2023-05-20T01:04:42.520 INFO:root:teuthology version: 0.0.1.dev98+g69caab8 2023-05-20T01:04:42.522 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/yuriw-2023-05-19_16:21:22-fs-wip-yuri7-testing-2023-05-18-0655-distro-default-smithi/7278970 --verbose --owner scheduled_yuriw@teuthology --name yuriw-2023-05-19_16:21:22-fs-wip-yuri7-testing-2023-05-18-0655-distro-default-smithi --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_20.04} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/data-scan} -- /home/teuthworker/archive/yuriw-2023-05-19_16:21:22-fs-wip-yuri7-testing-2023-05-18-0655-distro-default-smithi/7278970/orig.config.yaml 2023-05-20T01:04:42.578 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-05-20T01:04:42.692 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2023-05-19_16:21:22-fs-wip-yuri7-testing-2023-05-18-0655-distro-default-smithi/7278970 branch: wip-yuri7-testing-2023-05-18-0655 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_20.04} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/data-scan} email: yweinste@redhat.com first_in_suite: false job_id: '7278970' kernel: client: sha1: distro kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: yuriw-2023-05-19_16:21:22-fs-wip-yuri7-testing-2023-05-18-0655-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: wip-yuri7-testing-2023-05-18-0655 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - bad backtrace - object missing on disk - error reading table object - error reading sessionmap - unmatched fragstat - unmatched rstat - was unreadable, recreating it now - Scrub error on inode - Metadata damage detected - inconsistent rstat on inode - Error recovering journal sha1: 5ca1fdcfd7e6cec340f637cba9f0087e6db9c2c1 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 5ca1fdcfd7e6cec340f637cba9f0087e6db9c2c1 kclient: syntax: v2 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-yuri7-testing-2023-05-18-0655 sha1: 5ca1fdcfd7e6cec340f637cba9f0087e6db9c2c1 owner: scheduled_yuriw@teuthology priority: 75 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 6431 sha1: 5ca1fdcfd7e6cec340f637cba9f0087e6db9c2c1 sleep_before_teardown: 0 subset: 111/120000 suite: fs suite_branch: wip-yuri7-testing-2023-05-18-0655 suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_5ca1fdcfd7e6cec340f637cba9f0087e6db9c2c1/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 5ca1fdcfd7e6cec340f637cba9f0087e6db9c2c1 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi016.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac= smithi132.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: null - kclient: null - cephfs_test_runner: modules: - tasks.cephfs.test_data_scan teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 69caab8ec6f44c61c991ae693e8e7b8cd859a896 timestamp: 2023-05-19_16:21:22 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.23459 2023-05-20T01:04:42.693 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_5ca1fdcfd7e6cec340f637cba9f0087e6db9c2c1/qa; will attempt to use it 2023-05-20T01:04:42.693 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_5ca1fdcfd7e6cec340f637cba9f0087e6db9c2c1/qa/tasks 2023-05-20T01:04:42.694 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-05-20T01:04:42.695 INFO:teuthology.task.internal:Checking packages... 2023-05-20T01:04:42.724 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash '5ca1fdcfd7e6cec340f637cba9f0087e6db9c2c1' 2023-05-20T01:04:42.724 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-05-20T01:04:42.725 INFO:teuthology.packaging:ref: None 2023-05-20T01:04:42.725 INFO:teuthology.packaging:tag: None 2023-05-20T01:04:42.725 INFO:teuthology.packaging:branch: wip-yuri7-testing-2023-05-18-0655 2023-05-20T01:04:42.726 INFO:teuthology.packaging:sha1: 5ca1fdcfd7e6cec340f637cba9f0087e6db9c2c1 2023-05-20T01:04:42.726 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-yuri7-testing-2023-05-18-0655 2023-05-20T01:04:42.865 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-3996.g5ca1fdcf 2023-05-20T01:04:42.865 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-05-20T01:04:42.872 INFO:teuthology.task.internal:no buildpackages task found 2023-05-20T01:04:42.872 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-05-20T01:04:42.894 INFO:teuthology.task.internal:Saving configuration 2023-05-20T01:04:42.921 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-05-20T01:04:42.928 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-05-20T01:04:42.959 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi016.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-05-19_16:21:22-fs-wip-yuri7-testing-2023-05-18-0655-distro-default-smithi/7278970', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-05-20 00:59:34.950964', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-05-20T01:04:42.987 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi132.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-05-19_16:21:22-fs-wip-yuri7-testing-2023-05-18-0655-distro-default-smithi/7278970', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-05-20 00:59:34.952579', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-05-20T01:04:42.987 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-05-20T01:04:42.995 INFO:teuthology.task.internal:roles: ubuntu@smithi016.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2023-05-20T01:04:42.995 INFO:teuthology.task.internal:roles: ubuntu@smithi132.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2023-05-20T01:04:42.995 INFO:teuthology.run_tasks:Running task console_log... 2023-05-20T01:04:43.084 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f49a48e1ee0>, signals=[15]) 2023-05-20T01:04:43.085 INFO:teuthology.run_tasks:Running task internal.connect... 2023-05-20T01:04:43.091 INFO:teuthology.task.internal:Opening connections... 2023-05-20T01:04:43.092 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi016.front.sepia.ceph.com 2023-05-20T01:04:43.093 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-05-20T01:04:43.167 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi132.front.sepia.ceph.com 2023-05-20T01:04:43.168 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi132.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-05-20T01:04:43.238 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-05-20T01:04:43.250 DEBUG:teuthology.orchestra.run.smithi016:> uname -m 2023-05-20T01:04:43.271 INFO:teuthology.orchestra.run.smithi016.stdout:x86_64 2023-05-20T01:04:43.272 DEBUG:teuthology.orchestra.run.smithi016:> cat /etc/os-release 2023-05-20T01:04:43.332 INFO:teuthology.orchestra.run.smithi016.stdout:NAME="Red Hat Enterprise Linux" 2023-05-20T01:04:43.332 INFO:teuthology.orchestra.run.smithi016.stdout:VERSION="8.6 (Ootpa)" 2023-05-20T01:04:43.333 INFO:teuthology.orchestra.run.smithi016.stdout:ID="rhel" 2023-05-20T01:04:43.333 INFO:teuthology.orchestra.run.smithi016.stdout:ID_LIKE="fedora" 2023-05-20T01:04:43.333 INFO:teuthology.orchestra.run.smithi016.stdout:VERSION_ID="8.6" 2023-05-20T01:04:43.333 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM_ID="platform:el8" 2023-05-20T01:04:43.333 INFO:teuthology.orchestra.run.smithi016.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-05-20T01:04:43.333 INFO:teuthology.orchestra.run.smithi016.stdout:ANSI_COLOR="0;31" 2023-05-20T01:04:43.333 INFO:teuthology.orchestra.run.smithi016.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-05-20T01:04:43.333 INFO:teuthology.orchestra.run.smithi016.stdout:HOME_URL="https://www.redhat.com/" 2023-05-20T01:04:43.334 INFO:teuthology.orchestra.run.smithi016.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-05-20T01:04:43.334 INFO:teuthology.orchestra.run.smithi016.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-05-20T01:04:43.334 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:04:43.334 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-05-20T01:04:43.334 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-05-20T01:04:43.334 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-05-20T01:04:43.334 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-05-20T01:04:43.335 INFO:teuthology.lock.ops:Updating smithi016.front.sepia.ceph.com on lock server 2023-05-20T01:04:43.365 DEBUG:teuthology.orchestra.run.smithi132:> uname -m 2023-05-20T01:04:43.386 INFO:teuthology.orchestra.run.smithi132.stdout:x86_64 2023-05-20T01:04:43.386 DEBUG:teuthology.orchestra.run.smithi132:> cat /etc/os-release 2023-05-20T01:04:43.450 INFO:teuthology.orchestra.run.smithi132.stdout:NAME="Red Hat Enterprise Linux" 2023-05-20T01:04:43.450 INFO:teuthology.orchestra.run.smithi132.stdout:VERSION="8.6 (Ootpa)" 2023-05-20T01:04:43.451 INFO:teuthology.orchestra.run.smithi132.stdout:ID="rhel" 2023-05-20T01:04:43.451 INFO:teuthology.orchestra.run.smithi132.stdout:ID_LIKE="fedora" 2023-05-20T01:04:43.451 INFO:teuthology.orchestra.run.smithi132.stdout:VERSION_ID="8.6" 2023-05-20T01:04:43.452 INFO:teuthology.orchestra.run.smithi132.stdout:PLATFORM_ID="platform:el8" 2023-05-20T01:04:43.452 INFO:teuthology.orchestra.run.smithi132.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-05-20T01:04:43.452 INFO:teuthology.orchestra.run.smithi132.stdout:ANSI_COLOR="0;31" 2023-05-20T01:04:43.452 INFO:teuthology.orchestra.run.smithi132.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-05-20T01:04:43.453 INFO:teuthology.orchestra.run.smithi132.stdout:HOME_URL="https://www.redhat.com/" 2023-05-20T01:04:43.453 INFO:teuthology.orchestra.run.smithi132.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-05-20T01:04:43.453 INFO:teuthology.orchestra.run.smithi132.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-05-20T01:04:43.453 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:04:43.454 INFO:teuthology.orchestra.run.smithi132.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-05-20T01:04:43.454 INFO:teuthology.orchestra.run.smithi132.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-05-20T01:04:43.454 INFO:teuthology.orchestra.run.smithi132.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-05-20T01:04:43.454 INFO:teuthology.orchestra.run.smithi132.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-05-20T01:04:43.455 INFO:teuthology.lock.ops:Updating smithi132.front.sepia.ceph.com on lock server 2023-05-20T01:04:43.489 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-05-20T01:04:43.504 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-05-20T01:04:43.510 INFO:teuthology.task.internal:Checking for old test directory... 2023-05-20T01:04:43.511 DEBUG:teuthology.orchestra.run.smithi016:> test '!' -e /home/ubuntu/cephtest 2023-05-20T01:04:43.513 DEBUG:teuthology.orchestra.run.smithi132:> test '!' -e /home/ubuntu/cephtest 2023-05-20T01:04:43.531 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-05-20T01:04:43.537 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-05-20T01:04:43.537 DEBUG:teuthology.orchestra.run.smithi016:> test -z $(ls -A /var/lib/ceph) 2023-05-20T01:04:43.573 DEBUG:teuthology.orchestra.run.smithi132:> test -z $(ls -A /var/lib/ceph) 2023-05-20T01:04:43.606 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-05-20T01:04:43.675 INFO:teuthology.run_tasks:Running task kernel... 2023-05-20T01:04:43.695 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2023-05-20T01:04:43.696 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}, 'client.2': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}, 'client.3': {'sha1': 'distro'}} 2023-05-20T01:04:43.696 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}}, timeout 300 2023-05-20T01:04:43.697 DEBUG:teuthology.orchestra.run.smithi016:> test -f /run/.containerenv -o -f /.dockerenv 2023-05-20T01:04:43.697 DEBUG:teuthology.orchestra.run.smithi132:> test -f /run/.containerenv -o -f /.dockerenv 2023-05-20T01:04:43.716 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-05-20T01:04:43.717 DEBUG:teuthology.orchestra.run.smithi132:> uname -r 2023-05-20T01:04:43.718 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-05-20T01:04:43.720 DEBUG:teuthology.orchestra.run.smithi016:> uname -r 2023-05-20T01:04:43.776 INFO:teuthology.orchestra.run.smithi132.stdout:4.18.0-372.9.1.el8.x86_64 2023-05-20T01:04:43.776 INFO:teuthology.task.kernel:Running kernel on smithi132: 4.18.0-372.9.1.el8.x86_64 2023-05-20T01:04:43.777 DEBUG:teuthology.orchestra.run.smithi132:> sudo yum install -y kernel 2023-05-20T01:04:43.780 INFO:teuthology.orchestra.run.smithi016.stdout:4.18.0-372.9.1.el8.x86_64 2023-05-20T01:04:43.781 INFO:teuthology.task.kernel:Running kernel on smithi016: 4.18.0-372.9.1.el8.x86_64 2023-05-20T01:04:43.781 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum install -y kernel 2023-05-20T01:04:44.979 INFO:teuthology.orchestra.run.smithi132.stdout:Updating Subscription Management repositories. 2023-05-20T01:04:44.979 INFO:teuthology.orchestra.run.smithi132.stdout:Unable to read consumer identity 2023-05-20T01:04:45.140 INFO:teuthology.orchestra.run.smithi016.stdout:Updating Subscription Management repositories. 2023-05-20T01:04:45.141 INFO:teuthology.orchestra.run.smithi016.stdout:Unable to read consumer identity 2023-05-20T01:04:46.558 INFO:teuthology.orchestra.run.smithi132.stdout:Copr repo for python3-asyncssh owned by ceph 12 kB/s | 3.5 kB 00:00 2023-05-20T01:04:46.701 INFO:teuthology.orchestra.run.smithi016.stdout:Copr repo for python3-asyncssh owned by ceph 12 kB/s | 3.5 kB 00:00 2023-05-20T01:04:48.242 INFO:teuthology.orchestra.run.smithi132.stdout:Extra Packages for Enterprise Linux 8.3 MB/s | 14 MB 00:01 2023-05-20T01:04:49.809 INFO:teuthology.orchestra.run.smithi016.stdout:Extra Packages for Enterprise Linux 4.5 MB/s | 14 MB 00:03 2023-05-20T01:04:51.651 INFO:teuthology.orchestra.run.smithi132.stdout:lab-extras 74 kB/s | 24 kB 00:00 2023-05-20T01:04:52.294 INFO:teuthology.orchestra.run.smithi132.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2023-05-20T01:04:52.349 INFO:teuthology.orchestra.run.smithi132.stdout:Dependencies resolved. 2023-05-20T01:04:52.350 INFO:teuthology.orchestra.run.smithi132.stdout:Nothing to do. 2023-05-20T01:04:52.350 INFO:teuthology.orchestra.run.smithi132.stdout:Complete! 2023-05-20T01:04:52.467 DEBUG:teuthology.orchestra.run.smithi132:> echo no | sudo yum reinstall kernel || true 2023-05-20T01:04:52.797 INFO:teuthology.orchestra.run.smithi132.stdout:Updating Subscription Management repositories. 2023-05-20T01:04:52.797 INFO:teuthology.orchestra.run.smithi132.stdout:Unable to read consumer identity 2023-05-20T01:04:53.049 INFO:teuthology.orchestra.run.smithi132.stdout:Last metadata expiration check: 0:00:02 ago on Sat 20 May 2023 01:04:51 AM UTC. 2023-05-20T01:04:53.089 INFO:teuthology.orchestra.run.smithi132.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-05-20T01:04:53.131 INFO:teuthology.orchestra.run.smithi132.stderr:Error: No packages marked for reinstall. 2023-05-20T01:04:53.177 DEBUG:teuthology.orchestra.run.smithi132:> sudo yum reinstall -y kernel || true 2023-05-20T01:04:53.234 INFO:teuthology.orchestra.run.smithi016.stdout:lab-extras 443 kB/s | 24 kB 00:00 2023-05-20T01:04:53.507 INFO:teuthology.orchestra.run.smithi132.stdout:Updating Subscription Management repositories. 2023-05-20T01:04:53.508 INFO:teuthology.orchestra.run.smithi132.stdout:Unable to read consumer identity 2023-05-20T01:04:53.763 INFO:teuthology.orchestra.run.smithi132.stdout:Last metadata expiration check: 0:00:02 ago on Sat 20 May 2023 01:04:51 AM UTC. 2023-05-20T01:04:53.803 INFO:teuthology.orchestra.run.smithi132.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-05-20T01:04:53.831 INFO:teuthology.orchestra.run.smithi132.stderr:Error: No packages marked for reinstall. 2023-05-20T01:04:53.877 DEBUG:teuthology.orchestra.run.smithi132:> rpm -q kernel --last | head -n 1 2023-05-20T01:04:53.920 INFO:teuthology.orchestra.run.smithi016.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2023-05-20T01:04:53.957 INFO:teuthology.orchestra.run.smithi132.stdout:kernel-4.18.0-372.9.1.el8.x86_64 Thu 14 Jul 2022 06:23:21 PM UTC 2023-05-20T01:04:53.958 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-372.9.1.el8.x86_64 2023-05-20T01:04:53.958 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2023-05-20T01:04:53.959 DEBUG:teuthology.parallel:result is None 2023-05-20T01:04:53.976 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2023-05-20T01:04:53.977 INFO:teuthology.orchestra.run.smithi016.stdout:Nothing to do. 2023-05-20T01:04:53.978 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2023-05-20T01:04:54.100 DEBUG:teuthology.orchestra.run.smithi016:> echo no | sudo yum reinstall kernel || true 2023-05-20T01:04:54.491 INFO:teuthology.orchestra.run.smithi016.stdout:Updating Subscription Management repositories. 2023-05-20T01:04:54.491 INFO:teuthology.orchestra.run.smithi016.stdout:Unable to read consumer identity 2023-05-20T01:04:54.757 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:00:01 ago on Sat 20 May 2023 01:04:53 AM UTC. 2023-05-20T01:04:54.803 INFO:teuthology.orchestra.run.smithi016.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-05-20T01:04:54.840 INFO:teuthology.orchestra.run.smithi016.stderr:Error: No packages marked for reinstall. 2023-05-20T01:04:54.888 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum reinstall -y kernel || true 2023-05-20T01:04:55.236 INFO:teuthology.orchestra.run.smithi016.stdout:Updating Subscription Management repositories. 2023-05-20T01:04:55.237 INFO:teuthology.orchestra.run.smithi016.stdout:Unable to read consumer identity 2023-05-20T01:04:55.501 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:00:02 ago on Sat 20 May 2023 01:04:53 AM UTC. 2023-05-20T01:04:55.543 INFO:teuthology.orchestra.run.smithi016.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-05-20T01:04:55.575 INFO:teuthology.orchestra.run.smithi016.stderr:Error: No packages marked for reinstall. 2023-05-20T01:04:55.628 DEBUG:teuthology.orchestra.run.smithi016:> rpm -q kernel --last | head -n 1 2023-05-20T01:04:55.710 INFO:teuthology.orchestra.run.smithi016.stdout:kernel-4.18.0-372.9.1.el8.x86_64 Thu 14 Jul 2022 06:23:21 PM UTC 2023-05-20T01:04:55.711 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-372.9.1.el8.x86_64 2023-05-20T01:04:55.712 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2023-05-20T01:04:55.712 DEBUG:teuthology.parallel:result is None 2023-05-20T01:04:55.712 INFO:teuthology.run_tasks:Running task internal.base... 2023-05-20T01:04:55.725 INFO:teuthology.task.internal:Creating test directory... 2023-05-20T01:04:55.725 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-05-20T01:04:55.728 DEBUG:teuthology.orchestra.run.smithi132:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-05-20T01:04:55.750 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-05-20T01:04:55.757 INFO:teuthology.run_tasks:Running task internal.archive... 2023-05-20T01:04:55.767 INFO:teuthology.task.internal:Creating archive directory... 2023-05-20T01:04:55.767 DEBUG:teuthology.orchestra.run.smithi016:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-05-20T01:04:55.785 DEBUG:teuthology.orchestra.run.smithi132:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-05-20T01:04:55.818 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-05-20T01:04:55.825 INFO:teuthology.task.internal:Enabling coredump saving... 2023-05-20T01:04:55.826 DEBUG:teuthology.orchestra.run.smithi016:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-05-20T01:04:55.842 DEBUG:teuthology.orchestra.run.smithi132:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-05-20T01:04:55.868 INFO:teuthology.orchestra.run.smithi016.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-05-20T01:04:55.880 INFO:teuthology.orchestra.run.smithi016.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-05-20T01:04:55.885 INFO:teuthology.orchestra.run.smithi132.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-05-20T01:04:55.898 INFO:teuthology.orchestra.run.smithi132.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-05-20T01:04:55.900 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-05-20T01:04:55.907 INFO:teuthology.task.internal:Configuring sudo... 2023-05-20T01:04:55.908 DEBUG:teuthology.orchestra.run.smithi016:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-05-20T01:04:55.924 DEBUG:teuthology.orchestra.run.smithi132:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-05-20T01:04:55.969 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-05-20T01:04:55.978 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-05-20T01:04:55.979 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-05-20T01:04:55.992 DEBUG:teuthology.orchestra.run.smithi132:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-05-20T01:04:56.028 DEBUG:teuthology.orchestra.run.smithi016:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-05-20T01:04:56.095 DEBUG:teuthology.orchestra.run.smithi016:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-05-20T01:04:56.148 DEBUG:teuthology.orchestra.run.smithi016:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-05-20T01:04:56.233 DEBUG:teuthology.orchestra.run.smithi016:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-05-20T01:04:56.275 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-05-20T01:04:56.276 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-05-20T01:04:56.348 DEBUG:teuthology.orchestra.run.smithi132:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-05-20T01:04:56.395 DEBUG:teuthology.orchestra.run.smithi132:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-05-20T01:04:56.445 DEBUG:teuthology.orchestra.run.smithi132:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-05-20T01:04:56.535 DEBUG:teuthology.orchestra.run.smithi132:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-05-20T01:04:56.573 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2023-05-20T01:04:56.574 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-05-20T01:04:56.642 DEBUG:teuthology.orchestra.run.smithi016:> sudo service rsyslog restart 2023-05-20T01:04:56.644 DEBUG:teuthology.orchestra.run.smithi132:> sudo service rsyslog restart 2023-05-20T01:04:56.700 INFO:teuthology.orchestra.run.smithi016.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-05-20T01:04:56.745 INFO:teuthology.orchestra.run.smithi132.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-05-20T01:04:57.159 INFO:teuthology.run_tasks:Running task internal.timer... 2023-05-20T01:04:57.172 INFO:teuthology.task.internal:Starting timer... 2023-05-20T01:04:57.173 INFO:teuthology.run_tasks:Running task pcp... 2023-05-20T01:04:57.212 INFO:teuthology.run_tasks:Running task selinux... 2023-05-20T01:04:57.225 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-05-20T01:04:57.226 DEBUG:teuthology.orchestra.run.smithi016:> sudo service auditd rotate 2023-05-20T01:04:57.275 INFO:teuthology.orchestra.run.smithi016.stdout:Rotating logs: 2023-05-20T01:04:57.277 DEBUG:teuthology.orchestra.run.smithi132:> sudo service auditd rotate 2023-05-20T01:04:57.333 INFO:teuthology.orchestra.run.smithi132.stdout:Rotating logs: 2023-05-20T01:04:57.334 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-05-20T01:04:57.335 DEBUG:teuthology.orchestra.run.smithi016:> /usr/sbin/getenforce 2023-05-20T01:04:57.362 INFO:teuthology.orchestra.run.smithi016.stdout:Permissive 2023-05-20T01:04:57.362 DEBUG:teuthology.orchestra.run.smithi132:> /usr/sbin/getenforce 2023-05-20T01:04:57.386 INFO:teuthology.orchestra.run.smithi132.stdout:Permissive 2023-05-20T01:04:57.386 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi016.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi132.front.sepia.ceph.com': 'permissive'} 2023-05-20T01:04:57.387 DEBUG:teuthology.orchestra.run.smithi016:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-05-20T01:04:57.429 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-05-20T01:04:57.430 DEBUG:teuthology.orchestra.run.smithi132:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-05-20T01:04:57.459 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-05-20T01:04:57.460 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-05-20T01:04:57.461 DEBUG:teuthology.orchestra.run.smithi016:> sudo /usr/sbin/setenforce permissive 2023-05-20T01:04:57.498 DEBUG:teuthology.orchestra.run.smithi132:> sudo /usr/sbin/setenforce permissive 2023-05-20T01:04:57.528 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-05-20T01:04:57.539 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-05-20T01:04:57.540 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-05-20T01:04:57.567 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-05-20T01:04:57.569 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi016.front.sepia.ceph.com,smithi132.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-05-20T01:13:07.650 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi016.front.sepia.ceph.com'), Remote(name='ubuntu@smithi132.front.sepia.ceph.com')] 2023-05-20T01:13:07.651 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi016.front.sepia.ceph.com' 2023-05-20T01:13:07.652 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-05-20T01:13:07.721 DEBUG:teuthology.orchestra.run.smithi016:> true 2023-05-20T01:13:07.790 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi132.front.sepia.ceph.com' 2023-05-20T01:13:07.791 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi132.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-05-20T01:13:07.859 DEBUG:teuthology.orchestra.run.smithi132:> true 2023-05-20T01:13:07.926 INFO:teuthology.run_tasks:Running task clock... 2023-05-20T01:13:07.937 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-05-20T01:13:07.938 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-05-20T01:13:07.938 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-05-20T01:13:07.944 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-05-20T01:13:07.945 DEBUG:teuthology.orchestra.run.smithi132:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-05-20T01:13:07.973 INFO:teuthology.orchestra.run.smithi016.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-05-20T01:13:07.988 INFO:teuthology.orchestra.run.smithi016.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-05-20T01:13:07.997 INFO:teuthology.orchestra.run.smithi132.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-05-20T01:13:08.012 INFO:teuthology.orchestra.run.smithi132.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-05-20T01:13:08.059 INFO:teuthology.orchestra.run.smithi016.stderr:sudo: ntpd: command not found 2023-05-20T01:13:08.071 INFO:teuthology.orchestra.run.smithi016.stdout:506 Cannot talk to daemon 2023-05-20T01:13:08.087 INFO:teuthology.orchestra.run.smithi016.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-05-20T01:13:08.087 INFO:teuthology.orchestra.run.smithi132.stderr:sudo: ntpd: command not found 2023-05-20T01:13:08.096 INFO:teuthology.orchestra.run.smithi132.stdout:506 Cannot talk to daemon 2023-05-20T01:13:08.100 INFO:teuthology.orchestra.run.smithi016.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-05-20T01:13:08.111 INFO:teuthology.orchestra.run.smithi132.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-05-20T01:13:08.124 INFO:teuthology.orchestra.run.smithi132.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-05-20T01:13:08.176 INFO:teuthology.orchestra.run.smithi016.stderr:bash: ntpq: command not found 2023-05-20T01:13:08.180 INFO:teuthology.orchestra.run.smithi016.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-05-20T01:13:08.180 INFO:teuthology.orchestra.run.smithi016.stdout:=============================================================================== 2023-05-20T01:13:08.180 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-05-20T01:13:08.180 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-05-20T01:13:08.181 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-05-20T01:13:08.181 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-05-20T01:13:08.192 INFO:teuthology.orchestra.run.smithi132.stderr:bash: ntpq: command not found 2023-05-20T01:13:08.196 INFO:teuthology.orchestra.run.smithi132.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-05-20T01:13:08.196 INFO:teuthology.orchestra.run.smithi132.stdout:=============================================================================== 2023-05-20T01:13:08.197 INFO:teuthology.orchestra.run.smithi132.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-05-20T01:13:08.197 INFO:teuthology.orchestra.run.smithi132.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-05-20T01:13:08.197 INFO:teuthology.orchestra.run.smithi132.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-05-20T01:13:08.197 INFO:teuthology.orchestra.run.smithi132.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-05-20T01:13:08.197 INFO:teuthology.run_tasks:Running task install... 2023-05-20T01:13:08.207 DEBUG:teuthology.task.install:project ceph 2023-05-20T01:13:08.207 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '5ca1fdcfd7e6cec340f637cba9f0087e6db9c2c1'}} 2023-05-20T01:13:08.208 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '5ca1fdcfd7e6cec340f637cba9f0087e6db9c2c1'} 2023-05-20T01:13:08.208 INFO:teuthology.task.install:Using flavor: default 2023-05-20T01:13:08.215 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-05-20T01:13:08.215 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-05-20T01:13:08.216 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '5ca1fdcfd7e6cec340f637cba9f0087e6db9c2c1', 'tag': None, 'wait_for_package': False} 2023-05-20T01:13:08.217 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=5ca1fdcfd7e6cec340f637cba9f0087e6db9c2c1 2023-05-20T01:13:08.219 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '5ca1fdcfd7e6cec340f637cba9f0087e6db9c2c1', 'tag': None, 'wait_for_package': False} 2023-05-20T01:13:08.220 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=5ca1fdcfd7e6cec340f637cba9f0087e6db9c2c1 2023-05-20T01:13:08.358 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-yuri7-testing-2023-05-18-0655/5ca1fdcfd7e6cec340f637cba9f0087e6db9c2c1/centos/8/flavors/default/ 2023-05-20T01:13:08.359 INFO:teuthology.task.install.rpm:Package version is 18.0.0-3996.g5ca1fdcf 2023-05-20T01:13:08.365 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-yuri7-testing-2023-05-18-0655/5ca1fdcfd7e6cec340f637cba9f0087e6db9c2c1/centos/8/flavors/default/ 2023-05-20T01:13:08.365 INFO:teuthology.task.install.rpm:Package version is 18.0.0-3996.g5ca1fdcf 2023-05-20T01:13:08.469 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-yuri7-testing-2023-05-18-0655/5ca1fdcfd7e6cec340f637cba9f0087e6db9c2c1/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-yuri7-testing-2023-05-18-0655/5ca1fdcfd7e6cec340f637cba9f0087e6db9c2c1/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-yuri7-testing-2023-05-18-0655/5ca1fdcfd7e6cec340f637cba9f0087e6db9c2c1/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-05-20T01:13:08.470 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2023-05-20T01:13:08.471 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-05-20T01:13:08.476 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-yuri7-testing-2023-05-18-0655/5ca1fdcfd7e6cec340f637cba9f0087e6db9c2c1/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-yuri7-testing-2023-05-18-0655/5ca1fdcfd7e6cec340f637cba9f0087e6db9c2c1/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-yuri7-testing-2023-05-18-0655/5ca1fdcfd7e6cec340f637cba9f0087e6db9c2c1/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-05-20T01:13:08.477 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-05-20T01:13:08.477 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-05-20T01:13:08.501 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-05-20T01:13:08.501 DEBUG:teuthology.orchestra.run.smithi132:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/5ca1fdcfd7e6cec340f637cba9f0087e6db9c2c1/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-05-20T01:13:08.506 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-05-20T01:13:08.506 DEBUG:teuthology.orchestra.run.smithi016:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/5ca1fdcfd7e6cec340f637cba9f0087e6db9c2c1/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-05-20T01:13:08.568 DEBUG:teuthology.orchestra.run.smithi132:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-05-20T01:13:08.574 DEBUG:teuthology.orchestra.run.smithi016:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-05-20T01:13:08.645 DEBUG:teuthology.orchestra.run.smithi132:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-05-20T01:13:08.654 DEBUG:teuthology.orchestra.run.smithi016:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-05-20T01:13:08.711 INFO:teuthology.orchestra.run.smithi132.stdout:check_obsoletes = 1 2023-05-20T01:13:08.712 DEBUG:teuthology.orchestra.run.smithi132:> sudo yum clean all 2023-05-20T01:13:08.720 INFO:teuthology.orchestra.run.smithi016.stdout:check_obsoletes = 1 2023-05-20T01:13:08.722 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum clean all 2023-05-20T01:13:09.080 INFO:teuthology.orchestra.run.smithi132.stdout:Updating Subscription Management repositories. 2023-05-20T01:13:09.113 INFO:teuthology.orchestra.run.smithi016.stdout:Updating Subscription Management repositories. 2023-05-20T01:13:09.318 INFO:teuthology.orchestra.run.smithi132.stdout:44 files removed 2023-05-20T01:13:09.362 INFO:teuthology.orchestra.run.smithi016.stdout:44 files removed 2023-05-20T01:13:09.366 DEBUG:teuthology.orchestra.run.smithi132:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-05-20T01:13:09.410 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-05-20T01:13:09.696 INFO:teuthology.orchestra.run.smithi132.stdout:Updating Subscription Management repositories. 2023-05-20T01:13:09.760 INFO:teuthology.orchestra.run.smithi016.stdout:Updating Subscription Management repositories. 2023-05-20T01:13:11.603 INFO:teuthology.orchestra.run.smithi132.stdout:ceph packages for x86_64 52 kB/s | 80 kB 00:01 2023-05-20T01:13:11.883 INFO:teuthology.orchestra.run.smithi016.stdout:ceph packages for x86_64 46 kB/s | 80 kB 00:01 2023-05-20T01:13:12.030 INFO:teuthology.orchestra.run.smithi132.stdout:ceph noarch packages 24 kB/s | 9.4 kB 00:00 2023-05-20T01:13:12.241 INFO:teuthology.orchestra.run.smithi016.stdout:ceph noarch packages 29 kB/s | 9.4 kB 00:00 2023-05-20T01:13:12.262 INFO:teuthology.orchestra.run.smithi132.stdout:ceph source packages 7.0 kB/s | 1.5 kB 00:00 2023-05-20T01:13:12.413 INFO:teuthology.orchestra.run.smithi132.stdout:Copr repo for python3-asyncssh owned by ceph 27 kB/s | 3.5 kB 00:00 2023-05-20T01:13:12.492 INFO:teuthology.orchestra.run.smithi016.stdout:ceph source packages 6.5 kB/s | 1.5 kB 00:00 2023-05-20T01:13:12.663 INFO:teuthology.orchestra.run.smithi016.stdout:Copr repo for python3-asyncssh owned by ceph 23 kB/s | 3.5 kB 00:00 2023-05-20T01:13:14.103 INFO:teuthology.orchestra.run.smithi016.stdout:Extra Packages for Enterprise Linux 9.8 MB/s | 14 MB 00:01 2023-05-20T01:13:14.566 INFO:teuthology.orchestra.run.smithi132.stdout:Extra Packages for Enterprise Linux 6.5 MB/s | 14 MB 00:02 2023-05-20T01:13:17.386 INFO:teuthology.orchestra.run.smithi016.stdout:lab-extras 497 kB/s | 24 kB 00:00 2023-05-20T01:13:17.640 INFO:teuthology.orchestra.run.smithi132.stdout:lab-extras 402 kB/s | 24 kB 00:00 2023-05-20T01:13:18.296 INFO:teuthology.orchestra.run.smithi016.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 57 MB/s | 47 MB 00:00 2023-05-20T01:13:18.567 INFO:teuthology.orchestra.run.smithi132.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 62 MB/s | 53 MB 00:00 2023-05-20T01:13:26.634 INFO:teuthology.orchestra.run.smithi132.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 60 MB/s | 47 MB 00:00 2023-05-20T01:13:26.745 INFO:teuthology.orchestra.run.smithi016.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 58 MB/s | 53 MB 00:00 2023-05-20T01:13:37.467 INFO:teuthology.orchestra.run.smithi132.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-05-20T01:13:37.467 INFO:teuthology.orchestra.run.smithi132.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-05-20T01:13:37.482 INFO:teuthology.orchestra.run.smithi132.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-05-20T01:13:37.483 INFO:teuthology.orchestra.run.smithi132.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-05-20T01:13:37.484 INFO:teuthology.orchestra.run.smithi132.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-05-20T01:13:37.485 INFO:teuthology.orchestra.run.smithi132.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-05-20T01:13:37.487 INFO:teuthology.orchestra.run.smithi132.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-05-20T01:13:37.490 INFO:teuthology.orchestra.run.smithi132.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-05-20T01:13:37.492 INFO:teuthology.orchestra.run.smithi132.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-05-20T01:13:37.623 INFO:teuthology.orchestra.run.smithi132.stdout:Dependencies resolved. 2023-05-20T01:13:37.639 INFO:teuthology.orchestra.run.smithi132.stdout:=========================================================================================================================== 2023-05-20T01:13:37.639 INFO:teuthology.orchestra.run.smithi132.stdout: Package Arch Version Repository Size 2023-05-20T01:13:37.639 INFO:teuthology.orchestra.run.smithi132.stdout:=========================================================================================================================== 2023-05-20T01:13:37.640 INFO:teuthology.orchestra.run.smithi132.stdout:Installing: 2023-05-20T01:13:37.640 INFO:teuthology.orchestra.run.smithi132.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-05-20T01:13:37.640 INFO:teuthology.orchestra.run.smithi132.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-05-20T01:13:37.640 INFO:teuthology.orchestra.run.smithi132.stdout: ceph x86_64 2:18.0.0-3996.g5ca1fdcf.el8 ceph 6.4 k 2023-05-20T01:13:37.640 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-base x86_64 2:18.0.0-3996.g5ca1fdcf.el8 ceph 5.3 M 2023-05-20T01:13:37.640 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-fuse x86_64 2:18.0.0-3996.g5ca1fdcf.el8 ceph 848 k 2023-05-20T01:13:37.640 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-3996.g5ca1fdcf.el8 ceph 165 k 2023-05-20T01:13:37.641 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr x86_64 2:18.0.0-3996.g5ca1fdcf.el8 ceph 1.5 M 2023-05-20T01:13:37.641 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-cephadm noarch 2:18.0.0-3996.g5ca1fdcf.el8 ceph-noarch 127 k 2023-05-20T01:13:37.641 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-dashboard noarch 2:18.0.0-3996.g5ca1fdcf.el8 ceph-noarch 1.7 M 2023-05-20T01:13:37.641 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-3996.g5ca1fdcf.el8 ceph-noarch 7.4 M 2023-05-20T01:13:37.641 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-rook noarch 2:18.0.0-3996.g5ca1fdcf.el8 ceph-noarch 47 k 2023-05-20T01:13:37.641 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-radosgw x86_64 2:18.0.0-3996.g5ca1fdcf.el8 ceph 13 M 2023-05-20T01:13:37.641 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-test x86_64 2:18.0.0-3996.g5ca1fdcf.el8 ceph 41 M 2023-05-20T01:13:37.642 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-volume noarch 2:18.0.0-3996.g5ca1fdcf.el8 ceph-noarch 264 k 2023-05-20T01:13:37.642 INFO:teuthology.orchestra.run.smithi132.stdout: cephadm noarch 2:18.0.0-3996.g5ca1fdcf.el8 ceph-noarch 83 k 2023-05-20T01:13:37.642 INFO:teuthology.orchestra.run.smithi132.stdout: cephfs-mirror x86_64 2:18.0.0-3996.g5ca1fdcf.el8 ceph 258 k 2023-05-20T01:13:37.642 INFO:teuthology.orchestra.run.smithi132.stdout: cephfs-top noarch 2:18.0.0-3996.g5ca1fdcf.el8 ceph-noarch 18 k 2023-05-20T01:13:37.642 INFO:teuthology.orchestra.run.smithi132.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-05-20T01:13:37.642 INFO:teuthology.orchestra.run.smithi132.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-05-20T01:13:37.642 INFO:teuthology.orchestra.run.smithi132.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-05-20T01:13:37.643 INFO:teuthology.orchestra.run.smithi132.stdout: libcephfs-devel x86_64 2:18.0.0-3996.g5ca1fdcf.el8 ceph 31 k 2023-05-20T01:13:37.643 INFO:teuthology.orchestra.run.smithi132.stdout: libcephfs2 x86_64 2:18.0.0-3996.g5ca1fdcf.el8 ceph 771 k 2023-05-20T01:13:37.643 INFO:teuthology.orchestra.run.smithi132.stdout: librados-devel x86_64 2:18.0.0-3996.g5ca1fdcf.el8 ceph 139 k 2023-05-20T01:13:37.643 INFO:teuthology.orchestra.run.smithi132.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-05-20T01:13:37.643 INFO:teuthology.orchestra.run.smithi132.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-05-20T01:13:37.643 INFO:teuthology.orchestra.run.smithi132.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-05-20T01:13:37.643 INFO:teuthology.orchestra.run.smithi132.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-05-20T01:13:37.644 INFO:teuthology.orchestra.run.smithi132.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-05-20T01:13:37.644 INFO:teuthology.orchestra.run.smithi132.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-05-20T01:13:37.644 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cephfs x86_64 2:18.0.0-3996.g5ca1fdcf.el8 ceph 197 k 2023-05-20T01:13:37.644 INFO:teuthology.orchestra.run.smithi132.stdout: python3-rados x86_64 2:18.0.0-3996.g5ca1fdcf.el8 ceph 370 k 2023-05-20T01:13:37.644 INFO:teuthology.orchestra.run.smithi132.stdout: python3-rbd x86_64 2:18.0.0-3996.g5ca1fdcf.el8 ceph 364 k 2023-05-20T01:13:37.644 INFO:teuthology.orchestra.run.smithi132.stdout: python3-rgw x86_64 2:18.0.0-3996.g5ca1fdcf.el8 ceph 107 k 2023-05-20T01:13:37.645 INFO:teuthology.orchestra.run.smithi132.stdout: rbd-fuse x86_64 2:18.0.0-3996.g5ca1fdcf.el8 ceph 94 k 2023-05-20T01:13:37.645 INFO:teuthology.orchestra.run.smithi132.stdout: rbd-mirror x86_64 2:18.0.0-3996.g5ca1fdcf.el8 ceph 3.4 M 2023-05-20T01:13:37.645 INFO:teuthology.orchestra.run.smithi132.stdout: rbd-nbd x86_64 2:18.0.0-3996.g5ca1fdcf.el8 ceph 180 k 2023-05-20T01:13:37.645 INFO:teuthology.orchestra.run.smithi132.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-05-20T01:13:37.645 INFO:teuthology.orchestra.run.smithi132.stdout:Upgrading: 2023-05-20T01:13:37.645 INFO:teuthology.orchestra.run.smithi132.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-05-20T01:13:37.645 INFO:teuthology.orchestra.run.smithi132.stdout: librados2 x86_64 2:18.0.0-3996.g5ca1fdcf.el8 ceph 3.7 M 2023-05-20T01:13:37.646 INFO:teuthology.orchestra.run.smithi132.stdout: librbd1 x86_64 2:18.0.0-3996.g5ca1fdcf.el8 ceph 3.6 M 2023-05-20T01:13:37.646 INFO:teuthology.orchestra.run.smithi132.stdout:Installing dependencies: 2023-05-20T01:13:37.646 INFO:teuthology.orchestra.run.smithi132.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-05-20T01:13:37.646 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-common x86_64 2:18.0.0-3996.g5ca1fdcf.el8 ceph 22 M 2023-05-20T01:13:37.646 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-grafana-dashboards noarch 2:18.0.0-3996.g5ca1fdcf.el8 ceph-noarch 23 k 2023-05-20T01:13:37.647 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mds x86_64 2:18.0.0-3996.g5ca1fdcf.el8 ceph 2.3 M 2023-05-20T01:13:37.647 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-modules-core noarch 2:18.0.0-3996.g5ca1fdcf.el8 ceph-noarch 240 k 2023-05-20T01:13:37.647 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mon x86_64 2:18.0.0-3996.g5ca1fdcf.el8 ceph 4.6 M 2023-05-20T01:13:37.647 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-osd x86_64 2:18.0.0-3996.g5ca1fdcf.el8 ceph 18 M 2023-05-20T01:13:37.647 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-prometheus-alerts noarch 2:18.0.0-3996.g5ca1fdcf.el8 ceph-noarch 15 k 2023-05-20T01:13:37.647 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-selinux x86_64 2:18.0.0-3996.g5ca1fdcf.el8 ceph 24 k 2023-05-20T01:13:37.648 INFO:teuthology.orchestra.run.smithi132.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-05-20T01:13:37.648 INFO:teuthology.orchestra.run.smithi132.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-05-20T01:13:37.648 INFO:teuthology.orchestra.run.smithi132.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-05-20T01:13:37.648 INFO:teuthology.orchestra.run.smithi132.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-05-20T01:13:37.648 INFO:teuthology.orchestra.run.smithi132.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-05-20T01:13:37.649 INFO:teuthology.orchestra.run.smithi132.stdout: libcephsqlite x86_64 2:18.0.0-3996.g5ca1fdcf.el8 ceph 137 k 2023-05-20T01:13:37.649 INFO:teuthology.orchestra.run.smithi132.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-05-20T01:13:37.649 INFO:teuthology.orchestra.run.smithi132.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-05-20T01:13:37.649 INFO:teuthology.orchestra.run.smithi132.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-05-20T01:13:37.649 INFO:teuthology.orchestra.run.smithi132.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-05-20T01:13:37.649 INFO:teuthology.orchestra.run.smithi132.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-05-20T01:13:37.650 INFO:teuthology.orchestra.run.smithi132.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-05-20T01:13:37.650 INFO:teuthology.orchestra.run.smithi132.stdout: libradosstriper1 x86_64 2:18.0.0-3996.g5ca1fdcf.el8 ceph 473 k 2023-05-20T01:13:37.650 INFO:teuthology.orchestra.run.smithi132.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-05-20T01:13:37.650 INFO:teuthology.orchestra.run.smithi132.stdout: librgw2 x86_64 2:18.0.0-3996.g5ca1fdcf.el8 ceph 6.5 M 2023-05-20T01:13:37.650 INFO:teuthology.orchestra.run.smithi132.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-05-20T01:13:37.650 INFO:teuthology.orchestra.run.smithi132.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-05-20T01:13:37.650 INFO:teuthology.orchestra.run.smithi132.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-05-20T01:13:37.651 INFO:teuthology.orchestra.run.smithi132.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-05-20T01:13:37.651 INFO:teuthology.orchestra.run.smithi132.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-05-20T01:13:37.651 INFO:teuthology.orchestra.run.smithi132.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-05-20T01:13:37.651 INFO:teuthology.orchestra.run.smithi132.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-05-20T01:13:37.651 INFO:teuthology.orchestra.run.smithi132.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-05-20T01:13:37.651 INFO:teuthology.orchestra.run.smithi132.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-05-20T01:13:37.651 INFO:teuthology.orchestra.run.smithi132.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-05-20T01:13:37.652 INFO:teuthology.orchestra.run.smithi132.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-05-20T01:13:37.652 INFO:teuthology.orchestra.run.smithi132.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-05-20T01:13:37.652 INFO:teuthology.orchestra.run.smithi132.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-05-20T01:13:37.652 INFO:teuthology.orchestra.run.smithi132.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-05-20T01:13:37.652 INFO:teuthology.orchestra.run.smithi132.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-05-20T01:13:37.652 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-05-20T01:13:37.652 INFO:teuthology.orchestra.run.smithi132.stdout: python3-ceph-argparse x86_64 2:18.0.0-3996.g5ca1fdcf.el8 ceph 45 k 2023-05-20T01:13:37.652 INFO:teuthology.orchestra.run.smithi132.stdout: python3-ceph-common x86_64 2:18.0.0-3996.g5ca1fdcf.el8 ceph 119 k 2023-05-20T01:13:37.653 INFO:teuthology.orchestra.run.smithi132.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-05-20T01:13:37.653 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-05-20T01:13:37.653 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-05-20T01:13:37.653 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-05-20T01:13:37.653 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-05-20T01:13:37.653 INFO:teuthology.orchestra.run.smithi132.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-05-20T01:13:37.653 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-05-20T01:13:37.654 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-05-20T01:13:37.654 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-05-20T01:13:37.654 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-05-20T01:13:37.654 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-05-20T01:13:37.654 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-05-20T01:13:37.654 INFO:teuthology.orchestra.run.smithi132.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-05-20T01:13:37.654 INFO:teuthology.orchestra.run.smithi132.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-05-20T01:13:37.654 INFO:teuthology.orchestra.run.smithi132.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-05-20T01:13:37.655 INFO:teuthology.orchestra.run.smithi132.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-05-20T01:13:37.655 INFO:teuthology.orchestra.run.smithi132.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-05-20T01:13:37.655 INFO:teuthology.orchestra.run.smithi132.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-05-20T01:13:37.655 INFO:teuthology.orchestra.run.smithi132.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-05-20T01:13:37.655 INFO:teuthology.orchestra.run.smithi132.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-05-20T01:13:37.655 INFO:teuthology.orchestra.run.smithi132.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-05-20T01:13:37.655 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-05-20T01:13:37.656 INFO:teuthology.orchestra.run.smithi132.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-05-20T01:13:37.656 INFO:teuthology.orchestra.run.smithi132.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-05-20T01:13:37.656 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-05-20T01:13:37.656 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-05-20T01:13:37.656 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-05-20T01:13:37.656 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-05-20T01:13:37.657 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-05-20T01:13:37.657 INFO:teuthology.orchestra.run.smithi132.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-05-20T01:13:37.657 INFO:teuthology.orchestra.run.smithi132.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-05-20T01:13:37.657 INFO:teuthology.orchestra.run.smithi132.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-05-20T01:13:37.657 INFO:teuthology.orchestra.run.smithi132.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-05-20T01:13:37.657 INFO:teuthology.orchestra.run.smithi132.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-05-20T01:13:37.658 INFO:teuthology.orchestra.run.smithi132.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-05-20T01:13:37.658 INFO:teuthology.orchestra.run.smithi132.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-05-20T01:13:37.658 INFO:teuthology.orchestra.run.smithi132.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-05-20T01:13:37.658 INFO:teuthology.orchestra.run.smithi132.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-05-20T01:13:37.658 INFO:teuthology.orchestra.run.smithi132.stdout: python3-waitress noarch 1.2.1-2.el8.1 epel 241 k 2023-05-20T01:13:37.658 INFO:teuthology.orchestra.run.smithi132.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-05-20T01:13:37.658 INFO:teuthology.orchestra.run.smithi132.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-05-20T01:13:37.659 INFO:teuthology.orchestra.run.smithi132.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-05-20T01:13:37.659 INFO:teuthology.orchestra.run.smithi132.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-05-20T01:13:37.659 INFO:teuthology.orchestra.run.smithi132.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-05-20T01:13:37.659 INFO:teuthology.orchestra.run.smithi132.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-05-20T01:13:37.659 INFO:teuthology.orchestra.run.smithi132.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-05-20T01:13:37.659 INFO:teuthology.orchestra.run.smithi132.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-05-20T01:13:37.660 INFO:teuthology.orchestra.run.smithi132.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-05-20T01:13:37.660 INFO:teuthology.orchestra.run.smithi132.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-05-20T01:13:37.660 INFO:teuthology.orchestra.run.smithi132.stdout:Enabling module streams: 2023-05-20T01:13:37.660 INFO:teuthology.orchestra.run.smithi132.stdout: postgresql 10 2023-05-20T01:13:37.660 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:13:37.660 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction Summary 2023-05-20T01:13:37.660 INFO:teuthology.orchestra.run.smithi132.stdout:=========================================================================================================================== 2023-05-20T01:13:37.661 INFO:teuthology.orchestra.run.smithi132.stdout:Install 128 Packages 2023-05-20T01:13:37.661 INFO:teuthology.orchestra.run.smithi132.stdout:Upgrade 3 Packages 2023-05-20T01:13:37.661 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:13:37.661 INFO:teuthology.orchestra.run.smithi132.stdout:Total download size: 197 M 2023-05-20T01:13:37.662 INFO:teuthology.orchestra.run.smithi132.stdout:Downloading Packages: 2023-05-20T01:13:38.085 INFO:teuthology.orchestra.run.smithi132.stdout:(1/131): ceph-18.0.0-3996.g5ca1fdcf.el8.x86_64. 51 kB/s | 6.4 kB 00:00 2023-05-20T01:13:38.310 INFO:teuthology.orchestra.run.smithi132.stdout:(2/131): ceph-fuse-18.0.0-3996.g5ca1fdcf.el8.x8 3.7 MB/s | 848 kB 00:00 2023-05-20T01:13:38.369 INFO:teuthology.orchestra.run.smithi132.stdout:(3/131): ceph-immutable-object-cache-18.0.0-399 2.8 MB/s | 165 kB 00:00 2023-05-20T01:13:38.619 INFO:teuthology.orchestra.run.smithi132.stdout:(4/131): ceph-base-18.0.0-3996.g5ca1fdcf.el8.x8 8.1 MB/s | 5.3 MB 00:00 2023-05-20T01:13:38.672 INFO:teuthology.orchestra.run.smithi016.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-05-20T01:13:38.673 INFO:teuthology.orchestra.run.smithi016.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-05-20T01:13:38.688 INFO:teuthology.orchestra.run.smithi016.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-05-20T01:13:38.689 INFO:teuthology.orchestra.run.smithi016.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-05-20T01:13:38.690 INFO:teuthology.orchestra.run.smithi016.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-05-20T01:13:38.692 INFO:teuthology.orchestra.run.smithi016.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-05-20T01:13:38.694 INFO:teuthology.orchestra.run.smithi016.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-05-20T01:13:38.697 INFO:teuthology.orchestra.run.smithi016.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-05-20T01:13:38.698 INFO:teuthology.orchestra.run.smithi016.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-05-20T01:13:38.761 INFO:teuthology.orchestra.run.smithi132.stdout:(5/131): ceph-mds-18.0.0-3996.g5ca1fdcf.el8.x86 5.8 MB/s | 2.3 MB 00:00 2023-05-20T01:13:38.819 INFO:teuthology.orchestra.run.smithi132.stdout:(6/131): ceph-mgr-18.0.0-3996.g5ca1fdcf.el8.x86 7.7 MB/s | 1.5 MB 00:00 2023-05-20T01:13:38.843 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2023-05-20T01:13:38.857 INFO:teuthology.orchestra.run.smithi016.stdout:=========================================================================================================================== 2023-05-20T01:13:38.861 INFO:teuthology.orchestra.run.smithi016.stdout: Package Arch Version Repository Size 2023-05-20T01:13:38.861 INFO:teuthology.orchestra.run.smithi016.stdout:=========================================================================================================================== 2023-05-20T01:13:38.862 INFO:teuthology.orchestra.run.smithi016.stdout:Installing: 2023-05-20T01:13:38.862 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-05-20T01:13:38.862 INFO:teuthology.orchestra.run.smithi016.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-05-20T01:13:38.863 INFO:teuthology.orchestra.run.smithi016.stdout: ceph x86_64 2:18.0.0-3996.g5ca1fdcf.el8 ceph 6.4 k 2023-05-20T01:13:38.863 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-base x86_64 2:18.0.0-3996.g5ca1fdcf.el8 ceph 5.3 M 2023-05-20T01:13:38.863 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-fuse x86_64 2:18.0.0-3996.g5ca1fdcf.el8 ceph 848 k 2023-05-20T01:13:38.864 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-3996.g5ca1fdcf.el8 ceph 165 k 2023-05-20T01:13:38.864 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr x86_64 2:18.0.0-3996.g5ca1fdcf.el8 ceph 1.5 M 2023-05-20T01:13:38.864 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-cephadm noarch 2:18.0.0-3996.g5ca1fdcf.el8 ceph-noarch 127 k 2023-05-20T01:13:38.865 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-dashboard noarch 2:18.0.0-3996.g5ca1fdcf.el8 ceph-noarch 1.7 M 2023-05-20T01:13:38.865 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-3996.g5ca1fdcf.el8 ceph-noarch 7.4 M 2023-05-20T01:13:38.865 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-rook noarch 2:18.0.0-3996.g5ca1fdcf.el8 ceph-noarch 47 k 2023-05-20T01:13:38.865 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-radosgw x86_64 2:18.0.0-3996.g5ca1fdcf.el8 ceph 13 M 2023-05-20T01:13:38.865 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-test x86_64 2:18.0.0-3996.g5ca1fdcf.el8 ceph 41 M 2023-05-20T01:13:38.865 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-volume noarch 2:18.0.0-3996.g5ca1fdcf.el8 ceph-noarch 264 k 2023-05-20T01:13:38.866 INFO:teuthology.orchestra.run.smithi016.stdout: cephadm noarch 2:18.0.0-3996.g5ca1fdcf.el8 ceph-noarch 83 k 2023-05-20T01:13:38.866 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-mirror x86_64 2:18.0.0-3996.g5ca1fdcf.el8 ceph 258 k 2023-05-20T01:13:38.866 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-top noarch 2:18.0.0-3996.g5ca1fdcf.el8 ceph-noarch 18 k 2023-05-20T01:13:38.866 INFO:teuthology.orchestra.run.smithi016.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-05-20T01:13:38.866 INFO:teuthology.orchestra.run.smithi016.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-05-20T01:13:38.866 INFO:teuthology.orchestra.run.smithi016.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-05-20T01:13:38.866 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs-devel x86_64 2:18.0.0-3996.g5ca1fdcf.el8 ceph 31 k 2023-05-20T01:13:38.866 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs2 x86_64 2:18.0.0-3996.g5ca1fdcf.el8 ceph 771 k 2023-05-20T01:13:38.867 INFO:teuthology.orchestra.run.smithi016.stdout: librados-devel x86_64 2:18.0.0-3996.g5ca1fdcf.el8 ceph 139 k 2023-05-20T01:13:38.867 INFO:teuthology.orchestra.run.smithi016.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-05-20T01:13:38.867 INFO:teuthology.orchestra.run.smithi016.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-05-20T01:13:38.867 INFO:teuthology.orchestra.run.smithi016.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-05-20T01:13:38.867 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-05-20T01:13:38.867 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-05-20T01:13:38.868 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-05-20T01:13:38.868 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cephfs x86_64 2:18.0.0-3996.g5ca1fdcf.el8 ceph 197 k 2023-05-20T01:13:38.868 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rados x86_64 2:18.0.0-3996.g5ca1fdcf.el8 ceph 370 k 2023-05-20T01:13:38.868 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rbd x86_64 2:18.0.0-3996.g5ca1fdcf.el8 ceph 364 k 2023-05-20T01:13:38.868 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rgw x86_64 2:18.0.0-3996.g5ca1fdcf.el8 ceph 107 k 2023-05-20T01:13:38.868 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-fuse x86_64 2:18.0.0-3996.g5ca1fdcf.el8 ceph 94 k 2023-05-20T01:13:38.869 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-mirror x86_64 2:18.0.0-3996.g5ca1fdcf.el8 ceph 3.4 M 2023-05-20T01:13:38.869 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-nbd x86_64 2:18.0.0-3996.g5ca1fdcf.el8 ceph 180 k 2023-05-20T01:13:38.869 INFO:teuthology.orchestra.run.smithi016.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-05-20T01:13:38.869 INFO:teuthology.orchestra.run.smithi016.stdout:Upgrading: 2023-05-20T01:13:38.869 INFO:teuthology.orchestra.run.smithi016.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-05-20T01:13:38.870 INFO:teuthology.orchestra.run.smithi016.stdout: librados2 x86_64 2:18.0.0-3996.g5ca1fdcf.el8 ceph 3.7 M 2023-05-20T01:13:38.870 INFO:teuthology.orchestra.run.smithi016.stdout: librbd1 x86_64 2:18.0.0-3996.g5ca1fdcf.el8 ceph 3.6 M 2023-05-20T01:13:38.870 INFO:teuthology.orchestra.run.smithi016.stdout:Installing dependencies: 2023-05-20T01:13:38.870 INFO:teuthology.orchestra.run.smithi016.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-05-20T01:13:38.870 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-common x86_64 2:18.0.0-3996.g5ca1fdcf.el8 ceph 22 M 2023-05-20T01:13:38.870 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-grafana-dashboards noarch 2:18.0.0-3996.g5ca1fdcf.el8 ceph-noarch 23 k 2023-05-20T01:13:38.870 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mds x86_64 2:18.0.0-3996.g5ca1fdcf.el8 ceph 2.3 M 2023-05-20T01:13:38.871 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-modules-core noarch 2:18.0.0-3996.g5ca1fdcf.el8 ceph-noarch 240 k 2023-05-20T01:13:38.871 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mon x86_64 2:18.0.0-3996.g5ca1fdcf.el8 ceph 4.6 M 2023-05-20T01:13:38.871 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-osd x86_64 2:18.0.0-3996.g5ca1fdcf.el8 ceph 18 M 2023-05-20T01:13:38.871 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-prometheus-alerts noarch 2:18.0.0-3996.g5ca1fdcf.el8 ceph-noarch 15 k 2023-05-20T01:13:38.871 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-selinux x86_64 2:18.0.0-3996.g5ca1fdcf.el8 ceph 24 k 2023-05-20T01:13:38.871 INFO:teuthology.orchestra.run.smithi016.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-05-20T01:13:38.871 INFO:teuthology.orchestra.run.smithi016.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-05-20T01:13:38.871 INFO:teuthology.orchestra.run.smithi016.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-05-20T01:13:38.872 INFO:teuthology.orchestra.run.smithi016.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-05-20T01:13:38.872 INFO:teuthology.orchestra.run.smithi016.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-05-20T01:13:38.872 INFO:teuthology.orchestra.run.smithi016.stdout: libcephsqlite x86_64 2:18.0.0-3996.g5ca1fdcf.el8 ceph 137 k 2023-05-20T01:13:38.872 INFO:teuthology.orchestra.run.smithi016.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-05-20T01:13:38.872 INFO:teuthology.orchestra.run.smithi016.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-05-20T01:13:38.872 INFO:teuthology.orchestra.run.smithi016.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-05-20T01:13:38.872 INFO:teuthology.orchestra.run.smithi016.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-05-20T01:13:38.873 INFO:teuthology.orchestra.run.smithi016.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-05-20T01:13:38.873 INFO:teuthology.orchestra.run.smithi016.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-05-20T01:13:38.873 INFO:teuthology.orchestra.run.smithi016.stdout: libradosstriper1 x86_64 2:18.0.0-3996.g5ca1fdcf.el8 ceph 473 k 2023-05-20T01:13:38.873 INFO:teuthology.orchestra.run.smithi016.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-05-20T01:13:38.873 INFO:teuthology.orchestra.run.smithi016.stdout: librgw2 x86_64 2:18.0.0-3996.g5ca1fdcf.el8 ceph 6.5 M 2023-05-20T01:13:38.873 INFO:teuthology.orchestra.run.smithi016.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-05-20T01:13:38.873 INFO:teuthology.orchestra.run.smithi016.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-05-20T01:13:38.874 INFO:teuthology.orchestra.run.smithi016.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-05-20T01:13:38.874 INFO:teuthology.orchestra.run.smithi016.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-05-20T01:13:38.874 INFO:teuthology.orchestra.run.smithi016.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-05-20T01:13:38.874 INFO:teuthology.orchestra.run.smithi016.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-05-20T01:13:38.874 INFO:teuthology.orchestra.run.smithi016.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-05-20T01:13:38.874 INFO:teuthology.orchestra.run.smithi016.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-05-20T01:13:38.874 INFO:teuthology.orchestra.run.smithi016.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-05-20T01:13:38.875 INFO:teuthology.orchestra.run.smithi016.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-05-20T01:13:38.875 INFO:teuthology.orchestra.run.smithi016.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-05-20T01:13:38.875 INFO:teuthology.orchestra.run.smithi016.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-05-20T01:13:38.875 INFO:teuthology.orchestra.run.smithi016.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-05-20T01:13:38.876 INFO:teuthology.orchestra.run.smithi016.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-05-20T01:13:38.877 INFO:teuthology.orchestra.run.smithi016.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-05-20T01:13:38.877 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-05-20T01:13:38.877 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-argparse x86_64 2:18.0.0-3996.g5ca1fdcf.el8 ceph 45 k 2023-05-20T01:13:38.878 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-common x86_64 2:18.0.0-3996.g5ca1fdcf.el8 ceph 119 k 2023-05-20T01:13:38.878 INFO:teuthology.orchestra.run.smithi016.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-05-20T01:13:38.878 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-05-20T01:13:38.878 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-05-20T01:13:38.879 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-05-20T01:13:38.879 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-05-20T01:13:38.879 INFO:teuthology.orchestra.run.smithi016.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-05-20T01:13:38.880 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-05-20T01:13:38.880 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-05-20T01:13:38.880 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-05-20T01:13:38.881 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-05-20T01:13:38.881 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-05-20T01:13:38.881 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-05-20T01:13:38.882 INFO:teuthology.orchestra.run.smithi016.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-05-20T01:13:38.882 INFO:teuthology.orchestra.run.smithi016.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-05-20T01:13:38.882 INFO:teuthology.orchestra.run.smithi016.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-05-20T01:13:38.882 INFO:teuthology.orchestra.run.smithi016.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-05-20T01:13:38.883 INFO:teuthology.orchestra.run.smithi016.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-05-20T01:13:38.883 INFO:teuthology.orchestra.run.smithi016.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-05-20T01:13:38.883 INFO:teuthology.orchestra.run.smithi016.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-05-20T01:13:38.883 INFO:teuthology.orchestra.run.smithi016.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-05-20T01:13:38.883 INFO:teuthology.orchestra.run.smithi016.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-05-20T01:13:38.884 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-05-20T01:13:38.884 INFO:teuthology.orchestra.run.smithi016.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-05-20T01:13:38.884 INFO:teuthology.orchestra.run.smithi016.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-05-20T01:13:38.885 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-05-20T01:13:38.885 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-05-20T01:13:38.885 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-05-20T01:13:38.885 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-05-20T01:13:38.886 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-05-20T01:13:38.886 INFO:teuthology.orchestra.run.smithi016.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-05-20T01:13:38.886 INFO:teuthology.orchestra.run.smithi016.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-05-20T01:13:38.886 INFO:teuthology.orchestra.run.smithi016.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-05-20T01:13:38.886 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-05-20T01:13:38.886 INFO:teuthology.orchestra.run.smithi016.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-05-20T01:13:38.887 INFO:teuthology.orchestra.run.smithi016.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-05-20T01:13:38.887 INFO:teuthology.orchestra.run.smithi016.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-05-20T01:13:38.887 INFO:teuthology.orchestra.run.smithi016.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-05-20T01:13:38.887 INFO:teuthology.orchestra.run.smithi016.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-05-20T01:13:38.887 INFO:teuthology.orchestra.run.smithi016.stdout: python3-waitress noarch 1.2.1-2.el8.1 epel 241 k 2023-05-20T01:13:38.887 INFO:teuthology.orchestra.run.smithi016.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-05-20T01:13:38.887 INFO:teuthology.orchestra.run.smithi016.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-05-20T01:13:38.887 INFO:teuthology.orchestra.run.smithi016.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-05-20T01:13:38.888 INFO:teuthology.orchestra.run.smithi016.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-05-20T01:13:38.888 INFO:teuthology.orchestra.run.smithi016.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-05-20T01:13:38.888 INFO:teuthology.orchestra.run.smithi016.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-05-20T01:13:38.888 INFO:teuthology.orchestra.run.smithi016.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-05-20T01:13:38.888 INFO:teuthology.orchestra.run.smithi016.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-05-20T01:13:38.889 INFO:teuthology.orchestra.run.smithi016.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-05-20T01:13:38.889 INFO:teuthology.orchestra.run.smithi016.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-05-20T01:13:38.889 INFO:teuthology.orchestra.run.smithi016.stdout:Enabling module streams: 2023-05-20T01:13:38.889 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql 10 2023-05-20T01:13:38.890 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:13:38.890 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction Summary 2023-05-20T01:13:38.890 INFO:teuthology.orchestra.run.smithi016.stdout:=========================================================================================================================== 2023-05-20T01:13:38.890 INFO:teuthology.orchestra.run.smithi016.stdout:Install 128 Packages 2023-05-20T01:13:38.890 INFO:teuthology.orchestra.run.smithi016.stdout:Upgrade 3 Packages 2023-05-20T01:13:38.890 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:13:38.891 INFO:teuthology.orchestra.run.smithi016.stdout:Total download size: 197 M 2023-05-20T01:13:38.891 INFO:teuthology.orchestra.run.smithi016.stdout:Downloading Packages: 2023-05-20T01:13:39.248 INFO:teuthology.orchestra.run.smithi016.stdout:(1/131): ceph-18.0.0-3996.g5ca1fdcf.el8.x86_64. 19 kB/s | 6.4 kB 00:00 2023-05-20T01:13:39.578 INFO:teuthology.orchestra.run.smithi132.stdout:(7/131): ceph-mon-18.0.0-3996.g5ca1fdcf.el8.x86 5.6 MB/s | 4.6 MB 00:00 2023-05-20T01:13:40.015 INFO:teuthology.orchestra.run.smithi016.stdout:(2/131): ceph-fuse-18.0.0-3996.g5ca1fdcf.el8.x8 1.1 MB/s | 848 kB 00:00 2023-05-20T01:13:40.095 INFO:teuthology.orchestra.run.smithi132.stdout:(8/131): ceph-common-18.0.0-3996.g5ca1fdcf.el8. 10 MB/s | 22 MB 00:02 2023-05-20T01:13:40.115 INFO:teuthology.orchestra.run.smithi016.stdout:(3/131): ceph-immutable-object-cache-18.0.0-399 1.6 MB/s | 165 kB 00:00 2023-05-20T01:13:40.162 INFO:teuthology.orchestra.run.smithi132.stdout:(9/131): ceph-selinux-18.0.0-3996.g5ca1fdcf.el8 364 kB/s | 24 kB 00:00 2023-05-20T01:13:40.441 INFO:teuthology.orchestra.run.smithi016.stdout:(4/131): ceph-base-18.0.0-3996.g5ca1fdcf.el8.x8 3.5 MB/s | 5.3 MB 00:01 2023-05-20T01:13:40.774 INFO:teuthology.orchestra.run.smithi016.stdout:(5/131): ceph-mgr-18.0.0-3996.g5ca1fdcf.el8.x86 4.6 MB/s | 1.5 MB 00:00 2023-05-20T01:13:40.829 INFO:teuthology.orchestra.run.smithi132.stdout:(10/131): ceph-osd-18.0.0-3996.g5ca1fdcf.el8.x8 8.8 MB/s | 18 MB 00:02 2023-05-20T01:13:40.920 INFO:teuthology.orchestra.run.smithi132.stdout:(11/131): cephfs-mirror-18.0.0-3996.g5ca1fdcf.e 2.8 MB/s | 258 kB 00:00 2023-05-20T01:13:41.004 INFO:teuthology.orchestra.run.smithi132.stdout:(12/131): libcephfs-devel-18.0.0-3996.g5ca1fdcf 372 kB/s | 31 kB 00:00 2023-05-20T01:13:41.163 INFO:teuthology.orchestra.run.smithi132.stdout:(13/131): libcephfs2-18.0.0-3996.g5ca1fdcf.el8. 4.8 MB/s | 771 kB 00:00 2023-05-20T01:13:41.246 INFO:teuthology.orchestra.run.smithi132.stdout:(14/131): libcephsqlite-18.0.0-3996.g5ca1fdcf.e 1.6 MB/s | 137 kB 00:00 2023-05-20T01:13:41.330 INFO:teuthology.orchestra.run.smithi132.stdout:(15/131): librados-devel-18.0.0-3996.g5ca1fdcf. 1.6 MB/s | 139 kB 00:00 2023-05-20T01:13:41.430 INFO:teuthology.orchestra.run.smithi132.stdout:(16/131): libradosstriper1-18.0.0-3996.g5ca1fdc 4.6 MB/s | 473 kB 00:00 2023-05-20T01:13:41.491 INFO:teuthology.orchestra.run.smithi016.stdout:(6/131): ceph-mds-18.0.0-3996.g5ca1fdcf.el8.x86 1.7 MB/s | 2.3 MB 00:01 2023-05-20T01:13:41.599 INFO:teuthology.orchestra.run.smithi016.stdout:(7/131): ceph-mon-18.0.0-3996.g5ca1fdcf.el8.x86 5.6 MB/s | 4.6 MB 00:00 2023-05-20T01:13:42.164 INFO:teuthology.orchestra.run.smithi132.stdout:(17/131): librgw2-18.0.0-3996.g5ca1fdcf.el8.x86 8.9 MB/s | 6.5 MB 00:00 2023-05-20T01:13:42.239 INFO:teuthology.orchestra.run.smithi132.stdout:(18/131): python3-ceph-argparse-18.0.0-3996.g5c 596 kB/s | 45 kB 00:00 2023-05-20T01:13:42.314 INFO:teuthology.orchestra.run.smithi132.stdout:(19/131): python3-ceph-common-18.0.0-3996.g5ca1 1.5 MB/s | 119 kB 00:00 2023-05-20T01:13:42.406 INFO:teuthology.orchestra.run.smithi132.stdout:(20/131): python3-cephfs-18.0.0-3996.g5ca1fdcf. 2.1 MB/s | 197 kB 00:00 2023-05-20T01:13:42.574 INFO:teuthology.orchestra.run.smithi132.stdout:(21/131): ceph-radosgw-18.0.0-3996.g5ca1fdcf.el 4.4 MB/s | 13 MB 00:02 2023-05-20T01:13:42.599 INFO:teuthology.orchestra.run.smithi132.stdout:(22/131): python3-rados-18.0.0-3996.g5ca1fdcf.e 1.9 MB/s | 370 kB 00:00 2023-05-20T01:13:42.674 INFO:teuthology.orchestra.run.smithi132.stdout:(23/131): python3-rgw-18.0.0-3996.g5ca1fdcf.el8 1.4 MB/s | 107 kB 00:00 2023-05-20T01:13:42.700 INFO:teuthology.orchestra.run.smithi132.stdout:(24/131): python3-rbd-18.0.0-3996.g5ca1fdcf.el8 2.8 MB/s | 364 kB 00:00 2023-05-20T01:13:42.742 INFO:teuthology.orchestra.run.smithi132.stdout:(25/131): rbd-fuse-18.0.0-3996.g5ca1fdcf.el8.x8 1.4 MB/s | 94 kB 00:00 2023-05-20T01:13:42.817 INFO:teuthology.orchestra.run.smithi132.stdout:(26/131): rbd-nbd-18.0.0-3996.g5ca1fdcf.el8.x86 2.3 MB/s | 180 kB 00:00 2023-05-20T01:13:42.892 INFO:teuthology.orchestra.run.smithi132.stdout:(27/131): ceph-grafana-dashboards-18.0.0-3996.g 308 kB/s | 23 kB 00:00 2023-05-20T01:13:42.976 INFO:teuthology.orchestra.run.smithi132.stdout:(28/131): ceph-mgr-cephadm-18.0.0-3996.g5ca1fdc 1.5 MB/s | 127 kB 00:00 2023-05-20T01:13:43.226 INFO:teuthology.orchestra.run.smithi132.stdout:(29/131): ceph-mgr-dashboard-18.0.0-3996.g5ca1f 6.8 MB/s | 1.7 MB 00:00 2023-05-20T01:13:43.435 INFO:teuthology.orchestra.run.smithi132.stdout:(30/131): rbd-mirror-18.0.0-3996.g5ca1fdcf.el8. 4.7 MB/s | 3.4 MB 00:00 2023-05-20T01:13:43.535 INFO:teuthology.orchestra.run.smithi132.stdout:(31/131): ceph-mgr-modules-core-18.0.0-3996.g5c 2.4 MB/s | 240 kB 00:00 2023-05-20T01:13:43.627 INFO:teuthology.orchestra.run.smithi132.stdout:(32/131): ceph-mgr-rook-18.0.0-3996.g5ca1fdcf.e 517 kB/s | 47 kB 00:00 2023-05-20T01:13:43.711 INFO:teuthology.orchestra.run.smithi132.stdout:(33/131): ceph-prometheus-alerts-18.0.0-3996.g5 175 kB/s | 15 kB 00:00 2023-05-20T01:13:43.811 INFO:teuthology.orchestra.run.smithi132.stdout:(34/131): ceph-volume-18.0.0-3996.g5ca1fdcf.el8 2.6 MB/s | 264 kB 00:00 2023-05-20T01:13:43.903 INFO:teuthology.orchestra.run.smithi132.stdout:(35/131): cephadm-18.0.0-3996.g5ca1fdcf.el8.noa 911 kB/s | 83 kB 00:00 2023-05-20T01:13:43.995 INFO:teuthology.orchestra.run.smithi132.stdout:(36/131): ceph-mgr-diskprediction-local-18.0.0- 9.6 MB/s | 7.4 MB 00:00 2023-05-20T01:13:44.012 INFO:teuthology.orchestra.run.smithi132.stdout:(37/131): cephfs-top-18.0.0-3996.g5ca1fdcf.el8. 166 kB/s | 18 kB 00:00 2023-05-20T01:13:44.096 INFO:teuthology.orchestra.run.smithi132.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 706 kB/s | 59 kB 00:00 2023-05-20T01:13:44.246 INFO:teuthology.orchestra.run.smithi132.stdout:(39/131): gperftools-libs-2.7-9.el8.x86_64.rpm 1.2 MB/s | 306 kB 00:00 2023-05-20T01:13:44.271 INFO:teuthology.orchestra.run.smithi132.stdout:(40/131): libunwind-1.3.1-3.el8.x86_64.rpm 428 kB/s | 75 kB 00:00 2023-05-20T01:13:44.305 INFO:teuthology.orchestra.run.smithi132.stdout:(41/131): python3-asyncssh-2.7.0-2.el8.noarch.r 7.9 MB/s | 476 kB 00:00 2023-05-20T01:13:44.330 INFO:teuthology.orchestra.run.smithi132.stdout:(42/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 745 kB/s | 44 kB 00:00 2023-05-20T01:13:44.333 INFO:teuthology.orchestra.run.smithi016.stdout:(8/131): ceph-radosgw-18.0.0-3996.g5ca1fdcf.el8 4.8 MB/s | 13 MB 00:02 2023-05-20T01:13:44.356 INFO:teuthology.orchestra.run.smithi132.stdout:(43/131): python3-beautifulsoup4-4.6.3-2.el8.1. 3.6 MB/s | 185 kB 00:00 2023-05-20T01:13:44.381 INFO:teuthology.orchestra.run.smithi132.stdout:(44/131): python3-cachetools-3.1.1-4.el8.noarch 653 kB/s | 33 kB 00:00 2023-05-20T01:13:44.391 INFO:teuthology.orchestra.run.smithi016.stdout:(9/131): ceph-selinux-18.0.0-3996.g5ca1fdcf.el8 419 kB/s | 24 kB 00:00 2023-05-20T01:13:44.406 INFO:teuthology.orchestra.run.smithi132.stdout:(45/131): python3-certifi-2018.10.15-7.el8.noar 325 kB/s | 16 kB 00:00 2023-05-20T01:13:44.440 INFO:teuthology.orchestra.run.smithi132.stdout:(46/131): python3-cheroot-8.5.2-1.el8.noarch.rp 2.9 MB/s | 173 kB 00:00 2023-05-20T01:13:44.474 INFO:teuthology.orchestra.run.smithi132.stdout:(47/131): python3-cherrypy-18.4.0-1.el8.noarch. 5.6 MB/s | 384 kB 00:00 2023-05-20T01:13:44.499 INFO:teuthology.orchestra.run.smithi132.stdout:(48/131): python3-google-auth-1.1.1-10.el8.noar 1.4 MB/s | 81 kB 00:00 2023-05-20T01:13:44.524 INFO:teuthology.orchestra.run.smithi132.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 221 kB/s | 11 kB 00:00 2023-05-20T01:13:44.550 INFO:teuthology.orchestra.run.smithi132.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 351 kB/s | 18 kB 00:00 2023-05-20T01:13:44.592 INFO:teuthology.orchestra.run.smithi132.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 1.1 MB/s | 49 kB 00:00 2023-05-20T01:13:44.625 INFO:teuthology.orchestra.run.smithi132.stdout:(52/131): python3-kubernetes-11.0.0-6.el8.noarc 12 MB/s | 1.2 MB 00:00 2023-05-20T01:13:44.650 INFO:teuthology.orchestra.run.smithi132.stdout:(53/131): python3-more-itertools-7.2.0-3.el8.no 1.0 MB/s | 59 kB 00:00 2023-05-20T01:13:44.676 INFO:teuthology.orchestra.run.smithi132.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 1.2 MB/s | 60 kB 00:00 2023-05-20T01:13:44.701 INFO:teuthology.orchestra.run.smithi132.stdout:(55/131): python3-pecan-1.3.2-9.el8.noarch.rpm 5.5 MB/s | 283 kB 00:00 2023-05-20T01:13:44.727 INFO:teuthology.orchestra.run.smithi132.stdout:(56/131): python3-portend-2.6-1.el8.noarch.rpm 309 kB/s | 16 kB 00:00 2023-05-20T01:13:44.752 INFO:teuthology.orchestra.run.smithi132.stdout:(57/131): python3-repoze-lru-0.7-6.el8.noarch.r 671 kB/s | 34 kB 00:00 2023-05-20T01:13:44.777 INFO:teuthology.orchestra.run.smithi132.stdout:(58/131): python3-routes-2.4.1-12.el8.noarch.rp 3.8 MB/s | 196 kB 00:00 2023-05-20T01:13:44.802 INFO:teuthology.orchestra.run.smithi132.stdout:(59/131): python3-rsa-4.8-1.el8.noarch.rpm 1.2 MB/s | 61 kB 00:00 2023-05-20T01:13:44.828 INFO:teuthology.orchestra.run.smithi132.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 373 kB/s | 19 kB 00:00 2023-05-20T01:13:44.853 INFO:teuthology.orchestra.run.smithi132.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 478 kB/s | 24 kB 00:00 2023-05-20T01:13:44.879 INFO:teuthology.orchestra.run.smithi132.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 581 kB/s | 29 kB 00:00 2023-05-20T01:13:44.904 INFO:teuthology.orchestra.run.smithi132.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 547 kB/s | 27 kB 00:00 2023-05-20T01:13:44.929 INFO:teuthology.orchestra.run.smithi132.stdout:(64/131): python3-waitress-1.2.1-2.el8.1.noarch 4.7 MB/s | 241 kB 00:00 2023-05-20T01:13:44.954 INFO:teuthology.orchestra.run.smithi132.stdout:(65/131): python3-webob-1.8.5-1.el8.1.noarch.rp 4.9 MB/s | 251 kB 00:00 2023-05-20T01:13:44.979 INFO:teuthology.orchestra.run.smithi132.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 1.2 MB/s | 61 kB 00:00 2023-05-20T01:13:44.997 INFO:teuthology.orchestra.run.smithi132.stdout:(67/131): python3-webtest-2.0.33-1.el8.noarch.r 2.0 MB/s | 85 kB 00:00 2023-05-20T01:13:45.022 INFO:teuthology.orchestra.run.smithi132.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 548 kB/s | 23 kB 00:00 2023-05-20T01:13:45.047 INFO:teuthology.orchestra.run.smithi132.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.7 MB/s | 70 kB 00:00 2023-05-20T01:13:45.090 INFO:teuthology.orchestra.run.smithi132.stdout:(70/131): thrift-0.13.0-2.el8.x86_64.rpm 19 MB/s | 1.7 MB 00:00 2023-05-20T01:13:45.140 INFO:teuthology.orchestra.run.smithi132.stdout:(71/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 510 kB/s | 47 kB 00:00 2023-05-20T01:13:45.165 INFO:teuthology.orchestra.run.smithi132.stdout:(72/131): libselinux-devel-2.9-5.el8.x86_64.rpm 2.8 MB/s | 200 kB 00:00 2023-05-20T01:13:45.190 INFO:teuthology.orchestra.run.smithi132.stdout:(73/131): python3-jwt-1.6.1-2.el8.noarch.rpm 847 kB/s | 43 kB 00:00 2023-05-20T01:13:45.207 INFO:teuthology.orchestra.run.smithi132.stdout:(74/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 5.2 MB/s | 220 kB 00:00 2023-05-20T01:13:45.233 INFO:teuthology.orchestra.run.smithi132.stdout:(75/131): keyutils-libs-devel-1.5.10-9.el8.x86_ 1.1 MB/s | 48 kB 00:00 2023-05-20T01:13:45.258 INFO:teuthology.orchestra.run.smithi132.stdout:(76/131): libsepol-devel-2.9-3.el8.x86_64.rpm 1.7 MB/s | 87 kB 00:00 2023-05-20T01:13:45.275 INFO:teuthology.orchestra.run.smithi132.stdout:(77/131): python3-requests-oauthlib-1.0.0-1.el8 1.0 MB/s | 43 kB 00:00 2023-05-20T01:13:45.308 INFO:teuthology.orchestra.run.smithi132.stdout:(78/131): python3-pycparser-2.14-14.el8.noarch. 2.1 MB/s | 109 kB 00:00 2023-05-20T01:13:45.325 INFO:teuthology.orchestra.run.smithi132.stdout:(79/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 4.5 MB/s | 229 kB 00:00 2023-05-20T01:13:45.368 INFO:teuthology.orchestra.run.smithi132.stdout:(80/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 9.3 MB/s | 560 kB 00:00 2023-05-20T01:13:45.393 INFO:teuthology.orchestra.run.smithi132.stdout:(81/131): libaio-devel-0.3.112-1.el8.x86_64.rpm 279 kB/s | 19 kB 00:00 2023-05-20T01:13:45.418 INFO:teuthology.orchestra.run.smithi132.stdout:(82/131): python3-oauthlib-2.1.0-1.el8.noarch.r 3.0 MB/s | 155 kB 00:00 2023-05-20T01:13:45.452 INFO:teuthology.orchestra.run.smithi132.stdout:(83/131): python3-cryptography-3.2.1-5.el8.x86_ 9.3 MB/s | 559 kB 00:00 2023-05-20T01:13:45.477 INFO:teuthology.orchestra.run.smithi132.stdout:(84/131): libverto-devel-0.3.0-5.el8.x86_64.rpm 306 kB/s | 18 kB 00:00 2023-05-20T01:13:45.540 INFO:teuthology.orchestra.run.smithi132.stdout:(85/131): pcre2-devel-10.32-2.el8.x86_64.rpm 6.7 MB/s | 605 kB 00:00 2023-05-20T01:13:45.569 INFO:teuthology.orchestra.run.smithi132.stdout:(86/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 2.5 MB/s | 238 kB 00:00 2023-05-20T01:13:45.620 INFO:teuthology.orchestra.run.smithi132.stdout:(87/131): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 1.4 MB/s | 72 kB 00:00 2023-05-20T01:13:45.661 INFO:teuthology.orchestra.run.smithi132.stdout:(88/131): openssl-devel-1.1.1k-7.el8_6.x86_64.r 19 MB/s | 2.3 MB 00:00 2023-05-20T01:13:45.687 INFO:teuthology.orchestra.run.smithi132.stdout:(89/131): fuse-2.9.7-15.el8.x86_64.rpm 1.2 MB/s | 83 kB 00:00 2023-05-20T01:13:46.046 INFO:teuthology.orchestra.run.smithi132.stdout:(90/131): ceph-test-18.0.0-3996.g5ca1fdcf.el8.x 6.9 MB/s | 41 MB 00:05 2023-05-20T01:13:46.071 INFO:teuthology.orchestra.run.smithi132.stdout:(91/131): elfutils-libelf-devel-0.186-1.el8.x86 146 kB/s | 60 kB 00:00 2023-05-20T01:13:46.096 INFO:teuthology.orchestra.run.smithi132.stdout:(92/131): libuuid-devel-2.32.1-35.el8.x86_64.rp 240 kB/s | 98 kB 00:00 2023-05-20T01:13:46.130 INFO:teuthology.orchestra.run.smithi132.stdout:(93/131): libcom_err-devel-1.45.6-4.el8.x86_64. 728 kB/s | 39 kB 00:00 2023-05-20T01:13:46.155 INFO:teuthology.orchestra.run.smithi132.stdout:(94/131): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 530 kB/s | 58 kB 00:00 2023-05-20T01:13:46.180 INFO:teuthology.orchestra.run.smithi132.stdout:(95/131): libquadmath-8.5.0-10.1.el8_6.x86_64.r 3.3 MB/s | 171 kB 00:00 2023-05-20T01:13:46.206 INFO:teuthology.orchestra.run.smithi132.stdout:(96/131): NetworkManager-initscripts-updown-1.3 2.7 MB/s | 138 kB 00:00 2023-05-20T01:13:46.240 INFO:teuthology.orchestra.run.smithi132.stdout:(97/131): libgfortran-8.5.0-10.1.el8_6.x86_64.r 4.4 MB/s | 644 kB 00:00 2023-05-20T01:13:46.265 INFO:teuthology.orchestra.run.smithi132.stdout:(98/131): python3-pyasn1-modules-0.3.7-6.el8.no 1.8 MB/s | 110 kB 00:00 2023-05-20T01:13:46.290 INFO:teuthology.orchestra.run.smithi132.stdout:(99/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.1 MB/s | 54 kB 00:00 2023-05-20T01:13:46.315 INFO:teuthology.orchestra.run.smithi132.stdout:(100/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 5.0 MB/s | 259 kB 00:00 2023-05-20T01:13:46.349 INFO:teuthology.orchestra.run.smithi132.stdout:(101/131): socat-1.7.4.1-1.el8.x86_64.rpm 5.4 MB/s | 323 kB 00:00 2023-05-20T01:13:46.374 INFO:teuthology.orchestra.run.smithi132.stdout:(102/131): python3-markupsafe-0.23-19.el8.x86_6 664 kB/s | 39 kB 00:00 2023-05-20T01:13:46.416 INFO:teuthology.orchestra.run.smithi132.stdout:(103/131): bison-3.0.4-10.el8.x86_64.rpm 10 MB/s | 688 kB 00:00 2023-05-20T01:13:46.450 INFO:teuthology.orchestra.run.smithi132.stdout:(104/131): python3-jinja2-2.10.1-3.el8.noarch.r 7.0 MB/s | 538 kB 00:00 2023-05-20T01:13:46.475 INFO:teuthology.orchestra.run.smithi132.stdout:(105/131): python3-mako-1.0.6-13.el8.noarch.rpm 2.6 MB/s | 157 kB 00:00 2023-05-20T01:13:46.500 INFO:teuthology.orchestra.run.smithi132.stdout:(106/131): oniguruma-6.8.2-2.el8.x86_64.rpm 3.6 MB/s | 187 kB 00:00 2023-05-20T01:13:46.534 INFO:teuthology.orchestra.run.smithi132.stdout:(107/131): automake-1.16.1-7.el8.noarch.rpm 12 MB/s | 713 kB 00:00 2023-05-20T01:13:46.559 INFO:teuthology.orchestra.run.smithi132.stdout:(108/131): python3-jsonpatch-1.21-2.el8.noarch. 1.0 MB/s | 27 kB 00:00 2023-05-20T01:13:46.601 INFO:teuthology.orchestra.run.smithi132.stdout:(109/131): flex-2.6.1-9.el8.x86_64.rpm 7.5 MB/s | 320 kB 00:00 2023-05-20T01:13:46.627 INFO:teuthology.orchestra.run.smithi132.stdout:(110/131): python3-prettytable-0.7.2-14.el8.noa 1.7 MB/s | 44 kB 00:00 2023-05-20T01:13:46.719 INFO:teuthology.orchestra.run.smithi132.stdout:(111/131): python3-babel-2.5.1-7.el8.noarch.rpm 22 MB/s | 4.8 MB 00:00 2023-05-20T01:13:46.769 INFO:teuthology.orchestra.run.smithi132.stdout:(112/131): python3-werkzeug-0.12.2-4.el8.noarch 9.0 MB/s | 457 kB 00:00 2023-05-20T01:13:46.811 INFO:teuthology.orchestra.run.smithi132.stdout:(113/131): python3-numpy-f2py-1.14.3-10.el8.x86 5.3 MB/s | 225 kB 00:00 2023-05-20T01:13:46.869 INFO:teuthology.orchestra.run.smithi132.stdout:(114/131): python3-numpy-1.14.3-10.el8.x86_64.r 15 MB/s | 3.7 MB 00:00 2023-05-20T01:13:46.894 INFO:teuthology.orchestra.run.smithi132.stdout:(115/131): uuid-1.6.2-43.el8.x86_64.rpm 766 kB/s | 64 kB 00:00 2023-05-20T01:13:46.928 INFO:teuthology.orchestra.run.smithi132.stdout:(116/131): python3-pyasn1-0.3.7-6.el8.noarch.rp 2.1 MB/s | 126 kB 00:00 2023-05-20T01:13:46.953 INFO:teuthology.orchestra.run.smithi132.stdout:(117/131): python3-jsonpointer-1.10-11.el8.noar 340 kB/s | 20 kB 00:00 2023-05-20T01:13:46.979 INFO:teuthology.orchestra.run.smithi132.stdout:(118/131): python3-pyOpenSSL-19.0.0-1.el8.noarc 2.0 MB/s | 103 kB 00:00 2023-05-20T01:13:47.012 INFO:teuthology.orchestra.run.smithi132.stdout:(119/131): libtool-2.4.6-25.el8.x86_64.rpm 12 MB/s | 709 kB 00:00 2023-05-20T01:13:47.055 INFO:teuthology.orchestra.run.smithi132.stdout:(120/131): libpq-13.5-1.el8.x86_64.rpm 4.6 MB/s | 198 kB 00:00 2023-05-20T01:13:47.168 INFO:teuthology.orchestra.run.smithi016.stdout:(10/131): ceph-common-18.0.0-3996.g5ca1fdcf.el8 2.6 MB/s | 22 MB 00:08 2023-05-20T01:13:47.222 INFO:teuthology.orchestra.run.smithi132.stdout:(121/131): python3-scipy-1.0.0-21.module+el8.5. 13 MB/s | 14 MB 00:01 2023-05-20T01:13:47.263 INFO:teuthology.orchestra.run.smithi132.stdout:(122/131): jq-1.6-3.el8.x86_64.rpm 4.8 MB/s | 202 kB 00:00 2023-05-20T01:13:47.305 INFO:teuthology.orchestra.run.smithi132.stdout:(123/131): librdkafka-0.11.4-3.el8.x86_64.rpm 8.3 MB/s | 354 kB 00:00 2023-05-20T01:13:47.389 INFO:teuthology.orchestra.run.smithi132.stdout:(124/131): openblas-threads-0.3.15-3.el8.x86_64 12 MB/s | 4.9 MB 00:00 2023-05-20T01:13:47.417 INFO:teuthology.orchestra.run.smithi016.stdout:(11/131): cephfs-mirror-18.0.0-3996.g5ca1fdcf.e 1.0 MB/s | 258 kB 00:00 2023-05-20T01:13:47.473 INFO:teuthology.orchestra.run.smithi132.stdout:(125/131): openblas-0.3.15-3.el8.x86_64.rpm 11 MB/s | 4.7 MB 00:00 2023-05-20T01:13:47.507 INFO:teuthology.orchestra.run.smithi132.stdout:(126/131): postgresql-contrib-10.21-2.module+el 6.7 MB/s | 808 kB 00:00 2023-05-20T01:13:47.518 INFO:teuthology.orchestra.run.smithi016.stdout:(12/131): libcephfs-devel-18.0.0-3996.g5ca1fdcf 310 kB/s | 31 kB 00:00 2023-05-20T01:13:47.557 INFO:teuthology.orchestra.run.smithi132.stdout:(127/131): postgresql-10.21-2.module+el8.6.0+15 18 MB/s | 1.5 MB 00:00 2023-05-20T01:13:47.632 INFO:teuthology.orchestra.run.smithi132.stdout:(128/131): postgresql-server-10.21-2.module+el8 16 MB/s | 5.1 MB 00:00 2023-05-20T01:13:47.691 INFO:teuthology.orchestra.run.smithi132.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 13 MB/s | 798 kB 00:00 2023-05-20T01:13:48.101 INFO:teuthology.orchestra.run.smithi016.stdout:(13/131): libcephfs2-18.0.0-3996.g5ca1fdcf.el8. 1.3 MB/s | 771 kB 00:00 2023-05-20T01:13:48.227 INFO:teuthology.orchestra.run.smithi016.stdout:(14/131): libcephsqlite-18.0.0-3996.g5ca1fdcf.e 1.1 MB/s | 137 kB 00:00 2023-05-20T01:13:48.377 INFO:teuthology.orchestra.run.smithi016.stdout:(15/131): librados-devel-18.0.0-3996.g5ca1fdcf. 927 kB/s | 139 kB 00:00 2023-05-20T01:13:48.735 INFO:teuthology.orchestra.run.smithi016.stdout:(16/131): libradosstriper1-18.0.0-3996.g5ca1fdc 1.3 MB/s | 473 kB 00:00 2023-05-20T01:13:50.800 INFO:teuthology.orchestra.run.smithi132.stdout:(130/131): librados2-18.0.0-3996.g5ca1fdcf.el8. 1.1 MB/s | 3.7 MB 00:03 2023-05-20T01:13:51.810 INFO:teuthology.orchestra.run.smithi016.stdout:(17/131): ceph-osd-18.0.0-3996.g5ca1fdcf.el8.x8 1.7 MB/s | 18 MB 00:10 2023-05-20T01:13:51.886 INFO:teuthology.orchestra.run.smithi016.stdout:(18/131): python3-ceph-argparse-18.0.0-3996.g5c 598 kB/s | 45 kB 00:00 2023-05-20T01:13:52.016 INFO:teuthology.orchestra.run.smithi132.stdout:(131/131): librbd1-18.0.0-3996.g5ca1fdcf.el8.x8 818 kB/s | 3.6 MB 00:04 2023-05-20T01:13:52.018 INFO:teuthology.orchestra.run.smithi132.stdout:-------------------------------------------------------------------------------- 2023-05-20T01:13:52.019 INFO:teuthology.orchestra.run.smithi132.stdout:Total 14 MB/s | 197 MB 00:14 2023-05-20T01:13:52.019 INFO:teuthology.orchestra.run.smithi016.stdout:(19/131): python3-ceph-common-18.0.0-3996.g5ca1 892 kB/s | 119 kB 00:00 2023-05-20T01:13:52.211 INFO:teuthology.orchestra.run.smithi016.stdout:(20/131): python3-cephfs-18.0.0-3996.g5ca1fdcf. 1.0 MB/s | 197 kB 00:00 2023-05-20T01:13:52.328 INFO:teuthology.orchestra.run.smithi016.stdout:(21/131): librgw2-18.0.0-3996.g5ca1fdcf.el8.x86 1.8 MB/s | 6.5 MB 00:03 2023-05-20T01:13:52.445 INFO:teuthology.orchestra.run.smithi016.stdout:(22/131): python3-rados-18.0.0-3996.g5ca1fdcf.e 1.5 MB/s | 370 kB 00:00 2023-05-20T01:13:52.521 INFO:teuthology.orchestra.run.smithi016.stdout:(23/131): python3-rbd-18.0.0-3996.g5ca1fdcf.el8 1.9 MB/s | 364 kB 00:00 2023-05-20T01:13:52.579 INFO:teuthology.orchestra.run.smithi016.stdout:(24/131): python3-rgw-18.0.0-3996.g5ca1fdcf.el8 799 kB/s | 107 kB 00:00 2023-05-20T01:13:52.604 INFO:teuthology.orchestra.run.smithi016.stdout:(25/131): rbd-fuse-18.0.0-3996.g5ca1fdcf.el8.x8 1.1 MB/s | 94 kB 00:00 2023-05-20T01:13:52.696 INFO:teuthology.orchestra.run.smithi016.stdout:(26/131): rbd-nbd-18.0.0-3996.g5ca1fdcf.el8.x86 1.9 MB/s | 180 kB 00:00 2023-05-20T01:13:52.763 INFO:teuthology.orchestra.run.smithi016.stdout:(27/131): ceph-grafana-dashboards-18.0.0-3996.g 345 kB/s | 23 kB 00:00 2023-05-20T01:13:52.848 INFO:teuthology.orchestra.run.smithi016.stdout:(28/131): ceph-mgr-cephadm-18.0.0-3996.g5ca1fdc 1.5 MB/s | 127 kB 00:00 2023-05-20T01:13:53.335 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction check 2023-05-20T01:13:53.336 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction check succeeded. 2023-05-20T01:13:53.337 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction test 2023-05-20T01:13:53.764 INFO:teuthology.orchestra.run.smithi016.stdout:(29/131): ceph-mgr-dashboard-18.0.0-3996.g5ca1f 1.9 MB/s | 1.7 MB 00:00 2023-05-20T01:13:54.058 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction test succeeded. 2023-05-20T01:13:54.063 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction 2023-05-20T01:13:54.997 INFO:teuthology.orchestra.run.smithi016.stdout:(30/131): rbd-mirror-18.0.0-3996.g5ca1fdcf.el8. 1.4 MB/s | 3.4 MB 00:02 2023-05-20T01:13:55.198 INFO:teuthology.orchestra.run.smithi016.stdout:(31/131): ceph-mgr-modules-core-18.0.0-3996.g5c 1.2 MB/s | 240 kB 00:00 2023-05-20T01:13:55.290 INFO:teuthology.orchestra.run.smithi016.stdout:(32/131): ceph-mgr-rook-18.0.0-3996.g5ca1fdcf.e 517 kB/s | 47 kB 00:00 2023-05-20T01:13:55.382 INFO:teuthology.orchestra.run.smithi016.stdout:(33/131): ceph-prometheus-alerts-18.0.0-3996.g5 159 kB/s | 15 kB 00:00 2023-05-20T01:13:55.632 INFO:teuthology.orchestra.run.smithi016.stdout:(34/131): ceph-volume-18.0.0-3996.g5ca1fdcf.el8 1.0 MB/s | 264 kB 00:00 2023-05-20T01:13:55.740 INFO:teuthology.orchestra.run.smithi016.stdout:(35/131): cephadm-18.0.0-3996.g5ca1fdcf.el8.noa 771 kB/s | 83 kB 00:00 2023-05-20T01:13:55.832 INFO:teuthology.orchestra.run.smithi016.stdout:(36/131): cephfs-top-18.0.0-3996.g5ca1fdcf.el8. 196 kB/s | 18 kB 00:00 2023-05-20T01:13:56.658 INFO:teuthology.orchestra.run.smithi016.stdout:(37/131): gperftools-libs-2.7-9.el8.x86_64.rpm 371 kB/s | 306 kB 00:00 2023-05-20T01:13:56.766 INFO:teuthology.orchestra.run.smithi016.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 544 kB/s | 59 kB 00:00 2023-05-20T01:13:56.850 INFO:teuthology.orchestra.run.smithi016.stdout:(39/131): ceph-mgr-diskprediction-local-18.0.0- 2.4 MB/s | 7.4 MB 00:03 2023-05-20T01:13:56.900 INFO:teuthology.orchestra.run.smithi016.stdout:(40/131): libunwind-1.3.1-3.el8.x86_64.rpm 560 kB/s | 75 kB 00:00 2023-05-20T01:13:57.001 INFO:teuthology.orchestra.run.smithi016.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 436 kB/s | 44 kB 00:00 2023-05-20T01:13:57.151 INFO:teuthology.orchestra.run.smithi016.stdout:(42/131): python3-beautifulsoup4-4.6.3-2.el8.1. 1.2 MB/s | 185 kB 00:00 2023-05-20T01:13:57.259 INFO:teuthology.orchestra.run.smithi016.stdout:(43/131): python3-cachetools-3.1.1-4.el8.noarch 304 kB/s | 33 kB 00:00 2023-05-20T01:13:57.368 INFO:teuthology.orchestra.run.smithi016.stdout:(44/131): python3-certifi-2018.10.15-7.el8.noar 150 kB/s | 16 kB 00:00 2023-05-20T01:13:57.527 INFO:teuthology.orchestra.run.smithi016.stdout:(45/131): python3-cheroot-8.5.2-1.el8.noarch.rp 1.1 MB/s | 173 kB 00:00 2023-05-20T01:13:57.552 INFO:teuthology.orchestra.run.smithi016.stdout:(46/131): python3-asyncssh-2.7.0-2.el8.noarch.r 678 kB/s | 476 kB 00:00 2023-05-20T01:13:57.677 INFO:teuthology.orchestra.run.smithi016.stdout:(47/131): python3-google-auth-1.1.1-10.el8.noar 651 kB/s | 81 kB 00:00 2023-05-20T01:13:57.703 INFO:teuthology.orchestra.run.smithi016.stdout:(48/131): python3-cherrypy-18.4.0-1.el8.noarch. 2.1 MB/s | 384 kB 00:00 2023-05-20T01:13:57.787 INFO:teuthology.orchestra.run.smithi016.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 102 kB/s | 11 kB 00:00 2023-05-20T01:13:57.812 INFO:teuthology.orchestra.run.smithi016.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 162 kB/s | 18 kB 00:00 2023-05-20T01:13:57.920 INFO:teuthology.orchestra.run.smithi016.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 450 kB/s | 49 kB 00:00 2023-05-20T01:13:58.037 INFO:teuthology.orchestra.run.smithi016.stdout:(52/131): python3-more-itertools-7.2.0-3.el8.no 504 kB/s | 59 kB 00:00 2023-05-20T01:13:58.079 INFO:teuthology.orchestra.run.smithi016.stdout:(53/131): python3-kubernetes-11.0.0-6.el8.noarc 4.0 MB/s | 1.2 MB 00:00 2023-05-20T01:13:58.179 INFO:teuthology.orchestra.run.smithi016.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 425 kB/s | 60 kB 00:00 2023-05-20T01:13:58.205 INFO:teuthology.orchestra.run.smithi016.stdout:(55/131): python3-pecan-1.3.2-9.el8.noarch.rpm 2.2 MB/s | 283 kB 00:00 2023-05-20T01:13:58.597 INFO:teuthology.orchestra.run.smithi016.stdout:(56/131): ceph-test-18.0.0-3996.g5ca1fdcf.el8.x 2.9 MB/s | 41 MB 00:14 2023-05-20T01:13:58.622 INFO:teuthology.orchestra.run.smithi016.stdout:(57/131): python3-portend-2.6-1.el8.noarch.rpm 35 kB/s | 16 kB 00:00 2023-05-20T01:13:58.647 INFO:teuthology.orchestra.run.smithi016.stdout:(58/131): python3-repoze-lru-0.7-6.el8.noarch.r 76 kB/s | 34 kB 00:00 2023-05-20T01:13:58.739 INFO:teuthology.orchestra.run.smithi016.stdout:(59/131): python3-routes-2.4.1-12.el8.noarch.rp 1.4 MB/s | 196 kB 00:00 2023-05-20T01:13:58.765 INFO:teuthology.orchestra.run.smithi016.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 160 kB/s | 19 kB 00:00 2023-05-20T01:13:58.856 INFO:teuthology.orchestra.run.smithi016.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 204 kB/s | 24 kB 00:00 2023-05-20T01:13:58.882 INFO:teuthology.orchestra.run.smithi016.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 250 kB/s | 29 kB 00:00 2023-05-20T01:13:58.974 INFO:teuthology.orchestra.run.smithi016.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 236 kB/s | 27 kB 00:00 2023-05-20T01:13:58.999 INFO:teuthology.orchestra.run.smithi016.stdout:(64/131): python3-waitress-1.2.1-2.el8.1.noarch 2.0 MB/s | 241 kB 00:00 2023-05-20T01:13:59.074 INFO:teuthology.orchestra.run.smithi016.stdout:(65/131): python3-rsa-4.8-1.el8.noarch.rpm 136 kB/s | 61 kB 00:00 2023-05-20T01:13:59.082 INFO:teuthology.orchestra.run.smithi132.stdout: Preparing : 1/1 2023-05-20T01:13:59.116 INFO:teuthology.orchestra.run.smithi016.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 524 kB/s | 61 kB 00:00 2023-05-20T01:13:59.141 INFO:teuthology.orchestra.run.smithi016.stdout:(67/131): python3-webob-1.8.5-1.el8.1.noarch.rp 1.5 MB/s | 251 kB 00:00 2023-05-20T01:13:59.225 INFO:teuthology.orchestra.run.smithi016.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 210 kB/s | 23 kB 00:00 2023-05-20T01:13:59.251 INFO:teuthology.orchestra.run.smithi016.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.8 MB/s | 70 kB 00:00 2023-05-20T01:13:59.272 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-05-20T01:13:59.276 INFO:teuthology.orchestra.run.smithi016.stdout:(70/131): python3-webtest-2.0.33-1.el8.noarch.r 424 kB/s | 85 kB 00:00 2023-05-20T01:13:59.334 INFO:teuthology.orchestra.run.smithi016.stdout:(71/131): python3-pyasn1-modules-0.3.7-6.el8.no 1.8 MB/s | 110 kB 00:00 2023-05-20T01:13:59.376 INFO:teuthology.orchestra.run.smithi016.stdout:(72/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.3 MB/s | 54 kB 00:00 2023-05-20T01:13:59.441 INFO:teuthology.orchestra.run.smithi016.stdout:(73/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 3.9 MB/s | 259 kB 00:00 2023-05-20T01:13:59.447 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-05-20T01:13:59.494 INFO:teuthology.orchestra.run.smithi016.stdout:(74/131): socat-1.7.4.1-1.el8.x86_64.rpm 6.1 MB/s | 323 kB 00:00 2023-05-20T01:13:59.527 INFO:teuthology.orchestra.run.smithi016.stdout:(75/131): thrift-0.13.0-2.el8.x86_64.rpm 4.5 MB/s | 1.7 MB 00:00 2023-05-20T01:13:59.552 INFO:teuthology.orchestra.run.smithi016.stdout:(76/131): python3-markupsafe-0.23-19.el8.x86_64 664 kB/s | 39 kB 00:00 2023-05-20T01:13:59.599 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 3/134 2023-05-20T01:13:59.611 INFO:teuthology.orchestra.run.smithi016.stdout:(77/131): python3-jinja2-2.10.1-3.el8.noarch.rp 11 MB/s | 538 kB 00:00 2023-05-20T01:13:59.645 INFO:teuthology.orchestra.run.smithi016.stdout:(78/131): bison-3.0.4-10.el8.x86_64.rpm 5.8 MB/s | 688 kB 00:00 2023-05-20T01:13:59.670 INFO:teuthology.orchestra.run.smithi016.stdout:(79/131): python3-mako-1.0.6-13.el8.noarch.rpm 2.6 MB/s | 157 kB 00:00 2023-05-20T01:13:59.717 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 3/134 2023-05-20T01:13:59.862 INFO:teuthology.orchestra.run.smithi016.stdout:(80/131): oniguruma-6.8.2-2.el8.x86_64.rpm 865 kB/s | 187 kB 00:00 2023-05-20T01:13:59.916 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-05-20T01:13:59.921 INFO:teuthology.orchestra.run.smithi016.stdout:(81/131): automake-1.16.1-7.el8.noarch.rpm 2.8 MB/s | 713 kB 00:00 2023-05-20T01:13:59.954 INFO:teuthology.orchestra.run.smithi016.stdout:(82/131): python3-jsonpatch-1.21-2.el8.noarch.r 798 kB/s | 27 kB 00:00 2023-05-20T01:13:59.997 INFO:teuthology.orchestra.run.smithi016.stdout:(83/131): flex-2.6.1-9.el8.x86_64.rpm 7.5 MB/s | 320 kB 00:00 2023-05-20T01:14:00.030 INFO:teuthology.orchestra.run.smithi016.stdout:(84/131): python3-prettytable-0.7.2-14.el8.noar 1.3 MB/s | 44 kB 00:00 2023-05-20T01:14:00.155 INFO:teuthology.orchestra.run.smithi016.stdout:(85/131): python3-babel-2.5.1-7.el8.noarch.rpm 16 MB/s | 4.8 MB 00:00 2023-05-20T01:14:00.205 INFO:teuthology.orchestra.run.smithi016.stdout:(86/131): python3-werkzeug-0.12.2-4.el8.noarch. 9.0 MB/s | 457 kB 00:00 2023-05-20T01:14:00.247 INFO:teuthology.orchestra.run.smithi016.stdout:(87/131): python3-numpy-f2py-1.14.3-10.el8.x86_ 5.3 MB/s | 225 kB 00:00 2023-05-20T01:14:00.297 INFO:teuthology.orchestra.run.smithi016.stdout:(88/131): python3-numpy-1.14.3-10.el8.x86_64.rp 14 MB/s | 3.7 MB 00:00 2023-05-20T01:14:00.323 INFO:teuthology.orchestra.run.smithi016.stdout:(89/131): uuid-1.6.2-43.el8.x86_64.rpm 849 kB/s | 64 kB 00:00 2023-05-20T01:14:00.332 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-05-20T01:14:00.348 INFO:teuthology.orchestra.run.smithi016.stdout:(90/131): python3-pyasn1-0.3.7-6.el8.noarch.rpm 2.5 MB/s | 126 kB 00:00 2023-05-20T01:14:00.365 INFO:teuthology.orchestra.run.smithi016.stdout:(91/131): python3-jsonpointer-1.10-11.el8.noarc 477 kB/s | 20 kB 00:00 2023-05-20T01:14:00.391 INFO:teuthology.orchestra.run.smithi016.stdout:(92/131): python3-pyOpenSSL-19.0.0-1.el8.noarch 2.4 MB/s | 103 kB 00:00 2023-05-20T01:14:00.432 INFO:teuthology.orchestra.run.smithi016.stdout:(93/131): libtool-2.4.6-25.el8.x86_64.rpm 10 MB/s | 709 kB 00:00 2023-05-20T01:14:00.574 INFO:teuthology.orchestra.run.smithi016.stdout:(94/131): python3-scipy-1.0.0-21.module+el8.5.0 10 MB/s | 14 MB 00:01 2023-05-20T01:14:00.600 INFO:teuthology.orchestra.run.smithi016.stdout:(95/131): libpq-13.5-1.el8.x86_64.rpm 1.2 MB/s | 198 kB 00:00 2023-05-20T01:14:00.641 INFO:teuthology.orchestra.run.smithi132.stdout: Upgrading : librados2-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 5/134 2023-05-20T01:14:00.642 INFO:teuthology.orchestra.run.smithi016.stdout:(96/131): jq-1.6-3.el8.x86_64.rpm 4.7 MB/s | 202 kB 00:00 2023-05-20T01:14:00.686 INFO:teuthology.orchestra.run.smithi016.stdout:(97/131): librdkafka-0.11.4-3.el8.x86_64.rpm 7.9 MB/s | 354 kB 00:00 2023-05-20T01:14:00.811 INFO:teuthology.orchestra.run.smithi016.stdout:(98/131): openblas-threads-0.3.15-3.el8.x86_64. 12 MB/s | 4.9 MB 00:00 2023-05-20T01:14:00.878 INFO:teuthology.orchestra.run.smithi016.stdout:(99/131): openblas-0.3.15-3.el8.x86_64.rpm 16 MB/s | 4.7 MB 00:00 2023-05-20T01:14:00.920 INFO:teuthology.orchestra.run.smithi016.stdout:(100/131): postgresql-contrib-10.21-2.module+el 7.3 MB/s | 808 kB 00:00 2023-05-20T01:14:00.962 INFO:teuthology.orchestra.run.smithi016.stdout:(101/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2023-05-20T01:14:01.012 INFO:teuthology.orchestra.run.smithi016.stdout:(102/131): postgresql-10.21-2.module+el8.6.0+15 11 MB/s | 1.5 MB 00:00 2023-05-20T01:14:01.033 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: librados2-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 5/134 2023-05-20T01:14:01.037 INFO:teuthology.orchestra.run.smithi016.stdout:(103/131): libselinux-devel-2.9-5.el8.x86_64.rp 2.6 MB/s | 200 kB 00:00 2023-05-20T01:14:01.063 INFO:teuthology.orchestra.run.smithi016.stdout:(104/131): python3-jwt-1.6.1-2.el8.noarch.rpm 847 kB/s | 43 kB 00:00 2023-05-20T01:14:01.088 INFO:teuthology.orchestra.run.smithi016.stdout:(105/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 4.3 MB/s | 220 kB 00:00 2023-05-20T01:14:01.113 INFO:teuthology.orchestra.run.smithi016.stdout:(106/131): keyutils-libs-devel-1.5.10-9.el8.x86 959 kB/s | 48 kB 00:00 2023-05-20T01:14:01.180 INFO:teuthology.orchestra.run.smithi016.stdout:(107/131): postgresql-server-10.21-2.module+el8 10 MB/s | 5.1 MB 00:00 2023-05-20T01:14:01.206 INFO:teuthology.orchestra.run.smithi016.stdout:(108/131): python3-requests-oauthlib-1.0.0-1.el 464 kB/s | 43 kB 00:00 2023-05-20T01:14:01.216 INFO:teuthology.orchestra.run.smithi132.stdout: Upgrading : librbd1-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 6/134 2023-05-20T01:14:01.231 INFO:teuthology.orchestra.run.smithi016.stdout:(109/131): libsepol-devel-2.9-3.el8.x86_64.rpm 611 kB/s | 87 kB 00:00 2023-05-20T01:14:01.256 INFO:teuthology.orchestra.run.smithi016.stdout:(110/131): python3-pycparser-2.14-14.el8.noarch 1.4 MB/s | 109 kB 00:00 2023-05-20T01:14:01.281 INFO:teuthology.orchestra.run.smithi016.stdout:(111/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 3.0 MB/s | 229 kB 00:00 2023-05-20T01:14:01.308 INFO:teuthology.orchestra.run.smithi016.stdout:(112/131): libaio-devel-0.3.112-1.el8.x86_64.rp 372 kB/s | 19 kB 00:00 2023-05-20T01:14:01.324 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: librbd1-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 6/134 2023-05-20T01:14:01.358 INFO:teuthology.orchestra.run.smithi016.stdout:(113/131): python3-oauthlib-2.1.0-1.el8.noarch. 2.0 MB/s | 155 kB 00:00 2023-05-20T01:14:01.391 INFO:teuthology.orchestra.run.smithi016.stdout:(114/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 3.4 MB/s | 560 kB 00:00 2023-05-20T01:14:01.416 INFO:teuthology.orchestra.run.smithi016.stdout:(115/131): libverto-devel-0.3.0-5.el8.x86_64.rp 314 kB/s | 18 kB 00:00 2023-05-20T01:14:01.449 INFO:teuthology.orchestra.run.smithi016.stdout:(116/131): python3-cryptography-3.2.1-5.el8.x86 3.8 MB/s | 559 kB 00:00 2023-05-20T01:14:01.474 INFO:teuthology.orchestra.run.smithi016.stdout:(117/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 4.0 MB/s | 238 kB 00:00 2023-05-20T01:14:01.500 INFO:teuthology.orchestra.run.smithi016.stdout:(118/131): pcre2-devel-10.32-2.el8.x86_64.rpm 5.4 MB/s | 605 kB 00:00 2023-05-20T01:14:01.525 INFO:teuthology.orchestra.run.smithi016.stdout:(119/131): xfsprogs-devel-5.0.0-10.el8.x86_64.r 1.4 MB/s | 72 kB 00:00 2023-05-20T01:14:01.534 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libcephfs2-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 7/134 2023-05-20T01:14:01.550 INFO:teuthology.orchestra.run.smithi016.stdout:(120/131): fuse-2.9.7-15.el8.x86_64.rpm 1.6 MB/s | 83 kB 00:00 2023-05-20T01:14:01.567 INFO:teuthology.orchestra.run.smithi016.stdout:(121/131): elfutils-libelf-devel-0.186-1.el8.x8 1.4 MB/s | 60 kB 00:00 2023-05-20T01:14:01.594 INFO:teuthology.orchestra.run.smithi016.stdout:(122/131): libuuid-devel-2.32.1-35.el8.x86_64.r 2.3 MB/s | 98 kB 00:00 2023-05-20T01:14:01.611 INFO:teuthology.orchestra.run.smithi016.stdout:(123/131): zlib-devel-1.2.11-18.el8_5.x86_64.rp 1.3 MB/s | 58 kB 00:00 2023-05-20T01:14:01.621 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: libcephfs2-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 7/134 2023-05-20T01:14:01.627 INFO:teuthology.orchestra.run.smithi016.stdout:(124/131): libcom_err-devel-1.45.6-4.el8.x86_64 1.1 MB/s | 39 kB 00:00 2023-05-20T01:14:01.678 INFO:teuthology.orchestra.run.smithi016.stdout:(125/131): libgfortran-8.5.0-10.1.el8_6.x86_64. 9.4 MB/s | 644 kB 00:00 2023-05-20T01:14:01.702 INFO:teuthology.orchestra.run.smithi016.stdout:(126/131): libquadmath-8.5.0-10.1.el8_6.x86_64. 2.2 MB/s | 171 kB 00:00 2023-05-20T01:14:01.736 INFO:teuthology.orchestra.run.smithi016.stdout:(127/131): NetworkManager-initscripts-updown-1. 2.3 MB/s | 138 kB 00:00 2023-05-20T01:14:01.787 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-rados-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 8/134 2023-05-20T01:14:01.803 INFO:teuthology.orchestra.run.smithi016.stdout:(128/131): openssl-devel-1.1.1k-7.el8_6.x86_64. 6.6 MB/s | 2.3 MB 00:00 2023-05-20T01:14:01.853 INFO:teuthology.orchestra.run.smithi016.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 16 MB/s | 798 kB 00:00 2023-05-20T01:14:01.958 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/134 2023-05-20T01:14:02.177 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-05-20T01:14:02.286 INFO:teuthology.orchestra.run.smithi016.stdout:(130/131): librbd1-18.0.0-3996.g5ca1fdcf.el8.x8 6.5 MB/s | 3.6 MB 00:00 2023-05-20T01:14:02.377 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-05-20T01:14:02.653 INFO:teuthology.orchestra.run.smithi016.stdout:(131/131): librados2-18.0.0-3996.g5ca1fdcf.el8. 3.8 MB/s | 3.7 MB 00:00 2023-05-20T01:14:02.659 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2023-05-20T01:14:02.659 INFO:teuthology.orchestra.run.smithi016.stdout:Total 8.3 MB/s | 197 MB 00:23 2023-05-20T01:14:02.782 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-05-20T01:14:03.047 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-05-20T01:14:03.215 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libcephsqlite-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 14/134 2023-05-20T01:14:03.320 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: libcephsqlite-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 14/134 2023-05-20T01:14:03.474 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libradosstriper1-2:18.0.0-3996.g5ca1fdcf.el8.x86 15/134 2023-05-20T01:14:03.505 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction check 2023-05-20T01:14:03.606 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: libradosstriper1-2:18.0.0-3996.g5ca1fdcf.el8.x86 15/134 2023-05-20T01:14:03.637 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction check succeeded. 2023-05-20T01:14:03.637 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction test 2023-05-20T01:14:03.785 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 16/134 2023-05-20T01:14:03.954 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 17/134 2023-05-20T01:14:04.130 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 18/134 2023-05-20T01:14:04.265 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-05-20T01:14:04.402 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-05-20T01:14:04.540 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-05-20T01:14:04.785 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction test succeeded. 2023-05-20T01:14:04.789 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction 2023-05-20T01:14:05.193 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-05-20T01:14:05.373 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-05-20T01:14:05.952 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-05-20T01:14:06.157 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : openblas-0.3.15-3.el8.x86_64 22/134 2023-05-20T01:14:06.887 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 22/134 2023-05-20T01:14:07.147 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 23/134 2023-05-20T01:14:07.347 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 24/134 2023-05-20T01:14:07.557 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-05-20T01:14:07.713 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-ceph-common-2:18.0.0-3996.g5ca1fdcf.el8. 26/134 2023-05-20T01:14:07.859 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-ceph-argparse-2:18.0.0-3996.g5ca1fdcf.el 27/134 2023-05-20T01:14:08.077 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-cephfs-2:18.0.0-3996.g5ca1fdcf.el8.x86_6 28/134 2023-05-20T01:14:08.256 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-05-20T01:14:10.010 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-05-20T01:14:10.110 INFO:teuthology.orchestra.run.smithi016.stdout: Preparing : 1/1 2023-05-20T01:14:10.280 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-05-20T01:14:10.460 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-05-20T01:14:10.674 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 3/134 2023-05-20T01:14:10.832 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 4/134 2023-05-20T01:14:10.947 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 4/134 2023-05-20T01:14:11.008 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 30/134 2023-05-20T01:14:11.174 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-05-20T01:14:11.302 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 31/134 2023-05-20T01:14:11.582 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 32/134 2023-05-20T01:14:11.598 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-05-20T01:14:11.800 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/134 2023-05-20T01:14:11.856 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : librados2-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 6/134 2023-05-20T01:14:12.025 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-rsa-4.8-1.el8.noarch 34/134 2023-05-20T01:14:12.216 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-05-20T01:14:12.262 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librados2-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 6/134 2023-05-20T01:14:12.378 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-rbd-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 36/134 2023-05-20T01:14:12.498 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : librbd1-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 7/134 2023-05-20T01:14:12.512 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : librados-devel-2:18.0.0-3996.g5ca1fdcf.el8.x86_6 37/134 2023-05-20T01:14:12.633 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librbd1-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 7/134 2023-05-20T01:14:12.679 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 38/134 2023-05-20T01:14:12.818 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 39/134 2023-05-20T01:14:12.823 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libcephfs2-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 8/134 2023-05-20T01:14:12.920 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libcephfs2-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 8/134 2023-05-20T01:14:12.972 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : uuid-1.6.2-43.el8.x86_64 40/134 2023-05-20T01:14:13.043 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 40/134 2023-05-20T01:14:13.225 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-rados-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 9/134 2023-05-20T01:14:13.252 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 41/134 2023-05-20T01:14:13.450 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : automake-1.16.1-7.el8.noarch 42/134 2023-05-20T01:14:13.463 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-05-20T01:14:13.597 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 43/134 2023-05-20T01:14:13.669 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-05-20T01:14:13.680 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 43/134 2023-05-20T01:14:13.893 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : jq-1.6-3.el8.x86_64 44/134 2023-05-20T01:14:14.067 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 45/134 2023-05-20T01:14:14.123 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-05-20T01:14:14.260 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 46/134 2023-05-20T01:14:14.423 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-05-20T01:14:14.449 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 47/134 2023-05-20T01:14:14.589 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libcephsqlite-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 14/134 2023-05-20T01:14:14.616 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : fuse-2.9.7-15.el8.x86_64 48/134 2023-05-20T01:14:14.702 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libcephsqlite-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 14/134 2023-05-20T01:14:14.759 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 49/134 2023-05-20T01:14:14.873 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libradosstriper1-2:18.0.0-3996.g5ca1fdcf.el8.x86 15/134 2023-05-20T01:14:14.946 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 50/134 2023-05-20T01:14:14.959 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libradosstriper1-2:18.0.0-3996.g5ca1fdcf.el8.x86 15/134 2023-05-20T01:14:15.123 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-05-20T01:14:15.128 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 51/134 2023-05-20T01:14:15.278 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-05-20T01:14:15.414 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-05-20T01:14:15.460 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 52/134 2023-05-20T01:14:15.706 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 53/134 2023-05-20T01:14:15.958 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 54/134 2023-05-20T01:14:16.060 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-05-20T01:14:16.184 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 55/134 2023-05-20T01:14:16.214 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-05-20T01:14:16.585 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 56/134 2023-05-20T01:14:16.876 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-05-20T01:14:17.013 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 57/134 2023-05-20T01:14:17.089 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : openblas-0.3.15-3.el8.x86_64 19/134 2023-05-20T01:14:17.204 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 58/134 2023-05-20T01:14:17.383 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 59/134 2023-05-20T01:14:17.601 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 60/134 2023-05-20T01:14:17.769 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 61/134 2023-05-20T01:14:17.842 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 19/134 2023-05-20T01:14:17.924 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 62/134 2023-05-20T01:14:18.080 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 20/134 2023-05-20T01:14:18.212 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 63/134 2023-05-20T01:14:18.349 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 21/134 2023-05-20T01:14:18.458 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 64/134 2023-05-20T01:14:18.544 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 22/134 2023-05-20T01:14:18.701 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 65/134 2023-05-20T01:14:18.721 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/134 2023-05-20T01:14:18.906 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 66/134 2023-05-20T01:14:18.960 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 24/134 2023-05-20T01:14:19.088 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-05-20T01:14:19.188 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-05-20T01:14:19.249 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-05-20T01:14:19.346 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-ceph-common-2:18.0.0-3996.g5ca1fdcf.el8. 26/134 2023-05-20T01:14:19.485 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-05-20T01:14:19.510 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-ceph-argparse-2:18.0.0-3996.g5ca1fdcf.el 27/134 2023-05-20T01:14:19.711 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-waitress-1.2.1-2.el8.1.noarch 70/134 2023-05-20T01:14:19.850 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-05-20T01:14:19.990 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-05-20T01:14:20.177 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-05-20T01:14:20.356 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-05-20T01:14:20.447 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cephfs-2:18.0.0-3996.g5ca1fdcf.el8.x86_6 28/134 2023-05-20T01:14:20.530 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-05-20T01:14:20.690 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-05-20T01:14:20.761 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 29/134 2023-05-20T01:14:20.834 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-05-20T01:14:21.003 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 30/134 2023-05-20T01:14:21.045 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-05-20T01:14:21.248 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-05-20T01:14:21.256 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 31/134 2023-05-20T01:14:21.434 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-05-20T01:14:21.480 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-05-20T01:14:21.687 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-05-20T01:14:21.928 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-05-20T01:14:22.085 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-05-20T01:14:22.308 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-05-20T01:14:23.349 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-05-20T01:14:23.384 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-05-20T01:14:23.625 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-05-20T01:14:23.668 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 33/134 2023-05-20T01:14:23.870 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-rsa-4.8-1.el8.noarch 34/134 2023-05-20T01:14:23.886 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-05-20T01:14:24.084 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-05-20T01:14:24.169 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-05-20T01:14:24.266 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-rbd-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 36/134 2023-05-20T01:14:24.435 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : librados-devel-2:18.0.0-3996.g5ca1fdcf.el8.x86_6 37/134 2023-05-20T01:14:24.437 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-05-20T01:14:24.606 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 38/134 2023-05-20T01:14:24.631 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-05-20T01:14:24.820 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 39/134 2023-05-20T01:14:25.016 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : fuse-2.9.7-15.el8.x86_64 40/134 2023-05-20T01:14:25.169 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 41/134 2023-05-20T01:14:25.381 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-05-20T01:14:25.433 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 42/134 2023-05-20T01:14:25.529 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : librgw2-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 92/134 2023-05-20T01:14:25.588 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: librgw2-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 92/134 2023-05-20T01:14:25.632 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 43/134 2023-05-20T01:14:25.729 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-rgw-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 93/134 2023-05-20T01:14:26.018 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 44/134 2023-05-20T01:14:26.293 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 45/134 2023-05-20T01:14:26.475 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 46/134 2023-05-20T01:14:26.716 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 47/134 2023-05-20T01:14:26.921 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 48/134 2023-05-20T01:14:27.249 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 49/134 2023-05-20T01:14:27.497 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 50/134 2023-05-20T01:14:27.700 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 51/134 2023-05-20T01:14:27.952 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 52/134 2023-05-20T01:14:28.137 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 53/134 2023-05-20T01:14:28.301 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 54/134 2023-05-20T01:14:28.644 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/134 2023-05-20T01:14:28.901 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/134 2023-05-20T01:14:29.146 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/134 2023-05-20T01:14:29.325 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-common-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 94/134 2023-05-20T01:14:29.378 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 58/134 2023-05-20T01:14:29.520 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-common-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 94/134 2023-05-20T01:14:29.537 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 59/134 2023-05-20T01:14:29.690 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 60/134 2023-05-20T01:14:29.912 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : uuid-1.6.2-43.el8.x86_64 61/134 2023-05-20T01:14:29.989 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 61/134 2023-05-20T01:14:30.116 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-common-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 94/134 2023-05-20T01:14:30.230 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 62/134 2023-05-20T01:14:30.388 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-base-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 95/134 2023-05-20T01:14:30.457 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-base-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 95/134 2023-05-20T01:14:30.459 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : automake-1.16.1-7.el8.noarch 63/134 2023-05-20T01:14:30.603 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-selinux-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 96/134 2023-05-20T01:14:30.613 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 64/134 2023-05-20T01:14:30.693 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 64/134 2023-05-20T01:14:30.914 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : jq-1.6-3.el8.x86_64 65/134 2023-05-20T01:14:31.115 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 66/134 2023-05-20T01:14:31.281 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-05-20T01:14:31.487 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-05-20T01:14:31.796 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-05-20T01:14:32.003 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-waitress-1.2.1-2.el8.1.noarch 70/134 2023-05-20T01:14:32.168 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-05-20T01:14:32.359 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-05-20T01:14:32.567 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-05-20T01:14:32.778 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-05-20T01:14:33.010 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-05-20T01:14:33.174 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-05-20T01:14:33.336 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-05-20T01:14:33.564 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-05-20T01:14:33.809 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-05-20T01:14:34.011 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-05-20T01:14:34.274 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-05-20T01:14:34.528 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-05-20T01:14:34.704 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-05-20T01:14:34.961 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-05-20T01:14:36.082 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-05-20T01:14:36.654 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-05-20T01:14:36.916 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-05-20T01:14:37.223 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-05-20T01:14:37.502 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-05-20T01:14:37.695 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-05-20T01:14:38.524 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-05-20T01:14:38.736 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : librgw2-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 92/134 2023-05-20T01:14:38.810 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librgw2-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 92/134 2023-05-20T01:14:39.012 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-rgw-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 93/134 2023-05-20T01:14:42.797 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-common-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 94/134 2023-05-20T01:14:42.995 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-common-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 94/134 2023-05-20T01:14:43.611 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-common-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 94/134 2023-05-20T01:14:43.929 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-base-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 95/134 2023-05-20T01:14:43.995 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-base-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 95/134 2023-05-20T01:14:44.144 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-selinux-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 96/134 2023-05-20T01:14:49.413 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-selinux-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 96/134 2023-05-20T01:14:49.414 INFO:teuthology.orchestra.run.smithi132.stdout:skipping the directory /sys 2023-05-20T01:14:49.414 INFO:teuthology.orchestra.run.smithi132.stdout:skipping the directory /proc 2023-05-20T01:14:49.415 INFO:teuthology.orchestra.run.smithi132.stdout:skipping the directory /dev 2023-05-20T01:14:49.415 INFO:teuthology.orchestra.run.smithi132.stdout:skipping the directory /run 2023-05-20T01:14:49.415 INFO:teuthology.orchestra.run.smithi132.stdout:skipping the directory /mnt 2023-05-20T01:14:49.416 INFO:teuthology.orchestra.run.smithi132.stdout:skipping the directory /var/tmp 2023-05-20T01:14:49.416 INFO:teuthology.orchestra.run.smithi132.stdout:skipping the directory /home 2023-05-20T01:14:49.416 INFO:teuthology.orchestra.run.smithi132.stdout:skipping the directory /tmp 2023-05-20T01:14:49.416 INFO:teuthology.orchestra.run.smithi132.stdout:skipping the directory /dev 2023-05-20T01:14:49.416 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:14:49.751 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-volume-2:18.0.0-3996.g5ca1fdcf.el8.noarch 97/134 2023-05-20T01:14:51.319 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-volume-2:18.0.0-3996.g5ca1fdcf.el8.noarch 97/134 2023-05-20T01:14:51.518 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-osd-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 98/134 2023-05-20T01:14:51.736 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-osd-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 98/134 2023-05-20T01:14:51.908 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-mds-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 99/134 2023-05-20T01:14:52.369 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-mds-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 99/134 2023-05-20T01:14:52.533 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-mon-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 100/134 2023-05-20T01:14:52.550 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-mon-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 100/134 2023-05-20T01:14:54.165 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: cephadm-2:18.0.0-3996.g5ca1fdcf.el8.noarch 101/134 2023-05-20T01:14:54.362 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : cephadm-2:18.0.0-3996.g5ca1fdcf.el8.noarch 101/134 2023-05-20T01:14:54.480 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-3996.g5ca1fdcf.e 102/134 2023-05-20T01:14:54.663 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-3996.g5ca1fdcf. 103/134 2023-05-20T01:14:54.840 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-3996.g5ca1fdcf.el8.noa 104/134 2023-05-20T01:14:55.055 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-3996.g5ca1fdcf.el8.noa 104/134 2023-05-20T01:14:55.326 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-3996.g5ca1fdcf.el8.n 105/134 2023-05-20T01:14:56.288 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-3996.g5ca1fdcf.el8.n 105/134 2023-05-20T01:14:56.465 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-3996.g5ca 106/134 2023-05-20T01:14:56.557 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-3996.g5ca 106/134 2023-05-20T01:14:56.928 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-3996.g5ca1fdcf.el 107/134 2023-05-20T01:14:57.118 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-mgr-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 108/134 2023-05-20T01:14:57.153 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-mgr-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 108/134 2023-05-20T01:14:57.315 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-mgr-rook-2:18.0.0-3996.g5ca1fdcf.el8.noarch 109/134 2023-05-20T01:14:57.327 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-3996.g5ca1fdcf.el8.noarch 109/134 2023-05-20T01:14:58.562 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 110/134 2023-05-20T01:14:58.798 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-radosgw-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 111/134 2023-05-20T01:14:58.831 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 111/134 2023-05-20T01:14:58.982 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-3996.g5ca1f 112/134 2023-05-20T01:14:59.025 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-3996.g5ca1f 112/134 2023-05-20T01:14:59.173 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : cephfs-mirror-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 113/134 2023-05-20T01:14:59.520 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 113/134 2023-05-20T01:14:59.682 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : rbd-mirror-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 114/134 2023-05-20T01:15:03.607 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: rbd-mirror-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 114/134 2023-05-20T01:15:03.839 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-selinux-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 96/134 2023-05-20T01:15:03.840 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /sys 2023-05-20T01:15:03.840 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /proc 2023-05-20T01:15:03.840 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /dev 2023-05-20T01:15:03.840 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /run 2023-05-20T01:15:03.840 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /mnt 2023-05-20T01:15:03.841 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /var/tmp 2023-05-20T01:15:03.841 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /home 2023-05-20T01:15:03.841 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /tmp 2023-05-20T01:15:03.841 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /dev 2023-05-20T01:15:03.841 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:15:04.175 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-volume-2:18.0.0-3996.g5ca1fdcf.el8.noarch 97/134 2023-05-20T01:15:04.495 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-test-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 115/134 2023-05-20T01:15:04.812 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 116/134 2023-05-20T01:15:05.004 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-fuse-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 117/134 2023-05-20T01:15:05.250 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 118/134 2023-05-20T01:15:05.414 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libtool-2.4.6-25.el8.x86_64 119/134 2023-05-20T01:15:05.609 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 119/134 2023-05-20T01:15:05.852 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-volume-2:18.0.0-3996.g5ca1fdcf.el8.noarch 97/134 2023-05-20T01:15:05.860 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 120/134 2023-05-20T01:15:06.027 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libcephfs-devel-2:18.0.0-3996.g5ca1fdcf.el8.x86_ 121/134 2023-05-20T01:15:06.076 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-osd-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 98/134 2023-05-20T01:15:06.157 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-05-20T01:15:06.306 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-osd-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 98/134 2023-05-20T01:15:06.450 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mds-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 99/134 2023-05-20T01:15:06.937 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mds-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 99/134 2023-05-20T01:15:07.116 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mon-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 100/134 2023-05-20T01:15:07.133 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mon-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 100/134 2023-05-20T01:15:08.225 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-05-20T01:15:08.472 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-05-20T01:15:08.491 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-05-20T01:15:08.664 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : cephfs-top-2:18.0.0-3996.g5ca1fdcf.el8.noarch 124/134 2023-05-20T01:15:08.955 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : rbd-fuse-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 125/134 2023-05-20T01:15:09.168 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: cephadm-2:18.0.0-3996.g5ca1fdcf.el8.noarch 101/134 2023-05-20T01:15:09.235 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : rbd-nbd-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 126/134 2023-05-20T01:15:09.362 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : cephadm-2:18.0.0-3996.g5ca1fdcf.el8.noarch 101/134 2023-05-20T01:15:09.473 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-3996.g5ca1fdcf.e 102/134 2023-05-20T01:15:09.569 INFO:teuthology.orchestra.run.smithi132.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-05-20T01:15:09.674 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-3996.g5ca1fdcf. 103/134 2023-05-20T01:15:09.755 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : flex-2.6.1-9.el8.x86_64 128/134 2023-05-20T01:15:09.833 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 128/134 2023-05-20T01:15:09.890 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-3996.g5ca1fdcf.el8.noa 104/134 2023-05-20T01:15:10.022 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : bison-3.0.4-10.el8.x86_64 129/134 2023-05-20T01:15:10.035 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 129/134 2023-05-20T01:15:10.110 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-3996.g5ca1fdcf.el8.noa 104/134 2023-05-20T01:15:10.172 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-05-20T01:15:10.180 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-05-20T01:15:10.315 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/134 2023-05-20T01:15:10.316 INFO:teuthology.orchestra.run.smithi132.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-05-20T01:15:10.358 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-3996.g5ca1fdcf.el8.n 105/134 2023-05-20T01:15:10.533 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-05-20T01:15:10.534 INFO:teuthology.orchestra.run.smithi132.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-05-20T01:15:10.735 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-05-20T01:15:10.895 INFO:teuthology.orchestra.run.smithi132.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-05-20T01:15:11.156 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-05-20T01:15:11.156 INFO:teuthology.orchestra.run.smithi132.stdout:[/usr/lib/tmpfiles.d/journal-nocow.conf:26] Failed to resolve specifier: uninitialized /etc detected, skipping 2023-05-20T01:15:11.156 INFO:teuthology.orchestra.run.smithi132.stdout:All rules containing unresolvable specifiers will be skipped. 2023-05-20T01:15:11.156 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:15:11.370 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 1/134 2023-05-20T01:15:11.370 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-base-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 2/134 2023-05-20T01:15:11.371 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-common-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 3/134 2023-05-20T01:15:11.371 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-fuse-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 4/134 2023-05-20T01:15:11.371 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-3996.g5ca1f 5/134 2023-05-20T01:15:11.371 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-mds-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 6/134 2023-05-20T01:15:11.372 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-mgr-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 7/134 2023-05-20T01:15:11.372 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-mon-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 8/134 2023-05-20T01:15:11.372 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-osd-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 9/134 2023-05-20T01:15:11.373 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-radosgw-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 10/134 2023-05-20T01:15:11.373 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-selinux-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 11/134 2023-05-20T01:15:11.373 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-test-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 12/134 2023-05-20T01:15:11.374 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : cephfs-mirror-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 13/134 2023-05-20T01:15:11.374 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libcephfs-devel-2:18.0.0-3996.g5ca1fdcf.el8.x86_ 14/134 2023-05-20T01:15:11.374 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libcephfs2-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 15/134 2023-05-20T01:15:11.374 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libcephsqlite-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 16/134 2023-05-20T01:15:11.375 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : librados-devel-2:18.0.0-3996.g5ca1fdcf.el8.x86_6 17/134 2023-05-20T01:15:11.375 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libradosstriper1-2:18.0.0-3996.g5ca1fdcf.el8.x86 18/134 2023-05-20T01:15:11.375 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : librgw2-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 19/134 2023-05-20T01:15:11.376 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-ceph-argparse-2:18.0.0-3996.g5ca1fdcf.el 20/134 2023-05-20T01:15:11.376 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-ceph-common-2:18.0.0-3996.g5ca1fdcf.el8. 21/134 2023-05-20T01:15:11.376 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-cephfs-2:18.0.0-3996.g5ca1fdcf.el8.x86_6 22/134 2023-05-20T01:15:11.377 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-rados-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 23/134 2023-05-20T01:15:11.377 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-rbd-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 24/134 2023-05-20T01:15:11.377 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-rgw-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 25/134 2023-05-20T01:15:11.377 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : rbd-fuse-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 26/134 2023-05-20T01:15:11.385 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-3996.g5ca1fdcf.el8.n 105/134 2023-05-20T01:15:11.387 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : rbd-mirror-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 27/134 2023-05-20T01:15:11.387 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : rbd-nbd-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 28/134 2023-05-20T01:15:11.387 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-3996.g5ca1fdcf. 29/134 2023-05-20T01:15:11.387 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-3996.g5ca1fdcf.el8.noa 30/134 2023-05-20T01:15:11.387 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-3996.g5ca1fdcf.el8.n 31/134 2023-05-20T01:15:11.388 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-3996.g5ca 32/134 2023-05-20T01:15:11.388 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-3996.g5ca1fdcf.el 33/134 2023-05-20T01:15:11.388 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-mgr-rook-2:18.0.0-3996.g5ca1fdcf.el8.noarch 34/134 2023-05-20T01:15:11.388 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-3996.g5ca1fdcf.e 35/134 2023-05-20T01:15:11.388 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-volume-2:18.0.0-3996.g5ca1fdcf.el8.noarch 36/134 2023-05-20T01:15:11.388 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : cephadm-2:18.0.0-3996.g5ca1fdcf.el8.noarch 37/134 2023-05-20T01:15:11.388 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : cephfs-top-2:18.0.0-3996.g5ca1fdcf.el8.noarch 38/134 2023-05-20T01:15:11.389 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-05-20T01:15:11.389 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-05-20T01:15:11.389 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-05-20T01:15:11.389 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-05-20T01:15:11.389 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-05-20T01:15:11.389 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-05-20T01:15:11.389 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-05-20T01:15:11.390 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-05-20T01:15:11.390 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-05-20T01:15:11.390 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-05-20T01:15:11.390 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-05-20T01:15:11.390 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-05-20T01:15:11.390 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-05-20T01:15:11.390 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-05-20T01:15:11.390 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-05-20T01:15:11.391 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-05-20T01:15:11.391 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-05-20T01:15:11.391 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-05-20T01:15:11.392 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-05-20T01:15:11.392 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-05-20T01:15:11.392 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-05-20T01:15:11.392 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 60/134 2023-05-20T01:15:11.392 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-05-20T01:15:11.392 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-05-20T01:15:11.392 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-05-20T01:15:11.393 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-05-20T01:15:11.393 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-waitress-1.2.1-2.el8.1.noarch 65/134 2023-05-20T01:15:11.393 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-05-20T01:15:11.393 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-05-20T01:15:11.393 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-05-20T01:15:11.393 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-05-20T01:15:11.393 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-05-20T01:15:11.394 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-05-20T01:15:11.394 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 72/134 2023-05-20T01:15:11.394 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 73/134 2023-05-20T01:15:11.394 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/134 2023-05-20T01:15:11.394 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/134 2023-05-20T01:15:11.394 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 76/134 2023-05-20T01:15:11.394 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 77/134 2023-05-20T01:15:11.394 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/134 2023-05-20T01:15:11.395 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 79/134 2023-05-20T01:15:11.395 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/134 2023-05-20T01:15:11.395 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 81/134 2023-05-20T01:15:11.395 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 82/134 2023-05-20T01:15:11.395 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 83/134 2023-05-20T01:15:11.395 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 84/134 2023-05-20T01:15:11.395 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 85/134 2023-05-20T01:15:11.395 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 86/134 2023-05-20T01:15:11.396 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 87/134 2023-05-20T01:15:11.396 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 88/134 2023-05-20T01:15:11.396 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 89/134 2023-05-20T01:15:11.396 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 90/134 2023-05-20T01:15:11.396 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 91/134 2023-05-20T01:15:11.396 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 92/134 2023-05-20T01:15:11.396 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 93/134 2023-05-20T01:15:11.397 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 94/134 2023-05-20T01:15:11.397 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 95/134 2023-05-20T01:15:11.397 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 96/134 2023-05-20T01:15:11.397 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 97/134 2023-05-20T01:15:11.398 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 98/134 2023-05-20T01:15:11.398 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 99/134 2023-05-20T01:15:11.398 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 100/134 2023-05-20T01:15:11.398 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 101/134 2023-05-20T01:15:11.399 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 102/134 2023-05-20T01:15:11.399 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 103/134 2023-05-20T01:15:11.399 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : bison-3.0.4-10.el8.x86_64 104/134 2023-05-20T01:15:11.399 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 105/134 2023-05-20T01:15:11.400 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 106/134 2023-05-20T01:15:11.400 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/134 2023-05-20T01:15:11.400 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : automake-1.16.1-7.el8.noarch 108/134 2023-05-20T01:15:11.401 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 109/134 2023-05-20T01:15:11.401 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 110/134 2023-05-20T01:15:11.401 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : flex-2.6.1-9.el8.x86_64 111/134 2023-05-20T01:15:11.401 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 112/134 2023-05-20T01:15:11.402 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 113/134 2023-05-20T01:15:11.402 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 114/134 2023-05-20T01:15:11.402 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 115/134 2023-05-20T01:15:11.402 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 116/134 2023-05-20T01:15:11.403 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 117/134 2023-05-20T01:15:11.403 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 118/134 2023-05-20T01:15:11.403 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 119/134 2023-05-20T01:15:11.403 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 120/134 2023-05-20T01:15:11.404 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 121/134 2023-05-20T01:15:11.404 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libpq-13.5-1.el8.x86_64 122/134 2023-05-20T01:15:11.404 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 123/134 2023-05-20T01:15:11.404 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : jq-1.6-3.el8.x86_64 124/134 2023-05-20T01:15:11.404 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 125/134 2023-05-20T01:15:11.404 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 126/134 2023-05-20T01:15:11.405 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 127/134 2023-05-20T01:15:11.405 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 128/134 2023-05-20T01:15:11.405 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : librados2-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 129/134 2023-05-20T01:15:11.405 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-05-20T01:15:11.405 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : librbd1-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 131/134 2023-05-20T01:15:11.405 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-05-20T01:15:11.406 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-05-20T01:15:11.540 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-3996.g5ca 106/134 2023-05-20T01:15:11.635 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-3996.g5ca 106/134 2023-05-20T01:15:12.003 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-3996.g5ca1fdcf.el 107/134 2023-05-20T01:15:12.184 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 108/134 2023-05-20T01:15:12.220 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 108/134 2023-05-20T01:15:12.415 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-rook-2:18.0.0-3996.g5ca1fdcf.el8.noarch 109/134 2023-05-20T01:15:12.427 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-3996.g5ca1fdcf.el8.noarch 109/134 2023-05-20T01:15:13.739 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 110/134 2023-05-20T01:15:13.949 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-radosgw-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 111/134 2023-05-20T01:15:13.983 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 111/134 2023-05-20T01:15:14.076 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-05-20T01:15:14.076 INFO:teuthology.orchestra.run.smithi132.stdout:Last metadata expiration check: 0:01:48 ago on Sat 20 May 2023 01:13:26 AM UTC. 2023-05-20T01:15:14.124 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-3996.g5ca1f 112/134 2023-05-20T01:15:14.166 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-3996.g5ca1f 112/134 2023-05-20T01:15:14.324 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : cephfs-mirror-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 113/134 2023-05-20T01:15:14.690 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 113/134 2023-05-20T01:15:14.890 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : rbd-mirror-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 114/134 2023-05-20T01:15:16.666 INFO:teuthology.orchestra.run.smithi132.stdout:Installed products updated. 2023-05-20T01:15:16.889 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:15:16.889 INFO:teuthology.orchestra.run.smithi132.stdout:Upgraded: 2023-05-20T01:15:16.890 INFO:teuthology.orchestra.run.smithi132.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-05-20T01:15:16.890 INFO:teuthology.orchestra.run.smithi132.stdout: librados2-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 2023-05-20T01:15:16.890 INFO:teuthology.orchestra.run.smithi132.stdout: librbd1-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 2023-05-20T01:15:16.890 INFO:teuthology.orchestra.run.smithi132.stdout:Installed: 2023-05-20T01:15:16.890 INFO:teuthology.orchestra.run.smithi132.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-05-20T01:15:16.890 INFO:teuthology.orchestra.run.smithi132.stdout: automake-1.16.1-7.el8.noarch 2023-05-20T01:15:16.890 INFO:teuthology.orchestra.run.smithi132.stdout: bison-3.0.4-10.el8.x86_64 2023-05-20T01:15:16.891 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 2023-05-20T01:15:16.891 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-base-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 2023-05-20T01:15:16.891 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-common-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 2023-05-20T01:15:16.891 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-fuse-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 2023-05-20T01:15:16.891 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-grafana-dashboards-2:18.0.0-3996.g5ca1fdcf.el8.noarch 2023-05-20T01:15:16.891 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-immutable-object-cache-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 2023-05-20T01:15:16.891 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mds-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 2023-05-20T01:15:16.891 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 2023-05-20T01:15:16.892 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-cephadm-2:18.0.0-3996.g5ca1fdcf.el8.noarch 2023-05-20T01:15:16.892 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-dashboard-2:18.0.0-3996.g5ca1fdcf.el8.noarch 2023-05-20T01:15:16.892 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-diskprediction-local-2:18.0.0-3996.g5ca1fdcf.el8.noarch 2023-05-20T01:15:16.892 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-modules-core-2:18.0.0-3996.g5ca1fdcf.el8.noarch 2023-05-20T01:15:16.892 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-rook-2:18.0.0-3996.g5ca1fdcf.el8.noarch 2023-05-20T01:15:16.892 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mon-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 2023-05-20T01:15:16.892 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-osd-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 2023-05-20T01:15:16.893 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-prometheus-alerts-2:18.0.0-3996.g5ca1fdcf.el8.noarch 2023-05-20T01:15:16.893 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-radosgw-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 2023-05-20T01:15:16.893 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-selinux-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 2023-05-20T01:15:16.893 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-test-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 2023-05-20T01:15:16.893 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-volume-2:18.0.0-3996.g5ca1fdcf.el8.noarch 2023-05-20T01:15:16.893 INFO:teuthology.orchestra.run.smithi132.stdout: cephadm-2:18.0.0-3996.g5ca1fdcf.el8.noarch 2023-05-20T01:15:16.893 INFO:teuthology.orchestra.run.smithi132.stdout: cephfs-mirror-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 2023-05-20T01:15:16.894 INFO:teuthology.orchestra.run.smithi132.stdout: cephfs-top-2:18.0.0-3996.g5ca1fdcf.el8.noarch 2023-05-20T01:15:16.894 INFO:teuthology.orchestra.run.smithi132.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-05-20T01:15:16.894 INFO:teuthology.orchestra.run.smithi132.stdout: flex-2.6.1-9.el8.x86_64 2023-05-20T01:15:16.894 INFO:teuthology.orchestra.run.smithi132.stdout: fuse-2.9.7-15.el8.x86_64 2023-05-20T01:15:16.894 INFO:teuthology.orchestra.run.smithi132.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-05-20T01:15:16.894 INFO:teuthology.orchestra.run.smithi132.stdout: jq-1.6-3.el8.x86_64 2023-05-20T01:15:16.894 INFO:teuthology.orchestra.run.smithi132.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-05-20T01:15:16.894 INFO:teuthology.orchestra.run.smithi132.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-05-20T01:15:16.895 INFO:teuthology.orchestra.run.smithi132.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-05-20T01:15:16.895 INFO:teuthology.orchestra.run.smithi132.stdout: libcephfs-devel-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 2023-05-20T01:15:16.895 INFO:teuthology.orchestra.run.smithi132.stdout: libcephfs2-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 2023-05-20T01:15:16.895 INFO:teuthology.orchestra.run.smithi132.stdout: libcephsqlite-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 2023-05-20T01:15:16.895 INFO:teuthology.orchestra.run.smithi132.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-05-20T01:15:16.895 INFO:teuthology.orchestra.run.smithi132.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-05-20T01:15:16.895 INFO:teuthology.orchestra.run.smithi132.stdout: liboath-2.6.2-3.el8.x86_64 2023-05-20T01:15:16.895 INFO:teuthology.orchestra.run.smithi132.stdout: libpq-13.5-1.el8.x86_64 2023-05-20T01:15:16.896 INFO:teuthology.orchestra.run.smithi132.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-05-20T01:15:16.896 INFO:teuthology.orchestra.run.smithi132.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-05-20T01:15:16.896 INFO:teuthology.orchestra.run.smithi132.stdout: librados-devel-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 2023-05-20T01:15:16.896 INFO:teuthology.orchestra.run.smithi132.stdout: libradosstriper1-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 2023-05-20T01:15:16.896 INFO:teuthology.orchestra.run.smithi132.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-05-20T01:15:16.896 INFO:teuthology.orchestra.run.smithi132.stdout: librgw2-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 2023-05-20T01:15:16.896 INFO:teuthology.orchestra.run.smithi132.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-05-20T01:15:16.897 INFO:teuthology.orchestra.run.smithi132.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-05-20T01:15:16.897 INFO:teuthology.orchestra.run.smithi132.stdout: libtool-2.4.6-25.el8.x86_64 2023-05-20T01:15:16.897 INFO:teuthology.orchestra.run.smithi132.stdout: libunwind-1.3.1-3.el8.x86_64 2023-05-20T01:15:16.897 INFO:teuthology.orchestra.run.smithi132.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-05-20T01:15:16.897 INFO:teuthology.orchestra.run.smithi132.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-05-20T01:15:16.897 INFO:teuthology.orchestra.run.smithi132.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-05-20T01:15:16.897 INFO:teuthology.orchestra.run.smithi132.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-05-20T01:15:16.898 INFO:teuthology.orchestra.run.smithi132.stdout: openblas-0.3.15-3.el8.x86_64 2023-05-20T01:15:16.898 INFO:teuthology.orchestra.run.smithi132.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-05-20T01:15:16.898 INFO:teuthology.orchestra.run.smithi132.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-05-20T01:15:16.898 INFO:teuthology.orchestra.run.smithi132.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-05-20T01:15:16.898 INFO:teuthology.orchestra.run.smithi132.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-05-20T01:15:16.898 INFO:teuthology.orchestra.run.smithi132.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-05-20T01:15:16.898 INFO:teuthology.orchestra.run.smithi132.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-05-20T01:15:16.898 INFO:teuthology.orchestra.run.smithi132.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-05-20T01:15:16.899 INFO:teuthology.orchestra.run.smithi132.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-05-20T01:15:16.899 INFO:teuthology.orchestra.run.smithi132.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-05-20T01:15:16.899 INFO:teuthology.orchestra.run.smithi132.stdout: python3-babel-2.5.1-7.el8.noarch 2023-05-20T01:15:16.899 INFO:teuthology.orchestra.run.smithi132.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-05-20T01:15:16.899 INFO:teuthology.orchestra.run.smithi132.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-05-20T01:15:16.899 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-05-20T01:15:16.899 INFO:teuthology.orchestra.run.smithi132.stdout: python3-ceph-argparse-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 2023-05-20T01:15:16.899 INFO:teuthology.orchestra.run.smithi132.stdout: python3-ceph-common-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 2023-05-20T01:15:16.900 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cephfs-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 2023-05-20T01:15:16.900 INFO:teuthology.orchestra.run.smithi132.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-05-20T01:15:16.900 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-05-20T01:15:16.900 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-05-20T01:15:16.900 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-05-20T01:15:16.900 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-05-20T01:15:16.900 INFO:teuthology.orchestra.run.smithi132.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-05-20T01:15:16.901 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jaraco-6.2-6.el8.noarch 2023-05-20T01:15:16.901 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-05-20T01:15:16.901 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-05-20T01:15:16.901 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-05-20T01:15:16.901 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-05-20T01:15:16.901 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-05-20T01:15:16.901 INFO:teuthology.orchestra.run.smithi132.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-05-20T01:15:16.902 INFO:teuthology.orchestra.run.smithi132.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-05-20T01:15:16.902 INFO:teuthology.orchestra.run.smithi132.stdout: python3-mako-1.0.6-13.el8.noarch 2023-05-20T01:15:16.902 INFO:teuthology.orchestra.run.smithi132.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-05-20T01:15:16.902 INFO:teuthology.orchestra.run.smithi132.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-05-20T01:15:16.902 INFO:teuthology.orchestra.run.smithi132.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-05-20T01:15:16.902 INFO:teuthology.orchestra.run.smithi132.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-05-20T01:15:16.902 INFO:teuthology.orchestra.run.smithi132.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-05-20T01:15:16.902 INFO:teuthology.orchestra.run.smithi132.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-05-20T01:15:16.903 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-05-20T01:15:16.903 INFO:teuthology.orchestra.run.smithi132.stdout: python3-portend-2.6-1.el8.noarch 2023-05-20T01:15:16.903 INFO:teuthology.orchestra.run.smithi132.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-05-20T01:15:16.903 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-05-20T01:15:16.903 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-05-20T01:15:16.904 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-05-20T01:15:16.904 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pycparser-2.14-14.el8.noarch 2023-05-20T01:15:16.904 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pytz-2017.2-9.el8.noarch 2023-05-20T01:15:16.904 INFO:teuthology.orchestra.run.smithi132.stdout: python3-rados-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 2023-05-20T01:15:16.905 INFO:teuthology.orchestra.run.smithi132.stdout: python3-rbd-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 2023-05-20T01:15:16.905 INFO:teuthology.orchestra.run.smithi132.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-05-20T01:15:16.905 INFO:teuthology.orchestra.run.smithi132.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-05-20T01:15:16.905 INFO:teuthology.orchestra.run.smithi132.stdout: python3-rgw-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 2023-05-20T01:15:16.905 INFO:teuthology.orchestra.run.smithi132.stdout: python3-routes-2.4.1-12.el8.noarch 2023-05-20T01:15:16.906 INFO:teuthology.orchestra.run.smithi132.stdout: python3-rsa-4.8-1.el8.noarch 2023-05-20T01:15:16.906 INFO:teuthology.orchestra.run.smithi132.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-05-20T01:15:16.906 INFO:teuthology.orchestra.run.smithi132.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-05-20T01:15:16.906 INFO:teuthology.orchestra.run.smithi132.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-05-20T01:15:16.907 INFO:teuthology.orchestra.run.smithi132.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-05-20T01:15:16.907 INFO:teuthology.orchestra.run.smithi132.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-05-20T01:15:16.907 INFO:teuthology.orchestra.run.smithi132.stdout: python3-waitress-1.2.1-2.el8.1.noarch 2023-05-20T01:15:16.907 INFO:teuthology.orchestra.run.smithi132.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-05-20T01:15:16.907 INFO:teuthology.orchestra.run.smithi132.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-05-20T01:15:16.908 INFO:teuthology.orchestra.run.smithi132.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-05-20T01:15:16.908 INFO:teuthology.orchestra.run.smithi132.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-05-20T01:15:16.908 INFO:teuthology.orchestra.run.smithi132.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-05-20T01:15:16.908 INFO:teuthology.orchestra.run.smithi132.stdout: rbd-fuse-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 2023-05-20T01:15:16.909 INFO:teuthology.orchestra.run.smithi132.stdout: rbd-mirror-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 2023-05-20T01:15:16.909 INFO:teuthology.orchestra.run.smithi132.stdout: rbd-nbd-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 2023-05-20T01:15:16.909 INFO:teuthology.orchestra.run.smithi132.stdout: socat-1.7.4.1-1.el8.x86_64 2023-05-20T01:15:16.909 INFO:teuthology.orchestra.run.smithi132.stdout: thrift-0.13.0-2.el8.x86_64 2023-05-20T01:15:16.910 INFO:teuthology.orchestra.run.smithi132.stdout: uuid-1.6.2-43.el8.x86_64 2023-05-20T01:15:16.910 INFO:teuthology.orchestra.run.smithi132.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-05-20T01:15:16.910 INFO:teuthology.orchestra.run.smithi132.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-05-20T01:15:16.911 INFO:teuthology.orchestra.run.smithi132.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-05-20T01:15:16.911 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:15:16.911 INFO:teuthology.orchestra.run.smithi132.stdout:Complete! 2023-05-20T01:15:17.083 DEBUG:teuthology.parallel:result is None 2023-05-20T01:15:19.042 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: rbd-mirror-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 114/134 2023-05-20T01:15:19.326 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-test-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 115/134 2023-05-20T01:15:19.498 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libtool-2.4.6-25.el8.x86_64 116/134 2023-05-20T01:15:19.702 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 116/134 2023-05-20T01:15:20.664 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 117/134 2023-05-20T01:15:21.109 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 118/134 2023-05-20T01:15:21.330 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-fuse-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 119/134 2023-05-20T01:15:21.510 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-05-20T01:15:21.669 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libcephfs-devel-2:18.0.0-3996.g5ca1fdcf.el8.x86_ 121/134 2023-05-20T01:15:21.815 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-05-20T01:15:24.113 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-05-20T01:15:24.372 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-05-20T01:15:24.390 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-05-20T01:15:24.650 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : cephfs-top-2:18.0.0-3996.g5ca1fdcf.el8.noarch 124/134 2023-05-20T01:15:24.840 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : rbd-fuse-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 125/134 2023-05-20T01:15:25.130 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : rbd-nbd-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 126/134 2023-05-20T01:15:25.424 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-05-20T01:15:25.572 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-05-20T01:15:25.581 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-05-20T01:15:25.801 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 129/134 2023-05-20T01:15:25.980 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : flex-2.6.1-9.el8.x86_64 130/134 2023-05-20T01:15:26.070 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 130/134 2023-05-20T01:15:26.272 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : bison-3.0.4-10.el8.x86_64 131/134 2023-05-20T01:15:26.286 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 131/134 2023-05-20T01:15:26.286 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-05-20T01:15:26.517 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-05-20T01:15:26.517 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-05-20T01:15:26.737 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-05-20T01:15:26.921 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-05-20T01:15:27.256 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-05-20T01:15:27.256 INFO:teuthology.orchestra.run.smithi016.stdout:[/usr/lib/tmpfiles.d/journal-nocow.conf:26] Failed to resolve specifier: uninitialized /etc detected, skipping 2023-05-20T01:15:27.256 INFO:teuthology.orchestra.run.smithi016.stdout:All rules containing unresolvable specifiers will be skipped. 2023-05-20T01:15:27.257 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:15:27.495 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 1/134 2023-05-20T01:15:27.495 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-base-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 2/134 2023-05-20T01:15:27.495 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-common-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 3/134 2023-05-20T01:15:27.495 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-fuse-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 4/134 2023-05-20T01:15:27.495 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-3996.g5ca1f 5/134 2023-05-20T01:15:27.495 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mds-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 6/134 2023-05-20T01:15:27.496 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 7/134 2023-05-20T01:15:27.496 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mon-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 8/134 2023-05-20T01:15:27.496 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-osd-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 9/134 2023-05-20T01:15:27.496 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-radosgw-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 10/134 2023-05-20T01:15:27.496 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-selinux-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 11/134 2023-05-20T01:15:27.496 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-test-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 12/134 2023-05-20T01:15:27.496 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : cephfs-mirror-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 13/134 2023-05-20T01:15:27.497 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libcephfs-devel-2:18.0.0-3996.g5ca1fdcf.el8.x86_ 14/134 2023-05-20T01:15:27.497 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libcephfs2-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 15/134 2023-05-20T01:15:27.497 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libcephsqlite-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 16/134 2023-05-20T01:15:27.497 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librados-devel-2:18.0.0-3996.g5ca1fdcf.el8.x86_6 17/134 2023-05-20T01:15:27.497 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libradosstriper1-2:18.0.0-3996.g5ca1fdcf.el8.x86 18/134 2023-05-20T01:15:27.497 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librgw2-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 19/134 2023-05-20T01:15:27.497 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-ceph-argparse-2:18.0.0-3996.g5ca1fdcf.el 20/134 2023-05-20T01:15:27.497 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-ceph-common-2:18.0.0-3996.g5ca1fdcf.el8. 21/134 2023-05-20T01:15:27.498 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cephfs-2:18.0.0-3996.g5ca1fdcf.el8.x86_6 22/134 2023-05-20T01:15:27.498 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rados-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 23/134 2023-05-20T01:15:27.498 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rbd-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 24/134 2023-05-20T01:15:27.498 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rgw-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 25/134 2023-05-20T01:15:27.498 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : rbd-fuse-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 26/134 2023-05-20T01:15:27.498 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : rbd-mirror-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 27/134 2023-05-20T01:15:27.498 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : rbd-nbd-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 28/134 2023-05-20T01:15:27.498 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-3996.g5ca1fdcf. 29/134 2023-05-20T01:15:27.499 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-3996.g5ca1fdcf.el8.noa 30/134 2023-05-20T01:15:27.499 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-3996.g5ca1fdcf.el8.n 31/134 2023-05-20T01:15:27.499 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-3996.g5ca 32/134 2023-05-20T01:15:27.499 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-3996.g5ca1fdcf.el 33/134 2023-05-20T01:15:27.499 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-rook-2:18.0.0-3996.g5ca1fdcf.el8.noarch 34/134 2023-05-20T01:15:27.499 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-3996.g5ca1fdcf.e 35/134 2023-05-20T01:15:27.499 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-volume-2:18.0.0-3996.g5ca1fdcf.el8.noarch 36/134 2023-05-20T01:15:27.500 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : cephadm-2:18.0.0-3996.g5ca1fdcf.el8.noarch 37/134 2023-05-20T01:15:27.500 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : cephfs-top-2:18.0.0-3996.g5ca1fdcf.el8.noarch 38/134 2023-05-20T01:15:27.500 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-05-20T01:15:27.500 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-05-20T01:15:27.500 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-05-20T01:15:27.500 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-05-20T01:15:27.500 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-05-20T01:15:27.500 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-05-20T01:15:27.501 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-05-20T01:15:27.501 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-05-20T01:15:27.501 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-05-20T01:15:27.501 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-05-20T01:15:27.501 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-05-20T01:15:27.501 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-05-20T01:15:27.501 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-05-20T01:15:27.502 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-05-20T01:15:27.502 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-05-20T01:15:27.502 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-05-20T01:15:27.502 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-05-20T01:15:27.502 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-05-20T01:15:27.502 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-05-20T01:15:27.502 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-05-20T01:15:27.502 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-05-20T01:15:27.503 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 60/134 2023-05-20T01:15:27.503 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-05-20T01:15:27.503 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-05-20T01:15:27.503 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-05-20T01:15:27.503 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-05-20T01:15:27.503 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-waitress-1.2.1-2.el8.1.noarch 65/134 2023-05-20T01:15:27.504 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-05-20T01:15:27.504 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-05-20T01:15:27.504 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-05-20T01:15:27.504 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-05-20T01:15:27.504 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-05-20T01:15:27.504 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-05-20T01:15:27.504 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 72/134 2023-05-20T01:15:27.505 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 73/134 2023-05-20T01:15:27.505 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 74/134 2023-05-20T01:15:27.505 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 75/134 2023-05-20T01:15:27.505 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 76/134 2023-05-20T01:15:27.505 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 77/134 2023-05-20T01:15:27.505 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/134 2023-05-20T01:15:27.505 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 79/134 2023-05-20T01:15:27.506 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 80/134 2023-05-20T01:15:27.506 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 81/134 2023-05-20T01:15:27.506 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : automake-1.16.1-7.el8.noarch 82/134 2023-05-20T01:15:27.506 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 83/134 2023-05-20T01:15:27.506 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 84/134 2023-05-20T01:15:27.506 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : flex-2.6.1-9.el8.x86_64 85/134 2023-05-20T01:15:27.506 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 86/134 2023-05-20T01:15:27.506 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/134 2023-05-20T01:15:27.507 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 88/134 2023-05-20T01:15:27.507 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/134 2023-05-20T01:15:27.507 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 90/134 2023-05-20T01:15:27.507 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 91/134 2023-05-20T01:15:27.507 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 92/134 2023-05-20T01:15:27.507 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 93/134 2023-05-20T01:15:27.507 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 94/134 2023-05-20T01:15:27.507 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 95/134 2023-05-20T01:15:27.508 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libpq-13.5-1.el8.x86_64 96/134 2023-05-20T01:15:27.508 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 97/134 2023-05-20T01:15:27.508 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : jq-1.6-3.el8.x86_64 98/134 2023-05-20T01:15:27.508 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 99/134 2023-05-20T01:15:27.508 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 100/134 2023-05-20T01:15:27.508 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 101/134 2023-05-20T01:15:27.508 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 102/134 2023-05-20T01:15:27.509 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 103/134 2023-05-20T01:15:27.509 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 104/134 2023-05-20T01:15:27.509 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/134 2023-05-20T01:15:27.509 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 106/134 2023-05-20T01:15:27.509 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 107/134 2023-05-20T01:15:27.509 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 108/134 2023-05-20T01:15:27.509 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 109/134 2023-05-20T01:15:27.509 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 110/134 2023-05-20T01:15:27.510 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 111/134 2023-05-20T01:15:27.510 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 112/134 2023-05-20T01:15:27.510 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 113/134 2023-05-20T01:15:27.510 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 114/134 2023-05-20T01:15:27.510 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 115/134 2023-05-20T01:15:27.510 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 116/134 2023-05-20T01:15:27.510 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 117/134 2023-05-20T01:15:27.511 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 118/134 2023-05-20T01:15:27.511 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 119/134 2023-05-20T01:15:27.511 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-05-20T01:15:27.511 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 121/134 2023-05-20T01:15:27.511 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-05-20T01:15:27.511 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 123/134 2023-05-20T01:15:27.512 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 124/134 2023-05-20T01:15:27.512 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 125/134 2023-05-20T01:15:27.513 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 126/134 2023-05-20T01:15:27.513 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 127/134 2023-05-20T01:15:27.513 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-05-20T01:15:27.513 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librados2-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 129/134 2023-05-20T01:15:27.513 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-05-20T01:15:27.514 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librbd1-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 131/134 2023-05-20T01:15:27.514 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-05-20T01:15:27.514 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-05-20T01:15:30.185 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-05-20T01:15:30.185 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:02:04 ago on Sat 20 May 2023 01:13:26 AM UTC. 2023-05-20T01:15:32.977 INFO:teuthology.orchestra.run.smithi016.stdout:Installed products updated. 2023-05-20T01:15:33.218 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:15:33.218 INFO:teuthology.orchestra.run.smithi016.stdout:Upgraded: 2023-05-20T01:15:33.219 INFO:teuthology.orchestra.run.smithi016.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-05-20T01:15:33.219 INFO:teuthology.orchestra.run.smithi016.stdout: librados2-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 2023-05-20T01:15:33.219 INFO:teuthology.orchestra.run.smithi016.stdout: librbd1-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 2023-05-20T01:15:33.219 INFO:teuthology.orchestra.run.smithi016.stdout:Installed: 2023-05-20T01:15:33.219 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-05-20T01:15:33.219 INFO:teuthology.orchestra.run.smithi016.stdout: automake-1.16.1-7.el8.noarch 2023-05-20T01:15:33.219 INFO:teuthology.orchestra.run.smithi016.stdout: bison-3.0.4-10.el8.x86_64 2023-05-20T01:15:33.220 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 2023-05-20T01:15:33.220 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-base-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 2023-05-20T01:15:33.220 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-common-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 2023-05-20T01:15:33.220 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-fuse-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 2023-05-20T01:15:33.220 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-grafana-dashboards-2:18.0.0-3996.g5ca1fdcf.el8.noarch 2023-05-20T01:15:33.220 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-immutable-object-cache-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 2023-05-20T01:15:33.221 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mds-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 2023-05-20T01:15:33.221 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 2023-05-20T01:15:33.221 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-cephadm-2:18.0.0-3996.g5ca1fdcf.el8.noarch 2023-05-20T01:15:33.221 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-dashboard-2:18.0.0-3996.g5ca1fdcf.el8.noarch 2023-05-20T01:15:33.221 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-diskprediction-local-2:18.0.0-3996.g5ca1fdcf.el8.noarch 2023-05-20T01:15:33.221 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-modules-core-2:18.0.0-3996.g5ca1fdcf.el8.noarch 2023-05-20T01:15:33.221 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-rook-2:18.0.0-3996.g5ca1fdcf.el8.noarch 2023-05-20T01:15:33.221 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mon-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 2023-05-20T01:15:33.222 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-osd-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 2023-05-20T01:15:33.222 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-prometheus-alerts-2:18.0.0-3996.g5ca1fdcf.el8.noarch 2023-05-20T01:15:33.222 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-radosgw-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 2023-05-20T01:15:33.222 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-selinux-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 2023-05-20T01:15:33.222 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-test-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 2023-05-20T01:15:33.222 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-volume-2:18.0.0-3996.g5ca1fdcf.el8.noarch 2023-05-20T01:15:33.222 INFO:teuthology.orchestra.run.smithi016.stdout: cephadm-2:18.0.0-3996.g5ca1fdcf.el8.noarch 2023-05-20T01:15:33.223 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-mirror-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 2023-05-20T01:15:33.223 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-top-2:18.0.0-3996.g5ca1fdcf.el8.noarch 2023-05-20T01:15:33.223 INFO:teuthology.orchestra.run.smithi016.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-05-20T01:15:33.223 INFO:teuthology.orchestra.run.smithi016.stdout: flex-2.6.1-9.el8.x86_64 2023-05-20T01:15:33.223 INFO:teuthology.orchestra.run.smithi016.stdout: fuse-2.9.7-15.el8.x86_64 2023-05-20T01:15:33.223 INFO:teuthology.orchestra.run.smithi016.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-05-20T01:15:33.223 INFO:teuthology.orchestra.run.smithi016.stdout: jq-1.6-3.el8.x86_64 2023-05-20T01:15:33.224 INFO:teuthology.orchestra.run.smithi016.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-05-20T01:15:33.224 INFO:teuthology.orchestra.run.smithi016.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-05-20T01:15:33.224 INFO:teuthology.orchestra.run.smithi016.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-05-20T01:15:33.224 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs-devel-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 2023-05-20T01:15:33.224 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs2-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 2023-05-20T01:15:33.224 INFO:teuthology.orchestra.run.smithi016.stdout: libcephsqlite-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 2023-05-20T01:15:33.224 INFO:teuthology.orchestra.run.smithi016.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-05-20T01:15:33.225 INFO:teuthology.orchestra.run.smithi016.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-05-20T01:15:33.225 INFO:teuthology.orchestra.run.smithi016.stdout: liboath-2.6.2-3.el8.x86_64 2023-05-20T01:15:33.225 INFO:teuthology.orchestra.run.smithi016.stdout: libpq-13.5-1.el8.x86_64 2023-05-20T01:15:33.225 INFO:teuthology.orchestra.run.smithi016.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-05-20T01:15:33.225 INFO:teuthology.orchestra.run.smithi016.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-05-20T01:15:33.225 INFO:teuthology.orchestra.run.smithi016.stdout: librados-devel-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 2023-05-20T01:15:33.226 INFO:teuthology.orchestra.run.smithi016.stdout: libradosstriper1-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 2023-05-20T01:15:33.226 INFO:teuthology.orchestra.run.smithi016.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-05-20T01:15:33.226 INFO:teuthology.orchestra.run.smithi016.stdout: librgw2-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 2023-05-20T01:15:33.226 INFO:teuthology.orchestra.run.smithi016.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-05-20T01:15:33.226 INFO:teuthology.orchestra.run.smithi016.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-05-20T01:15:33.226 INFO:teuthology.orchestra.run.smithi016.stdout: libtool-2.4.6-25.el8.x86_64 2023-05-20T01:15:33.226 INFO:teuthology.orchestra.run.smithi016.stdout: libunwind-1.3.1-3.el8.x86_64 2023-05-20T01:15:33.227 INFO:teuthology.orchestra.run.smithi016.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-05-20T01:15:33.227 INFO:teuthology.orchestra.run.smithi016.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-05-20T01:15:33.227 INFO:teuthology.orchestra.run.smithi016.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-05-20T01:15:33.227 INFO:teuthology.orchestra.run.smithi016.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-05-20T01:15:33.227 INFO:teuthology.orchestra.run.smithi016.stdout: openblas-0.3.15-3.el8.x86_64 2023-05-20T01:15:33.227 INFO:teuthology.orchestra.run.smithi016.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-05-20T01:15:33.227 INFO:teuthology.orchestra.run.smithi016.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-05-20T01:15:33.228 INFO:teuthology.orchestra.run.smithi016.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-05-20T01:15:33.228 INFO:teuthology.orchestra.run.smithi016.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-05-20T01:15:33.228 INFO:teuthology.orchestra.run.smithi016.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-05-20T01:15:33.228 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-05-20T01:15:33.228 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-05-20T01:15:33.229 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-05-20T01:15:33.229 INFO:teuthology.orchestra.run.smithi016.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-05-20T01:15:33.229 INFO:teuthology.orchestra.run.smithi016.stdout: python3-babel-2.5.1-7.el8.noarch 2023-05-20T01:15:33.229 INFO:teuthology.orchestra.run.smithi016.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-05-20T01:15:33.229 INFO:teuthology.orchestra.run.smithi016.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-05-20T01:15:33.229 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-05-20T01:15:33.229 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-argparse-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 2023-05-20T01:15:33.230 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-common-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 2023-05-20T01:15:33.230 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cephfs-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 2023-05-20T01:15:33.230 INFO:teuthology.orchestra.run.smithi016.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-05-20T01:15:33.230 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-05-20T01:15:33.230 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-05-20T01:15:33.230 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-05-20T01:15:33.231 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-05-20T01:15:33.231 INFO:teuthology.orchestra.run.smithi016.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-05-20T01:15:33.231 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-6.2-6.el8.noarch 2023-05-20T01:15:33.231 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-05-20T01:15:33.231 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-05-20T01:15:33.231 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-05-20T01:15:33.231 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-05-20T01:15:33.232 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-05-20T01:15:33.232 INFO:teuthology.orchestra.run.smithi016.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-05-20T01:15:33.232 INFO:teuthology.orchestra.run.smithi016.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-05-20T01:15:33.232 INFO:teuthology.orchestra.run.smithi016.stdout: python3-mako-1.0.6-13.el8.noarch 2023-05-20T01:15:33.232 INFO:teuthology.orchestra.run.smithi016.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-05-20T01:15:33.232 INFO:teuthology.orchestra.run.smithi016.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-05-20T01:15:33.232 INFO:teuthology.orchestra.run.smithi016.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-05-20T01:15:33.233 INFO:teuthology.orchestra.run.smithi016.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-05-20T01:15:33.233 INFO:teuthology.orchestra.run.smithi016.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-05-20T01:15:33.233 INFO:teuthology.orchestra.run.smithi016.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-05-20T01:15:33.233 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-05-20T01:15:33.233 INFO:teuthology.orchestra.run.smithi016.stdout: python3-portend-2.6-1.el8.noarch 2023-05-20T01:15:33.233 INFO:teuthology.orchestra.run.smithi016.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-05-20T01:15:33.233 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-05-20T01:15:33.233 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-05-20T01:15:33.234 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-05-20T01:15:33.234 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pycparser-2.14-14.el8.noarch 2023-05-20T01:15:33.234 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pytz-2017.2-9.el8.noarch 2023-05-20T01:15:33.234 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rados-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 2023-05-20T01:15:33.234 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rbd-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 2023-05-20T01:15:33.234 INFO:teuthology.orchestra.run.smithi016.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-05-20T01:15:33.234 INFO:teuthology.orchestra.run.smithi016.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-05-20T01:15:33.235 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rgw-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 2023-05-20T01:15:33.235 INFO:teuthology.orchestra.run.smithi016.stdout: python3-routes-2.4.1-12.el8.noarch 2023-05-20T01:15:33.235 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rsa-4.8-1.el8.noarch 2023-05-20T01:15:33.235 INFO:teuthology.orchestra.run.smithi016.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-05-20T01:15:33.235 INFO:teuthology.orchestra.run.smithi016.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-05-20T01:15:33.235 INFO:teuthology.orchestra.run.smithi016.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-05-20T01:15:33.235 INFO:teuthology.orchestra.run.smithi016.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-05-20T01:15:33.236 INFO:teuthology.orchestra.run.smithi016.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-05-20T01:15:33.236 INFO:teuthology.orchestra.run.smithi016.stdout: python3-waitress-1.2.1-2.el8.1.noarch 2023-05-20T01:15:33.236 INFO:teuthology.orchestra.run.smithi016.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-05-20T01:15:33.236 INFO:teuthology.orchestra.run.smithi016.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-05-20T01:15:33.236 INFO:teuthology.orchestra.run.smithi016.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-05-20T01:15:33.236 INFO:teuthology.orchestra.run.smithi016.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-05-20T01:15:33.237 INFO:teuthology.orchestra.run.smithi016.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-05-20T01:15:33.237 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-fuse-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 2023-05-20T01:15:33.237 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-mirror-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 2023-05-20T01:15:33.237 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-nbd-2:18.0.0-3996.g5ca1fdcf.el8.x86_64 2023-05-20T01:15:33.237 INFO:teuthology.orchestra.run.smithi016.stdout: socat-1.7.4.1-1.el8.x86_64 2023-05-20T01:15:33.237 INFO:teuthology.orchestra.run.smithi016.stdout: thrift-0.13.0-2.el8.x86_64 2023-05-20T01:15:33.237 INFO:teuthology.orchestra.run.smithi016.stdout: uuid-1.6.2-43.el8.x86_64 2023-05-20T01:15:33.237 INFO:teuthology.orchestra.run.smithi016.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-05-20T01:15:33.238 INFO:teuthology.orchestra.run.smithi016.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-05-20T01:15:33.238 INFO:teuthology.orchestra.run.smithi016.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-05-20T01:15:33.238 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:15:33.238 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2023-05-20T01:15:33.410 DEBUG:teuthology.parallel:result is None 2023-05-20T01:15:33.411 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=5ca1fdcfd7e6cec340f637cba9f0087e6db9c2c1 2023-05-20T01:15:33.559 DEBUG:teuthology.orchestra.run.smithi016:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-05-20T01:15:33.584 INFO:teuthology.orchestra.run.smithi016.stdout:18.0.0-3996.g5ca1fdcf.el8 2023-05-20T01:15:33.584 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-3996.g5ca1fdcf.el8 2023-05-20T01:15:33.585 INFO:teuthology.task.install:The correct ceph version 18.0.0-3996.g5ca1fdcf is installed. 2023-05-20T01:15:33.585 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=5ca1fdcfd7e6cec340f637cba9f0087e6db9c2c1 2023-05-20T01:15:33.724 DEBUG:teuthology.orchestra.run.smithi132:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-05-20T01:15:33.752 INFO:teuthology.orchestra.run.smithi132.stdout:18.0.0-3996.g5ca1fdcf.el8 2023-05-20T01:15:33.752 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-3996.g5ca1fdcf.el8 2023-05-20T01:15:33.752 INFO:teuthology.task.install:The correct ceph version 18.0.0-3996.g5ca1fdcf is installed. 2023-05-20T01:15:33.753 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-05-20T01:15:33.753 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-05-20T01:15:33.754 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-05-20T01:15:33.797 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2023-05-20T01:15:33.797 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-05-20T01:15:33.842 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-05-20T01:15:33.842 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-05-20T01:15:33.843 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/usr/bin/daemon-helper 2023-05-20T01:15:33.871 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-05-20T01:15:33.939 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2023-05-20T01:15:33.940 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd of=/usr/bin/daemon-helper 2023-05-20T01:15:33.969 DEBUG:teuthology.orchestra.run.smithi132:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-05-20T01:15:34.038 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-05-20T01:15:34.038 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-05-20T01:15:34.039 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/usr/bin/adjust-ulimits 2023-05-20T01:15:34.068 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-05-20T01:15:34.136 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2023-05-20T01:15:34.137 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd of=/usr/bin/adjust-ulimits 2023-05-20T01:15:34.166 DEBUG:teuthology.orchestra.run.smithi132:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-05-20T01:15:34.233 INFO:teuthology.run_tasks:Running task ceph... 2023-05-20T01:15:34.314 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-05-20T01:15:34.315 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 777 /var/log/ceph 2023-05-20T01:15:34.318 DEBUG:teuthology.orchestra.run.smithi132:> sudo chmod 777 /var/log/ceph 2023-05-20T01:15:34.345 INFO:tasks.ceph:Disabling ceph logrotate... 2023-05-20T01:15:34.345 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -f -- /etc/logrotate.d/ceph 2023-05-20T01:15:34.386 DEBUG:teuthology.orchestra.run.smithi132:> sudo rm -f -- /etc/logrotate.d/ceph 2023-05-20T01:15:34.415 INFO:tasks.ceph:Creating extra log directories... 2023-05-20T01:15:34.416 DEBUG:teuthology.orchestra.run.smithi016:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-05-20T01:15:34.455 DEBUG:teuthology.orchestra.run.smithi132:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-05-20T01:15:34.489 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-05-20T01:15:34.490 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-05-20T01:15:34.490 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-05-20T01:15:34.490 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-05-20T01:15:34.491 DEBUG:teuthology.orchestra.run.smithi016:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-05-20T01:15:34.540 DEBUG:teuthology.orchestra.run.smithi016:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-05-20T01:15:34.684 DEBUG:teuthology.orchestra.run.smithi016:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-05-20T01:15:34.712 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2023-05-20T01:15:34.712 DEBUG:teuthology.orchestra.run.smithi132:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-05-20T01:15:34.729 DEBUG:teuthology.orchestra.run.smithi132:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-05-20T01:15:34.860 DEBUG:teuthology.orchestra.run.smithi132:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-05-20T01:15:34.893 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-05-20T01:15:34.893 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'bad backtrace', 'object missing on disk', 'error reading table object', 'error reading sessionmap', 'unmatched fragstat', 'unmatched rstat', 'was unreadable, recreating it now', 'Scrub error on inode', 'Metadata damage detected', 'inconsistent rstat on inode', 'Error recovering journal'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-05-20T01:15:34.894 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/yuriw-2023-05-19_16:21:22-fs-wip-yuri7-testing-2023-05-18-0655-distro-default-smithi/7278970', 'branch': 'wip-yuri7-testing-2023-05-18-0655', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_20.04} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/data-scan}', 'email': 'yweinste@redhat.com', 'first_in_suite': False, 'job_id': '7278970', 'kernel': {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'yuriw-2023-05-19_16:21:22-fs-wip-yuri7-testing-2023-05-18-0655-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'rhel', 'os_version': '8.6', 'overrides': {'admin_socket': {'branch': 'wip-yuri7-testing-2023-05-18-0655'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'bad backtrace', 'object missing on disk', 'error reading table object', 'error reading sessionmap', 'unmatched fragstat', 'unmatched rstat', 'was unreadable, recreating it now', 'Scrub error on inode', 'Metadata damage detected', 'inconsistent rstat on inode', 'Error recovering journal'], 'sha1': '5ca1fdcfd7e6cec340f637cba9f0087e6db9c2c1'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '5ca1fdcfd7e6cec340f637cba9f0087e6db9c2c1'}}, 'kclient': {'syntax': 'v2'}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-yuri7-testing-2023-05-18-0655', 'sha1': '5ca1fdcfd7e6cec340f637cba9f0087e6db9c2c1'}}, 'owner': 'scheduled_yuriw@teuthology', 'priority': 75, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 6431, 'sha1': '5ca1fdcfd7e6cec340f637cba9f0087e6db9c2c1', 'sleep_before_teardown': 0, 'subset': '111/120000', 'suite': 'fs', 'suite_branch': 'wip-yuri7-testing-2023-05-18-0655', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_5ca1fdcfd7e6cec340f637cba9f0087e6db9c2c1/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': '5ca1fdcfd7e6cec340f637cba9f0087e6db9c2c1', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi016.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac=', 'smithi132.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '5ca1fdcfd7e6cec340f637cba9f0087e6db9c2c1'}}, {'ceph': None}, {'kclient': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_data_scan']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '69caab8ec6f44c61c991ae693e8e7b8cd859a896', 'timestamp': '2023-05-19_16:21:22', 'tube': 'smithi', 'user': 'yuriw', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.23459'} 2023-05-20T01:15:34.895 DEBUG:teuthology.orchestra.run.smithi016:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-05-20T01:15:34.897 DEBUG:teuthology.orchestra.run.smithi132:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-05-20T01:15:34.944 DEBUG:teuthology.orchestra.run.smithi016:> sudo install -d -m0777 -- /var/run/ceph 2023-05-20T01:15:34.955 DEBUG:teuthology.orchestra.run.smithi132:> sudo install -d -m0777 -- /var/run/ceph 2023-05-20T01:15:35.009 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-05-20T01:15:35.010 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/scratch_devs of=/dev/stdout 2023-05-20T01:15:35.038 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-05-20T01:15:35.038 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_1 2023-05-20T01:15:35.096 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-05-20T01:15:35.097 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-05-20T01:15:35.097 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 6h/6d Inode: 371230 Links: 1 2023-05-20T01:15:35.097 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-05-20T01:15:35.098 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:device_t:s0 2023-05-20T01:15:35.098 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2023-05-20 01:15:27.168761203 +0000 2023-05-20T01:15:35.098 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2023-05-20 01:12:38.493054907 +0000 2023-05-20T01:15:35.099 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2023-05-20 01:12:38.493054907 +0000 2023-05-20T01:15:35.099 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: - 2023-05-20T01:15:35.099 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-05-20T01:15:35.164 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2023-05-20T01:15:35.164 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2023-05-20T01:15:35.164 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 0.000126133 s, 4.1 MB/s 2023-05-20T01:15:35.165 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-05-20T01:15:35.223 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_2 2023-05-20T01:15:35.281 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-05-20T01:15:35.282 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-05-20T01:15:35.282 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 6h/6d Inode: 370049 Links: 1 2023-05-20T01:15:35.282 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-05-20T01:15:35.282 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:device_t:s0 2023-05-20T01:15:35.282 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2023-05-20 01:15:27.168761203 +0000 2023-05-20T01:15:35.282 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2023-05-20 01:12:39.019045983 +0000 2023-05-20T01:15:35.283 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2023-05-20 01:12:39.019045983 +0000 2023-05-20T01:15:35.283 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: - 2023-05-20T01:15:35.283 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-05-20T01:15:35.348 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2023-05-20T01:15:35.348 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2023-05-20T01:15:35.348 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 0.000108896 s, 4.7 MB/s 2023-05-20T01:15:35.350 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-05-20T01:15:35.408 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_3 2023-05-20T01:15:35.466 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-05-20T01:15:35.466 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-05-20T01:15:35.466 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 6h/6d Inode: 370207 Links: 1 2023-05-20T01:15:35.466 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-05-20T01:15:35.466 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:device_t:s0 2023-05-20T01:15:35.466 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2023-05-20 01:15:27.168761203 +0000 2023-05-20T01:15:35.467 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2023-05-20 01:12:39.537037194 +0000 2023-05-20T01:15:35.467 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2023-05-20 01:12:39.537037194 +0000 2023-05-20T01:15:35.467 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: - 2023-05-20T01:15:35.467 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-05-20T01:15:35.532 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2023-05-20T01:15:35.533 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2023-05-20T01:15:35.533 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 0.000113111 s, 4.5 MB/s 2023-05-20T01:15:35.536 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-05-20T01:15:35.593 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_4 2023-05-20T01:15:35.651 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-05-20T01:15:35.651 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-05-20T01:15:35.651 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 6h/6d Inode: 370315 Links: 1 2023-05-20T01:15:35.651 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-05-20T01:15:35.652 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:device_t:s0 2023-05-20T01:15:35.652 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2023-05-20 01:15:27.168761203 +0000 2023-05-20T01:15:35.652 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2023-05-20 01:12:40.076028050 +0000 2023-05-20T01:15:35.652 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2023-05-20 01:12:40.076028050 +0000 2023-05-20T01:15:35.652 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: - 2023-05-20T01:15:35.652 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-05-20T01:15:35.717 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2023-05-20T01:15:35.718 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2023-05-20T01:15:35.718 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 0.00011675 s, 4.4 MB/s 2023-05-20T01:15:35.719 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-05-20T01:15:35.779 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-05-20T01:15:35.780 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2023-05-20T01:15:35.780 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/scratch_devs of=/dev/stdout 2023-05-20T01:15:35.799 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-05-20T01:15:35.800 DEBUG:teuthology.orchestra.run.smithi132:> stat /dev/vg_nvme/lv_1 2023-05-20T01:15:35.859 INFO:teuthology.orchestra.run.smithi132.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-05-20T01:15:35.859 INFO:teuthology.orchestra.run.smithi132.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-05-20T01:15:35.860 INFO:teuthology.orchestra.run.smithi132.stdout:Device: 6h/6d Inode: 354927 Links: 1 2023-05-20T01:15:35.860 INFO:teuthology.orchestra.run.smithi132.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-05-20T01:15:35.861 INFO:teuthology.orchestra.run.smithi132.stdout:Context: system_u:object_r:device_t:s0 2023-05-20T01:15:35.861 INFO:teuthology.orchestra.run.smithi132.stdout:Access: 2023-05-20 01:15:11.073090798 +0000 2023-05-20T01:15:35.862 INFO:teuthology.orchestra.run.smithi132.stdout:Modify: 2023-05-20 01:12:22.000044011 +0000 2023-05-20T01:15:35.862 INFO:teuthology.orchestra.run.smithi132.stdout:Change: 2023-05-20 01:12:22.000044011 +0000 2023-05-20T01:15:35.862 INFO:teuthology.orchestra.run.smithi132.stdout: Birth: - 2023-05-20T01:15:35.862 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-05-20T01:15:35.926 INFO:teuthology.orchestra.run.smithi132.stderr:1+0 records in 2023-05-20T01:15:35.927 INFO:teuthology.orchestra.run.smithi132.stderr:1+0 records out 2023-05-20T01:15:35.927 INFO:teuthology.orchestra.run.smithi132.stderr:512 bytes copied, 0.000265962 s, 1.9 MB/s 2023-05-20T01:15:35.928 DEBUG:teuthology.orchestra.run.smithi132:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-05-20T01:15:35.989 DEBUG:teuthology.orchestra.run.smithi132:> stat /dev/vg_nvme/lv_2 2023-05-20T01:15:36.047 INFO:teuthology.orchestra.run.smithi132.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-05-20T01:15:36.048 INFO:teuthology.orchestra.run.smithi132.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-05-20T01:15:36.048 INFO:teuthology.orchestra.run.smithi132.stdout:Device: 6h/6d Inode: 363408 Links: 1 2023-05-20T01:15:36.048 INFO:teuthology.orchestra.run.smithi132.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-05-20T01:15:36.048 INFO:teuthology.orchestra.run.smithi132.stdout:Context: system_u:object_r:device_t:s0 2023-05-20T01:15:36.048 INFO:teuthology.orchestra.run.smithi132.stdout:Access: 2023-05-20 01:15:11.073090798 +0000 2023-05-20T01:15:36.049 INFO:teuthology.orchestra.run.smithi132.stdout:Modify: 2023-05-20 01:12:22.536029145 +0000 2023-05-20T01:15:36.049 INFO:teuthology.orchestra.run.smithi132.stdout:Change: 2023-05-20 01:12:22.536029145 +0000 2023-05-20T01:15:36.049 INFO:teuthology.orchestra.run.smithi132.stdout: Birth: - 2023-05-20T01:15:36.049 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-05-20T01:15:36.114 INFO:teuthology.orchestra.run.smithi132.stderr:1+0 records in 2023-05-20T01:15:36.115 INFO:teuthology.orchestra.run.smithi132.stderr:1+0 records out 2023-05-20T01:15:36.115 INFO:teuthology.orchestra.run.smithi132.stderr:512 bytes copied, 0.000231295 s, 2.2 MB/s 2023-05-20T01:15:36.116 DEBUG:teuthology.orchestra.run.smithi132:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-05-20T01:15:36.175 DEBUG:teuthology.orchestra.run.smithi132:> stat /dev/vg_nvme/lv_3 2023-05-20T01:15:36.233 INFO:teuthology.orchestra.run.smithi132.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-05-20T01:15:36.233 INFO:teuthology.orchestra.run.smithi132.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-05-20T01:15:36.233 INFO:teuthology.orchestra.run.smithi132.stdout:Device: 6h/6d Inode: 364403 Links: 1 2023-05-20T01:15:36.234 INFO:teuthology.orchestra.run.smithi132.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-05-20T01:15:36.234 INFO:teuthology.orchestra.run.smithi132.stdout:Context: system_u:object_r:device_t:s0 2023-05-20T01:15:36.234 INFO:teuthology.orchestra.run.smithi132.stdout:Access: 2023-05-20 01:15:11.073090798 +0000 2023-05-20T01:15:36.234 INFO:teuthology.orchestra.run.smithi132.stdout:Modify: 2023-05-20 01:12:23.056014722 +0000 2023-05-20T01:15:36.234 INFO:teuthology.orchestra.run.smithi132.stdout:Change: 2023-05-20 01:12:23.056014722 +0000 2023-05-20T01:15:36.234 INFO:teuthology.orchestra.run.smithi132.stdout: Birth: - 2023-05-20T01:15:36.234 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-05-20T01:15:36.299 INFO:teuthology.orchestra.run.smithi132.stderr:1+0 records in 2023-05-20T01:15:36.299 INFO:teuthology.orchestra.run.smithi132.stderr:1+0 records out 2023-05-20T01:15:36.300 INFO:teuthology.orchestra.run.smithi132.stderr:512 bytes copied, 0.000261805 s, 2.0 MB/s 2023-05-20T01:15:36.301 DEBUG:teuthology.orchestra.run.smithi132:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-05-20T01:15:36.359 DEBUG:teuthology.orchestra.run.smithi132:> stat /dev/vg_nvme/lv_4 2023-05-20T01:15:36.416 INFO:teuthology.orchestra.run.smithi132.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-05-20T01:15:36.416 INFO:teuthology.orchestra.run.smithi132.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-05-20T01:15:36.417 INFO:teuthology.orchestra.run.smithi132.stdout:Device: 6h/6d Inode: 365093 Links: 1 2023-05-20T01:15:36.417 INFO:teuthology.orchestra.run.smithi132.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-05-20T01:15:36.417 INFO:teuthology.orchestra.run.smithi132.stdout:Context: system_u:object_r:device_t:s0 2023-05-20T01:15:36.417 INFO:teuthology.orchestra.run.smithi132.stdout:Access: 2023-05-20 01:15:11.074090771 +0000 2023-05-20T01:15:36.417 INFO:teuthology.orchestra.run.smithi132.stdout:Modify: 2023-05-20 01:12:23.579000216 +0000 2023-05-20T01:15:36.417 INFO:teuthology.orchestra.run.smithi132.stdout:Change: 2023-05-20 01:12:23.579000216 +0000 2023-05-20T01:15:36.417 INFO:teuthology.orchestra.run.smithi132.stdout: Birth: - 2023-05-20T01:15:36.418 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-05-20T01:15:36.482 INFO:teuthology.orchestra.run.smithi132.stderr:1+0 records in 2023-05-20T01:15:36.482 INFO:teuthology.orchestra.run.smithi132.stderr:1+0 records out 2023-05-20T01:15:36.483 INFO:teuthology.orchestra.run.smithi132.stderr:512 bytes copied, 0.000217874 s, 2.3 MB/s 2023-05-20T01:15:36.484 DEBUG:teuthology.orchestra.run.smithi132:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-05-20T01:15:36.541 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-05-20T01:15:36.542 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi016.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi132.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-05-20T01:15:36.542 INFO:tasks.ceph:Generating config... 2023-05-20T01:15:36.543 INFO:tasks.ceph:[client] client mount timeout = 600 2023-05-20T01:15:36.543 INFO:tasks.ceph:[client] debug client = 20 2023-05-20T01:15:36.544 INFO:tasks.ceph:[client] debug ms = 1 2023-05-20T01:15:36.544 INFO:tasks.ceph:[client] pid file = 2023-05-20T01:15:36.544 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-05-20T01:15:36.544 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-05-20T01:15:36.544 INFO:tasks.ceph:[global] ms die on skipped message = False 2023-05-20T01:15:36.545 INFO:tasks.ceph:[mds] debug mds = 20 2023-05-20T01:15:36.545 INFO:tasks.ceph:[mds] debug mds balancer = 20 2023-05-20T01:15:36.545 INFO:tasks.ceph:[mds] debug ms = 1 2023-05-20T01:15:36.545 INFO:tasks.ceph:[mds] mds debug frag = True 2023-05-20T01:15:36.545 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-05-20T01:15:36.546 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-05-20T01:15:36.546 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-05-20T01:15:36.546 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-05-20T01:15:36.546 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-05-20T01:15:36.546 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-05-20T01:15:36.547 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-05-20T01:15:36.547 INFO:tasks.ceph:[mgr] debug ms = 1 2023-05-20T01:15:36.547 INFO:tasks.ceph:[mon] debug mon = 20 2023-05-20T01:15:36.547 INFO:tasks.ceph:[mon] debug ms = 1 2023-05-20T01:15:36.547 INFO:tasks.ceph:[mon] debug paxos = 20 2023-05-20T01:15:36.548 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-05-20T01:15:36.548 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-05-20T01:15:36.548 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-05-20T01:15:36.548 INFO:tasks.ceph:[osd] debug bluefs = 20 2023-05-20T01:15:36.548 INFO:tasks.ceph:[osd] debug bluestore = 20 2023-05-20T01:15:36.549 INFO:tasks.ceph:[osd] debug ms = 1 2023-05-20T01:15:36.549 INFO:tasks.ceph:[osd] debug osd = 20 2023-05-20T01:15:36.549 INFO:tasks.ceph:[osd] debug rocksdb = 10 2023-05-20T01:15:36.549 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-05-20T01:15:36.549 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-05-20T01:15:36.549 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-05-20T01:15:36.550 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-05-20T01:15:36.550 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-05-20T01:15:36.550 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-05-20T01:15:36.550 INFO:tasks.ceph:Setting up mon.a... 2023-05-20T01:15:36.551 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-05-20T01:15:36.642 INFO:teuthology.orchestra.run.smithi016.stdout:creating /etc/ceph/ceph.keyring 2023-05-20T01:15:36.645 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-05-20T01:15:36.695 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-05-20T01:15:36.723 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.16'), ('mon.b', '172.21.15.132'), ('mon.c', '[v2:172.21.15.132:3301,v1:172.21.15.132:6790]')] 2023-05-20T01:15:36.724 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.16,172.21.15.132,[v2:172.21.15.132:3301,v1:172.21.15.132:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-05-20T01:15:36.725 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-05-20T01:15:36.726 DEBUG:teuthology.orchestra.run.smithi016:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-05-20T01:15:36.780 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.16 --add b 172.21.15.132 --addv c '[v2:172.21.15.132:3301,v1:172.21.15.132:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-05-20T01:15:36.869 INFO:teuthology.orchestra.run.smithi016.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-05-20T01:15:36.869 INFO:teuthology.orchestra.run.smithi016.stdout:monmaptool: generated fsid c2245cc5-ecab-4087-a59b-bc81db901676 2023-05-20T01:15:36.869 INFO:teuthology.orchestra.run.smithi016.stdout:setting min_mon_release = pacific 2023-05-20T01:15:36.870 INFO:teuthology.orchestra.run.smithi016.stdout:epoch 0 2023-05-20T01:15:36.870 INFO:teuthology.orchestra.run.smithi016.stdout:fsid c2245cc5-ecab-4087-a59b-bc81db901676 2023-05-20T01:15:36.870 INFO:teuthology.orchestra.run.smithi016.stdout:last_changed 2023-05-20T01:15:36.867138+0000 2023-05-20T01:15:36.870 INFO:teuthology.orchestra.run.smithi016.stdout:created 2023-05-20T01:15:36.867138+0000 2023-05-20T01:15:36.870 INFO:teuthology.orchestra.run.smithi016.stdout:min_mon_release 16 (pacific) 2023-05-20T01:15:36.870 INFO:teuthology.orchestra.run.smithi016.stdout:election_strategy: 1 2023-05-20T01:15:36.870 INFO:teuthology.orchestra.run.smithi016.stdout:0: [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon.a 2023-05-20T01:15:36.871 INFO:teuthology.orchestra.run.smithi016.stdout:1: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.b 2023-05-20T01:15:36.871 INFO:teuthology.orchestra.run.smithi016.stdout:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2023-05-20T01:15:36.871 INFO:teuthology.orchestra.run.smithi016.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-05-20T01:15:36.872 DEBUG:teuthology.orchestra.run.smithi016:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-05-20T01:15:36.889 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID c2245cc5-ecab-4087-a59b-bc81db901676... 2023-05-20T01:15:36.890 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-05-20T01:15:36.931 DEBUG:teuthology.orchestra.run.smithi132:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-05-20T01:15:36.975 INFO:teuthology.orchestra.run.smithi016.stdout:[global] 2023-05-20T01:15:36.976 INFO:teuthology.orchestra.run.smithi016.stdout: chdir = "" 2023-05-20T01:15:36.976 INFO:teuthology.orchestra.run.smithi016.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-05-20T01:15:36.976 INFO:teuthology.orchestra.run.smithi016.stdout: auth supported = cephx 2023-05-20T01:15:36.976 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:15:36.976 INFO:teuthology.orchestra.run.smithi016.stdout: filestore xattr use omap = true 2023-05-20T01:15:36.976 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:15:36.976 INFO:teuthology.orchestra.run.smithi016.stdout: mon clock drift allowed = 1.000 2023-05-20T01:15:36.977 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:15:36.977 INFO:teuthology.orchestra.run.smithi016.stdout: osd crush chooseleaf type = 0 2023-05-20T01:15:36.977 INFO:teuthology.orchestra.run.smithi016.stdout: auth debug = true 2023-05-20T01:15:36.977 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:15:36.977 INFO:teuthology.orchestra.run.smithi016.stdout: ms die on old message = true 2023-05-20T01:15:36.977 INFO:teuthology.orchestra.run.smithi016.stdout: ms die on bug = true 2023-05-20T01:15:36.977 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:15:36.978 INFO:teuthology.orchestra.run.smithi016.stdout: mon max pg per osd = 10000 # >= luminous 2023-05-20T01:15:36.978 INFO:teuthology.orchestra.run.smithi016.stdout: mon pg warn max object skew = 0 2023-05-20T01:15:36.978 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:15:36.978 INFO:teuthology.orchestra.run.smithi016.stdout: # disable pg_autoscaler by default for new pools 2023-05-20T01:15:36.978 INFO:teuthology.orchestra.run.smithi016.stdout: osd_pool_default_pg_autoscale_mode = off 2023-05-20T01:15:36.978 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:15:36.979 INFO:teuthology.orchestra.run.smithi016.stdout: osd pool default size = 2 2023-05-20T01:15:36.979 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:15:36.979 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd allow primary affinity = true 2023-05-20T01:15:36.979 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd allow pg remap = true 2023-05-20T01:15:36.979 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on legacy crush tunables = false 2023-05-20T01:15:36.979 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on crush straw calc version zero = false 2023-05-20T01:15:36.979 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on no sortbitwise = false 2023-05-20T01:15:36.980 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on osd down out interval zero = false 2023-05-20T01:15:36.980 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on too few osds = false 2023-05-20T01:15:36.980 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-05-20T01:15:36.980 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_pool_no_redundancy = false 2023-05-20T01:15:36.980 INFO:teuthology.orchestra.run.smithi016.stdout: mon_allow_pool_size_one = true 2023-05-20T01:15:36.980 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:15:36.980 INFO:teuthology.orchestra.run.smithi016.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-05-20T01:15:36.980 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:15:36.981 INFO:teuthology.orchestra.run.smithi016.stdout: osd default data pool replay window = 5 2023-05-20T01:15:36.981 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:15:36.981 INFO:teuthology.orchestra.run.smithi016.stdout: mon allow pool delete = true 2023-05-20T01:15:36.981 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:15:36.981 INFO:teuthology.orchestra.run.smithi016.stdout: mon cluster log file level = debug 2023-05-20T01:15:36.981 INFO:teuthology.orchestra.run.smithi016.stdout: debug asserts on shutdown = true 2023-05-20T01:15:36.982 INFO:teuthology.orchestra.run.smithi016.stdout: mon health detail to clog = false 2023-05-20T01:15:36.982 INFO:teuthology.orchestra.run.smithi016.stdout: mon host = "172.21.15.16,172.21.15.132,[v2:172.21.15.132:3301,v1:172.21.15.132:6790]" 2023-05-20T01:15:36.982 INFO:teuthology.orchestra.run.smithi016.stdout: ms die on skipped message = False 2023-05-20T01:15:36.982 INFO:teuthology.orchestra.run.smithi016.stdout: fsid = c2245cc5-ecab-4087-a59b-bc81db901676 2023-05-20T01:15:36.982 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:15:36.983 INFO:teuthology.orchestra.run.smithi016.stdout:[osd] 2023-05-20T01:15:36.983 INFO:teuthology.orchestra.run.smithi016.stdout: osd journal size = 100 2023-05-20T01:15:36.983 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:15:36.983 INFO:teuthology.orchestra.run.smithi016.stdout: osd scrub load threshold = 5.0 2023-05-20T01:15:36.983 INFO:teuthology.orchestra.run.smithi016.stdout: osd scrub max interval = 600 2023-05-20T01:15:36.983 INFO:teuthology.orchestra.run.smithi016.stdout: osd mclock profile = high_recovery_ops 2023-05-20T01:15:36.983 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:15:36.984 INFO:teuthology.orchestra.run.smithi016.stdout: osd recover clone overlap = true 2023-05-20T01:15:36.984 INFO:teuthology.orchestra.run.smithi016.stdout: osd recovery max chunk = 1048576 2023-05-20T01:15:36.984 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:15:36.984 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug shutdown = true 2023-05-20T01:15:36.984 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug op order = true 2023-05-20T01:15:36.985 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug verify stray on activate = true 2023-05-20T01:15:36.985 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:15:36.985 INFO:teuthology.orchestra.run.smithi016.stdout: osd open classes on start = true 2023-05-20T01:15:36.986 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug pg log writeout = true 2023-05-20T01:15:36.986 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:15:36.986 INFO:teuthology.orchestra.run.smithi016.stdout: osd deep scrub update digest min age = 30 2023-05-20T01:15:36.986 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:15:36.987 INFO:teuthology.orchestra.run.smithi016.stdout: osd map max advance = 10 2023-05-20T01:15:36.987 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:15:36.987 INFO:teuthology.orchestra.run.smithi016.stdout: journal zero on create = true 2023-05-20T01:15:36.987 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:15:36.987 INFO:teuthology.orchestra.run.smithi016.stdout: filestore ondisk finisher threads = 3 2023-05-20T01:15:36.988 INFO:teuthology.orchestra.run.smithi016.stdout: filestore apply finisher threads = 3 2023-05-20T01:15:36.988 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:15:36.988 INFO:teuthology.orchestra.run.smithi016.stdout: bdev debug aio = true 2023-05-20T01:15:36.988 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug misdirected ops = true 2023-05-20T01:15:36.988 INFO:teuthology.orchestra.run.smithi016.stdout: bluestore block size = 96636764160 2023-05-20T01:15:36.989 INFO:teuthology.orchestra.run.smithi016.stdout: bluestore fsck on mount = True 2023-05-20T01:15:36.989 INFO:teuthology.orchestra.run.smithi016.stdout: debug bluefs = 20 2023-05-20T01:15:36.989 INFO:teuthology.orchestra.run.smithi016.stdout: debug bluestore = 20 2023-05-20T01:15:36.989 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2023-05-20T01:15:36.989 INFO:teuthology.orchestra.run.smithi016.stdout: debug osd = 20 2023-05-20T01:15:36.989 INFO:teuthology.orchestra.run.smithi016.stdout: debug rocksdb = 10 2023-05-20T01:15:36.989 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd backfillfull_ratio = 0.85 2023-05-20T01:15:36.990 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd full ratio = 0.9 2023-05-20T01:15:36.990 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd nearfull ratio = 0.8 2023-05-20T01:15:36.990 INFO:teuthology.orchestra.run.smithi016.stdout: osd failsafe full ratio = 0.95 2023-05-20T01:15:36.990 INFO:teuthology.orchestra.run.smithi016.stdout: osd objectstore = bluestore 2023-05-20T01:15:36.990 INFO:teuthology.orchestra.run.smithi016.stdout: osd op complaint time = 180 2023-05-20T01:15:36.990 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:15:36.991 INFO:teuthology.orchestra.run.smithi016.stdout:[mgr] 2023-05-20T01:15:36.991 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2023-05-20T01:15:36.991 INFO:teuthology.orchestra.run.smithi016.stdout: debug mgr = 20 2023-05-20T01:15:36.992 INFO:teuthology.orchestra.run.smithi016.stdout: debug mon = 20 2023-05-20T01:15:36.992 INFO:teuthology.orchestra.run.smithi016.stdout: debug auth = 20 2023-05-20T01:15:36.992 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min pgs per osd = 4 2023-05-20T01:15:36.992 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min bytes per osd = 10 2023-05-20T01:15:36.993 INFO:teuthology.orchestra.run.smithi016.stdout: mgr/telemetry/nag = false 2023-05-20T01:15:36.993 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:15:36.994 INFO:teuthology.orchestra.run.smithi016.stdout:[mon] 2023-05-20T01:15:36.994 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2023-05-20T01:15:36.994 INFO:teuthology.orchestra.run.smithi016.stdout: debug mon = 20 2023-05-20T01:15:36.994 INFO:teuthology.orchestra.run.smithi016.stdout: debug paxos = 20 2023-05-20T01:15:36.995 INFO:teuthology.orchestra.run.smithi016.stdout: debug auth = 20 2023-05-20T01:15:36.995 INFO:teuthology.orchestra.run.smithi016.stdout: mon data avail warn = 5 2023-05-20T01:15:36.995 INFO:teuthology.orchestra.run.smithi016.stdout: mon mgr mkfs grace = 240 2023-05-20T01:15:36.996 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min pgs per osd = 4 2023-05-20T01:15:36.996 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd reporter subtree level = osd 2023-05-20T01:15:36.996 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd prime pg temp = true 2023-05-20T01:15:36.997 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min bytes per osd = 10 2023-05-20T01:15:36.997 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:15:36.997 INFO:teuthology.orchestra.run.smithi016.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-05-20T01:15:36.997 INFO:teuthology.orchestra.run.smithi016.stdout: auth mon ticket ttl = 660 # 11m 2023-05-20T01:15:36.998 INFO:teuthology.orchestra.run.smithi016.stdout: auth service ticket ttl = 240 # 4m 2023-05-20T01:15:36.998 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:15:36.998 INFO:teuthology.orchestra.run.smithi016.stdout: # don't complain about insecure global_id in the test suite 2023-05-20T01:15:36.999 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-05-20T01:15:36.999 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-05-20T01:15:36.999 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:15:36.999 INFO:teuthology.orchestra.run.smithi016.stdout: # 1m isn't quite enough 2023-05-20T01:15:36.999 INFO:teuthology.orchestra.run.smithi016.stdout: mon_down_mkfs_grace = 2m 2023-05-20T01:15:37.000 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:15:37.000 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_filestore_osds = false 2023-05-20T01:15:37.000 INFO:teuthology.orchestra.run.smithi016.stdout: mon op complaint time = 120 2023-05-20T01:15:37.001 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:15:37.001 INFO:teuthology.orchestra.run.smithi016.stdout:[client] 2023-05-20T01:15:37.002 INFO:teuthology.orchestra.run.smithi016.stdout: rgw cache enabled = true 2023-05-20T01:15:37.002 INFO:teuthology.orchestra.run.smithi016.stdout: rgw enable ops log = true 2023-05-20T01:15:37.002 INFO:teuthology.orchestra.run.smithi016.stdout: rgw enable usage log = true 2023-05-20T01:15:37.003 INFO:teuthology.orchestra.run.smithi016.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-05-20T01:15:37.003 INFO:teuthology.orchestra.run.smithi016.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-05-20T01:15:37.003 INFO:teuthology.orchestra.run.smithi016.stdout: client mount timeout = 600 2023-05-20T01:15:37.003 INFO:teuthology.orchestra.run.smithi016.stdout: debug client = 20 2023-05-20T01:15:37.004 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2023-05-20T01:15:37.004 INFO:teuthology.orchestra.run.smithi016.stdout: pid file = "" 2023-05-20T01:15:37.004 INFO:teuthology.orchestra.run.smithi016.stdout: rados mon op timeout = 900 2023-05-20T01:15:37.004 INFO:teuthology.orchestra.run.smithi016.stdout: rados osd op timeout = 900 2023-05-20T01:15:37.005 INFO:teuthology.orchestra.run.smithi016.stdout:[mon.a] 2023-05-20T01:15:37.005 INFO:teuthology.orchestra.run.smithi016.stdout:[mon.b] 2023-05-20T01:15:37.005 INFO:teuthology.orchestra.run.smithi016.stdout:[mon.c] 2023-05-20T01:15:37.005 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.a] 2023-05-20T01:15:37.006 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.c] 2023-05-20T01:15:37.006 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.b] 2023-05-20T01:15:37.006 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.d] 2023-05-20T01:15:37.006 INFO:teuthology.orchestra.run.smithi016.stdout:[mds] 2023-05-20T01:15:37.007 INFO:teuthology.orchestra.run.smithi016.stdout: debug mds = 20 2023-05-20T01:15:37.007 INFO:teuthology.orchestra.run.smithi016.stdout: debug mds balancer = 20 2023-05-20T01:15:37.007 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2023-05-20T01:15:37.007 INFO:teuthology.orchestra.run.smithi016.stdout: mds debug frag = True 2023-05-20T01:15:37.008 INFO:teuthology.orchestra.run.smithi016.stdout: mds debug scatterstat = True 2023-05-20T01:15:37.008 INFO:teuthology.orchestra.run.smithi016.stdout: mds op complaint time = 180 2023-05-20T01:15:37.008 INFO:teuthology.orchestra.run.smithi016.stdout: mds verify scatter = True 2023-05-20T01:15:37.008 INFO:teuthology.orchestra.run.smithi016.stdout: osd op complaint time = 180 2023-05-20T01:15:37.009 INFO:teuthology.orchestra.run.smithi016.stdout: rados mon op timeout = 900 2023-05-20T01:15:37.009 INFO:teuthology.orchestra.run.smithi016.stdout: rados osd op timeout = 900 2023-05-20T01:15:37.011 INFO:teuthology.orchestra.run.smithi132.stdout:[global] 2023-05-20T01:15:37.011 INFO:teuthology.orchestra.run.smithi132.stdout: chdir = "" 2023-05-20T01:15:37.012 INFO:teuthology.orchestra.run.smithi132.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-05-20T01:15:37.012 INFO:teuthology.orchestra.run.smithi132.stdout: auth supported = cephx 2023-05-20T01:15:37.012 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:15:37.012 INFO:teuthology.orchestra.run.smithi132.stdout: filestore xattr use omap = true 2023-05-20T01:15:37.013 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:15:37.013 INFO:teuthology.orchestra.run.smithi132.stdout: mon clock drift allowed = 1.000 2023-05-20T01:15:37.013 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:15:37.013 INFO:teuthology.orchestra.run.smithi132.stdout: osd crush chooseleaf type = 0 2023-05-20T01:15:37.014 INFO:teuthology.orchestra.run.smithi132.stdout: auth debug = true 2023-05-20T01:15:37.014 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:15:37.014 INFO:teuthology.orchestra.run.smithi132.stdout: ms die on old message = true 2023-05-20T01:15:37.014 INFO:teuthology.orchestra.run.smithi132.stdout: ms die on bug = true 2023-05-20T01:15:37.015 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:15:37.015 INFO:teuthology.orchestra.run.smithi132.stdout: mon max pg per osd = 10000 # >= luminous 2023-05-20T01:15:37.015 INFO:teuthology.orchestra.run.smithi132.stdout: mon pg warn max object skew = 0 2023-05-20T01:15:37.016 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:15:37.016 INFO:teuthology.orchestra.run.smithi132.stdout: # disable pg_autoscaler by default for new pools 2023-05-20T01:15:37.016 INFO:teuthology.orchestra.run.smithi132.stdout: osd_pool_default_pg_autoscale_mode = off 2023-05-20T01:15:37.016 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:15:37.017 INFO:teuthology.orchestra.run.smithi132.stdout: osd pool default size = 2 2023-05-20T01:15:37.017 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:15:37.017 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd allow primary affinity = true 2023-05-20T01:15:37.017 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd allow pg remap = true 2023-05-20T01:15:37.017 INFO:teuthology.orchestra.run.smithi132.stdout: mon warn on legacy crush tunables = false 2023-05-20T01:15:37.018 INFO:teuthology.orchestra.run.smithi132.stdout: mon warn on crush straw calc version zero = false 2023-05-20T01:15:37.018 INFO:teuthology.orchestra.run.smithi132.stdout: mon warn on no sortbitwise = false 2023-05-20T01:15:37.018 INFO:teuthology.orchestra.run.smithi132.stdout: mon warn on osd down out interval zero = false 2023-05-20T01:15:37.018 INFO:teuthology.orchestra.run.smithi132.stdout: mon warn on too few osds = false 2023-05-20T01:15:37.018 INFO:teuthology.orchestra.run.smithi132.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-05-20T01:15:37.018 INFO:teuthology.orchestra.run.smithi132.stdout: mon_warn_on_pool_no_redundancy = false 2023-05-20T01:15:37.018 INFO:teuthology.orchestra.run.smithi132.stdout: mon_allow_pool_size_one = true 2023-05-20T01:15:37.019 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:15:37.019 INFO:teuthology.orchestra.run.smithi132.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-05-20T01:15:37.019 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:15:37.019 INFO:teuthology.orchestra.run.smithi132.stdout: osd default data pool replay window = 5 2023-05-20T01:15:37.019 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:15:37.019 INFO:teuthology.orchestra.run.smithi132.stdout: mon allow pool delete = true 2023-05-20T01:15:37.019 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:15:37.020 INFO:teuthology.orchestra.run.smithi132.stdout: mon cluster log file level = debug 2023-05-20T01:15:37.020 INFO:teuthology.orchestra.run.smithi132.stdout: debug asserts on shutdown = true 2023-05-20T01:15:37.020 INFO:teuthology.orchestra.run.smithi132.stdout: mon health detail to clog = false 2023-05-20T01:15:37.020 INFO:teuthology.orchestra.run.smithi132.stdout: mon host = "172.21.15.16,172.21.15.132,[v2:172.21.15.132:3301,v1:172.21.15.132:6790]" 2023-05-20T01:15:37.020 INFO:teuthology.orchestra.run.smithi132.stdout: ms die on skipped message = False 2023-05-20T01:15:37.020 INFO:teuthology.orchestra.run.smithi132.stdout: fsid = c2245cc5-ecab-4087-a59b-bc81db901676 2023-05-20T01:15:37.021 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:15:37.021 INFO:teuthology.orchestra.run.smithi132.stdout:[osd] 2023-05-20T01:15:37.021 INFO:teuthology.orchestra.run.smithi132.stdout: osd journal size = 100 2023-05-20T01:15:37.021 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:15:37.021 INFO:teuthology.orchestra.run.smithi132.stdout: osd scrub load threshold = 5.0 2023-05-20T01:15:37.021 INFO:teuthology.orchestra.run.smithi132.stdout: osd scrub max interval = 600 2023-05-20T01:15:37.021 INFO:teuthology.orchestra.run.smithi132.stdout: osd mclock profile = high_recovery_ops 2023-05-20T01:15:37.022 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:15:37.022 INFO:teuthology.orchestra.run.smithi132.stdout: osd recover clone overlap = true 2023-05-20T01:15:37.022 INFO:teuthology.orchestra.run.smithi132.stdout: osd recovery max chunk = 1048576 2023-05-20T01:15:37.022 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:15:37.022 INFO:teuthology.orchestra.run.smithi132.stdout: osd debug shutdown = true 2023-05-20T01:15:37.022 INFO:teuthology.orchestra.run.smithi132.stdout: osd debug op order = true 2023-05-20T01:15:37.022 INFO:teuthology.orchestra.run.smithi132.stdout: osd debug verify stray on activate = true 2023-05-20T01:15:37.023 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:15:37.023 INFO:teuthology.orchestra.run.smithi132.stdout: osd open classes on start = true 2023-05-20T01:15:37.023 INFO:teuthology.orchestra.run.smithi132.stdout: osd debug pg log writeout = true 2023-05-20T01:15:37.023 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:15:37.023 INFO:teuthology.orchestra.run.smithi132.stdout: osd deep scrub update digest min age = 30 2023-05-20T01:15:37.023 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:15:37.023 INFO:teuthology.orchestra.run.smithi132.stdout: osd map max advance = 10 2023-05-20T01:15:37.024 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:15:37.024 INFO:teuthology.orchestra.run.smithi132.stdout: journal zero on create = true 2023-05-20T01:15:37.024 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:15:37.024 INFO:teuthology.orchestra.run.smithi132.stdout: filestore ondisk finisher threads = 3 2023-05-20T01:15:37.024 INFO:teuthology.orchestra.run.smithi132.stdout: filestore apply finisher threads = 3 2023-05-20T01:15:37.024 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:15:37.025 INFO:teuthology.orchestra.run.smithi132.stdout: bdev debug aio = true 2023-05-20T01:15:37.025 INFO:teuthology.orchestra.run.smithi132.stdout: osd debug misdirected ops = true 2023-05-20T01:15:37.025 INFO:teuthology.orchestra.run.smithi132.stdout: bluestore block size = 96636764160 2023-05-20T01:15:37.025 INFO:teuthology.orchestra.run.smithi132.stdout: bluestore fsck on mount = True 2023-05-20T01:15:37.025 INFO:teuthology.orchestra.run.smithi132.stdout: debug bluefs = 20 2023-05-20T01:15:37.026 INFO:teuthology.orchestra.run.smithi132.stdout: debug bluestore = 20 2023-05-20T01:15:37.026 INFO:teuthology.orchestra.run.smithi132.stdout: debug ms = 1 2023-05-20T01:15:37.026 INFO:teuthology.orchestra.run.smithi132.stdout: debug osd = 20 2023-05-20T01:15:37.026 INFO:teuthology.orchestra.run.smithi132.stdout: debug rocksdb = 10 2023-05-20T01:15:37.026 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd backfillfull_ratio = 0.85 2023-05-20T01:15:37.026 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd full ratio = 0.9 2023-05-20T01:15:37.027 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd nearfull ratio = 0.8 2023-05-20T01:15:37.027 INFO:teuthology.orchestra.run.smithi132.stdout: osd failsafe full ratio = 0.95 2023-05-20T01:15:37.027 INFO:teuthology.orchestra.run.smithi132.stdout: osd objectstore = bluestore 2023-05-20T01:15:37.027 INFO:teuthology.orchestra.run.smithi132.stdout: osd op complaint time = 180 2023-05-20T01:15:37.027 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:15:37.027 INFO:teuthology.orchestra.run.smithi132.stdout:[mgr] 2023-05-20T01:15:37.027 INFO:teuthology.orchestra.run.smithi132.stdout: debug ms = 1 2023-05-20T01:15:37.028 INFO:teuthology.orchestra.run.smithi132.stdout: debug mgr = 20 2023-05-20T01:15:37.028 INFO:teuthology.orchestra.run.smithi132.stdout: debug mon = 20 2023-05-20T01:15:37.028 INFO:teuthology.orchestra.run.smithi132.stdout: debug auth = 20 2023-05-20T01:15:37.028 INFO:teuthology.orchestra.run.smithi132.stdout: mon reweight min pgs per osd = 4 2023-05-20T01:15:37.028 INFO:teuthology.orchestra.run.smithi132.stdout: mon reweight min bytes per osd = 10 2023-05-20T01:15:37.028 INFO:teuthology.orchestra.run.smithi132.stdout: mgr/telemetry/nag = false 2023-05-20T01:15:37.029 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:15:37.029 INFO:teuthology.orchestra.run.smithi132.stdout:[mon] 2023-05-20T01:15:37.029 INFO:teuthology.orchestra.run.smithi132.stdout: debug ms = 1 2023-05-20T01:15:37.029 INFO:teuthology.orchestra.run.smithi132.stdout: debug mon = 20 2023-05-20T01:15:37.029 INFO:teuthology.orchestra.run.smithi132.stdout: debug paxos = 20 2023-05-20T01:15:37.029 INFO:teuthology.orchestra.run.smithi132.stdout: debug auth = 20 2023-05-20T01:15:37.029 INFO:teuthology.orchestra.run.smithi132.stdout: mon data avail warn = 5 2023-05-20T01:15:37.030 INFO:teuthology.orchestra.run.smithi132.stdout: mon mgr mkfs grace = 240 2023-05-20T01:15:37.030 INFO:teuthology.orchestra.run.smithi132.stdout: mon reweight min pgs per osd = 4 2023-05-20T01:15:37.030 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd reporter subtree level = osd 2023-05-20T01:15:37.030 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd prime pg temp = true 2023-05-20T01:15:37.030 INFO:teuthology.orchestra.run.smithi132.stdout: mon reweight min bytes per osd = 10 2023-05-20T01:15:37.030 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:15:37.030 INFO:teuthology.orchestra.run.smithi132.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-05-20T01:15:37.031 INFO:teuthology.orchestra.run.smithi132.stdout: auth mon ticket ttl = 660 # 11m 2023-05-20T01:15:37.031 INFO:teuthology.orchestra.run.smithi132.stdout: auth service ticket ttl = 240 # 4m 2023-05-20T01:15:37.031 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:15:37.032 INFO:teuthology.orchestra.run.smithi132.stdout: # don't complain about insecure global_id in the test suite 2023-05-20T01:15:37.032 INFO:teuthology.orchestra.run.smithi132.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-05-20T01:15:37.032 INFO:teuthology.orchestra.run.smithi132.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-05-20T01:15:37.033 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:15:37.033 INFO:teuthology.orchestra.run.smithi132.stdout: # 1m isn't quite enough 2023-05-20T01:15:37.033 INFO:teuthology.orchestra.run.smithi132.stdout: mon_down_mkfs_grace = 2m 2023-05-20T01:15:37.033 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:15:37.033 INFO:teuthology.orchestra.run.smithi132.stdout: mon_warn_on_filestore_osds = false 2023-05-20T01:15:37.034 INFO:teuthology.orchestra.run.smithi132.stdout: mon op complaint time = 120 2023-05-20T01:15:37.034 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:15:37.034 INFO:teuthology.orchestra.run.smithi132.stdout:[client] 2023-05-20T01:15:37.035 INFO:teuthology.orchestra.run.smithi132.stdout: rgw cache enabled = true 2023-05-20T01:15:37.035 INFO:teuthology.orchestra.run.smithi132.stdout: rgw enable ops log = true 2023-05-20T01:15:37.035 INFO:teuthology.orchestra.run.smithi132.stdout: rgw enable usage log = true 2023-05-20T01:15:37.036 INFO:teuthology.orchestra.run.smithi132.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-05-20T01:15:37.036 INFO:teuthology.orchestra.run.smithi132.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-05-20T01:15:37.036 INFO:teuthology.orchestra.run.smithi132.stdout: client mount timeout = 600 2023-05-20T01:15:37.037 INFO:teuthology.orchestra.run.smithi132.stdout: debug client = 20 2023-05-20T01:15:37.037 INFO:teuthology.orchestra.run.smithi132.stdout: debug ms = 1 2023-05-20T01:15:37.037 INFO:teuthology.orchestra.run.smithi132.stdout: pid file = "" 2023-05-20T01:15:37.037 INFO:teuthology.orchestra.run.smithi132.stdout: rados mon op timeout = 900 2023-05-20T01:15:37.038 INFO:teuthology.orchestra.run.smithi132.stdout: rados osd op timeout = 900 2023-05-20T01:15:37.038 INFO:teuthology.orchestra.run.smithi132.stdout:[mon.a] 2023-05-20T01:15:37.038 INFO:teuthology.orchestra.run.smithi132.stdout:[mon.b] 2023-05-20T01:15:37.038 INFO:teuthology.orchestra.run.smithi132.stdout:[mon.c] 2023-05-20T01:15:37.039 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.a] 2023-05-20T01:15:37.039 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.c] 2023-05-20T01:15:37.039 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.b] 2023-05-20T01:15:37.039 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.d] 2023-05-20T01:15:37.040 INFO:teuthology.orchestra.run.smithi132.stdout:[mds] 2023-05-20T01:15:37.040 INFO:teuthology.orchestra.run.smithi132.stdout: debug mds = 20 2023-05-20T01:15:37.040 INFO:teuthology.orchestra.run.smithi132.stdout: debug mds balancer = 20 2023-05-20T01:15:37.041 INFO:teuthology.orchestra.run.smithi132.stdout: debug ms = 1 2023-05-20T01:15:37.041 INFO:teuthology.orchestra.run.smithi132.stdout: mds debug frag = True 2023-05-20T01:15:37.041 INFO:teuthology.orchestra.run.smithi132.stdout: mds debug scatterstat = True 2023-05-20T01:15:37.041 INFO:teuthology.orchestra.run.smithi132.stdout: mds op complaint time = 180 2023-05-20T01:15:37.041 INFO:teuthology.orchestra.run.smithi132.stdout: mds verify scatter = True 2023-05-20T01:15:37.042 INFO:teuthology.orchestra.run.smithi132.stdout: osd op complaint time = 180 2023-05-20T01:15:37.042 INFO:teuthology.orchestra.run.smithi132.stdout: rados mon op timeout = 900 2023-05-20T01:15:37.042 INFO:teuthology.orchestra.run.smithi132.stdout: rados osd op timeout = 900 2023-05-20T01:15:37.042 INFO:tasks.ceph:Creating admin key on mon.a... 2023-05-20T01:15:37.043 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-05-20T01:15:37.110 INFO:tasks.ceph:Copying monmap to all nodes... 2023-05-20T01:15:37.111 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-05-20T01:15:37.111 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-05-20T01:15:37.129 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-05-20T01:15:37.131 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-05-20T01:15:37.186 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi016.front.sepia.ceph.com 2023-05-20T01:15:37.186 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-05-20T01:15:37.186 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/ceph/ceph.keyring 2023-05-20T01:15:37.187 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-05-20T01:15:37.263 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-05-20T01:15:37.264 DEBUG:teuthology.orchestra.run.smithi016:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-05-20T01:15:37.320 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi132.front.sepia.ceph.com 2023-05-20T01:15:37.322 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2023-05-20T01:15:37.322 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd of=/etc/ceph/ceph.keyring 2023-05-20T01:15:37.323 DEBUG:teuthology.orchestra.run.smithi132:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-05-20T01:15:37.362 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2023-05-20T01:15:37.363 DEBUG:teuthology.orchestra.run.smithi132:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-05-20T01:15:37.420 INFO:tasks.ceph:Setting up mon nodes... 2023-05-20T01:15:37.420 INFO:tasks.ceph:Setting up mgr nodes... 2023-05-20T01:15:37.421 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-05-20T01:15:37.475 INFO:teuthology.orchestra.run.smithi016.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-05-20T01:15:37.478 DEBUG:teuthology.orchestra.run.smithi132:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-05-20T01:15:37.552 INFO:teuthology.orchestra.run.smithi132.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-05-20T01:15:37.555 INFO:tasks.ceph:Setting up mds nodes... 2023-05-20T01:15:37.555 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-05-20T01:15:37.610 INFO:teuthology.orchestra.run.smithi016.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-05-20T01:15:37.614 DEBUG:teuthology.orchestra.run.smithi016:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-05-20T01:15:37.641 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-05-20T01:15:37.735 INFO:teuthology.orchestra.run.smithi016.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-05-20T01:15:37.738 DEBUG:teuthology.orchestra.run.smithi016:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-05-20T01:15:37.765 DEBUG:teuthology.orchestra.run.smithi132:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-05-20T01:15:37.821 INFO:teuthology.orchestra.run.smithi132.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-05-20T01:15:37.823 DEBUG:teuthology.orchestra.run.smithi132:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-05-20T01:15:37.849 DEBUG:teuthology.orchestra.run.smithi132:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-05-20T01:15:37.942 INFO:teuthology.orchestra.run.smithi132.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-05-20T01:15:37.945 DEBUG:teuthology.orchestra.run.smithi132:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-05-20T01:15:37.970 INFO:tasks.ceph_client:Setting up client nodes... 2023-05-20T01:15:37.971 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-05-20T01:15:38.015 INFO:teuthology.orchestra.run.smithi016.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-05-20T01:15:38.028 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2023-05-20T01:15:38.072 INFO:teuthology.orchestra.run.smithi016.stdout:creating /etc/ceph/ceph.client.2.keyring 2023-05-20T01:15:38.086 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-05-20T01:15:38.129 INFO:teuthology.orchestra.run.smithi132.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-05-20T01:15:38.142 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2023-05-20T01:15:38.224 INFO:teuthology.orchestra.run.smithi132.stdout:creating /etc/ceph/ceph.client.3.keyring 2023-05-20T01:15:38.237 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-05-20T01:15:38.237 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi016.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi132.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-05-20T01:15:38.237 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-05-20T01:15:38.264 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-05-20T01:15:38.265 INFO:tasks.ceph:role: osd.0 2023-05-20T01:15:38.265 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi016.front.sepia.ceph.com 2023-05-20T01:15:38.265 DEBUG:teuthology.orchestra.run.smithi016:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-05-20T01:15:38.332 INFO:teuthology.orchestra.run.smithi016.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-05-20T01:15:38.332 INFO:teuthology.orchestra.run.smithi016.stdout: = sectsz=512 attr=2, projid32bit=1 2023-05-20T01:15:38.332 INFO:teuthology.orchestra.run.smithi016.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-05-20T01:15:38.333 INFO:teuthology.orchestra.run.smithi016.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-05-20T01:15:38.333 INFO:teuthology.orchestra.run.smithi016.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-05-20T01:15:38.333 INFO:teuthology.orchestra.run.smithi016.stdout: = sunit=0 swidth=0 blks 2023-05-20T01:15:38.333 INFO:teuthology.orchestra.run.smithi016.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-05-20T01:15:38.333 INFO:teuthology.orchestra.run.smithi016.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-05-20T01:15:38.333 INFO:teuthology.orchestra.run.smithi016.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-05-20T01:15:38.333 INFO:teuthology.orchestra.run.smithi016.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-05-20T01:15:38.359 INFO:teuthology.orchestra.run.smithi016.stdout:Discarding blocks...Done. 2023-05-20T01:15:38.361 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi016.front.sepia.ceph.com -o noatime 2023-05-20T01:15:38.361 DEBUG:teuthology.orchestra.run.smithi016:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-05-20T01:15:38.442 DEBUG:teuthology.orchestra.run.smithi016:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-05-20T01:15:38.513 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-05-20T01:15:38.581 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-05-20T01:15:38.581 INFO:tasks.ceph:role: osd.1 2023-05-20T01:15:38.581 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi016.front.sepia.ceph.com 2023-05-20T01:15:38.582 DEBUG:teuthology.orchestra.run.smithi016:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-05-20T01:15:38.648 INFO:teuthology.orchestra.run.smithi016.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-05-20T01:15:38.648 INFO:teuthology.orchestra.run.smithi016.stdout: = sectsz=512 attr=2, projid32bit=1 2023-05-20T01:15:38.648 INFO:teuthology.orchestra.run.smithi016.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-05-20T01:15:38.649 INFO:teuthology.orchestra.run.smithi016.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-05-20T01:15:38.649 INFO:teuthology.orchestra.run.smithi016.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-05-20T01:15:38.649 INFO:teuthology.orchestra.run.smithi016.stdout: = sunit=0 swidth=0 blks 2023-05-20T01:15:38.649 INFO:teuthology.orchestra.run.smithi016.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-05-20T01:15:38.649 INFO:teuthology.orchestra.run.smithi016.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-05-20T01:15:38.649 INFO:teuthology.orchestra.run.smithi016.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-05-20T01:15:38.650 INFO:teuthology.orchestra.run.smithi016.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-05-20T01:15:38.675 INFO:teuthology.orchestra.run.smithi016.stdout:Discarding blocks...Done. 2023-05-20T01:15:38.677 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi016.front.sepia.ceph.com -o noatime 2023-05-20T01:15:38.677 DEBUG:teuthology.orchestra.run.smithi016:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-05-20T01:15:38.757 DEBUG:teuthology.orchestra.run.smithi016:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-05-20T01:15:38.830 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-05-20T01:15:38.897 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-05-20T01:15:38.897 INFO:tasks.ceph:role: osd.2 2023-05-20T01:15:38.898 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi016.front.sepia.ceph.com 2023-05-20T01:15:38.898 DEBUG:teuthology.orchestra.run.smithi016:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-05-20T01:15:38.963 INFO:teuthology.orchestra.run.smithi016.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-05-20T01:15:38.964 INFO:teuthology.orchestra.run.smithi016.stdout: = sectsz=512 attr=2, projid32bit=1 2023-05-20T01:15:38.964 INFO:teuthology.orchestra.run.smithi016.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-05-20T01:15:38.964 INFO:teuthology.orchestra.run.smithi016.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-05-20T01:15:38.964 INFO:teuthology.orchestra.run.smithi016.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-05-20T01:15:38.964 INFO:teuthology.orchestra.run.smithi016.stdout: = sunit=0 swidth=0 blks 2023-05-20T01:15:38.964 INFO:teuthology.orchestra.run.smithi016.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-05-20T01:15:38.965 INFO:teuthology.orchestra.run.smithi016.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-05-20T01:15:38.965 INFO:teuthology.orchestra.run.smithi016.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-05-20T01:15:38.965 INFO:teuthology.orchestra.run.smithi016.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-05-20T01:15:38.993 INFO:teuthology.orchestra.run.smithi016.stdout:Discarding blocks...Done. 2023-05-20T01:15:38.994 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi016.front.sepia.ceph.com -o noatime 2023-05-20T01:15:38.994 DEBUG:teuthology.orchestra.run.smithi016:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-05-20T01:15:39.074 DEBUG:teuthology.orchestra.run.smithi016:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-05-20T01:15:39.146 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-05-20T01:15:39.214 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-05-20T01:15:39.215 INFO:tasks.ceph:role: osd.3 2023-05-20T01:15:39.215 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi016.front.sepia.ceph.com 2023-05-20T01:15:39.216 DEBUG:teuthology.orchestra.run.smithi016:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-05-20T01:15:39.281 INFO:teuthology.orchestra.run.smithi016.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-05-20T01:15:39.281 INFO:teuthology.orchestra.run.smithi016.stdout: = sectsz=512 attr=2, projid32bit=1 2023-05-20T01:15:39.282 INFO:teuthology.orchestra.run.smithi016.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-05-20T01:15:39.282 INFO:teuthology.orchestra.run.smithi016.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-05-20T01:15:39.282 INFO:teuthology.orchestra.run.smithi016.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-05-20T01:15:39.282 INFO:teuthology.orchestra.run.smithi016.stdout: = sunit=0 swidth=0 blks 2023-05-20T01:15:39.282 INFO:teuthology.orchestra.run.smithi016.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-05-20T01:15:39.282 INFO:teuthology.orchestra.run.smithi016.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-05-20T01:15:39.282 INFO:teuthology.orchestra.run.smithi016.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-05-20T01:15:39.283 INFO:teuthology.orchestra.run.smithi016.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-05-20T01:15:39.308 INFO:teuthology.orchestra.run.smithi016.stdout:Discarding blocks...Done. 2023-05-20T01:15:39.310 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi016.front.sepia.ceph.com -o noatime 2023-05-20T01:15:39.310 DEBUG:teuthology.orchestra.run.smithi016:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-05-20T01:15:39.391 DEBUG:teuthology.orchestra.run.smithi016:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-05-20T01:15:39.462 DEBUG:teuthology.orchestra.run.smithi016:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-05-20T01:15:39.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:15:39.564+0000 7f8e245db640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-05-20T01:15:39.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:15:39.564+0000 7f8e245db640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-05-20T01:15:39.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:15:39.564+0000 7f8e245db640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-05-20T01:15:39.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:15:39.564+0000 7f8e245db640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-05-20T01:15:39.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:15:39.564+0000 7f8e245db640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-05-20T01:15:39.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:15:39.564+0000 7f8e245db640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-05-20T01:15:42.429 DEBUG:teuthology.orchestra.run.smithi016:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-05-20T01:15:42.462 DEBUG:teuthology.orchestra.run.smithi016:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-05-20T01:15:42.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:15:42.551+0000 7f93366f7640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-05-20T01:15:42.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:15:42.552+0000 7f93366f7640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-05-20T01:15:42.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:15:42.552+0000 7f93366f7640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-05-20T01:15:42.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:15:42.552+0000 7f93366f7640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-05-20T01:15:42.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:15:42.552+0000 7f93366f7640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-05-20T01:15:42.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:15:42.552+0000 7f93366f7640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-05-20T01:15:45.416 DEBUG:teuthology.orchestra.run.smithi016:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-05-20T01:15:45.449 DEBUG:teuthology.orchestra.run.smithi016:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-05-20T01:15:45.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:15:45.539+0000 7f52eca53640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-05-20T01:15:45.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:15:45.539+0000 7f52eca53640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-05-20T01:15:45.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:15:45.539+0000 7f52eca53640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-05-20T01:15:45.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:15:45.539+0000 7f52eca53640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-05-20T01:15:45.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:15:45.539+0000 7f52eca53640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-05-20T01:15:45.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:15:45.540+0000 7f52eca53640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-05-20T01:15:48.411 DEBUG:teuthology.orchestra.run.smithi016:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-05-20T01:15:48.445 DEBUG:teuthology.orchestra.run.smithi016:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-05-20T01:15:48.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:15:48.535+0000 7f98b28f2640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-05-20T01:15:48.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:15:48.535+0000 7f98b28f2640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-05-20T01:15:48.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:15:48.535+0000 7f98b28f2640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-05-20T01:15:48.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:15:48.535+0000 7f98b28f2640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-05-20T01:15:48.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:15:48.535+0000 7f98b28f2640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-05-20T01:15:48.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:15:48.536+0000 7f98b28f2640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-05-20T01:15:51.395 DEBUG:teuthology.orchestra.run.smithi016:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-05-20T01:15:51.430 DEBUG:teuthology.orchestra.run.smithi132:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-05-20T01:15:51.463 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-05-20T01:15:51.463 INFO:tasks.ceph:role: osd.4 2023-05-20T01:15:51.463 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi132.front.sepia.ceph.com 2023-05-20T01:15:51.464 DEBUG:teuthology.orchestra.run.smithi132:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-05-20T01:15:51.543 INFO:teuthology.orchestra.run.smithi132.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-05-20T01:15:51.543 INFO:teuthology.orchestra.run.smithi132.stdout: = sectsz=512 attr=2, projid32bit=1 2023-05-20T01:15:51.544 INFO:teuthology.orchestra.run.smithi132.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-05-20T01:15:51.544 INFO:teuthology.orchestra.run.smithi132.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-05-20T01:15:51.544 INFO:teuthology.orchestra.run.smithi132.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-05-20T01:15:51.544 INFO:teuthology.orchestra.run.smithi132.stdout: = sunit=0 swidth=0 blks 2023-05-20T01:15:51.545 INFO:teuthology.orchestra.run.smithi132.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-05-20T01:15:51.545 INFO:teuthology.orchestra.run.smithi132.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-05-20T01:15:51.546 INFO:teuthology.orchestra.run.smithi132.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-05-20T01:15:51.546 INFO:teuthology.orchestra.run.smithi132.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-05-20T01:15:51.974 INFO:teuthology.orchestra.run.smithi132.stdout:Discarding blocks...Done. 2023-05-20T01:15:51.975 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi132.front.sepia.ceph.com -o noatime 2023-05-20T01:15:51.976 DEBUG:teuthology.orchestra.run.smithi132:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-05-20T01:15:52.161 DEBUG:teuthology.orchestra.run.smithi132:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-05-20T01:15:52.199 DEBUG:teuthology.orchestra.run.smithi132:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-05-20T01:15:52.269 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-05-20T01:15:52.270 INFO:tasks.ceph:role: osd.5 2023-05-20T01:15:52.270 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi132.front.sepia.ceph.com 2023-05-20T01:15:52.270 DEBUG:teuthology.orchestra.run.smithi132:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-05-20T01:15:52.345 INFO:teuthology.orchestra.run.smithi132.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-05-20T01:15:52.345 INFO:teuthology.orchestra.run.smithi132.stdout: = sectsz=512 attr=2, projid32bit=1 2023-05-20T01:15:52.346 INFO:teuthology.orchestra.run.smithi132.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-05-20T01:15:52.346 INFO:teuthology.orchestra.run.smithi132.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-05-20T01:15:52.346 INFO:teuthology.orchestra.run.smithi132.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-05-20T01:15:52.346 INFO:teuthology.orchestra.run.smithi132.stdout: = sunit=0 swidth=0 blks 2023-05-20T01:15:52.347 INFO:teuthology.orchestra.run.smithi132.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-05-20T01:15:52.347 INFO:teuthology.orchestra.run.smithi132.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-05-20T01:15:52.347 INFO:teuthology.orchestra.run.smithi132.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-05-20T01:15:52.348 INFO:teuthology.orchestra.run.smithi132.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-05-20T01:15:52.760 INFO:teuthology.orchestra.run.smithi132.stdout:Discarding blocks...Done. 2023-05-20T01:15:52.762 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi132.front.sepia.ceph.com -o noatime 2023-05-20T01:15:52.762 DEBUG:teuthology.orchestra.run.smithi132:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-05-20T01:15:52.872 DEBUG:teuthology.orchestra.run.smithi132:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-05-20T01:15:52.904 DEBUG:teuthology.orchestra.run.smithi132:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-05-20T01:15:52.973 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-05-20T01:15:52.973 INFO:tasks.ceph:role: osd.6 2023-05-20T01:15:52.974 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi132.front.sepia.ceph.com 2023-05-20T01:15:52.974 DEBUG:teuthology.orchestra.run.smithi132:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-05-20T01:15:53.047 INFO:teuthology.orchestra.run.smithi132.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-05-20T01:15:53.047 INFO:teuthology.orchestra.run.smithi132.stdout: = sectsz=512 attr=2, projid32bit=1 2023-05-20T01:15:53.048 INFO:teuthology.orchestra.run.smithi132.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-05-20T01:15:53.048 INFO:teuthology.orchestra.run.smithi132.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-05-20T01:15:53.048 INFO:teuthology.orchestra.run.smithi132.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-05-20T01:15:53.048 INFO:teuthology.orchestra.run.smithi132.stdout: = sunit=0 swidth=0 blks 2023-05-20T01:15:53.048 INFO:teuthology.orchestra.run.smithi132.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-05-20T01:15:53.048 INFO:teuthology.orchestra.run.smithi132.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-05-20T01:15:53.048 INFO:teuthology.orchestra.run.smithi132.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-05-20T01:15:53.049 INFO:teuthology.orchestra.run.smithi132.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-05-20T01:15:53.491 INFO:teuthology.orchestra.run.smithi132.stdout:Discarding blocks...Done. 2023-05-20T01:15:53.493 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi132.front.sepia.ceph.com -o noatime 2023-05-20T01:15:53.493 DEBUG:teuthology.orchestra.run.smithi132:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-05-20T01:15:53.651 DEBUG:teuthology.orchestra.run.smithi132:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-05-20T01:15:53.685 DEBUG:teuthology.orchestra.run.smithi132:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-05-20T01:15:53.753 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-05-20T01:15:53.753 INFO:tasks.ceph:role: osd.7 2023-05-20T01:15:53.755 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi132.front.sepia.ceph.com 2023-05-20T01:15:53.755 DEBUG:teuthology.orchestra.run.smithi132:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-05-20T01:15:53.826 INFO:teuthology.orchestra.run.smithi132.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-05-20T01:15:53.827 INFO:teuthology.orchestra.run.smithi132.stdout: = sectsz=512 attr=2, projid32bit=1 2023-05-20T01:15:53.827 INFO:teuthology.orchestra.run.smithi132.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-05-20T01:15:53.827 INFO:teuthology.orchestra.run.smithi132.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-05-20T01:15:53.827 INFO:teuthology.orchestra.run.smithi132.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-05-20T01:15:53.827 INFO:teuthology.orchestra.run.smithi132.stdout: = sunit=0 swidth=0 blks 2023-05-20T01:15:53.827 INFO:teuthology.orchestra.run.smithi132.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-05-20T01:15:53.827 INFO:teuthology.orchestra.run.smithi132.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-05-20T01:15:53.827 INFO:teuthology.orchestra.run.smithi132.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-05-20T01:15:53.828 INFO:teuthology.orchestra.run.smithi132.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-05-20T01:15:54.280 INFO:teuthology.orchestra.run.smithi132.stdout:Discarding blocks...Done. 2023-05-20T01:15:54.282 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi132.front.sepia.ceph.com -o noatime 2023-05-20T01:15:54.283 DEBUG:teuthology.orchestra.run.smithi132:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-05-20T01:15:54.469 DEBUG:teuthology.orchestra.run.smithi132:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-05-20T01:15:54.502 DEBUG:teuthology.orchestra.run.smithi132:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-05-20T01:15:54.619 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:15:54.614+0000 7f4b9d709640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-05-20T01:15:54.619 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:15:54.614+0000 7f4b9d709640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-05-20T01:15:54.619 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:15:54.614+0000 7f4b9d709640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-05-20T01:15:54.619 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:15:54.614+0000 7f4b9d709640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-05-20T01:15:54.619 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:15:54.614+0000 7f4b9d709640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-05-20T01:15:54.620 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:15:54.615+0000 7f4b9d709640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-05-20T01:15:57.487 DEBUG:teuthology.orchestra.run.smithi132:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-05-20T01:15:57.520 DEBUG:teuthology.orchestra.run.smithi132:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-05-20T01:15:57.610 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:15:57.605+0000 7fb28fb94640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-05-20T01:15:57.610 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:15:57.606+0000 7fb28fb94640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-05-20T01:15:57.611 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:15:57.606+0000 7fb28fb94640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-05-20T01:15:57.611 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:15:57.606+0000 7fb28fb94640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-05-20T01:15:57.611 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:15:57.606+0000 7fb28fb94640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-05-20T01:15:57.611 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:15:57.606+0000 7fb28fb94640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-05-20T01:16:00.477 DEBUG:teuthology.orchestra.run.smithi132:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-05-20T01:16:00.510 DEBUG:teuthology.orchestra.run.smithi132:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-05-20T01:16:00.602 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:00.598+0000 7fa9a47da640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-05-20T01:16:00.602 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:00.598+0000 7fa9a47da640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-05-20T01:16:00.603 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:00.598+0000 7fa9a47da640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-05-20T01:16:00.603 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:00.598+0000 7fa9a47da640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-05-20T01:16:00.603 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:00.598+0000 7fa9a47da640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-05-20T01:16:00.603 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:00.599+0000 7fa9a47da640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-05-20T01:16:03.469 DEBUG:teuthology.orchestra.run.smithi132:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-05-20T01:16:03.501 DEBUG:teuthology.orchestra.run.smithi132:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-05-20T01:16:03.591 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:03.587+0000 7fc99525b640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-05-20T01:16:03.592 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:03.587+0000 7fc99525b640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-05-20T01:16:03.592 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:03.587+0000 7fc99525b640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-05-20T01:16:03.592 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:03.587+0000 7fc99525b640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-05-20T01:16:03.592 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:03.587+0000 7fc99525b640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-05-20T01:16:03.593 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:03.588+0000 7fc99525b640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-05-20T01:16:04.896 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:16:04.899 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:16:06.447 DEBUG:teuthology.orchestra.run.smithi132:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-05-20T01:16:06.478 INFO:tasks.ceph:Reading keys from all nodes... 2023-05-20T01:16:06.478 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-05-20T01:16:06.478 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-05-20T01:16:06.512 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-05-20T01:16:06.512 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-05-20T01:16:06.583 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-05-20T01:16:06.584 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-05-20T01:16:06.653 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-05-20T01:16:06.653 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-05-20T01:16:06.722 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-05-20T01:16:06.723 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-05-20T01:16:06.791 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-05-20T01:16:06.792 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-05-20T01:16:06.859 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-05-20T01:16:06.860 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-05-20T01:16:06.927 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2023-05-20T01:16:06.927 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-05-20T01:16:06.960 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2023-05-20T01:16:06.960 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-05-20T01:16:07.030 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2023-05-20T01:16:07.031 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-05-20T01:16:07.098 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2023-05-20T01:16:07.099 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-05-20T01:16:07.166 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2023-05-20T01:16:07.166 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-05-20T01:16:07.232 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2023-05-20T01:16:07.233 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-05-20T01:16:07.300 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2023-05-20T01:16:07.301 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-05-20T01:16:07.366 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-05-20T01:16:07.366 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-05-20T01:16:07.385 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-05-20T01:16:07.385 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2023-05-20T01:16:07.444 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2023-05-20T01:16:07.444 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-05-20T01:16:07.460 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2023-05-20T01:16:07.461 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2023-05-20T01:16:07.517 INFO:tasks.ceph:Adding keys to all mons... 2023-05-20T01:16:07.518 DEBUG:teuthology.orchestra.run.smithi016:> sudo tee -a /etc/ceph/ceph.keyring 2023-05-20T01:16:07.520 DEBUG:teuthology.orchestra.run.smithi132:> sudo tee -a /etc/ceph/ceph.keyring 2023-05-20T01:16:07.560 INFO:teuthology.orchestra.run.smithi016.stdout:[mgr.y] 2023-05-20T01:16:07.561 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQA5H2hkBQg7HBAAbOJZ+55rRACVBMd2eyjqtg== 2023-05-20T01:16:07.561 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.a] 2023-05-20T01:16:07.561 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQA5H2hkq5FEJBAA1qUUPSLmwo43hTQS7GSENA== 2023-05-20T01:16:07.561 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.c] 2023-05-20T01:16:07.561 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQA5H2hkBVK8KxAAgxy9+gbttxZeygwrAX+PHQ== 2023-05-20T01:16:07.561 INFO:teuthology.orchestra.run.smithi016.stdout:[osd.0] 2023-05-20T01:16:07.561 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQA7H2hkC4WwIRAA5KMQFqldcekzEuH92rZYHA== 2023-05-20T01:16:07.562 INFO:teuthology.orchestra.run.smithi016.stdout:[osd.1] 2023-05-20T01:16:07.562 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQA+H2hkvMH1IBAAhOQOjCiv3r2Pl3HcSS6eAA== 2023-05-20T01:16:07.562 INFO:teuthology.orchestra.run.smithi016.stdout:[osd.2] 2023-05-20T01:16:07.562 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQBBH2hkSJU5IBAAdFd4IhXxISVybFyT24gb0g== 2023-05-20T01:16:07.562 INFO:teuthology.orchestra.run.smithi016.stdout:[osd.3] 2023-05-20T01:16:07.562 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQBEH2hkrcH8HxAANlLWDyIYycLl1TrYCBN1hQ== 2023-05-20T01:16:07.562 INFO:teuthology.orchestra.run.smithi016.stdout:[mgr.x] 2023-05-20T01:16:07.563 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQA5H2hksQbMIBAAQFm/pY5stJHNudg6zcPBdQ== 2023-05-20T01:16:07.563 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.b] 2023-05-20T01:16:07.563 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQA5H2hk/2XSMBAAnGCIm7TJZsO9HyXwOWxO7w== 2023-05-20T01:16:07.563 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.d] 2023-05-20T01:16:07.563 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQA5H2hkFxEMOBAAMPvue7F+/qqOFUD/C//U+g== 2023-05-20T01:16:07.563 INFO:teuthology.orchestra.run.smithi016.stdout:[osd.4] 2023-05-20T01:16:07.563 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQBKH2hkZr+4JBAAzbzKd4ulGI8kVGqXpjCDog== 2023-05-20T01:16:07.564 INFO:teuthology.orchestra.run.smithi016.stdout:[osd.5] 2023-05-20T01:16:07.564 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQBNH2hkNV02JBAALrC7eOTnD1dQM9njk+BgLQ== 2023-05-20T01:16:07.564 INFO:teuthology.orchestra.run.smithi016.stdout:[osd.6] 2023-05-20T01:16:07.564 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQBQH2hkGOfBIxAABCTVPLBi5+drKpqOC8xqdw== 2023-05-20T01:16:07.564 INFO:teuthology.orchestra.run.smithi016.stdout:[osd.7] 2023-05-20T01:16:07.565 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQBTH2hkpwEdIxAAaI3X5ZAVJ9MfVG0MjFJvSQ== 2023-05-20T01:16:07.565 INFO:teuthology.orchestra.run.smithi016.stdout:[client.0] 2023-05-20T01:16:07.565 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQA6H2hkph3EABAA0277JHCqplw1LklDfidPWQ== 2023-05-20T01:16:07.566 INFO:teuthology.orchestra.run.smithi016.stdout:[client.2] 2023-05-20T01:16:07.566 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQA6H2hk+TgtBBAAc4+W2eOVgJfNd3xXjGvz4Q== 2023-05-20T01:16:07.566 INFO:teuthology.orchestra.run.smithi016.stdout:[client.1] 2023-05-20T01:16:07.566 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQA6H2hk43WQBxAApSfBSu1qEaKSGWaYAvz/Jg== 2023-05-20T01:16:07.567 INFO:teuthology.orchestra.run.smithi016.stdout:[client.3] 2023-05-20T01:16:07.567 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQA6H2hkifVADRAAQP6vtdVNYyeNPV0d+agOCw== 2023-05-20T01:16:07.581 INFO:teuthology.orchestra.run.smithi132.stdout:[mgr.y] 2023-05-20T01:16:07.582 INFO:teuthology.orchestra.run.smithi132.stdout: key = AQA5H2hkBQg7HBAAbOJZ+55rRACVBMd2eyjqtg== 2023-05-20T01:16:07.582 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.a] 2023-05-20T01:16:07.582 INFO:teuthology.orchestra.run.smithi132.stdout: key = AQA5H2hkq5FEJBAA1qUUPSLmwo43hTQS7GSENA== 2023-05-20T01:16:07.582 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.c] 2023-05-20T01:16:07.583 INFO:teuthology.orchestra.run.smithi132.stdout: key = AQA5H2hkBVK8KxAAgxy9+gbttxZeygwrAX+PHQ== 2023-05-20T01:16:07.583 INFO:teuthology.orchestra.run.smithi132.stdout:[osd.0] 2023-05-20T01:16:07.583 INFO:teuthology.orchestra.run.smithi132.stdout: key = AQA7H2hkC4WwIRAA5KMQFqldcekzEuH92rZYHA== 2023-05-20T01:16:07.584 INFO:teuthology.orchestra.run.smithi132.stdout:[osd.1] 2023-05-20T01:16:07.584 INFO:teuthology.orchestra.run.smithi132.stdout: key = AQA+H2hkvMH1IBAAhOQOjCiv3r2Pl3HcSS6eAA== 2023-05-20T01:16:07.584 INFO:teuthology.orchestra.run.smithi132.stdout:[osd.2] 2023-05-20T01:16:07.584 INFO:teuthology.orchestra.run.smithi132.stdout: key = AQBBH2hkSJU5IBAAdFd4IhXxISVybFyT24gb0g== 2023-05-20T01:16:07.585 INFO:teuthology.orchestra.run.smithi132.stdout:[osd.3] 2023-05-20T01:16:07.585 INFO:teuthology.orchestra.run.smithi132.stdout: key = AQBEH2hkrcH8HxAANlLWDyIYycLl1TrYCBN1hQ== 2023-05-20T01:16:07.585 INFO:teuthology.orchestra.run.smithi132.stdout:[mgr.x] 2023-05-20T01:16:07.586 INFO:teuthology.orchestra.run.smithi132.stdout: key = AQA5H2hksQbMIBAAQFm/pY5stJHNudg6zcPBdQ== 2023-05-20T01:16:07.586 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.b] 2023-05-20T01:16:07.586 INFO:teuthology.orchestra.run.smithi132.stdout: key = AQA5H2hk/2XSMBAAnGCIm7TJZsO9HyXwOWxO7w== 2023-05-20T01:16:07.587 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.d] 2023-05-20T01:16:07.587 INFO:teuthology.orchestra.run.smithi132.stdout: key = AQA5H2hkFxEMOBAAMPvue7F+/qqOFUD/C//U+g== 2023-05-20T01:16:07.587 INFO:teuthology.orchestra.run.smithi132.stdout:[osd.4] 2023-05-20T01:16:07.587 INFO:teuthology.orchestra.run.smithi132.stdout: key = AQBKH2hkZr+4JBAAzbzKd4ulGI8kVGqXpjCDog== 2023-05-20T01:16:07.588 INFO:teuthology.orchestra.run.smithi132.stdout:[osd.5] 2023-05-20T01:16:07.588 INFO:teuthology.orchestra.run.smithi132.stdout: key = AQBNH2hkNV02JBAALrC7eOTnD1dQM9njk+BgLQ== 2023-05-20T01:16:07.588 INFO:teuthology.orchestra.run.smithi132.stdout:[osd.6] 2023-05-20T01:16:07.588 INFO:teuthology.orchestra.run.smithi132.stdout: key = AQBQH2hkGOfBIxAABCTVPLBi5+drKpqOC8xqdw== 2023-05-20T01:16:07.589 INFO:teuthology.orchestra.run.smithi132.stdout:[osd.7] 2023-05-20T01:16:07.589 INFO:teuthology.orchestra.run.smithi132.stdout: key = AQBTH2hkpwEdIxAAaI3X5ZAVJ9MfVG0MjFJvSQ== 2023-05-20T01:16:07.589 INFO:teuthology.orchestra.run.smithi132.stdout:[client.0] 2023-05-20T01:16:07.590 INFO:teuthology.orchestra.run.smithi132.stdout: key = AQA6H2hkph3EABAA0277JHCqplw1LklDfidPWQ== 2023-05-20T01:16:07.591 INFO:teuthology.orchestra.run.smithi132.stdout:[client.2] 2023-05-20T01:16:07.591 INFO:teuthology.orchestra.run.smithi132.stdout: key = AQA6H2hk+TgtBBAAc4+W2eOVgJfNd3xXjGvz4Q== 2023-05-20T01:16:07.591 INFO:teuthology.orchestra.run.smithi132.stdout:[client.1] 2023-05-20T01:16:07.591 INFO:teuthology.orchestra.run.smithi132.stdout: key = AQA6H2hk43WQBxAApSfBSu1qEaKSGWaYAvz/Jg== 2023-05-20T01:16:07.591 INFO:teuthology.orchestra.run.smithi132.stdout:[client.3] 2023-05-20T01:16:07.591 INFO:teuthology.orchestra.run.smithi132.stdout: key = AQA6H2hkifVADRAAQP6vtdVNYyeNPV0d+agOCw== 2023-05-20T01:16:07.592 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-05-20T01:16:07.610 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-05-20T01:16:07.685 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-05-20T01:16:07.687 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-05-20T01:16:07.744 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-05-20T01:16:07.746 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-05-20T01:16:07.802 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-05-20T01:16:07.804 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-05-20T01:16:07.857 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-05-20T01:16:07.861 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-05-20T01:16:07.914 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-05-20T01:16:07.916 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-05-20T01:16:07.970 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-05-20T01:16:07.974 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-05-20T01:16:08.028 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-05-20T01:16:08.030 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-05-20T01:16:08.083 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-05-20T01:16:08.085 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-05-20T01:16:08.141 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-05-20T01:16:08.143 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-05-20T01:16:08.200 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-05-20T01:16:08.203 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-05-20T01:16:08.256 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-05-20T01:16:08.258 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-05-20T01:16:08.310 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-05-20T01:16:08.312 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-05-20T01:16:08.366 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-05-20T01:16:08.369 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-05-20T01:16:08.423 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-05-20T01:16:08.425 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-05-20T01:16:08.481 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-05-20T01:16:08.484 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-05-20T01:16:08.540 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-05-20T01:16:08.543 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-05-20T01:16:08.598 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-05-20T01:16:08.600 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-05-20T01:16:08.656 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-05-20T01:16:08.657 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-05-20T01:16:08.683 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-05-20T01:16:08.783 DEBUG:teuthology.orchestra.run.smithi016:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-05-20T01:16:08.810 DEBUG:teuthology.orchestra.run.smithi132:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-05-20T01:16:08.837 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-05-20T01:16:08.936 DEBUG:teuthology.orchestra.run.smithi132:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-05-20T01:16:08.962 DEBUG:teuthology.orchestra.run.smithi132:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-05-20T01:16:09.029 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-05-20T01:16:09.127 DEBUG:teuthology.orchestra.run.smithi132:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-05-20T01:16:09.154 DEBUG:teuthology.orchestra.run.smithi016:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-05-20T01:16:09.156 DEBUG:teuthology.orchestra.run.smithi132:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-05-20T01:16:09.210 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-05-20T01:16:09.211 INFO:tasks.ceph.mon.a:Restarting daemon 2023-05-20T01:16:09.211 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-05-20T01:16:09.213 INFO:tasks.ceph.mon.a:Started 2023-05-20T01:16:09.214 INFO:tasks.ceph.mon.b:Restarting daemon 2023-05-20T01:16:09.214 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-05-20T01:16:09.252 INFO:tasks.ceph.mon.b:Started 2023-05-20T01:16:09.253 INFO:tasks.ceph.mon.c:Restarting daemon 2023-05-20T01:16:09.253 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-05-20T01:16:09.256 INFO:tasks.ceph.mon.c:Started 2023-05-20T01:16:09.256 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-05-20T01:16:09.257 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-05-20T01:16:09.257 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-05-20T01:16:09.259 INFO:tasks.ceph.mgr.y:Started 2023-05-20T01:16:09.259 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-05-20T01:16:09.260 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-05-20T01:16:09.261 INFO:tasks.ceph.mgr.x:Started 2023-05-20T01:16:09.262 INFO:tasks.ceph:Setting crush tunables to default 2023-05-20T01:16:09.262 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --cluster ceph osd crush tunables default 2023-05-20T01:16:09.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:09.385+0000 7f38df67c700 1 Processor -- start 2023-05-20T01:16:09.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:09.385+0000 7f38df67c700 1 -- start start 2023-05-20T01:16:09.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:09.386+0000 7f38df67c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f38d8159b50 0x7f38d806e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:09.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:09.386+0000 7f38df67c700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f38d80717c0 0x7f38d80ca870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:09.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:09.386+0000 7f38df67c700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38d80ccd30 con 0x7f38d80cd1c0 2023-05-20T01:16:09.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:09.386+0000 7f38df67c700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38d80ce990 con 0x7f38d80717c0 2023-05-20T01:16:09.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:09.386+0000 7f38df67c700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38d80cc6e0 con 0x7f38d8159b50 2023-05-20T01:16:09.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:09.386+0000 7f38dcc17700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f38d80cd1c0 0x7f38d80710b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:40146/0 (socket says 172.21.15.16:40146) 2023-05-20T01:16:09.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:09.386+0000 7f38dcc17700 1 -- 172.21.15.16:0/652152058 learned_addr learned my addr 172.21.15.16:0/652152058 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:16:09.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:09.386+0000 7f38ddc19700 1 --2- 172.21.15.16:0/652152058 >> v2:172.21.15.132:3300/0 conn(0x7f38d80717c0 0x7f38d80ca870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:09.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:09.386+0000 7f38dd418700 1 --2- 172.21.15.16:0/652152058 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f38d8159b50 0x7f38d806e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:09.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:09.386+0000 7f38ddc19700 1 -- 172.21.15.16:0/652152058 >> v2:172.21.15.132:3300/0 conn(0x7f38d80717c0 msgr2=0x7f38d80ca870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-05-20T01:16:09.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:09.386+0000 7f38ddc19700 1 -- 172.21.15.16:0/652152058 >> v2:172.21.15.132:3300/0 conn(0x7f38d80717c0 msgr2=0x7f38d80ca870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-05-20T01:16:09.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:09.386+0000 7f38ddc19700 1 --2- 172.21.15.16:0/652152058 >> v2:172.21.15.132:3300/0 conn(0x7f38d80717c0 0x7f38d80ca870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-05-20T01:16:09.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:09.386+0000 7f38ddc19700 1 --2- 172.21.15.16:0/652152058 >> v2:172.21.15.132:3300/0 conn(0x7f38d80717c0 0x7f38d80ca870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-05-20T01:16:09.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:09.386+0000 7f38dd418700 1 -- 172.21.15.16:0/652152058 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f38d8159b50 msgr2=0x7f38d806e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-05-20T01:16:09.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:09.386+0000 7f38dd418700 1 -- 172.21.15.16:0/652152058 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f38d8159b50 msgr2=0x7f38d806e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-05-20T01:16:09.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:09.386+0000 7f38dd418700 1 --2- 172.21.15.16:0/652152058 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f38d8159b50 0x7f38d806e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-05-20T01:16:09.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:09.386+0000 7f38dd418700 1 --2- 172.21.15.16:0/652152058 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f38d8159b50 0x7f38d806e0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-05-20T01:16:09.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:09.569+0000 7f38cffff700 1 -- 172.21.15.16:0/652152058 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 83575712 0 0) 0x7f38d80ccd30 con 0x7f38d80cd1c0 2023-05-20T01:16:09.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:09.569+0000 7f38cffff700 1 -- 172.21.15.16:0/652152058 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38c40035b0 con 0x7f38d80cd1c0 2023-05-20T01:16:09.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:09.569+0000 7f38cffff700 1 -- 172.21.15.16:0/652152058 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3522801599 0 0) 0x7f38c40035b0 con 0x7f38d80cd1c0 2023-05-20T01:16:09.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:09.569+0000 7f38cffff700 1 -- 172.21.15.16:0/652152058 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f38d8159b50 msgr2=0x7f38d806e0a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-05-20T01:16:09.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:09.569+0000 7f38cffff700 1 --2- 172.21.15.16:0/652152058 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f38d8159b50 0x7f38d806e0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:09.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:09.569+0000 7f38cffff700 1 -- 172.21.15.16:0/652152058 >> v2:172.21.15.132:3300/0 conn(0x7f38d80717c0 msgr2=0x7f38d80ca870 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-05-20T01:16:09.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:09.569+0000 7f38cffff700 1 --2- 172.21.15.16:0/652152058 >> v2:172.21.15.132:3300/0 conn(0x7f38d80717c0 0x7f38d80ca870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:09.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:09.569+0000 7f38cffff700 1 -- 172.21.15.16:0/652152058 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38d80cc9c0 con 0x7f38d80cd1c0 2023-05-20T01:16:09.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:09.569+0000 7f38cffff700 1 -- 172.21.15.16:0/652152058 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7f38c8001670 con 0x7f38d80cd1c0 2023-05-20T01:16:09.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:09.570+0000 7f38cffff700 1 -- 172.21.15.16:0/652152058 >> v1:172.21.15.16:6789/0 conn(0x7f38d80cd1c0 legacy=0x7f38d80710b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:09.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:09.570+0000 7f38cffff700 1 --2- 172.21.15.16:0/652152058 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f38c4003a00 0x7f38c4003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:09.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:09.570+0000 7f38cffff700 1 --2- 172.21.15.16:0/652152058 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f38d8159b50 0x7f38c4004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:09.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:09.570+0000 7f38cffff700 1 --2- 172.21.15.16:0/652152058 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38c4004d90 0x7f38c4005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:09.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:09.570+0000 7f38cffff700 1 -- 172.21.15.16:0/652152058 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38d80ce990 con 0x7f38c4004d90 2023-05-20T01:16:09.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:09.570+0000 7f38cffff700 1 -- 172.21.15.16:0/652152058 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38d80cc6e0 con 0x7f38c4003a00 2023-05-20T01:16:09.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:09.570+0000 7f38cffff700 1 -- 172.21.15.16:0/652152058 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38c400bac0 con 0x7f38d8159b50 2023-05-20T01:16:09.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:09.570+0000 7f38ddc19700 1 --2- 172.21.15.16:0/652152058 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38c4004d90 0x7f38c4005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:09.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:09.570+0000 7f38dcc17700 1 --2- 172.21.15.16:0/652152058 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f38d8159b50 0x7f38c4004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:09.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:09.570+0000 7f38dd418700 1 --2- 172.21.15.16:0/652152058 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f38c4003a00 0x7f38c4003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:09.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:09.570+0000 7f38ddc19700 1 -- 172.21.15.16:0/652152058 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f38d8159b50 msgr2=0x7f38c4004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:09.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:09.570+0000 7f38ddc19700 1 --2- 172.21.15.16:0/652152058 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f38d8159b50 0x7f38c4004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:09.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:09.570+0000 7f38ddc19700 1 -- 172.21.15.16:0/652152058 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f38c4003a00 msgr2=0x7f38c4003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:09.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:09.570+0000 7f38ddc19700 1 --2- 172.21.15.16:0/652152058 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f38c4003a00 0x7f38c4003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:09.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:09.570+0000 7f38ddc19700 1 -- 172.21.15.16:0/652152058 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38c40035b0 con 0x7f38c4004d90 2023-05-20T01:16:09.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:09.570+0000 7f38ddc19700 1 --2- 172.21.15.16:0/652152058 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38c4004d90 0x7f38c4005200 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f38d0009fd0 tx=0x7f38c4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73c210cc44b33f99 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:09.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:09.570+0000 7f38cffff700 1 -- 172.21.15.16:0/652152058 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d0007e20 con 0x7f38c4004d90 2023-05-20T01:16:09.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:09.570+0000 7f38cffff700 1 -- 172.21.15.16:0/652152058 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38d0003a20 con 0x7f38c4004d90 2023-05-20T01:16:09.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:09.571+0000 7f38cffff700 1 -- 172.21.15.16:0/652152058 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d0003dd0 con 0x7f38c4004d90 2023-05-20T01:16:09.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:09.571+0000 7f38df67c700 1 -- 172.21.15.16:0/652152058 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38c4004d90 msgr2=0x7f38c4005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:09.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:09.571+0000 7f38df67c700 1 --2- 172.21.15.16:0/652152058 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38c4004d90 0x7f38c4005200 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f38d0009fd0 tx=0x7f38c4005880 comp rx=0 tx=0).stop 2023-05-20T01:16:09.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:09.571+0000 7f38ddc19700 1 -- 172.21.15.16:0/652152058 reap_dead start 2023-05-20T01:16:09.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:09.571+0000 7f38df67c700 1 -- 172.21.15.16:0/652152058 shutdown_connections 2023-05-20T01:16:09.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:09.571+0000 7f38df67c700 1 -- 172.21.15.16:0/652152058 >> 172.21.15.16:0/652152058 conn(0x7f38d80cffd0 msgr2=0x7f38d80d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:09.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:09.571+0000 7f38df67c700 1 -- 172.21.15.16:0/652152058 shutdown_connections 2023-05-20T01:16:09.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:09.571+0000 7f38df67c700 1 -- 172.21.15.16:0/652152058 wait complete. 2023-05-20T01:16:09.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:09.573+0000 7f38df67c700 1 Processor -- start 2023-05-20T01:16:09.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:09.573+0000 7f38df67c700 1 -- start start 2023-05-20T01:16:09.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:09.575+0000 7f38df67c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f38c4004d90 0x7f38d80c6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:09.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:09.575+0000 7f38df67c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f38d80b5910 0x7f38d80c7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:09.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:09.575+0000 7f38df67c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38d8149f10 0x7f38d814a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:09.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:09.575+0000 7f38df67c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38d80c5540 con 0x7f38d8149f10 2023-05-20T01:16:09.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:09.575+0000 7f38df67c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38d80c5de0 con 0x7f38c4004d90 2023-05-20T01:16:09.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:09.575+0000 7f38df67c700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38d814a8c0 con 0x7f38d80b5910 2023-05-20T01:16:09.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:09.575+0000 7f38dd418700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f38c4004d90 0x7f38d80c6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:09.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:09.575+0000 7f38dd418700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f38c4004d90 0x7f38d80c6270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.16:53904/0 (socket says 172.21.15.16:53904) 2023-05-20T01:16:09.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:09.575+0000 7f38dd418700 1 -- 172.21.15.16:0/2849215780 learned_addr learned my addr 172.21.15.16:0/2849215780 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:09.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:09.575+0000 7f38ddc19700 1 --2- 172.21.15.16:0/2849215780 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38d8149f10 0x7f38d814a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:09.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:09.575+0000 7f38dcc17700 1 --2- 172.21.15.16:0/2849215780 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f38d80b5910 0x7f38d80c7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:09.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:09.575+0000 7f38dd418700 1 -- 172.21.15.16:0/2849215780 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f38d80b5910 msgr2=0x7f38d80c7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:09.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:09.575+0000 7f38dd418700 1 --2- 172.21.15.16:0/2849215780 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f38d80b5910 0x7f38d80c7150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:09.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:09.575+0000 7f38dd418700 1 -- 172.21.15.16:0/2849215780 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38d8149f10 msgr2=0x7f38d814a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:09.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:09.575+0000 7f38dd418700 1 --2- 172.21.15.16:0/2849215780 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38d8149f10 0x7f38d814a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:09.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:09.575+0000 7f38dd418700 1 -- 172.21.15.16:0/2849215780 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38d80cec70 con 0x7f38c4004d90 2023-05-20T01:16:09.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:09.575+0000 7f38ddc19700 1 --2- 172.21.15.16:0/2849215780 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38d8149f10 0x7f38d814a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-05-20T01:16:09.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:09.575+0000 7f38dd418700 1 --2- 172.21.15.16:0/2849215780 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f38c4004d90 0x7f38d80c6270 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f38d4011620 tx=0x7f38d4011930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:09.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:09.576+0000 7f38cdffb700 1 -- 172.21.15.16:0/2849215780 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d4006050 con 0x7f38c4004d90 2023-05-20T01:16:09.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:09.576+0000 7f38cdffb700 1 -- 172.21.15.16:0/2849215780 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38d40061b0 con 0x7f38c4004d90 2023-05-20T01:16:09.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:09.576+0000 7f38df67c700 1 -- 172.21.15.16:0/2849215780 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38d8071e80 con 0x7f38c4004d90 2023-05-20T01:16:09.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:09.576+0000 7f38cdffb700 1 -- 172.21.15.16:0/2849215780 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d4006560 con 0x7f38c4004d90 2023-05-20T01:16:09.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:09.576+0000 7f38cdffb700 1 -- 172.21.15.16:0/2849215780 <== mon.1 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f38d401a3f0 con 0x7f38c4004d90 2023-05-20T01:16:09.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:09.576+0000 7f38df67c700 1 -- 172.21.15.16:0/2849215780 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38d80ce5b0 con 0x7f38c4004d90 2023-05-20T01:16:09.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:09.576+0000 7f38cdffb700 1 -- 172.21.15.16:0/2849215780 <== mon.1 v2:172.21.15.132:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f38d401ab70 con 0x7f38c4004d90 2023-05-20T01:16:09.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:09.577+0000 7f38df67c700 1 -- 172.21.15.16:0/2849215780 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38d80c60c0 con 0x7f38c4004d90 2023-05-20T01:16:09.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:09.579+0000 7f38cdffb700 1 -- 172.21.15.16:0/2849215780 <== mon.1 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72437 (secure 0 0 0) 0x7f38d401f060 con 0x7f38c4004d90 2023-05-20T01:16:09.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:09.655+0000 7f38df67c700 1 -- 172.21.15.16:0/2849215780 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f38d80747b0 con 0x7f38c4004d90 2023-05-20T01:16:09.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:09.663+0000 7f38cdffb700 1 -- 172.21.15.16:0/2849215780 <== mon.1 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f38d400c070 con 0x7f38c4004d90 2023-05-20T01:16:09.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:09.664+0000 7f38df67c700 1 -- 172.21.15.16:0/2849215780 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f38c4004d90 msgr2=0x7f38d80c6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:09.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:09.664+0000 7f38df67c700 1 --2- 172.21.15.16:0/2849215780 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f38c4004d90 0x7f38d80c6270 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f38d4011620 tx=0x7f38d4011930 comp rx=0 tx=0).stop 2023-05-20T01:16:09.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:09.664+0000 7f38df67c700 1 -- 172.21.15.16:0/2849215780 shutdown_connections 2023-05-20T01:16:09.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:09.664+0000 7f38df67c700 1 --2- 172.21.15.16:0/2849215780 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38d8149f10 0x7f38d814a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:09.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:09.664+0000 7f38df67c700 1 --2- 172.21.15.16:0/2849215780 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f38c4004d90 0x7f38d80c6270 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:09.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:09.664+0000 7f38df67c700 1 --2- 172.21.15.16:0/2849215780 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f38d80b5910 0x7f38d80c7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:09.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:09.664+0000 7f38df67c700 1 -- 172.21.15.16:0/2849215780 >> 172.21.15.16:0/2849215780 conn(0x7f38d80cffd0 msgr2=0x7f38d80ca870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:09.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:09.664+0000 7f38df67c700 1 -- 172.21.15.16:0/2849215780 shutdown_connections 2023-05-20T01:16:09.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:09.664+0000 7f38df67c700 1 -- 172.21.15.16:0/2849215780 wait complete. 2023-05-20T01:16:09.668 INFO:teuthology.orchestra.run.smithi016.stderr:adjusted tunables profile to default 2023-05-20T01:16:09.678 INFO:tasks.ceph:check_enable_crimson: False 2023-05-20T01:16:09.679 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-05-20T01:16:09.679 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-05-20T01:16:09.679 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-05-20T01:16:09.692 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-05-20T01:16:09.690+0000 7fa73c17e200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-05-20T01:16:09.706 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-05-20T01:16:09.706 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-05-20T01:16:09.762 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-05-20T01:16:09.760+0000 7fa73c17e200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-05-20T01:16:09.773 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-05-20T01:16:09.774 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-05-20T01:16:09.840 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-05-20T01:16:09.841 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-05-20T01:16:09.844 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-05-20T01:16:09.841+0000 7fa73c17e200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-05-20T01:16:09.870 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2023-05-20T01:16:09.870 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-05-20T01:16:09.899 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2023-05-20T01:16:09.900 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-05-20T01:16:09.930 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-05-20T01:16:09.927+0000 7fa73c17e200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-05-20T01:16:09.968 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2023-05-20T01:16:09.969 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-05-20T01:16:10.034 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2023-05-20T01:16:10.034 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-05-20T01:16:10.063 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-05-20T01:16:10.060+0000 7f1da015b200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-05-20T01:16:10.079 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-05-20T01:16:10.076+0000 7fa73c17e200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-05-20T01:16:10.088 DEBUG:teuthology.orchestra.run.smithi132:> sudo ceph --cluster ceph osd new 317a83cd-19c8-4958-a301-6087ef342289 0 2023-05-20T01:16:10.131 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-05-20T01:16:10.128+0000 7f1da015b200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-05-20T01:16:10.221 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-05-20T01:16:10.218+0000 7f1da015b200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-05-20T01:16:10.260 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.257+0000 7f77ced40700 1 Processor -- start 2023-05-20T01:16:10.261 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.257+0000 7f77ced40700 1 -- start start 2023-05-20T01:16:10.262 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.258+0000 7f77ced40700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f77c80cd070 0x7f77c806e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:10.262 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.258+0000 7f77ced40700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f77c80710d0 0x7f77c80cabb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:10.262 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.258+0000 7f77ced40700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77c8072430 con 0x7f77c815cf40 2023-05-20T01:16:10.263 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.258+0000 7f77ced40700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77c8071570 con 0x7f77c80710d0 2023-05-20T01:16:10.263 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.258+0000 7f77ced40700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77c8071820 con 0x7f77c80cd070 2023-05-20T01:16:10.263 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.258+0000 7f77c7fff700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f77c80cd070 0x7f77c806e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:10.263 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.258+0000 7f77cd2dd700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f77c80710d0 0x7f77c80cabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:10.263 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.258+0000 7f77cd2dd700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f77c80710d0 0x7f77c80cabb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:57850/0 (socket says 172.21.15.132:57850) 2023-05-20T01:16:10.263 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.258+0000 7f77c7fff700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f77c80cd070 0x7f77c806e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.132:51576/0 (socket says 172.21.15.132:51576) 2023-05-20T01:16:10.263 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.258+0000 7f77cd2dd700 1 -- 172.21.15.132:0/767246826 learned_addr learned my addr 172.21.15.132:0/767246826 (peer_addr_for_me v2:172.21.15.132:0/0) 2023-05-20T01:16:10.263 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.259+0000 7f77cd2dd700 1 -- 172.21.15.132:0/767246826 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f77c80cd070 msgr2=0x7f77c806e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:10.264 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.259+0000 7f77cd2dd700 1 --2- 172.21.15.132:0/767246826 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f77c80cd070 0x7f77c806e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:10.264 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.259+0000 7f77cd2dd700 1 -- 172.21.15.132:0/767246826 >> v1:172.21.15.16:6789/0 conn(0x7f77c815cf40 legacy=0x7f77c80cc960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:10.264 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.259+0000 7f77cd2dd700 1 -- 172.21.15.132:0/767246826 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77c8071b00 con 0x7f77c80710d0 2023-05-20T01:16:10.264 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.259+0000 7f77cd2dd700 1 --2- 172.21.15.132:0/767246826 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f77c80710d0 0x7f77c80cabb0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f77c0009e50 tx=0x7f77c000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1857b057e8965c0f server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:10.264 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.259+0000 7f77c77fe700 1 -- 172.21.15.132:0/767246826 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c000e070 con 0x7f77c80710d0 2023-05-20T01:16:10.264 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.259+0000 7f77c77fe700 1 -- 172.21.15.132:0/767246826 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77c000b910 con 0x7f77c80710d0 2023-05-20T01:16:10.264 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.259+0000 7f77c77fe700 1 -- 172.21.15.132:0/767246826 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c000bcc0 con 0x7f77c80710d0 2023-05-20T01:16:10.267 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.259+0000 7f77ced40700 1 -- 172.21.15.132:0/767246826 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f77c80710d0 msgr2=0x7f77c80cabb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:10.267 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.259+0000 7f77ced40700 1 --2- 172.21.15.132:0/767246826 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f77c80710d0 0x7f77c80cabb0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f77c0009e50 tx=0x7f77c000b330 comp rx=0 tx=0).stop 2023-05-20T01:16:10.267 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.260+0000 7f77ced40700 1 -- 172.21.15.132:0/767246826 shutdown_connections 2023-05-20T01:16:10.267 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.260+0000 7f77ced40700 1 --2- 172.21.15.132:0/767246826 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f77c80710d0 0x7f77c80cabb0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:10.267 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.260+0000 7f77ced40700 1 --2- 172.21.15.132:0/767246826 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f77c80cd070 0x7f77c806e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:10.268 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.260+0000 7f77ced40700 1 -- 172.21.15.132:0/767246826 >> 172.21.15.132:0/767246826 conn(0x7f77c806c260 msgr2=0x7f77c80cffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:10.268 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.260+0000 7f77ced40700 1 -- 172.21.15.132:0/767246826 shutdown_connections 2023-05-20T01:16:10.268 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.260+0000 7f77ced40700 1 -- 172.21.15.132:0/767246826 wait complete. 2023-05-20T01:16:10.268 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.261+0000 7f77ced40700 1 Processor -- start 2023-05-20T01:16:10.268 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.261+0000 7f77ced40700 1 -- start start 2023-05-20T01:16:10.268 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.262+0000 7f77ced40700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f77c80710d0 0x7f77c814f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:10.268 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.262+0000 7f77ced40700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f77c80cd070 0x7f77c81e37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:10.269 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.262+0000 7f77ced40700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77c815cf40 0x7f77c81e3d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:10.269 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.262+0000 7f77ced40700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77c80c5d40 con 0x7f77c815cf40 2023-05-20T01:16:10.269 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.262+0000 7f77ced40700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77c814fa10 con 0x7f77c80cd070 2023-05-20T01:16:10.269 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.262+0000 7f77ced40700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77c80c5130 con 0x7f77c80710d0 2023-05-20T01:16:10.269 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.262+0000 7f77c7fff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f77c80cd070 0x7f77c81e37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:10.269 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.262+0000 7f77ccadc700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f77c80710d0 0x7f77c814f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:10.269 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.262+0000 7f77c7fff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f77c80cd070 0x7f77c81e37c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:57854/0 (socket says 172.21.15.132:57854) 2023-05-20T01:16:10.270 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.262+0000 7f77ccadc700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f77c80710d0 0x7f77c814f4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.132:51580/0 (socket says 172.21.15.132:51580) 2023-05-20T01:16:10.270 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.262+0000 7f77c7fff700 1 -- 172.21.15.132:0/3546861206 learned_addr learned my addr 172.21.15.132:0/3546861206 (peer_addr_for_me v2:172.21.15.132:0/0) 2023-05-20T01:16:10.270 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.262+0000 7f77cd2dd700 1 --2- 172.21.15.132:0/3546861206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77c815cf40 0x7f77c81e3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:10.270 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.262+0000 7f77ccadc700 1 -- 172.21.15.132:0/3546861206 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f77c80cd070 msgr2=0x7f77c81e37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:10.270 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.262+0000 7f77ccadc700 1 --2- 172.21.15.132:0/3546861206 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f77c80cd070 0x7f77c81e37c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:10.270 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.262+0000 7f77ccadc700 1 -- 172.21.15.132:0/3546861206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77c815cf40 msgr2=0x7f77c81e3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:10.270 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.262+0000 7f77ccadc700 1 --2- 172.21.15.132:0/3546861206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77c815cf40 0x7f77c81e3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:10.270 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.262+0000 7f77ccadc700 1 -- 172.21.15.132:0/3546861206 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77c0009b30 con 0x7f77c80710d0 2023-05-20T01:16:10.271 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.262+0000 7f77c7fff700 1 --2- 172.21.15.132:0/3546861206 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f77c80cd070 0x7f77c81e37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-05-20T01:16:10.271 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.262+0000 7f77ccadc700 1 --2- 172.21.15.132:0/3546861206 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f77c80710d0 0x7f77c814f4d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f77bc00c400 tx=0x7f77bc00c710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:10.271 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.262+0000 7f77c57fa700 1 -- 172.21.15.132:0/3546861206 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77bc0052d0 con 0x7f77c80710d0 2023-05-20T01:16:10.271 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.262+0000 7f77c57fa700 1 -- 172.21.15.132:0/3546861206 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77bc005430 con 0x7f77c80710d0 2023-05-20T01:16:10.271 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.262+0000 7f77ced40700 1 -- 172.21.15.132:0/3546861206 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77c80720c0 con 0x7f77c80710d0 2023-05-20T01:16:10.271 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.262+0000 7f77c57fa700 1 -- 172.21.15.132:0/3546861206 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77bc0038a0 con 0x7f77c80710d0 2023-05-20T01:16:10.271 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.263+0000 7f77ced40700 1 -- 172.21.15.132:0/3546861206 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77c8071de0 con 0x7f77c80710d0 2023-05-20T01:16:10.271 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.263+0000 7f77c57fa700 1 -- 172.21.15.132:0/3546861206 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f77bc003d00 con 0x7f77c80710d0 2023-05-20T01:16:10.272 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.263+0000 7f77c57fa700 1 -- 172.21.15.132:0/3546861206 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7f77bc012760 con 0x7f77c80710d0 2023-05-20T01:16:10.272 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.263+0000 7f77ced40700 1 -- 172.21.15.132:0/3546861206 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77c80cb0f0 con 0x7f77c80710d0 2023-05-20T01:16:10.272 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.266+0000 7f77c57fa700 1 -- 172.21.15.132:0/3546861206 <== mon.2 v2:172.21.15.132:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72437 (secure 0 0 0) 0x7f77bc017070 con 0x7f77c80710d0 2023-05-20T01:16:10.305 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-05-20T01:16:10.302+0000 7f1da015b200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-05-20T01:16:10.353 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.350+0000 7f77ced40700 1 -- 172.21.15.132:0/3546861206 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "osd new", "uuid": "317a83cd-19c8-4958-a301-6087ef342289", "id": 0} v 0) v1 -- 0x7f77c80747b0 con 0x7f77c80710d0 2023-05-20T01:16:10.359 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.356+0000 7f77c57fa700 1 -- 172.21.15.132:0/3546861206 <== mon.2 v2:172.21.15.132:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "317a83cd-19c8-4958-a301-6087ef342289", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f77bc01c440 con 0x7f77c80710d0 2023-05-20T01:16:10.359 INFO:teuthology.orchestra.run.smithi132.stdout:0 2023-05-20T01:16:10.360 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.357+0000 7f77ced40700 1 -- 172.21.15.132:0/3546861206 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f77c80710d0 msgr2=0x7f77c814f4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:10.360 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.357+0000 7f77ced40700 1 --2- 172.21.15.132:0/3546861206 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f77c80710d0 0x7f77c814f4d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f77bc00c400 tx=0x7f77bc00c710 comp rx=0 tx=0).stop 2023-05-20T01:16:10.360 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.357+0000 7f77ced40700 1 -- 172.21.15.132:0/3546861206 shutdown_connections 2023-05-20T01:16:10.361 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.357+0000 7f77ced40700 1 --2- 172.21.15.132:0/3546861206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77c815cf40 0x7f77c81e3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:10.361 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.357+0000 7f77ced40700 1 --2- 172.21.15.132:0/3546861206 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f77c80710d0 0x7f77c814f4d0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:10.361 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.357+0000 7f77ced40700 1 --2- 172.21.15.132:0/3546861206 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f77c80cd070 0x7f77c81e37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:10.361 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.357+0000 7f77ced40700 1 -- 172.21.15.132:0/3546861206 >> 172.21.15.132:0/3546861206 conn(0x7f77c806c260 msgr2=0x7f77c80cc210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:10.361 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.357+0000 7f77ced40700 1 -- 172.21.15.132:0/3546861206 shutdown_connections 2023-05-20T01:16:10.361 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.357+0000 7f77ced40700 1 -- 172.21.15.132:0/3546861206 wait complete. 2023-05-20T01:16:10.372 DEBUG:teuthology.orchestra.run.smithi132:> sudo ceph --cluster ceph osd new 5581046f-8219-4089-8d7d-b7846399505e 1 2023-05-20T01:16:10.421 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-05-20T01:16:10.418+0000 7fa73c17e200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-05-20T01:16:10.455 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-05-20T01:16:10.452+0000 7f1da015b200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-05-20T01:16:10.492 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.489+0000 7efe68096700 1 Processor -- start 2023-05-20T01:16:10.493 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.490+0000 7efe68096700 1 -- start start 2023-05-20T01:16:10.494 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.490+0000 7efe68096700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efe6015cd20 0x7efe600b1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:10.494 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.490+0000 7efe68096700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7efe600b14c0 0x7efe6006b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:10.495 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.490+0000 7efe68096700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe6006bfc0 con 0x7efe6006bb60 2023-05-20T01:16:10.495 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.490+0000 7efe68096700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe600cb700 con 0x7efe600b14c0 2023-05-20T01:16:10.495 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.490+0000 7efe68096700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe600b2430 con 0x7efe6015cd20 2023-05-20T01:16:10.495 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.491+0000 7efe65e32700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efe6015cd20 0x7efe600b1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:10.495 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.491+0000 7efe65631700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7efe600b14c0 0x7efe6006b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:10.496 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.491+0000 7efe65e32700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efe6015cd20 0x7efe600b1e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.132:51586/0 (socket says 172.21.15.132:51586) 2023-05-20T01:16:10.496 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.491+0000 7efe65631700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7efe600b14c0 0x7efe6006b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:57860/0 (socket says 172.21.15.132:57860) 2023-05-20T01:16:10.496 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.491+0000 7efe65e32700 1 -- 172.21.15.132:0/1785827557 learned_addr learned my addr 172.21.15.132:0/1785827557 (peer_addr_for_me v2:172.21.15.132:0/0) 2023-05-20T01:16:10.496 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.491+0000 7efe65e32700 1 -- 172.21.15.132:0/1785827557 >> v2:172.21.15.132:3300/0 conn(0x7efe600b14c0 msgr2=0x7efe6006b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:10.497 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.491+0000 7efe65e32700 1 --2- 172.21.15.132:0/1785827557 >> v2:172.21.15.132:3300/0 conn(0x7efe600b14c0 0x7efe6006b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:10.497 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.491+0000 7efe65e32700 1 -- 172.21.15.132:0/1785827557 >> v1:172.21.15.16:6789/0 conn(0x7efe6006bb60 legacy=0x7efe600cd930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:10.497 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.491+0000 7efe65e32700 1 -- 172.21.15.132:0/1785827557 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe600b2710 con 0x7efe6015cd20 2023-05-20T01:16:10.497 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.491+0000 7efe65631700 1 --2- 172.21.15.132:0/1785827557 >> v2:172.21.15.132:3300/0 conn(0x7efe600b14c0 0x7efe6006b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-05-20T01:16:10.498 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.491+0000 7efe65e32700 1 --2- 172.21.15.132:0/1785827557 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efe6015cd20 0x7efe600b1e50 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7efe5c0098f0 tx=0x7efe5c009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=496dbdbe61d2a67c server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:10.498 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.491+0000 7efe64e30700 1 -- 172.21.15.132:0/1785827557 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe5c011830 con 0x7efe6015cd20 2023-05-20T01:16:10.498 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.491+0000 7efe64e30700 1 -- 172.21.15.132:0/1785827557 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe5c011990 con 0x7efe6015cd20 2023-05-20T01:16:10.499 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.491+0000 7efe64e30700 1 -- 172.21.15.132:0/1785827557 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe5c011d20 con 0x7efe6015cd20 2023-05-20T01:16:10.500 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-05-20T01:16:10.496+0000 7fa73c17e200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-05-20T01:16:10.502 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.492+0000 7efe68096700 1 -- 172.21.15.132:0/1785827557 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efe6015cd20 msgr2=0x7efe600b1e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:10.503 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.492+0000 7efe68096700 1 --2- 172.21.15.132:0/1785827557 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efe6015cd20 0x7efe600b1e50 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7efe5c0098f0 tx=0x7efe5c009c00 comp rx=0 tx=0).stop 2023-05-20T01:16:10.503 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.492+0000 7efe68096700 1 -- 172.21.15.132:0/1785827557 shutdown_connections 2023-05-20T01:16:10.503 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.492+0000 7efe68096700 1 --2- 172.21.15.132:0/1785827557 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efe6015cd20 0x7efe600b1e50 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:10.503 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.492+0000 7efe68096700 1 --2- 172.21.15.132:0/1785827557 >> v2:172.21.15.132:3300/0 conn(0x7efe600b14c0 0x7efe6006b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:10.503 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.492+0000 7efe68096700 1 -- 172.21.15.132:0/1785827557 >> 172.21.15.132:0/1785827557 conn(0x7efe6006ae30 msgr2=0x7efe6009a400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:10.503 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.492+0000 7efe68096700 1 -- 172.21.15.132:0/1785827557 shutdown_connections 2023-05-20T01:16:10.504 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.492+0000 7efe68096700 1 -- 172.21.15.132:0/1785827557 wait complete. 2023-05-20T01:16:10.504 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.493+0000 7efe68096700 1 Processor -- start 2023-05-20T01:16:10.504 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.493+0000 7efe68096700 1 -- start start 2023-05-20T01:16:10.504 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.494+0000 7efe68096700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efe6006bb60 0x7efe600cfef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:10.504 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.494+0000 7efe68096700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe600b14c0 0x7efe600c52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:10.504 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.494+0000 7efe68096700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efe6015cd20 0x7efe600c5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:10.505 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.494+0000 7efe68096700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe60156e70 con 0x7efe600b14c0 2023-05-20T01:16:10.505 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.494+0000 7efe68096700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe600b5370 con 0x7efe6015cd20 2023-05-20T01:16:10.505 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.494+0000 7efe68096700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe600c5e40 con 0x7efe6006bb60 2023-05-20T01:16:10.505 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.494+0000 7efe65e32700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efe6006bb60 0x7efe600cfef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:10.505 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.494+0000 7efe65e32700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efe6006bb60 0x7efe600cfef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.132:51592/0 (socket says 172.21.15.132:51592) 2023-05-20T01:16:10.505 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.494+0000 7efe65e32700 1 -- 172.21.15.132:0/2087093589 learned_addr learned my addr 172.21.15.132:0/2087093589 (peer_addr_for_me v2:172.21.15.132:0/0) 2023-05-20T01:16:10.505 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.494+0000 7efe66633700 1 --2- 172.21.15.132:0/2087093589 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efe6015cd20 0x7efe600c5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:10.506 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.494+0000 7efe65631700 1 --2- 172.21.15.132:0/2087093589 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe600b14c0 0x7efe600c52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:10.506 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.494+0000 7efe65e32700 1 -- 172.21.15.132:0/2087093589 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efe6015cd20 msgr2=0x7efe600c5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:10.506 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.494+0000 7efe65e32700 1 --2- 172.21.15.132:0/2087093589 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efe6015cd20 0x7efe600c5810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:10.506 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.494+0000 7efe65e32700 1 -- 172.21.15.132:0/2087093589 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe600b14c0 msgr2=0x7efe600c52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:10.506 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.494+0000 7efe65e32700 1 --2- 172.21.15.132:0/2087093589 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe600b14c0 0x7efe600c52d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:10.507 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.494+0000 7efe65e32700 1 -- 172.21.15.132:0/2087093589 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe500094a0 con 0x7efe6006bb60 2023-05-20T01:16:10.507 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.495+0000 7efe65e32700 1 --2- 172.21.15.132:0/2087093589 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efe6006bb60 0x7efe600cfef0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7efe5c009c40 tx=0x7efe5c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:10.507 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.495+0000 7efe567fc700 1 -- 172.21.15.132:0/2087093589 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe5c004510 con 0x7efe6006bb60 2023-05-20T01:16:10.507 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.495+0000 7efe567fc700 1 -- 172.21.15.132:0/2087093589 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe5c0036a0 con 0x7efe6006bb60 2023-05-20T01:16:10.507 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.495+0000 7efe68096700 1 -- 172.21.15.132:0/2087093589 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe5c009550 con 0x7efe6006bb60 2023-05-20T01:16:10.507 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.495+0000 7efe567fc700 1 -- 172.21.15.132:0/2087093589 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe5c003a50 con 0x7efe6006bb60 2023-05-20T01:16:10.507 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.495+0000 7efe68096700 1 -- 172.21.15.132:0/2087093589 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe600cbc30 con 0x7efe6006bb60 2023-05-20T01:16:10.507 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.495+0000 7efe567fc700 1 -- 172.21.15.132:0/2087093589 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7efe5c02d440 con 0x7efe6006bb60 2023-05-20T01:16:10.508 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.495+0000 7efe567fc700 1 -- 172.21.15.132:0/2087093589 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7efe5c01a7d0 con 0x7efe6006bb60 2023-05-20T01:16:10.508 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.496+0000 7efe68096700 1 -- 172.21.15.132:0/2087093589 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe600cb950 con 0x7efe6006bb60 2023-05-20T01:16:10.508 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.498+0000 7efe567fc700 1 -- 172.21.15.132:0/2087093589 <== mon.2 v2:172.21.15.132:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72437 (secure 0 0 0) 0x7efe5c028070 con 0x7efe6006bb60 2023-05-20T01:16:10.586 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.583+0000 7efe68096700 1 -- 172.21.15.132:0/2087093589 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "osd new", "uuid": "5581046f-8219-4089-8d7d-b7846399505e", "id": 1} v 0) v1 -- 0x7efe600d1f60 con 0x7efe6006bb60 2023-05-20T01:16:10.593 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.590+0000 7efe567fc700 1 -- 172.21.15.132:0/2087093589 <== mon.2 v2:172.21.15.132:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "5581046f-8219-4089-8d7d-b7846399505e", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7efe5c015020 con 0x7efe6006bb60 2023-05-20T01:16:10.594 INFO:teuthology.orchestra.run.smithi132.stdout:1 2023-05-20T01:16:10.594 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.591+0000 7efe68096700 1 -- 172.21.15.132:0/2087093589 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efe6006bb60 msgr2=0x7efe600cfef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:10.595 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.591+0000 7efe68096700 1 --2- 172.21.15.132:0/2087093589 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efe6006bb60 0x7efe600cfef0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7efe5c009c40 tx=0x7efe5c004750 comp rx=0 tx=0).stop 2023-05-20T01:16:10.595 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.591+0000 7efe68096700 1 -- 172.21.15.132:0/2087093589 shutdown_connections 2023-05-20T01:16:10.595 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.591+0000 7efe68096700 1 --2- 172.21.15.132:0/2087093589 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efe6015cd20 0x7efe600c5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:10.595 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.591+0000 7efe68096700 1 --2- 172.21.15.132:0/2087093589 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efe6006bb60 0x7efe600cfef0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:10.595 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.591+0000 7efe68096700 1 --2- 172.21.15.132:0/2087093589 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe600b14c0 0x7efe600c52d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:10.595 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.591+0000 7efe68096700 1 -- 172.21.15.132:0/2087093589 >> 172.21.15.132:0/2087093589 conn(0x7efe6006ae30 msgr2=0x7efe600cd700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:10.595 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.591+0000 7efe68096700 1 -- 172.21.15.132:0/2087093589 shutdown_connections 2023-05-20T01:16:10.596 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.591+0000 7efe68096700 1 -- 172.21.15.132:0/2087093589 wait complete. 2023-05-20T01:16:10.606 DEBUG:teuthology.orchestra.run.smithi132:> sudo ceph --cluster ceph osd new 072e9e66-4133-478e-8d4a-64d46c2963f7 2 2023-05-20T01:16:10.654 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-05-20T01:16:10.651+0000 7fa73c17e200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-05-20T01:16:10.726 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.723+0000 7f29557ee700 1 Processor -- start 2023-05-20T01:16:10.726 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.723+0000 7f29557ee700 1 -- start start 2023-05-20T01:16:10.728 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.723+0000 7f29557ee700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f295015cec0 0x7f2950071bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:10.728 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.723+0000 7f29557ee700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f295015d290 0x7f2950072130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:10.728 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.723+0000 7f29557ee700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29500cfe30 con 0x7f29500cf2c0 2023-05-20T01:16:10.729 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.723+0000 7f29557ee700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29500cd630 con 0x7f295015cec0 2023-05-20T01:16:10.729 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.723+0000 7f29557ee700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29500cca60 con 0x7f295015d290 2023-05-20T01:16:10.729 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.724+0000 7f294e7fc700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f295015d290 0x7f2950072130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:10.730 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.724+0000 7f294e7fc700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f295015d290 0x7f2950072130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.132:51598/0 (socket says 172.21.15.132:51598) 2023-05-20T01:16:10.730 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.724+0000 7f294e7fc700 1 -- 172.21.15.132:0/1380540999 learned_addr learned my addr 172.21.15.132:0/1380540999 (peer_addr_for_me v2:172.21.15.132:0/0) 2023-05-20T01:16:10.730 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.724+0000 7f294effd700 1 --2- 172.21.15.132:0/1380540999 >> v2:172.21.15.16:3300/0 conn(0x7f295015cec0 0x7f2950071bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:10.731 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.724+0000 7f294e7fc700 1 -- 172.21.15.132:0/1380540999 >> v2:172.21.15.16:3300/0 conn(0x7f295015cec0 msgr2=0x7f2950071bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:10.731 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.724+0000 7f294e7fc700 1 --2- 172.21.15.132:0/1380540999 >> v2:172.21.15.16:3300/0 conn(0x7f295015cec0 0x7f2950071bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:10.731 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.724+0000 7f294e7fc700 1 -- 172.21.15.132:0/1380540999 >> v1:172.21.15.16:6789/0 conn(0x7f29500cf2c0 legacy=0x7f29500cf720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:10.731 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.724+0000 7f294e7fc700 1 -- 172.21.15.132:0/1380540999 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29500ccd00 con 0x7f295015d290 2023-05-20T01:16:10.732 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.724+0000 7f294e7fc700 1 --2- 172.21.15.132:0/1380540999 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f295015d290 0x7f2950072130 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f29400098f0 tx=0x7f2940009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=740db600101a5b49 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:10.732 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.724+0000 7f294dffb700 1 -- 172.21.15.132:0/1380540999 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2940011830 con 0x7f295015d290 2023-05-20T01:16:10.732 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.724+0000 7f294dffb700 1 -- 172.21.15.132:0/1380540999 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2940011990 con 0x7f295015d290 2023-05-20T01:16:10.733 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.724+0000 7f294dffb700 1 -- 172.21.15.132:0/1380540999 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2940011d20 con 0x7f295015d290 2023-05-20T01:16:10.733 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.725+0000 7f29557ee700 1 -- 172.21.15.132:0/1380540999 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f295015d290 msgr2=0x7f2950072130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:10.733 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.725+0000 7f29557ee700 1 --2- 172.21.15.132:0/1380540999 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f295015d290 0x7f2950072130 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f29400098f0 tx=0x7f2940009c00 comp rx=0 tx=0).stop 2023-05-20T01:16:10.734 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.725+0000 7f29557ee700 1 -- 172.21.15.132:0/1380540999 shutdown_connections 2023-05-20T01:16:10.734 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.725+0000 7f29557ee700 1 --2- 172.21.15.132:0/1380540999 >> v2:172.21.15.16:3300/0 conn(0x7f295015cec0 0x7f2950071bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:10.734 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.725+0000 7f29557ee700 1 --2- 172.21.15.132:0/1380540999 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f295015d290 0x7f2950072130 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:10.734 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.725+0000 7f29557ee700 1 -- 172.21.15.132:0/1380540999 >> 172.21.15.132:0/1380540999 conn(0x7f29500cffd0 msgr2=0x7f29500d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:10.734 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.725+0000 7f29557ee700 1 -- 172.21.15.132:0/1380540999 shutdown_connections 2023-05-20T01:16:10.734 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.725+0000 7f29557ee700 1 -- 172.21.15.132:0/1380540999 wait complete. 2023-05-20T01:16:10.737 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.726+0000 7f29557ee700 1 Processor -- start 2023-05-20T01:16:10.738 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.727+0000 7f29557ee700 1 -- start start 2023-05-20T01:16:10.738 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.727+0000 7f29557ee700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f29500cf2c0 0x7f29501503e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:10.738 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.727+0000 7f29557ee700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f295015cec0 0x7f2950151950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:10.738 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.727+0000 7f29557ee700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f295015d290 0x7f29501e5f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:10.738 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.727+0000 7f29557ee700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29500b4b20 con 0x7f295015d290 2023-05-20T01:16:10.739 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.727+0000 7f29557ee700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29500b4e00 con 0x7f295015cec0 2023-05-20T01:16:10.739 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.727+0000 7f29557ee700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29500b50e0 con 0x7f29500cf2c0 2023-05-20T01:16:10.739 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.728+0000 7f294effd700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f29500cf2c0 0x7f29501503e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:10.739 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.728+0000 7f294e7fc700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f295015cec0 0x7f2950151950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:10.739 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.728+0000 7f294effd700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f29500cf2c0 0x7f29501503e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.132:51602/0 (socket says 172.21.15.132:51602) 2023-05-20T01:16:10.739 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.728+0000 7f294effd700 1 -- 172.21.15.132:0/1807402568 learned_addr learned my addr 172.21.15.132:0/1807402568 (peer_addr_for_me v2:172.21.15.132:0/0) 2023-05-20T01:16:10.740 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.728+0000 7f294e7fc700 1 --2- 172.21.15.132:0/1807402568 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f295015cec0 0x7f2950151950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:57876/0 (socket says 172.21.15.132:57876) 2023-05-20T01:16:10.740 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.728+0000 7f294f7fe700 1 --2- 172.21.15.132:0/1807402568 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f295015d290 0x7f29501e5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:10.740 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.728+0000 7f294e7fc700 1 -- 172.21.15.132:0/1807402568 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f29500cf2c0 msgr2=0x7f29501503e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:10.740 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.728+0000 7f294e7fc700 1 --2- 172.21.15.132:0/1807402568 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f29500cf2c0 0x7f29501503e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:10.740 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.728+0000 7f294e7fc700 1 -- 172.21.15.132:0/1807402568 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f295015d290 msgr2=0x7f29501e5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:10.740 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.728+0000 7f294e7fc700 1 --2- 172.21.15.132:0/1807402568 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f295015d290 0x7f29501e5f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:10.741 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.728+0000 7f294e7fc700 1 -- 172.21.15.132:0/1807402568 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2940009550 con 0x7f295015cec0 2023-05-20T01:16:10.741 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.728+0000 7f294effd700 1 --2- 172.21.15.132:0/1807402568 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f29500cf2c0 0x7f29501503e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-05-20T01:16:10.741 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.728+0000 7f294e7fc700 1 --2- 172.21.15.132:0/1807402568 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f295015cec0 0x7f2950151950 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2940016000 tx=0x7f2940004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:10.742 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.728+0000 7f29377fe700 1 -- 172.21.15.132:0/1807402568 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2940004510 con 0x7f295015cec0 2023-05-20T01:16:10.742 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.728+0000 7f29377fe700 1 -- 172.21.15.132:0/1807402568 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f294001e070 con 0x7f295015cec0 2023-05-20T01:16:10.742 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.728+0000 7f29377fe700 1 -- 172.21.15.132:0/1807402568 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29400038c0 con 0x7f295015cec0 2023-05-20T01:16:10.742 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.728+0000 7f29557ee700 1 -- 172.21.15.132:0/1807402568 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29500cd2c0 con 0x7f295015cec0 2023-05-20T01:16:10.742 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.728+0000 7f29557ee700 1 -- 172.21.15.132:0/1807402568 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29500ccfe0 con 0x7f295015cec0 2023-05-20T01:16:10.742 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.728+0000 7f29377fe700 1 -- 172.21.15.132:0/1807402568 <== mon.1 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f2940003d20 con 0x7f295015cec0 2023-05-20T01:16:10.743 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.728+0000 7f29377fe700 1 -- 172.21.15.132:0/1807402568 <== mon.1 v2:172.21.15.132:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7f294001a420 con 0x7f295015cec0 2023-05-20T01:16:10.743 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.729+0000 7f29557ee700 1 -- 172.21.15.132:0/1807402568 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2950160890 con 0x7f295015cec0 2023-05-20T01:16:10.743 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.731+0000 7f29377fe700 1 -- 172.21.15.132:0/1807402568 <== mon.1 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72437 (secure 0 0 0) 0x7f2940020070 con 0x7f295015cec0 2023-05-20T01:16:10.749 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-05-20T01:16:10.747+0000 7fa73c17e200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-05-20T01:16:10.801 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-05-20T01:16:10.798+0000 7f1da015b200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-05-20T01:16:10.818 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.815+0000 7f29557ee700 1 -- 172.21.15.132:0/1807402568 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd new", "uuid": "072e9e66-4133-478e-8d4a-64d46c2963f7", "id": 2} v 0) v1 -- 0x7f29500b4f40 con 0x7f295015cec0 2023-05-20T01:16:10.825 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.822+0000 7f29377fe700 1 -- 172.21.15.132:0/1807402568 <== mon.1 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "072e9e66-4133-478e-8d4a-64d46c2963f7", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f29500b4f40 con 0x7f295015cec0 2023-05-20T01:16:10.826 INFO:teuthology.orchestra.run.smithi132.stdout:2 2023-05-20T01:16:10.827 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.823+0000 7f29557ee700 1 -- 172.21.15.132:0/1807402568 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f295015cec0 msgr2=0x7f2950151950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:10.827 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.823+0000 7f29557ee700 1 --2- 172.21.15.132:0/1807402568 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f295015cec0 0x7f2950151950 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2940016000 tx=0x7f2940004750 comp rx=0 tx=0).stop 2023-05-20T01:16:10.827 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.823+0000 7f29557ee700 1 -- 172.21.15.132:0/1807402568 shutdown_connections 2023-05-20T01:16:10.827 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.823+0000 7f29557ee700 1 --2- 172.21.15.132:0/1807402568 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f295015d290 0x7f29501e5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:10.827 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.823+0000 7f29557ee700 1 --2- 172.21.15.132:0/1807402568 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f29500cf2c0 0x7f29501503e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:10.827 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.823+0000 7f29557ee700 1 --2- 172.21.15.132:0/1807402568 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f295015cec0 0x7f2950151950 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:10.828 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.823+0000 7f29557ee700 1 -- 172.21.15.132:0/1807402568 >> 172.21.15.132:0/1807402568 conn(0x7f29500cffd0 msgr2=0x7f295015fa70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:10.828 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.823+0000 7f29557ee700 1 -- 172.21.15.132:0/1807402568 shutdown_connections 2023-05-20T01:16:10.828 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.823+0000 7f29557ee700 1 -- 172.21.15.132:0/1807402568 wait complete. 2023-05-20T01:16:10.838 DEBUG:teuthology.orchestra.run.smithi132:> sudo ceph --cluster ceph osd new 11e1bba6-00f8-48dd-9028-d8d716b7f145 3 2023-05-20T01:16:10.883 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-05-20T01:16:10.880+0000 7f1da015b200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-05-20T01:16:10.960 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.957+0000 7f8275ecb700 1 Processor -- start 2023-05-20T01:16:10.961 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.957+0000 7f8275ecb700 1 -- start start 2023-05-20T01:16:10.963 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.959+0000 7f8275ecb700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f82701572c0 0x7f8270159e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:10.963 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.959+0000 7f8275ecb700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8270073c10 con 0x7f827006e150 2023-05-20T01:16:10.963 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.959+0000 7f8275ecb700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f827015a3d0 con 0x7f8270071e10 2023-05-20T01:16:10.964 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.959+0000 7f8275ecb700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f827015a600 con 0x7f82701572c0 2023-05-20T01:16:10.964 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.959+0000 7f826effd700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f8270071e10 0x7f827015db20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.132:38842/0 (socket says 172.21.15.132:38842) 2023-05-20T01:16:10.964 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.959+0000 7f826effd700 1 -- 172.21.15.132:0/375202852 learned_addr learned my addr 172.21.15.132:0/375202852 (peer_addr_for_me v1:172.21.15.132:0/0) 2023-05-20T01:16:10.964 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.959+0000 7f826f7fe700 1 --2- 172.21.15.132:0/375202852 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f82701572c0 0x7f8270159e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:10.964 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.959+0000 7f826e7fc700 1 -- 172.21.15.132:0/375202852 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1091151036 0 0) 0x7f827015a3d0 con 0x7f8270071e10 2023-05-20T01:16:10.965 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.959+0000 7f826e7fc700 1 -- 172.21.15.132:0/375202852 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f825c0035b0 con 0x7f8270071e10 2023-05-20T01:16:10.965 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.959+0000 7f826f7fe700 1 -- 172.21.15.132:0/375202852 >> v1:172.21.15.132:6789/0 conn(0x7f8270071e10 legacy=0x7f827015db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:10.965 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.959+0000 7f826f7fe700 1 -- 172.21.15.132:0/375202852 >> v1:172.21.15.16:6789/0 conn(0x7f827006e150 legacy=0x7f82700caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:10.965 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.959+0000 7f826f7fe700 1 -- 172.21.15.132:0/375202852 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f827015a8e0 con 0x7f82701572c0 2023-05-20T01:16:10.965 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.959+0000 7f826f7fe700 1 --2- 172.21.15.132:0/375202852 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f82701572c0 0x7f8270159e90 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f8260009dc0 tx=0x7f826000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42204f88a60b2162 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:10.965 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.960+0000 7f826e7fc700 1 -- 172.21.15.132:0/375202852 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f826000e070 con 0x7f82701572c0 2023-05-20T01:16:10.965 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.960+0000 7f826e7fc700 1 -- 172.21.15.132:0/375202852 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f826000bb20 con 0x7f82701572c0 2023-05-20T01:16:10.966 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.960+0000 7f826e7fc700 1 -- 172.21.15.132:0/375202852 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8260012430 con 0x7f82701572c0 2023-05-20T01:16:10.968 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.961+0000 7f8275ecb700 1 -- 172.21.15.132:0/375202852 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f82701572c0 msgr2=0x7f8270159e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:10.969 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.961+0000 7f8275ecb700 1 --2- 172.21.15.132:0/375202852 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f82701572c0 0x7f8270159e90 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f8260009dc0 tx=0x7f826000b330 comp rx=0 tx=0).stop 2023-05-20T01:16:10.970 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.961+0000 7f8275ecb700 1 -- 172.21.15.132:0/375202852 shutdown_connections 2023-05-20T01:16:10.970 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.961+0000 7f8275ecb700 1 --2- 172.21.15.132:0/375202852 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f82701572c0 0x7f8270159e90 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:10.970 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.961+0000 7f8275ecb700 1 -- 172.21.15.132:0/375202852 >> 172.21.15.132:0/375202852 conn(0x7f827006cbb0 msgr2=0x7f827006cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:10.970 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.961+0000 7f8275ecb700 1 -- 172.21.15.132:0/375202852 shutdown_connections 2023-05-20T01:16:10.970 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.961+0000 7f8275ecb700 1 -- 172.21.15.132:0/375202852 wait complete. 2023-05-20T01:16:10.970 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.962+0000 7f8275ecb700 1 Processor -- start 2023-05-20T01:16:10.970 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.963+0000 7f8275ecb700 1 -- start start 2023-05-20T01:16:10.971 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.964+0000 7f8275ecb700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f827006e150 0x7f8270144300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:10.971 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.964+0000 7f8275ecb700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8270071e10 0x7f82700c5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:10.971 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.964+0000 7f8275ecb700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82701572c0 0x7f82700c5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:10.971 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.964+0000 7f8275ecb700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82700c7070 con 0x7f82701572c0 2023-05-20T01:16:10.971 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.964+0000 7f8275ecb700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82700b49d0 con 0x7f8270071e10 2023-05-20T01:16:10.971 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.964+0000 7f8275ecb700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82700c6a20 con 0x7f827006e150 2023-05-20T01:16:10.971 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.964+0000 7f826effd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8270071e10 0x7f82700c5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:10.971 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.964+0000 7f826effd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8270071e10 0x7f82700c5660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:57888/0 (socket says 172.21.15.132:57888) 2023-05-20T01:16:10.972 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.964+0000 7f826effd700 1 -- 172.21.15.132:0/2305339741 learned_addr learned my addr 172.21.15.132:0/2305339741 (peer_addr_for_me v2:172.21.15.132:0/0) 2023-05-20T01:16:10.972 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.964+0000 7f826f7fe700 1 --2- 172.21.15.132:0/2305339741 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f827006e150 0x7f8270144300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:10.972 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.964+0000 7f826ffff700 1 --2- 172.21.15.132:0/2305339741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82701572c0 0x7f82700c5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:10.972 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.964+0000 7f826effd700 1 -- 172.21.15.132:0/2305339741 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f827006e150 msgr2=0x7f8270144300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:10.972 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.964+0000 7f826effd700 1 --2- 172.21.15.132:0/2305339741 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f827006e150 0x7f8270144300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:10.973 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.964+0000 7f826effd700 1 -- 172.21.15.132:0/2305339741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82701572c0 msgr2=0x7f82700c5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:10.973 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.964+0000 7f826effd700 1 --2- 172.21.15.132:0/2305339741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82701572c0 0x7f82700c5ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:10.973 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.964+0000 7f826effd700 1 -- 172.21.15.132:0/2305339741 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8260009a70 con 0x7f8270071e10 2023-05-20T01:16:10.973 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.964+0000 7f826f7fe700 1 --2- 172.21.15.132:0/2305339741 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f827006e150 0x7f8270144300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-05-20T01:16:10.974 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.965+0000 7f826effd700 1 --2- 172.21.15.132:0/2305339741 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8270071e10 0x7f82700c5660 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f825800c700 tx=0x7f825800ca10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:10.974 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.965+0000 7f8267fff700 1 -- 172.21.15.132:0/2305339741 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8258005d70 con 0x7f8270071e10 2023-05-20T01:16:10.974 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.965+0000 7f8267fff700 1 -- 172.21.15.132:0/2305339741 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82580070d0 con 0x7f8270071e10 2023-05-20T01:16:10.974 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.965+0000 7f8275ecb700 1 -- 172.21.15.132:0/2305339741 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f827015d600 con 0x7f8270071e10 2023-05-20T01:16:10.974 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.965+0000 7f8267fff700 1 -- 172.21.15.132:0/2305339741 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8258007460 con 0x7f8270071e10 2023-05-20T01:16:10.974 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.965+0000 7f8275ecb700 1 -- 172.21.15.132:0/2305339741 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f827015abc0 con 0x7f8270071e10 2023-05-20T01:16:10.974 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.965+0000 7f8267fff700 1 -- 172.21.15.132:0/2305339741 <== mon.1 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f825801c440 con 0x7f8270071e10 2023-05-20T01:16:10.975 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.965+0000 7f8267fff700 1 -- 172.21.15.132:0/2305339741 <== mon.1 v2:172.21.15.132:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7f8258012890 con 0x7f8270071e10 2023-05-20T01:16:10.975 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.966+0000 7f8275ecb700 1 -- 172.21.15.132:0/2305339741 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f827015e230 con 0x7f8270071e10 2023-05-20T01:16:10.975 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:10.968+0000 7f8267fff700 1 -- 172.21.15.132:0/2305339741 <== mon.1 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72437 (secure 0 0 0) 0x7f8258017070 con 0x7f8270071e10 2023-05-20T01:16:11.018 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-05-20T01:16:11.015+0000 7f1da015b200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-05-20T01:16:11.054 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.051+0000 7f8275ecb700 1 -- 172.21.15.132:0/2305339741 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd new", "uuid": "11e1bba6-00f8-48dd-9028-d8d716b7f145", "id": 3} v 0) v1 -- 0x7f82700c6860 con 0x7f8270071e10 2023-05-20T01:16:11.057 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.054+0000 7f8267fff700 1 -- 172.21.15.132:0/2305339741 <== mon.1 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "11e1bba6-00f8-48dd-9028-d8d716b7f145", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f8258012570 con 0x7f8270071e10 2023-05-20T01:16:11.058 INFO:teuthology.orchestra.run.smithi132.stdout:3 2023-05-20T01:16:11.059 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.055+0000 7f8275ecb700 1 -- 172.21.15.132:0/2305339741 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8270071e10 msgr2=0x7f82700c5660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:11.059 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.055+0000 7f8275ecb700 1 --2- 172.21.15.132:0/2305339741 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8270071e10 0x7f82700c5660 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f825800c700 tx=0x7f825800ca10 comp rx=0 tx=0).stop 2023-05-20T01:16:11.059 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.055+0000 7f8275ecb700 1 -- 172.21.15.132:0/2305339741 shutdown_connections 2023-05-20T01:16:11.059 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.055+0000 7f8275ecb700 1 --2- 172.21.15.132:0/2305339741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82701572c0 0x7f82700c5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:11.059 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.055+0000 7f8275ecb700 1 --2- 172.21.15.132:0/2305339741 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f827006e150 0x7f8270144300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:11.059 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.056+0000 7f8275ecb700 1 --2- 172.21.15.132:0/2305339741 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8270071e10 0x7f82700c5660 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:11.059 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.056+0000 7f8275ecb700 1 -- 172.21.15.132:0/2305339741 >> 172.21.15.132:0/2305339741 conn(0x7f827006cbb0 msgr2=0x7f82700cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:11.060 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.056+0000 7f8275ecb700 1 -- 172.21.15.132:0/2305339741 shutdown_connections 2023-05-20T01:16:11.060 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.056+0000 7f8275ecb700 1 -- 172.21.15.132:0/2305339741 wait complete. 2023-05-20T01:16:11.071 DEBUG:teuthology.orchestra.run.smithi132:> sudo ceph --cluster ceph osd new b8909c3a-cf70-46cb-8d0b-009c707cf9d3 4 2023-05-20T01:16:11.105 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-05-20T01:16:11.102+0000 7f1da015b200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-05-20T01:16:11.192 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.189+0000 7f6144642700 1 Processor -- start 2023-05-20T01:16:11.193 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.189+0000 7f6144642700 1 -- start start 2023-05-20T01:16:11.194 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.190+0000 7f6144642700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f613c15cec0 0x7f613c071bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:11.194 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.190+0000 7f6144642700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f613c15d290 0x7f613c072130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:11.194 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.190+0000 7f6144642700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f613c0cd630 con 0x7f613c0cf2c0 2023-05-20T01:16:11.194 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.190+0000 7f6144642700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f613c0cca60 con 0x7f613c15d290 2023-05-20T01:16:11.194 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.190+0000 7f6144642700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f613c0cfdd0 con 0x7f613c15cec0 2023-05-20T01:16:11.195 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.190+0000 7f61423de700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f613c15cec0 0x7f613c071bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:11.195 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.190+0000 7f61423de700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f613c15cec0 0x7f613c071bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:57892/0 (socket says 172.21.15.132:57892) 2023-05-20T01:16:11.195 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.190+0000 7f61423de700 1 -- 172.21.15.132:0/941484689 learned_addr learned my addr 172.21.15.132:0/941484689 (peer_addr_for_me v2:172.21.15.132:0/0) 2023-05-20T01:16:11.195 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.190+0000 7f6141bdd700 1 --2- 172.21.15.132:0/941484689 >> v2:172.21.15.16:3300/0 conn(0x7f613c15d290 0x7f613c072130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:11.195 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.190+0000 7f61423de700 1 -- 172.21.15.132:0/941484689 >> v2:172.21.15.16:3300/0 conn(0x7f613c15d290 msgr2=0x7f613c072130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:11.195 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.190+0000 7f61423de700 1 --2- 172.21.15.132:0/941484689 >> v2:172.21.15.16:3300/0 conn(0x7f613c15d290 0x7f613c072130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:11.195 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.190+0000 7f61423de700 1 -- 172.21.15.132:0/941484689 >> v1:172.21.15.16:6789/0 conn(0x7f613c0cf2c0 legacy=0x7f613c0cf6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:11.196 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.190+0000 7f61423de700 1 -- 172.21.15.132:0/941484689 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f613c0ccd00 con 0x7f613c15cec0 2023-05-20T01:16:11.196 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.191+0000 7f61423de700 1 --2- 172.21.15.132:0/941484689 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f613c15cec0 0x7f613c071bf0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f613c072770 tx=0x7f6138009bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34802e55c4c42003 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:11.196 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.191+0000 7f61413dc700 1 -- 172.21.15.132:0/941484689 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6138005e30 con 0x7f613c15cec0 2023-05-20T01:16:11.196 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.191+0000 7f61413dc700 1 -- 172.21.15.132:0/941484689 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6138004030 con 0x7f613c15cec0 2023-05-20T01:16:11.196 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.191+0000 7f61413dc700 1 -- 172.21.15.132:0/941484689 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61380043e0 con 0x7f613c15cec0 2023-05-20T01:16:11.197 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-05-20T01:16:11.191+0000 7fa73c17e200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-05-20T01:16:11.199 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.191+0000 7f6144642700 1 -- 172.21.15.132:0/941484689 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f613c15cec0 msgr2=0x7f613c071bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:11.199 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.191+0000 7f6144642700 1 --2- 172.21.15.132:0/941484689 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f613c15cec0 0x7f613c071bf0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f613c072770 tx=0x7f6138009bd0 comp rx=0 tx=0).stop 2023-05-20T01:16:11.199 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.191+0000 7f6144642700 1 -- 172.21.15.132:0/941484689 shutdown_connections 2023-05-20T01:16:11.199 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.191+0000 7f6144642700 1 --2- 172.21.15.132:0/941484689 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f613c15cec0 0x7f613c071bf0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:11.199 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.191+0000 7f6144642700 1 --2- 172.21.15.132:0/941484689 >> v2:172.21.15.16:3300/0 conn(0x7f613c15d290 0x7f613c072130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:11.199 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.191+0000 7f6144642700 1 -- 172.21.15.132:0/941484689 >> 172.21.15.132:0/941484689 conn(0x7f613c0cffd0 msgr2=0x7f613c0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:11.200 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.192+0000 7f6144642700 1 -- 172.21.15.132:0/941484689 shutdown_connections 2023-05-20T01:16:11.200 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.192+0000 7f6144642700 1 -- 172.21.15.132:0/941484689 wait complete. 2023-05-20T01:16:11.200 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.193+0000 7f6144642700 1 Processor -- start 2023-05-20T01:16:11.200 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.193+0000 7f6144642700 1 -- start start 2023-05-20T01:16:11.200 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.193+0000 7f6144642700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f613c0cf2c0 0x7f613c0c9c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:11.200 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.193+0000 7f6144642700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f613c15cec0 0x7f613c151a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:11.200 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.193+0000 7f6144642700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f613c15d290 0x7f613c15aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:11.201 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.193+0000 7f6144642700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f613c0c8910 con 0x7f613c15d290 2023-05-20T01:16:11.201 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.193+0000 7f6144642700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f613c0c82b0 con 0x7f613c15cec0 2023-05-20T01:16:11.201 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.193+0000 7f6144642700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f613c0c8b60 con 0x7f613c0cf2c0 2023-05-20T01:16:11.201 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.194+0000 7f6141bdd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f613c15cec0 0x7f613c151a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:11.201 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.194+0000 7f6141bdd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f613c15cec0 0x7f613c151a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:57900/0 (socket says 172.21.15.132:57900) 2023-05-20T01:16:11.201 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.194+0000 7f6141bdd700 1 -- 172.21.15.132:0/3139357202 learned_addr learned my addr 172.21.15.132:0/3139357202 (peer_addr_for_me v2:172.21.15.132:0/0) 2023-05-20T01:16:11.201 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.194+0000 7f61423de700 1 --2- 172.21.15.132:0/3139357202 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f613c0cf2c0 0x7f613c0c9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:11.201 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.194+0000 7f6142bdf700 1 --2- 172.21.15.132:0/3139357202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f613c15d290 0x7f613c15aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:11.202 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.194+0000 7f6141bdd700 1 -- 172.21.15.132:0/3139357202 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f613c0cf2c0 msgr2=0x7f613c0c9c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:11.202 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.194+0000 7f6141bdd700 1 --2- 172.21.15.132:0/3139357202 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f613c0cf2c0 0x7f613c0c9c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:11.202 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.194+0000 7f6141bdd700 1 -- 172.21.15.132:0/3139357202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f613c15d290 msgr2=0x7f613c15aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:11.202 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.194+0000 7f6141bdd700 1 --2- 172.21.15.132:0/3139357202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f613c15d290 0x7f613c15aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:11.202 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.194+0000 7f6141bdd700 1 -- 172.21.15.132:0/3139357202 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6138009500 con 0x7f613c15cec0 2023-05-20T01:16:11.202 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.194+0000 7f6141bdd700 1 --2- 172.21.15.132:0/3139357202 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f613c15cec0 0x7f613c151a40 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f612c00d800 tx=0x7f612c00db10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:11.202 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.194+0000 7f6132ffd700 1 -- 172.21.15.132:0/3139357202 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f612c0125f0 con 0x7f613c15cec0 2023-05-20T01:16:11.202 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.194+0000 7f6132ffd700 1 -- 172.21.15.132:0/3139357202 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f612c012750 con 0x7f613c15cec0 2023-05-20T01:16:11.203 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.194+0000 7f6144642700 1 -- 172.21.15.132:0/3139357202 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f613c0cd2c0 con 0x7f613c15cec0 2023-05-20T01:16:11.203 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.194+0000 7f6132ffd700 1 -- 172.21.15.132:0/3139357202 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f612c012b00 con 0x7f613c15cec0 2023-05-20T01:16:11.203 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.194+0000 7f6144642700 1 -- 172.21.15.132:0/3139357202 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f613c0ccfe0 con 0x7f613c15cec0 2023-05-20T01:16:11.203 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.195+0000 7f6132ffd700 1 -- 172.21.15.132:0/3139357202 <== mon.1 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f612c013480 con 0x7f613c15cec0 2023-05-20T01:16:11.203 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.195+0000 7f6132ffd700 1 -- 172.21.15.132:0/3139357202 <== mon.1 v2:172.21.15.132:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7f612c01d9c0 con 0x7f613c15cec0 2023-05-20T01:16:11.203 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.195+0000 7f6144642700 1 -- 172.21.15.132:0/3139357202 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f613c160890 con 0x7f613c15cec0 2023-05-20T01:16:11.204 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.197+0000 7f6132ffd700 1 -- 172.21.15.132:0/3139357202 <== mon.1 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72437 (secure 0 0 0) 0x7f612c01d560 con 0x7f613c15cec0 2023-05-20T01:16:11.283 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.280+0000 7f6144642700 1 -- 172.21.15.132:0/3139357202 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd new", "uuid": "b8909c3a-cf70-46cb-8d0b-009c707cf9d3", "id": 4} v 0) v1 -- 0x7f613c0747b0 con 0x7f613c15cec0 2023-05-20T01:16:11.286 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.283+0000 7f6132ffd700 1 -- 172.21.15.132:0/3139357202 <== mon.1 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b8909c3a-cf70-46cb-8d0b-009c707cf9d3", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f612c02c0b0 con 0x7f613c15cec0 2023-05-20T01:16:11.286 INFO:teuthology.orchestra.run.smithi132.stdout:4 2023-05-20T01:16:11.287 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.284+0000 7f6144642700 1 -- 172.21.15.132:0/3139357202 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f613c15cec0 msgr2=0x7f613c151a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:11.288 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.284+0000 7f6144642700 1 --2- 172.21.15.132:0/3139357202 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f613c15cec0 0x7f613c151a40 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f612c00d800 tx=0x7f612c00db10 comp rx=0 tx=0).stop 2023-05-20T01:16:11.288 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.284+0000 7f6144642700 1 -- 172.21.15.132:0/3139357202 shutdown_connections 2023-05-20T01:16:11.288 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.284+0000 7f6144642700 1 --2- 172.21.15.132:0/3139357202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f613c15d290 0x7f613c15aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:11.288 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.284+0000 7f6144642700 1 --2- 172.21.15.132:0/3139357202 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f613c0cf2c0 0x7f613c0c9c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:11.288 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.284+0000 7f6144642700 1 --2- 172.21.15.132:0/3139357202 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f613c15cec0 0x7f613c151a40 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:11.288 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.284+0000 7f6144642700 1 -- 172.21.15.132:0/3139357202 >> 172.21.15.132:0/3139357202 conn(0x7f613c0cffd0 msgr2=0x7f613c0cac50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:11.288 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.284+0000 7f6144642700 1 -- 172.21.15.132:0/3139357202 shutdown_connections 2023-05-20T01:16:11.289 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.284+0000 7f6144642700 1 -- 172.21.15.132:0/3139357202 wait complete. 2023-05-20T01:16:11.299 DEBUG:teuthology.orchestra.run.smithi132:> sudo ceph --cluster ceph osd new b1818304-4583-47bf-a6ec-0bfbb5166233 5 2023-05-20T01:16:11.458 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.455+0000 7fe47871b700 1 Processor -- start 2023-05-20T01:16:11.459 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.455+0000 7fe47871b700 1 -- start start 2023-05-20T01:16:11.460 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.456+0000 7fe47871b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe4700cd1c0 0x7fe4700710b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:11.461 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.456+0000 7fe47871b700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe4700cd630 0x7fe4700715f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:11.461 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.456+0000 7fe47871b700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4700ccd30 con 0x7fe470159b50 2023-05-20T01:16:11.461 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.456+0000 7fe47871b700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4700ce990 con 0x7fe4700cd1c0 2023-05-20T01:16:11.461 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.456+0000 7fe47871b700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4700cc6e0 con 0x7fe4700cd630 2023-05-20T01:16:11.461 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.457+0000 7fe476cb8700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe4700cd630 0x7fe4700715f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:11.461 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.457+0000 7fe476cb8700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe4700cd630 0x7fe4700715f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.132:51636/0 (socket says 172.21.15.132:51636) 2023-05-20T01:16:11.462 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.457+0000 7fe476cb8700 1 -- 172.21.15.132:0/2205502478 learned_addr learned my addr 172.21.15.132:0/2205502478 (peer_addr_for_me v2:172.21.15.132:0/0) 2023-05-20T01:16:11.462 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.457+0000 7fe475cb6700 1 --2- 172.21.15.132:0/2205502478 >> v2:172.21.15.16:3300/0 conn(0x7fe4700cd1c0 0x7fe4700710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:11.462 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.457+0000 7fe476cb8700 1 -- 172.21.15.132:0/2205502478 >> v2:172.21.15.16:3300/0 conn(0x7fe4700cd1c0 msgr2=0x7fe4700710b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:11.462 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.457+0000 7fe476cb8700 1 --2- 172.21.15.132:0/2205502478 >> v2:172.21.15.16:3300/0 conn(0x7fe4700cd1c0 0x7fe4700710b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:11.462 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.457+0000 7fe476cb8700 1 -- 172.21.15.132:0/2205502478 >> v1:172.21.15.16:6789/0 conn(0x7fe470159b50 legacy=0x7fe47006e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:11.463 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.457+0000 7fe476cb8700 1 -- 172.21.15.132:0/2205502478 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4700cc9c0 con 0x7fe4700cd630 2023-05-20T01:16:11.463 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.457+0000 7fe476cb8700 1 --2- 172.21.15.132:0/2205502478 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe4700cd630 0x7fe4700715f0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe468009e50 tx=0x7fe46800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df496f2d9f2222d3 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:11.463 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.457+0000 7fe4754b5700 1 -- 172.21.15.132:0/2205502478 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe46800e070 con 0x7fe4700cd630 2023-05-20T01:16:11.463 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.457+0000 7fe4754b5700 1 -- 172.21.15.132:0/2205502478 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe46800b910 con 0x7fe4700cd630 2023-05-20T01:16:11.463 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.457+0000 7fe4754b5700 1 -- 172.21.15.132:0/2205502478 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe46800bcc0 con 0x7fe4700cd630 2023-05-20T01:16:11.466 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.458+0000 7fe47871b700 1 -- 172.21.15.132:0/2205502478 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe4700cd630 msgr2=0x7fe4700715f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:11.466 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.458+0000 7fe47871b700 1 --2- 172.21.15.132:0/2205502478 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe4700cd630 0x7fe4700715f0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe468009e50 tx=0x7fe46800b330 comp rx=0 tx=0).stop 2023-05-20T01:16:11.466 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.458+0000 7fe47871b700 1 -- 172.21.15.132:0/2205502478 shutdown_connections 2023-05-20T01:16:11.466 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.458+0000 7fe47871b700 1 --2- 172.21.15.132:0/2205502478 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe4700cd630 0x7fe4700715f0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:11.466 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.458+0000 7fe47871b700 1 --2- 172.21.15.132:0/2205502478 >> v2:172.21.15.16:3300/0 conn(0x7fe4700cd1c0 0x7fe4700710b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:11.466 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.458+0000 7fe47871b700 1 -- 172.21.15.132:0/2205502478 >> 172.21.15.132:0/2205502478 conn(0x7fe4700cffd0 msgr2=0x7fe4700d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:11.467 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.458+0000 7fe47871b700 1 -- 172.21.15.132:0/2205502478 shutdown_connections 2023-05-20T01:16:11.467 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.458+0000 7fe47871b700 1 -- 172.21.15.132:0/2205502478 wait complete. 2023-05-20T01:16:11.467 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.459+0000 7fe47871b700 1 Processor -- start 2023-05-20T01:16:11.467 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.459+0000 7fe47871b700 1 -- start start 2023-05-20T01:16:11.467 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.460+0000 7fe47871b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4700cd1c0 0x7fe4701510c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:11.467 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.460+0000 7fe47871b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe4700cd630 0x7fe4700cb000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:11.467 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.460+0000 7fe47871b700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe470159b50 0x7fe4700cb540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:11.468 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.460+0000 7fe47871b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4700c7e50 con 0x7fe4700cd1c0 2023-05-20T01:16:11.468 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.460+0000 7fe47871b700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe470151600 con 0x7fe4700cd630 2023-05-20T01:16:11.468 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.460+0000 7fe47871b700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4700c7240 con 0x7fe470159b50 2023-05-20T01:16:11.468 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.460+0000 7fe476cb8700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe470159b50 0x7fe4700cb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:11.468 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.460+0000 7fe475cb6700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe4700cd630 0x7fe4700cb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:11.468 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.460+0000 7fe476cb8700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe470159b50 0x7fe4700cb540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.132:51642/0 (socket says 172.21.15.132:51642) 2023-05-20T01:16:11.468 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.460+0000 7fe476cb8700 1 -- 172.21.15.132:0/481719701 learned_addr learned my addr 172.21.15.132:0/481719701 (peer_addr_for_me v2:172.21.15.132:0/0) 2023-05-20T01:16:11.468 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.460+0000 7fe4764b7700 1 --2- 172.21.15.132:0/481719701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4700cd1c0 0x7fe4701510c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:11.469 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.460+0000 7fe476cb8700 1 -- 172.21.15.132:0/481719701 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe4700cd630 msgr2=0x7fe4700cb000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:11.469 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.460+0000 7fe476cb8700 1 --2- 172.21.15.132:0/481719701 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe4700cd630 0x7fe4700cb000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:11.469 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.460+0000 7fe476cb8700 1 -- 172.21.15.132:0/481719701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4700cd1c0 msgr2=0x7fe4701510c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:11.469 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.460+0000 7fe476cb8700 1 --2- 172.21.15.132:0/481719701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4700cd1c0 0x7fe4701510c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:11.469 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.460+0000 7fe476cb8700 1 -- 172.21.15.132:0/481719701 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe468009b30 con 0x7fe470159b50 2023-05-20T01:16:11.469 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.460+0000 7fe475cb6700 1 --2- 172.21.15.132:0/481719701 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe4700cd630 0x7fe4700cb000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-05-20T01:16:11.469 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.460+0000 7fe476cb8700 1 --2- 172.21.15.132:0/481719701 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe470159b50 0x7fe4700cb540 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fe46800b680 tx=0x7fe468004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:11.470 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.461+0000 7fe462ffd700 1 -- 172.21.15.132:0/481719701 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe468004830 con 0x7fe470159b50 2023-05-20T01:16:11.470 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.461+0000 7fe462ffd700 1 -- 172.21.15.132:0/481719701 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4680092e0 con 0x7fe470159b50 2023-05-20T01:16:11.470 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.461+0000 7fe47871b700 1 -- 172.21.15.132:0/481719701 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4700cec70 con 0x7fe470159b50 2023-05-20T01:16:11.470 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.461+0000 7fe462ffd700 1 -- 172.21.15.132:0/481719701 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe46801c680 con 0x7fe470159b50 2023-05-20T01:16:11.470 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.461+0000 7fe47871b700 1 -- 172.21.15.132:0/481719701 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe470071e80 con 0x7fe470159b50 2023-05-20T01:16:11.470 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.461+0000 7fe462ffd700 1 -- 172.21.15.132:0/481719701 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fe46801cae0 con 0x7fe470159b50 2023-05-20T01:16:11.470 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.461+0000 7fe462ffd700 1 -- 172.21.15.132:0/481719701 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7fe4680123f0 con 0x7fe470159b50 2023-05-20T01:16:11.470 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.461+0000 7fe47871b700 1 -- 172.21.15.132:0/481719701 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4700cf550 con 0x7fe470159b50 2023-05-20T01:16:11.471 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.465+0000 7fe462ffd700 1 -- 172.21.15.132:0/481719701 <== mon.2 v2:172.21.15.132:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72437 (secure 0 0 0) 0x7fe468020070 con 0x7fe470159b50 2023-05-20T01:16:11.552 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.549+0000 7fe47871b700 1 -- 172.21.15.132:0/481719701 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "osd new", "uuid": "b1818304-4583-47bf-a6ec-0bfbb5166233", "id": 5} v 0) v1 -- 0x7fe4700747b0 con 0x7fe470159b50 2023-05-20T01:16:11.556 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.553+0000 7fe462ffd700 1 -- 172.21.15.132:0/481719701 <== mon.2 v2:172.21.15.132:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b1818304-4583-47bf-a6ec-0bfbb5166233", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7fe468025450 con 0x7fe470159b50 2023-05-20T01:16:11.556 INFO:teuthology.orchestra.run.smithi132.stdout:5 2023-05-20T01:16:11.557 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.554+0000 7fe47871b700 1 -- 172.21.15.132:0/481719701 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe470159b50 msgr2=0x7fe4700cb540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:11.557 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.554+0000 7fe47871b700 1 --2- 172.21.15.132:0/481719701 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe470159b50 0x7fe4700cb540 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fe46800b680 tx=0x7fe468004ad0 comp rx=0 tx=0).stop 2023-05-20T01:16:11.557 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.554+0000 7fe47871b700 1 -- 172.21.15.132:0/481719701 shutdown_connections 2023-05-20T01:16:11.558 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.554+0000 7fe47871b700 1 --2- 172.21.15.132:0/481719701 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe470159b50 0x7fe4700cb540 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:11.558 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.554+0000 7fe47871b700 1 --2- 172.21.15.132:0/481719701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4700cd1c0 0x7fe4701510c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:11.558 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.554+0000 7fe47871b700 1 --2- 172.21.15.132:0/481719701 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe4700cd630 0x7fe4700cb000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:11.558 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.554+0000 7fe47871b700 1 -- 172.21.15.132:0/481719701 >> 172.21.15.132:0/481719701 conn(0x7fe4700cffd0 msgr2=0x7fe4700b5210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:11.558 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.554+0000 7fe47871b700 1 -- 172.21.15.132:0/481719701 shutdown_connections 2023-05-20T01:16:11.559 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.554+0000 7fe47871b700 1 -- 172.21.15.132:0/481719701 wait complete. 2023-05-20T01:16:11.569 DEBUG:teuthology.orchestra.run.smithi132:> sudo ceph --cluster ceph osd new d37f4691-eed1-4a93-8818-18354f03f4a4 6 2023-05-20T01:16:11.573 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-05-20T01:16:11.570+0000 7f1da015b200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-05-20T01:16:11.692 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.689+0000 7fd330194700 1 Processor -- start 2023-05-20T01:16:11.693 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.690+0000 7fd330194700 1 -- start start 2023-05-20T01:16:11.694 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.690+0000 7fd330194700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd32815cec0 0x7fd328071bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:11.694 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.690+0000 7fd330194700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd32815d290 0x7fd328072130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:11.695 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.690+0000 7fd330194700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3280cfe30 con 0x7fd3280cf2c0 2023-05-20T01:16:11.695 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.690+0000 7fd330194700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3280cd630 con 0x7fd32815d290 2023-05-20T01:16:11.695 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.690+0000 7fd330194700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3280cca60 con 0x7fd32815cec0 2023-05-20T01:16:11.695 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.691+0000 7fd32df30700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd32815cec0 0x7fd328071bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:11.695 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.691+0000 7fd32e731700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7fd3280cf2c0 0x7fd3280cf720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.132:38878/0 (socket says 172.21.15.132:38878) 2023-05-20T01:16:11.695 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.691+0000 7fd32e731700 1 -- 172.21.15.132:0/1350562499 learned_addr learned my addr 172.21.15.132:0/1350562499 (peer_addr_for_me v1:172.21.15.132:0/0) 2023-05-20T01:16:11.695 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.691+0000 7fd32d72f700 1 --2- 172.21.15.132:0/1350562499 >> v2:172.21.15.16:3300/0 conn(0x7fd32815d290 0x7fd328072130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:11.696 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.691+0000 7fd32cf2e700 1 -- 172.21.15.132:0/1350562499 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1012853530 0 0) 0x7fd320001490 con 0x7fd3280cf2c0 2023-05-20T01:16:11.696 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.691+0000 7fd32df30700 1 -- 172.21.15.132:0/1350562499 >> v2:172.21.15.16:3300/0 conn(0x7fd32815d290 msgr2=0x7fd328072130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:11.696 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.691+0000 7fd32df30700 1 --2- 172.21.15.132:0/1350562499 >> v2:172.21.15.16:3300/0 conn(0x7fd32815d290 0x7fd328072130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:11.696 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.691+0000 7fd32df30700 1 -- 172.21.15.132:0/1350562499 >> v1:172.21.15.132:6789/0 conn(0x7fd3280cf2c0 legacy=0x7fd3280cf720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:11.696 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.691+0000 7fd32df30700 1 -- 172.21.15.132:0/1350562499 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3280ccd00 con 0x7fd32815cec0 2023-05-20T01:16:11.696 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.691+0000 7fd32df30700 1 --2- 172.21.15.132:0/1350562499 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd32815cec0 0x7fd328071bf0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fd318009820 tx=0x7fd318009b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd56b4f999b01444 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:11.696 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.691+0000 7fd32cf2e700 1 -- 172.21.15.132:0/1350562499 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd318005e30 con 0x7fd32815cec0 2023-05-20T01:16:11.697 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.691+0000 7fd32cf2e700 1 -- 172.21.15.132:0/1350562499 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd318006030 con 0x7fd32815cec0 2023-05-20T01:16:11.697 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.691+0000 7fd32cf2e700 1 -- 172.21.15.132:0/1350562499 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3180063c0 con 0x7fd32815cec0 2023-05-20T01:16:11.699 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.692+0000 7fd330194700 1 -- 172.21.15.132:0/1350562499 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd32815cec0 msgr2=0x7fd328071bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:11.699 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.692+0000 7fd330194700 1 --2- 172.21.15.132:0/1350562499 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd32815cec0 0x7fd328071bf0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fd318009820 tx=0x7fd318009b30 comp rx=0 tx=0).stop 2023-05-20T01:16:11.699 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.692+0000 7fd330194700 1 -- 172.21.15.132:0/1350562499 shutdown_connections 2023-05-20T01:16:11.699 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.692+0000 7fd330194700 1 --2- 172.21.15.132:0/1350562499 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd32815cec0 0x7fd328071bf0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:11.699 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.692+0000 7fd330194700 1 --2- 172.21.15.132:0/1350562499 >> v2:172.21.15.16:3300/0 conn(0x7fd32815d290 0x7fd328072130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:11.699 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.692+0000 7fd330194700 1 -- 172.21.15.132:0/1350562499 >> 172.21.15.132:0/1350562499 conn(0x7fd3280cffd0 msgr2=0x7fd3280d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:11.700 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.692+0000 7fd330194700 1 -- 172.21.15.132:0/1350562499 shutdown_connections 2023-05-20T01:16:11.700 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.692+0000 7fd330194700 1 -- 172.21.15.132:0/1350562499 wait complete. 2023-05-20T01:16:11.700 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.693+0000 7fd330194700 1 Processor -- start 2023-05-20T01:16:11.700 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.693+0000 7fd330194700 1 -- start start 2023-05-20T01:16:11.700 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.694+0000 7fd330194700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3280cf2c0 0x7fd328152470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:11.700 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.694+0000 7fd330194700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd32815cec0 0x7fd3280c70a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:11.700 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.694+0000 7fd330194700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd32815d290 0x7fd3280c75e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:11.701 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.694+0000 7fd330194700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3280c84d0 con 0x7fd3280cf2c0 2023-05-20T01:16:11.701 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.694+0000 7fd330194700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd328155e40 con 0x7fd32815cec0 2023-05-20T01:16:11.701 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.694+0000 7fd330194700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3280c7c60 con 0x7fd32815d290 2023-05-20T01:16:11.701 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.694+0000 7fd32d72f700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd32815cec0 0x7fd3280c70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:11.701 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.694+0000 7fd32d72f700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd32815cec0 0x7fd3280c70a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:57924/0 (socket says 172.21.15.132:57924) 2023-05-20T01:16:11.701 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.694+0000 7fd32e731700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd32815d290 0x7fd3280c75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:11.701 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.694+0000 7fd32d72f700 1 -- 172.21.15.132:0/3485258008 learned_addr learned my addr 172.21.15.132:0/3485258008 (peer_addr_for_me v2:172.21.15.132:0/0) 2023-05-20T01:16:11.702 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.694+0000 7fd32df30700 1 --2- 172.21.15.132:0/3485258008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3280cf2c0 0x7fd328152470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:11.702 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.694+0000 7fd32d72f700 1 -- 172.21.15.132:0/3485258008 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd32815d290 msgr2=0x7fd3280c75e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:11.702 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.694+0000 7fd32d72f700 1 --2- 172.21.15.132:0/3485258008 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd32815d290 0x7fd3280c75e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:11.702 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.694+0000 7fd32d72f700 1 -- 172.21.15.132:0/3485258008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3280cf2c0 msgr2=0x7fd328152470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:11.702 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.694+0000 7fd32d72f700 1 --2- 172.21.15.132:0/3485258008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3280cf2c0 0x7fd328152470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:11.702 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.694+0000 7fd32d72f700 1 -- 172.21.15.132:0/3485258008 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd318009430 con 0x7fd32815cec0 2023-05-20T01:16:11.702 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.694+0000 7fd32d72f700 1 --2- 172.21.15.132:0/3485258008 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd32815cec0 0x7fd3280c70a0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd32400e870 tx=0x7fd32400eb80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:11.703 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.695+0000 7fd31e7fc700 1 -- 172.21.15.132:0/3485258008 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd324009590 con 0x7fd32815cec0 2023-05-20T01:16:11.703 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.695+0000 7fd31e7fc700 1 -- 172.21.15.132:0/3485258008 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd324013680 con 0x7fd32815cec0 2023-05-20T01:16:11.703 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.695+0000 7fd31e7fc700 1 -- 172.21.15.132:0/3485258008 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3240139f0 con 0x7fd32815cec0 2023-05-20T01:16:11.703 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.695+0000 7fd330194700 1 -- 172.21.15.132:0/3485258008 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3280cd2c0 con 0x7fd32815cec0 2023-05-20T01:16:11.703 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.695+0000 7fd330194700 1 -- 172.21.15.132:0/3485258008 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3280ccfe0 con 0x7fd32815cec0 2023-05-20T01:16:11.703 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.695+0000 7fd31e7fc700 1 -- 172.21.15.132:0/3485258008 <== mon.1 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fd324005a20 con 0x7fd32815cec0 2023-05-20T01:16:11.703 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.695+0000 7fd31e7fc700 1 -- 172.21.15.132:0/3485258008 <== mon.1 v2:172.21.15.132:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7fd3240066b0 con 0x7fd32815cec0 2023-05-20T01:16:11.703 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.695+0000 7fd330194700 1 -- 172.21.15.132:0/3485258008 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd328160890 con 0x7fd32815cec0 2023-05-20T01:16:11.704 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.698+0000 7fd31e7fc700 1 -- 172.21.15.132:0/3485258008 <== mon.1 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72437 (secure 0 0 0) 0x7fd324021060 con 0x7fd32815cec0 2023-05-20T01:16:11.783 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-05-20T01:16:11.781+0000 7fa73c17e200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-05-20T01:16:11.784 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.781+0000 7fd330194700 1 -- 172.21.15.132:0/3485258008 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd new", "uuid": "d37f4691-eed1-4a93-8818-18354f03f4a4", "id": 6} v 0) v1 -- 0x7fd3280747b0 con 0x7fd32815cec0 2023-05-20T01:16:11.794 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.791+0000 7fd31e7fc700 1 -- 172.21.15.132:0/3485258008 <== mon.1 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d37f4691-eed1-4a93-8818-18354f03f4a4", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7fd324017070 con 0x7fd32815cec0 2023-05-20T01:16:11.795 INFO:teuthology.orchestra.run.smithi132.stdout:6 2023-05-20T01:16:11.795 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.792+0000 7fd330194700 1 -- 172.21.15.132:0/3485258008 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd32815cec0 msgr2=0x7fd3280c70a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:11.796 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.792+0000 7fd330194700 1 --2- 172.21.15.132:0/3485258008 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd32815cec0 0x7fd3280c70a0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd32400e870 tx=0x7fd32400eb80 comp rx=0 tx=0).stop 2023-05-20T01:16:11.796 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.792+0000 7fd330194700 1 -- 172.21.15.132:0/3485258008 shutdown_connections 2023-05-20T01:16:11.796 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.792+0000 7fd330194700 1 --2- 172.21.15.132:0/3485258008 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd32815d290 0x7fd3280c75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:11.796 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.792+0000 7fd330194700 1 --2- 172.21.15.132:0/3485258008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3280cf2c0 0x7fd328152470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:11.796 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.792+0000 7fd330194700 1 --2- 172.21.15.132:0/3485258008 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd32815cec0 0x7fd3280c70a0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:11.796 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.792+0000 7fd330194700 1 -- 172.21.15.132:0/3485258008 >> 172.21.15.132:0/3485258008 conn(0x7fd3280cffd0 msgr2=0x7fd3280cf720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:11.797 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.792+0000 7fd330194700 1 -- 172.21.15.132:0/3485258008 shutdown_connections 2023-05-20T01:16:11.797 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.792+0000 7fd330194700 1 -- 172.21.15.132:0/3485258008 wait complete. 2023-05-20T01:16:11.807 DEBUG:teuthology.orchestra.run.smithi132:> sudo ceph --cluster ceph osd new 4f712f62-5d1d-4816-8042-03209144f8b0 7 2023-05-20T01:16:11.945 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-05-20T01:16:11.941+0000 7fa73c17e200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-05-20T01:16:11.973 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.970+0000 7f408a983700 1 Processor -- start 2023-05-20T01:16:11.974 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.970+0000 7f408a983700 1 -- start start 2023-05-20T01:16:11.975 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.971+0000 7f408a983700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f40840cbfb0 0x7f4084165510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:11.975 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.971+0000 7f408a983700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40840d0e10 con 0x7f4084071880 2023-05-20T01:16:11.975 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.971+0000 7f408a983700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40840cc580 con 0x7f40840cd620 2023-05-20T01:16:11.976 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.971+0000 7f408a983700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40840cd200 con 0x7f40840cbfb0 2023-05-20T01:16:11.978 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.972+0000 7f40837fe700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f40840cd620 0x7f40840cb8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.132:38890/0 (socket says 172.21.15.132:38890) 2023-05-20T01:16:11.978 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.972+0000 7f40837fe700 1 -- 172.21.15.132:0/3924303122 learned_addr learned my addr 172.21.15.132:0/3924303122 (peer_addr_for_me v1:172.21.15.132:0/0) 2023-05-20T01:16:11.979 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.972+0000 7f4088f20700 1 --2- 172.21.15.132:0/3924303122 >> v2:172.21.15.16:3300/0 conn(0x7f40840cbfb0 0x7f4084165510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:11.979 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.973+0000 7f4082ffd700 1 -- 172.21.15.132:0/3924303122 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1276840113 0 0) 0x7f40840cc580 con 0x7f40840cd620 2023-05-20T01:16:11.979 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.973+0000 7f4082ffd700 1 -- 172.21.15.132:0/3924303122 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f406c0035b0 con 0x7f40840cd620 2023-05-20T01:16:11.979 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.973+0000 7f4082ffd700 1 -- 172.21.15.132:0/3924303122 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1893133324 0 0) 0x7f40840d0e10 con 0x7f4084071880 2023-05-20T01:16:11.979 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.973+0000 7f4082ffd700 1 -- 172.21.15.132:0/3924303122 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f40840cc580 con 0x7f4084071880 2023-05-20T01:16:11.979 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.973+0000 7f4082ffd700 1 -- 172.21.15.132:0/3924303122 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 463969222 0 0) 0x7f40840cc580 con 0x7f4084071880 2023-05-20T01:16:11.980 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.973+0000 7f4088f20700 1 -- 172.21.15.132:0/3924303122 >> v1:172.21.15.132:6789/0 conn(0x7f40840cd620 legacy=0x7f40840cb8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:11.980 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.973+0000 7f4088f20700 1 -- 172.21.15.132:0/3924303122 >> v1:172.21.15.16:6789/0 conn(0x7f4084071880 legacy=0x7f4084071c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:11.980 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.973+0000 7f4088f20700 1 -- 172.21.15.132:0/3924303122 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40840cc860 con 0x7f40840cbfb0 2023-05-20T01:16:11.980 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.973+0000 7f4088f20700 1 --2- 172.21.15.132:0/3924303122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40840cbfb0 0x7f4084165510 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f4078009840 tx=0x7f4078009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2e00232d5e3806a server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:11.980 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.973+0000 7f4082ffd700 1 -- 172.21.15.132:0/3924303122 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f407c001a40 con 0x7f40840cbfb0 2023-05-20T01:16:11.980 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.973+0000 7f4082ffd700 1 -- 172.21.15.132:0/3924303122 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4078004030 con 0x7f40840cbfb0 2023-05-20T01:16:11.981 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.974+0000 7f4082ffd700 1 -- 172.21.15.132:0/3924303122 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40780043c0 con 0x7f40840cbfb0 2023-05-20T01:16:11.981 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.974+0000 7f408a983700 1 -- 172.21.15.132:0/3924303122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40840cbfb0 msgr2=0x7f4084165510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:11.981 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.974+0000 7f408a983700 1 --2- 172.21.15.132:0/3924303122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40840cbfb0 0x7f4084165510 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f4078009840 tx=0x7f4078009b50 comp rx=0 tx=0).stop 2023-05-20T01:16:11.981 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.974+0000 7f408a983700 1 -- 172.21.15.132:0/3924303122 shutdown_connections 2023-05-20T01:16:11.981 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.974+0000 7f408a983700 1 --2- 172.21.15.132:0/3924303122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40840cbfb0 0x7f4084165510 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:11.981 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.974+0000 7f408a983700 1 -- 172.21.15.132:0/3924303122 >> 172.21.15.132:0/3924303122 conn(0x7f40840cffd0 msgr2=0x7f40840d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:11.981 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.974+0000 7f408a983700 1 -- 172.21.15.132:0/3924303122 shutdown_connections 2023-05-20T01:16:11.983 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.974+0000 7f408a983700 1 -- 172.21.15.132:0/3924303122 wait complete. 2023-05-20T01:16:11.983 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.976+0000 7f408a983700 1 Processor -- start 2023-05-20T01:16:11.983 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.976+0000 7f408a983700 1 -- start start 2023-05-20T01:16:11.983 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.978+0000 7f408a983700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4084071880 0x7f40840c3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:11.983 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.978+0000 7f408a983700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40840cbfb0 0x7f4084156dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:11.983 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.978+0000 7f408a983700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f40840cd620 0x7f40840b0980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:11.984 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.978+0000 7f408a983700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40840cdba0 con 0x7f40840cbfb0 2023-05-20T01:16:11.984 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.978+0000 7f408a983700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40840c4020 con 0x7f4084071880 2023-05-20T01:16:11.984 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.978+0000 7f408a983700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40841f6cf0 con 0x7f40840cd620 2023-05-20T01:16:11.984 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.978+0000 7f4083fff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4084071880 0x7f40840c3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:11.984 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.978+0000 7f4083fff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4084071880 0x7f40840c3ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:57934/0 (socket says 172.21.15.132:57934) 2023-05-20T01:16:11.984 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.978+0000 7f4083fff700 1 -- 172.21.15.132:0/2816017387 learned_addr learned my addr 172.21.15.132:0/2816017387 (peer_addr_for_me v2:172.21.15.132:0/0) 2023-05-20T01:16:11.985 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.978+0000 7f4088f20700 1 --2- 172.21.15.132:0/2816017387 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f40840cd620 0x7f40840b0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:11.985 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.978+0000 7f40837fe700 1 --2- 172.21.15.132:0/2816017387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40840cbfb0 0x7f4084156dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:11.985 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.978+0000 7f4083fff700 1 -- 172.21.15.132:0/2816017387 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f40840cd620 msgr2=0x7f40840b0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:11.985 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.978+0000 7f4083fff700 1 --2- 172.21.15.132:0/2816017387 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f40840cd620 0x7f40840b0980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:11.985 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.978+0000 7f4083fff700 1 -- 172.21.15.132:0/2816017387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40840cbfb0 msgr2=0x7f4084156dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:11.985 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.978+0000 7f4083fff700 1 --2- 172.21.15.132:0/2816017387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40840cbfb0 0x7f4084156dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:11.986 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.978+0000 7f4083fff700 1 -- 172.21.15.132:0/2816017387 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40780094a0 con 0x7f4084071880 2023-05-20T01:16:11.986 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.978+0000 7f40837fe700 1 --2- 172.21.15.132:0/2816017387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40840cbfb0 0x7f4084156dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:16:11.986 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.978+0000 7f4083fff700 1 --2- 172.21.15.132:0/2816017387 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4084071880 0x7f40840c3ae0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f407c00c7c0 tx=0x7f407c00cad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:11.986 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.978+0000 7f4080ff9700 1 -- 172.21.15.132:0/2816017387 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f407c003e10 con 0x7f4084071880 2023-05-20T01:16:11.986 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.979+0000 7f4080ff9700 1 -- 172.21.15.132:0/2816017387 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f407c005070 con 0x7f4084071880 2023-05-20T01:16:11.986 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.979+0000 7f4080ff9700 1 -- 172.21.15.132:0/2816017387 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f407c005420 con 0x7f4084071880 2023-05-20T01:16:11.986 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.979+0000 7f408a983700 1 -- 172.21.15.132:0/2816017387 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40840cce20 con 0x7f4084071880 2023-05-20T01:16:11.987 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.979+0000 7f408a983700 1 -- 172.21.15.132:0/2816017387 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40840ccb40 con 0x7f4084071880 2023-05-20T01:16:11.987 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.979+0000 7f4080ff9700 1 -- 172.21.15.132:0/2816017387 <== mon.1 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f407c012400 con 0x7f4084071880 2023-05-20T01:16:11.987 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.979+0000 7f4080ff9700 1 -- 172.21.15.132:0/2816017387 <== mon.1 v2:172.21.15.132:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7f407c01bb60 con 0x7f4084071880 2023-05-20T01:16:11.987 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.980+0000 7f408a983700 1 -- 172.21.15.132:0/2816017387 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f408415acf0 con 0x7f4084071880 2023-05-20T01:16:11.988 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:11.983+0000 7f4080ff9700 1 -- 172.21.15.132:0/2816017387 <== mon.1 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72437 (secure 0 0 0) 0x7f407c021060 con 0x7f4084071880 2023-05-20T01:16:12.094 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:12.090+0000 7f408a983700 1 -- 172.21.15.132:0/2816017387 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd new", "uuid": "4f712f62-5d1d-4816-8042-03209144f8b0", "id": 7} v 0) v1 -- 0x7f40840cdd10 con 0x7f4084071880 2023-05-20T01:16:12.101 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:12.098+0000 7f4080ff9700 1 -- 172.21.15.132:0/2816017387 <== mon.1 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "4f712f62-5d1d-4816-8042-03209144f8b0", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f40840cdd10 con 0x7f4084071880 2023-05-20T01:16:12.102 INFO:teuthology.orchestra.run.smithi132.stdout:7 2023-05-20T01:16:12.103 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:12.099+0000 7f408a983700 1 -- 172.21.15.132:0/2816017387 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4084071880 msgr2=0x7f40840c3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:12.103 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:12.099+0000 7f408a983700 1 --2- 172.21.15.132:0/2816017387 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4084071880 0x7f40840c3ae0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f407c00c7c0 tx=0x7f407c00cad0 comp rx=0 tx=0).stop 2023-05-20T01:16:12.103 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:12.099+0000 7f408a983700 1 -- 172.21.15.132:0/2816017387 shutdown_connections 2023-05-20T01:16:12.103 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:12.099+0000 7f408a983700 1 --2- 172.21.15.132:0/2816017387 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f40840cd620 0x7f40840b0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:12.103 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:12.099+0000 7f408a983700 1 --2- 172.21.15.132:0/2816017387 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4084071880 0x7f40840c3ae0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:12.103 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:12.099+0000 7f408a983700 1 --2- 172.21.15.132:0/2816017387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40840cbfb0 0x7f4084156dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:12.104 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:12.099+0000 7f408a983700 1 -- 172.21.15.132:0/2816017387 >> 172.21.15.132:0/2816017387 conn(0x7f40840cffd0 msgr2=0x7f408415cfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:12.104 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:12.100+0000 7f408a983700 1 -- 172.21.15.132:0/2816017387 shutdown_connections 2023-05-20T01:16:12.104 INFO:teuthology.orchestra.run.smithi132.stderr:2023-05-20T01:16:12.100+0000 7f408a983700 1 -- 172.21.15.132:0/2816017387 wait complete. 2023-05-20T01:16:12.115 INFO:tasks.ceph.osd.0:Restarting daemon 2023-05-20T01:16:12.116 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-05-20T01:16:12.118 INFO:tasks.ceph.osd.0:Started 2023-05-20T01:16:12.119 INFO:tasks.ceph.osd.1:Restarting daemon 2023-05-20T01:16:12.119 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-05-20T01:16:12.121 INFO:tasks.ceph.osd.1:Started 2023-05-20T01:16:12.122 INFO:tasks.ceph.osd.2:Restarting daemon 2023-05-20T01:16:12.122 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-05-20T01:16:12.124 INFO:tasks.ceph.osd.2:Started 2023-05-20T01:16:12.124 INFO:tasks.ceph.osd.3:Restarting daemon 2023-05-20T01:16:12.124 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-05-20T01:16:12.126 INFO:tasks.ceph.osd.3:Started 2023-05-20T01:16:12.127 INFO:tasks.ceph.osd.4:Restarting daemon 2023-05-20T01:16:12.127 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-05-20T01:16:12.129 INFO:tasks.ceph.osd.4:Started 2023-05-20T01:16:12.129 INFO:tasks.ceph.osd.5:Restarting daemon 2023-05-20T01:16:12.130 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-05-20T01:16:12.131 INFO:tasks.ceph.osd.5:Started 2023-05-20T01:16:12.132 INFO:tasks.ceph.osd.6:Restarting daemon 2023-05-20T01:16:12.133 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-05-20T01:16:12.134 INFO:tasks.ceph.osd.6:Started 2023-05-20T01:16:12.135 INFO:tasks.ceph.osd.7:Restarting daemon 2023-05-20T01:16:12.135 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-05-20T01:16:12.137 INFO:tasks.ceph.osd.7:Started 2023-05-20T01:16:12.138 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-20T01:16:12.169 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-05-20T01:16:12.166+0000 7f1da015b200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-05-20T01:16:12.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:12.297+0000 7f61b2da1700 1 Processor -- start 2023-05-20T01:16:12.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:12.297+0000 7f61b2da1700 1 -- start start 2023-05-20T01:16:12.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:12.298+0000 7f61b2da1700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f61ac06bd30 0x7f61ac0cdab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:12.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:12.298+0000 7f61b2da1700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61ac0cb6a0 con 0x7f61ac1466f0 2023-05-20T01:16:12.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:12.298+0000 7f61b2da1700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61ac0ce820 con 0x7f61ac0cdff0 2023-05-20T01:16:12.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:12.298+0000 7f61b2da1700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61ac0cb8f0 con 0x7f61ac06bd30 2023-05-20T01:16:12.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:12.298+0000 7f61b0b3d700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f61ac1466f0 0x7f61ac06b620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:40208/0 (socket says 172.21.15.16:40208) 2023-05-20T01:16:12.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:12.298+0000 7f61b0b3d700 1 -- 172.21.15.16:0/4088208873 learned_addr learned my addr 172.21.15.16:0/4088208873 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:16:12.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:12.298+0000 7f61abfff700 1 --2- 172.21.15.16:0/4088208873 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f61ac06bd30 0x7f61ac0cdab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:12.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:12.298+0000 7f61ab7fe700 1 -- 172.21.15.16:0/4088208873 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3697908615 0 0) 0x7f61ac0cb6a0 con 0x7f61ac1466f0 2023-05-20T01:16:12.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:12.298+0000 7f61ab7fe700 1 -- 172.21.15.16:0/4088208873 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f619c0035b0 con 0x7f61ac1466f0 2023-05-20T01:16:12.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:12.298+0000 7f61ab7fe700 1 -- 172.21.15.16:0/4088208873 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 602170274 0 0) 0x7f61ac0ce820 con 0x7f61ac0cdff0 2023-05-20T01:16:12.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:12.298+0000 7f61ab7fe700 1 -- 172.21.15.16:0/4088208873 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61ac0cb6a0 con 0x7f61ac0cdff0 2023-05-20T01:16:12.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:12.298+0000 7f61ab7fe700 1 -- 172.21.15.16:0/4088208873 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2632848064 0 0) 0x7f619c0035b0 con 0x7f61ac1466f0 2023-05-20T01:16:12.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:12.298+0000 7f61abfff700 1 -- 172.21.15.16:0/4088208873 >> v1:172.21.15.132:6789/0 conn(0x7f61ac0cdff0 legacy=0x7f61ac0cc3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:12.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:12.298+0000 7f61abfff700 1 -- 172.21.15.16:0/4088208873 >> v1:172.21.15.16:6789/0 conn(0x7f61ac1466f0 legacy=0x7f61ac06b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:12.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:12.298+0000 7f61abfff700 1 -- 172.21.15.16:0/4088208873 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61ac0cbbd0 con 0x7f61ac06bd30 2023-05-20T01:16:12.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:12.298+0000 7f61abfff700 1 --2- 172.21.15.16:0/4088208873 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f61ac06bd30 0x7f61ac0cdab0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f6198009870 tx=0x7f6198009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a89196613a0bd523 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:12.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:12.299+0000 7f61ab7fe700 1 -- 172.21.15.16:0/4088208873 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61a0001670 con 0x7f61ac06bd30 2023-05-20T01:16:12.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:12.299+0000 7f61ab7fe700 1 -- 172.21.15.16:0/4088208873 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6198004030 con 0x7f61ac06bd30 2023-05-20T01:16:12.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:12.299+0000 7f61ab7fe700 1 -- 172.21.15.16:0/4088208873 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61980043c0 con 0x7f61ac06bd30 2023-05-20T01:16:12.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:12.299+0000 7f61b2da1700 1 -- 172.21.15.16:0/4088208873 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f61ac06bd30 msgr2=0x7f61ac0cdab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:12.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:12.299+0000 7f61b2da1700 1 --2- 172.21.15.16:0/4088208873 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f61ac06bd30 0x7f61ac0cdab0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f6198009870 tx=0x7f6198009b80 comp rx=0 tx=0).stop 2023-05-20T01:16:12.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:12.299+0000 7f61b2da1700 1 -- 172.21.15.16:0/4088208873 shutdown_connections 2023-05-20T01:16:12.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:12.299+0000 7f61b2da1700 1 --2- 172.21.15.16:0/4088208873 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f61ac06bd30 0x7f61ac0cdab0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:12.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:12.299+0000 7f61b2da1700 1 -- 172.21.15.16:0/4088208873 >> 172.21.15.16:0/4088208873 conn(0x7f61ac06ae30 msgr2=0x7f61ac09a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:12.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:12.299+0000 7f61b2da1700 1 -- 172.21.15.16:0/4088208873 shutdown_connections 2023-05-20T01:16:12.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:12.299+0000 7f61b2da1700 1 -- 172.21.15.16:0/4088208873 wait complete. 2023-05-20T01:16:12.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:12.301+0000 7f61b2da1700 1 Processor -- start 2023-05-20T01:16:12.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:12.301+0000 7f61b2da1700 1 -- start start 2023-05-20T01:16:12.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:12.301+0000 7f61b2da1700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f61ac06bd30 0x7f61ac06f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:12.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:12.301+0000 7f61b2da1700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f61ac0cdff0 0x7f61ac1f1f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:12.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:12.301+0000 7f61b2da1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61ac1466f0 0x7f61ac157b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:12.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:12.301+0000 7f61b2da1700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61ac0c5ac0 con 0x7f61ac1466f0 2023-05-20T01:16:12.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:12.301+0000 7f61b2da1700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61ac0c4eb0 con 0x7f61ac0cdff0 2023-05-20T01:16:12.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:12.301+0000 7f61b2da1700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61ac0c5190 con 0x7f61ac06bd30 2023-05-20T01:16:12.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:12.302+0000 7f61b133e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61ac1466f0 0x7f61ac157b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:12.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:12.302+0000 7f61b0b3d700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f61ac06bd30 0x7f61ac06f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:12.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:12.302+0000 7f61b133e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61ac1466f0 0x7f61ac157b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56922/0 (socket says 172.21.15.16:56922) 2023-05-20T01:16:12.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:12.302+0000 7f61b0b3d700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f61ac06bd30 0x7f61ac06f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.16:56956/0 (socket says 172.21.15.16:56956) 2023-05-20T01:16:12.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:12.302+0000 7f61b133e700 1 -- 172.21.15.16:0/2946562605 learned_addr learned my addr 172.21.15.16:0/2946562605 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:12.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:12.302+0000 7f61abfff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f61ac0cdff0 0x7f61ac1f1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:12.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:12.302+0000 7f61b133e700 1 -- 172.21.15.16:0/2946562605 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f61ac06bd30 msgr2=0x7f61ac06f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:12.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:12.302+0000 7f61b133e700 1 --2- 172.21.15.16:0/2946562605 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f61ac06bd30 0x7f61ac06f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:12.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:12.302+0000 7f61b133e700 1 -- 172.21.15.16:0/2946562605 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f61ac0cdff0 msgr2=0x7f61ac1f1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:12.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:12.302+0000 7f61b133e700 1 --2- 172.21.15.16:0/2946562605 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f61ac0cdff0 0x7f61ac1f1f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:12.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:12.302+0000 7f61b133e700 1 -- 172.21.15.16:0/2946562605 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61980094a0 con 0x7f61ac1466f0 2023-05-20T01:16:12.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:12.302+0000 7f61abfff700 1 --2- 172.21.15.16:0/2946562605 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f61ac0cdff0 0x7f61ac1f1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-05-20T01:16:12.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:12.302+0000 7f61b0b3d700 1 --2- 172.21.15.16:0/2946562605 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f61ac06bd30 0x7f61ac06f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-05-20T01:16:12.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:12.302+0000 7f61b133e700 1 --2- 172.21.15.16:0/2946562605 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61ac1466f0 0x7f61ac157b80 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f61a400cc20 tx=0x7f61a400cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:12.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:12.302+0000 7f61a97fa700 1 -- 172.21.15.16:0/2946562605 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61a4003c00 con 0x7f61ac1466f0 2023-05-20T01:16:12.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:12.302+0000 7f61a97fa700 1 -- 172.21.15.16:0/2946562605 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61a4003d60 con 0x7f61ac1466f0 2023-05-20T01:16:12.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:12.302+0000 7f61a97fa700 1 -- 172.21.15.16:0/2946562605 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61a400aae0 con 0x7f61ac1466f0 2023-05-20T01:16:12.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:12.302+0000 7f61b2da1700 1 -- 172.21.15.16:0/2946562605 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61ac0ce470 con 0x7f61ac1466f0 2023-05-20T01:16:12.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:12.302+0000 7f61b2da1700 1 -- 172.21.15.16:0/2946562605 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61ac0cbeb0 con 0x7f61ac1466f0 2023-05-20T01:16:12.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:12.303+0000 7f61a97fa700 1 -- 172.21.15.16:0/2946562605 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f61a40050d0 con 0x7f61ac1466f0 2023-05-20T01:16:12.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:12.303+0000 7f61a97fa700 1 -- 172.21.15.16:0/2946562605 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f61a4012cd0 con 0x7f61ac1466f0 2023-05-20T01:16:12.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:12.303+0000 7f61b2da1700 1 -- 172.21.15.16:0/2946562605 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61ac0ccae0 con 0x7f61ac1466f0 2023-05-20T01:16:12.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:12.306+0000 7f61a97fa700 1 -- 172.21.15.16:0/2946562605 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72437 (secure 0 0 0) 0x7f61a4005560 con 0x7f61ac1466f0 2023-05-20T01:16:12.339 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-05-20T01:16:12.336+0000 7f1da015b200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-05-20T01:16:12.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:12.376+0000 7f61b2da1700 1 -- 172.21.15.16:0/2946562605 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f61ac0c5c30 con 0x7f61ac1466f0 2023-05-20T01:16:12.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:12.376+0000 7f61a97fa700 1 -- 172.21.15.16:0/2946562605 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f61a4005700 con 0x7f61ac1466f0 2023-05-20T01:16:12.380 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:16:12.380 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":10,"fsid":"c2245cc5-ecab-4087-a59b-bc81db901676","created":"2023-05-20T01:16:09.549230+0000","modified":"2023-05-20T01:16:12.092160+0000","last_up_change":"0.000000","last_in_change":"2023-05-20T01:16:12.092160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"317a83cd-19c8-4958-a301-6087ef342289","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"5581046f-8219-4089-8d7d-b7846399505e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"072e9e66-4133-478e-8d4a-64d46c2963f7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"11e1bba6-00f8-48dd-9028-d8d716b7f145","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"b8909c3a-cf70-46cb-8d0b-009c707cf9d3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"b1818304-4583-47bf-a6ec-0bfbb5166233","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"d37f4691-eed1-4a93-8818-18354f03f4a4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"4f712f62-5d1d-4816-8042-03209144f8b0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-20T01:16:12.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:12.377+0000 7f61b2da1700 1 -- 172.21.15.16:0/2946562605 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61ac1466f0 msgr2=0x7f61ac157b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:12.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:12.377+0000 7f61b2da1700 1 --2- 172.21.15.16:0/2946562605 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61ac1466f0 0x7f61ac157b80 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f61a400cc20 tx=0x7f61a400cf30 comp rx=0 tx=0).stop 2023-05-20T01:16:12.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:12.377+0000 7f61b2da1700 1 -- 172.21.15.16:0/2946562605 shutdown_connections 2023-05-20T01:16:12.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:12.377+0000 7f61b2da1700 1 --2- 172.21.15.16:0/2946562605 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61ac1466f0 0x7f61ac157b80 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:12.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:12.377+0000 7f61b2da1700 1 --2- 172.21.15.16:0/2946562605 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f61ac06bd30 0x7f61ac06f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:12.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:12.377+0000 7f61b2da1700 1 --2- 172.21.15.16:0/2946562605 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f61ac0cdff0 0x7f61ac1f1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:12.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:12.377+0000 7f61b2da1700 1 -- 172.21.15.16:0/2946562605 >> 172.21.15.16:0/2946562605 conn(0x7f61ac06ae30 msgr2=0x7f61ac06c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:12.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:12.377+0000 7f61b2da1700 1 -- 172.21.15.16:0/2946562605 shutdown_connections 2023-05-20T01:16:12.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:12.377+0000 7f61b2da1700 1 -- 172.21.15.16:0/2946562605 wait complete. 2023-05-20T01:16:12.393 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-05-20T01:16:12.394 INFO:tasks.ceph:Waiting for OSDs to come up 2023-05-20T01:16:12.703 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-05-20T01:16:12.700+0000 7fa73c17e200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-05-20T01:16:12.735 INFO:tasks.ceph.osd.6.smithi132.stderr:2023-05-20T01:16:12.732+0000 7fc769562640 -1 Falling back to public interface 2023-05-20T01:16:12.736 INFO:tasks.ceph.osd.7.smithi132.stderr:2023-05-20T01:16:12.732+0000 7f7bc5e3e640 -1 Falling back to public interface 2023-05-20T01:16:12.736 INFO:tasks.ceph.osd.6.smithi132.stderr:2023-05-20T01:16:12.732+0000 7fc760920700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6800/0: (98) Address already in use 2023-05-20T01:16:12.737 INFO:tasks.ceph.osd.7.smithi132.stderr:2023-05-20T01:16:12.732+0000 7f7bbd1fc700 -1 PosixStack listen unable to listen on v1:0.0.0.0:6801/0: (98) Address already in use 2023-05-20T01:16:12.737 INFO:tasks.ceph.osd.7.smithi132.stderr:2023-05-20T01:16:12.732+0000 7f7bbd1fc700 -1 PosixStack listen unable to listen on v1:0.0.0.0:6802/0: (98) Address already in use 2023-05-20T01:16:12.738 INFO:tasks.ceph.osd.6.smithi132.stderr:2023-05-20T01:16:12.732+0000 7fc760920700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6804/0: (98) Address already in use 2023-05-20T01:16:12.739 INFO:tasks.ceph.osd.3.smithi016.stderr:2023-05-20T01:16:12.733+0000 7f8f4a606640 -1 Falling back to public interface 2023-05-20T01:16:12.749 INFO:tasks.ceph.osd.1.smithi016.stderr:2023-05-20T01:16:12.745+0000 7f4de9481640 -1 Falling back to public interface 2023-05-20T01:16:12.753 INFO:tasks.ceph.osd.2.smithi016.stderr:2023-05-20T01:16:12.749+0000 7f1bce8e8640 -1 Falling back to public interface 2023-05-20T01:16:12.757 INFO:tasks.ceph.osd.4.smithi132.stderr:2023-05-20T01:16:12.754+0000 7f45c880c640 -1 Falling back to public interface 2023-05-20T01:16:12.758 INFO:tasks.ceph.osd.5.smithi132.stderr:2023-05-20T01:16:12.754+0000 7f3555339640 -1 Falling back to public interface 2023-05-20T01:16:12.759 INFO:tasks.ceph.osd.4.smithi132.stderr:2023-05-20T01:16:12.755+0000 7f45bfbca700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6816/0: (98) Address already in use 2023-05-20T01:16:12.760 INFO:tasks.ceph.osd.0.smithi016.stderr:2023-05-20T01:16:12.756+0000 7f43a6587640 -1 Falling back to public interface 2023-05-20T01:16:12.786 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-05-20T01:16:12.782+0000 7fa73c17e200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-05-20T01:16:12.931 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-05-20T01:16:12.928+0000 7fa73c17e200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-05-20T01:16:13.043 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-05-20T01:16:13.040+0000 7f1da015b200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-05-20T01:16:13.121 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-05-20T01:16:13.118+0000 7f1da015b200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-05-20T01:16:13.175 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-05-20T01:16:13.171+0000 7fa73c17e200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-05-20T01:16:13.198 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-05-20T01:16:13.253 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-05-20T01:16:13.250+0000 7fa73c17e200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-05-20T01:16:13.258 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-05-20T01:16:13.255+0000 7f1da015b200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-05-20T01:16:13.315 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:13.311+0000 7f3861672700 1 Processor -- start 2023-05-20T01:16:13.315 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:13.311+0000 7f3861672700 1 -- start start 2023-05-20T01:16:13.317 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:13.312+0000 7f3861672700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f385c06c0d0 0x7f385c0ce320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:13.317 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:13.312+0000 7f3861672700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f385c071090 con 0x7f385c0ce860 2023-05-20T01:16:13.317 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:13.312+0000 7f3861672700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f385c15e2d0 con 0x7f385c159b40 2023-05-20T01:16:13.318 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:13.312+0000 7f3861672700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f385c15d6c0 con 0x7f385c06c0d0 2023-05-20T01:16:13.318 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:13.312+0000 7f385b7fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f385c0ce860 0x7f385c0cab30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:40224/0 (socket says 172.21.15.16:40224) 2023-05-20T01:16:13.318 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:13.312+0000 7f385b7fe700 1 -- 172.21.15.16:0/3228185996 learned_addr learned my addr 172.21.15.16:0/3228185996 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:16:13.318 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:13.312+0000 7f385a7fc700 1 --2- 172.21.15.16:0/3228185996 >> v2:172.21.15.132:3300/0 conn(0x7f385c06c0d0 0x7f385c0ce320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:13.318 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:13.312+0000 7f3859ffb700 1 -- 172.21.15.16:0/3228185996 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3508697238 0 0) 0x7f385c071090 con 0x7f385c0ce860 2023-05-20T01:16:13.319 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:13.312+0000 7f3859ffb700 1 -- 172.21.15.16:0/3228185996 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38480035b0 con 0x7f385c0ce860 2023-05-20T01:16:13.319 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:13.313+0000 7f385a7fc700 1 -- 172.21.15.16:0/3228185996 >> v1:172.21.15.132:6789/0 conn(0x7f385c159b40 legacy=0x7f385c06b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:13.319 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:13.313+0000 7f3859ffb700 1 -- 172.21.15.16:0/3228185996 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 818413595 0 0) 0x7f385c15e2d0 con 0x7f385c159b40 2023-05-20T01:16:13.319 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:13.313+0000 7f385a7fc700 1 -- 172.21.15.16:0/3228185996 >> v1:172.21.15.16:6789/0 conn(0x7f385c0ce860 legacy=0x7f385c0cab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:13.319 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:13.313+0000 7f385a7fc700 1 -- 172.21.15.16:0/3228185996 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f385c15d9a0 con 0x7f385c06c0d0 2023-05-20T01:16:13.319 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:13.313+0000 7f385a7fc700 1 --2- 172.21.15.16:0/3228185996 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f385c06c0d0 0x7f385c0ce320 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f3844009840 tx=0x7f3844009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3bc4cc757b83fb12 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:13.319 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:13.313+0000 7f3859ffb700 1 -- 172.21.15.16:0/3228185996 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3844011830 con 0x7f385c06c0d0 2023-05-20T01:16:13.319 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:13.313+0000 7f3859ffb700 1 -- 172.21.15.16:0/3228185996 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3844011990 con 0x7f385c06c0d0 2023-05-20T01:16:13.320 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:13.313+0000 7f3859ffb700 1 -- 172.21.15.16:0/3228185996 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3844011d20 con 0x7f385c06c0d0 2023-05-20T01:16:13.322 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:13.314+0000 7f3861672700 1 -- 172.21.15.16:0/3228185996 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f385c06c0d0 msgr2=0x7f385c0ce320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:13.322 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:13.314+0000 7f3861672700 1 --2- 172.21.15.16:0/3228185996 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f385c06c0d0 0x7f385c0ce320 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f3844009840 tx=0x7f3844009b50 comp rx=0 tx=0).stop 2023-05-20T01:16:13.323 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:13.314+0000 7f3861672700 1 -- 172.21.15.16:0/3228185996 shutdown_connections 2023-05-20T01:16:13.323 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:13.314+0000 7f3861672700 1 --2- 172.21.15.16:0/3228185996 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f385c06c0d0 0x7f385c0ce320 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:13.323 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:13.314+0000 7f3861672700 1 -- 172.21.15.16:0/3228185996 >> 172.21.15.16:0/3228185996 conn(0x7f385c06f110 msgr2=0x7f385c06f4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:13.323 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:13.314+0000 7f3861672700 1 -- 172.21.15.16:0/3228185996 shutdown_connections 2023-05-20T01:16:13.323 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:13.314+0000 7f3861672700 1 -- 172.21.15.16:0/3228185996 wait complete. 2023-05-20T01:16:13.323 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:13.316+0000 7f3861672700 1 Processor -- start 2023-05-20T01:16:13.324 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:13.316+0000 7f3861672700 1 -- start start 2023-05-20T01:16:13.324 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:13.317+0000 7f3861672700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f385c06c0d0 0x7f385c150680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:13.324 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:13.317+0000 7f3861672700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f385c0ce860 0x7f385c0aeed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:13.324 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:13.317+0000 7f3861672700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f385c159b40 0x7f385c06a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:13.324 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:13.317+0000 7f3861672700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f385c0adb50 con 0x7f385c159b40 2023-05-20T01:16:13.324 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:13.317+0000 7f3861672700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f385c0ade30 con 0x7f385c06c0d0 2023-05-20T01:16:13.324 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:13.317+0000 7f3861672700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f385c0ae110 con 0x7f385c0ce860 2023-05-20T01:16:13.325 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:13.317+0000 7f385b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f385c159b40 0x7f385c06a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:13.325 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:13.317+0000 7f385affd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f385c06c0d0 0x7f385c150680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:13.325 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:13.317+0000 7f385b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f385c159b40 0x7f385c06a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56934/0 (socket says 172.21.15.16:56934) 2023-05-20T01:16:13.325 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:13.317+0000 7f385affd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f385c06c0d0 0x7f385c150680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.16:53952/0 (socket says 172.21.15.16:53952) 2023-05-20T01:16:13.325 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:13.317+0000 7f385b7fe700 1 -- 172.21.15.16:0/1740541963 learned_addr learned my addr 172.21.15.16:0/1740541963 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:13.325 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:13.317+0000 7f385a7fc700 1 --2- 172.21.15.16:0/1740541963 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f385c0ce860 0x7f385c0aeed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:13.325 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:13.317+0000 7f385affd700 1 -- 172.21.15.16:0/1740541963 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f385c0ce860 msgr2=0x7f385c0aeed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:13.325 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:13.317+0000 7f385affd700 1 --2- 172.21.15.16:0/1740541963 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f385c0ce860 0x7f385c0aeed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:13.326 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:13.317+0000 7f385affd700 1 -- 172.21.15.16:0/1740541963 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f385c159b40 msgr2=0x7f385c06a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:13.326 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:13.317+0000 7f385affd700 1 --2- 172.21.15.16:0/1740541963 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f385c159b40 0x7f385c06a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:13.326 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:13.317+0000 7f385affd700 1 -- 172.21.15.16:0/1740541963 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38440094a0 con 0x7f385c06c0d0 2023-05-20T01:16:13.326 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:13.317+0000 7f385a7fc700 1 --2- 172.21.15.16:0/1740541963 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f385c0ce860 0x7f385c0aeed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-05-20T01:16:13.326 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:13.318+0000 7f385affd700 1 --2- 172.21.15.16:0/1740541963 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f385c06c0d0 0x7f385c150680 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f384c00c710 tx=0x7f384c00ca20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:13.326 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:13.318+0000 7f38537fe700 1 -- 172.21.15.16:0/1740541963 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f384c003e30 con 0x7f385c06c0d0 2023-05-20T01:16:13.326 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:13.318+0000 7f38537fe700 1 -- 172.21.15.16:0/1740541963 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f384c0050d0 con 0x7f385c06c0d0 2023-05-20T01:16:13.327 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:13.318+0000 7f3861672700 1 -- 172.21.15.16:0/1740541963 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f385c15df60 con 0x7f385c06c0d0 2023-05-20T01:16:13.327 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:13.318+0000 7f38537fe700 1 -- 172.21.15.16:0/1740541963 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f384c005460 con 0x7f385c06c0d0 2023-05-20T01:16:13.327 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:13.318+0000 7f3861672700 1 -- 172.21.15.16:0/1740541963 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f385c15dc80 con 0x7f385c06c0d0 2023-05-20T01:16:13.327 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:13.318+0000 7f38537fe700 1 -- 172.21.15.16:0/1740541963 <== mon.1 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f384c01c440 con 0x7f385c06c0d0 2023-05-20T01:16:13.327 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:13.318+0000 7f38537fe700 1 -- 172.21.15.16:0/1740541963 <== mon.1 v2:172.21.15.132:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f384c012ad0 con 0x7f385c06c0d0 2023-05-20T01:16:13.327 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:13.319+0000 7f3861672700 1 -- 172.21.15.16:0/1740541963 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f385c0ceca0 con 0x7f385c06c0d0 2023-05-20T01:16:13.328 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:13.321+0000 7f38537fe700 1 -- 172.21.15.16:0/1740541963 <== mon.1 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72437 (secure 0 0 0) 0x7f384c017070 con 0x7f385c06c0d0 2023-05-20T01:16:13.396 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:13.393+0000 7f3861672700 1 -- 172.21.15.16:0/1740541963 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f385c0747b0 con 0x7f385c06c0d0 2023-05-20T01:16:13.397 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:13.394+0000 7f38537fe700 1 -- 172.21.15.16:0/1740541963 <== mon.1 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f384c0124b0 con 0x7f385c06c0d0 2023-05-20T01:16:13.397 INFO:teuthology.misc.health.smithi016.stdout: 2023-05-20T01:16:13.398 INFO:teuthology.misc.health.smithi016.stdout:{"epoch":10,"fsid":"c2245cc5-ecab-4087-a59b-bc81db901676","created":"2023-05-20T01:16:09.549230+0000","modified":"2023-05-20T01:16:12.092160+0000","last_up_change":"0.000000","last_in_change":"2023-05-20T01:16:12.092160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"317a83cd-19c8-4958-a301-6087ef342289","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"5581046f-8219-4089-8d7d-b7846399505e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"072e9e66-4133-478e-8d4a-64d46c2963f7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"11e1bba6-00f8-48dd-9028-d8d716b7f145","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"b8909c3a-cf70-46cb-8d0b-009c707cf9d3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"b1818304-4583-47bf-a6ec-0bfbb5166233","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"d37f4691-eed1-4a93-8818-18354f03f4a4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"4f712f62-5d1d-4816-8042-03209144f8b0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-20T01:16:13.398 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:13.395+0000 7f3861672700 1 -- 172.21.15.16:0/1740541963 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f385c06c0d0 msgr2=0x7f385c150680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:13.399 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:13.395+0000 7f3861672700 1 --2- 172.21.15.16:0/1740541963 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f385c06c0d0 0x7f385c150680 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f384c00c710 tx=0x7f384c00ca20 comp rx=0 tx=0).stop 2023-05-20T01:16:13.399 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:13.395+0000 7f3861672700 1 -- 172.21.15.16:0/1740541963 shutdown_connections 2023-05-20T01:16:13.399 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:13.395+0000 7f3861672700 1 --2- 172.21.15.16:0/1740541963 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f385c159b40 0x7f385c06a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:13.399 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:13.395+0000 7f3861672700 1 --2- 172.21.15.16:0/1740541963 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f385c06c0d0 0x7f385c150680 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:13.399 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:13.395+0000 7f3861672700 1 --2- 172.21.15.16:0/1740541963 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f385c0ce860 0x7f385c0aeed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:13.399 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:13.395+0000 7f3861672700 1 -- 172.21.15.16:0/1740541963 >> 172.21.15.16:0/1740541963 conn(0x7f385c06f110 msgr2=0x7f385c0cab30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:13.399 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:13.395+0000 7f3861672700 1 -- 172.21.15.16:0/1740541963 shutdown_connections 2023-05-20T01:16:13.400 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:13.395+0000 7f3861672700 1 -- 172.21.15.16:0/1740541963 wait complete. 2023-05-20T01:16:13.409 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-05-20T01:16:13.492 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-05-20T01:16:13.489+0000 7f1da015b200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-05-20T01:16:13.538 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-05-20T01:16:13.535+0000 7fa73c17e200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-05-20T01:16:13.565 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-05-20T01:16:13.562+0000 7f1da015b200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-05-20T01:16:13.765 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-05-20T01:16:13.762+0000 7fa73c17e200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-05-20T01:16:13.840 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-05-20T01:16:13.837+0000 7f1da015b200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-05-20T01:16:13.927 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-05-20T01:16:13.923+0000 7fa73c17e200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-05-20T01:16:14.047 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-05-20T01:16:14.043+0000 7fa73c17e200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-05-20T01:16:14.057 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-05-20T01:16:14.054+0000 7f1da015b200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-05-20T01:16:14.214 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-05-20T01:16:14.211+0000 7f1da015b200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-05-20T01:16:14.328 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-05-20T01:16:14.325+0000 7f1da015b200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-05-20T01:16:14.381 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-05-20T01:16:14.378+0000 7fa73c17e200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-05-20T01:16:14.485 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-05-20T01:16:14.482+0000 7fa73c17e200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-05-20T01:16:14.653 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-05-20T01:16:14.650+0000 7f1da015b200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-05-20T01:16:14.680 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-05-20T01:16:14.674+0000 7fa73c17e200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-05-20T01:16:14.754 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-05-20T01:16:14.751+0000 7f1da015b200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-05-20T01:16:14.938 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-05-20T01:16:14.935+0000 7f1da015b200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-05-20T01:16:15.299 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-05-20T01:16:15.296+0000 7fa73c17e200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-05-20T01:16:15.586 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-05-20T01:16:15.583+0000 7f1da015b200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-05-20T01:16:16.231 INFO:tasks.ceph.osd.3.smithi016.stderr:2023-05-20T01:16:16.228+0000 7f8f4a606640 -1 osd.3 0 log_to_monitors true 2023-05-20T01:16:16.279 INFO:tasks.ceph.osd.6.smithi132.stderr:2023-05-20T01:16:16.276+0000 7fc769562640 -1 osd.6 0 log_to_monitors true 2023-05-20T01:16:16.286 INFO:tasks.ceph.osd.7.smithi132.stderr:2023-05-20T01:16:16.283+0000 7f7bc5e3e640 -1 osd.7 0 log_to_monitors true 2023-05-20T01:16:16.302 INFO:tasks.ceph.osd.4.smithi132.stderr:2023-05-20T01:16:16.298+0000 7f45c880c640 -1 osd.4 0 log_to_monitors true 2023-05-20T01:16:16.306 INFO:tasks.ceph.osd.1.smithi016.stderr:2023-05-20T01:16:16.303+0000 7f4de9481640 -1 osd.1 0 log_to_monitors true 2023-05-20T01:16:16.307 INFO:tasks.ceph.osd.0.smithi016.stderr:2023-05-20T01:16:16.304+0000 7f43a6587640 -1 osd.0 0 log_to_monitors true 2023-05-20T01:16:16.308 INFO:tasks.ceph.osd.2.smithi016.stderr:2023-05-20T01:16:16.304+0000 7f1bce8e8640 -1 osd.2 0 log_to_monitors true 2023-05-20T01:16:16.324 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-05-20T01:16:16.321+0000 7fa6f43aa700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-05-20T01:16:16.325 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-05-20T01:16:16.321+0000 7fa6f43aa700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-05-20T01:16:16.331 INFO:tasks.ceph.osd.5.smithi132.stderr:2023-05-20T01:16:16.328+0000 7f3555339640 -1 osd.5 0 log_to_monitors true 2023-05-20T01:16:18.763 INFO:tasks.ceph.osd.0.smithi016.stderr:2023-05-20T01:16:18.760+0000 7f439a93f700 -1 osd.0 0 waiting for initial osdmap 2023-05-20T01:16:18.781 INFO:tasks.ceph.osd.0.smithi016.stderr:2023-05-20T01:16:18.777+0000 7f4392519700 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-05-20T01:16:18.882 INFO:tasks.ceph.osd.2.smithi016.stderr:2023-05-20T01:16:18.878+0000 7f1bc2ca0700 -1 osd.2 0 waiting for initial osdmap 2023-05-20T01:16:18.883 INFO:tasks.ceph.osd.1.smithi016.stderr:2023-05-20T01:16:18.878+0000 7f4ddd839700 -1 osd.1 0 waiting for initial osdmap 2023-05-20T01:16:18.893 INFO:tasks.ceph.osd.1.smithi016.stderr:2023-05-20T01:16:18.890+0000 7f4dd5413700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-05-20T01:16:18.895 INFO:tasks.ceph.osd.3.smithi016.stderr:2023-05-20T01:16:18.892+0000 7f8f3e9be700 -1 osd.3 0 waiting for initial osdmap 2023-05-20T01:16:18.896 INFO:tasks.ceph.osd.2.smithi016.stderr:2023-05-20T01:16:18.893+0000 7f1bba87a700 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-05-20T01:16:18.907 INFO:tasks.ceph.osd.3.smithi016.stderr:2023-05-20T01:16:18.904+0000 7f8f36598700 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-05-20T01:16:19.365 INFO:tasks.ceph.osd.5.smithi132.stderr:2023-05-20T01:16:19.361+0000 7f35496f1700 -1 osd.5 0 waiting for initial osdmap 2023-05-20T01:16:19.380 INFO:tasks.ceph.osd.5.smithi132.stderr:2023-05-20T01:16:19.377+0000 7f35412cb700 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-05-20T01:16:19.386 INFO:tasks.ceph.osd.6.smithi132.stderr:2023-05-20T01:16:19.383+0000 7fc75d91a700 -1 osd.6 0 waiting for initial osdmap 2023-05-20T01:16:19.399 INFO:tasks.ceph.osd.6.smithi132.stderr:2023-05-20T01:16:19.396+0000 7fc7554f4700 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-05-20T01:16:19.420 INFO:tasks.ceph.osd.7.smithi132.stderr:2023-05-20T01:16:19.417+0000 7f7bba1f6700 -1 osd.7 0 waiting for initial osdmap 2023-05-20T01:16:19.430 INFO:tasks.ceph.osd.4.smithi132.stderr:2023-05-20T01:16:19.427+0000 7f45bcbc4700 -1 osd.4 0 waiting for initial osdmap 2023-05-20T01:16:19.433 INFO:tasks.ceph.osd.7.smithi132.stderr:2023-05-20T01:16:19.430+0000 7f7bb1dd0700 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-05-20T01:16:19.449 INFO:tasks.ceph.osd.4.smithi132.stderr:2023-05-20T01:16:19.446+0000 7f45b479e700 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-05-20T01:16:20.214 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-05-20T01:16:20.343 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:20.339+0000 7fde49c14700 1 Processor -- start 2023-05-20T01:16:20.343 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:20.339+0000 7fde49c14700 1 -- start start 2023-05-20T01:16:20.344 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:20.340+0000 7fde49c14700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fde4406ac20 0x7fde440d0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:20.345 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:20.340+0000 7fde49c14700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fde4406f540 0x7fde44161010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:20.345 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:20.340+0000 7fde49c14700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde44160b80 con 0x7fde4406ef80 2023-05-20T01:16:20.345 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:20.340+0000 7fde49c14700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde44160630 con 0x7fde4406f540 2023-05-20T01:16:20.345 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:20.340+0000 7fde49c14700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde44161550 con 0x7fde4406ac20 2023-05-20T01:16:20.345 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:20.340+0000 7fde42ffd700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fde4406ac20 0x7fde440d0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:20.345 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:20.340+0000 7fde43fff700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fde4406f540 0x7fde44161010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:20.345 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:20.340+0000 7fde42ffd700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fde4406ac20 0x7fde440d0bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.16:57208/0 (socket says 172.21.15.16:57208) 2023-05-20T01:16:20.346 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:20.340+0000 7fde43fff700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fde4406f540 0x7fde44161010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.16:54194/0 (socket says 172.21.15.16:54194) 2023-05-20T01:16:20.346 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:20.340+0000 7fde437fe700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7fde4406ef80 0x7fde4406a510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.16:50178/0 (socket says 172.21.15.16:50178) 2023-05-20T01:16:20.346 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:20.340+0000 7fde42ffd700 1 -- 172.21.15.16:0/3039257141 learned_addr learned my addr 172.21.15.16:0/3039257141 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:20.346 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:20.341+0000 7fde43fff700 1 -- 172.21.15.16:0/3039257141 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fde4406ac20 msgr2=0x7fde440d0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:20.346 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:20.341+0000 7fde43fff700 1 --2- 172.21.15.16:0/3039257141 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fde4406ac20 0x7fde440d0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:20.346 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:20.341+0000 7fde43fff700 1 -- 172.21.15.16:0/3039257141 >> v1:172.21.15.132:6789/0 conn(0x7fde4406ef80 legacy=0x7fde4406a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:20.346 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:20.341+0000 7fde43fff700 1 -- 172.21.15.16:0/3039257141 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde4406f350 con 0x7fde4406f540 2023-05-20T01:16:20.346 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:20.341+0000 7fde427fc700 1 -- 172.21.15.16:0/3039257141 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1391210873 0 0) 0x7fde44160b80 con 0x7fde4406ef80 2023-05-20T01:16:20.347 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:20.341+0000 7fde42ffd700 1 --2- 172.21.15.16:0/3039257141 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fde4406ac20 0x7fde440d0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-05-20T01:16:20.347 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:20.341+0000 7fde43fff700 1 --2- 172.21.15.16:0/3039257141 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fde4406f540 0x7fde44161010 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fde3c009dc0 tx=0x7fde3c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e55b9bcaad37656f server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:20.347 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:20.341+0000 7fde427fc700 1 -- 172.21.15.16:0/3039257141 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde3c00e070 con 0x7fde4406f540 2023-05-20T01:16:20.347 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:20.341+0000 7fde427fc700 1 -- 172.21.15.16:0/3039257141 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde3c00bae0 con 0x7fde4406f540 2023-05-20T01:16:20.347 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:20.341+0000 7fde427fc700 1 -- 172.21.15.16:0/3039257141 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde3c00be90 con 0x7fde4406f540 2023-05-20T01:16:20.350 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:20.342+0000 7fde49c14700 1 -- 172.21.15.16:0/3039257141 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fde4406f540 msgr2=0x7fde44161010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:20.350 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:20.342+0000 7fde49c14700 1 --2- 172.21.15.16:0/3039257141 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fde4406f540 0x7fde44161010 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fde3c009dc0 tx=0x7fde3c00b330 comp rx=0 tx=0).stop 2023-05-20T01:16:20.350 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:20.342+0000 7fde49c14700 1 -- 172.21.15.16:0/3039257141 shutdown_connections 2023-05-20T01:16:20.350 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:20.342+0000 7fde49c14700 1 --2- 172.21.15.16:0/3039257141 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fde4406f540 0x7fde44161010 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:20.350 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:20.342+0000 7fde49c14700 1 --2- 172.21.15.16:0/3039257141 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fde4406ac20 0x7fde440d0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:20.350 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:20.342+0000 7fde49c14700 1 -- 172.21.15.16:0/3039257141 >> 172.21.15.16:0/3039257141 conn(0x7fde440b27d0 msgr2=0x7fde440d0010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:20.351 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:20.342+0000 7fde49c14700 1 -- 172.21.15.16:0/3039257141 shutdown_connections 2023-05-20T01:16:20.351 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:20.342+0000 7fde49c14700 1 -- 172.21.15.16:0/3039257141 wait complete. 2023-05-20T01:16:20.351 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:20.343+0000 7fde49c14700 1 Processor -- start 2023-05-20T01:16:20.351 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:20.343+0000 7fde49c14700 1 -- start start 2023-05-20T01:16:20.351 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:20.344+0000 7fde49c14700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fde4406ac20 0x7fde440cbce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:20.351 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:20.344+0000 7fde49c14700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde4406ef80 0x7fde441f1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:20.351 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:20.344+0000 7fde49c14700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fde4406f540 0x7fde440cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:20.351 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:20.344+0000 7fde49c14700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde440cc220 con 0x7fde4406ef80 2023-05-20T01:16:20.352 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:20.344+0000 7fde49c14700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde440cc4a0 con 0x7fde4406ac20 2023-05-20T01:16:20.352 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:20.344+0000 7fde49c14700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde440cc780 con 0x7fde4406f540 2023-05-20T01:16:20.352 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:20.344+0000 7fde42ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde4406ef80 0x7fde441f1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:20.352 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:20.344+0000 7fde42ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde4406ef80 0x7fde441f1ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57176/0 (socket says 172.21.15.16:57176) 2023-05-20T01:16:20.352 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:20.344+0000 7fde42ffd700 1 -- 172.21.15.16:0/2130399484 learned_addr learned my addr 172.21.15.16:0/2130399484 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:20.352 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:20.344+0000 7fde43fff700 1 --2- 172.21.15.16:0/2130399484 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fde4406f540 0x7fde440cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:20.352 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:20.344+0000 7fde437fe700 1 --2- 172.21.15.16:0/2130399484 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fde4406ac20 0x7fde440cbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:20.353 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:20.344+0000 7fde42ffd700 1 -- 172.21.15.16:0/2130399484 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fde4406f540 msgr2=0x7fde440cd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:20.353 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:20.344+0000 7fde42ffd700 1 --2- 172.21.15.16:0/2130399484 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fde4406f540 0x7fde440cd9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:20.353 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:20.344+0000 7fde42ffd700 1 -- 172.21.15.16:0/2130399484 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fde4406ac20 msgr2=0x7fde440cbce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:20.353 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:20.345+0000 7fde42ffd700 1 --2- 172.21.15.16:0/2130399484 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fde4406ac20 0x7fde440cbce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:20.353 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:20.345+0000 7fde42ffd700 1 -- 172.21.15.16:0/2130399484 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde2c009550 con 0x7fde4406ef80 2023-05-20T01:16:20.353 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:20.345+0000 7fde43fff700 1 --2- 172.21.15.16:0/2130399484 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fde4406f540 0x7fde440cd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-05-20T01:16:20.353 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:20.345+0000 7fde42ffd700 1 --2- 172.21.15.16:0/2130399484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde4406ef80 0x7fde441f1ca0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fde2c00ebb0 tx=0x7fde2c00eec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:20.354 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:20.345+0000 7fde3bfff700 1 -- 172.21.15.16:0/2130399484 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde2c00c850 con 0x7fde4406ef80 2023-05-20T01:16:20.354 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:20.345+0000 7fde3bfff700 1 -- 172.21.15.16:0/2130399484 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde2c00c9b0 con 0x7fde4406ef80 2023-05-20T01:16:20.354 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:20.345+0000 7fde49c14700 1 -- 172.21.15.16:0/2130399484 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde3c009a70 con 0x7fde4406ef80 2023-05-20T01:16:20.354 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:20.345+0000 7fde3bfff700 1 -- 172.21.15.16:0/2130399484 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde2c00cd40 con 0x7fde4406ef80 2023-05-20T01:16:20.354 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:20.345+0000 7fde49c14700 1 -- 172.21.15.16:0/2130399484 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde440cf950 con 0x7fde4406ef80 2023-05-20T01:16:20.354 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:20.346+0000 7fde49c14700 1 -- 172.21.15.16:0/2130399484 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde440cfc30 con 0x7fde4406ef80 2023-05-20T01:16:20.354 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:20.346+0000 7fde3bfff700 1 -- 172.21.15.16:0/2130399484 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 4) v1 ==== 88981+0+0 (secure 0 0 0) 0x7fde2c013470 con 0x7fde4406ef80 2023-05-20T01:16:20.356 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:20.347+0000 7fde3bfff700 1 --2- 172.21.15.16:0/2130399484 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fde2406ac50 0x7fde2406d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:20.356 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:20.347+0000 7fde3bfff700 1 -- 172.21.15.16:0/2130399484 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 4471+0+0 (secure 0 0 0) 0x7fde2c088e40 con 0x7fde4406ef80 2023-05-20T01:16:20.356 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:20.347+0000 7fde437fe700 1 --2- 172.21.15.16:0/2130399484 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fde2406ac50 0x7fde2406d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:20.356 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:20.347+0000 7fde437fe700 1 --2- 172.21.15.16:0/2130399484 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fde2406ac50 0x7fde2406d100 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fde34007fd0 tx=0x7fde34007de0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:20.357 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:20.352+0000 7fde3bfff700 1 -- 172.21.15.16:0/2130399484 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fde2c054610 con 0x7fde4406ef80 2023-05-20T01:16:20.557 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:20.553+0000 7fde49c14700 1 -- 172.21.15.16:0/2130399484 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fde44161160 con 0x7fde4406ef80 2023-05-20T01:16:20.557 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:20.554+0000 7fde3bfff700 1 -- 172.21.15.16:0/2130399484 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) v1 ==== 74+0+10872 (secure 0 0 0) 0x7fde2c057c30 con 0x7fde4406ef80 2023-05-20T01:16:20.558 INFO:teuthology.misc.health.smithi016.stdout: 2023-05-20T01:16:20.558 INFO:teuthology.misc.health.smithi016.stdout:{"epoch":13,"fsid":"c2245cc5-ecab-4087-a59b-bc81db901676","created":"2023-05-20T01:16:09.549230+0000","modified":"2023-05-20T01:16:19.788624+0000","last_up_change":"2023-05-20T01:16:19.788624+0000","last_in_change":"2023-05-20T01:16:12.092160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"317a83cd-19c8-4958-a301-6087ef342289","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6825","nonce":3923634179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6827","nonce":3923634179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6831","nonce":3923634179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6829","nonce":3923634179}]},"public_addr":"172.21.15.16:6825/3923634179","cluster_addr":"172.21.15.16:6827/3923634179","heartbeat_back_addr":"172.21.15.16:6831/3923634179","heartbeat_front_addr":"172.21.15.16:6829/3923634179","state":["exists","up"]},{"osd":1,"uuid":"5581046f-8219-4089-8d7d-b7846399505e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6809","nonce":3860264220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6811","nonce":3860264220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6815","nonce":3860264220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6813","nonce":3860264220}]},"public_addr":"172.21.15.16:6809/3860264220","cluster_addr":"172.21.15.16:6811/3860264220","heartbeat_back_addr":"172.21.15.16:6815/3860264220","heartbeat_front_addr":"172.21.15.16:6813/3860264220","state":["exists","up"]},{"osd":2,"uuid":"072e9e66-4133-478e-8d4a-64d46c2963f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6817","nonce":293187030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6819","nonce":293187030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6823","nonce":293187030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6821","nonce":293187030}]},"public_addr":"172.21.15.16:6817/293187030","cluster_addr":"172.21.15.16:6819/293187030","heartbeat_back_addr":"172.21.15.16:6823/293187030","heartbeat_front_addr":"172.21.15.16:6821/293187030","state":["exists","up"]},{"osd":3,"uuid":"11e1bba6-00f8-48dd-9028-d8d716b7f145","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6801","nonce":3669781890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6803","nonce":3669781890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6807","nonce":3669781890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6805","nonce":3669781890}]},"public_addr":"172.21.15.16:6801/3669781890","cluster_addr":"172.21.15.16:6803/3669781890","heartbeat_back_addr":"172.21.15.16:6807/3669781890","heartbeat_front_addr":"172.21.15.16:6805/3669781890","state":["exists","up"]},{"osd":4,"uuid":"b8909c3a-cf70-46cb-8d0b-009c707cf9d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6819","nonce":506317223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6823","nonce":506317223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6829","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6831","nonce":506317223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6827","nonce":506317223}]},"public_addr":"172.21.15.132:6819/506317223","cluster_addr":"172.21.15.132:6823/506317223","heartbeat_back_addr":"172.21.15.132:6831/506317223","heartbeat_front_addr":"172.21.15.132:6827/506317223","state":["exists","up"]},{"osd":5,"uuid":"b1818304-4583-47bf-a6ec-0bfbb5166233","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6818","nonce":1844689995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6822","nonce":1844689995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6830","nonce":1844689995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6826","nonce":1844689995}]},"public_addr":"172.21.15.132:6818/1844689995","cluster_addr":"172.21.15.132:6822/1844689995","heartbeat_back_addr":"172.21.15.132:6830/1844689995","heartbeat_front_addr":"172.21.15.132:6826/1844689995","state":["exists","up"]},{"osd":6,"uuid":"d37f4691-eed1-4a93-8818-18354f03f4a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6801","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6802","nonce":727581123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6805","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6807","nonce":727581123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6813","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6815","nonce":727581123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6811","nonce":727581123}]},"public_addr":"172.21.15.132:6802/727581123","cluster_addr":"172.21.15.132:6807/727581123","heartbeat_back_addr":"172.21.15.132:6815/727581123","heartbeat_front_addr":"172.21.15.132:6811/727581123","state":["exists","up"]},{"osd":7,"uuid":"4f712f62-5d1d-4816-8042-03209144f8b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6803","nonce":466001149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6806","nonce":466001149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6814","nonce":466001149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6810","nonce":466001149}]},"public_addr":"172.21.15.132:6803/466001149","cluster_addr":"172.21.15.132:6806/466001149","heartbeat_back_addr":"172.21.15.132:6814/466001149","heartbeat_front_addr":"172.21.15.132:6810/466001149","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-20T01:16:20.559 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:20.556+0000 7fde49c14700 1 -- 172.21.15.16:0/2130399484 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fde2406ac50 msgr2=0x7fde2406d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:20.560 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:20.556+0000 7fde49c14700 1 --2- 172.21.15.16:0/2130399484 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fde2406ac50 0x7fde2406d100 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fde34007fd0 tx=0x7fde34007de0 comp rx=0 tx=0).stop 2023-05-20T01:16:20.560 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:20.556+0000 7fde49c14700 1 -- 172.21.15.16:0/2130399484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde4406ef80 msgr2=0x7fde441f1ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:20.560 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:20.556+0000 7fde49c14700 1 --2- 172.21.15.16:0/2130399484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde4406ef80 0x7fde441f1ca0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fde2c00ebb0 tx=0x7fde2c00eec0 comp rx=0 tx=0).stop 2023-05-20T01:16:20.560 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:20.556+0000 7fde49c14700 1 -- 172.21.15.16:0/2130399484 shutdown_connections 2023-05-20T01:16:20.560 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:20.556+0000 7fde49c14700 1 --2- 172.21.15.16:0/2130399484 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fde2406ac50 0x7fde2406d100 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:20.560 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:20.556+0000 7fde49c14700 1 --2- 172.21.15.16:0/2130399484 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fde4406f540 0x7fde440cd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:20.561 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:20.556+0000 7fde49c14700 1 --2- 172.21.15.16:0/2130399484 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fde4406ac20 0x7fde440cbce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:20.561 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:20.556+0000 7fde49c14700 1 --2- 172.21.15.16:0/2130399484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde4406ef80 0x7fde441f1ca0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:20.561 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:20.556+0000 7fde49c14700 1 -- 172.21.15.16:0/2130399484 >> 172.21.15.16:0/2130399484 conn(0x7fde440b27d0 msgr2=0x7fde440c71e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:20.561 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:20.556+0000 7fde49c14700 1 -- 172.21.15.16:0/2130399484 shutdown_connections 2023-05-20T01:16:20.561 INFO:teuthology.misc.health.smithi016.stderr:2023-05-20T01:16:20.556+0000 7fde49c14700 1 -- 172.21.15.16:0/2130399484 wait complete. 2023-05-20T01:16:20.570 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-05-20T01:16:20.571 INFO:tasks.ceph:Creating RBD pool 2023-05-20T01:16:20.571 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --cluster ceph osd pool create rbd 8 2023-05-20T01:16:20.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:20.691+0000 7f9075294700 1 Processor -- start 2023-05-20T01:16:20.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:20.691+0000 7f9075294700 1 -- start start 2023-05-20T01:16:20.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:20.692+0000 7f9075294700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f907015cec0 0x7f9070071bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:20.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:20.692+0000 7f9075294700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90700cd630 con 0x7f9070072130 2023-05-20T01:16:20.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:20.692+0000 7f9075294700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9070072530 con 0x7f907015d290 2023-05-20T01:16:20.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:20.692+0000 7f9075294700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90700cca60 con 0x7f907015cec0 2023-05-20T01:16:20.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:20.692+0000 7f906f7fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f9070072130 0x7f9070160890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:40490/0 (socket says 172.21.15.16:40490) 2023-05-20T01:16:20.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:20.692+0000 7f906f7fe700 1 -- 172.21.15.16:0/968031334 learned_addr learned my addr 172.21.15.16:0/968031334 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:16:20.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:20.693+0000 7f906effd700 1 --2- 172.21.15.16:0/968031334 >> v2:172.21.15.132:3300/0 conn(0x7f907015cec0 0x7f9070071bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:20.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:20.693+0000 7f906dffb700 1 -- 172.21.15.16:0/968031334 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 14770709 0 0) 0x7f90700cd630 con 0x7f9070072130 2023-05-20T01:16:20.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:20.693+0000 7f906dffb700 1 -- 172.21.15.16:0/968031334 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f905c0035b0 con 0x7f9070072130 2023-05-20T01:16:20.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:20.693+0000 7f906effd700 1 -- 172.21.15.16:0/968031334 >> v1:172.21.15.132:6789/0 conn(0x7f907015d290 legacy=0x7f90700cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:20.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:20.693+0000 7f906effd700 1 -- 172.21.15.16:0/968031334 >> v1:172.21.15.16:6789/0 conn(0x7f9070072130 legacy=0x7f9070160890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:20.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:20.693+0000 7f906effd700 1 -- 172.21.15.16:0/968031334 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90700ccd00 con 0x7f907015cec0 2023-05-20T01:16:20.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:20.693+0000 7f906effd700 1 --2- 172.21.15.16:0/968031334 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f907015cec0 0x7f9070071bf0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f9058009820 tx=0x7f9058009b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3586b53f67db706 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:20.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:20.693+0000 7f906dffb700 1 -- 172.21.15.16:0/968031334 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9058005e30 con 0x7f907015cec0 2023-05-20T01:16:20.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:20.693+0000 7f906dffb700 1 -- 172.21.15.16:0/968031334 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9058006030 con 0x7f907015cec0 2023-05-20T01:16:20.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:20.693+0000 7f906dffb700 1 -- 172.21.15.16:0/968031334 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90580063c0 con 0x7f907015cec0 2023-05-20T01:16:20.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:20.694+0000 7f9075294700 1 -- 172.21.15.16:0/968031334 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f907015cec0 msgr2=0x7f9070071bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:20.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:20.694+0000 7f9075294700 1 --2- 172.21.15.16:0/968031334 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f907015cec0 0x7f9070071bf0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f9058009820 tx=0x7f9058009b30 comp rx=0 tx=0).stop 2023-05-20T01:16:20.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:20.694+0000 7f9075294700 1 -- 172.21.15.16:0/968031334 shutdown_connections 2023-05-20T01:16:20.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:20.694+0000 7f9075294700 1 --2- 172.21.15.16:0/968031334 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f907015cec0 0x7f9070071bf0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:20.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:20.694+0000 7f9075294700 1 -- 172.21.15.16:0/968031334 >> 172.21.15.16:0/968031334 conn(0x7f90700cffd0 msgr2=0x7f90700d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:20.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:20.694+0000 7f9075294700 1 -- 172.21.15.16:0/968031334 shutdown_connections 2023-05-20T01:16:20.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:20.694+0000 7f9075294700 1 -- 172.21.15.16:0/968031334 wait complete. 2023-05-20T01:16:20.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:20.695+0000 7f9075294700 1 Processor -- start 2023-05-20T01:16:20.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:20.695+0000 7f9075294700 1 -- start start 2023-05-20T01:16:20.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:20.696+0000 7f9075294700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9070072130 0x7f9070151e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:20.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:20.696+0000 7f9075294700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f907015cec0 0x7f90701f62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:20.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:20.696+0000 7f9075294700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f907015d290 0x7f90700ad140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:20.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:20.696+0000 7f9075294700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90700acd20 con 0x7f907015d290 2023-05-20T01:16:20.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:20.696+0000 7f9075294700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90700b29e0 con 0x7f907015cec0 2023-05-20T01:16:20.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:20.696+0000 7f9075294700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90701f7160 con 0x7f9070072130 2023-05-20T01:16:20.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:20.696+0000 7f906f7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f907015d290 0x7f90700ad140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:20.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:20.696+0000 7f906f7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f907015d290 0x7f90700ad140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57200/0 (socket says 172.21.15.16:57200) 2023-05-20T01:16:20.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:20.696+0000 7f906f7fe700 1 -- 172.21.15.16:0/475688944 learned_addr learned my addr 172.21.15.16:0/475688944 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:20.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:20.696+0000 7f906effd700 1 --2- 172.21.15.16:0/475688944 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9070072130 0x7f9070151e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:20.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:20.696+0000 7f906e7fc700 1 --2- 172.21.15.16:0/475688944 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f907015cec0 0x7f90701f62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:20.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:20.696+0000 7f906f7fe700 1 -- 172.21.15.16:0/475688944 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9070072130 msgr2=0x7f9070151e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:20.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:20.696+0000 7f906f7fe700 1 --2- 172.21.15.16:0/475688944 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9070072130 0x7f9070151e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:20.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:20.696+0000 7f906f7fe700 1 -- 172.21.15.16:0/475688944 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f907015cec0 msgr2=0x7f90701f62c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:20.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:20.696+0000 7f906f7fe700 1 --2- 172.21.15.16:0/475688944 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f907015cec0 0x7f90701f62c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:20.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:20.696+0000 7f906f7fe700 1 -- 172.21.15.16:0/475688944 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9058009430 con 0x7f907015d290 2023-05-20T01:16:20.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:20.697+0000 7f906f7fe700 1 --2- 172.21.15.16:0/475688944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f907015d290 0x7f90700ad140 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f906800cb40 tx=0x7f906800cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:20.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:20.697+0000 7f90677fe700 1 -- 172.21.15.16:0/475688944 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f906800ab50 con 0x7f907015d290 2023-05-20T01:16:20.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:20.697+0000 7f90677fe700 1 -- 172.21.15.16:0/475688944 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f906800acb0 con 0x7f907015d290 2023-05-20T01:16:20.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:20.697+0000 7f9075294700 1 -- 172.21.15.16:0/475688944 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90700cd2c0 con 0x7f907015d290 2023-05-20T01:16:20.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:20.697+0000 7f90677fe700 1 -- 172.21.15.16:0/475688944 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9068012620 con 0x7f907015d290 2023-05-20T01:16:20.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:20.697+0000 7f9075294700 1 -- 172.21.15.16:0/475688944 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90700ccfe0 con 0x7f907015d290 2023-05-20T01:16:20.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:20.698+0000 7f9075294700 1 -- 172.21.15.16:0/475688944 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90700ace60 con 0x7f907015d290 2023-05-20T01:16:20.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:20.698+0000 7f90677fe700 1 -- 172.21.15.16:0/475688944 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 4) v1 ==== 88981+0+0 (secure 0 0 0) 0x7f906801b3b0 con 0x7f907015d290 2023-05-20T01:16:20.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:20.699+0000 7f90677fe700 1 --2- 172.21.15.16:0/475688944 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f905006ac50 0x7f905006d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:20.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:20.699+0000 7f90677fe700 1 -- 172.21.15.16:0/475688944 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 4471+0+0 (secure 0 0 0) 0x7f9068088c50 con 0x7f907015d290 2023-05-20T01:16:20.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:20.699+0000 7f906effd700 1 --2- 172.21.15.16:0/475688944 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f905006ac50 0x7f905006d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:20.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:20.699+0000 7f906effd700 1 --2- 172.21.15.16:0/475688944 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f905006ac50 0x7f905006d100 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9058009b70 tx=0x7f9058008070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:20.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:20.703+0000 7f90677fe700 1 -- 172.21.15.16:0/475688944 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f90700ace60 con 0x7f907015d290 2023-05-20T01:16:20.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:20.903+0000 7f9075294700 1 -- 172.21.15.16:0/475688944 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7f90700ace60 con 0x7f907015d290 2023-05-20T01:16:21.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:21.814+0000 7f90677fe700 1 -- 172.21.15.16:0/475688944 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v15) v1 ==== 109+0+0 (secure 0 0 0) 0x7f90700ace60 con 0x7f907015d290 2023-05-20T01:16:21.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:21.817+0000 7f9075294700 1 -- 172.21.15.16:0/475688944 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f905006ac50 msgr2=0x7f905006d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:21.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:21.817+0000 7f9075294700 1 --2- 172.21.15.16:0/475688944 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f905006ac50 0x7f905006d100 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9058009b70 tx=0x7f9058008070 comp rx=0 tx=0).stop 2023-05-20T01:16:21.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:21.817+0000 7f9075294700 1 -- 172.21.15.16:0/475688944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f907015d290 msgr2=0x7f90700ad140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:21.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:21.817+0000 7f9075294700 1 --2- 172.21.15.16:0/475688944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f907015d290 0x7f90700ad140 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f906800cb40 tx=0x7f906800cf00 comp rx=0 tx=0).stop 2023-05-20T01:16:21.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:21.817+0000 7f9075294700 1 -- 172.21.15.16:0/475688944 shutdown_connections 2023-05-20T01:16:21.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:21.817+0000 7f9075294700 1 --2- 172.21.15.16:0/475688944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f907015d290 0x7f90700ad140 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:21.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:21.817+0000 7f9075294700 1 --2- 172.21.15.16:0/475688944 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f905006ac50 0x7f905006d100 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:21.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:21.817+0000 7f9075294700 1 --2- 172.21.15.16:0/475688944 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9070072130 0x7f9070151e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:21.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:21.818+0000 7f9075294700 1 --2- 172.21.15.16:0/475688944 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f907015cec0 0x7f90701f62c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:21.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:21.818+0000 7f9075294700 1 -- 172.21.15.16:0/475688944 >> 172.21.15.16:0/475688944 conn(0x7f90700cffd0 msgr2=0x7f907006a510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:21.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:21.818+0000 7f9075294700 1 -- 172.21.15.16:0/475688944 shutdown_connections 2023-05-20T01:16:21.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:21.818+0000 7f9075294700 1 -- 172.21.15.16:0/475688944 wait complete. 2023-05-20T01:16:21.824 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'rbd' created 2023-05-20T01:16:21.836 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2023-05-20T01:16:21.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:21.966+0000 7f8e5d4e6700 1 Processor -- start 2023-05-20T01:16:21.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:21.966+0000 7f8e5d4e6700 1 -- start start 2023-05-20T01:16:21.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:21.966+0000 7f8e5d4e6700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8e580d1bb0 0x7f8e58160820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:21.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:21.967+0000 7f8e5d4e6700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f8e580704f0 0x7f8e580cbe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:21.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:21.967+0000 7f8e5d4e6700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e580cb6f0 con 0x7f8e580cdab0 2023-05-20T01:16:21.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:21.967+0000 7f8e5d4e6700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e580d1f80 con 0x7f8e580d1bb0 2023-05-20T01:16:21.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:21.967+0000 7f8e5d4e6700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e5815d0d0 con 0x7f8e580704f0 2023-05-20T01:16:21.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:21.967+0000 7f8e56ffd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8e580d1bb0 0x7f8e58160820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:21.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:21.967+0000 7f8e567fc700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f8e580cdab0 0x7f8e5806fde0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:40520/0 (socket says 172.21.15.16:40520) 2023-05-20T01:16:21.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:21.967+0000 7f8e56ffd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8e580d1bb0 0x7f8e58160820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57224/0 (socket says 172.21.15.16:57224) 2023-05-20T01:16:21.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:21.967+0000 7f8e567fc700 1 -- 172.21.15.16:0/1186660750 learned_addr learned my addr 172.21.15.16:0/1186660750 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:16:21.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:21.967+0000 7f8e577fe700 1 --2- 172.21.15.16:0/1186660750 >> v2:172.21.15.132:3300/0 conn(0x7f8e580704f0 0x7f8e580cbe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:21.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:21.967+0000 7f8e56ffd700 1 -- 172.21.15.16:0/1186660750 >> v2:172.21.15.132:3300/0 conn(0x7f8e580704f0 msgr2=0x7f8e580cbe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:21.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:21.967+0000 7f8e56ffd700 1 --2- 172.21.15.16:0/1186660750 >> v2:172.21.15.132:3300/0 conn(0x7f8e580704f0 0x7f8e580cbe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:21.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:21.967+0000 7f8e56ffd700 1 -- 172.21.15.16:0/1186660750 >> v1:172.21.15.16:6789/0 conn(0x7f8e580cdab0 legacy=0x7f8e5806fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:21.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:21.967+0000 7f8e56ffd700 1 -- 172.21.15.16:0/1186660750 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e58160d60 con 0x7f8e580d1bb0 2023-05-20T01:16:21.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:21.967+0000 7f8e577fe700 1 --2- 172.21.15.16:0/1186660750 >> v2:172.21.15.132:3300/0 conn(0x7f8e580704f0 0x7f8e580cbe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:16:21.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:21.967+0000 7f8e55ffb700 1 -- 172.21.15.16:0/1186660750 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3805286439 0 0) 0x7f8e580cb6f0 con 0x7f8e580cdab0 2023-05-20T01:16:21.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:21.967+0000 7f8e56ffd700 1 --2- 172.21.15.16:0/1186660750 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e580d1bb0 0x7f8e58160820 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f8e580cb680 tx=0x7f8e48009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=937d8b8c60b59ef2 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:21.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:21.967+0000 7f8e55ffb700 1 -- 172.21.15.16:0/1186660750 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e48011830 con 0x7f8e580d1bb0 2023-05-20T01:16:21.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:21.967+0000 7f8e55ffb700 1 -- 172.21.15.16:0/1186660750 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e48011990 con 0x7f8e580d1bb0 2023-05-20T01:16:21.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:21.967+0000 7f8e55ffb700 1 -- 172.21.15.16:0/1186660750 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e48011d20 con 0x7f8e580d1bb0 2023-05-20T01:16:21.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:21.968+0000 7f8e5d4e6700 1 -- 172.21.15.16:0/1186660750 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e580d1bb0 msgr2=0x7f8e58160820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:21.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:21.968+0000 7f8e5d4e6700 1 --2- 172.21.15.16:0/1186660750 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e580d1bb0 0x7f8e58160820 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f8e580cb680 tx=0x7f8e48009b20 comp rx=0 tx=0).stop 2023-05-20T01:16:21.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:21.968+0000 7f8e5d4e6700 1 -- 172.21.15.16:0/1186660750 shutdown_connections 2023-05-20T01:16:21.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:21.968+0000 7f8e5d4e6700 1 --2- 172.21.15.16:0/1186660750 >> v2:172.21.15.132:3300/0 conn(0x7f8e580704f0 0x7f8e580cbe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:21.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:21.968+0000 7f8e5d4e6700 1 --2- 172.21.15.16:0/1186660750 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e580d1bb0 0x7f8e58160820 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:21.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:21.968+0000 7f8e5d4e6700 1 -- 172.21.15.16:0/1186660750 >> 172.21.15.16:0/1186660750 conn(0x7f8e5806e3b0 msgr2=0x7f8e5806e780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:21.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:21.968+0000 7f8e5d4e6700 1 -- 172.21.15.16:0/1186660750 shutdown_connections 2023-05-20T01:16:21.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:21.968+0000 7f8e5d4e6700 1 -- 172.21.15.16:0/1186660750 wait complete. 2023-05-20T01:16:21.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:21.969+0000 7f8e5d4e6700 1 Processor -- start 2023-05-20T01:16:21.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:21.969+0000 7f8e5d4e6700 1 -- start start 2023-05-20T01:16:21.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:21.970+0000 7f8e5d4e6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e580704f0 0x7f8e580b4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:21.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:21.970+0000 7f8e5d4e6700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8e580cdab0 0x7f8e581f3b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:21.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:21.970+0000 7f8e5d4e6700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8e580d1bb0 0x7f8e580b5820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:21.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:21.970+0000 7f8e5d4e6700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e580b5d60 con 0x7f8e580704f0 2023-05-20T01:16:21.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:21.970+0000 7f8e5d4e6700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e580c4d50 con 0x7f8e580cdab0 2023-05-20T01:16:21.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:21.970+0000 7f8e5d4e6700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e580c5030 con 0x7f8e580d1bb0 2023-05-20T01:16:21.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:21.970+0000 7f8e56ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e580704f0 0x7f8e580b4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:21.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:21.970+0000 7f8e56ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e580704f0 0x7f8e580b4ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57230/0 (socket says 172.21.15.16:57230) 2023-05-20T01:16:21.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:21.970+0000 7f8e56ffd700 1 -- 172.21.15.16:0/1779703734 learned_addr learned my addr 172.21.15.16:0/1779703734 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:21.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:21.970+0000 7f8e577fe700 1 --2- 172.21.15.16:0/1779703734 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8e580d1bb0 0x7f8e580b5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:21.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:21.970+0000 7f8e567fc700 1 --2- 172.21.15.16:0/1779703734 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8e580cdab0 0x7f8e581f3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:21.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:21.970+0000 7f8e56ffd700 1 -- 172.21.15.16:0/1779703734 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8e580d1bb0 msgr2=0x7f8e580b5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:21.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:21.970+0000 7f8e56ffd700 1 --2- 172.21.15.16:0/1779703734 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8e580d1bb0 0x7f8e580b5820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:21.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:21.970+0000 7f8e56ffd700 1 -- 172.21.15.16:0/1779703734 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8e580cdab0 msgr2=0x7f8e581f3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:21.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:21.970+0000 7f8e56ffd700 1 --2- 172.21.15.16:0/1779703734 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8e580cdab0 0x7f8e581f3b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:21.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:21.970+0000 7f8e56ffd700 1 -- 172.21.15.16:0/1779703734 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e480094a0 con 0x7f8e580704f0 2023-05-20T01:16:21.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:21.971+0000 7f8e56ffd700 1 --2- 172.21.15.16:0/1779703734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e580704f0 0x7f8e580b4ae0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f8e48014040 tx=0x7f8e48004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:21.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:21.971+0000 7f8e3f7fe700 1 -- 172.21.15.16:0/1779703734 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e480038a0 con 0x7f8e580704f0 2023-05-20T01:16:21.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:21.971+0000 7f8e3f7fe700 1 -- 172.21.15.16:0/1779703734 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e4801e070 con 0x7f8e580704f0 2023-05-20T01:16:21.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:21.971+0000 7f8e5d4e6700 1 -- 172.21.15.16:0/1779703734 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e581612e0 con 0x7f8e580704f0 2023-05-20T01:16:21.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:21.971+0000 7f8e3f7fe700 1 -- 172.21.15.16:0/1779703734 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e48003c40 con 0x7f8e580704f0 2023-05-20T01:16:21.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:21.971+0000 7f8e5d4e6700 1 -- 172.21.15.16:0/1779703734 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e58161000 con 0x7f8e580704f0 2023-05-20T01:16:21.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:21.972+0000 7f8e5d4e6700 1 -- 172.21.15.16:0/1779703734 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e581615c0 con 0x7f8e580704f0 2023-05-20T01:16:21.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:21.972+0000 7f8e3f7fe700 1 -- 172.21.15.16:0/1779703734 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 4) v1 ==== 88981+0+0 (secure 0 0 0) 0x7f8e4802d450 con 0x7f8e580704f0 2023-05-20T01:16:21.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:21.973+0000 7f8e3f7fe700 1 --2- 172.21.15.16:0/1779703734 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f8e4406ac50 0x7f8e4406d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:21.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:21.973+0000 7f8e567fc700 1 --2- 172.21.15.16:0/1779703734 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f8e4406ac50 0x7f8e4406d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:21.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:21.973+0000 7f8e3f7fe700 1 -- 172.21.15.16:0/1779703734 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 5188+0+0 (secure 0 0 0) 0x7f8e4808a200 con 0x7f8e580704f0 2023-05-20T01:16:21.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:21.973+0000 7f8e567fc700 1 --2- 172.21.15.16:0/1779703734 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f8e4406ac50 0x7f8e4406d100 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c000f60 tx=0x7f8e4c007be0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:21.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:21.978+0000 7f8e3f7fe700 1 -- 172.21.15.16:0/1779703734 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f8e480556e0 con 0x7f8e580704f0 2023-05-20T01:16:22.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:22.175+0000 7f8e5d4e6700 1 -- 172.21.15.16:0/1779703734 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f8e58066d50 con 0x7f8e580704f0 2023-05-20T01:16:22.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:22.820+0000 7f8e3f7fe700 1 -- 172.21.15.16:0/1779703734 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v16) v1 ==== 173+0+0 (secure 0 0 0) 0x7f8e480200c0 con 0x7f8e580704f0 2023-05-20T01:16:22.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:22.823+0000 7f8e5d4e6700 1 -- 172.21.15.16:0/1779703734 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f8e4406ac50 msgr2=0x7f8e4406d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:22.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:22.823+0000 7f8e5d4e6700 1 --2- 172.21.15.16:0/1779703734 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f8e4406ac50 0x7f8e4406d100 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c000f60 tx=0x7f8e4c007be0 comp rx=0 tx=0).stop 2023-05-20T01:16:22.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:22.823+0000 7f8e5d4e6700 1 -- 172.21.15.16:0/1779703734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e580704f0 msgr2=0x7f8e580b4ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:22.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:22.823+0000 7f8e5d4e6700 1 --2- 172.21.15.16:0/1779703734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e580704f0 0x7f8e580b4ae0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f8e48014040 tx=0x7f8e48004750 comp rx=0 tx=0).stop 2023-05-20T01:16:22.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:22.823+0000 7f8e5d4e6700 1 -- 172.21.15.16:0/1779703734 shutdown_connections 2023-05-20T01:16:22.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:22.823+0000 7f8e5d4e6700 1 --2- 172.21.15.16:0/1779703734 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f8e4406ac50 0x7f8e4406d100 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:22.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:22.823+0000 7f8e5d4e6700 1 --2- 172.21.15.16:0/1779703734 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8e580d1bb0 0x7f8e580b5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:22.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:22.823+0000 7f8e5d4e6700 1 --2- 172.21.15.16:0/1779703734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e580704f0 0x7f8e580b4ae0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:22.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:22.823+0000 7f8e5d4e6700 1 --2- 172.21.15.16:0/1779703734 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8e580cdab0 0x7f8e581f3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:22.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:22.823+0000 7f8e5d4e6700 1 -- 172.21.15.16:0/1779703734 >> 172.21.15.16:0/1779703734 conn(0x7f8e5806e3b0 msgr2=0x7f8e5815c470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:22.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:22.823+0000 7f8e5d4e6700 1 -- 172.21.15.16:0/1779703734 shutdown_connections 2023-05-20T01:16:22.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:22.823+0000 7f8e5d4e6700 1 -- 172.21.15.16:0/1779703734 wait complete. 2023-05-20T01:16:22.831 INFO:teuthology.orchestra.run.smithi016.stderr:enabled application 'rbd' on pool 'rbd' 2023-05-20T01:16:22.844 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-05-20T01:16:22.844 INFO:tasks.ceph.mds.a:Restarting daemon 2023-05-20T01:16:22.845 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-05-20T01:16:22.848 INFO:tasks.ceph.mds.a:Started 2023-05-20T01:16:22.848 INFO:tasks.ceph.mds.c:Restarting daemon 2023-05-20T01:16:22.849 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-05-20T01:16:22.851 INFO:tasks.ceph.mds.c:Started 2023-05-20T01:16:22.852 INFO:tasks.ceph.mds.b:Restarting daemon 2023-05-20T01:16:22.852 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-05-20T01:16:22.855 INFO:tasks.ceph.mds.b:Started 2023-05-20T01:16:22.855 INFO:tasks.ceph.mds.d:Restarting daemon 2023-05-20T01:16:22.856 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-05-20T01:16:22.859 INFO:tasks.ceph.mds.d:Started 2023-05-20T01:16:22.859 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-05-20T01:16:22.860 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-20T01:16:22.948 INFO:tasks.ceph.mds.c.smithi016.stdout:starting mds.c at 2023-05-20T01:16:22.949 INFO:tasks.ceph.mds.b.smithi132.stdout:starting mds.b at 2023-05-20T01:16:22.950 INFO:tasks.ceph.mds.d.smithi132.stdout:starting mds.d at 2023-05-20T01:16:22.951 INFO:tasks.ceph.mds.a.smithi016.stdout:starting mds.a at 2023-05-20T01:16:23.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.022+0000 7fa8cd087700 1 Processor -- start 2023-05-20T01:16:23.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.022+0000 7fa8cd087700 1 -- start start 2023-05-20T01:16:23.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.022+0000 7fa8cd087700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fa8c80cd650 0x7fa8c815cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:23.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.022+0000 7fa8cd087700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa8c80cdad0 0x7fa8c815d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:23.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.022+0000 7fa8cd087700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8c80b13b0 con 0x7fa8c8156d30 2023-05-20T01:16:23.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.022+0000 7fa8cd087700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8c80b1970 con 0x7fa8c80cdad0 2023-05-20T01:16:23.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.022+0000 7fa8cd087700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8c80cec60 con 0x7fa8c80cd650 2023-05-20T01:16:23.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.023+0000 7fa8c759e700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa8c80cdad0 0x7fa8c815d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:23.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.023+0000 7fa8c759e700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa8c80cdad0 0x7fa8c815d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57300/0 (socket says 172.21.15.16:57300) 2023-05-20T01:16:23.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.023+0000 7fa8c759e700 1 -- 172.21.15.16:0/3441196361 learned_addr learned my addr 172.21.15.16:0/3441196361 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:23.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.023+0000 7fa8c659c700 1 --2- 172.21.15.16:0/3441196361 >> v2:172.21.15.132:3300/0 conn(0x7fa8c80cd650 0x7fa8c815cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:23.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.023+0000 7fa8c759e700 1 -- 172.21.15.16:0/3441196361 >> v2:172.21.15.132:3300/0 conn(0x7fa8c80cd650 msgr2=0x7fa8c815cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:23.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.023+0000 7fa8c759e700 1 --2- 172.21.15.16:0/3441196361 >> v2:172.21.15.132:3300/0 conn(0x7fa8c80cd650 0x7fa8c815cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:23.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.023+0000 7fa8c759e700 1 -- 172.21.15.16:0/3441196361 >> v1:172.21.15.132:6789/0 conn(0x7fa8c8156d30 legacy=0x7fa8c80ce550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:23.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.023+0000 7fa8c759e700 1 -- 172.21.15.16:0/3441196361 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8c80b1690 con 0x7fa8c80cdad0 2023-05-20T01:16:23.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.023+0000 7fa8c759e700 1 --2- 172.21.15.16:0/3441196361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8c80cdad0 0x7fa8c815d0a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa8c806fa00 tx=0x7fa8c000d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73f2066a5d401976 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:23.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.023+0000 7fa8c5d9b700 1 -- 172.21.15.16:0/3441196361 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8c0010070 con 0x7fa8c80cdad0 2023-05-20T01:16:23.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.023+0000 7fa8c5d9b700 1 -- 172.21.15.16:0/3441196361 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8c000dd50 con 0x7fa8c80cdad0 2023-05-20T01:16:23.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.023+0000 7fa8c5d9b700 1 -- 172.21.15.16:0/3441196361 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8c000b310 con 0x7fa8c80cdad0 2023-05-20T01:16:23.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.024+0000 7fa8cd087700 1 -- 172.21.15.16:0/3441196361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8c80cdad0 msgr2=0x7fa8c815d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:23.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.024+0000 7fa8cd087700 1 --2- 172.21.15.16:0/3441196361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8c80cdad0 0x7fa8c815d0a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa8c806fa00 tx=0x7fa8c000d430 comp rx=0 tx=0).stop 2023-05-20T01:16:23.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.024+0000 7fa8cd087700 1 -- 172.21.15.16:0/3441196361 shutdown_connections 2023-05-20T01:16:23.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.024+0000 7fa8cd087700 1 --2- 172.21.15.16:0/3441196361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8c80cdad0 0x7fa8c815d0a0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:23.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.024+0000 7fa8cd087700 1 --2- 172.21.15.16:0/3441196361 >> v2:172.21.15.132:3300/0 conn(0x7fa8c80cd650 0x7fa8c815cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:23.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.024+0000 7fa8cd087700 1 -- 172.21.15.16:0/3441196361 >> 172.21.15.16:0/3441196361 conn(0x7fa8c80cb910 msgr2=0x7fa8c80cbd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:23.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.024+0000 7fa8cd087700 1 -- 172.21.15.16:0/3441196361 shutdown_connections 2023-05-20T01:16:23.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.024+0000 7fa8cd087700 1 -- 172.21.15.16:0/3441196361 wait complete. 2023-05-20T01:16:23.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.025+0000 7fa8cd087700 1 Processor -- start 2023-05-20T01:16:23.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.025+0000 7fa8cd087700 1 -- start start 2023-05-20T01:16:23.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.026+0000 7fa8cd087700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa8c80cd650 0x7fa8c8150f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:23.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.026+0000 7fa8cd087700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8c80cdad0 0x7fa8c80b47a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:23.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.026+0000 7fa8cd087700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa8c8156d30 0x7fa8c80b2640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:23.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.026+0000 7fa8cd087700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c8153630 con 0x7fa8c80cdad0 2023-05-20T01:16:23.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.026+0000 7fa8cd087700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c8153ed0 con 0x7fa8c8156d30 2023-05-20T01:16:23.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.026+0000 7fa8cd087700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c8152770 con 0x7fa8c80cd650 2023-05-20T01:16:23.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.026+0000 7fa8c659c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8c80cdad0 0x7fa8c80b47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:23.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.026+0000 7fa8c659c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8c80cdad0 0x7fa8c80b47a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57304/0 (socket says 172.21.15.16:57304) 2023-05-20T01:16:23.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.026+0000 7fa8c659c700 1 -- 172.21.15.16:0/4109014804 learned_addr learned my addr 172.21.15.16:0/4109014804 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:23.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.026+0000 7fa8c759e700 1 --2- 172.21.15.16:0/4109014804 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa8c8156d30 0x7fa8c80b2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:23.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.026+0000 7fa8c6d9d700 1 --2- 172.21.15.16:0/4109014804 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa8c80cd650 0x7fa8c8150f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:23.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.027+0000 7fa8c659c700 1 -- 172.21.15.16:0/4109014804 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa8c80cd650 msgr2=0x7fa8c8150f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:23.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.027+0000 7fa8c659c700 1 --2- 172.21.15.16:0/4109014804 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa8c80cd650 0x7fa8c8150f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:23.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.027+0000 7fa8c659c700 1 -- 172.21.15.16:0/4109014804 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa8c8156d30 msgr2=0x7fa8c80b2640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:23.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.027+0000 7fa8c659c700 1 --2- 172.21.15.16:0/4109014804 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa8c8156d30 0x7fa8c80b2640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:23.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.027+0000 7fa8c659c700 1 -- 172.21.15.16:0/4109014804 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8c000bd40 con 0x7fa8c80cdad0 2023-05-20T01:16:23.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.027+0000 7fa8c6d9d700 1 --2- 172.21.15.16:0/4109014804 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa8c80cd650 0x7fa8c8150f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:16:23.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.027+0000 7fa8c759e700 1 --2- 172.21.15.16:0/4109014804 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa8c8156d30 0x7fa8c80b2640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:16:23.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.027+0000 7fa8c659c700 1 --2- 172.21.15.16:0/4109014804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8c80cdad0 0x7fa8c80b47a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc00c820 tx=0x7fa8bc00cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:23.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.027+0000 7fa8af7fe700 1 -- 172.21.15.16:0/4109014804 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8bc011800 con 0x7fa8c80cdad0 2023-05-20T01:16:23.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.027+0000 7fa8af7fe700 1 -- 172.21.15.16:0/4109014804 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8bc011960 con 0x7fa8c80cdad0 2023-05-20T01:16:23.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.027+0000 7fa8af7fe700 1 -- 172.21.15.16:0/4109014804 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8bc011cf0 con 0x7fa8c80cdad0 2023-05-20T01:16:23.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.027+0000 7fa8cd087700 1 -- 172.21.15.16:0/4109014804 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8c80cd130 con 0x7fa8c80cdad0 2023-05-20T01:16:23.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.027+0000 7fa8cd087700 1 -- 172.21.15.16:0/4109014804 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8c80cce50 con 0x7fa8c80cdad0 2023-05-20T01:16:23.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.029+0000 7fa8af7fe700 1 -- 172.21.15.16:0/4109014804 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 4) v1 ==== 88981+0+0 (secure 0 0 0) 0x7fa8bc018360 con 0x7fa8c80cdad0 2023-05-20T01:16:23.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.029+0000 7fa8cd087700 1 -- 172.21.15.16:0/4109014804 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8c80cb4a0 con 0x7fa8c80cdad0 2023-05-20T01:16:23.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.029+0000 7fa8af7fe700 1 --2- 172.21.15.16:0/4109014804 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa8b406ac50 0x7fa8b406d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:23.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.029+0000 7fa8af7fe700 1 -- 172.21.15.16:0/4109014804 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fa8bc088ef0 con 0x7fa8c80cdad0 2023-05-20T01:16:23.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.029+0000 7fa8c6d9d700 1 --2- 172.21.15.16:0/4109014804 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa8b406ac50 0x7fa8b406d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:23.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.029+0000 7fa8c6d9d700 1 --2- 172.21.15.16:0/4109014804 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa8b406ac50 0x7fa8b406d100 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa8b8003fc0 tx=0x7fa8b8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:23.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.034+0000 7fa8af7fe700 1 -- 172.21.15.16:0/4109014804 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fa8bc054460 con 0x7fa8c80cdad0 2023-05-20T01:16:23.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.248+0000 7fa8cd087700 1 -- 172.21.15.16:0/4109014804 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa8c804e8c0 con 0x7fa8c80cdad0 2023-05-20T01:16:23.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.249+0000 7fa8af7fe700 1 -- 172.21.15.16:0/4109014804 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+14569 (secure 0 0 0) 0x7fa8bc01f090 con 0x7fa8c80cdad0 2023-05-20T01:16:23.252 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:16:23.253 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":16,"fsid":"c2245cc5-ecab-4087-a59b-bc81db901676","created":"2023-05-20T01:16:09.549230+0000","modified":"2023-05-20T01:16:22.813338+0000","last_up_change":"2023-05-20T01:16:19.788624+0000","last_in_change":"2023-05-20T01:16:12.092160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-05-20T01:16:20.934366+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-05-20T01:16:21.372294+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"317a83cd-19c8-4958-a301-6087ef342289","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6825","nonce":3923634179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6827","nonce":3923634179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6831","nonce":3923634179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6829","nonce":3923634179}]},"public_addr":"172.21.15.16:6825/3923634179","cluster_addr":"172.21.15.16:6827/3923634179","heartbeat_back_addr":"172.21.15.16:6831/3923634179","heartbeat_front_addr":"172.21.15.16:6829/3923634179","state":["exists","up"]},{"osd":1,"uuid":"5581046f-8219-4089-8d7d-b7846399505e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6809","nonce":3860264220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6811","nonce":3860264220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6815","nonce":3860264220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6813","nonce":3860264220}]},"public_addr":"172.21.15.16:6809/3860264220","cluster_addr":"172.21.15.16:6811/3860264220","heartbeat_back_addr":"172.21.15.16:6815/3860264220","heartbeat_front_addr":"172.21.15.16:6813/3860264220","state":["exists","up"]},{"osd":2,"uuid":"072e9e66-4133-478e-8d4a-64d46c2963f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6817","nonce":293187030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6819","nonce":293187030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6823","nonce":293187030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6821","nonce":293187030}]},"public_addr":"172.21.15.16:6817/293187030","cluster_addr":"172.21.15.16:6819/293187030","heartbeat_back_addr":"172.21.15.16:6823/293187030","heartbeat_front_addr":"172.21.15.16:6821/293187030","state":["exists","up"]},{"osd":3,"uuid":"11e1bba6-00f8-48dd-9028-d8d716b7f145","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6801","nonce":3669781890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6803","nonce":3669781890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6807","nonce":3669781890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6805","nonce":3669781890}]},"public_addr":"172.21.15.16:6801/3669781890","cluster_addr":"172.21.15.16:6803/3669781890","heartbeat_back_addr":"172.21.15.16:6807/3669781890","heartbeat_front_addr":"172.21.15.16:6805/3669781890","state":["exists","up"]},{"osd":4,"uuid":"b8909c3a-cf70-46cb-8d0b-009c707cf9d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6819","nonce":506317223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6823","nonce":506317223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6829","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6831","nonce":506317223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6827","nonce":506317223}]},"public_addr":"172.21.15.132:6819/506317223","cluster_addr":"172.21.15.132:6823/506317223","heartbeat_back_addr":"172.21.15.132:6831/506317223","heartbeat_front_addr":"172.21.15.132:6827/506317223","state":["exists","up"]},{"osd":5,"uuid":"b1818304-4583-47bf-a6ec-0bfbb5166233","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6818","nonce":1844689995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6822","nonce":1844689995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6830","nonce":1844689995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6826","nonce":1844689995}]},"public_addr":"172.21.15.132:6818/1844689995","cluster_addr":"172.21.15.132:6822/1844689995","heartbeat_back_addr":"172.21.15.132:6830/1844689995","heartbeat_front_addr":"172.21.15.132:6826/1844689995","state":["exists","up"]},{"osd":6,"uuid":"d37f4691-eed1-4a93-8818-18354f03f4a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6801","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6802","nonce":727581123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6805","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6807","nonce":727581123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6813","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6815","nonce":727581123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6811","nonce":727581123}]},"public_addr":"172.21.15.132:6802/727581123","cluster_addr":"172.21.15.132:6807/727581123","heartbeat_back_addr":"172.21.15.132:6815/727581123","heartbeat_front_addr":"172.21.15.132:6811/727581123","state":["exists","up"]},{"osd":7,"uuid":"4f712f62-5d1d-4816-8042-03209144f8b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6803","nonce":466001149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6806","nonce":466001149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6814","nonce":466001149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6810","nonce":466001149}]},"public_addr":"172.21.15.132:6803/466001149","cluster_addr":"172.21.15.132:6806/466001149","heartbeat_back_addr":"172.21.15.132:6814/466001149","heartbeat_front_addr":"172.21.15.132:6810/466001149","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.338863+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.278680+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305694+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.246835+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.276237+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.304441+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305421+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.267050+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-20T01:16:23.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.251+0000 7fa8cd087700 1 -- 172.21.15.16:0/4109014804 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa8b406ac50 msgr2=0x7fa8b406d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:23.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.251+0000 7fa8cd087700 1 --2- 172.21.15.16:0/4109014804 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa8b406ac50 0x7fa8b406d100 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa8b8003fc0 tx=0x7fa8b8009040 comp rx=0 tx=0).stop 2023-05-20T01:16:23.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.251+0000 7fa8cd087700 1 -- 172.21.15.16:0/4109014804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8c80cdad0 msgr2=0x7fa8c80b47a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:23.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.251+0000 7fa8cd087700 1 --2- 172.21.15.16:0/4109014804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8c80cdad0 0x7fa8c80b47a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc00c820 tx=0x7fa8bc00cbe0 comp rx=0 tx=0).stop 2023-05-20T01:16:23.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.251+0000 7fa8cd087700 1 -- 172.21.15.16:0/4109014804 shutdown_connections 2023-05-20T01:16:23.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.251+0000 7fa8cd087700 1 --2- 172.21.15.16:0/4109014804 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa8c8156d30 0x7fa8c80b2640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:23.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.251+0000 7fa8cd087700 1 --2- 172.21.15.16:0/4109014804 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa8b406ac50 0x7fa8b406d100 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:23.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.251+0000 7fa8cd087700 1 --2- 172.21.15.16:0/4109014804 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa8c80cd650 0x7fa8c8150f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:23.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.251+0000 7fa8cd087700 1 --2- 172.21.15.16:0/4109014804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8c80cdad0 0x7fa8c80b47a0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:23.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.251+0000 7fa8cd087700 1 -- 172.21.15.16:0/4109014804 >> 172.21.15.16:0/4109014804 conn(0x7fa8c80cb910 msgr2=0x7fa8c80cbce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:23.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.251+0000 7fa8cd087700 1 -- 172.21.15.16:0/4109014804 shutdown_connections 2023-05-20T01:16:23.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.251+0000 7fa8cd087700 1 -- 172.21.15.16:0/4109014804 wait complete. 2023-05-20T01:16:23.267 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-05-20T01:16:20.934366+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-05-20T01:16:21.372294+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-05-20T01:16:23.267 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-05-20T01:16:23.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.396+0000 7f010fa63700 1 Processor -- start 2023-05-20T01:16:23.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.397+0000 7f010fa63700 1 -- start start 2023-05-20T01:16:23.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.398+0000 7f010fa63700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f010815d290 0x7f01080cf9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:23.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.398+0000 7f010fa63700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01080cd630 con 0x7f010815cec0 2023-05-20T01:16:23.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.398+0000 7f010fa63700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01080cca60 con 0x7f0108072300 2023-05-20T01:16:23.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.398+0000 7f010fa63700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01080ccd00 con 0x7f010815d290 2023-05-20T01:16:23.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.398+0000 7f010e000700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f010815d290 0x7f01080cf9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:23.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.398+0000 7f010d7ff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f010815cec0 0x7f0108071bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:40614/0 (socket says 172.21.15.16:40614) 2023-05-20T01:16:23.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.398+0000 7f010d7ff700 1 -- 172.21.15.16:0/790531038 learned_addr learned my addr 172.21.15.16:0/790531038 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:16:23.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.398+0000 7f010cffe700 1 --1- 172.21.15.16:0/790531038 >> v1:172.21.15.132:6789/0 conn(0x7f0108072300 0x7f01080cf2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.16:50328/0 (socket says 172.21.15.16:50328) 2023-05-20T01:16:23.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.398+0000 7f010e000700 1 -- 172.21.15.16:0/790531038 >> v1:172.21.15.132:6789/0 conn(0x7f0108072300 legacy=0x7f01080cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:23.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.398+0000 7f010e000700 1 -- 172.21.15.16:0/790531038 >> v1:172.21.15.16:6789/0 conn(0x7f010815cec0 legacy=0x7f0108071bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:23.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.398+0000 7f010e000700 1 -- 172.21.15.16:0/790531038 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01080ccfe0 con 0x7f010815d290 2023-05-20T01:16:23.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.398+0000 7f00fffff700 1 -- 172.21.15.16:0/790531038 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 484689774 0 0) 0x7f01080cca60 con 0x7f0108072300 2023-05-20T01:16:23.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.398+0000 7f010e000700 1 --2- 172.21.15.16:0/790531038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f010815d290 0x7f01080cf9d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0100009d90 tx=0x7f010000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6edff5d4a7bc46c server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:23.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.398+0000 7f00fffff700 1 -- 172.21.15.16:0/790531038 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f010000e070 con 0x7f010815d290 2023-05-20T01:16:23.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.399+0000 7f00fffff700 1 -- 172.21.15.16:0/790531038 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f010000bad0 con 0x7f010815d290 2023-05-20T01:16:23.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.399+0000 7f00fffff700 1 -- 172.21.15.16:0/790531038 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f010000be80 con 0x7f010815d290 2023-05-20T01:16:23.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.399+0000 7f010fa63700 1 -- 172.21.15.16:0/790531038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f010815d290 msgr2=0x7f01080cf9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:23.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.399+0000 7f010fa63700 1 --2- 172.21.15.16:0/790531038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f010815d290 0x7f01080cf9d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0100009d90 tx=0x7f010000b330 comp rx=0 tx=0).stop 2023-05-20T01:16:23.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.399+0000 7f010fa63700 1 -- 172.21.15.16:0/790531038 shutdown_connections 2023-05-20T01:16:23.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.399+0000 7f010fa63700 1 --2- 172.21.15.16:0/790531038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f010815d290 0x7f01080cf9d0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:23.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.399+0000 7f010fa63700 1 -- 172.21.15.16:0/790531038 >> 172.21.15.16:0/790531038 conn(0x7f01080cffd0 msgr2=0x7f01080d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:23.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.399+0000 7f010fa63700 1 -- 172.21.15.16:0/790531038 shutdown_connections 2023-05-20T01:16:23.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.399+0000 7f010fa63700 1 -- 172.21.15.16:0/790531038 wait complete. 2023-05-20T01:16:23.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.401+0000 7f010fa63700 1 Processor -- start 2023-05-20T01:16:23.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.401+0000 7f010fa63700 1 -- start start 2023-05-20T01:16:23.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.402+0000 7f010fa63700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0108072300 0x7f01080b58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:23.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.402+0000 7f010fa63700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f010815cec0 0x7f01080b4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:23.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.402+0000 7f010fa63700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f010815d290 0x7f01081528b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:23.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.402+0000 7f010fa63700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0108152df0 con 0x7f0108072300 2023-05-20T01:16:23.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.402+0000 7f010fa63700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01080b4840 con 0x7f010815d290 2023-05-20T01:16:23.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.402+0000 7f010fa63700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01080b4b20 con 0x7f010815cec0 2023-05-20T01:16:23.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.402+0000 7f010d7ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0108072300 0x7f01080b58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:23.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.402+0000 7f010d7ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0108072300 0x7f01080b58e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57324/0 (socket says 172.21.15.16:57324) 2023-05-20T01:16:23.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.402+0000 7f010d7ff700 1 -- 172.21.15.16:0/203344254 learned_addr learned my addr 172.21.15.16:0/203344254 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:23.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.402+0000 7f010cffe700 1 --2- 172.21.15.16:0/203344254 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f010815cec0 0x7f01080b4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:23.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.402+0000 7f010e000700 1 --2- 172.21.15.16:0/203344254 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f010815d290 0x7f01081528b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:23.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.402+0000 7f010d7ff700 1 -- 172.21.15.16:0/203344254 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f010815cec0 msgr2=0x7f01080b4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:23.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.402+0000 7f010d7ff700 1 --2- 172.21.15.16:0/203344254 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f010815cec0 0x7f01080b4140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:23.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.402+0000 7f010d7ff700 1 -- 172.21.15.16:0/203344254 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f010815d290 msgr2=0x7f01081528b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:23.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.402+0000 7f010d7ff700 1 --2- 172.21.15.16:0/203344254 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f010815d290 0x7f01081528b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:23.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.402+0000 7f010d7ff700 1 -- 172.21.15.16:0/203344254 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0100009a70 con 0x7f0108072300 2023-05-20T01:16:23.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.402+0000 7f010e000700 1 --2- 172.21.15.16:0/203344254 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f010815d290 0x7f01081528b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:16:23.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.402+0000 7f010d7ff700 1 --2- 172.21.15.16:0/203344254 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0108072300 0x7f01080b58e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f00f800c780 tx=0x7f00f800cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:23.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.403+0000 7f00fdffb700 1 -- 172.21.15.16:0/203344254 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f8005c40 con 0x7f0108072300 2023-05-20T01:16:23.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.403+0000 7f00fdffb700 1 -- 172.21.15.16:0/203344254 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00f8005da0 con 0x7f0108072300 2023-05-20T01:16:23.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.403+0000 7f010fa63700 1 -- 172.21.15.16:0/203344254 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0108167b30 con 0x7f0108072300 2023-05-20T01:16:23.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.403+0000 7f00fdffb700 1 -- 172.21.15.16:0/203344254 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f8007270 con 0x7f0108072300 2023-05-20T01:16:23.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.403+0000 7f010fa63700 1 -- 172.21.15.16:0/203344254 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01080cd2c0 con 0x7f0108072300 2023-05-20T01:16:23.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.404+0000 7f010fa63700 1 -- 172.21.15.16:0/203344254 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0108167cc0 con 0x7f0108072300 2023-05-20T01:16:23.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.404+0000 7f00fdffb700 1 -- 172.21.15.16:0/203344254 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 4) v1 ==== 88981+0+0 (secure 0 0 0) 0x7f00f80073d0 con 0x7f0108072300 2023-05-20T01:16:23.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.404+0000 7f00fdffb700 1 --2- 172.21.15.16:0/203344254 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f00ec06ac50 0x7f00ec06d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:23.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.404+0000 7f00fdffb700 1 -- 172.21.15.16:0/203344254 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f00f8088b90 con 0x7f0108072300 2023-05-20T01:16:23.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.409+0000 7f00fdffb700 1 -- 172.21.15.16:0/203344254 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f00f8054100 con 0x7f0108072300 2023-05-20T01:16:23.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.413+0000 7f010cffe700 1 --2- 172.21.15.16:0/203344254 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f00ec06ac50 0x7f00ec06d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:23.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.413+0000 7f010cffe700 1 --2- 172.21.15.16:0/203344254 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f00ec06ac50 0x7f00ec06d100 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0104007fd0 tx=0x7f0104007ba0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:23.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.608+0000 7f010fa63700 1 -- 172.21.15.16:0/203344254 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f01080ce9e0 con 0x7f0108072300 2023-05-20T01:16:23.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.608+0000 7f00fdffb700 1 -- 172.21.15.16:0/203344254 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v16) v1 ==== 92+0+10 (secure 0 0 0) 0x7f00f8057720 con 0x7f0108072300 2023-05-20T01:16:23.613 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 8 2023-05-20T01:16:23.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.611+0000 7f010fa63700 1 -- 172.21.15.16:0/203344254 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f00ec06ac50 msgr2=0x7f00ec06d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:23.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.611+0000 7f010fa63700 1 --2- 172.21.15.16:0/203344254 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f00ec06ac50 0x7f00ec06d100 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0104007fd0 tx=0x7f0104007ba0 comp rx=0 tx=0).stop 2023-05-20T01:16:23.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.611+0000 7f010fa63700 1 -- 172.21.15.16:0/203344254 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0108072300 msgr2=0x7f01080b58e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:23.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.611+0000 7f010fa63700 1 --2- 172.21.15.16:0/203344254 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0108072300 0x7f01080b58e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f00f800c780 tx=0x7f00f800cb40 comp rx=0 tx=0).stop 2023-05-20T01:16:23.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.611+0000 7f010fa63700 1 -- 172.21.15.16:0/203344254 shutdown_connections 2023-05-20T01:16:23.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.611+0000 7f010fa63700 1 --2- 172.21.15.16:0/203344254 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f010815d290 0x7f01081528b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:23.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.611+0000 7f010fa63700 1 --2- 172.21.15.16:0/203344254 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0108072300 0x7f01080b58e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:23.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.611+0000 7f010fa63700 1 --2- 172.21.15.16:0/203344254 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f00ec06ac50 0x7f00ec06d100 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:23.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.611+0000 7f010fa63700 1 --2- 172.21.15.16:0/203344254 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f010815cec0 0x7f01080b4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:23.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.611+0000 7f010fa63700 1 -- 172.21.15.16:0/203344254 >> 172.21.15.16:0/203344254 conn(0x7f01080cffd0 msgr2=0x7f010815c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:23.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.611+0000 7f010fa63700 1 -- 172.21.15.16:0/203344254 shutdown_connections 2023-05-20T01:16:23.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.611+0000 7f010fa63700 1 -- 172.21.15.16:0/203344254 wait complete. 2023-05-20T01:16:23.626 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-05-20T01:16:23.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.751+0000 7f8b22271700 1 Processor -- start 2023-05-20T01:16:23.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.751+0000 7f8b22271700 1 -- start start 2023-05-20T01:16:23.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.752+0000 7f8b22271700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8b1c1572c0 0x7f8b1c159e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:23.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.752+0000 7f8b22271700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f8b1c06e150 0x7f8b1c06e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:23.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.752+0000 7f8b22271700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b1c073c10 con 0x7f8b1c071e10 2023-05-20T01:16:23.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.752+0000 7f8b22271700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b1c06ead0 con 0x7f8b1c1572c0 2023-05-20T01:16:23.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.752+0000 7f8b22271700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b1c15a600 con 0x7f8b1c06e150 2023-05-20T01:16:23.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.752+0000 7f8b1bfff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8b1c1572c0 0x7f8b1c159e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:23.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.752+0000 7f8b2080e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f8b1c06e150 0x7f8b1c06e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:23.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.752+0000 7f8b1b7fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f8b1c071e10 0x7f8b1c15db20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:40638/0 (socket says 172.21.15.16:40638) 2023-05-20T01:16:23.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.752+0000 7f8b1b7fe700 1 -- 172.21.15.16:0/2194684415 learned_addr learned my addr 172.21.15.16:0/2194684415 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:16:23.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.753+0000 7f8b2080e700 1 -- 172.21.15.16:0/2194684415 >> v2:172.21.15.16:3300/0 conn(0x7f8b1c1572c0 msgr2=0x7f8b1c159e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:23.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.753+0000 7f8b2080e700 1 --2- 172.21.15.16:0/2194684415 >> v2:172.21.15.16:3300/0 conn(0x7f8b1c1572c0 0x7f8b1c159e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:23.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.753+0000 7f8b2080e700 1 -- 172.21.15.16:0/2194684415 >> v1:172.21.15.16:6789/0 conn(0x7f8b1c071e10 legacy=0x7f8b1c15db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:23.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.753+0000 7f8b2080e700 1 -- 172.21.15.16:0/2194684415 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b1c15a3d0 con 0x7f8b1c06e150 2023-05-20T01:16:23.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.753+0000 7f8b2080e700 1 --2- 172.21.15.16:0/2194684415 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8b1c06e150 0x7f8b1c06e590 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c0706e0 tx=0x7f8b1400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b79c05db6a1bcd7a server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:23.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.753+0000 7f8b1affd700 1 -- 172.21.15.16:0/2194684415 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b1400e070 con 0x7f8b1c06e150 2023-05-20T01:16:23.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.753+0000 7f8b1affd700 1 -- 172.21.15.16:0/2194684415 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b1400bb20 con 0x7f8b1c06e150 2023-05-20T01:16:23.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.753+0000 7f8b1affd700 1 -- 172.21.15.16:0/2194684415 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b14012430 con 0x7f8b1c06e150 2023-05-20T01:16:23.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.754+0000 7f8b22271700 1 -- 172.21.15.16:0/2194684415 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8b1c06e150 msgr2=0x7f8b1c06e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:23.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.754+0000 7f8b22271700 1 --2- 172.21.15.16:0/2194684415 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8b1c06e150 0x7f8b1c06e590 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c0706e0 tx=0x7f8b1400b330 comp rx=0 tx=0).stop 2023-05-20T01:16:23.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.754+0000 7f8b22271700 1 -- 172.21.15.16:0/2194684415 shutdown_connections 2023-05-20T01:16:23.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.754+0000 7f8b22271700 1 --2- 172.21.15.16:0/2194684415 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8b1c06e150 0x7f8b1c06e590 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:23.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.754+0000 7f8b22271700 1 --2- 172.21.15.16:0/2194684415 >> v2:172.21.15.16:3300/0 conn(0x7f8b1c1572c0 0x7f8b1c159e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:23.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.754+0000 7f8b22271700 1 -- 172.21.15.16:0/2194684415 >> 172.21.15.16:0/2194684415 conn(0x7f8b1c06cbb0 msgr2=0x7f8b1c06cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:23.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.754+0000 7f8b22271700 1 -- 172.21.15.16:0/2194684415 shutdown_connections 2023-05-20T01:16:23.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.754+0000 7f8b22271700 1 -- 172.21.15.16:0/2194684415 wait complete. 2023-05-20T01:16:23.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.756+0000 7f8b22271700 1 Processor -- start 2023-05-20T01:16:23.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.756+0000 7f8b22271700 1 -- start start 2023-05-20T01:16:23.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.757+0000 7f8b22271700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8b1c06e150 0x7f8b1c0b2720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:23.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.757+0000 7f8b22271700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8b1c071e10 0x7f8b1c0b2c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:23.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.757+0000 7f8b22271700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b1c1572c0 0x7f8b1c1ee0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:23.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.757+0000 7f8b22271700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b1c1ef640 con 0x7f8b1c1572c0 2023-05-20T01:16:23.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.757+0000 7f8b22271700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b1c0c99f0 con 0x7f8b1c06e150 2023-05-20T01:16:23.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.757+0000 7f8b22271700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b1c1ef280 con 0x7f8b1c071e10 2023-05-20T01:16:23.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.757+0000 7f8b2080e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b1c1572c0 0x7f8b1c1ee0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:23.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.757+0000 7f8b2080e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b1c1572c0 0x7f8b1c1ee0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57350/0 (socket says 172.21.15.16:57350) 2023-05-20T01:16:23.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.757+0000 7f8b2080e700 1 -- 172.21.15.16:0/2518549524 learned_addr learned my addr 172.21.15.16:0/2518549524 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:23.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.757+0000 7f8b1bfff700 1 --2- 172.21.15.16:0/2518549524 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8b1c06e150 0x7f8b1c0b2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:23.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.757+0000 7f8b1b7fe700 1 --2- 172.21.15.16:0/2518549524 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8b1c071e10 0x7f8b1c0b2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:23.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.757+0000 7f8b2080e700 1 -- 172.21.15.16:0/2518549524 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8b1c071e10 msgr2=0x7f8b1c0b2c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:23.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.757+0000 7f8b2080e700 1 --2- 172.21.15.16:0/2518549524 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8b1c071e10 0x7f8b1c0b2c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:23.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.757+0000 7f8b2080e700 1 -- 172.21.15.16:0/2518549524 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8b1c06e150 msgr2=0x7f8b1c0b2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:23.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.757+0000 7f8b2080e700 1 --2- 172.21.15.16:0/2518549524 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8b1c06e150 0x7f8b1c0b2720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:23.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.757+0000 7f8b2080e700 1 -- 172.21.15.16:0/2518549524 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b14009a70 con 0x7f8b1c1572c0 2023-05-20T01:16:23.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.757+0000 7f8b1bfff700 1 --2- 172.21.15.16:0/2518549524 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8b1c06e150 0x7f8b1c0b2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:16:23.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.757+0000 7f8b2080e700 1 --2- 172.21.15.16:0/2518549524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b1c1572c0 0x7f8b1c1ee0b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8b14014040 tx=0x7f8b14004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:23.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.808+0000 7f8b18ff9700 1 -- 172.21.15.16:0/2518549524 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b14012430 con 0x7f8b1c1572c0 2023-05-20T01:16:23.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.808+0000 7f8b22271700 1 -- 172.21.15.16:0/2518549524 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b1c15abc0 con 0x7f8b1c1572c0 2023-05-20T01:16:23.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.808+0000 7f8b18ff9700 1 -- 172.21.15.16:0/2518549524 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b14020070 con 0x7f8b1c1572c0 2023-05-20T01:16:23.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.808+0000 7f8b18ff9700 1 -- 172.21.15.16:0/2518549524 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b14004280 con 0x7f8b1c1572c0 2023-05-20T01:16:23.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.808+0000 7f8b22271700 1 -- 172.21.15.16:0/2518549524 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b1c15a8e0 con 0x7f8b1c1572c0 2023-05-20T01:16:23.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.809+0000 7f8b22271700 1 -- 172.21.15.16:0/2518549524 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b1c15d600 con 0x7f8b1c1572c0 2023-05-20T01:16:23.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.809+0000 7f8b18ff9700 1 -- 172.21.15.16:0/2518549524 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 4) v1 ==== 88981+0+0 (secure 0 0 0) 0x7f8b140043e0 con 0x7f8b1c1572c0 2023-05-20T01:16:23.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.810+0000 7f8b18ff9700 1 --2- 172.21.15.16:0/2518549524 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f8b0806ac50 0x7f8b0806d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:23.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.810+0000 7f8b18ff9700 1 -- 172.21.15.16:0/2518549524 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f8b14022070 con 0x7f8b1c1572c0 2023-05-20T01:16:23.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.810+0000 7f8b1bfff700 1 --2- 172.21.15.16:0/2518549524 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f8b0806ac50 0x7f8b0806d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:23.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.810+0000 7f8b1bfff700 1 --2- 172.21.15.16:0/2518549524 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f8b0806ac50 0x7f8b0806d100 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8b0c00be30 tx=0x7f8b0c00b420 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:23.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:23.815+0000 7f8b18ff9700 1 -- 172.21.15.16:0/2518549524 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f8b14056270 con 0x7f8b1c1572c0 2023-05-20T01:16:24.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.017+0000 7f8b22271700 1 -- 172.21.15.16:0/2518549524 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8b1c0747b0 con 0x7f8b1c1572c0 2023-05-20T01:16:24.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.018+0000 7f8b18ff9700 1 -- 172.21.15.16:0/2518549524 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8b1402c0a0 con 0x7f8b1c1572c0 2023-05-20T01:16:24.022 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-05-20T01:16:24.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.020+0000 7f8b22271700 1 -- 172.21.15.16:0/2518549524 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f8b0806ac50 msgr2=0x7f8b0806d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:24.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.020+0000 7f8b22271700 1 --2- 172.21.15.16:0/2518549524 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f8b0806ac50 0x7f8b0806d100 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8b0c00be30 tx=0x7f8b0c00b420 comp rx=0 tx=0).stop 2023-05-20T01:16:24.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.020+0000 7f8b22271700 1 -- 172.21.15.16:0/2518549524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b1c1572c0 msgr2=0x7f8b1c1ee0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:24.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.020+0000 7f8b22271700 1 --2- 172.21.15.16:0/2518549524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b1c1572c0 0x7f8b1c1ee0b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8b14014040 tx=0x7f8b14004ad0 comp rx=0 tx=0).stop 2023-05-20T01:16:24.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.020+0000 7f8b22271700 1 -- 172.21.15.16:0/2518549524 shutdown_connections 2023-05-20T01:16:24.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.020+0000 7f8b22271700 1 --2- 172.21.15.16:0/2518549524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b1c1572c0 0x7f8b1c1ee0b0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:24.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.020+0000 7f8b22271700 1 --2- 172.21.15.16:0/2518549524 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8b1c06e150 0x7f8b1c0b2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:24.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.020+0000 7f8b22271700 1 --2- 172.21.15.16:0/2518549524 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f8b0806ac50 0x7f8b0806d100 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:24.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.020+0000 7f8b22271700 1 --2- 172.21.15.16:0/2518549524 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8b1c071e10 0x7f8b1c0b2c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:24.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.020+0000 7f8b22271700 1 -- 172.21.15.16:0/2518549524 >> 172.21.15.16:0/2518549524 conn(0x7f8b1c06cbb0 msgr2=0x7f8b1c06cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:24.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.020+0000 7f8b22271700 1 -- 172.21.15.16:0/2518549524 shutdown_connections 2023-05-20T01:16:24.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.020+0000 7f8b22271700 1 -- 172.21.15.16:0/2518549524 wait complete. 2023-05-20T01:16:24.036 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-20T01:16:24.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.160+0000 7f0ca5c10700 1 Processor -- start 2023-05-20T01:16:24.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.160+0000 7f0ca5c10700 1 -- start start 2023-05-20T01:16:24.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.161+0000 7f0ca5c10700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0ca0157190 0x7f0ca00cc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:24.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.161+0000 7f0ca5c10700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0ca0071ad0 0x7f0ca00ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:24.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.161+0000 7f0ca5c10700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ca0071f20 con 0x7f0ca00cccf0 2023-05-20T01:16:24.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.161+0000 7f0ca5c10700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ca00cb590 con 0x7f0ca0071ad0 2023-05-20T01:16:24.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.161+0000 7f0ca5c10700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca00ca9a0 con 0x7f0ca0157190 2023-05-20T01:16:24.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.161+0000 7f0c9ffff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0ca0071ad0 0x7f0ca00ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:24.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.161+0000 7f0c9ffff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0ca0071ad0 0x7f0ca00ce740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57364/0 (socket says 172.21.15.16:57364) 2023-05-20T01:16:24.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.161+0000 7f0c9ffff700 1 -- 172.21.15.16:0/3050923445 learned_addr learned my addr 172.21.15.16:0/3050923445 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:24.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.161+0000 7f0c9f7fe700 1 --2- 172.21.15.16:0/3050923445 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0ca0157190 0x7f0ca00cc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:24.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.161+0000 7f0c9ffff700 1 -- 172.21.15.16:0/3050923445 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0ca0157190 msgr2=0x7f0ca00cc7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:24.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.161+0000 7f0c9ffff700 1 --2- 172.21.15.16:0/3050923445 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0ca0157190 0x7f0ca00cc7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:24.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.161+0000 7f0c9ffff700 1 -- 172.21.15.16:0/3050923445 >> v1:172.21.15.132:6789/0 conn(0x7f0ca00cccf0 legacy=0x7f0ca00713c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:24.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.161+0000 7f0c9ffff700 1 -- 172.21.15.16:0/3050923445 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ca00cac60 con 0x7f0ca0071ad0 2023-05-20T01:16:24.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.162+0000 7f0c9ffff700 1 --2- 172.21.15.16:0/3050923445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ca0071ad0 0x7f0ca00ce740 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0c9800d150 tx=0x7f0c9800d460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7eb6c39fa6f13679 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:24.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.162+0000 7f0c9e7fc700 1 -- 172.21.15.16:0/3050923445 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c98010070 con 0x7f0ca0071ad0 2023-05-20T01:16:24.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.162+0000 7f0c9e7fc700 1 -- 172.21.15.16:0/3050923445 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c9800dd10 con 0x7f0ca0071ad0 2023-05-20T01:16:24.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.162+0000 7f0c9e7fc700 1 -- 172.21.15.16:0/3050923445 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c9800b310 con 0x7f0ca0071ad0 2023-05-20T01:16:24.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.162+0000 7f0ca5c10700 1 -- 172.21.15.16:0/3050923445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ca0071ad0 msgr2=0x7f0ca00ce740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:24.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.162+0000 7f0ca5c10700 1 --2- 172.21.15.16:0/3050923445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ca0071ad0 0x7f0ca00ce740 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0c9800d150 tx=0x7f0c9800d460 comp rx=0 tx=0).stop 2023-05-20T01:16:24.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.162+0000 7f0ca5c10700 1 -- 172.21.15.16:0/3050923445 shutdown_connections 2023-05-20T01:16:24.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.162+0000 7f0ca5c10700 1 --2- 172.21.15.16:0/3050923445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ca0071ad0 0x7f0ca00ce740 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:24.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.162+0000 7f0ca5c10700 1 --2- 172.21.15.16:0/3050923445 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0ca0157190 0x7f0ca00cc7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:24.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.162+0000 7f0ca5c10700 1 -- 172.21.15.16:0/3050923445 >> 172.21.15.16:0/3050923445 conn(0x7f0ca00cffb0 msgr2=0x7f0ca00d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:24.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.162+0000 7f0ca5c10700 1 -- 172.21.15.16:0/3050923445 shutdown_connections 2023-05-20T01:16:24.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.162+0000 7f0ca5c10700 1 -- 172.21.15.16:0/3050923445 wait complete. 2023-05-20T01:16:24.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.164+0000 7f0ca5c10700 1 Processor -- start 2023-05-20T01:16:24.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.164+0000 7f0ca5c10700 1 -- start start 2023-05-20T01:16:24.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.164+0000 7f0ca5c10700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0ca0071ad0 0x7f0ca01f6760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:24.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.165+0000 7f0ca5c10700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0ca00cccf0 0x7f0ca01f25a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:24.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.165+0000 7f0ca5c10700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ca0157190 0x7f0ca006f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:24.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.165+0000 7f0ca5c10700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca00c4b30 con 0x7f0ca0157190 2023-05-20T01:16:24.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.165+0000 7f0ca5c10700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca01f2df0 con 0x7f0ca0071ad0 2023-05-20T01:16:24.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.165+0000 7f0ca5c10700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca01f30d0 con 0x7f0ca00cccf0 2023-05-20T01:16:24.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.165+0000 7f0c9ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ca0157190 0x7f0ca006f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:24.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.165+0000 7f0c9ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ca0157190 0x7f0ca006f1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57370/0 (socket says 172.21.15.16:57370) 2023-05-20T01:16:24.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.165+0000 7f0c9ffff700 1 -- 172.21.15.16:0/1206715272 learned_addr learned my addr 172.21.15.16:0/1206715272 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:24.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.165+0000 7f0c9f7fe700 1 --2- 172.21.15.16:0/1206715272 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0ca0071ad0 0x7f0ca01f6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:24.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.165+0000 7f0c9effd700 1 --2- 172.21.15.16:0/1206715272 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0ca00cccf0 0x7f0ca01f25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:24.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.165+0000 7f0c9ffff700 1 -- 172.21.15.16:0/1206715272 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0ca00cccf0 msgr2=0x7f0ca01f25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:24.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.165+0000 7f0c9ffff700 1 --2- 172.21.15.16:0/1206715272 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0ca00cccf0 0x7f0ca01f25a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:24.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.165+0000 7f0c9ffff700 1 -- 172.21.15.16:0/1206715272 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0ca0071ad0 msgr2=0x7f0ca01f6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:24.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.165+0000 7f0c9ffff700 1 --2- 172.21.15.16:0/1206715272 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0ca0071ad0 0x7f0ca01f6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:24.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.165+0000 7f0c9ffff700 1 -- 172.21.15.16:0/1206715272 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c9800bd40 con 0x7f0ca0157190 2023-05-20T01:16:24.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.165+0000 7f0c9ffff700 1 --2- 172.21.15.16:0/1206715272 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ca0157190 0x7f0ca006f1f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0ca00d0f50 tx=0x7f0c98004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:24.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.165+0000 7f0c97fff700 1 -- 172.21.15.16:0/1206715272 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c98009a10 con 0x7f0ca0157190 2023-05-20T01:16:24.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.165+0000 7f0c97fff700 1 -- 172.21.15.16:0/1206715272 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c98009b70 con 0x7f0ca0157190 2023-05-20T01:16:24.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.165+0000 7f0ca5c10700 1 -- 172.21.15.16:0/1206715272 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ca00cb220 con 0x7f0ca0157190 2023-05-20T01:16:24.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.165+0000 7f0c97fff700 1 -- 172.21.15.16:0/1206715272 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c9800b4c0 con 0x7f0ca0157190 2023-05-20T01:16:24.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.165+0000 7f0ca5c10700 1 -- 172.21.15.16:0/1206715272 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ca00caf40 con 0x7f0ca0157190 2023-05-20T01:16:24.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.166+0000 7f0ca5c10700 1 -- 172.21.15.16:0/1206715272 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ca0157560 con 0x7f0ca0157190 2023-05-20T01:16:24.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.167+0000 7f0c97fff700 1 -- 172.21.15.16:0/1206715272 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0c98004040 con 0x7f0ca0157190 2023-05-20T01:16:24.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.167+0000 7f0c97fff700 1 --2- 172.21.15.16:0/1206715272 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f0c8006ace0 0x7f0c8006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:24.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.167+0000 7f0c97fff700 1 -- 172.21.15.16:0/1206715272 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f0c9808a5b0 con 0x7f0ca0157190 2023-05-20T01:16:24.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.167+0000 7f0c9f7fe700 1 --2- 172.21.15.16:0/1206715272 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f0c8006ace0 0x7f0c8006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:24.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.168+0000 7f0c9f7fe700 1 --2- 172.21.15.16:0/1206715272 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f0c8006ace0 0x7f0c8006d190 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0c90009b40 tx=0x7f0c90009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:24.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.172+0000 7f0c97fff700 1 -- 172.21.15.16:0/1206715272 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f0c98055a40 con 0x7f0ca0157190 2023-05-20T01:16:24.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.414+0000 7f0ca5c10700 1 -- 172.21.15.16:0/1206715272 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0ca00747b0 con 0x7f0ca0157190 2023-05-20T01:16:24.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.415+0000 7f0c97fff700 1 -- 172.21.15.16:0/1206715272 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3530 (secure 0 0 0) 0x7f0c98021de0 con 0x7f0ca0157190 2023-05-20T01:16:24.419 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:16:24.419 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4273,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/1444124146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1444124146},{"type":"v1","addr":"172.21.15.132:6834","nonce":1444124146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4293,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4092119206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4092119206},{"type":"v1","addr":"172.21.15.16:6835","nonce":4092119206}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2162680372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":2162680372},{"type":"v1","addr":"172.21.15.132:6835","nonce":2162680372}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4299,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2330551626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2330551626},{"type":"v1","addr":"172.21.15.16:6837","nonce":2330551626}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2023-05-20T01:16:24.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.417+0000 7f0ca5c10700 1 -- 172.21.15.16:0/1206715272 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f0c8006ace0 msgr2=0x7f0c8006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:24.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.417+0000 7f0ca5c10700 1 --2- 172.21.15.16:0/1206715272 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f0c8006ace0 0x7f0c8006d190 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0c90009b40 tx=0x7f0c90009210 comp rx=0 tx=0).stop 2023-05-20T01:16:24.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.417+0000 7f0ca5c10700 1 -- 172.21.15.16:0/1206715272 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ca0157190 msgr2=0x7f0ca006f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:24.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.417+0000 7f0ca5c10700 1 --2- 172.21.15.16:0/1206715272 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ca0157190 0x7f0ca006f1f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0ca00d0f50 tx=0x7f0c98004ad0 comp rx=0 tx=0).stop 2023-05-20T01:16:24.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.417+0000 7f0ca5c10700 1 -- 172.21.15.16:0/1206715272 shutdown_connections 2023-05-20T01:16:24.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.417+0000 7f0ca5c10700 1 --2- 172.21.15.16:0/1206715272 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ca0157190 0x7f0ca006f1f0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:24.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.417+0000 7f0ca5c10700 1 --2- 172.21.15.16:0/1206715272 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0ca0071ad0 0x7f0ca01f6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:24.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.417+0000 7f0ca5c10700 1 --2- 172.21.15.16:0/1206715272 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f0c8006ace0 0x7f0c8006d190 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:24.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.417+0000 7f0ca5c10700 1 --2- 172.21.15.16:0/1206715272 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0ca00cccf0 0x7f0ca01f25a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:24.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.417+0000 7f0ca5c10700 1 -- 172.21.15.16:0/1206715272 >> 172.21.15.16:0/1206715272 conn(0x7f0ca00cffb0 msgr2=0x7f0ca00713c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:24.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.417+0000 7f0ca5c10700 1 -- 172.21.15.16:0/1206715272 shutdown_connections 2023-05-20T01:16:24.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.417+0000 7f0ca5c10700 1 -- 172.21.15.16:0/1206715272 wait complete. 2023-05-20T01:16:24.422 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 3 2023-05-20T01:16:24.432 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-20T01:16:24.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.553+0000 7f1015e6a700 1 Processor -- start 2023-05-20T01:16:24.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.553+0000 7f1015e6a700 1 -- start start 2023-05-20T01:16:24.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.554+0000 7f1015e6a700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f101006f450 0x7f10100cf890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:24.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.554+0000 7f1015e6a700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f101006d4b0 0x7f1010161080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:24.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.554+0000 7f1015e6a700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10100cc510 con 0x7f101006a510 2023-05-20T01:16:24.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.554+0000 7f1015e6a700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10101615c0 con 0x7f101006f450 2023-05-20T01:16:24.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.554+0000 7f1015e6a700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1010071ed0 con 0x7f101006d4b0 2023-05-20T01:16:24.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.554+0000 7f100f7fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f101006f450 0x7f10100cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:24.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.554+0000 7f100ffff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f101006a510 0x7f10100cdb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:40674/0 (socket says 172.21.15.16:40674) 2023-05-20T01:16:24.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.554+0000 7f100ffff700 1 -- 172.21.15.16:0/1078128320 learned_addr learned my addr 172.21.15.16:0/1078128320 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:16:24.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.554+0000 7f100effd700 1 --2- 172.21.15.16:0/1078128320 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f101006d4b0 0x7f1010161080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:24.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.554+0000 7f100f7fe700 1 -- 172.21.15.16:0/1078128320 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f101006d4b0 msgr2=0x7f1010161080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:24.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.554+0000 7f100f7fe700 1 --2- 172.21.15.16:0/1078128320 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f101006d4b0 0x7f1010161080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:24.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.554+0000 7f100f7fe700 1 -- 172.21.15.16:0/1078128320 >> v1:172.21.15.16:6789/0 conn(0x7f101006a510 legacy=0x7f10100cdb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:24.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.554+0000 7f100f7fe700 1 -- 172.21.15.16:0/1078128320 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10100721b0 con 0x7f101006f450 2023-05-20T01:16:24.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.554+0000 7f100effd700 1 --2- 172.21.15.16:0/1078128320 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f101006d4b0 0x7f1010161080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:16:24.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.554+0000 7f100f7fe700 1 --2- 172.21.15.16:0/1078128320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f101006f450 0x7f10100cf890 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0ff8009840 tx=0x7f0ff8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a861b4a3abb09bab server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:24.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.555+0000 7f100e7fc700 1 -- 172.21.15.16:0/1078128320 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ff8011830 con 0x7f101006f450 2023-05-20T01:16:24.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.555+0000 7f100e7fc700 1 -- 172.21.15.16:0/1078128320 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ff8011990 con 0x7f101006f450 2023-05-20T01:16:24.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.555+0000 7f100e7fc700 1 -- 172.21.15.16:0/1078128320 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ff8011d20 con 0x7f101006f450 2023-05-20T01:16:24.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.555+0000 7f1015e6a700 1 -- 172.21.15.16:0/1078128320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f101006f450 msgr2=0x7f10100cf890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:24.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.555+0000 7f1015e6a700 1 --2- 172.21.15.16:0/1078128320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f101006f450 0x7f10100cf890 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0ff8009840 tx=0x7f0ff8009b50 comp rx=0 tx=0).stop 2023-05-20T01:16:24.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.555+0000 7f1015e6a700 1 -- 172.21.15.16:0/1078128320 shutdown_connections 2023-05-20T01:16:24.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.555+0000 7f1015e6a700 1 --2- 172.21.15.16:0/1078128320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f101006f450 0x7f10100cf890 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:24.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.555+0000 7f1015e6a700 1 --2- 172.21.15.16:0/1078128320 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f101006d4b0 0x7f1010161080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:24.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.555+0000 7f1015e6a700 1 -- 172.21.15.16:0/1078128320 >> 172.21.15.16:0/1078128320 conn(0x7f10100cffb0 msgr2=0x7f10100d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:24.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.555+0000 7f1015e6a700 1 -- 172.21.15.16:0/1078128320 shutdown_connections 2023-05-20T01:16:24.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.555+0000 7f1015e6a700 1 -- 172.21.15.16:0/1078128320 wait complete. 2023-05-20T01:16:24.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.557+0000 7f1015e6a700 1 Processor -- start 2023-05-20T01:16:24.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.557+0000 7f1015e6a700 1 -- start start 2023-05-20T01:16:24.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.558+0000 7f1015e6a700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f101006a510 0x7f1010151110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:24.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.558+0000 7f1015e6a700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f101006d4b0 0x7f1010156ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:24.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.558+0000 7f1015e6a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f101006f450 0x7f10100b5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:24.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.558+0000 7f1015e6a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1010158070 con 0x7f101006f450 2023-05-20T01:16:24.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.558+0000 7f1015e6a700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10101e66b0 con 0x7f101006d4b0 2023-05-20T01:16:24.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.558+0000 7f1015e6a700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1010151650 con 0x7f101006a510 2023-05-20T01:16:24.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.558+0000 7f100ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f101006f450 0x7f10100b5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:24.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.558+0000 7f100ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f101006f450 0x7f10100b5190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57384/0 (socket says 172.21.15.16:57384) 2023-05-20T01:16:24.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.558+0000 7f100ffff700 1 -- 172.21.15.16:0/3109711100 learned_addr learned my addr 172.21.15.16:0/3109711100 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:24.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.558+0000 7f100effd700 1 --2- 172.21.15.16:0/3109711100 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f101006d4b0 0x7f1010156ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:24.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.558+0000 7f100f7fe700 1 --2- 172.21.15.16:0/3109711100 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f101006a510 0x7f1010151110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:24.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.558+0000 7f100ffff700 1 -- 172.21.15.16:0/3109711100 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f101006a510 msgr2=0x7f1010151110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:24.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.558+0000 7f100ffff700 1 --2- 172.21.15.16:0/3109711100 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f101006a510 0x7f1010151110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:24.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.558+0000 7f100ffff700 1 -- 172.21.15.16:0/3109711100 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f101006d4b0 msgr2=0x7f1010156ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:24.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.558+0000 7f100ffff700 1 --2- 172.21.15.16:0/3109711100 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f101006d4b0 0x7f1010156ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:24.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.558+0000 7f100ffff700 1 -- 172.21.15.16:0/3109711100 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ff80094a0 con 0x7f101006f450 2023-05-20T01:16:24.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.558+0000 7f100ffff700 1 --2- 172.21.15.16:0/3109711100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f101006f450 0x7f10100b5190 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f100800cbc0 tx=0x7f100800ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:24.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.559+0000 7f1007fff700 1 -- 172.21.15.16:0/3109711100 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f100800a9e0 con 0x7f101006f450 2023-05-20T01:16:24.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.559+0000 7f1007fff700 1 -- 172.21.15.16:0/3109711100 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f100800ab40 con 0x7f101006f450 2023-05-20T01:16:24.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.559+0000 7f1015e6a700 1 -- 172.21.15.16:0/3109711100 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10100cbe50 con 0x7f101006f450 2023-05-20T01:16:24.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.559+0000 7f1007fff700 1 -- 172.21.15.16:0/3109711100 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1008012400 con 0x7f101006f450 2023-05-20T01:16:24.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.559+0000 7f1015e6a700 1 -- 172.21.15.16:0/3109711100 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10100cc1a0 con 0x7f101006f450 2023-05-20T01:16:24.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.560+0000 7f1015e6a700 1 -- 172.21.15.16:0/3109711100 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f101006a9b0 con 0x7f101006f450 2023-05-20T01:16:24.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.560+0000 7f1007fff700 1 -- 172.21.15.16:0/3109711100 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f100801b3b0 con 0x7f101006f450 2023-05-20T01:16:24.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.560+0000 7f1007fff700 1 --2- 172.21.15.16:0/3109711100 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f0ff006ace0 0x7f0ff006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:24.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.560+0000 7f1007fff700 1 -- 172.21.15.16:0/3109711100 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f1008088c70 con 0x7f101006f450 2023-05-20T01:16:24.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.560+0000 7f100f7fe700 1 --2- 172.21.15.16:0/3109711100 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f0ff006ace0 0x7f0ff006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:24.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.561+0000 7f100f7fe700 1 --2- 172.21.15.16:0/3109711100 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f0ff006ace0 0x7f0ff006d190 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0ff80090b0 tx=0x7f0ff801c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:24.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.565+0000 7f1007fff700 1 -- 172.21.15.16:0/3109711100 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f1008054130 con 0x7f101006f450 2023-05-20T01:16:24.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.764+0000 7f1015e6a700 1 -- 172.21.15.16:0/3109711100 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f10101e6410 con 0x7f101006f450 2023-05-20T01:16:24.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.765+0000 7f1007fff700 1 -- 172.21.15.16:0/3109711100 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+14543 (secure 0 0 0) 0x7f10101e6410 con 0x7f101006f450 2023-05-20T01:16:24.768 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:16:24.769 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":17,"fsid":"c2245cc5-ecab-4087-a59b-bc81db901676","created":"2023-05-20T01:16:09.549230+0000","modified":"2023-05-20T01:16:23.820327+0000","last_up_change":"2023-05-20T01:16:19.788624+0000","last_in_change":"2023-05-20T01:16:12.092160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-05-20T01:16:20.934366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-05-20T01:16:21.372294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"317a83cd-19c8-4958-a301-6087ef342289","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6825","nonce":3923634179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6827","nonce":3923634179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6831","nonce":3923634179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6829","nonce":3923634179}]},"public_addr":"172.21.15.16:6825/3923634179","cluster_addr":"172.21.15.16:6827/3923634179","heartbeat_back_addr":"172.21.15.16:6831/3923634179","heartbeat_front_addr":"172.21.15.16:6829/3923634179","state":["exists","up"]},{"osd":1,"uuid":"5581046f-8219-4089-8d7d-b7846399505e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6809","nonce":3860264220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6811","nonce":3860264220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6815","nonce":3860264220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6813","nonce":3860264220}]},"public_addr":"172.21.15.16:6809/3860264220","cluster_addr":"172.21.15.16:6811/3860264220","heartbeat_back_addr":"172.21.15.16:6815/3860264220","heartbeat_front_addr":"172.21.15.16:6813/3860264220","state":["exists","up"]},{"osd":2,"uuid":"072e9e66-4133-478e-8d4a-64d46c2963f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6817","nonce":293187030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6819","nonce":293187030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6823","nonce":293187030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6821","nonce":293187030}]},"public_addr":"172.21.15.16:6817/293187030","cluster_addr":"172.21.15.16:6819/293187030","heartbeat_back_addr":"172.21.15.16:6823/293187030","heartbeat_front_addr":"172.21.15.16:6821/293187030","state":["exists","up"]},{"osd":3,"uuid":"11e1bba6-00f8-48dd-9028-d8d716b7f145","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6801","nonce":3669781890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6803","nonce":3669781890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6807","nonce":3669781890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6805","nonce":3669781890}]},"public_addr":"172.21.15.16:6801/3669781890","cluster_addr":"172.21.15.16:6803/3669781890","heartbeat_back_addr":"172.21.15.16:6807/3669781890","heartbeat_front_addr":"172.21.15.16:6805/3669781890","state":["exists","up"]},{"osd":4,"uuid":"b8909c3a-cf70-46cb-8d0b-009c707cf9d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6819","nonce":506317223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6823","nonce":506317223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6829","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6831","nonce":506317223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6827","nonce":506317223}]},"public_addr":"172.21.15.132:6819/506317223","cluster_addr":"172.21.15.132:6823/506317223","heartbeat_back_addr":"172.21.15.132:6831/506317223","heartbeat_front_addr":"172.21.15.132:6827/506317223","state":["exists","up"]},{"osd":5,"uuid":"b1818304-4583-47bf-a6ec-0bfbb5166233","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6818","nonce":1844689995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6822","nonce":1844689995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6830","nonce":1844689995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6826","nonce":1844689995}]},"public_addr":"172.21.15.132:6818/1844689995","cluster_addr":"172.21.15.132:6822/1844689995","heartbeat_back_addr":"172.21.15.132:6830/1844689995","heartbeat_front_addr":"172.21.15.132:6826/1844689995","state":["exists","up"]},{"osd":6,"uuid":"d37f4691-eed1-4a93-8818-18354f03f4a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6801","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6802","nonce":727581123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6805","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6807","nonce":727581123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6813","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6815","nonce":727581123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6811","nonce":727581123}]},"public_addr":"172.21.15.132:6802/727581123","cluster_addr":"172.21.15.132:6807/727581123","heartbeat_back_addr":"172.21.15.132:6815/727581123","heartbeat_front_addr":"172.21.15.132:6811/727581123","state":["exists","up"]},{"osd":7,"uuid":"4f712f62-5d1d-4816-8042-03209144f8b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6803","nonce":466001149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6806","nonce":466001149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6814","nonce":466001149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6810","nonce":466001149}]},"public_addr":"172.21.15.132:6803/466001149","cluster_addr":"172.21.15.132:6806/466001149","heartbeat_back_addr":"172.21.15.132:6814/466001149","heartbeat_front_addr":"172.21.15.132:6810/466001149","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.338863+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.278680+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305694+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.246835+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.276237+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.304441+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305421+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.267050+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-20T01:16:24.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.767+0000 7f1015e6a700 1 -- 172.21.15.16:0/3109711100 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f0ff006ace0 msgr2=0x7f0ff006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:24.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.767+0000 7f1015e6a700 1 --2- 172.21.15.16:0/3109711100 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f0ff006ace0 0x7f0ff006d190 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0ff80090b0 tx=0x7f0ff801c040 comp rx=0 tx=0).stop 2023-05-20T01:16:24.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.767+0000 7f1015e6a700 1 -- 172.21.15.16:0/3109711100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f101006f450 msgr2=0x7f10100b5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:24.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.767+0000 7f1015e6a700 1 --2- 172.21.15.16:0/3109711100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f101006f450 0x7f10100b5190 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f100800cbc0 tx=0x7f100800ced0 comp rx=0 tx=0).stop 2023-05-20T01:16:24.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.767+0000 7f1015e6a700 1 -- 172.21.15.16:0/3109711100 shutdown_connections 2023-05-20T01:16:24.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.767+0000 7f1015e6a700 1 --2- 172.21.15.16:0/3109711100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f101006f450 0x7f10100b5190 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:24.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.767+0000 7f1015e6a700 1 --2- 172.21.15.16:0/3109711100 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f0ff006ace0 0x7f0ff006d190 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:24.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.767+0000 7f1015e6a700 1 --2- 172.21.15.16:0/3109711100 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f101006a510 0x7f1010151110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:24.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.767+0000 7f1015e6a700 1 --2- 172.21.15.16:0/3109711100 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f101006d4b0 0x7f1010156ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:24.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.767+0000 7f1015e6a700 1 -- 172.21.15.16:0/3109711100 >> 172.21.15.16:0/3109711100 conn(0x7f10100cffb0 msgr2=0x7f1010072fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:24.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.767+0000 7f1015e6a700 1 -- 172.21.15.16:0/3109711100 shutdown_connections 2023-05-20T01:16:24.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.767+0000 7f1015e6a700 1 -- 172.21.15.16:0/3109711100 wait complete. 2023-05-20T01:16:24.783 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-05-20T01:16:20.934366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-05-20T01:16:21.372294+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-05-20T01:16:24.784 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-05-20T01:16:24.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.909+0000 7f7117a7c700 1 Processor -- start 2023-05-20T01:16:24.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.909+0000 7f7117a7c700 1 -- start start 2023-05-20T01:16:24.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.910+0000 7f7117a7c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f711015cec0 0x7f7110071bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:24.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.910+0000 7f7117a7c700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f7110072130 0x7f71100cf9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:24.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.910+0000 7f7117a7c700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71100cd630 con 0x7f711015d290 2023-05-20T01:16:24.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.910+0000 7f7117a7c700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7110072530 con 0x7f711015cec0 2023-05-20T01:16:24.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.910+0000 7f7117a7c700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71100cca60 con 0x7f7110072130 2023-05-20T01:16:24.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.910+0000 7f7115818700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f711015cec0 0x7f7110071bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:24.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.910+0000 7f7115818700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f711015cec0 0x7f7110071bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57388/0 (socket says 172.21.15.16:57388) 2023-05-20T01:16:24.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.910+0000 7f7115818700 1 -- 172.21.15.16:0/4096906280 learned_addr learned my addr 172.21.15.16:0/4096906280 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:24.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.910+0000 7f7116019700 1 --2- 172.21.15.16:0/4096906280 >> v2:172.21.15.132:3300/0 conn(0x7f7110072130 0x7f71100cf9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:24.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.910+0000 7f7115818700 1 -- 172.21.15.16:0/4096906280 >> v2:172.21.15.132:3300/0 conn(0x7f7110072130 msgr2=0x7f71100cf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:24.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.910+0000 7f7115818700 1 --2- 172.21.15.16:0/4096906280 >> v2:172.21.15.132:3300/0 conn(0x7f7110072130 0x7f71100cf9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:24.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.910+0000 7f7115818700 1 -- 172.21.15.16:0/4096906280 >> v1:172.21.15.132:6789/0 conn(0x7f711015d290 legacy=0x7f71100cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:24.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.910+0000 7f7115818700 1 -- 172.21.15.16:0/4096906280 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71100ccd00 con 0x7f711015cec0 2023-05-20T01:16:24.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.910+0000 7f7115818700 1 --2- 172.21.15.16:0/4096906280 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f711015cec0 0x7f7110071bf0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f71100b6120 tx=0x7f7100009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f77ed1b4effc0d53 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:24.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.910+0000 7f7114816700 1 -- 172.21.15.16:0/4096906280 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7100005e30 con 0x7f711015cec0 2023-05-20T01:16:24.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.910+0000 7f7114816700 1 -- 172.21.15.16:0/4096906280 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7100004030 con 0x7f711015cec0 2023-05-20T01:16:24.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.911+0000 7f7114816700 1 -- 172.21.15.16:0/4096906280 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71000043e0 con 0x7f711015cec0 2023-05-20T01:16:24.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.911+0000 7f7117a7c700 1 -- 172.21.15.16:0/4096906280 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f711015cec0 msgr2=0x7f7110071bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:24.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.911+0000 7f7117a7c700 1 --2- 172.21.15.16:0/4096906280 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f711015cec0 0x7f7110071bf0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f71100b6120 tx=0x7f7100009bd0 comp rx=0 tx=0).stop 2023-05-20T01:16:24.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.911+0000 7f7117a7c700 1 -- 172.21.15.16:0/4096906280 shutdown_connections 2023-05-20T01:16:24.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.911+0000 7f7117a7c700 1 --2- 172.21.15.16:0/4096906280 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f711015cec0 0x7f7110071bf0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:24.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.911+0000 7f7117a7c700 1 --2- 172.21.15.16:0/4096906280 >> v2:172.21.15.132:3300/0 conn(0x7f7110072130 0x7f71100cf9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:24.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.911+0000 7f7117a7c700 1 -- 172.21.15.16:0/4096906280 >> 172.21.15.16:0/4096906280 conn(0x7f71100cffd0 msgr2=0x7f71100d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:24.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.911+0000 7f7117a7c700 1 -- 172.21.15.16:0/4096906280 shutdown_connections 2023-05-20T01:16:24.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.911+0000 7f7117a7c700 1 -- 172.21.15.16:0/4096906280 wait complete. 2023-05-20T01:16:24.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.913+0000 7f7117a7c700 1 Processor -- start 2023-05-20T01:16:24.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.913+0000 7f7117a7c700 1 -- start start 2023-05-20T01:16:24.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.914+0000 7f7117a7c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7110072130 0x7f7110156560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:24.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.914+0000 7f7117a7c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f711015cec0 0x7f7110156d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:24.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.914+0000 7f7117a7c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f711015d290 0x7f71100c9ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:24.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.914+0000 7f7117a7c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71101572c0 con 0x7f711015cec0 2023-05-20T01:16:24.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.914+0000 7f7117a7c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71100c9ff0 con 0x7f711015d290 2023-05-20T01:16:24.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.914+0000 7f7117a7c700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71101574b0 con 0x7f7110072130 2023-05-20T01:16:24.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.914+0000 7f7115017700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f711015cec0 0x7f7110156d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:24.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.914+0000 7f7115017700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f711015cec0 0x7f7110156d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57396/0 (socket says 172.21.15.16:57396) 2023-05-20T01:16:24.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.914+0000 7f7115017700 1 -- 172.21.15.16:0/838799536 learned_addr learned my addr 172.21.15.16:0/838799536 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:24.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.914+0000 7f7116019700 1 --2- 172.21.15.16:0/838799536 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f711015d290 0x7f71100c9ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:24.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.914+0000 7f7115818700 1 --2- 172.21.15.16:0/838799536 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7110072130 0x7f7110156560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:24.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.914+0000 7f7115017700 1 -- 172.21.15.16:0/838799536 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7110072130 msgr2=0x7f7110156560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:24.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.914+0000 7f7115017700 1 --2- 172.21.15.16:0/838799536 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7110072130 0x7f7110156560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:24.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.914+0000 7f7115017700 1 -- 172.21.15.16:0/838799536 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f711015d290 msgr2=0x7f71100c9ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:24.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.914+0000 7f7115017700 1 --2- 172.21.15.16:0/838799536 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f711015d290 0x7f71100c9ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:24.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.914+0000 7f7115017700 1 -- 172.21.15.16:0/838799536 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7100009500 con 0x7f711015cec0 2023-05-20T01:16:24.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.914+0000 7f7116019700 1 --2- 172.21.15.16:0/838799536 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f711015d290 0x7f71100c9ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:16:24.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.914+0000 7f7115818700 1 --2- 172.21.15.16:0/838799536 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7110072130 0x7f7110156560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:16:24.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.914+0000 7f7115017700 1 --2- 172.21.15.16:0/838799536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f711015cec0 0x7f7110156d80 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f710c00c450 tx=0x7f710c00c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:24.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.914+0000 7f71067fc700 1 -- 172.21.15.16:0/838799536 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f710c0052d0 con 0x7f711015cec0 2023-05-20T01:16:24.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.914+0000 7f71067fc700 1 -- 172.21.15.16:0/838799536 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f710c005430 con 0x7f711015cec0 2023-05-20T01:16:24.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.914+0000 7f7117a7c700 1 -- 172.21.15.16:0/838799536 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71100cd2c0 con 0x7f711015cec0 2023-05-20T01:16:24.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.914+0000 7f71067fc700 1 -- 172.21.15.16:0/838799536 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f710c0038c0 con 0x7f711015cec0 2023-05-20T01:16:24.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.915+0000 7f7117a7c700 1 -- 172.21.15.16:0/838799536 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71100ccfe0 con 0x7f711015cec0 2023-05-20T01:16:24.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.916+0000 7f7117a7c700 1 -- 172.21.15.16:0/838799536 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7110160870 con 0x7f711015cec0 2023-05-20T01:16:24.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.916+0000 7f71067fc700 1 -- 172.21.15.16:0/838799536 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f710c003a20 con 0x7f711015cec0 2023-05-20T01:16:24.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.916+0000 7f71067fc700 1 --2- 172.21.15.16:0/838799536 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f70f406ace0 0x7f70f406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:24.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.916+0000 7f71067fc700 1 -- 172.21.15.16:0/838799536 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f710c088440 con 0x7f711015cec0 2023-05-20T01:16:24.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.917+0000 7f7115818700 1 --2- 172.21.15.16:0/838799536 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f70f406ace0 0x7f70f406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:24.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.917+0000 7f7115818700 1 --2- 172.21.15.16:0/838799536 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f70f406ace0 0x7f70f406d190 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7100009c10 tx=0x7f7100008070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:24.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:24.921+0000 7f71067fc700 1 -- 172.21.15.16:0/838799536 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f710c0538d0 con 0x7f711015cec0 2023-05-20T01:16:25.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.121+0000 7f7117a7c700 1 -- 172.21.15.16:0/838799536 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f71100747b0 con 0x7f711015cec0 2023-05-20T01:16:25.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.121+0000 7f71067fc700 1 -- 172.21.15.16:0/838799536 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v17) v1 ==== 92+0+10 (secure 0 0 0) 0x7f710c056ef0 con 0x7f711015cec0 2023-05-20T01:16:25.125 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 8 2023-05-20T01:16:25.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.123+0000 7f7117a7c700 1 -- 172.21.15.16:0/838799536 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f70f406ace0 msgr2=0x7f70f406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:25.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.123+0000 7f7117a7c700 1 --2- 172.21.15.16:0/838799536 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f70f406ace0 0x7f70f406d190 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7100009c10 tx=0x7f7100008070 comp rx=0 tx=0).stop 2023-05-20T01:16:25.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.123+0000 7f7117a7c700 1 -- 172.21.15.16:0/838799536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f711015cec0 msgr2=0x7f7110156d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:25.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.123+0000 7f7117a7c700 1 --2- 172.21.15.16:0/838799536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f711015cec0 0x7f7110156d80 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f710c00c450 tx=0x7f710c00c810 comp rx=0 tx=0).stop 2023-05-20T01:16:25.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.124+0000 7f7117a7c700 1 -- 172.21.15.16:0/838799536 shutdown_connections 2023-05-20T01:16:25.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.124+0000 7f7117a7c700 1 --2- 172.21.15.16:0/838799536 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f711015d290 0x7f71100c9ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:25.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.124+0000 7f7117a7c700 1 --2- 172.21.15.16:0/838799536 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f70f406ace0 0x7f70f406d190 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:25.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.124+0000 7f7117a7c700 1 --2- 172.21.15.16:0/838799536 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7110072130 0x7f7110156560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:25.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.124+0000 7f7117a7c700 1 --2- 172.21.15.16:0/838799536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f711015cec0 0x7f7110156d80 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:25.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.124+0000 7f7117a7c700 1 -- 172.21.15.16:0/838799536 >> 172.21.15.16:0/838799536 conn(0x7f71100cffd0 msgr2=0x7f711015e3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:25.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.124+0000 7f7117a7c700 1 -- 172.21.15.16:0/838799536 shutdown_connections 2023-05-20T01:16:25.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.124+0000 7f7117a7c700 1 -- 172.21.15.16:0/838799536 wait complete. 2023-05-20T01:16:25.139 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-05-20T01:16:25.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.263+0000 7ff52778b700 1 Processor -- start 2023-05-20T01:16:25.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.263+0000 7ff52778b700 1 -- start start 2023-05-20T01:16:25.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.264+0000 7ff52778b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff520159b50 0x7ff52006e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:25.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.264+0000 7ff52778b700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7ff5200717c0 0x7ff5200ca870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:25.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.264+0000 7ff52778b700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5200ccd30 con 0x7ff5200cd1c0 2023-05-20T01:16:25.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.264+0000 7ff52778b700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5200ce990 con 0x7ff520159b50 2023-05-20T01:16:25.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.264+0000 7ff52778b700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5200cc6e0 con 0x7ff5200717c0 2023-05-20T01:16:25.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.264+0000 7ff525527700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff520159b50 0x7ff52006e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:25.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.264+0000 7ff524d26700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ff5200cd1c0 0x7ff5200710b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:40698/0 (socket says 172.21.15.16:40698) 2023-05-20T01:16:25.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.264+0000 7ff524d26700 1 -- 172.21.15.16:0/551367540 learned_addr learned my addr 172.21.15.16:0/551367540 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:16:25.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.264+0000 7ff525527700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff520159b50 0x7ff52006e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57402/0 (socket says 172.21.15.16:57402) 2023-05-20T01:16:25.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.264+0000 7ff525d28700 1 --2- 172.21.15.16:0/551367540 >> v2:172.21.15.132:3300/0 conn(0x7ff5200717c0 0x7ff5200ca870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:25.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.264+0000 7ff525527700 1 -- 172.21.15.16:0/551367540 >> v2:172.21.15.132:3300/0 conn(0x7ff5200717c0 msgr2=0x7ff5200ca870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:25.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.264+0000 7ff525527700 1 --2- 172.21.15.16:0/551367540 >> v2:172.21.15.132:3300/0 conn(0x7ff5200717c0 0x7ff5200ca870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:25.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.264+0000 7ff525527700 1 -- 172.21.15.16:0/551367540 >> v1:172.21.15.16:6789/0 conn(0x7ff5200cd1c0 legacy=0x7ff5200710b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:25.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.264+0000 7ff525527700 1 -- 172.21.15.16:0/551367540 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5200cc9c0 con 0x7ff520159b50 2023-05-20T01:16:25.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.264+0000 7ff517fff700 1 -- 172.21.15.16:0/551367540 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 818203290 0 0) 0x7ff5200ccd30 con 0x7ff5200cd1c0 2023-05-20T01:16:25.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.264+0000 7ff525d28700 1 --2- 172.21.15.16:0/551367540 >> v2:172.21.15.132:3300/0 conn(0x7ff5200717c0 0x7ff5200ca870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:16:25.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.264+0000 7ff525527700 1 --2- 172.21.15.16:0/551367540 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff520159b50 0x7ff52006e0a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff520072750 tx=0x7ff51c00bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7764a6450465c0d server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:25.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.265+0000 7ff517fff700 1 -- 172.21.15.16:0/551367540 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff51c005e30 con 0x7ff520159b50 2023-05-20T01:16:25.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.265+0000 7ff517fff700 1 -- 172.21.15.16:0/551367540 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff51c00d860 con 0x7ff520159b50 2023-05-20T01:16:25.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.265+0000 7ff517fff700 1 -- 172.21.15.16:0/551367540 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff51c00dc10 con 0x7ff520159b50 2023-05-20T01:16:25.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.265+0000 7ff52778b700 1 -- 172.21.15.16:0/551367540 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff520159b50 msgr2=0x7ff52006e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:25.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.265+0000 7ff52778b700 1 --2- 172.21.15.16:0/551367540 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff520159b50 0x7ff52006e0a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff520072750 tx=0x7ff51c00bda0 comp rx=0 tx=0).stop 2023-05-20T01:16:25.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.265+0000 7ff52778b700 1 -- 172.21.15.16:0/551367540 shutdown_connections 2023-05-20T01:16:25.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.265+0000 7ff52778b700 1 --2- 172.21.15.16:0/551367540 >> v2:172.21.15.132:3300/0 conn(0x7ff5200717c0 0x7ff5200ca870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:25.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.265+0000 7ff52778b700 1 --2- 172.21.15.16:0/551367540 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff520159b50 0x7ff52006e0a0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:25.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.265+0000 7ff52778b700 1 -- 172.21.15.16:0/551367540 >> 172.21.15.16:0/551367540 conn(0x7ff5200cffd0 msgr2=0x7ff5200d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:25.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.265+0000 7ff52778b700 1 -- 172.21.15.16:0/551367540 shutdown_connections 2023-05-20T01:16:25.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.265+0000 7ff52778b700 1 -- 172.21.15.16:0/551367540 wait complete. 2023-05-20T01:16:25.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.266+0000 7ff52778b700 1 Processor -- start 2023-05-20T01:16:25.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.266+0000 7ff52778b700 1 -- start start 2023-05-20T01:16:25.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.267+0000 7ff52778b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff5200717c0 0x7ff5200c9600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:25.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.267+0000 7ff52778b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5200cd1c0 0x7ff520148360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:25.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.267+0000 7ff52778b700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff520159b50 0x7ff5200b4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:25.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.267+0000 7ff52778b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5200c9170 con 0x7ff5200cd1c0 2023-05-20T01:16:25.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.267+0000 7ff52778b700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5200c82c0 con 0x7ff5200717c0 2023-05-20T01:16:25.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.267+0000 7ff52778b700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5200c8560 con 0x7ff520159b50 2023-05-20T01:16:25.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.267+0000 7ff524d26700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5200cd1c0 0x7ff520148360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:25.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.267+0000 7ff524d26700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5200cd1c0 0x7ff520148360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57410/0 (socket says 172.21.15.16:57410) 2023-05-20T01:16:25.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.267+0000 7ff524d26700 1 -- 172.21.15.16:0/1952352498 learned_addr learned my addr 172.21.15.16:0/1952352498 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:25.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.267+0000 7ff525527700 1 --2- 172.21.15.16:0/1952352498 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff5200717c0 0x7ff5200c9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:25.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.267+0000 7ff525d28700 1 --2- 172.21.15.16:0/1952352498 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff520159b50 0x7ff5200b4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:25.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.268+0000 7ff524d26700 1 -- 172.21.15.16:0/1952352498 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff520159b50 msgr2=0x7ff5200b4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:25.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.268+0000 7ff524d26700 1 --2- 172.21.15.16:0/1952352498 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff520159b50 0x7ff5200b4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:25.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.268+0000 7ff524d26700 1 -- 172.21.15.16:0/1952352498 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff5200717c0 msgr2=0x7ff5200c9600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:25.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.268+0000 7ff524d26700 1 --2- 172.21.15.16:0/1952352498 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff5200717c0 0x7ff5200c9600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:25.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.268+0000 7ff524d26700 1 -- 172.21.15.16:0/1952352498 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff51c00b6d0 con 0x7ff5200cd1c0 2023-05-20T01:16:25.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.268+0000 7ff524d26700 1 --2- 172.21.15.16:0/1952352498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5200cd1c0 0x7ff520148360 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff5100011e0 tx=0x7ff51000ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:25.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.268+0000 7ff515ffb700 1 -- 172.21.15.16:0/1952352498 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff510004c80 con 0x7ff5200cd1c0 2023-05-20T01:16:25.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.268+0000 7ff515ffb700 1 -- 172.21.15.16:0/1952352498 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff510004e00 con 0x7ff5200cd1c0 2023-05-20T01:16:25.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.268+0000 7ff52778b700 1 -- 172.21.15.16:0/1952352498 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff520071e80 con 0x7ff5200cd1c0 2023-05-20T01:16:25.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.268+0000 7ff515ffb700 1 -- 172.21.15.16:0/1952352498 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff510007310 con 0x7ff5200cd1c0 2023-05-20T01:16:25.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.268+0000 7ff52778b700 1 -- 172.21.15.16:0/1952352498 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff52006e5e0 con 0x7ff5200cd1c0 2023-05-20T01:16:25.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.269+0000 7ff52778b700 1 -- 172.21.15.16:0/1952352498 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5200cec70 con 0x7ff5200cd1c0 2023-05-20T01:16:25.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.269+0000 7ff515ffb700 1 -- 172.21.15.16:0/1952352498 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff510007470 con 0x7ff5200cd1c0 2023-05-20T01:16:25.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.270+0000 7ff515ffb700 1 --2- 172.21.15.16:0/1952352498 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff50406ace0 0x7ff50406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:25.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.270+0000 7ff525527700 1 --2- 172.21.15.16:0/1952352498 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff50406ace0 0x7ff50406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:25.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.270+0000 7ff515ffb700 1 -- 172.21.15.16:0/1952352498 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5210+0+0 (secure 0 0 0) 0x7ff510088d80 con 0x7ff5200cd1c0 2023-05-20T01:16:25.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.270+0000 7ff525527700 1 --2- 172.21.15.16:0/1952352498 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff50406ace0 0x7ff50406d190 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff51c00bde0 tx=0x7ff51c00a000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:25.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.274+0000 7ff515ffb700 1 -- 172.21.15.16:0/1952352498 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7ff510054180 con 0x7ff5200cd1c0 2023-05-20T01:16:25.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.472+0000 7ff52778b700 1 -- 172.21.15.16:0/1952352498 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff5200747b0 con 0x7ff5200cd1c0 2023-05-20T01:16:25.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.472+0000 7ff515ffb700 1 -- 172.21.15.16:0/1952352498 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff5100577a0 con 0x7ff5200cd1c0 2023-05-20T01:16:25.477 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-05-20T01:16:25.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.474+0000 7ff52778b700 1 -- 172.21.15.16:0/1952352498 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff50406ace0 msgr2=0x7ff50406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:25.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.474+0000 7ff52778b700 1 --2- 172.21.15.16:0/1952352498 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff50406ace0 0x7ff50406d190 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff51c00bde0 tx=0x7ff51c00a000 comp rx=0 tx=0).stop 2023-05-20T01:16:25.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.474+0000 7ff52778b700 1 -- 172.21.15.16:0/1952352498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5200cd1c0 msgr2=0x7ff520148360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:25.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.474+0000 7ff52778b700 1 --2- 172.21.15.16:0/1952352498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5200cd1c0 0x7ff520148360 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff5100011e0 tx=0x7ff51000ca30 comp rx=0 tx=0).stop 2023-05-20T01:16:25.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.474+0000 7ff52778b700 1 -- 172.21.15.16:0/1952352498 shutdown_connections 2023-05-20T01:16:25.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.474+0000 7ff52778b700 1 --2- 172.21.15.16:0/1952352498 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff50406ace0 0x7ff50406d190 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:25.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.474+0000 7ff52778b700 1 --2- 172.21.15.16:0/1952352498 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff520159b50 0x7ff5200b4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:25.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.474+0000 7ff52778b700 1 --2- 172.21.15.16:0/1952352498 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff5200717c0 0x7ff5200c9600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:25.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.474+0000 7ff52778b700 1 --2- 172.21.15.16:0/1952352498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5200cd1c0 0x7ff520148360 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:25.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.475+0000 7ff52778b700 1 -- 172.21.15.16:0/1952352498 >> 172.21.15.16:0/1952352498 conn(0x7ff5200cffd0 msgr2=0x7ff5200710b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:25.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.475+0000 7ff52778b700 1 -- 172.21.15.16:0/1952352498 shutdown_connections 2023-05-20T01:16:25.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.475+0000 7ff52778b700 1 -- 172.21.15.16:0/1952352498 wait complete. 2023-05-20T01:16:25.490 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-05-20T01:16:25.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.610+0000 7fa116fee700 1 Processor -- start 2023-05-20T01:16:25.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.611+0000 7fa116fee700 1 -- start start 2023-05-20T01:16:25.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.611+0000 7fa116fee700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa11006cb90 0x7fa11015ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:25.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.611+0000 7fa116fee700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa1100b54f0 0x7fa1100b29e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:25.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.611+0000 7fa116fee700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fa11006cf60 0x7fa110070910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:25.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.611+0000 7fa116fee700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1100cc990 con 0x7fa1100b54f0 2023-05-20T01:16:25.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.611+0000 7fa116fee700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1100b2f20 con 0x7fa11006cf60 2023-05-20T01:16:25.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.611+0000 7fa116fee700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1100b31a0 con 0x7fa11006cb90 2023-05-20T01:16:25.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.611+0000 7fa10ffff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa1100b54f0 0x7fa1100b29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:25.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.612+0000 7fa10ffff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa1100b54f0 0x7fa1100b29e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57418/0 (socket says 172.21.15.16:57418) 2023-05-20T01:16:25.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.612+0000 7fa10ffff700 1 -- 172.21.15.16:0/3117407751 learned_addr learned my addr 172.21.15.16:0/3117407751 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:25.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.612+0000 7fa114d8a700 1 --2- 172.21.15.16:0/3117407751 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa11006cb90 0x7fa11015ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:25.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.612+0000 7fa11558b700 1 --2- 172.21.15.16:0/3117407751 >> v2:172.21.15.132:3300/0 conn(0x7fa11006cf60 0x7fa110070910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:25.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.612+0000 7fa10ffff700 1 -- 172.21.15.16:0/3117407751 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa11006cb90 msgr2=0x7fa11015ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:25.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.612+0000 7fa10ffff700 1 --2- 172.21.15.16:0/3117407751 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa11006cb90 0x7fa11015ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:25.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.612+0000 7fa10ffff700 1 -- 172.21.15.16:0/3117407751 >> v2:172.21.15.132:3300/0 conn(0x7fa11006cf60 msgr2=0x7fa110070910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:25.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.612+0000 7fa10ffff700 1 --2- 172.21.15.16:0/3117407751 >> v2:172.21.15.132:3300/0 conn(0x7fa11006cf60 0x7fa110070910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:25.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.612+0000 7fa10ffff700 1 -- 172.21.15.16:0/3117407751 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1100b3480 con 0x7fa1100b54f0 2023-05-20T01:16:25.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.612+0000 7fa114d8a700 1 --2- 172.21.15.16:0/3117407751 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa11006cb90 0x7fa11015ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:16:25.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.612+0000 7fa11558b700 1 --2- 172.21.15.16:0/3117407751 >> v2:172.21.15.132:3300/0 conn(0x7fa11006cf60 0x7fa110070910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:16:25.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.612+0000 7fa10ffff700 1 --2- 172.21.15.16:0/3117407751 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1100b54f0 0x7fa1100b29e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa110158a20 tx=0x7fa0fc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e139e528bb03b05e server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:25.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.612+0000 7fa10f7fe700 1 -- 172.21.15.16:0/3117407751 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0fc00b870 con 0x7fa1100b54f0 2023-05-20T01:16:25.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.612+0000 7fa10f7fe700 1 -- 172.21.15.16:0/3117407751 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0fc00b9d0 con 0x7fa1100b54f0 2023-05-20T01:16:25.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.612+0000 7fa10f7fe700 1 -- 172.21.15.16:0/3117407751 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0fc00bd60 con 0x7fa1100b54f0 2023-05-20T01:16:25.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.613+0000 7fa116fee700 1 -- 172.21.15.16:0/3117407751 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1100b54f0 msgr2=0x7fa1100b29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:25.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.613+0000 7fa116fee700 1 --2- 172.21.15.16:0/3117407751 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1100b54f0 0x7fa1100b29e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa110158a20 tx=0x7fa0fc009bd0 comp rx=0 tx=0).stop 2023-05-20T01:16:25.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.613+0000 7fa116fee700 1 -- 172.21.15.16:0/3117407751 shutdown_connections 2023-05-20T01:16:25.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.613+0000 7fa116fee700 1 --2- 172.21.15.16:0/3117407751 >> v2:172.21.15.132:3300/0 conn(0x7fa11006cf60 0x7fa110070910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:25.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.613+0000 7fa116fee700 1 --2- 172.21.15.16:0/3117407751 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa11006cb90 0x7fa11015ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:25.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.613+0000 7fa116fee700 1 --2- 172.21.15.16:0/3117407751 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1100b54f0 0x7fa1100b29e0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:25.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.613+0000 7fa116fee700 1 -- 172.21.15.16:0/3117407751 >> 172.21.15.16:0/3117407751 conn(0x7fa1100661b0 msgr2=0x7fa110066580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:25.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.613+0000 7fa116fee700 1 -- 172.21.15.16:0/3117407751 shutdown_connections 2023-05-20T01:16:25.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.613+0000 7fa116fee700 1 -- 172.21.15.16:0/3117407751 wait complete. 2023-05-20T01:16:25.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.614+0000 7fa116fee700 1 Processor -- start 2023-05-20T01:16:25.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.614+0000 7fa116fee700 1 -- start start 2023-05-20T01:16:25.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.615+0000 7fa116fee700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa11006cb90 0x7fa1100b0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:25.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.615+0000 7fa116fee700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa11006cf60 0x7fa1100a8480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:25.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.615+0000 7fa116fee700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa1100b54f0 0x7fa1100a89c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:25.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.615+0000 7fa116fee700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa110147150 con 0x7fa11006cf60 2023-05-20T01:16:25.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.615+0000 7fa116fee700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1101473a0 con 0x7fa1100b54f0 2023-05-20T01:16:25.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.615+0000 7fa116fee700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa110147680 con 0x7fa11006cb90 2023-05-20T01:16:25.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.615+0000 7fa10ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa11006cf60 0x7fa1100a8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:25.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.615+0000 7fa10ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa11006cf60 0x7fa1100a8480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57424/0 (socket says 172.21.15.16:57424) 2023-05-20T01:16:25.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.615+0000 7fa10ffff700 1 -- 172.21.15.16:0/2997983701 learned_addr learned my addr 172.21.15.16:0/2997983701 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:25.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.615+0000 7fa11558b700 1 --2- 172.21.15.16:0/2997983701 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa1100b54f0 0x7fa1100a89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:25.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.615+0000 7fa114d8a700 1 --2- 172.21.15.16:0/2997983701 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa11006cb90 0x7fa1100b0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:25.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.615+0000 7fa10ffff700 1 -- 172.21.15.16:0/2997983701 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa11006cb90 msgr2=0x7fa1100b0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:25.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.615+0000 7fa10ffff700 1 --2- 172.21.15.16:0/2997983701 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa11006cb90 0x7fa1100b0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:25.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.615+0000 7fa10ffff700 1 -- 172.21.15.16:0/2997983701 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa1100b54f0 msgr2=0x7fa1100a89c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:25.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.615+0000 7fa10ffff700 1 --2- 172.21.15.16:0/2997983701 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa1100b54f0 0x7fa1100a89c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:25.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.615+0000 7fa10ffff700 1 -- 172.21.15.16:0/2997983701 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0fc009550 con 0x7fa11006cf60 2023-05-20T01:16:25.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.615+0000 7fa11558b700 1 --2- 172.21.15.16:0/2997983701 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa1100b54f0 0x7fa1100a89c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:16:25.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.615+0000 7fa114d8a700 1 --2- 172.21.15.16:0/2997983701 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa11006cb90 0x7fa1100b0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:16:25.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.615+0000 7fa10ffff700 1 --2- 172.21.15.16:0/2997983701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa11006cf60 0x7fa1100a8480 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc014040 tx=0x7fa0fc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:25.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.616+0000 7fa10d7fa700 1 -- 172.21.15.16:0/2997983701 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0fc004310 con 0x7fa11006cf60 2023-05-20T01:16:25.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.616+0000 7fa10d7fa700 1 -- 172.21.15.16:0/2997983701 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0fc01f070 con 0x7fa11006cf60 2023-05-20T01:16:25.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.616+0000 7fa116fee700 1 -- 172.21.15.16:0/2997983701 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa11015b240 con 0x7fa11006cf60 2023-05-20T01:16:25.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.616+0000 7fa10d7fa700 1 -- 172.21.15.16:0/2997983701 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0fc0036a0 con 0x7fa11006cf60 2023-05-20T01:16:25.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.616+0000 7fa116fee700 1 -- 172.21.15.16:0/2997983701 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa11015af60 con 0x7fa11006cf60 2023-05-20T01:16:25.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.617+0000 7fa116fee700 1 -- 172.21.15.16:0/2997983701 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa11015b520 con 0x7fa11006cf60 2023-05-20T01:16:25.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.617+0000 7fa10d7fa700 1 -- 172.21.15.16:0/2997983701 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa0fc021030 con 0x7fa11006cf60 2023-05-20T01:16:25.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.617+0000 7fa10d7fa700 1 --2- 172.21.15.16:0/2997983701 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa0f406ace0 0x7fa0f406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:25.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.618+0000 7fa10d7fa700 1 -- 172.21.15.16:0/2997983701 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fa0fc08a500 con 0x7fa11006cf60 2023-05-20T01:16:25.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.618+0000 7fa114d8a700 1 --2- 172.21.15.16:0/2997983701 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa0f406ace0 0x7fa0f406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:25.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.618+0000 7fa114d8a700 1 --2- 172.21.15.16:0/2997983701 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa0f406ace0 0x7fa0f406d190 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa104006fd0 tx=0x7fa104008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:25.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.622+0000 7fa10d7fa700 1 -- 172.21.15.16:0/2997983701 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fa0fc055ab0 con 0x7fa11006cf60 2023-05-20T01:16:25.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.821+0000 7fa116fee700 1 -- 172.21.15.16:0/2997983701 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fa1100747b0 con 0x7fa11006cf60 2023-05-20T01:16:25.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.821+0000 7fa10d7fa700 1 -- 172.21.15.16:0/2997983701 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v17) v1 ==== 84+0+126 (secure 0 0 0) 0x7fa0fc015090 con 0x7fa11006cf60 2023-05-20T01:16:25.825 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:16:25.825 INFO:teuthology.orchestra.run.smithi016.stdout:[ 2023-05-20T01:16:25.825 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-05-20T01:16:25.825 INFO:teuthology.orchestra.run.smithi016.stdout: "poolnum": 1, 2023-05-20T01:16:25.825 INFO:teuthology.orchestra.run.smithi016.stdout: "poolname": "rbd" 2023-05-20T01:16:25.826 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-05-20T01:16:25.826 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-05-20T01:16:25.826 INFO:teuthology.orchestra.run.smithi016.stdout: "poolnum": 2, 2023-05-20T01:16:25.826 INFO:teuthology.orchestra.run.smithi016.stdout: "poolname": ".mgr" 2023-05-20T01:16:25.826 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:16:25.826 INFO:teuthology.orchestra.run.smithi016.stdout:] 2023-05-20T01:16:25.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.823+0000 7fa116fee700 1 -- 172.21.15.16:0/2997983701 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa0f406ace0 msgr2=0x7fa0f406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:25.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.823+0000 7fa116fee700 1 --2- 172.21.15.16:0/2997983701 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa0f406ace0 0x7fa0f406d190 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa104006fd0 tx=0x7fa104008040 comp rx=0 tx=0).stop 2023-05-20T01:16:25.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.823+0000 7fa116fee700 1 -- 172.21.15.16:0/2997983701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa11006cf60 msgr2=0x7fa1100a8480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:25.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.823+0000 7fa116fee700 1 --2- 172.21.15.16:0/2997983701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa11006cf60 0x7fa1100a8480 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc014040 tx=0x7fa0fc004750 comp rx=0 tx=0).stop 2023-05-20T01:16:25.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.823+0000 7fa116fee700 1 -- 172.21.15.16:0/2997983701 shutdown_connections 2023-05-20T01:16:25.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.823+0000 7fa116fee700 1 --2- 172.21.15.16:0/2997983701 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa1100b54f0 0x7fa1100a89c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:25.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.823+0000 7fa116fee700 1 --2- 172.21.15.16:0/2997983701 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa0f406ace0 0x7fa0f406d190 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:25.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.823+0000 7fa116fee700 1 --2- 172.21.15.16:0/2997983701 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa11006cb90 0x7fa1100b0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:25.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.823+0000 7fa116fee700 1 --2- 172.21.15.16:0/2997983701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa11006cf60 0x7fa1100a8480 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:25.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.823+0000 7fa116fee700 1 -- 172.21.15.16:0/2997983701 >> 172.21.15.16:0/2997983701 conn(0x7fa1100661b0 msgr2=0x7fa11015d8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:25.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.823+0000 7fa116fee700 1 -- 172.21.15.16:0/2997983701 shutdown_connections 2023-05-20T01:16:25.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.823+0000 7fa116fee700 1 -- 172.21.15.16:0/2997983701 wait complete. 2023-05-20T01:16:25.838 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-05-20T01:16:25.839 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-05-20T01:16:25.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.964+0000 7f985f735700 1 Processor -- start 2023-05-20T01:16:25.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.965+0000 7f985f735700 1 -- start start 2023-05-20T01:16:25.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.965+0000 7f985f735700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f98580ce310 0x7f985815a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:25.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.965+0000 7f985f735700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f985815adc0 con 0x7f9858159880 2023-05-20T01:16:25.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.965+0000 7f985f735700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f985815a800 con 0x7f9858159c50 2023-05-20T01:16:25.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.965+0000 7f985f735700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f985815a610 con 0x7f98580ce310 2023-05-20T01:16:25.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.965+0000 7f985d4d1700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f9858159880 0x7f985815dd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:40726/0 (socket says 172.21.15.16:40726) 2023-05-20T01:16:25.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.965+0000 7f985d4d1700 1 -- 172.21.15.16:0/3013377809 learned_addr learned my addr 172.21.15.16:0/3013377809 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:16:25.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.966+0000 7f985dcd2700 1 --2- 172.21.15.16:0/3013377809 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f98580ce310 0x7f985815a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:25.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.966+0000 7f984ffff700 1 -- 172.21.15.16:0/3013377809 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3393837876 0 0) 0x7f985815adc0 con 0x7f9858159880 2023-05-20T01:16:25.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.966+0000 7f984ffff700 1 -- 172.21.15.16:0/3013377809 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f98440035b0 con 0x7f9858159880 2023-05-20T01:16:25.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.966+0000 7f984ffff700 1 -- 172.21.15.16:0/3013377809 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1153383763 0 0) 0x7f985815a800 con 0x7f9858159c50 2023-05-20T01:16:25.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.966+0000 7f984ffff700 1 -- 172.21.15.16:0/3013377809 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f985815adc0 con 0x7f9858159c50 2023-05-20T01:16:25.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.966+0000 7f984ffff700 1 -- 172.21.15.16:0/3013377809 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2934404097 0 0) 0x7f98440035b0 con 0x7f9858159880 2023-05-20T01:16:25.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.966+0000 7f984ffff700 1 -- 172.21.15.16:0/3013377809 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f98580ce310 msgr2=0x7f985815a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:25.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.966+0000 7f984ffff700 1 --2- 172.21.15.16:0/3013377809 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f98580ce310 0x7f985815a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:25.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.966+0000 7f984ffff700 1 -- 172.21.15.16:0/3013377809 >> v1:172.21.15.132:6789/0 conn(0x7f9858159c50 legacy=0x7f98580cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:25.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.966+0000 7f984ffff700 1 -- 172.21.15.16:0/3013377809 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f985815aae0 con 0x7f9858159880 2023-05-20T01:16:25.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.966+0000 7f984ffff700 1 -- 172.21.15.16:0/3013377809 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7f9848001670 con 0x7f9858159880 2023-05-20T01:16:25.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.966+0000 7f984ffff700 1 -- 172.21.15.16:0/3013377809 >> v1:172.21.15.16:6789/0 conn(0x7f9858159880 legacy=0x7f985815dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:25.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.966+0000 7f984ffff700 1 --2- 172.21.15.16:0/3013377809 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f98580ce310 0x7f9844003bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:25.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.966+0000 7f984ffff700 1 --2- 172.21.15.16:0/3013377809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9844004120 0x7f9844004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:25.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.966+0000 7f984ffff700 1 --2- 172.21.15.16:0/3013377809 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9844004d90 0x7f9844005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:25.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.966+0000 7f984ffff700 1 -- 172.21.15.16:0/3013377809 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f984400bac0 con 0x7f9844004120 2023-05-20T01:16:25.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.966+0000 7f984ffff700 1 -- 172.21.15.16:0/3013377809 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f984400bc00 con 0x7f9844004d90 2023-05-20T01:16:25.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.966+0000 7f984ffff700 1 -- 172.21.15.16:0/3013377809 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f984400bdc0 con 0x7f98580ce310 2023-05-20T01:16:25.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.966+0000 7f985ccd0700 1 --2- 172.21.15.16:0/3013377809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9844004120 0x7f9844004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:25.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.966+0000 7f985d4d1700 1 --2- 172.21.15.16:0/3013377809 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f98580ce310 0x7f9844003bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:25.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.966+0000 7f985dcd2700 1 --2- 172.21.15.16:0/3013377809 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9844004d90 0x7f9844005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:25.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.966+0000 7f985ccd0700 1 -- 172.21.15.16:0/3013377809 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f98580ce310 msgr2=0x7f9844003bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:25.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.966+0000 7f985ccd0700 1 --2- 172.21.15.16:0/3013377809 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f98580ce310 0x7f9844003bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:25.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.966+0000 7f985ccd0700 1 -- 172.21.15.16:0/3013377809 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9844004d90 msgr2=0x7f9844005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:25.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.966+0000 7f985ccd0700 1 --2- 172.21.15.16:0/3013377809 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9844004d90 0x7f9844005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:25.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.966+0000 7f985ccd0700 1 -- 172.21.15.16:0/3013377809 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98440035b0 con 0x7f9844004120 2023-05-20T01:16:25.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.966+0000 7f985dcd2700 1 --2- 172.21.15.16:0/3013377809 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9844004d90 0x7f9844005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:16:25.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.967+0000 7f985ccd0700 1 --2- 172.21.15.16:0/3013377809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9844004120 0x7f9844004850 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9854007fd0 tx=0x7f9844005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=825b0469c1939440 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:25.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.967+0000 7f984ffff700 1 -- 172.21.15.16:0/3013377809 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9854001430 con 0x7f9844004120 2023-05-20T01:16:25.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.967+0000 7f984ffff700 1 -- 172.21.15.16:0/3013377809 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98540052d0 con 0x7f9844004120 2023-05-20T01:16:25.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.967+0000 7f984ffff700 1 -- 172.21.15.16:0/3013377809 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9854001d40 con 0x7f9844004120 2023-05-20T01:16:25.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.967+0000 7f985f735700 1 -- 172.21.15.16:0/3013377809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9844004120 msgr2=0x7f9844004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:25.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.967+0000 7f985f735700 1 --2- 172.21.15.16:0/3013377809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9844004120 0x7f9844004850 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9854007fd0 tx=0x7f9844005880 comp rx=0 tx=0).stop 2023-05-20T01:16:25.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.967+0000 7f985dcd2700 1 -- 172.21.15.16:0/3013377809 reap_dead start 2023-05-20T01:16:25.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.967+0000 7f985f735700 1 -- 172.21.15.16:0/3013377809 shutdown_connections 2023-05-20T01:16:25.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.967+0000 7f985f735700 1 -- 172.21.15.16:0/3013377809 >> 172.21.15.16:0/3013377809 conn(0x7f98580ccab0 msgr2=0x7f98580cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:25.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.967+0000 7f985f735700 1 -- 172.21.15.16:0/3013377809 shutdown_connections 2023-05-20T01:16:25.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.967+0000 7f985f735700 1 -- 172.21.15.16:0/3013377809 wait complete. 2023-05-20T01:16:25.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.969+0000 7f985f735700 1 Processor -- start 2023-05-20T01:16:25.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.969+0000 7f985f735700 1 -- start start 2023-05-20T01:16:25.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.969+0000 7f985f735700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9844004120 0x7f98580c3e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:25.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.969+0000 7f985f735700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f985814f980 0x7f98580b2e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:25.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.969+0000 7f985f735700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f985815be30 0x7f98581583d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:25.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.969+0000 7f985f735700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98581550e0 con 0x7f985814f980 2023-05-20T01:16:25.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.969+0000 7f985f735700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f985814fdf0 con 0x7f985815be30 2023-05-20T01:16:25.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.969+0000 7f985f735700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98581558b0 con 0x7f9844004120 2023-05-20T01:16:25.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.970+0000 7f985ccd0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f985814f980 0x7f98580b2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:25.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.970+0000 7f985ccd0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f985814f980 0x7f98580b2e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57444/0 (socket says 172.21.15.16:57444) 2023-05-20T01:16:25.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.970+0000 7f985ccd0700 1 -- 172.21.15.16:0/2046873518 learned_addr learned my addr 172.21.15.16:0/2046873518 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:25.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.970+0000 7f985d4d1700 1 --2- 172.21.15.16:0/2046873518 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9844004120 0x7f98580c3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:25.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.970+0000 7f985dcd2700 1 --2- 172.21.15.16:0/2046873518 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f985815be30 0x7f98581583d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:25.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.970+0000 7f985ccd0700 1 -- 172.21.15.16:0/2046873518 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9844004120 msgr2=0x7f98580c3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:25.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.970+0000 7f985ccd0700 1 --2- 172.21.15.16:0/2046873518 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9844004120 0x7f98580c3e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:25.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.970+0000 7f985ccd0700 1 -- 172.21.15.16:0/2046873518 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f985815be30 msgr2=0x7f98581583d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:25.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.970+0000 7f985ccd0700 1 --2- 172.21.15.16:0/2046873518 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f985815be30 0x7f98581583d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:25.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.970+0000 7f985ccd0700 1 -- 172.21.15.16:0/2046873518 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f985815d820 con 0x7f985814f980 2023-05-20T01:16:25.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.970+0000 7f985ccd0700 1 --2- 172.21.15.16:0/2046873518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f985814f980 0x7f98580b2e60 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9854005690 tx=0x7f98540012d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:25.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.970+0000 7f984dffb700 1 -- 172.21.15.16:0/2046873518 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98540013d0 con 0x7f985814f980 2023-05-20T01:16:25.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.970+0000 7f984dffb700 1 -- 172.21.15.16:0/2046873518 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9854006ca0 con 0x7f985814f980 2023-05-20T01:16:25.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.970+0000 7f985f735700 1 -- 172.21.15.16:0/2046873518 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f985815d540 con 0x7f985814f980 2023-05-20T01:16:25.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.970+0000 7f984dffb700 1 -- 172.21.15.16:0/2046873518 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9854007280 con 0x7f985814f980 2023-05-20T01:16:25.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.971+0000 7f985f735700 1 -- 172.21.15.16:0/2046873518 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98580b1bd0 con 0x7f985814f980 2023-05-20T01:16:25.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.971+0000 7f985f735700 1 -- 172.21.15.16:0/2046873518 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98580747b0 con 0x7f985814f980 2023-05-20T01:16:25.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.972+0000 7f984dffb700 1 -- 172.21.15.16:0/2046873518 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f98540073e0 con 0x7f985814f980 2023-05-20T01:16:25.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.973+0000 7f984dffb700 1 --2- 172.21.15.16:0/2046873518 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f983c06ad10 0x7f983c06d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:25.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.973+0000 7f984dffb700 1 -- 172.21.15.16:0/2046873518 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f985408bd50 con 0x7f985814f980 2023-05-20T01:16:25.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.973+0000 7f985d4d1700 1 --2- 172.21.15.16:0/2046873518 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f983c06ad10 0x7f983c06d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:25.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.973+0000 7f985d4d1700 1 --2- 172.21.15.16:0/2046873518 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f983c06ad10 0x7f983c06d1c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9848001c20 tx=0x7f9848001e00 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:25.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:25.976+0000 7f984dffb700 1 -- 172.21.15.16:0/2046873518 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f98540573e0 con 0x7f985814f980 2023-05-20T01:16:26.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:26.177+0000 7f985f735700 1 -- 172.21.15.16:0/2046873518 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f985815b200 con 0x7f985814f980 2023-05-20T01:16:26.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:26.894+0000 7f984dffb700 1 -- 172.21.15.16:0/2046873518 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v18) v1 ==== 138+0+0 (secure 0 0 0) 0x7f98540170b0 con 0x7f985814f980 2023-05-20T01:16:26.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:26.898+0000 7f985f735700 1 -- 172.21.15.16:0/2046873518 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f983c06ad10 msgr2=0x7f983c06d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:26.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:26.898+0000 7f985f735700 1 --2- 172.21.15.16:0/2046873518 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f983c06ad10 0x7f983c06d1c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9848001c20 tx=0x7f9848001e00 comp rx=0 tx=0).stop 2023-05-20T01:16:26.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:26.898+0000 7f985f735700 1 -- 172.21.15.16:0/2046873518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f985814f980 msgr2=0x7f98580b2e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:26.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:26.898+0000 7f985f735700 1 --2- 172.21.15.16:0/2046873518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f985814f980 0x7f98580b2e60 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9854005690 tx=0x7f98540012d0 comp rx=0 tx=0).stop 2023-05-20T01:16:26.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:26.899+0000 7f985f735700 1 -- 172.21.15.16:0/2046873518 shutdown_connections 2023-05-20T01:16:26.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:26.899+0000 7f985f735700 1 --2- 172.21.15.16:0/2046873518 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f985815be30 0x7f98581583d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:26.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:26.899+0000 7f985f735700 1 --2- 172.21.15.16:0/2046873518 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f983c06ad10 0x7f983c06d1c0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:26.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:26.899+0000 7f985f735700 1 --2- 172.21.15.16:0/2046873518 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9844004120 0x7f98580c3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:26.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:26.899+0000 7f985f735700 1 --2- 172.21.15.16:0/2046873518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f985814f980 0x7f98580b2e60 secure :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9854005690 tx=0x7f98540012d0 comp rx=0 tx=0).stop 2023-05-20T01:16:26.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:26.899+0000 7f985f735700 1 -- 172.21.15.16:0/2046873518 >> 172.21.15.16:0/2046873518 conn(0x7f98580ccab0 msgr2=0x7f98580afde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:26.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:26.900+0000 7f985f735700 1 -- 172.21.15.16:0/2046873518 shutdown_connections 2023-05-20T01:16:26.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:26.900+0000 7f985f735700 1 -- 172.21.15.16:0/2046873518 wait complete. 2023-05-20T01:16:26.905 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' created 2023-05-20T01:16:26.919 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2023-05-20T01:16:27.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:27.053+0000 7efdd31f8700 1 Processor -- start 2023-05-20T01:16:27.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:27.053+0000 7efdd31f8700 1 -- start start 2023-05-20T01:16:27.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:27.054+0000 7efdd31f8700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7efdcc15cec0 0x7efdcc071bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:27.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:27.054+0000 7efdd31f8700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efdcc15d290 0x7efdcc072130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:27.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:27.054+0000 7efdd31f8700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7efdcc0cf2c0 0x7efdcc0cf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:27.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:27.054+0000 7efdd31f8700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdcc0cd630 con 0x7efdcc15cec0 2023-05-20T01:16:27.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:27.054+0000 7efdd31f8700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdcc0cca60 con 0x7efdcc0cf2c0 2023-05-20T01:16:27.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:27.054+0000 7efdd31f8700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdcc0ccd00 con 0x7efdcc15d290 2023-05-20T01:16:27.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:27.054+0000 7efdd0f94700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7efdcc15cec0 0x7efdcc071bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:27.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:27.054+0000 7efdd0f94700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7efdcc15cec0 0x7efdcc071bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57450/0 (socket says 172.21.15.16:57450) 2023-05-20T01:16:27.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:27.054+0000 7efdd0f94700 1 -- 172.21.15.16:0/1430970182 learned_addr learned my addr 172.21.15.16:0/1430970182 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:27.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:27.054+0000 7efdd1795700 1 --2- 172.21.15.16:0/1430970182 >> v2:172.21.15.132:3300/0 conn(0x7efdcc0cf2c0 0x7efdcc0cf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:27.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:27.054+0000 7efdcbfff700 1 --2- 172.21.15.16:0/1430970182 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efdcc15d290 0x7efdcc072130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:27.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:27.054+0000 7efdd0f94700 1 -- 172.21.15.16:0/1430970182 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efdcc15d290 msgr2=0x7efdcc072130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:27.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:27.054+0000 7efdd0f94700 1 --2- 172.21.15.16:0/1430970182 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efdcc15d290 0x7efdcc072130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:27.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:27.054+0000 7efdd0f94700 1 -- 172.21.15.16:0/1430970182 >> v2:172.21.15.132:3300/0 conn(0x7efdcc0cf2c0 msgr2=0x7efdcc0cf720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:27.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:27.054+0000 7efdd0f94700 1 --2- 172.21.15.16:0/1430970182 >> v2:172.21.15.132:3300/0 conn(0x7efdcc0cf2c0 0x7efdcc0cf720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:27.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:27.054+0000 7efdd0f94700 1 -- 172.21.15.16:0/1430970182 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdcc0ccfe0 con 0x7efdcc15cec0 2023-05-20T01:16:27.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:27.055+0000 7efdd0f94700 1 --2- 172.21.15.16:0/1430970182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efdcc15cec0 0x7efdcc071bf0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7efdcc071140 tx=0x7efdc0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=805e9ec5069b032a server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:27.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:27.055+0000 7efdcb7fe700 1 -- 172.21.15.16:0/1430970182 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdc0005e30 con 0x7efdcc15cec0 2023-05-20T01:16:27.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:27.055+0000 7efdcb7fe700 1 -- 172.21.15.16:0/1430970182 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdc0004030 con 0x7efdcc15cec0 2023-05-20T01:16:27.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:27.055+0000 7efdcb7fe700 1 -- 172.21.15.16:0/1430970182 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdc00043e0 con 0x7efdcc15cec0 2023-05-20T01:16:27.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:27.055+0000 7efdd31f8700 1 -- 172.21.15.16:0/1430970182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efdcc15cec0 msgr2=0x7efdcc071bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:27.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:27.055+0000 7efdd31f8700 1 --2- 172.21.15.16:0/1430970182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efdcc15cec0 0x7efdcc071bf0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7efdcc071140 tx=0x7efdc0009bd0 comp rx=0 tx=0).stop 2023-05-20T01:16:27.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:27.055+0000 7efdd31f8700 1 -- 172.21.15.16:0/1430970182 shutdown_connections 2023-05-20T01:16:27.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:27.055+0000 7efdd31f8700 1 --2- 172.21.15.16:0/1430970182 >> v2:172.21.15.132:3300/0 conn(0x7efdcc0cf2c0 0x7efdcc0cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:27.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:27.055+0000 7efdd31f8700 1 --2- 172.21.15.16:0/1430970182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efdcc15cec0 0x7efdcc071bf0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:27.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:27.055+0000 7efdd31f8700 1 --2- 172.21.15.16:0/1430970182 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efdcc15d290 0x7efdcc072130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:27.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:27.055+0000 7efdd31f8700 1 -- 172.21.15.16:0/1430970182 >> 172.21.15.16:0/1430970182 conn(0x7efdcc0cffd0 msgr2=0x7efdcc0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:27.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:27.055+0000 7efdd31f8700 1 -- 172.21.15.16:0/1430970182 shutdown_connections 2023-05-20T01:16:27.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:27.055+0000 7efdd31f8700 1 -- 172.21.15.16:0/1430970182 wait complete. 2023-05-20T01:16:27.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:27.056+0000 7efdd31f8700 1 Processor -- start 2023-05-20T01:16:27.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:27.057+0000 7efdd31f8700 1 -- start start 2023-05-20T01:16:27.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:27.057+0000 7efdd31f8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efdcc0cf2c0 0x7efdcc0b54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:27.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:27.057+0000 7efdd31f8700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efdcc15cec0 0x7efdcc0c5e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:27.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:27.057+0000 7efdd31f8700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efdcc15d290 0x7efdcc153150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:27.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:27.057+0000 7efdd31f8700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdcc154540 con 0x7efdcc0cf2c0 2023-05-20T01:16:27.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:27.057+0000 7efdd31f8700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdcc0c63c0 con 0x7efdcc15cec0 2023-05-20T01:16:27.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:27.057+0000 7efdd31f8700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdcc153690 con 0x7efdcc15d290 2023-05-20T01:16:27.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:27.057+0000 7efdd0f94700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efdcc0cf2c0 0x7efdcc0b54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:27.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:27.057+0000 7efdd0f94700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efdcc0cf2c0 0x7efdcc0b54c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57456/0 (socket says 172.21.15.16:57456) 2023-05-20T01:16:27.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:27.057+0000 7efdd0f94700 1 -- 172.21.15.16:0/332386180 learned_addr learned my addr 172.21.15.16:0/332386180 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:27.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:27.058+0000 7efdd1795700 1 --2- 172.21.15.16:0/332386180 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efdcc15d290 0x7efdcc153150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:27.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:27.058+0000 7efdcbfff700 1 --2- 172.21.15.16:0/332386180 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efdcc15cec0 0x7efdcc0c5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:27.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:27.058+0000 7efdd0f94700 1 -- 172.21.15.16:0/332386180 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efdcc15d290 msgr2=0x7efdcc153150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:27.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:27.058+0000 7efdd0f94700 1 --2- 172.21.15.16:0/332386180 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efdcc15d290 0x7efdcc153150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:27.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:27.058+0000 7efdd0f94700 1 -- 172.21.15.16:0/332386180 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efdcc15cec0 msgr2=0x7efdcc0c5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:27.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:27.058+0000 7efdd0f94700 1 --2- 172.21.15.16:0/332386180 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efdcc15cec0 0x7efdcc0c5e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:27.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:27.058+0000 7efdd0f94700 1 -- 172.21.15.16:0/332386180 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdc0009500 con 0x7efdcc0cf2c0 2023-05-20T01:16:27.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:27.058+0000 7efdd0f94700 1 --2- 172.21.15.16:0/332386180 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efdcc0cf2c0 0x7efdcc0b54c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7efdc0009c10 tx=0x7efdc0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:27.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:27.058+0000 7efdc97fa700 1 -- 172.21.15.16:0/332386180 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdc0003ce0 con 0x7efdcc0cf2c0 2023-05-20T01:16:27.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:27.058+0000 7efdc97fa700 1 -- 172.21.15.16:0/332386180 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdc0003e40 con 0x7efdcc0cf2c0 2023-05-20T01:16:27.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:27.058+0000 7efdd31f8700 1 -- 172.21.15.16:0/332386180 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdcc0cfc60 con 0x7efdcc0cf2c0 2023-05-20T01:16:27.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:27.058+0000 7efdc97fa700 1 -- 172.21.15.16:0/332386180 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdc001c640 con 0x7efdcc0cf2c0 2023-05-20T01:16:27.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:27.058+0000 7efdd31f8700 1 -- 172.21.15.16:0/332386180 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdcc0cd2c0 con 0x7efdcc0cf2c0 2023-05-20T01:16:27.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:27.059+0000 7efdd31f8700 1 -- 172.21.15.16:0/332386180 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdcc0cfdf0 con 0x7efdcc0cf2c0 2023-05-20T01:16:27.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:27.061+0000 7efdc97fa700 1 -- 172.21.15.16:0/332386180 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7efdc001c7a0 con 0x7efdcc0cf2c0 2023-05-20T01:16:27.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:27.061+0000 7efdc97fa700 1 --2- 172.21.15.16:0/332386180 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7efdb006ace0 0x7efdb006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:27.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:27.061+0000 7efdc97fa700 1 -- 172.21.15.16:0/332386180 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5580+0+0 (secure 0 0 0) 0x7efdc008a580 con 0x7efdcc0cf2c0 2023-05-20T01:16:27.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:27.061+0000 7efdcbfff700 1 --2- 172.21.15.16:0/332386180 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7efdb006ace0 0x7efdb006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:27.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:27.061+0000 7efdcbfff700 1 --2- 172.21.15.16:0/332386180 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7efdb006ace0 0x7efdb006d190 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7efdcc0b5d40 tx=0x7efdb8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:27.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:27.065+0000 7efdc97fa700 1 -- 172.21.15.16:0/332386180 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7efdc0055980 con 0x7efdcc0cf2c0 2023-05-20T01:16:27.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:27.261+0000 7efdd31f8700 1 -- 172.21.15.16:0/332386180 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7efdcc0747b0 con 0x7efdcc0cf2c0 2023-05-20T01:16:27.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:27.899+0000 7efdc97fa700 1 -- 172.21.15.16:0/332386180 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v19) v1 ==== 171+0+0 (secure 0 0 0) 0x7efdc0058fa0 con 0x7efdcc0cf2c0 2023-05-20T01:16:27.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:27.904+0000 7efdd31f8700 1 -- 172.21.15.16:0/332386180 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7efdb006ace0 msgr2=0x7efdb006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:27.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:27.904+0000 7efdd31f8700 1 --2- 172.21.15.16:0/332386180 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7efdb006ace0 0x7efdb006d190 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7efdcc0b5d40 tx=0x7efdb8008040 comp rx=0 tx=0).stop 2023-05-20T01:16:27.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:27.904+0000 7efdd31f8700 1 -- 172.21.15.16:0/332386180 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efdcc0cf2c0 msgr2=0x7efdcc0b54c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:27.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:27.904+0000 7efdd31f8700 1 --2- 172.21.15.16:0/332386180 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efdcc0cf2c0 0x7efdcc0b54c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7efdc0009c10 tx=0x7efdc0004750 comp rx=0 tx=0).stop 2023-05-20T01:16:27.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:27.904+0000 7efdd31f8700 1 -- 172.21.15.16:0/332386180 shutdown_connections 2023-05-20T01:16:27.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:27.904+0000 7efdd31f8700 1 --2- 172.21.15.16:0/332386180 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7efdb006ace0 0x7efdb006d190 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:27.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:27.904+0000 7efdd31f8700 1 --2- 172.21.15.16:0/332386180 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efdcc15d290 0x7efdcc153150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:27.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:27.904+0000 7efdd31f8700 1 --2- 172.21.15.16:0/332386180 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efdcc0cf2c0 0x7efdcc0b54c0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:27.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:27.904+0000 7efdd31f8700 1 --2- 172.21.15.16:0/332386180 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efdcc15cec0 0x7efdcc0c5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:27.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:27.904+0000 7efdd31f8700 1 -- 172.21.15.16:0/332386180 >> 172.21.15.16:0/332386180 conn(0x7efdcc0cffd0 msgr2=0x7efdcc160050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:27.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:27.907+0000 7efdd31f8700 1 -- 172.21.15.16:0/332386180 shutdown_connections 2023-05-20T01:16:27.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:27.910+0000 7efdd31f8700 1 -- 172.21.15.16:0/332386180 wait complete. 2023-05-20T01:16:27.923 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' created 2023-05-20T01:16:27.935 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-05-20T01:16:28.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.062+0000 7f689fa2f700 1 Processor -- start 2023-05-20T01:16:28.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.062+0000 7f689fa2f700 1 -- start start 2023-05-20T01:16:28.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.062+0000 7f689fa2f700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6898159b50 0x7f689806e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:28.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.063+0000 7f689fa2f700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68980ccd30 con 0x7f68980cd1c0 2023-05-20T01:16:28.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.063+0000 7f689fa2f700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68980ce990 con 0x7f68980717c0 2023-05-20T01:16:28.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.063+0000 7f689fa2f700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68980cc6e0 con 0x7f6898159b50 2023-05-20T01:16:28.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.063+0000 7f689cfca700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f68980cd1c0 0x7f68980710b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:40760/0 (socket says 172.21.15.16:40760) 2023-05-20T01:16:28.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.063+0000 7f689cfca700 1 -- 172.21.15.16:0/718803473 learned_addr learned my addr 172.21.15.16:0/718803473 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:16:28.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.063+0000 7f689d7cb700 1 --2- 172.21.15.16:0/718803473 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6898159b50 0x7f689806e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:28.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.063+0000 7f688ffff700 1 -- 172.21.15.16:0/718803473 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 116813749 0 0) 0x7f68980ccd30 con 0x7f68980cd1c0 2023-05-20T01:16:28.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.063+0000 7f688ffff700 1 -- 172.21.15.16:0/718803473 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68840035b0 con 0x7f68980cd1c0 2023-05-20T01:16:28.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.063+0000 7f688ffff700 1 -- 172.21.15.16:0/718803473 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2949414750 0 0) 0x7f68840035b0 con 0x7f68980cd1c0 2023-05-20T01:16:28.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.063+0000 7f688ffff700 1 -- 172.21.15.16:0/718803473 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6898159b50 msgr2=0x7f689806e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:28.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.063+0000 7f688ffff700 1 --2- 172.21.15.16:0/718803473 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6898159b50 0x7f689806e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:28.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.063+0000 7f688ffff700 1 -- 172.21.15.16:0/718803473 >> v1:172.21.15.132:6789/0 conn(0x7f68980717c0 legacy=0x7f68980ca870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:28.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.063+0000 7f688ffff700 1 -- 172.21.15.16:0/718803473 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68980cc9c0 con 0x7f68980cd1c0 2023-05-20T01:16:28.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.063+0000 7f688ffff700 1 -- 172.21.15.16:0/718803473 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7f6894001670 con 0x7f68980cd1c0 2023-05-20T01:16:28.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.063+0000 7f688ffff700 1 -- 172.21.15.16:0/718803473 >> v1:172.21.15.16:6789/0 conn(0x7f68980cd1c0 legacy=0x7f68980710b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:28.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.063+0000 7f688ffff700 1 --2- 172.21.15.16:0/718803473 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6898159b50 0x7f6884003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:28.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.063+0000 7f688ffff700 1 --2- 172.21.15.16:0/718803473 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6884003f70 0x7f6884004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:28.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.063+0000 7f688ffff700 1 --2- 172.21.15.16:0/718803473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6884004d90 0x7f6884005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:28.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.063+0000 7f688ffff700 1 -- 172.21.15.16:0/718803473 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68980cc6e0 con 0x7f6884004d90 2023-05-20T01:16:28.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.063+0000 7f688ffff700 1 -- 172.21.15.16:0/718803473 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f688400ba40 con 0x7f6884003f70 2023-05-20T01:16:28.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.063+0000 7f688ffff700 1 -- 172.21.15.16:0/718803473 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f688400bc00 con 0x7f6898159b50 2023-05-20T01:16:28.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.064+0000 7f689d7cb700 1 --2- 172.21.15.16:0/718803473 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6898159b50 0x7f6884003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:28.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.063+0000 7f689dfcc700 1 --2- 172.21.15.16:0/718803473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6884004d90 0x7f6884005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:28.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.064+0000 7f689cfca700 1 --2- 172.21.15.16:0/718803473 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6884003f70 0x7f6884004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:28.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.064+0000 7f689dfcc700 1 -- 172.21.15.16:0/718803473 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6898159b50 msgr2=0x7f6884003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:28.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.064+0000 7f689dfcc700 1 --2- 172.21.15.16:0/718803473 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6898159b50 0x7f6884003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:28.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.064+0000 7f689dfcc700 1 -- 172.21.15.16:0/718803473 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6884003f70 msgr2=0x7f6884004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:28.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.064+0000 7f689dfcc700 1 --2- 172.21.15.16:0/718803473 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6884003f70 0x7f6884004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:28.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.064+0000 7f689dfcc700 1 -- 172.21.15.16:0/718803473 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68840035b0 con 0x7f6884004d90 2023-05-20T01:16:28.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.064+0000 7f689dfcc700 1 --2- 172.21.15.16:0/718803473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6884004d90 0x7f6884005200 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f689000c290 tx=0x7f6884005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=156e7e11bc73fed7 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:28.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.064+0000 7f688ffff700 1 -- 172.21.15.16:0/718803473 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6890001e70 con 0x7f6884004d90 2023-05-20T01:16:28.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.064+0000 7f688ffff700 1 -- 172.21.15.16:0/718803473 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f689000ce50 con 0x7f6884004d90 2023-05-20T01:16:28.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.064+0000 7f688ffff700 1 -- 172.21.15.16:0/718803473 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6890012660 con 0x7f6884004d90 2023-05-20T01:16:28.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.065+0000 7f689fa2f700 1 -- 172.21.15.16:0/718803473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6884004d90 msgr2=0x7f6884005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:28.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.065+0000 7f689fa2f700 1 --2- 172.21.15.16:0/718803473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6884004d90 0x7f6884005200 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f689000c290 tx=0x7f6884005880 comp rx=0 tx=0).stop 2023-05-20T01:16:28.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.065+0000 7f689dfcc700 1 -- 172.21.15.16:0/718803473 reap_dead start 2023-05-20T01:16:28.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.065+0000 7f689fa2f700 1 -- 172.21.15.16:0/718803473 shutdown_connections 2023-05-20T01:16:28.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.065+0000 7f689fa2f700 1 -- 172.21.15.16:0/718803473 >> 172.21.15.16:0/718803473 conn(0x7f68980cffd0 msgr2=0x7f68980d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:28.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.065+0000 7f689fa2f700 1 -- 172.21.15.16:0/718803473 shutdown_connections 2023-05-20T01:16:28.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.065+0000 7f689fa2f700 1 -- 172.21.15.16:0/718803473 wait complete. 2023-05-20T01:16:28.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.066+0000 7f689fa2f700 1 Processor -- start 2023-05-20T01:16:28.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.066+0000 7f689fa2f700 1 -- start start 2023-05-20T01:16:28.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.067+0000 7f689fa2f700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6884004d90 0x7f68980c3c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:28.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.067+0000 7f689fa2f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68981f6310 0x7f68980c8140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:28.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.067+0000 7f689fa2f700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f68981f6780 0x7f68981f3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:28.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.067+0000 7f689fa2f700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68980c7cb0 con 0x7f68981f6310 2023-05-20T01:16:28.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.067+0000 7f689fa2f700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68980c70a0 con 0x7f68981f6780 2023-05-20T01:16:28.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.067+0000 7f689fa2f700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68980c7380 con 0x7f6884004d90 2023-05-20T01:16:28.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.067+0000 7f689cfca700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68981f6310 0x7f68980c8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:28.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.067+0000 7f689cfca700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68981f6310 0x7f68980c8140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57478/0 (socket says 172.21.15.16:57478) 2023-05-20T01:16:28.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.067+0000 7f689cfca700 1 -- 172.21.15.16:0/2122153397 learned_addr learned my addr 172.21.15.16:0/2122153397 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:28.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.067+0000 7f689d7cb700 1 --2- 172.21.15.16:0/2122153397 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6884004d90 0x7f68980c3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:28.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.067+0000 7f689dfcc700 1 --2- 172.21.15.16:0/2122153397 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f68981f6780 0x7f68981f3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:28.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.067+0000 7f689cfca700 1 -- 172.21.15.16:0/2122153397 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6884004d90 msgr2=0x7f68980c3c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:28.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.067+0000 7f689cfca700 1 --2- 172.21.15.16:0/2122153397 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6884004d90 0x7f68980c3c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:28.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.067+0000 7f689cfca700 1 -- 172.21.15.16:0/2122153397 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f68981f6780 msgr2=0x7f68981f3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:28.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.067+0000 7f689cfca700 1 --2- 172.21.15.16:0/2122153397 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f68981f6780 0x7f68981f3bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:28.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.067+0000 7f689cfca700 1 -- 172.21.15.16:0/2122153397 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68980cec70 con 0x7f68981f6310 2023-05-20T01:16:28.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.067+0000 7f689dfcc700 1 --2- 172.21.15.16:0/2122153397 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f68981f6780 0x7f68981f3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:16:28.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.067+0000 7f689cfca700 1 --2- 172.21.15.16:0/2122153397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68981f6310 0x7f68980c8140 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6894007600 tx=0x7f689400f4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:28.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.068+0000 7f688dffb700 1 -- 172.21.15.16:0/2122153397 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68940077a0 con 0x7f68981f6310 2023-05-20T01:16:28.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.068+0000 7f688dffb700 1 -- 172.21.15.16:0/2122153397 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f689400fa40 con 0x7f68981f6310 2023-05-20T01:16:28.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.068+0000 7f689fa2f700 1 -- 172.21.15.16:0/2122153397 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6898071e80 con 0x7f68981f6310 2023-05-20T01:16:28.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.068+0000 7f688dffb700 1 -- 172.21.15.16:0/2122153397 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f689400fdf0 con 0x7f68981f6310 2023-05-20T01:16:28.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.068+0000 7f689fa2f700 1 -- 172.21.15.16:0/2122153397 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68980ce5b0 con 0x7f68981f6310 2023-05-20T01:16:28.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.069+0000 7f689fa2f700 1 -- 172.21.15.16:0/2122153397 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68980747b0 con 0x7f68981f6310 2023-05-20T01:16:28.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.069+0000 7f688dffb700 1 -- 172.21.15.16:0/2122153397 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f689401d460 con 0x7f68981f6310 2023-05-20T01:16:28.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.069+0000 7f688dffb700 1 --2- 172.21.15.16:0/2122153397 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f687c06ace0 0x7f687c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:28.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.070+0000 7f688dffb700 1 -- 172.21.15.16:0/2122153397 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5962+0+0 (secure 0 0 0) 0x7f6894089520 con 0x7f68981f6310 2023-05-20T01:16:28.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.070+0000 7f689d7cb700 1 --2- 172.21.15.16:0/2122153397 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f687c06ace0 0x7f687c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:28.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.070+0000 7f689d7cb700 1 --2- 172.21.15.16:0/2122153397 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f687c06ace0 0x7f687c06d190 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6888009b80 tx=0x7f6888008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:28.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.074+0000 7f688dffb700 1 -- 172.21.15.16:0/2122153397 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f6894054630 con 0x7f68981f6310 2023-05-20T01:16:28.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.314+0000 7f689fa2f700 1 -- 172.21.15.16:0/2122153397 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f68980b5f00 con 0x7f68981f6310 2023-05-20T01:16:28.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.320+0000 7f688dffb700 1 -- 172.21.15.16:0/2122153397 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 3 and data pool 4 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f6894057c50 con 0x7f68981f6310 2023-05-20T01:16:28.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.333+0000 7f689fa2f700 1 -- 172.21.15.16:0/2122153397 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f687c06ace0 msgr2=0x7f687c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:28.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.333+0000 7f689fa2f700 1 --2- 172.21.15.16:0/2122153397 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f687c06ace0 0x7f687c06d190 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6888009b80 tx=0x7f6888008040 comp rx=0 tx=0).stop 2023-05-20T01:16:28.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.333+0000 7f689fa2f700 1 -- 172.21.15.16:0/2122153397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68981f6310 msgr2=0x7f68980c8140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:28.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.333+0000 7f689fa2f700 1 --2- 172.21.15.16:0/2122153397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68981f6310 0x7f68980c8140 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6894007600 tx=0x7f689400f4b0 comp rx=0 tx=0).stop 2023-05-20T01:16:28.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.333+0000 7f689fa2f700 1 -- 172.21.15.16:0/2122153397 shutdown_connections 2023-05-20T01:16:28.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.333+0000 7f689fa2f700 1 --2- 172.21.15.16:0/2122153397 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f68981f6780 0x7f68981f3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:28.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.333+0000 7f689fa2f700 1 --2- 172.21.15.16:0/2122153397 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f687c06ace0 0x7f687c06d190 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:28.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.333+0000 7f689fa2f700 1 --2- 172.21.15.16:0/2122153397 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6884004d90 0x7f68980c3c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:28.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.333+0000 7f689fa2f700 1 --2- 172.21.15.16:0/2122153397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68981f6310 0x7f68980c8140 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:28.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.333+0000 7f689fa2f700 1 -- 172.21.15.16:0/2122153397 >> 172.21.15.16:0/2122153397 conn(0x7f68980cffd0 msgr2=0x7f68980b2270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:28.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.333+0000 7f689fa2f700 1 -- 172.21.15.16:0/2122153397 shutdown_connections 2023-05-20T01:16:28.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.333+0000 7f689fa2f700 1 -- 172.21.15.16:0/2122153397 wait complete. 2023-05-20T01:16:28.339 INFO:teuthology.orchestra.run.smithi016.stderr:new fs with metadata pool 3 and data pool 4 2023-05-20T01:16:28.359 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2023-05-20T01:16:28.359 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2023-05-20T01:16:28.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.486+0000 7fe001c63700 1 Processor -- start 2023-05-20T01:16:28.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.486+0000 7fe001c63700 1 -- start start 2023-05-20T01:16:28.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.487+0000 7fe001c63700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fdffc06aad0 0x7fdffc06dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:28.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.487+0000 7fe001c63700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fdffc06aea0 0x7fdffc06e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:28.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.487+0000 7fe001c63700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdffc161440 con 0x7fdffc0b1b90 2023-05-20T01:16:28.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.487+0000 7fe001c63700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdffc0b28c0 con 0x7fdffc06aad0 2023-05-20T01:16:28.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.487+0000 7fe001c63700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdffc160680 con 0x7fdffc06aea0 2023-05-20T01:16:28.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.487+0000 7fdffb7fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fdffc06aad0 0x7fdffc06dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:28.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.487+0000 7fdffb7fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fdffc06aad0 0x7fdffc06dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57496/0 (socket says 172.21.15.16:57496) 2023-05-20T01:16:28.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.487+0000 7fdffb7fe700 1 -- 172.21.15.16:0/153489429 learned_addr learned my addr 172.21.15.16:0/153489429 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:28.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.487+0000 7fdffaffd700 1 --2- 172.21.15.16:0/153489429 >> v2:172.21.15.132:3300/0 conn(0x7fdffc06aea0 0x7fdffc06e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:28.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.488+0000 7fdffb7fe700 1 -- 172.21.15.16:0/153489429 >> v2:172.21.15.132:3300/0 conn(0x7fdffc06aea0 msgr2=0x7fdffc06e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:28.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.488+0000 7fdffb7fe700 1 --2- 172.21.15.16:0/153489429 >> v2:172.21.15.132:3300/0 conn(0x7fdffc06aea0 0x7fdffc06e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:28.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.488+0000 7fdffb7fe700 1 -- 172.21.15.16:0/153489429 >> v1:172.21.15.132:6789/0 conn(0x7fdffc0b1b90 legacy=0x7fdffc0b2010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:28.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.488+0000 7fdffb7fe700 1 -- 172.21.15.16:0/153489429 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdffc160960 con 0x7fdffc06aad0 2023-05-20T01:16:28.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.488+0000 7fdffb7fe700 1 --2- 172.21.15.16:0/153489429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdffc06aad0 0x7fdffc06dca0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fdffc070f10 tx=0x7fdfe4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca749517ae017f73 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:28.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.488+0000 7fdffa7fc700 1 -- 172.21.15.16:0/153489429 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe4011830 con 0x7fdffc06aad0 2023-05-20T01:16:28.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.488+0000 7fdffa7fc700 1 -- 172.21.15.16:0/153489429 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfe4011990 con 0x7fdffc06aad0 2023-05-20T01:16:28.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.488+0000 7fdffa7fc700 1 -- 172.21.15.16:0/153489429 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe4011d20 con 0x7fdffc06aad0 2023-05-20T01:16:28.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.488+0000 7fe001c63700 1 -- 172.21.15.16:0/153489429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdffc06aad0 msgr2=0x7fdffc06dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:28.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.488+0000 7fe001c63700 1 --2- 172.21.15.16:0/153489429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdffc06aad0 0x7fdffc06dca0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fdffc070f10 tx=0x7fdfe4009bd0 comp rx=0 tx=0).stop 2023-05-20T01:16:28.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.489+0000 7fe001c63700 1 -- 172.21.15.16:0/153489429 shutdown_connections 2023-05-20T01:16:28.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.489+0000 7fe001c63700 1 --2- 172.21.15.16:0/153489429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdffc06aad0 0x7fdffc06dca0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:28.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.489+0000 7fe001c63700 1 --2- 172.21.15.16:0/153489429 >> v2:172.21.15.132:3300/0 conn(0x7fdffc06aea0 0x7fdffc06e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:28.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.489+0000 7fe001c63700 1 -- 172.21.15.16:0/153489429 >> 172.21.15.16:0/153489429 conn(0x7fdffc0ccab0 msgr2=0x7fdffc0cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:28.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.489+0000 7fe001c63700 1 -- 172.21.15.16:0/153489429 shutdown_connections 2023-05-20T01:16:28.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.489+0000 7fe001c63700 1 -- 172.21.15.16:0/153489429 wait complete. 2023-05-20T01:16:28.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.490+0000 7fe001c63700 1 Processor -- start 2023-05-20T01:16:28.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.490+0000 7fe001c63700 1 -- start start 2023-05-20T01:16:28.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.491+0000 7fe001c63700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdffc06aad0 0x7fdffc0c4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:28.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.491+0000 7fe001c63700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdffc06aea0 0x7fdffc152710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:28.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.491+0000 7fe001c63700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdffc0b1b90 0x7fdffc0addb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:28.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.491+0000 7fe001c63700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdffc152c50 con 0x7fdffc06aad0 2023-05-20T01:16:28.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.491+0000 7fe001c63700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdffc154080 con 0x7fdffc0b1b90 2023-05-20T01:16:28.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.491+0000 7fe001c63700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdffc154360 con 0x7fdffc06aea0 2023-05-20T01:16:28.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.491+0000 7fdffb7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdffc06aad0 0x7fdffc0c4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:28.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.491+0000 7fdffb7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdffc06aad0 0x7fdffc0c4090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57502/0 (socket says 172.21.15.16:57502) 2023-05-20T01:16:28.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.491+0000 7fdffb7fe700 1 -- 172.21.15.16:0/4170125556 learned_addr learned my addr 172.21.15.16:0/4170125556 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:28.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.491+0000 7fdffbfff700 1 --2- 172.21.15.16:0/4170125556 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdffc0b1b90 0x7fdffc0addb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:28.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.491+0000 7fdffb7fe700 1 -- 172.21.15.16:0/4170125556 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdffc06aea0 msgr2=0x7fdffc152710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:28.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.491+0000 7fdffb7fe700 1 --2- 172.21.15.16:0/4170125556 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdffc06aea0 0x7fdffc152710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:28.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.491+0000 7fdffb7fe700 1 -- 172.21.15.16:0/4170125556 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdffc0b1b90 msgr2=0x7fdffc0addb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:28.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.491+0000 7fdffb7fe700 1 --2- 172.21.15.16:0/4170125556 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdffc0b1b90 0x7fdffc0addb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:28.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.491+0000 7fdffb7fe700 1 -- 172.21.15.16:0/4170125556 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfe4009550 con 0x7fdffc06aad0 2023-05-20T01:16:28.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.491+0000 7fdffb7fe700 1 --2- 172.21.15.16:0/4170125556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdffc06aad0 0x7fdffc0c4090 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fdfe4000c00 tx=0x7fdfe4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:28.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.491+0000 7fdff3fff700 1 -- 172.21.15.16:0/4170125556 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe4004570 con 0x7fdffc06aad0 2023-05-20T01:16:28.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.491+0000 7fdff3fff700 1 -- 172.21.15.16:0/4170125556 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfe401f070 con 0x7fdffc06aad0 2023-05-20T01:16:28.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.491+0000 7fe001c63700 1 -- 172.21.15.16:0/4170125556 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdffc160f20 con 0x7fdffc06aad0 2023-05-20T01:16:28.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.491+0000 7fdff3fff700 1 -- 172.21.15.16:0/4170125556 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe40038e0 con 0x7fdffc06aad0 2023-05-20T01:16:28.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.491+0000 7fe001c63700 1 -- 172.21.15.16:0/4170125556 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdffc160c40 con 0x7fdffc06aad0 2023-05-20T01:16:28.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.492+0000 7fe001c63700 1 -- 172.21.15.16:0/4170125556 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdffc06cb80 con 0x7fdffc06aad0 2023-05-20T01:16:28.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.493+0000 7fdff3fff700 1 -- 172.21.15.16:0/4170125556 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fdfe4021030 con 0x7fdffc06aad0 2023-05-20T01:16:28.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.493+0000 7fdff3fff700 1 --2- 172.21.15.16:0/4170125556 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fdfdc06ace0 0x7fdfdc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:28.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.493+0000 7fdff3fff700 1 -- 172.21.15.16:0/4170125556 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fdfe408a910 con 0x7fdffc06aad0 2023-05-20T01:16:28.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.493+0000 7fdffaffd700 1 --2- 172.21.15.16:0/4170125556 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fdfdc06ace0 0x7fdfdc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:28.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.494+0000 7fdffaffd700 1 --2- 172.21.15.16:0/4170125556 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fdfdc06ace0 0x7fdfdc06d190 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fdfec006b20 tx=0x7fdfec006ab0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:28.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.498+0000 7fdff3fff700 1 -- 172.21.15.16:0/4170125556 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fdfe4055ba0 con 0x7fdffc06aad0 2023-05-20T01:16:28.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:28.705+0000 7fe001c63700 1 -- 172.21.15.16:0/4170125556 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7fdffc0747b0 con 0x7fdffc06aad0 2023-05-20T01:16:29.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:29.322+0000 7fdff3fff700 1 -- 172.21.15.16:0/4170125556 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v21) v1 ==== 157+0+0 (secure 0 0 0) 0x7fdfe40150e0 con 0x7fdffc06aad0 2023-05-20T01:16:29.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:29.325+0000 7fe001c63700 1 -- 172.21.15.16:0/4170125556 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fdfdc06ace0 msgr2=0x7fdfdc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:29.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:29.325+0000 7fe001c63700 1 --2- 172.21.15.16:0/4170125556 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fdfdc06ace0 0x7fdfdc06d190 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fdfec006b20 tx=0x7fdfec006ab0 comp rx=0 tx=0).stop 2023-05-20T01:16:29.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:29.325+0000 7fe001c63700 1 -- 172.21.15.16:0/4170125556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdffc06aad0 msgr2=0x7fdffc0c4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:29.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:29.325+0000 7fe001c63700 1 --2- 172.21.15.16:0/4170125556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdffc06aad0 0x7fdffc0c4090 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fdfe4000c00 tx=0x7fdfe4004750 comp rx=0 tx=0).stop 2023-05-20T01:16:29.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:29.325+0000 7fe001c63700 1 -- 172.21.15.16:0/4170125556 shutdown_connections 2023-05-20T01:16:29.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:29.325+0000 7fe001c63700 1 --2- 172.21.15.16:0/4170125556 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdffc0b1b90 0x7fdffc0addb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:29.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:29.325+0000 7fe001c63700 1 --2- 172.21.15.16:0/4170125556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdffc06aad0 0x7fdffc0c4090 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:29.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:29.325+0000 7fe001c63700 1 --2- 172.21.15.16:0/4170125556 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fdfdc06ace0 0x7fdfdc06d190 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:29.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:29.325+0000 7fe001c63700 1 --2- 172.21.15.16:0/4170125556 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdffc06aea0 0x7fdffc152710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:29.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:29.325+0000 7fe001c63700 1 -- 172.21.15.16:0/4170125556 >> 172.21.15.16:0/4170125556 conn(0x7fdffc0ccab0 msgr2=0x7fdffc0b1210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:29.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:29.325+0000 7fe001c63700 1 -- 172.21.15.16:0/4170125556 shutdown_connections 2023-05-20T01:16:29.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:29.325+0000 7fe001c63700 1 -- 172.21.15.16:0/4170125556 wait complete. 2023-05-20T01:16:29.344 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2023-05-20T01:16:29.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:29.480+0000 7f1b85ae8700 1 Processor -- start 2023-05-20T01:16:29.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:29.480+0000 7f1b85ae8700 1 -- start start 2023-05-20T01:16:29.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:29.481+0000 7f1b85ae8700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1b800cd1c0 0x7f1b800710b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:29.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:29.481+0000 7f1b85ae8700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1b800cd630 0x7f1b800715f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:29.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:29.481+0000 7f1b85ae8700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b800ccd30 con 0x7f1b80159b50 2023-05-20T01:16:29.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:29.481+0000 7f1b85ae8700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b800ce990 con 0x7f1b800cd630 2023-05-20T01:16:29.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:29.481+0000 7f1b85ae8700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b800cc6e0 con 0x7f1b800cd1c0 2023-05-20T01:16:29.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:29.481+0000 7f1b7ffff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1b800cd630 0x7f1b800715f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:29.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:29.481+0000 7f1b7f7fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f1b80159b50 0x7f1b8006e0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:40806/0 (socket says 172.21.15.16:40806) 2023-05-20T01:16:29.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:29.481+0000 7f1b7f7fe700 1 -- 172.21.15.16:0/863889860 learned_addr learned my addr 172.21.15.16:0/863889860 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:16:29.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:29.481+0000 7f1b7effd700 1 --2- 172.21.15.16:0/863889860 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1b800cd1c0 0x7f1b800710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:29.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:29.482+0000 7f1b7ffff700 1 -- 172.21.15.16:0/863889860 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1b800cd1c0 msgr2=0x7f1b800710b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:29.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:29.482+0000 7f1b7ffff700 1 --2- 172.21.15.16:0/863889860 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1b800cd1c0 0x7f1b800710b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:29.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:29.482+0000 7f1b7ffff700 1 -- 172.21.15.16:0/863889860 >> v1:172.21.15.16:6789/0 conn(0x7f1b80159b50 legacy=0x7f1b8006e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:29.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:29.482+0000 7f1b7ffff700 1 -- 172.21.15.16:0/863889860 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b800cc9c0 con 0x7f1b800cd630 2023-05-20T01:16:29.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:29.482+0000 7f1b7e7fc700 1 -- 172.21.15.16:0/863889860 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3224652417 0 0) 0x7f1b800ccd30 con 0x7f1b80159b50 2023-05-20T01:16:29.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:29.482+0000 7f1b7ffff700 1 --2- 172.21.15.16:0/863889860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b800cd630 0x7f1b800715f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1b78009d90 tx=0x7f1b7800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd4759c13991cc5f server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:29.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:29.482+0000 7f1b7e7fc700 1 -- 172.21.15.16:0/863889860 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b7800e070 con 0x7f1b800cd630 2023-05-20T01:16:29.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:29.482+0000 7f1b7e7fc700 1 -- 172.21.15.16:0/863889860 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b7800bbd0 con 0x7f1b800cd630 2023-05-20T01:16:29.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:29.482+0000 7f1b7e7fc700 1 -- 172.21.15.16:0/863889860 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b78012430 con 0x7f1b800cd630 2023-05-20T01:16:29.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:29.482+0000 7f1b85ae8700 1 -- 172.21.15.16:0/863889860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b800cd630 msgr2=0x7f1b800715f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:29.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:29.482+0000 7f1b85ae8700 1 --2- 172.21.15.16:0/863889860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b800cd630 0x7f1b800715f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1b78009d90 tx=0x7f1b7800b330 comp rx=0 tx=0).stop 2023-05-20T01:16:29.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:29.482+0000 7f1b85ae8700 1 -- 172.21.15.16:0/863889860 shutdown_connections 2023-05-20T01:16:29.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:29.482+0000 7f1b85ae8700 1 --2- 172.21.15.16:0/863889860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b800cd630 0x7f1b800715f0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:29.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:29.482+0000 7f1b85ae8700 1 --2- 172.21.15.16:0/863889860 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1b800cd1c0 0x7f1b800710b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:29.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:29.482+0000 7f1b85ae8700 1 -- 172.21.15.16:0/863889860 >> 172.21.15.16:0/863889860 conn(0x7f1b800cffd0 msgr2=0x7f1b800d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:29.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:29.483+0000 7f1b85ae8700 1 -- 172.21.15.16:0/863889860 shutdown_connections 2023-05-20T01:16:29.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:29.483+0000 7f1b85ae8700 1 -- 172.21.15.16:0/863889860 wait complete. 2023-05-20T01:16:29.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:29.484+0000 7f1b85ae8700 1 Processor -- start 2023-05-20T01:16:29.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:29.484+0000 7f1b85ae8700 1 -- start start 2023-05-20T01:16:29.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:29.485+0000 7f1b85ae8700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1b800cd1c0 0x7f1b8015a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:29.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:29.485+0000 7f1b85ae8700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1b800cd630 0x7f1b8014b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:29.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:29.485+0000 7f1b85ae8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b80159b50 0x7f1b80156630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:29.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:29.485+0000 7f1b85ae8700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b8015a8c0 con 0x7f1b80159b50 2023-05-20T01:16:29.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:29.485+0000 7f1b85ae8700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b800c7140 con 0x7f1b800cd1c0 2023-05-20T01:16:29.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:29.485+0000 7f1b85ae8700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b800c73e0 con 0x7f1b800cd630 2023-05-20T01:16:29.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:29.485+0000 7f1b7ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b80159b50 0x7f1b80156630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:29.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:29.485+0000 7f1b7ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b80159b50 0x7f1b80156630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57520/0 (socket says 172.21.15.16:57520) 2023-05-20T01:16:29.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:29.485+0000 7f1b7ffff700 1 -- 172.21.15.16:0/450148888 learned_addr learned my addr 172.21.15.16:0/450148888 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:29.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:29.485+0000 7f1b7effd700 1 --2- 172.21.15.16:0/450148888 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1b800cd630 0x7f1b8014b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:29.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:29.485+0000 7f1b7f7fe700 1 --2- 172.21.15.16:0/450148888 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1b800cd1c0 0x7f1b8015a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:29.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:29.485+0000 7f1b7ffff700 1 -- 172.21.15.16:0/450148888 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1b800cd630 msgr2=0x7f1b8014b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:29.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:29.485+0000 7f1b7ffff700 1 --2- 172.21.15.16:0/450148888 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1b800cd630 0x7f1b8014b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:29.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:29.485+0000 7f1b7ffff700 1 -- 172.21.15.16:0/450148888 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1b800cd1c0 msgr2=0x7f1b8015a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:29.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:29.485+0000 7f1b7ffff700 1 --2- 172.21.15.16:0/450148888 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1b800cd1c0 0x7f1b8015a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:29.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:29.485+0000 7f1b7ffff700 1 -- 172.21.15.16:0/450148888 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b78009a70 con 0x7f1b80159b50 2023-05-20T01:16:29.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:29.485+0000 7f1b7ffff700 1 --2- 172.21.15.16:0/450148888 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b80159b50 0x7f1b80156630 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1b800675d0 tx=0x7f1b78004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:29.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:29.485+0000 7f1b77fff700 1 -- 172.21.15.16:0/450148888 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b78012430 con 0x7f1b80159b50 2023-05-20T01:16:29.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:29.485+0000 7f1b77fff700 1 -- 172.21.15.16:0/450148888 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b78020070 con 0x7f1b80159b50 2023-05-20T01:16:29.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:29.485+0000 7f1b77fff700 1 -- 172.21.15.16:0/450148888 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b78004260 con 0x7f1b80159b50 2023-05-20T01:16:29.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:29.485+0000 7f1b85ae8700 1 -- 172.21.15.16:0/450148888 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b800cec70 con 0x7f1b80159b50 2023-05-20T01:16:29.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:29.485+0000 7f1b85ae8700 1 -- 172.21.15.16:0/450148888 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b80071e80 con 0x7f1b80159b50 2023-05-20T01:16:29.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:29.486+0000 7f1b85ae8700 1 -- 172.21.15.16:0/450148888 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b800747b0 con 0x7f1b80159b50 2023-05-20T01:16:29.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:29.487+0000 7f1b77fff700 1 -- 172.21.15.16:0/450148888 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1b780043c0 con 0x7f1b80159b50 2023-05-20T01:16:29.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:29.487+0000 7f1b77fff700 1 --2- 172.21.15.16:0/450148888 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f1b6006ace0 0x7f1b6006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:29.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:29.487+0000 7f1b77fff700 1 -- 172.21.15.16:0/450148888 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6288+0+0 (secure 0 0 0) 0x7f1b78022070 con 0x7f1b80159b50 2023-05-20T01:16:29.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:29.487+0000 7f1b7f7fe700 1 --2- 172.21.15.16:0/450148888 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f1b6006ace0 0x7f1b6006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:29.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:29.488+0000 7f1b7f7fe700 1 --2- 172.21.15.16:0/450148888 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f1b6006ace0 0x7f1b6006d190 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1b800ca110 tx=0x7f1b70009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:29.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:29.492+0000 7f1b77fff700 1 -- 172.21.15.16:0/450148888 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f1b780563e0 con 0x7f1b80159b50 2023-05-20T01:16:29.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:29.689+0000 7f1b85ae8700 1 -- 172.21.15.16:0/450148888 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f1b800b4490 con 0x7f1b80159b50 2023-05-20T01:16:31.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:31.328+0000 7f1b77fff700 1 -- 172.21.15.16:0/450148888 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v23) v1 ==== 228+0+0 (secure 0 0 0) 0x7f1b78059a00 con 0x7f1b80159b50 2023-05-20T01:16:31.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:31.339+0000 7f1b85ae8700 1 -- 172.21.15.16:0/450148888 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f1b6006ace0 msgr2=0x7f1b6006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:31.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:31.339+0000 7f1b85ae8700 1 --2- 172.21.15.16:0/450148888 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f1b6006ace0 0x7f1b6006d190 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1b800ca110 tx=0x7f1b70009040 comp rx=0 tx=0).stop 2023-05-20T01:16:31.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:31.339+0000 7f1b85ae8700 1 -- 172.21.15.16:0/450148888 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b80159b50 msgr2=0x7f1b80156630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:31.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:31.339+0000 7f1b85ae8700 1 --2- 172.21.15.16:0/450148888 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b80159b50 0x7f1b80156630 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1b800675d0 tx=0x7f1b78004ad0 comp rx=0 tx=0).stop 2023-05-20T01:16:31.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:31.339+0000 7f1b85ae8700 1 -- 172.21.15.16:0/450148888 shutdown_connections 2023-05-20T01:16:31.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:31.339+0000 7f1b85ae8700 1 --2- 172.21.15.16:0/450148888 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b80159b50 0x7f1b80156630 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:31.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:31.339+0000 7f1b85ae8700 1 --2- 172.21.15.16:0/450148888 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1b800cd1c0 0x7f1b8015a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:31.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:31.339+0000 7f1b85ae8700 1 --2- 172.21.15.16:0/450148888 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f1b6006ace0 0x7f1b6006d190 secure :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1b800ca110 tx=0x7f1b70009040 comp rx=0 tx=0).stop 2023-05-20T01:16:31.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:31.339+0000 7f1b85ae8700 1 --2- 172.21.15.16:0/450148888 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1b800cd630 0x7f1b8014b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:31.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:31.339+0000 7f1b85ae8700 1 -- 172.21.15.16:0/450148888 >> 172.21.15.16:0/450148888 conn(0x7f1b800cffd0 msgr2=0x7f1b800cbc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:31.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:31.341+0000 7f1b85ae8700 1 -- 172.21.15.16:0/450148888 shutdown_connections 2023-05-20T01:16:31.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:31.341+0000 7f1b85ae8700 1 -- 172.21.15.16:0/450148888 wait complete. 2023-05-20T01:16:31.347 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data_ec' created 2023-05-20T01:16:31.361 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2023-05-20T01:16:31.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:31.491+0000 7fcd81727700 1 Processor -- start 2023-05-20T01:16:31.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:31.491+0000 7fcd81727700 1 -- start start 2023-05-20T01:16:31.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:31.492+0000 7fcd81727700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcd7c0d0cd0 0x7fcd7c0b6930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:31.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:31.492+0000 7fcd81727700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcd7c0b6e70 0x7fcd7c0cb7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:31.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:31.492+0000 7fcd81727700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd7c0b64a0 con 0x7fcd7c0cefe0 2023-05-20T01:16:31.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:31.492+0000 7fcd81727700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd7c0b59c0 con 0x7fcd7c0b6e70 2023-05-20T01:16:31.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:31.492+0000 7fcd81727700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7c0b2540 con 0x7fcd7c0d0cd0 2023-05-20T01:16:31.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:31.492+0000 7fcd7b7fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcd7c0b6e70 0x7fcd7c0cb7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:31.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:31.492+0000 7fcd7b7fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcd7c0b6e70 0x7fcd7c0cb7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57528/0 (socket says 172.21.15.16:57528) 2023-05-20T01:16:31.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:31.492+0000 7fcd7b7fe700 1 -- 172.21.15.16:0/2702527079 learned_addr learned my addr 172.21.15.16:0/2702527079 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:31.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:31.492+0000 7fcd7a7fc700 1 --2- 172.21.15.16:0/2702527079 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcd7c0d0cd0 0x7fcd7c0b6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:31.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:31.493+0000 7fcd7b7fe700 1 -- 172.21.15.16:0/2702527079 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcd7c0d0cd0 msgr2=0x7fcd7c0b6930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:31.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:31.493+0000 7fcd7b7fe700 1 --2- 172.21.15.16:0/2702527079 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcd7c0d0cd0 0x7fcd7c0b6930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:31.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:31.493+0000 7fcd7b7fe700 1 -- 172.21.15.16:0/2702527079 >> v1:172.21.15.132:6789/0 conn(0x7fcd7c0cefe0 legacy=0x7fcd7c0b1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:31.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:31.493+0000 7fcd7b7fe700 1 -- 172.21.15.16:0/2702527079 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd7c0b5bb0 con 0x7fcd7c0b6e70 2023-05-20T01:16:31.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:31.493+0000 7fcd7b7fe700 1 --2- 172.21.15.16:0/2702527079 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd7c0b6e70 0x7fcd7c0cb7f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fcd74009e50 tx=0x7fcd7400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=801a55e5819393c9 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:31.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:31.493+0000 7fcd79ffb700 1 -- 172.21.15.16:0/2702527079 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd7400e070 con 0x7fcd7c0b6e70 2023-05-20T01:16:31.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:31.493+0000 7fcd79ffb700 1 -- 172.21.15.16:0/2702527079 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd7400b9c0 con 0x7fcd7c0b6e70 2023-05-20T01:16:31.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:31.493+0000 7fcd79ffb700 1 -- 172.21.15.16:0/2702527079 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd7400bd70 con 0x7fcd7c0b6e70 2023-05-20T01:16:31.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:31.493+0000 7fcd81727700 1 -- 172.21.15.16:0/2702527079 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd7c0b6e70 msgr2=0x7fcd7c0cb7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:31.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:31.493+0000 7fcd81727700 1 --2- 172.21.15.16:0/2702527079 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd7c0b6e70 0x7fcd7c0cb7f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fcd74009e50 tx=0x7fcd7400b330 comp rx=0 tx=0).stop 2023-05-20T01:16:31.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:31.494+0000 7fcd81727700 1 -- 172.21.15.16:0/2702527079 shutdown_connections 2023-05-20T01:16:31.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:31.494+0000 7fcd81727700 1 --2- 172.21.15.16:0/2702527079 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd7c0b6e70 0x7fcd7c0cb7f0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:31.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:31.494+0000 7fcd81727700 1 --2- 172.21.15.16:0/2702527079 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcd7c0d0cd0 0x7fcd7c0b6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:31.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:31.494+0000 7fcd81727700 1 -- 172.21.15.16:0/2702527079 >> 172.21.15.16:0/2702527079 conn(0x7fcd7c09a470 msgr2=0x7fcd7c0d11a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:31.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:31.494+0000 7fcd81727700 1 -- 172.21.15.16:0/2702527079 shutdown_connections 2023-05-20T01:16:31.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:31.494+0000 7fcd81727700 1 -- 172.21.15.16:0/2702527079 wait complete. 2023-05-20T01:16:31.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:31.495+0000 7fcd81727700 1 Processor -- start 2023-05-20T01:16:31.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:31.495+0000 7fcd81727700 1 -- start start 2023-05-20T01:16:31.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:31.496+0000 7fcd81727700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcd7c0b6e70 0x7fcd7c14b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:31.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:31.496+0000 7fcd81727700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcd7c0cefe0 0x7fcd7c0c5e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:31.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:31.496+0000 7fcd81727700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd7c0d0cd0 0x7fcd7c14c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:31.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:31.496+0000 7fcd81727700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7c1e44c0 con 0x7fcd7c0d0cd0 2023-05-20T01:16:31.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:31.496+0000 7fcd81727700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7c14cb10 con 0x7fcd7c0cefe0 2023-05-20T01:16:31.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:31.496+0000 7fcd81727700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7c1f0370 con 0x7fcd7c0b6e70 2023-05-20T01:16:31.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:31.496+0000 7fcd7b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd7c0d0cd0 0x7fcd7c14c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:31.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:31.496+0000 7fcd7b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd7c0d0cd0 0x7fcd7c14c230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57534/0 (socket says 172.21.15.16:57534) 2023-05-20T01:16:31.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:31.496+0000 7fcd7b7fe700 1 -- 172.21.15.16:0/2676141999 learned_addr learned my addr 172.21.15.16:0/2676141999 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:31.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:31.496+0000 7fcd7a7fc700 1 --2- 172.21.15.16:0/2676141999 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcd7c0cefe0 0x7fcd7c0c5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:31.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:31.496+0000 7fcd7affd700 1 --2- 172.21.15.16:0/2676141999 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcd7c0b6e70 0x7fcd7c14b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:31.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:31.496+0000 7fcd7b7fe700 1 -- 172.21.15.16:0/2676141999 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcd7c0b6e70 msgr2=0x7fcd7c14b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:31.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:31.496+0000 7fcd7b7fe700 1 --2- 172.21.15.16:0/2676141999 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcd7c0b6e70 0x7fcd7c14b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:31.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:31.496+0000 7fcd7b7fe700 1 -- 172.21.15.16:0/2676141999 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcd7c0cefe0 msgr2=0x7fcd7c0c5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:31.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:31.496+0000 7fcd7b7fe700 1 --2- 172.21.15.16:0/2676141999 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcd7c0cefe0 0x7fcd7c0c5e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:31.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:31.496+0000 7fcd7b7fe700 1 -- 172.21.15.16:0/2676141999 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd74009b30 con 0x7fcd7c0d0cd0 2023-05-20T01:16:31.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:31.496+0000 7fcd7affd700 1 --2- 172.21.15.16:0/2676141999 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcd7c0b6e70 0x7fcd7c14b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:16:31.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:31.496+0000 7fcd7a7fc700 1 --2- 172.21.15.16:0/2676141999 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcd7c0cefe0 0x7fcd7c0c5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:16:31.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:31.496+0000 7fcd7b7fe700 1 --2- 172.21.15.16:0/2676141999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd7c0d0cd0 0x7fcd7c14c230 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c0716f0 tx=0x7fcd74004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:31.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:31.496+0000 7fcd737fe700 1 -- 172.21.15.16:0/2676141999 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd74004960 con 0x7fcd7c0d0cd0 2023-05-20T01:16:31.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:31.496+0000 7fcd737fe700 1 -- 172.21.15.16:0/2676141999 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd740099b0 con 0x7fcd7c0d0cd0 2023-05-20T01:16:31.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:31.496+0000 7fcd737fe700 1 -- 172.21.15.16:0/2676141999 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd7401c680 con 0x7fcd7c0d0cd0 2023-05-20T01:16:31.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:31.496+0000 7fcd81727700 1 -- 172.21.15.16:0/2676141999 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd7c0b6130 con 0x7fcd7c0d0cd0 2023-05-20T01:16:31.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:31.497+0000 7fcd81727700 1 -- 172.21.15.16:0/2676141999 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd7c0b5e50 con 0x7fcd7c0d0cd0 2023-05-20T01:16:31.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:31.498+0000 7fcd81727700 1 -- 172.21.15.16:0/2676141999 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd7c0cf3b0 con 0x7fcd7c0d0cd0 2023-05-20T01:16:31.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:31.498+0000 7fcd737fe700 1 -- 172.21.15.16:0/2676141999 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fcd7401c7e0 con 0x7fcd7c0d0cd0 2023-05-20T01:16:31.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:31.498+0000 7fcd737fe700 1 --2- 172.21.15.16:0/2676141999 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fcd5c06ace0 0x7fcd5c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:31.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:31.498+0000 7fcd737fe700 1 -- 172.21.15.16:0/2676141999 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6781+0+0 (secure 0 0 0) 0x7fcd7408b3f0 con 0x7fcd7c0d0cd0 2023-05-20T01:16:31.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:31.498+0000 7fcd7affd700 1 --2- 172.21.15.16:0/2676141999 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fcd5c06ace0 0x7fcd5c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:31.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:31.499+0000 7fcd7affd700 1 --2- 172.21.15.16:0/2676141999 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fcd5c06ace0 0x7fcd5c06d190 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fcd64000f60 tx=0x7fcd64009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:31.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:31.503+0000 7fcd737fe700 1 -- 172.21.15.16:0/2676141999 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fcd74056260 con 0x7fcd7c0d0cd0 2023-05-20T01:16:31.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:31.699+0000 7fcd81727700 1 -- 172.21.15.16:0/2676141999 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7fcd7c1e4600 con 0x7fcd7c0d0cd0 2023-05-20T01:16:32.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.331+0000 7fcd737fe700 1 -- 172.21.15.16:0/2676141999 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 5 allow_ec_overwrites to true v24) v1 ==== 169+0+0 (secure 0 0 0) 0x7fcd7c1e4600 con 0x7fcd7c0d0cd0 2023-05-20T01:16:32.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.346+0000 7fcd81727700 1 -- 172.21.15.16:0/2676141999 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fcd5c06ace0 msgr2=0x7fcd5c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:32.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.346+0000 7fcd81727700 1 --2- 172.21.15.16:0/2676141999 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fcd5c06ace0 0x7fcd5c06d190 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fcd64000f60 tx=0x7fcd64009040 comp rx=0 tx=0).stop 2023-05-20T01:16:32.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.346+0000 7fcd81727700 1 -- 172.21.15.16:0/2676141999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd7c0d0cd0 msgr2=0x7fcd7c14c230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:32.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.346+0000 7fcd81727700 1 --2- 172.21.15.16:0/2676141999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd7c0d0cd0 0x7fcd7c14c230 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c0716f0 tx=0x7fcd74004ad0 comp rx=0 tx=0).stop 2023-05-20T01:16:32.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.347+0000 7fcd81727700 1 -- 172.21.15.16:0/2676141999 shutdown_connections 2023-05-20T01:16:32.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.347+0000 7fcd81727700 1 --2- 172.21.15.16:0/2676141999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd7c0d0cd0 0x7fcd7c14c230 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:32.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.347+0000 7fcd81727700 1 --2- 172.21.15.16:0/2676141999 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fcd5c06ace0 0x7fcd5c06d190 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:32.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.347+0000 7fcd81727700 1 --2- 172.21.15.16:0/2676141999 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcd7c0b6e70 0x7fcd7c14b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:32.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.347+0000 7fcd81727700 1 --2- 172.21.15.16:0/2676141999 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcd7c0cefe0 0x7fcd7c0c5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:32.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.347+0000 7fcd81727700 1 -- 172.21.15.16:0/2676141999 >> 172.21.15.16:0/2676141999 conn(0x7fcd7c09a470 msgr2=0x7fcd7c0ce4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:32.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.347+0000 7fcd81727700 1 -- 172.21.15.16:0/2676141999 shutdown_connections 2023-05-20T01:16:32.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.350+0000 7fcd81727700 1 -- 172.21.15.16:0/2676141999 wait complete. 2023-05-20T01:16:32.365 INFO:teuthology.orchestra.run.smithi016.stderr:set pool 5 allow_ec_overwrites to true 2023-05-20T01:16:32.378 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2023-05-20T01:16:32.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.505+0000 7f9d1814f700 1 Processor -- start 2023-05-20T01:16:32.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.506+0000 7f9d1814f700 1 -- start start 2023-05-20T01:16:32.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.507+0000 7f9d1814f700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f9d10159b50 0x7f9d1006e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:32.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.507+0000 7f9d1814f700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9d100cd1c0 0x7f9d100710b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:32.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.507+0000 7f9d1814f700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9d100cd630 0x7f9d100715f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:32.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.507+0000 7f9d1814f700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d100ccd30 con 0x7f9d100cd630 2023-05-20T01:16:32.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.507+0000 7f9d1814f700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d100ce990 con 0x7f9d10159b50 2023-05-20T01:16:32.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.507+0000 7f9d1814f700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d100cc6e0 con 0x7f9d100cd1c0 2023-05-20T01:16:32.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.507+0000 7f9d166ec700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9d100cd630 0x7f9d100715f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:32.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.507+0000 7f9d166ec700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9d100cd630 0x7f9d100715f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57542/0 (socket says 172.21.15.16:57542) 2023-05-20T01:16:32.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.507+0000 7f9d166ec700 1 -- 172.21.15.16:0/3832205607 learned_addr learned my addr 172.21.15.16:0/3832205607 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:32.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.507+0000 7f9d15eeb700 1 --2- 172.21.15.16:0/3832205607 >> v2:172.21.15.132:3300/0 conn(0x7f9d10159b50 0x7f9d1006e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:32.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.507+0000 7f9d166ec700 1 -- 172.21.15.16:0/3832205607 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9d100cd1c0 msgr2=0x7f9d100710b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:32.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.507+0000 7f9d156ea700 1 --2- 172.21.15.16:0/3832205607 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9d100cd1c0 0x7f9d100710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:32.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.507+0000 7f9d166ec700 1 --2- 172.21.15.16:0/3832205607 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9d100cd1c0 0x7f9d100710b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:32.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.507+0000 7f9d166ec700 1 -- 172.21.15.16:0/3832205607 >> v2:172.21.15.132:3300/0 conn(0x7f9d10159b50 msgr2=0x7f9d1006e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:32.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.507+0000 7f9d166ec700 1 --2- 172.21.15.16:0/3832205607 >> v2:172.21.15.132:3300/0 conn(0x7f9d10159b50 0x7f9d1006e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:32.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.507+0000 7f9d166ec700 1 -- 172.21.15.16:0/3832205607 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d100cc9c0 con 0x7f9d100cd630 2023-05-20T01:16:32.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.507+0000 7f9d15eeb700 1 --2- 172.21.15.16:0/3832205607 >> v2:172.21.15.132:3300/0 conn(0x7f9d10159b50 0x7f9d1006e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:16:32.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.507+0000 7f9d166ec700 1 --2- 172.21.15.16:0/3832205607 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d100cd630 0x7f9d100715f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9d10079820 tx=0x7f9d0800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d347f712e64ff74 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:32.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.507+0000 7f9d14ee9700 1 -- 172.21.15.16:0/3832205607 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d0800e070 con 0x7f9d100cd630 2023-05-20T01:16:32.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.507+0000 7f9d14ee9700 1 -- 172.21.15.16:0/3832205607 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d0800b9c0 con 0x7f9d100cd630 2023-05-20T01:16:32.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.507+0000 7f9d14ee9700 1 -- 172.21.15.16:0/3832205607 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d0800bd70 con 0x7f9d100cd630 2023-05-20T01:16:32.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.508+0000 7f9d1814f700 1 -- 172.21.15.16:0/3832205607 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d100cd630 msgr2=0x7f9d100715f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:32.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.508+0000 7f9d1814f700 1 --2- 172.21.15.16:0/3832205607 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d100cd630 0x7f9d100715f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9d10079820 tx=0x7f9d0800b330 comp rx=0 tx=0).stop 2023-05-20T01:16:32.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.508+0000 7f9d1814f700 1 -- 172.21.15.16:0/3832205607 shutdown_connections 2023-05-20T01:16:32.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.508+0000 7f9d1814f700 1 --2- 172.21.15.16:0/3832205607 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d100cd630 0x7f9d100715f0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:32.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.508+0000 7f9d1814f700 1 --2- 172.21.15.16:0/3832205607 >> v2:172.21.15.132:3300/0 conn(0x7f9d10159b50 0x7f9d1006e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:32.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.508+0000 7f9d1814f700 1 --2- 172.21.15.16:0/3832205607 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9d100cd1c0 0x7f9d100710b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:32.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.508+0000 7f9d1814f700 1 -- 172.21.15.16:0/3832205607 >> 172.21.15.16:0/3832205607 conn(0x7f9d100cffd0 msgr2=0x7f9d100d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:32.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.508+0000 7f9d1814f700 1 -- 172.21.15.16:0/3832205607 shutdown_connections 2023-05-20T01:16:32.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.508+0000 7f9d1814f700 1 -- 172.21.15.16:0/3832205607 wait complete. 2023-05-20T01:16:32.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.509+0000 7f9d1814f700 1 Processor -- start 2023-05-20T01:16:32.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.509+0000 7f9d1814f700 1 -- start start 2023-05-20T01:16:32.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.510+0000 7f9d1814f700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9d100cd1c0 0x7f9d1014b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:32.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.510+0000 7f9d1814f700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9d100cd630 0x7f9d100b3e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:32.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.510+0000 7f9d1814f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d10159b50 0x7f9d100c59f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:32.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.510+0000 7f9d1814f700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d100b4790 con 0x7f9d10159b50 2023-05-20T01:16:32.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.510+0000 7f9d1814f700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d100b4a70 con 0x7f9d100cd1c0 2023-05-20T01:16:32.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.510+0000 7f9d1814f700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d100c51f0 con 0x7f9d100cd630 2023-05-20T01:16:32.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.510+0000 7f9d166ec700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d10159b50 0x7f9d100c59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:32.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.510+0000 7f9d166ec700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d10159b50 0x7f9d100c59f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57548/0 (socket says 172.21.15.16:57548) 2023-05-20T01:16:32.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.510+0000 7f9d166ec700 1 -- 172.21.15.16:0/178647750 learned_addr learned my addr 172.21.15.16:0/178647750 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:32.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.510+0000 7f9d156ea700 1 --2- 172.21.15.16:0/178647750 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9d100cd630 0x7f9d100b3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:32.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.510+0000 7f9d15eeb700 1 --2- 172.21.15.16:0/178647750 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9d100cd1c0 0x7f9d1014b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:32.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.510+0000 7f9d166ec700 1 -- 172.21.15.16:0/178647750 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9d100cd630 msgr2=0x7f9d100b3e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:32.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.510+0000 7f9d166ec700 1 --2- 172.21.15.16:0/178647750 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9d100cd630 0x7f9d100b3e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:32.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.510+0000 7f9d166ec700 1 -- 172.21.15.16:0/178647750 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9d100cd1c0 msgr2=0x7f9d1014b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:32.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.510+0000 7f9d166ec700 1 --2- 172.21.15.16:0/178647750 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9d100cd1c0 0x7f9d1014b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:32.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.510+0000 7f9d166ec700 1 -- 172.21.15.16:0/178647750 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d08009b30 con 0x7f9d10159b50 2023-05-20T01:16:32.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.510+0000 7f9d166ec700 1 --2- 172.21.15.16:0/178647750 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d10159b50 0x7f9d100c59f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9d0800b370 tx=0x7f9d08004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:32.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.511+0000 7f9d067fc700 1 -- 172.21.15.16:0/178647750 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d080099b0 con 0x7f9d10159b50 2023-05-20T01:16:32.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.511+0000 7f9d067fc700 1 -- 172.21.15.16:0/178647750 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d08004250 con 0x7f9d10159b50 2023-05-20T01:16:32.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.511+0000 7f9d1814f700 1 -- 172.21.15.16:0/178647750 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d10071e80 con 0x7f9d10159b50 2023-05-20T01:16:32.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.511+0000 7f9d067fc700 1 -- 172.21.15.16:0/178647750 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d080045e0 con 0x7f9d10159b50 2023-05-20T01:16:32.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.511+0000 7f9d1814f700 1 -- 172.21.15.16:0/178647750 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d1006e5e0 con 0x7f9d10159b50 2023-05-20T01:16:32.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.512+0000 7f9d1814f700 1 -- 172.21.15.16:0/178647750 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d100cec70 con 0x7f9d10159b50 2023-05-20T01:16:32.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.512+0000 7f9d067fc700 1 -- 172.21.15.16:0/178647750 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9d08004740 con 0x7f9d10159b50 2023-05-20T01:16:32.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.512+0000 7f9d067fc700 1 --2- 172.21.15.16:0/178647750 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f9cf406ace0 0x7f9cf406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:32.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.513+0000 7f9d067fc700 1 -- 172.21.15.16:0/178647750 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6781+0+0 (secure 0 0 0) 0x7f9d0808ac90 con 0x7f9d10159b50 2023-05-20T01:16:32.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.513+0000 7f9d15eeb700 1 --2- 172.21.15.16:0/178647750 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f9cf406ace0 0x7f9cf406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:32.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.513+0000 7f9d15eeb700 1 --2- 172.21.15.16:0/178647750 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f9cf406ace0 0x7f9cf406d190 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9d100cfa00 tx=0x7f9d00008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:32.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.517+0000 7f9d067fc700 1 -- 172.21.15.16:0/178647750 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f9d08055a60 con 0x7f9d10159b50 2023-05-20T01:16:32.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.767+0000 7f9d1814f700 1 -- 172.21.15.16:0/178647750 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7f9d100747b0 con 0x7f9d10159b50 2023-05-20T01:16:32.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.771+0000 7f9d067fc700 1 -- 172.21.15.16:0/178647750 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 5 to fsmap v6) v1 ==== 137+0+0 (secure 0 0 0) 0x7f9d080200b0 con 0x7f9d10159b50 2023-05-20T01:16:32.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.773+0000 7f9d1814f700 1 -- 172.21.15.16:0/178647750 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f9cf406ace0 msgr2=0x7f9cf406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:32.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.773+0000 7f9d1814f700 1 --2- 172.21.15.16:0/178647750 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f9cf406ace0 0x7f9cf406d190 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9d100cfa00 tx=0x7f9d00008040 comp rx=0 tx=0).stop 2023-05-20T01:16:32.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.773+0000 7f9d1814f700 1 -- 172.21.15.16:0/178647750 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d10159b50 msgr2=0x7f9d100c59f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:32.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.773+0000 7f9d1814f700 1 --2- 172.21.15.16:0/178647750 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d10159b50 0x7f9d100c59f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9d0800b370 tx=0x7f9d08004ad0 comp rx=0 tx=0).stop 2023-05-20T01:16:32.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.773+0000 7f9d1814f700 1 -- 172.21.15.16:0/178647750 shutdown_connections 2023-05-20T01:16:32.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.773+0000 7f9d1814f700 1 --2- 172.21.15.16:0/178647750 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d10159b50 0x7f9d100c59f0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:32.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.773+0000 7f9d1814f700 1 --2- 172.21.15.16:0/178647750 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9d100cd1c0 0x7f9d1014b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:32.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.773+0000 7f9d1814f700 1 --2- 172.21.15.16:0/178647750 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f9cf406ace0 0x7f9cf406d190 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:32.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.773+0000 7f9d1814f700 1 --2- 172.21.15.16:0/178647750 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9d100cd630 0x7f9d100b3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:32.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.773+0000 7f9d1814f700 1 -- 172.21.15.16:0/178647750 >> 172.21.15.16:0/178647750 conn(0x7f9d100cffd0 msgr2=0x7f9d1006a510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:32.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.774+0000 7f9d1814f700 1 -- 172.21.15.16:0/178647750 shutdown_connections 2023-05-20T01:16:32.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.774+0000 7f9d1814f700 1 -- 172.21.15.16:0/178647750 wait complete. 2023-05-20T01:16:32.779 INFO:teuthology.orchestra.run.smithi016.stderr:added data pool 5 to fsmap 2023-05-20T01:16:32.789 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-20T01:16:32.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.910+0000 7f770b1c2700 1 Processor -- start 2023-05-20T01:16:32.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.910+0000 7f770b1c2700 1 -- start start 2023-05-20T01:16:32.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.911+0000 7f770b1c2700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f770415caf0 0x7f770406cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:32.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.911+0000 7f770b1c2700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f77040ccf00 0x7f770406d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:32.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.911+0000 7f770b1c2700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f770415cec0 con 0x7f77040b1e00 2023-05-20T01:16:32.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.911+0000 7f770b1c2700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77040d0490 con 0x7f770415caf0 2023-05-20T01:16:32.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.911+0000 7f770b1c2700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7704160610 con 0x7f77040ccf00 2023-05-20T01:16:32.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.911+0000 7f7708f5e700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f770415caf0 0x7f770406cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:32.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.911+0000 7f7708f5e700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f770415caf0 0x7f770406cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57552/0 (socket says 172.21.15.16:57552) 2023-05-20T01:16:32.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.911+0000 7f7708f5e700 1 -- 172.21.15.16:0/82668692 learned_addr learned my addr 172.21.15.16:0/82668692 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:32.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.911+0000 7f7703fff700 1 --2- 172.21.15.16:0/82668692 >> v2:172.21.15.132:3300/0 conn(0x7f77040ccf00 0x7f770406d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:32.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.911+0000 7f7708f5e700 1 -- 172.21.15.16:0/82668692 >> v2:172.21.15.132:3300/0 conn(0x7f77040ccf00 msgr2=0x7f770406d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:32.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.911+0000 7f7708f5e700 1 --2- 172.21.15.16:0/82668692 >> v2:172.21.15.132:3300/0 conn(0x7f77040ccf00 0x7f770406d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:32.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.911+0000 7f7708f5e700 1 -- 172.21.15.16:0/82668692 >> v1:172.21.15.132:6789/0 conn(0x7f77040b1e00 legacy=0x7f77040c8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:32.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.911+0000 7f7708f5e700 1 -- 172.21.15.16:0/82668692 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77041608f0 con 0x7f770415caf0 2023-05-20T01:16:32.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.911+0000 7f7708f5e700 1 --2- 172.21.15.16:0/82668692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f770415caf0 0x7f770406cb90 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f77040cf6d0 tx=0x7f76f8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25fc0c2df517b98f server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:32.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.912+0000 7f77037fe700 1 -- 172.21.15.16:0/82668692 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f8004230 con 0x7f770415caf0 2023-05-20T01:16:32.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.912+0000 7f77037fe700 1 -- 172.21.15.16:0/82668692 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76f8004390 con 0x7f770415caf0 2023-05-20T01:16:32.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.912+0000 7f77037fe700 1 -- 172.21.15.16:0/82668692 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f80036a0 con 0x7f770415caf0 2023-05-20T01:16:32.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.912+0000 7f770b1c2700 1 -- 172.21.15.16:0/82668692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f770415caf0 msgr2=0x7f770406cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:32.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.912+0000 7f770b1c2700 1 --2- 172.21.15.16:0/82668692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f770415caf0 0x7f770406cb90 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f77040cf6d0 tx=0x7f76f8009bd0 comp rx=0 tx=0).stop 2023-05-20T01:16:32.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.912+0000 7f770b1c2700 1 -- 172.21.15.16:0/82668692 shutdown_connections 2023-05-20T01:16:32.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.912+0000 7f770b1c2700 1 --2- 172.21.15.16:0/82668692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f770415caf0 0x7f770406cb90 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:32.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.912+0000 7f770b1c2700 1 --2- 172.21.15.16:0/82668692 >> v2:172.21.15.132:3300/0 conn(0x7f77040ccf00 0x7f770406d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:32.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.912+0000 7f770b1c2700 1 -- 172.21.15.16:0/82668692 >> 172.21.15.16:0/82668692 conn(0x7f77040661b0 msgr2=0x7f7704066580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:32.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.912+0000 7f770b1c2700 1 -- 172.21.15.16:0/82668692 shutdown_connections 2023-05-20T01:16:32.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.912+0000 7f770b1c2700 1 -- 172.21.15.16:0/82668692 wait complete. 2023-05-20T01:16:32.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.913+0000 7f770b1c2700 1 Processor -- start 2023-05-20T01:16:32.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.913+0000 7f770b1c2700 1 -- start start 2023-05-20T01:16:32.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.914+0000 7f770b1c2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77040b1e00 0x7f77040b43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:32.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.914+0000 7f770b1c2700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f77040ccf00 0x7f77040b2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:32.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.914+0000 7f770b1c2700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f770415caf0 0x7f7704147130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:32.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.914+0000 7f770b1c2700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7704154a10 con 0x7f77040b1e00 2023-05-20T01:16:32.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.914+0000 7f770b1c2700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77040b3110 con 0x7f770415caf0 2023-05-20T01:16:32.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.914+0000 7f770b1c2700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77040b3340 con 0x7f77040ccf00 2023-05-20T01:16:32.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.914+0000 7f7708f5e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77040b1e00 0x7f77040b43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:32.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.914+0000 7f7708f5e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77040b1e00 0x7f77040b43e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57558/0 (socket says 172.21.15.16:57558) 2023-05-20T01:16:32.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.914+0000 7f7708f5e700 1 -- 172.21.15.16:0/2454340437 learned_addr learned my addr 172.21.15.16:0/2454340437 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:32.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.914+0000 7f770975f700 1 --2- 172.21.15.16:0/2454340437 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f770415caf0 0x7f7704147130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:32.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.914+0000 7f7703fff700 1 --2- 172.21.15.16:0/2454340437 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f77040ccf00 0x7f77040b2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:32.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.915+0000 7f7708f5e700 1 -- 172.21.15.16:0/2454340437 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f77040ccf00 msgr2=0x7f77040b2bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:32.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.915+0000 7f7708f5e700 1 --2- 172.21.15.16:0/2454340437 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f77040ccf00 0x7f77040b2bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:32.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.915+0000 7f7708f5e700 1 -- 172.21.15.16:0/2454340437 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f770415caf0 msgr2=0x7f7704147130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:32.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.915+0000 7f7708f5e700 1 --2- 172.21.15.16:0/2454340437 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f770415caf0 0x7f7704147130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:32.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.915+0000 7f7708f5e700 1 -- 172.21.15.16:0/2454340437 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76f8009550 con 0x7f77040b1e00 2023-05-20T01:16:32.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.915+0000 7f7703fff700 1 --2- 172.21.15.16:0/2454340437 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f77040ccf00 0x7f77040b2bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:16:32.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.915+0000 7f7708f5e700 1 --2- 172.21.15.16:0/2454340437 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77040b1e00 0x7f77040b43e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f77040d0870 tx=0x7f76f8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:32.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.915+0000 7f77017fa700 1 -- 172.21.15.16:0/2454340437 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f8011630 con 0x7f77040b1e00 2023-05-20T01:16:32.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.915+0000 7f77017fa700 1 -- 172.21.15.16:0/2454340437 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76f8011790 con 0x7f77040b1e00 2023-05-20T01:16:32.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.915+0000 7f770b1c2700 1 -- 172.21.15.16:0/2454340437 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7704160eb0 con 0x7f77040b1e00 2023-05-20T01:16:32.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.915+0000 7f77017fa700 1 -- 172.21.15.16:0/2454340437 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f8011b40 con 0x7f77040b1e00 2023-05-20T01:16:32.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.915+0000 7f770b1c2700 1 -- 172.21.15.16:0/2454340437 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7704160bd0 con 0x7f77040b1e00 2023-05-20T01:16:32.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.916+0000 7f770b1c2700 1 -- 172.21.15.16:0/2454340437 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77040cbe30 con 0x7f77040b1e00 2023-05-20T01:16:32.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.916+0000 7f77017fa700 1 -- 172.21.15.16:0/2454340437 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f76f8011ca0 con 0x7f77040b1e00 2023-05-20T01:16:32.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.917+0000 7f77017fa700 1 --2- 172.21.15.16:0/2454340437 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f76e806ace0 0x7f76e806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:32.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.917+0000 7f7703fff700 1 --2- 172.21.15.16:0/2454340437 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f76e806ace0 0x7f76e806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:32.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.917+0000 7f77017fa700 1 -- 172.21.15.16:0/2454340437 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f76f808ae10 con 0x7f77040b1e00 2023-05-20T01:16:32.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.917+0000 7f7703fff700 1 --2- 172.21.15.16:0/2454340437 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f76e806ace0 0x7f76e806d190 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7704157a60 tx=0x7f76f0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:32.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:32.921+0000 7f77017fa700 1 -- 172.21.15.16:0/2454340437 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f76f8055c60 con 0x7f77040b1e00 2023-05-20T01:16:33.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.165+0000 7f770b1c2700 1 -- 172.21.15.16:0/2454340437 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f77040747b0 con 0x7f77040b1e00 2023-05-20T01:16:33.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.165+0000 7f77017fa700 1 -- 172.21.15.16:0/2454340437 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4673 (secure 0 0 0) 0x7f76f8059280 con 0x7f77040b1e00 2023-05-20T01:16:33.169 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:16:33.169 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4273,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/1444124146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1444124146},{"type":"v1","addr":"172.21.15.132:6834","nonce":1444124146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4293,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4092119206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4092119206},{"type":"v1","addr":"172.21.15.16:6835","nonce":4092119206}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2162680372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":2162680372},{"type":"v1","addr":"172.21.15.132:6835","nonce":2162680372}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-20T01:16:28.316109+0000","modified":"2023-05-20T01:16:32.768709+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4299},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4299":{"gid":4299,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6837/2330551626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2330551626},{"type":"v1","addr":"172.21.15.16:6837","nonce":2330551626}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-05-20T01:16:33.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.168+0000 7f770b1c2700 1 -- 172.21.15.16:0/2454340437 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f76e806ace0 msgr2=0x7f76e806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:33.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.168+0000 7f770b1c2700 1 --2- 172.21.15.16:0/2454340437 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f76e806ace0 0x7f76e806d190 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7704157a60 tx=0x7f76f0008040 comp rx=0 tx=0).stop 2023-05-20T01:16:33.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.168+0000 7f770b1c2700 1 -- 172.21.15.16:0/2454340437 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77040b1e00 msgr2=0x7f77040b43e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:33.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.168+0000 7f770b1c2700 1 --2- 172.21.15.16:0/2454340437 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77040b1e00 0x7f77040b43e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f77040d0870 tx=0x7f76f8004750 comp rx=0 tx=0).stop 2023-05-20T01:16:33.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.168+0000 7f770b1c2700 1 -- 172.21.15.16:0/2454340437 shutdown_connections 2023-05-20T01:16:33.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.168+0000 7f770b1c2700 1 --2- 172.21.15.16:0/2454340437 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f770415caf0 0x7f7704147130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:33.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.168+0000 7f770b1c2700 1 --2- 172.21.15.16:0/2454340437 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77040b1e00 0x7f77040b43e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:33.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.168+0000 7f770b1c2700 1 --2- 172.21.15.16:0/2454340437 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f76e806ace0 0x7f76e806d190 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:33.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.168+0000 7f770b1c2700 1 --2- 172.21.15.16:0/2454340437 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f77040ccf00 0x7f77040b2bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:33.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.168+0000 7f770b1c2700 1 -- 172.21.15.16:0/2454340437 >> 172.21.15.16:0/2454340437 conn(0x7f77040661b0 msgr2=0x7f77040cae20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:33.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.168+0000 7f770b1c2700 1 -- 172.21.15.16:0/2454340437 shutdown_connections 2023-05-20T01:16:33.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.168+0000 7f770b1c2700 1 -- 172.21.15.16:0/2454340437 wait complete. 2023-05-20T01:16:33.173 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 6 2023-05-20T01:16:33.183 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-20T01:16:33.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.304+0000 7fb3b2ba3700 1 Processor -- start 2023-05-20T01:16:33.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.304+0000 7fb3b2ba3700 1 -- start start 2023-05-20T01:16:33.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.305+0000 7fb3b2ba3700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fb3ac157190 0x7fb3ac0cc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:33.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.305+0000 7fb3b2ba3700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb3ac0cccf0 0x7fb3ac0713c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:33.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.305+0000 7fb3b2ba3700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb3ac071900 0x7fb3ac15d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:33.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.305+0000 7fb3b2ba3700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3ac0cb590 con 0x7fb3ac071900 2023-05-20T01:16:33.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.305+0000 7fb3b2ba3700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3ac0ca9a0 con 0x7fb3ac157190 2023-05-20T01:16:33.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.305+0000 7fb3b2ba3700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3ac0cac60 con 0x7fb3ac0cccf0 2023-05-20T01:16:33.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.305+0000 7fb3b1140700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb3ac071900 0x7fb3ac15d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:33.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.305+0000 7fb3b1140700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb3ac071900 0x7fb3ac15d220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57570/0 (socket says 172.21.15.16:57570) 2023-05-20T01:16:33.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.305+0000 7fb3b1140700 1 -- 172.21.15.16:0/2688308361 learned_addr learned my addr 172.21.15.16:0/2688308361 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:33.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.305+0000 7fb3b093f700 1 --2- 172.21.15.16:0/2688308361 >> v2:172.21.15.132:3300/0 conn(0x7fb3ac157190 0x7fb3ac0cc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:33.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.305+0000 7fb3abfff700 1 --2- 172.21.15.16:0/2688308361 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb3ac0cccf0 0x7fb3ac0713c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:33.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.305+0000 7fb3b1140700 1 -- 172.21.15.16:0/2688308361 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb3ac0cccf0 msgr2=0x7fb3ac0713c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:33.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.305+0000 7fb3b1140700 1 --2- 172.21.15.16:0/2688308361 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb3ac0cccf0 0x7fb3ac0713c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:33.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.305+0000 7fb3b1140700 1 -- 172.21.15.16:0/2688308361 >> v2:172.21.15.132:3300/0 conn(0x7fb3ac157190 msgr2=0x7fb3ac0cc7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:33.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.305+0000 7fb3b1140700 1 --2- 172.21.15.16:0/2688308361 >> v2:172.21.15.132:3300/0 conn(0x7fb3ac157190 0x7fb3ac0cc7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:33.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.305+0000 7fb3b1140700 1 -- 172.21.15.16:0/2688308361 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3ac0caf40 con 0x7fb3ac071900 2023-05-20T01:16:33.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.305+0000 7fb3b1140700 1 --2- 172.21.15.16:0/2688308361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3ac071900 0x7fb3ac15d220 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac070c40 tx=0x7fb3a400d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2650ebb3fc01e8a2 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:33.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.306+0000 7fb3ab7fe700 1 -- 172.21.15.16:0/2688308361 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3a4010070 con 0x7fb3ac071900 2023-05-20T01:16:33.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.306+0000 7fb3ab7fe700 1 -- 172.21.15.16:0/2688308361 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3a400dd90 con 0x7fb3ac071900 2023-05-20T01:16:33.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.306+0000 7fb3ab7fe700 1 -- 172.21.15.16:0/2688308361 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3a400b310 con 0x7fb3ac071900 2023-05-20T01:16:33.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.306+0000 7fb3b2ba3700 1 -- 172.21.15.16:0/2688308361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3ac071900 msgr2=0x7fb3ac15d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:33.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.306+0000 7fb3b2ba3700 1 --2- 172.21.15.16:0/2688308361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3ac071900 0x7fb3ac15d220 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac070c40 tx=0x7fb3a400d430 comp rx=0 tx=0).stop 2023-05-20T01:16:33.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.306+0000 7fb3b2ba3700 1 -- 172.21.15.16:0/2688308361 shutdown_connections 2023-05-20T01:16:33.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.306+0000 7fb3b2ba3700 1 --2- 172.21.15.16:0/2688308361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3ac071900 0x7fb3ac15d220 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:33.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.306+0000 7fb3b2ba3700 1 --2- 172.21.15.16:0/2688308361 >> v2:172.21.15.132:3300/0 conn(0x7fb3ac157190 0x7fb3ac0cc7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:33.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.306+0000 7fb3b2ba3700 1 --2- 172.21.15.16:0/2688308361 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb3ac0cccf0 0x7fb3ac0713c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:33.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.306+0000 7fb3b2ba3700 1 -- 172.21.15.16:0/2688308361 >> 172.21.15.16:0/2688308361 conn(0x7fb3ac0cffb0 msgr2=0x7fb3ac0d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:33.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.306+0000 7fb3b2ba3700 1 -- 172.21.15.16:0/2688308361 shutdown_connections 2023-05-20T01:16:33.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.306+0000 7fb3b2ba3700 1 -- 172.21.15.16:0/2688308361 wait complete. 2023-05-20T01:16:33.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.308+0000 7fb3b2ba3700 1 Processor -- start 2023-05-20T01:16:33.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.308+0000 7fb3b2ba3700 1 -- start start 2023-05-20T01:16:33.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.309+0000 7fb3b2ba3700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb3ac071900 0x7fb3ac14c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:33.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.309+0000 7fb3b2ba3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3ac0cccf0 0x7fb3ac06ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:33.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.309+0000 7fb3b2ba3700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb3ac157190 0x7fb3ac06f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:33.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.309+0000 7fb3b2ba3700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3ac0b2e40 con 0x7fb3ac0cccf0 2023-05-20T01:16:33.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.309+0000 7fb3b2ba3700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3ac0c65b0 con 0x7fb3ac157190 2023-05-20T01:16:33.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.309+0000 7fb3b2ba3700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3ac0c6890 con 0x7fb3ac071900 2023-05-20T01:16:33.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.309+0000 7fb3abfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3ac0cccf0 0x7fb3ac06ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:33.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.309+0000 7fb3abfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3ac0cccf0 0x7fb3ac06ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57574/0 (socket says 172.21.15.16:57574) 2023-05-20T01:16:33.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.309+0000 7fb3abfff700 1 -- 172.21.15.16:0/3638980174 learned_addr learned my addr 172.21.15.16:0/3638980174 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:33.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.310+0000 7fb3b093f700 1 --2- 172.21.15.16:0/3638980174 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb3ac071900 0x7fb3ac14c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:33.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.310+0000 7fb3abfff700 1 -- 172.21.15.16:0/3638980174 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb3ac071900 msgr2=0x7fb3ac14c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:33.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.310+0000 7fb3abfff700 1 --2- 172.21.15.16:0/3638980174 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb3ac071900 0x7fb3ac14c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:33.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.310+0000 7fb3abfff700 1 -- 172.21.15.16:0/3638980174 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb3ac157190 msgr2=0x7fb3ac06f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:33.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.310+0000 7fb3abfff700 1 --2- 172.21.15.16:0/3638980174 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb3ac157190 0x7fb3ac06f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:33.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.310+0000 7fb3abfff700 1 -- 172.21.15.16:0/3638980174 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3a400bd40 con 0x7fb3ac0cccf0 2023-05-20T01:16:33.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.310+0000 7fb3b093f700 1 --2- 172.21.15.16:0/3638980174 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb3ac071900 0x7fb3ac14c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:16:33.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.310+0000 7fb3abfff700 1 --2- 172.21.15.16:0/3638980174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3ac0cccf0 0x7fb3ac06ef80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb39800e810 tx=0x7fb39800ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:33.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.310+0000 7fb3a97fa700 1 -- 172.21.15.16:0/3638980174 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb398013610 con 0x7fb3ac0cccf0 2023-05-20T01:16:33.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.310+0000 7fb3a97fa700 1 -- 172.21.15.16:0/3638980174 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb398013770 con 0x7fb3ac0cccf0 2023-05-20T01:16:33.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.310+0000 7fb3a97fa700 1 -- 172.21.15.16:0/3638980174 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb398013b20 con 0x7fb3ac0cccf0 2023-05-20T01:16:33.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.310+0000 7fb3b2ba3700 1 -- 172.21.15.16:0/3638980174 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3ac157560 con 0x7fb3ac0cccf0 2023-05-20T01:16:33.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.310+0000 7fb3b2ba3700 1 -- 172.21.15.16:0/3638980174 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3ac0cb220 con 0x7fb3ac0cccf0 2023-05-20T01:16:33.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.311+0000 7fb3b2ba3700 1 -- 172.21.15.16:0/3638980174 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3ac0cd120 con 0x7fb3ac0cccf0 2023-05-20T01:16:33.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.311+0000 7fb3a97fa700 1 -- 172.21.15.16:0/3638980174 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb398013c80 con 0x7fb3ac0cccf0 2023-05-20T01:16:33.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.312+0000 7fb3a97fa700 1 --2- 172.21.15.16:0/3638980174 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb39006ace0 0x7fb39006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:33.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.312+0000 7fb3a97fa700 1 -- 172.21.15.16:0/3638980174 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fb398089420 con 0x7fb3ac0cccf0 2023-05-20T01:16:33.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.312+0000 7fb3b093f700 1 --2- 172.21.15.16:0/3638980174 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb39006ace0 0x7fb39006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:33.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.312+0000 7fb3b093f700 1 --2- 172.21.15.16:0/3638980174 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb39006ace0 0x7fb39006d190 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb3a00097a0 tx=0x7fb3a0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:33.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.316+0000 7fb3a97fa700 1 -- 172.21.15.16:0/3638980174 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fb398057c80 con 0x7fb3ac0cccf0 2023-05-20T01:16:33.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.514+0000 7fb3b2ba3700 1 -- 172.21.15.16:0/3638980174 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb3ac0747b0 con 0x7fb3ac0cccf0 2023-05-20T01:16:33.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.515+0000 7fb3a97fa700 1 -- 172.21.15.16:0/3638980174 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+20026 (secure 0 0 0) 0x7fb398057810 con 0x7fb3ac0cccf0 2023-05-20T01:16:33.519 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:16:33.519 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"fsid":"c2245cc5-ecab-4087-a59b-bc81db901676","created":"2023-05-20T01:16:09.549230+0000","modified":"2023-05-20T01:16:32.768495+0000","last_up_change":"2023-05-20T01:16:19.788624+0000","last_in_change":"2023-05-20T01:16:12.092160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-05-20T01:16:20.934366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-05-20T01:16:21.372294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-05-20T01:16:26.208287+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-05-20T01:16:27.292268+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-05-20T01:16:30.357285+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"317a83cd-19c8-4958-a301-6087ef342289","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6825","nonce":3923634179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6827","nonce":3923634179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6831","nonce":3923634179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6829","nonce":3923634179}]},"public_addr":"172.21.15.16:6825/3923634179","cluster_addr":"172.21.15.16:6827/3923634179","heartbeat_back_addr":"172.21.15.16:6831/3923634179","heartbeat_front_addr":"172.21.15.16:6829/3923634179","state":["exists","up"]},{"osd":1,"uuid":"5581046f-8219-4089-8d7d-b7846399505e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6809","nonce":3860264220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6811","nonce":3860264220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6815","nonce":3860264220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6813","nonce":3860264220}]},"public_addr":"172.21.15.16:6809/3860264220","cluster_addr":"172.21.15.16:6811/3860264220","heartbeat_back_addr":"172.21.15.16:6815/3860264220","heartbeat_front_addr":"172.21.15.16:6813/3860264220","state":["exists","up"]},{"osd":2,"uuid":"072e9e66-4133-478e-8d4a-64d46c2963f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6817","nonce":293187030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6819","nonce":293187030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6823","nonce":293187030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6821","nonce":293187030}]},"public_addr":"172.21.15.16:6817/293187030","cluster_addr":"172.21.15.16:6819/293187030","heartbeat_back_addr":"172.21.15.16:6823/293187030","heartbeat_front_addr":"172.21.15.16:6821/293187030","state":["exists","up"]},{"osd":3,"uuid":"11e1bba6-00f8-48dd-9028-d8d716b7f145","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6801","nonce":3669781890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6803","nonce":3669781890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6807","nonce":3669781890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6805","nonce":3669781890}]},"public_addr":"172.21.15.16:6801/3669781890","cluster_addr":"172.21.15.16:6803/3669781890","heartbeat_back_addr":"172.21.15.16:6807/3669781890","heartbeat_front_addr":"172.21.15.16:6805/3669781890","state":["exists","up"]},{"osd":4,"uuid":"b8909c3a-cf70-46cb-8d0b-009c707cf9d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6819","nonce":506317223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6823","nonce":506317223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6829","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6831","nonce":506317223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6827","nonce":506317223}]},"public_addr":"172.21.15.132:6819/506317223","cluster_addr":"172.21.15.132:6823/506317223","heartbeat_back_addr":"172.21.15.132:6831/506317223","heartbeat_front_addr":"172.21.15.132:6827/506317223","state":["exists","up"]},{"osd":5,"uuid":"b1818304-4583-47bf-a6ec-0bfbb5166233","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6818","nonce":1844689995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6822","nonce":1844689995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6830","nonce":1844689995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6826","nonce":1844689995}]},"public_addr":"172.21.15.132:6818/1844689995","cluster_addr":"172.21.15.132:6822/1844689995","heartbeat_back_addr":"172.21.15.132:6830/1844689995","heartbeat_front_addr":"172.21.15.132:6826/1844689995","state":["exists","up"]},{"osd":6,"uuid":"d37f4691-eed1-4a93-8818-18354f03f4a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6801","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6802","nonce":727581123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6805","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6807","nonce":727581123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6813","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6815","nonce":727581123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6811","nonce":727581123}]},"public_addr":"172.21.15.132:6802/727581123","cluster_addr":"172.21.15.132:6807/727581123","heartbeat_back_addr":"172.21.15.132:6815/727581123","heartbeat_front_addr":"172.21.15.132:6811/727581123","state":["exists","up"]},{"osd":7,"uuid":"4f712f62-5d1d-4816-8042-03209144f8b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6803","nonce":466001149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6806","nonce":466001149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6814","nonce":466001149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6810","nonce":466001149}]},"public_addr":"172.21.15.132:6803/466001149","cluster_addr":"172.21.15.132:6806/466001149","heartbeat_back_addr":"172.21.15.132:6814/466001149","heartbeat_front_addr":"172.21.15.132:6810/466001149","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.338863+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.278680+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305694+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.246835+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.276237+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.304441+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305421+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.267050+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-20T01:16:33.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.517+0000 7fb3b2ba3700 1 -- 172.21.15.16:0/3638980174 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb39006ace0 msgr2=0x7fb39006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:33.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.517+0000 7fb3b2ba3700 1 --2- 172.21.15.16:0/3638980174 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb39006ace0 0x7fb39006d190 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb3a00097a0 tx=0x7fb3a0008040 comp rx=0 tx=0).stop 2023-05-20T01:16:33.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.517+0000 7fb3b2ba3700 1 -- 172.21.15.16:0/3638980174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3ac0cccf0 msgr2=0x7fb3ac06ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:33.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.517+0000 7fb3b2ba3700 1 --2- 172.21.15.16:0/3638980174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3ac0cccf0 0x7fb3ac06ef80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb39800e810 tx=0x7fb39800ebd0 comp rx=0 tx=0).stop 2023-05-20T01:16:33.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.517+0000 7fb3b2ba3700 1 -- 172.21.15.16:0/3638980174 shutdown_connections 2023-05-20T01:16:33.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.517+0000 7fb3b2ba3700 1 --2- 172.21.15.16:0/3638980174 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb3ac157190 0x7fb3ac06f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:33.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.517+0000 7fb3b2ba3700 1 --2- 172.21.15.16:0/3638980174 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb39006ace0 0x7fb39006d190 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:33.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.517+0000 7fb3b2ba3700 1 --2- 172.21.15.16:0/3638980174 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb3ac071900 0x7fb3ac14c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:33.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.517+0000 7fb3b2ba3700 1 --2- 172.21.15.16:0/3638980174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3ac0cccf0 0x7fb3ac06ef80 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:33.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.517+0000 7fb3b2ba3700 1 -- 172.21.15.16:0/3638980174 >> 172.21.15.16:0/3638980174 conn(0x7fb3ac0cffb0 msgr2=0x7fb3ac0cf5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:33.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.517+0000 7fb3b2ba3700 1 -- 172.21.15.16:0/3638980174 shutdown_connections 2023-05-20T01:16:33.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.517+0000 7fb3b2ba3700 1 -- 172.21.15.16:0/3638980174 wait complete. 2023-05-20T01:16:33.533 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-20T01:16:33.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.653+0000 7f6604a45700 1 Processor -- start 2023-05-20T01:16:33.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.653+0000 7f6604a45700 1 -- start start 2023-05-20T01:16:33.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.654+0000 7f6604a45700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f660006d4b0 0x7f66000cf890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:33.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.654+0000 7f6604a45700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f660006a510 0x7f660006a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:33.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.654+0000 7f6604a45700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66000cc510 con 0x7f660006f450 2023-05-20T01:16:33.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.654+0000 7f6604a45700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f660006aef0 con 0x7f660006a510 2023-05-20T01:16:33.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.654+0000 7f6604a45700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6600071ed0 con 0x7f660006d4b0 2023-05-20T01:16:33.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.654+0000 7f65fed9d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f660006a510 0x7f660006a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:33.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.654+0000 7f65fed9d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f660006a510 0x7f660006a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57584/0 (socket says 172.21.15.16:57584) 2023-05-20T01:16:33.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.654+0000 7f65fed9d700 1 -- 172.21.15.16:0/2508529647 learned_addr learned my addr 172.21.15.16:0/2508529647 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:33.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.654+0000 7f65fdd9b700 1 --2- 172.21.15.16:0/2508529647 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f660006d4b0 0x7f66000cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:33.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.654+0000 7f65fed9d700 1 -- 172.21.15.16:0/2508529647 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f660006d4b0 msgr2=0x7f66000cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:33.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.654+0000 7f65fed9d700 1 --2- 172.21.15.16:0/2508529647 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f660006d4b0 0x7f66000cf890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:33.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.654+0000 7f65fed9d700 1 -- 172.21.15.16:0/2508529647 >> v1:172.21.15.132:6789/0 conn(0x7f660006f450 legacy=0x7f6600161080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:33.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.654+0000 7f65fed9d700 1 -- 172.21.15.16:0/2508529647 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66000721b0 con 0x7f660006a510 2023-05-20T01:16:33.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.654+0000 7f65fdd9b700 1 --2- 172.21.15.16:0/2508529647 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f660006d4b0 0x7f66000cf890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:16:33.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.654+0000 7f65fed9d700 1 --2- 172.21.15.16:0/2508529647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f660006a510 0x7f660006a9b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f65f8009e50 tx=0x7f65f800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=893251d6686ec3c7 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:33.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.654+0000 7f65fd59a700 1 -- 172.21.15.16:0/2508529647 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65f800e070 con 0x7f660006a510 2023-05-20T01:16:33.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.654+0000 7f65fd59a700 1 -- 172.21.15.16:0/2508529647 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65f800b9c0 con 0x7f660006a510 2023-05-20T01:16:33.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.654+0000 7f65fd59a700 1 -- 172.21.15.16:0/2508529647 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65f800bd70 con 0x7f660006a510 2023-05-20T01:16:33.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.655+0000 7f6604a45700 1 -- 172.21.15.16:0/2508529647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f660006a510 msgr2=0x7f660006a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:33.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.655+0000 7f6604a45700 1 --2- 172.21.15.16:0/2508529647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f660006a510 0x7f660006a9b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f65f8009e50 tx=0x7f65f800b330 comp rx=0 tx=0).stop 2023-05-20T01:16:33.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.655+0000 7f6604a45700 1 -- 172.21.15.16:0/2508529647 shutdown_connections 2023-05-20T01:16:33.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.655+0000 7f6604a45700 1 --2- 172.21.15.16:0/2508529647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f660006a510 0x7f660006a9b0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:33.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.655+0000 7f6604a45700 1 --2- 172.21.15.16:0/2508529647 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f660006d4b0 0x7f66000cf890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:33.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.655+0000 7f6604a45700 1 -- 172.21.15.16:0/2508529647 >> 172.21.15.16:0/2508529647 conn(0x7f66000cffb0 msgr2=0x7f66000d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:33.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.655+0000 7f6604a45700 1 -- 172.21.15.16:0/2508529647 shutdown_connections 2023-05-20T01:16:33.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.655+0000 7f6604a45700 1 -- 172.21.15.16:0/2508529647 wait complete. 2023-05-20T01:16:33.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.656+0000 7f6604a45700 1 Processor -- start 2023-05-20T01:16:33.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.656+0000 7f6604a45700 1 -- start start 2023-05-20T01:16:33.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.657+0000 7f6604a45700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f660006a510 0x7f6600154920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:33.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.657+0000 7f6604a45700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f660006d4b0 0x7f6600153e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:33.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.657+0000 7f6604a45700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f660006f450 0x7f66000b3e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:33.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.657+0000 7f6604a45700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66001f22d0 con 0x7f660006f450 2023-05-20T01:16:33.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.657+0000 7f6604a45700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6600157e40 con 0x7f660006d4b0 2023-05-20T01:16:33.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.657+0000 7f6604a45700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6600158120 con 0x7f660006a510 2023-05-20T01:16:33.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.657+0000 7f65fed9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f660006f450 0x7f66000b3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:33.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.657+0000 7f65fed9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f660006f450 0x7f66000b3e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57590/0 (socket says 172.21.15.16:57590) 2023-05-20T01:16:33.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.657+0000 7f65fed9d700 1 -- 172.21.15.16:0/296942101 learned_addr learned my addr 172.21.15.16:0/296942101 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:33.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.657+0000 7f65fdd9b700 1 --2- 172.21.15.16:0/296942101 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f660006d4b0 0x7f6600153e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:33.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.657+0000 7f65fe59c700 1 --2- 172.21.15.16:0/296942101 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f660006a510 0x7f6600154920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:33.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.658+0000 7f65fed9d700 1 -- 172.21.15.16:0/296942101 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f660006a510 msgr2=0x7f6600154920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:33.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.658+0000 7f65fed9d700 1 --2- 172.21.15.16:0/296942101 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f660006a510 0x7f6600154920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:33.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.658+0000 7f65fed9d700 1 -- 172.21.15.16:0/296942101 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f660006d4b0 msgr2=0x7f6600153e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:33.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.658+0000 7f65fed9d700 1 --2- 172.21.15.16:0/296942101 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f660006d4b0 0x7f6600153e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:33.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.658+0000 7f65fed9d700 1 -- 172.21.15.16:0/296942101 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65f8009b30 con 0x7f660006f450 2023-05-20T01:16:33.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.658+0000 7f65fed9d700 1 --2- 172.21.15.16:0/296942101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f660006f450 0x7f66000b3e90 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f66000d0e30 tx=0x7f65f8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:33.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.658+0000 7f65eeffd700 1 -- 172.21.15.16:0/296942101 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65f8004840 con 0x7f660006f450 2023-05-20T01:16:33.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.658+0000 7f65eeffd700 1 -- 172.21.15.16:0/296942101 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65f80099b0 con 0x7f660006f450 2023-05-20T01:16:33.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.658+0000 7f6604a45700 1 -- 172.21.15.16:0/296942101 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66000cbe50 con 0x7f660006f450 2023-05-20T01:16:33.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.658+0000 7f65eeffd700 1 -- 172.21.15.16:0/296942101 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65f801c680 con 0x7f660006f450 2023-05-20T01:16:33.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.658+0000 7f6604a45700 1 -- 172.21.15.16:0/296942101 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66000cc1a0 con 0x7f660006f450 2023-05-20T01:16:33.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.659+0000 7f6604a45700 1 -- 172.21.15.16:0/296942101 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66000cc9a0 con 0x7f660006f450 2023-05-20T01:16:33.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.659+0000 7f65eeffd700 1 -- 172.21.15.16:0/296942101 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f65f801c7e0 con 0x7f660006f450 2023-05-20T01:16:33.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.660+0000 7f65eeffd700 1 --2- 172.21.15.16:0/296942101 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f65e006ace0 0x7f65e006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:33.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.660+0000 7f65eeffd700 1 -- 172.21.15.16:0/296942101 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f65f808b3d0 con 0x7f660006f450 2023-05-20T01:16:33.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.660+0000 7f65fe59c700 1 --2- 172.21.15.16:0/296942101 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f65e006ace0 0x7f65e006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:33.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.660+0000 7f65fe59c700 1 --2- 172.21.15.16:0/296942101 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f65e006ace0 0x7f65e006d190 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f65f0007fd0 tx=0x7f65f0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:33.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.664+0000 7f65eeffd700 1 -- 172.21.15.16:0/296942101 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f65f8056180 con 0x7f660006f450 2023-05-20T01:16:33.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.864+0000 7f6604a45700 1 -- 172.21.15.16:0/296942101 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f66000747b0 con 0x7f660006f450 2023-05-20T01:16:33.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.865+0000 7f65eeffd700 1 -- 172.21.15.16:0/296942101 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+20026 (secure 0 0 0) 0x7f65f80597a0 con 0x7f660006f450 2023-05-20T01:16:33.869 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:16:33.870 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"fsid":"c2245cc5-ecab-4087-a59b-bc81db901676","created":"2023-05-20T01:16:09.549230+0000","modified":"2023-05-20T01:16:32.768495+0000","last_up_change":"2023-05-20T01:16:19.788624+0000","last_in_change":"2023-05-20T01:16:12.092160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-05-20T01:16:20.934366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-05-20T01:16:21.372294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-05-20T01:16:26.208287+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-05-20T01:16:27.292268+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-05-20T01:16:30.357285+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"317a83cd-19c8-4958-a301-6087ef342289","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6825","nonce":3923634179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6827","nonce":3923634179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6831","nonce":3923634179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6829","nonce":3923634179}]},"public_addr":"172.21.15.16:6825/3923634179","cluster_addr":"172.21.15.16:6827/3923634179","heartbeat_back_addr":"172.21.15.16:6831/3923634179","heartbeat_front_addr":"172.21.15.16:6829/3923634179","state":["exists","up"]},{"osd":1,"uuid":"5581046f-8219-4089-8d7d-b7846399505e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6809","nonce":3860264220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6811","nonce":3860264220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6815","nonce":3860264220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6813","nonce":3860264220}]},"public_addr":"172.21.15.16:6809/3860264220","cluster_addr":"172.21.15.16:6811/3860264220","heartbeat_back_addr":"172.21.15.16:6815/3860264220","heartbeat_front_addr":"172.21.15.16:6813/3860264220","state":["exists","up"]},{"osd":2,"uuid":"072e9e66-4133-478e-8d4a-64d46c2963f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6817","nonce":293187030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6819","nonce":293187030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6823","nonce":293187030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6821","nonce":293187030}]},"public_addr":"172.21.15.16:6817/293187030","cluster_addr":"172.21.15.16:6819/293187030","heartbeat_back_addr":"172.21.15.16:6823/293187030","heartbeat_front_addr":"172.21.15.16:6821/293187030","state":["exists","up"]},{"osd":3,"uuid":"11e1bba6-00f8-48dd-9028-d8d716b7f145","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6801","nonce":3669781890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6803","nonce":3669781890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6807","nonce":3669781890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6805","nonce":3669781890}]},"public_addr":"172.21.15.16:6801/3669781890","cluster_addr":"172.21.15.16:6803/3669781890","heartbeat_back_addr":"172.21.15.16:6807/3669781890","heartbeat_front_addr":"172.21.15.16:6805/3669781890","state":["exists","up"]},{"osd":4,"uuid":"b8909c3a-cf70-46cb-8d0b-009c707cf9d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6819","nonce":506317223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6823","nonce":506317223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6829","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6831","nonce":506317223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6827","nonce":506317223}]},"public_addr":"172.21.15.132:6819/506317223","cluster_addr":"172.21.15.132:6823/506317223","heartbeat_back_addr":"172.21.15.132:6831/506317223","heartbeat_front_addr":"172.21.15.132:6827/506317223","state":["exists","up"]},{"osd":5,"uuid":"b1818304-4583-47bf-a6ec-0bfbb5166233","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6818","nonce":1844689995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6822","nonce":1844689995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6830","nonce":1844689995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6826","nonce":1844689995}]},"public_addr":"172.21.15.132:6818/1844689995","cluster_addr":"172.21.15.132:6822/1844689995","heartbeat_back_addr":"172.21.15.132:6830/1844689995","heartbeat_front_addr":"172.21.15.132:6826/1844689995","state":["exists","up"]},{"osd":6,"uuid":"d37f4691-eed1-4a93-8818-18354f03f4a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6801","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6802","nonce":727581123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6805","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6807","nonce":727581123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6813","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6815","nonce":727581123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6811","nonce":727581123}]},"public_addr":"172.21.15.132:6802/727581123","cluster_addr":"172.21.15.132:6807/727581123","heartbeat_back_addr":"172.21.15.132:6815/727581123","heartbeat_front_addr":"172.21.15.132:6811/727581123","state":["exists","up"]},{"osd":7,"uuid":"4f712f62-5d1d-4816-8042-03209144f8b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6803","nonce":466001149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6806","nonce":466001149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6814","nonce":466001149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6810","nonce":466001149}]},"public_addr":"172.21.15.132:6803/466001149","cluster_addr":"172.21.15.132:6806/466001149","heartbeat_back_addr":"172.21.15.132:6814/466001149","heartbeat_front_addr":"172.21.15.132:6810/466001149","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.338863+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.278680+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305694+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.246835+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.276237+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.304441+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305421+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.267050+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-20T01:16:33.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.867+0000 7f6604a45700 1 -- 172.21.15.16:0/296942101 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f65e006ace0 msgr2=0x7f65e006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:33.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.867+0000 7f6604a45700 1 --2- 172.21.15.16:0/296942101 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f65e006ace0 0x7f65e006d190 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f65f0007fd0 tx=0x7f65f0009040 comp rx=0 tx=0).stop 2023-05-20T01:16:33.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.867+0000 7f6604a45700 1 -- 172.21.15.16:0/296942101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f660006f450 msgr2=0x7f66000b3e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:33.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.867+0000 7f6604a45700 1 --2- 172.21.15.16:0/296942101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f660006f450 0x7f66000b3e90 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f66000d0e30 tx=0x7f65f8004ad0 comp rx=0 tx=0).stop 2023-05-20T01:16:33.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.868+0000 7f6604a45700 1 -- 172.21.15.16:0/296942101 shutdown_connections 2023-05-20T01:16:33.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.868+0000 7f6604a45700 1 --2- 172.21.15.16:0/296942101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f660006f450 0x7f66000b3e90 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:33.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.868+0000 7f6604a45700 1 --2- 172.21.15.16:0/296942101 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f65e006ace0 0x7f65e006d190 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:33.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.868+0000 7f6604a45700 1 --2- 172.21.15.16:0/296942101 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f660006a510 0x7f6600154920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:33.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.868+0000 7f6604a45700 1 --2- 172.21.15.16:0/296942101 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f660006d4b0 0x7f6600153e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:33.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.868+0000 7f6604a45700 1 -- 172.21.15.16:0/296942101 >> 172.21.15.16:0/296942101 conn(0x7f66000cffb0 msgr2=0x7f66000c98f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:33.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.868+0000 7f6604a45700 1 -- 172.21.15.16:0/296942101 shutdown_connections 2023-05-20T01:16:33.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:33.868+0000 7f6604a45700 1 -- 172.21.15.16:0/296942101 wait complete. 2023-05-20T01:16:33.898 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-20T01:16:33.975 INFO:tasks.ceph.mds.a.smithi016.stderr:2023-05-20T01:16:33.971+0000 7f93a1f3b700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-05-20T01:16:34.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.018+0000 7ffae71a4700 1 Processor -- start 2023-05-20T01:16:34.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.018+0000 7ffae71a4700 1 -- start start 2023-05-20T01:16:34.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.019+0000 7ffae71a4700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffae01598a0 0x7ffae00b3860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:34.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.019+0000 7ffae71a4700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ffae015d340 0x7ffae0071890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:34.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.019+0000 7ffae71a4700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7ffae006cb90 0x7ffae006d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:34.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.019+0000 7ffae71a4700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffae0159c70 con 0x7ffae015d340 2023-05-20T01:16:34.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.019+0000 7ffae71a4700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffae006d570 con 0x7ffae006cb90 2023-05-20T01:16:34.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.019+0000 7ffae71a4700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffae00cbeb0 con 0x7ffae01598a0 2023-05-20T01:16:34.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.019+0000 7ffadffff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ffae015d340 0x7ffae0071890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:34.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.019+0000 7ffadffff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ffae015d340 0x7ffae0071890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57596/0 (socket says 172.21.15.16:57596) 2023-05-20T01:16:34.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.019+0000 7ffadffff700 1 -- 172.21.15.16:0/2346775680 learned_addr learned my addr 172.21.15.16:0/2346775680 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:34.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.019+0000 7ffae4f40700 1 --2- 172.21.15.16:0/2346775680 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffae01598a0 0x7ffae00b3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:34.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.019+0000 7ffae5741700 1 --2- 172.21.15.16:0/2346775680 >> v2:172.21.15.132:3300/0 conn(0x7ffae006cb90 0x7ffae006d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:34.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.019+0000 7ffadffff700 1 -- 172.21.15.16:0/2346775680 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffae01598a0 msgr2=0x7ffae00b3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:34.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.019+0000 7ffadffff700 1 --2- 172.21.15.16:0/2346775680 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffae01598a0 0x7ffae00b3860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:34.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.019+0000 7ffadffff700 1 -- 172.21.15.16:0/2346775680 >> v2:172.21.15.132:3300/0 conn(0x7ffae006cb90 msgr2=0x7ffae006d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:34.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.019+0000 7ffadffff700 1 --2- 172.21.15.16:0/2346775680 >> v2:172.21.15.132:3300/0 conn(0x7ffae006cb90 0x7ffae006d030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:34.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.019+0000 7ffadffff700 1 -- 172.21.15.16:0/2346775680 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffae00b3da0 con 0x7ffae015d340 2023-05-20T01:16:34.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.020+0000 7ffadffff700 1 --2- 172.21.15.16:0/2346775680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffae015d340 0x7ffae0071890 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ffae006fa80 tx=0x7ffacc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=975b5754fe1a245e server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:34.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.020+0000 7ffadf7fe700 1 -- 172.21.15.16:0/2346775680 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffacc00b870 con 0x7ffae015d340 2023-05-20T01:16:34.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.020+0000 7ffadf7fe700 1 -- 172.21.15.16:0/2346775680 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffacc00b9d0 con 0x7ffae015d340 2023-05-20T01:16:34.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.020+0000 7ffadf7fe700 1 -- 172.21.15.16:0/2346775680 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffacc00bd60 con 0x7ffae015d340 2023-05-20T01:16:34.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.020+0000 7ffae71a4700 1 -- 172.21.15.16:0/2346775680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffae015d340 msgr2=0x7ffae0071890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:34.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.020+0000 7ffae71a4700 1 --2- 172.21.15.16:0/2346775680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffae015d340 0x7ffae0071890 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ffae006fa80 tx=0x7ffacc009bd0 comp rx=0 tx=0).stop 2023-05-20T01:16:34.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.020+0000 7ffae71a4700 1 -- 172.21.15.16:0/2346775680 shutdown_connections 2023-05-20T01:16:34.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.020+0000 7ffae71a4700 1 --2- 172.21.15.16:0/2346775680 >> v2:172.21.15.132:3300/0 conn(0x7ffae006cb90 0x7ffae006d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:34.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.020+0000 7ffae71a4700 1 --2- 172.21.15.16:0/2346775680 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffae01598a0 0x7ffae00b3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:34.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.020+0000 7ffae71a4700 1 --2- 172.21.15.16:0/2346775680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffae015d340 0x7ffae0071890 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:34.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.020+0000 7ffae71a4700 1 -- 172.21.15.16:0/2346775680 >> 172.21.15.16:0/2346775680 conn(0x7ffae00661b0 msgr2=0x7ffae0066580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:34.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.020+0000 7ffae71a4700 1 -- 172.21.15.16:0/2346775680 shutdown_connections 2023-05-20T01:16:34.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.020+0000 7ffae71a4700 1 -- 172.21.15.16:0/2346775680 wait complete. 2023-05-20T01:16:34.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.022+0000 7ffae71a4700 1 Processor -- start 2023-05-20T01:16:34.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.022+0000 7ffae71a4700 1 -- start start 2023-05-20T01:16:34.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.022+0000 7ffae71a4700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffae006cb90 0x7ffae0147950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:34.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.022+0000 7ffae71a4700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffae01598a0 0x7ffae014b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:34.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.022+0000 7ffae71a4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffae015d340 0x7ffae014b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:34.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.022+0000 7ffae71a4700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffae015c2f0 con 0x7ffae015d340 2023-05-20T01:16:34.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.022+0000 7ffae71a4700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffae015d0a0 con 0x7ffae006cb90 2023-05-20T01:16:34.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.022+0000 7ffae71a4700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffae01e54f0 con 0x7ffae01598a0 2023-05-20T01:16:34.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.023+0000 7ffae5741700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffae015d340 0x7ffae014b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:34.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.023+0000 7ffae5741700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffae015d340 0x7ffae014b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57604/0 (socket says 172.21.15.16:57604) 2023-05-20T01:16:34.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.023+0000 7ffae5741700 1 -- 172.21.15.16:0/1061307148 learned_addr learned my addr 172.21.15.16:0/1061307148 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:34.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.023+0000 7ffadffff700 1 --2- 172.21.15.16:0/1061307148 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffae01598a0 0x7ffae014b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:34.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.023+0000 7ffae4f40700 1 --2- 172.21.15.16:0/1061307148 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffae006cb90 0x7ffae0147950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:34.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.023+0000 7ffae5741700 1 -- 172.21.15.16:0/1061307148 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffae01598a0 msgr2=0x7ffae014b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:34.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.023+0000 7ffae5741700 1 --2- 172.21.15.16:0/1061307148 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffae01598a0 0x7ffae014b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:34.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.023+0000 7ffae5741700 1 -- 172.21.15.16:0/1061307148 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffae006cb90 msgr2=0x7ffae0147950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:34.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.023+0000 7ffae5741700 1 --2- 172.21.15.16:0/1061307148 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffae006cb90 0x7ffae0147950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:34.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.023+0000 7ffae5741700 1 -- 172.21.15.16:0/1061307148 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffacc009550 con 0x7ffae015d340 2023-05-20T01:16:34.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.023+0000 7ffae5741700 1 --2- 172.21.15.16:0/1061307148 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffae015d340 0x7ffae014b9c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ffad800cd20 tx=0x7ffad800a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:34.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.023+0000 7ffadd7fa700 1 -- 172.21.15.16:0/1061307148 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffad800ce70 con 0x7ffae015d340 2023-05-20T01:16:34.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.023+0000 7ffadd7fa700 1 -- 172.21.15.16:0/1061307148 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffad8003a20 con 0x7ffae015d340 2023-05-20T01:16:34.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.023+0000 7ffae71a4700 1 -- 172.21.15.16:0/1061307148 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffae015da90 con 0x7ffae015d340 2023-05-20T01:16:34.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.023+0000 7ffadd7fa700 1 -- 172.21.15.16:0/1061307148 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffad8003db0 con 0x7ffae015d340 2023-05-20T01:16:34.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.023+0000 7ffae71a4700 1 -- 172.21.15.16:0/1061307148 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffae015d7b0 con 0x7ffae015d340 2023-05-20T01:16:34.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.024+0000 7ffae71a4700 1 -- 172.21.15.16:0/1061307148 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffae015dd70 con 0x7ffae015d340 2023-05-20T01:16:34.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.025+0000 7ffadd7fa700 1 -- 172.21.15.16:0/1061307148 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ffad8003f30 con 0x7ffae015d340 2023-05-20T01:16:34.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.025+0000 7ffadd7fa700 1 --2- 172.21.15.16:0/1061307148 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ffac406ace0 0x7ffac406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:34.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.025+0000 7ffadd7fa700 1 -- 172.21.15.16:0/1061307148 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7ffad8012070 con 0x7ffae015d340 2023-05-20T01:16:34.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.025+0000 7ffae4f40700 1 --2- 172.21.15.16:0/1061307148 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ffac406ace0 0x7ffac406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:34.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.026+0000 7ffae4f40700 1 --2- 172.21.15.16:0/1061307148 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ffac406ace0 0x7ffac406d190 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ffad400ab60 tx=0x7ffad4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:34.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.030+0000 7ffadd7fa700 1 -- 172.21.15.16:0/1061307148 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7ffad8054bc0 con 0x7ffae015d340 2023-05-20T01:16:34.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.270+0000 7ffae71a4700 1 -- 172.21.15.16:0/1061307148 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffae01e3f60 con 0x7ffae015d340 2023-05-20T01:16:34.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.270+0000 7ffadd7fa700 1 -- 172.21.15.16:0/1061307148 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4673 (secure 0 0 0) 0x7ffad80581e0 con 0x7ffae015d340 2023-05-20T01:16:34.274 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:16:34.274 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4273,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/1444124146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1444124146},{"type":"v1","addr":"172.21.15.132:6834","nonce":1444124146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4293,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4092119206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4092119206},{"type":"v1","addr":"172.21.15.16:6835","nonce":4092119206}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2162680372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":2162680372},{"type":"v1","addr":"172.21.15.132:6835","nonce":2162680372}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-20T01:16:28.316109+0000","modified":"2023-05-20T01:16:32.768709+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4299},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4299":{"gid":4299,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6837/2330551626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2330551626},{"type":"v1","addr":"172.21.15.16:6837","nonce":2330551626}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-05-20T01:16:34.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.272+0000 7ffae71a4700 1 -- 172.21.15.16:0/1061307148 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ffac406ace0 msgr2=0x7ffac406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:34.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.272+0000 7ffae71a4700 1 --2- 172.21.15.16:0/1061307148 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ffac406ace0 0x7ffac406d190 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ffad400ab60 tx=0x7ffad4008040 comp rx=0 tx=0).stop 2023-05-20T01:16:34.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.272+0000 7ffae71a4700 1 -- 172.21.15.16:0/1061307148 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffae015d340 msgr2=0x7ffae014b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:34.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.272+0000 7ffae71a4700 1 --2- 172.21.15.16:0/1061307148 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffae015d340 0x7ffae014b9c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ffad800cd20 tx=0x7ffad800a570 comp rx=0 tx=0).stop 2023-05-20T01:16:34.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.272+0000 7ffae71a4700 1 -- 172.21.15.16:0/1061307148 shutdown_connections 2023-05-20T01:16:34.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.272+0000 7ffae71a4700 1 --2- 172.21.15.16:0/1061307148 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffae015d340 0x7ffae014b9c0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:34.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.272+0000 7ffae71a4700 1 --2- 172.21.15.16:0/1061307148 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffae006cb90 0x7ffae0147950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:34.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.272+0000 7ffae71a4700 1 --2- 172.21.15.16:0/1061307148 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ffac406ace0 0x7ffac406d190 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:34.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.272+0000 7ffae71a4700 1 --2- 172.21.15.16:0/1061307148 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffae01598a0 0x7ffae014b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:34.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.272+0000 7ffae71a4700 1 -- 172.21.15.16:0/1061307148 >> 172.21.15.16:0/1061307148 conn(0x7ffae00661b0 msgr2=0x7ffae00b47f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:34.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.272+0000 7ffae71a4700 1 -- 172.21.15.16:0/1061307148 shutdown_connections 2023-05-20T01:16:34.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.272+0000 7ffae71a4700 1 -- 172.21.15.16:0/1061307148 wait complete. 2023-05-20T01:16:34.278 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 6 2023-05-20T01:16:34.288 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-20T01:16:34.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.409+0000 7f2257e10700 1 Processor -- start 2023-05-20T01:16:34.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.409+0000 7f2257e10700 1 -- start start 2023-05-20T01:16:34.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.410+0000 7f2257e10700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f225006f450 0x7f22500cf890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:34.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.410+0000 7f2257e10700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f225006d4b0 0x7f2250161080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:34.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.410+0000 7f2257e10700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f225006a510 0x7f225006a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:34.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.410+0000 7f2257e10700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f225006aed0 con 0x7f225006f450 2023-05-20T01:16:34.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.410+0000 7f2257e10700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22500cc510 con 0x7f225006d4b0 2023-05-20T01:16:34.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.410+0000 7f2257e10700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22501615c0 con 0x7f225006a510 2023-05-20T01:16:34.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.410+0000 7f2255bac700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f225006f450 0x7f22500cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:34.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.410+0000 7f2255bac700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f225006f450 0x7f22500cf890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57608/0 (socket says 172.21.15.16:57608) 2023-05-20T01:16:34.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.410+0000 7f2255bac700 1 -- 172.21.15.16:0/2450053555 learned_addr learned my addr 172.21.15.16:0/2450053555 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:34.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.410+0000 7f22563ad700 1 --2- 172.21.15.16:0/2450053555 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f225006a510 0x7f225006a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:34.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.410+0000 7f22553ab700 1 --2- 172.21.15.16:0/2450053555 >> v2:172.21.15.132:3300/0 conn(0x7f225006d4b0 0x7f2250161080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:34.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.410+0000 7f2255bac700 1 -- 172.21.15.16:0/2450053555 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f225006a510 msgr2=0x7f225006a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:34.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.410+0000 7f2255bac700 1 --2- 172.21.15.16:0/2450053555 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f225006a510 0x7f225006a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:34.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.410+0000 7f2255bac700 1 -- 172.21.15.16:0/2450053555 >> v2:172.21.15.132:3300/0 conn(0x7f225006d4b0 msgr2=0x7f2250161080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:34.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.410+0000 7f2255bac700 1 --2- 172.21.15.16:0/2450053555 >> v2:172.21.15.132:3300/0 conn(0x7f225006d4b0 0x7f2250161080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:34.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.410+0000 7f2255bac700 1 -- 172.21.15.16:0/2450053555 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2250071ed0 con 0x7f225006f450 2023-05-20T01:16:34.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.410+0000 7f22553ab700 1 --2- 172.21.15.16:0/2450053555 >> v2:172.21.15.132:3300/0 conn(0x7f225006d4b0 0x7f2250161080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:16:34.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.410+0000 7f2255bac700 1 --2- 172.21.15.16:0/2450053555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f225006f450 0x7f22500cf890 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2250070c40 tx=0x7f2240009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6da16b374ac39bb server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:34.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.411+0000 7f2254baa700 1 -- 172.21.15.16:0/2450053555 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2240011830 con 0x7f225006f450 2023-05-20T01:16:34.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.411+0000 7f2254baa700 1 -- 172.21.15.16:0/2450053555 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2240011990 con 0x7f225006f450 2023-05-20T01:16:34.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.411+0000 7f2254baa700 1 -- 172.21.15.16:0/2450053555 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2240011d20 con 0x7f225006f450 2023-05-20T01:16:34.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.411+0000 7f2257e10700 1 -- 172.21.15.16:0/2450053555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f225006f450 msgr2=0x7f22500cf890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:34.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.411+0000 7f2257e10700 1 --2- 172.21.15.16:0/2450053555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f225006f450 0x7f22500cf890 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2250070c40 tx=0x7f2240009bd0 comp rx=0 tx=0).stop 2023-05-20T01:16:34.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.411+0000 7f2257e10700 1 -- 172.21.15.16:0/2450053555 shutdown_connections 2023-05-20T01:16:34.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.411+0000 7f2257e10700 1 --2- 172.21.15.16:0/2450053555 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f225006a510 0x7f225006a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:34.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.411+0000 7f2257e10700 1 --2- 172.21.15.16:0/2450053555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f225006f450 0x7f22500cf890 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:34.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.411+0000 7f2257e10700 1 --2- 172.21.15.16:0/2450053555 >> v2:172.21.15.132:3300/0 conn(0x7f225006d4b0 0x7f2250161080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:34.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.411+0000 7f2257e10700 1 -- 172.21.15.16:0/2450053555 >> 172.21.15.16:0/2450053555 conn(0x7f22500cffb0 msgr2=0x7f22500d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:34.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.411+0000 7f2257e10700 1 -- 172.21.15.16:0/2450053555 shutdown_connections 2023-05-20T01:16:34.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.411+0000 7f2257e10700 1 -- 172.21.15.16:0/2450053555 wait complete. 2023-05-20T01:16:34.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.412+0000 7f2257e10700 1 Processor -- start 2023-05-20T01:16:34.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.412+0000 7f2257e10700 1 -- start start 2023-05-20T01:16:34.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.413+0000 7f2257e10700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f225006a510 0x7f225015cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:34.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.413+0000 7f2257e10700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f225006d4b0 0x7f22500b4110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:34.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.413+0000 7f2257e10700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f225006f450 0x7f22501f1e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:34.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.413+0000 7f2257e10700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f225014ef10 con 0x7f225006d4b0 2023-05-20T01:16:34.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.413+0000 7f2257e10700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22501f5e80 con 0x7f225006a510 2023-05-20T01:16:34.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.413+0000 7f2257e10700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22501f6160 con 0x7f225006f450 2023-05-20T01:16:34.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.413+0000 7f22553ab700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f225006d4b0 0x7f22500b4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:34.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.413+0000 7f22553ab700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f225006d4b0 0x7f22500b4110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57614/0 (socket says 172.21.15.16:57614) 2023-05-20T01:16:34.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.413+0000 7f22553ab700 1 -- 172.21.15.16:0/1141654519 learned_addr learned my addr 172.21.15.16:0/1141654519 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:34.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.413+0000 7f2255bac700 1 --2- 172.21.15.16:0/1141654519 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f225006a510 0x7f225015cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:34.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.413+0000 7f22563ad700 1 --2- 172.21.15.16:0/1141654519 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f225006f450 0x7f22501f1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:34.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.414+0000 7f22553ab700 1 -- 172.21.15.16:0/1141654519 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f225006f450 msgr2=0x7f22501f1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:34.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.414+0000 7f22553ab700 1 --2- 172.21.15.16:0/1141654519 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f225006f450 0x7f22501f1e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:34.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.414+0000 7f22553ab700 1 -- 172.21.15.16:0/1141654519 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f225006a510 msgr2=0x7f225015cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:34.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.414+0000 7f22553ab700 1 --2- 172.21.15.16:0/1141654519 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f225006a510 0x7f225015cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:34.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.414+0000 7f22553ab700 1 -- 172.21.15.16:0/1141654519 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2240009550 con 0x7f225006d4b0 2023-05-20T01:16:34.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.414+0000 7f22553ab700 1 --2- 172.21.15.16:0/1141654519 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f225006d4b0 0x7f22500b4110 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f224c00e810 tx=0x7f224c00eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:34.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.414+0000 7f22467fc700 1 -- 172.21.15.16:0/1141654519 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f224c00c070 con 0x7f225006d4b0 2023-05-20T01:16:34.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.414+0000 7f22467fc700 1 -- 172.21.15.16:0/1141654519 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f224c004230 con 0x7f225006d4b0 2023-05-20T01:16:34.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.414+0000 7f2257e10700 1 -- 172.21.15.16:0/1141654519 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22500cc1a0 con 0x7f225006d4b0 2023-05-20T01:16:34.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.414+0000 7f22467fc700 1 -- 172.21.15.16:0/1141654519 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f224c0036a0 con 0x7f225006d4b0 2023-05-20T01:16:34.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.414+0000 7f2257e10700 1 -- 172.21.15.16:0/1141654519 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22500721b0 con 0x7f225006d4b0 2023-05-20T01:16:34.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.415+0000 7f2257e10700 1 -- 172.21.15.16:0/1141654519 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22500cbe50 con 0x7f225006d4b0 2023-05-20T01:16:34.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.416+0000 7f22467fc700 1 -- 172.21.15.16:0/1141654519 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f224c003800 con 0x7f225006d4b0 2023-05-20T01:16:34.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.416+0000 7f22467fc700 1 --2- 172.21.15.16:0/1141654519 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f223406ace0 0x7f223406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:34.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.416+0000 7f22467fc700 1 -- 172.21.15.16:0/1141654519 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f224c089750 con 0x7f225006d4b0 2023-05-20T01:16:34.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.416+0000 7f2255bac700 1 --2- 172.21.15.16:0/1141654519 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f223406ace0 0x7f223406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:34.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.416+0000 7f2255bac700 1 --2- 172.21.15.16:0/1141654519 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f223406ace0 0x7f223406d190 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2240014040 tx=0x7f2240008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:34.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.420+0000 7f22467fc700 1 -- 172.21.15.16:0/1141654519 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f224c0545f0 con 0x7f225006d4b0 2023-05-20T01:16:34.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.617+0000 7f2257e10700 1 -- 172.21.15.16:0/1141654519 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f22500747b0 con 0x7f225006d4b0 2023-05-20T01:16:34.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.618+0000 7f22467fc700 1 -- 172.21.15.16:0/1141654519 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+20026 (secure 0 0 0) 0x7f224c057c10 con 0x7f225006d4b0 2023-05-20T01:16:34.622 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:16:34.622 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"fsid":"c2245cc5-ecab-4087-a59b-bc81db901676","created":"2023-05-20T01:16:09.549230+0000","modified":"2023-05-20T01:16:32.768495+0000","last_up_change":"2023-05-20T01:16:19.788624+0000","last_in_change":"2023-05-20T01:16:12.092160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-05-20T01:16:20.934366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-05-20T01:16:21.372294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-05-20T01:16:26.208287+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-05-20T01:16:27.292268+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-05-20T01:16:30.357285+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"317a83cd-19c8-4958-a301-6087ef342289","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6825","nonce":3923634179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6827","nonce":3923634179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6831","nonce":3923634179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6829","nonce":3923634179}]},"public_addr":"172.21.15.16:6825/3923634179","cluster_addr":"172.21.15.16:6827/3923634179","heartbeat_back_addr":"172.21.15.16:6831/3923634179","heartbeat_front_addr":"172.21.15.16:6829/3923634179","state":["exists","up"]},{"osd":1,"uuid":"5581046f-8219-4089-8d7d-b7846399505e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6809","nonce":3860264220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6811","nonce":3860264220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6815","nonce":3860264220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6813","nonce":3860264220}]},"public_addr":"172.21.15.16:6809/3860264220","cluster_addr":"172.21.15.16:6811/3860264220","heartbeat_back_addr":"172.21.15.16:6815/3860264220","heartbeat_front_addr":"172.21.15.16:6813/3860264220","state":["exists","up"]},{"osd":2,"uuid":"072e9e66-4133-478e-8d4a-64d46c2963f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6817","nonce":293187030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6819","nonce":293187030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6823","nonce":293187030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6821","nonce":293187030}]},"public_addr":"172.21.15.16:6817/293187030","cluster_addr":"172.21.15.16:6819/293187030","heartbeat_back_addr":"172.21.15.16:6823/293187030","heartbeat_front_addr":"172.21.15.16:6821/293187030","state":["exists","up"]},{"osd":3,"uuid":"11e1bba6-00f8-48dd-9028-d8d716b7f145","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6801","nonce":3669781890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6803","nonce":3669781890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6807","nonce":3669781890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6805","nonce":3669781890}]},"public_addr":"172.21.15.16:6801/3669781890","cluster_addr":"172.21.15.16:6803/3669781890","heartbeat_back_addr":"172.21.15.16:6807/3669781890","heartbeat_front_addr":"172.21.15.16:6805/3669781890","state":["exists","up"]},{"osd":4,"uuid":"b8909c3a-cf70-46cb-8d0b-009c707cf9d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6819","nonce":506317223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6823","nonce":506317223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6829","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6831","nonce":506317223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6827","nonce":506317223}]},"public_addr":"172.21.15.132:6819/506317223","cluster_addr":"172.21.15.132:6823/506317223","heartbeat_back_addr":"172.21.15.132:6831/506317223","heartbeat_front_addr":"172.21.15.132:6827/506317223","state":["exists","up"]},{"osd":5,"uuid":"b1818304-4583-47bf-a6ec-0bfbb5166233","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6818","nonce":1844689995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6822","nonce":1844689995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6830","nonce":1844689995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6826","nonce":1844689995}]},"public_addr":"172.21.15.132:6818/1844689995","cluster_addr":"172.21.15.132:6822/1844689995","heartbeat_back_addr":"172.21.15.132:6830/1844689995","heartbeat_front_addr":"172.21.15.132:6826/1844689995","state":["exists","up"]},{"osd":6,"uuid":"d37f4691-eed1-4a93-8818-18354f03f4a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6801","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6802","nonce":727581123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6805","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6807","nonce":727581123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6813","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6815","nonce":727581123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6811","nonce":727581123}]},"public_addr":"172.21.15.132:6802/727581123","cluster_addr":"172.21.15.132:6807/727581123","heartbeat_back_addr":"172.21.15.132:6815/727581123","heartbeat_front_addr":"172.21.15.132:6811/727581123","state":["exists","up"]},{"osd":7,"uuid":"4f712f62-5d1d-4816-8042-03209144f8b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6803","nonce":466001149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6806","nonce":466001149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6814","nonce":466001149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6810","nonce":466001149}]},"public_addr":"172.21.15.132:6803/466001149","cluster_addr":"172.21.15.132:6806/466001149","heartbeat_back_addr":"172.21.15.132:6814/466001149","heartbeat_front_addr":"172.21.15.132:6810/466001149","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.338863+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.278680+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305694+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.246835+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.276237+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.304441+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305421+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.267050+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-20T01:16:34.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.620+0000 7f2257e10700 1 -- 172.21.15.16:0/1141654519 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f223406ace0 msgr2=0x7f223406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:34.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.620+0000 7f2257e10700 1 --2- 172.21.15.16:0/1141654519 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f223406ace0 0x7f223406d190 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2240014040 tx=0x7f2240008030 comp rx=0 tx=0).stop 2023-05-20T01:16:34.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.620+0000 7f2257e10700 1 -- 172.21.15.16:0/1141654519 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f225006d4b0 msgr2=0x7f22500b4110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:34.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.620+0000 7f2257e10700 1 --2- 172.21.15.16:0/1141654519 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f225006d4b0 0x7f22500b4110 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f224c00e810 tx=0x7f224c00eb20 comp rx=0 tx=0).stop 2023-05-20T01:16:34.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.620+0000 7f2257e10700 1 -- 172.21.15.16:0/1141654519 shutdown_connections 2023-05-20T01:16:34.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.620+0000 7f2257e10700 1 --2- 172.21.15.16:0/1141654519 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f223406ace0 0x7f223406d190 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:34.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.620+0000 7f2257e10700 1 --2- 172.21.15.16:0/1141654519 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f225006f450 0x7f22501f1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:34.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.620+0000 7f2257e10700 1 --2- 172.21.15.16:0/1141654519 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f225006a510 0x7f225015cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:34.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.620+0000 7f2257e10700 1 --2- 172.21.15.16:0/1141654519 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f225006d4b0 0x7f22500b4110 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:34.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.620+0000 7f2257e10700 1 -- 172.21.15.16:0/1141654519 >> 172.21.15.16:0/1141654519 conn(0x7f22500cffb0 msgr2=0x7f225006e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:34.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.620+0000 7f2257e10700 1 -- 172.21.15.16:0/1141654519 shutdown_connections 2023-05-20T01:16:34.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:34.620+0000 7f2257e10700 1 -- 172.21.15.16:0/1141654519 wait complete. 2023-05-20T01:16:35.000 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:16:35.002 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:16:35.039 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 6, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-05-20T01:16:28.316109+0000', 'modified': '2023-05-20T01:16:32.768709+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4299}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4299': {'gid': 4299, 'name': 'a', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.16:6837/2330551626', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6836', 'nonce': 2330551626}, {'type': 'v1', 'addr': '172.21.15.16:6837', 'nonce': 2330551626}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [4, 5], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1} 2023-05-20T01:16:35.040 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-05-20T01:16:35.040 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-05-20T01:16:35.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.170+0000 7f554edcc700 1 Processor -- start 2023-05-20T01:16:35.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.170+0000 7f554edcc700 1 -- start start 2023-05-20T01:16:35.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.171+0000 7f554edcc700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f55480ccec0 0x7f55480713c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:35.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.171+0000 7f554edcc700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5548071900 0x7f55480ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:35.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.171+0000 7f554edcc700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55480cb590 con 0x7f5548157190 2023-05-20T01:16:35.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.171+0000 7f554edcc700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55480ca9a0 con 0x7f55480ccec0 2023-05-20T01:16:35.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.171+0000 7f554edcc700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55480cac60 con 0x7f5548071900 2023-05-20T01:16:35.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.171+0000 7f5547fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f55480ccec0 0x7f55480713c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:35.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.171+0000 7f5547fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f55480ccec0 0x7f55480713c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57622/0 (socket says 172.21.15.16:57622) 2023-05-20T01:16:35.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.171+0000 7f5547fff700 1 -- 172.21.15.16:0/1435003408 learned_addr learned my addr 172.21.15.16:0/1435003408 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:35.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.171+0000 7f554d369700 1 --2- 172.21.15.16:0/1435003408 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5548071900 0x7f55480ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:35.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.171+0000 7f5547fff700 1 -- 172.21.15.16:0/1435003408 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5548071900 msgr2=0x7f55480ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:35.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.171+0000 7f5547fff700 1 --2- 172.21.15.16:0/1435003408 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5548071900 0x7f55480ce740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:35.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.171+0000 7f5547fff700 1 -- 172.21.15.16:0/1435003408 >> v1:172.21.15.132:6789/0 conn(0x7f5548157190 legacy=0x7f55480cc7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:35.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.171+0000 7f5547fff700 1 -- 172.21.15.16:0/1435003408 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55480caf40 con 0x7f55480ccec0 2023-05-20T01:16:35.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.171+0000 7f5547fff700 1 --2- 172.21.15.16:0/1435003408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55480ccec0 0x7f55480713c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f553400bb90 tx=0x7f553400bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=480558800dd66579 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:35.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.171+0000 7f55477fe700 1 -- 172.21.15.16:0/1435003408 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f553400dd50 con 0x7f55480ccec0 2023-05-20T01:16:35.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.171+0000 7f55477fe700 1 -- 172.21.15.16:0/1435003408 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5534010470 con 0x7f55480ccec0 2023-05-20T01:16:35.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.171+0000 7f55477fe700 1 -- 172.21.15.16:0/1435003408 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5534010800 con 0x7f55480ccec0 2023-05-20T01:16:35.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.172+0000 7f554edcc700 1 -- 172.21.15.16:0/1435003408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55480ccec0 msgr2=0x7f55480713c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:35.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.172+0000 7f554edcc700 1 --2- 172.21.15.16:0/1435003408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55480ccec0 0x7f55480713c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f553400bb90 tx=0x7f553400bea0 comp rx=0 tx=0).stop 2023-05-20T01:16:35.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.172+0000 7f554edcc700 1 -- 172.21.15.16:0/1435003408 shutdown_connections 2023-05-20T01:16:35.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.172+0000 7f554edcc700 1 --2- 172.21.15.16:0/1435003408 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5548071900 0x7f55480ce740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:35.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.172+0000 7f554edcc700 1 --2- 172.21.15.16:0/1435003408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55480ccec0 0x7f55480713c0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:35.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.172+0000 7f554edcc700 1 -- 172.21.15.16:0/1435003408 >> 172.21.15.16:0/1435003408 conn(0x7f55480cffb0 msgr2=0x7f55480d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:35.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.172+0000 7f554edcc700 1 -- 172.21.15.16:0/1435003408 shutdown_connections 2023-05-20T01:16:35.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.172+0000 7f554edcc700 1 -- 172.21.15.16:0/1435003408 wait complete. 2023-05-20T01:16:35.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.173+0000 7f554edcc700 1 Processor -- start 2023-05-20T01:16:35.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.173+0000 7f554edcc700 1 -- start start 2023-05-20T01:16:35.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.174+0000 7f554edcc700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5548071900 0x7f5548149120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:35.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.174+0000 7f554edcc700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f55480ccec0 0x7f55481587d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:35.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.174+0000 7f554edcc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5548157190 0x7f5548158d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:35.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.174+0000 7f554edcc700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55480c81b0 con 0x7f5548157190 2023-05-20T01:16:35.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.174+0000 7f554edcc700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5548149660 con 0x7f5548071900 2023-05-20T01:16:35.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.174+0000 7f554edcc700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55481f67c0 con 0x7f55480ccec0 2023-05-20T01:16:35.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.174+0000 7f554d369700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5548157190 0x7f5548158d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:35.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.174+0000 7f554d369700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5548157190 0x7f5548158d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57630/0 (socket says 172.21.15.16:57630) 2023-05-20T01:16:35.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.174+0000 7f554d369700 1 -- 172.21.15.16:0/63427141 learned_addr learned my addr 172.21.15.16:0/63427141 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:35.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.174+0000 7f5547fff700 1 --2- 172.21.15.16:0/63427141 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f55480ccec0 0x7f55481587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:35.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.174+0000 7f554cb68700 1 --2- 172.21.15.16:0/63427141 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5548071900 0x7f5548149120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:35.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.175+0000 7f554d369700 1 -- 172.21.15.16:0/63427141 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f55480ccec0 msgr2=0x7f55481587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:35.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.175+0000 7f554d369700 1 --2- 172.21.15.16:0/63427141 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f55480ccec0 0x7f55481587d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:35.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.175+0000 7f554d369700 1 -- 172.21.15.16:0/63427141 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5548071900 msgr2=0x7f5548149120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:35.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.175+0000 7f554d369700 1 --2- 172.21.15.16:0/63427141 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5548071900 0x7f5548149120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:35.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.175+0000 7f554d369700 1 -- 172.21.15.16:0/63427141 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f553400b7f0 con 0x7f5548157190 2023-05-20T01:16:35.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.175+0000 7f554d369700 1 --2- 172.21.15.16:0/63427141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5548157190 0x7f5548158d10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5540007e40 tx=0x7f554000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:35.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.175+0000 7f55457fa700 1 -- 172.21.15.16:0/63427141 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f554000ed30 con 0x7f5548157190 2023-05-20T01:16:35.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.175+0000 7f55457fa700 1 -- 172.21.15.16:0/63427141 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5540014420 con 0x7f5548157190 2023-05-20T01:16:35.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.175+0000 7f554edcc700 1 -- 172.21.15.16:0/63427141 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5548157560 con 0x7f5548157190 2023-05-20T01:16:35.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.175+0000 7f55457fa700 1 -- 172.21.15.16:0/63427141 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55400147d0 con 0x7f5548157190 2023-05-20T01:16:35.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.175+0000 7f554edcc700 1 -- 172.21.15.16:0/63427141 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55480cb220 con 0x7f5548157190 2023-05-20T01:16:35.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.176+0000 7f55457fa700 1 -- 172.21.15.16:0/63427141 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5540014930 con 0x7f5548157190 2023-05-20T01:16:35.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.177+0000 7f55457fa700 1 --2- 172.21.15.16:0/63427141 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f552c06ace0 0x7f552c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:35.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.177+0000 7f55457fa700 1 -- 172.21.15.16:0/63427141 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f5540089ec0 con 0x7f5548157190 2023-05-20T01:16:35.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.177+0000 7f554cb68700 1 --2- 172.21.15.16:0/63427141 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f552c06ace0 0x7f552c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:35.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.177+0000 7f554cb68700 1 --2- 172.21.15.16:0/63427141 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f552c06ace0 0x7f552c06d190 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f553c007fd0 tx=0x7f553c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:35.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.179+0000 7f555af58b80 1 Processor -- start 2023-05-20T01:16:35.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.179+0000 7f555af58b80 1 -- start start 2023-05-20T01:16:35.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.179+0000 7f555af58b80 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556ba70c9400 0x556ba70c97d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:35.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.179+0000 7f555af58b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ba70c5700 0x556ba709e460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:35.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.179+0000 7f555af58b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556ba709c870 0x556ba709e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:35.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.179+0000 7f555af58b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ba6fa4010 con 0x556ba70c5700 2023-05-20T01:16:35.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.179+0000 7f555af58b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ba6ff6850 con 0x556ba709c870 2023-05-20T01:16:35.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.179+0000 7f555af58b80 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x556ba6fefce0 con 0x556ba70c9400 2023-05-20T01:16:35.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.180+0000 7f5547fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ba70c5700 0x556ba709e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:35.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.180+0000 7f5547fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ba70c5700 0x556ba709e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57636/0 (socket says 172.21.15.16:57636) 2023-05-20T01:16:35.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.180+0000 7f5547fff700 1 -- 172.21.15.16:0/3887060907 learned_addr learned my addr 172.21.15.16:0/3887060907 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:35.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.180+0000 7f554cb68700 1 --2- 172.21.15.16:0/3887060907 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556ba70c9400 0x556ba70c97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:35.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.180+0000 7f554d369700 1 --2- 172.21.15.16:0/3887060907 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556ba709c870 0x556ba709e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:35.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.180+0000 7f5547fff700 1 -- 172.21.15.16:0/3887060907 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556ba70c9400 msgr2=0x556ba70c97d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:35.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.180+0000 7f5547fff700 1 --2- 172.21.15.16:0/3887060907 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556ba70c9400 0x556ba70c97d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:35.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.180+0000 7f5547fff700 1 -- 172.21.15.16:0/3887060907 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556ba709c870 msgr2=0x556ba709e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:35.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.180+0000 7f5547fff700 1 --2- 172.21.15.16:0/3887060907 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556ba709c870 0x556ba709e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:35.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.180+0000 7f5547fff700 1 -- 172.21.15.16:0/3887060907 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556ba6e715e0 con 0x556ba70c5700 2023-05-20T01:16:35.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.180+0000 7f5547fff700 1 --2- 172.21.15.16:0/3887060907 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ba70c5700 0x556ba709e460 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f55480d0f50 tx=0x7f5534012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4f87433f34c0072 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:35.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.180+0000 7f55327fc700 1 -- 172.21.15.16:0/3887060907 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f553401e070 con 0x556ba70c5700 2023-05-20T01:16:35.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.180+0000 7f55327fc700 1 -- 172.21.15.16:0/3887060907 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f553400dd80 con 0x556ba70c5700 2023-05-20T01:16:35.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.180+0000 7f55327fc700 1 -- 172.21.15.16:0/3887060907 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5534004270 con 0x556ba70c5700 2023-05-20T01:16:35.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.181+0000 7f555af58b80 1 -- 172.21.15.16:0/3887060907 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ba70c5700 msgr2=0x556ba709e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:35.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.181+0000 7f555af58b80 1 --2- 172.21.15.16:0/3887060907 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ba70c5700 0x556ba709e460 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f55480d0f50 tx=0x7f5534012000 comp rx=0 tx=0).stop 2023-05-20T01:16:35.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.181+0000 7f555af58b80 1 -- 172.21.15.16:0/3887060907 shutdown_connections 2023-05-20T01:16:35.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.181+0000 7f555af58b80 1 --2- 172.21.15.16:0/3887060907 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556ba709c870 0x556ba709e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:35.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.181+0000 7f555af58b80 1 --2- 172.21.15.16:0/3887060907 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556ba70c9400 0x556ba70c97d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:35.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.181+0000 7f555af58b80 1 --2- 172.21.15.16:0/3887060907 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ba70c5700 0x556ba709e460 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:35.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.181+0000 7f555af58b80 1 -- 172.21.15.16:0/3887060907 >> 172.21.15.16:0/3887060907 conn(0x556ba7093f30 msgr2=0x556ba70e4210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:35.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.181+0000 7f555af58b80 1 -- 172.21.15.16:0/3887060907 shutdown_connections 2023-05-20T01:16:35.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.181+0000 7f555af58b80 1 -- 172.21.15.16:0/3887060907 wait complete. 2023-05-20T01:16:35.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.181+0000 7f555af58b80 1 -- start start 2023-05-20T01:16:35.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.182+0000 7f555af58b80 1 Processor -- start 2023-05-20T01:16:35.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.182+0000 7f555af58b80 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556ba709c870 0x556ba70c1d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:35.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.182+0000 7f555af58b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556ba70c5700 0x556ba709deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:35.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.182+0000 7f555af58b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ba70c9400 0x556ba709e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:35.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.182+0000 7f555af58b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ba6f91890 con 0x556ba70c9400 2023-05-20T01:16:35.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.182+0000 7f555af58b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ba6f91530 con 0x556ba70c5700 2023-05-20T01:16:35.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.182+0000 7f555af58b80 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x556ba6e738d0 con 0x556ba709c870 2023-05-20T01:16:35.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.182+0000 7f555af58b80 10 client.0 _ll_register_callbacks cb 0x556ba6faf730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-05-20T01:16:35.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.183+0000 7f554d369700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ba70c9400 0x556ba709e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:35.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.183+0000 7f554d369700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ba70c9400 0x556ba709e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57644/0 (socket says 172.21.15.16:57644) 2023-05-20T01:16:35.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.183+0000 7f554d369700 1 -- 172.21.15.16:0/3632196415 learned_addr learned my addr 172.21.15.16:0/3632196415 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:35.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.183+0000 7f554cb68700 1 --2- 172.21.15.16:0/3632196415 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556ba709c870 0x556ba70c1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:35.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.183+0000 7f5547fff700 1 --2- 172.21.15.16:0/3632196415 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556ba70c5700 0x556ba709deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:35.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.183+0000 7f554d369700 1 -- 172.21.15.16:0/3632196415 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556ba709c870 msgr2=0x556ba70c1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:35.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.183+0000 7f554d369700 1 --2- 172.21.15.16:0/3632196415 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556ba709c870 0x556ba70c1d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:35.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.183+0000 7f554d369700 1 -- 172.21.15.16:0/3632196415 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556ba70c5700 msgr2=0x556ba709deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:35.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.183+0000 7f554d369700 1 --2- 172.21.15.16:0/3632196415 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556ba70c5700 0x556ba709deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:35.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.183+0000 7f554d369700 1 -- 172.21.15.16:0/3632196415 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55480caf40 con 0x556ba70c9400 2023-05-20T01:16:35.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.183+0000 7f554d369700 1 --2- 172.21.15.16:0/3632196415 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ba70c9400 0x556ba709e3f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5540054ef0 tx=0x7f554005c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a022cb9e9843a8c4 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:35.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.183+0000 7f55277fe700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-05-20T01:16:35.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.183+0000 7f55277fe700 1 -- 172.21.15.16:0/3632196415 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55400580e0 con 0x556ba70c9400 2023-05-20T01:16:35.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.183+0000 7f55277fe700 1 -- 172.21.15.16:0/3632196415 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f5540060800 con 0x556ba70c9400 2023-05-20T01:16:35.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.183+0000 7f555af58b80 1 -- 172.21.15.16:0/3632196415 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556ba70916f0 con 0x556ba70c9400 2023-05-20T01:16:35.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.183+0000 7f55277fe700 1 -- 172.21.15.16:0/3632196415 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) v2 ==== 24+0+0 (secure 0 0 0) 0x7f55400609c0 con 0x556ba70c9400 2023-05-20T01:16:35.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.183+0000 7f555af58b80 10 client.4446 fetch_fsmap learned FSMap version 6 2023-05-20T01:16:35.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.183+0000 7f555af58b80 1 -- 172.21.15.16:0/3632196415 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) v3 -- 0x556ba6fb6140 con 0x556ba70c9400 2023-05-20T01:16:35.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.183+0000 7f55277fe700 1 -- 172.21.15.16:0/3632196415 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5540077020 con 0x556ba70c9400 2023-05-20T01:16:35.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.183+0000 7f55277fe700 1 -- 172.21.15.16:0/3632196415 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 6) v1 ==== 3122+0+0 (secure 0 0 0) 0x7f5540077190 con 0x556ba70c9400 2023-05-20T01:16:35.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.183+0000 7f555af58b80 10 client.4446 fetch_fsmap finished waiting for FSMap version 6 2023-05-20T01:16:35.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.184+0000 7f555af58b80 10 client.4446 resolve_mds: resolved name 'a' to daemon mds.a 2023-05-20T01:16:35.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.184+0000 7f555af58b80 20 client.4446 populate_metadata read hostname 'smithi016' 2023-05-20T01:16:35.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.184+0000 7f555af58b80 1 --2- 172.21.15.16:0/3632196415 >> [v2:172.21.15.16:6836/2330551626,v1:172.21.15.16:6837/2330551626] conn(0x556ba709e930 0x556ba70be730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:35.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.184+0000 7f555af58b80 4 client.4446 mds_command: new command op to 4299 tid=0[{"prefix": "get_command_descriptions"}] 2023-05-20T01:16:35.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.184+0000 7f555af58b80 1 -- 172.21.15.16:0/3632196415 --> [v2:172.21.15.16:6836/2330551626,v1:172.21.15.16:6837/2330551626] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556ba6fb2050 con 0x556ba709e930 2023-05-20T01:16:35.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.184+0000 7f554cb68700 1 --2- 172.21.15.16:0/3632196415 >> [v2:172.21.15.16:6836/2330551626,v1:172.21.15.16:6837/2330551626] conn(0x556ba709e930 0x556ba70be730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:35.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.184+0000 7f55277fe700 10 client.4446 ms_handle_connect on v2:172.21.15.16:6836/2330551626 2023-05-20T01:16:35.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.184+0000 7f554cb68700 1 --2- 172.21.15.16:0/3632196415 >> [v2:172.21.15.16:6836/2330551626,v1:172.21.15.16:6837/2330551626] conn(0x556ba709e930 0x556ba70be730 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ecb2280f960af96 server_cookie=5cf368d36d0335e in_seq=0 out_seq=0 2023-05-20T01:16:35.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.185+0000 7f55277fe700 1 -- 172.21.15.16:0/3632196415 <== mds.0 v2:172.21.15.16:6836/2330551626 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x556ba6fb2050 con 0x556ba709e930 2023-05-20T01:16:35.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.185+0000 7f55277fe700 10 client.4446 handle_command_reply: tid=0 2023-05-20T01:16:35.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.185+0000 7f555af58b80 1 client.4446 shutdown 2023-05-20T01:16:35.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.185+0000 7f555af58b80 1 -- 172.21.15.16:0/3632196415 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ba70c9400 msgr2=0x556ba709e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:35.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.185+0000 7f555af58b80 1 --2- 172.21.15.16:0/3632196415 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ba70c9400 0x556ba709e3f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5540054ef0 tx=0x7f554005c9d0 comp rx=0 tx=0).stop 2023-05-20T01:16:35.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.185+0000 7f555af58b80 1 -- 172.21.15.16:0/3632196415 shutdown_connections 2023-05-20T01:16:35.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.185+0000 7f555af58b80 1 --2- 172.21.15.16:0/3632196415 >> [v2:172.21.15.16:6836/2330551626,v1:172.21.15.16:6837/2330551626] conn(0x556ba709e930 0x556ba70be730 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:35.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.185+0000 7f555af58b80 1 --2- 172.21.15.16:0/3632196415 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ba70c9400 0x556ba709e3f0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:35.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.185+0000 7f555af58b80 1 --2- 172.21.15.16:0/3632196415 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556ba709c870 0x556ba70c1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:35.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.185+0000 7f55277fe700 0 client.4446 ms_handle_reset on v2:172.21.15.16:6836/2330551626 2023-05-20T01:16:35.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.185+0000 7f555af58b80 1 --2- 172.21.15.16:0/3632196415 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556ba70c5700 0x556ba709deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:35.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.185+0000 7f555af58b80 1 -- 172.21.15.16:0/3632196415 >> 172.21.15.16:0/3632196415 conn(0x556ba7093f30 msgr2=0x556ba70993e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:35.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.185+0000 7f555af58b80 1 -- 172.21.15.16:0/3632196415 shutdown_connections 2023-05-20T01:16:35.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.185+0000 7f555af58b80 1 -- 172.21.15.16:0/3632196415 wait complete. 2023-05-20T01:16:35.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.185+0000 7f555af58b80 20 client.4446 trim_cache size 0 max 16384 2023-05-20T01:16:35.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.203+0000 7f555af58b80 1 Processor -- start 2023-05-20T01:16:35.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.203+0000 7f555af58b80 1 -- start start 2023-05-20T01:16:35.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.203+0000 7f555af58b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556ba709c870 0x556ba70e4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:35.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.203+0000 7f555af58b80 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556ba70c5700 0x556ba709ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:35.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.203+0000 7f555af58b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ba70c9400 0x556ba709f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:35.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.203+0000 7f555af58b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ba709d810 con 0x556ba70c9400 2023-05-20T01:16:35.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.203+0000 7f555af58b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ba70a02d0 con 0x556ba709c870 2023-05-20T01:16:35.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.203+0000 7f555af58b80 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x556ba70c9e40 con 0x556ba70c5700 2023-05-20T01:16:35.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.204+0000 7f554d369700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ba70c9400 0x556ba709f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:35.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.204+0000 7f554d369700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ba70c9400 0x556ba709f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57652/0 (socket says 172.21.15.16:57652) 2023-05-20T01:16:35.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.204+0000 7f554d369700 1 -- 172.21.15.16:0/2405312848 learned_addr learned my addr 172.21.15.16:0/2405312848 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:35.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.204+0000 7f554d369700 1 -- 172.21.15.16:0/2405312848 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556ba70c5700 msgr2=0x556ba709ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:35.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.204+0000 7f554d369700 1 --2- 172.21.15.16:0/2405312848 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556ba70c5700 0x556ba709ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:35.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.204+0000 7f554d369700 1 -- 172.21.15.16:0/2405312848 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556ba709c870 msgr2=0x556ba70e4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:35.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.204+0000 7f554cb68700 1 --2- 172.21.15.16:0/2405312848 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556ba709c870 0x556ba70e4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:35.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.204+0000 7f554d369700 1 --2- 172.21.15.16:0/2405312848 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556ba709c870 0x556ba70e4630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:35.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.204+0000 7f554d369700 1 -- 172.21.15.16:0/2405312848 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556ba6fb5e90 con 0x556ba70c9400 2023-05-20T01:16:35.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.204+0000 7f554d369700 1 --2- 172.21.15.16:0/2405312848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ba70c9400 0x556ba709f470 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f5540077510 tx=0x7f554005c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb76a7cfc82f7424 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:35.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.204+0000 7f5532ffd700 1 -- 172.21.15.16:0/2405312848 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5540060800 con 0x556ba70c9400 2023-05-20T01:16:35.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.204+0000 7f5532ffd700 1 -- 172.21.15.16:0/2405312848 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5540077040 con 0x556ba70c9400 2023-05-20T01:16:35.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.204+0000 7f5532ffd700 1 -- 172.21.15.16:0/2405312848 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5540057ee0 con 0x556ba70c9400 2023-05-20T01:16:35.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.205+0000 7f555af58b80 1 -- 172.21.15.16:0/2405312848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ba70c9400 msgr2=0x556ba709f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:35.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.205+0000 7f555af58b80 1 --2- 172.21.15.16:0/2405312848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ba70c9400 0x556ba709f470 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f5540077510 tx=0x7f554005c000 comp rx=0 tx=0).stop 2023-05-20T01:16:35.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.205+0000 7f555af58b80 1 -- 172.21.15.16:0/2405312848 shutdown_connections 2023-05-20T01:16:35.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.205+0000 7f555af58b80 1 --2- 172.21.15.16:0/2405312848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ba70c9400 0x556ba709f470 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:35.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.205+0000 7f555af58b80 1 --2- 172.21.15.16:0/2405312848 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556ba709c870 0x556ba70e4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:35.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.205+0000 7f555af58b80 1 --2- 172.21.15.16:0/2405312848 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556ba70c5700 0x556ba709ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:35.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.205+0000 7f555af58b80 1 -- 172.21.15.16:0/2405312848 >> 172.21.15.16:0/2405312848 conn(0x556ba7093f30 msgr2=0x556ba7099380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:35.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.205+0000 7f555af58b80 1 -- 172.21.15.16:0/2405312848 shutdown_connections 2023-05-20T01:16:35.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.205+0000 7f555af58b80 1 -- 172.21.15.16:0/2405312848 wait complete. 2023-05-20T01:16:35.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.205+0000 7f555af58b80 1 -- start start 2023-05-20T01:16:35.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.205+0000 7f555af58b80 1 Processor -- start 2023-05-20T01:16:35.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.206+0000 7f555af58b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556ba709c870 0x556ba70c5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:35.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.206+0000 7f555af58b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ba70c5700 0x556ba70c6070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:35.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.206+0000 7f555af58b80 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556ba70c9400 0x556ba709f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:35.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.206+0000 7f555af58b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ba70f6650 con 0x556ba70c5700 2023-05-20T01:16:35.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.206+0000 7f555af58b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ba70b9da0 con 0x556ba709c870 2023-05-20T01:16:35.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.206+0000 7f555af58b80 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x556ba70daa80 con 0x556ba70c9400 2023-05-20T01:16:35.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.206+0000 7f555af58b80 10 client.0 _ll_register_callbacks cb 0x7f5540058440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-05-20T01:16:35.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.206+0000 7f5547fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ba70c5700 0x556ba70c6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:35.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.206+0000 7f5547fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ba70c5700 0x556ba70c6070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57656/0 (socket says 172.21.15.16:57656) 2023-05-20T01:16:35.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.206+0000 7f5547fff700 1 -- 172.21.15.16:0/883874717 learned_addr learned my addr 172.21.15.16:0/883874717 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:35.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.206+0000 7f554cb68700 1 --2- 172.21.15.16:0/883874717 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556ba709c870 0x556ba70c5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:35.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.206+0000 7f554d369700 1 --2- 172.21.15.16:0/883874717 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556ba70c9400 0x556ba709f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:35.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.207+0000 7f5547fff700 1 -- 172.21.15.16:0/883874717 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556ba70c9400 msgr2=0x556ba709f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:35.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.207+0000 7f5547fff700 1 --2- 172.21.15.16:0/883874717 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556ba70c9400 0x556ba709f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:35.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.207+0000 7f5547fff700 1 -- 172.21.15.16:0/883874717 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556ba709c870 msgr2=0x556ba70c5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:35.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.207+0000 7f5547fff700 1 --2- 172.21.15.16:0/883874717 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556ba709c870 0x556ba70c5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:35.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.207+0000 7f5547fff700 1 -- 172.21.15.16:0/883874717 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556ba6fb6140 con 0x556ba70c5700 2023-05-20T01:16:35.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.207+0000 7f5547fff700 1 --2- 172.21.15.16:0/883874717 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ba70c5700 0x556ba70c6070 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f553400b6a0 tx=0x7f5534004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94ea815c0bfddce3 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:35.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.207+0000 7f55277fe700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-05-20T01:16:35.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.207+0000 7f55277fe700 1 -- 172.21.15.16:0/883874717 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5534027070 con 0x556ba70c5700 2023-05-20T01:16:35.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.207+0000 7f55277fe700 1 -- 172.21.15.16:0/883874717 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f55340038b0 con 0x556ba70c5700 2023-05-20T01:16:35.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.207+0000 7f555af58b80 1 -- 172.21.15.16:0/883874717 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f553400dd80 con 0x556ba70c5700 2023-05-20T01:16:35.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.207+0000 7f55277fe700 1 -- 172.21.15.16:0/883874717 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) v2 ==== 24+0+0 (secure 0 0 0) 0x7f553401e040 con 0x556ba70c5700 2023-05-20T01:16:35.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.207+0000 7f555af58b80 10 client.4452 fetch_fsmap learned FSMap version 6 2023-05-20T01:16:35.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.207+0000 7f555af58b80 1 -- 172.21.15.16:0/883874717 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) v3 -- 0x556ba6e06c10 con 0x556ba70c5700 2023-05-20T01:16:35.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.207+0000 7f55277fe700 1 -- 172.21.15.16:0/883874717 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f553400dd80 con 0x556ba70c5700 2023-05-20T01:16:35.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.207+0000 7f55277fe700 1 -- 172.21.15.16:0/883874717 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 6) v1 ==== 3122+0+0 (secure 0 0 0) 0x7f553402a020 con 0x556ba70c5700 2023-05-20T01:16:35.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.207+0000 7f555af58b80 10 client.4452 fetch_fsmap finished waiting for FSMap version 6 2023-05-20T01:16:35.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.207+0000 7f555af58b80 10 client.4452 resolve_mds: resolved name 'a' to daemon mds.a 2023-05-20T01:16:35.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.207+0000 7f555af58b80 20 client.4452 populate_metadata read hostname 'smithi016' 2023-05-20T01:16:35.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.207+0000 7f555af58b80 1 --2- 172.21.15.16:0/883874717 >> [v2:172.21.15.16:6836/2330551626,v1:172.21.15.16:6837/2330551626] conn(0x556ba70c2930 0x556ba70c2d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:35.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.207+0000 7f555af58b80 4 client.4452 mds_command: new command op to 4299 tid=0[{"prefix": "status"}] 2023-05-20T01:16:35.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.207+0000 7f555af58b80 1 -- 172.21.15.16:0/883874717 --> [v2:172.21.15.16:6836/2330551626,v1:172.21.15.16:6837/2330551626] -- command(tid 0: {"prefix": "status"}) v1 -- 0x556ba6fbe610 con 0x556ba70c2930 2023-05-20T01:16:35.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.207+0000 7f554cb68700 1 --2- 172.21.15.16:0/883874717 >> [v2:172.21.15.16:6836/2330551626,v1:172.21.15.16:6837/2330551626] conn(0x556ba70c2930 0x556ba70c2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:35.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.208+0000 7f55277fe700 10 client.4452 ms_handle_connect on v2:172.21.15.16:6836/2330551626 2023-05-20T01:16:35.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.208+0000 7f554cb68700 1 --2- 172.21.15.16:0/883874717 >> [v2:172.21.15.16:6836/2330551626,v1:172.21.15.16:6837/2330551626] conn(0x556ba70c2930 0x556ba70c2d00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbd10538c01a7b52 server_cookie=b8d68eee392c273b in_seq=0 out_seq=0 2023-05-20T01:16:35.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.208+0000 7f55277fe700 1 -- 172.21.15.16:0/883874717 <== mds.0 v2:172.21.15.16:6836/2330551626 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x556ba6fbe610 con 0x556ba70c2930 2023-05-20T01:16:35.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.208+0000 7f55277fe700 10 client.4452 handle_command_reply: tid=0 2023-05-20T01:16:35.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.208+0000 7f555af58b80 1 client.4452 shutdown 2023-05-20T01:16:35.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.208+0000 7f555af58b80 1 -- 172.21.15.16:0/883874717 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ba70c5700 msgr2=0x556ba70c6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:35.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.208+0000 7f555af58b80 1 --2- 172.21.15.16:0/883874717 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ba70c5700 0x556ba70c6070 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f553400b6a0 tx=0x7f5534004750 comp rx=0 tx=0).stop 2023-05-20T01:16:35.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.208+0000 7f555af58b80 1 -- 172.21.15.16:0/883874717 shutdown_connections 2023-05-20T01:16:35.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.208+0000 7f555af58b80 1 --2- 172.21.15.16:0/883874717 >> [v2:172.21.15.16:6836/2330551626,v1:172.21.15.16:6837/2330551626] conn(0x556ba70c2930 0x556ba70c2d00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:35.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.208+0000 7f555af58b80 1 --2- 172.21.15.16:0/883874717 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556ba70c9400 0x556ba709f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:35.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.208+0000 7f55277fe700 0 client.4452 ms_handle_reset on v2:172.21.15.16:6836/2330551626 2023-05-20T01:16:35.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.208+0000 7f555af58b80 1 --2- 172.21.15.16:0/883874717 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556ba709c870 0x556ba70c5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:35.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.208+0000 7f555af58b80 1 --2- 172.21.15.16:0/883874717 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ba70c5700 0x556ba70c6070 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:35.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.208+0000 7f555af58b80 1 -- 172.21.15.16:0/883874717 >> 172.21.15.16:0/883874717 conn(0x556ba7093f30 msgr2=0x556ba7099380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:35.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.208+0000 7f555af58b80 1 -- 172.21.15.16:0/883874717 shutdown_connections 2023-05-20T01:16:35.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.208+0000 7f555af58b80 1 -- 172.21.15.16:0/883874717 wait complete. 2023-05-20T01:16:35.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.209+0000 7f555af58b80 20 client.4452 trim_cache size 0 max 16384 2023-05-20T01:16:35.222 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-05-20T01:16:35.222 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "c2245cc5-ecab-4087-a59b-bc81db901676", 2023-05-20T01:16:35.222 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2023-05-20T01:16:35.222 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 4299, 2023-05-20T01:16:35.222 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2023-05-20T01:16:35.222 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2023-05-20T01:16:35.222 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2023-05-20T01:16:35.223 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 6.8914255779999998, 2023-05-20T01:16:35.223 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 6, 2023-05-20T01:16:35.223 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 25, 2023-05-20T01:16:35.223 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 24, 2023-05-20T01:16:35.223 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 12.259240256 2023-05-20T01:16:35.223 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-05-20T01:16:35.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.209+0000 7f554edcc700 1 -- 172.21.15.16:0/63427141 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f552c06ace0 msgr2=0x7f552c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:35.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.209+0000 7f554edcc700 1 --2- 172.21.15.16:0/63427141 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f552c06ace0 0x7f552c06d190 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f553c007fd0 tx=0x7f553c009040 comp rx=0 tx=0).stop 2023-05-20T01:16:35.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.209+0000 7f554edcc700 1 -- 172.21.15.16:0/63427141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5548157190 msgr2=0x7f5548158d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:35.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.209+0000 7f554edcc700 1 --2- 172.21.15.16:0/63427141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5548157190 0x7f5548158d10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5540007e40 tx=0x7f554000e3c0 comp rx=0 tx=0).stop 2023-05-20T01:16:35.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.209+0000 7f554edcc700 1 -- 172.21.15.16:0/63427141 shutdown_connections 2023-05-20T01:16:35.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.209+0000 7f554edcc700 1 --2- 172.21.15.16:0/63427141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5548157190 0x7f5548158d10 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:35.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.209+0000 7f554edcc700 1 --2- 172.21.15.16:0/63427141 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5548071900 0x7f5548149120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:35.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.209+0000 7f554edcc700 1 --2- 172.21.15.16:0/63427141 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f552c06ace0 0x7f552c06d190 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:35.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.209+0000 7f554edcc700 1 --2- 172.21.15.16:0/63427141 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f55480ccec0 0x7f55481587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:35.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.209+0000 7f554edcc700 1 -- 172.21.15.16:0/63427141 >> 172.21.15.16:0/63427141 conn(0x7f55480cffb0 msgr2=0x7f55480cc7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:35.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.209+0000 7f554edcc700 1 -- 172.21.15.16:0/63427141 shutdown_connections 2023-05-20T01:16:35.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.209+0000 7f554edcc700 1 -- 172.21.15.16:0/63427141 wait complete. 2023-05-20T01:16:35.226 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-05-20T01:16:35.227 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-05-20T01:16:35.228 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-05-20T01:16:35.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.355+0000 7fa3fe85a700 1 Processor -- start 2023-05-20T01:16:35.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.355+0000 7fa3fe85a700 1 -- start start 2023-05-20T01:16:35.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.356+0000 7fa3fe85a700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa3f80b4750 0x7fa3f80d2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:35.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.356+0000 7fa3fe85a700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3f80b3b50 con 0x7fa3f80cd970 2023-05-20T01:16:35.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.356+0000 7fa3fe85a700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3f80d0b90 con 0x7fa3f80cfc50 2023-05-20T01:16:35.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.356+0000 7fa3fe85a700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3f80cfad0 con 0x7fa3f80b4750 2023-05-20T01:16:35.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.356+0000 7fa3fcdf7700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fa3f80cd970 0x7fa3f80cdd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:40958/0 (socket says 172.21.15.16:40958) 2023-05-20T01:16:35.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.356+0000 7fa3fcdf7700 1 -- 172.21.15.16:0/2585981654 learned_addr learned my addr 172.21.15.16:0/2585981654 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:16:35.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.356+0000 7fa3f77fe700 1 --2- 172.21.15.16:0/2585981654 >> v2:172.21.15.16:3300/0 conn(0x7fa3f80b4750 0x7fa3f80d2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:35.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.356+0000 7fa3f77fe700 1 -- 172.21.15.16:0/2585981654 >> v1:172.21.15.132:6789/0 conn(0x7fa3f80cfc50 legacy=0x7fa3f80d0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:35.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.356+0000 7fa3f77fe700 1 -- 172.21.15.16:0/2585981654 >> v1:172.21.15.16:6789/0 conn(0x7fa3f80cd970 legacy=0x7fa3f80cdd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:35.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.356+0000 7fa3f77fe700 1 -- 172.21.15.16:0/2585981654 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3f815d0b0 con 0x7fa3f80b4750 2023-05-20T01:16:35.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.356+0000 7fa3f6ffd700 1 -- 172.21.15.16:0/2585981654 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1130711362 0 0) 0x7fa3f80b3b50 con 0x7fa3f80cd970 2023-05-20T01:16:35.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.357+0000 7fa3f77fe700 1 --2- 172.21.15.16:0/2585981654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3f80b4750 0x7fa3f80d2c10 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa3e0009840 tx=0x7fa3e0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e1e8d5a772be4dc server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:35.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.357+0000 7fa3f6ffd700 1 -- 172.21.15.16:0/2585981654 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3e0004230 con 0x7fa3f80b4750 2023-05-20T01:16:35.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.357+0000 7fa3f6ffd700 1 -- 172.21.15.16:0/2585981654 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3e0004390 con 0x7fa3f80b4750 2023-05-20T01:16:35.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.357+0000 7fa3f6ffd700 1 -- 172.21.15.16:0/2585981654 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3e00036a0 con 0x7fa3f80b4750 2023-05-20T01:16:35.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.357+0000 7fa3fe85a700 1 -- 172.21.15.16:0/2585981654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3f80b4750 msgr2=0x7fa3f80d2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:35.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.357+0000 7fa3fe85a700 1 --2- 172.21.15.16:0/2585981654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3f80b4750 0x7fa3f80d2c10 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa3e0009840 tx=0x7fa3e0009b50 comp rx=0 tx=0).stop 2023-05-20T01:16:35.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.357+0000 7fa3fe85a700 1 -- 172.21.15.16:0/2585981654 shutdown_connections 2023-05-20T01:16:35.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.357+0000 7fa3fe85a700 1 --2- 172.21.15.16:0/2585981654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3f80b4750 0x7fa3f80d2c10 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:35.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.357+0000 7fa3fe85a700 1 -- 172.21.15.16:0/2585981654 >> 172.21.15.16:0/2585981654 conn(0x7fa3f809c590 msgr2=0x7fa3f80d3240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:35.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.357+0000 7fa3fe85a700 1 -- 172.21.15.16:0/2585981654 shutdown_connections 2023-05-20T01:16:35.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.358+0000 7fa3fe85a700 1 -- 172.21.15.16:0/2585981654 wait complete. 2023-05-20T01:16:35.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.359+0000 7fa3fe85a700 1 Processor -- start 2023-05-20T01:16:35.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.359+0000 7fa3fe85a700 1 -- start start 2023-05-20T01:16:35.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.360+0000 7fa3fe85a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3f80b4750 0x7fa3f81f3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:35.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.360+0000 7fa3fe85a700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa3f80cd970 0x7fa3f815a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:35.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.360+0000 7fa3fe85a700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa3f80cfc50 0x7fa3f815a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:35.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.360+0000 7fa3fe85a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f80b62a0 con 0x7fa3f80b4750 2023-05-20T01:16:35.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.360+0000 7fa3fe85a700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f8159ff0 con 0x7fa3f80cfc50 2023-05-20T01:16:35.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.360+0000 7fa3fe85a700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f8159100 con 0x7fa3f80cd970 2023-05-20T01:16:35.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.360+0000 7fa3f7fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3f80b4750 0x7fa3f81f3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:35.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.360+0000 7fa3f7fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3f80b4750 0x7fa3f81f3690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57666/0 (socket says 172.21.15.16:57666) 2023-05-20T01:16:35.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.360+0000 7fa3f7fff700 1 -- 172.21.15.16:0/2898897404 learned_addr learned my addr 172.21.15.16:0/2898897404 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:35.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.360+0000 7fa3f77fe700 1 --2- 172.21.15.16:0/2898897404 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa3f80cd970 0x7fa3f815a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:35.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.360+0000 7fa3fcdf7700 1 --2- 172.21.15.16:0/2898897404 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa3f80cfc50 0x7fa3f815a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:35.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.360+0000 7fa3f7fff700 1 -- 172.21.15.16:0/2898897404 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa3f80cd970 msgr2=0x7fa3f815a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:35.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.360+0000 7fa3f7fff700 1 --2- 172.21.15.16:0/2898897404 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa3f80cd970 0x7fa3f815a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:35.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.360+0000 7fa3f7fff700 1 -- 172.21.15.16:0/2898897404 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa3f80cfc50 msgr2=0x7fa3f815a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:35.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.360+0000 7fa3f7fff700 1 --2- 172.21.15.16:0/2898897404 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa3f80cfc50 0x7fa3f815a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:35.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.360+0000 7fa3f7fff700 1 -- 172.21.15.16:0/2898897404 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3e00094a0 con 0x7fa3f80b4750 2023-05-20T01:16:35.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.360+0000 7fa3fcdf7700 1 --2- 172.21.15.16:0/2898897404 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa3f80cfc50 0x7fa3f815a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:16:35.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.360+0000 7fa3f77fe700 1 --2- 172.21.15.16:0/2898897404 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa3f80cd970 0x7fa3f815a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:16:35.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.360+0000 7fa3f7fff700 1 --2- 172.21.15.16:0/2898897404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3f80b4750 0x7fa3f81f3690 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa3e800b4a0 tx=0x7fa3e800b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:35.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.360+0000 7fa3f4ff9700 1 -- 172.21.15.16:0/2898897404 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3e8011830 con 0x7fa3f80b4750 2023-05-20T01:16:35.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.360+0000 7fa3f4ff9700 1 -- 172.21.15.16:0/2898897404 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3e8011990 con 0x7fa3f80b4750 2023-05-20T01:16:35.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.360+0000 7fa3fe85a700 1 -- 172.21.15.16:0/2898897404 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3f80d4000 con 0x7fa3f80b4750 2023-05-20T01:16:35.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.360+0000 7fa3f4ff9700 1 -- 172.21.15.16:0/2898897404 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3e8011d40 con 0x7fa3f80b4750 2023-05-20T01:16:35.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.360+0000 7fa3fe85a700 1 -- 172.21.15.16:0/2898897404 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3f80b4380 con 0x7fa3f80b4750 2023-05-20T01:16:35.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.361+0000 7fa3fe85a700 1 -- 172.21.15.16:0/2898897404 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3f80b3da0 con 0x7fa3f80b4750 2023-05-20T01:16:35.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.362+0000 7fa3f4ff9700 1 -- 172.21.15.16:0/2898897404 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa3e801a400 con 0x7fa3f80b4750 2023-05-20T01:16:35.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.362+0000 7fa3f4ff9700 1 --2- 172.21.15.16:0/2898897404 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa3d806ace0 0x7fa3d806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:35.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.362+0000 7fa3f77fe700 1 --2- 172.21.15.16:0/2898897404 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa3d806ace0 0x7fa3d806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:35.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.362+0000 7fa3f4ff9700 1 -- 172.21.15.16:0/2898897404 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fa3e80894f0 con 0x7fa3f80b4750 2023-05-20T01:16:35.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.363+0000 7fa3f77fe700 1 --2- 172.21.15.16:0/2898897404 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa3d806ace0 0x7fa3d806d190 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa3e00090b0 tx=0x7fa3e0008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:35.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.367+0000 7fa3f4ff9700 1 -- 172.21.15.16:0/2898897404 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fa3e80542b0 con 0x7fa3f80b4750 2023-05-20T01:16:35.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.616+0000 7fa3fe85a700 1 -- 172.21.15.16:0/2898897404 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fa3f806b130 con 0x7fa3f80b4750 2023-05-20T01:16:35.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.616+0000 7fa3f4ff9700 1 -- 172.21.15.16:0/2898897404 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) v1 ==== 53+0+88 (secure 0 0 0) 0x7fa3e80578d0 con 0x7fa3f80b4750 2023-05-20T01:16:35.620 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-05-20T01:16:35.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.618+0000 7fa3fe85a700 1 -- 172.21.15.16:0/2898897404 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa3d806ace0 msgr2=0x7fa3d806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:35.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.618+0000 7fa3fe85a700 1 --2- 172.21.15.16:0/2898897404 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa3d806ace0 0x7fa3d806d190 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa3e00090b0 tx=0x7fa3e0008030 comp rx=0 tx=0).stop 2023-05-20T01:16:35.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.618+0000 7fa3fe85a700 1 -- 172.21.15.16:0/2898897404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3f80b4750 msgr2=0x7fa3f81f3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:35.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.618+0000 7fa3fe85a700 1 --2- 172.21.15.16:0/2898897404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3f80b4750 0x7fa3f81f3690 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa3e800b4a0 tx=0x7fa3e800b860 comp rx=0 tx=0).stop 2023-05-20T01:16:35.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.618+0000 7fa3fe85a700 1 -- 172.21.15.16:0/2898897404 shutdown_connections 2023-05-20T01:16:35.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.618+0000 7fa3fe85a700 1 --2- 172.21.15.16:0/2898897404 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa3f80cfc50 0x7fa3f815a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:35.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.618+0000 7fa3fe85a700 1 --2- 172.21.15.16:0/2898897404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3f80b4750 0x7fa3f81f3690 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:35.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.618+0000 7fa3fe85a700 1 --2- 172.21.15.16:0/2898897404 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa3d806ace0 0x7fa3d806d190 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:35.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.618+0000 7fa3fe85a700 1 --2- 172.21.15.16:0/2898897404 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa3f80cd970 0x7fa3f815a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:35.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.618+0000 7fa3fe85a700 1 -- 172.21.15.16:0/2898897404 >> 172.21.15.16:0/2898897404 conn(0x7fa3f809c590 msgr2=0x7fa3f80cead0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:35.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.618+0000 7fa3fe85a700 1 -- 172.21.15.16:0/2898897404 shutdown_connections 2023-05-20T01:16:35.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:35.618+0000 7fa3fe85a700 1 -- 172.21.15.16:0/2898897404 wait complete. 2023-05-20T01:16:35.634 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-05-20T01:16:35.634 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-05-20T01:16:35.635 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2023-05-20T01:16:35.635 INFO:tasks.cephfs.mount:self.client.name = client.admin 2023-05-20T01:16:35.635 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-05-20T01:16:35.635 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-05-20T01:16:35.635 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-05-20T01:16:35.636 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-05-20T01:16:35.636 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2023-05-20T01:16:35.636 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:16:35.637 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2023-05-20T01:16:35.654 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-05-20T01:16:35.654 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-05-20T01:16:35.654 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2023-05-20T01:16:35.654 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-05-20T01:16:35.654 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2023-05-20T01:16:35.655 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-05-20T01:16:35.655 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-05-20T01:16:35.655 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2023-05-20T01:16:35.655 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-05-20T01:16:35.655 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2023-05-20T01:16:35.655 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-05-20T01:16:35.656 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2023-05-20T01:16:35.656 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-05-20T01:16:35.656 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 42438sec preferred_lft 42438sec 2023-05-20T01:16:35.656 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link 2023-05-20T01:16:35.656 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-05-20T01:16:35.656 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-05-20T01:16:35.656 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2023-05-20T01:16:35.657 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-05-20T01:16:35.657 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:16:35.658 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2023-05-20T01:16:35.658 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2023-05-20T01:16:35.659 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link add name ceph-brx type bridge 2023-05-20T01:16:35.659 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip addr flush dev ceph-brx 2023-05-20T01:16:35.660 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set ceph-brx up 2023-05-20T01:16:35.661 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-05-20T01:16:35.661 DEBUG:teuthology.orchestra.run.smithi016:> ') 2023-05-20T01:16:35.827 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:16:35.828 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-05-20T01:16:35.857 INFO:teuthology.orchestra.run.smithi016.stdout:1 2023-05-20T01:16:35.858 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:16:35.859 DEBUG:teuthology.orchestra.run.smithi016:> route 2023-05-20T01:16:35.926 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2023-05-20T01:16:35.927 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-05-20T01:16:35.927 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-05-20T01:16:35.927 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-05-20T01:16:35.927 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-05-20T01:16:35.928 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:16:35.928 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2023-05-20T01:16:35.928 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2023-05-20T01:16:35.929 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-05-20T01:16:35.929 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-05-20T01:16:35.929 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-05-20T01:16:35.929 DEBUG:teuthology.orchestra.run.smithi016:> ') 2023-05-20T01:16:36.193 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:16:36.194 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2023-05-20T01:16:36.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:16:36.211 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list-id 2023-05-20T01:16:36.268 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:16:36.269 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2023-05-20T01:16:36.269 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2023-05-20T01:16:36.269 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-05-20T01:16:36.269 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2023-05-20T01:16:36.270 DEBUG:teuthology.orchestra.run.smithi016:> ') 2023-05-20T01:16:36.366 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2023-05-20T01:16:36.367 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:16:36.367 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2023-05-20T01:16:36.368 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2023-05-20T01:16:36.368 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2023-05-20T01:16:36.369 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-05-20T01:16:36.370 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2023-05-20T01:16:36.370 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2023-05-20T01:16:36.371 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2023-05-20T01:16:36.372 DEBUG:teuthology.orchestra.run.smithi016:> ') 2023-05-20T01:16:36.577 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:16:36.577 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2023-05-20T01:16:36.577 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2023-05-20T01:16:36.578 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.0 up 2023-05-20T01:16:36.578 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set dev brx.0 master ceph-brx 2023-05-20T01:16:36.578 DEBUG:teuthology.orchestra.run.smithi016:> ') 2023-05-20T01:16:36.691 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2023-05-20T01:16:36.691 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-05-20T01:16:36.691 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2023-05-20T01:16:36.718 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2023-05-20T01:16:36.718 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-05-20T01:16:36.719 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2023-05-20T01:16:36.775 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2023-05-20T01:16:36.843 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-05-20T01:16:36.844 DEBUG:teuthology.orchestra.run.smithi016:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-05-20T01:16:36.910 INFO:teuthology.orchestra.run.smithi016.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-05-20T01:16:36.911 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-05-20T01:16:36.912 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:16:36.912 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2023-05-20T01:16:36.970 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-05-20T01:16:36.970 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2023-05-20T01:16:37.011 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2023-05-20T01:16:37.039 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-05-20T01:16:37.040 DEBUG:teuthology.orchestra.run.smithi016:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-05-20T01:16:37.106 INFO:teuthology.orchestra.run.smithi016.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-05-20T01:16:37.107 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-05-20T01:16:37.108 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:16:37.108 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2023-05-20T01:16:37.113 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi016.stderr:2023-05-20T01:16:37.109+0000 7f8e22d635c0 -1 init, newargv = 0x560b6be93020 newargc=15 2023-05-20T01:16:37.113 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi016.stderr:ceph-fuse[105788]: starting ceph client 2023-05-20T01:16:37.120 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi016.stderr:ceph-fuse[105788]: starting fuse 2023-05-20T01:16:37.130 INFO:teuthology.orchestra.run.smithi016.stdout:47 2023-05-20T01:16:37.131 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [47] 2023-05-20T01:16:37.131 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-05-20T01:16:37.132 DEBUG:teuthology.orchestra.run.smithi016:> import glob 2023-05-20T01:16:37.133 DEBUG:teuthology.orchestra.run.smithi016:> import re 2023-05-20T01:16:37.133 DEBUG:teuthology.orchestra.run.smithi016:> import os 2023-05-20T01:16:37.134 DEBUG:teuthology.orchestra.run.smithi016:> import subprocess 2023-05-20T01:16:37.134 DEBUG:teuthology.orchestra.run.smithi016:> 2023-05-20T01:16:37.135 DEBUG:teuthology.orchestra.run.smithi016:> def _find_admin_socket(client_name): 2023-05-20T01:16:37.136 DEBUG:teuthology.orchestra.run.smithi016:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2023-05-20T01:16:37.136 DEBUG:teuthology.orchestra.run.smithi016:> files = glob.glob(asok_path) 2023-05-20T01:16:37.137 DEBUG:teuthology.orchestra.run.smithi016:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2023-05-20T01:16:37.138 DEBUG:teuthology.orchestra.run.smithi016:> 2023-05-20T01:16:37.138 DEBUG:teuthology.orchestra.run.smithi016:> # Given a non-glob path, it better be there 2023-05-20T01:16:37.138 DEBUG:teuthology.orchestra.run.smithi016:> if "*" not in asok_path: 2023-05-20T01:16:37.139 DEBUG:teuthology.orchestra.run.smithi016:> assert(len(files) == 1) 2023-05-20T01:16:37.139 DEBUG:teuthology.orchestra.run.smithi016:> return files[0] 2023-05-20T01:16:37.139 DEBUG:teuthology.orchestra.run.smithi016:> 2023-05-20T01:16:37.139 DEBUG:teuthology.orchestra.run.smithi016:> for f in files: 2023-05-20T01:16:37.140 DEBUG:teuthology.orchestra.run.smithi016:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-05-20T01:16:37.140 DEBUG:teuthology.orchestra.run.smithi016:> if os.path.exists("/proc/{0}".format(pid)): 2023-05-20T01:16:37.140 DEBUG:teuthology.orchestra.run.smithi016:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-05-20T01:16:37.141 DEBUG:teuthology.orchestra.run.smithi016:> contents = proc_f.read() 2023-05-20T01:16:37.141 DEBUG:teuthology.orchestra.run.smithi016:> if mountpoint in contents: 2023-05-20T01:16:37.141 DEBUG:teuthology.orchestra.run.smithi016:> return f 2023-05-20T01:16:37.141 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-05-20T01:16:37.142 DEBUG:teuthology.orchestra.run.smithi016:> 2023-05-20T01:16:37.142 DEBUG:teuthology.orchestra.run.smithi016:> print(_find_admin_socket("client.admin")) 2023-05-20T01:16:37.142 DEBUG:teuthology.orchestra.run.smithi016:> ' 2023-05-20T01:16:37.255 INFO:teuthology.orchestra.run.smithi016.stdout:/var/run/ceph/ceph-client.admin.105788.asok 2023-05-20T01:16:37.437 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.105788.asok 2023-05-20T01:16:37.438 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:16:37.438 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.105788.asok status 2023-05-20T01:16:37.599 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-05-20T01:16:37.600 INFO:teuthology.orchestra.run.smithi016.stdout: "metadata": { 2023-05-20T01:16:37.600 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_sha1": "5ca1fdcfd7e6cec340f637cba9f0087e6db9c2c1", 2023-05-20T01:16:37.600 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_version": "ceph version 18.0.0-3996-g5ca1fdcf (5ca1fdcfd7e6cec340f637cba9f0087e6db9c2c1) reef (dev)", 2023-05-20T01:16:37.600 INFO:teuthology.orchestra.run.smithi016.stdout: "entity_id": "admin", 2023-05-20T01:16:37.600 INFO:teuthology.orchestra.run.smithi016.stdout: "hostname": "smithi016", 2023-05-20T01:16:37.600 INFO:teuthology.orchestra.run.smithi016.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2023-05-20T01:16:37.601 INFO:teuthology.orchestra.run.smithi016.stdout: "pid": "105788", 2023-05-20T01:16:37.601 INFO:teuthology.orchestra.run.smithi016.stdout: "root": "/" 2023-05-20T01:16:37.601 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-05-20T01:16:37.601 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_count": 0, 2023-05-20T01:16:37.601 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_pinned_count": 0, 2023-05-20T01:16:37.601 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 4470, 2023-05-20T01:16:37.601 INFO:teuthology.orchestra.run.smithi016.stdout: "inst": { 2023-05-20T01:16:37.601 INFO:teuthology.orchestra.run.smithi016.stdout: "name": { 2023-05-20T01:16:37.602 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2023-05-20T01:16:37.602 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 4470 2023-05-20T01:16:37.602 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-05-20T01:16:37.602 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2023-05-20T01:16:37.602 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2023-05-20T01:16:37.602 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.1:0", 2023-05-20T01:16:37.602 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 4128709911 2023-05-20T01:16:37.603 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:16:37.603 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-05-20T01:16:37.603 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2023-05-20T01:16:37.603 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2023-05-20T01:16:37.603 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.1:0", 2023-05-20T01:16:37.604 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 4128709911 2023-05-20T01:16:37.604 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-05-20T01:16:37.604 INFO:teuthology.orchestra.run.smithi016.stdout: "inst_str": "client.4470 192.168.0.1:0/4128709911", 2023-05-20T01:16:37.604 INFO:teuthology.orchestra.run.smithi016.stdout: "addr_str": "192.168.0.1:0/4128709911", 2023-05-20T01:16:37.605 INFO:teuthology.orchestra.run.smithi016.stdout: "inode_count": 1, 2023-05-20T01:16:37.605 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_epoch": 6, 2023-05-20T01:16:37.605 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch": 25, 2023-05-20T01:16:37.605 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch_barrier": 0, 2023-05-20T01:16:37.606 INFO:teuthology.orchestra.run.smithi016.stdout: "blocklisted": false, 2023-05-20T01:16:37.606 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs" 2023-05-20T01:16:37.606 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-05-20T01:16:37.609 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:16:37.610 DEBUG:teuthology.orchestra.run.smithi016:> stat --file-system '--printf=%T 2023-05-20T01:16:37.610 DEBUG:teuthology.orchestra.run.smithi016:> ' -- /home/ubuntu/cephtest/mnt.admin 2023-05-20T01:16:37.627 INFO:teuthology.orchestra.run.smithi016.stdout:fuseblk 2023-05-20T01:16:37.627 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2023-05-20T01:16:37.628 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:16:37.628 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2023-05-20T01:16:37.695 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:16:37.695 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2023-05-20T01:16:37.793 INFO:teuthology.orchestra.run.smithi016.stdout:# file: . 2023-05-20T01:16:37.793 INFO:teuthology.orchestra.run.smithi016.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2023-05-20T01:16:37.793 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:16:37.794 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-05-20T01:16:37.794 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2023-05-20T01:16:37.812 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-20T01:16:37.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:37.974+0000 7f0d9a142700 1 Processor -- start 2023-05-20T01:16:37.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:37.974+0000 7f0d9a142700 1 -- start start 2023-05-20T01:16:37.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:37.975+0000 7f0d9a142700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0d9415a910 0x7f0d940cac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:37.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:37.975+0000 7f0d9a142700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d9415ace0 con 0x7f0d940cb150 2023-05-20T01:16:37.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:37.975+0000 7f0d9a142700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d940723b0 con 0x7f0d940b1020 2023-05-20T01:16:37.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:37.975+0000 7f0d9a142700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d940cb5c0 con 0x7f0d9415a910 2023-05-20T01:16:37.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:37.975+0000 7f0d937fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0d9415a910 0x7f0d940cac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:37.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:37.975+0000 7f0d92ffd700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f0d940cb150 0x7f0d940b0910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:40994/0 (socket says 172.21.15.16:40994) 2023-05-20T01:16:37.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:37.975+0000 7f0d937fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0d9415a910 0x7f0d940cac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57698/0 (socket says 172.21.15.16:57698) 2023-05-20T01:16:37.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:37.975+0000 7f0d92ffd700 1 -- 172.21.15.16:0/2425124712 learned_addr learned my addr 172.21.15.16:0/2425124712 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:16:37.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:37.975+0000 7f0d937fe700 1 -- 172.21.15.16:0/2425124712 >> v1:172.21.15.132:6789/0 conn(0x7f0d940b1020 legacy=0x7f0d940c94e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:37.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:37.975+0000 7f0d937fe700 1 -- 172.21.15.16:0/2425124712 >> v1:172.21.15.16:6789/0 conn(0x7f0d940cb150 legacy=0x7f0d940b0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:37.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:37.975+0000 7f0d937fe700 1 -- 172.21.15.16:0/2425124712 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d9406ba70 con 0x7f0d9415a910 2023-05-20T01:16:37.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:37.975+0000 7f0d937fe700 1 --2- 172.21.15.16:0/2425124712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d9415a910 0x7f0d940cac10 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f0d88009840 tx=0x7f0d88009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4643914ce31b9d5 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:37.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:37.975+0000 7f0d927fc700 1 -- 172.21.15.16:0/2425124712 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d88004230 con 0x7f0d9415a910 2023-05-20T01:16:37.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:37.975+0000 7f0d927fc700 1 -- 172.21.15.16:0/2425124712 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d88004390 con 0x7f0d9415a910 2023-05-20T01:16:37.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:37.975+0000 7f0d927fc700 1 -- 172.21.15.16:0/2425124712 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d880036a0 con 0x7f0d9415a910 2023-05-20T01:16:37.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:37.976+0000 7f0d9a142700 1 -- 172.21.15.16:0/2425124712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d9415a910 msgr2=0x7f0d940cac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:37.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:37.976+0000 7f0d9a142700 1 --2- 172.21.15.16:0/2425124712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d9415a910 0x7f0d940cac10 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f0d88009840 tx=0x7f0d88009b50 comp rx=0 tx=0).stop 2023-05-20T01:16:37.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:37.976+0000 7f0d9a142700 1 -- 172.21.15.16:0/2425124712 shutdown_connections 2023-05-20T01:16:37.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:37.976+0000 7f0d9a142700 1 --2- 172.21.15.16:0/2425124712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d9415a910 0x7f0d940cac10 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:37.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:37.976+0000 7f0d9a142700 1 -- 172.21.15.16:0/2425124712 >> 172.21.15.16:0/2425124712 conn(0x7f0d940cb910 msgr2=0x7f0d940cbd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:37.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:37.976+0000 7f0d9a142700 1 -- 172.21.15.16:0/2425124712 shutdown_connections 2023-05-20T01:16:37.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:37.976+0000 7f0d9a142700 1 -- 172.21.15.16:0/2425124712 wait complete. 2023-05-20T01:16:37.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:37.977+0000 7f0d9a142700 1 Processor -- start 2023-05-20T01:16:37.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:37.978+0000 7f0d9a142700 1 -- start start 2023-05-20T01:16:37.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:37.978+0000 7f0d9a142700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0d940b1020 0x7f0d9413ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:37.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:37.978+0000 7f0d9a142700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d940cb150 0x7f0d940c29c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:37.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:37.978+0000 7f0d9a142700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0d9415a910 0x7f0d940cddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:37.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:37.978+0000 7f0d9a142700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d940cd960 con 0x7f0d940cb150 2023-05-20T01:16:37.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:37.978+0000 7f0d9a142700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d94145590 con 0x7f0d940b1020 2023-05-20T01:16:37.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:37.978+0000 7f0d9a142700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d940c5880 con 0x7f0d9415a910 2023-05-20T01:16:37.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:37.978+0000 7f0d92ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d940cb150 0x7f0d940c29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:37.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:37.978+0000 7f0d92ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d940cb150 0x7f0d940c29c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57704/0 (socket says 172.21.15.16:57704) 2023-05-20T01:16:37.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:37.978+0000 7f0d92ffd700 1 -- 172.21.15.16:0/2281140078 learned_addr learned my addr 172.21.15.16:0/2281140078 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:37.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:37.978+0000 7f0d937fe700 1 --2- 172.21.15.16:0/2281140078 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0d940b1020 0x7f0d9413ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:37.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:37.978+0000 7f0d93fff700 1 --2- 172.21.15.16:0/2281140078 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0d9415a910 0x7f0d940cddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:37.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:37.979+0000 7f0d92ffd700 1 -- 172.21.15.16:0/2281140078 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0d9415a910 msgr2=0x7f0d940cddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:37.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:37.979+0000 7f0d92ffd700 1 --2- 172.21.15.16:0/2281140078 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0d9415a910 0x7f0d940cddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:37.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:37.979+0000 7f0d92ffd700 1 -- 172.21.15.16:0/2281140078 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0d940b1020 msgr2=0x7f0d9413ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:37.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:37.979+0000 7f0d92ffd700 1 --2- 172.21.15.16:0/2281140078 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0d940b1020 0x7f0d9413ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:37.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:37.979+0000 7f0d92ffd700 1 -- 172.21.15.16:0/2281140078 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d880094a0 con 0x7f0d940cb150 2023-05-20T01:16:37.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:37.979+0000 7f0d92ffd700 1 --2- 172.21.15.16:0/2281140078 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d940cb150 0x7f0d940c29c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0d8400c500 tx=0x7f0d8400c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:37.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:37.979+0000 7f0d7bfff700 1 -- 172.21.15.16:0/2281140078 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d84005d70 con 0x7f0d940cb150 2023-05-20T01:16:37.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:37.979+0000 7f0d7bfff700 1 -- 172.21.15.16:0/2281140078 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d840070d0 con 0x7f0d940cb150 2023-05-20T01:16:37.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:37.979+0000 7f0d9a142700 1 -- 172.21.15.16:0/2281140078 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d9406c030 con 0x7f0d940cb150 2023-05-20T01:16:37.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:37.979+0000 7f0d7bfff700 1 -- 172.21.15.16:0/2281140078 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d84007460 con 0x7f0d940cb150 2023-05-20T01:16:37.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:37.979+0000 7f0d9a142700 1 -- 172.21.15.16:0/2281140078 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d9406bd50 con 0x7f0d940cb150 2023-05-20T01:16:37.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:37.980+0000 7f0d9a142700 1 -- 172.21.15.16:0/2281140078 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d9406c310 con 0x7f0d940cb150 2023-05-20T01:16:37.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:37.981+0000 7f0d7bfff700 1 -- 172.21.15.16:0/2281140078 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0d840075c0 con 0x7f0d940cb150 2023-05-20T01:16:37.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:37.981+0000 7f0d7bfff700 1 --2- 172.21.15.16:0/2281140078 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f0d8006ace0 0x7f0d8006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:37.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:37.981+0000 7f0d937fe700 1 --2- 172.21.15.16:0/2281140078 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f0d8006ace0 0x7f0d8006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:37.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:37.981+0000 7f0d7bfff700 1 -- 172.21.15.16:0/2281140078 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f0d84088b20 con 0x7f0d940cb150 2023-05-20T01:16:37.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:37.981+0000 7f0d937fe700 1 --2- 172.21.15.16:0/2281140078 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f0d8006ace0 0x7f0d8006d190 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0d880090b0 tx=0x7f0d8801c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:37.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:37.986+0000 7f0d7bfff700 1 -- 172.21.15.16:0/2281140078 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f0d840538e0 con 0x7f0d940cb150 2023-05-20T01:16:38.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.183+0000 7f0d9a142700 1 -- 172.21.15.16:0/2281140078 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0d940c5da0 con 0x7f0d940cb150 2023-05-20T01:16:38.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.185+0000 7f0d7bfff700 1 -- 172.21.15.16:0/2281140078 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+20026 (secure 0 0 0) 0x7f0d84017090 con 0x7f0d940cb150 2023-05-20T01:16:38.189 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:16:38.189 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"fsid":"c2245cc5-ecab-4087-a59b-bc81db901676","created":"2023-05-20T01:16:09.549230+0000","modified":"2023-05-20T01:16:32.768495+0000","last_up_change":"2023-05-20T01:16:19.788624+0000","last_in_change":"2023-05-20T01:16:12.092160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-05-20T01:16:20.934366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-05-20T01:16:21.372294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-05-20T01:16:26.208287+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-05-20T01:16:27.292268+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-05-20T01:16:30.357285+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"317a83cd-19c8-4958-a301-6087ef342289","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6825","nonce":3923634179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6827","nonce":3923634179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6831","nonce":3923634179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6829","nonce":3923634179}]},"public_addr":"172.21.15.16:6825/3923634179","cluster_addr":"172.21.15.16:6827/3923634179","heartbeat_back_addr":"172.21.15.16:6831/3923634179","heartbeat_front_addr":"172.21.15.16:6829/3923634179","state":["exists","up"]},{"osd":1,"uuid":"5581046f-8219-4089-8d7d-b7846399505e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6809","nonce":3860264220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6811","nonce":3860264220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6815","nonce":3860264220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6813","nonce":3860264220}]},"public_addr":"172.21.15.16:6809/3860264220","cluster_addr":"172.21.15.16:6811/3860264220","heartbeat_back_addr":"172.21.15.16:6815/3860264220","heartbeat_front_addr":"172.21.15.16:6813/3860264220","state":["exists","up"]},{"osd":2,"uuid":"072e9e66-4133-478e-8d4a-64d46c2963f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6817","nonce":293187030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6819","nonce":293187030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6823","nonce":293187030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6821","nonce":293187030}]},"public_addr":"172.21.15.16:6817/293187030","cluster_addr":"172.21.15.16:6819/293187030","heartbeat_back_addr":"172.21.15.16:6823/293187030","heartbeat_front_addr":"172.21.15.16:6821/293187030","state":["exists","up"]},{"osd":3,"uuid":"11e1bba6-00f8-48dd-9028-d8d716b7f145","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6801","nonce":3669781890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6803","nonce":3669781890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6807","nonce":3669781890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6805","nonce":3669781890}]},"public_addr":"172.21.15.16:6801/3669781890","cluster_addr":"172.21.15.16:6803/3669781890","heartbeat_back_addr":"172.21.15.16:6807/3669781890","heartbeat_front_addr":"172.21.15.16:6805/3669781890","state":["exists","up"]},{"osd":4,"uuid":"b8909c3a-cf70-46cb-8d0b-009c707cf9d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6819","nonce":506317223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6823","nonce":506317223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6829","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6831","nonce":506317223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6827","nonce":506317223}]},"public_addr":"172.21.15.132:6819/506317223","cluster_addr":"172.21.15.132:6823/506317223","heartbeat_back_addr":"172.21.15.132:6831/506317223","heartbeat_front_addr":"172.21.15.132:6827/506317223","state":["exists","up"]},{"osd":5,"uuid":"b1818304-4583-47bf-a6ec-0bfbb5166233","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6818","nonce":1844689995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6822","nonce":1844689995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6830","nonce":1844689995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6826","nonce":1844689995}]},"public_addr":"172.21.15.132:6818/1844689995","cluster_addr":"172.21.15.132:6822/1844689995","heartbeat_back_addr":"172.21.15.132:6830/1844689995","heartbeat_front_addr":"172.21.15.132:6826/1844689995","state":["exists","up"]},{"osd":6,"uuid":"d37f4691-eed1-4a93-8818-18354f03f4a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6801","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6802","nonce":727581123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6805","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6807","nonce":727581123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6813","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6815","nonce":727581123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6811","nonce":727581123}]},"public_addr":"172.21.15.132:6802/727581123","cluster_addr":"172.21.15.132:6807/727581123","heartbeat_back_addr":"172.21.15.132:6815/727581123","heartbeat_front_addr":"172.21.15.132:6811/727581123","state":["exists","up"]},{"osd":7,"uuid":"4f712f62-5d1d-4816-8042-03209144f8b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6803","nonce":466001149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6806","nonce":466001149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6814","nonce":466001149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6810","nonce":466001149}]},"public_addr":"172.21.15.132:6803/466001149","cluster_addr":"172.21.15.132:6806/466001149","heartbeat_back_addr":"172.21.15.132:6814/466001149","heartbeat_front_addr":"172.21.15.132:6810/466001149","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.338863+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.278680+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305694+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.246835+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.276237+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.304441+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305421+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.267050+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-20T01:16:38.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.187+0000 7f0d9a142700 1 -- 172.21.15.16:0/2281140078 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f0d8006ace0 msgr2=0x7f0d8006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:38.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.187+0000 7f0d9a142700 1 --2- 172.21.15.16:0/2281140078 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f0d8006ace0 0x7f0d8006d190 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0d880090b0 tx=0x7f0d8801c040 comp rx=0 tx=0).stop 2023-05-20T01:16:38.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.187+0000 7f0d9a142700 1 -- 172.21.15.16:0/2281140078 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d940cb150 msgr2=0x7f0d940c29c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:38.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.187+0000 7f0d9a142700 1 --2- 172.21.15.16:0/2281140078 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d940cb150 0x7f0d940c29c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0d8400c500 tx=0x7f0d8400c810 comp rx=0 tx=0).stop 2023-05-20T01:16:38.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.187+0000 7f0d9a142700 1 -- 172.21.15.16:0/2281140078 shutdown_connections 2023-05-20T01:16:38.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.187+0000 7f0d9a142700 1 --2- 172.21.15.16:0/2281140078 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f0d8006ace0 0x7f0d8006d190 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:38.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.187+0000 7f0d9a142700 1 --2- 172.21.15.16:0/2281140078 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0d9415a910 0x7f0d940cddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:38.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.187+0000 7f0d9a142700 1 --2- 172.21.15.16:0/2281140078 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0d940b1020 0x7f0d9413ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:38.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.187+0000 7f0d9a142700 1 --2- 172.21.15.16:0/2281140078 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d940cb150 0x7f0d940c29c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:38.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.187+0000 7f0d9a142700 1 -- 172.21.15.16:0/2281140078 >> 172.21.15.16:0/2281140078 conn(0x7f0d940cb910 msgr2=0x7f0d9415af30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:38.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.187+0000 7f0d9a142700 1 -- 172.21.15.16:0/2281140078 shutdown_connections 2023-05-20T01:16:38.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.187+0000 7f0d9a142700 1 -- 172.21.15.16:0/2281140078 wait complete. 2023-05-20T01:16:38.203 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-05-20T01:16:20.934366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-05-20T01:16:21.372294+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-05-20T01:16:26.208287+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-05-20T01:16:27.292268+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-05-20T01:16:30.357285+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-05-20T01:16:38.204 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-05-20T01:16:38.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.327+0000 7f370e7a6700 1 Processor -- start 2023-05-20T01:16:38.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.327+0000 7f370e7a6700 1 -- start start 2023-05-20T01:16:38.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.328+0000 7f370e7a6700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f37080cd070 0x7f370806e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:38.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.328+0000 7f370e7a6700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f37080710d0 0x7f37080cabb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:38.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.328+0000 7f370e7a6700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3708072430 con 0x7f370815cf40 2023-05-20T01:16:38.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.328+0000 7f370e7a6700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3708071820 con 0x7f37080710d0 2023-05-20T01:16:38.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.328+0000 7f370e7a6700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3708071b00 con 0x7f37080cd070 2023-05-20T01:16:38.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.328+0000 7f370cd43700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f37080710d0 0x7f37080cabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:38.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.328+0000 7f370cd43700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f37080710d0 0x7f37080cabb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57714/0 (socket says 172.21.15.16:57714) 2023-05-20T01:16:38.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.328+0000 7f370cd43700 1 -- 172.21.15.16:0/2271828627 learned_addr learned my addr 172.21.15.16:0/2271828627 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:38.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.329+0000 7f37077fe700 1 --2- 172.21.15.16:0/2271828627 >> v2:172.21.15.132:3300/0 conn(0x7f37080cd070 0x7f370806e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:38.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.329+0000 7f370cd43700 1 -- 172.21.15.16:0/2271828627 >> v2:172.21.15.132:3300/0 conn(0x7f37080cd070 msgr2=0x7f370806e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:38.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.329+0000 7f370cd43700 1 --2- 172.21.15.16:0/2271828627 >> v2:172.21.15.132:3300/0 conn(0x7f37080cd070 0x7f370806e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:38.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.329+0000 7f370cd43700 1 -- 172.21.15.16:0/2271828627 >> v1:172.21.15.132:6789/0 conn(0x7f370815cf40 legacy=0x7f37080cc960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:38.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.329+0000 7f370cd43700 1 -- 172.21.15.16:0/2271828627 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3708071de0 con 0x7f37080710d0 2023-05-20T01:16:38.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.329+0000 7f370cd43700 1 --2- 172.21.15.16:0/2271828627 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37080710d0 0x7f37080cabb0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f37080d0c30 tx=0x7f370000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aee0d8fb9c10950 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:38.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.329+0000 7f3706ffd700 1 -- 172.21.15.16:0/2271828627 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f370000e070 con 0x7f37080710d0 2023-05-20T01:16:38.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.329+0000 7f3706ffd700 1 -- 172.21.15.16:0/2271828627 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f370000b9c0 con 0x7f37080710d0 2023-05-20T01:16:38.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.329+0000 7f3706ffd700 1 -- 172.21.15.16:0/2271828627 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f370000bd70 con 0x7f37080710d0 2023-05-20T01:16:38.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.330+0000 7f370e7a6700 1 -- 172.21.15.16:0/2271828627 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37080710d0 msgr2=0x7f37080cabb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:38.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.330+0000 7f370e7a6700 1 --2- 172.21.15.16:0/2271828627 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37080710d0 0x7f37080cabb0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f37080d0c30 tx=0x7f370000b330 comp rx=0 tx=0).stop 2023-05-20T01:16:38.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.330+0000 7f370e7a6700 1 -- 172.21.15.16:0/2271828627 shutdown_connections 2023-05-20T01:16:38.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.330+0000 7f370e7a6700 1 --2- 172.21.15.16:0/2271828627 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37080710d0 0x7f37080cabb0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:38.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.330+0000 7f370e7a6700 1 --2- 172.21.15.16:0/2271828627 >> v2:172.21.15.132:3300/0 conn(0x7f37080cd070 0x7f370806e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:38.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.330+0000 7f370e7a6700 1 -- 172.21.15.16:0/2271828627 >> 172.21.15.16:0/2271828627 conn(0x7f370806c260 msgr2=0x7f37080cffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:38.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.330+0000 7f370e7a6700 1 -- 172.21.15.16:0/2271828627 shutdown_connections 2023-05-20T01:16:38.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.330+0000 7f370e7a6700 1 -- 172.21.15.16:0/2271828627 wait complete. 2023-05-20T01:16:38.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.331+0000 7f370e7a6700 1 Processor -- start 2023-05-20T01:16:38.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.331+0000 7f370e7a6700 1 -- start start 2023-05-20T01:16:38.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.332+0000 7f370e7a6700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f37080710d0 0x7f370815bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:38.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.332+0000 7f370e7a6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37080cd070 0x7f370814b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:38.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.332+0000 7f370e7a6700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f370815cf40 0x7f370814b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:38.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.332+0000 7f370e7a6700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f370814e320 con 0x7f37080cd070 2023-05-20T01:16:38.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.332+0000 7f370e7a6700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f370814c890 con 0x7f37080710d0 2023-05-20T01:16:38.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.332+0000 7f370e7a6700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f370814cb70 con 0x7f370815cf40 2023-05-20T01:16:38.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.332+0000 7f37077fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37080cd070 0x7f370814b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:38.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.332+0000 7f37077fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37080cd070 0x7f370814b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57718/0 (socket says 172.21.15.16:57718) 2023-05-20T01:16:38.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.332+0000 7f37077fe700 1 -- 172.21.15.16:0/879220608 learned_addr learned my addr 172.21.15.16:0/879220608 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:38.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.332+0000 7f370cd43700 1 --2- 172.21.15.16:0/879220608 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f370815cf40 0x7f370814b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:38.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.332+0000 7f3707fff700 1 --2- 172.21.15.16:0/879220608 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f37080710d0 0x7f370815bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:38.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.332+0000 7f37077fe700 1 -- 172.21.15.16:0/879220608 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f370815cf40 msgr2=0x7f370814b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:38.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.332+0000 7f37077fe700 1 --2- 172.21.15.16:0/879220608 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f370815cf40 0x7f370814b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:38.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.332+0000 7f37077fe700 1 -- 172.21.15.16:0/879220608 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f37080710d0 msgr2=0x7f370815bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:38.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.332+0000 7f37077fe700 1 --2- 172.21.15.16:0/879220608 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f37080710d0 0x7f370815bec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:38.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.332+0000 7f37077fe700 1 -- 172.21.15.16:0/879220608 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3700009b30 con 0x7f37080cd070 2023-05-20T01:16:38.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.332+0000 7f37077fe700 1 --2- 172.21.15.16:0/879220608 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37080cd070 0x7f370814b330 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f36f800e810 tx=0x7f36f800ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:38.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.332+0000 7f3704ff9700 1 -- 172.21.15.16:0/879220608 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36f8004230 con 0x7f37080cd070 2023-05-20T01:16:38.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.333+0000 7f3704ff9700 1 -- 172.21.15.16:0/879220608 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36f8004390 con 0x7f37080cd070 2023-05-20T01:16:38.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.333+0000 7f370e7a6700 1 -- 172.21.15.16:0/879220608 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3708071510 con 0x7f37080cd070 2023-05-20T01:16:38.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.333+0000 7f3704ff9700 1 -- 172.21.15.16:0/879220608 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36f80036a0 con 0x7f37080cd070 2023-05-20T01:16:38.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.333+0000 7f370e7a6700 1 -- 172.21.15.16:0/879220608 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37080720c0 con 0x7f37080cd070 2023-05-20T01:16:38.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.333+0000 7f370e7a6700 1 -- 172.21.15.16:0/879220608 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37080cb0f0 con 0x7f37080cd070 2023-05-20T01:16:38.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.334+0000 7f3704ff9700 1 -- 172.21.15.16:0/879220608 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f36f8003800 con 0x7f37080cd070 2023-05-20T01:16:38.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.334+0000 7f3704ff9700 1 --2- 172.21.15.16:0/879220608 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f36e806ace0 0x7f36e806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:38.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.335+0000 7f3704ff9700 1 -- 172.21.15.16:0/879220608 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f36f80896a0 con 0x7f37080cd070 2023-05-20T01:16:38.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.335+0000 7f3707fff700 1 --2- 172.21.15.16:0/879220608 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f36e806ace0 0x7f36e806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:38.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.335+0000 7f3707fff700 1 --2- 172.21.15.16:0/879220608 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f36e806ace0 0x7f36e806d190 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f36f0007fd0 tx=0x7f36f0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:38.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.339+0000 7f3704ff9700 1 -- 172.21.15.16:0/879220608 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f36f8057e60 con 0x7f37080cd070 2023-05-20T01:16:38.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.533+0000 7f370e7a6700 1 -- 172.21.15.16:0/879220608 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f37080747b0 con 0x7f37080cd070 2023-05-20T01:16:38.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.534+0000 7f3704ff9700 1 -- 172.21.15.16:0/879220608 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7f36f80579f0 con 0x7f37080cd070 2023-05-20T01:16:38.538 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 8 2023-05-20T01:16:38.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.536+0000 7f370e7a6700 1 -- 172.21.15.16:0/879220608 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f36e806ace0 msgr2=0x7f36e806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:38.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.536+0000 7f370e7a6700 1 --2- 172.21.15.16:0/879220608 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f36e806ace0 0x7f36e806d190 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f36f0007fd0 tx=0x7f36f0009040 comp rx=0 tx=0).stop 2023-05-20T01:16:38.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.536+0000 7f370e7a6700 1 -- 172.21.15.16:0/879220608 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37080cd070 msgr2=0x7f370814b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:38.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.536+0000 7f370e7a6700 1 --2- 172.21.15.16:0/879220608 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37080cd070 0x7f370814b330 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f36f800e810 tx=0x7f36f800ebd0 comp rx=0 tx=0).stop 2023-05-20T01:16:38.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.536+0000 7f370e7a6700 1 -- 172.21.15.16:0/879220608 shutdown_connections 2023-05-20T01:16:38.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.536+0000 7f370e7a6700 1 --2- 172.21.15.16:0/879220608 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f36e806ace0 0x7f36e806d190 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:38.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.536+0000 7f370e7a6700 1 --2- 172.21.15.16:0/879220608 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f370815cf40 0x7f370814b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:38.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.536+0000 7f370e7a6700 1 --2- 172.21.15.16:0/879220608 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f37080710d0 0x7f370815bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:38.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.536+0000 7f370e7a6700 1 --2- 172.21.15.16:0/879220608 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37080cd070 0x7f370814b330 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:38.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.536+0000 7f370e7a6700 1 -- 172.21.15.16:0/879220608 >> 172.21.15.16:0/879220608 conn(0x7f370806c260 msgr2=0x7f3708150b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:38.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.536+0000 7f370e7a6700 1 -- 172.21.15.16:0/879220608 shutdown_connections 2023-05-20T01:16:38.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.536+0000 7f370e7a6700 1 -- 172.21.15.16:0/879220608 wait complete. 2023-05-20T01:16:38.551 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-05-20T01:16:38.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.675+0000 7f3dd1afc700 1 Processor -- start 2023-05-20T01:16:38.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.675+0000 7f3dd1afc700 1 -- start start 2023-05-20T01:16:38.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.676+0000 7f3dd1afc700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3dcc0ce7b0 0x7f3dcc0cb7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:38.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.676+0000 7f3dd1afc700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3dcc159360 con 0x7f3dcc158f90 2023-05-20T01:16:38.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.676+0000 7f3dd1afc700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3dcc0ccba0 con 0x7f3dcc0cc050 2023-05-20T01:16:38.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.676+0000 7f3dd1afc700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dcc0cce10 con 0x7f3dcc0ce7b0 2023-05-20T01:16:38.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.676+0000 7f3dcb7fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f3dcc158f90 0x7f3dcc0ce0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:41020/0 (socket says 172.21.15.16:41020) 2023-05-20T01:16:38.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.676+0000 7f3dcb7fe700 1 -- 172.21.15.16:0/4064367924 learned_addr learned my addr 172.21.15.16:0/4064367924 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:16:38.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.676+0000 7f3dcaffd700 1 --2- 172.21.15.16:0/4064367924 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3dcc0ce7b0 0x7f3dcc0cb7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:38.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.676+0000 7f3dca7fc700 1 -- 172.21.15.16:0/4064367924 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3581157375 0 0) 0x7f3dc0001110 con 0x7f3dcc158f90 2023-05-20T01:16:38.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.676+0000 7f3dca7fc700 1 -- 172.21.15.16:0/4064367924 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3db40035b0 con 0x7f3dcc158f90 2023-05-20T01:16:38.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.677+0000 7f3dca7fc700 1 -- 172.21.15.16:0/4064367924 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3747690595 0 0) 0x7f3dcc0ccba0 con 0x7f3dcc0cc050 2023-05-20T01:16:38.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.677+0000 7f3dca7fc700 1 -- 172.21.15.16:0/4064367924 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3dc0001110 con 0x7f3dcc0cc050 2023-05-20T01:16:38.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.677+0000 7f3dca7fc700 1 -- 172.21.15.16:0/4064367924 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1409814767 0 0) 0x7f3db40035b0 con 0x7f3dcc158f90 2023-05-20T01:16:38.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.677+0000 7f3dca7fc700 1 -- 172.21.15.16:0/4064367924 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3dcc0ce7b0 msgr2=0x7f3dcc0cb7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:38.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.677+0000 7f3dca7fc700 1 --2- 172.21.15.16:0/4064367924 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3dcc0ce7b0 0x7f3dcc0cb7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:38.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.677+0000 7f3dca7fc700 1 -- 172.21.15.16:0/4064367924 >> v1:172.21.15.132:6789/0 conn(0x7f3dcc0cc050 legacy=0x7f3dcc0cc490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:38.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.677+0000 7f3dca7fc700 1 -- 172.21.15.16:0/4064367924 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dcc0ca750 con 0x7f3dcc158f90 2023-05-20T01:16:38.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.677+0000 7f3dca7fc700 1 -- 172.21.15.16:0/4064367924 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7f3dc0001800 con 0x7f3dcc158f90 2023-05-20T01:16:38.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.677+0000 7f3dcaffd700 1 --2- 172.21.15.16:0/4064367924 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3dcc0ce7b0 0x7f3dcc0cb7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-05-20T01:16:38.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.677+0000 7f3dca7fc700 1 -- 172.21.15.16:0/4064367924 >> v1:172.21.15.16:6789/0 conn(0x7f3dcc158f90 legacy=0x7f3dcc0ce0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:38.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.677+0000 7f3dca7fc700 1 --2- 172.21.15.16:0/4064367924 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3db4003bc0 0x7f3db4003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:38.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.677+0000 7f3dca7fc700 1 --2- 172.21.15.16:0/4064367924 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3db4004850 0x7f3db4004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:38.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.677+0000 7f3dca7fc700 1 --2- 172.21.15.16:0/4064367924 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3dcc0ce7b0 0x7f3db4005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:38.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.677+0000 7f3dca7fc700 1 -- 172.21.15.16:0/4064367924 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3db400dc50 con 0x7f3db4004850 2023-05-20T01:16:38.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.677+0000 7f3dca7fc700 1 -- 172.21.15.16:0/4064367924 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3db400ddc0 con 0x7f3db4003bc0 2023-05-20T01:16:38.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.677+0000 7f3dca7fc700 1 -- 172.21.15.16:0/4064367924 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3db400df80 con 0x7f3dcc0ce7b0 2023-05-20T01:16:38.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.677+0000 7f3dcaffd700 1 --2- 172.21.15.16:0/4064367924 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3db4004850 0x7f3db4004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:38.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.677+0000 7f3dcb7fe700 1 --2- 172.21.15.16:0/4064367924 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3db4003bc0 0x7f3db4003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:38.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.677+0000 7f3dcbfff700 1 --2- 172.21.15.16:0/4064367924 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3dcc0ce7b0 0x7f3db4005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:38.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.677+0000 7f3dcaffd700 1 -- 172.21.15.16:0/4064367924 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3dcc0ce7b0 msgr2=0x7f3db4005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:38.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.677+0000 7f3dcaffd700 1 --2- 172.21.15.16:0/4064367924 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3dcc0ce7b0 0x7f3db4005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:38.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.677+0000 7f3dcaffd700 1 -- 172.21.15.16:0/4064367924 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3db4003bc0 msgr2=0x7f3db4003fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:38.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.677+0000 7f3dcaffd700 1 --2- 172.21.15.16:0/4064367924 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3db4003bc0 0x7f3db4003fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:38.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.677+0000 7f3dcaffd700 1 -- 172.21.15.16:0/4064367924 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dbc00a640 con 0x7f3db4004850 2023-05-20T01:16:38.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.677+0000 7f3dcaffd700 1 --2- 172.21.15.16:0/4064367924 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3db4004850 0x7f3db4004cc0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc00ac60 tx=0x7f3db4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1001e32517c7e9c4 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:38.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.677+0000 7f3dca7fc700 1 -- 172.21.15.16:0/4064367924 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dbc00e960 con 0x7f3db4004850 2023-05-20T01:16:38.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.677+0000 7f3dca7fc700 1 -- 172.21.15.16:0/4064367924 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dbc00eac0 con 0x7f3db4004850 2023-05-20T01:16:38.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.678+0000 7f3dca7fc700 1 -- 172.21.15.16:0/4064367924 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dbc00ee30 con 0x7f3db4004850 2023-05-20T01:16:38.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.678+0000 7f3dd1afc700 1 -- 172.21.15.16:0/4064367924 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3db4004850 msgr2=0x7f3db4004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:38.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.678+0000 7f3dd1afc700 1 --2- 172.21.15.16:0/4064367924 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3db4004850 0x7f3db4004cc0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc00ac60 tx=0x7f3db4005880 comp rx=0 tx=0).stop 2023-05-20T01:16:38.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.678+0000 7f3dcbfff700 1 -- 172.21.15.16:0/4064367924 reap_dead start 2023-05-20T01:16:38.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.678+0000 7f3dd1afc700 1 -- 172.21.15.16:0/4064367924 shutdown_connections 2023-05-20T01:16:38.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.678+0000 7f3dd1afc700 1 -- 172.21.15.16:0/4064367924 >> 172.21.15.16:0/4064367924 conn(0x7f3dcc06cbb0 msgr2=0x7f3dcc06cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:38.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.678+0000 7f3dd1afc700 1 -- 172.21.15.16:0/4064367924 shutdown_connections 2023-05-20T01:16:38.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.678+0000 7f3dd1afc700 1 -- 172.21.15.16:0/4064367924 wait complete. 2023-05-20T01:16:38.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.679+0000 7f3dd1afc700 1 Processor -- start 2023-05-20T01:16:38.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.680+0000 7f3dd1afc700 1 -- start start 2023-05-20T01:16:38.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.680+0000 7f3dd1afc700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3db4004850 0x7f3dcc1f68e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:38.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.680+0000 7f3dd1afc700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3dcc1575f0 0x7f3dcc155110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:38.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.680+0000 7f3dd1afc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3dcc1451e0 0x7f3dcc145680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:38.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.680+0000 7f3dd1afc700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dcc0c9630 con 0x7f3dcc1451e0 2023-05-20T01:16:38.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.680+0000 7f3dd1afc700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dcc145bc0 con 0x7f3dcc1575f0 2023-05-20T01:16:38.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.680+0000 7f3dd1afc700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dcc0c6880 con 0x7f3db4004850 2023-05-20T01:16:38.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.680+0000 7f3dcbfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3dcc1451e0 0x7f3dcc145680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:38.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.680+0000 7f3dcbfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3dcc1451e0 0x7f3dcc145680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57740/0 (socket says 172.21.15.16:57740) 2023-05-20T01:16:38.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.680+0000 7f3dcbfff700 1 -- 172.21.15.16:0/3052690485 learned_addr learned my addr 172.21.15.16:0/3052690485 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:38.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.680+0000 7f3dcb7fe700 1 --2- 172.21.15.16:0/3052690485 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3db4004850 0x7f3dcc1f68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:38.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.680+0000 7f3dcaffd700 1 --2- 172.21.15.16:0/3052690485 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3dcc1575f0 0x7f3dcc155110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:38.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.681+0000 7f3dcbfff700 1 -- 172.21.15.16:0/3052690485 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3db4004850 msgr2=0x7f3dcc1f68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:38.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.681+0000 7f3dcbfff700 1 --2- 172.21.15.16:0/3052690485 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3db4004850 0x7f3dcc1f68e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:38.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.681+0000 7f3dcbfff700 1 -- 172.21.15.16:0/3052690485 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3dcc1575f0 msgr2=0x7f3dcc155110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:38.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.681+0000 7f3dcbfff700 1 --2- 172.21.15.16:0/3052690485 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3dcc1575f0 0x7f3dcc155110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:38.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.681+0000 7f3dcbfff700 1 -- 172.21.15.16:0/3052690485 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dcc0cacd0 con 0x7f3dcc1451e0 2023-05-20T01:16:38.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.681+0000 7f3dcbfff700 1 --2- 172.21.15.16:0/3052690485 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3dcc1451e0 0x7f3dcc145680 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc0cbcf0 tx=0x7f3dc400d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:38.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.681+0000 7f3db3fff700 1 -- 172.21.15.16:0/3052690485 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dc400fe70 con 0x7f3dcc1451e0 2023-05-20T01:16:38.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.681+0000 7f3db3fff700 1 -- 172.21.15.16:0/3052690485 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dc4003c10 con 0x7f3dcc1451e0 2023-05-20T01:16:38.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.681+0000 7f3dd1afc700 1 -- 172.21.15.16:0/3052690485 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dcc0ca9f0 con 0x7f3dcc1451e0 2023-05-20T01:16:38.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.681+0000 7f3db3fff700 1 -- 172.21.15.16:0/3052690485 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dc400d960 con 0x7f3dcc1451e0 2023-05-20T01:16:38.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.681+0000 7f3dd1afc700 1 -- 172.21.15.16:0/3052690485 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dcc0cafb0 con 0x7f3dcc1451e0 2023-05-20T01:16:38.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.682+0000 7f3dd1afc700 1 -- 172.21.15.16:0/3052690485 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3dcc0747b0 con 0x7f3dcc1451e0 2023-05-20T01:16:38.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.682+0000 7f3db3fff700 1 -- 172.21.15.16:0/3052690485 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3dc400dac0 con 0x7f3dcc1451e0 2023-05-20T01:16:38.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.683+0000 7f3db3fff700 1 --2- 172.21.15.16:0/3052690485 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f3db806ace0 0x7f3db806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:38.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.683+0000 7f3dcb7fe700 1 --2- 172.21.15.16:0/3052690485 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f3db806ace0 0x7f3db806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:38.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.683+0000 7f3db3fff700 1 -- 172.21.15.16:0/3052690485 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f3dc40898d0 con 0x7f3dcc1451e0 2023-05-20T01:16:38.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.683+0000 7f3dcb7fe700 1 --2- 172.21.15.16:0/3052690485 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f3db806ace0 0x7f3db806d190 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3dc0001050 tx=0x7f3dc0007a80 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:38.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.687+0000 7f3db3fff700 1 -- 172.21.15.16:0/3052690485 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f3dc4054720 con 0x7f3dcc1451e0 2023-05-20T01:16:38.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.885+0000 7f3dd1afc700 1 -- 172.21.15.16:0/3052690485 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3dcc06ec80 con 0x7f3dcc1451e0 2023-05-20T01:16:38.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.885+0000 7f3db3fff700 1 -- 172.21.15.16:0/3052690485 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3dc4057d40 con 0x7f3dcc1451e0 2023-05-20T01:16:38.889 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-05-20T01:16:38.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.887+0000 7f3dd1afc700 1 -- 172.21.15.16:0/3052690485 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f3db806ace0 msgr2=0x7f3db806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:38.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.887+0000 7f3dd1afc700 1 --2- 172.21.15.16:0/3052690485 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f3db806ace0 0x7f3db806d190 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3dc0001050 tx=0x7f3dc0007a80 comp rx=0 tx=0).stop 2023-05-20T01:16:38.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.887+0000 7f3dd1afc700 1 -- 172.21.15.16:0/3052690485 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3dcc1451e0 msgr2=0x7f3dcc145680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:38.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.887+0000 7f3dd1afc700 1 --2- 172.21.15.16:0/3052690485 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3dcc1451e0 0x7f3dcc145680 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc0cbcf0 tx=0x7f3dc400d430 comp rx=0 tx=0).stop 2023-05-20T01:16:38.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.887+0000 7f3dd1afc700 1 -- 172.21.15.16:0/3052690485 shutdown_connections 2023-05-20T01:16:38.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.887+0000 7f3dd1afc700 1 --2- 172.21.15.16:0/3052690485 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3dcc1451e0 0x7f3dcc145680 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:38.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.887+0000 7f3dd1afc700 1 --2- 172.21.15.16:0/3052690485 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f3db806ace0 0x7f3db806d190 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:38.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.887+0000 7f3dd1afc700 1 --2- 172.21.15.16:0/3052690485 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3db4004850 0x7f3dcc1f68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:38.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.887+0000 7f3dd1afc700 1 --2- 172.21.15.16:0/3052690485 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3dcc1575f0 0x7f3dcc155110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:38.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.887+0000 7f3dd1afc700 1 -- 172.21.15.16:0/3052690485 >> 172.21.15.16:0/3052690485 conn(0x7f3dcc06cbb0 msgr2=0x7f3dcc1e3000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:38.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.887+0000 7f3dd1afc700 1 -- 172.21.15.16:0/3052690485 shutdown_connections 2023-05-20T01:16:38.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:38.887+0000 7f3dd1afc700 1 -- 172.21.15.16:0/3052690485 wait complete. 2023-05-20T01:16:38.903 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-05-20T01:16:39.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.027+0000 7fc54c67a700 1 Processor -- start 2023-05-20T01:16:39.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.027+0000 7fc54c67a700 1 -- start start 2023-05-20T01:16:39.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.028+0000 7fc54c67a700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc54415cec0 0x7fc544071bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:39.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.028+0000 7fc54c67a700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fc54415d290 0x7fc544072130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:39.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.028+0000 7fc54c67a700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5440cd630 con 0x7fc5440cf2c0 2023-05-20T01:16:39.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.028+0000 7fc54c67a700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5440cca60 con 0x7fc54415cec0 2023-05-20T01:16:39.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.028+0000 7fc54c67a700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5440cfdd0 con 0x7fc54415d290 2023-05-20T01:16:39.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.028+0000 7fc54a416700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc54415cec0 0x7fc544071bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:39.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.028+0000 7fc54a416700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc54415cec0 0x7fc544071bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57744/0 (socket says 172.21.15.16:57744) 2023-05-20T01:16:39.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.028+0000 7fc54a416700 1 -- 172.21.15.16:0/1880900300 learned_addr learned my addr 172.21.15.16:0/1880900300 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:39.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.028+0000 7fc549c15700 1 --2- 172.21.15.16:0/1880900300 >> v2:172.21.15.132:3300/0 conn(0x7fc54415d290 0x7fc544072130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:39.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.029+0000 7fc54a416700 1 -- 172.21.15.16:0/1880900300 >> v2:172.21.15.132:3300/0 conn(0x7fc54415d290 msgr2=0x7fc544072130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:39.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.029+0000 7fc54a416700 1 --2- 172.21.15.16:0/1880900300 >> v2:172.21.15.132:3300/0 conn(0x7fc54415d290 0x7fc544072130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:39.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.029+0000 7fc54a416700 1 -- 172.21.15.16:0/1880900300 >> v1:172.21.15.132:6789/0 conn(0x7fc5440cf2c0 legacy=0x7fc5440cf6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:39.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.029+0000 7fc54a416700 1 -- 172.21.15.16:0/1880900300 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5440ccd00 con 0x7fc54415cec0 2023-05-20T01:16:39.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.029+0000 7fc54a416700 1 --2- 172.21.15.16:0/1880900300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc54415cec0 0x7fc544071bf0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc5440cc970 tx=0x7fc534009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2401e8b762d4ceb server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:39.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.029+0000 7fc549414700 1 -- 172.21.15.16:0/1880900300 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc534005e30 con 0x7fc54415cec0 2023-05-20T01:16:39.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.029+0000 7fc549414700 1 -- 172.21.15.16:0/1880900300 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc534004030 con 0x7fc54415cec0 2023-05-20T01:16:39.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.029+0000 7fc549414700 1 -- 172.21.15.16:0/1880900300 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5340043e0 con 0x7fc54415cec0 2023-05-20T01:16:39.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.029+0000 7fc54c67a700 1 -- 172.21.15.16:0/1880900300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc54415cec0 msgr2=0x7fc544071bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:39.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.029+0000 7fc54c67a700 1 --2- 172.21.15.16:0/1880900300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc54415cec0 0x7fc544071bf0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc5440cc970 tx=0x7fc534009bd0 comp rx=0 tx=0).stop 2023-05-20T01:16:39.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.030+0000 7fc54c67a700 1 -- 172.21.15.16:0/1880900300 shutdown_connections 2023-05-20T01:16:39.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.030+0000 7fc54c67a700 1 --2- 172.21.15.16:0/1880900300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc54415cec0 0x7fc544071bf0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:39.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.030+0000 7fc54c67a700 1 --2- 172.21.15.16:0/1880900300 >> v2:172.21.15.132:3300/0 conn(0x7fc54415d290 0x7fc544072130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:39.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.030+0000 7fc54c67a700 1 -- 172.21.15.16:0/1880900300 >> 172.21.15.16:0/1880900300 conn(0x7fc5440cffd0 msgr2=0x7fc5440d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:39.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.030+0000 7fc54c67a700 1 -- 172.21.15.16:0/1880900300 shutdown_connections 2023-05-20T01:16:39.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.030+0000 7fc54c67a700 1 -- 172.21.15.16:0/1880900300 wait complete. 2023-05-20T01:16:39.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.031+0000 7fc54c67a700 1 Processor -- start 2023-05-20T01:16:39.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.031+0000 7fc54c67a700 1 -- start start 2023-05-20T01:16:39.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.032+0000 7fc54c67a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5440cf2c0 0x7fc544152400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:39.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.032+0000 7fc54c67a700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc54415cec0 0x7fc5440ae170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:39.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.032+0000 7fc54c67a700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc54415d290 0x7fc5440ac8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:39.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.032+0000 7fc54c67a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc544152940 con 0x7fc5440cf2c0 2023-05-20T01:16:39.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.032+0000 7fc54c67a700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5440ae6b0 con 0x7fc54415d290 2023-05-20T01:16:39.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.032+0000 7fc54c67a700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5440ad0d0 con 0x7fc54415cec0 2023-05-20T01:16:39.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.032+0000 7fc54a416700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5440cf2c0 0x7fc544152400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:39.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.032+0000 7fc54a416700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5440cf2c0 0x7fc544152400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57750/0 (socket says 172.21.15.16:57750) 2023-05-20T01:16:39.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.032+0000 7fc54a416700 1 -- 172.21.15.16:0/1475372840 learned_addr learned my addr 172.21.15.16:0/1475372840 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:39.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.032+0000 7fc549c15700 1 --2- 172.21.15.16:0/1475372840 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc54415cec0 0x7fc5440ae170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:39.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.032+0000 7fc54ac17700 1 --2- 172.21.15.16:0/1475372840 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc54415d290 0x7fc5440ac8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:39.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.032+0000 7fc54a416700 1 -- 172.21.15.16:0/1475372840 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc54415cec0 msgr2=0x7fc5440ae170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:39.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.032+0000 7fc54a416700 1 --2- 172.21.15.16:0/1475372840 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc54415cec0 0x7fc5440ae170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:39.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.032+0000 7fc54a416700 1 -- 172.21.15.16:0/1475372840 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc54415d290 msgr2=0x7fc5440ac8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:39.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.032+0000 7fc54a416700 1 --2- 172.21.15.16:0/1475372840 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc54415d290 0x7fc5440ac8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:39.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.032+0000 7fc54a416700 1 -- 172.21.15.16:0/1475372840 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc534009500 con 0x7fc5440cf2c0 2023-05-20T01:16:39.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.032+0000 7fc54ac17700 1 --2- 172.21.15.16:0/1475372840 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc54415d290 0x7fc5440ac8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:16:39.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.032+0000 7fc549c15700 1 --2- 172.21.15.16:0/1475372840 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc54415cec0 0x7fc5440ae170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:16:39.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.032+0000 7fc54a416700 1 --2- 172.21.15.16:0/1475372840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5440cf2c0 0x7fc544152400 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc534009c10 tx=0x7fc534004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:39.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.032+0000 7fc53affd700 1 -- 172.21.15.16:0/1475372840 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc534003d90 con 0x7fc5440cf2c0 2023-05-20T01:16:39.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.032+0000 7fc53affd700 1 -- 172.21.15.16:0/1475372840 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc53401c420 con 0x7fc5440cf2c0 2023-05-20T01:16:39.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.032+0000 7fc54c67a700 1 -- 172.21.15.16:0/1475372840 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5440cd2c0 con 0x7fc5440cf2c0 2023-05-20T01:16:39.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.032+0000 7fc53affd700 1 -- 172.21.15.16:0/1475372840 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc53401c7b0 con 0x7fc5440cf2c0 2023-05-20T01:16:39.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.032+0000 7fc54c67a700 1 -- 172.21.15.16:0/1475372840 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5440ccfe0 con 0x7fc5440cf2c0 2023-05-20T01:16:39.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.033+0000 7fc54c67a700 1 -- 172.21.15.16:0/1475372840 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc544160890 con 0x7fc5440cf2c0 2023-05-20T01:16:39.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.034+0000 7fc53affd700 1 -- 172.21.15.16:0/1475372840 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc53401c910 con 0x7fc5440cf2c0 2023-05-20T01:16:39.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.034+0000 7fc53affd700 1 --2- 172.21.15.16:0/1475372840 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc52806ace0 0x7fc52806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:39.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.034+0000 7fc53affd700 1 -- 172.21.15.16:0/1475372840 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fc53408a870 con 0x7fc5440cf2c0 2023-05-20T01:16:39.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.034+0000 7fc549c15700 1 --2- 172.21.15.16:0/1475372840 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc52806ace0 0x7fc52806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:39.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.035+0000 7fc549c15700 1 --2- 172.21.15.16:0/1475372840 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc52806ace0 0x7fc52806d190 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc5441f6480 tx=0x7fc540008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:39.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.039+0000 7fc53affd700 1 -- 172.21.15.16:0/1475372840 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fc534055620 con 0x7fc5440cf2c0 2023-05-20T01:16:39.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.237+0000 7fc54c67a700 1 -- 172.21.15.16:0/1475372840 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc5440747b0 con 0x7fc5440cf2c0 2023-05-20T01:16:39.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.238+0000 7fc53affd700 1 -- 172.21.15.16:0/1475372840 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc534058c40 con 0x7fc5440cf2c0 2023-05-20T01:16:39.242 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-05-20T01:16:39.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.240+0000 7fc54c67a700 1 -- 172.21.15.16:0/1475372840 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc52806ace0 msgr2=0x7fc52806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:39.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.240+0000 7fc54c67a700 1 --2- 172.21.15.16:0/1475372840 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc52806ace0 0x7fc52806d190 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc5441f6480 tx=0x7fc540008040 comp rx=0 tx=0).stop 2023-05-20T01:16:39.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.240+0000 7fc54c67a700 1 -- 172.21.15.16:0/1475372840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5440cf2c0 msgr2=0x7fc544152400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:39.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.240+0000 7fc54c67a700 1 --2- 172.21.15.16:0/1475372840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5440cf2c0 0x7fc544152400 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc534009c10 tx=0x7fc534004750 comp rx=0 tx=0).stop 2023-05-20T01:16:39.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.240+0000 7fc54c67a700 1 -- 172.21.15.16:0/1475372840 shutdown_connections 2023-05-20T01:16:39.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.240+0000 7fc54c67a700 1 --2- 172.21.15.16:0/1475372840 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc54415d290 0x7fc5440ac8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:39.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.240+0000 7fc54c67a700 1 --2- 172.21.15.16:0/1475372840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5440cf2c0 0x7fc544152400 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:39.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.240+0000 7fc54c67a700 1 --2- 172.21.15.16:0/1475372840 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc52806ace0 0x7fc52806d190 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:39.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.240+0000 7fc54c67a700 1 --2- 172.21.15.16:0/1475372840 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc54415cec0 0x7fc5440ae170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:39.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.240+0000 7fc54c67a700 1 -- 172.21.15.16:0/1475372840 >> 172.21.15.16:0/1475372840 conn(0x7fc5440cffd0 msgr2=0x7fc5440cf6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:39.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.240+0000 7fc54c67a700 1 -- 172.21.15.16:0/1475372840 shutdown_connections 2023-05-20T01:16:39.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.240+0000 7fc54c67a700 1 -- 172.21.15.16:0/1475372840 wait complete. 2023-05-20T01:16:39.256 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-05-20T01:16:39.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.381+0000 7fe571186700 1 Processor -- start 2023-05-20T01:16:39.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.381+0000 7fe571186700 1 -- start start 2023-05-20T01:16:39.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.382+0000 7fe571186700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fe56c072010 0x7fe56c0711b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:39.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.382+0000 7fe571186700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe56c0d0b80 0x7fe56c0716f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:39.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.382+0000 7fe571186700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe56c06cbd0 con 0x7fe56c0af4d0 2023-05-20T01:16:39.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.382+0000 7fe571186700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe56c0cadb0 con 0x7fe56c0d0b80 2023-05-20T01:16:39.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.382+0000 7fe571186700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe56c0af8a0 con 0x7fe56c072010 2023-05-20T01:16:39.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.382+0000 7fe56b59e700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe56c0d0b80 0x7fe56c0716f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:39.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.382+0000 7fe56b59e700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe56c0d0b80 0x7fe56c0716f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57762/0 (socket says 172.21.15.16:57762) 2023-05-20T01:16:39.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.382+0000 7fe56ad9d700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fe56c0af4d0 0x7fe56c0ca6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:41054/0 (socket says 172.21.15.16:41054) 2023-05-20T01:16:39.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.382+0000 7fe56b59e700 1 -- 172.21.15.16:0/3581656736 learned_addr learned my addr 172.21.15.16:0/3581656736 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:39.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.382+0000 7fe56a59c700 1 --2- 172.21.15.16:0/3581656736 >> v2:172.21.15.132:3300/0 conn(0x7fe56c072010 0x7fe56c0711b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:39.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.382+0000 7fe56b59e700 1 -- 172.21.15.16:0/3581656736 >> v2:172.21.15.132:3300/0 conn(0x7fe56c072010 msgr2=0x7fe56c0711b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:39.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.382+0000 7fe56b59e700 1 --2- 172.21.15.16:0/3581656736 >> v2:172.21.15.132:3300/0 conn(0x7fe56c072010 0x7fe56c0711b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:39.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.382+0000 7fe56b59e700 1 -- 172.21.15.16:0/3581656736 >> v1:172.21.15.16:6789/0 conn(0x7fe56c0af4d0 legacy=0x7fe56c0ca6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:39.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.382+0000 7fe56b59e700 1 -- 172.21.15.16:0/3581656736 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe56c06ce20 con 0x7fe56c0d0b80 2023-05-20T01:16:39.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.382+0000 7fe569d9b700 1 -- 172.21.15.16:0/3581656736 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 39823960 0 0) 0x7fe56c06cbd0 con 0x7fe56c0af4d0 2023-05-20T01:16:39.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.382+0000 7fe56a59c700 1 --2- 172.21.15.16:0/3581656736 >> v2:172.21.15.132:3300/0 conn(0x7fe56c072010 0x7fe56c0711b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:16:39.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.382+0000 7fe56b59e700 1 --2- 172.21.15.16:0/3581656736 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe56c0d0b80 0x7fe56c0716f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe56c0cf4a0 tx=0x7fe56400d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5bf78a68419e1d7 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:39.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.382+0000 7fe569d9b700 1 -- 172.21.15.16:0/3581656736 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe564010070 con 0x7fe56c0d0b80 2023-05-20T01:16:39.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.382+0000 7fe569d9b700 1 -- 172.21.15.16:0/3581656736 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe56400dd90 con 0x7fe56c0d0b80 2023-05-20T01:16:39.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.382+0000 7fe569d9b700 1 -- 172.21.15.16:0/3581656736 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe56400b310 con 0x7fe56c0d0b80 2023-05-20T01:16:39.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.383+0000 7fe571186700 1 -- 172.21.15.16:0/3581656736 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe56c0d0b80 msgr2=0x7fe56c0716f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:39.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.383+0000 7fe571186700 1 --2- 172.21.15.16:0/3581656736 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe56c0d0b80 0x7fe56c0716f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe56c0cf4a0 tx=0x7fe56400d430 comp rx=0 tx=0).stop 2023-05-20T01:16:39.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.383+0000 7fe571186700 1 -- 172.21.15.16:0/3581656736 shutdown_connections 2023-05-20T01:16:39.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.383+0000 7fe571186700 1 --2- 172.21.15.16:0/3581656736 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe56c0d0b80 0x7fe56c0716f0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:39.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.383+0000 7fe571186700 1 --2- 172.21.15.16:0/3581656736 >> v2:172.21.15.132:3300/0 conn(0x7fe56c072010 0x7fe56c0711b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:39.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.383+0000 7fe571186700 1 -- 172.21.15.16:0/3581656736 >> 172.21.15.16:0/3581656736 conn(0x7fe56c0cd760 msgr2=0x7fe56c0d0050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:39.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.383+0000 7fe571186700 1 -- 172.21.15.16:0/3581656736 shutdown_connections 2023-05-20T01:16:39.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.383+0000 7fe571186700 1 -- 172.21.15.16:0/3581656736 wait complete. 2023-05-20T01:16:39.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.384+0000 7fe571186700 1 Processor -- start 2023-05-20T01:16:39.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.385+0000 7fe571186700 1 -- start start 2023-05-20T01:16:39.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.385+0000 7fe571186700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe56c072010 0x7fe56c0c5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:39.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.385+0000 7fe571186700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe56c0af4d0 0x7fe56c1503a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:39.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.385+0000 7fe571186700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe56c0d0b80 0x7fe56c0c3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:39.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.385+0000 7fe571186700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe56c0c8f70 con 0x7fe56c072010 2023-05-20T01:16:39.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.385+0000 7fe571186700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe56c1508e0 con 0x7fe56c0af4d0 2023-05-20T01:16:39.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.385+0000 7fe571186700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe56c0b32a0 con 0x7fe56c0d0b80 2023-05-20T01:16:39.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.385+0000 7fe56ad9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe56c072010 0x7fe56c0c5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:39.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.385+0000 7fe56ad9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe56c072010 0x7fe56c0c5840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57764/0 (socket says 172.21.15.16:57764) 2023-05-20T01:16:39.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.385+0000 7fe56ad9d700 1 -- 172.21.15.16:0/2093516915 learned_addr learned my addr 172.21.15.16:0/2093516915 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:39.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.386+0000 7fe56a59c700 1 --2- 172.21.15.16:0/2093516915 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe56c0af4d0 0x7fe56c1503a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:39.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.386+0000 7fe56b59e700 1 --2- 172.21.15.16:0/2093516915 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe56c0d0b80 0x7fe56c0c3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:39.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.386+0000 7fe56ad9d700 1 -- 172.21.15.16:0/2093516915 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe56c0d0b80 msgr2=0x7fe56c0c3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:39.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.386+0000 7fe56ad9d700 1 --2- 172.21.15.16:0/2093516915 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe56c0d0b80 0x7fe56c0c3690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:39.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.386+0000 7fe56ad9d700 1 -- 172.21.15.16:0/2093516915 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe56c0af4d0 msgr2=0x7fe56c1503a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:39.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.386+0000 7fe56ad9d700 1 --2- 172.21.15.16:0/2093516915 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe56c0af4d0 0x7fe56c1503a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:39.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.386+0000 7fe56ad9d700 1 -- 172.21.15.16:0/2093516915 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe56400bd40 con 0x7fe56c072010 2023-05-20T01:16:39.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.386+0000 7fe56ad9d700 1 --2- 172.21.15.16:0/2093516915 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe56c072010 0x7fe56c0c5840 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe55c0011e0 tx=0x7fe55c00cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:39.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.386+0000 7fe5537fe700 1 -- 172.21.15.16:0/2093516915 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe55c004c80 con 0x7fe56c072010 2023-05-20T01:16:39.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.386+0000 7fe5537fe700 1 -- 172.21.15.16:0/2093516915 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe55c004e00 con 0x7fe56c072010 2023-05-20T01:16:39.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.386+0000 7fe571186700 1 -- 172.21.15.16:0/2093516915 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe56c06d3e0 con 0x7fe56c072010 2023-05-20T01:16:39.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.386+0000 7fe5537fe700 1 -- 172.21.15.16:0/2093516915 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe55c007310 con 0x7fe56c072010 2023-05-20T01:16:39.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.386+0000 7fe571186700 1 -- 172.21.15.16:0/2093516915 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe56c06d100 con 0x7fe56c072010 2023-05-20T01:16:39.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.387+0000 7fe571186700 1 -- 172.21.15.16:0/2093516915 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe56c06d6c0 con 0x7fe56c072010 2023-05-20T01:16:39.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.388+0000 7fe5537fe700 1 -- 172.21.15.16:0/2093516915 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe55c007470 con 0x7fe56c072010 2023-05-20T01:16:39.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.388+0000 7fe5537fe700 1 --2- 172.21.15.16:0/2093516915 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe55806ace0 0x7fe55806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:39.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.389+0000 7fe5537fe700 1 -- 172.21.15.16:0/2093516915 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fe55c089a40 con 0x7fe56c072010 2023-05-20T01:16:39.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.389+0000 7fe56a59c700 1 --2- 172.21.15.16:0/2093516915 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe55806ace0 0x7fe55806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:39.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.389+0000 7fe56a59c700 1 --2- 172.21.15.16:0/2093516915 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe55806ace0 0x7fe55806d190 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe56c159a80 tx=0x7fe560008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:39.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.392+0000 7fe5537fe700 1 -- 172.21.15.16:0/2093516915 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fe55c054970 con 0x7fe56c072010 2023-05-20T01:16:39.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.588+0000 7fe571186700 1 -- 172.21.15.16:0/2093516915 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe56c0747b0 con 0x7fe56c072010 2023-05-20T01:16:39.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.589+0000 7fe5537fe700 1 -- 172.21.15.16:0/2093516915 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe55c057f90 con 0x7fe56c072010 2023-05-20T01:16:39.593 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2023-05-20T01:16:39.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.591+0000 7fe571186700 1 -- 172.21.15.16:0/2093516915 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe55806ace0 msgr2=0x7fe55806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:39.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.591+0000 7fe571186700 1 --2- 172.21.15.16:0/2093516915 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe55806ace0 0x7fe55806d190 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe56c159a80 tx=0x7fe560008040 comp rx=0 tx=0).stop 2023-05-20T01:16:39.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.591+0000 7fe571186700 1 -- 172.21.15.16:0/2093516915 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe56c072010 msgr2=0x7fe56c0c5840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:39.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.591+0000 7fe571186700 1 --2- 172.21.15.16:0/2093516915 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe56c072010 0x7fe56c0c5840 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe55c0011e0 tx=0x7fe55c00cae0 comp rx=0 tx=0).stop 2023-05-20T01:16:39.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.591+0000 7fe571186700 1 -- 172.21.15.16:0/2093516915 shutdown_connections 2023-05-20T01:16:39.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.591+0000 7fe571186700 1 --2- 172.21.15.16:0/2093516915 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe55806ace0 0x7fe55806d190 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:39.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.591+0000 7fe571186700 1 --2- 172.21.15.16:0/2093516915 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe56c0d0b80 0x7fe56c0c3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:39.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.591+0000 7fe571186700 1 --2- 172.21.15.16:0/2093516915 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe56c072010 0x7fe56c0c5840 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:39.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.591+0000 7fe571186700 1 --2- 172.21.15.16:0/2093516915 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe56c0af4d0 0x7fe56c1503a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:39.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.591+0000 7fe571186700 1 -- 172.21.15.16:0/2093516915 >> 172.21.15.16:0/2093516915 conn(0x7fe56c0cd760 msgr2=0x7fe56c0d0170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:39.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.591+0000 7fe571186700 1 -- 172.21.15.16:0/2093516915 shutdown_connections 2023-05-20T01:16:39.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.591+0000 7fe571186700 1 -- 172.21.15.16:0/2093516915 wait complete. 2023-05-20T01:16:39.606 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-05-20T01:16:39.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.730+0000 7fa358c4f700 1 Processor -- start 2023-05-20T01:16:39.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.730+0000 7fa358c4f700 1 -- start start 2023-05-20T01:16:39.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.731+0000 7fa358c4f700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa35415cec0 0x7fa354071bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:39.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.731+0000 7fa358c4f700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa354072590 con 0x7fa354072130 2023-05-20T01:16:39.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.731+0000 7fa358c4f700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3540cd630 con 0x7fa35415d290 2023-05-20T01:16:39.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.731+0000 7fa358c4f700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3540cca60 con 0x7fa35415cec0 2023-05-20T01:16:39.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.731+0000 7fa352d9d700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fa354072130 0x7fa354160890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:41072/0 (socket says 172.21.15.16:41072) 2023-05-20T01:16:39.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.731+0000 7fa352d9d700 1 -- 172.21.15.16:0/4278521299 learned_addr learned my addr 172.21.15.16:0/4278521299 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:16:39.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.731+0000 7fa35259c700 1 --2- 172.21.15.16:0/4278521299 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa35415cec0 0x7fa354071bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:39.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.731+0000 7fa35159a700 1 -- 172.21.15.16:0/4278521299 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 383242650 0 0) 0x7fa354072590 con 0x7fa354072130 2023-05-20T01:16:39.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.731+0000 7fa35159a700 1 -- 172.21.15.16:0/4278521299 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa33c0035b0 con 0x7fa354072130 2023-05-20T01:16:39.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.732+0000 7fa35159a700 1 -- 172.21.15.16:0/4278521299 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1629707666 0 0) 0x7fa33c0035b0 con 0x7fa354072130 2023-05-20T01:16:39.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.732+0000 7fa35159a700 1 -- 172.21.15.16:0/4278521299 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa35415cec0 msgr2=0x7fa354071bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:39.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.732+0000 7fa35159a700 1 --2- 172.21.15.16:0/4278521299 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa35415cec0 0x7fa354071bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:39.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.732+0000 7fa35159a700 1 -- 172.21.15.16:0/4278521299 >> v1:172.21.15.132:6789/0 conn(0x7fa35415d290 legacy=0x7fa3540cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:39.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.732+0000 7fa35159a700 1 -- 172.21.15.16:0/4278521299 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3540ccd00 con 0x7fa354072130 2023-05-20T01:16:39.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.732+0000 7fa35159a700 1 -- 172.21.15.16:0/4278521299 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7fa34c0019f0 con 0x7fa354072130 2023-05-20T01:16:39.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.732+0000 7fa35159a700 1 -- 172.21.15.16:0/4278521299 >> v1:172.21.15.16:6789/0 conn(0x7fa354072130 legacy=0x7fa354160890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:39.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.732+0000 7fa35159a700 1 --2- 172.21.15.16:0/4278521299 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa35415cec0 0x7fa33c003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:39.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.732+0000 7fa35159a700 1 --2- 172.21.15.16:0/4278521299 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa33c003fb0 0x7fa33c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:39.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.732+0000 7fa35159a700 1 --2- 172.21.15.16:0/4278521299 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa33c004d90 0x7fa33c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:39.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.732+0000 7fa35159a700 1 -- 172.21.15.16:0/4278521299 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3540cca60 con 0x7fa33c003fb0 2023-05-20T01:16:39.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.732+0000 7fa35159a700 1 -- 172.21.15.16:0/4278521299 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa33c00bba0 con 0x7fa33c004d90 2023-05-20T01:16:39.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.732+0000 7fa35159a700 1 -- 172.21.15.16:0/4278521299 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa33c00bd60 con 0x7fa35415cec0 2023-05-20T01:16:39.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.732+0000 7fa351d9b700 1 --2- 172.21.15.16:0/4278521299 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa33c003fb0 0x7fa33c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:39.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.732+0000 7fa35259c700 1 --2- 172.21.15.16:0/4278521299 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa35415cec0 0x7fa33c003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:39.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.732+0000 7fa352d9d700 1 --2- 172.21.15.16:0/4278521299 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa33c004d90 0x7fa33c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:39.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.732+0000 7fa351d9b700 1 -- 172.21.15.16:0/4278521299 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa35415cec0 msgr2=0x7fa33c003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:39.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.732+0000 7fa351d9b700 1 --2- 172.21.15.16:0/4278521299 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa35415cec0 0x7fa33c003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:39.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.732+0000 7fa351d9b700 1 -- 172.21.15.16:0/4278521299 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa33c004d90 msgr2=0x7fa33c005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:39.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.732+0000 7fa351d9b700 1 --2- 172.21.15.16:0/4278521299 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa33c004d90 0x7fa33c005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:39.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.732+0000 7fa351d9b700 1 -- 172.21.15.16:0/4278521299 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa33c0035b0 con 0x7fa33c003fb0 2023-05-20T01:16:39.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.732+0000 7fa352d9d700 1 --2- 172.21.15.16:0/4278521299 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa33c004d90 0x7fa33c005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:16:39.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.732+0000 7fa351d9b700 1 --2- 172.21.15.16:0/4278521299 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa33c003fb0 0x7fa33c004850 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa344007fd0 tx=0x7fa33c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87e2f76e6c3b685f server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:39.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.732+0000 7fa35159a700 1 -- 172.21.15.16:0/4278521299 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3440052d0 con 0x7fa33c003fb0 2023-05-20T01:16:39.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.732+0000 7fa35159a700 1 -- 172.21.15.16:0/4278521299 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa344005470 con 0x7fa33c003fb0 2023-05-20T01:16:39.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.733+0000 7fa35159a700 1 -- 172.21.15.16:0/4278521299 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3440038c0 con 0x7fa33c003fb0 2023-05-20T01:16:39.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.733+0000 7fa358c4f700 1 -- 172.21.15.16:0/4278521299 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa33c003fb0 msgr2=0x7fa33c004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:39.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.733+0000 7fa358c4f700 1 --2- 172.21.15.16:0/4278521299 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa33c003fb0 0x7fa33c004850 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa344007fd0 tx=0x7fa33c005880 comp rx=0 tx=0).stop 2023-05-20T01:16:39.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.733+0000 7fa352d9d700 1 -- 172.21.15.16:0/4278521299 reap_dead start 2023-05-20T01:16:39.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.733+0000 7fa358c4f700 1 -- 172.21.15.16:0/4278521299 shutdown_connections 2023-05-20T01:16:39.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.733+0000 7fa358c4f700 1 -- 172.21.15.16:0/4278521299 >> 172.21.15.16:0/4278521299 conn(0x7fa3540cffd0 msgr2=0x7fa3540d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:39.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.733+0000 7fa358c4f700 1 -- 172.21.15.16:0/4278521299 shutdown_connections 2023-05-20T01:16:39.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.733+0000 7fa358c4f700 1 -- 172.21.15.16:0/4278521299 wait complete. 2023-05-20T01:16:39.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.734+0000 7fa358c4f700 1 Processor -- start 2023-05-20T01:16:39.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.734+0000 7fa358c4f700 1 -- start start 2023-05-20T01:16:39.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.735+0000 7fa358c4f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa33c003fb0 0x7fa3540c70a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:39.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.735+0000 7fa358c4f700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa3540b5860 0x7fa35415b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:39.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.735+0000 7fa358c4f700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa3541e65d0 0x7fa3541e6a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:39.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.735+0000 7fa358c4f700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3541e6fb0 con 0x7fa33c003fb0 2023-05-20T01:16:39.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.735+0000 7fa358c4f700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3540b3880 con 0x7fa3540b5860 2023-05-20T01:16:39.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.735+0000 7fa358c4f700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3540c7f20 con 0x7fa3541e65d0 2023-05-20T01:16:39.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.735+0000 7fa35259c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa33c003fb0 0x7fa3540c70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:39.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.735+0000 7fa35259c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa33c003fb0 0x7fa3540c70a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57784/0 (socket says 172.21.15.16:57784) 2023-05-20T01:16:39.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.735+0000 7fa35259c700 1 -- 172.21.15.16:0/821178697 learned_addr learned my addr 172.21.15.16:0/821178697 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:39.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.735+0000 7fa351d9b700 1 --2- 172.21.15.16:0/821178697 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa3540b5860 0x7fa35415b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:39.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.735+0000 7fa352d9d700 1 --2- 172.21.15.16:0/821178697 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa3541e65d0 0x7fa3541e6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:39.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.736+0000 7fa35259c700 1 -- 172.21.15.16:0/821178697 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa3541e65d0 msgr2=0x7fa3541e6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:39.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.736+0000 7fa35259c700 1 --2- 172.21.15.16:0/821178697 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa3541e65d0 0x7fa3541e6a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:39.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.736+0000 7fa35259c700 1 -- 172.21.15.16:0/821178697 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa3540b5860 msgr2=0x7fa35415b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:39.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.736+0000 7fa35259c700 1 --2- 172.21.15.16:0/821178697 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa3540b5860 0x7fa35415b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:39.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.736+0000 7fa35259c700 1 -- 172.21.15.16:0/821178697 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3540cd2c0 con 0x7fa33c003fb0 2023-05-20T01:16:39.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.736+0000 7fa351d9b700 1 --2- 172.21.15.16:0/821178697 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa3540b5860 0x7fa35415b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:16:39.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.736+0000 7fa35259c700 1 --2- 172.21.15.16:0/821178697 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa33c003fb0 0x7fa3540c70a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa3480119f0 tx=0x7fa348011db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:39.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.736+0000 7fa342ffd700 1 -- 172.21.15.16:0/821178697 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa348005c30 con 0x7fa33c003fb0 2023-05-20T01:16:39.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.736+0000 7fa342ffd700 1 -- 172.21.15.16:0/821178697 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa348005d90 con 0x7fa33c003fb0 2023-05-20T01:16:39.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.736+0000 7fa358c4f700 1 -- 172.21.15.16:0/821178697 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3540ccfe0 con 0x7fa33c003fb0 2023-05-20T01:16:39.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.736+0000 7fa342ffd700 1 -- 172.21.15.16:0/821178697 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa34800f950 con 0x7fa33c003fb0 2023-05-20T01:16:39.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.736+0000 7fa358c4f700 1 -- 172.21.15.16:0/821178697 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa354071bf0 con 0x7fa33c003fb0 2023-05-20T01:16:39.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.737+0000 7fa358c4f700 1 -- 172.21.15.16:0/821178697 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa354066d90 con 0x7fa33c003fb0 2023-05-20T01:16:39.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.737+0000 7fa342ffd700 1 -- 172.21.15.16:0/821178697 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa34800fab0 con 0x7fa33c003fb0 2023-05-20T01:16:39.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.738+0000 7fa342ffd700 1 --2- 172.21.15.16:0/821178697 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa33406ace0 0x7fa33406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:39.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.738+0000 7fa342ffd700 1 -- 172.21.15.16:0/821178697 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fa348089fd0 con 0x7fa33c003fb0 2023-05-20T01:16:39.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.738+0000 7fa351d9b700 1 --2- 172.21.15.16:0/821178697 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa33406ace0 0x7fa33406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:39.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.738+0000 7fa351d9b700 1 --2- 172.21.15.16:0/821178697 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa33406ace0 0x7fa33406d190 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa344007fd0 tx=0x7fa344009030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:39.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.742+0000 7fa342ffd700 1 -- 172.21.15.16:0/821178697 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fa348054e20 con 0x7fa33c003fb0 2023-05-20T01:16:39.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.940+0000 7fa358c4f700 1 -- 172.21.15.16:0/821178697 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fa3540747b0 con 0x7fa33c003fb0 2023-05-20T01:16:39.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.940+0000 7fa342ffd700 1 -- 172.21.15.16:0/821178697 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7fa348058440 con 0x7fa33c003fb0 2023-05-20T01:16:39.944 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-05-20T01:16:39.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.942+0000 7fa358c4f700 1 -- 172.21.15.16:0/821178697 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa33406ace0 msgr2=0x7fa33406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:39.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.942+0000 7fa358c4f700 1 --2- 172.21.15.16:0/821178697 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa33406ace0 0x7fa33406d190 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa344007fd0 tx=0x7fa344009030 comp rx=0 tx=0).stop 2023-05-20T01:16:39.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.942+0000 7fa358c4f700 1 -- 172.21.15.16:0/821178697 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa33c003fb0 msgr2=0x7fa3540c70a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:39.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.942+0000 7fa358c4f700 1 --2- 172.21.15.16:0/821178697 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa33c003fb0 0x7fa3540c70a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa3480119f0 tx=0x7fa348011db0 comp rx=0 tx=0).stop 2023-05-20T01:16:39.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.942+0000 7fa358c4f700 1 -- 172.21.15.16:0/821178697 shutdown_connections 2023-05-20T01:16:39.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.942+0000 7fa358c4f700 1 --2- 172.21.15.16:0/821178697 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa33406ace0 0x7fa33406d190 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:39.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.942+0000 7fa358c4f700 1 --2- 172.21.15.16:0/821178697 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa3541e65d0 0x7fa3541e6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:39.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.942+0000 7fa358c4f700 1 --2- 172.21.15.16:0/821178697 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa33c003fb0 0x7fa3540c70a0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:39.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.942+0000 7fa358c4f700 1 --2- 172.21.15.16:0/821178697 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa3540b5860 0x7fa35415b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:39.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.942+0000 7fa358c4f700 1 -- 172.21.15.16:0/821178697 >> 172.21.15.16:0/821178697 conn(0x7fa3540cffd0 msgr2=0x7fa35406a510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:39.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.942+0000 7fa358c4f700 1 -- 172.21.15.16:0/821178697 shutdown_connections 2023-05-20T01:16:39.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:39.942+0000 7fa358c4f700 1 -- 172.21.15.16:0/821178697 wait complete. 2023-05-20T01:16:39.958 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-05-20T01:16:40.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.079+0000 7f5135be0700 1 Processor -- start 2023-05-20T01:16:40.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.079+0000 7f5135be0700 1 -- start start 2023-05-20T01:16:40.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.080+0000 7f5135be0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f51301585d0 0x7f513015ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:40.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.080+0000 7f5135be0700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f513006ac20 0x7f513015e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:40.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.080+0000 7f5135be0700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51300ccf40 con 0x7f513015b360 2023-05-20T01:16:40.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.080+0000 7f5135be0700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f513015e540 con 0x7f513006ac20 2023-05-20T01:16:40.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.080+0000 7f5135be0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51300cd220 con 0x7f51301585d0 2023-05-20T01:16:40.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.080+0000 7f512ffff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f513006ac20 0x7f513015e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:40.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.080+0000 7f512ffff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f513006ac20 0x7f513015e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57796/0 (socket says 172.21.15.16:57796) 2023-05-20T01:16:40.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.080+0000 7f512ffff700 1 -- 172.21.15.16:0/3721144700 learned_addr learned my addr 172.21.15.16:0/3721144700 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:40.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.080+0000 7f512f7fe700 1 --2- 172.21.15.16:0/3721144700 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f51301585d0 0x7f513015ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:40.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.081+0000 7f512ffff700 1 -- 172.21.15.16:0/3721144700 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f51301585d0 msgr2=0x7f513015ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:40.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.081+0000 7f512ffff700 1 --2- 172.21.15.16:0/3721144700 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f51301585d0 0x7f513015ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:40.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.081+0000 7f512ffff700 1 -- 172.21.15.16:0/3721144700 >> v1:172.21.15.132:6789/0 conn(0x7f513015b360 legacy=0x7f513006a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:40.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.081+0000 7f512ffff700 1 -- 172.21.15.16:0/3721144700 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51300cd500 con 0x7f513006ac20 2023-05-20T01:16:40.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.081+0000 7f512ffff700 1 --2- 172.21.15.16:0/3721144700 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f513006ac20 0x7f513015e990 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5128009e50 tx=0x7f512800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10865c0e07f218a6 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:40.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.081+0000 7f512e7fc700 1 -- 172.21.15.16:0/3721144700 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f512800e070 con 0x7f513006ac20 2023-05-20T01:16:40.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.081+0000 7f512e7fc700 1 -- 172.21.15.16:0/3721144700 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f512800b910 con 0x7f513006ac20 2023-05-20T01:16:40.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.081+0000 7f512e7fc700 1 -- 172.21.15.16:0/3721144700 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f512800bcc0 con 0x7f513006ac20 2023-05-20T01:16:40.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.081+0000 7f5135be0700 1 -- 172.21.15.16:0/3721144700 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f513006ac20 msgr2=0x7f513015e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:40.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.081+0000 7f5135be0700 1 --2- 172.21.15.16:0/3721144700 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f513006ac20 0x7f513015e990 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5128009e50 tx=0x7f512800b330 comp rx=0 tx=0).stop 2023-05-20T01:16:40.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.082+0000 7f5135be0700 1 -- 172.21.15.16:0/3721144700 shutdown_connections 2023-05-20T01:16:40.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.082+0000 7f5135be0700 1 --2- 172.21.15.16:0/3721144700 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f513006ac20 0x7f513015e990 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:40.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.082+0000 7f5135be0700 1 --2- 172.21.15.16:0/3721144700 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f51301585d0 0x7f513015ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:40.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.082+0000 7f5135be0700 1 -- 172.21.15.16:0/3721144700 >> 172.21.15.16:0/3721144700 conn(0x7f51300cffb0 msgr2=0x7f51300d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:40.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.082+0000 7f5135be0700 1 -- 172.21.15.16:0/3721144700 shutdown_connections 2023-05-20T01:16:40.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.082+0000 7f5135be0700 1 -- 172.21.15.16:0/3721144700 wait complete. 2023-05-20T01:16:40.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.083+0000 7f5135be0700 1 Processor -- start 2023-05-20T01:16:40.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.083+0000 7f5135be0700 1 -- start start 2023-05-20T01:16:40.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.084+0000 7f5135be0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f513006ac20 0x7f51300cea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:40.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.084+0000 7f5135be0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51301585d0 0x7f51300ad190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:40.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.084+0000 7f5135be0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f513015b360 0x7f51300c4310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:40.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.084+0000 7f5135be0700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5130156d60 con 0x7f51301585d0 2023-05-20T01:16:40.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.084+0000 7f5135be0700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51301e3680 con 0x7f513006ac20 2023-05-20T01:16:40.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.084+0000 7f5135be0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51301e3960 con 0x7f513015b360 2023-05-20T01:16:40.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.084+0000 7f512effd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51301585d0 0x7f51300ad190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:40.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.084+0000 7f512effd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51301585d0 0x7f51300ad190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57800/0 (socket says 172.21.15.16:57800) 2023-05-20T01:16:40.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.084+0000 7f512effd700 1 -- 172.21.15.16:0/3626247341 learned_addr learned my addr 172.21.15.16:0/3626247341 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:40.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.084+0000 7f512f7fe700 1 --2- 172.21.15.16:0/3626247341 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f513006ac20 0x7f51300cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:40.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.084+0000 7f512ffff700 1 --2- 172.21.15.16:0/3626247341 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f513015b360 0x7f51300c4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:40.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.084+0000 7f512effd700 1 -- 172.21.15.16:0/3626247341 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f513015b360 msgr2=0x7f51300c4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:40.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.084+0000 7f512effd700 1 --2- 172.21.15.16:0/3626247341 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f513015b360 0x7f51300c4310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:40.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.084+0000 7f512effd700 1 -- 172.21.15.16:0/3626247341 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f513006ac20 msgr2=0x7f51300cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:40.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.084+0000 7f512effd700 1 --2- 172.21.15.16:0/3626247341 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f513006ac20 0x7f51300cea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:40.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.084+0000 7f512effd700 1 -- 172.21.15.16:0/3626247341 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5128009b30 con 0x7f51301585d0 2023-05-20T01:16:40.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.084+0000 7f512f7fe700 1 --2- 172.21.15.16:0/3626247341 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f513006ac20 0x7f51300cea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:16:40.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.084+0000 7f512effd700 1 --2- 172.21.15.16:0/3626247341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51301585d0 0x7f51300ad190 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f511800c420 tx=0x7f511800c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:40.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.084+0000 7f5127fff700 1 -- 172.21.15.16:0/3626247341 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51180052d0 con 0x7f51301585d0 2023-05-20T01:16:40.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.084+0000 7f5127fff700 1 -- 172.21.15.16:0/3626247341 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5118005430 con 0x7f51301585d0 2023-05-20T01:16:40.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.084+0000 7f5135be0700 1 -- 172.21.15.16:0/3626247341 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51300719a0 con 0x7f51301585d0 2023-05-20T01:16:40.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.084+0000 7f5127fff700 1 -- 172.21.15.16:0/3626247341 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51180038c0 con 0x7f51301585d0 2023-05-20T01:16:40.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.084+0000 7f5135be0700 1 -- 172.21.15.16:0/3626247341 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51300cd7e0 con 0x7f51301585d0 2023-05-20T01:16:40.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.086+0000 7f5135be0700 1 -- 172.21.15.16:0/3626247341 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51301e4570 con 0x7f51301585d0 2023-05-20T01:16:40.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.086+0000 7f5127fff700 1 -- 172.21.15.16:0/3626247341 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5118003a20 con 0x7f51301585d0 2023-05-20T01:16:40.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.086+0000 7f5127fff700 1 --2- 172.21.15.16:0/3626247341 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f511006ace0 0x7f511006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:40.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.087+0000 7f512f7fe700 1 --2- 172.21.15.16:0/3626247341 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f511006ace0 0x7f511006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:40.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.087+0000 7f5127fff700 1 -- 172.21.15.16:0/3626247341 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f5118088b10 con 0x7f51301585d0 2023-05-20T01:16:40.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.087+0000 7f512f7fe700 1 --2- 172.21.15.16:0/3626247341 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f511006ace0 0x7f511006d190 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f51301f5a30 tx=0x7f5120008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:40.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.091+0000 7f5127fff700 1 -- 172.21.15.16:0/3626247341 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f5118053960 con 0x7f51301585d0 2023-05-20T01:16:40.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.302+0000 7f5135be0700 1 -- 172.21.15.16:0/3626247341 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f51300747b0 con 0x7f51301585d0 2023-05-20T01:16:40.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.303+0000 7f5127fff700 1 -- 172.21.15.16:0/3626247341 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) v1 ==== 80+0+0 (secure 0 0 0) 0x7f5118056f80 con 0x7f51301585d0 2023-05-20T01:16:40.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.304+0000 7f5135be0700 1 -- 172.21.15.16:0/3626247341 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f511006ace0 msgr2=0x7f511006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:40.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.305+0000 7f5135be0700 1 --2- 172.21.15.16:0/3626247341 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f511006ace0 0x7f511006d190 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f51301f5a30 tx=0x7f5120008040 comp rx=0 tx=0).stop 2023-05-20T01:16:40.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.305+0000 7f5135be0700 1 -- 172.21.15.16:0/3626247341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51301585d0 msgr2=0x7f51300ad190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:40.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.305+0000 7f5135be0700 1 --2- 172.21.15.16:0/3626247341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51301585d0 0x7f51300ad190 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f511800c420 tx=0x7f511800c7e0 comp rx=0 tx=0).stop 2023-05-20T01:16:40.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.305+0000 7f5135be0700 1 -- 172.21.15.16:0/3626247341 shutdown_connections 2023-05-20T01:16:40.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.305+0000 7f5135be0700 1 --2- 172.21.15.16:0/3626247341 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f511006ace0 0x7f511006d190 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:40.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.305+0000 7f5135be0700 1 --2- 172.21.15.16:0/3626247341 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f513015b360 0x7f51300c4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:40.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.305+0000 7f5135be0700 1 --2- 172.21.15.16:0/3626247341 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f513006ac20 0x7f51300cea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:40.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.305+0000 7f5135be0700 1 --2- 172.21.15.16:0/3626247341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51301585d0 0x7f51300ad190 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:40.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.305+0000 7f5135be0700 1 -- 172.21.15.16:0/3626247341 >> 172.21.15.16:0/3626247341 conn(0x7f51300cffb0 msgr2=0x7f5130157440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:40.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.305+0000 7f5135be0700 1 -- 172.21.15.16:0/3626247341 shutdown_connections 2023-05-20T01:16:40.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.305+0000 7f5135be0700 1 -- 172.21.15.16:0/3626247341 wait complete. 2023-05-20T01:16:40.310 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2023-05-20T01:16:40.320 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-05-20T01:16:40.321 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2023-05-20T01:16:40.338 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-20T01:16:40.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.499+0000 7f5497010700 1 Processor -- start 2023-05-20T01:16:40.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.499+0000 7f5497010700 1 -- start start 2023-05-20T01:16:40.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.500+0000 7f5497010700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5490159000 0x7f549006cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:40.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.500+0000 7f5497010700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54900d0500 con 0x7f54900cb320 2023-05-20T01:16:40.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.500+0000 7f5497010700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5490071920 con 0x7f54900b1b00 2023-05-20T01:16:40.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.500+0000 7f5497010700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54900ca350 con 0x7f5490159000 2023-05-20T01:16:40.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.500+0000 7f548ffff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f54900cb320 0x7f54900cf6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:41104/0 (socket says 172.21.15.16:41104) 2023-05-20T01:16:40.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.500+0000 7f548ffff700 1 -- 172.21.15.16:0/4170455484 learned_addr learned my addr 172.21.15.16:0/4170455484 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:16:40.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.500+0000 7f5494dac700 1 --2- 172.21.15.16:0/4170455484 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5490159000 0x7f549006cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:40.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.501+0000 7f548f7fe700 1 -- 172.21.15.16:0/4170455484 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4175740203 0 0) 0x7f54900d0500 con 0x7f54900cb320 2023-05-20T01:16:40.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.501+0000 7f548f7fe700 1 -- 172.21.15.16:0/4170455484 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f54800046c0 con 0x7f54900cb320 2023-05-20T01:16:40.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.501+0000 7f548f7fe700 1 -- 172.21.15.16:0/4170455484 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4224915885 0 0) 0x7f5490071920 con 0x7f54900b1b00 2023-05-20T01:16:40.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.501+0000 7f548f7fe700 1 -- 172.21.15.16:0/4170455484 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f54900d0500 con 0x7f54900b1b00 2023-05-20T01:16:40.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.501+0000 7f548f7fe700 1 -- 172.21.15.16:0/4170455484 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2109239243 0 0) 0x7f54800046c0 con 0x7f54900cb320 2023-05-20T01:16:40.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.501+0000 7f5494dac700 1 -- 172.21.15.16:0/4170455484 >> v1:172.21.15.132:6789/0 conn(0x7f54900b1b00 legacy=0x7f54900cfdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:40.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.501+0000 7f5494dac700 1 -- 172.21.15.16:0/4170455484 >> v1:172.21.15.16:6789/0 conn(0x7f54900cb320 legacy=0x7f54900cf6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:40.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.501+0000 7f5494dac700 1 -- 172.21.15.16:0/4170455484 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f549015cac0 con 0x7f5490159000 2023-05-20T01:16:40.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.501+0000 7f5494dac700 1 --2- 172.21.15.16:0/4170455484 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5490159000 0x7f549006cb90 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f5484009870 tx=0x7f5484009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=535dd9ee9fa7668d server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:40.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.501+0000 7f548f7fe700 1 -- 172.21.15.16:0/4170455484 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f547c001670 con 0x7f5490159000 2023-05-20T01:16:40.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.501+0000 7f548f7fe700 1 -- 172.21.15.16:0/4170455484 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5484004030 con 0x7f5490159000 2023-05-20T01:16:40.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.501+0000 7f548f7fe700 1 -- 172.21.15.16:0/4170455484 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54840043a0 con 0x7f5490159000 2023-05-20T01:16:40.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.502+0000 7f5497010700 1 -- 172.21.15.16:0/4170455484 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5490159000 msgr2=0x7f549006cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:40.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.502+0000 7f5497010700 1 --2- 172.21.15.16:0/4170455484 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5490159000 0x7f549006cb90 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f5484009870 tx=0x7f5484009b80 comp rx=0 tx=0).stop 2023-05-20T01:16:40.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.502+0000 7f5497010700 1 -- 172.21.15.16:0/4170455484 shutdown_connections 2023-05-20T01:16:40.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.502+0000 7f5497010700 1 --2- 172.21.15.16:0/4170455484 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5490159000 0x7f549006cb90 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:40.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.502+0000 7f5497010700 1 -- 172.21.15.16:0/4170455484 >> 172.21.15.16:0/4170455484 conn(0x7f54900661b0 msgr2=0x7f5490066580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:40.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.502+0000 7f5497010700 1 -- 172.21.15.16:0/4170455484 shutdown_connections 2023-05-20T01:16:40.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.502+0000 7f5497010700 1 -- 172.21.15.16:0/4170455484 wait complete. 2023-05-20T01:16:40.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.503+0000 7f5497010700 1 Processor -- start 2023-05-20T01:16:40.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.503+0000 7f5497010700 1 -- start start 2023-05-20T01:16:40.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.504+0000 7f5497010700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f54900b1b00 0x7f54901e3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:40.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.504+0000 7f5497010700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f54900cb320 0x7f5490156ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:40.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.504+0000 7f5497010700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5490159000 0x7f549013f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:40.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.504+0000 7f5497010700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5490141fc0 con 0x7f5490159000 2023-05-20T01:16:40.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.504+0000 7f5497010700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5490142d70 con 0x7f54900b1b00 2023-05-20T01:16:40.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.504+0000 7f5497010700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54901421a0 con 0x7f54900cb320 2023-05-20T01:16:40.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.504+0000 7f54955ad700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5490159000 0x7f549013f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:40.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.504+0000 7f54955ad700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5490159000 0x7f549013f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57816/0 (socket says 172.21.15.16:57816) 2023-05-20T01:16:40.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.504+0000 7f54955ad700 1 -- 172.21.15.16:0/1380876358 learned_addr learned my addr 172.21.15.16:0/1380876358 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:40.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.504+0000 7f5494dac700 1 --2- 172.21.15.16:0/1380876358 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f54900b1b00 0x7f54901e3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:40.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.504+0000 7f548ffff700 1 --2- 172.21.15.16:0/1380876358 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f54900cb320 0x7f5490156ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:40.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.504+0000 7f54955ad700 1 -- 172.21.15.16:0/1380876358 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f54900cb320 msgr2=0x7f5490156ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:40.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.504+0000 7f54955ad700 1 --2- 172.21.15.16:0/1380876358 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f54900cb320 0x7f5490156ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:40.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.504+0000 7f54955ad700 1 -- 172.21.15.16:0/1380876358 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f54900b1b00 msgr2=0x7f54901e3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:40.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.504+0000 7f54955ad700 1 --2- 172.21.15.16:0/1380876358 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f54900b1b00 0x7f54901e3a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:40.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.504+0000 7f54955ad700 1 -- 172.21.15.16:0/1380876358 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54840094a0 con 0x7f5490159000 2023-05-20T01:16:40.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.504+0000 7f548ffff700 1 --2- 172.21.15.16:0/1380876358 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f54900cb320 0x7f5490156ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:16:40.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.505+0000 7f54955ad700 1 --2- 172.21.15.16:0/1380876358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5490159000 0x7f549013f0b0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f548800cb50 tx=0x7f548800ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:40.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.505+0000 7f548d7fa700 1 -- 172.21.15.16:0/1380876358 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5488003c00 con 0x7f5490159000 2023-05-20T01:16:40.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.505+0000 7f548d7fa700 1 -- 172.21.15.16:0/1380876358 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5488003d60 con 0x7f5490159000 2023-05-20T01:16:40.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.505+0000 7f5497010700 1 -- 172.21.15.16:0/1380876358 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f549015d080 con 0x7f5490159000 2023-05-20T01:16:40.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.505+0000 7f548d7fa700 1 -- 172.21.15.16:0/1380876358 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f548800aac0 con 0x7f5490159000 2023-05-20T01:16:40.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.505+0000 7f5497010700 1 -- 172.21.15.16:0/1380876358 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f549015cda0 con 0x7f5490159000 2023-05-20T01:16:40.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.506+0000 7f5497010700 1 -- 172.21.15.16:0/1380876358 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f549015d360 con 0x7f5490159000 2023-05-20T01:16:40.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.506+0000 7f548d7fa700 1 -- 172.21.15.16:0/1380876358 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f548800ac20 con 0x7f5490159000 2023-05-20T01:16:40.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.507+0000 7f548d7fa700 1 --2- 172.21.15.16:0/1380876358 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f547406ace0 0x7f547406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:40.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.507+0000 7f5494dac700 1 --2- 172.21.15.16:0/1380876358 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f547406ace0 0x7f547406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:40.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.507+0000 7f548d7fa700 1 -- 172.21.15.16:0/1380876358 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f54880897c0 con 0x7f5490159000 2023-05-20T01:16:40.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.507+0000 7f5494dac700 1 --2- 172.21.15.16:0/1380876358 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f547406ace0 0x7f547406d190 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f5484000c00 tx=0x7f5484008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:40.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.511+0000 7f548d7fa700 1 -- 172.21.15.16:0/1380876358 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f5488054570 con 0x7f5490159000 2023-05-20T01:16:40.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.711+0000 7f5497010700 1 -- 172.21.15.16:0/1380876358 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5490142ee0 con 0x7f5490159000 2023-05-20T01:16:40.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.712+0000 7f548d7fa700 1 -- 172.21.15.16:0/1380876358 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+20026 (secure 0 0 0) 0x7f5488057b90 con 0x7f5490159000 2023-05-20T01:16:40.716 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:16:40.717 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"fsid":"c2245cc5-ecab-4087-a59b-bc81db901676","created":"2023-05-20T01:16:09.549230+0000","modified":"2023-05-20T01:16:32.768495+0000","last_up_change":"2023-05-20T01:16:19.788624+0000","last_in_change":"2023-05-20T01:16:12.092160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-05-20T01:16:20.934366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-05-20T01:16:21.372294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-05-20T01:16:26.208287+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-05-20T01:16:27.292268+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-05-20T01:16:30.357285+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"317a83cd-19c8-4958-a301-6087ef342289","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6825","nonce":3923634179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6827","nonce":3923634179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6831","nonce":3923634179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6829","nonce":3923634179}]},"public_addr":"172.21.15.16:6825/3923634179","cluster_addr":"172.21.15.16:6827/3923634179","heartbeat_back_addr":"172.21.15.16:6831/3923634179","heartbeat_front_addr":"172.21.15.16:6829/3923634179","state":["exists","up"]},{"osd":1,"uuid":"5581046f-8219-4089-8d7d-b7846399505e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6809","nonce":3860264220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6811","nonce":3860264220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6815","nonce":3860264220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6813","nonce":3860264220}]},"public_addr":"172.21.15.16:6809/3860264220","cluster_addr":"172.21.15.16:6811/3860264220","heartbeat_back_addr":"172.21.15.16:6815/3860264220","heartbeat_front_addr":"172.21.15.16:6813/3860264220","state":["exists","up"]},{"osd":2,"uuid":"072e9e66-4133-478e-8d4a-64d46c2963f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6817","nonce":293187030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6819","nonce":293187030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6823","nonce":293187030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6821","nonce":293187030}]},"public_addr":"172.21.15.16:6817/293187030","cluster_addr":"172.21.15.16:6819/293187030","heartbeat_back_addr":"172.21.15.16:6823/293187030","heartbeat_front_addr":"172.21.15.16:6821/293187030","state":["exists","up"]},{"osd":3,"uuid":"11e1bba6-00f8-48dd-9028-d8d716b7f145","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6801","nonce":3669781890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6803","nonce":3669781890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6807","nonce":3669781890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6805","nonce":3669781890}]},"public_addr":"172.21.15.16:6801/3669781890","cluster_addr":"172.21.15.16:6803/3669781890","heartbeat_back_addr":"172.21.15.16:6807/3669781890","heartbeat_front_addr":"172.21.15.16:6805/3669781890","state":["exists","up"]},{"osd":4,"uuid":"b8909c3a-cf70-46cb-8d0b-009c707cf9d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6819","nonce":506317223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6823","nonce":506317223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6829","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6831","nonce":506317223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6827","nonce":506317223}]},"public_addr":"172.21.15.132:6819/506317223","cluster_addr":"172.21.15.132:6823/506317223","heartbeat_back_addr":"172.21.15.132:6831/506317223","heartbeat_front_addr":"172.21.15.132:6827/506317223","state":["exists","up"]},{"osd":5,"uuid":"b1818304-4583-47bf-a6ec-0bfbb5166233","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6818","nonce":1844689995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6822","nonce":1844689995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6830","nonce":1844689995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6826","nonce":1844689995}]},"public_addr":"172.21.15.132:6818/1844689995","cluster_addr":"172.21.15.132:6822/1844689995","heartbeat_back_addr":"172.21.15.132:6830/1844689995","heartbeat_front_addr":"172.21.15.132:6826/1844689995","state":["exists","up"]},{"osd":6,"uuid":"d37f4691-eed1-4a93-8818-18354f03f4a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6801","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6802","nonce":727581123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6805","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6807","nonce":727581123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6813","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6815","nonce":727581123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6811","nonce":727581123}]},"public_addr":"172.21.15.132:6802/727581123","cluster_addr":"172.21.15.132:6807/727581123","heartbeat_back_addr":"172.21.15.132:6815/727581123","heartbeat_front_addr":"172.21.15.132:6811/727581123","state":["exists","up"]},{"osd":7,"uuid":"4f712f62-5d1d-4816-8042-03209144f8b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6803","nonce":466001149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6806","nonce":466001149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6814","nonce":466001149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6810","nonce":466001149}]},"public_addr":"172.21.15.132:6803/466001149","cluster_addr":"172.21.15.132:6806/466001149","heartbeat_back_addr":"172.21.15.132:6814/466001149","heartbeat_front_addr":"172.21.15.132:6810/466001149","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.338863+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.278680+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305694+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.246835+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.276237+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.304441+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305421+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.267050+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-20T01:16:40.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.714+0000 7f5497010700 1 -- 172.21.15.16:0/1380876358 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f547406ace0 msgr2=0x7f547406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:40.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.714+0000 7f5497010700 1 --2- 172.21.15.16:0/1380876358 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f547406ace0 0x7f547406d190 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f5484000c00 tx=0x7f5484008030 comp rx=0 tx=0).stop 2023-05-20T01:16:40.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.714+0000 7f5497010700 1 -- 172.21.15.16:0/1380876358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5490159000 msgr2=0x7f549013f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:40.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.714+0000 7f5497010700 1 --2- 172.21.15.16:0/1380876358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5490159000 0x7f549013f0b0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f548800cb50 tx=0x7f548800ce60 comp rx=0 tx=0).stop 2023-05-20T01:16:40.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.714+0000 7f5497010700 1 -- 172.21.15.16:0/1380876358 shutdown_connections 2023-05-20T01:16:40.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.714+0000 7f5497010700 1 --2- 172.21.15.16:0/1380876358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5490159000 0x7f549013f0b0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:40.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.714+0000 7f5497010700 1 --2- 172.21.15.16:0/1380876358 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f54900b1b00 0x7f54901e3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:40.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.714+0000 7f5497010700 1 --2- 172.21.15.16:0/1380876358 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f547406ace0 0x7f547406d190 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:40.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.714+0000 7f5497010700 1 --2- 172.21.15.16:0/1380876358 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f54900cb320 0x7f5490156ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:40.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.714+0000 7f5497010700 1 -- 172.21.15.16:0/1380876358 >> 172.21.15.16:0/1380876358 conn(0x7f54900661b0 msgr2=0x7f549006a510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:40.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.715+0000 7f5497010700 1 -- 172.21.15.16:0/1380876358 shutdown_connections 2023-05-20T01:16:40.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.715+0000 7f5497010700 1 -- 172.21.15.16:0/1380876358 wait complete. 2023-05-20T01:16:40.730 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-05-20T01:16:20.934366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-05-20T01:16:21.372294+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-05-20T01:16:26.208287+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-05-20T01:16:27.292268+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-05-20T01:16:30.357285+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-05-20T01:16:40.731 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-05-20T01:16:40.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.855+0000 7f242c933700 1 Processor -- start 2023-05-20T01:16:40.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.855+0000 7f242c933700 1 -- start start 2023-05-20T01:16:40.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.856+0000 7f242c933700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f2428071880 0x7f2428071c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:40.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.856+0000 7f242c933700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f24280cd620 0x7f24280cb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:40.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.856+0000 7f242c933700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24280d0e10 con 0x7f24280cbde0 2023-05-20T01:16:40.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.856+0000 7f242c933700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24280cd200 con 0x7f2428071880 2023-05-20T01:16:40.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.856+0000 7f242c933700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2428072190 con 0x7f24280cd620 2023-05-20T01:16:40.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.856+0000 7f2426d9d700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f24280cbde0 0x7f242806d690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:41120/0 (socket says 172.21.15.16:41120) 2023-05-20T01:16:40.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.856+0000 7f2426d9d700 1 -- 172.21.15.16:0/3807924705 learned_addr learned my addr 172.21.15.16:0/3807924705 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:16:40.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.857+0000 7f2425d9b700 1 --2- 172.21.15.16:0/3807924705 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f24280cd620 0x7f24280cb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:40.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.857+0000 7f242659c700 1 --2- 172.21.15.16:0/3807924705 >> v2:172.21.15.132:3300/0 conn(0x7f2428071880 0x7f2428071c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:40.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.857+0000 7f242559a700 1 -- 172.21.15.16:0/3807924705 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2865920464 0 0) 0x7f24280d0e10 con 0x7f24280cbde0 2023-05-20T01:16:40.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.857+0000 7f242559a700 1 -- 172.21.15.16:0/3807924705 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f24100035b0 con 0x7f24280cbde0 2023-05-20T01:16:40.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.857+0000 7f242559a700 1 -- 172.21.15.16:0/3807924705 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3979978861 0 0) 0x7f24100035b0 con 0x7f24280cbde0 2023-05-20T01:16:40.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.857+0000 7f242659c700 1 -- 172.21.15.16:0/3807924705 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f24280cd620 msgr2=0x7f24280cb8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:40.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.857+0000 7f242659c700 1 --2- 172.21.15.16:0/3807924705 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f24280cd620 0x7f24280cb8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:40.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.857+0000 7f242659c700 1 -- 172.21.15.16:0/3807924705 >> v1:172.21.15.16:6789/0 conn(0x7f24280cbde0 legacy=0x7f242806d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:40.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.857+0000 7f242659c700 1 -- 172.21.15.16:0/3807924705 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24280cc580 con 0x7f2428071880 2023-05-20T01:16:40.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.857+0000 7f2425d9b700 1 --2- 172.21.15.16:0/3807924705 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f24280cd620 0x7f24280cb8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-05-20T01:16:40.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.857+0000 7f242659c700 1 --2- 172.21.15.16:0/3807924705 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2428071880 0x7f2428071c50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2418009820 tx=0x7f2418009b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ff39ad144a65443 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:40.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.857+0000 7f242559a700 1 -- 172.21.15.16:0/3807924705 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24200019f0 con 0x7f2428071880 2023-05-20T01:16:40.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.857+0000 7f242559a700 1 -- 172.21.15.16:0/3807924705 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2418005c30 con 0x7f2428071880 2023-05-20T01:16:40.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.857+0000 7f242559a700 1 -- 172.21.15.16:0/3807924705 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2418004030 con 0x7f2428071880 2023-05-20T01:16:40.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.858+0000 7f242c933700 1 -- 172.21.15.16:0/3807924705 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2428071880 msgr2=0x7f2428071c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:40.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.858+0000 7f242c933700 1 --2- 172.21.15.16:0/3807924705 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2428071880 0x7f2428071c50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2418009820 tx=0x7f2418009b30 comp rx=0 tx=0).stop 2023-05-20T01:16:40.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.858+0000 7f242c933700 1 -- 172.21.15.16:0/3807924705 shutdown_connections 2023-05-20T01:16:40.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.858+0000 7f242c933700 1 --2- 172.21.15.16:0/3807924705 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2428071880 0x7f2428071c50 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:40.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.858+0000 7f242c933700 1 --2- 172.21.15.16:0/3807924705 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f24280cd620 0x7f24280cb8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:40.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.858+0000 7f242c933700 1 -- 172.21.15.16:0/3807924705 >> 172.21.15.16:0/3807924705 conn(0x7f24280cffd0 msgr2=0x7f24280d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:40.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.858+0000 7f242c933700 1 -- 172.21.15.16:0/3807924705 shutdown_connections 2023-05-20T01:16:40.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.858+0000 7f242c933700 1 -- 172.21.15.16:0/3807924705 wait complete. 2023-05-20T01:16:40.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.860+0000 7f242c933700 1 Processor -- start 2023-05-20T01:16:40.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.860+0000 7f242c933700 1 -- start start 2023-05-20T01:16:40.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.861+0000 7f242c933700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2428071880 0x7f24280ae210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:40.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.861+0000 7f242c933700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24280cbde0 0x7f2428157890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:40.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.861+0000 7f242c933700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f24280cd620 0x7f24280b1620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:40.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.861+0000 7f242c933700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24280cdba0 con 0x7f24280cbde0 2023-05-20T01:16:40.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.861+0000 7f242c933700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24280c9d00 con 0x7f24280cd620 2023-05-20T01:16:40.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.861+0000 7f242c933700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24280ae910 con 0x7f2428071880 2023-05-20T01:16:40.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.861+0000 7f2425d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24280cbde0 0x7f2428157890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:40.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.861+0000 7f2425d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24280cbde0 0x7f2428157890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57828/0 (socket says 172.21.15.16:57828) 2023-05-20T01:16:40.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.861+0000 7f2425d9b700 1 -- 172.21.15.16:0/1130149143 learned_addr learned my addr 172.21.15.16:0/1130149143 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:40.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.861+0000 7f242659c700 1 --2- 172.21.15.16:0/1130149143 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2428071880 0x7f24280ae210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:40.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.861+0000 7f2426d9d700 1 --2- 172.21.15.16:0/1130149143 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f24280cd620 0x7f24280b1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:40.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.861+0000 7f2425d9b700 1 -- 172.21.15.16:0/1130149143 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2428071880 msgr2=0x7f24280ae210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:40.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.861+0000 7f2425d9b700 1 --2- 172.21.15.16:0/1130149143 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2428071880 0x7f24280ae210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:40.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.861+0000 7f2425d9b700 1 -- 172.21.15.16:0/1130149143 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f24280cd620 msgr2=0x7f24280b1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:40.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.861+0000 7f2425d9b700 1 --2- 172.21.15.16:0/1130149143 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f24280cd620 0x7f24280b1620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:40.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.861+0000 7f2425d9b700 1 -- 172.21.15.16:0/1130149143 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f241c0094a0 con 0x7f24280cbde0 2023-05-20T01:16:40.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.862+0000 7f2425d9b700 1 --2- 172.21.15.16:0/1130149143 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24280cbde0 0x7f2428157890 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f241c00e9b0 tx=0x7f241c00ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:40.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.862+0000 7f2416ffd700 1 -- 172.21.15.16:0/1130149143 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f241c00ee50 con 0x7f24280cbde0 2023-05-20T01:16:40.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.862+0000 7f2416ffd700 1 -- 172.21.15.16:0/1130149143 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f241c00c850 con 0x7f24280cbde0 2023-05-20T01:16:40.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.862+0000 7f242c933700 1 -- 172.21.15.16:0/1130149143 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2418009430 con 0x7f24280cbde0 2023-05-20T01:16:40.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.862+0000 7f2416ffd700 1 -- 172.21.15.16:0/1130149143 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f241c00cc00 con 0x7f24280cbde0 2023-05-20T01:16:40.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.862+0000 7f242c933700 1 -- 172.21.15.16:0/1130149143 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24280ccb40 con 0x7f24280cbde0 2023-05-20T01:16:40.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.863+0000 7f242c933700 1 -- 172.21.15.16:0/1130149143 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24280cc860 con 0x7f24280cbde0 2023-05-20T01:16:40.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.863+0000 7f2416ffd700 1 -- 172.21.15.16:0/1130149143 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f241c013470 con 0x7f24280cbde0 2023-05-20T01:16:40.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.864+0000 7f2416ffd700 1 --2- 172.21.15.16:0/1130149143 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f240806ace0 0x7f240806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:40.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.864+0000 7f242659c700 1 --2- 172.21.15.16:0/1130149143 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f240806ace0 0x7f240806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:40.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.864+0000 7f2416ffd700 1 -- 172.21.15.16:0/1130149143 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f241c089550 con 0x7f24280cbde0 2023-05-20T01:16:40.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.864+0000 7f242659c700 1 --2- 172.21.15.16:0/1130149143 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f240806ace0 0x7f240806d190 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2418000c00 tx=0x7f2418008070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:40.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:40.868+0000 7f2416ffd700 1 -- 172.21.15.16:0/1130149143 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f241c057c80 con 0x7f24280cbde0 2023-05-20T01:16:41.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.063+0000 7f242c933700 1 -- 172.21.15.16:0/1130149143 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f24280cce20 con 0x7f24280cbde0 2023-05-20T01:16:41.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.064+0000 7f2416ffd700 1 -- 172.21.15.16:0/1130149143 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7f241c057810 con 0x7f24280cbde0 2023-05-20T01:16:41.068 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 8 2023-05-20T01:16:41.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.065+0000 7f242c933700 1 -- 172.21.15.16:0/1130149143 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f240806ace0 msgr2=0x7f240806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:41.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.065+0000 7f242c933700 1 --2- 172.21.15.16:0/1130149143 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f240806ace0 0x7f240806d190 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2418000c00 tx=0x7f2418008070 comp rx=0 tx=0).stop 2023-05-20T01:16:41.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.065+0000 7f242c933700 1 -- 172.21.15.16:0/1130149143 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24280cbde0 msgr2=0x7f2428157890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:41.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.065+0000 7f242c933700 1 --2- 172.21.15.16:0/1130149143 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24280cbde0 0x7f2428157890 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f241c00e9b0 tx=0x7f241c00ecc0 comp rx=0 tx=0).stop 2023-05-20T01:16:41.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.066+0000 7f242c933700 1 -- 172.21.15.16:0/1130149143 shutdown_connections 2023-05-20T01:16:41.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.066+0000 7f242c933700 1 --2- 172.21.15.16:0/1130149143 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f24280cd620 0x7f24280b1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:41.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.066+0000 7f242c933700 1 --2- 172.21.15.16:0/1130149143 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f240806ace0 0x7f240806d190 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:41.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.066+0000 7f242c933700 1 --2- 172.21.15.16:0/1130149143 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2428071880 0x7f24280ae210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:41.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.066+0000 7f242c933700 1 --2- 172.21.15.16:0/1130149143 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24280cbde0 0x7f2428157890 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:41.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.066+0000 7f242c933700 1 -- 172.21.15.16:0/1130149143 >> 172.21.15.16:0/1130149143 conn(0x7f24280cffd0 msgr2=0x7f24280d03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:41.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.066+0000 7f242c933700 1 -- 172.21.15.16:0/1130149143 shutdown_connections 2023-05-20T01:16:41.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.066+0000 7f242c933700 1 -- 172.21.15.16:0/1130149143 wait complete. 2023-05-20T01:16:41.081 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-05-20T01:16:41.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.205+0000 7fda942e8700 1 Processor -- start 2023-05-20T01:16:41.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.205+0000 7fda942e8700 1 -- start start 2023-05-20T01:16:41.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.206+0000 7fda942e8700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fda8c15cec0 0x7fda8c071bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:41.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.206+0000 7fda942e8700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fda8c15d290 0x7fda8c072130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:41.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.206+0000 7fda942e8700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda8c0cd630 con 0x7fda8c0cf2c0 2023-05-20T01:16:41.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.206+0000 7fda942e8700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda8c0cca60 con 0x7fda8c15cec0 2023-05-20T01:16:41.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.206+0000 7fda942e8700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda8c0cfdd0 con 0x7fda8c15d290 2023-05-20T01:16:41.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.206+0000 7fda92084700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fda8c15cec0 0x7fda8c071bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:41.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.207+0000 7fda92084700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fda8c15cec0 0x7fda8c071bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57834/0 (socket says 172.21.15.16:57834) 2023-05-20T01:16:41.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.207+0000 7fda92084700 1 -- 172.21.15.16:0/613147708 learned_addr learned my addr 172.21.15.16:0/613147708 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:41.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.207+0000 7fda91883700 1 --2- 172.21.15.16:0/613147708 >> v2:172.21.15.132:3300/0 conn(0x7fda8c15d290 0x7fda8c072130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:41.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.207+0000 7fda92084700 1 -- 172.21.15.16:0/613147708 >> v2:172.21.15.132:3300/0 conn(0x7fda8c15d290 msgr2=0x7fda8c072130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:41.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.207+0000 7fda92084700 1 --2- 172.21.15.16:0/613147708 >> v2:172.21.15.132:3300/0 conn(0x7fda8c15d290 0x7fda8c072130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:41.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.207+0000 7fda92084700 1 -- 172.21.15.16:0/613147708 >> v1:172.21.15.132:6789/0 conn(0x7fda8c0cf2c0 legacy=0x7fda8c0cf6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:41.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.207+0000 7fda92084700 1 -- 172.21.15.16:0/613147708 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda8c0ccd00 con 0x7fda8c15cec0 2023-05-20T01:16:41.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.207+0000 7fda91883700 1 --2- 172.21.15.16:0/613147708 >> v2:172.21.15.132:3300/0 conn(0x7fda8c15d290 0x7fda8c072130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:16:41.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.207+0000 7fda92084700 1 --2- 172.21.15.16:0/613147708 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda8c15cec0 0x7fda8c071bf0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fda8c0cc970 tx=0x7fda80009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20e4e3a522a7e1bc server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:41.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.207+0000 7fda91082700 1 -- 172.21.15.16:0/613147708 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda80005e30 con 0x7fda8c15cec0 2023-05-20T01:16:41.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.207+0000 7fda91082700 1 -- 172.21.15.16:0/613147708 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda80006030 con 0x7fda8c15cec0 2023-05-20T01:16:41.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.207+0000 7fda91082700 1 -- 172.21.15.16:0/613147708 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda800063e0 con 0x7fda8c15cec0 2023-05-20T01:16:41.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.208+0000 7fda942e8700 1 -- 172.21.15.16:0/613147708 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda8c15cec0 msgr2=0x7fda8c071bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:41.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.208+0000 7fda942e8700 1 --2- 172.21.15.16:0/613147708 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda8c15cec0 0x7fda8c071bf0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fda8c0cc970 tx=0x7fda80009bd0 comp rx=0 tx=0).stop 2023-05-20T01:16:41.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.208+0000 7fda942e8700 1 -- 172.21.15.16:0/613147708 shutdown_connections 2023-05-20T01:16:41.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.208+0000 7fda942e8700 1 --2- 172.21.15.16:0/613147708 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda8c15cec0 0x7fda8c071bf0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:41.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.208+0000 7fda942e8700 1 --2- 172.21.15.16:0/613147708 >> v2:172.21.15.132:3300/0 conn(0x7fda8c15d290 0x7fda8c072130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:41.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.208+0000 7fda942e8700 1 -- 172.21.15.16:0/613147708 >> 172.21.15.16:0/613147708 conn(0x7fda8c0cffd0 msgr2=0x7fda8c0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:41.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.208+0000 7fda942e8700 1 -- 172.21.15.16:0/613147708 shutdown_connections 2023-05-20T01:16:41.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.208+0000 7fda942e8700 1 -- 172.21.15.16:0/613147708 wait complete. 2023-05-20T01:16:41.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.209+0000 7fda942e8700 1 Processor -- start 2023-05-20T01:16:41.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.209+0000 7fda942e8700 1 -- start start 2023-05-20T01:16:41.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.210+0000 7fda942e8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda8c0cf2c0 0x7fda8c0b36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:41.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.210+0000 7fda942e8700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fda8c15cec0 0x7fda8c1f2190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:41.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.210+0000 7fda942e8700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fda8c15d290 0x7fda8c0c9270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:41.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.210+0000 7fda942e8700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda8c1f3820 con 0x7fda8c0cf2c0 2023-05-20T01:16:41.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.210+0000 7fda942e8700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda8c1f2be0 con 0x7fda8c15cec0 2023-05-20T01:16:41.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.210+0000 7fda942e8700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda8c1f2e80 con 0x7fda8c15d290 2023-05-20T01:16:41.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.210+0000 7fda92084700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda8c0cf2c0 0x7fda8c0b36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:41.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.210+0000 7fda92084700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda8c0cf2c0 0x7fda8c0b36c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57838/0 (socket says 172.21.15.16:57838) 2023-05-20T01:16:41.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.210+0000 7fda92084700 1 -- 172.21.15.16:0/3604464433 learned_addr learned my addr 172.21.15.16:0/3604464433 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:41.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.210+0000 7fda92885700 1 --2- 172.21.15.16:0/3604464433 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fda8c15d290 0x7fda8c0c9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:41.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.210+0000 7fda91883700 1 --2- 172.21.15.16:0/3604464433 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fda8c15cec0 0x7fda8c1f2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:41.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.210+0000 7fda92084700 1 -- 172.21.15.16:0/3604464433 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fda8c15d290 msgr2=0x7fda8c0c9270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:41.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.210+0000 7fda92084700 1 --2- 172.21.15.16:0/3604464433 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fda8c15d290 0x7fda8c0c9270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:41.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.210+0000 7fda92084700 1 -- 172.21.15.16:0/3604464433 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fda8c15cec0 msgr2=0x7fda8c1f2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:41.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.210+0000 7fda92084700 1 --2- 172.21.15.16:0/3604464433 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fda8c15cec0 0x7fda8c1f2190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:41.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.210+0000 7fda92084700 1 -- 172.21.15.16:0/3604464433 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda80009500 con 0x7fda8c0cf2c0 2023-05-20T01:16:41.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.210+0000 7fda91883700 1 --2- 172.21.15.16:0/3604464433 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fda8c15cec0 0x7fda8c1f2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:16:41.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.211+0000 7fda92084700 1 --2- 172.21.15.16:0/3604464433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda8c0cf2c0 0x7fda8c0b36c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fda80013040 tx=0x7fda80004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:41.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.211+0000 7fda7effd700 1 -- 172.21.15.16:0/3604464433 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda80004250 con 0x7fda8c0cf2c0 2023-05-20T01:16:41.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.211+0000 7fda7effd700 1 -- 172.21.15.16:0/3604464433 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda800043f0 con 0x7fda8c0cf2c0 2023-05-20T01:16:41.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.211+0000 7fda7effd700 1 -- 172.21.15.16:0/3604464433 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda800038e0 con 0x7fda8c0cf2c0 2023-05-20T01:16:41.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.211+0000 7fda942e8700 1 -- 172.21.15.16:0/3604464433 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda8c0cd2c0 con 0x7fda8c0cf2c0 2023-05-20T01:16:41.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.211+0000 7fda942e8700 1 -- 172.21.15.16:0/3604464433 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda8c0ccfe0 con 0x7fda8c0cf2c0 2023-05-20T01:16:41.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.212+0000 7fda942e8700 1 -- 172.21.15.16:0/3604464433 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda8c160890 con 0x7fda8c0cf2c0 2023-05-20T01:16:41.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.212+0000 7fda7effd700 1 -- 172.21.15.16:0/3604464433 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fda80022070 con 0x7fda8c0cf2c0 2023-05-20T01:16:41.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.213+0000 7fda7effd700 1 --2- 172.21.15.16:0/3604464433 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fda7006ace0 0x7fda7006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:41.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.213+0000 7fda7effd700 1 -- 172.21.15.16:0/3604464433 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fda8008aff0 con 0x7fda8c0cf2c0 2023-05-20T01:16:41.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.213+0000 7fda91883700 1 --2- 172.21.15.16:0/3604464433 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fda7006ace0 0x7fda7006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:41.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.213+0000 7fda91883700 1 --2- 172.21.15.16:0/3604464433 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fda7006ace0 0x7fda7006d190 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fda8c0b3520 tx=0x7fda88008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:41.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.217+0000 7fda7effd700 1 -- 172.21.15.16:0/3604464433 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fda800bd2d0 con 0x7fda8c0cf2c0 2023-05-20T01:16:41.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.416+0000 7fda942e8700 1 -- 172.21.15.16:0/3604464433 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fda8c1f3960 con 0x7fda8c0cf2c0 2023-05-20T01:16:41.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.417+0000 7fda7effd700 1 -- 172.21.15.16:0/3604464433 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7fda80055e40 con 0x7fda8c0cf2c0 2023-05-20T01:16:41.421 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-05-20T01:16:41.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.418+0000 7fda942e8700 1 -- 172.21.15.16:0/3604464433 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fda7006ace0 msgr2=0x7fda7006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:41.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.418+0000 7fda942e8700 1 --2- 172.21.15.16:0/3604464433 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fda7006ace0 0x7fda7006d190 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fda8c0b3520 tx=0x7fda88008040 comp rx=0 tx=0).stop 2023-05-20T01:16:41.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.418+0000 7fda942e8700 1 -- 172.21.15.16:0/3604464433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda8c0cf2c0 msgr2=0x7fda8c0b36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:41.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.418+0000 7fda942e8700 1 --2- 172.21.15.16:0/3604464433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda8c0cf2c0 0x7fda8c0b36c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fda80013040 tx=0x7fda80004750 comp rx=0 tx=0).stop 2023-05-20T01:16:41.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.418+0000 7fda942e8700 1 -- 172.21.15.16:0/3604464433 shutdown_connections 2023-05-20T01:16:41.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.418+0000 7fda942e8700 1 --2- 172.21.15.16:0/3604464433 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fda7006ace0 0x7fda7006d190 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:41.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.418+0000 7fda942e8700 1 --2- 172.21.15.16:0/3604464433 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fda8c15d290 0x7fda8c0c9270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:41.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.418+0000 7fda942e8700 1 --2- 172.21.15.16:0/3604464433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda8c0cf2c0 0x7fda8c0b36c0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:41.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.418+0000 7fda942e8700 1 --2- 172.21.15.16:0/3604464433 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fda8c15cec0 0x7fda8c1f2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:41.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.418+0000 7fda942e8700 1 -- 172.21.15.16:0/3604464433 >> 172.21.15.16:0/3604464433 conn(0x7fda8c0cffd0 msgr2=0x7fda8c0cf6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:41.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.419+0000 7fda942e8700 1 -- 172.21.15.16:0/3604464433 shutdown_connections 2023-05-20T01:16:41.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.419+0000 7fda942e8700 1 -- 172.21.15.16:0/3604464433 wait complete. 2023-05-20T01:16:41.434 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-05-20T01:16:41.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.557+0000 7fa20ae28700 1 Processor -- start 2023-05-20T01:16:41.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.558+0000 7fa20ae28700 1 -- start start 2023-05-20T01:16:41.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.558+0000 7fa20ae28700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fa20415cec0 0x7fa204071bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:41.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.558+0000 7fa20ae28700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa20415d290 0x7fa204072130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:41.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.558+0000 7fa20ae28700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa2040cf2c0 0x7fa2040cf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:41.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.558+0000 7fa20ae28700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2040cd630 con 0x7fa2040cf2c0 2023-05-20T01:16:41.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.558+0000 7fa20ae28700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2040cca60 con 0x7fa20415cec0 2023-05-20T01:16:41.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.558+0000 7fa20ae28700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2040ccd00 con 0x7fa20415d290 2023-05-20T01:16:41.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.558+0000 7fa2093c5700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa2040cf2c0 0x7fa2040cf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:41.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.559+0000 7fa2093c5700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa2040cf2c0 0x7fa2040cf720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57848/0 (socket says 172.21.15.16:57848) 2023-05-20T01:16:41.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.559+0000 7fa2093c5700 1 -- 172.21.15.16:0/2558823666 learned_addr learned my addr 172.21.15.16:0/2558823666 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:41.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.559+0000 7fa208bc4700 1 --2- 172.21.15.16:0/2558823666 >> v2:172.21.15.132:3300/0 conn(0x7fa20415cec0 0x7fa204071bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:41.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.559+0000 7fa203fff700 1 --2- 172.21.15.16:0/2558823666 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa20415d290 0x7fa204072130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:41.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.559+0000 7fa2093c5700 1 -- 172.21.15.16:0/2558823666 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa20415d290 msgr2=0x7fa204072130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:41.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.559+0000 7fa2093c5700 1 --2- 172.21.15.16:0/2558823666 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa20415d290 0x7fa204072130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:41.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.559+0000 7fa2093c5700 1 -- 172.21.15.16:0/2558823666 >> v2:172.21.15.132:3300/0 conn(0x7fa20415cec0 msgr2=0x7fa204071bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:41.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.559+0000 7fa2093c5700 1 --2- 172.21.15.16:0/2558823666 >> v2:172.21.15.132:3300/0 conn(0x7fa20415cec0 0x7fa204071bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:41.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.559+0000 7fa2093c5700 1 -- 172.21.15.16:0/2558823666 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2040ccfe0 con 0x7fa2040cf2c0 2023-05-20T01:16:41.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.559+0000 7fa2093c5700 1 --2- 172.21.15.16:0/2558823666 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2040cf2c0 0x7fa2040cf720 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa204072770 tx=0x7fa1fc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42ef065b2959b28f server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:41.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.559+0000 7fa2037fe700 1 -- 172.21.15.16:0/2558823666 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1fc00e070 con 0x7fa2040cf2c0 2023-05-20T01:16:41.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.559+0000 7fa2037fe700 1 -- 172.21.15.16:0/2558823666 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1fc00b9c0 con 0x7fa2040cf2c0 2023-05-20T01:16:41.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.559+0000 7fa2037fe700 1 -- 172.21.15.16:0/2558823666 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1fc00bd70 con 0x7fa2040cf2c0 2023-05-20T01:16:41.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.560+0000 7fa20ae28700 1 -- 172.21.15.16:0/2558823666 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2040cf2c0 msgr2=0x7fa2040cf720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:41.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.560+0000 7fa20ae28700 1 --2- 172.21.15.16:0/2558823666 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2040cf2c0 0x7fa2040cf720 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa204072770 tx=0x7fa1fc00b330 comp rx=0 tx=0).stop 2023-05-20T01:16:41.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.560+0000 7fa20ae28700 1 -- 172.21.15.16:0/2558823666 shutdown_connections 2023-05-20T01:16:41.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.560+0000 7fa20ae28700 1 --2- 172.21.15.16:0/2558823666 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2040cf2c0 0x7fa2040cf720 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:41.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.560+0000 7fa20ae28700 1 --2- 172.21.15.16:0/2558823666 >> v2:172.21.15.132:3300/0 conn(0x7fa20415cec0 0x7fa204071bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:41.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.560+0000 7fa20ae28700 1 --2- 172.21.15.16:0/2558823666 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa20415d290 0x7fa204072130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:41.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.560+0000 7fa20ae28700 1 -- 172.21.15.16:0/2558823666 >> 172.21.15.16:0/2558823666 conn(0x7fa2040cffd0 msgr2=0x7fa2040d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:41.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.560+0000 7fa20ae28700 1 -- 172.21.15.16:0/2558823666 shutdown_connections 2023-05-20T01:16:41.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.560+0000 7fa20ae28700 1 -- 172.21.15.16:0/2558823666 wait complete. 2023-05-20T01:16:41.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.561+0000 7fa20ae28700 1 Processor -- start 2023-05-20T01:16:41.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.561+0000 7fa20ae28700 1 -- start start 2023-05-20T01:16:41.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.562+0000 7fa20ae28700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa2040cf2c0 0x7fa2040b56c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:41.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.562+0000 7fa20ae28700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa20415cec0 0x7fa2040ac8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:41.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.562+0000 7fa20ae28700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa20415d290 0x7fa20415b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:41.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.562+0000 7fa20ae28700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2041f34e0 con 0x7fa20415cec0 2023-05-20T01:16:41.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.562+0000 7fa20ae28700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2041f37c0 con 0x7fa2040cf2c0 2023-05-20T01:16:41.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.562+0000 7fa20ae28700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2041f3aa0 con 0x7fa20415d290 2023-05-20T01:16:41.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.562+0000 7fa203fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa20415cec0 0x7fa2040ac8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:41.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.562+0000 7fa203fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa20415cec0 0x7fa2040ac8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57852/0 (socket says 172.21.15.16:57852) 2023-05-20T01:16:41.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.562+0000 7fa203fff700 1 -- 172.21.15.16:0/2741364992 learned_addr learned my addr 172.21.15.16:0/2741364992 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:41.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.562+0000 7fa208bc4700 1 --2- 172.21.15.16:0/2741364992 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa2040cf2c0 0x7fa2040b56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:41.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.562+0000 7fa2093c5700 1 --2- 172.21.15.16:0/2741364992 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa20415d290 0x7fa20415b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:41.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.562+0000 7fa203fff700 1 -- 172.21.15.16:0/2741364992 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa20415d290 msgr2=0x7fa20415b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:41.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.562+0000 7fa203fff700 1 --2- 172.21.15.16:0/2741364992 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa20415d290 0x7fa20415b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:41.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.562+0000 7fa203fff700 1 -- 172.21.15.16:0/2741364992 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa2040cf2c0 msgr2=0x7fa2040b56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:41.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.562+0000 7fa203fff700 1 --2- 172.21.15.16:0/2741364992 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa2040cf2c0 0x7fa2040b56c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:41.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.562+0000 7fa203fff700 1 -- 172.21.15.16:0/2741364992 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1fc009b30 con 0x7fa20415cec0 2023-05-20T01:16:41.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.562+0000 7fa203fff700 1 --2- 172.21.15.16:0/2741364992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa20415cec0 0x7fa2040ac8a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa1f000e810 tx=0x7fa1f000eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:41.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.563+0000 7fa2017fa700 1 -- 172.21.15.16:0/2741364992 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f000c070 con 0x7fa20415cec0 2023-05-20T01:16:41.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.563+0000 7fa2017fa700 1 -- 172.21.15.16:0/2741364992 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1f0013610 con 0x7fa20415cec0 2023-05-20T01:16:41.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.563+0000 7fa20ae28700 1 -- 172.21.15.16:0/2741364992 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2040cfc60 con 0x7fa20415cec0 2023-05-20T01:16:41.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.563+0000 7fa2017fa700 1 -- 172.21.15.16:0/2741364992 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f00139c0 con 0x7fa20415cec0 2023-05-20T01:16:41.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.563+0000 7fa20ae28700 1 -- 172.21.15.16:0/2741364992 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2040cd2c0 con 0x7fa20415cec0 2023-05-20T01:16:41.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.564+0000 7fa20ae28700 1 -- 172.21.15.16:0/2741364992 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2040cfdf0 con 0x7fa20415cec0 2023-05-20T01:16:41.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.564+0000 7fa2017fa700 1 -- 172.21.15.16:0/2741364992 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa1f0013b20 con 0x7fa20415cec0 2023-05-20T01:16:41.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.565+0000 7fa2017fa700 1 --2- 172.21.15.16:0/2741364992 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa1e806ace0 0x7fa1e806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:41.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.565+0000 7fa2017fa700 1 -- 172.21.15.16:0/2741364992 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fa1f00894a0 con 0x7fa20415cec0 2023-05-20T01:16:41.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.565+0000 7fa208bc4700 1 --2- 172.21.15.16:0/2741364992 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa1e806ace0 0x7fa1e806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:41.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.565+0000 7fa208bc4700 1 --2- 172.21.15.16:0/2741364992 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa1e806ace0 0x7fa1e806d190 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa1f8006fd0 tx=0x7fa1f8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:41.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.569+0000 7fa2017fa700 1 -- 172.21.15.16:0/2741364992 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fa1f0057d00 con 0x7fa20415cec0 2023-05-20T01:16:41.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.766+0000 7fa20ae28700 1 -- 172.21.15.16:0/2741364992 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa2040747b0 con 0x7fa20415cec0 2023-05-20T01:16:41.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.767+0000 7fa2017fa700 1 -- 172.21.15.16:0/2741364992 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa1f0057890 con 0x7fa20415cec0 2023-05-20T01:16:41.771 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-05-20T01:16:41.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.769+0000 7fa20ae28700 1 -- 172.21.15.16:0/2741364992 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa1e806ace0 msgr2=0x7fa1e806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:41.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.769+0000 7fa20ae28700 1 --2- 172.21.15.16:0/2741364992 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa1e806ace0 0x7fa1e806d190 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa1f8006fd0 tx=0x7fa1f8008040 comp rx=0 tx=0).stop 2023-05-20T01:16:41.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.769+0000 7fa20ae28700 1 -- 172.21.15.16:0/2741364992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa20415cec0 msgr2=0x7fa2040ac8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:41.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.769+0000 7fa20ae28700 1 --2- 172.21.15.16:0/2741364992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa20415cec0 0x7fa2040ac8a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa1f000e810 tx=0x7fa1f000eb20 comp rx=0 tx=0).stop 2023-05-20T01:16:41.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.769+0000 7fa20ae28700 1 -- 172.21.15.16:0/2741364992 shutdown_connections 2023-05-20T01:16:41.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.769+0000 7fa20ae28700 1 --2- 172.21.15.16:0/2741364992 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa1e806ace0 0x7fa1e806d190 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:41.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.769+0000 7fa20ae28700 1 --2- 172.21.15.16:0/2741364992 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa20415d290 0x7fa20415b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:41.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.769+0000 7fa20ae28700 1 --2- 172.21.15.16:0/2741364992 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa2040cf2c0 0x7fa2040b56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:41.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.769+0000 7fa20ae28700 1 --2- 172.21.15.16:0/2741364992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa20415cec0 0x7fa2040ac8a0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:41.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.769+0000 7fa20ae28700 1 -- 172.21.15.16:0/2741364992 >> 172.21.15.16:0/2741364992 conn(0x7fa2040cffd0 msgr2=0x7fa20415fdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:41.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.769+0000 7fa20ae28700 1 -- 172.21.15.16:0/2741364992 shutdown_connections 2023-05-20T01:16:41.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.769+0000 7fa20ae28700 1 -- 172.21.15.16:0/2741364992 wait complete. 2023-05-20T01:16:41.785 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-05-20T01:16:41.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.908+0000 7fbebd74b700 1 Processor -- start 2023-05-20T01:16:41.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.908+0000 7fbebd74b700 1 -- start start 2023-05-20T01:16:41.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.909+0000 7fbebd74b700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbeb80717c0 0x7fbeb80ca870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:41.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.909+0000 7fbebd74b700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbeb80ccd30 con 0x7fbeb80cd1c0 2023-05-20T01:16:41.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.909+0000 7fbebd74b700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbeb80ce990 con 0x7fbeb8159b50 2023-05-20T01:16:41.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.909+0000 7fbebd74b700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb80cc6e0 con 0x7fbeb80717c0 2023-05-20T01:16:41.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.909+0000 7fbeb67fc700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fbeb80cd1c0 0x7fbeb80710b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:41156/0 (socket says 172.21.15.16:41156) 2023-05-20T01:16:41.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.909+0000 7fbeb67fc700 1 -- 172.21.15.16:0/1413870387 learned_addr learned my addr 172.21.15.16:0/1413870387 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:16:41.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.910+0000 7fbeb77fe700 1 --2- 172.21.15.16:0/1413870387 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbeb80717c0 0x7fbeb80ca870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:41.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.910+0000 7fbeb5ffb700 1 -- 172.21.15.16:0/1413870387 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1276089631 0 0) 0x7fbeb80ccd30 con 0x7fbeb80cd1c0 2023-05-20T01:16:41.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.910+0000 7fbeb5ffb700 1 -- 172.21.15.16:0/1413870387 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbea00035b0 con 0x7fbeb80cd1c0 2023-05-20T01:16:41.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.910+0000 7fbeb5ffb700 1 -- 172.21.15.16:0/1413870387 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4170962161 0 0) 0x7fbeb80ce990 con 0x7fbeb8159b50 2023-05-20T01:16:41.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.910+0000 7fbeb5ffb700 1 -- 172.21.15.16:0/1413870387 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbeb80ccd30 con 0x7fbeb8159b50 2023-05-20T01:16:41.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.910+0000 7fbeb5ffb700 1 -- 172.21.15.16:0/1413870387 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1616533456 0 0) 0x7fbea00035b0 con 0x7fbeb80cd1c0 2023-05-20T01:16:41.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.910+0000 7fbeb5ffb700 1 -- 172.21.15.16:0/1413870387 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbeb80717c0 msgr2=0x7fbeb80ca870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:41.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.910+0000 7fbeb5ffb700 1 --2- 172.21.15.16:0/1413870387 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbeb80717c0 0x7fbeb80ca870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:41.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.910+0000 7fbeb5ffb700 1 -- 172.21.15.16:0/1413870387 >> v1:172.21.15.132:6789/0 conn(0x7fbeb8159b50 legacy=0x7fbeb806e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:41.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.910+0000 7fbeb5ffb700 1 -- 172.21.15.16:0/1413870387 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbeb80cc9c0 con 0x7fbeb80cd1c0 2023-05-20T01:16:41.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.910+0000 7fbeb5ffb700 1 -- 172.21.15.16:0/1413870387 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7fbea8001670 con 0x7fbeb80cd1c0 2023-05-20T01:16:41.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.910+0000 7fbeb77fe700 1 --2- 172.21.15.16:0/1413870387 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbeb80717c0 0x7fbeb80ca870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-05-20T01:16:41.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.910+0000 7fbeb5ffb700 1 -- 172.21.15.16:0/1413870387 >> v1:172.21.15.16:6789/0 conn(0x7fbeb80cd1c0 legacy=0x7fbeb80710b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:41.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.910+0000 7fbeb5ffb700 1 --2- 172.21.15.16:0/1413870387 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbea0003bc0 0x7fbea0003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:41.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.910+0000 7fbeb5ffb700 1 --2- 172.21.15.16:0/1413870387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbea0004850 0x7fbea0004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:41.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.910+0000 7fbeb5ffb700 1 --2- 172.21.15.16:0/1413870387 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbeb80717c0 0x7fbea0005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:41.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.910+0000 7fbeb5ffb700 1 -- 172.21.15.16:0/1413870387 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb80cc6e0 con 0x7fbea0004850 2023-05-20T01:16:41.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.910+0000 7fbeb5ffb700 1 -- 172.21.15.16:0/1413870387 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbea000ba70 con 0x7fbea0003bc0 2023-05-20T01:16:41.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.910+0000 7fbeb5ffb700 1 -- 172.21.15.16:0/1413870387 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbea000bc30 con 0x7fbeb80717c0 2023-05-20T01:16:41.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.910+0000 7fbeb67fc700 1 --2- 172.21.15.16:0/1413870387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbea0004850 0x7fbea0004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:41.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.910+0000 7fbeb6ffd700 1 --2- 172.21.15.16:0/1413870387 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbea0003bc0 0x7fbea0003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:41.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.910+0000 7fbeb77fe700 1 --2- 172.21.15.16:0/1413870387 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbeb80717c0 0x7fbea0005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:41.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.910+0000 7fbeb67fc700 1 -- 172.21.15.16:0/1413870387 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbeb80717c0 msgr2=0x7fbea0005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:41.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.910+0000 7fbeb67fc700 1 --2- 172.21.15.16:0/1413870387 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbeb80717c0 0x7fbea0005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:41.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.910+0000 7fbeb67fc700 1 -- 172.21.15.16:0/1413870387 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbea0003bc0 msgr2=0x7fbea0003fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:41.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.910+0000 7fbeb67fc700 1 --2- 172.21.15.16:0/1413870387 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbea0003bc0 0x7fbea0003fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:41.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.910+0000 7fbeb67fc700 1 -- 172.21.15.16:0/1413870387 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbeb0009a70 con 0x7fbea0004850 2023-05-20T01:16:41.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.911+0000 7fbeb67fc700 1 --2- 172.21.15.16:0/1413870387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbea0004850 0x7fbea0004cc0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fbea800c210 tx=0x7fbea0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c952d3bf144f5135 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:41.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.911+0000 7fbeb5ffb700 1 -- 172.21.15.16:0/1413870387 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea800cc90 con 0x7fbea0004850 2023-05-20T01:16:41.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.911+0000 7fbeb5ffb700 1 -- 172.21.15.16:0/1413870387 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbea800ce10 con 0x7fbea0004850 2023-05-20T01:16:41.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.911+0000 7fbeb5ffb700 1 -- 172.21.15.16:0/1413870387 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea8005ba0 con 0x7fbea0004850 2023-05-20T01:16:41.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.911+0000 7fbebd74b700 1 -- 172.21.15.16:0/1413870387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbea0004850 msgr2=0x7fbea0004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:41.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.911+0000 7fbebd74b700 1 --2- 172.21.15.16:0/1413870387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbea0004850 0x7fbea0004cc0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fbea800c210 tx=0x7fbea0005880 comp rx=0 tx=0).stop 2023-05-20T01:16:41.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.911+0000 7fbeb77fe700 1 -- 172.21.15.16:0/1413870387 reap_dead start 2023-05-20T01:16:41.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.911+0000 7fbebd74b700 1 -- 172.21.15.16:0/1413870387 shutdown_connections 2023-05-20T01:16:41.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.911+0000 7fbebd74b700 1 -- 172.21.15.16:0/1413870387 >> 172.21.15.16:0/1413870387 conn(0x7fbeb80cffd0 msgr2=0x7fbeb80d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:41.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.911+0000 7fbebd74b700 1 -- 172.21.15.16:0/1413870387 shutdown_connections 2023-05-20T01:16:41.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.911+0000 7fbebd74b700 1 -- 172.21.15.16:0/1413870387 wait complete. 2023-05-20T01:16:41.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.913+0000 7fbebd74b700 1 Processor -- start 2023-05-20T01:16:41.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.913+0000 7fbebd74b700 1 -- start start 2023-05-20T01:16:41.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.913+0000 7fbebd74b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbea0004850 0x7fbeb80a8490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:41.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.914+0000 7fbebd74b700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbeb814b4d0 0x7fbeb814a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:41.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.914+0000 7fbebd74b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbeb80a89d0 0x7fbeb814ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:41.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.914+0000 7fbebd74b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb80a8ee0 con 0x7fbea0004850 2023-05-20T01:16:41.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.914+0000 7fbebd74b700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb8148180 con 0x7fbeb80a89d0 2023-05-20T01:16:41.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.914+0000 7fbebd74b700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb8148460 con 0x7fbeb814b4d0 2023-05-20T01:16:41.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.914+0000 7fbeb6ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbea0004850 0x7fbeb80a8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:41.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.914+0000 7fbeb6ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbea0004850 0x7fbeb80a8490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57870/0 (socket says 172.21.15.16:57870) 2023-05-20T01:16:41.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.914+0000 7fbeb6ffd700 1 -- 172.21.15.16:0/665806759 learned_addr learned my addr 172.21.15.16:0/665806759 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:41.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.914+0000 7fbeb67fc700 1 --2- 172.21.15.16:0/665806759 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbeb814b4d0 0x7fbeb814a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:41.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.914+0000 7fbeb77fe700 1 --2- 172.21.15.16:0/665806759 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbeb80a89d0 0x7fbeb814ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:41.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.914+0000 7fbeb6ffd700 1 -- 172.21.15.16:0/665806759 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbeb814b4d0 msgr2=0x7fbeb814a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:41.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.914+0000 7fbeb6ffd700 1 --2- 172.21.15.16:0/665806759 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbeb814b4d0 0x7fbeb814a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:41.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.914+0000 7fbeb6ffd700 1 -- 172.21.15.16:0/665806759 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbeb80a89d0 msgr2=0x7fbeb814ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:41.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.914+0000 7fbeb6ffd700 1 --2- 172.21.15.16:0/665806759 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbeb80a89d0 0x7fbeb814ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:41.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.914+0000 7fbeb6ffd700 1 -- 172.21.15.16:0/665806759 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbeb80cec70 con 0x7fbea0004850 2023-05-20T01:16:41.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.914+0000 7fbeb77fe700 1 --2- 172.21.15.16:0/665806759 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbeb80a89d0 0x7fbeb814ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:16:41.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.914+0000 7fbeb6ffd700 1 --2- 172.21.15.16:0/665806759 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbea0004850 0x7fbeb80a8490 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fbeac00f7a0 tx=0x7fbeac00fb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:41.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.914+0000 7fbe9f7fe700 1 -- 172.21.15.16:0/665806759 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeac005320 con 0x7fbea0004850 2023-05-20T01:16:41.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.914+0000 7fbe9f7fe700 1 -- 172.21.15.16:0/665806759 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbeac0054a0 con 0x7fbea0004850 2023-05-20T01:16:41.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.914+0000 7fbebd74b700 1 -- 172.21.15.16:0/665806759 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbeb8071e80 con 0x7fbea0004850 2023-05-20T01:16:41.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.914+0000 7fbe9f7fe700 1 -- 172.21.15.16:0/665806759 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeac006af0 con 0x7fbea0004850 2023-05-20T01:16:41.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.914+0000 7fbebd74b700 1 -- 172.21.15.16:0/665806759 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbeb80ce5b0 con 0x7fbea0004850 2023-05-20T01:16:41.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.915+0000 7fbebd74b700 1 -- 172.21.15.16:0/665806759 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbeb81482c0 con 0x7fbea0004850 2023-05-20T01:16:41.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.916+0000 7fbe9f7fe700 1 -- 172.21.15.16:0/665806759 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fbeac006c50 con 0x7fbea0004850 2023-05-20T01:16:41.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.916+0000 7fbe9f7fe700 1 --2- 172.21.15.16:0/665806759 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fbea406acd0 0x7fbea406d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:41.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.916+0000 7fbe9f7fe700 1 -- 172.21.15.16:0/665806759 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fbeac089110 con 0x7fbea0004850 2023-05-20T01:16:41.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.916+0000 7fbeb67fc700 1 --2- 172.21.15.16:0/665806759 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fbea406acd0 0x7fbea406d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:41.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.917+0000 7fbeb67fc700 1 --2- 172.21.15.16:0/665806759 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fbea406acd0 0x7fbea406d180 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fbea8015040 tx=0x7fbea8007f90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:41.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:41.920+0000 7fbe9f7fe700 1 -- 172.21.15.16:0/665806759 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fbeb81482c0 con 0x7fbea0004850 2023-05-20T01:16:42.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.119+0000 7fbebd74b700 1 -- 172.21.15.16:0/665806759 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fbeb81482c0 con 0x7fbea0004850 2023-05-20T01:16:42.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.119+0000 7fbe9f7fe700 1 -- 172.21.15.16:0/665806759 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7fbeb81482c0 con 0x7fbea0004850 2023-05-20T01:16:42.123 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2023-05-20T01:16:42.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.121+0000 7fbebd74b700 1 -- 172.21.15.16:0/665806759 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fbea406acd0 msgr2=0x7fbea406d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:42.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.122+0000 7fbebd74b700 1 --2- 172.21.15.16:0/665806759 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fbea406acd0 0x7fbea406d180 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fbea8015040 tx=0x7fbea8007f90 comp rx=0 tx=0).stop 2023-05-20T01:16:42.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.122+0000 7fbebd74b700 1 -- 172.21.15.16:0/665806759 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbea0004850 msgr2=0x7fbeb80a8490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:42.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.122+0000 7fbebd74b700 1 --2- 172.21.15.16:0/665806759 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbea0004850 0x7fbeb80a8490 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fbeac00f7a0 tx=0x7fbeac00fb60 comp rx=0 tx=0).stop 2023-05-20T01:16:42.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.122+0000 7fbebd74b700 1 -- 172.21.15.16:0/665806759 shutdown_connections 2023-05-20T01:16:42.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.122+0000 7fbebd74b700 1 --2- 172.21.15.16:0/665806759 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbeb80a89d0 0x7fbeb814ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:42.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.122+0000 7fbebd74b700 1 --2- 172.21.15.16:0/665806759 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbea0004850 0x7fbeb80a8490 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:42.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.122+0000 7fbebd74b700 1 --2- 172.21.15.16:0/665806759 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fbea406acd0 0x7fbea406d180 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:42.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.122+0000 7fbebd74b700 1 --2- 172.21.15.16:0/665806759 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbeb814b4d0 0x7fbeb814a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:42.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.122+0000 7fbebd74b700 1 -- 172.21.15.16:0/665806759 >> 172.21.15.16:0/665806759 conn(0x7fbeb80cffd0 msgr2=0x7fbeb80c6880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:42.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.122+0000 7fbebd74b700 1 -- 172.21.15.16:0/665806759 shutdown_connections 2023-05-20T01:16:42.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.122+0000 7fbebd74b700 1 -- 172.21.15.16:0/665806759 wait complete. 2023-05-20T01:16:42.137 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-05-20T01:16:42.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.261+0000 7f8463adf700 1 Processor -- start 2023-05-20T01:16:42.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.261+0000 7f8463adf700 1 -- start start 2023-05-20T01:16:42.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.262+0000 7f8463adf700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f845c159b50 0x7f845c06e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:42.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.262+0000 7f8463adf700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f845c0717c0 0x7f845c0ca870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:42.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.262+0000 7f8463adf700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f845c0ccd30 con 0x7f845c0cd1c0 2023-05-20T01:16:42.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.262+0000 7f8463adf700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f845c0ce990 con 0x7f845c159b50 2023-05-20T01:16:42.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.262+0000 7f8463adf700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f845c0cc6e0 con 0x7f845c0717c0 2023-05-20T01:16:42.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.262+0000 7f846187b700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f845c159b50 0x7f845c06e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:42.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.262+0000 7f846207c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f845c0717c0 0x7f845c0ca870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:42.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.262+0000 7f846187b700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f845c159b50 0x7f845c06e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.16:54900/0 (socket says 172.21.15.16:54900) 2023-05-20T01:16:42.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.262+0000 7f846187b700 1 -- 172.21.15.16:0/1359519634 learned_addr learned my addr 172.21.15.16:0/1359519634 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:42.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.262+0000 7f846107a700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f845c0cd1c0 0x7f845c0710b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.16:50890/0 (socket says 172.21.15.16:50890) 2023-05-20T01:16:42.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.263+0000 7f846187b700 1 -- 172.21.15.16:0/1359519634 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f845c0717c0 msgr2=0x7f845c0ca870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:42.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.263+0000 7f846187b700 1 --2- 172.21.15.16:0/1359519634 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f845c0717c0 0x7f845c0ca870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:42.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.263+0000 7f846187b700 1 -- 172.21.15.16:0/1359519634 >> v1:172.21.15.132:6789/0 conn(0x7f845c0cd1c0 legacy=0x7f845c0710b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:42.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.263+0000 7f846187b700 1 -- 172.21.15.16:0/1359519634 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f845c0cc9c0 con 0x7f845c159b50 2023-05-20T01:16:42.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.263+0000 7f8460879700 1 -- 172.21.15.16:0/1359519634 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2547509187 0 0) 0x7f845c0ccd30 con 0x7f845c0cd1c0 2023-05-20T01:16:42.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.263+0000 7f846187b700 1 --2- 172.21.15.16:0/1359519634 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f845c159b50 0x7f845c06e0a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f844c006fd0 tx=0x7f844c00be70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b62c4e89d50219f2 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:42.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.263+0000 7f8460879700 1 -- 172.21.15.16:0/1359519634 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f844c005e30 con 0x7f845c159b50 2023-05-20T01:16:42.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.263+0000 7f8460879700 1 -- 172.21.15.16:0/1359519634 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f844c004030 con 0x7f845c159b50 2023-05-20T01:16:42.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.263+0000 7f8460879700 1 -- 172.21.15.16:0/1359519634 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f844c0043e0 con 0x7f845c159b50 2023-05-20T01:16:42.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.264+0000 7f8463adf700 1 -- 172.21.15.16:0/1359519634 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f845c159b50 msgr2=0x7f845c06e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:42.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.264+0000 7f8463adf700 1 --2- 172.21.15.16:0/1359519634 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f845c159b50 0x7f845c06e0a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f844c006fd0 tx=0x7f844c00be70 comp rx=0 tx=0).stop 2023-05-20T01:16:42.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.264+0000 7f8463adf700 1 -- 172.21.15.16:0/1359519634 shutdown_connections 2023-05-20T01:16:42.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.264+0000 7f8463adf700 1 --2- 172.21.15.16:0/1359519634 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f845c0717c0 0x7f845c0ca870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:42.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.264+0000 7f8463adf700 1 --2- 172.21.15.16:0/1359519634 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f845c159b50 0x7f845c06e0a0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:42.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.264+0000 7f8463adf700 1 -- 172.21.15.16:0/1359519634 >> 172.21.15.16:0/1359519634 conn(0x7f845c0cffd0 msgr2=0x7f845c0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:42.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.264+0000 7f8463adf700 1 -- 172.21.15.16:0/1359519634 shutdown_connections 2023-05-20T01:16:42.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.264+0000 7f8463adf700 1 -- 172.21.15.16:0/1359519634 wait complete. 2023-05-20T01:16:42.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.265+0000 7f8463adf700 1 Processor -- start 2023-05-20T01:16:42.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.265+0000 7f8463adf700 1 -- start start 2023-05-20T01:16:42.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.266+0000 7f8463adf700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f845c0717c0 0x7f845c0c4560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:42.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.266+0000 7f8463adf700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f845c0cd1c0 0x7f845c1f1d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:42.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.266+0000 7f8463adf700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f845c159b50 0x7f845c1f2240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:42.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.266+0000 7f8463adf700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f845c1f3440 con 0x7f845c0cd1c0 2023-05-20T01:16:42.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.266+0000 7f8463adf700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f845c0c4aa0 con 0x7f845c159b50 2023-05-20T01:16:42.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.266+0000 7f8463adf700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f845c1f28c0 con 0x7f845c0717c0 2023-05-20T01:16:42.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.266+0000 7f846107a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f845c0cd1c0 0x7f845c1f1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:42.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.266+0000 7f846107a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f845c0cd1c0 0x7f845c1f1d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57886/0 (socket says 172.21.15.16:57886) 2023-05-20T01:16:42.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.266+0000 7f846107a700 1 -- 172.21.15.16:0/2697614123 learned_addr learned my addr 172.21.15.16:0/2697614123 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:42.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.266+0000 7f846187b700 1 --2- 172.21.15.16:0/2697614123 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f845c0717c0 0x7f845c0c4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:42.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.266+0000 7f846207c700 1 --2- 172.21.15.16:0/2697614123 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f845c159b50 0x7f845c1f2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:42.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.266+0000 7f846107a700 1 -- 172.21.15.16:0/2697614123 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f845c0717c0 msgr2=0x7f845c0c4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:42.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.266+0000 7f846107a700 1 --2- 172.21.15.16:0/2697614123 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f845c0717c0 0x7f845c0c4560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:42.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.266+0000 7f846107a700 1 -- 172.21.15.16:0/2697614123 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f845c159b50 msgr2=0x7f845c1f2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:42.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.266+0000 7f846107a700 1 --2- 172.21.15.16:0/2697614123 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f845c159b50 0x7f845c1f2240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:42.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.266+0000 7f846107a700 1 -- 172.21.15.16:0/2697614123 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f844c00b7a0 con 0x7f845c0cd1c0 2023-05-20T01:16:42.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.266+0000 7f846207c700 1 --2- 172.21.15.16:0/2697614123 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f845c159b50 0x7f845c1f2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:16:42.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.266+0000 7f846187b700 1 --2- 172.21.15.16:0/2697614123 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f845c0717c0 0x7f845c0c4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:16:42.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.266+0000 7f846107a700 1 --2- 172.21.15.16:0/2697614123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f845c0cd1c0 0x7f845c1f1d00 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f84580011e0 tx=0x7f845800cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:42.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.267+0000 7f84527fc700 1 -- 172.21.15.16:0/2697614123 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8458004c80 con 0x7f845c0cd1c0 2023-05-20T01:16:42.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.267+0000 7f8463adf700 1 -- 172.21.15.16:0/2697614123 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f845c071e80 con 0x7f845c0cd1c0 2023-05-20T01:16:42.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.267+0000 7f84527fc700 1 -- 172.21.15.16:0/2697614123 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8458004e00 con 0x7f845c0cd1c0 2023-05-20T01:16:42.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.267+0000 7f84527fc700 1 -- 172.21.15.16:0/2697614123 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8458007310 con 0x7f845c0cd1c0 2023-05-20T01:16:42.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.267+0000 7f8463adf700 1 -- 172.21.15.16:0/2697614123 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f845c06e5e0 con 0x7f845c0cd1c0 2023-05-20T01:16:42.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.267+0000 7f8463adf700 1 -- 172.21.15.16:0/2697614123 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f845c0cec70 con 0x7f845c0cd1c0 2023-05-20T01:16:42.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.268+0000 7f84527fc700 1 -- 172.21.15.16:0/2697614123 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8458007470 con 0x7f845c0cd1c0 2023-05-20T01:16:42.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.268+0000 7f84527fc700 1 --2- 172.21.15.16:0/2697614123 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f844006ace0 0x7f844006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:42.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.268+0000 7f84527fc700 1 -- 172.21.15.16:0/2697614123 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f8458089570 con 0x7f845c0cd1c0 2023-05-20T01:16:42.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.268+0000 7f846187b700 1 --2- 172.21.15.16:0/2697614123 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f844006ace0 0x7f844006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:42.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.269+0000 7f846187b700 1 --2- 172.21.15.16:0/2697614123 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f844006ace0 0x7f844006d190 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f844c00b340 tx=0x7f844c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:42.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.273+0000 7f84527fc700 1 -- 172.21.15.16:0/2697614123 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f84580542b0 con 0x7f845c0cd1c0 2023-05-20T01:16:42.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.472+0000 7f8463adf700 1 -- 172.21.15.16:0/2697614123 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f845c0747b0 con 0x7f845c0cd1c0 2023-05-20T01:16:42.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.472+0000 7f84527fc700 1 -- 172.21.15.16:0/2697614123 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7f84580578d0 con 0x7f845c0cd1c0 2023-05-20T01:16:42.476 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-05-20T01:16:42.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.474+0000 7f8463adf700 1 -- 172.21.15.16:0/2697614123 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f844006ace0 msgr2=0x7f844006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:42.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.474+0000 7f8463adf700 1 --2- 172.21.15.16:0/2697614123 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f844006ace0 0x7f844006d190 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f844c00b340 tx=0x7f844c008040 comp rx=0 tx=0).stop 2023-05-20T01:16:42.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.474+0000 7f8463adf700 1 -- 172.21.15.16:0/2697614123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f845c0cd1c0 msgr2=0x7f845c1f1d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:42.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.474+0000 7f8463adf700 1 --2- 172.21.15.16:0/2697614123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f845c0cd1c0 0x7f845c1f1d00 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f84580011e0 tx=0x7f845800cae0 comp rx=0 tx=0).stop 2023-05-20T01:16:42.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.474+0000 7f8463adf700 1 -- 172.21.15.16:0/2697614123 shutdown_connections 2023-05-20T01:16:42.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.474+0000 7f8463adf700 1 --2- 172.21.15.16:0/2697614123 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f845c159b50 0x7f845c1f2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:42.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.474+0000 7f8463adf700 1 --2- 172.21.15.16:0/2697614123 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f844006ace0 0x7f844006d190 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:42.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.474+0000 7f8463adf700 1 --2- 172.21.15.16:0/2697614123 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f845c0717c0 0x7f845c0c4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:42.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.474+0000 7f8463adf700 1 --2- 172.21.15.16:0/2697614123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f845c0cd1c0 0x7f845c1f1d00 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:42.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.474+0000 7f8463adf700 1 -- 172.21.15.16:0/2697614123 >> 172.21.15.16:0/2697614123 conn(0x7f845c0cffd0 msgr2=0x7f845c154760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:42.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.475+0000 7f8463adf700 1 -- 172.21.15.16:0/2697614123 shutdown_connections 2023-05-20T01:16:42.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.475+0000 7f8463adf700 1 -- 172.21.15.16:0/2697614123 wait complete. 2023-05-20T01:16:42.490 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-05-20T01:16:42.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.611+0000 7f6d4d17b700 1 Processor -- start 2023-05-20T01:16:42.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.611+0000 7f6d4d17b700 1 -- start start 2023-05-20T01:16:42.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.612+0000 7f6d4d17b700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f6d4806f450 0x7f6d480cf890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:42.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.612+0000 7f6d4d17b700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6d4806a510 0x7f6d4806a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:42.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.612+0000 7f6d4d17b700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d480cc510 con 0x7f6d4806d4b0 2023-05-20T01:16:42.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.612+0000 7f6d4d17b700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d4806aed0 con 0x7f6d4806f450 2023-05-20T01:16:42.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.612+0000 7f6d4d17b700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d48071ed0 con 0x7f6d4806a510 2023-05-20T01:16:42.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.612+0000 7f6d4659c700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f6d4806d4b0 0x7f6d48161080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:41190/0 (socket says 172.21.15.16:41190) 2023-05-20T01:16:42.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.612+0000 7f6d4659c700 1 -- 172.21.15.16:0/770319982 learned_addr learned my addr 172.21.15.16:0/770319982 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:16:42.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.612+0000 7f6d46d9d700 1 --2- 172.21.15.16:0/770319982 >> v2:172.21.15.132:3300/0 conn(0x7f6d4806f450 0x7f6d480cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:42.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.612+0000 7f6d4759e700 1 --2- 172.21.15.16:0/770319982 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6d4806a510 0x7f6d4806a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:42.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.612+0000 7f6d45d9b700 1 -- 172.21.15.16:0/770319982 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1265867507 0 0) 0x7f6d480cc510 con 0x7f6d4806d4b0 2023-05-20T01:16:42.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.613+0000 7f6d45d9b700 1 -- 172.21.15.16:0/770319982 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d300035b0 con 0x7f6d4806d4b0 2023-05-20T01:16:42.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.613+0000 7f6d45d9b700 1 -- 172.21.15.16:0/770319982 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2838920304 0 0) 0x7f6d300035b0 con 0x7f6d4806d4b0 2023-05-20T01:16:42.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.613+0000 7f6d46d9d700 1 -- 172.21.15.16:0/770319982 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6d4806a510 msgr2=0x7f6d4806a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:42.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.613+0000 7f6d46d9d700 1 --2- 172.21.15.16:0/770319982 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6d4806a510 0x7f6d4806a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:42.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.613+0000 7f6d46d9d700 1 -- 172.21.15.16:0/770319982 >> v1:172.21.15.16:6789/0 conn(0x7f6d4806d4b0 legacy=0x7f6d48161080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:42.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.613+0000 7f6d46d9d700 1 -- 172.21.15.16:0/770319982 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d480721b0 con 0x7f6d4806f450 2023-05-20T01:16:42.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.613+0000 7f6d46d9d700 1 --2- 172.21.15.16:0/770319982 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6d4806f450 0x7f6d480cf890 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c009840 tx=0x7f6d3c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2c80b868bbbecf4 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:42.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.613+0000 7f6d45d9b700 1 -- 172.21.15.16:0/770319982 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d38001670 con 0x7f6d4806f450 2023-05-20T01:16:42.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.613+0000 7f6d45d9b700 1 -- 172.21.15.16:0/770319982 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d3c004030 con 0x7f6d4806f450 2023-05-20T01:16:42.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.613+0000 7f6d45d9b700 1 -- 172.21.15.16:0/770319982 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d3c0043c0 con 0x7f6d4806f450 2023-05-20T01:16:42.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.614+0000 7f6d4d17b700 1 -- 172.21.15.16:0/770319982 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6d4806f450 msgr2=0x7f6d480cf890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:42.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.614+0000 7f6d4d17b700 1 --2- 172.21.15.16:0/770319982 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6d4806f450 0x7f6d480cf890 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c009840 tx=0x7f6d3c009b50 comp rx=0 tx=0).stop 2023-05-20T01:16:42.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.614+0000 7f6d4d17b700 1 -- 172.21.15.16:0/770319982 shutdown_connections 2023-05-20T01:16:42.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.614+0000 7f6d4d17b700 1 --2- 172.21.15.16:0/770319982 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6d4806a510 0x7f6d4806a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:42.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.614+0000 7f6d4d17b700 1 --2- 172.21.15.16:0/770319982 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6d4806f450 0x7f6d480cf890 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:42.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.614+0000 7f6d4d17b700 1 -- 172.21.15.16:0/770319982 >> 172.21.15.16:0/770319982 conn(0x7f6d480cffb0 msgr2=0x7f6d480d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:42.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.614+0000 7f6d4d17b700 1 -- 172.21.15.16:0/770319982 shutdown_connections 2023-05-20T01:16:42.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.614+0000 7f6d4d17b700 1 -- 172.21.15.16:0/770319982 wait complete. 2023-05-20T01:16:42.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.615+0000 7f6d4d17b700 1 Processor -- start 2023-05-20T01:16:42.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.615+0000 7f6d4d17b700 1 -- start start 2023-05-20T01:16:42.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.616+0000 7f6d4d17b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d4806a510 0x7f6d4814d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:42.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.616+0000 7f6d4d17b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6d4806d4b0 0x7f6d480c5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:42.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.616+0000 7f6d4d17b700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6d4806f450 0x7f6d480c3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:42.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.616+0000 7f6d4d17b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d480c5170 con 0x7f6d4806a510 2023-05-20T01:16:42.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.616+0000 7f6d4d17b700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d480c4560 con 0x7f6d4806d4b0 2023-05-20T01:16:42.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.616+0000 7f6d4d17b700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d480c4840 con 0x7f6d4806f450 2023-05-20T01:16:42.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.616+0000 7f6d46d9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d4806a510 0x7f6d4814d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:42.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.616+0000 7f6d46d9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d4806a510 0x7f6d4814d390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57898/0 (socket says 172.21.15.16:57898) 2023-05-20T01:16:42.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.616+0000 7f6d46d9d700 1 -- 172.21.15.16:0/3911535446 learned_addr learned my addr 172.21.15.16:0/3911535446 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:42.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.616+0000 7f6d4659c700 1 --2- 172.21.15.16:0/3911535446 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6d4806d4b0 0x7f6d480c5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:42.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.616+0000 7f6d4759e700 1 --2- 172.21.15.16:0/3911535446 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6d4806f450 0x7f6d480c3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:42.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.616+0000 7f6d46d9d700 1 -- 172.21.15.16:0/3911535446 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6d4806f450 msgr2=0x7f6d480c3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:42.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.616+0000 7f6d46d9d700 1 --2- 172.21.15.16:0/3911535446 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6d4806f450 0x7f6d480c3ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:42.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.616+0000 7f6d46d9d700 1 -- 172.21.15.16:0/3911535446 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6d4806d4b0 msgr2=0x7f6d480c5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:42.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.616+0000 7f6d46d9d700 1 --2- 172.21.15.16:0/3911535446 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6d4806d4b0 0x7f6d480c5600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:42.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.616+0000 7f6d46d9d700 1 -- 172.21.15.16:0/3911535446 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d3c0094a0 con 0x7f6d4806a510 2023-05-20T01:16:42.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.617+0000 7f6d46d9d700 1 --2- 172.21.15.16:0/3911535446 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d4806a510 0x7f6d4814d390 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c00a010 tx=0x7f6d3c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:42.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.617+0000 7f6d2f7fe700 1 -- 172.21.15.16:0/3911535446 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d3c003c00 con 0x7f6d4806a510 2023-05-20T01:16:42.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.617+0000 7f6d2f7fe700 1 -- 172.21.15.16:0/3911535446 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d3c01f070 con 0x7f6d4806a510 2023-05-20T01:16:42.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.617+0000 7f6d4d17b700 1 -- 172.21.15.16:0/3911535446 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d480cbe50 con 0x7f6d4806a510 2023-05-20T01:16:42.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.617+0000 7f6d2f7fe700 1 -- 172.21.15.16:0/3911535446 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d3c011630 con 0x7f6d4806a510 2023-05-20T01:16:42.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.617+0000 7f6d4d17b700 1 -- 172.21.15.16:0/3911535446 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d480cc1a0 con 0x7f6d4806a510 2023-05-20T01:16:42.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.618+0000 7f6d4d17b700 1 -- 172.21.15.16:0/3911535446 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d480cc9a0 con 0x7f6d4806a510 2023-05-20T01:16:42.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.618+0000 7f6d2f7fe700 1 -- 172.21.15.16:0/3911535446 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6d3c021030 con 0x7f6d4806a510 2023-05-20T01:16:42.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.619+0000 7f6d2f7fe700 1 --2- 172.21.15.16:0/3911535446 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f6d3406ace0 0x7f6d3406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:42.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.619+0000 7f6d2f7fe700 1 -- 172.21.15.16:0/3911535446 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f6d3c08aa50 con 0x7f6d4806a510 2023-05-20T01:16:42.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.619+0000 7f6d4659c700 1 --2- 172.21.15.16:0/3911535446 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f6d3406ace0 0x7f6d3406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:42.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.619+0000 7f6d4659c700 1 --2- 172.21.15.16:0/3911535446 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f6d3406ace0 0x7f6d3406d190 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6d4815cae0 tx=0x7f6d38001ce0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:42.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.623+0000 7f6d2f7fe700 1 -- 172.21.15.16:0/3911535446 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f6d3c0558a0 con 0x7f6d4806a510 2023-05-20T01:16:42.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.835+0000 7f6d4d17b700 1 -- 172.21.15.16:0/3911535446 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f6d480cc9a0 con 0x7f6d4806a510 2023-05-20T01:16:42.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.836+0000 7f6d2f7fe700 1 -- 172.21.15.16:0/3911535446 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) v1 ==== 80+0+0 (secure 0 0 0) 0x7f6d3c058ec0 con 0x7f6d4806a510 2023-05-20T01:16:42.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.838+0000 7f6d4d17b700 1 -- 172.21.15.16:0/3911535446 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f6d3406ace0 msgr2=0x7f6d3406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:42.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.838+0000 7f6d4d17b700 1 --2- 172.21.15.16:0/3911535446 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f6d3406ace0 0x7f6d3406d190 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6d4815cae0 tx=0x7f6d38001ce0 comp rx=0 tx=0).stop 2023-05-20T01:16:42.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.838+0000 7f6d4d17b700 1 -- 172.21.15.16:0/3911535446 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d4806a510 msgr2=0x7f6d4814d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:42.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.838+0000 7f6d4d17b700 1 --2- 172.21.15.16:0/3911535446 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d4806a510 0x7f6d4814d390 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c00a010 tx=0x7f6d3c004750 comp rx=0 tx=0).stop 2023-05-20T01:16:42.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.838+0000 7f6d4d17b700 1 -- 172.21.15.16:0/3911535446 shutdown_connections 2023-05-20T01:16:42.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.838+0000 7f6d4d17b700 1 --2- 172.21.15.16:0/3911535446 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f6d3406ace0 0x7f6d3406d190 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:42.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.838+0000 7f6d4d17b700 1 --2- 172.21.15.16:0/3911535446 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6d4806f450 0x7f6d480c3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:42.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.838+0000 7f6d4d17b700 1 --2- 172.21.15.16:0/3911535446 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d4806a510 0x7f6d4814d390 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:42.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.838+0000 7f6d4d17b700 1 --2- 172.21.15.16:0/3911535446 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6d4806d4b0 0x7f6d480c5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:42.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.838+0000 7f6d4d17b700 1 -- 172.21.15.16:0/3911535446 >> 172.21.15.16:0/3911535446 conn(0x7f6d480cffb0 msgr2=0x7f6d4806d880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:42.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.838+0000 7f6d4d17b700 1 -- 172.21.15.16:0/3911535446 shutdown_connections 2023-05-20T01:16:42.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:42.838+0000 7f6d4d17b700 1 -- 172.21.15.16:0/3911535446 wait complete. 2023-05-20T01:16:42.843 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2023-05-20T01:16:42.854 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi016.front.sepia.ceph.com... 2023-05-20T01:16:42.855 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:16:42.855 DEBUG:teuthology.orchestra.run.smithi016:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2023-05-20T01:16:42.892 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi016.stderr:ceph-fuse[105788]: fuse finished with error 0 and tester_r 0 2023-05-20T01:16:42.893 INFO:teuthology.orchestra.run:waiting for 300 2023-05-20T01:16:42.905 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2023-05-20T01:16:42.906 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:16:42.906 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2023-05-20T01:16:42.969 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-20T01:16:43.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.130+0000 7f6c2a351700 1 Processor -- start 2023-05-20T01:16:43.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.130+0000 7f6c2a351700 1 -- start start 2023-05-20T01:16:43.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.131+0000 7f6c2a351700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6c240ccec0 0x7f6c240713c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:43.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.131+0000 7f6c2a351700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6c24071900 0x7f6c240ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:43.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.131+0000 7f6c2a351700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c240cb590 con 0x7f6c24157190 2023-05-20T01:16:43.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.131+0000 7f6c2a351700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c240ca9a0 con 0x7f6c24071900 2023-05-20T01:16:43.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.131+0000 7f6c2a351700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c240cac60 con 0x7f6c240ccec0 2023-05-20T01:16:43.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.131+0000 7f6c23fff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f6c24157190 0x7f6c240cc7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:41202/0 (socket says 172.21.15.16:41202) 2023-05-20T01:16:43.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.131+0000 7f6c23fff700 1 -- 172.21.15.16:0/3076371662 learned_addr learned my addr 172.21.15.16:0/3076371662 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:16:43.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.131+0000 7f6c288ee700 1 --2- 172.21.15.16:0/3076371662 >> v2:172.21.15.16:3300/0 conn(0x7f6c24071900 0x7f6c240ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:43.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.131+0000 7f6c237fe700 1 --2- 172.21.15.16:0/3076371662 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6c240ccec0 0x7f6c240713c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:43.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.131+0000 7f6c22ffd700 1 -- 172.21.15.16:0/3076371662 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2012254914 0 0) 0x7f6c240cb590 con 0x7f6c24157190 2023-05-20T01:16:43.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.131+0000 7f6c22ffd700 1 -- 172.21.15.16:0/3076371662 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c0c0057c0 con 0x7f6c24157190 2023-05-20T01:16:43.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.131+0000 7f6c288ee700 1 -- 172.21.15.16:0/3076371662 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6c240ccec0 msgr2=0x7f6c240713c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:43.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.131+0000 7f6c288ee700 1 --2- 172.21.15.16:0/3076371662 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6c240ccec0 0x7f6c240713c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:43.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.131+0000 7f6c288ee700 1 -- 172.21.15.16:0/3076371662 >> v1:172.21.15.16:6789/0 conn(0x7f6c24157190 legacy=0x7f6c240cc7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:43.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.131+0000 7f6c288ee700 1 -- 172.21.15.16:0/3076371662 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c240caf40 con 0x7f6c24071900 2023-05-20T01:16:43.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.131+0000 7f6c288ee700 1 --2- 172.21.15.16:0/3076371662 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c24071900 0x7f6c240ce740 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c009d90 tx=0x7f6c1c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=833c556acf288271 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:43.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.131+0000 7f6c22ffd700 1 -- 172.21.15.16:0/3076371662 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c1c00e070 con 0x7f6c24071900 2023-05-20T01:16:43.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.131+0000 7f6c22ffd700 1 -- 172.21.15.16:0/3076371662 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c1c00bb20 con 0x7f6c24071900 2023-05-20T01:16:43.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.131+0000 7f6c22ffd700 1 -- 172.21.15.16:0/3076371662 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c1c012430 con 0x7f6c24071900 2023-05-20T01:16:43.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.132+0000 7f6c2a351700 1 -- 172.21.15.16:0/3076371662 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c24071900 msgr2=0x7f6c240ce740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:43.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.132+0000 7f6c2a351700 1 --2- 172.21.15.16:0/3076371662 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c24071900 0x7f6c240ce740 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c009d90 tx=0x7f6c1c00b330 comp rx=0 tx=0).stop 2023-05-20T01:16:43.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.132+0000 7f6c2a351700 1 -- 172.21.15.16:0/3076371662 shutdown_connections 2023-05-20T01:16:43.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.132+0000 7f6c2a351700 1 --2- 172.21.15.16:0/3076371662 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c24071900 0x7f6c240ce740 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:43.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.132+0000 7f6c2a351700 1 --2- 172.21.15.16:0/3076371662 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6c240ccec0 0x7f6c240713c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:43.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.132+0000 7f6c2a351700 1 -- 172.21.15.16:0/3076371662 >> 172.21.15.16:0/3076371662 conn(0x7f6c240cffb0 msgr2=0x7f6c240d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:43.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.132+0000 7f6c2a351700 1 -- 172.21.15.16:0/3076371662 shutdown_connections 2023-05-20T01:16:43.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.132+0000 7f6c2a351700 1 -- 172.21.15.16:0/3076371662 wait complete. 2023-05-20T01:16:43.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.133+0000 7f6c2a351700 1 Processor -- start 2023-05-20T01:16:43.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.133+0000 7f6c2a351700 1 -- start start 2023-05-20T01:16:43.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.134+0000 7f6c2a351700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6c24071900 0x7f6c24140fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:43.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.134+0000 7f6c2a351700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c240ccec0 0x7f6c241e2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:43.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.134+0000 7f6c2a351700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6c24157190 0x7f6c240c6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:43.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.134+0000 7f6c2a351700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c24142320 con 0x7f6c240ccec0 2023-05-20T01:16:43.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.134+0000 7f6c2a351700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c240c34c0 con 0x7f6c24157190 2023-05-20T01:16:43.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.134+0000 7f6c2a351700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c241e2650 con 0x7f6c24071900 2023-05-20T01:16:43.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.134+0000 7f6c237fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c240ccec0 0x7f6c241e2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:43.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.134+0000 7f6c237fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c240ccec0 0x7f6c241e2110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57914/0 (socket says 172.21.15.16:57914) 2023-05-20T01:16:43.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.134+0000 7f6c237fe700 1 -- 172.21.15.16:0/1744481025 learned_addr learned my addr 172.21.15.16:0/1744481025 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:43.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.134+0000 7f6c23fff700 1 --2- 172.21.15.16:0/1744481025 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6c24071900 0x7f6c24140fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:43.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.134+0000 7f6c288ee700 1 --2- 172.21.15.16:0/1744481025 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6c24157190 0x7f6c240c6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:43.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.135+0000 7f6c237fe700 1 -- 172.21.15.16:0/1744481025 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6c24071900 msgr2=0x7f6c24140fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:43.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.135+0000 7f6c237fe700 1 --2- 172.21.15.16:0/1744481025 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6c24071900 0x7f6c24140fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:43.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.135+0000 7f6c237fe700 1 -- 172.21.15.16:0/1744481025 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6c24157190 msgr2=0x7f6c240c6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:43.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.135+0000 7f6c237fe700 1 --2- 172.21.15.16:0/1744481025 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6c24157190 0x7f6c240c6300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:43.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.135+0000 7f6c237fe700 1 -- 172.21.15.16:0/1744481025 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c1c009a70 con 0x7f6c240ccec0 2023-05-20T01:16:43.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.135+0000 7f6c237fe700 1 --2- 172.21.15.16:0/1744481025 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c240ccec0 0x7f6c241e2110 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6c1800e840 tx=0x7f6c1800eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:43.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.135+0000 7f6c20ff9700 1 -- 172.21.15.16:0/1744481025 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c18013610 con 0x7f6c240ccec0 2023-05-20T01:16:43.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.135+0000 7f6c20ff9700 1 -- 172.21.15.16:0/1744481025 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c18013770 con 0x7f6c240ccec0 2023-05-20T01:16:43.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.135+0000 7f6c20ff9700 1 -- 172.21.15.16:0/1744481025 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c18013b20 con 0x7f6c240ccec0 2023-05-20T01:16:43.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.135+0000 7f6c2a351700 1 -- 172.21.15.16:0/1744481025 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c24157560 con 0x7f6c240ccec0 2023-05-20T01:16:43.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.135+0000 7f6c2a351700 1 -- 172.21.15.16:0/1744481025 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c240cb220 con 0x7f6c240ccec0 2023-05-20T01:16:43.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.136+0000 7f6c2a351700 1 -- 172.21.15.16:0/1744481025 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c240cba20 con 0x7f6c240ccec0 2023-05-20T01:16:43.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.136+0000 7f6c20ff9700 1 -- 172.21.15.16:0/1744481025 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6c18013c80 con 0x7f6c240ccec0 2023-05-20T01:16:43.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.137+0000 7f6c20ff9700 1 --2- 172.21.15.16:0/1744481025 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f6c1006ace0 0x7f6c1006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:43.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.137+0000 7f6c23fff700 1 --2- 172.21.15.16:0/1744481025 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f6c1006ace0 0x7f6c1006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:43.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.137+0000 7f6c20ff9700 1 -- 172.21.15.16:0/1744481025 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f6c18089300 con 0x7f6c240ccec0 2023-05-20T01:16:43.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.137+0000 7f6c23fff700 1 --2- 172.21.15.16:0/1744481025 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f6c1006ace0 0x7f6c1006d190 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6c14007fd0 tx=0x7f6c14007dc0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:43.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.142+0000 7f6c20ff9700 1 -- 172.21.15.16:0/1744481025 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f6c18057b60 con 0x7f6c240ccec0 2023-05-20T01:16:43.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.341+0000 7f6c2a351700 1 -- 172.21.15.16:0/1744481025 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6c240747b0 con 0x7f6c240ccec0 2023-05-20T01:16:43.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.342+0000 7f6c20ff9700 1 -- 172.21.15.16:0/1744481025 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+20026 (secure 0 0 0) 0x7f6c180576f0 con 0x7f6c240ccec0 2023-05-20T01:16:43.346 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:16:43.347 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"fsid":"c2245cc5-ecab-4087-a59b-bc81db901676","created":"2023-05-20T01:16:09.549230+0000","modified":"2023-05-20T01:16:32.768495+0000","last_up_change":"2023-05-20T01:16:19.788624+0000","last_in_change":"2023-05-20T01:16:12.092160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-05-20T01:16:20.934366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-05-20T01:16:21.372294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-05-20T01:16:26.208287+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-05-20T01:16:27.292268+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-05-20T01:16:30.357285+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"317a83cd-19c8-4958-a301-6087ef342289","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6825","nonce":3923634179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6827","nonce":3923634179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6831","nonce":3923634179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6829","nonce":3923634179}]},"public_addr":"172.21.15.16:6825/3923634179","cluster_addr":"172.21.15.16:6827/3923634179","heartbeat_back_addr":"172.21.15.16:6831/3923634179","heartbeat_front_addr":"172.21.15.16:6829/3923634179","state":["exists","up"]},{"osd":1,"uuid":"5581046f-8219-4089-8d7d-b7846399505e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6809","nonce":3860264220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6811","nonce":3860264220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6815","nonce":3860264220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6813","nonce":3860264220}]},"public_addr":"172.21.15.16:6809/3860264220","cluster_addr":"172.21.15.16:6811/3860264220","heartbeat_back_addr":"172.21.15.16:6815/3860264220","heartbeat_front_addr":"172.21.15.16:6813/3860264220","state":["exists","up"]},{"osd":2,"uuid":"072e9e66-4133-478e-8d4a-64d46c2963f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6817","nonce":293187030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6819","nonce":293187030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6823","nonce":293187030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6821","nonce":293187030}]},"public_addr":"172.21.15.16:6817/293187030","cluster_addr":"172.21.15.16:6819/293187030","heartbeat_back_addr":"172.21.15.16:6823/293187030","heartbeat_front_addr":"172.21.15.16:6821/293187030","state":["exists","up"]},{"osd":3,"uuid":"11e1bba6-00f8-48dd-9028-d8d716b7f145","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6801","nonce":3669781890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6803","nonce":3669781890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6807","nonce":3669781890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6805","nonce":3669781890}]},"public_addr":"172.21.15.16:6801/3669781890","cluster_addr":"172.21.15.16:6803/3669781890","heartbeat_back_addr":"172.21.15.16:6807/3669781890","heartbeat_front_addr":"172.21.15.16:6805/3669781890","state":["exists","up"]},{"osd":4,"uuid":"b8909c3a-cf70-46cb-8d0b-009c707cf9d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6819","nonce":506317223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6823","nonce":506317223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6829","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6831","nonce":506317223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6827","nonce":506317223}]},"public_addr":"172.21.15.132:6819/506317223","cluster_addr":"172.21.15.132:6823/506317223","heartbeat_back_addr":"172.21.15.132:6831/506317223","heartbeat_front_addr":"172.21.15.132:6827/506317223","state":["exists","up"]},{"osd":5,"uuid":"b1818304-4583-47bf-a6ec-0bfbb5166233","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6818","nonce":1844689995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6822","nonce":1844689995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6830","nonce":1844689995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6826","nonce":1844689995}]},"public_addr":"172.21.15.132:6818/1844689995","cluster_addr":"172.21.15.132:6822/1844689995","heartbeat_back_addr":"172.21.15.132:6830/1844689995","heartbeat_front_addr":"172.21.15.132:6826/1844689995","state":["exists","up"]},{"osd":6,"uuid":"d37f4691-eed1-4a93-8818-18354f03f4a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6801","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6802","nonce":727581123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6805","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6807","nonce":727581123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6813","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6815","nonce":727581123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6811","nonce":727581123}]},"public_addr":"172.21.15.132:6802/727581123","cluster_addr":"172.21.15.132:6807/727581123","heartbeat_back_addr":"172.21.15.132:6815/727581123","heartbeat_front_addr":"172.21.15.132:6811/727581123","state":["exists","up"]},{"osd":7,"uuid":"4f712f62-5d1d-4816-8042-03209144f8b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6803","nonce":466001149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6806","nonce":466001149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6814","nonce":466001149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6810","nonce":466001149}]},"public_addr":"172.21.15.132:6803/466001149","cluster_addr":"172.21.15.132:6806/466001149","heartbeat_back_addr":"172.21.15.132:6814/466001149","heartbeat_front_addr":"172.21.15.132:6810/466001149","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.338863+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.278680+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305694+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.246835+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.276237+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.304441+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305421+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.267050+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-20T01:16:43.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.345+0000 7f6c2a351700 1 -- 172.21.15.16:0/1744481025 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f6c1006ace0 msgr2=0x7f6c1006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:43.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.345+0000 7f6c2a351700 1 --2- 172.21.15.16:0/1744481025 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f6c1006ace0 0x7f6c1006d190 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6c14007fd0 tx=0x7f6c14007dc0 comp rx=0 tx=0).stop 2023-05-20T01:16:43.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.345+0000 7f6c2a351700 1 -- 172.21.15.16:0/1744481025 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c240ccec0 msgr2=0x7f6c241e2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:43.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.345+0000 7f6c2a351700 1 --2- 172.21.15.16:0/1744481025 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c240ccec0 0x7f6c241e2110 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6c1800e840 tx=0x7f6c1800eb50 comp rx=0 tx=0).stop 2023-05-20T01:16:43.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.345+0000 7f6c2a351700 1 -- 172.21.15.16:0/1744481025 shutdown_connections 2023-05-20T01:16:43.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.345+0000 7f6c2a351700 1 --2- 172.21.15.16:0/1744481025 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6c24157190 0x7f6c240c6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:43.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.345+0000 7f6c2a351700 1 --2- 172.21.15.16:0/1744481025 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f6c1006ace0 0x7f6c1006d190 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:43.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.345+0000 7f6c2a351700 1 --2- 172.21.15.16:0/1744481025 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6c24071900 0x7f6c24140fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:43.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.345+0000 7f6c2a351700 1 --2- 172.21.15.16:0/1744481025 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c240ccec0 0x7f6c241e2110 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:43.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.345+0000 7f6c2a351700 1 -- 172.21.15.16:0/1744481025 >> 172.21.15.16:0/1744481025 conn(0x7f6c240cffb0 msgr2=0x7f6c240d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:43.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.345+0000 7f6c2a351700 1 -- 172.21.15.16:0/1744481025 shutdown_connections 2023-05-20T01:16:43.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.345+0000 7f6c2a351700 1 -- 172.21.15.16:0/1744481025 wait complete. 2023-05-20T01:16:43.361 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-20T01:16:43.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.482+0000 7f427da33700 1 Processor -- start 2023-05-20T01:16:43.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.482+0000 7f427da33700 1 -- start start 2023-05-20T01:16:43.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.482+0000 7f427da33700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f427806f450 0x7f42780cf890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:43.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.482+0000 7f427da33700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f427806d4b0 0x7f4278161080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:43.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.482+0000 7f427da33700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42780cc510 con 0x7f427806a510 2023-05-20T01:16:43.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.483+0000 7f427da33700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42781615c0 con 0x7f427806d4b0 2023-05-20T01:16:43.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.483+0000 7f427da33700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4278071ed0 con 0x7f427806f450 2023-05-20T01:16:43.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.483+0000 7f42767fc700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f427806d4b0 0x7f4278161080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:43.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.483+0000 7f4276ffd700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f427806f450 0x7f42780cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:43.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.483+0000 7f42777fe700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f427806a510 0x7f42780cdb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.16:50932/0 (socket says 172.21.15.16:50932) 2023-05-20T01:16:43.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.483+0000 7f42777fe700 1 -- 172.21.15.16:0/1959914433 learned_addr learned my addr 172.21.15.16:0/1959914433 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:16:43.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.483+0000 7f42767fc700 1 -- 172.21.15.16:0/1959914433 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f427806f450 msgr2=0x7f42780cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:43.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.483+0000 7f42767fc700 1 --2- 172.21.15.16:0/1959914433 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f427806f450 0x7f42780cf890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:43.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.483+0000 7f42767fc700 1 -- 172.21.15.16:0/1959914433 >> v1:172.21.15.132:6789/0 conn(0x7f427806a510 legacy=0x7f42780cdb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:43.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.483+0000 7f42767fc700 1 -- 172.21.15.16:0/1959914433 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42780721b0 con 0x7f427806d4b0 2023-05-20T01:16:43.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.483+0000 7f4275ffb700 1 -- 172.21.15.16:0/1959914433 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2124007217 0 0) 0x7f42780cc510 con 0x7f427806a510 2023-05-20T01:16:43.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.483+0000 7f42767fc700 1 --2- 172.21.15.16:0/1959914433 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f427806d4b0 0x7f4278161080 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f426c009840 tx=0x7f426c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ff73978bfe0f103 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:43.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.483+0000 7f4275ffb700 1 -- 172.21.15.16:0/1959914433 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f426c011830 con 0x7f427806d4b0 2023-05-20T01:16:43.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.484+0000 7f4275ffb700 1 -- 172.21.15.16:0/1959914433 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f426c011990 con 0x7f427806d4b0 2023-05-20T01:16:43.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.484+0000 7f4275ffb700 1 -- 172.21.15.16:0/1959914433 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f426c011d20 con 0x7f427806d4b0 2023-05-20T01:16:43.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.484+0000 7f427da33700 1 -- 172.21.15.16:0/1959914433 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f427806d4b0 msgr2=0x7f4278161080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:43.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.484+0000 7f427da33700 1 --2- 172.21.15.16:0/1959914433 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f427806d4b0 0x7f4278161080 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f426c009840 tx=0x7f426c009b50 comp rx=0 tx=0).stop 2023-05-20T01:16:43.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.484+0000 7f427da33700 1 -- 172.21.15.16:0/1959914433 shutdown_connections 2023-05-20T01:16:43.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.484+0000 7f427da33700 1 --2- 172.21.15.16:0/1959914433 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f427806f450 0x7f42780cf890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:43.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.484+0000 7f427da33700 1 --2- 172.21.15.16:0/1959914433 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f427806d4b0 0x7f4278161080 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:43.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.484+0000 7f427da33700 1 -- 172.21.15.16:0/1959914433 >> 172.21.15.16:0/1959914433 conn(0x7f42780cffb0 msgr2=0x7f42780d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:43.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.484+0000 7f427da33700 1 -- 172.21.15.16:0/1959914433 shutdown_connections 2023-05-20T01:16:43.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.484+0000 7f427da33700 1 -- 172.21.15.16:0/1959914433 wait complete. 2023-05-20T01:16:43.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.485+0000 7f427da33700 1 Processor -- start 2023-05-20T01:16:43.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.486+0000 7f427da33700 1 -- start start 2023-05-20T01:16:43.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.486+0000 7f427da33700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f427806a510 0x7f4278149d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:43.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.486+0000 7f427da33700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f427806d4b0 0x7f427814c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:43.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.486+0000 7f427da33700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f427806f450 0x7f427814cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:43.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.486+0000 7f427da33700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4278161430 con 0x7f427806d4b0 2023-05-20T01:16:43.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.486+0000 7f427da33700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f427814a2b0 con 0x7f427806f450 2023-05-20T01:16:43.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.486+0000 7f427da33700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f427814ac40 con 0x7f427806a510 2023-05-20T01:16:43.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.486+0000 7f42767fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f427806d4b0 0x7f427814c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:43.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.486+0000 7f42767fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f427806d4b0 0x7f427814c5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57924/0 (socket says 172.21.15.16:57924) 2023-05-20T01:16:43.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.486+0000 7f42767fc700 1 -- 172.21.15.16:0/2861357922 learned_addr learned my addr 172.21.15.16:0/2861357922 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:43.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.486+0000 7f42777fe700 1 --2- 172.21.15.16:0/2861357922 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f427806f450 0x7f427814cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:43.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.486+0000 7f4276ffd700 1 --2- 172.21.15.16:0/2861357922 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f427806a510 0x7f4278149d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:43.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.487+0000 7f42767fc700 1 -- 172.21.15.16:0/2861357922 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f427806a510 msgr2=0x7f4278149d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:43.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.487+0000 7f42767fc700 1 --2- 172.21.15.16:0/2861357922 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f427806a510 0x7f4278149d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:43.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.487+0000 7f42767fc700 1 -- 172.21.15.16:0/2861357922 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f427806f450 msgr2=0x7f427814cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:43.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.487+0000 7f42767fc700 1 --2- 172.21.15.16:0/2861357922 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f427806f450 0x7f427814cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:43.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.487+0000 7f42767fc700 1 -- 172.21.15.16:0/2861357922 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f426c0094a0 con 0x7f427806d4b0 2023-05-20T01:16:43.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.487+0000 7f42767fc700 1 --2- 172.21.15.16:0/2861357922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f427806d4b0 0x7f427814c5e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f426c009b90 tx=0x7f426c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:43.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.487+0000 7f425ffff700 1 -- 172.21.15.16:0/2861357922 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f426c0036a0 con 0x7f427806d4b0 2023-05-20T01:16:43.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.487+0000 7f425ffff700 1 -- 172.21.15.16:0/2861357922 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f426c01f070 con 0x7f427806d4b0 2023-05-20T01:16:43.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.487+0000 7f427da33700 1 -- 172.21.15.16:0/2861357922 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42780cbe50 con 0x7f427806d4b0 2023-05-20T01:16:43.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.487+0000 7f425ffff700 1 -- 172.21.15.16:0/2861357922 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f426c003a40 con 0x7f427806d4b0 2023-05-20T01:16:43.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.487+0000 7f427da33700 1 -- 172.21.15.16:0/2861357922 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42780cc1a0 con 0x7f427806d4b0 2023-05-20T01:16:43.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.488+0000 7f427da33700 1 -- 172.21.15.16:0/2861357922 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f427806a9b0 con 0x7f427806d4b0 2023-05-20T01:16:43.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.488+0000 7f425ffff700 1 -- 172.21.15.16:0/2861357922 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f426c003ba0 con 0x7f427806d4b0 2023-05-20T01:16:43.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.489+0000 7f425ffff700 1 --2- 172.21.15.16:0/2861357922 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f426406ace0 0x7f426406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:43.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.489+0000 7f425ffff700 1 -- 172.21.15.16:0/2861357922 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f426c021070 con 0x7f427806d4b0 2023-05-20T01:16:43.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.489+0000 7f4276ffd700 1 --2- 172.21.15.16:0/2861357922 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f426406ace0 0x7f426406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:43.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.489+0000 7f4276ffd700 1 --2- 172.21.15.16:0/2861357922 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f426406ace0 0x7f426406d190 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f42781f6180 tx=0x7f4268009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:43.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.493+0000 7f425ffff700 1 -- 172.21.15.16:0/2861357922 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f426c055d90 con 0x7f427806d4b0 2023-05-20T01:16:43.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.690+0000 7f427da33700 1 -- 172.21.15.16:0/2861357922 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4278066620 con 0x7f427806d4b0 2023-05-20T01:16:43.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.692+0000 7f425ffff700 1 -- 172.21.15.16:0/2861357922 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+20026 (secure 0 0 0) 0x7f426c0593b0 con 0x7f427806d4b0 2023-05-20T01:16:43.695 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:16:43.696 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"fsid":"c2245cc5-ecab-4087-a59b-bc81db901676","created":"2023-05-20T01:16:09.549230+0000","modified":"2023-05-20T01:16:32.768495+0000","last_up_change":"2023-05-20T01:16:19.788624+0000","last_in_change":"2023-05-20T01:16:12.092160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-05-20T01:16:20.934366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-05-20T01:16:21.372294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-05-20T01:16:26.208287+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-05-20T01:16:27.292268+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-05-20T01:16:30.357285+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"317a83cd-19c8-4958-a301-6087ef342289","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6825","nonce":3923634179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6827","nonce":3923634179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6831","nonce":3923634179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6829","nonce":3923634179}]},"public_addr":"172.21.15.16:6825/3923634179","cluster_addr":"172.21.15.16:6827/3923634179","heartbeat_back_addr":"172.21.15.16:6831/3923634179","heartbeat_front_addr":"172.21.15.16:6829/3923634179","state":["exists","up"]},{"osd":1,"uuid":"5581046f-8219-4089-8d7d-b7846399505e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6809","nonce":3860264220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6811","nonce":3860264220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6815","nonce":3860264220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6813","nonce":3860264220}]},"public_addr":"172.21.15.16:6809/3860264220","cluster_addr":"172.21.15.16:6811/3860264220","heartbeat_back_addr":"172.21.15.16:6815/3860264220","heartbeat_front_addr":"172.21.15.16:6813/3860264220","state":["exists","up"]},{"osd":2,"uuid":"072e9e66-4133-478e-8d4a-64d46c2963f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6817","nonce":293187030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6819","nonce":293187030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6823","nonce":293187030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6821","nonce":293187030}]},"public_addr":"172.21.15.16:6817/293187030","cluster_addr":"172.21.15.16:6819/293187030","heartbeat_back_addr":"172.21.15.16:6823/293187030","heartbeat_front_addr":"172.21.15.16:6821/293187030","state":["exists","up"]},{"osd":3,"uuid":"11e1bba6-00f8-48dd-9028-d8d716b7f145","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6801","nonce":3669781890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6803","nonce":3669781890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6807","nonce":3669781890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6805","nonce":3669781890}]},"public_addr":"172.21.15.16:6801/3669781890","cluster_addr":"172.21.15.16:6803/3669781890","heartbeat_back_addr":"172.21.15.16:6807/3669781890","heartbeat_front_addr":"172.21.15.16:6805/3669781890","state":["exists","up"]},{"osd":4,"uuid":"b8909c3a-cf70-46cb-8d0b-009c707cf9d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6819","nonce":506317223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6823","nonce":506317223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6829","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6831","nonce":506317223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6827","nonce":506317223}]},"public_addr":"172.21.15.132:6819/506317223","cluster_addr":"172.21.15.132:6823/506317223","heartbeat_back_addr":"172.21.15.132:6831/506317223","heartbeat_front_addr":"172.21.15.132:6827/506317223","state":["exists","up"]},{"osd":5,"uuid":"b1818304-4583-47bf-a6ec-0bfbb5166233","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6818","nonce":1844689995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6822","nonce":1844689995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6830","nonce":1844689995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6826","nonce":1844689995}]},"public_addr":"172.21.15.132:6818/1844689995","cluster_addr":"172.21.15.132:6822/1844689995","heartbeat_back_addr":"172.21.15.132:6830/1844689995","heartbeat_front_addr":"172.21.15.132:6826/1844689995","state":["exists","up"]},{"osd":6,"uuid":"d37f4691-eed1-4a93-8818-18354f03f4a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6801","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6802","nonce":727581123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6805","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6807","nonce":727581123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6813","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6815","nonce":727581123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6811","nonce":727581123}]},"public_addr":"172.21.15.132:6802/727581123","cluster_addr":"172.21.15.132:6807/727581123","heartbeat_back_addr":"172.21.15.132:6815/727581123","heartbeat_front_addr":"172.21.15.132:6811/727581123","state":["exists","up"]},{"osd":7,"uuid":"4f712f62-5d1d-4816-8042-03209144f8b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6803","nonce":466001149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6806","nonce":466001149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6814","nonce":466001149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6810","nonce":466001149}]},"public_addr":"172.21.15.132:6803/466001149","cluster_addr":"172.21.15.132:6806/466001149","heartbeat_back_addr":"172.21.15.132:6814/466001149","heartbeat_front_addr":"172.21.15.132:6810/466001149","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.338863+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.278680+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305694+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.246835+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.276237+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.304441+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305421+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.267050+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-20T01:16:43.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.694+0000 7f427da33700 1 -- 172.21.15.16:0/2861357922 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f426406ace0 msgr2=0x7f426406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:43.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.694+0000 7f427da33700 1 --2- 172.21.15.16:0/2861357922 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f426406ace0 0x7f426406d190 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f42781f6180 tx=0x7f4268009210 comp rx=0 tx=0).stop 2023-05-20T01:16:43.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.694+0000 7f427da33700 1 -- 172.21.15.16:0/2861357922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f427806d4b0 msgr2=0x7f427814c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:43.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.694+0000 7f427da33700 1 --2- 172.21.15.16:0/2861357922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f427806d4b0 0x7f427814c5e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f426c009b90 tx=0x7f426c004750 comp rx=0 tx=0).stop 2023-05-20T01:16:43.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.694+0000 7f427da33700 1 -- 172.21.15.16:0/2861357922 shutdown_connections 2023-05-20T01:16:43.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.694+0000 7f427da33700 1 --2- 172.21.15.16:0/2861357922 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f427806f450 0x7f427814cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:43.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.694+0000 7f427da33700 1 --2- 172.21.15.16:0/2861357922 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f426406ace0 0x7f426406d190 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:43.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.694+0000 7f427da33700 1 --2- 172.21.15.16:0/2861357922 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f427806a510 0x7f4278149d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:43.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.694+0000 7f427da33700 1 --2- 172.21.15.16:0/2861357922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f427806d4b0 0x7f427814c5e0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:43.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.694+0000 7f427da33700 1 -- 172.21.15.16:0/2861357922 >> 172.21.15.16:0/2861357922 conn(0x7f42780cffb0 msgr2=0x7f42780c95e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:43.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.694+0000 7f427da33700 1 -- 172.21.15.16:0/2861357922 shutdown_connections 2023-05-20T01:16:43.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.694+0000 7f427da33700 1 -- 172.21.15.16:0/2861357922 wait complete. 2023-05-20T01:16:43.710 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-05-20T01:16:43.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.834+0000 7fda3b495700 1 Processor -- start 2023-05-20T01:16:43.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.834+0000 7fda3b495700 1 -- start start 2023-05-20T01:16:43.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.835+0000 7fda3b495700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fda3415cec0 0x7fda34071bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:43.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.835+0000 7fda3b495700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fda34072130 0x7fda340cf9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:43.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.835+0000 7fda3b495700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda34072590 con 0x7fda3415d290 2023-05-20T01:16:43.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.835+0000 7fda3b495700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda340cd630 con 0x7fda34072130 2023-05-20T01:16:43.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.835+0000 7fda3b495700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda340cca60 con 0x7fda3415cec0 2023-05-20T01:16:43.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.835+0000 7fda39a32700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fda34072130 0x7fda340cf9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:43.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.835+0000 7fda39a32700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fda34072130 0x7fda340cf9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57936/0 (socket says 172.21.15.16:57936) 2023-05-20T01:16:43.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.835+0000 7fda39a32700 1 -- 172.21.15.16:0/3044003467 learned_addr learned my addr 172.21.15.16:0/3044003467 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:43.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.835+0000 7fda39231700 1 --2- 172.21.15.16:0/3044003467 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fda3415cec0 0x7fda34071bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:43.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.835+0000 7fda39a32700 1 -- 172.21.15.16:0/3044003467 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fda3415cec0 msgr2=0x7fda34071bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:43.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.835+0000 7fda39a32700 1 --2- 172.21.15.16:0/3044003467 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fda3415cec0 0x7fda34071bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:43.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.835+0000 7fda39a32700 1 -- 172.21.15.16:0/3044003467 >> v1:172.21.15.132:6789/0 conn(0x7fda3415d290 legacy=0x7fda340cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:43.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.835+0000 7fda39a32700 1 -- 172.21.15.16:0/3044003467 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda340ccd00 con 0x7fda34072130 2023-05-20T01:16:43.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.835+0000 7fda39a32700 1 --2- 172.21.15.16:0/3044003467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda34072130 0x7fda340cf9d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fda2c009e50 tx=0x7fda2c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f19ca494ff8c95c server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:43.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.836+0000 7fda2bfff700 1 -- 172.21.15.16:0/3044003467 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda2c00e070 con 0x7fda34072130 2023-05-20T01:16:43.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.836+0000 7fda2bfff700 1 -- 172.21.15.16:0/3044003467 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda2c00b910 con 0x7fda34072130 2023-05-20T01:16:43.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.836+0000 7fda2bfff700 1 -- 172.21.15.16:0/3044003467 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda2c00bcc0 con 0x7fda34072130 2023-05-20T01:16:43.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.836+0000 7fda3b495700 1 -- 172.21.15.16:0/3044003467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda34072130 msgr2=0x7fda340cf9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:43.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.836+0000 7fda3b495700 1 --2- 172.21.15.16:0/3044003467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda34072130 0x7fda340cf9d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fda2c009e50 tx=0x7fda2c00b330 comp rx=0 tx=0).stop 2023-05-20T01:16:43.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.836+0000 7fda3b495700 1 -- 172.21.15.16:0/3044003467 shutdown_connections 2023-05-20T01:16:43.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.836+0000 7fda3b495700 1 --2- 172.21.15.16:0/3044003467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda34072130 0x7fda340cf9d0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:43.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.836+0000 7fda3b495700 1 --2- 172.21.15.16:0/3044003467 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fda3415cec0 0x7fda34071bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:43.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.836+0000 7fda3b495700 1 -- 172.21.15.16:0/3044003467 >> 172.21.15.16:0/3044003467 conn(0x7fda340cffd0 msgr2=0x7fda340d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:43.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.836+0000 7fda3b495700 1 -- 172.21.15.16:0/3044003467 shutdown_connections 2023-05-20T01:16:43.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.836+0000 7fda3b495700 1 -- 172.21.15.16:0/3044003467 wait complete. 2023-05-20T01:16:43.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.837+0000 7fda3b495700 1 Processor -- start 2023-05-20T01:16:43.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.837+0000 7fda3b495700 1 -- start start 2023-05-20T01:16:43.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.838+0000 7fda3b495700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fda34072130 0x7fda341f3130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:43.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.838+0000 7fda3b495700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda3415cec0 0x7fda34155640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:43.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.838+0000 7fda3b495700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fda3415d290 0x7fda34151ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:43.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.838+0000 7fda3b495700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda340c7750 con 0x7fda3415cec0 2023-05-20T01:16:43.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.838+0000 7fda3b495700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda34153db0 con 0x7fda3415d290 2023-05-20T01:16:43.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.838+0000 7fda3b495700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda341f3670 con 0x7fda34072130 2023-05-20T01:16:43.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.838+0000 7fda38a30700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda3415cec0 0x7fda34155640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:43.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.838+0000 7fda38a30700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda3415cec0 0x7fda34155640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57940/0 (socket says 172.21.15.16:57940) 2023-05-20T01:16:43.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.838+0000 7fda38a30700 1 -- 172.21.15.16:0/2530846484 learned_addr learned my addr 172.21.15.16:0/2530846484 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:43.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.838+0000 7fda39231700 1 --2- 172.21.15.16:0/2530846484 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fda34072130 0x7fda341f3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:43.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.838+0000 7fda39a32700 1 --2- 172.21.15.16:0/2530846484 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fda3415d290 0x7fda34151ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:43.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.839+0000 7fda38a30700 1 -- 172.21.15.16:0/2530846484 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fda34072130 msgr2=0x7fda341f3130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:43.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.839+0000 7fda38a30700 1 --2- 172.21.15.16:0/2530846484 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fda34072130 0x7fda341f3130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:43.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.839+0000 7fda38a30700 1 -- 172.21.15.16:0/2530846484 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fda3415d290 msgr2=0x7fda34151ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:43.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.839+0000 7fda38a30700 1 --2- 172.21.15.16:0/2530846484 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fda3415d290 0x7fda34151ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:43.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.839+0000 7fda38a30700 1 -- 172.21.15.16:0/2530846484 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda2c009b30 con 0x7fda3415cec0 2023-05-20T01:16:43.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.839+0000 7fda38a30700 1 --2- 172.21.15.16:0/2530846484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda3415cec0 0x7fda34155640 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fda2400c450 tx=0x7fda2400c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:43.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.839+0000 7fda29ffb700 1 -- 172.21.15.16:0/2530846484 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda240052d0 con 0x7fda3415cec0 2023-05-20T01:16:43.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.839+0000 7fda29ffb700 1 -- 172.21.15.16:0/2530846484 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda24005430 con 0x7fda3415cec0 2023-05-20T01:16:43.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.839+0000 7fda3b495700 1 -- 172.21.15.16:0/2530846484 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda340cd2c0 con 0x7fda3415cec0 2023-05-20T01:16:43.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.839+0000 7fda29ffb700 1 -- 172.21.15.16:0/2530846484 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda240038c0 con 0x7fda3415cec0 2023-05-20T01:16:43.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.839+0000 7fda3b495700 1 -- 172.21.15.16:0/2530846484 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda340ccfe0 con 0x7fda3415cec0 2023-05-20T01:16:43.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.840+0000 7fda3b495700 1 -- 172.21.15.16:0/2530846484 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda34160890 con 0x7fda3415cec0 2023-05-20T01:16:43.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.840+0000 7fda29ffb700 1 -- 172.21.15.16:0/2530846484 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fda24003a20 con 0x7fda3415cec0 2023-05-20T01:16:43.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.841+0000 7fda29ffb700 1 --2- 172.21.15.16:0/2530846484 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fda1806ace0 0x7fda1806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:43.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.841+0000 7fda29ffb700 1 -- 172.21.15.16:0/2530846484 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fda24088a00 con 0x7fda3415cec0 2023-05-20T01:16:43.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.841+0000 7fda39231700 1 --2- 172.21.15.16:0/2530846484 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fda1806ace0 0x7fda1806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:43.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.841+0000 7fda39231700 1 --2- 172.21.15.16:0/2530846484 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fda1806ace0 0x7fda1806d190 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fda30009bc0 tx=0x7fda300091c0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:43.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:43.845+0000 7fda29ffb700 1 -- 172.21.15.16:0/2530846484 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fda24053850 con 0x7fda3415cec0 2023-05-20T01:16:44.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:44.089+0000 7fda3b495700 1 -- 172.21.15.16:0/2530846484 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fda340c7890 con 0x7fda3415cec0 2023-05-20T01:16:44.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:44.360+0000 7fda29ffb700 1 -- 172.21.15.16:0/2530846484 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) v1 ==== 118+0+0 (secure 0 0 0) 0x7fda340c7890 con 0x7fda3415cec0 2023-05-20T01:16:44.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:44.362+0000 7fda3b495700 1 -- 172.21.15.16:0/2530846484 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fda1806ace0 msgr2=0x7fda1806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:44.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:44.362+0000 7fda3b495700 1 --2- 172.21.15.16:0/2530846484 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fda1806ace0 0x7fda1806d190 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fda30009bc0 tx=0x7fda300091c0 comp rx=0 tx=0).stop 2023-05-20T01:16:44.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:44.362+0000 7fda3b495700 1 -- 172.21.15.16:0/2530846484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda3415cec0 msgr2=0x7fda34155640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:44.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:44.362+0000 7fda3b495700 1 --2- 172.21.15.16:0/2530846484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda3415cec0 0x7fda34155640 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fda2400c450 tx=0x7fda2400c760 comp rx=0 tx=0).stop 2023-05-20T01:16:44.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:44.362+0000 7fda3b495700 1 -- 172.21.15.16:0/2530846484 shutdown_connections 2023-05-20T01:16:44.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:44.362+0000 7fda3b495700 1 --2- 172.21.15.16:0/2530846484 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fda3415d290 0x7fda34151ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:44.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:44.362+0000 7fda3b495700 1 --2- 172.21.15.16:0/2530846484 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fda1806ace0 0x7fda1806d190 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:44.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:44.362+0000 7fda3b495700 1 --2- 172.21.15.16:0/2530846484 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fda34072130 0x7fda341f3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:44.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:44.362+0000 7fda3b495700 1 --2- 172.21.15.16:0/2530846484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda3415cec0 0x7fda34155640 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:44.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:44.362+0000 7fda3b495700 1 -- 172.21.15.16:0/2530846484 >> 172.21.15.16:0/2530846484 conn(0x7fda340cffd0 msgr2=0x7fda3406a510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:44.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:44.362+0000 7fda3b495700 1 -- 172.21.15.16:0/2530846484 shutdown_connections 2023-05-20T01:16:44.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:44.362+0000 7fda3b495700 1 -- 172.21.15.16:0/2530846484 wait complete. 2023-05-20T01:16:44.378 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']} 2023-05-20T01:16:44.379 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-05-20T01:16:44.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:44.503+0000 7fc3eb470700 1 Processor -- start 2023-05-20T01:16:44.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:44.504+0000 7fc3eb470700 1 -- start start 2023-05-20T01:16:44.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:44.505+0000 7fc3eb470700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fc3e40d16e0 0x7fc3e4144350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:44.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:44.505+0000 7fc3eb470700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc3e40b19b0 0x7fc3e40cf0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:44.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:44.505+0000 7fc3eb470700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3e40b1df0 con 0x7fc3e40cec40 2023-05-20T01:16:44.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:44.505+0000 7fc3eb470700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3e40cf620 con 0x7fc3e40b19b0 2023-05-20T01:16:44.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:44.505+0000 7fc3eb470700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3e4144890 con 0x7fc3e40d16e0 2023-05-20T01:16:44.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:44.505+0000 7fc3e9a0d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc3e40b19b0 0x7fc3e40cf0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:44.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:44.505+0000 7fc3e9a0d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc3e40b19b0 0x7fc3e40cf0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57950/0 (socket says 172.21.15.16:57950) 2023-05-20T01:16:44.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:44.505+0000 7fc3e9a0d700 1 -- 172.21.15.16:0/127250009 learned_addr learned my addr 172.21.15.16:0/127250009 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:44.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:44.505+0000 7fc3e920c700 1 --2- 172.21.15.16:0/127250009 >> v2:172.21.15.132:3300/0 conn(0x7fc3e40d16e0 0x7fc3e4144350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:44.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:44.505+0000 7fc3e9a0d700 1 -- 172.21.15.16:0/127250009 >> v2:172.21.15.132:3300/0 conn(0x7fc3e40d16e0 msgr2=0x7fc3e4144350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:44.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:44.505+0000 7fc3e9a0d700 1 --2- 172.21.15.16:0/127250009 >> v2:172.21.15.132:3300/0 conn(0x7fc3e40d16e0 0x7fc3e4144350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:44.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:44.505+0000 7fc3e9a0d700 1 -- 172.21.15.16:0/127250009 >> v1:172.21.15.132:6789/0 conn(0x7fc3e40cec40 legacy=0x7fc3e40ae2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:44.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:44.505+0000 7fc3e9a0d700 1 -- 172.21.15.16:0/127250009 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3e40b2020 con 0x7fc3e40b19b0 2023-05-20T01:16:44.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:44.505+0000 7fc3e9a0d700 1 --2- 172.21.15.16:0/127250009 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3e40b19b0 0x7fc3e40cf0e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc009e50 tx=0x7fc3dc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=494a51fa376ff20a server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:44.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:44.506+0000 7fc3dbfff700 1 -- 172.21.15.16:0/127250009 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3dc00e070 con 0x7fc3e40b19b0 2023-05-20T01:16:44.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:44.506+0000 7fc3dbfff700 1 -- 172.21.15.16:0/127250009 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3dc00b980 con 0x7fc3e40b19b0 2023-05-20T01:16:44.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:44.506+0000 7fc3dbfff700 1 -- 172.21.15.16:0/127250009 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3dc00bd30 con 0x7fc3e40b19b0 2023-05-20T01:16:44.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:44.506+0000 7fc3eb470700 1 -- 172.21.15.16:0/127250009 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3e40b19b0 msgr2=0x7fc3e40cf0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:44.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:44.506+0000 7fc3eb470700 1 --2- 172.21.15.16:0/127250009 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3e40b19b0 0x7fc3e40cf0e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc009e50 tx=0x7fc3dc00b330 comp rx=0 tx=0).stop 2023-05-20T01:16:44.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:44.506+0000 7fc3eb470700 1 -- 172.21.15.16:0/127250009 shutdown_connections 2023-05-20T01:16:44.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:44.506+0000 7fc3eb470700 1 --2- 172.21.15.16:0/127250009 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3e40b19b0 0x7fc3e40cf0e0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:44.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:44.506+0000 7fc3eb470700 1 --2- 172.21.15.16:0/127250009 >> v2:172.21.15.132:3300/0 conn(0x7fc3e40d16e0 0x7fc3e4144350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:44.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:44.506+0000 7fc3eb470700 1 -- 172.21.15.16:0/127250009 >> 172.21.15.16:0/127250009 conn(0x7fc3e406ae30 msgr2=0x7fc3e409a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:44.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:44.506+0000 7fc3eb470700 1 -- 172.21.15.16:0/127250009 shutdown_connections 2023-05-20T01:16:44.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:44.506+0000 7fc3eb470700 1 -- 172.21.15.16:0/127250009 wait complete. 2023-05-20T01:16:44.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:44.507+0000 7fc3eb470700 1 Processor -- start 2023-05-20T01:16:44.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:44.507+0000 7fc3eb470700 1 -- start start 2023-05-20T01:16:44.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:44.508+0000 7fc3eb470700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3e40b19b0 0x7fc3e40b4860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:44.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:44.508+0000 7fc3eb470700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc3e40cec40 0x7fc3e40c66e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:44.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:44.508+0000 7fc3eb470700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc3e40d16e0 0x7fc3e40cd470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:44.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:44.508+0000 7fc3eb470700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3e40c71b0 con 0x7fc3e40b19b0 2023-05-20T01:16:44.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:44.508+0000 7fc3eb470700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3e40ccfe0 con 0x7fc3e40cec40 2023-05-20T01:16:44.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:44.508+0000 7fc3eb470700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3e40cc6f0 con 0x7fc3e40d16e0 2023-05-20T01:16:44.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:44.508+0000 7fc3e920c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3e40b19b0 0x7fc3e40b4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:44.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:44.508+0000 7fc3e920c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3e40b19b0 0x7fc3e40b4860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57952/0 (socket says 172.21.15.16:57952) 2023-05-20T01:16:44.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:44.508+0000 7fc3e920c700 1 -- 172.21.15.16:0/3304022207 learned_addr learned my addr 172.21.15.16:0/3304022207 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:44.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:44.508+0000 7fc3e8a0b700 1 --2- 172.21.15.16:0/3304022207 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc3e40cec40 0x7fc3e40c66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:44.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:44.508+0000 7fc3e9a0d700 1 --2- 172.21.15.16:0/3304022207 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc3e40d16e0 0x7fc3e40cd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:44.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:44.509+0000 7fc3e920c700 1 -- 172.21.15.16:0/3304022207 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc3e40d16e0 msgr2=0x7fc3e40cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:44.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:44.509+0000 7fc3e920c700 1 --2- 172.21.15.16:0/3304022207 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc3e40d16e0 0x7fc3e40cd470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:44.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:44.509+0000 7fc3e920c700 1 -- 172.21.15.16:0/3304022207 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc3e40cec40 msgr2=0x7fc3e40c66e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:44.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:44.509+0000 7fc3e920c700 1 --2- 172.21.15.16:0/3304022207 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc3e40cec40 0x7fc3e40c66e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:44.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:44.509+0000 7fc3e920c700 1 -- 172.21.15.16:0/3304022207 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3dc009b30 con 0x7fc3e40b19b0 2023-05-20T01:16:44.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:44.509+0000 7fc3e920c700 1 --2- 172.21.15.16:0/3304022207 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3e40b19b0 0x7fc3e40b4860 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc3d400e810 tx=0x7fc3d400eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:44.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:44.509+0000 7fc3d9ffb700 1 -- 172.21.15.16:0/3304022207 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3d400c070 con 0x7fc3e40b19b0 2023-05-20T01:16:44.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:44.509+0000 7fc3d9ffb700 1 -- 172.21.15.16:0/3304022207 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3d4013610 con 0x7fc3e40b19b0 2023-05-20T01:16:44.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:44.509+0000 7fc3eb470700 1 -- 172.21.15.16:0/3304022207 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3e40b25e0 con 0x7fc3e40b19b0 2023-05-20T01:16:44.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:44.509+0000 7fc3d9ffb700 1 -- 172.21.15.16:0/3304022207 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3d40139c0 con 0x7fc3e40b19b0 2023-05-20T01:16:44.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:44.509+0000 7fc3eb470700 1 -- 172.21.15.16:0/3304022207 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3e40b2300 con 0x7fc3e40b19b0 2023-05-20T01:16:44.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:44.510+0000 7fc3eb470700 1 -- 172.21.15.16:0/3304022207 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3e40addc0 con 0x7fc3e40b19b0 2023-05-20T01:16:44.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:44.510+0000 7fc3d9ffb700 1 -- 172.21.15.16:0/3304022207 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc3d4013b20 con 0x7fc3e40b19b0 2023-05-20T01:16:44.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:44.511+0000 7fc3d9ffb700 1 --2- 172.21.15.16:0/3304022207 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc3c806ace0 0x7fc3c806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:44.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:44.511+0000 7fc3e8a0b700 1 --2- 172.21.15.16:0/3304022207 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc3c806ace0 0x7fc3c806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:44.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:44.511+0000 7fc3d9ffb700 1 -- 172.21.15.16:0/3304022207 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fc3d4089460 con 0x7fc3e40b19b0 2023-05-20T01:16:44.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:44.511+0000 7fc3e8a0b700 1 --2- 172.21.15.16:0/3304022207 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc3c806ace0 0x7fc3c806d190 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc3e40b28f0 tx=0x7fc3e0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:44.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:44.516+0000 7fc3d9ffb700 1 -- 172.21.15.16:0/3304022207 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fc3d4057cc0 con 0x7fc3e40b19b0 2023-05-20T01:16:44.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:44.757+0000 7fc3eb470700 1 -- 172.21.15.16:0/3304022207 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fc3e40cd2c0 con 0x7fc3e40b19b0 2023-05-20T01:16:45.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.367+0000 7fc3d9ffb700 1 -- 172.21.15.16:0/3304022207 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7fc3d4057850 con 0x7fc3e40b19b0 2023-05-20T01:16:45.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.370+0000 7fc3eb470700 1 -- 172.21.15.16:0/3304022207 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc3c806ace0 msgr2=0x7fc3c806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:45.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.370+0000 7fc3eb470700 1 --2- 172.21.15.16:0/3304022207 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc3c806ace0 0x7fc3c806d190 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc3e40b28f0 tx=0x7fc3e0009040 comp rx=0 tx=0).stop 2023-05-20T01:16:45.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.370+0000 7fc3eb470700 1 -- 172.21.15.16:0/3304022207 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3e40b19b0 msgr2=0x7fc3e40b4860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:45.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.370+0000 7fc3eb470700 1 --2- 172.21.15.16:0/3304022207 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3e40b19b0 0x7fc3e40b4860 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc3d400e810 tx=0x7fc3d400eb20 comp rx=0 tx=0).stop 2023-05-20T01:16:45.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.370+0000 7fc3eb470700 1 -- 172.21.15.16:0/3304022207 shutdown_connections 2023-05-20T01:16:45.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.370+0000 7fc3eb470700 1 --2- 172.21.15.16:0/3304022207 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc3c806ace0 0x7fc3c806d190 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:45.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.370+0000 7fc3eb470700 1 --2- 172.21.15.16:0/3304022207 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc3e40d16e0 0x7fc3e40cd470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:45.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.370+0000 7fc3eb470700 1 --2- 172.21.15.16:0/3304022207 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3e40b19b0 0x7fc3e40b4860 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:45.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.370+0000 7fc3eb470700 1 --2- 172.21.15.16:0/3304022207 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc3e40cec40 0x7fc3e40c66e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:45.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.370+0000 7fc3eb470700 1 -- 172.21.15.16:0/3304022207 >> 172.21.15.16:0/3304022207 conn(0x7fc3e406ae30 msgr2=0x7fc3e406e3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:45.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.370+0000 7fc3eb470700 1 -- 172.21.15.16:0/3304022207 shutdown_connections 2023-05-20T01:16:45.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.370+0000 7fc3eb470700 1 -- 172.21.15.16:0/3304022207 wait complete. 2023-05-20T01:16:45.389 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-05-20T01:16:45.389 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-20T01:16:45.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.518+0000 7f2278622700 1 Processor -- start 2023-05-20T01:16:45.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.518+0000 7f2278622700 1 -- start start 2023-05-20T01:16:45.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.519+0000 7f2278622700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f227006cb90 0x7f227015ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:45.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.519+0000 7f2278622700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f22700b54f0 0x7f22700b29e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:45.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.519+0000 7f2278622700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22700b2f20 con 0x7f227006cf60 2023-05-20T01:16:45.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.519+0000 7f2278622700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22700cc990 con 0x7f227006cb90 2023-05-20T01:16:45.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.519+0000 7f2278622700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22700b31a0 con 0x7f22700b54f0 2023-05-20T01:16:45.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.519+0000 7f22763be700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f227006cb90 0x7f227015ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:45.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.519+0000 7f22763be700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f227006cb90 0x7f227015ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57960/0 (socket says 172.21.15.16:57960) 2023-05-20T01:16:45.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.519+0000 7f22763be700 1 -- 172.21.15.16:0/4103075631 learned_addr learned my addr 172.21.15.16:0/4103075631 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:45.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.519+0000 7f2275bbd700 1 --2- 172.21.15.16:0/4103075631 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f22700b54f0 0x7f22700b29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:45.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.519+0000 7f22763be700 1 -- 172.21.15.16:0/4103075631 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f22700b54f0 msgr2=0x7f22700b29e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:45.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.519+0000 7f22763be700 1 --2- 172.21.15.16:0/4103075631 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f22700b54f0 0x7f22700b29e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:45.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.519+0000 7f22763be700 1 -- 172.21.15.16:0/4103075631 >> v1:172.21.15.132:6789/0 conn(0x7f227006cf60 legacy=0x7f2270070910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:45.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.519+0000 7f22763be700 1 -- 172.21.15.16:0/4103075631 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22700b3480 con 0x7f227006cb90 2023-05-20T01:16:45.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.519+0000 7f22763be700 1 --2- 172.21.15.16:0/4103075631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f227006cb90 0x7f227015ba40 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f226c0098f0 tx=0x7f226c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5ce8a667e89e0ee server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:45.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.519+0000 7f22753bc700 1 -- 172.21.15.16:0/4103075631 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f226c011830 con 0x7f227006cb90 2023-05-20T01:16:45.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.520+0000 7f22753bc700 1 -- 172.21.15.16:0/4103075631 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f226c011990 con 0x7f227006cb90 2023-05-20T01:16:45.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.520+0000 7f22753bc700 1 -- 172.21.15.16:0/4103075631 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f226c011d20 con 0x7f227006cb90 2023-05-20T01:16:45.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.520+0000 7f2278622700 1 -- 172.21.15.16:0/4103075631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f227006cb90 msgr2=0x7f227015ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:45.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.520+0000 7f2278622700 1 --2- 172.21.15.16:0/4103075631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f227006cb90 0x7f227015ba40 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f226c0098f0 tx=0x7f226c009c00 comp rx=0 tx=0).stop 2023-05-20T01:16:45.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.520+0000 7f2278622700 1 -- 172.21.15.16:0/4103075631 shutdown_connections 2023-05-20T01:16:45.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.520+0000 7f2278622700 1 --2- 172.21.15.16:0/4103075631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f227006cb90 0x7f227015ba40 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:45.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.520+0000 7f2278622700 1 --2- 172.21.15.16:0/4103075631 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f22700b54f0 0x7f22700b29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:45.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.520+0000 7f2278622700 1 -- 172.21.15.16:0/4103075631 >> 172.21.15.16:0/4103075631 conn(0x7f22700661b0 msgr2=0x7f2270066580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:45.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.520+0000 7f2278622700 1 -- 172.21.15.16:0/4103075631 shutdown_connections 2023-05-20T01:16:45.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.520+0000 7f2278622700 1 -- 172.21.15.16:0/4103075631 wait complete. 2023-05-20T01:16:45.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.521+0000 7f2278622700 1 Processor -- start 2023-05-20T01:16:45.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.521+0000 7f2278622700 1 -- start start 2023-05-20T01:16:45.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.522+0000 7f2278622700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f227006cb90 0x7f22700caba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:45.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.522+0000 7f2278622700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f227006cf60 0x7f22701ef520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:45.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.522+0000 7f2278622700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22700b54f0 0x7f22700cb610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:45.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.522+0000 7f2278622700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22700ca780 con 0x7f22700b54f0 2023-05-20T01:16:45.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.522+0000 7f2278622700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22700cbd10 con 0x7f227006cb90 2023-05-20T01:16:45.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.522+0000 7f2278622700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22700c9de0 con 0x7f227006cf60 2023-05-20T01:16:45.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.522+0000 7f2276bbf700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22700b54f0 0x7f22700cb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:45.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.522+0000 7f2276bbf700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22700b54f0 0x7f22700cb610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57970/0 (socket says 172.21.15.16:57970) 2023-05-20T01:16:45.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.522+0000 7f2276bbf700 1 -- 172.21.15.16:0/2062923097 learned_addr learned my addr 172.21.15.16:0/2062923097 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:45.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.522+0000 7f22763be700 1 --2- 172.21.15.16:0/2062923097 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f227006cb90 0x7f22700caba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:45.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.522+0000 7f2275bbd700 1 --2- 172.21.15.16:0/2062923097 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f227006cf60 0x7f22701ef520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:45.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.523+0000 7f2276bbf700 1 -- 172.21.15.16:0/2062923097 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f227006cf60 msgr2=0x7f22701ef520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:45.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.523+0000 7f2276bbf700 1 --2- 172.21.15.16:0/2062923097 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f227006cf60 0x7f22701ef520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:45.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.523+0000 7f2276bbf700 1 -- 172.21.15.16:0/2062923097 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f227006cb90 msgr2=0x7f22700caba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:45.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.523+0000 7f2276bbf700 1 --2- 172.21.15.16:0/2062923097 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f227006cb90 0x7f22700caba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:45.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.523+0000 7f2276bbf700 1 -- 172.21.15.16:0/2062923097 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f226c009550 con 0x7f22700b54f0 2023-05-20T01:16:45.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.523+0000 7f2276bbf700 1 --2- 172.21.15.16:0/2062923097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22700b54f0 0x7f22700cb610 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f226800cb50 tx=0x7f226800ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:45.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.523+0000 7f2266ffd700 1 -- 172.21.15.16:0/2062923097 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2268003c00 con 0x7f22700b54f0 2023-05-20T01:16:45.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.523+0000 7f2266ffd700 1 -- 172.21.15.16:0/2062923097 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2268003d60 con 0x7f22700b54f0 2023-05-20T01:16:45.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.523+0000 7f2266ffd700 1 -- 172.21.15.16:0/2062923097 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f226800aa20 con 0x7f22700b54f0 2023-05-20T01:16:45.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.523+0000 7f2278622700 1 -- 172.21.15.16:0/2062923097 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f227015b240 con 0x7f22700b54f0 2023-05-20T01:16:45.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.523+0000 7f2278622700 1 -- 172.21.15.16:0/2062923097 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f227015af60 con 0x7f22700b54f0 2023-05-20T01:16:45.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.524+0000 7f2278622700 1 -- 172.21.15.16:0/2062923097 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f227015b520 con 0x7f22700b54f0 2023-05-20T01:16:45.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.524+0000 7f2266ffd700 1 -- 172.21.15.16:0/2062923097 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f226800ab80 con 0x7f22700b54f0 2023-05-20T01:16:45.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.525+0000 7f2266ffd700 1 --2- 172.21.15.16:0/2062923097 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f225406ace0 0x7f225406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:45.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.525+0000 7f2266ffd700 1 -- 172.21.15.16:0/2062923097 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f2268089660 con 0x7f22700b54f0 2023-05-20T01:16:45.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.525+0000 7f22763be700 1 --2- 172.21.15.16:0/2062923097 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f225406ace0 0x7f225406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:45.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.525+0000 7f22763be700 1 --2- 172.21.15.16:0/2062923097 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f225406ace0 0x7f225406d190 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f226c014040 tx=0x7f226c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:45.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.529+0000 7f2266ffd700 1 -- 172.21.15.16:0/2062923097 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f22680544b0 con 0x7f22700b54f0 2023-05-20T01:16:45.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.770+0000 7f2278622700 1 -- 172.21.15.16:0/2062923097 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f22700747b0 con 0x7f22700b54f0 2023-05-20T01:16:45.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.771+0000 7f2266ffd700 1 -- 172.21.15.16:0/2062923097 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4673 (secure 0 0 0) 0x7f2268057ad0 con 0x7f22700b54f0 2023-05-20T01:16:45.775 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:16:45.776 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4273,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/1444124146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1444124146},{"type":"v1","addr":"172.21.15.132:6834","nonce":1444124146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4293,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4092119206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4092119206},{"type":"v1","addr":"172.21.15.16:6835","nonce":4092119206}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2162680372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":2162680372},{"type":"v1","addr":"172.21.15.132:6835","nonce":2162680372}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-20T01:16:28.316109+0000","modified":"2023-05-20T01:16:45.362572+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4299},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4299":{"gid":4299,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6837/2330551626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2330551626},{"type":"v1","addr":"172.21.15.16:6837","nonce":2330551626}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-05-20T01:16:45.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.773+0000 7f2278622700 1 -- 172.21.15.16:0/2062923097 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f225406ace0 msgr2=0x7f225406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:45.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.773+0000 7f2278622700 1 --2- 172.21.15.16:0/2062923097 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f225406ace0 0x7f225406d190 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f226c014040 tx=0x7f226c008030 comp rx=0 tx=0).stop 2023-05-20T01:16:45.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.773+0000 7f2278622700 1 -- 172.21.15.16:0/2062923097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22700b54f0 msgr2=0x7f22700cb610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:45.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.773+0000 7f2278622700 1 --2- 172.21.15.16:0/2062923097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22700b54f0 0x7f22700cb610 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f226800cb50 tx=0x7f226800ce60 comp rx=0 tx=0).stop 2023-05-20T01:16:45.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.773+0000 7f2278622700 1 -- 172.21.15.16:0/2062923097 shutdown_connections 2023-05-20T01:16:45.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.773+0000 7f2278622700 1 --2- 172.21.15.16:0/2062923097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22700b54f0 0x7f22700cb610 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:45.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.773+0000 7f2278622700 1 --2- 172.21.15.16:0/2062923097 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f227006cb90 0x7f22700caba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:45.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.773+0000 7f2278622700 1 --2- 172.21.15.16:0/2062923097 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f225406ace0 0x7f225406d190 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:45.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.773+0000 7f2278622700 1 --2- 172.21.15.16:0/2062923097 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f227006cf60 0x7f22701ef520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:45.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.773+0000 7f2278622700 1 -- 172.21.15.16:0/2062923097 >> 172.21.15.16:0/2062923097 conn(0x7f22700661b0 msgr2=0x7f2270154b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:45.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.773+0000 7f2278622700 1 -- 172.21.15.16:0/2062923097 shutdown_connections 2023-05-20T01:16:45.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.773+0000 7f2278622700 1 -- 172.21.15.16:0/2062923097 wait complete. 2023-05-20T01:16:45.779 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 8 2023-05-20T01:16:45.789 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-20T01:16:45.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.911+0000 7f48649da700 1 Processor -- start 2023-05-20T01:16:45.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.911+0000 7f48649da700 1 -- start start 2023-05-20T01:16:45.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.912+0000 7f48649da700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f48600cc9c0 0x7f486006a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:45.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.912+0000 7f48649da700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48600ccf40 con 0x7f4860072170 2023-05-20T01:16:45.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.912+0000 7f48649da700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48600cd220 con 0x7f48601585d0 2023-05-20T01:16:45.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.912+0000 7f48649da700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f486015e540 con 0x7f48600cc9c0 2023-05-20T01:16:45.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.913+0000 7f485ed9d700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f4860072170 0x7f486015e990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:41272/0 (socket says 172.21.15.16:41272) 2023-05-20T01:16:45.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.913+0000 7f485ed9d700 1 -- 172.21.15.16:0/3386520996 learned_addr learned my addr 172.21.15.16:0/3386520996 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:16:45.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.913+0000 7f485dd9b700 1 --2- 172.21.15.16:0/3386520996 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f48600cc9c0 0x7f486006a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:45.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.913+0000 7f485d59a700 1 -- 172.21.15.16:0/3386520996 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3559215935 0 0) 0x7f48600ccf40 con 0x7f4860072170 2023-05-20T01:16:45.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.913+0000 7f485d59a700 1 -- 172.21.15.16:0/3386520996 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48480035b0 con 0x7f4860072170 2023-05-20T01:16:45.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.913+0000 7f485d59a700 1 -- 172.21.15.16:0/3386520996 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2650078945 0 0) 0x7f48600cd220 con 0x7f48601585d0 2023-05-20T01:16:45.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.913+0000 7f485d59a700 1 -- 172.21.15.16:0/3386520996 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48600ccf40 con 0x7f48601585d0 2023-05-20T01:16:45.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.913+0000 7f485dd9b700 1 -- 172.21.15.16:0/3386520996 >> v1:172.21.15.132:6789/0 conn(0x7f48601585d0 legacy=0x7f486015ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:45.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.913+0000 7f485dd9b700 1 -- 172.21.15.16:0/3386520996 >> v1:172.21.15.16:6789/0 conn(0x7f4860072170 legacy=0x7f486015e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:45.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.913+0000 7f485dd9b700 1 -- 172.21.15.16:0/3386520996 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48600cd500 con 0x7f48600cc9c0 2023-05-20T01:16:45.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.913+0000 7f485dd9b700 1 --2- 172.21.15.16:0/3386520996 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f48600cc9c0 0x7f486006a510 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f4854009870 tx=0x7f4854009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e60948a8bee0f84d server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:45.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.914+0000 7f485d59a700 1 -- 172.21.15.16:0/3386520996 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4854011830 con 0x7f48600cc9c0 2023-05-20T01:16:45.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.914+0000 7f485d59a700 1 -- 172.21.15.16:0/3386520996 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4854011990 con 0x7f48600cc9c0 2023-05-20T01:16:45.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.914+0000 7f485d59a700 1 -- 172.21.15.16:0/3386520996 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4854011d00 con 0x7f48600cc9c0 2023-05-20T01:16:45.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.914+0000 7f48649da700 1 -- 172.21.15.16:0/3386520996 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f48600cc9c0 msgr2=0x7f486006a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:45.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.914+0000 7f48649da700 1 --2- 172.21.15.16:0/3386520996 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f48600cc9c0 0x7f486006a510 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f4854009870 tx=0x7f4854009b80 comp rx=0 tx=0).stop 2023-05-20T01:16:45.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.914+0000 7f48649da700 1 -- 172.21.15.16:0/3386520996 shutdown_connections 2023-05-20T01:16:45.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.914+0000 7f48649da700 1 --2- 172.21.15.16:0/3386520996 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f48600cc9c0 0x7f486006a510 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:45.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.914+0000 7f48649da700 1 -- 172.21.15.16:0/3386520996 >> 172.21.15.16:0/3386520996 conn(0x7f48600cffb0 msgr2=0x7f48600d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:45.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.914+0000 7f48649da700 1 -- 172.21.15.16:0/3386520996 shutdown_connections 2023-05-20T01:16:45.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.914+0000 7f48649da700 1 -- 172.21.15.16:0/3386520996 wait complete. 2023-05-20T01:16:45.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.916+0000 7f48649da700 1 Processor -- start 2023-05-20T01:16:45.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.916+0000 7f48649da700 1 -- start start 2023-05-20T01:16:45.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.916+0000 7f48649da700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4860072170 0x7f4860155830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:45.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.916+0000 7f48649da700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48600cc9c0 0x7f48601f2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:45.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.916+0000 7f48649da700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f48601585d0 0x7f486015c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:45.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.916+0000 7f48649da700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48600c7490 con 0x7f48600cc9c0 2023-05-20T01:16:45.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.916+0000 7f48649da700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48601f2d80 con 0x7f4860072170 2023-05-20T01:16:45.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.916+0000 7f48649da700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48601f3060 con 0x7f48601585d0 2023-05-20T01:16:45.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.916+0000 7f485dd9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48600cc9c0 0x7f48601f2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:45.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.917+0000 7f485dd9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48600cc9c0 0x7f48601f2410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57982/0 (socket says 172.21.15.16:57982) 2023-05-20T01:16:45.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.917+0000 7f485dd9b700 1 -- 172.21.15.16:0/1555530480 learned_addr learned my addr 172.21.15.16:0/1555530480 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:45.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.917+0000 7f485e59c700 1 --2- 172.21.15.16:0/1555530480 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4860072170 0x7f4860155830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:45.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.917+0000 7f485ed9d700 1 --2- 172.21.15.16:0/1555530480 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f48601585d0 0x7f486015c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:45.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.917+0000 7f485dd9b700 1 -- 172.21.15.16:0/1555530480 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f48601585d0 msgr2=0x7f486015c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:45.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.917+0000 7f485dd9b700 1 --2- 172.21.15.16:0/1555530480 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f48601585d0 0x7f486015c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:45.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.917+0000 7f485dd9b700 1 -- 172.21.15.16:0/1555530480 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4860072170 msgr2=0x7f4860155830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:45.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.917+0000 7f485dd9b700 1 --2- 172.21.15.16:0/1555530480 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4860072170 0x7f4860155830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:45.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.917+0000 7f485dd9b700 1 -- 172.21.15.16:0/1555530480 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48540094a0 con 0x7f48600cc9c0 2023-05-20T01:16:45.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.917+0000 7f485e59c700 1 --2- 172.21.15.16:0/1555530480 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4860072170 0x7f4860155830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:16:45.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.917+0000 7f485dd9b700 1 --2- 172.21.15.16:0/1555530480 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48600cc9c0 0x7f48601f2410 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f48540097c0 tx=0x7f4854004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:45.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.917+0000 7f484effd700 1 -- 172.21.15.16:0/1555530480 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48540045d0 con 0x7f48600cc9c0 2023-05-20T01:16:45.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.917+0000 7f484effd700 1 -- 172.21.15.16:0/1555530480 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f485401f070 con 0x7f48600cc9c0 2023-05-20T01:16:45.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.917+0000 7f484effd700 1 -- 172.21.15.16:0/1555530480 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48540038c0 con 0x7f48600cc9c0 2023-05-20T01:16:45.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.917+0000 7f48649da700 1 -- 172.21.15.16:0/1555530480 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48600719a0 con 0x7f48600cc9c0 2023-05-20T01:16:45.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.917+0000 7f48649da700 1 -- 172.21.15.16:0/1555530480 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48600cd7e0 con 0x7f48600cc9c0 2023-05-20T01:16:45.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.918+0000 7f48649da700 1 -- 172.21.15.16:0/1555530480 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f486015b530 con 0x7f48600cc9c0 2023-05-20T01:16:45.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.918+0000 7f484effd700 1 -- 172.21.15.16:0/1555530480 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4854003a20 con 0x7f48600cc9c0 2023-05-20T01:16:45.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.919+0000 7f484effd700 1 --2- 172.21.15.16:0/1555530480 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f484006ace0 0x7f484006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:45.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.919+0000 7f484effd700 1 -- 172.21.15.16:0/1555530480 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f4854021070 con 0x7f48600cc9c0 2023-05-20T01:16:45.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.919+0000 7f485e59c700 1 --2- 172.21.15.16:0/1555530480 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f484006ace0 0x7f484006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:45.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.919+0000 7f485e59c700 1 --2- 172.21.15.16:0/1555530480 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f484006ace0 0x7f484006d190 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f48500017b0 tx=0x7f4850009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:45.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:45.923+0000 7f484effd700 1 -- 172.21.15.16:0/1555530480 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f4854055e80 con 0x7f48600cc9c0 2023-05-20T01:16:46.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.123+0000 7f48649da700 1 -- 172.21.15.16:0/1555530480 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f48600747b0 con 0x7f48600cc9c0 2023-05-20T01:16:46.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.124+0000 7f484effd700 1 -- 172.21.15.16:0/1555530480 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+20026 (secure 0 0 0) 0x7f48540594a0 con 0x7f48600cc9c0 2023-05-20T01:16:46.128 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:16:46.128 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"fsid":"c2245cc5-ecab-4087-a59b-bc81db901676","created":"2023-05-20T01:16:09.549230+0000","modified":"2023-05-20T01:16:32.768495+0000","last_up_change":"2023-05-20T01:16:19.788624+0000","last_in_change":"2023-05-20T01:16:12.092160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-05-20T01:16:20.934366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-05-20T01:16:21.372294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-05-20T01:16:26.208287+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-05-20T01:16:27.292268+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-05-20T01:16:30.357285+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"317a83cd-19c8-4958-a301-6087ef342289","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6825","nonce":3923634179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6827","nonce":3923634179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6831","nonce":3923634179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6829","nonce":3923634179}]},"public_addr":"172.21.15.16:6825/3923634179","cluster_addr":"172.21.15.16:6827/3923634179","heartbeat_back_addr":"172.21.15.16:6831/3923634179","heartbeat_front_addr":"172.21.15.16:6829/3923634179","state":["exists","up"]},{"osd":1,"uuid":"5581046f-8219-4089-8d7d-b7846399505e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6809","nonce":3860264220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6811","nonce":3860264220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6815","nonce":3860264220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6813","nonce":3860264220}]},"public_addr":"172.21.15.16:6809/3860264220","cluster_addr":"172.21.15.16:6811/3860264220","heartbeat_back_addr":"172.21.15.16:6815/3860264220","heartbeat_front_addr":"172.21.15.16:6813/3860264220","state":["exists","up"]},{"osd":2,"uuid":"072e9e66-4133-478e-8d4a-64d46c2963f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6817","nonce":293187030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6819","nonce":293187030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6823","nonce":293187030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6821","nonce":293187030}]},"public_addr":"172.21.15.16:6817/293187030","cluster_addr":"172.21.15.16:6819/293187030","heartbeat_back_addr":"172.21.15.16:6823/293187030","heartbeat_front_addr":"172.21.15.16:6821/293187030","state":["exists","up"]},{"osd":3,"uuid":"11e1bba6-00f8-48dd-9028-d8d716b7f145","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6801","nonce":3669781890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6803","nonce":3669781890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6807","nonce":3669781890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6805","nonce":3669781890}]},"public_addr":"172.21.15.16:6801/3669781890","cluster_addr":"172.21.15.16:6803/3669781890","heartbeat_back_addr":"172.21.15.16:6807/3669781890","heartbeat_front_addr":"172.21.15.16:6805/3669781890","state":["exists","up"]},{"osd":4,"uuid":"b8909c3a-cf70-46cb-8d0b-009c707cf9d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6819","nonce":506317223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6823","nonce":506317223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6829","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6831","nonce":506317223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6827","nonce":506317223}]},"public_addr":"172.21.15.132:6819/506317223","cluster_addr":"172.21.15.132:6823/506317223","heartbeat_back_addr":"172.21.15.132:6831/506317223","heartbeat_front_addr":"172.21.15.132:6827/506317223","state":["exists","up"]},{"osd":5,"uuid":"b1818304-4583-47bf-a6ec-0bfbb5166233","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6818","nonce":1844689995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6822","nonce":1844689995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6830","nonce":1844689995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6826","nonce":1844689995}]},"public_addr":"172.21.15.132:6818/1844689995","cluster_addr":"172.21.15.132:6822/1844689995","heartbeat_back_addr":"172.21.15.132:6830/1844689995","heartbeat_front_addr":"172.21.15.132:6826/1844689995","state":["exists","up"]},{"osd":6,"uuid":"d37f4691-eed1-4a93-8818-18354f03f4a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6801","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6802","nonce":727581123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6805","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6807","nonce":727581123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6813","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6815","nonce":727581123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6811","nonce":727581123}]},"public_addr":"172.21.15.132:6802/727581123","cluster_addr":"172.21.15.132:6807/727581123","heartbeat_back_addr":"172.21.15.132:6815/727581123","heartbeat_front_addr":"172.21.15.132:6811/727581123","state":["exists","up"]},{"osd":7,"uuid":"4f712f62-5d1d-4816-8042-03209144f8b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6803","nonce":466001149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6806","nonce":466001149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6814","nonce":466001149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6810","nonce":466001149}]},"public_addr":"172.21.15.132:6803/466001149","cluster_addr":"172.21.15.132:6806/466001149","heartbeat_back_addr":"172.21.15.132:6814/466001149","heartbeat_front_addr":"172.21.15.132:6810/466001149","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.338863+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.278680+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305694+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.246835+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.276237+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.304441+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305421+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.267050+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-20T01:16:46.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.126+0000 7f48649da700 1 -- 172.21.15.16:0/1555530480 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f484006ace0 msgr2=0x7f484006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:46.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.126+0000 7f48649da700 1 --2- 172.21.15.16:0/1555530480 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f484006ace0 0x7f484006d190 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f48500017b0 tx=0x7f4850009040 comp rx=0 tx=0).stop 2023-05-20T01:16:46.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.126+0000 7f48649da700 1 -- 172.21.15.16:0/1555530480 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48600cc9c0 msgr2=0x7f48601f2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:46.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.126+0000 7f48649da700 1 --2- 172.21.15.16:0/1555530480 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48600cc9c0 0x7f48601f2410 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f48540097c0 tx=0x7f4854004750 comp rx=0 tx=0).stop 2023-05-20T01:16:46.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.126+0000 7f48649da700 1 -- 172.21.15.16:0/1555530480 shutdown_connections 2023-05-20T01:16:46.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.126+0000 7f48649da700 1 --2- 172.21.15.16:0/1555530480 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f484006ace0 0x7f484006d190 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:46.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.126+0000 7f48649da700 1 --2- 172.21.15.16:0/1555530480 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f48601585d0 0x7f486015c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:46.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.126+0000 7f48649da700 1 --2- 172.21.15.16:0/1555530480 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4860072170 0x7f4860155830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:46.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.126+0000 7f48649da700 1 --2- 172.21.15.16:0/1555530480 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48600cc9c0 0x7f48601f2410 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:46.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.126+0000 7f48649da700 1 -- 172.21.15.16:0/1555530480 >> 172.21.15.16:0/1555530480 conn(0x7f48600cffb0 msgr2=0x7f486006a510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:46.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.126+0000 7f48649da700 1 -- 172.21.15.16:0/1555530480 shutdown_connections 2023-05-20T01:16:46.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.126+0000 7f48649da700 1 -- 172.21.15.16:0/1555530480 wait complete. 2023-05-20T01:16:46.142 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-05-20T01:16:46.143 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-05-20T01:16:46.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.264+0000 7f26aa4fc700 1 Processor -- start 2023-05-20T01:16:46.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.264+0000 7f26aa4fc700 1 -- start start 2023-05-20T01:16:46.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.265+0000 7f26aa4fc700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f26a4161190 0x7f26a406de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:46.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.265+0000 7f26aa4fc700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f26a406ac20 0x7f26a4071140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:46.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.265+0000 7f26aa4fc700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26a415d020 con 0x7f26a406e370 2023-05-20T01:16:46.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.265+0000 7f26aa4fc700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26a406e7c0 con 0x7f26a406ac20 2023-05-20T01:16:46.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.265+0000 7f26aa4fc700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26a40cebf0 con 0x7f26a4161190 2023-05-20T01:16:46.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.265+0000 7f26a3fff700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f26a4161190 0x7f26a406de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:46.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.265+0000 7f26a8a99700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f26a406ac20 0x7f26a4071140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:46.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.265+0000 7f26a3fff700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f26a4161190 0x7f26a406de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.16:58026/0 (socket says 172.21.15.16:58026) 2023-05-20T01:16:46.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.265+0000 7f26a3fff700 1 -- 172.21.15.16:0/3679776128 learned_addr learned my addr 172.21.15.16:0/3679776128 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:46.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.266+0000 7f26a3fff700 1 -- 172.21.15.16:0/3679776128 >> v2:172.21.15.132:3300/0 conn(0x7f26a406ac20 msgr2=0x7f26a4071140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:46.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.266+0000 7f26a3fff700 1 --2- 172.21.15.16:0/3679776128 >> v2:172.21.15.132:3300/0 conn(0x7f26a406ac20 0x7f26a4071140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:46.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.266+0000 7f26a3fff700 1 -- 172.21.15.16:0/3679776128 >> v1:172.21.15.132:6789/0 conn(0x7f26a406e370 legacy=0x7f26a406a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:46.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.266+0000 7f26a3fff700 1 -- 172.21.15.16:0/3679776128 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26a406e9a0 con 0x7f26a4161190 2023-05-20T01:16:46.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.266+0000 7f26a3fff700 1 --2- 172.21.15.16:0/3679776128 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f26a4161190 0x7f26a406de30 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f26940098f0 tx=0x7f2694009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a457bff068bae16 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:46.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.266+0000 7f26a2ffd700 1 -- 172.21.15.16:0/3679776128 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2694011830 con 0x7f26a4161190 2023-05-20T01:16:46.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.266+0000 7f26a2ffd700 1 -- 172.21.15.16:0/3679776128 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2694011990 con 0x7f26a4161190 2023-05-20T01:16:46.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.266+0000 7f26a2ffd700 1 -- 172.21.15.16:0/3679776128 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2694011d20 con 0x7f26a4161190 2023-05-20T01:16:46.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.267+0000 7f26aa4fc700 1 -- 172.21.15.16:0/3679776128 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f26a4161190 msgr2=0x7f26a406de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:46.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.267+0000 7f26aa4fc700 1 --2- 172.21.15.16:0/3679776128 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f26a4161190 0x7f26a406de30 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f26940098f0 tx=0x7f2694009c00 comp rx=0 tx=0).stop 2023-05-20T01:16:46.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.267+0000 7f26aa4fc700 1 -- 172.21.15.16:0/3679776128 shutdown_connections 2023-05-20T01:16:46.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.267+0000 7f26aa4fc700 1 --2- 172.21.15.16:0/3679776128 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f26a4161190 0x7f26a406de30 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:46.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.267+0000 7f26aa4fc700 1 --2- 172.21.15.16:0/3679776128 >> v2:172.21.15.132:3300/0 conn(0x7f26a406ac20 0x7f26a4071140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:46.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.267+0000 7f26aa4fc700 1 -- 172.21.15.16:0/3679776128 >> 172.21.15.16:0/3679776128 conn(0x7f26a40d0c70 msgr2=0x7f26a40b1af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:46.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.267+0000 7f26aa4fc700 1 -- 172.21.15.16:0/3679776128 shutdown_connections 2023-05-20T01:16:46.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.267+0000 7f26aa4fc700 1 -- 172.21.15.16:0/3679776128 wait complete. 2023-05-20T01:16:46.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.268+0000 7f26aa4fc700 1 Processor -- start 2023-05-20T01:16:46.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.268+0000 7f26aa4fc700 1 -- start start 2023-05-20T01:16:46.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.269+0000 7f26aa4fc700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f26a406ac20 0x7f26a41496c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:46.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.269+0000 7f26aa4fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26a406e370 0x7f26a40c5c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:46.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.269+0000 7f26aa4fc700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f26a4161190 0x7f26a40c61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:46.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.269+0000 7f26aa4fc700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26a41e7c00 con 0x7f26a406e370 2023-05-20T01:16:46.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.269+0000 7f26aa4fc700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26a41e78b0 con 0x7f26a4161190 2023-05-20T01:16:46.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.269+0000 7f26aa4fc700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26a4149c00 con 0x7f26a406ac20 2023-05-20T01:16:46.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.269+0000 7f26a37fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26a406e370 0x7f26a40c5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:46.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.269+0000 7f26a37fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26a406e370 0x7f26a40c5c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57994/0 (socket says 172.21.15.16:57994) 2023-05-20T01:16:46.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.269+0000 7f26a37fe700 1 -- 172.21.15.16:0/3727533332 learned_addr learned my addr 172.21.15.16:0/3727533332 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:46.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.269+0000 7f26a3fff700 1 --2- 172.21.15.16:0/3727533332 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f26a406ac20 0x7f26a41496c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:46.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.269+0000 7f26a8a99700 1 --2- 172.21.15.16:0/3727533332 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f26a4161190 0x7f26a40c61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:46.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.269+0000 7f26a37fe700 1 -- 172.21.15.16:0/3727533332 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f26a406ac20 msgr2=0x7f26a41496c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:46.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.269+0000 7f26a37fe700 1 --2- 172.21.15.16:0/3727533332 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f26a406ac20 0x7f26a41496c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:46.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.269+0000 7f26a37fe700 1 -- 172.21.15.16:0/3727533332 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f26a4161190 msgr2=0x7f26a40c61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:46.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.269+0000 7f26a37fe700 1 --2- 172.21.15.16:0/3727533332 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f26a4161190 0x7f26a40c61c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:46.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.269+0000 7f26a37fe700 1 -- 172.21.15.16:0/3727533332 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2694009550 con 0x7f26a406e370 2023-05-20T01:16:46.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.269+0000 7f26a8a99700 1 --2- 172.21.15.16:0/3727533332 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f26a4161190 0x7f26a40c61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:16:46.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.270+0000 7f26a37fe700 1 --2- 172.21.15.16:0/3727533332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26a406e370 0x7f26a40c5c80 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f269800c4e0 tx=0x7f269800c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:46.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.270+0000 7f26a0ff9700 1 -- 172.21.15.16:0/3727533332 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26980052d0 con 0x7f26a406e370 2023-05-20T01:16:46.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.270+0000 7f26a0ff9700 1 -- 172.21.15.16:0/3727533332 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2698005430 con 0x7f26a406e370 2023-05-20T01:16:46.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.270+0000 7f26a0ff9700 1 -- 172.21.15.16:0/3727533332 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26980038c0 con 0x7f26a406e370 2023-05-20T01:16:46.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.270+0000 7f26aa4fc700 1 -- 172.21.15.16:0/3727533332 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26a40cf170 con 0x7f26a406e370 2023-05-20T01:16:46.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.270+0000 7f26aa4fc700 1 -- 172.21.15.16:0/3727533332 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26a40cee90 con 0x7f26a406e370 2023-05-20T01:16:46.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.271+0000 7f26aa4fc700 1 -- 172.21.15.16:0/3727533332 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26a40cf450 con 0x7f26a406e370 2023-05-20T01:16:46.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.271+0000 7f26a0ff9700 1 -- 172.21.15.16:0/3727533332 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2698003a20 con 0x7f26a406e370 2023-05-20T01:16:46.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.272+0000 7f26a0ff9700 1 --2- 172.21.15.16:0/3727533332 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f269006ace0 0x7f269006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:46.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.272+0000 7f26a0ff9700 1 -- 172.21.15.16:0/3727533332 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f2698088c10 con 0x7f26a406e370 2023-05-20T01:16:46.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.272+0000 7f26a3fff700 1 --2- 172.21.15.16:0/3727533332 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f269006ace0 0x7f269006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:46.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.272+0000 7f26a3fff700 1 --2- 172.21.15.16:0/3727533332 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f269006ace0 0x7f269006d190 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2694009c40 tx=0x7f2694008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:46.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.276+0000 7f26a0ff9700 1 -- 172.21.15.16:0/3727533332 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f26980539d0 con 0x7f26a406e370 2023-05-20T01:16:46.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.472+0000 7f26aa4fc700 1 -- 172.21.15.16:0/3727533332 --> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f26a41e7d40 con 0x7f269006ace0 2023-05-20T01:16:46.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.484+0000 7f26a0ff9700 1 -- 172.21.15.16:0/3727533332 <== mgr.4104 v2:172.21.15.16:6832/3817237998 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+367552 (secure 0 0 0) 0x7f26a41e7d40 con 0x7f269006ace0 2023-05-20T01:16:46.493 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:16:46.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.487+0000 7f26aa4fc700 1 -- 172.21.15.16:0/3727533332 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f269006ace0 msgr2=0x7f269006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:46.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.487+0000 7f26aa4fc700 1 --2- 172.21.15.16:0/3727533332 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f269006ace0 0x7f269006d190 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2694009c40 tx=0x7f2694008030 comp rx=0 tx=0).stop 2023-05-20T01:16:46.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.487+0000 7f26aa4fc700 1 -- 172.21.15.16:0/3727533332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26a406e370 msgr2=0x7f26a40c5c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:46.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.487+0000 7f26aa4fc700 1 --2- 172.21.15.16:0/3727533332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26a406e370 0x7f26a40c5c80 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f269800c4e0 tx=0x7f269800c7f0 comp rx=0 tx=0).stop 2023-05-20T01:16:46.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.487+0000 7f26aa4fc700 1 -- 172.21.15.16:0/3727533332 shutdown_connections 2023-05-20T01:16:46.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.487+0000 7f26aa4fc700 1 --2- 172.21.15.16:0/3727533332 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f26a4161190 0x7f26a40c61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:46.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.487+0000 7f26aa4fc700 1 --2- 172.21.15.16:0/3727533332 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f269006ace0 0x7f269006d190 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:46.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.487+0000 7f26aa4fc700 1 --2- 172.21.15.16:0/3727533332 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f26a406ac20 0x7f26a41496c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:46.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.487+0000 7f26aa4fc700 1 --2- 172.21.15.16:0/3727533332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26a406e370 0x7f26a40c5c80 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:46.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.487+0000 7f26aa4fc700 1 -- 172.21.15.16:0/3727533332 >> 172.21.15.16:0/3727533332 conn(0x7f26a40d0c70 msgr2=0x7f26a40acdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:46.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.487+0000 7f26aa4fc700 1 -- 172.21.15.16:0/3727533332 shutdown_connections 2023-05-20T01:16:46.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.487+0000 7f26aa4fc700 1 -- 172.21.15.16:0/3727533332 wait complete. 2023-05-20T01:16:46.498 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2023-05-20T01:16:46.503 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":31,"stamp":"2023-05-20T01:16:45.321252+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594819,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":206,"num_write_kb":2433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158,"ondisk_log_size":158,"up":338,"acting":338,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":338,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":67296,"kb_used_data":3296,"kb_used_omap":0,"kb_used_meta":63936,"kb_avail":754907424,"statfs":{"total":773094113280,"available":773025202176,"internally_reserved":0,"allocated":3375104,"data_stored":2062924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":65470464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2089,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002510"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.310521+0000","last_change":"2023-05-20T01:16:28.345449+0000","last_active":"2023-05-20T01:16:33.310521+0000","last_peered":"2023-05-20T01:16:33.310521+0000","last_clean":"2023-05-20T01:16:33.310521+0000","last_became_active":"2023-05-20T01:16:28.345161+0000","last_became_peered":"2023-05-20T01:16:28.345161+0000","last_unstale":"2023-05-20T01:16:33.310521+0000","last_undegraded":"2023-05-20T01:16:33.310521+0000","last_fullsized":"2023-05-20T01:16:33.310521+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:10:54.451689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.310465+0000","last_change":"2023-05-20T01:16:28.337973+0000","last_active":"2023-05-20T01:16:33.310465+0000","last_peered":"2023-05-20T01:16:33.310465+0000","last_clean":"2023-05-20T01:16:33.310465+0000","last_became_active":"2023-05-20T01:16:28.337679+0000","last_became_peered":"2023-05-20T01:16:28.337679+0000","last_unstale":"2023-05-20T01:16:33.310465+0000","last_undegraded":"2023-05-20T01:16:33.310465+0000","last_fullsized":"2023-05-20T01:16:33.310465+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:59:48.441604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.774449+0000","last_change":"2023-05-20T01:16:28.342781+0000","last_active":"2023-05-20T01:16:32.774449+0000","last_peered":"2023-05-20T01:16:32.774449+0000","last_clean":"2023-05-20T01:16:32.774449+0000","last_became_active":"2023-05-20T01:16:28.342488+0000","last_became_peered":"2023-05-20T01:16:28.342488+0000","last_unstale":"2023-05-20T01:16:32.774449+0000","last_undegraded":"2023-05-20T01:16:32.774449+0000","last_fullsized":"2023-05-20T01:16:32.774449+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:35:02.629395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.773758+0000","last_change":"2023-05-20T01:16:28.345746+0000","last_active":"2023-05-20T01:16:32.773758+0000","last_peered":"2023-05-20T01:16:32.773758+0000","last_clean":"2023-05-20T01:16:32.773758+0000","last_became_active":"2023-05-20T01:16:28.345477+0000","last_became_peered":"2023-05-20T01:16:28.345477+0000","last_unstale":"2023-05-20T01:16:32.773758+0000","last_undegraded":"2023-05-20T01:16:32.773758+0000","last_fullsized":"2023-05-20T01:16:32.773758+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:08:32.412558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.954652+0000","last_change":"2023-05-20T01:16:28.352984+0000","last_active":"2023-05-20T01:16:32.954652+0000","last_peered":"2023-05-20T01:16:32.954652+0000","last_clean":"2023-05-20T01:16:32.954652+0000","last_became_active":"2023-05-20T01:16:28.352711+0000","last_became_peered":"2023-05-20T01:16:28.352711+0000","last_unstale":"2023-05-20T01:16:32.954652+0000","last_undegraded":"2023-05-20T01:16:32.954652+0000","last_fullsized":"2023-05-20T01:16:32.954652+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:34:08.601566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.774271+0000","last_change":"2023-05-20T01:16:28.347827+0000","last_active":"2023-05-20T01:16:32.774271+0000","last_peered":"2023-05-20T01:16:32.774271+0000","last_clean":"2023-05-20T01:16:32.774271+0000","last_became_active":"2023-05-20T01:16:28.347476+0000","last_became_peered":"2023-05-20T01:16:28.347476+0000","last_unstale":"2023-05-20T01:16:32.774271+0000","last_undegraded":"2023-05-20T01:16:32.774271+0000","last_fullsized":"2023-05-20T01:16:32.774271+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:21:02.720695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.311877+0000","last_change":"2023-05-20T01:16:28.338564+0000","last_active":"2023-05-20T01:16:33.311877+0000","last_peered":"2023-05-20T01:16:33.311877+0000","last_clean":"2023-05-20T01:16:33.311877+0000","last_became_active":"2023-05-20T01:16:28.337927+0000","last_became_peered":"2023-05-20T01:16:28.337927+0000","last_unstale":"2023-05-20T01:16:33.311877+0000","last_undegraded":"2023-05-20T01:16:33.311877+0000","last_fullsized":"2023-05-20T01:16:33.311877+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:24:36.047155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.310151+0000","last_change":"2023-05-20T01:16:28.340008+0000","last_active":"2023-05-20T01:16:33.310151+0000","last_peered":"2023-05-20T01:16:33.310151+0000","last_clean":"2023-05-20T01:16:33.310151+0000","last_became_active":"2023-05-20T01:16:28.339724+0000","last_became_peered":"2023-05-20T01:16:28.339724+0000","last_unstale":"2023-05-20T01:16:33.310151+0000","last_undegraded":"2023-05-20T01:16:33.310151+0000","last_fullsized":"2023-05-20T01:16:33.310151+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:55:50.059228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.954657+0000","last_change":"2023-05-20T01:16:28.351002+0000","last_active":"2023-05-20T01:16:32.954657+0000","last_peered":"2023-05-20T01:16:32.954657+0000","last_clean":"2023-05-20T01:16:32.954657+0000","last_became_active":"2023-05-20T01:16:28.350672+0000","last_became_peered":"2023-05-20T01:16:28.350672+0000","last_unstale":"2023-05-20T01:16:32.954657+0000","last_undegraded":"2023-05-20T01:16:32.954657+0000","last_fullsized":"2023-05-20T01:16:32.954657+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:23:33.732284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.310970+0000","last_change":"2023-05-20T01:16:28.349410+0000","last_active":"2023-05-20T01:16:33.310970+0000","last_peered":"2023-05-20T01:16:33.310970+0000","last_clean":"2023-05-20T01:16:33.310970+0000","last_became_active":"2023-05-20T01:16:28.349177+0000","last_became_peered":"2023-05-20T01:16:28.349177+0000","last_unstale":"2023-05-20T01:16:33.310970+0000","last_undegraded":"2023-05-20T01:16:33.310970+0000","last_fullsized":"2023-05-20T01:16:33.310970+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:34:08.601566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.774486+0000","last_change":"2023-05-20T01:16:28.338627+0000","last_active":"2023-05-20T01:16:32.774486+0000","last_peered":"2023-05-20T01:16:32.774486+0000","last_clean":"2023-05-20T01:16:32.774486+0000","last_became_active":"2023-05-20T01:16:28.338363+0000","last_became_peered":"2023-05-20T01:16:28.338363+0000","last_unstale":"2023-05-20T01:16:32.774486+0000","last_undegraded":"2023-05-20T01:16:32.774486+0000","last_fullsized":"2023-05-20T01:16:32.774486+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:20:15.094065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.310156+0000","last_change":"2023-05-20T01:16:28.341603+0000","last_active":"2023-05-20T01:16:33.310156+0000","last_peered":"2023-05-20T01:16:33.310156+0000","last_clean":"2023-05-20T01:16:33.310156+0000","last_became_active":"2023-05-20T01:16:28.341242+0000","last_became_peered":"2023-05-20T01:16:28.341242+0000","last_unstale":"2023-05-20T01:16:33.310156+0000","last_undegraded":"2023-05-20T01:16:33.310156+0000","last_fullsized":"2023-05-20T01:16:33.310156+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:15:50.986060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.310269+0000","last_change":"2023-05-20T01:16:28.345977+0000","last_active":"2023-05-20T01:16:33.310269+0000","last_peered":"2023-05-20T01:16:33.310269+0000","last_clean":"2023-05-20T01:16:33.310269+0000","last_became_active":"2023-05-20T01:16:28.345732+0000","last_became_peered":"2023-05-20T01:16:28.345732+0000","last_unstale":"2023-05-20T01:16:33.310269+0000","last_undegraded":"2023-05-20T01:16:33.310269+0000","last_fullsized":"2023-05-20T01:16:33.310269+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:39:45.746298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.311067+0000","last_change":"2023-05-20T01:16:28.334227+0000","last_active":"2023-05-20T01:16:33.311067+0000","last_peered":"2023-05-20T01:16:33.311067+0000","last_clean":"2023-05-20T01:16:33.311067+0000","last_became_active":"2023-05-20T01:16:28.333922+0000","last_became_peered":"2023-05-20T01:16:28.333922+0000","last_unstale":"2023-05-20T01:16:33.311067+0000","last_undegraded":"2023-05-20T01:16:33.311067+0000","last_fullsized":"2023-05-20T01:16:33.311067+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:23:33.732284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.311509+0000","last_change":"2023-05-20T01:16:28.338772+0000","last_active":"2023-05-20T01:16:33.311509+0000","last_peered":"2023-05-20T01:16:33.311509+0000","last_clean":"2023-05-20T01:16:33.311509+0000","last_became_active":"2023-05-20T01:16:28.338300+0000","last_became_peered":"2023-05-20T01:16:28.338300+0000","last_unstale":"2023-05-20T01:16:33.311509+0000","last_undegraded":"2023-05-20T01:16:33.311509+0000","last_fullsized":"2023-05-20T01:16:33.311509+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:19:51.520080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.774491+0000","last_change":"2023-05-20T01:16:28.343926+0000","last_active":"2023-05-20T01:16:32.774491+0000","last_peered":"2023-05-20T01:16:32.774491+0000","last_clean":"2023-05-20T01:16:32.774491+0000","last_became_active":"2023-05-20T01:16:28.343696+0000","last_became_peered":"2023-05-20T01:16:28.343696+0000","last_unstale":"2023-05-20T01:16:32.774491+0000","last_undegraded":"2023-05-20T01:16:32.774491+0000","last_fullsized":"2023-05-20T01:16:32.774491+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:42:48.561844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.954664+0000","last_change":"2023-05-20T01:16:28.350876+0000","last_active":"2023-05-20T01:16:32.954664+0000","last_peered":"2023-05-20T01:16:32.954664+0000","last_clean":"2023-05-20T01:16:32.954664+0000","last_became_active":"2023-05-20T01:16:28.350242+0000","last_became_peered":"2023-05-20T01:16:28.350242+0000","last_unstale":"2023-05-20T01:16:32.954664+0000","last_undegraded":"2023-05-20T01:16:32.954664+0000","last_fullsized":"2023-05-20T01:16:32.954664+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:19:51.520080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.773740+0000","last_change":"2023-05-20T01:16:28.334794+0000","last_active":"2023-05-20T01:16:32.773740+0000","last_peered":"2023-05-20T01:16:32.773740+0000","last_clean":"2023-05-20T01:16:32.773740+0000","last_became_active":"2023-05-20T01:16:28.334535+0000","last_became_peered":"2023-05-20T01:16:28.334535+0000","last_unstale":"2023-05-20T01:16:32.773740+0000","last_undegraded":"2023-05-20T01:16:32.773740+0000","last_fullsized":"2023-05-20T01:16:32.773740+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T01:37:30.926061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.310464+0000","last_change":"2023-05-20T01:16:28.347597+0000","last_active":"2023-05-20T01:16:33.310464+0000","last_peered":"2023-05-20T01:16:33.310464+0000","last_clean":"2023-05-20T01:16:33.310464+0000","last_became_active":"2023-05-20T01:16:28.347245+0000","last_became_peered":"2023-05-20T01:16:28.347245+0000","last_unstale":"2023-05-20T01:16:33.310464+0000","last_undegraded":"2023-05-20T01:16:33.310464+0000","last_fullsized":"2023-05-20T01:16:33.310464+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:02:17.539187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.773711+0000","last_change":"2023-05-20T01:16:28.347994+0000","last_active":"2023-05-20T01:16:32.773711+0000","last_peered":"2023-05-20T01:16:32.773711+0000","last_clean":"2023-05-20T01:16:32.773711+0000","last_became_active":"2023-05-20T01:16:28.347690+0000","last_became_peered":"2023-05-20T01:16:28.347690+0000","last_unstale":"2023-05-20T01:16:32.773711+0000","last_undegraded":"2023-05-20T01:16:32.773711+0000","last_fullsized":"2023-05-20T01:16:32.773711+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:46:24.892963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.311730+0000","last_change":"2023-05-20T01:16:28.346262+0000","last_active":"2023-05-20T01:16:33.311730+0000","last_peered":"2023-05-20T01:16:33.311730+0000","last_clean":"2023-05-20T01:16:33.311730+0000","last_became_active":"2023-05-20T01:16:28.345513+0000","last_became_peered":"2023-05-20T01:16:28.345513+0000","last_unstale":"2023-05-20T01:16:33.311730+0000","last_undegraded":"2023-05-20T01:16:33.311730+0000","last_fullsized":"2023-05-20T01:16:33.311730+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:46:14.388881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.954206+0000","last_change":"2023-05-20T01:16:28.344269+0000","last_active":"2023-05-20T01:16:32.954206+0000","last_peered":"2023-05-20T01:16:32.954206+0000","last_clean":"2023-05-20T01:16:32.954206+0000","last_became_active":"2023-05-20T01:16:28.343991+0000","last_became_peered":"2023-05-20T01:16:28.343991+0000","last_unstale":"2023-05-20T01:16:32.954206+0000","last_undegraded":"2023-05-20T01:16:32.954206+0000","last_fullsized":"2023-05-20T01:16:32.954206+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:33:00.800050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.771776+0000","last_change":"2023-05-20T01:16:28.344552+0000","last_active":"2023-05-20T01:16:32.771776+0000","last_peered":"2023-05-20T01:16:32.771776+0000","last_clean":"2023-05-20T01:16:32.771776+0000","last_became_active":"2023-05-20T01:16:28.344125+0000","last_became_peered":"2023-05-20T01:16:28.344125+0000","last_unstale":"2023-05-20T01:16:32.771776+0000","last_undegraded":"2023-05-20T01:16:32.771776+0000","last_fullsized":"2023-05-20T01:16:32.771776+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:06:59.447318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.311284+0000","last_change":"2023-05-20T01:16:28.345899+0000","last_active":"2023-05-20T01:16:33.311284+0000","last_peered":"2023-05-20T01:16:33.311284+0000","last_clean":"2023-05-20T01:16:33.311284+0000","last_became_active":"2023-05-20T01:16:28.345629+0000","last_became_peered":"2023-05-20T01:16:28.345629+0000","last_unstale":"2023-05-20T01:16:33.311284+0000","last_undegraded":"2023-05-20T01:16:33.311284+0000","last_fullsized":"2023-05-20T01:16:33.311284+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:13:57.518009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.957468+0000","last_change":"2023-05-20T01:16:28.346168+0000","last_active":"2023-05-20T01:16:32.957468+0000","last_peered":"2023-05-20T01:16:32.957468+0000","last_clean":"2023-05-20T01:16:32.957468+0000","last_became_active":"2023-05-20T01:16:28.345889+0000","last_became_peered":"2023-05-20T01:16:28.345889+0000","last_unstale":"2023-05-20T01:16:32.957468+0000","last_undegraded":"2023-05-20T01:16:32.957468+0000","last_fullsized":"2023-05-20T01:16:32.957468+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:05:20.974728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.957396+0000","last_change":"2023-05-20T01:16:28.335035+0000","last_active":"2023-05-20T01:16:32.957396+0000","last_peered":"2023-05-20T01:16:32.957396+0000","last_clean":"2023-05-20T01:16:32.957396+0000","last_became_active":"2023-05-20T01:16:28.334787+0000","last_became_peered":"2023-05-20T01:16:28.334787+0000","last_unstale":"2023-05-20T01:16:32.957396+0000","last_undegraded":"2023-05-20T01:16:32.957396+0000","last_fullsized":"2023-05-20T01:16:32.957396+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:55:50.059228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.310780+0000","last_change":"2023-05-20T01:16:28.335038+0000","last_active":"2023-05-20T01:16:33.310780+0000","last_peered":"2023-05-20T01:16:33.310780+0000","last_clean":"2023-05-20T01:16:33.310780+0000","last_became_active":"2023-05-20T01:16:28.334750+0000","last_became_peered":"2023-05-20T01:16:28.334750+0000","last_unstale":"2023-05-20T01:16:33.310780+0000","last_undegraded":"2023-05-20T01:16:33.310780+0000","last_fullsized":"2023-05-20T01:16:33.310780+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:13:57.518009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.774853+0000","last_change":"2023-05-20T01:16:32.363983+0000","last_active":"2023-05-20T01:16:32.774853+0000","last_peered":"2023-05-20T01:16:32.774853+0000","last_clean":"2023-05-20T01:16:32.774853+0000","last_became_active":"2023-05-20T01:16:32.363545+0000","last_became_peered":"2023-05-20T01:16:32.363545+0000","last_unstale":"2023-05-20T01:16:32.774853+0000","last_undegraded":"2023-05-20T01:16:32.774853+0000","last_fullsized":"2023-05-20T01:16:32.774853+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:15:31.121265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2,0],"acting":[4,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"20'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.774070+0000","last_change":"2023-05-20T01:16:27.924758+0000","last_active":"2023-05-20T01:16:32.774070+0000","last_peered":"2023-05-20T01:16:32.774070+0000","last_clean":"2023-05-20T01:16:32.774070+0000","last_became_active":"2023-05-20T01:16:27.924471+0000","last_became_peered":"2023-05-20T01:16:27.924471+0000","last_unstale":"2023-05-20T01:16:32.774070+0000","last_undegraded":"2023-05-20T01:16:32.774070+0000","last_fullsized":"2023-05-20T01:16:32.774070+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:40:53.545036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.772324+0000","last_change":"2023-05-20T01:16:28.340516+0000","last_active":"2023-05-20T01:16:32.772324+0000","last_peered":"2023-05-20T01:16:32.772324+0000","last_clean":"2023-05-20T01:16:32.772324+0000","last_became_active":"2023-05-20T01:16:28.340255+0000","last_became_peered":"2023-05-20T01:16:28.340255+0000","last_unstale":"2023-05-20T01:16:32.772324+0000","last_undegraded":"2023-05-20T01:16:32.772324+0000","last_fullsized":"2023-05-20T01:16:32.772324+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:17:27.361087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.774064+0000","last_change":"2023-05-20T01:16:32.348270+0000","last_active":"2023-05-20T01:16:32.774064+0000","last_peered":"2023-05-20T01:16:32.774064+0000","last_clean":"2023-05-20T01:16:32.774064+0000","last_became_active":"2023-05-20T01:16:32.347460+0000","last_became_peered":"2023-05-20T01:16:32.347460+0000","last_unstale":"2023-05-20T01:16:32.774064+0000","last_undegraded":"2023-05-20T01:16:32.774064+0000","last_fullsized":"2023-05-20T01:16:32.774064+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:49:18.438950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3,7],"acting":[6,4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.956713+0000","last_change":"2023-05-20T01:16:27.922074+0000","last_active":"2023-05-20T01:16:32.956713+0000","last_peered":"2023-05-20T01:16:32.956713+0000","last_clean":"2023-05-20T01:16:32.956713+0000","last_became_active":"2023-05-20T01:16:27.921759+0000","last_became_peered":"2023-05-20T01:16:27.921759+0000","last_unstale":"2023-05-20T01:16:32.956713+0000","last_undegraded":"2023-05-20T01:16:32.956713+0000","last_fullsized":"2023-05-20T01:16:32.956713+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:08:26.231228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.954625+0000","last_change":"2023-05-20T01:16:28.348595+0000","last_active":"2023-05-20T01:16:32.954625+0000","last_peered":"2023-05-20T01:16:32.954625+0000","last_clean":"2023-05-20T01:16:32.954625+0000","last_became_active":"2023-05-20T01:16:28.347911+0000","last_became_peered":"2023-05-20T01:16:28.347911+0000","last_unstale":"2023-05-20T01:16:32.954625+0000","last_undegraded":"2023-05-20T01:16:32.954625+0000","last_fullsized":"2023-05-20T01:16:32.954625+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:46:42.721514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.954742+0000","last_change":"2023-05-20T01:16:32.362037+0000","last_active":"2023-05-20T01:16:32.954742+0000","last_peered":"2023-05-20T01:16:32.954742+0000","last_clean":"2023-05-20T01:16:32.954742+0000","last_became_active":"2023-05-20T01:16:32.361564+0000","last_became_peered":"2023-05-20T01:16:32.361564+0000","last_unstale":"2023-05-20T01:16:32.954742+0000","last_undegraded":"2023-05-20T01:16:32.954742+0000","last_fullsized":"2023-05-20T01:16:32.954742+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:14:35.048109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1,2],"acting":[0,7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.771855+0000","last_change":"2023-05-20T01:16:27.910268+0000","last_active":"2023-05-20T01:16:32.771855+0000","last_peered":"2023-05-20T01:16:32.771855+0000","last_clean":"2023-05-20T01:16:32.771855+0000","last_became_active":"2023-05-20T01:16:27.909963+0000","last_became_peered":"2023-05-20T01:16:27.909963+0000","last_unstale":"2023-05-20T01:16:32.771855+0000","last_undegraded":"2023-05-20T01:16:32.771855+0000","last_fullsized":"2023-05-20T01:16:32.771855+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:22:46.049477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.774156+0000","last_change":"2023-05-20T01:16:28.345078+0000","last_active":"2023-05-20T01:16:32.774156+0000","last_peered":"2023-05-20T01:16:32.774156+0000","last_clean":"2023-05-20T01:16:32.774156+0000","last_became_active":"2023-05-20T01:16:28.344742+0000","last_became_peered":"2023-05-20T01:16:28.344742+0000","last_unstale":"2023-05-20T01:16:32.774156+0000","last_undegraded":"2023-05-20T01:16:32.774156+0000","last_fullsized":"2023-05-20T01:16:32.774156+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:53:18.904132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.773906+0000","last_change":"2023-05-20T01:16:32.348805+0000","last_active":"2023-05-20T01:16:32.773906+0000","last_peered":"2023-05-20T01:16:32.773906+0000","last_clean":"2023-05-20T01:16:32.773906+0000","last_became_active":"2023-05-20T01:16:32.348352+0000","last_became_peered":"2023-05-20T01:16:32.348352+0000","last_unstale":"2023-05-20T01:16:32.773906+0000","last_undegraded":"2023-05-20T01:16:32.773906+0000","last_fullsized":"2023-05-20T01:16:32.773906+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:04:37.665737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7,1],"acting":[6,4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.954805+0000","last_change":"2023-05-20T01:16:27.915492+0000","last_active":"2023-05-20T01:16:32.954805+0000","last_peered":"2023-05-20T01:16:32.954805+0000","last_clean":"2023-05-20T01:16:32.954805+0000","last_became_active":"2023-05-20T01:16:27.915015+0000","last_became_peered":"2023-05-20T01:16:27.915015+0000","last_unstale":"2023-05-20T01:16:32.954805+0000","last_undegraded":"2023-05-20T01:16:32.954805+0000","last_fullsized":"2023-05-20T01:16:32.954805+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:25:21.983559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.771743+0000","last_change":"2023-05-20T01:16:28.339100+0000","last_active":"2023-05-20T01:16:32.771743+0000","last_peered":"2023-05-20T01:16:32.771743+0000","last_clean":"2023-05-20T01:16:32.771743+0000","last_became_active":"2023-05-20T01:16:28.338829+0000","last_became_peered":"2023-05-20T01:16:28.338829+0000","last_unstale":"2023-05-20T01:16:32.771743+0000","last_undegraded":"2023-05-20T01:16:32.771743+0000","last_fullsized":"2023-05-20T01:16:32.771743+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:38:13.734511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.310111+0000","last_change":"2023-05-20T01:16:32.364639+0000","last_active":"2023-05-20T01:16:33.310111+0000","last_peered":"2023-05-20T01:16:33.310111+0000","last_clean":"2023-05-20T01:16:33.310111+0000","last_became_active":"2023-05-20T01:16:32.364259+0000","last_became_peered":"2023-05-20T01:16:32.364259+0000","last_unstale":"2023-05-20T01:16:33.310111+0000","last_undegraded":"2023-05-20T01:16:33.310111+0000","last_fullsized":"2023-05-20T01:16:33.310111+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:58:39.806550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6,0],"acting":[7,4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.310717+0000","last_change":"2023-05-20T01:16:27.914406+0000","last_active":"2023-05-20T01:16:33.310717+0000","last_peered":"2023-05-20T01:16:33.310717+0000","last_clean":"2023-05-20T01:16:33.310717+0000","last_became_active":"2023-05-20T01:16:27.913932+0000","last_became_peered":"2023-05-20T01:16:27.913932+0000","last_unstale":"2023-05-20T01:16:33.310717+0000","last_undegraded":"2023-05-20T01:16:33.310717+0000","last_fullsized":"2023-05-20T01:16:33.310717+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:58:35.370081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.771638+0000","last_change":"2023-05-20T01:16:28.330729+0000","last_active":"2023-05-20T01:16:32.771638+0000","last_peered":"2023-05-20T01:16:32.771638+0000","last_clean":"2023-05-20T01:16:32.771638+0000","last_became_active":"2023-05-20T01:16:28.330474+0000","last_became_peered":"2023-05-20T01:16:28.330474+0000","last_unstale":"2023-05-20T01:16:32.771638+0000","last_undegraded":"2023-05-20T01:16:32.771638+0000","last_fullsized":"2023-05-20T01:16:32.771638+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:49:21.195547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.772392+0000","last_change":"2023-05-20T01:16:32.363881+0000","last_active":"2023-05-20T01:16:32.772392+0000","last_peered":"2023-05-20T01:16:32.772392+0000","last_clean":"2023-05-20T01:16:32.772392+0000","last_became_active":"2023-05-20T01:16:32.363269+0000","last_became_peered":"2023-05-20T01:16:32.363269+0000","last_unstale":"2023-05-20T01:16:32.772392+0000","last_undegraded":"2023-05-20T01:16:32.772392+0000","last_fullsized":"2023-05-20T01:16:32.772392+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:06:25.331494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,5],"acting":[3,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"20'3","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.311909+0000","last_change":"2023-05-20T01:16:27.919005+0000","last_active":"2023-05-20T01:16:33.311909+0000","last_peered":"2023-05-20T01:16:33.311909+0000","last_clean":"2023-05-20T01:16:33.311909+0000","last_became_active":"2023-05-20T01:16:27.918679+0000","last_became_peered":"2023-05-20T01:16:27.918679+0000","last_unstale":"2023-05-20T01:16:33.311909+0000","last_undegraded":"2023-05-20T01:16:33.311909+0000","last_fullsized":"2023-05-20T01:16:33.311909+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:06:58.439555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.957017+0000","last_change":"2023-05-20T01:16:28.346905+0000","last_active":"2023-05-20T01:16:32.957017+0000","last_peered":"2023-05-20T01:16:32.957017+0000","last_clean":"2023-05-20T01:16:32.957017+0000","last_became_active":"2023-05-20T01:16:28.345822+0000","last_became_peered":"2023-05-20T01:16:28.345822+0000","last_unstale":"2023-05-20T01:16:32.957017+0000","last_undegraded":"2023-05-20T01:16:32.957017+0000","last_fullsized":"2023-05-20T01:16:32.957017+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:45:24.050486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.957316+0000","last_change":"2023-05-20T01:16:32.354753+0000","last_active":"2023-05-20T01:16:32.957316+0000","last_peered":"2023-05-20T01:16:32.957316+0000","last_clean":"2023-05-20T01:16:32.957316+0000","last_became_active":"2023-05-20T01:16:32.354017+0000","last_became_peered":"2023-05-20T01:16:32.354017+0000","last_unstale":"2023-05-20T01:16:32.957316+0000","last_undegraded":"2023-05-20T01:16:32.957316+0000","last_fullsized":"2023-05-20T01:16:32.957316+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:24:19.113785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3,7],"acting":[1,5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"25'8","reported_seq":29,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:37.116885+0000","last_change":"2023-05-20T01:16:27.924627+0000","last_active":"2023-05-20T01:16:37.116885+0000","last_peered":"2023-05-20T01:16:37.116885+0000","last_clean":"2023-05-20T01:16:37.116885+0000","last_became_active":"2023-05-20T01:16:27.924339+0000","last_became_peered":"2023-05-20T01:16:27.924339+0000","last_unstale":"2023-05-20T01:16:37.116885+0000","last_undegraded":"2023-05-20T01:16:37.116885+0000","last_fullsized":"2023-05-20T01:16:37.116885+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:27:33.065117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3653,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.773847+0000","last_change":"2023-05-20T01:16:28.348715+0000","last_active":"2023-05-20T01:16:32.773847+0000","last_peered":"2023-05-20T01:16:32.773847+0000","last_clean":"2023-05-20T01:16:32.773847+0000","last_became_active":"2023-05-20T01:16:28.346893+0000","last_became_peered":"2023-05-20T01:16:28.346893+0000","last_unstale":"2023-05-20T01:16:32.773847+0000","last_undegraded":"2023-05-20T01:16:32.773847+0000","last_fullsized":"2023-05-20T01:16:32.773847+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:44:10.386273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.311010+0000","last_change":"2023-05-20T01:16:32.362978+0000","last_active":"2023-05-20T01:16:33.311010+0000","last_peered":"2023-05-20T01:16:33.311010+0000","last_clean":"2023-05-20T01:16:33.311010+0000","last_became_active":"2023-05-20T01:16:32.362592+0000","last_became_peered":"2023-05-20T01:16:32.362592+0000","last_unstale":"2023-05-20T01:16:33.311010+0000","last_undegraded":"2023-05-20T01:16:33.311010+0000","last_fullsized":"2023-05-20T01:16:33.311010+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:46:46.150220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3,0],"acting":[5,1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.311751+0000","last_change":"2023-05-20T01:16:27.925307+0000","last_active":"2023-05-20T01:16:33.311751+0000","last_peered":"2023-05-20T01:16:33.311751+0000","last_clean":"2023-05-20T01:16:33.311751+0000","last_became_active":"2023-05-20T01:16:27.925044+0000","last_became_peered":"2023-05-20T01:16:27.925044+0000","last_unstale":"2023-05-20T01:16:33.311751+0000","last_undegraded":"2023-05-20T01:16:33.311751+0000","last_fullsized":"2023-05-20T01:16:33.311751+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:54:11.307738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.772200+0000","last_change":"2023-05-20T01:16:28.340299+0000","last_active":"2023-05-20T01:16:32.772200+0000","last_peered":"2023-05-20T01:16:32.772200+0000","last_clean":"2023-05-20T01:16:32.772200+0000","last_became_active":"2023-05-20T01:16:28.340002+0000","last_became_peered":"2023-05-20T01:16:28.340002+0000","last_unstale":"2023-05-20T01:16:32.772200+0000","last_undegraded":"2023-05-20T01:16:32.772200+0000","last_fullsized":"2023-05-20T01:16:32.772200+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T01:44:35.746448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.772637+0000","last_change":"2023-05-20T01:16:32.354337+0000","last_active":"2023-05-20T01:16:32.772637+0000","last_peered":"2023-05-20T01:16:32.772637+0000","last_clean":"2023-05-20T01:16:32.772637+0000","last_became_active":"2023-05-20T01:16:32.353900+0000","last_became_peered":"2023-05-20T01:16:32.353900+0000","last_unstale":"2023-05-20T01:16:32.772637+0000","last_undegraded":"2023-05-20T01:16:32.772637+0000","last_fullsized":"2023-05-20T01:16:32.772637+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:50:05.163209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2,1],"acting":[3,7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.954416+0000","last_change":"2023-05-20T01:16:27.916514+0000","last_active":"2023-05-20T01:16:32.954416+0000","last_peered":"2023-05-20T01:16:32.954416+0000","last_clean":"2023-05-20T01:16:32.954416+0000","last_became_active":"2023-05-20T01:16:27.916199+0000","last_became_peered":"2023-05-20T01:16:27.916199+0000","last_unstale":"2023-05-20T01:16:32.954416+0000","last_undegraded":"2023-05-20T01:16:32.954416+0000","last_fullsized":"2023-05-20T01:16:32.954416+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:21:51.991048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.310678+0000","last_change":"2023-05-20T01:16:28.335549+0000","last_active":"2023-05-20T01:16:33.310678+0000","last_peered":"2023-05-20T01:16:33.310678+0000","last_clean":"2023-05-20T01:16:33.310678+0000","last_became_active":"2023-05-20T01:16:28.335284+0000","last_became_peered":"2023-05-20T01:16:28.335284+0000","last_unstale":"2023-05-20T01:16:33.310678+0000","last_undegraded":"2023-05-20T01:16:33.310678+0000","last_fullsized":"2023-05-20T01:16:33.310678+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:21:52.998812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.772615+0000","last_change":"2023-05-20T01:16:32.362344+0000","last_active":"2023-05-20T01:16:32.772615+0000","last_peered":"2023-05-20T01:16:32.772615+0000","last_clean":"2023-05-20T01:16:32.772615+0000","last_became_active":"2023-05-20T01:16:32.361944+0000","last_became_peered":"2023-05-20T01:16:32.361944+0000","last_unstale":"2023-05-20T01:16:32.772615+0000","last_undegraded":"2023-05-20T01:16:32.772615+0000","last_fullsized":"2023-05-20T01:16:32.772615+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:44:02.823618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,0],"acting":[3,1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.311580+0000","last_change":"2023-05-20T01:16:27.926496+0000","last_active":"2023-05-20T01:16:33.311580+0000","last_peered":"2023-05-20T01:16:33.311580+0000","last_clean":"2023-05-20T01:16:33.311580+0000","last_became_active":"2023-05-20T01:16:27.926265+0000","last_became_peered":"2023-05-20T01:16:27.926265+0000","last_unstale":"2023-05-20T01:16:33.311580+0000","last_undegraded":"2023-05-20T01:16:33.311580+0000","last_fullsized":"2023-05-20T01:16:33.311580+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:23:32.724521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.954832+0000","last_change":"2023-05-20T01:16:28.348384+0000","last_active":"2023-05-20T01:16:32.954832+0000","last_peered":"2023-05-20T01:16:32.954832+0000","last_clean":"2023-05-20T01:16:32.954832+0000","last_became_active":"2023-05-20T01:16:28.347652+0000","last_became_peered":"2023-05-20T01:16:28.347652+0000","last_unstale":"2023-05-20T01:16:32.954832+0000","last_undegraded":"2023-05-20T01:16:32.954832+0000","last_fullsized":"2023-05-20T01:16:32.954832+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:49:21.195547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.310448+0000","last_change":"2023-05-20T01:16:32.352418+0000","last_active":"2023-05-20T01:16:33.310448+0000","last_peered":"2023-05-20T01:16:33.310448+0000","last_clean":"2023-05-20T01:16:33.310448+0000","last_became_active":"2023-05-20T01:16:32.352031+0000","last_became_peered":"2023-05-20T01:16:32.352031+0000","last_unstale":"2023-05-20T01:16:33.310448+0000","last_undegraded":"2023-05-20T01:16:33.310448+0000","last_fullsized":"2023-05-20T01:16:33.310448+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:05:24.403434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1,6],"acting":[5,3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"20'8","reported_seq":29,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.773007+0000","last_change":"2023-05-20T01:16:27.916957+0000","last_active":"2023-05-20T01:16:32.773007+0000","last_peered":"2023-05-20T01:16:32.773007+0000","last_clean":"2023-05-20T01:16:32.773007+0000","last_became_active":"2023-05-20T01:16:27.916682+0000","last_became_peered":"2023-05-20T01:16:27.916682+0000","last_unstale":"2023-05-20T01:16:32.773007+0000","last_undegraded":"2023-05-20T01:16:32.773007+0000","last_fullsized":"2023-05-20T01:16:32.773007+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:15:10.960002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.772882+0000","last_change":"2023-05-20T01:16:28.339442+0000","last_active":"2023-05-20T01:16:32.772882+0000","last_peered":"2023-05-20T01:16:32.772882+0000","last_clean":"2023-05-20T01:16:32.772882+0000","last_became_active":"2023-05-20T01:16:28.339182+0000","last_became_peered":"2023-05-20T01:16:28.339182+0000","last_unstale":"2023-05-20T01:16:32.772882+0000","last_undegraded":"2023-05-20T01:16:32.772882+0000","last_fullsized":"2023-05-20T01:16:32.772882+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:31:47.073658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.311789+0000","last_change":"2023-05-20T01:16:32.360635+0000","last_active":"2023-05-20T01:16:33.311789+0000","last_peered":"2023-05-20T01:16:33.311789+0000","last_clean":"2023-05-20T01:16:33.311789+0000","last_became_active":"2023-05-20T01:16:32.360032+0000","last_became_peered":"2023-05-20T01:16:32.360032+0000","last_unstale":"2023-05-20T01:16:33.311789+0000","last_undegraded":"2023-05-20T01:16:33.311789+0000","last_fullsized":"2023-05-20T01:16:33.311789+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:49:24.624253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3,4],"acting":[7,6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"20'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.312203+0000","last_change":"2023-05-20T01:16:27.926418+0000","last_active":"2023-05-20T01:16:33.312203+0000","last_peered":"2023-05-20T01:16:33.312203+0000","last_clean":"2023-05-20T01:16:33.312203+0000","last_became_active":"2023-05-20T01:16:27.926135+0000","last_became_peered":"2023-05-20T01:16:27.926135+0000","last_unstale":"2023-05-20T01:16:33.312203+0000","last_undegraded":"2023-05-20T01:16:33.312203+0000","last_fullsized":"2023-05-20T01:16:33.312203+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:53:31.196496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.310060+0000","last_change":"2023-05-20T01:16:28.333381+0000","last_active":"2023-05-20T01:16:33.310060+0000","last_peered":"2023-05-20T01:16:33.310060+0000","last_clean":"2023-05-20T01:16:33.310060+0000","last_became_active":"2023-05-20T01:16:28.333096+0000","last_became_peered":"2023-05-20T01:16:28.333096+0000","last_unstale":"2023-05-20T01:16:33.310060+0000","last_undegraded":"2023-05-20T01:16:33.310060+0000","last_fullsized":"2023-05-20T01:16:33.310060+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:47:13.680746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.310001+0000","last_change":"2023-05-20T01:16:32.353695+0000","last_active":"2023-05-20T01:16:33.310001+0000","last_peered":"2023-05-20T01:16:33.310001+0000","last_clean":"2023-05-20T01:16:33.310001+0000","last_became_active":"2023-05-20T01:16:32.353280+0000","last_became_peered":"2023-05-20T01:16:32.353280+0000","last_unstale":"2023-05-20T01:16:33.310001+0000","last_undegraded":"2023-05-20T01:16:33.310001+0000","last_fullsized":"2023-05-20T01:16:33.310001+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:53:47.536905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1,5],"acting":[2,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.312036+0000","last_change":"2023-05-20T01:16:27.926342+0000","last_active":"2023-05-20T01:16:33.312036+0000","last_peered":"2023-05-20T01:16:33.312036+0000","last_clean":"2023-05-20T01:16:33.312036+0000","last_became_active":"2023-05-20T01:16:27.926093+0000","last_became_peered":"2023-05-20T01:16:27.926093+0000","last_unstale":"2023-05-20T01:16:33.312036+0000","last_undegraded":"2023-05-20T01:16:33.312036+0000","last_fullsized":"2023-05-20T01:16:33.312036+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:22:46.049477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.774721+0000","last_change":"2023-05-20T01:16:28.338568+0000","last_active":"2023-05-20T01:16:32.774721+0000","last_peered":"2023-05-20T01:16:32.774721+0000","last_clean":"2023-05-20T01:16:32.774721+0000","last_became_active":"2023-05-20T01:16:28.338293+0000","last_became_peered":"2023-05-20T01:16:28.338293+0000","last_unstale":"2023-05-20T01:16:32.774721+0000","last_undegraded":"2023-05-20T01:16:32.774721+0000","last_fullsized":"2023-05-20T01:16:32.774721+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:49:07.725723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.310172+0000","last_change":"2023-05-20T01:16:32.358154+0000","last_active":"2023-05-20T01:16:33.310172+0000","last_peered":"2023-05-20T01:16:33.310172+0000","last_clean":"2023-05-20T01:16:33.310172+0000","last_became_active":"2023-05-20T01:16:32.357690+0000","last_became_peered":"2023-05-20T01:16:32.357690+0000","last_unstale":"2023-05-20T01:16:33.310172+0000","last_undegraded":"2023-05-20T01:16:33.310172+0000","last_fullsized":"2023-05-20T01:16:33.310172+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:04:33.803901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5,6],"acting":[2,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"20'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.311721+0000","last_change":"2023-05-20T01:16:27.922534+0000","last_active":"2023-05-20T01:16:33.311721+0000","last_peered":"2023-05-20T01:16:33.311721+0000","last_clean":"2023-05-20T01:16:33.311721+0000","last_became_active":"2023-05-20T01:16:27.922263+0000","last_became_peered":"2023-05-20T01:16:27.922263+0000","last_unstale":"2023-05-20T01:16:33.311721+0000","last_undegraded":"2023-05-20T01:16:33.311721+0000","last_fullsized":"2023-05-20T01:16:33.311721+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:25:21.983559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.773343+0000","last_change":"2023-05-20T01:16:28.345113+0000","last_active":"2023-05-20T01:16:32.773343+0000","last_peered":"2023-05-20T01:16:32.773343+0000","last_clean":"2023-05-20T01:16:32.773343+0000","last_became_active":"2023-05-20T01:16:28.344807+0000","last_became_peered":"2023-05-20T01:16:28.344807+0000","last_unstale":"2023-05-20T01:16:32.773343+0000","last_undegraded":"2023-05-20T01:16:32.773343+0000","last_fullsized":"2023-05-20T01:16:32.773343+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:31:16.197740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.310276+0000","last_change":"2023-05-20T01:16:32.346858+0000","last_active":"2023-05-20T01:16:33.310276+0000","last_peered":"2023-05-20T01:16:33.310276+0000","last_clean":"2023-05-20T01:16:33.310276+0000","last_became_active":"2023-05-20T01:16:32.346456+0000","last_became_peered":"2023-05-20T01:16:32.346456+0000","last_unstale":"2023-05-20T01:16:33.310276+0000","last_undegraded":"2023-05-20T01:16:33.310276+0000","last_fullsized":"2023-05-20T01:16:33.310276+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:30:20.804962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3,6],"acting":[2,4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.310878+0000","last_change":"2023-05-20T01:16:27.912388+0000","last_active":"2023-05-20T01:16:33.310878+0000","last_peered":"2023-05-20T01:16:33.310878+0000","last_clean":"2023-05-20T01:16:33.310878+0000","last_became_active":"2023-05-20T01:16:27.911977+0000","last_became_peered":"2023-05-20T01:16:27.911977+0000","last_unstale":"2023-05-20T01:16:33.310878+0000","last_undegraded":"2023-05-20T01:16:33.310878+0000","last_fullsized":"2023-05-20T01:16:33.310878+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:19:50.512317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.954435+0000","last_change":"2023-05-20T01:16:28.342633+0000","last_active":"2023-05-20T01:16:32.954435+0000","last_peered":"2023-05-20T01:16:32.954435+0000","last_clean":"2023-05-20T01:16:32.954435+0000","last_became_active":"2023-05-20T01:16:28.339419+0000","last_became_peered":"2023-05-20T01:16:28.339419+0000","last_unstale":"2023-05-20T01:16:32.954435+0000","last_undegraded":"2023-05-20T01:16:32.954435+0000","last_fullsized":"2023-05-20T01:16:32.954435+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:27:57.460519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.310304+0000","last_change":"2023-05-20T01:16:32.347930+0000","last_active":"2023-05-20T01:16:33.310304+0000","last_peered":"2023-05-20T01:16:33.310304+0000","last_clean":"2023-05-20T01:16:33.310304+0000","last_became_active":"2023-05-20T01:16:32.347502+0000","last_became_peered":"2023-05-20T01:16:32.347502+0000","last_unstale":"2023-05-20T01:16:33.310304+0000","last_undegraded":"2023-05-20T01:16:33.310304+0000","last_fullsized":"2023-05-20T01:16:33.310304+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:05:15.989168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5,6],"acting":[2,7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.772049+0000","last_change":"2023-05-20T01:16:27.909757+0000","last_active":"2023-05-20T01:16:32.772049+0000","last_peered":"2023-05-20T01:16:32.772049+0000","last_clean":"2023-05-20T01:16:32.772049+0000","last_became_active":"2023-05-20T01:16:27.909458+0000","last_became_peered":"2023-05-20T01:16:27.909458+0000","last_unstale":"2023-05-20T01:16:32.772049+0000","last_undegraded":"2023-05-20T01:16:32.772049+0000","last_fullsized":"2023-05-20T01:16:32.772049+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:26:58.729159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.773572+0000","last_change":"2023-05-20T01:16:28.340078+0000","last_active":"2023-05-20T01:16:32.773572+0000","last_peered":"2023-05-20T01:16:32.773572+0000","last_clean":"2023-05-20T01:16:32.773572+0000","last_became_active":"2023-05-20T01:16:28.339673+0000","last_became_peered":"2023-05-20T01:16:28.339673+0000","last_unstale":"2023-05-20T01:16:32.773572+0000","last_undegraded":"2023-05-20T01:16:32.773572+0000","last_fullsized":"2023-05-20T01:16:32.773572+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:13:55.558140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.957039+0000","last_change":"2023-05-20T01:16:32.364538+0000","last_active":"2023-05-20T01:16:32.957039+0000","last_peered":"2023-05-20T01:16:32.957039+0000","last_clean":"2023-05-20T01:16:32.957039+0000","last_became_active":"2023-05-20T01:16:32.364025+0000","last_became_peered":"2023-05-20T01:16:32.364025+0000","last_unstale":"2023-05-20T01:16:32.957039+0000","last_undegraded":"2023-05-20T01:16:32.957039+0000","last_fullsized":"2023-05-20T01:16:32.957039+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:01:04.644797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0,6],"acting":[1,4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.774177+0000","last_change":"2023-05-20T01:16:27.922354+0000","last_active":"2023-05-20T01:16:32.774177+0000","last_peered":"2023-05-20T01:16:32.774177+0000","last_clean":"2023-05-20T01:16:32.774177+0000","last_became_active":"2023-05-20T01:16:27.922000+0000","last_became_peered":"2023-05-20T01:16:27.922000+0000","last_unstale":"2023-05-20T01:16:32.774177+0000","last_undegraded":"2023-05-20T01:16:32.774177+0000","last_fullsized":"2023-05-20T01:16:32.774177+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:56:49.903435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.774102+0000","last_change":"2023-05-20T01:16:28.334441+0000","last_active":"2023-05-20T01:16:32.774102+0000","last_peered":"2023-05-20T01:16:32.774102+0000","last_clean":"2023-05-20T01:16:32.774102+0000","last_became_active":"2023-05-20T01:16:28.334157+0000","last_became_peered":"2023-05-20T01:16:28.334157+0000","last_unstale":"2023-05-20T01:16:32.774102+0000","last_undegraded":"2023-05-20T01:16:32.774102+0000","last_fullsized":"2023-05-20T01:16:32.774102+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:51:16.225196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.310897+0000","last_change":"2023-05-20T01:16:32.346834+0000","last_active":"2023-05-20T01:16:33.310897+0000","last_peered":"2023-05-20T01:16:33.310897+0000","last_clean":"2023-05-20T01:16:33.310897+0000","last_became_active":"2023-05-20T01:16:32.346410+0000","last_became_peered":"2023-05-20T01:16:32.346410+0000","last_unstale":"2023-05-20T01:16:33.310897+0000","last_undegraded":"2023-05-20T01:16:33.310897+0000","last_fullsized":"2023-05-20T01:16:33.310897+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:07:08.123139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6,2],"acting":[5,4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.774525+0000","last_change":"2023-05-20T01:16:27.921658+0000","last_active":"2023-05-20T01:16:32.774525+0000","last_peered":"2023-05-20T01:16:32.774525+0000","last_clean":"2023-05-20T01:16:32.774525+0000","last_became_active":"2023-05-20T01:16:27.921399+0000","last_became_peered":"2023-05-20T01:16:27.921399+0000","last_unstale":"2023-05-20T01:16:32.774525+0000","last_undegraded":"2023-05-20T01:16:32.774525+0000","last_fullsized":"2023-05-20T01:16:32.774525+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:43:09.775404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.775593+0000","last_change":"2023-05-20T01:16:28.347570+0000","last_active":"2023-05-20T01:16:32.775593+0000","last_peered":"2023-05-20T01:16:32.775593+0000","last_clean":"2023-05-20T01:16:32.775593+0000","last_became_active":"2023-05-20T01:16:28.347140+0000","last_became_peered":"2023-05-20T01:16:28.347140+0000","last_unstale":"2023-05-20T01:16:32.775593+0000","last_undegraded":"2023-05-20T01:16:32.775593+0000","last_fullsized":"2023-05-20T01:16:32.775593+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T01:30:46.937626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.773539+0000","last_change":"2023-05-20T01:16:32.363537+0000","last_active":"2023-05-20T01:16:32.773539+0000","last_peered":"2023-05-20T01:16:32.773539+0000","last_clean":"2023-05-20T01:16:32.773539+0000","last_became_active":"2023-05-20T01:16:32.363119+0000","last_became_peered":"2023-05-20T01:16:32.363119+0000","last_unstale":"2023-05-20T01:16:32.773539+0000","last_undegraded":"2023-05-20T01:16:32.773539+0000","last_fullsized":"2023-05-20T01:16:32.773539+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:40:04.730139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0,2],"acting":[4,5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.771744+0000","last_change":"2023-05-20T01:16:27.924294+0000","last_active":"2023-05-20T01:16:32.771744+0000","last_peered":"2023-05-20T01:16:32.771744+0000","last_clean":"2023-05-20T01:16:32.771744+0000","last_became_active":"2023-05-20T01:16:27.923930+0000","last_became_peered":"2023-05-20T01:16:27.923930+0000","last_unstale":"2023-05-20T01:16:32.771744+0000","last_undegraded":"2023-05-20T01:16:32.771744+0000","last_fullsized":"2023-05-20T01:16:32.771744+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:24:35.039391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.957544+0000","last_change":"2023-05-20T01:16:28.349390+0000","last_active":"2023-05-20T01:16:32.957544+0000","last_peered":"2023-05-20T01:16:32.957544+0000","last_clean":"2023-05-20T01:16:32.957544+0000","last_became_active":"2023-05-20T01:16:28.349074+0000","last_became_peered":"2023-05-20T01:16:28.349074+0000","last_unstale":"2023-05-20T01:16:32.957544+0000","last_undegraded":"2023-05-20T01:16:32.957544+0000","last_fullsized":"2023-05-20T01:16:32.957544+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:59:48.441604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.774884+0000","last_change":"2023-05-20T01:16:32.353210+0000","last_active":"2023-05-20T01:16:32.774884+0000","last_peered":"2023-05-20T01:16:32.774884+0000","last_clean":"2023-05-20T01:16:32.774884+0000","last_became_active":"2023-05-20T01:16:32.352807+0000","last_became_peered":"2023-05-20T01:16:32.352807+0000","last_unstale":"2023-05-20T01:16:32.774884+0000","last_undegraded":"2023-05-20T01:16:32.774884+0000","last_fullsized":"2023-05-20T01:16:32.774884+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:42:22.830436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7,0],"acting":[4,3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.772776+0000","last_change":"2023-05-20T01:16:27.913270+0000","last_active":"2023-05-20T01:16:32.772776+0000","last_peered":"2023-05-20T01:16:32.772776+0000","last_clean":"2023-05-20T01:16:32.772776+0000","last_became_active":"2023-05-20T01:16:27.912952+0000","last_became_peered":"2023-05-20T01:16:27.912952+0000","last_unstale":"2023-05-20T01:16:32.772776+0000","last_undegraded":"2023-05-20T01:16:32.772776+0000","last_fullsized":"2023-05-20T01:16:32.772776+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:34:35.188240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.774248+0000","last_change":"2023-05-20T01:16:23.289068+0000","last_active":"2023-05-20T01:16:32.774248+0000","last_peered":"2023-05-20T01:16:32.774248+0000","last_clean":"2023-05-20T01:16:32.774248+0000","last_became_active":"2023-05-20T01:16:23.288781+0000","last_became_peered":"2023-05-20T01:16:23.288781+0000","last_unstale":"2023-05-20T01:16:32.774248+0000","last_undegraded":"2023-05-20T01:16:32.774248+0000","last_fullsized":"2023-05-20T01:16:32.774248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:02:03.239372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.771607+0000","last_change":"2023-05-20T01:16:28.329739+0000","last_active":"2023-05-20T01:16:32.771607+0000","last_peered":"2023-05-20T01:16:32.771607+0000","last_clean":"2023-05-20T01:16:32.771607+0000","last_became_active":"2023-05-20T01:16:28.329483+0000","last_became_peered":"2023-05-20T01:16:28.329483+0000","last_unstale":"2023-05-20T01:16:32.771607+0000","last_undegraded":"2023-05-20T01:16:32.771607+0000","last_fullsized":"2023-05-20T01:16:32.771607+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:46:14.388881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.771640+0000","last_change":"2023-05-20T01:16:32.353144+0000","last_active":"2023-05-20T01:16:32.771640+0000","last_peered":"2023-05-20T01:16:32.771640+0000","last_clean":"2023-05-20T01:16:32.771640+0000","last_became_active":"2023-05-20T01:16:32.352355+0000","last_became_peered":"2023-05-20T01:16:32.352355+0000","last_unstale":"2023-05-20T01:16:32.771640+0000","last_undegraded":"2023-05-20T01:16:32.771640+0000","last_fullsized":"2023-05-20T01:16:32.771640+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:25:26.420028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,0],"acting":[3,1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.772224+0000","last_change":"2023-05-20T01:16:22.829324+0000","last_active":"2023-05-20T01:16:32.772224+0000","last_peered":"2023-05-20T01:16:32.772224+0000","last_clean":"2023-05-20T01:16:32.772224+0000","last_became_active":"2023-05-20T01:16:22.828964+0000","last_became_peered":"2023-05-20T01:16:22.828964+0000","last_unstale":"2023-05-20T01:16:32.772224+0000","last_undegraded":"2023-05-20T01:16:32.772224+0000","last_fullsized":"2023-05-20T01:16:32.772224+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:33:22.739978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"25'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.954588+0000","last_change":"2023-05-20T01:16:27.922339+0000","last_active":"2023-05-20T01:16:32.954588+0000","last_peered":"2023-05-20T01:16:32.954588+0000","last_clean":"2023-05-20T01:16:32.954588+0000","last_became_active":"2023-05-20T01:16:27.922033+0000","last_became_peered":"2023-05-20T01:16:27.922033+0000","last_unstale":"2023-05-20T01:16:32.954588+0000","last_undegraded":"2023-05-20T01:16:32.954588+0000","last_fullsized":"2023-05-20T01:16:32.954588+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:01:20.476920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.774598+0000","last_change":"2023-05-20T01:16:28.335846+0000","last_active":"2023-05-20T01:16:32.774598+0000","last_peered":"2023-05-20T01:16:32.774598+0000","last_clean":"2023-05-20T01:16:32.774598+0000","last_became_active":"2023-05-20T01:16:28.335523+0000","last_became_peered":"2023-05-20T01:16:28.335523+0000","last_unstale":"2023-05-20T01:16:32.774598+0000","last_undegraded":"2023-05-20T01:16:32.774598+0000","last_fullsized":"2023-05-20T01:16:32.774598+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:18:05.626837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.310654+0000","last_change":"2023-05-20T01:16:32.363029+0000","last_active":"2023-05-20T01:16:33.310654+0000","last_peered":"2023-05-20T01:16:33.310654+0000","last_clean":"2023-05-20T01:16:33.310654+0000","last_became_active":"2023-05-20T01:16:32.362637+0000","last_became_peered":"2023-05-20T01:16:32.362637+0000","last_unstale":"2023-05-20T01:16:33.310654+0000","last_undegraded":"2023-05-20T01:16:33.310654+0000","last_fullsized":"2023-05-20T01:16:33.310654+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:28:00.889225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0,4],"acting":[5,1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.956901+0000","last_change":"2023-05-20T01:16:22.834754+0000","last_active":"2023-05-20T01:16:32.956901+0000","last_peered":"2023-05-20T01:16:32.956901+0000","last_clean":"2023-05-20T01:16:32.956901+0000","last_became_active":"2023-05-20T01:16:22.834437+0000","last_became_peered":"2023-05-20T01:16:22.834437+0000","last_unstale":"2023-05-20T01:16:32.956901+0000","last_undegraded":"2023-05-20T01:16:32.956901+0000","last_fullsized":"2023-05-20T01:16:32.956901+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:51:36.657132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":177,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.311443+0000","last_change":"2023-05-20T01:16:22.834920+0000","last_active":"2023-05-20T01:16:33.311443+0000","last_peered":"2023-05-20T01:16:33.311443+0000","last_clean":"2023-05-20T01:16:33.311443+0000","last_became_active":"2023-05-20T01:16:22.833924+0000","last_became_peered":"2023-05-20T01:16:22.833924+0000","last_unstale":"2023-05-20T01:16:33.311443+0000","last_undegraded":"2023-05-20T01:16:33.311443+0000","last_fullsized":"2023-05-20T01:16:33.311443+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:34:02.513729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.954099+0000","last_change":"2023-05-20T01:16:27.923011+0000","last_active":"2023-05-20T01:16:32.954099+0000","last_peered":"2023-05-20T01:16:32.954099+0000","last_clean":"2023-05-20T01:16:32.954099+0000","last_became_active":"2023-05-20T01:16:27.922736+0000","last_became_peered":"2023-05-20T01:16:27.922736+0000","last_unstale":"2023-05-20T01:16:32.954099+0000","last_undegraded":"2023-05-20T01:16:32.954099+0000","last_fullsized":"2023-05-20T01:16:32.954099+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:59:47.433840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.954829+0000","last_change":"2023-05-20T01:16:28.348199+0000","last_active":"2023-05-20T01:16:32.954829+0000","last_peered":"2023-05-20T01:16:32.954829+0000","last_clean":"2023-05-20T01:16:32.954829+0000","last_became_active":"2023-05-20T01:16:28.347370+0000","last_became_peered":"2023-05-20T01:16:28.347370+0000","last_unstale":"2023-05-20T01:16:32.954829+0000","last_undegraded":"2023-05-20T01:16:32.954829+0000","last_fullsized":"2023-05-20T01:16:32.954829+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:46:14.388881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.310467+0000","last_change":"2023-05-20T01:16:32.357823+0000","last_active":"2023-05-20T01:16:33.310467+0000","last_peered":"2023-05-20T01:16:33.310467+0000","last_clean":"2023-05-20T01:16:33.310467+0000","last_became_active":"2023-05-20T01:16:32.357437+0000","last_became_peered":"2023-05-20T01:16:32.357437+0000","last_unstale":"2023-05-20T01:16:33.310467+0000","last_undegraded":"2023-05-20T01:16:33.310467+0000","last_fullsized":"2023-05-20T01:16:33.310467+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:16:08.791218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1,7],"acting":[2,5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.311398+0000","last_change":"2023-05-20T01:16:23.290259+0000","last_active":"2023-05-20T01:16:33.311398+0000","last_peered":"2023-05-20T01:16:33.311398+0000","last_clean":"2023-05-20T01:16:33.311398+0000","last_became_active":"2023-05-20T01:16:23.289911+0000","last_became_peered":"2023-05-20T01:16:23.289911+0000","last_unstale":"2023-05-20T01:16:33.311398+0000","last_undegraded":"2023-05-20T01:16:33.311398+0000","last_fullsized":"2023-05-20T01:16:33.311398+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:46:36.633677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.956681+0000","last_change":"2023-05-20T01:16:27.919248+0000","last_active":"2023-05-20T01:16:32.956681+0000","last_peered":"2023-05-20T01:16:32.956681+0000","last_clean":"2023-05-20T01:16:32.956681+0000","last_became_active":"2023-05-20T01:16:27.918957+0000","last_became_peered":"2023-05-20T01:16:27.918957+0000","last_unstale":"2023-05-20T01:16:32.956681+0000","last_undegraded":"2023-05-20T01:16:32.956681+0000","last_fullsized":"2023-05-20T01:16:32.956681+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:51:16.461763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.957471+0000","last_change":"2023-05-20T01:16:28.337136+0000","last_active":"2023-05-20T01:16:32.957471+0000","last_peered":"2023-05-20T01:16:32.957471+0000","last_clean":"2023-05-20T01:16:32.957471+0000","last_became_active":"2023-05-20T01:16:28.336322+0000","last_became_peered":"2023-05-20T01:16:28.336322+0000","last_unstale":"2023-05-20T01:16:32.957471+0000","last_undegraded":"2023-05-20T01:16:32.957471+0000","last_fullsized":"2023-05-20T01:16:32.957471+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:42:59.927187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.311713+0000","last_change":"2023-05-20T01:16:32.358840+0000","last_active":"2023-05-20T01:16:33.311713+0000","last_peered":"2023-05-20T01:16:33.311713+0000","last_clean":"2023-05-20T01:16:33.311713+0000","last_became_active":"2023-05-20T01:16:32.358470+0000","last_became_peered":"2023-05-20T01:16:32.358470+0000","last_unstale":"2023-05-20T01:16:33.311713+0000","last_undegraded":"2023-05-20T01:16:33.311713+0000","last_fullsized":"2023-05-20T01:16:33.311713+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:14:35.048109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5,3],"acting":[7,2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.311131+0000","last_change":"2023-05-20T01:16:23.290207+0000","last_active":"2023-05-20T01:16:33.311131+0000","last_peered":"2023-05-20T01:16:33.311131+0000","last_clean":"2023-05-20T01:16:33.311131+0000","last_became_active":"2023-05-20T01:16:23.289928+0000","last_became_peered":"2023-05-20T01:16:23.289928+0000","last_unstale":"2023-05-20T01:16:33.311131+0000","last_undegraded":"2023-05-20T01:16:33.311131+0000","last_fullsized":"2023-05-20T01:16:33.311131+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:01:15.396847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.771970+0000","last_change":"2023-05-20T01:16:27.911483+0000","last_active":"2023-05-20T01:16:32.771970+0000","last_peered":"2023-05-20T01:16:32.771970+0000","last_clean":"2023-05-20T01:16:32.771970+0000","last_became_active":"2023-05-20T01:16:27.911180+0000","last_became_peered":"2023-05-20T01:16:27.911180+0000","last_unstale":"2023-05-20T01:16:32.771970+0000","last_undegraded":"2023-05-20T01:16:32.771970+0000","last_fullsized":"2023-05-20T01:16:32.771970+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:13:48.951321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.773506+0000","last_change":"2023-05-20T01:16:28.345161+0000","last_active":"2023-05-20T01:16:32.773506+0000","last_peered":"2023-05-20T01:16:32.773506+0000","last_clean":"2023-05-20T01:16:32.773506+0000","last_became_active":"2023-05-20T01:16:28.344872+0000","last_became_peered":"2023-05-20T01:16:28.344872+0000","last_unstale":"2023-05-20T01:16:32.773506+0000","last_undegraded":"2023-05-20T01:16:32.773506+0000","last_fullsized":"2023-05-20T01:16:32.773506+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:49:51.917546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.954428+0000","last_change":"2023-05-20T01:16:32.362192+0000","last_active":"2023-05-20T01:16:32.954428+0000","last_peered":"2023-05-20T01:16:32.954428+0000","last_clean":"2023-05-20T01:16:32.954428+0000","last_became_active":"2023-05-20T01:16:32.361656+0000","last_became_peered":"2023-05-20T01:16:32.361656+0000","last_unstale":"2023-05-20T01:16:32.954428+0000","last_undegraded":"2023-05-20T01:16:32.954428+0000","last_fullsized":"2023-05-20T01:16:32.954428+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:58:35.944694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,6],"acting":[0,1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.310220+0000","last_change":"2023-05-20T01:16:23.290538+0000","last_active":"2023-05-20T01:16:33.310220+0000","last_peered":"2023-05-20T01:16:33.310220+0000","last_clean":"2023-05-20T01:16:33.310220+0000","last_became_active":"2023-05-20T01:16:23.290230+0000","last_became_peered":"2023-05-20T01:16:23.290230+0000","last_unstale":"2023-05-20T01:16:33.310220+0000","last_undegraded":"2023-05-20T01:16:33.310220+0000","last_fullsized":"2023-05-20T01:16:33.310220+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:24:09.597242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.773435+0000","last_change":"2023-05-20T01:16:27.921749+0000","last_active":"2023-05-20T01:16:32.773435+0000","last_peered":"2023-05-20T01:16:32.773435+0000","last_clean":"2023-05-20T01:16:32.773435+0000","last_became_active":"2023-05-20T01:16:27.921461+0000","last_became_peered":"2023-05-20T01:16:27.921461+0000","last_unstale":"2023-05-20T01:16:32.773435+0000","last_undegraded":"2023-05-20T01:16:32.773435+0000","last_fullsized":"2023-05-20T01:16:32.773435+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:01:30.293877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.957033+0000","last_change":"2023-05-20T01:16:28.335769+0000","last_active":"2023-05-20T01:16:32.957033+0000","last_peered":"2023-05-20T01:16:32.957033+0000","last_clean":"2023-05-20T01:16:32.957033+0000","last_became_active":"2023-05-20T01:16:28.335232+0000","last_became_peered":"2023-05-20T01:16:28.335232+0000","last_unstale":"2023-05-20T01:16:32.957033+0000","last_undegraded":"2023-05-20T01:16:32.957033+0000","last_fullsized":"2023-05-20T01:16:32.957033+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:23:18.405128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.774369+0000","last_change":"2023-05-20T01:16:32.354871+0000","last_active":"2023-05-20T01:16:32.774369+0000","last_peered":"2023-05-20T01:16:32.774369+0000","last_clean":"2023-05-20T01:16:32.774369+0000","last_became_active":"2023-05-20T01:16:32.353771+0000","last_became_peered":"2023-05-20T01:16:32.353771+0000","last_unstale":"2023-05-20T01:16:32.774369+0000","last_undegraded":"2023-05-20T01:16:32.774369+0000","last_fullsized":"2023-05-20T01:16:32.774369+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:22:02.793310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5,2],"acting":[6,0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.773241+0000","last_change":"2023-05-20T01:16:23.290341+0000","last_active":"2023-05-20T01:16:32.773241+0000","last_peered":"2023-05-20T01:16:32.773241+0000","last_clean":"2023-05-20T01:16:32.773241+0000","last_became_active":"2023-05-20T01:16:23.290053+0000","last_became_peered":"2023-05-20T01:16:23.290053+0000","last_unstale":"2023-05-20T01:16:32.773241+0000","last_undegraded":"2023-05-20T01:16:32.773241+0000","last_fullsized":"2023-05-20T01:16:32.773241+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:11:18.465994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.957177+0000","last_change":"2023-05-20T01:16:27.920036+0000","last_active":"2023-05-20T01:16:32.957177+0000","last_peered":"2023-05-20T01:16:32.957177+0000","last_clean":"2023-05-20T01:16:32.957177+0000","last_became_active":"2023-05-20T01:16:27.919757+0000","last_became_peered":"2023-05-20T01:16:27.919757+0000","last_unstale":"2023-05-20T01:16:32.957177+0000","last_undegraded":"2023-05-20T01:16:32.957177+0000","last_fullsized":"2023-05-20T01:16:32.957177+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:49:54.418261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.954564+0000","last_change":"2023-05-20T01:16:28.343036+0000","last_active":"2023-05-20T01:16:32.954564+0000","last_peered":"2023-05-20T01:16:32.954564+0000","last_clean":"2023-05-20T01:16:32.954564+0000","last_became_active":"2023-05-20T01:16:28.342814+0000","last_became_peered":"2023-05-20T01:16:28.342814+0000","last_unstale":"2023-05-20T01:16:32.954564+0000","last_undegraded":"2023-05-20T01:16:32.954564+0000","last_fullsized":"2023-05-20T01:16:32.954564+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:13:57.518009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.954367+0000","last_change":"2023-05-20T01:16:32.356221+0000","last_active":"2023-05-20T01:16:32.954367+0000","last_peered":"2023-05-20T01:16:32.954367+0000","last_clean":"2023-05-20T01:16:32.954367+0000","last_became_active":"2023-05-20T01:16:32.355772+0000","last_became_peered":"2023-05-20T01:16:32.355772+0000","last_unstale":"2023-05-20T01:16:32.954367+0000","last_undegraded":"2023-05-20T01:16:32.954367+0000","last_fullsized":"2023-05-20T01:16:32.954367+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:07:08.123139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6,5],"acting":[0,1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.774068+0000","last_change":"2023-05-20T01:16:22.834330+0000","last_active":"2023-05-20T01:16:32.774068+0000","last_peered":"2023-05-20T01:16:32.774068+0000","last_clean":"2023-05-20T01:16:32.774068+0000","last_became_active":"2023-05-20T01:16:22.833946+0000","last_became_peered":"2023-05-20T01:16:22.833946+0000","last_unstale":"2023-05-20T01:16:32.774068+0000","last_undegraded":"2023-05-20T01:16:32.774068+0000","last_fullsized":"2023-05-20T01:16:32.774068+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:22:15.807596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.310582+0000","last_change":"2023-05-20T01:16:27.914231+0000","last_active":"2023-05-20T01:16:33.310582+0000","last_peered":"2023-05-20T01:16:33.310582+0000","last_clean":"2023-05-20T01:16:33.310582+0000","last_became_active":"2023-05-20T01:16:27.913946+0000","last_became_peered":"2023-05-20T01:16:27.913946+0000","last_unstale":"2023-05-20T01:16:33.310582+0000","last_undegraded":"2023-05-20T01:16:33.310582+0000","last_fullsized":"2023-05-20T01:16:33.310582+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:58:31.508225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.956705+0000","last_change":"2023-05-20T01:16:28.336767+0000","last_active":"2023-05-20T01:16:32.956705+0000","last_peered":"2023-05-20T01:16:32.956705+0000","last_clean":"2023-05-20T01:16:32.956705+0000","last_became_active":"2023-05-20T01:16:28.336172+0000","last_became_peered":"2023-05-20T01:16:28.336172+0000","last_unstale":"2023-05-20T01:16:32.956705+0000","last_undegraded":"2023-05-20T01:16:32.956705+0000","last_fullsized":"2023-05-20T01:16:32.956705+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:45:17.217540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.957277+0000","last_change":"2023-05-20T01:16:32.364585+0000","last_active":"2023-05-20T01:16:32.957277+0000","last_peered":"2023-05-20T01:16:32.957277+0000","last_clean":"2023-05-20T01:16:32.957277+0000","last_became_active":"2023-05-20T01:16:32.364096+0000","last_became_peered":"2023-05-20T01:16:32.364096+0000","last_unstale":"2023-05-20T01:16:32.957277+0000","last_undegraded":"2023-05-20T01:16:32.957277+0000","last_fullsized":"2023-05-20T01:16:32.957277+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:47:17.109451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,0],"acting":[1,3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"20'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.954518+0000","last_change":"2023-05-20T01:16:27.922900+0000","last_active":"2023-05-20T01:16:32.954518+0000","last_peered":"2023-05-20T01:16:32.954518+0000","last_clean":"2023-05-20T01:16:32.954518+0000","last_became_active":"2023-05-20T01:16:27.922597+0000","last_became_peered":"2023-05-20T01:16:27.922597+0000","last_unstale":"2023-05-20T01:16:32.954518+0000","last_undegraded":"2023-05-20T01:16:32.954518+0000","last_fullsized":"2023-05-20T01:16:32.954518+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:58:35.370081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.310293+0000","last_change":"2023-05-20T01:16:28.341446+0000","last_active":"2023-05-20T01:16:33.310293+0000","last_peered":"2023-05-20T01:16:33.310293+0000","last_clean":"2023-05-20T01:16:33.310293+0000","last_became_active":"2023-05-20T01:16:28.340943+0000","last_became_peered":"2023-05-20T01:16:28.340943+0000","last_unstale":"2023-05-20T01:16:33.310293+0000","last_undegraded":"2023-05-20T01:16:33.310293+0000","last_fullsized":"2023-05-20T01:16:33.310293+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:24:03.794336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.311921+0000","last_change":"2023-05-20T01:16:32.360851+0000","last_active":"2023-05-20T01:16:33.311921+0000","last_peered":"2023-05-20T01:16:33.311921+0000","last_clean":"2023-05-20T01:16:33.311921+0000","last_became_active":"2023-05-20T01:16:32.360366+0000","last_became_peered":"2023-05-20T01:16:32.360366+0000","last_unstale":"2023-05-20T01:16:33.311921+0000","last_undegraded":"2023-05-20T01:16:33.311921+0000","last_fullsized":"2023-05-20T01:16:33.311921+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:38:17.163216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1,6],"acting":[7,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.310447+0000","last_change":"2023-05-20T01:16:27.925398+0000","last_active":"2023-05-20T01:16:33.310447+0000","last_peered":"2023-05-20T01:16:33.310447+0000","last_clean":"2023-05-20T01:16:33.310447+0000","last_became_active":"2023-05-20T01:16:27.924943+0000","last_became_peered":"2023-05-20T01:16:27.924943+0000","last_unstale":"2023-05-20T01:16:33.310447+0000","last_undegraded":"2023-05-20T01:16:33.310447+0000","last_fullsized":"2023-05-20T01:16:33.310447+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:42:58.919423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.773260+0000","last_change":"2023-05-20T01:16:28.332091+0000","last_active":"2023-05-20T01:16:32.773260+0000","last_peered":"2023-05-20T01:16:32.773260+0000","last_clean":"2023-05-20T01:16:32.773260+0000","last_became_active":"2023-05-20T01:16:28.331785+0000","last_became_peered":"2023-05-20T01:16:28.331785+0000","last_unstale":"2023-05-20T01:16:32.773260+0000","last_undegraded":"2023-05-20T01:16:32.773260+0000","last_fullsized":"2023-05-20T01:16:32.773260+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:31:47.073658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.310796+0000","last_change":"2023-05-20T01:16:32.362120+0000","last_active":"2023-05-20T01:16:33.310796+0000","last_peered":"2023-05-20T01:16:33.310796+0000","last_clean":"2023-05-20T01:16:33.310796+0000","last_became_active":"2023-05-20T01:16:32.361727+0000","last_became_peered":"2023-05-20T01:16:32.361727+0000","last_unstale":"2023-05-20T01:16:33.310796+0000","last_undegraded":"2023-05-20T01:16:33.310796+0000","last_fullsized":"2023-05-20T01:16:33.310796+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:01:24.913389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7,4],"acting":[5,0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.310449+0000","last_change":"2023-05-20T01:16:27.925288+0000","last_active":"2023-05-20T01:16:33.310449+0000","last_peered":"2023-05-20T01:16:33.310449+0000","last_clean":"2023-05-20T01:16:33.310449+0000","last_became_active":"2023-05-20T01:16:27.924988+0000","last_became_peered":"2023-05-20T01:16:27.924988+0000","last_unstale":"2023-05-20T01:16:33.310449+0000","last_undegraded":"2023-05-20T01:16:33.310449+0000","last_fullsized":"2023-05-20T01:16:33.310449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:32:59.792286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.773973+0000","last_change":"2023-05-20T01:16:28.334255+0000","last_active":"2023-05-20T01:16:32.773973+0000","last_peered":"2023-05-20T01:16:32.773973+0000","last_clean":"2023-05-20T01:16:32.773973+0000","last_became_active":"2023-05-20T01:16:28.333966+0000","last_became_peered":"2023-05-20T01:16:28.333966+0000","last_unstale":"2023-05-20T01:16:32.773973+0000","last_undegraded":"2023-05-20T01:16:32.773973+0000","last_fullsized":"2023-05-20T01:16:32.773973+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:04:05.601168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.772980+0000","last_change":"2023-05-20T01:16:32.346878+0000","last_active":"2023-05-20T01:16:32.772980+0000","last_peered":"2023-05-20T01:16:32.772980+0000","last_clean":"2023-05-20T01:16:32.772980+0000","last_became_active":"2023-05-20T01:16:32.346493+0000","last_became_peered":"2023-05-20T01:16:32.346493+0000","last_unstale":"2023-05-20T01:16:32.772980+0000","last_undegraded":"2023-05-20T01:16:32.772980+0000","last_fullsized":"2023-05-20T01:16:32.772980+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:50:05.163209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,2],"acting":[4,6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"20'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.772568+0000","last_change":"2023-05-20T01:16:27.906889+0000","last_active":"2023-05-20T01:16:32.772568+0000","last_peered":"2023-05-20T01:16:32.772568+0000","last_clean":"2023-05-20T01:16:32.772568+0000","last_became_active":"2023-05-20T01:16:27.906574+0000","last_became_peered":"2023-05-20T01:16:27.906574+0000","last_unstale":"2023-05-20T01:16:32.772568+0000","last_undegraded":"2023-05-20T01:16:32.772568+0000","last_fullsized":"2023-05-20T01:16:32.772568+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:49:41.443268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.771941+0000","last_change":"2023-05-20T01:16:28.345350+0000","last_active":"2023-05-20T01:16:32.771941+0000","last_peered":"2023-05-20T01:16:32.771941+0000","last_clean":"2023-05-20T01:16:32.771941+0000","last_became_active":"2023-05-20T01:16:28.344838+0000","last_became_peered":"2023-05-20T01:16:28.344838+0000","last_unstale":"2023-05-20T01:16:32.771941+0000","last_undegraded":"2023-05-20T01:16:32.771941+0000","last_fullsized":"2023-05-20T01:16:32.771941+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:53:32.204259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.956979+0000","last_change":"2023-05-20T01:16:32.352947+0000","last_active":"2023-05-20T01:16:32.956979+0000","last_peered":"2023-05-20T01:16:32.956979+0000","last_clean":"2023-05-20T01:16:32.956979+0000","last_became_active":"2023-05-20T01:16:32.352495+0000","last_became_peered":"2023-05-20T01:16:32.352495+0000","last_unstale":"2023-05-20T01:16:32.956979+0000","last_undegraded":"2023-05-20T01:16:32.956979+0000","last_fullsized":"2023-05-20T01:16:32.956979+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:29:18.453884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7,2],"acting":[1,5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"20'4","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.954845+0000","last_change":"2023-05-20T01:16:27.917645+0000","last_active":"2023-05-20T01:16:32.954845+0000","last_peered":"2023-05-20T01:16:32.954845+0000","last_clean":"2023-05-20T01:16:32.954845+0000","last_became_active":"2023-05-20T01:16:27.917362+0000","last_became_peered":"2023-05-20T01:16:27.917362+0000","last_unstale":"2023-05-20T01:16:32.954845+0000","last_undegraded":"2023-05-20T01:16:32.954845+0000","last_fullsized":"2023-05-20T01:16:32.954845+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:54:11.307738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.773261+0000","last_change":"2023-05-20T01:16:28.347617+0000","last_active":"2023-05-20T01:16:32.773261+0000","last_peered":"2023-05-20T01:16:32.773261+0000","last_clean":"2023-05-20T01:16:32.773261+0000","last_became_active":"2023-05-20T01:16:28.347032+0000","last_became_peered":"2023-05-20T01:16:28.347032+0000","last_unstale":"2023-05-20T01:16:32.773261+0000","last_undegraded":"2023-05-20T01:16:32.773261+0000","last_fullsized":"2023-05-20T01:16:32.773261+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:34:36.196004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.773474+0000","last_change":"2023-05-20T01:16:28.336509+0000","last_active":"2023-05-20T01:16:32.773474+0000","last_peered":"2023-05-20T01:16:32.773474+0000","last_clean":"2023-05-20T01:16:32.773474+0000","last_became_active":"2023-05-20T01:16:28.336170+0000","last_became_peered":"2023-05-20T01:16:28.336170+0000","last_unstale":"2023-05-20T01:16:32.773474+0000","last_undegraded":"2023-05-20T01:16:32.773474+0000","last_fullsized":"2023-05-20T01:16:32.773474+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:16:31.755218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.771733+0000","last_change":"2023-05-20T01:16:28.340216+0000","last_active":"2023-05-20T01:16:32.771733+0000","last_peered":"2023-05-20T01:16:32.771733+0000","last_clean":"2023-05-20T01:16:32.771733+0000","last_became_active":"2023-05-20T01:16:28.339662+0000","last_became_peered":"2023-05-20T01:16:28.339662+0000","last_unstale":"2023-05-20T01:16:32.771733+0000","last_undegraded":"2023-05-20T01:16:32.771733+0000","last_fullsized":"2023-05-20T01:16:32.771733+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:54:12.315501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.774267+0000","last_change":"2023-05-20T01:16:28.347840+0000","last_active":"2023-05-20T01:16:32.774267+0000","last_peered":"2023-05-20T01:16:32.774267+0000","last_clean":"2023-05-20T01:16:32.774267+0000","last_became_active":"2023-05-20T01:16:28.347370+0000","last_became_peered":"2023-05-20T01:16:28.347370+0000","last_unstale":"2023-05-20T01:16:32.774267+0000","last_undegraded":"2023-05-20T01:16:32.774267+0000","last_fullsized":"2023-05-20T01:16:32.774267+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T01:49:49.650741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.772038+0000","last_change":"2023-05-20T01:16:28.328611+0000","last_active":"2023-05-20T01:16:32.772038+0000","last_peered":"2023-05-20T01:16:32.772038+0000","last_clean":"2023-05-20T01:16:32.772038+0000","last_became_active":"2023-05-20T01:16:28.328311+0000","last_became_peered":"2023-05-20T01:16:28.328311+0000","last_unstale":"2023-05-20T01:16:32.772038+0000","last_undegraded":"2023-05-20T01:16:32.772038+0000","last_fullsized":"2023-05-20T01:16:32.772038+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:09:39.569369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.957125+0000","last_change":"2023-05-20T01:16:28.336980+0000","last_active":"2023-05-20T01:16:32.957125+0000","last_peered":"2023-05-20T01:16:32.957125+0000","last_clean":"2023-05-20T01:16:32.957125+0000","last_became_active":"2023-05-20T01:16:28.336778+0000","last_became_peered":"2023-05-20T01:16:28.336778+0000","last_unstale":"2023-05-20T01:16:32.957125+0000","last_undegraded":"2023-05-20T01:16:32.957125+0000","last_fullsized":"2023-05-20T01:16:32.957125+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:03:20.405510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":4451,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":8902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9260,"kb_used_data":868,"kb_used_omap":0,"kb_used_meta":8384,"kb_avail":94362580,"statfs":{"total":96636764160,"available":96627281920,"internally_reserved":0,"allocated":888832,"data_stored":703638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8585216},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.314}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41799999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38800000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51000000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54100000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8076,"kb_used_data":260,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363764,"statfs":{"total":96636764160,"available":96628494336,"internally_reserved":0,"allocated":266240,"data_stored":109401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.433}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49399999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63200000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70199999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64900000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67900000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9232,"kb_used_data":840,"kb_used_omap":0,"kb_used_meta":8384,"kb_avail":94362608,"statfs":{"total":96636764160,"available":96627310592,"internally_reserved":0,"allocated":860160,"data_stored":699769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8585216},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40699999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28899999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32300000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64100000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54500000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8140,"kb_used_data":260,"kb_used_omap":0,"kb_used_meta":7872,"kb_avail":94363700,"statfs":{"total":96636764160,"available":96628428800,"internally_reserved":0,"allocated":266240,"data_stored":109022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8060928},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28000000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32000000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47299999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51900000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8076,"kb_used_data":260,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363764,"statfs":{"total":96636764160,"available":96628494336,"internally_reserved":0,"allocated":266240,"data_stored":109401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.252}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34200000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41899999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58899999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63400000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84199999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88100000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8144,"kb_used_data":264,"kb_used_omap":0,"kb_used_meta":7872,"kb_avail":94363696,"statfs":{"total":96636764160,"available":96628424704,"internally_reserved":0,"allocated":270336,"data_stored":109491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8060928},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28699999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70499999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8224,"kb_used_data":280,"kb_used_omap":0,"kb_used_meta":7936,"kb_avail":94363616,"statfs":{"total":96636764160,"available":96628342784,"internally_reserved":0,"allocated":286720,"data_stored":113110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8126464},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39600000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51600000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45400000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54200000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.5}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55800000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56899999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8144,"kb_used_data":264,"kb_used_omap":0,"kb_used_meta":7872,"kb_avail":94363696,"statfs":{"total":96636764160,"available":96628424704,"internally_reserved":0,"allocated":270336,"data_stored":109092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8060928},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67900000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73499999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61099999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59799999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51500000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-05-20T01:16:46.513 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-05-20T01:16:46.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.664+0000 7fbd8b455700 1 Processor -- start 2023-05-20T01:16:46.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.664+0000 7fbd8b455700 1 -- start start 2023-05-20T01:16:46.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.665+0000 7fbd8b455700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fbd8406e2d0 0x7fbd840cbd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:46.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.665+0000 7fbd8b455700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd8415e400 con 0x7fbd8415a950 2023-05-20T01:16:46.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.665+0000 7fbd8b455700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd840724c0 con 0x7fbd8406e7d0 2023-05-20T01:16:46.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.665+0000 7fbd8b455700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd8415ad20 con 0x7fbd8406e2d0 2023-05-20T01:16:46.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.665+0000 7fbd891f1700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fbd8415a950 0x7fbd840c9950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:41298/0 (socket says 172.21.15.16:41298) 2023-05-20T01:16:46.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.665+0000 7fbd891f1700 1 -- 172.21.15.16:0/2060076204 learned_addr learned my addr 172.21.15.16:0/2060076204 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:16:46.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.665+0000 7fbd889f0700 1 --2- 172.21.15.16:0/2060076204 >> v2:172.21.15.132:3300/0 conn(0x7fbd8406e2d0 0x7fbd840cbd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:46.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.665+0000 7fbd7bfff700 1 -- 172.21.15.16:0/2060076204 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3839219822 0 0) 0x7fbd8415e400 con 0x7fbd8415a950 2023-05-20T01:16:46.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.665+0000 7fbd7bfff700 1 -- 172.21.15.16:0/2060076204 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd700035b0 con 0x7fbd8415a950 2023-05-20T01:16:46.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.666+0000 7fbd7bfff700 1 -- 172.21.15.16:0/2060076204 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2808485061 0 0) 0x7fbd700035b0 con 0x7fbd8415a950 2023-05-20T01:16:46.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.666+0000 7fbd7bfff700 1 -- 172.21.15.16:0/2060076204 >> v2:172.21.15.132:3300/0 conn(0x7fbd8406e2d0 msgr2=0x7fbd840cbd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:46.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.666+0000 7fbd7bfff700 1 --2- 172.21.15.16:0/2060076204 >> v2:172.21.15.132:3300/0 conn(0x7fbd8406e2d0 0x7fbd840cbd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:46.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.666+0000 7fbd7bfff700 1 -- 172.21.15.16:0/2060076204 >> v1:172.21.15.132:6789/0 conn(0x7fbd8406e7d0 legacy=0x7fbd8415e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:46.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.666+0000 7fbd7bfff700 1 -- 172.21.15.16:0/2060076204 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd840b4de0 con 0x7fbd8415a950 2023-05-20T01:16:46.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.666+0000 7fbd7bfff700 1 -- 172.21.15.16:0/2060076204 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7fbd74001670 con 0x7fbd8415a950 2023-05-20T01:16:46.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.666+0000 7fbd889f0700 1 --2- 172.21.15.16:0/2060076204 >> v2:172.21.15.132:3300/0 conn(0x7fbd8406e2d0 0x7fbd840cbd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-05-20T01:16:46.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.666+0000 7fbd7bfff700 1 -- 172.21.15.16:0/2060076204 >> v1:172.21.15.16:6789/0 conn(0x7fbd8415a950 legacy=0x7fbd840c9950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:46.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.666+0000 7fbd7bfff700 1 --2- 172.21.15.16:0/2060076204 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbd70003a70 0x7fbd70003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:46.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.666+0000 7fbd7bfff700 1 --2- 172.21.15.16:0/2060076204 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbd70004850 0x7fbd70004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:46.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.666+0000 7fbd7bfff700 1 --2- 172.21.15.16:0/2060076204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd70005250 0x7fbd7000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:46.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.666+0000 7fbd7bfff700 1 -- 172.21.15.16:0/2060076204 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd8415ad20 con 0x7fbd70005250 2023-05-20T01:16:46.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.666+0000 7fbd7bfff700 1 -- 172.21.15.16:0/2060076204 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd7000c190 con 0x7fbd70004850 2023-05-20T01:16:46.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.666+0000 7fbd7bfff700 1 -- 172.21.15.16:0/2060076204 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd7000c350 con 0x7fbd70003a70 2023-05-20T01:16:46.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.666+0000 7fbd899f2700 1 --2- 172.21.15.16:0/2060076204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd70005250 0x7fbd7000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:46.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.666+0000 7fbd891f1700 1 --2- 172.21.15.16:0/2060076204 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbd70003a70 0x7fbd70003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:46.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.666+0000 7fbd889f0700 1 --2- 172.21.15.16:0/2060076204 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbd70004850 0x7fbd70004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:46.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.666+0000 7fbd899f2700 1 -- 172.21.15.16:0/2060076204 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbd70003a70 msgr2=0x7fbd70003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:46.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.666+0000 7fbd899f2700 1 --2- 172.21.15.16:0/2060076204 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbd70003a70 0x7fbd70003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:46.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.666+0000 7fbd899f2700 1 -- 172.21.15.16:0/2060076204 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbd70004850 msgr2=0x7fbd70004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:46.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.666+0000 7fbd899f2700 1 --2- 172.21.15.16:0/2060076204 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbd70004850 0x7fbd70004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:46.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.666+0000 7fbd899f2700 1 -- 172.21.15.16:0/2060076204 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd700035b0 con 0x7fbd70005250 2023-05-20T01:16:46.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.666+0000 7fbd899f2700 1 -- 172.21.15.16:0/2060076204 reap_dead start 2023-05-20T01:16:46.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.666+0000 7fbd899f2700 1 --2- 172.21.15.16:0/2060076204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd70005250 0x7fbd7000ba10 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c00c160 tx=0x7fbd70005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79ecb9e661907570 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:46.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.666+0000 7fbd7bfff700 1 -- 172.21.15.16:0/2060076204 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd7c001db0 con 0x7fbd70005250 2023-05-20T01:16:46.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.667+0000 7fbd7bfff700 1 -- 172.21.15.16:0/2060076204 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd7c00cca0 con 0x7fbd70005250 2023-05-20T01:16:46.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.667+0000 7fbd7bfff700 1 -- 172.21.15.16:0/2060076204 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd7c003c20 con 0x7fbd70005250 2023-05-20T01:16:46.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.667+0000 7fbd8b455700 1 -- 172.21.15.16:0/2060076204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd70005250 msgr2=0x7fbd7000ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:46.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.667+0000 7fbd8b455700 1 --2- 172.21.15.16:0/2060076204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd70005250 0x7fbd7000ba10 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c00c160 tx=0x7fbd70005880 comp rx=0 tx=0).stop 2023-05-20T01:16:46.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.667+0000 7fbd8b455700 1 -- 172.21.15.16:0/2060076204 shutdown_connections 2023-05-20T01:16:46.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.667+0000 7fbd8b455700 1 --2- 172.21.15.16:0/2060076204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd70005250 0x7fbd7000ba10 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:46.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.667+0000 7fbd8b455700 1 -- 172.21.15.16:0/2060076204 >> 172.21.15.16:0/2060076204 conn(0x7fbd840b0e90 msgr2=0x7fbd840b1260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:46.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.667+0000 7fbd8b455700 1 -- 172.21.15.16:0/2060076204 shutdown_connections 2023-05-20T01:16:46.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.667+0000 7fbd8b455700 1 -- 172.21.15.16:0/2060076204 wait complete. 2023-05-20T01:16:46.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.668+0000 7fbd8b455700 1 Processor -- start 2023-05-20T01:16:46.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.669+0000 7fbd8b455700 1 -- start start 2023-05-20T01:16:46.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.669+0000 7fbd8b455700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbd70005250 0x7fbd84142840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:46.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.669+0000 7fbd8b455700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbd84156380 0x7fbd840c9de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:46.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.669+0000 7fbd8b455700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd841567f0 0x7fbd840ca320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:46.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.669+0000 7fbd8b455700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd840c6d60 con 0x7fbd841567f0 2023-05-20T01:16:46.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.669+0000 7fbd8b455700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd840c6470 con 0x7fbd70005250 2023-05-20T01:16:46.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.669+0000 7fbd8b455700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd840c6710 con 0x7fbd84156380 2023-05-20T01:16:46.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.669+0000 7fbd899f2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd841567f0 0x7fbd840ca320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:46.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.669+0000 7fbd889f0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbd84156380 0x7fbd840c9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:46.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.669+0000 7fbd891f1700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbd70005250 0x7fbd84142840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:46.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.669+0000 7fbd899f2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd841567f0 0x7fbd840ca320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58018/0 (socket says 172.21.15.16:58018) 2023-05-20T01:16:46.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.669+0000 7fbd899f2700 1 -- 172.21.15.16:0/3442255295 learned_addr learned my addr 172.21.15.16:0/3442255295 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:46.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.670+0000 7fbd899f2700 1 -- 172.21.15.16:0/3442255295 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbd84156380 msgr2=0x7fbd840c9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:46.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.670+0000 7fbd899f2700 1 --2- 172.21.15.16:0/3442255295 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbd84156380 0x7fbd840c9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:46.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.670+0000 7fbd899f2700 1 -- 172.21.15.16:0/3442255295 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbd70005250 msgr2=0x7fbd84142840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:46.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.670+0000 7fbd899f2700 1 --2- 172.21.15.16:0/3442255295 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbd70005250 0x7fbd84142840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:46.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.670+0000 7fbd899f2700 1 -- 172.21.15.16:0/3442255295 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd8406ee20 con 0x7fbd841567f0 2023-05-20T01:16:46.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.670+0000 7fbd899f2700 1 --2- 172.21.15.16:0/3442255295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd841567f0 0x7fbd840ca320 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c00c160 tx=0x7fbd7c0014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:46.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.670+0000 7fbd79ffb700 1 -- 172.21.15.16:0/3442255295 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd7c0052d0 con 0x7fbd841567f0 2023-05-20T01:16:46.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.670+0000 7fbd79ffb700 1 -- 172.21.15.16:0/3442255295 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd7c005430 con 0x7fbd841567f0 2023-05-20T01:16:46.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.670+0000 7fbd8b455700 1 -- 172.21.15.16:0/3442255295 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd840b50c0 con 0x7fbd841567f0 2023-05-20T01:16:46.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.670+0000 7fbd79ffb700 1 -- 172.21.15.16:0/3442255295 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd7c0057e0 con 0x7fbd841567f0 2023-05-20T01:16:46.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.670+0000 7fbd8b455700 1 -- 172.21.15.16:0/3442255295 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd840727a0 con 0x7fbd841567f0 2023-05-20T01:16:46.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.671+0000 7fbd8b455700 1 -- 172.21.15.16:0/3442255295 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd841e57d0 con 0x7fbd841567f0 2023-05-20T01:16:46.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.672+0000 7fbd79ffb700 1 -- 172.21.15.16:0/3442255295 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fbd7c01d480 con 0x7fbd841567f0 2023-05-20T01:16:46.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.672+0000 7fbd79ffb700 1 --2- 172.21.15.16:0/3442255295 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fbd6806ace0 0x7fbd6806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:46.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.672+0000 7fbd79ffb700 1 -- 172.21.15.16:0/3442255295 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fbd7c08a490 con 0x7fbd841567f0 2023-05-20T01:16:46.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.672+0000 7fbd891f1700 1 --2- 172.21.15.16:0/3442255295 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fbd6806ace0 0x7fbd6806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:46.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.673+0000 7fbd891f1700 1 --2- 172.21.15.16:0/3442255295 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fbd6806ace0 0x7fbd6806d190 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fbd74000c00 tx=0x7fbd74001bf0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:46.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.677+0000 7fbd79ffb700 1 -- 172.21.15.16:0/3442255295 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fbd7c0552e0 con 0x7fbd841567f0 2023-05-20T01:16:46.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.874+0000 7fbd8b455700 1 -- 172.21.15.16:0/3442255295 --> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbd8406e0b0 con 0x7fbd6806ace0 2023-05-20T01:16:46.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.884+0000 7fbd79ffb700 1 -- 172.21.15.16:0/3442255295 <== mgr.4104 v2:172.21.15.16:6832/3817237998 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+367552 (secure 0 0 0) 0x7fbd8406e0b0 con 0x7fbd6806ace0 2023-05-20T01:16:46.893 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:16:46.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.888+0000 7fbd8b455700 1 -- 172.21.15.16:0/3442255295 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fbd6806ace0 msgr2=0x7fbd6806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:46.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.888+0000 7fbd8b455700 1 --2- 172.21.15.16:0/3442255295 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fbd6806ace0 0x7fbd6806d190 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fbd74000c00 tx=0x7fbd74001bf0 comp rx=0 tx=0).stop 2023-05-20T01:16:46.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.888+0000 7fbd8b455700 1 -- 172.21.15.16:0/3442255295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd841567f0 msgr2=0x7fbd840ca320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:46.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.888+0000 7fbd8b455700 1 --2- 172.21.15.16:0/3442255295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd841567f0 0x7fbd840ca320 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c00c160 tx=0x7fbd7c0014a0 comp rx=0 tx=0).stop 2023-05-20T01:16:46.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.888+0000 7fbd8b455700 1 -- 172.21.15.16:0/3442255295 shutdown_connections 2023-05-20T01:16:46.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.888+0000 7fbd8b455700 1 --2- 172.21.15.16:0/3442255295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd841567f0 0x7fbd840ca320 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:46.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.888+0000 7fbd8b455700 1 --2- 172.21.15.16:0/3442255295 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbd70005250 0x7fbd84142840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:46.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.888+0000 7fbd8b455700 1 --2- 172.21.15.16:0/3442255295 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fbd6806ace0 0x7fbd6806d190 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:46.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.888+0000 7fbd8b455700 1 --2- 172.21.15.16:0/3442255295 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbd84156380 0x7fbd840c9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:46.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.888+0000 7fbd8b455700 1 -- 172.21.15.16:0/3442255295 >> 172.21.15.16:0/3442255295 conn(0x7fbd840b0e90 msgr2=0x7fbd840b1260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:46.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.888+0000 7fbd8b455700 1 -- 172.21.15.16:0/3442255295 shutdown_connections 2023-05-20T01:16:46.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:46.888+0000 7fbd8b455700 1 -- 172.21.15.16:0/3442255295 wait complete. 2023-05-20T01:16:46.898 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2023-05-20T01:16:46.904 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":31,"stamp":"2023-05-20T01:16:45.321252+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594819,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":206,"num_write_kb":2433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158,"ondisk_log_size":158,"up":338,"acting":338,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":338,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":67296,"kb_used_data":3296,"kb_used_omap":0,"kb_used_meta":63936,"kb_avail":754907424,"statfs":{"total":773094113280,"available":773025202176,"internally_reserved":0,"allocated":3375104,"data_stored":2062924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":65470464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2089,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002510"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.310521+0000","last_change":"2023-05-20T01:16:28.345449+0000","last_active":"2023-05-20T01:16:33.310521+0000","last_peered":"2023-05-20T01:16:33.310521+0000","last_clean":"2023-05-20T01:16:33.310521+0000","last_became_active":"2023-05-20T01:16:28.345161+0000","last_became_peered":"2023-05-20T01:16:28.345161+0000","last_unstale":"2023-05-20T01:16:33.310521+0000","last_undegraded":"2023-05-20T01:16:33.310521+0000","last_fullsized":"2023-05-20T01:16:33.310521+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:10:54.451689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.310465+0000","last_change":"2023-05-20T01:16:28.337973+0000","last_active":"2023-05-20T01:16:33.310465+0000","last_peered":"2023-05-20T01:16:33.310465+0000","last_clean":"2023-05-20T01:16:33.310465+0000","last_became_active":"2023-05-20T01:16:28.337679+0000","last_became_peered":"2023-05-20T01:16:28.337679+0000","last_unstale":"2023-05-20T01:16:33.310465+0000","last_undegraded":"2023-05-20T01:16:33.310465+0000","last_fullsized":"2023-05-20T01:16:33.310465+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:59:48.441604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.774449+0000","last_change":"2023-05-20T01:16:28.342781+0000","last_active":"2023-05-20T01:16:32.774449+0000","last_peered":"2023-05-20T01:16:32.774449+0000","last_clean":"2023-05-20T01:16:32.774449+0000","last_became_active":"2023-05-20T01:16:28.342488+0000","last_became_peered":"2023-05-20T01:16:28.342488+0000","last_unstale":"2023-05-20T01:16:32.774449+0000","last_undegraded":"2023-05-20T01:16:32.774449+0000","last_fullsized":"2023-05-20T01:16:32.774449+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:35:02.629395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.773758+0000","last_change":"2023-05-20T01:16:28.345746+0000","last_active":"2023-05-20T01:16:32.773758+0000","last_peered":"2023-05-20T01:16:32.773758+0000","last_clean":"2023-05-20T01:16:32.773758+0000","last_became_active":"2023-05-20T01:16:28.345477+0000","last_became_peered":"2023-05-20T01:16:28.345477+0000","last_unstale":"2023-05-20T01:16:32.773758+0000","last_undegraded":"2023-05-20T01:16:32.773758+0000","last_fullsized":"2023-05-20T01:16:32.773758+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:08:32.412558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.954652+0000","last_change":"2023-05-20T01:16:28.352984+0000","last_active":"2023-05-20T01:16:32.954652+0000","last_peered":"2023-05-20T01:16:32.954652+0000","last_clean":"2023-05-20T01:16:32.954652+0000","last_became_active":"2023-05-20T01:16:28.352711+0000","last_became_peered":"2023-05-20T01:16:28.352711+0000","last_unstale":"2023-05-20T01:16:32.954652+0000","last_undegraded":"2023-05-20T01:16:32.954652+0000","last_fullsized":"2023-05-20T01:16:32.954652+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:34:08.601566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.774271+0000","last_change":"2023-05-20T01:16:28.347827+0000","last_active":"2023-05-20T01:16:32.774271+0000","last_peered":"2023-05-20T01:16:32.774271+0000","last_clean":"2023-05-20T01:16:32.774271+0000","last_became_active":"2023-05-20T01:16:28.347476+0000","last_became_peered":"2023-05-20T01:16:28.347476+0000","last_unstale":"2023-05-20T01:16:32.774271+0000","last_undegraded":"2023-05-20T01:16:32.774271+0000","last_fullsized":"2023-05-20T01:16:32.774271+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:21:02.720695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.311877+0000","last_change":"2023-05-20T01:16:28.338564+0000","last_active":"2023-05-20T01:16:33.311877+0000","last_peered":"2023-05-20T01:16:33.311877+0000","last_clean":"2023-05-20T01:16:33.311877+0000","last_became_active":"2023-05-20T01:16:28.337927+0000","last_became_peered":"2023-05-20T01:16:28.337927+0000","last_unstale":"2023-05-20T01:16:33.311877+0000","last_undegraded":"2023-05-20T01:16:33.311877+0000","last_fullsized":"2023-05-20T01:16:33.311877+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:24:36.047155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.310151+0000","last_change":"2023-05-20T01:16:28.340008+0000","last_active":"2023-05-20T01:16:33.310151+0000","last_peered":"2023-05-20T01:16:33.310151+0000","last_clean":"2023-05-20T01:16:33.310151+0000","last_became_active":"2023-05-20T01:16:28.339724+0000","last_became_peered":"2023-05-20T01:16:28.339724+0000","last_unstale":"2023-05-20T01:16:33.310151+0000","last_undegraded":"2023-05-20T01:16:33.310151+0000","last_fullsized":"2023-05-20T01:16:33.310151+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:55:50.059228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.954657+0000","last_change":"2023-05-20T01:16:28.351002+0000","last_active":"2023-05-20T01:16:32.954657+0000","last_peered":"2023-05-20T01:16:32.954657+0000","last_clean":"2023-05-20T01:16:32.954657+0000","last_became_active":"2023-05-20T01:16:28.350672+0000","last_became_peered":"2023-05-20T01:16:28.350672+0000","last_unstale":"2023-05-20T01:16:32.954657+0000","last_undegraded":"2023-05-20T01:16:32.954657+0000","last_fullsized":"2023-05-20T01:16:32.954657+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:23:33.732284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.310970+0000","last_change":"2023-05-20T01:16:28.349410+0000","last_active":"2023-05-20T01:16:33.310970+0000","last_peered":"2023-05-20T01:16:33.310970+0000","last_clean":"2023-05-20T01:16:33.310970+0000","last_became_active":"2023-05-20T01:16:28.349177+0000","last_became_peered":"2023-05-20T01:16:28.349177+0000","last_unstale":"2023-05-20T01:16:33.310970+0000","last_undegraded":"2023-05-20T01:16:33.310970+0000","last_fullsized":"2023-05-20T01:16:33.310970+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:34:08.601566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.774486+0000","last_change":"2023-05-20T01:16:28.338627+0000","last_active":"2023-05-20T01:16:32.774486+0000","last_peered":"2023-05-20T01:16:32.774486+0000","last_clean":"2023-05-20T01:16:32.774486+0000","last_became_active":"2023-05-20T01:16:28.338363+0000","last_became_peered":"2023-05-20T01:16:28.338363+0000","last_unstale":"2023-05-20T01:16:32.774486+0000","last_undegraded":"2023-05-20T01:16:32.774486+0000","last_fullsized":"2023-05-20T01:16:32.774486+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:20:15.094065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.310156+0000","last_change":"2023-05-20T01:16:28.341603+0000","last_active":"2023-05-20T01:16:33.310156+0000","last_peered":"2023-05-20T01:16:33.310156+0000","last_clean":"2023-05-20T01:16:33.310156+0000","last_became_active":"2023-05-20T01:16:28.341242+0000","last_became_peered":"2023-05-20T01:16:28.341242+0000","last_unstale":"2023-05-20T01:16:33.310156+0000","last_undegraded":"2023-05-20T01:16:33.310156+0000","last_fullsized":"2023-05-20T01:16:33.310156+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:15:50.986060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.310269+0000","last_change":"2023-05-20T01:16:28.345977+0000","last_active":"2023-05-20T01:16:33.310269+0000","last_peered":"2023-05-20T01:16:33.310269+0000","last_clean":"2023-05-20T01:16:33.310269+0000","last_became_active":"2023-05-20T01:16:28.345732+0000","last_became_peered":"2023-05-20T01:16:28.345732+0000","last_unstale":"2023-05-20T01:16:33.310269+0000","last_undegraded":"2023-05-20T01:16:33.310269+0000","last_fullsized":"2023-05-20T01:16:33.310269+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:39:45.746298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.311067+0000","last_change":"2023-05-20T01:16:28.334227+0000","last_active":"2023-05-20T01:16:33.311067+0000","last_peered":"2023-05-20T01:16:33.311067+0000","last_clean":"2023-05-20T01:16:33.311067+0000","last_became_active":"2023-05-20T01:16:28.333922+0000","last_became_peered":"2023-05-20T01:16:28.333922+0000","last_unstale":"2023-05-20T01:16:33.311067+0000","last_undegraded":"2023-05-20T01:16:33.311067+0000","last_fullsized":"2023-05-20T01:16:33.311067+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:23:33.732284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.311509+0000","last_change":"2023-05-20T01:16:28.338772+0000","last_active":"2023-05-20T01:16:33.311509+0000","last_peered":"2023-05-20T01:16:33.311509+0000","last_clean":"2023-05-20T01:16:33.311509+0000","last_became_active":"2023-05-20T01:16:28.338300+0000","last_became_peered":"2023-05-20T01:16:28.338300+0000","last_unstale":"2023-05-20T01:16:33.311509+0000","last_undegraded":"2023-05-20T01:16:33.311509+0000","last_fullsized":"2023-05-20T01:16:33.311509+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:19:51.520080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.774491+0000","last_change":"2023-05-20T01:16:28.343926+0000","last_active":"2023-05-20T01:16:32.774491+0000","last_peered":"2023-05-20T01:16:32.774491+0000","last_clean":"2023-05-20T01:16:32.774491+0000","last_became_active":"2023-05-20T01:16:28.343696+0000","last_became_peered":"2023-05-20T01:16:28.343696+0000","last_unstale":"2023-05-20T01:16:32.774491+0000","last_undegraded":"2023-05-20T01:16:32.774491+0000","last_fullsized":"2023-05-20T01:16:32.774491+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:42:48.561844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.954664+0000","last_change":"2023-05-20T01:16:28.350876+0000","last_active":"2023-05-20T01:16:32.954664+0000","last_peered":"2023-05-20T01:16:32.954664+0000","last_clean":"2023-05-20T01:16:32.954664+0000","last_became_active":"2023-05-20T01:16:28.350242+0000","last_became_peered":"2023-05-20T01:16:28.350242+0000","last_unstale":"2023-05-20T01:16:32.954664+0000","last_undegraded":"2023-05-20T01:16:32.954664+0000","last_fullsized":"2023-05-20T01:16:32.954664+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:19:51.520080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.773740+0000","last_change":"2023-05-20T01:16:28.334794+0000","last_active":"2023-05-20T01:16:32.773740+0000","last_peered":"2023-05-20T01:16:32.773740+0000","last_clean":"2023-05-20T01:16:32.773740+0000","last_became_active":"2023-05-20T01:16:28.334535+0000","last_became_peered":"2023-05-20T01:16:28.334535+0000","last_unstale":"2023-05-20T01:16:32.773740+0000","last_undegraded":"2023-05-20T01:16:32.773740+0000","last_fullsized":"2023-05-20T01:16:32.773740+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T01:37:30.926061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.310464+0000","last_change":"2023-05-20T01:16:28.347597+0000","last_active":"2023-05-20T01:16:33.310464+0000","last_peered":"2023-05-20T01:16:33.310464+0000","last_clean":"2023-05-20T01:16:33.310464+0000","last_became_active":"2023-05-20T01:16:28.347245+0000","last_became_peered":"2023-05-20T01:16:28.347245+0000","last_unstale":"2023-05-20T01:16:33.310464+0000","last_undegraded":"2023-05-20T01:16:33.310464+0000","last_fullsized":"2023-05-20T01:16:33.310464+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:02:17.539187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.773711+0000","last_change":"2023-05-20T01:16:28.347994+0000","last_active":"2023-05-20T01:16:32.773711+0000","last_peered":"2023-05-20T01:16:32.773711+0000","last_clean":"2023-05-20T01:16:32.773711+0000","last_became_active":"2023-05-20T01:16:28.347690+0000","last_became_peered":"2023-05-20T01:16:28.347690+0000","last_unstale":"2023-05-20T01:16:32.773711+0000","last_undegraded":"2023-05-20T01:16:32.773711+0000","last_fullsized":"2023-05-20T01:16:32.773711+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:46:24.892963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.311730+0000","last_change":"2023-05-20T01:16:28.346262+0000","last_active":"2023-05-20T01:16:33.311730+0000","last_peered":"2023-05-20T01:16:33.311730+0000","last_clean":"2023-05-20T01:16:33.311730+0000","last_became_active":"2023-05-20T01:16:28.345513+0000","last_became_peered":"2023-05-20T01:16:28.345513+0000","last_unstale":"2023-05-20T01:16:33.311730+0000","last_undegraded":"2023-05-20T01:16:33.311730+0000","last_fullsized":"2023-05-20T01:16:33.311730+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:46:14.388881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.954206+0000","last_change":"2023-05-20T01:16:28.344269+0000","last_active":"2023-05-20T01:16:32.954206+0000","last_peered":"2023-05-20T01:16:32.954206+0000","last_clean":"2023-05-20T01:16:32.954206+0000","last_became_active":"2023-05-20T01:16:28.343991+0000","last_became_peered":"2023-05-20T01:16:28.343991+0000","last_unstale":"2023-05-20T01:16:32.954206+0000","last_undegraded":"2023-05-20T01:16:32.954206+0000","last_fullsized":"2023-05-20T01:16:32.954206+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:33:00.800050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.771776+0000","last_change":"2023-05-20T01:16:28.344552+0000","last_active":"2023-05-20T01:16:32.771776+0000","last_peered":"2023-05-20T01:16:32.771776+0000","last_clean":"2023-05-20T01:16:32.771776+0000","last_became_active":"2023-05-20T01:16:28.344125+0000","last_became_peered":"2023-05-20T01:16:28.344125+0000","last_unstale":"2023-05-20T01:16:32.771776+0000","last_undegraded":"2023-05-20T01:16:32.771776+0000","last_fullsized":"2023-05-20T01:16:32.771776+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:06:59.447318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.311284+0000","last_change":"2023-05-20T01:16:28.345899+0000","last_active":"2023-05-20T01:16:33.311284+0000","last_peered":"2023-05-20T01:16:33.311284+0000","last_clean":"2023-05-20T01:16:33.311284+0000","last_became_active":"2023-05-20T01:16:28.345629+0000","last_became_peered":"2023-05-20T01:16:28.345629+0000","last_unstale":"2023-05-20T01:16:33.311284+0000","last_undegraded":"2023-05-20T01:16:33.311284+0000","last_fullsized":"2023-05-20T01:16:33.311284+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:13:57.518009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.957468+0000","last_change":"2023-05-20T01:16:28.346168+0000","last_active":"2023-05-20T01:16:32.957468+0000","last_peered":"2023-05-20T01:16:32.957468+0000","last_clean":"2023-05-20T01:16:32.957468+0000","last_became_active":"2023-05-20T01:16:28.345889+0000","last_became_peered":"2023-05-20T01:16:28.345889+0000","last_unstale":"2023-05-20T01:16:32.957468+0000","last_undegraded":"2023-05-20T01:16:32.957468+0000","last_fullsized":"2023-05-20T01:16:32.957468+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:05:20.974728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.957396+0000","last_change":"2023-05-20T01:16:28.335035+0000","last_active":"2023-05-20T01:16:32.957396+0000","last_peered":"2023-05-20T01:16:32.957396+0000","last_clean":"2023-05-20T01:16:32.957396+0000","last_became_active":"2023-05-20T01:16:28.334787+0000","last_became_peered":"2023-05-20T01:16:28.334787+0000","last_unstale":"2023-05-20T01:16:32.957396+0000","last_undegraded":"2023-05-20T01:16:32.957396+0000","last_fullsized":"2023-05-20T01:16:32.957396+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:55:50.059228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.310780+0000","last_change":"2023-05-20T01:16:28.335038+0000","last_active":"2023-05-20T01:16:33.310780+0000","last_peered":"2023-05-20T01:16:33.310780+0000","last_clean":"2023-05-20T01:16:33.310780+0000","last_became_active":"2023-05-20T01:16:28.334750+0000","last_became_peered":"2023-05-20T01:16:28.334750+0000","last_unstale":"2023-05-20T01:16:33.310780+0000","last_undegraded":"2023-05-20T01:16:33.310780+0000","last_fullsized":"2023-05-20T01:16:33.310780+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:13:57.518009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.774853+0000","last_change":"2023-05-20T01:16:32.363983+0000","last_active":"2023-05-20T01:16:32.774853+0000","last_peered":"2023-05-20T01:16:32.774853+0000","last_clean":"2023-05-20T01:16:32.774853+0000","last_became_active":"2023-05-20T01:16:32.363545+0000","last_became_peered":"2023-05-20T01:16:32.363545+0000","last_unstale":"2023-05-20T01:16:32.774853+0000","last_undegraded":"2023-05-20T01:16:32.774853+0000","last_fullsized":"2023-05-20T01:16:32.774853+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:15:31.121265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2,0],"acting":[4,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"20'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.774070+0000","last_change":"2023-05-20T01:16:27.924758+0000","last_active":"2023-05-20T01:16:32.774070+0000","last_peered":"2023-05-20T01:16:32.774070+0000","last_clean":"2023-05-20T01:16:32.774070+0000","last_became_active":"2023-05-20T01:16:27.924471+0000","last_became_peered":"2023-05-20T01:16:27.924471+0000","last_unstale":"2023-05-20T01:16:32.774070+0000","last_undegraded":"2023-05-20T01:16:32.774070+0000","last_fullsized":"2023-05-20T01:16:32.774070+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:40:53.545036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.772324+0000","last_change":"2023-05-20T01:16:28.340516+0000","last_active":"2023-05-20T01:16:32.772324+0000","last_peered":"2023-05-20T01:16:32.772324+0000","last_clean":"2023-05-20T01:16:32.772324+0000","last_became_active":"2023-05-20T01:16:28.340255+0000","last_became_peered":"2023-05-20T01:16:28.340255+0000","last_unstale":"2023-05-20T01:16:32.772324+0000","last_undegraded":"2023-05-20T01:16:32.772324+0000","last_fullsized":"2023-05-20T01:16:32.772324+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:17:27.361087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.774064+0000","last_change":"2023-05-20T01:16:32.348270+0000","last_active":"2023-05-20T01:16:32.774064+0000","last_peered":"2023-05-20T01:16:32.774064+0000","last_clean":"2023-05-20T01:16:32.774064+0000","last_became_active":"2023-05-20T01:16:32.347460+0000","last_became_peered":"2023-05-20T01:16:32.347460+0000","last_unstale":"2023-05-20T01:16:32.774064+0000","last_undegraded":"2023-05-20T01:16:32.774064+0000","last_fullsized":"2023-05-20T01:16:32.774064+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:49:18.438950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3,7],"acting":[6,4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.956713+0000","last_change":"2023-05-20T01:16:27.922074+0000","last_active":"2023-05-20T01:16:32.956713+0000","last_peered":"2023-05-20T01:16:32.956713+0000","last_clean":"2023-05-20T01:16:32.956713+0000","last_became_active":"2023-05-20T01:16:27.921759+0000","last_became_peered":"2023-05-20T01:16:27.921759+0000","last_unstale":"2023-05-20T01:16:32.956713+0000","last_undegraded":"2023-05-20T01:16:32.956713+0000","last_fullsized":"2023-05-20T01:16:32.956713+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:08:26.231228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.954625+0000","last_change":"2023-05-20T01:16:28.348595+0000","last_active":"2023-05-20T01:16:32.954625+0000","last_peered":"2023-05-20T01:16:32.954625+0000","last_clean":"2023-05-20T01:16:32.954625+0000","last_became_active":"2023-05-20T01:16:28.347911+0000","last_became_peered":"2023-05-20T01:16:28.347911+0000","last_unstale":"2023-05-20T01:16:32.954625+0000","last_undegraded":"2023-05-20T01:16:32.954625+0000","last_fullsized":"2023-05-20T01:16:32.954625+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:46:42.721514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.954742+0000","last_change":"2023-05-20T01:16:32.362037+0000","last_active":"2023-05-20T01:16:32.954742+0000","last_peered":"2023-05-20T01:16:32.954742+0000","last_clean":"2023-05-20T01:16:32.954742+0000","last_became_active":"2023-05-20T01:16:32.361564+0000","last_became_peered":"2023-05-20T01:16:32.361564+0000","last_unstale":"2023-05-20T01:16:32.954742+0000","last_undegraded":"2023-05-20T01:16:32.954742+0000","last_fullsized":"2023-05-20T01:16:32.954742+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:14:35.048109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1,2],"acting":[0,7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.771855+0000","last_change":"2023-05-20T01:16:27.910268+0000","last_active":"2023-05-20T01:16:32.771855+0000","last_peered":"2023-05-20T01:16:32.771855+0000","last_clean":"2023-05-20T01:16:32.771855+0000","last_became_active":"2023-05-20T01:16:27.909963+0000","last_became_peered":"2023-05-20T01:16:27.909963+0000","last_unstale":"2023-05-20T01:16:32.771855+0000","last_undegraded":"2023-05-20T01:16:32.771855+0000","last_fullsized":"2023-05-20T01:16:32.771855+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:22:46.049477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.774156+0000","last_change":"2023-05-20T01:16:28.345078+0000","last_active":"2023-05-20T01:16:32.774156+0000","last_peered":"2023-05-20T01:16:32.774156+0000","last_clean":"2023-05-20T01:16:32.774156+0000","last_became_active":"2023-05-20T01:16:28.344742+0000","last_became_peered":"2023-05-20T01:16:28.344742+0000","last_unstale":"2023-05-20T01:16:32.774156+0000","last_undegraded":"2023-05-20T01:16:32.774156+0000","last_fullsized":"2023-05-20T01:16:32.774156+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:53:18.904132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.773906+0000","last_change":"2023-05-20T01:16:32.348805+0000","last_active":"2023-05-20T01:16:32.773906+0000","last_peered":"2023-05-20T01:16:32.773906+0000","last_clean":"2023-05-20T01:16:32.773906+0000","last_became_active":"2023-05-20T01:16:32.348352+0000","last_became_peered":"2023-05-20T01:16:32.348352+0000","last_unstale":"2023-05-20T01:16:32.773906+0000","last_undegraded":"2023-05-20T01:16:32.773906+0000","last_fullsized":"2023-05-20T01:16:32.773906+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:04:37.665737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7,1],"acting":[6,4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.954805+0000","last_change":"2023-05-20T01:16:27.915492+0000","last_active":"2023-05-20T01:16:32.954805+0000","last_peered":"2023-05-20T01:16:32.954805+0000","last_clean":"2023-05-20T01:16:32.954805+0000","last_became_active":"2023-05-20T01:16:27.915015+0000","last_became_peered":"2023-05-20T01:16:27.915015+0000","last_unstale":"2023-05-20T01:16:32.954805+0000","last_undegraded":"2023-05-20T01:16:32.954805+0000","last_fullsized":"2023-05-20T01:16:32.954805+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:25:21.983559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.771743+0000","last_change":"2023-05-20T01:16:28.339100+0000","last_active":"2023-05-20T01:16:32.771743+0000","last_peered":"2023-05-20T01:16:32.771743+0000","last_clean":"2023-05-20T01:16:32.771743+0000","last_became_active":"2023-05-20T01:16:28.338829+0000","last_became_peered":"2023-05-20T01:16:28.338829+0000","last_unstale":"2023-05-20T01:16:32.771743+0000","last_undegraded":"2023-05-20T01:16:32.771743+0000","last_fullsized":"2023-05-20T01:16:32.771743+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:38:13.734511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.310111+0000","last_change":"2023-05-20T01:16:32.364639+0000","last_active":"2023-05-20T01:16:33.310111+0000","last_peered":"2023-05-20T01:16:33.310111+0000","last_clean":"2023-05-20T01:16:33.310111+0000","last_became_active":"2023-05-20T01:16:32.364259+0000","last_became_peered":"2023-05-20T01:16:32.364259+0000","last_unstale":"2023-05-20T01:16:33.310111+0000","last_undegraded":"2023-05-20T01:16:33.310111+0000","last_fullsized":"2023-05-20T01:16:33.310111+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:58:39.806550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6,0],"acting":[7,4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.310717+0000","last_change":"2023-05-20T01:16:27.914406+0000","last_active":"2023-05-20T01:16:33.310717+0000","last_peered":"2023-05-20T01:16:33.310717+0000","last_clean":"2023-05-20T01:16:33.310717+0000","last_became_active":"2023-05-20T01:16:27.913932+0000","last_became_peered":"2023-05-20T01:16:27.913932+0000","last_unstale":"2023-05-20T01:16:33.310717+0000","last_undegraded":"2023-05-20T01:16:33.310717+0000","last_fullsized":"2023-05-20T01:16:33.310717+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:58:35.370081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.771638+0000","last_change":"2023-05-20T01:16:28.330729+0000","last_active":"2023-05-20T01:16:32.771638+0000","last_peered":"2023-05-20T01:16:32.771638+0000","last_clean":"2023-05-20T01:16:32.771638+0000","last_became_active":"2023-05-20T01:16:28.330474+0000","last_became_peered":"2023-05-20T01:16:28.330474+0000","last_unstale":"2023-05-20T01:16:32.771638+0000","last_undegraded":"2023-05-20T01:16:32.771638+0000","last_fullsized":"2023-05-20T01:16:32.771638+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:49:21.195547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.772392+0000","last_change":"2023-05-20T01:16:32.363881+0000","last_active":"2023-05-20T01:16:32.772392+0000","last_peered":"2023-05-20T01:16:32.772392+0000","last_clean":"2023-05-20T01:16:32.772392+0000","last_became_active":"2023-05-20T01:16:32.363269+0000","last_became_peered":"2023-05-20T01:16:32.363269+0000","last_unstale":"2023-05-20T01:16:32.772392+0000","last_undegraded":"2023-05-20T01:16:32.772392+0000","last_fullsized":"2023-05-20T01:16:32.772392+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:06:25.331494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,5],"acting":[3,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"20'3","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.311909+0000","last_change":"2023-05-20T01:16:27.919005+0000","last_active":"2023-05-20T01:16:33.311909+0000","last_peered":"2023-05-20T01:16:33.311909+0000","last_clean":"2023-05-20T01:16:33.311909+0000","last_became_active":"2023-05-20T01:16:27.918679+0000","last_became_peered":"2023-05-20T01:16:27.918679+0000","last_unstale":"2023-05-20T01:16:33.311909+0000","last_undegraded":"2023-05-20T01:16:33.311909+0000","last_fullsized":"2023-05-20T01:16:33.311909+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:06:58.439555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.957017+0000","last_change":"2023-05-20T01:16:28.346905+0000","last_active":"2023-05-20T01:16:32.957017+0000","last_peered":"2023-05-20T01:16:32.957017+0000","last_clean":"2023-05-20T01:16:32.957017+0000","last_became_active":"2023-05-20T01:16:28.345822+0000","last_became_peered":"2023-05-20T01:16:28.345822+0000","last_unstale":"2023-05-20T01:16:32.957017+0000","last_undegraded":"2023-05-20T01:16:32.957017+0000","last_fullsized":"2023-05-20T01:16:32.957017+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:45:24.050486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.957316+0000","last_change":"2023-05-20T01:16:32.354753+0000","last_active":"2023-05-20T01:16:32.957316+0000","last_peered":"2023-05-20T01:16:32.957316+0000","last_clean":"2023-05-20T01:16:32.957316+0000","last_became_active":"2023-05-20T01:16:32.354017+0000","last_became_peered":"2023-05-20T01:16:32.354017+0000","last_unstale":"2023-05-20T01:16:32.957316+0000","last_undegraded":"2023-05-20T01:16:32.957316+0000","last_fullsized":"2023-05-20T01:16:32.957316+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:24:19.113785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3,7],"acting":[1,5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"25'8","reported_seq":29,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:37.116885+0000","last_change":"2023-05-20T01:16:27.924627+0000","last_active":"2023-05-20T01:16:37.116885+0000","last_peered":"2023-05-20T01:16:37.116885+0000","last_clean":"2023-05-20T01:16:37.116885+0000","last_became_active":"2023-05-20T01:16:27.924339+0000","last_became_peered":"2023-05-20T01:16:27.924339+0000","last_unstale":"2023-05-20T01:16:37.116885+0000","last_undegraded":"2023-05-20T01:16:37.116885+0000","last_fullsized":"2023-05-20T01:16:37.116885+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:27:33.065117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3653,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.773847+0000","last_change":"2023-05-20T01:16:28.348715+0000","last_active":"2023-05-20T01:16:32.773847+0000","last_peered":"2023-05-20T01:16:32.773847+0000","last_clean":"2023-05-20T01:16:32.773847+0000","last_became_active":"2023-05-20T01:16:28.346893+0000","last_became_peered":"2023-05-20T01:16:28.346893+0000","last_unstale":"2023-05-20T01:16:32.773847+0000","last_undegraded":"2023-05-20T01:16:32.773847+0000","last_fullsized":"2023-05-20T01:16:32.773847+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:44:10.386273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.311010+0000","last_change":"2023-05-20T01:16:32.362978+0000","last_active":"2023-05-20T01:16:33.311010+0000","last_peered":"2023-05-20T01:16:33.311010+0000","last_clean":"2023-05-20T01:16:33.311010+0000","last_became_active":"2023-05-20T01:16:32.362592+0000","last_became_peered":"2023-05-20T01:16:32.362592+0000","last_unstale":"2023-05-20T01:16:33.311010+0000","last_undegraded":"2023-05-20T01:16:33.311010+0000","last_fullsized":"2023-05-20T01:16:33.311010+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:46:46.150220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3,0],"acting":[5,1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.311751+0000","last_change":"2023-05-20T01:16:27.925307+0000","last_active":"2023-05-20T01:16:33.311751+0000","last_peered":"2023-05-20T01:16:33.311751+0000","last_clean":"2023-05-20T01:16:33.311751+0000","last_became_active":"2023-05-20T01:16:27.925044+0000","last_became_peered":"2023-05-20T01:16:27.925044+0000","last_unstale":"2023-05-20T01:16:33.311751+0000","last_undegraded":"2023-05-20T01:16:33.311751+0000","last_fullsized":"2023-05-20T01:16:33.311751+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:54:11.307738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.772200+0000","last_change":"2023-05-20T01:16:28.340299+0000","last_active":"2023-05-20T01:16:32.772200+0000","last_peered":"2023-05-20T01:16:32.772200+0000","last_clean":"2023-05-20T01:16:32.772200+0000","last_became_active":"2023-05-20T01:16:28.340002+0000","last_became_peered":"2023-05-20T01:16:28.340002+0000","last_unstale":"2023-05-20T01:16:32.772200+0000","last_undegraded":"2023-05-20T01:16:32.772200+0000","last_fullsized":"2023-05-20T01:16:32.772200+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T01:44:35.746448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.772637+0000","last_change":"2023-05-20T01:16:32.354337+0000","last_active":"2023-05-20T01:16:32.772637+0000","last_peered":"2023-05-20T01:16:32.772637+0000","last_clean":"2023-05-20T01:16:32.772637+0000","last_became_active":"2023-05-20T01:16:32.353900+0000","last_became_peered":"2023-05-20T01:16:32.353900+0000","last_unstale":"2023-05-20T01:16:32.772637+0000","last_undegraded":"2023-05-20T01:16:32.772637+0000","last_fullsized":"2023-05-20T01:16:32.772637+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:50:05.163209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2,1],"acting":[3,7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.954416+0000","last_change":"2023-05-20T01:16:27.916514+0000","last_active":"2023-05-20T01:16:32.954416+0000","last_peered":"2023-05-20T01:16:32.954416+0000","last_clean":"2023-05-20T01:16:32.954416+0000","last_became_active":"2023-05-20T01:16:27.916199+0000","last_became_peered":"2023-05-20T01:16:27.916199+0000","last_unstale":"2023-05-20T01:16:32.954416+0000","last_undegraded":"2023-05-20T01:16:32.954416+0000","last_fullsized":"2023-05-20T01:16:32.954416+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:21:51.991048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.310678+0000","last_change":"2023-05-20T01:16:28.335549+0000","last_active":"2023-05-20T01:16:33.310678+0000","last_peered":"2023-05-20T01:16:33.310678+0000","last_clean":"2023-05-20T01:16:33.310678+0000","last_became_active":"2023-05-20T01:16:28.335284+0000","last_became_peered":"2023-05-20T01:16:28.335284+0000","last_unstale":"2023-05-20T01:16:33.310678+0000","last_undegraded":"2023-05-20T01:16:33.310678+0000","last_fullsized":"2023-05-20T01:16:33.310678+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:21:52.998812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.772615+0000","last_change":"2023-05-20T01:16:32.362344+0000","last_active":"2023-05-20T01:16:32.772615+0000","last_peered":"2023-05-20T01:16:32.772615+0000","last_clean":"2023-05-20T01:16:32.772615+0000","last_became_active":"2023-05-20T01:16:32.361944+0000","last_became_peered":"2023-05-20T01:16:32.361944+0000","last_unstale":"2023-05-20T01:16:32.772615+0000","last_undegraded":"2023-05-20T01:16:32.772615+0000","last_fullsized":"2023-05-20T01:16:32.772615+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:44:02.823618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,0],"acting":[3,1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.311580+0000","last_change":"2023-05-20T01:16:27.926496+0000","last_active":"2023-05-20T01:16:33.311580+0000","last_peered":"2023-05-20T01:16:33.311580+0000","last_clean":"2023-05-20T01:16:33.311580+0000","last_became_active":"2023-05-20T01:16:27.926265+0000","last_became_peered":"2023-05-20T01:16:27.926265+0000","last_unstale":"2023-05-20T01:16:33.311580+0000","last_undegraded":"2023-05-20T01:16:33.311580+0000","last_fullsized":"2023-05-20T01:16:33.311580+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:23:32.724521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.954832+0000","last_change":"2023-05-20T01:16:28.348384+0000","last_active":"2023-05-20T01:16:32.954832+0000","last_peered":"2023-05-20T01:16:32.954832+0000","last_clean":"2023-05-20T01:16:32.954832+0000","last_became_active":"2023-05-20T01:16:28.347652+0000","last_became_peered":"2023-05-20T01:16:28.347652+0000","last_unstale":"2023-05-20T01:16:32.954832+0000","last_undegraded":"2023-05-20T01:16:32.954832+0000","last_fullsized":"2023-05-20T01:16:32.954832+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:49:21.195547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.310448+0000","last_change":"2023-05-20T01:16:32.352418+0000","last_active":"2023-05-20T01:16:33.310448+0000","last_peered":"2023-05-20T01:16:33.310448+0000","last_clean":"2023-05-20T01:16:33.310448+0000","last_became_active":"2023-05-20T01:16:32.352031+0000","last_became_peered":"2023-05-20T01:16:32.352031+0000","last_unstale":"2023-05-20T01:16:33.310448+0000","last_undegraded":"2023-05-20T01:16:33.310448+0000","last_fullsized":"2023-05-20T01:16:33.310448+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:05:24.403434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1,6],"acting":[5,3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"20'8","reported_seq":29,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.773007+0000","last_change":"2023-05-20T01:16:27.916957+0000","last_active":"2023-05-20T01:16:32.773007+0000","last_peered":"2023-05-20T01:16:32.773007+0000","last_clean":"2023-05-20T01:16:32.773007+0000","last_became_active":"2023-05-20T01:16:27.916682+0000","last_became_peered":"2023-05-20T01:16:27.916682+0000","last_unstale":"2023-05-20T01:16:32.773007+0000","last_undegraded":"2023-05-20T01:16:32.773007+0000","last_fullsized":"2023-05-20T01:16:32.773007+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:15:10.960002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.772882+0000","last_change":"2023-05-20T01:16:28.339442+0000","last_active":"2023-05-20T01:16:32.772882+0000","last_peered":"2023-05-20T01:16:32.772882+0000","last_clean":"2023-05-20T01:16:32.772882+0000","last_became_active":"2023-05-20T01:16:28.339182+0000","last_became_peered":"2023-05-20T01:16:28.339182+0000","last_unstale":"2023-05-20T01:16:32.772882+0000","last_undegraded":"2023-05-20T01:16:32.772882+0000","last_fullsized":"2023-05-20T01:16:32.772882+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:31:47.073658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.311789+0000","last_change":"2023-05-20T01:16:32.360635+0000","last_active":"2023-05-20T01:16:33.311789+0000","last_peered":"2023-05-20T01:16:33.311789+0000","last_clean":"2023-05-20T01:16:33.311789+0000","last_became_active":"2023-05-20T01:16:32.360032+0000","last_became_peered":"2023-05-20T01:16:32.360032+0000","last_unstale":"2023-05-20T01:16:33.311789+0000","last_undegraded":"2023-05-20T01:16:33.311789+0000","last_fullsized":"2023-05-20T01:16:33.311789+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:49:24.624253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3,4],"acting":[7,6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"20'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.312203+0000","last_change":"2023-05-20T01:16:27.926418+0000","last_active":"2023-05-20T01:16:33.312203+0000","last_peered":"2023-05-20T01:16:33.312203+0000","last_clean":"2023-05-20T01:16:33.312203+0000","last_became_active":"2023-05-20T01:16:27.926135+0000","last_became_peered":"2023-05-20T01:16:27.926135+0000","last_unstale":"2023-05-20T01:16:33.312203+0000","last_undegraded":"2023-05-20T01:16:33.312203+0000","last_fullsized":"2023-05-20T01:16:33.312203+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:53:31.196496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.310060+0000","last_change":"2023-05-20T01:16:28.333381+0000","last_active":"2023-05-20T01:16:33.310060+0000","last_peered":"2023-05-20T01:16:33.310060+0000","last_clean":"2023-05-20T01:16:33.310060+0000","last_became_active":"2023-05-20T01:16:28.333096+0000","last_became_peered":"2023-05-20T01:16:28.333096+0000","last_unstale":"2023-05-20T01:16:33.310060+0000","last_undegraded":"2023-05-20T01:16:33.310060+0000","last_fullsized":"2023-05-20T01:16:33.310060+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:47:13.680746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.310001+0000","last_change":"2023-05-20T01:16:32.353695+0000","last_active":"2023-05-20T01:16:33.310001+0000","last_peered":"2023-05-20T01:16:33.310001+0000","last_clean":"2023-05-20T01:16:33.310001+0000","last_became_active":"2023-05-20T01:16:32.353280+0000","last_became_peered":"2023-05-20T01:16:32.353280+0000","last_unstale":"2023-05-20T01:16:33.310001+0000","last_undegraded":"2023-05-20T01:16:33.310001+0000","last_fullsized":"2023-05-20T01:16:33.310001+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:53:47.536905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1,5],"acting":[2,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.312036+0000","last_change":"2023-05-20T01:16:27.926342+0000","last_active":"2023-05-20T01:16:33.312036+0000","last_peered":"2023-05-20T01:16:33.312036+0000","last_clean":"2023-05-20T01:16:33.312036+0000","last_became_active":"2023-05-20T01:16:27.926093+0000","last_became_peered":"2023-05-20T01:16:27.926093+0000","last_unstale":"2023-05-20T01:16:33.312036+0000","last_undegraded":"2023-05-20T01:16:33.312036+0000","last_fullsized":"2023-05-20T01:16:33.312036+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:22:46.049477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.774721+0000","last_change":"2023-05-20T01:16:28.338568+0000","last_active":"2023-05-20T01:16:32.774721+0000","last_peered":"2023-05-20T01:16:32.774721+0000","last_clean":"2023-05-20T01:16:32.774721+0000","last_became_active":"2023-05-20T01:16:28.338293+0000","last_became_peered":"2023-05-20T01:16:28.338293+0000","last_unstale":"2023-05-20T01:16:32.774721+0000","last_undegraded":"2023-05-20T01:16:32.774721+0000","last_fullsized":"2023-05-20T01:16:32.774721+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:49:07.725723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.310172+0000","last_change":"2023-05-20T01:16:32.358154+0000","last_active":"2023-05-20T01:16:33.310172+0000","last_peered":"2023-05-20T01:16:33.310172+0000","last_clean":"2023-05-20T01:16:33.310172+0000","last_became_active":"2023-05-20T01:16:32.357690+0000","last_became_peered":"2023-05-20T01:16:32.357690+0000","last_unstale":"2023-05-20T01:16:33.310172+0000","last_undegraded":"2023-05-20T01:16:33.310172+0000","last_fullsized":"2023-05-20T01:16:33.310172+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:04:33.803901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5,6],"acting":[2,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"20'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.311721+0000","last_change":"2023-05-20T01:16:27.922534+0000","last_active":"2023-05-20T01:16:33.311721+0000","last_peered":"2023-05-20T01:16:33.311721+0000","last_clean":"2023-05-20T01:16:33.311721+0000","last_became_active":"2023-05-20T01:16:27.922263+0000","last_became_peered":"2023-05-20T01:16:27.922263+0000","last_unstale":"2023-05-20T01:16:33.311721+0000","last_undegraded":"2023-05-20T01:16:33.311721+0000","last_fullsized":"2023-05-20T01:16:33.311721+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:25:21.983559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.773343+0000","last_change":"2023-05-20T01:16:28.345113+0000","last_active":"2023-05-20T01:16:32.773343+0000","last_peered":"2023-05-20T01:16:32.773343+0000","last_clean":"2023-05-20T01:16:32.773343+0000","last_became_active":"2023-05-20T01:16:28.344807+0000","last_became_peered":"2023-05-20T01:16:28.344807+0000","last_unstale":"2023-05-20T01:16:32.773343+0000","last_undegraded":"2023-05-20T01:16:32.773343+0000","last_fullsized":"2023-05-20T01:16:32.773343+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:31:16.197740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.310276+0000","last_change":"2023-05-20T01:16:32.346858+0000","last_active":"2023-05-20T01:16:33.310276+0000","last_peered":"2023-05-20T01:16:33.310276+0000","last_clean":"2023-05-20T01:16:33.310276+0000","last_became_active":"2023-05-20T01:16:32.346456+0000","last_became_peered":"2023-05-20T01:16:32.346456+0000","last_unstale":"2023-05-20T01:16:33.310276+0000","last_undegraded":"2023-05-20T01:16:33.310276+0000","last_fullsized":"2023-05-20T01:16:33.310276+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:30:20.804962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3,6],"acting":[2,4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.310878+0000","last_change":"2023-05-20T01:16:27.912388+0000","last_active":"2023-05-20T01:16:33.310878+0000","last_peered":"2023-05-20T01:16:33.310878+0000","last_clean":"2023-05-20T01:16:33.310878+0000","last_became_active":"2023-05-20T01:16:27.911977+0000","last_became_peered":"2023-05-20T01:16:27.911977+0000","last_unstale":"2023-05-20T01:16:33.310878+0000","last_undegraded":"2023-05-20T01:16:33.310878+0000","last_fullsized":"2023-05-20T01:16:33.310878+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:19:50.512317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.954435+0000","last_change":"2023-05-20T01:16:28.342633+0000","last_active":"2023-05-20T01:16:32.954435+0000","last_peered":"2023-05-20T01:16:32.954435+0000","last_clean":"2023-05-20T01:16:32.954435+0000","last_became_active":"2023-05-20T01:16:28.339419+0000","last_became_peered":"2023-05-20T01:16:28.339419+0000","last_unstale":"2023-05-20T01:16:32.954435+0000","last_undegraded":"2023-05-20T01:16:32.954435+0000","last_fullsized":"2023-05-20T01:16:32.954435+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:27:57.460519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.310304+0000","last_change":"2023-05-20T01:16:32.347930+0000","last_active":"2023-05-20T01:16:33.310304+0000","last_peered":"2023-05-20T01:16:33.310304+0000","last_clean":"2023-05-20T01:16:33.310304+0000","last_became_active":"2023-05-20T01:16:32.347502+0000","last_became_peered":"2023-05-20T01:16:32.347502+0000","last_unstale":"2023-05-20T01:16:33.310304+0000","last_undegraded":"2023-05-20T01:16:33.310304+0000","last_fullsized":"2023-05-20T01:16:33.310304+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:05:15.989168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5,6],"acting":[2,7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.772049+0000","last_change":"2023-05-20T01:16:27.909757+0000","last_active":"2023-05-20T01:16:32.772049+0000","last_peered":"2023-05-20T01:16:32.772049+0000","last_clean":"2023-05-20T01:16:32.772049+0000","last_became_active":"2023-05-20T01:16:27.909458+0000","last_became_peered":"2023-05-20T01:16:27.909458+0000","last_unstale":"2023-05-20T01:16:32.772049+0000","last_undegraded":"2023-05-20T01:16:32.772049+0000","last_fullsized":"2023-05-20T01:16:32.772049+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:26:58.729159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.773572+0000","last_change":"2023-05-20T01:16:28.340078+0000","last_active":"2023-05-20T01:16:32.773572+0000","last_peered":"2023-05-20T01:16:32.773572+0000","last_clean":"2023-05-20T01:16:32.773572+0000","last_became_active":"2023-05-20T01:16:28.339673+0000","last_became_peered":"2023-05-20T01:16:28.339673+0000","last_unstale":"2023-05-20T01:16:32.773572+0000","last_undegraded":"2023-05-20T01:16:32.773572+0000","last_fullsized":"2023-05-20T01:16:32.773572+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:13:55.558140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.957039+0000","last_change":"2023-05-20T01:16:32.364538+0000","last_active":"2023-05-20T01:16:32.957039+0000","last_peered":"2023-05-20T01:16:32.957039+0000","last_clean":"2023-05-20T01:16:32.957039+0000","last_became_active":"2023-05-20T01:16:32.364025+0000","last_became_peered":"2023-05-20T01:16:32.364025+0000","last_unstale":"2023-05-20T01:16:32.957039+0000","last_undegraded":"2023-05-20T01:16:32.957039+0000","last_fullsized":"2023-05-20T01:16:32.957039+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:01:04.644797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0,6],"acting":[1,4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.774177+0000","last_change":"2023-05-20T01:16:27.922354+0000","last_active":"2023-05-20T01:16:32.774177+0000","last_peered":"2023-05-20T01:16:32.774177+0000","last_clean":"2023-05-20T01:16:32.774177+0000","last_became_active":"2023-05-20T01:16:27.922000+0000","last_became_peered":"2023-05-20T01:16:27.922000+0000","last_unstale":"2023-05-20T01:16:32.774177+0000","last_undegraded":"2023-05-20T01:16:32.774177+0000","last_fullsized":"2023-05-20T01:16:32.774177+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:56:49.903435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.774102+0000","last_change":"2023-05-20T01:16:28.334441+0000","last_active":"2023-05-20T01:16:32.774102+0000","last_peered":"2023-05-20T01:16:32.774102+0000","last_clean":"2023-05-20T01:16:32.774102+0000","last_became_active":"2023-05-20T01:16:28.334157+0000","last_became_peered":"2023-05-20T01:16:28.334157+0000","last_unstale":"2023-05-20T01:16:32.774102+0000","last_undegraded":"2023-05-20T01:16:32.774102+0000","last_fullsized":"2023-05-20T01:16:32.774102+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:51:16.225196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.310897+0000","last_change":"2023-05-20T01:16:32.346834+0000","last_active":"2023-05-20T01:16:33.310897+0000","last_peered":"2023-05-20T01:16:33.310897+0000","last_clean":"2023-05-20T01:16:33.310897+0000","last_became_active":"2023-05-20T01:16:32.346410+0000","last_became_peered":"2023-05-20T01:16:32.346410+0000","last_unstale":"2023-05-20T01:16:33.310897+0000","last_undegraded":"2023-05-20T01:16:33.310897+0000","last_fullsized":"2023-05-20T01:16:33.310897+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:07:08.123139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6,2],"acting":[5,4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.774525+0000","last_change":"2023-05-20T01:16:27.921658+0000","last_active":"2023-05-20T01:16:32.774525+0000","last_peered":"2023-05-20T01:16:32.774525+0000","last_clean":"2023-05-20T01:16:32.774525+0000","last_became_active":"2023-05-20T01:16:27.921399+0000","last_became_peered":"2023-05-20T01:16:27.921399+0000","last_unstale":"2023-05-20T01:16:32.774525+0000","last_undegraded":"2023-05-20T01:16:32.774525+0000","last_fullsized":"2023-05-20T01:16:32.774525+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:43:09.775404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.775593+0000","last_change":"2023-05-20T01:16:28.347570+0000","last_active":"2023-05-20T01:16:32.775593+0000","last_peered":"2023-05-20T01:16:32.775593+0000","last_clean":"2023-05-20T01:16:32.775593+0000","last_became_active":"2023-05-20T01:16:28.347140+0000","last_became_peered":"2023-05-20T01:16:28.347140+0000","last_unstale":"2023-05-20T01:16:32.775593+0000","last_undegraded":"2023-05-20T01:16:32.775593+0000","last_fullsized":"2023-05-20T01:16:32.775593+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T01:30:46.937626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.773539+0000","last_change":"2023-05-20T01:16:32.363537+0000","last_active":"2023-05-20T01:16:32.773539+0000","last_peered":"2023-05-20T01:16:32.773539+0000","last_clean":"2023-05-20T01:16:32.773539+0000","last_became_active":"2023-05-20T01:16:32.363119+0000","last_became_peered":"2023-05-20T01:16:32.363119+0000","last_unstale":"2023-05-20T01:16:32.773539+0000","last_undegraded":"2023-05-20T01:16:32.773539+0000","last_fullsized":"2023-05-20T01:16:32.773539+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:40:04.730139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0,2],"acting":[4,5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.771744+0000","last_change":"2023-05-20T01:16:27.924294+0000","last_active":"2023-05-20T01:16:32.771744+0000","last_peered":"2023-05-20T01:16:32.771744+0000","last_clean":"2023-05-20T01:16:32.771744+0000","last_became_active":"2023-05-20T01:16:27.923930+0000","last_became_peered":"2023-05-20T01:16:27.923930+0000","last_unstale":"2023-05-20T01:16:32.771744+0000","last_undegraded":"2023-05-20T01:16:32.771744+0000","last_fullsized":"2023-05-20T01:16:32.771744+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:24:35.039391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.957544+0000","last_change":"2023-05-20T01:16:28.349390+0000","last_active":"2023-05-20T01:16:32.957544+0000","last_peered":"2023-05-20T01:16:32.957544+0000","last_clean":"2023-05-20T01:16:32.957544+0000","last_became_active":"2023-05-20T01:16:28.349074+0000","last_became_peered":"2023-05-20T01:16:28.349074+0000","last_unstale":"2023-05-20T01:16:32.957544+0000","last_undegraded":"2023-05-20T01:16:32.957544+0000","last_fullsized":"2023-05-20T01:16:32.957544+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:59:48.441604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.774884+0000","last_change":"2023-05-20T01:16:32.353210+0000","last_active":"2023-05-20T01:16:32.774884+0000","last_peered":"2023-05-20T01:16:32.774884+0000","last_clean":"2023-05-20T01:16:32.774884+0000","last_became_active":"2023-05-20T01:16:32.352807+0000","last_became_peered":"2023-05-20T01:16:32.352807+0000","last_unstale":"2023-05-20T01:16:32.774884+0000","last_undegraded":"2023-05-20T01:16:32.774884+0000","last_fullsized":"2023-05-20T01:16:32.774884+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:42:22.830436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7,0],"acting":[4,3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.772776+0000","last_change":"2023-05-20T01:16:27.913270+0000","last_active":"2023-05-20T01:16:32.772776+0000","last_peered":"2023-05-20T01:16:32.772776+0000","last_clean":"2023-05-20T01:16:32.772776+0000","last_became_active":"2023-05-20T01:16:27.912952+0000","last_became_peered":"2023-05-20T01:16:27.912952+0000","last_unstale":"2023-05-20T01:16:32.772776+0000","last_undegraded":"2023-05-20T01:16:32.772776+0000","last_fullsized":"2023-05-20T01:16:32.772776+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:34:35.188240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.774248+0000","last_change":"2023-05-20T01:16:23.289068+0000","last_active":"2023-05-20T01:16:32.774248+0000","last_peered":"2023-05-20T01:16:32.774248+0000","last_clean":"2023-05-20T01:16:32.774248+0000","last_became_active":"2023-05-20T01:16:23.288781+0000","last_became_peered":"2023-05-20T01:16:23.288781+0000","last_unstale":"2023-05-20T01:16:32.774248+0000","last_undegraded":"2023-05-20T01:16:32.774248+0000","last_fullsized":"2023-05-20T01:16:32.774248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:02:03.239372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.771607+0000","last_change":"2023-05-20T01:16:28.329739+0000","last_active":"2023-05-20T01:16:32.771607+0000","last_peered":"2023-05-20T01:16:32.771607+0000","last_clean":"2023-05-20T01:16:32.771607+0000","last_became_active":"2023-05-20T01:16:28.329483+0000","last_became_peered":"2023-05-20T01:16:28.329483+0000","last_unstale":"2023-05-20T01:16:32.771607+0000","last_undegraded":"2023-05-20T01:16:32.771607+0000","last_fullsized":"2023-05-20T01:16:32.771607+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:46:14.388881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.771640+0000","last_change":"2023-05-20T01:16:32.353144+0000","last_active":"2023-05-20T01:16:32.771640+0000","last_peered":"2023-05-20T01:16:32.771640+0000","last_clean":"2023-05-20T01:16:32.771640+0000","last_became_active":"2023-05-20T01:16:32.352355+0000","last_became_peered":"2023-05-20T01:16:32.352355+0000","last_unstale":"2023-05-20T01:16:32.771640+0000","last_undegraded":"2023-05-20T01:16:32.771640+0000","last_fullsized":"2023-05-20T01:16:32.771640+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:25:26.420028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,0],"acting":[3,1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.772224+0000","last_change":"2023-05-20T01:16:22.829324+0000","last_active":"2023-05-20T01:16:32.772224+0000","last_peered":"2023-05-20T01:16:32.772224+0000","last_clean":"2023-05-20T01:16:32.772224+0000","last_became_active":"2023-05-20T01:16:22.828964+0000","last_became_peered":"2023-05-20T01:16:22.828964+0000","last_unstale":"2023-05-20T01:16:32.772224+0000","last_undegraded":"2023-05-20T01:16:32.772224+0000","last_fullsized":"2023-05-20T01:16:32.772224+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:33:22.739978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"25'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.954588+0000","last_change":"2023-05-20T01:16:27.922339+0000","last_active":"2023-05-20T01:16:32.954588+0000","last_peered":"2023-05-20T01:16:32.954588+0000","last_clean":"2023-05-20T01:16:32.954588+0000","last_became_active":"2023-05-20T01:16:27.922033+0000","last_became_peered":"2023-05-20T01:16:27.922033+0000","last_unstale":"2023-05-20T01:16:32.954588+0000","last_undegraded":"2023-05-20T01:16:32.954588+0000","last_fullsized":"2023-05-20T01:16:32.954588+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:01:20.476920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.774598+0000","last_change":"2023-05-20T01:16:28.335846+0000","last_active":"2023-05-20T01:16:32.774598+0000","last_peered":"2023-05-20T01:16:32.774598+0000","last_clean":"2023-05-20T01:16:32.774598+0000","last_became_active":"2023-05-20T01:16:28.335523+0000","last_became_peered":"2023-05-20T01:16:28.335523+0000","last_unstale":"2023-05-20T01:16:32.774598+0000","last_undegraded":"2023-05-20T01:16:32.774598+0000","last_fullsized":"2023-05-20T01:16:32.774598+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:18:05.626837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.310654+0000","last_change":"2023-05-20T01:16:32.363029+0000","last_active":"2023-05-20T01:16:33.310654+0000","last_peered":"2023-05-20T01:16:33.310654+0000","last_clean":"2023-05-20T01:16:33.310654+0000","last_became_active":"2023-05-20T01:16:32.362637+0000","last_became_peered":"2023-05-20T01:16:32.362637+0000","last_unstale":"2023-05-20T01:16:33.310654+0000","last_undegraded":"2023-05-20T01:16:33.310654+0000","last_fullsized":"2023-05-20T01:16:33.310654+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:28:00.889225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0,4],"acting":[5,1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.956901+0000","last_change":"2023-05-20T01:16:22.834754+0000","last_active":"2023-05-20T01:16:32.956901+0000","last_peered":"2023-05-20T01:16:32.956901+0000","last_clean":"2023-05-20T01:16:32.956901+0000","last_became_active":"2023-05-20T01:16:22.834437+0000","last_became_peered":"2023-05-20T01:16:22.834437+0000","last_unstale":"2023-05-20T01:16:32.956901+0000","last_undegraded":"2023-05-20T01:16:32.956901+0000","last_fullsized":"2023-05-20T01:16:32.956901+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:51:36.657132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":177,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.311443+0000","last_change":"2023-05-20T01:16:22.834920+0000","last_active":"2023-05-20T01:16:33.311443+0000","last_peered":"2023-05-20T01:16:33.311443+0000","last_clean":"2023-05-20T01:16:33.311443+0000","last_became_active":"2023-05-20T01:16:22.833924+0000","last_became_peered":"2023-05-20T01:16:22.833924+0000","last_unstale":"2023-05-20T01:16:33.311443+0000","last_undegraded":"2023-05-20T01:16:33.311443+0000","last_fullsized":"2023-05-20T01:16:33.311443+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:34:02.513729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.954099+0000","last_change":"2023-05-20T01:16:27.923011+0000","last_active":"2023-05-20T01:16:32.954099+0000","last_peered":"2023-05-20T01:16:32.954099+0000","last_clean":"2023-05-20T01:16:32.954099+0000","last_became_active":"2023-05-20T01:16:27.922736+0000","last_became_peered":"2023-05-20T01:16:27.922736+0000","last_unstale":"2023-05-20T01:16:32.954099+0000","last_undegraded":"2023-05-20T01:16:32.954099+0000","last_fullsized":"2023-05-20T01:16:32.954099+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:59:47.433840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.954829+0000","last_change":"2023-05-20T01:16:28.348199+0000","last_active":"2023-05-20T01:16:32.954829+0000","last_peered":"2023-05-20T01:16:32.954829+0000","last_clean":"2023-05-20T01:16:32.954829+0000","last_became_active":"2023-05-20T01:16:28.347370+0000","last_became_peered":"2023-05-20T01:16:28.347370+0000","last_unstale":"2023-05-20T01:16:32.954829+0000","last_undegraded":"2023-05-20T01:16:32.954829+0000","last_fullsized":"2023-05-20T01:16:32.954829+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:46:14.388881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.310467+0000","last_change":"2023-05-20T01:16:32.357823+0000","last_active":"2023-05-20T01:16:33.310467+0000","last_peered":"2023-05-20T01:16:33.310467+0000","last_clean":"2023-05-20T01:16:33.310467+0000","last_became_active":"2023-05-20T01:16:32.357437+0000","last_became_peered":"2023-05-20T01:16:32.357437+0000","last_unstale":"2023-05-20T01:16:33.310467+0000","last_undegraded":"2023-05-20T01:16:33.310467+0000","last_fullsized":"2023-05-20T01:16:33.310467+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:16:08.791218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1,7],"acting":[2,5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.311398+0000","last_change":"2023-05-20T01:16:23.290259+0000","last_active":"2023-05-20T01:16:33.311398+0000","last_peered":"2023-05-20T01:16:33.311398+0000","last_clean":"2023-05-20T01:16:33.311398+0000","last_became_active":"2023-05-20T01:16:23.289911+0000","last_became_peered":"2023-05-20T01:16:23.289911+0000","last_unstale":"2023-05-20T01:16:33.311398+0000","last_undegraded":"2023-05-20T01:16:33.311398+0000","last_fullsized":"2023-05-20T01:16:33.311398+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:46:36.633677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.956681+0000","last_change":"2023-05-20T01:16:27.919248+0000","last_active":"2023-05-20T01:16:32.956681+0000","last_peered":"2023-05-20T01:16:32.956681+0000","last_clean":"2023-05-20T01:16:32.956681+0000","last_became_active":"2023-05-20T01:16:27.918957+0000","last_became_peered":"2023-05-20T01:16:27.918957+0000","last_unstale":"2023-05-20T01:16:32.956681+0000","last_undegraded":"2023-05-20T01:16:32.956681+0000","last_fullsized":"2023-05-20T01:16:32.956681+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:51:16.461763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.957471+0000","last_change":"2023-05-20T01:16:28.337136+0000","last_active":"2023-05-20T01:16:32.957471+0000","last_peered":"2023-05-20T01:16:32.957471+0000","last_clean":"2023-05-20T01:16:32.957471+0000","last_became_active":"2023-05-20T01:16:28.336322+0000","last_became_peered":"2023-05-20T01:16:28.336322+0000","last_unstale":"2023-05-20T01:16:32.957471+0000","last_undegraded":"2023-05-20T01:16:32.957471+0000","last_fullsized":"2023-05-20T01:16:32.957471+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:42:59.927187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.311713+0000","last_change":"2023-05-20T01:16:32.358840+0000","last_active":"2023-05-20T01:16:33.311713+0000","last_peered":"2023-05-20T01:16:33.311713+0000","last_clean":"2023-05-20T01:16:33.311713+0000","last_became_active":"2023-05-20T01:16:32.358470+0000","last_became_peered":"2023-05-20T01:16:32.358470+0000","last_unstale":"2023-05-20T01:16:33.311713+0000","last_undegraded":"2023-05-20T01:16:33.311713+0000","last_fullsized":"2023-05-20T01:16:33.311713+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:14:35.048109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5,3],"acting":[7,2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.311131+0000","last_change":"2023-05-20T01:16:23.290207+0000","last_active":"2023-05-20T01:16:33.311131+0000","last_peered":"2023-05-20T01:16:33.311131+0000","last_clean":"2023-05-20T01:16:33.311131+0000","last_became_active":"2023-05-20T01:16:23.289928+0000","last_became_peered":"2023-05-20T01:16:23.289928+0000","last_unstale":"2023-05-20T01:16:33.311131+0000","last_undegraded":"2023-05-20T01:16:33.311131+0000","last_fullsized":"2023-05-20T01:16:33.311131+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:01:15.396847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.771970+0000","last_change":"2023-05-20T01:16:27.911483+0000","last_active":"2023-05-20T01:16:32.771970+0000","last_peered":"2023-05-20T01:16:32.771970+0000","last_clean":"2023-05-20T01:16:32.771970+0000","last_became_active":"2023-05-20T01:16:27.911180+0000","last_became_peered":"2023-05-20T01:16:27.911180+0000","last_unstale":"2023-05-20T01:16:32.771970+0000","last_undegraded":"2023-05-20T01:16:32.771970+0000","last_fullsized":"2023-05-20T01:16:32.771970+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:13:48.951321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.773506+0000","last_change":"2023-05-20T01:16:28.345161+0000","last_active":"2023-05-20T01:16:32.773506+0000","last_peered":"2023-05-20T01:16:32.773506+0000","last_clean":"2023-05-20T01:16:32.773506+0000","last_became_active":"2023-05-20T01:16:28.344872+0000","last_became_peered":"2023-05-20T01:16:28.344872+0000","last_unstale":"2023-05-20T01:16:32.773506+0000","last_undegraded":"2023-05-20T01:16:32.773506+0000","last_fullsized":"2023-05-20T01:16:32.773506+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:49:51.917546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.954428+0000","last_change":"2023-05-20T01:16:32.362192+0000","last_active":"2023-05-20T01:16:32.954428+0000","last_peered":"2023-05-20T01:16:32.954428+0000","last_clean":"2023-05-20T01:16:32.954428+0000","last_became_active":"2023-05-20T01:16:32.361656+0000","last_became_peered":"2023-05-20T01:16:32.361656+0000","last_unstale":"2023-05-20T01:16:32.954428+0000","last_undegraded":"2023-05-20T01:16:32.954428+0000","last_fullsized":"2023-05-20T01:16:32.954428+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:58:35.944694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,6],"acting":[0,1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.310220+0000","last_change":"2023-05-20T01:16:23.290538+0000","last_active":"2023-05-20T01:16:33.310220+0000","last_peered":"2023-05-20T01:16:33.310220+0000","last_clean":"2023-05-20T01:16:33.310220+0000","last_became_active":"2023-05-20T01:16:23.290230+0000","last_became_peered":"2023-05-20T01:16:23.290230+0000","last_unstale":"2023-05-20T01:16:33.310220+0000","last_undegraded":"2023-05-20T01:16:33.310220+0000","last_fullsized":"2023-05-20T01:16:33.310220+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:24:09.597242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.773435+0000","last_change":"2023-05-20T01:16:27.921749+0000","last_active":"2023-05-20T01:16:32.773435+0000","last_peered":"2023-05-20T01:16:32.773435+0000","last_clean":"2023-05-20T01:16:32.773435+0000","last_became_active":"2023-05-20T01:16:27.921461+0000","last_became_peered":"2023-05-20T01:16:27.921461+0000","last_unstale":"2023-05-20T01:16:32.773435+0000","last_undegraded":"2023-05-20T01:16:32.773435+0000","last_fullsized":"2023-05-20T01:16:32.773435+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:01:30.293877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.957033+0000","last_change":"2023-05-20T01:16:28.335769+0000","last_active":"2023-05-20T01:16:32.957033+0000","last_peered":"2023-05-20T01:16:32.957033+0000","last_clean":"2023-05-20T01:16:32.957033+0000","last_became_active":"2023-05-20T01:16:28.335232+0000","last_became_peered":"2023-05-20T01:16:28.335232+0000","last_unstale":"2023-05-20T01:16:32.957033+0000","last_undegraded":"2023-05-20T01:16:32.957033+0000","last_fullsized":"2023-05-20T01:16:32.957033+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:23:18.405128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.774369+0000","last_change":"2023-05-20T01:16:32.354871+0000","last_active":"2023-05-20T01:16:32.774369+0000","last_peered":"2023-05-20T01:16:32.774369+0000","last_clean":"2023-05-20T01:16:32.774369+0000","last_became_active":"2023-05-20T01:16:32.353771+0000","last_became_peered":"2023-05-20T01:16:32.353771+0000","last_unstale":"2023-05-20T01:16:32.774369+0000","last_undegraded":"2023-05-20T01:16:32.774369+0000","last_fullsized":"2023-05-20T01:16:32.774369+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:22:02.793310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5,2],"acting":[6,0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.773241+0000","last_change":"2023-05-20T01:16:23.290341+0000","last_active":"2023-05-20T01:16:32.773241+0000","last_peered":"2023-05-20T01:16:32.773241+0000","last_clean":"2023-05-20T01:16:32.773241+0000","last_became_active":"2023-05-20T01:16:23.290053+0000","last_became_peered":"2023-05-20T01:16:23.290053+0000","last_unstale":"2023-05-20T01:16:32.773241+0000","last_undegraded":"2023-05-20T01:16:32.773241+0000","last_fullsized":"2023-05-20T01:16:32.773241+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:11:18.465994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.957177+0000","last_change":"2023-05-20T01:16:27.920036+0000","last_active":"2023-05-20T01:16:32.957177+0000","last_peered":"2023-05-20T01:16:32.957177+0000","last_clean":"2023-05-20T01:16:32.957177+0000","last_became_active":"2023-05-20T01:16:27.919757+0000","last_became_peered":"2023-05-20T01:16:27.919757+0000","last_unstale":"2023-05-20T01:16:32.957177+0000","last_undegraded":"2023-05-20T01:16:32.957177+0000","last_fullsized":"2023-05-20T01:16:32.957177+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:49:54.418261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.954564+0000","last_change":"2023-05-20T01:16:28.343036+0000","last_active":"2023-05-20T01:16:32.954564+0000","last_peered":"2023-05-20T01:16:32.954564+0000","last_clean":"2023-05-20T01:16:32.954564+0000","last_became_active":"2023-05-20T01:16:28.342814+0000","last_became_peered":"2023-05-20T01:16:28.342814+0000","last_unstale":"2023-05-20T01:16:32.954564+0000","last_undegraded":"2023-05-20T01:16:32.954564+0000","last_fullsized":"2023-05-20T01:16:32.954564+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:13:57.518009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.954367+0000","last_change":"2023-05-20T01:16:32.356221+0000","last_active":"2023-05-20T01:16:32.954367+0000","last_peered":"2023-05-20T01:16:32.954367+0000","last_clean":"2023-05-20T01:16:32.954367+0000","last_became_active":"2023-05-20T01:16:32.355772+0000","last_became_peered":"2023-05-20T01:16:32.355772+0000","last_unstale":"2023-05-20T01:16:32.954367+0000","last_undegraded":"2023-05-20T01:16:32.954367+0000","last_fullsized":"2023-05-20T01:16:32.954367+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:07:08.123139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6,5],"acting":[0,1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.774068+0000","last_change":"2023-05-20T01:16:22.834330+0000","last_active":"2023-05-20T01:16:32.774068+0000","last_peered":"2023-05-20T01:16:32.774068+0000","last_clean":"2023-05-20T01:16:32.774068+0000","last_became_active":"2023-05-20T01:16:22.833946+0000","last_became_peered":"2023-05-20T01:16:22.833946+0000","last_unstale":"2023-05-20T01:16:32.774068+0000","last_undegraded":"2023-05-20T01:16:32.774068+0000","last_fullsized":"2023-05-20T01:16:32.774068+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:22:15.807596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.310582+0000","last_change":"2023-05-20T01:16:27.914231+0000","last_active":"2023-05-20T01:16:33.310582+0000","last_peered":"2023-05-20T01:16:33.310582+0000","last_clean":"2023-05-20T01:16:33.310582+0000","last_became_active":"2023-05-20T01:16:27.913946+0000","last_became_peered":"2023-05-20T01:16:27.913946+0000","last_unstale":"2023-05-20T01:16:33.310582+0000","last_undegraded":"2023-05-20T01:16:33.310582+0000","last_fullsized":"2023-05-20T01:16:33.310582+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:58:31.508225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.956705+0000","last_change":"2023-05-20T01:16:28.336767+0000","last_active":"2023-05-20T01:16:32.956705+0000","last_peered":"2023-05-20T01:16:32.956705+0000","last_clean":"2023-05-20T01:16:32.956705+0000","last_became_active":"2023-05-20T01:16:28.336172+0000","last_became_peered":"2023-05-20T01:16:28.336172+0000","last_unstale":"2023-05-20T01:16:32.956705+0000","last_undegraded":"2023-05-20T01:16:32.956705+0000","last_fullsized":"2023-05-20T01:16:32.956705+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:45:17.217540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.957277+0000","last_change":"2023-05-20T01:16:32.364585+0000","last_active":"2023-05-20T01:16:32.957277+0000","last_peered":"2023-05-20T01:16:32.957277+0000","last_clean":"2023-05-20T01:16:32.957277+0000","last_became_active":"2023-05-20T01:16:32.364096+0000","last_became_peered":"2023-05-20T01:16:32.364096+0000","last_unstale":"2023-05-20T01:16:32.957277+0000","last_undegraded":"2023-05-20T01:16:32.957277+0000","last_fullsized":"2023-05-20T01:16:32.957277+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:47:17.109451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,0],"acting":[1,3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"20'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.954518+0000","last_change":"2023-05-20T01:16:27.922900+0000","last_active":"2023-05-20T01:16:32.954518+0000","last_peered":"2023-05-20T01:16:32.954518+0000","last_clean":"2023-05-20T01:16:32.954518+0000","last_became_active":"2023-05-20T01:16:27.922597+0000","last_became_peered":"2023-05-20T01:16:27.922597+0000","last_unstale":"2023-05-20T01:16:32.954518+0000","last_undegraded":"2023-05-20T01:16:32.954518+0000","last_fullsized":"2023-05-20T01:16:32.954518+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:58:35.370081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.310293+0000","last_change":"2023-05-20T01:16:28.341446+0000","last_active":"2023-05-20T01:16:33.310293+0000","last_peered":"2023-05-20T01:16:33.310293+0000","last_clean":"2023-05-20T01:16:33.310293+0000","last_became_active":"2023-05-20T01:16:28.340943+0000","last_became_peered":"2023-05-20T01:16:28.340943+0000","last_unstale":"2023-05-20T01:16:33.310293+0000","last_undegraded":"2023-05-20T01:16:33.310293+0000","last_fullsized":"2023-05-20T01:16:33.310293+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:24:03.794336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.311921+0000","last_change":"2023-05-20T01:16:32.360851+0000","last_active":"2023-05-20T01:16:33.311921+0000","last_peered":"2023-05-20T01:16:33.311921+0000","last_clean":"2023-05-20T01:16:33.311921+0000","last_became_active":"2023-05-20T01:16:32.360366+0000","last_became_peered":"2023-05-20T01:16:32.360366+0000","last_unstale":"2023-05-20T01:16:33.311921+0000","last_undegraded":"2023-05-20T01:16:33.311921+0000","last_fullsized":"2023-05-20T01:16:33.311921+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:38:17.163216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1,6],"acting":[7,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.310447+0000","last_change":"2023-05-20T01:16:27.925398+0000","last_active":"2023-05-20T01:16:33.310447+0000","last_peered":"2023-05-20T01:16:33.310447+0000","last_clean":"2023-05-20T01:16:33.310447+0000","last_became_active":"2023-05-20T01:16:27.924943+0000","last_became_peered":"2023-05-20T01:16:27.924943+0000","last_unstale":"2023-05-20T01:16:33.310447+0000","last_undegraded":"2023-05-20T01:16:33.310447+0000","last_fullsized":"2023-05-20T01:16:33.310447+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:42:58.919423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.773260+0000","last_change":"2023-05-20T01:16:28.332091+0000","last_active":"2023-05-20T01:16:32.773260+0000","last_peered":"2023-05-20T01:16:32.773260+0000","last_clean":"2023-05-20T01:16:32.773260+0000","last_became_active":"2023-05-20T01:16:28.331785+0000","last_became_peered":"2023-05-20T01:16:28.331785+0000","last_unstale":"2023-05-20T01:16:32.773260+0000","last_undegraded":"2023-05-20T01:16:32.773260+0000","last_fullsized":"2023-05-20T01:16:32.773260+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:31:47.073658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.310796+0000","last_change":"2023-05-20T01:16:32.362120+0000","last_active":"2023-05-20T01:16:33.310796+0000","last_peered":"2023-05-20T01:16:33.310796+0000","last_clean":"2023-05-20T01:16:33.310796+0000","last_became_active":"2023-05-20T01:16:32.361727+0000","last_became_peered":"2023-05-20T01:16:32.361727+0000","last_unstale":"2023-05-20T01:16:33.310796+0000","last_undegraded":"2023-05-20T01:16:33.310796+0000","last_fullsized":"2023-05-20T01:16:33.310796+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:01:24.913389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7,4],"acting":[5,0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.310449+0000","last_change":"2023-05-20T01:16:27.925288+0000","last_active":"2023-05-20T01:16:33.310449+0000","last_peered":"2023-05-20T01:16:33.310449+0000","last_clean":"2023-05-20T01:16:33.310449+0000","last_became_active":"2023-05-20T01:16:27.924988+0000","last_became_peered":"2023-05-20T01:16:27.924988+0000","last_unstale":"2023-05-20T01:16:33.310449+0000","last_undegraded":"2023-05-20T01:16:33.310449+0000","last_fullsized":"2023-05-20T01:16:33.310449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:32:59.792286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.773973+0000","last_change":"2023-05-20T01:16:28.334255+0000","last_active":"2023-05-20T01:16:32.773973+0000","last_peered":"2023-05-20T01:16:32.773973+0000","last_clean":"2023-05-20T01:16:32.773973+0000","last_became_active":"2023-05-20T01:16:28.333966+0000","last_became_peered":"2023-05-20T01:16:28.333966+0000","last_unstale":"2023-05-20T01:16:32.773973+0000","last_undegraded":"2023-05-20T01:16:32.773973+0000","last_fullsized":"2023-05-20T01:16:32.773973+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:04:05.601168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.772980+0000","last_change":"2023-05-20T01:16:32.346878+0000","last_active":"2023-05-20T01:16:32.772980+0000","last_peered":"2023-05-20T01:16:32.772980+0000","last_clean":"2023-05-20T01:16:32.772980+0000","last_became_active":"2023-05-20T01:16:32.346493+0000","last_became_peered":"2023-05-20T01:16:32.346493+0000","last_unstale":"2023-05-20T01:16:32.772980+0000","last_undegraded":"2023-05-20T01:16:32.772980+0000","last_fullsized":"2023-05-20T01:16:32.772980+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:50:05.163209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,2],"acting":[4,6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"20'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.772568+0000","last_change":"2023-05-20T01:16:27.906889+0000","last_active":"2023-05-20T01:16:32.772568+0000","last_peered":"2023-05-20T01:16:32.772568+0000","last_clean":"2023-05-20T01:16:32.772568+0000","last_became_active":"2023-05-20T01:16:27.906574+0000","last_became_peered":"2023-05-20T01:16:27.906574+0000","last_unstale":"2023-05-20T01:16:32.772568+0000","last_undegraded":"2023-05-20T01:16:32.772568+0000","last_fullsized":"2023-05-20T01:16:32.772568+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:49:41.443268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.771941+0000","last_change":"2023-05-20T01:16:28.345350+0000","last_active":"2023-05-20T01:16:32.771941+0000","last_peered":"2023-05-20T01:16:32.771941+0000","last_clean":"2023-05-20T01:16:32.771941+0000","last_became_active":"2023-05-20T01:16:28.344838+0000","last_became_peered":"2023-05-20T01:16:28.344838+0000","last_unstale":"2023-05-20T01:16:32.771941+0000","last_undegraded":"2023-05-20T01:16:32.771941+0000","last_fullsized":"2023-05-20T01:16:32.771941+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:53:32.204259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.956979+0000","last_change":"2023-05-20T01:16:32.352947+0000","last_active":"2023-05-20T01:16:32.956979+0000","last_peered":"2023-05-20T01:16:32.956979+0000","last_clean":"2023-05-20T01:16:32.956979+0000","last_became_active":"2023-05-20T01:16:32.352495+0000","last_became_peered":"2023-05-20T01:16:32.352495+0000","last_unstale":"2023-05-20T01:16:32.956979+0000","last_undegraded":"2023-05-20T01:16:32.956979+0000","last_fullsized":"2023-05-20T01:16:32.956979+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:29:18.453884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7,2],"acting":[1,5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"20'4","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.954845+0000","last_change":"2023-05-20T01:16:27.917645+0000","last_active":"2023-05-20T01:16:32.954845+0000","last_peered":"2023-05-20T01:16:32.954845+0000","last_clean":"2023-05-20T01:16:32.954845+0000","last_became_active":"2023-05-20T01:16:27.917362+0000","last_became_peered":"2023-05-20T01:16:27.917362+0000","last_unstale":"2023-05-20T01:16:32.954845+0000","last_undegraded":"2023-05-20T01:16:32.954845+0000","last_fullsized":"2023-05-20T01:16:32.954845+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:54:11.307738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.773261+0000","last_change":"2023-05-20T01:16:28.347617+0000","last_active":"2023-05-20T01:16:32.773261+0000","last_peered":"2023-05-20T01:16:32.773261+0000","last_clean":"2023-05-20T01:16:32.773261+0000","last_became_active":"2023-05-20T01:16:28.347032+0000","last_became_peered":"2023-05-20T01:16:28.347032+0000","last_unstale":"2023-05-20T01:16:32.773261+0000","last_undegraded":"2023-05-20T01:16:32.773261+0000","last_fullsized":"2023-05-20T01:16:32.773261+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:34:36.196004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.773474+0000","last_change":"2023-05-20T01:16:28.336509+0000","last_active":"2023-05-20T01:16:32.773474+0000","last_peered":"2023-05-20T01:16:32.773474+0000","last_clean":"2023-05-20T01:16:32.773474+0000","last_became_active":"2023-05-20T01:16:28.336170+0000","last_became_peered":"2023-05-20T01:16:28.336170+0000","last_unstale":"2023-05-20T01:16:32.773474+0000","last_undegraded":"2023-05-20T01:16:32.773474+0000","last_fullsized":"2023-05-20T01:16:32.773474+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:16:31.755218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.771733+0000","last_change":"2023-05-20T01:16:28.340216+0000","last_active":"2023-05-20T01:16:32.771733+0000","last_peered":"2023-05-20T01:16:32.771733+0000","last_clean":"2023-05-20T01:16:32.771733+0000","last_became_active":"2023-05-20T01:16:28.339662+0000","last_became_peered":"2023-05-20T01:16:28.339662+0000","last_unstale":"2023-05-20T01:16:32.771733+0000","last_undegraded":"2023-05-20T01:16:32.771733+0000","last_fullsized":"2023-05-20T01:16:32.771733+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:54:12.315501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.774267+0000","last_change":"2023-05-20T01:16:28.347840+0000","last_active":"2023-05-20T01:16:32.774267+0000","last_peered":"2023-05-20T01:16:32.774267+0000","last_clean":"2023-05-20T01:16:32.774267+0000","last_became_active":"2023-05-20T01:16:28.347370+0000","last_became_peered":"2023-05-20T01:16:28.347370+0000","last_unstale":"2023-05-20T01:16:32.774267+0000","last_undegraded":"2023-05-20T01:16:32.774267+0000","last_fullsized":"2023-05-20T01:16:32.774267+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T01:49:49.650741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.772038+0000","last_change":"2023-05-20T01:16:28.328611+0000","last_active":"2023-05-20T01:16:32.772038+0000","last_peered":"2023-05-20T01:16:32.772038+0000","last_clean":"2023-05-20T01:16:32.772038+0000","last_became_active":"2023-05-20T01:16:28.328311+0000","last_became_peered":"2023-05-20T01:16:28.328311+0000","last_unstale":"2023-05-20T01:16:32.772038+0000","last_undegraded":"2023-05-20T01:16:32.772038+0000","last_fullsized":"2023-05-20T01:16:32.772038+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:09:39.569369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.957125+0000","last_change":"2023-05-20T01:16:28.336980+0000","last_active":"2023-05-20T01:16:32.957125+0000","last_peered":"2023-05-20T01:16:32.957125+0000","last_clean":"2023-05-20T01:16:32.957125+0000","last_became_active":"2023-05-20T01:16:28.336778+0000","last_became_peered":"2023-05-20T01:16:28.336778+0000","last_unstale":"2023-05-20T01:16:32.957125+0000","last_undegraded":"2023-05-20T01:16:32.957125+0000","last_fullsized":"2023-05-20T01:16:32.957125+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:03:20.405510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":4451,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":8902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9260,"kb_used_data":868,"kb_used_omap":0,"kb_used_meta":8384,"kb_avail":94362580,"statfs":{"total":96636764160,"available":96627281920,"internally_reserved":0,"allocated":888832,"data_stored":703638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8585216},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.314}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41799999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38800000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51000000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54100000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8076,"kb_used_data":260,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363764,"statfs":{"total":96636764160,"available":96628494336,"internally_reserved":0,"allocated":266240,"data_stored":109401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.433}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49399999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63200000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70199999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64900000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67900000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9232,"kb_used_data":840,"kb_used_omap":0,"kb_used_meta":8384,"kb_avail":94362608,"statfs":{"total":96636764160,"available":96627310592,"internally_reserved":0,"allocated":860160,"data_stored":699769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8585216},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40699999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28899999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32300000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64100000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54500000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8140,"kb_used_data":260,"kb_used_omap":0,"kb_used_meta":7872,"kb_avail":94363700,"statfs":{"total":96636764160,"available":96628428800,"internally_reserved":0,"allocated":266240,"data_stored":109022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8060928},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28000000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32000000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47299999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51900000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8076,"kb_used_data":260,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363764,"statfs":{"total":96636764160,"available":96628494336,"internally_reserved":0,"allocated":266240,"data_stored":109401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.252}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34200000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41899999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58899999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63400000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84199999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88100000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8144,"kb_used_data":264,"kb_used_omap":0,"kb_used_meta":7872,"kb_avail":94363696,"statfs":{"total":96636764160,"available":96628424704,"internally_reserved":0,"allocated":270336,"data_stored":109491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8060928},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28699999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70499999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8224,"kb_used_data":280,"kb_used_omap":0,"kb_used_meta":7936,"kb_avail":94363616,"statfs":{"total":96636764160,"available":96628342784,"internally_reserved":0,"allocated":286720,"data_stored":113110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8126464},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39600000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51600000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45400000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54200000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.5}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55800000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56899999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8144,"kb_used_data":264,"kb_used_omap":0,"kb_used_meta":7872,"kb_avail":94363696,"statfs":{"total":96636764160,"available":96628424704,"internally_reserved":0,"allocated":270336,"data_stored":109092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8060928},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67900000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73499999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61099999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59799999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51500000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-05-20T01:16:46.914 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-05-20T01:16:46.915 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-05-20T01:16:46.915 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-05-20T01:16:46.916 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-05-20T01:16:46.916 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-05-20T01:16:47.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.066+0000 7f2142638700 1 Processor -- start 2023-05-20T01:16:47.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.066+0000 7f2142638700 1 -- start start 2023-05-20T01:16:47.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.067+0000 7f2142638700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f213c0cb670 0x7f213c0ce340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:47.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.067+0000 7f2142638700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f213c0caaa0 0x7f213c0ce880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:47.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.067+0000 7f2142638700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f213c159d90 con 0x7f213c1599c0 2023-05-20T01:16:47.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.067+0000 7f2142638700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f213c0cdf20 con 0x7f213c0caaa0 2023-05-20T01:16:47.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.067+0000 7f2142638700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f213c0cdb80 con 0x7f213c0cb670 2023-05-20T01:16:47.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.068+0000 7f213bfff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f213c1599c0 0x7f213c0caf60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:41318/0 (socket says 172.21.15.16:41318) 2023-05-20T01:16:47.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.068+0000 7f213bfff700 1 -- 172.21.15.16:0/363402882 learned_addr learned my addr 172.21.15.16:0/363402882 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:16:47.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.068+0000 7f2140bd5700 1 --2- 172.21.15.16:0/363402882 >> v2:172.21.15.16:3300/0 conn(0x7f213c0caaa0 0x7f213c0ce880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:47.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.068+0000 7f213b7fe700 1 --2- 172.21.15.16:0/363402882 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f213c0cb670 0x7f213c0ce340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:47.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.068+0000 7f213affd700 1 -- 172.21.15.16:0/363402882 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2445722963 0 0) 0x7f212c001110 con 0x7f213c1599c0 2023-05-20T01:16:47.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.068+0000 7f213affd700 1 -- 172.21.15.16:0/363402882 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21280035b0 con 0x7f213c1599c0 2023-05-20T01:16:47.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.068+0000 7f2140bd5700 1 -- 172.21.15.16:0/363402882 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f213c0cb670 msgr2=0x7f213c0ce340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:47.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.068+0000 7f2140bd5700 1 --2- 172.21.15.16:0/363402882 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f213c0cb670 0x7f213c0ce340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:47.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.068+0000 7f2140bd5700 1 -- 172.21.15.16:0/363402882 >> v1:172.21.15.16:6789/0 conn(0x7f213c1599c0 legacy=0x7f213c0caf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:47.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.068+0000 7f2140bd5700 1 -- 172.21.15.16:0/363402882 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f213c06eaf0 con 0x7f213c0caaa0 2023-05-20T01:16:47.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.068+0000 7f2140bd5700 1 --2- 172.21.15.16:0/363402882 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f213c0caaa0 0x7f213c0ce880 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2134009d90 tx=0x7f213400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adff89ee0f756877 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:47.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.068+0000 7f213affd700 1 -- 172.21.15.16:0/363402882 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f213400e070 con 0x7f213c0caaa0 2023-05-20T01:16:47.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.068+0000 7f213affd700 1 -- 172.21.15.16:0/363402882 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f213400bb20 con 0x7f213c0caaa0 2023-05-20T01:16:47.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.068+0000 7f213affd700 1 -- 172.21.15.16:0/363402882 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2134012430 con 0x7f213c0caaa0 2023-05-20T01:16:47.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.069+0000 7f2142638700 1 -- 172.21.15.16:0/363402882 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f213c0caaa0 msgr2=0x7f213c0ce880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:47.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.069+0000 7f2142638700 1 --2- 172.21.15.16:0/363402882 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f213c0caaa0 0x7f213c0ce880 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2134009d90 tx=0x7f213400b330 comp rx=0 tx=0).stop 2023-05-20T01:16:47.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.069+0000 7f2142638700 1 -- 172.21.15.16:0/363402882 shutdown_connections 2023-05-20T01:16:47.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.069+0000 7f2142638700 1 --2- 172.21.15.16:0/363402882 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f213c0caaa0 0x7f213c0ce880 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:47.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.069+0000 7f2142638700 1 --2- 172.21.15.16:0/363402882 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f213c0cb670 0x7f213c0ce340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:47.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.069+0000 7f2142638700 1 -- 172.21.15.16:0/363402882 >> 172.21.15.16:0/363402882 conn(0x7f213c0d0c30 msgr2=0x7f213c0b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:47.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.069+0000 7f2142638700 1 -- 172.21.15.16:0/363402882 shutdown_connections 2023-05-20T01:16:47.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.069+0000 7f2142638700 1 -- 172.21.15.16:0/363402882 wait complete. 2023-05-20T01:16:47.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.070+0000 7f2142638700 1 Processor -- start 2023-05-20T01:16:47.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.070+0000 7f2142638700 1 -- start start 2023-05-20T01:16:47.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.071+0000 7f2142638700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f213c0caaa0 0x7f213c14de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:47.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.071+0000 7f2142638700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f213c0cb670 0x7f213c14e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:47.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.071+0000 7f2142638700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f213c1599c0 0x7f213c1f5b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:47.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.071+0000 7f2142638700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f213c0b04b0 con 0x7f213c0cb670 2023-05-20T01:16:47.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.071+0000 7f2142638700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f213c0afb50 con 0x7f213c0caaa0 2023-05-20T01:16:47.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.071+0000 7f2142638700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f213c0afdf0 con 0x7f213c1599c0 2023-05-20T01:16:47.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.071+0000 7f213b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f213c0cb670 0x7f213c14e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:47.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.071+0000 7f213b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f213c0cb670 0x7f213c14e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58030/0 (socket says 172.21.15.16:58030) 2023-05-20T01:16:47.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.071+0000 7f213b7fe700 1 -- 172.21.15.16:0/491780201 learned_addr learned my addr 172.21.15.16:0/491780201 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:47.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.071+0000 7f2140bd5700 1 --2- 172.21.15.16:0/491780201 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f213c1599c0 0x7f213c1f5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:47.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.071+0000 7f213bfff700 1 --2- 172.21.15.16:0/491780201 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f213c0caaa0 0x7f213c14de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:47.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.072+0000 7f213b7fe700 1 -- 172.21.15.16:0/491780201 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f213c1599c0 msgr2=0x7f213c1f5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:47.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.072+0000 7f213b7fe700 1 --2- 172.21.15.16:0/491780201 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f213c1599c0 0x7f213c1f5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:47.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.072+0000 7f213b7fe700 1 -- 172.21.15.16:0/491780201 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f213c0caaa0 msgr2=0x7f213c14de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:47.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.072+0000 7f213b7fe700 1 --2- 172.21.15.16:0/491780201 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f213c0caaa0 0x7f213c14de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:47.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.072+0000 7f213b7fe700 1 -- 172.21.15.16:0/491780201 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2134009a70 con 0x7f213c0cb670 2023-05-20T01:16:47.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.072+0000 7f213b7fe700 1 --2- 172.21.15.16:0/491780201 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f213c0cb670 0x7f213c14e3b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f212400e980 tx=0x7f212400ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:47.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.072+0000 7f2138ff9700 1 -- 172.21.15.16:0/491780201 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2124009710 con 0x7f213c0cb670 2023-05-20T01:16:47.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.072+0000 7f2138ff9700 1 -- 172.21.15.16:0/491780201 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f212400c690 con 0x7f213c0cb670 2023-05-20T01:16:47.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.072+0000 7f2138ff9700 1 -- 172.21.15.16:0/491780201 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f212400ca40 con 0x7f213c0cb670 2023-05-20T01:16:47.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.072+0000 7f2142638700 1 -- 172.21.15.16:0/491780201 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f213c06f0b0 con 0x7f213c0cb670 2023-05-20T01:16:47.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.072+0000 7f2142638700 1 -- 172.21.15.16:0/491780201 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f213c06edd0 con 0x7f213c0cb670 2023-05-20T01:16:47.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.073+0000 7f2142638700 1 -- 172.21.15.16:0/491780201 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f213c06e750 con 0x7f213c0cb670 2023-05-20T01:16:47.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.073+0000 7f2138ff9700 1 -- 172.21.15.16:0/491780201 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f212400cba0 con 0x7f213c0cb670 2023-05-20T01:16:47.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.074+0000 7f2138ff9700 1 --2- 172.21.15.16:0/491780201 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f211c06ace0 0x7f211c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:47.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.074+0000 7f213bfff700 1 --2- 172.21.15.16:0/491780201 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f211c06ace0 0x7f211c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:47.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.074+0000 7f2138ff9700 1 -- 172.21.15.16:0/491780201 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f2124089760 con 0x7f213c0cb670 2023-05-20T01:16:47.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.074+0000 7f213bfff700 1 --2- 172.21.15.16:0/491780201 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f211c06ace0 0x7f211c06d190 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f212c007fd0 tx=0x7f212c007c40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:47.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.078+0000 7f2138ff9700 1 -- 172.21.15.16:0/491780201 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f2124057fa0 con 0x7f213c0cb670 2023-05-20T01:16:47.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.331+0000 7f2142638700 1 -- 172.21.15.16:0/491780201 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f213c14e9b0 con 0x7f213c0cb670 2023-05-20T01:16:47.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.334+0000 7f2138ff9700 1 -- 172.21.15.16:0/491780201 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+170841 (secure 0 0 0) 0x7f213c14e9b0 con 0x7f213c0cb670 2023-05-20T01:16:47.340 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:16:47.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.337+0000 7f2142638700 1 -- 172.21.15.16:0/491780201 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f211c06ace0 msgr2=0x7f211c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:47.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.337+0000 7f2142638700 1 --2- 172.21.15.16:0/491780201 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f211c06ace0 0x7f211c06d190 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f212c007fd0 tx=0x7f212c007c40 comp rx=0 tx=0).stop 2023-05-20T01:16:47.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.337+0000 7f2142638700 1 -- 172.21.15.16:0/491780201 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f213c0cb670 msgr2=0x7f213c14e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:47.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.337+0000 7f2142638700 1 --2- 172.21.15.16:0/491780201 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f213c0cb670 0x7f213c14e3b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f212400e980 tx=0x7f212400ec90 comp rx=0 tx=0).stop 2023-05-20T01:16:47.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.337+0000 7f2142638700 1 -- 172.21.15.16:0/491780201 shutdown_connections 2023-05-20T01:16:47.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.337+0000 7f2142638700 1 --2- 172.21.15.16:0/491780201 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f211c06ace0 0x7f211c06d190 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:47.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.337+0000 7f2142638700 1 --2- 172.21.15.16:0/491780201 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f213c1599c0 0x7f213c1f5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:47.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.337+0000 7f2142638700 1 --2- 172.21.15.16:0/491780201 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f213c0caaa0 0x7f213c14de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:47.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.337+0000 7f2142638700 1 --2- 172.21.15.16:0/491780201 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f213c0cb670 0x7f213c14e3b0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:47.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.337+0000 7f2142638700 1 -- 172.21.15.16:0/491780201 >> 172.21.15.16:0/491780201 conn(0x7f213c0d0c30 msgr2=0x7f213c0b2ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:47.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.337+0000 7f2142638700 1 -- 172.21.15.16:0/491780201 shutdown_connections 2023-05-20T01:16:47.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.337+0000 7f2142638700 1 -- 172.21.15.16:0/491780201 wait complete. 2023-05-20T01:16:47.352 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":5,"active_gid":4104,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3817237998},{"type":"v1","addr":"172.21.15.16:6833","nonce":3817237998}]},"active_addr":"172.21.15.16:6833/3817237998","active_change":"2023-05-20T01:16:15.309844+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4108,"name":"x","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.16:0","nonce":2935994262}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.16:0","nonce":1671517057}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.16:0","nonce":966059318}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.16:0","nonce":3877064489}]}]} 2023-05-20T01:16:47.357 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-05-20T01:16:47.357 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-05-20T01:16:47.357 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-20T01:16:47.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.479+0000 7f5a2c26a700 1 Processor -- start 2023-05-20T01:16:47.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.479+0000 7f5a2c26a700 1 -- start start 2023-05-20T01:16:47.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.480+0000 7f5a2c26a700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f5a24157190 0x7f5a240cc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:47.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.480+0000 7f5a2c26a700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5a24071ad0 0x7f5a240ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:47.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.480+0000 7f5a2c26a700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a240cb590 con 0x7f5a240cccf0 2023-05-20T01:16:47.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.480+0000 7f5a2c26a700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a24071f50 con 0x7f5a24071ad0 2023-05-20T01:16:47.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.480+0000 7f5a2c26a700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a240ca9a0 con 0x7f5a24157190 2023-05-20T01:16:47.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.480+0000 7f5a2a807700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5a24071ad0 0x7f5a240ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:47.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.480+0000 7f5a2a807700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5a24071ad0 0x7f5a240ce740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58040/0 (socket says 172.21.15.16:58040) 2023-05-20T01:16:47.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.480+0000 7f5a2a807700 1 -- 172.21.15.16:0/2300228393 learned_addr learned my addr 172.21.15.16:0/2300228393 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:47.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.480+0000 7f5a2a006700 1 --2- 172.21.15.16:0/2300228393 >> v2:172.21.15.132:3300/0 conn(0x7f5a24157190 0x7f5a240cc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:47.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.480+0000 7f5a2a807700 1 -- 172.21.15.16:0/2300228393 >> v2:172.21.15.132:3300/0 conn(0x7f5a24157190 msgr2=0x7f5a240cc7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:47.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.480+0000 7f5a2a807700 1 --2- 172.21.15.16:0/2300228393 >> v2:172.21.15.132:3300/0 conn(0x7f5a24157190 0x7f5a240cc7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:47.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.480+0000 7f5a2a807700 1 -- 172.21.15.16:0/2300228393 >> v1:172.21.15.132:6789/0 conn(0x7f5a240cccf0 legacy=0x7f5a240713c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:47.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.480+0000 7f5a2a807700 1 -- 172.21.15.16:0/2300228393 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a240cac60 con 0x7f5a24071ad0 2023-05-20T01:16:47.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.480+0000 7f5a2a006700 1 --2- 172.21.15.16:0/2300228393 >> v2:172.21.15.132:3300/0 conn(0x7f5a24157190 0x7f5a240cc7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:16:47.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.481+0000 7f5a2a807700 1 --2- 172.21.15.16:0/2300228393 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a24071ad0 0x7f5a240ce740 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5a24070c40 tx=0x7f5a1c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42b4206d74aaf4e server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:47.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.481+0000 7f5a29004700 1 -- 172.21.15.16:0/2300228393 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a1c00e070 con 0x7f5a24071ad0 2023-05-20T01:16:47.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.481+0000 7f5a29004700 1 -- 172.21.15.16:0/2300228393 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a1c003a20 con 0x7f5a24071ad0 2023-05-20T01:16:47.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.481+0000 7f5a29004700 1 -- 172.21.15.16:0/2300228393 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a1c003dd0 con 0x7f5a24071ad0 2023-05-20T01:16:47.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.481+0000 7f5a2c26a700 1 -- 172.21.15.16:0/2300228393 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a24071ad0 msgr2=0x7f5a240ce740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:47.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.481+0000 7f5a2c26a700 1 --2- 172.21.15.16:0/2300228393 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a24071ad0 0x7f5a240ce740 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5a24070c40 tx=0x7f5a1c00b330 comp rx=0 tx=0).stop 2023-05-20T01:16:47.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.481+0000 7f5a2c26a700 1 -- 172.21.15.16:0/2300228393 shutdown_connections 2023-05-20T01:16:47.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.481+0000 7f5a2c26a700 1 --2- 172.21.15.16:0/2300228393 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a24071ad0 0x7f5a240ce740 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:47.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.481+0000 7f5a2c26a700 1 --2- 172.21.15.16:0/2300228393 >> v2:172.21.15.132:3300/0 conn(0x7f5a24157190 0x7f5a240cc7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:47.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.481+0000 7f5a2c26a700 1 -- 172.21.15.16:0/2300228393 >> 172.21.15.16:0/2300228393 conn(0x7f5a240cffb0 msgr2=0x7f5a240d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:47.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.481+0000 7f5a2c26a700 1 -- 172.21.15.16:0/2300228393 shutdown_connections 2023-05-20T01:16:47.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.481+0000 7f5a2c26a700 1 -- 172.21.15.16:0/2300228393 wait complete. 2023-05-20T01:16:47.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.483+0000 7f5a2c26a700 1 Processor -- start 2023-05-20T01:16:47.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.483+0000 7f5a2c26a700 1 -- start start 2023-05-20T01:16:47.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.483+0000 7f5a2c26a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a24071ad0 0x7f5a241f5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:47.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.483+0000 7f5a2c26a700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5a240cccf0 0x7f5a240c9160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:47.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.483+0000 7f5a2c26a700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5a24157190 0x7f5a240b5720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:47.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.483+0000 7f5a2c26a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a241f69d0 con 0x7f5a24071ad0 2023-05-20T01:16:47.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.483+0000 7f5a2c26a700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a241f6130 con 0x7f5a240cccf0 2023-05-20T01:16:47.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.484+0000 7f5a2c26a700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a241f6410 con 0x7f5a24157190 2023-05-20T01:16:47.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.484+0000 7f5a2a006700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a24071ad0 0x7f5a241f5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:47.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.484+0000 7f5a2a006700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a24071ad0 0x7f5a241f5ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58042/0 (socket says 172.21.15.16:58042) 2023-05-20T01:16:47.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.484+0000 7f5a2a006700 1 -- 172.21.15.16:0/3229209534 learned_addr learned my addr 172.21.15.16:0/3229209534 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:47.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.484+0000 7f5a29805700 1 --2- 172.21.15.16:0/3229209534 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5a240cccf0 0x7f5a240c9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:47.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.484+0000 7f5a2a807700 1 --2- 172.21.15.16:0/3229209534 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5a24157190 0x7f5a240b5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:47.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.484+0000 7f5a2a006700 1 -- 172.21.15.16:0/3229209534 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5a24157190 msgr2=0x7f5a240b5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:47.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.484+0000 7f5a2a006700 1 --2- 172.21.15.16:0/3229209534 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5a24157190 0x7f5a240b5720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:47.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.484+0000 7f5a2a006700 1 -- 172.21.15.16:0/3229209534 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5a240cccf0 msgr2=0x7f5a240c9160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:47.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.484+0000 7f5a2a006700 1 --2- 172.21.15.16:0/3229209534 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5a240cccf0 0x7f5a240c9160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:47.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.484+0000 7f5a2a006700 1 -- 172.21.15.16:0/3229209534 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a1c009b30 con 0x7f5a24071ad0 2023-05-20T01:16:47.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.484+0000 7f5a2a006700 1 --2- 172.21.15.16:0/3229209534 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a24071ad0 0x7f5a241f5ac0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5a1400c7f0 tx=0x7f5a1400cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:47.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.484+0000 7f5a1affd700 1 -- 172.21.15.16:0/3229209534 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a14011800 con 0x7f5a24071ad0 2023-05-20T01:16:47.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.484+0000 7f5a1affd700 1 -- 172.21.15.16:0/3229209534 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a14011960 con 0x7f5a24071ad0 2023-05-20T01:16:47.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.484+0000 7f5a1affd700 1 -- 172.21.15.16:0/3229209534 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a14011d10 con 0x7f5a24071ad0 2023-05-20T01:16:47.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.484+0000 7f5a2c26a700 1 -- 172.21.15.16:0/3229209534 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a240cb220 con 0x7f5a24071ad0 2023-05-20T01:16:47.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.484+0000 7f5a2c26a700 1 -- 172.21.15.16:0/3229209534 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a240caf40 con 0x7f5a24071ad0 2023-05-20T01:16:47.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.485+0000 7f5a2c26a700 1 -- 172.21.15.16:0/3229209534 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a24157560 con 0x7f5a24071ad0 2023-05-20T01:16:47.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.486+0000 7f5a1affd700 1 -- 172.21.15.16:0/3229209534 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5a14018360 con 0x7f5a24071ad0 2023-05-20T01:16:47.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.486+0000 7f5a1affd700 1 --2- 172.21.15.16:0/3229209534 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f5a0806ace0 0x7f5a0806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:47.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.486+0000 7f5a1affd700 1 -- 172.21.15.16:0/3229209534 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f5a140896c0 con 0x7f5a24071ad0 2023-05-20T01:16:47.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.486+0000 7f5a29805700 1 --2- 172.21.15.16:0/3229209534 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f5a0806ace0 0x7f5a0806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:47.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.486+0000 7f5a29805700 1 --2- 172.21.15.16:0/3229209534 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f5a0806ace0 0x7f5a0806d190 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5a240d09b0 tx=0x7f5a20009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:47.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.490+0000 7f5a1affd700 1 -- 172.21.15.16:0/3229209534 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f5a14057fa0 con 0x7f5a24071ad0 2023-05-20T01:16:47.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.687+0000 7f5a2c26a700 1 -- 172.21.15.16:0/3229209534 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5a240747b0 con 0x7f5a24071ad0 2023-05-20T01:16:47.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.689+0000 7f5a1affd700 1 -- 172.21.15.16:0/3229209534 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+20026 (secure 0 0 0) 0x7f5a14057b30 con 0x7f5a24071ad0 2023-05-20T01:16:47.693 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:16:47.693 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"fsid":"c2245cc5-ecab-4087-a59b-bc81db901676","created":"2023-05-20T01:16:09.549230+0000","modified":"2023-05-20T01:16:32.768495+0000","last_up_change":"2023-05-20T01:16:19.788624+0000","last_in_change":"2023-05-20T01:16:12.092160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-05-20T01:16:20.934366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-05-20T01:16:21.372294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-05-20T01:16:26.208287+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-05-20T01:16:27.292268+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-05-20T01:16:30.357285+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"317a83cd-19c8-4958-a301-6087ef342289","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6825","nonce":3923634179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6827","nonce":3923634179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6831","nonce":3923634179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6829","nonce":3923634179}]},"public_addr":"172.21.15.16:6825/3923634179","cluster_addr":"172.21.15.16:6827/3923634179","heartbeat_back_addr":"172.21.15.16:6831/3923634179","heartbeat_front_addr":"172.21.15.16:6829/3923634179","state":["exists","up"]},{"osd":1,"uuid":"5581046f-8219-4089-8d7d-b7846399505e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6809","nonce":3860264220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6811","nonce":3860264220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6815","nonce":3860264220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6813","nonce":3860264220}]},"public_addr":"172.21.15.16:6809/3860264220","cluster_addr":"172.21.15.16:6811/3860264220","heartbeat_back_addr":"172.21.15.16:6815/3860264220","heartbeat_front_addr":"172.21.15.16:6813/3860264220","state":["exists","up"]},{"osd":2,"uuid":"072e9e66-4133-478e-8d4a-64d46c2963f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6817","nonce":293187030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6819","nonce":293187030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6823","nonce":293187030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6821","nonce":293187030}]},"public_addr":"172.21.15.16:6817/293187030","cluster_addr":"172.21.15.16:6819/293187030","heartbeat_back_addr":"172.21.15.16:6823/293187030","heartbeat_front_addr":"172.21.15.16:6821/293187030","state":["exists","up"]},{"osd":3,"uuid":"11e1bba6-00f8-48dd-9028-d8d716b7f145","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6801","nonce":3669781890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6803","nonce":3669781890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6807","nonce":3669781890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6805","nonce":3669781890}]},"public_addr":"172.21.15.16:6801/3669781890","cluster_addr":"172.21.15.16:6803/3669781890","heartbeat_back_addr":"172.21.15.16:6807/3669781890","heartbeat_front_addr":"172.21.15.16:6805/3669781890","state":["exists","up"]},{"osd":4,"uuid":"b8909c3a-cf70-46cb-8d0b-009c707cf9d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6819","nonce":506317223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6823","nonce":506317223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6829","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6831","nonce":506317223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6827","nonce":506317223}]},"public_addr":"172.21.15.132:6819/506317223","cluster_addr":"172.21.15.132:6823/506317223","heartbeat_back_addr":"172.21.15.132:6831/506317223","heartbeat_front_addr":"172.21.15.132:6827/506317223","state":["exists","up"]},{"osd":5,"uuid":"b1818304-4583-47bf-a6ec-0bfbb5166233","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6818","nonce":1844689995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6822","nonce":1844689995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6830","nonce":1844689995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6826","nonce":1844689995}]},"public_addr":"172.21.15.132:6818/1844689995","cluster_addr":"172.21.15.132:6822/1844689995","heartbeat_back_addr":"172.21.15.132:6830/1844689995","heartbeat_front_addr":"172.21.15.132:6826/1844689995","state":["exists","up"]},{"osd":6,"uuid":"d37f4691-eed1-4a93-8818-18354f03f4a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6801","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6802","nonce":727581123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6805","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6807","nonce":727581123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6813","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6815","nonce":727581123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6811","nonce":727581123}]},"public_addr":"172.21.15.132:6802/727581123","cluster_addr":"172.21.15.132:6807/727581123","heartbeat_back_addr":"172.21.15.132:6815/727581123","heartbeat_front_addr":"172.21.15.132:6811/727581123","state":["exists","up"]},{"osd":7,"uuid":"4f712f62-5d1d-4816-8042-03209144f8b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6803","nonce":466001149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6806","nonce":466001149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6814","nonce":466001149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6810","nonce":466001149}]},"public_addr":"172.21.15.132:6803/466001149","cluster_addr":"172.21.15.132:6806/466001149","heartbeat_back_addr":"172.21.15.132:6814/466001149","heartbeat_front_addr":"172.21.15.132:6810/466001149","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.338863+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.278680+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305694+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.246835+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.276237+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.304441+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305421+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.267050+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-20T01:16:47.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.691+0000 7f5a2c26a700 1 -- 172.21.15.16:0/3229209534 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f5a0806ace0 msgr2=0x7f5a0806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:47.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.691+0000 7f5a2c26a700 1 --2- 172.21.15.16:0/3229209534 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f5a0806ace0 0x7f5a0806d190 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5a240d09b0 tx=0x7f5a20009040 comp rx=0 tx=0).stop 2023-05-20T01:16:47.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.691+0000 7f5a2c26a700 1 -- 172.21.15.16:0/3229209534 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a24071ad0 msgr2=0x7f5a241f5ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:47.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.691+0000 7f5a2c26a700 1 --2- 172.21.15.16:0/3229209534 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a24071ad0 0x7f5a241f5ac0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5a1400c7f0 tx=0x7f5a1400cbb0 comp rx=0 tx=0).stop 2023-05-20T01:16:47.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.691+0000 7f5a2c26a700 1 -- 172.21.15.16:0/3229209534 shutdown_connections 2023-05-20T01:16:47.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.691+0000 7f5a2c26a700 1 --2- 172.21.15.16:0/3229209534 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f5a0806ace0 0x7f5a0806d190 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:47.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.691+0000 7f5a2c26a700 1 --2- 172.21.15.16:0/3229209534 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5a24157190 0x7f5a240b5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:47.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.691+0000 7f5a2c26a700 1 --2- 172.21.15.16:0/3229209534 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a24071ad0 0x7f5a241f5ac0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:47.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.691+0000 7f5a2c26a700 1 --2- 172.21.15.16:0/3229209534 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5a240cccf0 0x7f5a240c9160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:47.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.691+0000 7f5a2c26a700 1 -- 172.21.15.16:0/3229209534 >> 172.21.15.16:0/3229209534 conn(0x7f5a240cffb0 msgr2=0x7f5a24153dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:47.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.691+0000 7f5a2c26a700 1 -- 172.21.15.16:0/3229209534 shutdown_connections 2023-05-20T01:16:47.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.691+0000 7f5a2c26a700 1 -- 172.21.15.16:0/3229209534 wait complete. 2023-05-20T01:16:47.708 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-05-20T01:16:47.709 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-20T01:16:47.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.830+0000 7fc6bcbc0700 1 Processor -- start 2023-05-20T01:16:47.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.830+0000 7fc6bcbc0700 1 -- start start 2023-05-20T01:16:47.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.830+0000 7fc6bcbc0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc6b8157190 0x7fc6b80cc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:47.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.830+0000 7fc6bcbc0700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fc6b80cccf0 0x7fc6b80713c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:47.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.831+0000 7fc6bcbc0700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6b80cb590 con 0x7fc6b8071900 2023-05-20T01:16:47.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.831+0000 7fc6bcbc0700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6b80ca9a0 con 0x7fc6b80cccf0 2023-05-20T01:16:47.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.831+0000 7fc6bcbc0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6b80cac60 con 0x7fc6b8157190 2023-05-20T01:16:47.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.831+0000 7fc6b659c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc6b8157190 0x7fc6b80cc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:47.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.831+0000 7fc6b5d9b700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fc6b80cccf0 0x7fc6b80713c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:47.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.831+0000 7fc6b5d9b700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fc6b80cccf0 0x7fc6b80713c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.16:55072/0 (socket says 172.21.15.16:55072) 2023-05-20T01:16:47.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.831+0000 7fc6b659c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc6b8157190 0x7fc6b80cc7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.16:58088/0 (socket says 172.21.15.16:58088) 2023-05-20T01:16:47.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.831+0000 7fc6b6d9d700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7fc6b8071900 0x7fc6b815d220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.16:51062/0 (socket says 172.21.15.16:51062) 2023-05-20T01:16:47.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.831+0000 7fc6b659c700 1 -- 172.21.15.16:0/1374304985 learned_addr learned my addr 172.21.15.16:0/1374304985 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:47.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.831+0000 7fc6b5d9b700 1 -- 172.21.15.16:0/1374304985 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc6b8157190 msgr2=0x7fc6b80cc7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:47.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.831+0000 7fc6b5d9b700 1 --2- 172.21.15.16:0/1374304985 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc6b8157190 0x7fc6b80cc7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:47.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.831+0000 7fc6b5d9b700 1 -- 172.21.15.16:0/1374304985 >> v1:172.21.15.132:6789/0 conn(0x7fc6b8071900 legacy=0x7fc6b815d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:47.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.831+0000 7fc6b5d9b700 1 -- 172.21.15.16:0/1374304985 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6b80caf40 con 0x7fc6b80cccf0 2023-05-20T01:16:47.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.831+0000 7fc6b659c700 1 --2- 172.21.15.16:0/1374304985 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc6b8157190 0x7fc6b80cc7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-05-20T01:16:47.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.831+0000 7fc6b5d9b700 1 --2- 172.21.15.16:0/1374304985 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc6b80cccf0 0x7fc6b80713c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc6a0009840 tx=0x7fc6a0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f6d4715d0f6f2b34 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:47.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.832+0000 7fc6b559a700 1 -- 172.21.15.16:0/1374304985 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6a0011830 con 0x7fc6b80cccf0 2023-05-20T01:16:47.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.832+0000 7fc6b559a700 1 -- 172.21.15.16:0/1374304985 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6a0011990 con 0x7fc6b80cccf0 2023-05-20T01:16:47.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.832+0000 7fc6b559a700 1 -- 172.21.15.16:0/1374304985 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6a0011d20 con 0x7fc6b80cccf0 2023-05-20T01:16:47.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.832+0000 7fc6bcbc0700 1 -- 172.21.15.16:0/1374304985 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc6b80cccf0 msgr2=0x7fc6b80713c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:47.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.832+0000 7fc6bcbc0700 1 --2- 172.21.15.16:0/1374304985 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc6b80cccf0 0x7fc6b80713c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc6a0009840 tx=0x7fc6a0009b50 comp rx=0 tx=0).stop 2023-05-20T01:16:47.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.832+0000 7fc6bcbc0700 1 -- 172.21.15.16:0/1374304985 shutdown_connections 2023-05-20T01:16:47.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.832+0000 7fc6bcbc0700 1 --2- 172.21.15.16:0/1374304985 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc6b8157190 0x7fc6b80cc7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:47.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.832+0000 7fc6bcbc0700 1 --2- 172.21.15.16:0/1374304985 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc6b80cccf0 0x7fc6b80713c0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:47.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.832+0000 7fc6bcbc0700 1 -- 172.21.15.16:0/1374304985 >> 172.21.15.16:0/1374304985 conn(0x7fc6b80cffb0 msgr2=0x7fc6b80d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:47.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.832+0000 7fc6bcbc0700 1 -- 172.21.15.16:0/1374304985 shutdown_connections 2023-05-20T01:16:47.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.832+0000 7fc6bcbc0700 1 -- 172.21.15.16:0/1374304985 wait complete. 2023-05-20T01:16:47.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.834+0000 7fc6bcbc0700 1 Processor -- start 2023-05-20T01:16:47.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.834+0000 7fc6bcbc0700 1 -- start start 2023-05-20T01:16:47.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.835+0000 7fc6bcbc0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6b8071900 0x7fc6b80b5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:47.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.835+0000 7fc6bcbc0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc6b80cccf0 0x7fc6b81f1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:47.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.835+0000 7fc6bcbc0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc6b8157190 0x7fc6b80c4b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:47.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.835+0000 7fc6bcbc0700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6b80b5e00 con 0x7fc6b8071900 2023-05-20T01:16:47.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.835+0000 7fc6bcbc0700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6b8154950 con 0x7fc6b8157190 2023-05-20T01:16:47.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.835+0000 7fc6bcbc0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6b8145680 con 0x7fc6b80cccf0 2023-05-20T01:16:47.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.835+0000 7fc6b659c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6b8071900 0x7fc6b80b5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:47.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.835+0000 7fc6b659c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6b8071900 0x7fc6b80b5790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58054/0 (socket says 172.21.15.16:58054) 2023-05-20T01:16:47.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.835+0000 7fc6b659c700 1 -- 172.21.15.16:0/969660544 learned_addr learned my addr 172.21.15.16:0/969660544 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:47.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.835+0000 7fc6b5d9b700 1 --2- 172.21.15.16:0/969660544 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc6b80cccf0 0x7fc6b81f1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:47.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.835+0000 7fc6b6d9d700 1 --2- 172.21.15.16:0/969660544 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc6b8157190 0x7fc6b80c4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:47.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.835+0000 7fc6b659c700 1 -- 172.21.15.16:0/969660544 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc6b80cccf0 msgr2=0x7fc6b81f1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:47.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.835+0000 7fc6b659c700 1 --2- 172.21.15.16:0/969660544 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc6b80cccf0 0x7fc6b81f1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:47.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.835+0000 7fc6b659c700 1 -- 172.21.15.16:0/969660544 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc6b8157190 msgr2=0x7fc6b80c4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:47.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.835+0000 7fc6b659c700 1 --2- 172.21.15.16:0/969660544 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc6b8157190 0x7fc6b80c4b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:47.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.835+0000 7fc6b659c700 1 -- 172.21.15.16:0/969660544 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6a8009550 con 0x7fc6b8071900 2023-05-20T01:16:47.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.835+0000 7fc6b659c700 1 --2- 172.21.15.16:0/969660544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6b8071900 0x7fc6b80b5790 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc6a800e9b0 tx=0x7fc6a800ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:47.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.835+0000 7fc6aeffd700 1 -- 172.21.15.16:0/969660544 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6a800ee50 con 0x7fc6b8071900 2023-05-20T01:16:47.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.835+0000 7fc6aeffd700 1 -- 172.21.15.16:0/969660544 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6a800c850 con 0x7fc6b8071900 2023-05-20T01:16:47.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.835+0000 7fc6bcbc0700 1 -- 172.21.15.16:0/969660544 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6a00094a0 con 0x7fc6b8071900 2023-05-20T01:16:47.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.835+0000 7fc6aeffd700 1 -- 172.21.15.16:0/969660544 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6a800cbc0 con 0x7fc6b8071900 2023-05-20T01:16:47.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.835+0000 7fc6bcbc0700 1 -- 172.21.15.16:0/969660544 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6b8157560 con 0x7fc6b8071900 2023-05-20T01:16:47.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.836+0000 7fc6bcbc0700 1 -- 172.21.15.16:0/969660544 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6b80cb220 con 0x7fc6b8071900 2023-05-20T01:16:47.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.837+0000 7fc6aeffd700 1 -- 172.21.15.16:0/969660544 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc6a800cd20 con 0x7fc6b8071900 2023-05-20T01:16:47.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.837+0000 7fc6aeffd700 1 --2- 172.21.15.16:0/969660544 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc69806ace0 0x7fc69806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:47.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.837+0000 7fc6b5d9b700 1 --2- 172.21.15.16:0/969660544 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc69806ace0 0x7fc69806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:47.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.838+0000 7fc6aeffd700 1 -- 172.21.15.16:0/969660544 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fc6a80896e0 con 0x7fc6b8071900 2023-05-20T01:16:47.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.838+0000 7fc6b5d9b700 1 --2- 172.21.15.16:0/969660544 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc69806ace0 0x7fc69806d190 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc6a0000c00 tx=0x7fc6a0008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:47.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:47.842+0000 7fc6aeffd700 1 -- 172.21.15.16:0/969660544 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fc6a80544b0 con 0x7fc6b8071900 2023-05-20T01:16:48.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.043+0000 7fc6bcbc0700 1 -- 172.21.15.16:0/969660544 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc6b80cd120 con 0x7fc6b8071900 2023-05-20T01:16:48.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.044+0000 7fc6aeffd700 1 -- 172.21.15.16:0/969660544 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+20026 (secure 0 0 0) 0x7fc6a8057ad0 con 0x7fc6b8071900 2023-05-20T01:16:48.048 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:16:48.049 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"fsid":"c2245cc5-ecab-4087-a59b-bc81db901676","created":"2023-05-20T01:16:09.549230+0000","modified":"2023-05-20T01:16:32.768495+0000","last_up_change":"2023-05-20T01:16:19.788624+0000","last_in_change":"2023-05-20T01:16:12.092160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-05-20T01:16:20.934366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-05-20T01:16:21.372294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-05-20T01:16:26.208287+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-05-20T01:16:27.292268+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-05-20T01:16:30.357285+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"317a83cd-19c8-4958-a301-6087ef342289","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6825","nonce":3923634179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6827","nonce":3923634179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6831","nonce":3923634179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6829","nonce":3923634179}]},"public_addr":"172.21.15.16:6825/3923634179","cluster_addr":"172.21.15.16:6827/3923634179","heartbeat_back_addr":"172.21.15.16:6831/3923634179","heartbeat_front_addr":"172.21.15.16:6829/3923634179","state":["exists","up"]},{"osd":1,"uuid":"5581046f-8219-4089-8d7d-b7846399505e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6809","nonce":3860264220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6811","nonce":3860264220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6815","nonce":3860264220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6813","nonce":3860264220}]},"public_addr":"172.21.15.16:6809/3860264220","cluster_addr":"172.21.15.16:6811/3860264220","heartbeat_back_addr":"172.21.15.16:6815/3860264220","heartbeat_front_addr":"172.21.15.16:6813/3860264220","state":["exists","up"]},{"osd":2,"uuid":"072e9e66-4133-478e-8d4a-64d46c2963f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6817","nonce":293187030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6819","nonce":293187030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6823","nonce":293187030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6821","nonce":293187030}]},"public_addr":"172.21.15.16:6817/293187030","cluster_addr":"172.21.15.16:6819/293187030","heartbeat_back_addr":"172.21.15.16:6823/293187030","heartbeat_front_addr":"172.21.15.16:6821/293187030","state":["exists","up"]},{"osd":3,"uuid":"11e1bba6-00f8-48dd-9028-d8d716b7f145","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6801","nonce":3669781890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6803","nonce":3669781890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6807","nonce":3669781890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6805","nonce":3669781890}]},"public_addr":"172.21.15.16:6801/3669781890","cluster_addr":"172.21.15.16:6803/3669781890","heartbeat_back_addr":"172.21.15.16:6807/3669781890","heartbeat_front_addr":"172.21.15.16:6805/3669781890","state":["exists","up"]},{"osd":4,"uuid":"b8909c3a-cf70-46cb-8d0b-009c707cf9d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6819","nonce":506317223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6823","nonce":506317223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6829","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6831","nonce":506317223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6827","nonce":506317223}]},"public_addr":"172.21.15.132:6819/506317223","cluster_addr":"172.21.15.132:6823/506317223","heartbeat_back_addr":"172.21.15.132:6831/506317223","heartbeat_front_addr":"172.21.15.132:6827/506317223","state":["exists","up"]},{"osd":5,"uuid":"b1818304-4583-47bf-a6ec-0bfbb5166233","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6818","nonce":1844689995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6822","nonce":1844689995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6830","nonce":1844689995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6826","nonce":1844689995}]},"public_addr":"172.21.15.132:6818/1844689995","cluster_addr":"172.21.15.132:6822/1844689995","heartbeat_back_addr":"172.21.15.132:6830/1844689995","heartbeat_front_addr":"172.21.15.132:6826/1844689995","state":["exists","up"]},{"osd":6,"uuid":"d37f4691-eed1-4a93-8818-18354f03f4a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6801","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6802","nonce":727581123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6805","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6807","nonce":727581123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6813","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6815","nonce":727581123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6811","nonce":727581123}]},"public_addr":"172.21.15.132:6802/727581123","cluster_addr":"172.21.15.132:6807/727581123","heartbeat_back_addr":"172.21.15.132:6815/727581123","heartbeat_front_addr":"172.21.15.132:6811/727581123","state":["exists","up"]},{"osd":7,"uuid":"4f712f62-5d1d-4816-8042-03209144f8b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6803","nonce":466001149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6806","nonce":466001149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6814","nonce":466001149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6810","nonce":466001149}]},"public_addr":"172.21.15.132:6803/466001149","cluster_addr":"172.21.15.132:6806/466001149","heartbeat_back_addr":"172.21.15.132:6814/466001149","heartbeat_front_addr":"172.21.15.132:6810/466001149","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.338863+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.278680+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305694+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.246835+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.276237+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.304441+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305421+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.267050+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-20T01:16:48.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.046+0000 7fc6bcbc0700 1 -- 172.21.15.16:0/969660544 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc69806ace0 msgr2=0x7fc69806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:48.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.046+0000 7fc6bcbc0700 1 --2- 172.21.15.16:0/969660544 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc69806ace0 0x7fc69806d190 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc6a0000c00 tx=0x7fc6a0008030 comp rx=0 tx=0).stop 2023-05-20T01:16:48.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.046+0000 7fc6bcbc0700 1 -- 172.21.15.16:0/969660544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6b8071900 msgr2=0x7fc6b80b5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:48.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.046+0000 7fc6bcbc0700 1 --2- 172.21.15.16:0/969660544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6b8071900 0x7fc6b80b5790 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc6a800e9b0 tx=0x7fc6a800ecc0 comp rx=0 tx=0).stop 2023-05-20T01:16:48.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.046+0000 7fc6bcbc0700 1 -- 172.21.15.16:0/969660544 shutdown_connections 2023-05-20T01:16:48.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.046+0000 7fc6bcbc0700 1 --2- 172.21.15.16:0/969660544 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc6b8157190 0x7fc6b80c4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:48.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.046+0000 7fc6bcbc0700 1 --2- 172.21.15.16:0/969660544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6b8071900 0x7fc6b80b5790 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:48.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.046+0000 7fc6bcbc0700 1 --2- 172.21.15.16:0/969660544 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc69806ace0 0x7fc69806d190 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:48.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.046+0000 7fc6bcbc0700 1 --2- 172.21.15.16:0/969660544 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc6b80cccf0 0x7fc6b81f1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:48.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.046+0000 7fc6bcbc0700 1 -- 172.21.15.16:0/969660544 >> 172.21.15.16:0/969660544 conn(0x7fc6b80cffb0 msgr2=0x7fc6b806ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:48.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.046+0000 7fc6bcbc0700 1 -- 172.21.15.16:0/969660544 shutdown_connections 2023-05-20T01:16:48.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.046+0000 7fc6bcbc0700 1 -- 172.21.15.16:0/969660544 wait complete. 2023-05-20T01:16:48.063 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-05-20T01:16:48.063 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-05-20T01:16:48.064 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-05-20T01:16:48.064 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-05-20T01:16:48.065 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-05-20T01:16:48.065 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-05-20T01:16:48.066 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-05-20T01:16:48.066 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-05-20T01:16:48.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.272+0000 7f81a1ee0700 1 Processor -- start 2023-05-20T01:16:48.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.272+0000 7f81a1ee0700 1 -- start start 2023-05-20T01:16:48.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.274+0000 7f81a1ee0700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f819c0ce0a0 0x7f819c0ce470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:48.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.274+0000 7f81a1ee0700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f819c1590f0 con 0x7f819c0b1020 2023-05-20T01:16:48.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.274+0000 7f81a1ee0700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f819c1593b0 con 0x7f819c0700c0 2023-05-20T01:16:48.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.274+0000 7f81a1ee0700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f819c159690 con 0x7f819c0ce0a0 2023-05-20T01:16:48.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.274+0000 7f819affd700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f819c0700c0 0x7f819c0b0910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.16:51072/0 (socket says 172.21.15.16:51072) 2023-05-20T01:16:48.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.274+0000 7f819affd700 1 -- 172.21.15.16:0/3727583261 learned_addr learned my addr 172.21.15.16:0/3727583261 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:16:48.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.275+0000 7f819b7fe700 1 --2- 172.21.15.16:0/3727583261 >> v2:172.21.15.16:3300/0 conn(0x7f819c0ce0a0 0x7f819c0ce470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:48.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.275+0000 7f819a7fc700 1 -- 172.21.15.16:0/3727583261 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2767383218 0 0) 0x7f819c1593b0 con 0x7f819c0700c0 2023-05-20T01:16:48.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.275+0000 7f819a7fc700 1 -- 172.21.15.16:0/3727583261 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81840035b0 con 0x7f819c0700c0 2023-05-20T01:16:48.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.275+0000 7f819b7fe700 1 -- 172.21.15.16:0/3727583261 >> v1:172.21.15.132:6789/0 conn(0x7f819c0700c0 legacy=0x7f819c0b0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:48.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.275+0000 7f819b7fe700 1 -- 172.21.15.16:0/3727583261 >> v1:172.21.15.16:6789/0 conn(0x7f819c0b1020 legacy=0x7f819c06bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:48.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.275+0000 7f819b7fe700 1 -- 172.21.15.16:0/3727583261 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f819c159970 con 0x7f819c0ce0a0 2023-05-20T01:16:48.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.276+0000 7f819b7fe700 1 --2- 172.21.15.16:0/3727583261 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f819c0ce0a0 0x7f819c0ce470 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f819000a9e0 tx=0x7f819000acf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8075cebeeadb8de0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:48.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.276+0000 7f819a7fc700 1 -- 172.21.15.16:0/3727583261 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f819000cb30 con 0x7f819c0ce0a0 2023-05-20T01:16:48.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.276+0000 7f819a7fc700 1 -- 172.21.15.16:0/3727583261 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f819000cc90 con 0x7f819c0ce0a0 2023-05-20T01:16:48.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.276+0000 7f819a7fc700 1 -- 172.21.15.16:0/3727583261 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f819000ae80 con 0x7f819c0ce0a0 2023-05-20T01:16:48.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.277+0000 7f81a1ee0700 1 -- 172.21.15.16:0/3727583261 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f819c0ce0a0 msgr2=0x7f819c0ce470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:48.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.277+0000 7f81a1ee0700 1 --2- 172.21.15.16:0/3727583261 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f819c0ce0a0 0x7f819c0ce470 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f819000a9e0 tx=0x7f819000acf0 comp rx=0 tx=0).stop 2023-05-20T01:16:48.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.279+0000 7f81a1ee0700 1 -- 172.21.15.16:0/3727583261 shutdown_connections 2023-05-20T01:16:48.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.279+0000 7f81a1ee0700 1 --2- 172.21.15.16:0/3727583261 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f819c0ce0a0 0x7f819c0ce470 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:48.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.279+0000 7f81a1ee0700 1 -- 172.21.15.16:0/3727583261 >> 172.21.15.16:0/3727583261 conn(0x7f819c0cb910 msgr2=0x7f819c0cbd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:48.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.280+0000 7f81a1ee0700 1 -- 172.21.15.16:0/3727583261 shutdown_connections 2023-05-20T01:16:48.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.280+0000 7f81a1ee0700 1 -- 172.21.15.16:0/3727583261 wait complete. 2023-05-20T01:16:48.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.282+0000 7f81a1ee0700 1 Processor -- start 2023-05-20T01:16:48.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.282+0000 7f81a1ee0700 1 -- start start 2023-05-20T01:16:48.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.283+0000 7f81a1ee0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f819c0700c0 0x7f819c0c3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:48.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.283+0000 7f81a1ee0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f819c0b1020 0x7f819c16c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:48.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.283+0000 7f81a1ee0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f819c0ce0a0 0x7f819c155d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:48.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.283+0000 7f81a1ee0700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f819c0c7150 con 0x7f819c0b1020 2023-05-20T01:16:48.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.283+0000 7f81a1ee0700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f819c0c3c90 con 0x7f819c0ce0a0 2023-05-20T01:16:48.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.283+0000 7f81a1ee0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f819c0c4500 con 0x7f819c0700c0 2023-05-20T01:16:48.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.284+0000 7f819b7fe700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f819c0700c0 0x7f819c0c3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:48.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.284+0000 7f819b7fe700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f819c0700c0 0x7f819c0c3750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.16:58106/0 (socket says 172.21.15.16:58106) 2023-05-20T01:16:48.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.284+0000 7f819b7fe700 1 -- 172.21.15.16:0/1294154832 learned_addr learned my addr 172.21.15.16:0/1294154832 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:48.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.284+0000 7f819bfff700 1 --2- 172.21.15.16:0/1294154832 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f819c0ce0a0 0x7f819c155d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:48.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.284+0000 7f819b7fe700 1 -- 172.21.15.16:0/1294154832 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f819c0ce0a0 msgr2=0x7f819c155d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:48.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.284+0000 7f819b7fe700 1 --2- 172.21.15.16:0/1294154832 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f819c0ce0a0 0x7f819c155d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:48.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.284+0000 7f819b7fe700 1 -- 172.21.15.16:0/1294154832 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f819c0b1020 msgr2=0x7f819c16c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:48.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.284+0000 7f819b7fe700 1 --2- 172.21.15.16:0/1294154832 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f819c0b1020 0x7f819c16c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:48.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.284+0000 7f819b7fe700 1 -- 172.21.15.16:0/1294154832 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f819000a640 con 0x7f819c0700c0 2023-05-20T01:16:48.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.284+0000 7f819b7fe700 1 --2- 172.21.15.16:0/1294154832 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f819c0700c0 0x7f819c0c3750 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f819000a1c0 tx=0x7f8190004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:48.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.285+0000 7f8183fff700 1 -- 172.21.15.16:0/1294154832 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f819001d070 con 0x7f819c0700c0 2023-05-20T01:16:48.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.285+0000 7f81a1ee0700 1 -- 172.21.15.16:0/1294154832 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f819c159f30 con 0x7f819c0700c0 2023-05-20T01:16:48.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.285+0000 7f81a1ee0700 1 -- 172.21.15.16:0/1294154832 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f819c159c50 con 0x7f819c0700c0 2023-05-20T01:16:48.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.285+0000 7f8183fff700 1 -- 172.21.15.16:0/1294154832 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81900036a0 con 0x7f819c0700c0 2023-05-20T01:16:48.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.285+0000 7f8183fff700 1 -- 172.21.15.16:0/1294154832 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8190003a50 con 0x7f819c0700c0 2023-05-20T01:16:48.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.287+0000 7f81a1ee0700 1 -- 172.21.15.16:0/1294154832 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f817c000fb0 con 0x7f819c0700c0 2023-05-20T01:16:48.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.287+0000 7f292a617700 1 Processor -- start 2023-05-20T01:16:48.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.287+0000 7f292a617700 1 -- start start 2023-05-20T01:16:48.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.287+0000 7f8183fff700 1 -- 172.21.15.16:0/1294154832 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8190003bb0 con 0x7f819c0700c0 2023-05-20T01:16:48.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.288+0000 7f7e49188700 1 Processor -- start 2023-05-20T01:16:48.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.288+0000 7f8183fff700 1 --2- 172.21.15.16:0/1294154832 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f818806ace0 0x7f818806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:48.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.289+0000 7fd600984700 1 Processor -- start 2023-05-20T01:16:48.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.288+0000 7f735f74a700 1 Processor -- start 2023-05-20T01:16:48.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.288+0000 7f735f74a700 1 -- start start 2023-05-20T01:16:48.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.288+0000 7f7e49188700 1 -- start start 2023-05-20T01:16:48.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.289+0000 7f7e49188700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f7e440ce0a0 0x7f7e440ce470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:48.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.289+0000 7f7e49188700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7e440700c0 0x7f7e440b0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:48.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.289+0000 7f7e49188700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e441590f0 con 0x7f7e440ce9b0 2023-05-20T01:16:48.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.289+0000 7f7e49188700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e441593b0 con 0x7f7e440ce0a0 2023-05-20T01:16:48.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.289+0000 7f7e49188700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e44159690 con 0x7f7e440700c0 2023-05-20T01:16:48.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.289+0000 7f8183fff700 1 -- 172.21.15.16:0/1294154832 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f819008acb0 con 0x7f819c0700c0 2023-05-20T01:16:48.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.289+0000 7f8183fff700 1 --2- 172.21.15.16:0/1294154832 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x7f81880748b0 0x7f8188076ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:48.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.289+0000 7f8183fff700 1 -- 172.21.15.16:0/1294154832 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f81880773b0 con 0x7f81880748b0 2023-05-20T01:16:48.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.289+0000 7f819affd700 1 --2- 172.21.15.16:0/1294154832 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f818806ace0 0x7f818806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:48.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.289+0000 7f819bfff700 1 --2- 172.21.15.16:0/1294154832 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x7f81880748b0 0x7f8188076ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:48.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.289+0000 7f26ed39f700 1 Processor -- start 2023-05-20T01:16:48.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.289+0000 7f26ed39f700 1 -- start start 2023-05-20T01:16:48.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.289+0000 7f735f74a700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f735806e170 0x7f73580716d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:48.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.289+0000 7f735f74a700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f73580b1020 0x7f7358158fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:48.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.289+0000 7f735f74a700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f735815d530 con 0x7f735815d090 2023-05-20T01:16:48.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.289+0000 7f735f74a700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73580c9790 con 0x7f73580b1020 2023-05-20T01:16:48.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.289+0000 7f735f74a700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7358159ba0 con 0x7f735806e170 2023-05-20T01:16:48.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.290+0000 7f735dce7700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f73580b1020 0x7f7358158fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:48.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.290+0000 7f735d4e6700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f735806e170 0x7f73580716d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:48.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.290+0000 7f735d4e6700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f735806e170 0x7f73580716d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.16:58126/0 (socket says 172.21.15.16:58126) 2023-05-20T01:16:48.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.290+0000 7f735d4e6700 1 -- 172.21.15.16:0/2748429869 learned_addr learned my addr 172.21.15.16:0/2748429869 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:48.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.289+0000 7fd600984700 1 -- start start 2023-05-20T01:16:48.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.290+0000 7f819affd700 1 --2- 172.21.15.16:0/1294154832 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f818806ace0 0x7f818806d190 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f819c2025d0 tx=0x7f8194005d70 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:48.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.290+0000 7f819bfff700 1 --2- 172.21.15.16:0/1294154832 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x7f81880748b0 0x7f8188076ce0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:48.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.290+0000 7f735d4e6700 1 -- 172.21.15.16:0/2748429869 >> v2:172.21.15.132:3300/0 conn(0x7f73580b1020 msgr2=0x7f7358158fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:48.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.290+0000 7f735d4e6700 1 --2- 172.21.15.16:0/2748429869 >> v2:172.21.15.132:3300/0 conn(0x7f73580b1020 0x7f7358158fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:48.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.290+0000 7f735d4e6700 1 -- 172.21.15.16:0/2748429869 >> v1:172.21.15.16:6789/0 conn(0x7f735815d090 legacy=0x7f73580b0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:48.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.290+0000 7f735d4e6700 1 -- 172.21.15.16:0/2748429869 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7358159e80 con 0x7f735806e170 2023-05-20T01:16:48.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.290+0000 7fd600984700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd5fc0d2760 0x7fd5fc0d3d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:48.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.290+0000 7fd600984700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd5fc0b2010 0x7fd5fc09c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:48.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.290+0000 7fd600984700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fd5fc0b66a0 0x7fd5fc0b6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:48.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.290+0000 7fd600984700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5fc164a70 con 0x7fd5fc0d2760 2023-05-20T01:16:48.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.290+0000 7fd600984700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5fc09cb40 con 0x7fd5fc0b66a0 2023-05-20T01:16:48.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.290+0000 7fd600984700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5fc09bbb0 con 0x7fd5fc0b2010 2023-05-20T01:16:48.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.290+0000 7fd5f9d9b700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd5fc0b2010 0x7fd5fc09c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:48.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.290+0000 7fd5f9d9b700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd5fc0b2010 0x7fd5fc09c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.16:58130/0 (socket says 172.21.15.16:58130) 2023-05-20T01:16:48.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.290+0000 7fd5f9d9b700 1 -- 172.21.15.16:0/1274212391 learned_addr learned my addr 172.21.15.16:0/1274212391 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:48.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.290+0000 7fd5fad9d700 1 --2- 172.21.15.16:0/1274212391 >> v2:172.21.15.132:3300/0 conn(0x7fd5fc0b66a0 0x7fd5fc0b6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:48.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.290+0000 7fd5fa59c700 1 --2- 172.21.15.16:0/1274212391 >> v2:172.21.15.16:3300/0 conn(0x7fd5fc0d2760 0x7fd5fc0d3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:48.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.290+0000 7f7e4259c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7e440700c0 0x7f7e440b0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:48.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.291+0000 7f7e4359e700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f7e440ce9b0 0x7f7e4406bd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.16:51092/0 (socket says 172.21.15.16:51092) 2023-05-20T01:16:48.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.291+0000 7f7e4359e700 1 -- 172.21.15.16:0/772034026 learned_addr learned my addr 172.21.15.16:0/772034026 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:16:48.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.291+0000 7f7e42d9d700 1 --2- 172.21.15.16:0/772034026 >> v2:172.21.15.132:3300/0 conn(0x7f7e440ce0a0 0x7f7e440ce470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:48.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.291+0000 7f26ed39f700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f26e815a910 0x7f26e80cac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:48.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.291+0000 7f26ed39f700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26e815ace0 con 0x7f26e80cb150 2023-05-20T01:16:48.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.291+0000 7f26ed39f700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26e80723b0 con 0x7f26e80b1020 2023-05-20T01:16:48.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.291+0000 7f26ed39f700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26e80cb5c0 con 0x7f26e815a910 2023-05-20T01:16:48.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.291+0000 7f735d4e6700 1 --2- 172.21.15.16:0/2748429869 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f735806e170 0x7f73580716d0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f73540098f0 tx=0x7f7354009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba01d95036bfcec2 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:48.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.291+0000 7fd5f9d9b700 1 -- 172.21.15.16:0/1274212391 >> v2:172.21.15.132:3300/0 conn(0x7fd5fc0b66a0 msgr2=0x7fd5fc0b6ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:48.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.291+0000 7fd5f9d9b700 1 --2- 172.21.15.16:0/1274212391 >> v2:172.21.15.132:3300/0 conn(0x7fd5fc0b66a0 0x7fd5fc0b6ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:48.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.291+0000 7fd5f9d9b700 1 -- 172.21.15.16:0/1274212391 >> v2:172.21.15.16:3300/0 conn(0x7fd5fc0d2760 msgr2=0x7fd5fc0d3d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:48.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.291+0000 7fd5f9d9b700 1 --2- 172.21.15.16:0/1274212391 >> v2:172.21.15.16:3300/0 conn(0x7fd5fc0d2760 0x7fd5fc0d3d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:48.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.291+0000 7fd5f9d9b700 1 -- 172.21.15.16:0/1274212391 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5fc0d2b30 con 0x7fd5fc0b2010 2023-05-20T01:16:48.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.291+0000 7f8183fff700 1 -- 172.21.15.16:0/1294154832 <== mon.2 v2:172.21.15.132:3301/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8190055c20 con 0x7f819c0700c0 2023-05-20T01:16:48.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.291+0000 7f97ecb42700 1 Processor -- start 2023-05-20T01:16:48.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.288+0000 7f292a617700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f292406d1e0 0x7f29240cb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:48.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.288+0000 7f292a617700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f292416dbe0 0x7f29240b0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:48.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.288+0000 7f292a617700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f292406c2c0 con 0x7f29240700c0 2023-05-20T01:16:48.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.288+0000 7f292a617700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f292406c5a0 con 0x7f292416dbe0 2023-05-20T01:16:48.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.288+0000 7f292a617700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f292406c880 con 0x7f292406d1e0 2023-05-20T01:16:48.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.289+0000 7f2928bb4700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f292416dbe0 0x7f29240b0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:48.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.289+0000 7f2928bb4700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f292416dbe0 0x7f29240b0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58074/0 (socket says 172.21.15.16:58074) 2023-05-20T01:16:48.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.289+0000 7f2928bb4700 1 -- 172.21.15.16:0/3702944612 learned_addr learned my addr 172.21.15.16:0/3702944612 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:48.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.289+0000 7f2928bb4700 1 -- 172.21.15.16:0/3702944612 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f292406d1e0 msgr2=0x7f29240cb2c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-05-20T01:16:48.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.289+0000 7f2928bb4700 1 --2- 172.21.15.16:0/3702944612 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f292406d1e0 0x7f29240cb2c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:48.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.291+0000 7f8183fff700 1 -- 172.21.15.16:0/1294154832 <== osd.1 v2:172.21.15.16:6808/3860264220 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f81880773b0 con 0x7f81880748b0 2023-05-20T01:16:48.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.291+0000 7f26e77fe700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f26e80b1020 0x7f26e80c94e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.16:51110/0 (socket says 172.21.15.16:51110) 2023-05-20T01:16:48.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.291+0000 7f26e77fe700 1 -- 172.21.15.16:0/216855643 learned_addr learned my addr 172.21.15.16:0/216855643 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:16:48.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.289+0000 7f2928bb4700 1 -- 172.21.15.16:0/3702944612 >> v1:172.21.15.16:6789/0 conn(0x7f29240700c0 legacy=0x7f292406ba70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-05-20T01:16:48.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.289+0000 7f2928bb4700 1 -- 172.21.15.16:0/3702944612 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f292416d3e0 con 0x7f292416dbe0 2023-05-20T01:16:48.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.290+0000 7f2928bb4700 1 --2- 172.21.15.16:0/3702944612 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f292416dbe0 0x7f29240b0910 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f291c009e50 tx=0x7f291c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45a731e5b0de2ae2 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:48.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.290+0000 7f2922ffd700 1 -- 172.21.15.16:0/3702944612 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f291c0092d0 con 0x7f292416dbe0 2023-05-20T01:16:48.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.290+0000 7f2922ffd700 1 -- 172.21.15.16:0/3702944612 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f291c01f070 con 0x7f292416dbe0 2023-05-20T01:16:48.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.290+0000 7f2922ffd700 1 -- 172.21.15.16:0/3702944612 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f291c003c60 con 0x7f292416dbe0 2023-05-20T01:16:48.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.291+0000 7f292a617700 1 -- 172.21.15.16:0/3702944612 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f292416dbe0 msgr2=0x7f29240b0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:48.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.291+0000 7f292a617700 1 --2- 172.21.15.16:0/3702944612 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f292416dbe0 0x7f29240b0910 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f291c009e50 tx=0x7f291c00b330 comp rx=0 tx=0).stop 2023-05-20T01:16:48.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.291+0000 7f292a617700 1 -- 172.21.15.16:0/3702944612 shutdown_connections 2023-05-20T01:16:48.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.291+0000 7f292a617700 1 --2- 172.21.15.16:0/3702944612 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f292416dbe0 0x7f29240b0910 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:48.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.291+0000 7f292a617700 1 --2- 172.21.15.16:0/3702944612 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f292406d1e0 0x7f29240cb2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:48.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.291+0000 7f292a617700 1 -- 172.21.15.16:0/3702944612 >> 172.21.15.16:0/3702944612 conn(0x7f29240cb910 msgr2=0x7f29240cbd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:48.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.291+0000 7f734ffff700 1 -- 172.21.15.16:0/2748429869 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7354004230 con 0x7f735806e170 2023-05-20T01:16:48.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.291+0000 7f734ffff700 1 -- 172.21.15.16:0/2748429869 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7354004390 con 0x7f735806e170 2023-05-20T01:16:48.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.291+0000 7f734ffff700 1 -- 172.21.15.16:0/2748429869 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73540036a0 con 0x7f735806e170 2023-05-20T01:16:48.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.291+0000 7f97ecb42700 1 -- start start 2023-05-20T01:16:48.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.292+0000 7f7e4259c700 1 -- 172.21.15.16:0/772034026 >> v2:172.21.15.132:3300/0 conn(0x7f7e440ce0a0 msgr2=0x7f7e440ce470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:48.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.292+0000 7f7e4259c700 1 --2- 172.21.15.16:0/772034026 >> v2:172.21.15.132:3300/0 conn(0x7f7e440ce0a0 0x7f7e440ce470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:48.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.292+0000 7f7e4259c700 1 -- 172.21.15.16:0/772034026 >> v1:172.21.15.132:6789/0 conn(0x7f7e440ce9b0 legacy=0x7f7e4406bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:48.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.292+0000 7f7e4259c700 1 -- 172.21.15.16:0/772034026 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e44159970 con 0x7f7e440700c0 2023-05-20T01:16:48.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.292+0000 7f26e5ffb700 1 -- 172.21.15.16:0/216855643 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1297791615 0 0) 0x7f26e80723b0 con 0x7f26e80b1020 2023-05-20T01:16:48.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.292+0000 7f26e5ffb700 1 -- 172.21.15.16:0/216855643 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f26d00035b0 con 0x7f26e80b1020 2023-05-20T01:16:48.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.291+0000 7fd5f9d9b700 1 --2- 172.21.15.16:0/1274212391 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd5fc0b2010 0x7fd5fc09c600 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc08c6c0 tx=0x7fd5f0009bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=129bbb5711e200e3 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:48.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.292+0000 7f26e5ffb700 1 -- 172.21.15.16:0/216855643 <== mon.1 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1477006126 0 0) 0x7f26d00035b0 con 0x7f26e80b1020 2023-05-20T01:16:48.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.292+0000 7f26e5ffb700 1 -- 172.21.15.16:0/216855643 >> v2:172.21.15.16:3300/0 conn(0x7f26e815a910 msgr2=0x7f26e80cac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:48.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.292+0000 7f26e5ffb700 1 --2- 172.21.15.16:0/216855643 >> v2:172.21.15.16:3300/0 conn(0x7f26e815a910 0x7f26e80cac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:48.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.292+0000 7f26e5ffb700 1 -- 172.21.15.16:0/216855643 >> v1:172.21.15.16:6789/0 conn(0x7f26e80cb150 legacy=0x7f26e80b0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:48.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.292+0000 7f26e5ffb700 1 -- 172.21.15.16:0/216855643 --> v1:172.21.15.132:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26e806ba70 con 0x7f26e80b1020 2023-05-20T01:16:48.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.292+0000 7f26e5ffb700 1 -- 172.21.15.16:0/216855643 <== mon.1 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7f26e00019f0 con 0x7f26e80b1020 2023-05-20T01:16:48.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.292+0000 7f26e5ffb700 1 -- 172.21.15.16:0/216855643 >> v1:172.21.15.132:6789/0 conn(0x7f26e80b1020 legacy=0x7f26e80c94e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:48.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.292+0000 7f26e5ffb700 1 --2- 172.21.15.16:0/216855643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26d0003a70 0x7f26d0003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:48.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.292+0000 7f26e5ffb700 1 --2- 172.21.15.16:0/216855643 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f26d0004850 0x7f26d0004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:48.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.292+0000 7f26e5ffb700 1 --2- 172.21.15.16:0/216855643 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f26d0005320 0x7f26d000fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:48.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.292+0000 7f26e5ffb700 1 -- 172.21.15.16:0/216855643 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26e80cb5c0 con 0x7f26d0003a70 2023-05-20T01:16:48.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.292+0000 7f26e5ffb700 1 -- 172.21.15.16:0/216855643 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26d0010660 con 0x7f26d0004850 2023-05-20T01:16:48.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.292+0000 7f26e5ffb700 1 -- 172.21.15.16:0/216855643 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26d0010820 con 0x7f26d0005320 2023-05-20T01:16:48.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.292+0000 7f7e4259c700 1 --2- 172.21.15.16:0/772034026 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7e440700c0 0x7f7e440b0910 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f7e38009840 tx=0x7f7e38009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4200480d391c42f server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:48.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.292+0000 7f735f74a700 1 -- 172.21.15.16:0/2748429869 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f735806e170 msgr2=0x7f73580716d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:48.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.292+0000 7f735f74a700 1 --2- 172.21.15.16:0/2748429869 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f735806e170 0x7f73580716d0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f73540098f0 tx=0x7f7354009c00 comp rx=0 tx=0).stop 2023-05-20T01:16:48.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.292+0000 7fd5f959a700 1 -- 172.21.15.16:0/1274212391 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f0011830 con 0x7fd5fc0b2010 2023-05-20T01:16:48.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.292+0000 7fd5f959a700 1 -- 172.21.15.16:0/1274212391 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5f0011990 con 0x7fd5fc0b2010 2023-05-20T01:16:48.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.292+0000 7fd5f959a700 1 -- 172.21.15.16:0/1274212391 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f0011d20 con 0x7fd5fc0b2010 2023-05-20T01:16:48.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.293+0000 7f7e41d9b700 1 -- 172.21.15.16:0/772034026 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e38004230 con 0x7f7e440700c0 2023-05-20T01:16:48.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.293+0000 7f7e41d9b700 1 -- 172.21.15.16:0/772034026 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e38004390 con 0x7f7e440700c0 2023-05-20T01:16:48.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.293+0000 7f7e41d9b700 1 -- 172.21.15.16:0/772034026 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e380036a0 con 0x7f7e440700c0 2023-05-20T01:16:48.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.293+0000 7f26e6ffd700 1 --2- 172.21.15.16:0/216855643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26d0003a70 0x7f26d0003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:48.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.293+0000 7fd600984700 1 -- 172.21.15.16:0/1274212391 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd5fc0b2010 msgr2=0x7fd5fc09c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:48.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.293+0000 7fd600984700 1 --2- 172.21.15.16:0/1274212391 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd5fc0b2010 0x7fd5fc09c600 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc08c6c0 tx=0x7fd5f0009bd0 comp rx=0 tx=0).stop 2023-05-20T01:16:48.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.292+0000 7f735f74a700 1 -- 172.21.15.16:0/2748429869 shutdown_connections 2023-05-20T01:16:48.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.292+0000 7f735f74a700 1 --2- 172.21.15.16:0/2748429869 >> v2:172.21.15.132:3300/0 conn(0x7f73580b1020 0x7f7358158fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:48.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.292+0000 7f735f74a700 1 --2- 172.21.15.16:0/2748429869 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f735806e170 0x7f73580716d0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:48.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.293+0000 7f735f74a700 1 -- 172.21.15.16:0/2748429869 >> 172.21.15.16:0/2748429869 conn(0x7f73580cb910 msgr2=0x7f73580cbd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:48.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.293+0000 7f735f74a700 1 -- 172.21.15.16:0/2748429869 shutdown_connections 2023-05-20T01:16:48.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.291+0000 7f292a617700 1 -- 172.21.15.16:0/3702944612 shutdown_connections 2023-05-20T01:16:48.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.292+0000 7f292a617700 1 -- 172.21.15.16:0/3702944612 wait complete. 2023-05-20T01:16:48.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.293+0000 7f292a617700 1 Processor -- start 2023-05-20T01:16:48.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.293+0000 7f97ecb42700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f97e8160e70 0x7f97e8164b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:48.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.293+0000 7f97ecb42700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f97e8161240 0x7f97e80cadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:48.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.293+0000 7f97ecb42700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f97e80c2670 0x7f97e80ce4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:48.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.293+0000 7f97ecb42700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97e8165b00 con 0x7f97e80c2670 2023-05-20T01:16:48.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.293+0000 7f97ecb42700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97e80c6250 con 0x7f97e8160e70 2023-05-20T01:16:48.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.293+0000 7f97ecb42700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97e8165260 con 0x7f97e8161240 2023-05-20T01:16:48.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.294+0000 7f26e6ffd700 1 -- 172.21.15.16:0/216855643 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f26d0005320 msgr2=0x7f26d000fe20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-05-20T01:16:48.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.294+0000 7f26e6ffd700 1 --2- 172.21.15.16:0/216855643 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f26d0005320 0x7f26d000fe20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:48.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.294+0000 7f26e6ffd700 1 -- 172.21.15.16:0/216855643 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f26d0004850 msgr2=0x7f26d0004cc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-05-20T01:16:48.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.294+0000 7f26e6ffd700 1 --2- 172.21.15.16:0/216855643 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f26d0004850 0x7f26d0004cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:48.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.294+0000 7f26e6ffd700 1 -- 172.21.15.16:0/216855643 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26e806ba70 con 0x7f26d0003a70 2023-05-20T01:16:48.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.294+0000 7f26e77fe700 1 -- 172.21.15.16:0/216855643 reap_dead start 2023-05-20T01:16:48.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.294+0000 7f7e49188700 1 -- 172.21.15.16:0/772034026 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7e440700c0 msgr2=0x7f7e440b0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:48.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.294+0000 7f7e49188700 1 --2- 172.21.15.16:0/772034026 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7e440700c0 0x7f7e440b0910 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f7e38009840 tx=0x7f7e38009b50 comp rx=0 tx=0).stop 2023-05-20T01:16:48.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.294+0000 7f97e5d9b700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f97e8161240 0x7f97e80cadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:48.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.294+0000 7f97e5d9b700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f97e8161240 0x7f97e80cadb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.16:58146/0 (socket says 172.21.15.16:58146) 2023-05-20T01:16:48.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.294+0000 7f97e5d9b700 1 -- 172.21.15.16:0/3529444794 learned_addr learned my addr 172.21.15.16:0/3529444794 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:48.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.294+0000 7f26e6ffd700 1 --2- 172.21.15.16:0/216855643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26d0003a70 0x7f26d0003e60 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f26d8009910 tx=0x7f26d0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a73a83354825cc4 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:48.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.294+0000 7f7e49188700 1 -- 172.21.15.16:0/772034026 shutdown_connections 2023-05-20T01:16:48.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.294+0000 7f7e49188700 1 --2- 172.21.15.16:0/772034026 >> v2:172.21.15.132:3300/0 conn(0x7f7e440ce0a0 0x7f7e440ce470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:48.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.294+0000 7f7e49188700 1 --2- 172.21.15.16:0/772034026 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7e440700c0 0x7f7e440b0910 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:48.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.294+0000 7f7e49188700 1 -- 172.21.15.16:0/772034026 >> 172.21.15.16:0/772034026 conn(0x7f7e440cb910 msgr2=0x7f7e440cbd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:48.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.294+0000 7f7e49188700 1 -- 172.21.15.16:0/772034026 shutdown_connections 2023-05-20T01:16:48.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.294+0000 7f26e5ffb700 1 -- 172.21.15.16:0/216855643 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d8004230 con 0x7f26d0003a70 2023-05-20T01:16:48.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.294+0000 7f26e5ffb700 1 -- 172.21.15.16:0/216855643 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26d80043b0 con 0x7f26d0003a70 2023-05-20T01:16:48.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.294+0000 7f26e5ffb700 1 -- 172.21.15.16:0/216855643 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d80038e0 con 0x7f26d0003a70 2023-05-20T01:16:48.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.294+0000 7f97e659c700 1 --2- 172.21.15.16:0/3529444794 >> v2:172.21.15.132:3300/0 conn(0x7f97e8160e70 0x7f97e8164b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:48.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.294+0000 7f97e5d9b700 1 -- 172.21.15.16:0/3529444794 >> v2:172.21.15.132:3300/0 conn(0x7f97e8160e70 msgr2=0x7f97e8164b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:48.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.295+0000 7f97e5d9b700 1 --2- 172.21.15.16:0/3529444794 >> v2:172.21.15.132:3300/0 conn(0x7f97e8160e70 0x7f97e8164b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:48.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.295+0000 7f97e5d9b700 1 -- 172.21.15.16:0/3529444794 >> v2:172.21.15.16:3300/0 conn(0x7f97e80c2670 msgr2=0x7f97e80ce4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:48.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.295+0000 7f97e5d9b700 1 --2- 172.21.15.16:0/3529444794 >> v2:172.21.15.16:3300/0 conn(0x7f97e80c2670 0x7f97e80ce4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:48.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.295+0000 7f97e5d9b700 1 -- 172.21.15.16:0/3529444794 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97e8165540 con 0x7f97e8161240 2023-05-20T01:16:48.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.293+0000 7f735f74a700 1 -- 172.21.15.16:0/2748429869 wait complete. 2023-05-20T01:16:48.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.295+0000 7f735f74a700 1 Processor -- start 2023-05-20T01:16:48.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.295+0000 7fd600984700 1 -- 172.21.15.16:0/1274212391 shutdown_connections 2023-05-20T01:16:48.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.295+0000 7fd600984700 1 --2- 172.21.15.16:0/1274212391 >> v2:172.21.15.132:3300/0 conn(0x7fd5fc0b66a0 0x7fd5fc0b6ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:48.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.295+0000 7fd600984700 1 --2- 172.21.15.16:0/1274212391 >> v2:172.21.15.16:3300/0 conn(0x7fd5fc0d2760 0x7fd5fc0d3d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:48.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.295+0000 7fd600984700 1 --2- 172.21.15.16:0/1274212391 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd5fc0b2010 0x7fd5fc09c600 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:48.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.295+0000 7fd600984700 1 -- 172.21.15.16:0/1274212391 >> 172.21.15.16:0/1274212391 conn(0x7fd5fc0d0950 msgr2=0x7fd5fc0d3240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:48.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.295+0000 7f97e5d9b700 1 --2- 172.21.15.16:0/3529444794 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f97e8161240 0x7f97e80cadb0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f97e80b1c90 tx=0x7f97dc009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c29e06ef9f3de7d9 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:48.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.294+0000 7f7e49188700 1 -- 172.21.15.16:0/772034026 wait complete. 2023-05-20T01:16:48.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.296+0000 7f7e49188700 1 Processor -- start 2023-05-20T01:16:48.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.295+0000 7f26ed39f700 1 -- 172.21.15.16:0/216855643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26d0003a70 msgr2=0x7f26d0003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:48.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.295+0000 7f26ed39f700 1 --2- 172.21.15.16:0/216855643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26d0003a70 0x7f26d0003e60 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f26d8009910 tx=0x7f26d0005880 comp rx=0 tx=0).stop 2023-05-20T01:16:48.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.295+0000 7f26ed39f700 1 -- 172.21.15.16:0/216855643 shutdown_connections 2023-05-20T01:16:48.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.295+0000 7f26ed39f700 1 --2- 172.21.15.16:0/216855643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26d0003a70 0x7f26d0003e60 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:48.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.295+0000 7f26ed39f700 1 -- 172.21.15.16:0/216855643 >> 172.21.15.16:0/216855643 conn(0x7f26e80cb910 msgr2=0x7f26e80cbd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:48.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.296+0000 7f26ed39f700 1 -- 172.21.15.16:0/216855643 shutdown_connections 2023-05-20T01:16:48.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.296+0000 7f26ed39f700 1 -- 172.21.15.16:0/216855643 wait complete. 2023-05-20T01:16:48.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.295+0000 7fd600984700 1 -- 172.21.15.16:0/1274212391 shutdown_connections 2023-05-20T01:16:48.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.295+0000 7fd600984700 1 -- 172.21.15.16:0/1274212391 wait complete. 2023-05-20T01:16:48.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.297+0000 7fd600984700 1 Processor -- start 2023-05-20T01:16:48.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.294+0000 7f292a617700 1 -- start start 2023-05-20T01:16:48.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.295+0000 7f292a617700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f292416dbe0 0x7f29240c76a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:48.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.295+0000 7f292a617700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2924089990 0x7f2924079fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:48.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.295+0000 7f292a617700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2924165810 0x7f2924165c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:48.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.295+0000 7f292a617700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2924164520 con 0x7f2924165810 2023-05-20T01:16:48.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.295+0000 7f292a617700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2924164770 con 0x7f292416dbe0 2023-05-20T01:16:48.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.295+0000 7f292a617700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2924164a50 con 0x7f2924089990 2023-05-20T01:16:48.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.296+0000 7f2928bb4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2924165810 0x7f2924165c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:48.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.296+0000 7f2928bb4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2924165810 0x7f2924165c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58114/0 (socket says 172.21.15.16:58114) 2023-05-20T01:16:48.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.296+0000 7f2928bb4700 1 -- 172.21.15.16:0/759793321 learned_addr learned my addr 172.21.15.16:0/759793321 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:48.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.296+0000 7f2928bb4700 1 -- 172.21.15.16:0/759793321 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2924089990 msgr2=0x7f2924079fa0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-20T01:16:48.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.296+0000 7f2928bb4700 1 --2- 172.21.15.16:0/759793321 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2924089990 0x7f2924079fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:48.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.296+0000 7f2928bb4700 1 -- 172.21.15.16:0/759793321 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f292416dbe0 msgr2=0x7f29240c76a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-20T01:16:48.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.296+0000 7f2928bb4700 1 --2- 172.21.15.16:0/759793321 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f292416dbe0 0x7f29240c76a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:48.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.296+0000 7f2928bb4700 1 -- 172.21.15.16:0/759793321 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f291c009b30 con 0x7f2924165810 2023-05-20T01:16:48.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.296+0000 7f2928bb4700 1 --2- 172.21.15.16:0/759793321 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2924165810 0x7f2924165c80 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f292406ebf0 tx=0x7f291c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:48.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.295+0000 7f97e559a700 1 -- 172.21.15.16:0/3529444794 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97dc00b860 con 0x7f97e8161240 2023-05-20T01:16:48.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.295+0000 7f97e559a700 1 -- 172.21.15.16:0/3529444794 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97dc00b9c0 con 0x7f97e8161240 2023-05-20T01:16:48.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.295+0000 7f97e559a700 1 -- 172.21.15.16:0/3529444794 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97dc00bd50 con 0x7f97e8161240 2023-05-20T01:16:48.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.296+0000 7f97ecb42700 1 -- 172.21.15.16:0/3529444794 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f97e8161240 msgr2=0x7f97e80cadb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:48.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.296+0000 7f97ecb42700 1 --2- 172.21.15.16:0/3529444794 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f97e8161240 0x7f97e80cadb0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f97e80b1c90 tx=0x7f97dc009c00 comp rx=0 tx=0).stop 2023-05-20T01:16:48.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.296+0000 7f97ecb42700 1 -- 172.21.15.16:0/3529444794 shutdown_connections 2023-05-20T01:16:48.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.296+0000 7f97ecb42700 1 --2- 172.21.15.16:0/3529444794 >> v2:172.21.15.16:3300/0 conn(0x7f97e80c2670 0x7f97e80ce4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:48.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.296+0000 7f97ecb42700 1 --2- 172.21.15.16:0/3529444794 >> v2:172.21.15.132:3300/0 conn(0x7f97e8160e70 0x7f97e8164b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:48.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.296+0000 7f97ecb42700 1 --2- 172.21.15.16:0/3529444794 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f97e8161240 0x7f97e80cadb0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:48.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.296+0000 7f97ecb42700 1 -- 172.21.15.16:0/3529444794 >> 172.21.15.16:0/3529444794 conn(0x7f97e80b0e90 msgr2=0x7f97e80b1260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:48.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.297+0000 7f97ecb42700 1 -- 172.21.15.16:0/3529444794 shutdown_connections 2023-05-20T01:16:48.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.297+0000 7f7e49188700 1 -- start start 2023-05-20T01:16:48.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.298+0000 7f7e49188700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e440700c0 0x7f7e440c5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:48.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.298+0000 7f7e49188700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7e440ce0a0 0x7f7e440b45b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:48.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.298+0000 7f7e49188700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7e440ce9b0 0x7f7e440b0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:48.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.298+0000 7f7e49188700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e440b2800 con 0x7f7e440700c0 2023-05-20T01:16:48.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.298+0000 7f7e49188700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e440b2ae0 con 0x7f7e440ce0a0 2023-05-20T01:16:48.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.298+0000 7f7e49188700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e440b2dc0 con 0x7f7e440ce9b0 2023-05-20T01:16:48.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.298+0000 7f7e42d9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e440700c0 0x7f7e440c5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:48.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.298+0000 7f7e42d9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e440700c0 0x7f7e440c5970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58116/0 (socket says 172.21.15.16:58116) 2023-05-20T01:16:48.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.298+0000 7f7e42d9d700 1 -- 172.21.15.16:0/297189136 learned_addr learned my addr 172.21.15.16:0/297189136 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:48.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.299+0000 7f7e4359e700 1 --2- 172.21.15.16:0/297189136 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7e440ce9b0 0x7f7e440b0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:48.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.297+0000 7f97ecb42700 1 -- 172.21.15.16:0/3529444794 wait complete. 2023-05-20T01:16:48.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.299+0000 7f97ecb42700 1 Processor -- start 2023-05-20T01:16:48.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.298+0000 7f26ed39f700 1 Processor -- start 2023-05-20T01:16:48.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.299+0000 7f26ed39f700 1 -- start start 2023-05-20T01:16:48.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.297+0000 7f2920ff9700 1 -- 172.21.15.16:0/759793321 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f291c004230 con 0x7f2924165810 2023-05-20T01:16:48.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.297+0000 7f2920ff9700 1 -- 172.21.15.16:0/759793321 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f291c004390 con 0x7f2924165810 2023-05-20T01:16:48.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.297+0000 7f2920ff9700 1 -- 172.21.15.16:0/759793321 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f291c004740 con 0x7f2924165810 2023-05-20T01:16:48.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.297+0000 7f292a617700 1 -- 172.21.15.16:0/759793321 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f292416e020 con 0x7f2924165810 2023-05-20T01:16:48.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.297+0000 7f292a617700 1 -- 172.21.15.16:0/759793321 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f292416d6c0 con 0x7f2924165810 2023-05-20T01:16:48.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.298+0000 7fd600984700 1 -- start start 2023-05-20T01:16:48.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.299+0000 7fd600984700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd5fc0b2010 0x7fd5fc1ea910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:48.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.299+0000 7fd600984700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5fc0b66a0 0x7fd5fc1eae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:48.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.299+0000 7fd600984700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd5fc0d2760 0x7fd5fc0cc850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:48.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.299+0000 7fd600984700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5fc0cba90 con 0x7fd5fc0b66a0 2023-05-20T01:16:48.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.299+0000 7fd600984700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5fc0cbd70 con 0x7fd5fc0b2010 2023-05-20T01:16:48.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.299+0000 7fd600984700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5fc0cc050 con 0x7fd5fc0d2760 2023-05-20T01:16:48.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.299+0000 7fd5fa59c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd5fc0b2010 0x7fd5fc1ea910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:48.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.299+0000 7fd5f9d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5fc0b66a0 0x7fd5fc1eae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:48.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.299+0000 7fd5f9d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5fc0b66a0 0x7fd5fc1eae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58122/0 (socket says 172.21.15.16:58122) 2023-05-20T01:16:48.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.299+0000 7fd5f9d9b700 1 -- 172.21.15.16:0/1959318617 learned_addr learned my addr 172.21.15.16:0/1959318617 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:48.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.300+0000 7f26ed39f700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f26d0003a70 0x7f26e80c33d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:48.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.300+0000 7f26ed39f700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f26e815d540 0x7f26e80c5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:48.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.300+0000 7f26ed39f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26e80ccb60 0x7f26e814e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:48.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.300+0000 7f26ed39f700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26e80c3c90 con 0x7f26e80ccb60 2023-05-20T01:16:48.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.300+0000 7f26ed39f700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26e815d0b0 con 0x7f26e815d540 2023-05-20T01:16:48.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.300+0000 7f26ed39f700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26e80c39b0 con 0x7f26d0003a70 2023-05-20T01:16:48.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.299+0000 7fd5fad9d700 1 --2- 172.21.15.16:0/1959318617 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd5fc0d2760 0x7fd5fc0cc850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:48.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.300+0000 7fd5f9d9b700 1 -- 172.21.15.16:0/1959318617 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd5fc0d2760 msgr2=0x7fd5fc0cc850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:48.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.300+0000 7fd5f9d9b700 1 --2- 172.21.15.16:0/1959318617 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd5fc0d2760 0x7fd5fc0cc850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:48.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.300+0000 7fd5f9d9b700 1 -- 172.21.15.16:0/1959318617 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd5fc0b2010 msgr2=0x7fd5fc1ea910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:48.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.300+0000 7fd5f9d9b700 1 --2- 172.21.15.16:0/1959318617 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd5fc0b2010 0x7fd5fc1ea910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:48.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.300+0000 7fd5f9d9b700 1 -- 172.21.15.16:0/1959318617 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5f0009550 con 0x7fd5fc0b66a0 2023-05-20T01:16:48.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.300+0000 7fd5f9d9b700 1 --2- 172.21.15.16:0/1959318617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5fc0b66a0 0x7fd5fc1eae50 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd5f0000c00 tx=0x7fd5f0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:48.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.299+0000 7f97ecb42700 1 -- start start 2023-05-20T01:16:48.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.300+0000 7f97ecb42700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f97e80c2670 0x7f97e81ed030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:48.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.300+0000 7f97ecb42700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f97e8160e70 0x7f97e81573d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:48.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.300+0000 7f97ecb42700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97e8161240 0x7f97e815d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:48.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.300+0000 7f97ecb42700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97e8157910 con 0x7f97e8161240 2023-05-20T01:16:48.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.300+0000 7f97ecb42700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97e815d740 con 0x7f97e8160e70 2023-05-20T01:16:48.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.300+0000 7f97ecb42700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97e815d9e0 con 0x7f97e80c2670 2023-05-20T01:16:48.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.300+0000 7f97e5d9b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f97e8160e70 0x7f97e81573d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:48.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.300+0000 7f97e5d9b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f97e8160e70 0x7f97e81573d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.16:55154/0 (socket says 172.21.15.16:55154) 2023-05-20T01:16:48.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.300+0000 7f97e5d9b700 1 -- 172.21.15.16:0/1016053349 learned_addr learned my addr 172.21.15.16:0/1016053349 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:48.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.300+0000 7f26e77fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26e80ccb60 0x7f26e814e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:48.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.300+0000 7f26e77fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26e80ccb60 0x7f26e814e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58130/0 (socket says 172.21.15.16:58130) 2023-05-20T01:16:48.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.300+0000 7f26e77fe700 1 -- 172.21.15.16:0/1959311532 learned_addr learned my addr 172.21.15.16:0/1959311532 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:48.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.300+0000 7f26e6ffd700 1 --2- 172.21.15.16:0/1959311532 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f26d0003a70 0x7f26e80c33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:48.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.300+0000 7f26e67fc700 1 --2- 172.21.15.16:0/1959311532 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f26e815d540 0x7f26e80c5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:48.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.300+0000 7f26e77fe700 1 -- 172.21.15.16:0/1959311532 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f26d0003a70 msgr2=0x7f26e80c33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:48.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.300+0000 7f26e77fe700 1 --2- 172.21.15.16:0/1959311532 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f26d0003a70 0x7f26e80c33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:48.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.300+0000 7f26e77fe700 1 -- 172.21.15.16:0/1959311532 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f26e815d540 msgr2=0x7f26e80c5290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:48.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.300+0000 7f26e77fe700 1 --2- 172.21.15.16:0/1959311532 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f26e815d540 0x7f26e80c5290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:48.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.300+0000 7f26e77fe700 1 -- 172.21.15.16:0/1959311532 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26e806c030 con 0x7f26e80ccb60 2023-05-20T01:16:48.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.300+0000 7f97e6d9d700 1 --2- 172.21.15.16:0/1016053349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97e8161240 0x7f97e815d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:48.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.301+0000 7f97e659c700 1 --2- 172.21.15.16:0/1016053349 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f97e80c2670 0x7f97e81ed030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:48.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.301+0000 7f26e77fe700 1 --2- 172.21.15.16:0/1959311532 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26e80ccb60 0x7f26e814e020 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f26e0001330 tx=0x7f26e000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:48.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.301+0000 7f7e4359e700 1 -- 172.21.15.16:0/297189136 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7e440ce0a0 msgr2=0x7f7e440b45b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-20T01:16:48.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.301+0000 7f7e4359e700 1 --2- 172.21.15.16:0/297189136 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7e440ce0a0 0x7f7e440b45b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:48.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.301+0000 7f7e4359e700 1 -- 172.21.15.16:0/297189136 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e440700c0 msgr2=0x7f7e440c5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:48.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.301+0000 7f7e4359e700 1 --2- 172.21.15.16:0/297189136 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e440700c0 0x7f7e440c5970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:48.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.301+0000 7f7e4359e700 1 -- 172.21.15.16:0/297189136 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e380094a0 con 0x7f7e440ce9b0 2023-05-20T01:16:48.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.300+0000 7f735f74a700 1 -- start start 2023-05-20T01:16:48.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.301+0000 7f735f74a700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f735806e170 0x7f73580c4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:48.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.301+0000 7f735f74a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73580b1020 0x7f73580c22f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:48.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.301+0000 7f735f74a700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f735815d090 0x7f73580ceae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:48.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.301+0000 7f735f74a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73580c4460 con 0x7f73580b1020 2023-05-20T01:16:48.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.301+0000 7f735f74a700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73580c2830 con 0x7f735806e170 2023-05-20T01:16:48.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.301+0000 7f735f74a700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73581406e0 con 0x7f735815d090 2023-05-20T01:16:48.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.301+0000 7f97e5d9b700 1 -- 172.21.15.16:0/1016053349 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f97e80c2670 msgr2=0x7f97e81ed030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:48.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.301+0000 7f97e5d9b700 1 --2- 172.21.15.16:0/1016053349 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f97e80c2670 0x7f97e81ed030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:48.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.301+0000 7f97e5d9b700 1 -- 172.21.15.16:0/1016053349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97e8161240 msgr2=0x7f97e815d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:48.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.301+0000 7f97e5d9b700 1 --2- 172.21.15.16:0/1016053349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97e8161240 0x7f97e815d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:48.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.301+0000 7f97e5d9b700 1 -- 172.21.15.16:0/1016053349 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97dc009550 con 0x7f97e8160e70 2023-05-20T01:16:48.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.301+0000 7f735dce7700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f735815d090 0x7f73580ceae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:48.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.301+0000 7f735dce7700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f735815d090 0x7f73580ceae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.16:58176/0 (socket says 172.21.15.16:58176) 2023-05-20T01:16:48.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.301+0000 7f735dce7700 1 -- 172.21.15.16:0/873941470 learned_addr learned my addr 172.21.15.16:0/873941470 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:48.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.301+0000 7f97e5d9b700 1 --2- 172.21.15.16:0/1016053349 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f97e8160e70 0x7f97e81573d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f97dc009870 tx=0x7f97dc004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:48.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.302+0000 7f2920ff9700 1 -- 172.21.15.16:0/759793321 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f291c023440 con 0x7f2924165810 2023-05-20T01:16:48.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.302+0000 7f7e4359e700 1 --2- 172.21.15.16:0/297189136 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7e440ce9b0 0x7f7e440b0790 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f7e3400c480 tx=0x7f7e3400c790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:48.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.302+0000 7f26cf7fe700 1 -- 172.21.15.16:0/1959311532 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26e0002100 con 0x7f26e80ccb60 2023-05-20T01:16:48.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.302+0000 7f26cf7fe700 1 -- 172.21.15.16:0/1959311532 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26e000acc0 con 0x7f26e80ccb60 2023-05-20T01:16:48.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.302+0000 7f26cf7fe700 1 -- 172.21.15.16:0/1959311532 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26e00070d0 con 0x7f26e80ccb60 2023-05-20T01:16:48.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.302+0000 7f26ed39f700 1 -- 172.21.15.16:0/1959311532 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26e806bd50 con 0x7f26e80ccb60 2023-05-20T01:16:48.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.302+0000 7f26ed39f700 1 -- 172.21.15.16:0/1959311532 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26e806c310 con 0x7f26e80ccb60 2023-05-20T01:16:48.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.301+0000 7fd5eaffd700 1 -- 172.21.15.16:0/1959318617 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f0004450 con 0x7fd5fc0b66a0 2023-05-20T01:16:48.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.301+0000 7fd5eaffd700 1 -- 172.21.15.16:0/1959318617 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5f001f070 con 0x7fd5fc0b66a0 2023-05-20T01:16:48.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.301+0000 7fd5eaffd700 1 -- 172.21.15.16:0/1959318617 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f00038e0 con 0x7fd5fc0b66a0 2023-05-20T01:16:48.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.301+0000 7fd5fad9d700 1 --2- 172.21.15.16:0/1959318617 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd5fc0d2760 0x7fd5fc0cc850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:16:48.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.302+0000 7f735d4e6700 1 --2- 172.21.15.16:0/873941470 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f735806e170 0x7f73580c4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:48.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.302+0000 7f735cce5700 1 --2- 172.21.15.16:0/873941470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73580b1020 0x7f73580c22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:48.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.302+0000 7f2920ff9700 1 --2- 172.21.15.16:0/759793321 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f290806ad00 0x7f290806d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:48.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.302+0000 7f7e2b7fe700 1 -- 172.21.15.16:0/297189136 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e34004c80 con 0x7f7e440ce9b0 2023-05-20T01:16:48.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.302+0000 7f7e2b7fe700 1 -- 172.21.15.16:0/297189136 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e34004e00 con 0x7f7e440ce9b0 2023-05-20T01:16:48.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.302+0000 7f7e2b7fe700 1 -- 172.21.15.16:0/297189136 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e34007310 con 0x7f7e440ce9b0 2023-05-20T01:16:48.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.302+0000 7f2920ff9700 1 -- 172.21.15.16:0/759793321 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f291c09c180 con 0x7f2924165810 2023-05-20T01:16:48.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.302+0000 7f7e49188700 1 -- 172.21.15.16:0/297189136 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e44159f30 con 0x7f7e440ce9b0 2023-05-20T01:16:48.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.303+0000 7f7e49188700 1 -- 172.21.15.16:0/297189136 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e44159c50 con 0x7f7e440ce9b0 2023-05-20T01:16:48.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.302+0000 7f735d4e6700 1 -- 172.21.15.16:0/873941470 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f735815d090 msgr2=0x7f73580ceae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:48.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.302+0000 7f735d4e6700 1 --2- 172.21.15.16:0/873941470 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f735815d090 0x7f73580ceae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:48.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.302+0000 7f735d4e6700 1 -- 172.21.15.16:0/873941470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73580b1020 msgr2=0x7f73580c22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:48.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.302+0000 7f735d4e6700 1 --2- 172.21.15.16:0/873941470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73580b1020 0x7f73580c22f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:48.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.302+0000 7f735d4e6700 1 -- 172.21.15.16:0/873941470 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7354009550 con 0x7f735806e170 2023-05-20T01:16:48.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.306+0000 7f7e2b7fe700 1 -- 172.21.15.16:0/297189136 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7e34007470 con 0x7f7e440ce9b0 2023-05-20T01:16:48.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.304+0000 7f26ed39f700 1 -- 172.21.15.16:0/1959311532 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f26c8000fb0 con 0x7f26e80ccb60 2023-05-20T01:16:48.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.306+0000 7f26cf7fe700 1 -- 172.21.15.16:0/1959311532 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f26e0007230 con 0x7f26e80ccb60 2023-05-20T01:16:48.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.302+0000 7fd600984700 1 -- 172.21.15.16:0/1959318617 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5fc06cd60 con 0x7fd5fc0b66a0 2023-05-20T01:16:48.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.302+0000 7fd600984700 1 -- 172.21.15.16:0/1959318617 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5fc06ca80 con 0x7fd5fc0b66a0 2023-05-20T01:16:48.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.303+0000 7fd600984700 1 -- 172.21.15.16:0/1959318617 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd5fc09be00 con 0x7fd5fc0b66a0 2023-05-20T01:16:48.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.303+0000 7f735d4e6700 1 --2- 172.21.15.16:0/873941470 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f735806e170 0x7f73580c4be0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7354015000 tx=0x7f7354004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:48.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.303+0000 7f734dffb700 1 -- 172.21.15.16:0/873941470 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7354003ce0 con 0x7f735806e170 2023-05-20T01:16:48.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.303+0000 7f734dffb700 1 -- 172.21.15.16:0/873941470 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7354003e40 con 0x7f735806e170 2023-05-20T01:16:48.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.303+0000 7f734dffb700 1 -- 172.21.15.16:0/873941470 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7354011850 con 0x7f735806e170 2023-05-20T01:16:48.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.303+0000 7f735f74a700 1 -- 172.21.15.16:0/873941470 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f735815c890 con 0x7f735806e170 2023-05-20T01:16:48.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.304+0000 7f735f74a700 1 -- 172.21.15.16:0/873941470 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f735815c5b0 con 0x7f735806e170 2023-05-20T01:16:48.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.303+0000 7f292a617700 1 --2- 172.21.15.16:0/759793321 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x7f29000015d0 0x7f2900003a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:48.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.303+0000 7f292a617700 1 -- 172.21.15.16:0/759793321 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2900006bb0 con 0x7f29000015d0 2023-05-20T01:16:48.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.303+0000 7f29237fe700 1 --2- 172.21.15.16:0/759793321 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x7f29000015d0 0x7f2900003a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:48.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.304+0000 7f2923fff700 1 --2- 172.21.15.16:0/759793321 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f290806ad00 0x7f290806d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:48.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.303+0000 7f97d6ffd700 1 -- 172.21.15.16:0/1016053349 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97dc004450 con 0x7f97e8160e70 2023-05-20T01:16:48.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.303+0000 7f97d6ffd700 1 -- 172.21.15.16:0/1016053349 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97dc0036a0 con 0x7f97e8160e70 2023-05-20T01:16:48.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.303+0000 7f97d6ffd700 1 -- 172.21.15.16:0/1016053349 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97dc0045f0 con 0x7f97e8160e70 2023-05-20T01:16:48.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.303+0000 7f97ecb42700 1 -- 172.21.15.16:0/1016053349 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97e80cd070 con 0x7f97e8160e70 2023-05-20T01:16:48.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.303+0000 7f97ecb42700 1 -- 172.21.15.16:0/1016053349 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97e8165820 con 0x7f97e8160e70 2023-05-20T01:16:48.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.306+0000 7f97d6ffd700 1 -- 172.21.15.16:0/1016053349 <== mon.1 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f97dc003a50 con 0x7f97e8160e70 2023-05-20T01:16:48.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.306+0000 7fd5eaffd700 1 -- 172.21.15.16:0/1959318617 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd5f0003a40 con 0x7fd5fc0b66a0 2023-05-20T01:16:48.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.306+0000 7fd5eaffd700 1 --2- 172.21.15.16:0/1959318617 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd5dc06ad30 0x7fd5dc06d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:48.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.307+0000 7fd5eaffd700 1 -- 172.21.15.16:0/1959318617 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fd5f0021070 con 0x7fd5fc0b66a0 2023-05-20T01:16:48.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.307+0000 7f7e49188700 1 -- 172.21.15.16:0/297189136 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7e440c96a0 con 0x7f7e440ce9b0 2023-05-20T01:16:48.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.307+0000 7f7e2b7fe700 1 --2- 172.21.15.16:0/297189136 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f7e3006ad30 0x7f7e3006d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:48.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.306+0000 7f97d6ffd700 1 --2- 172.21.15.16:0/1016053349 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f97c806ad30 0x7f97c806d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:48.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.306+0000 7f97d6ffd700 1 -- 172.21.15.16:0/1016053349 <== mon.1 v2:172.21.15.132:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f97dc08aa30 con 0x7f97e8160e70 2023-05-20T01:16:48.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.307+0000 7fd5eaffd700 1 --2- 172.21.15.16:0/1959318617 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x7fd5dc074a00 0x7fd5dc076e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:48.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.307+0000 7fd5eaffd700 1 -- 172.21.15.16:0/1959318617 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd5dc077500 con 0x7fd5dc074a00 2023-05-20T01:16:48.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.307+0000 7fd5eaffd700 1 -- 172.21.15.16:0/1959318617 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd5f008b4b0 con 0x7fd5fc0b66a0 2023-05-20T01:16:48.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.307+0000 7f29237fe700 1 --2- 172.21.15.16:0/759793321 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x7f29000015d0 0x7f2900003a80 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:48.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.307+0000 7f7e2b7fe700 1 -- 172.21.15.16:0/297189136 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f7e340891d0 con 0x7f7e440ce9b0 2023-05-20T01:16:48.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.307+0000 7f7e2b7fe700 1 --2- 172.21.15.16:0/297189136 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x7f7e30074980 0x7f7e30076db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:48.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.307+0000 7f7e2b7fe700 1 -- 172.21.15.16:0/297189136 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7e30077480 con 0x7f7e30074980 2023-05-20T01:16:48.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.307+0000 7f7e4259c700 1 --2- 172.21.15.16:0/297189136 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x7f7e30074980 0x7f7e30076db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:48.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.308+0000 7f7e42d9d700 1 --2- 172.21.15.16:0/297189136 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f7e3006ad30 0x7f7e3006d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:48.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.308+0000 7f7e2b7fe700 1 -- 172.21.15.16:0/297189136 <== mon.2 v2:172.21.15.132:3301/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7e340541c0 con 0x7f7e440ce9b0 2023-05-20T01:16:48.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.308+0000 7f7e4259c700 1 --2- 172.21.15.16:0/297189136 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x7f7e30074980 0x7f7e30076db0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:48.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.307+0000 7f26cf7fe700 1 --2- 172.21.15.16:0/1959311532 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f26d406adb0 0x7f26d406d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:48.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.307+0000 7f26cf7fe700 1 -- 172.21.15.16:0/1959311532 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f26e0089cc0 con 0x7f26e80ccb60 2023-05-20T01:16:48.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.307+0000 7f26cf7fe700 1 --2- 172.21.15.16:0/1959311532 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x7f26d4074a80 0x7f26d4076eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:48.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.307+0000 7f26cf7fe700 1 -- 172.21.15.16:0/1959311532 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f26d4077580 con 0x7f26d4074a80 2023-05-20T01:16:48.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.307+0000 7f26cf7fe700 1 -- 172.21.15.16:0/1959311532 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f26e008a070 con 0x7f26e80ccb60 2023-05-20T01:16:48.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.307+0000 7f26e6ffd700 1 --2- 172.21.15.16:0/1959311532 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f26d406adb0 0x7f26d406d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:48.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.308+0000 7f26e67fc700 1 --2- 172.21.15.16:0/1959311532 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x7f26d4074a80 0x7f26d4076eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:48.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.307+0000 7fd5fad9d700 1 --2- 172.21.15.16:0/1959318617 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x7fd5dc074a00 0x7fd5dc076e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:48.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.308+0000 7fd5fa59c700 1 --2- 172.21.15.16:0/1959318617 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd5dc06ad30 0x7fd5dc06d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:48.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.308+0000 7fd5fad9d700 1 --2- 172.21.15.16:0/1959318617 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x7fd5dc074a00 0x7fd5dc076e30 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:48.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.307+0000 7f734dffb700 1 -- 172.21.15.16:0/873941470 <== mon.1 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7354022070 con 0x7f735806e170 2023-05-20T01:16:48.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.308+0000 7f734dffb700 1 --2- 172.21.15.16:0/873941470 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f733c06ad30 0x7f733c06d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:48.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.308+0000 7f734dffb700 1 -- 172.21.15.16:0/873941470 <== mon.1 v2:172.21.15.132:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f735408b050 con 0x7f735806e170 2023-05-20T01:16:48.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.308+0000 7f735f74a700 1 --2- 172.21.15.16:0/873941470 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x7f735813ce10 0x7f735814ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:48.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.308+0000 7f735f74a700 1 -- 172.21.15.16:0/873941470 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f735804e8c0 con 0x7f735813ce10 2023-05-20T01:16:48.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.308+0000 7f2920ff9700 1 -- 172.21.15.16:0/759793321 <== osd.6 v2:172.21.15.132:6801/727581123 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f2900006bb0 con 0x7f29000015d0 2023-05-20T01:16:48.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.307+0000 7f97e659c700 1 --2- 172.21.15.16:0/1016053349 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f97c806ad30 0x7f97c806d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:48.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.308+0000 7f97ecb42700 1 --2- 172.21.15.16:0/1016053349 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x7f97cc0015d0 0x7f97cc003a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:48.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.308+0000 7f97ecb42700 1 -- 172.21.15.16:0/1016053349 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f97cc006bb0 con 0x7f97cc0015d0 2023-05-20T01:16:48.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.308+0000 7f735dce7700 1 --2- 172.21.15.16:0/873941470 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x7f735813ce10 0x7f735814ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:48.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.309+0000 7f26e67fc700 1 --2- 172.21.15.16:0/1959311532 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x7f26d4074a80 0x7f26d4076eb0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:48.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.309+0000 7f97e6d9d700 1 --2- 172.21.15.16:0/1016053349 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x7f97cc0015d0 0x7f97cc003a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:48.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.309+0000 7f735cce5700 1 --2- 172.21.15.16:0/873941470 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f733c06ad30 0x7f733c06d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:48.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.309+0000 7f97e6d9d700 1 --2- 172.21.15.16:0/1016053349 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x7f97cc0015d0 0x7f97cc003a80 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:48.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.309+0000 7f7e42d9d700 1 --2- 172.21.15.16:0/297189136 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f7e3006ad30 0x7f7e3006d1e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7e3c00a230 tx=0x7f7e3c00c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:48.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.309+0000 7f7e2b7fe700 1 -- 172.21.15.16:0/297189136 <== osd.0 v2:172.21.15.16:6824/3923634179 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f7e30077480 con 0x7f7e30074980 2023-05-20T01:16:48.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.309+0000 7f2923fff700 1 --2- 172.21.15.16:0/759793321 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f290806ad00 0x7f290806d1b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f292406fba0 tx=0x7f2914006c60 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:48.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.309+0000 7f26e6ffd700 1 --2- 172.21.15.16:0/1959311532 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f26d406adb0 0x7f26d406d260 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f26d80090b0 tx=0x7f26d8008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:48.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.309+0000 7f97e659c700 1 --2- 172.21.15.16:0/1016053349 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f97c806ad30 0x7f97c806d1e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f97d8006b20 tx=0x7f97d8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:48.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.310+0000 7fd5fa59c700 1 --2- 172.21.15.16:0/1959318617 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd5dc06ad30 0x7fd5dc06d1e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec009b60 tx=0x7fd5ec009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:48.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.310+0000 7f735dce7700 1 --2- 172.21.15.16:0/873941470 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x7f735813ce10 0x7f735814ea20 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:48.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.310+0000 7f735cce5700 1 --2- 172.21.15.16:0/873941470 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f733c06ad30 0x7f733c06d1e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7358071960 tx=0x7f7348008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:48.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.310+0000 7f26cf7fe700 1 -- 172.21.15.16:0/1959311532 <== osd.3 v2:172.21.15.16:6800/3669781890 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f26d4077580 con 0x7f26d4074a80 2023-05-20T01:16:48.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.310+0000 7fd5eaffd700 1 -- 172.21.15.16:0/1959318617 <== osd.4 v2:172.21.15.132:6817/506317223 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fd5dc077500 con 0x7fd5dc074a00 2023-05-20T01:16:48.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.310+0000 7f90e23bd700 1 Processor -- start 2023-05-20T01:16:48.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.310+0000 7f97d6ffd700 1 -- 172.21.15.16:0/1016053349 <== osd.7 v2:172.21.15.132:6800/466001149 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f97cc006bb0 con 0x7f97cc0015d0 2023-05-20T01:16:48.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.312+0000 7f734dffb700 1 -- 172.21.15.16:0/873941470 <== osd.5 v2:172.21.15.132:6816/1844689995 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f735804e8c0 con 0x7f735813ce10 2023-05-20T01:16:48.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.310+0000 7f90e23bd700 1 -- start start 2023-05-20T01:16:48.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.312+0000 7f90e23bd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f90dc071a60 0x7f90dc071f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:48.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.312+0000 7f90e23bd700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f90dc164ae0 0x7f90dc164ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:48.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.312+0000 7f90e23bd700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90dc0ce0f0 con 0x7f90dc161000 2023-05-20T01:16:48.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.312+0000 7f90e23bd700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90dc072440 con 0x7f90dc071a60 2023-05-20T01:16:48.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.312+0000 7f90e23bd700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90dc0ce990 con 0x7f90dc164ae0 2023-05-20T01:16:48.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.312+0000 7f90dbfff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f90dc161000 0x7f90dc0caa20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:41464/0 (socket says 172.21.15.16:41464) 2023-05-20T01:16:48.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.312+0000 7f90dbfff700 1 -- 172.21.15.16:0/3055315690 learned_addr learned my addr 172.21.15.16:0/3055315690 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:16:48.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.317+0000 7f90e095a700 1 --2- 172.21.15.16:0/3055315690 >> v2:172.21.15.132:3300/0 conn(0x7f90dc164ae0 0x7f90dc164ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:48.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.317+0000 7f90daffd700 1 -- 172.21.15.16:0/3055315690 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1059562609 0 0) 0x7f90dc0ce0f0 con 0x7f90dc161000 2023-05-20T01:16:48.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.317+0000 7f90daffd700 1 -- 172.21.15.16:0/3055315690 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f90c40035b0 con 0x7f90dc161000 2023-05-20T01:16:48.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.317+0000 7f90daffd700 1 -- 172.21.15.16:0/3055315690 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 375239827 0 0) 0x7f90c40035b0 con 0x7f90dc161000 2023-05-20T01:16:48.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.317+0000 7f90daffd700 1 -- 172.21.15.16:0/3055315690 >> v2:172.21.15.132:3300/0 conn(0x7f90dc164ae0 msgr2=0x7f90dc164ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:48.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.317+0000 7f90daffd700 1 --2- 172.21.15.16:0/3055315690 >> v2:172.21.15.132:3300/0 conn(0x7f90dc164ae0 0x7f90dc164ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:48.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.317+0000 7f90daffd700 1 -- 172.21.15.16:0/3055315690 >> v2:172.21.15.16:3300/0 conn(0x7f90dc071a60 msgr2=0x7f90dc071f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:48.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.317+0000 7f90daffd700 1 --2- 172.21.15.16:0/3055315690 >> v2:172.21.15.16:3300/0 conn(0x7f90dc071a60 0x7f90dc071f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:48.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.317+0000 7f90daffd700 1 -- 172.21.15.16:0/3055315690 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90dc0cb500 con 0x7f90dc161000 2023-05-20T01:16:48.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.317+0000 7f90daffd700 1 -- 172.21.15.16:0/3055315690 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7f90d40019f0 con 0x7f90dc161000 2023-05-20T01:16:48.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.317+0000 7f90daffd700 1 -- 172.21.15.16:0/3055315690 >> v1:172.21.15.16:6789/0 conn(0x7f90dc161000 legacy=0x7f90dc0caa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:48.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.317+0000 7f90daffd700 1 --2- 172.21.15.16:0/3055315690 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90c4003a70 0x7f90c4003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:48.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.317+0000 7f90daffd700 1 --2- 172.21.15.16:0/3055315690 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f90c4004850 0x7f90c4004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:48.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.317+0000 7f90daffd700 1 --2- 172.21.15.16:0/3055315690 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f90c4005200 0x7f90c400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:48.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.317+0000 7f90daffd700 1 -- 172.21.15.16:0/3055315690 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90dc072440 con 0x7f90c4003a70 2023-05-20T01:16:48.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.317+0000 7f90daffd700 1 -- 172.21.15.16:0/3055315690 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90dc0ce990 con 0x7f90c4005200 2023-05-20T01:16:48.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.317+0000 7f90daffd700 1 -- 172.21.15.16:0/3055315690 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90c400c160 con 0x7f90c4004850 2023-05-20T01:16:48.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.318+0000 7f90dbfff700 1 --2- 172.21.15.16:0/3055315690 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90c4003a70 0x7f90c4003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:48.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.318+0000 7f90dbfff700 1 -- 172.21.15.16:0/3055315690 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f90c4004850 msgr2=0x7f90c4004cc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-05-20T01:16:48.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.318+0000 7f90dbfff700 1 --2- 172.21.15.16:0/3055315690 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f90c4004850 0x7f90c4004cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:48.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.318+0000 7f90dbfff700 1 -- 172.21.15.16:0/3055315690 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f90c4005200 msgr2=0x7f90c400ba10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-05-20T01:16:48.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.318+0000 7f90dbfff700 1 --2- 172.21.15.16:0/3055315690 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f90c4005200 0x7f90c400ba10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:48.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.318+0000 7f90dbfff700 1 -- 172.21.15.16:0/3055315690 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90dc0cb500 con 0x7f90c4003a70 2023-05-20T01:16:48.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.318+0000 7f90dbfff700 1 --2- 172.21.15.16:0/3055315690 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90c4003a70 0x7f90c4003e80 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f90d400a8c0 tx=0x7f90c4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b41f42c0105731b server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:48.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.319+0000 7f90daffd700 1 -- 172.21.15.16:0/3055315690 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90d4012600 con 0x7f90c4003a70 2023-05-20T01:16:48.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.319+0000 7f90daffd700 1 -- 172.21.15.16:0/3055315690 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90d4012760 con 0x7f90c4003a70 2023-05-20T01:16:48.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.319+0000 7f90daffd700 1 -- 172.21.15.16:0/3055315690 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90d4012af0 con 0x7f90c4003a70 2023-05-20T01:16:48.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.320+0000 7f90e23bd700 1 -- 172.21.15.16:0/3055315690 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90c4003a70 msgr2=0x7f90c4003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:48.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.320+0000 7f90e23bd700 1 --2- 172.21.15.16:0/3055315690 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90c4003a70 0x7f90c4003e80 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f90d400a8c0 tx=0x7f90c4005880 comp rx=0 tx=0).stop 2023-05-20T01:16:48.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.321+0000 7f90e095a700 1 -- 172.21.15.16:0/3055315690 reap_dead start 2023-05-20T01:16:48.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.321+0000 7f90e095a700 1 -- 172.21.15.16:0/3055315690 reap_dead start 2023-05-20T01:16:48.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.321+0000 7f90e23bd700 1 -- 172.21.15.16:0/3055315690 shutdown_connections 2023-05-20T01:16:48.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.321+0000 7f90e23bd700 1 -- 172.21.15.16:0/3055315690 >> 172.21.15.16:0/3055315690 conn(0x7f90dc0cb910 msgr2=0x7f90dc0cbd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:48.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.328+0000 7f81a1ee0700 1 -- 172.21.15.16:0/1294154832 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f817c002d30 con 0x7f81880748b0 2023-05-20T01:16:48.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.328+0000 7f90e23bd700 1 -- 172.21.15.16:0/3055315690 shutdown_connections 2023-05-20T01:16:48.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.329+0000 7f8183fff700 1 -- 172.21.15.16:0/1294154832 <== osd.1 v2:172.21.15.16:6808/3860264220 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f817c002d30 con 0x7f81880748b0 2023-05-20T01:16:48.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.329+0000 7f90e23bd700 1 -- 172.21.15.16:0/3055315690 wait complete. 2023-05-20T01:16:48.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.330+0000 7f90e23bd700 1 Processor -- start 2023-05-20T01:16:48.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.331+0000 7f90e23bd700 1 -- start start 2023-05-20T01:16:48.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.332+0000 7f90e23bd700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f90dc16c250 0x7f90dc0c3b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:48.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.332+0000 7f90e23bd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f90dc0c4060 0x7f90dc143c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:48.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.332+0000 7f90e23bd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90dc16c620 0x7f90dc144190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:48.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.332+0000 7f90e23bd700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90dc0c45e0 con 0x7f90dc16c620 2023-05-20T01:16:48.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.332+0000 7f90e23bd700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90dc0c48c0 con 0x7f90dc0c4060 2023-05-20T01:16:48.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.332+0000 7f90e23bd700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90dc0c4ba0 con 0x7f90dc16c250 2023-05-20T01:16:48.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.332+0000 7f90e095a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90dc16c620 0x7f90dc144190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:48.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.332+0000 7f90dbfff700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f90dc16c250 0x7f90dc0c3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:48.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.332+0000 7f90e095a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90dc16c620 0x7f90dc144190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58182/0 (socket says 172.21.15.16:58182) 2023-05-20T01:16:48.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.332+0000 7f90dbfff700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f90dc16c250 0x7f90dc0c3b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.16:58216/0 (socket says 172.21.15.16:58216) 2023-05-20T01:16:48.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.332+0000 7f90e095a700 1 -- 172.21.15.16:0/703449014 learned_addr learned my addr 172.21.15.16:0/703449014 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:48.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.333+0000 7f90db7fe700 1 --2- 172.21.15.16:0/703449014 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f90dc0c4060 0x7f90dc143c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:48.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.333+0000 7f90dbfff700 1 -- 172.21.15.16:0/703449014 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f90dc0c4060 msgr2=0x7f90dc143c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:48.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.333+0000 7f90dbfff700 1 --2- 172.21.15.16:0/703449014 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f90dc0c4060 0x7f90dc143c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:48.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.333+0000 7f90dbfff700 1 -- 172.21.15.16:0/703449014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90dc16c620 msgr2=0x7f90dc144190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:48.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.333+0000 7f90dbfff700 1 --2- 172.21.15.16:0/703449014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90dc16c620 0x7f90dc144190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:48.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.333+0000 7f90dbfff700 1 -- 172.21.15.16:0/703449014 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90dc0ce620 con 0x7f90dc16c250 2023-05-20T01:16:48.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.333+0000 7f90db7fe700 1 --2- 172.21.15.16:0/703449014 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f90dc0c4060 0x7f90dc143c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-05-20T01:16:48.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.333+0000 7f90dbfff700 1 --2- 172.21.15.16:0/703449014 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f90dc16c250 0x7f90dc0c3b20 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f90d4014000 tx=0x7f90d4001cb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:48.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.334+0000 7f90d8ff9700 1 -- 172.21.15.16:0/703449014 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90d4001fa0 con 0x7f90dc16c250 2023-05-20T01:16:48.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.334+0000 7f90d8ff9700 1 -- 172.21.15.16:0/703449014 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90d401d420 con 0x7f90dc16c250 2023-05-20T01:16:48.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.334+0000 7f90d8ff9700 1 -- 172.21.15.16:0/703449014 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90d401d7d0 con 0x7f90dc16c250 2023-05-20T01:16:48.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.334+0000 7f90e23bd700 1 -- 172.21.15.16:0/703449014 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90dc0ce340 con 0x7f90dc16c250 2023-05-20T01:16:48.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.334+0000 7f90e23bd700 1 -- 172.21.15.16:0/703449014 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90dc0cb130 con 0x7f90dc16c250 2023-05-20T01:16:48.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.336+0000 7f90e23bd700 1 -- 172.21.15.16:0/703449014 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f90c8000fe0 con 0x7f90dc16c250 2023-05-20T01:16:48.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.336+0000 7fd600984700 1 -- 172.21.15.16:0/1959318617 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd5fc0cbbd0 con 0x7fd5dc074a00 2023-05-20T01:16:48.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.337+0000 7fd5eaffd700 1 -- 172.21.15.16:0/1959318617 <== osd.4 v2:172.21.15.132:6817/506317223 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fd5fc0cbbd0 con 0x7fd5dc074a00 2023-05-20T01:16:48.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.337+0000 7f97ecb42700 1 -- 172.21.15.16:0/1016053349 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f97cc005c90 con 0x7f97cc0015d0 2023-05-20T01:16:48.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.338+0000 7fd600984700 1 -- 172.21.15.16:0/1959318617 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x7fd5dc074a00 msgr2=0x7fd5dc076e30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:48.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.338+0000 7fd600984700 1 --2- 172.21.15.16:0/1959318617 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x7fd5dc074a00 0x7fd5dc076e30 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:48.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.338+0000 7f97d6ffd700 1 -- 172.21.15.16:0/1016053349 <== osd.7 v2:172.21.15.132:6800/466001149 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f97cc005c90 con 0x7f97cc0015d0 2023-05-20T01:16:48.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.338+0000 7f81a1ee0700 1 -- 172.21.15.16:0/1294154832 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x7f81880748b0 msgr2=0x7f8188076ce0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:48.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.338+0000 7f81a1ee0700 1 --2- 172.21.15.16:0/1294154832 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x7f81880748b0 0x7f8188076ce0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:48.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.338+0000 7f90d8ff9700 1 -- 172.21.15.16:0/703449014 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f90d401d930 con 0x7f90dc16c250 2023-05-20T01:16:48.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.338+0000 7f97ecb42700 1 -- 172.21.15.16:0/1016053349 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x7f97cc0015d0 msgr2=0x7f97cc003a80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:48.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.338+0000 7f97ecb42700 1 --2- 172.21.15.16:0/1016053349 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x7f97cc0015d0 0x7f97cc003a80 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:48.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.338+0000 7fd600984700 1 -- 172.21.15.16:0/1959318617 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd5dc06ad30 msgr2=0x7fd5dc06d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:48.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.339+0000 7fd600984700 1 --2- 172.21.15.16:0/1959318617 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd5dc06ad30 0x7fd5dc06d1e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec009b60 tx=0x7fd5ec009210 comp rx=0 tx=0).stop 2023-05-20T01:16:48.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.339+0000 7fd600984700 1 -- 172.21.15.16:0/1959318617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5fc0b66a0 msgr2=0x7fd5fc1eae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:48.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.339+0000 7fd600984700 1 --2- 172.21.15.16:0/1959318617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5fc0b66a0 0x7fd5fc1eae50 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd5f0000c00 tx=0x7fd5f0004750 comp rx=0 tx=0).stop 2023-05-20T01:16:48.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.339+0000 7f97ecb42700 1 -- 172.21.15.16:0/1016053349 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f97c806ad30 msgr2=0x7f97c806d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:48.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.339+0000 7f97ecb42700 1 --2- 172.21.15.16:0/1016053349 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f97c806ad30 0x7f97c806d1e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f97d8006b20 tx=0x7f97d8008040 comp rx=0 tx=0).stop 2023-05-20T01:16:48.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.339+0000 7f97ecb42700 1 -- 172.21.15.16:0/1016053349 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f97e8160e70 msgr2=0x7f97e81573d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:48.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.339+0000 7f97ecb42700 1 --2- 172.21.15.16:0/1016053349 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f97e8160e70 0x7f97e81573d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f97dc009870 tx=0x7f97dc004750 comp rx=0 tx=0).stop 2023-05-20T01:16:48.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.339+0000 7f97e6d9d700 1 -- 172.21.15.16:0/1016053349 reap_dead start 2023-05-20T01:16:48.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.339+0000 7f90d8ff9700 1 --2- 172.21.15.16:0/703449014 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f90c406c6a0 0x7f90c406eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:48.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.339+0000 7fd5fad9d700 1 -- 172.21.15.16:0/1959318617 reap_dead start 2023-05-20T01:16:48.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.339+0000 7fd600984700 1 -- 172.21.15.16:0/1959318617 shutdown_connections 2023-05-20T01:16:48.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.339+0000 7fd600984700 1 -- 172.21.15.16:0/1959318617 >> 172.21.15.16:0/1959318617 conn(0x7fd5fc0d0950 msgr2=0x7fd5fc0b0d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:48.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.339+0000 7f90d8ff9700 1 -- 172.21.15.16:0/703449014 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f90d408b4a0 con 0x7f90dc16c250 2023-05-20T01:16:48.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.339+0000 7f26ed39f700 1 -- 172.21.15.16:0/1959311532 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f26c8002d30 con 0x7f26d4074a80 2023-05-20T01:16:48.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.339+0000 7f81a1ee0700 1 -- 172.21.15.16:0/1294154832 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f818806ace0 msgr2=0x7f818806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:48.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.339+0000 7f81a1ee0700 1 --2- 172.21.15.16:0/1294154832 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f818806ace0 0x7f818806d190 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f819c2025d0 tx=0x7f8194005d70 comp rx=0 tx=0).stop 2023-05-20T01:16:48.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.339+0000 7f81a1ee0700 1 -- 172.21.15.16:0/1294154832 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f819c0700c0 msgr2=0x7f819c0c3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:48.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.339+0000 7f81a1ee0700 1 --2- 172.21.15.16:0/1294154832 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f819c0700c0 0x7f819c0c3750 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f819000a1c0 tx=0x7f8190004750 comp rx=0 tx=0).stop 2023-05-20T01:16:48.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.339+0000 7f90db7fe700 1 --2- 172.21.15.16:0/703449014 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f90c406c6a0 0x7f90c406eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:48.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.339+0000 7f90d8ff9700 1 --2- 172.21.15.16:0/703449014 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x7f90c40762e0 0x7f90c4078710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:48.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.339+0000 7f90d8ff9700 1 -- 172.21.15.16:0/703449014 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f90c4078de0 con 0x7f90c40762e0 2023-05-20T01:16:48.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.339+0000 7f90d8ff9700 1 -- 172.21.15.16:0/703449014 <== mon.2 v2:172.21.15.132:3301/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f90d408b850 con 0x7f90dc16c250 2023-05-20T01:16:48.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.339+0000 7f819bfff700 1 -- 172.21.15.16:0/1294154832 reap_dead start 2023-05-20T01:16:48.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.339+0000 7f90e095a700 1 --2- 172.21.15.16:0/703449014 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x7f90c40762e0 0x7f90c4078710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:48.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.339+0000 7f97ecb42700 1 -- 172.21.15.16:0/1016053349 shutdown_connections 2023-05-20T01:16:48.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.339+0000 7f97ecb42700 1 -- 172.21.15.16:0/1016053349 >> 172.21.15.16:0/1016053349 conn(0x7f97e80b0e90 msgr2=0x7f97e80cf200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:48.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.339+0000 7f97ecb42700 1 -- 172.21.15.16:0/1016053349 shutdown_connections 2023-05-20T01:16:48.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.339+0000 7f97ecb42700 1 -- 172.21.15.16:0/1016053349 wait complete. 2023-05-20T01:16:48.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.340+0000 7f90e095a700 1 --2- 172.21.15.16:0/703449014 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x7f90c40762e0 0x7f90c4078710 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:48.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.340+0000 7f90db7fe700 1 --2- 172.21.15.16:0/703449014 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f90c406c6a0 0x7f90c406eb50 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f90dc151da0 tx=0x7f90d0009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:48.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.341+0000 7f90d8ff9700 1 -- 172.21.15.16:0/703449014 <== osd.2 v2:172.21.15.16:6816/293187030 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f90c4078de0 con 0x7f90c40762e0 2023-05-20T01:16:48.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.341+0000 7f292a617700 1 -- 172.21.15.16:0/759793321 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2900005c90 con 0x7f29000015d0 2023-05-20T01:16:48.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.341+0000 7f26cf7fe700 1 -- 172.21.15.16:0/1959311532 <== osd.3 v2:172.21.15.16:6800/3669781890 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f26c8002d30 con 0x7f26d4074a80 2023-05-20T01:16:48.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.341+0000 7f26ed39f700 1 -- 172.21.15.16:0/1959311532 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x7f26d4074a80 msgr2=0x7f26d4076eb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:48.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.341+0000 7f26ed39f700 1 --2- 172.21.15.16:0/1959311532 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x7f26d4074a80 0x7f26d4076eb0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:48.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.342+0000 7f26ed39f700 1 -- 172.21.15.16:0/1959311532 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f26d406adb0 msgr2=0x7f26d406d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:48.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.342+0000 7f26ed39f700 1 --2- 172.21.15.16:0/1959311532 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f26d406adb0 0x7f26d406d260 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f26d80090b0 tx=0x7f26d8008030 comp rx=0 tx=0).stop 2023-05-20T01:16:48.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.342+0000 7f26ed39f700 1 -- 172.21.15.16:0/1959311532 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26e80ccb60 msgr2=0x7f26e814e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:48.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.342+0000 7f26ed39f700 1 --2- 172.21.15.16:0/1959311532 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26e80ccb60 0x7f26e814e020 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f26e0001330 tx=0x7f26e000cf90 comp rx=0 tx=0).stop 2023-05-20T01:16:48.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.342+0000 7f26e77fe700 1 -- 172.21.15.16:0/1959311532 reap_dead start 2023-05-20T01:16:48.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.342+0000 7f26ed39f700 1 -- 172.21.15.16:0/1959311532 shutdown_connections 2023-05-20T01:16:48.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.342+0000 7f26ed39f700 1 -- 172.21.15.16:0/1959311532 >> 172.21.15.16:0/1959311532 conn(0x7f26e80cb910 msgr2=0x7f26e8159420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:48.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.342+0000 7f26ed39f700 1 -- 172.21.15.16:0/1959311532 shutdown_connections 2023-05-20T01:16:48.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.342+0000 7f26ed39f700 1 -- 172.21.15.16:0/1959311532 wait complete. 2023-05-20T01:16:48.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.343+0000 7f81a1ee0700 1 -- 172.21.15.16:0/1294154832 shutdown_connections 2023-05-20T01:16:48.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.343+0000 7f81a1ee0700 1 -- 172.21.15.16:0/1294154832 >> 172.21.15.16:0/1294154832 conn(0x7f819c0cb910 msgr2=0x7f819c156c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:48.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.343+0000 7fd600984700 1 -- 172.21.15.16:0/1959318617 shutdown_connections 2023-05-20T01:16:48.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.343+0000 7f81a1ee0700 1 -- 172.21.15.16:0/1294154832 shutdown_connections 2023-05-20T01:16:48.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.343+0000 7fd600984700 1 -- 172.21.15.16:0/1959318617 wait complete. 2023-05-20T01:16:48.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.343+0000 7f81a1ee0700 1 -- 172.21.15.16:0/1294154832 wait complete. 2023-05-20T01:16:48.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.343+0000 7f7e49188700 1 -- 172.21.15.16:0/297189136 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f7e440c38d0 con 0x7f7e30074980 2023-05-20T01:16:48.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.343+0000 7f7e2b7fe700 1 -- 172.21.15.16:0/297189136 <== osd.0 v2:172.21.15.16:6824/3923634179 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f7e440c38d0 con 0x7f7e30074980 2023-05-20T01:16:48.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.346+0000 7f7e49188700 1 -- 172.21.15.16:0/297189136 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x7f7e30074980 msgr2=0x7f7e30076db0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:48.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.346+0000 7f7e49188700 1 --2- 172.21.15.16:0/297189136 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x7f7e30074980 0x7f7e30076db0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:48.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.346+0000 7f735f74a700 1 -- 172.21.15.16:0/873941470 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f7358073fe0 con 0x7f735813ce10 2023-05-20T01:16:48.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.348+0000 7f734dffb700 1 -- 172.21.15.16:0/873941470 <== osd.5 v2:172.21.15.132:6816/1844689995 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f7358073fe0 con 0x7f735813ce10 2023-05-20T01:16:48.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.351+0000 7f735f74a700 1 -- 172.21.15.16:0/873941470 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x7f735813ce10 msgr2=0x7f735814ea20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:48.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.351+0000 7f735f74a700 1 --2- 172.21.15.16:0/873941470 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x7f735813ce10 0x7f735814ea20 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:48.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.351+0000 7f735f74a700 1 -- 172.21.15.16:0/873941470 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f733c06ad30 msgr2=0x7f733c06d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:48.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.351+0000 7f735f74a700 1 --2- 172.21.15.16:0/873941470 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f733c06ad30 0x7f733c06d1e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7358071960 tx=0x7f7348008040 comp rx=0 tx=0).stop 2023-05-20T01:16:48.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.351+0000 7f735f74a700 1 -- 172.21.15.16:0/873941470 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f735806e170 msgr2=0x7f73580c4be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:48.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.351+0000 7f735f74a700 1 --2- 172.21.15.16:0/873941470 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f735806e170 0x7f73580c4be0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7354015000 tx=0x7f7354004750 comp rx=0 tx=0).stop 2023-05-20T01:16:48.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.351+0000 7f735dce7700 1 -- 172.21.15.16:0/873941470 reap_dead start 2023-05-20T01:16:48.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.355+0000 7f735f74a700 1 -- 172.21.15.16:0/873941470 shutdown_connections 2023-05-20T01:16:48.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.355+0000 7f735f74a700 1 -- 172.21.15.16:0/873941470 >> 172.21.15.16:0/873941470 conn(0x7f73580cb910 msgr2=0x7f735815b6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:48.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.355+0000 7f7e49188700 1 -- 172.21.15.16:0/297189136 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f7e3006ad30 msgr2=0x7f7e3006d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:48.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.355+0000 7f7e49188700 1 --2- 172.21.15.16:0/297189136 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f7e3006ad30 0x7f7e3006d1e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7e3c00a230 tx=0x7f7e3c00c040 comp rx=0 tx=0).stop 2023-05-20T01:16:48.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.355+0000 7f7e49188700 1 -- 172.21.15.16:0/297189136 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7e440ce9b0 msgr2=0x7f7e440b0790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:48.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.355+0000 7f7e49188700 1 --2- 172.21.15.16:0/297189136 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7e440ce9b0 0x7f7e440b0790 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f7e3400c480 tx=0x7f7e3400c790 comp rx=0 tx=0).stop 2023-05-20T01:16:48.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.355+0000 7f2920ff9700 1 -- 172.21.15.16:0/759793321 <== osd.6 v2:172.21.15.132:6801/727581123 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f2900005c90 con 0x7f29000015d0 2023-05-20T01:16:48.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.355+0000 7f7e4359e700 1 -- 172.21.15.16:0/297189136 reap_dead start 2023-05-20T01:16:48.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.356+0000 7f7e49188700 1 -- 172.21.15.16:0/297189136 shutdown_connections 2023-05-20T01:16:48.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.356+0000 7f7e49188700 1 -- 172.21.15.16:0/297189136 >> 172.21.15.16:0/297189136 conn(0x7f7e440cb910 msgr2=0x7f7e44156000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:48.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.357+0000 7f735f74a700 1 -- 172.21.15.16:0/873941470 shutdown_connections 2023-05-20T01:16:48.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.357+0000 7f735f74a700 1 -- 172.21.15.16:0/873941470 wait complete. 2023-05-20T01:16:48.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.359+0000 7f292a617700 1 -- 172.21.15.16:0/759793321 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x7f29000015d0 msgr2=0x7f2900003a80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:48.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.359+0000 7f292a617700 1 --2- 172.21.15.16:0/759793321 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x7f29000015d0 0x7f2900003a80 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:48.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.360+0000 7f7e49188700 1 -- 172.21.15.16:0/297189136 shutdown_connections 2023-05-20T01:16:48.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.361+0000 7f7e49188700 1 -- 172.21.15.16:0/297189136 wait complete. 2023-05-20T01:16:48.409 INFO:teuthology.orchestra.run.smithi016.stdout:55834574857 2023-05-20T01:16:48.409 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-05-20T01:16:48.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.364+0000 7f292a617700 1 -- 172.21.15.16:0/759793321 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f290806ad00 msgr2=0x7f290806d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:48.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.364+0000 7f292a617700 1 --2- 172.21.15.16:0/759793321 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f290806ad00 0x7f290806d1b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f292406fba0 tx=0x7f2914006c60 comp rx=0 tx=0).stop 2023-05-20T01:16:48.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.364+0000 7f292a617700 1 -- 172.21.15.16:0/759793321 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2924165810 msgr2=0x7f2924165c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:48.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.364+0000 7f292a617700 1 --2- 172.21.15.16:0/759793321 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2924165810 0x7f2924165c80 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f292406ebf0 tx=0x7f291c004ad0 comp rx=0 tx=0).stop 2023-05-20T01:16:48.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.364+0000 7f2928bb4700 1 -- 172.21.15.16:0/759793321 reap_dead start 2023-05-20T01:16:48.411 INFO:teuthology.orchestra.run.smithi016.stdout:55834574856 2023-05-20T01:16:48.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.366+0000 7f90e23bd700 1 -- 172.21.15.16:0/703449014 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f90c8002d60 con 0x7f90c40762e0 2023-05-20T01:16:48.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.367+0000 7f90d8ff9700 1 -- 172.21.15.16:0/703449014 <== osd.2 v2:172.21.15.16:6816/293187030 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f90c8002d60 con 0x7f90c40762e0 2023-05-20T01:16:48.412 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-05-20T01:16:48.413 INFO:teuthology.orchestra.run.smithi016.stdout:55834574856 2023-05-20T01:16:48.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.365+0000 7f292a617700 1 -- 172.21.15.16:0/759793321 shutdown_connections 2023-05-20T01:16:48.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.365+0000 7f292a617700 1 -- 172.21.15.16:0/759793321 >> 172.21.15.16:0/759793321 conn(0x7f29240cb910 msgr2=0x7f2924167d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:48.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.365+0000 7f292a617700 1 -- 172.21.15.16:0/759793321 shutdown_connections 2023-05-20T01:16:48.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.365+0000 7f292a617700 1 -- 172.21.15.16:0/759793321 wait complete. 2023-05-20T01:16:48.414 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-05-20T01:16:48.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.368+0000 7f90e23bd700 1 -- 172.21.15.16:0/703449014 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x7f90c40762e0 msgr2=0x7f90c4078710 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:48.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.368+0000 7f90e23bd700 1 --2- 172.21.15.16:0/703449014 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x7f90c40762e0 0x7f90c4078710 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:48.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.368+0000 7f90e23bd700 1 -- 172.21.15.16:0/703449014 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f90c406c6a0 msgr2=0x7f90c406eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:48.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.368+0000 7f90e23bd700 1 --2- 172.21.15.16:0/703449014 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f90c406c6a0 0x7f90c406eb50 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f90dc151da0 tx=0x7f90d0009190 comp rx=0 tx=0).stop 2023-05-20T01:16:48.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.368+0000 7f90e23bd700 1 -- 172.21.15.16:0/703449014 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f90dc16c250 msgr2=0x7f90dc0c3b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:48.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.368+0000 7f90e23bd700 1 --2- 172.21.15.16:0/703449014 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f90dc16c250 0x7f90dc0c3b20 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f90d4014000 tx=0x7f90d4001cb0 comp rx=0 tx=0).stop 2023-05-20T01:16:48.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.368+0000 7f90e095a700 1 -- 172.21.15.16:0/703449014 reap_dead start 2023-05-20T01:16:48.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.368+0000 7f90e23bd700 1 -- 172.21.15.16:0/703449014 shutdown_connections 2023-05-20T01:16:48.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.369+0000 7f90e23bd700 1 -- 172.21.15.16:0/703449014 >> 172.21.15.16:0/703449014 conn(0x7f90dc0cb910 msgr2=0x7f90dc072fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:48.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.369+0000 7f90e23bd700 1 -- 172.21.15.16:0/703449014 shutdown_connections 2023-05-20T01:16:48.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.369+0000 7f90e23bd700 1 -- 172.21.15.16:0/703449014 wait complete. 2023-05-20T01:16:48.417 INFO:teuthology.orchestra.run.smithi016.stdout:55834574856 2023-05-20T01:16:48.417 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-05-20T01:16:48.418 INFO:teuthology.orchestra.run.smithi016.stdout:55834574857 2023-05-20T01:16:48.418 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-05-20T01:16:48.419 INFO:teuthology.orchestra.run.smithi016.stdout:55834574856 2023-05-20T01:16:48.419 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-05-20T01:16:48.420 INFO:teuthology.orchestra.run.smithi016.stdout:55834574856 2023-05-20T01:16:48.421 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-05-20T01:16:48.421 INFO:teuthology.orchestra.run.smithi016.stdout:55834574856 2023-05-20T01:16:48.422 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-05-20T01:16:48.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.663+0000 7fa66b0cf700 1 Processor -- start 2023-05-20T01:16:48.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.663+0000 7fa66b0cf700 1 -- start start 2023-05-20T01:16:48.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.665+0000 7fa66b0cf700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa664160f30 0x7fa6640ce090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:48.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.665+0000 7fa66b0cf700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa664161300 con 0x7fa6640cc900 2023-05-20T01:16:48.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.665+0000 7fa66b0cf700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6640caad0 con 0x7fa6640b0f50 2023-05-20T01:16:48.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.665+0000 7fa66b0cf700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6640cd9d0 con 0x7fa664160f30 2023-05-20T01:16:48.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.666+0000 7fa66966c700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7fa6640b0f50 0x7fa664164a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.16:51200/0 (socket says 172.21.15.16:51200) 2023-05-20T01:16:48.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.666+0000 7fa66966c700 1 -- 172.21.15.16:0/3937931400 learned_addr learned my addr 172.21.15.16:0/3937931400 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:16:48.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.667+0000 7fa668e6b700 1 --2- 172.21.15.16:0/3937931400 >> v2:172.21.15.16:3300/0 conn(0x7fa664160f30 0x7fa6640ce090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:48.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.667+0000 7fa668e6b700 1 -- 172.21.15.16:0/3937931400 >> v1:172.21.15.132:6789/0 conn(0x7fa6640b0f50 legacy=0x7fa664164a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:48.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.667+0000 7fa668e6b700 1 -- 172.21.15.16:0/3937931400 >> v1:172.21.15.16:6789/0 conn(0x7fa6640cc900 legacy=0x7fa6640b0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:48.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.667+0000 7fa668e6b700 1 -- 172.21.15.16:0/3937931400 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6640cadb0 con 0x7fa664160f30 2023-05-20T01:16:48.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.667+0000 7fa668e6b700 1 --2- 172.21.15.16:0/3937931400 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa664160f30 0x7fa6640ce090 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa65c009d90 tx=0x7fa65c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57e1cffff4b780be server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:48.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.669+0000 7fa6637fe700 1 -- 172.21.15.16:0/3937931400 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa65c00e070 con 0x7fa664160f30 2023-05-20T01:16:48.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.669+0000 7fa6637fe700 1 -- 172.21.15.16:0/3937931400 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa65c00bb90 con 0x7fa664160f30 2023-05-20T01:16:48.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.669+0000 7fa6637fe700 1 -- 172.21.15.16:0/3937931400 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa65c003a20 con 0x7fa664160f30 2023-05-20T01:16:48.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.669+0000 7fa66b0cf700 1 -- 172.21.15.16:0/3937931400 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa664160f30 msgr2=0x7fa6640ce090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:48.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.669+0000 7fa66b0cf700 1 --2- 172.21.15.16:0/3937931400 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa664160f30 0x7fa6640ce090 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa65c009d90 tx=0x7fa65c00b330 comp rx=0 tx=0).stop 2023-05-20T01:16:48.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.670+0000 7fa66b0cf700 1 -- 172.21.15.16:0/3937931400 shutdown_connections 2023-05-20T01:16:48.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.670+0000 7fa66b0cf700 1 --2- 172.21.15.16:0/3937931400 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa664160f30 0x7fa6640ce090 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:48.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.670+0000 7fa66b0cf700 1 -- 172.21.15.16:0/3937931400 >> 172.21.15.16:0/3937931400 conn(0x7fa6640cb840 msgr2=0x7fa6640cbc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:48.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.670+0000 7fa66b0cf700 1 -- 172.21.15.16:0/3937931400 shutdown_connections 2023-05-20T01:16:48.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.670+0000 7fa66b0cf700 1 -- 172.21.15.16:0/3937931400 wait complete. 2023-05-20T01:16:48.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.670+0000 7f6c52269700 1 Processor -- start 2023-05-20T01:16:48.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.671+0000 7f6c52269700 1 -- start start 2023-05-20T01:16:48.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.671+0000 7fc62a0da700 1 Processor -- start 2023-05-20T01:16:48.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.671+0000 7fc62a0da700 1 -- start start 2023-05-20T01:16:48.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.671+0000 7fa66b0cf700 1 Processor -- start 2023-05-20T01:16:48.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.672+0000 7fdc2090a700 1 Processor -- start 2023-05-20T01:16:48.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.671+0000 7fa66b0cf700 1 -- start start 2023-05-20T01:16:48.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.672+0000 7fc62a0da700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc624161000 0x7fc6240b0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:48.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.672+0000 7fc62a0da700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6241613d0 con 0x7fc6240ce740 2023-05-20T01:16:48.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.672+0000 7fc62a0da700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6240b12a0 con 0x7fc6240b0e50 2023-05-20T01:16:48.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.672+0000 7fc62a0da700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6240cecf0 con 0x7fc624161000 2023-05-20T01:16:48.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.673+0000 7fc623fff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fc6240ce740 0x7fc6240c1a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:41498/0 (socket says 172.21.15.16:41498) 2023-05-20T01:16:48.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.673+0000 7fc623fff700 1 -- 172.21.15.16:0/2932852117 learned_addr learned my addr 172.21.15.16:0/2932852117 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:16:48.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.672+0000 7f6c52269700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6c4c161000 0x7f6c4c0b0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:48.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.672+0000 7f6c52269700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6c4c0b0e50 0x7f6c4c0ce030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:48.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.672+0000 7f6c52269700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c4c1613d0 con 0x7f6c4c0ce570 2023-05-20T01:16:48.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.672+0000 7f6c52269700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c4c0cecf0 con 0x7f6c4c0b0e50 2023-05-20T01:16:48.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.672+0000 7f6c52269700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c4c0b12a0 con 0x7f6c4c161000 2023-05-20T01:16:48.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.672+0000 7f6c4b7fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6c4c0b0e50 0x7f6c4c0ce030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:48.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.672+0000 7f6c50806700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f6c4c0ce570 0x7f6c4c0c1a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:41494/0 (socket says 172.21.15.16:41494) 2023-05-20T01:16:48.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.672+0000 7f6c50806700 1 -- 172.21.15.16:0/3719404108 learned_addr learned my addr 172.21.15.16:0/3719404108 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:16:48.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.673+0000 7f6c4b7fe700 1 -- 172.21.15.16:0/3719404108 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6c4c161000 msgr2=0x7f6c4c0b0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:48.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.673+0000 7f6c4b7fe700 1 --2- 172.21.15.16:0/3719404108 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6c4c161000 0x7f6c4c0b0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:48.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.673+0000 7f6c4b7fe700 1 -- 172.21.15.16:0/3719404108 >> v1:172.21.15.16:6789/0 conn(0x7f6c4c0ce570 legacy=0x7f6c4c0c1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:48.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.673+0000 7f6c4b7fe700 1 -- 172.21.15.16:0/3719404108 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c4c06db20 con 0x7f6c4c0b0e50 2023-05-20T01:16:48.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.673+0000 7fa66b0cf700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa6640b0f50 0x7fa6641569b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:48.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.673+0000 7fa66b0cf700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6640cc900 0x7fa6640b2e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:48.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.673+0000 7fa66b0cf700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa664160f30 0x7fa6640aeb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:48.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.673+0000 7fa66b0cf700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6640c6fd0 con 0x7fa6640cc900 2023-05-20T01:16:48.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.673+0000 7fa66b0cf700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6640c63c0 con 0x7fa664160f30 2023-05-20T01:16:48.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.673+0000 7fa66b0cf700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6640c66a0 con 0x7fa6640b0f50 2023-05-20T01:16:48.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.673+0000 7fa668e6b700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa6640b0f50 0x7fa6641569b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:48.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.673+0000 7fa66966c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa664160f30 0x7fa6640aeb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:48.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.673+0000 7fa66966c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa664160f30 0x7fa6640aeb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.16:55228/0 (socket says 172.21.15.16:55228) 2023-05-20T01:16:48.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.673+0000 7fa66966c700 1 -- 172.21.15.16:0/3796166182 learned_addr learned my addr 172.21.15.16:0/3796166182 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:48.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.673+0000 7fc6237fe700 1 --2- 172.21.15.16:0/2932852117 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc624161000 0x7fc6240b0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:48.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.673+0000 7f6c4b7fe700 1 --2- 172.21.15.16:0/3719404108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c4c0b0e50 0x7f6c4c0ce030 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c009840 tx=0x7f6c3c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cc51a9c8d9a22ad server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:48.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.673+0000 7fa663fff700 1 --2- 172.21.15.16:0/3796166182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6640cc900 0x7fa6640b2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:48.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.673+0000 7fdc2090a700 1 -- start start 2023-05-20T01:16:48.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.674+0000 7fa66966c700 1 -- 172.21.15.16:0/3796166182 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa6640b0f50 msgr2=0x7fa6641569b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:48.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.674+0000 7fa66966c700 1 --2- 172.21.15.16:0/3796166182 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa6640b0f50 0x7fa6641569b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:48.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.674+0000 7fa66966c700 1 -- 172.21.15.16:0/3796166182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6640cc900 msgr2=0x7fa6640b2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:48.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.674+0000 7fa66966c700 1 --2- 172.21.15.16:0/3796166182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6640cc900 0x7fa6640b2e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:48.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.674+0000 7fa66966c700 1 -- 172.21.15.16:0/3796166182 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa65c009a70 con 0x7fa664160f30 2023-05-20T01:16:48.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.674+0000 7fc6237fe700 1 -- 172.21.15.16:0/2932852117 >> v1:172.21.15.132:6789/0 conn(0x7fc6240b0e50 legacy=0x7fc6240ce030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:48.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.674+0000 7fc6237fe700 1 -- 172.21.15.16:0/2932852117 >> v1:172.21.15.16:6789/0 conn(0x7fc6240ce740 legacy=0x7fc6240c1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:48.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.674+0000 7fc6237fe700 1 -- 172.21.15.16:0/2932852117 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc62406db20 con 0x7fc624161000 2023-05-20T01:16:48.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.674+0000 7fc6237fe700 1 --2- 172.21.15.16:0/2932852117 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc624161000 0x7fc6240b0910 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc614009870 tx=0x7fc614009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9064601a9a33b6d0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:48.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.674+0000 7fdc2090a700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fdc1c161000 0x7fdc1c0caa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:48.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.674+0000 7fdc2090a700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdc1c0caf60 0x7fdc1c071a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:48.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.675+0000 7fdc2090a700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc1c072400 con 0x7fdc1c071fa0 2023-05-20T01:16:48.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.675+0000 7fdc2090a700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc1c0ce0f0 con 0x7fdc1c161000 2023-05-20T01:16:48.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.675+0000 7fdc2090a700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1c0ce990 con 0x7fdc1c0caf60 2023-05-20T01:16:48.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.675+0000 7fdc1a59c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fdc1c161000 0x7fdc1c0caa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:48.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.675+0000 7fdc1a59c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fdc1c161000 0x7fdc1c0caa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58212/0 (socket says 172.21.15.16:58212) 2023-05-20T01:16:48.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.675+0000 7fdc1a59c700 1 -- 172.21.15.16:0/3548198422 learned_addr learned my addr 172.21.15.16:0/3548198422 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:48.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.674+0000 7fa66966c700 1 --2- 172.21.15.16:0/3796166182 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa664160f30 0x7fa6640aeb60 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa65400c450 tx=0x7fa65400c760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:48.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.675+0000 7fa6617fa700 1 -- 172.21.15.16:0/3796166182 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa654005d70 con 0x7fa664160f30 2023-05-20T01:16:48.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.675+0000 7fa6617fa700 1 -- 172.21.15.16:0/3796166182 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa65400ce20 con 0x7fa664160f30 2023-05-20T01:16:48.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.675+0000 7fa6617fa700 1 -- 172.21.15.16:0/3796166182 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6540072f0 con 0x7fa664160f30 2023-05-20T01:16:48.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.675+0000 7fa66b0cf700 1 -- 172.21.15.16:0/3796166182 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6640cb370 con 0x7fa664160f30 2023-05-20T01:16:48.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.675+0000 7fa66b0cf700 1 -- 172.21.15.16:0/3796166182 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6640cb090 con 0x7fa664160f30 2023-05-20T01:16:48.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.675+0000 7fdc19d9b700 1 --2- 172.21.15.16:0/3548198422 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdc1c0caf60 0x7fdc1c071a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:48.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.675+0000 7fdc1a59c700 1 -- 172.21.15.16:0/3548198422 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdc1c0caf60 msgr2=0x7fdc1c071a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:48.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.675+0000 7fdc1a59c700 1 --2- 172.21.15.16:0/3548198422 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdc1c0caf60 0x7fdc1c071a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:48.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.675+0000 7fdc1959a700 1 -- 172.21.15.16:0/3548198422 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1403889477 0 0) 0x7fdc10001110 con 0x7fdc1c071fa0 2023-05-20T01:16:48.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.675+0000 7fdc1a59c700 1 -- 172.21.15.16:0/3548198422 >> v1:172.21.15.16:6789/0 conn(0x7fdc1c071fa0 legacy=0x7fdc1c06ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:48.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.675+0000 7fdc1a59c700 1 -- 172.21.15.16:0/3548198422 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc1c0cb500 con 0x7fdc1c161000 2023-05-20T01:16:48.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.675+0000 7fdc1a59c700 1 --2- 172.21.15.16:0/3548198422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc1c161000 0x7fdc1c0caa20 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c0098f0 tx=0x7fdc0c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5170da77ca6be8d3 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:48.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.676+0000 7fdc1959a700 1 -- 172.21.15.16:0/3548198422 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc0c00b8a0 con 0x7fdc1c161000 2023-05-20T01:16:48.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.676+0000 7fdc1959a700 1 -- 172.21.15.16:0/3548198422 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc0c00ba00 con 0x7fdc1c161000 2023-05-20T01:16:48.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.676+0000 7fdc1959a700 1 -- 172.21.15.16:0/3548198422 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc0c00bd90 con 0x7fdc1c161000 2023-05-20T01:16:48.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.677+0000 7fdc2090a700 1 -- 172.21.15.16:0/3548198422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc1c161000 msgr2=0x7fdc1c0caa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:48.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.677+0000 7fdc2090a700 1 --2- 172.21.15.16:0/3548198422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc1c161000 0x7fdc1c0caa20 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c0098f0 tx=0x7fdc0c009c00 comp rx=0 tx=0).stop 2023-05-20T01:16:48.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.677+0000 7f6c4affd700 1 -- 172.21.15.16:0/3719404108 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c3c004230 con 0x7f6c4c0b0e50 2023-05-20T01:16:48.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.677+0000 7fa66b0cf700 1 -- 172.21.15.16:0/3796166182 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6440052e0 con 0x7fa664160f30 2023-05-20T01:16:48.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.677+0000 7f6c4affd700 1 -- 172.21.15.16:0/3719404108 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c3c004390 con 0x7f6c4c0b0e50 2023-05-20T01:16:48.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.677+0000 7f6c4affd700 1 -- 172.21.15.16:0/3719404108 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c3c0036a0 con 0x7f6c4c0b0e50 2023-05-20T01:16:48.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.677+0000 7fdc2090a700 1 -- 172.21.15.16:0/3548198422 shutdown_connections 2023-05-20T01:16:48.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.677+0000 7fdc2090a700 1 --2- 172.21.15.16:0/3548198422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc1c161000 0x7fdc1c0caa20 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:48.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.677+0000 7fdc2090a700 1 --2- 172.21.15.16:0/3548198422 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdc1c0caf60 0x7fdc1c071a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:48.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.677+0000 7fdc2090a700 1 -- 172.21.15.16:0/3548198422 >> 172.21.15.16:0/3548198422 conn(0x7fdc1c0cb910 msgr2=0x7fdc1c0cbd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:48.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.677+0000 7fdc2090a700 1 -- 172.21.15.16:0/3548198422 shutdown_connections 2023-05-20T01:16:48.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.677+0000 7fdc2090a700 1 -- 172.21.15.16:0/3548198422 wait complete. 2023-05-20T01:16:48.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.678+0000 7fa6617fa700 1 -- 172.21.15.16:0/3796166182 <== mon.1 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa654007450 con 0x7fa664160f30 2023-05-20T01:16:48.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.678+0000 7f6c52269700 1 -- 172.21.15.16:0/3719404108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c4c0b0e50 msgr2=0x7f6c4c0ce030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:48.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.678+0000 7f6c52269700 1 --2- 172.21.15.16:0/3719404108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c4c0b0e50 0x7f6c4c0ce030 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c009840 tx=0x7f6c3c009b50 comp rx=0 tx=0).stop 2023-05-20T01:16:48.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.678+0000 7f6c52269700 1 -- 172.21.15.16:0/3719404108 shutdown_connections 2023-05-20T01:16:48.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.678+0000 7f6c52269700 1 --2- 172.21.15.16:0/3719404108 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6c4c161000 0x7f6c4c0b0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:48.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.678+0000 7f6c52269700 1 --2- 172.21.15.16:0/3719404108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c4c0b0e50 0x7f6c4c0ce030 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:48.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.678+0000 7f6c52269700 1 -- 172.21.15.16:0/3719404108 >> 172.21.15.16:0/3719404108 conn(0x7f6c4c0cb910 msgr2=0x7f6c4c0cbd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:48.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.678+0000 7f6c52269700 1 -- 172.21.15.16:0/3719404108 shutdown_connections 2023-05-20T01:16:48.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.678+0000 7f6c52269700 1 -- 172.21.15.16:0/3719404108 wait complete. 2023-05-20T01:16:48.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.678+0000 7fa6617fa700 1 --2- 172.21.15.16:0/3796166182 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa65006ad30 0x7fa65006d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:48.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.678+0000 7fa6617fa700 1 -- 172.21.15.16:0/3796166182 <== mon.1 v2:172.21.15.132:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fa654088940 con 0x7fa664160f30 2023-05-20T01:16:48.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.679+0000 7fa668e6b700 1 --2- 172.21.15.16:0/3796166182 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa65006ad30 0x7fa65006d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:48.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.679+0000 7fdc2090a700 1 Processor -- start 2023-05-20T01:16:48.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.679+0000 7eff56856700 1 Processor -- start 2023-05-20T01:16:48.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.679+0000 7eff56856700 1 -- start start 2023-05-20T01:16:48.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.679+0000 7fa668e6b700 1 --2- 172.21.15.16:0/3796166182 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa65006ad30 0x7fa65006d1e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa65c00b370 tx=0x7fa65c014040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:48.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.680+0000 7f6c52269700 1 Processor -- start 2023-05-20T01:16:48.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.680+0000 7f6c52269700 1 -- start start 2023-05-20T01:16:48.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.680+0000 7fc6227fc700 1 -- 172.21.15.16:0/2932852117 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc614011830 con 0x7fc624161000 2023-05-20T01:16:48.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.680+0000 7fc6227fc700 1 -- 172.21.15.16:0/2932852117 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc614011990 con 0x7fc624161000 2023-05-20T01:16:48.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.680+0000 7fc6227fc700 1 -- 172.21.15.16:0/2932852117 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc614011d20 con 0x7fc624161000 2023-05-20T01:16:48.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.679+0000 7fdc2090a700 1 -- start start 2023-05-20T01:16:48.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.680+0000 7fdc2090a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc1c071fa0 0x7fdc1c0c3d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:48.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.680+0000 7fdc2090a700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdc1c0caf60 0x7fdc1c1f9c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:48.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.680+0000 7fdc2090a700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdc1c161000 0x7fdc1c0b4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:48.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.680+0000 7fdc2090a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1c1ec900 con 0x7fdc1c071fa0 2023-05-20T01:16:48.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.680+0000 7fdc2090a700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1c1ecbe0 con 0x7fdc1c0caf60 2023-05-20T01:16:48.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.680+0000 7fdc2090a700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1c1ecec0 con 0x7fdc1c161000 2023-05-20T01:16:48.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.680+0000 7fdc1a59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc1c071fa0 0x7fdc1c0c3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:48.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.680+0000 7fdc1a59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc1c071fa0 0x7fdc1c0c3d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58220/0 (socket says 172.21.15.16:58220) 2023-05-20T01:16:48.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.680+0000 7fdc1a59c700 1 -- 172.21.15.16:0/1193055875 learned_addr learned my addr 172.21.15.16:0/1193055875 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:48.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.680+0000 7fdc1ad9d700 1 --2- 172.21.15.16:0/1193055875 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdc1c161000 0x7fdc1c0b4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:48.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.680+0000 7fdc19d9b700 1 --2- 172.21.15.16:0/1193055875 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdc1c0caf60 0x7fdc1c1f9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:48.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.681+0000 7eff56856700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7eff5015a8c0 0x7eff500caba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:48.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.681+0000 7eff56856700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff5015af40 con 0x7eff500cb0e0 2023-05-20T01:16:48.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.681+0000 7eff56856700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff5015ac90 con 0x7eff500b1020 2023-05-20T01:16:48.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.681+0000 7eff56856700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff5015b1e0 con 0x7eff5015a8c0 2023-05-20T01:16:48.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.681+0000 7fdc19d9b700 1 -- 172.21.15.16:0/1193055875 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdc1c161000 msgr2=0x7fdc1c0b4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:48.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.681+0000 7fdc19d9b700 1 --2- 172.21.15.16:0/1193055875 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdc1c161000 0x7fdc1c0b4140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:48.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.681+0000 7fdc19d9b700 1 -- 172.21.15.16:0/1193055875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc1c071fa0 msgr2=0x7fdc1c0c3d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:48.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.681+0000 7fdc19d9b700 1 --2- 172.21.15.16:0/1193055875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc1c071fa0 0x7fdc1c0c3d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:48.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.681+0000 7fdc19d9b700 1 -- 172.21.15.16:0/1193055875 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc0c009550 con 0x7fdc1c0caf60 2023-05-20T01:16:48.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.681+0000 7fdc1ad9d700 1 --2- 172.21.15.16:0/1193055875 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdc1c161000 0x7fdc1c0b4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:16:48.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.681+0000 7fdc1a59c700 1 --2- 172.21.15.16:0/1193055875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc1c071fa0 0x7fdc1c0c3d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-05-20T01:16:48.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.681+0000 7eff54df3700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7eff500b1020 0x7eff5006ba70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.16:51238/0 (socket says 172.21.15.16:51238) 2023-05-20T01:16:48.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.681+0000 7eff54df3700 1 -- 172.21.15.16:0/4015206694 learned_addr learned my addr 172.21.15.16:0/4015206694 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:16:48.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.681+0000 7fdc19d9b700 1 --2- 172.21.15.16:0/1193055875 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdc1c0caf60 0x7fdc1c1f9c70 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fdc1400cc10 tx=0x7fdc1400cf20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:48.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.681+0000 7fc62a0da700 1 -- 172.21.15.16:0/2932852117 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc624161000 msgr2=0x7fc6240b0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:48.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.681+0000 7fc62a0da700 1 --2- 172.21.15.16:0/2932852117 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc624161000 0x7fc6240b0910 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc614009870 tx=0x7fc614009b80 comp rx=0 tx=0).stop 2023-05-20T01:16:48.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.681+0000 7f6c52269700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6c4c0b0e50 0x7f6c4c1565b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:48.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.681+0000 7f6c52269700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6c4c0ce570 0x7f6c4c1577e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:48.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.681+0000 7f6c52269700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c4c161000 0x7f6c4c0afec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:48.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.681+0000 7f6c52269700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c4c161570 con 0x7f6c4c161000 2023-05-20T01:16:48.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.681+0000 7f6c52269700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c4c157ec0 con 0x7f6c4c0ce570 2023-05-20T01:16:48.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.681+0000 7f6c52269700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c4c1581a0 con 0x7f6c4c0b0e50 2023-05-20T01:16:48.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.681+0000 7fc62a0da700 1 -- 172.21.15.16:0/2932852117 shutdown_connections 2023-05-20T01:16:48.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.681+0000 7fc62a0da700 1 --2- 172.21.15.16:0/2932852117 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc624161000 0x7fc6240b0910 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:48.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.681+0000 7fc62a0da700 1 -- 172.21.15.16:0/2932852117 >> 172.21.15.16:0/2932852117 conn(0x7fc6240cb910 msgr2=0x7fc6240cbd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:48.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.681+0000 7f6c50806700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c4c161000 0x7f6c4c0afec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:48.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.682+0000 7f6c50806700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c4c161000 0x7f6c4c0afec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58230/0 (socket says 172.21.15.16:58230) 2023-05-20T01:16:48.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.682+0000 7f6c50806700 1 -- 172.21.15.16:0/2385770827 learned_addr learned my addr 172.21.15.16:0/2385770827 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:48.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.682+0000 7fc62a0da700 1 -- 172.21.15.16:0/2932852117 shutdown_connections 2023-05-20T01:16:48.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.682+0000 7fc62a0da700 1 -- 172.21.15.16:0/2932852117 wait complete. 2023-05-20T01:16:48.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.682+0000 7f6c50806700 1 -- 172.21.15.16:0/2385770827 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6c4c0b0e50 msgr2=0x7f6c4c1565b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-20T01:16:48.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.682+0000 7f6c50806700 1 --2- 172.21.15.16:0/2385770827 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6c4c0b0e50 0x7f6c4c1565b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:48.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.682+0000 7f6c50806700 1 -- 172.21.15.16:0/2385770827 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6c4c0ce570 msgr2=0x7f6c4c1577e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-20T01:16:48.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.682+0000 7f6c50806700 1 --2- 172.21.15.16:0/2385770827 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6c4c0ce570 0x7f6c4c1577e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:48.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.682+0000 7f6c50806700 1 -- 172.21.15.16:0/2385770827 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c3c0094a0 con 0x7f6c4c161000 2023-05-20T01:16:48.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.682+0000 7f6c50806700 1 --2- 172.21.15.16:0/2385770827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c4c161000 0x7f6c4c0afec0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f6c44002190 tx=0x7f6c4400a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:48.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.682+0000 7eff4effd700 1 -- 172.21.15.16:0/4015206694 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1065130348 0 0) 0x7eff5015af40 con 0x7eff500cb0e0 2023-05-20T01:16:48.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.682+0000 7eff4effd700 1 -- 172.21.15.16:0/4015206694 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7eff380035b0 con 0x7eff500cb0e0 2023-05-20T01:16:48.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.682+0000 7f6c48ff9700 1 -- 172.21.15.16:0/2385770827 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c440070d0 con 0x7f6c4c161000 2023-05-20T01:16:48.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.682+0000 7f96c3791700 1 Processor -- start 2023-05-20T01:16:48.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.682+0000 7f6c48ff9700 1 -- 172.21.15.16:0/2385770827 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c44007230 con 0x7f6c4c161000 2023-05-20T01:16:48.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.682+0000 7f6c48ff9700 1 -- 172.21.15.16:0/2385770827 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c440075e0 con 0x7f6c4c161000 2023-05-20T01:16:48.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.683+0000 7eff4effd700 1 -- 172.21.15.16:0/4015206694 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1504462913 0 0) 0x7eff380035b0 con 0x7eff500cb0e0 2023-05-20T01:16:48.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.683+0000 7eff4effd700 1 -- 172.21.15.16:0/4015206694 >> v2:172.21.15.16:3300/0 conn(0x7eff5015a8c0 msgr2=0x7eff500caba0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-05-20T01:16:48.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.683+0000 7eff4effd700 1 --2- 172.21.15.16:0/4015206694 >> v2:172.21.15.16:3300/0 conn(0x7eff5015a8c0 0x7eff500caba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:48.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.683+0000 7eff4effd700 1 -- 172.21.15.16:0/4015206694 >> v1:172.21.15.132:6789/0 conn(0x7eff500b1020 legacy=0x7eff5006ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:48.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.683+0000 7eff4effd700 1 -- 172.21.15.16:0/4015206694 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff5015b4c0 con 0x7eff500cb0e0 2023-05-20T01:16:48.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.683+0000 7eff4effd700 1 -- 172.21.15.16:0/4015206694 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7eff44001670 con 0x7eff500cb0e0 2023-05-20T01:16:48.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.683+0000 7eff4effd700 1 -- 172.21.15.16:0/4015206694 >> v1:172.21.15.16:6789/0 conn(0x7eff500cb0e0 legacy=0x7eff500b0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:48.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.683+0000 7eff4effd700 1 --2- 172.21.15.16:0/4015206694 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7eff38003a70 0x7eff38003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:48.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.683+0000 7eff4effd700 1 --2- 172.21.15.16:0/4015206694 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7eff38004850 0x7eff38004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:48.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.683+0000 7eff4effd700 1 --2- 172.21.15.16:0/4015206694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff38005200 0x7eff38014610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:48.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.683+0000 7eff4effd700 1 -- 172.21.15.16:0/4015206694 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff5015b1e0 con 0x7eff38005200 2023-05-20T01:16:48.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.683+0000 7eff4effd700 1 -- 172.21.15.16:0/4015206694 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff38014b50 con 0x7eff38003a70 2023-05-20T01:16:48.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.683+0000 7eff4effd700 1 -- 172.21.15.16:0/4015206694 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff38014c90 con 0x7eff38004850 2023-05-20T01:16:48.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.683+0000 7eff4f7fe700 1 --2- 172.21.15.16:0/4015206694 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7eff38004850 0x7eff38004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:48.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.682+0000 7f6c52269700 1 -- 172.21.15.16:0/2385770827 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c4c06bef0 con 0x7f6c4c161000 2023-05-20T01:16:48.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.683+0000 7f6c52269700 1 -- 172.21.15.16:0/2385770827 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c4c06bc10 con 0x7f6c4c161000 2023-05-20T01:16:48.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.684+0000 7fc62a0da700 1 Processor -- start 2023-05-20T01:16:48.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.684+0000 7fc62a0da700 1 -- start start 2023-05-20T01:16:48.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.684+0000 7fa6617fa700 1 -- 172.21.15.16:0/3796166182 <== mon.1 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fa654053870 con 0x7fa664160f30 2023-05-20T01:16:48.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.685+0000 7fc62a0da700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc6240b0e50 0x7fc62415f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:48.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.685+0000 7fc62a0da700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc6240ce740 0x7fc6241ec7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:48.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.685+0000 7fc62a0da700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc624161000 0x7fc6241eccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:48.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.685+0000 7fc62a0da700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6241f72e0 con 0x7fc624161000 2023-05-20T01:16:48.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.685+0000 7fc62a0da700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6240b3690 con 0x7fc6240ce740 2023-05-20T01:16:48.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.685+0000 7fc62a0da700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6240c75f0 con 0x7fc6240b0e50 2023-05-20T01:16:48.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.685+0000 7fc6237fe700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc6240b0e50 0x7fc62415f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:48.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.685+0000 7fc6237fe700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc6240b0e50 0x7fc62415f510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.16:58284/0 (socket says 172.21.15.16:58284) 2023-05-20T01:16:48.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.685+0000 7fc6237fe700 1 -- 172.21.15.16:0/1131580079 learned_addr learned my addr 172.21.15.16:0/1131580079 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:48.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.684+0000 7fdc0affd700 1 -- 172.21.15.16:0/1193055875 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc1400aaa0 con 0x7fdc1c0caf60 2023-05-20T01:16:48.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.684+0000 7fdc0affd700 1 -- 172.21.15.16:0/1193055875 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc1400ac00 con 0x7fdc1c0caf60 2023-05-20T01:16:48.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.684+0000 7fdc0affd700 1 -- 172.21.15.16:0/1193055875 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc14003a20 con 0x7fdc1c0caf60 2023-05-20T01:16:48.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.684+0000 7fdc2090a700 1 -- 172.21.15.16:0/1193055875 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc1c0ce620 con 0x7fdc1c0caf60 2023-05-20T01:16:48.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.684+0000 7fdc2090a700 1 -- 172.21.15.16:0/1193055875 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc1c0ce340 con 0x7fdc1c0caf60 2023-05-20T01:16:48.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.684+0000 7eff4f7fe700 1 -- 172.21.15.16:0/4015206694 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7eff38003a70 msgr2=0x7eff38003e60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-05-20T01:16:48.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.684+0000 7eff4f7fe700 1 --2- 172.21.15.16:0/4015206694 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7eff38003a70 0x7eff38003e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:48.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.684+0000 7eff4f7fe700 1 -- 172.21.15.16:0/4015206694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff38005200 msgr2=0x7eff38014610 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-05-20T01:16:48.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.684+0000 7eff4f7fe700 1 --2- 172.21.15.16:0/4015206694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff38005200 0x7eff38014610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:48.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.684+0000 7eff4f7fe700 1 -- 172.21.15.16:0/4015206694 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff5015b4c0 con 0x7eff38004850 2023-05-20T01:16:48.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.685+0000 7eff4f7fe700 1 --2- 172.21.15.16:0/4015206694 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7eff38004850 0x7eff38004cc0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7eff500cb520 tx=0x7eff38005880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7446f223eaa34c6 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:48.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.683+0000 7f96c3791700 1 -- start start 2023-05-20T01:16:48.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.684+0000 7f96c3791700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f96bc0ccec0 0x7f96bc0713c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:48.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.684+0000 7f96c3791700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96bc0cb590 con 0x7f96bc071900 2023-05-20T01:16:48.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.684+0000 7f96c3791700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96bc0ca9a0 con 0x7f96bc157190 2023-05-20T01:16:48.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.684+0000 7f96c3791700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96bc0cac60 con 0x7f96bc0ccec0 2023-05-20T01:16:48.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.684+0000 7f96c1d2e700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f96bc071900 0x7f96bc0ce740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:41532/0 (socket says 172.21.15.16:41532) 2023-05-20T01:16:48.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.684+0000 7f96c1d2e700 1 -- 172.21.15.16:0/2725397243 learned_addr learned my addr 172.21.15.16:0/2725397243 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:16:48.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.685+0000 7f96affff700 1 -- 172.21.15.16:0/2725397243 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3393962514 0 0) 0x7f96bc0cb590 con 0x7f96bc071900 2023-05-20T01:16:48.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.685+0000 7f96affff700 1 -- 172.21.15.16:0/2725397243 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f96a80035b0 con 0x7f96bc071900 2023-05-20T01:16:48.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.685+0000 7f96affff700 1 -- 172.21.15.16:0/2725397243 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1676704561 0 0) 0x7f96a80035b0 con 0x7f96bc071900 2023-05-20T01:16:48.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.685+0000 7f96affff700 1 -- 172.21.15.16:0/2725397243 >> v2:172.21.15.16:3300/0 conn(0x7f96bc0ccec0 msgr2=0x7f96bc0713c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:48.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.685+0000 7f96affff700 1 --2- 172.21.15.16:0/2725397243 >> v2:172.21.15.16:3300/0 conn(0x7f96bc0ccec0 0x7f96bc0713c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:48.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.685+0000 7f96affff700 1 -- 172.21.15.16:0/2725397243 >> v1:172.21.15.132:6789/0 conn(0x7f96bc157190 legacy=0x7f96bc0cc7b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-05-20T01:16:48.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.685+0000 7f96affff700 1 -- 172.21.15.16:0/2725397243 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96bc0caf40 con 0x7f96bc071900 2023-05-20T01:16:48.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.685+0000 7f96affff700 1 -- 172.21.15.16:0/2725397243 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7f96b8001670 con 0x7f96bc071900 2023-05-20T01:16:48.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.685+0000 7f96affff700 1 -- 172.21.15.16:0/2725397243 >> v1:172.21.15.16:6789/0 conn(0x7f96bc071900 legacy=0x7f96bc0ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:48.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.685+0000 7f96affff700 1 --2- 172.21.15.16:0/2725397243 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f96a8003a70 0x7f96a8003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:48.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.685+0000 7f96affff700 1 --2- 172.21.15.16:0/2725397243 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f96a8004850 0x7f96a8004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:48.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.685+0000 7f96affff700 1 --2- 172.21.15.16:0/2725397243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96a8005400 0x7f96a800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:48.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.685+0000 7f96affff700 1 -- 172.21.15.16:0/2725397243 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96bc0cac60 con 0x7f96a8005400 2023-05-20T01:16:48.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.685+0000 7f96affff700 1 -- 172.21.15.16:0/2725397243 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96a802e300 con 0x7f96a8004850 2023-05-20T01:16:48.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.685+0000 7f96affff700 1 -- 172.21.15.16:0/2725397243 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96a802e4c0 con 0x7f96a8003a70 2023-05-20T01:16:48.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.686+0000 7fdc2090a700 1 -- 172.21.15.16:0/1193055875 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc000052e0 con 0x7fdc1c0caf60 2023-05-20T01:16:48.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.685+0000 7f6c52269700 1 -- 172.21.15.16:0/2385770827 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c4c06c1d0 con 0x7f6c4c161000 2023-05-20T01:16:48.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.686+0000 7f96c0d2c700 1 --2- 172.21.15.16:0/2725397243 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f96a8004850 0x7f96a8004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:48.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.686+0000 7fc622ffd700 1 --2- 172.21.15.16:0/1131580079 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc6240ce740 0x7fc6241ec7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:48.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.686+0000 7fc623fff700 1 --2- 172.21.15.16:0/1131580079 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc624161000 0x7fc6241eccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:48.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.685+0000 7eff4effd700 1 -- 172.21.15.16:0/4015206694 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff4400cd70 con 0x7eff38004850 2023-05-20T01:16:48.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.685+0000 7eff54df3700 1 -- 172.21.15.16:0/4015206694 reap_dead start 2023-05-20T01:16:48.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.685+0000 7eff4effd700 1 -- 172.21.15.16:0/4015206694 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff44006860 con 0x7eff38004850 2023-05-20T01:16:48.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.685+0000 7eff4effd700 1 -- 172.21.15.16:0/4015206694 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff44006c10 con 0x7eff38004850 2023-05-20T01:16:48.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.686+0000 7eff56856700 1 -- 172.21.15.16:0/4015206694 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7eff38004850 msgr2=0x7eff38004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:48.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.686+0000 7eff56856700 1 --2- 172.21.15.16:0/4015206694 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7eff38004850 0x7eff38004cc0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7eff500cb520 tx=0x7eff38005880 comp rx=0 tx=0).stop 2023-05-20T01:16:48.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.686+0000 7eff56856700 1 -- 172.21.15.16:0/4015206694 shutdown_connections 2023-05-20T01:16:48.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.686+0000 7eff56856700 1 --2- 172.21.15.16:0/4015206694 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7eff38004850 0x7eff38004cc0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:48.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.686+0000 7eff56856700 1 -- 172.21.15.16:0/4015206694 >> 172.21.15.16:0/4015206694 conn(0x7eff500cb910 msgr2=0x7eff500cbd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:48.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.686+0000 7f96c0d2c700 1 -- 172.21.15.16:0/2725397243 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f96a8003a70 msgr2=0x7f96a8003e60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-05-20T01:16:48.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.686+0000 7f96c0d2c700 1 --2- 172.21.15.16:0/2725397243 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f96a8003a70 0x7f96a8003e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:48.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.686+0000 7f96c0d2c700 1 -- 172.21.15.16:0/2725397243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96a8005400 msgr2=0x7f96a800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:48.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.686+0000 7f96c0d2c700 1 --2- 172.21.15.16:0/2725397243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96a8005400 0x7f96a800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:48.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.686+0000 7f96c0d2c700 1 -- 172.21.15.16:0/2725397243 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96bc0caf40 con 0x7f96a8004850 2023-05-20T01:16:48.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.686+0000 7fc6237fe700 1 -- 172.21.15.16:0/1131580079 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc6240ce740 msgr2=0x7fc6241ec7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:48.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.686+0000 7fc6237fe700 1 --2- 172.21.15.16:0/1131580079 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc6240ce740 0x7fc6241ec7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:48.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.686+0000 7fc6237fe700 1 -- 172.21.15.16:0/1131580079 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc624161000 msgr2=0x7fc6241eccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:48.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.686+0000 7fc6237fe700 1 --2- 172.21.15.16:0/1131580079 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc624161000 0x7fc6241eccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:48.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.686+0000 7fc6237fe700 1 -- 172.21.15.16:0/1131580079 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6140094a0 con 0x7fc6240b0e50 2023-05-20T01:16:48.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.686+0000 7f96c0d2c700 1 --2- 172.21.15.16:0/2725397243 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f96a8004850 0x7f96a8004cc0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f96b0009e40 tx=0x7f96a8005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2b57a46860b36f9 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:48.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.687+0000 7f96c1d2e700 1 -- 172.21.15.16:0/2725397243 reap_dead start 2023-05-20T01:16:48.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.687+0000 7fc6237fe700 1 --2- 172.21.15.16:0/1131580079 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc6240b0e50 0x7fc62415f510 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc614014040 tx=0x7fc614004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:48.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.687+0000 7fc60bfff700 1 -- 172.21.15.16:0/1131580079 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc614004510 con 0x7fc6240b0e50 2023-05-20T01:16:48.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.687+0000 7fc60bfff700 1 -- 172.21.15.16:0/1131580079 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc61401f070 con 0x7fc6240b0e50 2023-05-20T01:16:48.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.687+0000 7fc60bfff700 1 -- 172.21.15.16:0/1131580079 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6140038c0 con 0x7fc6240b0e50 2023-05-20T01:16:48.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.687+0000 7fc62a0da700 1 -- 172.21.15.16:0/1131580079 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc62406bef0 con 0x7fc6240b0e50 2023-05-20T01:16:48.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.687+0000 7fc62a0da700 1 -- 172.21.15.16:0/1131580079 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc62406bc10 con 0x7fc6240b0e50 2023-05-20T01:16:48.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.688+0000 7fdc0affd700 1 -- 172.21.15.16:0/1193055875 <== mon.1 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fdc14022480 con 0x7fdc1c0caf60 2023-05-20T01:16:48.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.688+0000 7fc62a0da700 1 -- 172.21.15.16:0/1131580079 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6040052e0 con 0x7fc6240b0e50 2023-05-20T01:16:48.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.688+0000 7f6c48ff9700 1 -- 172.21.15.16:0/2385770827 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6c44007740 con 0x7f6c4c161000 2023-05-20T01:16:48.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.688+0000 7f6c48ff9700 1 --2- 172.21.15.16:0/2385770827 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f6c3806adb0 0x7f6c3806d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:48.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.688+0000 7f6c48ff9700 1 -- 172.21.15.16:0/2385770827 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f6c44089cc0 con 0x7f6c4c161000 2023-05-20T01:16:48.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.689+0000 7f6c4bfff700 1 --2- 172.21.15.16:0/2385770827 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f6c3806adb0 0x7f6c3806d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:48.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.689+0000 7f5467be3700 1 Processor -- start 2023-05-20T01:16:48.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.689+0000 7fdc0affd700 1 --2- 172.21.15.16:0/1193055875 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fdbfc06adb0 0x7fdbfc06d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:48.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.689+0000 7fdc0affd700 1 -- 172.21.15.16:0/1193055875 <== mon.1 v2:172.21.15.132:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fdc14089cb0 con 0x7fdc1c0caf60 2023-05-20T01:16:48.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.689+0000 7f6c4bfff700 1 --2- 172.21.15.16:0/2385770827 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f6c3806adb0 0x7f6c3806d260 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c0cb280 tx=0x7f6c40006c10 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:48.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.689+0000 7fdc1a59c700 1 --2- 172.21.15.16:0/1193055875 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fdbfc06adb0 0x7fdbfc06d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:48.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.690+0000 7fdc1a59c700 1 --2- 172.21.15.16:0/1193055875 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fdbfc06adb0 0x7fdbfc06d260 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c071730 tx=0x7fdc0c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:48.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.690+0000 7fc60bfff700 1 -- 172.21.15.16:0/1131580079 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc614003a20 con 0x7fc6240b0e50 2023-05-20T01:16:48.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.690+0000 7f5467be3700 1 -- start start 2023-05-20T01:16:48.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.690+0000 7f5467be3700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f5460161000 0x7f54600caa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:48.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.690+0000 7f5467be3700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f54600caf60 0x7f5460071a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:48.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.691+0000 7f5467be3700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5460072400 con 0x7f5460071fa0 2023-05-20T01:16:48.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.691+0000 7f5467be3700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54600ce0f0 con 0x7f5460161000 2023-05-20T01:16:48.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.691+0000 7f5467be3700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54600ce990 con 0x7f54600caf60 2023-05-20T01:16:48.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.691+0000 7f546597f700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f5460161000 0x7f54600caa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:48.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.691+0000 7f5466180700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f5460071fa0 0x7f546006ba70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:41552/0 (socket says 172.21.15.16:41552) 2023-05-20T01:16:48.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.691+0000 7f5466180700 1 -- 172.21.15.16:0/1143739444 learned_addr learned my addr 172.21.15.16:0/1143739444 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:16:48.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.691+0000 7fc60bfff700 1 --2- 172.21.15.16:0/1131580079 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc60c06ae00 0x7fc60c06d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:48.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.691+0000 7fc60bfff700 1 -- 172.21.15.16:0/1131580079 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fc614021070 con 0x7fc6240b0e50 2023-05-20T01:16:48.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.691+0000 7f546517e700 1 --2- 172.21.15.16:0/1143739444 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f54600caf60 0x7f5460071a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:48.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.691+0000 7fc622ffd700 1 --2- 172.21.15.16:0/1131580079 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc60c06ae00 0x7fc60c06d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:48.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.691+0000 7f546597f700 1 -- 172.21.15.16:0/1143739444 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f54600caf60 msgr2=0x7f5460071a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:48.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.691+0000 7f546597f700 1 --2- 172.21.15.16:0/1143739444 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f54600caf60 0x7f5460071a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:48.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.691+0000 7f546597f700 1 -- 172.21.15.16:0/1143739444 >> v1:172.21.15.16:6789/0 conn(0x7f5460071fa0 legacy=0x7f546006ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:48.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.691+0000 7f546597f700 1 -- 172.21.15.16:0/1143739444 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54600cb500 con 0x7f5460161000 2023-05-20T01:16:48.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.691+0000 7f546597f700 1 --2- 172.21.15.16:0/1143739444 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5460161000 0x7f54600caa20 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5450009840 tx=0x7f5450009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d294a684f0754e server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:48.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.691+0000 7f96affff700 1 -- 172.21.15.16:0/2725397243 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b000baa0 con 0x7f96a8004850 2023-05-20T01:16:48.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.692+0000 7f546497d700 1 -- 172.21.15.16:0/1143739444 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5450011830 con 0x7f5460161000 2023-05-20T01:16:48.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.692+0000 7f4747196700 1 Processor -- start 2023-05-20T01:16:48.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.692+0000 7fdc0affd700 1 -- 172.21.15.16:0/1193055875 <== mon.1 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fdc14054ae0 con 0x7fdc1c0caf60 2023-05-20T01:16:48.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.692+0000 7f546497d700 1 -- 172.21.15.16:0/1143739444 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5450011990 con 0x7f5460161000 2023-05-20T01:16:48.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.692+0000 7f546497d700 1 -- 172.21.15.16:0/1143739444 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5450011d20 con 0x7f5460161000 2023-05-20T01:16:48.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.692+0000 7fc622ffd700 1 --2- 172.21.15.16:0/1131580079 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc60c06ae00 0x7fc60c06d2b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc6240c7300 tx=0x7fc618009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:48.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.693+0000 7f5467be3700 1 -- 172.21.15.16:0/1143739444 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5460161000 msgr2=0x7f54600caa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:48.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.693+0000 7f5467be3700 1 --2- 172.21.15.16:0/1143739444 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5460161000 0x7f54600caa20 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5450009840 tx=0x7f5450009b50 comp rx=0 tx=0).stop 2023-05-20T01:16:48.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.691+0000 7f96affff700 1 -- 172.21.15.16:0/2725397243 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96b000bc00 con 0x7f96a8004850 2023-05-20T01:16:48.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.691+0000 7f96affff700 1 -- 172.21.15.16:0/2725397243 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b0015420 con 0x7f96a8004850 2023-05-20T01:16:48.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.692+0000 7f96c3791700 1 -- 172.21.15.16:0/2725397243 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f96a8004850 msgr2=0x7f96a8004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:48.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.692+0000 7f96c3791700 1 --2- 172.21.15.16:0/2725397243 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f96a8004850 0x7f96a8004cc0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f96b0009e40 tx=0x7f96a8005880 comp rx=0 tx=0).stop 2023-05-20T01:16:48.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.693+0000 7f96c3791700 1 -- 172.21.15.16:0/2725397243 shutdown_connections 2023-05-20T01:16:48.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.693+0000 7f96c3791700 1 --2- 172.21.15.16:0/2725397243 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f96a8004850 0x7f96a8004cc0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:48.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.693+0000 7f96c3791700 1 -- 172.21.15.16:0/2725397243 >> 172.21.15.16:0/2725397243 conn(0x7f96bc0cffb0 msgr2=0x7f96bc0d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:48.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.692+0000 7f4747196700 1 -- start start 2023-05-20T01:16:48.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.693+0000 7f4747196700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4740161000 0x7f47400caa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:48.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.693+0000 7f4747196700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f47400caf60 0x7f4740071a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:48.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.693+0000 7f4747196700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f4740071fa0 0x7f474006ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:48.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.693+0000 7f4747196700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4740072400 con 0x7f4740161000 2023-05-20T01:16:48.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.693+0000 7f4747196700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47400ce0f0 con 0x7f4740071fa0 2023-05-20T01:16:48.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.693+0000 7f4747196700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47400ce990 con 0x7f47400caf60 2023-05-20T01:16:48.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.693+0000 7f4744f32700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4740161000 0x7f47400caa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:48.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.693+0000 7f473ffff700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f47400caf60 0x7f4740071a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:48.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.694+0000 7f4744f32700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4740161000 0x7f47400caa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58262/0 (socket says 172.21.15.16:58262) 2023-05-20T01:16:48.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.694+0000 7f4744f32700 1 -- 172.21.15.16:0/1787554146 learned_addr learned my addr 172.21.15.16:0/1787554146 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:48.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.694+0000 7f4744f32700 1 -- 172.21.15.16:0/1787554146 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f47400caf60 msgr2=0x7f4740071a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:48.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.694+0000 7f4744f32700 1 --2- 172.21.15.16:0/1787554146 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f47400caf60 0x7f4740071a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:48.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.694+0000 7f4744f32700 1 -- 172.21.15.16:0/1787554146 >> v2:172.21.15.132:3300/0 conn(0x7f4740071fa0 msgr2=0x7f474006ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:48.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.694+0000 7f4744f32700 1 --2- 172.21.15.16:0/1787554146 >> v2:172.21.15.132:3300/0 conn(0x7f4740071fa0 0x7f474006ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:48.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.694+0000 7f4744f32700 1 -- 172.21.15.16:0/1787554146 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47400cb500 con 0x7f4740161000 2023-05-20T01:16:48.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.694+0000 7f4744f32700 1 --2- 172.21.15.16:0/1787554146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4740161000 0x7f47400caa20 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f474006fa60 tx=0x7f4734009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=458fc81f2ef903e9 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:48.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.692+0000 7eff56856700 1 -- 172.21.15.16:0/4015206694 shutdown_connections 2023-05-20T01:16:48.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.693+0000 7eff56856700 1 -- 172.21.15.16:0/4015206694 wait complete. 2023-05-20T01:16:48.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.694+0000 7eff56856700 1 Processor -- start 2023-05-20T01:16:48.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.694+0000 7f473f7fe700 1 -- 172.21.15.16:0/1787554146 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4734011830 con 0x7f4740161000 2023-05-20T01:16:48.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.694+0000 7f473f7fe700 1 -- 172.21.15.16:0/1787554146 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4734011990 con 0x7f4740161000 2023-05-20T01:16:48.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.694+0000 7f473f7fe700 1 -- 172.21.15.16:0/1787554146 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4734011d20 con 0x7f4740161000 2023-05-20T01:16:48.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.695+0000 7f6c48ff9700 1 -- 172.21.15.16:0/2385770827 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f6c44054a60 con 0x7f6c4c161000 2023-05-20T01:16:48.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.695+0000 7eff56856700 1 -- start start 2023-05-20T01:16:48.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.695+0000 7f4747196700 1 -- 172.21.15.16:0/1787554146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4740161000 msgr2=0x7f47400caa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:48.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.695+0000 7f4747196700 1 --2- 172.21.15.16:0/1787554146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4740161000 0x7f47400caa20 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f474006fa60 tx=0x7f4734009bd0 comp rx=0 tx=0).stop 2023-05-20T01:16:48.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.695+0000 7f4747196700 1 -- 172.21.15.16:0/1787554146 shutdown_connections 2023-05-20T01:16:48.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.695+0000 7f4747196700 1 --2- 172.21.15.16:0/1787554146 >> v2:172.21.15.132:3300/0 conn(0x7f4740071fa0 0x7f474006ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:48.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.695+0000 7f4747196700 1 --2- 172.21.15.16:0/1787554146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4740161000 0x7f47400caa20 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:48.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.695+0000 7f4747196700 1 --2- 172.21.15.16:0/1787554146 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f47400caf60 0x7f4740071a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:48.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.695+0000 7f4747196700 1 -- 172.21.15.16:0/1787554146 >> 172.21.15.16:0/1787554146 conn(0x7f47400cb910 msgr2=0x7f47400cbd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:48.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.695+0000 7f4747196700 1 -- 172.21.15.16:0/1787554146 shutdown_connections 2023-05-20T01:16:48.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.696+0000 7f4747196700 1 -- 172.21.15.16:0/1787554146 wait complete. 2023-05-20T01:16:48.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.696+0000 7eff56856700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7eff38004850 0x7eff501585a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:48.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.696+0000 7eff56856700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff50159db0 0x7eff500b2850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:48.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.696+0000 7eff56856700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7eff500b2d90 0x7eff50150de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:48.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.696+0000 7eff56856700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff500b3210 con 0x7eff50159db0 2023-05-20T01:16:48.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.696+0000 7eff56856700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff50158ae0 con 0x7eff38004850 2023-05-20T01:16:48.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.696+0000 7eff56856700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff500b4c70 con 0x7eff500b2d90 2023-05-20T01:16:48.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.696+0000 7eff4f7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff50159db0 0x7eff500b2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:48.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.696+0000 7eff4f7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff50159db0 0x7eff500b2850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58268/0 (socket says 172.21.15.16:58268) 2023-05-20T01:16:48.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.696+0000 7eff4f7fe700 1 -- 172.21.15.16:0/1405918066 learned_addr learned my addr 172.21.15.16:0/1405918066 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:48.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.696+0000 7eff54df3700 1 --2- 172.21.15.16:0/1405918066 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7eff500b2d90 0x7eff50150de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:48.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.696+0000 7eff4ffff700 1 --2- 172.21.15.16:0/1405918066 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7eff38004850 0x7eff501585a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:48.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.697+0000 7fc60bfff700 1 -- 172.21.15.16:0/1131580079 <== mon.2 v2:172.21.15.132:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fc61408f060 con 0x7fc6240b0e50 2023-05-20T01:16:48.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.697+0000 7eff54df3700 1 -- 172.21.15.16:0/1405918066 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7eff38004850 msgr2=0x7eff501585a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:48.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.697+0000 7eff54df3700 1 --2- 172.21.15.16:0/1405918066 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7eff38004850 0x7eff501585a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:48.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.697+0000 7eff54df3700 1 -- 172.21.15.16:0/1405918066 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff50159db0 msgr2=0x7eff500b2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:48.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.697+0000 7eff54df3700 1 --2- 172.21.15.16:0/1405918066 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff50159db0 0x7eff500b2850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:48.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.697+0000 7eff54df3700 1 -- 172.21.15.16:0/1405918066 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff500cb5f0 con 0x7eff500b2d90 2023-05-20T01:16:48.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.697+0000 7eff54df3700 1 --2- 172.21.15.16:0/1405918066 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7eff500b2d90 0x7eff50150de0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7eff500ced20 tx=0x7eff4000c670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:48.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.697+0000 7eff4cff9700 1 -- 172.21.15.16:0/1405918066 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff40005960 con 0x7eff500b2d90 2023-05-20T01:16:48.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.697+0000 7eff4cff9700 1 -- 172.21.15.16:0/1405918066 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff40005ac0 con 0x7eff500b2d90 2023-05-20T01:16:48.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.697+0000 7eff4cff9700 1 -- 172.21.15.16:0/1405918066 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff400070d0 con 0x7eff500b2d90 2023-05-20T01:16:48.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.698+0000 7f4747196700 1 Processor -- start 2023-05-20T01:16:48.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.698+0000 7f4747196700 1 -- start start 2023-05-20T01:16:48.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.699+0000 7f96c3791700 1 -- 172.21.15.16:0/2725397243 shutdown_connections 2023-05-20T01:16:48.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.700+0000 7f4747196700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4740071fa0 0x7f47400c1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:48.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.700+0000 7f4747196700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f47400caf60 0x7f47400be910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:48.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.700+0000 7f4747196700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4740161000 0x7f47400ae790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:48.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.700+0000 7f4747196700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47400c83c0 con 0x7f4740071fa0 2023-05-20T01:16:48.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.700+0000 7f4747196700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47400c86a0 con 0x7f47400caf60 2023-05-20T01:16:48.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.700+0000 7f4747196700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47400c8980 con 0x7f4740161000 2023-05-20T01:16:48.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.700+0000 7f4744f32700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4740071fa0 0x7f47400c1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:48.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.700+0000 7f4744f32700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4740071fa0 0x7f47400c1ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58274/0 (socket says 172.21.15.16:58274) 2023-05-20T01:16:48.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.700+0000 7f4744f32700 1 -- 172.21.15.16:0/1875492496 learned_addr learned my addr 172.21.15.16:0/1875492496 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:48.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.700+0000 7f473ffff700 1 --2- 172.21.15.16:0/1875492496 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f47400caf60 0x7f47400be910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:48.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.700+0000 7f4744f32700 1 -- 172.21.15.16:0/1875492496 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4740161000 msgr2=0x7f47400ae790 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-05-20T01:16:48.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.701+0000 7f4744f32700 1 --2- 172.21.15.16:0/1875492496 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4740161000 0x7f47400ae790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:48.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.701+0000 7f4744f32700 1 -- 172.21.15.16:0/1875492496 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f47400caf60 msgr2=0x7f47400be910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:48.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.701+0000 7f4744f32700 1 --2- 172.21.15.16:0/1875492496 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f47400caf60 0x7f47400be910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:48.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.701+0000 7f4744f32700 1 -- 172.21.15.16:0/1875492496 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4734009550 con 0x7f4740071fa0 2023-05-20T01:16:48.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.701+0000 7f473ffff700 1 --2- 172.21.15.16:0/1875492496 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f47400caf60 0x7f47400be910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-05-20T01:16:48.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.701+0000 7f4744f32700 1 --2- 172.21.15.16:0/1875492496 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4740071fa0 0x7f47400c1ab0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4734000c00 tx=0x7f4734004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:48.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.701+0000 7f473d7fa700 1 -- 172.21.15.16:0/1875492496 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47340044d0 con 0x7f4740071fa0 2023-05-20T01:16:48.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.701+0000 7f4747196700 1 -- 172.21.15.16:0/1875492496 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47400ce620 con 0x7f4740071fa0 2023-05-20T01:16:48.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.701+0000 7f4747196700 1 -- 172.21.15.16:0/1875492496 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47400ce340 con 0x7f4740071fa0 2023-05-20T01:16:48.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.701+0000 7f473d7fa700 1 -- 172.21.15.16:0/1875492496 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f473401e070 con 0x7f4740071fa0 2023-05-20T01:16:48.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.701+0000 7f473d7fa700 1 -- 172.21.15.16:0/1875492496 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47340038a0 con 0x7f4740071fa0 2023-05-20T01:16:48.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.702+0000 7f472affd700 1 -- 172.21.15.16:0/1875492496 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47200052e0 con 0x7f4740071fa0 2023-05-20T01:16:48.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.703+0000 7f473d7fa700 1 -- 172.21.15.16:0/1875492496 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4734003a00 con 0x7f4740071fa0 2023-05-20T01:16:48.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.704+0000 7f473d7fa700 1 --2- 172.21.15.16:0/1875492496 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f472406ace0 0x7f472406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:48.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.704+0000 7f473d7fa700 1 -- 172.21.15.16:0/1875492496 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f473408a760 con 0x7f4740071fa0 2023-05-20T01:16:48.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.704+0000 7f473ffff700 1 --2- 172.21.15.16:0/1875492496 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f472406ace0 0x7f472406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:48.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.704+0000 7f473ffff700 1 --2- 172.21.15.16:0/1875492496 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f472406ace0 0x7f472406d190 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f474016cc80 tx=0x7f472c009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:48.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.707+0000 7f96c3791700 1 -- 172.21.15.16:0/2725397243 wait complete. 2023-05-20T01:16:48.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.707+0000 7eff56856700 1 -- 172.21.15.16:0/1405918066 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff5015b7a0 con 0x7eff500b2d90 2023-05-20T01:16:48.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.707+0000 7eff56856700 1 -- 172.21.15.16:0/1405918066 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff5015bcc0 con 0x7eff500b2d90 2023-05-20T01:16:48.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.699+0000 7f5467be3700 1 -- 172.21.15.16:0/1143739444 shutdown_connections 2023-05-20T01:16:48.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.699+0000 7f5467be3700 1 --2- 172.21.15.16:0/1143739444 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5460161000 0x7f54600caa20 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:48.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.699+0000 7f5467be3700 1 --2- 172.21.15.16:0/1143739444 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f54600caf60 0x7f5460071a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:48.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.699+0000 7f5467be3700 1 -- 172.21.15.16:0/1143739444 >> 172.21.15.16:0/1143739444 conn(0x7f54600cb910 msgr2=0x7f54600cbd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:48.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.707+0000 7f5467be3700 1 -- 172.21.15.16:0/1143739444 shutdown_connections 2023-05-20T01:16:48.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.708+0000 7eff56856700 1 -- 172.21.15.16:0/1405918066 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff50157c80 con 0x7eff500b2d90 2023-05-20T01:16:48.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.709+0000 7f96c3791700 1 Processor -- start 2023-05-20T01:16:48.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.709+0000 7eff4cff9700 1 -- 172.21.15.16:0/1405918066 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7eff40007230 con 0x7eff500b2d90 2023-05-20T01:16:48.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.709+0000 7eff4cff9700 1 --2- 172.21.15.16:0/1405918066 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7eff3c06ace0 0x7eff3c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:48.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.710+0000 7eff4cff9700 1 -- 172.21.15.16:0/1405918066 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7eff40088790 con 0x7eff500b2d90 2023-05-20T01:16:48.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.710+0000 7eff4ffff700 1 --2- 172.21.15.16:0/1405918066 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7eff3c06ace0 0x7eff3c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:48.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.709+0000 7f96c3791700 1 -- start start 2023-05-20T01:16:48.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.710+0000 7f96c3791700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96a8004850 0x7f96bc0c6510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:48.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.710+0000 7f96c3791700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f96bc0b2e40 0x7f96bc147f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:48.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.710+0000 7f96c3791700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f96bc1e8ea0 0x7f96bc1e9310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:48.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.710+0000 7f96c3791700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96bc0c6a50 con 0x7f96a8004850 2023-05-20T01:16:48.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.710+0000 7f96c3791700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96bc08a3e0 con 0x7f96bc1e8ea0 2023-05-20T01:16:48.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.710+0000 7f96c3791700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96bc08a6c0 con 0x7f96bc0b2e40 2023-05-20T01:16:48.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.710+0000 7eff4ffff700 1 --2- 172.21.15.16:0/1405918066 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7eff3c06ace0 0x7eff3c06d190 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7eff500cec10 tx=0x7eff4800d040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:48.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.710+0000 7f96c0d2c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f96bc0b2e40 0x7f96bc147f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:48.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.710+0000 7f96c1d2e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f96bc1e8ea0 0x7f96bc1e9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:48.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.710+0000 7f96c0d2c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f96bc0b2e40 0x7f96bc147f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.16:58324/0 (socket says 172.21.15.16:58324) 2023-05-20T01:16:48.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.710+0000 7f96c0d2c700 1 -- 172.21.15.16:0/752514872 learned_addr learned my addr 172.21.15.16:0/752514872 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:48.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.711+0000 7f96c0d2c700 1 -- 172.21.15.16:0/752514872 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f96bc1e8ea0 msgr2=0x7f96bc1e9310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:48.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.711+0000 7f96c0d2c700 1 --2- 172.21.15.16:0/752514872 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f96bc1e8ea0 0x7f96bc1e9310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:48.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.711+0000 7f96c0d2c700 1 -- 172.21.15.16:0/752514872 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96a8004850 msgr2=0x7f96bc0c6510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:48.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.711+0000 7f96c0d2c700 1 --2- 172.21.15.16:0/752514872 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96a8004850 0x7f96bc0c6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:48.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.711+0000 7f96c0d2c700 1 -- 172.21.15.16:0/752514872 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96bc157560 con 0x7f96bc0b2e40 2023-05-20T01:16:48.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.711+0000 7f96c0d2c700 1 --2- 172.21.15.16:0/752514872 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f96bc0b2e40 0x7f96bc147f50 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f96b0017040 tx=0x7f96b0004be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:48.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.712+0000 7f5467be3700 1 -- 172.21.15.16:0/1143739444 wait complete. 2023-05-20T01:16:48.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.712+0000 7f473d7fa700 1 -- 172.21.15.16:0/1875492496 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f4734055530 con 0x7f4740071fa0 2023-05-20T01:16:48.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.713+0000 7f96adffb700 1 -- 172.21.15.16:0/752514872 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b000baa0 con 0x7f96bc0b2e40 2023-05-20T01:16:48.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.713+0000 7f96adffb700 1 -- 172.21.15.16:0/752514872 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96b001e420 con 0x7f96bc0b2e40 2023-05-20T01:16:48.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.713+0000 7f96c3791700 1 -- 172.21.15.16:0/752514872 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96bc0cb220 con 0x7f96bc0b2e40 2023-05-20T01:16:48.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.713+0000 7f96c3791700 1 -- 172.21.15.16:0/752514872 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96bc0cba20 con 0x7f96bc0b2e40 2023-05-20T01:16:48.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.714+0000 7f96adffb700 1 -- 172.21.15.16:0/752514872 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b001e7d0 con 0x7f96bc0b2e40 2023-05-20T01:16:48.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.716+0000 7f96adffb700 1 -- 172.21.15.16:0/752514872 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f96b001e9f0 con 0x7f96bc0b2e40 2023-05-20T01:16:48.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.716+0000 7f96adffb700 1 --2- 172.21.15.16:0/752514872 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f96a006acc0 0x7f96a006d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:48.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.717+0000 7f96adffb700 1 -- 172.21.15.16:0/752514872 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f96b0011070 con 0x7f96bc0b2e40 2023-05-20T01:16:48.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.717+0000 7f96c152d700 1 --2- 172.21.15.16:0/752514872 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f96a006acc0 0x7f96a006d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:48.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.717+0000 7f96c152d700 1 --2- 172.21.15.16:0/752514872 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f96a006acc0 0x7f96a006d170 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f96bc0c5f20 tx=0x7f96b4006c80 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:48.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.714+0000 7f5467be3700 1 Processor -- start 2023-05-20T01:16:48.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.718+0000 7f5467be3700 1 -- start start 2023-05-20T01:16:48.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.719+0000 7f5467be3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5460071fa0 0x7f546014d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:48.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.719+0000 7f5467be3700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f54600caf60 0x7f54600aff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:48.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.719+0000 7f5467be3700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5460161000 0x7f5460149dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:48.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.719+0000 7f5467be3700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54600b3b90 con 0x7f5460071fa0 2023-05-20T01:16:48.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.719+0000 7f5467be3700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f546014da90 con 0x7f5460161000 2023-05-20T01:16:48.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.719+0000 7f5467be3700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54601fa5a0 con 0x7f54600caf60 2023-05-20T01:16:48.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.719+0000 7f546597f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5460071fa0 0x7f546014d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:48.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.719+0000 7f546597f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5460071fa0 0x7f546014d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58292/0 (socket says 172.21.15.16:58292) 2023-05-20T01:16:48.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.719+0000 7f546597f700 1 -- 172.21.15.16:0/3295818194 learned_addr learned my addr 172.21.15.16:0/3295818194 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:48.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.719+0000 7f5466180700 1 --2- 172.21.15.16:0/3295818194 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5460161000 0x7f5460149dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:48.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.720+0000 7f546597f700 1 -- 172.21.15.16:0/3295818194 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f54600caf60 msgr2=0x7f54600aff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:48.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.720+0000 7f546597f700 1 --2- 172.21.15.16:0/3295818194 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f54600caf60 0x7f54600aff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:48.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.720+0000 7f546597f700 1 -- 172.21.15.16:0/3295818194 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5460161000 msgr2=0x7f5460149dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:48.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.720+0000 7f546597f700 1 --2- 172.21.15.16:0/3295818194 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5460161000 0x7f5460149dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:48.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.720+0000 7f546597f700 1 -- 172.21.15.16:0/3295818194 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54500094a0 con 0x7f5460071fa0 2023-05-20T01:16:48.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.720+0000 7f546597f700 1 --2- 172.21.15.16:0/3295818194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5460071fa0 0x7f546014d550 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f5450009fd0 tx=0x7f5450004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:48.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.720+0000 7f54567fc700 1 -- 172.21.15.16:0/3295818194 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54500036a0 con 0x7f5460071fa0 2023-05-20T01:16:48.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.720+0000 7f54567fc700 1 -- 172.21.15.16:0/3295818194 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f545001e070 con 0x7f5460071fa0 2023-05-20T01:16:48.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.720+0000 7f54567fc700 1 -- 172.21.15.16:0/3295818194 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5450003a40 con 0x7f5460071fa0 2023-05-20T01:16:48.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.720+0000 7f5467be3700 1 -- 172.21.15.16:0/3295818194 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54600ce620 con 0x7f5460071fa0 2023-05-20T01:16:48.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.720+0000 7f5467be3700 1 -- 172.21.15.16:0/3295818194 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54600ce340 con 0x7f5460071fa0 2023-05-20T01:16:48.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.724+0000 7eff4cff9700 1 -- 172.21.15.16:0/1405918066 <== mon.2 v2:172.21.15.132:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7eff40053630 con 0x7eff500b2d90 2023-05-20T01:16:48.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.724+0000 7f54567fc700 1 -- 172.21.15.16:0/3295818194 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5450003ba0 con 0x7f5460071fa0 2023-05-20T01:16:48.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.724+0000 7f54567fc700 1 --2- 172.21.15.16:0/3295818194 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f544406ace0 0x7f544406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:48.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.725+0000 7f54567fc700 1 -- 172.21.15.16:0/3295818194 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f545008a660 con 0x7f5460071fa0 2023-05-20T01:16:48.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.725+0000 7f546517e700 1 --2- 172.21.15.16:0/3295818194 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f544406ace0 0x7f544406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:48.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.726+0000 7f546517e700 1 --2- 172.21.15.16:0/3295818194 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f544406ace0 0x7f544406d190 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f54601fa360 tx=0x7f545800b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:48.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.727+0000 7f5467be3700 1 -- 172.21.15.16:0/3295818194 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f546006d5e0 con 0x7f5460071fa0 2023-05-20T01:16:48.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.728+0000 7f96c3791700 1 -- 172.21.15.16:0/752514872 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f969c0052e0 con 0x7f96bc0b2e40 2023-05-20T01:16:48.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.734+0000 7f96adffb700 1 -- 172.21.15.16:0/752514872 <== mon.2 v2:172.21.15.132:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f96b0088e60 con 0x7f96bc0b2e40 2023-05-20T01:16:48.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:48.740+0000 7f54567fc700 1 -- 172.21.15.16:0/3295818194 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f5450020080 con 0x7f5460071fa0 2023-05-20T01:16:49.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:49.089+0000 7fdc2090a700 1 -- 172.21.15.16:0/1193055875 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fdc00005860 con 0x7fdc1c0caf60 2023-05-20T01:16:49.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:49.090+0000 7fdc0affd700 1 -- 172.21.15.16:0/1193055875 <== mon.1 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fdc1401d020 con 0x7fdc1c0caf60 2023-05-20T01:16:49.096 INFO:teuthology.orchestra.run.smithi016.stdout:55834574855 2023-05-20T01:16:49.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:49.093+0000 7fdc2090a700 1 -- 172.21.15.16:0/1193055875 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fdbfc06adb0 msgr2=0x7fdbfc06d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:49.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:49.093+0000 7fdc2090a700 1 --2- 172.21.15.16:0/1193055875 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fdbfc06adb0 0x7fdbfc06d260 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c071730 tx=0x7fdc0c008030 comp rx=0 tx=0).stop 2023-05-20T01:16:49.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:49.093+0000 7fdc2090a700 1 -- 172.21.15.16:0/1193055875 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdc1c0caf60 msgr2=0x7fdc1c1f9c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:49.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:49.093+0000 7fdc2090a700 1 --2- 172.21.15.16:0/1193055875 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdc1c0caf60 0x7fdc1c1f9c70 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fdc1400cc10 tx=0x7fdc1400cf20 comp rx=0 tx=0).stop 2023-05-20T01:16:49.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:49.094+0000 7fa66b0cf700 1 -- 172.21.15.16:0/3796166182 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fa644005860 con 0x7fa664160f30 2023-05-20T01:16:49.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:49.094+0000 7fa6617fa700 1 -- 172.21.15.16:0/3796166182 <== mon.1 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa654017020 con 0x7fa664160f30 2023-05-20T01:16:49.099 INFO:teuthology.orchestra.run.smithi016.stdout:55834574855 2023-05-20T01:16:49.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:49.094+0000 7fdc2090a700 1 -- 172.21.15.16:0/1193055875 shutdown_connections 2023-05-20T01:16:49.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:49.094+0000 7fdc2090a700 1 --2- 172.21.15.16:0/1193055875 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fdbfc06adb0 0x7fdbfc06d260 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:49.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:49.094+0000 7fdc2090a700 1 --2- 172.21.15.16:0/1193055875 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdc1c161000 0x7fdc1c0b4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:49.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:49.094+0000 7fdc2090a700 1 --2- 172.21.15.16:0/1193055875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc1c071fa0 0x7fdc1c0c3d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:49.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:49.094+0000 7fdc2090a700 1 --2- 172.21.15.16:0/1193055875 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdc1c0caf60 0x7fdc1c1f9c70 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:49.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:49.094+0000 7fdc2090a700 1 -- 172.21.15.16:0/1193055875 >> 172.21.15.16:0/1193055875 conn(0x7fdc1c0cb910 msgr2=0x7fdc1c06ffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:49.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:49.095+0000 7fdc2090a700 1 -- 172.21.15.16:0/1193055875 shutdown_connections 2023-05-20T01:16:49.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:49.095+0000 7fdc2090a700 1 -- 172.21.15.16:0/1193055875 wait complete. 2023-05-20T01:16:49.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:49.109+0000 7fa66b0cf700 1 -- 172.21.15.16:0/3796166182 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa65006ad30 msgr2=0x7fa65006d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:49.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:49.109+0000 7fa66b0cf700 1 --2- 172.21.15.16:0/3796166182 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa65006ad30 0x7fa65006d1e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa65c00b370 tx=0x7fa65c014040 comp rx=0 tx=0).stop 2023-05-20T01:16:49.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:49.109+0000 7fa66b0cf700 1 -- 172.21.15.16:0/3796166182 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa664160f30 msgr2=0x7fa6640aeb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:49.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:49.109+0000 7fa66b0cf700 1 --2- 172.21.15.16:0/3796166182 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa664160f30 0x7fa6640aeb60 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa65400c450 tx=0x7fa65400c760 comp rx=0 tx=0).stop 2023-05-20T01:16:49.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:49.109+0000 7fa66b0cf700 1 -- 172.21.15.16:0/3796166182 shutdown_connections 2023-05-20T01:16:49.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:49.109+0000 7fa66b0cf700 1 --2- 172.21.15.16:0/3796166182 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa664160f30 0x7fa6640aeb60 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:49.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:49.109+0000 7fa66b0cf700 1 --2- 172.21.15.16:0/3796166182 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa65006ad30 0x7fa65006d1e0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:49.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:49.109+0000 7fa66b0cf700 1 --2- 172.21.15.16:0/3796166182 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa6640b0f50 0x7fa6641569b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:49.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:49.109+0000 7fa66b0cf700 1 --2- 172.21.15.16:0/3796166182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6640cc900 0x7fa6640b2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:49.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:49.110+0000 7fa66b0cf700 1 -- 172.21.15.16:0/3796166182 >> 172.21.15.16:0/3796166182 conn(0x7fa6640cb840 msgr2=0x7fa6640701c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:49.115 INFO:teuthology.orchestra.run.smithi016.stdout:55834574855 2023-05-20T01:16:49.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:49.111+0000 7fc62a0da700 1 -- 172.21.15.16:0/1131580079 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fc604005860 con 0x7fc6240b0e50 2023-05-20T01:16:49.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:49.109+0000 7f6c52269700 1 -- 172.21.15.16:0/2385770827 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f6c4c1613d0 con 0x7f6c4c161000 2023-05-20T01:16:49.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:49.110+0000 7f6c48ff9700 1 -- 172.21.15.16:0/2385770827 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6c4c1613d0 con 0x7f6c4c161000 2023-05-20T01:16:49.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:49.111+0000 7fa66b0cf700 1 -- 172.21.15.16:0/3796166182 shutdown_connections 2023-05-20T01:16:49.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:49.111+0000 7fa66b0cf700 1 -- 172.21.15.16:0/3796166182 wait complete. 2023-05-20T01:16:49.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:49.112+0000 7fc60bfff700 1 -- 172.21.15.16:0/1131580079 <== mon.2 v2:172.21.15.132:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc614015090 con 0x7fc6240b0e50 2023-05-20T01:16:49.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:49.111+0000 7f472affd700 1 -- 172.21.15.16:0/1875492496 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f4720005860 con 0x7f4740071fa0 2023-05-20T01:16:49.118 INFO:teuthology.orchestra.run.smithi016.stdout:55834574856 2023-05-20T01:16:49.119 INFO:teuthology.orchestra.run.smithi016.stdout:55834574855 2023-05-20T01:16:49.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:49.113+0000 7f473d7fa700 1 -- 172.21.15.16:0/1875492496 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4734003cb0 con 0x7f4740071fa0 2023-05-20T01:16:49.119 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.4 2023-05-20T01:16:49.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:49.115+0000 7fc62a0da700 1 -- 172.21.15.16:0/1131580079 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc60c06ae00 msgr2=0x7fc60c06d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:49.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:49.115+0000 7fc62a0da700 1 --2- 172.21.15.16:0/1131580079 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc60c06ae00 0x7fc60c06d2b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc6240c7300 tx=0x7fc618009040 comp rx=0 tx=0).stop 2023-05-20T01:16:49.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:49.115+0000 7fc62a0da700 1 -- 172.21.15.16:0/1131580079 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc6240b0e50 msgr2=0x7fc62415f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:49.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:49.115+0000 7fc62a0da700 1 --2- 172.21.15.16:0/1131580079 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc6240b0e50 0x7fc62415f510 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc614014040 tx=0x7fc614004750 comp rx=0 tx=0).stop 2023-05-20T01:16:49.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:49.119+0000 7eff56856700 1 -- 172.21.15.16:0/1405918066 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7eff5004e8c0 con 0x7eff500b2d90 2023-05-20T01:16:49.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:49.114+0000 7f6c52269700 1 -- 172.21.15.16:0/2385770827 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f6c3806adb0 msgr2=0x7f6c3806d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:49.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:49.114+0000 7f6c52269700 1 --2- 172.21.15.16:0/2385770827 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f6c3806adb0 0x7f6c3806d260 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c0cb280 tx=0x7f6c40006c10 comp rx=0 tx=0).stop 2023-05-20T01:16:49.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:49.114+0000 7f6c52269700 1 -- 172.21.15.16:0/2385770827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c4c161000 msgr2=0x7f6c4c0afec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:49.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:49.114+0000 7f6c52269700 1 --2- 172.21.15.16:0/2385770827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c4c161000 0x7f6c4c0afec0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f6c44002190 tx=0x7f6c4400a3f0 comp rx=0 tx=0).stop 2023-05-20T01:16:49.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:49.116+0000 7f6c52269700 1 -- 172.21.15.16:0/2385770827 shutdown_connections 2023-05-20T01:16:49.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:49.116+0000 7f6c52269700 1 --2- 172.21.15.16:0/2385770827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c4c161000 0x7f6c4c0afec0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:49.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:49.116+0000 7f6c52269700 1 --2- 172.21.15.16:0/2385770827 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f6c3806adb0 0x7f6c3806d260 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:49.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:49.116+0000 7f6c52269700 1 --2- 172.21.15.16:0/2385770827 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6c4c0b0e50 0x7f6c4c1565b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:49.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:49.116+0000 7f6c52269700 1 --2- 172.21.15.16:0/2385770827 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6c4c0ce570 0x7f6c4c1577e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:49.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:49.116+0000 7f6c52269700 1 -- 172.21.15.16:0/2385770827 >> 172.21.15.16:0/2385770827 conn(0x7f6c4c0cb910 msgr2=0x7f6c4c06ffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:49.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:49.116+0000 7f6c52269700 1 -- 172.21.15.16:0/2385770827 shutdown_connections 2023-05-20T01:16:49.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:49.116+0000 7f6c52269700 1 -- 172.21.15.16:0/2385770827 wait complete. 2023-05-20T01:16:49.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:49.120+0000 7eff4cff9700 1 -- 172.21.15.16:0/1405918066 <== mon.2 v2:172.21.15.132:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7eff40056c50 con 0x7eff500b2d90 2023-05-20T01:16:49.127 INFO:teuthology.orchestra.run.smithi016.stdout:55834574855 2023-05-20T01:16:49.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:49.123+0000 7fc62a0da700 1 -- 172.21.15.16:0/1131580079 shutdown_connections 2023-05-20T01:16:49.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:49.123+0000 7fc62a0da700 1 --2- 172.21.15.16:0/1131580079 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc624161000 0x7fc6241eccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:49.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:49.123+0000 7fc62a0da700 1 --2- 172.21.15.16:0/1131580079 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc60c06ae00 0x7fc60c06d2b0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:49.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:49.123+0000 7fc62a0da700 1 --2- 172.21.15.16:0/1131580079 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc6240b0e50 0x7fc62415f510 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:49.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:49.123+0000 7fc62a0da700 1 --2- 172.21.15.16:0/1131580079 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc6240ce740 0x7fc6241ec7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:49.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:49.123+0000 7fc62a0da700 1 -- 172.21.15.16:0/1131580079 >> 172.21.15.16:0/1131580079 conn(0x7fc6240cb910 msgr2=0x7fc6240affa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:49.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:49.123+0000 7fc62a0da700 1 -- 172.21.15.16:0/1131580079 shutdown_connections 2023-05-20T01:16:49.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:49.123+0000 7fc62a0da700 1 -- 172.21.15.16:0/1131580079 wait complete. 2023-05-20T01:16:49.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:49.123+0000 7eff56856700 1 -- 172.21.15.16:0/1405918066 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7eff3c06ace0 msgr2=0x7eff3c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:49.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:49.123+0000 7eff56856700 1 --2- 172.21.15.16:0/1405918066 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7eff3c06ace0 0x7eff3c06d190 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7eff500cec10 tx=0x7eff4800d040 comp rx=0 tx=0).stop 2023-05-20T01:16:49.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:49.123+0000 7eff56856700 1 -- 172.21.15.16:0/1405918066 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7eff500b2d90 msgr2=0x7eff50150de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:49.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:49.123+0000 7eff56856700 1 --2- 172.21.15.16:0/1405918066 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7eff500b2d90 0x7eff50150de0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7eff500ced20 tx=0x7eff4000c670 comp rx=0 tx=0).stop 2023-05-20T01:16:49.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:49.123+0000 7eff56856700 1 -- 172.21.15.16:0/1405918066 shutdown_connections 2023-05-20T01:16:49.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:49.123+0000 7eff56856700 1 --2- 172.21.15.16:0/1405918066 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7eff3c06ace0 0x7eff3c06d190 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:49.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:49.123+0000 7eff56856700 1 --2- 172.21.15.16:0/1405918066 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7eff500b2d90 0x7eff50150de0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:49.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:49.123+0000 7eff56856700 1 --2- 172.21.15.16:0/1405918066 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7eff38004850 0x7eff501585a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:49.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:49.123+0000 7eff56856700 1 --2- 172.21.15.16:0/1405918066 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff50159db0 0x7eff500b2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:49.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:49.123+0000 7eff56856700 1 -- 172.21.15.16:0/1405918066 >> 172.21.15.16:0/1405918066 conn(0x7eff500cb910 msgr2=0x7eff50071b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:49.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:49.124+0000 7eff56856700 1 -- 172.21.15.16:0/1405918066 shutdown_connections 2023-05-20T01:16:49.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:49.127+0000 7f472affd700 1 -- 172.21.15.16:0/1875492496 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f472406ace0 msgr2=0x7f472406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:49.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:49.127+0000 7f472affd700 1 --2- 172.21.15.16:0/1875492496 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f472406ace0 0x7f472406d190 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f474016cc80 tx=0x7f472c009210 comp rx=0 tx=0).stop 2023-05-20T01:16:49.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:49.127+0000 7f472affd700 1 -- 172.21.15.16:0/1875492496 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4740071fa0 msgr2=0x7f47400c1ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:49.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:49.127+0000 7f472affd700 1 --2- 172.21.15.16:0/1875492496 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4740071fa0 0x7f47400c1ab0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4734000c00 tx=0x7f4734004750 comp rx=0 tx=0).stop 2023-05-20T01:16:49.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:49.132+0000 7f472affd700 1 -- 172.21.15.16:0/1875492496 shutdown_connections 2023-05-20T01:16:49.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:49.132+0000 7f472affd700 1 --2- 172.21.15.16:0/1875492496 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f472406ace0 0x7f472406d190 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:49.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:49.132+0000 7f472affd700 1 --2- 172.21.15.16:0/1875492496 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4740161000 0x7f47400ae790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:49.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:49.132+0000 7f472affd700 1 --2- 172.21.15.16:0/1875492496 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4740071fa0 0x7f47400c1ab0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:49.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:49.132+0000 7f472affd700 1 --2- 172.21.15.16:0/1875492496 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f47400caf60 0x7f47400be910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:49.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:49.132+0000 7f472affd700 1 -- 172.21.15.16:0/1875492496 >> 172.21.15.16:0/1875492496 conn(0x7f47400cb910 msgr2=0x7f47400b0320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:49.138 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.1 2023-05-20T01:16:49.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:49.133+0000 7f472affd700 1 -- 172.21.15.16:0/1875492496 shutdown_connections 2023-05-20T01:16:49.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:49.133+0000 7f472affd700 1 -- 172.21.15.16:0/1875492496 wait complete. 2023-05-20T01:16:49.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:49.135+0000 7eff56856700 1 -- 172.21.15.16:0/1405918066 wait complete. 2023-05-20T01:16:49.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:49.135+0000 7f5467be3700 1 -- 172.21.15.16:0/3295818194 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f546004e8c0 con 0x7f5460071fa0 2023-05-20T01:16:49.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:49.135+0000 7f54567fc700 1 -- 172.21.15.16:0/3295818194 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5450015030 con 0x7f5460071fa0 2023-05-20T01:16:49.139 INFO:teuthology.orchestra.run.smithi016.stdout:55834574856 2023-05-20T01:16:49.140 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.6 2023-05-20T01:16:49.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:49.138+0000 7f5467be3700 1 -- 172.21.15.16:0/3295818194 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f544406ace0 msgr2=0x7f544406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:49.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:49.138+0000 7f5467be3700 1 --2- 172.21.15.16:0/3295818194 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f544406ace0 0x7f544406d190 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f54601fa360 tx=0x7f545800b040 comp rx=0 tx=0).stop 2023-05-20T01:16:49.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:49.138+0000 7f5467be3700 1 -- 172.21.15.16:0/3295818194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5460071fa0 msgr2=0x7f546014d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:49.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:49.138+0000 7f5467be3700 1 --2- 172.21.15.16:0/3295818194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5460071fa0 0x7f546014d550 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f5450009fd0 tx=0x7f5450004750 comp rx=0 tx=0).stop 2023-05-20T01:16:49.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:49.139+0000 7f5467be3700 1 -- 172.21.15.16:0/3295818194 shutdown_connections 2023-05-20T01:16:49.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:49.139+0000 7f5467be3700 1 --2- 172.21.15.16:0/3295818194 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5460161000 0x7f5460149dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:49.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:49.139+0000 7f5467be3700 1 --2- 172.21.15.16:0/3295818194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5460071fa0 0x7f546014d550 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:49.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:49.139+0000 7f5467be3700 1 --2- 172.21.15.16:0/3295818194 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f544406ace0 0x7f544406d190 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:49.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:49.139+0000 7f5467be3700 1 --2- 172.21.15.16:0/3295818194 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f54600caf60 0x7f54600aff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:49.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:49.139+0000 7f5467be3700 1 -- 172.21.15.16:0/3295818194 >> 172.21.15.16:0/3295818194 conn(0x7f54600cb910 msgr2=0x7f54600cf710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:49.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:49.139+0000 7f5467be3700 1 -- 172.21.15.16:0/3295818194 shutdown_connections 2023-05-20T01:16:49.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:49.139+0000 7f5467be3700 1 -- 172.21.15.16:0/3295818194 wait complete. 2023-05-20T01:16:49.144 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.7 2023-05-20T01:16:49.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:49.140+0000 7f96c3791700 1 -- 172.21.15.16:0/752514872 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f969c005860 con 0x7f96bc0b2e40 2023-05-20T01:16:49.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:49.140+0000 7f96adffb700 1 -- 172.21.15.16:0/752514872 <== mon.2 v2:172.21.15.132:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f96b0025020 con 0x7f96bc0b2e40 2023-05-20T01:16:49.145 INFO:teuthology.orchestra.run.smithi016.stdout:55834574855 2023-05-20T01:16:49.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:49.143+0000 7f96c3791700 1 -- 172.21.15.16:0/752514872 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f96a006acc0 msgr2=0x7f96a006d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:49.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:49.143+0000 7f96c3791700 1 --2- 172.21.15.16:0/752514872 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f96a006acc0 0x7f96a006d170 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f96bc0c5f20 tx=0x7f96b4006c80 comp rx=0 tx=0).stop 2023-05-20T01:16:49.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:49.143+0000 7f96c3791700 1 -- 172.21.15.16:0/752514872 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f96bc0b2e40 msgr2=0x7f96bc147f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:49.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:49.143+0000 7f96c3791700 1 --2- 172.21.15.16:0/752514872 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f96bc0b2e40 0x7f96bc147f50 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f96b0017040 tx=0x7f96b0004be0 comp rx=0 tx=0).stop 2023-05-20T01:16:49.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:49.143+0000 7f96c3791700 1 -- 172.21.15.16:0/752514872 shutdown_connections 2023-05-20T01:16:49.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:49.143+0000 7f96c3791700 1 --2- 172.21.15.16:0/752514872 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f96bc1e8ea0 0x7f96bc1e9310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:49.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:49.143+0000 7f96c3791700 1 --2- 172.21.15.16:0/752514872 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96a8004850 0x7f96bc0c6510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:49.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:49.143+0000 7f96c3791700 1 --2- 172.21.15.16:0/752514872 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f96a006acc0 0x7f96a006d170 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:49.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:49.143+0000 7f96c3791700 1 --2- 172.21.15.16:0/752514872 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f96bc0b2e40 0x7f96bc147f50 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:49.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:49.143+0000 7f96c3791700 1 -- 172.21.15.16:0/752514872 >> 172.21.15.16:0/752514872 conn(0x7f96bc0cffb0 msgr2=0x7f96bc06acd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:49.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:49.143+0000 7f96c3791700 1 -- 172.21.15.16:0/752514872 shutdown_connections 2023-05-20T01:16:49.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:49.143+0000 7f96c3791700 1 -- 172.21.15.16:0/752514872 wait complete. 2023-05-20T01:16:49.155 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.2 2023-05-20T01:16:49.159 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.5 2023-05-20T01:16:49.161 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.3 2023-05-20T01:16:49.162 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.0 2023-05-20T01:16:50.121 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-05-20T01:16:50.139 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-05-20T01:16:50.140 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-05-20T01:16:50.145 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-05-20T01:16:50.157 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-05-20T01:16:50.161 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-05-20T01:16:50.161 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-05-20T01:16:50.163 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-05-20T01:16:50.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.319+0000 7f342080e700 1 Processor -- start 2023-05-20T01:16:50.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.319+0000 7f342080e700 1 -- start start 2023-05-20T01:16:50.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.320+0000 7f342080e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f341c0cb420 0x7f341c16dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:50.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.320+0000 7f342080e700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f341c06d1e0 0x7f341c0b0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:50.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.320+0000 7f342080e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f341c16d750 con 0x7f341c0cb050 2023-05-20T01:16:50.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.320+0000 7f342080e700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f341c06c2c0 con 0x7f341c06d1e0 2023-05-20T01:16:50.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.320+0000 7f342080e700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f341c06c5a0 con 0x7f341c0cb420 2023-05-20T01:16:50.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.321+0000 7f3419d9b700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f341c0cb420 0x7f341c16dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:50.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.322+0000 7f3419d9b700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f341c0cb420 0x7f341c16dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.16:55324/0 (socket says 172.21.15.16:55324) 2023-05-20T01:16:50.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.322+0000 7f3419d9b700 1 -- 172.21.15.16:0/3233746311 learned_addr learned my addr 172.21.15.16:0/3233746311 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:50.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.322+0000 7f341ad9d700 1 --2- 172.21.15.16:0/3233746311 >> v2:172.21.15.16:3300/0 conn(0x7f341c06d1e0 0x7f341c0b0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:50.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.322+0000 7f341ad9d700 1 -- 172.21.15.16:0/3233746311 >> v2:172.21.15.132:3300/0 conn(0x7f341c0cb420 msgr2=0x7f341c16dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:50.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.322+0000 7f341ad9d700 1 --2- 172.21.15.16:0/3233746311 >> v2:172.21.15.132:3300/0 conn(0x7f341c0cb420 0x7f341c16dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:50.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.322+0000 7f341ad9d700 1 -- 172.21.15.16:0/3233746311 >> v1:172.21.15.132:6789/0 conn(0x7f341c0cb050 legacy=0x7f341c06ba70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-05-20T01:16:50.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.322+0000 7f341ad9d700 1 -- 172.21.15.16:0/3233746311 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f341c06c880 con 0x7f341c06d1e0 2023-05-20T01:16:50.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.323+0000 7f341ad9d700 1 --2- 172.21.15.16:0/3233746311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f341c06d1e0 0x7f341c0b0910 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f341c0ca440 tx=0x7f341400f650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5f04a4f4d7f956b server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:50.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.327+0000 7f341959a700 1 -- 172.21.15.16:0/3233746311 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3414003a20 con 0x7f341c06d1e0 2023-05-20T01:16:50.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.327+0000 7f341959a700 1 -- 172.21.15.16:0/3233746311 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3414003b80 con 0x7f341c06d1e0 2023-05-20T01:16:50.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.327+0000 7f341959a700 1 -- 172.21.15.16:0/3233746311 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f341400d0d0 con 0x7f341c06d1e0 2023-05-20T01:16:50.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.328+0000 7f342080e700 1 -- 172.21.15.16:0/3233746311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f341c06d1e0 msgr2=0x7f341c0b0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:50.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.328+0000 7f342080e700 1 --2- 172.21.15.16:0/3233746311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f341c06d1e0 0x7f341c0b0910 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f341c0ca440 tx=0x7f341400f650 comp rx=0 tx=0).stop 2023-05-20T01:16:50.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.329+0000 7f342080e700 1 -- 172.21.15.16:0/3233746311 shutdown_connections 2023-05-20T01:16:50.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.329+0000 7f342080e700 1 --2- 172.21.15.16:0/3233746311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f341c06d1e0 0x7f341c0b0910 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:50.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.329+0000 7f342080e700 1 --2- 172.21.15.16:0/3233746311 >> v2:172.21.15.132:3300/0 conn(0x7f341c0cb420 0x7f341c16dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:50.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.329+0000 7f342080e700 1 -- 172.21.15.16:0/3233746311 >> 172.21.15.16:0/3233746311 conn(0x7f341c0cb910 msgr2=0x7f341c0cbd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:50.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.330+0000 7f342080e700 1 -- 172.21.15.16:0/3233746311 shutdown_connections 2023-05-20T01:16:50.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.330+0000 7f342080e700 1 -- 172.21.15.16:0/3233746311 wait complete. 2023-05-20T01:16:50.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.332+0000 7f342080e700 1 Processor -- start 2023-05-20T01:16:50.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.333+0000 7f342080e700 1 -- start start 2023-05-20T01:16:50.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.335+0000 7f342080e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f341c06d1e0 0x7f341c0c5f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:50.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.335+0000 7f342080e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f341c0cb050 0x7f341c0c7670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:50.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.335+0000 7f342080e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f341c0cb420 0x7f341c0b3a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:50.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.335+0000 7f342080e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f341c1687a0 con 0x7f341c0cb420 2023-05-20T01:16:50.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.335+0000 7f342080e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f341c0c65d0 con 0x7f341c06d1e0 2023-05-20T01:16:50.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.335+0000 7f342080e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f341c0c68b0 con 0x7f341c0cb050 2023-05-20T01:16:50.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.335+0000 7f3419d9b700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f341c0cb050 0x7f341c0c7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:50.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.335+0000 7f3419d9b700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f341c0cb050 0x7f341c0c7670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.16:58344/0 (socket says 172.21.15.16:58344) 2023-05-20T01:16:50.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.335+0000 7f3419d9b700 1 -- 172.21.15.16:0/1426147969 learned_addr learned my addr 172.21.15.16:0/1426147969 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:50.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.335+0000 7f341ad9d700 1 --2- 172.21.15.16:0/1426147969 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f341c0cb420 0x7f341c0b3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:50.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.336+0000 7f3419d9b700 1 -- 172.21.15.16:0/1426147969 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f341c06d1e0 msgr2=0x7f341c0c5f60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-20T01:16:50.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.336+0000 7f3419d9b700 1 --2- 172.21.15.16:0/1426147969 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f341c06d1e0 0x7f341c0c5f60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:50.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.336+0000 7f3419d9b700 1 -- 172.21.15.16:0/1426147969 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f341c0cb420 msgr2=0x7f341c0b3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:50.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.336+0000 7f3419d9b700 1 --2- 172.21.15.16:0/1426147969 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f341c0cb420 0x7f341c0b3a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:50.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.336+0000 7f3419d9b700 1 -- 172.21.15.16:0/1426147969 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f341400f050 con 0x7f341c0cb050 2023-05-20T01:16:50.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.337+0000 7f3419d9b700 1 --2- 172.21.15.16:0/1426147969 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f341c0cb050 0x7f341c0c7670 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f340c00ac40 tx=0x7f340c00af50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:50.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.338+0000 7f340affd700 1 -- 172.21.15.16:0/1426147969 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f340c010070 con 0x7f341c0cb050 2023-05-20T01:16:50.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.338+0000 7f340affd700 1 -- 172.21.15.16:0/1426147969 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f340c007ac0 con 0x7f341c0cb050 2023-05-20T01:16:50.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.338+0000 7f340affd700 1 -- 172.21.15.16:0/1426147969 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f340c007e70 con 0x7f341c0cb050 2023-05-20T01:16:50.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.338+0000 7f342080e700 1 -- 172.21.15.16:0/1426147969 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f341c16e120 con 0x7f341c0cb050 2023-05-20T01:16:50.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.338+0000 7f342080e700 1 -- 172.21.15.16:0/1426147969 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f341c16d3e0 con 0x7f341c0cb050 2023-05-20T01:16:50.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.343+0000 7f340affd700 1 -- 172.21.15.16:0/1426147969 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f340c00b040 con 0x7f341c0cb050 2023-05-20T01:16:50.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.343+0000 7f340affd700 1 --2- 172.21.15.16:0/1426147969 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f340406af70 0x7f340406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:50.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.344+0000 7f340affd700 1 -- 172.21.15.16:0/1426147969 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f340c088e10 con 0x7f341c0cb050 2023-05-20T01:16:50.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.344+0000 7f342080e700 1 -- 172.21.15.16:0/1426147969 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34000052e0 con 0x7f341c0cb050 2023-05-20T01:16:50.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.345+0000 7f341a59c700 1 --2- 172.21.15.16:0/1426147969 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f340406af70 0x7f340406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:50.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.346+0000 7f341a59c700 1 --2- 172.21.15.16:0/1426147969 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f340406af70 0x7f340406d420 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f3410009710 tx=0x7f3410006c60 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:50.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.350+0000 7f340affd700 1 -- 172.21.15.16:0/1426147969 <== mon.2 v2:172.21.15.132:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f340c053cf0 con 0x7f341c0cb050 2023-05-20T01:16:50.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.391+0000 7f08ec00d700 1 Processor -- start 2023-05-20T01:16:50.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.391+0000 7f08ec00d700 1 -- start start 2023-05-20T01:16:50.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.392+0000 7f08ec00d700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f08e40cb670 0x7f08e40ce340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:50.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.392+0000 7f08ec00d700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f08e40caaa0 0x7f08e40ce880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:50.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.392+0000 7f08ec00d700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08e4159d90 con 0x7f08e41599c0 2023-05-20T01:16:50.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.392+0000 7f08ec00d700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08e40cdf20 con 0x7f08e40cb670 2023-05-20T01:16:50.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.392+0000 7f08ec00d700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08e40cdb80 con 0x7f08e40caaa0 2023-05-20T01:16:50.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.392+0000 7f08e95a8700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f08e40cb670 0x7f08e40ce340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:50.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.392+0000 7f08e95a8700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f08e40cb670 0x7f08e40ce340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.16:55338/0 (socket says 172.21.15.16:55338) 2023-05-20T01:16:50.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.392+0000 7f08e95a8700 1 -- 172.21.15.16:0/3243107778 learned_addr learned my addr 172.21.15.16:0/3243107778 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:50.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.392+0000 7f08ea5aa700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f08e40caaa0 0x7f08e40ce880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:50.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.393+0000 7ff7bd734700 1 Processor -- start 2023-05-20T01:16:50.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.393+0000 7ff7bd734700 1 -- start start 2023-05-20T01:16:50.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.393+0000 7f08e95a8700 1 -- 172.21.15.16:0/3243107778 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f08e40caaa0 msgr2=0x7f08e40ce880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:50.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.393+0000 7f08e95a8700 1 --2- 172.21.15.16:0/3243107778 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f08e40caaa0 0x7f08e40ce880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:50.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.393+0000 7f08e95a8700 1 -- 172.21.15.16:0/3243107778 >> v1:172.21.15.132:6789/0 conn(0x7f08e41599c0 legacy=0x7f08e40caf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:50.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.393+0000 7f08e95a8700 1 -- 172.21.15.16:0/3243107778 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08e406eaf0 con 0x7f08e40cb670 2023-05-20T01:16:50.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.394+0000 7f08e95a8700 1 --2- 172.21.15.16:0/3243107778 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f08e40cb670 0x7f08e40ce340 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f08e000bb90 tx=0x7f08e000bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2e64d37f019736b server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:50.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.394+0000 7ff7bd734700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff7b8157190 0x7ff7b80cc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:50.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.394+0000 7ff7bd734700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7b8071f20 con 0x7ff7b80cccf0 2023-05-20T01:16:50.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.394+0000 7ff7bd734700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7b80cb590 con 0x7ff7b8071ad0 2023-05-20T01:16:50.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.394+0000 7ff7bd734700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b80ca9a0 con 0x7ff7b8157190 2023-05-20T01:16:50.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.395+0000 7ff7b6ffd700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff7b8157190 0x7ff7b80cc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:50.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.395+0000 7ff7b6ffd700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff7b8157190 0x7ff7b80cc7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.16:58358/0 (socket says 172.21.15.16:58358) 2023-05-20T01:16:50.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.395+0000 7ff7b6ffd700 1 -- 172.21.15.16:0/1431712074 learned_addr learned my addr 172.21.15.16:0/1431712074 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:50.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.394+0000 7f08e8da7700 1 -- 172.21.15.16:0/3243107778 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e000dcf0 con 0x7f08e40cb670 2023-05-20T01:16:50.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.394+0000 7f08e8da7700 1 -- 172.21.15.16:0/3243107778 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08e000de50 con 0x7f08e40cb670 2023-05-20T01:16:50.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.394+0000 7f08e8da7700 1 -- 172.21.15.16:0/3243107778 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e0004250 con 0x7f08e40cb670 2023-05-20T01:16:50.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.395+0000 7f08ec00d700 1 -- 172.21.15.16:0/3243107778 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f08e40cb670 msgr2=0x7f08e40ce340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:50.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.395+0000 7f08ec00d700 1 --2- 172.21.15.16:0/3243107778 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f08e40cb670 0x7f08e40ce340 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f08e000bb90 tx=0x7f08e000bea0 comp rx=0 tx=0).stop 2023-05-20T01:16:50.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.394+0000 7f99a2f68700 1 Processor -- start 2023-05-20T01:16:50.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.394+0000 7f99a2f68700 1 -- start start 2023-05-20T01:16:50.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.395+0000 7f99a2f68700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f999c0cebd0 0x7f999c164b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:50.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.395+0000 7f99a2f68700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f999c165260 con 0x7f999c161240 2023-05-20T01:16:50.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.395+0000 7f99a2f68700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f999c165540 con 0x7f999c160e70 2023-05-20T01:16:50.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.395+0000 7f99a2f68700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f999c165b00 con 0x7f999c0cebd0 2023-05-20T01:16:50.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.395+0000 7f999bfff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f999c161240 0x7f999c0ce4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:41620/0 (socket says 172.21.15.16:41620) 2023-05-20T01:16:50.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.395+0000 7f999bfff700 1 -- 172.21.15.16:0/1205620923 learned_addr learned my addr 172.21.15.16:0/1205620923 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:16:50.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.396+0000 7ff7b6ffd700 1 -- 172.21.15.16:0/1431712074 >> v1:172.21.15.132:6789/0 conn(0x7ff7b8071ad0 legacy=0x7ff7b806a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:50.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.396+0000 7ff7b6ffd700 1 -- 172.21.15.16:0/1431712074 >> v1:172.21.15.16:6789/0 conn(0x7ff7b80cccf0 legacy=0x7ff7b80713c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:50.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.396+0000 7ff7b6ffd700 1 -- 172.21.15.16:0/1431712074 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7b80cac60 con 0x7ff7b8157190 2023-05-20T01:16:50.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.396+0000 7f99a1505700 1 --2- 172.21.15.16:0/1205620923 >> v2:172.21.15.16:3300/0 conn(0x7f999c0cebd0 0x7f999c164b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:50.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.396+0000 7ff7b6ffd700 1 --2- 172.21.15.16:0/1431712074 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff7b8157190 0x7ff7b80cc7b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac0098a0 tx=0x7ff7ac009bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c878dd19cc2da37 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:50.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.396+0000 7f999b7fe700 1 -- 172.21.15.16:0/1205620923 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2194242664 0 0) 0x7f999c165540 con 0x7f999c160e70 2023-05-20T01:16:50.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.396+0000 7f999b7fe700 1 -- 172.21.15.16:0/1205620923 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f99840035b0 con 0x7f999c160e70 2023-05-20T01:16:50.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.396+0000 7f99a1505700 1 -- 172.21.15.16:0/1205620923 >> v1:172.21.15.132:6789/0 conn(0x7f999c160e70 legacy=0x7f999c0cadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:50.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.396+0000 7f99a1505700 1 -- 172.21.15.16:0/1205620923 >> v1:172.21.15.16:6789/0 conn(0x7f999c161240 legacy=0x7f999c0ce4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:50.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.396+0000 7f99a1505700 1 -- 172.21.15.16:0/1205620923 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f999c165820 con 0x7f999c0cebd0 2023-05-20T01:16:50.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.396+0000 7f2fa5481700 1 Processor -- start 2023-05-20T01:16:50.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.396+0000 7ff7b5ffb700 1 -- 172.21.15.16:0/1431712074 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7ac004230 con 0x7ff7b8157190 2023-05-20T01:16:50.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.396+0000 7ff7b5ffb700 1 -- 172.21.15.16:0/1431712074 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7ac004390 con 0x7ff7b8157190 2023-05-20T01:16:50.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.396+0000 7ff7b5ffb700 1 -- 172.21.15.16:0/1431712074 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7ac0036a0 con 0x7ff7b8157190 2023-05-20T01:16:50.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.396+0000 7f2fa5481700 1 -- start start 2023-05-20T01:16:50.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.397+0000 7f08ec00d700 1 -- 172.21.15.16:0/3243107778 shutdown_connections 2023-05-20T01:16:50.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.397+0000 7f08ec00d700 1 --2- 172.21.15.16:0/3243107778 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f08e40caaa0 0x7f08e40ce880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:50.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.397+0000 7f08ec00d700 1 --2- 172.21.15.16:0/3243107778 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f08e40cb670 0x7f08e40ce340 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:50.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.397+0000 7f08ec00d700 1 -- 172.21.15.16:0/3243107778 >> 172.21.15.16:0/3243107778 conn(0x7f08e40d0c30 msgr2=0x7f08e40b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:50.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.398+0000 7ff7bd734700 1 -- 172.21.15.16:0/1431712074 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff7b8157190 msgr2=0x7ff7b80cc7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:50.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.398+0000 7ff7bd734700 1 --2- 172.21.15.16:0/1431712074 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff7b8157190 0x7ff7b80cc7b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac0098a0 tx=0x7ff7ac009bb0 comp rx=0 tx=0).stop 2023-05-20T01:16:50.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.398+0000 7ff7bd734700 1 -- 172.21.15.16:0/1431712074 shutdown_connections 2023-05-20T01:16:50.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.398+0000 7ff7bd734700 1 --2- 172.21.15.16:0/1431712074 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff7b8157190 0x7ff7b80cc7b0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:50.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.398+0000 7ff7bd734700 1 -- 172.21.15.16:0/1431712074 >> 172.21.15.16:0/1431712074 conn(0x7ff7b80cffb0 msgr2=0x7ff7b80d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:50.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.398+0000 7f2fa5481700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2fa0071a60 0x7f2fa0071ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:50.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.398+0000 7f2fa5481700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f2fa0164ae0 0x7f2fa0164f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:50.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.398+0000 7f2fa5481700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fa0072410 con 0x7f2fa0161000 2023-05-20T01:16:50.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.398+0000 7f2fa5481700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fa00ce0f0 con 0x7f2fa0164ae0 2023-05-20T01:16:50.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.398+0000 7f2fa5481700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa00ce990 con 0x7f2fa0071a60 2023-05-20T01:16:50.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.398+0000 7f99a1505700 1 --2- 172.21.15.16:0/1205620923 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f999c0cebd0 0x7f999c164b80 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9994009d90 tx=0x7f999400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=698a9d1ef079fc77 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:50.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.398+0000 7ff7bd734700 1 -- 172.21.15.16:0/1431712074 shutdown_connections 2023-05-20T01:16:50.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.398+0000 7f2f9e7fc700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2fa0071a60 0x7f2fa0071ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:50.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.398+0000 7f2f9e7fc700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2fa0071a60 0x7f2fa0071ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.16:58368/0 (socket says 172.21.15.16:58368) 2023-05-20T01:16:50.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.398+0000 7f2f9e7fc700 1 -- 172.21.15.16:0/10662484 learned_addr learned my addr 172.21.15.16:0/10662484 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:50.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.398+0000 7f999b7fe700 1 -- 172.21.15.16:0/1205620923 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f999400e070 con 0x7f999c0cebd0 2023-05-20T01:16:50.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.398+0000 7f999b7fe700 1 -- 172.21.15.16:0/1205620923 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f999400bb20 con 0x7f999c0cebd0 2023-05-20T01:16:50.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.398+0000 7f999b7fe700 1 -- 172.21.15.16:0/1205620923 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9994012430 con 0x7f999c0cebd0 2023-05-20T01:16:50.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.399+0000 7f99a2f68700 1 -- 172.21.15.16:0/1205620923 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f999c0cebd0 msgr2=0x7f999c164b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:50.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.399+0000 7f99a2f68700 1 --2- 172.21.15.16:0/1205620923 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f999c0cebd0 0x7f999c164b80 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9994009d90 tx=0x7f999400b330 comp rx=0 tx=0).stop 2023-05-20T01:16:50.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.398+0000 7f18ecbca700 1 Processor -- start 2023-05-20T01:16:50.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.398+0000 7f18ecbca700 1 -- start start 2023-05-20T01:16:50.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.400+0000 7f18ecbca700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f18e81653d0 0x7f18e80ca310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:50.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.400+0000 7f18ecbca700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f18e80d0210 0x7f18e8165c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:50.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.400+0000 7f18ecbca700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18e80cf670 con 0x7f18e81657a0 2023-05-20T01:16:50.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.400+0000 7f18ecbca700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18e80d1840 con 0x7f18e81653d0 2023-05-20T01:16:50.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.400+0000 7f18ecbca700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18e80d2c10 con 0x7f18e80d0210 2023-05-20T01:16:50.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.399+0000 7f08ec00d700 1 -- 172.21.15.16:0/3243107778 shutdown_connections 2023-05-20T01:16:50.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.400+0000 7f18e6d9d700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f18e80d0210 0x7f18e8165c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:50.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.400+0000 7f18e6d9d700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f18e80d0210 0x7f18e8165c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.16:58376/0 (socket says 172.21.15.16:58376) 2023-05-20T01:16:50.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.400+0000 7f18e6d9d700 1 -- 172.21.15.16:0/2226480156 learned_addr learned my addr 172.21.15.16:0/2226480156 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:50.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.398+0000 7ff7bd734700 1 -- 172.21.15.16:0/1431712074 wait complete. 2023-05-20T01:16:50.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.400+0000 7ff7bd734700 1 Processor -- start 2023-05-20T01:16:50.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.400+0000 7f99a2f68700 1 -- 172.21.15.16:0/1205620923 shutdown_connections 2023-05-20T01:16:50.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.400+0000 7f99a2f68700 1 --2- 172.21.15.16:0/1205620923 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f999c0cebd0 0x7f999c164b80 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:50.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.400+0000 7f99a2f68700 1 -- 172.21.15.16:0/1205620923 >> 172.21.15.16:0/1205620923 conn(0x7f999c0b0e90 msgr2=0x7f999c0b1260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:50.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.400+0000 7f2f9f7fe700 1 --2- 172.21.15.16:0/10662484 >> v2:172.21.15.132:3300/0 conn(0x7f2fa0164ae0 0x7f2fa0164f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:50.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.400+0000 7f2f9f7fe700 1 -- 172.21.15.16:0/10662484 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2fa0071a60 msgr2=0x7f2fa0071ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:50.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.400+0000 7f2f9f7fe700 1 --2- 172.21.15.16:0/10662484 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2fa0071a60 0x7f2fa0071ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:50.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.400+0000 7f2f9f7fe700 1 -- 172.21.15.16:0/10662484 >> v1:172.21.15.132:6789/0 conn(0x7f2fa0161000 legacy=0x7f2fa00caa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:50.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.400+0000 7f2f9f7fe700 1 -- 172.21.15.16:0/10662484 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fa00cb500 con 0x7f2fa0164ae0 2023-05-20T01:16:50.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.400+0000 7f99a2f68700 1 -- 172.21.15.16:0/1205620923 shutdown_connections 2023-05-20T01:16:50.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.400+0000 7f99a2f68700 1 -- 172.21.15.16:0/1205620923 wait complete. 2023-05-20T01:16:50.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.400+0000 7f18e6d9d700 1 -- 172.21.15.16:0/2226480156 >> v2:172.21.15.16:3300/0 conn(0x7f18e81653d0 msgr2=0x7f18e80ca310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:50.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.401+0000 7f18e6d9d700 1 --2- 172.21.15.16:0/2226480156 >> v2:172.21.15.16:3300/0 conn(0x7f18e81653d0 0x7f18e80ca310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:50.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.401+0000 7f18e6d9d700 1 -- 172.21.15.16:0/2226480156 >> v1:172.21.15.16:6789/0 conn(0x7f18e81657a0 legacy=0x7f18e80cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:50.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.401+0000 7f18e6d9d700 1 -- 172.21.15.16:0/2226480156 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18e80b4550 con 0x7f18e80d0210 2023-05-20T01:16:50.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.401+0000 7f18e6d9d700 1 --2- 172.21.15.16:0/2226480156 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f18e80d0210 0x7f18e8165c20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f18e0009e50 tx=0x7f18e000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62a8dce9eddd0570 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:50.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.401+0000 7f08ec00d700 1 -- 172.21.15.16:0/3243107778 wait complete. 2023-05-20T01:16:50.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.401+0000 7f18e559a700 1 -- 172.21.15.16:0/2226480156 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18e000e070 con 0x7f18e80d0210 2023-05-20T01:16:50.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.401+0000 7f18e559a700 1 -- 172.21.15.16:0/2226480156 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18e0003a20 con 0x7f18e80d0210 2023-05-20T01:16:50.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.401+0000 7f18e559a700 1 -- 172.21.15.16:0/2226480156 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18e0003dd0 con 0x7f18e80d0210 2023-05-20T01:16:50.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.400+0000 7ff7bd734700 1 -- start start 2023-05-20T01:16:50.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.402+0000 7ff7bd734700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff7b8071ad0 0x7ff7b806ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:50.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.402+0000 7ff7bd734700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7b80cccf0 0x7ff7b80b0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:50.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.402+0000 7ff7bd734700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff7b8157190 0x7ff7b80b11e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:50.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.402+0000 7ff7bd734700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b80b5d80 con 0x7ff7b80cccf0 2023-05-20T01:16:50.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.402+0000 7ff7bd734700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b8156520 con 0x7ff7b8071ad0 2023-05-20T01:16:50.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.402+0000 7ff7bd734700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b80b57c0 con 0x7ff7b8157190 2023-05-20T01:16:50.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.402+0000 7ff7b77fe700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff7b8157190 0x7ff7b80b11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:50.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.402+0000 7ff7b77fe700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff7b8157190 0x7ff7b80b11e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.16:58382/0 (socket says 172.21.15.16:58382) 2023-05-20T01:16:50.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.402+0000 7ff7b77fe700 1 -- 172.21.15.16:0/1373792653 learned_addr learned my addr 172.21.15.16:0/1373792653 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:50.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.402+0000 7f99a2f68700 1 Processor -- start 2023-05-20T01:16:50.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.402+0000 7ff7b6ffd700 1 --2- 172.21.15.16:0/1373792653 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff7b8071ad0 0x7ff7b806ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:50.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.403+0000 7ff7b6ffd700 1 -- 172.21.15.16:0/1373792653 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff7b8157190 msgr2=0x7ff7b80b11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:50.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.403+0000 7ff7b6ffd700 1 --2- 172.21.15.16:0/1373792653 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff7b8157190 0x7ff7b80b11e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:50.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.403+0000 7ff7b6ffd700 1 -- 172.21.15.16:0/1373792653 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7b80cccf0 msgr2=0x7ff7b80b0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:50.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.403+0000 7ff7b6ffd700 1 --2- 172.21.15.16:0/1373792653 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7b80cccf0 0x7ff7b80b0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:50.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.403+0000 7ff7b6ffd700 1 -- 172.21.15.16:0/1373792653 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7ac009550 con 0x7ff7b8071ad0 2023-05-20T01:16:50.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.403+0000 7f08ec00d700 1 Processor -- start 2023-05-20T01:16:50.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.402+0000 7f99a2f68700 1 -- start start 2023-05-20T01:16:50.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.403+0000 7ff7b6ffd700 1 --2- 172.21.15.16:0/1373792653 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff7b8071ad0 0x7ff7b806ed90 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac015000 tx=0x7ff7ac004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:50.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.404+0000 7ff79f7fe700 1 -- 172.21.15.16:0/1373792653 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7ac003b50 con 0x7ff7b8071ad0 2023-05-20T01:16:50.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.404+0000 7ff79f7fe700 1 -- 172.21.15.16:0/1373792653 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7ac020070 con 0x7ff7b8071ad0 2023-05-20T01:16:50.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.404+0000 7ff79f7fe700 1 -- 172.21.15.16:0/1373792653 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7ac011630 con 0x7ff7b8071ad0 2023-05-20T01:16:50.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.403+0000 7f18ecbca700 1 -- 172.21.15.16:0/2226480156 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f18e80d0210 msgr2=0x7f18e8165c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:50.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.403+0000 7f18ecbca700 1 --2- 172.21.15.16:0/2226480156 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f18e80d0210 0x7f18e8165c20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f18e0009e50 tx=0x7f18e000b330 comp rx=0 tx=0).stop 2023-05-20T01:16:50.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.404+0000 7f18ecbca700 1 -- 172.21.15.16:0/2226480156 shutdown_connections 2023-05-20T01:16:50.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.404+0000 7f18ecbca700 1 --2- 172.21.15.16:0/2226480156 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f18e80d0210 0x7f18e8165c20 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:50.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.404+0000 7f18ecbca700 1 --2- 172.21.15.16:0/2226480156 >> v2:172.21.15.16:3300/0 conn(0x7f18e81653d0 0x7f18e80ca310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:50.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.404+0000 7f18ecbca700 1 -- 172.21.15.16:0/2226480156 >> 172.21.15.16:0/2226480156 conn(0x7f18e809c590 msgr2=0x7f18e80d3240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:50.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.404+0000 7f99a2f68700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f999c0cebd0 0x7f999c1487c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:50.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.404+0000 7f99a2f68700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f999c160e70 0x7f999c1616b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:50.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.404+0000 7f99a2f68700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f999c161240 0x7f999c15f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:50.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.404+0000 7f99a2f68700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f999c14bd20 con 0x7f999c0cebd0 2023-05-20T01:16:50.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.404+0000 7f99a2f68700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f999c147b90 con 0x7f999c160e70 2023-05-20T01:16:50.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.404+0000 7f99a2f68700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f999c147e70 con 0x7f999c161240 2023-05-20T01:16:50.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.404+0000 7f99a0d04700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f999c0cebd0 0x7f999c1487c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:50.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.404+0000 7f999bfff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f999c160e70 0x7f999c1616b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:50.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.404+0000 7f99a0d04700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f999c0cebd0 0x7f999c1487c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58348/0 (socket says 172.21.15.16:58348) 2023-05-20T01:16:50.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.404+0000 7f99a1505700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f999c161240 0x7f999c15f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:50.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.404+0000 7f99a0d04700 1 -- 172.21.15.16:0/2556497117 learned_addr learned my addr 172.21.15.16:0/2556497117 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:50.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.404+0000 7f2f9f7fe700 1 --2- 172.21.15.16:0/10662484 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2fa0164ae0 0x7f2fa0164f40 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2f98009d90 tx=0x7f2f9800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf65da9c12526a52 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:50.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.404+0000 7f18ecbca700 1 -- 172.21.15.16:0/2226480156 shutdown_connections 2023-05-20T01:16:50.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.404+0000 7f18ecbca700 1 -- 172.21.15.16:0/2226480156 wait complete. 2023-05-20T01:16:50.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.404+0000 7ff7bd734700 1 -- 172.21.15.16:0/1373792653 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7b80cb220 con 0x7ff7b8071ad0 2023-05-20T01:16:50.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.404+0000 7ff7bd734700 1 -- 172.21.15.16:0/1373792653 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7b80caf40 con 0x7ff7b8071ad0 2023-05-20T01:16:50.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.403+0000 7f08ec00d700 1 -- start start 2023-05-20T01:16:50.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.404+0000 7f08ec00d700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f08e40caaa0 0x7f08e40c10a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:50.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.404+0000 7f08ec00d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08e40cb670 0x7f08e4146ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:50.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.404+0000 7f08ec00d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f08e41599c0 0x7f08e40710d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:50.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.404+0000 7f08ec00d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08e40b5c90 con 0x7f08e40cb670 2023-05-20T01:16:50.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.404+0000 7f08ec00d700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08e40b53f0 con 0x7f08e41599c0 2023-05-20T01:16:50.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.404+0000 7f08ec00d700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08e40b56d0 con 0x7f08e40caaa0 2023-05-20T01:16:50.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.405+0000 7f08e95a8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08e40cb670 0x7f08e4146ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:50.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.405+0000 7f08e95a8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08e40cb670 0x7f08e4146ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58356/0 (socket says 172.21.15.16:58356) 2023-05-20T01:16:50.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.405+0000 7f08e95a8700 1 -- 172.21.15.16:0/2260346013 learned_addr learned my addr 172.21.15.16:0/2260346013 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:50.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.404+0000 7f99a1505700 1 -- 172.21.15.16:0/2556497117 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f999c160e70 msgr2=0x7f999c1616b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:50.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.404+0000 7f99a1505700 1 --2- 172.21.15.16:0/2556497117 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f999c160e70 0x7f999c1616b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:50.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.404+0000 7f99a1505700 1 -- 172.21.15.16:0/2556497117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f999c0cebd0 msgr2=0x7f999c1487c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:50.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.404+0000 7f99a1505700 1 --2- 172.21.15.16:0/2556497117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f999c0cebd0 0x7f999c1487c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:50.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.405+0000 7f08e9da9700 1 --2- 172.21.15.16:0/2260346013 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f08e40caaa0 0x7f08e40c10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:50.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.404+0000 7f99a1505700 1 -- 172.21.15.16:0/2556497117 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9994009a70 con 0x7f999c161240 2023-05-20T01:16:50.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.404+0000 7f999bfff700 1 --2- 172.21.15.16:0/2556497117 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f999c160e70 0x7f999c1616b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:16:50.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.405+0000 7f99a1505700 1 --2- 172.21.15.16:0/2556497117 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f999c161240 0x7f999c15f720 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f9994016000 tx=0x7f9994004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:50.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.405+0000 7f99997fa700 1 -- 172.21.15.16:0/2556497117 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9994004830 con 0x7f999c161240 2023-05-20T01:16:50.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.405+0000 7f99997fa700 1 -- 172.21.15.16:0/2556497117 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9994020070 con 0x7f999c161240 2023-05-20T01:16:50.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.405+0000 7f99997fa700 1 -- 172.21.15.16:0/2556497117 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99940276c0 con 0x7f999c161240 2023-05-20T01:16:50.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.405+0000 7f99a2f68700 1 -- 172.21.15.16:0/2556497117 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f999c0c2670 con 0x7f999c161240 2023-05-20T01:16:50.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.405+0000 7f99a2f68700 1 -- 172.21.15.16:0/2556497117 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f999c0cd070 con 0x7f999c161240 2023-05-20T01:16:50.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.406+0000 7ff79f7fe700 1 -- 172.21.15.16:0/1373792653 <== mon.1 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff7ac022030 con 0x7ff7b8071ad0 2023-05-20T01:16:50.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.406+0000 7f18ecbca700 1 Processor -- start 2023-05-20T01:16:50.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.406+0000 7f18ecbca700 1 -- start start 2023-05-20T01:16:50.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.405+0000 7f08e9da9700 1 -- 172.21.15.16:0/2260346013 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f08e41599c0 msgr2=0x7f08e40710d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-05-20T01:16:50.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.405+0000 7f08e9da9700 1 --2- 172.21.15.16:0/2260346013 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f08e41599c0 0x7f08e40710d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:50.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.405+0000 7f08e9da9700 1 -- 172.21.15.16:0/2260346013 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08e40cb670 msgr2=0x7f08e4146ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:50.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.405+0000 7f08e9da9700 1 --2- 172.21.15.16:0/2260346013 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08e40cb670 0x7f08e4146ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:50.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.405+0000 7f08e9da9700 1 -- 172.21.15.16:0/2260346013 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08e000b7f0 con 0x7f08e40caaa0 2023-05-20T01:16:50.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.406+0000 7f08e9da9700 1 --2- 172.21.15.16:0/2260346013 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f08e40caaa0 0x7f08e40c10a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f08d400b410 tx=0x7f08d4009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:50.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.406+0000 7f08da7fc700 1 -- 172.21.15.16:0/2260346013 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08d4011830 con 0x7f08e40caaa0 2023-05-20T01:16:50.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.406+0000 7f08da7fc700 1 -- 172.21.15.16:0/2260346013 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08d40119b0 con 0x7f08e40caaa0 2023-05-20T01:16:50.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.406+0000 7f08da7fc700 1 -- 172.21.15.16:0/2260346013 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08d4011d60 con 0x7f08e40caaa0 2023-05-20T01:16:50.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.407+0000 7ff7bd734700 1 -- 172.21.15.16:0/1373792653 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7b8157560 con 0x7ff7b8071ad0 2023-05-20T01:16:50.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.407+0000 7ff79f7fe700 1 --2- 172.21.15.16:0/1373792653 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff7a406ace0 0x7ff7a406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:50.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.407+0000 7ff79f7fe700 1 -- 172.21.15.16:0/1373792653 <== mon.1 v2:172.21.15.132:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7ff7ac08af60 con 0x7ff7b8071ad0 2023-05-20T01:16:50.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.407+0000 7f08ec00d700 1 -- 172.21.15.16:0/2260346013 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08e406f0b0 con 0x7f08e40caaa0 2023-05-20T01:16:50.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.407+0000 7f08ec00d700 1 -- 172.21.15.16:0/2260346013 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08e406edd0 con 0x7f08e40caaa0 2023-05-20T01:16:50.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.407+0000 7f18ecbca700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18e80d0210 0x7f18e815ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:50.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.407+0000 7f18ecbca700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f18e81657a0 0x7f18e80cc180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:50.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.407+0000 7f18ecbca700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f18e80cc6c0 0x7f18e80c7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:50.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.407+0000 7f18ecbca700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18e80ccb40 con 0x7f18e80d0210 2023-05-20T01:16:50.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.407+0000 7f18ecbca700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18e80c7db0 con 0x7f18e81657a0 2023-05-20T01:16:50.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.407+0000 7f18ecbca700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18e80c8090 con 0x7f18e80cc6c0 2023-05-20T01:16:50.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.407+0000 7f18e659c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18e80d0210 0x7f18e815ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:50.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.407+0000 7f18e659c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18e80d0210 0x7f18e815ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58366/0 (socket says 172.21.15.16:58366) 2023-05-20T01:16:50.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.407+0000 7f18e659c700 1 -- 172.21.15.16:0/573021336 learned_addr learned my addr 172.21.15.16:0/573021336 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:50.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.408+0000 7f18e659c700 1 -- 172.21.15.16:0/573021336 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f18e80cc6c0 msgr2=0x7f18e80c7080 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-05-20T01:16:50.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.408+0000 7f18e659c700 1 --2- 172.21.15.16:0/573021336 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f18e80cc6c0 0x7f18e80c7080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:50.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.408+0000 7f18e659c700 1 -- 172.21.15.16:0/573021336 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f18e81657a0 msgr2=0x7f18e80cc180 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-05-20T01:16:50.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.408+0000 7f18e659c700 1 --2- 172.21.15.16:0/573021336 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f18e81657a0 0x7f18e80cc180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:50.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.408+0000 7f18e659c700 1 -- 172.21.15.16:0/573021336 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18e0009b30 con 0x7f18e80d0210 2023-05-20T01:16:50.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.408+0000 7ff7b67fc700 1 --2- 172.21.15.16:0/1373792653 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff7a406ace0 0x7ff7a406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:50.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.408+0000 7f08ec00d700 1 -- 172.21.15.16:0/2260346013 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08e406e750 con 0x7f08e40caaa0 2023-05-20T01:16:50.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.408+0000 7f99a2f68700 1 -- 172.21.15.16:0/2556497117 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f997c0052e0 con 0x7f999c161240 2023-05-20T01:16:50.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.409+0000 7fbcd067a700 1 Processor -- start 2023-05-20T01:16:50.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.409+0000 7fbcd067a700 1 -- start start 2023-05-20T01:16:50.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.409+0000 7ff7b67fc700 1 --2- 172.21.15.16:0/1373792653 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff7a406ace0 0x7ff7a406d190 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff7a8000f60 tx=0x7ff7a8007a40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:50.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.409+0000 7f99997fa700 1 -- 172.21.15.16:0/2556497117 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9994012430 con 0x7f999c161240 2023-05-20T01:16:50.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.409+0000 7f08da7fc700 1 -- 172.21.15.16:0/2260346013 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f08d401a400 con 0x7f08e40caaa0 2023-05-20T01:16:50.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.410+0000 7f2f9dffb700 1 -- 172.21.15.16:0/10662484 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f9800e070 con 0x7f2fa0164ae0 2023-05-20T01:16:50.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.410+0000 7f2f9dffb700 1 -- 172.21.15.16:0/10662484 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f98003a20 con 0x7f2fa0164ae0 2023-05-20T01:16:50.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.410+0000 7f2f9dffb700 1 -- 172.21.15.16:0/10662484 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f98003db0 con 0x7f2fa0164ae0 2023-05-20T01:16:50.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.410+0000 7f18e659c700 1 --2- 172.21.15.16:0/573021336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18e80d0210 0x7f18e815ca90 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f18dc00b4a0 tx=0x7f18dc00b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:50.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.410+0000 7f99997fa700 1 --2- 172.21.15.16:0/2556497117 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f998406b160 0x7f998406d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:50.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.410+0000 7f99997fa700 1 -- 172.21.15.16:0/2556497117 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f9994022070 con 0x7f999c161240 2023-05-20T01:16:50.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.410+0000 7f99a0d04700 1 --2- 172.21.15.16:0/2556497117 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f998406b160 0x7f998406d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:50.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.410+0000 7fbcd067a700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbcc806ac20 0x7fbcc815e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:50.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.410+0000 7fbcd067a700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcc80ccf40 con 0x7fbcc81585d0 2023-05-20T01:16:50.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.410+0000 7fbcd067a700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcc80cd220 con 0x7fbcc80cc9c0 2023-05-20T01:16:50.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.410+0000 7fbcd067a700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcc815e540 con 0x7fbcc806ac20 2023-05-20T01:16:50.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.410+0000 7f08da7fc700 1 --2- 172.21.15.16:0/2260346013 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f08c806ace0 0x7f08c806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:50.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.410+0000 7f08da7fc700 1 -- 172.21.15.16:0/2260346013 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f08d4089760 con 0x7f08e40caaa0 2023-05-20T01:16:50.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.410+0000 7fbcce416700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fbcc81585d0 0x7fbcc815ae20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:41668/0 (socket says 172.21.15.16:41668) 2023-05-20T01:16:50.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.410+0000 7fbcce416700 1 -- 172.21.15.16:0/2459449820 learned_addr learned my addr 172.21.15.16:0/2459449820 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:16:50.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.410+0000 7fbccec17700 1 --2- 172.21.15.16:0/2459449820 >> v2:172.21.15.16:3300/0 conn(0x7fbcc806ac20 0x7fbcc815e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:50.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.411+0000 7f99a0d04700 1 --2- 172.21.15.16:0/2556497117 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f998406b160 0x7f998406d610 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f998c001010 tx=0x7f998c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:50.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.411+0000 7f2fa5481700 1 -- 172.21.15.16:0/10662484 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2fa0164ae0 msgr2=0x7f2fa0164f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:50.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.411+0000 7f2fa5481700 1 --2- 172.21.15.16:0/10662484 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2fa0164ae0 0x7f2fa0164f40 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2f98009d90 tx=0x7f2f9800b330 comp rx=0 tx=0).stop 2023-05-20T01:16:50.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.411+0000 7fbccec17700 1 -- 172.21.15.16:0/2459449820 >> v1:172.21.15.132:6789/0 conn(0x7fbcc80cc9c0 legacy=0x7fbcc806a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:50.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.411+0000 7fbccec17700 1 -- 172.21.15.16:0/2459449820 >> v1:172.21.15.16:6789/0 conn(0x7fbcc81585d0 legacy=0x7fbcc815ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:50.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.411+0000 7fbccec17700 1 -- 172.21.15.16:0/2459449820 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcc80cd500 con 0x7fbcc806ac20 2023-05-20T01:16:50.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.411+0000 7f2fa5481700 1 -- 172.21.15.16:0/10662484 shutdown_connections 2023-05-20T01:16:50.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.411+0000 7f2fa5481700 1 --2- 172.21.15.16:0/10662484 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2fa0164ae0 0x7f2fa0164f40 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:50.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.411+0000 7f2fa5481700 1 --2- 172.21.15.16:0/10662484 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2fa0071a60 0x7f2fa0071ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:50.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.411+0000 7f2fa5481700 1 -- 172.21.15.16:0/10662484 >> 172.21.15.16:0/10662484 conn(0x7f2fa00cb910 msgr2=0x7f2fa00cbd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:50.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.411+0000 7fbccec17700 1 --2- 172.21.15.16:0/2459449820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcc806ac20 0x7fbcc815e990 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fbcb8009840 tx=0x7fbcb8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7aa676cea2ec0ae4 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:50.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.411+0000 7f2fa5481700 1 -- 172.21.15.16:0/10662484 shutdown_connections 2023-05-20T01:16:50.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.411+0000 7f2fa5481700 1 -- 172.21.15.16:0/10662484 wait complete. 2023-05-20T01:16:50.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.411+0000 7f18d6ffd700 1 -- 172.21.15.16:0/573021336 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18dc011830 con 0x7f18e80d0210 2023-05-20T01:16:50.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.411+0000 7f18d6ffd700 1 -- 172.21.15.16:0/573021336 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18dc0119b0 con 0x7f18e80d0210 2023-05-20T01:16:50.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.411+0000 7f18ecbca700 1 -- 172.21.15.16:0/573021336 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18e809ca70 con 0x7f18e80d0210 2023-05-20T01:16:50.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.412+0000 7f18ecbca700 1 -- 172.21.15.16:0/573021336 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18e80d2e90 con 0x7f18e80d0210 2023-05-20T01:16:50.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.413+0000 7f2fa5481700 1 Processor -- start 2023-05-20T01:16:50.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.413+0000 7f2fa5481700 1 -- start start 2023-05-20T01:16:50.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.413+0000 7fbccd414700 1 -- 172.21.15.16:0/2459449820 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb8004230 con 0x7fbcc806ac20 2023-05-20T01:16:50.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.413+0000 7fbccd414700 1 -- 172.21.15.16:0/2459449820 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcb8004390 con 0x7fbcc806ac20 2023-05-20T01:16:50.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.413+0000 7f18d6ffd700 1 -- 172.21.15.16:0/573021336 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18dc011d60 con 0x7f18e80d0210 2023-05-20T01:16:50.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.413+0000 7f08e95a8700 1 --2- 172.21.15.16:0/2260346013 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f08c806ace0 0x7f08c806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:50.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.413+0000 7f08e95a8700 1 --2- 172.21.15.16:0/2260346013 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f08c806ace0 0x7f08c806d190 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f08e4153770 tx=0x7f08e001b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:50.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.414+0000 7ff79f7fe700 1 -- 172.21.15.16:0/1373792653 <== mon.1 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7ff7ac055d10 con 0x7ff7b8071ad0 2023-05-20T01:16:50.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.414+0000 7f2fa5481700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fa0071a60 0x7f2fa0156e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:50.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.414+0000 7f2fa5481700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2fa0161000 0x7f2fa01564a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:50.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.414+0000 7f2fa5481700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2fa0164ae0 0x7f2fa00b1e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:50.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.414+0000 7f2fa5481700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa00c62a0 con 0x7f2fa0071a60 2023-05-20T01:16:50.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.414+0000 7f2fa5481700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa00c6510 con 0x7f2fa0161000 2023-05-20T01:16:50.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.414+0000 7f2fa5481700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa00c67f0 con 0x7f2fa0164ae0 2023-05-20T01:16:50.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.415+0000 7f2f9effd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fa0071a60 0x7f2fa0156e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:50.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.415+0000 7f2f9effd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fa0071a60 0x7f2fa0156e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58380/0 (socket says 172.21.15.16:58380) 2023-05-20T01:16:50.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.415+0000 7f2f9effd700 1 -- 172.21.15.16:0/3461452057 learned_addr learned my addr 172.21.15.16:0/3461452057 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:50.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.414+0000 7f18d6ffd700 1 -- 172.21.15.16:0/573021336 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f18dc01a400 con 0x7f18e80d0210 2023-05-20T01:16:50.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.414+0000 7f18ecbca700 1 -- 172.21.15.16:0/573021336 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18cc0052e0 con 0x7f18e80d0210 2023-05-20T01:16:50.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.414+0000 7f18d6ffd700 1 --2- 172.21.15.16:0/573021336 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f18c806ace0 0x7f18c806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:50.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.414+0000 7f18d6ffd700 1 -- 172.21.15.16:0/573021336 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f18dc089760 con 0x7f18e80d0210 2023-05-20T01:16:50.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.414+0000 7f08da7fc700 1 -- 172.21.15.16:0/2260346013 <== mon.2 v2:172.21.15.132:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f08d4054690 con 0x7f08e40caaa0 2023-05-20T01:16:50.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.415+0000 7f2f9f7fe700 1 --2- 172.21.15.16:0/3461452057 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2fa0164ae0 0x7f2fa00b1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:50.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.415+0000 7f18e5d9b700 1 --2- 172.21.15.16:0/573021336 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f18c806ace0 0x7f18c806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:50.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.415+0000 7fbccd414700 1 -- 172.21.15.16:0/2459449820 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb80036a0 con 0x7fbcc806ac20 2023-05-20T01:16:50.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.415+0000 7f2f9effd700 1 -- 172.21.15.16:0/3461452057 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2fa0164ae0 msgr2=0x7f2fa00b1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:50.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.415+0000 7f2f9effd700 1 --2- 172.21.15.16:0/3461452057 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2fa0164ae0 0x7f2fa00b1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:50.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.415+0000 7f2f9effd700 1 -- 172.21.15.16:0/3461452057 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2fa0161000 msgr2=0x7f2fa01564a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-20T01:16:50.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.415+0000 7f2f9effd700 1 --2- 172.21.15.16:0/3461452057 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2fa0161000 0x7f2fa01564a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:50.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.415+0000 7f2f9effd700 1 -- 172.21.15.16:0/3461452057 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f98009a70 con 0x7f2fa0071a60 2023-05-20T01:16:50.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.415+0000 7f2f9effd700 1 --2- 172.21.15.16:0/3461452057 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fa0071a60 0x7f2fa0156e80 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2f9000c700 tx=0x7f2f9000cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:50.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.416+0000 7fbcd067a700 1 -- 172.21.15.16:0/2459449820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcc806ac20 msgr2=0x7fbcc815e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:50.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.416+0000 7fbcd067a700 1 --2- 172.21.15.16:0/2459449820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcc806ac20 0x7fbcc815e990 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fbcb8009840 tx=0x7fbcb8009b50 comp rx=0 tx=0).stop 2023-05-20T01:16:50.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.415+0000 7f18e5d9b700 1 --2- 172.21.15.16:0/573021336 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f18c806ace0 0x7f18c806d190 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f18e806ca20 tx=0x7f18d8007a20 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:50.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.416+0000 7fbcd067a700 1 -- 172.21.15.16:0/2459449820 shutdown_connections 2023-05-20T01:16:50.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.416+0000 7fbcd067a700 1 --2- 172.21.15.16:0/2459449820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcc806ac20 0x7fbcc815e990 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:50.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.416+0000 7fbcd067a700 1 -- 172.21.15.16:0/2459449820 >> 172.21.15.16:0/2459449820 conn(0x7fbcc80cffb0 msgr2=0x7fbcc80d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:50.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.418+0000 7f99997fa700 1 -- 172.21.15.16:0/2556497117 <== mon.2 v2:172.21.15.132:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f9994056110 con 0x7f999c161240 2023-05-20T01:16:50.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.422+0000 7fbcd067a700 1 -- 172.21.15.16:0/2459449820 shutdown_connections 2023-05-20T01:16:50.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.422+0000 7f18d6ffd700 1 -- 172.21.15.16:0/573021336 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f18dc054520 con 0x7f18e80d0210 2023-05-20T01:16:50.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.422+0000 7fbcd067a700 1 -- 172.21.15.16:0/2459449820 wait complete. 2023-05-20T01:16:50.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.424+0000 7fbcd067a700 1 Processor -- start 2023-05-20T01:16:50.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.424+0000 7fbcd067a700 1 -- start start 2023-05-20T01:16:50.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.425+0000 7f2f877fe700 1 -- 172.21.15.16:0/3461452057 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f900054e0 con 0x7f2fa0071a60 2023-05-20T01:16:50.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.425+0000 7f2f877fe700 1 -- 172.21.15.16:0/3461452057 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f90003680 con 0x7f2fa0071a60 2023-05-20T01:16:50.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.425+0000 7f2f877fe700 1 -- 172.21.15.16:0/3461452057 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f90003a30 con 0x7f2fa0071a60 2023-05-20T01:16:50.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.426+0000 7fbcd067a700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbcc806ac20 0x7fbcc81576e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:50.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.426+0000 7fbcd067a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcc80cc9c0 0x7fbcc80b0d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:50.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.426+0000 7fbcd067a700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbcc81585d0 0x7fbcc80b12a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:50.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.426+0000 7fbcd067a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcc8157c20 con 0x7fbcc80cc9c0 2023-05-20T01:16:50.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.426+0000 7fbcd067a700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcc80b2ee0 con 0x7fbcc806ac20 2023-05-20T01:16:50.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.426+0000 7fbcd067a700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcc80b5930 con 0x7fbcc81585d0 2023-05-20T01:16:50.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.426+0000 7f2fa5481700 1 -- 172.21.15.16:0/3461452057 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fa00ce620 con 0x7f2fa0071a60 2023-05-20T01:16:50.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.426+0000 7f2fa5481700 1 -- 172.21.15.16:0/3461452057 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fa00ce340 con 0x7f2fa0071a60 2023-05-20T01:16:50.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.426+0000 7fbccec17700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbcc81585d0 0x7fbcc80b12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:50.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.426+0000 7fbccec17700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbcc81585d0 0x7fbcc80b12a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.16:58426/0 (socket says 172.21.15.16:58426) 2023-05-20T01:16:50.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.426+0000 7fbccec17700 1 -- 172.21.15.16:0/707674566 learned_addr learned my addr 172.21.15.16:0/707674566 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:50.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.426+0000 7fbccec17700 1 -- 172.21.15.16:0/707674566 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbcc806ac20 msgr2=0x7fbcc81576e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:50.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.426+0000 7fbccec17700 1 --2- 172.21.15.16:0/707674566 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbcc806ac20 0x7fbcc81576e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:50.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.426+0000 7fbccec17700 1 -- 172.21.15.16:0/707674566 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcc80cc9c0 msgr2=0x7fbcc80b0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:50.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.426+0000 7fbccec17700 1 --2- 172.21.15.16:0/707674566 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcc80cc9c0 0x7fbcc80b0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:50.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.426+0000 7fbccec17700 1 -- 172.21.15.16:0/707674566 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcb80094a0 con 0x7fbcc81585d0 2023-05-20T01:16:50.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.427+0000 7fbccec17700 1 --2- 172.21.15.16:0/707674566 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbcc81585d0 0x7fbcc80b12a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fbcb8004000 tx=0x7fbcb8004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:50.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.428+0000 7f2f877fe700 1 -- 172.21.15.16:0/3461452057 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2f90003b90 con 0x7f2fa0071a60 2023-05-20T01:16:50.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.429+0000 7f2f877fe700 1 --2- 172.21.15.16:0/3461452057 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f2f8c06ad30 0x7f2f8c06d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:50.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.429+0000 7f2f877fe700 1 -- 172.21.15.16:0/3461452057 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f2f90088ac0 con 0x7f2fa0071a60 2023-05-20T01:16:50.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.429+0000 7f2f9e7fc700 1 --2- 172.21.15.16:0/3461452057 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f2f8c06ad30 0x7f2f8c06d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:50.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.430+0000 7f2f9e7fc700 1 --2- 172.21.15.16:0/3461452057 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f2f8c06ad30 0x7f2f8c06d1e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2fa015fb80 tx=0x7f2f940092c0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:50.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.435+0000 7fbcbeffd700 1 -- 172.21.15.16:0/707674566 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb8011630 con 0x7fbcc81585d0 2023-05-20T01:16:50.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.435+0000 7fbcbeffd700 1 -- 172.21.15.16:0/707674566 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcb801e070 con 0x7fbcc81585d0 2023-05-20T01:16:50.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.435+0000 7fbcbeffd700 1 -- 172.21.15.16:0/707674566 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb80119d0 con 0x7fbcc81585d0 2023-05-20T01:16:50.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.435+0000 7fbcd067a700 1 -- 172.21.15.16:0/707674566 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcc80719a0 con 0x7fbcc81585d0 2023-05-20T01:16:50.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.435+0000 7fbcd067a700 1 -- 172.21.15.16:0/707674566 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcc80cd7e0 con 0x7fbcc81585d0 2023-05-20T01:16:50.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.437+0000 7f2fa5481700 1 -- 172.21.15.16:0/3461452057 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f800052e0 con 0x7f2fa0071a60 2023-05-20T01:16:50.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.440+0000 7fbcbeffd700 1 -- 172.21.15.16:0/707674566 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fbcb8011b30 con 0x7fbcc81585d0 2023-05-20T01:16:50.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.440+0000 7fbcbeffd700 1 --2- 172.21.15.16:0/707674566 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fbcac06adb0 0x7fbcac06d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:50.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.441+0000 7fbcbeffd700 1 -- 172.21.15.16:0/707674566 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fbcb808aa50 con 0x7fbcc81585d0 2023-05-20T01:16:50.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.441+0000 7fbcce416700 1 --2- 172.21.15.16:0/707674566 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fbcac06adb0 0x7fbcac06d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:50.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.442+0000 7fbcd067a700 1 -- 172.21.15.16:0/707674566 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcb00052e0 con 0x7fbcc81585d0 2023-05-20T01:16:50.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.442+0000 7fbcce416700 1 --2- 172.21.15.16:0/707674566 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fbcac06adb0 0x7fbcac06d260 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fbcc0001fd0 tx=0x7fbcc00015c0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:50.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.442+0000 7f5c87808700 1 Processor -- start 2023-05-20T01:16:50.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.443+0000 7f5c87808700 1 -- start start 2023-05-20T01:16:50.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.444+0000 7f5c87808700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5c800700c0 0x7f5c800b0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:50.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.444+0000 7f5c87808700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5c800b0e50 0x7f5c8006bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:50.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.444+0000 7f5c87808700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c800b1290 con 0x7f5c800ce0a0 2023-05-20T01:16:50.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.444+0000 7f5c87808700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c801590f0 con 0x7f5c800700c0 2023-05-20T01:16:50.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.444+0000 7f5c87808700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c801593b0 con 0x7f5c800b0e50 2023-05-20T01:16:50.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.444+0000 7f5c84da3700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5c800700c0 0x7f5c800b0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:50.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.445+0000 7f5c84da3700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5c800700c0 0x7f5c800b0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58396/0 (socket says 172.21.15.16:58396) 2023-05-20T01:16:50.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.445+0000 7f5c84da3700 1 -- 172.21.15.16:0/1145873895 learned_addr learned my addr 172.21.15.16:0/1145873895 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:50.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.445+0000 7f5c855a4700 1 --1- 172.21.15.16:0/1145873895 >> v1:172.21.15.16:6789/0 conn(0x7f5c800ce0a0 0x7f5c800ce470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:41690/0 (socket says 172.21.15.16:41690) 2023-05-20T01:16:50.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.445+0000 7f5c85da5700 1 --2- 172.21.15.16:0/1145873895 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5c800b0e50 0x7f5c8006bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:50.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.445+0000 7f5c84da3700 1 -- 172.21.15.16:0/1145873895 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5c800b0e50 msgr2=0x7f5c8006bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:50.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.445+0000 7f5c84da3700 1 --2- 172.21.15.16:0/1145873895 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5c800b0e50 0x7f5c8006bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:50.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.445+0000 7f5c84da3700 1 -- 172.21.15.16:0/1145873895 >> v1:172.21.15.16:6789/0 conn(0x7f5c800ce0a0 legacy=0x7f5c800ce470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:50.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.445+0000 7f5c84da3700 1 -- 172.21.15.16:0/1145873895 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c80159690 con 0x7f5c800700c0 2023-05-20T01:16:50.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.445+0000 7f5c84da3700 1 --2- 172.21.15.16:0/1145873895 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c800700c0 0x7f5c800b0910 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c0088f0 tx=0x7f5c7c00d330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=845f3f6d80ffb17a server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:50.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.446+0000 7f5c77fff700 1 -- 172.21.15.16:0/1145873895 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c7c010070 con 0x7f5c800700c0 2023-05-20T01:16:50.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.447+0000 7f2f877fe700 1 -- 172.21.15.16:0/3461452057 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f2f90053aa0 con 0x7f2fa0071a60 2023-05-20T01:16:50.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.448+0000 7fbcbeffd700 1 -- 172.21.15.16:0/707674566 <== mon.2 v2:172.21.15.132:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fbcb8055880 con 0x7fbcc81585d0 2023-05-20T01:16:50.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.447+0000 7f5c77fff700 1 -- 172.21.15.16:0/1145873895 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c7c00da80 con 0x7f5c800700c0 2023-05-20T01:16:50.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.447+0000 7f5c77fff700 1 -- 172.21.15.16:0/1145873895 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c7c00de10 con 0x7f5c800700c0 2023-05-20T01:16:50.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.448+0000 7f5c87808700 1 -- 172.21.15.16:0/1145873895 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c800700c0 msgr2=0x7f5c800b0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:50.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.448+0000 7f5c87808700 1 --2- 172.21.15.16:0/1145873895 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c800700c0 0x7f5c800b0910 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c0088f0 tx=0x7f5c7c00d330 comp rx=0 tx=0).stop 2023-05-20T01:16:50.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.449+0000 7f5c87808700 1 -- 172.21.15.16:0/1145873895 shutdown_connections 2023-05-20T01:16:50.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.449+0000 7f5c87808700 1 --2- 172.21.15.16:0/1145873895 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5c800b0e50 0x7f5c8006bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:50.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.449+0000 7f5c87808700 1 --2- 172.21.15.16:0/1145873895 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c800700c0 0x7f5c800b0910 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:50.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.449+0000 7f5c87808700 1 -- 172.21.15.16:0/1145873895 >> 172.21.15.16:0/1145873895 conn(0x7f5c800cb910 msgr2=0x7f5c800cbd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:50.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.449+0000 7f5c87808700 1 -- 172.21.15.16:0/1145873895 shutdown_connections 2023-05-20T01:16:50.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.449+0000 7f5c87808700 1 -- 172.21.15.16:0/1145873895 wait complete. 2023-05-20T01:16:50.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.451+0000 7f5c87808700 1 Processor -- start 2023-05-20T01:16:50.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.451+0000 7f5c87808700 1 -- start start 2023-05-20T01:16:50.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.452+0000 7f5c87808700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5c800700c0 0x7f5c800c5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:50.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.452+0000 7f5c87808700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5c800b0e50 0x7f5c8015aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:50.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.452+0000 7f5c87808700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c800ce0a0 0x7f5c800c2860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:50.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.452+0000 7f5c87808700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c8015ba10 con 0x7f5c800ce0a0 2023-05-20T01:16:50.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.452+0000 7f5c87808700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c8015b730 con 0x7f5c800700c0 2023-05-20T01:16:50.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.452+0000 7f5c87808700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c800b3080 con 0x7f5c800b0e50 2023-05-20T01:16:50.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.453+0000 7f5c85da5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c800ce0a0 0x7f5c800c2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:50.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.453+0000 7f5c85da5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c800ce0a0 0x7f5c800c2860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58404/0 (socket says 172.21.15.16:58404) 2023-05-20T01:16:50.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.453+0000 7f5c85da5700 1 -- 172.21.15.16:0/150966937 learned_addr learned my addr 172.21.15.16:0/150966937 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:50.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.454+0000 7f5c85da5700 1 -- 172.21.15.16:0/150966937 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5c800b0e50 msgr2=0x7f5c8015aee0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-05-20T01:16:50.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.454+0000 7f5c85da5700 1 --2- 172.21.15.16:0/150966937 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5c800b0e50 0x7f5c8015aee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:50.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.454+0000 7f5c85da5700 1 -- 172.21.15.16:0/150966937 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5c800700c0 msgr2=0x7f5c800c5790 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-05-20T01:16:50.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.454+0000 7f5c85da5700 1 --2- 172.21.15.16:0/150966937 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5c800700c0 0x7f5c800c5790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:50.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.454+0000 7f5c85da5700 1 -- 172.21.15.16:0/150966937 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c7c0085d0 con 0x7f5c800ce0a0 2023-05-20T01:16:50.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.454+0000 7f5c85da5700 1 --2- 172.21.15.16:0/150966937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c800ce0a0 0x7f5c800c2860 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5c7000c7c0 tx=0x7f5c7000cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:50.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.461+0000 7f5c75ffb700 1 -- 172.21.15.16:0/150966937 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c70004220 con 0x7f5c800ce0a0 2023-05-20T01:16:50.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.461+0000 7f5c75ffb700 1 -- 172.21.15.16:0/150966937 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c70004380 con 0x7f5c800ce0a0 2023-05-20T01:16:50.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.461+0000 7f5c75ffb700 1 -- 172.21.15.16:0/150966937 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c700036a0 con 0x7f5c800ce0a0 2023-05-20T01:16:50.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.461+0000 7f5c87808700 1 -- 172.21.15.16:0/150966937 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c80159c50 con 0x7f5c800ce0a0 2023-05-20T01:16:50.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.461+0000 7f5c87808700 1 -- 172.21.15.16:0/150966937 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c80159970 con 0x7f5c800ce0a0 2023-05-20T01:16:50.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.465+0000 7f5c75ffb700 1 -- 172.21.15.16:0/150966937 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5c70003800 con 0x7f5c800ce0a0 2023-05-20T01:16:50.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.466+0000 7f5c75ffb700 1 --2- 172.21.15.16:0/150966937 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f5c6406adb0 0x7f5c6406d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:50.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.466+0000 7f5c75ffb700 1 -- 172.21.15.16:0/150966937 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f5c700892a0 con 0x7f5c800ce0a0 2023-05-20T01:16:50.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.468+0000 7f5c87808700 1 -- 172.21.15.16:0/150966937 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c600052e0 con 0x7f5c800ce0a0 2023-05-20T01:16:50.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.469+0000 7f5c855a4700 1 --2- 172.21.15.16:0/150966937 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f5c6406adb0 0x7f5c6406d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:50.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.469+0000 7f5c855a4700 1 --2- 172.21.15.16:0/150966937 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f5c6406adb0 0x7f5c6406d260 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f5c780020b0 tx=0x7f5c780016a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:50.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.477+0000 7f5c75ffb700 1 -- 172.21.15.16:0/150966937 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f5c700540d0 con 0x7f5c800ce0a0 2023-05-20T01:16:50.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.757+0000 7f342080e700 1 -- 172.21.15.16:0/1426147969 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f3400005590 con 0x7f341c0cb050 2023-05-20T01:16:50.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.758+0000 7f340affd700 1 -- 172.21.15.16:0/1426147969 <== mon.2 v2:172.21.15.132:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f340c019070 con 0x7f341c0cb050 2023-05-20T01:16:50.764 INFO:teuthology.orchestra.run.smithi016.stdout:55834574856 2023-05-20T01:16:50.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.761+0000 7f342080e700 1 -- 172.21.15.16:0/1426147969 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f340406af70 msgr2=0x7f340406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:50.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.761+0000 7f342080e700 1 --2- 172.21.15.16:0/1426147969 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f340406af70 0x7f340406d420 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f3410009710 tx=0x7f3410006c60 comp rx=0 tx=0).stop 2023-05-20T01:16:50.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.761+0000 7f342080e700 1 -- 172.21.15.16:0/1426147969 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f341c0cb050 msgr2=0x7f341c0c7670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:50.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.761+0000 7f342080e700 1 --2- 172.21.15.16:0/1426147969 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f341c0cb050 0x7f341c0c7670 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f340c00ac40 tx=0x7f340c00af50 comp rx=0 tx=0).stop 2023-05-20T01:16:50.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.762+0000 7f342080e700 1 -- 172.21.15.16:0/1426147969 shutdown_connections 2023-05-20T01:16:50.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.762+0000 7f342080e700 1 --2- 172.21.15.16:0/1426147969 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f341c0cb420 0x7f341c0b3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:50.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.762+0000 7f342080e700 1 --2- 172.21.15.16:0/1426147969 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f341c06d1e0 0x7f341c0c5f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:50.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.762+0000 7f342080e700 1 --2- 172.21.15.16:0/1426147969 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f340406af70 0x7f340406d420 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:50.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.762+0000 7f342080e700 1 --2- 172.21.15.16:0/1426147969 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f341c0cb050 0x7f341c0c7670 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:50.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.762+0000 7f342080e700 1 -- 172.21.15.16:0/1426147969 >> 172.21.15.16:0/1426147969 conn(0x7f341c0cb910 msgr2=0x7f341c0cbce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:50.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.762+0000 7f342080e700 1 -- 172.21.15.16:0/1426147969 shutdown_connections 2023-05-20T01:16:50.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.762+0000 7f342080e700 1 -- 172.21.15.16:0/1426147969 wait complete. 2023-05-20T01:16:50.784 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.4 2023-05-20T01:16:50.785 DEBUG:teuthology.parallel:result is None 2023-05-20T01:16:50.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.804+0000 7f08ec00d700 1 -- 172.21.15.16:0/2260346013 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f08e40b5810 con 0x7f08e40caaa0 2023-05-20T01:16:50.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.805+0000 7f08da7fc700 1 -- 172.21.15.16:0/2260346013 <== mon.2 v2:172.21.15.132:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f08e40b5810 con 0x7f08e40caaa0 2023-05-20T01:16:50.809 INFO:teuthology.orchestra.run.smithi016.stdout:55834574856 2023-05-20T01:16:50.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.808+0000 7f08ec00d700 1 -- 172.21.15.16:0/2260346013 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f08c806ace0 msgr2=0x7f08c806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:50.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.808+0000 7f08ec00d700 1 --2- 172.21.15.16:0/2260346013 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f08c806ace0 0x7f08c806d190 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f08e4153770 tx=0x7f08e001b040 comp rx=0 tx=0).stop 2023-05-20T01:16:50.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.808+0000 7f08ec00d700 1 -- 172.21.15.16:0/2260346013 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f08e40caaa0 msgr2=0x7f08e40c10a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:50.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.808+0000 7f08ec00d700 1 --2- 172.21.15.16:0/2260346013 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f08e40caaa0 0x7f08e40c10a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f08d400b410 tx=0x7f08d4009f90 comp rx=0 tx=0).stop 2023-05-20T01:16:50.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.808+0000 7f08ec00d700 1 -- 172.21.15.16:0/2260346013 shutdown_connections 2023-05-20T01:16:50.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.808+0000 7f08ec00d700 1 --2- 172.21.15.16:0/2260346013 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f08e41599c0 0x7f08e40710d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:50.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.808+0000 7f08ec00d700 1 --2- 172.21.15.16:0/2260346013 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f08c806ace0 0x7f08c806d190 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:50.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.808+0000 7f08ec00d700 1 --2- 172.21.15.16:0/2260346013 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f08e40caaa0 0x7f08e40c10a0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:50.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.808+0000 7f08ec00d700 1 --2- 172.21.15.16:0/2260346013 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08e40cb670 0x7f08e4146ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:50.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.808+0000 7f08ec00d700 1 -- 172.21.15.16:0/2260346013 >> 172.21.15.16:0/2260346013 conn(0x7f08e40d0c30 msgr2=0x7f08e4150b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:50.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.808+0000 7ff7bd734700 1 -- 172.21.15.16:0/1373792653 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7ff7b8157560 con 0x7ff7b8071ad0 2023-05-20T01:16:50.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.808+0000 7f08ec00d700 1 -- 172.21.15.16:0/2260346013 shutdown_connections 2023-05-20T01:16:50.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.808+0000 7f08ec00d700 1 -- 172.21.15.16:0/2260346013 wait complete. 2023-05-20T01:16:50.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.808+0000 7ff79f7fe700 1 -- 172.21.15.16:0/1373792653 <== mon.1 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff7ac059330 con 0x7ff7b8071ad0 2023-05-20T01:16:50.817 INFO:teuthology.orchestra.run.smithi016.stdout:55834574856 2023-05-20T01:16:50.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.813+0000 7ff7bd734700 1 -- 172.21.15.16:0/1373792653 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff7a406ace0 msgr2=0x7ff7a406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:50.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.813+0000 7ff7bd734700 1 --2- 172.21.15.16:0/1373792653 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff7a406ace0 0x7ff7a406d190 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff7a8000f60 tx=0x7ff7a8007a40 comp rx=0 tx=0).stop 2023-05-20T01:16:50.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.813+0000 7ff7bd734700 1 -- 172.21.15.16:0/1373792653 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff7b8071ad0 msgr2=0x7ff7b806ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:50.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.813+0000 7ff7bd734700 1 --2- 172.21.15.16:0/1373792653 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff7b8071ad0 0x7ff7b806ed90 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac015000 tx=0x7ff7ac004750 comp rx=0 tx=0).stop 2023-05-20T01:16:50.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.814+0000 7ff7bd734700 1 -- 172.21.15.16:0/1373792653 shutdown_connections 2023-05-20T01:16:50.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.814+0000 7ff7bd734700 1 --2- 172.21.15.16:0/1373792653 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff7a406ace0 0x7ff7a406d190 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:50.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.814+0000 7ff7bd734700 1 --2- 172.21.15.16:0/1373792653 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff7b8157190 0x7ff7b80b11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:50.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.814+0000 7ff7bd734700 1 --2- 172.21.15.16:0/1373792653 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff7b8071ad0 0x7ff7b806ed90 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:50.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.814+0000 7ff7bd734700 1 --2- 172.21.15.16:0/1373792653 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7b80cccf0 0x7ff7b80b0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:50.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.814+0000 7ff7bd734700 1 -- 172.21.15.16:0/1373792653 >> 172.21.15.16:0/1373792653 conn(0x7ff7b80cffb0 msgr2=0x7ff7b806ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:50.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.814+0000 7ff7bd734700 1 -- 172.21.15.16:0/1373792653 shutdown_connections 2023-05-20T01:16:50.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.814+0000 7ff7bd734700 1 -- 172.21.15.16:0/1373792653 wait complete. 2023-05-20T01:16:50.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.824+0000 7f99a2f68700 1 -- 172.21.15.16:0/2556497117 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f997c005860 con 0x7f999c161240 2023-05-20T01:16:50.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.825+0000 7f99997fa700 1 -- 172.21.15.16:0/2556497117 <== mon.2 v2:172.21.15.132:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9994059730 con 0x7f999c161240 2023-05-20T01:16:50.829 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.0 2023-05-20T01:16:50.829 INFO:teuthology.orchestra.run.smithi016.stdout:55834574856 2023-05-20T01:16:50.830 DEBUG:teuthology.parallel:result is None 2023-05-20T01:16:50.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.828+0000 7f99a2f68700 1 -- 172.21.15.16:0/2556497117 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f998406b160 msgr2=0x7f998406d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:50.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.828+0000 7f99a2f68700 1 --2- 172.21.15.16:0/2556497117 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f998406b160 0x7f998406d610 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f998c001010 tx=0x7f998c009040 comp rx=0 tx=0).stop 2023-05-20T01:16:50.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.828+0000 7f99a2f68700 1 -- 172.21.15.16:0/2556497117 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f999c161240 msgr2=0x7f999c15f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:50.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.828+0000 7f99a2f68700 1 --2- 172.21.15.16:0/2556497117 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f999c161240 0x7f999c15f720 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f9994016000 tx=0x7f9994004ad0 comp rx=0 tx=0).stop 2023-05-20T01:16:50.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.828+0000 7f99a2f68700 1 -- 172.21.15.16:0/2556497117 shutdown_connections 2023-05-20T01:16:50.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.828+0000 7f99a2f68700 1 --2- 172.21.15.16:0/2556497117 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f998406b160 0x7f998406d610 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:50.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.828+0000 7f99a2f68700 1 --2- 172.21.15.16:0/2556497117 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f999c161240 0x7f999c15f720 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:50.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.828+0000 7f99a2f68700 1 --2- 172.21.15.16:0/2556497117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f999c0cebd0 0x7f999c1487c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:50.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.828+0000 7f99a2f68700 1 --2- 172.21.15.16:0/2556497117 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f999c160e70 0x7f999c1616b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:50.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.828+0000 7f99a2f68700 1 -- 172.21.15.16:0/2556497117 >> 172.21.15.16:0/2556497117 conn(0x7f999c0b0e90 msgr2=0x7f999c0cbdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:50.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.828+0000 7f99a2f68700 1 -- 172.21.15.16:0/2556497117 shutdown_connections 2023-05-20T01:16:50.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.828+0000 7f99a2f68700 1 -- 172.21.15.16:0/2556497117 wait complete. 2023-05-20T01:16:50.833 INFO:teuthology.orchestra.run.smithi016.stdout:55834574856 2023-05-20T01:16:50.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.829+0000 7f18ecbca700 1 -- 172.21.15.16:0/573021336 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f18cc005860 con 0x7f18e80d0210 2023-05-20T01:16:50.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.830+0000 7f18d6ffd700 1 -- 172.21.15.16:0/573021336 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f18dc057b40 con 0x7f18e80d0210 2023-05-20T01:16:50.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.833+0000 7f18ecbca700 1 -- 172.21.15.16:0/573021336 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f18c806ace0 msgr2=0x7f18c806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:50.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.833+0000 7f18ecbca700 1 --2- 172.21.15.16:0/573021336 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f18c806ace0 0x7f18c806d190 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f18e806ca20 tx=0x7f18d8007a20 comp rx=0 tx=0).stop 2023-05-20T01:16:50.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.833+0000 7f18ecbca700 1 -- 172.21.15.16:0/573021336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18e80d0210 msgr2=0x7f18e815ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:50.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.833+0000 7f18ecbca700 1 --2- 172.21.15.16:0/573021336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18e80d0210 0x7f18e815ca90 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f18dc00b4a0 tx=0x7f18dc00b860 comp rx=0 tx=0).stop 2023-05-20T01:16:50.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.833+0000 7f18ecbca700 1 -- 172.21.15.16:0/573021336 shutdown_connections 2023-05-20T01:16:50.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.833+0000 7f18ecbca700 1 --2- 172.21.15.16:0/573021336 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f18c806ace0 0x7f18c806d190 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:50.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.833+0000 7f18ecbca700 1 --2- 172.21.15.16:0/573021336 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f18e80cc6c0 0x7f18e80c7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:50.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.833+0000 7f18ecbca700 1 --2- 172.21.15.16:0/573021336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18e80d0210 0x7f18e815ca90 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:50.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.833+0000 7f18ecbca700 1 --2- 172.21.15.16:0/573021336 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f18e81657a0 0x7f18e80cc180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:50.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.833+0000 7f18ecbca700 1 -- 172.21.15.16:0/573021336 >> 172.21.15.16:0/573021336 conn(0x7f18e809c590 msgr2=0x7f18e806b920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:50.839 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.6 2023-05-20T01:16:50.839 DEBUG:teuthology.parallel:result is None 2023-05-20T01:16:50.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.833+0000 7f18ecbca700 1 -- 172.21.15.16:0/573021336 shutdown_connections 2023-05-20T01:16:50.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.833+0000 7f18ecbca700 1 -- 172.21.15.16:0/573021336 wait complete. 2023-05-20T01:16:50.848 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.1 2023-05-20T01:16:50.848 DEBUG:teuthology.parallel:result is None 2023-05-20T01:16:50.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.848+0000 7f2fa5481700 1 -- 172.21.15.16:0/3461452057 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f2f80005860 con 0x7f2fa0071a60 2023-05-20T01:16:50.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.848+0000 7f2f877fe700 1 -- 172.21.15.16:0/3461452057 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2f90017020 con 0x7f2fa0071a60 2023-05-20T01:16:50.852 INFO:teuthology.orchestra.run.smithi016.stdout:55834574857 2023-05-20T01:16:50.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.850+0000 7f2fa5481700 1 -- 172.21.15.16:0/3461452057 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f2f8c06ad30 msgr2=0x7f2f8c06d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:50.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.850+0000 7f2fa5481700 1 --2- 172.21.15.16:0/3461452057 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f2f8c06ad30 0x7f2f8c06d1e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2fa015fb80 tx=0x7f2f940092c0 comp rx=0 tx=0).stop 2023-05-20T01:16:50.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.850+0000 7f2fa5481700 1 -- 172.21.15.16:0/3461452057 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fa0071a60 msgr2=0x7f2fa0156e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:50.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.850+0000 7f2fa5481700 1 --2- 172.21.15.16:0/3461452057 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fa0071a60 0x7f2fa0156e80 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2f9000c700 tx=0x7f2f9000cac0 comp rx=0 tx=0).stop 2023-05-20T01:16:50.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.850+0000 7f2fa5481700 1 -- 172.21.15.16:0/3461452057 shutdown_connections 2023-05-20T01:16:50.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.850+0000 7f2fa5481700 1 --2- 172.21.15.16:0/3461452057 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f2f8c06ad30 0x7f2f8c06d1e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:50.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.850+0000 7f2fa5481700 1 --2- 172.21.15.16:0/3461452057 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2fa0164ae0 0x7f2fa00b1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:50.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.850+0000 7f2fa5481700 1 --2- 172.21.15.16:0/3461452057 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fa0071a60 0x7f2fa0156e80 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:50.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.850+0000 7f2fa5481700 1 --2- 172.21.15.16:0/3461452057 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2fa0161000 0x7f2fa01564a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:50.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.850+0000 7f2fa5481700 1 -- 172.21.15.16:0/3461452057 >> 172.21.15.16:0/3461452057 conn(0x7f2fa00cb910 msgr2=0x7f2fa00704a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:50.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.851+0000 7f2fa5481700 1 -- 172.21.15.16:0/3461452057 shutdown_connections 2023-05-20T01:16:50.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.851+0000 7f2fa5481700 1 -- 172.21.15.16:0/3461452057 wait complete. 2023-05-20T01:16:50.858 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.3 2023-05-20T01:16:50.859 DEBUG:teuthology.parallel:result is None 2023-05-20T01:16:50.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.858+0000 7fbcd067a700 1 -- 172.21.15.16:0/707674566 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fbcb0005590 con 0x7fbcc81585d0 2023-05-20T01:16:50.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.858+0000 7fbcbeffd700 1 -- 172.21.15.16:0/707674566 <== mon.2 v2:172.21.15.132:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fbcb8020070 con 0x7fbcc81585d0 2023-05-20T01:16:50.862 INFO:teuthology.orchestra.run.smithi016.stdout:55834574856 2023-05-20T01:16:50.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.860+0000 7fbcd067a700 1 -- 172.21.15.16:0/707674566 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fbcac06adb0 msgr2=0x7fbcac06d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:50.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.860+0000 7fbcd067a700 1 --2- 172.21.15.16:0/707674566 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fbcac06adb0 0x7fbcac06d260 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fbcc0001fd0 tx=0x7fbcc00015c0 comp rx=0 tx=0).stop 2023-05-20T01:16:50.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.860+0000 7fbcd067a700 1 -- 172.21.15.16:0/707674566 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbcc81585d0 msgr2=0x7fbcc80b12a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:50.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.860+0000 7fbcd067a700 1 --2- 172.21.15.16:0/707674566 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbcc81585d0 0x7fbcc80b12a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fbcb8004000 tx=0x7fbcb8004750 comp rx=0 tx=0).stop 2023-05-20T01:16:50.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.860+0000 7fbcd067a700 1 -- 172.21.15.16:0/707674566 shutdown_connections 2023-05-20T01:16:50.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.860+0000 7fbcd067a700 1 --2- 172.21.15.16:0/707674566 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fbcac06adb0 0x7fbcac06d260 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:50.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.860+0000 7fbcd067a700 1 --2- 172.21.15.16:0/707674566 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbcc81585d0 0x7fbcc80b12a0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:50.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.860+0000 7fbcd067a700 1 --2- 172.21.15.16:0/707674566 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbcc806ac20 0x7fbcc81576e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:50.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.860+0000 7fbcd067a700 1 --2- 172.21.15.16:0/707674566 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcc80cc9c0 0x7fbcc80b0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:50.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.860+0000 7fbcd067a700 1 -- 172.21.15.16:0/707674566 >> 172.21.15.16:0/707674566 conn(0x7fbcc80cffb0 msgr2=0x7fbcc80cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:50.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.861+0000 7fbcd067a700 1 -- 172.21.15.16:0/707674566 shutdown_connections 2023-05-20T01:16:50.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.861+0000 7fbcd067a700 1 -- 172.21.15.16:0/707674566 wait complete. 2023-05-20T01:16:50.867 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.7 2023-05-20T01:16:50.867 DEBUG:teuthology.parallel:result is None 2023-05-20T01:16:50.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.865+0000 7f5c87808700 1 -- 172.21.15.16:0/150966937 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f5c60005590 con 0x7f5c800ce0a0 2023-05-20T01:16:50.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.866+0000 7f5c75ffb700 1 -- 172.21.15.16:0/150966937 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5c7001e020 con 0x7f5c800ce0a0 2023-05-20T01:16:50.869 INFO:teuthology.orchestra.run.smithi016.stdout:55834574857 2023-05-20T01:16:50.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.868+0000 7f5c87808700 1 -- 172.21.15.16:0/150966937 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f5c6406adb0 msgr2=0x7f5c6406d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:50.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.868+0000 7f5c87808700 1 --2- 172.21.15.16:0/150966937 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f5c6406adb0 0x7f5c6406d260 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f5c780020b0 tx=0x7f5c780016a0 comp rx=0 tx=0).stop 2023-05-20T01:16:50.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.868+0000 7f5c87808700 1 -- 172.21.15.16:0/150966937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c800ce0a0 msgr2=0x7f5c800c2860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:50.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.868+0000 7f5c87808700 1 --2- 172.21.15.16:0/150966937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c800ce0a0 0x7f5c800c2860 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5c7000c7c0 tx=0x7f5c7000cb80 comp rx=0 tx=0).stop 2023-05-20T01:16:50.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.868+0000 7f5c87808700 1 -- 172.21.15.16:0/150966937 shutdown_connections 2023-05-20T01:16:50.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.868+0000 7f5c87808700 1 --2- 172.21.15.16:0/150966937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c800ce0a0 0x7f5c800c2860 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:50.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.868+0000 7f5c87808700 1 --2- 172.21.15.16:0/150966937 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5c800700c0 0x7f5c800c5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:50.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.868+0000 7f5c87808700 1 --2- 172.21.15.16:0/150966937 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f5c6406adb0 0x7f5c6406d260 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:50.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.868+0000 7f5c87808700 1 --2- 172.21.15.16:0/150966937 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5c800b0e50 0x7f5c8015aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:50.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.868+0000 7f5c87808700 1 -- 172.21.15.16:0/150966937 >> 172.21.15.16:0/150966937 conn(0x7f5c800cb910 msgr2=0x7f5c80070510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:50.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.868+0000 7f5c87808700 1 -- 172.21.15.16:0/150966937 shutdown_connections 2023-05-20T01:16:50.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:50.868+0000 7f5c87808700 1 -- 172.21.15.16:0/150966937 wait complete. 2023-05-20T01:16:50.878 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.2 2023-05-20T01:16:50.878 DEBUG:teuthology.parallel:result is None 2023-05-20T01:16:50.884 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.5 2023-05-20T01:16:50.885 DEBUG:teuthology.parallel:result is None 2023-05-20T01:16:50.885 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-05-20T01:16:50.885 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-05-20T01:16:51.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.045+0000 7fa5feb33700 1 Processor -- start 2023-05-20T01:16:51.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.045+0000 7fa5feb33700 1 -- start start 2023-05-20T01:16:51.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.046+0000 7fa5feb33700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa5f806d4b0 0x7fa5f80cf890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:51.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.046+0000 7fa5feb33700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa5f806a510 0x7fa5f806a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:51.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.046+0000 7fa5feb33700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5f80cc510 con 0x7fa5f806f450 2023-05-20T01:16:51.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.046+0000 7fa5feb33700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5f806aed0 con 0x7fa5f806d4b0 2023-05-20T01:16:51.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.046+0000 7fa5feb33700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5f8071ed0 con 0x7fa5f806a510 2023-05-20T01:16:51.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.046+0000 7fa5f7fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa5f806d4b0 0x7fa5f80cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:51.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.046+0000 7fa5f7fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa5f806d4b0 0x7fa5f80cf890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58410/0 (socket says 172.21.15.16:58410) 2023-05-20T01:16:51.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.046+0000 7fa5f7fff700 1 -- 172.21.15.16:0/3924720171 learned_addr learned my addr 172.21.15.16:0/3924720171 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:51.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.046+0000 7fa5fd0d0700 1 --2- 172.21.15.16:0/3924720171 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa5f806a510 0x7fa5f806a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:51.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.046+0000 7fa5f7fff700 1 -- 172.21.15.16:0/3924720171 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa5f806a510 msgr2=0x7fa5f806a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:51.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.046+0000 7fa5f7fff700 1 --2- 172.21.15.16:0/3924720171 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa5f806a510 0x7fa5f806a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:51.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.046+0000 7fa5f7fff700 1 -- 172.21.15.16:0/3924720171 >> v1:172.21.15.132:6789/0 conn(0x7fa5f806f450 legacy=0x7fa5f8161080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:51.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.046+0000 7fa5f7fff700 1 -- 172.21.15.16:0/3924720171 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5f80721b0 con 0x7fa5f806d4b0 2023-05-20T01:16:51.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.047+0000 7fa5f7fff700 1 --2- 172.21.15.16:0/3924720171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5f806d4b0 0x7fa5f80cf890 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa5e40098f0 tx=0x7fa5e4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efda31f9fca9286e server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:51.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.047+0000 7fa5f77fe700 1 -- 172.21.15.16:0/3924720171 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5e400b8a0 con 0x7fa5f806d4b0 2023-05-20T01:16:51.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.047+0000 7fa5f77fe700 1 -- 172.21.15.16:0/3924720171 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5e400ba00 con 0x7fa5f806d4b0 2023-05-20T01:16:51.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.047+0000 7fa5f77fe700 1 -- 172.21.15.16:0/3924720171 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5e400bd90 con 0x7fa5f806d4b0 2023-05-20T01:16:51.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.047+0000 7fa5feb33700 1 -- 172.21.15.16:0/3924720171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5f806d4b0 msgr2=0x7fa5f80cf890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:51.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.047+0000 7fa5feb33700 1 --2- 172.21.15.16:0/3924720171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5f806d4b0 0x7fa5f80cf890 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa5e40098f0 tx=0x7fa5e4009c00 comp rx=0 tx=0).stop 2023-05-20T01:16:51.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.047+0000 7fa5feb33700 1 -- 172.21.15.16:0/3924720171 shutdown_connections 2023-05-20T01:16:51.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.047+0000 7fa5feb33700 1 --2- 172.21.15.16:0/3924720171 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa5f806a510 0x7fa5f806a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:51.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.047+0000 7fa5feb33700 1 --2- 172.21.15.16:0/3924720171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5f806d4b0 0x7fa5f80cf890 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:51.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.047+0000 7fa5feb33700 1 -- 172.21.15.16:0/3924720171 >> 172.21.15.16:0/3924720171 conn(0x7fa5f80cffb0 msgr2=0x7fa5f80d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:51.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.047+0000 7fa5feb33700 1 -- 172.21.15.16:0/3924720171 shutdown_connections 2023-05-20T01:16:51.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.047+0000 7fa5feb33700 1 -- 172.21.15.16:0/3924720171 wait complete. 2023-05-20T01:16:51.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.049+0000 7fa5feb33700 1 Processor -- start 2023-05-20T01:16:51.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.049+0000 7fa5feb33700 1 -- start start 2023-05-20T01:16:51.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.049+0000 7fa5feb33700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa5f806a510 0x7fa5f814d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:51.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.050+0000 7fa5feb33700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa5f806d4b0 0x7fa5f815b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:51.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.050+0000 7fa5feb33700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5f806f450 0x7fa5f80c8510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:51.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.050+0000 7fa5feb33700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5f814dd80 con 0x7fa5f806f450 2023-05-20T01:16:51.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.050+0000 7fa5feb33700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5f80b3bf0 con 0x7fa5f806a510 2023-05-20T01:16:51.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.050+0000 7fa5feb33700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5f80b2fe0 con 0x7fa5f806d4b0 2023-05-20T01:16:51.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.050+0000 7fa5fd0d0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5f806f450 0x7fa5f80c8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:51.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.050+0000 7fa5fd0d0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5f806f450 0x7fa5f80c8510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58418/0 (socket says 172.21.15.16:58418) 2023-05-20T01:16:51.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.050+0000 7fa5fd0d0700 1 -- 172.21.15.16:0/2195034014 learned_addr learned my addr 172.21.15.16:0/2195034014 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:51.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.050+0000 7fa5fc8cf700 1 --2- 172.21.15.16:0/2195034014 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa5f806a510 0x7fa5f814d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:51.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.050+0000 7fa5f7fff700 1 --2- 172.21.15.16:0/2195034014 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa5f806d4b0 0x7fa5f815b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:51.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.050+0000 7fa5fd0d0700 1 -- 172.21.15.16:0/2195034014 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa5f806d4b0 msgr2=0x7fa5f815b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:51.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.050+0000 7fa5fd0d0700 1 --2- 172.21.15.16:0/2195034014 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa5f806d4b0 0x7fa5f815b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:51.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.050+0000 7fa5fd0d0700 1 -- 172.21.15.16:0/2195034014 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa5f806a510 msgr2=0x7fa5f814d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:51.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.050+0000 7fa5fd0d0700 1 --2- 172.21.15.16:0/2195034014 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa5f806a510 0x7fa5f814d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:51.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.050+0000 7fa5fd0d0700 1 -- 172.21.15.16:0/2195034014 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5e4009550 con 0x7fa5f806f450 2023-05-20T01:16:51.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.050+0000 7fa5fd0d0700 1 --2- 172.21.15.16:0/2195034014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5f806f450 0x7fa5f80c8510 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa5f0007e40 tx=0x7fa5f000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:51.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.050+0000 7fa5f57fa700 1 -- 172.21.15.16:0/2195034014 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5f000eea0 con 0x7fa5f806f450 2023-05-20T01:16:51.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.050+0000 7fa5f57fa700 1 -- 172.21.15.16:0/2195034014 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5f0014420 con 0x7fa5f806f450 2023-05-20T01:16:51.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.050+0000 7fa5feb33700 1 -- 172.21.15.16:0/2195034014 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5f80cbe50 con 0x7fa5f806f450 2023-05-20T01:16:51.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.050+0000 7fa5f57fa700 1 -- 172.21.15.16:0/2195034014 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5f00147b0 con 0x7fa5f806f450 2023-05-20T01:16:51.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.050+0000 7fa5feb33700 1 -- 172.21.15.16:0/2195034014 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5f80cc1a0 con 0x7fa5f806f450 2023-05-20T01:16:51.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.051+0000 7fa5feb33700 1 -- 172.21.15.16:0/2195034014 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5f80cc9a0 con 0x7fa5f806f450 2023-05-20T01:16:51.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.052+0000 7fa5f57fa700 1 -- 172.21.15.16:0/2195034014 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa5f0014910 con 0x7fa5f806f450 2023-05-20T01:16:51.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.052+0000 7fa5f57fa700 1 --2- 172.21.15.16:0/2195034014 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa5dc06ace0 0x7fa5dc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:51.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.052+0000 7fa5f57fa700 1 -- 172.21.15.16:0/2195034014 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fa5f0089f50 con 0x7fa5f806f450 2023-05-20T01:16:51.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.052+0000 7fa5fc8cf700 1 --2- 172.21.15.16:0/2195034014 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa5dc06ace0 0x7fa5dc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:51.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.053+0000 7fa5fc8cf700 1 --2- 172.21.15.16:0/2195034014 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa5dc06ace0 0x7fa5dc06d190 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa5f815af20 tx=0x7fa5ec009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:51.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.057+0000 7fa5f57fa700 1 -- 172.21.15.16:0/2195034014 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fa5f0054d10 con 0x7fa5f806f450 2023-05-20T01:16:51.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.249+0000 7fa5feb33700 1 -- 172.21.15.16:0/2195034014 --> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa5f80b3dd0 con 0x7fa5dc06ace0 2023-05-20T01:16:51.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.260+0000 7fa5f57fa700 1 -- 172.21.15.16:0/2195034014 <== mgr.4104 v2:172.21.15.16:6832/3817237998 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+367505 (secure 0 0 0) 0x7fa5f80b3dd0 con 0x7fa5dc06ace0 2023-05-20T01:16:51.267 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:16:51.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.263+0000 7fa5feb33700 1 -- 172.21.15.16:0/2195034014 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa5dc06ace0 msgr2=0x7fa5dc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:51.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.263+0000 7fa5feb33700 1 --2- 172.21.15.16:0/2195034014 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa5dc06ace0 0x7fa5dc06d190 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa5f815af20 tx=0x7fa5ec009040 comp rx=0 tx=0).stop 2023-05-20T01:16:51.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.263+0000 7fa5feb33700 1 -- 172.21.15.16:0/2195034014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5f806f450 msgr2=0x7fa5f80c8510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:51.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.263+0000 7fa5feb33700 1 --2- 172.21.15.16:0/2195034014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5f806f450 0x7fa5f80c8510 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa5f0007e40 tx=0x7fa5f000e3c0 comp rx=0 tx=0).stop 2023-05-20T01:16:51.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.263+0000 7fa5feb33700 1 -- 172.21.15.16:0/2195034014 shutdown_connections 2023-05-20T01:16:51.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.263+0000 7fa5feb33700 1 --2- 172.21.15.16:0/2195034014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5f806f450 0x7fa5f80c8510 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:51.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.263+0000 7fa5feb33700 1 --2- 172.21.15.16:0/2195034014 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa5f806a510 0x7fa5f814d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:51.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.263+0000 7fa5feb33700 1 --2- 172.21.15.16:0/2195034014 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa5dc06ace0 0x7fa5dc06d190 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:51.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.263+0000 7fa5feb33700 1 --2- 172.21.15.16:0/2195034014 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa5f806d4b0 0x7fa5f815b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:51.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.263+0000 7fa5feb33700 1 -- 172.21.15.16:0/2195034014 >> 172.21.15.16:0/2195034014 conn(0x7fa5f80cffb0 msgr2=0x7fa5f81595e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:51.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.263+0000 7fa5feb33700 1 -- 172.21.15.16:0/2195034014 shutdown_connections 2023-05-20T01:16:51.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.263+0000 7fa5feb33700 1 -- 172.21.15.16:0/2195034014 wait complete. 2023-05-20T01:16:51.276 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2023-05-20T01:16:51.279 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":33,"stamp":"2023-05-20T01:16:49.322226+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594960,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":338,"acting":338,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":338,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":67296,"kb_used_data":3296,"kb_used_omap":0,"kb_used_meta":63936,"kb_avail":754907424,"statfs":{"total":773094113280,"available":773025202176,"internally_reserved":0,"allocated":3375104,"data_stored":2063206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":65470464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002753"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.310521+0000","last_change":"2023-05-20T01:16:28.345449+0000","last_active":"2023-05-20T01:16:33.310521+0000","last_peered":"2023-05-20T01:16:33.310521+0000","last_clean":"2023-05-20T01:16:33.310521+0000","last_became_active":"2023-05-20T01:16:28.345161+0000","last_became_peered":"2023-05-20T01:16:28.345161+0000","last_unstale":"2023-05-20T01:16:33.310521+0000","last_undegraded":"2023-05-20T01:16:33.310521+0000","last_fullsized":"2023-05-20T01:16:33.310521+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:10:54.451689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.310465+0000","last_change":"2023-05-20T01:16:28.337973+0000","last_active":"2023-05-20T01:16:33.310465+0000","last_peered":"2023-05-20T01:16:33.310465+0000","last_clean":"2023-05-20T01:16:33.310465+0000","last_became_active":"2023-05-20T01:16:28.337679+0000","last_became_peered":"2023-05-20T01:16:28.337679+0000","last_unstale":"2023-05-20T01:16:33.310465+0000","last_undegraded":"2023-05-20T01:16:33.310465+0000","last_fullsized":"2023-05-20T01:16:33.310465+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:59:48.441604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.774449+0000","last_change":"2023-05-20T01:16:28.342781+0000","last_active":"2023-05-20T01:16:32.774449+0000","last_peered":"2023-05-20T01:16:32.774449+0000","last_clean":"2023-05-20T01:16:32.774449+0000","last_became_active":"2023-05-20T01:16:28.342488+0000","last_became_peered":"2023-05-20T01:16:28.342488+0000","last_unstale":"2023-05-20T01:16:32.774449+0000","last_undegraded":"2023-05-20T01:16:32.774449+0000","last_fullsized":"2023-05-20T01:16:32.774449+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:35:02.629395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.773758+0000","last_change":"2023-05-20T01:16:28.345746+0000","last_active":"2023-05-20T01:16:32.773758+0000","last_peered":"2023-05-20T01:16:32.773758+0000","last_clean":"2023-05-20T01:16:32.773758+0000","last_became_active":"2023-05-20T01:16:28.345477+0000","last_became_peered":"2023-05-20T01:16:28.345477+0000","last_unstale":"2023-05-20T01:16:32.773758+0000","last_undegraded":"2023-05-20T01:16:32.773758+0000","last_fullsized":"2023-05-20T01:16:32.773758+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:08:32.412558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.954652+0000","last_change":"2023-05-20T01:16:28.352984+0000","last_active":"2023-05-20T01:16:32.954652+0000","last_peered":"2023-05-20T01:16:32.954652+0000","last_clean":"2023-05-20T01:16:32.954652+0000","last_became_active":"2023-05-20T01:16:28.352711+0000","last_became_peered":"2023-05-20T01:16:28.352711+0000","last_unstale":"2023-05-20T01:16:32.954652+0000","last_undegraded":"2023-05-20T01:16:32.954652+0000","last_fullsized":"2023-05-20T01:16:32.954652+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:34:08.601566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.774271+0000","last_change":"2023-05-20T01:16:28.347827+0000","last_active":"2023-05-20T01:16:32.774271+0000","last_peered":"2023-05-20T01:16:32.774271+0000","last_clean":"2023-05-20T01:16:32.774271+0000","last_became_active":"2023-05-20T01:16:28.347476+0000","last_became_peered":"2023-05-20T01:16:28.347476+0000","last_unstale":"2023-05-20T01:16:32.774271+0000","last_undegraded":"2023-05-20T01:16:32.774271+0000","last_fullsized":"2023-05-20T01:16:32.774271+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:21:02.720695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.311877+0000","last_change":"2023-05-20T01:16:28.338564+0000","last_active":"2023-05-20T01:16:33.311877+0000","last_peered":"2023-05-20T01:16:33.311877+0000","last_clean":"2023-05-20T01:16:33.311877+0000","last_became_active":"2023-05-20T01:16:28.337927+0000","last_became_peered":"2023-05-20T01:16:28.337927+0000","last_unstale":"2023-05-20T01:16:33.311877+0000","last_undegraded":"2023-05-20T01:16:33.311877+0000","last_fullsized":"2023-05-20T01:16:33.311877+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:24:36.047155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.310151+0000","last_change":"2023-05-20T01:16:28.340008+0000","last_active":"2023-05-20T01:16:33.310151+0000","last_peered":"2023-05-20T01:16:33.310151+0000","last_clean":"2023-05-20T01:16:33.310151+0000","last_became_active":"2023-05-20T01:16:28.339724+0000","last_became_peered":"2023-05-20T01:16:28.339724+0000","last_unstale":"2023-05-20T01:16:33.310151+0000","last_undegraded":"2023-05-20T01:16:33.310151+0000","last_fullsized":"2023-05-20T01:16:33.310151+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:55:50.059228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.954657+0000","last_change":"2023-05-20T01:16:28.351002+0000","last_active":"2023-05-20T01:16:32.954657+0000","last_peered":"2023-05-20T01:16:32.954657+0000","last_clean":"2023-05-20T01:16:32.954657+0000","last_became_active":"2023-05-20T01:16:28.350672+0000","last_became_peered":"2023-05-20T01:16:28.350672+0000","last_unstale":"2023-05-20T01:16:32.954657+0000","last_undegraded":"2023-05-20T01:16:32.954657+0000","last_fullsized":"2023-05-20T01:16:32.954657+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:23:33.732284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.310970+0000","last_change":"2023-05-20T01:16:28.349410+0000","last_active":"2023-05-20T01:16:33.310970+0000","last_peered":"2023-05-20T01:16:33.310970+0000","last_clean":"2023-05-20T01:16:33.310970+0000","last_became_active":"2023-05-20T01:16:28.349177+0000","last_became_peered":"2023-05-20T01:16:28.349177+0000","last_unstale":"2023-05-20T01:16:33.310970+0000","last_undegraded":"2023-05-20T01:16:33.310970+0000","last_fullsized":"2023-05-20T01:16:33.310970+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:34:08.601566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.774486+0000","last_change":"2023-05-20T01:16:28.338627+0000","last_active":"2023-05-20T01:16:32.774486+0000","last_peered":"2023-05-20T01:16:32.774486+0000","last_clean":"2023-05-20T01:16:32.774486+0000","last_became_active":"2023-05-20T01:16:28.338363+0000","last_became_peered":"2023-05-20T01:16:28.338363+0000","last_unstale":"2023-05-20T01:16:32.774486+0000","last_undegraded":"2023-05-20T01:16:32.774486+0000","last_fullsized":"2023-05-20T01:16:32.774486+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:20:15.094065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.310156+0000","last_change":"2023-05-20T01:16:28.341603+0000","last_active":"2023-05-20T01:16:33.310156+0000","last_peered":"2023-05-20T01:16:33.310156+0000","last_clean":"2023-05-20T01:16:33.310156+0000","last_became_active":"2023-05-20T01:16:28.341242+0000","last_became_peered":"2023-05-20T01:16:28.341242+0000","last_unstale":"2023-05-20T01:16:33.310156+0000","last_undegraded":"2023-05-20T01:16:33.310156+0000","last_fullsized":"2023-05-20T01:16:33.310156+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:15:50.986060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.310269+0000","last_change":"2023-05-20T01:16:28.345977+0000","last_active":"2023-05-20T01:16:33.310269+0000","last_peered":"2023-05-20T01:16:33.310269+0000","last_clean":"2023-05-20T01:16:33.310269+0000","last_became_active":"2023-05-20T01:16:28.345732+0000","last_became_peered":"2023-05-20T01:16:28.345732+0000","last_unstale":"2023-05-20T01:16:33.310269+0000","last_undegraded":"2023-05-20T01:16:33.310269+0000","last_fullsized":"2023-05-20T01:16:33.310269+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:39:45.746298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.311067+0000","last_change":"2023-05-20T01:16:28.334227+0000","last_active":"2023-05-20T01:16:33.311067+0000","last_peered":"2023-05-20T01:16:33.311067+0000","last_clean":"2023-05-20T01:16:33.311067+0000","last_became_active":"2023-05-20T01:16:28.333922+0000","last_became_peered":"2023-05-20T01:16:28.333922+0000","last_unstale":"2023-05-20T01:16:33.311067+0000","last_undegraded":"2023-05-20T01:16:33.311067+0000","last_fullsized":"2023-05-20T01:16:33.311067+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:23:33.732284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.311509+0000","last_change":"2023-05-20T01:16:28.338772+0000","last_active":"2023-05-20T01:16:33.311509+0000","last_peered":"2023-05-20T01:16:33.311509+0000","last_clean":"2023-05-20T01:16:33.311509+0000","last_became_active":"2023-05-20T01:16:28.338300+0000","last_became_peered":"2023-05-20T01:16:28.338300+0000","last_unstale":"2023-05-20T01:16:33.311509+0000","last_undegraded":"2023-05-20T01:16:33.311509+0000","last_fullsized":"2023-05-20T01:16:33.311509+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:19:51.520080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.774491+0000","last_change":"2023-05-20T01:16:28.343926+0000","last_active":"2023-05-20T01:16:32.774491+0000","last_peered":"2023-05-20T01:16:32.774491+0000","last_clean":"2023-05-20T01:16:32.774491+0000","last_became_active":"2023-05-20T01:16:28.343696+0000","last_became_peered":"2023-05-20T01:16:28.343696+0000","last_unstale":"2023-05-20T01:16:32.774491+0000","last_undegraded":"2023-05-20T01:16:32.774491+0000","last_fullsized":"2023-05-20T01:16:32.774491+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:42:48.561844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.954664+0000","last_change":"2023-05-20T01:16:28.350876+0000","last_active":"2023-05-20T01:16:32.954664+0000","last_peered":"2023-05-20T01:16:32.954664+0000","last_clean":"2023-05-20T01:16:32.954664+0000","last_became_active":"2023-05-20T01:16:28.350242+0000","last_became_peered":"2023-05-20T01:16:28.350242+0000","last_unstale":"2023-05-20T01:16:32.954664+0000","last_undegraded":"2023-05-20T01:16:32.954664+0000","last_fullsized":"2023-05-20T01:16:32.954664+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:19:51.520080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.773740+0000","last_change":"2023-05-20T01:16:28.334794+0000","last_active":"2023-05-20T01:16:32.773740+0000","last_peered":"2023-05-20T01:16:32.773740+0000","last_clean":"2023-05-20T01:16:32.773740+0000","last_became_active":"2023-05-20T01:16:28.334535+0000","last_became_peered":"2023-05-20T01:16:28.334535+0000","last_unstale":"2023-05-20T01:16:32.773740+0000","last_undegraded":"2023-05-20T01:16:32.773740+0000","last_fullsized":"2023-05-20T01:16:32.773740+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T01:37:30.926061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.310464+0000","last_change":"2023-05-20T01:16:28.347597+0000","last_active":"2023-05-20T01:16:33.310464+0000","last_peered":"2023-05-20T01:16:33.310464+0000","last_clean":"2023-05-20T01:16:33.310464+0000","last_became_active":"2023-05-20T01:16:28.347245+0000","last_became_peered":"2023-05-20T01:16:28.347245+0000","last_unstale":"2023-05-20T01:16:33.310464+0000","last_undegraded":"2023-05-20T01:16:33.310464+0000","last_fullsized":"2023-05-20T01:16:33.310464+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:02:17.539187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.773711+0000","last_change":"2023-05-20T01:16:28.347994+0000","last_active":"2023-05-20T01:16:32.773711+0000","last_peered":"2023-05-20T01:16:32.773711+0000","last_clean":"2023-05-20T01:16:32.773711+0000","last_became_active":"2023-05-20T01:16:28.347690+0000","last_became_peered":"2023-05-20T01:16:28.347690+0000","last_unstale":"2023-05-20T01:16:32.773711+0000","last_undegraded":"2023-05-20T01:16:32.773711+0000","last_fullsized":"2023-05-20T01:16:32.773711+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:46:24.892963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.311730+0000","last_change":"2023-05-20T01:16:28.346262+0000","last_active":"2023-05-20T01:16:33.311730+0000","last_peered":"2023-05-20T01:16:33.311730+0000","last_clean":"2023-05-20T01:16:33.311730+0000","last_became_active":"2023-05-20T01:16:28.345513+0000","last_became_peered":"2023-05-20T01:16:28.345513+0000","last_unstale":"2023-05-20T01:16:33.311730+0000","last_undegraded":"2023-05-20T01:16:33.311730+0000","last_fullsized":"2023-05-20T01:16:33.311730+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:46:14.388881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.954206+0000","last_change":"2023-05-20T01:16:28.344269+0000","last_active":"2023-05-20T01:16:32.954206+0000","last_peered":"2023-05-20T01:16:32.954206+0000","last_clean":"2023-05-20T01:16:32.954206+0000","last_became_active":"2023-05-20T01:16:28.343991+0000","last_became_peered":"2023-05-20T01:16:28.343991+0000","last_unstale":"2023-05-20T01:16:32.954206+0000","last_undegraded":"2023-05-20T01:16:32.954206+0000","last_fullsized":"2023-05-20T01:16:32.954206+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:33:00.800050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.771776+0000","last_change":"2023-05-20T01:16:28.344552+0000","last_active":"2023-05-20T01:16:32.771776+0000","last_peered":"2023-05-20T01:16:32.771776+0000","last_clean":"2023-05-20T01:16:32.771776+0000","last_became_active":"2023-05-20T01:16:28.344125+0000","last_became_peered":"2023-05-20T01:16:28.344125+0000","last_unstale":"2023-05-20T01:16:32.771776+0000","last_undegraded":"2023-05-20T01:16:32.771776+0000","last_fullsized":"2023-05-20T01:16:32.771776+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:06:59.447318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.311284+0000","last_change":"2023-05-20T01:16:28.345899+0000","last_active":"2023-05-20T01:16:33.311284+0000","last_peered":"2023-05-20T01:16:33.311284+0000","last_clean":"2023-05-20T01:16:33.311284+0000","last_became_active":"2023-05-20T01:16:28.345629+0000","last_became_peered":"2023-05-20T01:16:28.345629+0000","last_unstale":"2023-05-20T01:16:33.311284+0000","last_undegraded":"2023-05-20T01:16:33.311284+0000","last_fullsized":"2023-05-20T01:16:33.311284+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:13:57.518009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.957468+0000","last_change":"2023-05-20T01:16:28.346168+0000","last_active":"2023-05-20T01:16:32.957468+0000","last_peered":"2023-05-20T01:16:32.957468+0000","last_clean":"2023-05-20T01:16:32.957468+0000","last_became_active":"2023-05-20T01:16:28.345889+0000","last_became_peered":"2023-05-20T01:16:28.345889+0000","last_unstale":"2023-05-20T01:16:32.957468+0000","last_undegraded":"2023-05-20T01:16:32.957468+0000","last_fullsized":"2023-05-20T01:16:32.957468+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:05:20.974728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.957396+0000","last_change":"2023-05-20T01:16:28.335035+0000","last_active":"2023-05-20T01:16:32.957396+0000","last_peered":"2023-05-20T01:16:32.957396+0000","last_clean":"2023-05-20T01:16:32.957396+0000","last_became_active":"2023-05-20T01:16:28.334787+0000","last_became_peered":"2023-05-20T01:16:28.334787+0000","last_unstale":"2023-05-20T01:16:32.957396+0000","last_undegraded":"2023-05-20T01:16:32.957396+0000","last_fullsized":"2023-05-20T01:16:32.957396+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:55:50.059228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.310780+0000","last_change":"2023-05-20T01:16:28.335038+0000","last_active":"2023-05-20T01:16:33.310780+0000","last_peered":"2023-05-20T01:16:33.310780+0000","last_clean":"2023-05-20T01:16:33.310780+0000","last_became_active":"2023-05-20T01:16:28.334750+0000","last_became_peered":"2023-05-20T01:16:28.334750+0000","last_unstale":"2023-05-20T01:16:33.310780+0000","last_undegraded":"2023-05-20T01:16:33.310780+0000","last_fullsized":"2023-05-20T01:16:33.310780+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:13:57.518009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.774853+0000","last_change":"2023-05-20T01:16:32.363983+0000","last_active":"2023-05-20T01:16:32.774853+0000","last_peered":"2023-05-20T01:16:32.774853+0000","last_clean":"2023-05-20T01:16:32.774853+0000","last_became_active":"2023-05-20T01:16:32.363545+0000","last_became_peered":"2023-05-20T01:16:32.363545+0000","last_unstale":"2023-05-20T01:16:32.774853+0000","last_undegraded":"2023-05-20T01:16:32.774853+0000","last_fullsized":"2023-05-20T01:16:32.774853+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:15:31.121265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2,0],"acting":[4,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"20'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.774070+0000","last_change":"2023-05-20T01:16:27.924758+0000","last_active":"2023-05-20T01:16:32.774070+0000","last_peered":"2023-05-20T01:16:32.774070+0000","last_clean":"2023-05-20T01:16:32.774070+0000","last_became_active":"2023-05-20T01:16:27.924471+0000","last_became_peered":"2023-05-20T01:16:27.924471+0000","last_unstale":"2023-05-20T01:16:32.774070+0000","last_undegraded":"2023-05-20T01:16:32.774070+0000","last_fullsized":"2023-05-20T01:16:32.774070+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:40:53.545036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.772324+0000","last_change":"2023-05-20T01:16:28.340516+0000","last_active":"2023-05-20T01:16:32.772324+0000","last_peered":"2023-05-20T01:16:32.772324+0000","last_clean":"2023-05-20T01:16:32.772324+0000","last_became_active":"2023-05-20T01:16:28.340255+0000","last_became_peered":"2023-05-20T01:16:28.340255+0000","last_unstale":"2023-05-20T01:16:32.772324+0000","last_undegraded":"2023-05-20T01:16:32.772324+0000","last_fullsized":"2023-05-20T01:16:32.772324+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:17:27.361087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.774064+0000","last_change":"2023-05-20T01:16:32.348270+0000","last_active":"2023-05-20T01:16:32.774064+0000","last_peered":"2023-05-20T01:16:32.774064+0000","last_clean":"2023-05-20T01:16:32.774064+0000","last_became_active":"2023-05-20T01:16:32.347460+0000","last_became_peered":"2023-05-20T01:16:32.347460+0000","last_unstale":"2023-05-20T01:16:32.774064+0000","last_undegraded":"2023-05-20T01:16:32.774064+0000","last_fullsized":"2023-05-20T01:16:32.774064+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:49:18.438950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3,7],"acting":[6,4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.956713+0000","last_change":"2023-05-20T01:16:27.922074+0000","last_active":"2023-05-20T01:16:32.956713+0000","last_peered":"2023-05-20T01:16:32.956713+0000","last_clean":"2023-05-20T01:16:32.956713+0000","last_became_active":"2023-05-20T01:16:27.921759+0000","last_became_peered":"2023-05-20T01:16:27.921759+0000","last_unstale":"2023-05-20T01:16:32.956713+0000","last_undegraded":"2023-05-20T01:16:32.956713+0000","last_fullsized":"2023-05-20T01:16:32.956713+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:08:26.231228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.954625+0000","last_change":"2023-05-20T01:16:28.348595+0000","last_active":"2023-05-20T01:16:32.954625+0000","last_peered":"2023-05-20T01:16:32.954625+0000","last_clean":"2023-05-20T01:16:32.954625+0000","last_became_active":"2023-05-20T01:16:28.347911+0000","last_became_peered":"2023-05-20T01:16:28.347911+0000","last_unstale":"2023-05-20T01:16:32.954625+0000","last_undegraded":"2023-05-20T01:16:32.954625+0000","last_fullsized":"2023-05-20T01:16:32.954625+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:46:42.721514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.954742+0000","last_change":"2023-05-20T01:16:32.362037+0000","last_active":"2023-05-20T01:16:32.954742+0000","last_peered":"2023-05-20T01:16:32.954742+0000","last_clean":"2023-05-20T01:16:32.954742+0000","last_became_active":"2023-05-20T01:16:32.361564+0000","last_became_peered":"2023-05-20T01:16:32.361564+0000","last_unstale":"2023-05-20T01:16:32.954742+0000","last_undegraded":"2023-05-20T01:16:32.954742+0000","last_fullsized":"2023-05-20T01:16:32.954742+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:14:35.048109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1,2],"acting":[0,7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.771855+0000","last_change":"2023-05-20T01:16:27.910268+0000","last_active":"2023-05-20T01:16:32.771855+0000","last_peered":"2023-05-20T01:16:32.771855+0000","last_clean":"2023-05-20T01:16:32.771855+0000","last_became_active":"2023-05-20T01:16:27.909963+0000","last_became_peered":"2023-05-20T01:16:27.909963+0000","last_unstale":"2023-05-20T01:16:32.771855+0000","last_undegraded":"2023-05-20T01:16:32.771855+0000","last_fullsized":"2023-05-20T01:16:32.771855+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:22:46.049477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.774156+0000","last_change":"2023-05-20T01:16:28.345078+0000","last_active":"2023-05-20T01:16:32.774156+0000","last_peered":"2023-05-20T01:16:32.774156+0000","last_clean":"2023-05-20T01:16:32.774156+0000","last_became_active":"2023-05-20T01:16:28.344742+0000","last_became_peered":"2023-05-20T01:16:28.344742+0000","last_unstale":"2023-05-20T01:16:32.774156+0000","last_undegraded":"2023-05-20T01:16:32.774156+0000","last_fullsized":"2023-05-20T01:16:32.774156+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:53:18.904132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.773906+0000","last_change":"2023-05-20T01:16:32.348805+0000","last_active":"2023-05-20T01:16:32.773906+0000","last_peered":"2023-05-20T01:16:32.773906+0000","last_clean":"2023-05-20T01:16:32.773906+0000","last_became_active":"2023-05-20T01:16:32.348352+0000","last_became_peered":"2023-05-20T01:16:32.348352+0000","last_unstale":"2023-05-20T01:16:32.773906+0000","last_undegraded":"2023-05-20T01:16:32.773906+0000","last_fullsized":"2023-05-20T01:16:32.773906+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:04:37.665737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7,1],"acting":[6,4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.954805+0000","last_change":"2023-05-20T01:16:27.915492+0000","last_active":"2023-05-20T01:16:32.954805+0000","last_peered":"2023-05-20T01:16:32.954805+0000","last_clean":"2023-05-20T01:16:32.954805+0000","last_became_active":"2023-05-20T01:16:27.915015+0000","last_became_peered":"2023-05-20T01:16:27.915015+0000","last_unstale":"2023-05-20T01:16:32.954805+0000","last_undegraded":"2023-05-20T01:16:32.954805+0000","last_fullsized":"2023-05-20T01:16:32.954805+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:25:21.983559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.771743+0000","last_change":"2023-05-20T01:16:28.339100+0000","last_active":"2023-05-20T01:16:32.771743+0000","last_peered":"2023-05-20T01:16:32.771743+0000","last_clean":"2023-05-20T01:16:32.771743+0000","last_became_active":"2023-05-20T01:16:28.338829+0000","last_became_peered":"2023-05-20T01:16:28.338829+0000","last_unstale":"2023-05-20T01:16:32.771743+0000","last_undegraded":"2023-05-20T01:16:32.771743+0000","last_fullsized":"2023-05-20T01:16:32.771743+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:38:13.734511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.310111+0000","last_change":"2023-05-20T01:16:32.364639+0000","last_active":"2023-05-20T01:16:33.310111+0000","last_peered":"2023-05-20T01:16:33.310111+0000","last_clean":"2023-05-20T01:16:33.310111+0000","last_became_active":"2023-05-20T01:16:32.364259+0000","last_became_peered":"2023-05-20T01:16:32.364259+0000","last_unstale":"2023-05-20T01:16:33.310111+0000","last_undegraded":"2023-05-20T01:16:33.310111+0000","last_fullsized":"2023-05-20T01:16:33.310111+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:58:39.806550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6,0],"acting":[7,4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.310717+0000","last_change":"2023-05-20T01:16:27.914406+0000","last_active":"2023-05-20T01:16:33.310717+0000","last_peered":"2023-05-20T01:16:33.310717+0000","last_clean":"2023-05-20T01:16:33.310717+0000","last_became_active":"2023-05-20T01:16:27.913932+0000","last_became_peered":"2023-05-20T01:16:27.913932+0000","last_unstale":"2023-05-20T01:16:33.310717+0000","last_undegraded":"2023-05-20T01:16:33.310717+0000","last_fullsized":"2023-05-20T01:16:33.310717+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:58:35.370081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.771638+0000","last_change":"2023-05-20T01:16:28.330729+0000","last_active":"2023-05-20T01:16:32.771638+0000","last_peered":"2023-05-20T01:16:32.771638+0000","last_clean":"2023-05-20T01:16:32.771638+0000","last_became_active":"2023-05-20T01:16:28.330474+0000","last_became_peered":"2023-05-20T01:16:28.330474+0000","last_unstale":"2023-05-20T01:16:32.771638+0000","last_undegraded":"2023-05-20T01:16:32.771638+0000","last_fullsized":"2023-05-20T01:16:32.771638+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:49:21.195547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.772392+0000","last_change":"2023-05-20T01:16:32.363881+0000","last_active":"2023-05-20T01:16:32.772392+0000","last_peered":"2023-05-20T01:16:32.772392+0000","last_clean":"2023-05-20T01:16:32.772392+0000","last_became_active":"2023-05-20T01:16:32.363269+0000","last_became_peered":"2023-05-20T01:16:32.363269+0000","last_unstale":"2023-05-20T01:16:32.772392+0000","last_undegraded":"2023-05-20T01:16:32.772392+0000","last_fullsized":"2023-05-20T01:16:32.772392+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:06:25.331494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,5],"acting":[3,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"20'3","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.311909+0000","last_change":"2023-05-20T01:16:27.919005+0000","last_active":"2023-05-20T01:16:33.311909+0000","last_peered":"2023-05-20T01:16:33.311909+0000","last_clean":"2023-05-20T01:16:33.311909+0000","last_became_active":"2023-05-20T01:16:27.918679+0000","last_became_peered":"2023-05-20T01:16:27.918679+0000","last_unstale":"2023-05-20T01:16:33.311909+0000","last_undegraded":"2023-05-20T01:16:33.311909+0000","last_fullsized":"2023-05-20T01:16:33.311909+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:06:58.439555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.957017+0000","last_change":"2023-05-20T01:16:28.346905+0000","last_active":"2023-05-20T01:16:32.957017+0000","last_peered":"2023-05-20T01:16:32.957017+0000","last_clean":"2023-05-20T01:16:32.957017+0000","last_became_active":"2023-05-20T01:16:28.345822+0000","last_became_peered":"2023-05-20T01:16:28.345822+0000","last_unstale":"2023-05-20T01:16:32.957017+0000","last_undegraded":"2023-05-20T01:16:32.957017+0000","last_fullsized":"2023-05-20T01:16:32.957017+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:45:24.050486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.957316+0000","last_change":"2023-05-20T01:16:32.354753+0000","last_active":"2023-05-20T01:16:32.957316+0000","last_peered":"2023-05-20T01:16:32.957316+0000","last_clean":"2023-05-20T01:16:32.957316+0000","last_became_active":"2023-05-20T01:16:32.354017+0000","last_became_peered":"2023-05-20T01:16:32.354017+0000","last_unstale":"2023-05-20T01:16:32.957316+0000","last_undegraded":"2023-05-20T01:16:32.957316+0000","last_fullsized":"2023-05-20T01:16:32.957316+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:24:19.113785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3,7],"acting":[1,5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"25'9","reported_seq":30,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:37.788145+0000","last_change":"2023-05-20T01:16:27.924627+0000","last_active":"2023-05-20T01:16:37.788145+0000","last_peered":"2023-05-20T01:16:37.788145+0000","last_clean":"2023-05-20T01:16:37.788145+0000","last_became_active":"2023-05-20T01:16:27.924339+0000","last_became_peered":"2023-05-20T01:16:27.924339+0000","last_unstale":"2023-05-20T01:16:37.788145+0000","last_undegraded":"2023-05-20T01:16:37.788145+0000","last_fullsized":"2023-05-20T01:16:37.788145+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:27:33.065117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3794,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.773847+0000","last_change":"2023-05-20T01:16:28.348715+0000","last_active":"2023-05-20T01:16:32.773847+0000","last_peered":"2023-05-20T01:16:32.773847+0000","last_clean":"2023-05-20T01:16:32.773847+0000","last_became_active":"2023-05-20T01:16:28.346893+0000","last_became_peered":"2023-05-20T01:16:28.346893+0000","last_unstale":"2023-05-20T01:16:32.773847+0000","last_undegraded":"2023-05-20T01:16:32.773847+0000","last_fullsized":"2023-05-20T01:16:32.773847+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:44:10.386273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.311010+0000","last_change":"2023-05-20T01:16:32.362978+0000","last_active":"2023-05-20T01:16:33.311010+0000","last_peered":"2023-05-20T01:16:33.311010+0000","last_clean":"2023-05-20T01:16:33.311010+0000","last_became_active":"2023-05-20T01:16:32.362592+0000","last_became_peered":"2023-05-20T01:16:32.362592+0000","last_unstale":"2023-05-20T01:16:33.311010+0000","last_undegraded":"2023-05-20T01:16:33.311010+0000","last_fullsized":"2023-05-20T01:16:33.311010+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:46:46.150220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3,0],"acting":[5,1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.311751+0000","last_change":"2023-05-20T01:16:27.925307+0000","last_active":"2023-05-20T01:16:33.311751+0000","last_peered":"2023-05-20T01:16:33.311751+0000","last_clean":"2023-05-20T01:16:33.311751+0000","last_became_active":"2023-05-20T01:16:27.925044+0000","last_became_peered":"2023-05-20T01:16:27.925044+0000","last_unstale":"2023-05-20T01:16:33.311751+0000","last_undegraded":"2023-05-20T01:16:33.311751+0000","last_fullsized":"2023-05-20T01:16:33.311751+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:54:11.307738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.772200+0000","last_change":"2023-05-20T01:16:28.340299+0000","last_active":"2023-05-20T01:16:32.772200+0000","last_peered":"2023-05-20T01:16:32.772200+0000","last_clean":"2023-05-20T01:16:32.772200+0000","last_became_active":"2023-05-20T01:16:28.340002+0000","last_became_peered":"2023-05-20T01:16:28.340002+0000","last_unstale":"2023-05-20T01:16:32.772200+0000","last_undegraded":"2023-05-20T01:16:32.772200+0000","last_fullsized":"2023-05-20T01:16:32.772200+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T01:44:35.746448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.772637+0000","last_change":"2023-05-20T01:16:32.354337+0000","last_active":"2023-05-20T01:16:32.772637+0000","last_peered":"2023-05-20T01:16:32.772637+0000","last_clean":"2023-05-20T01:16:32.772637+0000","last_became_active":"2023-05-20T01:16:32.353900+0000","last_became_peered":"2023-05-20T01:16:32.353900+0000","last_unstale":"2023-05-20T01:16:32.772637+0000","last_undegraded":"2023-05-20T01:16:32.772637+0000","last_fullsized":"2023-05-20T01:16:32.772637+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:50:05.163209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2,1],"acting":[3,7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.954416+0000","last_change":"2023-05-20T01:16:27.916514+0000","last_active":"2023-05-20T01:16:32.954416+0000","last_peered":"2023-05-20T01:16:32.954416+0000","last_clean":"2023-05-20T01:16:32.954416+0000","last_became_active":"2023-05-20T01:16:27.916199+0000","last_became_peered":"2023-05-20T01:16:27.916199+0000","last_unstale":"2023-05-20T01:16:32.954416+0000","last_undegraded":"2023-05-20T01:16:32.954416+0000","last_fullsized":"2023-05-20T01:16:32.954416+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:21:51.991048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.310678+0000","last_change":"2023-05-20T01:16:28.335549+0000","last_active":"2023-05-20T01:16:33.310678+0000","last_peered":"2023-05-20T01:16:33.310678+0000","last_clean":"2023-05-20T01:16:33.310678+0000","last_became_active":"2023-05-20T01:16:28.335284+0000","last_became_peered":"2023-05-20T01:16:28.335284+0000","last_unstale":"2023-05-20T01:16:33.310678+0000","last_undegraded":"2023-05-20T01:16:33.310678+0000","last_fullsized":"2023-05-20T01:16:33.310678+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:21:52.998812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.772615+0000","last_change":"2023-05-20T01:16:32.362344+0000","last_active":"2023-05-20T01:16:32.772615+0000","last_peered":"2023-05-20T01:16:32.772615+0000","last_clean":"2023-05-20T01:16:32.772615+0000","last_became_active":"2023-05-20T01:16:32.361944+0000","last_became_peered":"2023-05-20T01:16:32.361944+0000","last_unstale":"2023-05-20T01:16:32.772615+0000","last_undegraded":"2023-05-20T01:16:32.772615+0000","last_fullsized":"2023-05-20T01:16:32.772615+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:44:02.823618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,0],"acting":[3,1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.311580+0000","last_change":"2023-05-20T01:16:27.926496+0000","last_active":"2023-05-20T01:16:33.311580+0000","last_peered":"2023-05-20T01:16:33.311580+0000","last_clean":"2023-05-20T01:16:33.311580+0000","last_became_active":"2023-05-20T01:16:27.926265+0000","last_became_peered":"2023-05-20T01:16:27.926265+0000","last_unstale":"2023-05-20T01:16:33.311580+0000","last_undegraded":"2023-05-20T01:16:33.311580+0000","last_fullsized":"2023-05-20T01:16:33.311580+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:23:32.724521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.954832+0000","last_change":"2023-05-20T01:16:28.348384+0000","last_active":"2023-05-20T01:16:32.954832+0000","last_peered":"2023-05-20T01:16:32.954832+0000","last_clean":"2023-05-20T01:16:32.954832+0000","last_became_active":"2023-05-20T01:16:28.347652+0000","last_became_peered":"2023-05-20T01:16:28.347652+0000","last_unstale":"2023-05-20T01:16:32.954832+0000","last_undegraded":"2023-05-20T01:16:32.954832+0000","last_fullsized":"2023-05-20T01:16:32.954832+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:49:21.195547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.310448+0000","last_change":"2023-05-20T01:16:32.352418+0000","last_active":"2023-05-20T01:16:33.310448+0000","last_peered":"2023-05-20T01:16:33.310448+0000","last_clean":"2023-05-20T01:16:33.310448+0000","last_became_active":"2023-05-20T01:16:32.352031+0000","last_became_peered":"2023-05-20T01:16:32.352031+0000","last_unstale":"2023-05-20T01:16:33.310448+0000","last_undegraded":"2023-05-20T01:16:33.310448+0000","last_fullsized":"2023-05-20T01:16:33.310448+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:05:24.403434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1,6],"acting":[5,3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"20'8","reported_seq":29,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.773007+0000","last_change":"2023-05-20T01:16:27.916957+0000","last_active":"2023-05-20T01:16:32.773007+0000","last_peered":"2023-05-20T01:16:32.773007+0000","last_clean":"2023-05-20T01:16:32.773007+0000","last_became_active":"2023-05-20T01:16:27.916682+0000","last_became_peered":"2023-05-20T01:16:27.916682+0000","last_unstale":"2023-05-20T01:16:32.773007+0000","last_undegraded":"2023-05-20T01:16:32.773007+0000","last_fullsized":"2023-05-20T01:16:32.773007+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:15:10.960002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.772882+0000","last_change":"2023-05-20T01:16:28.339442+0000","last_active":"2023-05-20T01:16:32.772882+0000","last_peered":"2023-05-20T01:16:32.772882+0000","last_clean":"2023-05-20T01:16:32.772882+0000","last_became_active":"2023-05-20T01:16:28.339182+0000","last_became_peered":"2023-05-20T01:16:28.339182+0000","last_unstale":"2023-05-20T01:16:32.772882+0000","last_undegraded":"2023-05-20T01:16:32.772882+0000","last_fullsized":"2023-05-20T01:16:32.772882+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:31:47.073658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.311789+0000","last_change":"2023-05-20T01:16:32.360635+0000","last_active":"2023-05-20T01:16:33.311789+0000","last_peered":"2023-05-20T01:16:33.311789+0000","last_clean":"2023-05-20T01:16:33.311789+0000","last_became_active":"2023-05-20T01:16:32.360032+0000","last_became_peered":"2023-05-20T01:16:32.360032+0000","last_unstale":"2023-05-20T01:16:33.311789+0000","last_undegraded":"2023-05-20T01:16:33.311789+0000","last_fullsized":"2023-05-20T01:16:33.311789+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:49:24.624253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3,4],"acting":[7,6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"20'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.312203+0000","last_change":"2023-05-20T01:16:27.926418+0000","last_active":"2023-05-20T01:16:33.312203+0000","last_peered":"2023-05-20T01:16:33.312203+0000","last_clean":"2023-05-20T01:16:33.312203+0000","last_became_active":"2023-05-20T01:16:27.926135+0000","last_became_peered":"2023-05-20T01:16:27.926135+0000","last_unstale":"2023-05-20T01:16:33.312203+0000","last_undegraded":"2023-05-20T01:16:33.312203+0000","last_fullsized":"2023-05-20T01:16:33.312203+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:53:31.196496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.310060+0000","last_change":"2023-05-20T01:16:28.333381+0000","last_active":"2023-05-20T01:16:33.310060+0000","last_peered":"2023-05-20T01:16:33.310060+0000","last_clean":"2023-05-20T01:16:33.310060+0000","last_became_active":"2023-05-20T01:16:28.333096+0000","last_became_peered":"2023-05-20T01:16:28.333096+0000","last_unstale":"2023-05-20T01:16:33.310060+0000","last_undegraded":"2023-05-20T01:16:33.310060+0000","last_fullsized":"2023-05-20T01:16:33.310060+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:47:13.680746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.310001+0000","last_change":"2023-05-20T01:16:32.353695+0000","last_active":"2023-05-20T01:16:33.310001+0000","last_peered":"2023-05-20T01:16:33.310001+0000","last_clean":"2023-05-20T01:16:33.310001+0000","last_became_active":"2023-05-20T01:16:32.353280+0000","last_became_peered":"2023-05-20T01:16:32.353280+0000","last_unstale":"2023-05-20T01:16:33.310001+0000","last_undegraded":"2023-05-20T01:16:33.310001+0000","last_fullsized":"2023-05-20T01:16:33.310001+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:53:47.536905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1,5],"acting":[2,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.312036+0000","last_change":"2023-05-20T01:16:27.926342+0000","last_active":"2023-05-20T01:16:33.312036+0000","last_peered":"2023-05-20T01:16:33.312036+0000","last_clean":"2023-05-20T01:16:33.312036+0000","last_became_active":"2023-05-20T01:16:27.926093+0000","last_became_peered":"2023-05-20T01:16:27.926093+0000","last_unstale":"2023-05-20T01:16:33.312036+0000","last_undegraded":"2023-05-20T01:16:33.312036+0000","last_fullsized":"2023-05-20T01:16:33.312036+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:22:46.049477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.774721+0000","last_change":"2023-05-20T01:16:28.338568+0000","last_active":"2023-05-20T01:16:32.774721+0000","last_peered":"2023-05-20T01:16:32.774721+0000","last_clean":"2023-05-20T01:16:32.774721+0000","last_became_active":"2023-05-20T01:16:28.338293+0000","last_became_peered":"2023-05-20T01:16:28.338293+0000","last_unstale":"2023-05-20T01:16:32.774721+0000","last_undegraded":"2023-05-20T01:16:32.774721+0000","last_fullsized":"2023-05-20T01:16:32.774721+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:49:07.725723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.310172+0000","last_change":"2023-05-20T01:16:32.358154+0000","last_active":"2023-05-20T01:16:33.310172+0000","last_peered":"2023-05-20T01:16:33.310172+0000","last_clean":"2023-05-20T01:16:33.310172+0000","last_became_active":"2023-05-20T01:16:32.357690+0000","last_became_peered":"2023-05-20T01:16:32.357690+0000","last_unstale":"2023-05-20T01:16:33.310172+0000","last_undegraded":"2023-05-20T01:16:33.310172+0000","last_fullsized":"2023-05-20T01:16:33.310172+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:04:33.803901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5,6],"acting":[2,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"20'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.311721+0000","last_change":"2023-05-20T01:16:27.922534+0000","last_active":"2023-05-20T01:16:33.311721+0000","last_peered":"2023-05-20T01:16:33.311721+0000","last_clean":"2023-05-20T01:16:33.311721+0000","last_became_active":"2023-05-20T01:16:27.922263+0000","last_became_peered":"2023-05-20T01:16:27.922263+0000","last_unstale":"2023-05-20T01:16:33.311721+0000","last_undegraded":"2023-05-20T01:16:33.311721+0000","last_fullsized":"2023-05-20T01:16:33.311721+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:25:21.983559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.773343+0000","last_change":"2023-05-20T01:16:28.345113+0000","last_active":"2023-05-20T01:16:32.773343+0000","last_peered":"2023-05-20T01:16:32.773343+0000","last_clean":"2023-05-20T01:16:32.773343+0000","last_became_active":"2023-05-20T01:16:28.344807+0000","last_became_peered":"2023-05-20T01:16:28.344807+0000","last_unstale":"2023-05-20T01:16:32.773343+0000","last_undegraded":"2023-05-20T01:16:32.773343+0000","last_fullsized":"2023-05-20T01:16:32.773343+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:31:16.197740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.310276+0000","last_change":"2023-05-20T01:16:32.346858+0000","last_active":"2023-05-20T01:16:33.310276+0000","last_peered":"2023-05-20T01:16:33.310276+0000","last_clean":"2023-05-20T01:16:33.310276+0000","last_became_active":"2023-05-20T01:16:32.346456+0000","last_became_peered":"2023-05-20T01:16:32.346456+0000","last_unstale":"2023-05-20T01:16:33.310276+0000","last_undegraded":"2023-05-20T01:16:33.310276+0000","last_fullsized":"2023-05-20T01:16:33.310276+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:30:20.804962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3,6],"acting":[2,4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.310878+0000","last_change":"2023-05-20T01:16:27.912388+0000","last_active":"2023-05-20T01:16:33.310878+0000","last_peered":"2023-05-20T01:16:33.310878+0000","last_clean":"2023-05-20T01:16:33.310878+0000","last_became_active":"2023-05-20T01:16:27.911977+0000","last_became_peered":"2023-05-20T01:16:27.911977+0000","last_unstale":"2023-05-20T01:16:33.310878+0000","last_undegraded":"2023-05-20T01:16:33.310878+0000","last_fullsized":"2023-05-20T01:16:33.310878+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:19:50.512317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.954435+0000","last_change":"2023-05-20T01:16:28.342633+0000","last_active":"2023-05-20T01:16:32.954435+0000","last_peered":"2023-05-20T01:16:32.954435+0000","last_clean":"2023-05-20T01:16:32.954435+0000","last_became_active":"2023-05-20T01:16:28.339419+0000","last_became_peered":"2023-05-20T01:16:28.339419+0000","last_unstale":"2023-05-20T01:16:32.954435+0000","last_undegraded":"2023-05-20T01:16:32.954435+0000","last_fullsized":"2023-05-20T01:16:32.954435+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:27:57.460519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.310304+0000","last_change":"2023-05-20T01:16:32.347930+0000","last_active":"2023-05-20T01:16:33.310304+0000","last_peered":"2023-05-20T01:16:33.310304+0000","last_clean":"2023-05-20T01:16:33.310304+0000","last_became_active":"2023-05-20T01:16:32.347502+0000","last_became_peered":"2023-05-20T01:16:32.347502+0000","last_unstale":"2023-05-20T01:16:33.310304+0000","last_undegraded":"2023-05-20T01:16:33.310304+0000","last_fullsized":"2023-05-20T01:16:33.310304+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:05:15.989168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5,6],"acting":[2,7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.772049+0000","last_change":"2023-05-20T01:16:27.909757+0000","last_active":"2023-05-20T01:16:32.772049+0000","last_peered":"2023-05-20T01:16:32.772049+0000","last_clean":"2023-05-20T01:16:32.772049+0000","last_became_active":"2023-05-20T01:16:27.909458+0000","last_became_peered":"2023-05-20T01:16:27.909458+0000","last_unstale":"2023-05-20T01:16:32.772049+0000","last_undegraded":"2023-05-20T01:16:32.772049+0000","last_fullsized":"2023-05-20T01:16:32.772049+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:26:58.729159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.773572+0000","last_change":"2023-05-20T01:16:28.340078+0000","last_active":"2023-05-20T01:16:32.773572+0000","last_peered":"2023-05-20T01:16:32.773572+0000","last_clean":"2023-05-20T01:16:32.773572+0000","last_became_active":"2023-05-20T01:16:28.339673+0000","last_became_peered":"2023-05-20T01:16:28.339673+0000","last_unstale":"2023-05-20T01:16:32.773572+0000","last_undegraded":"2023-05-20T01:16:32.773572+0000","last_fullsized":"2023-05-20T01:16:32.773572+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:13:55.558140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.957039+0000","last_change":"2023-05-20T01:16:32.364538+0000","last_active":"2023-05-20T01:16:32.957039+0000","last_peered":"2023-05-20T01:16:32.957039+0000","last_clean":"2023-05-20T01:16:32.957039+0000","last_became_active":"2023-05-20T01:16:32.364025+0000","last_became_peered":"2023-05-20T01:16:32.364025+0000","last_unstale":"2023-05-20T01:16:32.957039+0000","last_undegraded":"2023-05-20T01:16:32.957039+0000","last_fullsized":"2023-05-20T01:16:32.957039+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:01:04.644797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0,6],"acting":[1,4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.774177+0000","last_change":"2023-05-20T01:16:27.922354+0000","last_active":"2023-05-20T01:16:32.774177+0000","last_peered":"2023-05-20T01:16:32.774177+0000","last_clean":"2023-05-20T01:16:32.774177+0000","last_became_active":"2023-05-20T01:16:27.922000+0000","last_became_peered":"2023-05-20T01:16:27.922000+0000","last_unstale":"2023-05-20T01:16:32.774177+0000","last_undegraded":"2023-05-20T01:16:32.774177+0000","last_fullsized":"2023-05-20T01:16:32.774177+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:56:49.903435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.774102+0000","last_change":"2023-05-20T01:16:28.334441+0000","last_active":"2023-05-20T01:16:32.774102+0000","last_peered":"2023-05-20T01:16:32.774102+0000","last_clean":"2023-05-20T01:16:32.774102+0000","last_became_active":"2023-05-20T01:16:28.334157+0000","last_became_peered":"2023-05-20T01:16:28.334157+0000","last_unstale":"2023-05-20T01:16:32.774102+0000","last_undegraded":"2023-05-20T01:16:32.774102+0000","last_fullsized":"2023-05-20T01:16:32.774102+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:51:16.225196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.310897+0000","last_change":"2023-05-20T01:16:32.346834+0000","last_active":"2023-05-20T01:16:33.310897+0000","last_peered":"2023-05-20T01:16:33.310897+0000","last_clean":"2023-05-20T01:16:33.310897+0000","last_became_active":"2023-05-20T01:16:32.346410+0000","last_became_peered":"2023-05-20T01:16:32.346410+0000","last_unstale":"2023-05-20T01:16:33.310897+0000","last_undegraded":"2023-05-20T01:16:33.310897+0000","last_fullsized":"2023-05-20T01:16:33.310897+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:07:08.123139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6,2],"acting":[5,4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.774525+0000","last_change":"2023-05-20T01:16:27.921658+0000","last_active":"2023-05-20T01:16:32.774525+0000","last_peered":"2023-05-20T01:16:32.774525+0000","last_clean":"2023-05-20T01:16:32.774525+0000","last_became_active":"2023-05-20T01:16:27.921399+0000","last_became_peered":"2023-05-20T01:16:27.921399+0000","last_unstale":"2023-05-20T01:16:32.774525+0000","last_undegraded":"2023-05-20T01:16:32.774525+0000","last_fullsized":"2023-05-20T01:16:32.774525+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:43:09.775404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.775593+0000","last_change":"2023-05-20T01:16:28.347570+0000","last_active":"2023-05-20T01:16:32.775593+0000","last_peered":"2023-05-20T01:16:32.775593+0000","last_clean":"2023-05-20T01:16:32.775593+0000","last_became_active":"2023-05-20T01:16:28.347140+0000","last_became_peered":"2023-05-20T01:16:28.347140+0000","last_unstale":"2023-05-20T01:16:32.775593+0000","last_undegraded":"2023-05-20T01:16:32.775593+0000","last_fullsized":"2023-05-20T01:16:32.775593+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T01:30:46.937626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.773539+0000","last_change":"2023-05-20T01:16:32.363537+0000","last_active":"2023-05-20T01:16:32.773539+0000","last_peered":"2023-05-20T01:16:32.773539+0000","last_clean":"2023-05-20T01:16:32.773539+0000","last_became_active":"2023-05-20T01:16:32.363119+0000","last_became_peered":"2023-05-20T01:16:32.363119+0000","last_unstale":"2023-05-20T01:16:32.773539+0000","last_undegraded":"2023-05-20T01:16:32.773539+0000","last_fullsized":"2023-05-20T01:16:32.773539+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:40:04.730139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0,2],"acting":[4,5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.771744+0000","last_change":"2023-05-20T01:16:27.924294+0000","last_active":"2023-05-20T01:16:32.771744+0000","last_peered":"2023-05-20T01:16:32.771744+0000","last_clean":"2023-05-20T01:16:32.771744+0000","last_became_active":"2023-05-20T01:16:27.923930+0000","last_became_peered":"2023-05-20T01:16:27.923930+0000","last_unstale":"2023-05-20T01:16:32.771744+0000","last_undegraded":"2023-05-20T01:16:32.771744+0000","last_fullsized":"2023-05-20T01:16:32.771744+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:24:35.039391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.957544+0000","last_change":"2023-05-20T01:16:28.349390+0000","last_active":"2023-05-20T01:16:32.957544+0000","last_peered":"2023-05-20T01:16:32.957544+0000","last_clean":"2023-05-20T01:16:32.957544+0000","last_became_active":"2023-05-20T01:16:28.349074+0000","last_became_peered":"2023-05-20T01:16:28.349074+0000","last_unstale":"2023-05-20T01:16:32.957544+0000","last_undegraded":"2023-05-20T01:16:32.957544+0000","last_fullsized":"2023-05-20T01:16:32.957544+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:59:48.441604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.774884+0000","last_change":"2023-05-20T01:16:32.353210+0000","last_active":"2023-05-20T01:16:32.774884+0000","last_peered":"2023-05-20T01:16:32.774884+0000","last_clean":"2023-05-20T01:16:32.774884+0000","last_became_active":"2023-05-20T01:16:32.352807+0000","last_became_peered":"2023-05-20T01:16:32.352807+0000","last_unstale":"2023-05-20T01:16:32.774884+0000","last_undegraded":"2023-05-20T01:16:32.774884+0000","last_fullsized":"2023-05-20T01:16:32.774884+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:42:22.830436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7,0],"acting":[4,3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.772776+0000","last_change":"2023-05-20T01:16:27.913270+0000","last_active":"2023-05-20T01:16:32.772776+0000","last_peered":"2023-05-20T01:16:32.772776+0000","last_clean":"2023-05-20T01:16:32.772776+0000","last_became_active":"2023-05-20T01:16:27.912952+0000","last_became_peered":"2023-05-20T01:16:27.912952+0000","last_unstale":"2023-05-20T01:16:32.772776+0000","last_undegraded":"2023-05-20T01:16:32.772776+0000","last_fullsized":"2023-05-20T01:16:32.772776+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:34:35.188240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.774248+0000","last_change":"2023-05-20T01:16:23.289068+0000","last_active":"2023-05-20T01:16:32.774248+0000","last_peered":"2023-05-20T01:16:32.774248+0000","last_clean":"2023-05-20T01:16:32.774248+0000","last_became_active":"2023-05-20T01:16:23.288781+0000","last_became_peered":"2023-05-20T01:16:23.288781+0000","last_unstale":"2023-05-20T01:16:32.774248+0000","last_undegraded":"2023-05-20T01:16:32.774248+0000","last_fullsized":"2023-05-20T01:16:32.774248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:02:03.239372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.771607+0000","last_change":"2023-05-20T01:16:28.329739+0000","last_active":"2023-05-20T01:16:32.771607+0000","last_peered":"2023-05-20T01:16:32.771607+0000","last_clean":"2023-05-20T01:16:32.771607+0000","last_became_active":"2023-05-20T01:16:28.329483+0000","last_became_peered":"2023-05-20T01:16:28.329483+0000","last_unstale":"2023-05-20T01:16:32.771607+0000","last_undegraded":"2023-05-20T01:16:32.771607+0000","last_fullsized":"2023-05-20T01:16:32.771607+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:46:14.388881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.771640+0000","last_change":"2023-05-20T01:16:32.353144+0000","last_active":"2023-05-20T01:16:32.771640+0000","last_peered":"2023-05-20T01:16:32.771640+0000","last_clean":"2023-05-20T01:16:32.771640+0000","last_became_active":"2023-05-20T01:16:32.352355+0000","last_became_peered":"2023-05-20T01:16:32.352355+0000","last_unstale":"2023-05-20T01:16:32.771640+0000","last_undegraded":"2023-05-20T01:16:32.771640+0000","last_fullsized":"2023-05-20T01:16:32.771640+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:25:26.420028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,0],"acting":[3,1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.772224+0000","last_change":"2023-05-20T01:16:22.829324+0000","last_active":"2023-05-20T01:16:32.772224+0000","last_peered":"2023-05-20T01:16:32.772224+0000","last_clean":"2023-05-20T01:16:32.772224+0000","last_became_active":"2023-05-20T01:16:22.828964+0000","last_became_peered":"2023-05-20T01:16:22.828964+0000","last_unstale":"2023-05-20T01:16:32.772224+0000","last_undegraded":"2023-05-20T01:16:32.772224+0000","last_fullsized":"2023-05-20T01:16:32.772224+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:33:22.739978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"25'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.954588+0000","last_change":"2023-05-20T01:16:27.922339+0000","last_active":"2023-05-20T01:16:32.954588+0000","last_peered":"2023-05-20T01:16:32.954588+0000","last_clean":"2023-05-20T01:16:32.954588+0000","last_became_active":"2023-05-20T01:16:27.922033+0000","last_became_peered":"2023-05-20T01:16:27.922033+0000","last_unstale":"2023-05-20T01:16:32.954588+0000","last_undegraded":"2023-05-20T01:16:32.954588+0000","last_fullsized":"2023-05-20T01:16:32.954588+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:01:20.476920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.774598+0000","last_change":"2023-05-20T01:16:28.335846+0000","last_active":"2023-05-20T01:16:32.774598+0000","last_peered":"2023-05-20T01:16:32.774598+0000","last_clean":"2023-05-20T01:16:32.774598+0000","last_became_active":"2023-05-20T01:16:28.335523+0000","last_became_peered":"2023-05-20T01:16:28.335523+0000","last_unstale":"2023-05-20T01:16:32.774598+0000","last_undegraded":"2023-05-20T01:16:32.774598+0000","last_fullsized":"2023-05-20T01:16:32.774598+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:18:05.626837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.310654+0000","last_change":"2023-05-20T01:16:32.363029+0000","last_active":"2023-05-20T01:16:33.310654+0000","last_peered":"2023-05-20T01:16:33.310654+0000","last_clean":"2023-05-20T01:16:33.310654+0000","last_became_active":"2023-05-20T01:16:32.362637+0000","last_became_peered":"2023-05-20T01:16:32.362637+0000","last_unstale":"2023-05-20T01:16:33.310654+0000","last_undegraded":"2023-05-20T01:16:33.310654+0000","last_fullsized":"2023-05-20T01:16:33.310654+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:28:00.889225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0,4],"acting":[5,1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.956901+0000","last_change":"2023-05-20T01:16:22.834754+0000","last_active":"2023-05-20T01:16:32.956901+0000","last_peered":"2023-05-20T01:16:32.956901+0000","last_clean":"2023-05-20T01:16:32.956901+0000","last_became_active":"2023-05-20T01:16:22.834437+0000","last_became_peered":"2023-05-20T01:16:22.834437+0000","last_unstale":"2023-05-20T01:16:32.956901+0000","last_undegraded":"2023-05-20T01:16:32.956901+0000","last_fullsized":"2023-05-20T01:16:32.956901+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:51:36.657132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":177,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.311443+0000","last_change":"2023-05-20T01:16:22.834920+0000","last_active":"2023-05-20T01:16:33.311443+0000","last_peered":"2023-05-20T01:16:33.311443+0000","last_clean":"2023-05-20T01:16:33.311443+0000","last_became_active":"2023-05-20T01:16:22.833924+0000","last_became_peered":"2023-05-20T01:16:22.833924+0000","last_unstale":"2023-05-20T01:16:33.311443+0000","last_undegraded":"2023-05-20T01:16:33.311443+0000","last_fullsized":"2023-05-20T01:16:33.311443+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:34:02.513729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.954099+0000","last_change":"2023-05-20T01:16:27.923011+0000","last_active":"2023-05-20T01:16:32.954099+0000","last_peered":"2023-05-20T01:16:32.954099+0000","last_clean":"2023-05-20T01:16:32.954099+0000","last_became_active":"2023-05-20T01:16:27.922736+0000","last_became_peered":"2023-05-20T01:16:27.922736+0000","last_unstale":"2023-05-20T01:16:32.954099+0000","last_undegraded":"2023-05-20T01:16:32.954099+0000","last_fullsized":"2023-05-20T01:16:32.954099+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:59:47.433840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.954829+0000","last_change":"2023-05-20T01:16:28.348199+0000","last_active":"2023-05-20T01:16:32.954829+0000","last_peered":"2023-05-20T01:16:32.954829+0000","last_clean":"2023-05-20T01:16:32.954829+0000","last_became_active":"2023-05-20T01:16:28.347370+0000","last_became_peered":"2023-05-20T01:16:28.347370+0000","last_unstale":"2023-05-20T01:16:32.954829+0000","last_undegraded":"2023-05-20T01:16:32.954829+0000","last_fullsized":"2023-05-20T01:16:32.954829+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:46:14.388881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.310467+0000","last_change":"2023-05-20T01:16:32.357823+0000","last_active":"2023-05-20T01:16:33.310467+0000","last_peered":"2023-05-20T01:16:33.310467+0000","last_clean":"2023-05-20T01:16:33.310467+0000","last_became_active":"2023-05-20T01:16:32.357437+0000","last_became_peered":"2023-05-20T01:16:32.357437+0000","last_unstale":"2023-05-20T01:16:33.310467+0000","last_undegraded":"2023-05-20T01:16:33.310467+0000","last_fullsized":"2023-05-20T01:16:33.310467+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:16:08.791218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1,7],"acting":[2,5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.311398+0000","last_change":"2023-05-20T01:16:23.290259+0000","last_active":"2023-05-20T01:16:33.311398+0000","last_peered":"2023-05-20T01:16:33.311398+0000","last_clean":"2023-05-20T01:16:33.311398+0000","last_became_active":"2023-05-20T01:16:23.289911+0000","last_became_peered":"2023-05-20T01:16:23.289911+0000","last_unstale":"2023-05-20T01:16:33.311398+0000","last_undegraded":"2023-05-20T01:16:33.311398+0000","last_fullsized":"2023-05-20T01:16:33.311398+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:46:36.633677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.956681+0000","last_change":"2023-05-20T01:16:27.919248+0000","last_active":"2023-05-20T01:16:32.956681+0000","last_peered":"2023-05-20T01:16:32.956681+0000","last_clean":"2023-05-20T01:16:32.956681+0000","last_became_active":"2023-05-20T01:16:27.918957+0000","last_became_peered":"2023-05-20T01:16:27.918957+0000","last_unstale":"2023-05-20T01:16:32.956681+0000","last_undegraded":"2023-05-20T01:16:32.956681+0000","last_fullsized":"2023-05-20T01:16:32.956681+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:51:16.461763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.957471+0000","last_change":"2023-05-20T01:16:28.337136+0000","last_active":"2023-05-20T01:16:32.957471+0000","last_peered":"2023-05-20T01:16:32.957471+0000","last_clean":"2023-05-20T01:16:32.957471+0000","last_became_active":"2023-05-20T01:16:28.336322+0000","last_became_peered":"2023-05-20T01:16:28.336322+0000","last_unstale":"2023-05-20T01:16:32.957471+0000","last_undegraded":"2023-05-20T01:16:32.957471+0000","last_fullsized":"2023-05-20T01:16:32.957471+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:42:59.927187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.311713+0000","last_change":"2023-05-20T01:16:32.358840+0000","last_active":"2023-05-20T01:16:33.311713+0000","last_peered":"2023-05-20T01:16:33.311713+0000","last_clean":"2023-05-20T01:16:33.311713+0000","last_became_active":"2023-05-20T01:16:32.358470+0000","last_became_peered":"2023-05-20T01:16:32.358470+0000","last_unstale":"2023-05-20T01:16:33.311713+0000","last_undegraded":"2023-05-20T01:16:33.311713+0000","last_fullsized":"2023-05-20T01:16:33.311713+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:14:35.048109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5,3],"acting":[7,2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.311131+0000","last_change":"2023-05-20T01:16:23.290207+0000","last_active":"2023-05-20T01:16:33.311131+0000","last_peered":"2023-05-20T01:16:33.311131+0000","last_clean":"2023-05-20T01:16:33.311131+0000","last_became_active":"2023-05-20T01:16:23.289928+0000","last_became_peered":"2023-05-20T01:16:23.289928+0000","last_unstale":"2023-05-20T01:16:33.311131+0000","last_undegraded":"2023-05-20T01:16:33.311131+0000","last_fullsized":"2023-05-20T01:16:33.311131+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:01:15.396847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.771970+0000","last_change":"2023-05-20T01:16:27.911483+0000","last_active":"2023-05-20T01:16:32.771970+0000","last_peered":"2023-05-20T01:16:32.771970+0000","last_clean":"2023-05-20T01:16:32.771970+0000","last_became_active":"2023-05-20T01:16:27.911180+0000","last_became_peered":"2023-05-20T01:16:27.911180+0000","last_unstale":"2023-05-20T01:16:32.771970+0000","last_undegraded":"2023-05-20T01:16:32.771970+0000","last_fullsized":"2023-05-20T01:16:32.771970+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:13:48.951321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.773506+0000","last_change":"2023-05-20T01:16:28.345161+0000","last_active":"2023-05-20T01:16:32.773506+0000","last_peered":"2023-05-20T01:16:32.773506+0000","last_clean":"2023-05-20T01:16:32.773506+0000","last_became_active":"2023-05-20T01:16:28.344872+0000","last_became_peered":"2023-05-20T01:16:28.344872+0000","last_unstale":"2023-05-20T01:16:32.773506+0000","last_undegraded":"2023-05-20T01:16:32.773506+0000","last_fullsized":"2023-05-20T01:16:32.773506+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:49:51.917546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.954428+0000","last_change":"2023-05-20T01:16:32.362192+0000","last_active":"2023-05-20T01:16:32.954428+0000","last_peered":"2023-05-20T01:16:32.954428+0000","last_clean":"2023-05-20T01:16:32.954428+0000","last_became_active":"2023-05-20T01:16:32.361656+0000","last_became_peered":"2023-05-20T01:16:32.361656+0000","last_unstale":"2023-05-20T01:16:32.954428+0000","last_undegraded":"2023-05-20T01:16:32.954428+0000","last_fullsized":"2023-05-20T01:16:32.954428+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:58:35.944694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,6],"acting":[0,1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.310220+0000","last_change":"2023-05-20T01:16:23.290538+0000","last_active":"2023-05-20T01:16:33.310220+0000","last_peered":"2023-05-20T01:16:33.310220+0000","last_clean":"2023-05-20T01:16:33.310220+0000","last_became_active":"2023-05-20T01:16:23.290230+0000","last_became_peered":"2023-05-20T01:16:23.290230+0000","last_unstale":"2023-05-20T01:16:33.310220+0000","last_undegraded":"2023-05-20T01:16:33.310220+0000","last_fullsized":"2023-05-20T01:16:33.310220+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:24:09.597242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.773435+0000","last_change":"2023-05-20T01:16:27.921749+0000","last_active":"2023-05-20T01:16:32.773435+0000","last_peered":"2023-05-20T01:16:32.773435+0000","last_clean":"2023-05-20T01:16:32.773435+0000","last_became_active":"2023-05-20T01:16:27.921461+0000","last_became_peered":"2023-05-20T01:16:27.921461+0000","last_unstale":"2023-05-20T01:16:32.773435+0000","last_undegraded":"2023-05-20T01:16:32.773435+0000","last_fullsized":"2023-05-20T01:16:32.773435+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:01:30.293877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.957033+0000","last_change":"2023-05-20T01:16:28.335769+0000","last_active":"2023-05-20T01:16:32.957033+0000","last_peered":"2023-05-20T01:16:32.957033+0000","last_clean":"2023-05-20T01:16:32.957033+0000","last_became_active":"2023-05-20T01:16:28.335232+0000","last_became_peered":"2023-05-20T01:16:28.335232+0000","last_unstale":"2023-05-20T01:16:32.957033+0000","last_undegraded":"2023-05-20T01:16:32.957033+0000","last_fullsized":"2023-05-20T01:16:32.957033+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:23:18.405128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.774369+0000","last_change":"2023-05-20T01:16:32.354871+0000","last_active":"2023-05-20T01:16:32.774369+0000","last_peered":"2023-05-20T01:16:32.774369+0000","last_clean":"2023-05-20T01:16:32.774369+0000","last_became_active":"2023-05-20T01:16:32.353771+0000","last_became_peered":"2023-05-20T01:16:32.353771+0000","last_unstale":"2023-05-20T01:16:32.774369+0000","last_undegraded":"2023-05-20T01:16:32.774369+0000","last_fullsized":"2023-05-20T01:16:32.774369+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:22:02.793310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5,2],"acting":[6,0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.773241+0000","last_change":"2023-05-20T01:16:23.290341+0000","last_active":"2023-05-20T01:16:32.773241+0000","last_peered":"2023-05-20T01:16:32.773241+0000","last_clean":"2023-05-20T01:16:32.773241+0000","last_became_active":"2023-05-20T01:16:23.290053+0000","last_became_peered":"2023-05-20T01:16:23.290053+0000","last_unstale":"2023-05-20T01:16:32.773241+0000","last_undegraded":"2023-05-20T01:16:32.773241+0000","last_fullsized":"2023-05-20T01:16:32.773241+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:11:18.465994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.957177+0000","last_change":"2023-05-20T01:16:27.920036+0000","last_active":"2023-05-20T01:16:32.957177+0000","last_peered":"2023-05-20T01:16:32.957177+0000","last_clean":"2023-05-20T01:16:32.957177+0000","last_became_active":"2023-05-20T01:16:27.919757+0000","last_became_peered":"2023-05-20T01:16:27.919757+0000","last_unstale":"2023-05-20T01:16:32.957177+0000","last_undegraded":"2023-05-20T01:16:32.957177+0000","last_fullsized":"2023-05-20T01:16:32.957177+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:49:54.418261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.954564+0000","last_change":"2023-05-20T01:16:28.343036+0000","last_active":"2023-05-20T01:16:32.954564+0000","last_peered":"2023-05-20T01:16:32.954564+0000","last_clean":"2023-05-20T01:16:32.954564+0000","last_became_active":"2023-05-20T01:16:28.342814+0000","last_became_peered":"2023-05-20T01:16:28.342814+0000","last_unstale":"2023-05-20T01:16:32.954564+0000","last_undegraded":"2023-05-20T01:16:32.954564+0000","last_fullsized":"2023-05-20T01:16:32.954564+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:13:57.518009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.954367+0000","last_change":"2023-05-20T01:16:32.356221+0000","last_active":"2023-05-20T01:16:32.954367+0000","last_peered":"2023-05-20T01:16:32.954367+0000","last_clean":"2023-05-20T01:16:32.954367+0000","last_became_active":"2023-05-20T01:16:32.355772+0000","last_became_peered":"2023-05-20T01:16:32.355772+0000","last_unstale":"2023-05-20T01:16:32.954367+0000","last_undegraded":"2023-05-20T01:16:32.954367+0000","last_fullsized":"2023-05-20T01:16:32.954367+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:07:08.123139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6,5],"acting":[0,1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.774068+0000","last_change":"2023-05-20T01:16:22.834330+0000","last_active":"2023-05-20T01:16:32.774068+0000","last_peered":"2023-05-20T01:16:32.774068+0000","last_clean":"2023-05-20T01:16:32.774068+0000","last_became_active":"2023-05-20T01:16:22.833946+0000","last_became_peered":"2023-05-20T01:16:22.833946+0000","last_unstale":"2023-05-20T01:16:32.774068+0000","last_undegraded":"2023-05-20T01:16:32.774068+0000","last_fullsized":"2023-05-20T01:16:32.774068+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:22:15.807596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.310582+0000","last_change":"2023-05-20T01:16:27.914231+0000","last_active":"2023-05-20T01:16:33.310582+0000","last_peered":"2023-05-20T01:16:33.310582+0000","last_clean":"2023-05-20T01:16:33.310582+0000","last_became_active":"2023-05-20T01:16:27.913946+0000","last_became_peered":"2023-05-20T01:16:27.913946+0000","last_unstale":"2023-05-20T01:16:33.310582+0000","last_undegraded":"2023-05-20T01:16:33.310582+0000","last_fullsized":"2023-05-20T01:16:33.310582+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:58:31.508225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.956705+0000","last_change":"2023-05-20T01:16:28.336767+0000","last_active":"2023-05-20T01:16:32.956705+0000","last_peered":"2023-05-20T01:16:32.956705+0000","last_clean":"2023-05-20T01:16:32.956705+0000","last_became_active":"2023-05-20T01:16:28.336172+0000","last_became_peered":"2023-05-20T01:16:28.336172+0000","last_unstale":"2023-05-20T01:16:32.956705+0000","last_undegraded":"2023-05-20T01:16:32.956705+0000","last_fullsized":"2023-05-20T01:16:32.956705+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:45:17.217540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.957277+0000","last_change":"2023-05-20T01:16:32.364585+0000","last_active":"2023-05-20T01:16:32.957277+0000","last_peered":"2023-05-20T01:16:32.957277+0000","last_clean":"2023-05-20T01:16:32.957277+0000","last_became_active":"2023-05-20T01:16:32.364096+0000","last_became_peered":"2023-05-20T01:16:32.364096+0000","last_unstale":"2023-05-20T01:16:32.957277+0000","last_undegraded":"2023-05-20T01:16:32.957277+0000","last_fullsized":"2023-05-20T01:16:32.957277+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:47:17.109451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,0],"acting":[1,3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"20'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.954518+0000","last_change":"2023-05-20T01:16:27.922900+0000","last_active":"2023-05-20T01:16:32.954518+0000","last_peered":"2023-05-20T01:16:32.954518+0000","last_clean":"2023-05-20T01:16:32.954518+0000","last_became_active":"2023-05-20T01:16:27.922597+0000","last_became_peered":"2023-05-20T01:16:27.922597+0000","last_unstale":"2023-05-20T01:16:32.954518+0000","last_undegraded":"2023-05-20T01:16:32.954518+0000","last_fullsized":"2023-05-20T01:16:32.954518+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:58:35.370081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.310293+0000","last_change":"2023-05-20T01:16:28.341446+0000","last_active":"2023-05-20T01:16:33.310293+0000","last_peered":"2023-05-20T01:16:33.310293+0000","last_clean":"2023-05-20T01:16:33.310293+0000","last_became_active":"2023-05-20T01:16:28.340943+0000","last_became_peered":"2023-05-20T01:16:28.340943+0000","last_unstale":"2023-05-20T01:16:33.310293+0000","last_undegraded":"2023-05-20T01:16:33.310293+0000","last_fullsized":"2023-05-20T01:16:33.310293+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:24:03.794336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.311921+0000","last_change":"2023-05-20T01:16:32.360851+0000","last_active":"2023-05-20T01:16:33.311921+0000","last_peered":"2023-05-20T01:16:33.311921+0000","last_clean":"2023-05-20T01:16:33.311921+0000","last_became_active":"2023-05-20T01:16:32.360366+0000","last_became_peered":"2023-05-20T01:16:32.360366+0000","last_unstale":"2023-05-20T01:16:33.311921+0000","last_undegraded":"2023-05-20T01:16:33.311921+0000","last_fullsized":"2023-05-20T01:16:33.311921+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:38:17.163216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1,6],"acting":[7,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.310447+0000","last_change":"2023-05-20T01:16:27.925398+0000","last_active":"2023-05-20T01:16:33.310447+0000","last_peered":"2023-05-20T01:16:33.310447+0000","last_clean":"2023-05-20T01:16:33.310447+0000","last_became_active":"2023-05-20T01:16:27.924943+0000","last_became_peered":"2023-05-20T01:16:27.924943+0000","last_unstale":"2023-05-20T01:16:33.310447+0000","last_undegraded":"2023-05-20T01:16:33.310447+0000","last_fullsized":"2023-05-20T01:16:33.310447+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:42:58.919423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.773260+0000","last_change":"2023-05-20T01:16:28.332091+0000","last_active":"2023-05-20T01:16:32.773260+0000","last_peered":"2023-05-20T01:16:32.773260+0000","last_clean":"2023-05-20T01:16:32.773260+0000","last_became_active":"2023-05-20T01:16:28.331785+0000","last_became_peered":"2023-05-20T01:16:28.331785+0000","last_unstale":"2023-05-20T01:16:32.773260+0000","last_undegraded":"2023-05-20T01:16:32.773260+0000","last_fullsized":"2023-05-20T01:16:32.773260+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:31:47.073658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.310796+0000","last_change":"2023-05-20T01:16:32.362120+0000","last_active":"2023-05-20T01:16:33.310796+0000","last_peered":"2023-05-20T01:16:33.310796+0000","last_clean":"2023-05-20T01:16:33.310796+0000","last_became_active":"2023-05-20T01:16:32.361727+0000","last_became_peered":"2023-05-20T01:16:32.361727+0000","last_unstale":"2023-05-20T01:16:33.310796+0000","last_undegraded":"2023-05-20T01:16:33.310796+0000","last_fullsized":"2023-05-20T01:16:33.310796+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:01:24.913389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7,4],"acting":[5,0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.310449+0000","last_change":"2023-05-20T01:16:27.925288+0000","last_active":"2023-05-20T01:16:33.310449+0000","last_peered":"2023-05-20T01:16:33.310449+0000","last_clean":"2023-05-20T01:16:33.310449+0000","last_became_active":"2023-05-20T01:16:27.924988+0000","last_became_peered":"2023-05-20T01:16:27.924988+0000","last_unstale":"2023-05-20T01:16:33.310449+0000","last_undegraded":"2023-05-20T01:16:33.310449+0000","last_fullsized":"2023-05-20T01:16:33.310449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:32:59.792286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.773973+0000","last_change":"2023-05-20T01:16:28.334255+0000","last_active":"2023-05-20T01:16:32.773973+0000","last_peered":"2023-05-20T01:16:32.773973+0000","last_clean":"2023-05-20T01:16:32.773973+0000","last_became_active":"2023-05-20T01:16:28.333966+0000","last_became_peered":"2023-05-20T01:16:28.333966+0000","last_unstale":"2023-05-20T01:16:32.773973+0000","last_undegraded":"2023-05-20T01:16:32.773973+0000","last_fullsized":"2023-05-20T01:16:32.773973+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:04:05.601168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.772980+0000","last_change":"2023-05-20T01:16:32.346878+0000","last_active":"2023-05-20T01:16:32.772980+0000","last_peered":"2023-05-20T01:16:32.772980+0000","last_clean":"2023-05-20T01:16:32.772980+0000","last_became_active":"2023-05-20T01:16:32.346493+0000","last_became_peered":"2023-05-20T01:16:32.346493+0000","last_unstale":"2023-05-20T01:16:32.772980+0000","last_undegraded":"2023-05-20T01:16:32.772980+0000","last_fullsized":"2023-05-20T01:16:32.772980+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:50:05.163209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,2],"acting":[4,6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"20'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.772568+0000","last_change":"2023-05-20T01:16:27.906889+0000","last_active":"2023-05-20T01:16:32.772568+0000","last_peered":"2023-05-20T01:16:32.772568+0000","last_clean":"2023-05-20T01:16:32.772568+0000","last_became_active":"2023-05-20T01:16:27.906574+0000","last_became_peered":"2023-05-20T01:16:27.906574+0000","last_unstale":"2023-05-20T01:16:32.772568+0000","last_undegraded":"2023-05-20T01:16:32.772568+0000","last_fullsized":"2023-05-20T01:16:32.772568+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:49:41.443268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.771941+0000","last_change":"2023-05-20T01:16:28.345350+0000","last_active":"2023-05-20T01:16:32.771941+0000","last_peered":"2023-05-20T01:16:32.771941+0000","last_clean":"2023-05-20T01:16:32.771941+0000","last_became_active":"2023-05-20T01:16:28.344838+0000","last_became_peered":"2023-05-20T01:16:28.344838+0000","last_unstale":"2023-05-20T01:16:32.771941+0000","last_undegraded":"2023-05-20T01:16:32.771941+0000","last_fullsized":"2023-05-20T01:16:32.771941+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:53:32.204259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.956979+0000","last_change":"2023-05-20T01:16:32.352947+0000","last_active":"2023-05-20T01:16:32.956979+0000","last_peered":"2023-05-20T01:16:32.956979+0000","last_clean":"2023-05-20T01:16:32.956979+0000","last_became_active":"2023-05-20T01:16:32.352495+0000","last_became_peered":"2023-05-20T01:16:32.352495+0000","last_unstale":"2023-05-20T01:16:32.956979+0000","last_undegraded":"2023-05-20T01:16:32.956979+0000","last_fullsized":"2023-05-20T01:16:32.956979+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:29:18.453884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7,2],"acting":[1,5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"20'4","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.954845+0000","last_change":"2023-05-20T01:16:27.917645+0000","last_active":"2023-05-20T01:16:32.954845+0000","last_peered":"2023-05-20T01:16:32.954845+0000","last_clean":"2023-05-20T01:16:32.954845+0000","last_became_active":"2023-05-20T01:16:27.917362+0000","last_became_peered":"2023-05-20T01:16:27.917362+0000","last_unstale":"2023-05-20T01:16:32.954845+0000","last_undegraded":"2023-05-20T01:16:32.954845+0000","last_fullsized":"2023-05-20T01:16:32.954845+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:54:11.307738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.773261+0000","last_change":"2023-05-20T01:16:28.347617+0000","last_active":"2023-05-20T01:16:32.773261+0000","last_peered":"2023-05-20T01:16:32.773261+0000","last_clean":"2023-05-20T01:16:32.773261+0000","last_became_active":"2023-05-20T01:16:28.347032+0000","last_became_peered":"2023-05-20T01:16:28.347032+0000","last_unstale":"2023-05-20T01:16:32.773261+0000","last_undegraded":"2023-05-20T01:16:32.773261+0000","last_fullsized":"2023-05-20T01:16:32.773261+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:34:36.196004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.773474+0000","last_change":"2023-05-20T01:16:28.336509+0000","last_active":"2023-05-20T01:16:32.773474+0000","last_peered":"2023-05-20T01:16:32.773474+0000","last_clean":"2023-05-20T01:16:32.773474+0000","last_became_active":"2023-05-20T01:16:28.336170+0000","last_became_peered":"2023-05-20T01:16:28.336170+0000","last_unstale":"2023-05-20T01:16:32.773474+0000","last_undegraded":"2023-05-20T01:16:32.773474+0000","last_fullsized":"2023-05-20T01:16:32.773474+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:16:31.755218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.771733+0000","last_change":"2023-05-20T01:16:28.340216+0000","last_active":"2023-05-20T01:16:32.771733+0000","last_peered":"2023-05-20T01:16:32.771733+0000","last_clean":"2023-05-20T01:16:32.771733+0000","last_became_active":"2023-05-20T01:16:28.339662+0000","last_became_peered":"2023-05-20T01:16:28.339662+0000","last_unstale":"2023-05-20T01:16:32.771733+0000","last_undegraded":"2023-05-20T01:16:32.771733+0000","last_fullsized":"2023-05-20T01:16:32.771733+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:54:12.315501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.774267+0000","last_change":"2023-05-20T01:16:28.347840+0000","last_active":"2023-05-20T01:16:32.774267+0000","last_peered":"2023-05-20T01:16:32.774267+0000","last_clean":"2023-05-20T01:16:32.774267+0000","last_became_active":"2023-05-20T01:16:28.347370+0000","last_became_peered":"2023-05-20T01:16:28.347370+0000","last_unstale":"2023-05-20T01:16:32.774267+0000","last_undegraded":"2023-05-20T01:16:32.774267+0000","last_fullsized":"2023-05-20T01:16:32.774267+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T01:49:49.650741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.772038+0000","last_change":"2023-05-20T01:16:28.328611+0000","last_active":"2023-05-20T01:16:32.772038+0000","last_peered":"2023-05-20T01:16:32.772038+0000","last_clean":"2023-05-20T01:16:32.772038+0000","last_became_active":"2023-05-20T01:16:28.328311+0000","last_became_peered":"2023-05-20T01:16:28.328311+0000","last_unstale":"2023-05-20T01:16:32.772038+0000","last_undegraded":"2023-05-20T01:16:32.772038+0000","last_fullsized":"2023-05-20T01:16:32.772038+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:09:39.569369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.957125+0000","last_change":"2023-05-20T01:16:28.336980+0000","last_active":"2023-05-20T01:16:32.957125+0000","last_peered":"2023-05-20T01:16:32.957125+0000","last_clean":"2023-05-20T01:16:32.957125+0000","last_became_active":"2023-05-20T01:16:28.336778+0000","last_became_peered":"2023-05-20T01:16:28.336778+0000","last_unstale":"2023-05-20T01:16:32.957125+0000","last_undegraded":"2023-05-20T01:16:32.957125+0000","last_fullsized":"2023-05-20T01:16:32.957125+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:03:20.405510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":4592,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574857,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9260,"kb_used_data":868,"kb_used_omap":0,"kb_used_meta":8384,"kb_avail":94362580,"statfs":{"total":96636764160,"available":96627281920,"internally_reserved":0,"allocated":888832,"data_stored":703779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8585216},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53800000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52500000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35399999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47699999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8076,"kb_used_data":260,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363764,"statfs":{"total":96636764160,"available":96628494336,"internally_reserved":0,"allocated":266240,"data_stored":109401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54200000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.43099999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61299999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59899999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58599999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70499999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9232,"kb_used_data":840,"kb_used_omap":0,"kb_used_meta":8384,"kb_avail":94362608,"statfs":{"total":96636764160,"available":96627310592,"internally_reserved":0,"allocated":860160,"data_stored":699769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8585216},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34100000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34999999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54200000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53000000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8140,"kb_used_data":260,"kb_used_omap":0,"kb_used_meta":7872,"kb_avail":94363700,"statfs":{"total":96636764160,"available":96628428800,"internally_reserved":0,"allocated":266240,"data_stored":109022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8060928},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.24299999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.371}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.309}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.62}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54000000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53000000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8076,"kb_used_data":260,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363764,"statfs":{"total":96636764160,"available":96628494336,"internally_reserved":0,"allocated":266240,"data_stored":109401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45600000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.37}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57399999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66200000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74299999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78200000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8144,"kb_used_data":264,"kb_used_omap":0,"kb_used_meta":7872,"kb_avail":94363696,"statfs":{"total":96636764160,"available":96628424704,"internally_reserved":0,"allocated":270336,"data_stored":109491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8060928},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.317}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.4260000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1619999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0700000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.5379999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.347}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1359999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8224,"kb_used_data":280,"kb_used_omap":0,"kb_used_meta":7936,"kb_avail":94363616,"statfs":{"total":96636764160,"available":96628342784,"internally_reserved":0,"allocated":286720,"data_stored":113251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8126464},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.497}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44600000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48599999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.378}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54000000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60699999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8144,"kb_used_data":264,"kb_used_omap":0,"kb_used_meta":7872,"kb_avail":94363696,"statfs":{"total":96636764160,"available":96628424704,"internally_reserved":0,"allocated":270336,"data_stored":109092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8060928},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44600000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46700000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48999999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51800000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50700000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55800000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-05-20T01:16:51.289 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-05-20T01:16:51.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.442+0000 7f26ddaf8700 1 Processor -- start 2023-05-20T01:16:51.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.442+0000 7f26ddaf8700 1 -- start start 2023-05-20T01:16:51.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.443+0000 7f26ddaf8700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f26d80d16c0 0x7f26d8144330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:51.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.443+0000 7f26ddaf8700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f26d80cec20 0x7f26d80cf090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:51.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.443+0000 7f26ddaf8700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26d8144870 con 0x7f26d80ae2c0 2023-05-20T01:16:51.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.443+0000 7f26ddaf8700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26d80cf5d0 con 0x7f26d80d16c0 2023-05-20T01:16:51.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.443+0000 7f26ddaf8700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26d80b2000 con 0x7f26d80cec20 2023-05-20T01:16:51.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.443+0000 7f26d77fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f26d80d16c0 0x7f26d8144330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:51.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.443+0000 7f26d77fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f26d80d16c0 0x7f26d8144330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58422/0 (socket says 172.21.15.16:58422) 2023-05-20T01:16:51.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.443+0000 7f26d77fe700 1 -- 172.21.15.16:0/3217951597 learned_addr learned my addr 172.21.15.16:0/3217951597 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:51.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.443+0000 7f26d6ffd700 1 --2- 172.21.15.16:0/3217951597 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f26d80cec20 0x7f26d80cf090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:51.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.443+0000 7f26d77fe700 1 -- 172.21.15.16:0/3217951597 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f26d80cec20 msgr2=0x7f26d80cf090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:51.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.443+0000 7f26d77fe700 1 --2- 172.21.15.16:0/3217951597 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f26d80cec20 0x7f26d80cf090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:51.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.443+0000 7f26d77fe700 1 -- 172.21.15.16:0/3217951597 >> v1:172.21.15.132:6789/0 conn(0x7f26d80ae2c0 legacy=0x7f26d80ae740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:51.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.443+0000 7f26d77fe700 1 -- 172.21.15.16:0/3217951597 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26d80b22e0 con 0x7f26d80d16c0 2023-05-20T01:16:51.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.443+0000 7f26d6ffd700 1 --2- 172.21.15.16:0/3217951597 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f26d80cec20 0x7f26d80cf090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:16:51.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.443+0000 7f26d77fe700 1 --2- 172.21.15.16:0/3217951597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26d80d16c0 0x7f26d8144330 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f26c80098f0 tx=0x7f26c8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=256c37103175c44f server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:51.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.444+0000 7f26d67fc700 1 -- 172.21.15.16:0/3217951597 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26c800b8a0 con 0x7f26d80d16c0 2023-05-20T01:16:51.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.444+0000 7f26d67fc700 1 -- 172.21.15.16:0/3217951597 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26c800ba00 con 0x7f26d80d16c0 2023-05-20T01:16:51.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.444+0000 7f26d67fc700 1 -- 172.21.15.16:0/3217951597 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26c800bd90 con 0x7f26d80d16c0 2023-05-20T01:16:51.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.444+0000 7f26ddaf8700 1 -- 172.21.15.16:0/3217951597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26d80d16c0 msgr2=0x7f26d8144330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:51.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.444+0000 7f26ddaf8700 1 --2- 172.21.15.16:0/3217951597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26d80d16c0 0x7f26d8144330 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f26c80098f0 tx=0x7f26c8009c00 comp rx=0 tx=0).stop 2023-05-20T01:16:51.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.444+0000 7f26ddaf8700 1 -- 172.21.15.16:0/3217951597 shutdown_connections 2023-05-20T01:16:51.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.444+0000 7f26ddaf8700 1 --2- 172.21.15.16:0/3217951597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26d80d16c0 0x7f26d8144330 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:51.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.444+0000 7f26ddaf8700 1 --2- 172.21.15.16:0/3217951597 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f26d80cec20 0x7f26d80cf090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:51.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.444+0000 7f26ddaf8700 1 -- 172.21.15.16:0/3217951597 >> 172.21.15.16:0/3217951597 conn(0x7f26d806ae30 msgr2=0x7f26d809a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:51.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.444+0000 7f26ddaf8700 1 -- 172.21.15.16:0/3217951597 shutdown_connections 2023-05-20T01:16:51.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.444+0000 7f26ddaf8700 1 -- 172.21.15.16:0/3217951597 wait complete. 2023-05-20T01:16:51.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.445+0000 7f26ddaf8700 1 Processor -- start 2023-05-20T01:16:51.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.446+0000 7f26ddaf8700 1 -- start start 2023-05-20T01:16:51.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.446+0000 7f26ddaf8700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f26d80ae2c0 0x7f26d806b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:51.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.446+0000 7f26ddaf8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26d80cec20 0x7f26d8143650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:51.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.446+0000 7f26ddaf8700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f26d80d16c0 0x7f26d8159e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:51.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.446+0000 7f26ddaf8700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26d80c8750 con 0x7f26d80cec20 2023-05-20T01:16:51.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.446+0000 7f26ddaf8700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26d80c8120 con 0x7f26d80d16c0 2023-05-20T01:16:51.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.446+0000 7f26ddaf8700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26d8143b90 con 0x7f26d80ae2c0 2023-05-20T01:16:51.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.446+0000 7f26d6ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26d80cec20 0x7f26d8143650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:51.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.446+0000 7f26d6ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26d80cec20 0x7f26d8143650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58430/0 (socket says 172.21.15.16:58430) 2023-05-20T01:16:51.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.447+0000 7f26d6ffd700 1 -- 172.21.15.16:0/1510010650 learned_addr learned my addr 172.21.15.16:0/1510010650 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:51.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.447+0000 7f26d77fe700 1 --2- 172.21.15.16:0/1510010650 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f26d80ae2c0 0x7f26d806b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:51.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.447+0000 7f26d7fff700 1 --2- 172.21.15.16:0/1510010650 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f26d80d16c0 0x7f26d8159e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:51.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.447+0000 7f26d6ffd700 1 -- 172.21.15.16:0/1510010650 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f26d80ae2c0 msgr2=0x7f26d806b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:51.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.447+0000 7f26d6ffd700 1 --2- 172.21.15.16:0/1510010650 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f26d80ae2c0 0x7f26d806b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:51.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.447+0000 7f26d6ffd700 1 -- 172.21.15.16:0/1510010650 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f26d80d16c0 msgr2=0x7f26d8159e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:51.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.447+0000 7f26d6ffd700 1 --2- 172.21.15.16:0/1510010650 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f26d80d16c0 0x7f26d8159e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:51.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.447+0000 7f26d6ffd700 1 -- 172.21.15.16:0/1510010650 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26c8009550 con 0x7f26d80cec20 2023-05-20T01:16:51.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.447+0000 7f26d6ffd700 1 --2- 172.21.15.16:0/1510010650 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26d80cec20 0x7f26d8143650 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f26c000e810 tx=0x7f26c000eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:51.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.447+0000 7f26cffff700 1 -- 172.21.15.16:0/1510010650 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26c000c070 con 0x7f26d80cec20 2023-05-20T01:16:51.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.447+0000 7f26cffff700 1 -- 172.21.15.16:0/1510010650 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26c0004230 con 0x7f26d80cec20 2023-05-20T01:16:51.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.447+0000 7f26ddaf8700 1 -- 172.21.15.16:0/1510010650 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26d80adda0 con 0x7f26d80cec20 2023-05-20T01:16:51.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.447+0000 7f26cffff700 1 -- 172.21.15.16:0/1510010650 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26c00036a0 con 0x7f26d80cec20 2023-05-20T01:16:51.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.447+0000 7f26ddaf8700 1 -- 172.21.15.16:0/1510010650 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26d80b25c0 con 0x7f26d80cec20 2023-05-20T01:16:51.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.448+0000 7f26ddaf8700 1 -- 172.21.15.16:0/1510010650 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26d80b28a0 con 0x7f26d80cec20 2023-05-20T01:16:51.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.449+0000 7f26cffff700 1 -- 172.21.15.16:0/1510010650 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f26c0003800 con 0x7f26d80cec20 2023-05-20T01:16:51.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.449+0000 7f26cffff700 1 --2- 172.21.15.16:0/1510010650 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f26b806ace0 0x7f26b806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:51.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.449+0000 7f26cffff700 1 -- 172.21.15.16:0/1510010650 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f26c0089390 con 0x7f26d80cec20 2023-05-20T01:16:51.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.449+0000 7f26d77fe700 1 --2- 172.21.15.16:0/1510010650 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f26b806ace0 0x7f26b806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:51.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.450+0000 7f26d77fe700 1 --2- 172.21.15.16:0/1510010650 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f26b806ace0 0x7f26b806d190 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f26c8000c00 tx=0x7f26c8008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:51.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.454+0000 7f26cffff700 1 -- 172.21.15.16:0/1510010650 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f26c00541e0 con 0x7f26d80cec20 2023-05-20T01:16:51.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.647+0000 7f26ddaf8700 1 -- 172.21.15.16:0/1510010650 --> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f26d80c8890 con 0x7f26b806ace0 2023-05-20T01:16:51.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.657+0000 7f26cffff700 1 -- 172.21.15.16:0/1510010650 <== mgr.4104 v2:172.21.15.16:6832/3817237998 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+367546 (secure 0 0 0) 0x7f26d80c8890 con 0x7f26b806ace0 2023-05-20T01:16:51.663 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:16:51.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.660+0000 7f26ddaf8700 1 -- 172.21.15.16:0/1510010650 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f26b806ace0 msgr2=0x7f26b806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:51.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.660+0000 7f26ddaf8700 1 --2- 172.21.15.16:0/1510010650 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f26b806ace0 0x7f26b806d190 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f26c8000c00 tx=0x7f26c8008030 comp rx=0 tx=0).stop 2023-05-20T01:16:51.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.660+0000 7f26ddaf8700 1 -- 172.21.15.16:0/1510010650 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26d80cec20 msgr2=0x7f26d8143650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:51.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.660+0000 7f26ddaf8700 1 --2- 172.21.15.16:0/1510010650 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26d80cec20 0x7f26d8143650 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f26c000e810 tx=0x7f26c000eb20 comp rx=0 tx=0).stop 2023-05-20T01:16:51.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.660+0000 7f26ddaf8700 1 -- 172.21.15.16:0/1510010650 shutdown_connections 2023-05-20T01:16:51.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.660+0000 7f26ddaf8700 1 --2- 172.21.15.16:0/1510010650 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f26d80d16c0 0x7f26d8159e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:51.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.660+0000 7f26ddaf8700 1 --2- 172.21.15.16:0/1510010650 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f26b806ace0 0x7f26b806d190 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:51.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.660+0000 7f26ddaf8700 1 --2- 172.21.15.16:0/1510010650 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f26d80ae2c0 0x7f26d806b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:51.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.660+0000 7f26ddaf8700 1 --2- 172.21.15.16:0/1510010650 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26d80cec20 0x7f26d8143650 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:51.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.660+0000 7f26ddaf8700 1 -- 172.21.15.16:0/1510010650 >> 172.21.15.16:0/1510010650 conn(0x7f26d806ae30 msgr2=0x7f26d80ae690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:51.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.660+0000 7f26ddaf8700 1 -- 172.21.15.16:0/1510010650 shutdown_connections 2023-05-20T01:16:51.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.660+0000 7f26ddaf8700 1 -- 172.21.15.16:0/1510010650 wait complete. 2023-05-20T01:16:51.671 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2023-05-20T01:16:51.676 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":34,"stamp":"2023-05-20T01:16:51.322761+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594960,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":338,"acting":338,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":338,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":67296,"kb_used_data":3296,"kb_used_omap":0,"kb_used_meta":63936,"kb_avail":754907424,"statfs":{"total":773094113280,"available":773025202176,"internally_reserved":0,"allocated":3375104,"data_stored":2063206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":65470464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002722"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.310521+0000","last_change":"2023-05-20T01:16:28.345449+0000","last_active":"2023-05-20T01:16:33.310521+0000","last_peered":"2023-05-20T01:16:33.310521+0000","last_clean":"2023-05-20T01:16:33.310521+0000","last_became_active":"2023-05-20T01:16:28.345161+0000","last_became_peered":"2023-05-20T01:16:28.345161+0000","last_unstale":"2023-05-20T01:16:33.310521+0000","last_undegraded":"2023-05-20T01:16:33.310521+0000","last_fullsized":"2023-05-20T01:16:33.310521+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:10:54.451689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.310465+0000","last_change":"2023-05-20T01:16:28.337973+0000","last_active":"2023-05-20T01:16:33.310465+0000","last_peered":"2023-05-20T01:16:33.310465+0000","last_clean":"2023-05-20T01:16:33.310465+0000","last_became_active":"2023-05-20T01:16:28.337679+0000","last_became_peered":"2023-05-20T01:16:28.337679+0000","last_unstale":"2023-05-20T01:16:33.310465+0000","last_undegraded":"2023-05-20T01:16:33.310465+0000","last_fullsized":"2023-05-20T01:16:33.310465+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:59:48.441604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.774449+0000","last_change":"2023-05-20T01:16:28.342781+0000","last_active":"2023-05-20T01:16:32.774449+0000","last_peered":"2023-05-20T01:16:32.774449+0000","last_clean":"2023-05-20T01:16:32.774449+0000","last_became_active":"2023-05-20T01:16:28.342488+0000","last_became_peered":"2023-05-20T01:16:28.342488+0000","last_unstale":"2023-05-20T01:16:32.774449+0000","last_undegraded":"2023-05-20T01:16:32.774449+0000","last_fullsized":"2023-05-20T01:16:32.774449+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:35:02.629395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.773758+0000","last_change":"2023-05-20T01:16:28.345746+0000","last_active":"2023-05-20T01:16:32.773758+0000","last_peered":"2023-05-20T01:16:32.773758+0000","last_clean":"2023-05-20T01:16:32.773758+0000","last_became_active":"2023-05-20T01:16:28.345477+0000","last_became_peered":"2023-05-20T01:16:28.345477+0000","last_unstale":"2023-05-20T01:16:32.773758+0000","last_undegraded":"2023-05-20T01:16:32.773758+0000","last_fullsized":"2023-05-20T01:16:32.773758+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:08:32.412558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.954652+0000","last_change":"2023-05-20T01:16:28.352984+0000","last_active":"2023-05-20T01:16:32.954652+0000","last_peered":"2023-05-20T01:16:32.954652+0000","last_clean":"2023-05-20T01:16:32.954652+0000","last_became_active":"2023-05-20T01:16:28.352711+0000","last_became_peered":"2023-05-20T01:16:28.352711+0000","last_unstale":"2023-05-20T01:16:32.954652+0000","last_undegraded":"2023-05-20T01:16:32.954652+0000","last_fullsized":"2023-05-20T01:16:32.954652+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:34:08.601566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.774271+0000","last_change":"2023-05-20T01:16:28.347827+0000","last_active":"2023-05-20T01:16:32.774271+0000","last_peered":"2023-05-20T01:16:32.774271+0000","last_clean":"2023-05-20T01:16:32.774271+0000","last_became_active":"2023-05-20T01:16:28.347476+0000","last_became_peered":"2023-05-20T01:16:28.347476+0000","last_unstale":"2023-05-20T01:16:32.774271+0000","last_undegraded":"2023-05-20T01:16:32.774271+0000","last_fullsized":"2023-05-20T01:16:32.774271+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:21:02.720695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.311877+0000","last_change":"2023-05-20T01:16:28.338564+0000","last_active":"2023-05-20T01:16:33.311877+0000","last_peered":"2023-05-20T01:16:33.311877+0000","last_clean":"2023-05-20T01:16:33.311877+0000","last_became_active":"2023-05-20T01:16:28.337927+0000","last_became_peered":"2023-05-20T01:16:28.337927+0000","last_unstale":"2023-05-20T01:16:33.311877+0000","last_undegraded":"2023-05-20T01:16:33.311877+0000","last_fullsized":"2023-05-20T01:16:33.311877+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:24:36.047155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.310151+0000","last_change":"2023-05-20T01:16:28.340008+0000","last_active":"2023-05-20T01:16:33.310151+0000","last_peered":"2023-05-20T01:16:33.310151+0000","last_clean":"2023-05-20T01:16:33.310151+0000","last_became_active":"2023-05-20T01:16:28.339724+0000","last_became_peered":"2023-05-20T01:16:28.339724+0000","last_unstale":"2023-05-20T01:16:33.310151+0000","last_undegraded":"2023-05-20T01:16:33.310151+0000","last_fullsized":"2023-05-20T01:16:33.310151+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:55:50.059228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.954657+0000","last_change":"2023-05-20T01:16:28.351002+0000","last_active":"2023-05-20T01:16:32.954657+0000","last_peered":"2023-05-20T01:16:32.954657+0000","last_clean":"2023-05-20T01:16:32.954657+0000","last_became_active":"2023-05-20T01:16:28.350672+0000","last_became_peered":"2023-05-20T01:16:28.350672+0000","last_unstale":"2023-05-20T01:16:32.954657+0000","last_undegraded":"2023-05-20T01:16:32.954657+0000","last_fullsized":"2023-05-20T01:16:32.954657+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:23:33.732284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.310970+0000","last_change":"2023-05-20T01:16:28.349410+0000","last_active":"2023-05-20T01:16:33.310970+0000","last_peered":"2023-05-20T01:16:33.310970+0000","last_clean":"2023-05-20T01:16:33.310970+0000","last_became_active":"2023-05-20T01:16:28.349177+0000","last_became_peered":"2023-05-20T01:16:28.349177+0000","last_unstale":"2023-05-20T01:16:33.310970+0000","last_undegraded":"2023-05-20T01:16:33.310970+0000","last_fullsized":"2023-05-20T01:16:33.310970+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:34:08.601566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.774486+0000","last_change":"2023-05-20T01:16:28.338627+0000","last_active":"2023-05-20T01:16:32.774486+0000","last_peered":"2023-05-20T01:16:32.774486+0000","last_clean":"2023-05-20T01:16:32.774486+0000","last_became_active":"2023-05-20T01:16:28.338363+0000","last_became_peered":"2023-05-20T01:16:28.338363+0000","last_unstale":"2023-05-20T01:16:32.774486+0000","last_undegraded":"2023-05-20T01:16:32.774486+0000","last_fullsized":"2023-05-20T01:16:32.774486+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:20:15.094065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.310156+0000","last_change":"2023-05-20T01:16:28.341603+0000","last_active":"2023-05-20T01:16:33.310156+0000","last_peered":"2023-05-20T01:16:33.310156+0000","last_clean":"2023-05-20T01:16:33.310156+0000","last_became_active":"2023-05-20T01:16:28.341242+0000","last_became_peered":"2023-05-20T01:16:28.341242+0000","last_unstale":"2023-05-20T01:16:33.310156+0000","last_undegraded":"2023-05-20T01:16:33.310156+0000","last_fullsized":"2023-05-20T01:16:33.310156+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:15:50.986060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.310269+0000","last_change":"2023-05-20T01:16:28.345977+0000","last_active":"2023-05-20T01:16:33.310269+0000","last_peered":"2023-05-20T01:16:33.310269+0000","last_clean":"2023-05-20T01:16:33.310269+0000","last_became_active":"2023-05-20T01:16:28.345732+0000","last_became_peered":"2023-05-20T01:16:28.345732+0000","last_unstale":"2023-05-20T01:16:33.310269+0000","last_undegraded":"2023-05-20T01:16:33.310269+0000","last_fullsized":"2023-05-20T01:16:33.310269+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:39:45.746298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.311067+0000","last_change":"2023-05-20T01:16:28.334227+0000","last_active":"2023-05-20T01:16:33.311067+0000","last_peered":"2023-05-20T01:16:33.311067+0000","last_clean":"2023-05-20T01:16:33.311067+0000","last_became_active":"2023-05-20T01:16:28.333922+0000","last_became_peered":"2023-05-20T01:16:28.333922+0000","last_unstale":"2023-05-20T01:16:33.311067+0000","last_undegraded":"2023-05-20T01:16:33.311067+0000","last_fullsized":"2023-05-20T01:16:33.311067+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:23:33.732284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.311509+0000","last_change":"2023-05-20T01:16:28.338772+0000","last_active":"2023-05-20T01:16:33.311509+0000","last_peered":"2023-05-20T01:16:33.311509+0000","last_clean":"2023-05-20T01:16:33.311509+0000","last_became_active":"2023-05-20T01:16:28.338300+0000","last_became_peered":"2023-05-20T01:16:28.338300+0000","last_unstale":"2023-05-20T01:16:33.311509+0000","last_undegraded":"2023-05-20T01:16:33.311509+0000","last_fullsized":"2023-05-20T01:16:33.311509+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:19:51.520080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.774491+0000","last_change":"2023-05-20T01:16:28.343926+0000","last_active":"2023-05-20T01:16:32.774491+0000","last_peered":"2023-05-20T01:16:32.774491+0000","last_clean":"2023-05-20T01:16:32.774491+0000","last_became_active":"2023-05-20T01:16:28.343696+0000","last_became_peered":"2023-05-20T01:16:28.343696+0000","last_unstale":"2023-05-20T01:16:32.774491+0000","last_undegraded":"2023-05-20T01:16:32.774491+0000","last_fullsized":"2023-05-20T01:16:32.774491+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:42:48.561844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.954664+0000","last_change":"2023-05-20T01:16:28.350876+0000","last_active":"2023-05-20T01:16:32.954664+0000","last_peered":"2023-05-20T01:16:32.954664+0000","last_clean":"2023-05-20T01:16:32.954664+0000","last_became_active":"2023-05-20T01:16:28.350242+0000","last_became_peered":"2023-05-20T01:16:28.350242+0000","last_unstale":"2023-05-20T01:16:32.954664+0000","last_undegraded":"2023-05-20T01:16:32.954664+0000","last_fullsized":"2023-05-20T01:16:32.954664+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:19:51.520080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.773740+0000","last_change":"2023-05-20T01:16:28.334794+0000","last_active":"2023-05-20T01:16:32.773740+0000","last_peered":"2023-05-20T01:16:32.773740+0000","last_clean":"2023-05-20T01:16:32.773740+0000","last_became_active":"2023-05-20T01:16:28.334535+0000","last_became_peered":"2023-05-20T01:16:28.334535+0000","last_unstale":"2023-05-20T01:16:32.773740+0000","last_undegraded":"2023-05-20T01:16:32.773740+0000","last_fullsized":"2023-05-20T01:16:32.773740+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T01:37:30.926061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.310464+0000","last_change":"2023-05-20T01:16:28.347597+0000","last_active":"2023-05-20T01:16:33.310464+0000","last_peered":"2023-05-20T01:16:33.310464+0000","last_clean":"2023-05-20T01:16:33.310464+0000","last_became_active":"2023-05-20T01:16:28.347245+0000","last_became_peered":"2023-05-20T01:16:28.347245+0000","last_unstale":"2023-05-20T01:16:33.310464+0000","last_undegraded":"2023-05-20T01:16:33.310464+0000","last_fullsized":"2023-05-20T01:16:33.310464+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:02:17.539187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.773711+0000","last_change":"2023-05-20T01:16:28.347994+0000","last_active":"2023-05-20T01:16:32.773711+0000","last_peered":"2023-05-20T01:16:32.773711+0000","last_clean":"2023-05-20T01:16:32.773711+0000","last_became_active":"2023-05-20T01:16:28.347690+0000","last_became_peered":"2023-05-20T01:16:28.347690+0000","last_unstale":"2023-05-20T01:16:32.773711+0000","last_undegraded":"2023-05-20T01:16:32.773711+0000","last_fullsized":"2023-05-20T01:16:32.773711+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:46:24.892963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.311730+0000","last_change":"2023-05-20T01:16:28.346262+0000","last_active":"2023-05-20T01:16:33.311730+0000","last_peered":"2023-05-20T01:16:33.311730+0000","last_clean":"2023-05-20T01:16:33.311730+0000","last_became_active":"2023-05-20T01:16:28.345513+0000","last_became_peered":"2023-05-20T01:16:28.345513+0000","last_unstale":"2023-05-20T01:16:33.311730+0000","last_undegraded":"2023-05-20T01:16:33.311730+0000","last_fullsized":"2023-05-20T01:16:33.311730+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:46:14.388881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.954206+0000","last_change":"2023-05-20T01:16:28.344269+0000","last_active":"2023-05-20T01:16:32.954206+0000","last_peered":"2023-05-20T01:16:32.954206+0000","last_clean":"2023-05-20T01:16:32.954206+0000","last_became_active":"2023-05-20T01:16:28.343991+0000","last_became_peered":"2023-05-20T01:16:28.343991+0000","last_unstale":"2023-05-20T01:16:32.954206+0000","last_undegraded":"2023-05-20T01:16:32.954206+0000","last_fullsized":"2023-05-20T01:16:32.954206+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:33:00.800050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.771776+0000","last_change":"2023-05-20T01:16:28.344552+0000","last_active":"2023-05-20T01:16:32.771776+0000","last_peered":"2023-05-20T01:16:32.771776+0000","last_clean":"2023-05-20T01:16:32.771776+0000","last_became_active":"2023-05-20T01:16:28.344125+0000","last_became_peered":"2023-05-20T01:16:28.344125+0000","last_unstale":"2023-05-20T01:16:32.771776+0000","last_undegraded":"2023-05-20T01:16:32.771776+0000","last_fullsized":"2023-05-20T01:16:32.771776+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:06:59.447318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.311284+0000","last_change":"2023-05-20T01:16:28.345899+0000","last_active":"2023-05-20T01:16:33.311284+0000","last_peered":"2023-05-20T01:16:33.311284+0000","last_clean":"2023-05-20T01:16:33.311284+0000","last_became_active":"2023-05-20T01:16:28.345629+0000","last_became_peered":"2023-05-20T01:16:28.345629+0000","last_unstale":"2023-05-20T01:16:33.311284+0000","last_undegraded":"2023-05-20T01:16:33.311284+0000","last_fullsized":"2023-05-20T01:16:33.311284+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:13:57.518009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.957468+0000","last_change":"2023-05-20T01:16:28.346168+0000","last_active":"2023-05-20T01:16:32.957468+0000","last_peered":"2023-05-20T01:16:32.957468+0000","last_clean":"2023-05-20T01:16:32.957468+0000","last_became_active":"2023-05-20T01:16:28.345889+0000","last_became_peered":"2023-05-20T01:16:28.345889+0000","last_unstale":"2023-05-20T01:16:32.957468+0000","last_undegraded":"2023-05-20T01:16:32.957468+0000","last_fullsized":"2023-05-20T01:16:32.957468+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:05:20.974728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.957396+0000","last_change":"2023-05-20T01:16:28.335035+0000","last_active":"2023-05-20T01:16:32.957396+0000","last_peered":"2023-05-20T01:16:32.957396+0000","last_clean":"2023-05-20T01:16:32.957396+0000","last_became_active":"2023-05-20T01:16:28.334787+0000","last_became_peered":"2023-05-20T01:16:28.334787+0000","last_unstale":"2023-05-20T01:16:32.957396+0000","last_undegraded":"2023-05-20T01:16:32.957396+0000","last_fullsized":"2023-05-20T01:16:32.957396+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:55:50.059228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.310780+0000","last_change":"2023-05-20T01:16:28.335038+0000","last_active":"2023-05-20T01:16:33.310780+0000","last_peered":"2023-05-20T01:16:33.310780+0000","last_clean":"2023-05-20T01:16:33.310780+0000","last_became_active":"2023-05-20T01:16:28.334750+0000","last_became_peered":"2023-05-20T01:16:28.334750+0000","last_unstale":"2023-05-20T01:16:33.310780+0000","last_undegraded":"2023-05-20T01:16:33.310780+0000","last_fullsized":"2023-05-20T01:16:33.310780+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:13:57.518009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.774853+0000","last_change":"2023-05-20T01:16:32.363983+0000","last_active":"2023-05-20T01:16:32.774853+0000","last_peered":"2023-05-20T01:16:32.774853+0000","last_clean":"2023-05-20T01:16:32.774853+0000","last_became_active":"2023-05-20T01:16:32.363545+0000","last_became_peered":"2023-05-20T01:16:32.363545+0000","last_unstale":"2023-05-20T01:16:32.774853+0000","last_undegraded":"2023-05-20T01:16:32.774853+0000","last_fullsized":"2023-05-20T01:16:32.774853+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:15:31.121265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2,0],"acting":[4,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"20'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.774070+0000","last_change":"2023-05-20T01:16:27.924758+0000","last_active":"2023-05-20T01:16:32.774070+0000","last_peered":"2023-05-20T01:16:32.774070+0000","last_clean":"2023-05-20T01:16:32.774070+0000","last_became_active":"2023-05-20T01:16:27.924471+0000","last_became_peered":"2023-05-20T01:16:27.924471+0000","last_unstale":"2023-05-20T01:16:32.774070+0000","last_undegraded":"2023-05-20T01:16:32.774070+0000","last_fullsized":"2023-05-20T01:16:32.774070+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:40:53.545036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.772324+0000","last_change":"2023-05-20T01:16:28.340516+0000","last_active":"2023-05-20T01:16:32.772324+0000","last_peered":"2023-05-20T01:16:32.772324+0000","last_clean":"2023-05-20T01:16:32.772324+0000","last_became_active":"2023-05-20T01:16:28.340255+0000","last_became_peered":"2023-05-20T01:16:28.340255+0000","last_unstale":"2023-05-20T01:16:32.772324+0000","last_undegraded":"2023-05-20T01:16:32.772324+0000","last_fullsized":"2023-05-20T01:16:32.772324+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:17:27.361087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.774064+0000","last_change":"2023-05-20T01:16:32.348270+0000","last_active":"2023-05-20T01:16:32.774064+0000","last_peered":"2023-05-20T01:16:32.774064+0000","last_clean":"2023-05-20T01:16:32.774064+0000","last_became_active":"2023-05-20T01:16:32.347460+0000","last_became_peered":"2023-05-20T01:16:32.347460+0000","last_unstale":"2023-05-20T01:16:32.774064+0000","last_undegraded":"2023-05-20T01:16:32.774064+0000","last_fullsized":"2023-05-20T01:16:32.774064+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:49:18.438950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3,7],"acting":[6,4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.956713+0000","last_change":"2023-05-20T01:16:27.922074+0000","last_active":"2023-05-20T01:16:32.956713+0000","last_peered":"2023-05-20T01:16:32.956713+0000","last_clean":"2023-05-20T01:16:32.956713+0000","last_became_active":"2023-05-20T01:16:27.921759+0000","last_became_peered":"2023-05-20T01:16:27.921759+0000","last_unstale":"2023-05-20T01:16:32.956713+0000","last_undegraded":"2023-05-20T01:16:32.956713+0000","last_fullsized":"2023-05-20T01:16:32.956713+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:08:26.231228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.954625+0000","last_change":"2023-05-20T01:16:28.348595+0000","last_active":"2023-05-20T01:16:32.954625+0000","last_peered":"2023-05-20T01:16:32.954625+0000","last_clean":"2023-05-20T01:16:32.954625+0000","last_became_active":"2023-05-20T01:16:28.347911+0000","last_became_peered":"2023-05-20T01:16:28.347911+0000","last_unstale":"2023-05-20T01:16:32.954625+0000","last_undegraded":"2023-05-20T01:16:32.954625+0000","last_fullsized":"2023-05-20T01:16:32.954625+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:46:42.721514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.954742+0000","last_change":"2023-05-20T01:16:32.362037+0000","last_active":"2023-05-20T01:16:32.954742+0000","last_peered":"2023-05-20T01:16:32.954742+0000","last_clean":"2023-05-20T01:16:32.954742+0000","last_became_active":"2023-05-20T01:16:32.361564+0000","last_became_peered":"2023-05-20T01:16:32.361564+0000","last_unstale":"2023-05-20T01:16:32.954742+0000","last_undegraded":"2023-05-20T01:16:32.954742+0000","last_fullsized":"2023-05-20T01:16:32.954742+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:14:35.048109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1,2],"acting":[0,7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.771855+0000","last_change":"2023-05-20T01:16:27.910268+0000","last_active":"2023-05-20T01:16:32.771855+0000","last_peered":"2023-05-20T01:16:32.771855+0000","last_clean":"2023-05-20T01:16:32.771855+0000","last_became_active":"2023-05-20T01:16:27.909963+0000","last_became_peered":"2023-05-20T01:16:27.909963+0000","last_unstale":"2023-05-20T01:16:32.771855+0000","last_undegraded":"2023-05-20T01:16:32.771855+0000","last_fullsized":"2023-05-20T01:16:32.771855+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:22:46.049477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.774156+0000","last_change":"2023-05-20T01:16:28.345078+0000","last_active":"2023-05-20T01:16:32.774156+0000","last_peered":"2023-05-20T01:16:32.774156+0000","last_clean":"2023-05-20T01:16:32.774156+0000","last_became_active":"2023-05-20T01:16:28.344742+0000","last_became_peered":"2023-05-20T01:16:28.344742+0000","last_unstale":"2023-05-20T01:16:32.774156+0000","last_undegraded":"2023-05-20T01:16:32.774156+0000","last_fullsized":"2023-05-20T01:16:32.774156+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:53:18.904132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.773906+0000","last_change":"2023-05-20T01:16:32.348805+0000","last_active":"2023-05-20T01:16:32.773906+0000","last_peered":"2023-05-20T01:16:32.773906+0000","last_clean":"2023-05-20T01:16:32.773906+0000","last_became_active":"2023-05-20T01:16:32.348352+0000","last_became_peered":"2023-05-20T01:16:32.348352+0000","last_unstale":"2023-05-20T01:16:32.773906+0000","last_undegraded":"2023-05-20T01:16:32.773906+0000","last_fullsized":"2023-05-20T01:16:32.773906+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:04:37.665737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7,1],"acting":[6,4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.954805+0000","last_change":"2023-05-20T01:16:27.915492+0000","last_active":"2023-05-20T01:16:32.954805+0000","last_peered":"2023-05-20T01:16:32.954805+0000","last_clean":"2023-05-20T01:16:32.954805+0000","last_became_active":"2023-05-20T01:16:27.915015+0000","last_became_peered":"2023-05-20T01:16:27.915015+0000","last_unstale":"2023-05-20T01:16:32.954805+0000","last_undegraded":"2023-05-20T01:16:32.954805+0000","last_fullsized":"2023-05-20T01:16:32.954805+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:25:21.983559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.771743+0000","last_change":"2023-05-20T01:16:28.339100+0000","last_active":"2023-05-20T01:16:32.771743+0000","last_peered":"2023-05-20T01:16:32.771743+0000","last_clean":"2023-05-20T01:16:32.771743+0000","last_became_active":"2023-05-20T01:16:28.338829+0000","last_became_peered":"2023-05-20T01:16:28.338829+0000","last_unstale":"2023-05-20T01:16:32.771743+0000","last_undegraded":"2023-05-20T01:16:32.771743+0000","last_fullsized":"2023-05-20T01:16:32.771743+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:38:13.734511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.310111+0000","last_change":"2023-05-20T01:16:32.364639+0000","last_active":"2023-05-20T01:16:33.310111+0000","last_peered":"2023-05-20T01:16:33.310111+0000","last_clean":"2023-05-20T01:16:33.310111+0000","last_became_active":"2023-05-20T01:16:32.364259+0000","last_became_peered":"2023-05-20T01:16:32.364259+0000","last_unstale":"2023-05-20T01:16:33.310111+0000","last_undegraded":"2023-05-20T01:16:33.310111+0000","last_fullsized":"2023-05-20T01:16:33.310111+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:58:39.806550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6,0],"acting":[7,4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.310717+0000","last_change":"2023-05-20T01:16:27.914406+0000","last_active":"2023-05-20T01:16:33.310717+0000","last_peered":"2023-05-20T01:16:33.310717+0000","last_clean":"2023-05-20T01:16:33.310717+0000","last_became_active":"2023-05-20T01:16:27.913932+0000","last_became_peered":"2023-05-20T01:16:27.913932+0000","last_unstale":"2023-05-20T01:16:33.310717+0000","last_undegraded":"2023-05-20T01:16:33.310717+0000","last_fullsized":"2023-05-20T01:16:33.310717+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:58:35.370081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.771638+0000","last_change":"2023-05-20T01:16:28.330729+0000","last_active":"2023-05-20T01:16:32.771638+0000","last_peered":"2023-05-20T01:16:32.771638+0000","last_clean":"2023-05-20T01:16:32.771638+0000","last_became_active":"2023-05-20T01:16:28.330474+0000","last_became_peered":"2023-05-20T01:16:28.330474+0000","last_unstale":"2023-05-20T01:16:32.771638+0000","last_undegraded":"2023-05-20T01:16:32.771638+0000","last_fullsized":"2023-05-20T01:16:32.771638+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:49:21.195547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.772392+0000","last_change":"2023-05-20T01:16:32.363881+0000","last_active":"2023-05-20T01:16:32.772392+0000","last_peered":"2023-05-20T01:16:32.772392+0000","last_clean":"2023-05-20T01:16:32.772392+0000","last_became_active":"2023-05-20T01:16:32.363269+0000","last_became_peered":"2023-05-20T01:16:32.363269+0000","last_unstale":"2023-05-20T01:16:32.772392+0000","last_undegraded":"2023-05-20T01:16:32.772392+0000","last_fullsized":"2023-05-20T01:16:32.772392+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:06:25.331494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,5],"acting":[3,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"20'3","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.311909+0000","last_change":"2023-05-20T01:16:27.919005+0000","last_active":"2023-05-20T01:16:33.311909+0000","last_peered":"2023-05-20T01:16:33.311909+0000","last_clean":"2023-05-20T01:16:33.311909+0000","last_became_active":"2023-05-20T01:16:27.918679+0000","last_became_peered":"2023-05-20T01:16:27.918679+0000","last_unstale":"2023-05-20T01:16:33.311909+0000","last_undegraded":"2023-05-20T01:16:33.311909+0000","last_fullsized":"2023-05-20T01:16:33.311909+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:06:58.439555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.957017+0000","last_change":"2023-05-20T01:16:28.346905+0000","last_active":"2023-05-20T01:16:32.957017+0000","last_peered":"2023-05-20T01:16:32.957017+0000","last_clean":"2023-05-20T01:16:32.957017+0000","last_became_active":"2023-05-20T01:16:28.345822+0000","last_became_peered":"2023-05-20T01:16:28.345822+0000","last_unstale":"2023-05-20T01:16:32.957017+0000","last_undegraded":"2023-05-20T01:16:32.957017+0000","last_fullsized":"2023-05-20T01:16:32.957017+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:45:24.050486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.957316+0000","last_change":"2023-05-20T01:16:32.354753+0000","last_active":"2023-05-20T01:16:32.957316+0000","last_peered":"2023-05-20T01:16:32.957316+0000","last_clean":"2023-05-20T01:16:32.957316+0000","last_became_active":"2023-05-20T01:16:32.354017+0000","last_became_peered":"2023-05-20T01:16:32.354017+0000","last_unstale":"2023-05-20T01:16:32.957316+0000","last_undegraded":"2023-05-20T01:16:32.957316+0000","last_fullsized":"2023-05-20T01:16:32.957316+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:24:19.113785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3,7],"acting":[1,5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"25'9","reported_seq":30,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:37.788145+0000","last_change":"2023-05-20T01:16:27.924627+0000","last_active":"2023-05-20T01:16:37.788145+0000","last_peered":"2023-05-20T01:16:37.788145+0000","last_clean":"2023-05-20T01:16:37.788145+0000","last_became_active":"2023-05-20T01:16:27.924339+0000","last_became_peered":"2023-05-20T01:16:27.924339+0000","last_unstale":"2023-05-20T01:16:37.788145+0000","last_undegraded":"2023-05-20T01:16:37.788145+0000","last_fullsized":"2023-05-20T01:16:37.788145+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:27:33.065117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3794,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.773847+0000","last_change":"2023-05-20T01:16:28.348715+0000","last_active":"2023-05-20T01:16:32.773847+0000","last_peered":"2023-05-20T01:16:32.773847+0000","last_clean":"2023-05-20T01:16:32.773847+0000","last_became_active":"2023-05-20T01:16:28.346893+0000","last_became_peered":"2023-05-20T01:16:28.346893+0000","last_unstale":"2023-05-20T01:16:32.773847+0000","last_undegraded":"2023-05-20T01:16:32.773847+0000","last_fullsized":"2023-05-20T01:16:32.773847+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:44:10.386273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.311010+0000","last_change":"2023-05-20T01:16:32.362978+0000","last_active":"2023-05-20T01:16:33.311010+0000","last_peered":"2023-05-20T01:16:33.311010+0000","last_clean":"2023-05-20T01:16:33.311010+0000","last_became_active":"2023-05-20T01:16:32.362592+0000","last_became_peered":"2023-05-20T01:16:32.362592+0000","last_unstale":"2023-05-20T01:16:33.311010+0000","last_undegraded":"2023-05-20T01:16:33.311010+0000","last_fullsized":"2023-05-20T01:16:33.311010+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:46:46.150220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3,0],"acting":[5,1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.311751+0000","last_change":"2023-05-20T01:16:27.925307+0000","last_active":"2023-05-20T01:16:33.311751+0000","last_peered":"2023-05-20T01:16:33.311751+0000","last_clean":"2023-05-20T01:16:33.311751+0000","last_became_active":"2023-05-20T01:16:27.925044+0000","last_became_peered":"2023-05-20T01:16:27.925044+0000","last_unstale":"2023-05-20T01:16:33.311751+0000","last_undegraded":"2023-05-20T01:16:33.311751+0000","last_fullsized":"2023-05-20T01:16:33.311751+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:54:11.307738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.772200+0000","last_change":"2023-05-20T01:16:28.340299+0000","last_active":"2023-05-20T01:16:32.772200+0000","last_peered":"2023-05-20T01:16:32.772200+0000","last_clean":"2023-05-20T01:16:32.772200+0000","last_became_active":"2023-05-20T01:16:28.340002+0000","last_became_peered":"2023-05-20T01:16:28.340002+0000","last_unstale":"2023-05-20T01:16:32.772200+0000","last_undegraded":"2023-05-20T01:16:32.772200+0000","last_fullsized":"2023-05-20T01:16:32.772200+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T01:44:35.746448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.772637+0000","last_change":"2023-05-20T01:16:32.354337+0000","last_active":"2023-05-20T01:16:32.772637+0000","last_peered":"2023-05-20T01:16:32.772637+0000","last_clean":"2023-05-20T01:16:32.772637+0000","last_became_active":"2023-05-20T01:16:32.353900+0000","last_became_peered":"2023-05-20T01:16:32.353900+0000","last_unstale":"2023-05-20T01:16:32.772637+0000","last_undegraded":"2023-05-20T01:16:32.772637+0000","last_fullsized":"2023-05-20T01:16:32.772637+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:50:05.163209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2,1],"acting":[3,7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.954416+0000","last_change":"2023-05-20T01:16:27.916514+0000","last_active":"2023-05-20T01:16:32.954416+0000","last_peered":"2023-05-20T01:16:32.954416+0000","last_clean":"2023-05-20T01:16:32.954416+0000","last_became_active":"2023-05-20T01:16:27.916199+0000","last_became_peered":"2023-05-20T01:16:27.916199+0000","last_unstale":"2023-05-20T01:16:32.954416+0000","last_undegraded":"2023-05-20T01:16:32.954416+0000","last_fullsized":"2023-05-20T01:16:32.954416+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:21:51.991048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.310678+0000","last_change":"2023-05-20T01:16:28.335549+0000","last_active":"2023-05-20T01:16:33.310678+0000","last_peered":"2023-05-20T01:16:33.310678+0000","last_clean":"2023-05-20T01:16:33.310678+0000","last_became_active":"2023-05-20T01:16:28.335284+0000","last_became_peered":"2023-05-20T01:16:28.335284+0000","last_unstale":"2023-05-20T01:16:33.310678+0000","last_undegraded":"2023-05-20T01:16:33.310678+0000","last_fullsized":"2023-05-20T01:16:33.310678+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:21:52.998812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.772615+0000","last_change":"2023-05-20T01:16:32.362344+0000","last_active":"2023-05-20T01:16:32.772615+0000","last_peered":"2023-05-20T01:16:32.772615+0000","last_clean":"2023-05-20T01:16:32.772615+0000","last_became_active":"2023-05-20T01:16:32.361944+0000","last_became_peered":"2023-05-20T01:16:32.361944+0000","last_unstale":"2023-05-20T01:16:32.772615+0000","last_undegraded":"2023-05-20T01:16:32.772615+0000","last_fullsized":"2023-05-20T01:16:32.772615+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:44:02.823618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,0],"acting":[3,1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.311580+0000","last_change":"2023-05-20T01:16:27.926496+0000","last_active":"2023-05-20T01:16:33.311580+0000","last_peered":"2023-05-20T01:16:33.311580+0000","last_clean":"2023-05-20T01:16:33.311580+0000","last_became_active":"2023-05-20T01:16:27.926265+0000","last_became_peered":"2023-05-20T01:16:27.926265+0000","last_unstale":"2023-05-20T01:16:33.311580+0000","last_undegraded":"2023-05-20T01:16:33.311580+0000","last_fullsized":"2023-05-20T01:16:33.311580+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:23:32.724521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.954832+0000","last_change":"2023-05-20T01:16:28.348384+0000","last_active":"2023-05-20T01:16:32.954832+0000","last_peered":"2023-05-20T01:16:32.954832+0000","last_clean":"2023-05-20T01:16:32.954832+0000","last_became_active":"2023-05-20T01:16:28.347652+0000","last_became_peered":"2023-05-20T01:16:28.347652+0000","last_unstale":"2023-05-20T01:16:32.954832+0000","last_undegraded":"2023-05-20T01:16:32.954832+0000","last_fullsized":"2023-05-20T01:16:32.954832+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:49:21.195547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.310448+0000","last_change":"2023-05-20T01:16:32.352418+0000","last_active":"2023-05-20T01:16:33.310448+0000","last_peered":"2023-05-20T01:16:33.310448+0000","last_clean":"2023-05-20T01:16:33.310448+0000","last_became_active":"2023-05-20T01:16:32.352031+0000","last_became_peered":"2023-05-20T01:16:32.352031+0000","last_unstale":"2023-05-20T01:16:33.310448+0000","last_undegraded":"2023-05-20T01:16:33.310448+0000","last_fullsized":"2023-05-20T01:16:33.310448+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:05:24.403434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1,6],"acting":[5,3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"20'8","reported_seq":29,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.773007+0000","last_change":"2023-05-20T01:16:27.916957+0000","last_active":"2023-05-20T01:16:32.773007+0000","last_peered":"2023-05-20T01:16:32.773007+0000","last_clean":"2023-05-20T01:16:32.773007+0000","last_became_active":"2023-05-20T01:16:27.916682+0000","last_became_peered":"2023-05-20T01:16:27.916682+0000","last_unstale":"2023-05-20T01:16:32.773007+0000","last_undegraded":"2023-05-20T01:16:32.773007+0000","last_fullsized":"2023-05-20T01:16:32.773007+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:15:10.960002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.772882+0000","last_change":"2023-05-20T01:16:28.339442+0000","last_active":"2023-05-20T01:16:32.772882+0000","last_peered":"2023-05-20T01:16:32.772882+0000","last_clean":"2023-05-20T01:16:32.772882+0000","last_became_active":"2023-05-20T01:16:28.339182+0000","last_became_peered":"2023-05-20T01:16:28.339182+0000","last_unstale":"2023-05-20T01:16:32.772882+0000","last_undegraded":"2023-05-20T01:16:32.772882+0000","last_fullsized":"2023-05-20T01:16:32.772882+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:31:47.073658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.311789+0000","last_change":"2023-05-20T01:16:32.360635+0000","last_active":"2023-05-20T01:16:33.311789+0000","last_peered":"2023-05-20T01:16:33.311789+0000","last_clean":"2023-05-20T01:16:33.311789+0000","last_became_active":"2023-05-20T01:16:32.360032+0000","last_became_peered":"2023-05-20T01:16:32.360032+0000","last_unstale":"2023-05-20T01:16:33.311789+0000","last_undegraded":"2023-05-20T01:16:33.311789+0000","last_fullsized":"2023-05-20T01:16:33.311789+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:49:24.624253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3,4],"acting":[7,6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"20'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.312203+0000","last_change":"2023-05-20T01:16:27.926418+0000","last_active":"2023-05-20T01:16:33.312203+0000","last_peered":"2023-05-20T01:16:33.312203+0000","last_clean":"2023-05-20T01:16:33.312203+0000","last_became_active":"2023-05-20T01:16:27.926135+0000","last_became_peered":"2023-05-20T01:16:27.926135+0000","last_unstale":"2023-05-20T01:16:33.312203+0000","last_undegraded":"2023-05-20T01:16:33.312203+0000","last_fullsized":"2023-05-20T01:16:33.312203+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:53:31.196496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.310060+0000","last_change":"2023-05-20T01:16:28.333381+0000","last_active":"2023-05-20T01:16:33.310060+0000","last_peered":"2023-05-20T01:16:33.310060+0000","last_clean":"2023-05-20T01:16:33.310060+0000","last_became_active":"2023-05-20T01:16:28.333096+0000","last_became_peered":"2023-05-20T01:16:28.333096+0000","last_unstale":"2023-05-20T01:16:33.310060+0000","last_undegraded":"2023-05-20T01:16:33.310060+0000","last_fullsized":"2023-05-20T01:16:33.310060+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:47:13.680746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.310001+0000","last_change":"2023-05-20T01:16:32.353695+0000","last_active":"2023-05-20T01:16:33.310001+0000","last_peered":"2023-05-20T01:16:33.310001+0000","last_clean":"2023-05-20T01:16:33.310001+0000","last_became_active":"2023-05-20T01:16:32.353280+0000","last_became_peered":"2023-05-20T01:16:32.353280+0000","last_unstale":"2023-05-20T01:16:33.310001+0000","last_undegraded":"2023-05-20T01:16:33.310001+0000","last_fullsized":"2023-05-20T01:16:33.310001+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:53:47.536905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1,5],"acting":[2,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.312036+0000","last_change":"2023-05-20T01:16:27.926342+0000","last_active":"2023-05-20T01:16:33.312036+0000","last_peered":"2023-05-20T01:16:33.312036+0000","last_clean":"2023-05-20T01:16:33.312036+0000","last_became_active":"2023-05-20T01:16:27.926093+0000","last_became_peered":"2023-05-20T01:16:27.926093+0000","last_unstale":"2023-05-20T01:16:33.312036+0000","last_undegraded":"2023-05-20T01:16:33.312036+0000","last_fullsized":"2023-05-20T01:16:33.312036+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:22:46.049477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.774721+0000","last_change":"2023-05-20T01:16:28.338568+0000","last_active":"2023-05-20T01:16:32.774721+0000","last_peered":"2023-05-20T01:16:32.774721+0000","last_clean":"2023-05-20T01:16:32.774721+0000","last_became_active":"2023-05-20T01:16:28.338293+0000","last_became_peered":"2023-05-20T01:16:28.338293+0000","last_unstale":"2023-05-20T01:16:32.774721+0000","last_undegraded":"2023-05-20T01:16:32.774721+0000","last_fullsized":"2023-05-20T01:16:32.774721+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:49:07.725723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.310172+0000","last_change":"2023-05-20T01:16:32.358154+0000","last_active":"2023-05-20T01:16:33.310172+0000","last_peered":"2023-05-20T01:16:33.310172+0000","last_clean":"2023-05-20T01:16:33.310172+0000","last_became_active":"2023-05-20T01:16:32.357690+0000","last_became_peered":"2023-05-20T01:16:32.357690+0000","last_unstale":"2023-05-20T01:16:33.310172+0000","last_undegraded":"2023-05-20T01:16:33.310172+0000","last_fullsized":"2023-05-20T01:16:33.310172+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:04:33.803901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5,6],"acting":[2,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"20'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.311721+0000","last_change":"2023-05-20T01:16:27.922534+0000","last_active":"2023-05-20T01:16:33.311721+0000","last_peered":"2023-05-20T01:16:33.311721+0000","last_clean":"2023-05-20T01:16:33.311721+0000","last_became_active":"2023-05-20T01:16:27.922263+0000","last_became_peered":"2023-05-20T01:16:27.922263+0000","last_unstale":"2023-05-20T01:16:33.311721+0000","last_undegraded":"2023-05-20T01:16:33.311721+0000","last_fullsized":"2023-05-20T01:16:33.311721+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:25:21.983559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.773343+0000","last_change":"2023-05-20T01:16:28.345113+0000","last_active":"2023-05-20T01:16:32.773343+0000","last_peered":"2023-05-20T01:16:32.773343+0000","last_clean":"2023-05-20T01:16:32.773343+0000","last_became_active":"2023-05-20T01:16:28.344807+0000","last_became_peered":"2023-05-20T01:16:28.344807+0000","last_unstale":"2023-05-20T01:16:32.773343+0000","last_undegraded":"2023-05-20T01:16:32.773343+0000","last_fullsized":"2023-05-20T01:16:32.773343+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:31:16.197740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.310276+0000","last_change":"2023-05-20T01:16:32.346858+0000","last_active":"2023-05-20T01:16:33.310276+0000","last_peered":"2023-05-20T01:16:33.310276+0000","last_clean":"2023-05-20T01:16:33.310276+0000","last_became_active":"2023-05-20T01:16:32.346456+0000","last_became_peered":"2023-05-20T01:16:32.346456+0000","last_unstale":"2023-05-20T01:16:33.310276+0000","last_undegraded":"2023-05-20T01:16:33.310276+0000","last_fullsized":"2023-05-20T01:16:33.310276+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:30:20.804962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3,6],"acting":[2,4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.310878+0000","last_change":"2023-05-20T01:16:27.912388+0000","last_active":"2023-05-20T01:16:33.310878+0000","last_peered":"2023-05-20T01:16:33.310878+0000","last_clean":"2023-05-20T01:16:33.310878+0000","last_became_active":"2023-05-20T01:16:27.911977+0000","last_became_peered":"2023-05-20T01:16:27.911977+0000","last_unstale":"2023-05-20T01:16:33.310878+0000","last_undegraded":"2023-05-20T01:16:33.310878+0000","last_fullsized":"2023-05-20T01:16:33.310878+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:19:50.512317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.954435+0000","last_change":"2023-05-20T01:16:28.342633+0000","last_active":"2023-05-20T01:16:32.954435+0000","last_peered":"2023-05-20T01:16:32.954435+0000","last_clean":"2023-05-20T01:16:32.954435+0000","last_became_active":"2023-05-20T01:16:28.339419+0000","last_became_peered":"2023-05-20T01:16:28.339419+0000","last_unstale":"2023-05-20T01:16:32.954435+0000","last_undegraded":"2023-05-20T01:16:32.954435+0000","last_fullsized":"2023-05-20T01:16:32.954435+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:27:57.460519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.310304+0000","last_change":"2023-05-20T01:16:32.347930+0000","last_active":"2023-05-20T01:16:33.310304+0000","last_peered":"2023-05-20T01:16:33.310304+0000","last_clean":"2023-05-20T01:16:33.310304+0000","last_became_active":"2023-05-20T01:16:32.347502+0000","last_became_peered":"2023-05-20T01:16:32.347502+0000","last_unstale":"2023-05-20T01:16:33.310304+0000","last_undegraded":"2023-05-20T01:16:33.310304+0000","last_fullsized":"2023-05-20T01:16:33.310304+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:05:15.989168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5,6],"acting":[2,7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.772049+0000","last_change":"2023-05-20T01:16:27.909757+0000","last_active":"2023-05-20T01:16:32.772049+0000","last_peered":"2023-05-20T01:16:32.772049+0000","last_clean":"2023-05-20T01:16:32.772049+0000","last_became_active":"2023-05-20T01:16:27.909458+0000","last_became_peered":"2023-05-20T01:16:27.909458+0000","last_unstale":"2023-05-20T01:16:32.772049+0000","last_undegraded":"2023-05-20T01:16:32.772049+0000","last_fullsized":"2023-05-20T01:16:32.772049+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:26:58.729159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.773572+0000","last_change":"2023-05-20T01:16:28.340078+0000","last_active":"2023-05-20T01:16:32.773572+0000","last_peered":"2023-05-20T01:16:32.773572+0000","last_clean":"2023-05-20T01:16:32.773572+0000","last_became_active":"2023-05-20T01:16:28.339673+0000","last_became_peered":"2023-05-20T01:16:28.339673+0000","last_unstale":"2023-05-20T01:16:32.773572+0000","last_undegraded":"2023-05-20T01:16:32.773572+0000","last_fullsized":"2023-05-20T01:16:32.773572+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:13:55.558140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.957039+0000","last_change":"2023-05-20T01:16:32.364538+0000","last_active":"2023-05-20T01:16:32.957039+0000","last_peered":"2023-05-20T01:16:32.957039+0000","last_clean":"2023-05-20T01:16:32.957039+0000","last_became_active":"2023-05-20T01:16:32.364025+0000","last_became_peered":"2023-05-20T01:16:32.364025+0000","last_unstale":"2023-05-20T01:16:32.957039+0000","last_undegraded":"2023-05-20T01:16:32.957039+0000","last_fullsized":"2023-05-20T01:16:32.957039+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:01:04.644797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0,6],"acting":[1,4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.774177+0000","last_change":"2023-05-20T01:16:27.922354+0000","last_active":"2023-05-20T01:16:32.774177+0000","last_peered":"2023-05-20T01:16:32.774177+0000","last_clean":"2023-05-20T01:16:32.774177+0000","last_became_active":"2023-05-20T01:16:27.922000+0000","last_became_peered":"2023-05-20T01:16:27.922000+0000","last_unstale":"2023-05-20T01:16:32.774177+0000","last_undegraded":"2023-05-20T01:16:32.774177+0000","last_fullsized":"2023-05-20T01:16:32.774177+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:56:49.903435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.774102+0000","last_change":"2023-05-20T01:16:28.334441+0000","last_active":"2023-05-20T01:16:32.774102+0000","last_peered":"2023-05-20T01:16:32.774102+0000","last_clean":"2023-05-20T01:16:32.774102+0000","last_became_active":"2023-05-20T01:16:28.334157+0000","last_became_peered":"2023-05-20T01:16:28.334157+0000","last_unstale":"2023-05-20T01:16:32.774102+0000","last_undegraded":"2023-05-20T01:16:32.774102+0000","last_fullsized":"2023-05-20T01:16:32.774102+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:51:16.225196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.310897+0000","last_change":"2023-05-20T01:16:32.346834+0000","last_active":"2023-05-20T01:16:33.310897+0000","last_peered":"2023-05-20T01:16:33.310897+0000","last_clean":"2023-05-20T01:16:33.310897+0000","last_became_active":"2023-05-20T01:16:32.346410+0000","last_became_peered":"2023-05-20T01:16:32.346410+0000","last_unstale":"2023-05-20T01:16:33.310897+0000","last_undegraded":"2023-05-20T01:16:33.310897+0000","last_fullsized":"2023-05-20T01:16:33.310897+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:07:08.123139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6,2],"acting":[5,4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.774525+0000","last_change":"2023-05-20T01:16:27.921658+0000","last_active":"2023-05-20T01:16:32.774525+0000","last_peered":"2023-05-20T01:16:32.774525+0000","last_clean":"2023-05-20T01:16:32.774525+0000","last_became_active":"2023-05-20T01:16:27.921399+0000","last_became_peered":"2023-05-20T01:16:27.921399+0000","last_unstale":"2023-05-20T01:16:32.774525+0000","last_undegraded":"2023-05-20T01:16:32.774525+0000","last_fullsized":"2023-05-20T01:16:32.774525+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:43:09.775404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.775593+0000","last_change":"2023-05-20T01:16:28.347570+0000","last_active":"2023-05-20T01:16:32.775593+0000","last_peered":"2023-05-20T01:16:32.775593+0000","last_clean":"2023-05-20T01:16:32.775593+0000","last_became_active":"2023-05-20T01:16:28.347140+0000","last_became_peered":"2023-05-20T01:16:28.347140+0000","last_unstale":"2023-05-20T01:16:32.775593+0000","last_undegraded":"2023-05-20T01:16:32.775593+0000","last_fullsized":"2023-05-20T01:16:32.775593+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T01:30:46.937626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.773539+0000","last_change":"2023-05-20T01:16:32.363537+0000","last_active":"2023-05-20T01:16:32.773539+0000","last_peered":"2023-05-20T01:16:32.773539+0000","last_clean":"2023-05-20T01:16:32.773539+0000","last_became_active":"2023-05-20T01:16:32.363119+0000","last_became_peered":"2023-05-20T01:16:32.363119+0000","last_unstale":"2023-05-20T01:16:32.773539+0000","last_undegraded":"2023-05-20T01:16:32.773539+0000","last_fullsized":"2023-05-20T01:16:32.773539+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:40:04.730139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0,2],"acting":[4,5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.771744+0000","last_change":"2023-05-20T01:16:27.924294+0000","last_active":"2023-05-20T01:16:32.771744+0000","last_peered":"2023-05-20T01:16:32.771744+0000","last_clean":"2023-05-20T01:16:32.771744+0000","last_became_active":"2023-05-20T01:16:27.923930+0000","last_became_peered":"2023-05-20T01:16:27.923930+0000","last_unstale":"2023-05-20T01:16:32.771744+0000","last_undegraded":"2023-05-20T01:16:32.771744+0000","last_fullsized":"2023-05-20T01:16:32.771744+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:24:35.039391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.957544+0000","last_change":"2023-05-20T01:16:28.349390+0000","last_active":"2023-05-20T01:16:32.957544+0000","last_peered":"2023-05-20T01:16:32.957544+0000","last_clean":"2023-05-20T01:16:32.957544+0000","last_became_active":"2023-05-20T01:16:28.349074+0000","last_became_peered":"2023-05-20T01:16:28.349074+0000","last_unstale":"2023-05-20T01:16:32.957544+0000","last_undegraded":"2023-05-20T01:16:32.957544+0000","last_fullsized":"2023-05-20T01:16:32.957544+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:59:48.441604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.774884+0000","last_change":"2023-05-20T01:16:32.353210+0000","last_active":"2023-05-20T01:16:32.774884+0000","last_peered":"2023-05-20T01:16:32.774884+0000","last_clean":"2023-05-20T01:16:32.774884+0000","last_became_active":"2023-05-20T01:16:32.352807+0000","last_became_peered":"2023-05-20T01:16:32.352807+0000","last_unstale":"2023-05-20T01:16:32.774884+0000","last_undegraded":"2023-05-20T01:16:32.774884+0000","last_fullsized":"2023-05-20T01:16:32.774884+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:42:22.830436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7,0],"acting":[4,3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.772776+0000","last_change":"2023-05-20T01:16:27.913270+0000","last_active":"2023-05-20T01:16:32.772776+0000","last_peered":"2023-05-20T01:16:32.772776+0000","last_clean":"2023-05-20T01:16:32.772776+0000","last_became_active":"2023-05-20T01:16:27.912952+0000","last_became_peered":"2023-05-20T01:16:27.912952+0000","last_unstale":"2023-05-20T01:16:32.772776+0000","last_undegraded":"2023-05-20T01:16:32.772776+0000","last_fullsized":"2023-05-20T01:16:32.772776+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:34:35.188240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.774248+0000","last_change":"2023-05-20T01:16:23.289068+0000","last_active":"2023-05-20T01:16:32.774248+0000","last_peered":"2023-05-20T01:16:32.774248+0000","last_clean":"2023-05-20T01:16:32.774248+0000","last_became_active":"2023-05-20T01:16:23.288781+0000","last_became_peered":"2023-05-20T01:16:23.288781+0000","last_unstale":"2023-05-20T01:16:32.774248+0000","last_undegraded":"2023-05-20T01:16:32.774248+0000","last_fullsized":"2023-05-20T01:16:32.774248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:02:03.239372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.771607+0000","last_change":"2023-05-20T01:16:28.329739+0000","last_active":"2023-05-20T01:16:32.771607+0000","last_peered":"2023-05-20T01:16:32.771607+0000","last_clean":"2023-05-20T01:16:32.771607+0000","last_became_active":"2023-05-20T01:16:28.329483+0000","last_became_peered":"2023-05-20T01:16:28.329483+0000","last_unstale":"2023-05-20T01:16:32.771607+0000","last_undegraded":"2023-05-20T01:16:32.771607+0000","last_fullsized":"2023-05-20T01:16:32.771607+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:46:14.388881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.771640+0000","last_change":"2023-05-20T01:16:32.353144+0000","last_active":"2023-05-20T01:16:32.771640+0000","last_peered":"2023-05-20T01:16:32.771640+0000","last_clean":"2023-05-20T01:16:32.771640+0000","last_became_active":"2023-05-20T01:16:32.352355+0000","last_became_peered":"2023-05-20T01:16:32.352355+0000","last_unstale":"2023-05-20T01:16:32.771640+0000","last_undegraded":"2023-05-20T01:16:32.771640+0000","last_fullsized":"2023-05-20T01:16:32.771640+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:25:26.420028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,0],"acting":[3,1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.772224+0000","last_change":"2023-05-20T01:16:22.829324+0000","last_active":"2023-05-20T01:16:32.772224+0000","last_peered":"2023-05-20T01:16:32.772224+0000","last_clean":"2023-05-20T01:16:32.772224+0000","last_became_active":"2023-05-20T01:16:22.828964+0000","last_became_peered":"2023-05-20T01:16:22.828964+0000","last_unstale":"2023-05-20T01:16:32.772224+0000","last_undegraded":"2023-05-20T01:16:32.772224+0000","last_fullsized":"2023-05-20T01:16:32.772224+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:33:22.739978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"25'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.954588+0000","last_change":"2023-05-20T01:16:27.922339+0000","last_active":"2023-05-20T01:16:32.954588+0000","last_peered":"2023-05-20T01:16:32.954588+0000","last_clean":"2023-05-20T01:16:32.954588+0000","last_became_active":"2023-05-20T01:16:27.922033+0000","last_became_peered":"2023-05-20T01:16:27.922033+0000","last_unstale":"2023-05-20T01:16:32.954588+0000","last_undegraded":"2023-05-20T01:16:32.954588+0000","last_fullsized":"2023-05-20T01:16:32.954588+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:01:20.476920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.774598+0000","last_change":"2023-05-20T01:16:28.335846+0000","last_active":"2023-05-20T01:16:32.774598+0000","last_peered":"2023-05-20T01:16:32.774598+0000","last_clean":"2023-05-20T01:16:32.774598+0000","last_became_active":"2023-05-20T01:16:28.335523+0000","last_became_peered":"2023-05-20T01:16:28.335523+0000","last_unstale":"2023-05-20T01:16:32.774598+0000","last_undegraded":"2023-05-20T01:16:32.774598+0000","last_fullsized":"2023-05-20T01:16:32.774598+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:18:05.626837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.310654+0000","last_change":"2023-05-20T01:16:32.363029+0000","last_active":"2023-05-20T01:16:33.310654+0000","last_peered":"2023-05-20T01:16:33.310654+0000","last_clean":"2023-05-20T01:16:33.310654+0000","last_became_active":"2023-05-20T01:16:32.362637+0000","last_became_peered":"2023-05-20T01:16:32.362637+0000","last_unstale":"2023-05-20T01:16:33.310654+0000","last_undegraded":"2023-05-20T01:16:33.310654+0000","last_fullsized":"2023-05-20T01:16:33.310654+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:28:00.889225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0,4],"acting":[5,1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.956901+0000","last_change":"2023-05-20T01:16:22.834754+0000","last_active":"2023-05-20T01:16:32.956901+0000","last_peered":"2023-05-20T01:16:32.956901+0000","last_clean":"2023-05-20T01:16:32.956901+0000","last_became_active":"2023-05-20T01:16:22.834437+0000","last_became_peered":"2023-05-20T01:16:22.834437+0000","last_unstale":"2023-05-20T01:16:32.956901+0000","last_undegraded":"2023-05-20T01:16:32.956901+0000","last_fullsized":"2023-05-20T01:16:32.956901+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:51:36.657132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":177,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.311443+0000","last_change":"2023-05-20T01:16:22.834920+0000","last_active":"2023-05-20T01:16:33.311443+0000","last_peered":"2023-05-20T01:16:33.311443+0000","last_clean":"2023-05-20T01:16:33.311443+0000","last_became_active":"2023-05-20T01:16:22.833924+0000","last_became_peered":"2023-05-20T01:16:22.833924+0000","last_unstale":"2023-05-20T01:16:33.311443+0000","last_undegraded":"2023-05-20T01:16:33.311443+0000","last_fullsized":"2023-05-20T01:16:33.311443+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:34:02.513729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.954099+0000","last_change":"2023-05-20T01:16:27.923011+0000","last_active":"2023-05-20T01:16:32.954099+0000","last_peered":"2023-05-20T01:16:32.954099+0000","last_clean":"2023-05-20T01:16:32.954099+0000","last_became_active":"2023-05-20T01:16:27.922736+0000","last_became_peered":"2023-05-20T01:16:27.922736+0000","last_unstale":"2023-05-20T01:16:32.954099+0000","last_undegraded":"2023-05-20T01:16:32.954099+0000","last_fullsized":"2023-05-20T01:16:32.954099+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:59:47.433840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.954829+0000","last_change":"2023-05-20T01:16:28.348199+0000","last_active":"2023-05-20T01:16:32.954829+0000","last_peered":"2023-05-20T01:16:32.954829+0000","last_clean":"2023-05-20T01:16:32.954829+0000","last_became_active":"2023-05-20T01:16:28.347370+0000","last_became_peered":"2023-05-20T01:16:28.347370+0000","last_unstale":"2023-05-20T01:16:32.954829+0000","last_undegraded":"2023-05-20T01:16:32.954829+0000","last_fullsized":"2023-05-20T01:16:32.954829+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:46:14.388881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.310467+0000","last_change":"2023-05-20T01:16:32.357823+0000","last_active":"2023-05-20T01:16:33.310467+0000","last_peered":"2023-05-20T01:16:33.310467+0000","last_clean":"2023-05-20T01:16:33.310467+0000","last_became_active":"2023-05-20T01:16:32.357437+0000","last_became_peered":"2023-05-20T01:16:32.357437+0000","last_unstale":"2023-05-20T01:16:33.310467+0000","last_undegraded":"2023-05-20T01:16:33.310467+0000","last_fullsized":"2023-05-20T01:16:33.310467+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:16:08.791218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1,7],"acting":[2,5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.311398+0000","last_change":"2023-05-20T01:16:23.290259+0000","last_active":"2023-05-20T01:16:33.311398+0000","last_peered":"2023-05-20T01:16:33.311398+0000","last_clean":"2023-05-20T01:16:33.311398+0000","last_became_active":"2023-05-20T01:16:23.289911+0000","last_became_peered":"2023-05-20T01:16:23.289911+0000","last_unstale":"2023-05-20T01:16:33.311398+0000","last_undegraded":"2023-05-20T01:16:33.311398+0000","last_fullsized":"2023-05-20T01:16:33.311398+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:46:36.633677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.956681+0000","last_change":"2023-05-20T01:16:27.919248+0000","last_active":"2023-05-20T01:16:32.956681+0000","last_peered":"2023-05-20T01:16:32.956681+0000","last_clean":"2023-05-20T01:16:32.956681+0000","last_became_active":"2023-05-20T01:16:27.918957+0000","last_became_peered":"2023-05-20T01:16:27.918957+0000","last_unstale":"2023-05-20T01:16:32.956681+0000","last_undegraded":"2023-05-20T01:16:32.956681+0000","last_fullsized":"2023-05-20T01:16:32.956681+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:51:16.461763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.957471+0000","last_change":"2023-05-20T01:16:28.337136+0000","last_active":"2023-05-20T01:16:32.957471+0000","last_peered":"2023-05-20T01:16:32.957471+0000","last_clean":"2023-05-20T01:16:32.957471+0000","last_became_active":"2023-05-20T01:16:28.336322+0000","last_became_peered":"2023-05-20T01:16:28.336322+0000","last_unstale":"2023-05-20T01:16:32.957471+0000","last_undegraded":"2023-05-20T01:16:32.957471+0000","last_fullsized":"2023-05-20T01:16:32.957471+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:42:59.927187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.311713+0000","last_change":"2023-05-20T01:16:32.358840+0000","last_active":"2023-05-20T01:16:33.311713+0000","last_peered":"2023-05-20T01:16:33.311713+0000","last_clean":"2023-05-20T01:16:33.311713+0000","last_became_active":"2023-05-20T01:16:32.358470+0000","last_became_peered":"2023-05-20T01:16:32.358470+0000","last_unstale":"2023-05-20T01:16:33.311713+0000","last_undegraded":"2023-05-20T01:16:33.311713+0000","last_fullsized":"2023-05-20T01:16:33.311713+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:14:35.048109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5,3],"acting":[7,2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.311131+0000","last_change":"2023-05-20T01:16:23.290207+0000","last_active":"2023-05-20T01:16:33.311131+0000","last_peered":"2023-05-20T01:16:33.311131+0000","last_clean":"2023-05-20T01:16:33.311131+0000","last_became_active":"2023-05-20T01:16:23.289928+0000","last_became_peered":"2023-05-20T01:16:23.289928+0000","last_unstale":"2023-05-20T01:16:33.311131+0000","last_undegraded":"2023-05-20T01:16:33.311131+0000","last_fullsized":"2023-05-20T01:16:33.311131+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:01:15.396847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.771970+0000","last_change":"2023-05-20T01:16:27.911483+0000","last_active":"2023-05-20T01:16:32.771970+0000","last_peered":"2023-05-20T01:16:32.771970+0000","last_clean":"2023-05-20T01:16:32.771970+0000","last_became_active":"2023-05-20T01:16:27.911180+0000","last_became_peered":"2023-05-20T01:16:27.911180+0000","last_unstale":"2023-05-20T01:16:32.771970+0000","last_undegraded":"2023-05-20T01:16:32.771970+0000","last_fullsized":"2023-05-20T01:16:32.771970+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:13:48.951321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.773506+0000","last_change":"2023-05-20T01:16:28.345161+0000","last_active":"2023-05-20T01:16:32.773506+0000","last_peered":"2023-05-20T01:16:32.773506+0000","last_clean":"2023-05-20T01:16:32.773506+0000","last_became_active":"2023-05-20T01:16:28.344872+0000","last_became_peered":"2023-05-20T01:16:28.344872+0000","last_unstale":"2023-05-20T01:16:32.773506+0000","last_undegraded":"2023-05-20T01:16:32.773506+0000","last_fullsized":"2023-05-20T01:16:32.773506+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:49:51.917546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.954428+0000","last_change":"2023-05-20T01:16:32.362192+0000","last_active":"2023-05-20T01:16:32.954428+0000","last_peered":"2023-05-20T01:16:32.954428+0000","last_clean":"2023-05-20T01:16:32.954428+0000","last_became_active":"2023-05-20T01:16:32.361656+0000","last_became_peered":"2023-05-20T01:16:32.361656+0000","last_unstale":"2023-05-20T01:16:32.954428+0000","last_undegraded":"2023-05-20T01:16:32.954428+0000","last_fullsized":"2023-05-20T01:16:32.954428+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:58:35.944694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,6],"acting":[0,1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.310220+0000","last_change":"2023-05-20T01:16:23.290538+0000","last_active":"2023-05-20T01:16:33.310220+0000","last_peered":"2023-05-20T01:16:33.310220+0000","last_clean":"2023-05-20T01:16:33.310220+0000","last_became_active":"2023-05-20T01:16:23.290230+0000","last_became_peered":"2023-05-20T01:16:23.290230+0000","last_unstale":"2023-05-20T01:16:33.310220+0000","last_undegraded":"2023-05-20T01:16:33.310220+0000","last_fullsized":"2023-05-20T01:16:33.310220+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:24:09.597242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.773435+0000","last_change":"2023-05-20T01:16:27.921749+0000","last_active":"2023-05-20T01:16:32.773435+0000","last_peered":"2023-05-20T01:16:32.773435+0000","last_clean":"2023-05-20T01:16:32.773435+0000","last_became_active":"2023-05-20T01:16:27.921461+0000","last_became_peered":"2023-05-20T01:16:27.921461+0000","last_unstale":"2023-05-20T01:16:32.773435+0000","last_undegraded":"2023-05-20T01:16:32.773435+0000","last_fullsized":"2023-05-20T01:16:32.773435+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:01:30.293877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.957033+0000","last_change":"2023-05-20T01:16:28.335769+0000","last_active":"2023-05-20T01:16:32.957033+0000","last_peered":"2023-05-20T01:16:32.957033+0000","last_clean":"2023-05-20T01:16:32.957033+0000","last_became_active":"2023-05-20T01:16:28.335232+0000","last_became_peered":"2023-05-20T01:16:28.335232+0000","last_unstale":"2023-05-20T01:16:32.957033+0000","last_undegraded":"2023-05-20T01:16:32.957033+0000","last_fullsized":"2023-05-20T01:16:32.957033+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:23:18.405128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.774369+0000","last_change":"2023-05-20T01:16:32.354871+0000","last_active":"2023-05-20T01:16:32.774369+0000","last_peered":"2023-05-20T01:16:32.774369+0000","last_clean":"2023-05-20T01:16:32.774369+0000","last_became_active":"2023-05-20T01:16:32.353771+0000","last_became_peered":"2023-05-20T01:16:32.353771+0000","last_unstale":"2023-05-20T01:16:32.774369+0000","last_undegraded":"2023-05-20T01:16:32.774369+0000","last_fullsized":"2023-05-20T01:16:32.774369+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:22:02.793310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5,2],"acting":[6,0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.773241+0000","last_change":"2023-05-20T01:16:23.290341+0000","last_active":"2023-05-20T01:16:32.773241+0000","last_peered":"2023-05-20T01:16:32.773241+0000","last_clean":"2023-05-20T01:16:32.773241+0000","last_became_active":"2023-05-20T01:16:23.290053+0000","last_became_peered":"2023-05-20T01:16:23.290053+0000","last_unstale":"2023-05-20T01:16:32.773241+0000","last_undegraded":"2023-05-20T01:16:32.773241+0000","last_fullsized":"2023-05-20T01:16:32.773241+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:11:18.465994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.957177+0000","last_change":"2023-05-20T01:16:27.920036+0000","last_active":"2023-05-20T01:16:32.957177+0000","last_peered":"2023-05-20T01:16:32.957177+0000","last_clean":"2023-05-20T01:16:32.957177+0000","last_became_active":"2023-05-20T01:16:27.919757+0000","last_became_peered":"2023-05-20T01:16:27.919757+0000","last_unstale":"2023-05-20T01:16:32.957177+0000","last_undegraded":"2023-05-20T01:16:32.957177+0000","last_fullsized":"2023-05-20T01:16:32.957177+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:49:54.418261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.954564+0000","last_change":"2023-05-20T01:16:28.343036+0000","last_active":"2023-05-20T01:16:32.954564+0000","last_peered":"2023-05-20T01:16:32.954564+0000","last_clean":"2023-05-20T01:16:32.954564+0000","last_became_active":"2023-05-20T01:16:28.342814+0000","last_became_peered":"2023-05-20T01:16:28.342814+0000","last_unstale":"2023-05-20T01:16:32.954564+0000","last_undegraded":"2023-05-20T01:16:32.954564+0000","last_fullsized":"2023-05-20T01:16:32.954564+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:13:57.518009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.954367+0000","last_change":"2023-05-20T01:16:32.356221+0000","last_active":"2023-05-20T01:16:32.954367+0000","last_peered":"2023-05-20T01:16:32.954367+0000","last_clean":"2023-05-20T01:16:32.954367+0000","last_became_active":"2023-05-20T01:16:32.355772+0000","last_became_peered":"2023-05-20T01:16:32.355772+0000","last_unstale":"2023-05-20T01:16:32.954367+0000","last_undegraded":"2023-05-20T01:16:32.954367+0000","last_fullsized":"2023-05-20T01:16:32.954367+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:07:08.123139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6,5],"acting":[0,1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.774068+0000","last_change":"2023-05-20T01:16:22.834330+0000","last_active":"2023-05-20T01:16:32.774068+0000","last_peered":"2023-05-20T01:16:32.774068+0000","last_clean":"2023-05-20T01:16:32.774068+0000","last_became_active":"2023-05-20T01:16:22.833946+0000","last_became_peered":"2023-05-20T01:16:22.833946+0000","last_unstale":"2023-05-20T01:16:32.774068+0000","last_undegraded":"2023-05-20T01:16:32.774068+0000","last_fullsized":"2023-05-20T01:16:32.774068+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:22:15.807596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.310582+0000","last_change":"2023-05-20T01:16:27.914231+0000","last_active":"2023-05-20T01:16:33.310582+0000","last_peered":"2023-05-20T01:16:33.310582+0000","last_clean":"2023-05-20T01:16:33.310582+0000","last_became_active":"2023-05-20T01:16:27.913946+0000","last_became_peered":"2023-05-20T01:16:27.913946+0000","last_unstale":"2023-05-20T01:16:33.310582+0000","last_undegraded":"2023-05-20T01:16:33.310582+0000","last_fullsized":"2023-05-20T01:16:33.310582+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:58:31.508225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.956705+0000","last_change":"2023-05-20T01:16:28.336767+0000","last_active":"2023-05-20T01:16:32.956705+0000","last_peered":"2023-05-20T01:16:32.956705+0000","last_clean":"2023-05-20T01:16:32.956705+0000","last_became_active":"2023-05-20T01:16:28.336172+0000","last_became_peered":"2023-05-20T01:16:28.336172+0000","last_unstale":"2023-05-20T01:16:32.956705+0000","last_undegraded":"2023-05-20T01:16:32.956705+0000","last_fullsized":"2023-05-20T01:16:32.956705+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:45:17.217540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.957277+0000","last_change":"2023-05-20T01:16:32.364585+0000","last_active":"2023-05-20T01:16:32.957277+0000","last_peered":"2023-05-20T01:16:32.957277+0000","last_clean":"2023-05-20T01:16:32.957277+0000","last_became_active":"2023-05-20T01:16:32.364096+0000","last_became_peered":"2023-05-20T01:16:32.364096+0000","last_unstale":"2023-05-20T01:16:32.957277+0000","last_undegraded":"2023-05-20T01:16:32.957277+0000","last_fullsized":"2023-05-20T01:16:32.957277+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:47:17.109451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,0],"acting":[1,3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"20'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.954518+0000","last_change":"2023-05-20T01:16:27.922900+0000","last_active":"2023-05-20T01:16:32.954518+0000","last_peered":"2023-05-20T01:16:32.954518+0000","last_clean":"2023-05-20T01:16:32.954518+0000","last_became_active":"2023-05-20T01:16:27.922597+0000","last_became_peered":"2023-05-20T01:16:27.922597+0000","last_unstale":"2023-05-20T01:16:32.954518+0000","last_undegraded":"2023-05-20T01:16:32.954518+0000","last_fullsized":"2023-05-20T01:16:32.954518+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:58:35.370081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.310293+0000","last_change":"2023-05-20T01:16:28.341446+0000","last_active":"2023-05-20T01:16:33.310293+0000","last_peered":"2023-05-20T01:16:33.310293+0000","last_clean":"2023-05-20T01:16:33.310293+0000","last_became_active":"2023-05-20T01:16:28.340943+0000","last_became_peered":"2023-05-20T01:16:28.340943+0000","last_unstale":"2023-05-20T01:16:33.310293+0000","last_undegraded":"2023-05-20T01:16:33.310293+0000","last_fullsized":"2023-05-20T01:16:33.310293+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:24:03.794336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.311921+0000","last_change":"2023-05-20T01:16:32.360851+0000","last_active":"2023-05-20T01:16:33.311921+0000","last_peered":"2023-05-20T01:16:33.311921+0000","last_clean":"2023-05-20T01:16:33.311921+0000","last_became_active":"2023-05-20T01:16:32.360366+0000","last_became_peered":"2023-05-20T01:16:32.360366+0000","last_unstale":"2023-05-20T01:16:33.311921+0000","last_undegraded":"2023-05-20T01:16:33.311921+0000","last_fullsized":"2023-05-20T01:16:33.311921+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:38:17.163216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1,6],"acting":[7,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.310447+0000","last_change":"2023-05-20T01:16:27.925398+0000","last_active":"2023-05-20T01:16:33.310447+0000","last_peered":"2023-05-20T01:16:33.310447+0000","last_clean":"2023-05-20T01:16:33.310447+0000","last_became_active":"2023-05-20T01:16:27.924943+0000","last_became_peered":"2023-05-20T01:16:27.924943+0000","last_unstale":"2023-05-20T01:16:33.310447+0000","last_undegraded":"2023-05-20T01:16:33.310447+0000","last_fullsized":"2023-05-20T01:16:33.310447+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:42:58.919423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.773260+0000","last_change":"2023-05-20T01:16:28.332091+0000","last_active":"2023-05-20T01:16:32.773260+0000","last_peered":"2023-05-20T01:16:32.773260+0000","last_clean":"2023-05-20T01:16:32.773260+0000","last_became_active":"2023-05-20T01:16:28.331785+0000","last_became_peered":"2023-05-20T01:16:28.331785+0000","last_unstale":"2023-05-20T01:16:32.773260+0000","last_undegraded":"2023-05-20T01:16:32.773260+0000","last_fullsized":"2023-05-20T01:16:32.773260+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:31:47.073658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.310796+0000","last_change":"2023-05-20T01:16:32.362120+0000","last_active":"2023-05-20T01:16:33.310796+0000","last_peered":"2023-05-20T01:16:33.310796+0000","last_clean":"2023-05-20T01:16:33.310796+0000","last_became_active":"2023-05-20T01:16:32.361727+0000","last_became_peered":"2023-05-20T01:16:32.361727+0000","last_unstale":"2023-05-20T01:16:33.310796+0000","last_undegraded":"2023-05-20T01:16:33.310796+0000","last_fullsized":"2023-05-20T01:16:33.310796+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:01:24.913389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7,4],"acting":[5,0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:33.310449+0000","last_change":"2023-05-20T01:16:27.925288+0000","last_active":"2023-05-20T01:16:33.310449+0000","last_peered":"2023-05-20T01:16:33.310449+0000","last_clean":"2023-05-20T01:16:33.310449+0000","last_became_active":"2023-05-20T01:16:27.924988+0000","last_became_peered":"2023-05-20T01:16:27.924988+0000","last_unstale":"2023-05-20T01:16:33.310449+0000","last_undegraded":"2023-05-20T01:16:33.310449+0000","last_fullsized":"2023-05-20T01:16:33.310449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:32:59.792286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.773973+0000","last_change":"2023-05-20T01:16:28.334255+0000","last_active":"2023-05-20T01:16:32.773973+0000","last_peered":"2023-05-20T01:16:32.773973+0000","last_clean":"2023-05-20T01:16:32.773973+0000","last_became_active":"2023-05-20T01:16:28.333966+0000","last_became_peered":"2023-05-20T01:16:28.333966+0000","last_unstale":"2023-05-20T01:16:32.773973+0000","last_undegraded":"2023-05-20T01:16:32.773973+0000","last_fullsized":"2023-05-20T01:16:32.773973+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:04:05.601168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.772980+0000","last_change":"2023-05-20T01:16:32.346878+0000","last_active":"2023-05-20T01:16:32.772980+0000","last_peered":"2023-05-20T01:16:32.772980+0000","last_clean":"2023-05-20T01:16:32.772980+0000","last_became_active":"2023-05-20T01:16:32.346493+0000","last_became_peered":"2023-05-20T01:16:32.346493+0000","last_unstale":"2023-05-20T01:16:32.772980+0000","last_undegraded":"2023-05-20T01:16:32.772980+0000","last_fullsized":"2023-05-20T01:16:32.772980+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:50:05.163209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,2],"acting":[4,6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"20'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.772568+0000","last_change":"2023-05-20T01:16:27.906889+0000","last_active":"2023-05-20T01:16:32.772568+0000","last_peered":"2023-05-20T01:16:32.772568+0000","last_clean":"2023-05-20T01:16:32.772568+0000","last_became_active":"2023-05-20T01:16:27.906574+0000","last_became_peered":"2023-05-20T01:16:27.906574+0000","last_unstale":"2023-05-20T01:16:32.772568+0000","last_undegraded":"2023-05-20T01:16:32.772568+0000","last_fullsized":"2023-05-20T01:16:32.772568+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:49:41.443268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.771941+0000","last_change":"2023-05-20T01:16:28.345350+0000","last_active":"2023-05-20T01:16:32.771941+0000","last_peered":"2023-05-20T01:16:32.771941+0000","last_clean":"2023-05-20T01:16:32.771941+0000","last_became_active":"2023-05-20T01:16:28.344838+0000","last_became_peered":"2023-05-20T01:16:28.344838+0000","last_unstale":"2023-05-20T01:16:32.771941+0000","last_undegraded":"2023-05-20T01:16:32.771941+0000","last_fullsized":"2023-05-20T01:16:32.771941+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:53:32.204259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.956979+0000","last_change":"2023-05-20T01:16:32.352947+0000","last_active":"2023-05-20T01:16:32.956979+0000","last_peered":"2023-05-20T01:16:32.956979+0000","last_clean":"2023-05-20T01:16:32.956979+0000","last_became_active":"2023-05-20T01:16:32.352495+0000","last_became_peered":"2023-05-20T01:16:32.352495+0000","last_unstale":"2023-05-20T01:16:32.956979+0000","last_undegraded":"2023-05-20T01:16:32.956979+0000","last_fullsized":"2023-05-20T01:16:32.956979+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:31.322067+0000","last_clean_scrub_stamp":"2023-05-20T01:16:31.322067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:29:18.453884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7,2],"acting":[1,5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"20'4","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.954845+0000","last_change":"2023-05-20T01:16:27.917645+0000","last_active":"2023-05-20T01:16:32.954845+0000","last_peered":"2023-05-20T01:16:32.954845+0000","last_clean":"2023-05-20T01:16:32.954845+0000","last_became_active":"2023-05-20T01:16:27.917362+0000","last_became_peered":"2023-05-20T01:16:27.917362+0000","last_unstale":"2023-05-20T01:16:32.954845+0000","last_undegraded":"2023-05-20T01:16:32.954845+0000","last_fullsized":"2023-05-20T01:16:32.954845+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:26.885598+0000","last_clean_scrub_stamp":"2023-05-20T01:16:26.885598+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:54:11.307738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.773261+0000","last_change":"2023-05-20T01:16:28.347617+0000","last_active":"2023-05-20T01:16:32.773261+0000","last_peered":"2023-05-20T01:16:32.773261+0000","last_clean":"2023-05-20T01:16:32.773261+0000","last_became_active":"2023-05-20T01:16:28.347032+0000","last_became_peered":"2023-05-20T01:16:28.347032+0000","last_unstale":"2023-05-20T01:16:32.773261+0000","last_undegraded":"2023-05-20T01:16:32.773261+0000","last_fullsized":"2023-05-20T01:16:32.773261+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:34:36.196004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.773474+0000","last_change":"2023-05-20T01:16:28.336509+0000","last_active":"2023-05-20T01:16:32.773474+0000","last_peered":"2023-05-20T01:16:32.773474+0000","last_clean":"2023-05-20T01:16:32.773474+0000","last_became_active":"2023-05-20T01:16:28.336170+0000","last_became_peered":"2023-05-20T01:16:28.336170+0000","last_unstale":"2023-05-20T01:16:32.773474+0000","last_undegraded":"2023-05-20T01:16:32.773474+0000","last_fullsized":"2023-05-20T01:16:32.773474+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:16:31.755218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.771733+0000","last_change":"2023-05-20T01:16:28.340216+0000","last_active":"2023-05-20T01:16:32.771733+0000","last_peered":"2023-05-20T01:16:32.771733+0000","last_clean":"2023-05-20T01:16:32.771733+0000","last_became_active":"2023-05-20T01:16:28.339662+0000","last_became_peered":"2023-05-20T01:16:28.339662+0000","last_unstale":"2023-05-20T01:16:32.771733+0000","last_undegraded":"2023-05-20T01:16:32.771733+0000","last_fullsized":"2023-05-20T01:16:32.771733+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:54:12.315501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.774267+0000","last_change":"2023-05-20T01:16:28.347840+0000","last_active":"2023-05-20T01:16:32.774267+0000","last_peered":"2023-05-20T01:16:32.774267+0000","last_clean":"2023-05-20T01:16:32.774267+0000","last_became_active":"2023-05-20T01:16:28.347370+0000","last_became_peered":"2023-05-20T01:16:28.347370+0000","last_unstale":"2023-05-20T01:16:32.774267+0000","last_undegraded":"2023-05-20T01:16:32.774267+0000","last_fullsized":"2023-05-20T01:16:32.774267+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T01:49:49.650741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.772038+0000","last_change":"2023-05-20T01:16:28.328611+0000","last_active":"2023-05-20T01:16:32.772038+0000","last_peered":"2023-05-20T01:16:32.772038+0000","last_clean":"2023-05-20T01:16:32.772038+0000","last_became_active":"2023-05-20T01:16:28.328311+0000","last_became_peered":"2023-05-20T01:16:28.328311+0000","last_unstale":"2023-05-20T01:16:32.772038+0000","last_undegraded":"2023-05-20T01:16:32.772038+0000","last_fullsized":"2023-05-20T01:16:32.772038+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:09:39.569369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-05-20T01:16:32.957125+0000","last_change":"2023-05-20T01:16:28.336980+0000","last_active":"2023-05-20T01:16:32.957125+0000","last_peered":"2023-05-20T01:16:32.957125+0000","last_clean":"2023-05-20T01:16:32.957125+0000","last_became_active":"2023-05-20T01:16:28.336778+0000","last_became_peered":"2023-05-20T01:16:28.336778+0000","last_unstale":"2023-05-20T01:16:32.957125+0000","last_undegraded":"2023-05-20T01:16:32.957125+0000","last_fullsized":"2023-05-20T01:16:32.957125+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:27.893361+0000","last_clean_scrub_stamp":"2023-05-20T01:16:27.893361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:03:20.405510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":4592,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574858,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9260,"kb_used_data":868,"kb_used_omap":0,"kb_used_meta":8384,"kb_avail":94362580,"statfs":{"total":96636764160,"available":96627281920,"internally_reserved":0,"allocated":888832,"data_stored":703779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8585216},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53800000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52500000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35399999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47699999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574857,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8076,"kb_used_data":260,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363764,"statfs":{"total":96636764160,"available":96628494336,"internally_reserved":0,"allocated":266240,"data_stored":109401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54200000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.43099999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61299999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59899999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58599999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70499999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574857,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9232,"kb_used_data":840,"kb_used_omap":0,"kb_used_meta":8384,"kb_avail":94362608,"statfs":{"total":96636764160,"available":96627310592,"internally_reserved":0,"allocated":860160,"data_stored":699769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8585216},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34100000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34999999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54200000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53000000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574857,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8140,"kb_used_data":260,"kb_used_omap":0,"kb_used_meta":7872,"kb_avail":94363700,"statfs":{"total":96636764160,"available":96628428800,"internally_reserved":0,"allocated":266240,"data_stored":109022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8060928},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.25900000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28999999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34799999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55800000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53600000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60799999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52500000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574857,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8076,"kb_used_data":260,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363764,"statfs":{"total":96636764160,"available":96628494336,"internally_reserved":0,"allocated":266240,"data_stored":109401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.245}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30499999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.376}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55800000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64700000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59399999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65900000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8144,"kb_used_data":264,"kb_used_omap":0,"kb_used_meta":7872,"kb_avail":94363696,"statfs":{"total":96636764160,"available":96628424704,"internally_reserved":0,"allocated":270336,"data_stored":109491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8060928},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.317}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.4260000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1619999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0700000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.5379999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.347}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1359999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8224,"kb_used_data":280,"kb_used_omap":0,"kb_used_meta":7936,"kb_avail":94363616,"statfs":{"total":96636764160,"available":96628342784,"internally_reserved":0,"allocated":286720,"data_stored":113251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8126464},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82799999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58599999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36199999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38900000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40899999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574858,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8144,"kb_used_data":264,"kb_used_omap":0,"kb_used_meta":7872,"kb_avail":94363696,"statfs":{"total":96636764160,"available":96628424704,"internally_reserved":0,"allocated":270336,"data_stored":109092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8060928},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":6.3929999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.2450000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1639999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":6.351}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46300000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52500000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-05-20T01:16:51.684 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-05-20T01:16:51.684 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-05-20T01:16:51.684 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-05-20T01:16:51.685 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-05-20T01:16:51.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.837+0000 7f91d9a01700 1 Processor -- start 2023-05-20T01:16:51.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.837+0000 7f91d9a01700 1 -- start start 2023-05-20T01:16:51.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.838+0000 7f91d9a01700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f91d40cfc50 0x7f91d40d2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:51.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.838+0000 7f91d9a01700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f91d40b4750 0x7f91d40d0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:51.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.838+0000 7f91d9a01700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91d40d0b90 con 0x7f91d40b2b40 2023-05-20T01:16:51.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.838+0000 7f91d9a01700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91d40cfad0 con 0x7f91d40b4750 2023-05-20T01:16:51.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.838+0000 7f91d9a01700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91d40d3df0 con 0x7f91d40cfc50 2023-05-20T01:16:51.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.838+0000 7f91d27fc700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f91d40b4750 0x7f91d40d0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:51.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.838+0000 7f91d37fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f91d40b2b40 0x7f91d40b2f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:41736/0 (socket says 172.21.15.16:41736) 2023-05-20T01:16:51.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.838+0000 7f91d37fe700 1 -- 172.21.15.16:0/3711473367 learned_addr learned my addr 172.21.15.16:0/3711473367 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:16:51.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.838+0000 7f91d2ffd700 1 --2- 172.21.15.16:0/3711473367 >> v2:172.21.15.132:3300/0 conn(0x7f91d40cfc50 0x7f91d40d2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:51.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.838+0000 7f91d27fc700 1 -- 172.21.15.16:0/3711473367 >> v2:172.21.15.132:3300/0 conn(0x7f91d40cfc50 msgr2=0x7f91d40d2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:51.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.838+0000 7f91d27fc700 1 --2- 172.21.15.16:0/3711473367 >> v2:172.21.15.132:3300/0 conn(0x7f91d40cfc50 0x7f91d40d2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:51.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.838+0000 7f91d27fc700 1 -- 172.21.15.16:0/3711473367 >> v1:172.21.15.16:6789/0 conn(0x7f91d40b2b40 legacy=0x7f91d40b2f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:51.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.838+0000 7f91d1ffb700 1 -- 172.21.15.16:0/3711473367 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2548180584 0 0) 0x7f91d40d0b90 con 0x7f91d40b2b40 2023-05-20T01:16:51.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.838+0000 7f91d27fc700 1 -- 172.21.15.16:0/3711473367 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91d40d0560 con 0x7f91d40b4750 2023-05-20T01:16:51.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.839+0000 7f91d27fc700 1 --2- 172.21.15.16:0/3711473367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91d40b4750 0x7f91d40d0020 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f91d40b5de0 tx=0x7f91c4009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d5d67d3a9239a2f server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:51.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.839+0000 7f91d1ffb700 1 -- 172.21.15.16:0/3711473367 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c4011830 con 0x7f91d40b4750 2023-05-20T01:16:51.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.839+0000 7f91d1ffb700 1 -- 172.21.15.16:0/3711473367 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91c4011990 con 0x7f91d40b4750 2023-05-20T01:16:51.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.839+0000 7f91d1ffb700 1 -- 172.21.15.16:0/3711473367 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c4011d20 con 0x7f91d40b4750 2023-05-20T01:16:51.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.839+0000 7f91d9a01700 1 -- 172.21.15.16:0/3711473367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91d40b4750 msgr2=0x7f91d40d0020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:51.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.839+0000 7f91d9a01700 1 --2- 172.21.15.16:0/3711473367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91d40b4750 0x7f91d40d0020 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f91d40b5de0 tx=0x7f91c4009b20 comp rx=0 tx=0).stop 2023-05-20T01:16:51.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.839+0000 7f91d9a01700 1 -- 172.21.15.16:0/3711473367 shutdown_connections 2023-05-20T01:16:51.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.839+0000 7f91d9a01700 1 --2- 172.21.15.16:0/3711473367 >> v2:172.21.15.132:3300/0 conn(0x7f91d40cfc50 0x7f91d40d2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:51.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.839+0000 7f91d9a01700 1 --2- 172.21.15.16:0/3711473367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91d40b4750 0x7f91d40d0020 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:51.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.839+0000 7f91d9a01700 1 -- 172.21.15.16:0/3711473367 >> 172.21.15.16:0/3711473367 conn(0x7f91d409c590 msgr2=0x7f91d40d3240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:51.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.839+0000 7f91d9a01700 1 -- 172.21.15.16:0/3711473367 shutdown_connections 2023-05-20T01:16:51.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.839+0000 7f91d9a01700 1 -- 172.21.15.16:0/3711473367 wait complete. 2023-05-20T01:16:51.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.841+0000 7f91d9a01700 1 Processor -- start 2023-05-20T01:16:51.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.841+0000 7f91d9a01700 1 -- start start 2023-05-20T01:16:51.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.842+0000 7f91d9a01700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91d40b2b40 0x7f91d415ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:51.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.842+0000 7f91d9a01700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f91d40b4750 0x7f91d40b6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:51.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.842+0000 7f91d9a01700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f91d40cfc50 0x7f91d40cdd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:51.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.842+0000 7f91d9a01700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91d40c8390 con 0x7f91d40b2b40 2023-05-20T01:16:51.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.842+0000 7f91d9a01700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91d41f6c00 con 0x7f91d40cfc50 2023-05-20T01:16:51.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.842+0000 7f91d9a01700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91d41f5e40 con 0x7f91d40b4750 2023-05-20T01:16:51.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.842+0000 7f91d2ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91d40b2b40 0x7f91d415ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:51.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.843+0000 7f91d2ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91d40b2b40 0x7f91d415ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58442/0 (socket says 172.21.15.16:58442) 2023-05-20T01:16:51.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.843+0000 7f91d2ffd700 1 -- 172.21.15.16:0/2626492636 learned_addr learned my addr 172.21.15.16:0/2626492636 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:51.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.843+0000 7f91d37fe700 1 --2- 172.21.15.16:0/2626492636 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f91d40cfc50 0x7f91d40cdd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:51.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.843+0000 7f91d27fc700 1 --2- 172.21.15.16:0/2626492636 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f91d40b4750 0x7f91d40b6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:51.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.843+0000 7f91d2ffd700 1 -- 172.21.15.16:0/2626492636 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f91d40b4750 msgr2=0x7f91d40b6fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:51.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.843+0000 7f91d2ffd700 1 --2- 172.21.15.16:0/2626492636 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f91d40b4750 0x7f91d40b6fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:51.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.843+0000 7f91d2ffd700 1 -- 172.21.15.16:0/2626492636 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f91d40cfc50 msgr2=0x7f91d40cdd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:51.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.843+0000 7f91d2ffd700 1 --2- 172.21.15.16:0/2626492636 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f91d40cfc50 0x7f91d40cdd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:51.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.843+0000 7f91d2ffd700 1 -- 172.21.15.16:0/2626492636 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91c40094a0 con 0x7f91d40b2b40 2023-05-20T01:16:51.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.843+0000 7f91d2ffd700 1 --2- 172.21.15.16:0/2626492636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91d40b2b40 0x7f91d415ada0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f91c800e870 tx=0x7f91c800ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:51.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.843+0000 7f91bbfff700 1 -- 172.21.15.16:0/2626492636 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c8009590 con 0x7f91d40b2b40 2023-05-20T01:16:51.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.843+0000 7f91bbfff700 1 -- 172.21.15.16:0/2626492636 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91c800ee70 con 0x7f91d40b2b40 2023-05-20T01:16:51.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.843+0000 7f91d9a01700 1 -- 172.21.15.16:0/2626492636 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91d40b4380 con 0x7f91d40b2b40 2023-05-20T01:16:51.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.843+0000 7f91bbfff700 1 -- 172.21.15.16:0/2626492636 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c800ca50 con 0x7f91d40b2b40 2023-05-20T01:16:51.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.843+0000 7f91d9a01700 1 -- 172.21.15.16:0/2626492636 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91d415d0b0 con 0x7f91d40b2b40 2023-05-20T01:16:51.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.844+0000 7f91d9a01700 1 -- 172.21.15.16:0/2626492636 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91d40d4000 con 0x7f91d40b2b40 2023-05-20T01:16:51.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.845+0000 7f91bbfff700 1 -- 172.21.15.16:0/2626492636 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f91c800cbb0 con 0x7f91d40b2b40 2023-05-20T01:16:51.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.845+0000 7f91bbfff700 1 --2- 172.21.15.16:0/2626492636 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f91c006ace0 0x7f91c006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:51.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.845+0000 7f91d27fc700 1 --2- 172.21.15.16:0/2626492636 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f91c006ace0 0x7f91c006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:51.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.845+0000 7f91bbfff700 1 -- 172.21.15.16:0/2626492636 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f91c80896b0 con 0x7f91d40b2b40 2023-05-20T01:16:51.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.846+0000 7f91d27fc700 1 --2- 172.21.15.16:0/2626492636 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f91c006ace0 0x7f91c006d190 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f91c40090b0 tx=0x7f91c401c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:51.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:51.850+0000 7f91bbfff700 1 -- 172.21.15.16:0/2626492636 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f91c8057f90 con 0x7f91d40b2b40 2023-05-20T01:16:52.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.096+0000 7f91d9a01700 1 -- 172.21.15.16:0/2626492636 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f91d404e8c0 con 0x7f91d40b2b40 2023-05-20T01:16:52.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.096+0000 7f91bbfff700 1 -- 172.21.15.16:0/2626492636 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f91c800ce60 con 0x7f91d40b2b40 2023-05-20T01:16:52.100 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:16:52.100 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-05-20T01:16:52.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.098+0000 7f91d9a01700 1 -- 172.21.15.16:0/2626492636 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f91c006ace0 msgr2=0x7f91c006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:52.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.098+0000 7f91d9a01700 1 --2- 172.21.15.16:0/2626492636 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f91c006ace0 0x7f91c006d190 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f91c40090b0 tx=0x7f91c401c040 comp rx=0 tx=0).stop 2023-05-20T01:16:52.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.098+0000 7f91d9a01700 1 -- 172.21.15.16:0/2626492636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91d40b2b40 msgr2=0x7f91d415ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:52.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.098+0000 7f91d9a01700 1 --2- 172.21.15.16:0/2626492636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91d40b2b40 0x7f91d415ada0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f91c800e870 tx=0x7f91c800ec30 comp rx=0 tx=0).stop 2023-05-20T01:16:52.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.098+0000 7f91d9a01700 1 -- 172.21.15.16:0/2626492636 shutdown_connections 2023-05-20T01:16:52.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.098+0000 7f91d9a01700 1 --2- 172.21.15.16:0/2626492636 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f91d40cfc50 0x7f91d40cdd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:52.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.098+0000 7f91d9a01700 1 --2- 172.21.15.16:0/2626492636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91d40b2b40 0x7f91d415ada0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:52.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.098+0000 7f91d9a01700 1 --2- 172.21.15.16:0/2626492636 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f91c006ace0 0x7f91c006d190 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:52.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.098+0000 7f91d9a01700 1 --2- 172.21.15.16:0/2626492636 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f91d40b4750 0x7f91d40b6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:52.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.098+0000 7f91d9a01700 1 -- 172.21.15.16:0/2626492636 >> 172.21.15.16:0/2626492636 conn(0x7f91d409c590 msgr2=0x7f91d406c060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:52.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.098+0000 7f91d9a01700 1 -- 172.21.15.16:0/2626492636 shutdown_connections 2023-05-20T01:16:52.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.098+0000 7f91d9a01700 1 -- 172.21.15.16:0/2626492636 wait complete. 2023-05-20T01:16:52.114 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-05-20T01:16:52.114 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-20T01:16:52.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.235+0000 7f2c00de0700 1 Processor -- start 2023-05-20T01:16:52.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.235+0000 7f2c00de0700 1 -- start start 2023-05-20T01:16:52.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.236+0000 7f2c00de0700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f2bfc06f450 0x7f2bfc0cf890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:52.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.236+0000 7f2c00de0700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bfc0cc510 con 0x7f2bfc06a510 2023-05-20T01:16:52.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.236+0000 7f2c00de0700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bfc071ed0 con 0x7f2bfc06d4b0 2023-05-20T01:16:52.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.236+0000 7f2c00de0700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bfc0721b0 con 0x7f2bfc06f450 2023-05-20T01:16:52.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.236+0000 7f2bfad9d700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f2bfc06a510 0x7f2bfc0cc9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:41750/0 (socket says 172.21.15.16:41750) 2023-05-20T01:16:52.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.236+0000 7f2bfad9d700 1 -- 172.21.15.16:0/2169096103 learned_addr learned my addr 172.21.15.16:0/2169096103 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:16:52.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.236+0000 7f2bfa59c700 1 --2- 172.21.15.16:0/2169096103 >> v2:172.21.15.132:3300/0 conn(0x7f2bfc06f450 0x7f2bfc0cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:52.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.236+0000 7f2bf959a700 1 -- 172.21.15.16:0/2169096103 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 179264494 0 0) 0x7f2bfc0cc510 con 0x7f2bfc06a510 2023-05-20T01:16:52.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.236+0000 7f2bf959a700 1 -- 172.21.15.16:0/2169096103 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2be40035b0 con 0x7f2bfc06a510 2023-05-20T01:16:52.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.236+0000 7f2bf959a700 1 -- 172.21.15.16:0/2169096103 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3841786719 0 0) 0x7f2be40035b0 con 0x7f2bfc06a510 2023-05-20T01:16:52.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.236+0000 7f2bf959a700 1 -- 172.21.15.16:0/2169096103 >> v2:172.21.15.132:3300/0 conn(0x7f2bfc06f450 msgr2=0x7f2bfc0cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:52.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.236+0000 7f2bf959a700 1 --2- 172.21.15.16:0/2169096103 >> v2:172.21.15.132:3300/0 conn(0x7f2bfc06f450 0x7f2bfc0cf890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:52.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.236+0000 7f2bf959a700 1 -- 172.21.15.16:0/2169096103 >> v1:172.21.15.132:6789/0 conn(0x7f2bfc06d4b0 legacy=0x7f2bfc161080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:52.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.236+0000 7f2bf959a700 1 -- 172.21.15.16:0/2169096103 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bfc0cc1a0 con 0x7f2bfc06a510 2023-05-20T01:16:52.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.236+0000 7f2bf959a700 1 -- 172.21.15.16:0/2169096103 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7f2bf40019f0 con 0x7f2bfc06a510 2023-05-20T01:16:52.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.236+0000 7f2bf959a700 1 -- 172.21.15.16:0/2169096103 >> v1:172.21.15.16:6789/0 conn(0x7f2bfc06a510 legacy=0x7f2bfc0cc9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:52.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.237+0000 7f2bf959a700 1 --2- 172.21.15.16:0/2169096103 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2be4003a70 0x7f2be4003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:52.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.237+0000 7f2bf959a700 1 --2- 172.21.15.16:0/2169096103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2be4004850 0x7f2be4004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:52.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.237+0000 7f2bf959a700 1 --2- 172.21.15.16:0/2169096103 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2be4005250 0x7f2be400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:52.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.237+0000 7f2bf959a700 1 -- 172.21.15.16:0/2169096103 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bfc0721b0 con 0x7f2be4004850 2023-05-20T01:16:52.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.237+0000 7f2bf959a700 1 -- 172.21.15.16:0/2169096103 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2be400c190 con 0x7f2be4003a70 2023-05-20T01:16:52.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.237+0000 7f2bf959a700 1 -- 172.21.15.16:0/2169096103 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2be400c350 con 0x7f2be4005250 2023-05-20T01:16:52.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.237+0000 7f2bf9d9b700 1 --2- 172.21.15.16:0/2169096103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2be4004850 0x7f2be4004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:52.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.237+0000 7f2bfa59c700 1 --2- 172.21.15.16:0/2169096103 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2be4003a70 0x7f2be4003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:52.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.237+0000 7f2bfad9d700 1 --2- 172.21.15.16:0/2169096103 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2be4005250 0x7f2be400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:52.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.237+0000 7f2bf9d9b700 1 -- 172.21.15.16:0/2169096103 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2be4005250 msgr2=0x7f2be400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:52.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.237+0000 7f2bf9d9b700 1 --2- 172.21.15.16:0/2169096103 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2be4005250 0x7f2be400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:52.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.237+0000 7f2bf9d9b700 1 -- 172.21.15.16:0/2169096103 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2be4003a70 msgr2=0x7f2be4003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:52.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.237+0000 7f2bf9d9b700 1 --2- 172.21.15.16:0/2169096103 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2be4003a70 0x7f2be4003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:52.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.237+0000 7f2bfad9d700 1 -- 172.21.15.16:0/2169096103 reap_dead start 2023-05-20T01:16:52.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.237+0000 7f2bf9d9b700 1 -- 172.21.15.16:0/2169096103 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2be40035b0 con 0x7f2be4004850 2023-05-20T01:16:52.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.237+0000 7f2bf9d9b700 1 --2- 172.21.15.16:0/2169096103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2be4004850 0x7f2be4004cc0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f2bf0007fd0 tx=0x7f2be4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3b7d86066e16777 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:52.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.237+0000 7f2bf959a700 1 -- 172.21.15.16:0/2169096103 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bf00052d0 con 0x7f2be4004850 2023-05-20T01:16:52.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.237+0000 7f2bf959a700 1 -- 172.21.15.16:0/2169096103 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bf0005430 con 0x7f2be4004850 2023-05-20T01:16:52.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.237+0000 7f2bf959a700 1 -- 172.21.15.16:0/2169096103 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bf00038c0 con 0x7f2be4004850 2023-05-20T01:16:52.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.238+0000 7f2c00de0700 1 -- 172.21.15.16:0/2169096103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2be4004850 msgr2=0x7f2be4004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:52.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.238+0000 7f2c00de0700 1 --2- 172.21.15.16:0/2169096103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2be4004850 0x7f2be4004cc0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f2bf0007fd0 tx=0x7f2be4005880 comp rx=0 tx=0).stop 2023-05-20T01:16:52.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.238+0000 7f2c00de0700 1 -- 172.21.15.16:0/2169096103 shutdown_connections 2023-05-20T01:16:52.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.238+0000 7f2c00de0700 1 --2- 172.21.15.16:0/2169096103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2be4004850 0x7f2be4004cc0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:52.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.238+0000 7f2c00de0700 1 -- 172.21.15.16:0/2169096103 >> 172.21.15.16:0/2169096103 conn(0x7f2bfc0cffb0 msgr2=0x7f2bfc0d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:52.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.238+0000 7f2c00de0700 1 -- 172.21.15.16:0/2169096103 shutdown_connections 2023-05-20T01:16:52.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.238+0000 7f2c00de0700 1 -- 172.21.15.16:0/2169096103 wait complete. 2023-05-20T01:16:52.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.239+0000 7f2c00de0700 1 Processor -- start 2023-05-20T01:16:52.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.239+0000 7f2c00de0700 1 -- start start 2023-05-20T01:16:52.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.240+0000 7f2c00de0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2be4004850 0x7f2bfc155410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:52.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.240+0000 7f2c00de0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bfc0ad7f0 0x7f2bfc0adc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:52.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.240+0000 7f2c00de0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2bfc0cc7f0 0x7f2bfc0ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:52.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.240+0000 7f2c00de0700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bfc0af7e0 con 0x7f2bfc0ad7f0 2023-05-20T01:16:52.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.240+0000 7f2c00de0700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bfc0cd1d0 con 0x7f2bfc0cc7f0 2023-05-20T01:16:52.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.240+0000 7f2c00de0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bfc0b50a0 con 0x7f2be4004850 2023-05-20T01:16:52.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.240+0000 7f2bf9d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bfc0ad7f0 0x7f2bfc0adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:52.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.240+0000 7f2bf9d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bfc0ad7f0 0x7f2bfc0adc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58464/0 (socket says 172.21.15.16:58464) 2023-05-20T01:16:52.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.240+0000 7f2bf9d9b700 1 -- 172.21.15.16:0/1004265228 learned_addr learned my addr 172.21.15.16:0/1004265228 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:52.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.240+0000 7f2bfa59c700 1 --2- 172.21.15.16:0/1004265228 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2be4004850 0x7f2bfc155410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:52.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.240+0000 7f2bfad9d700 1 --2- 172.21.15.16:0/1004265228 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2bfc0cc7f0 0x7f2bfc0ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:52.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.240+0000 7f2bf9d9b700 1 -- 172.21.15.16:0/1004265228 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2be4004850 msgr2=0x7f2bfc155410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:52.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.240+0000 7f2bf9d9b700 1 --2- 172.21.15.16:0/1004265228 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2be4004850 0x7f2bfc155410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:52.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.240+0000 7f2bf9d9b700 1 -- 172.21.15.16:0/1004265228 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2bfc0cc7f0 msgr2=0x7f2bfc0ccc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:52.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.240+0000 7f2bf9d9b700 1 --2- 172.21.15.16:0/1004265228 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2bfc0cc7f0 0x7f2bfc0ccc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:52.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.241+0000 7f2bf9d9b700 1 -- 172.21.15.16:0/1004265228 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bfc0cf890 con 0x7f2bfc0ad7f0 2023-05-20T01:16:52.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.241+0000 7f2bf9d9b700 1 --2- 172.21.15.16:0/1004265228 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bfc0ad7f0 0x7f2bfc0adc60 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f2bf0015040 tx=0x7f2bf0001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:52.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.241+0000 7f2beaffd700 1 -- 172.21.15.16:0/1004265228 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bf0001290 con 0x7f2bfc0ad7f0 2023-05-20T01:16:52.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.241+0000 7f2beaffd700 1 -- 172.21.15.16:0/1004265228 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bf001d480 con 0x7f2bfc0ad7f0 2023-05-20T01:16:52.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.241+0000 7f2c00de0700 1 -- 172.21.15.16:0/1004265228 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bfc0cbe50 con 0x7f2bfc0ad7f0 2023-05-20T01:16:52.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.241+0000 7f2beaffd700 1 -- 172.21.15.16:0/1004265228 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bf001d830 con 0x7f2bfc0ad7f0 2023-05-20T01:16:52.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.241+0000 7f2c00de0700 1 -- 172.21.15.16:0/1004265228 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bfc0cfa20 con 0x7f2bfc0ad7f0 2023-05-20T01:16:52.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.242+0000 7f2c00de0700 1 -- 172.21.15.16:0/1004265228 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bfc0747b0 con 0x7f2bfc0ad7f0 2023-05-20T01:16:52.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.242+0000 7f2beaffd700 1 -- 172.21.15.16:0/1004265228 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2bf0019040 con 0x7f2bfc0ad7f0 2023-05-20T01:16:52.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.243+0000 7f2beaffd700 1 --2- 172.21.15.16:0/1004265228 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f2bdc06ace0 0x7f2bdc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:52.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.243+0000 7f2beaffd700 1 -- 172.21.15.16:0/1004265228 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f2bf008a0d0 con 0x7f2bfc0ad7f0 2023-05-20T01:16:52.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.243+0000 7f2bfa59c700 1 --2- 172.21.15.16:0/1004265228 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f2bdc06ace0 0x7f2bdc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:52.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.243+0000 7f2bfa59c700 1 --2- 172.21.15.16:0/1004265228 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f2bdc06ace0 0x7f2bdc06d190 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc050070 tx=0x7f2bec009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:52.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.247+0000 7f2beaffd700 1 -- 172.21.15.16:0/1004265228 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f2bf0054fe0 con 0x7f2bfc0ad7f0 2023-05-20T01:16:52.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.448+0000 7f2c00de0700 1 -- 172.21.15.16:0/1004265228 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2bfc0730d0 con 0x7f2bfc0ad7f0 2023-05-20T01:16:52.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.449+0000 7f2beaffd700 1 -- 172.21.15.16:0/1004265228 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+20026 (secure 0 0 0) 0x7f2bf0058600 con 0x7f2bfc0ad7f0 2023-05-20T01:16:52.453 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:16:52.454 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"fsid":"c2245cc5-ecab-4087-a59b-bc81db901676","created":"2023-05-20T01:16:09.549230+0000","modified":"2023-05-20T01:16:32.768495+0000","last_up_change":"2023-05-20T01:16:19.788624+0000","last_in_change":"2023-05-20T01:16:12.092160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-05-20T01:16:20.934366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-05-20T01:16:21.372294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-05-20T01:16:26.208287+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-05-20T01:16:27.292268+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-05-20T01:16:30.357285+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"317a83cd-19c8-4958-a301-6087ef342289","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6825","nonce":3923634179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6827","nonce":3923634179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6831","nonce":3923634179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6829","nonce":3923634179}]},"public_addr":"172.21.15.16:6825/3923634179","cluster_addr":"172.21.15.16:6827/3923634179","heartbeat_back_addr":"172.21.15.16:6831/3923634179","heartbeat_front_addr":"172.21.15.16:6829/3923634179","state":["exists","up"]},{"osd":1,"uuid":"5581046f-8219-4089-8d7d-b7846399505e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6809","nonce":3860264220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6811","nonce":3860264220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6815","nonce":3860264220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6813","nonce":3860264220}]},"public_addr":"172.21.15.16:6809/3860264220","cluster_addr":"172.21.15.16:6811/3860264220","heartbeat_back_addr":"172.21.15.16:6815/3860264220","heartbeat_front_addr":"172.21.15.16:6813/3860264220","state":["exists","up"]},{"osd":2,"uuid":"072e9e66-4133-478e-8d4a-64d46c2963f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6817","nonce":293187030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6819","nonce":293187030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6823","nonce":293187030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6821","nonce":293187030}]},"public_addr":"172.21.15.16:6817/293187030","cluster_addr":"172.21.15.16:6819/293187030","heartbeat_back_addr":"172.21.15.16:6823/293187030","heartbeat_front_addr":"172.21.15.16:6821/293187030","state":["exists","up"]},{"osd":3,"uuid":"11e1bba6-00f8-48dd-9028-d8d716b7f145","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6801","nonce":3669781890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6803","nonce":3669781890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6807","nonce":3669781890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6805","nonce":3669781890}]},"public_addr":"172.21.15.16:6801/3669781890","cluster_addr":"172.21.15.16:6803/3669781890","heartbeat_back_addr":"172.21.15.16:6807/3669781890","heartbeat_front_addr":"172.21.15.16:6805/3669781890","state":["exists","up"]},{"osd":4,"uuid":"b8909c3a-cf70-46cb-8d0b-009c707cf9d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6819","nonce":506317223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6823","nonce":506317223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6829","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6831","nonce":506317223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6827","nonce":506317223}]},"public_addr":"172.21.15.132:6819/506317223","cluster_addr":"172.21.15.132:6823/506317223","heartbeat_back_addr":"172.21.15.132:6831/506317223","heartbeat_front_addr":"172.21.15.132:6827/506317223","state":["exists","up"]},{"osd":5,"uuid":"b1818304-4583-47bf-a6ec-0bfbb5166233","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6818","nonce":1844689995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6822","nonce":1844689995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6830","nonce":1844689995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6826","nonce":1844689995}]},"public_addr":"172.21.15.132:6818/1844689995","cluster_addr":"172.21.15.132:6822/1844689995","heartbeat_back_addr":"172.21.15.132:6830/1844689995","heartbeat_front_addr":"172.21.15.132:6826/1844689995","state":["exists","up"]},{"osd":6,"uuid":"d37f4691-eed1-4a93-8818-18354f03f4a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6801","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6802","nonce":727581123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6805","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6807","nonce":727581123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6813","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6815","nonce":727581123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6811","nonce":727581123}]},"public_addr":"172.21.15.132:6802/727581123","cluster_addr":"172.21.15.132:6807/727581123","heartbeat_back_addr":"172.21.15.132:6815/727581123","heartbeat_front_addr":"172.21.15.132:6811/727581123","state":["exists","up"]},{"osd":7,"uuid":"4f712f62-5d1d-4816-8042-03209144f8b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6803","nonce":466001149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6806","nonce":466001149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6814","nonce":466001149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6810","nonce":466001149}]},"public_addr":"172.21.15.132:6803/466001149","cluster_addr":"172.21.15.132:6806/466001149","heartbeat_back_addr":"172.21.15.132:6814/466001149","heartbeat_front_addr":"172.21.15.132:6810/466001149","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.338863+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.278680+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305694+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.246835+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.276237+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.304441+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305421+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.267050+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-20T01:16:52.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.451+0000 7f2c00de0700 1 -- 172.21.15.16:0/1004265228 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f2bdc06ace0 msgr2=0x7f2bdc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:52.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.451+0000 7f2c00de0700 1 --2- 172.21.15.16:0/1004265228 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f2bdc06ace0 0x7f2bdc06d190 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc050070 tx=0x7f2bec009210 comp rx=0 tx=0).stop 2023-05-20T01:16:52.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.451+0000 7f2c00de0700 1 -- 172.21.15.16:0/1004265228 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bfc0ad7f0 msgr2=0x7f2bfc0adc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:52.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.451+0000 7f2c00de0700 1 --2- 172.21.15.16:0/1004265228 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bfc0ad7f0 0x7f2bfc0adc60 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f2bf0015040 tx=0x7f2bf0001120 comp rx=0 tx=0).stop 2023-05-20T01:16:52.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.451+0000 7f2c00de0700 1 -- 172.21.15.16:0/1004265228 shutdown_connections 2023-05-20T01:16:52.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.451+0000 7f2c00de0700 1 --2- 172.21.15.16:0/1004265228 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2bfc0cc7f0 0x7f2bfc0ccc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:52.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.451+0000 7f2c00de0700 1 --2- 172.21.15.16:0/1004265228 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f2bdc06ace0 0x7f2bdc06d190 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:52.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.451+0000 7f2c00de0700 1 --2- 172.21.15.16:0/1004265228 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2be4004850 0x7f2bfc155410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:52.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.451+0000 7f2c00de0700 1 --2- 172.21.15.16:0/1004265228 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bfc0ad7f0 0x7f2bfc0adc60 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:52.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.452+0000 7f2c00de0700 1 -- 172.21.15.16:0/1004265228 >> 172.21.15.16:0/1004265228 conn(0x7f2bfc0cffb0 msgr2=0x7f2bfc0c9570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:52.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.452+0000 7f2c00de0700 1 -- 172.21.15.16:0/1004265228 shutdown_connections 2023-05-20T01:16:52.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.452+0000 7f2c00de0700 1 -- 172.21.15.16:0/1004265228 wait complete. 2023-05-20T01:16:52.467 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-05-20T01:16:20.934366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-05-20T01:16:21.372294+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-05-20T01:16:26.208287+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-05-20T01:16:27.292268+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-05-20T01:16:30.357285+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-05-20T01:16:52.469 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-05-20T01:16:52.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.592+0000 7ff05544d700 1 Processor -- start 2023-05-20T01:16:52.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.592+0000 7ff05544d700 1 -- start start 2023-05-20T01:16:52.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.593+0000 7ff05544d700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff0500cd890 0x7ff0500b6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:52.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.593+0000 7ff05544d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff0500b65d0 0x7ff0500cf570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:52.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.593+0000 7ff05544d700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0500d1820 con 0x7ff05015ca90 2023-05-20T01:16:52.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.593+0000 7ff05544d700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0500d0c50 con 0x7ff0500b65d0 2023-05-20T01:16:52.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.593+0000 7ff05544d700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0500cdce0 con 0x7ff0500cd890 2023-05-20T01:16:52.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.593+0000 7ff04f7fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff0500b65d0 0x7ff0500cf570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:52.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.593+0000 7ff04effd700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ff05015ca90 0x7ff0500d2550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:41766/0 (socket says 172.21.15.16:41766) 2023-05-20T01:16:52.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.593+0000 7ff04effd700 1 -- 172.21.15.16:0/2742825804 learned_addr learned my addr 172.21.15.16:0/2742825804 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:16:52.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.593+0000 7ff04f7fe700 1 -- 172.21.15.16:0/2742825804 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff0500cd890 msgr2=0x7ff0500b6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:52.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.593+0000 7ff04f7fe700 1 --2- 172.21.15.16:0/2742825804 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff0500cd890 0x7ff0500b6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:52.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.593+0000 7ff04f7fe700 1 -- 172.21.15.16:0/2742825804 >> v1:172.21.15.16:6789/0 conn(0x7ff05015ca90 legacy=0x7ff0500d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:52.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.593+0000 7ff04dffb700 1 -- 172.21.15.16:0/2742825804 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3056201389 0 0) 0x7ff0500d1820 con 0x7ff05015ca90 2023-05-20T01:16:52.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.593+0000 7ff04f7fe700 1 -- 172.21.15.16:0/2742825804 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0500d2c60 con 0x7ff0500b65d0 2023-05-20T01:16:52.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.594+0000 7ff04f7fe700 1 --2- 172.21.15.16:0/2742825804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0500b65d0 0x7ff0500cf570 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff048009d90 tx=0x7ff04800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7766ac590de11f72 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:52.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.594+0000 7ff04dffb700 1 -- 172.21.15.16:0/2742825804 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff04800e070 con 0x7ff0500b65d0 2023-05-20T01:16:52.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.594+0000 7ff04dffb700 1 -- 172.21.15.16:0/2742825804 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff04800bbd0 con 0x7ff0500b65d0 2023-05-20T01:16:52.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.594+0000 7ff04dffb700 1 -- 172.21.15.16:0/2742825804 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff048012430 con 0x7ff0500b65d0 2023-05-20T01:16:52.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.594+0000 7ff05544d700 1 -- 172.21.15.16:0/2742825804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0500b65d0 msgr2=0x7ff0500cf570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:52.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.594+0000 7ff05544d700 1 --2- 172.21.15.16:0/2742825804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0500b65d0 0x7ff0500cf570 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff048009d90 tx=0x7ff04800b330 comp rx=0 tx=0).stop 2023-05-20T01:16:52.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.594+0000 7ff05544d700 1 -- 172.21.15.16:0/2742825804 shutdown_connections 2023-05-20T01:16:52.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.594+0000 7ff05544d700 1 --2- 172.21.15.16:0/2742825804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0500b65d0 0x7ff0500cf570 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:52.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.594+0000 7ff05544d700 1 --2- 172.21.15.16:0/2742825804 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff0500cd890 0x7ff0500b6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:52.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.594+0000 7ff05544d700 1 -- 172.21.15.16:0/2742825804 >> 172.21.15.16:0/2742825804 conn(0x7ff05006b000 msgr2=0x7ff05006b400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:52.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.594+0000 7ff05544d700 1 -- 172.21.15.16:0/2742825804 shutdown_connections 2023-05-20T01:16:52.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.594+0000 7ff05544d700 1 -- 172.21.15.16:0/2742825804 wait complete. 2023-05-20T01:16:52.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.596+0000 7ff05544d700 1 Processor -- start 2023-05-20T01:16:52.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.596+0000 7ff05544d700 1 -- start start 2023-05-20T01:16:52.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.596+0000 7ff05544d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0500b65d0 0x7ff05014edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:52.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.596+0000 7ff05544d700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff0500cd890 0x7ff050157100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:52.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.596+0000 7ff05544d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff05015ca90 0x7ff050157640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:52.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.596+0000 7ff05544d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0500caa00 con 0x7ff0500b65d0 2023-05-20T01:16:52.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.596+0000 7ff05544d700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0500c8040 con 0x7ff05015ca90 2023-05-20T01:16:52.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.596+0000 7ff05544d700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0500c7d60 con 0x7ff0500cd890 2023-05-20T01:16:52.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.596+0000 7ff04effd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0500b65d0 0x7ff05014edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:52.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.597+0000 7ff04effd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0500b65d0 0x7ff05014edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58476/0 (socket says 172.21.15.16:58476) 2023-05-20T01:16:52.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.597+0000 7ff04effd700 1 -- 172.21.15.16:0/192103348 learned_addr learned my addr 172.21.15.16:0/192103348 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:52.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.597+0000 7ff04f7fe700 1 --2- 172.21.15.16:0/192103348 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff05015ca90 0x7ff050157640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:52.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.597+0000 7ff04e7fc700 1 --2- 172.21.15.16:0/192103348 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff0500cd890 0x7ff050157100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:52.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.597+0000 7ff04effd700 1 -- 172.21.15.16:0/192103348 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff0500cd890 msgr2=0x7ff050157100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:52.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.597+0000 7ff04effd700 1 --2- 172.21.15.16:0/192103348 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff0500cd890 0x7ff050157100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:52.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.597+0000 7ff04effd700 1 -- 172.21.15.16:0/192103348 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff05015ca90 msgr2=0x7ff050157640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:52.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.597+0000 7ff04effd700 1 --2- 172.21.15.16:0/192103348 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff05015ca90 0x7ff050157640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:52.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.597+0000 7ff04effd700 1 -- 172.21.15.16:0/192103348 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff048009a70 con 0x7ff0500b65d0 2023-05-20T01:16:52.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.597+0000 7ff04f7fe700 1 --2- 172.21.15.16:0/192103348 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff05015ca90 0x7ff050157640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:16:52.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.597+0000 7ff04effd700 1 --2- 172.21.15.16:0/192103348 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0500b65d0 0x7ff05014edb0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff0440011e0 tx=0x7ff04400cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:52.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.597+0000 7ff0377fe700 1 -- 172.21.15.16:0/192103348 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff044004c80 con 0x7ff0500b65d0 2023-05-20T01:16:52.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.597+0000 7ff0377fe700 1 -- 172.21.15.16:0/192103348 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff044004e00 con 0x7ff0500b65d0 2023-05-20T01:16:52.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.597+0000 7ff05544d700 1 -- 172.21.15.16:0/192103348 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0500cdf00 con 0x7ff0500b65d0 2023-05-20T01:16:52.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.597+0000 7ff0377fe700 1 -- 172.21.15.16:0/192103348 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff044007310 con 0x7ff0500b65d0 2023-05-20T01:16:52.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.597+0000 7ff05544d700 1 -- 172.21.15.16:0/192103348 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0500b6a50 con 0x7ff0500b65d0 2023-05-20T01:16:52.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.598+0000 7ff05544d700 1 -- 172.21.15.16:0/192103348 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0500d1a70 con 0x7ff0500b65d0 2023-05-20T01:16:52.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.598+0000 7ff0377fe700 1 -- 172.21.15.16:0/192103348 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff044007470 con 0x7ff0500b65d0 2023-05-20T01:16:52.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.599+0000 7ff0377fe700 1 --2- 172.21.15.16:0/192103348 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff03c06ace0 0x7ff03c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:52.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.599+0000 7ff0377fe700 1 -- 172.21.15.16:0/192103348 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7ff0440895a0 con 0x7ff0500b65d0 2023-05-20T01:16:52.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.599+0000 7ff04e7fc700 1 --2- 172.21.15.16:0/192103348 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff03c06ace0 0x7ff03c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:52.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.599+0000 7ff04e7fc700 1 --2- 172.21.15.16:0/192103348 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff03c06ace0 0x7ff03c06d190 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff040006fd0 tx=0x7ff040008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:52.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.603+0000 7ff0377fe700 1 -- 172.21.15.16:0/192103348 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7ff044054360 con 0x7ff0500b65d0 2023-05-20T01:16:52.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.800+0000 7ff05544d700 1 -- 172.21.15.16:0/192103348 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ff05004e8c0 con 0x7ff0500b65d0 2023-05-20T01:16:52.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.800+0000 7ff0377fe700 1 -- 172.21.15.16:0/192103348 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7ff044057980 con 0x7ff0500b65d0 2023-05-20T01:16:52.805 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 8 2023-05-20T01:16:52.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.802+0000 7ff05544d700 1 -- 172.21.15.16:0/192103348 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff03c06ace0 msgr2=0x7ff03c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:52.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.802+0000 7ff05544d700 1 --2- 172.21.15.16:0/192103348 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff03c06ace0 0x7ff03c06d190 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff040006fd0 tx=0x7ff040008040 comp rx=0 tx=0).stop 2023-05-20T01:16:52.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.802+0000 7ff05544d700 1 -- 172.21.15.16:0/192103348 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0500b65d0 msgr2=0x7ff05014edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:52.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.802+0000 7ff05544d700 1 --2- 172.21.15.16:0/192103348 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0500b65d0 0x7ff05014edb0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff0440011e0 tx=0x7ff04400cae0 comp rx=0 tx=0).stop 2023-05-20T01:16:52.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.802+0000 7ff05544d700 1 -- 172.21.15.16:0/192103348 shutdown_connections 2023-05-20T01:16:52.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.802+0000 7ff05544d700 1 --2- 172.21.15.16:0/192103348 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff05015ca90 0x7ff050157640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:52.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.802+0000 7ff05544d700 1 --2- 172.21.15.16:0/192103348 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0500b65d0 0x7ff05014edb0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:52.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.802+0000 7ff05544d700 1 --2- 172.21.15.16:0/192103348 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff03c06ace0 0x7ff03c06d190 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:52.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.802+0000 7ff05544d700 1 --2- 172.21.15.16:0/192103348 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff0500cd890 0x7ff050157100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:52.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.802+0000 7ff05544d700 1 -- 172.21.15.16:0/192103348 >> 172.21.15.16:0/192103348 conn(0x7ff05006b000 msgr2=0x7ff05015b920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:52.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.802+0000 7ff05544d700 1 -- 172.21.15.16:0/192103348 shutdown_connections 2023-05-20T01:16:52.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.802+0000 7ff05544d700 1 -- 172.21.15.16:0/192103348 wait complete. 2023-05-20T01:16:52.818 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-05-20T01:16:52.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.942+0000 7f892ccdb700 1 Processor -- start 2023-05-20T01:16:52.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.942+0000 7f892ccdb700 1 -- start start 2023-05-20T01:16:52.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.943+0000 7f892ccdb700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f892815cec0 0x7f89280cd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:52.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.943+0000 7f892ccdb700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f892806e190 0x7f89280cf1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:52.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.943+0000 7f892ccdb700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f892806a510 0x7f892806a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:52.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.943+0000 7f892ccdb700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f892806aed0 con 0x7f892806e190 2023-05-20T01:16:52.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.943+0000 7f892ccdb700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f892806dd00 con 0x7f892815cec0 2023-05-20T01:16:52.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.943+0000 7f892ccdb700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89280cfa60 con 0x7f892806a510 2023-05-20T01:16:52.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.943+0000 7f8925d9b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f892806e190 0x7f89280cf1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:52.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.943+0000 7f8925d9b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f892806e190 0x7f89280cf1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58484/0 (socket says 172.21.15.16:58484) 2023-05-20T01:16:52.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.943+0000 7f8925d9b700 1 -- 172.21.15.16:0/3304124029 learned_addr learned my addr 172.21.15.16:0/3304124029 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:52.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.943+0000 7f8926d9d700 1 --2- 172.21.15.16:0/3304124029 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f892806a510 0x7f892806a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:52.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.943+0000 7f892659c700 1 --2- 172.21.15.16:0/3304124029 >> v2:172.21.15.132:3300/0 conn(0x7f892815cec0 0x7f89280cd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:52.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.944+0000 7f8925d9b700 1 -- 172.21.15.16:0/3304124029 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f892806a510 msgr2=0x7f892806a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:52.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.944+0000 7f8925d9b700 1 --2- 172.21.15.16:0/3304124029 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f892806a510 0x7f892806a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:52.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.944+0000 7f8925d9b700 1 -- 172.21.15.16:0/3304124029 >> v2:172.21.15.132:3300/0 conn(0x7f892815cec0 msgr2=0x7f89280cd4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:52.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.944+0000 7f8925d9b700 1 --2- 172.21.15.16:0/3304124029 >> v2:172.21.15.132:3300/0 conn(0x7f892815cec0 0x7f89280cd4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:52.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.944+0000 7f8925d9b700 1 -- 172.21.15.16:0/3304124029 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f892806d6b0 con 0x7f892806e190 2023-05-20T01:16:52.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.944+0000 7f8925d9b700 1 --2- 172.21.15.16:0/3304124029 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f892806e190 0x7f89280cf1f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f892806b5f0 tx=0x7f8918009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6df3fe8ea3cc39b8 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:52.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.944+0000 7f892559a700 1 -- 172.21.15.16:0/3304124029 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f891800b870 con 0x7f892806e190 2023-05-20T01:16:52.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.944+0000 7f892559a700 1 -- 172.21.15.16:0/3304124029 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f891800b9d0 con 0x7f892806e190 2023-05-20T01:16:52.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.944+0000 7f892559a700 1 -- 172.21.15.16:0/3304124029 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f891800bd60 con 0x7f892806e190 2023-05-20T01:16:52.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.944+0000 7f892ccdb700 1 -- 172.21.15.16:0/3304124029 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f892806e190 msgr2=0x7f89280cf1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:52.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.944+0000 7f892ccdb700 1 --2- 172.21.15.16:0/3304124029 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f892806e190 0x7f89280cf1f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f892806b5f0 tx=0x7f8918009bd0 comp rx=0 tx=0).stop 2023-05-20T01:16:52.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.945+0000 7f892ccdb700 1 -- 172.21.15.16:0/3304124029 shutdown_connections 2023-05-20T01:16:52.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.945+0000 7f892ccdb700 1 --2- 172.21.15.16:0/3304124029 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f892806a510 0x7f892806a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:52.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.945+0000 7f892ccdb700 1 --2- 172.21.15.16:0/3304124029 >> v2:172.21.15.132:3300/0 conn(0x7f892815cec0 0x7f89280cd4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:52.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.945+0000 7f892ccdb700 1 --2- 172.21.15.16:0/3304124029 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f892806e190 0x7f89280cf1f0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:52.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.945+0000 7f892ccdb700 1 -- 172.21.15.16:0/3304124029 >> 172.21.15.16:0/3304124029 conn(0x7f892806c260 msgr2=0x7f89280cffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:52.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.945+0000 7f892ccdb700 1 -- 172.21.15.16:0/3304124029 shutdown_connections 2023-05-20T01:16:52.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.945+0000 7f892ccdb700 1 -- 172.21.15.16:0/3304124029 wait complete. 2023-05-20T01:16:52.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.946+0000 7f892ccdb700 1 Processor -- start 2023-05-20T01:16:52.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.946+0000 7f892ccdb700 1 -- start start 2023-05-20T01:16:52.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.947+0000 7f892ccdb700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f892806a510 0x7f89280c5860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:52.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.947+0000 7f892ccdb700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f892806e190 0x7f89281552e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:52.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.947+0000 7f892ccdb700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f892815cec0 0x7f8928155820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:52.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.947+0000 7f892ccdb700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89280acb60 con 0x7f892806e190 2023-05-20T01:16:52.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.947+0000 7f892ccdb700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89280ace40 con 0x7f892815cec0 2023-05-20T01:16:52.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.947+0000 7f892ccdb700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89280ad120 con 0x7f892806a510 2023-05-20T01:16:52.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.947+0000 7f8925d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f892806e190 0x7f89281552e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:52.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.947+0000 7f8925d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f892806e190 0x7f89281552e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58492/0 (socket says 172.21.15.16:58492) 2023-05-20T01:16:52.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.947+0000 7f8925d9b700 1 -- 172.21.15.16:0/2562458593 learned_addr learned my addr 172.21.15.16:0/2562458593 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:52.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.947+0000 7f892659c700 1 --2- 172.21.15.16:0/2562458593 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f892806a510 0x7f89280c5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:52.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.947+0000 7f8926d9d700 1 --2- 172.21.15.16:0/2562458593 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f892815cec0 0x7f8928155820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:52.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.947+0000 7f8925d9b700 1 -- 172.21.15.16:0/2562458593 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f892806a510 msgr2=0x7f89280c5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:52.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.947+0000 7f8925d9b700 1 --2- 172.21.15.16:0/2562458593 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f892806a510 0x7f89280c5860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:52.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.947+0000 7f8925d9b700 1 -- 172.21.15.16:0/2562458593 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f892815cec0 msgr2=0x7f8928155820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:52.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.947+0000 7f8925d9b700 1 --2- 172.21.15.16:0/2562458593 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f892815cec0 0x7f8928155820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:52.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.947+0000 7f8925d9b700 1 -- 172.21.15.16:0/2562458593 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8918009550 con 0x7f892806e190 2023-05-20T01:16:52.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.947+0000 7f8925d9b700 1 --2- 172.21.15.16:0/2562458593 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f892806e190 0x7f89281552e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f8918000c00 tx=0x7f8918004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:52.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.947+0000 7f8916ffd700 1 -- 172.21.15.16:0/2562458593 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8918004250 con 0x7f892806e190 2023-05-20T01:16:52.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.948+0000 7f8916ffd700 1 -- 172.21.15.16:0/2562458593 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f891801f070 con 0x7f892806e190 2023-05-20T01:16:52.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.948+0000 7f8916ffd700 1 -- 172.21.15.16:0/2562458593 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89180045d0 con 0x7f892806e190 2023-05-20T01:16:52.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.948+0000 7f892ccdb700 1 -- 172.21.15.16:0/2562458593 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f892815d290 con 0x7f892806e190 2023-05-20T01:16:52.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.948+0000 7f892ccdb700 1 -- 172.21.15.16:0/2562458593 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f892806d990 con 0x7f892806e190 2023-05-20T01:16:52.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.949+0000 7f892ccdb700 1 -- 172.21.15.16:0/2562458593 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89280cfd40 con 0x7f892806e190 2023-05-20T01:16:52.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.949+0000 7f8916ffd700 1 -- 172.21.15.16:0/2562458593 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8918021030 con 0x7f892806e190 2023-05-20T01:16:52.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.950+0000 7f8916ffd700 1 --2- 172.21.15.16:0/2562458593 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f890806ace0 0x7f890806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:52.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.950+0000 7f892659c700 1 --2- 172.21.15.16:0/2562458593 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f890806ace0 0x7f890806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:52.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.950+0000 7f8916ffd700 1 -- 172.21.15.16:0/2562458593 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f891808a890 con 0x7f892806e190 2023-05-20T01:16:52.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.950+0000 7f892659c700 1 --2- 172.21.15.16:0/2562458593 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f890806ace0 0x7f890806d190 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f891c006b20 tx=0x7f891c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:52.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:52.954+0000 7f8916ffd700 1 -- 172.21.15.16:0/2562458593 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f8918055800 con 0x7f892806e190 2023-05-20T01:16:53.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.151+0000 7f892ccdb700 1 -- 172.21.15.16:0/2562458593 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f89280acca0 con 0x7f892806e190 2023-05-20T01:16:53.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.152+0000 7f8916ffd700 1 -- 172.21.15.16:0/2562458593 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8918015020 con 0x7f892806e190 2023-05-20T01:16:53.156 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-05-20T01:16:53.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.154+0000 7f892ccdb700 1 -- 172.21.15.16:0/2562458593 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f890806ace0 msgr2=0x7f890806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:53.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.154+0000 7f892ccdb700 1 --2- 172.21.15.16:0/2562458593 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f890806ace0 0x7f890806d190 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f891c006b20 tx=0x7f891c008040 comp rx=0 tx=0).stop 2023-05-20T01:16:53.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.154+0000 7f892ccdb700 1 -- 172.21.15.16:0/2562458593 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f892806e190 msgr2=0x7f89281552e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:53.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.154+0000 7f892ccdb700 1 --2- 172.21.15.16:0/2562458593 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f892806e190 0x7f89281552e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f8918000c00 tx=0x7f8918004750 comp rx=0 tx=0).stop 2023-05-20T01:16:53.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.154+0000 7f892ccdb700 1 -- 172.21.15.16:0/2562458593 shutdown_connections 2023-05-20T01:16:53.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.154+0000 7f892ccdb700 1 --2- 172.21.15.16:0/2562458593 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f892815cec0 0x7f8928155820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:53.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.154+0000 7f892ccdb700 1 --2- 172.21.15.16:0/2562458593 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f890806ace0 0x7f890806d190 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:53.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.154+0000 7f892ccdb700 1 --2- 172.21.15.16:0/2562458593 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f892806a510 0x7f89280c5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:53.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.154+0000 7f892ccdb700 1 --2- 172.21.15.16:0/2562458593 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f892806e190 0x7f89281552e0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:53.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.154+0000 7f892ccdb700 1 -- 172.21.15.16:0/2562458593 >> 172.21.15.16:0/2562458593 conn(0x7f892806c260 msgr2=0x7f89280b4080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:53.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.154+0000 7f892ccdb700 1 -- 172.21.15.16:0/2562458593 shutdown_connections 2023-05-20T01:16:53.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.154+0000 7f892ccdb700 1 -- 172.21.15.16:0/2562458593 wait complete. 2023-05-20T01:16:53.170 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-05-20T01:16:53.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.295+0000 7f5092028700 1 Processor -- start 2023-05-20T01:16:53.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.295+0000 7f5092028700 1 -- start start 2023-05-20T01:16:53.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.296+0000 7f5092028700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f508c15cec0 0x7f508c071bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:53.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.296+0000 7f5092028700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f508c072130 0x7f508c0cf9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:53.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.296+0000 7f5092028700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f508c072590 con 0x7f508c15d290 2023-05-20T01:16:53.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.296+0000 7f5092028700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f508c0cd630 con 0x7f508c072130 2023-05-20T01:16:53.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.296+0000 7f5092028700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f508c0cca60 con 0x7f508c15cec0 2023-05-20T01:16:53.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.296+0000 7f508bfff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f508c072130 0x7f508c0cf9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:53.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.296+0000 7f508affd700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f508c15d290 0x7f508c0cf2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:41794/0 (socket says 172.21.15.16:41794) 2023-05-20T01:16:53.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.296+0000 7f508affd700 1 -- 172.21.15.16:0/109955770 learned_addr learned my addr 172.21.15.16:0/109955770 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:16:53.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.296+0000 7f508b7fe700 1 --2- 172.21.15.16:0/109955770 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f508c15cec0 0x7f508c071bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:53.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.296+0000 7f508bfff700 1 -- 172.21.15.16:0/109955770 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f508c15cec0 msgr2=0x7f508c071bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:53.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.296+0000 7f508bfff700 1 --2- 172.21.15.16:0/109955770 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f508c15cec0 0x7f508c071bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:53.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.296+0000 7f508bfff700 1 -- 172.21.15.16:0/109955770 >> v1:172.21.15.16:6789/0 conn(0x7f508c15d290 legacy=0x7f508c0cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:53.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.296+0000 7f508a7fc700 1 -- 172.21.15.16:0/109955770 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 901463183 0 0) 0x7f508c072590 con 0x7f508c15d290 2023-05-20T01:16:53.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.296+0000 7f508bfff700 1 -- 172.21.15.16:0/109955770 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f508c0ccd00 con 0x7f508c072130 2023-05-20T01:16:53.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.296+0000 7f508bfff700 1 --2- 172.21.15.16:0/109955770 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f508c072130 0x7f508c0cf9d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f5084009d90 tx=0x7f508400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afa47fb8f2eb57d5 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:53.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.297+0000 7f508a7fc700 1 -- 172.21.15.16:0/109955770 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f508400e070 con 0x7f508c072130 2023-05-20T01:16:53.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.297+0000 7f508a7fc700 1 -- 172.21.15.16:0/109955770 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f508400bbd0 con 0x7f508c072130 2023-05-20T01:16:53.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.297+0000 7f508a7fc700 1 -- 172.21.15.16:0/109955770 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5084012430 con 0x7f508c072130 2023-05-20T01:16:53.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.297+0000 7f5092028700 1 -- 172.21.15.16:0/109955770 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f508c072130 msgr2=0x7f508c0cf9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:53.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.297+0000 7f5092028700 1 --2- 172.21.15.16:0/109955770 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f508c072130 0x7f508c0cf9d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f5084009d90 tx=0x7f508400b330 comp rx=0 tx=0).stop 2023-05-20T01:16:53.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.297+0000 7f5092028700 1 -- 172.21.15.16:0/109955770 shutdown_connections 2023-05-20T01:16:53.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.297+0000 7f5092028700 1 --2- 172.21.15.16:0/109955770 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f508c072130 0x7f508c0cf9d0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:53.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.297+0000 7f5092028700 1 --2- 172.21.15.16:0/109955770 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f508c15cec0 0x7f508c071bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:53.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.297+0000 7f5092028700 1 -- 172.21.15.16:0/109955770 >> 172.21.15.16:0/109955770 conn(0x7f508c0cffd0 msgr2=0x7f508c0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:53.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.297+0000 7f5092028700 1 -- 172.21.15.16:0/109955770 shutdown_connections 2023-05-20T01:16:53.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.297+0000 7f5092028700 1 -- 172.21.15.16:0/109955770 wait complete. 2023-05-20T01:16:53.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.298+0000 7f5092028700 1 Processor -- start 2023-05-20T01:16:53.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.299+0000 7f5092028700 1 -- start start 2023-05-20T01:16:53.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.299+0000 7f5092028700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f508c072130 0x7f508c0b3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:53.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.299+0000 7f5092028700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f508c15cec0 0x7f508c0b4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:53.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.299+0000 7f5092028700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f508c15d290 0x7f508c0c5d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:53.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.299+0000 7f5092028700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f508c0b5e70 con 0x7f508c15d290 2023-05-20T01:16:53.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.299+0000 7f5092028700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f508c0b53c0 con 0x7f508c072130 2023-05-20T01:16:53.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.299+0000 7f5092028700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f508c0c6280 con 0x7f508c15cec0 2023-05-20T01:16:53.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.300+0000 7f508bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f508c15d290 0x7f508c0c5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:53.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.300+0000 7f508bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f508c15d290 0x7f508c0c5d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58508/0 (socket says 172.21.15.16:58508) 2023-05-20T01:16:53.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.300+0000 7f508bfff700 1 -- 172.21.15.16:0/776715954 learned_addr learned my addr 172.21.15.16:0/776715954 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:53.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.300+0000 7f508b7fe700 1 --2- 172.21.15.16:0/776715954 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f508c072130 0x7f508c0b3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:53.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.300+0000 7f508affd700 1 --2- 172.21.15.16:0/776715954 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f508c15cec0 0x7f508c0b4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:53.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.300+0000 7f508bfff700 1 -- 172.21.15.16:0/776715954 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f508c15cec0 msgr2=0x7f508c0b4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:53.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.300+0000 7f508bfff700 1 --2- 172.21.15.16:0/776715954 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f508c15cec0 0x7f508c0b4e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:53.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.300+0000 7f508bfff700 1 -- 172.21.15.16:0/776715954 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f508c072130 msgr2=0x7f508c0b3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:53.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.300+0000 7f508bfff700 1 --2- 172.21.15.16:0/776715954 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f508c072130 0x7f508c0b3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:53.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.300+0000 7f508bfff700 1 -- 172.21.15.16:0/776715954 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5084009a70 con 0x7f508c15d290 2023-05-20T01:16:53.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.300+0000 7f508bfff700 1 --2- 172.21.15.16:0/776715954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f508c15d290 0x7f508c0c5d40 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f508c0cdb50 tx=0x7f5084004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:53.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.300+0000 7f5083fff700 1 -- 172.21.15.16:0/776715954 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5084012430 con 0x7f508c15d290 2023-05-20T01:16:53.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.300+0000 7f5083fff700 1 -- 172.21.15.16:0/776715954 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5084020070 con 0x7f508c15d290 2023-05-20T01:16:53.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.300+0000 7f5092028700 1 -- 172.21.15.16:0/776715954 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f508c0cd2c0 con 0x7f508c15d290 2023-05-20T01:16:53.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.300+0000 7f5083fff700 1 -- 172.21.15.16:0/776715954 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5084004280 con 0x7f508c15d290 2023-05-20T01:16:53.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.300+0000 7f5092028700 1 -- 172.21.15.16:0/776715954 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f508c0ccfe0 con 0x7f508c15d290 2023-05-20T01:16:53.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.301+0000 7f5092028700 1 -- 172.21.15.16:0/776715954 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f508c160890 con 0x7f508c15d290 2023-05-20T01:16:53.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.302+0000 7f5083fff700 1 -- 172.21.15.16:0/776715954 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f50840043e0 con 0x7f508c15d290 2023-05-20T01:16:53.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.302+0000 7f5083fff700 1 --2- 172.21.15.16:0/776715954 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f506c06ace0 0x7f506c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:53.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.302+0000 7f5083fff700 1 -- 172.21.15.16:0/776715954 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f5084022070 con 0x7f508c15d290 2023-05-20T01:16:53.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.302+0000 7f508b7fe700 1 --2- 172.21.15.16:0/776715954 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f506c06ace0 0x7f506c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:53.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.303+0000 7f508b7fe700 1 --2- 172.21.15.16:0/776715954 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f506c06ace0 0x7f506c06d190 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f507c006fd0 tx=0x7f507c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:53.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.307+0000 7f5083fff700 1 -- 172.21.15.16:0/776715954 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f5084056790 con 0x7f508c15d290 2023-05-20T01:16:53.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.506+0000 7f5092028700 1 -- 172.21.15.16:0/776715954 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f508c0c75b0 con 0x7f508c15d290 2023-05-20T01:16:53.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.506+0000 7f5083fff700 1 -- 172.21.15.16:0/776715954 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5084059db0 con 0x7f508c15d290 2023-05-20T01:16:53.510 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-05-20T01:16:53.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.509+0000 7f5092028700 1 -- 172.21.15.16:0/776715954 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f506c06ace0 msgr2=0x7f506c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:53.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.509+0000 7f5092028700 1 --2- 172.21.15.16:0/776715954 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f506c06ace0 0x7f506c06d190 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f507c006fd0 tx=0x7f507c008040 comp rx=0 tx=0).stop 2023-05-20T01:16:53.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.509+0000 7f5092028700 1 -- 172.21.15.16:0/776715954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f508c15d290 msgr2=0x7f508c0c5d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:53.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.509+0000 7f5092028700 1 --2- 172.21.15.16:0/776715954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f508c15d290 0x7f508c0c5d40 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f508c0cdb50 tx=0x7f5084004ad0 comp rx=0 tx=0).stop 2023-05-20T01:16:53.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.509+0000 7f5092028700 1 -- 172.21.15.16:0/776715954 shutdown_connections 2023-05-20T01:16:53.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.509+0000 7f5092028700 1 --2- 172.21.15.16:0/776715954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f508c15d290 0x7f508c0c5d40 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:53.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.509+0000 7f5092028700 1 --2- 172.21.15.16:0/776715954 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f508c072130 0x7f508c0b3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:53.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.509+0000 7f5092028700 1 --2- 172.21.15.16:0/776715954 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f506c06ace0 0x7f506c06d190 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:53.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.509+0000 7f5092028700 1 --2- 172.21.15.16:0/776715954 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f508c15cec0 0x7f508c0b4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:53.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.509+0000 7f5092028700 1 -- 172.21.15.16:0/776715954 >> 172.21.15.16:0/776715954 conn(0x7f508c0cffd0 msgr2=0x7f508c0c68a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:53.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.509+0000 7f5092028700 1 -- 172.21.15.16:0/776715954 shutdown_connections 2023-05-20T01:16:53.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.509+0000 7f5092028700 1 -- 172.21.15.16:0/776715954 wait complete. 2023-05-20T01:16:53.525 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-05-20T01:16:53.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.649+0000 7f8a95420700 1 Processor -- start 2023-05-20T01:16:53.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.649+0000 7f8a95420700 1 -- start start 2023-05-20T01:16:53.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.650+0000 7f8a95420700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8a900cb7b0 0x7f8a900cc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:53.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.650+0000 7f8a95420700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a900cbbd0 con 0x7f8a90158f90 2023-05-20T01:16:53.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.650+0000 7f8a95420700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a90159360 con 0x7f8a900ce7b0 2023-05-20T01:16:53.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.650+0000 7f8a95420700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a900cce10 con 0x7f8a900cb7b0 2023-05-20T01:16:53.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.650+0000 7f8a8effd700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f8a90158f90 0x7f8a900ce0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:41808/0 (socket says 172.21.15.16:41808) 2023-05-20T01:16:53.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.650+0000 7f8a8f7fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8a900cb7b0 0x7f8a900cc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:53.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.650+0000 7f8a8effd700 1 -- 172.21.15.16:0/4238173089 learned_addr learned my addr 172.21.15.16:0/4238173089 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:16:53.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.651+0000 7f8a8dffb700 1 -- 172.21.15.16:0/4238173089 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4021298390 0 0) 0x7f8a900cbbd0 con 0x7f8a90158f90 2023-05-20T01:16:53.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.651+0000 7f8a8dffb700 1 -- 172.21.15.16:0/4238173089 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a780035b0 con 0x7f8a90158f90 2023-05-20T01:16:53.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.651+0000 7f8a8f7fe700 1 -- 172.21.15.16:0/4238173089 >> v1:172.21.15.132:6789/0 conn(0x7f8a900ce7b0 legacy=0x7f8a900cc050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:53.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.651+0000 7f8a8f7fe700 1 -- 172.21.15.16:0/4238173089 >> v1:172.21.15.16:6789/0 conn(0x7f8a90158f90 legacy=0x7f8a900ce0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:53.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.651+0000 7f8a8f7fe700 1 -- 172.21.15.16:0/4238173089 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a900ca750 con 0x7f8a900cb7b0 2023-05-20T01:16:53.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.651+0000 7f8a8f7fe700 1 --2- 172.21.15.16:0/4238173089 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a900cb7b0 0x7f8a900cc760 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8a88009d90 tx=0x7f8a8800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14f5da8b5de6e5fa server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:53.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.651+0000 7f8a8dffb700 1 -- 172.21.15.16:0/4238173089 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a8800e070 con 0x7f8a900cb7b0 2023-05-20T01:16:53.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.651+0000 7f8a8dffb700 1 -- 172.21.15.16:0/4238173089 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a8800bb20 con 0x7f8a900cb7b0 2023-05-20T01:16:53.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.651+0000 7f8a8dffb700 1 -- 172.21.15.16:0/4238173089 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a88012430 con 0x7f8a900cb7b0 2023-05-20T01:16:53.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.652+0000 7f8a95420700 1 -- 172.21.15.16:0/4238173089 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a900cb7b0 msgr2=0x7f8a900cc760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:53.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.652+0000 7f8a95420700 1 --2- 172.21.15.16:0/4238173089 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a900cb7b0 0x7f8a900cc760 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8a88009d90 tx=0x7f8a8800b330 comp rx=0 tx=0).stop 2023-05-20T01:16:53.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.652+0000 7f8a95420700 1 -- 172.21.15.16:0/4238173089 shutdown_connections 2023-05-20T01:16:53.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.652+0000 7f8a95420700 1 --2- 172.21.15.16:0/4238173089 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a900cb7b0 0x7f8a900cc760 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:53.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.652+0000 7f8a95420700 1 -- 172.21.15.16:0/4238173089 >> 172.21.15.16:0/4238173089 conn(0x7f8a9006cbb0 msgr2=0x7f8a9006cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:53.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.652+0000 7f8a95420700 1 -- 172.21.15.16:0/4238173089 shutdown_connections 2023-05-20T01:16:53.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.652+0000 7f8a95420700 1 -- 172.21.15.16:0/4238173089 wait complete. 2023-05-20T01:16:53.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.653+0000 7f8a95420700 1 Processor -- start 2023-05-20T01:16:53.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.653+0000 7f8a95420700 1 -- start start 2023-05-20T01:16:53.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.654+0000 7f8a95420700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8a900cb7b0 0x7f8a90146380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:53.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.654+0000 7f8a95420700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a900ce7b0 0x7f8a900cbfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:53.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.654+0000 7f8a95420700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8a90158f90 0x7f8a900cc500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:53.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.654+0000 7f8a95420700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a90147750 con 0x7f8a900ce7b0 2023-05-20T01:16:53.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.654+0000 7f8a95420700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a90146bd0 con 0x7f8a90158f90 2023-05-20T01:16:53.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.654+0000 7f8a95420700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a90146eb0 con 0x7f8a900cb7b0 2023-05-20T01:16:53.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.654+0000 7f8a8e7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a900ce7b0 0x7f8a900cbfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:53.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.654+0000 7f8a8e7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a900ce7b0 0x7f8a900cbfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58520/0 (socket says 172.21.15.16:58520) 2023-05-20T01:16:53.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.654+0000 7f8a8e7fc700 1 -- 172.21.15.16:0/210634655 learned_addr learned my addr 172.21.15.16:0/210634655 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:53.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.654+0000 7f8a8f7fe700 1 --2- 172.21.15.16:0/210634655 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8a90158f90 0x7f8a900cc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:53.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.654+0000 7f8a8effd700 1 --2- 172.21.15.16:0/210634655 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8a900cb7b0 0x7f8a90146380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:53.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.654+0000 7f8a8e7fc700 1 -- 172.21.15.16:0/210634655 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8a900cb7b0 msgr2=0x7f8a90146380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:53.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.654+0000 7f8a8e7fc700 1 --2- 172.21.15.16:0/210634655 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8a900cb7b0 0x7f8a90146380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:53.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.654+0000 7f8a8e7fc700 1 -- 172.21.15.16:0/210634655 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8a90158f90 msgr2=0x7f8a900cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:53.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.654+0000 7f8a8e7fc700 1 --2- 172.21.15.16:0/210634655 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8a90158f90 0x7f8a900cc500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:53.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.654+0000 7f8a8e7fc700 1 -- 172.21.15.16:0/210634655 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a88009a70 con 0x7f8a900ce7b0 2023-05-20T01:16:53.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.654+0000 7f8a8e7fc700 1 --2- 172.21.15.16:0/210634655 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a900ce7b0 0x7f8a900cbfc0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8a8000c480 tx=0x7f8a8000c790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:53.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.654+0000 7f8a777fe700 1 -- 172.21.15.16:0/210634655 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a800052d0 con 0x7f8a900ce7b0 2023-05-20T01:16:53.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.655+0000 7f8a777fe700 1 -- 172.21.15.16:0/210634655 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a8000ce90 con 0x7f8a900ce7b0 2023-05-20T01:16:53.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.655+0000 7f8a95420700 1 -- 172.21.15.16:0/210634655 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a900cacd0 con 0x7f8a900ce7b0 2023-05-20T01:16:53.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.655+0000 7f8a777fe700 1 -- 172.21.15.16:0/210634655 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a80003680 con 0x7f8a900ce7b0 2023-05-20T01:16:53.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.655+0000 7f8a95420700 1 -- 172.21.15.16:0/210634655 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a900ca9f0 con 0x7f8a900ce7b0 2023-05-20T01:16:53.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.655+0000 7f8a95420700 1 -- 172.21.15.16:0/210634655 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a900cafb0 con 0x7f8a900ce7b0 2023-05-20T01:16:53.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.656+0000 7f8a777fe700 1 -- 172.21.15.16:0/210634655 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8a800037e0 con 0x7f8a900ce7b0 2023-05-20T01:16:53.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.656+0000 7f8a777fe700 1 --2- 172.21.15.16:0/210634655 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f8a7c06ace0 0x7f8a7c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:53.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.657+0000 7f8a8effd700 1 --2- 172.21.15.16:0/210634655 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f8a7c06ace0 0x7f8a7c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:53.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.657+0000 7f8a777fe700 1 -- 172.21.15.16:0/210634655 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f8a80088950 con 0x7f8a900ce7b0 2023-05-20T01:16:53.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.657+0000 7f8a8effd700 1 --2- 172.21.15.16:0/210634655 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f8a7c06ace0 0x7f8a7c06d190 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8a84007fd0 tx=0x7f8a84007dc0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:53.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.661+0000 7f8a777fe700 1 -- 172.21.15.16:0/210634655 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f8a80053710 con 0x7f8a900ce7b0 2023-05-20T01:16:53.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.858+0000 7f8a95420700 1 -- 172.21.15.16:0/210634655 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8a90159360 con 0x7f8a900ce7b0 2023-05-20T01:16:53.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.858+0000 7f8a777fe700 1 -- 172.21.15.16:0/210634655 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8a90159360 con 0x7f8a900ce7b0 2023-05-20T01:16:53.862 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2023-05-20T01:16:53.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.860+0000 7f8a95420700 1 -- 172.21.15.16:0/210634655 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f8a7c06ace0 msgr2=0x7f8a7c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:53.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.860+0000 7f8a95420700 1 --2- 172.21.15.16:0/210634655 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f8a7c06ace0 0x7f8a7c06d190 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8a84007fd0 tx=0x7f8a84007dc0 comp rx=0 tx=0).stop 2023-05-20T01:16:53.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.860+0000 7f8a95420700 1 -- 172.21.15.16:0/210634655 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a900ce7b0 msgr2=0x7f8a900cbfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:53.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.860+0000 7f8a95420700 1 --2- 172.21.15.16:0/210634655 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a900ce7b0 0x7f8a900cbfc0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8a8000c480 tx=0x7f8a8000c790 comp rx=0 tx=0).stop 2023-05-20T01:16:53.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.860+0000 7f8a95420700 1 -- 172.21.15.16:0/210634655 shutdown_connections 2023-05-20T01:16:53.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.860+0000 7f8a95420700 1 --2- 172.21.15.16:0/210634655 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8a90158f90 0x7f8a900cc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:53.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.860+0000 7f8a95420700 1 --2- 172.21.15.16:0/210634655 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f8a7c06ace0 0x7f8a7c06d190 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:53.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.860+0000 7f8a95420700 1 --2- 172.21.15.16:0/210634655 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8a900cb7b0 0x7f8a90146380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:53.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.860+0000 7f8a95420700 1 --2- 172.21.15.16:0/210634655 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a900ce7b0 0x7f8a900cbfc0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:53.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.860+0000 7f8a95420700 1 -- 172.21.15.16:0/210634655 >> 172.21.15.16:0/210634655 conn(0x7f8a9006cbb0 msgr2=0x7f8a90157890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:53.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.860+0000 7f8a95420700 1 -- 172.21.15.16:0/210634655 shutdown_connections 2023-05-20T01:16:53.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:53.860+0000 7f8a95420700 1 -- 172.21.15.16:0/210634655 wait complete. 2023-05-20T01:16:53.876 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-05-20T01:16:54.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.000+0000 7ff4ad93c700 1 Processor -- start 2023-05-20T01:16:54.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.000+0000 7ff4ad93c700 1 -- start start 2023-05-20T01:16:54.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.001+0000 7ff4ad93c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff4a80b1e50 0x7ff4a806b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:54.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.001+0000 7ff4ad93c700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7ff4a806bb60 0x7ff4a80cc150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:54.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.001+0000 7ff4ad93c700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4a80cb700 con 0x7ff4a815cd20 2023-05-20T01:16:54.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.001+0000 7ff4ad93c700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4a806bf80 con 0x7ff4a80b1e50 2023-05-20T01:16:54.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.001+0000 7ff4ad93c700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4a80b2430 con 0x7ff4a806bb60 2023-05-20T01:16:54.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.001+0000 7ff4a67fc700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff4a80b1e50 0x7ff4a806b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:54.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.001+0000 7ff4a67fc700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff4a80b1e50 0x7ff4a806b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58528/0 (socket says 172.21.15.16:58528) 2023-05-20T01:16:54.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.001+0000 7ff4a67fc700 1 -- 172.21.15.16:0/2482040183 learned_addr learned my addr 172.21.15.16:0/2482040183 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:54.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.001+0000 7ff4a77fe700 1 --2- 172.21.15.16:0/2482040183 >> v2:172.21.15.132:3300/0 conn(0x7ff4a806bb60 0x7ff4a80cc150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:54.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.002+0000 7ff4a67fc700 1 -- 172.21.15.16:0/2482040183 >> v2:172.21.15.132:3300/0 conn(0x7ff4a806bb60 msgr2=0x7ff4a80cc150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:54.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.002+0000 7ff4a67fc700 1 --2- 172.21.15.16:0/2482040183 >> v2:172.21.15.132:3300/0 conn(0x7ff4a806bb60 0x7ff4a80cc150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:54.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.002+0000 7ff4a67fc700 1 -- 172.21.15.16:0/2482040183 >> v1:172.21.15.132:6789/0 conn(0x7ff4a815cd20 legacy=0x7ff4a80b14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:54.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.002+0000 7ff4a67fc700 1 -- 172.21.15.16:0/2482040183 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4a80b2710 con 0x7ff4a80b1e50 2023-05-20T01:16:54.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.002+0000 7ff4a67fc700 1 --2- 172.21.15.16:0/2482040183 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4a80b1e50 0x7ff4a806b620 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff4a80ceaf0 tx=0x7ff498009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=826c8f3b40afae6 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:54.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.002+0000 7ff4a5ffb700 1 -- 172.21.15.16:0/2482040183 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff49800b870 con 0x7ff4a80b1e50 2023-05-20T01:16:54.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.002+0000 7ff4a5ffb700 1 -- 172.21.15.16:0/2482040183 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff49800b9d0 con 0x7ff4a80b1e50 2023-05-20T01:16:54.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.002+0000 7ff4a5ffb700 1 -- 172.21.15.16:0/2482040183 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff49800bd60 con 0x7ff4a80b1e50 2023-05-20T01:16:54.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.002+0000 7ff4ad93c700 1 -- 172.21.15.16:0/2482040183 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4a80b1e50 msgr2=0x7ff4a806b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:54.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.002+0000 7ff4ad93c700 1 --2- 172.21.15.16:0/2482040183 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4a80b1e50 0x7ff4a806b620 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff4a80ceaf0 tx=0x7ff498009bd0 comp rx=0 tx=0).stop 2023-05-20T01:16:54.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.003+0000 7ff4ad93c700 1 -- 172.21.15.16:0/2482040183 shutdown_connections 2023-05-20T01:16:54.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.003+0000 7ff4ad93c700 1 --2- 172.21.15.16:0/2482040183 >> v2:172.21.15.132:3300/0 conn(0x7ff4a806bb60 0x7ff4a80cc150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:54.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.003+0000 7ff4ad93c700 1 --2- 172.21.15.16:0/2482040183 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4a80b1e50 0x7ff4a806b620 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:54.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.003+0000 7ff4ad93c700 1 -- 172.21.15.16:0/2482040183 >> 172.21.15.16:0/2482040183 conn(0x7ff4a806ae30 msgr2=0x7ff4a809a400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:54.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.003+0000 7ff4ad93c700 1 -- 172.21.15.16:0/2482040183 shutdown_connections 2023-05-20T01:16:54.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.003+0000 7ff4ad93c700 1 -- 172.21.15.16:0/2482040183 wait complete. 2023-05-20T01:16:54.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.004+0000 7ff4ad93c700 1 Processor -- start 2023-05-20T01:16:54.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.004+0000 7ff4ad93c700 1 -- start start 2023-05-20T01:16:54.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.005+0000 7ff4ad93c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4a806bb60 0x7ff4a8158050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:54.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.005+0000 7ff4ad93c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff4a80b1e50 0x7ff4a80c8180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:54.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.005+0000 7ff4ad93c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff4a815cd20 0x7ff4a80c71f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:54.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.005+0000 7ff4ad93c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a8158590 con 0x7ff4a806bb60 2023-05-20T01:16:54.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.005+0000 7ff4ad93c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a80c4c80 con 0x7ff4a815cd20 2023-05-20T01:16:54.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.005+0000 7ff4ad93c700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a80c4f60 con 0x7ff4a80b1e50 2023-05-20T01:16:54.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.005+0000 7ff4a6ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4a806bb60 0x7ff4a8158050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:54.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.005+0000 7ff4a6ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4a806bb60 0x7ff4a8158050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58532/0 (socket says 172.21.15.16:58532) 2023-05-20T01:16:54.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.005+0000 7ff4a6ffd700 1 -- 172.21.15.16:0/3432236624 learned_addr learned my addr 172.21.15.16:0/3432236624 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:54.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.006+0000 7ff4a67fc700 1 --2- 172.21.15.16:0/3432236624 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff4a80b1e50 0x7ff4a80c8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:54.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.006+0000 7ff4a77fe700 1 --2- 172.21.15.16:0/3432236624 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff4a815cd20 0x7ff4a80c71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:54.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.006+0000 7ff4a6ffd700 1 -- 172.21.15.16:0/3432236624 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff4a80b1e50 msgr2=0x7ff4a80c8180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:54.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.006+0000 7ff4a6ffd700 1 --2- 172.21.15.16:0/3432236624 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff4a80b1e50 0x7ff4a80c8180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:54.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.006+0000 7ff4a6ffd700 1 -- 172.21.15.16:0/3432236624 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff4a815cd20 msgr2=0x7ff4a80c71f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:54.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.006+0000 7ff4a6ffd700 1 --2- 172.21.15.16:0/3432236624 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff4a815cd20 0x7ff4a80c71f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:54.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.006+0000 7ff4a6ffd700 1 -- 172.21.15.16:0/3432236624 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff498009550 con 0x7ff4a806bb60 2023-05-20T01:16:54.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.006+0000 7ff4a77fe700 1 --2- 172.21.15.16:0/3432236624 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff4a815cd20 0x7ff4a80c71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:16:54.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.006+0000 7ff4a6ffd700 1 --2- 172.21.15.16:0/3432236624 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4a806bb60 0x7ff4a8158050 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ff49c00c4e0 tx=0x7ff49c00c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:54.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.006+0000 7ff48ffff700 1 -- 172.21.15.16:0/3432236624 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff49c0052d0 con 0x7ff4a806bb60 2023-05-20T01:16:54.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.006+0000 7ff48ffff700 1 -- 172.21.15.16:0/3432236624 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff49c005430 con 0x7ff4a806bb60 2023-05-20T01:16:54.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.006+0000 7ff48ffff700 1 -- 172.21.15.16:0/3432236624 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff49c0038c0 con 0x7ff4a806bb60 2023-05-20T01:16:54.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.006+0000 7ff4ad93c700 1 -- 172.21.15.16:0/3432236624 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4a80cbc30 con 0x7ff4a806bb60 2023-05-20T01:16:54.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.006+0000 7ff4ad93c700 1 -- 172.21.15.16:0/3432236624 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4a80cb950 con 0x7ff4a806bb60 2023-05-20T01:16:54.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.007+0000 7ff4ad93c700 1 -- 172.21.15.16:0/3432236624 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4a815d0f0 con 0x7ff4a806bb60 2023-05-20T01:16:54.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.008+0000 7ff48ffff700 1 -- 172.21.15.16:0/3432236624 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff49c003a20 con 0x7ff4a806bb60 2023-05-20T01:16:54.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.008+0000 7ff48ffff700 1 --2- 172.21.15.16:0/3432236624 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff49406ace0 0x7ff49406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:54.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.008+0000 7ff48ffff700 1 -- 172.21.15.16:0/3432236624 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7ff49c088c10 con 0x7ff4a806bb60 2023-05-20T01:16:54.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.008+0000 7ff4a67fc700 1 --2- 172.21.15.16:0/3432236624 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff49406ace0 0x7ff49406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:54.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.009+0000 7ff4a67fc700 1 --2- 172.21.15.16:0/3432236624 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff49406ace0 0x7ff49406d190 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff498000c00 tx=0x7ff498008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:54.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.012+0000 7ff48ffff700 1 -- 172.21.15.16:0/3432236624 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7ff49c0539d0 con 0x7ff4a806bb60 2023-05-20T01:16:54.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.211+0000 7ff4ad93c700 1 -- 172.21.15.16:0/3432236624 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7ff4a80d1f60 con 0x7ff4a806bb60 2023-05-20T01:16:54.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.212+0000 7ff48ffff700 1 -- 172.21.15.16:0/3432236624 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7ff49c017020 con 0x7ff4a806bb60 2023-05-20T01:16:54.216 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-05-20T01:16:54.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.214+0000 7ff4ad93c700 1 -- 172.21.15.16:0/3432236624 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff49406ace0 msgr2=0x7ff49406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:54.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.214+0000 7ff4ad93c700 1 --2- 172.21.15.16:0/3432236624 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff49406ace0 0x7ff49406d190 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff498000c00 tx=0x7ff498008030 comp rx=0 tx=0).stop 2023-05-20T01:16:54.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.214+0000 7ff4ad93c700 1 -- 172.21.15.16:0/3432236624 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4a806bb60 msgr2=0x7ff4a8158050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:54.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.214+0000 7ff4ad93c700 1 --2- 172.21.15.16:0/3432236624 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4a806bb60 0x7ff4a8158050 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ff49c00c4e0 tx=0x7ff49c00c7f0 comp rx=0 tx=0).stop 2023-05-20T01:16:54.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.214+0000 7ff4ad93c700 1 -- 172.21.15.16:0/3432236624 shutdown_connections 2023-05-20T01:16:54.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.214+0000 7ff4ad93c700 1 --2- 172.21.15.16:0/3432236624 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff4a815cd20 0x7ff4a80c71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:54.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.214+0000 7ff4ad93c700 1 --2- 172.21.15.16:0/3432236624 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4a806bb60 0x7ff4a8158050 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:54.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.214+0000 7ff4ad93c700 1 --2- 172.21.15.16:0/3432236624 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff49406ace0 0x7ff49406d190 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:54.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.214+0000 7ff4ad93c700 1 --2- 172.21.15.16:0/3432236624 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff4a80b1e50 0x7ff4a80c8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:54.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.214+0000 7ff4ad93c700 1 -- 172.21.15.16:0/3432236624 >> 172.21.15.16:0/3432236624 conn(0x7ff4a806ae30 msgr2=0x7ff4a806fc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:54.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.214+0000 7ff4ad93c700 1 -- 172.21.15.16:0/3432236624 shutdown_connections 2023-05-20T01:16:54.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.214+0000 7ff4ad93c700 1 -- 172.21.15.16:0/3432236624 wait complete. 2023-05-20T01:16:54.229 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-20T01:16:54.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.350+0000 7fd6ed047700 1 Processor -- start 2023-05-20T01:16:54.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.350+0000 7fd6ed047700 1 -- start start 2023-05-20T01:16:54.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.351+0000 7fd6ed047700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fd6e8157190 0x7fd6e80cc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:54.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.351+0000 7fd6ed047700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd6e8071ad0 0x7fd6e80ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:54.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.351+0000 7fd6ed047700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6e80cb590 con 0x7fd6e80cccf0 2023-05-20T01:16:54.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.351+0000 7fd6ed047700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6e8071f50 con 0x7fd6e8071ad0 2023-05-20T01:16:54.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.351+0000 7fd6ed047700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6e80ca9a0 con 0x7fd6e8157190 2023-05-20T01:16:54.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.351+0000 7fd6e759e700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd6e8071ad0 0x7fd6e80ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:54.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.351+0000 7fd6e759e700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd6e8071ad0 0x7fd6e80ce740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58544/0 (socket says 172.21.15.16:58544) 2023-05-20T01:16:54.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.351+0000 7fd6e759e700 1 -- 172.21.15.16:0/2839400841 learned_addr learned my addr 172.21.15.16:0/2839400841 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:54.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.351+0000 7fd6e6d9d700 1 --2- 172.21.15.16:0/2839400841 >> v2:172.21.15.132:3300/0 conn(0x7fd6e8157190 0x7fd6e80cc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:54.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.352+0000 7fd6e759e700 1 -- 172.21.15.16:0/2839400841 >> v2:172.21.15.132:3300/0 conn(0x7fd6e8157190 msgr2=0x7fd6e80cc7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:54.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.352+0000 7fd6e759e700 1 --2- 172.21.15.16:0/2839400841 >> v2:172.21.15.132:3300/0 conn(0x7fd6e8157190 0x7fd6e80cc7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:54.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.352+0000 7fd6e759e700 1 -- 172.21.15.16:0/2839400841 >> v1:172.21.15.132:6789/0 conn(0x7fd6e80cccf0 legacy=0x7fd6e80713c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:54.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.352+0000 7fd6e759e700 1 -- 172.21.15.16:0/2839400841 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6e80cac60 con 0x7fd6e8071ad0 2023-05-20T01:16:54.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.352+0000 7fd6e6d9d700 1 --2- 172.21.15.16:0/2839400841 >> v2:172.21.15.132:3300/0 conn(0x7fd6e8157190 0x7fd6e80cc7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:16:54.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.352+0000 7fd6e759e700 1 --2- 172.21.15.16:0/2839400841 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6e8071ad0 0x7fd6e80ce740 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd6e8070c40 tx=0x7fd6e000d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=861bd4b701e77701 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:54.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.352+0000 7fd6e5d9b700 1 -- 172.21.15.16:0/2839400841 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6e0010070 con 0x7fd6e8071ad0 2023-05-20T01:16:54.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.352+0000 7fd6e5d9b700 1 -- 172.21.15.16:0/2839400841 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6e000dd90 con 0x7fd6e8071ad0 2023-05-20T01:16:54.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.352+0000 7fd6e5d9b700 1 -- 172.21.15.16:0/2839400841 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6e000b310 con 0x7fd6e8071ad0 2023-05-20T01:16:54.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.352+0000 7fd6ed047700 1 -- 172.21.15.16:0/2839400841 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6e8071ad0 msgr2=0x7fd6e80ce740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:54.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.352+0000 7fd6ed047700 1 --2- 172.21.15.16:0/2839400841 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6e8071ad0 0x7fd6e80ce740 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd6e8070c40 tx=0x7fd6e000d430 comp rx=0 tx=0).stop 2023-05-20T01:16:54.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.352+0000 7fd6ed047700 1 -- 172.21.15.16:0/2839400841 shutdown_connections 2023-05-20T01:16:54.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.352+0000 7fd6ed047700 1 --2- 172.21.15.16:0/2839400841 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6e8071ad0 0x7fd6e80ce740 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:54.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.352+0000 7fd6ed047700 1 --2- 172.21.15.16:0/2839400841 >> v2:172.21.15.132:3300/0 conn(0x7fd6e8157190 0x7fd6e80cc7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:54.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.352+0000 7fd6ed047700 1 -- 172.21.15.16:0/2839400841 >> 172.21.15.16:0/2839400841 conn(0x7fd6e80cffb0 msgr2=0x7fd6e80d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:54.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.352+0000 7fd6ed047700 1 -- 172.21.15.16:0/2839400841 shutdown_connections 2023-05-20T01:16:54.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.353+0000 7fd6ed047700 1 -- 172.21.15.16:0/2839400841 wait complete. 2023-05-20T01:16:54.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.354+0000 7fd6ed047700 1 Processor -- start 2023-05-20T01:16:54.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.354+0000 7fd6ed047700 1 -- start start 2023-05-20T01:16:54.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.355+0000 7fd6ed047700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6e8071ad0 0x7fd6e80b4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:54.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.355+0000 7fd6ed047700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd6e80cccf0 0x7fd6e8140fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:54.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.355+0000 7fd6ed047700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd6e8157190 0x7fd6e80b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:54.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.355+0000 7fd6ed047700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e8143ff0 con 0x7fd6e8071ad0 2023-05-20T01:16:54.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.355+0000 7fd6ed047700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e80afbb0 con 0x7fd6e8157190 2023-05-20T01:16:54.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.355+0000 7fd6ed047700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e80afe90 con 0x7fd6e80cccf0 2023-05-20T01:16:54.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.355+0000 7fd6e6d9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6e8071ad0 0x7fd6e80b4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:54.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.355+0000 7fd6e6d9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6e8071ad0 0x7fd6e80b4100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58546/0 (socket says 172.21.15.16:58546) 2023-05-20T01:16:54.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.355+0000 7fd6e6d9d700 1 -- 172.21.15.16:0/4231774324 learned_addr learned my addr 172.21.15.16:0/4231774324 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:54.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.355+0000 7fd6e659c700 1 --2- 172.21.15.16:0/4231774324 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd6e80cccf0 0x7fd6e8140fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:54.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.355+0000 7fd6e759e700 1 --2- 172.21.15.16:0/4231774324 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd6e8157190 0x7fd6e80b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:54.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.355+0000 7fd6e6d9d700 1 -- 172.21.15.16:0/4231774324 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd6e80cccf0 msgr2=0x7fd6e8140fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:54.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.355+0000 7fd6e6d9d700 1 --2- 172.21.15.16:0/4231774324 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd6e80cccf0 0x7fd6e8140fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:54.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.355+0000 7fd6e6d9d700 1 -- 172.21.15.16:0/4231774324 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd6e8157190 msgr2=0x7fd6e80b4eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:54.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.355+0000 7fd6e6d9d700 1 --2- 172.21.15.16:0/4231774324 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd6e8157190 0x7fd6e80b4eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:54.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.355+0000 7fd6e6d9d700 1 -- 172.21.15.16:0/4231774324 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6e000bd40 con 0x7fd6e8071ad0 2023-05-20T01:16:54.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.355+0000 7fd6e659c700 1 --2- 172.21.15.16:0/4231774324 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd6e80cccf0 0x7fd6e8140fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:16:54.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.356+0000 7fd6e6d9d700 1 --2- 172.21.15.16:0/4231774324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6e8071ad0 0x7fd6e80b4100 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd6dc00e760 tx=0x7fd6dc00ea70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:54.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.356+0000 7fd6cf7fe700 1 -- 172.21.15.16:0/4231774324 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6dc013610 con 0x7fd6e8071ad0 2023-05-20T01:16:54.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.356+0000 7fd6cf7fe700 1 -- 172.21.15.16:0/4231774324 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6dc013770 con 0x7fd6e8071ad0 2023-05-20T01:16:54.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.356+0000 7fd6ed047700 1 -- 172.21.15.16:0/4231774324 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6e80cb220 con 0x7fd6e8071ad0 2023-05-20T01:16:54.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.356+0000 7fd6cf7fe700 1 -- 172.21.15.16:0/4231774324 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6dc013b20 con 0x7fd6e8071ad0 2023-05-20T01:16:54.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.356+0000 7fd6ed047700 1 -- 172.21.15.16:0/4231774324 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6e80caf40 con 0x7fd6e8071ad0 2023-05-20T01:16:54.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.357+0000 7fd6ed047700 1 -- 172.21.15.16:0/4231774324 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6e8157560 con 0x7fd6e8071ad0 2023-05-20T01:16:54.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.357+0000 7fd6cf7fe700 1 -- 172.21.15.16:0/4231774324 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd6dc013c80 con 0x7fd6e8071ad0 2023-05-20T01:16:54.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.357+0000 7fd6cf7fe700 1 --2- 172.21.15.16:0/4231774324 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd6d406ace0 0x7fd6d406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:54.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.358+0000 7fd6cf7fe700 1 -- 172.21.15.16:0/4231774324 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fd6dc089280 con 0x7fd6e8071ad0 2023-05-20T01:16:54.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.358+0000 7fd6e659c700 1 --2- 172.21.15.16:0/4231774324 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd6d406ace0 0x7fd6d406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:54.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.358+0000 7fd6e659c700 1 --2- 172.21.15.16:0/4231774324 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd6d406ace0 0x7fd6d406d190 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd6e8140eb0 tx=0x7fd6d8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:54.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.362+0000 7fd6cf7fe700 1 -- 172.21.15.16:0/4231774324 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fd6dc057b60 con 0x7fd6e8071ad0 2023-05-20T01:16:54.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.605+0000 7fd6ed047700 1 -- 172.21.15.16:0/4231774324 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd6e80747b0 con 0x7fd6e8071ad0 2023-05-20T01:16:54.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.605+0000 7fd6cf7fe700 1 -- 172.21.15.16:0/4231774324 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4673 (secure 0 0 0) 0x7fd6dc0576f0 con 0x7fd6e8071ad0 2023-05-20T01:16:54.609 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:16:54.609 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4273,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/1444124146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1444124146},{"type":"v1","addr":"172.21.15.132:6834","nonce":1444124146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4293,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4092119206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4092119206},{"type":"v1","addr":"172.21.15.16:6835","nonce":4092119206}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2162680372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":2162680372},{"type":"v1","addr":"172.21.15.132:6835","nonce":2162680372}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-20T01:16:28.316109+0000","modified":"2023-05-20T01:16:45.362572+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4299},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4299":{"gid":4299,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6837/2330551626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2330551626},{"type":"v1","addr":"172.21.15.16:6837","nonce":2330551626}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-05-20T01:16:54.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.607+0000 7fd6ed047700 1 -- 172.21.15.16:0/4231774324 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd6d406ace0 msgr2=0x7fd6d406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:54.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.607+0000 7fd6ed047700 1 --2- 172.21.15.16:0/4231774324 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd6d406ace0 0x7fd6d406d190 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd6e8140eb0 tx=0x7fd6d8009040 comp rx=0 tx=0).stop 2023-05-20T01:16:54.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.607+0000 7fd6ed047700 1 -- 172.21.15.16:0/4231774324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6e8071ad0 msgr2=0x7fd6e80b4100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:54.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.607+0000 7fd6ed047700 1 --2- 172.21.15.16:0/4231774324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6e8071ad0 0x7fd6e80b4100 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd6dc00e760 tx=0x7fd6dc00ea70 comp rx=0 tx=0).stop 2023-05-20T01:16:54.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.607+0000 7fd6ed047700 1 -- 172.21.15.16:0/4231774324 shutdown_connections 2023-05-20T01:16:54.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.607+0000 7fd6ed047700 1 --2- 172.21.15.16:0/4231774324 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd6e8157190 0x7fd6e80b4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:54.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.607+0000 7fd6ed047700 1 --2- 172.21.15.16:0/4231774324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6e8071ad0 0x7fd6e80b4100 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:54.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.607+0000 7fd6ed047700 1 --2- 172.21.15.16:0/4231774324 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd6d406ace0 0x7fd6d406d190 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:54.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.607+0000 7fd6ed047700 1 --2- 172.21.15.16:0/4231774324 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd6e80cccf0 0x7fd6e8140fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:54.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.608+0000 7fd6ed047700 1 -- 172.21.15.16:0/4231774324 >> 172.21.15.16:0/4231774324 conn(0x7fd6e80cffb0 msgr2=0x7fd6e8156560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:54.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.608+0000 7fd6ed047700 1 -- 172.21.15.16:0/4231774324 shutdown_connections 2023-05-20T01:16:54.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.608+0000 7fd6ed047700 1 -- 172.21.15.16:0/4231774324 wait complete. 2023-05-20T01:16:54.613 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 8 2023-05-20T01:16:54.624 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-20T01:16:54.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.746+0000 7feecf436700 1 Processor -- start 2023-05-20T01:16:54.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.746+0000 7feecf436700 1 -- start start 2023-05-20T01:16:54.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.747+0000 7feecf436700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7feec806f450 0x7feec80cf890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:54.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.747+0000 7feecf436700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7feec806d4b0 0x7feec8161080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:54.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.747+0000 7feecf436700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feec80cc510 con 0x7feec806a510 2023-05-20T01:16:54.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.747+0000 7feecf436700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feec81615c0 con 0x7feec806f450 2023-05-20T01:16:54.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.747+0000 7feecf436700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feec8071ed0 con 0x7feec806d4b0 2023-05-20T01:16:54.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.747+0000 7feecd1d2700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7feec806f450 0x7feec80cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:54.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.747+0000 7feecd1d2700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7feec806f450 0x7feec80cf890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58554/0 (socket says 172.21.15.16:58554) 2023-05-20T01:16:54.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.747+0000 7feecd1d2700 1 -- 172.21.15.16:0/3164870895 learned_addr learned my addr 172.21.15.16:0/3164870895 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:54.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.747+0000 7feecc9d1700 1 --2- 172.21.15.16:0/3164870895 >> v2:172.21.15.132:3300/0 conn(0x7feec806d4b0 0x7feec8161080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:54.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.747+0000 7feecd1d2700 1 -- 172.21.15.16:0/3164870895 >> v2:172.21.15.132:3300/0 conn(0x7feec806d4b0 msgr2=0x7feec8161080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:54.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.747+0000 7feecd1d2700 1 --2- 172.21.15.16:0/3164870895 >> v2:172.21.15.132:3300/0 conn(0x7feec806d4b0 0x7feec8161080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:54.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.747+0000 7feecd1d2700 1 -- 172.21.15.16:0/3164870895 >> v1:172.21.15.132:6789/0 conn(0x7feec806a510 legacy=0x7feec80cdb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:54.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.747+0000 7feecd1d2700 1 -- 172.21.15.16:0/3164870895 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feec80721b0 con 0x7feec806f450 2023-05-20T01:16:54.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.747+0000 7feecc9d1700 1 --2- 172.21.15.16:0/3164870895 >> v2:172.21.15.132:3300/0 conn(0x7feec806d4b0 0x7feec8161080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:16:54.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.747+0000 7feecd1d2700 1 --2- 172.21.15.16:0/3164870895 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feec806f450 0x7feec80cf890 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7feec8070c40 tx=0x7feec4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=447329f137c1bf79 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:54.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.747+0000 7feebffff700 1 -- 172.21.15.16:0/3164870895 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec400b870 con 0x7feec806f450 2023-05-20T01:16:54.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.747+0000 7feebffff700 1 -- 172.21.15.16:0/3164870895 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feec400b9d0 con 0x7feec806f450 2023-05-20T01:16:54.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.747+0000 7feebffff700 1 -- 172.21.15.16:0/3164870895 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec400bd60 con 0x7feec806f450 2023-05-20T01:16:54.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.748+0000 7feecf436700 1 -- 172.21.15.16:0/3164870895 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feec806f450 msgr2=0x7feec80cf890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:54.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.748+0000 7feecf436700 1 --2- 172.21.15.16:0/3164870895 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feec806f450 0x7feec80cf890 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7feec8070c40 tx=0x7feec4009bd0 comp rx=0 tx=0).stop 2023-05-20T01:16:54.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.748+0000 7feecf436700 1 -- 172.21.15.16:0/3164870895 shutdown_connections 2023-05-20T01:16:54.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.748+0000 7feecf436700 1 --2- 172.21.15.16:0/3164870895 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feec806f450 0x7feec80cf890 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:54.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.748+0000 7feecf436700 1 --2- 172.21.15.16:0/3164870895 >> v2:172.21.15.132:3300/0 conn(0x7feec806d4b0 0x7feec8161080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:54.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.748+0000 7feecf436700 1 -- 172.21.15.16:0/3164870895 >> 172.21.15.16:0/3164870895 conn(0x7feec80cffb0 msgr2=0x7feec80d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:54.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.748+0000 7feecf436700 1 -- 172.21.15.16:0/3164870895 shutdown_connections 2023-05-20T01:16:54.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.748+0000 7feecf436700 1 -- 172.21.15.16:0/3164870895 wait complete. 2023-05-20T01:16:54.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.749+0000 7feecf436700 1 Processor -- start 2023-05-20T01:16:54.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.749+0000 7feecf436700 1 -- start start 2023-05-20T01:16:54.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.750+0000 7feecf436700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7feec806a510 0x7feec8155e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:54.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.750+0000 7feecf436700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feec806d4b0 0x7feec8153ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:54.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.750+0000 7feecf436700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feec806f450 0x7feec80c5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:54.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.750+0000 7feecf436700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feec81551e0 con 0x7feec806f450 2023-05-20T01:16:54.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.750+0000 7feecf436700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feec81546d0 con 0x7feec806d4b0 2023-05-20T01:16:54.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.750+0000 7feecf436700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feec80c5940 con 0x7feec806a510 2023-05-20T01:16:54.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.750+0000 7feecd9d3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feec806f450 0x7feec80c5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:54.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.750+0000 7feecc9d1700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feec806d4b0 0x7feec8153ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:54.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.750+0000 7feecd9d3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feec806f450 0x7feec80c5dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58564/0 (socket says 172.21.15.16:58564) 2023-05-20T01:16:54.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.750+0000 7feecd9d3700 1 -- 172.21.15.16:0/2266330143 learned_addr learned my addr 172.21.15.16:0/2266330143 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:54.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.750+0000 7feecd1d2700 1 --2- 172.21.15.16:0/2266330143 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7feec806a510 0x7feec8155e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:54.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.750+0000 7feecd9d3700 1 -- 172.21.15.16:0/2266330143 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7feec806a510 msgr2=0x7feec8155e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:54.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.750+0000 7feecd9d3700 1 --2- 172.21.15.16:0/2266330143 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7feec806a510 0x7feec8155e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:54.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.750+0000 7feecd9d3700 1 -- 172.21.15.16:0/2266330143 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feec806d4b0 msgr2=0x7feec8153ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:54.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.750+0000 7feecd9d3700 1 --2- 172.21.15.16:0/2266330143 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feec806d4b0 0x7feec8153ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:54.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.751+0000 7feecd9d3700 1 -- 172.21.15.16:0/2266330143 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feec4009550 con 0x7feec806f450 2023-05-20T01:16:54.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.751+0000 7feecc9d1700 1 --2- 172.21.15.16:0/2266330143 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feec806d4b0 0x7feec8153ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-05-20T01:16:54.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.751+0000 7feecd9d3700 1 --2- 172.21.15.16:0/2266330143 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feec806f450 0x7feec80c5dd0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7feec000cc80 tx=0x7feec000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:54.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.751+0000 7feebdffb700 1 -- 172.21.15.16:0/2266330143 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec0003c00 con 0x7feec806f450 2023-05-20T01:16:54.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.751+0000 7feebdffb700 1 -- 172.21.15.16:0/2266330143 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feec0003d60 con 0x7feec806f450 2023-05-20T01:16:54.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.751+0000 7feecf436700 1 -- 172.21.15.16:0/2266330143 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feec80cbe50 con 0x7feec806f450 2023-05-20T01:16:54.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.751+0000 7feebdffb700 1 -- 172.21.15.16:0/2266330143 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec000aaa0 con 0x7feec806f450 2023-05-20T01:16:54.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.751+0000 7feecf436700 1 -- 172.21.15.16:0/2266330143 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feec80cc1a0 con 0x7feec806f450 2023-05-20T01:16:54.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.752+0000 7feecf436700 1 -- 172.21.15.16:0/2266330143 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feec806a950 con 0x7feec806f450 2023-05-20T01:16:54.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.752+0000 7feebdffb700 1 -- 172.21.15.16:0/2266330143 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7feec000ac00 con 0x7feec806f450 2023-05-20T01:16:54.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.753+0000 7feebdffb700 1 --2- 172.21.15.16:0/2266330143 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7feeac06ace0 0x7feeac06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:54.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.753+0000 7feebdffb700 1 -- 172.21.15.16:0/2266330143 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7feec00896e0 con 0x7feec806f450 2023-05-20T01:16:54.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.753+0000 7feecd1d2700 1 --2- 172.21.15.16:0/2266330143 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7feeac06ace0 0x7feeac06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:54.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.753+0000 7feecd1d2700 1 --2- 172.21.15.16:0/2266330143 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7feeac06ace0 0x7feeac06d190 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7feec4014040 tx=0x7feec4008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:54.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.757+0000 7feebdffb700 1 -- 172.21.15.16:0/2266330143 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7feec00544a0 con 0x7feec806f450 2023-05-20T01:16:54.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.955+0000 7feecf436700 1 -- 172.21.15.16:0/2266330143 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feec80747b0 con 0x7feec806f450 2023-05-20T01:16:54.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.956+0000 7feebdffb700 1 -- 172.21.15.16:0/2266330143 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+20026 (secure 0 0 0) 0x7feec0057ac0 con 0x7feec806f450 2023-05-20T01:16:54.960 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:16:54.960 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"fsid":"c2245cc5-ecab-4087-a59b-bc81db901676","created":"2023-05-20T01:16:09.549230+0000","modified":"2023-05-20T01:16:32.768495+0000","last_up_change":"2023-05-20T01:16:19.788624+0000","last_in_change":"2023-05-20T01:16:12.092160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-05-20T01:16:20.934366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-05-20T01:16:21.372294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-05-20T01:16:26.208287+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-05-20T01:16:27.292268+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-05-20T01:16:30.357285+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"317a83cd-19c8-4958-a301-6087ef342289","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6825","nonce":3923634179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6827","nonce":3923634179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6831","nonce":3923634179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6829","nonce":3923634179}]},"public_addr":"172.21.15.16:6825/3923634179","cluster_addr":"172.21.15.16:6827/3923634179","heartbeat_back_addr":"172.21.15.16:6831/3923634179","heartbeat_front_addr":"172.21.15.16:6829/3923634179","state":["exists","up"]},{"osd":1,"uuid":"5581046f-8219-4089-8d7d-b7846399505e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6809","nonce":3860264220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6811","nonce":3860264220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6815","nonce":3860264220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6813","nonce":3860264220}]},"public_addr":"172.21.15.16:6809/3860264220","cluster_addr":"172.21.15.16:6811/3860264220","heartbeat_back_addr":"172.21.15.16:6815/3860264220","heartbeat_front_addr":"172.21.15.16:6813/3860264220","state":["exists","up"]},{"osd":2,"uuid":"072e9e66-4133-478e-8d4a-64d46c2963f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6817","nonce":293187030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6819","nonce":293187030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6823","nonce":293187030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6821","nonce":293187030}]},"public_addr":"172.21.15.16:6817/293187030","cluster_addr":"172.21.15.16:6819/293187030","heartbeat_back_addr":"172.21.15.16:6823/293187030","heartbeat_front_addr":"172.21.15.16:6821/293187030","state":["exists","up"]},{"osd":3,"uuid":"11e1bba6-00f8-48dd-9028-d8d716b7f145","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6801","nonce":3669781890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6803","nonce":3669781890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6807","nonce":3669781890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6805","nonce":3669781890}]},"public_addr":"172.21.15.16:6801/3669781890","cluster_addr":"172.21.15.16:6803/3669781890","heartbeat_back_addr":"172.21.15.16:6807/3669781890","heartbeat_front_addr":"172.21.15.16:6805/3669781890","state":["exists","up"]},{"osd":4,"uuid":"b8909c3a-cf70-46cb-8d0b-009c707cf9d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6819","nonce":506317223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6823","nonce":506317223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6829","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6831","nonce":506317223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6827","nonce":506317223}]},"public_addr":"172.21.15.132:6819/506317223","cluster_addr":"172.21.15.132:6823/506317223","heartbeat_back_addr":"172.21.15.132:6831/506317223","heartbeat_front_addr":"172.21.15.132:6827/506317223","state":["exists","up"]},{"osd":5,"uuid":"b1818304-4583-47bf-a6ec-0bfbb5166233","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6818","nonce":1844689995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6822","nonce":1844689995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6830","nonce":1844689995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6826","nonce":1844689995}]},"public_addr":"172.21.15.132:6818/1844689995","cluster_addr":"172.21.15.132:6822/1844689995","heartbeat_back_addr":"172.21.15.132:6830/1844689995","heartbeat_front_addr":"172.21.15.132:6826/1844689995","state":["exists","up"]},{"osd":6,"uuid":"d37f4691-eed1-4a93-8818-18354f03f4a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6801","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6802","nonce":727581123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6805","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6807","nonce":727581123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6813","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6815","nonce":727581123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6811","nonce":727581123}]},"public_addr":"172.21.15.132:6802/727581123","cluster_addr":"172.21.15.132:6807/727581123","heartbeat_back_addr":"172.21.15.132:6815/727581123","heartbeat_front_addr":"172.21.15.132:6811/727581123","state":["exists","up"]},{"osd":7,"uuid":"4f712f62-5d1d-4816-8042-03209144f8b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6803","nonce":466001149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6806","nonce":466001149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6814","nonce":466001149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6810","nonce":466001149}]},"public_addr":"172.21.15.132:6803/466001149","cluster_addr":"172.21.15.132:6806/466001149","heartbeat_back_addr":"172.21.15.132:6814/466001149","heartbeat_front_addr":"172.21.15.132:6810/466001149","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.338863+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.278680+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305694+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.246835+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.276237+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.304441+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305421+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.267050+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-20T01:16:54.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.958+0000 7feecf436700 1 -- 172.21.15.16:0/2266330143 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7feeac06ace0 msgr2=0x7feeac06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:54.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.958+0000 7feecf436700 1 --2- 172.21.15.16:0/2266330143 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7feeac06ace0 0x7feeac06d190 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7feec4014040 tx=0x7feec4008030 comp rx=0 tx=0).stop 2023-05-20T01:16:54.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.958+0000 7feecf436700 1 -- 172.21.15.16:0/2266330143 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feec806f450 msgr2=0x7feec80c5dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:54.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.958+0000 7feecf436700 1 --2- 172.21.15.16:0/2266330143 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feec806f450 0x7feec80c5dd0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7feec000cc80 tx=0x7feec000cf90 comp rx=0 tx=0).stop 2023-05-20T01:16:54.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.958+0000 7feecf436700 1 -- 172.21.15.16:0/2266330143 shutdown_connections 2023-05-20T01:16:54.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.958+0000 7feecf436700 1 --2- 172.21.15.16:0/2266330143 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feec806f450 0x7feec80c5dd0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:54.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.958+0000 7feecf436700 1 --2- 172.21.15.16:0/2266330143 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7feeac06ace0 0x7feeac06d190 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:54.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.958+0000 7feecf436700 1 --2- 172.21.15.16:0/2266330143 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7feec806a510 0x7feec8155e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:54.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.958+0000 7feecf436700 1 --2- 172.21.15.16:0/2266330143 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feec806d4b0 0x7feec8153ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:54.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.958+0000 7feecf436700 1 -- 172.21.15.16:0/2266330143 >> 172.21.15.16:0/2266330143 conn(0x7feec80cffb0 msgr2=0x7feec815b360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:54.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.958+0000 7feecf436700 1 -- 172.21.15.16:0/2266330143 shutdown_connections 2023-05-20T01:16:54.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:54.958+0000 7feecf436700 1 -- 172.21.15.16:0/2266330143 wait complete. 2023-05-20T01:16:54.975 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-05-20T01:16:20.934366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-05-20T01:16:21.372294+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-05-20T01:16:26.208287+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-05-20T01:16:27.292268+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-05-20T01:16:30.357285+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-05-20T01:16:54.976 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-05-20T01:16:55.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.100+0000 7ff4763d9700 1 Processor -- start 2023-05-20T01:16:55.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.100+0000 7ff4763d9700 1 -- start start 2023-05-20T01:16:55.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.101+0000 7ff4763d9700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7ff470072300 0x7ff4700cf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:55.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.101+0000 7ff4763d9700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff47015d290 0x7ff4700cf800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:55.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.101+0000 7ff4763d9700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4700cd630 con 0x7ff47015cec0 2023-05-20T01:16:55.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.101+0000 7ff4763d9700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4700cca60 con 0x7ff47015d290 2023-05-20T01:16:55.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.101+0000 7ff4763d9700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4700ccd00 con 0x7ff470072300 2023-05-20T01:16:55.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.101+0000 7ff474976700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff47015d290 0x7ff4700cf800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:55.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.101+0000 7ff46ffff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ff47015cec0 0x7ff470071bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:41864/0 (socket says 172.21.15.16:41864) 2023-05-20T01:16:55.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.101+0000 7ff46ffff700 1 -- 172.21.15.16:0/4100153672 learned_addr learned my addr 172.21.15.16:0/4100153672 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:16:55.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.101+0000 7ff46f7fe700 1 --2- 172.21.15.16:0/4100153672 >> v2:172.21.15.132:3300/0 conn(0x7ff470072300 0x7ff4700cf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:55.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.102+0000 7ff474976700 1 -- 172.21.15.16:0/4100153672 >> v2:172.21.15.132:3300/0 conn(0x7ff470072300 msgr2=0x7ff4700cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:55.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.102+0000 7ff474976700 1 --2- 172.21.15.16:0/4100153672 >> v2:172.21.15.132:3300/0 conn(0x7ff470072300 0x7ff4700cf2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:55.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.102+0000 7ff474976700 1 -- 172.21.15.16:0/4100153672 >> v1:172.21.15.16:6789/0 conn(0x7ff47015cec0 legacy=0x7ff470071bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:55.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.102+0000 7ff46effd700 1 -- 172.21.15.16:0/4100153672 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1775214300 0 0) 0x7ff4700cd630 con 0x7ff47015cec0 2023-05-20T01:16:55.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.102+0000 7ff474976700 1 -- 172.21.15.16:0/4100153672 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4700ccfe0 con 0x7ff47015d290 2023-05-20T01:16:55.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.102+0000 7ff474976700 1 --2- 172.21.15.16:0/4100153672 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff47015d290 0x7ff4700cf800 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ff4700cc970 tx=0x7ff46800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=831b63cfea005a81 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:55.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.102+0000 7ff46effd700 1 -- 172.21.15.16:0/4100153672 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff46800e070 con 0x7ff47015d290 2023-05-20T01:16:55.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.102+0000 7ff46effd700 1 -- 172.21.15.16:0/4100153672 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff46800bb20 con 0x7ff47015d290 2023-05-20T01:16:55.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.102+0000 7ff46effd700 1 -- 172.21.15.16:0/4100153672 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff468012430 con 0x7ff47015d290 2023-05-20T01:16:55.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.103+0000 7ff4763d9700 1 -- 172.21.15.16:0/4100153672 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff47015d290 msgr2=0x7ff4700cf800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:55.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.103+0000 7ff4763d9700 1 --2- 172.21.15.16:0/4100153672 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff47015d290 0x7ff4700cf800 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ff4700cc970 tx=0x7ff46800b330 comp rx=0 tx=0).stop 2023-05-20T01:16:55.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.103+0000 7ff4763d9700 1 -- 172.21.15.16:0/4100153672 shutdown_connections 2023-05-20T01:16:55.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.103+0000 7ff4763d9700 1 --2- 172.21.15.16:0/4100153672 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff47015d290 0x7ff4700cf800 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:55.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.103+0000 7ff4763d9700 1 --2- 172.21.15.16:0/4100153672 >> v2:172.21.15.132:3300/0 conn(0x7ff470072300 0x7ff4700cf2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:55.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.103+0000 7ff4763d9700 1 -- 172.21.15.16:0/4100153672 >> 172.21.15.16:0/4100153672 conn(0x7ff4700cffd0 msgr2=0x7ff4700d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:55.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.103+0000 7ff4763d9700 1 -- 172.21.15.16:0/4100153672 shutdown_connections 2023-05-20T01:16:55.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.103+0000 7ff4763d9700 1 -- 172.21.15.16:0/4100153672 wait complete. 2023-05-20T01:16:55.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.104+0000 7ff4763d9700 1 Processor -- start 2023-05-20T01:16:55.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.104+0000 7ff4763d9700 1 -- start start 2023-05-20T01:16:55.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.105+0000 7ff4763d9700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff470072300 0x7ff470149370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:55.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.105+0000 7ff4763d9700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff47015cec0 0x7ff4700b0d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:55.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.105+0000 7ff4763d9700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff47015d290 0x7ff4700b1280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:55.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.105+0000 7ff4763d9700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4700b30c0 con 0x7ff470072300 2023-05-20T01:16:55.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.105+0000 7ff4763d9700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4700c9940 con 0x7ff47015cec0 2023-05-20T01:16:55.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.105+0000 7ff4763d9700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4700c9c20 con 0x7ff47015d290 2023-05-20T01:16:55.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.105+0000 7ff46ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff470072300 0x7ff470149370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:55.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.105+0000 7ff46ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff470072300 0x7ff470149370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58574/0 (socket says 172.21.15.16:58574) 2023-05-20T01:16:55.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.105+0000 7ff46ffff700 1 -- 172.21.15.16:0/2295767498 learned_addr learned my addr 172.21.15.16:0/2295767498 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:55.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.105+0000 7ff46f7fe700 1 --2- 172.21.15.16:0/2295767498 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff47015cec0 0x7ff4700b0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:55.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.105+0000 7ff474976700 1 --2- 172.21.15.16:0/2295767498 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff47015d290 0x7ff4700b1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:55.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.105+0000 7ff46ffff700 1 -- 172.21.15.16:0/2295767498 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff47015d290 msgr2=0x7ff4700b1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:55.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.105+0000 7ff46ffff700 1 --2- 172.21.15.16:0/2295767498 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff47015d290 0x7ff4700b1280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:55.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.105+0000 7ff46ffff700 1 -- 172.21.15.16:0/2295767498 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff47015cec0 msgr2=0x7ff4700b0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:55.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.105+0000 7ff46ffff700 1 --2- 172.21.15.16:0/2295767498 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff47015cec0 0x7ff4700b0d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:55.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.105+0000 7ff46ffff700 1 -- 172.21.15.16:0/2295767498 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff468009a70 con 0x7ff470072300 2023-05-20T01:16:55.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.105+0000 7ff46f7fe700 1 --2- 172.21.15.16:0/2295767498 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff47015cec0 0x7ff4700b0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:16:55.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.105+0000 7ff46ffff700 1 --2- 172.21.15.16:0/2295767498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff470072300 0x7ff470149370 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff4600010e0 tx=0x7ff46000cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:55.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.105+0000 7ff46cff9700 1 -- 172.21.15.16:0/2295767498 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4600074e0 con 0x7ff470072300 2023-05-20T01:16:55.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.106+0000 7ff46cff9700 1 -- 172.21.15.16:0/2295767498 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff460007640 con 0x7ff470072300 2023-05-20T01:16:55.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.106+0000 7ff4763d9700 1 -- 172.21.15.16:0/2295767498 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4700cfd40 con 0x7ff470072300 2023-05-20T01:16:55.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.106+0000 7ff46cff9700 1 -- 172.21.15.16:0/2295767498 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff460005a70 con 0x7ff470072300 2023-05-20T01:16:55.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.106+0000 7ff4763d9700 1 -- 172.21.15.16:0/2295767498 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4700cd2c0 con 0x7ff470072300 2023-05-20T01:16:55.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.107+0000 7ff4763d9700 1 -- 172.21.15.16:0/2295767498 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff470160870 con 0x7ff470072300 2023-05-20T01:16:55.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.107+0000 7ff46cff9700 1 -- 172.21.15.16:0/2295767498 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff460005bd0 con 0x7ff470072300 2023-05-20T01:16:55.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.107+0000 7ff46cff9700 1 --2- 172.21.15.16:0/2295767498 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff45006ace0 0x7ff45006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:55.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.108+0000 7ff46f7fe700 1 --2- 172.21.15.16:0/2295767498 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff45006ace0 0x7ff45006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:55.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.108+0000 7ff46cff9700 1 -- 172.21.15.16:0/2295767498 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7ff460088d20 con 0x7ff470072300 2023-05-20T01:16:55.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.108+0000 7ff46f7fe700 1 --2- 172.21.15.16:0/2295767498 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff45006ace0 0x7ff45006d190 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ff4701f60d0 tx=0x7ff458008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:55.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.112+0000 7ff46cff9700 1 -- 172.21.15.16:0/2295767498 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7ff460053af0 con 0x7ff470072300 2023-05-20T01:16:55.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.309+0000 7ff4763d9700 1 -- 172.21.15.16:0/2295767498 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ff4700b3200 con 0x7ff470072300 2023-05-20T01:16:55.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.310+0000 7ff46cff9700 1 -- 172.21.15.16:0/2295767498 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7ff460017020 con 0x7ff470072300 2023-05-20T01:16:55.314 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 8 2023-05-20T01:16:55.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.312+0000 7ff4763d9700 1 -- 172.21.15.16:0/2295767498 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff45006ace0 msgr2=0x7ff45006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:55.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.312+0000 7ff4763d9700 1 --2- 172.21.15.16:0/2295767498 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff45006ace0 0x7ff45006d190 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ff4701f60d0 tx=0x7ff458008040 comp rx=0 tx=0).stop 2023-05-20T01:16:55.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.312+0000 7ff4763d9700 1 -- 172.21.15.16:0/2295767498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff470072300 msgr2=0x7ff470149370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:55.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.312+0000 7ff4763d9700 1 --2- 172.21.15.16:0/2295767498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff470072300 0x7ff470149370 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff4600010e0 tx=0x7ff46000cae0 comp rx=0 tx=0).stop 2023-05-20T01:16:55.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.312+0000 7ff4763d9700 1 -- 172.21.15.16:0/2295767498 shutdown_connections 2023-05-20T01:16:55.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.312+0000 7ff4763d9700 1 --2- 172.21.15.16:0/2295767498 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff45006ace0 0x7ff45006d190 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:55.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.312+0000 7ff4763d9700 1 --2- 172.21.15.16:0/2295767498 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff47015d290 0x7ff4700b1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:55.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.312+0000 7ff4763d9700 1 --2- 172.21.15.16:0/2295767498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff470072300 0x7ff470149370 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:55.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.312+0000 7ff4763d9700 1 --2- 172.21.15.16:0/2295767498 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff47015cec0 0x7ff4700b0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:55.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.312+0000 7ff4763d9700 1 -- 172.21.15.16:0/2295767498 >> 172.21.15.16:0/2295767498 conn(0x7ff4700cffd0 msgr2=0x7ff47006a510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:55.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.312+0000 7ff4763d9700 1 -- 172.21.15.16:0/2295767498 shutdown_connections 2023-05-20T01:16:55.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.312+0000 7ff4763d9700 1 -- 172.21.15.16:0/2295767498 wait complete. 2023-05-20T01:16:55.328 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-05-20T01:16:55.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.453+0000 7fde8c21d700 1 Processor -- start 2023-05-20T01:16:55.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.454+0000 7fde8c21d700 1 -- start start 2023-05-20T01:16:55.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.454+0000 7fde8c21d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fde840cd1c0 0x7fde840710b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:55.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.454+0000 7fde8c21d700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fde840cd630 0x7fde840715f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:55.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.454+0000 7fde8c21d700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde840ccd30 con 0x7fde84159b50 2023-05-20T01:16:55.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.454+0000 7fde8c21d700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde840ce990 con 0x7fde840cd1c0 2023-05-20T01:16:55.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.454+0000 7fde8c21d700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde840cc6e0 con 0x7fde840cd630 2023-05-20T01:16:55.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.454+0000 7fde897b8700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fde840cd1c0 0x7fde840710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:55.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.455+0000 7fde897b8700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fde840cd1c0 0x7fde840710b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58584/0 (socket says 172.21.15.16:58584) 2023-05-20T01:16:55.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.455+0000 7fde89fb9700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fde84159b50 0x7fde8406e0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:41878/0 (socket says 172.21.15.16:41878) 2023-05-20T01:16:55.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.455+0000 7fde897b8700 1 -- 172.21.15.16:0/2596549973 learned_addr learned my addr 172.21.15.16:0/2596549973 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:55.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.455+0000 7fde8a7ba700 1 --2- 172.21.15.16:0/2596549973 >> v2:172.21.15.132:3300/0 conn(0x7fde840cd630 0x7fde840715f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:55.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.455+0000 7fde897b8700 1 -- 172.21.15.16:0/2596549973 >> v2:172.21.15.132:3300/0 conn(0x7fde840cd630 msgr2=0x7fde840715f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:55.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.455+0000 7fde897b8700 1 --2- 172.21.15.16:0/2596549973 >> v2:172.21.15.132:3300/0 conn(0x7fde840cd630 0x7fde840715f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:55.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.455+0000 7fde897b8700 1 -- 172.21.15.16:0/2596549973 >> v1:172.21.15.16:6789/0 conn(0x7fde84159b50 legacy=0x7fde8406e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:55.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.455+0000 7fde88fb7700 1 -- 172.21.15.16:0/2596549973 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 296297860 0 0) 0x7fde840ccd30 con 0x7fde84159b50 2023-05-20T01:16:55.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.455+0000 7fde897b8700 1 -- 172.21.15.16:0/2596549973 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde840cc9c0 con 0x7fde840cd1c0 2023-05-20T01:16:55.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.455+0000 7fde897b8700 1 --2- 172.21.15.16:0/2596549973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde840cd1c0 0x7fde840710b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fde8406c340 tx=0x7fde8000be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fea4de34f96131a8 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:55.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.455+0000 7fde88fb7700 1 -- 172.21.15.16:0/2596549973 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde8000dd30 con 0x7fde840cd1c0 2023-05-20T01:16:55.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.455+0000 7fde88fb7700 1 -- 172.21.15.16:0/2596549973 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde8000de90 con 0x7fde840cd1c0 2023-05-20T01:16:55.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.455+0000 7fde88fb7700 1 -- 172.21.15.16:0/2596549973 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde80010690 con 0x7fde840cd1c0 2023-05-20T01:16:55.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.456+0000 7fde8c21d700 1 -- 172.21.15.16:0/2596549973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde840cd1c0 msgr2=0x7fde840710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:55.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.456+0000 7fde8c21d700 1 --2- 172.21.15.16:0/2596549973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde840cd1c0 0x7fde840710b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fde8406c340 tx=0x7fde8000be70 comp rx=0 tx=0).stop 2023-05-20T01:16:55.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.456+0000 7fde8c21d700 1 -- 172.21.15.16:0/2596549973 shutdown_connections 2023-05-20T01:16:55.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.456+0000 7fde8c21d700 1 --2- 172.21.15.16:0/2596549973 >> v2:172.21.15.132:3300/0 conn(0x7fde840cd630 0x7fde840715f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:55.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.456+0000 7fde8c21d700 1 --2- 172.21.15.16:0/2596549973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde840cd1c0 0x7fde840710b0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:55.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.456+0000 7fde8c21d700 1 -- 172.21.15.16:0/2596549973 >> 172.21.15.16:0/2596549973 conn(0x7fde840cffd0 msgr2=0x7fde840d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:55.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.456+0000 7fde8c21d700 1 -- 172.21.15.16:0/2596549973 shutdown_connections 2023-05-20T01:16:55.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.456+0000 7fde8c21d700 1 -- 172.21.15.16:0/2596549973 wait complete. 2023-05-20T01:16:55.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.457+0000 7fde8c21d700 1 Processor -- start 2023-05-20T01:16:55.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.457+0000 7fde8c21d700 1 -- start start 2023-05-20T01:16:55.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.458+0000 7fde8c21d700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fde840cd1c0 0x7fde841f1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:55.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.458+0000 7fde8c21d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fde840cd630 0x7fde84157880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:55.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.458+0000 7fde8c21d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde84159b50 0x7fde84157dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:55.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.458+0000 7fde8c21d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde840c66b0 con 0x7fde84159b50 2023-05-20T01:16:55.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.458+0000 7fde8c21d700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde840c63d0 con 0x7fde840cd630 2023-05-20T01:16:55.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.458+0000 7fde8c21d700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde84156940 con 0x7fde840cd1c0 2023-05-20T01:16:55.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.458+0000 7fde8a7ba700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde84159b50 0x7fde84157dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:55.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.458+0000 7fde8a7ba700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde84159b50 0x7fde84157dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58592/0 (socket says 172.21.15.16:58592) 2023-05-20T01:16:55.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.458+0000 7fde8a7ba700 1 -- 172.21.15.16:0/2932808368 learned_addr learned my addr 172.21.15.16:0/2932808368 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:55.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.458+0000 7fde897b8700 1 --2- 172.21.15.16:0/2932808368 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fde840cd630 0x7fde84157880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:55.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.458+0000 7fde89fb9700 1 --2- 172.21.15.16:0/2932808368 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fde840cd1c0 0x7fde841f1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:55.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.458+0000 7fde8a7ba700 1 -- 172.21.15.16:0/2932808368 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fde840cd1c0 msgr2=0x7fde841f1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:55.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.458+0000 7fde8a7ba700 1 --2- 172.21.15.16:0/2932808368 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fde840cd1c0 0x7fde841f1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:55.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.458+0000 7fde8a7ba700 1 -- 172.21.15.16:0/2932808368 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fde840cd630 msgr2=0x7fde84157880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:55.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.458+0000 7fde8a7ba700 1 --2- 172.21.15.16:0/2932808368 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fde840cd630 0x7fde84157880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:55.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.458+0000 7fde8a7ba700 1 -- 172.21.15.16:0/2932808368 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde8000b7f0 con 0x7fde84159b50 2023-05-20T01:16:55.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.458+0000 7fde897b8700 1 --2- 172.21.15.16:0/2932808368 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fde840cd630 0x7fde84157880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:16:55.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.458+0000 7fde89fb9700 1 --2- 172.21.15.16:0/2932808368 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fde840cd1c0 0x7fde841f1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:16:55.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.458+0000 7fde8a7ba700 1 --2- 172.21.15.16:0/2932808368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde84159b50 0x7fde84157dc0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fde7c007e40 tx=0x7fde7c00e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:55.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.459+0000 7fde7a7fc700 1 -- 172.21.15.16:0/2932808368 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde7c00eea0 con 0x7fde84159b50 2023-05-20T01:16:55.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.459+0000 7fde7a7fc700 1 -- 172.21.15.16:0/2932808368 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde7c014420 con 0x7fde84159b50 2023-05-20T01:16:55.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.459+0000 7fde8c21d700 1 -- 172.21.15.16:0/2932808368 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde840cec70 con 0x7fde84159b50 2023-05-20T01:16:55.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.459+0000 7fde7a7fc700 1 -- 172.21.15.16:0/2932808368 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde7c0147b0 con 0x7fde84159b50 2023-05-20T01:16:55.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.459+0000 7fde8c21d700 1 -- 172.21.15.16:0/2932808368 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde84071e80 con 0x7fde84159b50 2023-05-20T01:16:55.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.459+0000 7fde8c21d700 1 -- 172.21.15.16:0/2932808368 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde840747b0 con 0x7fde84159b50 2023-05-20T01:16:55.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.460+0000 7fde7a7fc700 1 -- 172.21.15.16:0/2932808368 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fde7c014910 con 0x7fde84159b50 2023-05-20T01:16:55.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.460+0000 7fde7a7fc700 1 --2- 172.21.15.16:0/2932808368 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fde6806ace0 0x7fde6806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:55.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.460+0000 7fde7a7fc700 1 -- 172.21.15.16:0/2932808368 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fde7c089f30 con 0x7fde84159b50 2023-05-20T01:16:55.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.460+0000 7fde89fb9700 1 --2- 172.21.15.16:0/2932808368 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fde6806ace0 0x7fde6806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:55.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.461+0000 7fde89fb9700 1 --2- 172.21.15.16:0/2932808368 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fde6806ace0 0x7fde6806d190 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fde740017e0 tx=0x7fde74009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:55.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.465+0000 7fde7a7fc700 1 -- 172.21.15.16:0/2932808368 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fde7c054db0 con 0x7fde84159b50 2023-05-20T01:16:55.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.661+0000 7fde8c21d700 1 -- 172.21.15.16:0/2932808368 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fde8406e1c0 con 0x7fde84159b50 2023-05-20T01:16:55.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.661+0000 7fde7a7fc700 1 -- 172.21.15.16:0/2932808368 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7fde7c0583d0 con 0x7fde84159b50 2023-05-20T01:16:55.666 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-05-20T01:16:55.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.663+0000 7fde8c21d700 1 -- 172.21.15.16:0/2932808368 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fde6806ace0 msgr2=0x7fde6806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:55.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.663+0000 7fde8c21d700 1 --2- 172.21.15.16:0/2932808368 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fde6806ace0 0x7fde6806d190 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fde740017e0 tx=0x7fde74009040 comp rx=0 tx=0).stop 2023-05-20T01:16:55.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.663+0000 7fde8c21d700 1 -- 172.21.15.16:0/2932808368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde84159b50 msgr2=0x7fde84157dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:55.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.663+0000 7fde8c21d700 1 --2- 172.21.15.16:0/2932808368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde84159b50 0x7fde84157dc0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fde7c007e40 tx=0x7fde7c00e3c0 comp rx=0 tx=0).stop 2023-05-20T01:16:55.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.663+0000 7fde8c21d700 1 -- 172.21.15.16:0/2932808368 shutdown_connections 2023-05-20T01:16:55.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.663+0000 7fde8c21d700 1 --2- 172.21.15.16:0/2932808368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde84159b50 0x7fde84157dc0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:55.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.663+0000 7fde8c21d700 1 --2- 172.21.15.16:0/2932808368 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fde6806ace0 0x7fde6806d190 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:55.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.663+0000 7fde8c21d700 1 --2- 172.21.15.16:0/2932808368 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fde840cd1c0 0x7fde841f1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:55.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.663+0000 7fde8c21d700 1 --2- 172.21.15.16:0/2932808368 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fde840cd630 0x7fde84157880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:55.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.663+0000 7fde8c21d700 1 -- 172.21.15.16:0/2932808368 >> 172.21.15.16:0/2932808368 conn(0x7fde840cffd0 msgr2=0x7fde8406a510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:55.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.664+0000 7fde8c21d700 1 -- 172.21.15.16:0/2932808368 shutdown_connections 2023-05-20T01:16:55.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.664+0000 7fde8c21d700 1 -- 172.21.15.16:0/2932808368 wait complete. 2023-05-20T01:16:55.680 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-05-20T01:16:55.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.805+0000 7fc93d4e0700 1 Processor -- start 2023-05-20T01:16:55.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.805+0000 7fc93d4e0700 1 -- start start 2023-05-20T01:16:55.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.806+0000 7fc93d4e0700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc9380cc0a0 0x7fc9380cc520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:55.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.806+0000 7fc93d4e0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc938070950 0x7fc938070d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:55.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.806+0000 7fc93d4e0700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc93815e4b0 con 0x7fc93815a8d0 2023-05-20T01:16:55.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.806+0000 7fc93d4e0700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9380cca60 con 0x7fc9380cc0a0 2023-05-20T01:16:55.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.806+0000 7fc93d4e0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9380b21c0 con 0x7fc938070950 2023-05-20T01:16:55.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.806+0000 7fc9367fc700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc9380cc0a0 0x7fc9380cc520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:55.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.806+0000 7fc936ffd700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fc93815a8d0 0x7fc9380b1720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:41892/0 (socket says 172.21.15.16:41892) 2023-05-20T01:16:55.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.806+0000 7fc936ffd700 1 -- 172.21.15.16:0/3275276287 learned_addr learned my addr 172.21.15.16:0/3275276287 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:16:55.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.806+0000 7fc9377fe700 1 --2- 172.21.15.16:0/3275276287 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc938070950 0x7fc938070d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:55.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.806+0000 7fc9367fc700 1 -- 172.21.15.16:0/3275276287 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc938070950 msgr2=0x7fc938070d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:55.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.806+0000 7fc9367fc700 1 --2- 172.21.15.16:0/3275276287 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc938070950 0x7fc938070d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:55.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.806+0000 7fc9367fc700 1 -- 172.21.15.16:0/3275276287 >> v1:172.21.15.16:6789/0 conn(0x7fc93815a8d0 legacy=0x7fc9380b1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:55.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.806+0000 7fc9367fc700 1 -- 172.21.15.16:0/3275276287 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc93815e700 con 0x7fc9380cc0a0 2023-05-20T01:16:55.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.806+0000 7fc935ffb700 1 -- 172.21.15.16:0/3275276287 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 630037815 0 0) 0x7fc93815e4b0 con 0x7fc93815a8d0 2023-05-20T01:16:55.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.806+0000 7fc9377fe700 1 --2- 172.21.15.16:0/3275276287 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc938070950 0x7fc938070d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:16:55.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.807+0000 7fc9367fc700 1 --2- 172.21.15.16:0/3275276287 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9380cc0a0 0x7fc9380cc520 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc928009840 tx=0x7fc928009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b39cfe3fc17f03ee server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:55.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.807+0000 7fc935ffb700 1 -- 172.21.15.16:0/3275276287 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc928011830 con 0x7fc9380cc0a0 2023-05-20T01:16:55.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.807+0000 7fc935ffb700 1 -- 172.21.15.16:0/3275276287 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc928011990 con 0x7fc9380cc0a0 2023-05-20T01:16:55.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.807+0000 7fc935ffb700 1 -- 172.21.15.16:0/3275276287 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc928011d20 con 0x7fc9380cc0a0 2023-05-20T01:16:55.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.807+0000 7fc93d4e0700 1 -- 172.21.15.16:0/3275276287 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9380cc0a0 msgr2=0x7fc9380cc520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:55.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.807+0000 7fc93d4e0700 1 --2- 172.21.15.16:0/3275276287 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9380cc0a0 0x7fc9380cc520 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc928009840 tx=0x7fc928009b50 comp rx=0 tx=0).stop 2023-05-20T01:16:55.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.807+0000 7fc93d4e0700 1 -- 172.21.15.16:0/3275276287 shutdown_connections 2023-05-20T01:16:55.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.807+0000 7fc93d4e0700 1 --2- 172.21.15.16:0/3275276287 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc938070950 0x7fc938070d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:55.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.807+0000 7fc93d4e0700 1 --2- 172.21.15.16:0/3275276287 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9380cc0a0 0x7fc9380cc520 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:55.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.807+0000 7fc93d4e0700 1 -- 172.21.15.16:0/3275276287 >> 172.21.15.16:0/3275276287 conn(0x7fc93806cbd0 msgr2=0x7fc93806cfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:55.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.807+0000 7fc93d4e0700 1 -- 172.21.15.16:0/3275276287 shutdown_connections 2023-05-20T01:16:55.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.807+0000 7fc93d4e0700 1 -- 172.21.15.16:0/3275276287 wait complete. 2023-05-20T01:16:55.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.809+0000 7fc93d4e0700 1 Processor -- start 2023-05-20T01:16:55.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.809+0000 7fc93d4e0700 1 -- start start 2023-05-20T01:16:55.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.809+0000 7fc93d4e0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc938070950 0x7fc9381563b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:55.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.809+0000 7fc93d4e0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc9380cc0a0 0x7fc9380c6880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:55.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.809+0000 7fc93d4e0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc93815a8d0 0x7fc9380b38a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:55.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.809+0000 7fc93d4e0700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9381568f0 con 0x7fc938070950 2023-05-20T01:16:55.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.809+0000 7fc93d4e0700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9380cd380 con 0x7fc9380cc0a0 2023-05-20T01:16:55.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.809+0000 7fc93d4e0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9380c7320 con 0x7fc93815a8d0 2023-05-20T01:16:55.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.810+0000 7fc936ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc938070950 0x7fc9381563b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:55.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.810+0000 7fc936ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc938070950 0x7fc9381563b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58602/0 (socket says 172.21.15.16:58602) 2023-05-20T01:16:55.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.810+0000 7fc936ffd700 1 -- 172.21.15.16:0/201790087 learned_addr learned my addr 172.21.15.16:0/201790087 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:55.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.810+0000 7fc9367fc700 1 --2- 172.21.15.16:0/201790087 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc9380cc0a0 0x7fc9380c6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:55.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.810+0000 7fc9377fe700 1 --2- 172.21.15.16:0/201790087 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc93815a8d0 0x7fc9380b38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:55.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.810+0000 7fc936ffd700 1 -- 172.21.15.16:0/201790087 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc93815a8d0 msgr2=0x7fc9380b38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:55.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.810+0000 7fc936ffd700 1 --2- 172.21.15.16:0/201790087 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc93815a8d0 0x7fc9380b38a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:55.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.810+0000 7fc936ffd700 1 -- 172.21.15.16:0/201790087 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc9380cc0a0 msgr2=0x7fc9380c6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:55.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.810+0000 7fc936ffd700 1 --2- 172.21.15.16:0/201790087 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc9380cc0a0 0x7fc9380c6880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:55.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.810+0000 7fc936ffd700 1 -- 172.21.15.16:0/201790087 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9280094a0 con 0x7fc938070950 2023-05-20T01:16:55.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.810+0000 7fc9367fc700 1 --2- 172.21.15.16:0/201790087 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc9380cc0a0 0x7fc9380c6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:16:55.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.810+0000 7fc936ffd700 1 --2- 172.21.15.16:0/201790087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc938070950 0x7fc9381563b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc92c00c550 tx=0x7fc92c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:55.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.810+0000 7fc91f7fe700 1 -- 172.21.15.16:0/201790087 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc92c004c80 con 0x7fc938070950 2023-05-20T01:16:55.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.810+0000 7fc91f7fe700 1 -- 172.21.15.16:0/201790087 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc92c004e00 con 0x7fc938070950 2023-05-20T01:16:55.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.810+0000 7fc93d4e0700 1 -- 172.21.15.16:0/201790087 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc93815ecc0 con 0x7fc938070950 2023-05-20T01:16:55.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.810+0000 7fc91f7fe700 1 -- 172.21.15.16:0/201790087 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc92c007310 con 0x7fc938070950 2023-05-20T01:16:55.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.810+0000 7fc93d4e0700 1 -- 172.21.15.16:0/201790087 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc93815e9e0 con 0x7fc938070950 2023-05-20T01:16:55.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.811+0000 7fc93d4e0700 1 -- 172.21.15.16:0/201790087 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9380cab40 con 0x7fc938070950 2023-05-20T01:16:55.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.812+0000 7fc91f7fe700 1 -- 172.21.15.16:0/201790087 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc92c007470 con 0x7fc938070950 2023-05-20T01:16:55.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.812+0000 7fc91f7fe700 1 --2- 172.21.15.16:0/201790087 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc92406ace0 0x7fc92406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:55.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.812+0000 7fc91f7fe700 1 -- 172.21.15.16:0/201790087 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fc92c0893f0 con 0x7fc938070950 2023-05-20T01:16:55.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.812+0000 7fc9367fc700 1 --2- 172.21.15.16:0/201790087 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc92406ace0 0x7fc92406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:55.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.813+0000 7fc9367fc700 1 --2- 172.21.15.16:0/201790087 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc92406ace0 0x7fc92406d190 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc9280090b0 tx=0x7fc928008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:55.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:55.816+0000 7fc91f7fe700 1 -- 172.21.15.16:0/201790087 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fc92c054240 con 0x7fc938070950 2023-05-20T01:16:56.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.014+0000 7fc93d4e0700 1 -- 172.21.15.16:0/201790087 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc9380747b0 con 0x7fc938070950 2023-05-20T01:16:56.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.015+0000 7fc91f7fe700 1 -- 172.21.15.16:0/201790087 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc92c057860 con 0x7fc938070950 2023-05-20T01:16:56.019 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-05-20T01:16:56.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.017+0000 7fc93d4e0700 1 -- 172.21.15.16:0/201790087 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc92406ace0 msgr2=0x7fc92406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:56.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.017+0000 7fc93d4e0700 1 --2- 172.21.15.16:0/201790087 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc92406ace0 0x7fc92406d190 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc9280090b0 tx=0x7fc928008030 comp rx=0 tx=0).stop 2023-05-20T01:16:56.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.017+0000 7fc93d4e0700 1 -- 172.21.15.16:0/201790087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc938070950 msgr2=0x7fc9381563b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:56.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.017+0000 7fc93d4e0700 1 --2- 172.21.15.16:0/201790087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc938070950 0x7fc9381563b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc92c00c550 tx=0x7fc92c00c910 comp rx=0 tx=0).stop 2023-05-20T01:16:56.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.017+0000 7fc93d4e0700 1 -- 172.21.15.16:0/201790087 shutdown_connections 2023-05-20T01:16:56.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.017+0000 7fc93d4e0700 1 --2- 172.21.15.16:0/201790087 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc92406ace0 0x7fc92406d190 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:56.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.017+0000 7fc93d4e0700 1 --2- 172.21.15.16:0/201790087 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc93815a8d0 0x7fc9380b38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:56.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.017+0000 7fc93d4e0700 1 --2- 172.21.15.16:0/201790087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc938070950 0x7fc9381563b0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:56.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.017+0000 7fc93d4e0700 1 --2- 172.21.15.16:0/201790087 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc9380cc0a0 0x7fc9380c6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:56.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.017+0000 7fc93d4e0700 1 -- 172.21.15.16:0/201790087 >> 172.21.15.16:0/201790087 conn(0x7fc93806cbd0 msgr2=0x7fc9380c4890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:56.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.017+0000 7fc93d4e0700 1 -- 172.21.15.16:0/201790087 shutdown_connections 2023-05-20T01:16:56.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.017+0000 7fc93d4e0700 1 -- 172.21.15.16:0/201790087 wait complete. 2023-05-20T01:16:56.033 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-05-20T01:16:56.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.157+0000 7fc1eb79c700 1 Processor -- start 2023-05-20T01:16:56.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.158+0000 7fc1eb79c700 1 -- start start 2023-05-20T01:16:56.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.158+0000 7fc1eb79c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc1e40cd400 0x7fc1e40ce1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:56.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.158+0000 7fc1eb79c700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1e40cb6c0 con 0x7fc1e4146710 2023-05-20T01:16:56.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.158+0000 7fc1eb79c700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1e40ce840 con 0x7fc1e406bd30 2023-05-20T01:16:56.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.158+0000 7fc1eb79c700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1e40cb910 con 0x7fc1e40cd400 2023-05-20T01:16:56.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.159+0000 7fc1e9d39700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc1e40cd400 0x7fc1e40ce1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:56.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.159+0000 7fc1e9538700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fc1e4146710 0x7fc1e406b620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:41906/0 (socket says 172.21.15.16:41906) 2023-05-20T01:16:56.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.159+0000 7fc1e9538700 1 -- 172.21.15.16:0/1938420319 learned_addr learned my addr 172.21.15.16:0/1938420319 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:16:56.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.159+0000 7fc1e9d39700 1 -- 172.21.15.16:0/1938420319 >> v1:172.21.15.132:6789/0 conn(0x7fc1e406bd30 legacy=0x7fc1e40cdad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:56.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.159+0000 7fc1e9d39700 1 -- 172.21.15.16:0/1938420319 >> v1:172.21.15.16:6789/0 conn(0x7fc1e4146710 legacy=0x7fc1e406b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:56.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.159+0000 7fc1e9d39700 1 -- 172.21.15.16:0/1938420319 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1e40cbbf0 con 0x7fc1e40cd400 2023-05-20T01:16:56.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.159+0000 7fc1dbfff700 1 -- 172.21.15.16:0/1938420319 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4115290789 0 0) 0x7fc1e40cb6c0 con 0x7fc1e4146710 2023-05-20T01:16:56.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.159+0000 7fc1e9d39700 1 --2- 172.21.15.16:0/1938420319 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1e40cd400 0x7fc1e40ce1e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc1dc009d90 tx=0x7fc1dc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffd9b0955a4d644a server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:56.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.159+0000 7fc1dbfff700 1 -- 172.21.15.16:0/1938420319 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1dc00e070 con 0x7fc1e40cd400 2023-05-20T01:16:56.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.159+0000 7fc1dbfff700 1 -- 172.21.15.16:0/1938420319 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1dc00bbd0 con 0x7fc1e40cd400 2023-05-20T01:16:56.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.159+0000 7fc1dbfff700 1 -- 172.21.15.16:0/1938420319 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1dc012430 con 0x7fc1e40cd400 2023-05-20T01:16:56.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.160+0000 7fc1eb79c700 1 -- 172.21.15.16:0/1938420319 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1e40cd400 msgr2=0x7fc1e40ce1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:56.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.160+0000 7fc1eb79c700 1 --2- 172.21.15.16:0/1938420319 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1e40cd400 0x7fc1e40ce1e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc1dc009d90 tx=0x7fc1dc00b330 comp rx=0 tx=0).stop 2023-05-20T01:16:56.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.160+0000 7fc1eb79c700 1 -- 172.21.15.16:0/1938420319 shutdown_connections 2023-05-20T01:16:56.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.160+0000 7fc1eb79c700 1 --2- 172.21.15.16:0/1938420319 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1e40cd400 0x7fc1e40ce1e0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:56.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.160+0000 7fc1eb79c700 1 -- 172.21.15.16:0/1938420319 >> 172.21.15.16:0/1938420319 conn(0x7fc1e406ae30 msgr2=0x7fc1e409a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:56.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.160+0000 7fc1eb79c700 1 -- 172.21.15.16:0/1938420319 shutdown_connections 2023-05-20T01:16:56.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.160+0000 7fc1eb79c700 1 -- 172.21.15.16:0/1938420319 wait complete. 2023-05-20T01:16:56.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.161+0000 7fc1eb79c700 1 Processor -- start 2023-05-20T01:16:56.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.161+0000 7fc1eb79c700 1 -- start start 2023-05-20T01:16:56.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.162+0000 7fc1eb79c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc1e406bd30 0x7fc1e40b34a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:56.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.162+0000 7fc1eb79c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc1e4146710 0x7fc1e4154700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:56.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.162+0000 7fc1eb79c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1e415bb40 0x7fc1e40cd9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:56.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.162+0000 7fc1eb79c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1e40b39e0 con 0x7fc1e415bb40 2023-05-20T01:16:56.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.162+0000 7fc1eb79c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1e415c040 con 0x7fc1e4146710 2023-05-20T01:16:56.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.162+0000 7fc1eb79c700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1e415c320 con 0x7fc1e406bd30 2023-05-20T01:16:56.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.162+0000 7fc1e9d39700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1e415bb40 0x7fc1e40cd9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:56.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.162+0000 7fc1e9d39700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1e415bb40 0x7fc1e40cd9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58620/0 (socket says 172.21.15.16:58620) 2023-05-20T01:16:56.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.162+0000 7fc1e9d39700 1 -- 172.21.15.16:0/1375258097 learned_addr learned my addr 172.21.15.16:0/1375258097 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:56.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.162+0000 7fc1e8d37700 1 --2- 172.21.15.16:0/1375258097 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc1e4146710 0x7fc1e4154700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:56.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.162+0000 7fc1e9538700 1 --2- 172.21.15.16:0/1375258097 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc1e406bd30 0x7fc1e40b34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:56.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.162+0000 7fc1e9d39700 1 -- 172.21.15.16:0/1375258097 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc1e406bd30 msgr2=0x7fc1e40b34a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:56.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.162+0000 7fc1e9d39700 1 --2- 172.21.15.16:0/1375258097 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc1e406bd30 0x7fc1e40b34a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:56.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.162+0000 7fc1e9d39700 1 -- 172.21.15.16:0/1375258097 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc1e4146710 msgr2=0x7fc1e4154700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:56.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.162+0000 7fc1e9d39700 1 --2- 172.21.15.16:0/1375258097 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc1e4146710 0x7fc1e4154700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:56.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.162+0000 7fc1e9d39700 1 -- 172.21.15.16:0/1375258097 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1dc009a70 con 0x7fc1e415bb40 2023-05-20T01:16:56.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.162+0000 7fc1e8d37700 1 --2- 172.21.15.16:0/1375258097 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc1e4146710 0x7fc1e4154700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:16:56.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.162+0000 7fc1e9d39700 1 --2- 172.21.15.16:0/1375258097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1e415bb40 0x7fc1e40cd9f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc1dc014040 tx=0x7fc1dc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:56.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.163+0000 7fc1d9ffb700 1 -- 172.21.15.16:0/1375258097 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1dc012430 con 0x7fc1e415bb40 2023-05-20T01:16:56.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.163+0000 7fc1d9ffb700 1 -- 172.21.15.16:0/1375258097 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1dc020070 con 0x7fc1e415bb40 2023-05-20T01:16:56.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.163+0000 7fc1eb79c700 1 -- 172.21.15.16:0/1375258097 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1e4167890 con 0x7fc1e415bb40 2023-05-20T01:16:56.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.163+0000 7fc1d9ffb700 1 -- 172.21.15.16:0/1375258097 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1dc004280 con 0x7fc1e415bb40 2023-05-20T01:16:56.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.163+0000 7fc1eb79c700 1 -- 172.21.15.16:0/1375258097 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1e40cbed0 con 0x7fc1e415bb40 2023-05-20T01:16:56.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.164+0000 7fc1eb79c700 1 -- 172.21.15.16:0/1375258097 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1e4167a20 con 0x7fc1e415bb40 2023-05-20T01:16:56.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.164+0000 7fc1d9ffb700 1 -- 172.21.15.16:0/1375258097 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc1dc0043e0 con 0x7fc1e415bb40 2023-05-20T01:16:56.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.165+0000 7fc1d9ffb700 1 --2- 172.21.15.16:0/1375258097 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc1c806ace0 0x7fc1c806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:56.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.165+0000 7fc1d9ffb700 1 -- 172.21.15.16:0/1375258097 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fc1dc022070 con 0x7fc1e415bb40 2023-05-20T01:16:56.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.165+0000 7fc1e9538700 1 --2- 172.21.15.16:0/1375258097 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc1c806ace0 0x7fc1c806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:56.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.165+0000 7fc1e9538700 1 --2- 172.21.15.16:0/1375258097 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc1c806ace0 0x7fc1c806d190 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fc1e0000f60 tx=0x7fc1e0007d90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:56.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.169+0000 7fc1d9ffb700 1 -- 172.21.15.16:0/1375258097 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fc1dc0565e0 con 0x7fc1e415bb40 2023-05-20T01:16:56.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.369+0000 7fc1eb79c700 1 -- 172.21.15.16:0/1375258097 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc1e40b0a60 con 0x7fc1e415bb40 2023-05-20T01:16:56.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.369+0000 7fc1d9ffb700 1 -- 172.21.15.16:0/1375258097 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc1e40b0a60 con 0x7fc1e415bb40 2023-05-20T01:16:56.373 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2023-05-20T01:16:56.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.371+0000 7fc1eb79c700 1 -- 172.21.15.16:0/1375258097 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc1c806ace0 msgr2=0x7fc1c806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:56.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.371+0000 7fc1eb79c700 1 --2- 172.21.15.16:0/1375258097 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc1c806ace0 0x7fc1c806d190 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fc1e0000f60 tx=0x7fc1e0007d90 comp rx=0 tx=0).stop 2023-05-20T01:16:56.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.371+0000 7fc1eb79c700 1 -- 172.21.15.16:0/1375258097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1e415bb40 msgr2=0x7fc1e40cd9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:56.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.371+0000 7fc1eb79c700 1 --2- 172.21.15.16:0/1375258097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1e415bb40 0x7fc1e40cd9f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc1dc014040 tx=0x7fc1dc004ad0 comp rx=0 tx=0).stop 2023-05-20T01:16:56.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.371+0000 7fc1eb79c700 1 -- 172.21.15.16:0/1375258097 shutdown_connections 2023-05-20T01:16:56.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.371+0000 7fc1eb79c700 1 --2- 172.21.15.16:0/1375258097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1e415bb40 0x7fc1e40cd9f0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:56.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.371+0000 7fc1eb79c700 1 --2- 172.21.15.16:0/1375258097 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc1c806ace0 0x7fc1c806d190 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:56.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.371+0000 7fc1eb79c700 1 --2- 172.21.15.16:0/1375258097 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc1e406bd30 0x7fc1e40b34a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:56.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.371+0000 7fc1eb79c700 1 --2- 172.21.15.16:0/1375258097 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc1e4146710 0x7fc1e4154700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:56.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.371+0000 7fc1eb79c700 1 -- 172.21.15.16:0/1375258097 >> 172.21.15.16:0/1375258097 conn(0x7fc1e406ae30 msgr2=0x7fc1e406e7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:56.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.372+0000 7fc1eb79c700 1 -- 172.21.15.16:0/1375258097 shutdown_connections 2023-05-20T01:16:56.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.372+0000 7fc1eb79c700 1 -- 172.21.15.16:0/1375258097 wait complete. 2023-05-20T01:16:56.387 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-05-20T01:16:56.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.511+0000 7f33f7641700 1 Processor -- start 2023-05-20T01:16:56.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.512+0000 7f33f7641700 1 -- start start 2023-05-20T01:16:56.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.512+0000 7f33f7641700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f33f00cb700 0x7f33f0159e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:56.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.512+0000 7f33f7641700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f33f0071e10 0x7f33f015da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:56.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.512+0000 7f33f7641700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33f015d5b0 con 0x7f33f0157450 2023-05-20T01:16:56.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.512+0000 7f33f7641700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33f0157820 con 0x7f33f0071e10 2023-05-20T01:16:56.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.512+0000 7f33f7641700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33f015a4c0 con 0x7f33f00cb700 2023-05-20T01:16:56.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.513+0000 7f33f5bde700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f33f0071e10 0x7f33f015da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:56.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.513+0000 7f33f5bde700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f33f0071e10 0x7f33f015da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58628/0 (socket says 172.21.15.16:58628) 2023-05-20T01:16:56.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.513+0000 7f33f5bde700 1 -- 172.21.15.16:0/2935884127 learned_addr learned my addr 172.21.15.16:0/2935884127 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:56.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.513+0000 7f33f4bdc700 1 --2- 172.21.15.16:0/2935884127 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f33f00cb700 0x7f33f0159e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:56.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.513+0000 7f33f5bde700 1 -- 172.21.15.16:0/2935884127 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f33f00cb700 msgr2=0x7f33f0159e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:56.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.513+0000 7f33f5bde700 1 --2- 172.21.15.16:0/2935884127 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f33f00cb700 0x7f33f0159e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:56.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.513+0000 7f33f5bde700 1 -- 172.21.15.16:0/2935884127 >> v1:172.21.15.132:6789/0 conn(0x7f33f0157450 legacy=0x7f33f00caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:56.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.513+0000 7f33f5bde700 1 -- 172.21.15.16:0/2935884127 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33f015a7a0 con 0x7f33f0071e10 2023-05-20T01:16:56.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.513+0000 7f33f4bdc700 1 --2- 172.21.15.16:0/2935884127 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f33f00cb700 0x7f33f0159e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:16:56.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.513+0000 7f33f5bde700 1 --2- 172.21.15.16:0/2935884127 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33f0071e10 0x7f33f015da40 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f33e800d150 tx=0x7f33e800d460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28502fc4afa008f3 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:56.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.513+0000 7f33e3fff700 1 -- 172.21.15.16:0/2935884127 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e8010070 con 0x7f33f0071e10 2023-05-20T01:16:56.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.513+0000 7f33e3fff700 1 -- 172.21.15.16:0/2935884127 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33e800ddc0 con 0x7f33f0071e10 2023-05-20T01:16:56.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.513+0000 7f33e3fff700 1 -- 172.21.15.16:0/2935884127 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e800b310 con 0x7f33f0071e10 2023-05-20T01:16:56.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.514+0000 7f33f7641700 1 -- 172.21.15.16:0/2935884127 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33f0071e10 msgr2=0x7f33f015da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:56.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.514+0000 7f33f7641700 1 --2- 172.21.15.16:0/2935884127 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33f0071e10 0x7f33f015da40 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f33e800d150 tx=0x7f33e800d460 comp rx=0 tx=0).stop 2023-05-20T01:16:56.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.514+0000 7f33f7641700 1 -- 172.21.15.16:0/2935884127 shutdown_connections 2023-05-20T01:16:56.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.514+0000 7f33f7641700 1 --2- 172.21.15.16:0/2935884127 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33f0071e10 0x7f33f015da40 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:56.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.514+0000 7f33f7641700 1 --2- 172.21.15.16:0/2935884127 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f33f00cb700 0x7f33f0159e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:56.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.514+0000 7f33f7641700 1 -- 172.21.15.16:0/2935884127 >> 172.21.15.16:0/2935884127 conn(0x7f33f006cbb0 msgr2=0x7f33f006cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:56.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.514+0000 7f33f7641700 1 -- 172.21.15.16:0/2935884127 shutdown_connections 2023-05-20T01:16:56.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.514+0000 7f33f7641700 1 -- 172.21.15.16:0/2935884127 wait complete. 2023-05-20T01:16:56.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.515+0000 7f33f7641700 1 Processor -- start 2023-05-20T01:16:56.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.515+0000 7f33f7641700 1 -- start start 2023-05-20T01:16:56.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.516+0000 7f33f7641700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33f0071e10 0x7f33f0144290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:56.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.516+0000 7f33f7641700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f33f00cb700 0x7f33f00cdd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:56.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.516+0000 7f33f7641700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f33f0157450 0x7f33f00ce2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:56.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.516+0000 7f33f7641700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33f01447d0 con 0x7f33f0071e10 2023-05-20T01:16:56.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.516+0000 7f33f7641700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33f014ae20 con 0x7f33f00cb700 2023-05-20T01:16:56.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.516+0000 7f33f7641700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33f00ce940 con 0x7f33f0157450 2023-05-20T01:16:56.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.516+0000 7f33f53dd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33f0071e10 0x7f33f0144290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:56.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.516+0000 7f33f53dd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33f0071e10 0x7f33f0144290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58630/0 (socket says 172.21.15.16:58630) 2023-05-20T01:16:56.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.516+0000 7f33f53dd700 1 -- 172.21.15.16:0/805390158 learned_addr learned my addr 172.21.15.16:0/805390158 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:56.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.516+0000 7f33f4bdc700 1 --2- 172.21.15.16:0/805390158 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f33f00cb700 0x7f33f00cdd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:56.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.516+0000 7f33f53dd700 1 -- 172.21.15.16:0/805390158 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f33f0157450 msgr2=0x7f33f00ce2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:56.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.516+0000 7f33f5bde700 1 --2- 172.21.15.16:0/805390158 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f33f0157450 0x7f33f00ce2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:56.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.516+0000 7f33f53dd700 1 --2- 172.21.15.16:0/805390158 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f33f0157450 0x7f33f00ce2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:56.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.516+0000 7f33f53dd700 1 -- 172.21.15.16:0/805390158 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f33f00cb700 msgr2=0x7f33f00cdd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:56.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.516+0000 7f33f53dd700 1 --2- 172.21.15.16:0/805390158 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f33f00cb700 0x7f33f00cdd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:56.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.516+0000 7f33f53dd700 1 -- 172.21.15.16:0/805390158 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33e800bd40 con 0x7f33f0071e10 2023-05-20T01:16:56.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.516+0000 7f33f4bdc700 1 --2- 172.21.15.16:0/805390158 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f33f00cb700 0x7f33f00cdd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:16:56.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.517+0000 7f33f53dd700 1 --2- 172.21.15.16:0/805390158 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33f0071e10 0x7f33f0144290 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f33e400c490 tx=0x7f33e400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:56.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.517+0000 7f33e1ffb700 1 -- 172.21.15.16:0/805390158 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e40052d0 con 0x7f33f0071e10 2023-05-20T01:16:56.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.517+0000 7f33e1ffb700 1 -- 172.21.15.16:0/805390158 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33e4005450 con 0x7f33f0071e10 2023-05-20T01:16:56.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.517+0000 7f33f7641700 1 -- 172.21.15.16:0/805390158 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33f015ad60 con 0x7f33f0071e10 2023-05-20T01:16:56.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.517+0000 7f33e1ffb700 1 -- 172.21.15.16:0/805390158 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e4006aa0 con 0x7f33f0071e10 2023-05-20T01:16:56.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.517+0000 7f33f7641700 1 -- 172.21.15.16:0/805390158 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33f015aa80 con 0x7f33f0071e10 2023-05-20T01:16:56.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.518+0000 7f33f7641700 1 -- 172.21.15.16:0/805390158 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33f0073b00 con 0x7f33f0071e10 2023-05-20T01:16:56.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.518+0000 7f33e1ffb700 1 -- 172.21.15.16:0/805390158 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f33e4003680 con 0x7f33f0071e10 2023-05-20T01:16:56.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.519+0000 7f33e1ffb700 1 --2- 172.21.15.16:0/805390158 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f33d406ace0 0x7f33d406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:56.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.519+0000 7f33e1ffb700 1 -- 172.21.15.16:0/805390158 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f33e40889e0 con 0x7f33f0071e10 2023-05-20T01:16:56.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.519+0000 7f33f4bdc700 1 --2- 172.21.15.16:0/805390158 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f33d406ace0 0x7f33d406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:56.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.519+0000 7f33f4bdc700 1 --2- 172.21.15.16:0/805390158 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f33d406ace0 0x7f33d406d190 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f33f00c9840 tx=0x7f33ec008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:56.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.523+0000 7f33e1ffb700 1 -- 172.21.15.16:0/805390158 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f33e4053830 con 0x7f33f0071e10 2023-05-20T01:16:56.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.722+0000 7f33f7641700 1 -- 172.21.15.16:0/805390158 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f33f00747b0 con 0x7f33f0071e10 2023-05-20T01:16:56.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.723+0000 7f33e1ffb700 1 -- 172.21.15.16:0/805390158 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7f33e4056e50 con 0x7f33f0071e10 2023-05-20T01:16:56.727 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-05-20T01:16:56.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.725+0000 7f33f7641700 1 -- 172.21.15.16:0/805390158 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f33d406ace0 msgr2=0x7f33d406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:56.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.725+0000 7f33f7641700 1 --2- 172.21.15.16:0/805390158 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f33d406ace0 0x7f33d406d190 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f33f00c9840 tx=0x7f33ec008040 comp rx=0 tx=0).stop 2023-05-20T01:16:56.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.725+0000 7f33f7641700 1 -- 172.21.15.16:0/805390158 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33f0071e10 msgr2=0x7f33f0144290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:56.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.725+0000 7f33f7641700 1 --2- 172.21.15.16:0/805390158 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33f0071e10 0x7f33f0144290 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f33e400c490 tx=0x7f33e400c850 comp rx=0 tx=0).stop 2023-05-20T01:16:56.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.725+0000 7f33f7641700 1 -- 172.21.15.16:0/805390158 shutdown_connections 2023-05-20T01:16:56.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.725+0000 7f33f7641700 1 --2- 172.21.15.16:0/805390158 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f33d406ace0 0x7f33d406d190 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:56.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.725+0000 7f33f7641700 1 --2- 172.21.15.16:0/805390158 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f33f0157450 0x7f33f00ce2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:56.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.725+0000 7f33f7641700 1 --2- 172.21.15.16:0/805390158 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33f0071e10 0x7f33f0144290 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:56.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.725+0000 7f33f7641700 1 --2- 172.21.15.16:0/805390158 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f33f00cb700 0x7f33f00cdd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:56.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.725+0000 7f33f7641700 1 -- 172.21.15.16:0/805390158 >> 172.21.15.16:0/805390158 conn(0x7f33f006cbb0 msgr2=0x7f33f01562c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:56.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.725+0000 7f33f7641700 1 -- 172.21.15.16:0/805390158 shutdown_connections 2023-05-20T01:16:56.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.725+0000 7f33f7641700 1 -- 172.21.15.16:0/805390158 wait complete. 2023-05-20T01:16:56.741 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-20T01:16:56.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.862+0000 7f37a99af700 1 Processor -- start 2023-05-20T01:16:56.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.862+0000 7f37a99af700 1 -- start start 2023-05-20T01:16:56.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.862+0000 7f37a99af700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f37a406d4b0 0x7f37a40cf890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:56.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.862+0000 7f37a99af700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f37a406a510 0x7f37a406a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:56.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.862+0000 7f37a99af700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37a40cc510 con 0x7f37a406f450 2023-05-20T01:16:56.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.862+0000 7f37a99af700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37a406aef0 con 0x7f37a406a510 2023-05-20T01:16:56.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.862+0000 7f37a99af700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37a4071ed0 con 0x7f37a406d4b0 2023-05-20T01:16:56.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.863+0000 7f37a27fc700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f37a406d4b0 0x7f37a40cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:56.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.863+0000 7f37a37fe700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f37a406a510 0x7f37a406a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:56.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.863+0000 7f37a2ffd700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f37a406f450 0x7f37a4161080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.16:51648/0 (socket says 172.21.15.16:51648) 2023-05-20T01:16:56.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.863+0000 7f37a27fc700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f37a406d4b0 0x7f37a40cf890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.16:58678/0 (socket says 172.21.15.16:58678) 2023-05-20T01:16:56.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.863+0000 7f37a2ffd700 1 -- 172.21.15.16:0/399241709 learned_addr learned my addr 172.21.15.16:0/399241709 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:16:56.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.863+0000 7f37a37fe700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f37a406a510 0x7f37a406a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.16:55664/0 (socket says 172.21.15.16:55664) 2023-05-20T01:16:56.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.863+0000 7f37a37fe700 1 -- 172.21.15.16:0/399241709 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f37a406d4b0 msgr2=0x7f37a40cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:56.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.863+0000 7f37a37fe700 1 --2- 172.21.15.16:0/399241709 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f37a406d4b0 0x7f37a40cf890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:56.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.863+0000 7f37a37fe700 1 -- 172.21.15.16:0/399241709 >> v1:172.21.15.132:6789/0 conn(0x7f37a406f450 legacy=0x7f37a4161080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:56.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.863+0000 7f37a1ffb700 1 -- 172.21.15.16:0/399241709 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3500244952 0 0) 0x7f37a40cc510 con 0x7f37a406f450 2023-05-20T01:16:56.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.863+0000 7f37a37fe700 1 -- 172.21.15.16:0/399241709 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37a40721b0 con 0x7f37a406a510 2023-05-20T01:16:56.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.863+0000 7f37a27fc700 1 --2- 172.21.15.16:0/399241709 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f37a406d4b0 0x7f37a40cf890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-05-20T01:16:56.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.863+0000 7f37a37fe700 1 --2- 172.21.15.16:0/399241709 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f37a406a510 0x7f37a406a9b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f379c009d90 tx=0x7f379c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30270371620ac800 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:56.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.863+0000 7f37a1ffb700 1 -- 172.21.15.16:0/399241709 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f379c00e070 con 0x7f37a406a510 2023-05-20T01:16:56.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.864+0000 7f37a1ffb700 1 -- 172.21.15.16:0/399241709 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f379c00bb20 con 0x7f37a406a510 2023-05-20T01:16:56.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.864+0000 7f37a1ffb700 1 -- 172.21.15.16:0/399241709 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f379c012430 con 0x7f37a406a510 2023-05-20T01:16:56.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.864+0000 7f37a99af700 1 -- 172.21.15.16:0/399241709 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f37a406a510 msgr2=0x7f37a406a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:56.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.864+0000 7f37a99af700 1 --2- 172.21.15.16:0/399241709 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f37a406a510 0x7f37a406a9b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f379c009d90 tx=0x7f379c00b330 comp rx=0 tx=0).stop 2023-05-20T01:16:56.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.864+0000 7f37a99af700 1 -- 172.21.15.16:0/399241709 shutdown_connections 2023-05-20T01:16:56.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.864+0000 7f37a99af700 1 --2- 172.21.15.16:0/399241709 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f37a406a510 0x7f37a406a9b0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:56.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.864+0000 7f37a99af700 1 --2- 172.21.15.16:0/399241709 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f37a406d4b0 0x7f37a40cf890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:56.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.864+0000 7f37a99af700 1 -- 172.21.15.16:0/399241709 >> 172.21.15.16:0/399241709 conn(0x7f37a40cffb0 msgr2=0x7f37a40d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:56.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.864+0000 7f37a99af700 1 -- 172.21.15.16:0/399241709 shutdown_connections 2023-05-20T01:16:56.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.864+0000 7f37a99af700 1 -- 172.21.15.16:0/399241709 wait complete. 2023-05-20T01:16:56.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.865+0000 7f37a99af700 1 Processor -- start 2023-05-20T01:16:56.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.866+0000 7f37a99af700 1 -- start start 2023-05-20T01:16:56.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.866+0000 7f37a99af700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f37a406a510 0x7f37a414e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:56.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.866+0000 7f37a99af700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f37a406d4b0 0x7f37a41f24d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:56.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.866+0000 7f37a99af700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37a406f450 0x7f37a40b4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:56.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.866+0000 7f37a99af700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37a415b9e0 con 0x7f37a406f450 2023-05-20T01:16:56.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.866+0000 7f37a99af700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37a414ea70 con 0x7f37a406a510 2023-05-20T01:16:56.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.866+0000 7f37a99af700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37a41e3d50 con 0x7f37a406d4b0 2023-05-20T01:16:56.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.866+0000 7f37a37fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37a406f450 0x7f37a40b4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:56.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.866+0000 7f37a2ffd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f37a406a510 0x7f37a414e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:56.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.867+0000 7f37a37fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37a406f450 0x7f37a40b4880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58648/0 (socket says 172.21.15.16:58648) 2023-05-20T01:16:56.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.867+0000 7f37a37fe700 1 -- 172.21.15.16:0/874101812 learned_addr learned my addr 172.21.15.16:0/874101812 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:56.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.867+0000 7f37a27fc700 1 --2- 172.21.15.16:0/874101812 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f37a406d4b0 0x7f37a41f24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:56.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.867+0000 7f37a37fe700 1 -- 172.21.15.16:0/874101812 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f37a406d4b0 msgr2=0x7f37a41f24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:56.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.867+0000 7f37a37fe700 1 --2- 172.21.15.16:0/874101812 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f37a406d4b0 0x7f37a41f24d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:56.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.867+0000 7f37a37fe700 1 -- 172.21.15.16:0/874101812 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f37a406a510 msgr2=0x7f37a414e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:56.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.867+0000 7f37a37fe700 1 --2- 172.21.15.16:0/874101812 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f37a406a510 0x7f37a414e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:56.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.867+0000 7f37a37fe700 1 -- 172.21.15.16:0/874101812 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37940094a0 con 0x7f37a406f450 2023-05-20T01:16:56.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.867+0000 7f37a2ffd700 1 --2- 172.21.15.16:0/874101812 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f37a406a510 0x7f37a414e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-05-20T01:16:56.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.867+0000 7f37a37fe700 1 --2- 172.21.15.16:0/874101812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37a406f450 0x7f37a40b4880 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f37a40d0e30 tx=0x7f379c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:56.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.867+0000 7f379bfff700 1 -- 172.21.15.16:0/874101812 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f379c012430 con 0x7f37a406f450 2023-05-20T01:16:56.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.867+0000 7f379bfff700 1 -- 172.21.15.16:0/874101812 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f379c020070 con 0x7f37a406f450 2023-05-20T01:16:56.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.867+0000 7f37a99af700 1 -- 172.21.15.16:0/874101812 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f379c009a70 con 0x7f37a406f450 2023-05-20T01:16:56.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.867+0000 7f379bfff700 1 -- 172.21.15.16:0/874101812 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f379c004280 con 0x7f37a406f450 2023-05-20T01:16:56.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.867+0000 7f37a99af700 1 -- 172.21.15.16:0/874101812 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37a40cbe50 con 0x7f37a406f450 2023-05-20T01:16:56.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.868+0000 7f37a99af700 1 -- 172.21.15.16:0/874101812 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37a40cc1a0 con 0x7f37a406f450 2023-05-20T01:16:56.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.869+0000 7f379bfff700 1 -- 172.21.15.16:0/874101812 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f379c0043e0 con 0x7f37a406f450 2023-05-20T01:16:56.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.869+0000 7f379bfff700 1 --2- 172.21.15.16:0/874101812 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f378406ace0 0x7f378406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:56.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.869+0000 7f379bfff700 1 -- 172.21.15.16:0/874101812 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f379c022070 con 0x7f37a406f450 2023-05-20T01:16:56.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.869+0000 7f37a2ffd700 1 --2- 172.21.15.16:0/874101812 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f378406ace0 0x7f378406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:56.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.870+0000 7f37a2ffd700 1 --2- 172.21.15.16:0/874101812 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f378406ace0 0x7f378406d190 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f378c000f60 tx=0x7f378c007e70 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:56.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:56.874+0000 7f379bfff700 1 -- 172.21.15.16:0/874101812 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f379c056190 con 0x7f37a406f450 2023-05-20T01:16:57.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.114+0000 7f37a99af700 1 -- 172.21.15.16:0/874101812 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f37a40cc980 con 0x7f37a406f450 2023-05-20T01:16:57.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.114+0000 7f379bfff700 1 -- 172.21.15.16:0/874101812 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4673 (secure 0 0 0) 0x7f379c0597b0 con 0x7f37a406f450 2023-05-20T01:16:57.118 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:16:57.119 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4273,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/1444124146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1444124146},{"type":"v1","addr":"172.21.15.132:6834","nonce":1444124146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4293,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4092119206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4092119206},{"type":"v1","addr":"172.21.15.16:6835","nonce":4092119206}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2162680372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":2162680372},{"type":"v1","addr":"172.21.15.132:6835","nonce":2162680372}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-20T01:16:28.316109+0000","modified":"2023-05-20T01:16:45.362572+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4299},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4299":{"gid":4299,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6837/2330551626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2330551626},{"type":"v1","addr":"172.21.15.16:6837","nonce":2330551626}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-05-20T01:16:57.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.116+0000 7f37a99af700 1 -- 172.21.15.16:0/874101812 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f378406ace0 msgr2=0x7f378406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:57.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.116+0000 7f37a99af700 1 --2- 172.21.15.16:0/874101812 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f378406ace0 0x7f378406d190 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f378c000f60 tx=0x7f378c007e70 comp rx=0 tx=0).stop 2023-05-20T01:16:57.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.116+0000 7f37a99af700 1 -- 172.21.15.16:0/874101812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37a406f450 msgr2=0x7f37a40b4880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:57.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.116+0000 7f37a99af700 1 --2- 172.21.15.16:0/874101812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37a406f450 0x7f37a40b4880 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f37a40d0e30 tx=0x7f379c004ad0 comp rx=0 tx=0).stop 2023-05-20T01:16:57.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.117+0000 7f37a99af700 1 -- 172.21.15.16:0/874101812 shutdown_connections 2023-05-20T01:16:57.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.117+0000 7f37a99af700 1 --2- 172.21.15.16:0/874101812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37a406f450 0x7f37a40b4880 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:57.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.117+0000 7f37a99af700 1 --2- 172.21.15.16:0/874101812 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f37a406a510 0x7f37a414e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:57.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.117+0000 7f37a99af700 1 --2- 172.21.15.16:0/874101812 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f378406ace0 0x7f378406d190 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:57.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.117+0000 7f37a99af700 1 --2- 172.21.15.16:0/874101812 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f37a406d4b0 0x7f37a41f24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:57.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.117+0000 7f37a99af700 1 -- 172.21.15.16:0/874101812 >> 172.21.15.16:0/874101812 conn(0x7f37a40cffb0 msgr2=0x7f37a40c98e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:57.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.117+0000 7f37a99af700 1 -- 172.21.15.16:0/874101812 shutdown_connections 2023-05-20T01:16:57.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.117+0000 7f37a99af700 1 -- 172.21.15.16:0/874101812 wait complete. 2023-05-20T01:16:57.122 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 8 2023-05-20T01:16:57.132 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-20T01:16:57.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.254+0000 7f6b485f0700 1 Processor -- start 2023-05-20T01:16:57.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.254+0000 7f6b485f0700 1 -- start start 2023-05-20T01:16:57.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.255+0000 7f6b485f0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6b4015a9d0 0x7f6b4015efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:57.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.255+0000 7f6b485f0700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6b4006ac20 0x7f6b400710d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:57.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.255+0000 7f6b485f0700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b4006d940 con 0x7f6b400b2250 2023-05-20T01:16:57.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.255+0000 7f6b485f0700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b400cc8d0 con 0x7f6b4006ac20 2023-05-20T01:16:57.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.255+0000 7f6b485f0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b4015f500 con 0x7f6b4015a9d0 2023-05-20T01:16:57.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.255+0000 7f6b46b8d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6b4006ac20 0x7f6b400710d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:57.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.255+0000 7f6b46b8d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6b4006ac20 0x7f6b400710d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58654/0 (socket says 172.21.15.16:58654) 2023-05-20T01:16:57.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.255+0000 7f6b46b8d700 1 -- 172.21.15.16:0/1855002375 learned_addr learned my addr 172.21.15.16:0/1855002375 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:57.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.255+0000 7f6b46b8d700 1 -- 172.21.15.16:0/1855002375 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6b4015a9d0 msgr2=0x7f6b4015efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:57.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.255+0000 7f6b4638c700 1 --2- 172.21.15.16:0/1855002375 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6b4015a9d0 0x7f6b4015efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:57.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.255+0000 7f6b46b8d700 1 --2- 172.21.15.16:0/1855002375 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6b4015a9d0 0x7f6b4015efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:57.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.255+0000 7f6b46b8d700 1 -- 172.21.15.16:0/1855002375 >> v1:172.21.15.132:6789/0 conn(0x7f6b400b2250 legacy=0x7f6b4006a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:57.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.255+0000 7f6b46b8d700 1 -- 172.21.15.16:0/1855002375 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b400cc030 con 0x7f6b4006ac20 2023-05-20T01:16:57.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.255+0000 7f6b46b8d700 1 --2- 172.21.15.16:0/1855002375 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b4006ac20 0x7f6b400710d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6b38009e50 tx=0x7f6b3800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b8968a52647095e server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:57.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.256+0000 7f6b4538a700 1 -- 172.21.15.16:0/1855002375 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b3800e070 con 0x7f6b4006ac20 2023-05-20T01:16:57.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.256+0000 7f6b4538a700 1 -- 172.21.15.16:0/1855002375 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b38003a20 con 0x7f6b4006ac20 2023-05-20T01:16:57.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.256+0000 7f6b4538a700 1 -- 172.21.15.16:0/1855002375 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b38003dd0 con 0x7f6b4006ac20 2023-05-20T01:16:57.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.256+0000 7f6b485f0700 1 -- 172.21.15.16:0/1855002375 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b4006ac20 msgr2=0x7f6b400710d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:57.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.256+0000 7f6b485f0700 1 --2- 172.21.15.16:0/1855002375 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b4006ac20 0x7f6b400710d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6b38009e50 tx=0x7f6b3800b330 comp rx=0 tx=0).stop 2023-05-20T01:16:57.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.256+0000 7f6b485f0700 1 -- 172.21.15.16:0/1855002375 shutdown_connections 2023-05-20T01:16:57.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.256+0000 7f6b485f0700 1 --2- 172.21.15.16:0/1855002375 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b4006ac20 0x7f6b400710d0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:57.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.256+0000 7f6b485f0700 1 --2- 172.21.15.16:0/1855002375 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6b4015a9d0 0x7f6b4015efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:57.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.256+0000 7f6b485f0700 1 -- 172.21.15.16:0/1855002375 >> 172.21.15.16:0/1855002375 conn(0x7f6b400d0c30 msgr2=0x7f6b400b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:57.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.256+0000 7f6b485f0700 1 -- 172.21.15.16:0/1855002375 shutdown_connections 2023-05-20T01:16:57.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.256+0000 7f6b485f0700 1 -- 172.21.15.16:0/1855002375 wait complete. 2023-05-20T01:16:57.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.257+0000 7f6b485f0700 1 Processor -- start 2023-05-20T01:16:57.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.258+0000 7f6b485f0700 1 -- start start 2023-05-20T01:16:57.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.258+0000 7f6b485f0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6b4006ac20 0x7f6b40144ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:57.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.258+0000 7f6b485f0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b400b2250 0x7f6b40145420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:57.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.258+0000 7f6b485f0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6b4015a9d0 0x7f6b400ceff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:57.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.258+0000 7f6b485f0700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b400c35c0 con 0x7f6b400b2250 2023-05-20T01:16:57.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.258+0000 7f6b485f0700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b400b4530 con 0x7f6b4015a9d0 2023-05-20T01:16:57.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.258+0000 7f6b485f0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b400b4810 con 0x7f6b4006ac20 2023-05-20T01:16:57.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.258+0000 7f6b45b8b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b400b2250 0x7f6b40145420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:57.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.258+0000 7f6b45b8b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b400b2250 0x7f6b40145420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58658/0 (socket says 172.21.15.16:58658) 2023-05-20T01:16:57.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.258+0000 7f6b45b8b700 1 -- 172.21.15.16:0/1408646932 learned_addr learned my addr 172.21.15.16:0/1408646932 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:57.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.258+0000 7f6b4638c700 1 --2- 172.21.15.16:0/1408646932 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6b4006ac20 0x7f6b40144ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:57.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.258+0000 7f6b46b8d700 1 --2- 172.21.15.16:0/1408646932 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6b4015a9d0 0x7f6b400ceff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:57.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.259+0000 7f6b45b8b700 1 -- 172.21.15.16:0/1408646932 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6b4006ac20 msgr2=0x7f6b40144ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:57.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.259+0000 7f6b45b8b700 1 --2- 172.21.15.16:0/1408646932 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6b4006ac20 0x7f6b40144ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:57.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.259+0000 7f6b45b8b700 1 -- 172.21.15.16:0/1408646932 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6b4015a9d0 msgr2=0x7f6b400ceff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:57.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.259+0000 7f6b45b8b700 1 --2- 172.21.15.16:0/1408646932 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6b4015a9d0 0x7f6b400ceff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:57.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.259+0000 7f6b45b8b700 1 -- 172.21.15.16:0/1408646932 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b38009b30 con 0x7f6b400b2250 2023-05-20T01:16:57.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.259+0000 7f6b45b8b700 1 --2- 172.21.15.16:0/1408646932 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b400b2250 0x7f6b40145420 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f6b3000c4d0 tx=0x7f6b3000c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:57.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.259+0000 7f6b36ffd700 1 -- 172.21.15.16:0/1408646932 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b30011790 con 0x7f6b400b2250 2023-05-20T01:16:57.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.259+0000 7f6b36ffd700 1 -- 172.21.15.16:0/1408646932 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b300118f0 con 0x7f6b400b2250 2023-05-20T01:16:57.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.259+0000 7f6b485f0700 1 -- 172.21.15.16:0/1408646932 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b400cc5f0 con 0x7f6b400b2250 2023-05-20T01:16:57.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.259+0000 7f6b36ffd700 1 -- 172.21.15.16:0/1408646932 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b30011ca0 con 0x7f6b400b2250 2023-05-20T01:16:57.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.259+0000 7f6b485f0700 1 -- 172.21.15.16:0/1408646932 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b400cc310 con 0x7f6b400b2250 2023-05-20T01:16:57.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.260+0000 7f6b485f0700 1 -- 172.21.15.16:0/1408646932 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b4015e7c0 con 0x7f6b400b2250 2023-05-20T01:16:57.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.260+0000 7f6b36ffd700 1 -- 172.21.15.16:0/1408646932 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6b3001c460 con 0x7f6b400b2250 2023-05-20T01:16:57.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.261+0000 7f6b36ffd700 1 --2- 172.21.15.16:0/1408646932 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f6b2406ace0 0x7f6b2406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:57.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.261+0000 7f6b36ffd700 1 -- 172.21.15.16:0/1408646932 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f6b30089400 con 0x7f6b400b2250 2023-05-20T01:16:57.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.261+0000 7f6b4638c700 1 --2- 172.21.15.16:0/1408646932 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f6b2406ace0 0x7f6b2406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:57.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.261+0000 7f6b4638c700 1 --2- 172.21.15.16:0/1408646932 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f6b2406ace0 0x7f6b2406d190 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c006fd0 tx=0x7f6b3c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:57.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.265+0000 7f6b36ffd700 1 -- 172.21.15.16:0/1408646932 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f6b30054250 con 0x7f6b400b2250 2023-05-20T01:16:57.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.464+0000 7f6b485f0700 1 -- 172.21.15.16:0/1408646932 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6b400747b0 con 0x7f6b400b2250 2023-05-20T01:16:57.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.466+0000 7f6b36ffd700 1 -- 172.21.15.16:0/1408646932 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+20026 (secure 0 0 0) 0x7f6b30057870 con 0x7f6b400b2250 2023-05-20T01:16:57.469 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:16:57.470 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"fsid":"c2245cc5-ecab-4087-a59b-bc81db901676","created":"2023-05-20T01:16:09.549230+0000","modified":"2023-05-20T01:16:32.768495+0000","last_up_change":"2023-05-20T01:16:19.788624+0000","last_in_change":"2023-05-20T01:16:12.092160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-05-20T01:16:20.934366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-05-20T01:16:21.372294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-05-20T01:16:26.208287+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-05-20T01:16:27.292268+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-05-20T01:16:30.357285+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"317a83cd-19c8-4958-a301-6087ef342289","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6825","nonce":3923634179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6827","nonce":3923634179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6831","nonce":3923634179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6829","nonce":3923634179}]},"public_addr":"172.21.15.16:6825/3923634179","cluster_addr":"172.21.15.16:6827/3923634179","heartbeat_back_addr":"172.21.15.16:6831/3923634179","heartbeat_front_addr":"172.21.15.16:6829/3923634179","state":["exists","up"]},{"osd":1,"uuid":"5581046f-8219-4089-8d7d-b7846399505e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6809","nonce":3860264220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6811","nonce":3860264220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6815","nonce":3860264220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6813","nonce":3860264220}]},"public_addr":"172.21.15.16:6809/3860264220","cluster_addr":"172.21.15.16:6811/3860264220","heartbeat_back_addr":"172.21.15.16:6815/3860264220","heartbeat_front_addr":"172.21.15.16:6813/3860264220","state":["exists","up"]},{"osd":2,"uuid":"072e9e66-4133-478e-8d4a-64d46c2963f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6817","nonce":293187030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6819","nonce":293187030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6823","nonce":293187030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6821","nonce":293187030}]},"public_addr":"172.21.15.16:6817/293187030","cluster_addr":"172.21.15.16:6819/293187030","heartbeat_back_addr":"172.21.15.16:6823/293187030","heartbeat_front_addr":"172.21.15.16:6821/293187030","state":["exists","up"]},{"osd":3,"uuid":"11e1bba6-00f8-48dd-9028-d8d716b7f145","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6801","nonce":3669781890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6803","nonce":3669781890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6807","nonce":3669781890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6805","nonce":3669781890}]},"public_addr":"172.21.15.16:6801/3669781890","cluster_addr":"172.21.15.16:6803/3669781890","heartbeat_back_addr":"172.21.15.16:6807/3669781890","heartbeat_front_addr":"172.21.15.16:6805/3669781890","state":["exists","up"]},{"osd":4,"uuid":"b8909c3a-cf70-46cb-8d0b-009c707cf9d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6819","nonce":506317223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6823","nonce":506317223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6829","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6831","nonce":506317223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6827","nonce":506317223}]},"public_addr":"172.21.15.132:6819/506317223","cluster_addr":"172.21.15.132:6823/506317223","heartbeat_back_addr":"172.21.15.132:6831/506317223","heartbeat_front_addr":"172.21.15.132:6827/506317223","state":["exists","up"]},{"osd":5,"uuid":"b1818304-4583-47bf-a6ec-0bfbb5166233","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6818","nonce":1844689995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6822","nonce":1844689995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6830","nonce":1844689995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6826","nonce":1844689995}]},"public_addr":"172.21.15.132:6818/1844689995","cluster_addr":"172.21.15.132:6822/1844689995","heartbeat_back_addr":"172.21.15.132:6830/1844689995","heartbeat_front_addr":"172.21.15.132:6826/1844689995","state":["exists","up"]},{"osd":6,"uuid":"d37f4691-eed1-4a93-8818-18354f03f4a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6801","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6802","nonce":727581123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6805","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6807","nonce":727581123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6813","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6815","nonce":727581123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6811","nonce":727581123}]},"public_addr":"172.21.15.132:6802/727581123","cluster_addr":"172.21.15.132:6807/727581123","heartbeat_back_addr":"172.21.15.132:6815/727581123","heartbeat_front_addr":"172.21.15.132:6811/727581123","state":["exists","up"]},{"osd":7,"uuid":"4f712f62-5d1d-4816-8042-03209144f8b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6803","nonce":466001149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6806","nonce":466001149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6814","nonce":466001149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6810","nonce":466001149}]},"public_addr":"172.21.15.132:6803/466001149","cluster_addr":"172.21.15.132:6806/466001149","heartbeat_back_addr":"172.21.15.132:6814/466001149","heartbeat_front_addr":"172.21.15.132:6810/466001149","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.338863+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.278680+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305694+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.246835+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.276237+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.304441+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305421+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.267050+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-20T01:16:57.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.468+0000 7f6b485f0700 1 -- 172.21.15.16:0/1408646932 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f6b2406ace0 msgr2=0x7f6b2406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:57.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.468+0000 7f6b485f0700 1 --2- 172.21.15.16:0/1408646932 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f6b2406ace0 0x7f6b2406d190 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c006fd0 tx=0x7f6b3c008040 comp rx=0 tx=0).stop 2023-05-20T01:16:57.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.468+0000 7f6b485f0700 1 -- 172.21.15.16:0/1408646932 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b400b2250 msgr2=0x7f6b40145420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:57.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.468+0000 7f6b485f0700 1 --2- 172.21.15.16:0/1408646932 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b400b2250 0x7f6b40145420 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f6b3000c4d0 tx=0x7f6b3000c7e0 comp rx=0 tx=0).stop 2023-05-20T01:16:57.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.468+0000 7f6b485f0700 1 -- 172.21.15.16:0/1408646932 shutdown_connections 2023-05-20T01:16:57.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.468+0000 7f6b485f0700 1 --2- 172.21.15.16:0/1408646932 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6b4015a9d0 0x7f6b400ceff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:57.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.468+0000 7f6b485f0700 1 --2- 172.21.15.16:0/1408646932 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f6b2406ace0 0x7f6b2406d190 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:57.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.468+0000 7f6b485f0700 1 --2- 172.21.15.16:0/1408646932 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6b4006ac20 0x7f6b40144ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:57.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.468+0000 7f6b485f0700 1 --2- 172.21.15.16:0/1408646932 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b400b2250 0x7f6b40145420 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:57.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.468+0000 7f6b485f0700 1 -- 172.21.15.16:0/1408646932 >> 172.21.15.16:0/1408646932 conn(0x7f6b400d0c30 msgr2=0x7f6b401589d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:57.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.468+0000 7f6b485f0700 1 -- 172.21.15.16:0/1408646932 shutdown_connections 2023-05-20T01:16:57.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.468+0000 7f6b485f0700 1 -- 172.21.15.16:0/1408646932 wait complete. 2023-05-20T01:16:57.484 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-20T01:16:57.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.604+0000 7fabb3776700 1 Processor -- start 2023-05-20T01:16:57.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.604+0000 7fabb3776700 1 -- start start 2023-05-20T01:16:57.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.605+0000 7fabb3776700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fabac0cf370 0x7fabac0d1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:57.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.605+0000 7fabb3776700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabac0cc500 con 0x7fabac0b2760 2023-05-20T01:16:57.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.605+0000 7fabb3776700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabac0cb8f0 con 0x7fabac144020 2023-05-20T01:16:57.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.605+0000 7fabb3776700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabac0cb6e0 con 0x7fabac0cf370 2023-05-20T01:16:57.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.605+0000 7fabb1512700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fabac0b2760 0x7fabac0b0980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:41960/0 (socket says 172.21.15.16:41960) 2023-05-20T01:16:57.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.605+0000 7fabb1512700 1 -- 172.21.15.16:0/3747269989 learned_addr learned my addr 172.21.15.16:0/3747269989 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:16:57.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.606+0000 7fabb1d13700 1 --2- 172.21.15.16:0/3747269989 >> v2:172.21.15.132:3300/0 conn(0x7fabac0cf370 0x7fabac0d1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:57.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.606+0000 7fab9ffff700 1 -- 172.21.15.16:0/3747269989 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1480019152 0 0) 0x7fabac0cc500 con 0x7fabac0b2760 2023-05-20T01:16:57.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.606+0000 7fab9ffff700 1 -- 172.21.15.16:0/3747269989 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab980035b0 con 0x7fabac0b2760 2023-05-20T01:16:57.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.606+0000 7fab9ffff700 1 -- 172.21.15.16:0/3747269989 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2365735158 0 0) 0x7fabac0cb8f0 con 0x7fabac144020 2023-05-20T01:16:57.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.606+0000 7fab9ffff700 1 -- 172.21.15.16:0/3747269989 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fabac0cc500 con 0x7fabac144020 2023-05-20T01:16:57.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.606+0000 7fab9ffff700 1 -- 172.21.15.16:0/3747269989 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2077069387 0 0) 0x7fab980035b0 con 0x7fabac0b2760 2023-05-20T01:16:57.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.606+0000 7fab9ffff700 1 -- 172.21.15.16:0/3747269989 >> v2:172.21.15.132:3300/0 conn(0x7fabac0cf370 msgr2=0x7fabac0d1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:57.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.606+0000 7fab9ffff700 1 --2- 172.21.15.16:0/3747269989 >> v2:172.21.15.132:3300/0 conn(0x7fabac0cf370 0x7fabac0d1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:57.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.606+0000 7fab9ffff700 1 -- 172.21.15.16:0/3747269989 >> v1:172.21.15.132:6789/0 conn(0x7fabac144020 legacy=0x7fabac0cec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:57.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.606+0000 7fab9ffff700 1 -- 172.21.15.16:0/3747269989 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabac0cbbd0 con 0x7fabac0b2760 2023-05-20T01:16:57.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.606+0000 7fab9ffff700 1 -- 172.21.15.16:0/3747269989 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7faba8001670 con 0x7fabac0b2760 2023-05-20T01:16:57.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.606+0000 7fab9ffff700 1 -- 172.21.15.16:0/3747269989 >> v1:172.21.15.16:6789/0 conn(0x7fabac0b2760 legacy=0x7fabac0b0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:57.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.606+0000 7fab9ffff700 1 --2- 172.21.15.16:0/3747269989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab98003c50 0x7fab98004040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:57.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.606+0000 7fab9ffff700 1 --2- 172.21.15.16:0/3747269989 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fab98004850 0x7fab98004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:57.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.606+0000 7fab9ffff700 1 --2- 172.21.15.16:0/3747269989 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fab98005200 0x7fab9800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:57.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.606+0000 7fab9ffff700 1 -- 172.21.15.16:0/3747269989 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabac0cb6e0 con 0x7fab98003c50 2023-05-20T01:16:57.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.606+0000 7fab9ffff700 1 -- 172.21.15.16:0/3747269989 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab9800c110 con 0x7fab98005200 2023-05-20T01:16:57.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.606+0000 7fab9ffff700 1 -- 172.21.15.16:0/3747269989 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab9800c2d0 con 0x7fab98004850 2023-05-20T01:16:57.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.606+0000 7fabb1512700 1 --2- 172.21.15.16:0/3747269989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab98003c50 0x7fab98004040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:57.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.606+0000 7fabb1d13700 1 --2- 172.21.15.16:0/3747269989 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fab98005200 0x7fab9800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:57.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.606+0000 7fabb0d11700 1 --2- 172.21.15.16:0/3747269989 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fab98004850 0x7fab98004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:57.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.607+0000 7fabb1512700 1 -- 172.21.15.16:0/3747269989 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fab98004850 msgr2=0x7fab98004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:57.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.607+0000 7fabb1512700 1 --2- 172.21.15.16:0/3747269989 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fab98004850 0x7fab98004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:57.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.607+0000 7fabb1512700 1 -- 172.21.15.16:0/3747269989 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fab98005200 msgr2=0x7fab9800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:57.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.607+0000 7fabb1512700 1 --2- 172.21.15.16:0/3747269989 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fab98005200 0x7fab9800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:57.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.607+0000 7fabb1512700 1 -- 172.21.15.16:0/3747269989 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab980035b0 con 0x7fab98003c50 2023-05-20T01:16:57.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.607+0000 7fabb1d13700 1 --2- 172.21.15.16:0/3747269989 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fab98005200 0x7fab9800ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-05-20T01:16:57.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.607+0000 7fabb1d13700 1 -- 172.21.15.16:0/3747269989 reap_dead start 2023-05-20T01:16:57.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.607+0000 7fabb1512700 1 --2- 172.21.15.16:0/3747269989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab98003c50 0x7fab98004040 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7faba800c110 tx=0x7fab98005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2135fe473aba7ae3 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:57.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.607+0000 7fab9ffff700 1 -- 172.21.15.16:0/3747269989 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba800ccb0 con 0x7fab98003c50 2023-05-20T01:16:57.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.607+0000 7fab9ffff700 1 -- 172.21.15.16:0/3747269989 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faba800ce10 con 0x7fab98003c50 2023-05-20T01:16:57.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.607+0000 7fab9ffff700 1 -- 172.21.15.16:0/3747269989 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba8005ba0 con 0x7fab98003c50 2023-05-20T01:16:57.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.607+0000 7fabb3776700 1 -- 172.21.15.16:0/3747269989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab98003c50 msgr2=0x7fab98004040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:57.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.607+0000 7fabb3776700 1 --2- 172.21.15.16:0/3747269989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab98003c50 0x7fab98004040 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7faba800c110 tx=0x7fab98005880 comp rx=0 tx=0).stop 2023-05-20T01:16:57.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.607+0000 7fabb3776700 1 -- 172.21.15.16:0/3747269989 shutdown_connections 2023-05-20T01:16:57.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.607+0000 7fabb3776700 1 --2- 172.21.15.16:0/3747269989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab98003c50 0x7fab98004040 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:57.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.607+0000 7fabb3776700 1 -- 172.21.15.16:0/3747269989 >> 172.21.15.16:0/3747269989 conn(0x7fabac06ae30 msgr2=0x7fabac09a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:57.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.608+0000 7fabb3776700 1 -- 172.21.15.16:0/3747269989 shutdown_connections 2023-05-20T01:16:57.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.608+0000 7fabb3776700 1 -- 172.21.15.16:0/3747269989 wait complete. 2023-05-20T01:16:57.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.609+0000 7fabb3776700 1 Processor -- start 2023-05-20T01:16:57.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.609+0000 7fabb3776700 1 -- start start 2023-05-20T01:16:57.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.609+0000 7fabb3776700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fab98003c50 0x7fabac158e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:57.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.609+0000 7fabb3776700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fabac15d050 0x7fabac15d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:57.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.609+0000 7fabb3776700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabac15b880 0x7fabac0b60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:57.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.610+0000 7fabb3776700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabac159370 con 0x7fabac15b880 2023-05-20T01:16:57.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.610+0000 7fabb3776700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabac15dbd0 con 0x7fab98003c50 2023-05-20T01:16:57.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.610+0000 7fabb3776700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabac15c290 con 0x7fabac15d050 2023-05-20T01:16:57.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.610+0000 7fabb1d13700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabac15b880 0x7fabac0b60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:57.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.610+0000 7fabb1d13700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabac15b880 0x7fabac0b60b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58680/0 (socket says 172.21.15.16:58680) 2023-05-20T01:16:57.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.610+0000 7fabb1d13700 1 -- 172.21.15.16:0/2792670345 learned_addr learned my addr 172.21.15.16:0/2792670345 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:57.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.610+0000 7fabb1512700 1 --2- 172.21.15.16:0/2792670345 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fab98003c50 0x7fabac158e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:57.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.610+0000 7fabb0d11700 1 --2- 172.21.15.16:0/2792670345 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fabac15d050 0x7fabac15d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:57.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.610+0000 7fabb1d13700 1 -- 172.21.15.16:0/2792670345 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fabac15d050 msgr2=0x7fabac15d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:57.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.610+0000 7fabb1d13700 1 --2- 172.21.15.16:0/2792670345 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fabac15d050 0x7fabac15d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:57.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.610+0000 7fabb1d13700 1 -- 172.21.15.16:0/2792670345 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fab98003c50 msgr2=0x7fabac158e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:57.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.610+0000 7fabb1d13700 1 --2- 172.21.15.16:0/2792670345 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fab98003c50 0x7fabac158e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:57.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.610+0000 7fabb1d13700 1 -- 172.21.15.16:0/2792670345 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabac0cc190 con 0x7fabac15b880 2023-05-20T01:16:57.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.610+0000 7fabb1d13700 1 --2- 172.21.15.16:0/2792670345 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabac15b880 0x7fabac0b60b0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fabac0cf740 tx=0x7faba400f490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:57.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.610+0000 7fab9dffb700 1 -- 172.21.15.16:0/2792670345 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba401c070 con 0x7fabac15b880 2023-05-20T01:16:57.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.610+0000 7fab9dffb700 1 -- 172.21.15.16:0/2792670345 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faba400f040 con 0x7fabac15b880 2023-05-20T01:16:57.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.610+0000 7fabb3776700 1 -- 172.21.15.16:0/2792670345 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabac0cbeb0 con 0x7fabac15b880 2023-05-20T01:16:57.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.610+0000 7fab9dffb700 1 -- 172.21.15.16:0/2792670345 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba400e040 con 0x7fabac15b880 2023-05-20T01:16:57.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.611+0000 7fabb3776700 1 -- 172.21.15.16:0/2792670345 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabac0d1680 con 0x7fabac15b880 2023-05-20T01:16:57.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.612+0000 7fab9dffb700 1 -- 172.21.15.16:0/2792670345 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7faba400fd10 con 0x7fabac15b880 2023-05-20T01:16:57.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.612+0000 7fabb3776700 1 -- 172.21.15.16:0/2792670345 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabac0747b0 con 0x7fabac15b880 2023-05-20T01:16:57.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.612+0000 7fab9dffb700 1 --2- 172.21.15.16:0/2792670345 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fab9006ace0 0x7fab9006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:57.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.612+0000 7fab9dffb700 1 -- 172.21.15.16:0/2792670345 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7faba4089410 con 0x7fabac15b880 2023-05-20T01:16:57.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.613+0000 7fabb1512700 1 --2- 172.21.15.16:0/2792670345 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fab9006ace0 0x7fab9006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:57.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.613+0000 7fabb1512700 1 --2- 172.21.15.16:0/2792670345 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fab9006ace0 0x7fab9006d190 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7faba800c110 tx=0x7faba8007000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:57.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.617+0000 7fab9dffb700 1 -- 172.21.15.16:0/2792670345 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7faba40541d0 con 0x7fabac15b880 2023-05-20T01:16:57.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.860+0000 7fabb3776700 1 -- 172.21.15.16:0/2792670345 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fabac15ae40 con 0x7fabac15b880 2023-05-20T01:16:57.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.860+0000 7fab9dffb700 1 -- 172.21.15.16:0/2792670345 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4673 (secure 0 0 0) 0x7faba40577f0 con 0x7fabac15b880 2023-05-20T01:16:57.864 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:16:57.864 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4273,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/1444124146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1444124146},{"type":"v1","addr":"172.21.15.132:6834","nonce":1444124146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4293,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4092119206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4092119206},{"type":"v1","addr":"172.21.15.16:6835","nonce":4092119206}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2162680372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":2162680372},{"type":"v1","addr":"172.21.15.132:6835","nonce":2162680372}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-20T01:16:28.316109+0000","modified":"2023-05-20T01:16:45.362572+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4299},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4299":{"gid":4299,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6837/2330551626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2330551626},{"type":"v1","addr":"172.21.15.16:6837","nonce":2330551626}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-05-20T01:16:57.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.862+0000 7fabb3776700 1 -- 172.21.15.16:0/2792670345 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fab9006ace0 msgr2=0x7fab9006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:57.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.862+0000 7fabb3776700 1 --2- 172.21.15.16:0/2792670345 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fab9006ace0 0x7fab9006d190 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7faba800c110 tx=0x7faba8007000 comp rx=0 tx=0).stop 2023-05-20T01:16:57.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.862+0000 7fabb3776700 1 -- 172.21.15.16:0/2792670345 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabac15b880 msgr2=0x7fabac0b60b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:57.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.862+0000 7fabb3776700 1 --2- 172.21.15.16:0/2792670345 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabac15b880 0x7fabac0b60b0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fabac0cf740 tx=0x7faba400f490 comp rx=0 tx=0).stop 2023-05-20T01:16:57.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.862+0000 7fabb3776700 1 -- 172.21.15.16:0/2792670345 shutdown_connections 2023-05-20T01:16:57.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.862+0000 7fabb3776700 1 --2- 172.21.15.16:0/2792670345 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabac15b880 0x7fabac0b60b0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:57.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.862+0000 7fabb3776700 1 --2- 172.21.15.16:0/2792670345 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fab98003c50 0x7fabac158e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:57.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.862+0000 7fabb3776700 1 --2- 172.21.15.16:0/2792670345 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fab9006ace0 0x7fab9006d190 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:57.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.862+0000 7fabb3776700 1 --2- 172.21.15.16:0/2792670345 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fabac15d050 0x7fabac15d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:57.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.862+0000 7fabb3776700 1 -- 172.21.15.16:0/2792670345 >> 172.21.15.16:0/2792670345 conn(0x7fabac06ae30 msgr2=0x7fabac15a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:57.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.862+0000 7fabb3776700 1 -- 172.21.15.16:0/2792670345 shutdown_connections 2023-05-20T01:16:57.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:57.862+0000 7fabb3776700 1 -- 172.21.15.16:0/2792670345 wait complete. 2023-05-20T01:16:57.868 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 8 2023-05-20T01:16:58.280 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-05-20T01:16:28.316109+0000', 'modified': '2023-05-20T01:16:45.362572+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4299}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4299': {'gid': 4299, 'name': 'a', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.16:6837/2330551626', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6836', 'nonce': 2330551626}, {'type': 'v1', 'addr': '172.21.15.16:6837', 'nonce': 2330551626}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [4, 5], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-05-20T01:16:58.280 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-05-20T01:16:58.281 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-05-20T01:16:58.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.409+0000 7fd4b3362700 1 Processor -- start 2023-05-20T01:16:58.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.410+0000 7fd4b3362700 1 -- start start 2023-05-20T01:16:58.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.410+0000 7fd4b3362700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd4ac159000 0x7fd4ac06cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:58.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.410+0000 7fd4b3362700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fd4ac0cb320 0x7fd4ac06d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:58.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.410+0000 7fd4b3362700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd4ac0b1b00 0x7fd4ac0cf6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:58.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.410+0000 7fd4b3362700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4ac0ca350 con 0x7fd4ac0b1b00 2023-05-20T01:16:58.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.410+0000 7fd4b3362700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4ac15cac0 con 0x7fd4ac0cb320 2023-05-20T01:16:58.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.410+0000 7fd4b3362700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4ac15cda0 con 0x7fd4ac159000 2023-05-20T01:16:58.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.411+0000 7fd4b18ff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd4ac0b1b00 0x7fd4ac0cf6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:58.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.411+0000 7fd4b18ff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd4ac0b1b00 0x7fd4ac0cf6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58688/0 (socket says 172.21.15.16:58688) 2023-05-20T01:16:58.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.411+0000 7fd4b18ff700 1 -- 172.21.15.16:0/1409489061 learned_addr learned my addr 172.21.15.16:0/1409489061 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:58.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.411+0000 7fd4b08fd700 1 --2- 172.21.15.16:0/1409489061 >> v2:172.21.15.132:3300/0 conn(0x7fd4ac0cb320 0x7fd4ac06d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:58.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.411+0000 7fd4b10fe700 1 --2- 172.21.15.16:0/1409489061 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd4ac159000 0x7fd4ac06cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:58.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.411+0000 7fd4b18ff700 1 -- 172.21.15.16:0/1409489061 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd4ac159000 msgr2=0x7fd4ac06cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:58.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.411+0000 7fd4b18ff700 1 --2- 172.21.15.16:0/1409489061 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd4ac159000 0x7fd4ac06cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:58.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.411+0000 7fd4b18ff700 1 -- 172.21.15.16:0/1409489061 >> v2:172.21.15.132:3300/0 conn(0x7fd4ac0cb320 msgr2=0x7fd4ac06d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:58.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.411+0000 7fd4b18ff700 1 --2- 172.21.15.16:0/1409489061 >> v2:172.21.15.132:3300/0 conn(0x7fd4ac0cb320 0x7fd4ac06d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:58.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.411+0000 7fd4b18ff700 1 -- 172.21.15.16:0/1409489061 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4ac15d080 con 0x7fd4ac0b1b00 2023-05-20T01:16:58.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.411+0000 7fd4b18ff700 1 --2- 172.21.15.16:0/1409489061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4ac0b1b00 0x7fd4ac0cf6e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fd4ac0d07f0 tx=0x7fd4a400be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e412623a5597ced8 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:58.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.411+0000 7fd4abfff700 1 -- 172.21.15.16:0/1409489061 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a400db60 con 0x7fd4ac0b1b00 2023-05-20T01:16:58.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.411+0000 7fd4abfff700 1 -- 172.21.15.16:0/1409489061 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4a400dcc0 con 0x7fd4ac0b1b00 2023-05-20T01:16:58.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.411+0000 7fd4abfff700 1 -- 172.21.15.16:0/1409489061 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a4017630 con 0x7fd4ac0b1b00 2023-05-20T01:16:58.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.412+0000 7fd4b3362700 1 -- 172.21.15.16:0/1409489061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4ac0b1b00 msgr2=0x7fd4ac0cf6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:58.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.412+0000 7fd4b3362700 1 --2- 172.21.15.16:0/1409489061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4ac0b1b00 0x7fd4ac0cf6e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fd4ac0d07f0 tx=0x7fd4a400be70 comp rx=0 tx=0).stop 2023-05-20T01:16:58.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.412+0000 7fd4b3362700 1 -- 172.21.15.16:0/1409489061 shutdown_connections 2023-05-20T01:16:58.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.412+0000 7fd4b3362700 1 --2- 172.21.15.16:0/1409489061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4ac0b1b00 0x7fd4ac0cf6e0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:58.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.412+0000 7fd4b3362700 1 --2- 172.21.15.16:0/1409489061 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd4ac159000 0x7fd4ac06cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:58.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.412+0000 7fd4b3362700 1 --2- 172.21.15.16:0/1409489061 >> v2:172.21.15.132:3300/0 conn(0x7fd4ac0cb320 0x7fd4ac06d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:58.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.412+0000 7fd4b3362700 1 -- 172.21.15.16:0/1409489061 >> 172.21.15.16:0/1409489061 conn(0x7fd4ac0661b0 msgr2=0x7fd4ac066580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:58.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.412+0000 7fd4b3362700 1 -- 172.21.15.16:0/1409489061 shutdown_connections 2023-05-20T01:16:58.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.412+0000 7fd4b3362700 1 -- 172.21.15.16:0/1409489061 wait complete. 2023-05-20T01:16:58.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.413+0000 7fd4b3362700 1 Processor -- start 2023-05-20T01:16:58.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.413+0000 7fd4b3362700 1 -- start start 2023-05-20T01:16:58.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.414+0000 7fd4b3362700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4ac0b1b00 0x7fd4ac1ef460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:58.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.414+0000 7fd4b3362700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd4ac0cb320 0x7fd4ac0b3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:58.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.414+0000 7fd4b3362700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd4ac159000 0x7fd4ac0b43f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:58.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.414+0000 7fd4b3362700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4ac1ef1d0 con 0x7fd4ac0b1b00 2023-05-20T01:16:58.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.414+0000 7fd4b3362700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4ac0b30f0 con 0x7fd4ac159000 2023-05-20T01:16:58.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.414+0000 7fd4b3362700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4ac0b33d0 con 0x7fd4ac0cb320 2023-05-20T01:16:58.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.414+0000 7fd4b10fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4ac0b1b00 0x7fd4ac1ef460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:58.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.414+0000 7fd4b10fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4ac0b1b00 0x7fd4ac1ef460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58690/0 (socket says 172.21.15.16:58690) 2023-05-20T01:16:58.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.414+0000 7fd4b10fe700 1 -- 172.21.15.16:0/3293483820 learned_addr learned my addr 172.21.15.16:0/3293483820 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:58.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.414+0000 7fd4b18ff700 1 --2- 172.21.15.16:0/3293483820 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd4ac159000 0x7fd4ac0b43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:58.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.414+0000 7fd4b08fd700 1 --2- 172.21.15.16:0/3293483820 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd4ac0cb320 0x7fd4ac0b3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:58.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.414+0000 7fd4b10fe700 1 -- 172.21.15.16:0/3293483820 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd4ac0cb320 msgr2=0x7fd4ac0b3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:58.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.414+0000 7fd4b10fe700 1 --2- 172.21.15.16:0/3293483820 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd4ac0cb320 0x7fd4ac0b3eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:58.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.414+0000 7fd4b10fe700 1 -- 172.21.15.16:0/3293483820 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd4ac159000 msgr2=0x7fd4ac0b43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:58.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.414+0000 7fd4b10fe700 1 --2- 172.21.15.16:0/3293483820 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd4ac159000 0x7fd4ac0b43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:58.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.414+0000 7fd4b10fe700 1 -- 172.21.15.16:0/3293483820 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4a400b7f0 con 0x7fd4ac0b1b00 2023-05-20T01:16:58.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.414+0000 7fd4b08fd700 1 --2- 172.21.15.16:0/3293483820 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd4ac0cb320 0x7fd4ac0b3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:16:58.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.414+0000 7fd4b10fe700 1 --2- 172.21.15.16:0/3293483820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4ac0b1b00 0x7fd4ac1ef460 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd49c00efc0 tx=0x7fd49c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:58.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.415+0000 7fd4a9ffb700 1 -- 172.21.15.16:0/3293483820 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd49c009b80 con 0x7fd4ac0b1b00 2023-05-20T01:16:58.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.415+0000 7fd4a9ffb700 1 -- 172.21.15.16:0/3293483820 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd49c00f040 con 0x7fd4ac0b1b00 2023-05-20T01:16:58.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.415+0000 7fd4b3362700 1 -- 172.21.15.16:0/3293483820 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4ac15d880 con 0x7fd4ac0b1b00 2023-05-20T01:16:58.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.415+0000 7fd4a9ffb700 1 -- 172.21.15.16:0/3293483820 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd49c014480 con 0x7fd4ac0b1b00 2023-05-20T01:16:58.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.415+0000 7fd4b3362700 1 -- 172.21.15.16:0/3293483820 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4ac15d360 con 0x7fd4ac0b1b00 2023-05-20T01:16:58.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.416+0000 7fd4a9ffb700 1 -- 172.21.15.16:0/3293483820 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd49c0145e0 con 0x7fd4ac0b1b00 2023-05-20T01:16:58.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.417+0000 7fd4a9ffb700 1 --2- 172.21.15.16:0/3293483820 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd49806ace0 0x7fd49806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:58.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.417+0000 7fd4a9ffb700 1 -- 172.21.15.16:0/3293483820 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fd49c08a2c0 con 0x7fd4ac0b1b00 2023-05-20T01:16:58.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.417+0000 7fd4b08fd700 1 --2- 172.21.15.16:0/3293483820 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd49806ace0 0x7fd49806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:58.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.417+0000 7fd4b08fd700 1 --2- 172.21.15.16:0/3293483820 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd49806ace0 0x7fd49806d190 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd4ac158460 tx=0x7fd4a0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:58.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.419+0000 7fd4bf4eeb80 1 Processor -- start 2023-05-20T01:16:58.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.419+0000 7fd4bf4eeb80 1 -- start start 2023-05-20T01:16:58.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.419+0000 7fd4bf4eeb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558b7e2f7e60 0x558b7e317690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:58.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.419+0000 7fd4bf4eeb80 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x558b7e2ce870 0x558b7e2f8230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:58.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.419+0000 7fd4bf4eeb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b7e2d0460 0x558b7e2d08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:58.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.419+0000 7fd4bf4eeb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b7e1d6010 con 0x558b7e2d0460 2023-05-20T01:16:58.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.419+0000 7fd4bf4eeb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b7e228850 con 0x558b7e2f7e60 2023-05-20T01:16:58.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.419+0000 7fd4bf4eeb80 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x558b7e221ce0 con 0x558b7e2ce870 2023-05-20T01:16:58.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.419+0000 7fd4b18ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b7e2d0460 0x558b7e2d08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:58.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.420+0000 7fd4b18ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b7e2d0460 0x558b7e2d08d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58702/0 (socket says 172.21.15.16:58702) 2023-05-20T01:16:58.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.420+0000 7fd4b18ff700 1 -- 172.21.15.16:0/2101449666 learned_addr learned my addr 172.21.15.16:0/2101449666 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:58.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.420+0000 7fd4b10fe700 1 --2- 172.21.15.16:0/2101449666 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558b7e2f7e60 0x558b7e317690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:58.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.420+0000 7fd4b08fd700 1 --2- 172.21.15.16:0/2101449666 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x558b7e2ce870 0x558b7e2f8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:58.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.420+0000 7fd4b18ff700 1 -- 172.21.15.16:0/2101449666 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x558b7e2ce870 msgr2=0x558b7e2f8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:58.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.420+0000 7fd4b18ff700 1 --2- 172.21.15.16:0/2101449666 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x558b7e2ce870 0x558b7e2f8230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:58.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.420+0000 7fd4b18ff700 1 -- 172.21.15.16:0/2101449666 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558b7e2f7e60 msgr2=0x558b7e317690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:58.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.420+0000 7fd4b18ff700 1 --2- 172.21.15.16:0/2101449666 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558b7e2f7e60 0x558b7e317690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:58.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.420+0000 7fd4b18ff700 1 -- 172.21.15.16:0/2101449666 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558b7e0a35e0 con 0x558b7e2d0460 2023-05-20T01:16:58.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.420+0000 7fd4b10fe700 1 --2- 172.21.15.16:0/2101449666 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558b7e2f7e60 0x558b7e317690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-05-20T01:16:58.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.420+0000 7fd4b18ff700 1 --2- 172.21.15.16:0/2101449666 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b7e2d0460 0x558b7e2d08d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd4a4008000 tx=0x7fd4a4010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=877410c60c2481b8 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:58.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.420+0000 7fd492ffd700 1 -- 172.21.15.16:0/2101449666 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a4025070 con 0x558b7e2d0460 2023-05-20T01:16:58.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.420+0000 7fd492ffd700 1 -- 172.21.15.16:0/2101449666 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4a4006d20 con 0x558b7e2d0460 2023-05-20T01:16:58.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.420+0000 7fd492ffd700 1 -- 172.21.15.16:0/2101449666 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a4004040 con 0x558b7e2d0460 2023-05-20T01:16:58.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.421+0000 7fd4bf4eeb80 1 -- 172.21.15.16:0/2101449666 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b7e2d0460 msgr2=0x558b7e2d08d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:58.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.421+0000 7fd4bf4eeb80 1 --2- 172.21.15.16:0/2101449666 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b7e2d0460 0x558b7e2d08d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd4a4008000 tx=0x7fd4a4010000 comp rx=0 tx=0).stop 2023-05-20T01:16:58.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.421+0000 7fd4bf4eeb80 1 -- 172.21.15.16:0/2101449666 shutdown_connections 2023-05-20T01:16:58.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.421+0000 7fd4bf4eeb80 1 --2- 172.21.15.16:0/2101449666 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b7e2d0460 0x558b7e2d08d0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:58.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.421+0000 7fd4bf4eeb80 1 --2- 172.21.15.16:0/2101449666 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558b7e2f7e60 0x558b7e317690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:58.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.421+0000 7fd4bf4eeb80 1 --2- 172.21.15.16:0/2101449666 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x558b7e2ce870 0x558b7e2f8230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:58.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.421+0000 7fd4bf4eeb80 1 -- 172.21.15.16:0/2101449666 >> 172.21.15.16:0/2101449666 conn(0x558b7e2c5f30 msgr2=0x558b7e316210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:58.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.421+0000 7fd4bf4eeb80 1 -- 172.21.15.16:0/2101449666 shutdown_connections 2023-05-20T01:16:58.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.421+0000 7fd4bf4eeb80 1 -- 172.21.15.16:0/2101449666 wait complete. 2023-05-20T01:16:58.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.421+0000 7fd4bf4eeb80 1 -- start start 2023-05-20T01:16:58.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.422+0000 7fd4bf4eeb80 1 Processor -- start 2023-05-20T01:16:58.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.422+0000 7fd4bf4eeb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b7e2ce870 0x558b7e2e6f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:58.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.422+0000 7fd4bf4eeb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558b7e2d0460 0x558b7e2ea130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:58.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.423+0000 7fd4bf4eeb80 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x558b7e2f7e60 0x558b7e2e8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:58.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.423+0000 7fd4bf4eeb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b7e1c3890 con 0x558b7e2ce870 2023-05-20T01:16:58.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.423+0000 7fd4bf4eeb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b7e1c3530 con 0x558b7e2d0460 2023-05-20T01:16:58.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.423+0000 7fd4bf4eeb80 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x558b7e0a58d0 con 0x558b7e2f7e60 2023-05-20T01:16:58.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.423+0000 7fd4bf4eeb80 10 client.0 _ll_register_callbacks cb 0x558b7e1e1730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-05-20T01:16:58.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.423+0000 7fd4b10fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b7e2ce870 0x558b7e2e6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:58.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.423+0000 7fd4b10fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b7e2ce870 0x558b7e2e6f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58704/0 (socket says 172.21.15.16:58704) 2023-05-20T01:16:58.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.423+0000 7fd4b10fe700 1 -- 172.21.15.16:0/3479986805 learned_addr learned my addr 172.21.15.16:0/3479986805 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:58.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.423+0000 7fd4b08fd700 1 --2- 172.21.15.16:0/3479986805 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558b7e2d0460 0x558b7e2ea130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:58.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.423+0000 7fd4b18ff700 1 --2- 172.21.15.16:0/3479986805 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x558b7e2f7e60 0x558b7e2e8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:58.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.423+0000 7fd4b10fe700 1 -- 172.21.15.16:0/3479986805 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x558b7e2f7e60 msgr2=0x558b7e2e8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:58.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.423+0000 7fd4b10fe700 1 --2- 172.21.15.16:0/3479986805 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x558b7e2f7e60 0x558b7e2e8b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:58.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.423+0000 7fd4b10fe700 1 -- 172.21.15.16:0/3479986805 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558b7e2d0460 msgr2=0x558b7e2ea130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:58.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.423+0000 7fd4b10fe700 1 --2- 172.21.15.16:0/3479986805 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558b7e2d0460 0x558b7e2ea130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:58.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.423+0000 7fd4b10fe700 1 -- 172.21.15.16:0/3479986805 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4ac15d080 con 0x558b7e2ce870 2023-05-20T01:16:58.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.423+0000 7fd4b08fd700 1 --2- 172.21.15.16:0/3479986805 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558b7e2d0460 0x558b7e2ea130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:16:58.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.423+0000 7fd4b10fe700 1 --2- 172.21.15.16:0/3479986805 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b7e2ce870 0x558b7e2e6f40 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd49c00c7a0 tx=0x7fd49c0612a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e9bc4186e63620 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:58.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.423+0000 7fd48bfff700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-05-20T01:16:58.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.423+0000 7fd48bfff700 1 -- 172.21.15.16:0/3479986805 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd49c024bd0 con 0x558b7e2ce870 2023-05-20T01:16:58.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.423+0000 7fd48bfff700 1 -- 172.21.15.16:0/3479986805 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fd49c024d30 con 0x558b7e2ce870 2023-05-20T01:16:58.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.423+0000 7fd4bf4eeb80 1 -- 172.21.15.16:0/3479986805 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558b7e2c36f0 con 0x558b7e2ce870 2023-05-20T01:16:58.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.423+0000 7fd48bfff700 1 -- 172.21.15.16:0/3479986805 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd49c024ef0 con 0x558b7e2ce870 2023-05-20T01:16:58.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.423+0000 7fd4bf4eeb80 10 client.4884 fetch_fsmap learned FSMap version 8 2023-05-20T01:16:58.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.423+0000 7fd4bf4eeb80 1 -- 172.21.15.16:0/3479986805 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x558b7e1e8140 con 0x558b7e2ce870 2023-05-20T01:16:58.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.423+0000 7fd48bfff700 1 -- 172.21.15.16:0/3479986805 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558b7e2c36f0 con 0x558b7e2ce870 2023-05-20T01:16:58.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.424+0000 7fd48bfff700 1 -- 172.21.15.16:0/3479986805 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 8) v1 ==== 3122+0+0 (secure 0 0 0) 0x7fd49c054bc0 con 0x558b7e2ce870 2023-05-20T01:16:58.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.424+0000 7fd4bf4eeb80 10 client.4884 fetch_fsmap finished waiting for FSMap version 8 2023-05-20T01:16:58.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.424+0000 7fd4bf4eeb80 10 client.4884 resolve_mds: resolved name 'a' to daemon mds.a 2023-05-20T01:16:58.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.424+0000 7fd4bf4eeb80 20 client.4884 populate_metadata read hostname 'smithi016' 2023-05-20T01:16:58.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.424+0000 7fd4bf4eeb80 1 --2- 172.21.15.16:0/3479986805 >> [v2:172.21.15.16:6836/2330551626,v1:172.21.15.16:6837/2330551626] conn(0x558b7e2f55f0 0x558b7e2ced10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:58.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.424+0000 7fd4bf4eeb80 4 client.4884 mds_command: new command op to 4299 tid=0[{"prefix": "get_command_descriptions"}] 2023-05-20T01:16:58.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.424+0000 7fd4bf4eeb80 1 -- 172.21.15.16:0/3479986805 --> [v2:172.21.15.16:6836/2330551626,v1:172.21.15.16:6837/2330551626] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558b7e1e4050 con 0x558b7e2f55f0 2023-05-20T01:16:58.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.424+0000 7fd4b08fd700 1 --2- 172.21.15.16:0/3479986805 >> [v2:172.21.15.16:6836/2330551626,v1:172.21.15.16:6837/2330551626] conn(0x558b7e2f55f0 0x558b7e2ced10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:58.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.424+0000 7fd48bfff700 10 client.4884 ms_handle_connect on v2:172.21.15.16:6836/2330551626 2023-05-20T01:16:58.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.424+0000 7fd4b08fd700 1 --2- 172.21.15.16:0/3479986805 >> [v2:172.21.15.16:6836/2330551626,v1:172.21.15.16:6837/2330551626] conn(0x558b7e2f55f0 0x558b7e2ced10 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2db228496053093 server_cookie=a58203b15f7b5b1f in_seq=0 out_seq=0 2023-05-20T01:16:58.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.425+0000 7fd48bfff700 1 -- 172.21.15.16:0/3479986805 <== mds.0 v2:172.21.15.16:6836/2330551626 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x558b7e1e4050 con 0x558b7e2f55f0 2023-05-20T01:16:58.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.425+0000 7fd48bfff700 10 client.4884 handle_command_reply: tid=0 2023-05-20T01:16:58.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.425+0000 7fd4bf4eeb80 1 client.4884 shutdown 2023-05-20T01:16:58.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.425+0000 7fd4bf4eeb80 1 -- 172.21.15.16:0/3479986805 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b7e2ce870 msgr2=0x558b7e2e6f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:58.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.425+0000 7fd4bf4eeb80 1 --2- 172.21.15.16:0/3479986805 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b7e2ce870 0x558b7e2e6f40 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd49c00c7a0 tx=0x7fd49c0612a0 comp rx=0 tx=0).stop 2023-05-20T01:16:58.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.425+0000 7fd4bf4eeb80 1 -- 172.21.15.16:0/3479986805 shutdown_connections 2023-05-20T01:16:58.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.425+0000 7fd4bf4eeb80 1 --2- 172.21.15.16:0/3479986805 >> [v2:172.21.15.16:6836/2330551626,v1:172.21.15.16:6837/2330551626] conn(0x558b7e2f55f0 0x558b7e2ced10 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:58.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.425+0000 7fd4bf4eeb80 1 --2- 172.21.15.16:0/3479986805 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x558b7e2f7e60 0x558b7e2e8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:58.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.425+0000 7fd4bf4eeb80 1 --2- 172.21.15.16:0/3479986805 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b7e2ce870 0x558b7e2e6f40 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:58.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.425+0000 7fd48bfff700 0 client.4884 ms_handle_reset on v2:172.21.15.16:6836/2330551626 2023-05-20T01:16:58.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.425+0000 7fd4bf4eeb80 1 --2- 172.21.15.16:0/3479986805 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558b7e2d0460 0x558b7e2ea130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:58.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.425+0000 7fd4bf4eeb80 1 -- 172.21.15.16:0/3479986805 >> 172.21.15.16:0/3479986805 conn(0x558b7e2c5f30 msgr2=0x558b7e2ca120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:58.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.425+0000 7fd4bf4eeb80 1 -- 172.21.15.16:0/3479986805 shutdown_connections 2023-05-20T01:16:58.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.425+0000 7fd4bf4eeb80 1 -- 172.21.15.16:0/3479986805 wait complete. 2023-05-20T01:16:58.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.425+0000 7fd4bf4eeb80 20 client.4884 trim_cache size 0 max 16384 2023-05-20T01:16:58.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.443+0000 7fd4bf4eeb80 1 Processor -- start 2023-05-20T01:16:58.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.443+0000 7fd4bf4eeb80 1 -- start start 2023-05-20T01:16:58.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.443+0000 7fd4bf4eeb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b7e2ce870 0x558b7e311aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:58.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.443+0000 7fd4bf4eeb80 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x558b7e2d0460 0x558b7e2e64c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:58.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.443+0000 7fd4bf4eeb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558b7e2f7e60 0x558b7e2f46f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:58.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.443+0000 7fd4bf4eeb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b7e2e6a00 con 0x558b7e2ce870 2023-05-20T01:16:58.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.443+0000 7fd4bf4eeb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b7e2e7670 con 0x558b7e2f7e60 2023-05-20T01:16:58.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.443+0000 7fd4bf4eeb80 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x558b7e2e8ea0 con 0x558b7e2d0460 2023-05-20T01:16:58.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.444+0000 7fd4b10fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b7e2ce870 0x558b7e311aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:58.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.444+0000 7fd4b10fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b7e2ce870 0x558b7e311aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58712/0 (socket says 172.21.15.16:58712) 2023-05-20T01:16:58.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.444+0000 7fd4b10fe700 1 -- 172.21.15.16:0/4224168048 learned_addr learned my addr 172.21.15.16:0/4224168048 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:58.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.444+0000 7fd4b10fe700 1 -- 172.21.15.16:0/4224168048 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x558b7e2d0460 msgr2=0x558b7e2e64c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:58.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.444+0000 7fd4b10fe700 1 --2- 172.21.15.16:0/4224168048 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x558b7e2d0460 0x558b7e2e64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:58.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.444+0000 7fd4b10fe700 1 -- 172.21.15.16:0/4224168048 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558b7e2f7e60 msgr2=0x558b7e2f46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:58.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.444+0000 7fd4b18ff700 1 --2- 172.21.15.16:0/4224168048 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558b7e2f7e60 0x558b7e2f46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:58.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.444+0000 7fd4b10fe700 1 --2- 172.21.15.16:0/4224168048 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558b7e2f7e60 0x558b7e2f46f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:58.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.444+0000 7fd4b10fe700 1 -- 172.21.15.16:0/4224168048 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558b7e1e7e90 con 0x558b7e2ce870 2023-05-20T01:16:58.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.444+0000 7fd4b10fe700 1 --2- 172.21.15.16:0/4224168048 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b7e2ce870 0x558b7e311aa0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fd49c009720 tx=0x7fd49c06f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c38aac127c268dea server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:58.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.444+0000 7fd4937fe700 1 -- 172.21.15.16:0/4224168048 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd49c060ea0 con 0x558b7e2ce870 2023-05-20T01:16:58.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.444+0000 7fd4937fe700 1 -- 172.21.15.16:0/4224168048 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd49c054bc0 con 0x558b7e2ce870 2023-05-20T01:16:58.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.444+0000 7fd4937fe700 1 -- 172.21.15.16:0/4224168048 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd49c024dd0 con 0x558b7e2ce870 2023-05-20T01:16:58.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.445+0000 7fd4bf4eeb80 1 -- 172.21.15.16:0/4224168048 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b7e2ce870 msgr2=0x558b7e311aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:58.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.445+0000 7fd4bf4eeb80 1 --2- 172.21.15.16:0/4224168048 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b7e2ce870 0x558b7e311aa0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fd49c009720 tx=0x7fd49c06f000 comp rx=0 tx=0).stop 2023-05-20T01:16:58.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.445+0000 7fd4bf4eeb80 1 -- 172.21.15.16:0/4224168048 shutdown_connections 2023-05-20T01:16:58.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.445+0000 7fd4bf4eeb80 1 --2- 172.21.15.16:0/4224168048 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558b7e2f7e60 0x558b7e2f46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:58.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.445+0000 7fd4bf4eeb80 1 --2- 172.21.15.16:0/4224168048 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b7e2ce870 0x558b7e311aa0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:58.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.445+0000 7fd4bf4eeb80 1 --2- 172.21.15.16:0/4224168048 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x558b7e2d0460 0x558b7e2e64c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:58.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.445+0000 7fd4bf4eeb80 1 -- 172.21.15.16:0/4224168048 >> 172.21.15.16:0/4224168048 conn(0x558b7e2c5f30 msgr2=0x558b7e33c410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:58.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.445+0000 7fd4bf4eeb80 1 -- 172.21.15.16:0/4224168048 shutdown_connections 2023-05-20T01:16:58.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.445+0000 7fd4bf4eeb80 1 -- 172.21.15.16:0/4224168048 wait complete. 2023-05-20T01:16:58.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.445+0000 7fd4bf4eeb80 1 -- start start 2023-05-20T01:16:58.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.445+0000 7fd4bf4eeb80 1 Processor -- start 2023-05-20T01:16:58.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.446+0000 7fd4bf4eeb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b7e2ce870 0x558b7e32baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:58.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.446+0000 7fd4bf4eeb80 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x558b7e2d0460 0x558b7e319980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:58.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.446+0000 7fd4bf4eeb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558b7e2f7e60 0x558b7e2ca120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:58.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.446+0000 7fd4bf4eeb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b7e32cad0 con 0x558b7e2ce870 2023-05-20T01:16:58.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.446+0000 7fd4bf4eeb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b7e2e5c70 con 0x558b7e2f7e60 2023-05-20T01:16:58.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.446+0000 7fd4bf4eeb80 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x558b7e317c20 con 0x558b7e2d0460 2023-05-20T01:16:58.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.446+0000 7fd4bf4eeb80 10 client.0 _ll_register_callbacks cb 0x7fd49c061140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-05-20T01:16:58.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.446+0000 7fd4b10fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b7e2ce870 0x558b7e32baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:58.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.446+0000 7fd4b10fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b7e2ce870 0x558b7e32baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58718/0 (socket says 172.21.15.16:58718) 2023-05-20T01:16:58.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.446+0000 7fd4b10fe700 1 -- 172.21.15.16:0/1247526411 learned_addr learned my addr 172.21.15.16:0/1247526411 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:58.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.446+0000 7fd4b18ff700 1 --2- 172.21.15.16:0/1247526411 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558b7e2f7e60 0x558b7e2ca120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:58.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.446+0000 7fd4b08fd700 1 --2- 172.21.15.16:0/1247526411 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x558b7e2d0460 0x558b7e319980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:58.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.447+0000 7fd4b10fe700 1 -- 172.21.15.16:0/1247526411 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x558b7e2d0460 msgr2=0x558b7e319980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:58.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.447+0000 7fd4b10fe700 1 --2- 172.21.15.16:0/1247526411 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x558b7e2d0460 0x558b7e319980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:58.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.447+0000 7fd4b10fe700 1 -- 172.21.15.16:0/1247526411 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558b7e2f7e60 msgr2=0x558b7e2ca120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:58.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.447+0000 7fd4b10fe700 1 --2- 172.21.15.16:0/1247526411 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558b7e2f7e60 0x558b7e2ca120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:58.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.447+0000 7fd4b10fe700 1 -- 172.21.15.16:0/1247526411 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558b7e1e8140 con 0x558b7e2ce870 2023-05-20T01:16:58.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.447+0000 7fd4b10fe700 1 --2- 172.21.15.16:0/1247526411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b7e2ce870 0x558b7e32baf0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fd49c0587b0 tx=0x7fd49c088300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=204043ba353c6625 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:58.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.447+0000 7fd48bfff700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-05-20T01:16:58.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.447+0000 7fd48bfff700 1 -- 172.21.15.16:0/1247526411 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd49c054d30 con 0x558b7e2ce870 2023-05-20T01:16:58.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.447+0000 7fd48bfff700 1 -- 172.21.15.16:0/1247526411 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fd49c072b60 con 0x558b7e2ce870 2023-05-20T01:16:58.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.447+0000 7fd4bf4eeb80 1 -- 172.21.15.16:0/1247526411 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd4a4006d20 con 0x558b7e2ce870 2023-05-20T01:16:58.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.447+0000 7fd48bfff700 1 -- 172.21.15.16:0/1247526411 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd49c072d20 con 0x558b7e2ce870 2023-05-20T01:16:58.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.447+0000 7fd4bf4eeb80 10 client.4890 fetch_fsmap learned FSMap version 8 2023-05-20T01:16:58.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.447+0000 7fd4bf4eeb80 1 -- 172.21.15.16:0/1247526411 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x558b7e038c10 con 0x558b7e2ce870 2023-05-20T01:16:58.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.447+0000 7fd48bfff700 1 -- 172.21.15.16:0/1247526411 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4a4006d20 con 0x558b7e2ce870 2023-05-20T01:16:58.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.447+0000 7fd48bfff700 1 -- 172.21.15.16:0/1247526411 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 8) v1 ==== 3122+0+0 (secure 0 0 0) 0x7fd49c077af0 con 0x558b7e2ce870 2023-05-20T01:16:58.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.447+0000 7fd4bf4eeb80 10 client.4890 fetch_fsmap finished waiting for FSMap version 8 2023-05-20T01:16:58.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.447+0000 7fd4bf4eeb80 10 client.4890 resolve_mds: resolved name 'a' to daemon mds.a 2023-05-20T01:16:58.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.447+0000 7fd4bf4eeb80 20 client.4890 populate_metadata read hostname 'smithi016' 2023-05-20T01:16:58.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.447+0000 7fd4bf4eeb80 1 --2- 172.21.15.16:0/1247526411 >> [v2:172.21.15.16:6836/2330551626,v1:172.21.15.16:6837/2330551626] conn(0x558b7e2f4b10 0x558b7e2ca660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:58.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.447+0000 7fd4bf4eeb80 4 client.4890 mds_command: new command op to 4299 tid=0[{"prefix": "status"}] 2023-05-20T01:16:58.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.447+0000 7fd4bf4eeb80 1 -- 172.21.15.16:0/1247526411 --> [v2:172.21.15.16:6836/2330551626,v1:172.21.15.16:6837/2330551626] -- command(tid 0: {"prefix": "status"}) v1 -- 0x558b7e1f0610 con 0x558b7e2f4b10 2023-05-20T01:16:58.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.448+0000 7fd4b08fd700 1 --2- 172.21.15.16:0/1247526411 >> [v2:172.21.15.16:6836/2330551626,v1:172.21.15.16:6837/2330551626] conn(0x558b7e2f4b10 0x558b7e2ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:58.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.448+0000 7fd48bfff700 10 client.4890 ms_handle_connect on v2:172.21.15.16:6836/2330551626 2023-05-20T01:16:58.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.448+0000 7fd4b08fd700 1 --2- 172.21.15.16:0/1247526411 >> [v2:172.21.15.16:6836/2330551626,v1:172.21.15.16:6837/2330551626] conn(0x558b7e2f4b10 0x558b7e2ca660 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ec7219fabc6b49a server_cookie=2924c29534332b92 in_seq=0 out_seq=0 2023-05-20T01:16:58.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.448+0000 7fd48bfff700 1 -- 172.21.15.16:0/1247526411 <== mds.0 v2:172.21.15.16:6836/2330551626 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x558b7e1f0610 con 0x558b7e2f4b10 2023-05-20T01:16:58.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.448+0000 7fd48bfff700 10 client.4890 handle_command_reply: tid=0 2023-05-20T01:16:58.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.448+0000 7fd4bf4eeb80 1 client.4890 shutdown 2023-05-20T01:16:58.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.448+0000 7fd4bf4eeb80 1 -- 172.21.15.16:0/1247526411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b7e2ce870 msgr2=0x558b7e32baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:58.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.448+0000 7fd4bf4eeb80 1 --2- 172.21.15.16:0/1247526411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b7e2ce870 0x558b7e32baf0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fd49c0587b0 tx=0x7fd49c088300 comp rx=0 tx=0).stop 2023-05-20T01:16:58.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.448+0000 7fd4bf4eeb80 1 -- 172.21.15.16:0/1247526411 shutdown_connections 2023-05-20T01:16:58.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.448+0000 7fd4bf4eeb80 1 --2- 172.21.15.16:0/1247526411 >> [v2:172.21.15.16:6836/2330551626,v1:172.21.15.16:6837/2330551626] conn(0x558b7e2f4b10 0x558b7e2ca660 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:58.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.448+0000 7fd4bf4eeb80 1 --2- 172.21.15.16:0/1247526411 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558b7e2f7e60 0x558b7e2ca120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:58.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.448+0000 7fd4bf4eeb80 1 --2- 172.21.15.16:0/1247526411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b7e2ce870 0x558b7e32baf0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:58.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.448+0000 7fd48bfff700 0 client.4890 ms_handle_reset on v2:172.21.15.16:6836/2330551626 2023-05-20T01:16:58.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.448+0000 7fd4bf4eeb80 1 --2- 172.21.15.16:0/1247526411 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x558b7e2d0460 0x558b7e319980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:58.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.448+0000 7fd4bf4eeb80 1 -- 172.21.15.16:0/1247526411 >> 172.21.15.16:0/1247526411 conn(0x558b7e2c5f30 msgr2=0x558b7e310af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:58.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.448+0000 7fd4bf4eeb80 1 -- 172.21.15.16:0/1247526411 shutdown_connections 2023-05-20T01:16:58.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.448+0000 7fd4bf4eeb80 1 -- 172.21.15.16:0/1247526411 wait complete. 2023-05-20T01:16:58.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.448+0000 7fd4bf4eeb80 20 client.4890 trim_cache size 0 max 16384 2023-05-20T01:16:58.463 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-05-20T01:16:58.463 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "c2245cc5-ecab-4087-a59b-bc81db901676", 2023-05-20T01:16:58.463 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2023-05-20T01:16:58.463 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 4299, 2023-05-20T01:16:58.463 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2023-05-20T01:16:58.463 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2023-05-20T01:16:58.463 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2023-05-20T01:16:58.464 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 30.131428330999999, 2023-05-20T01:16:58.464 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 8, 2023-05-20T01:16:58.464 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 25, 2023-05-20T01:16:58.464 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 25, 2023-05-20T01:16:58.464 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 35.499243102999998 2023-05-20T01:16:58.464 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-05-20T01:16:58.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.449+0000 7fd4b3362700 1 -- 172.21.15.16:0/3293483820 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd49806ace0 msgr2=0x7fd49806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:58.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.449+0000 7fd4b3362700 1 --2- 172.21.15.16:0/3293483820 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd49806ace0 0x7fd49806d190 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd4ac158460 tx=0x7fd4a0008040 comp rx=0 tx=0).stop 2023-05-20T01:16:58.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.449+0000 7fd4b3362700 1 -- 172.21.15.16:0/3293483820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4ac0b1b00 msgr2=0x7fd4ac1ef460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:58.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.449+0000 7fd4b3362700 1 --2- 172.21.15.16:0/3293483820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4ac0b1b00 0x7fd4ac1ef460 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd49c00efc0 tx=0x7fd49c00c3a0 comp rx=0 tx=0).stop 2023-05-20T01:16:58.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.449+0000 7fd4b3362700 1 -- 172.21.15.16:0/3293483820 shutdown_connections 2023-05-20T01:16:58.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.449+0000 7fd4b3362700 1 --2- 172.21.15.16:0/3293483820 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd4ac159000 0x7fd4ac0b43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:58.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.449+0000 7fd4b3362700 1 --2- 172.21.15.16:0/3293483820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4ac0b1b00 0x7fd4ac1ef460 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:58.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.449+0000 7fd4b3362700 1 --2- 172.21.15.16:0/3293483820 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd49806ace0 0x7fd49806d190 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:58.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.449+0000 7fd4b3362700 1 --2- 172.21.15.16:0/3293483820 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd4ac0cb320 0x7fd4ac0b3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:58.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.449+0000 7fd4b3362700 1 -- 172.21.15.16:0/3293483820 >> 172.21.15.16:0/3293483820 conn(0x7fd4ac0661b0 msgr2=0x7fd4ac066580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:58.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.449+0000 7fd4b3362700 1 -- 172.21.15.16:0/3293483820 shutdown_connections 2023-05-20T01:16:58.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:58.449+0000 7fd4b3362700 1 -- 172.21.15.16:0/3293483820 wait complete. 2023-05-20T01:16:58.468 INFO:teuthology.run_tasks:Running task kclient... 2023-05-20T01:16:58.481 INFO:tasks.kclient:Mounting kernel clients... 2023-05-20T01:16:58.481 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2023-05-20T01:16:58.482 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:16:58.482 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2023-05-20T01:16:58.499 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2023-05-20T01:16:58.500 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:16:58.500 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-05-20T01:16:58.580 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:16:58.580 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link delete ceph-brx 2023-05-20T01:16:58.685 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:16:58.686 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2023-05-20T01:16:58.704 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:16:58.705 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link delete ceph-brx 2023-05-20T01:16:58.775 INFO:teuthology.orchestra.run.smithi016.stderr:Cannot find device "ceph-brx" 2023-05-20T01:16:58.775 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-05-20T01:16:58.776 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:16:58.777 DEBUG:teuthology.orchestra.run.smithi132:> ip netns list 2023-05-20T01:16:58.798 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:16:58.798 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip link delete ceph-brx 2023-05-20T01:16:58.872 INFO:teuthology.orchestra.run.smithi132.stderr:Cannot find device "ceph-brx" 2023-05-20T01:16:58.873 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-05-20T01:16:58.874 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:16:58.875 DEBUG:teuthology.orchestra.run.smithi132:> ip netns list 2023-05-20T01:16:58.936 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:16:58.936 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip link delete ceph-brx 2023-05-20T01:16:59.008 INFO:teuthology.orchestra.run.smithi132.stderr:Cannot find device "ceph-brx" 2023-05-20T01:16:59.010 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-05-20T01:16:59.011 INFO:tasks.kclient:client.0 config is {'syntax': 'v2'} 2023-05-20T01:16:59.011 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-05-20T01:16:59.011 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-05-20T01:16:59.012 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-05-20T01:16:59.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:59.134+0000 7f4408a4a700 1 Processor -- start 2023-05-20T01:16:59.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:59.134+0000 7f4408a4a700 1 -- start start 2023-05-20T01:16:59.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:59.135+0000 7f4408a4a700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4404156d30 0x7f44040ce550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:59.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:59.135+0000 7f4408a4a700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f44040cd650 0x7f440415cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:59.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:59.135+0000 7f4408a4a700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44040b13b0 con 0x7f44040cdac0 2023-05-20T01:16:59.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:59.135+0000 7f4408a4a700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44040b1970 con 0x7f44040cd650 2023-05-20T01:16:59.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:59.135+0000 7f4408a4a700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44040b1690 con 0x7f4404156d30 2023-05-20T01:16:59.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:59.135+0000 7f4401d9b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f44040cd650 0x7f440415cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:59.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:59.135+0000 7f4401d9b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f44040cd650 0x7f440415cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58726/0 (socket says 172.21.15.16:58726) 2023-05-20T01:16:59.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:59.135+0000 7f4401d9b700 1 -- 172.21.15.16:0/1916597215 learned_addr learned my addr 172.21.15.16:0/1916597215 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:59.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:59.135+0000 7f440259c700 1 --2- 172.21.15.16:0/1916597215 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4404156d30 0x7f44040ce550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:59.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:59.135+0000 7f4401d9b700 1 -- 172.21.15.16:0/1916597215 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4404156d30 msgr2=0x7f44040ce550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:16:59.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:59.135+0000 7f4401d9b700 1 --2- 172.21.15.16:0/1916597215 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4404156d30 0x7f44040ce550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:59.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:59.135+0000 7f4401d9b700 1 -- 172.21.15.16:0/1916597215 >> v1:172.21.15.132:6789/0 conn(0x7f44040cdac0 legacy=0x7f440406ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:59.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:59.135+0000 7f4401d9b700 1 -- 172.21.15.16:0/1916597215 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44040cce50 con 0x7f44040cd650 2023-05-20T01:16:59.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:59.135+0000 7f4401d9b700 1 --2- 172.21.15.16:0/1916597215 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f44040cd650 0x7f440415cb60 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f43f80098f0 tx=0x7f43f8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dd71e6a74c1139c server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:59.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:59.135+0000 7f440159a700 1 -- 172.21.15.16:0/1916597215 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43f800b8a0 con 0x7f44040cd650 2023-05-20T01:16:59.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:59.136+0000 7f440159a700 1 -- 172.21.15.16:0/1916597215 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43f800ba00 con 0x7f44040cd650 2023-05-20T01:16:59.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:59.136+0000 7f440159a700 1 -- 172.21.15.16:0/1916597215 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43f800bd90 con 0x7f44040cd650 2023-05-20T01:16:59.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:59.136+0000 7f4408a4a700 1 -- 172.21.15.16:0/1916597215 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f44040cd650 msgr2=0x7f440415cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:59.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:59.136+0000 7f4408a4a700 1 --2- 172.21.15.16:0/1916597215 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f44040cd650 0x7f440415cb60 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f43f80098f0 tx=0x7f43f8009c00 comp rx=0 tx=0).stop 2023-05-20T01:16:59.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:59.136+0000 7f4408a4a700 1 -- 172.21.15.16:0/1916597215 shutdown_connections 2023-05-20T01:16:59.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:59.136+0000 7f4408a4a700 1 --2- 172.21.15.16:0/1916597215 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4404156d30 0x7f44040ce550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:59.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:59.136+0000 7f4408a4a700 1 --2- 172.21.15.16:0/1916597215 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f44040cd650 0x7f440415cb60 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:59.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:59.136+0000 7f4408a4a700 1 -- 172.21.15.16:0/1916597215 >> 172.21.15.16:0/1916597215 conn(0x7f44040cb910 msgr2=0x7f44040cbd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:59.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:59.136+0000 7f4408a4a700 1 -- 172.21.15.16:0/1916597215 shutdown_connections 2023-05-20T01:16:59.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:59.136+0000 7f4408a4a700 1 -- 172.21.15.16:0/1916597215 wait complete. 2023-05-20T01:16:59.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:59.138+0000 7f4408a4a700 1 Processor -- start 2023-05-20T01:16:59.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:59.138+0000 7f4408a4a700 1 -- start start 2023-05-20T01:16:59.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:59.138+0000 7f4408a4a700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f44040cd650 0x7f44040c2e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:59.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:59.138+0000 7f4408a4a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f44040cdac0 0x7f44041f21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:59.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:59.138+0000 7f4408a4a700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4404156d30 0x7f4404157190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:59.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:59.138+0000 7f4408a4a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44041593e0 con 0x7f44040cdac0 2023-05-20T01:16:59.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:59.138+0000 7f4408a4a700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4404070470 con 0x7f44040cd650 2023-05-20T01:16:59.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:59.138+0000 7f4408a4a700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44040c3f50 con 0x7f4404156d30 2023-05-20T01:16:59.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:59.139+0000 7f4401d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f44040cdac0 0x7f44041f21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:59.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:59.139+0000 7f4401d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f44040cdac0 0x7f44041f21a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58734/0 (socket says 172.21.15.16:58734) 2023-05-20T01:16:59.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:59.139+0000 7f4401d9b700 1 -- 172.21.15.16:0/1817887494 learned_addr learned my addr 172.21.15.16:0/1817887494 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:16:59.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:59.139+0000 7f440259c700 1 --2- 172.21.15.16:0/1817887494 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f44040cd650 0x7f44040c2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:59.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:59.139+0000 7f4402d9d700 1 --2- 172.21.15.16:0/1817887494 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4404156d30 0x7f4404157190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:59.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:59.139+0000 7f4401d9b700 1 -- 172.21.15.16:0/1817887494 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4404156d30 msgr2=0x7f4404157190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:59.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:59.139+0000 7f4401d9b700 1 --2- 172.21.15.16:0/1817887494 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4404156d30 0x7f4404157190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:59.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:59.139+0000 7f4401d9b700 1 -- 172.21.15.16:0/1817887494 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f44040cd650 msgr2=0x7f44040c2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:59.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:59.139+0000 7f4401d9b700 1 --2- 172.21.15.16:0/1817887494 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f44040cd650 0x7f44040c2e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:59.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:59.139+0000 7f4401d9b700 1 -- 172.21.15.16:0/1817887494 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43f8009550 con 0x7f44040cdac0 2023-05-20T01:16:59.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:59.139+0000 7f4402d9d700 1 --2- 172.21.15.16:0/1817887494 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4404156d30 0x7f4404157190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:16:59.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:59.139+0000 7f4401d9b700 1 --2- 172.21.15.16:0/1817887494 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f44040cdac0 0x7f44041f21a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f43f8000c00 tx=0x7f43f8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:59.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:59.139+0000 7f43f2ffd700 1 -- 172.21.15.16:0/1817887494 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43f80038a0 con 0x7f44040cdac0 2023-05-20T01:16:59.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:59.139+0000 7f43f2ffd700 1 -- 172.21.15.16:0/1817887494 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43f8003a00 con 0x7f44040cdac0 2023-05-20T01:16:59.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:59.139+0000 7f4408a4a700 1 -- 172.21.15.16:0/1817887494 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44040cea90 con 0x7f44040cdac0 2023-05-20T01:16:59.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:59.139+0000 7f43f2ffd700 1 -- 172.21.15.16:0/1817887494 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43f8003db0 con 0x7f44040cdac0 2023-05-20T01:16:59.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:59.139+0000 7f4408a4a700 1 -- 172.21.15.16:0/1817887494 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44040cd130 con 0x7f44040cdac0 2023-05-20T01:16:59.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:59.140+0000 7f4408a4a700 1 -- 172.21.15.16:0/1817887494 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44040cb4a0 con 0x7f44040cdac0 2023-05-20T01:16:59.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:59.141+0000 7f43f2ffd700 1 -- 172.21.15.16:0/1817887494 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f43f8018400 con 0x7f44040cdac0 2023-05-20T01:16:59.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:59.141+0000 7f43f2ffd700 1 --2- 172.21.15.16:0/1817887494 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f43e406ace0 0x7f43e406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:16:59.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:59.141+0000 7f43f2ffd700 1 -- 172.21.15.16:0/1817887494 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f43f808aa90 con 0x7f44040cdac0 2023-05-20T01:16:59.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:59.141+0000 7f440259c700 1 --2- 172.21.15.16:0/1817887494 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f43e406ace0 0x7f43e406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:16:59.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:59.142+0000 7f440259c700 1 --2- 172.21.15.16:0/1817887494 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f43e406ace0 0x7f43e406d190 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f43f4006b20 tx=0x7f43f4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:16:59.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:59.145+0000 7f43f2ffd700 1 -- 172.21.15.16:0/1817887494 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f43f8055a00 con 0x7f44040cdac0 2023-05-20T01:16:59.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:59.390+0000 7f4408a4a700 1 -- 172.21.15.16:0/1817887494 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f440404e8c0 con 0x7f44040cdac0 2023-05-20T01:16:59.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:59.390+0000 7f43f2ffd700 1 -- 172.21.15.16:0/1817887494 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+88 (secure 0 0 0) 0x7f43f8059020 con 0x7f44040cdac0 2023-05-20T01:16:59.395 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-05-20T01:16:59.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:59.392+0000 7f4408a4a700 1 -- 172.21.15.16:0/1817887494 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f43e406ace0 msgr2=0x7f43e406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:59.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:59.392+0000 7f4408a4a700 1 --2- 172.21.15.16:0/1817887494 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f43e406ace0 0x7f43e406d190 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f43f4006b20 tx=0x7f43f4008040 comp rx=0 tx=0).stop 2023-05-20T01:16:59.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:59.392+0000 7f4408a4a700 1 -- 172.21.15.16:0/1817887494 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f44040cdac0 msgr2=0x7f44041f21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:16:59.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:59.392+0000 7f4408a4a700 1 --2- 172.21.15.16:0/1817887494 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f44040cdac0 0x7f44041f21a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f43f8000c00 tx=0x7f43f8004750 comp rx=0 tx=0).stop 2023-05-20T01:16:59.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:59.392+0000 7f4408a4a700 1 -- 172.21.15.16:0/1817887494 shutdown_connections 2023-05-20T01:16:59.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:59.392+0000 7f4408a4a700 1 --2- 172.21.15.16:0/1817887494 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f43e406ace0 0x7f43e406d190 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:59.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:59.392+0000 7f4408a4a700 1 --2- 172.21.15.16:0/1817887494 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4404156d30 0x7f4404157190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:59.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:59.392+0000 7f4408a4a700 1 --2- 172.21.15.16:0/1817887494 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f44040cd650 0x7f44040c2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:59.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:59.392+0000 7f4408a4a700 1 --2- 172.21.15.16:0/1817887494 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f44040cdac0 0x7f44041f21a0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:16:59.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:59.392+0000 7f4408a4a700 1 -- 172.21.15.16:0/1817887494 >> 172.21.15.16:0/1817887494 conn(0x7f44040cb910 msgr2=0x7f4404158770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:16:59.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:59.392+0000 7f4408a4a700 1 -- 172.21.15.16:0/1817887494 shutdown_connections 2023-05-20T01:16:59.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:16:59.392+0000 7f4408a4a700 1 -- 172.21.15.16:0/1817887494 wait complete. 2023-05-20T01:16:59.408 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-05-20T01:16:59.408 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-05-20T01:16:59.408 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2023-05-20T01:16:59.409 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-05-20T01:16:59.409 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-05-20T01:16:59.409 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-05-20T01:16:59.409 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-05-20T01:16:59.409 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-05-20T01:16:59.410 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-05-20T01:16:59.410 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:16:59.410 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2023-05-20T01:16:59.428 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-05-20T01:16:59.428 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-05-20T01:16:59.428 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2023-05-20T01:16:59.428 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-05-20T01:16:59.428 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2023-05-20T01:16:59.428 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-05-20T01:16:59.429 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-05-20T01:16:59.429 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2023-05-20T01:16:59.429 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-05-20T01:16:59.429 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2023-05-20T01:16:59.429 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-05-20T01:16:59.429 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2023-05-20T01:16:59.429 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-05-20T01:16:59.430 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 42414sec preferred_lft 42414sec 2023-05-20T01:16:59.430 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link 2023-05-20T01:16:59.430 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-05-20T01:16:59.430 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-05-20T01:16:59.430 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2023-05-20T01:16:59.430 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-05-20T01:16:59.431 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:16:59.431 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2023-05-20T01:16:59.431 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2023-05-20T01:16:59.432 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link add name ceph-brx type bridge 2023-05-20T01:16:59.432 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip addr flush dev ceph-brx 2023-05-20T01:16:59.432 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set ceph-brx up 2023-05-20T01:16:59.432 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-05-20T01:16:59.432 DEBUG:teuthology.orchestra.run.smithi016:> ') 2023-05-20T01:16:59.546 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:16:59.547 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-05-20T01:16:59.574 INFO:teuthology.orchestra.run.smithi016.stdout:1 2023-05-20T01:16:59.576 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:16:59.576 DEBUG:teuthology.orchestra.run.smithi016:> route 2023-05-20T01:16:59.636 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2023-05-20T01:16:59.638 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-05-20T01:16:59.638 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-05-20T01:16:59.638 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-05-20T01:16:59.638 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-05-20T01:16:59.639 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:16:59.639 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2023-05-20T01:16:59.639 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2023-05-20T01:16:59.640 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-05-20T01:16:59.640 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-05-20T01:16:59.640 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-05-20T01:16:59.641 DEBUG:teuthology.orchestra.run.smithi016:> ') 2023-05-20T01:16:59.744 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:16:59.744 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2023-05-20T01:16:59.761 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:16:59.762 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list-id 2023-05-20T01:16:59.818 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:16:59.818 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2023-05-20T01:16:59.819 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2023-05-20T01:16:59.819 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-05-20T01:16:59.819 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-05-20T01:16:59.819 DEBUG:teuthology.orchestra.run.smithi016:> ') 2023-05-20T01:16:59.913 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-05-20T01:16:59.913 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:16:59.914 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2023-05-20T01:16:59.914 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2023-05-20T01:16:59.914 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-05-20T01:16:59.914 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-05-20T01:16:59.915 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-05-20T01:16:59.915 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-05-20T01:16:59.915 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-05-20T01:16:59.915 DEBUG:teuthology.orchestra.run.smithi016:> ') 2023-05-20T01:17:00.146 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:17:00.146 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2023-05-20T01:17:00.146 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2023-05-20T01:17:00.147 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.0 up 2023-05-20T01:17:00.147 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set dev brx.0 master ceph-brx 2023-05-20T01:17:00.147 DEBUG:teuthology.orchestra.run.smithi016:> ') 2023-05-20T01:17:00.268 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-05-20T01:17:00.269 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-05-20T01:17:00.291 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-05-20T01:17:00.292 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-05-20T01:17:00.292 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-05-20T01:17:00.349 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-05-20T01:17:00.349 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:17:00.349 DEBUG:teuthology.orchestra.run.smithi016:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-05-20T01:17:00.436 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-05-20T01:17:00.436 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-05-20T01:17:00.570 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-05-20T01:17:00.571 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-05-20T01:17:00.571 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.132:3300,172.21.15.132:3301" 2023-05-20T01:17:00.571 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: trying mount with new device syntax: 0@c2245cc5-ecab-4087-a59b-bc81db901676.cephfs=/ 2023-05-20T01:17:00.571 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.16:3300/172.21.15.132:3300/172.21.15.132:3301" will pass to kernel 2023-05-20T01:17:00.571 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: v2 mount support check returned 2 2023-05-20T01:17:00.571 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: kernel does not support v2 syntax 2023-05-20T01:17:00.572 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: switching to using v1 address with old syntax 2023-05-20T01:17:00.646 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-05-20T01:17:00.646 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-05-20T01:17:00.646 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.132:3300,172.21.15.132:3301" 2023-05-20T01:17:00.646 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: trying mount with new device syntax: 0@c2245cc5-ecab-4087-a59b-bc81db901676.cephfs=/ 2023-05-20T01:17:00.646 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.16:3300/172.21.15.132:3300/172.21.15.132:3301" will pass to kernel 2023-05-20T01:17:00.646 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: v2 mount support check returned 2 2023-05-20T01:17:00.647 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: kernel does not support v2 syntax 2023-05-20T01:17:00.647 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: switching to using v1 address with old syntax 2023-05-20T01:17:00.647 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:6789,172.21.15.132:6789,172.21.15.132:6790" 2023-05-20T01:17:00.647 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.16:6789,172.21.15.132:6789,172.21.15.132:6790:/ 2023-05-20T01:17:00.647 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=c2245cc5-ecab-4087-a59b-bc81db901676" will pass to kernel 2023-05-20T01:17:00.648 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-05-20T01:17:00.649 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:17:00.649 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-05-20T01:17:00.676 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:17:00.677 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-05-20T01:17:00.749 INFO:teuthology.orchestra.run.smithi016.stdout:client4771 2023-05-20T01:17:00.749 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-05-20T01:17:00.750 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2023-05-20T01:17:00.807 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:17:00.808 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-05-20T01:17:00.880 INFO:teuthology.orchestra.run.smithi016.stdout:client4771 2023-05-20T01:17:00.881 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:17:00.882 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-05-20T01:17:00.953 INFO:teuthology.orchestra.run.smithi016.stdout:client4771 2023-05-20T01:17:00.953 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:17:00.954 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/c2245cc5-ecab-4087-a59b-bc81db901676.client4771/status') 2023-05-20T01:17:01.034 INFO:teuthology.orchestra.run.smithi016.stdout:instance: client.4771 (3)172.21.15.16:0/4019002403 2023-05-20T01:17:01.034 INFO:teuthology.orchestra.run.smithi016.stdout:blocklisted: false 2023-05-20T01:17:01.035 INFO:teuthology.orchestra.run.smithi016.stderr:0+1 records in 2023-05-20T01:17:01.035 INFO:teuthology.orchestra.run.smithi016.stderr:0+1 records out 2023-05-20T01:17:01.035 INFO:teuthology.orchestra.run.smithi016.stderr:70 bytes copied, 2.4344e-05 s, 2.9 MB/s 2023-05-20T01:17:01.036 INFO:tasks.kclient:client.2 config is {'syntax': 'v2'} 2023-05-20T01:17:01.036 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-05-20T01:17:01.036 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-05-20T01:17:01.037 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-05-20T01:17:01.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:01.197+0000 7f261febd700 1 Processor -- start 2023-05-20T01:17:01.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:01.197+0000 7f261febd700 1 -- start start 2023-05-20T01:17:01.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:01.198+0000 7f261febd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f26180ccf00 0x7f26180cb530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:01.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:01.198+0000 7f261febd700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f261815cec0 con 0x7f261815caf0 2023-05-20T01:17:01.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:01.198+0000 7f261febd700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26180d0490 con 0x7f261806d2a0 2023-05-20T01:17:01.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:01.198+0000 7f261febd700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2618160610 con 0x7f26180ccf00 2023-05-20T01:17:01.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:01.198+0000 7f261e45a700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f26180ccf00 0x7f26180cb530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:01.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:01.198+0000 7f261dc59700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f261815caf0 0x7f261806cb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:42042/0 (socket says 172.21.15.16:42042) 2023-05-20T01:17:01.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:01.198+0000 7f261dc59700 1 -- 172.21.15.16:0/1535478437 learned_addr learned my addr 172.21.15.16:0/1535478437 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:17:01.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:01.198+0000 7f261e45a700 1 -- 172.21.15.16:0/1535478437 >> v1:172.21.15.132:6789/0 conn(0x7f261806d2a0 legacy=0x7f26180cae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:01.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:01.198+0000 7f261e45a700 1 -- 172.21.15.16:0/1535478437 >> v1:172.21.15.16:6789/0 conn(0x7f261815caf0 legacy=0x7f261806cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:01.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:01.198+0000 7f261e45a700 1 -- 172.21.15.16:0/1535478437 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26181608f0 con 0x7f26180ccf00 2023-05-20T01:17:01.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:01.198+0000 7f261cc57700 1 -- 172.21.15.16:0/1535478437 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1050865218 0 0) 0x7f261815cec0 con 0x7f261815caf0 2023-05-20T01:17:01.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:01.198+0000 7f261e45a700 1 --2- 172.21.15.16:0/1535478437 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26180ccf00 0x7f26180cb530 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f2610009d90 tx=0x7f261000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=348403e7bc29923b server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:01.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:01.198+0000 7f261cc57700 1 -- 172.21.15.16:0/1535478437 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f261000e070 con 0x7f26180ccf00 2023-05-20T01:17:01.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:01.198+0000 7f261cc57700 1 -- 172.21.15.16:0/1535478437 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f261000bb90 con 0x7f26180ccf00 2023-05-20T01:17:01.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:01.198+0000 7f261cc57700 1 -- 172.21.15.16:0/1535478437 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2610003a20 con 0x7f26180ccf00 2023-05-20T01:17:01.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:01.199+0000 7f261febd700 1 -- 172.21.15.16:0/1535478437 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26180ccf00 msgr2=0x7f26180cb530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:01.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:01.199+0000 7f261febd700 1 --2- 172.21.15.16:0/1535478437 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26180ccf00 0x7f26180cb530 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f2610009d90 tx=0x7f261000b330 comp rx=0 tx=0).stop 2023-05-20T01:17:01.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:01.199+0000 7f261febd700 1 -- 172.21.15.16:0/1535478437 shutdown_connections 2023-05-20T01:17:01.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:01.199+0000 7f261febd700 1 --2- 172.21.15.16:0/1535478437 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26180ccf00 0x7f26180cb530 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:01.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:01.199+0000 7f261febd700 1 -- 172.21.15.16:0/1535478437 >> 172.21.15.16:0/1535478437 conn(0x7f26180661b0 msgr2=0x7f2618066580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:01.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:01.199+0000 7f261febd700 1 -- 172.21.15.16:0/1535478437 shutdown_connections 2023-05-20T01:17:01.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:01.199+0000 7f261febd700 1 -- 172.21.15.16:0/1535478437 wait complete. 2023-05-20T01:17:01.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:01.200+0000 7f261febd700 1 Processor -- start 2023-05-20T01:17:01.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:01.200+0000 7f261febd700 1 -- start start 2023-05-20T01:17:01.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:01.201+0000 7f261febd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f261806d2a0 0x7f261815d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:01.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:01.201+0000 7f261febd700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f26180ccf00 0x7f2618153d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:01.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:01.201+0000 7f261febd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f261815caf0 0x7f2618154250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:01.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:01.201+0000 7f261febd700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26181598c0 con 0x7f261806d2a0 2023-05-20T01:17:01.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:01.201+0000 7f261febd700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f261815f070 con 0x7f261815caf0 2023-05-20T01:17:01.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:01.201+0000 7f261febd700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f261815f310 con 0x7f26180ccf00 2023-05-20T01:17:01.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:01.201+0000 7f261dc59700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f261806d2a0 0x7f261815d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:01.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:01.201+0000 7f261dc59700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f261806d2a0 0x7f261815d130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58750/0 (socket says 172.21.15.16:58750) 2023-05-20T01:17:01.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:01.201+0000 7f261dc59700 1 -- 172.21.15.16:0/3868160263 learned_addr learned my addr 172.21.15.16:0/3868160263 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:01.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:01.201+0000 7f261e45a700 1 --2- 172.21.15.16:0/3868160263 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f261815caf0 0x7f2618154250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:01.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:01.201+0000 7f261d458700 1 --2- 172.21.15.16:0/3868160263 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f26180ccf00 0x7f2618153d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:01.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:01.201+0000 7f261dc59700 1 -- 172.21.15.16:0/3868160263 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f26180ccf00 msgr2=0x7f2618153d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:01.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:01.201+0000 7f261dc59700 1 --2- 172.21.15.16:0/3868160263 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f26180ccf00 0x7f2618153d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:01.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:01.201+0000 7f261dc59700 1 -- 172.21.15.16:0/3868160263 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f261815caf0 msgr2=0x7f2618154250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:01.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:01.201+0000 7f261dc59700 1 --2- 172.21.15.16:0/3868160263 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f261815caf0 0x7f2618154250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:01.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:01.202+0000 7f261dc59700 1 -- 172.21.15.16:0/3868160263 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2610009a70 con 0x7f261806d2a0 2023-05-20T01:17:01.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:01.202+0000 7f261e45a700 1 --2- 172.21.15.16:0/3868160263 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f261815caf0 0x7f2618154250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:17:01.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:01.202+0000 7f261dc59700 1 --2- 172.21.15.16:0/3868160263 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f261806d2a0 0x7f261815d130 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f260800c550 tx=0x7f260800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:01.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:01.202+0000 7f260e7fc700 1 -- 172.21.15.16:0/3868160263 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2608004c80 con 0x7f261806d2a0 2023-05-20T01:17:01.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:01.202+0000 7f260e7fc700 1 -- 172.21.15.16:0/3868160263 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2608004e00 con 0x7f261806d2a0 2023-05-20T01:17:01.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:01.202+0000 7f261febd700 1 -- 172.21.15.16:0/3868160263 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2618160eb0 con 0x7f261806d2a0 2023-05-20T01:17:01.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:01.202+0000 7f260e7fc700 1 -- 172.21.15.16:0/3868160263 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2608007310 con 0x7f261806d2a0 2023-05-20T01:17:01.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:01.202+0000 7f261febd700 1 -- 172.21.15.16:0/3868160263 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2618160bd0 con 0x7f261806d2a0 2023-05-20T01:17:01.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:01.203+0000 7f261febd700 1 -- 172.21.15.16:0/3868160263 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26180d0110 con 0x7f261806d2a0 2023-05-20T01:17:01.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:01.203+0000 7f260e7fc700 1 -- 172.21.15.16:0/3868160263 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2608007470 con 0x7f261806d2a0 2023-05-20T01:17:01.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:01.204+0000 7f260e7fc700 1 --2- 172.21.15.16:0/3868160263 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f25fc06ace0 0x7f25fc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:01.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:01.204+0000 7f260e7fc700 1 -- 172.21.15.16:0/3868160263 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f2608089470 con 0x7f261806d2a0 2023-05-20T01:17:01.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:01.204+0000 7f261d458700 1 --2- 172.21.15.16:0/3868160263 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f25fc06ace0 0x7f25fc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:01.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:01.204+0000 7f261d458700 1 --2- 172.21.15.16:0/3868160263 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f25fc06ace0 0x7f25fc06d190 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f261806d690 tx=0x7f2614009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:01.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:01.208+0000 7f260e7fc700 1 -- 172.21.15.16:0/3868160263 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f2608054240 con 0x7f261806d2a0 2023-05-20T01:17:01.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:01.452+0000 7f261febd700 1 -- 172.21.15.16:0/3868160263 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f2618157250 con 0x7f261806d2a0 2023-05-20T01:17:01.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:01.452+0000 7f260e7fc700 1 -- 172.21.15.16:0/3868160263 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+88 (secure 0 0 0) 0x7f2608057860 con 0x7f261806d2a0 2023-05-20T01:17:01.456 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-05-20T01:17:01.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:01.454+0000 7f261febd700 1 -- 172.21.15.16:0/3868160263 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f25fc06ace0 msgr2=0x7f25fc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:01.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:01.454+0000 7f261febd700 1 --2- 172.21.15.16:0/3868160263 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f25fc06ace0 0x7f25fc06d190 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f261806d690 tx=0x7f2614009040 comp rx=0 tx=0).stop 2023-05-20T01:17:01.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:01.454+0000 7f261febd700 1 -- 172.21.15.16:0/3868160263 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f261806d2a0 msgr2=0x7f261815d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:01.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:01.454+0000 7f261febd700 1 --2- 172.21.15.16:0/3868160263 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f261806d2a0 0x7f261815d130 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f260800c550 tx=0x7f260800c910 comp rx=0 tx=0).stop 2023-05-20T01:17:01.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:01.454+0000 7f261febd700 1 -- 172.21.15.16:0/3868160263 shutdown_connections 2023-05-20T01:17:01.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:01.454+0000 7f261febd700 1 --2- 172.21.15.16:0/3868160263 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f261815caf0 0x7f2618154250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:01.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:01.454+0000 7f261febd700 1 --2- 172.21.15.16:0/3868160263 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f261806d2a0 0x7f261815d130 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:01.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:01.454+0000 7f261febd700 1 --2- 172.21.15.16:0/3868160263 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f25fc06ace0 0x7f25fc06d190 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:01.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:01.454+0000 7f261febd700 1 --2- 172.21.15.16:0/3868160263 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f26180ccf00 0x7f2618153d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:01.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:01.454+0000 7f261febd700 1 -- 172.21.15.16:0/3868160263 >> 172.21.15.16:0/3868160263 conn(0x7f26180661b0 msgr2=0x7f26180cf6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:01.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:01.454+0000 7f261febd700 1 -- 172.21.15.16:0/3868160263 shutdown_connections 2023-05-20T01:17:01.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:01.454+0000 7f261febd700 1 -- 172.21.15.16:0/3868160263 wait complete. 2023-05-20T01:17:01.470 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-05-20T01:17:01.470 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-05-20T01:17:01.470 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2023-05-20T01:17:01.470 INFO:tasks.cephfs.mount:self.client.name = client.2 2023-05-20T01:17:01.470 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-05-20T01:17:01.471 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-05-20T01:17:01.471 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-05-20T01:17:01.471 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-05-20T01:17:01.471 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2023-05-20T01:17:01.471 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:17:01.472 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2023-05-20T01:17:01.489 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-05-20T01:17:01.490 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-05-20T01:17:01.490 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2023-05-20T01:17:01.490 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-05-20T01:17:01.490 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2023-05-20T01:17:01.490 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-05-20T01:17:01.490 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-05-20T01:17:01.490 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2023-05-20T01:17:01.490 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-05-20T01:17:01.491 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2023-05-20T01:17:01.491 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-05-20T01:17:01.491 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2023-05-20T01:17:01.491 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-05-20T01:17:01.491 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 42412sec preferred_lft 42412sec 2023-05-20T01:17:01.491 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link 2023-05-20T01:17:01.491 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-05-20T01:17:01.492 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-05-20T01:17:01.492 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2023-05-20T01:17:01.492 INFO:teuthology.orchestra.run.smithi016.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-05-20T01:17:01.492 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether d2:e4:70:a7:b9:4b brd ff:ff:ff:ff:ff:ff 2023-05-20T01:17:01.492 INFO:teuthology.orchestra.run.smithi016.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-05-20T01:17:01.492 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-05-20T01:17:01.492 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::7c00:b2ff:fef4:946/64 scope link tentative 2023-05-20T01:17:01.492 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-05-20T01:17:01.493 INFO:teuthology.orchestra.run.smithi016.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-05-20T01:17:01.493 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether d2:e4:70:a7:b9:4b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-05-20T01:17:01.493 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::d0e4:70ff:fea7:b94b/64 scope link 2023-05-20T01:17:01.493 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-05-20T01:17:01.493 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:17:01.494 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-05-20T01:17:01.555 INFO:teuthology.orchestra.run.smithi016.stdout:1 2023-05-20T01:17:01.557 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:17:01.557 DEBUG:teuthology.orchestra.run.smithi016:> route 2023-05-20T01:17:01.616 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2023-05-20T01:17:01.617 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-05-20T01:17:01.617 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-05-20T01:17:01.617 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-05-20T01:17:01.617 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-05-20T01:17:01.618 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:17:01.618 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2023-05-20T01:17:01.618 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2023-05-20T01:17:01.619 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-05-20T01:17:01.619 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-05-20T01:17:01.619 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-05-20T01:17:01.619 DEBUG:teuthology.orchestra.run.smithi016:> ') 2023-05-20T01:17:01.723 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:17:01.724 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2023-05-20T01:17:01.740 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-05-20T01:17:01.741 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:17:01.741 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list-id 2023-05-20T01:17:01.798 INFO:teuthology.orchestra.run.smithi016.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2023-05-20T01:17:01.799 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:17:01.799 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2023-05-20T01:17:01.800 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2023-05-20T01:17:01.801 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2023-05-20T01:17:01.801 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2023-05-20T01:17:01.802 DEBUG:teuthology.orchestra.run.smithi016:> ') 2023-05-20T01:17:01.894 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:17:01.895 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-05-20T01:17:01.934 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-05-20T01:17:01.934 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-05-20T01:17:01.935 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2023-05-20T01:17:01.935 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-05-20T01:17:01.935 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2023-05-20T01:17:01.935 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-05-20T01:17:01.935 INFO:teuthology.orchestra.run.smithi016.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-05-20T01:17:01.935 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 96:52:9b:de:19:bd brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-05-20T01:17:01.936 INFO:teuthology.orchestra.run.smithi016.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-05-20T01:17:01.936 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-05-20T01:17:01.936 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::9452:9bff:fede:19bd/64 scope link 2023-05-20T01:17:01.936 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-05-20T01:17:01.947 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:17:01.947 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-05-20T01:17:02.031 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-05-20T01:17:02.032 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-05-20T01:17:02.032 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2023-05-20T01:17:02.032 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-05-20T01:17:02.032 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2023-05-20T01:17:02.032 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-05-20T01:17:02.032 INFO:teuthology.orchestra.run.smithi016.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-05-20T01:17:02.033 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 96:52:9b:de:19:bd brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-05-20T01:17:02.033 INFO:teuthology.orchestra.run.smithi016.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-05-20T01:17:02.033 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-05-20T01:17:02.033 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::9452:9bff:fede:19bd/64 scope link 2023-05-20T01:17:02.033 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-05-20T01:17:02.048 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2023-05-20T01:17:02.049 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:17:02.049 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2023-05-20T01:17:02.049 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2023-05-20T01:17:02.050 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2023-05-20T01:17:02.050 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-05-20T01:17:02.050 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2023-05-20T01:17:02.051 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2023-05-20T01:17:02.051 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2023-05-20T01:17:02.051 DEBUG:teuthology.orchestra.run.smithi016:> ') 2023-05-20T01:17:02.261 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:17:02.261 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2023-05-20T01:17:02.262 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2023-05-20T01:17:02.262 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.1 up 2023-05-20T01:17:02.262 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set dev brx.1 master ceph-brx 2023-05-20T01:17:02.262 DEBUG:teuthology.orchestra.run.smithi016:> ') 2023-05-20T01:17:02.383 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-05-20T01:17:02.384 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2023-05-20T01:17:02.406 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2023-05-20T01:17:02.406 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-05-20T01:17:02.406 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2023-05-20T01:17:02.463 INFO:tasks.cephfs.kernel_mount:mounting using device: 2@.cephfs=/ 2023-05-20T01:17:02.464 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:17:02.464 DEBUG:teuthology.orchestra.run.smithi016:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 2@.cephfs=/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-05-20T01:17:02.550 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-05-20T01:17:02.550 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-05-20T01:17:02.579 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-05-20T01:17:02.579 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-05-20T01:17:02.579 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.132:3300,172.21.15.132:3301" 2023-05-20T01:17:02.579 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: trying mount with new device syntax: 2@c2245cc5-ecab-4087-a59b-bc81db901676.cephfs=/ 2023-05-20T01:17:02.580 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,ms_mode=prefer-crc,key=2,mon_addr=172.21.15.16:3300/172.21.15.132:3300/172.21.15.132:3301" will pass to kernel 2023-05-20T01:17:02.580 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: v2 mount support check returned 2 2023-05-20T01:17:02.580 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: kernel does not support v2 syntax 2023-05-20T01:17:02.580 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: switching to using v1 address with old syntax 2023-05-20T01:17:02.628 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-05-20T01:17:02.629 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-05-20T01:17:02.629 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.132:3300,172.21.15.132:3301" 2023-05-20T01:17:02.629 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: trying mount with new device syntax: 2@c2245cc5-ecab-4087-a59b-bc81db901676.cephfs=/ 2023-05-20T01:17:02.629 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,ms_mode=prefer-crc,key=2,mon_addr=172.21.15.16:3300/172.21.15.132:3300/172.21.15.132:3301" will pass to kernel 2023-05-20T01:17:02.629 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: v2 mount support check returned 2 2023-05-20T01:17:02.629 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: kernel does not support v2 syntax 2023-05-20T01:17:02.629 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: switching to using v1 address with old syntax 2023-05-20T01:17:02.630 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:6789,172.21.15.132:6789,172.21.15.132:6790" 2023-05-20T01:17:02.630 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.16:6789,172.21.15.132:6789,172.21.15.132:6790:/ 2023-05-20T01:17:02.630 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,key=2,mds_namespace=cephfs,fsid=c2245cc5-ecab-4087-a59b-bc81db901676" will pass to kernel 2023-05-20T01:17:02.640 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-05-20T01:17:02.640 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:17:02.640 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2023-05-20T01:17:02.667 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:17:02.668 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-05-20T01:17:02.739 INFO:teuthology.orchestra.run.smithi016.stdout:client4908 2023-05-20T01:17:02.740 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-05-20T01:17:02.740 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2023-05-20T01:17:02.796 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:17:02.797 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-05-20T01:17:02.868 INFO:teuthology.orchestra.run.smithi016.stdout:client4908 2023-05-20T01:17:02.868 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:17:02.869 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-05-20T01:17:02.941 INFO:teuthology.orchestra.run.smithi016.stdout:client4908 2023-05-20T01:17:02.942 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:17:02.943 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/c2245cc5-ecab-4087-a59b-bc81db901676.client4908/status') 2023-05-20T01:17:03.022 INFO:teuthology.orchestra.run.smithi016.stdout:instance: client.4908 (3)192.168.0.2:0/2254448509 2023-05-20T01:17:03.023 INFO:teuthology.orchestra.run.smithi016.stdout:blocklisted: false 2023-05-20T01:17:03.023 INFO:teuthology.orchestra.run.smithi016.stderr:0+1 records in 2023-05-20T01:17:03.023 INFO:teuthology.orchestra.run.smithi016.stderr:0+1 records out 2023-05-20T01:17:03.023 INFO:teuthology.orchestra.run.smithi016.stderr:69 bytes copied, 2.4932e-05 s, 2.8 MB/s 2023-05-20T01:17:03.024 INFO:tasks.kclient:client.1 config is {'syntax': 'v2'} 2023-05-20T01:17:03.025 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-05-20T01:17:03.025 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-05-20T01:17:03.025 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-05-20T01:17:03.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:03.185+0000 7f6b0d2e7700 1 Processor -- start 2023-05-20T01:17:03.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:03.185+0000 7f6b0d2e7700 1 -- start start 2023-05-20T01:17:03.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:03.186+0000 7f6b0d2e7700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6b08156d30 0x7f6b080ce550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:03.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:03.186+0000 7f6b0d2e7700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b080b13b0 con 0x7f6b080cd650 2023-05-20T01:17:03.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:03.186+0000 7f6b0d2e7700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b080b1690 con 0x7f6b0815d270 2023-05-20T01:17:03.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:03.186+0000 7f6b0d2e7700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b080b1970 con 0x7f6b08156d30 2023-05-20T01:17:03.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:03.186+0000 7f6b06ffd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6b08156d30 0x7f6b080ce550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:03.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:03.186+0000 7f6b067fc700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f6b080cd650 0x7f6b0815cb60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:42060/0 (socket says 172.21.15.16:42060) 2023-05-20T01:17:03.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:03.186+0000 7f6b067fc700 1 -- 172.21.15.16:0/4014156285 learned_addr learned my addr 172.21.15.16:0/4014156285 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:17:03.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:03.187+0000 7f6b06ffd700 1 -- 172.21.15.16:0/4014156285 >> v1:172.21.15.132:6789/0 conn(0x7f6b0815d270 legacy=0x7f6b0806ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:03.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:03.187+0000 7f6b06ffd700 1 -- 172.21.15.16:0/4014156285 >> v1:172.21.15.16:6789/0 conn(0x7f6b080cd650 legacy=0x7f6b0815cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:03.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:03.187+0000 7f6b06ffd700 1 -- 172.21.15.16:0/4014156285 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b080cce50 con 0x7f6b08156d30 2023-05-20T01:17:03.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:03.187+0000 7f6b06ffd700 1 --2- 172.21.15.16:0/4014156285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b08156d30 0x7f6b080ce550 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f6afc00a9e0 tx=0x7f6afc00acf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=982feab982f92dc5 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:03.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:03.187+0000 7f6b05ffb700 1 -- 172.21.15.16:0/4014156285 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6afc00cb30 con 0x7f6b08156d30 2023-05-20T01:17:03.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:03.187+0000 7f6b05ffb700 1 -- 172.21.15.16:0/4014156285 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6afc00cc90 con 0x7f6b08156d30 2023-05-20T01:17:03.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:03.187+0000 7f6b05ffb700 1 -- 172.21.15.16:0/4014156285 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6afc004030 con 0x7f6b08156d30 2023-05-20T01:17:03.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:03.187+0000 7f6b0d2e7700 1 -- 172.21.15.16:0/4014156285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b08156d30 msgr2=0x7f6b080ce550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:03.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:03.187+0000 7f6b0d2e7700 1 --2- 172.21.15.16:0/4014156285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b08156d30 0x7f6b080ce550 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f6afc00a9e0 tx=0x7f6afc00acf0 comp rx=0 tx=0).stop 2023-05-20T01:17:03.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:03.188+0000 7f6b0d2e7700 1 -- 172.21.15.16:0/4014156285 shutdown_connections 2023-05-20T01:17:03.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:03.188+0000 7f6b0d2e7700 1 --2- 172.21.15.16:0/4014156285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b08156d30 0x7f6b080ce550 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:03.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:03.188+0000 7f6b0d2e7700 1 -- 172.21.15.16:0/4014156285 >> 172.21.15.16:0/4014156285 conn(0x7f6b080cb910 msgr2=0x7f6b080cbd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:03.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:03.188+0000 7f6b0d2e7700 1 -- 172.21.15.16:0/4014156285 shutdown_connections 2023-05-20T01:17:03.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:03.188+0000 7f6b0d2e7700 1 -- 172.21.15.16:0/4014156285 wait complete. 2023-05-20T01:17:03.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:03.189+0000 7f6b0d2e7700 1 Processor -- start 2023-05-20T01:17:03.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:03.189+0000 7f6b0d2e7700 1 -- start start 2023-05-20T01:17:03.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:03.190+0000 7f6b0d2e7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b080cd650 0x7f6b080ad420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:03.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:03.190+0000 7f6b0d2e7700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6b08156d30 0x7f6b08070580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:03.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:03.190+0000 7f6b0d2e7700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6b0815d270 0x7f6b080b0440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:03.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:03.190+0000 7f6b0d2e7700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b080acf90 con 0x7f6b080cd650 2023-05-20T01:17:03.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:03.190+0000 7f6b0d2e7700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b081f3380 con 0x7f6b08156d30 2023-05-20T01:17:03.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:03.190+0000 7f6b0d2e7700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b081f30a0 con 0x7f6b0815d270 2023-05-20T01:17:03.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:03.190+0000 7f6b06ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b080cd650 0x7f6b080ad420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:03.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:03.190+0000 7f6b06ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b080cd650 0x7f6b080ad420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58768/0 (socket says 172.21.15.16:58768) 2023-05-20T01:17:03.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:03.190+0000 7f6b06ffd700 1 -- 172.21.15.16:0/1673760291 learned_addr learned my addr 172.21.15.16:0/1673760291 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:03.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:03.190+0000 7f6b067fc700 1 --2- 172.21.15.16:0/1673760291 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6b08156d30 0x7f6b08070580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:03.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:03.190+0000 7f6b06ffd700 1 -- 172.21.15.16:0/1673760291 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6b0815d270 msgr2=0x7f6b080b0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:03.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:03.190+0000 7f6b06ffd700 1 --2- 172.21.15.16:0/1673760291 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6b0815d270 0x7f6b080b0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:03.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:03.190+0000 7f6b06ffd700 1 -- 172.21.15.16:0/1673760291 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6b08156d30 msgr2=0x7f6b08070580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:03.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:03.190+0000 7f6b06ffd700 1 --2- 172.21.15.16:0/1673760291 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6b08156d30 0x7f6b08070580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:03.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:03.190+0000 7f6b06ffd700 1 -- 172.21.15.16:0/1673760291 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6afc00a640 con 0x7f6b080cd650 2023-05-20T01:17:03.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:03.190+0000 7f6b067fc700 1 --2- 172.21.15.16:0/1673760291 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6b08156d30 0x7f6b08070580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:17:03.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:03.190+0000 7f6b06ffd700 1 --2- 172.21.15.16:0/1673760291 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b080cd650 0x7f6b080ad420 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f6afc00a1c0 tx=0x7f6afc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:03.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:03.190+0000 7f6aef7fe700 1 -- 172.21.15.16:0/1673760291 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6afc01d070 con 0x7f6b080cd650 2023-05-20T01:17:03.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:03.190+0000 7f6aef7fe700 1 -- 172.21.15.16:0/1673760291 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6afc0039f0 con 0x7f6b080cd650 2023-05-20T01:17:03.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:03.190+0000 7f6b0d2e7700 1 -- 172.21.15.16:0/1673760291 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b080cea90 con 0x7f6b080cd650 2023-05-20T01:17:03.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:03.190+0000 7f6aef7fe700 1 -- 172.21.15.16:0/1673760291 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6afc003da0 con 0x7f6b080cd650 2023-05-20T01:17:03.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:03.190+0000 7f6b0d2e7700 1 -- 172.21.15.16:0/1673760291 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b080cd130 con 0x7f6b080cd650 2023-05-20T01:17:03.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:03.191+0000 7f6b0d2e7700 1 -- 172.21.15.16:0/1673760291 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b080cb4a0 con 0x7f6b080cd650 2023-05-20T01:17:03.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:03.192+0000 7f6aef7fe700 1 -- 172.21.15.16:0/1673760291 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6afc0183f0 con 0x7f6b080cd650 2023-05-20T01:17:03.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:03.192+0000 7f6aef7fe700 1 --2- 172.21.15.16:0/1673760291 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f6af406ace0 0x7f6af406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:03.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:03.192+0000 7f6aef7fe700 1 -- 172.21.15.16:0/1673760291 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f6afc08b070 con 0x7f6b080cd650 2023-05-20T01:17:03.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:03.192+0000 7f6b067fc700 1 --2- 172.21.15.16:0/1673760291 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f6af406ace0 0x7f6af406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:03.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:03.193+0000 7f6b067fc700 1 --2- 172.21.15.16:0/1673760291 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f6af406ace0 0x7f6af406d190 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6b080b0f90 tx=0x7f6af8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:03.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:03.197+0000 7f6aef7fe700 1 -- 172.21.15.16:0/1673760291 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f6afc055f60 con 0x7f6b080cd650 2023-05-20T01:17:03.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:03.444+0000 7f6b0d2e7700 1 -- 172.21.15.16:0/1673760291 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f6b0804e8c0 con 0x7f6b080cd650 2023-05-20T01:17:03.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:03.445+0000 7f6aef7fe700 1 -- 172.21.15.16:0/1673760291 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+88 (secure 0 0 0) 0x7f6afc059580 con 0x7f6b080cd650 2023-05-20T01:17:03.448 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-05-20T01:17:03.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:03.447+0000 7f6b0d2e7700 1 -- 172.21.15.16:0/1673760291 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f6af406ace0 msgr2=0x7f6af406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:03.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:03.447+0000 7f6b0d2e7700 1 --2- 172.21.15.16:0/1673760291 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f6af406ace0 0x7f6af406d190 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6b080b0f90 tx=0x7f6af8009040 comp rx=0 tx=0).stop 2023-05-20T01:17:03.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:03.447+0000 7f6b0d2e7700 1 -- 172.21.15.16:0/1673760291 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b080cd650 msgr2=0x7f6b080ad420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:03.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:03.447+0000 7f6b0d2e7700 1 --2- 172.21.15.16:0/1673760291 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b080cd650 0x7f6b080ad420 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f6afc00a1c0 tx=0x7f6afc004750 comp rx=0 tx=0).stop 2023-05-20T01:17:03.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:03.447+0000 7f6b0d2e7700 1 -- 172.21.15.16:0/1673760291 shutdown_connections 2023-05-20T01:17:03.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:03.447+0000 7f6b0d2e7700 1 --2- 172.21.15.16:0/1673760291 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f6af406ace0 0x7f6af406d190 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:03.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:03.447+0000 7f6b0d2e7700 1 --2- 172.21.15.16:0/1673760291 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6b0815d270 0x7f6b080b0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:03.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:03.447+0000 7f6b0d2e7700 1 --2- 172.21.15.16:0/1673760291 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b080cd650 0x7f6b080ad420 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:03.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:03.447+0000 7f6b0d2e7700 1 --2- 172.21.15.16:0/1673760291 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6b08156d30 0x7f6b08070580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:03.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:03.447+0000 7f6b0d2e7700 1 -- 172.21.15.16:0/1673760291 >> 172.21.15.16:0/1673760291 conn(0x7f6b080cb910 msgr2=0x7f6b080cdad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:03.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:03.447+0000 7f6b0d2e7700 1 -- 172.21.15.16:0/1673760291 shutdown_connections 2023-05-20T01:17:03.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:03.447+0000 7f6b0d2e7700 1 -- 172.21.15.16:0/1673760291 wait complete. 2023-05-20T01:17:03.463 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-05-20T01:17:03.463 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-05-20T01:17:03.463 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi132.front.sepia.ceph.com 2023-05-20T01:17:03.464 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-05-20T01:17:03.464 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-05-20T01:17:03.465 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-05-20T01:17:03.465 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-05-20T01:17:03.465 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-05-20T01:17:03.465 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2023-05-20T01:17:03.466 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:17:03.466 DEBUG:teuthology.orchestra.run.smithi132:> ip addr 2023-05-20T01:17:03.487 INFO:teuthology.orchestra.run.smithi132.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-05-20T01:17:03.487 INFO:teuthology.orchestra.run.smithi132.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-05-20T01:17:03.487 INFO:teuthology.orchestra.run.smithi132.stdout: inet 127.0.0.1/8 scope host lo 2023-05-20T01:17:03.488 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft forever preferred_lft forever 2023-05-20T01:17:03.488 INFO:teuthology.orchestra.run.smithi132.stdout: inet6 ::1/128 scope host 2023-05-20T01:17:03.488 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft forever preferred_lft forever 2023-05-20T01:17:03.488 INFO:teuthology.orchestra.run.smithi132.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-05-20T01:17:03.488 INFO:teuthology.orchestra.run.smithi132.stdout: link/ether 0c:c4:7a:d9:35:cc brd ff:ff:ff:ff:ff:ff 2023-05-20T01:17:03.489 INFO:teuthology.orchestra.run.smithi132.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-05-20T01:17:03.489 INFO:teuthology.orchestra.run.smithi132.stdout: link/ether 0c:c4:7a:d9:35:cd brd ff:ff:ff:ff:ff:ff 2023-05-20T01:17:03.489 INFO:teuthology.orchestra.run.smithi132.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-05-20T01:17:03.489 INFO:teuthology.orchestra.run.smithi132.stdout: link/ether 0c:c4:7a:88:77:3e brd ff:ff:ff:ff:ff:ff 2023-05-20T01:17:03.490 INFO:teuthology.orchestra.run.smithi132.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-05-20T01:17:03.490 INFO:teuthology.orchestra.run.smithi132.stdout: link/ether 0c:c4:7a:88:77:3f brd ff:ff:ff:ff:ff:ff 2023-05-20T01:17:03.490 INFO:teuthology.orchestra.run.smithi132.stdout: inet 172.21.15.132/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-05-20T01:17:03.490 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft 42422sec preferred_lft 42422sec 2023-05-20T01:17:03.490 INFO:teuthology.orchestra.run.smithi132.stdout: inet6 fe80::bddc:c837:86ee:be67/64 scope link noprefixroute 2023-05-20T01:17:03.491 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft forever preferred_lft forever 2023-05-20T01:17:03.491 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-05-20T01:17:03.492 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:17:03.492 DEBUG:teuthology.orchestra.run.smithi132:> (cd / && exec bash -c ' 2023-05-20T01:17:03.492 DEBUG:teuthology.orchestra.run.smithi132:> set -e 2023-05-20T01:17:03.493 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip link add name ceph-brx type bridge 2023-05-20T01:17:03.493 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip addr flush dev ceph-brx 2023-05-20T01:17:03.494 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip link set ceph-brx up 2023-05-20T01:17:03.494 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-05-20T01:17:03.494 DEBUG:teuthology.orchestra.run.smithi132:> ') 2023-05-20T01:17:03.854 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:17:03.854 DEBUG:teuthology.orchestra.run.smithi132:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-05-20T01:17:03.884 INFO:teuthology.orchestra.run.smithi132.stdout:1 2023-05-20T01:17:03.885 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:17:03.885 DEBUG:teuthology.orchestra.run.smithi132:> route 2023-05-20T01:17:03.970 INFO:teuthology.orchestra.run.smithi132.stdout:Kernel IP routing table 2023-05-20T01:17:03.971 INFO:teuthology.orchestra.run.smithi132.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-05-20T01:17:03.971 INFO:teuthology.orchestra.run.smithi132.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-05-20T01:17:03.971 INFO:teuthology.orchestra.run.smithi132.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-05-20T01:17:03.971 INFO:teuthology.orchestra.run.smithi132.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-05-20T01:17:03.971 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:17:03.972 DEBUG:teuthology.orchestra.run.smithi132:> (cd / && exec bash -c ' 2023-05-20T01:17:03.972 DEBUG:teuthology.orchestra.run.smithi132:> set -e 2023-05-20T01:17:03.972 DEBUG:teuthology.orchestra.run.smithi132:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-05-20T01:17:03.972 DEBUG:teuthology.orchestra.run.smithi132:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-05-20T01:17:03.973 DEBUG:teuthology.orchestra.run.smithi132:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-05-20T01:17:03.973 DEBUG:teuthology.orchestra.run.smithi132:> ') 2023-05-20T01:17:04.212 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:17:04.213 DEBUG:teuthology.orchestra.run.smithi132:> ip netns list 2023-05-20T01:17:04.231 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:17:04.231 DEBUG:teuthology.orchestra.run.smithi132:> ip netns list-id 2023-05-20T01:17:04.287 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:17:04.288 DEBUG:teuthology.orchestra.run.smithi132:> (cd / && exec bash -c ' 2023-05-20T01:17:04.288 DEBUG:teuthology.orchestra.run.smithi132:> set -e 2023-05-20T01:17:04.289 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2023-05-20T01:17:04.289 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2023-05-20T01:17:04.289 DEBUG:teuthology.orchestra.run.smithi132:> ') 2023-05-20T01:17:04.384 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2023-05-20T01:17:04.384 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:17:04.385 DEBUG:teuthology.orchestra.run.smithi132:> (cd / && exec bash -c ' 2023-05-20T01:17:04.385 DEBUG:teuthology.orchestra.run.smithi132:> set -e 2023-05-20T01:17:04.385 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2023-05-20T01:17:04.385 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-05-20T01:17:04.386 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2023-05-20T01:17:04.386 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2023-05-20T01:17:04.386 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2023-05-20T01:17:04.386 DEBUG:teuthology.orchestra.run.smithi132:> ') 2023-05-20T01:17:04.624 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:17:04.624 DEBUG:teuthology.orchestra.run.smithi132:> (cd / && exec bash -c ' 2023-05-20T01:17:04.624 DEBUG:teuthology.orchestra.run.smithi132:> set -e 2023-05-20T01:17:04.625 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip link set brx.0 up 2023-05-20T01:17:04.625 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip link set dev brx.0 master ceph-brx 2023-05-20T01:17:04.625 DEBUG:teuthology.orchestra.run.smithi132:> ') 2023-05-20T01:17:04.725 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-05-20T01:17:04.725 DEBUG:teuthology.orchestra.run.smithi132:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-05-20T01:17:04.745 INFO:teuthology.orchestra.run.smithi132.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-05-20T01:17:04.746 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-05-20T01:17:04.746 DEBUG:teuthology.orchestra.run.smithi132:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-05-20T01:17:04.802 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-05-20T01:17:05.031 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:17:05.032 DEBUG:teuthology.orchestra.run.smithi132:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-05-20T01:17:05.061 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:17:05.064 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:17:05.082 INFO:teuthology.orchestra.run.smithi132.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-05-20T01:17:05.082 INFO:teuthology.orchestra.run.smithi132.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-05-20T01:17:05.206 INFO:teuthology.orchestra.run.smithi132.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-05-20T01:17:05.206 INFO:teuthology.orchestra.run.smithi132.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-05-20T01:17:05.207 INFO:teuthology.orchestra.run.smithi132.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.132:3300,172.21.15.132:3301" 2023-05-20T01:17:05.207 INFO:teuthology.orchestra.run.smithi132.stdout:mount.ceph: trying mount with new device syntax: 1@c2245cc5-ecab-4087-a59b-bc81db901676.cephfs=/ 2023-05-20T01:17:05.207 INFO:teuthology.orchestra.run.smithi132.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.16:3300/172.21.15.132:3300/172.21.15.132:3301" will pass to kernel 2023-05-20T01:17:05.207 INFO:teuthology.orchestra.run.smithi132.stdout:mount.ceph: v2 mount support check returned 2 2023-05-20T01:17:05.207 INFO:teuthology.orchestra.run.smithi132.stdout:mount.ceph: kernel does not support v2 syntax 2023-05-20T01:17:05.207 INFO:teuthology.orchestra.run.smithi132.stdout:mount.ceph: switching to using v1 address with old syntax 2023-05-20T01:17:05.237 INFO:teuthology.orchestra.run.smithi132.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-05-20T01:17:05.238 INFO:teuthology.orchestra.run.smithi132.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-05-20T01:17:05.238 INFO:teuthology.orchestra.run.smithi132.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.132:3300,172.21.15.132:3301" 2023-05-20T01:17:05.238 INFO:teuthology.orchestra.run.smithi132.stdout:mount.ceph: trying mount with new device syntax: 1@c2245cc5-ecab-4087-a59b-bc81db901676.cephfs=/ 2023-05-20T01:17:05.238 INFO:teuthology.orchestra.run.smithi132.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.16:3300/172.21.15.132:3300/172.21.15.132:3301" will pass to kernel 2023-05-20T01:17:05.238 INFO:teuthology.orchestra.run.smithi132.stdout:mount.ceph: v2 mount support check returned 2 2023-05-20T01:17:05.239 INFO:teuthology.orchestra.run.smithi132.stdout:mount.ceph: kernel does not support v2 syntax 2023-05-20T01:17:05.239 INFO:teuthology.orchestra.run.smithi132.stdout:mount.ceph: switching to using v1 address with old syntax 2023-05-20T01:17:05.239 INFO:teuthology.orchestra.run.smithi132.stdout:mount.ceph: resolved to: "172.21.15.16:6789,172.21.15.132:6789,172.21.15.132:6790" 2023-05-20T01:17:05.239 INFO:teuthology.orchestra.run.smithi132.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.16:6789,172.21.15.132:6789,172.21.15.132:6790:/ 2023-05-20T01:17:05.239 INFO:teuthology.orchestra.run.smithi132.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,key=1,mds_namespace=cephfs,fsid=c2245cc5-ecab-4087-a59b-bc81db901676" will pass to kernel 2023-05-20T01:17:05.241 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-05-20T01:17:05.241 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:17:05.242 DEBUG:teuthology.orchestra.run.smithi132:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-05-20T01:17:05.275 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:17:05.275 DEBUG:teuthology.orchestra.run.smithi132:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-05-20T01:17:05.347 INFO:teuthology.orchestra.run.smithi132.stdout:client4649 2023-05-20T01:17:05.348 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2023-05-20T01:17:05.348 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/proc/self/mounts of=/dev/stdout 2023-05-20T01:17:05.405 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:17:05.406 DEBUG:teuthology.orchestra.run.smithi132:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-05-20T01:17:05.477 INFO:teuthology.orchestra.run.smithi132.stdout:client4649 2023-05-20T01:17:05.477 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:17:05.478 DEBUG:teuthology.orchestra.run.smithi132:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-05-20T01:17:05.549 INFO:teuthology.orchestra.run.smithi132.stdout:client4649 2023-05-20T01:17:05.549 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:17:05.550 DEBUG:teuthology.orchestra.run.smithi132:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/c2245cc5-ecab-4087-a59b-bc81db901676.client4649/status') 2023-05-20T01:17:05.629 INFO:teuthology.orchestra.run.smithi132.stdout:instance: client.4649 (3)192.168.0.1:0/1899844421 2023-05-20T01:17:05.629 INFO:teuthology.orchestra.run.smithi132.stdout:blocklisted: false 2023-05-20T01:17:05.629 INFO:teuthology.orchestra.run.smithi132.stderr:0+1 records in 2023-05-20T01:17:05.629 INFO:teuthology.orchestra.run.smithi132.stderr:0+1 records out 2023-05-20T01:17:05.630 INFO:teuthology.orchestra.run.smithi132.stderr:69 bytes copied, 2.3699e-05 s, 2.9 MB/s 2023-05-20T01:17:05.630 INFO:tasks.kclient:client.3 config is {'syntax': 'v2'} 2023-05-20T01:17:05.631 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-05-20T01:17:05.631 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-05-20T01:17:05.631 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-05-20T01:17:05.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:05.767+0000 7f7baafb0700 1 Processor -- start 2023-05-20T01:17:05.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:05.767+0000 7f7baafb0700 1 -- start start 2023-05-20T01:17:05.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:05.767+0000 7f7baafb0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7ba415a9d0 0x7f7ba415efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:05.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:05.768+0000 7f7baafb0700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ba406d940 con 0x7f7ba40b2250 2023-05-20T01:17:05.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:05.768+0000 7f7baafb0700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ba415f500 con 0x7f7ba406ac20 2023-05-20T01:17:05.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:05.768+0000 7f7baafb0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba40cc8d0 con 0x7f7ba415a9d0 2023-05-20T01:17:05.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:05.768+0000 7f7ba3fff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f7ba40b2250 0x7f7ba406a510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:42076/0 (socket says 172.21.15.16:42076) 2023-05-20T01:17:05.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:05.768+0000 7f7ba3fff700 1 -- 172.21.15.16:0/3151317003 learned_addr learned my addr 172.21.15.16:0/3151317003 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:17:05.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:05.768+0000 7f7ba8d4c700 1 --2- 172.21.15.16:0/3151317003 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7ba415a9d0 0x7f7ba415efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:05.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:05.768+0000 7f7ba37fe700 1 -- 172.21.15.16:0/3151317003 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2333989081 0 0) 0x7f7ba406d940 con 0x7f7ba40b2250 2023-05-20T01:17:05.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:05.768+0000 7f7ba37fe700 1 -- 172.21.15.16:0/3151317003 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b940035b0 con 0x7f7ba40b2250 2023-05-20T01:17:05.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:05.768+0000 7f7ba37fe700 1 -- 172.21.15.16:0/3151317003 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 972444633 0 0) 0x7f7ba415f500 con 0x7f7ba406ac20 2023-05-20T01:17:05.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:05.768+0000 7f7ba37fe700 1 -- 172.21.15.16:0/3151317003 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ba406d940 con 0x7f7ba406ac20 2023-05-20T01:17:05.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:05.768+0000 7f7ba8d4c700 1 -- 172.21.15.16:0/3151317003 >> v1:172.21.15.132:6789/0 conn(0x7f7ba406ac20 legacy=0x7f7ba40710d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:05.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:05.768+0000 7f7ba8d4c700 1 -- 172.21.15.16:0/3151317003 >> v1:172.21.15.16:6789/0 conn(0x7f7ba40b2250 legacy=0x7f7ba406a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:05.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:05.768+0000 7f7ba8d4c700 1 -- 172.21.15.16:0/3151317003 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ba40cc030 con 0x7f7ba415a9d0 2023-05-20T01:17:05.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:05.768+0000 7f7ba8d4c700 1 --2- 172.21.15.16:0/3151317003 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7ba415a9d0 0x7f7ba415efc0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f7b98009870 tx=0x7f7b98009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a7c08f92073a809 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:05.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:05.769+0000 7f7ba37fe700 1 -- 172.21.15.16:0/3151317003 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b98011830 con 0x7f7ba415a9d0 2023-05-20T01:17:05.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:05.769+0000 7f7ba37fe700 1 -- 172.21.15.16:0/3151317003 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b98011990 con 0x7f7ba415a9d0 2023-05-20T01:17:05.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:05.769+0000 7f7ba37fe700 1 -- 172.21.15.16:0/3151317003 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b98011d20 con 0x7f7ba415a9d0 2023-05-20T01:17:05.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:05.769+0000 7f7baafb0700 1 -- 172.21.15.16:0/3151317003 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7ba415a9d0 msgr2=0x7f7ba415efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:05.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:05.769+0000 7f7baafb0700 1 --2- 172.21.15.16:0/3151317003 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7ba415a9d0 0x7f7ba415efc0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f7b98009870 tx=0x7f7b98009b80 comp rx=0 tx=0).stop 2023-05-20T01:17:05.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:05.769+0000 7f7baafb0700 1 -- 172.21.15.16:0/3151317003 shutdown_connections 2023-05-20T01:17:05.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:05.769+0000 7f7baafb0700 1 --2- 172.21.15.16:0/3151317003 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7ba415a9d0 0x7f7ba415efc0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:05.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:05.769+0000 7f7baafb0700 1 -- 172.21.15.16:0/3151317003 >> 172.21.15.16:0/3151317003 conn(0x7f7ba40d0c30 msgr2=0x7f7ba40b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:05.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:05.769+0000 7f7baafb0700 1 -- 172.21.15.16:0/3151317003 shutdown_connections 2023-05-20T01:17:05.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:05.769+0000 7f7baafb0700 1 -- 172.21.15.16:0/3151317003 wait complete. 2023-05-20T01:17:05.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:05.770+0000 7f7baafb0700 1 Processor -- start 2023-05-20T01:17:05.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:05.770+0000 7f7baafb0700 1 -- start start 2023-05-20T01:17:05.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:05.771+0000 7f7baafb0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ba406ac20 0x7f7ba41e58d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:05.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:05.771+0000 7f7baafb0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7ba40b2250 0x7f7ba4158660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:05.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:05.771+0000 7f7baafb0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7ba415a9d0 0x7f7ba41414c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:05.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:05.771+0000 7f7baafb0700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba4140140 con 0x7f7ba406ac20 2023-05-20T01:17:05.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:05.771+0000 7f7baafb0700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba4140420 con 0x7f7ba40b2250 2023-05-20T01:17:05.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:05.771+0000 7f7baafb0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba4140700 con 0x7f7ba415a9d0 2023-05-20T01:17:05.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:05.771+0000 7f7ba8d4c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ba406ac20 0x7f7ba41e58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:05.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:05.771+0000 7f7ba3fff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7ba40b2250 0x7f7ba4158660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:05.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:05.771+0000 7f7ba8d4c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ba406ac20 0x7f7ba41e58d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58784/0 (socket says 172.21.15.16:58784) 2023-05-20T01:17:05.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:05.771+0000 7f7ba954d700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7ba415a9d0 0x7f7ba41414c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:05.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:05.771+0000 7f7ba8d4c700 1 -- 172.21.15.16:0/2316797600 learned_addr learned my addr 172.21.15.16:0/2316797600 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:05.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:05.772+0000 7f7ba8d4c700 1 -- 172.21.15.16:0/2316797600 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7ba415a9d0 msgr2=0x7f7ba41414c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:05.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:05.772+0000 7f7ba8d4c700 1 --2- 172.21.15.16:0/2316797600 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7ba415a9d0 0x7f7ba41414c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:05.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:05.772+0000 7f7ba8d4c700 1 -- 172.21.15.16:0/2316797600 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7ba40b2250 msgr2=0x7f7ba4158660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:05.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:05.772+0000 7f7ba8d4c700 1 --2- 172.21.15.16:0/2316797600 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7ba40b2250 0x7f7ba4158660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:05.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:05.772+0000 7f7ba8d4c700 1 -- 172.21.15.16:0/2316797600 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b980094a0 con 0x7f7ba406ac20 2023-05-20T01:17:05.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:05.772+0000 7f7ba954d700 1 --2- 172.21.15.16:0/2316797600 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7ba415a9d0 0x7f7ba41414c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-05-20T01:17:05.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:05.772+0000 7f7ba3fff700 1 --2- 172.21.15.16:0/2316797600 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7ba40b2250 0x7f7ba4158660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-05-20T01:17:05.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:05.772+0000 7f7ba8d4c700 1 --2- 172.21.15.16:0/2316797600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ba406ac20 0x7f7ba41e58d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f7b980097c0 tx=0x7f7b98004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:05.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:05.772+0000 7f7ba17fa700 1 -- 172.21.15.16:0/2316797600 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b9801a8f0 con 0x7f7ba406ac20 2023-05-20T01:17:05.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:05.772+0000 7f7ba17fa700 1 -- 172.21.15.16:0/2316797600 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b9801aa70 con 0x7f7ba406ac20 2023-05-20T01:17:05.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:05.772+0000 7f7baafb0700 1 -- 172.21.15.16:0/2316797600 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ba40cc5f0 con 0x7f7ba406ac20 2023-05-20T01:17:05.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:05.772+0000 7f7ba17fa700 1 -- 172.21.15.16:0/2316797600 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b9801ae20 con 0x7f7ba406ac20 2023-05-20T01:17:05.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:05.772+0000 7f7baafb0700 1 -- 172.21.15.16:0/2316797600 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ba40cc310 con 0x7f7ba406ac20 2023-05-20T01:17:05.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:05.773+0000 7f7baafb0700 1 -- 172.21.15.16:0/2316797600 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ba415e7c0 con 0x7f7ba406ac20 2023-05-20T01:17:05.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:05.774+0000 7f7ba17fa700 1 -- 172.21.15.16:0/2316797600 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7b9802e450 con 0x7f7ba406ac20 2023-05-20T01:17:05.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:05.774+0000 7f7ba17fa700 1 --2- 172.21.15.16:0/2316797600 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f7b8806ace0 0x7f7b8806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:05.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:05.774+0000 7f7ba17fa700 1 -- 172.21.15.16:0/2316797600 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f7b9808aa80 con 0x7f7ba406ac20 2023-05-20T01:17:05.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:05.774+0000 7f7ba3fff700 1 --2- 172.21.15.16:0/2316797600 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f7b8806ace0 0x7f7b8806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:05.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:05.775+0000 7f7ba3fff700 1 --2- 172.21.15.16:0/2316797600 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f7b8806ace0 0x7f7b8806d190 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7ba40b5d50 tx=0x7f7b90007e00 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:05.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:05.779+0000 7f7ba17fa700 1 -- 172.21.15.16:0/2316797600 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f7b980558d0 con 0x7f7ba406ac20 2023-05-20T01:17:06.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:06.028+0000 7f7baafb0700 1 -- 172.21.15.16:0/2316797600 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f7ba4141310 con 0x7f7ba406ac20 2023-05-20T01:17:06.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:06.028+0000 7f7ba17fa700 1 -- 172.21.15.16:0/2316797600 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+88 (secure 0 0 0) 0x7f7b98058ef0 con 0x7f7ba406ac20 2023-05-20T01:17:06.032 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-05-20T01:17:06.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:06.030+0000 7f7baafb0700 1 -- 172.21.15.16:0/2316797600 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f7b8806ace0 msgr2=0x7f7b8806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:06.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:06.030+0000 7f7baafb0700 1 --2- 172.21.15.16:0/2316797600 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f7b8806ace0 0x7f7b8806d190 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7ba40b5d50 tx=0x7f7b90007e00 comp rx=0 tx=0).stop 2023-05-20T01:17:06.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:06.030+0000 7f7baafb0700 1 -- 172.21.15.16:0/2316797600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ba406ac20 msgr2=0x7f7ba41e58d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:06.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:06.030+0000 7f7baafb0700 1 --2- 172.21.15.16:0/2316797600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ba406ac20 0x7f7ba41e58d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f7b980097c0 tx=0x7f7b98004750 comp rx=0 tx=0).stop 2023-05-20T01:17:06.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:06.031+0000 7f7baafb0700 1 -- 172.21.15.16:0/2316797600 shutdown_connections 2023-05-20T01:17:06.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:06.031+0000 7f7baafb0700 1 --2- 172.21.15.16:0/2316797600 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f7b8806ace0 0x7f7b8806d190 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:06.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:06.031+0000 7f7baafb0700 1 --2- 172.21.15.16:0/2316797600 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7ba415a9d0 0x7f7ba41414c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:06.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:06.031+0000 7f7baafb0700 1 --2- 172.21.15.16:0/2316797600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ba406ac20 0x7f7ba41e58d0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:06.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:06.031+0000 7f7baafb0700 1 --2- 172.21.15.16:0/2316797600 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7ba40b2250 0x7f7ba4158660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:06.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:06.031+0000 7f7baafb0700 1 -- 172.21.15.16:0/2316797600 >> 172.21.15.16:0/2316797600 conn(0x7f7ba40d0c30 msgr2=0x7f7ba40b38d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:06.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:06.031+0000 7f7baafb0700 1 -- 172.21.15.16:0/2316797600 shutdown_connections 2023-05-20T01:17:06.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:06.031+0000 7f7baafb0700 1 -- 172.21.15.16:0/2316797600 wait complete. 2023-05-20T01:17:06.048 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-05-20T01:17:06.049 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-05-20T01:17:06.049 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi132.front.sepia.ceph.com 2023-05-20T01:17:06.050 INFO:tasks.cephfs.mount:self.client.name = client.3 2023-05-20T01:17:06.050 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-05-20T01:17:06.050 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-05-20T01:17:06.051 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-05-20T01:17:06.051 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-05-20T01:17:06.051 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2023-05-20T01:17:06.052 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:17:06.052 DEBUG:teuthology.orchestra.run.smithi132:> ip addr 2023-05-20T01:17:06.070 INFO:teuthology.orchestra.run.smithi132.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-05-20T01:17:06.070 INFO:teuthology.orchestra.run.smithi132.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-05-20T01:17:06.070 INFO:teuthology.orchestra.run.smithi132.stdout: inet 127.0.0.1/8 scope host lo 2023-05-20T01:17:06.071 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft forever preferred_lft forever 2023-05-20T01:17:06.071 INFO:teuthology.orchestra.run.smithi132.stdout: inet6 ::1/128 scope host 2023-05-20T01:17:06.071 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft forever preferred_lft forever 2023-05-20T01:17:06.071 INFO:teuthology.orchestra.run.smithi132.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-05-20T01:17:06.071 INFO:teuthology.orchestra.run.smithi132.stdout: link/ether 0c:c4:7a:d9:35:cc brd ff:ff:ff:ff:ff:ff 2023-05-20T01:17:06.071 INFO:teuthology.orchestra.run.smithi132.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-05-20T01:17:06.072 INFO:teuthology.orchestra.run.smithi132.stdout: link/ether 0c:c4:7a:d9:35:cd brd ff:ff:ff:ff:ff:ff 2023-05-20T01:17:06.072 INFO:teuthology.orchestra.run.smithi132.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-05-20T01:17:06.072 INFO:teuthology.orchestra.run.smithi132.stdout: link/ether 0c:c4:7a:88:77:3e brd ff:ff:ff:ff:ff:ff 2023-05-20T01:17:06.073 INFO:teuthology.orchestra.run.smithi132.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-05-20T01:17:06.073 INFO:teuthology.orchestra.run.smithi132.stdout: link/ether 0c:c4:7a:88:77:3f brd ff:ff:ff:ff:ff:ff 2023-05-20T01:17:06.073 INFO:teuthology.orchestra.run.smithi132.stdout: inet 172.21.15.132/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-05-20T01:17:06.073 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft 42420sec preferred_lft 42420sec 2023-05-20T01:17:06.073 INFO:teuthology.orchestra.run.smithi132.stdout: inet6 fe80::bddc:c837:86ee:be67/64 scope link noprefixroute 2023-05-20T01:17:06.073 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft forever preferred_lft forever 2023-05-20T01:17:06.073 INFO:teuthology.orchestra.run.smithi132.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-05-20T01:17:06.074 INFO:teuthology.orchestra.run.smithi132.stdout: link/ether 5e:cd:1e:b4:8f:b0 brd ff:ff:ff:ff:ff:ff 2023-05-20T01:17:06.074 INFO:teuthology.orchestra.run.smithi132.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-05-20T01:17:06.074 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft forever preferred_lft forever 2023-05-20T01:17:06.074 INFO:teuthology.orchestra.run.smithi132.stdout: inet6 fe80::5434:dff:fe13:c4e3/64 scope link 2023-05-20T01:17:06.074 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft forever preferred_lft forever 2023-05-20T01:17:06.074 INFO:teuthology.orchestra.run.smithi132.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-05-20T01:17:06.074 INFO:teuthology.orchestra.run.smithi132.stdout: link/ether 5e:cd:1e:b4:8f:b0 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-05-20T01:17:06.075 INFO:teuthology.orchestra.run.smithi132.stdout: inet6 fe80::5ccd:1eff:feb4:8fb0/64 scope link tentative 2023-05-20T01:17:06.075 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft forever preferred_lft forever 2023-05-20T01:17:06.075 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:17:06.075 DEBUG:teuthology.orchestra.run.smithi132:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-05-20T01:17:06.135 INFO:teuthology.orchestra.run.smithi132.stdout:1 2023-05-20T01:17:06.136 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:17:06.136 DEBUG:teuthology.orchestra.run.smithi132:> route 2023-05-20T01:17:06.195 INFO:teuthology.orchestra.run.smithi132.stdout:Kernel IP routing table 2023-05-20T01:17:06.195 INFO:teuthology.orchestra.run.smithi132.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-05-20T01:17:06.195 INFO:teuthology.orchestra.run.smithi132.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-05-20T01:17:06.196 INFO:teuthology.orchestra.run.smithi132.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-05-20T01:17:06.196 INFO:teuthology.orchestra.run.smithi132.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-05-20T01:17:06.196 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:17:06.196 DEBUG:teuthology.orchestra.run.smithi132:> (cd / && exec bash -c ' 2023-05-20T01:17:06.197 DEBUG:teuthology.orchestra.run.smithi132:> set -e 2023-05-20T01:17:06.197 DEBUG:teuthology.orchestra.run.smithi132:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-05-20T01:17:06.197 DEBUG:teuthology.orchestra.run.smithi132:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-05-20T01:17:06.197 DEBUG:teuthology.orchestra.run.smithi132:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-05-20T01:17:06.198 DEBUG:teuthology.orchestra.run.smithi132:> ') 2023-05-20T01:17:06.299 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:17:06.300 DEBUG:teuthology.orchestra.run.smithi132:> ip netns list 2023-05-20T01:17:06.317 INFO:teuthology.orchestra.run.smithi132.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-05-20T01:17:06.318 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:17:06.318 DEBUG:teuthology.orchestra.run.smithi132:> ip netns list-id 2023-05-20T01:17:06.373 INFO:teuthology.orchestra.run.smithi132.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2023-05-20T01:17:06.374 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:17:06.374 DEBUG:teuthology.orchestra.run.smithi132:> (cd / && exec bash -c ' 2023-05-20T01:17:06.374 DEBUG:teuthology.orchestra.run.smithi132:> set -e 2023-05-20T01:17:06.374 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2023-05-20T01:17:06.375 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2023-05-20T01:17:06.375 DEBUG:teuthology.orchestra.run.smithi132:> ') 2023-05-20T01:17:06.469 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:17:06.470 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-05-20T01:17:06.509 INFO:teuthology.orchestra.run.smithi132.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-05-20T01:17:06.509 INFO:teuthology.orchestra.run.smithi132.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-05-20T01:17:06.509 INFO:teuthology.orchestra.run.smithi132.stdout: inet 127.0.0.1/8 scope host lo 2023-05-20T01:17:06.509 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft forever preferred_lft forever 2023-05-20T01:17:06.509 INFO:teuthology.orchestra.run.smithi132.stdout: inet6 ::1/128 scope host 2023-05-20T01:17:06.510 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft forever preferred_lft forever 2023-05-20T01:17:06.510 INFO:teuthology.orchestra.run.smithi132.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-05-20T01:17:06.510 INFO:teuthology.orchestra.run.smithi132.stdout: link/ether a2:86:98:1c:f5:b6 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-05-20T01:17:06.510 INFO:teuthology.orchestra.run.smithi132.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-05-20T01:17:06.510 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft forever preferred_lft forever 2023-05-20T01:17:06.510 INFO:teuthology.orchestra.run.smithi132.stdout: inet6 fe80::a086:98ff:fe1c:f5b6/64 scope link 2023-05-20T01:17:06.510 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft forever preferred_lft forever 2023-05-20T01:17:06.524 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:17:06.525 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-05-20T01:17:06.599 INFO:teuthology.orchestra.run.smithi132.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-05-20T01:17:06.599 INFO:teuthology.orchestra.run.smithi132.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-05-20T01:17:06.600 INFO:teuthology.orchestra.run.smithi132.stdout: inet 127.0.0.1/8 scope host lo 2023-05-20T01:17:06.600 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft forever preferred_lft forever 2023-05-20T01:17:06.600 INFO:teuthology.orchestra.run.smithi132.stdout: inet6 ::1/128 scope host 2023-05-20T01:17:06.601 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft forever preferred_lft forever 2023-05-20T01:17:06.601 INFO:teuthology.orchestra.run.smithi132.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-05-20T01:17:06.601 INFO:teuthology.orchestra.run.smithi132.stdout: link/ether a2:86:98:1c:f5:b6 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-05-20T01:17:06.602 INFO:teuthology.orchestra.run.smithi132.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-05-20T01:17:06.602 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft forever preferred_lft forever 2023-05-20T01:17:06.602 INFO:teuthology.orchestra.run.smithi132.stdout: inet6 fe80::a086:98ff:fe1c:f5b6/64 scope link 2023-05-20T01:17:06.603 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft forever preferred_lft forever 2023-05-20T01:17:06.615 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2023-05-20T01:17:06.616 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:17:06.616 DEBUG:teuthology.orchestra.run.smithi132:> (cd / && exec bash -c ' 2023-05-20T01:17:06.617 DEBUG:teuthology.orchestra.run.smithi132:> set -e 2023-05-20T01:17:06.617 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2023-05-20T01:17:06.617 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-05-20T01:17:06.617 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2023-05-20T01:17:06.617 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2023-05-20T01:17:06.618 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2023-05-20T01:17:06.618 DEBUG:teuthology.orchestra.run.smithi132:> ') 2023-05-20T01:17:06.881 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:17:06.882 DEBUG:teuthology.orchestra.run.smithi132:> (cd / && exec bash -c ' 2023-05-20T01:17:06.882 DEBUG:teuthology.orchestra.run.smithi132:> set -e 2023-05-20T01:17:06.882 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip link set brx.1 up 2023-05-20T01:17:06.883 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip link set dev brx.1 master ceph-brx 2023-05-20T01:17:06.883 DEBUG:teuthology.orchestra.run.smithi132:> ') 2023-05-20T01:17:06.988 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-05-20T01:17:06.988 DEBUG:teuthology.orchestra.run.smithi132:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2023-05-20T01:17:07.009 INFO:teuthology.orchestra.run.smithi132.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2023-05-20T01:17:07.009 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-05-20T01:17:07.009 DEBUG:teuthology.orchestra.run.smithi132:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2023-05-20T01:17:07.065 INFO:tasks.cephfs.kernel_mount:mounting using device: 3@.cephfs=/ 2023-05-20T01:17:07.065 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:17:07.065 DEBUG:teuthology.orchestra.run.smithi132:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 3@.cephfs=/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-05-20T01:17:07.150 INFO:teuthology.orchestra.run.smithi132.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-05-20T01:17:07.150 INFO:teuthology.orchestra.run.smithi132.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-05-20T01:17:07.173 INFO:teuthology.orchestra.run.smithi132.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-05-20T01:17:07.173 INFO:teuthology.orchestra.run.smithi132.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-05-20T01:17:07.174 INFO:teuthology.orchestra.run.smithi132.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.132:3300,172.21.15.132:3301" 2023-05-20T01:17:07.174 INFO:teuthology.orchestra.run.smithi132.stdout:mount.ceph: trying mount with new device syntax: 3@c2245cc5-ecab-4087-a59b-bc81db901676.cephfs=/ 2023-05-20T01:17:07.174 INFO:teuthology.orchestra.run.smithi132.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,ms_mode=prefer-crc,key=3,mon_addr=172.21.15.16:3300/172.21.15.132:3300/172.21.15.132:3301" will pass to kernel 2023-05-20T01:17:07.174 INFO:teuthology.orchestra.run.smithi132.stdout:mount.ceph: v2 mount support check returned 2 2023-05-20T01:17:07.174 INFO:teuthology.orchestra.run.smithi132.stdout:mount.ceph: kernel does not support v2 syntax 2023-05-20T01:17:07.174 INFO:teuthology.orchestra.run.smithi132.stdout:mount.ceph: switching to using v1 address with old syntax 2023-05-20T01:17:07.212 INFO:teuthology.orchestra.run.smithi132.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-05-20T01:17:07.213 INFO:teuthology.orchestra.run.smithi132.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-05-20T01:17:07.213 INFO:teuthology.orchestra.run.smithi132.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.132:3300,172.21.15.132:3301" 2023-05-20T01:17:07.213 INFO:teuthology.orchestra.run.smithi132.stdout:mount.ceph: trying mount with new device syntax: 3@c2245cc5-ecab-4087-a59b-bc81db901676.cephfs=/ 2023-05-20T01:17:07.213 INFO:teuthology.orchestra.run.smithi132.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,ms_mode=prefer-crc,key=3,mon_addr=172.21.15.16:3300/172.21.15.132:3300/172.21.15.132:3301" will pass to kernel 2023-05-20T01:17:07.213 INFO:teuthology.orchestra.run.smithi132.stdout:mount.ceph: v2 mount support check returned 2 2023-05-20T01:17:07.213 INFO:teuthology.orchestra.run.smithi132.stdout:mount.ceph: kernel does not support v2 syntax 2023-05-20T01:17:07.214 INFO:teuthology.orchestra.run.smithi132.stdout:mount.ceph: switching to using v1 address with old syntax 2023-05-20T01:17:07.214 INFO:teuthology.orchestra.run.smithi132.stdout:mount.ceph: resolved to: "172.21.15.16:6789,172.21.15.132:6789,172.21.15.132:6790" 2023-05-20T01:17:07.214 INFO:teuthology.orchestra.run.smithi132.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.16:6789,172.21.15.132:6789,172.21.15.132:6790:/ 2023-05-20T01:17:07.214 INFO:teuthology.orchestra.run.smithi132.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,key=3,mds_namespace=cephfs,fsid=c2245cc5-ecab-4087-a59b-bc81db901676" will pass to kernel 2023-05-20T01:17:07.232 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-05-20T01:17:07.233 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:17:07.233 DEBUG:teuthology.orchestra.run.smithi132:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2023-05-20T01:17:07.259 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:17:07.260 DEBUG:teuthology.orchestra.run.smithi132:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-05-20T01:17:07.333 INFO:teuthology.orchestra.run.smithi132.stdout:client4658 2023-05-20T01:17:07.334 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2023-05-20T01:17:07.334 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/proc/self/mounts of=/dev/stdout 2023-05-20T01:17:07.390 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:17:07.390 DEBUG:teuthology.orchestra.run.smithi132:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-05-20T01:17:07.462 INFO:teuthology.orchestra.run.smithi132.stdout:client4658 2023-05-20T01:17:07.463 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:17:07.463 DEBUG:teuthology.orchestra.run.smithi132:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-05-20T01:17:07.535 INFO:teuthology.orchestra.run.smithi132.stdout:client4658 2023-05-20T01:17:07.535 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:17:07.535 DEBUG:teuthology.orchestra.run.smithi132:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/c2245cc5-ecab-4087-a59b-bc81db901676.client4658/status') 2023-05-20T01:17:07.614 INFO:teuthology.orchestra.run.smithi132.stdout:instance: client.4658 (3)192.168.0.2:0/2641152315 2023-05-20T01:17:07.615 INFO:teuthology.orchestra.run.smithi132.stdout:blocklisted: false 2023-05-20T01:17:07.617 INFO:teuthology.orchestra.run.smithi132.stderr:0+1 records in 2023-05-20T01:17:07.617 INFO:teuthology.orchestra.run.smithi132.stderr:0+1 records out 2023-05-20T01:17:07.618 INFO:teuthology.orchestra.run.smithi132.stderr:69 bytes copied, 3.7459e-05 s, 1.8 MB/s 2023-05-20T01:17:07.618 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2023-05-20T01:17:07.637 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-20T01:17:07.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:07.774+0000 7fe295c58700 1 Processor -- start 2023-05-20T01:17:07.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:07.774+0000 7fe295c58700 1 -- start start 2023-05-20T01:17:07.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:07.774+0000 7fe295c58700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fe2900cfc50 0x7fe2900d2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:07.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:07.774+0000 7fe295c58700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe2900b4750 0x7fe2900d0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:07.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:07.774+0000 7fe295c58700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe2900b2b40 0x7fe2900b2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:07.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:07.774+0000 7fe295c58700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2900cfad0 con 0x7fe2900b2b40 2023-05-20T01:17:07.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:07.774+0000 7fe295c58700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2900d3df0 con 0x7fe2900cfc50 2023-05-20T01:17:07.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:07.774+0000 7fe295c58700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2900d0b90 con 0x7fe2900b4750 2023-05-20T01:17:07.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:07.774+0000 7fe28ffff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe2900b2b40 0x7fe2900b2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:07.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:07.774+0000 7fe28ffff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe2900b2b40 0x7fe2900b2fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58794/0 (socket says 172.21.15.16:58794) 2023-05-20T01:17:07.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:07.774+0000 7fe28ffff700 1 -- 172.21.15.16:0/3638102951 learned_addr learned my addr 172.21.15.16:0/3638102951 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:07.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:07.774+0000 7fe28f7fe700 1 --2- 172.21.15.16:0/3638102951 >> v2:172.21.15.132:3300/0 conn(0x7fe2900cfc50 0x7fe2900d2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:07.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:07.774+0000 7fe28effd700 1 --2- 172.21.15.16:0/3638102951 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe2900b4750 0x7fe2900d0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:07.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:07.774+0000 7fe28ffff700 1 -- 172.21.15.16:0/3638102951 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe2900b4750 msgr2=0x7fe2900d0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:07.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:07.774+0000 7fe28ffff700 1 --2- 172.21.15.16:0/3638102951 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe2900b4750 0x7fe2900d0020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:07.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:07.774+0000 7fe28ffff700 1 -- 172.21.15.16:0/3638102951 >> v2:172.21.15.132:3300/0 conn(0x7fe2900cfc50 msgr2=0x7fe2900d2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:07.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:07.774+0000 7fe28ffff700 1 --2- 172.21.15.16:0/3638102951 >> v2:172.21.15.132:3300/0 conn(0x7fe2900cfc50 0x7fe2900d2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:07.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:07.774+0000 7fe28ffff700 1 -- 172.21.15.16:0/3638102951 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2900d0560 con 0x7fe2900b2b40 2023-05-20T01:17:07.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:07.774+0000 7fe28ffff700 1 --2- 172.21.15.16:0/3638102951 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2900b2b40 0x7fe2900b2fc0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe2900b5de0 tx=0x7fe28800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a253557790d58ebd server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:07.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:07.775+0000 7fe28e7fc700 1 -- 172.21.15.16:0/3638102951 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe28800e070 con 0x7fe2900b2b40 2023-05-20T01:17:07.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:07.775+0000 7fe28e7fc700 1 -- 172.21.15.16:0/3638102951 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe28800b910 con 0x7fe2900b2b40 2023-05-20T01:17:07.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:07.775+0000 7fe28e7fc700 1 -- 172.21.15.16:0/3638102951 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe28800bcc0 con 0x7fe2900b2b40 2023-05-20T01:17:07.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:07.775+0000 7fe295c58700 1 -- 172.21.15.16:0/3638102951 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2900b2b40 msgr2=0x7fe2900b2fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:07.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:07.775+0000 7fe295c58700 1 --2- 172.21.15.16:0/3638102951 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2900b2b40 0x7fe2900b2fc0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe2900b5de0 tx=0x7fe28800b330 comp rx=0 tx=0).stop 2023-05-20T01:17:07.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:07.775+0000 7fe295c58700 1 -- 172.21.15.16:0/3638102951 shutdown_connections 2023-05-20T01:17:07.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:07.775+0000 7fe295c58700 1 --2- 172.21.15.16:0/3638102951 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2900b2b40 0x7fe2900b2fc0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:07.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:07.775+0000 7fe295c58700 1 --2- 172.21.15.16:0/3638102951 >> v2:172.21.15.132:3300/0 conn(0x7fe2900cfc50 0x7fe2900d2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:07.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:07.775+0000 7fe295c58700 1 --2- 172.21.15.16:0/3638102951 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe2900b4750 0x7fe2900d0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:07.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:07.775+0000 7fe295c58700 1 -- 172.21.15.16:0/3638102951 >> 172.21.15.16:0/3638102951 conn(0x7fe29009c590 msgr2=0x7fe2900d3240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:07.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:07.775+0000 7fe295c58700 1 -- 172.21.15.16:0/3638102951 shutdown_connections 2023-05-20T01:17:07.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:07.775+0000 7fe295c58700 1 -- 172.21.15.16:0/3638102951 wait complete. 2023-05-20T01:17:07.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:07.776+0000 7fe295c58700 1 Processor -- start 2023-05-20T01:17:07.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:07.777+0000 7fe295c58700 1 -- start start 2023-05-20T01:17:07.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:07.777+0000 7fe295c58700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe2900b2b40 0x7fe2900cacf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:07.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:07.777+0000 7fe295c58700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2900b4750 0x7fe290157820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:07.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:07.777+0000 7fe295c58700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2900cfc50 0x7fe2900cd870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:07.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:07.777+0000 7fe295c58700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2900b7730 con 0x7fe2900b4750 2023-05-20T01:17:07.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:07.777+0000 7fe295c58700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2900cb440 con 0x7fe2900cfc50 2023-05-20T01:17:07.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:07.777+0000 7fe295c58700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2900b8fa0 con 0x7fe2900b2b40 2023-05-20T01:17:07.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:07.777+0000 7fe28effd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2900b4750 0x7fe290157820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:07.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:07.778+0000 7fe28effd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2900b4750 0x7fe290157820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58798/0 (socket says 172.21.15.16:58798) 2023-05-20T01:17:07.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:07.778+0000 7fe28effd700 1 -- 172.21.15.16:0/3010326084 learned_addr learned my addr 172.21.15.16:0/3010326084 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:07.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:07.778+0000 7fe28f7fe700 1 --2- 172.21.15.16:0/3010326084 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe2900b2b40 0x7fe2900cacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:07.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:07.778+0000 7fe28ffff700 1 --2- 172.21.15.16:0/3010326084 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2900cfc50 0x7fe2900cd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:07.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:07.778+0000 7fe28effd700 1 -- 172.21.15.16:0/3010326084 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe2900b2b40 msgr2=0x7fe2900cacf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:07.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:07.778+0000 7fe28effd700 1 --2- 172.21.15.16:0/3010326084 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe2900b2b40 0x7fe2900cacf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:07.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:07.778+0000 7fe28effd700 1 -- 172.21.15.16:0/3010326084 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2900cfc50 msgr2=0x7fe2900cd870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:07.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:07.778+0000 7fe28effd700 1 --2- 172.21.15.16:0/3010326084 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2900cfc50 0x7fe2900cd870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:07.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:07.778+0000 7fe28effd700 1 -- 172.21.15.16:0/3010326084 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe288009b30 con 0x7fe2900b4750 2023-05-20T01:17:07.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:07.778+0000 7fe28effd700 1 --2- 172.21.15.16:0/3010326084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2900b4750 0x7fe290157820 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe28400e870 tx=0x7fe28400eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:07.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:07.778+0000 7fe277fff700 1 -- 172.21.15.16:0/3010326084 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe284013680 con 0x7fe2900b4750 2023-05-20T01:17:07.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:07.778+0000 7fe277fff700 1 -- 172.21.15.16:0/3010326084 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2840137e0 con 0x7fe2900b4750 2023-05-20T01:17:07.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:07.778+0000 7fe295c58700 1 -- 172.21.15.16:0/3010326084 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2900b4380 con 0x7fe2900b4750 2023-05-20T01:17:07.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:07.778+0000 7fe277fff700 1 -- 172.21.15.16:0/3010326084 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe284013b90 con 0x7fe2900b4750 2023-05-20T01:17:07.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:07.778+0000 7fe295c58700 1 -- 172.21.15.16:0/3010326084 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe29015d0b0 con 0x7fe2900b4750 2023-05-20T01:17:07.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:07.779+0000 7fe295c58700 1 -- 172.21.15.16:0/3010326084 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2900d4000 con 0x7fe2900b4750 2023-05-20T01:17:07.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:07.780+0000 7fe277fff700 1 -- 172.21.15.16:0/3010326084 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe284013cf0 con 0x7fe2900b4750 2023-05-20T01:17:07.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:07.780+0000 7fe277fff700 1 --2- 172.21.15.16:0/3010326084 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe27c06ace0 0x7fe27c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:07.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:07.780+0000 7fe277fff700 1 -- 172.21.15.16:0/3010326084 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fe284089570 con 0x7fe2900b4750 2023-05-20T01:17:07.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:07.780+0000 7fe28f7fe700 1 --2- 172.21.15.16:0/3010326084 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe27c06ace0 0x7fe27c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:07.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:07.781+0000 7fe28f7fe700 1 --2- 172.21.15.16:0/3010326084 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe27c06ace0 0x7fe27c06d190 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe2900ccef0 tx=0x7fe280009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:07.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:07.785+0000 7fe277fff700 1 -- 172.21.15.16:0/3010326084 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fe284054330 con 0x7fe2900b4750 2023-05-20T01:17:07.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:07.984+0000 7fe295c58700 1 -- 172.21.15.16:0/3010326084 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe2900d1cd0 con 0x7fe2900b4750 2023-05-20T01:17:07.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:07.986+0000 7fe277fff700 1 -- 172.21.15.16:0/3010326084 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+20026 (secure 0 0 0) 0x7fe284057950 con 0x7fe2900b4750 2023-05-20T01:17:07.990 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:17:07.990 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"fsid":"c2245cc5-ecab-4087-a59b-bc81db901676","created":"2023-05-20T01:16:09.549230+0000","modified":"2023-05-20T01:16:32.768495+0000","last_up_change":"2023-05-20T01:16:19.788624+0000","last_in_change":"2023-05-20T01:16:12.092160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-05-20T01:16:20.934366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-05-20T01:16:21.372294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-05-20T01:16:26.208287+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-05-20T01:16:27.292268+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-05-20T01:16:30.357285+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"317a83cd-19c8-4958-a301-6087ef342289","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6825","nonce":3923634179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6827","nonce":3923634179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6831","nonce":3923634179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6829","nonce":3923634179}]},"public_addr":"172.21.15.16:6825/3923634179","cluster_addr":"172.21.15.16:6827/3923634179","heartbeat_back_addr":"172.21.15.16:6831/3923634179","heartbeat_front_addr":"172.21.15.16:6829/3923634179","state":["exists","up"]},{"osd":1,"uuid":"5581046f-8219-4089-8d7d-b7846399505e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6809","nonce":3860264220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6811","nonce":3860264220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6815","nonce":3860264220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6813","nonce":3860264220}]},"public_addr":"172.21.15.16:6809/3860264220","cluster_addr":"172.21.15.16:6811/3860264220","heartbeat_back_addr":"172.21.15.16:6815/3860264220","heartbeat_front_addr":"172.21.15.16:6813/3860264220","state":["exists","up"]},{"osd":2,"uuid":"072e9e66-4133-478e-8d4a-64d46c2963f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6817","nonce":293187030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6819","nonce":293187030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6823","nonce":293187030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6821","nonce":293187030}]},"public_addr":"172.21.15.16:6817/293187030","cluster_addr":"172.21.15.16:6819/293187030","heartbeat_back_addr":"172.21.15.16:6823/293187030","heartbeat_front_addr":"172.21.15.16:6821/293187030","state":["exists","up"]},{"osd":3,"uuid":"11e1bba6-00f8-48dd-9028-d8d716b7f145","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6801","nonce":3669781890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6803","nonce":3669781890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6807","nonce":3669781890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6805","nonce":3669781890}]},"public_addr":"172.21.15.16:6801/3669781890","cluster_addr":"172.21.15.16:6803/3669781890","heartbeat_back_addr":"172.21.15.16:6807/3669781890","heartbeat_front_addr":"172.21.15.16:6805/3669781890","state":["exists","up"]},{"osd":4,"uuid":"b8909c3a-cf70-46cb-8d0b-009c707cf9d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6819","nonce":506317223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6823","nonce":506317223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6829","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6831","nonce":506317223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6827","nonce":506317223}]},"public_addr":"172.21.15.132:6819/506317223","cluster_addr":"172.21.15.132:6823/506317223","heartbeat_back_addr":"172.21.15.132:6831/506317223","heartbeat_front_addr":"172.21.15.132:6827/506317223","state":["exists","up"]},{"osd":5,"uuid":"b1818304-4583-47bf-a6ec-0bfbb5166233","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6818","nonce":1844689995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6822","nonce":1844689995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6830","nonce":1844689995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6826","nonce":1844689995}]},"public_addr":"172.21.15.132:6818/1844689995","cluster_addr":"172.21.15.132:6822/1844689995","heartbeat_back_addr":"172.21.15.132:6830/1844689995","heartbeat_front_addr":"172.21.15.132:6826/1844689995","state":["exists","up"]},{"osd":6,"uuid":"d37f4691-eed1-4a93-8818-18354f03f4a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6801","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6802","nonce":727581123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6805","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6807","nonce":727581123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6813","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6815","nonce":727581123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6811","nonce":727581123}]},"public_addr":"172.21.15.132:6802/727581123","cluster_addr":"172.21.15.132:6807/727581123","heartbeat_back_addr":"172.21.15.132:6815/727581123","heartbeat_front_addr":"172.21.15.132:6811/727581123","state":["exists","up"]},{"osd":7,"uuid":"4f712f62-5d1d-4816-8042-03209144f8b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6803","nonce":466001149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6806","nonce":466001149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6814","nonce":466001149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6810","nonce":466001149}]},"public_addr":"172.21.15.132:6803/466001149","cluster_addr":"172.21.15.132:6806/466001149","heartbeat_back_addr":"172.21.15.132:6814/466001149","heartbeat_front_addr":"172.21.15.132:6810/466001149","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.338863+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.278680+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305694+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.246835+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.276237+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.304441+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305421+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.267050+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-20T01:17:07.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:07.988+0000 7fe295c58700 1 -- 172.21.15.16:0/3010326084 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe27c06ace0 msgr2=0x7fe27c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:07.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:07.988+0000 7fe295c58700 1 --2- 172.21.15.16:0/3010326084 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe27c06ace0 0x7fe27c06d190 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe2900ccef0 tx=0x7fe280009210 comp rx=0 tx=0).stop 2023-05-20T01:17:07.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:07.988+0000 7fe295c58700 1 -- 172.21.15.16:0/3010326084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2900b4750 msgr2=0x7fe290157820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:07.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:07.988+0000 7fe295c58700 1 --2- 172.21.15.16:0/3010326084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2900b4750 0x7fe290157820 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe28400e870 tx=0x7fe28400eb80 comp rx=0 tx=0).stop 2023-05-20T01:17:07.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:07.988+0000 7fe295c58700 1 -- 172.21.15.16:0/3010326084 shutdown_connections 2023-05-20T01:17:07.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:07.988+0000 7fe295c58700 1 --2- 172.21.15.16:0/3010326084 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2900cfc50 0x7fe2900cd870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:07.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:07.988+0000 7fe295c58700 1 --2- 172.21.15.16:0/3010326084 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe27c06ace0 0x7fe27c06d190 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:07.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:07.988+0000 7fe295c58700 1 --2- 172.21.15.16:0/3010326084 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe2900b2b40 0x7fe2900cacf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:07.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:07.988+0000 7fe295c58700 1 --2- 172.21.15.16:0/3010326084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2900b4750 0x7fe290157820 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:07.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:07.988+0000 7fe295c58700 1 -- 172.21.15.16:0/3010326084 >> 172.21.15.16:0/3010326084 conn(0x7fe29009c590 msgr2=0x7fe2901e7e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:07.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:07.988+0000 7fe295c58700 1 -- 172.21.15.16:0/3010326084 shutdown_connections 2023-05-20T01:17:07.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:07.988+0000 7fe295c58700 1 -- 172.21.15.16:0/3010326084 wait complete. 2023-05-20T01:17:08.004 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-05-20T01:16:20.934366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-05-20T01:16:21.372294+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-05-20T01:16:26.208287+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-05-20T01:16:27.292268+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-05-20T01:16:30.357285+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-05-20T01:17:08.005 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-05-20T01:17:08.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.131+0000 7f001ebce700 1 Processor -- start 2023-05-20T01:17:08.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.131+0000 7f001ebce700 1 -- start start 2023-05-20T01:17:08.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.131+0000 7f001ebce700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f00180718c0 0x7f001806dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:08.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.131+0000 7f001ebce700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f001806cbd0 con 0x7f00180af4d0 2023-05-20T01:17:08.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.131+0000 7f001ebce700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00180cadb0 con 0x7f0018072010 2023-05-20T01:17:08.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.131+0000 7f001ebce700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00180af8a0 con 0x7f00180718c0 2023-05-20T01:17:08.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.132+0000 7f001c96a700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f00180af4d0 0x7f00180ca6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:42100/0 (socket says 172.21.15.16:42100) 2023-05-20T01:17:08.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.132+0000 7f001c96a700 1 -- 172.21.15.16:0/2147598100 learned_addr learned my addr 172.21.15.16:0/2147598100 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:17:08.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.132+0000 7f001d16b700 1 --2- 172.21.15.16:0/2147598100 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f00180718c0 0x7f001806dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:08.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.132+0000 7f00177fe700 1 -- 172.21.15.16:0/2147598100 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3617619354 0 0) 0x7f001806cbd0 con 0x7f00180af4d0 2023-05-20T01:17:08.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.132+0000 7f00177fe700 1 -- 172.21.15.16:0/2147598100 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00080035b0 con 0x7f00180af4d0 2023-05-20T01:17:08.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.132+0000 7f00177fe700 1 -- 172.21.15.16:0/2147598100 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 990355569 0 0) 0x7f00180cadb0 con 0x7f0018072010 2023-05-20T01:17:08.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.132+0000 7f00177fe700 1 -- 172.21.15.16:0/2147598100 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f001806cbd0 con 0x7f0018072010 2023-05-20T01:17:08.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.132+0000 7f001d16b700 1 -- 172.21.15.16:0/2147598100 >> v1:172.21.15.132:6789/0 conn(0x7f0018072010 legacy=0x7f00180711b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:08.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.132+0000 7f001d16b700 1 -- 172.21.15.16:0/2147598100 >> v1:172.21.15.16:6789/0 conn(0x7f00180af4d0 legacy=0x7f00180ca6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:08.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.132+0000 7f00177fe700 1 -- 172.21.15.16:0/2147598100 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2318700073 0 0) 0x7f00080035b0 con 0x7f00180af4d0 2023-05-20T01:17:08.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.132+0000 7f001d16b700 1 -- 172.21.15.16:0/2147598100 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f001806ce20 con 0x7f00180718c0 2023-05-20T01:17:08.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.132+0000 7f001d16b700 1 --2- 172.21.15.16:0/2147598100 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f00180718c0 0x7f001806dec0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0010009dc0 tx=0x7f001000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f522f143bd93573e server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:08.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.133+0000 7f00177fe700 1 -- 172.21.15.16:0/2147598100 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f001000e070 con 0x7f00180718c0 2023-05-20T01:17:08.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.133+0000 7f00177fe700 1 -- 172.21.15.16:0/2147598100 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0010003a20 con 0x7f00180718c0 2023-05-20T01:17:08.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.133+0000 7f00177fe700 1 -- 172.21.15.16:0/2147598100 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f000c001670 con 0x7f00180718c0 2023-05-20T01:17:08.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.133+0000 7f001ebce700 1 -- 172.21.15.16:0/2147598100 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f00180718c0 msgr2=0x7f001806dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:08.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.133+0000 7f001ebce700 1 --2- 172.21.15.16:0/2147598100 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f00180718c0 0x7f001806dec0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0010009dc0 tx=0x7f001000b330 comp rx=0 tx=0).stop 2023-05-20T01:17:08.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.133+0000 7f001ebce700 1 -- 172.21.15.16:0/2147598100 shutdown_connections 2023-05-20T01:17:08.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.133+0000 7f001ebce700 1 --2- 172.21.15.16:0/2147598100 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f00180718c0 0x7f001806dec0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:08.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.133+0000 7f001ebce700 1 -- 172.21.15.16:0/2147598100 >> 172.21.15.16:0/2147598100 conn(0x7f00180cd760 msgr2=0x7f00180d0050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:08.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.133+0000 7f001ebce700 1 -- 172.21.15.16:0/2147598100 shutdown_connections 2023-05-20T01:17:08.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.133+0000 7f001ebce700 1 -- 172.21.15.16:0/2147598100 wait complete. 2023-05-20T01:17:08.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.134+0000 7f001ebce700 1 Processor -- start 2023-05-20T01:17:08.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.134+0000 7f001ebce700 1 -- start start 2023-05-20T01:17:08.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.135+0000 7f001ebce700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f00180718c0 0x7f001814d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:08.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.135+0000 7f001ebce700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0018072010 0x7f00180c8470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:08.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.135+0000 7f001ebce700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00180af4d0 0x7f00180c89b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:08.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.135+0000 7f001ebce700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f001814d290 con 0x7f00180af4d0 2023-05-20T01:17:08.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.135+0000 7f001ebce700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00180b5500 con 0x7f00180718c0 2023-05-20T01:17:08.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.135+0000 7f001ebce700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00180b57a0 con 0x7f0018072010 2023-05-20T01:17:08.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.135+0000 7f001d16b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00180af4d0 0x7f00180c89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:08.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.135+0000 7f001d16b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00180af4d0 0x7f00180c89b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58814/0 (socket says 172.21.15.16:58814) 2023-05-20T01:17:08.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.135+0000 7f001c96a700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f00180718c0 0x7f001814d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:08.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.135+0000 7f0017fff700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0018072010 0x7f00180c8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:08.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.135+0000 7f001d16b700 1 -- 172.21.15.16:0/1540747966 learned_addr learned my addr 172.21.15.16:0/1540747966 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:08.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.136+0000 7f001d16b700 1 -- 172.21.15.16:0/1540747966 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0018072010 msgr2=0x7f00180c8470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:08.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.136+0000 7f001d16b700 1 --2- 172.21.15.16:0/1540747966 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0018072010 0x7f00180c8470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:08.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.136+0000 7f001d16b700 1 -- 172.21.15.16:0/1540747966 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f00180718c0 msgr2=0x7f001814d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:08.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.136+0000 7f001d16b700 1 --2- 172.21.15.16:0/1540747966 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f00180718c0 0x7f001814d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:08.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.136+0000 7f001d16b700 1 -- 172.21.15.16:0/1540747966 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0010009a70 con 0x7f00180af4d0 2023-05-20T01:17:08.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.136+0000 7f001c96a700 1 --2- 172.21.15.16:0/1540747966 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f00180718c0 0x7f001814d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-05-20T01:17:08.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.136+0000 7f001d16b700 1 --2- 172.21.15.16:0/1540747966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00180af4d0 0x7f00180c89b0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f0010009d90 tx=0x7f0010004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:08.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.136+0000 7f00157fa700 1 -- 172.21.15.16:0/1540747966 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00100044c0 con 0x7f00180af4d0 2023-05-20T01:17:08.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.136+0000 7f00157fa700 1 -- 172.21.15.16:0/1540747966 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0010004620 con 0x7f00180af4d0 2023-05-20T01:17:08.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.136+0000 7f00157fa700 1 -- 172.21.15.16:0/1540747966 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f001001d470 con 0x7f00180af4d0 2023-05-20T01:17:08.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.136+0000 7f001ebce700 1 -- 172.21.15.16:0/1540747966 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f001806d3e0 con 0x7f00180af4d0 2023-05-20T01:17:08.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.136+0000 7f001ebce700 1 -- 172.21.15.16:0/1540747966 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f001806d100 con 0x7f00180af4d0 2023-05-20T01:17:08.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.137+0000 7f001ebce700 1 -- 172.21.15.16:0/1540747966 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f001806d6c0 con 0x7f00180af4d0 2023-05-20T01:17:08.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.138+0000 7f00157fa700 1 -- 172.21.15.16:0/1540747966 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f001001d5d0 con 0x7f00180af4d0 2023-05-20T01:17:08.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.138+0000 7f00157fa700 1 --2- 172.21.15.16:0/1540747966 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7efffc06ace0 0x7efffc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:08.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.138+0000 7f00157fa700 1 -- 172.21.15.16:0/1540747966 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f001008b060 con 0x7f00180af4d0 2023-05-20T01:17:08.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.138+0000 7f001c96a700 1 --2- 172.21.15.16:0/1540747966 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7efffc06ace0 0x7efffc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:08.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.138+0000 7f001c96a700 1 --2- 172.21.15.16:0/1540747966 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7efffc06ace0 0x7efffc06d190 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f000c001eb0 tx=0x7f000c001670 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:08.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.142+0000 7f00157fa700 1 -- 172.21.15.16:0/1540747966 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f00100598b0 con 0x7f00180af4d0 2023-05-20T01:17:08.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.341+0000 7f001ebce700 1 -- 172.21.15.16:0/1540747966 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f00180747b0 con 0x7f00180af4d0 2023-05-20T01:17:08.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.342+0000 7f00157fa700 1 -- 172.21.15.16:0/1540747966 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7f0010059440 con 0x7f00180af4d0 2023-05-20T01:17:08.345 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 8 2023-05-20T01:17:08.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.344+0000 7f001ebce700 1 -- 172.21.15.16:0/1540747966 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7efffc06ace0 msgr2=0x7efffc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:08.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.344+0000 7f001ebce700 1 --2- 172.21.15.16:0/1540747966 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7efffc06ace0 0x7efffc06d190 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f000c001eb0 tx=0x7f000c001670 comp rx=0 tx=0).stop 2023-05-20T01:17:08.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.344+0000 7f001ebce700 1 -- 172.21.15.16:0/1540747966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00180af4d0 msgr2=0x7f00180c89b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:08.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.344+0000 7f001ebce700 1 --2- 172.21.15.16:0/1540747966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00180af4d0 0x7f00180c89b0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f0010009d90 tx=0x7f0010004ad0 comp rx=0 tx=0).stop 2023-05-20T01:17:08.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.344+0000 7f001ebce700 1 -- 172.21.15.16:0/1540747966 shutdown_connections 2023-05-20T01:17:08.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.344+0000 7f001ebce700 1 --2- 172.21.15.16:0/1540747966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00180af4d0 0x7f00180c89b0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:08.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.344+0000 7f001ebce700 1 --2- 172.21.15.16:0/1540747966 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f00180718c0 0x7f001814d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:08.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.344+0000 7f001ebce700 1 --2- 172.21.15.16:0/1540747966 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7efffc06ace0 0x7efffc06d190 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:08.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.344+0000 7f001ebce700 1 --2- 172.21.15.16:0/1540747966 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0018072010 0x7f00180c8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:08.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.344+0000 7f001ebce700 1 -- 172.21.15.16:0/1540747966 >> 172.21.15.16:0/1540747966 conn(0x7f00180cd760 msgr2=0x7f001815c210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:08.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.344+0000 7f001ebce700 1 -- 172.21.15.16:0/1540747966 shutdown_connections 2023-05-20T01:17:08.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.344+0000 7f001ebce700 1 -- 172.21.15.16:0/1540747966 wait complete. 2023-05-20T01:17:08.359 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-05-20T01:17:08.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.484+0000 7f7e11a78700 1 Processor -- start 2023-05-20T01:17:08.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.484+0000 7f7e11a78700 1 -- start start 2023-05-20T01:17:08.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.485+0000 7f7e11a78700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7e0c1598b0 0x7f7e0c15dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:08.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.485+0000 7f7e11a78700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f7e0c06f800 0x7f7e0c0ca660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:08.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.485+0000 7f7e11a78700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e0c071e50 con 0x7f7e0c0b1a60 2023-05-20T01:17:08.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.485+0000 7f7e11a78700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e0c159c80 con 0x7f7e0c1598b0 2023-05-20T01:17:08.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.485+0000 7f7e11a78700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e0c15e2a0 con 0x7f7e0c06f800 2023-05-20T01:17:08.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.485+0000 7f7e0b7fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7e0c1598b0 0x7f7e0c15dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:08.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.485+0000 7f7e0b7fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7e0c1598b0 0x7f7e0c15dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58818/0 (socket says 172.21.15.16:58818) 2023-05-20T01:17:08.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.485+0000 7f7e0b7fe700 1 -- 172.21.15.16:0/2874608059 learned_addr learned my addr 172.21.15.16:0/2874608059 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:08.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.485+0000 7f7e0affd700 1 --2- 172.21.15.16:0/2874608059 >> v2:172.21.15.132:3300/0 conn(0x7f7e0c06f800 0x7f7e0c0ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:08.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.486+0000 7f7e0b7fe700 1 -- 172.21.15.16:0/2874608059 >> v2:172.21.15.132:3300/0 conn(0x7f7e0c06f800 msgr2=0x7f7e0c0ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:08.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.486+0000 7f7e0b7fe700 1 --2- 172.21.15.16:0/2874608059 >> v2:172.21.15.132:3300/0 conn(0x7f7e0c06f800 0x7f7e0c0ca660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:08.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.486+0000 7f7e0b7fe700 1 -- 172.21.15.16:0/2874608059 >> v1:172.21.15.132:6789/0 conn(0x7f7e0c0b1a60 legacy=0x7f7e0c0b1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:08.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.486+0000 7f7e0b7fe700 1 -- 172.21.15.16:0/2874608059 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e0c0ce670 con 0x7f7e0c1598b0 2023-05-20T01:17:08.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.486+0000 7f7e0affd700 1 --2- 172.21.15.16:0/2874608059 >> v2:172.21.15.132:3300/0 conn(0x7f7e0c06f800 0x7f7e0c0ca660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:17:08.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.486+0000 7f7e0b7fe700 1 --2- 172.21.15.16:0/2874608059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e0c1598b0 0x7f7e0c15dd60 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c071960 tx=0x7f7dfc00be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33de4bb7bf01c3b1 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:08.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.486+0000 7f7e0a7fc700 1 -- 172.21.15.16:0/2874608059 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dfc00dd30 con 0x7f7e0c1598b0 2023-05-20T01:17:08.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.486+0000 7f7e0a7fc700 1 -- 172.21.15.16:0/2874608059 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dfc00de90 con 0x7f7e0c1598b0 2023-05-20T01:17:08.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.486+0000 7f7e0a7fc700 1 -- 172.21.15.16:0/2874608059 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dfc010690 con 0x7f7e0c1598b0 2023-05-20T01:17:08.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.486+0000 7f7e11a78700 1 -- 172.21.15.16:0/2874608059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e0c1598b0 msgr2=0x7f7e0c15dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:08.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.486+0000 7f7e11a78700 1 --2- 172.21.15.16:0/2874608059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e0c1598b0 0x7f7e0c15dd60 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c071960 tx=0x7f7dfc00be70 comp rx=0 tx=0).stop 2023-05-20T01:17:08.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.486+0000 7f7e11a78700 1 -- 172.21.15.16:0/2874608059 shutdown_connections 2023-05-20T01:17:08.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.486+0000 7f7e11a78700 1 --2- 172.21.15.16:0/2874608059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e0c1598b0 0x7f7e0c15dd60 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:08.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.486+0000 7f7e11a78700 1 --2- 172.21.15.16:0/2874608059 >> v2:172.21.15.132:3300/0 conn(0x7f7e0c06f800 0x7f7e0c0ca660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:08.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.486+0000 7f7e11a78700 1 -- 172.21.15.16:0/2874608059 >> 172.21.15.16:0/2874608059 conn(0x7f7e0c06d220 msgr2=0x7f7e0c06d5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:08.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.487+0000 7f7e11a78700 1 -- 172.21.15.16:0/2874608059 shutdown_connections 2023-05-20T01:17:08.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.487+0000 7f7e11a78700 1 -- 172.21.15.16:0/2874608059 wait complete. 2023-05-20T01:17:08.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.488+0000 7f7e11a78700 1 Processor -- start 2023-05-20T01:17:08.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.488+0000 7f7e11a78700 1 -- start start 2023-05-20T01:17:08.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.489+0000 7f7e11a78700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e0c06f800 0x7f7e0c148f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:08.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.489+0000 7f7e11a78700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7e0c0b1a60 0x7f7e0c0c9650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:08.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.489+0000 7f7e11a78700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7e0c1598b0 0x7f7e0c1e4220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:08.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.489+0000 7f7e11a78700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e0c0c9230 con 0x7f7e0c06f800 2023-05-20T01:17:08.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.489+0000 7f7e11a78700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e0c1e5760 con 0x7f7e0c1598b0 2023-05-20T01:17:08.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.489+0000 7f7e11a78700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e0c149480 con 0x7f7e0c0b1a60 2023-05-20T01:17:08.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.489+0000 7f7e0b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e0c06f800 0x7f7e0c148f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:08.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.489+0000 7f7e0b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e0c06f800 0x7f7e0c148f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58822/0 (socket says 172.21.15.16:58822) 2023-05-20T01:17:08.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.489+0000 7f7e0b7fe700 1 -- 172.21.15.16:0/2252884000 learned_addr learned my addr 172.21.15.16:0/2252884000 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:08.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.489+0000 7f7e0bfff700 1 --2- 172.21.15.16:0/2252884000 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7e0c1598b0 0x7f7e0c1e4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:08.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.490+0000 7f7e0affd700 1 --2- 172.21.15.16:0/2252884000 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7e0c0b1a60 0x7f7e0c0c9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:08.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.490+0000 7f7e0b7fe700 1 -- 172.21.15.16:0/2252884000 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7e0c0b1a60 msgr2=0x7f7e0c0c9650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:08.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.490+0000 7f7e0b7fe700 1 --2- 172.21.15.16:0/2252884000 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7e0c0b1a60 0x7f7e0c0c9650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:08.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.490+0000 7f7e0b7fe700 1 -- 172.21.15.16:0/2252884000 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7e0c1598b0 msgr2=0x7f7e0c1e4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:08.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.490+0000 7f7e0b7fe700 1 --2- 172.21.15.16:0/2252884000 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7e0c1598b0 0x7f7e0c1e4220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:08.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.490+0000 7f7e0b7fe700 1 -- 172.21.15.16:0/2252884000 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dfc00b7f0 con 0x7f7e0c06f800 2023-05-20T01:17:08.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.490+0000 7f7e0b7fe700 1 --2- 172.21.15.16:0/2252884000 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e0c06f800 0x7f7e0c148f40 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc000c00 tx=0x7f7dfc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:08.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.490+0000 7f7e03fff700 1 -- 172.21.15.16:0/2252884000 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dfc01f070 con 0x7f7e0c06f800 2023-05-20T01:17:08.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.490+0000 7f7e03fff700 1 -- 172.21.15.16:0/2252884000 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dfc0039f0 con 0x7f7e0c06f800 2023-05-20T01:17:08.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.490+0000 7f7e11a78700 1 -- 172.21.15.16:0/2252884000 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e0c0ce910 con 0x7f7e0c06f800 2023-05-20T01:17:08.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.490+0000 7f7e03fff700 1 -- 172.21.15.16:0/2252884000 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dfc003d80 con 0x7f7e0c06f800 2023-05-20T01:17:08.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.490+0000 7f7e11a78700 1 -- 172.21.15.16:0/2252884000 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e0c0caba0 con 0x7f7e0c06f800 2023-05-20T01:17:08.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.491+0000 7f7e11a78700 1 -- 172.21.15.16:0/2252884000 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e0c0720a0 con 0x7f7e0c06f800 2023-05-20T01:17:08.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.492+0000 7f7e03fff700 1 -- 172.21.15.16:0/2252884000 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7dfc019400 con 0x7f7e0c06f800 2023-05-20T01:17:08.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.492+0000 7f7e03fff700 1 --2- 172.21.15.16:0/2252884000 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f7dec06ace0 0x7f7dec06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:08.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.492+0000 7f7e03fff700 1 -- 172.21.15.16:0/2252884000 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f7dfc08cac0 con 0x7f7e0c06f800 2023-05-20T01:17:08.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.492+0000 7f7e0affd700 1 --2- 172.21.15.16:0/2252884000 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f7dec06ace0 0x7f7dec06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:08.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.493+0000 7f7e0affd700 1 --2- 172.21.15.16:0/2252884000 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f7dec06ace0 0x7f7dec06d190 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c0c6700 tx=0x7f7df4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:08.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.496+0000 7f7e03fff700 1 -- 172.21.15.16:0/2252884000 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f7dfc0579f0 con 0x7f7e0c06f800 2023-05-20T01:17:08.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.692+0000 7f7e11a78700 1 -- 172.21.15.16:0/2252884000 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7e0c0c9370 con 0x7f7e0c06f800 2023-05-20T01:17:08.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.692+0000 7f7e03fff700 1 -- 172.21.15.16:0/2252884000 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7dfc05b010 con 0x7f7e0c06f800 2023-05-20T01:17:08.695 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-05-20T01:17:08.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.694+0000 7f7e11a78700 1 -- 172.21.15.16:0/2252884000 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f7dec06ace0 msgr2=0x7f7dec06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:08.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.694+0000 7f7e11a78700 1 --2- 172.21.15.16:0/2252884000 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f7dec06ace0 0x7f7dec06d190 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c0c6700 tx=0x7f7df4008040 comp rx=0 tx=0).stop 2023-05-20T01:17:08.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.694+0000 7f7e11a78700 1 -- 172.21.15.16:0/2252884000 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e0c06f800 msgr2=0x7f7e0c148f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:08.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.694+0000 7f7e11a78700 1 --2- 172.21.15.16:0/2252884000 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e0c06f800 0x7f7e0c148f40 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc000c00 tx=0x7f7dfc004750 comp rx=0 tx=0).stop 2023-05-20T01:17:08.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.694+0000 7f7e11a78700 1 -- 172.21.15.16:0/2252884000 shutdown_connections 2023-05-20T01:17:08.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.694+0000 7f7e11a78700 1 --2- 172.21.15.16:0/2252884000 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7e0c1598b0 0x7f7e0c1e4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:08.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.694+0000 7f7e11a78700 1 --2- 172.21.15.16:0/2252884000 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e0c06f800 0x7f7e0c148f40 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:08.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.694+0000 7f7e11a78700 1 --2- 172.21.15.16:0/2252884000 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f7dec06ace0 0x7f7dec06d190 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:08.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.694+0000 7f7e11a78700 1 --2- 172.21.15.16:0/2252884000 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7e0c0b1a60 0x7f7e0c0c9650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:08.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.694+0000 7f7e11a78700 1 -- 172.21.15.16:0/2252884000 >> 172.21.15.16:0/2252884000 conn(0x7f7e0c06d220 msgr2=0x7f7e0c0cae90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:08.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.694+0000 7f7e11a78700 1 -- 172.21.15.16:0/2252884000 shutdown_connections 2023-05-20T01:17:08.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.694+0000 7f7e11a78700 1 -- 172.21.15.16:0/2252884000 wait complete. 2023-05-20T01:17:08.709 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-05-20T01:17:08.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.834+0000 7fd704362700 1 Processor -- start 2023-05-20T01:17:08.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.834+0000 7fd704362700 1 -- start start 2023-05-20T01:17:08.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.835+0000 7fd704362700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd6fc0cf350 0x7fd6fc0b19b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:08.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.835+0000 7fd704362700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6fc0b2020 con 0x7fd6fc0d16e0 2023-05-20T01:17:08.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.835+0000 7fd704362700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6fc0b2300 con 0x7fd6fc0af2f0 2023-05-20T01:17:08.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.835+0000 7fd704362700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6fc0b25e0 con 0x7fd6fc0cf350 2023-05-20T01:17:08.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.835+0000 7fd7028ff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd6fc0cf350 0x7fd6fc0b19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:08.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.835+0000 7fd7028ff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd6fc0cf350 0x7fd6fc0b19b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58834/0 (socket says 172.21.15.16:58834) 2023-05-20T01:17:08.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.835+0000 7fd7028ff700 1 -- 172.21.15.16:0/3037366107 learned_addr learned my addr 172.21.15.16:0/3037366107 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:08.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.836+0000 7fd7028ff700 1 -- 172.21.15.16:0/3037366107 >> v1:172.21.15.132:6789/0 conn(0x7fd6fc0af2f0 legacy=0x7fd6fc0cec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:08.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.836+0000 7fd7028ff700 1 -- 172.21.15.16:0/3037366107 >> v1:172.21.15.16:6789/0 conn(0x7fd6fc0d16e0 legacy=0x7fd6fc144350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:08.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.836+0000 7fd7028ff700 1 -- 172.21.15.16:0/3037366107 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6fc0addc0 con 0x7fd6fc0cf350 2023-05-20T01:17:08.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.836+0000 7fd7010fc700 1 -- 172.21.15.16:0/3037366107 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4042897140 0 0) 0x7fd6fc0b2020 con 0x7fd6fc0d16e0 2023-05-20T01:17:08.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.836+0000 7fd7028ff700 1 --2- 172.21.15.16:0/3037366107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6fc0cf350 0x7fd6fc0b19b0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd6f4009e80 tx=0x7fd6f400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aacad18327d9553f server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:08.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.836+0000 7fd7010fc700 1 -- 172.21.15.16:0/3037366107 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f400e070 con 0x7fd6fc0cf350 2023-05-20T01:17:08.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.836+0000 7fd7010fc700 1 -- 172.21.15.16:0/3037366107 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6f400b980 con 0x7fd6fc0cf350 2023-05-20T01:17:08.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.836+0000 7fd7010fc700 1 -- 172.21.15.16:0/3037366107 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f400bd30 con 0x7fd6fc0cf350 2023-05-20T01:17:08.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.837+0000 7fd704362700 1 -- 172.21.15.16:0/3037366107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6fc0cf350 msgr2=0x7fd6fc0b19b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:08.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.837+0000 7fd704362700 1 --2- 172.21.15.16:0/3037366107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6fc0cf350 0x7fd6fc0b19b0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd6f4009e80 tx=0x7fd6f400b330 comp rx=0 tx=0).stop 2023-05-20T01:17:08.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.837+0000 7fd704362700 1 -- 172.21.15.16:0/3037366107 shutdown_connections 2023-05-20T01:17:08.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.837+0000 7fd704362700 1 --2- 172.21.15.16:0/3037366107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6fc0cf350 0x7fd6fc0b19b0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:08.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.837+0000 7fd704362700 1 -- 172.21.15.16:0/3037366107 >> 172.21.15.16:0/3037366107 conn(0x7fd6fc06ae30 msgr2=0x7fd6fc09a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:08.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.837+0000 7fd704362700 1 -- 172.21.15.16:0/3037366107 shutdown_connections 2023-05-20T01:17:08.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.837+0000 7fd704362700 1 -- 172.21.15.16:0/3037366107 wait complete. 2023-05-20T01:17:08.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.838+0000 7fd704362700 1 Processor -- start 2023-05-20T01:17:08.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.838+0000 7fd704362700 1 -- start start 2023-05-20T01:17:08.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.839+0000 7fd704362700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd6fc0af2f0 0x7fd6fc09a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:08.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.839+0000 7fd704362700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd6fc0cf350 0x7fd6fc1511b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:08.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.839+0000 7fd704362700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6fc0d16e0 0x7fd6fc0b4110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:08.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.839+0000 7fd704362700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6fc156a10 con 0x7fd6fc0d16e0 2023-05-20T01:17:08.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.839+0000 7fd704362700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6fc1560e0 con 0x7fd6fc0cf350 2023-05-20T01:17:08.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.839+0000 7fd704362700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6fc1563c0 con 0x7fd6fc0af2f0 2023-05-20T01:17:08.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.839+0000 7fd7028ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6fc0d16e0 0x7fd6fc0b4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:08.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.839+0000 7fd7028ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6fc0d16e0 0x7fd6fc0b4110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58840/0 (socket says 172.21.15.16:58840) 2023-05-20T01:17:08.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.839+0000 7fd7028ff700 1 -- 172.21.15.16:0/1320211785 learned_addr learned my addr 172.21.15.16:0/1320211785 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:08.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.839+0000 7fd7018fd700 1 --2- 172.21.15.16:0/1320211785 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd6fc0cf350 0x7fd6fc1511b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:08.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.839+0000 7fd7020fe700 1 --2- 172.21.15.16:0/1320211785 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd6fc0af2f0 0x7fd6fc09a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:08.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.839+0000 7fd7028ff700 1 -- 172.21.15.16:0/1320211785 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd6fc0af2f0 msgr2=0x7fd6fc09a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:08.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.839+0000 7fd7028ff700 1 --2- 172.21.15.16:0/1320211785 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd6fc0af2f0 0x7fd6fc09a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:08.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.839+0000 7fd7028ff700 1 -- 172.21.15.16:0/1320211785 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd6fc0cf350 msgr2=0x7fd6fc1511b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:08.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.839+0000 7fd7028ff700 1 --2- 172.21.15.16:0/1320211785 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd6fc0cf350 0x7fd6fc1511b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:08.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.839+0000 7fd7028ff700 1 -- 172.21.15.16:0/1320211785 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6f4009b30 con 0x7fd6fc0d16e0 2023-05-20T01:17:08.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.839+0000 7fd7020fe700 1 --2- 172.21.15.16:0/1320211785 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd6fc0af2f0 0x7fd6fc09a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:17:08.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.839+0000 7fd7028ff700 1 --2- 172.21.15.16:0/1320211785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6fc0d16e0 0x7fd6fc0b4110 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fd6f4012000 tx=0x7fd6f4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:08.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.839+0000 7fd6f2ffd700 1 -- 172.21.15.16:0/1320211785 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f40099b0 con 0x7fd6fc0d16e0 2023-05-20T01:17:08.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.840+0000 7fd6f2ffd700 1 -- 172.21.15.16:0/1320211785 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6f40092e0 con 0x7fd6fc0d16e0 2023-05-20T01:17:08.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.840+0000 7fd6f2ffd700 1 -- 172.21.15.16:0/1320211785 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f401c660 con 0x7fd6fc0d16e0 2023-05-20T01:17:08.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.840+0000 7fd704362700 1 -- 172.21.15.16:0/1320211785 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6fc1655f0 con 0x7fd6fc0d16e0 2023-05-20T01:17:08.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.840+0000 7fd704362700 1 -- 172.21.15.16:0/1320211785 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6fc0b28c0 con 0x7fd6fc0d16e0 2023-05-20T01:17:08.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.841+0000 7fd704362700 1 -- 172.21.15.16:0/1320211785 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6fc165780 con 0x7fd6fc0d16e0 2023-05-20T01:17:08.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.841+0000 7fd6f2ffd700 1 -- 172.21.15.16:0/1320211785 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd6f401c7c0 con 0x7fd6fc0d16e0 2023-05-20T01:17:08.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.841+0000 7fd6f2ffd700 1 --2- 172.21.15.16:0/1320211785 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd6e006ace0 0x7fd6e006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:08.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.842+0000 7fd6f2ffd700 1 -- 172.21.15.16:0/1320211785 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fd6f408b100 con 0x7fd6fc0d16e0 2023-05-20T01:17:08.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.842+0000 7fd7020fe700 1 --2- 172.21.15.16:0/1320211785 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd6e006ace0 0x7fd6e006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:08.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.842+0000 7fd7020fe700 1 --2- 172.21.15.16:0/1320211785 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd6e006ace0 0x7fd6e006d190 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec000f60 tx=0x7fd6ec009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:08.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:08.846+0000 7fd6f2ffd700 1 -- 172.21.15.16:0/1320211785 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fd6f4055f50 con 0x7fd6fc0d16e0 2023-05-20T01:17:09.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.044+0000 7fd704362700 1 -- 172.21.15.16:0/1320211785 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd6fc157650 con 0x7fd6fc0d16e0 2023-05-20T01:17:09.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.045+0000 7fd6f2ffd700 1 -- 172.21.15.16:0/1320211785 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd6f4059570 con 0x7fd6fc0d16e0 2023-05-20T01:17:09.048 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-05-20T01:17:09.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.047+0000 7fd704362700 1 -- 172.21.15.16:0/1320211785 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd6e006ace0 msgr2=0x7fd6e006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:09.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.047+0000 7fd704362700 1 --2- 172.21.15.16:0/1320211785 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd6e006ace0 0x7fd6e006d190 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec000f60 tx=0x7fd6ec009040 comp rx=0 tx=0).stop 2023-05-20T01:17:09.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.047+0000 7fd704362700 1 -- 172.21.15.16:0/1320211785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6fc0d16e0 msgr2=0x7fd6fc0b4110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:09.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.047+0000 7fd704362700 1 --2- 172.21.15.16:0/1320211785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6fc0d16e0 0x7fd6fc0b4110 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fd6f4012000 tx=0x7fd6f4004ad0 comp rx=0 tx=0).stop 2023-05-20T01:17:09.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.047+0000 7fd704362700 1 -- 172.21.15.16:0/1320211785 shutdown_connections 2023-05-20T01:17:09.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.047+0000 7fd704362700 1 --2- 172.21.15.16:0/1320211785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6fc0d16e0 0x7fd6fc0b4110 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:09.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.047+0000 7fd704362700 1 --2- 172.21.15.16:0/1320211785 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd6e006ace0 0x7fd6e006d190 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:09.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.047+0000 7fd704362700 1 --2- 172.21.15.16:0/1320211785 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd6fc0af2f0 0x7fd6fc09a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:09.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.047+0000 7fd704362700 1 --2- 172.21.15.16:0/1320211785 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd6fc0cf350 0x7fd6fc1511b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:09.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.047+0000 7fd704362700 1 -- 172.21.15.16:0/1320211785 >> 172.21.15.16:0/1320211785 conn(0x7fd6fc06ae30 msgr2=0x7fd6fc15d780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:09.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.047+0000 7fd704362700 1 -- 172.21.15.16:0/1320211785 shutdown_connections 2023-05-20T01:17:09.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.047+0000 7fd704362700 1 -- 172.21.15.16:0/1320211785 wait complete. 2023-05-20T01:17:09.062 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-05-20T01:17:09.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.188+0000 7fabfa318700 1 Processor -- start 2023-05-20T01:17:09.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.188+0000 7fabfa318700 1 -- start start 2023-05-20T01:17:09.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.188+0000 7fabfa318700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fabf415cec0 0x7fabf4071bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:09.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.188+0000 7fabfa318700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fabf415d290 0x7fabf4072130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:09.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.188+0000 7fabfa318700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fabf40cf2c0 0x7fabf40cf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:09.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.188+0000 7fabfa318700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabf40cd630 con 0x7fabf40cf2c0 2023-05-20T01:17:09.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.188+0000 7fabfa318700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabf40cca60 con 0x7fabf415cec0 2023-05-20T01:17:09.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.188+0000 7fabfa318700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabf40ccd00 con 0x7fabf415d290 2023-05-20T01:17:09.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.189+0000 7fabf88b5700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fabf40cf2c0 0x7fabf40cf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:09.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.189+0000 7fabf88b5700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fabf40cf2c0 0x7fabf40cf720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58848/0 (socket says 172.21.15.16:58848) 2023-05-20T01:17:09.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.189+0000 7fabf88b5700 1 -- 172.21.15.16:0/3879847262 learned_addr learned my addr 172.21.15.16:0/3879847262 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:09.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.189+0000 7fabf3fff700 1 --2- 172.21.15.16:0/3879847262 >> v2:172.21.15.132:3300/0 conn(0x7fabf415cec0 0x7fabf4071bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:09.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.189+0000 7fabf37fe700 1 --2- 172.21.15.16:0/3879847262 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fabf415d290 0x7fabf4072130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:09.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.189+0000 7fabf88b5700 1 -- 172.21.15.16:0/3879847262 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fabf415d290 msgr2=0x7fabf4072130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:09.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.189+0000 7fabf88b5700 1 --2- 172.21.15.16:0/3879847262 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fabf415d290 0x7fabf4072130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:09.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.189+0000 7fabf88b5700 1 -- 172.21.15.16:0/3879847262 >> v2:172.21.15.132:3300/0 conn(0x7fabf415cec0 msgr2=0x7fabf4071bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:09.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.189+0000 7fabf88b5700 1 --2- 172.21.15.16:0/3879847262 >> v2:172.21.15.132:3300/0 conn(0x7fabf415cec0 0x7fabf4071bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:09.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.189+0000 7fabf88b5700 1 -- 172.21.15.16:0/3879847262 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabf40ccfe0 con 0x7fabf40cf2c0 2023-05-20T01:17:09.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.189+0000 7fabf88b5700 1 --2- 172.21.15.16:0/3879847262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabf40cf2c0 0x7fabf40cf720 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fabf4072770 tx=0x7fabec00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b7da4d05daa0c46 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:09.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.189+0000 7fabf2ffd700 1 -- 172.21.15.16:0/3879847262 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabec00e070 con 0x7fabf40cf2c0 2023-05-20T01:17:09.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.189+0000 7fabf2ffd700 1 -- 172.21.15.16:0/3879847262 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabec00b9c0 con 0x7fabf40cf2c0 2023-05-20T01:17:09.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.189+0000 7fabf2ffd700 1 -- 172.21.15.16:0/3879847262 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabec00bd70 con 0x7fabf40cf2c0 2023-05-20T01:17:09.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.190+0000 7fabfa318700 1 -- 172.21.15.16:0/3879847262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabf40cf2c0 msgr2=0x7fabf40cf720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:09.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.190+0000 7fabfa318700 1 --2- 172.21.15.16:0/3879847262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabf40cf2c0 0x7fabf40cf720 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fabf4072770 tx=0x7fabec00b330 comp rx=0 tx=0).stop 2023-05-20T01:17:09.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.190+0000 7fabfa318700 1 -- 172.21.15.16:0/3879847262 shutdown_connections 2023-05-20T01:17:09.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.190+0000 7fabfa318700 1 --2- 172.21.15.16:0/3879847262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabf40cf2c0 0x7fabf40cf720 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:09.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.190+0000 7fabfa318700 1 --2- 172.21.15.16:0/3879847262 >> v2:172.21.15.132:3300/0 conn(0x7fabf415cec0 0x7fabf4071bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:09.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.190+0000 7fabfa318700 1 --2- 172.21.15.16:0/3879847262 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fabf415d290 0x7fabf4072130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:09.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.190+0000 7fabfa318700 1 -- 172.21.15.16:0/3879847262 >> 172.21.15.16:0/3879847262 conn(0x7fabf40cffd0 msgr2=0x7fabf40d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:09.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.190+0000 7fabfa318700 1 -- 172.21.15.16:0/3879847262 shutdown_connections 2023-05-20T01:17:09.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.190+0000 7fabfa318700 1 -- 172.21.15.16:0/3879847262 wait complete. 2023-05-20T01:17:09.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.191+0000 7fabfa318700 1 Processor -- start 2023-05-20T01:17:09.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.191+0000 7fabfa318700 1 -- start start 2023-05-20T01:17:09.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.192+0000 7fabfa318700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fabf40cf2c0 0x7fabf4152420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:09.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.192+0000 7fabfa318700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabf415cec0 0x7fabf40b2e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:09.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.192+0000 7fabfa318700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fabf415d290 0x7fabf40b4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:09.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.192+0000 7fabfa318700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabf40b3690 con 0x7fabf415cec0 2023-05-20T01:17:09.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.192+0000 7fabfa318700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabf40b3970 con 0x7fabf415d290 2023-05-20T01:17:09.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.192+0000 7fabfa318700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabf40b3c50 con 0x7fabf40cf2c0 2023-05-20T01:17:09.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.192+0000 7fabf37fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabf415cec0 0x7fabf40b2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:09.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.192+0000 7fabf37fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabf415cec0 0x7fabf40b2e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58852/0 (socket says 172.21.15.16:58852) 2023-05-20T01:17:09.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.192+0000 7fabf37fe700 1 -- 172.21.15.16:0/963374500 learned_addr learned my addr 172.21.15.16:0/963374500 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:09.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.192+0000 7fabf88b5700 1 --2- 172.21.15.16:0/963374500 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fabf415d290 0x7fabf40b4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:09.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.192+0000 7fabf3fff700 1 --2- 172.21.15.16:0/963374500 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fabf40cf2c0 0x7fabf4152420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:09.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.192+0000 7fabf37fe700 1 -- 172.21.15.16:0/963374500 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fabf40cf2c0 msgr2=0x7fabf4152420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:09.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.192+0000 7fabf37fe700 1 --2- 172.21.15.16:0/963374500 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fabf40cf2c0 0x7fabf4152420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:09.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.192+0000 7fabf37fe700 1 -- 172.21.15.16:0/963374500 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fabf415d290 msgr2=0x7fabf40b4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:09.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.192+0000 7fabf37fe700 1 --2- 172.21.15.16:0/963374500 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fabf415d290 0x7fabf40b4730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:09.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.192+0000 7fabf37fe700 1 -- 172.21.15.16:0/963374500 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabec009b30 con 0x7fabf415cec0 2023-05-20T01:17:09.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.192+0000 7fabf3fff700 1 --2- 172.21.15.16:0/963374500 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fabf40cf2c0 0x7fabf4152420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:17:09.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.193+0000 7fabf37fe700 1 --2- 172.21.15.16:0/963374500 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabf415cec0 0x7fabf40b2e40 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fabe400e810 tx=0x7fabe400eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:09.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.193+0000 7fabf0ff9700 1 -- 172.21.15.16:0/963374500 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabe4013610 con 0x7fabf415cec0 2023-05-20T01:17:09.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.193+0000 7fabf0ff9700 1 -- 172.21.15.16:0/963374500 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabe4013770 con 0x7fabf415cec0 2023-05-20T01:17:09.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.193+0000 7fabfa318700 1 -- 172.21.15.16:0/963374500 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabf40cfc60 con 0x7fabf415cec0 2023-05-20T01:17:09.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.193+0000 7fabf0ff9700 1 -- 172.21.15.16:0/963374500 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabe4013b20 con 0x7fabf415cec0 2023-05-20T01:17:09.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.193+0000 7fabfa318700 1 -- 172.21.15.16:0/963374500 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabf40cd2c0 con 0x7fabf415cec0 2023-05-20T01:17:09.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.194+0000 7fabfa318700 1 -- 172.21.15.16:0/963374500 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabf40cfdf0 con 0x7fabf415cec0 2023-05-20T01:17:09.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.194+0000 7fabf0ff9700 1 -- 172.21.15.16:0/963374500 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fabe4013c80 con 0x7fabf415cec0 2023-05-20T01:17:09.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.195+0000 7fabf0ff9700 1 --2- 172.21.15.16:0/963374500 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fabd406ace0 0x7fabd406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:09.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.195+0000 7fabf0ff9700 1 -- 172.21.15.16:0/963374500 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fabe4089400 con 0x7fabf415cec0 2023-05-20T01:17:09.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.195+0000 7fabf3fff700 1 --2- 172.21.15.16:0/963374500 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fabd406ace0 0x7fabd406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:09.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.195+0000 7fabf3fff700 1 --2- 172.21.15.16:0/963374500 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fabd406ace0 0x7fabd406d190 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fabdc006fd0 tx=0x7fabdc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:09.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.199+0000 7fabf0ff9700 1 -- 172.21.15.16:0/963374500 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fabe4057bc0 con 0x7fabf415cec0 2023-05-20T01:17:09.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.399+0000 7fabfa318700 1 -- 172.21.15.16:0/963374500 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fabf40747b0 con 0x7fabf415cec0 2023-05-20T01:17:09.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.399+0000 7fabf0ff9700 1 -- 172.21.15.16:0/963374500 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7fabe4057750 con 0x7fabf415cec0 2023-05-20T01:17:09.403 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2023-05-20T01:17:09.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.402+0000 7fabfa318700 1 -- 172.21.15.16:0/963374500 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fabd406ace0 msgr2=0x7fabd406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:09.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.402+0000 7fabfa318700 1 --2- 172.21.15.16:0/963374500 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fabd406ace0 0x7fabd406d190 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fabdc006fd0 tx=0x7fabdc008040 comp rx=0 tx=0).stop 2023-05-20T01:17:09.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.402+0000 7fabfa318700 1 -- 172.21.15.16:0/963374500 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabf415cec0 msgr2=0x7fabf40b2e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:09.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.402+0000 7fabfa318700 1 --2- 172.21.15.16:0/963374500 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabf415cec0 0x7fabf40b2e40 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fabe400e810 tx=0x7fabe400eb20 comp rx=0 tx=0).stop 2023-05-20T01:17:09.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.402+0000 7fabfa318700 1 -- 172.21.15.16:0/963374500 shutdown_connections 2023-05-20T01:17:09.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.402+0000 7fabfa318700 1 --2- 172.21.15.16:0/963374500 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fabf415d290 0x7fabf40b4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:09.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.402+0000 7fabfa318700 1 --2- 172.21.15.16:0/963374500 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fabd406ace0 0x7fabd406d190 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:09.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.402+0000 7fabfa318700 1 --2- 172.21.15.16:0/963374500 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fabf40cf2c0 0x7fabf4152420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:09.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.402+0000 7fabfa318700 1 --2- 172.21.15.16:0/963374500 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabf415cec0 0x7fabf40b2e40 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:09.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.402+0000 7fabfa318700 1 -- 172.21.15.16:0/963374500 >> 172.21.15.16:0/963374500 conn(0x7fabf40cffd0 msgr2=0x7fabf40cba40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:09.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.402+0000 7fabfa318700 1 -- 172.21.15.16:0/963374500 shutdown_connections 2023-05-20T01:17:09.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.402+0000 7fabfa318700 1 -- 172.21.15.16:0/963374500 wait complete. 2023-05-20T01:17:09.416 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-05-20T01:17:09.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.541+0000 7fd2e60cf700 1 Processor -- start 2023-05-20T01:17:09.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.541+0000 7fd2e60cf700 1 -- start start 2023-05-20T01:17:09.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.542+0000 7fd2e60cf700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd2e015cec0 0x7fd2e0071bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:09.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.542+0000 7fd2e60cf700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd2e015d290 0x7fd2e0072130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:09.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.542+0000 7fd2e60cf700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2e00cfe30 con 0x7fd2e00cf2c0 2023-05-20T01:17:09.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.542+0000 7fd2e60cf700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2e00cd630 con 0x7fd2e015d290 2023-05-20T01:17:09.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.542+0000 7fd2e60cf700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2e00cca60 con 0x7fd2e015cec0 2023-05-20T01:17:09.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.542+0000 7fd2deffd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd2e015d290 0x7fd2e0072130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:09.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.542+0000 7fd2deffd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd2e015d290 0x7fd2e0072130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58860/0 (socket says 172.21.15.16:58860) 2023-05-20T01:17:09.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.542+0000 7fd2deffd700 1 -- 172.21.15.16:0/394201380 learned_addr learned my addr 172.21.15.16:0/394201380 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:09.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.542+0000 7fd2df7fe700 1 --2- 172.21.15.16:0/394201380 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd2e015cec0 0x7fd2e0071bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:09.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.542+0000 7fd2deffd700 1 -- 172.21.15.16:0/394201380 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd2e015cec0 msgr2=0x7fd2e0071bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:09.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.542+0000 7fd2deffd700 1 --2- 172.21.15.16:0/394201380 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd2e015cec0 0x7fd2e0071bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:09.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.542+0000 7fd2deffd700 1 -- 172.21.15.16:0/394201380 >> v1:172.21.15.132:6789/0 conn(0x7fd2e00cf2c0 legacy=0x7fd2e00cf720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:09.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.542+0000 7fd2deffd700 1 -- 172.21.15.16:0/394201380 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2e00ccd00 con 0x7fd2e015d290 2023-05-20T01:17:09.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.543+0000 7fd2deffd700 1 --2- 172.21.15.16:0/394201380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2e015d290 0x7fd2e0072130 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fd2c80098f0 tx=0x7fd2c8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=622928b8efbddbc3 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:09.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.543+0000 7fd2de7fc700 1 -- 172.21.15.16:0/394201380 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c8011830 con 0x7fd2e015d290 2023-05-20T01:17:09.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.543+0000 7fd2de7fc700 1 -- 172.21.15.16:0/394201380 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2c8011990 con 0x7fd2e015d290 2023-05-20T01:17:09.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.543+0000 7fd2de7fc700 1 -- 172.21.15.16:0/394201380 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c8011d20 con 0x7fd2e015d290 2023-05-20T01:17:09.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.543+0000 7fd2e60cf700 1 -- 172.21.15.16:0/394201380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2e015d290 msgr2=0x7fd2e0072130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:09.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.543+0000 7fd2e60cf700 1 --2- 172.21.15.16:0/394201380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2e015d290 0x7fd2e0072130 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fd2c80098f0 tx=0x7fd2c8009c00 comp rx=0 tx=0).stop 2023-05-20T01:17:09.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.543+0000 7fd2e60cf700 1 -- 172.21.15.16:0/394201380 shutdown_connections 2023-05-20T01:17:09.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.543+0000 7fd2e60cf700 1 --2- 172.21.15.16:0/394201380 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd2e015cec0 0x7fd2e0071bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:09.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.543+0000 7fd2e60cf700 1 --2- 172.21.15.16:0/394201380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2e015d290 0x7fd2e0072130 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:09.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.543+0000 7fd2e60cf700 1 -- 172.21.15.16:0/394201380 >> 172.21.15.16:0/394201380 conn(0x7fd2e00cffd0 msgr2=0x7fd2e00d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:09.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.543+0000 7fd2e60cf700 1 -- 172.21.15.16:0/394201380 shutdown_connections 2023-05-20T01:17:09.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.543+0000 7fd2e60cf700 1 -- 172.21.15.16:0/394201380 wait complete. 2023-05-20T01:17:09.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.545+0000 7fd2e60cf700 1 Processor -- start 2023-05-20T01:17:09.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.545+0000 7fd2e60cf700 1 -- start start 2023-05-20T01:17:09.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.545+0000 7fd2e60cf700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2e00cf2c0 0x7fd2e0155530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:09.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.546+0000 7fd2e60cf700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd2e015cec0 0x7fd2e015ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:09.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.546+0000 7fd2e60cf700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd2e015d290 0x7fd2e015b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:09.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.546+0000 7fd2e60cf700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2e0155a70 con 0x7fd2e00cf2c0 2023-05-20T01:17:09.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.546+0000 7fd2e60cf700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2e00b50e0 con 0x7fd2e015d290 2023-05-20T01:17:09.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.546+0000 7fd2e60cf700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2e01565f0 con 0x7fd2e015cec0 2023-05-20T01:17:09.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.546+0000 7fd2df7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2e00cf2c0 0x7fd2e0155530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:09.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.546+0000 7fd2df7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2e00cf2c0 0x7fd2e0155530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58864/0 (socket says 172.21.15.16:58864) 2023-05-20T01:17:09.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.546+0000 7fd2df7fe700 1 -- 172.21.15.16:0/4126942895 learned_addr learned my addr 172.21.15.16:0/4126942895 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:09.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.546+0000 7fd2dffff700 1 --2- 172.21.15.16:0/4126942895 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd2e015d290 0x7fd2e015b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:09.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.546+0000 7fd2df7fe700 1 -- 172.21.15.16:0/4126942895 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd2e015cec0 msgr2=0x7fd2e015ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:09.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.546+0000 7fd2df7fe700 1 --2- 172.21.15.16:0/4126942895 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd2e015cec0 0x7fd2e015ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:09.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.546+0000 7fd2df7fe700 1 -- 172.21.15.16:0/4126942895 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd2e015d290 msgr2=0x7fd2e015b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:09.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.546+0000 7fd2df7fe700 1 --2- 172.21.15.16:0/4126942895 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd2e015d290 0x7fd2e015b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:09.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.546+0000 7fd2df7fe700 1 -- 172.21.15.16:0/4126942895 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2c8009550 con 0x7fd2e00cf2c0 2023-05-20T01:17:09.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.546+0000 7fd2dffff700 1 --2- 172.21.15.16:0/4126942895 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd2e015d290 0x7fd2e015b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:17:09.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.546+0000 7fd2df7fe700 1 --2- 172.21.15.16:0/4126942895 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2e00cf2c0 0x7fd2e0155530 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fd2d000e990 tx=0x7fd2d000ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:09.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.546+0000 7fd2d7fff700 1 -- 172.21.15.16:0/4126942895 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2d00096f0 con 0x7fd2e00cf2c0 2023-05-20T01:17:09.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.546+0000 7fd2d7fff700 1 -- 172.21.15.16:0/4126942895 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2d000c850 con 0x7fd2e00cf2c0 2023-05-20T01:17:09.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.546+0000 7fd2d7fff700 1 -- 172.21.15.16:0/4126942895 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2d000cbc0 con 0x7fd2e00cf2c0 2023-05-20T01:17:09.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.546+0000 7fd2e60cf700 1 -- 172.21.15.16:0/4126942895 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2e00cd2c0 con 0x7fd2e00cf2c0 2023-05-20T01:17:09.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.546+0000 7fd2e60cf700 1 -- 172.21.15.16:0/4126942895 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2e00ccfe0 con 0x7fd2e00cf2c0 2023-05-20T01:17:09.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.547+0000 7fd2e60cf700 1 -- 172.21.15.16:0/4126942895 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2e0160890 con 0x7fd2e00cf2c0 2023-05-20T01:17:09.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.548+0000 7fd2d7fff700 1 -- 172.21.15.16:0/4126942895 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd2d000cd20 con 0x7fd2e00cf2c0 2023-05-20T01:17:09.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.548+0000 7fd2d7fff700 1 --2- 172.21.15.16:0/4126942895 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd2c006ace0 0x7fd2c006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:09.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.548+0000 7fd2d7fff700 1 -- 172.21.15.16:0/4126942895 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fd2d0089bc0 con 0x7fd2e00cf2c0 2023-05-20T01:17:09.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.548+0000 7fd2deffd700 1 --2- 172.21.15.16:0/4126942895 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd2c006ace0 0x7fd2c006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:09.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.549+0000 7fd2deffd700 1 --2- 172.21.15.16:0/4126942895 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd2c006ace0 0x7fd2c006d190 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd2e015b810 tx=0x7fd2c8010000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:09.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.553+0000 7fd2d7fff700 1 -- 172.21.15.16:0/4126942895 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fd2d0054a10 con 0x7fd2e00cf2c0 2023-05-20T01:17:09.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.749+0000 7fd2e60cf700 1 -- 172.21.15.16:0/4126942895 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fd2e0156730 con 0x7fd2e00cf2c0 2023-05-20T01:17:09.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.750+0000 7fd2d7fff700 1 -- 172.21.15.16:0/4126942895 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7fd2d0021020 con 0x7fd2e00cf2c0 2023-05-20T01:17:09.753 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-05-20T01:17:09.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.752+0000 7fd2e60cf700 1 -- 172.21.15.16:0/4126942895 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd2c006ace0 msgr2=0x7fd2c006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:09.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.752+0000 7fd2e60cf700 1 --2- 172.21.15.16:0/4126942895 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd2c006ace0 0x7fd2c006d190 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd2e015b810 tx=0x7fd2c8010000 comp rx=0 tx=0).stop 2023-05-20T01:17:09.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.752+0000 7fd2e60cf700 1 -- 172.21.15.16:0/4126942895 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2e00cf2c0 msgr2=0x7fd2e0155530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:09.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.752+0000 7fd2e60cf700 1 --2- 172.21.15.16:0/4126942895 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2e00cf2c0 0x7fd2e0155530 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fd2d000e990 tx=0x7fd2d000ed50 comp rx=0 tx=0).stop 2023-05-20T01:17:09.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.752+0000 7fd2e60cf700 1 -- 172.21.15.16:0/4126942895 shutdown_connections 2023-05-20T01:17:09.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.752+0000 7fd2e60cf700 1 --2- 172.21.15.16:0/4126942895 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd2e015d290 0x7fd2e015b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:09.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.752+0000 7fd2e60cf700 1 --2- 172.21.15.16:0/4126942895 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2e00cf2c0 0x7fd2e0155530 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:09.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.752+0000 7fd2e60cf700 1 --2- 172.21.15.16:0/4126942895 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd2c006ace0 0x7fd2c006d190 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:09.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.752+0000 7fd2e60cf700 1 --2- 172.21.15.16:0/4126942895 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd2e015cec0 0x7fd2e015ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:09.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.752+0000 7fd2e60cf700 1 -- 172.21.15.16:0/4126942895 >> 172.21.15.16:0/4126942895 conn(0x7fd2e00cffd0 msgr2=0x7fd2e015bb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:09.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.752+0000 7fd2e60cf700 1 -- 172.21.15.16:0/4126942895 shutdown_connections 2023-05-20T01:17:09.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.752+0000 7fd2e60cf700 1 -- 172.21.15.16:0/4126942895 wait complete. 2023-05-20T01:17:09.767 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-20T01:17:09.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.890+0000 7fa524d7d700 1 Processor -- start 2023-05-20T01:17:09.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.890+0000 7fa524d7d700 1 -- start start 2023-05-20T01:17:09.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.891+0000 7fa524d7d700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fa5200cf370 0x7fa5200d1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:09.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.891+0000 7fa524d7d700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5200cc500 con 0x7fa520144020 2023-05-20T01:17:09.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.891+0000 7fa524d7d700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5200cb8f0 con 0x7fa5200b2760 2023-05-20T01:17:09.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.891+0000 7fa524d7d700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5200cb6e0 con 0x7fa5200cf370 2023-05-20T01:17:09.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.891+0000 7fa51dd9b700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fa520144020 0x7fa5200cec60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:42168/0 (socket says 172.21.15.16:42168) 2023-05-20T01:17:09.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.891+0000 7fa51dd9b700 1 -- 172.21.15.16:0/3059130121 learned_addr learned my addr 172.21.15.16:0/3059130121 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:17:09.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.891+0000 7fa51ed9d700 1 --2- 172.21.15.16:0/3059130121 >> v2:172.21.15.132:3300/0 conn(0x7fa5200cf370 0x7fa5200d1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:09.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.891+0000 7fa51d59a700 1 -- 172.21.15.16:0/3059130121 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 528906582 0 0) 0x7fa5200cc500 con 0x7fa520144020 2023-05-20T01:17:09.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.891+0000 7fa51d59a700 1 -- 172.21.15.16:0/3059130121 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5080035b0 con 0x7fa520144020 2023-05-20T01:17:09.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.891+0000 7fa51d59a700 1 -- 172.21.15.16:0/3059130121 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 343763048 0 0) 0x7fa5200cb8f0 con 0x7fa5200b2760 2023-05-20T01:17:09.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.891+0000 7fa51d59a700 1 -- 172.21.15.16:0/3059130121 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5200cc500 con 0x7fa5200b2760 2023-05-20T01:17:09.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.891+0000 7fa51d59a700 1 -- 172.21.15.16:0/3059130121 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2613341232 0 0) 0x7fa5080035b0 con 0x7fa520144020 2023-05-20T01:17:09.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.891+0000 7fa51d59a700 1 -- 172.21.15.16:0/3059130121 >> v2:172.21.15.132:3300/0 conn(0x7fa5200cf370 msgr2=0x7fa5200d1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:09.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.891+0000 7fa51d59a700 1 --2- 172.21.15.16:0/3059130121 >> v2:172.21.15.132:3300/0 conn(0x7fa5200cf370 0x7fa5200d1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:09.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.891+0000 7fa51d59a700 1 -- 172.21.15.16:0/3059130121 >> v1:172.21.15.132:6789/0 conn(0x7fa5200b2760 legacy=0x7fa5200b0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:09.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.891+0000 7fa51d59a700 1 -- 172.21.15.16:0/3059130121 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5200cbbd0 con 0x7fa520144020 2023-05-20T01:17:09.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.891+0000 7fa51d59a700 1 -- 172.21.15.16:0/3059130121 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7fa514001670 con 0x7fa520144020 2023-05-20T01:17:09.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.891+0000 7fa51d59a700 1 -- 172.21.15.16:0/3059130121 >> v1:172.21.15.16:6789/0 conn(0x7fa520144020 legacy=0x7fa5200cec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:09.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.891+0000 7fa51d59a700 1 --2- 172.21.15.16:0/3059130121 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa508003c50 0x7fa508004040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:09.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.891+0000 7fa51d59a700 1 --2- 172.21.15.16:0/3059130121 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa508004850 0x7fa508004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:09.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.891+0000 7fa51d59a700 1 --2- 172.21.15.16:0/3059130121 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa508005200 0x7fa50800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:09.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.891+0000 7fa51d59a700 1 -- 172.21.15.16:0/3059130121 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5200cb6e0 con 0x7fa508003c50 2023-05-20T01:17:09.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.891+0000 7fa51d59a700 1 -- 172.21.15.16:0/3059130121 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa50800c110 con 0x7fa508005200 2023-05-20T01:17:09.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.891+0000 7fa51d59a700 1 -- 172.21.15.16:0/3059130121 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa50800c2d0 con 0x7fa508004850 2023-05-20T01:17:09.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.891+0000 7fa51ed9d700 1 --2- 172.21.15.16:0/3059130121 >> v2:172.21.15.132:3300/0 conn(0x7fa5200cf370 0x7fa5200d1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-05-20T01:17:09.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.892+0000 7fa51e59c700 1 --2- 172.21.15.16:0/3059130121 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa508003c50 0x7fa508004040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:09.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.892+0000 7fa51dd9b700 1 --2- 172.21.15.16:0/3059130121 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa508004850 0x7fa508004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:09.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.892+0000 7fa51ed9d700 1 --2- 172.21.15.16:0/3059130121 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa508005200 0x7fa50800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:09.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.892+0000 7fa51e59c700 1 -- 172.21.15.16:0/3059130121 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa508004850 msgr2=0x7fa508004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:09.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.892+0000 7fa51e59c700 1 --2- 172.21.15.16:0/3059130121 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa508004850 0x7fa508004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:09.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.892+0000 7fa51e59c700 1 -- 172.21.15.16:0/3059130121 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa508005200 msgr2=0x7fa50800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:09.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.892+0000 7fa51e59c700 1 --2- 172.21.15.16:0/3059130121 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa508005200 0x7fa50800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:09.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.892+0000 7fa51e59c700 1 -- 172.21.15.16:0/3059130121 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5080035b0 con 0x7fa508003c50 2023-05-20T01:17:09.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.892+0000 7fa51ed9d700 1 -- 172.21.15.16:0/3059130121 reap_dead start 2023-05-20T01:17:09.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.892+0000 7fa51e59c700 1 --2- 172.21.15.16:0/3059130121 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa508003c50 0x7fa508004040 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa510001930 tx=0x7fa508005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99d8ca98685902ef server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:09.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.892+0000 7fa51d59a700 1 -- 172.21.15.16:0/3059130121 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5100052d0 con 0x7fa508003c50 2023-05-20T01:17:09.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.892+0000 7fa51d59a700 1 -- 172.21.15.16:0/3059130121 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa510005450 con 0x7fa508003c50 2023-05-20T01:17:09.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.892+0000 7fa51d59a700 1 -- 172.21.15.16:0/3059130121 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5100038c0 con 0x7fa508003c50 2023-05-20T01:17:09.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.893+0000 7fa524d7d700 1 -- 172.21.15.16:0/3059130121 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa508003c50 msgr2=0x7fa508004040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:09.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.893+0000 7fa524d7d700 1 --2- 172.21.15.16:0/3059130121 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa508003c50 0x7fa508004040 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa510001930 tx=0x7fa508005880 comp rx=0 tx=0).stop 2023-05-20T01:17:09.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.893+0000 7fa524d7d700 1 -- 172.21.15.16:0/3059130121 shutdown_connections 2023-05-20T01:17:09.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.893+0000 7fa524d7d700 1 --2- 172.21.15.16:0/3059130121 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa508003c50 0x7fa508004040 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:09.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.893+0000 7fa524d7d700 1 -- 172.21.15.16:0/3059130121 >> 172.21.15.16:0/3059130121 conn(0x7fa52006ae30 msgr2=0x7fa52009a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:09.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.893+0000 7fa524d7d700 1 -- 172.21.15.16:0/3059130121 shutdown_connections 2023-05-20T01:17:09.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.893+0000 7fa524d7d700 1 -- 172.21.15.16:0/3059130121 wait complete. 2023-05-20T01:17:09.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.894+0000 7fa524d7d700 1 Processor -- start 2023-05-20T01:17:09.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.894+0000 7fa524d7d700 1 -- start start 2023-05-20T01:17:09.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.895+0000 7fa524d7d700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa508003c50 0x7fa52015be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:09.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.895+0000 7fa524d7d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa520159060 0x7fa5200b66e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:09.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.895+0000 7fa524d7d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5200b6c20 0x7fa5200b0220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:09.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.895+0000 7fa524d7d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5200b6250 con 0x7fa5200b6c20 2023-05-20T01:17:09.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.895+0000 7fa524d7d700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5200b5640 con 0x7fa520159060 2023-05-20T01:17:09.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.895+0000 7fa524d7d700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5200b5920 con 0x7fa508003c50 2023-05-20T01:17:09.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.895+0000 7fa51ed9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5200b6c20 0x7fa5200b0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:09.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.895+0000 7fa51ed9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5200b6c20 0x7fa5200b0220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58888/0 (socket says 172.21.15.16:58888) 2023-05-20T01:17:09.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.895+0000 7fa51ed9d700 1 -- 172.21.15.16:0/2536401642 learned_addr learned my addr 172.21.15.16:0/2536401642 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:09.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.895+0000 7fa51dd9b700 1 --2- 172.21.15.16:0/2536401642 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa520159060 0x7fa5200b66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:09.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.895+0000 7fa51e59c700 1 --2- 172.21.15.16:0/2536401642 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa508003c50 0x7fa52015be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:09.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.895+0000 7fa51ed9d700 1 -- 172.21.15.16:0/2536401642 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa508003c50 msgr2=0x7fa52015be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:09.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.895+0000 7fa51ed9d700 1 --2- 172.21.15.16:0/2536401642 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa508003c50 0x7fa52015be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:09.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.895+0000 7fa51ed9d700 1 -- 172.21.15.16:0/2536401642 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa520159060 msgr2=0x7fa5200b66e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:09.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.895+0000 7fa51ed9d700 1 --2- 172.21.15.16:0/2536401642 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa520159060 0x7fa5200b66e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:09.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.895+0000 7fa51ed9d700 1 -- 172.21.15.16:0/2536401642 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5200cc190 con 0x7fa5200b6c20 2023-05-20T01:17:09.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.895+0000 7fa51e59c700 1 --2- 172.21.15.16:0/2536401642 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa508003c50 0x7fa52015be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:17:09.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.895+0000 7fa51ed9d700 1 --2- 172.21.15.16:0/2536401642 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5200b6c20 0x7fa5200b0220 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa5200cf740 tx=0x7fa5180114b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:09.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.896+0000 7fa50effd700 1 -- 172.21.15.16:0/2536401642 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa51800e070 con 0x7fa5200b6c20 2023-05-20T01:17:09.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.896+0000 7fa50effd700 1 -- 172.21.15.16:0/2536401642 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa518003cf0 con 0x7fa5200b6c20 2023-05-20T01:17:09.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.896+0000 7fa524d7d700 1 -- 172.21.15.16:0/2536401642 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5200cbeb0 con 0x7fa5200b6c20 2023-05-20T01:17:09.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.896+0000 7fa50effd700 1 -- 172.21.15.16:0/2536401642 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa518003e80 con 0x7fa5200b6c20 2023-05-20T01:17:09.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.896+0000 7fa524d7d700 1 -- 172.21.15.16:0/2536401642 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5200d1680 con 0x7fa5200b6c20 2023-05-20T01:17:09.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.896+0000 7fa524d7d700 1 -- 172.21.15.16:0/2536401642 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5200747b0 con 0x7fa5200b6c20 2023-05-20T01:17:09.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.897+0000 7fa50effd700 1 -- 172.21.15.16:0/2536401642 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa518011a20 con 0x7fa5200b6c20 2023-05-20T01:17:09.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.897+0000 7fa50effd700 1 --2- 172.21.15.16:0/2536401642 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa50006ace0 0x7fa50006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:09.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.897+0000 7fa50effd700 1 -- 172.21.15.16:0/2536401642 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fa518089f30 con 0x7fa5200b6c20 2023-05-20T01:17:09.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.898+0000 7fa51e59c700 1 --2- 172.21.15.16:0/2536401642 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa50006ace0 0x7fa50006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:09.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.898+0000 7fa51e59c700 1 --2- 172.21.15.16:0/2536401642 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa50006ace0 0x7fa50006d190 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa510005040 tx=0x7fa510009030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:09.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:09.902+0000 7fa50effd700 1 -- 172.21.15.16:0/2536401642 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fa518054cf0 con 0x7fa5200b6c20 2023-05-20T01:17:10.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.099+0000 7fa524d7d700 1 -- 172.21.15.16:0/2536401642 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa5200b6390 con 0x7fa5200b6c20 2023-05-20T01:17:10.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.101+0000 7fa50effd700 1 -- 172.21.15.16:0/2536401642 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+20026 (secure 0 0 0) 0x7fa518058310 con 0x7fa5200b6c20 2023-05-20T01:17:10.104 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:17:10.104 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"fsid":"c2245cc5-ecab-4087-a59b-bc81db901676","created":"2023-05-20T01:16:09.549230+0000","modified":"2023-05-20T01:16:32.768495+0000","last_up_change":"2023-05-20T01:16:19.788624+0000","last_in_change":"2023-05-20T01:16:12.092160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-05-20T01:16:20.934366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-05-20T01:16:21.372294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-05-20T01:16:26.208287+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-05-20T01:16:27.292268+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-05-20T01:16:30.357285+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"317a83cd-19c8-4958-a301-6087ef342289","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6825","nonce":3923634179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6827","nonce":3923634179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6831","nonce":3923634179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6829","nonce":3923634179}]},"public_addr":"172.21.15.16:6825/3923634179","cluster_addr":"172.21.15.16:6827/3923634179","heartbeat_back_addr":"172.21.15.16:6831/3923634179","heartbeat_front_addr":"172.21.15.16:6829/3923634179","state":["exists","up"]},{"osd":1,"uuid":"5581046f-8219-4089-8d7d-b7846399505e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6809","nonce":3860264220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6811","nonce":3860264220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6815","nonce":3860264220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6813","nonce":3860264220}]},"public_addr":"172.21.15.16:6809/3860264220","cluster_addr":"172.21.15.16:6811/3860264220","heartbeat_back_addr":"172.21.15.16:6815/3860264220","heartbeat_front_addr":"172.21.15.16:6813/3860264220","state":["exists","up"]},{"osd":2,"uuid":"072e9e66-4133-478e-8d4a-64d46c2963f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6817","nonce":293187030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6819","nonce":293187030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6823","nonce":293187030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6821","nonce":293187030}]},"public_addr":"172.21.15.16:6817/293187030","cluster_addr":"172.21.15.16:6819/293187030","heartbeat_back_addr":"172.21.15.16:6823/293187030","heartbeat_front_addr":"172.21.15.16:6821/293187030","state":["exists","up"]},{"osd":3,"uuid":"11e1bba6-00f8-48dd-9028-d8d716b7f145","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6801","nonce":3669781890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6803","nonce":3669781890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6807","nonce":3669781890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6805","nonce":3669781890}]},"public_addr":"172.21.15.16:6801/3669781890","cluster_addr":"172.21.15.16:6803/3669781890","heartbeat_back_addr":"172.21.15.16:6807/3669781890","heartbeat_front_addr":"172.21.15.16:6805/3669781890","state":["exists","up"]},{"osd":4,"uuid":"b8909c3a-cf70-46cb-8d0b-009c707cf9d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6819","nonce":506317223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6823","nonce":506317223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6829","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6831","nonce":506317223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6827","nonce":506317223}]},"public_addr":"172.21.15.132:6819/506317223","cluster_addr":"172.21.15.132:6823/506317223","heartbeat_back_addr":"172.21.15.132:6831/506317223","heartbeat_front_addr":"172.21.15.132:6827/506317223","state":["exists","up"]},{"osd":5,"uuid":"b1818304-4583-47bf-a6ec-0bfbb5166233","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6818","nonce":1844689995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6822","nonce":1844689995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6830","nonce":1844689995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6826","nonce":1844689995}]},"public_addr":"172.21.15.132:6818/1844689995","cluster_addr":"172.21.15.132:6822/1844689995","heartbeat_back_addr":"172.21.15.132:6830/1844689995","heartbeat_front_addr":"172.21.15.132:6826/1844689995","state":["exists","up"]},{"osd":6,"uuid":"d37f4691-eed1-4a93-8818-18354f03f4a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6801","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6802","nonce":727581123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6805","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6807","nonce":727581123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6813","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6815","nonce":727581123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6811","nonce":727581123}]},"public_addr":"172.21.15.132:6802/727581123","cluster_addr":"172.21.15.132:6807/727581123","heartbeat_back_addr":"172.21.15.132:6815/727581123","heartbeat_front_addr":"172.21.15.132:6811/727581123","state":["exists","up"]},{"osd":7,"uuid":"4f712f62-5d1d-4816-8042-03209144f8b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6803","nonce":466001149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6806","nonce":466001149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6814","nonce":466001149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6810","nonce":466001149}]},"public_addr":"172.21.15.132:6803/466001149","cluster_addr":"172.21.15.132:6806/466001149","heartbeat_back_addr":"172.21.15.132:6814/466001149","heartbeat_front_addr":"172.21.15.132:6810/466001149","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.338863+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.278680+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305694+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.246835+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.276237+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.304441+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305421+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.267050+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-20T01:17:10.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.103+0000 7fa524d7d700 1 -- 172.21.15.16:0/2536401642 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa50006ace0 msgr2=0x7fa50006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:10.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.103+0000 7fa524d7d700 1 --2- 172.21.15.16:0/2536401642 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa50006ace0 0x7fa50006d190 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa510005040 tx=0x7fa510009030 comp rx=0 tx=0).stop 2023-05-20T01:17:10.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.103+0000 7fa524d7d700 1 -- 172.21.15.16:0/2536401642 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5200b6c20 msgr2=0x7fa5200b0220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:10.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.103+0000 7fa524d7d700 1 --2- 172.21.15.16:0/2536401642 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5200b6c20 0x7fa5200b0220 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa5200cf740 tx=0x7fa5180114b0 comp rx=0 tx=0).stop 2023-05-20T01:17:10.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.103+0000 7fa524d7d700 1 -- 172.21.15.16:0/2536401642 shutdown_connections 2023-05-20T01:17:10.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.103+0000 7fa524d7d700 1 --2- 172.21.15.16:0/2536401642 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5200b6c20 0x7fa5200b0220 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:10.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.103+0000 7fa524d7d700 1 --2- 172.21.15.16:0/2536401642 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa50006ace0 0x7fa50006d190 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:10.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.103+0000 7fa524d7d700 1 --2- 172.21.15.16:0/2536401642 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa508003c50 0x7fa52015be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:10.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.103+0000 7fa524d7d700 1 --2- 172.21.15.16:0/2536401642 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa520159060 0x7fa5200b66e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:10.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.103+0000 7fa524d7d700 1 -- 172.21.15.16:0/2536401642 >> 172.21.15.16:0/2536401642 conn(0x7fa52006ae30 msgr2=0x7fa52006e390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:10.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.103+0000 7fa524d7d700 1 -- 172.21.15.16:0/2536401642 shutdown_connections 2023-05-20T01:17:10.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.103+0000 7fa524d7d700 1 -- 172.21.15.16:0/2536401642 wait complete. 2023-05-20T01:17:10.118 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-05-20T01:16:20.934366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-05-20T01:16:21.372294+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-05-20T01:16:26.208287+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-05-20T01:16:27.292268+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-05-20T01:16:30.357285+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-05-20T01:17:10.119 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-05-20T01:17:10.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.243+0000 7fb86875a700 1 Processor -- start 2023-05-20T01:17:10.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.243+0000 7fb86875a700 1 -- start start 2023-05-20T01:17:10.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.244+0000 7fb86875a700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb86015cec0 0x7fb860071bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:10.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.244+0000 7fb86875a700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fb860072130 0x7fb8600cf9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:10.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.244+0000 7fb86875a700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb860072590 con 0x7fb86015d290 2023-05-20T01:17:10.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.244+0000 7fb86875a700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8600cd630 con 0x7fb860072130 2023-05-20T01:17:10.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.244+0000 7fb86875a700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8600cca60 con 0x7fb86015cec0 2023-05-20T01:17:10.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.245+0000 7fb865cf5700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fb86015d290 0x7fb8600cf2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:42188/0 (socket says 172.21.15.16:42188) 2023-05-20T01:17:10.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.245+0000 7fb865cf5700 1 -- 172.21.15.16:0/1032733415 learned_addr learned my addr 172.21.15.16:0/1032733415 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:17:10.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.245+0000 7fb866cf7700 1 --2- 172.21.15.16:0/1032733415 >> v2:172.21.15.132:3300/0 conn(0x7fb860072130 0x7fb8600cf9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:10.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.245+0000 7fb8664f6700 1 --2- 172.21.15.16:0/1032733415 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb86015cec0 0x7fb860071bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:10.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.245+0000 7fb8654f4700 1 -- 172.21.15.16:0/1032733415 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1317446582 0 0) 0x7fb860072590 con 0x7fb86015d290 2023-05-20T01:17:10.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.245+0000 7fb8654f4700 1 -- 172.21.15.16:0/1032733415 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb84c0035b0 con 0x7fb86015d290 2023-05-20T01:17:10.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.245+0000 7fb866cf7700 1 -- 172.21.15.16:0/1032733415 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb86015cec0 msgr2=0x7fb860071bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:10.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.245+0000 7fb866cf7700 1 --2- 172.21.15.16:0/1032733415 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb86015cec0 0x7fb860071bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:10.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.245+0000 7fb866cf7700 1 -- 172.21.15.16:0/1032733415 >> v1:172.21.15.16:6789/0 conn(0x7fb86015d290 legacy=0x7fb8600cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:10.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.245+0000 7fb866cf7700 1 -- 172.21.15.16:0/1032733415 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8600ccd00 con 0x7fb860072130 2023-05-20T01:17:10.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.245+0000 7fb866cf7700 1 --2- 172.21.15.16:0/1032733415 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb860072130 0x7fb8600cf9d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb858009d90 tx=0x7fb85800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b05f20cb294a1dc8 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:10.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.245+0000 7fb8654f4700 1 -- 172.21.15.16:0/1032733415 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb85800e070 con 0x7fb860072130 2023-05-20T01:17:10.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.246+0000 7fb8654f4700 1 -- 172.21.15.16:0/1032733415 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb85800bb20 con 0x7fb860072130 2023-05-20T01:17:10.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.246+0000 7fb8654f4700 1 -- 172.21.15.16:0/1032733415 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb858012430 con 0x7fb860072130 2023-05-20T01:17:10.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.246+0000 7fb86875a700 1 -- 172.21.15.16:0/1032733415 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb860072130 msgr2=0x7fb8600cf9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:10.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.246+0000 7fb86875a700 1 --2- 172.21.15.16:0/1032733415 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb860072130 0x7fb8600cf9d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb858009d90 tx=0x7fb85800b330 comp rx=0 tx=0).stop 2023-05-20T01:17:10.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.246+0000 7fb86875a700 1 -- 172.21.15.16:0/1032733415 shutdown_connections 2023-05-20T01:17:10.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.246+0000 7fb86875a700 1 --2- 172.21.15.16:0/1032733415 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb860072130 0x7fb8600cf9d0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:10.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.246+0000 7fb86875a700 1 --2- 172.21.15.16:0/1032733415 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb86015cec0 0x7fb860071bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:10.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.246+0000 7fb86875a700 1 -- 172.21.15.16:0/1032733415 >> 172.21.15.16:0/1032733415 conn(0x7fb8600cffd0 msgr2=0x7fb8600d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:10.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.246+0000 7fb86875a700 1 -- 172.21.15.16:0/1032733415 shutdown_connections 2023-05-20T01:17:10.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.246+0000 7fb86875a700 1 -- 172.21.15.16:0/1032733415 wait complete. 2023-05-20T01:17:10.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.247+0000 7fb86875a700 1 Processor -- start 2023-05-20T01:17:10.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.247+0000 7fb86875a700 1 -- start start 2023-05-20T01:17:10.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.248+0000 7fb86875a700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb860072130 0x7fb860157a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:10.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.248+0000 7fb86875a700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb86015cec0 0x7fb8600c9b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:10.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.248+0000 7fb86875a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb86015d290 0x7fb8600c7310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:10.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.248+0000 7fb86875a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb860157fb0 con 0x7fb86015d290 2023-05-20T01:17:10.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.248+0000 7fb86875a700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8600ca070 con 0x7fb86015cec0 2023-05-20T01:17:10.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.248+0000 7fb86875a700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8601581c0 con 0x7fb860072130 2023-05-20T01:17:10.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.248+0000 7fb866cf7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb86015d290 0x7fb8600c7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:10.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.248+0000 7fb866cf7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb86015d290 0x7fb8600c7310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58902/0 (socket says 172.21.15.16:58902) 2023-05-20T01:17:10.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.248+0000 7fb866cf7700 1 -- 172.21.15.16:0/963301319 learned_addr learned my addr 172.21.15.16:0/963301319 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:10.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.248+0000 7fb8664f6700 1 --2- 172.21.15.16:0/963301319 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb860072130 0x7fb860157a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:10.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.248+0000 7fb865cf5700 1 --2- 172.21.15.16:0/963301319 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb86015cec0 0x7fb8600c9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:10.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.249+0000 7fb866cf7700 1 -- 172.21.15.16:0/963301319 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb860072130 msgr2=0x7fb860157a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:10.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.249+0000 7fb866cf7700 1 --2- 172.21.15.16:0/963301319 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb860072130 0x7fb860157a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:10.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.249+0000 7fb866cf7700 1 -- 172.21.15.16:0/963301319 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb86015cec0 msgr2=0x7fb8600c9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:10.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.249+0000 7fb866cf7700 1 --2- 172.21.15.16:0/963301319 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb86015cec0 0x7fb8600c9b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:10.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.249+0000 7fb866cf7700 1 -- 172.21.15.16:0/963301319 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb858009a70 con 0x7fb86015d290 2023-05-20T01:17:10.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.249+0000 7fb8664f6700 1 --2- 172.21.15.16:0/963301319 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb860072130 0x7fb860157a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:17:10.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.249+0000 7fb866cf7700 1 --2- 172.21.15.16:0/963301319 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb86015d290 0x7fb8600c7310 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb858014040 tx=0x7fb858004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:10.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.249+0000 7fb856ffd700 1 -- 172.21.15.16:0/963301319 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8580129f0 con 0x7fb86015d290 2023-05-20T01:17:10.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.249+0000 7fb856ffd700 1 -- 172.21.15.16:0/963301319 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb858020070 con 0x7fb86015d290 2023-05-20T01:17:10.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.249+0000 7fb86875a700 1 -- 172.21.15.16:0/963301319 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8600cd2c0 con 0x7fb86015d290 2023-05-20T01:17:10.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.249+0000 7fb856ffd700 1 -- 172.21.15.16:0/963301319 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb858012db0 con 0x7fb86015d290 2023-05-20T01:17:10.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.249+0000 7fb86875a700 1 -- 172.21.15.16:0/963301319 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8600ccfe0 con 0x7fb86015d290 2023-05-20T01:17:10.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.250+0000 7fb86875a700 1 -- 172.21.15.16:0/963301319 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb860160890 con 0x7fb86015d290 2023-05-20T01:17:10.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.250+0000 7fb856ffd700 1 -- 172.21.15.16:0/963301319 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb858004040 con 0x7fb86015d290 2023-05-20T01:17:10.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.251+0000 7fb856ffd700 1 --2- 172.21.15.16:0/963301319 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb84406ace0 0x7fb84406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:10.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.251+0000 7fb856ffd700 1 -- 172.21.15.16:0/963301319 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fb858022070 con 0x7fb86015d290 2023-05-20T01:17:10.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.251+0000 7fb8664f6700 1 --2- 172.21.15.16:0/963301319 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb84406ace0 0x7fb84406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:10.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.251+0000 7fb8664f6700 1 --2- 172.21.15.16:0/963301319 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb84406ace0 0x7fb84406d190 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb850009c10 tx=0x7fb850008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:10.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.255+0000 7fb856ffd700 1 -- 172.21.15.16:0/963301319 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fb858056380 con 0x7fb86015d290 2023-05-20T01:17:10.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.456+0000 7fb86875a700 1 -- 172.21.15.16:0/963301319 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fb8600747b0 con 0x7fb86015d290 2023-05-20T01:17:10.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.456+0000 7fb856ffd700 1 -- 172.21.15.16:0/963301319 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7fb8580599a0 con 0x7fb86015d290 2023-05-20T01:17:10.460 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 8 2023-05-20T01:17:10.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.458+0000 7fb86875a700 1 -- 172.21.15.16:0/963301319 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb84406ace0 msgr2=0x7fb84406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:10.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.458+0000 7fb86875a700 1 --2- 172.21.15.16:0/963301319 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb84406ace0 0x7fb84406d190 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb850009c10 tx=0x7fb850008040 comp rx=0 tx=0).stop 2023-05-20T01:17:10.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.458+0000 7fb86875a700 1 -- 172.21.15.16:0/963301319 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb86015d290 msgr2=0x7fb8600c7310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:10.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.458+0000 7fb86875a700 1 --2- 172.21.15.16:0/963301319 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb86015d290 0x7fb8600c7310 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb858014040 tx=0x7fb858004ad0 comp rx=0 tx=0).stop 2023-05-20T01:17:10.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.458+0000 7fb86875a700 1 -- 172.21.15.16:0/963301319 shutdown_connections 2023-05-20T01:17:10.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.458+0000 7fb86875a700 1 --2- 172.21.15.16:0/963301319 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb86015d290 0x7fb8600c7310 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:10.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.458+0000 7fb86875a700 1 --2- 172.21.15.16:0/963301319 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb84406ace0 0x7fb84406d190 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:10.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.458+0000 7fb86875a700 1 --2- 172.21.15.16:0/963301319 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb860072130 0x7fb860157a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:10.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.458+0000 7fb86875a700 1 --2- 172.21.15.16:0/963301319 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb86015cec0 0x7fb8600c9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:10.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.458+0000 7fb86875a700 1 -- 172.21.15.16:0/963301319 >> 172.21.15.16:0/963301319 conn(0x7fb8600cffd0 msgr2=0x7fb86015ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:10.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.458+0000 7fb86875a700 1 -- 172.21.15.16:0/963301319 shutdown_connections 2023-05-20T01:17:10.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.458+0000 7fb86875a700 1 -- 172.21.15.16:0/963301319 wait complete. 2023-05-20T01:17:10.473 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-05-20T01:17:10.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.600+0000 7f87bc221700 1 Processor -- start 2023-05-20T01:17:10.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.600+0000 7f87bc221700 1 -- start start 2023-05-20T01:17:10.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.601+0000 7f87bc221700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87b4071880 0x7f87b4071c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:10.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.601+0000 7f87bc221700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f87b40cd620 0x7f87b40cb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:10.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.601+0000 7f87bc221700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f87b40cbde0 0x7f87b406d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:10.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.601+0000 7f87bc221700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87b40cd200 con 0x7f87b40cd620 2023-05-20T01:17:10.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.601+0000 7f87bc221700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87b40d0e10 con 0x7f87b40cbde0 2023-05-20T01:17:10.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.601+0000 7f87bc221700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87b4072190 con 0x7f87b4071880 2023-05-20T01:17:10.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.601+0000 7f87b97bc700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f87b40cd620 0x7f87b40cb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:10.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.601+0000 7f87b97bc700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f87b40cd620 0x7f87b40cb8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58908/0 (socket says 172.21.15.16:58908) 2023-05-20T01:17:10.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.601+0000 7f87b97bc700 1 -- 172.21.15.16:0/1625229797 learned_addr learned my addr 172.21.15.16:0/1625229797 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:10.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.601+0000 7f87ba7be700 1 --2- 172.21.15.16:0/1625229797 >> v2:172.21.15.132:3300/0 conn(0x7f87b40cbde0 0x7f87b406d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:10.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.601+0000 7f87b9fbd700 1 --2- 172.21.15.16:0/1625229797 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87b4071880 0x7f87b4071c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:10.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.601+0000 7f87b97bc700 1 -- 172.21.15.16:0/1625229797 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87b4071880 msgr2=0x7f87b4071c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:10.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.601+0000 7f87b97bc700 1 --2- 172.21.15.16:0/1625229797 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87b4071880 0x7f87b4071c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:10.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.601+0000 7f87b97bc700 1 -- 172.21.15.16:0/1625229797 >> v2:172.21.15.132:3300/0 conn(0x7f87b40cbde0 msgr2=0x7f87b406d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:10.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.601+0000 7f87b97bc700 1 --2- 172.21.15.16:0/1625229797 >> v2:172.21.15.132:3300/0 conn(0x7f87b40cbde0 0x7f87b406d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:10.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.601+0000 7f87b97bc700 1 -- 172.21.15.16:0/1625229797 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87b40cc580 con 0x7f87b40cd620 2023-05-20T01:17:10.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.601+0000 7f87ba7be700 1 --2- 172.21.15.16:0/1625229797 >> v2:172.21.15.132:3300/0 conn(0x7f87b40cbde0 0x7f87b406d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:17:10.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.602+0000 7f87b97bc700 1 --2- 172.21.15.16:0/1625229797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87b40cd620 0x7f87b40cb8a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f87b406c600 tx=0x7f87b0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dced450ec6f01671 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:10.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.602+0000 7f87b8fbb700 1 -- 172.21.15.16:0/1625229797 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b000b870 con 0x7f87b40cd620 2023-05-20T01:17:10.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.602+0000 7f87b8fbb700 1 -- 172.21.15.16:0/1625229797 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87b000b9d0 con 0x7f87b40cd620 2023-05-20T01:17:10.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.602+0000 7f87b8fbb700 1 -- 172.21.15.16:0/1625229797 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b000bd60 con 0x7f87b40cd620 2023-05-20T01:17:10.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.602+0000 7f87bc221700 1 -- 172.21.15.16:0/1625229797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87b40cd620 msgr2=0x7f87b40cb8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:10.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.602+0000 7f87bc221700 1 --2- 172.21.15.16:0/1625229797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87b40cd620 0x7f87b40cb8a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f87b406c600 tx=0x7f87b0009bd0 comp rx=0 tx=0).stop 2023-05-20T01:17:10.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.602+0000 7f87bc221700 1 -- 172.21.15.16:0/1625229797 shutdown_connections 2023-05-20T01:17:10.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.602+0000 7f87bc221700 1 --2- 172.21.15.16:0/1625229797 >> v2:172.21.15.132:3300/0 conn(0x7f87b40cbde0 0x7f87b406d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:10.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.602+0000 7f87bc221700 1 --2- 172.21.15.16:0/1625229797 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87b4071880 0x7f87b4071c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:10.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.602+0000 7f87bc221700 1 --2- 172.21.15.16:0/1625229797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87b40cd620 0x7f87b40cb8a0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:10.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.602+0000 7f87bc221700 1 -- 172.21.15.16:0/1625229797 >> 172.21.15.16:0/1625229797 conn(0x7f87b40cffd0 msgr2=0x7f87b40d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:10.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.602+0000 7f87bc221700 1 -- 172.21.15.16:0/1625229797 shutdown_connections 2023-05-20T01:17:10.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.602+0000 7f87bc221700 1 -- 172.21.15.16:0/1625229797 wait complete. 2023-05-20T01:17:10.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.604+0000 7f87bc221700 1 Processor -- start 2023-05-20T01:17:10.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.604+0000 7f87bc221700 1 -- start start 2023-05-20T01:17:10.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.604+0000 7f87bc221700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f87b4071880 0x7f87b40aefe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:10.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.604+0000 7f87bc221700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87b40cbde0 0x7f87b40ac8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:10.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.604+0000 7f87bc221700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87b40cd620 0x7f87b40acde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:10.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.604+0000 7f87bc221700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87b40cf630 con 0x7f87b40cd620 2023-05-20T01:17:10.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.604+0000 7f87bc221700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87b40af520 con 0x7f87b4071880 2023-05-20T01:17:10.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.604+0000 7f87bc221700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87b40ce9b0 con 0x7f87b40cbde0 2023-05-20T01:17:10.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.604+0000 7f87ba7be700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87b40cd620 0x7f87b40acde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:10.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.605+0000 7f87ba7be700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87b40cd620 0x7f87b40acde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58916/0 (socket says 172.21.15.16:58916) 2023-05-20T01:17:10.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.605+0000 7f87ba7be700 1 -- 172.21.15.16:0/2343967456 learned_addr learned my addr 172.21.15.16:0/2343967456 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:10.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.605+0000 7f87b9fbd700 1 --2- 172.21.15.16:0/2343967456 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f87b4071880 0x7f87b40aefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:10.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.605+0000 7f87b97bc700 1 --2- 172.21.15.16:0/2343967456 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87b40cbde0 0x7f87b40ac8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:10.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.605+0000 7f87ba7be700 1 -- 172.21.15.16:0/2343967456 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87b40cbde0 msgr2=0x7f87b40ac8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:10.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.605+0000 7f87ba7be700 1 --2- 172.21.15.16:0/2343967456 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87b40cbde0 0x7f87b40ac8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:10.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.605+0000 7f87ba7be700 1 -- 172.21.15.16:0/2343967456 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f87b4071880 msgr2=0x7f87b40aefe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:10.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.605+0000 7f87ba7be700 1 --2- 172.21.15.16:0/2343967456 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f87b4071880 0x7f87b40aefe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:10.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.605+0000 7f87ba7be700 1 -- 172.21.15.16:0/2343967456 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87b0009550 con 0x7f87b40cd620 2023-05-20T01:17:10.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.605+0000 7f87ba7be700 1 --2- 172.21.15.16:0/2343967456 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87b40cd620 0x7f87b40acde0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f87ac00cd50 tx=0x7f87ac00a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:10.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.605+0000 7f87aa7fc700 1 -- 172.21.15.16:0/2343967456 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87ac00cea0 con 0x7f87b40cd620 2023-05-20T01:17:10.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.605+0000 7f87aa7fc700 1 -- 172.21.15.16:0/2343967456 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87ac00f480 con 0x7f87b40cd620 2023-05-20T01:17:10.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.605+0000 7f87bc221700 1 -- 172.21.15.16:0/2343967456 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87b40ccb40 con 0x7f87b40cd620 2023-05-20T01:17:10.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.605+0000 7f87aa7fc700 1 -- 172.21.15.16:0/2343967456 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87ac00f830 con 0x7f87b40cd620 2023-05-20T01:17:10.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.605+0000 7f87bc221700 1 -- 172.21.15.16:0/2343967456 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87b40cc860 con 0x7f87b40cd620 2023-05-20T01:17:10.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.606+0000 7f87bc221700 1 -- 172.21.15.16:0/2343967456 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87b40cce20 con 0x7f87b40cd620 2023-05-20T01:17:10.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.607+0000 7f87aa7fc700 1 -- 172.21.15.16:0/2343967456 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f87ac00b040 con 0x7f87b40cd620 2023-05-20T01:17:10.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.607+0000 7f87aa7fc700 1 --2- 172.21.15.16:0/2343967456 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f879806ace0 0x7f879806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:10.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.607+0000 7f87aa7fc700 1 -- 172.21.15.16:0/2343967456 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f87ac089dd0 con 0x7f87b40cd620 2023-05-20T01:17:10.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.607+0000 7f87b9fbd700 1 --2- 172.21.15.16:0/2343967456 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f879806ace0 0x7f879806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:10.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.608+0000 7f87b9fbd700 1 --2- 172.21.15.16:0/2343967456 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f879806ace0 0x7f879806d190 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f87a4006fd0 tx=0x7f87a4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:10.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.612+0000 7f87aa7fc700 1 -- 172.21.15.16:0/2343967456 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f87ac054c20 con 0x7f87b40cd620 2023-05-20T01:17:10.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.809+0000 7f87bc221700 1 -- 172.21.15.16:0/2343967456 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f87b40cf770 con 0x7f87b40cd620 2023-05-20T01:17:10.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.809+0000 7f87aa7fc700 1 -- 172.21.15.16:0/2343967456 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7f87b40cf770 con 0x7f87b40cd620 2023-05-20T01:17:10.812 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-05-20T01:17:10.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.811+0000 7f87bc221700 1 -- 172.21.15.16:0/2343967456 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f879806ace0 msgr2=0x7f879806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:10.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.811+0000 7f87bc221700 1 --2- 172.21.15.16:0/2343967456 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f879806ace0 0x7f879806d190 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f87a4006fd0 tx=0x7f87a4008040 comp rx=0 tx=0).stop 2023-05-20T01:17:10.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.811+0000 7f87bc221700 1 -- 172.21.15.16:0/2343967456 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87b40cd620 msgr2=0x7f87b40acde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:10.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.811+0000 7f87bc221700 1 --2- 172.21.15.16:0/2343967456 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87b40cd620 0x7f87b40acde0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f87ac00cd50 tx=0x7f87ac00a560 comp rx=0 tx=0).stop 2023-05-20T01:17:10.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.811+0000 7f87bc221700 1 -- 172.21.15.16:0/2343967456 shutdown_connections 2023-05-20T01:17:10.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.811+0000 7f87bc221700 1 --2- 172.21.15.16:0/2343967456 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87b40cd620 0x7f87b40acde0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:10.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.811+0000 7f87bc221700 1 --2- 172.21.15.16:0/2343967456 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f87b4071880 0x7f87b40aefe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:10.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.811+0000 7f87bc221700 1 --2- 172.21.15.16:0/2343967456 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f879806ace0 0x7f879806d190 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:10.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.811+0000 7f87bc221700 1 --2- 172.21.15.16:0/2343967456 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87b40cbde0 0x7f87b40ac8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:10.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.811+0000 7f87bc221700 1 -- 172.21.15.16:0/2343967456 >> 172.21.15.16:0/2343967456 conn(0x7f87b40cffd0 msgr2=0x7f87b406dde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:10.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.811+0000 7f87bc221700 1 -- 172.21.15.16:0/2343967456 shutdown_connections 2023-05-20T01:17:10.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.812+0000 7f87bc221700 1 -- 172.21.15.16:0/2343967456 wait complete. 2023-05-20T01:17:10.827 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-05-20T01:17:10.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.954+0000 7f3324ff0700 1 Processor -- start 2023-05-20T01:17:10.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.954+0000 7f3324ff0700 1 -- start start 2023-05-20T01:17:10.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.955+0000 7f3324ff0700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f332015ab80 0x7f33200cda80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:10.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.955+0000 7f3324ff0700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f33200cf390 0x7f33200cc110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:10.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.955+0000 7f3324ff0700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33200ce4d0 con 0x7f33200cdfc0 2023-05-20T01:17:10.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.955+0000 7f3324ff0700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33200cb700 con 0x7f332015ab80 2023-05-20T01:17:10.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.955+0000 7f3324ff0700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33200ce7b0 con 0x7f33200cf390 2023-05-20T01:17:10.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.955+0000 7f331e59c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f332015ab80 0x7f33200cda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:10.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.955+0000 7f331e59c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f332015ab80 0x7f33200cda80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58920/0 (socket says 172.21.15.16:58920) 2023-05-20T01:17:10.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.955+0000 7f331e59c700 1 -- 172.21.15.16:0/4235911340 learned_addr learned my addr 172.21.15.16:0/4235911340 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:10.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.955+0000 7f331ed9d700 1 --2- 172.21.15.16:0/4235911340 >> v2:172.21.15.132:3300/0 conn(0x7f33200cf390 0x7f33200cc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:10.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.955+0000 7f331e59c700 1 -- 172.21.15.16:0/4235911340 >> v2:172.21.15.132:3300/0 conn(0x7f33200cf390 msgr2=0x7f33200cc110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:10.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.955+0000 7f331e59c700 1 --2- 172.21.15.16:0/4235911340 >> v2:172.21.15.132:3300/0 conn(0x7f33200cf390 0x7f33200cc110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:10.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.955+0000 7f331e59c700 1 -- 172.21.15.16:0/4235911340 >> v1:172.21.15.132:6789/0 conn(0x7f33200cdfc0 legacy=0x7f33200cec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:10.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.955+0000 7f331e59c700 1 -- 172.21.15.16:0/4235911340 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33200cb910 con 0x7f332015ab80 2023-05-20T01:17:10.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.955+0000 7f331e59c700 1 --2- 172.21.15.16:0/4235911340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f332015ab80 0x7f33200cda80 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f33200b2b50 tx=0x7f3314009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4206f8d2a79324e server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:10.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.955+0000 7f331d59a700 1 -- 172.21.15.16:0/4235911340 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f331400b870 con 0x7f332015ab80 2023-05-20T01:17:10.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.956+0000 7f331d59a700 1 -- 172.21.15.16:0/4235911340 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f331400b9d0 con 0x7f332015ab80 2023-05-20T01:17:10.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.956+0000 7f331d59a700 1 -- 172.21.15.16:0/4235911340 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f331400bd60 con 0x7f332015ab80 2023-05-20T01:17:10.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.956+0000 7f3324ff0700 1 -- 172.21.15.16:0/4235911340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f332015ab80 msgr2=0x7f33200cda80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:10.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.956+0000 7f3324ff0700 1 --2- 172.21.15.16:0/4235911340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f332015ab80 0x7f33200cda80 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f33200b2b50 tx=0x7f3314009bd0 comp rx=0 tx=0).stop 2023-05-20T01:17:10.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.956+0000 7f3324ff0700 1 -- 172.21.15.16:0/4235911340 shutdown_connections 2023-05-20T01:17:10.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.956+0000 7f3324ff0700 1 --2- 172.21.15.16:0/4235911340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f332015ab80 0x7f33200cda80 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:10.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.956+0000 7f3324ff0700 1 --2- 172.21.15.16:0/4235911340 >> v2:172.21.15.132:3300/0 conn(0x7f33200cf390 0x7f33200cc110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:10.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.956+0000 7f3324ff0700 1 -- 172.21.15.16:0/4235911340 >> 172.21.15.16:0/4235911340 conn(0x7f332006ae30 msgr2=0x7f332009a400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:10.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.956+0000 7f3324ff0700 1 -- 172.21.15.16:0/4235911340 shutdown_connections 2023-05-20T01:17:10.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.956+0000 7f3324ff0700 1 -- 172.21.15.16:0/4235911340 wait complete. 2023-05-20T01:17:10.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.958+0000 7f3324ff0700 1 Processor -- start 2023-05-20T01:17:10.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.958+0000 7f3324ff0700 1 -- start start 2023-05-20T01:17:10.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.958+0000 7f3324ff0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f33200cdfc0 0x7f33201545f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:10.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.958+0000 7f3324ff0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33200cf390 0x7f33200c8180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:10.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.958+0000 7f3324ff0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f332015ab80 0x7f3320159e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:10.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.958+0000 7f3324ff0700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33200b50a0 con 0x7f33200cf390 2023-05-20T01:17:10.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.958+0000 7f3324ff0700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33200c8ca0 con 0x7f332015ab80 2023-05-20T01:17:10.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.958+0000 7f3324ff0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33200c8f80 con 0x7f33200cdfc0 2023-05-20T01:17:10.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.958+0000 7f331dd9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33200cf390 0x7f33200c8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:10.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.959+0000 7f331dd9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33200cf390 0x7f33200c8180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58928/0 (socket says 172.21.15.16:58928) 2023-05-20T01:17:10.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.959+0000 7f331dd9b700 1 -- 172.21.15.16:0/3256047161 learned_addr learned my addr 172.21.15.16:0/3256047161 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:10.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.959+0000 7f331ed9d700 1 --2- 172.21.15.16:0/3256047161 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f332015ab80 0x7f3320159e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:10.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.959+0000 7f331e59c700 1 --2- 172.21.15.16:0/3256047161 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f33200cdfc0 0x7f33201545f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:10.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.959+0000 7f331dd9b700 1 -- 172.21.15.16:0/3256047161 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f33200cdfc0 msgr2=0x7f33201545f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:10.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.959+0000 7f331dd9b700 1 --2- 172.21.15.16:0/3256047161 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f33200cdfc0 0x7f33201545f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:10.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.959+0000 7f331dd9b700 1 -- 172.21.15.16:0/3256047161 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f332015ab80 msgr2=0x7f3320159e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:10.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.959+0000 7f331dd9b700 1 --2- 172.21.15.16:0/3256047161 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f332015ab80 0x7f3320159e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:10.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.959+0000 7f331dd9b700 1 -- 172.21.15.16:0/3256047161 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3314009550 con 0x7f33200cf390 2023-05-20T01:17:10.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.959+0000 7f331e59c700 1 --2- 172.21.15.16:0/3256047161 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f33200cdfc0 0x7f33201545f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:17:10.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.959+0000 7f331dd9b700 1 --2- 172.21.15.16:0/3256047161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33200cf390 0x7f33200c8180 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f331000c490 tx=0x7f331000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:10.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.959+0000 7f330effd700 1 -- 172.21.15.16:0/3256047161 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33100052d0 con 0x7f33200cf390 2023-05-20T01:17:10.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.959+0000 7f330effd700 1 -- 172.21.15.16:0/3256047161 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3310005450 con 0x7f33200cf390 2023-05-20T01:17:10.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.959+0000 7f3324ff0700 1 -- 172.21.15.16:0/3256047161 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33200d0e30 con 0x7f33200cf390 2023-05-20T01:17:10.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.959+0000 7f330effd700 1 -- 172.21.15.16:0/3256047161 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3310006aa0 con 0x7f33200cf390 2023-05-20T01:17:10.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.959+0000 7f3324ff0700 1 -- 172.21.15.16:0/3256047161 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33200cbbf0 con 0x7f33200cf390 2023-05-20T01:17:10.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.960+0000 7f3324ff0700 1 -- 172.21.15.16:0/3256047161 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33200d0fc0 con 0x7f33200cf390 2023-05-20T01:17:10.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.961+0000 7f330effd700 1 -- 172.21.15.16:0/3256047161 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3310003680 con 0x7f33200cf390 2023-05-20T01:17:10.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.961+0000 7f330effd700 1 --2- 172.21.15.16:0/3256047161 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f330006ace0 0x7f330006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:10.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.961+0000 7f330effd700 1 -- 172.21.15.16:0/3256047161 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f3310088a70 con 0x7f33200cf390 2023-05-20T01:17:10.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.961+0000 7f331e59c700 1 --2- 172.21.15.16:0/3256047161 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f330006ace0 0x7f330006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:10.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.961+0000 7f331e59c700 1 --2- 172.21.15.16:0/3256047161 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f330006ace0 0x7f330006d190 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f33200aec70 tx=0x7f3314008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:10.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:10.965+0000 7f330effd700 1 -- 172.21.15.16:0/3256047161 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f3310053830 con 0x7f33200cf390 2023-05-20T01:17:11.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.158+0000 7f3324ff0700 1 -- 172.21.15.16:0/3256047161 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f33200747b0 con 0x7f33200cf390 2023-05-20T01:17:11.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.159+0000 7f330effd700 1 -- 172.21.15.16:0/3256047161 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3310056e50 con 0x7f33200cf390 2023-05-20T01:17:11.163 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-05-20T01:17:11.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.160+0000 7f3324ff0700 1 -- 172.21.15.16:0/3256047161 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f330006ace0 msgr2=0x7f330006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:11.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.160+0000 7f3324ff0700 1 --2- 172.21.15.16:0/3256047161 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f330006ace0 0x7f330006d190 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f33200aec70 tx=0x7f3314008030 comp rx=0 tx=0).stop 2023-05-20T01:17:11.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.160+0000 7f3324ff0700 1 -- 172.21.15.16:0/3256047161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33200cf390 msgr2=0x7f33200c8180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:11.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.160+0000 7f3324ff0700 1 --2- 172.21.15.16:0/3256047161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33200cf390 0x7f33200c8180 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f331000c490 tx=0x7f331000c850 comp rx=0 tx=0).stop 2023-05-20T01:17:11.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.161+0000 7f3324ff0700 1 -- 172.21.15.16:0/3256047161 shutdown_connections 2023-05-20T01:17:11.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.161+0000 7f3324ff0700 1 --2- 172.21.15.16:0/3256047161 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f332015ab80 0x7f3320159e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:11.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.161+0000 7f3324ff0700 1 --2- 172.21.15.16:0/3256047161 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f330006ace0 0x7f330006d190 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:11.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.161+0000 7f3324ff0700 1 --2- 172.21.15.16:0/3256047161 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f33200cdfc0 0x7f33201545f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:11.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.161+0000 7f3324ff0700 1 --2- 172.21.15.16:0/3256047161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33200cf390 0x7f33200c8180 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:11.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.161+0000 7f3324ff0700 1 -- 172.21.15.16:0/3256047161 >> 172.21.15.16:0/3256047161 conn(0x7f332006ae30 msgr2=0x7f33200aeee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:11.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.161+0000 7f3324ff0700 1 -- 172.21.15.16:0/3256047161 shutdown_connections 2023-05-20T01:17:11.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.161+0000 7f3324ff0700 1 -- 172.21.15.16:0/3256047161 wait complete. 2023-05-20T01:17:11.176 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-05-20T01:17:11.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.301+0000 7f9a2194f700 1 Processor -- start 2023-05-20T01:17:11.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.301+0000 7f9a2194f700 1 -- start start 2023-05-20T01:17:11.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.301+0000 7f9a2194f700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9a1c0cd070 0x7f9a1c06e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:11.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.301+0000 7f9a2194f700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a1c072430 con 0x7f9a1c15cf40 2023-05-20T01:17:11.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.301+0000 7f9a2194f700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a1c071820 con 0x7f9a1c0710d0 2023-05-20T01:17:11.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.301+0000 7f9a2194f700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a1c071b00 con 0x7f9a1c0cd070 2023-05-20T01:17:11.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.301+0000 7f9a1a7fc700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9a1c0cd070 0x7f9a1c06e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:11.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.302+0000 7f9a1a7fc700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9a1c0cd070 0x7f9a1c06e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58936/0 (socket says 172.21.15.16:58936) 2023-05-20T01:17:11.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.302+0000 7f9a1affd700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f9a1c15cf40 0x7f9a1c0cc960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:42230/0 (socket says 172.21.15.16:42230) 2023-05-20T01:17:11.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.302+0000 7f9a1a7fc700 1 -- 172.21.15.16:0/1923019630 learned_addr learned my addr 172.21.15.16:0/1923019630 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:11.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.302+0000 7f9a1a7fc700 1 -- 172.21.15.16:0/1923019630 >> v1:172.21.15.132:6789/0 conn(0x7f9a1c0710d0 legacy=0x7f9a1c0cabb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:11.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.302+0000 7f9a1a7fc700 1 -- 172.21.15.16:0/1923019630 >> v1:172.21.15.16:6789/0 conn(0x7f9a1c15cf40 legacy=0x7f9a1c0cc960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:11.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.302+0000 7f9a19ffb700 1 -- 172.21.15.16:0/1923019630 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3354958957 0 0) 0x7f9a1c072430 con 0x7f9a1c15cf40 2023-05-20T01:17:11.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.302+0000 7f9a1a7fc700 1 -- 172.21.15.16:0/1923019630 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a1c071de0 con 0x7f9a1c0cd070 2023-05-20T01:17:11.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.302+0000 7f9a1a7fc700 1 --2- 172.21.15.16:0/1923019630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a1c0cd070 0x7f9a1c06e020 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f9a040098f0 tx=0x7f9a04009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d78bf432ec8ef06 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:11.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.302+0000 7f9a19ffb700 1 -- 172.21.15.16:0/1923019630 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a0400b8a0 con 0x7f9a1c0cd070 2023-05-20T01:17:11.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.302+0000 7f9a19ffb700 1 -- 172.21.15.16:0/1923019630 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a0400ba00 con 0x7f9a1c0cd070 2023-05-20T01:17:11.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.302+0000 7f9a19ffb700 1 -- 172.21.15.16:0/1923019630 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a0400bd90 con 0x7f9a1c0cd070 2023-05-20T01:17:11.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.303+0000 7f9a2194f700 1 -- 172.21.15.16:0/1923019630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a1c0cd070 msgr2=0x7f9a1c06e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:11.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.303+0000 7f9a2194f700 1 --2- 172.21.15.16:0/1923019630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a1c0cd070 0x7f9a1c06e020 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f9a040098f0 tx=0x7f9a04009c00 comp rx=0 tx=0).stop 2023-05-20T01:17:11.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.303+0000 7f9a2194f700 1 -- 172.21.15.16:0/1923019630 shutdown_connections 2023-05-20T01:17:11.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.303+0000 7f9a2194f700 1 --2- 172.21.15.16:0/1923019630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a1c0cd070 0x7f9a1c06e020 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:11.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.303+0000 7f9a2194f700 1 -- 172.21.15.16:0/1923019630 >> 172.21.15.16:0/1923019630 conn(0x7f9a1c06c260 msgr2=0x7f9a1c0cffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:11.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.303+0000 7f9a2194f700 1 -- 172.21.15.16:0/1923019630 shutdown_connections 2023-05-20T01:17:11.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.303+0000 7f9a2194f700 1 -- 172.21.15.16:0/1923019630 wait complete. 2023-05-20T01:17:11.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.304+0000 7f9a2194f700 1 Processor -- start 2023-05-20T01:17:11.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.304+0000 7f9a2194f700 1 -- start start 2023-05-20T01:17:11.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.305+0000 7f9a2194f700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9a1c0710d0 0x7f9a1c1f5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:11.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.305+0000 7f9a2194f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a1c0cd070 0x7f9a1c158960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:11.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.305+0000 7f9a2194f700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9a1c15cf40 0x7f9a1c1f2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:11.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.305+0000 7f9a2194f700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a1c1f5f60 con 0x7f9a1c0cd070 2023-05-20T01:17:11.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.305+0000 7f9a2194f700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a1c1f6240 con 0x7f9a1c0710d0 2023-05-20T01:17:11.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.305+0000 7f9a2194f700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a1c1f6520 con 0x7f9a1c15cf40 2023-05-20T01:17:11.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.305+0000 7f9a1a7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a1c0cd070 0x7f9a1c158960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:11.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.305+0000 7f9a1a7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a1c0cd070 0x7f9a1c158960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58942/0 (socket says 172.21.15.16:58942) 2023-05-20T01:17:11.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.305+0000 7f9a1a7fc700 1 -- 172.21.15.16:0/1293457194 learned_addr learned my addr 172.21.15.16:0/1293457194 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:11.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.305+0000 7f9a1affd700 1 --2- 172.21.15.16:0/1293457194 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9a1c0710d0 0x7f9a1c1f5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:11.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.305+0000 7f9a1b7fe700 1 --2- 172.21.15.16:0/1293457194 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9a1c15cf40 0x7f9a1c1f2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:11.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.305+0000 7f9a1a7fc700 1 -- 172.21.15.16:0/1293457194 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9a1c15cf40 msgr2=0x7f9a1c1f2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:11.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.305+0000 7f9a1a7fc700 1 --2- 172.21.15.16:0/1293457194 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9a1c15cf40 0x7f9a1c1f2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:11.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.305+0000 7f9a1a7fc700 1 -- 172.21.15.16:0/1293457194 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9a1c0710d0 msgr2=0x7f9a1c1f5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:11.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.305+0000 7f9a1a7fc700 1 --2- 172.21.15.16:0/1293457194 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9a1c0710d0 0x7f9a1c1f5710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:11.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.305+0000 7f9a1a7fc700 1 -- 172.21.15.16:0/1293457194 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a04009550 con 0x7f9a1c0cd070 2023-05-20T01:17:11.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.305+0000 7f9a1a7fc700 1 --2- 172.21.15.16:0/1293457194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a1c0cd070 0x7f9a1c158960 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f9a04013000 tx=0x7f9a04004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:11.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.306+0000 7f9a13fff700 1 -- 172.21.15.16:0/1293457194 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a04011a70 con 0x7f9a1c0cd070 2023-05-20T01:17:11.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.306+0000 7f9a13fff700 1 -- 172.21.15.16:0/1293457194 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a0401f070 con 0x7f9a1c0cd070 2023-05-20T01:17:11.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.306+0000 7f9a2194f700 1 -- 172.21.15.16:0/1293457194 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a1c071510 con 0x7f9a1c0cd070 2023-05-20T01:17:11.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.306+0000 7f9a13fff700 1 -- 172.21.15.16:0/1293457194 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a04011dd0 con 0x7f9a1c0cd070 2023-05-20T01:17:11.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.306+0000 7f9a2194f700 1 -- 172.21.15.16:0/1293457194 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a1c0720c0 con 0x7f9a1c0cd070 2023-05-20T01:17:11.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.307+0000 7f9a2194f700 1 -- 172.21.15.16:0/1293457194 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a1c0cb2c0 con 0x7f9a1c0cd070 2023-05-20T01:17:11.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.307+0000 7f9a13fff700 1 -- 172.21.15.16:0/1293457194 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9a0401a430 con 0x7f9a1c0cd070 2023-05-20T01:17:11.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.307+0000 7f9a13fff700 1 --2- 172.21.15.16:0/1293457194 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f99fc06ace0 0x7f99fc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:11.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.308+0000 7f9a13fff700 1 -- 172.21.15.16:0/1293457194 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f9a04021070 con 0x7f9a1c0cd070 2023-05-20T01:17:11.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.308+0000 7f9a1affd700 1 --2- 172.21.15.16:0/1293457194 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f99fc06ace0 0x7f99fc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:11.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.308+0000 7f9a1affd700 1 --2- 172.21.15.16:0/1293457194 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f99fc06ace0 0x7f99fc06d190 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c000f60 tx=0x7f9a0c007d90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:11.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.312+0000 7f9a13fff700 1 -- 172.21.15.16:0/1293457194 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f9a04056050 con 0x7f9a1c0cd070 2023-05-20T01:17:11.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.510+0000 7f9a2194f700 1 -- 172.21.15.16:0/1293457194 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9a1c0747b0 con 0x7f9a1c0cd070 2023-05-20T01:17:11.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.510+0000 7f9a13fff700 1 -- 172.21.15.16:0/1293457194 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9a04059670 con 0x7f9a1c0cd070 2023-05-20T01:17:11.515 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2023-05-20T01:17:11.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.513+0000 7f9a2194f700 1 -- 172.21.15.16:0/1293457194 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f99fc06ace0 msgr2=0x7f99fc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:11.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.513+0000 7f9a2194f700 1 --2- 172.21.15.16:0/1293457194 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f99fc06ace0 0x7f99fc06d190 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c000f60 tx=0x7f9a0c007d90 comp rx=0 tx=0).stop 2023-05-20T01:17:11.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.513+0000 7f9a2194f700 1 -- 172.21.15.16:0/1293457194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a1c0cd070 msgr2=0x7f9a1c158960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:11.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.513+0000 7f9a2194f700 1 --2- 172.21.15.16:0/1293457194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a1c0cd070 0x7f9a1c158960 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f9a04013000 tx=0x7f9a04004750 comp rx=0 tx=0).stop 2023-05-20T01:17:11.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.513+0000 7f9a2194f700 1 -- 172.21.15.16:0/1293457194 shutdown_connections 2023-05-20T01:17:11.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.513+0000 7f9a2194f700 1 --2- 172.21.15.16:0/1293457194 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f99fc06ace0 0x7f99fc06d190 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:11.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.513+0000 7f9a2194f700 1 --2- 172.21.15.16:0/1293457194 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9a1c15cf40 0x7f9a1c1f2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:11.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.513+0000 7f9a2194f700 1 --2- 172.21.15.16:0/1293457194 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9a1c0710d0 0x7f9a1c1f5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:11.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.513+0000 7f9a2194f700 1 --2- 172.21.15.16:0/1293457194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a1c0cd070 0x7f9a1c158960 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:11.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.513+0000 7f9a2194f700 1 -- 172.21.15.16:0/1293457194 >> 172.21.15.16:0/1293457194 conn(0x7f9a1c06c260 msgr2=0x7f9a1c0ce530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:11.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.513+0000 7f9a2194f700 1 -- 172.21.15.16:0/1293457194 shutdown_connections 2023-05-20T01:17:11.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.513+0000 7f9a2194f700 1 -- 172.21.15.16:0/1293457194 wait complete. 2023-05-20T01:17:11.529 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-05-20T01:17:11.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.653+0000 7f4cb2a08700 1 Processor -- start 2023-05-20T01:17:11.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.653+0000 7f4cb2a08700 1 -- start start 2023-05-20T01:17:11.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.654+0000 7f4cb2a08700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4cac0cd1c0 0x7f4cac0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:11.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.654+0000 7f4cb2a08700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f4cac0cd630 0x7f4cac0715f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:11.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.654+0000 7f4cb2a08700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4cac0ccd30 con 0x7f4cac159b50 2023-05-20T01:17:11.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.654+0000 7f4cb2a08700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cac0ce990 con 0x7f4cac0cd630 2023-05-20T01:17:11.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.654+0000 7f4cb2a08700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cac0cc6e0 con 0x7f4cac0cd1c0 2023-05-20T01:17:11.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.654+0000 7f4cab7fe700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4cac0cd1c0 0x7f4cac0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:11.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.654+0000 7f4cb0fa5700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f4cac0cd630 0x7f4cac0715f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:11.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.654+0000 7f4cabfff700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f4cac159b50 0x7f4cac06e0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.16:51958/0 (socket says 172.21.15.16:51958) 2023-05-20T01:17:11.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.654+0000 7f4cabfff700 1 -- 172.21.15.16:0/41022319 learned_addr learned my addr 172.21.15.16:0/41022319 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:17:11.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.655+0000 7f4caaffd700 1 -- 172.21.15.16:0/41022319 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2359920473 0 0) 0x7f4cac0ccd30 con 0x7f4cac159b50 2023-05-20T01:17:11.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.655+0000 7f4caaffd700 1 -- 172.21.15.16:0/41022319 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c980035b0 con 0x7f4cac159b50 2023-05-20T01:17:11.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.655+0000 7f4cb0fa5700 1 -- 172.21.15.16:0/41022319 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4cac0cd1c0 msgr2=0x7f4cac0710b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:11.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.655+0000 7f4cb0fa5700 1 --2- 172.21.15.16:0/41022319 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4cac0cd1c0 0x7f4cac0710b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:11.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.655+0000 7f4cb0fa5700 1 -- 172.21.15.16:0/41022319 >> v1:172.21.15.132:6789/0 conn(0x7f4cac159b50 legacy=0x7f4cac06e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:11.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.655+0000 7f4cb0fa5700 1 -- 172.21.15.16:0/41022319 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cac0cc9c0 con 0x7f4cac0cd630 2023-05-20T01:17:11.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.655+0000 7f4cab7fe700 1 --2- 172.21.15.16:0/41022319 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4cac0cd1c0 0x7f4cac0710b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-05-20T01:17:11.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.655+0000 7f4cb0fa5700 1 --2- 172.21.15.16:0/41022319 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4cac0cd630 0x7f4cac0715f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4ca4009d90 tx=0x7f4ca400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2e444910514c99a server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:11.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.655+0000 7f4caaffd700 1 -- 172.21.15.16:0/41022319 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ca400e070 con 0x7f4cac0cd630 2023-05-20T01:17:11.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.655+0000 7f4caaffd700 1 -- 172.21.15.16:0/41022319 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ca400bb20 con 0x7f4cac0cd630 2023-05-20T01:17:11.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.655+0000 7f4caaffd700 1 -- 172.21.15.16:0/41022319 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ca4012430 con 0x7f4cac0cd630 2023-05-20T01:17:11.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.656+0000 7f4cb2a08700 1 -- 172.21.15.16:0/41022319 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4cac0cd630 msgr2=0x7f4cac0715f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:11.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.656+0000 7f4cb2a08700 1 --2- 172.21.15.16:0/41022319 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4cac0cd630 0x7f4cac0715f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4ca4009d90 tx=0x7f4ca400b330 comp rx=0 tx=0).stop 2023-05-20T01:17:11.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.656+0000 7f4cb2a08700 1 -- 172.21.15.16:0/41022319 shutdown_connections 2023-05-20T01:17:11.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.656+0000 7f4cb2a08700 1 --2- 172.21.15.16:0/41022319 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4cac0cd630 0x7f4cac0715f0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:11.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.656+0000 7f4cb2a08700 1 --2- 172.21.15.16:0/41022319 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4cac0cd1c0 0x7f4cac0710b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:11.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.656+0000 7f4cb2a08700 1 -- 172.21.15.16:0/41022319 >> 172.21.15.16:0/41022319 conn(0x7f4cac0cffd0 msgr2=0x7f4cac0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:11.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.656+0000 7f4cb2a08700 1 -- 172.21.15.16:0/41022319 shutdown_connections 2023-05-20T01:17:11.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.656+0000 7f4cb2a08700 1 -- 172.21.15.16:0/41022319 wait complete. 2023-05-20T01:17:11.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.657+0000 7f4cb2a08700 1 Processor -- start 2023-05-20T01:17:11.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.657+0000 7f4cb2a08700 1 -- start start 2023-05-20T01:17:11.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.658+0000 7f4cb2a08700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4cac0cd1c0 0x7f4cac151730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:11.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.658+0000 7f4cb2a08700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4cac0cd630 0x7f4cac0b2e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:11.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.658+0000 7f4cb2a08700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4cac159b50 0x7f4cac0b0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:11.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.658+0000 7f4cb2a08700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cac156ef0 con 0x7f4cac159b50 2023-05-20T01:17:11.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.658+0000 7f4cb2a08700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cac0b4510 con 0x7f4cac0cd1c0 2023-05-20T01:17:11.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.658+0000 7f4cb2a08700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cac0b3890 con 0x7f4cac0cd630 2023-05-20T01:17:11.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.658+0000 7f4cb0fa5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4cac159b50 0x7f4cac0b0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:11.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.658+0000 7f4cb0fa5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4cac159b50 0x7f4cac0b0ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58958/0 (socket says 172.21.15.16:58958) 2023-05-20T01:17:11.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.658+0000 7f4cb0fa5700 1 -- 172.21.15.16:0/67754184 learned_addr learned my addr 172.21.15.16:0/67754184 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:11.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.659+0000 7f4cab7fe700 1 --2- 172.21.15.16:0/67754184 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4cac0cd630 0x7f4cac0b2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:11.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.659+0000 7f4cabfff700 1 --2- 172.21.15.16:0/67754184 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4cac0cd1c0 0x7f4cac151730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:11.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.659+0000 7f4cb0fa5700 1 -- 172.21.15.16:0/67754184 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4cac0cd630 msgr2=0x7f4cac0b2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:11.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.659+0000 7f4cb0fa5700 1 --2- 172.21.15.16:0/67754184 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4cac0cd630 0x7f4cac0b2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:11.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.659+0000 7f4cb0fa5700 1 -- 172.21.15.16:0/67754184 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4cac0cd1c0 msgr2=0x7f4cac151730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:11.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.659+0000 7f4cb0fa5700 1 --2- 172.21.15.16:0/67754184 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4cac0cd1c0 0x7f4cac151730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:11.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.659+0000 7f4cb0fa5700 1 -- 172.21.15.16:0/67754184 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c940094a0 con 0x7f4cac159b50 2023-05-20T01:17:11.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.659+0000 7f4cabfff700 1 --2- 172.21.15.16:0/67754184 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4cac0cd1c0 0x7f4cac151730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:17:11.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.659+0000 7f4cab7fe700 1 --2- 172.21.15.16:0/67754184 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4cac0cd630 0x7f4cac0b2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:17:11.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.659+0000 7f4cb0fa5700 1 --2- 172.21.15.16:0/67754184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4cac159b50 0x7f4cac0b0ec0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f4ca4014040 tx=0x7f4ca4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:11.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.659+0000 7f4ca8ff9700 1 -- 172.21.15.16:0/67754184 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ca40129f0 con 0x7f4cac159b50 2023-05-20T01:17:11.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.659+0000 7f4ca8ff9700 1 -- 172.21.15.16:0/67754184 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ca4020070 con 0x7f4cac159b50 2023-05-20T01:17:11.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.659+0000 7f4ca8ff9700 1 -- 172.21.15.16:0/67754184 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ca4012db0 con 0x7f4cac159b50 2023-05-20T01:17:11.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.659+0000 7f4cb2a08700 1 -- 172.21.15.16:0/67754184 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ca4009a70 con 0x7f4cac159b50 2023-05-20T01:17:11.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.659+0000 7f4cb2a08700 1 -- 172.21.15.16:0/67754184 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cac0cec70 con 0x7f4cac159b50 2023-05-20T01:17:11.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.660+0000 7f4ca27fc700 1 -- 172.21.15.16:0/67754184 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c840052e0 con 0x7f4cac159b50 2023-05-20T01:17:11.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.661+0000 7f4ca8ff9700 1 -- 172.21.15.16:0/67754184 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4ca4004040 con 0x7f4cac159b50 2023-05-20T01:17:11.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.661+0000 7f4ca8ff9700 1 --2- 172.21.15.16:0/67754184 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f4c8c06ace0 0x7f4c8c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:11.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.661+0000 7f4ca8ff9700 1 -- 172.21.15.16:0/67754184 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f4ca4022070 con 0x7f4cac159b50 2023-05-20T01:17:11.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.661+0000 7f4cabfff700 1 --2- 172.21.15.16:0/67754184 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f4c8c06ace0 0x7f4c8c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:11.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.662+0000 7f4cabfff700 1 --2- 172.21.15.16:0/67754184 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f4c8c06ace0 0x7f4c8c06d190 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f4c9c001230 tx=0x7f4c9c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:11.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.666+0000 7f4ca8ff9700 1 -- 172.21.15.16:0/67754184 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f4ca4056220 con 0x7f4cac159b50 2023-05-20T01:17:11.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.861+0000 7f4ca27fc700 1 -- 172.21.15.16:0/67754184 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f4c84005b10 con 0x7f4cac159b50 2023-05-20T01:17:11.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.862+0000 7f4ca8ff9700 1 -- 172.21.15.16:0/67754184 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7f4ca4059840 con 0x7f4cac159b50 2023-05-20T01:17:11.866 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-05-20T01:17:11.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.864+0000 7f4ca27fc700 1 -- 172.21.15.16:0/67754184 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f4c8c06ace0 msgr2=0x7f4c8c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:11.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.864+0000 7f4ca27fc700 1 --2- 172.21.15.16:0/67754184 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f4c8c06ace0 0x7f4c8c06d190 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f4c9c001230 tx=0x7f4c9c009040 comp rx=0 tx=0).stop 2023-05-20T01:17:11.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.864+0000 7f4ca27fc700 1 -- 172.21.15.16:0/67754184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4cac159b50 msgr2=0x7f4cac0b0ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:11.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.864+0000 7f4ca27fc700 1 --2- 172.21.15.16:0/67754184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4cac159b50 0x7f4cac0b0ec0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f4ca4014040 tx=0x7f4ca4004ad0 comp rx=0 tx=0).stop 2023-05-20T01:17:11.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.864+0000 7f4ca27fc700 1 -- 172.21.15.16:0/67754184 shutdown_connections 2023-05-20T01:17:11.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.864+0000 7f4ca27fc700 1 --2- 172.21.15.16:0/67754184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4cac159b50 0x7f4cac0b0ec0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:11.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.864+0000 7f4ca27fc700 1 --2- 172.21.15.16:0/67754184 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4cac0cd1c0 0x7f4cac151730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:11.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.864+0000 7f4ca27fc700 1 --2- 172.21.15.16:0/67754184 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f4c8c06ace0 0x7f4c8c06d190 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:11.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.864+0000 7f4ca27fc700 1 --2- 172.21.15.16:0/67754184 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4cac0cd630 0x7f4cac0b2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:11.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.864+0000 7f4ca27fc700 1 -- 172.21.15.16:0/67754184 >> 172.21.15.16:0/67754184 conn(0x7f4cac0cffd0 msgr2=0x7f4cac0b1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:11.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.864+0000 7f4ca27fc700 1 -- 172.21.15.16:0/67754184 shutdown_connections 2023-05-20T01:17:11.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:11.864+0000 7f4ca27fc700 1 -- 172.21.15.16:0/67754184 wait complete. 2023-05-20T01:17:11.880 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-20T01:17:12.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.001+0000 7fa05f509700 1 Processor -- start 2023-05-20T01:17:12.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.001+0000 7fa05f509700 1 -- start start 2023-05-20T01:17:12.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.002+0000 7fa05f509700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa0580d16c0 0x7fa058144330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:12.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.002+0000 7fa05f509700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa058144870 con 0x7fa0580cec20 2023-05-20T01:17:12.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.002+0000 7fa05f509700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0580b2000 con 0x7fa0580b1990 2023-05-20T01:17:12.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.002+0000 7fa05f509700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0580b1e10 con 0x7fa0580d16c0 2023-05-20T01:17:12.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.002+0000 7fa05caa4700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fa0580cec20 0x7fa0580ae2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:42258/0 (socket says 172.21.15.16:42258) 2023-05-20T01:17:12.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.002+0000 7fa05caa4700 1 -- 172.21.15.16:0/1716750544 learned_addr learned my addr 172.21.15.16:0/1716750544 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:17:12.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.002+0000 7fa05d2a5700 1 --2- 172.21.15.16:0/1716750544 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa0580d16c0 0x7fa058144330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:12.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.002+0000 7fa04ffff700 1 -- 172.21.15.16:0/1716750544 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1764880706 0 0) 0x7fa058144870 con 0x7fa0580cec20 2023-05-20T01:17:12.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.002+0000 7fa04ffff700 1 -- 172.21.15.16:0/1716750544 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0440035b0 con 0x7fa0580cec20 2023-05-20T01:17:12.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.002+0000 7fa04ffff700 1 -- 172.21.15.16:0/1716750544 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1974566466 0 0) 0x7fa0440035b0 con 0x7fa0580cec20 2023-05-20T01:17:12.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.002+0000 7fa04ffff700 1 -- 172.21.15.16:0/1716750544 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa0580d16c0 msgr2=0x7fa058144330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:12.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.002+0000 7fa04ffff700 1 --2- 172.21.15.16:0/1716750544 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa0580d16c0 0x7fa058144330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:12.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.002+0000 7fa04ffff700 1 -- 172.21.15.16:0/1716750544 >> v1:172.21.15.132:6789/0 conn(0x7fa0580b1990 legacy=0x7fa0580ae9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:12.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.002+0000 7fa04ffff700 1 -- 172.21.15.16:0/1716750544 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0580b22e0 con 0x7fa0580cec20 2023-05-20T01:17:12.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.002+0000 7fa04ffff700 1 -- 172.21.15.16:0/1716750544 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7fa048001670 con 0x7fa0580cec20 2023-05-20T01:17:12.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.003+0000 7fa04ffff700 1 -- 172.21.15.16:0/1716750544 >> v1:172.21.15.16:6789/0 conn(0x7fa0580cec20 legacy=0x7fa0580ae2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:12.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.003+0000 7fa04ffff700 1 --2- 172.21.15.16:0/1716750544 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa0580d16c0 0x7fa044003aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:12.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.003+0000 7fa04ffff700 1 --2- 172.21.15.16:0/1716750544 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa044003fe0 0x7fa044004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:12.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.003+0000 7fa04ffff700 1 --2- 172.21.15.16:0/1716750544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa044004d90 0x7fa044005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:12.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.003+0000 7fa04ffff700 1 -- 172.21.15.16:0/1716750544 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa04400bac0 con 0x7fa044004d90 2023-05-20T01:17:12.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.003+0000 7fa04ffff700 1 -- 172.21.15.16:0/1716750544 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa04400bc00 con 0x7fa044003fe0 2023-05-20T01:17:12.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.003+0000 7fa04ffff700 1 -- 172.21.15.16:0/1716750544 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa04400bdc0 con 0x7fa0580d16c0 2023-05-20T01:17:12.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.003+0000 7fa05daa6700 1 --2- 172.21.15.16:0/1716750544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa044004d90 0x7fa044005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:12.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.003+0000 7fa05d2a5700 1 --2- 172.21.15.16:0/1716750544 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa0580d16c0 0x7fa044003aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:12.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.003+0000 7fa05caa4700 1 --2- 172.21.15.16:0/1716750544 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa044003fe0 0x7fa044004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:12.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.003+0000 7fa05daa6700 1 -- 172.21.15.16:0/1716750544 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa0580d16c0 msgr2=0x7fa044003aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:12.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.003+0000 7fa05daa6700 1 --2- 172.21.15.16:0/1716750544 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa0580d16c0 0x7fa044003aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:12.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.003+0000 7fa05daa6700 1 -- 172.21.15.16:0/1716750544 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa044003fe0 msgr2=0x7fa044004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:12.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.003+0000 7fa05daa6700 1 --2- 172.21.15.16:0/1716750544 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa044003fe0 0x7fa044004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:12.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.003+0000 7fa05daa6700 1 -- 172.21.15.16:0/1716750544 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0440035b0 con 0x7fa044004d90 2023-05-20T01:17:12.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.003+0000 7fa05daa6700 1 --2- 172.21.15.16:0/1716750544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa044004d90 0x7fa044005200 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa05000c260 tx=0x7fa044005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=267965653dfaa8f5 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:12.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.003+0000 7fa04ffff700 1 -- 172.21.15.16:0/1716750544 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa050001e70 con 0x7fa044004d90 2023-05-20T01:17:12.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.003+0000 7fa04ffff700 1 -- 172.21.15.16:0/1716750544 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa05000ce20 con 0x7fa044004d90 2023-05-20T01:17:12.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.003+0000 7fa04ffff700 1 -- 172.21.15.16:0/1716750544 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa050012660 con 0x7fa044004d90 2023-05-20T01:17:12.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.004+0000 7fa05f509700 1 -- 172.21.15.16:0/1716750544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa044004d90 msgr2=0x7fa044005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:12.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.004+0000 7fa05f509700 1 --2- 172.21.15.16:0/1716750544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa044004d90 0x7fa044005200 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa05000c260 tx=0x7fa044005880 comp rx=0 tx=0).stop 2023-05-20T01:17:12.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.004+0000 7fa05daa6700 1 -- 172.21.15.16:0/1716750544 reap_dead start 2023-05-20T01:17:12.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.004+0000 7fa05f509700 1 -- 172.21.15.16:0/1716750544 shutdown_connections 2023-05-20T01:17:12.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.004+0000 7fa05f509700 1 -- 172.21.15.16:0/1716750544 >> 172.21.15.16:0/1716750544 conn(0x7fa05806ae30 msgr2=0x7fa05809a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:12.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.004+0000 7fa05f509700 1 -- 172.21.15.16:0/1716750544 shutdown_connections 2023-05-20T01:17:12.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.004+0000 7fa05f509700 1 -- 172.21.15.16:0/1716750544 wait complete. 2023-05-20T01:17:12.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.005+0000 7fa05f509700 1 Processor -- start 2023-05-20T01:17:12.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.005+0000 7fa05f509700 1 -- start start 2023-05-20T01:17:12.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.006+0000 7fa05f509700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa0581553c0 0x7fa0580ca420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:12.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.006+0000 7fa05f509700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa058155790 0x7fa0580c8120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:12.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.006+0000 7fa05f509700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0580ca960 0x7fa0580c8660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:12.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.006+0000 7fa05f509700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0580c9f30 con 0x7fa0580ca960 2023-05-20T01:17:12.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.006+0000 7fa05f509700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0580cb4a0 con 0x7fa058155790 2023-05-20T01:17:12.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.006+0000 7fa05f509700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0580caee0 con 0x7fa0581553c0 2023-05-20T01:17:12.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.006+0000 7fa05daa6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0580ca960 0x7fa0580c8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:12.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.006+0000 7fa05daa6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0580ca960 0x7fa0580c8660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58976/0 (socket says 172.21.15.16:58976) 2023-05-20T01:17:12.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.006+0000 7fa05daa6700 1 -- 172.21.15.16:0/3470247730 learned_addr learned my addr 172.21.15.16:0/3470247730 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:12.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.006+0000 7fa05d2a5700 1 --2- 172.21.15.16:0/3470247730 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa0581553c0 0x7fa0580ca420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:12.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.006+0000 7fa05caa4700 1 --2- 172.21.15.16:0/3470247730 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa058155790 0x7fa0580c8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:12.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.006+0000 7fa05daa6700 1 -- 172.21.15.16:0/3470247730 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa0581553c0 msgr2=0x7fa0580ca420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:12.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.006+0000 7fa05daa6700 1 --2- 172.21.15.16:0/3470247730 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa0581553c0 0x7fa0580ca420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:12.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.006+0000 7fa05daa6700 1 -- 172.21.15.16:0/3470247730 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa058155790 msgr2=0x7fa0580c8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:12.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.006+0000 7fa05daa6700 1 --2- 172.21.15.16:0/3470247730 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa058155790 0x7fa0580c8120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:12.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.006+0000 7fa05daa6700 1 -- 172.21.15.16:0/3470247730 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0580adda0 con 0x7fa0580ca960 2023-05-20T01:17:12.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.006+0000 7fa05d2a5700 1 --2- 172.21.15.16:0/3470247730 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa0581553c0 0x7fa0580ca420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:17:12.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.007+0000 7fa05daa6700 1 --2- 172.21.15.16:0/3470247730 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0580ca960 0x7fa0580c8660 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fa050013000 tx=0x7fa0500015b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:12.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.007+0000 7fa04dffb700 1 -- 172.21.15.16:0/3470247730 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa050003c00 con 0x7fa0580ca960 2023-05-20T01:17:12.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.007+0000 7fa04dffb700 1 -- 172.21.15.16:0/3470247730 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa050020070 con 0x7fa0580ca960 2023-05-20T01:17:12.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.007+0000 7fa05f509700 1 -- 172.21.15.16:0/3470247730 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0580b25c0 con 0x7fa0580ca960 2023-05-20T01:17:12.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.007+0000 7fa04dffb700 1 -- 172.21.15.16:0/3470247730 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0500050c0 con 0x7fa0580ca960 2023-05-20T01:17:12.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.007+0000 7fa05f509700 1 -- 172.21.15.16:0/3470247730 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0580b28a0 con 0x7fa0580ca960 2023-05-20T01:17:12.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.008+0000 7fa05f509700 1 -- 172.21.15.16:0/3470247730 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0580ccbd0 con 0x7fa0580ca960 2023-05-20T01:17:12.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.008+0000 7fa04dffb700 1 -- 172.21.15.16:0/3470247730 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa050022030 con 0x7fa0580ca960 2023-05-20T01:17:12.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.009+0000 7fa04dffb700 1 --2- 172.21.15.16:0/3470247730 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa03c06ace0 0x7fa03c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:12.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.009+0000 7fa04dffb700 1 -- 172.21.15.16:0/3470247730 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fa05008a8e0 con 0x7fa0580ca960 2023-05-20T01:17:12.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.009+0000 7fa05d2a5700 1 --2- 172.21.15.16:0/3470247730 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa03c06ace0 0x7fa03c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:12.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.009+0000 7fa05d2a5700 1 --2- 172.21.15.16:0/3470247730 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa03c06ace0 0x7fa03c06d190 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa054009c60 tx=0x7fa054008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:12.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.013+0000 7fa04dffb700 1 -- 172.21.15.16:0/3470247730 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fa0500556b0 con 0x7fa0580ca960 2023-05-20T01:17:12.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.214+0000 7fa05f509700 1 -- 172.21.15.16:0/3470247730 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa0580747b0 con 0x7fa0580ca960 2023-05-20T01:17:12.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.215+0000 7fa04dffb700 1 -- 172.21.15.16:0/3470247730 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+20026 (secure 0 0 0) 0x7fa050058cd0 con 0x7fa0580ca960 2023-05-20T01:17:12.219 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:17:12.220 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"fsid":"c2245cc5-ecab-4087-a59b-bc81db901676","created":"2023-05-20T01:16:09.549230+0000","modified":"2023-05-20T01:16:32.768495+0000","last_up_change":"2023-05-20T01:16:19.788624+0000","last_in_change":"2023-05-20T01:16:12.092160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-05-20T01:16:20.934366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-05-20T01:16:21.372294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-05-20T01:16:26.208287+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-05-20T01:16:27.292268+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-05-20T01:16:30.357285+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"317a83cd-19c8-4958-a301-6087ef342289","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6825","nonce":3923634179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6827","nonce":3923634179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6831","nonce":3923634179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6829","nonce":3923634179}]},"public_addr":"172.21.15.16:6825/3923634179","cluster_addr":"172.21.15.16:6827/3923634179","heartbeat_back_addr":"172.21.15.16:6831/3923634179","heartbeat_front_addr":"172.21.15.16:6829/3923634179","state":["exists","up"]},{"osd":1,"uuid":"5581046f-8219-4089-8d7d-b7846399505e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6809","nonce":3860264220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6811","nonce":3860264220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6815","nonce":3860264220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6813","nonce":3860264220}]},"public_addr":"172.21.15.16:6809/3860264220","cluster_addr":"172.21.15.16:6811/3860264220","heartbeat_back_addr":"172.21.15.16:6815/3860264220","heartbeat_front_addr":"172.21.15.16:6813/3860264220","state":["exists","up"]},{"osd":2,"uuid":"072e9e66-4133-478e-8d4a-64d46c2963f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6817","nonce":293187030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6819","nonce":293187030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6823","nonce":293187030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6821","nonce":293187030}]},"public_addr":"172.21.15.16:6817/293187030","cluster_addr":"172.21.15.16:6819/293187030","heartbeat_back_addr":"172.21.15.16:6823/293187030","heartbeat_front_addr":"172.21.15.16:6821/293187030","state":["exists","up"]},{"osd":3,"uuid":"11e1bba6-00f8-48dd-9028-d8d716b7f145","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6801","nonce":3669781890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6803","nonce":3669781890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6807","nonce":3669781890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6805","nonce":3669781890}]},"public_addr":"172.21.15.16:6801/3669781890","cluster_addr":"172.21.15.16:6803/3669781890","heartbeat_back_addr":"172.21.15.16:6807/3669781890","heartbeat_front_addr":"172.21.15.16:6805/3669781890","state":["exists","up"]},{"osd":4,"uuid":"b8909c3a-cf70-46cb-8d0b-009c707cf9d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6819","nonce":506317223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6823","nonce":506317223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6829","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6831","nonce":506317223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6827","nonce":506317223}]},"public_addr":"172.21.15.132:6819/506317223","cluster_addr":"172.21.15.132:6823/506317223","heartbeat_back_addr":"172.21.15.132:6831/506317223","heartbeat_front_addr":"172.21.15.132:6827/506317223","state":["exists","up"]},{"osd":5,"uuid":"b1818304-4583-47bf-a6ec-0bfbb5166233","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6818","nonce":1844689995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6822","nonce":1844689995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6830","nonce":1844689995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6826","nonce":1844689995}]},"public_addr":"172.21.15.132:6818/1844689995","cluster_addr":"172.21.15.132:6822/1844689995","heartbeat_back_addr":"172.21.15.132:6830/1844689995","heartbeat_front_addr":"172.21.15.132:6826/1844689995","state":["exists","up"]},{"osd":6,"uuid":"d37f4691-eed1-4a93-8818-18354f03f4a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6801","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6802","nonce":727581123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6805","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6807","nonce":727581123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6813","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6815","nonce":727581123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6811","nonce":727581123}]},"public_addr":"172.21.15.132:6802/727581123","cluster_addr":"172.21.15.132:6807/727581123","heartbeat_back_addr":"172.21.15.132:6815/727581123","heartbeat_front_addr":"172.21.15.132:6811/727581123","state":["exists","up"]},{"osd":7,"uuid":"4f712f62-5d1d-4816-8042-03209144f8b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6803","nonce":466001149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6806","nonce":466001149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6814","nonce":466001149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6810","nonce":466001149}]},"public_addr":"172.21.15.132:6803/466001149","cluster_addr":"172.21.15.132:6806/466001149","heartbeat_back_addr":"172.21.15.132:6814/466001149","heartbeat_front_addr":"172.21.15.132:6810/466001149","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.338863+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.278680+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305694+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.246835+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.276237+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.304441+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305421+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.267050+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-20T01:17:12.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.217+0000 7fa05f509700 1 -- 172.21.15.16:0/3470247730 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa03c06ace0 msgr2=0x7fa03c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:12.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.217+0000 7fa05f509700 1 --2- 172.21.15.16:0/3470247730 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa03c06ace0 0x7fa03c06d190 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa054009c60 tx=0x7fa054008040 comp rx=0 tx=0).stop 2023-05-20T01:17:12.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.217+0000 7fa05f509700 1 -- 172.21.15.16:0/3470247730 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0580ca960 msgr2=0x7fa0580c8660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:12.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.217+0000 7fa05f509700 1 --2- 172.21.15.16:0/3470247730 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0580ca960 0x7fa0580c8660 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fa050013000 tx=0x7fa0500015b0 comp rx=0 tx=0).stop 2023-05-20T01:17:12.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.218+0000 7fa05f509700 1 -- 172.21.15.16:0/3470247730 shutdown_connections 2023-05-20T01:17:12.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.218+0000 7fa05f509700 1 --2- 172.21.15.16:0/3470247730 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0580ca960 0x7fa0580c8660 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:12.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.218+0000 7fa05f509700 1 --2- 172.21.15.16:0/3470247730 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa03c06ace0 0x7fa03c06d190 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:12.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.218+0000 7fa05f509700 1 --2- 172.21.15.16:0/3470247730 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa0581553c0 0x7fa0580ca420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:12.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.218+0000 7fa05f509700 1 --2- 172.21.15.16:0/3470247730 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa058155790 0x7fa0580c8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:12.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.218+0000 7fa05f509700 1 -- 172.21.15.16:0/3470247730 >> 172.21.15.16:0/3470247730 conn(0x7fa05806ae30 msgr2=0x7fa0580b0e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:12.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.218+0000 7fa05f509700 1 -- 172.21.15.16:0/3470247730 shutdown_connections 2023-05-20T01:17:12.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.218+0000 7fa05f509700 1 -- 172.21.15.16:0/3470247730 wait complete. 2023-05-20T01:17:12.234 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-05-20T01:16:20.934366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-05-20T01:16:21.372294+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-05-20T01:16:26.208287+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-05-20T01:16:27.292268+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-05-20T01:16:30.357285+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-05-20T01:17:12.235 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-05-20T01:17:12.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.360+0000 7fa4517cb700 1 Processor -- start 2023-05-20T01:17:12.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.360+0000 7fa4517cb700 1 -- start start 2023-05-20T01:17:12.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.361+0000 7fa4517cb700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fa44c0b67a0 0x7fa44c167730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:12.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.361+0000 7fa4517cb700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa44c0d1820 con 0x7fa44c15ca90 2023-05-20T01:17:12.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.361+0000 7fa4517cb700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa44c0d2c60 con 0x7fa44c0cd890 2023-05-20T01:17:12.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.361+0000 7fa4517cb700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa44c0d0c50 con 0x7fa44c0b67a0 2023-05-20T01:17:12.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.361+0000 7fa44affd700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fa44c15ca90 0x7fa44c0d2550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:42276/0 (socket says 172.21.15.16:42276) 2023-05-20T01:17:12.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.361+0000 7fa44affd700 1 -- 172.21.15.16:0/242902996 learned_addr learned my addr 172.21.15.16:0/242902996 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:17:12.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.361+0000 7fa44b7fe700 1 --2- 172.21.15.16:0/242902996 >> v2:172.21.15.132:3300/0 conn(0x7fa44c0b67a0 0x7fa44c167730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:12.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.361+0000 7fa449ffb700 1 -- 172.21.15.16:0/242902996 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2090288081 0 0) 0x7fa44c0d2c60 con 0x7fa44c0cd890 2023-05-20T01:17:12.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.361+0000 7fa449ffb700 1 -- 172.21.15.16:0/242902996 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa4340035b0 con 0x7fa44c0cd890 2023-05-20T01:17:12.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.361+0000 7fa449ffb700 1 -- 172.21.15.16:0/242902996 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 579498570 0 0) 0x7fa44c0d1820 con 0x7fa44c15ca90 2023-05-20T01:17:12.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.361+0000 7fa449ffb700 1 -- 172.21.15.16:0/242902996 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa44c0d2c60 con 0x7fa44c15ca90 2023-05-20T01:17:12.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.361+0000 7fa44b7fe700 1 -- 172.21.15.16:0/242902996 >> v1:172.21.15.132:6789/0 conn(0x7fa44c0cd890 legacy=0x7fa44c0b6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:12.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.361+0000 7fa44b7fe700 1 -- 172.21.15.16:0/242902996 >> v1:172.21.15.16:6789/0 conn(0x7fa44c15ca90 legacy=0x7fa44c0d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:12.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.361+0000 7fa44b7fe700 1 -- 172.21.15.16:0/242902996 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa44c0cdf00 con 0x7fa44c0b67a0 2023-05-20T01:17:12.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.362+0000 7fa44b7fe700 1 --2- 172.21.15.16:0/242902996 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa44c0b67a0 0x7fa44c167730 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa444009d90 tx=0x7fa44400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7bcf82a6fa0d928 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:12.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.362+0000 7fa449ffb700 1 -- 172.21.15.16:0/242902996 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa44400e070 con 0x7fa44c0b67a0 2023-05-20T01:17:12.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.362+0000 7fa449ffb700 1 -- 172.21.15.16:0/242902996 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa44400bb20 con 0x7fa44c0b67a0 2023-05-20T01:17:12.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.362+0000 7fa449ffb700 1 -- 172.21.15.16:0/242902996 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa44400be90 con 0x7fa44c0b67a0 2023-05-20T01:17:12.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.363+0000 7fa4517cb700 1 -- 172.21.15.16:0/242902996 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa44c0b67a0 msgr2=0x7fa44c167730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:12.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.363+0000 7fa4517cb700 1 --2- 172.21.15.16:0/242902996 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa44c0b67a0 0x7fa44c167730 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa444009d90 tx=0x7fa44400b330 comp rx=0 tx=0).stop 2023-05-20T01:17:12.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.363+0000 7fa4517cb700 1 -- 172.21.15.16:0/242902996 shutdown_connections 2023-05-20T01:17:12.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.363+0000 7fa4517cb700 1 --2- 172.21.15.16:0/242902996 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa44c0b67a0 0x7fa44c167730 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:12.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.363+0000 7fa4517cb700 1 -- 172.21.15.16:0/242902996 >> 172.21.15.16:0/242902996 conn(0x7fa44c06b000 msgr2=0x7fa44c06b400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:12.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.363+0000 7fa4517cb700 1 -- 172.21.15.16:0/242902996 shutdown_connections 2023-05-20T01:17:12.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.363+0000 7fa4517cb700 1 -- 172.21.15.16:0/242902996 wait complete. 2023-05-20T01:17:12.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.364+0000 7fa4517cb700 1 Processor -- start 2023-05-20T01:17:12.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.365+0000 7fa4517cb700 1 -- start start 2023-05-20T01:17:12.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.365+0000 7fa4517cb700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa44c0b67a0 0x7fa44c1f1b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:12.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.365+0000 7fa4517cb700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa44c0cd890 0x7fa44c1f2a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:12.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.365+0000 7fa4517cb700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa44c15ca90 0x7fa44c15b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:12.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.365+0000 7fa4517cb700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa44c1f5720 con 0x7fa44c15ca90 2023-05-20T01:17:12.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.365+0000 7fa4517cb700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa44c1f5a00 con 0x7fa44c0b67a0 2023-05-20T01:17:12.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.365+0000 7fa4517cb700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa44c1f5ce0 con 0x7fa44c0cd890 2023-05-20T01:17:12.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.365+0000 7fa44b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa44c15ca90 0x7fa44c15b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:12.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.365+0000 7fa44b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa44c15ca90 0x7fa44c15b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58990/0 (socket says 172.21.15.16:58990) 2023-05-20T01:17:12.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.365+0000 7fa44affd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa44c0b67a0 0x7fa44c1f1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:12.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.365+0000 7fa44b7fe700 1 -- 172.21.15.16:0/45283859 learned_addr learned my addr 172.21.15.16:0/45283859 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:12.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.365+0000 7fa44a7fc700 1 --2- 172.21.15.16:0/45283859 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa44c0cd890 0x7fa44c1f2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:12.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.366+0000 7fa44b7fe700 1 -- 172.21.15.16:0/45283859 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa44c0cd890 msgr2=0x7fa44c1f2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:12.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.366+0000 7fa44b7fe700 1 --2- 172.21.15.16:0/45283859 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa44c0cd890 0x7fa44c1f2a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:12.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.366+0000 7fa44b7fe700 1 -- 172.21.15.16:0/45283859 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa44c0b67a0 msgr2=0x7fa44c1f1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:12.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.366+0000 7fa44b7fe700 1 --2- 172.21.15.16:0/45283859 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa44c0b67a0 0x7fa44c1f1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:12.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.366+0000 7fa44b7fe700 1 -- 172.21.15.16:0/45283859 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa444009a70 con 0x7fa44c15ca90 2023-05-20T01:17:12.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.366+0000 7fa44b7fe700 1 --2- 172.21.15.16:0/45283859 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa44c15ca90 0x7fa44c15b090 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fa44400b370 tx=0x7fa444004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:12.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.366+0000 7fa4337fe700 1 -- 172.21.15.16:0/45283859 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4440098f0 con 0x7fa44c15ca90 2023-05-20T01:17:12.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.366+0000 7fa4337fe700 1 -- 172.21.15.16:0/45283859 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa44401f070 con 0x7fa44c15ca90 2023-05-20T01:17:12.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.366+0000 7fa4517cb700 1 -- 172.21.15.16:0/45283859 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa44c0d1d50 con 0x7fa44c15ca90 2023-05-20T01:17:12.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.366+0000 7fa4337fe700 1 -- 172.21.15.16:0/45283859 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa444003c40 con 0x7fa44c15ca90 2023-05-20T01:17:12.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.366+0000 7fa4517cb700 1 -- 172.21.15.16:0/45283859 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa44c0d1a70 con 0x7fa44c15ca90 2023-05-20T01:17:12.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.367+0000 7fa4517cb700 1 -- 172.21.15.16:0/45283859 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa44c0d2030 con 0x7fa44c15ca90 2023-05-20T01:17:12.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.368+0000 7fa4337fe700 1 -- 172.21.15.16:0/45283859 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa444004030 con 0x7fa44c15ca90 2023-05-20T01:17:12.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.368+0000 7fa4337fe700 1 --2- 172.21.15.16:0/45283859 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa43806ace0 0x7fa43806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:12.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.368+0000 7fa4337fe700 1 -- 172.21.15.16:0/45283859 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fa44408af20 con 0x7fa44c15ca90 2023-05-20T01:17:12.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.368+0000 7fa44affd700 1 --2- 172.21.15.16:0/45283859 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa43806ace0 0x7fa43806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:12.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.369+0000 7fa44affd700 1 --2- 172.21.15.16:0/45283859 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa43806ace0 0x7fa43806d190 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa43c007fd0 tx=0x7fa43c007dc0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:12.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.372+0000 7fa4337fe700 1 -- 172.21.15.16:0/45283859 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fa444055ce0 con 0x7fa44c15ca90 2023-05-20T01:17:12.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.569+0000 7fa4517cb700 1 -- 172.21.15.16:0/45283859 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fa44c04e8c0 con 0x7fa44c15ca90 2023-05-20T01:17:12.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.570+0000 7fa4337fe700 1 -- 172.21.15.16:0/45283859 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7fa444059300 con 0x7fa44c15ca90 2023-05-20T01:17:12.575 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 8 2023-05-20T01:17:12.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.572+0000 7fa4517cb700 1 -- 172.21.15.16:0/45283859 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa43806ace0 msgr2=0x7fa43806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:12.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.572+0000 7fa4517cb700 1 --2- 172.21.15.16:0/45283859 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa43806ace0 0x7fa43806d190 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa43c007fd0 tx=0x7fa43c007dc0 comp rx=0 tx=0).stop 2023-05-20T01:17:12.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.572+0000 7fa4517cb700 1 -- 172.21.15.16:0/45283859 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa44c15ca90 msgr2=0x7fa44c15b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:12.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.572+0000 7fa4517cb700 1 --2- 172.21.15.16:0/45283859 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa44c15ca90 0x7fa44c15b090 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fa44400b370 tx=0x7fa444004ad0 comp rx=0 tx=0).stop 2023-05-20T01:17:12.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.572+0000 7fa4517cb700 1 -- 172.21.15.16:0/45283859 shutdown_connections 2023-05-20T01:17:12.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.572+0000 7fa4517cb700 1 --2- 172.21.15.16:0/45283859 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa44c15ca90 0x7fa44c15b090 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:12.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.572+0000 7fa4517cb700 1 --2- 172.21.15.16:0/45283859 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa44c0b67a0 0x7fa44c1f1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:12.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.572+0000 7fa4517cb700 1 --2- 172.21.15.16:0/45283859 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa43806ace0 0x7fa43806d190 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:12.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.572+0000 7fa4517cb700 1 --2- 172.21.15.16:0/45283859 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa44c0cd890 0x7fa44c1f2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:12.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.572+0000 7fa4517cb700 1 -- 172.21.15.16:0/45283859 >> 172.21.15.16:0/45283859 conn(0x7fa44c06b000 msgr2=0x7fa44c06b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:12.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.572+0000 7fa4517cb700 1 -- 172.21.15.16:0/45283859 shutdown_connections 2023-05-20T01:17:12.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.572+0000 7fa4517cb700 1 -- 172.21.15.16:0/45283859 wait complete. 2023-05-20T01:17:12.588 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-05-20T01:17:12.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.712+0000 7f28aa78e700 1 Processor -- start 2023-05-20T01:17:12.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.712+0000 7f28aa78e700 1 -- start start 2023-05-20T01:17:12.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.713+0000 7f28aa78e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f28a40ce7b0 0x7f28a40cb7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:12.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.713+0000 7f28aa78e700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f28a40cc050 0x7f28a40cc470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:12.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.713+0000 7f28aa78e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28a4159360 con 0x7f28a4158f90 2023-05-20T01:17:12.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.713+0000 7f28aa78e700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28a40cce10 con 0x7f28a40cc050 2023-05-20T01:17:12.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.713+0000 7f28aa78e700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28a40ca750 con 0x7f28a40ce7b0 2023-05-20T01:17:12.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.713+0000 7f28a8d2b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f28a40cc050 0x7f28a40cc470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:12.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.713+0000 7f28a8d2b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f28a40cc050 0x7f28a40cc470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58998/0 (socket says 172.21.15.16:58998) 2023-05-20T01:17:12.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.713+0000 7f28a8d2b700 1 -- 172.21.15.16:0/2660884300 learned_addr learned my addr 172.21.15.16:0/2660884300 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:12.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.713+0000 7f28a37fe700 1 --2- 172.21.15.16:0/2660884300 >> v2:172.21.15.132:3300/0 conn(0x7f28a40ce7b0 0x7f28a40cb7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:12.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.713+0000 7f28a8d2b700 1 -- 172.21.15.16:0/2660884300 >> v2:172.21.15.132:3300/0 conn(0x7f28a40ce7b0 msgr2=0x7f28a40cb7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:12.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.713+0000 7f28a8d2b700 1 --2- 172.21.15.16:0/2660884300 >> v2:172.21.15.132:3300/0 conn(0x7f28a40ce7b0 0x7f28a40cb7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:12.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.713+0000 7f28a8d2b700 1 -- 172.21.15.16:0/2660884300 >> v1:172.21.15.132:6789/0 conn(0x7f28a4158f90 legacy=0x7f28a40ce0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:12.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.713+0000 7f28a8d2b700 1 -- 172.21.15.16:0/2660884300 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28a40ca9f0 con 0x7f28a40cc050 2023-05-20T01:17:12.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.713+0000 7f28a37fe700 1 --2- 172.21.15.16:0/2660884300 >> v2:172.21.15.132:3300/0 conn(0x7f28a40ce7b0 0x7f28a40cb7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:17:12.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.714+0000 7f28a8d2b700 1 --2- 172.21.15.16:0/2660884300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28a40cc050 0x7f28a40cc470 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f28a40ceb80 tx=0x7f289c00d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=946841553f9406fd server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:12.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.714+0000 7f28a2ffd700 1 -- 172.21.15.16:0/2660884300 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f289c010070 con 0x7f28a40cc050 2023-05-20T01:17:12.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.714+0000 7f28a2ffd700 1 -- 172.21.15.16:0/2660884300 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f289c00db40 con 0x7f28a40cc050 2023-05-20T01:17:12.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.714+0000 7f28a2ffd700 1 -- 172.21.15.16:0/2660884300 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f289c003a20 con 0x7f28a40cc050 2023-05-20T01:17:12.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.714+0000 7f28aa78e700 1 -- 172.21.15.16:0/2660884300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28a40cc050 msgr2=0x7f28a40cc470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:12.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.714+0000 7f28aa78e700 1 --2- 172.21.15.16:0/2660884300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28a40cc050 0x7f28a40cc470 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f28a40ceb80 tx=0x7f289c00d430 comp rx=0 tx=0).stop 2023-05-20T01:17:12.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.714+0000 7f28aa78e700 1 -- 172.21.15.16:0/2660884300 shutdown_connections 2023-05-20T01:17:12.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.714+0000 7f28aa78e700 1 --2- 172.21.15.16:0/2660884300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28a40cc050 0x7f28a40cc470 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:12.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.714+0000 7f28aa78e700 1 --2- 172.21.15.16:0/2660884300 >> v2:172.21.15.132:3300/0 conn(0x7f28a40ce7b0 0x7f28a40cb7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:12.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.714+0000 7f28aa78e700 1 -- 172.21.15.16:0/2660884300 >> 172.21.15.16:0/2660884300 conn(0x7f28a406cbb0 msgr2=0x7f28a406cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:12.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.714+0000 7f28aa78e700 1 -- 172.21.15.16:0/2660884300 shutdown_connections 2023-05-20T01:17:12.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.714+0000 7f28aa78e700 1 -- 172.21.15.16:0/2660884300 wait complete. 2023-05-20T01:17:12.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.716+0000 7f28aa78e700 1 Processor -- start 2023-05-20T01:17:12.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.716+0000 7f28aa78e700 1 -- start start 2023-05-20T01:17:12.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.717+0000 7f28aa78e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28a40cc050 0x7f28a4144ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:12.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.717+0000 7f28aa78e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f28a40ce7b0 0x7f28a414e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:12.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.717+0000 7f28aa78e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f28a4158f90 0x7f28a4157e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:12.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.717+0000 7f28aa78e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28a40b4780 con 0x7f28a40cc050 2023-05-20T01:17:12.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.717+0000 7f28aa78e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28a4145410 con 0x7f28a4158f90 2023-05-20T01:17:12.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.717+0000 7f28aa78e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28a40b3b70 con 0x7f28a40ce7b0 2023-05-20T01:17:12.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.717+0000 7f28a3fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28a40cc050 0x7f28a4144ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:12.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.717+0000 7f28a3fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28a40cc050 0x7f28a4144ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59000/0 (socket says 172.21.15.16:59000) 2023-05-20T01:17:12.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.717+0000 7f28a3fff700 1 -- 172.21.15.16:0/2821782369 learned_addr learned my addr 172.21.15.16:0/2821782369 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:12.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.717+0000 7f28a8d2b700 1 --2- 172.21.15.16:0/2821782369 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f28a4158f90 0x7f28a4157e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:12.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.717+0000 7f28a37fe700 1 --2- 172.21.15.16:0/2821782369 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f28a40ce7b0 0x7f28a414e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:12.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.717+0000 7f28a3fff700 1 -- 172.21.15.16:0/2821782369 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f28a40ce7b0 msgr2=0x7f28a414e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:12.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.717+0000 7f28a3fff700 1 --2- 172.21.15.16:0/2821782369 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f28a40ce7b0 0x7f28a414e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:12.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.718+0000 7f28a3fff700 1 -- 172.21.15.16:0/2821782369 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f28a4158f90 msgr2=0x7f28a4157e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:12.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.718+0000 7f28a3fff700 1 --2- 172.21.15.16:0/2821782369 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f28a4158f90 0x7f28a4157e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:12.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.718+0000 7f28a3fff700 1 -- 172.21.15.16:0/2821782369 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f289c00bd40 con 0x7f28a40cc050 2023-05-20T01:17:12.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.718+0000 7f28a8d2b700 1 --2- 172.21.15.16:0/2821782369 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f28a4158f90 0x7f28a4157e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:17:12.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.718+0000 7f28a37fe700 1 --2- 172.21.15.16:0/2821782369 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f28a40ce7b0 0x7f28a414e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:17:12.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.718+0000 7f28a3fff700 1 --2- 172.21.15.16:0/2821782369 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28a40cc050 0x7f28a4144ed0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f288c00b4a0 tx=0x7f288c00b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:12.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.718+0000 7f28a0ff9700 1 -- 172.21.15.16:0/2821782369 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f288c011830 con 0x7f28a40cc050 2023-05-20T01:17:12.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.718+0000 7f28a0ff9700 1 -- 172.21.15.16:0/2821782369 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f288c011990 con 0x7f28a40cc050 2023-05-20T01:17:12.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.718+0000 7f28a0ff9700 1 -- 172.21.15.16:0/2821782369 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f288c011d40 con 0x7f28a40cc050 2023-05-20T01:17:12.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.718+0000 7f28aa78e700 1 -- 172.21.15.16:0/2821782369 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28a40cafb0 con 0x7f28a40cc050 2023-05-20T01:17:12.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.718+0000 7f28aa78e700 1 -- 172.21.15.16:0/2821782369 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28a40cacd0 con 0x7f28a40cc050 2023-05-20T01:17:12.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.719+0000 7f28aa78e700 1 -- 172.21.15.16:0/2821782369 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28a40cb290 con 0x7f28a40cc050 2023-05-20T01:17:12.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.719+0000 7f28a0ff9700 1 -- 172.21.15.16:0/2821782369 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f288c01a400 con 0x7f28a40cc050 2023-05-20T01:17:12.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.720+0000 7f28a0ff9700 1 --2- 172.21.15.16:0/2821782369 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f288406ace0 0x7f288406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:12.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.720+0000 7f28a0ff9700 1 -- 172.21.15.16:0/2821782369 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f288c089510 con 0x7f28a40cc050 2023-05-20T01:17:12.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.720+0000 7f28a37fe700 1 --2- 172.21.15.16:0/2821782369 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f288406ace0 0x7f288406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:12.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.720+0000 7f28a37fe700 1 --2- 172.21.15.16:0/2821782369 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f288406ace0 0x7f288406d190 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f28a4150480 tx=0x7f2894008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:12.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.724+0000 7f28a0ff9700 1 -- 172.21.15.16:0/2821782369 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f288c0542c0 con 0x7f28a40cc050 2023-05-20T01:17:12.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.920+0000 7f28aa78e700 1 -- 172.21.15.16:0/2821782369 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f28a40747b0 con 0x7f28a40cc050 2023-05-20T01:17:12.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.921+0000 7f28a0ff9700 1 -- 172.21.15.16:0/2821782369 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7f288c0578e0 con 0x7f28a40cc050 2023-05-20T01:17:12.925 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-05-20T01:17:12.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.923+0000 7f28aa78e700 1 -- 172.21.15.16:0/2821782369 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f288406ace0 msgr2=0x7f288406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:12.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.923+0000 7f28aa78e700 1 --2- 172.21.15.16:0/2821782369 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f288406ace0 0x7f288406d190 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f28a4150480 tx=0x7f2894008040 comp rx=0 tx=0).stop 2023-05-20T01:17:12.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.923+0000 7f28aa78e700 1 -- 172.21.15.16:0/2821782369 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28a40cc050 msgr2=0x7f28a4144ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:12.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.923+0000 7f28aa78e700 1 --2- 172.21.15.16:0/2821782369 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28a40cc050 0x7f28a4144ed0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f288c00b4a0 tx=0x7f288c00b860 comp rx=0 tx=0).stop 2023-05-20T01:17:12.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.923+0000 7f28aa78e700 1 -- 172.21.15.16:0/2821782369 shutdown_connections 2023-05-20T01:17:12.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.923+0000 7f28aa78e700 1 --2- 172.21.15.16:0/2821782369 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f28a4158f90 0x7f28a4157e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:12.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.923+0000 7f28aa78e700 1 --2- 172.21.15.16:0/2821782369 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28a40cc050 0x7f28a4144ed0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:12.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.923+0000 7f28aa78e700 1 --2- 172.21.15.16:0/2821782369 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f288406ace0 0x7f288406d190 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:12.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.923+0000 7f28aa78e700 1 --2- 172.21.15.16:0/2821782369 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f28a40ce7b0 0x7f28a414e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:12.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.923+0000 7f28aa78e700 1 -- 172.21.15.16:0/2821782369 >> 172.21.15.16:0/2821782369 conn(0x7f28a406cbb0 msgr2=0x7f28a4145da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:12.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.923+0000 7f28aa78e700 1 -- 172.21.15.16:0/2821782369 shutdown_connections 2023-05-20T01:17:12.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:12.923+0000 7f28aa78e700 1 -- 172.21.15.16:0/2821782369 wait complete. 2023-05-20T01:17:12.938 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-05-20T01:17:13.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.062+0000 7fcbee4b1700 1 Processor -- start 2023-05-20T01:17:13.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.062+0000 7fcbee4b1700 1 -- start start 2023-05-20T01:17:13.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.063+0000 7fcbee4b1700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcbe815ca90 0x7fcbe80cd890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:13.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.063+0000 7fcbee4b1700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbe80d1820 con 0x7fcbe80d29a0 2023-05-20T01:17:13.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.063+0000 7fcbee4b1700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbe80cdf00 con 0x7fcbe80d2550 2023-05-20T01:17:13.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.063+0000 7fcbee4b1700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbe80d0c50 con 0x7fcbe815ca90 2023-05-20T01:17:13.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.063+0000 7fcbe7fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcbe815ca90 0x7fcbe80cd890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:13.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.063+0000 7fcbeca4e700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fcbe80d29a0 0x7fcbe80cf570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:42308/0 (socket says 172.21.15.16:42308) 2023-05-20T01:17:13.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.063+0000 7fcbe7fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcbe815ca90 0x7fcbe80cd890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59008/0 (socket says 172.21.15.16:59008) 2023-05-20T01:17:13.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.063+0000 7fcbeca4e700 1 -- 172.21.15.16:0/547630709 learned_addr learned my addr 172.21.15.16:0/547630709 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:17:13.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.064+0000 7fcbe7fff700 1 -- 172.21.15.16:0/547630709 >> v1:172.21.15.132:6789/0 conn(0x7fcbe80d2550 legacy=0x7fcbe80b6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:13.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.064+0000 7fcbe7fff700 1 -- 172.21.15.16:0/547630709 >> v1:172.21.15.16:6789/0 conn(0x7fcbe80d29a0 legacy=0x7fcbe80cf570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:13.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.064+0000 7fcbe6ffd700 1 -- 172.21.15.16:0/547630709 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 379130002 0 0) 0x7fcbe80d1820 con 0x7fcbe80d29a0 2023-05-20T01:17:13.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.064+0000 7fcbe7fff700 1 -- 172.21.15.16:0/547630709 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbe80d1a70 con 0x7fcbe815ca90 2023-05-20T01:17:13.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.064+0000 7fcbe7fff700 1 --2- 172.21.15.16:0/547630709 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcbe815ca90 0x7fcbe80cd890 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fcbd8009840 tx=0x7fcbd8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fde9cd128989d715 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:13.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.064+0000 7fcbe6ffd700 1 -- 172.21.15.16:0/547630709 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbd8011830 con 0x7fcbe815ca90 2023-05-20T01:17:13.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.064+0000 7fcbe6ffd700 1 -- 172.21.15.16:0/547630709 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbd8011990 con 0x7fcbe815ca90 2023-05-20T01:17:13.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.064+0000 7fcbe6ffd700 1 -- 172.21.15.16:0/547630709 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbd8011d20 con 0x7fcbe815ca90 2023-05-20T01:17:13.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.064+0000 7fcbee4b1700 1 -- 172.21.15.16:0/547630709 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcbe815ca90 msgr2=0x7fcbe80cd890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:13.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.064+0000 7fcbee4b1700 1 --2- 172.21.15.16:0/547630709 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcbe815ca90 0x7fcbe80cd890 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fcbd8009840 tx=0x7fcbd8009b50 comp rx=0 tx=0).stop 2023-05-20T01:17:13.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.065+0000 7fcbee4b1700 1 -- 172.21.15.16:0/547630709 shutdown_connections 2023-05-20T01:17:13.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.065+0000 7fcbee4b1700 1 --2- 172.21.15.16:0/547630709 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcbe815ca90 0x7fcbe80cd890 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:13.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.065+0000 7fcbee4b1700 1 -- 172.21.15.16:0/547630709 >> 172.21.15.16:0/547630709 conn(0x7fcbe806b000 msgr2=0x7fcbe806b400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:13.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.065+0000 7fcbee4b1700 1 -- 172.21.15.16:0/547630709 shutdown_connections 2023-05-20T01:17:13.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.065+0000 7fcbee4b1700 1 -- 172.21.15.16:0/547630709 wait complete. 2023-05-20T01:17:13.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.066+0000 7fcbee4b1700 1 Processor -- start 2023-05-20T01:17:13.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.066+0000 7fcbee4b1700 1 -- start start 2023-05-20T01:17:13.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.067+0000 7fcbee4b1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcbe80d2550 0x7fcbe815fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:13.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.067+0000 7fcbee4b1700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcbe80d29a0 0x7fcbe814a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:13.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.067+0000 7fcbee4b1700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcbe815ca90 0x7fcbe80b3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:13.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.067+0000 7fcbee4b1700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbe815d4d0 con 0x7fcbe80d2550 2023-05-20T01:17:13.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.067+0000 7fcbee4b1700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbe814ac00 con 0x7fcbe815ca90 2023-05-20T01:17:13.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.067+0000 7fcbee4b1700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbe80b4310 con 0x7fcbe80d29a0 2023-05-20T01:17:13.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.067+0000 7fcbe7fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcbe80d2550 0x7fcbe815fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:13.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.067+0000 7fcbe7fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcbe80d2550 0x7fcbe815fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59014/0 (socket says 172.21.15.16:59014) 2023-05-20T01:17:13.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.067+0000 7fcbe7fff700 1 -- 172.21.15.16:0/1635923546 learned_addr learned my addr 172.21.15.16:0/1635923546 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:13.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.067+0000 7fcbe77fe700 1 --2- 172.21.15.16:0/1635923546 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcbe80d29a0 0x7fcbe814a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:13.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.067+0000 7fcbe7fff700 1 -- 172.21.15.16:0/1635923546 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcbe80d29a0 msgr2=0x7fcbe814a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:13.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.067+0000 7fcbe7fff700 1 --2- 172.21.15.16:0/1635923546 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcbe80d29a0 0x7fcbe814a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:13.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.067+0000 7fcbe7fff700 1 -- 172.21.15.16:0/1635923546 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcbe815ca90 msgr2=0x7fcbe80b3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:13.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.067+0000 7fcbe7fff700 1 --2- 172.21.15.16:0/1635923546 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcbe815ca90 0x7fcbe80b3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:13.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.067+0000 7fcbe7fff700 1 -- 172.21.15.16:0/1635923546 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbd80094a0 con 0x7fcbe80d2550 2023-05-20T01:17:13.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.067+0000 7fcbe7fff700 1 --2- 172.21.15.16:0/1635923546 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcbe80d2550 0x7fcbe815fba0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fcbd80090b0 tx=0x7fcbd8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:13.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.067+0000 7fcbe4ff9700 1 -- 172.21.15.16:0/1635923546 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbd80036a0 con 0x7fcbe80d2550 2023-05-20T01:17:13.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.067+0000 7fcbe4ff9700 1 -- 172.21.15.16:0/1635923546 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbd801e070 con 0x7fcbe80d2550 2023-05-20T01:17:13.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.067+0000 7fcbee4b1700 1 -- 172.21.15.16:0/1635923546 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbe80d2030 con 0x7fcbe80d2550 2023-05-20T01:17:13.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.067+0000 7fcbe4ff9700 1 -- 172.21.15.16:0/1635923546 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbd8003a20 con 0x7fcbe80d2550 2023-05-20T01:17:13.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.067+0000 7fcbee4b1700 1 -- 172.21.15.16:0/1635923546 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbe80d1d50 con 0x7fcbe80d2550 2023-05-20T01:17:13.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.068+0000 7fcbee4b1700 1 -- 172.21.15.16:0/1635923546 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbe815ce60 con 0x7fcbe80d2550 2023-05-20T01:17:13.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.069+0000 7fcbe4ff9700 1 -- 172.21.15.16:0/1635923546 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fcbd8003b80 con 0x7fcbe80d2550 2023-05-20T01:17:13.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.069+0000 7fcbe4ff9700 1 --2- 172.21.15.16:0/1635923546 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fcbc806ace0 0x7fcbc806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:13.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.069+0000 7fcbe4ff9700 1 -- 172.21.15.16:0/1635923546 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fcbd808a9b0 con 0x7fcbe80d2550 2023-05-20T01:17:13.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.069+0000 7fcbe77fe700 1 --2- 172.21.15.16:0/1635923546 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fcbc806ace0 0x7fcbc806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:13.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.070+0000 7fcbe77fe700 1 --2- 172.21.15.16:0/1635923546 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fcbc806ace0 0x7fcbc806d190 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fcbe815d8a0 tx=0x7fcbd0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:13.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.074+0000 7fcbe4ff9700 1 -- 172.21.15.16:0/1635923546 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fcbd8055760 con 0x7fcbe80d2550 2023-05-20T01:17:13.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.270+0000 7fcbee4b1700 1 -- 172.21.15.16:0/1635923546 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fcbe80b4540 con 0x7fcbe80d2550 2023-05-20T01:17:13.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.270+0000 7fcbe4ff9700 1 -- 172.21.15.16:0/1635923546 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7fcbd8020020 con 0x7fcbe80d2550 2023-05-20T01:17:13.274 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-05-20T01:17:13.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.272+0000 7fcbee4b1700 1 -- 172.21.15.16:0/1635923546 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fcbc806ace0 msgr2=0x7fcbc806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:13.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.272+0000 7fcbee4b1700 1 --2- 172.21.15.16:0/1635923546 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fcbc806ace0 0x7fcbc806d190 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fcbe815d8a0 tx=0x7fcbd0009040 comp rx=0 tx=0).stop 2023-05-20T01:17:13.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.272+0000 7fcbee4b1700 1 -- 172.21.15.16:0/1635923546 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcbe80d2550 msgr2=0x7fcbe815fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:13.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.272+0000 7fcbee4b1700 1 --2- 172.21.15.16:0/1635923546 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcbe80d2550 0x7fcbe815fba0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fcbd80090b0 tx=0x7fcbd8004750 comp rx=0 tx=0).stop 2023-05-20T01:17:13.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.273+0000 7fcbee4b1700 1 -- 172.21.15.16:0/1635923546 shutdown_connections 2023-05-20T01:17:13.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.273+0000 7fcbee4b1700 1 --2- 172.21.15.16:0/1635923546 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcbe815ca90 0x7fcbe80b3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:13.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.273+0000 7fcbee4b1700 1 --2- 172.21.15.16:0/1635923546 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcbe80d2550 0x7fcbe815fba0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:13.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.273+0000 7fcbee4b1700 1 --2- 172.21.15.16:0/1635923546 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fcbc806ace0 0x7fcbc806d190 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:13.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.273+0000 7fcbee4b1700 1 --2- 172.21.15.16:0/1635923546 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcbe80d29a0 0x7fcbe814a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:13.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.273+0000 7fcbee4b1700 1 -- 172.21.15.16:0/1635923546 >> 172.21.15.16:0/1635923546 conn(0x7fcbe806b000 msgr2=0x7fcbe80b62a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:13.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.273+0000 7fcbee4b1700 1 -- 172.21.15.16:0/1635923546 shutdown_connections 2023-05-20T01:17:13.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.273+0000 7fcbee4b1700 1 -- 172.21.15.16:0/1635923546 wait complete. 2023-05-20T01:17:13.288 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-05-20T01:17:13.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.415+0000 7f64a689b700 1 Processor -- start 2023-05-20T01:17:13.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.415+0000 7f64a689b700 1 -- start start 2023-05-20T01:17:13.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.416+0000 7f64a689b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f64a015cec0 0x7f64a0071bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:13.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.416+0000 7f64a689b700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f64a015d290 0x7f64a0072130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:13.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.416+0000 7f64a689b700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64a00cd630 con 0x7f64a00cf2c0 2023-05-20T01:17:13.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.416+0000 7f64a689b700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64a00cca60 con 0x7f64a015cec0 2023-05-20T01:17:13.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.416+0000 7f64a689b700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64a00cfdd0 con 0x7f64a015d290 2023-05-20T01:17:13.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.416+0000 7f649ffff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f64a015cec0 0x7f64a0071bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:13.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.416+0000 7f64a4e38700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f64a00cf2c0 0x7f64a00cf6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:42322/0 (socket says 172.21.15.16:42322) 2023-05-20T01:17:13.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.416+0000 7f64a4e38700 1 -- 172.21.15.16:0/1384281892 learned_addr learned my addr 172.21.15.16:0/1384281892 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:17:13.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.416+0000 7f649f7fe700 1 --2- 172.21.15.16:0/1384281892 >> v2:172.21.15.132:3300/0 conn(0x7f64a015d290 0x7f64a0072130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:13.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.416+0000 7f649ffff700 1 -- 172.21.15.16:0/1384281892 >> v2:172.21.15.132:3300/0 conn(0x7f64a015d290 msgr2=0x7f64a0072130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:13.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.416+0000 7f649ffff700 1 --2- 172.21.15.16:0/1384281892 >> v2:172.21.15.132:3300/0 conn(0x7f64a015d290 0x7f64a0072130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:13.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.416+0000 7f649ffff700 1 -- 172.21.15.16:0/1384281892 >> v1:172.21.15.16:6789/0 conn(0x7f64a00cf2c0 legacy=0x7f64a00cf6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:13.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.416+0000 7f649effd700 1 -- 172.21.15.16:0/1384281892 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1935610262 0 0) 0x7f64a00cd630 con 0x7f64a00cf2c0 2023-05-20T01:17:13.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.416+0000 7f649ffff700 1 -- 172.21.15.16:0/1384281892 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64a00ccd00 con 0x7f64a015cec0 2023-05-20T01:17:13.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.416+0000 7f649ffff700 1 --2- 172.21.15.16:0/1384281892 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64a015cec0 0x7f64a0071bf0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f64a00cc970 tx=0x7f6490009b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ec1383ba3a20b14 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:13.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.417+0000 7f649effd700 1 -- 172.21.15.16:0/1384281892 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6490005e30 con 0x7f64a015cec0 2023-05-20T01:17:13.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.417+0000 7f649effd700 1 -- 172.21.15.16:0/1384281892 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6490006030 con 0x7f64a015cec0 2023-05-20T01:17:13.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.417+0000 7f649effd700 1 -- 172.21.15.16:0/1384281892 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64900063c0 con 0x7f64a015cec0 2023-05-20T01:17:13.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.417+0000 7f64a689b700 1 -- 172.21.15.16:0/1384281892 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64a015cec0 msgr2=0x7f64a0071bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:13.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.417+0000 7f64a689b700 1 --2- 172.21.15.16:0/1384281892 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64a015cec0 0x7f64a0071bf0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f64a00cc970 tx=0x7f6490009b00 comp rx=0 tx=0).stop 2023-05-20T01:17:13.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.417+0000 7f64a689b700 1 -- 172.21.15.16:0/1384281892 shutdown_connections 2023-05-20T01:17:13.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.417+0000 7f64a689b700 1 --2- 172.21.15.16:0/1384281892 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64a015cec0 0x7f64a0071bf0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:13.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.417+0000 7f64a689b700 1 --2- 172.21.15.16:0/1384281892 >> v2:172.21.15.132:3300/0 conn(0x7f64a015d290 0x7f64a0072130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:13.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.417+0000 7f64a689b700 1 -- 172.21.15.16:0/1384281892 >> 172.21.15.16:0/1384281892 conn(0x7f64a00cffd0 msgr2=0x7f64a00d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:13.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.417+0000 7f64a689b700 1 -- 172.21.15.16:0/1384281892 shutdown_connections 2023-05-20T01:17:13.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.417+0000 7f64a689b700 1 -- 172.21.15.16:0/1384281892 wait complete. 2023-05-20T01:17:13.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.418+0000 7f64a689b700 1 Processor -- start 2023-05-20T01:17:13.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.419+0000 7f64a689b700 1 -- start start 2023-05-20T01:17:13.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.419+0000 7f64a689b700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f64a00cf2c0 0x7f64a0150e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:13.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.419+0000 7f64a689b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f64a015cec0 0x7f64a00c8910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:13.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.419+0000 7f64a689b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64a015d290 0x7f64a015d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:13.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.419+0000 7f64a689b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64a0152970 con 0x7f64a015d290 2023-05-20T01:17:13.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.419+0000 7f64a689b700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64a01523b0 con 0x7f64a015cec0 2023-05-20T01:17:13.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.419+0000 7f64a689b700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64a0152690 con 0x7f64a00cf2c0 2023-05-20T01:17:13.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.419+0000 7f64a4e38700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64a015d290 0x7f64a015d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:13.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.419+0000 7f64a4e38700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64a015d290 0x7f64a015d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59032/0 (socket says 172.21.15.16:59032) 2023-05-20T01:17:13.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.419+0000 7f64a4e38700 1 -- 172.21.15.16:0/553631746 learned_addr learned my addr 172.21.15.16:0/553631746 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:13.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.419+0000 7f649f7fe700 1 --2- 172.21.15.16:0/553631746 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f64a015cec0 0x7f64a00c8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:13.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.419+0000 7f649ffff700 1 --2- 172.21.15.16:0/553631746 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f64a00cf2c0 0x7f64a0150e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:13.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.420+0000 7f64a4e38700 1 -- 172.21.15.16:0/553631746 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f64a00cf2c0 msgr2=0x7f64a0150e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:13.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.420+0000 7f64a4e38700 1 --2- 172.21.15.16:0/553631746 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f64a00cf2c0 0x7f64a0150e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:13.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.420+0000 7f64a4e38700 1 -- 172.21.15.16:0/553631746 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f64a015cec0 msgr2=0x7f64a00c8910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:13.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.420+0000 7f64a4e38700 1 --2- 172.21.15.16:0/553631746 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f64a015cec0 0x7f64a00c8910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:13.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.420+0000 7f64a4e38700 1 -- 172.21.15.16:0/553631746 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6490009430 con 0x7f64a015d290 2023-05-20T01:17:13.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.420+0000 7f64a4e38700 1 --2- 172.21.15.16:0/553631746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64a015d290 0x7f64a015d720 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f6498002230 tx=0x7f649800cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:13.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.420+0000 7f649cff9700 1 -- 172.21.15.16:0/553631746 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f649800aac0 con 0x7f64a015d290 2023-05-20T01:17:13.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.420+0000 7f649cff9700 1 -- 172.21.15.16:0/553631746 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f649800ac40 con 0x7f64a015d290 2023-05-20T01:17:13.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.420+0000 7f64a689b700 1 -- 172.21.15.16:0/553631746 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64a00cd2c0 con 0x7f64a015d290 2023-05-20T01:17:13.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.420+0000 7f649cff9700 1 -- 172.21.15.16:0/553631746 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64980070d0 con 0x7f64a015d290 2023-05-20T01:17:13.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.420+0000 7f64a689b700 1 -- 172.21.15.16:0/553631746 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64a00ccfe0 con 0x7f64a015d290 2023-05-20T01:17:13.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.421+0000 7f64a689b700 1 -- 172.21.15.16:0/553631746 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64a0160890 con 0x7f64a015d290 2023-05-20T01:17:13.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.421+0000 7f649cff9700 1 -- 172.21.15.16:0/553631746 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6498007250 con 0x7f64a015d290 2023-05-20T01:17:13.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.422+0000 7f649cff9700 1 --2- 172.21.15.16:0/553631746 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f648006ace0 0x7f648006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:13.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.422+0000 7f649cff9700 1 -- 172.21.15.16:0/553631746 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f64980894c0 con 0x7f64a015d290 2023-05-20T01:17:13.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.422+0000 7f649ffff700 1 --2- 172.21.15.16:0/553631746 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f648006ace0 0x7f648006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:13.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.422+0000 7f649ffff700 1 --2- 172.21.15.16:0/553631746 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f648006ace0 0x7f648006d190 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6490000c00 tx=0x7f6490008070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:13.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.426+0000 7f649cff9700 1 -- 172.21.15.16:0/553631746 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f6498054310 con 0x7f64a015d290 2023-05-20T01:17:13.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.622+0000 7f64a689b700 1 -- 172.21.15.16:0/553631746 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f64a00ce140 con 0x7f64a015d290 2023-05-20T01:17:13.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.622+0000 7f649cff9700 1 -- 172.21.15.16:0/553631746 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6498057930 con 0x7f64a015d290 2023-05-20T01:17:13.626 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2023-05-20T01:17:13.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.624+0000 7f64a689b700 1 -- 172.21.15.16:0/553631746 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f648006ace0 msgr2=0x7f648006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:13.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.624+0000 7f64a689b700 1 --2- 172.21.15.16:0/553631746 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f648006ace0 0x7f648006d190 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6490000c00 tx=0x7f6490008070 comp rx=0 tx=0).stop 2023-05-20T01:17:13.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.624+0000 7f64a689b700 1 -- 172.21.15.16:0/553631746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64a015d290 msgr2=0x7f64a015d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:13.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.624+0000 7f64a689b700 1 --2- 172.21.15.16:0/553631746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64a015d290 0x7f64a015d720 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f6498002230 tx=0x7f649800cf50 comp rx=0 tx=0).stop 2023-05-20T01:17:13.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.624+0000 7f64a689b700 1 -- 172.21.15.16:0/553631746 shutdown_connections 2023-05-20T01:17:13.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.624+0000 7f64a689b700 1 --2- 172.21.15.16:0/553631746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64a015d290 0x7f64a015d720 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:13.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.624+0000 7f64a689b700 1 --2- 172.21.15.16:0/553631746 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f648006ace0 0x7f648006d190 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:13.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.624+0000 7f64a689b700 1 --2- 172.21.15.16:0/553631746 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f64a00cf2c0 0x7f64a0150e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:13.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.624+0000 7f64a689b700 1 --2- 172.21.15.16:0/553631746 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f64a015cec0 0x7f64a00c8910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:13.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.624+0000 7f64a689b700 1 -- 172.21.15.16:0/553631746 >> 172.21.15.16:0/553631746 conn(0x7f64a00cffd0 msgr2=0x7f64a0071490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:13.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.624+0000 7f64a689b700 1 -- 172.21.15.16:0/553631746 shutdown_connections 2023-05-20T01:17:13.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.624+0000 7f64a689b700 1 -- 172.21.15.16:0/553631746 wait complete. 2023-05-20T01:17:13.640 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-05-20T01:17:13.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.765+0000 7ff1d077a700 1 Processor -- start 2023-05-20T01:17:13.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.765+0000 7ff1d077a700 1 -- start start 2023-05-20T01:17:13.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.766+0000 7ff1d077a700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff1c8072300 0x7ff1c80cf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:13.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.766+0000 7ff1d077a700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1c80cd630 con 0x7ff1c815cec0 2023-05-20T01:17:13.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.766+0000 7ff1d077a700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1c80cca60 con 0x7ff1c815d290 2023-05-20T01:17:13.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.766+0000 7ff1d077a700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1c80ccd00 con 0x7ff1c8072300 2023-05-20T01:17:13.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.766+0000 7ff1ce516700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ff1c815cec0 0x7ff1c8071bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:42332/0 (socket says 172.21.15.16:42332) 2023-05-20T01:17:13.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.766+0000 7ff1ce516700 1 -- 172.21.15.16:0/693510291 learned_addr learned my addr 172.21.15.16:0/693510291 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:17:13.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.767+0000 7ff1cdd15700 1 --2- 172.21.15.16:0/693510291 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff1c8072300 0x7ff1c80cf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:13.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.767+0000 7ff1cd514700 1 -- 172.21.15.16:0/693510291 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1323884001 0 0) 0x7ff1c80cd630 con 0x7ff1c815cec0 2023-05-20T01:17:13.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.767+0000 7ff1cd514700 1 -- 172.21.15.16:0/693510291 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1b40035b0 con 0x7ff1c815cec0 2023-05-20T01:17:13.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.767+0000 7ff1cd514700 1 -- 172.21.15.16:0/693510291 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2747305796 0 0) 0x7ff1b40035b0 con 0x7ff1c815cec0 2023-05-20T01:17:13.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.767+0000 7ff1cd514700 1 -- 172.21.15.16:0/693510291 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff1c8072300 msgr2=0x7ff1c80cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:13.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.767+0000 7ff1cd514700 1 --2- 172.21.15.16:0/693510291 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff1c8072300 0x7ff1c80cf2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:13.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.767+0000 7ff1cd514700 1 -- 172.21.15.16:0/693510291 >> v1:172.21.15.132:6789/0 conn(0x7ff1c815d290 legacy=0x7ff1c80cf800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:13.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.767+0000 7ff1cd514700 1 -- 172.21.15.16:0/693510291 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1c80ccfe0 con 0x7ff1c815cec0 2023-05-20T01:17:13.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.767+0000 7ff1cd514700 1 -- 172.21.15.16:0/693510291 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7ff1b80017f0 con 0x7ff1c815cec0 2023-05-20T01:17:13.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.767+0000 7ff1cd514700 1 -- 172.21.15.16:0/693510291 >> v1:172.21.15.16:6789/0 conn(0x7ff1c815cec0 legacy=0x7ff1c8071bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:13.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.767+0000 7ff1cd514700 1 --2- 172.21.15.16:0/693510291 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff1b4003a30 0x7ff1b4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:13.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.767+0000 7ff1cd514700 1 --2- 172.21.15.16:0/693510291 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1b40043d0 0x7ff1b4004890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:13.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.767+0000 7ff1cd514700 1 --2- 172.21.15.16:0/693510291 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff1c8072300 0x7ff1b4005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:13.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.767+0000 7ff1cd514700 1 -- 172.21.15.16:0/693510291 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1c80ccd00 con 0x7ff1b40043d0 2023-05-20T01:17:13.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.767+0000 7ff1cd514700 1 -- 172.21.15.16:0/693510291 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b400bac0 con 0x7ff1b4003a30 2023-05-20T01:17:13.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.767+0000 7ff1cd514700 1 -- 172.21.15.16:0/693510291 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b400bc50 con 0x7ff1c8072300 2023-05-20T01:17:13.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.767+0000 7ff1cdd15700 1 --2- 172.21.15.16:0/693510291 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1b40043d0 0x7ff1b4004890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:13.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.767+0000 7ff1ce516700 1 --2- 172.21.15.16:0/693510291 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff1b4003a30 0x7ff1b4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:13.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.767+0000 7ff1ced17700 1 --2- 172.21.15.16:0/693510291 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff1c8072300 0x7ff1b4005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:13.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.767+0000 7ff1cdd15700 1 -- 172.21.15.16:0/693510291 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff1c8072300 msgr2=0x7ff1b4005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:13.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.767+0000 7ff1cdd15700 1 --2- 172.21.15.16:0/693510291 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff1c8072300 0x7ff1b4005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:13.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.767+0000 7ff1cdd15700 1 -- 172.21.15.16:0/693510291 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff1b4003a30 msgr2=0x7ff1b4003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:13.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.767+0000 7ff1cdd15700 1 --2- 172.21.15.16:0/693510291 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff1b4003a30 0x7ff1b4003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:13.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.768+0000 7ff1cdd15700 1 -- 172.21.15.16:0/693510291 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1b40035b0 con 0x7ff1b40043d0 2023-05-20T01:17:13.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.768+0000 7ff1cdd15700 1 --2- 172.21.15.16:0/693510291 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1b40043d0 0x7ff1b4004890 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7ff1c4009b60 tx=0x7ff1b4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4199ea48b99c529 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:13.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.768+0000 7ff1cd514700 1 -- 172.21.15.16:0/693510291 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1c400e6c0 con 0x7ff1b40043d0 2023-05-20T01:17:13.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.768+0000 7ff1cd514700 1 -- 172.21.15.16:0/693510291 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1c400e820 con 0x7ff1b40043d0 2023-05-20T01:17:13.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.768+0000 7ff1cd514700 1 -- 172.21.15.16:0/693510291 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1c400eb90 con 0x7ff1b40043d0 2023-05-20T01:17:13.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.768+0000 7ff1d077a700 1 -- 172.21.15.16:0/693510291 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1b40043d0 msgr2=0x7ff1b4004890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:13.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.768+0000 7ff1d077a700 1 --2- 172.21.15.16:0/693510291 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1b40043d0 0x7ff1b4004890 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7ff1c4009b60 tx=0x7ff1b4005880 comp rx=0 tx=0).stop 2023-05-20T01:17:13.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.768+0000 7ff1ced17700 1 -- 172.21.15.16:0/693510291 reap_dead start 2023-05-20T01:17:13.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.768+0000 7ff1d077a700 1 -- 172.21.15.16:0/693510291 shutdown_connections 2023-05-20T01:17:13.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.768+0000 7ff1d077a700 1 -- 172.21.15.16:0/693510291 >> 172.21.15.16:0/693510291 conn(0x7ff1c80cffd0 msgr2=0x7ff1c80d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:13.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.768+0000 7ff1d077a700 1 -- 172.21.15.16:0/693510291 shutdown_connections 2023-05-20T01:17:13.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.769+0000 7ff1d077a700 1 -- 172.21.15.16:0/693510291 wait complete. 2023-05-20T01:17:13.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.770+0000 7ff1d077a700 1 Processor -- start 2023-05-20T01:17:13.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.770+0000 7ff1d077a700 1 -- start start 2023-05-20T01:17:13.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.770+0000 7ff1d077a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1b40043d0 0x7ff1c815b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:13.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.770+0000 7ff1d077a700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff1c80ae1a0 0x7ff1c80ae610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:13.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.770+0000 7ff1d077a700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff1c81e93e0 0x7ff1c81e9850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:13.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.770+0000 7ff1d077a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1c80aed10 con 0x7ff1b40043d0 2023-05-20T01:17:13.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.770+0000 7ff1d077a700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1c81e9d90 con 0x7ff1c80ae1a0 2023-05-20T01:17:13.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.770+0000 7ff1d077a700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1c80b38d0 con 0x7ff1c81e93e0 2023-05-20T01:17:13.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.771+0000 7ff1ce516700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1b40043d0 0x7ff1c815b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:13.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.771+0000 7ff1ce516700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1b40043d0 0x7ff1c815b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59048/0 (socket says 172.21.15.16:59048) 2023-05-20T01:17:13.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.771+0000 7ff1ce516700 1 -- 172.21.15.16:0/2131463325 learned_addr learned my addr 172.21.15.16:0/2131463325 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:13.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.771+0000 7ff1cdd15700 1 --2- 172.21.15.16:0/2131463325 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff1c80ae1a0 0x7ff1c80ae610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:13.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.771+0000 7ff1ced17700 1 --2- 172.21.15.16:0/2131463325 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff1c81e93e0 0x7ff1c81e9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:13.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.771+0000 7ff1ce516700 1 -- 172.21.15.16:0/2131463325 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff1c81e93e0 msgr2=0x7ff1c81e9850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:13.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.771+0000 7ff1ce516700 1 --2- 172.21.15.16:0/2131463325 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff1c81e93e0 0x7ff1c81e9850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:13.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.771+0000 7ff1ce516700 1 -- 172.21.15.16:0/2131463325 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff1c80ae1a0 msgr2=0x7ff1c80ae610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:13.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.771+0000 7ff1ce516700 1 --2- 172.21.15.16:0/2131463325 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff1c80ae1a0 0x7ff1c80ae610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:13.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.771+0000 7ff1ce516700 1 -- 172.21.15.16:0/2131463325 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1c80cf2c0 con 0x7ff1b40043d0 2023-05-20T01:17:13.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.771+0000 7ff1cdd15700 1 --2- 172.21.15.16:0/2131463325 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff1c80ae1a0 0x7ff1c80ae610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:17:13.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.771+0000 7ff1ced17700 1 --2- 172.21.15.16:0/2131463325 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff1c81e93e0 0x7ff1c81e9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:17:13.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.771+0000 7ff1ce516700 1 --2- 172.21.15.16:0/2131463325 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1b40043d0 0x7ff1c815b7b0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7ff1b800f7e0 tx=0x7ff1b800fba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:13.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.771+0000 7ff1beffd700 1 -- 172.21.15.16:0/2131463325 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1b80072e0 con 0x7ff1b40043d0 2023-05-20T01:17:13.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.771+0000 7ff1beffd700 1 -- 172.21.15.16:0/2131463325 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1b8007460 con 0x7ff1b40043d0 2023-05-20T01:17:13.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.771+0000 7ff1d077a700 1 -- 172.21.15.16:0/2131463325 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1c80cd2c0 con 0x7ff1b40043d0 2023-05-20T01:17:13.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.771+0000 7ff1beffd700 1 -- 172.21.15.16:0/2131463325 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1b8005830 con 0x7ff1b40043d0 2023-05-20T01:17:13.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.771+0000 7ff1d077a700 1 -- 172.21.15.16:0/2131463325 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1c80cf450 con 0x7ff1b40043d0 2023-05-20T01:17:13.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.772+0000 7ff1d077a700 1 -- 172.21.15.16:0/2131463325 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1c80b3bb0 con 0x7ff1b40043d0 2023-05-20T01:17:13.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.773+0000 7ff1beffd700 1 -- 172.21.15.16:0/2131463325 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff1b8005990 con 0x7ff1b40043d0 2023-05-20T01:17:13.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.773+0000 7ff1beffd700 1 --2- 172.21.15.16:0/2131463325 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff1ac06ace0 0x7ff1ac06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:13.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.773+0000 7ff1beffd700 1 -- 172.21.15.16:0/2131463325 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7ff1b8089610 con 0x7ff1b40043d0 2023-05-20T01:17:13.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.773+0000 7ff1cdd15700 1 --2- 172.21.15.16:0/2131463325 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff1ac06ace0 0x7ff1ac06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:13.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.773+0000 7ff1cdd15700 1 --2- 172.21.15.16:0/2131463325 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff1ac06ace0 0x7ff1ac06d190 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff1c40090b0 tx=0x7ff1c4008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:13.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.777+0000 7ff1beffd700 1 -- 172.21.15.16:0/2131463325 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7ff1b8057e60 con 0x7ff1b40043d0 2023-05-20T01:17:13.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.977+0000 7ff1d077a700 1 -- 172.21.15.16:0/2131463325 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7ff1c80747b0 con 0x7ff1b40043d0 2023-05-20T01:17:13.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.977+0000 7ff1beffd700 1 -- 172.21.15.16:0/2131463325 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7ff1b801a310 con 0x7ff1b40043d0 2023-05-20T01:17:13.982 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-05-20T01:17:13.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.980+0000 7ff1d077a700 1 -- 172.21.15.16:0/2131463325 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff1ac06ace0 msgr2=0x7ff1ac06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:13.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.980+0000 7ff1d077a700 1 --2- 172.21.15.16:0/2131463325 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff1ac06ace0 0x7ff1ac06d190 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff1c40090b0 tx=0x7ff1c4008030 comp rx=0 tx=0).stop 2023-05-20T01:17:13.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.980+0000 7ff1d077a700 1 -- 172.21.15.16:0/2131463325 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1b40043d0 msgr2=0x7ff1c815b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:13.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.980+0000 7ff1d077a700 1 --2- 172.21.15.16:0/2131463325 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1b40043d0 0x7ff1c815b7b0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7ff1b800f7e0 tx=0x7ff1b800fba0 comp rx=0 tx=0).stop 2023-05-20T01:17:13.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.980+0000 7ff1d077a700 1 -- 172.21.15.16:0/2131463325 shutdown_connections 2023-05-20T01:17:13.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.980+0000 7ff1d077a700 1 --2- 172.21.15.16:0/2131463325 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff1ac06ace0 0x7ff1ac06d190 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:13.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.980+0000 7ff1d077a700 1 --2- 172.21.15.16:0/2131463325 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff1c81e93e0 0x7ff1c81e9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:13.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.980+0000 7ff1d077a700 1 --2- 172.21.15.16:0/2131463325 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1b40043d0 0x7ff1c815b7b0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:13.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.980+0000 7ff1d077a700 1 --2- 172.21.15.16:0/2131463325 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff1c80ae1a0 0x7ff1c80ae610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:13.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.980+0000 7ff1d077a700 1 -- 172.21.15.16:0/2131463325 >> 172.21.15.16:0/2131463325 conn(0x7ff1c80cffd0 msgr2=0x7ff1c80ca600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:13.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.980+0000 7ff1d077a700 1 -- 172.21.15.16:0/2131463325 shutdown_connections 2023-05-20T01:17:13.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:13.980+0000 7ff1d077a700 1 -- 172.21.15.16:0/2131463325 wait complete. 2023-05-20T01:17:13.996 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-20T01:17:14.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.116+0000 7f8c03ccb700 1 Processor -- start 2023-05-20T01:17:14.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.116+0000 7f8c03ccb700 1 -- start start 2023-05-20T01:17:14.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.117+0000 7f8c03ccb700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f8bfc157190 0x7f8bfc0cc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:14.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.117+0000 7f8c03ccb700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bfc0cb590 con 0x7f8bfc071ad0 2023-05-20T01:17:14.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.117+0000 7f8c03ccb700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bfc0ca9a0 con 0x7f8bfc0cccf0 2023-05-20T01:17:14.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.117+0000 7f8c03ccb700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bfc071f50 con 0x7f8bfc157190 2023-05-20T01:17:14.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.117+0000 7f8c02268700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f8bfc071ad0 0x7f8bfc0ce740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:42354/0 (socket says 172.21.15.16:42354) 2023-05-20T01:17:14.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.117+0000 7f8c02268700 1 -- 172.21.15.16:0/1506416176 learned_addr learned my addr 172.21.15.16:0/1506416176 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:17:14.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.117+0000 7f8c00a65700 1 -- 172.21.15.16:0/1506416176 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3277092572 0 0) 0x7f8bfc0cb590 con 0x7f8bfc071ad0 2023-05-20T01:17:14.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.117+0000 7f8c00a65700 1 -- 172.21.15.16:0/1506416176 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8be80035b0 con 0x7f8bfc071ad0 2023-05-20T01:17:14.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.117+0000 7f8c01a67700 1 --2- 172.21.15.16:0/1506416176 >> v2:172.21.15.132:3300/0 conn(0x7f8bfc157190 0x7f8bfc0cc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:14.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.117+0000 7f8c00a65700 1 -- 172.21.15.16:0/1506416176 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3194928448 0 0) 0x7f8be80035b0 con 0x7f8bfc071ad0 2023-05-20T01:17:14.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.118+0000 7f8c00a65700 1 -- 172.21.15.16:0/1506416176 >> v2:172.21.15.132:3300/0 conn(0x7f8bfc157190 msgr2=0x7f8bfc0cc7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:14.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.118+0000 7f8c00a65700 1 --2- 172.21.15.16:0/1506416176 >> v2:172.21.15.132:3300/0 conn(0x7f8bfc157190 0x7f8bfc0cc7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:14.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.118+0000 7f8c00a65700 1 -- 172.21.15.16:0/1506416176 >> v1:172.21.15.132:6789/0 conn(0x7f8bfc0cccf0 legacy=0x7f8bfc0713c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:14.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.118+0000 7f8c00a65700 1 -- 172.21.15.16:0/1506416176 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bfc0cac60 con 0x7f8bfc071ad0 2023-05-20T01:17:14.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.118+0000 7f8c00a65700 1 -- 172.21.15.16:0/1506416176 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7f8bf40019f0 con 0x7f8bfc071ad0 2023-05-20T01:17:14.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.118+0000 7f8c00a65700 1 -- 172.21.15.16:0/1506416176 >> v1:172.21.15.16:6789/0 conn(0x7f8bfc071ad0 legacy=0x7f8bfc0ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:14.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.118+0000 7f8c00a65700 1 --2- 172.21.15.16:0/1506416176 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8be8003a70 0x7f8be8003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:14.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.118+0000 7f8c00a65700 1 --2- 172.21.15.16:0/1506416176 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8be8004850 0x7f8be8004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:14.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.118+0000 7f8c00a65700 1 --2- 172.21.15.16:0/1506416176 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8be8005200 0x7f8be800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:14.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.118+0000 7f8c00a65700 1 -- 172.21.15.16:0/1506416176 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bfc071f50 con 0x7f8be8005200 2023-05-20T01:17:14.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.118+0000 7f8c00a65700 1 -- 172.21.15.16:0/1506416176 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8be800c0e0 con 0x7f8be8004850 2023-05-20T01:17:14.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.118+0000 7f8c00a65700 1 -- 172.21.15.16:0/1506416176 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8be800c2a0 con 0x7f8be8003a70 2023-05-20T01:17:14.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.118+0000 7f8c02268700 1 --2- 172.21.15.16:0/1506416176 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8be8005200 0x7f8be800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:14.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.118+0000 7f8c01266700 1 --2- 172.21.15.16:0/1506416176 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8be8004850 0x7f8be8004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:14.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.118+0000 7f8c01a67700 1 --2- 172.21.15.16:0/1506416176 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8be8003a70 0x7f8be8003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:14.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.118+0000 7f8c02268700 1 -- 172.21.15.16:0/1506416176 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8be8003a70 msgr2=0x7f8be8003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:14.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.118+0000 7f8c02268700 1 --2- 172.21.15.16:0/1506416176 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8be8003a70 0x7f8be8003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:14.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.118+0000 7f8c02268700 1 -- 172.21.15.16:0/1506416176 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8be8004850 msgr2=0x7f8be8004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:14.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.118+0000 7f8c02268700 1 --2- 172.21.15.16:0/1506416176 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8be8004850 0x7f8be8004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:14.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.118+0000 7f8c02268700 1 -- 172.21.15.16:0/1506416176 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8be80035b0 con 0x7f8be8005200 2023-05-20T01:17:14.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.118+0000 7f8c02268700 1 -- 172.21.15.16:0/1506416176 reap_dead start 2023-05-20T01:17:14.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.118+0000 7f8c02268700 1 --2- 172.21.15.16:0/1506416176 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8be8005200 0x7f8be800ba10 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f8bf400a410 tx=0x7f8be8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e737f20777098bfa server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:14.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.118+0000 7f8c00a65700 1 -- 172.21.15.16:0/1506416176 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf4010690 con 0x7f8be8005200 2023-05-20T01:17:14.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.118+0000 7f8c00a65700 1 -- 172.21.15.16:0/1506416176 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bf40107f0 con 0x7f8be8005200 2023-05-20T01:17:14.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.118+0000 7f8c00a65700 1 -- 172.21.15.16:0/1506416176 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf4010ba0 con 0x7f8be8005200 2023-05-20T01:17:14.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.119+0000 7f8c03ccb700 1 -- 172.21.15.16:0/1506416176 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8be8005200 msgr2=0x7f8be800ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:14.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.119+0000 7f8c03ccb700 1 --2- 172.21.15.16:0/1506416176 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8be8005200 0x7f8be800ba10 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f8bf400a410 tx=0x7f8be8005880 comp rx=0 tx=0).stop 2023-05-20T01:17:14.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.119+0000 7f8c03ccb700 1 -- 172.21.15.16:0/1506416176 shutdown_connections 2023-05-20T01:17:14.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.119+0000 7f8c03ccb700 1 --2- 172.21.15.16:0/1506416176 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8be8005200 0x7f8be800ba10 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:14.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.119+0000 7f8c03ccb700 1 -- 172.21.15.16:0/1506416176 >> 172.21.15.16:0/1506416176 conn(0x7f8bfc0cffb0 msgr2=0x7f8bfc0d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:14.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.119+0000 7f8c03ccb700 1 -- 172.21.15.16:0/1506416176 shutdown_connections 2023-05-20T01:17:14.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.119+0000 7f8c03ccb700 1 -- 172.21.15.16:0/1506416176 wait complete. 2023-05-20T01:17:14.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.120+0000 7f8c03ccb700 1 Processor -- start 2023-05-20T01:17:14.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.120+0000 7f8c03ccb700 1 -- start start 2023-05-20T01:17:14.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.121+0000 7f8c03ccb700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8be8005200 0x7f8bfc1f2040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:14.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.121+0000 7f8c03ccb700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8bfc1f2580 0x7f8bfc0ac8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:14.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.121+0000 7f8c03ccb700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8bfc0acde0 0x7f8bfc0ad250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:14.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.121+0000 7f8c03ccb700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bfc0b2ee0 con 0x7f8be8005200 2023-05-20T01:17:14.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.121+0000 7f8c03ccb700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bfc1f29f0 con 0x7f8bfc1f2580 2023-05-20T01:17:14.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.121+0000 7f8c03ccb700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bfc0b3150 con 0x7f8bfc0acde0 2023-05-20T01:17:14.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.121+0000 7f8c01a67700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8be8005200 0x7f8bfc1f2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:14.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.121+0000 7f8c01a67700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8be8005200 0x7f8bfc1f2040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59066/0 (socket says 172.21.15.16:59066) 2023-05-20T01:17:14.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.121+0000 7f8c01a67700 1 -- 172.21.15.16:0/3205458542 learned_addr learned my addr 172.21.15.16:0/3205458542 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:14.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.121+0000 7f8c01266700 1 --2- 172.21.15.16:0/3205458542 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8bfc1f2580 0x7f8bfc0ac8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:14.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.121+0000 7f8c02268700 1 --2- 172.21.15.16:0/3205458542 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8bfc0acde0 0x7f8bfc0ad250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:14.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.122+0000 7f8c01a67700 1 -- 172.21.15.16:0/3205458542 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8bfc0acde0 msgr2=0x7f8bfc0ad250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:14.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.122+0000 7f8c01a67700 1 --2- 172.21.15.16:0/3205458542 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8bfc0acde0 0x7f8bfc0ad250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:14.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.122+0000 7f8c01a67700 1 -- 172.21.15.16:0/3205458542 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8bfc1f2580 msgr2=0x7f8bfc0ac8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:14.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.122+0000 7f8c01a67700 1 --2- 172.21.15.16:0/3205458542 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8bfc1f2580 0x7f8bfc0ac8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:14.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.122+0000 7f8c01a67700 1 -- 172.21.15.16:0/3205458542 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bfc0cb220 con 0x7f8be8005200 2023-05-20T01:17:14.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.122+0000 7f8c01a67700 1 --2- 172.21.15.16:0/3205458542 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8be8005200 0x7f8bfc1f2040 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f8bec0119a0 tx=0x7f8bec011cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:14.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.122+0000 7f8bf27fc700 1 -- 172.21.15.16:0/3205458542 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bec00f780 con 0x7f8be8005200 2023-05-20T01:17:14.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.122+0000 7f8bf27fc700 1 -- 172.21.15.16:0/3205458542 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bec00f8e0 con 0x7f8be8005200 2023-05-20T01:17:14.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.122+0000 7f8bf27fc700 1 -- 172.21.15.16:0/3205458542 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bec00fc70 con 0x7f8be8005200 2023-05-20T01:17:14.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.122+0000 7f8c03ccb700 1 -- 172.21.15.16:0/3205458542 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bfc0caf40 con 0x7f8be8005200 2023-05-20T01:17:14.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.122+0000 7f8c03ccb700 1 -- 172.21.15.16:0/3205458542 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bfc157560 con 0x7f8be8005200 2023-05-20T01:17:14.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.123+0000 7f8c03ccb700 1 -- 172.21.15.16:0/3205458542 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bfc0747b0 con 0x7f8be8005200 2023-05-20T01:17:14.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.123+0000 7f8bf27fc700 1 -- 172.21.15.16:0/3205458542 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8bec014440 con 0x7f8be8005200 2023-05-20T01:17:14.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.124+0000 7f8bf27fc700 1 --2- 172.21.15.16:0/3205458542 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f8be006ace0 0x7f8be006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:14.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.124+0000 7f8bf27fc700 1 -- 172.21.15.16:0/3205458542 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f8bec089c50 con 0x7f8be8005200 2023-05-20T01:17:14.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.124+0000 7f8c01266700 1 --2- 172.21.15.16:0/3205458542 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f8be006ace0 0x7f8be006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:14.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.124+0000 7f8c01266700 1 --2- 172.21.15.16:0/3205458542 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f8be006ace0 0x7f8be006d190 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc06e7d0 tx=0x7f8bf8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:14.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.128+0000 7f8bf27fc700 1 -- 172.21.15.16:0/3205458542 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f8bec08f020 con 0x7f8be8005200 2023-05-20T01:17:14.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.327+0000 7f8c03ccb700 1 -- 172.21.15.16:0/3205458542 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8bfc0ce120 con 0x7f8be8005200 2023-05-20T01:17:14.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.329+0000 7f8bf27fc700 1 -- 172.21.15.16:0/3205458542 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+20026 (secure 0 0 0) 0x7f8bec022090 con 0x7f8be8005200 2023-05-20T01:17:14.333 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:17:14.333 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"fsid":"c2245cc5-ecab-4087-a59b-bc81db901676","created":"2023-05-20T01:16:09.549230+0000","modified":"2023-05-20T01:16:32.768495+0000","last_up_change":"2023-05-20T01:16:19.788624+0000","last_in_change":"2023-05-20T01:16:12.092160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-05-20T01:16:20.934366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-05-20T01:16:21.372294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-05-20T01:16:26.208287+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-05-20T01:16:27.292268+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-05-20T01:16:30.357285+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"317a83cd-19c8-4958-a301-6087ef342289","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6825","nonce":3923634179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6827","nonce":3923634179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6831","nonce":3923634179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6829","nonce":3923634179}]},"public_addr":"172.21.15.16:6825/3923634179","cluster_addr":"172.21.15.16:6827/3923634179","heartbeat_back_addr":"172.21.15.16:6831/3923634179","heartbeat_front_addr":"172.21.15.16:6829/3923634179","state":["exists","up"]},{"osd":1,"uuid":"5581046f-8219-4089-8d7d-b7846399505e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6809","nonce":3860264220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6811","nonce":3860264220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6815","nonce":3860264220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6813","nonce":3860264220}]},"public_addr":"172.21.15.16:6809/3860264220","cluster_addr":"172.21.15.16:6811/3860264220","heartbeat_back_addr":"172.21.15.16:6815/3860264220","heartbeat_front_addr":"172.21.15.16:6813/3860264220","state":["exists","up"]},{"osd":2,"uuid":"072e9e66-4133-478e-8d4a-64d46c2963f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6817","nonce":293187030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6819","nonce":293187030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6823","nonce":293187030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6821","nonce":293187030}]},"public_addr":"172.21.15.16:6817/293187030","cluster_addr":"172.21.15.16:6819/293187030","heartbeat_back_addr":"172.21.15.16:6823/293187030","heartbeat_front_addr":"172.21.15.16:6821/293187030","state":["exists","up"]},{"osd":3,"uuid":"11e1bba6-00f8-48dd-9028-d8d716b7f145","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6801","nonce":3669781890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6803","nonce":3669781890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6807","nonce":3669781890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6805","nonce":3669781890}]},"public_addr":"172.21.15.16:6801/3669781890","cluster_addr":"172.21.15.16:6803/3669781890","heartbeat_back_addr":"172.21.15.16:6807/3669781890","heartbeat_front_addr":"172.21.15.16:6805/3669781890","state":["exists","up"]},{"osd":4,"uuid":"b8909c3a-cf70-46cb-8d0b-009c707cf9d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6819","nonce":506317223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6823","nonce":506317223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6829","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6831","nonce":506317223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6827","nonce":506317223}]},"public_addr":"172.21.15.132:6819/506317223","cluster_addr":"172.21.15.132:6823/506317223","heartbeat_back_addr":"172.21.15.132:6831/506317223","heartbeat_front_addr":"172.21.15.132:6827/506317223","state":["exists","up"]},{"osd":5,"uuid":"b1818304-4583-47bf-a6ec-0bfbb5166233","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6818","nonce":1844689995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6822","nonce":1844689995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6830","nonce":1844689995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6826","nonce":1844689995}]},"public_addr":"172.21.15.132:6818/1844689995","cluster_addr":"172.21.15.132:6822/1844689995","heartbeat_back_addr":"172.21.15.132:6830/1844689995","heartbeat_front_addr":"172.21.15.132:6826/1844689995","state":["exists","up"]},{"osd":6,"uuid":"d37f4691-eed1-4a93-8818-18354f03f4a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6801","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6802","nonce":727581123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6805","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6807","nonce":727581123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6813","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6815","nonce":727581123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6811","nonce":727581123}]},"public_addr":"172.21.15.132:6802/727581123","cluster_addr":"172.21.15.132:6807/727581123","heartbeat_back_addr":"172.21.15.132:6815/727581123","heartbeat_front_addr":"172.21.15.132:6811/727581123","state":["exists","up"]},{"osd":7,"uuid":"4f712f62-5d1d-4816-8042-03209144f8b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6803","nonce":466001149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6806","nonce":466001149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6814","nonce":466001149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6810","nonce":466001149}]},"public_addr":"172.21.15.132:6803/466001149","cluster_addr":"172.21.15.132:6806/466001149","heartbeat_back_addr":"172.21.15.132:6814/466001149","heartbeat_front_addr":"172.21.15.132:6810/466001149","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.338863+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.278680+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305694+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.246835+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.276237+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.304441+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305421+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.267050+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-20T01:17:14.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.331+0000 7f8c03ccb700 1 -- 172.21.15.16:0/3205458542 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f8be006ace0 msgr2=0x7f8be006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:14.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.331+0000 7f8c03ccb700 1 --2- 172.21.15.16:0/3205458542 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f8be006ace0 0x7f8be006d190 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc06e7d0 tx=0x7f8bf8009040 comp rx=0 tx=0).stop 2023-05-20T01:17:14.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.331+0000 7f8c03ccb700 1 -- 172.21.15.16:0/3205458542 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8be8005200 msgr2=0x7f8bfc1f2040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:14.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.331+0000 7f8c03ccb700 1 --2- 172.21.15.16:0/3205458542 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8be8005200 0x7f8bfc1f2040 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f8bec0119a0 tx=0x7f8bec011cb0 comp rx=0 tx=0).stop 2023-05-20T01:17:14.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.331+0000 7f8c03ccb700 1 -- 172.21.15.16:0/3205458542 shutdown_connections 2023-05-20T01:17:14.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.331+0000 7f8c03ccb700 1 --2- 172.21.15.16:0/3205458542 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f8be006ace0 0x7f8be006d190 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:14.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.331+0000 7f8c03ccb700 1 --2- 172.21.15.16:0/3205458542 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8bfc0acde0 0x7f8bfc0ad250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:14.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.331+0000 7f8c03ccb700 1 --2- 172.21.15.16:0/3205458542 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8be8005200 0x7f8bfc1f2040 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:14.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.331+0000 7f8c03ccb700 1 --2- 172.21.15.16:0/3205458542 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8bfc1f2580 0x7f8bfc0ac8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:14.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.331+0000 7f8c03ccb700 1 -- 172.21.15.16:0/3205458542 >> 172.21.15.16:0/3205458542 conn(0x7f8bfc0cffb0 msgr2=0x7f8bfc06ed20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:14.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.331+0000 7f8c03ccb700 1 -- 172.21.15.16:0/3205458542 shutdown_connections 2023-05-20T01:17:14.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.331+0000 7f8c03ccb700 1 -- 172.21.15.16:0/3205458542 wait complete. 2023-05-20T01:17:14.348 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-05-20T01:16:20.934366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-05-20T01:16:21.372294+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-05-20T01:16:26.208287+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-05-20T01:16:27.292268+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-05-20T01:16:30.357285+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-05-20T01:17:14.349 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-05-20T01:17:14.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.474+0000 7fb51a6fd700 1 Processor -- start 2023-05-20T01:17:14.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.474+0000 7fb51a6fd700 1 -- start start 2023-05-20T01:17:14.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.475+0000 7fb51a6fd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb514158f90 0x7fb5140cb7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:14.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.475+0000 7fb51a6fd700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fb5140ce0a0 0x7fb5140ce540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:14.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.475+0000 7fb51a6fd700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb514159360 con 0x7fb5140cc050 2023-05-20T01:17:14.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.475+0000 7fb51a6fd700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5140cce10 con 0x7fb514158f90 2023-05-20T01:17:14.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.475+0000 7fb51a6fd700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5140cea80 con 0x7fb5140ce0a0 2023-05-20T01:17:14.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.475+0000 7fb513fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb514158f90 0x7fb5140cb7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:14.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.475+0000 7fb518c9a700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fb5140cc050 0x7fb5140cc450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:42374/0 (socket says 172.21.15.16:42374) 2023-05-20T01:17:14.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.475+0000 7fb513fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb514158f90 0x7fb5140cb7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59074/0 (socket says 172.21.15.16:59074) 2023-05-20T01:17:14.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.475+0000 7fb518c9a700 1 -- 172.21.15.16:0/1397661400 learned_addr learned my addr 172.21.15.16:0/1397661400 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:17:14.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.475+0000 7fb5137fe700 1 --2- 172.21.15.16:0/1397661400 >> v2:172.21.15.132:3300/0 conn(0x7fb5140ce0a0 0x7fb5140ce540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:14.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.475+0000 7fb513fff700 1 -- 172.21.15.16:0/1397661400 >> v2:172.21.15.132:3300/0 conn(0x7fb5140ce0a0 msgr2=0x7fb5140ce540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:14.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.475+0000 7fb513fff700 1 --2- 172.21.15.16:0/1397661400 >> v2:172.21.15.132:3300/0 conn(0x7fb5140ce0a0 0x7fb5140ce540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:14.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.475+0000 7fb513fff700 1 -- 172.21.15.16:0/1397661400 >> v1:172.21.15.16:6789/0 conn(0x7fb5140cc050 legacy=0x7fb5140cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:14.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.475+0000 7fb513fff700 1 -- 172.21.15.16:0/1397661400 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5140ca750 con 0x7fb514158f90 2023-05-20T01:17:14.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.475+0000 7fb512ffd700 1 -- 172.21.15.16:0/1397661400 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3859481177 0 0) 0x7fb50c001490 con 0x7fb5140cc050 2023-05-20T01:17:14.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.475+0000 7fb5137fe700 1 --2- 172.21.15.16:0/1397661400 >> v2:172.21.15.132:3300/0 conn(0x7fb5140ce0a0 0x7fb5140ce540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:17:14.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.475+0000 7fb513fff700 1 --2- 172.21.15.16:0/1397661400 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb514158f90 0x7fb5140cb7b0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fb5140cd2e0 tx=0x7fb50400acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72fba79a83844059 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:14.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.475+0000 7fb512ffd700 1 -- 172.21.15.16:0/1397661400 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb50400cb70 con 0x7fb514158f90 2023-05-20T01:17:14.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.476+0000 7fb512ffd700 1 -- 172.21.15.16:0/1397661400 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb50400ccd0 con 0x7fb514158f90 2023-05-20T01:17:14.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.476+0000 7fb512ffd700 1 -- 172.21.15.16:0/1397661400 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb504018620 con 0x7fb514158f90 2023-05-20T01:17:14.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.476+0000 7fb51a6fd700 1 -- 172.21.15.16:0/1397661400 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb514158f90 msgr2=0x7fb5140cb7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:14.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.476+0000 7fb51a6fd700 1 --2- 172.21.15.16:0/1397661400 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb514158f90 0x7fb5140cb7b0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fb5140cd2e0 tx=0x7fb50400acc0 comp rx=0 tx=0).stop 2023-05-20T01:17:14.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.476+0000 7fb51a6fd700 1 -- 172.21.15.16:0/1397661400 shutdown_connections 2023-05-20T01:17:14.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.476+0000 7fb51a6fd700 1 --2- 172.21.15.16:0/1397661400 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb514158f90 0x7fb5140cb7b0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:14.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.476+0000 7fb51a6fd700 1 --2- 172.21.15.16:0/1397661400 >> v2:172.21.15.132:3300/0 conn(0x7fb5140ce0a0 0x7fb5140ce540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:14.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.476+0000 7fb51a6fd700 1 -- 172.21.15.16:0/1397661400 >> 172.21.15.16:0/1397661400 conn(0x7fb51406cbb0 msgr2=0x7fb51406cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:14.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.476+0000 7fb51a6fd700 1 -- 172.21.15.16:0/1397661400 shutdown_connections 2023-05-20T01:17:14.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.476+0000 7fb51a6fd700 1 -- 172.21.15.16:0/1397661400 wait complete. 2023-05-20T01:17:14.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.477+0000 7fb51a6fd700 1 Processor -- start 2023-05-20T01:17:14.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.477+0000 7fb51a6fd700 1 -- start start 2023-05-20T01:17:14.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.478+0000 7fb51a6fd700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb5140cc050 0x7fb5141f1a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:14.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.478+0000 7fb51a6fd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb5140ce0a0 0x7fb5141eae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:14.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.478+0000 7fb51a6fd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb514158f90 0x7fb514159d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:14.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.478+0000 7fb51a6fd700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb51415acd0 con 0x7fb514158f90 2023-05-20T01:17:14.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.478+0000 7fb51a6fd700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5141f1fd0 con 0x7fb5140ce0a0 2023-05-20T01:17:14.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.478+0000 7fb51a6fd700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb514071550 con 0x7fb5140cc050 2023-05-20T01:17:14.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.478+0000 7fb518c9a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb514158f90 0x7fb514159d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:14.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.478+0000 7fb518c9a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb514158f90 0x7fb514159d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59084/0 (socket says 172.21.15.16:59084) 2023-05-20T01:17:14.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.478+0000 7fb518c9a700 1 -- 172.21.15.16:0/2265992966 learned_addr learned my addr 172.21.15.16:0/2265992966 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:14.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.478+0000 7fb5137fe700 1 --2- 172.21.15.16:0/2265992966 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb5140ce0a0 0x7fb5141eae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:14.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.479+0000 7fb513fff700 1 --2- 172.21.15.16:0/2265992966 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb5140cc050 0x7fb5141f1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:14.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.479+0000 7fb518c9a700 1 -- 172.21.15.16:0/2265992966 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb5140cc050 msgr2=0x7fb5141f1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:14.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.479+0000 7fb518c9a700 1 --2- 172.21.15.16:0/2265992966 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb5140cc050 0x7fb5141f1a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:14.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.479+0000 7fb518c9a700 1 -- 172.21.15.16:0/2265992966 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb5140ce0a0 msgr2=0x7fb5141eae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:14.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.479+0000 7fb518c9a700 1 --2- 172.21.15.16:0/2265992966 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb5140ce0a0 0x7fb5141eae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:14.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.479+0000 7fb518c9a700 1 -- 172.21.15.16:0/2265992966 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb50400a640 con 0x7fb514158f90 2023-05-20T01:17:14.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.479+0000 7fb5137fe700 1 --2- 172.21.15.16:0/2265992966 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb5140ce0a0 0x7fb5141eae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:17:14.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.479+0000 7fb513fff700 1 --2- 172.21.15.16:0/2265992966 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb5140cc050 0x7fb5141f1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:17:14.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.479+0000 7fb518c9a700 1 --2- 172.21.15.16:0/2265992966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb514158f90 0x7fb514159d80 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fb50c00cca0 tx=0x7fb50c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:14.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.479+0000 7fb510ff9700 1 -- 172.21.15.16:0/2265992966 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb50c00cea0 con 0x7fb514158f90 2023-05-20T01:17:14.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.479+0000 7fb510ff9700 1 -- 172.21.15.16:0/2265992966 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb50c00ade0 con 0x7fb514158f90 2023-05-20T01:17:14.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.479+0000 7fb51a6fd700 1 -- 172.21.15.16:0/2265992966 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5140ca9f0 con 0x7fb514158f90 2023-05-20T01:17:14.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.479+0000 7fb510ff9700 1 -- 172.21.15.16:0/2265992966 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb50c007310 con 0x7fb514158f90 2023-05-20T01:17:14.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.479+0000 7fb51a6fd700 1 -- 172.21.15.16:0/2265992966 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5140ccb60 con 0x7fb514158f90 2023-05-20T01:17:14.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.480+0000 7fb51a6fd700 1 -- 172.21.15.16:0/2265992966 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5140cacd0 con 0x7fb514158f90 2023-05-20T01:17:14.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.480+0000 7fb510ff9700 1 -- 172.21.15.16:0/2265992966 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb50c007470 con 0x7fb514158f90 2023-05-20T01:17:14.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.481+0000 7fb510ff9700 1 --2- 172.21.15.16:0/2265992966 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb50006ace0 0x7fb50006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:14.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.481+0000 7fb510ff9700 1 -- 172.21.15.16:0/2265992966 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fb50c089230 con 0x7fb514158f90 2023-05-20T01:17:14.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.481+0000 7fb513fff700 1 --2- 172.21.15.16:0/2265992966 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb50006ace0 0x7fb50006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:14.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.481+0000 7fb513fff700 1 --2- 172.21.15.16:0/2265992966 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb50006ace0 0x7fb50006d190 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb5140cdc50 tx=0x7fb504012000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:14.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.485+0000 7fb510ff9700 1 -- 172.21.15.16:0/2265992966 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fb50c054080 con 0x7fb514158f90 2023-05-20T01:17:14.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.681+0000 7fb51a6fd700 1 -- 172.21.15.16:0/2265992966 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fb514159360 con 0x7fb514158f90 2023-05-20T01:17:14.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.682+0000 7fb510ff9700 1 -- 172.21.15.16:0/2265992966 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7fb514159360 con 0x7fb514158f90 2023-05-20T01:17:14.686 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 8 2023-05-20T01:17:14.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.684+0000 7fb51a6fd700 1 -- 172.21.15.16:0/2265992966 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb50006ace0 msgr2=0x7fb50006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:14.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.684+0000 7fb51a6fd700 1 --2- 172.21.15.16:0/2265992966 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb50006ace0 0x7fb50006d190 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb5140cdc50 tx=0x7fb504012000 comp rx=0 tx=0).stop 2023-05-20T01:17:14.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.684+0000 7fb51a6fd700 1 -- 172.21.15.16:0/2265992966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb514158f90 msgr2=0x7fb514159d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:14.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.684+0000 7fb51a6fd700 1 --2- 172.21.15.16:0/2265992966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb514158f90 0x7fb514159d80 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fb50c00cca0 tx=0x7fb50c00a340 comp rx=0 tx=0).stop 2023-05-20T01:17:14.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.684+0000 7fb51a6fd700 1 -- 172.21.15.16:0/2265992966 shutdown_connections 2023-05-20T01:17:14.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.684+0000 7fb51a6fd700 1 --2- 172.21.15.16:0/2265992966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb514158f90 0x7fb514159d80 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:14.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.684+0000 7fb51a6fd700 1 --2- 172.21.15.16:0/2265992966 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb50006ace0 0x7fb50006d190 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:14.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.684+0000 7fb51a6fd700 1 --2- 172.21.15.16:0/2265992966 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb5140cc050 0x7fb5141f1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:14.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.684+0000 7fb51a6fd700 1 --2- 172.21.15.16:0/2265992966 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb5140ce0a0 0x7fb5141eae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:14.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.684+0000 7fb51a6fd700 1 -- 172.21.15.16:0/2265992966 >> 172.21.15.16:0/2265992966 conn(0x7fb51406cbb0 msgr2=0x7fb51414eb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:14.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.684+0000 7fb51a6fd700 1 -- 172.21.15.16:0/2265992966 shutdown_connections 2023-05-20T01:17:14.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.684+0000 7fb51a6fd700 1 -- 172.21.15.16:0/2265992966 wait complete. 2023-05-20T01:17:14.699 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-05-20T01:17:14.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.823+0000 7f573254b700 1 Processor -- start 2023-05-20T01:17:14.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.823+0000 7f573254b700 1 -- start start 2023-05-20T01:17:14.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.824+0000 7f573254b700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f572c159b50 0x7f572c06e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:14.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.824+0000 7f573254b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f572c0cd1c0 0x7f572c0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:14.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.824+0000 7f573254b700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f572c0ccd30 con 0x7f572c0cd630 2023-05-20T01:17:14.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.824+0000 7f573254b700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f572c0ce990 con 0x7f572c0cd1c0 2023-05-20T01:17:14.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.824+0000 7f573254b700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f572c0cc6e0 con 0x7f572c159b50 2023-05-20T01:17:14.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.824+0000 7f572b7fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f572c0cd1c0 0x7f572c0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:14.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.824+0000 7f572b7fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f572c0cd1c0 0x7f572c0710b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59090/0 (socket says 172.21.15.16:59090) 2023-05-20T01:17:14.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.824+0000 7f572b7fe700 1 -- 172.21.15.16:0/316118635 learned_addr learned my addr 172.21.15.16:0/316118635 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:14.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.824+0000 7f572bfff700 1 --2- 172.21.15.16:0/316118635 >> v2:172.21.15.132:3300/0 conn(0x7f572c159b50 0x7f572c06e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:14.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.824+0000 7f572b7fe700 1 -- 172.21.15.16:0/316118635 >> v2:172.21.15.132:3300/0 conn(0x7f572c159b50 msgr2=0x7f572c06e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:14.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.824+0000 7f572b7fe700 1 --2- 172.21.15.16:0/316118635 >> v2:172.21.15.132:3300/0 conn(0x7f572c159b50 0x7f572c06e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:14.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.824+0000 7f572b7fe700 1 -- 172.21.15.16:0/316118635 >> v1:172.21.15.132:6789/0 conn(0x7f572c0cd630 legacy=0x7f572c0ca870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:14.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.824+0000 7f572b7fe700 1 -- 172.21.15.16:0/316118635 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f572c0cc9c0 con 0x7f572c0cd1c0 2023-05-20T01:17:14.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.825+0000 7f572b7fe700 1 --2- 172.21.15.16:0/316118635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f572c0cd1c0 0x7f572c0710b0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f572c079820 tx=0x7f571c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9f9ee23c71e606d server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:14.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.825+0000 7f572affd700 1 -- 172.21.15.16:0/316118635 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f571c00b870 con 0x7f572c0cd1c0 2023-05-20T01:17:14.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.825+0000 7f572affd700 1 -- 172.21.15.16:0/316118635 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f571c00b9d0 con 0x7f572c0cd1c0 2023-05-20T01:17:14.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.825+0000 7f572affd700 1 -- 172.21.15.16:0/316118635 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f571c00bd60 con 0x7f572c0cd1c0 2023-05-20T01:17:14.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.825+0000 7f573254b700 1 -- 172.21.15.16:0/316118635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f572c0cd1c0 msgr2=0x7f572c0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:14.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.825+0000 7f573254b700 1 --2- 172.21.15.16:0/316118635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f572c0cd1c0 0x7f572c0710b0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f572c079820 tx=0x7f571c009bd0 comp rx=0 tx=0).stop 2023-05-20T01:17:14.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.825+0000 7f573254b700 1 -- 172.21.15.16:0/316118635 shutdown_connections 2023-05-20T01:17:14.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.825+0000 7f573254b700 1 --2- 172.21.15.16:0/316118635 >> v2:172.21.15.132:3300/0 conn(0x7f572c159b50 0x7f572c06e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:14.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.825+0000 7f573254b700 1 --2- 172.21.15.16:0/316118635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f572c0cd1c0 0x7f572c0710b0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:14.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.825+0000 7f573254b700 1 -- 172.21.15.16:0/316118635 >> 172.21.15.16:0/316118635 conn(0x7f572c0cffd0 msgr2=0x7f572c0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:14.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.825+0000 7f573254b700 1 -- 172.21.15.16:0/316118635 shutdown_connections 2023-05-20T01:17:14.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.825+0000 7f573254b700 1 -- 172.21.15.16:0/316118635 wait complete. 2023-05-20T01:17:14.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.827+0000 7f573254b700 1 Processor -- start 2023-05-20T01:17:14.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.827+0000 7f573254b700 1 -- start start 2023-05-20T01:17:14.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.828+0000 7f573254b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f572c0cd1c0 0x7f572c158870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:14.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.828+0000 7f573254b700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f572c0cd630 0x7f572c1f2690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:14.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.828+0000 7f573254b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f572c159b50 0x7f572c1e5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:14.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.828+0000 7f573254b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f572c144f20 con 0x7f572c159b50 2023-05-20T01:17:14.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.828+0000 7f573254b700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f572c1454e0 con 0x7f572c0cd1c0 2023-05-20T01:17:14.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.828+0000 7f573254b700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f572c1f3110 con 0x7f572c0cd630 2023-05-20T01:17:14.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.828+0000 7f5730ae8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f572c159b50 0x7f572c1e5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:14.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.828+0000 7f5730ae8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f572c159b50 0x7f572c1e5490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59098/0 (socket says 172.21.15.16:59098) 2023-05-20T01:17:14.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.828+0000 7f5730ae8700 1 -- 172.21.15.16:0/1159251327 learned_addr learned my addr 172.21.15.16:0/1159251327 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:14.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.828+0000 7f572bfff700 1 --2- 172.21.15.16:0/1159251327 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f572c0cd1c0 0x7f572c158870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:14.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.828+0000 7f572b7fe700 1 --2- 172.21.15.16:0/1159251327 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f572c0cd630 0x7f572c1f2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:14.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.829+0000 7f5730ae8700 1 -- 172.21.15.16:0/1159251327 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f572c0cd630 msgr2=0x7f572c1f2690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:14.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.829+0000 7f5730ae8700 1 --2- 172.21.15.16:0/1159251327 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f572c0cd630 0x7f572c1f2690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:14.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.829+0000 7f5730ae8700 1 -- 172.21.15.16:0/1159251327 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f572c0cd1c0 msgr2=0x7f572c158870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:14.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.829+0000 7f5730ae8700 1 --2- 172.21.15.16:0/1159251327 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f572c0cd1c0 0x7f572c158870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:14.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.829+0000 7f5730ae8700 1 -- 172.21.15.16:0/1159251327 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f571c009550 con 0x7f572c159b50 2023-05-20T01:17:14.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.829+0000 7f5730ae8700 1 --2- 172.21.15.16:0/1159251327 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f572c159b50 0x7f572c1e5490 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f572400cc80 tx=0x7f572400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:14.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.829+0000 7f5728ff9700 1 -- 172.21.15.16:0/1159251327 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5724003c00 con 0x7f572c159b50 2023-05-20T01:17:14.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.829+0000 7f5728ff9700 1 -- 172.21.15.16:0/1159251327 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5724003d60 con 0x7f572c159b50 2023-05-20T01:17:14.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.829+0000 7f573254b700 1 -- 172.21.15.16:0/1159251327 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f572c071e80 con 0x7f572c159b50 2023-05-20T01:17:14.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.829+0000 7f5728ff9700 1 -- 172.21.15.16:0/1159251327 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f572400aaa0 con 0x7f572c159b50 2023-05-20T01:17:14.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.829+0000 7f573254b700 1 -- 172.21.15.16:0/1159251327 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f572c06e5e0 con 0x7f572c159b50 2023-05-20T01:17:14.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.830+0000 7f573254b700 1 -- 172.21.15.16:0/1159251327 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f572c1f3630 con 0x7f572c159b50 2023-05-20T01:17:14.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.830+0000 7f5728ff9700 1 -- 172.21.15.16:0/1159251327 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f572400ac00 con 0x7f572c159b50 2023-05-20T01:17:14.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.831+0000 7f5728ff9700 1 --2- 172.21.15.16:0/1159251327 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f571806ace0 0x7f571806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:14.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.831+0000 7f5728ff9700 1 -- 172.21.15.16:0/1159251327 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f5724089720 con 0x7f572c159b50 2023-05-20T01:17:14.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.831+0000 7f572bfff700 1 --2- 172.21.15.16:0/1159251327 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f571806ace0 0x7f571806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:14.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.831+0000 7f572bfff700 1 --2- 172.21.15.16:0/1159251327 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f571806ace0 0x7f571806d190 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5720006fd0 tx=0x7f5720008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:14.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:14.835+0000 7f5728ff9700 1 -- 172.21.15.16:0/1159251327 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f57240544e0 con 0x7f572c159b50 2023-05-20T01:17:15.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.033+0000 7f573254b700 1 -- 172.21.15.16:0/1159251327 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f572c0cec70 con 0x7f572c159b50 2023-05-20T01:17:15.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.034+0000 7f5728ff9700 1 -- 172.21.15.16:0/1159251327 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5724057b00 con 0x7f572c159b50 2023-05-20T01:17:15.038 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-05-20T01:17:15.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.036+0000 7f573254b700 1 -- 172.21.15.16:0/1159251327 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f571806ace0 msgr2=0x7f571806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:15.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.036+0000 7f573254b700 1 --2- 172.21.15.16:0/1159251327 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f571806ace0 0x7f571806d190 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5720006fd0 tx=0x7f5720008040 comp rx=0 tx=0).stop 2023-05-20T01:17:15.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.036+0000 7f573254b700 1 -- 172.21.15.16:0/1159251327 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f572c159b50 msgr2=0x7f572c1e5490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:15.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.036+0000 7f573254b700 1 --2- 172.21.15.16:0/1159251327 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f572c159b50 0x7f572c1e5490 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f572400cc80 tx=0x7f572400cf90 comp rx=0 tx=0).stop 2023-05-20T01:17:15.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.036+0000 7f573254b700 1 -- 172.21.15.16:0/1159251327 shutdown_connections 2023-05-20T01:17:15.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.036+0000 7f573254b700 1 --2- 172.21.15.16:0/1159251327 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f572c159b50 0x7f572c1e5490 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:15.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.036+0000 7f573254b700 1 --2- 172.21.15.16:0/1159251327 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f572c0cd1c0 0x7f572c158870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:15.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.036+0000 7f573254b700 1 --2- 172.21.15.16:0/1159251327 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f571806ace0 0x7f571806d190 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:15.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.036+0000 7f573254b700 1 --2- 172.21.15.16:0/1159251327 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f572c0cd630 0x7f572c1f2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:15.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.036+0000 7f573254b700 1 -- 172.21.15.16:0/1159251327 >> 172.21.15.16:0/1159251327 conn(0x7f572c0cffd0 msgr2=0x7f572c0c5af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:15.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.036+0000 7f573254b700 1 -- 172.21.15.16:0/1159251327 shutdown_connections 2023-05-20T01:17:15.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.036+0000 7f573254b700 1 -- 172.21.15.16:0/1159251327 wait complete. 2023-05-20T01:17:15.051 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-05-20T01:17:15.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.176+0000 7f6e4ad1e700 1 Processor -- start 2023-05-20T01:17:15.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.176+0000 7f6e4ad1e700 1 -- start start 2023-05-20T01:17:15.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.177+0000 7f6e4ad1e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6e44159b50 0x7f6e4406e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:15.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.178+0000 7f6e4ad1e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f6e440cd1c0 0x7f6e440710b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:15.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.178+0000 7f6e4ad1e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e440ccd30 con 0x7f6e440cd630 2023-05-20T01:17:15.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.178+0000 7f6e4ad1e700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e440ce990 con 0x7f6e440cd1c0 2023-05-20T01:17:15.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.178+0000 7f6e4ad1e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e440cc6e0 con 0x7f6e44159b50 2023-05-20T01:17:15.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.178+0000 7f6e43fff700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f6e440cd1c0 0x7f6e440710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:15.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.178+0000 7f6e48aba700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6e44159b50 0x7f6e4406e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:15.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.178+0000 7f6e492bb700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f6e440cd630 0x7f6e440ca870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.16:52116/0 (socket says 172.21.15.16:52116) 2023-05-20T01:17:15.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.178+0000 7f6e43fff700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f6e440cd1c0 0x7f6e440710b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.16:56126/0 (socket says 172.21.15.16:56126) 2023-05-20T01:17:15.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.178+0000 7f6e48aba700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6e44159b50 0x7f6e4406e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.16:59140/0 (socket says 172.21.15.16:59140) 2023-05-20T01:17:15.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.178+0000 7f6e492bb700 1 -- 172.21.15.16:0/2308563317 learned_addr learned my addr 172.21.15.16:0/2308563317 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:17:15.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.178+0000 7f6e43fff700 1 -- 172.21.15.16:0/2308563317 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6e44159b50 msgr2=0x7f6e4406e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:15.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.178+0000 7f6e43fff700 1 --2- 172.21.15.16:0/2308563317 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6e44159b50 0x7f6e4406e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:15.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.178+0000 7f6e43fff700 1 -- 172.21.15.16:0/2308563317 >> v1:172.21.15.132:6789/0 conn(0x7f6e440cd630 legacy=0x7f6e440ca870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:15.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.178+0000 7f6e43fff700 1 -- 172.21.15.16:0/2308563317 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e440cc9c0 con 0x7f6e440cd1c0 2023-05-20T01:17:15.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.178+0000 7f6e437fe700 1 -- 172.21.15.16:0/2308563317 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 50032536 0 0) 0x7f6e440ccd30 con 0x7f6e440cd630 2023-05-20T01:17:15.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.178+0000 7f6e48aba700 1 --2- 172.21.15.16:0/2308563317 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6e44159b50 0x7f6e4406e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-05-20T01:17:15.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.179+0000 7f6e43fff700 1 --2- 172.21.15.16:0/2308563317 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6e440cd1c0 0x7f6e440710b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6e30009840 tx=0x7f6e30009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92fccfb75b6153dc server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:15.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.179+0000 7f6e437fe700 1 -- 172.21.15.16:0/2308563317 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e30011830 con 0x7f6e440cd1c0 2023-05-20T01:17:15.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.179+0000 7f6e437fe700 1 -- 172.21.15.16:0/2308563317 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e30011990 con 0x7f6e440cd1c0 2023-05-20T01:17:15.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.179+0000 7f6e437fe700 1 -- 172.21.15.16:0/2308563317 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e30011d20 con 0x7f6e440cd1c0 2023-05-20T01:17:15.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.179+0000 7f6e4ad1e700 1 -- 172.21.15.16:0/2308563317 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6e440cd1c0 msgr2=0x7f6e440710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:15.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.179+0000 7f6e4ad1e700 1 --2- 172.21.15.16:0/2308563317 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6e440cd1c0 0x7f6e440710b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6e30009840 tx=0x7f6e30009b50 comp rx=0 tx=0).stop 2023-05-20T01:17:15.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.179+0000 7f6e4ad1e700 1 -- 172.21.15.16:0/2308563317 shutdown_connections 2023-05-20T01:17:15.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.179+0000 7f6e4ad1e700 1 --2- 172.21.15.16:0/2308563317 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6e44159b50 0x7f6e4406e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:15.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.179+0000 7f6e4ad1e700 1 --2- 172.21.15.16:0/2308563317 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6e440cd1c0 0x7f6e440710b0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:15.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.179+0000 7f6e4ad1e700 1 -- 172.21.15.16:0/2308563317 >> 172.21.15.16:0/2308563317 conn(0x7f6e440cffd0 msgr2=0x7f6e440d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:15.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.179+0000 7f6e4ad1e700 1 -- 172.21.15.16:0/2308563317 shutdown_connections 2023-05-20T01:17:15.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.179+0000 7f6e4ad1e700 1 -- 172.21.15.16:0/2308563317 wait complete. 2023-05-20T01:17:15.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.181+0000 7f6e4ad1e700 1 Processor -- start 2023-05-20T01:17:15.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.181+0000 7f6e4ad1e700 1 -- start start 2023-05-20T01:17:15.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.182+0000 7f6e4ad1e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6e440cd1c0 0x7f6e440c9de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:15.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.182+0000 7f6e4ad1e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6e440cd630 0x7f6e440c5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:15.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.182+0000 7f6e4ad1e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e44159b50 0x7f6e44155240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:15.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.182+0000 7f6e4ad1e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e440c5850 con 0x7f6e44159b50 2023-05-20T01:17:15.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.182+0000 7f6e4ad1e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e440c4c40 con 0x7f6e440cd630 2023-05-20T01:17:15.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.182+0000 7f6e4ad1e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e440c4f20 con 0x7f6e440cd1c0 2023-05-20T01:17:15.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.182+0000 7f6e492bb700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e44159b50 0x7f6e44155240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:15.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.182+0000 7f6e492bb700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e44159b50 0x7f6e44155240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59112/0 (socket says 172.21.15.16:59112) 2023-05-20T01:17:15.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.182+0000 7f6e492bb700 1 -- 172.21.15.16:0/1417730855 learned_addr learned my addr 172.21.15.16:0/1417730855 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:15.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.182+0000 7f6e48aba700 1 --2- 172.21.15.16:0/1417730855 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6e440cd1c0 0x7f6e440c9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:15.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.182+0000 7f6e43fff700 1 --2- 172.21.15.16:0/1417730855 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6e440cd630 0x7f6e440c5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:15.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.183+0000 7f6e492bb700 1 -- 172.21.15.16:0/1417730855 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6e440cd1c0 msgr2=0x7f6e440c9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:15.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.183+0000 7f6e492bb700 1 --2- 172.21.15.16:0/1417730855 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6e440cd1c0 0x7f6e440c9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:15.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.183+0000 7f6e492bb700 1 -- 172.21.15.16:0/1417730855 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6e440cd630 msgr2=0x7f6e440c5ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:15.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.183+0000 7f6e492bb700 1 --2- 172.21.15.16:0/1417730855 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6e440cd630 0x7f6e440c5ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:15.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.183+0000 7f6e492bb700 1 -- 172.21.15.16:0/1417730855 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e38009430 con 0x7f6e44159b50 2023-05-20T01:17:15.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.183+0000 7f6e492bb700 1 --2- 172.21.15.16:0/1417730855 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e44159b50 0x7f6e44155240 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c002230 tx=0x7f6e3c00cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:15.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.183+0000 7f6e417fa700 1 -- 172.21.15.16:0/1417730855 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e3c00aac0 con 0x7f6e44159b50 2023-05-20T01:17:15.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.183+0000 7f6e417fa700 1 -- 172.21.15.16:0/1417730855 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e3c00ac40 con 0x7f6e44159b50 2023-05-20T01:17:15.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.183+0000 7f6e4ad1e700 1 -- 172.21.15.16:0/1417730855 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e300094a0 con 0x7f6e44159b50 2023-05-20T01:17:15.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.183+0000 7f6e417fa700 1 -- 172.21.15.16:0/1417730855 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e3c0070d0 con 0x7f6e44159b50 2023-05-20T01:17:15.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.183+0000 7f6e4ad1e700 1 -- 172.21.15.16:0/1417730855 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e44071e80 con 0x7f6e44159b50 2023-05-20T01:17:15.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.184+0000 7f6e4ad1e700 1 -- 172.21.15.16:0/1417730855 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e4406e5e0 con 0x7f6e44159b50 2023-05-20T01:17:15.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.184+0000 7f6e417fa700 1 -- 172.21.15.16:0/1417730855 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6e3c007250 con 0x7f6e44159b50 2023-05-20T01:17:15.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.185+0000 7f6e417fa700 1 --2- 172.21.15.16:0/1417730855 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f6e2806ace0 0x7f6e2806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:15.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.185+0000 7f6e417fa700 1 -- 172.21.15.16:0/1417730855 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f6e3c0894c0 con 0x7f6e44159b50 2023-05-20T01:17:15.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.185+0000 7f6e48aba700 1 --2- 172.21.15.16:0/1417730855 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f6e2806ace0 0x7f6e2806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:15.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.185+0000 7f6e48aba700 1 --2- 172.21.15.16:0/1417730855 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f6e2806ace0 0x7f6e2806d190 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6e38009cb0 tx=0x7f6e380091c0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:15.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.189+0000 7f6e417fa700 1 -- 172.21.15.16:0/1417730855 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f6e3c054310 con 0x7f6e44159b50 2023-05-20T01:17:15.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.388+0000 7f6e4ad1e700 1 -- 172.21.15.16:0/1417730855 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6e440cec70 con 0x7f6e44159b50 2023-05-20T01:17:15.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.388+0000 7f6e417fa700 1 -- 172.21.15.16:0/1417730855 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6e3c057930 con 0x7f6e44159b50 2023-05-20T01:17:15.392 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-05-20T01:17:15.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.390+0000 7f6e4ad1e700 1 -- 172.21.15.16:0/1417730855 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f6e2806ace0 msgr2=0x7f6e2806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:15.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.390+0000 7f6e4ad1e700 1 --2- 172.21.15.16:0/1417730855 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f6e2806ace0 0x7f6e2806d190 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6e38009cb0 tx=0x7f6e380091c0 comp rx=0 tx=0).stop 2023-05-20T01:17:15.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.390+0000 7f6e4ad1e700 1 -- 172.21.15.16:0/1417730855 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e44159b50 msgr2=0x7f6e44155240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:15.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.390+0000 7f6e4ad1e700 1 --2- 172.21.15.16:0/1417730855 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e44159b50 0x7f6e44155240 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c002230 tx=0x7f6e3c00cf50 comp rx=0 tx=0).stop 2023-05-20T01:17:15.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.390+0000 7f6e4ad1e700 1 -- 172.21.15.16:0/1417730855 shutdown_connections 2023-05-20T01:17:15.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.390+0000 7f6e4ad1e700 1 --2- 172.21.15.16:0/1417730855 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e44159b50 0x7f6e44155240 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:15.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.390+0000 7f6e4ad1e700 1 --2- 172.21.15.16:0/1417730855 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f6e2806ace0 0x7f6e2806d190 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:15.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.390+0000 7f6e4ad1e700 1 --2- 172.21.15.16:0/1417730855 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6e440cd1c0 0x7f6e440c9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:15.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.390+0000 7f6e4ad1e700 1 --2- 172.21.15.16:0/1417730855 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6e440cd630 0x7f6e440c5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:15.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.390+0000 7f6e4ad1e700 1 -- 172.21.15.16:0/1417730855 >> 172.21.15.16:0/1417730855 conn(0x7f6e440cffd0 msgr2=0x7f6e440b4e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:15.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.390+0000 7f6e4ad1e700 1 -- 172.21.15.16:0/1417730855 shutdown_connections 2023-05-20T01:17:15.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.391+0000 7f6e4ad1e700 1 -- 172.21.15.16:0/1417730855 wait complete. 2023-05-20T01:17:15.407 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-05-20T01:17:15.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.535+0000 7f6c3125c700 1 Processor -- start 2023-05-20T01:17:15.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.535+0000 7f6c3125c700 1 -- start start 2023-05-20T01:17:15.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.535+0000 7f6c3125c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6c2c0717c0 0x7f6c2c0ca870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:15.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.535+0000 7f6c3125c700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c2c0ccd30 con 0x7f6c2c0cd1c0 2023-05-20T01:17:15.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.535+0000 7f6c3125c700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c2c0ce990 con 0x7f6c2c159b50 2023-05-20T01:17:15.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.535+0000 7f6c3125c700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c2c0cc6e0 con 0x7f6c2c0717c0 2023-05-20T01:17:15.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.536+0000 7f6c2a59c700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f6c2c0cd1c0 0x7f6c2c0710b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:42418/0 (socket says 172.21.15.16:42418) 2023-05-20T01:17:15.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.536+0000 7f6c2b59e700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6c2c0717c0 0x7f6c2c0ca870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:15.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.536+0000 7f6c2a59c700 1 -- 172.21.15.16:0/5362085 learned_addr learned my addr 172.21.15.16:0/5362085 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:17:15.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.536+0000 7f6c29d9b700 1 -- 172.21.15.16:0/5362085 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3458261703 0 0) 0x7f6c2c0ccd30 con 0x7f6c2c0cd1c0 2023-05-20T01:17:15.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.536+0000 7f6c29d9b700 1 -- 172.21.15.16:0/5362085 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c180035b0 con 0x7f6c2c0cd1c0 2023-05-20T01:17:15.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.536+0000 7f6c2b59e700 1 -- 172.21.15.16:0/5362085 >> v1:172.21.15.132:6789/0 conn(0x7f6c2c159b50 legacy=0x7f6c2c06e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:15.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.536+0000 7f6c2b59e700 1 -- 172.21.15.16:0/5362085 >> v1:172.21.15.16:6789/0 conn(0x7f6c2c0cd1c0 legacy=0x7f6c2c0710b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:15.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.536+0000 7f6c2b59e700 1 -- 172.21.15.16:0/5362085 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c2c0cc9c0 con 0x7f6c2c0717c0 2023-05-20T01:17:15.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.536+0000 7f6c2b59e700 1 --2- 172.21.15.16:0/5362085 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c2c0717c0 0x7f6c2c0ca870 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f6c24009d90 tx=0x7f6c2400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfe0ca4b7e81ee55 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:15.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.536+0000 7f6c29d9b700 1 -- 172.21.15.16:0/5362085 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c2400e070 con 0x7f6c2c0717c0 2023-05-20T01:17:15.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.536+0000 7f6c29d9b700 1 -- 172.21.15.16:0/5362085 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c2400bb20 con 0x7f6c2c0717c0 2023-05-20T01:17:15.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.536+0000 7f6c29d9b700 1 -- 172.21.15.16:0/5362085 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c24012430 con 0x7f6c2c0717c0 2023-05-20T01:17:15.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.537+0000 7f6c3125c700 1 -- 172.21.15.16:0/5362085 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c2c0717c0 msgr2=0x7f6c2c0ca870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:15.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.537+0000 7f6c3125c700 1 --2- 172.21.15.16:0/5362085 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c2c0717c0 0x7f6c2c0ca870 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f6c24009d90 tx=0x7f6c2400b330 comp rx=0 tx=0).stop 2023-05-20T01:17:15.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.537+0000 7f6c3125c700 1 -- 172.21.15.16:0/5362085 shutdown_connections 2023-05-20T01:17:15.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.537+0000 7f6c3125c700 1 --2- 172.21.15.16:0/5362085 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c2c0717c0 0x7f6c2c0ca870 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:15.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.537+0000 7f6c3125c700 1 -- 172.21.15.16:0/5362085 >> 172.21.15.16:0/5362085 conn(0x7f6c2c0cffd0 msgr2=0x7f6c2c0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:15.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.537+0000 7f6c3125c700 1 -- 172.21.15.16:0/5362085 shutdown_connections 2023-05-20T01:17:15.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.537+0000 7f6c3125c700 1 -- 172.21.15.16:0/5362085 wait complete. 2023-05-20T01:17:15.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.538+0000 7f6c3125c700 1 Processor -- start 2023-05-20T01:17:15.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.538+0000 7f6c3125c700 1 -- start start 2023-05-20T01:17:15.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.539+0000 7f6c3125c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c2c0717c0 0x7f6c2c14f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:15.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.539+0000 7f6c3125c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6c2c0cd1c0 0x7f6c2c06a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:15.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.539+0000 7f6c3125c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6c2c159b50 0x7f6c2c06aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:15.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.539+0000 7f6c3125c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2c0c78f0 con 0x7f6c2c0717c0 2023-05-20T01:17:15.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.539+0000 7f6c3125c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2c0c7bd0 con 0x7f6c2c0cd1c0 2023-05-20T01:17:15.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.539+0000 7f6c3125c700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2c0c7eb0 con 0x7f6c2c159b50 2023-05-20T01:17:15.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.539+0000 7f6c2ad9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c2c0717c0 0x7f6c2c14f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:15.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.539+0000 7f6c2ad9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c2c0717c0 0x7f6c2c14f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59126/0 (socket says 172.21.15.16:59126) 2023-05-20T01:17:15.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.539+0000 7f6c2ad9d700 1 -- 172.21.15.16:0/2450693240 learned_addr learned my addr 172.21.15.16:0/2450693240 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:15.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.539+0000 7f6c2a59c700 1 --2- 172.21.15.16:0/2450693240 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6c2c0cd1c0 0x7f6c2c06a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:15.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.539+0000 7f6c2b59e700 1 --2- 172.21.15.16:0/2450693240 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6c2c159b50 0x7f6c2c06aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:15.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.540+0000 7f6c2ad9d700 1 -- 172.21.15.16:0/2450693240 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6c2c159b50 msgr2=0x7f6c2c06aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:15.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.540+0000 7f6c2ad9d700 1 --2- 172.21.15.16:0/2450693240 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6c2c159b50 0x7f6c2c06aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:15.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.540+0000 7f6c2ad9d700 1 -- 172.21.15.16:0/2450693240 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6c2c0cd1c0 msgr2=0x7f6c2c06a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:15.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.540+0000 7f6c2ad9d700 1 --2- 172.21.15.16:0/2450693240 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6c2c0cd1c0 0x7f6c2c06a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:15.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.540+0000 7f6c2ad9d700 1 -- 172.21.15.16:0/2450693240 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c24009a70 con 0x7f6c2c0717c0 2023-05-20T01:17:15.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.540+0000 7f6c2b59e700 1 --2- 172.21.15.16:0/2450693240 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6c2c159b50 0x7f6c2c06aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:17:15.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.540+0000 7f6c2ad9d700 1 --2- 172.21.15.16:0/2450693240 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c2c0717c0 0x7f6c2c14f5f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c00c4d0 tx=0x7f6c1c00c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:15.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.540+0000 7f6c237fe700 1 -- 172.21.15.16:0/2450693240 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c1c0052d0 con 0x7f6c2c0717c0 2023-05-20T01:17:15.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.540+0000 7f6c237fe700 1 -- 172.21.15.16:0/2450693240 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c1c005430 con 0x7f6c2c0717c0 2023-05-20T01:17:15.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.540+0000 7f6c3125c700 1 -- 172.21.15.16:0/2450693240 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c2c0cec70 con 0x7f6c2c0717c0 2023-05-20T01:17:15.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.540+0000 7f6c237fe700 1 -- 172.21.15.16:0/2450693240 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c1c0038f0 con 0x7f6c2c0717c0 2023-05-20T01:17:15.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.540+0000 7f6c3125c700 1 -- 172.21.15.16:0/2450693240 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c2c071e80 con 0x7f6c2c0717c0 2023-05-20T01:17:15.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.541+0000 7f6c3125c700 1 -- 172.21.15.16:0/2450693240 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c2c0747b0 con 0x7f6c2c0717c0 2023-05-20T01:17:15.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.541+0000 7f6c237fe700 1 -- 172.21.15.16:0/2450693240 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6c1c003a50 con 0x7f6c2c0717c0 2023-05-20T01:17:15.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.542+0000 7f6c237fe700 1 --2- 172.21.15.16:0/2450693240 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f6c0c06ace0 0x7f6c0c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:15.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.542+0000 7f6c237fe700 1 -- 172.21.15.16:0/2450693240 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f6c1c088b90 con 0x7f6c2c0717c0 2023-05-20T01:17:15.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.542+0000 7f6c2a59c700 1 --2- 172.21.15.16:0/2450693240 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f6c0c06ace0 0x7f6c0c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:15.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.542+0000 7f6c2a59c700 1 --2- 172.21.15.16:0/2450693240 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f6c0c06ace0 0x7f6c0c06d190 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c153270 tx=0x7f6c14007dc0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:15.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.546+0000 7f6c237fe700 1 -- 172.21.15.16:0/2450693240 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f6c1c053910 con 0x7f6c2c0717c0 2023-05-20T01:17:15.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.743+0000 7f6c3125c700 1 -- 172.21.15.16:0/2450693240 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6c2c1592b0 con 0x7f6c2c0717c0 2023-05-20T01:17:15.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.743+0000 7f6c237fe700 1 -- 172.21.15.16:0/2450693240 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6c1c017020 con 0x7f6c2c0717c0 2023-05-20T01:17:15.748 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2023-05-20T01:17:15.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.745+0000 7f6c3125c700 1 -- 172.21.15.16:0/2450693240 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f6c0c06ace0 msgr2=0x7f6c0c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:15.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.745+0000 7f6c3125c700 1 --2- 172.21.15.16:0/2450693240 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f6c0c06ace0 0x7f6c0c06d190 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c153270 tx=0x7f6c14007dc0 comp rx=0 tx=0).stop 2023-05-20T01:17:15.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.745+0000 7f6c3125c700 1 -- 172.21.15.16:0/2450693240 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c2c0717c0 msgr2=0x7f6c2c14f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:15.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.745+0000 7f6c3125c700 1 --2- 172.21.15.16:0/2450693240 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c2c0717c0 0x7f6c2c14f5f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c00c4d0 tx=0x7f6c1c00c7e0 comp rx=0 tx=0).stop 2023-05-20T01:17:15.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.746+0000 7f6c3125c700 1 -- 172.21.15.16:0/2450693240 shutdown_connections 2023-05-20T01:17:15.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.746+0000 7f6c3125c700 1 --2- 172.21.15.16:0/2450693240 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f6c0c06ace0 0x7f6c0c06d190 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:15.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.746+0000 7f6c3125c700 1 --2- 172.21.15.16:0/2450693240 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6c2c159b50 0x7f6c2c06aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:15.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.746+0000 7f6c3125c700 1 --2- 172.21.15.16:0/2450693240 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c2c0717c0 0x7f6c2c14f5f0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:15.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.746+0000 7f6c3125c700 1 --2- 172.21.15.16:0/2450693240 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6c2c0cd1c0 0x7f6c2c06a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:15.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.746+0000 7f6c3125c700 1 -- 172.21.15.16:0/2450693240 >> 172.21.15.16:0/2450693240 conn(0x7f6c2c0cffd0 msgr2=0x7f6c2c0b4510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:15.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.746+0000 7f6c3125c700 1 -- 172.21.15.16:0/2450693240 shutdown_connections 2023-05-20T01:17:15.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.746+0000 7f6c3125c700 1 -- 172.21.15.16:0/2450693240 wait complete. 2023-05-20T01:17:15.762 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-05-20T01:17:15.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.887+0000 7f8bdf214700 1 Processor -- start 2023-05-20T01:17:15.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.887+0000 7f8bdf214700 1 -- start start 2023-05-20T01:17:15.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.888+0000 7f8bdf214700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8bd815cec0 0x7f8bd80cd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:15.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.888+0000 7f8bdf214700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8bd806e190 0x7f8bd80cf1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:15.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.888+0000 7f8bdf214700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f8bd806a510 0x7f8bd806a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:15.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.888+0000 7f8bdf214700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bd806aed0 con 0x7f8bd815cec0 2023-05-20T01:17:15.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.888+0000 7f8bdf214700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bd806dd00 con 0x7f8bd806a510 2023-05-20T01:17:15.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.888+0000 7f8bdf214700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd80cfa60 con 0x7f8bd806e190 2023-05-20T01:17:15.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.888+0000 7f8bdcfb0700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8bd815cec0 0x7f8bd80cd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:15.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.888+0000 7f8bdcfb0700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8bd815cec0 0x7f8bd80cd4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59134/0 (socket says 172.21.15.16:59134) 2023-05-20T01:17:15.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.888+0000 7f8bdcfb0700 1 -- 172.21.15.16:0/1560580290 learned_addr learned my addr 172.21.15.16:0/1560580290 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:15.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.888+0000 7f8bdd7b1700 1 --2- 172.21.15.16:0/1560580290 >> v2:172.21.15.132:3300/0 conn(0x7f8bd806a510 0x7f8bd806a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:15.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.888+0000 7f8bd7fff700 1 --2- 172.21.15.16:0/1560580290 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8bd806e190 0x7f8bd80cf1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:15.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.888+0000 7f8bdcfb0700 1 -- 172.21.15.16:0/1560580290 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8bd806e190 msgr2=0x7f8bd80cf1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:15.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.888+0000 7f8bdcfb0700 1 --2- 172.21.15.16:0/1560580290 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8bd806e190 0x7f8bd80cf1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:15.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.888+0000 7f8bdcfb0700 1 -- 172.21.15.16:0/1560580290 >> v2:172.21.15.132:3300/0 conn(0x7f8bd806a510 msgr2=0x7f8bd806a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:15.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.888+0000 7f8bdcfb0700 1 --2- 172.21.15.16:0/1560580290 >> v2:172.21.15.132:3300/0 conn(0x7f8bd806a510 0x7f8bd806a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:15.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.888+0000 7f8bdcfb0700 1 -- 172.21.15.16:0/1560580290 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bd806d6b0 con 0x7f8bd815cec0 2023-05-20T01:17:15.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.889+0000 7f8bdcfb0700 1 --2- 172.21.15.16:0/1560580290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bd815cec0 0x7f8bd80cd4f0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f8bd80b2ba0 tx=0x7f8bcc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ac8fc8557531548 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:15.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.889+0000 7f8bd77fe700 1 -- 172.21.15.16:0/1560580290 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc011830 con 0x7f8bd815cec0 2023-05-20T01:17:15.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.889+0000 7f8bd77fe700 1 -- 172.21.15.16:0/1560580290 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bcc011990 con 0x7f8bd815cec0 2023-05-20T01:17:15.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.889+0000 7f8bd77fe700 1 -- 172.21.15.16:0/1560580290 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc011d20 con 0x7f8bd815cec0 2023-05-20T01:17:15.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.889+0000 7f8bdf214700 1 -- 172.21.15.16:0/1560580290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bd815cec0 msgr2=0x7f8bd80cd4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:15.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.889+0000 7f8bdf214700 1 --2- 172.21.15.16:0/1560580290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bd815cec0 0x7f8bd80cd4f0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f8bd80b2ba0 tx=0x7f8bcc009bd0 comp rx=0 tx=0).stop 2023-05-20T01:17:15.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.889+0000 7f8bdf214700 1 -- 172.21.15.16:0/1560580290 shutdown_connections 2023-05-20T01:17:15.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.889+0000 7f8bdf214700 1 --2- 172.21.15.16:0/1560580290 >> v2:172.21.15.132:3300/0 conn(0x7f8bd806a510 0x7f8bd806a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:15.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.889+0000 7f8bdf214700 1 --2- 172.21.15.16:0/1560580290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bd815cec0 0x7f8bd80cd4f0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:15.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.889+0000 7f8bdf214700 1 --2- 172.21.15.16:0/1560580290 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8bd806e190 0x7f8bd80cf1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:15.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.889+0000 7f8bdf214700 1 -- 172.21.15.16:0/1560580290 >> 172.21.15.16:0/1560580290 conn(0x7f8bd806c260 msgr2=0x7f8bd80cffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:15.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.889+0000 7f8bdf214700 1 -- 172.21.15.16:0/1560580290 shutdown_connections 2023-05-20T01:17:15.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.889+0000 7f8bdf214700 1 -- 172.21.15.16:0/1560580290 wait complete. 2023-05-20T01:17:15.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.890+0000 7f8bdf214700 1 Processor -- start 2023-05-20T01:17:15.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.891+0000 7f8bdf214700 1 -- start start 2023-05-20T01:17:15.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.891+0000 7f8bdf214700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bd806a510 0x7f8bd815f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:15.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.891+0000 7f8bdf214700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8bd806e190 0x7f8bd81f32d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:15.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.891+0000 7f8bdf214700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8bd815cec0 0x7f8bd80b4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:15.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.891+0000 7f8bdf214700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd81f6a50 con 0x7f8bd806a510 2023-05-20T01:17:15.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.891+0000 7f8bdf214700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd81f5e40 con 0x7f8bd806e190 2023-05-20T01:17:15.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.891+0000 7f8bdf214700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd81f6120 con 0x7f8bd815cec0 2023-05-20T01:17:15.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.891+0000 7f8bdcfb0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bd806a510 0x7f8bd815f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:15.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.891+0000 7f8bdcfb0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bd806a510 0x7f8bd815f4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59140/0 (socket says 172.21.15.16:59140) 2023-05-20T01:17:15.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.892+0000 7f8bdcfb0700 1 -- 172.21.15.16:0/602616494 learned_addr learned my addr 172.21.15.16:0/602616494 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:15.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.892+0000 7f8bd7fff700 1 --2- 172.21.15.16:0/602616494 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8bd806e190 0x7f8bd81f32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:15.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.892+0000 7f8bdd7b1700 1 --2- 172.21.15.16:0/602616494 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8bd815cec0 0x7f8bd80b4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:15.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.892+0000 7f8bdcfb0700 1 -- 172.21.15.16:0/602616494 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8bd815cec0 msgr2=0x7f8bd80b4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:15.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.892+0000 7f8bdcfb0700 1 --2- 172.21.15.16:0/602616494 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8bd815cec0 0x7f8bd80b4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:15.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.892+0000 7f8bdcfb0700 1 -- 172.21.15.16:0/602616494 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8bd806e190 msgr2=0x7f8bd81f32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:15.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.892+0000 7f8bdcfb0700 1 --2- 172.21.15.16:0/602616494 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8bd806e190 0x7f8bd81f32d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:15.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.892+0000 7f8bdcfb0700 1 -- 172.21.15.16:0/602616494 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bcc009550 con 0x7f8bd806a510 2023-05-20T01:17:15.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.892+0000 7f8bdd7b1700 1 --2- 172.21.15.16:0/602616494 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8bd815cec0 0x7f8bd80b4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:17:15.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.892+0000 7f8bd7fff700 1 --2- 172.21.15.16:0/602616494 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8bd806e190 0x7f8bd81f32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:17:15.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.892+0000 7f8bdcfb0700 1 --2- 172.21.15.16:0/602616494 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bd806a510 0x7f8bd815f4a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc000c00 tx=0x7f8bcc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:15.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.892+0000 7f8bd57fa700 1 -- 172.21.15.16:0/602616494 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc004300 con 0x7f8bd806a510 2023-05-20T01:17:15.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.892+0000 7f8bd57fa700 1 -- 172.21.15.16:0/602616494 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bcc01f070 con 0x7f8bd806a510 2023-05-20T01:17:15.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.892+0000 7f8bdf214700 1 -- 172.21.15.16:0/602616494 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bd815d290 con 0x7f8bd806a510 2023-05-20T01:17:15.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.892+0000 7f8bd57fa700 1 -- 172.21.15.16:0/602616494 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc0036a0 con 0x7f8bd806a510 2023-05-20T01:17:15.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.892+0000 7f8bdf214700 1 -- 172.21.15.16:0/602616494 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bd806d990 con 0x7f8bd806a510 2023-05-20T01:17:15.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.893+0000 7f8bdf214700 1 -- 172.21.15.16:0/602616494 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bd80cfd40 con 0x7f8bd806a510 2023-05-20T01:17:15.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.894+0000 7f8bd57fa700 1 -- 172.21.15.16:0/602616494 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8bcc021030 con 0x7f8bd806a510 2023-05-20T01:17:15.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.894+0000 7f8bd57fa700 1 --2- 172.21.15.16:0/602616494 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f8bbc06ace0 0x7f8bbc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:15.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.894+0000 7f8bd57fa700 1 -- 172.21.15.16:0/602616494 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f8bcc08ab50 con 0x7f8bd806a510 2023-05-20T01:17:15.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.894+0000 7f8bd7fff700 1 --2- 172.21.15.16:0/602616494 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f8bbc06ace0 0x7f8bbc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:15.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.894+0000 7f8bd7fff700 1 --2- 172.21.15.16:0/602616494 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f8bbc06ace0 0x7f8bbc06d190 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8bd8071140 tx=0x7f8bc4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:15.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:15.898+0000 7f8bd57fa700 1 -- 172.21.15.16:0/602616494 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f8bcc055a80 con 0x7f8bd806a510 2023-05-20T01:17:16.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:16.096+0000 7f8bdf214700 1 -- 172.21.15.16:0/602616494 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f8bd81f6b90 con 0x7f8bd806a510 2023-05-20T01:17:16.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:16.096+0000 7f8bd57fa700 1 -- 172.21.15.16:0/602616494 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f8bd81f6b90 con 0x7f8bd806a510 2023-05-20T01:17:16.100 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-05-20T01:17:16.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:16.098+0000 7f8bdf214700 1 -- 172.21.15.16:0/602616494 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f8bbc06ace0 msgr2=0x7f8bbc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:16.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:16.098+0000 7f8bdf214700 1 --2- 172.21.15.16:0/602616494 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f8bbc06ace0 0x7f8bbc06d190 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8bd8071140 tx=0x7f8bc4008040 comp rx=0 tx=0).stop 2023-05-20T01:17:16.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:16.098+0000 7f8bdf214700 1 -- 172.21.15.16:0/602616494 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bd806a510 msgr2=0x7f8bd815f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:16.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:16.098+0000 7f8bdf214700 1 --2- 172.21.15.16:0/602616494 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bd806a510 0x7f8bd815f4a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc000c00 tx=0x7f8bcc004750 comp rx=0 tx=0).stop 2023-05-20T01:17:16.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:16.098+0000 7f8bdf214700 1 -- 172.21.15.16:0/602616494 shutdown_connections 2023-05-20T01:17:16.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:16.098+0000 7f8bdf214700 1 --2- 172.21.15.16:0/602616494 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f8bbc06ace0 0x7f8bbc06d190 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:16.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:16.098+0000 7f8bdf214700 1 --2- 172.21.15.16:0/602616494 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8bd815cec0 0x7f8bd80b4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:16.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:16.098+0000 7f8bdf214700 1 --2- 172.21.15.16:0/602616494 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bd806a510 0x7f8bd815f4a0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:16.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:16.098+0000 7f8bdf214700 1 --2- 172.21.15.16:0/602616494 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8bd806e190 0x7f8bd81f32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:16.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:16.098+0000 7f8bdf214700 1 -- 172.21.15.16:0/602616494 >> 172.21.15.16:0/602616494 conn(0x7f8bd806c260 msgr2=0x7f8bd80cc1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:16.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:16.099+0000 7f8bdf214700 1 -- 172.21.15.16:0/602616494 shutdown_connections 2023-05-20T01:17:16.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:16.099+0000 7f8bdf214700 1 -- 172.21.15.16:0/602616494 wait complete. 2023-05-20T01:17:16.128 INFO:tasks.cephfs_test_runner:Starting test: test_extra_data_pool_rebuild_few_files (tasks.cephfs.test_data_scan.TestDataScan) 2023-05-20T01:17:16.129 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_data_scan.TestDataScan.test_extra_data_pool_rebuild_few_files' 2023-05-20T01:17:16.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:16.251+0000 7f3c6b7e1700 1 Processor -- start 2023-05-20T01:17:16.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:16.251+0000 7f3c6b7e1700 1 -- start start 2023-05-20T01:17:16.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:16.251+0000 7f3c6b7e1700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3c640cad50 0x7f3c64071140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:16.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:16.251+0000 7f3c6b7e1700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3c64071680 0x7f3c64071b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:16.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:16.252+0000 7f3c6b7e1700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c640ce920 con 0x7f3c640ca980 2023-05-20T01:17:16.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:16.252+0000 7f3c6b7e1700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c640af8f0 con 0x7f3c64071680 2023-05-20T01:17:16.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:16.252+0000 7f3c6b7e1700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c640725d0 con 0x7f3c640cad50 2023-05-20T01:17:16.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:16.252+0000 7f3c69d7e700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3c64071680 0x7f3c64071b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:16.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:16.252+0000 7f3c69d7e700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3c64071680 0x7f3c64071b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59152/0 (socket says 172.21.15.16:59152) 2023-05-20T01:17:16.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:16.252+0000 7f3c69d7e700 1 -- 172.21.15.16:0/2719573136 learned_addr learned my addr 172.21.15.16:0/2719573136 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:16.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:16.252+0000 7f3c68d7c700 1 --2- 172.21.15.16:0/2719573136 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3c640cad50 0x7f3c64071140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:16.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:16.252+0000 7f3c69d7e700 1 -- 172.21.15.16:0/2719573136 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3c640cad50 msgr2=0x7f3c64071140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:16.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:16.252+0000 7f3c69d7e700 1 --2- 172.21.15.16:0/2719573136 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3c640cad50 0x7f3c64071140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:16.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:16.252+0000 7f3c69d7e700 1 -- 172.21.15.16:0/2719573136 >> v1:172.21.15.132:6789/0 conn(0x7f3c640ca980 legacy=0x7f3c640aef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:16.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:16.252+0000 7f3c69d7e700 1 -- 172.21.15.16:0/2719573136 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c64072040 con 0x7f3c64071680 2023-05-20T01:17:16.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:16.252+0000 7f3c69d7e700 1 --2- 172.21.15.16:0/2719573136 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c64071680 0x7f3c64071b00 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c00d150 tx=0x7f3c5c00d460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e82708f614e23960 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:16.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:16.252+0000 7f3c5bfff700 1 -- 172.21.15.16:0/2719573136 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c5c010070 con 0x7f3c64071680 2023-05-20T01:17:16.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:16.252+0000 7f3c5bfff700 1 -- 172.21.15.16:0/2719573136 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c5c00dd80 con 0x7f3c64071680 2023-05-20T01:17:16.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:16.252+0000 7f3c5bfff700 1 -- 172.21.15.16:0/2719573136 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c5c00b310 con 0x7f3c64071680 2023-05-20T01:17:16.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:16.253+0000 7f3c6b7e1700 1 -- 172.21.15.16:0/2719573136 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c64071680 msgr2=0x7f3c64071b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:16.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:16.253+0000 7f3c6b7e1700 1 --2- 172.21.15.16:0/2719573136 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c64071680 0x7f3c64071b00 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c00d150 tx=0x7f3c5c00d460 comp rx=0 tx=0).stop 2023-05-20T01:17:16.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:16.253+0000 7f3c6b7e1700 1 -- 172.21.15.16:0/2719573136 shutdown_connections 2023-05-20T01:17:16.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:16.253+0000 7f3c6b7e1700 1 --2- 172.21.15.16:0/2719573136 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c64071680 0x7f3c64071b00 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:16.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:16.253+0000 7f3c6b7e1700 1 --2- 172.21.15.16:0/2719573136 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3c640cad50 0x7f3c64071140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:16.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:16.253+0000 7f3c6b7e1700 1 -- 172.21.15.16:0/2719573136 >> 172.21.15.16:0/2719573136 conn(0x7f3c640d0c70 msgr2=0x7f3c640b1af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:16.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:16.253+0000 7f3c6b7e1700 1 -- 172.21.15.16:0/2719573136 shutdown_connections 2023-05-20T01:17:16.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:16.253+0000 7f3c6b7e1700 1 -- 172.21.15.16:0/2719573136 wait complete. 2023-05-20T01:17:16.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:16.254+0000 7f3c6b7e1700 1 Processor -- start 2023-05-20T01:17:16.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:16.254+0000 7f3c6b7e1700 1 -- start start 2023-05-20T01:17:16.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:16.255+0000 7f3c6b7e1700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3c64071680 0x7f3c640ca0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:16.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:16.255+0000 7f3c6b7e1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c640ca980 0x7f3c641e2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:16.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:16.255+0000 7f3c6b7e1700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3c640cad50 0x7f3c640c7f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:16.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:16.255+0000 7f3c6b7e1700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c640c3bb0 con 0x7f3c640ca980 2023-05-20T01:17:16.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:16.255+0000 7f3c6b7e1700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c640c3e90 con 0x7f3c64071680 2023-05-20T01:17:16.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:16.255+0000 7f3c6b7e1700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c640c4170 con 0x7f3c640cad50 2023-05-20T01:17:16.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:16.255+0000 7f3c68d7c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c640ca980 0x7f3c641e2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:16.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:16.255+0000 7f3c68d7c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c640ca980 0x7f3c641e2a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59154/0 (socket says 172.21.15.16:59154) 2023-05-20T01:17:16.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:16.255+0000 7f3c68d7c700 1 -- 172.21.15.16:0/4032327707 learned_addr learned my addr 172.21.15.16:0/4032327707 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:16.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:16.255+0000 7f3c6957d700 1 --2- 172.21.15.16:0/4032327707 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3c64071680 0x7f3c640ca0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:16.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:16.255+0000 7f3c69d7e700 1 --2- 172.21.15.16:0/4032327707 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3c640cad50 0x7f3c640c7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:16.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:16.255+0000 7f3c68d7c700 1 -- 172.21.15.16:0/4032327707 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3c640cad50 msgr2=0x7f3c640c7f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:16.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:16.255+0000 7f3c68d7c700 1 --2- 172.21.15.16:0/4032327707 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3c640cad50 0x7f3c640c7f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:16.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:16.255+0000 7f3c68d7c700 1 -- 172.21.15.16:0/4032327707 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3c64071680 msgr2=0x7f3c640ca0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:16.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:16.255+0000 7f3c68d7c700 1 --2- 172.21.15.16:0/4032327707 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3c64071680 0x7f3c640ca0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:16.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:16.255+0000 7f3c68d7c700 1 -- 172.21.15.16:0/4032327707 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c5c00bd40 con 0x7f3c640ca980 2023-05-20T01:17:16.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:16.255+0000 7f3c6957d700 1 --2- 172.21.15.16:0/4032327707 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3c64071680 0x7f3c640ca0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:17:16.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:16.256+0000 7f3c68d7c700 1 --2- 172.21.15.16:0/4032327707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c640ca980 0x7f3c641e2a30 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f3c6000c7f0 tx=0x7f3c6000cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:16.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:16.256+0000 7f3c59ffb700 1 -- 172.21.15.16:0/4032327707 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c60011800 con 0x7f3c640ca980 2023-05-20T01:17:16.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:16.256+0000 7f3c59ffb700 1 -- 172.21.15.16:0/4032327707 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c60011960 con 0x7f3c640ca980 2023-05-20T01:17:16.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:16.256+0000 7f3c6b7e1700 1 -- 172.21.15.16:0/4032327707 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c640722f0 con 0x7f3c640ca980 2023-05-20T01:17:16.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:16.256+0000 7f3c59ffb700 1 -- 172.21.15.16:0/4032327707 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c60011d10 con 0x7f3c640ca980 2023-05-20T01:17:16.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:16.256+0000 7f3c6b7e1700 1 -- 172.21.15.16:0/4032327707 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c640af620 con 0x7f3c640ca980 2023-05-20T01:17:16.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:16.257+0000 7f3c6b7e1700 1 -- 172.21.15.16:0/4032327707 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c640ce5b0 con 0x7f3c640ca980 2023-05-20T01:17:16.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:16.257+0000 7f3c59ffb700 1 -- 172.21.15.16:0/4032327707 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3c60018360 con 0x7f3c640ca980 2023-05-20T01:17:16.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:16.258+0000 7f3c59ffb700 1 --2- 172.21.15.16:0/4032327707 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f3c4806ace0 0x7f3c4806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:16.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:16.258+0000 7f3c59ffb700 1 -- 172.21.15.16:0/4032327707 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f3c60089770 con 0x7f3c640ca980 2023-05-20T01:17:16.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:16.258+0000 7f3c6957d700 1 --2- 172.21.15.16:0/4032327707 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f3c4806ace0 0x7f3c4806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:16.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:16.258+0000 7f3c6957d700 1 --2- 172.21.15.16:0/4032327707 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f3c4806ace0 0x7f3c4806d190 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f3c640b1e40 tx=0x7f3c54008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:16.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:16.262+0000 7f3c59ffb700 1 -- 172.21.15.16:0/4032327707 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f3c60054580 con 0x7f3c640ca980 2023-05-20T01:17:16.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:16.549+0000 7f3c6b7e1700 1 -- 172.21.15.16:0/4032327707 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_data_scan.TestDataScan.test_extra_data_pool_rebuild_few_files"]} v 0) v1 -- 0x7f3c640c3cf0 con 0x7f3c640ca980 2023-05-20T01:17:17.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:17.423+0000 7f3c59ffb700 1 -- 172.21.15.16:0/4032327707 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_data_scan.TestDataScan.test_extra_data_pool_rebuild_few_files"]}]=0 v62) v1 ==== 161+0+0 (secure 0 0 0) 0x7f3c640c3cf0 con 0x7f3c640ca980 2023-05-20T01:17:17.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:17.425+0000 7f3c6b7e1700 1 -- 172.21.15.16:0/4032327707 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f3c4806ace0 msgr2=0x7f3c4806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:17.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:17.425+0000 7f3c6b7e1700 1 --2- 172.21.15.16:0/4032327707 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f3c4806ace0 0x7f3c4806d190 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f3c640b1e40 tx=0x7f3c54008040 comp rx=0 tx=0).stop 2023-05-20T01:17:17.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:17.425+0000 7f3c6b7e1700 1 -- 172.21.15.16:0/4032327707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c640ca980 msgr2=0x7f3c641e2a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:17.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:17.425+0000 7f3c6b7e1700 1 --2- 172.21.15.16:0/4032327707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c640ca980 0x7f3c641e2a30 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f3c6000c7f0 tx=0x7f3c6000cbb0 comp rx=0 tx=0).stop 2023-05-20T01:17:17.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:17.426+0000 7f3c6b7e1700 1 -- 172.21.15.16:0/4032327707 shutdown_connections 2023-05-20T01:17:17.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:17.426+0000 7f3c6b7e1700 1 --2- 172.21.15.16:0/4032327707 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f3c4806ace0 0x7f3c4806d190 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:17.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:17.426+0000 7f3c6b7e1700 1 --2- 172.21.15.16:0/4032327707 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3c640cad50 0x7f3c640c7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:17.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:17.426+0000 7f3c6b7e1700 1 --2- 172.21.15.16:0/4032327707 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3c64071680 0x7f3c640ca0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:17.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:17.426+0000 7f3c6b7e1700 1 --2- 172.21.15.16:0/4032327707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c640ca980 0x7f3c641e2a30 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:17.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:17.426+0000 7f3c6b7e1700 1 -- 172.21.15.16:0/4032327707 >> 172.21.15.16:0/4032327707 conn(0x7f3c640d0c70 msgr2=0x7f3c6406d9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:17.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:17.426+0000 7f3c6b7e1700 1 -- 172.21.15.16:0/4032327707 shutdown_connections 2023-05-20T01:17:17.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:17.426+0000 7f3c6b7e1700 1 -- 172.21.15.16:0/4032327707 wait complete. 2023-05-20T01:17:17.444 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-05-20T01:17:17.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:17.575+0000 7ff0be1a4700 1 Processor -- start 2023-05-20T01:17:17.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:17.575+0000 7ff0be1a4700 1 -- start start 2023-05-20T01:17:17.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:17.576+0000 7ff0be1a4700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff0b815cd20 0x7ff0b80b1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:17.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:17.576+0000 7ff0be1a4700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff0b80b14c0 0x7ff0b806b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:17.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:17.576+0000 7ff0be1a4700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0b806bfc0 con 0x7ff0b806bb60 2023-05-20T01:17:17.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:17.576+0000 7ff0be1a4700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0b80cb700 con 0x7ff0b80b14c0 2023-05-20T01:17:17.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:17.576+0000 7ff0be1a4700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b80b2430 con 0x7ff0b815cd20 2023-05-20T01:17:17.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:17.576+0000 7ff0b6ffd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff0b80b14c0 0x7ff0b806b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:17.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:17.576+0000 7ff0b6ffd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff0b80b14c0 0x7ff0b806b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59162/0 (socket says 172.21.15.16:59162) 2023-05-20T01:17:17.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:17.576+0000 7ff0b6ffd700 1 -- 172.21.15.16:0/3080950965 learned_addr learned my addr 172.21.15.16:0/3080950965 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:17.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:17.577+0000 7ff0b77fe700 1 --2- 172.21.15.16:0/3080950965 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff0b815cd20 0x7ff0b80b1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:17.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:17.577+0000 7ff0b6ffd700 1 -- 172.21.15.16:0/3080950965 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff0b815cd20 msgr2=0x7ff0b80b1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:17.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:17.577+0000 7ff0b6ffd700 1 --2- 172.21.15.16:0/3080950965 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff0b815cd20 0x7ff0b80b1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:17.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:17.577+0000 7ff0b6ffd700 1 -- 172.21.15.16:0/3080950965 >> v1:172.21.15.132:6789/0 conn(0x7ff0b806bb60 legacy=0x7ff0b80cd930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:17.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:17.577+0000 7ff0b6ffd700 1 -- 172.21.15.16:0/3080950965 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0b80b2710 con 0x7ff0b80b14c0 2023-05-20T01:17:17.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:17.577+0000 7ff0b77fe700 1 --2- 172.21.15.16:0/3080950965 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff0b815cd20 0x7ff0b80b1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:17:17.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:17.577+0000 7ff0b6ffd700 1 --2- 172.21.15.16:0/3080950965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0b80b14c0 0x7ff0b806b620 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ff0a00098f0 tx=0x7ff0a0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=341c90302e0af6a0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:17.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:17.577+0000 7ff0b67fc700 1 -- 172.21.15.16:0/3080950965 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a0004230 con 0x7ff0b80b14c0 2023-05-20T01:17:17.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:17.577+0000 7ff0b67fc700 1 -- 172.21.15.16:0/3080950965 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0a0004390 con 0x7ff0b80b14c0 2023-05-20T01:17:17.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:17.577+0000 7ff0b67fc700 1 -- 172.21.15.16:0/3080950965 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a00036a0 con 0x7ff0b80b14c0 2023-05-20T01:17:17.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:17.577+0000 7ff0be1a4700 1 -- 172.21.15.16:0/3080950965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0b80b14c0 msgr2=0x7ff0b806b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:17.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:17.577+0000 7ff0be1a4700 1 --2- 172.21.15.16:0/3080950965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0b80b14c0 0x7ff0b806b620 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ff0a00098f0 tx=0x7ff0a0009c00 comp rx=0 tx=0).stop 2023-05-20T01:17:17.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:17.578+0000 7ff0be1a4700 1 -- 172.21.15.16:0/3080950965 shutdown_connections 2023-05-20T01:17:17.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:17.578+0000 7ff0be1a4700 1 --2- 172.21.15.16:0/3080950965 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff0b815cd20 0x7ff0b80b1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:17.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:17.578+0000 7ff0be1a4700 1 --2- 172.21.15.16:0/3080950965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0b80b14c0 0x7ff0b806b620 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:17.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:17.578+0000 7ff0be1a4700 1 -- 172.21.15.16:0/3080950965 >> 172.21.15.16:0/3080950965 conn(0x7ff0b806ae30 msgr2=0x7ff0b809a400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:17.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:17.578+0000 7ff0be1a4700 1 -- 172.21.15.16:0/3080950965 shutdown_connections 2023-05-20T01:17:17.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:17.578+0000 7ff0be1a4700 1 -- 172.21.15.16:0/3080950965 wait complete. 2023-05-20T01:17:17.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:17.579+0000 7ff0be1a4700 1 Processor -- start 2023-05-20T01:17:17.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:17.579+0000 7ff0be1a4700 1 -- start start 2023-05-20T01:17:17.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:17.580+0000 7ff0be1a4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0b806bb60 0x7ff0b80b68f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:17.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:17.580+0000 7ff0be1a4700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff0b80b14c0 0x7ff0b80cc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:17.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:17.580+0000 7ff0be1a4700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff0b815cd20 0x7ff0b815ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:17.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:17.580+0000 7ff0be1a4700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b80ca4f0 con 0x7ff0b806bb60 2023-05-20T01:17:17.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:17.580+0000 7ff0be1a4700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b80ca7d0 con 0x7ff0b815cd20 2023-05-20T01:17:17.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:17.580+0000 7ff0be1a4700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b80caab0 con 0x7ff0b80b14c0 2023-05-20T01:17:17.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:17.580+0000 7ff0b77fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0b806bb60 0x7ff0b80b68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:17.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:17.580+0000 7ff0b77fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0b806bb60 0x7ff0b80b68f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59168/0 (socket says 172.21.15.16:59168) 2023-05-20T01:17:17.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:17.580+0000 7ff0b77fe700 1 -- 172.21.15.16:0/2811287119 learned_addr learned my addr 172.21.15.16:0/2811287119 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:17.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:17.581+0000 7ff0b6ffd700 1 --2- 172.21.15.16:0/2811287119 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff0b80b14c0 0x7ff0b80cc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:17.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:17.581+0000 7ff0b7fff700 1 --2- 172.21.15.16:0/2811287119 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff0b815cd20 0x7ff0b815ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:17.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:17.581+0000 7ff0b77fe700 1 -- 172.21.15.16:0/2811287119 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff0b80b14c0 msgr2=0x7ff0b80cc2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:17.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:17.581+0000 7ff0b77fe700 1 --2- 172.21.15.16:0/2811287119 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff0b80b14c0 0x7ff0b80cc2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:17.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:17.581+0000 7ff0b77fe700 1 -- 172.21.15.16:0/2811287119 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff0b815cd20 msgr2=0x7ff0b815ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:17.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:17.581+0000 7ff0b77fe700 1 --2- 172.21.15.16:0/2811287119 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff0b815cd20 0x7ff0b815ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:17.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:17.581+0000 7ff0b77fe700 1 -- 172.21.15.16:0/2811287119 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0a0009550 con 0x7ff0b806bb60 2023-05-20T01:17:17.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:17.581+0000 7ff0b6ffd700 1 --2- 172.21.15.16:0/2811287119 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff0b80b14c0 0x7ff0b80cc2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:17:17.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:17.581+0000 7ff0b77fe700 1 --2- 172.21.15.16:0/2811287119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0b806bb60 0x7ff0b80b68f0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7ff0a800c750 tx=0x7ff0a800cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:17.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:17.581+0000 7ff0affff700 1 -- 172.21.15.16:0/2811287119 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a8011800 con 0x7ff0b806bb60 2023-05-20T01:17:17.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:17.581+0000 7ff0affff700 1 -- 172.21.15.16:0/2811287119 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0a8011960 con 0x7ff0b806bb60 2023-05-20T01:17:17.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:17.581+0000 7ff0be1a4700 1 -- 172.21.15.16:0/2811287119 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0b80cbc30 con 0x7ff0b806bb60 2023-05-20T01:17:17.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:17.581+0000 7ff0affff700 1 -- 172.21.15.16:0/2811287119 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a8011cd0 con 0x7ff0b806bb60 2023-05-20T01:17:17.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:17.581+0000 7ff0be1a4700 1 -- 172.21.15.16:0/2811287119 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0b80cb950 con 0x7ff0b806bb60 2023-05-20T01:17:17.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:17.582+0000 7ff0be1a4700 1 -- 172.21.15.16:0/2811287119 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0b80b1930 con 0x7ff0b806bb60 2023-05-20T01:17:17.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:17.583+0000 7ff0affff700 1 -- 172.21.15.16:0/2811287119 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff0a8021350 con 0x7ff0b806bb60 2023-05-20T01:17:17.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:17.583+0000 7ff0affff700 1 --2- 172.21.15.16:0/2811287119 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff09806ace0 0x7ff09806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:17.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:17.583+0000 7ff0affff700 1 -- 172.21.15.16:0/2811287119 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7ff0a8089700 con 0x7ff0b806bb60 2023-05-20T01:17:17.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:17.583+0000 7ff0b6ffd700 1 --2- 172.21.15.16:0/2811287119 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff09806ace0 0x7ff09806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:17.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:17.583+0000 7ff0b6ffd700 1 --2- 172.21.15.16:0/2811287119 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff09806ace0 0x7ff09806d190 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff0b81f5be0 tx=0x7ff0a0008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:17.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:17.587+0000 7ff0affff700 1 -- 172.21.15.16:0/2811287119 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7ff0a8057fa0 con 0x7ff0b806bb60 2023-05-20T01:17:17.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:17.785+0000 7ff0be1a4700 1 -- 172.21.15.16:0/2811287119 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7ff0b80d1f60 con 0x7ff0b806bb60 2023-05-20T01:17:17.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:17.789+0000 7ff0affff700 1 -- 172.21.15.16:0/2811287119 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7ff0a8057b30 con 0x7ff0b806bb60 2023-05-20T01:17:17.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:17.791+0000 7ff0be1a4700 1 -- 172.21.15.16:0/2811287119 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff09806ace0 msgr2=0x7ff09806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:17.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:17.791+0000 7ff0be1a4700 1 --2- 172.21.15.16:0/2811287119 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff09806ace0 0x7ff09806d190 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff0b81f5be0 tx=0x7ff0a0008030 comp rx=0 tx=0).stop 2023-05-20T01:17:17.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:17.791+0000 7ff0be1a4700 1 -- 172.21.15.16:0/2811287119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0b806bb60 msgr2=0x7ff0b80b68f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:17.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:17.791+0000 7ff0be1a4700 1 --2- 172.21.15.16:0/2811287119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0b806bb60 0x7ff0b80b68f0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7ff0a800c750 tx=0x7ff0a800cb10 comp rx=0 tx=0).stop 2023-05-20T01:17:17.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:17.791+0000 7ff0be1a4700 1 -- 172.21.15.16:0/2811287119 shutdown_connections 2023-05-20T01:17:17.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:17.791+0000 7ff0be1a4700 1 --2- 172.21.15.16:0/2811287119 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff0b815cd20 0x7ff0b815ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:17.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:17.791+0000 7ff0be1a4700 1 --2- 172.21.15.16:0/2811287119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0b806bb60 0x7ff0b80b68f0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:17.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:17.791+0000 7ff0be1a4700 1 --2- 172.21.15.16:0/2811287119 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff09806ace0 0x7ff09806d190 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:17.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:17.791+0000 7ff0be1a4700 1 --2- 172.21.15.16:0/2811287119 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff0b80b14c0 0x7ff0b80cc2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:17.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:17.791+0000 7ff0be1a4700 1 -- 172.21.15.16:0/2811287119 >> 172.21.15.16:0/2811287119 conn(0x7ff0b806ae30 msgr2=0x7ff0b80afd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:17.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:17.791+0000 7ff0be1a4700 1 -- 172.21.15.16:0/2811287119 shutdown_connections 2023-05-20T01:17:17.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:17.791+0000 7ff0be1a4700 1 -- 172.21.15.16:0/2811287119 wait complete. 2023-05-20T01:17:17.807 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-05-20T01:17:17.808 DEBUG:teuthology.orchestra.run.smithi016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-05-20T01:17:17.848 DEBUG:teuthology.orchestra.run.smithi132:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-05-20T01:17:17.894 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-05-20T01:17:17.894 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-05-20T01:17:17.895 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2023-05-20T01:17:17.912 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-05-20T01:17:17.913 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2023-05-20T01:17:17.972 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-05-20T01:17:17.973 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2023-05-20T01:17:18.029 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-20T01:17:18.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.188+0000 7fd60f094700 1 Processor -- start 2023-05-20T01:17:18.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.189+0000 7fd60f094700 1 -- start start 2023-05-20T01:17:18.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.189+0000 7fd60f094700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fd6080d1000 0x7fd6080d13d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:18.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.189+0000 7fd60f094700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd60806ba90 0x7fd6080b10e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:18.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.189+0000 7fd60f094700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6080ce3e0 con 0x7fd60806b620 2023-05-20T01:17:18.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.189+0000 7fd60f094700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6080d1910 con 0x7fd6080d1000 2023-05-20T01:17:18.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.189+0000 7fd60f094700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6080cc6f0 con 0x7fd60806ba90 2023-05-20T01:17:18.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.190+0000 7fd60ce30700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fd6080d1000 0x7fd6080d13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:18.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.190+0000 7fd607fff700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7fd60806b620 0x7fd6080b09d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.16:52186/0 (socket says 172.21.15.16:52186) 2023-05-20T01:17:18.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.190+0000 7fd607fff700 1 -- 172.21.15.16:0/2114861857 learned_addr learned my addr 172.21.15.16:0/2114861857 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:17:18.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.190+0000 7fd60d631700 1 --2- 172.21.15.16:0/2114861857 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd60806ba90 0x7fd6080b10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:18.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.190+0000 7fd60d631700 1 -- 172.21.15.16:0/2114861857 >> v2:172.21.15.132:3300/0 conn(0x7fd6080d1000 msgr2=0x7fd6080d13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:18.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.190+0000 7fd60d631700 1 --2- 172.21.15.16:0/2114861857 >> v2:172.21.15.132:3300/0 conn(0x7fd6080d1000 0x7fd6080d13d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:18.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.190+0000 7fd60d631700 1 -- 172.21.15.16:0/2114861857 >> v1:172.21.15.132:6789/0 conn(0x7fd60806b620 legacy=0x7fd6080b09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:18.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.190+0000 7fd60d631700 1 -- 172.21.15.16:0/2114861857 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6080cdab0 con 0x7fd60806ba90 2023-05-20T01:17:18.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.190+0000 7fd60ce30700 1 --2- 172.21.15.16:0/2114861857 >> v2:172.21.15.132:3300/0 conn(0x7fd6080d1000 0x7fd6080d13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-05-20T01:17:18.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.190+0000 7fd60d631700 1 --2- 172.21.15.16:0/2114861857 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd60806ba90 0x7fd6080b10e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd600009d90 tx=0x7fd60000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3559f595f4ad7962 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:18.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.190+0000 7fd6077fe700 1 -- 172.21.15.16:0/2114861857 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd60000e070 con 0x7fd60806ba90 2023-05-20T01:17:18.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.191+0000 7fd6077fe700 1 -- 172.21.15.16:0/2114861857 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd60000bb20 con 0x7fd60806ba90 2023-05-20T01:17:18.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.191+0000 7fd6077fe700 1 -- 172.21.15.16:0/2114861857 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd600012430 con 0x7fd60806ba90 2023-05-20T01:17:18.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.191+0000 7fd60f094700 1 -- 172.21.15.16:0/2114861857 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd60806ba90 msgr2=0x7fd6080b10e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:18.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.191+0000 7fd60f094700 1 --2- 172.21.15.16:0/2114861857 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd60806ba90 0x7fd6080b10e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd600009d90 tx=0x7fd60000b330 comp rx=0 tx=0).stop 2023-05-20T01:17:18.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.191+0000 7fd60f094700 1 -- 172.21.15.16:0/2114861857 shutdown_connections 2023-05-20T01:17:18.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.191+0000 7fd60f094700 1 --2- 172.21.15.16:0/2114861857 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd60806ba90 0x7fd6080b10e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:18.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.191+0000 7fd60f094700 1 --2- 172.21.15.16:0/2114861857 >> v2:172.21.15.132:3300/0 conn(0x7fd6080d1000 0x7fd6080d13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:18.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.191+0000 7fd60f094700 1 -- 172.21.15.16:0/2114861857 >> 172.21.15.16:0/2114861857 conn(0x7fd60806ae30 msgr2=0x7fd60809a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:18.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.191+0000 7fd60f094700 1 -- 172.21.15.16:0/2114861857 shutdown_connections 2023-05-20T01:17:18.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.191+0000 7fd60f094700 1 -- 172.21.15.16:0/2114861857 wait complete. 2023-05-20T01:17:18.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.192+0000 7fd60f094700 1 Processor -- start 2023-05-20T01:17:18.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.192+0000 7fd60f094700 1 -- start start 2023-05-20T01:17:18.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.193+0000 7fd60f094700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd60806b620 0x7fd6080b51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:18.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.193+0000 7fd60f094700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd60806ba90 0x7fd6080c66c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:18.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.193+0000 7fd60f094700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6080d1000 0x7fd6080c6c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:18.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.193+0000 7fd60f094700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6080b5c40 con 0x7fd6080d1000 2023-05-20T01:17:18.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.193+0000 7fd60f094700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6080c5be0 con 0x7fd60806ba90 2023-05-20T01:17:18.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.193+0000 7fd60f094700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6080c5ec0 con 0x7fd60806b620 2023-05-20T01:17:18.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.193+0000 7fd60d631700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6080d1000 0x7fd6080c6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:18.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.193+0000 7fd60d631700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6080d1000 0x7fd6080c6c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59184/0 (socket says 172.21.15.16:59184) 2023-05-20T01:17:18.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.193+0000 7fd60d631700 1 -- 172.21.15.16:0/596993533 learned_addr learned my addr 172.21.15.16:0/596993533 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:18.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.193+0000 7fd607fff700 1 --2- 172.21.15.16:0/596993533 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd60806ba90 0x7fd6080c66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:18.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.193+0000 7fd60ce30700 1 --2- 172.21.15.16:0/596993533 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd60806b620 0x7fd6080b51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:18.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.194+0000 7fd60d631700 1 -- 172.21.15.16:0/596993533 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd60806b620 msgr2=0x7fd6080b51f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:18.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.194+0000 7fd60d631700 1 --2- 172.21.15.16:0/596993533 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd60806b620 0x7fd6080b51f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:18.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.194+0000 7fd60d631700 1 -- 172.21.15.16:0/596993533 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd60806ba90 msgr2=0x7fd6080c66c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:18.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.194+0000 7fd60d631700 1 --2- 172.21.15.16:0/596993533 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd60806ba90 0x7fd6080c66c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:18.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.194+0000 7fd60d631700 1 -- 172.21.15.16:0/596993533 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5fc0094a0 con 0x7fd6080d1000 2023-05-20T01:17:18.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.194+0000 7fd60d631700 1 --2- 172.21.15.16:0/596993533 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6080d1000 0x7fd6080c6c00 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fd600014040 tx=0x7fd600004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:18.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.194+0000 7fd6057fa700 1 -- 172.21.15.16:0/596993533 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd600012430 con 0x7fd6080d1000 2023-05-20T01:17:18.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.194+0000 7fd6057fa700 1 -- 172.21.15.16:0/596993533 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd600020070 con 0x7fd6080d1000 2023-05-20T01:17:18.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.194+0000 7fd60f094700 1 -- 172.21.15.16:0/596993533 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd600009a70 con 0x7fd6080d1000 2023-05-20T01:17:18.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.194+0000 7fd6057fa700 1 -- 172.21.15.16:0/596993533 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6000098f0 con 0x7fd6080d1000 2023-05-20T01:17:18.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.194+0000 7fd60f094700 1 -- 172.21.15.16:0/596993533 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6080ce070 con 0x7fd6080d1000 2023-05-20T01:17:18.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.195+0000 7fd60f094700 1 -- 172.21.15.16:0/596993533 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6080cdd90 con 0x7fd6080d1000 2023-05-20T01:17:18.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.195+0000 7fd6057fa700 1 -- 172.21.15.16:0/596993533 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd600004040 con 0x7fd6080d1000 2023-05-20T01:17:18.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.196+0000 7fd6057fa700 1 --2- 172.21.15.16:0/596993533 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd5ec06ace0 0x7fd5ec06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:18.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.196+0000 7fd6057fa700 1 -- 172.21.15.16:0/596993533 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fd600022070 con 0x7fd6080d1000 2023-05-20T01:17:18.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.196+0000 7fd60ce30700 1 --2- 172.21.15.16:0/596993533 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd5ec06ace0 0x7fd5ec06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:18.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.196+0000 7fd60ce30700 1 --2- 172.21.15.16:0/596993533 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd5ec06ace0 0x7fd5ec06d190 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc009d70 tx=0x7fd5fc009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:18.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.200+0000 7fd6057fa700 1 -- 172.21.15.16:0/596993533 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fd600056590 con 0x7fd6080d1000 2023-05-20T01:17:18.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.397+0000 7fd60f094700 1 -- 172.21.15.16:0/596993533 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd6080b2860 con 0x7fd6080d1000 2023-05-20T01:17:18.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.398+0000 7fd6057fa700 1 -- 172.21.15.16:0/596993533 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+20121 (secure 0 0 0) 0x7fd600059bb0 con 0x7fd6080d1000 2023-05-20T01:17:18.403 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:17:18.403 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":27,"fsid":"c2245cc5-ecab-4087-a59b-bc81db901676","created":"2023-05-20T01:16:09.549230+0000","modified":"2023-05-20T01:17:16.406712+0000","last_up_change":"2023-05-20T01:16:19.788624+0000","last_in_change":"2023-05-20T01:16:12.092160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-05-20T01:16:20.934366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-05-20T01:16:21.372294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-05-20T01:16:26.208287+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-05-20T01:16:27.292268+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-05-20T01:16:30.357285+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"317a83cd-19c8-4958-a301-6087ef342289","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6825","nonce":3923634179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6827","nonce":3923634179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6831","nonce":3923634179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6829","nonce":3923634179}]},"public_addr":"172.21.15.16:6825/3923634179","cluster_addr":"172.21.15.16:6827/3923634179","heartbeat_back_addr":"172.21.15.16:6831/3923634179","heartbeat_front_addr":"172.21.15.16:6829/3923634179","state":["exists","up"]},{"osd":1,"uuid":"5581046f-8219-4089-8d7d-b7846399505e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6809","nonce":3860264220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6811","nonce":3860264220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6815","nonce":3860264220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6813","nonce":3860264220}]},"public_addr":"172.21.15.16:6809/3860264220","cluster_addr":"172.21.15.16:6811/3860264220","heartbeat_back_addr":"172.21.15.16:6815/3860264220","heartbeat_front_addr":"172.21.15.16:6813/3860264220","state":["exists","up"]},{"osd":2,"uuid":"072e9e66-4133-478e-8d4a-64d46c2963f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6817","nonce":293187030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6819","nonce":293187030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6823","nonce":293187030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6821","nonce":293187030}]},"public_addr":"172.21.15.16:6817/293187030","cluster_addr":"172.21.15.16:6819/293187030","heartbeat_back_addr":"172.21.15.16:6823/293187030","heartbeat_front_addr":"172.21.15.16:6821/293187030","state":["exists","up"]},{"osd":3,"uuid":"11e1bba6-00f8-48dd-9028-d8d716b7f145","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6801","nonce":3669781890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6803","nonce":3669781890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6807","nonce":3669781890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6805","nonce":3669781890}]},"public_addr":"172.21.15.16:6801/3669781890","cluster_addr":"172.21.15.16:6803/3669781890","heartbeat_back_addr":"172.21.15.16:6807/3669781890","heartbeat_front_addr":"172.21.15.16:6805/3669781890","state":["exists","up"]},{"osd":4,"uuid":"b8909c3a-cf70-46cb-8d0b-009c707cf9d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6819","nonce":506317223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6823","nonce":506317223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6829","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6831","nonce":506317223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6827","nonce":506317223}]},"public_addr":"172.21.15.132:6819/506317223","cluster_addr":"172.21.15.132:6823/506317223","heartbeat_back_addr":"172.21.15.132:6831/506317223","heartbeat_front_addr":"172.21.15.132:6827/506317223","state":["exists","up"]},{"osd":5,"uuid":"b1818304-4583-47bf-a6ec-0bfbb5166233","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6818","nonce":1844689995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6822","nonce":1844689995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6830","nonce":1844689995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6826","nonce":1844689995}]},"public_addr":"172.21.15.132:6818/1844689995","cluster_addr":"172.21.15.132:6822/1844689995","heartbeat_back_addr":"172.21.15.132:6830/1844689995","heartbeat_front_addr":"172.21.15.132:6826/1844689995","state":["exists","up"]},{"osd":6,"uuid":"d37f4691-eed1-4a93-8818-18354f03f4a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6801","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6802","nonce":727581123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6805","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6807","nonce":727581123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6813","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6815","nonce":727581123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6811","nonce":727581123}]},"public_addr":"172.21.15.132:6802/727581123","cluster_addr":"172.21.15.132:6807/727581123","heartbeat_back_addr":"172.21.15.132:6815/727581123","heartbeat_front_addr":"172.21.15.132:6811/727581123","state":["exists","up"]},{"osd":7,"uuid":"4f712f62-5d1d-4816-8042-03209144f8b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6803","nonce":466001149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6806","nonce":466001149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6814","nonce":466001149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6810","nonce":466001149}]},"public_addr":"172.21.15.132:6803/466001149","cluster_addr":"172.21.15.132:6806/466001149","heartbeat_back_addr":"172.21.15.132:6814/466001149","heartbeat_front_addr":"172.21.15.132:6810/466001149","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.338863+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.278680+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305694+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.246835+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.276237+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.304441+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305421+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.267050+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.a","mappings":[{"from":1,"to":5}]},{"pgid":"4.11","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-20T01:17:18.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.401+0000 7fd60f094700 1 -- 172.21.15.16:0/596993533 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd5ec06ace0 msgr2=0x7fd5ec06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:18.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.401+0000 7fd60f094700 1 --2- 172.21.15.16:0/596993533 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd5ec06ace0 0x7fd5ec06d190 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc009d70 tx=0x7fd5fc009210 comp rx=0 tx=0).stop 2023-05-20T01:17:18.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.401+0000 7fd60f094700 1 -- 172.21.15.16:0/596993533 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6080d1000 msgr2=0x7fd6080c6c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:18.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.401+0000 7fd60f094700 1 --2- 172.21.15.16:0/596993533 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6080d1000 0x7fd6080c6c00 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fd600014040 tx=0x7fd600004ad0 comp rx=0 tx=0).stop 2023-05-20T01:17:18.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.401+0000 7fd60f094700 1 -- 172.21.15.16:0/596993533 shutdown_connections 2023-05-20T01:17:18.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.401+0000 7fd60f094700 1 --2- 172.21.15.16:0/596993533 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6080d1000 0x7fd6080c6c00 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:18.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.401+0000 7fd60f094700 1 --2- 172.21.15.16:0/596993533 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd5ec06ace0 0x7fd5ec06d190 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:18.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.401+0000 7fd60f094700 1 --2- 172.21.15.16:0/596993533 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd60806b620 0x7fd6080b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:18.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.401+0000 7fd60f094700 1 --2- 172.21.15.16:0/596993533 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd60806ba90 0x7fd6080c66c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:18.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.401+0000 7fd60f094700 1 -- 172.21.15.16:0/596993533 >> 172.21.15.16:0/596993533 conn(0x7fd60806ae30 msgr2=0x7fd60806fbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:18.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.401+0000 7fd60f094700 1 -- 172.21.15.16:0/596993533 shutdown_connections 2023-05-20T01:17:18.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.401+0000 7fd60f094700 1 -- 172.21.15.16:0/596993533 wait complete. 2023-05-20T01:17:18.418 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-05-20T01:16:20.934366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-05-20T01:16:21.372294+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-05-20T01:16:26.208287+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-05-20T01:16:27.292268+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-05-20T01:16:30.357285+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-05-20T01:17:18.419 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-05-20T01:17:18.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.544+0000 7f663dfa6700 1 Processor -- start 2023-05-20T01:17:18.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.544+0000 7f663dfa6700 1 -- start start 2023-05-20T01:17:18.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.545+0000 7f663dfa6700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f663815cd20 0x7f66380b1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:18.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.545+0000 7f663dfa6700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f66380b14c0 0x7f663806b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:18.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.545+0000 7f663dfa6700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f663806bb60 0x7f66380cd930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:18.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.545+0000 7f663dfa6700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f663806bfc0 con 0x7f66380b14c0 2023-05-20T01:17:18.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.545+0000 7f663dfa6700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66380cb700 con 0x7f663815cd20 2023-05-20T01:17:18.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.545+0000 7f663dfa6700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66380b2430 con 0x7f663806bb60 2023-05-20T01:17:18.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.545+0000 7f6636ffd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f66380b14c0 0x7f663806b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:18.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.545+0000 7f6636ffd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f66380b14c0 0x7f663806b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59188/0 (socket says 172.21.15.16:59188) 2023-05-20T01:17:18.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.545+0000 7f6636ffd700 1 -- 172.21.15.16:0/4106586754 learned_addr learned my addr 172.21.15.16:0/4106586754 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:18.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.545+0000 7f6637fff700 1 --2- 172.21.15.16:0/4106586754 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f663806bb60 0x7f66380cd930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:18.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.545+0000 7f66377fe700 1 --2- 172.21.15.16:0/4106586754 >> v2:172.21.15.132:3300/0 conn(0x7f663815cd20 0x7f66380b1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:18.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.545+0000 7f6636ffd700 1 -- 172.21.15.16:0/4106586754 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f663806bb60 msgr2=0x7f66380cd930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:18.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.545+0000 7f6636ffd700 1 --2- 172.21.15.16:0/4106586754 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f663806bb60 0x7f66380cd930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:18.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.545+0000 7f6636ffd700 1 -- 172.21.15.16:0/4106586754 >> v2:172.21.15.132:3300/0 conn(0x7f663815cd20 msgr2=0x7f66380b1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:18.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.545+0000 7f6636ffd700 1 --2- 172.21.15.16:0/4106586754 >> v2:172.21.15.132:3300/0 conn(0x7f663815cd20 0x7f66380b1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:18.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.545+0000 7f6636ffd700 1 -- 172.21.15.16:0/4106586754 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66380b2710 con 0x7f66380b14c0 2023-05-20T01:17:18.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.545+0000 7f6637fff700 1 --2- 172.21.15.16:0/4106586754 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f663806bb60 0x7f66380cd930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:17:18.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.545+0000 7f6636ffd700 1 --2- 172.21.15.16:0/4106586754 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66380b14c0 0x7f663806b620 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f66380cebd0 tx=0x7f662c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97619dc735f718fe server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:18.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.545+0000 7f66367fc700 1 -- 172.21.15.16:0/4106586754 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f662c00b870 con 0x7f66380b14c0 2023-05-20T01:17:18.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.545+0000 7f66367fc700 1 -- 172.21.15.16:0/4106586754 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f662c00b9d0 con 0x7f66380b14c0 2023-05-20T01:17:18.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.545+0000 7f66367fc700 1 -- 172.21.15.16:0/4106586754 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f662c00bd60 con 0x7f66380b14c0 2023-05-20T01:17:18.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.546+0000 7f663dfa6700 1 -- 172.21.15.16:0/4106586754 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66380b14c0 msgr2=0x7f663806b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:18.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.546+0000 7f663dfa6700 1 --2- 172.21.15.16:0/4106586754 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66380b14c0 0x7f663806b620 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f66380cebd0 tx=0x7f662c009bd0 comp rx=0 tx=0).stop 2023-05-20T01:17:18.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.546+0000 7f663dfa6700 1 -- 172.21.15.16:0/4106586754 shutdown_connections 2023-05-20T01:17:18.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.546+0000 7f663dfa6700 1 --2- 172.21.15.16:0/4106586754 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f663806bb60 0x7f66380cd930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:18.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.546+0000 7f663dfa6700 1 --2- 172.21.15.16:0/4106586754 >> v2:172.21.15.132:3300/0 conn(0x7f663815cd20 0x7f66380b1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:18.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.546+0000 7f663dfa6700 1 --2- 172.21.15.16:0/4106586754 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66380b14c0 0x7f663806b620 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:18.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.546+0000 7f663dfa6700 1 -- 172.21.15.16:0/4106586754 >> 172.21.15.16:0/4106586754 conn(0x7f663806ae30 msgr2=0x7f663809a400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:18.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.546+0000 7f663dfa6700 1 -- 172.21.15.16:0/4106586754 shutdown_connections 2023-05-20T01:17:18.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.546+0000 7f663dfa6700 1 -- 172.21.15.16:0/4106586754 wait complete. 2023-05-20T01:17:18.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.547+0000 7f663dfa6700 1 Processor -- start 2023-05-20T01:17:18.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.547+0000 7f663dfa6700 1 -- start start 2023-05-20T01:17:18.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.548+0000 7f663dfa6700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f663806bb60 0x7f6638156860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:18.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.548+0000 7f663dfa6700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f66380b14c0 0x7f663815d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:18.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.548+0000 7f663dfa6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f663815cd20 0x7f66380cd990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:18.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.548+0000 7f663dfa6700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f663815d630 con 0x7f663815cd20 2023-05-20T01:17:18.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.548+0000 7f663dfa6700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f663815d8d0 con 0x7f66380b14c0 2023-05-20T01:17:18.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.548+0000 7f663dfa6700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f663815dbb0 con 0x7f663806bb60 2023-05-20T01:17:18.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.548+0000 7f6637fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f663815cd20 0x7f66380cd990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:18.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.548+0000 7f6637fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f663815cd20 0x7f66380cd990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59196/0 (socket says 172.21.15.16:59196) 2023-05-20T01:17:18.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.548+0000 7f6637fff700 1 -- 172.21.15.16:0/763654572 learned_addr learned my addr 172.21.15.16:0/763654572 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:18.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.548+0000 7f66377fe700 1 --2- 172.21.15.16:0/763654572 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f663806bb60 0x7f6638156860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:18.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.548+0000 7f6636ffd700 1 --2- 172.21.15.16:0/763654572 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f66380b14c0 0x7f663815d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:18.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.549+0000 7f6637fff700 1 -- 172.21.15.16:0/763654572 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f663806bb60 msgr2=0x7f6638156860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:18.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.549+0000 7f6637fff700 1 --2- 172.21.15.16:0/763654572 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f663806bb60 0x7f6638156860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:18.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.549+0000 7f6637fff700 1 -- 172.21.15.16:0/763654572 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f66380b14c0 msgr2=0x7f663815d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:18.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.549+0000 7f6637fff700 1 --2- 172.21.15.16:0/763654572 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f66380b14c0 0x7f663815d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:18.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.549+0000 7f6637fff700 1 -- 172.21.15.16:0/763654572 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f662c009550 con 0x7f663815cd20 2023-05-20T01:17:18.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.549+0000 7f6637fff700 1 --2- 172.21.15.16:0/763654572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f663815cd20 0x7f66380cd990 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f6630007db0 tx=0x7f663000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:18.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.549+0000 7f661ffff700 1 -- 172.21.15.16:0/763654572 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f663000dd20 con 0x7f663815cd20 2023-05-20T01:17:18.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.549+0000 7f661ffff700 1 -- 172.21.15.16:0/763654572 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f663000f040 con 0x7f663815cd20 2023-05-20T01:17:18.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.549+0000 7f661ffff700 1 -- 172.21.15.16:0/763654572 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f663000dea0 con 0x7f663815cd20 2023-05-20T01:17:18.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.549+0000 7f663dfa6700 1 -- 172.21.15.16:0/763654572 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66380cbc30 con 0x7f663815cd20 2023-05-20T01:17:18.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.549+0000 7f663dfa6700 1 -- 172.21.15.16:0/763654572 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66380cb950 con 0x7f663815cd20 2023-05-20T01:17:18.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.550+0000 7f663dfa6700 1 -- 172.21.15.16:0/763654572 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66380b1960 con 0x7f663815cd20 2023-05-20T01:17:18.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.550+0000 7f661ffff700 1 -- 172.21.15.16:0/763654572 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6630003c60 con 0x7f663815cd20 2023-05-20T01:17:18.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.551+0000 7f661ffff700 1 --2- 172.21.15.16:0/763654572 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f662406ace0 0x7f662406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:18.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.551+0000 7f66377fe700 1 --2- 172.21.15.16:0/763654572 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f662406ace0 0x7f662406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:18.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.551+0000 7f661ffff700 1 -- 172.21.15.16:0/763654572 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f6630089750 con 0x7f663815cd20 2023-05-20T01:17:18.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.551+0000 7f66377fe700 1 --2- 172.21.15.16:0/763654572 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f662406ace0 0x7f662406d190 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6638073c10 tx=0x7f6628008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:18.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.555+0000 7f661ffff700 1 -- 172.21.15.16:0/763654572 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f66300544c0 con 0x7f663815cd20 2023-05-20T01:17:18.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.751+0000 7f663dfa6700 1 -- 172.21.15.16:0/763654572 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f66380d1f60 con 0x7f663815cd20 2023-05-20T01:17:18.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.751+0000 7f661ffff700 1 -- 172.21.15.16:0/763654572 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v27) v1 ==== 92+0+10 (secure 0 0 0) 0x7f6630057ae0 con 0x7f663815cd20 2023-05-20T01:17:18.755 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 8 2023-05-20T01:17:18.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.753+0000 7f663dfa6700 1 -- 172.21.15.16:0/763654572 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f662406ace0 msgr2=0x7f662406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:18.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.753+0000 7f663dfa6700 1 --2- 172.21.15.16:0/763654572 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f662406ace0 0x7f662406d190 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6638073c10 tx=0x7f6628008040 comp rx=0 tx=0).stop 2023-05-20T01:17:18.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.753+0000 7f663dfa6700 1 -- 172.21.15.16:0/763654572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f663815cd20 msgr2=0x7f66380cd990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:18.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.753+0000 7f663dfa6700 1 --2- 172.21.15.16:0/763654572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f663815cd20 0x7f66380cd990 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f6630007db0 tx=0x7f663000d3b0 comp rx=0 tx=0).stop 2023-05-20T01:17:18.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.753+0000 7f663dfa6700 1 -- 172.21.15.16:0/763654572 shutdown_connections 2023-05-20T01:17:18.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.753+0000 7f663dfa6700 1 --2- 172.21.15.16:0/763654572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f663815cd20 0x7f66380cd990 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:18.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.753+0000 7f663dfa6700 1 --2- 172.21.15.16:0/763654572 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f662406ace0 0x7f662406d190 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:18.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.753+0000 7f663dfa6700 1 --2- 172.21.15.16:0/763654572 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f663806bb60 0x7f6638156860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:18.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.753+0000 7f663dfa6700 1 --2- 172.21.15.16:0/763654572 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f66380b14c0 0x7f663815d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:18.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.753+0000 7f663dfa6700 1 -- 172.21.15.16:0/763654572 >> 172.21.15.16:0/763654572 conn(0x7f663806ae30 msgr2=0x7f66380afd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:18.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.753+0000 7f663dfa6700 1 -- 172.21.15.16:0/763654572 shutdown_connections 2023-05-20T01:17:18.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.753+0000 7f663dfa6700 1 -- 172.21.15.16:0/763654572 wait complete. 2023-05-20T01:17:18.769 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-05-20T01:17:18.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.894+0000 7fdffb2a8700 1 Processor -- start 2023-05-20T01:17:18.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.894+0000 7fdffb2a8700 1 -- start start 2023-05-20T01:17:18.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.895+0000 7fdffb2a8700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdff40cf840 0x7fdff40b2450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:18.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.895+0000 7fdffb2a8700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fdff4160590 0x7fdff406a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:18.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.895+0000 7fdffb2a8700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdff406e420 con 0x7fdff40cfc10 2023-05-20T01:17:18.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.895+0000 7fdffb2a8700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdff40b2990 con 0x7fdff4160590 2023-05-20T01:17:18.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.895+0000 7fdffb2a8700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdff40714b0 con 0x7fdff40cf840 2023-05-20T01:17:18.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.895+0000 7fdff9845700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fdff4160590 0x7fdff406a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:18.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.895+0000 7fdff9845700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fdff4160590 0x7fdff406a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59204/0 (socket says 172.21.15.16:59204) 2023-05-20T01:17:18.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.895+0000 7fdff9845700 1 -- 172.21.15.16:0/2914302892 learned_addr learned my addr 172.21.15.16:0/2914302892 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:18.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.895+0000 7fdff9044700 1 --2- 172.21.15.16:0/2914302892 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdff40cf840 0x7fdff40b2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:18.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.896+0000 7fdff9845700 1 -- 172.21.15.16:0/2914302892 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdff40cf840 msgr2=0x7fdff40b2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:18.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.896+0000 7fdff9845700 1 --2- 172.21.15.16:0/2914302892 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdff40cf840 0x7fdff40b2450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:18.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.896+0000 7fdff9845700 1 -- 172.21.15.16:0/2914302892 >> v1:172.21.15.132:6789/0 conn(0x7fdff40cfc10 legacy=0x7fdff40ceab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:18.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.896+0000 7fdff9845700 1 -- 172.21.15.16:0/2914302892 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdff4160ae0 con 0x7fdff4160590 2023-05-20T01:17:18.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.896+0000 7fdff9845700 1 --2- 172.21.15.16:0/2914302892 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdff4160590 0x7fdff406a510 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fdfec009e50 tx=0x7fdfec00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad7c2dbd314faec6 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:18.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.896+0000 7fdfebfff700 1 -- 172.21.15.16:0/2914302892 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfec00e070 con 0x7fdff4160590 2023-05-20T01:17:18.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.896+0000 7fdfebfff700 1 -- 172.21.15.16:0/2914302892 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfec003a20 con 0x7fdff4160590 2023-05-20T01:17:18.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.896+0000 7fdfebfff700 1 -- 172.21.15.16:0/2914302892 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfec003dd0 con 0x7fdff4160590 2023-05-20T01:17:18.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.896+0000 7fdffb2a8700 1 -- 172.21.15.16:0/2914302892 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdff4160590 msgr2=0x7fdff406a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:18.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.896+0000 7fdffb2a8700 1 --2- 172.21.15.16:0/2914302892 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdff4160590 0x7fdff406a510 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fdfec009e50 tx=0x7fdfec00b330 comp rx=0 tx=0).stop 2023-05-20T01:17:18.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.897+0000 7fdffb2a8700 1 -- 172.21.15.16:0/2914302892 shutdown_connections 2023-05-20T01:17:18.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.897+0000 7fdffb2a8700 1 --2- 172.21.15.16:0/2914302892 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdff4160590 0x7fdff406a510 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:18.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.897+0000 7fdffb2a8700 1 --2- 172.21.15.16:0/2914302892 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdff40cf840 0x7fdff40b2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:18.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.897+0000 7fdffb2a8700 1 -- 172.21.15.16:0/2914302892 >> 172.21.15.16:0/2914302892 conn(0x7fdff40d0010 msgr2=0x7fdff40d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:18.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.897+0000 7fdffb2a8700 1 -- 172.21.15.16:0/2914302892 shutdown_connections 2023-05-20T01:17:18.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.897+0000 7fdffb2a8700 1 -- 172.21.15.16:0/2914302892 wait complete. 2023-05-20T01:17:18.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.898+0000 7fdffb2a8700 1 Processor -- start 2023-05-20T01:17:18.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.898+0000 7fdffb2a8700 1 -- start start 2023-05-20T01:17:18.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.899+0000 7fdffb2a8700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdff40cf840 0x7fdff41f2350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:18.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.899+0000 7fdffb2a8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdff40cfc10 0x7fdff4159630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:18.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.899+0000 7fdffb2a8700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdff4160590 0x7fdff4159b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:18.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.899+0000 7fdffb2a8700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdff40c5550 con 0x7fdff40cfc10 2023-05-20T01:17:18.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.899+0000 7fdffb2a8700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdff4152240 con 0x7fdff40cf840 2023-05-20T01:17:18.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.899+0000 7fdffb2a8700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdff4152520 con 0x7fdff4160590 2023-05-20T01:17:18.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.899+0000 7fdff8843700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdff40cfc10 0x7fdff4159630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:18.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.899+0000 7fdff8843700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdff40cfc10 0x7fdff4159630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59208/0 (socket says 172.21.15.16:59208) 2023-05-20T01:17:18.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.899+0000 7fdff8843700 1 -- 172.21.15.16:0/632206383 learned_addr learned my addr 172.21.15.16:0/632206383 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:18.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.899+0000 7fdff9845700 1 --2- 172.21.15.16:0/632206383 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdff4160590 0x7fdff4159b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:18.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.899+0000 7fdff9044700 1 --2- 172.21.15.16:0/632206383 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdff40cf840 0x7fdff41f2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:18.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.899+0000 7fdff8843700 1 -- 172.21.15.16:0/632206383 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdff4160590 msgr2=0x7fdff4159b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:18.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.899+0000 7fdff8843700 1 --2- 172.21.15.16:0/632206383 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdff4160590 0x7fdff4159b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:18.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.899+0000 7fdff8843700 1 -- 172.21.15.16:0/632206383 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdff40cf840 msgr2=0x7fdff41f2350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:18.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.899+0000 7fdff8843700 1 --2- 172.21.15.16:0/632206383 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdff40cf840 0x7fdff41f2350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:18.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.899+0000 7fdff8843700 1 -- 172.21.15.16:0/632206383 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfec009b30 con 0x7fdff40cfc10 2023-05-20T01:17:18.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.899+0000 7fdff8843700 1 --2- 172.21.15.16:0/632206383 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdff40cfc10 0x7fdff4159630 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fdff000c4a0 tx=0x7fdff000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:18.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.899+0000 7fdfe9ffb700 1 -- 172.21.15.16:0/632206383 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdff0011790 con 0x7fdff40cfc10 2023-05-20T01:17:18.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.899+0000 7fdfe9ffb700 1 -- 172.21.15.16:0/632206383 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdff00118f0 con 0x7fdff40cfc10 2023-05-20T01:17:18.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.899+0000 7fdfe9ffb700 1 -- 172.21.15.16:0/632206383 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdff0011ca0 con 0x7fdff40cfc10 2023-05-20T01:17:18.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.899+0000 7fdffb2a8700 1 -- 172.21.15.16:0/632206383 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdff41610a0 con 0x7fdff40cfc10 2023-05-20T01:17:18.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.900+0000 7fdffb2a8700 1 -- 172.21.15.16:0/632206383 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdff4160dc0 con 0x7fdff40cfc10 2023-05-20T01:17:18.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.901+0000 7fdffb2a8700 1 -- 172.21.15.16:0/632206383 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdff4161380 con 0x7fdff40cfc10 2023-05-20T01:17:18.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.901+0000 7fdfe9ffb700 1 -- 172.21.15.16:0/632206383 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fdff001c460 con 0x7fdff40cfc10 2023-05-20T01:17:18.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.901+0000 7fdfe9ffb700 1 --2- 172.21.15.16:0/632206383 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fdfd806ace0 0x7fdfd806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:18.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.902+0000 7fdff9044700 1 --2- 172.21.15.16:0/632206383 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fdfd806ace0 0x7fdfd806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:18.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.902+0000 7fdfe9ffb700 1 -- 172.21.15.16:0/632206383 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fdff00896a0 con 0x7fdff40cfc10 2023-05-20T01:17:18.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.902+0000 7fdff9044700 1 --2- 172.21.15.16:0/632206383 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fdfd806ace0 0x7fdfd806d190 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fdfe4009950 tx=0x7fdfe4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:18.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:18.906+0000 7fdfe9ffb700 1 -- 172.21.15.16:0/632206383 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fdff0054380 con 0x7fdff40cfc10 2023-05-20T01:17:19.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.102+0000 7fdffb2a8700 1 -- 172.21.15.16:0/632206383 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdff40c69c0 con 0x7fdff40cfc10 2023-05-20T01:17:19.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.103+0000 7fdfe9ffb700 1 -- 172.21.15.16:0/632206383 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdff00579a0 con 0x7fdff40cfc10 2023-05-20T01:17:19.107 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-05-20T01:17:19.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.105+0000 7fdffb2a8700 1 -- 172.21.15.16:0/632206383 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fdfd806ace0 msgr2=0x7fdfd806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:19.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.105+0000 7fdffb2a8700 1 --2- 172.21.15.16:0/632206383 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fdfd806ace0 0x7fdfd806d190 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fdfe4009950 tx=0x7fdfe4008040 comp rx=0 tx=0).stop 2023-05-20T01:17:19.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.105+0000 7fdffb2a8700 1 -- 172.21.15.16:0/632206383 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdff40cfc10 msgr2=0x7fdff4159630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:19.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.105+0000 7fdffb2a8700 1 --2- 172.21.15.16:0/632206383 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdff40cfc10 0x7fdff4159630 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fdff000c4a0 tx=0x7fdff000c860 comp rx=0 tx=0).stop 2023-05-20T01:17:19.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.105+0000 7fdffb2a8700 1 -- 172.21.15.16:0/632206383 shutdown_connections 2023-05-20T01:17:19.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.105+0000 7fdffb2a8700 1 --2- 172.21.15.16:0/632206383 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fdfd806ace0 0x7fdfd806d190 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:19.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.105+0000 7fdffb2a8700 1 --2- 172.21.15.16:0/632206383 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdff4160590 0x7fdff4159b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:19.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.105+0000 7fdffb2a8700 1 --2- 172.21.15.16:0/632206383 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdff40cf840 0x7fdff41f2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:19.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.105+0000 7fdffb2a8700 1 --2- 172.21.15.16:0/632206383 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdff40cfc10 0x7fdff4159630 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:19.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.105+0000 7fdffb2a8700 1 -- 172.21.15.16:0/632206383 >> 172.21.15.16:0/632206383 conn(0x7fdff40d0010 msgr2=0x7fdff406e6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:19.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.105+0000 7fdffb2a8700 1 -- 172.21.15.16:0/632206383 shutdown_connections 2023-05-20T01:17:19.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.105+0000 7fdffb2a8700 1 -- 172.21.15.16:0/632206383 wait complete. 2023-05-20T01:17:19.120 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-05-20T01:17:19.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.244+0000 7f9d9f9d4700 1 Processor -- start 2023-05-20T01:17:19.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.244+0000 7f9d9f9d4700 1 -- start start 2023-05-20T01:17:19.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.245+0000 7f9d9f9d4700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9d9815cec0 0x7f9d980cd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:19.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.245+0000 7f9d9f9d4700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9d9806e190 0x7f9d980cf1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:19.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.245+0000 7f9d9f9d4700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d9806dd00 con 0x7f9d9806a510 2023-05-20T01:17:19.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.245+0000 7f9d9f9d4700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d980cfa60 con 0x7f9d9806e190 2023-05-20T01:17:19.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.245+0000 7f9d9f9d4700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d9806d6b0 con 0x7f9d9815cec0 2023-05-20T01:17:19.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.245+0000 7f9d9cf6f700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9d9806e190 0x7f9d980cf1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:19.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.245+0000 7f9d9cf6f700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9d9806e190 0x7f9d980cf1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59214/0 (socket says 172.21.15.16:59214) 2023-05-20T01:17:19.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.245+0000 7f9d9cf6f700 1 -- 172.21.15.16:0/3363309941 learned_addr learned my addr 172.21.15.16:0/3363309941 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:19.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.246+0000 7f9d9d770700 1 --2- 172.21.15.16:0/3363309941 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9d9815cec0 0x7f9d980cd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:19.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.246+0000 7f9d9cf6f700 1 -- 172.21.15.16:0/3363309941 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9d9815cec0 msgr2=0x7f9d980cd4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:19.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.246+0000 7f9d9cf6f700 1 --2- 172.21.15.16:0/3363309941 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9d9815cec0 0x7f9d980cd4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:19.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.246+0000 7f9d9cf6f700 1 -- 172.21.15.16:0/3363309941 >> v1:172.21.15.132:6789/0 conn(0x7f9d9806a510 legacy=0x7f9d980710d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:19.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.246+0000 7f9d9cf6f700 1 -- 172.21.15.16:0/3363309941 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d9806d990 con 0x7f9d9806e190 2023-05-20T01:17:19.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.246+0000 7f9d9d770700 1 --2- 172.21.15.16:0/3363309941 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9d9815cec0 0x7f9d980cd4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:17:19.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.246+0000 7f9d9cf6f700 1 --2- 172.21.15.16:0/3363309941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d9806e190 0x7f9d980cf1f0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f9d940098f0 tx=0x7f9d94009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a326525ec53cdf5b server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:19.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.246+0000 7f9d8ffff700 1 -- 172.21.15.16:0/3363309941 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d94004230 con 0x7f9d9806e190 2023-05-20T01:17:19.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.246+0000 7f9d8ffff700 1 -- 172.21.15.16:0/3363309941 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d94004390 con 0x7f9d9806e190 2023-05-20T01:17:19.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.246+0000 7f9d8ffff700 1 -- 172.21.15.16:0/3363309941 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d940036a0 con 0x7f9d9806e190 2023-05-20T01:17:19.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.246+0000 7f9d9f9d4700 1 -- 172.21.15.16:0/3363309941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d9806e190 msgr2=0x7f9d980cf1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:19.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.246+0000 7f9d9f9d4700 1 --2- 172.21.15.16:0/3363309941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d9806e190 0x7f9d980cf1f0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f9d940098f0 tx=0x7f9d94009c00 comp rx=0 tx=0).stop 2023-05-20T01:17:19.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.246+0000 7f9d9f9d4700 1 -- 172.21.15.16:0/3363309941 shutdown_connections 2023-05-20T01:17:19.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.247+0000 7f9d9f9d4700 1 --2- 172.21.15.16:0/3363309941 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9d9815cec0 0x7f9d980cd4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:19.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.247+0000 7f9d9f9d4700 1 --2- 172.21.15.16:0/3363309941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d9806e190 0x7f9d980cf1f0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:19.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.247+0000 7f9d9f9d4700 1 -- 172.21.15.16:0/3363309941 >> 172.21.15.16:0/3363309941 conn(0x7f9d9806c260 msgr2=0x7f9d980cffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:19.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.247+0000 7f9d9f9d4700 1 -- 172.21.15.16:0/3363309941 shutdown_connections 2023-05-20T01:17:19.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.247+0000 7f9d9f9d4700 1 -- 172.21.15.16:0/3363309941 wait complete. 2023-05-20T01:17:19.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.248+0000 7f9d9f9d4700 1 Processor -- start 2023-05-20T01:17:19.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.248+0000 7f9d9f9d4700 1 -- start start 2023-05-20T01:17:19.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.248+0000 7f9d9f9d4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d9806a510 0x7f9d980c6660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:19.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.249+0000 7f9d9f9d4700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9d9806e190 0x7f9d980af960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:19.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.249+0000 7f9d9f9d4700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9d9815cec0 0x7f9d9814a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:19.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.249+0000 7f9d9f9d4700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d980afea0 con 0x7f9d9806a510 2023-05-20T01:17:19.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.249+0000 7f9d9f9d4700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d980b21f0 con 0x7f9d9806e190 2023-05-20T01:17:19.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.249+0000 7f9d9f9d4700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d980b24d0 con 0x7f9d9815cec0 2023-05-20T01:17:19.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.249+0000 7f9d9d770700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d9806a510 0x7f9d980c6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:19.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.249+0000 7f9d9d770700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d9806a510 0x7f9d980c6660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59220/0 (socket says 172.21.15.16:59220) 2023-05-20T01:17:19.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.249+0000 7f9d9d770700 1 -- 172.21.15.16:0/3942666791 learned_addr learned my addr 172.21.15.16:0/3942666791 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:19.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.249+0000 7f9d9df71700 1 --2- 172.21.15.16:0/3942666791 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9d9815cec0 0x7f9d9814a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:19.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.249+0000 7f9d9cf6f700 1 --2- 172.21.15.16:0/3942666791 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9d9806e190 0x7f9d980af960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:19.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.249+0000 7f9d9d770700 1 -- 172.21.15.16:0/3942666791 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9d9815cec0 msgr2=0x7f9d9814a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:19.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.249+0000 7f9d9d770700 1 --2- 172.21.15.16:0/3942666791 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9d9815cec0 0x7f9d9814a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:19.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.249+0000 7f9d9d770700 1 -- 172.21.15.16:0/3942666791 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9d9806e190 msgr2=0x7f9d980af960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:19.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.249+0000 7f9d9d770700 1 --2- 172.21.15.16:0/3942666791 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9d9806e190 0x7f9d980af960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:19.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.249+0000 7f9d9d770700 1 -- 172.21.15.16:0/3942666791 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d94009550 con 0x7f9d9806a510 2023-05-20T01:17:19.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.249+0000 7f9d9df71700 1 --2- 172.21.15.16:0/3942666791 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9d9815cec0 0x7f9d9814a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:17:19.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.249+0000 7f9d9d770700 1 --2- 172.21.15.16:0/3942666791 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d9806a510 0x7f9d980c6660 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f9d8800c750 tx=0x7f9d8800cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:19.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.249+0000 7f9d8dffb700 1 -- 172.21.15.16:0/3942666791 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d88011800 con 0x7f9d9806a510 2023-05-20T01:17:19.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.249+0000 7f9d8dffb700 1 -- 172.21.15.16:0/3942666791 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d88011960 con 0x7f9d9806a510 2023-05-20T01:17:19.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.249+0000 7f9d8dffb700 1 -- 172.21.15.16:0/3942666791 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d88011cd0 con 0x7f9d9806a510 2023-05-20T01:17:19.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.249+0000 7f9d9f9d4700 1 -- 172.21.15.16:0/3942666791 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d980cfd40 con 0x7f9d9806a510 2023-05-20T01:17:19.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.250+0000 7f9d9f9d4700 1 -- 172.21.15.16:0/3942666791 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d9815d290 con 0x7f9d9806a510 2023-05-20T01:17:19.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.251+0000 7f9d9f9d4700 1 -- 172.21.15.16:0/3942666791 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d980cf730 con 0x7f9d9806a510 2023-05-20T01:17:19.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.251+0000 7f9d8dffb700 1 -- 172.21.15.16:0/3942666791 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9d88021350 con 0x7f9d9806a510 2023-05-20T01:17:19.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.251+0000 7f9d8dffb700 1 --2- 172.21.15.16:0/3942666791 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f9d7c06ace0 0x7f9d7c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:19.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.251+0000 7f9d8dffb700 1 -- 172.21.15.16:0/3942666791 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f9d88089720 con 0x7f9d9806a510 2023-05-20T01:17:19.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.251+0000 7f9d9cf6f700 1 --2- 172.21.15.16:0/3942666791 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f9d7c06ace0 0x7f9d7c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:19.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.252+0000 7f9d9cf6f700 1 --2- 172.21.15.16:0/3942666791 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f9d7c06ace0 0x7f9d7c06d190 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f9d980ca4f0 tx=0x7f9d94008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:19.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.256+0000 7f9d8dffb700 1 -- 172.21.15.16:0/3942666791 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f9d88054530 con 0x7f9d9806a510 2023-05-20T01:17:19.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.454+0000 7f9d9f9d4700 1 -- 172.21.15.16:0/3942666791 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9d980747b0 con 0x7f9d9806a510 2023-05-20T01:17:19.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.454+0000 7f9d8dffb700 1 -- 172.21.15.16:0/3942666791 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9d88057b50 con 0x7f9d9806a510 2023-05-20T01:17:19.458 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-05-20T01:17:19.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.456+0000 7f9d9f9d4700 1 -- 172.21.15.16:0/3942666791 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f9d7c06ace0 msgr2=0x7f9d7c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:19.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.456+0000 7f9d9f9d4700 1 --2- 172.21.15.16:0/3942666791 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f9d7c06ace0 0x7f9d7c06d190 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f9d980ca4f0 tx=0x7f9d94008030 comp rx=0 tx=0).stop 2023-05-20T01:17:19.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.456+0000 7f9d9f9d4700 1 -- 172.21.15.16:0/3942666791 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d9806a510 msgr2=0x7f9d980c6660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:19.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.456+0000 7f9d9f9d4700 1 --2- 172.21.15.16:0/3942666791 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d9806a510 0x7f9d980c6660 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f9d8800c750 tx=0x7f9d8800cb10 comp rx=0 tx=0).stop 2023-05-20T01:17:19.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.456+0000 7f9d9f9d4700 1 -- 172.21.15.16:0/3942666791 shutdown_connections 2023-05-20T01:17:19.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.456+0000 7f9d9f9d4700 1 --2- 172.21.15.16:0/3942666791 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f9d7c06ace0 0x7f9d7c06d190 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:19.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.456+0000 7f9d9f9d4700 1 --2- 172.21.15.16:0/3942666791 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9d9815cec0 0x7f9d9814a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:19.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.456+0000 7f9d9f9d4700 1 --2- 172.21.15.16:0/3942666791 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d9806a510 0x7f9d980c6660 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:19.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.456+0000 7f9d9f9d4700 1 --2- 172.21.15.16:0/3942666791 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9d9806e190 0x7f9d980af960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:19.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.456+0000 7f9d9f9d4700 1 -- 172.21.15.16:0/3942666791 >> 172.21.15.16:0/3942666791 conn(0x7f9d9806c260 msgr2=0x7f9d9815c340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:19.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.456+0000 7f9d9f9d4700 1 -- 172.21.15.16:0/3942666791 shutdown_connections 2023-05-20T01:17:19.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.456+0000 7f9d9f9d4700 1 -- 172.21.15.16:0/3942666791 wait complete. 2023-05-20T01:17:19.472 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-05-20T01:17:19.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.596+0000 7f49bfe15700 1 Processor -- start 2023-05-20T01:17:19.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.596+0000 7f49bfe15700 1 -- start start 2023-05-20T01:17:19.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.597+0000 7f49bfe15700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f49b815cec0 0x7f49b8071bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:19.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.597+0000 7f49bfe15700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f49b815d290 0x7f49b8072130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:19.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.597+0000 7f49bfe15700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49b80cfe30 con 0x7f49b80cf2c0 2023-05-20T01:17:19.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.597+0000 7f49bfe15700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49b80cd630 con 0x7f49b815cec0 2023-05-20T01:17:19.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.597+0000 7f49bfe15700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49b80cca60 con 0x7f49b815d290 2023-05-20T01:17:19.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.597+0000 7f49bdbb1700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f49b815cec0 0x7f49b8071bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:19.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.597+0000 7f49be3b2700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f49b80cf2c0 0x7f49b80cf720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:42528/0 (socket says 172.21.15.16:42528) 2023-05-20T01:17:19.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.597+0000 7f49be3b2700 1 -- 172.21.15.16:0/2495769099 learned_addr learned my addr 172.21.15.16:0/2495769099 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:17:19.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.597+0000 7f49bd3b0700 1 --2- 172.21.15.16:0/2495769099 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f49b815d290 0x7f49b8072130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:19.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.598+0000 7f49bdbb1700 1 -- 172.21.15.16:0/2495769099 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f49b815d290 msgr2=0x7f49b8072130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:19.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.598+0000 7f49bdbb1700 1 --2- 172.21.15.16:0/2495769099 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f49b815d290 0x7f49b8072130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:19.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.598+0000 7f49bdbb1700 1 -- 172.21.15.16:0/2495769099 >> v1:172.21.15.16:6789/0 conn(0x7f49b80cf2c0 legacy=0x7f49b80cf720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:19.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.598+0000 7f49bdbb1700 1 -- 172.21.15.16:0/2495769099 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49b80ccd00 con 0x7f49b815cec0 2023-05-20T01:17:19.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.598+0000 7f49bdbb1700 1 --2- 172.21.15.16:0/2495769099 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49b815cec0 0x7f49b8071bf0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f49a8009820 tx=0x7f49a8009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbc9999471e0303d server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:19.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.598+0000 7f49bcbaf700 1 -- 172.21.15.16:0/2495769099 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49a8005e30 con 0x7f49b815cec0 2023-05-20T01:17:19.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.598+0000 7f49bcbaf700 1 -- 172.21.15.16:0/2495769099 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49a8006030 con 0x7f49b815cec0 2023-05-20T01:17:19.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.598+0000 7f49bcbaf700 1 -- 172.21.15.16:0/2495769099 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49a80063e0 con 0x7f49b815cec0 2023-05-20T01:17:19.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.598+0000 7f49bfe15700 1 -- 172.21.15.16:0/2495769099 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49b815cec0 msgr2=0x7f49b8071bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:19.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.598+0000 7f49bfe15700 1 --2- 172.21.15.16:0/2495769099 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49b815cec0 0x7f49b8071bf0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f49a8009820 tx=0x7f49a8009b30 comp rx=0 tx=0).stop 2023-05-20T01:17:19.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.598+0000 7f49bfe15700 1 -- 172.21.15.16:0/2495769099 shutdown_connections 2023-05-20T01:17:19.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.598+0000 7f49bfe15700 1 --2- 172.21.15.16:0/2495769099 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49b815cec0 0x7f49b8071bf0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:19.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.598+0000 7f49bfe15700 1 --2- 172.21.15.16:0/2495769099 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f49b815d290 0x7f49b8072130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:19.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.598+0000 7f49bfe15700 1 -- 172.21.15.16:0/2495769099 >> 172.21.15.16:0/2495769099 conn(0x7f49b80cffd0 msgr2=0x7f49b80d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:19.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.599+0000 7f49bfe15700 1 -- 172.21.15.16:0/2495769099 shutdown_connections 2023-05-20T01:17:19.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.599+0000 7f49bfe15700 1 -- 172.21.15.16:0/2495769099 wait complete. 2023-05-20T01:17:19.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.600+0000 7f49bfe15700 1 Processor -- start 2023-05-20T01:17:19.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.600+0000 7f49bfe15700 1 -- start start 2023-05-20T01:17:19.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.601+0000 7f49bfe15700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49b80cf2c0 0x7f49b81f6df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:19.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.601+0000 7f49bfe15700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f49b815cec0 0x7f49b81508c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:19.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.601+0000 7f49bfe15700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49b815d290 0x7f49b815e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:19.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.601+0000 7f49bfe15700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49b81f7330 con 0x7f49b80cf2c0 2023-05-20T01:17:19.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.601+0000 7f49bfe15700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49b8153100 con 0x7f49b815d290 2023-05-20T01:17:19.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.601+0000 7f49bfe15700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49b81533e0 con 0x7f49b815cec0 2023-05-20T01:17:19.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.601+0000 7f49bdbb1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49b80cf2c0 0x7f49b81f6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:19.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.601+0000 7f49bdbb1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49b80cf2c0 0x7f49b81f6df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59234/0 (socket says 172.21.15.16:59234) 2023-05-20T01:17:19.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.601+0000 7f49bdbb1700 1 -- 172.21.15.16:0/1943339680 learned_addr learned my addr 172.21.15.16:0/1943339680 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:19.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.601+0000 7f49bd3b0700 1 --2- 172.21.15.16:0/1943339680 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f49b815cec0 0x7f49b81508c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:19.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.601+0000 7f49bdbb1700 1 -- 172.21.15.16:0/1943339680 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f49b815cec0 msgr2=0x7f49b81508c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:19.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.601+0000 7f49bdbb1700 1 --2- 172.21.15.16:0/1943339680 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f49b815cec0 0x7f49b81508c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:19.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.601+0000 7f49bdbb1700 1 -- 172.21.15.16:0/1943339680 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49b815d290 msgr2=0x7f49b815e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:19.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.601+0000 7f49bdbb1700 1 --2- 172.21.15.16:0/1943339680 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49b815d290 0x7f49b815e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:19.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.601+0000 7f49bdbb1700 1 -- 172.21.15.16:0/1943339680 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49a8009430 con 0x7f49b80cf2c0 2023-05-20T01:17:19.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.601+0000 7f49bdbb1700 1 --2- 172.21.15.16:0/1943339680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49b80cf2c0 0x7f49b81f6df0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f49a8009b70 tx=0x7f49a8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:19.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.601+0000 7f49ae7fc700 1 -- 172.21.15.16:0/1943339680 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49a8004240 con 0x7f49b80cf2c0 2023-05-20T01:17:19.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.601+0000 7f49ae7fc700 1 -- 172.21.15.16:0/1943339680 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49a80043c0 con 0x7f49b80cf2c0 2023-05-20T01:17:19.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.601+0000 7f49bfe15700 1 -- 172.21.15.16:0/1943339680 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49b80cd2c0 con 0x7f49b80cf2c0 2023-05-20T01:17:19.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.601+0000 7f49ae7fc700 1 -- 172.21.15.16:0/1943339680 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49a80036a0 con 0x7f49b80cf2c0 2023-05-20T01:17:19.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.601+0000 7f49bfe15700 1 -- 172.21.15.16:0/1943339680 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49b80ccfe0 con 0x7f49b80cf2c0 2023-05-20T01:17:19.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.602+0000 7f49bfe15700 1 -- 172.21.15.16:0/1943339680 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49b8160890 con 0x7f49b80cf2c0 2023-05-20T01:17:19.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.603+0000 7f49ae7fc700 1 -- 172.21.15.16:0/1943339680 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f49a8022070 con 0x7f49b80cf2c0 2023-05-20T01:17:19.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.603+0000 7f49ae7fc700 1 --2- 172.21.15.16:0/1943339680 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f499c06ace0 0x7f499c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:19.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.603+0000 7f49ae7fc700 1 -- 172.21.15.16:0/1943339680 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f49a808b060 con 0x7f49b80cf2c0 2023-05-20T01:17:19.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.603+0000 7f49bd3b0700 1 --2- 172.21.15.16:0/1943339680 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f499c06ace0 0x7f499c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:19.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.604+0000 7f49bd3b0700 1 --2- 172.21.15.16:0/1943339680 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f499c06ace0 0x7f499c06d190 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f49b4009770 tx=0x7f49b4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:19.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.608+0000 7f49ae7fc700 1 -- 172.21.15.16:0/1943339680 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f49a8055e70 con 0x7f49b80cf2c0 2023-05-20T01:17:19.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.804+0000 7f49bfe15700 1 -- 172.21.15.16:0/1943339680 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f49b80747b0 con 0x7f49b80cf2c0 2023-05-20T01:17:19.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.804+0000 7f49ae7fc700 1 -- 172.21.15.16:0/1943339680 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f49a8059490 con 0x7f49b80cf2c0 2023-05-20T01:17:19.808 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2023-05-20T01:17:19.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.806+0000 7f49bfe15700 1 -- 172.21.15.16:0/1943339680 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f499c06ace0 msgr2=0x7f499c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:19.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.806+0000 7f49bfe15700 1 --2- 172.21.15.16:0/1943339680 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f499c06ace0 0x7f499c06d190 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f49b4009770 tx=0x7f49b4008040 comp rx=0 tx=0).stop 2023-05-20T01:17:19.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.806+0000 7f49bfe15700 1 -- 172.21.15.16:0/1943339680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49b80cf2c0 msgr2=0x7f49b81f6df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:19.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.806+0000 7f49bfe15700 1 --2- 172.21.15.16:0/1943339680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49b80cf2c0 0x7f49b81f6df0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f49a8009b70 tx=0x7f49a8004750 comp rx=0 tx=0).stop 2023-05-20T01:17:19.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.806+0000 7f49bfe15700 1 -- 172.21.15.16:0/1943339680 shutdown_connections 2023-05-20T01:17:19.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.806+0000 7f49bfe15700 1 --2- 172.21.15.16:0/1943339680 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49b815d290 0x7f49b815e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:19.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.806+0000 7f49bfe15700 1 --2- 172.21.15.16:0/1943339680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49b80cf2c0 0x7f49b81f6df0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:19.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.806+0000 7f49bfe15700 1 --2- 172.21.15.16:0/1943339680 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f499c06ace0 0x7f499c06d190 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:19.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.806+0000 7f49bfe15700 1 --2- 172.21.15.16:0/1943339680 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f49b815cec0 0x7f49b81508c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:19.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.806+0000 7f49bfe15700 1 -- 172.21.15.16:0/1943339680 >> 172.21.15.16:0/1943339680 conn(0x7f49b80cffd0 msgr2=0x7f49b815d700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:19.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.806+0000 7f49bfe15700 1 -- 172.21.15.16:0/1943339680 shutdown_connections 2023-05-20T01:17:19.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.806+0000 7f49bfe15700 1 -- 172.21.15.16:0/1943339680 wait complete. 2023-05-20T01:17:19.822 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-05-20T01:17:19.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.947+0000 7fca89262700 1 Processor -- start 2023-05-20T01:17:19.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.947+0000 7fca89262700 1 -- start start 2023-05-20T01:17:19.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.947+0000 7fca89262700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fca8415ca90 0x7fca840cd890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:19.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.947+0000 7fca89262700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fca840d2550 0x7fca840b6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:19.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.947+0000 7fca89262700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca840d1820 con 0x7fca840d29a0 2023-05-20T01:17:19.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.948+0000 7fca89262700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca840d0c50 con 0x7fca8415ca90 2023-05-20T01:17:19.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.948+0000 7fca89262700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca840cdf00 con 0x7fca840d2550 2023-05-20T01:17:19.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.948+0000 7fca82ffd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fca8415ca90 0x7fca840cd890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:19.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.948+0000 7fca82ffd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fca8415ca90 0x7fca840cd890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59242/0 (socket says 172.21.15.16:59242) 2023-05-20T01:17:19.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.948+0000 7fca82ffd700 1 -- 172.21.15.16:0/722822465 learned_addr learned my addr 172.21.15.16:0/722822465 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:19.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.948+0000 7fca827fc700 1 --2- 172.21.15.16:0/722822465 >> v2:172.21.15.132:3300/0 conn(0x7fca840d2550 0x7fca840b6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:19.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.948+0000 7fca82ffd700 1 -- 172.21.15.16:0/722822465 >> v2:172.21.15.132:3300/0 conn(0x7fca840d2550 msgr2=0x7fca840b6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:19.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.948+0000 7fca82ffd700 1 --2- 172.21.15.16:0/722822465 >> v2:172.21.15.132:3300/0 conn(0x7fca840d2550 0x7fca840b6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:19.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.948+0000 7fca82ffd700 1 -- 172.21.15.16:0/722822465 >> v1:172.21.15.132:6789/0 conn(0x7fca840d29a0 legacy=0x7fca840b4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:19.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.948+0000 7fca82ffd700 1 -- 172.21.15.16:0/722822465 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca840d1a70 con 0x7fca8415ca90 2023-05-20T01:17:19.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.948+0000 7fca827fc700 1 --2- 172.21.15.16:0/722822465 >> v2:172.21.15.132:3300/0 conn(0x7fca840d2550 0x7fca840b6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:17:19.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.948+0000 7fca82ffd700 1 --2- 172.21.15.16:0/722822465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca8415ca90 0x7fca840cd890 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fca840b5e30 tx=0x7fca74009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bded530f581dc0fd server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:19.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.948+0000 7fca81ffb700 1 -- 172.21.15.16:0/722822465 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca7400b870 con 0x7fca8415ca90 2023-05-20T01:17:19.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.948+0000 7fca81ffb700 1 -- 172.21.15.16:0/722822465 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca7400b9d0 con 0x7fca8415ca90 2023-05-20T01:17:19.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.948+0000 7fca81ffb700 1 -- 172.21.15.16:0/722822465 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca7400bd60 con 0x7fca8415ca90 2023-05-20T01:17:19.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.949+0000 7fca89262700 1 -- 172.21.15.16:0/722822465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca8415ca90 msgr2=0x7fca840cd890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:19.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.949+0000 7fca89262700 1 --2- 172.21.15.16:0/722822465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca8415ca90 0x7fca840cd890 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fca840b5e30 tx=0x7fca74009bd0 comp rx=0 tx=0).stop 2023-05-20T01:17:19.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.949+0000 7fca89262700 1 -- 172.21.15.16:0/722822465 shutdown_connections 2023-05-20T01:17:19.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.949+0000 7fca89262700 1 --2- 172.21.15.16:0/722822465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca8415ca90 0x7fca840cd890 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:19.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.949+0000 7fca89262700 1 --2- 172.21.15.16:0/722822465 >> v2:172.21.15.132:3300/0 conn(0x7fca840d2550 0x7fca840b6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:19.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.949+0000 7fca89262700 1 -- 172.21.15.16:0/722822465 >> 172.21.15.16:0/722822465 conn(0x7fca8406b000 msgr2=0x7fca8406b400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:19.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.949+0000 7fca89262700 1 -- 172.21.15.16:0/722822465 shutdown_connections 2023-05-20T01:17:19.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.949+0000 7fca89262700 1 -- 172.21.15.16:0/722822465 wait complete. 2023-05-20T01:17:19.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.950+0000 7fca89262700 1 Processor -- start 2023-05-20T01:17:19.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.950+0000 7fca89262700 1 -- start start 2023-05-20T01:17:19.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.951+0000 7fca89262700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fca840d2550 0x7fca8415d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:19.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.951+0000 7fca89262700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca840d29a0 0x7fca840b3440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:19.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.951+0000 7fca89262700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fca8415ca90 0x7fca840c8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:19.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.951+0000 7fca89262700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca840c93e0 con 0x7fca840d29a0 2023-05-20T01:17:19.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.951+0000 7fca89262700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca84152d30 con 0x7fca840d2550 2023-05-20T01:17:19.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.951+0000 7fca89262700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca8415d890 con 0x7fca8415ca90 2023-05-20T01:17:19.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.951+0000 7fca827fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca840d29a0 0x7fca840b3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:19.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.951+0000 7fca827fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca840d29a0 0x7fca840b3440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59250/0 (socket says 172.21.15.16:59250) 2023-05-20T01:17:19.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.951+0000 7fca827fc700 1 -- 172.21.15.16:0/2624339482 learned_addr learned my addr 172.21.15.16:0/2624339482 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:19.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.951+0000 7fca837fe700 1 --2- 172.21.15.16:0/2624339482 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fca8415ca90 0x7fca840c8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:19.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.951+0000 7fca82ffd700 1 --2- 172.21.15.16:0/2624339482 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fca840d2550 0x7fca8415d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:19.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.951+0000 7fca827fc700 1 -- 172.21.15.16:0/2624339482 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fca8415ca90 msgr2=0x7fca840c8ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:19.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.951+0000 7fca827fc700 1 --2- 172.21.15.16:0/2624339482 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fca8415ca90 0x7fca840c8ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:19.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.951+0000 7fca827fc700 1 -- 172.21.15.16:0/2624339482 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fca840d2550 msgr2=0x7fca8415d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:19.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.951+0000 7fca827fc700 1 --2- 172.21.15.16:0/2624339482 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fca840d2550 0x7fca8415d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:19.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.951+0000 7fca827fc700 1 -- 172.21.15.16:0/2624339482 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca74009550 con 0x7fca840d29a0 2023-05-20T01:17:19.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.952+0000 7fca827fc700 1 --2- 172.21.15.16:0/2624339482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca840d29a0 0x7fca840b3440 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fca7800e810 tx=0x7fca7800ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:19.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.952+0000 7fca6b7fe700 1 -- 172.21.15.16:0/2624339482 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca78013610 con 0x7fca840d29a0 2023-05-20T01:17:19.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.952+0000 7fca6b7fe700 1 -- 172.21.15.16:0/2624339482 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca78013770 con 0x7fca840d29a0 2023-05-20T01:17:19.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.952+0000 7fca89262700 1 -- 172.21.15.16:0/2624339482 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca840d2030 con 0x7fca840d29a0 2023-05-20T01:17:19.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.952+0000 7fca6b7fe700 1 -- 172.21.15.16:0/2624339482 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca78013b20 con 0x7fca840d29a0 2023-05-20T01:17:19.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.952+0000 7fca89262700 1 -- 172.21.15.16:0/2624339482 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca840d1d50 con 0x7fca840d29a0 2023-05-20T01:17:19.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.953+0000 7fca89262700 1 -- 172.21.15.16:0/2624339482 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca8415ce60 con 0x7fca840d29a0 2023-05-20T01:17:19.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.953+0000 7fca6b7fe700 1 -- 172.21.15.16:0/2624339482 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fca78013c80 con 0x7fca840d29a0 2023-05-20T01:17:19.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.954+0000 7fca6b7fe700 1 --2- 172.21.15.16:0/2624339482 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fca7006ace0 0x7fca7006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:19.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.954+0000 7fca6b7fe700 1 -- 172.21.15.16:0/2624339482 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fca780898d0 con 0x7fca840d29a0 2023-05-20T01:17:19.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.954+0000 7fca82ffd700 1 --2- 172.21.15.16:0/2624339482 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fca7006ace0 0x7fca7006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:19.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.954+0000 7fca82ffd700 1 --2- 172.21.15.16:0/2624339482 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fca7006ace0 0x7fca7006d190 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fca74014040 tx=0x7fca74008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:19.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:19.958+0000 7fca6b7fe700 1 -- 172.21.15.16:0/2624339482 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fca78054730 con 0x7fca840d29a0 2023-05-20T01:17:20.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.154+0000 7fca89262700 1 -- 172.21.15.16:0/2624339482 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fca84066d20 con 0x7fca840d29a0 2023-05-20T01:17:20.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.155+0000 7fca6b7fe700 1 -- 172.21.15.16:0/2624339482 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7fca78057d50 con 0x7fca840d29a0 2023-05-20T01:17:20.159 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-05-20T01:17:20.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.157+0000 7fca89262700 1 -- 172.21.15.16:0/2624339482 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fca7006ace0 msgr2=0x7fca7006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:20.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.157+0000 7fca89262700 1 --2- 172.21.15.16:0/2624339482 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fca7006ace0 0x7fca7006d190 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fca74014040 tx=0x7fca74008030 comp rx=0 tx=0).stop 2023-05-20T01:17:20.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.157+0000 7fca89262700 1 -- 172.21.15.16:0/2624339482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca840d29a0 msgr2=0x7fca840b3440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:20.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.157+0000 7fca89262700 1 --2- 172.21.15.16:0/2624339482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca840d29a0 0x7fca840b3440 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fca7800e810 tx=0x7fca7800ebd0 comp rx=0 tx=0).stop 2023-05-20T01:17:20.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.157+0000 7fca89262700 1 -- 172.21.15.16:0/2624339482 shutdown_connections 2023-05-20T01:17:20.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.157+0000 7fca89262700 1 --2- 172.21.15.16:0/2624339482 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fca7006ace0 0x7fca7006d190 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:20.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.157+0000 7fca89262700 1 --2- 172.21.15.16:0/2624339482 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fca8415ca90 0x7fca840c8ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:20.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.157+0000 7fca89262700 1 --2- 172.21.15.16:0/2624339482 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fca840d2550 0x7fca8415d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:20.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.157+0000 7fca89262700 1 --2- 172.21.15.16:0/2624339482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca840d29a0 0x7fca840b3440 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:20.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.157+0000 7fca89262700 1 -- 172.21.15.16:0/2624339482 >> 172.21.15.16:0/2624339482 conn(0x7fca8406b000 msgr2=0x7fca840cf430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:20.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.157+0000 7fca89262700 1 -- 172.21.15.16:0/2624339482 shutdown_connections 2023-05-20T01:17:20.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.157+0000 7fca89262700 1 -- 172.21.15.16:0/2624339482 wait complete. 2023-05-20T01:17:20.173 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-05-20T01:17:20.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.296+0000 7f4d86aec700 1 Processor -- start 2023-05-20T01:17:20.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.296+0000 7f4d86aec700 1 -- start start 2023-05-20T01:17:20.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.297+0000 7f4d86aec700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4d800ccec0 0x7f4d800713c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:20.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.297+0000 7f4d86aec700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f4d80071900 0x7f4d800ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:20.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.297+0000 7f4d86aec700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d800cb590 con 0x7f4d80157190 2023-05-20T01:17:20.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.297+0000 7f4d86aec700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d800ca9a0 con 0x7f4d80071900 2023-05-20T01:17:20.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.297+0000 7f4d86aec700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d800cac60 con 0x7f4d800ccec0 2023-05-20T01:17:20.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.297+0000 7f4d7ffff700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4d800ccec0 0x7f4d800713c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:20.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.297+0000 7f4d85089700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f4d80071900 0x7f4d800ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:20.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.297+0000 7f4d7ffff700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4d800ccec0 0x7f4d800713c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.16:59296/0 (socket says 172.21.15.16:59296) 2023-05-20T01:17:20.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.297+0000 7f4d7ffff700 1 -- 172.21.15.16:0/1859895269 learned_addr learned my addr 172.21.15.16:0/1859895269 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:20.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.297+0000 7f4d85089700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f4d80071900 0x7f4d800ce740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.16:56282/0 (socket says 172.21.15.16:56282) 2023-05-20T01:17:20.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.298+0000 7f4d7ffff700 1 -- 172.21.15.16:0/1859895269 >> v2:172.21.15.132:3300/0 conn(0x7f4d80071900 msgr2=0x7f4d800ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:20.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.298+0000 7f4d7ffff700 1 --2- 172.21.15.16:0/1859895269 >> v2:172.21.15.132:3300/0 conn(0x7f4d80071900 0x7f4d800ce740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:20.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.298+0000 7f4d7ffff700 1 -- 172.21.15.16:0/1859895269 >> v1:172.21.15.132:6789/0 conn(0x7f4d80157190 legacy=0x7f4d800cc7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:20.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.298+0000 7f4d7ffff700 1 -- 172.21.15.16:0/1859895269 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d800caf40 con 0x7f4d800ccec0 2023-05-20T01:17:20.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.298+0000 7f4d85089700 1 --2- 172.21.15.16:0/1859895269 >> v2:172.21.15.132:3300/0 conn(0x7f4d80071900 0x7f4d800ce740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-05-20T01:17:20.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.298+0000 7f4d7ffff700 1 --2- 172.21.15.16:0/1859895269 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4d800ccec0 0x7f4d800713c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c0098f0 tx=0x7f4d6c009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1b1f4fc2dd12c6d server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:20.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.298+0000 7f4d7f7fe700 1 -- 172.21.15.16:0/1859895269 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d6c011830 con 0x7f4d800ccec0 2023-05-20T01:17:20.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.298+0000 7f4d7f7fe700 1 -- 172.21.15.16:0/1859895269 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d6c011990 con 0x7f4d800ccec0 2023-05-20T01:17:20.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.298+0000 7f4d7f7fe700 1 -- 172.21.15.16:0/1859895269 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d6c011d20 con 0x7f4d800ccec0 2023-05-20T01:17:20.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.299+0000 7f4d86aec700 1 -- 172.21.15.16:0/1859895269 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4d800ccec0 msgr2=0x7f4d800713c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:20.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.299+0000 7f4d86aec700 1 --2- 172.21.15.16:0/1859895269 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4d800ccec0 0x7f4d800713c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c0098f0 tx=0x7f4d6c009c00 comp rx=0 tx=0).stop 2023-05-20T01:17:20.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.299+0000 7f4d86aec700 1 -- 172.21.15.16:0/1859895269 shutdown_connections 2023-05-20T01:17:20.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.299+0000 7f4d86aec700 1 --2- 172.21.15.16:0/1859895269 >> v2:172.21.15.132:3300/0 conn(0x7f4d80071900 0x7f4d800ce740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:20.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.299+0000 7f4d86aec700 1 --2- 172.21.15.16:0/1859895269 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4d800ccec0 0x7f4d800713c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:20.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.299+0000 7f4d86aec700 1 -- 172.21.15.16:0/1859895269 >> 172.21.15.16:0/1859895269 conn(0x7f4d800cffb0 msgr2=0x7f4d800d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:20.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.299+0000 7f4d86aec700 1 -- 172.21.15.16:0/1859895269 shutdown_connections 2023-05-20T01:17:20.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.299+0000 7f4d86aec700 1 -- 172.21.15.16:0/1859895269 wait complete. 2023-05-20T01:17:20.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.300+0000 7f4d86aec700 1 Processor -- start 2023-05-20T01:17:20.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.300+0000 7f4d86aec700 1 -- start start 2023-05-20T01:17:20.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.301+0000 7f4d86aec700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4d80071900 0x7f4d801544e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:20.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.301+0000 7f4d86aec700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4d800ccec0 0x7f4d800b3e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:20.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.301+0000 7f4d86aec700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d80157190 0x7f4d800c5f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:20.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.301+0000 7f4d86aec700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d800b0e60 con 0x7f4d80157190 2023-05-20T01:17:20.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.301+0000 7f4d86aec700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d800b10e0 con 0x7f4d80071900 2023-05-20T01:17:20.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.301+0000 7f4d86aec700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d800b13c0 con 0x7f4d800ccec0 2023-05-20T01:17:20.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.301+0000 7f4d85089700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d80157190 0x7f4d800c5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:20.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.301+0000 7f4d85089700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d80157190 0x7f4d800c5f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59266/0 (socket says 172.21.15.16:59266) 2023-05-20T01:17:20.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.301+0000 7f4d85089700 1 -- 172.21.15.16:0/3109837781 learned_addr learned my addr 172.21.15.16:0/3109837781 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:20.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.301+0000 7f4d84888700 1 --2- 172.21.15.16:0/3109837781 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4d80071900 0x7f4d801544e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:20.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.301+0000 7f4d7ffff700 1 --2- 172.21.15.16:0/3109837781 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4d800ccec0 0x7f4d800b3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:20.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.301+0000 7f4d85089700 1 -- 172.21.15.16:0/3109837781 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4d800ccec0 msgr2=0x7f4d800b3e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:20.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.301+0000 7f4d85089700 1 --2- 172.21.15.16:0/3109837781 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4d800ccec0 0x7f4d800b3e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:20.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.301+0000 7f4d85089700 1 -- 172.21.15.16:0/3109837781 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4d80071900 msgr2=0x7f4d801544e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:20.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.301+0000 7f4d85089700 1 --2- 172.21.15.16:0/3109837781 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4d80071900 0x7f4d801544e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:20.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.301+0000 7f4d85089700 1 -- 172.21.15.16:0/3109837781 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d7800bc80 con 0x7f4d80157190 2023-05-20T01:17:20.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.302+0000 7f4d85089700 1 --2- 172.21.15.16:0/3109837781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d80157190 0x7f4d800c5f70 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f4d7800e160 tx=0x7f4d7800e470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:20.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.302+0000 7f4d7d7fa700 1 -- 172.21.15.16:0/3109837781 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d78019070 con 0x7f4d80157190 2023-05-20T01:17:20.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.302+0000 7f4d7d7fa700 1 -- 172.21.15.16:0/3109837781 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d78011040 con 0x7f4d80157190 2023-05-20T01:17:20.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.302+0000 7f4d86aec700 1 -- 172.21.15.16:0/3109837781 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d6c009550 con 0x7f4d80157190 2023-05-20T01:17:20.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.302+0000 7f4d7d7fa700 1 -- 172.21.15.16:0/3109837781 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d78010050 con 0x7f4d80157190 2023-05-20T01:17:20.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.302+0000 7f4d86aec700 1 -- 172.21.15.16:0/3109837781 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d80157560 con 0x7f4d80157190 2023-05-20T01:17:20.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.303+0000 7f4d86aec700 1 -- 172.21.15.16:0/3109837781 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d800cb220 con 0x7f4d80157190 2023-05-20T01:17:20.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.303+0000 7f4d7d7fa700 1 -- 172.21.15.16:0/3109837781 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4d7800ea70 con 0x7f4d80157190 2023-05-20T01:17:20.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.304+0000 7f4d7d7fa700 1 --2- 172.21.15.16:0/3109837781 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f4d6406ace0 0x7f4d6406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:20.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.304+0000 7f4d7d7fa700 1 -- 172.21.15.16:0/3109837781 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f4d7808b630 con 0x7f4d80157190 2023-05-20T01:17:20.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.304+0000 7f4d84888700 1 --2- 172.21.15.16:0/3109837781 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f4d6406ace0 0x7f4d6406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:20.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.304+0000 7f4d84888700 1 --2- 172.21.15.16:0/3109837781 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f4d6406ace0 0x7f4d6406d190 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f4d74007fd0 tx=0x7f4d74009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:20.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.308+0000 7f4d7d7fa700 1 -- 172.21.15.16:0/3109837781 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f4d780563b0 con 0x7f4d80157190 2023-05-20T01:17:20.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.527+0000 7f4d86aec700 1 -- 172.21.15.16:0/3109837781 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f4d800cba20 con 0x7f4d80157190 2023-05-20T01:17:20.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.528+0000 7f4d7d7fa700 1 -- 172.21.15.16:0/3109837781 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) v1 ==== 80+0+0 (secure 0 0 0) 0x7f4d780599d0 con 0x7f4d80157190 2023-05-20T01:17:20.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.530+0000 7f4d86aec700 1 -- 172.21.15.16:0/3109837781 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f4d6406ace0 msgr2=0x7f4d6406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:20.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.530+0000 7f4d86aec700 1 --2- 172.21.15.16:0/3109837781 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f4d6406ace0 0x7f4d6406d190 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f4d74007fd0 tx=0x7f4d74009040 comp rx=0 tx=0).stop 2023-05-20T01:17:20.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.530+0000 7f4d86aec700 1 -- 172.21.15.16:0/3109837781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d80157190 msgr2=0x7f4d800c5f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:20.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.530+0000 7f4d86aec700 1 --2- 172.21.15.16:0/3109837781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d80157190 0x7f4d800c5f70 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f4d7800e160 tx=0x7f4d7800e470 comp rx=0 tx=0).stop 2023-05-20T01:17:20.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.530+0000 7f4d86aec700 1 -- 172.21.15.16:0/3109837781 shutdown_connections 2023-05-20T01:17:20.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.530+0000 7f4d86aec700 1 --2- 172.21.15.16:0/3109837781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d80157190 0x7f4d800c5f70 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:20.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.530+0000 7f4d86aec700 1 --2- 172.21.15.16:0/3109837781 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4d80071900 0x7f4d801544e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:20.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.530+0000 7f4d86aec700 1 --2- 172.21.15.16:0/3109837781 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f4d6406ace0 0x7f4d6406d190 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:20.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.530+0000 7f4d86aec700 1 --2- 172.21.15.16:0/3109837781 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4d800ccec0 0x7f4d800b3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:20.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.530+0000 7f4d86aec700 1 -- 172.21.15.16:0/3109837781 >> 172.21.15.16:0/3109837781 conn(0x7f4d800cffb0 msgr2=0x7f4d800d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:20.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.530+0000 7f4d86aec700 1 -- 172.21.15.16:0/3109837781 shutdown_connections 2023-05-20T01:17:20.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.530+0000 7f4d86aec700 1 -- 172.21.15.16:0/3109837781 wait complete. 2023-05-20T01:17:20.535 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2023-05-20T01:17:20.548 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-05-20T01:17:20.549 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:17:20.549 DEBUG:teuthology.orchestra.run.smithi016:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2023-05-20T01:17:20.607 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2023-05-20T01:17:20.607 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:17:20.608 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-05-20T01:17:20.625 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2023-05-20T01:17:20.626 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/proc/self/mounts of=/dev/stdout 2023-05-20T01:17:20.646 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2023-05-20T01:17:20.647 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/proc/self/mounts of=/dev/stdout 2023-05-20T01:17:20.706 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2023-05-20T01:17:20.707 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/proc/self/mounts of=/dev/stdout 2023-05-20T01:17:20.768 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-20T01:17:20.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.890+0000 7f49aa084700 1 Processor -- start 2023-05-20T01:17:20.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.890+0000 7f49aa084700 1 -- start start 2023-05-20T01:17:20.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.890+0000 7f49aa084700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f49a41599c0 0x7f49a40caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:20.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.891+0000 7f49aa084700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f49a40cf350 0x7f49a40cb4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:20.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.891+0000 7f49aa084700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49a4159d90 con 0x7f49a40caaa0 2023-05-20T01:17:20.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.891+0000 7f49aa084700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49a40cdf20 con 0x7f49a40cf350 2023-05-20T01:17:20.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.891+0000 7f49aa084700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49a40cdb80 con 0x7f49a41599c0 2023-05-20T01:17:20.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.891+0000 7f49a37fe700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f49a41599c0 0x7f49a40caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:20.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.891+0000 7f49a3fff700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f49a40cf350 0x7f49a40cb4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:20.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.891+0000 7f49a37fe700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f49a41599c0 0x7f49a40caf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.16:59308/0 (socket says 172.21.15.16:59308) 2023-05-20T01:17:20.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.891+0000 7f49a2ffd700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f49a40caaa0 0x7f49a40710d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.16:52280/0 (socket says 172.21.15.16:52280) 2023-05-20T01:17:20.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.891+0000 7f49a37fe700 1 -- 172.21.15.16:0/2345600895 learned_addr learned my addr 172.21.15.16:0/2345600895 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:20.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.891+0000 7f49a37fe700 1 -- 172.21.15.16:0/2345600895 >> v2:172.21.15.132:3300/0 conn(0x7f49a40cf350 msgr2=0x7f49a40cb4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:20.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.891+0000 7f49a37fe700 1 --2- 172.21.15.16:0/2345600895 >> v2:172.21.15.132:3300/0 conn(0x7f49a40cf350 0x7f49a40cb4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:20.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.891+0000 7f49a37fe700 1 -- 172.21.15.16:0/2345600895 >> v1:172.21.15.132:6789/0 conn(0x7f49a40caaa0 legacy=0x7f49a40710d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:20.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.891+0000 7f49a37fe700 1 -- 172.21.15.16:0/2345600895 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49a406eaf0 con 0x7f49a41599c0 2023-05-20T01:17:20.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.892+0000 7f49a37fe700 1 --2- 172.21.15.16:0/2345600895 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f49a41599c0 0x7f49a40caf60 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f49940098f0 tx=0x7f4994009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3824cc070f0b4be9 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:20.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.892+0000 7f49a27fc700 1 -- 172.21.15.16:0/2345600895 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4994011830 con 0x7f49a41599c0 2023-05-20T01:17:20.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.892+0000 7f49a27fc700 1 -- 172.21.15.16:0/2345600895 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4994011990 con 0x7f49a41599c0 2023-05-20T01:17:20.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.892+0000 7f49a27fc700 1 -- 172.21.15.16:0/2345600895 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4994011d20 con 0x7f49a41599c0 2023-05-20T01:17:20.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.892+0000 7f49aa084700 1 -- 172.21.15.16:0/2345600895 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f49a41599c0 msgr2=0x7f49a40caf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:20.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.892+0000 7f49aa084700 1 --2- 172.21.15.16:0/2345600895 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f49a41599c0 0x7f49a40caf60 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f49940098f0 tx=0x7f4994009c00 comp rx=0 tx=0).stop 2023-05-20T01:17:20.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.892+0000 7f49aa084700 1 -- 172.21.15.16:0/2345600895 shutdown_connections 2023-05-20T01:17:20.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.892+0000 7f49aa084700 1 --2- 172.21.15.16:0/2345600895 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f49a41599c0 0x7f49a40caf60 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:20.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.892+0000 7f49aa084700 1 --2- 172.21.15.16:0/2345600895 >> v2:172.21.15.132:3300/0 conn(0x7f49a40cf350 0x7f49a40cb4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:20.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.892+0000 7f49aa084700 1 -- 172.21.15.16:0/2345600895 >> 172.21.15.16:0/2345600895 conn(0x7f49a40d0c30 msgr2=0x7f49a40b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:20.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.892+0000 7f49aa084700 1 -- 172.21.15.16:0/2345600895 shutdown_connections 2023-05-20T01:17:20.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.892+0000 7f49aa084700 1 -- 172.21.15.16:0/2345600895 wait complete. 2023-05-20T01:17:20.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.894+0000 7f49aa084700 1 Processor -- start 2023-05-20T01:17:20.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.894+0000 7f49aa084700 1 -- start start 2023-05-20T01:17:20.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.894+0000 7f49aa084700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f49a40caaa0 0x7f49a40c2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:20.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.894+0000 7f49aa084700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49a40cf350 0x7f49a40c3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:20.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.894+0000 7f49aa084700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49a41599c0 0x7f49a40c3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:20.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.894+0000 7f49aa084700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49a40c9f60 con 0x7f49a41599c0 2023-05-20T01:17:20.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.894+0000 7f49aa084700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49a4157db0 con 0x7f49a40cf350 2023-05-20T01:17:20.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.894+0000 7f49aa084700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49a4158c10 con 0x7f49a40caaa0 2023-05-20T01:17:20.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.895+0000 7f49a3fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49a41599c0 0x7f49a40c3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:20.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.895+0000 7f49a3fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49a41599c0 0x7f49a40c3ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59280/0 (socket says 172.21.15.16:59280) 2023-05-20T01:17:20.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.895+0000 7f49a3fff700 1 -- 172.21.15.16:0/1321480854 learned_addr learned my addr 172.21.15.16:0/1321480854 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:20.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.895+0000 7f49a37fe700 1 --2- 172.21.15.16:0/1321480854 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f49a40caaa0 0x7f49a40c2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:20.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.895+0000 7f49a2ffd700 1 --2- 172.21.15.16:0/1321480854 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49a40cf350 0x7f49a40c3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:20.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.895+0000 7f49a3fff700 1 -- 172.21.15.16:0/1321480854 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f49a40caaa0 msgr2=0x7f49a40c2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:20.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.895+0000 7f49a3fff700 1 --2- 172.21.15.16:0/1321480854 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f49a40caaa0 0x7f49a40c2bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:20.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.895+0000 7f49a3fff700 1 -- 172.21.15.16:0/1321480854 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49a40cf350 msgr2=0x7f49a40c3980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:20.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.895+0000 7f49a3fff700 1 --2- 172.21.15.16:0/1321480854 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49a40cf350 0x7f49a40c3980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:20.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.895+0000 7f49a3fff700 1 -- 172.21.15.16:0/1321480854 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4994009550 con 0x7f49a41599c0 2023-05-20T01:17:20.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.895+0000 7f49a3fff700 1 --2- 172.21.15.16:0/1321480854 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49a41599c0 0x7f49a40c3ec0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f499c009dd0 tx=0x7f499c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:20.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.895+0000 7f498bfff700 1 -- 172.21.15.16:0/1321480854 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f499c009c40 con 0x7f49a41599c0 2023-05-20T01:17:20.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.895+0000 7f498bfff700 1 -- 172.21.15.16:0/1321480854 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f499c00f040 con 0x7f49a41599c0 2023-05-20T01:17:20.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.895+0000 7f498bfff700 1 -- 172.21.15.16:0/1321480854 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f499c00ceb0 con 0x7f49a41599c0 2023-05-20T01:17:20.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.895+0000 7f49aa084700 1 -- 172.21.15.16:0/1321480854 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49a406f0b0 con 0x7f49a41599c0 2023-05-20T01:17:20.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.895+0000 7f49aa084700 1 -- 172.21.15.16:0/1321480854 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49a406edd0 con 0x7f49a41599c0 2023-05-20T01:17:20.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.896+0000 7f49aa084700 1 -- 172.21.15.16:0/1321480854 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49a406e750 con 0x7f49a41599c0 2023-05-20T01:17:20.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.897+0000 7f498bfff700 1 -- 172.21.15.16:0/1321480854 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f499c014480 con 0x7f49a41599c0 2023-05-20T01:17:20.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.897+0000 7f498bfff700 1 --2- 172.21.15.16:0/1321480854 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f499006ace0 0x7f499006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:20.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.897+0000 7f498bfff700 1 -- 172.21.15.16:0/1321480854 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f499c089f70 con 0x7f49a41599c0 2023-05-20T01:17:20.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.897+0000 7f49a37fe700 1 --2- 172.21.15.16:0/1321480854 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f499006ace0 0x7f499006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:20.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.898+0000 7f49a37fe700 1 --2- 172.21.15.16:0/1321480854 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f499006ace0 0x7f499006d190 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4994009c40 tx=0x7f499401d040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:20.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:20.902+0000 7f498bfff700 1 -- 172.21.15.16:0/1321480854 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f499c0bc2d0 con 0x7f49a41599c0 2023-05-20T01:17:21.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.099+0000 7f49aa084700 1 -- 172.21.15.16:0/1321480854 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f49a40747b0 con 0x7f49a41599c0 2023-05-20T01:17:21.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.100+0000 7f498bfff700 1 -- 172.21.15.16:0/1321480854 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+20121 (secure 0 0 0) 0x7f499c018090 con 0x7f49a41599c0 2023-05-20T01:17:21.104 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:17:21.105 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":27,"fsid":"c2245cc5-ecab-4087-a59b-bc81db901676","created":"2023-05-20T01:16:09.549230+0000","modified":"2023-05-20T01:17:16.406712+0000","last_up_change":"2023-05-20T01:16:19.788624+0000","last_in_change":"2023-05-20T01:16:12.092160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-05-20T01:16:20.934366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-05-20T01:16:21.372294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-05-20T01:16:26.208287+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-05-20T01:16:27.292268+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-05-20T01:16:30.357285+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"317a83cd-19c8-4958-a301-6087ef342289","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6825","nonce":3923634179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6827","nonce":3923634179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6831","nonce":3923634179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6829","nonce":3923634179}]},"public_addr":"172.21.15.16:6825/3923634179","cluster_addr":"172.21.15.16:6827/3923634179","heartbeat_back_addr":"172.21.15.16:6831/3923634179","heartbeat_front_addr":"172.21.15.16:6829/3923634179","state":["exists","up"]},{"osd":1,"uuid":"5581046f-8219-4089-8d7d-b7846399505e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6809","nonce":3860264220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6811","nonce":3860264220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6815","nonce":3860264220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6813","nonce":3860264220}]},"public_addr":"172.21.15.16:6809/3860264220","cluster_addr":"172.21.15.16:6811/3860264220","heartbeat_back_addr":"172.21.15.16:6815/3860264220","heartbeat_front_addr":"172.21.15.16:6813/3860264220","state":["exists","up"]},{"osd":2,"uuid":"072e9e66-4133-478e-8d4a-64d46c2963f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6817","nonce":293187030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6819","nonce":293187030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6823","nonce":293187030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6821","nonce":293187030}]},"public_addr":"172.21.15.16:6817/293187030","cluster_addr":"172.21.15.16:6819/293187030","heartbeat_back_addr":"172.21.15.16:6823/293187030","heartbeat_front_addr":"172.21.15.16:6821/293187030","state":["exists","up"]},{"osd":3,"uuid":"11e1bba6-00f8-48dd-9028-d8d716b7f145","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6801","nonce":3669781890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6803","nonce":3669781890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6807","nonce":3669781890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6805","nonce":3669781890}]},"public_addr":"172.21.15.16:6801/3669781890","cluster_addr":"172.21.15.16:6803/3669781890","heartbeat_back_addr":"172.21.15.16:6807/3669781890","heartbeat_front_addr":"172.21.15.16:6805/3669781890","state":["exists","up"]},{"osd":4,"uuid":"b8909c3a-cf70-46cb-8d0b-009c707cf9d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6819","nonce":506317223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6823","nonce":506317223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6829","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6831","nonce":506317223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6827","nonce":506317223}]},"public_addr":"172.21.15.132:6819/506317223","cluster_addr":"172.21.15.132:6823/506317223","heartbeat_back_addr":"172.21.15.132:6831/506317223","heartbeat_front_addr":"172.21.15.132:6827/506317223","state":["exists","up"]},{"osd":5,"uuid":"b1818304-4583-47bf-a6ec-0bfbb5166233","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6818","nonce":1844689995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6822","nonce":1844689995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6830","nonce":1844689995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6826","nonce":1844689995}]},"public_addr":"172.21.15.132:6818/1844689995","cluster_addr":"172.21.15.132:6822/1844689995","heartbeat_back_addr":"172.21.15.132:6830/1844689995","heartbeat_front_addr":"172.21.15.132:6826/1844689995","state":["exists","up"]},{"osd":6,"uuid":"d37f4691-eed1-4a93-8818-18354f03f4a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6801","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6802","nonce":727581123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6805","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6807","nonce":727581123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6813","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6815","nonce":727581123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6811","nonce":727581123}]},"public_addr":"172.21.15.132:6802/727581123","cluster_addr":"172.21.15.132:6807/727581123","heartbeat_back_addr":"172.21.15.132:6815/727581123","heartbeat_front_addr":"172.21.15.132:6811/727581123","state":["exists","up"]},{"osd":7,"uuid":"4f712f62-5d1d-4816-8042-03209144f8b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6803","nonce":466001149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6806","nonce":466001149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6814","nonce":466001149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6810","nonce":466001149}]},"public_addr":"172.21.15.132:6803/466001149","cluster_addr":"172.21.15.132:6806/466001149","heartbeat_back_addr":"172.21.15.132:6814/466001149","heartbeat_front_addr":"172.21.15.132:6810/466001149","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.338863+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.278680+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305694+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.246835+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.276237+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.304441+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305421+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.267050+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.a","mappings":[{"from":1,"to":5}]},{"pgid":"4.11","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-20T01:17:21.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.102+0000 7f49aa084700 1 -- 172.21.15.16:0/1321480854 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f499006ace0 msgr2=0x7f499006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:21.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.102+0000 7f49aa084700 1 --2- 172.21.15.16:0/1321480854 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f499006ace0 0x7f499006d190 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4994009c40 tx=0x7f499401d040 comp rx=0 tx=0).stop 2023-05-20T01:17:21.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.102+0000 7f49aa084700 1 -- 172.21.15.16:0/1321480854 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49a41599c0 msgr2=0x7f49a40c3ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:21.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.102+0000 7f49aa084700 1 --2- 172.21.15.16:0/1321480854 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49a41599c0 0x7f49a40c3ec0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f499c009dd0 tx=0x7f499c00c3a0 comp rx=0 tx=0).stop 2023-05-20T01:17:21.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.102+0000 7f49aa084700 1 -- 172.21.15.16:0/1321480854 shutdown_connections 2023-05-20T01:17:21.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.102+0000 7f49aa084700 1 --2- 172.21.15.16:0/1321480854 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49a41599c0 0x7f49a40c3ec0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:21.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.102+0000 7f49aa084700 1 --2- 172.21.15.16:0/1321480854 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f499006ace0 0x7f499006d190 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:21.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.102+0000 7f49aa084700 1 --2- 172.21.15.16:0/1321480854 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f49a40caaa0 0x7f49a40c2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:21.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.102+0000 7f49aa084700 1 --2- 172.21.15.16:0/1321480854 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49a40cf350 0x7f49a40c3980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:21.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.103+0000 7f49aa084700 1 -- 172.21.15.16:0/1321480854 >> 172.21.15.16:0/1321480854 conn(0x7f49a40d0c30 msgr2=0x7f49a40b1d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:21.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.103+0000 7f49aa084700 1 -- 172.21.15.16:0/1321480854 shutdown_connections 2023-05-20T01:17:21.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.103+0000 7f49aa084700 1 -- 172.21.15.16:0/1321480854 wait complete. 2023-05-20T01:17:21.118 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-05-20T01:16:20.934366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-05-20T01:16:21.372294+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-05-20T01:16:26.208287+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-05-20T01:16:27.292268+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-05-20T01:16:30.357285+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-05-20T01:17:21.119 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-05-20T01:17:21.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.245+0000 7fa9da2fd700 1 Processor -- start 2023-05-20T01:17:21.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.245+0000 7fa9da2fd700 1 -- start start 2023-05-20T01:17:21.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.245+0000 7fa9da2fd700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fa9d406e020 0x7fa9d40710d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:21.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.245+0000 7fa9da2fd700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9d4072430 con 0x7fa9d415cf40 2023-05-20T01:17:21.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.245+0000 7fa9da2fd700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9d406e460 con 0x7fa9d40cd070 2023-05-20T01:17:21.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.245+0000 7fa9da2fd700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9d4071610 con 0x7fa9d406e020 2023-05-20T01:17:21.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.246+0000 7fa9d3fff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fa9d415cf40 0x7fa9d40cc960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:42580/0 (socket says 172.21.15.16:42580) 2023-05-20T01:17:21.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.246+0000 7fa9d3fff700 1 -- 172.21.15.16:0/4089399309 learned_addr learned my addr 172.21.15.16:0/4089399309 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:17:21.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.246+0000 7fa9d889a700 1 --2- 172.21.15.16:0/4089399309 >> v2:172.21.15.132:3300/0 conn(0x7fa9d406e020 0x7fa9d40710d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:21.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.246+0000 7fa9d2ffd700 1 -- 172.21.15.16:0/4089399309 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2489281631 0 0) 0x7fa9d4072430 con 0x7fa9d415cf40 2023-05-20T01:17:21.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.246+0000 7fa9d2ffd700 1 -- 172.21.15.16:0/4089399309 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa9bc0035b0 con 0x7fa9d415cf40 2023-05-20T01:17:21.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.246+0000 7fa9d2ffd700 1 -- 172.21.15.16:0/4089399309 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 322684412 0 0) 0x7fa9bc0035b0 con 0x7fa9d415cf40 2023-05-20T01:17:21.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.246+0000 7fa9d2ffd700 1 -- 172.21.15.16:0/4089399309 >> v2:172.21.15.132:3300/0 conn(0x7fa9d406e020 msgr2=0x7fa9d40710d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:21.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.246+0000 7fa9d2ffd700 1 --2- 172.21.15.16:0/4089399309 >> v2:172.21.15.132:3300/0 conn(0x7fa9d406e020 0x7fa9d40710d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:21.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.246+0000 7fa9d2ffd700 1 -- 172.21.15.16:0/4089399309 >> v1:172.21.15.132:6789/0 conn(0x7fa9d40cd070 legacy=0x7fa9d40cabb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:21.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.246+0000 7fa9d2ffd700 1 -- 172.21.15.16:0/4089399309 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9d4071820 con 0x7fa9d415cf40 2023-05-20T01:17:21.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.246+0000 7fa9d2ffd700 1 -- 172.21.15.16:0/4089399309 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7fa9c4001670 con 0x7fa9d415cf40 2023-05-20T01:17:21.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.246+0000 7fa9d2ffd700 1 -- 172.21.15.16:0/4089399309 >> v1:172.21.15.16:6789/0 conn(0x7fa9d415cf40 legacy=0x7fa9d40cc960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:21.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.246+0000 7fa9d2ffd700 1 --2- 172.21.15.16:0/4089399309 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa9bc003a00 0x7fa9bc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:21.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.246+0000 7fa9d2ffd700 1 --2- 172.21.15.16:0/4089399309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9bc0043f0 0x7fa9bc0048b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:21.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.246+0000 7fa9d2ffd700 1 --2- 172.21.15.16:0/4089399309 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9bc0050b0 0x7fa9bc00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:21.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.246+0000 7fa9d2ffd700 1 -- 172.21.15.16:0/4089399309 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d4071610 con 0x7fa9bc0043f0 2023-05-20T01:17:21.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.246+0000 7fa9d2ffd700 1 -- 172.21.15.16:0/4089399309 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9bc00c000 con 0x7fa9bc0050b0 2023-05-20T01:17:21.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.246+0000 7fa9d2ffd700 1 -- 172.21.15.16:0/4089399309 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9bc00c1c0 con 0x7fa9bc003a00 2023-05-20T01:17:21.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.246+0000 7fa9d37fe700 1 --2- 172.21.15.16:0/4089399309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9bc0043f0 0x7fa9bc0048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:21.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.246+0000 7fa9d3fff700 1 --2- 172.21.15.16:0/4089399309 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa9bc003a00 0x7fa9bc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:21.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.246+0000 7fa9d889a700 1 --2- 172.21.15.16:0/4089399309 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9bc0050b0 0x7fa9bc00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:21.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.247+0000 7fa9d37fe700 1 -- 172.21.15.16:0/4089399309 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa9bc003a00 msgr2=0x7fa9bc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:21.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.247+0000 7fa9d37fe700 1 --2- 172.21.15.16:0/4089399309 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa9bc003a00 0x7fa9bc003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:21.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.247+0000 7fa9d37fe700 1 -- 172.21.15.16:0/4089399309 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9bc0050b0 msgr2=0x7fa9bc00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:21.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.247+0000 7fa9d37fe700 1 --2- 172.21.15.16:0/4089399309 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9bc0050b0 0x7fa9bc00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:21.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.247+0000 7fa9d37fe700 1 -- 172.21.15.16:0/4089399309 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9bc0035b0 con 0x7fa9bc0043f0 2023-05-20T01:17:21.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.247+0000 7fa9d889a700 1 -- 172.21.15.16:0/4089399309 reap_dead start 2023-05-20T01:17:21.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.247+0000 7fa9d37fe700 1 --2- 172.21.15.16:0/4089399309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9bc0043f0 0x7fa9bc0048b0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fa9d40cd4c0 tx=0x7fa9bc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4366d5171ac5a962 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:21.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.247+0000 7fa9d2ffd700 1 -- 172.21.15.16:0/4089399309 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9c80052d0 con 0x7fa9bc0043f0 2023-05-20T01:17:21.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.247+0000 7fa9d2ffd700 1 -- 172.21.15.16:0/4089399309 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9c8005430 con 0x7fa9bc0043f0 2023-05-20T01:17:21.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.247+0000 7fa9d2ffd700 1 -- 172.21.15.16:0/4089399309 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9c80038a0 con 0x7fa9bc0043f0 2023-05-20T01:17:21.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.247+0000 7fa9da2fd700 1 -- 172.21.15.16:0/4089399309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9bc0043f0 msgr2=0x7fa9bc0048b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:21.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.247+0000 7fa9da2fd700 1 --2- 172.21.15.16:0/4089399309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9bc0043f0 0x7fa9bc0048b0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fa9d40cd4c0 tx=0x7fa9bc005880 comp rx=0 tx=0).stop 2023-05-20T01:17:21.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.248+0000 7fa9da2fd700 1 -- 172.21.15.16:0/4089399309 shutdown_connections 2023-05-20T01:17:21.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.248+0000 7fa9da2fd700 1 --2- 172.21.15.16:0/4089399309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9bc0043f0 0x7fa9bc0048b0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:21.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.248+0000 7fa9da2fd700 1 -- 172.21.15.16:0/4089399309 >> 172.21.15.16:0/4089399309 conn(0x7fa9d406c260 msgr2=0x7fa9d40cffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:21.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.248+0000 7fa9da2fd700 1 -- 172.21.15.16:0/4089399309 shutdown_connections 2023-05-20T01:17:21.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.248+0000 7fa9da2fd700 1 -- 172.21.15.16:0/4089399309 wait complete. 2023-05-20T01:17:21.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.249+0000 7fa9da2fd700 1 Processor -- start 2023-05-20T01:17:21.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.249+0000 7fa9da2fd700 1 -- start start 2023-05-20T01:17:21.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.250+0000 7fa9da2fd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9bc0043f0 0x7fa9d414ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:21.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.250+0000 7fa9da2fd700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa9d40c8bb0 0x7fa9d40c9020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:21.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.250+0000 7fa9da2fd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9d40c9560 0x7fa9d40c54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:21.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.250+0000 7fa9da2fd700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d40c4de0 con 0x7fa9bc0043f0 2023-05-20T01:17:21.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.250+0000 7fa9da2fd700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d414e310 con 0x7fa9d40c9560 2023-05-20T01:17:21.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.250+0000 7fa9da2fd700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d414f850 con 0x7fa9d40c8bb0 2023-05-20T01:17:21.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.250+0000 7fa9d3fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9bc0043f0 0x7fa9d414ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:21.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.250+0000 7fa9d3fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9bc0043f0 0x7fa9d414ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59296/0 (socket says 172.21.15.16:59296) 2023-05-20T01:17:21.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.250+0000 7fa9d3fff700 1 -- 172.21.15.16:0/417202992 learned_addr learned my addr 172.21.15.16:0/417202992 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:21.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.250+0000 7fa9d889a700 1 --2- 172.21.15.16:0/417202992 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9d40c9560 0x7fa9d40c54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:21.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.250+0000 7fa9d37fe700 1 --2- 172.21.15.16:0/417202992 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa9d40c8bb0 0x7fa9d40c9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:21.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.250+0000 7fa9d3fff700 1 -- 172.21.15.16:0/417202992 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa9d40c8bb0 msgr2=0x7fa9d40c9020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:21.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.250+0000 7fa9d3fff700 1 --2- 172.21.15.16:0/417202992 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa9d40c8bb0 0x7fa9d40c9020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:21.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.250+0000 7fa9d3fff700 1 -- 172.21.15.16:0/417202992 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9d40c9560 msgr2=0x7fa9d40c54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:21.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.250+0000 7fa9d3fff700 1 --2- 172.21.15.16:0/417202992 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9d40c9560 0x7fa9d40c54f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:21.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.250+0000 7fa9d3fff700 1 -- 172.21.15.16:0/417202992 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9d4071de0 con 0x7fa9bc0043f0 2023-05-20T01:17:21.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.250+0000 7fa9d37fe700 1 --2- 172.21.15.16:0/417202992 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa9d40c8bb0 0x7fa9d40c9020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:17:21.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.250+0000 7fa9d889a700 1 --2- 172.21.15.16:0/417202992 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9d40c9560 0x7fa9d40c54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:17:21.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.250+0000 7fa9d3fff700 1 --2- 172.21.15.16:0/417202992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9bc0043f0 0x7fa9d414ddd0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fa9c4007e60 tx=0x7fa9c4007570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:21.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.250+0000 7fa9d0ff9700 1 -- 172.21.15.16:0/417202992 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9c4012780 con 0x7fa9bc0043f0 2023-05-20T01:17:21.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.250+0000 7fa9d0ff9700 1 -- 172.21.15.16:0/417202992 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9c40128e0 con 0x7fa9bc0043f0 2023-05-20T01:17:21.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.250+0000 7fa9da2fd700 1 -- 172.21.15.16:0/417202992 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9d4071b00 con 0x7fa9bc0043f0 2023-05-20T01:17:21.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.250+0000 7fa9d0ff9700 1 -- 172.21.15.16:0/417202992 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9c4012c90 con 0x7fa9bc0043f0 2023-05-20T01:17:21.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.251+0000 7fa9da2fd700 1 -- 172.21.15.16:0/417202992 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9d40720c0 con 0x7fa9bc0043f0 2023-05-20T01:17:21.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.251+0000 7fa9da2fd700 1 -- 172.21.15.16:0/417202992 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9d40c5340 con 0x7fa9bc0043f0 2023-05-20T01:17:21.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.252+0000 7fa9d0ff9700 1 -- 172.21.15.16:0/417202992 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa9c4018020 con 0x7fa9bc0043f0 2023-05-20T01:17:21.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.252+0000 7fa9d0ff9700 1 --2- 172.21.15.16:0/417202992 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa9c006ad10 0x7fa9c006d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:21.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.253+0000 7fa9d37fe700 1 --2- 172.21.15.16:0/417202992 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa9c006ad10 0x7fa9c006d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:21.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.253+0000 7fa9d0ff9700 1 -- 172.21.15.16:0/417202992 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fa9c408b1a0 con 0x7fa9bc0043f0 2023-05-20T01:17:21.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.253+0000 7fa9d37fe700 1 --2- 172.21.15.16:0/417202992 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa9c006ad10 0x7fa9c006d1c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa9c8015040 tx=0x7fa9c8005000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:21.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.257+0000 7fa9d0ff9700 1 -- 172.21.15.16:0/417202992 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fa9c4055e90 con 0x7fa9bc0043f0 2023-05-20T01:17:21.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.459+0000 7fa9da2fd700 1 -- 172.21.15.16:0/417202992 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fa9d40747b0 con 0x7fa9bc0043f0 2023-05-20T01:17:21.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.459+0000 7fa9d0ff9700 1 -- 172.21.15.16:0/417202992 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v27) v1 ==== 92+0+10 (secure 0 0 0) 0x7fa9c4078030 con 0x7fa9bc0043f0 2023-05-20T01:17:21.463 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 8 2023-05-20T01:17:21.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.461+0000 7fa9da2fd700 1 -- 172.21.15.16:0/417202992 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa9c006ad10 msgr2=0x7fa9c006d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:21.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.461+0000 7fa9da2fd700 1 --2- 172.21.15.16:0/417202992 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa9c006ad10 0x7fa9c006d1c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa9c8015040 tx=0x7fa9c8005000 comp rx=0 tx=0).stop 2023-05-20T01:17:21.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.461+0000 7fa9da2fd700 1 -- 172.21.15.16:0/417202992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9bc0043f0 msgr2=0x7fa9d414ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:21.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.461+0000 7fa9da2fd700 1 --2- 172.21.15.16:0/417202992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9bc0043f0 0x7fa9d414ddd0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fa9c4007e60 tx=0x7fa9c4007570 comp rx=0 tx=0).stop 2023-05-20T01:17:21.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.461+0000 7fa9da2fd700 1 -- 172.21.15.16:0/417202992 shutdown_connections 2023-05-20T01:17:21.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.461+0000 7fa9da2fd700 1 --2- 172.21.15.16:0/417202992 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9d40c9560 0x7fa9d40c54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:21.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.461+0000 7fa9da2fd700 1 --2- 172.21.15.16:0/417202992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9bc0043f0 0x7fa9d414ddd0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:21.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.461+0000 7fa9da2fd700 1 --2- 172.21.15.16:0/417202992 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa9c006ad10 0x7fa9c006d1c0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:21.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.461+0000 7fa9da2fd700 1 --2- 172.21.15.16:0/417202992 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa9d40c8bb0 0x7fa9d40c9020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:21.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.461+0000 7fa9da2fd700 1 -- 172.21.15.16:0/417202992 >> 172.21.15.16:0/417202992 conn(0x7fa9d406c260 msgr2=0x7fa9d40caec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:21.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.461+0000 7fa9da2fd700 1 -- 172.21.15.16:0/417202992 shutdown_connections 2023-05-20T01:17:21.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.462+0000 7fa9da2fd700 1 -- 172.21.15.16:0/417202992 wait complete. 2023-05-20T01:17:21.477 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-05-20T01:17:21.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.602+0000 7f70923f2700 1 Processor -- start 2023-05-20T01:17:21.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.603+0000 7f70923f2700 1 -- start start 2023-05-20T01:17:21.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.604+0000 7f70923f2700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f708c0cf350 0x7f708c0b19b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:21.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.604+0000 7f70923f2700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f708c0b2020 con 0x7f708c0d16e0 2023-05-20T01:17:21.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.604+0000 7f70923f2700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f708c0b2300 con 0x7f708c0af2f0 2023-05-20T01:17:21.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.604+0000 7f70923f2700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f708c0b25e0 con 0x7f708c0cf350 2023-05-20T01:17:21.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.604+0000 7f708bfff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f708c0d16e0 0x7f708c144350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:42600/0 (socket says 172.21.15.16:42600) 2023-05-20T01:17:21.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.604+0000 7f709098f700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f708c0cf350 0x7f708c0b19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:21.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.604+0000 7f708bfff700 1 -- 172.21.15.16:0/4200965566 learned_addr learned my addr 172.21.15.16:0/4200965566 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:17:21.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.604+0000 7f709098f700 1 -- 172.21.15.16:0/4200965566 >> v1:172.21.15.132:6789/0 conn(0x7f708c0af2f0 legacy=0x7f708c0cec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:21.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.604+0000 7f709098f700 1 -- 172.21.15.16:0/4200965566 >> v1:172.21.15.16:6789/0 conn(0x7f708c0d16e0 legacy=0x7f708c144350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:21.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.604+0000 7f709098f700 1 -- 172.21.15.16:0/4200965566 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f708c0addc0 con 0x7f708c0cf350 2023-05-20T01:17:21.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.604+0000 7f709098f700 1 --2- 172.21.15.16:0/4200965566 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f708c0cf350 0x7f708c0b19b0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f7084009dc0 tx=0x7f708400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f436312452ee5a2 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:21.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.604+0000 7f708affd700 1 -- 172.21.15.16:0/4200965566 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f708400e070 con 0x7f708c0cf350 2023-05-20T01:17:21.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.605+0000 7f708affd700 1 -- 172.21.15.16:0/4200965566 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f708400bb90 con 0x7f708c0cf350 2023-05-20T01:17:21.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.605+0000 7f708affd700 1 -- 172.21.15.16:0/4200965566 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7084012430 con 0x7f708c0cf350 2023-05-20T01:17:21.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.605+0000 7f70923f2700 1 -- 172.21.15.16:0/4200965566 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f708c0cf350 msgr2=0x7f708c0b19b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:21.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.605+0000 7f70923f2700 1 --2- 172.21.15.16:0/4200965566 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f708c0cf350 0x7f708c0b19b0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f7084009dc0 tx=0x7f708400b330 comp rx=0 tx=0).stop 2023-05-20T01:17:21.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.605+0000 7f70923f2700 1 -- 172.21.15.16:0/4200965566 shutdown_connections 2023-05-20T01:17:21.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.605+0000 7f70923f2700 1 --2- 172.21.15.16:0/4200965566 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f708c0cf350 0x7f708c0b19b0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:21.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.605+0000 7f70923f2700 1 -- 172.21.15.16:0/4200965566 >> 172.21.15.16:0/4200965566 conn(0x7f708c06ae30 msgr2=0x7f708c09a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:21.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.605+0000 7f70923f2700 1 -- 172.21.15.16:0/4200965566 shutdown_connections 2023-05-20T01:17:21.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.605+0000 7f70923f2700 1 -- 172.21.15.16:0/4200965566 wait complete. 2023-05-20T01:17:21.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.606+0000 7f70923f2700 1 Processor -- start 2023-05-20T01:17:21.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.607+0000 7f70923f2700 1 -- start start 2023-05-20T01:17:21.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.607+0000 7f70923f2700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f708c0af2f0 0x7f708c06f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:21.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.607+0000 7f70923f2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f708c0cf350 0x7f708c0cc390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:21.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.607+0000 7f70923f2700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f708c0d16e0 0x7f708c0cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:21.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.607+0000 7f70923f2700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f708c15aab0 con 0x7f708c0cf350 2023-05-20T01:17:21.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.607+0000 7f70923f2700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f708c15a4f0 con 0x7f708c0af2f0 2023-05-20T01:17:21.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.607+0000 7f70923f2700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f708c15a7d0 con 0x7f708c0d16e0 2023-05-20T01:17:21.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.607+0000 7f708b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f708c0cf350 0x7f708c0cc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:21.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.607+0000 7f708b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f708c0cf350 0x7f708c0cc390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59312/0 (socket says 172.21.15.16:59312) 2023-05-20T01:17:21.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.607+0000 7f708b7fe700 1 -- 172.21.15.16:0/1633921870 learned_addr learned my addr 172.21.15.16:0/1633921870 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:21.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.607+0000 7f709098f700 1 --2- 172.21.15.16:0/1633921870 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f708c0d16e0 0x7f708c0cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:21.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.607+0000 7f708bfff700 1 --2- 172.21.15.16:0/1633921870 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f708c0af2f0 0x7f708c06f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:21.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.608+0000 7f708b7fe700 1 -- 172.21.15.16:0/1633921870 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f708c0d16e0 msgr2=0x7f708c0cc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:21.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.608+0000 7f708b7fe700 1 --2- 172.21.15.16:0/1633921870 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f708c0d16e0 0x7f708c0cc8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:21.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.608+0000 7f708b7fe700 1 -- 172.21.15.16:0/1633921870 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f708c0af2f0 msgr2=0x7f708c06f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:21.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.608+0000 7f708b7fe700 1 --2- 172.21.15.16:0/1633921870 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f708c0af2f0 0x7f708c06f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:21.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.608+0000 7f708b7fe700 1 -- 172.21.15.16:0/1633921870 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7084009a70 con 0x7f708c0cf350 2023-05-20T01:17:21.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.608+0000 7f708b7fe700 1 --2- 172.21.15.16:0/1633921870 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f708c0cf350 0x7f708c0cc390 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f707c00c4e0 tx=0x7f707c00c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:21.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.608+0000 7f7088ff9700 1 -- 172.21.15.16:0/1633921870 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f707c0052d0 con 0x7f708c0cf350 2023-05-20T01:17:21.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.608+0000 7f7088ff9700 1 -- 172.21.15.16:0/1633921870 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f707c005430 con 0x7f708c0cf350 2023-05-20T01:17:21.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.608+0000 7f70923f2700 1 -- 172.21.15.16:0/1633921870 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f708c1655f0 con 0x7f708c0cf350 2023-05-20T01:17:21.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.608+0000 7f7088ff9700 1 -- 172.21.15.16:0/1633921870 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f707c0038c0 con 0x7f708c0cf350 2023-05-20T01:17:21.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.608+0000 7f70923f2700 1 -- 172.21.15.16:0/1633921870 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f708c0b28c0 con 0x7f708c0cf350 2023-05-20T01:17:21.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.609+0000 7f70923f2700 1 -- 172.21.15.16:0/1633921870 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f708c165780 con 0x7f708c0cf350 2023-05-20T01:17:21.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.609+0000 7f7088ff9700 1 -- 172.21.15.16:0/1633921870 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f707c003a20 con 0x7f708c0cf350 2023-05-20T01:17:21.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.610+0000 7f7088ff9700 1 --2- 172.21.15.16:0/1633921870 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f706c06ace0 0x7f706c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:21.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.610+0000 7f7088ff9700 1 -- 172.21.15.16:0/1633921870 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f707c088b50 con 0x7f708c0cf350 2023-05-20T01:17:21.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.610+0000 7f708bfff700 1 --2- 172.21.15.16:0/1633921870 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f706c06ace0 0x7f706c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:21.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.610+0000 7f708bfff700 1 --2- 172.21.15.16:0/1633921870 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f706c06ace0 0x7f706c06d190 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f7074005910 tx=0x7f7074009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:21.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.614+0000 7f7088ff9700 1 -- 172.21.15.16:0/1633921870 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f707c0538d0 con 0x7f708c0cf350 2023-05-20T01:17:21.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.811+0000 7f70923f2700 1 -- 172.21.15.16:0/1633921870 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f708c0747b0 con 0x7f708c0cf350 2023-05-20T01:17:21.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.811+0000 7f7088ff9700 1 -- 172.21.15.16:0/1633921870 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f707c017020 con 0x7f708c0cf350 2023-05-20T01:17:21.815 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-05-20T01:17:21.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.813+0000 7f70923f2700 1 -- 172.21.15.16:0/1633921870 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f706c06ace0 msgr2=0x7f706c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:21.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.813+0000 7f70923f2700 1 --2- 172.21.15.16:0/1633921870 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f706c06ace0 0x7f706c06d190 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f7074005910 tx=0x7f7074009040 comp rx=0 tx=0).stop 2023-05-20T01:17:21.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.813+0000 7f70923f2700 1 -- 172.21.15.16:0/1633921870 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f708c0cf350 msgr2=0x7f708c0cc390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:21.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.813+0000 7f70923f2700 1 --2- 172.21.15.16:0/1633921870 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f708c0cf350 0x7f708c0cc390 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f707c00c4e0 tx=0x7f707c00c7f0 comp rx=0 tx=0).stop 2023-05-20T01:17:21.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.813+0000 7f70923f2700 1 -- 172.21.15.16:0/1633921870 shutdown_connections 2023-05-20T01:17:21.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.813+0000 7f70923f2700 1 --2- 172.21.15.16:0/1633921870 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f706c06ace0 0x7f706c06d190 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:21.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.813+0000 7f70923f2700 1 --2- 172.21.15.16:0/1633921870 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f708c0d16e0 0x7f708c0cc8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:21.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.813+0000 7f70923f2700 1 --2- 172.21.15.16:0/1633921870 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f708c0af2f0 0x7f708c06f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:21.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.813+0000 7f70923f2700 1 --2- 172.21.15.16:0/1633921870 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f708c0cf350 0x7f708c0cc390 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:21.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.813+0000 7f70923f2700 1 -- 172.21.15.16:0/1633921870 >> 172.21.15.16:0/1633921870 conn(0x7f708c06ae30 msgr2=0x7f708c0b2a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:21.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.813+0000 7f70923f2700 1 -- 172.21.15.16:0/1633921870 shutdown_connections 2023-05-20T01:17:21.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.813+0000 7f70923f2700 1 -- 172.21.15.16:0/1633921870 wait complete. 2023-05-20T01:17:21.830 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-05-20T01:17:21.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.954+0000 7f5f22d57700 1 Processor -- start 2023-05-20T01:17:21.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.954+0000 7f5f22d57700 1 -- start start 2023-05-20T01:17:21.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.955+0000 7f5f22d57700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5f1c0cd1c0 0x7f5f1c0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:21.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.955+0000 7f5f22d57700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5f1c0cd630 0x7f5f1c0715f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:21.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.955+0000 7f5f22d57700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f1c0ccd30 con 0x7f5f1c159b50 2023-05-20T01:17:21.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.955+0000 7f5f22d57700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f1c0ce990 con 0x7f5f1c0cd1c0 2023-05-20T01:17:21.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.955+0000 7f5f22d57700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f1c0cc6e0 con 0x7f5f1c0cd630 2023-05-20T01:17:21.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.955+0000 7f5f20af3700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f5f1c159b50 0x7f5f1c06e0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:42614/0 (socket says 172.21.15.16:42614) 2023-05-20T01:17:21.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.955+0000 7f5f20af3700 1 -- 172.21.15.16:0/857229040 learned_addr learned my addr 172.21.15.16:0/857229040 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:17:21.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.955+0000 7f5f1bfff700 1 --2- 172.21.15.16:0/857229040 >> v2:172.21.15.16:3300/0 conn(0x7f5f1c0cd1c0 0x7f5f1c0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:21.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.956+0000 7f5f212f4700 1 --2- 172.21.15.16:0/857229040 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5f1c0cd630 0x7f5f1c0715f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:21.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.956+0000 7f5f1bfff700 1 -- 172.21.15.16:0/857229040 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5f1c0cd630 msgr2=0x7f5f1c0715f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:21.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.956+0000 7f5f1bfff700 1 --2- 172.21.15.16:0/857229040 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5f1c0cd630 0x7f5f1c0715f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:21.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.956+0000 7f5f1bfff700 1 -- 172.21.15.16:0/857229040 >> v1:172.21.15.16:6789/0 conn(0x7f5f1c159b50 legacy=0x7f5f1c06e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:21.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.956+0000 7f5f1bfff700 1 -- 172.21.15.16:0/857229040 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f1c0cc9c0 con 0x7f5f1c0cd1c0 2023-05-20T01:17:21.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.956+0000 7f5f1b7fe700 1 -- 172.21.15.16:0/857229040 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2030121960 0 0) 0x7f5f1c0ccd30 con 0x7f5f1c159b50 2023-05-20T01:17:21.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.956+0000 7f5f1bfff700 1 --2- 172.21.15.16:0/857229040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f1c0cd1c0 0x7f5f1c0710b0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f5f08009840 tx=0x7f5f08009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a42c115b81a18c8a server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:21.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.956+0000 7f5f1b7fe700 1 -- 172.21.15.16:0/857229040 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f08011830 con 0x7f5f1c0cd1c0 2023-05-20T01:17:21.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.956+0000 7f5f1b7fe700 1 -- 172.21.15.16:0/857229040 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f08011990 con 0x7f5f1c0cd1c0 2023-05-20T01:17:21.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.956+0000 7f5f1b7fe700 1 -- 172.21.15.16:0/857229040 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f08011d20 con 0x7f5f1c0cd1c0 2023-05-20T01:17:21.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.957+0000 7f5f22d57700 1 -- 172.21.15.16:0/857229040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f1c0cd1c0 msgr2=0x7f5f1c0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:21.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.957+0000 7f5f22d57700 1 --2- 172.21.15.16:0/857229040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f1c0cd1c0 0x7f5f1c0710b0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f5f08009840 tx=0x7f5f08009b50 comp rx=0 tx=0).stop 2023-05-20T01:17:21.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.957+0000 7f5f22d57700 1 -- 172.21.15.16:0/857229040 shutdown_connections 2023-05-20T01:17:21.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.957+0000 7f5f22d57700 1 --2- 172.21.15.16:0/857229040 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5f1c0cd630 0x7f5f1c0715f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:21.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.957+0000 7f5f22d57700 1 --2- 172.21.15.16:0/857229040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f1c0cd1c0 0x7f5f1c0710b0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:21.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.957+0000 7f5f22d57700 1 -- 172.21.15.16:0/857229040 >> 172.21.15.16:0/857229040 conn(0x7f5f1c0cffd0 msgr2=0x7f5f1c0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:21.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.957+0000 7f5f22d57700 1 -- 172.21.15.16:0/857229040 shutdown_connections 2023-05-20T01:17:21.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.957+0000 7f5f22d57700 1 -- 172.21.15.16:0/857229040 wait complete. 2023-05-20T01:17:21.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.958+0000 7f5f22d57700 1 Processor -- start 2023-05-20T01:17:21.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.958+0000 7f5f22d57700 1 -- start start 2023-05-20T01:17:21.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.959+0000 7f5f22d57700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5f1c0cd1c0 0x7f5f1c1525d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:21.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.959+0000 7f5f22d57700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5f1c0cd630 0x7f5f1c0c97e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:21.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.959+0000 7f5f22d57700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f1c159b50 0x7f5f1c0c70a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:21.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.959+0000 7f5f22d57700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f1c1539a0 con 0x7f5f1c159b50 2023-05-20T01:17:21.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.959+0000 7f5f22d57700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f1c152b10 con 0x7f5f1c0cd1c0 2023-05-20T01:17:21.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.959+0000 7f5f22d57700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f1c152d20 con 0x7f5f1c0cd630 2023-05-20T01:17:21.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.959+0000 7f5f212f4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f1c159b50 0x7f5f1c0c70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:21.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.959+0000 7f5f212f4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f1c159b50 0x7f5f1c0c70a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59328/0 (socket says 172.21.15.16:59328) 2023-05-20T01:17:21.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.959+0000 7f5f212f4700 1 -- 172.21.15.16:0/2975691889 learned_addr learned my addr 172.21.15.16:0/2975691889 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:21.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.959+0000 7f5f1bfff700 1 --2- 172.21.15.16:0/2975691889 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5f1c0cd630 0x7f5f1c0c97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:21.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.959+0000 7f5f20af3700 1 --2- 172.21.15.16:0/2975691889 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5f1c0cd1c0 0x7f5f1c1525d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:21.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.959+0000 7f5f212f4700 1 -- 172.21.15.16:0/2975691889 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5f1c0cd630 msgr2=0x7f5f1c0c97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:21.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.959+0000 7f5f212f4700 1 --2- 172.21.15.16:0/2975691889 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5f1c0cd630 0x7f5f1c0c97e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:21.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.959+0000 7f5f212f4700 1 -- 172.21.15.16:0/2975691889 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5f1c0cd1c0 msgr2=0x7f5f1c1525d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:21.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.959+0000 7f5f212f4700 1 --2- 172.21.15.16:0/2975691889 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5f1c0cd1c0 0x7f5f1c1525d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:21.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.959+0000 7f5f212f4700 1 -- 172.21.15.16:0/2975691889 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f080094a0 con 0x7f5f1c159b50 2023-05-20T01:17:21.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.959+0000 7f5f212f4700 1 --2- 172.21.15.16:0/2975691889 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f1c159b50 0x7f5f1c0c70a0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f5f1400efc0 tx=0x7f5f1400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:21.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.960+0000 7f5f197fa700 1 -- 172.21.15.16:0/2975691889 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f14009ea0 con 0x7f5f1c159b50 2023-05-20T01:17:21.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.960+0000 7f5f197fa700 1 -- 172.21.15.16:0/2975691889 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f1400f040 con 0x7f5f1c159b50 2023-05-20T01:17:21.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.960+0000 7f5f22d57700 1 -- 172.21.15.16:0/2975691889 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f1c0cec70 con 0x7f5f1c159b50 2023-05-20T01:17:21.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.960+0000 7f5f197fa700 1 -- 172.21.15.16:0/2975691889 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f1400ced0 con 0x7f5f1c159b50 2023-05-20T01:17:21.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.960+0000 7f5f22d57700 1 -- 172.21.15.16:0/2975691889 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f1c071e80 con 0x7f5f1c159b50 2023-05-20T01:17:21.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.961+0000 7f5f22d57700 1 -- 172.21.15.16:0/2975691889 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f1c0747b0 con 0x7f5f1c159b50 2023-05-20T01:17:21.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.961+0000 7f5f197fa700 1 -- 172.21.15.16:0/2975691889 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5f14014480 con 0x7f5f1c159b50 2023-05-20T01:17:21.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.961+0000 7f5f197fa700 1 --2- 172.21.15.16:0/2975691889 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f5f0006ace0 0x7f5f0006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:21.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.962+0000 7f5f197fa700 1 -- 172.21.15.16:0/2975691889 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f5f1408a210 con 0x7f5f1c159b50 2023-05-20T01:17:21.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.962+0000 7f5f20af3700 1 --2- 172.21.15.16:0/2975691889 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f5f0006ace0 0x7f5f0006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:21.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.962+0000 7f5f20af3700 1 --2- 172.21.15.16:0/2975691889 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f5f0006ace0 0x7f5f0006d190 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f5f10007fd0 tx=0x7f5f10009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:21.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:21.966+0000 7f5f197fa700 1 -- 172.21.15.16:0/2975691889 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f5f14055020 con 0x7f5f1c159b50 2023-05-20T01:17:22.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.166+0000 7f5f22d57700 1 -- 172.21.15.16:0/2975691889 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5f1c153ae0 con 0x7f5f1c159b50 2023-05-20T01:17:22.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.166+0000 7f5f197fa700 1 -- 172.21.15.16:0/2975691889 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5f1c153ae0 con 0x7f5f1c159b50 2023-05-20T01:17:22.171 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-05-20T01:17:22.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.168+0000 7f5f22d57700 1 -- 172.21.15.16:0/2975691889 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f5f0006ace0 msgr2=0x7f5f0006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:22.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.168+0000 7f5f22d57700 1 --2- 172.21.15.16:0/2975691889 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f5f0006ace0 0x7f5f0006d190 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f5f10007fd0 tx=0x7f5f10009040 comp rx=0 tx=0).stop 2023-05-20T01:17:22.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.168+0000 7f5f22d57700 1 -- 172.21.15.16:0/2975691889 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f1c159b50 msgr2=0x7f5f1c0c70a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:22.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.168+0000 7f5f22d57700 1 --2- 172.21.15.16:0/2975691889 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f1c159b50 0x7f5f1c0c70a0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f5f1400efc0 tx=0x7f5f1400c3a0 comp rx=0 tx=0).stop 2023-05-20T01:17:22.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.169+0000 7f5f22d57700 1 -- 172.21.15.16:0/2975691889 shutdown_connections 2023-05-20T01:17:22.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.169+0000 7f5f22d57700 1 --2- 172.21.15.16:0/2975691889 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f1c159b50 0x7f5f1c0c70a0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:22.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.169+0000 7f5f22d57700 1 --2- 172.21.15.16:0/2975691889 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5f1c0cd1c0 0x7f5f1c1525d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:22.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.169+0000 7f5f22d57700 1 --2- 172.21.15.16:0/2975691889 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f5f0006ace0 0x7f5f0006d190 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:22.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.169+0000 7f5f22d57700 1 --2- 172.21.15.16:0/2975691889 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5f1c0cd630 0x7f5f1c0c97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:22.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.169+0000 7f5f22d57700 1 -- 172.21.15.16:0/2975691889 >> 172.21.15.16:0/2975691889 conn(0x7f5f1c0cffd0 msgr2=0x7f5f1c0710b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:22.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.169+0000 7f5f22d57700 1 -- 172.21.15.16:0/2975691889 shutdown_connections 2023-05-20T01:17:22.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.169+0000 7f5f22d57700 1 -- 172.21.15.16:0/2975691889 wait complete. 2023-05-20T01:17:22.185 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-05-20T01:17:22.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.310+0000 7f235d08e700 1 Processor -- start 2023-05-20T01:17:22.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.310+0000 7f235d08e700 1 -- start start 2023-05-20T01:17:22.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.310+0000 7f235d08e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f2358071880 0x7f2358071c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:22.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.310+0000 7f235d08e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f23580cbfb0 0x7f23580b2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:22.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.310+0000 7f235d08e700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23580d0e10 con 0x7f23580cd620 2023-05-20T01:17:22.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.310+0000 7f235d08e700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23580cd200 con 0x7f2358071880 2023-05-20T01:17:22.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.310+0000 7f235d08e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2358072190 con 0x7f23580cbfb0 2023-05-20T01:17:22.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.311+0000 7f235659c700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f23580cd620 0x7f23580cb8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:42628/0 (socket says 172.21.15.16:42628) 2023-05-20T01:17:22.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.311+0000 7f235659c700 1 -- 172.21.15.16:0/3284582856 learned_addr learned my addr 172.21.15.16:0/3284582856 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:17:22.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.311+0000 7f235759e700 1 --2- 172.21.15.16:0/3284582856 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f23580cbfb0 0x7f23580b2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:22.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.311+0000 7f2356d9d700 1 --2- 172.21.15.16:0/3284582856 >> v2:172.21.15.132:3300/0 conn(0x7f2358071880 0x7f2358071c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:22.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.311+0000 7f2355d9b700 1 -- 172.21.15.16:0/3284582856 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 328631785 0 0) 0x7f23580d0e10 con 0x7f23580cd620 2023-05-20T01:17:22.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.311+0000 7f2355d9b700 1 -- 172.21.15.16:0/3284582856 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f23440035b0 con 0x7f23580cd620 2023-05-20T01:17:22.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.311+0000 7f2355d9b700 1 -- 172.21.15.16:0/3284582856 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 131529161 0 0) 0x7f23440035b0 con 0x7f23580cd620 2023-05-20T01:17:22.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.311+0000 7f2355d9b700 1 -- 172.21.15.16:0/3284582856 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f23580cbfb0 msgr2=0x7f23580b2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:22.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.311+0000 7f2355d9b700 1 --2- 172.21.15.16:0/3284582856 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f23580cbfb0 0x7f23580b2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:22.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.311+0000 7f2355d9b700 1 -- 172.21.15.16:0/3284582856 >> v2:172.21.15.132:3300/0 conn(0x7f2358071880 msgr2=0x7f2358071c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:22.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.311+0000 7f2355d9b700 1 --2- 172.21.15.16:0/3284582856 >> v2:172.21.15.132:3300/0 conn(0x7f2358071880 0x7f2358071c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:22.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.311+0000 7f2355d9b700 1 -- 172.21.15.16:0/3284582856 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23580cc580 con 0x7f23580cd620 2023-05-20T01:17:22.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.311+0000 7f2355d9b700 1 -- 172.21.15.16:0/3284582856 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7f2340001670 con 0x7f23580cd620 2023-05-20T01:17:22.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.311+0000 7f2355d9b700 1 -- 172.21.15.16:0/3284582856 >> v1:172.21.15.16:6789/0 conn(0x7f23580cd620 legacy=0x7f23580cb8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:22.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.311+0000 7f2355d9b700 1 --2- 172.21.15.16:0/3284582856 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2344003a00 0x7f2344003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:22.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.311+0000 7f2355d9b700 1 --2- 172.21.15.16:0/3284582856 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f23580cbfb0 0x7f2344004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:22.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.311+0000 7f2355d9b700 1 --2- 172.21.15.16:0/3284582856 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2344004d90 0x7f2344005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:22.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.311+0000 7f2355d9b700 1 -- 172.21.15.16:0/3284582856 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23580cd200 con 0x7f2344003a00 2023-05-20T01:17:22.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.311+0000 7f2355d9b700 1 -- 172.21.15.16:0/3284582856 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2358072190 con 0x7f2344004d90 2023-05-20T01:17:22.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.311+0000 7f2355d9b700 1 -- 172.21.15.16:0/3284582856 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f234400ba10 con 0x7f23580cbfb0 2023-05-20T01:17:22.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.311+0000 7f2356d9d700 1 --2- 172.21.15.16:0/3284582856 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2344003a00 0x7f2344003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:22.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.311+0000 7f235659c700 1 --2- 172.21.15.16:0/3284582856 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f23580cbfb0 0x7f2344004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:22.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.312+0000 7f235759e700 1 --2- 172.21.15.16:0/3284582856 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2344004d90 0x7f2344005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:22.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.312+0000 7f2356d9d700 1 -- 172.21.15.16:0/3284582856 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f23580cbfb0 msgr2=0x7f2344004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:22.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.312+0000 7f2356d9d700 1 --2- 172.21.15.16:0/3284582856 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f23580cbfb0 0x7f2344004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:22.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.312+0000 7f2356d9d700 1 -- 172.21.15.16:0/3284582856 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2344004d90 msgr2=0x7f2344005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:22.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.312+0000 7f2356d9d700 1 --2- 172.21.15.16:0/3284582856 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2344004d90 0x7f2344005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:22.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.312+0000 7f2356d9d700 1 -- 172.21.15.16:0/3284582856 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23440035b0 con 0x7f2344003a00 2023-05-20T01:17:22.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.312+0000 7f235659c700 1 --2- 172.21.15.16:0/3284582856 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f23580cbfb0 0x7f2344004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:17:22.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.312+0000 7f2356d9d700 1 --2- 172.21.15.16:0/3284582856 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2344003a00 0x7f2344003e10 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f2348009b40 tx=0x7f2344005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=508491e71cfcbaf6 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:22.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.312+0000 7f2355d9b700 1 -- 172.21.15.16:0/3284582856 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2348005a20 con 0x7f2344003a00 2023-05-20T01:17:22.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.312+0000 7f2355d9b700 1 -- 172.21.15.16:0/3284582856 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2348005b80 con 0x7f2344003a00 2023-05-20T01:17:22.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.312+0000 7f2355d9b700 1 -- 172.21.15.16:0/3284582856 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f234800e6c0 con 0x7f2344003a00 2023-05-20T01:17:22.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.312+0000 7f235d08e700 1 -- 172.21.15.16:0/3284582856 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2344003a00 msgr2=0x7f2344003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:22.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.312+0000 7f235d08e700 1 --2- 172.21.15.16:0/3284582856 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2344003a00 0x7f2344003e10 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f2348009b40 tx=0x7f2344005880 comp rx=0 tx=0).stop 2023-05-20T01:17:22.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.312+0000 7f235759e700 1 -- 172.21.15.16:0/3284582856 reap_dead start 2023-05-20T01:17:22.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.312+0000 7f235d08e700 1 -- 172.21.15.16:0/3284582856 shutdown_connections 2023-05-20T01:17:22.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.312+0000 7f235d08e700 1 -- 172.21.15.16:0/3284582856 >> 172.21.15.16:0/3284582856 conn(0x7f23580cffd0 msgr2=0x7f23580d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:22.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.312+0000 7f235d08e700 1 -- 172.21.15.16:0/3284582856 shutdown_connections 2023-05-20T01:17:22.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.313+0000 7f235d08e700 1 -- 172.21.15.16:0/3284582856 wait complete. 2023-05-20T01:17:22.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.314+0000 7f235d08e700 1 Processor -- start 2023-05-20T01:17:22.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.314+0000 7f235d08e700 1 -- start start 2023-05-20T01:17:22.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.315+0000 7f235d08e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2344003a00 0x7f23581534f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:22.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.315+0000 7f235d08e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f235815c0a0 0x7f23580afb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:22.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.315+0000 7f235d08e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f235815c510 0x7f23580b0060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:22.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.315+0000 7f235d08e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23580c4130 con 0x7f2344003a00 2023-05-20T01:17:22.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.315+0000 7f235d08e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2358155d60 con 0x7f235815c0a0 2023-05-20T01:17:22.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.315+0000 7f235d08e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2358156040 con 0x7f235815c510 2023-05-20T01:17:22.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.315+0000 7f2356d9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2344003a00 0x7f23581534f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:22.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.315+0000 7f2356d9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2344003a00 0x7f23581534f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59344/0 (socket says 172.21.15.16:59344) 2023-05-20T01:17:22.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.315+0000 7f2356d9d700 1 -- 172.21.15.16:0/1921837173 learned_addr learned my addr 172.21.15.16:0/1921837173 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:22.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.315+0000 7f235759e700 1 --2- 172.21.15.16:0/1921837173 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f235815c510 0x7f23580b0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:22.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.315+0000 7f235659c700 1 --2- 172.21.15.16:0/1921837173 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f235815c0a0 0x7f23580afb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:22.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.315+0000 7f2356d9d700 1 -- 172.21.15.16:0/1921837173 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f235815c510 msgr2=0x7f23580b0060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:22.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.315+0000 7f2356d9d700 1 --2- 172.21.15.16:0/1921837173 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f235815c510 0x7f23580b0060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:22.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.315+0000 7f2356d9d700 1 -- 172.21.15.16:0/1921837173 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f235815c0a0 msgr2=0x7f23580afb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:22.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.315+0000 7f2356d9d700 1 --2- 172.21.15.16:0/1921837173 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f235815c0a0 0x7f23580afb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:22.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.315+0000 7f2356d9d700 1 -- 172.21.15.16:0/1921837173 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23580ccb40 con 0x7f2344003a00 2023-05-20T01:17:22.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.316+0000 7f2356d9d700 1 --2- 172.21.15.16:0/1921837173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2344003a00 0x7f23581534f0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f2344004720 tx=0x7f23480047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:22.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.316+0000 7f234f7fe700 1 -- 172.21.15.16:0/1921837173 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2348021070 con 0x7f2344003a00 2023-05-20T01:17:22.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.316+0000 7f234f7fe700 1 -- 172.21.15.16:0/1921837173 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2348004030 con 0x7f2344003a00 2023-05-20T01:17:22.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.316+0000 7f235d08e700 1 -- 172.21.15.16:0/1921837173 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23580cc860 con 0x7f2344003a00 2023-05-20T01:17:22.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.316+0000 7f234f7fe700 1 -- 172.21.15.16:0/1921837173 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2348005a20 con 0x7f2344003a00 2023-05-20T01:17:22.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.316+0000 7f235d08e700 1 -- 172.21.15.16:0/1921837173 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23580cce20 con 0x7f2344003a00 2023-05-20T01:17:22.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.317+0000 7f235d08e700 1 -- 172.21.15.16:0/1921837173 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23580c4410 con 0x7f2344003a00 2023-05-20T01:17:22.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.317+0000 7f234f7fe700 1 -- 172.21.15.16:0/1921837173 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f23480043c0 con 0x7f2344003a00 2023-05-20T01:17:22.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.318+0000 7f234f7fe700 1 --2- 172.21.15.16:0/1921837173 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f233806ace0 0x7f233806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:22.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.318+0000 7f234f7fe700 1 -- 172.21.15.16:0/1921837173 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f234808a8e0 con 0x7f2344003a00 2023-05-20T01:17:22.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.318+0000 7f235659c700 1 --2- 172.21.15.16:0/1921837173 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f233806ace0 0x7f233806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:22.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.318+0000 7f235659c700 1 --2- 172.21.15.16:0/1921837173 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f233806ace0 0x7f233806d190 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f23581f3610 tx=0x7f2340007040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:22.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.322+0000 7f234f7fe700 1 -- 172.21.15.16:0/1921837173 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f23480556f0 con 0x7f2344003a00 2023-05-20T01:17:22.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.522+0000 7f235d08e700 1 -- 172.21.15.16:0/1921837173 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f23580747b0 con 0x7f2344003a00 2023-05-20T01:17:22.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.522+0000 7f234f7fe700 1 -- 172.21.15.16:0/1921837173 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2348058d10 con 0x7f2344003a00 2023-05-20T01:17:22.526 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2023-05-20T01:17:22.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.524+0000 7f235d08e700 1 -- 172.21.15.16:0/1921837173 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f233806ace0 msgr2=0x7f233806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:22.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.524+0000 7f235d08e700 1 --2- 172.21.15.16:0/1921837173 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f233806ace0 0x7f233806d190 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f23581f3610 tx=0x7f2340007040 comp rx=0 tx=0).stop 2023-05-20T01:17:22.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.524+0000 7f235d08e700 1 -- 172.21.15.16:0/1921837173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2344003a00 msgr2=0x7f23581534f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:22.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.524+0000 7f235d08e700 1 --2- 172.21.15.16:0/1921837173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2344003a00 0x7f23581534f0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f2344004720 tx=0x7f23480047c0 comp rx=0 tx=0).stop 2023-05-20T01:17:22.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.524+0000 7f235d08e700 1 -- 172.21.15.16:0/1921837173 shutdown_connections 2023-05-20T01:17:22.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.524+0000 7f235d08e700 1 --2- 172.21.15.16:0/1921837173 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f233806ace0 0x7f233806d190 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:22.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.524+0000 7f235d08e700 1 --2- 172.21.15.16:0/1921837173 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f235815c510 0x7f23580b0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:22.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.524+0000 7f235d08e700 1 --2- 172.21.15.16:0/1921837173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2344003a00 0x7f23581534f0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:22.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.524+0000 7f235d08e700 1 --2- 172.21.15.16:0/1921837173 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f235815c0a0 0x7f23580afb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:22.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.524+0000 7f235d08e700 1 -- 172.21.15.16:0/1921837173 >> 172.21.15.16:0/1921837173 conn(0x7f23580cffd0 msgr2=0x7f235806a510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:22.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.524+0000 7f235d08e700 1 -- 172.21.15.16:0/1921837173 shutdown_connections 2023-05-20T01:17:22.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.524+0000 7f235d08e700 1 -- 172.21.15.16:0/1921837173 wait complete. 2023-05-20T01:17:22.540 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-05-20T01:17:22.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.664+0000 7f867ad3e700 1 Processor -- start 2023-05-20T01:17:22.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.664+0000 7f867ad3e700 1 -- start start 2023-05-20T01:17:22.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.665+0000 7f867ad3e700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f86740cd1c0 0x7f86740710b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:22.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.665+0000 7f867ad3e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f86740cd630 0x7f86740715f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:22.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.665+0000 7f867ad3e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86740ccd30 con 0x7f8674159b50 2023-05-20T01:17:22.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.665+0000 7f867ad3e700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86740ce990 con 0x7f86740cd1c0 2023-05-20T01:17:22.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.665+0000 7f867ad3e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86740cc6e0 con 0x7f86740cd630 2023-05-20T01:17:22.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.665+0000 7f8673fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f86740cd1c0 0x7f86740710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:22.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.665+0000 7f8673fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f86740cd1c0 0x7f86740710b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59354/0 (socket says 172.21.15.16:59354) 2023-05-20T01:17:22.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.665+0000 7f8673fff700 1 -- 172.21.15.16:0/3713579385 learned_addr learned my addr 172.21.15.16:0/3713579385 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:22.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.665+0000 7f86792db700 1 --2- 172.21.15.16:0/3713579385 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f86740cd630 0x7f86740715f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:22.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.665+0000 7f8673fff700 1 -- 172.21.15.16:0/3713579385 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f86740cd630 msgr2=0x7f86740715f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:22.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.665+0000 7f8673fff700 1 --2- 172.21.15.16:0/3713579385 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f86740cd630 0x7f86740715f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:22.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.665+0000 7f8673fff700 1 -- 172.21.15.16:0/3713579385 >> v1:172.21.15.132:6789/0 conn(0x7f8674159b50 legacy=0x7f867406e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:22.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.665+0000 7f8673fff700 1 -- 172.21.15.16:0/3713579385 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86740cc9c0 con 0x7f86740cd1c0 2023-05-20T01:17:22.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.665+0000 7f8673fff700 1 --2- 172.21.15.16:0/3713579385 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86740cd1c0 0x7f86740710b0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f86600098f0 tx=0x7f8660009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17de228b91e3b8f5 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:22.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.665+0000 7f86737fe700 1 -- 172.21.15.16:0/3713579385 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8660011830 con 0x7f86740cd1c0 2023-05-20T01:17:22.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.666+0000 7f86737fe700 1 -- 172.21.15.16:0/3713579385 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8660011990 con 0x7f86740cd1c0 2023-05-20T01:17:22.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.666+0000 7f86737fe700 1 -- 172.21.15.16:0/3713579385 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8660011d20 con 0x7f86740cd1c0 2023-05-20T01:17:22.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.666+0000 7f867ad3e700 1 -- 172.21.15.16:0/3713579385 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86740cd1c0 msgr2=0x7f86740710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:22.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.666+0000 7f867ad3e700 1 --2- 172.21.15.16:0/3713579385 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86740cd1c0 0x7f86740710b0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f86600098f0 tx=0x7f8660009c00 comp rx=0 tx=0).stop 2023-05-20T01:17:22.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.666+0000 7f867ad3e700 1 -- 172.21.15.16:0/3713579385 shutdown_connections 2023-05-20T01:17:22.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.666+0000 7f867ad3e700 1 --2- 172.21.15.16:0/3713579385 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f86740cd630 0x7f86740715f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:22.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.666+0000 7f867ad3e700 1 --2- 172.21.15.16:0/3713579385 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86740cd1c0 0x7f86740710b0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:22.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.666+0000 7f867ad3e700 1 -- 172.21.15.16:0/3713579385 >> 172.21.15.16:0/3713579385 conn(0x7f86740cffd0 msgr2=0x7f86740d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:22.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.666+0000 7f867ad3e700 1 -- 172.21.15.16:0/3713579385 shutdown_connections 2023-05-20T01:17:22.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.666+0000 7f867ad3e700 1 -- 172.21.15.16:0/3713579385 wait complete. 2023-05-20T01:17:22.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.667+0000 7f867ad3e700 1 Processor -- start 2023-05-20T01:17:22.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.667+0000 7f867ad3e700 1 -- start start 2023-05-20T01:17:22.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.668+0000 7f867ad3e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86740cd1c0 0x7f867415da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:22.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.668+0000 7f867ad3e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f86740cd630 0x7f86740ca020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:22.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.668+0000 7f867ad3e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8674159b50 0x7f8674157520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:22.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.668+0000 7f867ad3e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86741ef030 con 0x7f86740cd1c0 2023-05-20T01:17:22.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.668+0000 7f867ad3e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86740c7240 con 0x7f86740cd630 2023-05-20T01:17:22.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.668+0000 7f867ad3e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8674156800 con 0x7f8674159b50 2023-05-20T01:17:22.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.668+0000 7f8678ada700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86740cd1c0 0x7f867415da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:22.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.668+0000 7f8678ada700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86740cd1c0 0x7f867415da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59358/0 (socket says 172.21.15.16:59358) 2023-05-20T01:17:22.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.668+0000 7f8678ada700 1 -- 172.21.15.16:0/2932772276 learned_addr learned my addr 172.21.15.16:0/2932772276 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:22.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.668+0000 7f8673fff700 1 --2- 172.21.15.16:0/2932772276 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f86740cd630 0x7f86740ca020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:22.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.668+0000 7f86792db700 1 --2- 172.21.15.16:0/2932772276 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8674159b50 0x7f8674157520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:22.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.669+0000 7f8678ada700 1 -- 172.21.15.16:0/2932772276 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8674159b50 msgr2=0x7f8674157520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:22.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.669+0000 7f8678ada700 1 --2- 172.21.15.16:0/2932772276 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8674159b50 0x7f8674157520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:22.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.669+0000 7f8678ada700 1 -- 172.21.15.16:0/2932772276 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f86740cd630 msgr2=0x7f86740ca020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:22.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.669+0000 7f8678ada700 1 --2- 172.21.15.16:0/2932772276 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f86740cd630 0x7f86740ca020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:22.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.669+0000 7f8678ada700 1 -- 172.21.15.16:0/2932772276 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8660009550 con 0x7f86740cd1c0 2023-05-20T01:17:22.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.669+0000 7f8678ada700 1 --2- 172.21.15.16:0/2932772276 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86740cd1c0 0x7f867415da60 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f866800c4d0 tx=0x7f866800c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:22.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.669+0000 7f86717fa700 1 -- 172.21.15.16:0/2932772276 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86680052d0 con 0x7f86740cd1c0 2023-05-20T01:17:22.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.669+0000 7f86717fa700 1 -- 172.21.15.16:0/2932772276 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8668005430 con 0x7f86740cd1c0 2023-05-20T01:17:22.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.669+0000 7f867ad3e700 1 -- 172.21.15.16:0/2932772276 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86740cec70 con 0x7f86740cd1c0 2023-05-20T01:17:22.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.669+0000 7f86717fa700 1 -- 172.21.15.16:0/2932772276 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86680038f0 con 0x7f86740cd1c0 2023-05-20T01:17:22.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.669+0000 7f867ad3e700 1 -- 172.21.15.16:0/2932772276 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8674071e80 con 0x7f86740cd1c0 2023-05-20T01:17:22.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.670+0000 7f867ad3e700 1 -- 172.21.15.16:0/2932772276 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86740747b0 con 0x7f86740cd1c0 2023-05-20T01:17:22.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.671+0000 7f86717fa700 1 -- 172.21.15.16:0/2932772276 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8668003a50 con 0x7f86740cd1c0 2023-05-20T01:17:22.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.671+0000 7f86717fa700 1 --2- 172.21.15.16:0/2932772276 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f865806ace0 0x7f865806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:22.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.671+0000 7f86717fa700 1 -- 172.21.15.16:0/2932772276 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f8668088a70 con 0x7f86740cd1c0 2023-05-20T01:17:22.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.671+0000 7f8673fff700 1 --2- 172.21.15.16:0/2932772276 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f865806ace0 0x7f865806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:22.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.672+0000 7f8673fff700 1 --2- 172.21.15.16:0/2932772276 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f865806ace0 0x7f865806d190 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8660009c40 tx=0x7f8660008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:22.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.676+0000 7f86717fa700 1 -- 172.21.15.16:0/2932772276 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f8668053880 con 0x7f86740cd1c0 2023-05-20T01:17:22.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.873+0000 7f867ad3e700 1 -- 172.21.15.16:0/2932772276 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f86740c9e00 con 0x7f86740cd1c0 2023-05-20T01:17:22.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.874+0000 7f86717fa700 1 -- 172.21.15.16:0/2932772276 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f8668056ea0 con 0x7f86740cd1c0 2023-05-20T01:17:22.878 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-05-20T01:17:22.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.876+0000 7f867ad3e700 1 -- 172.21.15.16:0/2932772276 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f865806ace0 msgr2=0x7f865806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:22.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.876+0000 7f867ad3e700 1 --2- 172.21.15.16:0/2932772276 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f865806ace0 0x7f865806d190 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8660009c40 tx=0x7f8660008030 comp rx=0 tx=0).stop 2023-05-20T01:17:22.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.876+0000 7f867ad3e700 1 -- 172.21.15.16:0/2932772276 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86740cd1c0 msgr2=0x7f867415da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:22.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.876+0000 7f867ad3e700 1 --2- 172.21.15.16:0/2932772276 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86740cd1c0 0x7f867415da60 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f866800c4d0 tx=0x7f866800c7e0 comp rx=0 tx=0).stop 2023-05-20T01:17:22.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.876+0000 7f867ad3e700 1 -- 172.21.15.16:0/2932772276 shutdown_connections 2023-05-20T01:17:22.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.876+0000 7f867ad3e700 1 --2- 172.21.15.16:0/2932772276 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f865806ace0 0x7f865806d190 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:22.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.876+0000 7f867ad3e700 1 --2- 172.21.15.16:0/2932772276 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8674159b50 0x7f8674157520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:22.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.876+0000 7f867ad3e700 1 --2- 172.21.15.16:0/2932772276 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86740cd1c0 0x7f867415da60 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:22.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.876+0000 7f867ad3e700 1 --2- 172.21.15.16:0/2932772276 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f86740cd630 0x7f86740ca020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:22.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.876+0000 7f867ad3e700 1 -- 172.21.15.16:0/2932772276 >> 172.21.15.16:0/2932772276 conn(0x7f86740cffd0 msgr2=0x7f86740cb6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:22.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.876+0000 7f867ad3e700 1 -- 172.21.15.16:0/2932772276 shutdown_connections 2023-05-20T01:17:22.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:22.876+0000 7f867ad3e700 1 -- 172.21.15.16:0/2932772276 wait complete. 2023-05-20T01:17:22.892 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-05-20T01:17:23.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.015+0000 7ffade4d4700 1 Processor -- start 2023-05-20T01:17:23.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.015+0000 7ffade4d4700 1 -- start start 2023-05-20T01:17:23.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.016+0000 7ffade4d4700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffad81466f0 0x7ffad806b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:23.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.016+0000 7ffade4d4700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7ffad806bb60 0x7ffad80cdab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:23.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.016+0000 7ffade4d4700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ffad80cdff0 0x7ffad8160710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:23.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.016+0000 7ffade4d4700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffad806bfd0 con 0x7ffad80cdff0 2023-05-20T01:17:23.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.016+0000 7ffade4d4700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffad80cb6a0 con 0x7ffad806bb60 2023-05-20T01:17:23.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.016+0000 7ffade4d4700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffad80ce820 con 0x7ffad81466f0 2023-05-20T01:17:23.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.016+0000 7ffadca71700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ffad80cdff0 0x7ffad8160710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:23.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.016+0000 7ffadca71700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ffad80cdff0 0x7ffad8160710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59370/0 (socket says 172.21.15.16:59370) 2023-05-20T01:17:23.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.016+0000 7ffadca71700 1 -- 172.21.15.16:0/1048221752 learned_addr learned my addr 172.21.15.16:0/1048221752 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:23.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.016+0000 7ffad7fff700 1 --2- 172.21.15.16:0/1048221752 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffad81466f0 0x7ffad806b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:23.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.016+0000 7ffad77fe700 1 --2- 172.21.15.16:0/1048221752 >> v2:172.21.15.132:3300/0 conn(0x7ffad806bb60 0x7ffad80cdab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:23.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.016+0000 7ffadca71700 1 -- 172.21.15.16:0/1048221752 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffad81466f0 msgr2=0x7ffad806b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:23.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.016+0000 7ffadca71700 1 --2- 172.21.15.16:0/1048221752 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffad81466f0 0x7ffad806b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:23.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.016+0000 7ffadca71700 1 -- 172.21.15.16:0/1048221752 >> v2:172.21.15.132:3300/0 conn(0x7ffad806bb60 msgr2=0x7ffad80cdab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:23.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.016+0000 7ffadca71700 1 --2- 172.21.15.16:0/1048221752 >> v2:172.21.15.132:3300/0 conn(0x7ffad806bb60 0x7ffad80cdab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:23.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.016+0000 7ffadca71700 1 -- 172.21.15.16:0/1048221752 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffad80cb8f0 con 0x7ffad80cdff0 2023-05-20T01:17:23.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.016+0000 7ffad7fff700 1 --2- 172.21.15.16:0/1048221752 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffad81466f0 0x7ffad806b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:17:23.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.017+0000 7ffadca71700 1 --2- 172.21.15.16:0/1048221752 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffad80cdff0 0x7ffad8160710 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7ffad80b3c40 tx=0x7ffad000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f953f63d094a3684 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:23.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.017+0000 7ffad6ffd700 1 -- 172.21.15.16:0/1048221752 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffad000e070 con 0x7ffad80cdff0 2023-05-20T01:17:23.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.017+0000 7ffad6ffd700 1 -- 172.21.15.16:0/1048221752 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffad000b9c0 con 0x7ffad80cdff0 2023-05-20T01:17:23.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.017+0000 7ffad6ffd700 1 -- 172.21.15.16:0/1048221752 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffad000bd70 con 0x7ffad80cdff0 2023-05-20T01:17:23.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.017+0000 7ffade4d4700 1 -- 172.21.15.16:0/1048221752 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffad80cdff0 msgr2=0x7ffad8160710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:23.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.017+0000 7ffade4d4700 1 --2- 172.21.15.16:0/1048221752 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffad80cdff0 0x7ffad8160710 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7ffad80b3c40 tx=0x7ffad000b330 comp rx=0 tx=0).stop 2023-05-20T01:17:23.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.017+0000 7ffade4d4700 1 -- 172.21.15.16:0/1048221752 shutdown_connections 2023-05-20T01:17:23.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.017+0000 7ffade4d4700 1 --2- 172.21.15.16:0/1048221752 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffad80cdff0 0x7ffad8160710 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:23.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.017+0000 7ffade4d4700 1 --2- 172.21.15.16:0/1048221752 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffad81466f0 0x7ffad806b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:23.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.017+0000 7ffade4d4700 1 --2- 172.21.15.16:0/1048221752 >> v2:172.21.15.132:3300/0 conn(0x7ffad806bb60 0x7ffad80cdab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:23.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.017+0000 7ffade4d4700 1 -- 172.21.15.16:0/1048221752 >> 172.21.15.16:0/1048221752 conn(0x7ffad806ae30 msgr2=0x7ffad809a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:23.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.017+0000 7ffade4d4700 1 -- 172.21.15.16:0/1048221752 shutdown_connections 2023-05-20T01:17:23.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.017+0000 7ffade4d4700 1 -- 172.21.15.16:0/1048221752 wait complete. 2023-05-20T01:17:23.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.018+0000 7ffade4d4700 1 Processor -- start 2023-05-20T01:17:23.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.019+0000 7ffade4d4700 1 -- start start 2023-05-20T01:17:23.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.019+0000 7ffade4d4700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffad806bb60 0x7ffad80d1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:23.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.019+0000 7ffade4d4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffad80cdff0 0x7ffad81f6550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:23.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.019+0000 7ffade4d4700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffad81466f0 0x7ffad80b2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:23.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.019+0000 7ffade4d4700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffad80b2180 con 0x7ffad80cdff0 2023-05-20T01:17:23.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.019+0000 7ffade4d4700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffad81f6a90 con 0x7ffad806bb60 2023-05-20T01:17:23.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.019+0000 7ffade4d4700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffad80b1570 con 0x7ffad81466f0 2023-05-20T01:17:23.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.020+0000 7ffad77fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffad80cdff0 0x7ffad81f6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:23.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.020+0000 7ffad77fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffad80cdff0 0x7ffad81f6550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59374/0 (socket says 172.21.15.16:59374) 2023-05-20T01:17:23.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.020+0000 7ffad77fe700 1 -- 172.21.15.16:0/3890769098 learned_addr learned my addr 172.21.15.16:0/3890769098 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:23.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.020+0000 7ffad7fff700 1 --2- 172.21.15.16:0/3890769098 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffad806bb60 0x7ffad80d1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:23.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.020+0000 7ffadca71700 1 --2- 172.21.15.16:0/3890769098 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffad81466f0 0x7ffad80b2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:23.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.020+0000 7ffad77fe700 1 -- 172.21.15.16:0/3890769098 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffad81466f0 msgr2=0x7ffad80b2610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:23.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.020+0000 7ffad77fe700 1 --2- 172.21.15.16:0/3890769098 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffad81466f0 0x7ffad80b2610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:23.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.020+0000 7ffad77fe700 1 -- 172.21.15.16:0/3890769098 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffad806bb60 msgr2=0x7ffad80d1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:23.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.020+0000 7ffad77fe700 1 --2- 172.21.15.16:0/3890769098 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffad806bb60 0x7ffad80d1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:23.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.020+0000 7ffad77fe700 1 -- 172.21.15.16:0/3890769098 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffad0009b30 con 0x7ffad80cdff0 2023-05-20T01:17:23.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.020+0000 7ffad7fff700 1 --2- 172.21.15.16:0/3890769098 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffad806bb60 0x7ffad80d1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:17:23.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.020+0000 7ffad77fe700 1 --2- 172.21.15.16:0/3890769098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffad80cdff0 0x7ffad81f6550 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7ffac000d830 tx=0x7ffac000dbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:23.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.020+0000 7ffad4ff9700 1 -- 172.21.15.16:0/3890769098 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac00125f0 con 0x7ffad80cdff0 2023-05-20T01:17:23.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.020+0000 7ffad4ff9700 1 -- 172.21.15.16:0/3890769098 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffac0012750 con 0x7ffad80cdff0 2023-05-20T01:17:23.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.020+0000 7ffad4ff9700 1 -- 172.21.15.16:0/3890769098 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac0012ae0 con 0x7ffad80cdff0 2023-05-20T01:17:23.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.020+0000 7ffade4d4700 1 -- 172.21.15.16:0/3890769098 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffad80cbeb0 con 0x7ffad80cdff0 2023-05-20T01:17:23.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.020+0000 7ffade4d4700 1 -- 172.21.15.16:0/3890769098 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffad80cbbd0 con 0x7ffad80cdff0 2023-05-20T01:17:23.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.021+0000 7ffade4d4700 1 -- 172.21.15.16:0/3890769098 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffad80ce470 con 0x7ffad80cdff0 2023-05-20T01:17:23.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.022+0000 7ffad4ff9700 1 -- 172.21.15.16:0/3890769098 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ffac0012c40 con 0x7ffad80cdff0 2023-05-20T01:17:23.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.022+0000 7ffad4ff9700 1 --2- 172.21.15.16:0/3890769098 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ffab806ace0 0x7ffab806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:23.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.022+0000 7ffad4ff9700 1 -- 172.21.15.16:0/3890769098 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7ffac0089b70 con 0x7ffad80cdff0 2023-05-20T01:17:23.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.022+0000 7ffad7fff700 1 --2- 172.21.15.16:0/3890769098 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ffab806ace0 0x7ffab806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:23.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.023+0000 7ffad7fff700 1 --2- 172.21.15.16:0/3890769098 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ffab806ace0 0x7ffab806d190 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ffac8009960 tx=0x7ffac8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:23.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.027+0000 7ffad4ff9700 1 -- 172.21.15.16:0/3890769098 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7ffac00548f0 con 0x7ffad80cdff0 2023-05-20T01:17:23.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.239+0000 7ffade4d4700 1 -- 172.21.15.16:0/3890769098 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ffad80b2460 con 0x7ffad80cdff0 2023-05-20T01:17:23.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.239+0000 7ffad4ff9700 1 -- 172.21.15.16:0/3890769098 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) v1 ==== 80+0+0 (secure 0 0 0) 0x7ffac0057f10 con 0x7ffad80cdff0 2023-05-20T01:17:23.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.241+0000 7ffade4d4700 1 -- 172.21.15.16:0/3890769098 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ffab806ace0 msgr2=0x7ffab806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:23.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.241+0000 7ffade4d4700 1 --2- 172.21.15.16:0/3890769098 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ffab806ace0 0x7ffab806d190 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ffac8009960 tx=0x7ffac8008040 comp rx=0 tx=0).stop 2023-05-20T01:17:23.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.241+0000 7ffade4d4700 1 -- 172.21.15.16:0/3890769098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffad80cdff0 msgr2=0x7ffad81f6550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:23.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.241+0000 7ffade4d4700 1 --2- 172.21.15.16:0/3890769098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffad80cdff0 0x7ffad81f6550 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7ffac000d830 tx=0x7ffac000dbf0 comp rx=0 tx=0).stop 2023-05-20T01:17:23.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.241+0000 7ffade4d4700 1 -- 172.21.15.16:0/3890769098 shutdown_connections 2023-05-20T01:17:23.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.241+0000 7ffade4d4700 1 --2- 172.21.15.16:0/3890769098 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ffab806ace0 0x7ffab806d190 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:23.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.241+0000 7ffade4d4700 1 --2- 172.21.15.16:0/3890769098 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffad81466f0 0x7ffad80b2610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:23.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.241+0000 7ffade4d4700 1 --2- 172.21.15.16:0/3890769098 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffad806bb60 0x7ffad80d1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:23.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.241+0000 7ffade4d4700 1 --2- 172.21.15.16:0/3890769098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffad80cdff0 0x7ffad81f6550 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:23.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.241+0000 7ffade4d4700 1 -- 172.21.15.16:0/3890769098 >> 172.21.15.16:0/3890769098 conn(0x7ffad806ae30 msgr2=0x7ffad806fc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:23.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.241+0000 7ffade4d4700 1 -- 172.21.15.16:0/3890769098 shutdown_connections 2023-05-20T01:17:23.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.241+0000 7ffade4d4700 1 -- 172.21.15.16:0/3890769098 wait complete. 2023-05-20T01:17:23.247 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2023-05-20T01:17:23.257 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-05-20T01:17:23.257 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:17:23.258 DEBUG:teuthology.orchestra.run.smithi132:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2023-05-20T01:17:23.303 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi132.front.sepia.ceph.com 2023-05-20T01:17:23.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:17:23.304 DEBUG:teuthology.orchestra.run.smithi132:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-05-20T01:17:23.330 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-05-20T01:17:23.330 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2023-05-20T01:17:23.349 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-05-20T01:17:23.349 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2023-05-20T01:17:23.406 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-05-20T01:17:23.406 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2023-05-20T01:17:23.464 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-20T01:17:23.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.626+0000 7f58ba1f0700 1 Processor -- start 2023-05-20T01:17:23.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.626+0000 7f58ba1f0700 1 -- start start 2023-05-20T01:17:23.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.626+0000 7f58ba1f0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f58b415cd40 0x7f58b40d3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:23.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.626+0000 7f58ba1f0700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58b40b60a0 con 0x7f58b40b4850 2023-05-20T01:17:23.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.626+0000 7f58ba1f0700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58b40b08d0 con 0x7f58b40d1530 2023-05-20T01:17:23.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.626+0000 7f58ba1f0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58b40affa0 con 0x7f58b415cd40 2023-05-20T01:17:23.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.627+0000 7f58b2ffd700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f58b40b4850 0x7f58b40d0e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:42678/0 (socket says 172.21.15.16:42678) 2023-05-20T01:17:23.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.627+0000 7f58b2ffd700 1 -- 172.21.15.16:0/3098521412 learned_addr learned my addr 172.21.15.16:0/3098521412 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:17:23.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.627+0000 7f58b37fe700 1 --2- 172.21.15.16:0/3098521412 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f58b415cd40 0x7f58b40d3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:23.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.627+0000 7f58b27fc700 1 -- 172.21.15.16:0/3098521412 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2354608845 0 0) 0x7f58b40b60a0 con 0x7f58b40b4850 2023-05-20T01:17:23.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.627+0000 7f58b27fc700 1 -- 172.21.15.16:0/3098521412 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f58a00035b0 con 0x7f58b40b4850 2023-05-20T01:17:23.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.627+0000 7f58b27fc700 1 -- 172.21.15.16:0/3098521412 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 508800449 0 0) 0x7f58b40b08d0 con 0x7f58b40d1530 2023-05-20T01:17:23.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.627+0000 7f58b27fc700 1 -- 172.21.15.16:0/3098521412 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f58b40b60a0 con 0x7f58b40d1530 2023-05-20T01:17:23.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.627+0000 7f58b27fc700 1 -- 172.21.15.16:0/3098521412 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1734073042 0 0) 0x7f58a00035b0 con 0x7f58b40b4850 2023-05-20T01:17:23.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.627+0000 7f58b27fc700 1 -- 172.21.15.16:0/3098521412 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f58b415cd40 msgr2=0x7f58b40d3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:23.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.627+0000 7f58b27fc700 1 --2- 172.21.15.16:0/3098521412 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f58b415cd40 0x7f58b40d3450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:23.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.627+0000 7f58b27fc700 1 -- 172.21.15.16:0/3098521412 >> v1:172.21.15.132:6789/0 conn(0x7f58b40d1530 legacy=0x7f58b40b0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:23.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.627+0000 7f58b27fc700 1 -- 172.21.15.16:0/3098521412 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58b40b0280 con 0x7f58b40b4850 2023-05-20T01:17:23.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.627+0000 7f58b27fc700 1 -- 172.21.15.16:0/3098521412 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7f58a4001670 con 0x7f58b40b4850 2023-05-20T01:17:23.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.627+0000 7f58b27fc700 1 -- 172.21.15.16:0/3098521412 >> v1:172.21.15.16:6789/0 conn(0x7f58b40b4850 legacy=0x7f58b40d0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:23.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.627+0000 7f58b27fc700 1 --2- 172.21.15.16:0/3098521412 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58a0003c30 0x7f58a0004020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:23.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.627+0000 7f58b27fc700 1 --2- 172.21.15.16:0/3098521412 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f58b415cd40 0x7f58a0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:23.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.627+0000 7f58b27fc700 1 --2- 172.21.15.16:0/3098521412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58a0004d90 0x7f58a0005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:23.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.627+0000 7f58b27fc700 1 -- 172.21.15.16:0/3098521412 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58b40affa0 con 0x7f58a0004d90 2023-05-20T01:17:23.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.627+0000 7f58b27fc700 1 -- 172.21.15.16:0/3098521412 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58a000baf0 con 0x7f58a0003c30 2023-05-20T01:17:23.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.627+0000 7f58b27fc700 1 -- 172.21.15.16:0/3098521412 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58a000bcb0 con 0x7f58b415cd40 2023-05-20T01:17:23.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.627+0000 7f58b3fff700 1 --2- 172.21.15.16:0/3098521412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58a0004d90 0x7f58a0005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:23.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.627+0000 7f58b37fe700 1 --2- 172.21.15.16:0/3098521412 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58a0003c30 0x7f58a0004020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:23.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.627+0000 7f58b2ffd700 1 --2- 172.21.15.16:0/3098521412 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f58b415cd40 0x7f58a0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:23.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.628+0000 7f58b3fff700 1 -- 172.21.15.16:0/3098521412 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f58b415cd40 msgr2=0x7f58a0004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:23.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.628+0000 7f58b3fff700 1 --2- 172.21.15.16:0/3098521412 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f58b415cd40 0x7f58a0004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:23.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.628+0000 7f58b3fff700 1 -- 172.21.15.16:0/3098521412 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58a0003c30 msgr2=0x7f58a0004020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:23.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.628+0000 7f58b3fff700 1 --2- 172.21.15.16:0/3098521412 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58a0003c30 0x7f58a0004020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:23.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.628+0000 7f58b3fff700 1 -- 172.21.15.16:0/3098521412 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58a00035b0 con 0x7f58a0004d90 2023-05-20T01:17:23.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.628+0000 7f58b3fff700 1 --2- 172.21.15.16:0/3098521412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58a0004d90 0x7f58a0005200 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f58b40b22b0 tx=0x7f58a0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3df3ec923a921cb2 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:23.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.628+0000 7f58b27fc700 1 -- 172.21.15.16:0/3098521412 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ac001f30 con 0x7f58a0004d90 2023-05-20T01:17:23.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.628+0000 7f58b27fc700 1 -- 172.21.15.16:0/3098521412 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58ac00ce30 con 0x7f58a0004d90 2023-05-20T01:17:23.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.628+0000 7f58b27fc700 1 -- 172.21.15.16:0/3098521412 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ac003c40 con 0x7f58a0004d90 2023-05-20T01:17:23.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.628+0000 7f58ba1f0700 1 -- 172.21.15.16:0/3098521412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58a0004d90 msgr2=0x7f58a0005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:23.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.628+0000 7f58ba1f0700 1 --2- 172.21.15.16:0/3098521412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58a0004d90 0x7f58a0005200 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f58b40b22b0 tx=0x7f58a0005880 comp rx=0 tx=0).stop 2023-05-20T01:17:23.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.629+0000 7f58b3fff700 1 -- 172.21.15.16:0/3098521412 reap_dead start 2023-05-20T01:17:23.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.629+0000 7f58ba1f0700 1 -- 172.21.15.16:0/3098521412 shutdown_connections 2023-05-20T01:17:23.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.629+0000 7f58ba1f0700 1 -- 172.21.15.16:0/3098521412 >> 172.21.15.16:0/3098521412 conn(0x7f58b4072560 msgr2=0x7f58b409c5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:23.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.629+0000 7f58ba1f0700 1 -- 172.21.15.16:0/3098521412 shutdown_connections 2023-05-20T01:17:23.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.629+0000 7f58ba1f0700 1 -- 172.21.15.16:0/3098521412 wait complete. 2023-05-20T01:17:23.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.630+0000 7f58ba1f0700 1 Processor -- start 2023-05-20T01:17:23.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.630+0000 7f58ba1f0700 1 -- start start 2023-05-20T01:17:23.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.631+0000 7f58ba1f0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f58b40b7f00 0x7f58b40b6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:23.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.631+0000 7f58ba1f0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58b41f27b0 0x7f58b41f2c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:23.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.631+0000 7f58ba1f0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58b40c8180 0x7f58b40c85f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:23.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.631+0000 7f58ba1f0700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58b41f37c0 con 0x7f58b40c8180 2023-05-20T01:17:23.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.631+0000 7f58ba1f0700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58b41f34e0 con 0x7f58b41f27b0 2023-05-20T01:17:23.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.631+0000 7f58ba1f0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58b40b63d0 con 0x7f58b40b7f00 2023-05-20T01:17:23.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.632+0000 7f58b3fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58b40c8180 0x7f58b40c85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:23.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.632+0000 7f58b3fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58b40c8180 0x7f58b40c85f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59396/0 (socket says 172.21.15.16:59396) 2023-05-20T01:17:23.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.632+0000 7f58b3fff700 1 -- 172.21.15.16:0/2878673809 learned_addr learned my addr 172.21.15.16:0/2878673809 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:23.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.632+0000 7f58b2ffd700 1 --2- 172.21.15.16:0/2878673809 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58b41f27b0 0x7f58b41f2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:23.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.632+0000 7f58b37fe700 1 --2- 172.21.15.16:0/2878673809 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f58b40b7f00 0x7f58b40b6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:23.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.632+0000 7f58b3fff700 1 -- 172.21.15.16:0/2878673809 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f58b40b7f00 msgr2=0x7f58b40b6eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:23.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.632+0000 7f58b3fff700 1 --2- 172.21.15.16:0/2878673809 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f58b40b7f00 0x7f58b40b6eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:23.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.632+0000 7f58b3fff700 1 -- 172.21.15.16:0/2878673809 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58b41f27b0 msgr2=0x7f58b41f2c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:23.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.632+0000 7f58b3fff700 1 --2- 172.21.15.16:0/2878673809 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58b41f27b0 0x7f58b41f2c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:23.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.632+0000 7f58b3fff700 1 -- 172.21.15.16:0/2878673809 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58b40d3450 con 0x7f58b40c8180 2023-05-20T01:17:23.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.632+0000 7f58b3fff700 1 --2- 172.21.15.16:0/2878673809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58b40c8180 0x7f58b40c85f0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f58ac005040 tx=0x7f58ac001650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:23.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.632+0000 7f58abfff700 1 -- 172.21.15.16:0/2878673809 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ac0050d0 con 0x7f58b40c8180 2023-05-20T01:17:23.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.632+0000 7f58abfff700 1 -- 172.21.15.16:0/2878673809 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58ac005250 con 0x7f58b40c8180 2023-05-20T01:17:23.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.632+0000 7f58ba1f0700 1 -- 172.21.15.16:0/2878673809 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58b40b0560 con 0x7f58b40c8180 2023-05-20T01:17:23.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.632+0000 7f58abfff700 1 -- 172.21.15.16:0/2878673809 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ac005600 con 0x7f58b40c8180 2023-05-20T01:17:23.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.632+0000 7f58ba1f0700 1 -- 172.21.15.16:0/2878673809 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58b40d35e0 con 0x7f58b40c8180 2023-05-20T01:17:23.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.633+0000 7f58ba1f0700 1 -- 172.21.15.16:0/2878673809 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58b40b6230 con 0x7f58b40c8180 2023-05-20T01:17:23.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.634+0000 7f58abfff700 1 -- 172.21.15.16:0/2878673809 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f58ac019040 con 0x7f58b40c8180 2023-05-20T01:17:23.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.634+0000 7f58abfff700 1 --2- 172.21.15.16:0/2878673809 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f589406ace0 0x7f589406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:23.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.634+0000 7f58abfff700 1 -- 172.21.15.16:0/2878673809 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f58ac089f30 con 0x7f58b40c8180 2023-05-20T01:17:23.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.634+0000 7f58b37fe700 1 --2- 172.21.15.16:0/2878673809 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f589406ace0 0x7f589406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:23.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.635+0000 7f58b37fe700 1 --2- 172.21.15.16:0/2878673809 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f589406ace0 0x7f589406d190 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f58b40b7720 tx=0x7f589c009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:23.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.638+0000 7f58abfff700 1 -- 172.21.15.16:0/2878673809 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f58b40b6230 con 0x7f58b40c8180 2023-05-20T01:17:23.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.838+0000 7f58ba1f0700 1 -- 172.21.15.16:0/2878673809 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f58b40b6230 con 0x7f58b40c8180 2023-05-20T01:17:23.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.839+0000 7f58abfff700 1 -- 172.21.15.16:0/2878673809 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+20121 (secure 0 0 0) 0x7f58b40b6230 con 0x7f58b40c8180 2023-05-20T01:17:23.843 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:17:23.844 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":27,"fsid":"c2245cc5-ecab-4087-a59b-bc81db901676","created":"2023-05-20T01:16:09.549230+0000","modified":"2023-05-20T01:17:16.406712+0000","last_up_change":"2023-05-20T01:16:19.788624+0000","last_in_change":"2023-05-20T01:16:12.092160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-05-20T01:16:20.934366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-05-20T01:16:21.372294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-05-20T01:16:26.208287+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-05-20T01:16:27.292268+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-05-20T01:16:30.357285+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"317a83cd-19c8-4958-a301-6087ef342289","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6825","nonce":3923634179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6827","nonce":3923634179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6831","nonce":3923634179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6829","nonce":3923634179}]},"public_addr":"172.21.15.16:6825/3923634179","cluster_addr":"172.21.15.16:6827/3923634179","heartbeat_back_addr":"172.21.15.16:6831/3923634179","heartbeat_front_addr":"172.21.15.16:6829/3923634179","state":["exists","up"]},{"osd":1,"uuid":"5581046f-8219-4089-8d7d-b7846399505e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6809","nonce":3860264220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6811","nonce":3860264220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6815","nonce":3860264220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6813","nonce":3860264220}]},"public_addr":"172.21.15.16:6809/3860264220","cluster_addr":"172.21.15.16:6811/3860264220","heartbeat_back_addr":"172.21.15.16:6815/3860264220","heartbeat_front_addr":"172.21.15.16:6813/3860264220","state":["exists","up"]},{"osd":2,"uuid":"072e9e66-4133-478e-8d4a-64d46c2963f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6817","nonce":293187030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6819","nonce":293187030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6823","nonce":293187030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6821","nonce":293187030}]},"public_addr":"172.21.15.16:6817/293187030","cluster_addr":"172.21.15.16:6819/293187030","heartbeat_back_addr":"172.21.15.16:6823/293187030","heartbeat_front_addr":"172.21.15.16:6821/293187030","state":["exists","up"]},{"osd":3,"uuid":"11e1bba6-00f8-48dd-9028-d8d716b7f145","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6801","nonce":3669781890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6803","nonce":3669781890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6807","nonce":3669781890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6805","nonce":3669781890}]},"public_addr":"172.21.15.16:6801/3669781890","cluster_addr":"172.21.15.16:6803/3669781890","heartbeat_back_addr":"172.21.15.16:6807/3669781890","heartbeat_front_addr":"172.21.15.16:6805/3669781890","state":["exists","up"]},{"osd":4,"uuid":"b8909c3a-cf70-46cb-8d0b-009c707cf9d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6819","nonce":506317223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6823","nonce":506317223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6829","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6831","nonce":506317223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6827","nonce":506317223}]},"public_addr":"172.21.15.132:6819/506317223","cluster_addr":"172.21.15.132:6823/506317223","heartbeat_back_addr":"172.21.15.132:6831/506317223","heartbeat_front_addr":"172.21.15.132:6827/506317223","state":["exists","up"]},{"osd":5,"uuid":"b1818304-4583-47bf-a6ec-0bfbb5166233","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6818","nonce":1844689995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6822","nonce":1844689995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6830","nonce":1844689995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6826","nonce":1844689995}]},"public_addr":"172.21.15.132:6818/1844689995","cluster_addr":"172.21.15.132:6822/1844689995","heartbeat_back_addr":"172.21.15.132:6830/1844689995","heartbeat_front_addr":"172.21.15.132:6826/1844689995","state":["exists","up"]},{"osd":6,"uuid":"d37f4691-eed1-4a93-8818-18354f03f4a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6801","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6802","nonce":727581123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6805","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6807","nonce":727581123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6813","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6815","nonce":727581123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6811","nonce":727581123}]},"public_addr":"172.21.15.132:6802/727581123","cluster_addr":"172.21.15.132:6807/727581123","heartbeat_back_addr":"172.21.15.132:6815/727581123","heartbeat_front_addr":"172.21.15.132:6811/727581123","state":["exists","up"]},{"osd":7,"uuid":"4f712f62-5d1d-4816-8042-03209144f8b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6803","nonce":466001149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6806","nonce":466001149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6814","nonce":466001149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6810","nonce":466001149}]},"public_addr":"172.21.15.132:6803/466001149","cluster_addr":"172.21.15.132:6806/466001149","heartbeat_back_addr":"172.21.15.132:6814/466001149","heartbeat_front_addr":"172.21.15.132:6810/466001149","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.338863+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.278680+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305694+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.246835+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.276237+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.304441+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305421+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.267050+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.a","mappings":[{"from":1,"to":5}]},{"pgid":"4.11","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-20T01:17:23.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.841+0000 7f58ba1f0700 1 -- 172.21.15.16:0/2878673809 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f589406ace0 msgr2=0x7f589406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:23.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.841+0000 7f58ba1f0700 1 --2- 172.21.15.16:0/2878673809 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f589406ace0 0x7f589406d190 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f58b40b7720 tx=0x7f589c009210 comp rx=0 tx=0).stop 2023-05-20T01:17:23.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.841+0000 7f58ba1f0700 1 -- 172.21.15.16:0/2878673809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58b40c8180 msgr2=0x7f58b40c85f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:23.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.841+0000 7f58ba1f0700 1 --2- 172.21.15.16:0/2878673809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58b40c8180 0x7f58b40c85f0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f58ac005040 tx=0x7f58ac001650 comp rx=0 tx=0).stop 2023-05-20T01:17:23.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.841+0000 7f58ba1f0700 1 -- 172.21.15.16:0/2878673809 shutdown_connections 2023-05-20T01:17:23.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.841+0000 7f58ba1f0700 1 --2- 172.21.15.16:0/2878673809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58b40c8180 0x7f58b40c85f0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:23.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.841+0000 7f58ba1f0700 1 --2- 172.21.15.16:0/2878673809 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f589406ace0 0x7f589406d190 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:23.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.842+0000 7f58ba1f0700 1 --2- 172.21.15.16:0/2878673809 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f58b40b7f00 0x7f58b40b6eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:23.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.842+0000 7f58ba1f0700 1 --2- 172.21.15.16:0/2878673809 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58b41f27b0 0x7f58b41f2c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:23.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.842+0000 7f58ba1f0700 1 -- 172.21.15.16:0/2878673809 >> 172.21.15.16:0/2878673809 conn(0x7f58b4072560 msgr2=0x7f58b406c920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:23.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.842+0000 7f58ba1f0700 1 -- 172.21.15.16:0/2878673809 shutdown_connections 2023-05-20T01:17:23.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.842+0000 7f58ba1f0700 1 -- 172.21.15.16:0/2878673809 wait complete. 2023-05-20T01:17:23.858 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-05-20T01:16:20.934366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-05-20T01:16:21.372294+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-05-20T01:16:26.208287+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-05-20T01:16:27.292268+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-05-20T01:16:30.357285+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-05-20T01:17:23.859 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-05-20T01:17:23.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.985+0000 7eff35f86700 1 Processor -- start 2023-05-20T01:17:23.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.985+0000 7eff35f86700 1 -- start start 2023-05-20T01:17:23.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.986+0000 7eff35f86700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7eff3015ca90 0x7eff300cd890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:23.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.986+0000 7eff35f86700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff300d1820 con 0x7eff300d2550 2023-05-20T01:17:23.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.986+0000 7eff35f86700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff300cdf00 con 0x7eff300d29a0 2023-05-20T01:17:23.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.986+0000 7eff35f86700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff300d0c50 con 0x7eff3015ca90 2023-05-20T01:17:23.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.986+0000 7eff2effd700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7eff300d2550 0x7eff300b6090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:42698/0 (socket says 172.21.15.16:42698) 2023-05-20T01:17:23.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.986+0000 7eff2effd700 1 -- 172.21.15.16:0/1943873696 learned_addr learned my addr 172.21.15.16:0/1943873696 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:17:23.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.986+0000 7eff2f7fe700 1 --2- 172.21.15.16:0/1943873696 >> v2:172.21.15.132:3300/0 conn(0x7eff3015ca90 0x7eff300cd890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:23.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.986+0000 7eff2e7fc700 1 -- 172.21.15.16:0/1943873696 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2711695994 0 0) 0x7eff300d1820 con 0x7eff300d2550 2023-05-20T01:17:23.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.986+0000 7eff2e7fc700 1 -- 172.21.15.16:0/1943873696 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7eff1c0035b0 con 0x7eff300d2550 2023-05-20T01:17:23.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.986+0000 7eff2f7fe700 1 -- 172.21.15.16:0/1943873696 >> v1:172.21.15.132:6789/0 conn(0x7eff300d29a0 legacy=0x7eff300cf570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:23.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.986+0000 7eff2f7fe700 1 -- 172.21.15.16:0/1943873696 >> v1:172.21.15.16:6789/0 conn(0x7eff300d2550 legacy=0x7eff300b6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:23.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.986+0000 7eff2f7fe700 1 -- 172.21.15.16:0/1943873696 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff300d1a70 con 0x7eff3015ca90 2023-05-20T01:17:23.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.987+0000 7eff2f7fe700 1 --2- 172.21.15.16:0/1943873696 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7eff3015ca90 0x7eff300cd890 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7eff20009d90 tx=0x7eff2000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fdcb37715a20f1c6 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:23.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.987+0000 7eff2e7fc700 1 -- 172.21.15.16:0/1943873696 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff2000e070 con 0x7eff3015ca90 2023-05-20T01:17:23.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.987+0000 7eff2e7fc700 1 -- 172.21.15.16:0/1943873696 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff2000bb20 con 0x7eff3015ca90 2023-05-20T01:17:23.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.987+0000 7eff2e7fc700 1 -- 172.21.15.16:0/1943873696 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff20012430 con 0x7eff3015ca90 2023-05-20T01:17:23.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.987+0000 7eff35f86700 1 -- 172.21.15.16:0/1943873696 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7eff3015ca90 msgr2=0x7eff300cd890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:23.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.987+0000 7eff35f86700 1 --2- 172.21.15.16:0/1943873696 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7eff3015ca90 0x7eff300cd890 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7eff20009d90 tx=0x7eff2000b330 comp rx=0 tx=0).stop 2023-05-20T01:17:23.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.987+0000 7eff35f86700 1 -- 172.21.15.16:0/1943873696 shutdown_connections 2023-05-20T01:17:23.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.987+0000 7eff35f86700 1 --2- 172.21.15.16:0/1943873696 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7eff3015ca90 0x7eff300cd890 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:23.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.987+0000 7eff35f86700 1 -- 172.21.15.16:0/1943873696 >> 172.21.15.16:0/1943873696 conn(0x7eff3006b000 msgr2=0x7eff3006b400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:23.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.987+0000 7eff35f86700 1 -- 172.21.15.16:0/1943873696 shutdown_connections 2023-05-20T01:17:23.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.987+0000 7eff35f86700 1 -- 172.21.15.16:0/1943873696 wait complete. 2023-05-20T01:17:23.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.989+0000 7eff35f86700 1 Processor -- start 2023-05-20T01:17:23.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.989+0000 7eff35f86700 1 -- start start 2023-05-20T01:17:23.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.989+0000 7eff35f86700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7eff300d2550 0x7eff300c8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:23.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.989+0000 7eff35f86700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7eff300d29a0 0x7eff300cc170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:23.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.989+0000 7eff35f86700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff3015ca90 0x7eff30152d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:23.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.989+0000 7eff35f86700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff300cc6b0 con 0x7eff3015ca90 2023-05-20T01:17:23.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.989+0000 7eff35f86700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff300c8770 con 0x7eff300d2550 2023-05-20T01:17:23.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.989+0000 7eff35f86700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff30153260 con 0x7eff300d29a0 2023-05-20T01:17:23.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.990+0000 7eff2ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff3015ca90 0x7eff30152d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:23.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.990+0000 7eff2ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff3015ca90 0x7eff30152d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59410/0 (socket says 172.21.15.16:59410) 2023-05-20T01:17:23.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.990+0000 7eff2ffff700 1 -- 172.21.15.16:0/3949003694 learned_addr learned my addr 172.21.15.16:0/3949003694 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:23.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.990+0000 7eff2f7fe700 1 --2- 172.21.15.16:0/3949003694 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7eff300d2550 0x7eff300c8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:23.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.990+0000 7eff2effd700 1 --2- 172.21.15.16:0/3949003694 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7eff300d29a0 0x7eff300cc170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:23.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.990+0000 7eff2ffff700 1 -- 172.21.15.16:0/3949003694 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7eff300d29a0 msgr2=0x7eff300cc170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:23.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.990+0000 7eff2ffff700 1 --2- 172.21.15.16:0/3949003694 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7eff300d29a0 0x7eff300cc170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:23.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.990+0000 7eff2ffff700 1 -- 172.21.15.16:0/3949003694 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7eff300d2550 msgr2=0x7eff300c8c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:23.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.990+0000 7eff2ffff700 1 --2- 172.21.15.16:0/3949003694 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7eff300d2550 0x7eff300c8c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:23.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.990+0000 7eff2ffff700 1 -- 172.21.15.16:0/3949003694 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff20009a70 con 0x7eff3015ca90 2023-05-20T01:17:23.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.990+0000 7eff2f7fe700 1 --2- 172.21.15.16:0/3949003694 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7eff300d2550 0x7eff300c8c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:17:23.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.990+0000 7eff2ffff700 1 --2- 172.21.15.16:0/3949003694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff3015ca90 0x7eff30152d20 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7eff2800c530 tx=0x7eff2800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:23.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.990+0000 7eff27fff700 1 -- 172.21.15.16:0/3949003694 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff280052d0 con 0x7eff3015ca90 2023-05-20T01:17:23.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.990+0000 7eff27fff700 1 -- 172.21.15.16:0/3949003694 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff28005430 con 0x7eff3015ca90 2023-05-20T01:17:23.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.990+0000 7eff35f86700 1 -- 172.21.15.16:0/3949003694 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff300d2030 con 0x7eff3015ca90 2023-05-20T01:17:23.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.990+0000 7eff27fff700 1 -- 172.21.15.16:0/3949003694 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff280038c0 con 0x7eff3015ca90 2023-05-20T01:17:23.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.990+0000 7eff35f86700 1 -- 172.21.15.16:0/3949003694 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff300d1d50 con 0x7eff3015ca90 2023-05-20T01:17:23.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.991+0000 7eff35f86700 1 -- 172.21.15.16:0/3949003694 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff3015ce60 con 0x7eff3015ca90 2023-05-20T01:17:24.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.992+0000 7eff27fff700 1 -- 172.21.15.16:0/3949003694 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7eff28003a20 con 0x7eff3015ca90 2023-05-20T01:17:24.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.992+0000 7eff27fff700 1 --2- 172.21.15.16:0/3949003694 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7eff1006ace0 0x7eff1006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:24.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.993+0000 7eff2f7fe700 1 --2- 172.21.15.16:0/3949003694 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7eff1006ace0 0x7eff1006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:24.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.993+0000 7eff27fff700 1 -- 172.21.15.16:0/3949003694 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7eff28088ce0 con 0x7eff3015ca90 2023-05-20T01:17:24.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.993+0000 7eff2f7fe700 1 --2- 172.21.15.16:0/3949003694 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7eff1006ace0 0x7eff1006d190 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7eff20000f80 tx=0x7eff20011000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:24.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:23.997+0000 7eff27fff700 1 -- 172.21.15.16:0/3949003694 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7eff280539c0 con 0x7eff3015ca90 2023-05-20T01:17:24.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.192+0000 7eff35f86700 1 -- 172.21.15.16:0/3949003694 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7eff3004e8c0 con 0x7eff3015ca90 2023-05-20T01:17:24.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.192+0000 7eff27fff700 1 -- 172.21.15.16:0/3949003694 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v27) v1 ==== 92+0+10 (secure 0 0 0) 0x7eff28017020 con 0x7eff3015ca90 2023-05-20T01:17:24.196 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 8 2023-05-20T01:17:24.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.194+0000 7eff35f86700 1 -- 172.21.15.16:0/3949003694 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7eff1006ace0 msgr2=0x7eff1006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:24.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.194+0000 7eff35f86700 1 --2- 172.21.15.16:0/3949003694 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7eff1006ace0 0x7eff1006d190 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7eff20000f80 tx=0x7eff20011000 comp rx=0 tx=0).stop 2023-05-20T01:17:24.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.194+0000 7eff35f86700 1 -- 172.21.15.16:0/3949003694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff3015ca90 msgr2=0x7eff30152d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:24.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.194+0000 7eff35f86700 1 --2- 172.21.15.16:0/3949003694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff3015ca90 0x7eff30152d20 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7eff2800c530 tx=0x7eff2800c8f0 comp rx=0 tx=0).stop 2023-05-20T01:17:24.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.194+0000 7eff35f86700 1 -- 172.21.15.16:0/3949003694 shutdown_connections 2023-05-20T01:17:24.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.194+0000 7eff35f86700 1 --2- 172.21.15.16:0/3949003694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff3015ca90 0x7eff30152d20 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:24.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.194+0000 7eff35f86700 1 --2- 172.21.15.16:0/3949003694 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7eff300d2550 0x7eff300c8c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:24.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.194+0000 7eff35f86700 1 --2- 172.21.15.16:0/3949003694 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7eff1006ace0 0x7eff1006d190 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:24.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.194+0000 7eff35f86700 1 --2- 172.21.15.16:0/3949003694 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7eff300d29a0 0x7eff300cc170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:24.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.194+0000 7eff35f86700 1 -- 172.21.15.16:0/3949003694 >> 172.21.15.16:0/3949003694 conn(0x7eff3006b000 msgr2=0x7eff300b61b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:24.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.194+0000 7eff35f86700 1 -- 172.21.15.16:0/3949003694 shutdown_connections 2023-05-20T01:17:24.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.195+0000 7eff35f86700 1 -- 172.21.15.16:0/3949003694 wait complete. 2023-05-20T01:17:24.210 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-05-20T01:17:24.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.335+0000 7f419df19700 1 Processor -- start 2023-05-20T01:17:24.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.335+0000 7f419df19700 1 -- start start 2023-05-20T01:17:24.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.336+0000 7f419df19700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f41980cd810 0x7f41980cc4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:24.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.336+0000 7f419df19700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f419815d170 0x7f419815d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:24.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.336+0000 7f419df19700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41980cca30 con 0x7f41980cc120 2023-05-20T01:17:24.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.336+0000 7f419df19700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41980cbfe0 con 0x7f419815d170 2023-05-20T01:17:24.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.336+0000 7f419df19700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41980cd3f0 con 0x7f41980cd810 2023-05-20T01:17:24.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.336+0000 7f41977fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f41980cc120 0x7f41980717e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:42710/0 (socket says 172.21.15.16:42710) 2023-05-20T01:17:24.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.336+0000 7f41977fe700 1 -- 172.21.15.16:0/239608443 learned_addr learned my addr 172.21.15.16:0/239608443 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:17:24.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.336+0000 7f4196ffd700 1 --2- 172.21.15.16:0/239608443 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f41980cd810 0x7f41980cc4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:24.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.336+0000 7f4197fff700 1 --2- 172.21.15.16:0/239608443 >> v2:172.21.15.132:3300/0 conn(0x7f419815d170 0x7f419815d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:24.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.336+0000 7f41967fc700 1 -- 172.21.15.16:0/239608443 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 466325312 0 0) 0x7f41980cca30 con 0x7f41980cc120 2023-05-20T01:17:24.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.336+0000 7f41967fc700 1 -- 172.21.15.16:0/239608443 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41800035b0 con 0x7f41980cc120 2023-05-20T01:17:24.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.337+0000 7f4196ffd700 1 -- 172.21.15.16:0/239608443 >> v2:172.21.15.132:3300/0 conn(0x7f419815d170 msgr2=0x7f419815d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:24.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.337+0000 7f4196ffd700 1 --2- 172.21.15.16:0/239608443 >> v2:172.21.15.132:3300/0 conn(0x7f419815d170 0x7f419815d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:24.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.337+0000 7f4196ffd700 1 -- 172.21.15.16:0/239608443 >> v1:172.21.15.16:6789/0 conn(0x7f41980cc120 legacy=0x7f41980717e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:24.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.337+0000 7f4196ffd700 1 -- 172.21.15.16:0/239608443 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41980cbd00 con 0x7f41980cd810 2023-05-20T01:17:24.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.337+0000 7f4197fff700 1 --2- 172.21.15.16:0/239608443 >> v2:172.21.15.132:3300/0 conn(0x7f419815d170 0x7f419815d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-05-20T01:17:24.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.337+0000 7f4196ffd700 1 --2- 172.21.15.16:0/239608443 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f41980cd810 0x7f41980cc4f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f418c009840 tx=0x7f418c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d42a800913f8025 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:24.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.337+0000 7f41967fc700 1 -- 172.21.15.16:0/239608443 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f418c011830 con 0x7f41980cd810 2023-05-20T01:17:24.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.337+0000 7f41967fc700 1 -- 172.21.15.16:0/239608443 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f418c011990 con 0x7f41980cd810 2023-05-20T01:17:24.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.337+0000 7f41967fc700 1 -- 172.21.15.16:0/239608443 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f418c011d00 con 0x7f41980cd810 2023-05-20T01:17:24.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.338+0000 7f419df19700 1 -- 172.21.15.16:0/239608443 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f41980cd810 msgr2=0x7f41980cc4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:24.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.338+0000 7f419df19700 1 --2- 172.21.15.16:0/239608443 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f41980cd810 0x7f41980cc4f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f418c009840 tx=0x7f418c009b50 comp rx=0 tx=0).stop 2023-05-20T01:17:24.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.338+0000 7f419df19700 1 -- 172.21.15.16:0/239608443 shutdown_connections 2023-05-20T01:17:24.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.338+0000 7f419df19700 1 --2- 172.21.15.16:0/239608443 >> v2:172.21.15.132:3300/0 conn(0x7f419815d170 0x7f419815d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:24.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.338+0000 7f419df19700 1 --2- 172.21.15.16:0/239608443 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f41980cd810 0x7f41980cc4f0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:24.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.338+0000 7f419df19700 1 -- 172.21.15.16:0/239608443 >> 172.21.15.16:0/239608443 conn(0x7f419806cbb0 msgr2=0x7f419806cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:24.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.338+0000 7f419df19700 1 -- 172.21.15.16:0/239608443 shutdown_connections 2023-05-20T01:17:24.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.338+0000 7f419df19700 1 -- 172.21.15.16:0/239608443 wait complete. 2023-05-20T01:17:24.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.339+0000 7f419df19700 1 Processor -- start 2023-05-20T01:17:24.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.339+0000 7f419df19700 1 -- start start 2023-05-20T01:17:24.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.340+0000 7f419df19700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f41980cc120 0x7f41980c9220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:24.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.340+0000 7f419df19700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f41980cd810 0x7f41980c6880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:24.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.340+0000 7f419df19700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f419815d170 0x7f419814c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:24.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.340+0000 7f419df19700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f419814c850 con 0x7f419815d170 2023-05-20T01:17:24.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.340+0000 7f419df19700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41980c7b90 con 0x7f41980cc120 2023-05-20T01:17:24.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.340+0000 7f419df19700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4198158be0 con 0x7f41980cd810 2023-05-20T01:17:24.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.340+0000 7f4197fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f419815d170 0x7f419814c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:24.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.340+0000 7f4197fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f419815d170 0x7f419814c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59424/0 (socket says 172.21.15.16:59424) 2023-05-20T01:17:24.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.340+0000 7f4197fff700 1 -- 172.21.15.16:0/4237452016 learned_addr learned my addr 172.21.15.16:0/4237452016 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:24.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.340+0000 7f4196ffd700 1 --2- 172.21.15.16:0/4237452016 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f41980cd810 0x7f41980c6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:24.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.340+0000 7f41977fe700 1 --2- 172.21.15.16:0/4237452016 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f41980cc120 0x7f41980c9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:24.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.341+0000 7f4197fff700 1 -- 172.21.15.16:0/4237452016 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f41980cd810 msgr2=0x7f41980c6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:24.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.341+0000 7f4197fff700 1 --2- 172.21.15.16:0/4237452016 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f41980cd810 0x7f41980c6880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:24.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.341+0000 7f4197fff700 1 -- 172.21.15.16:0/4237452016 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f41980cc120 msgr2=0x7f41980c9220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:24.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.341+0000 7f4197fff700 1 --2- 172.21.15.16:0/4237452016 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f41980cc120 0x7f41980c9220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:24.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.341+0000 7f4197fff700 1 -- 172.21.15.16:0/4237452016 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4190009a70 con 0x7f419815d170 2023-05-20T01:17:24.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.341+0000 7f4197fff700 1 --2- 172.21.15.16:0/4237452016 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f419815d170 0x7f419814c310 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f419000efc0 tx=0x7f419000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:24.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.341+0000 7f417ffff700 1 -- 172.21.15.16:0/4237452016 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f419000ca90 con 0x7f419815d170 2023-05-20T01:17:24.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.341+0000 7f417ffff700 1 -- 172.21.15.16:0/4237452016 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f419000f040 con 0x7f419815d170 2023-05-20T01:17:24.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.341+0000 7f419df19700 1 -- 172.21.15.16:0/4237452016 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f418c0094a0 con 0x7f419815d170 2023-05-20T01:17:24.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.341+0000 7f417ffff700 1 -- 172.21.15.16:0/4237452016 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f419000ce30 con 0x7f419815d170 2023-05-20T01:17:24.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.341+0000 7f419df19700 1 -- 172.21.15.16:0/4237452016 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41980ce1f0 con 0x7f419815d170 2023-05-20T01:17:24.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.342+0000 7f419df19700 1 -- 172.21.15.16:0/4237452016 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41980cdf10 con 0x7f419815d170 2023-05-20T01:17:24.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.342+0000 7f417ffff700 1 -- 172.21.15.16:0/4237452016 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4190014480 con 0x7f419815d170 2023-05-20T01:17:24.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.343+0000 7f417ffff700 1 --2- 172.21.15.16:0/4237452016 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f418406ace0 0x7f418406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:24.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.343+0000 7f417ffff700 1 -- 172.21.15.16:0/4237452016 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f4190089df0 con 0x7f419815d170 2023-05-20T01:17:24.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.343+0000 7f41977fe700 1 --2- 172.21.15.16:0/4237452016 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f418406ace0 0x7f418406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:24.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.343+0000 7f41977fe700 1 --2- 172.21.15.16:0/4237452016 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f418406ace0 0x7f418406d190 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4188007fd0 tx=0x7f4188009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:24.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.348+0000 7f417ffff700 1 -- 172.21.15.16:0/4237452016 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f4190054b70 con 0x7f419815d170 2023-05-20T01:17:24.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.549+0000 7f419df19700 1 -- 172.21.15.16:0/4237452016 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f41980cb9b0 con 0x7f419815d170 2023-05-20T01:17:24.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.549+0000 7f417ffff700 1 -- 172.21.15.16:0/4237452016 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f41900180a0 con 0x7f419815d170 2023-05-20T01:17:24.553 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-05-20T01:17:24.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.551+0000 7f419df19700 1 -- 172.21.15.16:0/4237452016 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f418406ace0 msgr2=0x7f418406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:24.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.551+0000 7f419df19700 1 --2- 172.21.15.16:0/4237452016 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f418406ace0 0x7f418406d190 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4188007fd0 tx=0x7f4188009040 comp rx=0 tx=0).stop 2023-05-20T01:17:24.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.551+0000 7f419df19700 1 -- 172.21.15.16:0/4237452016 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f419815d170 msgr2=0x7f419814c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:24.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.551+0000 7f419df19700 1 --2- 172.21.15.16:0/4237452016 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f419815d170 0x7f419814c310 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f419000efc0 tx=0x7f419000c3a0 comp rx=0 tx=0).stop 2023-05-20T01:17:24.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.551+0000 7f419df19700 1 -- 172.21.15.16:0/4237452016 shutdown_connections 2023-05-20T01:17:24.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.551+0000 7f419df19700 1 --2- 172.21.15.16:0/4237452016 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f419815d170 0x7f419814c310 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:24.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.551+0000 7f419df19700 1 --2- 172.21.15.16:0/4237452016 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f41980cc120 0x7f41980c9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:24.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.551+0000 7f419df19700 1 --2- 172.21.15.16:0/4237452016 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f418406ace0 0x7f418406d190 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:24.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.551+0000 7f419df19700 1 --2- 172.21.15.16:0/4237452016 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f41980cd810 0x7f41980c6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:24.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.551+0000 7f419df19700 1 -- 172.21.15.16:0/4237452016 >> 172.21.15.16:0/4237452016 conn(0x7f419806cbb0 msgr2=0x7f41980cf740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:24.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.551+0000 7f419df19700 1 -- 172.21.15.16:0/4237452016 shutdown_connections 2023-05-20T01:17:24.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.551+0000 7f419df19700 1 -- 172.21.15.16:0/4237452016 wait complete. 2023-05-20T01:17:24.567 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-05-20T01:17:24.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.691+0000 7f52c9b76700 1 Processor -- start 2023-05-20T01:17:24.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.691+0000 7f52c9b76700 1 -- start start 2023-05-20T01:17:24.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.692+0000 7f52c9b76700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f52c40b57b0 0x7f52c4071c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:24.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.692+0000 7f52c9b76700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f52c40ccc50 0x7f52c4072170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:24.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.692+0000 7f52c9b76700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52c406a5b0 con 0x7f52c415f000 2023-05-20T01:17:24.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.692+0000 7f52c9b76700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52c406a850 con 0x7f52c40ccc50 2023-05-20T01:17:24.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.692+0000 7f52c9b76700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52c406ab30 con 0x7f52c40b57b0 2023-05-20T01:17:24.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.692+0000 7f52c2ffd700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f52c415f000 0x7f52c406d4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:42726/0 (socket says 172.21.15.16:42726) 2023-05-20T01:17:24.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.692+0000 7f52c2ffd700 1 -- 172.21.15.16:0/3064384441 learned_addr learned my addr 172.21.15.16:0/3064384441 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:17:24.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.692+0000 7f52c3fff700 1 --2- 172.21.15.16:0/3064384441 >> v2:172.21.15.132:3300/0 conn(0x7f52c40ccc50 0x7f52c4072170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:24.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.692+0000 7f52c37fe700 1 --2- 172.21.15.16:0/3064384441 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f52c40b57b0 0x7f52c4071c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:24.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.693+0000 7f52c27fc700 1 -- 172.21.15.16:0/3064384441 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2447622302 0 0) 0x7f52c406a5b0 con 0x7f52c415f000 2023-05-20T01:17:24.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.693+0000 7f52c27fc700 1 -- 172.21.15.16:0/3064384441 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52b00035b0 con 0x7f52c415f000 2023-05-20T01:17:24.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.693+0000 7f52c3fff700 1 -- 172.21.15.16:0/3064384441 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f52c40b57b0 msgr2=0x7f52c4071c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:24.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.693+0000 7f52c3fff700 1 --2- 172.21.15.16:0/3064384441 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f52c40b57b0 0x7f52c4071c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:24.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.693+0000 7f52c3fff700 1 -- 172.21.15.16:0/3064384441 >> v1:172.21.15.16:6789/0 conn(0x7f52c415f000 legacy=0x7f52c406d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:24.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.693+0000 7f52c3fff700 1 -- 172.21.15.16:0/3064384441 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52c406ae10 con 0x7f52c40ccc50 2023-05-20T01:17:24.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.693+0000 7f52c37fe700 1 --2- 172.21.15.16:0/3064384441 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f52c40b57b0 0x7f52c4071c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-05-20T01:17:24.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.693+0000 7f52c3fff700 1 --2- 172.21.15.16:0/3064384441 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f52c40ccc50 0x7f52c4072170 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f52bc009d90 tx=0x7f52bc00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0149ed3533f586 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:24.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.693+0000 7f52c27fc700 1 -- 172.21.15.16:0/3064384441 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52bc00e070 con 0x7f52c40ccc50 2023-05-20T01:17:24.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.693+0000 7f52c27fc700 1 -- 172.21.15.16:0/3064384441 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52bc00bb20 con 0x7f52c40ccc50 2023-05-20T01:17:24.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.693+0000 7f52c27fc700 1 -- 172.21.15.16:0/3064384441 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52bc012430 con 0x7f52c40ccc50 2023-05-20T01:17:24.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.694+0000 7f52c9b76700 1 -- 172.21.15.16:0/3064384441 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f52c40ccc50 msgr2=0x7f52c4072170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:24.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.694+0000 7f52c9b76700 1 --2- 172.21.15.16:0/3064384441 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f52c40ccc50 0x7f52c4072170 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f52bc009d90 tx=0x7f52bc00b330 comp rx=0 tx=0).stop 2023-05-20T01:17:24.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.694+0000 7f52c9b76700 1 -- 172.21.15.16:0/3064384441 shutdown_connections 2023-05-20T01:17:24.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.694+0000 7f52c9b76700 1 --2- 172.21.15.16:0/3064384441 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f52c40ccc50 0x7f52c4072170 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:24.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.694+0000 7f52c9b76700 1 --2- 172.21.15.16:0/3064384441 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f52c40b57b0 0x7f52c4071c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:24.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.694+0000 7f52c9b76700 1 -- 172.21.15.16:0/3064384441 >> 172.21.15.16:0/3064384441 conn(0x7f52c406c260 msgr2=0x7f52c40cffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:24.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.694+0000 7f52c9b76700 1 -- 172.21.15.16:0/3064384441 shutdown_connections 2023-05-20T01:17:24.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.694+0000 7f52c9b76700 1 -- 172.21.15.16:0/3064384441 wait complete. 2023-05-20T01:17:24.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.695+0000 7f52c9b76700 1 Processor -- start 2023-05-20T01:17:24.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.695+0000 7f52c9b76700 1 -- start start 2023-05-20T01:17:24.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.696+0000 7f52c9b76700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52c40b57b0 0x7f52c414e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:24.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.696+0000 7f52c9b76700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f52c40ccc50 0x7f52c41efe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:24.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.696+0000 7f52c9b76700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f52c415f000 0x7f52c40c4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:24.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.696+0000 7f52c9b76700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52c40c4eb0 con 0x7f52c40b57b0 2023-05-20T01:17:24.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.696+0000 7f52c9b76700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52c40c8660 con 0x7f52c40ccc50 2023-05-20T01:17:24.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.696+0000 7f52c9b76700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52c40c8940 con 0x7f52c415f000 2023-05-20T01:17:24.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.696+0000 7f52c37fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52c40b57b0 0x7f52c414e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:24.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.696+0000 7f52c37fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52c40b57b0 0x7f52c414e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59434/0 (socket says 172.21.15.16:59434) 2023-05-20T01:17:24.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.696+0000 7f52c37fe700 1 -- 172.21.15.16:0/300572245 learned_addr learned my addr 172.21.15.16:0/300572245 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:24.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.696+0000 7f52c2ffd700 1 --2- 172.21.15.16:0/300572245 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f52c40ccc50 0x7f52c41efe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:24.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.696+0000 7f52c3fff700 1 --2- 172.21.15.16:0/300572245 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f52c415f000 0x7f52c40c4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:24.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.696+0000 7f52c37fe700 1 -- 172.21.15.16:0/300572245 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f52c415f000 msgr2=0x7f52c40c4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:24.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.696+0000 7f52c37fe700 1 --2- 172.21.15.16:0/300572245 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f52c415f000 0x7f52c40c4430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:24.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.696+0000 7f52c37fe700 1 -- 172.21.15.16:0/300572245 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f52c40ccc50 msgr2=0x7f52c41efe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:24.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.696+0000 7f52c37fe700 1 --2- 172.21.15.16:0/300572245 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f52c40ccc50 0x7f52c41efe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:24.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.696+0000 7f52c37fe700 1 -- 172.21.15.16:0/300572245 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52ac0094a0 con 0x7f52c40b57b0 2023-05-20T01:17:24.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.696+0000 7f52c37fe700 1 --2- 172.21.15.16:0/300572245 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52c40b57b0 0x7f52c414e5e0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f52ac00ebe0 tx=0x7f52ac009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:24.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.697+0000 7f52bbfff700 1 -- 172.21.15.16:0/300572245 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52ac00c910 con 0x7f52c40b57b0 2023-05-20T01:17:24.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.697+0000 7f52bbfff700 1 -- 172.21.15.16:0/300572245 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52ac00ca70 con 0x7f52c40b57b0 2023-05-20T01:17:24.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.697+0000 7f52bbfff700 1 -- 172.21.15.16:0/300572245 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52ac00cde0 con 0x7f52c40b57b0 2023-05-20T01:17:24.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.697+0000 7f52c9b76700 1 -- 172.21.15.16:0/300572245 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52bc009a70 con 0x7f52c40b57b0 2023-05-20T01:17:24.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.697+0000 7f52c9b76700 1 -- 172.21.15.16:0/300572245 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52c40af710 con 0x7f52c40b57b0 2023-05-20T01:17:24.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.698+0000 7f52c9b76700 1 -- 172.21.15.16:0/300572245 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52c415f4e0 con 0x7f52c40b57b0 2023-05-20T01:17:24.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.698+0000 7f52bbfff700 1 -- 172.21.15.16:0/300572245 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f52ac013470 con 0x7f52c40b57b0 2023-05-20T01:17:24.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.698+0000 7f52bbfff700 1 --2- 172.21.15.16:0/300572245 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f52a406ace0 0x7f52a406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:24.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.699+0000 7f52bbfff700 1 -- 172.21.15.16:0/300572245 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f52ac089a00 con 0x7f52c40b57b0 2023-05-20T01:17:24.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.699+0000 7f52c2ffd700 1 --2- 172.21.15.16:0/300572245 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f52a406ace0 0x7f52a406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:24.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.699+0000 7f52c2ffd700 1 --2- 172.21.15.16:0/300572245 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f52a406ace0 0x7f52a406d190 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f52c41f3330 tx=0x7f52b4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:24.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.703+0000 7f52bbfff700 1 -- 172.21.15.16:0/300572245 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f52ac054780 con 0x7f52c40b57b0 2023-05-20T01:17:24.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.901+0000 7f52c9b76700 1 -- 172.21.15.16:0/300572245 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f52c40af8a0 con 0x7f52c40b57b0 2023-05-20T01:17:24.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.902+0000 7f52bbfff700 1 -- 172.21.15.16:0/300572245 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f52ac057da0 con 0x7f52c40b57b0 2023-05-20T01:17:24.906 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-05-20T01:17:24.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.904+0000 7f52c9b76700 1 -- 172.21.15.16:0/300572245 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f52a406ace0 msgr2=0x7f52a406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:24.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.904+0000 7f52c9b76700 1 --2- 172.21.15.16:0/300572245 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f52a406ace0 0x7f52a406d190 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f52c41f3330 tx=0x7f52b4009040 comp rx=0 tx=0).stop 2023-05-20T01:17:24.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.904+0000 7f52c9b76700 1 -- 172.21.15.16:0/300572245 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52c40b57b0 msgr2=0x7f52c414e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:24.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.904+0000 7f52c9b76700 1 --2- 172.21.15.16:0/300572245 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52c40b57b0 0x7f52c414e5e0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f52ac00ebe0 tx=0x7f52ac009ec0 comp rx=0 tx=0).stop 2023-05-20T01:17:24.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.904+0000 7f52c9b76700 1 -- 172.21.15.16:0/300572245 shutdown_connections 2023-05-20T01:17:24.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.904+0000 7f52c9b76700 1 --2- 172.21.15.16:0/300572245 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f52a406ace0 0x7f52a406d190 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:24.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.904+0000 7f52c9b76700 1 --2- 172.21.15.16:0/300572245 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f52c415f000 0x7f52c40c4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:24.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.904+0000 7f52c9b76700 1 --2- 172.21.15.16:0/300572245 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52c40b57b0 0x7f52c414e5e0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:24.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.904+0000 7f52c9b76700 1 --2- 172.21.15.16:0/300572245 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f52c40ccc50 0x7f52c41efe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:24.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.904+0000 7f52c9b76700 1 -- 172.21.15.16:0/300572245 >> 172.21.15.16:0/300572245 conn(0x7f52c406c260 msgr2=0x7f52c4071c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:24.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.904+0000 7f52c9b76700 1 -- 172.21.15.16:0/300572245 shutdown_connections 2023-05-20T01:17:24.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:24.904+0000 7f52c9b76700 1 -- 172.21.15.16:0/300572245 wait complete. 2023-05-20T01:17:24.921 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-05-20T01:17:25.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.046+0000 7fe76b7ec700 1 Processor -- start 2023-05-20T01:17:25.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.046+0000 7fe76b7ec700 1 -- start start 2023-05-20T01:17:25.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.046+0000 7fe76b7ec700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe764159b50 0x7fe76406e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:25.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.047+0000 7fe76b7ec700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fe7640cd1c0 0x7fe7640710b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:25.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.047+0000 7fe76b7ec700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe7640cd630 0x7fe7640715f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:25.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.047+0000 7fe76b7ec700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7640ccd30 con 0x7fe764159b50 2023-05-20T01:17:25.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.047+0000 7fe76b7ec700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7640ce990 con 0x7fe7640cd1c0 2023-05-20T01:17:25.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.047+0000 7fe76b7ec700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7640cc6e0 con 0x7fe7640cd630 2023-05-20T01:17:25.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.047+0000 7fe769588700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe764159b50 0x7fe76406e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:25.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.047+0000 7fe769588700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe764159b50 0x7fe76406e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59442/0 (socket says 172.21.15.16:59442) 2023-05-20T01:17:25.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.047+0000 7fe769588700 1 -- 172.21.15.16:0/2416576284 learned_addr learned my addr 172.21.15.16:0/2416576284 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:25.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.047+0000 7fe768d87700 1 --2- 172.21.15.16:0/2416576284 >> v2:172.21.15.132:3300/0 conn(0x7fe7640cd1c0 0x7fe7640710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:25.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.047+0000 7fe769d89700 1 --2- 172.21.15.16:0/2416576284 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe7640cd630 0x7fe7640715f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:25.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.047+0000 7fe769588700 1 -- 172.21.15.16:0/2416576284 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe7640cd630 msgr2=0x7fe7640715f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:25.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.047+0000 7fe769588700 1 --2- 172.21.15.16:0/2416576284 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe7640cd630 0x7fe7640715f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:25.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.047+0000 7fe769588700 1 -- 172.21.15.16:0/2416576284 >> v2:172.21.15.132:3300/0 conn(0x7fe7640cd1c0 msgr2=0x7fe7640710b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:25.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.047+0000 7fe769588700 1 --2- 172.21.15.16:0/2416576284 >> v2:172.21.15.132:3300/0 conn(0x7fe7640cd1c0 0x7fe7640710b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:25.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.047+0000 7fe769588700 1 -- 172.21.15.16:0/2416576284 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7640cc9c0 con 0x7fe764159b50 2023-05-20T01:17:25.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.047+0000 7fe768d87700 1 --2- 172.21.15.16:0/2416576284 >> v2:172.21.15.132:3300/0 conn(0x7fe7640cd1c0 0x7fe7640710b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:17:25.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.047+0000 7fe769588700 1 --2- 172.21.15.16:0/2416576284 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe764159b50 0x7fe76406e0a0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fe7640b2bc0 tx=0x7fe760009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb9d348aad582569 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:25.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.047+0000 7fe75bfff700 1 -- 172.21.15.16:0/2416576284 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe760005e30 con 0x7fe764159b50 2023-05-20T01:17:25.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.047+0000 7fe75bfff700 1 -- 172.21.15.16:0/2416576284 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe760004030 con 0x7fe764159b50 2023-05-20T01:17:25.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.047+0000 7fe75bfff700 1 -- 172.21.15.16:0/2416576284 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7600043e0 con 0x7fe764159b50 2023-05-20T01:17:25.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.048+0000 7fe76b7ec700 1 -- 172.21.15.16:0/2416576284 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe764159b50 msgr2=0x7fe76406e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:25.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.048+0000 7fe76b7ec700 1 --2- 172.21.15.16:0/2416576284 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe764159b50 0x7fe76406e0a0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fe7640b2bc0 tx=0x7fe760009bd0 comp rx=0 tx=0).stop 2023-05-20T01:17:25.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.048+0000 7fe76b7ec700 1 -- 172.21.15.16:0/2416576284 shutdown_connections 2023-05-20T01:17:25.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.048+0000 7fe76b7ec700 1 --2- 172.21.15.16:0/2416576284 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe7640cd630 0x7fe7640715f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:25.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.048+0000 7fe76b7ec700 1 --2- 172.21.15.16:0/2416576284 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe764159b50 0x7fe76406e0a0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:25.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.048+0000 7fe76b7ec700 1 --2- 172.21.15.16:0/2416576284 >> v2:172.21.15.132:3300/0 conn(0x7fe7640cd1c0 0x7fe7640710b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:25.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.048+0000 7fe76b7ec700 1 -- 172.21.15.16:0/2416576284 >> 172.21.15.16:0/2416576284 conn(0x7fe7640cffd0 msgr2=0x7fe7640d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:25.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.048+0000 7fe76b7ec700 1 -- 172.21.15.16:0/2416576284 shutdown_connections 2023-05-20T01:17:25.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.048+0000 7fe76b7ec700 1 -- 172.21.15.16:0/2416576284 wait complete. 2023-05-20T01:17:25.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.049+0000 7fe76b7ec700 1 Processor -- start 2023-05-20T01:17:25.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.049+0000 7fe76b7ec700 1 -- start start 2023-05-20T01:17:25.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.050+0000 7fe76b7ec700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe7640cd1c0 0x7fe764155900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:25.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.050+0000 7fe76b7ec700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe7640cd630 0x7fe7641e6b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:25.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.050+0000 7fe76b7ec700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe764159b50 0x7fe7640c6050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:25.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.050+0000 7fe76b7ec700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7641589f0 con 0x7fe764159b50 2023-05-20T01:17:25.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.050+0000 7fe76b7ec700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7640c4fb0 con 0x7fe7640cd1c0 2023-05-20T01:17:25.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.050+0000 7fe76b7ec700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7640c5290 con 0x7fe7640cd630 2023-05-20T01:17:25.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.050+0000 7fe769d89700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe764159b50 0x7fe7640c6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:25.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.050+0000 7fe769d89700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe764159b50 0x7fe7640c6050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59452/0 (socket says 172.21.15.16:59452) 2023-05-20T01:17:25.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.050+0000 7fe769d89700 1 -- 172.21.15.16:0/2988135162 learned_addr learned my addr 172.21.15.16:0/2988135162 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:25.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.050+0000 7fe768d87700 1 --2- 172.21.15.16:0/2988135162 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe7640cd630 0x7fe7641e6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:25.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.050+0000 7fe769588700 1 --2- 172.21.15.16:0/2988135162 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe7640cd1c0 0x7fe764155900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:25.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.050+0000 7fe769d89700 1 -- 172.21.15.16:0/2988135162 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe7640cd630 msgr2=0x7fe7641e6b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:25.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.050+0000 7fe769d89700 1 --2- 172.21.15.16:0/2988135162 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe7640cd630 0x7fe7641e6b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:25.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.050+0000 7fe769d89700 1 -- 172.21.15.16:0/2988135162 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe7640cd1c0 msgr2=0x7fe764155900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:25.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.050+0000 7fe769d89700 1 --2- 172.21.15.16:0/2988135162 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe7640cd1c0 0x7fe764155900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:25.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.051+0000 7fe769d89700 1 -- 172.21.15.16:0/2988135162 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe760009500 con 0x7fe764159b50 2023-05-20T01:17:25.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.051+0000 7fe769d89700 1 --2- 172.21.15.16:0/2988135162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe764159b50 0x7fe7640c6050 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fe75c00cd70 tx=0x7fe75c00a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:25.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.051+0000 7fe759ffb700 1 -- 172.21.15.16:0/2988135162 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe75c003a20 con 0x7fe764159b50 2023-05-20T01:17:25.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.051+0000 7fe759ffb700 1 -- 172.21.15.16:0/2988135162 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe75c003b80 con 0x7fe764159b50 2023-05-20T01:17:25.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.051+0000 7fe76b7ec700 1 -- 172.21.15.16:0/2988135162 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe764071e80 con 0x7fe764159b50 2023-05-20T01:17:25.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.051+0000 7fe759ffb700 1 -- 172.21.15.16:0/2988135162 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe75c004020 con 0x7fe764159b50 2023-05-20T01:17:25.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.051+0000 7fe76b7ec700 1 -- 172.21.15.16:0/2988135162 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe76406e5e0 con 0x7fe764159b50 2023-05-20T01:17:25.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.052+0000 7fe76b7ec700 1 -- 172.21.15.16:0/2988135162 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7640cec70 con 0x7fe764159b50 2023-05-20T01:17:25.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.052+0000 7fe759ffb700 1 -- 172.21.15.16:0/2988135162 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe75c004180 con 0x7fe764159b50 2023-05-20T01:17:25.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.053+0000 7fe759ffb700 1 --2- 172.21.15.16:0/2988135162 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe74806ace0 0x7fe74806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:25.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.053+0000 7fe759ffb700 1 -- 172.21.15.16:0/2988135162 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fe75c012070 con 0x7fe764159b50 2023-05-20T01:17:25.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.053+0000 7fe769588700 1 --2- 172.21.15.16:0/2988135162 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe74806ace0 0x7fe74806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:25.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.053+0000 7fe769588700 1 --2- 172.21.15.16:0/2988135162 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe74806ace0 0x7fe74806d190 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe7640b2bc0 tx=0x7fe760008070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:25.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.057+0000 7fe759ffb700 1 -- 172.21.15.16:0/2988135162 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fe75c054d60 con 0x7fe764159b50 2023-05-20T01:17:25.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.253+0000 7fe76b7ec700 1 -- 172.21.15.16:0/2988135162 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe7640747b0 con 0x7fe764159b50 2023-05-20T01:17:25.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.253+0000 7fe759ffb700 1 -- 172.21.15.16:0/2988135162 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe75c058380 con 0x7fe764159b50 2023-05-20T01:17:25.258 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2023-05-20T01:17:25.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.256+0000 7fe76b7ec700 1 -- 172.21.15.16:0/2988135162 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe74806ace0 msgr2=0x7fe74806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:25.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.256+0000 7fe76b7ec700 1 --2- 172.21.15.16:0/2988135162 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe74806ace0 0x7fe74806d190 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe7640b2bc0 tx=0x7fe760008070 comp rx=0 tx=0).stop 2023-05-20T01:17:25.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.256+0000 7fe76b7ec700 1 -- 172.21.15.16:0/2988135162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe764159b50 msgr2=0x7fe7640c6050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:25.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.256+0000 7fe76b7ec700 1 --2- 172.21.15.16:0/2988135162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe764159b50 0x7fe7640c6050 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fe75c00cd70 tx=0x7fe75c00a570 comp rx=0 tx=0).stop 2023-05-20T01:17:25.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.256+0000 7fe76b7ec700 1 -- 172.21.15.16:0/2988135162 shutdown_connections 2023-05-20T01:17:25.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.256+0000 7fe76b7ec700 1 --2- 172.21.15.16:0/2988135162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe764159b50 0x7fe7640c6050 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:25.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.256+0000 7fe76b7ec700 1 --2- 172.21.15.16:0/2988135162 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe7640cd1c0 0x7fe764155900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:25.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.256+0000 7fe76b7ec700 1 --2- 172.21.15.16:0/2988135162 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe74806ace0 0x7fe74806d190 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:25.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.256+0000 7fe76b7ec700 1 --2- 172.21.15.16:0/2988135162 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe7640cd630 0x7fe7641e6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:25.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.256+0000 7fe76b7ec700 1 -- 172.21.15.16:0/2988135162 >> 172.21.15.16:0/2988135162 conn(0x7fe7640cffd0 msgr2=0x7fe7640b1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:25.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.256+0000 7fe76b7ec700 1 -- 172.21.15.16:0/2988135162 shutdown_connections 2023-05-20T01:17:25.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.256+0000 7fe76b7ec700 1 -- 172.21.15.16:0/2988135162 wait complete. 2023-05-20T01:17:25.272 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-05-20T01:17:25.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.398+0000 7fb447ca7700 1 Processor -- start 2023-05-20T01:17:25.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.398+0000 7fb447ca7700 1 -- start start 2023-05-20T01:17:25.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.399+0000 7fb447ca7700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb44006a510 0x7fb44006a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:25.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.399+0000 7fb447ca7700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb44006dd00 con 0x7fb44015cec0 2023-05-20T01:17:25.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.399+0000 7fb447ca7700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4400cfa60 con 0x7fb4400cd4f0 2023-05-20T01:17:25.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.399+0000 7fb447ca7700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4400cd940 con 0x7fb44006a510 2023-05-20T01:17:25.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.399+0000 7fb445a43700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fb44015cec0 0x7fb44006e190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:42752/0 (socket says 172.21.15.16:42752) 2023-05-20T01:17:25.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.399+0000 7fb445a43700 1 -- 172.21.15.16:0/2636887117 learned_addr learned my addr 172.21.15.16:0/2636887117 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:17:25.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.399+0000 7fb446244700 1 --2- 172.21.15.16:0/2636887117 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb44006a510 0x7fb44006a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:25.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.399+0000 7fb444a41700 1 -- 172.21.15.16:0/2636887117 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 960019095 0 0) 0x7fb44006dd00 con 0x7fb44015cec0 2023-05-20T01:17:25.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.399+0000 7fb444a41700 1 -- 172.21.15.16:0/2636887117 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb42c0035b0 con 0x7fb44015cec0 2023-05-20T01:17:25.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.400+0000 7fb444a41700 1 -- 172.21.15.16:0/2636887117 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3959207627 0 0) 0x7fb42c0035b0 con 0x7fb44015cec0 2023-05-20T01:17:25.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.400+0000 7fb444a41700 1 -- 172.21.15.16:0/2636887117 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb44006a510 msgr2=0x7fb44006a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:25.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.400+0000 7fb444a41700 1 --2- 172.21.15.16:0/2636887117 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb44006a510 0x7fb44006a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:25.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.400+0000 7fb444a41700 1 -- 172.21.15.16:0/2636887117 >> v1:172.21.15.132:6789/0 conn(0x7fb4400cd4f0 legacy=0x7fb4400cf1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:25.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.400+0000 7fb444a41700 1 -- 172.21.15.16:0/2636887117 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb44006d6b0 con 0x7fb44015cec0 2023-05-20T01:17:25.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.400+0000 7fb444a41700 1 -- 172.21.15.16:0/2636887117 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7fb430001670 con 0x7fb44015cec0 2023-05-20T01:17:25.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.400+0000 7fb444a41700 1 -- 172.21.15.16:0/2636887117 >> v1:172.21.15.16:6789/0 conn(0x7fb44015cec0 legacy=0x7fb44006e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:25.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.400+0000 7fb444a41700 1 --2- 172.21.15.16:0/2636887117 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb44006a510 0x7fb42c003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:25.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.400+0000 7fb444a41700 1 --2- 172.21.15.16:0/2636887117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb42c003fb0 0x7fb42c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:25.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.400+0000 7fb444a41700 1 --2- 172.21.15.16:0/2636887117 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb42c004d90 0x7fb42c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:25.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.400+0000 7fb444a41700 1 -- 172.21.15.16:0/2636887117 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4400cd940 con 0x7fb42c003fb0 2023-05-20T01:17:25.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.400+0000 7fb444a41700 1 -- 172.21.15.16:0/2636887117 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb42c00baf0 con 0x7fb42c004d90 2023-05-20T01:17:25.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.400+0000 7fb444a41700 1 -- 172.21.15.16:0/2636887117 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb42c00bcb0 con 0x7fb44006a510 2023-05-20T01:17:25.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.400+0000 7fb445242700 1 --2- 172.21.15.16:0/2636887117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb42c003fb0 0x7fb42c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:25.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.400+0000 7fb445a43700 1 --2- 172.21.15.16:0/2636887117 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb44006a510 0x7fb42c003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:25.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.400+0000 7fb446244700 1 --2- 172.21.15.16:0/2636887117 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb42c004d90 0x7fb42c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:25.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.400+0000 7fb445242700 1 -- 172.21.15.16:0/2636887117 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb44006a510 msgr2=0x7fb42c003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:25.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.400+0000 7fb445242700 1 --2- 172.21.15.16:0/2636887117 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb44006a510 0x7fb42c003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:25.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.400+0000 7fb445242700 1 -- 172.21.15.16:0/2636887117 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb42c004d90 msgr2=0x7fb42c005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:25.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.400+0000 7fb445242700 1 --2- 172.21.15.16:0/2636887117 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb42c004d90 0x7fb42c005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:25.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.400+0000 7fb445242700 1 -- 172.21.15.16:0/2636887117 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb42c0035b0 con 0x7fb42c003fb0 2023-05-20T01:17:25.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.400+0000 7fb446244700 1 --2- 172.21.15.16:0/2636887117 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb42c004d90 0x7fb42c005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:17:25.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.400+0000 7fb445242700 1 --2- 172.21.15.16:0/2636887117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb42c003fb0 0x7fb42c004850 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fb43c007fd0 tx=0x7fb42c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c95e789b0d098cd9 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:25.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.401+0000 7fb444a41700 1 -- 172.21.15.16:0/2636887117 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb43c0012f0 con 0x7fb42c003fb0 2023-05-20T01:17:25.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.401+0000 7fb444a41700 1 -- 172.21.15.16:0/2636887117 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb43c0052f0 con 0x7fb42c003fb0 2023-05-20T01:17:25.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.401+0000 7fb444a41700 1 -- 172.21.15.16:0/2636887117 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb43c006860 con 0x7fb42c003fb0 2023-05-20T01:17:25.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.401+0000 7fb447ca7700 1 -- 172.21.15.16:0/2636887117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb42c003fb0 msgr2=0x7fb42c004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:25.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.401+0000 7fb447ca7700 1 --2- 172.21.15.16:0/2636887117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb42c003fb0 0x7fb42c004850 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fb43c007fd0 tx=0x7fb42c005880 comp rx=0 tx=0).stop 2023-05-20T01:17:25.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.401+0000 7fb446244700 1 -- 172.21.15.16:0/2636887117 reap_dead start 2023-05-20T01:17:25.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.401+0000 7fb447ca7700 1 -- 172.21.15.16:0/2636887117 shutdown_connections 2023-05-20T01:17:25.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.401+0000 7fb447ca7700 1 -- 172.21.15.16:0/2636887117 >> 172.21.15.16:0/2636887117 conn(0x7fb44006c260 msgr2=0x7fb4400cffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:25.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.401+0000 7fb447ca7700 1 -- 172.21.15.16:0/2636887117 shutdown_connections 2023-05-20T01:17:25.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.401+0000 7fb447ca7700 1 -- 172.21.15.16:0/2636887117 wait complete. 2023-05-20T01:17:25.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.402+0000 7fb447ca7700 1 Processor -- start 2023-05-20T01:17:25.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.403+0000 7fb447ca7700 1 -- start start 2023-05-20T01:17:25.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.403+0000 7fb447ca7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb42c003fb0 0x7fb4400c75c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:25.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.403+0000 7fb447ca7700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb44014c4f0 0x7fb44014f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:25.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.403+0000 7fb447ca7700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb4400c4ee0 0x7fb4400c5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:25.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.403+0000 7fb447ca7700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4400c3b00 con 0x7fb42c003fb0 2023-05-20T01:17:25.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.403+0000 7fb447ca7700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4400c3da0 con 0x7fb44014c4f0 2023-05-20T01:17:25.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.403+0000 7fb447ca7700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb44014f8a0 con 0x7fb4400c4ee0 2023-05-20T01:17:25.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.403+0000 7fb445a43700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb42c003fb0 0x7fb4400c75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:25.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.403+0000 7fb445a43700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb42c003fb0 0x7fb4400c75c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59468/0 (socket says 172.21.15.16:59468) 2023-05-20T01:17:25.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.403+0000 7fb445a43700 1 -- 172.21.15.16:0/2360267815 learned_addr learned my addr 172.21.15.16:0/2360267815 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:25.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.403+0000 7fb445242700 1 --2- 172.21.15.16:0/2360267815 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb44014c4f0 0x7fb44014f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:25.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.403+0000 7fb446244700 1 --2- 172.21.15.16:0/2360267815 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb4400c4ee0 0x7fb4400c5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:25.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.404+0000 7fb445a43700 1 -- 172.21.15.16:0/2360267815 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb4400c4ee0 msgr2=0x7fb4400c5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:25.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.404+0000 7fb445a43700 1 --2- 172.21.15.16:0/2360267815 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb4400c4ee0 0x7fb4400c5350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:25.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.404+0000 7fb445a43700 1 -- 172.21.15.16:0/2360267815 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb44014c4f0 msgr2=0x7fb44014f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:25.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.404+0000 7fb445a43700 1 --2- 172.21.15.16:0/2360267815 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb44014c4f0 0x7fb44014f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:25.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.404+0000 7fb445a43700 1 -- 172.21.15.16:0/2360267815 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb44015d290 con 0x7fb42c003fb0 2023-05-20T01:17:25.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.404+0000 7fb446244700 1 --2- 172.21.15.16:0/2360267815 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb4400c4ee0 0x7fb4400c5350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:17:25.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.404+0000 7fb445a43700 1 --2- 172.21.15.16:0/2360267815 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb42c003fb0 0x7fb4400c75c0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fb4300072c0 tx=0x7fb4300075d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:25.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.404+0000 7fb4367fc700 1 -- 172.21.15.16:0/2360267815 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb43000f600 con 0x7fb42c003fb0 2023-05-20T01:17:25.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.404+0000 7fb4367fc700 1 -- 172.21.15.16:0/2360267815 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb43000f760 con 0x7fb42c003fb0 2023-05-20T01:17:25.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.404+0000 7fb447ca7700 1 -- 172.21.15.16:0/2360267815 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb44006d990 con 0x7fb42c003fb0 2023-05-20T01:17:25.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.404+0000 7fb4367fc700 1 -- 172.21.15.16:0/2360267815 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb43000faf0 con 0x7fb42c003fb0 2023-05-20T01:17:25.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.404+0000 7fb447ca7700 1 -- 172.21.15.16:0/2360267815 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4400cfd40 con 0x7fb42c003fb0 2023-05-20T01:17:25.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.405+0000 7fb447ca7700 1 -- 172.21.15.16:0/2360267815 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4400c4080 con 0x7fb42c003fb0 2023-05-20T01:17:25.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.405+0000 7fb4367fc700 1 -- 172.21.15.16:0/2360267815 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb43000fc50 con 0x7fb42c003fb0 2023-05-20T01:17:25.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.406+0000 7fb4367fc700 1 --2- 172.21.15.16:0/2360267815 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb42406ace0 0x7fb42406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:25.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.406+0000 7fb4367fc700 1 -- 172.21.15.16:0/2360267815 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fb4300896f0 con 0x7fb42c003fb0 2023-05-20T01:17:25.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.406+0000 7fb445242700 1 --2- 172.21.15.16:0/2360267815 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb42406ace0 0x7fb42406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:25.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.406+0000 7fb445242700 1 --2- 172.21.15.16:0/2360267815 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb42406ace0 0x7fb42406d190 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb4400b5b70 tx=0x7fb43c011000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:25.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.410+0000 7fb4367fc700 1 -- 172.21.15.16:0/2360267815 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fb430054470 con 0x7fb42c003fb0 2023-05-20T01:17:25.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.609+0000 7fb447ca7700 1 -- 172.21.15.16:0/2360267815 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fb4400b3610 con 0x7fb42c003fb0 2023-05-20T01:17:25.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.609+0000 7fb4367fc700 1 -- 172.21.15.16:0/2360267815 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7fb430057a90 con 0x7fb42c003fb0 2023-05-20T01:17:25.613 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-05-20T01:17:25.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.611+0000 7fb447ca7700 1 -- 172.21.15.16:0/2360267815 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb42406ace0 msgr2=0x7fb42406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:25.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.612+0000 7fb447ca7700 1 --2- 172.21.15.16:0/2360267815 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb42406ace0 0x7fb42406d190 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb4400b5b70 tx=0x7fb43c011000 comp rx=0 tx=0).stop 2023-05-20T01:17:25.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.612+0000 7fb447ca7700 1 -- 172.21.15.16:0/2360267815 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb42c003fb0 msgr2=0x7fb4400c75c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:25.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.612+0000 7fb447ca7700 1 --2- 172.21.15.16:0/2360267815 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb42c003fb0 0x7fb4400c75c0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fb4300072c0 tx=0x7fb4300075d0 comp rx=0 tx=0).stop 2023-05-20T01:17:25.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.612+0000 7fb447ca7700 1 -- 172.21.15.16:0/2360267815 shutdown_connections 2023-05-20T01:17:25.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.612+0000 7fb447ca7700 1 --2- 172.21.15.16:0/2360267815 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb42406ace0 0x7fb42406d190 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:25.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.612+0000 7fb447ca7700 1 --2- 172.21.15.16:0/2360267815 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb4400c4ee0 0x7fb4400c5350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:25.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.612+0000 7fb447ca7700 1 --2- 172.21.15.16:0/2360267815 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb42c003fb0 0x7fb4400c75c0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:25.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.612+0000 7fb447ca7700 1 --2- 172.21.15.16:0/2360267815 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb44014c4f0 0x7fb44014f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:25.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.612+0000 7fb447ca7700 1 -- 172.21.15.16:0/2360267815 >> 172.21.15.16:0/2360267815 conn(0x7fb44006c260 msgr2=0x7fb44006a990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:25.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.612+0000 7fb447ca7700 1 -- 172.21.15.16:0/2360267815 shutdown_connections 2023-05-20T01:17:25.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.612+0000 7fb447ca7700 1 -- 172.21.15.16:0/2360267815 wait complete. 2023-05-20T01:17:25.627 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-05-20T01:17:25.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.750+0000 7f9375f8c700 1 Processor -- start 2023-05-20T01:17:25.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.750+0000 7f9375f8c700 1 -- start start 2023-05-20T01:17:25.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.751+0000 7f9375f8c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f937006d4b0 0x7f93700cf890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:25.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.751+0000 7f9375f8c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f937006a510 0x7f937006a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:25.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.751+0000 7f9375f8c700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93700cc510 con 0x7f937006f450 2023-05-20T01:17:25.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.751+0000 7f9375f8c700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f937006aef0 con 0x7f937006a510 2023-05-20T01:17:25.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.751+0000 7f9375f8c700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9370071ed0 con 0x7f937006d4b0 2023-05-20T01:17:25.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.751+0000 7f936ffff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f937006a510 0x7f937006a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:25.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.751+0000 7f936ffff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f937006a510 0x7f937006a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59480/0 (socket says 172.21.15.16:59480) 2023-05-20T01:17:25.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.751+0000 7f936ffff700 1 -- 172.21.15.16:0/3864253506 learned_addr learned my addr 172.21.15.16:0/3864253506 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:25.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.751+0000 7f936effd700 1 --2- 172.21.15.16:0/3864253506 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f937006d4b0 0x7f93700cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:25.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.751+0000 7f936ffff700 1 -- 172.21.15.16:0/3864253506 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f937006d4b0 msgr2=0x7f93700cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:25.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.751+0000 7f936ffff700 1 --2- 172.21.15.16:0/3864253506 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f937006d4b0 0x7f93700cf890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:25.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.751+0000 7f936ffff700 1 -- 172.21.15.16:0/3864253506 >> v1:172.21.15.132:6789/0 conn(0x7f937006f450 legacy=0x7f9370161080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:25.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.751+0000 7f936ffff700 1 -- 172.21.15.16:0/3864253506 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93700721b0 con 0x7f937006a510 2023-05-20T01:17:25.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.751+0000 7f936ffff700 1 --2- 172.21.15.16:0/3864253506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f937006a510 0x7f937006a9b0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f9368009e50 tx=0x7f936800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b27d9e8a607377d0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:25.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.751+0000 7f936e7fc700 1 -- 172.21.15.16:0/3864253506 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f936800e070 con 0x7f937006a510 2023-05-20T01:17:25.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.751+0000 7f936e7fc700 1 -- 172.21.15.16:0/3864253506 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9368003a20 con 0x7f937006a510 2023-05-20T01:17:25.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.751+0000 7f936e7fc700 1 -- 172.21.15.16:0/3864253506 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9368003dd0 con 0x7f937006a510 2023-05-20T01:17:25.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.752+0000 7f9375f8c700 1 -- 172.21.15.16:0/3864253506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f937006a510 msgr2=0x7f937006a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:25.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.752+0000 7f9375f8c700 1 --2- 172.21.15.16:0/3864253506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f937006a510 0x7f937006a9b0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f9368009e50 tx=0x7f936800b330 comp rx=0 tx=0).stop 2023-05-20T01:17:25.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.752+0000 7f9375f8c700 1 -- 172.21.15.16:0/3864253506 shutdown_connections 2023-05-20T01:17:25.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.752+0000 7f9375f8c700 1 --2- 172.21.15.16:0/3864253506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f937006a510 0x7f937006a9b0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:25.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.752+0000 7f9375f8c700 1 --2- 172.21.15.16:0/3864253506 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f937006d4b0 0x7f93700cf890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:25.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.752+0000 7f9375f8c700 1 -- 172.21.15.16:0/3864253506 >> 172.21.15.16:0/3864253506 conn(0x7f93700cffb0 msgr2=0x7f93700d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:25.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.752+0000 7f9375f8c700 1 -- 172.21.15.16:0/3864253506 shutdown_connections 2023-05-20T01:17:25.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.752+0000 7f9375f8c700 1 -- 172.21.15.16:0/3864253506 wait complete. 2023-05-20T01:17:25.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.753+0000 7f9375f8c700 1 Processor -- start 2023-05-20T01:17:25.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.753+0000 7f9375f8c700 1 -- start start 2023-05-20T01:17:25.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.754+0000 7f9375f8c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f937006a510 0x7f93701e2450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:25.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.754+0000 7f9375f8c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f937006d4b0 0x7f93700c7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:25.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.754+0000 7f9375f8c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f937006f450 0x7f93701583f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:25.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.754+0000 7f9375f8c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93701e4700 con 0x7f937006d4b0 2023-05-20T01:17:25.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.754+0000 7f9375f8c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93701e49a0 con 0x7f937006a510 2023-05-20T01:17:25.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.754+0000 7f9375f8c700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93701e3260 con 0x7f937006f450 2023-05-20T01:17:25.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.754+0000 7f936effd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f937006d4b0 0x7f93700c7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:25.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.754+0000 7f936effd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f937006d4b0 0x7f93700c7bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59482/0 (socket says 172.21.15.16:59482) 2023-05-20T01:17:25.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.754+0000 7f936effd700 1 -- 172.21.15.16:0/381394818 learned_addr learned my addr 172.21.15.16:0/381394818 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:25.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.754+0000 7f936f7fe700 1 --2- 172.21.15.16:0/381394818 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f937006a510 0x7f93701e2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:25.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.754+0000 7f936ffff700 1 --2- 172.21.15.16:0/381394818 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f937006f450 0x7f93701583f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:25.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.755+0000 7f936effd700 1 -- 172.21.15.16:0/381394818 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f937006f450 msgr2=0x7f93701583f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:25.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.755+0000 7f936effd700 1 --2- 172.21.15.16:0/381394818 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f937006f450 0x7f93701583f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:25.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.755+0000 7f936effd700 1 -- 172.21.15.16:0/381394818 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f937006a510 msgr2=0x7f93701e2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:25.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.755+0000 7f936effd700 1 --2- 172.21.15.16:0/381394818 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f937006a510 0x7f93701e2450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:25.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.755+0000 7f936effd700 1 -- 172.21.15.16:0/381394818 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9368009b30 con 0x7f937006d4b0 2023-05-20T01:17:25.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.755+0000 7f936effd700 1 --2- 172.21.15.16:0/381394818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f937006d4b0 0x7f93700c7bc0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f936400e810 tx=0x7f936400eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:25.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.755+0000 7f9357fff700 1 -- 172.21.15.16:0/381394818 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f936400c070 con 0x7f937006d4b0 2023-05-20T01:17:25.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.755+0000 7f9357fff700 1 -- 172.21.15.16:0/381394818 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9364013610 con 0x7f937006d4b0 2023-05-20T01:17:25.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.755+0000 7f9375f8c700 1 -- 172.21.15.16:0/381394818 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93700cbe50 con 0x7f937006d4b0 2023-05-20T01:17:25.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.755+0000 7f9357fff700 1 -- 172.21.15.16:0/381394818 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93640139c0 con 0x7f937006d4b0 2023-05-20T01:17:25.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.755+0000 7f9375f8c700 1 -- 172.21.15.16:0/381394818 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93700cc1a0 con 0x7f937006d4b0 2023-05-20T01:17:25.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.756+0000 7f9375f8c700 1 -- 172.21.15.16:0/381394818 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93701e4fb0 con 0x7f937006d4b0 2023-05-20T01:17:25.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.756+0000 7f9357fff700 1 -- 172.21.15.16:0/381394818 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9364013b20 con 0x7f937006d4b0 2023-05-20T01:17:25.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.757+0000 7f9357fff700 1 --2- 172.21.15.16:0/381394818 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f935c06ace0 0x7f935c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:25.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.757+0000 7f9357fff700 1 -- 172.21.15.16:0/381394818 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f93640894a0 con 0x7f937006d4b0 2023-05-20T01:17:25.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.757+0000 7f936f7fe700 1 --2- 172.21.15.16:0/381394818 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f935c06ace0 0x7f935c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:25.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.757+0000 7f936f7fe700 1 --2- 172.21.15.16:0/381394818 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f935c06ace0 0x7f935c06d190 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9360009960 tx=0x7f93600090e0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:25.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.761+0000 7f9357fff700 1 -- 172.21.15.16:0/381394818 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f9364057cc0 con 0x7f937006d4b0 2023-05-20T01:17:25.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.977+0000 7f9375f8c700 1 -- 172.21.15.16:0/381394818 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f93700cc980 con 0x7f937006d4b0 2023-05-20T01:17:25.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.977+0000 7f9357fff700 1 -- 172.21.15.16:0/381394818 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) v1 ==== 80+0+0 (secure 0 0 0) 0x7f9364057850 con 0x7f937006d4b0 2023-05-20T01:17:25.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.979+0000 7f9375f8c700 1 -- 172.21.15.16:0/381394818 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f935c06ace0 msgr2=0x7f935c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:25.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.979+0000 7f9375f8c700 1 --2- 172.21.15.16:0/381394818 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f935c06ace0 0x7f935c06d190 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9360009960 tx=0x7f93600090e0 comp rx=0 tx=0).stop 2023-05-20T01:17:25.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.979+0000 7f9375f8c700 1 -- 172.21.15.16:0/381394818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f937006d4b0 msgr2=0x7f93700c7bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:25.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.979+0000 7f9375f8c700 1 --2- 172.21.15.16:0/381394818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f937006d4b0 0x7f93700c7bc0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f936400e810 tx=0x7f936400eb20 comp rx=0 tx=0).stop 2023-05-20T01:17:25.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.979+0000 7f9375f8c700 1 -- 172.21.15.16:0/381394818 shutdown_connections 2023-05-20T01:17:25.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.979+0000 7f9375f8c700 1 --2- 172.21.15.16:0/381394818 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f935c06ace0 0x7f935c06d190 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:25.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.979+0000 7f9375f8c700 1 --2- 172.21.15.16:0/381394818 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f937006f450 0x7f93701583f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:25.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.979+0000 7f9375f8c700 1 --2- 172.21.15.16:0/381394818 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f937006a510 0x7f93701e2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:25.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.979+0000 7f9375f8c700 1 --2- 172.21.15.16:0/381394818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f937006d4b0 0x7f93700c7bc0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:25.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.980+0000 7f9375f8c700 1 -- 172.21.15.16:0/381394818 >> 172.21.15.16:0/381394818 conn(0x7f93700cffb0 msgr2=0x7f937006e030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:25.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.980+0000 7f9375f8c700 1 -- 172.21.15.16:0/381394818 shutdown_connections 2023-05-20T01:17:25.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:25.980+0000 7f9375f8c700 1 -- 172.21.15.16:0/381394818 wait complete. 2023-05-20T01:17:25.985 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2023-05-20T01:17:25.995 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2023-05-20T01:17:25.997 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:17:25.998 DEBUG:teuthology.orchestra.run.smithi016:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2023-05-20T01:17:26.036 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2023-05-20T01:17:26.036 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:17:26.037 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-05-20T01:17:26.093 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2023-05-20T01:17:26.094 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/proc/self/mounts of=/dev/stdout 2023-05-20T01:17:26.115 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2023-05-20T01:17:26.116 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/proc/self/mounts of=/dev/stdout 2023-05-20T01:17:26.176 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2023-05-20T01:17:26.177 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/proc/self/mounts of=/dev/stdout 2023-05-20T01:17:26.238 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-20T01:17:26.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.360+0000 7f3964f5f700 1 Processor -- start 2023-05-20T01:17:26.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.360+0000 7f3964f5f700 1 -- start start 2023-05-20T01:17:26.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.361+0000 7f3964f5f700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f396006d4b0 0x7f39600cf890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:26.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.361+0000 7f3964f5f700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f396006a510 0x7f396006a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:26.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.361+0000 7f3964f5f700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39600cc510 con 0x7f396006f450 2023-05-20T01:17:26.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.361+0000 7f3964f5f700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f396006aef0 con 0x7f396006a510 2023-05-20T01:17:26.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.361+0000 7f3964f5f700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3960071ed0 con 0x7f396006d4b0 2023-05-20T01:17:26.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.361+0000 7f395dd9b700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f396006d4b0 0x7f39600cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:26.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.361+0000 7f395ed9d700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f396006a510 0x7f396006a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:26.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.361+0000 7f395e59c700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f396006f450 0x7f3960161080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.16:52500/0 (socket says 172.21.15.16:52500) 2023-05-20T01:17:26.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.361+0000 7f395e59c700 1 -- 172.21.15.16:0/296468634 learned_addr learned my addr 172.21.15.16:0/296468634 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:17:26.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.361+0000 7f395d59a700 1 -- 172.21.15.16:0/296468634 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1129340350 0 0) 0x7f39600cc510 con 0x7f396006f450 2023-05-20T01:17:26.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.361+0000 7f395d59a700 1 -- 172.21.15.16:0/296468634 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f39480035b0 con 0x7f396006f450 2023-05-20T01:17:26.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.362+0000 7f395dd9b700 1 -- 172.21.15.16:0/296468634 >> v2:172.21.15.132:3300/0 conn(0x7f396006a510 msgr2=0x7f396006a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:26.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.362+0000 7f395dd9b700 1 --2- 172.21.15.16:0/296468634 >> v2:172.21.15.132:3300/0 conn(0x7f396006a510 0x7f396006a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:26.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.362+0000 7f395dd9b700 1 -- 172.21.15.16:0/296468634 >> v1:172.21.15.132:6789/0 conn(0x7f396006f450 legacy=0x7f3960161080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:26.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.362+0000 7f395dd9b700 1 -- 172.21.15.16:0/296468634 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39600721b0 con 0x7f396006d4b0 2023-05-20T01:17:26.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.362+0000 7f395ed9d700 1 --2- 172.21.15.16:0/296468634 >> v2:172.21.15.132:3300/0 conn(0x7f396006a510 0x7f396006a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-05-20T01:17:26.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.362+0000 7f395dd9b700 1 --2- 172.21.15.16:0/296468634 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f396006d4b0 0x7f39600cf890 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3954009840 tx=0x7f3954009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1fcb7504cdb1c4b6 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:26.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.362+0000 7f395d59a700 1 -- 172.21.15.16:0/296468634 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3954011830 con 0x7f396006d4b0 2023-05-20T01:17:26.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.362+0000 7f395d59a700 1 -- 172.21.15.16:0/296468634 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3954011990 con 0x7f396006d4b0 2023-05-20T01:17:26.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.362+0000 7f395d59a700 1 -- 172.21.15.16:0/296468634 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3954011d00 con 0x7f396006d4b0 2023-05-20T01:17:26.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.362+0000 7f3964f5f700 1 -- 172.21.15.16:0/296468634 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f396006d4b0 msgr2=0x7f39600cf890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:26.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.362+0000 7f3964f5f700 1 --2- 172.21.15.16:0/296468634 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f396006d4b0 0x7f39600cf890 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3954009840 tx=0x7f3954009b50 comp rx=0 tx=0).stop 2023-05-20T01:17:26.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.363+0000 7f3964f5f700 1 -- 172.21.15.16:0/296468634 shutdown_connections 2023-05-20T01:17:26.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.363+0000 7f3964f5f700 1 --2- 172.21.15.16:0/296468634 >> v2:172.21.15.132:3300/0 conn(0x7f396006a510 0x7f396006a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:26.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.363+0000 7f3964f5f700 1 --2- 172.21.15.16:0/296468634 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f396006d4b0 0x7f39600cf890 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:26.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.363+0000 7f3964f5f700 1 -- 172.21.15.16:0/296468634 >> 172.21.15.16:0/296468634 conn(0x7f39600cffb0 msgr2=0x7f39600d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:26.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.363+0000 7f3964f5f700 1 -- 172.21.15.16:0/296468634 shutdown_connections 2023-05-20T01:17:26.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.363+0000 7f3964f5f700 1 -- 172.21.15.16:0/296468634 wait complete. 2023-05-20T01:17:26.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.364+0000 7f3964f5f700 1 Processor -- start 2023-05-20T01:17:26.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.364+0000 7f3964f5f700 1 -- start start 2023-05-20T01:17:26.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.365+0000 7f3964f5f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f396006a510 0x7f39600b57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:26.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.365+0000 7f3964f5f700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f396006d4b0 0x7f3960153940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:26.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.365+0000 7f3964f5f700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f396006f450 0x7f39600c90e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:26.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.365+0000 7f3964f5f700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3960155df0 con 0x7f396006a510 2023-05-20T01:17:26.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.365+0000 7f3964f5f700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39601f2330 con 0x7f396006d4b0 2023-05-20T01:17:26.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.365+0000 7f3964f5f700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39600b5cf0 con 0x7f396006f450 2023-05-20T01:17:26.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.365+0000 7f395e59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f396006a510 0x7f39600b57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:26.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.365+0000 7f395e59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f396006a510 0x7f39600b57b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59496/0 (socket says 172.21.15.16:59496) 2023-05-20T01:17:26.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.365+0000 7f395e59c700 1 -- 172.21.15.16:0/1962463937 learned_addr learned my addr 172.21.15.16:0/1962463937 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:26.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.365+0000 7f395dd9b700 1 --2- 172.21.15.16:0/1962463937 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f396006d4b0 0x7f3960153940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:26.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.365+0000 7f395ed9d700 1 --2- 172.21.15.16:0/1962463937 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f396006f450 0x7f39600c90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:26.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.365+0000 7f395e59c700 1 -- 172.21.15.16:0/1962463937 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f396006f450 msgr2=0x7f39600c90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:26.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.365+0000 7f395e59c700 1 --2- 172.21.15.16:0/1962463937 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f396006f450 0x7f39600c90e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:26.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.365+0000 7f395e59c700 1 -- 172.21.15.16:0/1962463937 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f396006d4b0 msgr2=0x7f3960153940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:26.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.365+0000 7f395e59c700 1 --2- 172.21.15.16:0/1962463937 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f396006d4b0 0x7f3960153940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:26.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.365+0000 7f395e59c700 1 -- 172.21.15.16:0/1962463937 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3958009a70 con 0x7f396006a510 2023-05-20T01:17:26.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.365+0000 7f395e59c700 1 --2- 172.21.15.16:0/1962463937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f396006a510 0x7f39600b57b0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f3950001eb0 tx=0x7f395000c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:26.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.366+0000 7f394effd700 1 -- 172.21.15.16:0/1962463937 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3950005d70 con 0x7f396006a510 2023-05-20T01:17:26.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.366+0000 7f394effd700 1 -- 172.21.15.16:0/1962463937 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f395000ce40 con 0x7f396006a510 2023-05-20T01:17:26.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.366+0000 7f394effd700 1 -- 172.21.15.16:0/1962463937 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39500072f0 con 0x7f396006a510 2023-05-20T01:17:26.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.366+0000 7f3964f5f700 1 -- 172.21.15.16:0/1962463937 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39540094a0 con 0x7f396006a510 2023-05-20T01:17:26.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.366+0000 7f3964f5f700 1 -- 172.21.15.16:0/1962463937 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39600cbe50 con 0x7f396006a510 2023-05-20T01:17:26.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.367+0000 7f3964f5f700 1 -- 172.21.15.16:0/1962463937 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39601560d0 con 0x7f396006a510 2023-05-20T01:17:26.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.367+0000 7f394effd700 1 -- 172.21.15.16:0/1962463937 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3950007450 con 0x7f396006a510 2023-05-20T01:17:26.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.368+0000 7f394effd700 1 --2- 172.21.15.16:0/1962463937 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f394006ace0 0x7f394006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:26.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.368+0000 7f395dd9b700 1 --2- 172.21.15.16:0/1962463937 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f394006ace0 0x7f394006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:26.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.368+0000 7f394effd700 1 -- 172.21.15.16:0/1962463937 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f3950088c90 con 0x7f396006a510 2023-05-20T01:17:26.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.368+0000 7f395dd9b700 1 --2- 172.21.15.16:0/1962463937 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f394006ace0 0x7f394006d190 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3954000c00 tx=0x7f3954008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:26.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.372+0000 7f394effd700 1 -- 172.21.15.16:0/1962463937 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f39500538e0 con 0x7f396006a510 2023-05-20T01:17:26.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.574+0000 7f3964f5f700 1 -- 172.21.15.16:0/1962463937 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f39600cc9a0 con 0x7f396006a510 2023-05-20T01:17:26.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.575+0000 7f394effd700 1 -- 172.21.15.16:0/1962463937 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+20121 (secure 0 0 0) 0x7f3950017020 con 0x7f396006a510 2023-05-20T01:17:26.579 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:17:26.580 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":27,"fsid":"c2245cc5-ecab-4087-a59b-bc81db901676","created":"2023-05-20T01:16:09.549230+0000","modified":"2023-05-20T01:17:16.406712+0000","last_up_change":"2023-05-20T01:16:19.788624+0000","last_in_change":"2023-05-20T01:16:12.092160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-05-20T01:16:20.934366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-05-20T01:16:21.372294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-05-20T01:16:26.208287+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-05-20T01:16:27.292268+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-05-20T01:16:30.357285+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"317a83cd-19c8-4958-a301-6087ef342289","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6825","nonce":3923634179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6827","nonce":3923634179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6831","nonce":3923634179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6829","nonce":3923634179}]},"public_addr":"172.21.15.16:6825/3923634179","cluster_addr":"172.21.15.16:6827/3923634179","heartbeat_back_addr":"172.21.15.16:6831/3923634179","heartbeat_front_addr":"172.21.15.16:6829/3923634179","state":["exists","up"]},{"osd":1,"uuid":"5581046f-8219-4089-8d7d-b7846399505e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6809","nonce":3860264220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6811","nonce":3860264220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6815","nonce":3860264220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6813","nonce":3860264220}]},"public_addr":"172.21.15.16:6809/3860264220","cluster_addr":"172.21.15.16:6811/3860264220","heartbeat_back_addr":"172.21.15.16:6815/3860264220","heartbeat_front_addr":"172.21.15.16:6813/3860264220","state":["exists","up"]},{"osd":2,"uuid":"072e9e66-4133-478e-8d4a-64d46c2963f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6817","nonce":293187030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6819","nonce":293187030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6823","nonce":293187030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6821","nonce":293187030}]},"public_addr":"172.21.15.16:6817/293187030","cluster_addr":"172.21.15.16:6819/293187030","heartbeat_back_addr":"172.21.15.16:6823/293187030","heartbeat_front_addr":"172.21.15.16:6821/293187030","state":["exists","up"]},{"osd":3,"uuid":"11e1bba6-00f8-48dd-9028-d8d716b7f145","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6801","nonce":3669781890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6803","nonce":3669781890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6807","nonce":3669781890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6805","nonce":3669781890}]},"public_addr":"172.21.15.16:6801/3669781890","cluster_addr":"172.21.15.16:6803/3669781890","heartbeat_back_addr":"172.21.15.16:6807/3669781890","heartbeat_front_addr":"172.21.15.16:6805/3669781890","state":["exists","up"]},{"osd":4,"uuid":"b8909c3a-cf70-46cb-8d0b-009c707cf9d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6819","nonce":506317223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6823","nonce":506317223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6829","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6831","nonce":506317223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6827","nonce":506317223}]},"public_addr":"172.21.15.132:6819/506317223","cluster_addr":"172.21.15.132:6823/506317223","heartbeat_back_addr":"172.21.15.132:6831/506317223","heartbeat_front_addr":"172.21.15.132:6827/506317223","state":["exists","up"]},{"osd":5,"uuid":"b1818304-4583-47bf-a6ec-0bfbb5166233","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6818","nonce":1844689995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6822","nonce":1844689995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6830","nonce":1844689995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6826","nonce":1844689995}]},"public_addr":"172.21.15.132:6818/1844689995","cluster_addr":"172.21.15.132:6822/1844689995","heartbeat_back_addr":"172.21.15.132:6830/1844689995","heartbeat_front_addr":"172.21.15.132:6826/1844689995","state":["exists","up"]},{"osd":6,"uuid":"d37f4691-eed1-4a93-8818-18354f03f4a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6801","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6802","nonce":727581123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6805","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6807","nonce":727581123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6813","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6815","nonce":727581123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6811","nonce":727581123}]},"public_addr":"172.21.15.132:6802/727581123","cluster_addr":"172.21.15.132:6807/727581123","heartbeat_back_addr":"172.21.15.132:6815/727581123","heartbeat_front_addr":"172.21.15.132:6811/727581123","state":["exists","up"]},{"osd":7,"uuid":"4f712f62-5d1d-4816-8042-03209144f8b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6803","nonce":466001149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6806","nonce":466001149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6814","nonce":466001149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6810","nonce":466001149}]},"public_addr":"172.21.15.132:6803/466001149","cluster_addr":"172.21.15.132:6806/466001149","heartbeat_back_addr":"172.21.15.132:6814/466001149","heartbeat_front_addr":"172.21.15.132:6810/466001149","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.338863+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.278680+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305694+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.246835+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.276237+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.304441+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305421+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.267050+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.a","mappings":[{"from":1,"to":5}]},{"pgid":"4.11","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-20T01:17:26.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.577+0000 7f3964f5f700 1 -- 172.21.15.16:0/1962463937 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f394006ace0 msgr2=0x7f394006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:26.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.577+0000 7f3964f5f700 1 --2- 172.21.15.16:0/1962463937 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f394006ace0 0x7f394006d190 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3954000c00 tx=0x7f3954008030 comp rx=0 tx=0).stop 2023-05-20T01:17:26.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.577+0000 7f3964f5f700 1 -- 172.21.15.16:0/1962463937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f396006a510 msgr2=0x7f39600b57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:26.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.577+0000 7f3964f5f700 1 --2- 172.21.15.16:0/1962463937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f396006a510 0x7f39600b57b0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f3950001eb0 tx=0x7f395000c7b0 comp rx=0 tx=0).stop 2023-05-20T01:17:26.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.577+0000 7f3964f5f700 1 -- 172.21.15.16:0/1962463937 shutdown_connections 2023-05-20T01:17:26.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.577+0000 7f3964f5f700 1 --2- 172.21.15.16:0/1962463937 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f394006ace0 0x7f394006d190 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:26.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.577+0000 7f3964f5f700 1 --2- 172.21.15.16:0/1962463937 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f396006f450 0x7f39600c90e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:26.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.577+0000 7f3964f5f700 1 --2- 172.21.15.16:0/1962463937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f396006a510 0x7f39600b57b0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:26.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.577+0000 7f3964f5f700 1 --2- 172.21.15.16:0/1962463937 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f396006d4b0 0x7f3960153940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:26.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.577+0000 7f3964f5f700 1 -- 172.21.15.16:0/1962463937 >> 172.21.15.16:0/1962463937 conn(0x7f39600cffb0 msgr2=0x7f39600c7720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:26.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.577+0000 7f3964f5f700 1 -- 172.21.15.16:0/1962463937 shutdown_connections 2023-05-20T01:17:26.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.577+0000 7f3964f5f700 1 -- 172.21.15.16:0/1962463937 wait complete. 2023-05-20T01:17:26.593 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-05-20T01:16:20.934366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-05-20T01:16:21.372294+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-05-20T01:16:26.208287+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-05-20T01:16:27.292268+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-05-20T01:16:30.357285+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-05-20T01:17:26.594 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-05-20T01:17:26.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.719+0000 7fc7efa1f700 1 Processor -- start 2023-05-20T01:17:26.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.719+0000 7fc7efa1f700 1 -- start start 2023-05-20T01:17:26.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.720+0000 7fc7efa1f700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc7e815cec0 0x7fc7e8071bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:26.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.720+0000 7fc7efa1f700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7e8072590 con 0x7fc7e8072130 2023-05-20T01:17:26.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.720+0000 7fc7efa1f700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7e80cd630 con 0x7fc7e815d290 2023-05-20T01:17:26.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.720+0000 7fc7efa1f700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7e80cca60 con 0x7fc7e815cec0 2023-05-20T01:17:26.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.720+0000 7fc7edfbc700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fc7e8072130 0x7fc7e8160890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:42804/0 (socket says 172.21.15.16:42804) 2023-05-20T01:17:26.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.720+0000 7fc7edfbc700 1 -- 172.21.15.16:0/2523855336 learned_addr learned my addr 172.21.15.16:0/2523855336 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:17:26.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.720+0000 7fc7ed7bb700 1 --2- 172.21.15.16:0/2523855336 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc7e815cec0 0x7fc7e8071bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:26.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.720+0000 7fc7dffff700 1 -- 172.21.15.16:0/2523855336 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3903099032 0 0) 0x7fc7e8072590 con 0x7fc7e8072130 2023-05-20T01:17:26.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.720+0000 7fc7dffff700 1 -- 172.21.15.16:0/2523855336 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7d40035b0 con 0x7fc7e8072130 2023-05-20T01:17:26.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.720+0000 7fc7dffff700 1 -- 172.21.15.16:0/2523855336 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3550693445 0 0) 0x7fc7e80cd630 con 0x7fc7e815d290 2023-05-20T01:17:26.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.720+0000 7fc7dffff700 1 -- 172.21.15.16:0/2523855336 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7e8072590 con 0x7fc7e815d290 2023-05-20T01:17:26.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.720+0000 7fc7dffff700 1 -- 172.21.15.16:0/2523855336 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2088800778 0 0) 0x7fc7d40035b0 con 0x7fc7e8072130 2023-05-20T01:17:26.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.720+0000 7fc7ed7bb700 1 -- 172.21.15.16:0/2523855336 >> v1:172.21.15.132:6789/0 conn(0x7fc7e815d290 legacy=0x7fc7e80cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:26.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.720+0000 7fc7ed7bb700 1 -- 172.21.15.16:0/2523855336 >> v1:172.21.15.16:6789/0 conn(0x7fc7e8072130 legacy=0x7fc7e8160890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:26.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.720+0000 7fc7ed7bb700 1 -- 172.21.15.16:0/2523855336 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7e80ccd00 con 0x7fc7e815cec0 2023-05-20T01:17:26.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.721+0000 7fc7ed7bb700 1 --2- 172.21.15.16:0/2523855336 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc7e815cec0 0x7fc7e8071bf0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc7e4009850 tx=0x7fc7e4009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e465de482634d53a server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:26.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.721+0000 7fc7dffff700 1 -- 172.21.15.16:0/2523855336 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7e0001970 con 0x7fc7e815cec0 2023-05-20T01:17:26.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.721+0000 7fc7dffff700 1 -- 172.21.15.16:0/2523855336 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7e4005c30 con 0x7fc7e815cec0 2023-05-20T01:17:26.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.721+0000 7fc7dffff700 1 -- 172.21.15.16:0/2523855336 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7e4004030 con 0x7fc7e815cec0 2023-05-20T01:17:26.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.721+0000 7fc7efa1f700 1 -- 172.21.15.16:0/2523855336 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc7e815cec0 msgr2=0x7fc7e8071bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:26.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.721+0000 7fc7efa1f700 1 --2- 172.21.15.16:0/2523855336 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc7e815cec0 0x7fc7e8071bf0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc7e4009850 tx=0x7fc7e4009b60 comp rx=0 tx=0).stop 2023-05-20T01:17:26.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.721+0000 7fc7efa1f700 1 -- 172.21.15.16:0/2523855336 shutdown_connections 2023-05-20T01:17:26.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.721+0000 7fc7efa1f700 1 --2- 172.21.15.16:0/2523855336 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc7e815cec0 0x7fc7e8071bf0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:26.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.721+0000 7fc7efa1f700 1 -- 172.21.15.16:0/2523855336 >> 172.21.15.16:0/2523855336 conn(0x7fc7e80cffd0 msgr2=0x7fc7e80d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:26.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.721+0000 7fc7efa1f700 1 -- 172.21.15.16:0/2523855336 shutdown_connections 2023-05-20T01:17:26.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.722+0000 7fc7efa1f700 1 -- 172.21.15.16:0/2523855336 wait complete. 2023-05-20T01:17:26.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.723+0000 7fc7efa1f700 1 Processor -- start 2023-05-20T01:17:26.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.723+0000 7fc7efa1f700 1 -- start start 2023-05-20T01:17:26.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.723+0000 7fc7efa1f700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc7e8072130 0x7fc7e81535a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:26.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.723+0000 7fc7efa1f700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc7e815cec0 0x7fc7e80b2e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:26.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.723+0000 7fc7efa1f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7e815d290 0x7fc7e80cb810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:26.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.723+0000 7fc7efa1f700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7e80c7fe0 con 0x7fc7e815d290 2023-05-20T01:17:26.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.723+0000 7fc7efa1f700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7e80c7cd0 con 0x7fc7e815cec0 2023-05-20T01:17:26.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.723+0000 7fc7efa1f700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7e80b3740 con 0x7fc7e8072130 2023-05-20T01:17:26.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.724+0000 7fc7edfbc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7e815d290 0x7fc7e80cb810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:26.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.724+0000 7fc7edfbc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7e815d290 0x7fc7e80cb810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59514/0 (socket says 172.21.15.16:59514) 2023-05-20T01:17:26.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.724+0000 7fc7edfbc700 1 -- 172.21.15.16:0/91690226 learned_addr learned my addr 172.21.15.16:0/91690226 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:26.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.724+0000 7fc7ed7bb700 1 --2- 172.21.15.16:0/91690226 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc7e8072130 0x7fc7e81535a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:26.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.724+0000 7fc7ecfba700 1 --2- 172.21.15.16:0/91690226 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc7e815cec0 0x7fc7e80b2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:26.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.724+0000 7fc7edfbc700 1 -- 172.21.15.16:0/91690226 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc7e8072130 msgr2=0x7fc7e81535a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:26.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.724+0000 7fc7edfbc700 1 --2- 172.21.15.16:0/91690226 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc7e8072130 0x7fc7e81535a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:26.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.724+0000 7fc7edfbc700 1 -- 172.21.15.16:0/91690226 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc7e815cec0 msgr2=0x7fc7e80b2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:26.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.724+0000 7fc7edfbc700 1 --2- 172.21.15.16:0/91690226 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc7e815cec0 0x7fc7e80b2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:26.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.724+0000 7fc7edfbc700 1 -- 172.21.15.16:0/91690226 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7e4009430 con 0x7fc7e815d290 2023-05-20T01:17:26.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.724+0000 7fc7edfbc700 1 --2- 172.21.15.16:0/91690226 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7e815d290 0x7fc7e80cb810 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fc7e000ccc0 tx=0x7fc7e000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:26.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.724+0000 7fc7ddffb700 1 -- 172.21.15.16:0/91690226 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7e000abb0 con 0x7fc7e815d290 2023-05-20T01:17:26.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.724+0000 7fc7ddffb700 1 -- 172.21.15.16:0/91690226 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7e000ad10 con 0x7fc7e815d290 2023-05-20T01:17:26.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.724+0000 7fc7efa1f700 1 -- 172.21.15.16:0/91690226 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7e80cd2c0 con 0x7fc7e815d290 2023-05-20T01:17:26.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.724+0000 7fc7ddffb700 1 -- 172.21.15.16:0/91690226 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7e0007310 con 0x7fc7e815d290 2023-05-20T01:17:26.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.724+0000 7fc7efa1f700 1 -- 172.21.15.16:0/91690226 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7e80ccfe0 con 0x7fc7e815d290 2023-05-20T01:17:26.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.725+0000 7fc7efa1f700 1 -- 172.21.15.16:0/91690226 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7e80cf9d0 con 0x7fc7e815d290 2023-05-20T01:17:26.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.726+0000 7fc7ddffb700 1 -- 172.21.15.16:0/91690226 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc7e0007470 con 0x7fc7e815d290 2023-05-20T01:17:26.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.726+0000 7fc7ddffb700 1 --2- 172.21.15.16:0/91690226 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc7cc06ace0 0x7fc7cc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:26.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.726+0000 7fc7ddffb700 1 -- 172.21.15.16:0/91690226 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fc7e00896e0 con 0x7fc7e815d290 2023-05-20T01:17:26.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.726+0000 7fc7ed7bb700 1 --2- 172.21.15.16:0/91690226 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc7cc06ace0 0x7fc7cc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:26.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.727+0000 7fc7ed7bb700 1 --2- 172.21.15.16:0/91690226 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc7cc06ace0 0x7fc7cc06d190 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc7e40097a0 tx=0x7fc7e4008070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:26.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.731+0000 7fc7ddffb700 1 -- 172.21.15.16:0/91690226 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fc7e0054460 con 0x7fc7e815d290 2023-05-20T01:17:26.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.928+0000 7fc7efa1f700 1 -- 172.21.15.16:0/91690226 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fc7e80c8190 con 0x7fc7e815d290 2023-05-20T01:17:26.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.929+0000 7fc7ddffb700 1 -- 172.21.15.16:0/91690226 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v27) v1 ==== 92+0+10 (secure 0 0 0) 0x7fc7e0018020 con 0x7fc7e815d290 2023-05-20T01:17:26.933 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 8 2023-05-20T01:17:26.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.931+0000 7fc7efa1f700 1 -- 172.21.15.16:0/91690226 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc7cc06ace0 msgr2=0x7fc7cc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:26.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.931+0000 7fc7efa1f700 1 --2- 172.21.15.16:0/91690226 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc7cc06ace0 0x7fc7cc06d190 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc7e40097a0 tx=0x7fc7e4008070 comp rx=0 tx=0).stop 2023-05-20T01:17:26.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.931+0000 7fc7efa1f700 1 -- 172.21.15.16:0/91690226 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7e815d290 msgr2=0x7fc7e80cb810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:26.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.931+0000 7fc7efa1f700 1 --2- 172.21.15.16:0/91690226 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7e815d290 0x7fc7e80cb810 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fc7e000ccc0 tx=0x7fc7e000a060 comp rx=0 tx=0).stop 2023-05-20T01:17:26.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.931+0000 7fc7efa1f700 1 -- 172.21.15.16:0/91690226 shutdown_connections 2023-05-20T01:17:26.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.931+0000 7fc7efa1f700 1 --2- 172.21.15.16:0/91690226 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7e815d290 0x7fc7e80cb810 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:26.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.931+0000 7fc7efa1f700 1 --2- 172.21.15.16:0/91690226 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc7cc06ace0 0x7fc7cc06d190 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:26.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.931+0000 7fc7efa1f700 1 --2- 172.21.15.16:0/91690226 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc7e8072130 0x7fc7e81535a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:26.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.931+0000 7fc7efa1f700 1 --2- 172.21.15.16:0/91690226 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc7e815cec0 0x7fc7e80b2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:26.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.931+0000 7fc7efa1f700 1 -- 172.21.15.16:0/91690226 >> 172.21.15.16:0/91690226 conn(0x7fc7e80cffd0 msgr2=0x7fc7e80b2340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:26.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.931+0000 7fc7efa1f700 1 -- 172.21.15.16:0/91690226 shutdown_connections 2023-05-20T01:17:26.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:26.931+0000 7fc7efa1f700 1 -- 172.21.15.16:0/91690226 wait complete. 2023-05-20T01:17:26.946 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-05-20T01:17:27.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.070+0000 7f7ce809c700 1 Processor -- start 2023-05-20T01:17:27.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.071+0000 7f7ce809c700 1 -- start start 2023-05-20T01:17:27.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.071+0000 7f7ce809c700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f7ce00cd1c0 0x7f7ce00710b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:27.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.071+0000 7f7ce809c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7ce00cd630 0x7f7ce00715f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:27.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.071+0000 7f7ce809c700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ce00ccd30 con 0x7f7ce0159b50 2023-05-20T01:17:27.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.071+0000 7f7ce809c700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ce00ce990 con 0x7f7ce00cd1c0 2023-05-20T01:17:27.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.071+0000 7f7ce809c700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce00cc6e0 con 0x7f7ce00cd630 2023-05-20T01:17:27.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.071+0000 7f7ce5e38700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f7ce0159b50 0x7f7ce006e0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:42814/0 (socket says 172.21.15.16:42814) 2023-05-20T01:17:27.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.071+0000 7f7ce5e38700 1 -- 172.21.15.16:0/653169799 learned_addr learned my addr 172.21.15.16:0/653169799 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:17:27.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.072+0000 7f7ce5637700 1 --2- 172.21.15.16:0/653169799 >> v2:172.21.15.132:3300/0 conn(0x7f7ce00cd1c0 0x7f7ce00710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:27.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.072+0000 7f7ce6639700 1 --2- 172.21.15.16:0/653169799 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7ce00cd630 0x7f7ce00715f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:27.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.072+0000 7f7ce4e36700 1 -- 172.21.15.16:0/653169799 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1843270424 0 0) 0x7f7ce00ccd30 con 0x7f7ce0159b50 2023-05-20T01:17:27.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.072+0000 7f7ce4e36700 1 -- 172.21.15.16:0/653169799 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ccc0035b0 con 0x7f7ce0159b50 2023-05-20T01:17:27.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.072+0000 7f7ce6639700 1 -- 172.21.15.16:0/653169799 >> v2:172.21.15.132:3300/0 conn(0x7f7ce00cd1c0 msgr2=0x7f7ce00710b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:27.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.072+0000 7f7ce6639700 1 --2- 172.21.15.16:0/653169799 >> v2:172.21.15.132:3300/0 conn(0x7f7ce00cd1c0 0x7f7ce00710b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:27.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.072+0000 7f7ce6639700 1 -- 172.21.15.16:0/653169799 >> v1:172.21.15.16:6789/0 conn(0x7f7ce0159b50 legacy=0x7f7ce006e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:27.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.072+0000 7f7ce6639700 1 -- 172.21.15.16:0/653169799 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ce00cc9c0 con 0x7f7ce00cd630 2023-05-20T01:17:27.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.072+0000 7f7ce6639700 1 --2- 172.21.15.16:0/653169799 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7ce00cd630 0x7f7ce00715f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7cd8009d90 tx=0x7f7cd800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ec945ba24de61b5 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:27.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.072+0000 7f7ce4e36700 1 -- 172.21.15.16:0/653169799 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cd800e070 con 0x7f7ce00cd630 2023-05-20T01:17:27.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.072+0000 7f7ce4e36700 1 -- 172.21.15.16:0/653169799 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cd800bb20 con 0x7f7ce00cd630 2023-05-20T01:17:27.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.072+0000 7f7ce4e36700 1 -- 172.21.15.16:0/653169799 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cd800be90 con 0x7f7ce00cd630 2023-05-20T01:17:27.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.073+0000 7f7ce809c700 1 -- 172.21.15.16:0/653169799 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7ce00cd630 msgr2=0x7f7ce00715f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:27.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.073+0000 7f7ce809c700 1 --2- 172.21.15.16:0/653169799 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7ce00cd630 0x7f7ce00715f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7cd8009d90 tx=0x7f7cd800b330 comp rx=0 tx=0).stop 2023-05-20T01:17:27.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.073+0000 7f7ce809c700 1 -- 172.21.15.16:0/653169799 shutdown_connections 2023-05-20T01:17:27.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.073+0000 7f7ce809c700 1 --2- 172.21.15.16:0/653169799 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7ce00cd630 0x7f7ce00715f0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:27.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.073+0000 7f7ce809c700 1 --2- 172.21.15.16:0/653169799 >> v2:172.21.15.132:3300/0 conn(0x7f7ce00cd1c0 0x7f7ce00710b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:27.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.073+0000 7f7ce809c700 1 -- 172.21.15.16:0/653169799 >> 172.21.15.16:0/653169799 conn(0x7f7ce00cffd0 msgr2=0x7f7ce00d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:27.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.073+0000 7f7ce809c700 1 -- 172.21.15.16:0/653169799 shutdown_connections 2023-05-20T01:17:27.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.073+0000 7f7ce809c700 1 -- 172.21.15.16:0/653169799 wait complete. 2023-05-20T01:17:27.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.074+0000 7f7ce809c700 1 Processor -- start 2023-05-20T01:17:27.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.074+0000 7f7ce809c700 1 -- start start 2023-05-20T01:17:27.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.075+0000 7f7ce809c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7ce00cd1c0 0x7f7ce00c80f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:27.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.075+0000 7f7ce809c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ce00cd630 0x7f7ce01ef380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:27.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.075+0000 7f7ce809c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7ce0159b50 0x7f7ce01ef8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:27.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.075+0000 7f7ce809c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce00c9ae0 con 0x7f7ce00cd630 2023-05-20T01:17:27.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.075+0000 7f7ce809c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce00c36e0 con 0x7f7ce0159b50 2023-05-20T01:17:27.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.075+0000 7f7ce809c700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce00c8a40 con 0x7f7ce00cd1c0 2023-05-20T01:17:27.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.075+0000 7f7ce5637700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ce00cd630 0x7f7ce01ef380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:27.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.075+0000 7f7ce5637700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ce00cd630 0x7f7ce01ef380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59526/0 (socket says 172.21.15.16:59526) 2023-05-20T01:17:27.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.075+0000 7f7ce5637700 1 -- 172.21.15.16:0/187647682 learned_addr learned my addr 172.21.15.16:0/187647682 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:27.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.075+0000 7f7ce5e38700 1 --2- 172.21.15.16:0/187647682 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7ce00cd1c0 0x7f7ce00c80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:27.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.075+0000 7f7ce6639700 1 --2- 172.21.15.16:0/187647682 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7ce0159b50 0x7f7ce01ef8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:27.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.075+0000 7f7ce5637700 1 -- 172.21.15.16:0/187647682 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7ce00cd1c0 msgr2=0x7f7ce00c80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:27.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.075+0000 7f7ce5637700 1 --2- 172.21.15.16:0/187647682 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7ce00cd1c0 0x7f7ce00c80f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:27.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.075+0000 7f7ce5637700 1 -- 172.21.15.16:0/187647682 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7ce0159b50 msgr2=0x7f7ce01ef8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:27.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.075+0000 7f7ce5637700 1 --2- 172.21.15.16:0/187647682 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7ce0159b50 0x7f7ce01ef8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:27.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.075+0000 7f7ce5637700 1 -- 172.21.15.16:0/187647682 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cd8009a70 con 0x7f7ce00cd630 2023-05-20T01:17:27.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.076+0000 7f7ce5637700 1 --2- 172.21.15.16:0/187647682 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ce00cd630 0x7f7ce01ef380 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f7cdc00e9b0 tx=0x7f7cdc00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:27.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.076+0000 7f7cd67fc700 1 -- 172.21.15.16:0/187647682 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cdc009710 con 0x7f7ce00cd630 2023-05-20T01:17:27.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.076+0000 7f7cd67fc700 1 -- 172.21.15.16:0/187647682 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cdc00c850 con 0x7f7ce00cd630 2023-05-20T01:17:27.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.076+0000 7f7ce809c700 1 -- 172.21.15.16:0/187647682 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ce00cec70 con 0x7f7ce00cd630 2023-05-20T01:17:27.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.076+0000 7f7cd67fc700 1 -- 172.21.15.16:0/187647682 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cdc00cbc0 con 0x7f7ce00cd630 2023-05-20T01:17:27.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.076+0000 7f7ce809c700 1 -- 172.21.15.16:0/187647682 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ce0071e80 con 0x7f7ce00cd630 2023-05-20T01:17:27.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.077+0000 7f7ce809c700 1 -- 172.21.15.16:0/187647682 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ce00747b0 con 0x7f7ce00cd630 2023-05-20T01:17:27.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.077+0000 7f7cd67fc700 1 -- 172.21.15.16:0/187647682 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7cdc00cd20 con 0x7f7ce00cd630 2023-05-20T01:17:27.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.077+0000 7f7cd67fc700 1 --2- 172.21.15.16:0/187647682 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f7cc406ace0 0x7f7cc406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:27.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.078+0000 7f7cd67fc700 1 -- 172.21.15.16:0/187647682 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f7cdc0898b0 con 0x7f7ce00cd630 2023-05-20T01:17:27.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.078+0000 7f7ce5e38700 1 --2- 172.21.15.16:0/187647682 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f7cc406ace0 0x7f7cc406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:27.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.078+0000 7f7ce5e38700 1 --2- 172.21.15.16:0/187647682 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f7cc406ace0 0x7f7cc406d190 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7cd0007fd0 tx=0x7f7cd0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:27.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.082+0000 7f7cd67fc700 1 -- 172.21.15.16:0/187647682 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f7cdc0580d0 con 0x7f7ce00cd630 2023-05-20T01:17:27.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.277+0000 7f7ce809c700 1 -- 172.21.15.16:0/187647682 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7ce00cdcd0 con 0x7f7ce00cd630 2023-05-20T01:17:27.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.278+0000 7f7cd67fc700 1 -- 172.21.15.16:0/187647682 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7cdc057c60 con 0x7f7ce00cd630 2023-05-20T01:17:27.281 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-05-20T01:17:27.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.279+0000 7f7ce809c700 1 -- 172.21.15.16:0/187647682 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f7cc406ace0 msgr2=0x7f7cc406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:27.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.279+0000 7f7ce809c700 1 --2- 172.21.15.16:0/187647682 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f7cc406ace0 0x7f7cc406d190 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7cd0007fd0 tx=0x7f7cd0009040 comp rx=0 tx=0).stop 2023-05-20T01:17:27.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.280+0000 7f7ce809c700 1 -- 172.21.15.16:0/187647682 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ce00cd630 msgr2=0x7f7ce01ef380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:27.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.280+0000 7f7ce809c700 1 --2- 172.21.15.16:0/187647682 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ce00cd630 0x7f7ce01ef380 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f7cdc00e9b0 tx=0x7f7cdc00ed70 comp rx=0 tx=0).stop 2023-05-20T01:17:27.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.280+0000 7f7ce809c700 1 -- 172.21.15.16:0/187647682 shutdown_connections 2023-05-20T01:17:27.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.280+0000 7f7ce809c700 1 --2- 172.21.15.16:0/187647682 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7ce0159b50 0x7f7ce01ef8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:27.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.280+0000 7f7ce809c700 1 --2- 172.21.15.16:0/187647682 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f7cc406ace0 0x7f7cc406d190 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:27.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.280+0000 7f7ce809c700 1 --2- 172.21.15.16:0/187647682 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7ce00cd1c0 0x7f7ce00c80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:27.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.280+0000 7f7ce809c700 1 --2- 172.21.15.16:0/187647682 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ce00cd630 0x7f7ce01ef380 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:27.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.280+0000 7f7ce809c700 1 -- 172.21.15.16:0/187647682 >> 172.21.15.16:0/187647682 conn(0x7f7ce00cffd0 msgr2=0x7f7ce00ca830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:27.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.280+0000 7f7ce809c700 1 -- 172.21.15.16:0/187647682 shutdown_connections 2023-05-20T01:17:27.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.280+0000 7f7ce809c700 1 -- 172.21.15.16:0/187647682 wait complete. 2023-05-20T01:17:27.295 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-05-20T01:17:27.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.421+0000 7fb67b0b2700 1 Processor -- start 2023-05-20T01:17:27.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.421+0000 7fb67b0b2700 1 -- start start 2023-05-20T01:17:27.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.421+0000 7fb67b0b2700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fb674159b50 0x7fb67406e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:27.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.421+0000 7fb67b0b2700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb6740717c0 0x7fb6740ca870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:27.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.421+0000 7fb67b0b2700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6740ccd30 con 0x7fb6740cd1c0 2023-05-20T01:17:27.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.421+0000 7fb67b0b2700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6740ce990 con 0x7fb674159b50 2023-05-20T01:17:27.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.421+0000 7fb67b0b2700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6740cc6e0 con 0x7fb6740717c0 2023-05-20T01:17:27.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.422+0000 7fb673fff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fb6740cd1c0 0x7fb6740710b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:42830/0 (socket says 172.21.15.16:42830) 2023-05-20T01:17:27.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.422+0000 7fb673fff700 1 -- 172.21.15.16:0/796764300 learned_addr learned my addr 172.21.15.16:0/796764300 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:17:27.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.422+0000 7fb678e4e700 1 --2- 172.21.15.16:0/796764300 >> v2:172.21.15.132:3300/0 conn(0x7fb674159b50 0x7fb67406e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:27.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.422+0000 7fb67964f700 1 --2- 172.21.15.16:0/796764300 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb6740717c0 0x7fb6740ca870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:27.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.422+0000 7fb6737fe700 1 -- 172.21.15.16:0/796764300 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1158774837 0 0) 0x7fb6740ccd30 con 0x7fb6740cd1c0 2023-05-20T01:17:27.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.422+0000 7fb6737fe700 1 -- 172.21.15.16:0/796764300 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb6640035b0 con 0x7fb6740cd1c0 2023-05-20T01:17:27.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.422+0000 7fb6737fe700 1 -- 172.21.15.16:0/796764300 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2622711408 0 0) 0x7fb6640035b0 con 0x7fb6740cd1c0 2023-05-20T01:17:27.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.422+0000 7fb6737fe700 1 -- 172.21.15.16:0/796764300 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb6740717c0 msgr2=0x7fb6740ca870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:27.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.422+0000 7fb6737fe700 1 --2- 172.21.15.16:0/796764300 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb6740717c0 0x7fb6740ca870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:27.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.422+0000 7fb6737fe700 1 -- 172.21.15.16:0/796764300 >> v2:172.21.15.132:3300/0 conn(0x7fb674159b50 msgr2=0x7fb67406e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:27.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.422+0000 7fb6737fe700 1 --2- 172.21.15.16:0/796764300 >> v2:172.21.15.132:3300/0 conn(0x7fb674159b50 0x7fb67406e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:27.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.422+0000 7fb6737fe700 1 -- 172.21.15.16:0/796764300 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6740cc9c0 con 0x7fb6740cd1c0 2023-05-20T01:17:27.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.422+0000 7fb6737fe700 1 -- 172.21.15.16:0/796764300 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7fb660001670 con 0x7fb6740cd1c0 2023-05-20T01:17:27.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.422+0000 7fb6737fe700 1 -- 172.21.15.16:0/796764300 >> v1:172.21.15.16:6789/0 conn(0x7fb6740cd1c0 legacy=0x7fb6740710b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:27.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.422+0000 7fb6737fe700 1 --2- 172.21.15.16:0/796764300 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb6740717c0 0x7fb664003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:27.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.422+0000 7fb6737fe700 1 --2- 172.21.15.16:0/796764300 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb664003f80 0x7fb664004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:27.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.422+0000 7fb6737fe700 1 --2- 172.21.15.16:0/796764300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb664004d90 0x7fb664005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:27.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.422+0000 7fb6737fe700 1 -- 172.21.15.16:0/796764300 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6740ce990 con 0x7fb664004d90 2023-05-20T01:17:27.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.422+0000 7fb6737fe700 1 -- 172.21.15.16:0/796764300 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6740cc6e0 con 0x7fb664003f80 2023-05-20T01:17:27.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.422+0000 7fb6737fe700 1 -- 172.21.15.16:0/796764300 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb66400ba10 con 0x7fb6740717c0 2023-05-20T01:17:27.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.422+0000 7fb67964f700 1 --2- 172.21.15.16:0/796764300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb664004d90 0x7fb664005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:27.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.422+0000 7fb678e4e700 1 --2- 172.21.15.16:0/796764300 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb6740717c0 0x7fb664003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:27.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.423+0000 7fb673fff700 1 --2- 172.21.15.16:0/796764300 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb664003f80 0x7fb664004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:27.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.423+0000 7fb67964f700 1 -- 172.21.15.16:0/796764300 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb6740717c0 msgr2=0x7fb664003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:27.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.423+0000 7fb67964f700 1 --2- 172.21.15.16:0/796764300 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb6740717c0 0x7fb664003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:27.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.423+0000 7fb67964f700 1 -- 172.21.15.16:0/796764300 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb664003f80 msgr2=0x7fb664004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:27.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.423+0000 7fb67964f700 1 --2- 172.21.15.16:0/796764300 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb664003f80 0x7fb664004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:27.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.423+0000 7fb67964f700 1 -- 172.21.15.16:0/796764300 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6640035b0 con 0x7fb664004d90 2023-05-20T01:17:27.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.423+0000 7fb678e4e700 1 --2- 172.21.15.16:0/796764300 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb6740717c0 0x7fb664003a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:17:27.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.423+0000 7fb67964f700 1 --2- 172.21.15.16:0/796764300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb664004d90 0x7fb664005200 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fb66c009f90 tx=0x7fb664005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b0cf82f54c15fd0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:27.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.423+0000 7fb6737fe700 1 -- 172.21.15.16:0/796764300 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb66c00ec30 con 0x7fb664004d90 2023-05-20T01:17:27.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.423+0000 7fb6737fe700 1 -- 172.21.15.16:0/796764300 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb66c00f040 con 0x7fb664004d90 2023-05-20T01:17:27.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.423+0000 7fb6737fe700 1 -- 172.21.15.16:0/796764300 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb66c014480 con 0x7fb664004d90 2023-05-20T01:17:27.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.423+0000 7fb67b0b2700 1 -- 172.21.15.16:0/796764300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb664004d90 msgr2=0x7fb664005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:27.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.423+0000 7fb67b0b2700 1 --2- 172.21.15.16:0/796764300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb664004d90 0x7fb664005200 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fb66c009f90 tx=0x7fb664005880 comp rx=0 tx=0).stop 2023-05-20T01:17:27.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.423+0000 7fb67964f700 1 -- 172.21.15.16:0/796764300 reap_dead start 2023-05-20T01:17:27.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.423+0000 7fb67b0b2700 1 -- 172.21.15.16:0/796764300 shutdown_connections 2023-05-20T01:17:27.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.423+0000 7fb67b0b2700 1 -- 172.21.15.16:0/796764300 >> 172.21.15.16:0/796764300 conn(0x7fb6740cffd0 msgr2=0x7fb6740d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:27.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.424+0000 7fb67b0b2700 1 -- 172.21.15.16:0/796764300 shutdown_connections 2023-05-20T01:17:27.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.424+0000 7fb67b0b2700 1 -- 172.21.15.16:0/796764300 wait complete. 2023-05-20T01:17:27.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.425+0000 7fb67b0b2700 1 Processor -- start 2023-05-20T01:17:27.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.425+0000 7fb67b0b2700 1 -- start start 2023-05-20T01:17:27.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.426+0000 7fb67b0b2700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb6741e4a20 0x7fb674148580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:27.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.426+0000 7fb67b0b2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb67415c200 0x7fb6740c9430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:27.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.426+0000 7fb67b0b2700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb674159f80 0x7fb67415a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:27.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.426+0000 7fb67b0b2700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb67415a940 con 0x7fb67415c200 2023-05-20T01:17:27.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.426+0000 7fb67b0b2700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb674148ac0 con 0x7fb674159f80 2023-05-20T01:17:27.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.426+0000 7fb67b0b2700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6740c80f0 con 0x7fb6741e4a20 2023-05-20T01:17:27.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.426+0000 7fb673fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb67415c200 0x7fb6740c9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:27.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.426+0000 7fb673fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb67415c200 0x7fb6740c9430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59546/0 (socket says 172.21.15.16:59546) 2023-05-20T01:17:27.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.426+0000 7fb673fff700 1 -- 172.21.15.16:0/2370369839 learned_addr learned my addr 172.21.15.16:0/2370369839 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:27.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.426+0000 7fb67964f700 1 --2- 172.21.15.16:0/2370369839 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb674159f80 0x7fb67415a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:27.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.426+0000 7fb673fff700 1 -- 172.21.15.16:0/2370369839 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb6741e4a20 msgr2=0x7fb674148580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:27.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.426+0000 7fb678e4e700 1 --2- 172.21.15.16:0/2370369839 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb6741e4a20 0x7fb674148580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:27.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.426+0000 7fb673fff700 1 --2- 172.21.15.16:0/2370369839 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb6741e4a20 0x7fb674148580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:27.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.426+0000 7fb673fff700 1 -- 172.21.15.16:0/2370369839 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb674159f80 msgr2=0x7fb67415a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:27.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.426+0000 7fb673fff700 1 --2- 172.21.15.16:0/2370369839 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb674159f80 0x7fb67415a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:27.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.426+0000 7fb673fff700 1 -- 172.21.15.16:0/2370369839 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb674071e80 con 0x7fb67415c200 2023-05-20T01:17:27.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.426+0000 7fb67964f700 1 --2- 172.21.15.16:0/2370369839 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb674159f80 0x7fb67415a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:17:27.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.426+0000 7fb673fff700 1 --2- 172.21.15.16:0/2370369839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb67415c200 0x7fb6740c9430 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fb660000fb0 tx=0x7fb66000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:27.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.426+0000 7fb6717fa700 1 -- 172.21.15.16:0/2370369839 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb660001bc0 con 0x7fb67415c200 2023-05-20T01:17:27.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.427+0000 7fb6717fa700 1 -- 172.21.15.16:0/2370369839 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb66000c780 con 0x7fb67415c200 2023-05-20T01:17:27.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.427+0000 7fb6717fa700 1 -- 172.21.15.16:0/2370369839 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb66000cb30 con 0x7fb67415c200 2023-05-20T01:17:27.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.427+0000 7fb67b0b2700 1 -- 172.21.15.16:0/2370369839 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb67406e5e0 con 0x7fb67415c200 2023-05-20T01:17:27.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.427+0000 7fb67b0b2700 1 -- 172.21.15.16:0/2370369839 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6740cec70 con 0x7fb67415c200 2023-05-20T01:17:27.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.427+0000 7fb67b0b2700 1 -- 172.21.15.16:0/2370369839 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6740747b0 con 0x7fb67415c200 2023-05-20T01:17:27.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.428+0000 7fb6717fa700 1 -- 172.21.15.16:0/2370369839 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb66000cc90 con 0x7fb67415c200 2023-05-20T01:17:27.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.428+0000 7fb6717fa700 1 --2- 172.21.15.16:0/2370369839 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb65806ad10 0x7fb65806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:27.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.429+0000 7fb6717fa700 1 -- 172.21.15.16:0/2370369839 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fb660089810 con 0x7fb67415c200 2023-05-20T01:17:27.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.429+0000 7fb678e4e700 1 --2- 172.21.15.16:0/2370369839 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb65806ad10 0x7fb65806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:27.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.429+0000 7fb678e4e700 1 --2- 172.21.15.16:0/2370369839 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb65806ad10 0x7fb65806d1c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb668009a30 tx=0x7fb668008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:27.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.433+0000 7fb6717fa700 1 -- 172.21.15.16:0/2370369839 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fb660054590 con 0x7fb67415c200 2023-05-20T01:17:27.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.631+0000 7fb67b0b2700 1 -- 172.21.15.16:0/2370369839 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb6741e4df0 con 0x7fb67415c200 2023-05-20T01:17:27.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.632+0000 7fb6717fa700 1 -- 172.21.15.16:0/2370369839 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb660057bb0 con 0x7fb67415c200 2023-05-20T01:17:27.636 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-05-20T01:17:27.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.634+0000 7fb67b0b2700 1 -- 172.21.15.16:0/2370369839 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb65806ad10 msgr2=0x7fb65806d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:27.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.634+0000 7fb67b0b2700 1 --2- 172.21.15.16:0/2370369839 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb65806ad10 0x7fb65806d1c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb668009a30 tx=0x7fb668008040 comp rx=0 tx=0).stop 2023-05-20T01:17:27.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.634+0000 7fb67b0b2700 1 -- 172.21.15.16:0/2370369839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb67415c200 msgr2=0x7fb6740c9430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:27.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.634+0000 7fb67b0b2700 1 --2- 172.21.15.16:0/2370369839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb67415c200 0x7fb6740c9430 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fb660000fb0 tx=0x7fb66000ee80 comp rx=0 tx=0).stop 2023-05-20T01:17:27.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.634+0000 7fb67b0b2700 1 -- 172.21.15.16:0/2370369839 shutdown_connections 2023-05-20T01:17:27.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.634+0000 7fb67b0b2700 1 --2- 172.21.15.16:0/2370369839 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb674159f80 0x7fb67415a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:27.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.634+0000 7fb67b0b2700 1 --2- 172.21.15.16:0/2370369839 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb65806ad10 0x7fb65806d1c0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:27.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.634+0000 7fb67b0b2700 1 --2- 172.21.15.16:0/2370369839 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb6741e4a20 0x7fb674148580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:27.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.634+0000 7fb67b0b2700 1 --2- 172.21.15.16:0/2370369839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb67415c200 0x7fb6740c9430 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:27.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.634+0000 7fb67b0b2700 1 -- 172.21.15.16:0/2370369839 >> 172.21.15.16:0/2370369839 conn(0x7fb6740cffd0 msgr2=0x7fb67406dde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:27.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.634+0000 7fb67b0b2700 1 -- 172.21.15.16:0/2370369839 shutdown_connections 2023-05-20T01:17:27.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.634+0000 7fb67b0b2700 1 -- 172.21.15.16:0/2370369839 wait complete. 2023-05-20T01:17:27.650 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-05-20T01:17:27.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.775+0000 7ff8a337f700 1 Processor -- start 2023-05-20T01:17:27.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.775+0000 7ff8a337f700 1 -- start start 2023-05-20T01:17:27.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.776+0000 7ff8a337f700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7ff89c15ab80 0x7ff89c0cda80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:27.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.776+0000 7ff8a337f700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff89c0cf390 0x7ff89c0cc110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:27.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.776+0000 7ff8a337f700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff89c0ce4d0 con 0x7ff89c0cdfc0 2023-05-20T01:17:27.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.776+0000 7ff8a337f700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff89c0cb700 con 0x7ff89c0cf390 2023-05-20T01:17:27.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.776+0000 7ff8a337f700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff89c0ce7b0 con 0x7ff89c15ab80 2023-05-20T01:17:27.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.776+0000 7ff8a191c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff89c0cf390 0x7ff89c0cc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:27.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.776+0000 7ff8a091a700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ff89c0cdfc0 0x7ff89c0cec80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:42848/0 (socket says 172.21.15.16:42848) 2023-05-20T01:17:27.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.776+0000 7ff8a091a700 1 -- 172.21.15.16:0/2296052407 learned_addr learned my addr 172.21.15.16:0/2296052407 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:17:27.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.776+0000 7ff8a111b700 1 --2- 172.21.15.16:0/2296052407 >> v2:172.21.15.132:3300/0 conn(0x7ff89c15ab80 0x7ff89c0cda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:27.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.776+0000 7ff8a191c700 1 -- 172.21.15.16:0/2296052407 >> v2:172.21.15.132:3300/0 conn(0x7ff89c15ab80 msgr2=0x7ff89c0cda80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:27.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.776+0000 7ff8a191c700 1 --2- 172.21.15.16:0/2296052407 >> v2:172.21.15.132:3300/0 conn(0x7ff89c15ab80 0x7ff89c0cda80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:27.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.776+0000 7ff8a191c700 1 -- 172.21.15.16:0/2296052407 >> v1:172.21.15.16:6789/0 conn(0x7ff89c0cdfc0 legacy=0x7ff89c0cec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:27.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.776+0000 7ff893fff700 1 -- 172.21.15.16:0/2296052407 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1459865915 0 0) 0x7ff89c0ce4d0 con 0x7ff89c0cdfc0 2023-05-20T01:17:27.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.776+0000 7ff8a191c700 1 -- 172.21.15.16:0/2296052407 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff89c0cb910 con 0x7ff89c0cf390 2023-05-20T01:17:27.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.776+0000 7ff8a111b700 1 --2- 172.21.15.16:0/2296052407 >> v2:172.21.15.132:3300/0 conn(0x7ff89c15ab80 0x7ff89c0cda80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:17:27.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.777+0000 7ff8a191c700 1 --2- 172.21.15.16:0/2296052407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff89c0cf390 0x7ff89c0cc110 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7ff89c0cebd0 tx=0x7ff89400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac17e68c79e547d4 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:27.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.777+0000 7ff893fff700 1 -- 172.21.15.16:0/2296052407 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff89400e070 con 0x7ff89c0cf390 2023-05-20T01:17:27.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.777+0000 7ff893fff700 1 -- 172.21.15.16:0/2296052407 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff89400bbd0 con 0x7ff89c0cf390 2023-05-20T01:17:27.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.777+0000 7ff893fff700 1 -- 172.21.15.16:0/2296052407 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff894012430 con 0x7ff89c0cf390 2023-05-20T01:17:27.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.777+0000 7ff8a337f700 1 -- 172.21.15.16:0/2296052407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff89c0cf390 msgr2=0x7ff89c0cc110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:27.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.777+0000 7ff8a337f700 1 --2- 172.21.15.16:0/2296052407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff89c0cf390 0x7ff89c0cc110 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7ff89c0cebd0 tx=0x7ff89400b330 comp rx=0 tx=0).stop 2023-05-20T01:17:27.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.777+0000 7ff8a337f700 1 -- 172.21.15.16:0/2296052407 shutdown_connections 2023-05-20T01:17:27.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.777+0000 7ff8a337f700 1 --2- 172.21.15.16:0/2296052407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff89c0cf390 0x7ff89c0cc110 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:27.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.777+0000 7ff8a337f700 1 --2- 172.21.15.16:0/2296052407 >> v2:172.21.15.132:3300/0 conn(0x7ff89c15ab80 0x7ff89c0cda80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:27.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.777+0000 7ff8a337f700 1 -- 172.21.15.16:0/2296052407 >> 172.21.15.16:0/2296052407 conn(0x7ff89c06ae30 msgr2=0x7ff89c09a400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:27.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.777+0000 7ff8a337f700 1 -- 172.21.15.16:0/2296052407 shutdown_connections 2023-05-20T01:17:27.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.777+0000 7ff8a337f700 1 -- 172.21.15.16:0/2296052407 wait complete. 2023-05-20T01:17:27.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.779+0000 7ff8a337f700 1 Processor -- start 2023-05-20T01:17:27.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.779+0000 7ff8a337f700 1 -- start start 2023-05-20T01:17:27.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.779+0000 7ff8a337f700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff89c0cdfc0 0x7ff89c0c4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:27.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.779+0000 7ff8a337f700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff89c0cf390 0x7ff89c1500f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:27.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.779+0000 7ff8a337f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff89c15ab80 0x7ff89c0b6660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:27.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.779+0000 7ff8a337f700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff89c155460 con 0x7ff89c15ab80 2023-05-20T01:17:27.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.779+0000 7ff8a337f700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff89c150630 con 0x7ff89c0cdfc0 2023-05-20T01:17:27.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.779+0000 7ff8a337f700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff89c0b46d0 con 0x7ff89c0cf390 2023-05-20T01:17:27.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.779+0000 7ff8a191c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff89c15ab80 0x7ff89c0b6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:27.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.779+0000 7ff8a191c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff89c15ab80 0x7ff89c0b6660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59562/0 (socket says 172.21.15.16:59562) 2023-05-20T01:17:27.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.779+0000 7ff8a191c700 1 -- 172.21.15.16:0/3652123873 learned_addr learned my addr 172.21.15.16:0/3652123873 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:27.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.780+0000 7ff8a091a700 1 --2- 172.21.15.16:0/3652123873 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff89c0cf390 0x7ff89c1500f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:27.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.780+0000 7ff8a111b700 1 --2- 172.21.15.16:0/3652123873 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff89c0cdfc0 0x7ff89c0c4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:27.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.780+0000 7ff8a191c700 1 -- 172.21.15.16:0/3652123873 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff89c0cf390 msgr2=0x7ff89c1500f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:27.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.780+0000 7ff8a191c700 1 --2- 172.21.15.16:0/3652123873 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff89c0cf390 0x7ff89c1500f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:27.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.780+0000 7ff8a191c700 1 -- 172.21.15.16:0/3652123873 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff89c0cdfc0 msgr2=0x7ff89c0c4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:27.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.780+0000 7ff8a191c700 1 --2- 172.21.15.16:0/3652123873 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff89c0cdfc0 0x7ff89c0c4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:27.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.780+0000 7ff8a191c700 1 -- 172.21.15.16:0/3652123873 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff894009a70 con 0x7ff89c15ab80 2023-05-20T01:17:27.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.780+0000 7ff8a191c700 1 --2- 172.21.15.16:0/3652123873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff89c15ab80 0x7ff89c0b6660 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7ff894013000 tx=0x7ff894004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:27.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.780+0000 7ff891ffb700 1 -- 172.21.15.16:0/3652123873 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff894012430 con 0x7ff89c15ab80 2023-05-20T01:17:27.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.780+0000 7ff891ffb700 1 -- 172.21.15.16:0/3652123873 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff894020070 con 0x7ff89c15ab80 2023-05-20T01:17:27.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.780+0000 7ff891ffb700 1 -- 172.21.15.16:0/3652123873 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff894004280 con 0x7ff89c15ab80 2023-05-20T01:17:27.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.780+0000 7ff8a337f700 1 -- 172.21.15.16:0/3652123873 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff89c0d0e30 con 0x7ff89c15ab80 2023-05-20T01:17:27.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.780+0000 7ff8a337f700 1 -- 172.21.15.16:0/3652123873 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff89c0cbbf0 con 0x7ff89c15ab80 2023-05-20T01:17:27.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.781+0000 7ff8a337f700 1 -- 172.21.15.16:0/3652123873 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff89c0d0fc0 con 0x7ff89c15ab80 2023-05-20T01:17:27.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.782+0000 7ff891ffb700 1 -- 172.21.15.16:0/3652123873 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff8940043e0 con 0x7ff89c15ab80 2023-05-20T01:17:27.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.782+0000 7ff891ffb700 1 --2- 172.21.15.16:0/3652123873 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff88006ace0 0x7ff88006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:27.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.782+0000 7ff891ffb700 1 -- 172.21.15.16:0/3652123873 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7ff894022070 con 0x7ff89c15ab80 2023-05-20T01:17:27.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.782+0000 7ff8a111b700 1 --2- 172.21.15.16:0/3652123873 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff88006ace0 0x7ff88006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:27.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.782+0000 7ff8a111b700 1 --2- 172.21.15.16:0/3652123873 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff88006ace0 0x7ff88006d190 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff89c1e4c70 tx=0x7ff898008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:27.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.786+0000 7ff891ffb700 1 -- 172.21.15.16:0/3652123873 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7ff894056340 con 0x7ff89c15ab80 2023-05-20T01:17:27.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.982+0000 7ff8a337f700 1 -- 172.21.15.16:0/3652123873 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff89c0747b0 con 0x7ff89c15ab80 2023-05-20T01:17:27.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.983+0000 7ff891ffb700 1 -- 172.21.15.16:0/3652123873 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff894059960 con 0x7ff89c15ab80 2023-05-20T01:17:27.987 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2023-05-20T01:17:27.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.984+0000 7ff8a337f700 1 -- 172.21.15.16:0/3652123873 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff88006ace0 msgr2=0x7ff88006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:27.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.984+0000 7ff8a337f700 1 --2- 172.21.15.16:0/3652123873 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff88006ace0 0x7ff88006d190 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff89c1e4c70 tx=0x7ff898008040 comp rx=0 tx=0).stop 2023-05-20T01:17:27.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.985+0000 7ff8a337f700 1 -- 172.21.15.16:0/3652123873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff89c15ab80 msgr2=0x7ff89c0b6660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:27.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.985+0000 7ff8a337f700 1 --2- 172.21.15.16:0/3652123873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff89c15ab80 0x7ff89c0b6660 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7ff894013000 tx=0x7ff894004ad0 comp rx=0 tx=0).stop 2023-05-20T01:17:27.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.985+0000 7ff8a337f700 1 -- 172.21.15.16:0/3652123873 shutdown_connections 2023-05-20T01:17:27.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.985+0000 7ff8a337f700 1 --2- 172.21.15.16:0/3652123873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff89c15ab80 0x7ff89c0b6660 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:27.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.985+0000 7ff8a337f700 1 --2- 172.21.15.16:0/3652123873 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff89c0cdfc0 0x7ff89c0c4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:27.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.985+0000 7ff8a337f700 1 --2- 172.21.15.16:0/3652123873 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff88006ace0 0x7ff88006d190 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:27.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.985+0000 7ff8a337f700 1 --2- 172.21.15.16:0/3652123873 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff89c0cf390 0x7ff89c1500f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:27.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.985+0000 7ff8a337f700 1 -- 172.21.15.16:0/3652123873 >> 172.21.15.16:0/3652123873 conn(0x7ff89c06ae30 msgr2=0x7ff89c0cc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:27.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.985+0000 7ff8a337f700 1 -- 172.21.15.16:0/3652123873 shutdown_connections 2023-05-20T01:17:27.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:27.985+0000 7ff8a337f700 1 -- 172.21.15.16:0/3652123873 wait complete. 2023-05-20T01:17:28.000 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-05-20T01:17:28.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.125+0000 7f2c72652700 1 Processor -- start 2023-05-20T01:17:28.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.125+0000 7f2c72652700 1 -- start start 2023-05-20T01:17:28.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.126+0000 7f2c72652700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2c6c15cec0 0x7f2c6c071bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:28.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.126+0000 7f2c72652700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2c6c15d290 0x7f2c6c072130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:28.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.126+0000 7f2c72652700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c6c0cfe30 con 0x7f2c6c0cf2c0 2023-05-20T01:17:28.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.126+0000 7f2c72652700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c6c0cd630 con 0x7f2c6c15d290 2023-05-20T01:17:28.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.126+0000 7f2c72652700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c6c0cca60 con 0x7f2c6c15cec0 2023-05-20T01:17:28.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.126+0000 7f2c6b7fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2c6c15d290 0x7f2c6c072130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:28.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.126+0000 7f2c6b7fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2c6c15d290 0x7f2c6c072130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59570/0 (socket says 172.21.15.16:59570) 2023-05-20T01:17:28.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.126+0000 7f2c6b7fe700 1 -- 172.21.15.16:0/3501072928 learned_addr learned my addr 172.21.15.16:0/3501072928 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:28.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.126+0000 7f2c6b7fe700 1 -- 172.21.15.16:0/3501072928 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2c6c15cec0 msgr2=0x7f2c6c071bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:28.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.126+0000 7f2c6bfff700 1 --2- 172.21.15.16:0/3501072928 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2c6c15cec0 0x7f2c6c071bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:28.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.126+0000 7f2c6b7fe700 1 --2- 172.21.15.16:0/3501072928 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2c6c15cec0 0x7f2c6c071bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:28.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.126+0000 7f2c6b7fe700 1 -- 172.21.15.16:0/3501072928 >> v1:172.21.15.132:6789/0 conn(0x7f2c6c0cf2c0 legacy=0x7f2c6c0cf720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:28.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.126+0000 7f2c6b7fe700 1 -- 172.21.15.16:0/3501072928 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c6c0ccd00 con 0x7f2c6c15d290 2023-05-20T01:17:28.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.127+0000 7f2c6b7fe700 1 --2- 172.21.15.16:0/3501072928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c6c15d290 0x7f2c6c072130 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c0098f0 tx=0x7f2c5c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7aa254ba8449b372 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:28.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.127+0000 7f2c6affd700 1 -- 172.21.15.16:0/3501072928 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c5c004230 con 0x7f2c6c15d290 2023-05-20T01:17:28.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.127+0000 7f2c6affd700 1 -- 172.21.15.16:0/3501072928 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c5c004390 con 0x7f2c6c15d290 2023-05-20T01:17:28.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.127+0000 7f2c6affd700 1 -- 172.21.15.16:0/3501072928 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c5c0036a0 con 0x7f2c6c15d290 2023-05-20T01:17:28.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.127+0000 7f2c72652700 1 -- 172.21.15.16:0/3501072928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c6c15d290 msgr2=0x7f2c6c072130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:28.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.127+0000 7f2c72652700 1 --2- 172.21.15.16:0/3501072928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c6c15d290 0x7f2c6c072130 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c0098f0 tx=0x7f2c5c009c00 comp rx=0 tx=0).stop 2023-05-20T01:17:28.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.127+0000 7f2c72652700 1 -- 172.21.15.16:0/3501072928 shutdown_connections 2023-05-20T01:17:28.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.127+0000 7f2c72652700 1 --2- 172.21.15.16:0/3501072928 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2c6c15cec0 0x7f2c6c071bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:28.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.127+0000 7f2c72652700 1 --2- 172.21.15.16:0/3501072928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c6c15d290 0x7f2c6c072130 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:28.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.127+0000 7f2c72652700 1 -- 172.21.15.16:0/3501072928 >> 172.21.15.16:0/3501072928 conn(0x7f2c6c0cffd0 msgr2=0x7f2c6c0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:28.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.127+0000 7f2c72652700 1 -- 172.21.15.16:0/3501072928 shutdown_connections 2023-05-20T01:17:28.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.127+0000 7f2c72652700 1 -- 172.21.15.16:0/3501072928 wait complete. 2023-05-20T01:17:28.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.129+0000 7f2c72652700 1 Processor -- start 2023-05-20T01:17:28.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.129+0000 7f2c72652700 1 -- start start 2023-05-20T01:17:28.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.130+0000 7f2c72652700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2c6c0cf2c0 0x7f2c6c1565c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:28.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.130+0000 7f2c72652700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c6c15cec0 0x7f2c6c1537e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:28.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.130+0000 7f2c72652700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2c6c15d290 0x7f2c6c0ca870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:28.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.130+0000 7f2c72652700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c6c0b3da0 con 0x7f2c6c15cec0 2023-05-20T01:17:28.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.130+0000 7f2c72652700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c6c156b00 con 0x7f2c6c15d290 2023-05-20T01:17:28.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.130+0000 7f2c72652700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c6c0b2e40 con 0x7f2c6c0cf2c0 2023-05-20T01:17:28.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.130+0000 7f2c6b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c6c15cec0 0x7f2c6c1537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:28.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.130+0000 7f2c6b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c6c15cec0 0x7f2c6c1537e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59574/0 (socket says 172.21.15.16:59574) 2023-05-20T01:17:28.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.130+0000 7f2c6b7fe700 1 -- 172.21.15.16:0/3650857955 learned_addr learned my addr 172.21.15.16:0/3650857955 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:28.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.130+0000 7f2c6bfff700 1 --2- 172.21.15.16:0/3650857955 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2c6c0cf2c0 0x7f2c6c1565c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:28.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.130+0000 7f2c70bef700 1 --2- 172.21.15.16:0/3650857955 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2c6c15d290 0x7f2c6c0ca870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:28.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.131+0000 7f2c6b7fe700 1 -- 172.21.15.16:0/3650857955 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2c6c0cf2c0 msgr2=0x7f2c6c1565c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:28.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.131+0000 7f2c6b7fe700 1 --2- 172.21.15.16:0/3650857955 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2c6c0cf2c0 0x7f2c6c1565c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:28.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.131+0000 7f2c6b7fe700 1 -- 172.21.15.16:0/3650857955 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2c6c15d290 msgr2=0x7f2c6c0ca870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:28.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.131+0000 7f2c6b7fe700 1 --2- 172.21.15.16:0/3650857955 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2c6c15d290 0x7f2c6c0ca870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:28.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.131+0000 7f2c6b7fe700 1 -- 172.21.15.16:0/3650857955 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c5c009550 con 0x7f2c6c15cec0 2023-05-20T01:17:28.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.131+0000 7f2c70bef700 1 --2- 172.21.15.16:0/3650857955 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2c6c15d290 0x7f2c6c0ca870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:17:28.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.131+0000 7f2c6b7fe700 1 --2- 172.21.15.16:0/3650857955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c6c15cec0 0x7f2c6c1537e0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c009130 tx=0x7f2c5c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:28.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.131+0000 7f2c68ff9700 1 -- 172.21.15.16:0/3650857955 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c5c00b660 con 0x7f2c6c15cec0 2023-05-20T01:17:28.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.131+0000 7f2c68ff9700 1 -- 172.21.15.16:0/3650857955 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c5c00b7c0 con 0x7f2c6c15cec0 2023-05-20T01:17:28.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.131+0000 7f2c72652700 1 -- 172.21.15.16:0/3650857955 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c6c0cd2c0 con 0x7f2c6c15cec0 2023-05-20T01:17:28.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.131+0000 7f2c68ff9700 1 -- 172.21.15.16:0/3650857955 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c5c00bb70 con 0x7f2c6c15cec0 2023-05-20T01:17:28.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.131+0000 7f2c72652700 1 -- 172.21.15.16:0/3650857955 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c6c0ccfe0 con 0x7f2c6c15cec0 2023-05-20T01:17:28.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.132+0000 7f2c72652700 1 -- 172.21.15.16:0/3650857955 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c6c0b31d0 con 0x7f2c6c15cec0 2023-05-20T01:17:28.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.132+0000 7f2c68ff9700 1 -- 172.21.15.16:0/3650857955 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2c5c00bcd0 con 0x7f2c6c15cec0 2023-05-20T01:17:28.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.133+0000 7f2c68ff9700 1 --2- 172.21.15.16:0/3650857955 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f2c5806ace0 0x7f2c5806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:28.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.133+0000 7f2c68ff9700 1 -- 172.21.15.16:0/3650857955 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f2c5c08af10 con 0x7f2c6c15cec0 2023-05-20T01:17:28.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.133+0000 7f2c6bfff700 1 --2- 172.21.15.16:0/3650857955 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f2c5806ace0 0x7f2c5806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:28.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.133+0000 7f2c6bfff700 1 --2- 172.21.15.16:0/3650857955 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f2c5806ace0 0x7f2c5806d190 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2c6000ab40 tx=0x7f2c60008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:28.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.137+0000 7f2c68ff9700 1 -- 172.21.15.16:0/3650857955 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f2c5c055d20 con 0x7f2c6c15cec0 2023-05-20T01:17:28.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.334+0000 7f2c72652700 1 -- 172.21.15.16:0/3650857955 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f2c6c160890 con 0x7f2c6c15cec0 2023-05-20T01:17:28.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.335+0000 7f2c68ff9700 1 -- 172.21.15.16:0/3650857955 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f2c5c01f020 con 0x7f2c6c15cec0 2023-05-20T01:17:28.339 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-05-20T01:17:28.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.337+0000 7f2c72652700 1 -- 172.21.15.16:0/3650857955 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f2c5806ace0 msgr2=0x7f2c5806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:28.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.337+0000 7f2c72652700 1 --2- 172.21.15.16:0/3650857955 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f2c5806ace0 0x7f2c5806d190 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2c6000ab40 tx=0x7f2c60008040 comp rx=0 tx=0).stop 2023-05-20T01:17:28.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.337+0000 7f2c72652700 1 -- 172.21.15.16:0/3650857955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c6c15cec0 msgr2=0x7f2c6c1537e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:28.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.337+0000 7f2c72652700 1 --2- 172.21.15.16:0/3650857955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c6c15cec0 0x7f2c6c1537e0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c009130 tx=0x7f2c5c004750 comp rx=0 tx=0).stop 2023-05-20T01:17:28.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.337+0000 7f2c72652700 1 -- 172.21.15.16:0/3650857955 shutdown_connections 2023-05-20T01:17:28.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.337+0000 7f2c72652700 1 --2- 172.21.15.16:0/3650857955 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2c6c15d290 0x7f2c6c0ca870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:28.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.337+0000 7f2c72652700 1 --2- 172.21.15.16:0/3650857955 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f2c5806ace0 0x7f2c5806d190 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:28.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.337+0000 7f2c72652700 1 --2- 172.21.15.16:0/3650857955 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2c6c0cf2c0 0x7f2c6c1565c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:28.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.337+0000 7f2c72652700 1 --2- 172.21.15.16:0/3650857955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c6c15cec0 0x7f2c6c1537e0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:28.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.337+0000 7f2c72652700 1 -- 172.21.15.16:0/3650857955 >> 172.21.15.16:0/3650857955 conn(0x7f2c6c0cffd0 msgr2=0x7f2c6c15d6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:28.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.337+0000 7f2c72652700 1 -- 172.21.15.16:0/3650857955 shutdown_connections 2023-05-20T01:17:28.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.337+0000 7f2c72652700 1 -- 172.21.15.16:0/3650857955 wait complete. 2023-05-20T01:17:28.354 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-05-20T01:17:28.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.479+0000 7fb117483700 1 Processor -- start 2023-05-20T01:17:28.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.479+0000 7fb117483700 1 -- start start 2023-05-20T01:17:28.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.480+0000 7fb117483700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb11006a510 0x7fb11006a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:28.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.480+0000 7fb117483700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb11006aea0 con 0x7fb110071140 2023-05-20T01:17:28.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.480+0000 7fb117483700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb11015d020 con 0x7fb110161190 2023-05-20T01:17:28.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.480+0000 7fb117483700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1100cebf0 con 0x7fb11006a510 2023-05-20T01:17:28.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.480+0000 7fb115a20700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fb110071140 0x7fb1100715a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:42880/0 (socket says 172.21.15.16:42880) 2023-05-20T01:17:28.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.480+0000 7fb115a20700 1 -- 172.21.15.16:0/3757218930 learned_addr learned my addr 172.21.15.16:0/3757218930 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:17:28.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.480+0000 7fb107fff700 1 -- 172.21.15.16:0/3757218930 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 323649879 0 0) 0x7fb108001490 con 0x7fb110071140 2023-05-20T01:17:28.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.480+0000 7fb107fff700 1 -- 172.21.15.16:0/3757218930 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb0fc0035b0 con 0x7fb110071140 2023-05-20T01:17:28.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.480+0000 7fb114a1e700 1 --2- 172.21.15.16:0/3757218930 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb11006a510 0x7fb11006a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:28.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.480+0000 7fb107fff700 1 -- 172.21.15.16:0/3757218930 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1735709373 0 0) 0x7fb0fc0035b0 con 0x7fb110071140 2023-05-20T01:17:28.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.480+0000 7fb107fff700 1 -- 172.21.15.16:0/3757218930 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb11006a510 msgr2=0x7fb11006a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:28.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.480+0000 7fb107fff700 1 --2- 172.21.15.16:0/3757218930 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb11006a510 0x7fb11006a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:28.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.480+0000 7fb107fff700 1 -- 172.21.15.16:0/3757218930 >> v1:172.21.15.132:6789/0 conn(0x7fb110161190 legacy=0x7fb11006de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:28.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.480+0000 7fb107fff700 1 -- 172.21.15.16:0/3757218930 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb11006e9a0 con 0x7fb110071140 2023-05-20T01:17:28.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.480+0000 7fb107fff700 1 -- 172.21.15.16:0/3757218930 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7fb108001b80 con 0x7fb110071140 2023-05-20T01:17:28.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.480+0000 7fb107fff700 1 -- 172.21.15.16:0/3757218930 >> v1:172.21.15.16:6789/0 conn(0x7fb110071140 legacy=0x7fb1100715a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:28.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.480+0000 7fb107fff700 1 --2- 172.21.15.16:0/3757218930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0fc003aa0 0x7fb0fc003eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:28.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.481+0000 7fb107fff700 1 --2- 172.21.15.16:0/3757218930 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb0fc004850 0x7fb0fc004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:28.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.481+0000 7fb107fff700 1 --2- 172.21.15.16:0/3757218930 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb11006a510 0x7fb0fc0052b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:28.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.481+0000 7fb107fff700 1 -- 172.21.15.16:0/3757218930 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1100cebf0 con 0x7fb0fc003aa0 2023-05-20T01:17:28.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.481+0000 7fb107fff700 1 -- 172.21.15.16:0/3757218930 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0fc00bc60 con 0x7fb0fc004850 2023-05-20T01:17:28.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.481+0000 7fb107fff700 1 -- 172.21.15.16:0/3757218930 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0fc00be20 con 0x7fb11006a510 2023-05-20T01:17:28.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.481+0000 7fb11521f700 1 --2- 172.21.15.16:0/3757218930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0fc003aa0 0x7fb0fc003eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:28.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.481+0000 7fb114a1e700 1 --2- 172.21.15.16:0/3757218930 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb0fc004850 0x7fb0fc004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:28.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.481+0000 7fb115a20700 1 --2- 172.21.15.16:0/3757218930 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb11006a510 0x7fb0fc0052b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:28.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.481+0000 7fb11521f700 1 -- 172.21.15.16:0/3757218930 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb11006a510 msgr2=0x7fb0fc0052b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:28.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.481+0000 7fb11521f700 1 --2- 172.21.15.16:0/3757218930 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb11006a510 0x7fb0fc0052b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:28.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.481+0000 7fb11521f700 1 -- 172.21.15.16:0/3757218930 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb0fc004850 msgr2=0x7fb0fc004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:28.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.481+0000 7fb11521f700 1 --2- 172.21.15.16:0/3757218930 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb0fc004850 0x7fb0fc004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:28.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.481+0000 7fb11521f700 1 -- 172.21.15.16:0/3757218930 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0fc0035b0 con 0x7fb0fc003aa0 2023-05-20T01:17:28.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.481+0000 7fb114a1e700 1 --2- 172.21.15.16:0/3757218930 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb0fc004850 0x7fb0fc004cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:17:28.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.481+0000 7fb115a20700 1 --2- 172.21.15.16:0/3757218930 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb11006a510 0x7fb0fc0052b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:17:28.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.481+0000 7fb11521f700 1 --2- 172.21.15.16:0/3757218930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0fc003aa0 0x7fb0fc003eb0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fb10c007fd0 tx=0x7fb0fc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75f89feed3b74957 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:28.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.481+0000 7fb107fff700 1 -- 172.21.15.16:0/3757218930 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb10c0052d0 con 0x7fb0fc003aa0 2023-05-20T01:17:28.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.481+0000 7fb107fff700 1 -- 172.21.15.16:0/3757218930 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb10c005470 con 0x7fb0fc003aa0 2023-05-20T01:17:28.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.481+0000 7fb107fff700 1 -- 172.21.15.16:0/3757218930 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb10c0038c0 con 0x7fb0fc003aa0 2023-05-20T01:17:28.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.482+0000 7fb117483700 1 -- 172.21.15.16:0/3757218930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0fc003aa0 msgr2=0x7fb0fc003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:28.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.482+0000 7fb117483700 1 --2- 172.21.15.16:0/3757218930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0fc003aa0 0x7fb0fc003eb0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fb10c007fd0 tx=0x7fb0fc005880 comp rx=0 tx=0).stop 2023-05-20T01:17:28.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.482+0000 7fb115a20700 1 -- 172.21.15.16:0/3757218930 reap_dead start 2023-05-20T01:17:28.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.482+0000 7fb117483700 1 -- 172.21.15.16:0/3757218930 shutdown_connections 2023-05-20T01:17:28.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.482+0000 7fb117483700 1 -- 172.21.15.16:0/3757218930 >> 172.21.15.16:0/3757218930 conn(0x7fb1100d0c70 msgr2=0x7fb1100b1af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:28.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.482+0000 7fb117483700 1 -- 172.21.15.16:0/3757218930 shutdown_connections 2023-05-20T01:17:28.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.482+0000 7fb117483700 1 -- 172.21.15.16:0/3757218930 wait complete. 2023-05-20T01:17:28.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.483+0000 7fb117483700 1 Processor -- start 2023-05-20T01:17:28.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.483+0000 7fb117483700 1 -- start start 2023-05-20T01:17:28.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.484+0000 7fb117483700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb0fc003aa0 0x7fb1101f26f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:28.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.484+0000 7fb117483700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb110159fc0 0x7fb1100c6670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:28.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.484+0000 7fb117483700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb11015a430 0x7fb110158030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:28.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.484+0000 7fb117483700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb110157ba0 con 0x7fb11015a430 2023-05-20T01:17:28.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.484+0000 7fb117483700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb110156f90 con 0x7fb110159fc0 2023-05-20T01:17:28.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.484+0000 7fb117483700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb110157270 con 0x7fb0fc003aa0 2023-05-20T01:17:28.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.484+0000 7fb115a20700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb11015a430 0x7fb110158030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:28.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.484+0000 7fb115a20700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb11015a430 0x7fb110158030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59596/0 (socket says 172.21.15.16:59596) 2023-05-20T01:17:28.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.484+0000 7fb115a20700 1 -- 172.21.15.16:0/3969128433 learned_addr learned my addr 172.21.15.16:0/3969128433 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:28.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.484+0000 7fb114a1e700 1 --2- 172.21.15.16:0/3969128433 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb110159fc0 0x7fb1100c6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:28.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.484+0000 7fb11521f700 1 --2- 172.21.15.16:0/3969128433 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb0fc003aa0 0x7fb1101f26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:28.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.484+0000 7fb115a20700 1 -- 172.21.15.16:0/3969128433 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb0fc003aa0 msgr2=0x7fb1101f26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:28.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.484+0000 7fb115a20700 1 --2- 172.21.15.16:0/3969128433 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb0fc003aa0 0x7fb1101f26f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:28.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.484+0000 7fb115a20700 1 -- 172.21.15.16:0/3969128433 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb110159fc0 msgr2=0x7fb1100c6670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:28.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.484+0000 7fb115a20700 1 --2- 172.21.15.16:0/3969128433 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb110159fc0 0x7fb1100c6670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:28.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.484+0000 7fb115a20700 1 -- 172.21.15.16:0/3969128433 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1100cf170 con 0x7fb11015a430 2023-05-20T01:17:28.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.485+0000 7fb115a20700 1 --2- 172.21.15.16:0/3969128433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb11015a430 0x7fb110158030 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fb110070cf0 tx=0x7fb10800e440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:28.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.485+0000 7fb105ffb700 1 -- 172.21.15.16:0/3969128433 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb10800eb00 con 0x7fb11015a430 2023-05-20T01:17:28.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.485+0000 7fb105ffb700 1 -- 172.21.15.16:0/3969128433 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb10800ec60 con 0x7fb11015a430 2023-05-20T01:17:28.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.485+0000 7fb117483700 1 -- 172.21.15.16:0/3969128433 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1100cee90 con 0x7fb11015a430 2023-05-20T01:17:28.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.485+0000 7fb105ffb700 1 -- 172.21.15.16:0/3969128433 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb108013420 con 0x7fb11015a430 2023-05-20T01:17:28.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.485+0000 7fb117483700 1 -- 172.21.15.16:0/3969128433 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1100cf450 con 0x7fb11015a430 2023-05-20T01:17:28.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.486+0000 7fb117483700 1 -- 172.21.15.16:0/3969128433 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb110157d10 con 0x7fb11015a430 2023-05-20T01:17:28.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.486+0000 7fb105ffb700 1 -- 172.21.15.16:0/3969128433 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb108013580 con 0x7fb11015a430 2023-05-20T01:17:28.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.487+0000 7fb105ffb700 1 --2- 172.21.15.16:0/3969128433 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb0f406ace0 0x7fb0f406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:28.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.487+0000 7fb105ffb700 1 -- 172.21.15.16:0/3969128433 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fb1080899b0 con 0x7fb11015a430 2023-05-20T01:17:28.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.487+0000 7fb11521f700 1 --2- 172.21.15.16:0/3969128433 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb0f406ace0 0x7fb0f406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:28.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.487+0000 7fb11521f700 1 --2- 172.21.15.16:0/3969128433 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb0f406ace0 0x7fb0f406d190 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb10c007fd0 tx=0x7fb10c009030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:28.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.491+0000 7fb105ffb700 1 -- 172.21.15.16:0/3969128433 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fb1080581c0 con 0x7fb11015a430 2023-05-20T01:17:28.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.706+0000 7fb117483700 1 -- 172.21.15.16:0/3969128433 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fb11006aea0 con 0x7fb11015a430 2023-05-20T01:17:28.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.706+0000 7fb105ffb700 1 -- 172.21.15.16:0/3969128433 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) v1 ==== 80+0+0 (secure 0 0 0) 0x7fb11006aea0 con 0x7fb11015a430 2023-05-20T01:17:28.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.708+0000 7fb117483700 1 -- 172.21.15.16:0/3969128433 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb0f406ace0 msgr2=0x7fb0f406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:28.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.708+0000 7fb117483700 1 --2- 172.21.15.16:0/3969128433 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb0f406ace0 0x7fb0f406d190 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb10c007fd0 tx=0x7fb10c009030 comp rx=0 tx=0).stop 2023-05-20T01:17:28.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.708+0000 7fb117483700 1 -- 172.21.15.16:0/3969128433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb11015a430 msgr2=0x7fb110158030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:28.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.708+0000 7fb117483700 1 --2- 172.21.15.16:0/3969128433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb11015a430 0x7fb110158030 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fb110070cf0 tx=0x7fb10800e440 comp rx=0 tx=0).stop 2023-05-20T01:17:28.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.708+0000 7fb117483700 1 -- 172.21.15.16:0/3969128433 shutdown_connections 2023-05-20T01:17:28.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.708+0000 7fb117483700 1 --2- 172.21.15.16:0/3969128433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb11015a430 0x7fb110158030 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:28.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.708+0000 7fb117483700 1 --2- 172.21.15.16:0/3969128433 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb0f406ace0 0x7fb0f406d190 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:28.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.708+0000 7fb117483700 1 --2- 172.21.15.16:0/3969128433 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb0fc003aa0 0x7fb1101f26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:28.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.708+0000 7fb117483700 1 --2- 172.21.15.16:0/3969128433 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb110159fc0 0x7fb1100c6670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:28.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.708+0000 7fb117483700 1 -- 172.21.15.16:0/3969128433 >> 172.21.15.16:0/3969128433 conn(0x7fb1100d0c70 msgr2=0x7fb1100b43a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:28.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.708+0000 7fb117483700 1 -- 172.21.15.16:0/3969128433 shutdown_connections 2023-05-20T01:17:28.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.708+0000 7fb117483700 1 -- 172.21.15.16:0/3969128433 wait complete. 2023-05-20T01:17:28.714 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2023-05-20T01:17:28.724 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2023-05-20T01:17:28.724 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:17:28.725 DEBUG:teuthology.orchestra.run.smithi132:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2023-05-20T01:17:28.769 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi132.front.sepia.ceph.com 2023-05-20T01:17:28.773 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:17:28.774 DEBUG:teuthology.orchestra.run.smithi132:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-05-20T01:17:28.829 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-05-20T01:17:28.829 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-20T01:17:28.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.951+0000 7f69df1bf700 1 Processor -- start 2023-05-20T01:17:28.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.951+0000 7f69df1bf700 1 -- start start 2023-05-20T01:17:28.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.951+0000 7f69df1bf700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f69d81599c0 0x7f69d80caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:28.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.951+0000 7f69df1bf700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f69d80caaa0 0x7f69d80cb4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:28.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.951+0000 7f69df1bf700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f69d806f3f0 0x7f69d80710d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:28.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.951+0000 7f69df1bf700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69d80cdf20 con 0x7f69d806f3f0 2023-05-20T01:17:28.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.951+0000 7f69df1bf700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69d8159d90 con 0x7f69d81599c0 2023-05-20T01:17:28.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.951+0000 7f69df1bf700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69d80cdb80 con 0x7f69d80caaa0 2023-05-20T01:17:28.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.952+0000 7f69dd75c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f69d806f3f0 0x7f69d80710d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:28.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.952+0000 7f69dd75c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f69d806f3f0 0x7f69d80710d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59602/0 (socket says 172.21.15.16:59602) 2023-05-20T01:17:28.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.952+0000 7f69dd75c700 1 -- 172.21.15.16:0/1302100423 learned_addr learned my addr 172.21.15.16:0/1302100423 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:28.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.952+0000 7f69dcf5b700 1 --2- 172.21.15.16:0/1302100423 >> v2:172.21.15.132:3300/0 conn(0x7f69d81599c0 0x7f69d80caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:28.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.952+0000 7f69d7fff700 1 --2- 172.21.15.16:0/1302100423 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f69d80caaa0 0x7f69d80cb4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:28.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.952+0000 7f69dd75c700 1 -- 172.21.15.16:0/1302100423 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f69d80caaa0 msgr2=0x7f69d80cb4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:28.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.952+0000 7f69dd75c700 1 --2- 172.21.15.16:0/1302100423 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f69d80caaa0 0x7f69d80cb4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:28.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.952+0000 7f69dd75c700 1 -- 172.21.15.16:0/1302100423 >> v2:172.21.15.132:3300/0 conn(0x7f69d81599c0 msgr2=0x7f69d80caf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:28.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.952+0000 7f69dd75c700 1 --2- 172.21.15.16:0/1302100423 >> v2:172.21.15.132:3300/0 conn(0x7f69d81599c0 0x7f69d80caf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:28.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.952+0000 7f69dd75c700 1 -- 172.21.15.16:0/1302100423 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69d806eaf0 con 0x7f69d806f3f0 2023-05-20T01:17:28.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.952+0000 7f69dcf5b700 1 --2- 172.21.15.16:0/1302100423 >> v2:172.21.15.132:3300/0 conn(0x7f69d81599c0 0x7f69d80caf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:17:28.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.952+0000 7f69dd75c700 1 --2- 172.21.15.16:0/1302100423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69d806f3f0 0x7f69d80710d0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f69d8074210 tx=0x7f69d000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68b9f8b170066b7a server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:28.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.952+0000 7f69d77fe700 1 -- 172.21.15.16:0/1302100423 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69d000e070 con 0x7f69d806f3f0 2023-05-20T01:17:28.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.952+0000 7f69d77fe700 1 -- 172.21.15.16:0/1302100423 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69d000b9c0 con 0x7f69d806f3f0 2023-05-20T01:17:28.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.952+0000 7f69d77fe700 1 -- 172.21.15.16:0/1302100423 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69d000bd70 con 0x7f69d806f3f0 2023-05-20T01:17:28.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.953+0000 7f69df1bf700 1 -- 172.21.15.16:0/1302100423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69d806f3f0 msgr2=0x7f69d80710d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:28.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.953+0000 7f69df1bf700 1 --2- 172.21.15.16:0/1302100423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69d806f3f0 0x7f69d80710d0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f69d8074210 tx=0x7f69d000b330 comp rx=0 tx=0).stop 2023-05-20T01:17:28.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.953+0000 7f69df1bf700 1 -- 172.21.15.16:0/1302100423 shutdown_connections 2023-05-20T01:17:28.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.953+0000 7f69df1bf700 1 --2- 172.21.15.16:0/1302100423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69d806f3f0 0x7f69d80710d0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:28.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.953+0000 7f69df1bf700 1 --2- 172.21.15.16:0/1302100423 >> v2:172.21.15.132:3300/0 conn(0x7f69d81599c0 0x7f69d80caf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:28.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.953+0000 7f69df1bf700 1 --2- 172.21.15.16:0/1302100423 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f69d80caaa0 0x7f69d80cb4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:28.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.953+0000 7f69df1bf700 1 -- 172.21.15.16:0/1302100423 >> 172.21.15.16:0/1302100423 conn(0x7f69d80d0c30 msgr2=0x7f69d80b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:28.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.953+0000 7f69df1bf700 1 -- 172.21.15.16:0/1302100423 shutdown_connections 2023-05-20T01:17:28.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.953+0000 7f69df1bf700 1 -- 172.21.15.16:0/1302100423 wait complete. 2023-05-20T01:17:28.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.954+0000 7f69df1bf700 1 Processor -- start 2023-05-20T01:17:28.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.954+0000 7f69df1bf700 1 -- start start 2023-05-20T01:17:28.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.955+0000 7f69df1bf700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f69d806f3f0 0x7f69d81f21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:28.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.955+0000 7f69df1bf700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69d80caaa0 0x7f69d8153480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:28.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.955+0000 7f69df1bf700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f69d81599c0 0x7f69d815c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:28.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.955+0000 7f69df1bf700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69d81f5930 con 0x7f69d80caaa0 2023-05-20T01:17:28.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.955+0000 7f69df1bf700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69d81f5c10 con 0x7f69d806f3f0 2023-05-20T01:17:28.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.955+0000 7f69df1bf700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69d81f5ef0 con 0x7f69d81599c0 2023-05-20T01:17:28.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.955+0000 7f69d7fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69d80caaa0 0x7f69d8153480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:28.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.955+0000 7f69d7fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69d80caaa0 0x7f69d8153480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59606/0 (socket says 172.21.15.16:59606) 2023-05-20T01:17:28.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.955+0000 7f69d7fff700 1 -- 172.21.15.16:0/204334856 learned_addr learned my addr 172.21.15.16:0/204334856 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:28.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.955+0000 7f69dcf5b700 1 --2- 172.21.15.16:0/204334856 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f69d806f3f0 0x7f69d81f21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:28.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.955+0000 7f69dd75c700 1 --2- 172.21.15.16:0/204334856 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f69d81599c0 0x7f69d815c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:28.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.955+0000 7f69d7fff700 1 -- 172.21.15.16:0/204334856 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f69d81599c0 msgr2=0x7f69d815c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:28.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.955+0000 7f69d7fff700 1 --2- 172.21.15.16:0/204334856 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f69d81599c0 0x7f69d815c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:28.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.955+0000 7f69d7fff700 1 -- 172.21.15.16:0/204334856 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f69d806f3f0 msgr2=0x7f69d81f21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:28.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.955+0000 7f69d7fff700 1 --2- 172.21.15.16:0/204334856 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f69d806f3f0 0x7f69d81f21a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:28.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.955+0000 7f69d7fff700 1 -- 172.21.15.16:0/204334856 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69d0009b30 con 0x7f69d80caaa0 2023-05-20T01:17:28.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.956+0000 7f69d7fff700 1 --2- 172.21.15.16:0/204334856 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69d80caaa0 0x7f69d8153480 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f69c400e810 tx=0x7f69c400eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:28.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.956+0000 7f69d57fa700 1 -- 172.21.15.16:0/204334856 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69c400c070 con 0x7f69d80caaa0 2023-05-20T01:17:28.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.956+0000 7f69d57fa700 1 -- 172.21.15.16:0/204334856 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69c4013610 con 0x7f69d80caaa0 2023-05-20T01:17:28.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.956+0000 7f69df1bf700 1 -- 172.21.15.16:0/204334856 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69d806f0b0 con 0x7f69d80caaa0 2023-05-20T01:17:28.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.956+0000 7f69d57fa700 1 -- 172.21.15.16:0/204334856 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69c40139a0 con 0x7f69d80caaa0 2023-05-20T01:17:28.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.956+0000 7f69df1bf700 1 -- 172.21.15.16:0/204334856 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69d806edd0 con 0x7f69d80caaa0 2023-05-20T01:17:28.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.957+0000 7f69df1bf700 1 -- 172.21.15.16:0/204334856 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69d806e750 con 0x7f69d80caaa0 2023-05-20T01:17:28.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.957+0000 7f69d57fa700 1 -- 172.21.15.16:0/204334856 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f69c4013b00 con 0x7f69d80caaa0 2023-05-20T01:17:28.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.958+0000 7f69d57fa700 1 --2- 172.21.15.16:0/204334856 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f69bc06ace0 0x7f69bc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:28.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.958+0000 7f69d57fa700 1 -- 172.21.15.16:0/204334856 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f69c4089420 con 0x7f69d80caaa0 2023-05-20T01:17:28.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.958+0000 7f69dcf5b700 1 --2- 172.21.15.16:0/204334856 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f69bc06ace0 0x7f69bc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:28.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.958+0000 7f69dcf5b700 1 --2- 172.21.15.16:0/204334856 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f69bc06ace0 0x7f69bc06d190 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f69cc0099d0 tx=0x7f69cc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:28.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:28.962+0000 7f69d57fa700 1 -- 172.21.15.16:0/204334856 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f69c4054230 con 0x7f69d80caaa0 2023-05-20T01:17:29.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.204+0000 7f69df1bf700 1 -- 172.21.15.16:0/204334856 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f69d80747b0 con 0x7f69d80caaa0 2023-05-20T01:17:29.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.205+0000 7f69d57fa700 1 -- 172.21.15.16:0/204334856 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4673 (secure 0 0 0) 0x7f69c4057850 con 0x7f69d80caaa0 2023-05-20T01:17:29.209 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:17:29.209 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4273,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/1444124146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1444124146},{"type":"v1","addr":"172.21.15.132:6834","nonce":1444124146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4293,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4092119206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4092119206},{"type":"v1","addr":"172.21.15.16:6835","nonce":4092119206}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2162680372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":2162680372},{"type":"v1","addr":"172.21.15.132:6835","nonce":2162680372}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-20T01:16:28.316109+0000","modified":"2023-05-20T01:16:45.362572+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4299},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4299":{"gid":4299,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6837/2330551626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2330551626},{"type":"v1","addr":"172.21.15.16:6837","nonce":2330551626}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-05-20T01:17:29.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.207+0000 7f69df1bf700 1 -- 172.21.15.16:0/204334856 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f69bc06ace0 msgr2=0x7f69bc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:29.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.207+0000 7f69df1bf700 1 --2- 172.21.15.16:0/204334856 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f69bc06ace0 0x7f69bc06d190 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f69cc0099d0 tx=0x7f69cc008040 comp rx=0 tx=0).stop 2023-05-20T01:17:29.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.207+0000 7f69df1bf700 1 -- 172.21.15.16:0/204334856 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69d80caaa0 msgr2=0x7f69d8153480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:29.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.207+0000 7f69df1bf700 1 --2- 172.21.15.16:0/204334856 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69d80caaa0 0x7f69d8153480 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f69c400e810 tx=0x7f69c400eb20 comp rx=0 tx=0).stop 2023-05-20T01:17:29.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.207+0000 7f69df1bf700 1 -- 172.21.15.16:0/204334856 shutdown_connections 2023-05-20T01:17:29.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.207+0000 7f69df1bf700 1 --2- 172.21.15.16:0/204334856 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f69bc06ace0 0x7f69bc06d190 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:29.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.207+0000 7f69df1bf700 1 --2- 172.21.15.16:0/204334856 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f69d81599c0 0x7f69d815c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:29.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.207+0000 7f69df1bf700 1 --2- 172.21.15.16:0/204334856 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f69d806f3f0 0x7f69d81f21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:29.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.207+0000 7f69df1bf700 1 --2- 172.21.15.16:0/204334856 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69d80caaa0 0x7f69d8153480 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:29.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.207+0000 7f69df1bf700 1 -- 172.21.15.16:0/204334856 >> 172.21.15.16:0/204334856 conn(0x7f69d80d0c30 msgr2=0x7f69d80ce340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:29.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.207+0000 7f69df1bf700 1 -- 172.21.15.16:0/204334856 shutdown_connections 2023-05-20T01:17:29.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.207+0000 7f69df1bf700 1 -- 172.21.15.16:0/204334856 wait complete. 2023-05-20T01:17:29.212 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 8 2023-05-20T01:17:29.224 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-20T01:17:29.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.346+0000 7f3903d11700 1 Processor -- start 2023-05-20T01:17:29.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.346+0000 7f3903d11700 1 -- start start 2023-05-20T01:17:29.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.347+0000 7f3903d11700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f38fc157190 0x7f38fc0cc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:29.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.347+0000 7f3903d11700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f38fc0cccf0 0x7f38fc0713c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:29.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.347+0000 7f3903d11700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38fc0cb590 con 0x7f38fc071900 2023-05-20T01:17:29.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.347+0000 7f3903d11700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38fc0ca9a0 con 0x7f38fc157190 2023-05-20T01:17:29.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.347+0000 7f3903d11700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38fc0cac60 con 0x7f38fc0cccf0 2023-05-20T01:17:29.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.347+0000 7f39022ae700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f38fc071900 0x7f38fc15d220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:42912/0 (socket says 172.21.15.16:42912) 2023-05-20T01:17:29.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.347+0000 7f39022ae700 1 -- 172.21.15.16:0/4169673627 learned_addr learned my addr 172.21.15.16:0/4169673627 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:17:29.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.347+0000 7f39012ac700 1 --2- 172.21.15.16:0/4169673627 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f38fc0cccf0 0x7f38fc0713c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:29.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.347+0000 7f3901aad700 1 --2- 172.21.15.16:0/4169673627 >> v2:172.21.15.132:3300/0 conn(0x7f38fc157190 0x7f38fc0cc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:29.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.347+0000 7f3900aab700 1 -- 172.21.15.16:0/4169673627 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1745732069 0 0) 0x7f38fc0cb590 con 0x7f38fc071900 2023-05-20T01:17:29.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.347+0000 7f3900aab700 1 -- 172.21.15.16:0/4169673627 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38e80035b0 con 0x7f38fc071900 2023-05-20T01:17:29.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.348+0000 7f3900aab700 1 -- 172.21.15.16:0/4169673627 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 196232077 0 0) 0x7f38e80035b0 con 0x7f38fc071900 2023-05-20T01:17:29.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.348+0000 7f3900aab700 1 -- 172.21.15.16:0/4169673627 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f38fc0cccf0 msgr2=0x7f38fc0713c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:29.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.348+0000 7f3900aab700 1 --2- 172.21.15.16:0/4169673627 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f38fc0cccf0 0x7f38fc0713c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:29.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.348+0000 7f3900aab700 1 -- 172.21.15.16:0/4169673627 >> v2:172.21.15.132:3300/0 conn(0x7f38fc157190 msgr2=0x7f38fc0cc7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:29.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.348+0000 7f3900aab700 1 --2- 172.21.15.16:0/4169673627 >> v2:172.21.15.132:3300/0 conn(0x7f38fc157190 0x7f38fc0cc7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:29.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.348+0000 7f3900aab700 1 -- 172.21.15.16:0/4169673627 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38fc0caf40 con 0x7f38fc071900 2023-05-20T01:17:29.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.348+0000 7f3900aab700 1 -- 172.21.15.16:0/4169673627 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7f38f40019f0 con 0x7f38fc071900 2023-05-20T01:17:29.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.348+0000 7f3900aab700 1 -- 172.21.15.16:0/4169673627 >> v1:172.21.15.16:6789/0 conn(0x7f38fc071900 legacy=0x7f38fc15d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:29.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.348+0000 7f3900aab700 1 --2- 172.21.15.16:0/4169673627 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f38fc0cccf0 0x7f38e8003a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:29.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.348+0000 7f3900aab700 1 --2- 172.21.15.16:0/4169673627 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f38e8003ff0 0x7f38e8004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:29.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.348+0000 7f3900aab700 1 --2- 172.21.15.16:0/4169673627 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38e8004d90 0x7f38e8005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:29.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.348+0000 7f3900aab700 1 -- 172.21.15.16:0/4169673627 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38fc0ca9a0 con 0x7f38e8004d90 2023-05-20T01:17:29.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.348+0000 7f3900aab700 1 -- 172.21.15.16:0/4169673627 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38fc0cac60 con 0x7f38e8003ff0 2023-05-20T01:17:29.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.348+0000 7f3900aab700 1 -- 172.21.15.16:0/4169673627 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38e800ba10 con 0x7f38fc0cccf0 2023-05-20T01:17:29.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.348+0000 7f39022ae700 1 --2- 172.21.15.16:0/4169673627 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38e8004d90 0x7f38e8005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:29.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.348+0000 7f39012ac700 1 --2- 172.21.15.16:0/4169673627 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f38e8003ff0 0x7f38e8004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:29.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.348+0000 7f3901aad700 1 --2- 172.21.15.16:0/4169673627 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f38fc0cccf0 0x7f38e8003a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:29.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.348+0000 7f39022ae700 1 -- 172.21.15.16:0/4169673627 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f38fc0cccf0 msgr2=0x7f38e8003a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:29.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.348+0000 7f39022ae700 1 --2- 172.21.15.16:0/4169673627 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f38fc0cccf0 0x7f38e8003a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:29.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.348+0000 7f39022ae700 1 -- 172.21.15.16:0/4169673627 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f38e8003ff0 msgr2=0x7f38e8004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:29.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.348+0000 7f39022ae700 1 --2- 172.21.15.16:0/4169673627 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f38e8003ff0 0x7f38e8004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:29.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.348+0000 7f39022ae700 1 -- 172.21.15.16:0/4169673627 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38e80035b0 con 0x7f38e8004d90 2023-05-20T01:17:29.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.348+0000 7f3901aad700 1 --2- 172.21.15.16:0/4169673627 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f38fc0cccf0 0x7f38e8003a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:17:29.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.348+0000 7f39022ae700 1 --2- 172.21.15.16:0/4169673627 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38e8004d90 0x7f38e8005200 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f38f400a900 tx=0x7f38e8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8584b2877fe3d7d server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:29.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.349+0000 7f3900aab700 1 -- 172.21.15.16:0/4169673627 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f400a340 con 0x7f38e8004d90 2023-05-20T01:17:29.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.349+0000 7f3900aab700 1 -- 172.21.15.16:0/4169673627 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38f4012620 con 0x7f38e8004d90 2023-05-20T01:17:29.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.349+0000 7f3900aab700 1 -- 172.21.15.16:0/4169673627 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f40129d0 con 0x7f38e8004d90 2023-05-20T01:17:29.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.349+0000 7f3903d11700 1 -- 172.21.15.16:0/4169673627 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38e8004d90 msgr2=0x7f38e8005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:29.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.349+0000 7f3903d11700 1 --2- 172.21.15.16:0/4169673627 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38e8004d90 0x7f38e8005200 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f38f400a900 tx=0x7f38e8005880 comp rx=0 tx=0).stop 2023-05-20T01:17:29.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.349+0000 7f39022ae700 1 -- 172.21.15.16:0/4169673627 reap_dead start 2023-05-20T01:17:29.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.349+0000 7f3903d11700 1 -- 172.21.15.16:0/4169673627 shutdown_connections 2023-05-20T01:17:29.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.349+0000 7f3903d11700 1 -- 172.21.15.16:0/4169673627 >> 172.21.15.16:0/4169673627 conn(0x7f38fc0cffb0 msgr2=0x7f38fc0d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:29.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.349+0000 7f3903d11700 1 -- 172.21.15.16:0/4169673627 shutdown_connections 2023-05-20T01:17:29.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.349+0000 7f3903d11700 1 -- 172.21.15.16:0/4169673627 wait complete. 2023-05-20T01:17:29.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.350+0000 7f3903d11700 1 Processor -- start 2023-05-20T01:17:29.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.351+0000 7f3903d11700 1 -- start start 2023-05-20T01:17:29.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.351+0000 7f3903d11700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f38fc151b40 0x7f38fc0c3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:29.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.351+0000 7f3903d11700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38fc0b1510 0x7f38fc153460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:29.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.351+0000 7f3903d11700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f38fc1528c0 0x7f38fc1539a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:29.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.351+0000 7f3903d11700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38fc0b19a0 con 0x7f38fc0b1510 2023-05-20T01:17:29.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.351+0000 7f3903d11700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38fc0b1c80 con 0x7f38fc1528c0 2023-05-20T01:17:29.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.351+0000 7f3903d11700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38fc0b1f60 con 0x7f38fc151b40 2023-05-20T01:17:29.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.351+0000 7f39012ac700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38fc0b1510 0x7f38fc153460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:29.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.351+0000 7f39012ac700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38fc0b1510 0x7f38fc153460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59626/0 (socket says 172.21.15.16:59626) 2023-05-20T01:17:29.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.351+0000 7f39012ac700 1 -- 172.21.15.16:0/2240658607 learned_addr learned my addr 172.21.15.16:0/2240658607 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:29.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.351+0000 7f3901aad700 1 --2- 172.21.15.16:0/2240658607 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f38fc151b40 0x7f38fc0c3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:29.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.351+0000 7f39022ae700 1 --2- 172.21.15.16:0/2240658607 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f38fc1528c0 0x7f38fc1539a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:29.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.352+0000 7f39012ac700 1 -- 172.21.15.16:0/2240658607 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f38fc151b40 msgr2=0x7f38fc0c3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:29.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.352+0000 7f39012ac700 1 --2- 172.21.15.16:0/2240658607 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f38fc151b40 0x7f38fc0c3ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:29.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.352+0000 7f39012ac700 1 -- 172.21.15.16:0/2240658607 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f38fc1528c0 msgr2=0x7f38fc1539a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:29.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.352+0000 7f39012ac700 1 --2- 172.21.15.16:0/2240658607 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f38fc1528c0 0x7f38fc1539a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:29.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.352+0000 7f39012ac700 1 -- 172.21.15.16:0/2240658607 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38fc157560 con 0x7f38fc0b1510 2023-05-20T01:17:29.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.352+0000 7f39022ae700 1 --2- 172.21.15.16:0/2240658607 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f38fc1528c0 0x7f38fc1539a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:17:29.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.352+0000 7f39012ac700 1 --2- 172.21.15.16:0/2240658607 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38fc0b1510 0x7f38fc153460 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f38f80119a0 tx=0x7f38f8011cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:29.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.352+0000 7f38f27fc700 1 -- 172.21.15.16:0/2240658607 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f800f940 con 0x7f38fc0b1510 2023-05-20T01:17:29.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.352+0000 7f38f27fc700 1 -- 172.21.15.16:0/2240658607 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38f800faa0 con 0x7f38fc0b1510 2023-05-20T01:17:29.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.352+0000 7f38f27fc700 1 -- 172.21.15.16:0/2240658607 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f800fe30 con 0x7f38fc0b1510 2023-05-20T01:17:29.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.352+0000 7f3903d11700 1 -- 172.21.15.16:0/2240658607 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38fc0cb220 con 0x7f38fc0b1510 2023-05-20T01:17:29.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.352+0000 7f3903d11700 1 -- 172.21.15.16:0/2240658607 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38fc0cd120 con 0x7f38fc0b1510 2023-05-20T01:17:29.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.353+0000 7f3903d11700 1 -- 172.21.15.16:0/2240658607 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38fc0c4780 con 0x7f38fc0b1510 2023-05-20T01:17:29.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.354+0000 7f38f27fc700 1 -- 172.21.15.16:0/2240658607 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f38f8014440 con 0x7f38fc0b1510 2023-05-20T01:17:29.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.354+0000 7f38f27fc700 1 --2- 172.21.15.16:0/2240658607 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f38e006ace0 0x7f38e006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:29.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.354+0000 7f38f27fc700 1 -- 172.21.15.16:0/2240658607 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f38f8089de0 con 0x7f38fc0b1510 2023-05-20T01:17:29.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.354+0000 7f3901aad700 1 --2- 172.21.15.16:0/2240658607 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f38e006ace0 0x7f38e006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:29.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.354+0000 7f3901aad700 1 --2- 172.21.15.16:0/2240658607 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f38e006ace0 0x7f38e006d190 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f38fc0724b0 tx=0x7f38ec008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:29.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.358+0000 7f38f27fc700 1 -- 172.21.15.16:0/2240658607 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f38f8054ca0 con 0x7f38fc0b1510 2023-05-20T01:17:29.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.557+0000 7f3903d11700 1 -- 172.21.15.16:0/2240658607 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f38fc0747b0 con 0x7f38fc0b1510 2023-05-20T01:17:29.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.558+0000 7f38f27fc700 1 -- 172.21.15.16:0/2240658607 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+20121 (secure 0 0 0) 0x7f38f80582c0 con 0x7f38fc0b1510 2023-05-20T01:17:29.562 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:17:29.562 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":27,"fsid":"c2245cc5-ecab-4087-a59b-bc81db901676","created":"2023-05-20T01:16:09.549230+0000","modified":"2023-05-20T01:17:16.406712+0000","last_up_change":"2023-05-20T01:16:19.788624+0000","last_in_change":"2023-05-20T01:16:12.092160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-05-20T01:16:20.934366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-05-20T01:16:21.372294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-05-20T01:16:26.208287+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-05-20T01:16:27.292268+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-05-20T01:16:30.357285+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"317a83cd-19c8-4958-a301-6087ef342289","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6825","nonce":3923634179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6827","nonce":3923634179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6831","nonce":3923634179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6829","nonce":3923634179}]},"public_addr":"172.21.15.16:6825/3923634179","cluster_addr":"172.21.15.16:6827/3923634179","heartbeat_back_addr":"172.21.15.16:6831/3923634179","heartbeat_front_addr":"172.21.15.16:6829/3923634179","state":["exists","up"]},{"osd":1,"uuid":"5581046f-8219-4089-8d7d-b7846399505e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6809","nonce":3860264220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6811","nonce":3860264220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6815","nonce":3860264220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6813","nonce":3860264220}]},"public_addr":"172.21.15.16:6809/3860264220","cluster_addr":"172.21.15.16:6811/3860264220","heartbeat_back_addr":"172.21.15.16:6815/3860264220","heartbeat_front_addr":"172.21.15.16:6813/3860264220","state":["exists","up"]},{"osd":2,"uuid":"072e9e66-4133-478e-8d4a-64d46c2963f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6817","nonce":293187030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6819","nonce":293187030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6823","nonce":293187030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6821","nonce":293187030}]},"public_addr":"172.21.15.16:6817/293187030","cluster_addr":"172.21.15.16:6819/293187030","heartbeat_back_addr":"172.21.15.16:6823/293187030","heartbeat_front_addr":"172.21.15.16:6821/293187030","state":["exists","up"]},{"osd":3,"uuid":"11e1bba6-00f8-48dd-9028-d8d716b7f145","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6801","nonce":3669781890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6803","nonce":3669781890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6807","nonce":3669781890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6805","nonce":3669781890}]},"public_addr":"172.21.15.16:6801/3669781890","cluster_addr":"172.21.15.16:6803/3669781890","heartbeat_back_addr":"172.21.15.16:6807/3669781890","heartbeat_front_addr":"172.21.15.16:6805/3669781890","state":["exists","up"]},{"osd":4,"uuid":"b8909c3a-cf70-46cb-8d0b-009c707cf9d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6819","nonce":506317223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6823","nonce":506317223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6829","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6831","nonce":506317223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6827","nonce":506317223}]},"public_addr":"172.21.15.132:6819/506317223","cluster_addr":"172.21.15.132:6823/506317223","heartbeat_back_addr":"172.21.15.132:6831/506317223","heartbeat_front_addr":"172.21.15.132:6827/506317223","state":["exists","up"]},{"osd":5,"uuid":"b1818304-4583-47bf-a6ec-0bfbb5166233","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6818","nonce":1844689995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6822","nonce":1844689995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6830","nonce":1844689995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6826","nonce":1844689995}]},"public_addr":"172.21.15.132:6818/1844689995","cluster_addr":"172.21.15.132:6822/1844689995","heartbeat_back_addr":"172.21.15.132:6830/1844689995","heartbeat_front_addr":"172.21.15.132:6826/1844689995","state":["exists","up"]},{"osd":6,"uuid":"d37f4691-eed1-4a93-8818-18354f03f4a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6801","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6802","nonce":727581123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6805","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6807","nonce":727581123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6813","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6815","nonce":727581123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6811","nonce":727581123}]},"public_addr":"172.21.15.132:6802/727581123","cluster_addr":"172.21.15.132:6807/727581123","heartbeat_back_addr":"172.21.15.132:6815/727581123","heartbeat_front_addr":"172.21.15.132:6811/727581123","state":["exists","up"]},{"osd":7,"uuid":"4f712f62-5d1d-4816-8042-03209144f8b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6803","nonce":466001149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6806","nonce":466001149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6814","nonce":466001149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6810","nonce":466001149}]},"public_addr":"172.21.15.132:6803/466001149","cluster_addr":"172.21.15.132:6806/466001149","heartbeat_back_addr":"172.21.15.132:6814/466001149","heartbeat_front_addr":"172.21.15.132:6810/466001149","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.338863+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.278680+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305694+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.246835+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.276237+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.304441+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305421+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.267050+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.a","mappings":[{"from":1,"to":5}]},{"pgid":"4.11","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-20T01:17:29.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.560+0000 7f3903d11700 1 -- 172.21.15.16:0/2240658607 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f38e006ace0 msgr2=0x7f38e006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:29.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.560+0000 7f3903d11700 1 --2- 172.21.15.16:0/2240658607 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f38e006ace0 0x7f38e006d190 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f38fc0724b0 tx=0x7f38ec008040 comp rx=0 tx=0).stop 2023-05-20T01:17:29.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.560+0000 7f3903d11700 1 -- 172.21.15.16:0/2240658607 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38fc0b1510 msgr2=0x7f38fc153460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:29.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.560+0000 7f3903d11700 1 --2- 172.21.15.16:0/2240658607 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38fc0b1510 0x7f38fc153460 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f38f80119a0 tx=0x7f38f8011cb0 comp rx=0 tx=0).stop 2023-05-20T01:17:29.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.560+0000 7f3903d11700 1 -- 172.21.15.16:0/2240658607 shutdown_connections 2023-05-20T01:17:29.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.560+0000 7f3903d11700 1 --2- 172.21.15.16:0/2240658607 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f38fc1528c0 0x7f38fc1539a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:29.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.560+0000 7f3903d11700 1 --2- 172.21.15.16:0/2240658607 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f38e006ace0 0x7f38e006d190 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:29.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.560+0000 7f3903d11700 1 --2- 172.21.15.16:0/2240658607 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f38fc151b40 0x7f38fc0c3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:29.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.560+0000 7f3903d11700 1 --2- 172.21.15.16:0/2240658607 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38fc0b1510 0x7f38fc153460 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:29.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.560+0000 7f3903d11700 1 -- 172.21.15.16:0/2240658607 >> 172.21.15.16:0/2240658607 conn(0x7f38fc0cffb0 msgr2=0x7f38fc0d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:29.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.560+0000 7f3903d11700 1 -- 172.21.15.16:0/2240658607 shutdown_connections 2023-05-20T01:17:29.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.560+0000 7f3903d11700 1 -- 172.21.15.16:0/2240658607 wait complete. 2023-05-20T01:17:29.576 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-05-20T01:16:20.934366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-05-20T01:16:21.372294+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-05-20T01:16:26.208287+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-05-20T01:16:27.292268+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-05-20T01:16:30.357285+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-05-20T01:17:29.577 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-05-20T01:17:29.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.701+0000 7fcee4d0f700 1 Processor -- start 2023-05-20T01:17:29.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.701+0000 7fcee4d0f700 1 -- start start 2023-05-20T01:17:29.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.702+0000 7fcee4d0f700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcee015cec0 0x7fcee00cd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:29.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.702+0000 7fcee4d0f700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcee006e190 0x7fcee00cf1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:29.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.702+0000 7fcee4d0f700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fcee006a510 0x7fcee006a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:29.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.702+0000 7fcee4d0f700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcee006aed0 con 0x7fcee006e190 2023-05-20T01:17:29.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.702+0000 7fcee4d0f700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcee006dd00 con 0x7fcee006a510 2023-05-20T01:17:29.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.702+0000 7fcee4d0f700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcee00cfa60 con 0x7fcee015cec0 2023-05-20T01:17:29.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.702+0000 7fceddd9b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcee006e190 0x7fcee00cf1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:29.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.702+0000 7fceddd9b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcee006e190 0x7fcee00cf1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59632/0 (socket says 172.21.15.16:59632) 2023-05-20T01:17:29.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.702+0000 7fceddd9b700 1 -- 172.21.15.16:0/4286436870 learned_addr learned my addr 172.21.15.16:0/4286436870 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:29.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.702+0000 7fceddd9b700 1 -- 172.21.15.16:0/4286436870 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcee015cec0 msgr2=0x7fcee00cd4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:29.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.702+0000 7fceddd9b700 1 --2- 172.21.15.16:0/4286436870 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcee015cec0 0x7fcee00cd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:29.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.702+0000 7fceddd9b700 1 -- 172.21.15.16:0/4286436870 >> v2:172.21.15.132:3300/0 conn(0x7fcee006a510 msgr2=0x7fcee006a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:29.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.702+0000 7fceded9d700 1 --2- 172.21.15.16:0/4286436870 >> v2:172.21.15.132:3300/0 conn(0x7fcee006a510 0x7fcee006a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:29.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.702+0000 7fceddd9b700 1 --2- 172.21.15.16:0/4286436870 >> v2:172.21.15.132:3300/0 conn(0x7fcee006a510 0x7fcee006a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:29.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.702+0000 7fceddd9b700 1 -- 172.21.15.16:0/4286436870 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcee006d6b0 con 0x7fcee006e190 2023-05-20T01:17:29.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.702+0000 7fceded9d700 1 --2- 172.21.15.16:0/4286436870 >> v2:172.21.15.132:3300/0 conn(0x7fcee006a510 0x7fcee006a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:17:29.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.702+0000 7fceddd9b700 1 --2- 172.21.15.16:0/4286436870 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcee006e190 0x7fcee00cf1f0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fcee00b2ba0 tx=0x7fced0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb3414a8e3a3d4c7 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:29.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.703+0000 7fcedd59a700 1 -- 172.21.15.16:0/4286436870 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced000b870 con 0x7fcee006e190 2023-05-20T01:17:29.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.703+0000 7fcedd59a700 1 -- 172.21.15.16:0/4286436870 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fced000b9d0 con 0x7fcee006e190 2023-05-20T01:17:29.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.703+0000 7fcedd59a700 1 -- 172.21.15.16:0/4286436870 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced000bd60 con 0x7fcee006e190 2023-05-20T01:17:29.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.703+0000 7fcee4d0f700 1 -- 172.21.15.16:0/4286436870 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcee006e190 msgr2=0x7fcee00cf1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:29.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.703+0000 7fcee4d0f700 1 --2- 172.21.15.16:0/4286436870 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcee006e190 0x7fcee00cf1f0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fcee00b2ba0 tx=0x7fced0009bd0 comp rx=0 tx=0).stop 2023-05-20T01:17:29.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.703+0000 7fcee4d0f700 1 -- 172.21.15.16:0/4286436870 shutdown_connections 2023-05-20T01:17:29.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.703+0000 7fcee4d0f700 1 --2- 172.21.15.16:0/4286436870 >> v2:172.21.15.132:3300/0 conn(0x7fcee006a510 0x7fcee006a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:29.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.703+0000 7fcee4d0f700 1 --2- 172.21.15.16:0/4286436870 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcee015cec0 0x7fcee00cd4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:29.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.703+0000 7fcee4d0f700 1 --2- 172.21.15.16:0/4286436870 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcee006e190 0x7fcee00cf1f0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:29.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.703+0000 7fcee4d0f700 1 -- 172.21.15.16:0/4286436870 >> 172.21.15.16:0/4286436870 conn(0x7fcee006c260 msgr2=0x7fcee00cffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:29.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.703+0000 7fcee4d0f700 1 -- 172.21.15.16:0/4286436870 shutdown_connections 2023-05-20T01:17:29.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.703+0000 7fcee4d0f700 1 -- 172.21.15.16:0/4286436870 wait complete. 2023-05-20T01:17:29.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.705+0000 7fcee4d0f700 1 Processor -- start 2023-05-20T01:17:29.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.705+0000 7fcee4d0f700 1 -- start start 2023-05-20T01:17:29.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.705+0000 7fcee4d0f700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcee006a510 0x7fcee015fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:29.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.705+0000 7fcee4d0f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcee006e190 0x7fcee01f5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:29.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.705+0000 7fcee4d0f700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcee015cec0 0x7fcee00c9660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:29.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.705+0000 7fcee4d0f700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcee015ed90 con 0x7fcee006e190 2023-05-20T01:17:29.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.705+0000 7fcee4d0f700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcee015de30 con 0x7fcee006a510 2023-05-20T01:17:29.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.705+0000 7fcee4d0f700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcee015e110 con 0x7fcee015cec0 2023-05-20T01:17:29.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.705+0000 7fceddd9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcee006e190 0x7fcee01f5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:29.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.706+0000 7fceddd9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcee006e190 0x7fcee01f5bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59638/0 (socket says 172.21.15.16:59638) 2023-05-20T01:17:29.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.706+0000 7fceddd9b700 1 -- 172.21.15.16:0/4108911924 learned_addr learned my addr 172.21.15.16:0/4108911924 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:29.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.706+0000 7fceded9d700 1 --2- 172.21.15.16:0/4108911924 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcee015cec0 0x7fcee00c9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:29.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.706+0000 7fceddd9b700 1 -- 172.21.15.16:0/4108911924 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcee015cec0 msgr2=0x7fcee00c9660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:29.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.706+0000 7fceddd9b700 1 --2- 172.21.15.16:0/4108911924 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcee015cec0 0x7fcee00c9660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:29.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.706+0000 7fcede59c700 1 --2- 172.21.15.16:0/4108911924 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcee006a510 0x7fcee015fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:29.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.706+0000 7fceddd9b700 1 -- 172.21.15.16:0/4108911924 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcee006a510 msgr2=0x7fcee015fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:29.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.706+0000 7fceddd9b700 1 --2- 172.21.15.16:0/4108911924 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcee006a510 0x7fcee015fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:29.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.706+0000 7fceddd9b700 1 -- 172.21.15.16:0/4108911924 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fced0009550 con 0x7fcee006e190 2023-05-20T01:17:29.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.706+0000 7fceddd9b700 1 --2- 172.21.15.16:0/4108911924 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcee006e190 0x7fcee01f5bc0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fcee00cdb50 tx=0x7fced0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:29.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.706+0000 7fceceffd700 1 -- 172.21.15.16:0/4108911924 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced0004310 con 0x7fcee006e190 2023-05-20T01:17:29.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.706+0000 7fceceffd700 1 -- 172.21.15.16:0/4108911924 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fced001f070 con 0x7fcee006e190 2023-05-20T01:17:29.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.706+0000 7fcee4d0f700 1 -- 172.21.15.16:0/4108911924 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcee015d290 con 0x7fcee006e190 2023-05-20T01:17:29.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.706+0000 7fceceffd700 1 -- 172.21.15.16:0/4108911924 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced00036a0 con 0x7fcee006e190 2023-05-20T01:17:29.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.706+0000 7fcee4d0f700 1 -- 172.21.15.16:0/4108911924 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcee006d990 con 0x7fcee006e190 2023-05-20T01:17:29.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.707+0000 7fcee4d0f700 1 -- 172.21.15.16:0/4108911924 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcee00cfd40 con 0x7fcee006e190 2023-05-20T01:17:29.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.708+0000 7fceceffd700 1 -- 172.21.15.16:0/4108911924 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fced0021030 con 0x7fcee006e190 2023-05-20T01:17:29.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.708+0000 7fceceffd700 1 --2- 172.21.15.16:0/4108911924 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fcec006ace0 0x7fcec006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:29.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.708+0000 7fceceffd700 1 -- 172.21.15.16:0/4108911924 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fced008ab80 con 0x7fcee006e190 2023-05-20T01:17:29.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.708+0000 7fcede59c700 1 --2- 172.21.15.16:0/4108911924 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fcec006ace0 0x7fcec006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:29.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.708+0000 7fcede59c700 1 --2- 172.21.15.16:0/4108911924 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fcec006ace0 0x7fcec006d190 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fced4006fd0 tx=0x7fced4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:29.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.712+0000 7fceceffd700 1 -- 172.21.15.16:0/4108911924 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fced0055990 con 0x7fcee006e190 2023-05-20T01:17:29.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.906+0000 7fcee4d0f700 1 -- 172.21.15.16:0/4108911924 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fcee00747b0 con 0x7fcee006e190 2023-05-20T01:17:29.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.907+0000 7fceceffd700 1 -- 172.21.15.16:0/4108911924 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v27) v1 ==== 92+0+10 (secure 0 0 0) 0x7fced0058fb0 con 0x7fcee006e190 2023-05-20T01:17:29.911 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 8 2023-05-20T01:17:29.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.909+0000 7fcee4d0f700 1 -- 172.21.15.16:0/4108911924 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fcec006ace0 msgr2=0x7fcec006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:29.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.909+0000 7fcee4d0f700 1 --2- 172.21.15.16:0/4108911924 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fcec006ace0 0x7fcec006d190 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fced4006fd0 tx=0x7fced4008040 comp rx=0 tx=0).stop 2023-05-20T01:17:29.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.909+0000 7fcee4d0f700 1 -- 172.21.15.16:0/4108911924 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcee006e190 msgr2=0x7fcee01f5bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:29.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.909+0000 7fcee4d0f700 1 --2- 172.21.15.16:0/4108911924 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcee006e190 0x7fcee01f5bc0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fcee00cdb50 tx=0x7fced0004750 comp rx=0 tx=0).stop 2023-05-20T01:17:29.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.909+0000 7fcee4d0f700 1 -- 172.21.15.16:0/4108911924 shutdown_connections 2023-05-20T01:17:29.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.909+0000 7fcee4d0f700 1 --2- 172.21.15.16:0/4108911924 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fcec006ace0 0x7fcec006d190 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:29.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.909+0000 7fcee4d0f700 1 --2- 172.21.15.16:0/4108911924 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcee015cec0 0x7fcee00c9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:29.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.909+0000 7fcee4d0f700 1 --2- 172.21.15.16:0/4108911924 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcee006a510 0x7fcee015fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:29.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.909+0000 7fcee4d0f700 1 --2- 172.21.15.16:0/4108911924 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcee006e190 0x7fcee01f5bc0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:29.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.909+0000 7fcee4d0f700 1 -- 172.21.15.16:0/4108911924 >> 172.21.15.16:0/4108911924 conn(0x7fcee006c260 msgr2=0x7fcee00b3060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:29.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.909+0000 7fcee4d0f700 1 -- 172.21.15.16:0/4108911924 shutdown_connections 2023-05-20T01:17:29.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:29.909+0000 7fcee4d0f700 1 -- 172.21.15.16:0/4108911924 wait complete. 2023-05-20T01:17:29.925 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-05-20T01:17:30.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.050+0000 7f25a1fb1700 1 Processor -- start 2023-05-20T01:17:30.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.050+0000 7f25a1fb1700 1 -- start start 2023-05-20T01:17:30.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.051+0000 7f25a1fb1700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f259c0cd070 0x7f259c06e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:30.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.051+0000 7f25a1fb1700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f259c072430 con 0x7f259c15cf40 2023-05-20T01:17:30.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.051+0000 7f25a1fb1700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f259c071570 con 0x7f259c0710d0 2023-05-20T01:17:30.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.051+0000 7f25a1fb1700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f259c071820 con 0x7f259c0cd070 2023-05-20T01:17:30.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.051+0000 7f259b7fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f259c15cf40 0x7f259c0cc960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:42940/0 (socket says 172.21.15.16:42940) 2023-05-20T01:17:30.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.051+0000 7f259b7fe700 1 -- 172.21.15.16:0/77115417 learned_addr learned my addr 172.21.15.16:0/77115417 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:17:30.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.051+0000 7f259affd700 1 --2- 172.21.15.16:0/77115417 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f259c0cd070 0x7f259c06e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:30.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.051+0000 7f259a7fc700 1 -- 172.21.15.16:0/77115417 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3744126830 0 0) 0x7f259c072430 con 0x7f259c15cf40 2023-05-20T01:17:30.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.051+0000 7f259a7fc700 1 -- 172.21.15.16:0/77115417 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f25840035b0 con 0x7f259c15cf40 2023-05-20T01:17:30.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.051+0000 7f259a7fc700 1 -- 172.21.15.16:0/77115417 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2542254490 0 0) 0x7f25840035b0 con 0x7f259c15cf40 2023-05-20T01:17:30.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.051+0000 7f259a7fc700 1 -- 172.21.15.16:0/77115417 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f259c0cd070 msgr2=0x7f259c06e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:30.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.051+0000 7f259a7fc700 1 --2- 172.21.15.16:0/77115417 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f259c0cd070 0x7f259c06e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:30.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.051+0000 7f259a7fc700 1 -- 172.21.15.16:0/77115417 >> v1:172.21.15.132:6789/0 conn(0x7f259c0710d0 legacy=0x7f259c0cabb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:30.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.051+0000 7f259a7fc700 1 -- 172.21.15.16:0/77115417 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f259c071b00 con 0x7f259c15cf40 2023-05-20T01:17:30.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.051+0000 7f259a7fc700 1 -- 172.21.15.16:0/77115417 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7f258c001670 con 0x7f259c15cf40 2023-05-20T01:17:30.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.051+0000 7f259a7fc700 1 -- 172.21.15.16:0/77115417 >> v1:172.21.15.16:6789/0 conn(0x7f259c15cf40 legacy=0x7f259c0cc960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:30.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.051+0000 7f259a7fc700 1 --2- 172.21.15.16:0/77115417 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f259c0cd070 0x7f2584003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:30.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.051+0000 7f259a7fc700 1 --2- 172.21.15.16:0/77115417 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2584003f70 0x7f2584004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:30.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.051+0000 7f259a7fc700 1 --2- 172.21.15.16:0/77115417 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2584004d90 0x7f2584005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:30.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.052+0000 7f259a7fc700 1 -- 172.21.15.16:0/77115417 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f259c071820 con 0x7f2584004d90 2023-05-20T01:17:30.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.052+0000 7f259a7fc700 1 -- 172.21.15.16:0/77115417 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f258400ba40 con 0x7f2584003f70 2023-05-20T01:17:30.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.052+0000 7f259a7fc700 1 -- 172.21.15.16:0/77115417 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f258400bc00 con 0x7f259c0cd070 2023-05-20T01:17:30.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.052+0000 7f259bfff700 1 --2- 172.21.15.16:0/77115417 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2584004d90 0x7f2584005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:30.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.052+0000 7f259b7fe700 1 --2- 172.21.15.16:0/77115417 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f259c0cd070 0x7f2584003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:30.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.052+0000 7f259affd700 1 --2- 172.21.15.16:0/77115417 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2584003f70 0x7f2584004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:30.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.052+0000 7f259bfff700 1 -- 172.21.15.16:0/77115417 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f259c0cd070 msgr2=0x7f2584003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:30.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.052+0000 7f259bfff700 1 --2- 172.21.15.16:0/77115417 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f259c0cd070 0x7f2584003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:30.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.052+0000 7f259bfff700 1 -- 172.21.15.16:0/77115417 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2584003f70 msgr2=0x7f2584004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:30.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.052+0000 7f259bfff700 1 --2- 172.21.15.16:0/77115417 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2584003f70 0x7f2584004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:30.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.052+0000 7f259bfff700 1 -- 172.21.15.16:0/77115417 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25840035b0 con 0x7f2584004d90 2023-05-20T01:17:30.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.052+0000 7f259bfff700 1 --2- 172.21.15.16:0/77115417 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2584004d90 0x7f2584005200 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f2594002220 tx=0x7f2584005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77b00b89aeccb6e8 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:30.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.052+0000 7f259a7fc700 1 -- 172.21.15.16:0/77115417 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f259400ce60 con 0x7f2584004d90 2023-05-20T01:17:30.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.052+0000 7f259a7fc700 1 -- 172.21.15.16:0/77115417 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2594011470 con 0x7f2584004d90 2023-05-20T01:17:30.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.052+0000 7f259a7fc700 1 -- 172.21.15.16:0/77115417 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2594011800 con 0x7f2584004d90 2023-05-20T01:17:30.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.053+0000 7f25a1fb1700 1 -- 172.21.15.16:0/77115417 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2584004d90 msgr2=0x7f2584005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:30.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.053+0000 7f25a1fb1700 1 --2- 172.21.15.16:0/77115417 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2584004d90 0x7f2584005200 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f2594002220 tx=0x7f2584005880 comp rx=0 tx=0).stop 2023-05-20T01:17:30.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.053+0000 7f259bfff700 1 -- 172.21.15.16:0/77115417 reap_dead start 2023-05-20T01:17:30.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.053+0000 7f25a1fb1700 1 -- 172.21.15.16:0/77115417 shutdown_connections 2023-05-20T01:17:30.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.053+0000 7f25a1fb1700 1 -- 172.21.15.16:0/77115417 >> 172.21.15.16:0/77115417 conn(0x7f259c06c260 msgr2=0x7f259c0cffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:30.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.053+0000 7f25a1fb1700 1 -- 172.21.15.16:0/77115417 shutdown_connections 2023-05-20T01:17:30.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.053+0000 7f25a1fb1700 1 -- 172.21.15.16:0/77115417 wait complete. 2023-05-20T01:17:30.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.054+0000 7f25a1fb1700 1 Processor -- start 2023-05-20T01:17:30.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.055+0000 7f25a1fb1700 1 -- start start 2023-05-20T01:17:30.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.056+0000 7f25a1fb1700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2584004d90 0x7f259c0c9620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:30.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.056+0000 7f25a1fb1700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f259c0c80f0 0x7f259c0c6440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:30.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.056+0000 7f25a1fb1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f259c0b2f90 0x7f259c13fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:30.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.056+0000 7f25a1fb1700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f259c0c6980 con 0x7f259c0b2f90 2023-05-20T01:17:30.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.056+0000 7f25a1fb1700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f259c0c8580 con 0x7f259c0c80f0 2023-05-20T01:17:30.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.056+0000 7f25a1fb1700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f259c0c8860 con 0x7f2584004d90 2023-05-20T01:17:30.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.056+0000 7f259bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f259c0b2f90 0x7f259c13fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:30.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.056+0000 7f259bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f259c0b2f90 0x7f259c13fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59660/0 (socket says 172.21.15.16:59660) 2023-05-20T01:17:30.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.056+0000 7f259bfff700 1 -- 172.21.15.16:0/1156120227 learned_addr learned my addr 172.21.15.16:0/1156120227 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:30.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.056+0000 7f259b7fe700 1 --2- 172.21.15.16:0/1156120227 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2584004d90 0x7f259c0c9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:30.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.056+0000 7f259bfff700 1 -- 172.21.15.16:0/1156120227 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2584004d90 msgr2=0x7f259c0c9620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:30.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.056+0000 7f259bfff700 1 --2- 172.21.15.16:0/1156120227 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2584004d90 0x7f259c0c9620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:30.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.056+0000 7f259affd700 1 --2- 172.21.15.16:0/1156120227 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f259c0c80f0 0x7f259c0c6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:30.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.056+0000 7f259bfff700 1 -- 172.21.15.16:0/1156120227 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f259c0c80f0 msgr2=0x7f259c0c6440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:30.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.056+0000 7f259bfff700 1 --2- 172.21.15.16:0/1156120227 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f259c0c80f0 0x7f259c0c6440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:30.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.056+0000 7f259bfff700 1 -- 172.21.15.16:0/1156120227 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f259c0720c0 con 0x7f259c0b2f90 2023-05-20T01:17:30.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.056+0000 7f259b7fe700 1 --2- 172.21.15.16:0/1156120227 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2584004d90 0x7f259c0c9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:17:30.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.056+0000 7f259affd700 1 --2- 172.21.15.16:0/1156120227 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f259c0c80f0 0x7f259c0c6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:17:30.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.056+0000 7f259bfff700 1 --2- 172.21.15.16:0/1156120227 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f259c0b2f90 0x7f259c13fbb0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f258400ba10 tx=0x7f2594001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:30.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.056+0000 7f2583fff700 1 -- 172.21.15.16:0/1156120227 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2594020070 con 0x7f259c0b2f90 2023-05-20T01:17:30.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.056+0000 7f2583fff700 1 -- 172.21.15.16:0/1156120227 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2594012e60 con 0x7f259c0b2f90 2023-05-20T01:17:30.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.056+0000 7f25a1fb1700 1 -- 172.21.15.16:0/1156120227 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f259c071de0 con 0x7f259c0b2f90 2023-05-20T01:17:30.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.056+0000 7f2583fff700 1 -- 172.21.15.16:0/1156120227 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2594003c20 con 0x7f259c0b2f90 2023-05-20T01:17:30.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.056+0000 7f25a1fb1700 1 -- 172.21.15.16:0/1156120227 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f259c06e020 con 0x7f259c0b2f90 2023-05-20T01:17:30.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.057+0000 7f25a1fb1700 1 -- 172.21.15.16:0/1156120227 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f259c0c9b60 con 0x7f259c0b2f90 2023-05-20T01:17:30.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.058+0000 7f2583fff700 1 -- 172.21.15.16:0/1156120227 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2594005070 con 0x7f259c0b2f90 2023-05-20T01:17:30.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.058+0000 7f2583fff700 1 --2- 172.21.15.16:0/1156120227 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f258806ace0 0x7f258806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:30.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.058+0000 7f2583fff700 1 -- 172.21.15.16:0/1156120227 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f259408c940 con 0x7f259c0b2f90 2023-05-20T01:17:30.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.058+0000 7f259b7fe700 1 --2- 172.21.15.16:0/1156120227 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f258806ace0 0x7f258806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:30.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.059+0000 7f259b7fe700 1 --2- 172.21.15.16:0/1156120227 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f258806ace0 0x7f258806d190 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f259c14ebb0 tx=0x7f258c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:30.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.063+0000 7f2583fff700 1 -- 172.21.15.16:0/1156120227 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f25940576d0 con 0x7f259c0b2f90 2023-05-20T01:17:30.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.258+0000 7f25a1fb1700 1 -- 172.21.15.16:0/1156120227 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f259c0747b0 con 0x7f259c0b2f90 2023-05-20T01:17:30.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.258+0000 7f2583fff700 1 -- 172.21.15.16:0/1156120227 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f259405acf0 con 0x7f259c0b2f90 2023-05-20T01:17:30.262 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-05-20T01:17:30.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.260+0000 7f25a1fb1700 1 -- 172.21.15.16:0/1156120227 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f258806ace0 msgr2=0x7f258806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:30.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.260+0000 7f25a1fb1700 1 --2- 172.21.15.16:0/1156120227 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f258806ace0 0x7f258806d190 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f259c14ebb0 tx=0x7f258c009040 comp rx=0 tx=0).stop 2023-05-20T01:17:30.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.260+0000 7f25a1fb1700 1 -- 172.21.15.16:0/1156120227 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f259c0b2f90 msgr2=0x7f259c13fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:30.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.260+0000 7f25a1fb1700 1 --2- 172.21.15.16:0/1156120227 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f259c0b2f90 0x7f259c13fbb0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f258400ba10 tx=0x7f2594001630 comp rx=0 tx=0).stop 2023-05-20T01:17:30.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.260+0000 7f25a1fb1700 1 -- 172.21.15.16:0/1156120227 shutdown_connections 2023-05-20T01:17:30.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.260+0000 7f25a1fb1700 1 --2- 172.21.15.16:0/1156120227 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f259c0b2f90 0x7f259c13fbb0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:30.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.260+0000 7f25a1fb1700 1 --2- 172.21.15.16:0/1156120227 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f258806ace0 0x7f258806d190 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:30.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.260+0000 7f25a1fb1700 1 --2- 172.21.15.16:0/1156120227 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2584004d90 0x7f259c0c9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:30.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.260+0000 7f25a1fb1700 1 --2- 172.21.15.16:0/1156120227 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f259c0c80f0 0x7f259c0c6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:30.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.260+0000 7f25a1fb1700 1 -- 172.21.15.16:0/1156120227 >> 172.21.15.16:0/1156120227 conn(0x7f259c06c260 msgr2=0x7f259c06a510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:30.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.260+0000 7f25a1fb1700 1 -- 172.21.15.16:0/1156120227 shutdown_connections 2023-05-20T01:17:30.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.261+0000 7f25a1fb1700 1 -- 172.21.15.16:0/1156120227 wait complete. 2023-05-20T01:17:30.276 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-05-20T01:17:30.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.402+0000 7f56d1f66700 1 Processor -- start 2023-05-20T01:17:30.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.402+0000 7f56d1f66700 1 -- start start 2023-05-20T01:17:30.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.403+0000 7f56d1f66700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f56cc15ab80 0x7f56cc0cda80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:30.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.403+0000 7f56d1f66700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f56cc0cdfc0 0x7f56cc0cec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:30.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.403+0000 7f56d1f66700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56cc0ce4d0 con 0x7f56cc0cf1c0 2023-05-20T01:17:30.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.403+0000 7f56d1f66700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56cc0cb700 con 0x7f56cc15ab80 2023-05-20T01:17:30.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.403+0000 7f56d1f66700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56cc0cf640 con 0x7f56cc0cdfc0 2023-05-20T01:17:30.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.403+0000 7f56cbfff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f56cc0cf1c0 0x7f56cc15e4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:42964/0 (socket says 172.21.15.16:42964) 2023-05-20T01:17:30.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.403+0000 7f56cbfff700 1 -- 172.21.15.16:0/2557254771 learned_addr learned my addr 172.21.15.16:0/2557254771 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:17:30.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.403+0000 7f56ca7fc700 1 -- 172.21.15.16:0/2557254771 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3922261527 0 0) 0x7f56cc0ce4d0 con 0x7f56cc0cf1c0 2023-05-20T01:17:30.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.403+0000 7f56ca7fc700 1 -- 172.21.15.16:0/2557254771 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56b40035b0 con 0x7f56cc0cf1c0 2023-05-20T01:17:30.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.403+0000 7f56caffd700 1 --2- 172.21.15.16:0/2557254771 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f56cc0cdfc0 0x7f56cc0cec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:30.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.403+0000 7f56cb7fe700 1 --2- 172.21.15.16:0/2557254771 >> v2:172.21.15.132:3300/0 conn(0x7f56cc15ab80 0x7f56cc0cda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:30.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.403+0000 7f56ca7fc700 1 -- 172.21.15.16:0/2557254771 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3226778770 0 0) 0x7f56b40035b0 con 0x7f56cc0cf1c0 2023-05-20T01:17:30.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.403+0000 7f56ca7fc700 1 -- 172.21.15.16:0/2557254771 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f56cc0cdfc0 msgr2=0x7f56cc0cec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:30.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.403+0000 7f56ca7fc700 1 --2- 172.21.15.16:0/2557254771 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f56cc0cdfc0 0x7f56cc0cec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:30.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.403+0000 7f56ca7fc700 1 -- 172.21.15.16:0/2557254771 >> v2:172.21.15.132:3300/0 conn(0x7f56cc15ab80 msgr2=0x7f56cc0cda80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:30.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.403+0000 7f56ca7fc700 1 --2- 172.21.15.16:0/2557254771 >> v2:172.21.15.132:3300/0 conn(0x7f56cc15ab80 0x7f56cc0cda80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:30.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.403+0000 7f56ca7fc700 1 -- 172.21.15.16:0/2557254771 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56cc0ce7b0 con 0x7f56cc0cf1c0 2023-05-20T01:17:30.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.403+0000 7f56ca7fc700 1 -- 172.21.15.16:0/2557254771 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7f56c40019f0 con 0x7f56cc0cf1c0 2023-05-20T01:17:30.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.403+0000 7f56ca7fc700 1 -- 172.21.15.16:0/2557254771 >> v1:172.21.15.16:6789/0 conn(0x7f56cc0cf1c0 legacy=0x7f56cc15e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:30.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.403+0000 7f56ca7fc700 1 --2- 172.21.15.16:0/2557254771 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56b4003a00 0x7f56b4003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:30.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.403+0000 7f56ca7fc700 1 --2- 172.21.15.16:0/2557254771 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f56b4004850 0x7f56b4004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:30.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.403+0000 7f56ca7fc700 1 --2- 172.21.15.16:0/2557254771 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f56cc0cdfc0 0x7f56b4005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:30.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.404+0000 7f56ca7fc700 1 -- 172.21.15.16:0/2557254771 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56cc0cb700 con 0x7f56b4003a00 2023-05-20T01:17:30.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.404+0000 7f56ca7fc700 1 -- 172.21.15.16:0/2557254771 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56cc0cf640 con 0x7f56b4004850 2023-05-20T01:17:30.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.404+0000 7f56ca7fc700 1 -- 172.21.15.16:0/2557254771 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56b400bac0 con 0x7f56cc0cdfc0 2023-05-20T01:17:30.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.404+0000 7f56cb7fe700 1 --2- 172.21.15.16:0/2557254771 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56b4003a00 0x7f56b4003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:30.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.404+0000 7f56caffd700 1 --2- 172.21.15.16:0/2557254771 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f56b4004850 0x7f56b4004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:30.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.404+0000 7f56cbfff700 1 --2- 172.21.15.16:0/2557254771 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f56cc0cdfc0 0x7f56b4005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:30.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.404+0000 7f56cb7fe700 1 -- 172.21.15.16:0/2557254771 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f56cc0cdfc0 msgr2=0x7f56b4005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:30.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.404+0000 7f56cb7fe700 1 --2- 172.21.15.16:0/2557254771 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f56cc0cdfc0 0x7f56b4005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:30.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.404+0000 7f56cb7fe700 1 -- 172.21.15.16:0/2557254771 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f56b4004850 msgr2=0x7f56b4004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:30.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.404+0000 7f56cb7fe700 1 --2- 172.21.15.16:0/2557254771 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f56b4004850 0x7f56b4004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:30.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.404+0000 7f56cb7fe700 1 -- 172.21.15.16:0/2557254771 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56b40035b0 con 0x7f56b4003a00 2023-05-20T01:17:30.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.404+0000 7f56cbfff700 1 --2- 172.21.15.16:0/2557254771 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f56cc0cdfc0 0x7f56b4005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:17:30.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.404+0000 7f56cb7fe700 1 --2- 172.21.15.16:0/2557254771 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56b4003a00 0x7f56b4003e10 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f56bc009890 tx=0x7f56b4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3790f1103b41dc91 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:30.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.404+0000 7f56ca7fc700 1 -- 172.21.15.16:0/2557254771 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56bc00c070 con 0x7f56b4003a00 2023-05-20T01:17:30.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.404+0000 7f56ca7fc700 1 -- 172.21.15.16:0/2557254771 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56bc013610 con 0x7f56b4003a00 2023-05-20T01:17:30.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.404+0000 7f56ca7fc700 1 -- 172.21.15.16:0/2557254771 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56bc0139a0 con 0x7f56b4003a00 2023-05-20T01:17:30.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.405+0000 7f56d1f66700 1 -- 172.21.15.16:0/2557254771 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56b4003a00 msgr2=0x7f56b4003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:30.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.405+0000 7f56d1f66700 1 --2- 172.21.15.16:0/2557254771 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56b4003a00 0x7f56b4003e10 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f56bc009890 tx=0x7f56b4005880 comp rx=0 tx=0).stop 2023-05-20T01:17:30.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.405+0000 7f56cbfff700 1 -- 172.21.15.16:0/2557254771 reap_dead start 2023-05-20T01:17:30.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.405+0000 7f56d1f66700 1 -- 172.21.15.16:0/2557254771 shutdown_connections 2023-05-20T01:17:30.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.405+0000 7f56d1f66700 1 -- 172.21.15.16:0/2557254771 >> 172.21.15.16:0/2557254771 conn(0x7f56cc06ae30 msgr2=0x7f56cc09a400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:30.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.405+0000 7f56d1f66700 1 -- 172.21.15.16:0/2557254771 shutdown_connections 2023-05-20T01:17:30.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.405+0000 7f56d1f66700 1 -- 172.21.15.16:0/2557254771 wait complete. 2023-05-20T01:17:30.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.406+0000 7f56d1f66700 1 Processor -- start 2023-05-20T01:17:30.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.406+0000 7f56d1f66700 1 -- start start 2023-05-20T01:17:30.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.407+0000 7f56d1f66700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f56b4003a00 0x7f56cc1efaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:30.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.407+0000 7f56d1f66700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f56cc144be0 0x7f56cc0b5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:30.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.407+0000 7f56d1f66700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56cc0b61b0 0x7f56cc0c6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:30.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.407+0000 7f56d1f66700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56cc0b6f10 con 0x7f56cc0b61b0 2023-05-20T01:17:30.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.407+0000 7f56d1f66700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56cc0b6670 con 0x7f56cc144be0 2023-05-20T01:17:30.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.407+0000 7f56d1f66700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56cc0b6950 con 0x7f56b4003a00 2023-05-20T01:17:30.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.407+0000 7f56cbfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56cc0b61b0 0x7f56cc0c6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:30.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.407+0000 7f56cbfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56cc0b61b0 0x7f56cc0c6bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59680/0 (socket says 172.21.15.16:59680) 2023-05-20T01:17:30.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.407+0000 7f56cbfff700 1 -- 172.21.15.16:0/1808527149 learned_addr learned my addr 172.21.15.16:0/1808527149 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:30.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.407+0000 7f56cb7fe700 1 --2- 172.21.15.16:0/1808527149 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f56b4003a00 0x7f56cc1efaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:30.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.407+0000 7f56caffd700 1 --2- 172.21.15.16:0/1808527149 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f56cc144be0 0x7f56cc0b5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:30.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.407+0000 7f56cbfff700 1 -- 172.21.15.16:0/1808527149 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f56b4003a00 msgr2=0x7f56cc1efaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:30.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.407+0000 7f56cbfff700 1 --2- 172.21.15.16:0/1808527149 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f56b4003a00 0x7f56cc1efaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:30.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.407+0000 7f56cbfff700 1 -- 172.21.15.16:0/1808527149 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f56cc144be0 msgr2=0x7f56cc0b5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:30.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.407+0000 7f56cbfff700 1 --2- 172.21.15.16:0/1808527149 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f56cc144be0 0x7f56cc0b5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:30.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.407+0000 7f56cbfff700 1 -- 172.21.15.16:0/1808527149 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56cc0cbbf0 con 0x7f56cc0b61b0 2023-05-20T01:17:30.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.407+0000 7f56caffd700 1 --2- 172.21.15.16:0/1808527149 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f56cc144be0 0x7f56cc0b5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:17:30.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.408+0000 7f56cbfff700 1 --2- 172.21.15.16:0/1808527149 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56cc0b61b0 0x7f56cc0c6bf0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f56cc0ceba0 tx=0x7f56c4001a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:30.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.408+0000 7f56b3fff700 1 -- 172.21.15.16:0/1808527149 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56c400bb70 con 0x7f56cc0b61b0 2023-05-20T01:17:30.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.408+0000 7f56b3fff700 1 -- 172.21.15.16:0/1808527149 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56c400bcd0 con 0x7f56cc0b61b0 2023-05-20T01:17:30.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.408+0000 7f56d1f66700 1 -- 172.21.15.16:0/1808527149 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56cc0cb910 con 0x7f56cc0b61b0 2023-05-20T01:17:30.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.408+0000 7f56b3fff700 1 -- 172.21.15.16:0/1808527149 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56c40070d0 con 0x7f56cc0b61b0 2023-05-20T01:17:30.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.408+0000 7f56d1f66700 1 -- 172.21.15.16:0/1808527149 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56cc0d0e30 con 0x7f56cc0b61b0 2023-05-20T01:17:30.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.409+0000 7f56d1f66700 1 -- 172.21.15.16:0/1808527149 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56cc0747b0 con 0x7f56cc0b61b0 2023-05-20T01:17:30.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.409+0000 7f56b3fff700 1 -- 172.21.15.16:0/1808527149 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f56c4007230 con 0x7f56cc0b61b0 2023-05-20T01:17:30.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.410+0000 7f56b3fff700 1 --2- 172.21.15.16:0/1808527149 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f56b806ace0 0x7f56b806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:30.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.410+0000 7f56b3fff700 1 -- 172.21.15.16:0/1808527149 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f56c4089880 con 0x7f56cc0b61b0 2023-05-20T01:17:30.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.410+0000 7f56cb7fe700 1 --2- 172.21.15.16:0/1808527149 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f56b806ace0 0x7f56b806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:30.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.410+0000 7f56cb7fe700 1 --2- 172.21.15.16:0/1808527149 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f56b806ace0 0x7f56b806d190 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f56bc000c00 tx=0x7f56bc020040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:30.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.414+0000 7f56b3fff700 1 -- 172.21.15.16:0/1808527149 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f56c4054690 con 0x7f56cc0b61b0 2023-05-20T01:17:30.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.614+0000 7f56d1f66700 1 -- 172.21.15.16:0/1808527149 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f56cc1effe0 con 0x7f56cc0b61b0 2023-05-20T01:17:30.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.614+0000 7f56b3fff700 1 -- 172.21.15.16:0/1808527149 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f56c4057cb0 con 0x7f56cc0b61b0 2023-05-20T01:17:30.618 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-05-20T01:17:30.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.617+0000 7f56d1f66700 1 -- 172.21.15.16:0/1808527149 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f56b806ace0 msgr2=0x7f56b806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:30.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.617+0000 7f56d1f66700 1 --2- 172.21.15.16:0/1808527149 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f56b806ace0 0x7f56b806d190 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f56bc000c00 tx=0x7f56bc020040 comp rx=0 tx=0).stop 2023-05-20T01:17:30.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.617+0000 7f56d1f66700 1 -- 172.21.15.16:0/1808527149 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56cc0b61b0 msgr2=0x7f56cc0c6bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:30.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.617+0000 7f56d1f66700 1 --2- 172.21.15.16:0/1808527149 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56cc0b61b0 0x7f56cc0c6bf0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f56cc0ceba0 tx=0x7f56c4001a90 comp rx=0 tx=0).stop 2023-05-20T01:17:30.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.617+0000 7f56d1f66700 1 -- 172.21.15.16:0/1808527149 shutdown_connections 2023-05-20T01:17:30.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.617+0000 7f56d1f66700 1 --2- 172.21.15.16:0/1808527149 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56cc0b61b0 0x7f56cc0c6bf0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:30.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.617+0000 7f56d1f66700 1 --2- 172.21.15.16:0/1808527149 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f56b806ace0 0x7f56b806d190 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:30.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.617+0000 7f56d1f66700 1 --2- 172.21.15.16:0/1808527149 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f56b4003a00 0x7f56cc1efaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:30.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.617+0000 7f56d1f66700 1 --2- 172.21.15.16:0/1808527149 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f56cc144be0 0x7f56cc0b5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:30.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.617+0000 7f56d1f66700 1 -- 172.21.15.16:0/1808527149 >> 172.21.15.16:0/1808527149 conn(0x7f56cc06ae30 msgr2=0x7f56cc06f4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:30.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.617+0000 7f56d1f66700 1 -- 172.21.15.16:0/1808527149 shutdown_connections 2023-05-20T01:17:30.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.617+0000 7f56d1f66700 1 -- 172.21.15.16:0/1808527149 wait complete. 2023-05-20T01:17:30.632 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-05-20T01:17:30.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.756+0000 7f154780c700 1 Processor -- start 2023-05-20T01:17:30.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.756+0000 7f154780c700 1 -- start start 2023-05-20T01:17:30.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.757+0000 7f154780c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f154006ddf0 0x7f154006e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:30.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.757+0000 7f154780c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f15401613e0 0x7f154015cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:30.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.757+0000 7f154780c700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f154006e700 con 0x7f154015d480 2023-05-20T01:17:30.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.757+0000 7f154780c700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f154015dea0 con 0x7f15401613e0 2023-05-20T01:17:30.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.757+0000 7f154780c700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1540160900 con 0x7f154006ddf0 2023-05-20T01:17:30.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.757+0000 7f1544da7700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f15401613e0 0x7f154015cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:30.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.757+0000 7f1545da9700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f154015d480 0x7f154006cbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:42984/0 (socket says 172.21.15.16:42984) 2023-05-20T01:17:30.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.757+0000 7f1545da9700 1 -- 172.21.15.16:0/1436820315 learned_addr learned my addr 172.21.15.16:0/1436820315 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:17:30.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.758+0000 7f15455a8700 1 --2- 172.21.15.16:0/1436820315 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f154006ddf0 0x7f154006e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:30.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.758+0000 7f1544da7700 1 -- 172.21.15.16:0/1436820315 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f154006ddf0 msgr2=0x7f154006e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:30.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.758+0000 7f1544da7700 1 --2- 172.21.15.16:0/1436820315 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f154006ddf0 0x7f154006e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:30.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.758+0000 7f1544da7700 1 -- 172.21.15.16:0/1436820315 >> v1:172.21.15.16:6789/0 conn(0x7f154015d480 legacy=0x7f154006cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:30.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.758+0000 7f1533fff700 1 -- 172.21.15.16:0/1436820315 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3506143853 0 0) 0x7f154006e700 con 0x7f154015d480 2023-05-20T01:17:30.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.758+0000 7f1544da7700 1 -- 172.21.15.16:0/1436820315 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1540160be0 con 0x7f15401613e0 2023-05-20T01:17:30.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.758+0000 7f1544da7700 1 --2- 172.21.15.16:0/1436820315 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15401613e0 0x7f154015cf40 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f153c009840 tx=0x7f153c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e05554e88338d4a server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:30.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.758+0000 7f1533fff700 1 -- 172.21.15.16:0/1436820315 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f153c011830 con 0x7f15401613e0 2023-05-20T01:17:30.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.758+0000 7f1533fff700 1 -- 172.21.15.16:0/1436820315 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f153c011990 con 0x7f15401613e0 2023-05-20T01:17:30.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.758+0000 7f1533fff700 1 -- 172.21.15.16:0/1436820315 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f153c011d20 con 0x7f15401613e0 2023-05-20T01:17:30.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.759+0000 7f154780c700 1 -- 172.21.15.16:0/1436820315 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15401613e0 msgr2=0x7f154015cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:30.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.759+0000 7f154780c700 1 --2- 172.21.15.16:0/1436820315 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15401613e0 0x7f154015cf40 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f153c009840 tx=0x7f153c009b50 comp rx=0 tx=0).stop 2023-05-20T01:17:30.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.759+0000 7f154780c700 1 -- 172.21.15.16:0/1436820315 shutdown_connections 2023-05-20T01:17:30.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.759+0000 7f154780c700 1 --2- 172.21.15.16:0/1436820315 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f154006ddf0 0x7f154006e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:30.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.759+0000 7f154780c700 1 --2- 172.21.15.16:0/1436820315 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15401613e0 0x7f154015cf40 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:30.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.759+0000 7f154780c700 1 -- 172.21.15.16:0/1436820315 >> 172.21.15.16:0/1436820315 conn(0x7f15400cd760 msgr2=0x7f15400d0050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:30.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.759+0000 7f154780c700 1 -- 172.21.15.16:0/1436820315 shutdown_connections 2023-05-20T01:17:30.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.759+0000 7f154780c700 1 -- 172.21.15.16:0/1436820315 wait complete. 2023-05-20T01:17:30.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.760+0000 7f154780c700 1 Processor -- start 2023-05-20T01:17:30.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.760+0000 7f154780c700 1 -- start start 2023-05-20T01:17:30.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.761+0000 7f154780c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f154006ddf0 0x7f15400b1b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:30.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.761+0000 7f154780c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f154015d480 0x7f15401576f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:30.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.761+0000 7f154780c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f15401613e0 0x7f154015af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:30.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.761+0000 7f154780c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15401f2a90 con 0x7f154015d480 2023-05-20T01:17:30.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.761+0000 7f154780c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1540157c30 con 0x7f15401613e0 2023-05-20T01:17:30.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.761+0000 7f154780c700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1540158660 con 0x7f154006ddf0 2023-05-20T01:17:30.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.762+0000 7f1544da7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f154015d480 0x7f15401576f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:30.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.762+0000 7f1544da7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f154015d480 0x7f15401576f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59692/0 (socket says 172.21.15.16:59692) 2023-05-20T01:17:30.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.762+0000 7f1544da7700 1 -- 172.21.15.16:0/3833533557 learned_addr learned my addr 172.21.15.16:0/3833533557 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:30.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.762+0000 7f15455a8700 1 --2- 172.21.15.16:0/3833533557 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f154006ddf0 0x7f15400b1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:30.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.762+0000 7f1545da9700 1 --2- 172.21.15.16:0/3833533557 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f15401613e0 0x7f154015af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:30.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.762+0000 7f1544da7700 1 -- 172.21.15.16:0/3833533557 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f154006ddf0 msgr2=0x7f15400b1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:30.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.762+0000 7f1544da7700 1 --2- 172.21.15.16:0/3833533557 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f154006ddf0 0x7f15400b1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:30.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.762+0000 7f1544da7700 1 -- 172.21.15.16:0/3833533557 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f15401613e0 msgr2=0x7f154015af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:30.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.762+0000 7f1544da7700 1 --2- 172.21.15.16:0/3833533557 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f15401613e0 0x7f154015af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:30.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.762+0000 7f1544da7700 1 -- 172.21.15.16:0/3833533557 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f153c0094a0 con 0x7f154015d480 2023-05-20T01:17:30.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.762+0000 7f15455a8700 1 --2- 172.21.15.16:0/3833533557 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f154006ddf0 0x7f15400b1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:17:30.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.762+0000 7f1544da7700 1 --2- 172.21.15.16:0/3833533557 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f154015d480 0x7f15401576f0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f153c014040 tx=0x7f153c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:30.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.762+0000 7f1531ffb700 1 -- 172.21.15.16:0/3833533557 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f153c0038a0 con 0x7f154015d480 2023-05-20T01:17:30.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.762+0000 7f1531ffb700 1 -- 172.21.15.16:0/3833533557 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f153c01e070 con 0x7f154015d480 2023-05-20T01:17:30.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.762+0000 7f154780c700 1 -- 172.21.15.16:0/3833533557 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f154015d8a0 con 0x7f154015d480 2023-05-20T01:17:30.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.762+0000 7f1531ffb700 1 -- 172.21.15.16:0/3833533557 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f153c003c20 con 0x7f154015d480 2023-05-20T01:17:30.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.762+0000 7f154780c700 1 -- 172.21.15.16:0/3833533557 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1540160ec0 con 0x7f154015d480 2023-05-20T01:17:30.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.763+0000 7f154780c700 1 -- 172.21.15.16:0/3833533557 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f154015da30 con 0x7f154015d480 2023-05-20T01:17:30.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.764+0000 7f1531ffb700 1 -- 172.21.15.16:0/3833533557 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f153c02d450 con 0x7f154015d480 2023-05-20T01:17:30.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.764+0000 7f1531ffb700 1 --2- 172.21.15.16:0/3833533557 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f152406ace0 0x7f152406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:30.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.764+0000 7f1531ffb700 1 -- 172.21.15.16:0/3833533557 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f153c08a870 con 0x7f154015d480 2023-05-20T01:17:30.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.764+0000 7f15455a8700 1 --2- 172.21.15.16:0/3833533557 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f152406ace0 0x7f152406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:30.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.765+0000 7f15455a8700 1 --2- 172.21.15.16:0/3833533557 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f152406ace0 0x7f152406d190 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f15340099f0 tx=0x7f1534008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:30.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.769+0000 7f1531ffb700 1 -- 172.21.15.16:0/3833533557 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f153c0555e0 con 0x7f154015d480 2023-05-20T01:17:30.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.967+0000 7f154780c700 1 -- 172.21.15.16:0/3833533557 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f15400747b0 con 0x7f154015d480 2023-05-20T01:17:30.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.967+0000 7f1531ffb700 1 -- 172.21.15.16:0/3833533557 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f153c02c3b0 con 0x7f154015d480 2023-05-20T01:17:30.971 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2023-05-20T01:17:30.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.969+0000 7f154780c700 1 -- 172.21.15.16:0/3833533557 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f152406ace0 msgr2=0x7f152406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:30.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.969+0000 7f154780c700 1 --2- 172.21.15.16:0/3833533557 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f152406ace0 0x7f152406d190 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f15340099f0 tx=0x7f1534008040 comp rx=0 tx=0).stop 2023-05-20T01:17:30.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.969+0000 7f154780c700 1 -- 172.21.15.16:0/3833533557 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f154015d480 msgr2=0x7f15401576f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:30.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.969+0000 7f154780c700 1 --2- 172.21.15.16:0/3833533557 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f154015d480 0x7f15401576f0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f153c014040 tx=0x7f153c004750 comp rx=0 tx=0).stop 2023-05-20T01:17:30.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.970+0000 7f154780c700 1 -- 172.21.15.16:0/3833533557 shutdown_connections 2023-05-20T01:17:30.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.970+0000 7f154780c700 1 --2- 172.21.15.16:0/3833533557 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f15401613e0 0x7f154015af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:30.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.970+0000 7f154780c700 1 --2- 172.21.15.16:0/3833533557 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f152406ace0 0x7f152406d190 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:30.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.970+0000 7f154780c700 1 --2- 172.21.15.16:0/3833533557 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f154006ddf0 0x7f15400b1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:30.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.970+0000 7f154780c700 1 --2- 172.21.15.16:0/3833533557 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f154015d480 0x7f15401576f0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:30.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.970+0000 7f154780c700 1 -- 172.21.15.16:0/3833533557 >> 172.21.15.16:0/3833533557 conn(0x7f15400cd760 msgr2=0x7f15400acd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:30.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.970+0000 7f154780c700 1 -- 172.21.15.16:0/3833533557 shutdown_connections 2023-05-20T01:17:30.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:30.970+0000 7f154780c700 1 -- 172.21.15.16:0/3833533557 wait complete. 2023-05-20T01:17:30.986 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-05-20T01:17:31.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.110+0000 7f1595902700 1 Processor -- start 2023-05-20T01:17:31.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.111+0000 7f1595902700 1 -- start start 2023-05-20T01:17:31.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.111+0000 7f1595902700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f15900b67a0 0x7f1590167730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:31.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.111+0000 7f1595902700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15900d1820 con 0x7f15900cd890 2023-05-20T01:17:31.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.111+0000 7f1595902700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15900d2c60 con 0x7f159015ca90 2023-05-20T01:17:31.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.111+0000 7f1595902700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15900d0c50 con 0x7f15900b67a0 2023-05-20T01:17:31.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.111+0000 7f158e7fc700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f15900cd890 0x7f15900b6090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:42994/0 (socket says 172.21.15.16:42994) 2023-05-20T01:17:31.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.111+0000 7f158e7fc700 1 -- 172.21.15.16:0/3858562909 learned_addr learned my addr 172.21.15.16:0/3858562909 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:17:31.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.112+0000 7f158f7fe700 1 --2- 172.21.15.16:0/3858562909 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f15900b67a0 0x7f1590167730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:31.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.112+0000 7f158dffb700 1 -- 172.21.15.16:0/3858562909 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1475959541 0 0) 0x7f15900d1820 con 0x7f15900cd890 2023-05-20T01:17:31.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.112+0000 7f158dffb700 1 -- 172.21.15.16:0/3858562909 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f15780035b0 con 0x7f15900cd890 2023-05-20T01:17:31.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.112+0000 7f158dffb700 1 -- 172.21.15.16:0/3858562909 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3239703858 0 0) 0x7f15780035b0 con 0x7f15900cd890 2023-05-20T01:17:31.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.112+0000 7f158dffb700 1 -- 172.21.15.16:0/3858562909 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f15900b67a0 msgr2=0x7f1590167730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:31.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.112+0000 7f158dffb700 1 --2- 172.21.15.16:0/3858562909 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f15900b67a0 0x7f1590167730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:31.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.112+0000 7f158dffb700 1 -- 172.21.15.16:0/3858562909 >> v1:172.21.15.132:6789/0 conn(0x7f159015ca90 legacy=0x7f15900d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:31.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.112+0000 7f158dffb700 1 -- 172.21.15.16:0/3858562909 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15900cdf00 con 0x7f15900cd890 2023-05-20T01:17:31.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.112+0000 7f158dffb700 1 -- 172.21.15.16:0/3858562909 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7f1580001670 con 0x7f15900cd890 2023-05-20T01:17:31.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.112+0000 7f158dffb700 1 -- 172.21.15.16:0/3858562909 >> v1:172.21.15.16:6789/0 conn(0x7f15900cd890 legacy=0x7f15900b6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:31.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.112+0000 7f158dffb700 1 --2- 172.21.15.16:0/3858562909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1578003a30 0x7f1578003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:31.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.112+0000 7f158dffb700 1 --2- 172.21.15.16:0/3858562909 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f15900b67a0 0x7f1578004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:31.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.112+0000 7f158dffb700 1 --2- 172.21.15.16:0/3858562909 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1578004d90 0x7f1578005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:31.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.112+0000 7f158dffb700 1 -- 172.21.15.16:0/3858562909 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15900d0c50 con 0x7f1578003a30 2023-05-20T01:17:31.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.112+0000 7f158dffb700 1 -- 172.21.15.16:0/3858562909 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f157800ba10 con 0x7f1578004d90 2023-05-20T01:17:31.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.112+0000 7f158dffb700 1 -- 172.21.15.16:0/3858562909 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f157800bba0 con 0x7f15900b67a0 2023-05-20T01:17:31.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.112+0000 7f158effd700 1 --2- 172.21.15.16:0/3858562909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1578003a30 0x7f1578003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:31.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.112+0000 7f158e7fc700 1 --2- 172.21.15.16:0/3858562909 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f15900b67a0 0x7f1578004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:31.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.112+0000 7f158f7fe700 1 --2- 172.21.15.16:0/3858562909 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1578004d90 0x7f1578005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:31.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.112+0000 7f158effd700 1 -- 172.21.15.16:0/3858562909 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f15900b67a0 msgr2=0x7f1578004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:31.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.112+0000 7f158effd700 1 --2- 172.21.15.16:0/3858562909 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f15900b67a0 0x7f1578004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:31.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.112+0000 7f158effd700 1 -- 172.21.15.16:0/3858562909 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1578004d90 msgr2=0x7f1578005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:31.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.112+0000 7f158effd700 1 --2- 172.21.15.16:0/3858562909 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1578004d90 0x7f1578005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:31.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.113+0000 7f158effd700 1 -- 172.21.15.16:0/3858562909 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15780035b0 con 0x7f1578003a30 2023-05-20T01:17:31.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.113+0000 7f158effd700 1 --2- 172.21.15.16:0/3858562909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1578003a30 0x7f1578003e00 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f15840018d0 tx=0x7f1578005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4c85831670deb80 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:31.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.113+0000 7f158dffb700 1 -- 172.21.15.16:0/3858562909 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f158400c690 con 0x7f1578003a30 2023-05-20T01:17:31.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.113+0000 7f158dffb700 1 -- 172.21.15.16:0/3858562909 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f158400c810 con 0x7f1578003a30 2023-05-20T01:17:31.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.113+0000 7f158dffb700 1 -- 172.21.15.16:0/3858562909 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f158400cbc0 con 0x7f1578003a30 2023-05-20T01:17:31.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.113+0000 7f1595902700 1 -- 172.21.15.16:0/3858562909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1578003a30 msgr2=0x7f1578003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:31.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.113+0000 7f1595902700 1 --2- 172.21.15.16:0/3858562909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1578003a30 0x7f1578003e00 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f15840018d0 tx=0x7f1578005880 comp rx=0 tx=0).stop 2023-05-20T01:17:31.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.113+0000 7f158f7fe700 1 -- 172.21.15.16:0/3858562909 reap_dead start 2023-05-20T01:17:31.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.113+0000 7f1595902700 1 -- 172.21.15.16:0/3858562909 shutdown_connections 2023-05-20T01:17:31.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.113+0000 7f1595902700 1 -- 172.21.15.16:0/3858562909 >> 172.21.15.16:0/3858562909 conn(0x7f159006b000 msgr2=0x7f159006b400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:31.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.113+0000 7f1595902700 1 -- 172.21.15.16:0/3858562909 shutdown_connections 2023-05-20T01:17:31.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.113+0000 7f1595902700 1 -- 172.21.15.16:0/3858562909 wait complete. 2023-05-20T01:17:31.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.115+0000 7f1595902700 1 Processor -- start 2023-05-20T01:17:31.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.115+0000 7f1595902700 1 -- start start 2023-05-20T01:17:31.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.116+0000 7f1595902700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1578003a30 0x7f15901579d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:31.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.116+0000 7f1595902700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f15900b3da0 0x7f15900b4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:31.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.116+0000 7f1595902700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15900cc750 0x7f15900ccbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:31.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.116+0000 7f1595902700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15900cd110 con 0x7f15900cc750 2023-05-20T01:17:31.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.116+0000 7f1595902700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15900b2d40 con 0x7f15900b3da0 2023-05-20T01:17:31.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.116+0000 7f1595902700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15900b2fe0 con 0x7f1578003a30 2023-05-20T01:17:31.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.116+0000 7f158f7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15900cc750 0x7f15900ccbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:31.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.116+0000 7f158f7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15900cc750 0x7f15900ccbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59714/0 (socket says 172.21.15.16:59714) 2023-05-20T01:17:31.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.116+0000 7f158f7fe700 1 -- 172.21.15.16:0/1713549604 learned_addr learned my addr 172.21.15.16:0/1713549604 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:31.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.116+0000 7f158e7fc700 1 --2- 172.21.15.16:0/1713549604 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f15900b3da0 0x7f15900b4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:31.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.116+0000 7f158effd700 1 --2- 172.21.15.16:0/1713549604 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1578003a30 0x7f15901579d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:31.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.116+0000 7f158f7fe700 1 -- 172.21.15.16:0/1713549604 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1578003a30 msgr2=0x7f15901579d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:31.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.116+0000 7f158f7fe700 1 --2- 172.21.15.16:0/1713549604 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1578003a30 0x7f15901579d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:31.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.116+0000 7f158f7fe700 1 -- 172.21.15.16:0/1713549604 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f15900b3da0 msgr2=0x7f15900b4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:31.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.116+0000 7f158f7fe700 1 --2- 172.21.15.16:0/1713549604 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f15900b3da0 0x7f15900b4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:31.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.116+0000 7f158f7fe700 1 -- 172.21.15.16:0/1713549604 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15900d1d50 con 0x7f15900cc750 2023-05-20T01:17:31.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.116+0000 7f158f7fe700 1 --2- 172.21.15.16:0/1713549604 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15900cc750 0x7f15900ccbd0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f1590160650 tx=0x7f1588011f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:31.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.116+0000 7f1577fff700 1 -- 172.21.15.16:0/1713549604 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f158800e040 con 0x7f15900cc750 2023-05-20T01:17:31.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.116+0000 7f1577fff700 1 -- 172.21.15.16:0/1713549604 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f158800f040 con 0x7f15900cc750 2023-05-20T01:17:31.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.116+0000 7f1577fff700 1 -- 172.21.15.16:0/1713549604 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f158800fdb0 con 0x7f15900cc750 2023-05-20T01:17:31.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.116+0000 7f1595902700 1 -- 172.21.15.16:0/1713549604 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15900d1a70 con 0x7f15900cc750 2023-05-20T01:17:31.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.116+0000 7f1595902700 1 -- 172.21.15.16:0/1713549604 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15900d2030 con 0x7f15900cc750 2023-05-20T01:17:31.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.117+0000 7f1595902700 1 -- 172.21.15.16:0/1713549604 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15900ceed0 con 0x7f15900cc750 2023-05-20T01:17:31.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.118+0000 7f1577fff700 1 -- 172.21.15.16:0/1713549604 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1588015430 con 0x7f15900cc750 2023-05-20T01:17:31.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.118+0000 7f1577fff700 1 --2- 172.21.15.16:0/1713549604 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f157c06ace0 0x7f157c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:31.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.118+0000 7f1577fff700 1 -- 172.21.15.16:0/1713549604 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f1588089fb0 con 0x7f15900cc750 2023-05-20T01:17:31.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.118+0000 7f158effd700 1 --2- 172.21.15.16:0/1713549604 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f157c06ace0 0x7f157c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:31.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.119+0000 7f158effd700 1 --2- 172.21.15.16:0/1713549604 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f157c06ace0 0x7f157c06d190 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1584014040 tx=0x7f1584005040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:31.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.123+0000 7f1577fff700 1 -- 172.21.15.16:0/1713549604 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f1588054dc0 con 0x7f15900cc750 2023-05-20T01:17:31.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.318+0000 7f1595902700 1 -- 172.21.15.16:0/1713549604 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f159006c310 con 0x7f15900cc750 2023-05-20T01:17:31.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.319+0000 7f1577fff700 1 -- 172.21.15.16:0/1713549604 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f15880583e0 con 0x7f15900cc750 2023-05-20T01:17:31.323 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-05-20T01:17:31.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.320+0000 7f1595902700 1 -- 172.21.15.16:0/1713549604 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f157c06ace0 msgr2=0x7f157c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:31.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.321+0000 7f1595902700 1 --2- 172.21.15.16:0/1713549604 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f157c06ace0 0x7f157c06d190 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1584014040 tx=0x7f1584005040 comp rx=0 tx=0).stop 2023-05-20T01:17:31.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.321+0000 7f1595902700 1 -- 172.21.15.16:0/1713549604 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15900cc750 msgr2=0x7f15900ccbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:31.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.321+0000 7f1595902700 1 --2- 172.21.15.16:0/1713549604 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15900cc750 0x7f15900ccbd0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f1590160650 tx=0x7f1588011f80 comp rx=0 tx=0).stop 2023-05-20T01:17:31.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.321+0000 7f1595902700 1 -- 172.21.15.16:0/1713549604 shutdown_connections 2023-05-20T01:17:31.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.321+0000 7f1595902700 1 --2- 172.21.15.16:0/1713549604 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15900cc750 0x7f15900ccbd0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:31.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.321+0000 7f1595902700 1 --2- 172.21.15.16:0/1713549604 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f157c06ace0 0x7f157c06d190 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:31.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.321+0000 7f1595902700 1 --2- 172.21.15.16:0/1713549604 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1578003a30 0x7f15901579d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:31.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.321+0000 7f1595902700 1 --2- 172.21.15.16:0/1713549604 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f15900b3da0 0x7f15900b4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:31.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.321+0000 7f1595902700 1 -- 172.21.15.16:0/1713549604 >> 172.21.15.16:0/1713549604 conn(0x7f159006b000 msgr2=0x7f1590159700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:31.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.321+0000 7f1595902700 1 -- 172.21.15.16:0/1713549604 shutdown_connections 2023-05-20T01:17:31.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.321+0000 7f1595902700 1 -- 172.21.15.16:0/1713549604 wait complete. 2023-05-20T01:17:31.337 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-20T01:17:31.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.459+0000 7fcb0849a700 1 Processor -- start 2023-05-20T01:17:31.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.459+0000 7fcb0849a700 1 -- start start 2023-05-20T01:17:31.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.459+0000 7fcb0849a700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fcb00157190 0x7fcb000cc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:31.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.460+0000 7fcb0849a700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcb00071ad0 0x7fcb000ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:31.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.460+0000 7fcb0849a700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb000cb590 con 0x7fcb000cccf0 2023-05-20T01:17:31.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.460+0000 7fcb0849a700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb00071f50 con 0x7fcb00157190 2023-05-20T01:17:31.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.460+0000 7fcb0849a700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb000ca9a0 con 0x7fcb00071ad0 2023-05-20T01:17:31.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.460+0000 7fcb06236700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fcb00157190 0x7fcb000cc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:31.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.460+0000 7fcb05a35700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7fcb000cccf0 0x7fcb000713c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.16:52730/0 (socket says 172.21.15.16:52730) 2023-05-20T01:17:31.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.460+0000 7fcb06236700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fcb00157190 0x7fcb000cc7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.16:56740/0 (socket says 172.21.15.16:56740) 2023-05-20T01:17:31.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.460+0000 7fcb05a35700 1 -- 172.21.15.16:0/1054393663 learned_addr learned my addr 172.21.15.16:0/1054393663 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:17:31.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.460+0000 7fcb06a37700 1 --2- 172.21.15.16:0/1054393663 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcb00071ad0 0x7fcb000ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:31.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.460+0000 7fcb06236700 1 -- 172.21.15.16:0/1054393663 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcb00071ad0 msgr2=0x7fcb000ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:31.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.460+0000 7fcb06236700 1 --2- 172.21.15.16:0/1054393663 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcb00071ad0 0x7fcb000ce740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:31.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.460+0000 7fcb06236700 1 -- 172.21.15.16:0/1054393663 >> v1:172.21.15.132:6789/0 conn(0x7fcb000cccf0 legacy=0x7fcb000713c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:31.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.460+0000 7fcb06236700 1 -- 172.21.15.16:0/1054393663 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb000cac60 con 0x7fcb00157190 2023-05-20T01:17:31.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.460+0000 7fcb06236700 1 --2- 172.21.15.16:0/1054393663 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcb00157190 0x7fcb000cc7b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fcafc009840 tx=0x7fcafc009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f377f9a6c59e46 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:31.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.461+0000 7fcb05234700 1 -- 172.21.15.16:0/1054393663 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcafc011830 con 0x7fcb00157190 2023-05-20T01:17:31.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.461+0000 7fcb05234700 1 -- 172.21.15.16:0/1054393663 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcafc011990 con 0x7fcb00157190 2023-05-20T01:17:31.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.461+0000 7fcb05234700 1 -- 172.21.15.16:0/1054393663 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcafc011d20 con 0x7fcb00157190 2023-05-20T01:17:31.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.461+0000 7fcb0849a700 1 -- 172.21.15.16:0/1054393663 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcb00157190 msgr2=0x7fcb000cc7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:31.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.461+0000 7fcb0849a700 1 --2- 172.21.15.16:0/1054393663 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcb00157190 0x7fcb000cc7b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fcafc009840 tx=0x7fcafc009b50 comp rx=0 tx=0).stop 2023-05-20T01:17:31.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.461+0000 7fcb0849a700 1 -- 172.21.15.16:0/1054393663 shutdown_connections 2023-05-20T01:17:31.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.461+0000 7fcb0849a700 1 --2- 172.21.15.16:0/1054393663 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcb00071ad0 0x7fcb000ce740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:31.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.461+0000 7fcb0849a700 1 --2- 172.21.15.16:0/1054393663 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcb00157190 0x7fcb000cc7b0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:31.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.461+0000 7fcb0849a700 1 -- 172.21.15.16:0/1054393663 >> 172.21.15.16:0/1054393663 conn(0x7fcb000cffb0 msgr2=0x7fcb000d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:31.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.461+0000 7fcb0849a700 1 -- 172.21.15.16:0/1054393663 shutdown_connections 2023-05-20T01:17:31.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.461+0000 7fcb0849a700 1 -- 172.21.15.16:0/1054393663 wait complete. 2023-05-20T01:17:31.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.462+0000 7fcb0849a700 1 Processor -- start 2023-05-20T01:17:31.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.462+0000 7fcb0849a700 1 -- start start 2023-05-20T01:17:31.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.463+0000 7fcb0849a700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcb00071ad0 0x7fcb000afab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:31.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.463+0000 7fcb0849a700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcb000cccf0 0x7fcb0006a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:31.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.463+0000 7fcb0849a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb00157190 0x7fcb000b40a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:31.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.463+0000 7fcb0849a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb000c5020 con 0x7fcb00157190 2023-05-20T01:17:31.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.463+0000 7fcb0849a700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb000b50e0 con 0x7fcb000cccf0 2023-05-20T01:17:31.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.463+0000 7fcb0849a700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb000b4b20 con 0x7fcb00071ad0 2023-05-20T01:17:31.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.463+0000 7fcb06a37700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb00157190 0x7fcb000b40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:31.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.463+0000 7fcb06a37700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb00157190 0x7fcb000b40a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59728/0 (socket says 172.21.15.16:59728) 2023-05-20T01:17:31.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.463+0000 7fcb06a37700 1 -- 172.21.15.16:0/396640919 learned_addr learned my addr 172.21.15.16:0/396640919 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:31.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.463+0000 7fcb06236700 1 --2- 172.21.15.16:0/396640919 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcb00071ad0 0x7fcb000afab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:31.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.463+0000 7fcb05a35700 1 --2- 172.21.15.16:0/396640919 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcb000cccf0 0x7fcb0006a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:31.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.464+0000 7fcb06a37700 1 -- 172.21.15.16:0/396640919 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcb00071ad0 msgr2=0x7fcb000afab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:31.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.464+0000 7fcb06a37700 1 --2- 172.21.15.16:0/396640919 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcb00071ad0 0x7fcb000afab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:31.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.464+0000 7fcb06a37700 1 -- 172.21.15.16:0/396640919 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcb000cccf0 msgr2=0x7fcb0006a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:31.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.464+0000 7fcb06a37700 1 --2- 172.21.15.16:0/396640919 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcb000cccf0 0x7fcb0006a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:31.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.464+0000 7fcb06a37700 1 -- 172.21.15.16:0/396640919 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcafc0094a0 con 0x7fcb00157190 2023-05-20T01:17:31.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.464+0000 7fcb06a37700 1 --2- 172.21.15.16:0/396640919 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb00157190 0x7fcb000b40a0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fcaf800efc0 tx=0x7fcaf800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:31.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.464+0000 7fcaf6ffd700 1 -- 172.21.15.16:0/396640919 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf8009ea0 con 0x7fcb00157190 2023-05-20T01:17:31.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.464+0000 7fcaf6ffd700 1 -- 172.21.15.16:0/396640919 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaf800f040 con 0x7fcb00157190 2023-05-20T01:17:31.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.464+0000 7fcaf6ffd700 1 -- 172.21.15.16:0/396640919 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf800ced0 con 0x7fcb00157190 2023-05-20T01:17:31.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.464+0000 7fcb0849a700 1 -- 172.21.15.16:0/396640919 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb000cb220 con 0x7fcb00157190 2023-05-20T01:17:31.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.464+0000 7fcb0849a700 1 -- 172.21.15.16:0/396640919 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb000caf40 con 0x7fcb00157190 2023-05-20T01:17:31.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.465+0000 7fcb0849a700 1 -- 172.21.15.16:0/396640919 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb00157560 con 0x7fcb00157190 2023-05-20T01:17:31.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.465+0000 7fcaf6ffd700 1 -- 172.21.15.16:0/396640919 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fcaf8014480 con 0x7fcb00157190 2023-05-20T01:17:31.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.466+0000 7fcaf6ffd700 1 --2- 172.21.15.16:0/396640919 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fcae406ace0 0x7fcae406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:31.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.466+0000 7fcaf6ffd700 1 -- 172.21.15.16:0/396640919 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fcaf808a1d0 con 0x7fcb00157190 2023-05-20T01:17:31.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.466+0000 7fcb06236700 1 --2- 172.21.15.16:0/396640919 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fcae406ace0 0x7fcae406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:31.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.467+0000 7fcb06236700 1 --2- 172.21.15.16:0/396640919 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fcae406ace0 0x7fcae406d190 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fcafc000c00 tx=0x7fcafc01c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:31.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.471+0000 7fcaf6ffd700 1 -- 172.21.15.16:0/396640919 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fcaf8054fe0 con 0x7fcb00157190 2023-05-20T01:17:31.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.713+0000 7fcb0849a700 1 -- 172.21.15.16:0/396640919 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcb000747b0 con 0x7fcb00157190 2023-05-20T01:17:31.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.713+0000 7fcaf6ffd700 1 -- 172.21.15.16:0/396640919 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4673 (secure 0 0 0) 0x7fcaf8058600 con 0x7fcb00157190 2023-05-20T01:17:31.717 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:17:31.717 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4273,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/1444124146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1444124146},{"type":"v1","addr":"172.21.15.132:6834","nonce":1444124146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4293,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4092119206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4092119206},{"type":"v1","addr":"172.21.15.16:6835","nonce":4092119206}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2162680372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":2162680372},{"type":"v1","addr":"172.21.15.132:6835","nonce":2162680372}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-20T01:16:28.316109+0000","modified":"2023-05-20T01:16:45.362572+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4299},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4299":{"gid":4299,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6837/2330551626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2330551626},{"type":"v1","addr":"172.21.15.16:6837","nonce":2330551626}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-05-20T01:17:31.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.715+0000 7fcb0849a700 1 -- 172.21.15.16:0/396640919 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fcae406ace0 msgr2=0x7fcae406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:31.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.715+0000 7fcb0849a700 1 --2- 172.21.15.16:0/396640919 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fcae406ace0 0x7fcae406d190 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fcafc000c00 tx=0x7fcafc01c040 comp rx=0 tx=0).stop 2023-05-20T01:17:31.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.716+0000 7fcb0849a700 1 -- 172.21.15.16:0/396640919 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb00157190 msgr2=0x7fcb000b40a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:31.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.716+0000 7fcb0849a700 1 --2- 172.21.15.16:0/396640919 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb00157190 0x7fcb000b40a0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fcaf800efc0 tx=0x7fcaf800c3a0 comp rx=0 tx=0).stop 2023-05-20T01:17:31.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.716+0000 7fcb0849a700 1 -- 172.21.15.16:0/396640919 shutdown_connections 2023-05-20T01:17:31.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.716+0000 7fcb0849a700 1 --2- 172.21.15.16:0/396640919 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb00157190 0x7fcb000b40a0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:31.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.716+0000 7fcb0849a700 1 --2- 172.21.15.16:0/396640919 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fcae406ace0 0x7fcae406d190 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:31.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.716+0000 7fcb0849a700 1 --2- 172.21.15.16:0/396640919 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcb00071ad0 0x7fcb000afab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:31.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.716+0000 7fcb0849a700 1 --2- 172.21.15.16:0/396640919 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcb000cccf0 0x7fcb0006a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:31.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.716+0000 7fcb0849a700 1 -- 172.21.15.16:0/396640919 >> 172.21.15.16:0/396640919 conn(0x7fcb000cffb0 msgr2=0x7fcb00153e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:31.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.716+0000 7fcb0849a700 1 -- 172.21.15.16:0/396640919 shutdown_connections 2023-05-20T01:17:31.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.716+0000 7fcb0849a700 1 -- 172.21.15.16:0/396640919 wait complete. 2023-05-20T01:17:31.721 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 8 2023-05-20T01:17:31.731 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-20T01:17:31.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.852+0000 7f293f26a700 1 Processor -- start 2023-05-20T01:17:31.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.852+0000 7f293f26a700 1 -- start start 2023-05-20T01:17:31.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.853+0000 7f293f26a700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f293815caf0 0x7f293806cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:31.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.853+0000 7f293f26a700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f293815cec0 con 0x7f29380ccf00 2023-05-20T01:17:31.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.853+0000 7f293f26a700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29380c98c0 con 0x7f29380b1e00 2023-05-20T01:17:31.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.853+0000 7f293f26a700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29380d0490 con 0x7f293815caf0 2023-05-20T01:17:31.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.853+0000 7f293c805700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f29380ccf00 0x7f29380c8aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:43028/0 (socket says 172.21.15.16:43028) 2023-05-20T01:17:31.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.853+0000 7f293c805700 1 -- 172.21.15.16:0/1690022083 learned_addr learned my addr 172.21.15.16:0/1690022083 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:17:31.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.854+0000 7f292ffff700 1 -- 172.21.15.16:0/1690022083 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2316652266 0 0) 0x7f293815cec0 con 0x7f29380ccf00 2023-05-20T01:17:31.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.854+0000 7f293d006700 1 --2- 172.21.15.16:0/1690022083 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f293815caf0 0x7f293806cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:31.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.854+0000 7f292ffff700 1 -- 172.21.15.16:0/1690022083 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f29240035b0 con 0x7f29380ccf00 2023-05-20T01:17:31.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.854+0000 7f292ffff700 1 -- 172.21.15.16:0/1690022083 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3309834701 0 0) 0x7f29240035b0 con 0x7f29380ccf00 2023-05-20T01:17:31.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.854+0000 7f292ffff700 1 -- 172.21.15.16:0/1690022083 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f293815caf0 msgr2=0x7f293806cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:31.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.854+0000 7f292ffff700 1 --2- 172.21.15.16:0/1690022083 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f293815caf0 0x7f293806cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:31.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.854+0000 7f292ffff700 1 -- 172.21.15.16:0/1690022083 >> v1:172.21.15.132:6789/0 conn(0x7f29380b1e00 legacy=0x7f29380c91b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:31.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.854+0000 7f292ffff700 1 -- 172.21.15.16:0/1690022083 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2938160610 con 0x7f29380ccf00 2023-05-20T01:17:31.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.854+0000 7f292ffff700 1 -- 172.21.15.16:0/1690022083 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7f2934001670 con 0x7f29380ccf00 2023-05-20T01:17:31.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.854+0000 7f292ffff700 1 -- 172.21.15.16:0/1690022083 >> v1:172.21.15.16:6789/0 conn(0x7f29380ccf00 legacy=0x7f29380c8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:31.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.854+0000 7f292ffff700 1 --2- 172.21.15.16:0/1690022083 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f293815caf0 0x7f2924003aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:31.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.854+0000 7f292ffff700 1 --2- 172.21.15.16:0/1690022083 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2924004020 0x7f2924004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:31.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.854+0000 7f292ffff700 1 --2- 172.21.15.16:0/1690022083 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2924004d90 0x7f2924005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:31.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.854+0000 7f292ffff700 1 -- 172.21.15.16:0/1690022083 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29380d0490 con 0x7f2924004d90 2023-05-20T01:17:31.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.854+0000 7f292ffff700 1 -- 172.21.15.16:0/1690022083 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f292400bb50 con 0x7f2924004020 2023-05-20T01:17:31.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.854+0000 7f292ffff700 1 -- 172.21.15.16:0/1690022083 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f292400bd10 con 0x7f293815caf0 2023-05-20T01:17:31.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.854+0000 7f293d807700 1 --2- 172.21.15.16:0/1690022083 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2924004d90 0x7f2924005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:31.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.854+0000 7f293d006700 1 --2- 172.21.15.16:0/1690022083 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f293815caf0 0x7f2924003aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:31.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.854+0000 7f293c805700 1 --2- 172.21.15.16:0/1690022083 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2924004020 0x7f2924004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:31.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.854+0000 7f293d807700 1 -- 172.21.15.16:0/1690022083 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f293815caf0 msgr2=0x7f2924003aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:31.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.855+0000 7f293d807700 1 --2- 172.21.15.16:0/1690022083 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f293815caf0 0x7f2924003aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:31.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.855+0000 7f293d807700 1 -- 172.21.15.16:0/1690022083 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2924004020 msgr2=0x7f2924004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:31.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.855+0000 7f293d807700 1 --2- 172.21.15.16:0/1690022083 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2924004020 0x7f2924004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:31.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.855+0000 7f293d807700 1 -- 172.21.15.16:0/1690022083 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29240035b0 con 0x7f2924004d90 2023-05-20T01:17:31.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.855+0000 7f293d807700 1 --2- 172.21.15.16:0/1690022083 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2924004d90 0x7f2924005200 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f293000c240 tx=0x7f2924005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2816f98c585772e3 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:31.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.855+0000 7f292ffff700 1 -- 172.21.15.16:0/1690022083 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2930001df0 con 0x7f2924004d90 2023-05-20T01:17:31.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.855+0000 7f292ffff700 1 -- 172.21.15.16:0/1690022083 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f293000cda0 con 0x7f2924004d90 2023-05-20T01:17:31.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.855+0000 7f292ffff700 1 -- 172.21.15.16:0/1690022083 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2930003c40 con 0x7f2924004d90 2023-05-20T01:17:31.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.855+0000 7f293f26a700 1 -- 172.21.15.16:0/1690022083 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2924004d90 msgr2=0x7f2924005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:31.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.855+0000 7f293f26a700 1 --2- 172.21.15.16:0/1690022083 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2924004d90 0x7f2924005200 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f293000c240 tx=0x7f2924005880 comp rx=0 tx=0).stop 2023-05-20T01:17:31.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.855+0000 7f293d807700 1 -- 172.21.15.16:0/1690022083 reap_dead start 2023-05-20T01:17:31.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.855+0000 7f293f26a700 1 -- 172.21.15.16:0/1690022083 shutdown_connections 2023-05-20T01:17:31.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.855+0000 7f293f26a700 1 -- 172.21.15.16:0/1690022083 >> 172.21.15.16:0/1690022083 conn(0x7f29380661b0 msgr2=0x7f2938066580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:31.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.855+0000 7f293f26a700 1 -- 172.21.15.16:0/1690022083 shutdown_connections 2023-05-20T01:17:31.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.856+0000 7f293f26a700 1 -- 172.21.15.16:0/1690022083 wait complete. 2023-05-20T01:17:31.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.857+0000 7f293f26a700 1 Processor -- start 2023-05-20T01:17:31.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.857+0000 7f293f26a700 1 -- start start 2023-05-20T01:17:31.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.858+0000 7f293f26a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2938152670 0x7f293806cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:31.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.858+0000 7f293f26a700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f293806d0d0 0x7f2938152e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:31.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.858+0000 7f293f26a700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2938153390 0x7f293815df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:31.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.858+0000 7f293f26a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f293806d540 con 0x7f2938152670 2023-05-20T01:17:31.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.858+0000 7f293f26a700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2938153ba0 con 0x7f293806d0d0 2023-05-20T01:17:31.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.858+0000 7f293f26a700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2938153e80 con 0x7f2938153390 2023-05-20T01:17:31.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.858+0000 7f293d006700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2938152670 0x7f293806cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:31.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.858+0000 7f293d006700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2938152670 0x7f293806cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59744/0 (socket says 172.21.15.16:59744) 2023-05-20T01:17:31.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.858+0000 7f293d006700 1 -- 172.21.15.16:0/4290237214 learned_addr learned my addr 172.21.15.16:0/4290237214 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:31.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.858+0000 7f293d807700 1 --2- 172.21.15.16:0/4290237214 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2938153390 0x7f293815df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:31.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.858+0000 7f293c805700 1 --2- 172.21.15.16:0/4290237214 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f293806d0d0 0x7f2938152e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:31.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.858+0000 7f293d006700 1 -- 172.21.15.16:0/4290237214 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2938153390 msgr2=0x7f293815df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:31.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.858+0000 7f293d006700 1 --2- 172.21.15.16:0/4290237214 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2938153390 0x7f293815df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:31.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.858+0000 7f293d006700 1 -- 172.21.15.16:0/4290237214 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f293806d0d0 msgr2=0x7f2938152e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:31.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.858+0000 7f293d006700 1 --2- 172.21.15.16:0/4290237214 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f293806d0d0 0x7f2938152e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:31.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.859+0000 7f293d006700 1 -- 172.21.15.16:0/4290237214 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2938160bd0 con 0x7f2938152670 2023-05-20T01:17:31.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.859+0000 7f293d807700 1 --2- 172.21.15.16:0/4290237214 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2938153390 0x7f293815df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:17:31.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.859+0000 7f293d006700 1 --2- 172.21.15.16:0/4290237214 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2938152670 0x7f293806cb90 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f2928011aa0 tx=0x7f2928011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:31.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.859+0000 7f292dffb700 1 -- 172.21.15.16:0/4290237214 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f292800f980 con 0x7f2938152670 2023-05-20T01:17:31.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.859+0000 7f292dffb700 1 -- 172.21.15.16:0/4290237214 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f292800fae0 con 0x7f2938152670 2023-05-20T01:17:31.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.859+0000 7f293f26a700 1 -- 172.21.15.16:0/4290237214 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29381608f0 con 0x7f2938152670 2023-05-20T01:17:31.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.859+0000 7f292dffb700 1 -- 172.21.15.16:0/4290237214 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f292800fe90 con 0x7f2938152670 2023-05-20T01:17:31.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.859+0000 7f293f26a700 1 -- 172.21.15.16:0/4290237214 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2938160eb0 con 0x7f2938152670 2023-05-20T01:17:31.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.860+0000 7f293f26a700 1 -- 172.21.15.16:0/4290237214 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29380747b0 con 0x7f2938152670 2023-05-20T01:17:31.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.860+0000 7f292dffb700 1 -- 172.21.15.16:0/4290237214 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2928014440 con 0x7f2938152670 2023-05-20T01:17:31.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.861+0000 7f292dffb700 1 --2- 172.21.15.16:0/4290237214 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f291c06ace0 0x7f291c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:31.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.861+0000 7f292dffb700 1 -- 172.21.15.16:0/4290237214 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f2928089f60 con 0x7f2938152670 2023-05-20T01:17:31.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.861+0000 7f293c805700 1 --2- 172.21.15.16:0/4290237214 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f291c06ace0 0x7f291c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:31.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.861+0000 7f293c805700 1 --2- 172.21.15.16:0/4290237214 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f291c06ace0 0x7f291c06d190 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f29340048d0 tx=0x7f2934009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:31.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:31.865+0000 7f292dffb700 1 -- 172.21.15.16:0/4290237214 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f2928058830 con 0x7f2938152670 2023-05-20T01:17:32.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.066+0000 7f293f26a700 1 -- 172.21.15.16:0/4290237214 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f29380d00e0 con 0x7f2938152670 2023-05-20T01:17:32.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.068+0000 7f292dffb700 1 -- 172.21.15.16:0/4290237214 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+20121 (secure 0 0 0) 0x7f2928022090 con 0x7f2938152670 2023-05-20T01:17:32.072 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:17:32.072 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":27,"fsid":"c2245cc5-ecab-4087-a59b-bc81db901676","created":"2023-05-20T01:16:09.549230+0000","modified":"2023-05-20T01:17:16.406712+0000","last_up_change":"2023-05-20T01:16:19.788624+0000","last_in_change":"2023-05-20T01:16:12.092160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-05-20T01:16:20.934366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-05-20T01:16:21.372294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-05-20T01:16:26.208287+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-05-20T01:16:27.292268+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-05-20T01:16:30.357285+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"317a83cd-19c8-4958-a301-6087ef342289","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6825","nonce":3923634179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6827","nonce":3923634179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6831","nonce":3923634179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6829","nonce":3923634179}]},"public_addr":"172.21.15.16:6825/3923634179","cluster_addr":"172.21.15.16:6827/3923634179","heartbeat_back_addr":"172.21.15.16:6831/3923634179","heartbeat_front_addr":"172.21.15.16:6829/3923634179","state":["exists","up"]},{"osd":1,"uuid":"5581046f-8219-4089-8d7d-b7846399505e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6809","nonce":3860264220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6811","nonce":3860264220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6815","nonce":3860264220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6813","nonce":3860264220}]},"public_addr":"172.21.15.16:6809/3860264220","cluster_addr":"172.21.15.16:6811/3860264220","heartbeat_back_addr":"172.21.15.16:6815/3860264220","heartbeat_front_addr":"172.21.15.16:6813/3860264220","state":["exists","up"]},{"osd":2,"uuid":"072e9e66-4133-478e-8d4a-64d46c2963f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6817","nonce":293187030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6819","nonce":293187030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6823","nonce":293187030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6821","nonce":293187030}]},"public_addr":"172.21.15.16:6817/293187030","cluster_addr":"172.21.15.16:6819/293187030","heartbeat_back_addr":"172.21.15.16:6823/293187030","heartbeat_front_addr":"172.21.15.16:6821/293187030","state":["exists","up"]},{"osd":3,"uuid":"11e1bba6-00f8-48dd-9028-d8d716b7f145","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6801","nonce":3669781890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6803","nonce":3669781890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6807","nonce":3669781890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6805","nonce":3669781890}]},"public_addr":"172.21.15.16:6801/3669781890","cluster_addr":"172.21.15.16:6803/3669781890","heartbeat_back_addr":"172.21.15.16:6807/3669781890","heartbeat_front_addr":"172.21.15.16:6805/3669781890","state":["exists","up"]},{"osd":4,"uuid":"b8909c3a-cf70-46cb-8d0b-009c707cf9d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6819","nonce":506317223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6823","nonce":506317223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6829","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6831","nonce":506317223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6827","nonce":506317223}]},"public_addr":"172.21.15.132:6819/506317223","cluster_addr":"172.21.15.132:6823/506317223","heartbeat_back_addr":"172.21.15.132:6831/506317223","heartbeat_front_addr":"172.21.15.132:6827/506317223","state":["exists","up"]},{"osd":5,"uuid":"b1818304-4583-47bf-a6ec-0bfbb5166233","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6818","nonce":1844689995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6822","nonce":1844689995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6830","nonce":1844689995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6826","nonce":1844689995}]},"public_addr":"172.21.15.132:6818/1844689995","cluster_addr":"172.21.15.132:6822/1844689995","heartbeat_back_addr":"172.21.15.132:6830/1844689995","heartbeat_front_addr":"172.21.15.132:6826/1844689995","state":["exists","up"]},{"osd":6,"uuid":"d37f4691-eed1-4a93-8818-18354f03f4a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6801","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6802","nonce":727581123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6805","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6807","nonce":727581123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6813","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6815","nonce":727581123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6811","nonce":727581123}]},"public_addr":"172.21.15.132:6802/727581123","cluster_addr":"172.21.15.132:6807/727581123","heartbeat_back_addr":"172.21.15.132:6815/727581123","heartbeat_front_addr":"172.21.15.132:6811/727581123","state":["exists","up"]},{"osd":7,"uuid":"4f712f62-5d1d-4816-8042-03209144f8b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6803","nonce":466001149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6806","nonce":466001149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6814","nonce":466001149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6810","nonce":466001149}]},"public_addr":"172.21.15.132:6803/466001149","cluster_addr":"172.21.15.132:6806/466001149","heartbeat_back_addr":"172.21.15.132:6814/466001149","heartbeat_front_addr":"172.21.15.132:6810/466001149","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.338863+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.278680+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305694+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.246835+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.276237+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.304441+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305421+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.267050+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.a","mappings":[{"from":1,"to":5}]},{"pgid":"4.11","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-20T01:17:32.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.070+0000 7f293f26a700 1 -- 172.21.15.16:0/4290237214 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f291c06ace0 msgr2=0x7f291c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:32.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.070+0000 7f293f26a700 1 --2- 172.21.15.16:0/4290237214 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f291c06ace0 0x7f291c06d190 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f29340048d0 tx=0x7f2934009040 comp rx=0 tx=0).stop 2023-05-20T01:17:32.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.070+0000 7f293f26a700 1 -- 172.21.15.16:0/4290237214 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2938152670 msgr2=0x7f293806cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:32.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.070+0000 7f293f26a700 1 --2- 172.21.15.16:0/4290237214 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2938152670 0x7f293806cb90 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f2928011aa0 tx=0x7f2928011e60 comp rx=0 tx=0).stop 2023-05-20T01:17:32.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.070+0000 7f293f26a700 1 -- 172.21.15.16:0/4290237214 shutdown_connections 2023-05-20T01:17:32.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.070+0000 7f293f26a700 1 --2- 172.21.15.16:0/4290237214 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f291c06ace0 0x7f291c06d190 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:32.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.070+0000 7f293f26a700 1 --2- 172.21.15.16:0/4290237214 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2938153390 0x7f293815df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:32.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.070+0000 7f293f26a700 1 --2- 172.21.15.16:0/4290237214 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2938152670 0x7f293806cb90 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:32.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.070+0000 7f293f26a700 1 --2- 172.21.15.16:0/4290237214 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f293806d0d0 0x7f2938152e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:32.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.070+0000 7f293f26a700 1 -- 172.21.15.16:0/4290237214 >> 172.21.15.16:0/4290237214 conn(0x7f29380661b0 msgr2=0x7f29380c8aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:32.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.070+0000 7f293f26a700 1 -- 172.21.15.16:0/4290237214 shutdown_connections 2023-05-20T01:17:32.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.070+0000 7f293f26a700 1 -- 172.21.15.16:0/4290237214 wait complete. 2023-05-20T01:17:32.086 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-05-20T01:17:32.086 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-20T01:17:32.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.207+0000 7f0acab3d700 1 Processor -- start 2023-05-20T01:17:32.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.207+0000 7f0acab3d700 1 -- start start 2023-05-20T01:17:32.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.208+0000 7f0acab3d700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f0ac406d4b0 0x7f0ac40cf890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:32.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.208+0000 7f0acab3d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0ac406a510 0x7f0ac406a950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:32.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.208+0000 7f0acab3d700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ac40cc510 con 0x7f0ac406f450 2023-05-20T01:17:32.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.208+0000 7f0acab3d700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ac406ae90 con 0x7f0ac406a510 2023-05-20T01:17:32.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.208+0000 7f0acab3d700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ac4071ed0 con 0x7f0ac406d4b0 2023-05-20T01:17:32.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.208+0000 7f0ac90da700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0ac406a510 0x7f0ac406a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:32.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.208+0000 7f0ac88d9700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f0ac406f450 0x7f0ac4161080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:43048/0 (socket says 172.21.15.16:43048) 2023-05-20T01:17:32.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.208+0000 7f0ac88d9700 1 -- 172.21.15.16:0/54473942 learned_addr learned my addr 172.21.15.16:0/54473942 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:17:32.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.208+0000 7f0ac3fff700 1 --2- 172.21.15.16:0/54473942 >> v2:172.21.15.132:3300/0 conn(0x7f0ac406d4b0 0x7f0ac40cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:32.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.209+0000 7f0ac90da700 1 -- 172.21.15.16:0/54473942 >> v2:172.21.15.132:3300/0 conn(0x7f0ac406d4b0 msgr2=0x7f0ac40cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:32.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.209+0000 7f0ac90da700 1 --2- 172.21.15.16:0/54473942 >> v2:172.21.15.132:3300/0 conn(0x7f0ac406d4b0 0x7f0ac40cf890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:32.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.209+0000 7f0ac90da700 1 -- 172.21.15.16:0/54473942 >> v1:172.21.15.16:6789/0 conn(0x7f0ac406f450 legacy=0x7f0ac4161080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:32.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.209+0000 7f0ac90da700 1 -- 172.21.15.16:0/54473942 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ac40721b0 con 0x7f0ac406a510 2023-05-20T01:17:32.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.209+0000 7f0ac37fe700 1 -- 172.21.15.16:0/54473942 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2235066448 0 0) 0x7f0ac40cc510 con 0x7f0ac406f450 2023-05-20T01:17:32.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.209+0000 7f0ac90da700 1 --2- 172.21.15.16:0/54473942 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ac406a510 0x7f0ac406a950 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f0ac4070c40 tx=0x7f0abc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79b35a3d7507535b server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:32.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.209+0000 7f0ac37fe700 1 -- 172.21.15.16:0/54473942 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0abc00e070 con 0x7f0ac406a510 2023-05-20T01:17:32.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.209+0000 7f0ac37fe700 1 -- 172.21.15.16:0/54473942 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0abc00bb20 con 0x7f0ac406a510 2023-05-20T01:17:32.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.209+0000 7f0ac37fe700 1 -- 172.21.15.16:0/54473942 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0abc012430 con 0x7f0ac406a510 2023-05-20T01:17:32.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.209+0000 7f0acab3d700 1 -- 172.21.15.16:0/54473942 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ac406a510 msgr2=0x7f0ac406a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:32.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.209+0000 7f0acab3d700 1 --2- 172.21.15.16:0/54473942 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ac406a510 0x7f0ac406a950 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f0ac4070c40 tx=0x7f0abc00b330 comp rx=0 tx=0).stop 2023-05-20T01:17:32.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.210+0000 7f0acab3d700 1 -- 172.21.15.16:0/54473942 shutdown_connections 2023-05-20T01:17:32.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.210+0000 7f0acab3d700 1 --2- 172.21.15.16:0/54473942 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ac406a510 0x7f0ac406a950 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:32.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.210+0000 7f0acab3d700 1 --2- 172.21.15.16:0/54473942 >> v2:172.21.15.132:3300/0 conn(0x7f0ac406d4b0 0x7f0ac40cf890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:32.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.210+0000 7f0acab3d700 1 -- 172.21.15.16:0/54473942 >> 172.21.15.16:0/54473942 conn(0x7f0ac40cffb0 msgr2=0x7f0ac40d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:32.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.210+0000 7f0acab3d700 1 -- 172.21.15.16:0/54473942 shutdown_connections 2023-05-20T01:17:32.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.210+0000 7f0acab3d700 1 -- 172.21.15.16:0/54473942 wait complete. 2023-05-20T01:17:32.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.211+0000 7f0acab3d700 1 Processor -- start 2023-05-20T01:17:32.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.211+0000 7f0acab3d700 1 -- start start 2023-05-20T01:17:32.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.212+0000 7f0acab3d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0ac406a510 0x7f0ac4146280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:32.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.212+0000 7f0acab3d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ac406d4b0 0x7f0ac40b5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:32.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.212+0000 7f0acab3d700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0ac406f450 0x7f0ac40c5af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:32.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.212+0000 7f0acab3d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac4147d50 con 0x7f0ac406d4b0 2023-05-20T01:17:32.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.212+0000 7f0acab3d700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac41470d0 con 0x7f0ac406a510 2023-05-20T01:17:32.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.212+0000 7f0acab3d700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac41473b0 con 0x7f0ac406f450 2023-05-20T01:17:32.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.212+0000 7f0ac3fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ac406d4b0 0x7f0ac40b5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:32.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.212+0000 7f0ac3fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ac406d4b0 0x7f0ac40b5110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59760/0 (socket says 172.21.15.16:59760) 2023-05-20T01:17:32.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.212+0000 7f0ac3fff700 1 -- 172.21.15.16:0/3641478697 learned_addr learned my addr 172.21.15.16:0/3641478697 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:32.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.212+0000 7f0ac88d9700 1 --2- 172.21.15.16:0/3641478697 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0ac406a510 0x7f0ac4146280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:32.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.212+0000 7f0ac90da700 1 --2- 172.21.15.16:0/3641478697 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0ac406f450 0x7f0ac40c5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:32.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.212+0000 7f0ac3fff700 1 -- 172.21.15.16:0/3641478697 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0ac406f450 msgr2=0x7f0ac40c5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:32.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.212+0000 7f0ac3fff700 1 --2- 172.21.15.16:0/3641478697 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0ac406f450 0x7f0ac40c5af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:32.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.212+0000 7f0ac3fff700 1 -- 172.21.15.16:0/3641478697 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0ac406a510 msgr2=0x7f0ac4146280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:32.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.212+0000 7f0ac3fff700 1 --2- 172.21.15.16:0/3641478697 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0ac406a510 0x7f0ac4146280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:32.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.212+0000 7f0ac3fff700 1 -- 172.21.15.16:0/3641478697 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0abc009a70 con 0x7f0ac406d4b0 2023-05-20T01:17:32.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.212+0000 7f0ac88d9700 1 --2- 172.21.15.16:0/3641478697 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0ac406a510 0x7f0ac4146280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:17:32.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.212+0000 7f0ac90da700 1 --2- 172.21.15.16:0/3641478697 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0ac406f450 0x7f0ac40c5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:17:32.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.212+0000 7f0ac3fff700 1 --2- 172.21.15.16:0/3641478697 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ac406d4b0 0x7f0ac40b5110 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f0ab000e9b0 tx=0x7f0ab000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:32.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.212+0000 7f0ac17fa700 1 -- 172.21.15.16:0/3641478697 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab0009710 con 0x7f0ac406d4b0 2023-05-20T01:17:32.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.212+0000 7f0ac17fa700 1 -- 172.21.15.16:0/3641478697 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ab000c850 con 0x7f0ac406d4b0 2023-05-20T01:17:32.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.212+0000 7f0acab3d700 1 -- 172.21.15.16:0/3641478697 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ac40cbe50 con 0x7f0ac406d4b0 2023-05-20T01:17:32.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.212+0000 7f0ac17fa700 1 -- 172.21.15.16:0/3641478697 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab000cbc0 con 0x7f0ac406d4b0 2023-05-20T01:17:32.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.212+0000 7f0acab3d700 1 -- 172.21.15.16:0/3641478697 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ac40cc1a0 con 0x7f0ac406d4b0 2023-05-20T01:17:32.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.213+0000 7f0acab3d700 1 -- 172.21.15.16:0/3641478697 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ac40cc980 con 0x7f0ac406d4b0 2023-05-20T01:17:32.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.214+0000 7f0ac17fa700 1 -- 172.21.15.16:0/3641478697 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0ab000cd20 con 0x7f0ac406d4b0 2023-05-20T01:17:32.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.214+0000 7f0ac17fa700 1 --2- 172.21.15.16:0/3641478697 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f0aa806ace0 0x7f0aa806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:32.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.214+0000 7f0ac17fa700 1 -- 172.21.15.16:0/3641478697 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f0ab00898c0 con 0x7f0ac406d4b0 2023-05-20T01:17:32.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.214+0000 7f0ac88d9700 1 --2- 172.21.15.16:0/3641478697 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f0aa806ace0 0x7f0aa806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:32.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.215+0000 7f0ac88d9700 1 --2- 172.21.15.16:0/3641478697 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f0aa806ace0 0x7f0aa806d190 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0ab8000f60 tx=0x7f0ab8007d30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:32.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.219+0000 7f0ac17fa700 1 -- 172.21.15.16:0/3641478697 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f0ab0058160 con 0x7f0ac406d4b0 2023-05-20T01:17:32.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.464+0000 7f0acab3d700 1 -- 172.21.15.16:0/3641478697 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0ac4147e90 con 0x7f0ac406d4b0 2023-05-20T01:17:32.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.465+0000 7f0ac17fa700 1 -- 172.21.15.16:0/3641478697 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4673 (secure 0 0 0) 0x7f0ac4147e90 con 0x7f0ac406d4b0 2023-05-20T01:17:32.469 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:17:32.469 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4273,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/1444124146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1444124146},{"type":"v1","addr":"172.21.15.132:6834","nonce":1444124146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4293,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4092119206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4092119206},{"type":"v1","addr":"172.21.15.16:6835","nonce":4092119206}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2162680372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":2162680372},{"type":"v1","addr":"172.21.15.132:6835","nonce":2162680372}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-20T01:16:28.316109+0000","modified":"2023-05-20T01:16:45.362572+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4299},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4299":{"gid":4299,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6837/2330551626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2330551626},{"type":"v1","addr":"172.21.15.16:6837","nonce":2330551626}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-05-20T01:17:32.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.467+0000 7f0acab3d700 1 -- 172.21.15.16:0/3641478697 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f0aa806ace0 msgr2=0x7f0aa806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:32.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.467+0000 7f0acab3d700 1 --2- 172.21.15.16:0/3641478697 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f0aa806ace0 0x7f0aa806d190 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0ab8000f60 tx=0x7f0ab8007d30 comp rx=0 tx=0).stop 2023-05-20T01:17:32.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.467+0000 7f0acab3d700 1 -- 172.21.15.16:0/3641478697 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ac406d4b0 msgr2=0x7f0ac40b5110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:32.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.467+0000 7f0acab3d700 1 --2- 172.21.15.16:0/3641478697 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ac406d4b0 0x7f0ac40b5110 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f0ab000e9b0 tx=0x7f0ab000ed70 comp rx=0 tx=0).stop 2023-05-20T01:17:32.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.467+0000 7f0acab3d700 1 -- 172.21.15.16:0/3641478697 shutdown_connections 2023-05-20T01:17:32.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.467+0000 7f0acab3d700 1 --2- 172.21.15.16:0/3641478697 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f0aa806ace0 0x7f0aa806d190 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:32.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.467+0000 7f0acab3d700 1 --2- 172.21.15.16:0/3641478697 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0ac406f450 0x7f0ac40c5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:32.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.467+0000 7f0acab3d700 1 --2- 172.21.15.16:0/3641478697 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0ac406a510 0x7f0ac4146280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:32.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.467+0000 7f0acab3d700 1 --2- 172.21.15.16:0/3641478697 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ac406d4b0 0x7f0ac40b5110 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:32.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.467+0000 7f0acab3d700 1 -- 172.21.15.16:0/3641478697 >> 172.21.15.16:0/3641478697 conn(0x7f0ac40cffb0 msgr2=0x7f0ac40ccda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:32.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.467+0000 7f0acab3d700 1 -- 172.21.15.16:0/3641478697 shutdown_connections 2023-05-20T01:17:32.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.467+0000 7f0acab3d700 1 -- 172.21.15.16:0/3641478697 wait complete. 2023-05-20T01:17:32.472 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 8 2023-05-20T01:17:32.482 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-20T01:17:32.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.603+0000 7f08afd60700 1 Processor -- start 2023-05-20T01:17:32.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.603+0000 7f08afd60700 1 -- start start 2023-05-20T01:17:32.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.604+0000 7f08afd60700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f08a8071340 0x7f08a8071760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:32.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.604+0000 7f08afd60700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08a815e060 con 0x7f08a815e2a0 2023-05-20T01:17:32.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.604+0000 7f08afd60700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08a815d450 con 0x7f08a806a510 2023-05-20T01:17:32.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.604+0000 7f08afd60700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08a815d730 con 0x7f08a8071340 2023-05-20T01:17:32.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.604+0000 7f08ae2fd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f08a8071340 0x7f08a8071760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:32.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.604+0000 7f08ad2fb700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f08a815e2a0 0x7f08a80cf5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:43064/0 (socket says 172.21.15.16:43064) 2023-05-20T01:17:32.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.604+0000 7f08ad2fb700 1 -- 172.21.15.16:0/3941380899 learned_addr learned my addr 172.21.15.16:0/3941380899 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:17:32.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.604+0000 7f08ae2fd700 1 -- 172.21.15.16:0/3941380899 >> v1:172.21.15.132:6789/0 conn(0x7f08a806a510 legacy=0x7f08a806a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:32.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.604+0000 7f08ae2fd700 1 -- 172.21.15.16:0/3941380899 >> v1:172.21.15.16:6789/0 conn(0x7f08a815e2a0 legacy=0x7f08a80cf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:32.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.604+0000 7f08acafa700 1 -- 172.21.15.16:0/3941380899 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2284892270 0 0) 0x7f08a815e060 con 0x7f08a815e2a0 2023-05-20T01:17:32.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.604+0000 7f08ae2fd700 1 -- 172.21.15.16:0/3941380899 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08a815da10 con 0x7f08a8071340 2023-05-20T01:17:32.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.604+0000 7f08ae2fd700 1 --2- 172.21.15.16:0/3941380899 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08a8071340 0x7f08a8071760 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f08a0009d90 tx=0x7f08a000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c45111f23bb8743 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:32.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.604+0000 7f08acafa700 1 -- 172.21.15.16:0/3941380899 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a000e070 con 0x7f08a8071340 2023-05-20T01:17:32.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.604+0000 7f08acafa700 1 -- 172.21.15.16:0/3941380899 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08a000bbd0 con 0x7f08a8071340 2023-05-20T01:17:32.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.605+0000 7f08acafa700 1 -- 172.21.15.16:0/3941380899 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a0012430 con 0x7f08a8071340 2023-05-20T01:17:32.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.605+0000 7f08afd60700 1 -- 172.21.15.16:0/3941380899 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08a8071340 msgr2=0x7f08a8071760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:32.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.605+0000 7f08afd60700 1 --2- 172.21.15.16:0/3941380899 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08a8071340 0x7f08a8071760 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f08a0009d90 tx=0x7f08a000b330 comp rx=0 tx=0).stop 2023-05-20T01:17:32.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.605+0000 7f08afd60700 1 -- 172.21.15.16:0/3941380899 shutdown_connections 2023-05-20T01:17:32.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.605+0000 7f08afd60700 1 --2- 172.21.15.16:0/3941380899 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08a8071340 0x7f08a8071760 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:32.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.605+0000 7f08afd60700 1 -- 172.21.15.16:0/3941380899 >> 172.21.15.16:0/3941380899 conn(0x7f08a80d0c30 msgr2=0x7f08a80b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:32.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.605+0000 7f08afd60700 1 -- 172.21.15.16:0/3941380899 shutdown_connections 2023-05-20T01:17:32.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.605+0000 7f08afd60700 1 -- 172.21.15.16:0/3941380899 wait complete. 2023-05-20T01:17:32.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.606+0000 7f08afd60700 1 Processor -- start 2023-05-20T01:17:32.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.606+0000 7f08afd60700 1 -- start start 2023-05-20T01:17:32.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.607+0000 7f08afd60700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08a806a510 0x7f08a8155a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:32.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.607+0000 7f08afd60700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f08a8071340 0x7f08a81f23f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:32.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.607+0000 7f08afd60700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f08a815e2a0 0x7f08a8156220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:32.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.607+0000 7f08afd60700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08a80b42e0 con 0x7f08a806a510 2023-05-20T01:17:32.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.607+0000 7f08afd60700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08a80b45c0 con 0x7f08a8071340 2023-05-20T01:17:32.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.607+0000 7f08afd60700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08a80b48a0 con 0x7f08a815e2a0 2023-05-20T01:17:32.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.607+0000 7f08adafc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08a806a510 0x7f08a8155a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:32.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.607+0000 7f08adafc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08a806a510 0x7f08a8155a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59772/0 (socket says 172.21.15.16:59772) 2023-05-20T01:17:32.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.607+0000 7f08adafc700 1 -- 172.21.15.16:0/4278989076 learned_addr learned my addr 172.21.15.16:0/4278989076 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:32.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.607+0000 7f08ad2fb700 1 --2- 172.21.15.16:0/4278989076 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f08a8071340 0x7f08a81f23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:32.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.607+0000 7f08ae2fd700 1 --2- 172.21.15.16:0/4278989076 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f08a815e2a0 0x7f08a8156220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:32.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.607+0000 7f08adafc700 1 -- 172.21.15.16:0/4278989076 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f08a815e2a0 msgr2=0x7f08a8156220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:32.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.607+0000 7f08adafc700 1 --2- 172.21.15.16:0/4278989076 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f08a815e2a0 0x7f08a8156220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:32.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.607+0000 7f08adafc700 1 -- 172.21.15.16:0/4278989076 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f08a8071340 msgr2=0x7f08a81f23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:32.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.607+0000 7f08adafc700 1 --2- 172.21.15.16:0/4278989076 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f08a8071340 0x7f08a81f23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:32.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.608+0000 7f08adafc700 1 -- 172.21.15.16:0/4278989076 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08a0009a70 con 0x7f08a806a510 2023-05-20T01:17:32.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.608+0000 7f08ad2fb700 1 --2- 172.21.15.16:0/4278989076 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f08a8071340 0x7f08a81f23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:17:32.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.608+0000 7f08ae2fd700 1 --2- 172.21.15.16:0/4278989076 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f08a815e2a0 0x7f08a8156220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:17:32.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.608+0000 7f08adafc700 1 --2- 172.21.15.16:0/4278989076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08a806a510 0x7f08a8155a40 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f08a400c4e0 tx=0x7f08a400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:32.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.608+0000 7f089e7fc700 1 -- 172.21.15.16:0/4278989076 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a40052d0 con 0x7f08a806a510 2023-05-20T01:17:32.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.608+0000 7f089e7fc700 1 -- 172.21.15.16:0/4278989076 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08a4005430 con 0x7f08a806a510 2023-05-20T01:17:32.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.608+0000 7f08afd60700 1 -- 172.21.15.16:0/4278989076 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08a80cfd00 con 0x7f08a806a510 2023-05-20T01:17:32.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.608+0000 7f089e7fc700 1 -- 172.21.15.16:0/4278989076 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a40038c0 con 0x7f08a806a510 2023-05-20T01:17:32.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.608+0000 7f08afd60700 1 -- 172.21.15.16:0/4278989076 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08a815dcf0 con 0x7f08a806a510 2023-05-20T01:17:32.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.609+0000 7f08afd60700 1 -- 172.21.15.16:0/4278989076 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08a815d080 con 0x7f08a806a510 2023-05-20T01:17:32.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.609+0000 7f089e7fc700 1 -- 172.21.15.16:0/4278989076 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f08a4003a20 con 0x7f08a806a510 2023-05-20T01:17:32.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.610+0000 7f089e7fc700 1 --2- 172.21.15.16:0/4278989076 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f088c06ace0 0x7f088c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:32.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.610+0000 7f089e7fc700 1 -- 172.21.15.16:0/4278989076 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f08a4088cd0 con 0x7f08a806a510 2023-05-20T01:17:32.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.610+0000 7f08ad2fb700 1 --2- 172.21.15.16:0/4278989076 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f088c06ace0 0x7f088c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:32.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.610+0000 7f08ad2fb700 1 --2- 172.21.15.16:0/4278989076 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f088c06ace0 0x7f088c06d190 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f0898000f60 tx=0x7f0898009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:32.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.614+0000 7f089e7fc700 1 -- 172.21.15.16:0/4278989076 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f08a4053a50 con 0x7f08a806a510 2023-05-20T01:17:32.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.857+0000 7f08afd60700 1 -- 172.21.15.16:0/4278989076 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f08a80747b0 con 0x7f08a806a510 2023-05-20T01:17:32.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.857+0000 7f089e7fc700 1 -- 172.21.15.16:0/4278989076 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4673 (secure 0 0 0) 0x7f08a4057070 con 0x7f08a806a510 2023-05-20T01:17:32.862 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:17:32.862 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4273,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/1444124146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1444124146},{"type":"v1","addr":"172.21.15.132:6834","nonce":1444124146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4293,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4092119206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4092119206},{"type":"v1","addr":"172.21.15.16:6835","nonce":4092119206}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2162680372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":2162680372},{"type":"v1","addr":"172.21.15.132:6835","nonce":2162680372}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-20T01:16:28.316109+0000","modified":"2023-05-20T01:16:45.362572+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4299},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4299":{"gid":4299,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6837/2330551626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2330551626},{"type":"v1","addr":"172.21.15.16:6837","nonce":2330551626}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-05-20T01:17:32.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.860+0000 7f08afd60700 1 -- 172.21.15.16:0/4278989076 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f088c06ace0 msgr2=0x7f088c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:32.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.860+0000 7f08afd60700 1 --2- 172.21.15.16:0/4278989076 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f088c06ace0 0x7f088c06d190 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f0898000f60 tx=0x7f0898009040 comp rx=0 tx=0).stop 2023-05-20T01:17:32.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.860+0000 7f08afd60700 1 -- 172.21.15.16:0/4278989076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08a806a510 msgr2=0x7f08a8155a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:32.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.860+0000 7f08afd60700 1 --2- 172.21.15.16:0/4278989076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08a806a510 0x7f08a8155a40 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f08a400c4e0 tx=0x7f08a400c8a0 comp rx=0 tx=0).stop 2023-05-20T01:17:32.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.860+0000 7f08afd60700 1 -- 172.21.15.16:0/4278989076 shutdown_connections 2023-05-20T01:17:32.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.860+0000 7f08afd60700 1 --2- 172.21.15.16:0/4278989076 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f088c06ace0 0x7f088c06d190 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:32.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.860+0000 7f08afd60700 1 --2- 172.21.15.16:0/4278989076 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f08a815e2a0 0x7f08a8156220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:32.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.860+0000 7f08afd60700 1 --2- 172.21.15.16:0/4278989076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08a806a510 0x7f08a8155a40 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:32.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.860+0000 7f08afd60700 1 --2- 172.21.15.16:0/4278989076 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f08a8071340 0x7f08a81f23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:32.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.860+0000 7f08afd60700 1 -- 172.21.15.16:0/4278989076 >> 172.21.15.16:0/4278989076 conn(0x7f08a80d0c30 msgr2=0x7f08a815f950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:32.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.860+0000 7f08afd60700 1 -- 172.21.15.16:0/4278989076 shutdown_connections 2023-05-20T01:17:32.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.860+0000 7f08afd60700 1 -- 172.21.15.16:0/4278989076 wait complete. 2023-05-20T01:17:32.867 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 8 2023-05-20T01:17:32.876 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-20T01:17:33.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.998+0000 7fe1f8b14700 1 Processor -- start 2023-05-20T01:17:33.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.998+0000 7fe1f8b14700 1 -- start start 2023-05-20T01:17:33.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.999+0000 7fe1f8b14700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fe1f40cb320 0x7fe1f4070910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:33.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.999+0000 7fe1f8b14700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1f40ca350 con 0x7fe1f406d2a0 2023-05-20T01:17:33.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.999+0000 7fe1f8b14700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1f415cac0 con 0x7fe1f4159000 2023-05-20T01:17:33.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.999+0000 7fe1f8b14700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1f415cda0 con 0x7fe1f40cb320 2023-05-20T01:17:33.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.999+0000 7fe1f2d9d700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fe1f406d2a0 0x7fe1f4070e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:43080/0 (socket says 172.21.15.16:43080) 2023-05-20T01:17:33.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.999+0000 7fe1f2d9d700 1 -- 172.21.15.16:0/1101985095 learned_addr learned my addr 172.21.15.16:0/1101985095 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:17:33.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.999+0000 7fe1f1d9b700 1 --2- 172.21.15.16:0/1101985095 >> v2:172.21.15.132:3300/0 conn(0x7fe1f40cb320 0x7fe1f4070910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:33.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.999+0000 7fe1f159a700 1 -- 172.21.15.16:0/1101985095 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1032262466 0 0) 0x7fe1f40ca350 con 0x7fe1f406d2a0 2023-05-20T01:17:33.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.999+0000 7fe1f159a700 1 -- 172.21.15.16:0/1101985095 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe1e00035b0 con 0x7fe1f406d2a0 2023-05-20T01:17:33.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.999+0000 7fe1f159a700 1 -- 172.21.15.16:0/1101985095 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 871213926 0 0) 0x7fe1f415cac0 con 0x7fe1f4159000 2023-05-20T01:17:33.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.999+0000 7fe1f159a700 1 -- 172.21.15.16:0/1101985095 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe1f40ca350 con 0x7fe1f4159000 2023-05-20T01:17:33.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.999+0000 7fe1f159a700 1 -- 172.21.15.16:0/1101985095 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3258964489 0 0) 0x7fe1e00035b0 con 0x7fe1f406d2a0 2023-05-20T01:17:33.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.999+0000 7fe1f159a700 1 -- 172.21.15.16:0/1101985095 >> v2:172.21.15.132:3300/0 conn(0x7fe1f40cb320 msgr2=0x7fe1f4070910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:33.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.999+0000 7fe1f159a700 1 --2- 172.21.15.16:0/1101985095 >> v2:172.21.15.132:3300/0 conn(0x7fe1f40cb320 0x7fe1f4070910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:33.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.999+0000 7fe1f159a700 1 -- 172.21.15.16:0/1101985095 >> v1:172.21.15.132:6789/0 conn(0x7fe1f4159000 legacy=0x7fe1f406cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:33.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.999+0000 7fe1f159a700 1 -- 172.21.15.16:0/1101985095 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1f415d080 con 0x7fe1f406d2a0 2023-05-20T01:17:33.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.999+0000 7fe1f159a700 1 -- 172.21.15.16:0/1101985095 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7fe1ec0019f0 con 0x7fe1f406d2a0 2023-05-20T01:17:33.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.999+0000 7fe1f159a700 1 -- 172.21.15.16:0/1101985095 >> v1:172.21.15.16:6789/0 conn(0x7fe1f406d2a0 legacy=0x7fe1f4070e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:33.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.999+0000 7fe1f159a700 1 --2- 172.21.15.16:0/1101985095 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe1e0003c00 0x7fe1e0003ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:33.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.999+0000 7fe1f159a700 1 --2- 172.21.15.16:0/1101985095 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe1e0004850 0x7fe1e0004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:33.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.999+0000 7fe1f159a700 1 --2- 172.21.15.16:0/1101985095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1e0005200 0x7fe1e000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:33.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.999+0000 7fe1f159a700 1 -- 172.21.15.16:0/1101985095 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1f415cda0 con 0x7fe1e0005200 2023-05-20T01:17:33.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.999+0000 7fe1f159a700 1 -- 172.21.15.16:0/1101985095 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e000c0e0 con 0x7fe1e0003c00 2023-05-20T01:17:33.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:32.999+0000 7fe1f159a700 1 -- 172.21.15.16:0/1101985095 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e000c2a0 con 0x7fe1e0004850 2023-05-20T01:17:33.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.000+0000 7fe1f2d9d700 1 --2- 172.21.15.16:0/1101985095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1e0005200 0x7fe1e000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:33.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.000+0000 7fe1f259c700 1 --2- 172.21.15.16:0/1101985095 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe1e0003c00 0x7fe1e0003ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:33.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.000+0000 7fe1f1d9b700 1 --2- 172.21.15.16:0/1101985095 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe1e0004850 0x7fe1e0004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:33.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.000+0000 7fe1f2d9d700 1 -- 172.21.15.16:0/1101985095 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe1e0004850 msgr2=0x7fe1e0004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:33.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.000+0000 7fe1f2d9d700 1 --2- 172.21.15.16:0/1101985095 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe1e0004850 0x7fe1e0004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:33.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.000+0000 7fe1f2d9d700 1 -- 172.21.15.16:0/1101985095 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe1e0003c00 msgr2=0x7fe1e0003ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:33.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.000+0000 7fe1f2d9d700 1 --2- 172.21.15.16:0/1101985095 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe1e0003c00 0x7fe1e0003ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:33.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.000+0000 7fe1f2d9d700 1 -- 172.21.15.16:0/1101985095 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1e00035b0 con 0x7fe1e0005200 2023-05-20T01:17:33.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.000+0000 7fe1f2d9d700 1 -- 172.21.15.16:0/1101985095 reap_dead start 2023-05-20T01:17:33.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.000+0000 7fe1f2d9d700 1 --2- 172.21.15.16:0/1101985095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1e0005200 0x7fe1e000ba10 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fe1ec002230 tx=0x7fe1e0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84cc3bef65b6f3d8 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:33.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.000+0000 7fe1f159a700 1 -- 172.21.15.16:0/1101985095 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1ec0072d0 con 0x7fe1e0005200 2023-05-20T01:17:33.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.000+0000 7fe1f159a700 1 -- 172.21.15.16:0/1101985095 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1ec007430 con 0x7fe1e0005200 2023-05-20T01:17:33.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.000+0000 7fe1f159a700 1 -- 172.21.15.16:0/1101985095 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1ec0077c0 con 0x7fe1e0005200 2023-05-20T01:17:33.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.001+0000 7fe1f8b14700 1 -- 172.21.15.16:0/1101985095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1e0005200 msgr2=0x7fe1e000ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:33.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.001+0000 7fe1f8b14700 1 --2- 172.21.15.16:0/1101985095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1e0005200 0x7fe1e000ba10 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fe1ec002230 tx=0x7fe1e0005880 comp rx=0 tx=0).stop 2023-05-20T01:17:33.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.001+0000 7fe1f8b14700 1 -- 172.21.15.16:0/1101985095 shutdown_connections 2023-05-20T01:17:33.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.001+0000 7fe1f8b14700 1 --2- 172.21.15.16:0/1101985095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1e0005200 0x7fe1e000ba10 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:33.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.001+0000 7fe1f8b14700 1 -- 172.21.15.16:0/1101985095 >> 172.21.15.16:0/1101985095 conn(0x7fe1f40661b0 msgr2=0x7fe1f4066580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:33.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.001+0000 7fe1f8b14700 1 -- 172.21.15.16:0/1101985095 shutdown_connections 2023-05-20T01:17:33.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.001+0000 7fe1f8b14700 1 -- 172.21.15.16:0/1101985095 wait complete. 2023-05-20T01:17:33.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.002+0000 7fe1f8b14700 1 Processor -- start 2023-05-20T01:17:33.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.002+0000 7fe1f8b14700 1 -- start start 2023-05-20T01:17:33.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.003+0000 7fe1f8b14700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe1e0005200 0x7fe1f40709f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:33.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.003+0000 7fe1f8b14700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1f4070f30 0x7fe1f4152f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:33.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.003+0000 7fe1f8b14700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe1f41534b0 0x7fe1f41406c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:33.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.003+0000 7fe1f8b14700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1f41435e0 con 0x7fe1f4070f30 2023-05-20T01:17:33.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.003+0000 7fe1f8b14700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1f4142ff0 con 0x7fe1e0005200 2023-05-20T01:17:33.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.003+0000 7fe1f8b14700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1f4153950 con 0x7fe1f41534b0 2023-05-20T01:17:33.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.003+0000 7fe1f1d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1f4070f30 0x7fe1f4152f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:33.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.003+0000 7fe1f1d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1f4070f30 0x7fe1f4152f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59794/0 (socket says 172.21.15.16:59794) 2023-05-20T01:17:33.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.003+0000 7fe1f1d9b700 1 -- 172.21.15.16:0/2435751422 learned_addr learned my addr 172.21.15.16:0/2435751422 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:33.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.003+0000 7fe1f1d9b700 1 -- 172.21.15.16:0/2435751422 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe1f41534b0 msgr2=0x7fe1f41406c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:33.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.003+0000 7fe1f1d9b700 1 --2- 172.21.15.16:0/2435751422 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe1f41534b0 0x7fe1f41406c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:33.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.003+0000 7fe1f1d9b700 1 -- 172.21.15.16:0/2435751422 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe1e0005200 msgr2=0x7fe1f40709f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:33.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.003+0000 7fe1f1d9b700 1 --2- 172.21.15.16:0/2435751422 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe1e0005200 0x7fe1f40709f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:33.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.003+0000 7fe1f1d9b700 1 -- 172.21.15.16:0/2435751422 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1f415d880 con 0x7fe1f4070f30 2023-05-20T01:17:33.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.003+0000 7fe1f1d9b700 1 --2- 172.21.15.16:0/2435751422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1f4070f30 0x7fe1f4152f70 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc011940 tx=0x7fe1dc011d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:33.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.004+0000 7fe1eaffd700 1 -- 172.21.15.16:0/2435751422 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1dc004240 con 0x7fe1f4070f30 2023-05-20T01:17:33.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.004+0000 7fe1eaffd700 1 -- 172.21.15.16:0/2435751422 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1dc0043a0 con 0x7fe1f4070f30 2023-05-20T01:17:33.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.004+0000 7fe1f8b14700 1 -- 172.21.15.16:0/2435751422 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1f415d360 con 0x7fe1f4070f30 2023-05-20T01:17:33.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.004+0000 7fe1eaffd700 1 -- 172.21.15.16:0/2435751422 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1dc00f950 con 0x7fe1f4070f30 2023-05-20T01:17:33.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.004+0000 7fe1f8b14700 1 -- 172.21.15.16:0/2435751422 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1f40b1b00 con 0x7fe1f4070f30 2023-05-20T01:17:33.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.005+0000 7fe1f8b14700 1 -- 172.21.15.16:0/2435751422 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1f40747b0 con 0x7fe1f4070f30 2023-05-20T01:17:33.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.005+0000 7fe1eaffd700 1 -- 172.21.15.16:0/2435751422 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe1dc00fab0 con 0x7fe1f4070f30 2023-05-20T01:17:33.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.005+0000 7fe1eaffd700 1 --2- 172.21.15.16:0/2435751422 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe1d406ad10 0x7fe1d406d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:33.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.006+0000 7fe1eaffd700 1 -- 172.21.15.16:0/2435751422 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fe1dc089c40 con 0x7fe1f4070f30 2023-05-20T01:17:33.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.006+0000 7fe1f259c700 1 --2- 172.21.15.16:0/2435751422 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe1d406ad10 0x7fe1d406d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:33.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.006+0000 7fe1f259c700 1 --2- 172.21.15.16:0/2435751422 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe1d406ad10 0x7fe1d406d1c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe1e40018f0 tx=0x7fe1e4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:33.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.010+0000 7fe1eaffd700 1 -- 172.21.15.16:0/2435751422 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fe1dc054a50 con 0x7fe1f4070f30 2023-05-20T01:17:33.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.208+0000 7fe1f8b14700 1 -- 172.21.15.16:0/2435751422 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe1f4143720 con 0x7fe1f4070f30 2023-05-20T01:17:33.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.209+0000 7fe1eaffd700 1 -- 172.21.15.16:0/2435751422 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+20121 (secure 0 0 0) 0x7fe1f4143720 con 0x7fe1f4070f30 2023-05-20T01:17:33.213 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:17:33.213 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":27,"fsid":"c2245cc5-ecab-4087-a59b-bc81db901676","created":"2023-05-20T01:16:09.549230+0000","modified":"2023-05-20T01:17:16.406712+0000","last_up_change":"2023-05-20T01:16:19.788624+0000","last_in_change":"2023-05-20T01:16:12.092160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-05-20T01:16:20.934366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-05-20T01:16:21.372294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-05-20T01:16:26.208287+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-05-20T01:16:27.292268+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-05-20T01:16:30.357285+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"317a83cd-19c8-4958-a301-6087ef342289","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6825","nonce":3923634179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6827","nonce":3923634179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6831","nonce":3923634179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6829","nonce":3923634179}]},"public_addr":"172.21.15.16:6825/3923634179","cluster_addr":"172.21.15.16:6827/3923634179","heartbeat_back_addr":"172.21.15.16:6831/3923634179","heartbeat_front_addr":"172.21.15.16:6829/3923634179","state":["exists","up"]},{"osd":1,"uuid":"5581046f-8219-4089-8d7d-b7846399505e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6809","nonce":3860264220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6811","nonce":3860264220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6815","nonce":3860264220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6813","nonce":3860264220}]},"public_addr":"172.21.15.16:6809/3860264220","cluster_addr":"172.21.15.16:6811/3860264220","heartbeat_back_addr":"172.21.15.16:6815/3860264220","heartbeat_front_addr":"172.21.15.16:6813/3860264220","state":["exists","up"]},{"osd":2,"uuid":"072e9e66-4133-478e-8d4a-64d46c2963f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6817","nonce":293187030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6819","nonce":293187030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6823","nonce":293187030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6821","nonce":293187030}]},"public_addr":"172.21.15.16:6817/293187030","cluster_addr":"172.21.15.16:6819/293187030","heartbeat_back_addr":"172.21.15.16:6823/293187030","heartbeat_front_addr":"172.21.15.16:6821/293187030","state":["exists","up"]},{"osd":3,"uuid":"11e1bba6-00f8-48dd-9028-d8d716b7f145","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6801","nonce":3669781890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6803","nonce":3669781890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6807","nonce":3669781890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6805","nonce":3669781890}]},"public_addr":"172.21.15.16:6801/3669781890","cluster_addr":"172.21.15.16:6803/3669781890","heartbeat_back_addr":"172.21.15.16:6807/3669781890","heartbeat_front_addr":"172.21.15.16:6805/3669781890","state":["exists","up"]},{"osd":4,"uuid":"b8909c3a-cf70-46cb-8d0b-009c707cf9d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6819","nonce":506317223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6823","nonce":506317223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6829","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6831","nonce":506317223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6827","nonce":506317223}]},"public_addr":"172.21.15.132:6819/506317223","cluster_addr":"172.21.15.132:6823/506317223","heartbeat_back_addr":"172.21.15.132:6831/506317223","heartbeat_front_addr":"172.21.15.132:6827/506317223","state":["exists","up"]},{"osd":5,"uuid":"b1818304-4583-47bf-a6ec-0bfbb5166233","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6818","nonce":1844689995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6822","nonce":1844689995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6830","nonce":1844689995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6826","nonce":1844689995}]},"public_addr":"172.21.15.132:6818/1844689995","cluster_addr":"172.21.15.132:6822/1844689995","heartbeat_back_addr":"172.21.15.132:6830/1844689995","heartbeat_front_addr":"172.21.15.132:6826/1844689995","state":["exists","up"]},{"osd":6,"uuid":"d37f4691-eed1-4a93-8818-18354f03f4a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6801","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6802","nonce":727581123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6805","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6807","nonce":727581123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6813","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6815","nonce":727581123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6811","nonce":727581123}]},"public_addr":"172.21.15.132:6802/727581123","cluster_addr":"172.21.15.132:6807/727581123","heartbeat_back_addr":"172.21.15.132:6815/727581123","heartbeat_front_addr":"172.21.15.132:6811/727581123","state":["exists","up"]},{"osd":7,"uuid":"4f712f62-5d1d-4816-8042-03209144f8b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6803","nonce":466001149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6806","nonce":466001149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6814","nonce":466001149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6810","nonce":466001149}]},"public_addr":"172.21.15.132:6803/466001149","cluster_addr":"172.21.15.132:6806/466001149","heartbeat_back_addr":"172.21.15.132:6814/466001149","heartbeat_front_addr":"172.21.15.132:6810/466001149","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.338863+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.278680+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305694+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.246835+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.276237+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.304441+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305421+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.267050+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.a","mappings":[{"from":1,"to":5}]},{"pgid":"4.11","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-20T01:17:33.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.211+0000 7fe1f8b14700 1 -- 172.21.15.16:0/2435751422 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe1d406ad10 msgr2=0x7fe1d406d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:33.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.211+0000 7fe1f8b14700 1 --2- 172.21.15.16:0/2435751422 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe1d406ad10 0x7fe1d406d1c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe1e40018f0 tx=0x7fe1e4009040 comp rx=0 tx=0).stop 2023-05-20T01:17:33.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.211+0000 7fe1f8b14700 1 -- 172.21.15.16:0/2435751422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1f4070f30 msgr2=0x7fe1f4152f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:33.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.211+0000 7fe1f8b14700 1 --2- 172.21.15.16:0/2435751422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1f4070f30 0x7fe1f4152f70 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc011940 tx=0x7fe1dc011d00 comp rx=0 tx=0).stop 2023-05-20T01:17:33.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.211+0000 7fe1f8b14700 1 -- 172.21.15.16:0/2435751422 shutdown_connections 2023-05-20T01:17:33.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.211+0000 7fe1f8b14700 1 --2- 172.21.15.16:0/2435751422 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe1d406ad10 0x7fe1d406d1c0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:33.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.211+0000 7fe1f8b14700 1 --2- 172.21.15.16:0/2435751422 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe1f41534b0 0x7fe1f41406c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:33.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.211+0000 7fe1f8b14700 1 --2- 172.21.15.16:0/2435751422 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe1e0005200 0x7fe1f40709f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:33.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.211+0000 7fe1f8b14700 1 --2- 172.21.15.16:0/2435751422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1f4070f30 0x7fe1f4152f70 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:33.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.211+0000 7fe1f8b14700 1 -- 172.21.15.16:0/2435751422 >> 172.21.15.16:0/2435751422 conn(0x7fe1f40661b0 msgr2=0x7fe1f415bde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:33.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.211+0000 7fe1f8b14700 1 -- 172.21.15.16:0/2435751422 shutdown_connections 2023-05-20T01:17:33.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.211+0000 7fe1f8b14700 1 -- 172.21.15.16:0/2435751422 wait complete. 2023-05-20T01:17:33.229 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-05-20T01:17:33.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.352+0000 7f54d291c700 1 Processor -- start 2023-05-20T01:17:33.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.352+0000 7f54d291c700 1 -- start start 2023-05-20T01:17:33.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.352+0000 7f54d291c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f54cc157190 0x7f54cc0cc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:33.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.353+0000 7f54d291c700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f54cc0cccf0 0x7f54cc0713c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:33.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.353+0000 7f54d291c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f54cc071900 0x7f54cc15d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:33.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.353+0000 7f54d291c700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54cc0cb590 con 0x7f54cc157190 2023-05-20T01:17:33.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.353+0000 7f54d291c700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54cc0ca9a0 con 0x7f54cc0cccf0 2023-05-20T01:17:33.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.353+0000 7f54d291c700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54cc0cac60 con 0x7f54cc071900 2023-05-20T01:17:33.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.353+0000 7f54cbfff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f54cc157190 0x7f54cc0cc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:33.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.353+0000 7f54cbfff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f54cc157190 0x7f54cc0cc7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59800/0 (socket says 172.21.15.16:59800) 2023-05-20T01:17:33.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.353+0000 7f54cbfff700 1 -- 172.21.15.16:0/3508173491 learned_addr learned my addr 172.21.15.16:0/3508173491 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:33.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.353+0000 7f54cb7fe700 1 --2- 172.21.15.16:0/3508173491 >> v2:172.21.15.132:3300/0 conn(0x7f54cc0cccf0 0x7f54cc0713c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:33.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.353+0000 7f54d0eb9700 1 --2- 172.21.15.16:0/3508173491 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f54cc071900 0x7f54cc15d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:33.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.353+0000 7f54cbfff700 1 -- 172.21.15.16:0/3508173491 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f54cc071900 msgr2=0x7f54cc15d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:33.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.353+0000 7f54cbfff700 1 --2- 172.21.15.16:0/3508173491 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f54cc071900 0x7f54cc15d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:33.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.353+0000 7f54cbfff700 1 -- 172.21.15.16:0/3508173491 >> v2:172.21.15.132:3300/0 conn(0x7f54cc0cccf0 msgr2=0x7f54cc0713c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:33.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.353+0000 7f54cbfff700 1 --2- 172.21.15.16:0/3508173491 >> v2:172.21.15.132:3300/0 conn(0x7f54cc0cccf0 0x7f54cc0713c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:33.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.353+0000 7f54cbfff700 1 -- 172.21.15.16:0/3508173491 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54cc0caf40 con 0x7f54cc157190 2023-05-20T01:17:33.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.353+0000 7f54cbfff700 1 --2- 172.21.15.16:0/3508173491 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54cc157190 0x7f54cc0cc7b0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f54cc0b6120 tx=0x7f54bc00be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd3f9e93428d270b server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:33.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.353+0000 7f54caffd700 1 -- 172.21.15.16:0/3508173491 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54bc00dd30 con 0x7f54cc157190 2023-05-20T01:17:33.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.353+0000 7f54caffd700 1 -- 172.21.15.16:0/3508173491 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54bc00de90 con 0x7f54cc157190 2023-05-20T01:17:33.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.353+0000 7f54caffd700 1 -- 172.21.15.16:0/3508173491 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54bc010690 con 0x7f54cc157190 2023-05-20T01:17:33.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.354+0000 7f54d291c700 1 -- 172.21.15.16:0/3508173491 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54cc157190 msgr2=0x7f54cc0cc7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:33.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.354+0000 7f54d291c700 1 --2- 172.21.15.16:0/3508173491 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54cc157190 0x7f54cc0cc7b0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f54cc0b6120 tx=0x7f54bc00be70 comp rx=0 tx=0).stop 2023-05-20T01:17:33.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.354+0000 7f54d291c700 1 -- 172.21.15.16:0/3508173491 shutdown_connections 2023-05-20T01:17:33.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.354+0000 7f54d291c700 1 --2- 172.21.15.16:0/3508173491 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f54cc071900 0x7f54cc15d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:33.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.354+0000 7f54d291c700 1 --2- 172.21.15.16:0/3508173491 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54cc157190 0x7f54cc0cc7b0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:33.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.354+0000 7f54d291c700 1 --2- 172.21.15.16:0/3508173491 >> v2:172.21.15.132:3300/0 conn(0x7f54cc0cccf0 0x7f54cc0713c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:33.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.354+0000 7f54d291c700 1 -- 172.21.15.16:0/3508173491 >> 172.21.15.16:0/3508173491 conn(0x7f54cc0cffb0 msgr2=0x7f54cc0d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:33.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.354+0000 7f54d291c700 1 -- 172.21.15.16:0/3508173491 shutdown_connections 2023-05-20T01:17:33.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.354+0000 7f54d291c700 1 -- 172.21.15.16:0/3508173491 wait complete. 2023-05-20T01:17:33.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.355+0000 7f54d291c700 1 Processor -- start 2023-05-20T01:17:33.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.355+0000 7f54d291c700 1 -- start start 2023-05-20T01:17:33.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.356+0000 7f54d291c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f54cc071900 0x7f54cc1555c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:33.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.356+0000 7f54d291c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54cc0cccf0 0x7f54cc0ae590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:33.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.356+0000 7f54d291c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f54cc157190 0x7f54cc1e4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:33.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.356+0000 7f54d291c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54cc155b00 con 0x7f54cc0cccf0 2023-05-20T01:17:33.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.356+0000 7f54d291c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54cc0b48f0 con 0x7f54cc157190 2023-05-20T01:17:33.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.356+0000 7f54d291c700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54cc0b3ce0 con 0x7f54cc071900 2023-05-20T01:17:33.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.356+0000 7f54cb7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54cc0cccf0 0x7f54cc0ae590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:33.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.356+0000 7f54cb7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54cc0cccf0 0x7f54cc0ae590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59808/0 (socket says 172.21.15.16:59808) 2023-05-20T01:17:33.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.356+0000 7f54cb7fe700 1 -- 172.21.15.16:0/868989976 learned_addr learned my addr 172.21.15.16:0/868989976 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:33.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.356+0000 7f54cbfff700 1 --2- 172.21.15.16:0/868989976 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f54cc071900 0x7f54cc1555c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:33.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.356+0000 7f54d0eb9700 1 --2- 172.21.15.16:0/868989976 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f54cc157190 0x7f54cc1e4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:33.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.357+0000 7f54cb7fe700 1 -- 172.21.15.16:0/868989976 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f54cc071900 msgr2=0x7f54cc1555c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:33.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.357+0000 7f54cb7fe700 1 --2- 172.21.15.16:0/868989976 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f54cc071900 0x7f54cc1555c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:33.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.357+0000 7f54cb7fe700 1 -- 172.21.15.16:0/868989976 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f54cc157190 msgr2=0x7f54cc1e4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:33.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.357+0000 7f54cb7fe700 1 --2- 172.21.15.16:0/868989976 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f54cc157190 0x7f54cc1e4000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:33.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.357+0000 7f54cb7fe700 1 -- 172.21.15.16:0/868989976 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54bc00b7f0 con 0x7f54cc0cccf0 2023-05-20T01:17:33.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.357+0000 7f54cb7fe700 1 --2- 172.21.15.16:0/868989976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54cc0cccf0 0x7f54cc0ae590 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f54c000e810 tx=0x7f54c000eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:33.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.357+0000 7f54c8ff9700 1 -- 172.21.15.16:0/868989976 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54c000c070 con 0x7f54cc0cccf0 2023-05-20T01:17:33.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.357+0000 7f54c8ff9700 1 -- 172.21.15.16:0/868989976 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54c0013610 con 0x7f54cc0cccf0 2023-05-20T01:17:33.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.357+0000 7f54c8ff9700 1 -- 172.21.15.16:0/868989976 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54c00139c0 con 0x7f54cc0cccf0 2023-05-20T01:17:33.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.357+0000 7f54d291c700 1 -- 172.21.15.16:0/868989976 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54cc157560 con 0x7f54cc0cccf0 2023-05-20T01:17:33.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.357+0000 7f54d291c700 1 -- 172.21.15.16:0/868989976 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54cc0cb220 con 0x7f54cc0cccf0 2023-05-20T01:17:33.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.358+0000 7f54d291c700 1 -- 172.21.15.16:0/868989976 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54cc0cd120 con 0x7f54cc0cccf0 2023-05-20T01:17:33.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.358+0000 7f54c8ff9700 1 -- 172.21.15.16:0/868989976 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f54c0013b20 con 0x7f54cc0cccf0 2023-05-20T01:17:33.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.359+0000 7f54c8ff9700 1 --2- 172.21.15.16:0/868989976 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f54b806ace0 0x7f54b806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:33.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.359+0000 7f54c8ff9700 1 -- 172.21.15.16:0/868989976 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f54c00894b0 con 0x7f54cc0cccf0 2023-05-20T01:17:33.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.359+0000 7f54cbfff700 1 --2- 172.21.15.16:0/868989976 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f54b806ace0 0x7f54b806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:33.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.359+0000 7f54cbfff700 1 --2- 172.21.15.16:0/868989976 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f54b806ace0 0x7f54b806d190 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f54bc00beb0 tx=0x7f54bc01c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:33.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.363+0000 7f54c8ff9700 1 -- 172.21.15.16:0/868989976 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f54c0057cd0 con 0x7f54cc0cccf0 2023-05-20T01:17:33.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:33.609+0000 7f54d291c700 1 -- 172.21.15.16:0/868989976 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f54cc0747b0 con 0x7f54cc0cccf0 2023-05-20T01:17:34.467 INFO:tasks.ceph.mon.a.smithi016.stderr:2023-05-20T01:17:34.463+0000 7f49eb8ae700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-05-20T01:17:34.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:34.467+0000 7f54c8ff9700 1 -- 172.21.15.16:0/868989976 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) v1 ==== 161+0+0 (secure 0 0 0) 0x7f54c0057860 con 0x7f54cc0cccf0 2023-05-20T01:17:34.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:34.470+0000 7f54d291c700 1 -- 172.21.15.16:0/868989976 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f54b806ace0 msgr2=0x7f54b806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:34.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:34.470+0000 7f54d291c700 1 --2- 172.21.15.16:0/868989976 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f54b806ace0 0x7f54b806d190 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f54bc00beb0 tx=0x7f54bc01c040 comp rx=0 tx=0).stop 2023-05-20T01:17:34.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:34.470+0000 7f54d291c700 1 -- 172.21.15.16:0/868989976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54cc0cccf0 msgr2=0x7f54cc0ae590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:34.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:34.470+0000 7f54d291c700 1 --2- 172.21.15.16:0/868989976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54cc0cccf0 0x7f54cc0ae590 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f54c000e810 tx=0x7f54c000eb20 comp rx=0 tx=0).stop 2023-05-20T01:17:34.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:34.470+0000 7f54d291c700 1 -- 172.21.15.16:0/868989976 shutdown_connections 2023-05-20T01:17:34.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:34.470+0000 7f54d291c700 1 --2- 172.21.15.16:0/868989976 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f54cc157190 0x7f54cc1e4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:34.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:34.470+0000 7f54d291c700 1 --2- 172.21.15.16:0/868989976 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f54b806ace0 0x7f54b806d190 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:34.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:34.470+0000 7f54d291c700 1 --2- 172.21.15.16:0/868989976 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f54cc071900 0x7f54cc1555c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:34.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:34.470+0000 7f54d291c700 1 --2- 172.21.15.16:0/868989976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54cc0cccf0 0x7f54cc0ae590 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:34.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:34.470+0000 7f54d291c700 1 -- 172.21.15.16:0/868989976 >> 172.21.15.16:0/868989976 conn(0x7f54cc0cffb0 msgr2=0x7f54cc0d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:34.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:34.470+0000 7f54d291c700 1 -- 172.21.15.16:0/868989976 shutdown_connections 2023-05-20T01:17:34.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:34.470+0000 7f54d291c700 1 -- 172.21.15.16:0/868989976 wait complete. 2023-05-20T01:17:34.476 INFO:teuthology.orchestra.run.smithi016.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-05-20T01:17:34.476 INFO:tasks.ceph.mds.a.smithi016.stderr: -5810> 2023-05-20T01:16:33.971+0000 7f93a1f3b700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-05-20T01:17:34.490 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-05-20T01:17:34.510 INFO:tasks.ceph.mds.a.smithi016.stdout:starting mds.a at 2023-05-20T01:17:34.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:34.623+0000 7fed92d95700 1 Processor -- start 2023-05-20T01:17:34.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:34.623+0000 7fed92d95700 1 -- start start 2023-05-20T01:17:34.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:34.624+0000 7fed92d95700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fed8c15f000 0x7fed8c0cbc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:34.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:34.624+0000 7fed92d95700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed8c06a5b0 con 0x7fed8c0b57b0 2023-05-20T01:17:34.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:34.624+0000 7fed92d95700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed8c06a850 con 0x7fed8c072340 2023-05-20T01:17:34.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:34.624+0000 7fed92d95700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed8c06ab30 con 0x7fed8c15f000 2023-05-20T01:17:34.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:34.624+0000 7fed90b31700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fed8c0b57b0 0x7fed8c071c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:43122/0 (socket says 172.21.15.16:43122) 2023-05-20T01:17:34.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:34.624+0000 7fed90b31700 1 -- 172.21.15.16:0/3116441058 learned_addr learned my addr 172.21.15.16:0/3116441058 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:17:34.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:34.624+0000 7fed8b7fe700 1 -- 172.21.15.16:0/3116441058 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2940047843 0 0) 0x7fed8c06a5b0 con 0x7fed8c0b57b0 2023-05-20T01:17:34.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:34.624+0000 7fed8b7fe700 1 -- 172.21.15.16:0/3116441058 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed7c0035b0 con 0x7fed8c0b57b0 2023-05-20T01:17:34.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:34.624+0000 7fed8bfff700 1 --2- 172.21.15.16:0/3116441058 >> v2:172.21.15.132:3300/0 conn(0x7fed8c15f000 0x7fed8c0cbc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:34.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:34.624+0000 7fed8b7fe700 1 -- 172.21.15.16:0/3116441058 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4198860107 0 0) 0x7fed7c0035b0 con 0x7fed8c0b57b0 2023-05-20T01:17:34.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:34.624+0000 7fed8b7fe700 1 -- 172.21.15.16:0/3116441058 >> v2:172.21.15.132:3300/0 conn(0x7fed8c15f000 msgr2=0x7fed8c0cbc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:34.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:34.624+0000 7fed8b7fe700 1 --2- 172.21.15.16:0/3116441058 >> v2:172.21.15.132:3300/0 conn(0x7fed8c15f000 0x7fed8c0cbc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:34.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:34.624+0000 7fed8b7fe700 1 -- 172.21.15.16:0/3116441058 >> v1:172.21.15.132:6789/0 conn(0x7fed8c072340 legacy=0x7fed8c0cc180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:34.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:34.624+0000 7fed8b7fe700 1 -- 172.21.15.16:0/3116441058 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed8c06ae10 con 0x7fed8c0b57b0 2023-05-20T01:17:34.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:34.624+0000 7fed8b7fe700 1 -- 172.21.15.16:0/3116441058 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7fed80001670 con 0x7fed8c0b57b0 2023-05-20T01:17:34.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:34.624+0000 7fed8b7fe700 1 -- 172.21.15.16:0/3116441058 >> v1:172.21.15.16:6789/0 conn(0x7fed8c0b57b0 legacy=0x7fed8c071c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:34.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:34.625+0000 7fed8b7fe700 1 --2- 172.21.15.16:0/3116441058 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fed7c003a00 0x7fed7c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:34.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:34.625+0000 7fed8b7fe700 1 --2- 172.21.15.16:0/3116441058 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed7c004850 0x7fed7c004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:34.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:34.625+0000 7fed8b7fe700 1 --2- 172.21.15.16:0/3116441058 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fed7c005200 0x7fed7c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:34.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:34.625+0000 7fed8b7fe700 1 -- 172.21.15.16:0/3116441058 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed8c06ab30 con 0x7fed7c004850 2023-05-20T01:17:34.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:34.625+0000 7fed8b7fe700 1 -- 172.21.15.16:0/3116441058 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed7c00c190 con 0x7fed7c003a00 2023-05-20T01:17:34.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:34.625+0000 7fed8b7fe700 1 -- 172.21.15.16:0/3116441058 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed7c00c350 con 0x7fed7c005200 2023-05-20T01:17:34.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:34.625+0000 7fed8bfff700 1 --2- 172.21.15.16:0/3116441058 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed7c004850 0x7fed7c004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:34.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:34.625+0000 7fed90b31700 1 --2- 172.21.15.16:0/3116441058 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fed7c003a00 0x7fed7c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:34.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:34.625+0000 7fed91332700 1 --2- 172.21.15.16:0/3116441058 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fed7c005200 0x7fed7c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:34.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:34.625+0000 7fed8bfff700 1 -- 172.21.15.16:0/3116441058 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fed7c005200 msgr2=0x7fed7c00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:34.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:34.625+0000 7fed8bfff700 1 --2- 172.21.15.16:0/3116441058 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fed7c005200 0x7fed7c00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:34.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:34.625+0000 7fed8bfff700 1 -- 172.21.15.16:0/3116441058 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fed7c003a00 msgr2=0x7fed7c003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:34.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:34.625+0000 7fed8bfff700 1 --2- 172.21.15.16:0/3116441058 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fed7c003a00 0x7fed7c003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:34.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:34.625+0000 7fed8bfff700 1 -- 172.21.15.16:0/3116441058 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed7c0035b0 con 0x7fed7c004850 2023-05-20T01:17:34.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:34.625+0000 7fed91332700 1 -- 172.21.15.16:0/3116441058 reap_dead start 2023-05-20T01:17:34.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:34.625+0000 7fed8bfff700 1 --2- 172.21.15.16:0/3116441058 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed7c004850 0x7fed7c004cc0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fed8c15f410 tx=0x7fed7c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89121206de75232b server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:34.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:34.625+0000 7fed8b7fe700 1 -- 172.21.15.16:0/3116441058 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed7800c070 con 0x7fed7c004850 2023-05-20T01:17:34.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:34.625+0000 7fed8b7fe700 1 -- 172.21.15.16:0/3116441058 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed78013610 con 0x7fed7c004850 2023-05-20T01:17:34.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:34.625+0000 7fed8b7fe700 1 -- 172.21.15.16:0/3116441058 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed780139c0 con 0x7fed7c004850 2023-05-20T01:17:34.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:34.626+0000 7fed92d95700 1 -- 172.21.15.16:0/3116441058 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed7c004850 msgr2=0x7fed7c004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:34.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:34.626+0000 7fed92d95700 1 --2- 172.21.15.16:0/3116441058 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed7c004850 0x7fed7c004cc0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fed8c15f410 tx=0x7fed7c005880 comp rx=0 tx=0).stop 2023-05-20T01:17:34.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:34.626+0000 7fed92d95700 1 -- 172.21.15.16:0/3116441058 shutdown_connections 2023-05-20T01:17:34.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:34.626+0000 7fed92d95700 1 --2- 172.21.15.16:0/3116441058 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed7c004850 0x7fed7c004cc0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:34.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:34.626+0000 7fed92d95700 1 -- 172.21.15.16:0/3116441058 >> 172.21.15.16:0/3116441058 conn(0x7fed8c06c260 msgr2=0x7fed8c0cffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:34.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:34.626+0000 7fed92d95700 1 -- 172.21.15.16:0/3116441058 shutdown_connections 2023-05-20T01:17:34.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:34.626+0000 7fed92d95700 1 -- 172.21.15.16:0/3116441058 wait complete. 2023-05-20T01:17:34.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:34.627+0000 7fed92d95700 1 Processor -- start 2023-05-20T01:17:34.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:34.628+0000 7fed92d95700 1 -- start start 2023-05-20T01:17:34.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:34.629+0000 7fed92d95700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed7c004850 0x7fed8c1effc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:34.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:34.629+0000 7fed92d95700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fed8c0c9b00 0x7fed8c0b2140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:34.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:34.629+0000 7fed92d95700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fed8c14dfb0 0x7fed8c14e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:34.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:34.629+0000 7fed92d95700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed8c1f0600 con 0x7fed7c004850 2023-05-20T01:17:34.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:34.629+0000 7fed92d95700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed8c1f08e0 con 0x7fed8c14dfb0 2023-05-20T01:17:34.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:34.629+0000 7fed92d95700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed8c1f0bc0 con 0x7fed8c0c9b00 2023-05-20T01:17:34.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:34.629+0000 7fed90b31700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed7c004850 0x7fed8c1effc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:34.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:34.629+0000 7fed90b31700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed7c004850 0x7fed8c1effc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59838/0 (socket says 172.21.15.16:59838) 2023-05-20T01:17:34.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:34.629+0000 7fed90b31700 1 -- 172.21.15.16:0/1379189334 learned_addr learned my addr 172.21.15.16:0/1379189334 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:34.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:34.629+0000 7fed91332700 1 --2- 172.21.15.16:0/1379189334 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fed8c14dfb0 0x7fed8c14e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:34.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:34.629+0000 7fed90b31700 1 -- 172.21.15.16:0/1379189334 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fed8c0c9b00 msgr2=0x7fed8c0b2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:34.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:34.629+0000 7fed8bfff700 1 --2- 172.21.15.16:0/1379189334 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fed8c0c9b00 0x7fed8c0b2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:34.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:34.629+0000 7fed90b31700 1 --2- 172.21.15.16:0/1379189334 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fed8c0c9b00 0x7fed8c0b2140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:34.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:34.629+0000 7fed90b31700 1 -- 172.21.15.16:0/1379189334 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fed8c14dfb0 msgr2=0x7fed8c14e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:34.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:34.629+0000 7fed90b31700 1 --2- 172.21.15.16:0/1379189334 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fed8c14dfb0 0x7fed8c14e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:34.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:34.629+0000 7fed90b31700 1 -- 172.21.15.16:0/1379189334 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed8c0af710 con 0x7fed7c004850 2023-05-20T01:17:34.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:34.629+0000 7fed91332700 1 --2- 172.21.15.16:0/1379189334 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fed8c14dfb0 0x7fed8c14e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:17:34.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:34.629+0000 7fed90b31700 1 --2- 172.21.15.16:0/1379189334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed7c004850 0x7fed8c1effc0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fed80007290 tx=0x7fed800075a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:34.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:34.629+0000 7fed897fa700 1 -- 172.21.15.16:0/1379189334 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed8000f5e0 con 0x7fed7c004850 2023-05-20T01:17:34.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:34.629+0000 7fed897fa700 1 -- 172.21.15.16:0/1379189334 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed8000f740 con 0x7fed7c004850 2023-05-20T01:17:34.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:34.629+0000 7fed92d95700 1 -- 172.21.15.16:0/1379189334 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed8c15f4e0 con 0x7fed7c004850 2023-05-20T01:17:34.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:34.629+0000 7fed897fa700 1 -- 172.21.15.16:0/1379189334 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed8000fab0 con 0x7fed7c004850 2023-05-20T01:17:34.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:34.630+0000 7fed92d95700 1 -- 172.21.15.16:0/1379189334 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed8c0af8a0 con 0x7fed7c004850 2023-05-20T01:17:34.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:34.630+0000 7fed92d95700 1 -- 172.21.15.16:0/1379189334 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed8c0747b0 con 0x7fed7c004850 2023-05-20T01:17:34.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:34.631+0000 7fed897fa700 1 -- 172.21.15.16:0/1379189334 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fed8000fc10 con 0x7fed7c004850 2023-05-20T01:17:34.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:34.631+0000 7fed897fa700 1 --2- 172.21.15.16:0/1379189334 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fed7006ace0 0x7fed7006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:34.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:34.631+0000 7fed897fa700 1 -- 172.21.15.16:0/1379189334 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6957+0+0 (secure 0 0 0) 0x7fed80089760 con 0x7fed7c004850 2023-05-20T01:17:34.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:34.631+0000 7fed8bfff700 1 --2- 172.21.15.16:0/1379189334 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fed7006ace0 0x7fed7006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:34.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:34.632+0000 7fed8bfff700 1 --2- 172.21.15.16:0/1379189334 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fed7006ace0 0x7fed7006d190 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fed8c0b2b60 tx=0x7fed7800c000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:34.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:34.636+0000 7fed897fa700 1 -- 172.21.15.16:0/1379189334 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fed80054490 con 0x7fed7c004850 2023-05-20T01:17:34.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:34.880+0000 7fed92d95700 1 -- 172.21.15.16:0/1379189334 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fed8c0d07b0 con 0x7fed7c004850 2023-05-20T01:17:35.142 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:17:35.145 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:17:35.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:35.471+0000 7fed897fa700 1 -- 172.21.15.16:0/1379189334 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) v1 ==== 104+0+0 (secure 0 0 0) 0x7fed80057ab0 con 0x7fed7c004850 2023-05-20T01:17:35.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:35.474+0000 7fed92d95700 1 -- 172.21.15.16:0/1379189334 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fed7006ace0 msgr2=0x7fed7006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:35.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:35.474+0000 7fed92d95700 1 --2- 172.21.15.16:0/1379189334 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fed7006ace0 0x7fed7006d190 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fed8c0b2b60 tx=0x7fed7800c000 comp rx=0 tx=0).stop 2023-05-20T01:17:35.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:35.474+0000 7fed92d95700 1 -- 172.21.15.16:0/1379189334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed7c004850 msgr2=0x7fed8c1effc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:35.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:35.474+0000 7fed92d95700 1 --2- 172.21.15.16:0/1379189334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed7c004850 0x7fed8c1effc0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fed80007290 tx=0x7fed800075a0 comp rx=0 tx=0).stop 2023-05-20T01:17:35.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:35.474+0000 7fed92d95700 1 -- 172.21.15.16:0/1379189334 shutdown_connections 2023-05-20T01:17:35.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:35.474+0000 7fed92d95700 1 --2- 172.21.15.16:0/1379189334 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fed8c14dfb0 0x7fed8c14e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:35.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:35.474+0000 7fed92d95700 1 --2- 172.21.15.16:0/1379189334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed7c004850 0x7fed8c1effc0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:35.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:35.474+0000 7fed92d95700 1 --2- 172.21.15.16:0/1379189334 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fed7006ace0 0x7fed7006d190 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:35.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:35.474+0000 7fed92d95700 1 --2- 172.21.15.16:0/1379189334 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fed8c0c9b00 0x7fed8c0b2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:35.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:35.474+0000 7fed92d95700 1 -- 172.21.15.16:0/1379189334 >> 172.21.15.16:0/1379189334 conn(0x7fed8c06c260 msgr2=0x7fed8c0cc8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:35.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:35.475+0000 7fed92d95700 1 -- 172.21.15.16:0/1379189334 shutdown_connections 2023-05-20T01:17:35.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:35.475+0000 7fed92d95700 1 -- 172.21.15.16:0/1379189334 wait complete. 2023-05-20T01:17:35.710 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-05-20T01:17:35.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:35.850+0000 7f1f630f1700 1 Processor -- start 2023-05-20T01:17:35.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:35.850+0000 7f1f630f1700 1 -- start start 2023-05-20T01:17:35.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:35.851+0000 7f1f630f1700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f1f5c0b1b30 0x7f1f5c0b1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:35.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:35.851+0000 7f1f630f1700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f5c15ccb0 con 0x7f1f5c0cfed0 2023-05-20T01:17:35.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:35.851+0000 7f1f630f1700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f5c15cf50 con 0x7f1f5c159060 2023-05-20T01:17:35.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:35.851+0000 7f1f630f1700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f5c15d230 con 0x7f1f5c0b1b30 2023-05-20T01:17:35.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:35.851+0000 7f1f5bfff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f1f5c0cfed0 0x7f1f5c0cc030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:43146/0 (socket says 172.21.15.16:43146) 2023-05-20T01:17:35.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:35.851+0000 7f1f5bfff700 1 -- 172.21.15.16:0/4268161554 learned_addr learned my addr 172.21.15.16:0/4268161554 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:17:35.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:35.851+0000 7f1f6168e700 1 --2- 172.21.15.16:0/4268161554 >> v2:172.21.15.132:3300/0 conn(0x7f1f5c0b1b30 0x7f1f5c0b1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:35.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:35.851+0000 7f1f5b7fe700 1 -- 172.21.15.16:0/4268161554 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3594363931 0 0) 0x7f1f5c15ccb0 con 0x7f1f5c0cfed0 2023-05-20T01:17:35.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:35.851+0000 7f1f5b7fe700 1 -- 172.21.15.16:0/4268161554 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f4c0046c0 con 0x7f1f5c0cfed0 2023-05-20T01:17:35.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:35.852+0000 7f1f5b7fe700 1 -- 172.21.15.16:0/4268161554 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1155772829 0 0) 0x7f1f5c15cf50 con 0x7f1f5c159060 2023-05-20T01:17:35.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:35.852+0000 7f1f5b7fe700 1 -- 172.21.15.16:0/4268161554 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f5c15ccb0 con 0x7f1f5c159060 2023-05-20T01:17:35.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:35.852+0000 7f1f5b7fe700 1 -- 172.21.15.16:0/4268161554 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1184683096 0 0) 0x7f1f4c0046c0 con 0x7f1f5c0cfed0 2023-05-20T01:17:35.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:35.852+0000 7f1f5b7fe700 1 -- 172.21.15.16:0/4268161554 >> v2:172.21.15.132:3300/0 conn(0x7f1f5c0b1b30 msgr2=0x7f1f5c0b1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:35.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:35.852+0000 7f1f5b7fe700 1 --2- 172.21.15.16:0/4268161554 >> v2:172.21.15.132:3300/0 conn(0x7f1f5c0b1b30 0x7f1f5c0b1fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:35.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:35.852+0000 7f1f5b7fe700 1 -- 172.21.15.16:0/4268161554 >> v1:172.21.15.132:6789/0 conn(0x7f1f5c159060 legacy=0x7f1f5c0cf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:35.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:35.852+0000 7f1f5b7fe700 1 -- 172.21.15.16:0/4268161554 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f5c15d510 con 0x7f1f5c0cfed0 2023-05-20T01:17:35.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:35.852+0000 7f1f5b7fe700 1 -- 172.21.15.16:0/4268161554 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7f1f48001670 con 0x7f1f5c0cfed0 2023-05-20T01:17:35.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:35.852+0000 7f1f5b7fe700 1 -- 172.21.15.16:0/4268161554 >> v1:172.21.15.16:6789/0 conn(0x7f1f5c0cfed0 legacy=0x7f1f5c0cc030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:35.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:35.852+0000 7f1f5b7fe700 1 --2- 172.21.15.16:0/4268161554 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1f4c004cf0 0x7f1f4c0050e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:35.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:35.852+0000 7f1f5b7fe700 1 --2- 172.21.15.16:0/4268161554 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1f4c005960 0x7f1f4c005dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:35.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:35.852+0000 7f1f5b7fe700 1 --2- 172.21.15.16:0/4268161554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f4c006310 0x7f1f4c00cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:35.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:35.852+0000 7f1f5b7fe700 1 -- 172.21.15.16:0/4268161554 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f5c15d230 con 0x7f1f4c006310 2023-05-20T01:17:35.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:35.852+0000 7f1f5b7fe700 1 -- 172.21.15.16:0/4268161554 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f4c00d1f0 con 0x7f1f4c005960 2023-05-20T01:17:35.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:35.852+0000 7f1f5b7fe700 1 -- 172.21.15.16:0/4268161554 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f4c00d380 con 0x7f1f4c004cf0 2023-05-20T01:17:35.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:35.852+0000 7f1f6168e700 1 --2- 172.21.15.16:0/4268161554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f4c006310 0x7f1f4c00cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:35.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:35.852+0000 7f1f5bfff700 1 --2- 172.21.15.16:0/4268161554 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1f4c005960 0x7f1f4c005dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:35.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:35.852+0000 7f1f60e8d700 1 --2- 172.21.15.16:0/4268161554 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1f4c004cf0 0x7f1f4c0050e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:35.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:35.852+0000 7f1f6168e700 1 -- 172.21.15.16:0/4268161554 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1f4c004cf0 msgr2=0x7f1f4c0050e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:35.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:35.852+0000 7f1f6168e700 1 --2- 172.21.15.16:0/4268161554 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1f4c004cf0 0x7f1f4c0050e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:35.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:35.852+0000 7f1f6168e700 1 -- 172.21.15.16:0/4268161554 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1f4c005960 msgr2=0x7f1f4c005dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:35.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:35.852+0000 7f1f6168e700 1 --2- 172.21.15.16:0/4268161554 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1f4c005960 0x7f1f4c005dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:35.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:35.852+0000 7f1f6168e700 1 -- 172.21.15.16:0/4268161554 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f4c0046c0 con 0x7f1f4c006310 2023-05-20T01:17:35.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:35.852+0000 7f1f60e8d700 1 --2- 172.21.15.16:0/4268161554 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1f4c004cf0 0x7f1f4c0050e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:17:35.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:35.852+0000 7f1f6168e700 1 -- 172.21.15.16:0/4268161554 reap_dead start 2023-05-20T01:17:35.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:35.852+0000 7f1f6168e700 1 --2- 172.21.15.16:0/4268161554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f4c006310 0x7f1f4c00cb20 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c0b1f30 tx=0x7f1f4c006990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d67991b2b4b6233b server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:35.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:35.853+0000 7f1f5b7fe700 1 -- 172.21.15.16:0/4268161554 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f54018070 con 0x7f1f4c006310 2023-05-20T01:17:35.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:35.853+0000 7f1f5b7fe700 1 -- 172.21.15.16:0/4268161554 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f5400f040 con 0x7f1f4c006310 2023-05-20T01:17:35.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:35.853+0000 7f1f5b7fe700 1 -- 172.21.15.16:0/4268161554 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5400ebe0 con 0x7f1f4c006310 2023-05-20T01:17:35.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:35.853+0000 7f1f630f1700 1 -- 172.21.15.16:0/4268161554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f4c006310 msgr2=0x7f1f4c00cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:35.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:35.853+0000 7f1f630f1700 1 --2- 172.21.15.16:0/4268161554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f4c006310 0x7f1f4c00cb20 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c0b1f30 tx=0x7f1f4c006990 comp rx=0 tx=0).stop 2023-05-20T01:17:35.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:35.853+0000 7f1f630f1700 1 -- 172.21.15.16:0/4268161554 shutdown_connections 2023-05-20T01:17:35.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:35.853+0000 7f1f630f1700 1 --2- 172.21.15.16:0/4268161554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f4c006310 0x7f1f4c00cb20 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:35.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:35.853+0000 7f1f630f1700 1 -- 172.21.15.16:0/4268161554 >> 172.21.15.16:0/4268161554 conn(0x7f1f5c06b150 msgr2=0x7f1f5c06cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:35.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:35.853+0000 7f1f630f1700 1 -- 172.21.15.16:0/4268161554 shutdown_connections 2023-05-20T01:17:35.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:35.853+0000 7f1f630f1700 1 -- 172.21.15.16:0/4268161554 wait complete. 2023-05-20T01:17:35.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:35.854+0000 7f1f630f1700 1 Processor -- start 2023-05-20T01:17:35.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:35.855+0000 7f1f630f1700 1 -- start start 2023-05-20T01:17:35.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:35.855+0000 7f1f630f1700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1f4c006310 0x7f1f5c159fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:35.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:35.855+0000 7f1f630f1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f5c0b47b0 0x7f1f5c146dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:35.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:35.855+0000 7f1f630f1700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1f5c155940 0x7f1f5c155db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:35.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:35.855+0000 7f1f630f1700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f5c1562f0 con 0x7f1f5c0b47b0 2023-05-20T01:17:35.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:35.855+0000 7f1f630f1700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f5c15b770 con 0x7f1f5c155940 2023-05-20T01:17:35.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:35.855+0000 7f1f630f1700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f5c147310 con 0x7f1f4c006310 2023-05-20T01:17:35.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:35.856+0000 7f1f5bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f5c0b47b0 0x7f1f5c146dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:35.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:35.856+0000 7f1f5bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f5c0b47b0 0x7f1f5c146dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59864/0 (socket says 172.21.15.16:59864) 2023-05-20T01:17:35.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:35.856+0000 7f1f5bfff700 1 -- 172.21.15.16:0/2278858615 learned_addr learned my addr 172.21.15.16:0/2278858615 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:35.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:35.856+0000 7f1f6168e700 1 --2- 172.21.15.16:0/2278858615 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1f5c155940 0x7f1f5c155db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:35.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:35.856+0000 7f1f60e8d700 1 --2- 172.21.15.16:0/2278858615 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1f4c006310 0x7f1f5c159fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:35.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:35.856+0000 7f1f5bfff700 1 -- 172.21.15.16:0/2278858615 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1f4c006310 msgr2=0x7f1f5c159fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:35.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:35.856+0000 7f1f5bfff700 1 --2- 172.21.15.16:0/2278858615 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1f4c006310 0x7f1f5c159fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:35.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:35.856+0000 7f1f5bfff700 1 -- 172.21.15.16:0/2278858615 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1f5c155940 msgr2=0x7f1f5c155db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:35.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:35.856+0000 7f1f5bfff700 1 --2- 172.21.15.16:0/2278858615 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1f5c155940 0x7f1f5c155db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:35.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:35.856+0000 7f1f5bfff700 1 -- 172.21.15.16:0/2278858615 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f5c159430 con 0x7f1f5c0b47b0 2023-05-20T01:17:35.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:35.856+0000 7f1f5bfff700 1 --2- 172.21.15.16:0/2278858615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f5c0b47b0 0x7f1f5c146dd0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f1f48004f50 tx=0x7f1f48007340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:35.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:35.856+0000 7f1f597fa700 1 -- 172.21.15.16:0/2278858615 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f48011580 con 0x7f1f5c0b47b0 2023-05-20T01:17:35.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:35.856+0000 7f1f597fa700 1 -- 172.21.15.16:0/2278858615 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f48011700 con 0x7f1f5c0b47b0 2023-05-20T01:17:35.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:35.856+0000 7f1f630f1700 1 -- 172.21.15.16:0/2278858615 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f5c070d70 con 0x7f1f5c0b47b0 2023-05-20T01:17:35.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:35.856+0000 7f1f597fa700 1 -- 172.21.15.16:0/2278858615 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f48011ab0 con 0x7f1f5c0b47b0 2023-05-20T01:17:35.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:35.856+0000 7f1f630f1700 1 -- 172.21.15.16:0/2278858615 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f5c15da30 con 0x7f1f5c0b47b0 2023-05-20T01:17:35.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:35.857+0000 7f1f630f1700 1 -- 172.21.15.16:0/2278858615 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f5c0747b0 con 0x7f1f5c0b47b0 2023-05-20T01:17:35.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:35.858+0000 7f1f597fa700 1 -- 172.21.15.16:0/2278858615 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1f48011c10 con 0x7f1f5c0b47b0 2023-05-20T01:17:35.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:35.859+0000 7f1f597fa700 1 --2- 172.21.15.16:0/2278858615 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f1f4006ad10 0x7f1f4006d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:35.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:35.859+0000 7f1f597fa700 1 -- 172.21.15.16:0/2278858615 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6957+0+0 (secure 0 0 0) 0x7f1f4808a300 con 0x7f1f5c0b47b0 2023-05-20T01:17:35.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:35.859+0000 7f1f60e8d700 1 --2- 172.21.15.16:0/2278858615 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f1f4006ad10 0x7f1f4006d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:35.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:35.859+0000 7f1f60e8d700 1 --2- 172.21.15.16:0/2278858615 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f1f4006ad10 0x7f1f4006d1c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f1f500011e0 tx=0x7f1f50009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:35.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:35.862+0000 7f1f597fa700 1 -- 172.21.15.16:0/2278858615 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f1f48055010 con 0x7f1f5c0b47b0 2023-05-20T01:17:36.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:36.059+0000 7f1f630f1700 1 -- 172.21.15.16:0/2278858615 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f1f5c0b1730 con 0x7f1f5c0b47b0 2023-05-20T01:17:36.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:36.530+0000 7f1f597fa700 1 -- 172.21.15.16:0/2278858615 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v29) v1 ==== 181+0+0 (secure 0 0 0) 0x7f1f48058630 con 0x7f1f5c0b47b0 2023-05-20T01:17:36.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:36.537+0000 7f1f630f1700 1 -- 172.21.15.16:0/2278858615 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f1f4006ad10 msgr2=0x7f1f4006d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:36.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:36.537+0000 7f1f630f1700 1 --2- 172.21.15.16:0/2278858615 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f1f4006ad10 0x7f1f4006d1c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f1f500011e0 tx=0x7f1f50009040 comp rx=0 tx=0).stop 2023-05-20T01:17:36.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:36.537+0000 7f1f630f1700 1 -- 172.21.15.16:0/2278858615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f5c0b47b0 msgr2=0x7f1f5c146dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:36.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:36.537+0000 7f1f630f1700 1 --2- 172.21.15.16:0/2278858615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f5c0b47b0 0x7f1f5c146dd0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f1f48004f50 tx=0x7f1f48007340 comp rx=0 tx=0).stop 2023-05-20T01:17:36.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:36.538+0000 7f1f630f1700 1 -- 172.21.15.16:0/2278858615 shutdown_connections 2023-05-20T01:17:36.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:36.538+0000 7f1f630f1700 1 --2- 172.21.15.16:0/2278858615 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1f5c155940 0x7f1f5c155db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:36.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:36.538+0000 7f1f630f1700 1 --2- 172.21.15.16:0/2278858615 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f1f4006ad10 0x7f1f4006d1c0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:36.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:36.538+0000 7f1f630f1700 1 --2- 172.21.15.16:0/2278858615 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1f4c006310 0x7f1f5c159fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:36.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:36.538+0000 7f1f630f1700 1 --2- 172.21.15.16:0/2278858615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f5c0b47b0 0x7f1f5c146dd0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:36.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:36.538+0000 7f1f630f1700 1 -- 172.21.15.16:0/2278858615 >> 172.21.15.16:0/2278858615 conn(0x7f1f5c06b150 msgr2=0x7f1f5c0caea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:36.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:36.539+0000 7f1f630f1700 1 -- 172.21.15.16:0/2278858615 shutdown_connections 2023-05-20T01:17:36.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:36.539+0000 7f1f630f1700 1 -- 172.21.15.16:0/2278858615 wait complete. 2023-05-20T01:17:36.544 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' removed 2023-05-20T01:17:36.557 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-05-20T01:17:36.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:36.685+0000 7f9958754700 1 Processor -- start 2023-05-20T01:17:36.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:36.685+0000 7f9958754700 1 -- start start 2023-05-20T01:17:36.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:36.686+0000 7f9958754700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f995006aad0 0x7f995006dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:36.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:36.686+0000 7f9958754700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f995006aea0 0x7f995006e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:36.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:36.686+0000 7f9958754700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9950161440 con 0x7f99500b1b90 2023-05-20T01:17:36.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:36.686+0000 7f9958754700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99500b28c0 con 0x7f995006aea0 2023-05-20T01:17:36.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:36.686+0000 7f9958754700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9950160680 con 0x7f995006aad0 2023-05-20T01:17:36.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:36.686+0000 7f9955cef700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f995006aea0 0x7f995006e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:36.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:36.686+0000 7f9955cef700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f995006aea0 0x7f995006e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59870/0 (socket says 172.21.15.16:59870) 2023-05-20T01:17:36.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:36.686+0000 7f9955cef700 1 -- 172.21.15.16:0/2915068902 learned_addr learned my addr 172.21.15.16:0/2915068902 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:36.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:36.686+0000 7f99564f0700 1 --2- 172.21.15.16:0/2915068902 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f995006aad0 0x7f995006dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:36.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:36.686+0000 7f9955cef700 1 -- 172.21.15.16:0/2915068902 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f995006aad0 msgr2=0x7f995006dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:36.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:36.686+0000 7f9955cef700 1 --2- 172.21.15.16:0/2915068902 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f995006aad0 0x7f995006dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:36.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:36.686+0000 7f9955cef700 1 -- 172.21.15.16:0/2915068902 >> v1:172.21.15.132:6789/0 conn(0x7f99500b1b90 legacy=0x7f99500b2010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:36.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:36.686+0000 7f9955cef700 1 -- 172.21.15.16:0/2915068902 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9950160960 con 0x7f995006aea0 2023-05-20T01:17:36.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:36.686+0000 7f9955cef700 1 --2- 172.21.15.16:0/2915068902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f995006aea0 0x7f995006e1e0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f99400098f0 tx=0x7f9940009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eab2071bf11d24f8 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:36.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:36.687+0000 7f99554ee700 1 -- 172.21.15.16:0/2915068902 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9940011830 con 0x7f995006aea0 2023-05-20T01:17:36.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:36.687+0000 7f99554ee700 1 -- 172.21.15.16:0/2915068902 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9940011990 con 0x7f995006aea0 2023-05-20T01:17:36.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:36.687+0000 7f99554ee700 1 -- 172.21.15.16:0/2915068902 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9940011d20 con 0x7f995006aea0 2023-05-20T01:17:36.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:36.687+0000 7f9958754700 1 -- 172.21.15.16:0/2915068902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f995006aea0 msgr2=0x7f995006e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:36.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:36.687+0000 7f9958754700 1 --2- 172.21.15.16:0/2915068902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f995006aea0 0x7f995006e1e0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f99400098f0 tx=0x7f9940009c00 comp rx=0 tx=0).stop 2023-05-20T01:17:36.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:36.687+0000 7f9958754700 1 -- 172.21.15.16:0/2915068902 shutdown_connections 2023-05-20T01:17:36.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:36.687+0000 7f9958754700 1 --2- 172.21.15.16:0/2915068902 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f995006aad0 0x7f995006dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:36.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:36.687+0000 7f9958754700 1 --2- 172.21.15.16:0/2915068902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f995006aea0 0x7f995006e1e0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:36.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:36.687+0000 7f9958754700 1 -- 172.21.15.16:0/2915068902 >> 172.21.15.16:0/2915068902 conn(0x7f99500ccab0 msgr2=0x7f99500cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:36.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:36.687+0000 7f9958754700 1 -- 172.21.15.16:0/2915068902 shutdown_connections 2023-05-20T01:17:36.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:36.687+0000 7f9958754700 1 -- 172.21.15.16:0/2915068902 wait complete. 2023-05-20T01:17:36.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:36.688+0000 7f9958754700 1 Processor -- start 2023-05-20T01:17:36.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:36.689+0000 7f9958754700 1 -- start start 2023-05-20T01:17:36.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:36.689+0000 7f9958754700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f995006aad0 0x7f995014f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:36.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:36.689+0000 7f9958754700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f995006aea0 0x7f995015e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:36.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:36.689+0000 7f9958754700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f99500b1b90 0x7f99500cb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:36.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:36.689+0000 7f9958754700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f995015f770 con 0x7f995006aad0 2023-05-20T01:17:36.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:36.689+0000 7f9958754700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f995015eaf0 con 0x7f995006aea0 2023-05-20T01:17:36.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:36.689+0000 7f9958754700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f995015edd0 con 0x7f99500b1b90 2023-05-20T01:17:36.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:36.689+0000 7f99564f0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f995006aad0 0x7f995014f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:36.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:36.689+0000 7f99564f0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f995006aad0 0x7f995014f410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59874/0 (socket says 172.21.15.16:59874) 2023-05-20T01:17:36.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:36.689+0000 7f99564f0700 1 -- 172.21.15.16:0/2308005566 learned_addr learned my addr 172.21.15.16:0/2308005566 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:36.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:36.689+0000 7f9955cef700 1 --2- 172.21.15.16:0/2308005566 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f995006aea0 0x7f995015e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:36.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:36.689+0000 7f9956cf1700 1 --2- 172.21.15.16:0/2308005566 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f99500b1b90 0x7f99500cb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:36.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:36.690+0000 7f99564f0700 1 -- 172.21.15.16:0/2308005566 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f99500b1b90 msgr2=0x7f99500cb5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:36.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:36.690+0000 7f99564f0700 1 --2- 172.21.15.16:0/2308005566 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f99500b1b90 0x7f99500cb5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:36.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:36.690+0000 7f99564f0700 1 -- 172.21.15.16:0/2308005566 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f995006aea0 msgr2=0x7f995015e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:36.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:36.690+0000 7f99564f0700 1 --2- 172.21.15.16:0/2308005566 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f995006aea0 0x7f995015e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:36.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:36.690+0000 7f99564f0700 1 -- 172.21.15.16:0/2308005566 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9940009550 con 0x7f995006aad0 2023-05-20T01:17:36.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:36.690+0000 7f99564f0700 1 --2- 172.21.15.16:0/2308005566 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f995006aad0 0x7f995014f410 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f994c00e9b0 tx=0x7f994c00ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:36.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:36.690+0000 7f9946ffd700 1 -- 172.21.15.16:0/2308005566 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f994c009710 con 0x7f995006aad0 2023-05-20T01:17:36.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:36.690+0000 7f9946ffd700 1 -- 172.21.15.16:0/2308005566 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f994c00ee50 con 0x7f995006aad0 2023-05-20T01:17:36.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:36.690+0000 7f9946ffd700 1 -- 172.21.15.16:0/2308005566 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f994c00ca50 con 0x7f995006aad0 2023-05-20T01:17:36.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:36.690+0000 7f9958754700 1 -- 172.21.15.16:0/2308005566 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9950160f20 con 0x7f995006aad0 2023-05-20T01:17:36.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:36.690+0000 7f9958754700 1 -- 172.21.15.16:0/2308005566 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9950160c40 con 0x7f995006aad0 2023-05-20T01:17:36.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:36.691+0000 7f9958754700 1 -- 172.21.15.16:0/2308005566 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f995006cb80 con 0x7f995006aad0 2023-05-20T01:17:36.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:36.692+0000 7f9946ffd700 1 -- 172.21.15.16:0/2308005566 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f994c00cbb0 con 0x7f995006aad0 2023-05-20T01:17:36.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:36.692+0000 7f9946ffd700 1 --2- 172.21.15.16:0/2308005566 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f993406ad10 0x7f993406d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:36.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:36.692+0000 7f9946ffd700 1 -- 172.21.15.16:0/2308005566 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6519+0+0 (secure 0 0 0) 0x7f994c089670 con 0x7f995006aad0 2023-05-20T01:17:36.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:36.692+0000 7f9955cef700 1 --2- 172.21.15.16:0/2308005566 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f993406ad10 0x7f993406d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:36.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:36.693+0000 7f9955cef700 1 --2- 172.21.15.16:0/2308005566 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f993406ad10 0x7f993406d1c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f99400098f0 tx=0x7f9940008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:36.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:36.697+0000 7f9946ffd700 1 -- 172.21.15.16:0/2308005566 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f994c057f50 con 0x7f995006aad0 2023-05-20T01:17:36.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:36.893+0000 7f9958754700 1 -- 172.21.15.16:0/2308005566 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f99500747b0 con 0x7f995006aad0 2023-05-20T01:17:37.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:37.535+0000 7f9946ffd700 1 -- 172.21.15.16:0/2308005566 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v30) v1 ==== 169+0+0 (secure 0 0 0) 0x7f994c00ce60 con 0x7f995006aad0 2023-05-20T01:17:37.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:37.540+0000 7f9958754700 1 -- 172.21.15.16:0/2308005566 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f993406ad10 msgr2=0x7f993406d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:37.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:37.540+0000 7f9958754700 1 --2- 172.21.15.16:0/2308005566 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f993406ad10 0x7f993406d1c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f99400098f0 tx=0x7f9940008030 comp rx=0 tx=0).stop 2023-05-20T01:17:37.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:37.540+0000 7f9958754700 1 -- 172.21.15.16:0/2308005566 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f995006aad0 msgr2=0x7f995014f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:37.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:37.540+0000 7f9958754700 1 --2- 172.21.15.16:0/2308005566 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f995006aad0 0x7f995014f410 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f994c00e9b0 tx=0x7f994c00ecc0 comp rx=0 tx=0).stop 2023-05-20T01:17:37.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:37.541+0000 7f9958754700 1 -- 172.21.15.16:0/2308005566 shutdown_connections 2023-05-20T01:17:37.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:37.541+0000 7f9958754700 1 --2- 172.21.15.16:0/2308005566 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f993406ad10 0x7f993406d1c0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:37.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:37.541+0000 7f9958754700 1 --2- 172.21.15.16:0/2308005566 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f99500b1b90 0x7f99500cb5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:37.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:37.541+0000 7f9958754700 1 --2- 172.21.15.16:0/2308005566 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f995006aad0 0x7f995014f410 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:37.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:37.541+0000 7f9958754700 1 --2- 172.21.15.16:0/2308005566 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f995006aea0 0x7f995015e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:37.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:37.541+0000 7f9958754700 1 -- 172.21.15.16:0/2308005566 >> 172.21.15.16:0/2308005566 conn(0x7f99500ccab0 msgr2=0x7f995006dca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:37.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:37.541+0000 7f9958754700 1 -- 172.21.15.16:0/2308005566 shutdown_connections 2023-05-20T01:17:37.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:37.541+0000 7f9958754700 1 -- 172.21.15.16:0/2308005566 wait complete. 2023-05-20T01:17:37.546 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' removed 2023-05-20T01:17:37.561 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2023-05-20T01:17:37.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:37.695+0000 7f4b53998700 1 Processor -- start 2023-05-20T01:17:37.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:37.696+0000 7f4b53998700 1 -- start start 2023-05-20T01:17:37.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:37.696+0000 7f4b53998700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4b4c159790 0x7f4b4c0caf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:37.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:37.696+0000 7f4b53998700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f4b4c0719b0 0x7f4b4c0c4f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:37.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:37.696+0000 7f4b53998700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b4c0cc320 con 0x7f4b4c0cdf90 2023-05-20T01:17:37.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:37.696+0000 7f4b53998700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b4c0cf410 con 0x7f4b4c159790 2023-05-20T01:17:37.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:37.696+0000 7f4b53998700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b4c0cb450 con 0x7f4b4c0719b0 2023-05-20T01:17:37.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:37.696+0000 7f4b51734700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4b4c159790 0x7f4b4c0caf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:37.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:37.696+0000 7f4b50f33700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f4b4c0cdf90 0x7f4b4c0712a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:43178/0 (socket says 172.21.15.16:43178) 2023-05-20T01:17:37.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:37.697+0000 7f4b50f33700 1 -- 172.21.15.16:0/3569336459 learned_addr learned my addr 172.21.15.16:0/3569336459 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:17:37.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:37.697+0000 7f4b51f35700 1 --2- 172.21.15.16:0/3569336459 >> v2:172.21.15.132:3300/0 conn(0x7f4b4c0719b0 0x7f4b4c0c4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:37.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:37.697+0000 7f4b51734700 1 -- 172.21.15.16:0/3569336459 >> v2:172.21.15.132:3300/0 conn(0x7f4b4c0719b0 msgr2=0x7f4b4c0c4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:37.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:37.697+0000 7f4b51734700 1 --2- 172.21.15.16:0/3569336459 >> v2:172.21.15.132:3300/0 conn(0x7f4b4c0719b0 0x7f4b4c0c4f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:37.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:37.697+0000 7f4b51734700 1 -- 172.21.15.16:0/3569336459 >> v1:172.21.15.16:6789/0 conn(0x7f4b4c0cdf90 legacy=0x7f4b4c0712a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:37.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:37.697+0000 7f4b43fff700 1 -- 172.21.15.16:0/3569336459 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2630565157 0 0) 0x7f4b4c0cc320 con 0x7f4b4c0cdf90 2023-05-20T01:17:37.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:37.697+0000 7f4b51734700 1 -- 172.21.15.16:0/3569336459 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b4c159b60 con 0x7f4b4c159790 2023-05-20T01:17:37.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:37.697+0000 7f4b51f35700 1 --2- 172.21.15.16:0/3569336459 >> v2:172.21.15.132:3300/0 conn(0x7f4b4c0719b0 0x7f4b4c0c4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:17:37.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:37.697+0000 7f4b51734700 1 --2- 172.21.15.16:0/3569336459 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b4c159790 0x7f4b4c0caf10 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c071f60 tx=0x7f4b3c00bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34ae79708294655b server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:37.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:37.697+0000 7f4b43fff700 1 -- 172.21.15.16:0/3569336459 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b3c00dc70 con 0x7f4b4c159790 2023-05-20T01:17:37.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:37.697+0000 7f4b43fff700 1 -- 172.21.15.16:0/3569336459 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b3c00ddd0 con 0x7f4b4c159790 2023-05-20T01:17:37.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:37.697+0000 7f4b43fff700 1 -- 172.21.15.16:0/3569336459 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b3c010690 con 0x7f4b4c159790 2023-05-20T01:17:37.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:37.698+0000 7f4b53998700 1 -- 172.21.15.16:0/3569336459 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b4c159790 msgr2=0x7f4b4c0caf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:37.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:37.698+0000 7f4b53998700 1 --2- 172.21.15.16:0/3569336459 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b4c159790 0x7f4b4c0caf10 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c071f60 tx=0x7f4b3c00bdc0 comp rx=0 tx=0).stop 2023-05-20T01:17:37.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:37.698+0000 7f4b53998700 1 -- 172.21.15.16:0/3569336459 shutdown_connections 2023-05-20T01:17:37.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:37.698+0000 7f4b53998700 1 --2- 172.21.15.16:0/3569336459 >> v2:172.21.15.132:3300/0 conn(0x7f4b4c0719b0 0x7f4b4c0c4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:37.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:37.698+0000 7f4b53998700 1 --2- 172.21.15.16:0/3569336459 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b4c159790 0x7f4b4c0caf10 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:37.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:37.698+0000 7f4b53998700 1 -- 172.21.15.16:0/3569336459 >> 172.21.15.16:0/3569336459 conn(0x7f4b4c0b2920 msgr2=0x7f4b4c0d0020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:37.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:37.698+0000 7f4b53998700 1 -- 172.21.15.16:0/3569336459 shutdown_connections 2023-05-20T01:17:37.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:37.698+0000 7f4b53998700 1 -- 172.21.15.16:0/3569336459 wait complete. 2023-05-20T01:17:37.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:37.699+0000 7f4b53998700 1 Processor -- start 2023-05-20T01:17:37.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:37.700+0000 7f4b53998700 1 -- start start 2023-05-20T01:17:37.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:37.700+0000 7f4b53998700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b4c0719b0 0x7f4b4c159d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:37.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:37.700+0000 7f4b53998700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b4c0cdf90 0x7f4b4c15b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:37.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:37.700+0000 7f4b53998700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b4c159790 0x7f4b4c0b58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:37.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:37.701+0000 7f4b53998700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b4c0b5e20 con 0x7f4b4c0719b0 2023-05-20T01:17:37.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:37.701+0000 7f4b51734700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b4c0719b0 0x7f4b4c159d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:37.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:37.701+0000 7f4b51734700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b4c0719b0 0x7f4b4c159d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59888/0 (socket says 172.21.15.16:59888) 2023-05-20T01:17:37.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:37.701+0000 7f4b51734700 1 -- 172.21.15.16:0/1127204422 learned_addr learned my addr 172.21.15.16:0/1127204422 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:37.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:37.701+0000 7f4b53998700 1 -- 172.21.15.16:0/1127204422 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b4c15a3a0 con 0x7f4b4c0cdf90 2023-05-20T01:17:37.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:37.701+0000 7f4b53998700 1 -- 172.21.15.16:0/1127204422 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b4c15a680 con 0x7f4b4c159790 2023-05-20T01:17:37.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:37.701+0000 7f4b50f33700 1 --2- 172.21.15.16:0/1127204422 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b4c0cdf90 0x7f4b4c15b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:37.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:37.701+0000 7f4b51f35700 1 --2- 172.21.15.16:0/1127204422 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b4c159790 0x7f4b4c0b58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:37.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:37.701+0000 7f4b51734700 1 -- 172.21.15.16:0/1127204422 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b4c159790 msgr2=0x7f4b4c0b58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:37.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:37.701+0000 7f4b51734700 1 --2- 172.21.15.16:0/1127204422 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b4c159790 0x7f4b4c0b58e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:37.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:37.701+0000 7f4b51734700 1 -- 172.21.15.16:0/1127204422 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b4c0cdf90 msgr2=0x7f4b4c15b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:37.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:37.701+0000 7f4b51734700 1 --2- 172.21.15.16:0/1127204422 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b4c0cdf90 0x7f4b4c15b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:37.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:37.701+0000 7f4b51734700 1 -- 172.21.15.16:0/1127204422 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b3c00b740 con 0x7f4b4c0719b0 2023-05-20T01:17:37.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:37.701+0000 7f4b51734700 1 --2- 172.21.15.16:0/1127204422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b4c0719b0 0x7f4b4c159d50 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c00b2c0 tx=0x7f4b3c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:37.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:37.701+0000 7f4b41ffb700 1 -- 172.21.15.16:0/1127204422 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b3c01d070 con 0x7f4b4c0719b0 2023-05-20T01:17:37.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:37.701+0000 7f4b41ffb700 1 -- 172.21.15.16:0/1127204422 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b3c01e040 con 0x7f4b4c0719b0 2023-05-20T01:17:37.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:37.701+0000 7f4b53998700 1 -- 172.21.15.16:0/1127204422 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b4c0cb9f0 con 0x7f4b4c0719b0 2023-05-20T01:17:37.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:37.701+0000 7f4b41ffb700 1 -- 172.21.15.16:0/1127204422 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b3c010c60 con 0x7f4b4c0719b0 2023-05-20T01:17:37.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:37.702+0000 7f4b53998700 1 -- 172.21.15.16:0/1127204422 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b4c0cb710 con 0x7f4b4c0719b0 2023-05-20T01:17:37.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:37.702+0000 7f4b53998700 1 -- 172.21.15.16:0/1127204422 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b4c0cbcd0 con 0x7f4b4c0719b0 2023-05-20T01:17:37.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:37.704+0000 7f4b41ffb700 1 -- 172.21.15.16:0/1127204422 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4b3c0036a0 con 0x7f4b4c0719b0 2023-05-20T01:17:37.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:37.704+0000 7f4b41ffb700 1 --2- 172.21.15.16:0/1127204422 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f4b3006ace0 0x7f4b3006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:37.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:37.704+0000 7f4b41ffb700 1 -- 172.21.15.16:0/1127204422 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6047+0+0 (secure 0 0 0) 0x7f4b3c010e40 con 0x7f4b4c0719b0 2023-05-20T01:17:37.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:37.704+0000 7f4b50f33700 1 --2- 172.21.15.16:0/1127204422 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f4b3006ace0 0x7f4b3006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:37.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:37.704+0000 7f4b50f33700 1 --2- 172.21.15.16:0/1127204422 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f4b3006ace0 0x7f4b3006d190 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f4b48007fd0 tx=0x7f4b48007de0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:37.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:37.707+0000 7f4b41ffb700 1 -- 172.21.15.16:0/1127204422 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f4b3c059240 con 0x7f4b4c0719b0 2023-05-20T01:17:37.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:37.919+0000 7f4b53998700 1 -- 172.21.15.16:0/1127204422 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f4b4c0747b0 con 0x7f4b4c0719b0 2023-05-20T01:17:38.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:38.540+0000 7f4b41ffb700 1 -- 172.21.15.16:0/1127204422 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v31) v1 ==== 178+0+0 (secure 0 0 0) 0x7f4b3c05c860 con 0x7f4b4c0719b0 2023-05-20T01:17:38.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:38.546+0000 7f4b53998700 1 -- 172.21.15.16:0/1127204422 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f4b3006ace0 msgr2=0x7f4b3006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:38.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:38.546+0000 7f4b53998700 1 --2- 172.21.15.16:0/1127204422 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f4b3006ace0 0x7f4b3006d190 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f4b48007fd0 tx=0x7f4b48007de0 comp rx=0 tx=0).stop 2023-05-20T01:17:38.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:38.546+0000 7f4b53998700 1 -- 172.21.15.16:0/1127204422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b4c0719b0 msgr2=0x7f4b4c159d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:38.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:38.546+0000 7f4b53998700 1 --2- 172.21.15.16:0/1127204422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b4c0719b0 0x7f4b4c159d50 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c00b2c0 tx=0x7f4b3c004750 comp rx=0 tx=0).stop 2023-05-20T01:17:38.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:38.546+0000 7f4b53998700 1 -- 172.21.15.16:0/1127204422 shutdown_connections 2023-05-20T01:17:38.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:38.546+0000 7f4b53998700 1 --2- 172.21.15.16:0/1127204422 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f4b3006ace0 0x7f4b3006d190 secure :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f4b48007fd0 tx=0x7f4b48007de0 comp rx=0 tx=0).stop 2023-05-20T01:17:38.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:38.546+0000 7f4b53998700 1 --2- 172.21.15.16:0/1127204422 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b4c159790 0x7f4b4c0b58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:38.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:38.546+0000 7f4b53998700 1 --2- 172.21.15.16:0/1127204422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b4c0719b0 0x7f4b4c159d50 secure :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c00b2c0 tx=0x7f4b3c004750 comp rx=0 tx=0).stop 2023-05-20T01:17:38.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:38.546+0000 7f4b53998700 1 --2- 172.21.15.16:0/1127204422 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b4c0cdf90 0x7f4b4c15b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:38.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:38.546+0000 7f4b53998700 1 -- 172.21.15.16:0/1127204422 >> 172.21.15.16:0/1127204422 conn(0x7f4b4c0b2920 msgr2=0x7f4b4c0afc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:38.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:38.547+0000 7f4b53998700 1 -- 172.21.15.16:0/1127204422 shutdown_connections 2023-05-20T01:17:38.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:38.548+0000 7f4b53998700 1 -- 172.21.15.16:0/1127204422 wait complete. 2023-05-20T01:17:38.554 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data_ec' removed 2023-05-20T01:17:38.569 INFO:tasks.ceph.mds.a:Restarting daemon 2023-05-20T01:17:38.569 INFO:tasks.ceph.mds.a:Stopping old one... 2023-05-20T01:17:38.570 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-05-20T01:17:38.570 INFO:teuthology.orchestra.run:waiting for 300 2023-05-20T01:17:38.570 INFO:tasks.ceph.mds.c:Restarting daemon 2023-05-20T01:17:38.571 INFO:tasks.ceph.mds.c:Stopping old one... 2023-05-20T01:17:38.571 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-05-20T01:17:38.571 INFO:teuthology.orchestra.run:waiting for 300 2023-05-20T01:17:38.572 INFO:tasks.ceph.mds.b:Restarting daemon 2023-05-20T01:17:38.572 INFO:tasks.ceph.mds.b:Stopping old one... 2023-05-20T01:17:38.573 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-05-20T01:17:38.573 INFO:teuthology.orchestra.run:waiting for 300 2023-05-20T01:17:38.573 INFO:tasks.ceph.mds.d:Restarting daemon 2023-05-20T01:17:38.574 INFO:tasks.ceph.mds.d:Stopping old one... 2023-05-20T01:17:38.574 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-05-20T01:17:38.575 INFO:teuthology.orchestra.run:waiting for 300 2023-05-20T01:17:38.580 INFO:tasks.ceph.mds.a:Stopped 2023-05-20T01:17:38.580 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-05-20T01:17:38.581 INFO:tasks.ceph.mds.c:Stopped 2023-05-20T01:17:38.582 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-05-20T01:17:38.584 INFO:tasks.ceph.mds.b:Stopped 2023-05-20T01:17:38.584 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-05-20T01:17:38.585 INFO:tasks.ceph.mds.d:Stopped 2023-05-20T01:17:38.585 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-05-20T01:17:38.622 INFO:tasks.ceph.mds.a:Started 2023-05-20T01:17:38.623 DEBUG:teuthology.parallel:result is None 2023-05-20T01:17:38.624 INFO:tasks.ceph.mds.c:Started 2023-05-20T01:17:38.624 DEBUG:teuthology.parallel:result is None 2023-05-20T01:17:38.627 INFO:tasks.ceph.mds.b:Started 2023-05-20T01:17:38.627 DEBUG:teuthology.parallel:result is None 2023-05-20T01:17:38.628 INFO:tasks.ceph.mds.d:Started 2023-05-20T01:17:38.628 DEBUG:teuthology.parallel:result is None 2023-05-20T01:17:38.629 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-05-20T01:17:38.710 INFO:tasks.ceph.mds.a.smithi016.stdout:starting mds.a at 2023-05-20T01:17:38.711 INFO:tasks.ceph.mds.c.smithi016.stdout:starting mds.c at 2023-05-20T01:17:38.715 INFO:tasks.ceph.mds.b.smithi132.stdout:starting mds.b at 2023-05-20T01:17:38.715 INFO:tasks.ceph.mds.d.smithi132.stdout:starting mds.d at 2023-05-20T01:17:38.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:38.766+0000 7efc1a3b5700 1 Processor -- start 2023-05-20T01:17:38.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:38.766+0000 7efc1a3b5700 1 -- start start 2023-05-20T01:17:38.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:38.767+0000 7efc1a3b5700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7efc140b2760 0x7efc140d1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:38.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:38.767+0000 7efc1a3b5700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efc140cec60 0x7efc140cf0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:38.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:38.767+0000 7efc1a3b5700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc140cc500 con 0x7efc14144020 2023-05-20T01:17:38.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:38.767+0000 7efc1a3b5700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc140cb6e0 con 0x7efc140b2760 2023-05-20T01:17:38.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:38.767+0000 7efc1a3b5700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc140cf620 con 0x7efc140cec60 2023-05-20T01:17:38.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:38.767+0000 7efc137fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7efc14144020 0x7efc140b0980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:43218/0 (socket says 172.21.15.16:43218) 2023-05-20T01:17:38.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:38.767+0000 7efc137fe700 1 -- 172.21.15.16:0/286832101 learned_addr learned my addr 172.21.15.16:0/286832101 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:17:38.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:38.767+0000 7efc13fff700 1 --2- 172.21.15.16:0/286832101 >> v2:172.21.15.132:3300/0 conn(0x7efc140b2760 0x7efc140d1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:38.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:38.767+0000 7efc18952700 1 --2- 172.21.15.16:0/286832101 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efc140cec60 0x7efc140cf0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:38.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:38.767+0000 7efc12ffd700 1 -- 172.21.15.16:0/286832101 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2592949423 0 0) 0x7efc140cc500 con 0x7efc14144020 2023-05-20T01:17:38.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:38.767+0000 7efc12ffd700 1 -- 172.21.15.16:0/286832101 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efbfc0035b0 con 0x7efc14144020 2023-05-20T01:17:38.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:38.768+0000 7efc13fff700 1 -- 172.21.15.16:0/286832101 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efc140cec60 msgr2=0x7efc140cf0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:38.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:38.768+0000 7efc13fff700 1 --2- 172.21.15.16:0/286832101 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efc140cec60 0x7efc140cf0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:38.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:38.768+0000 7efc13fff700 1 -- 172.21.15.16:0/286832101 >> v1:172.21.15.16:6789/0 conn(0x7efc14144020 legacy=0x7efc140b0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:38.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:38.768+0000 7efc12ffd700 1 -- 172.21.15.16:0/286832101 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 618348369 0 0) 0x7efbfc0035b0 con 0x7efc14144020 2023-05-20T01:17:38.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:38.768+0000 7efc13fff700 1 -- 172.21.15.16:0/286832101 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc140cb8f0 con 0x7efc140b2760 2023-05-20T01:17:38.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:38.768+0000 7efc18952700 1 --2- 172.21.15.16:0/286832101 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efc140cec60 0x7efc140cf0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-05-20T01:17:38.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:38.768+0000 7efc13fff700 1 --2- 172.21.15.16:0/286832101 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efc140b2760 0x7efc140d1680 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7efc04009840 tx=0x7efc04009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec4881abe7544439 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:38.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:38.768+0000 7efc12ffd700 1 -- 172.21.15.16:0/286832101 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc08001670 con 0x7efc140b2760 2023-05-20T01:17:38.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:38.768+0000 7efc12ffd700 1 -- 172.21.15.16:0/286832101 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc04004030 con 0x7efc140b2760 2023-05-20T01:17:38.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:38.768+0000 7efc12ffd700 1 -- 172.21.15.16:0/286832101 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc040043c0 con 0x7efc140b2760 2023-05-20T01:17:38.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:38.769+0000 7efc1a3b5700 1 -- 172.21.15.16:0/286832101 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efc140b2760 msgr2=0x7efc140d1680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:38.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:38.769+0000 7efc1a3b5700 1 --2- 172.21.15.16:0/286832101 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efc140b2760 0x7efc140d1680 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7efc04009840 tx=0x7efc04009b50 comp rx=0 tx=0).stop 2023-05-20T01:17:38.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:38.769+0000 7efc1a3b5700 1 -- 172.21.15.16:0/286832101 shutdown_connections 2023-05-20T01:17:38.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:38.769+0000 7efc1a3b5700 1 --2- 172.21.15.16:0/286832101 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efc140cec60 0x7efc140cf0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:38.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:38.769+0000 7efc1a3b5700 1 --2- 172.21.15.16:0/286832101 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efc140b2760 0x7efc140d1680 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:38.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:38.769+0000 7efc1a3b5700 1 -- 172.21.15.16:0/286832101 >> 172.21.15.16:0/286832101 conn(0x7efc1406ae30 msgr2=0x7efc1409a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:38.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:38.769+0000 7efc1a3b5700 1 -- 172.21.15.16:0/286832101 shutdown_connections 2023-05-20T01:17:38.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:38.769+0000 7efc1a3b5700 1 -- 172.21.15.16:0/286832101 wait complete. 2023-05-20T01:17:38.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:38.770+0000 7efc1a3b5700 1 Processor -- start 2023-05-20T01:17:38.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:38.770+0000 7efc1a3b5700 1 -- start start 2023-05-20T01:17:38.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:38.771+0000 7efc1a3b5700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efc140b2760 0x7efc1406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:38.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:38.771+0000 7efc1a3b5700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efc140cec60 0x7efc141595b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:38.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:38.771+0000 7efc1a3b5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc14144020 0x7efc1415b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:38.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:38.771+0000 7efc1a3b5700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc14159af0 con 0x7efc14144020 2023-05-20T01:17:38.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:38.771+0000 7efc1a3b5700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc140b4330 con 0x7efc140cec60 2023-05-20T01:17:38.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:38.771+0000 7efc1a3b5700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc140b4610 con 0x7efc140b2760 2023-05-20T01:17:38.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:38.771+0000 7efc18952700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc14144020 0x7efc1415b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:38.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:38.771+0000 7efc18952700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc14144020 0x7efc1415b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59930/0 (socket says 172.21.15.16:59930) 2023-05-20T01:17:38.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:38.771+0000 7efc18952700 1 -- 172.21.15.16:0/3298384754 learned_addr learned my addr 172.21.15.16:0/3298384754 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:38.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:38.771+0000 7efc13fff700 1 --2- 172.21.15.16:0/3298384754 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efc140b2760 0x7efc1406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:38.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:38.771+0000 7efc137fe700 1 --2- 172.21.15.16:0/3298384754 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efc140cec60 0x7efc141595b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:38.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:38.771+0000 7efc18952700 1 -- 172.21.15.16:0/3298384754 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efc140b2760 msgr2=0x7efc1406c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:38.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:38.771+0000 7efc18952700 1 --2- 172.21.15.16:0/3298384754 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efc140b2760 0x7efc1406c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:38.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:38.771+0000 7efc18952700 1 -- 172.21.15.16:0/3298384754 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efc140cec60 msgr2=0x7efc141595b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:38.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:38.771+0000 7efc18952700 1 --2- 172.21.15.16:0/3298384754 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efc140cec60 0x7efc141595b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:38.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:38.771+0000 7efc18952700 1 -- 172.21.15.16:0/3298384754 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc0c009a70 con 0x7efc14144020 2023-05-20T01:17:38.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:38.771+0000 7efc13fff700 1 --2- 172.21.15.16:0/3298384754 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efc140b2760 0x7efc1406c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:17:38.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:38.772+0000 7efc18952700 1 --2- 172.21.15.16:0/3298384754 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc14144020 0x7efc1415b350 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7efc0c00efc0 tx=0x7efc0c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:38.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:38.772+0000 7efc10ff9700 1 -- 172.21.15.16:0/3298384754 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc0c009cd0 con 0x7efc14144020 2023-05-20T01:17:38.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:38.772+0000 7efc10ff9700 1 -- 172.21.15.16:0/3298384754 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc0c00f040 con 0x7efc14144020 2023-05-20T01:17:38.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:38.772+0000 7efc1a3b5700 1 -- 172.21.15.16:0/3298384754 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc040094a0 con 0x7efc14144020 2023-05-20T01:17:38.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:38.772+0000 7efc10ff9700 1 -- 172.21.15.16:0/3298384754 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc0c00cd60 con 0x7efc14144020 2023-05-20T01:17:38.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:38.772+0000 7efc1a3b5700 1 -- 172.21.15.16:0/3298384754 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc140cbeb0 con 0x7efc14144020 2023-05-20T01:17:38.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:38.773+0000 7efc1a3b5700 1 -- 172.21.15.16:0/3298384754 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc140cbbd0 con 0x7efc14144020 2023-05-20T01:17:38.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:38.773+0000 7efc10ff9700 1 -- 172.21.15.16:0/3298384754 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7efc0c014480 con 0x7efc14144020 2023-05-20T01:17:38.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:38.774+0000 7efc10ff9700 1 --2- 172.21.15.16:0/3298384754 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7efc0006ace0 0x7efc0006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:38.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:38.774+0000 7efc13fff700 1 --2- 172.21.15.16:0/3298384754 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7efc0006ace0 0x7efc0006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:38.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:38.774+0000 7efc10ff9700 1 -- 172.21.15.16:0/3298384754 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5616+0+0 (secure 0 0 0) 0x7efc0c089a50 con 0x7efc14144020 2023-05-20T01:17:38.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:38.774+0000 7efc13fff700 1 --2- 172.21.15.16:0/3298384754 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7efc0006ace0 0x7efc0006d190 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7efc0400a010 tx=0x7efc04013040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:38.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:38.778+0000 7efc10ff9700 1 -- 172.21.15.16:0/3298384754 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7efc0c054cc0 con 0x7efc14144020 2023-05-20T01:17:38.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:38.989+0000 7efc1a3b5700 1 -- 172.21.15.16:0/3298384754 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7efc140cc190 con 0x7efc14144020 2023-05-20T01:17:39.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:39.751+0000 7efc10ff9700 1 -- 172.21.15.16:0/3298384754 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v36) v1 ==== 97+0+0 (secure 0 0 0) 0x7efc0c0582e0 con 0x7efc14144020 2023-05-20T01:17:39.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:39.753+0000 7efc1a3b5700 1 -- 172.21.15.16:0/3298384754 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7efc0006ace0 msgr2=0x7efc0006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:39.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:39.753+0000 7efc1a3b5700 1 --2- 172.21.15.16:0/3298384754 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7efc0006ace0 0x7efc0006d190 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7efc0400a010 tx=0x7efc04013040 comp rx=0 tx=0).stop 2023-05-20T01:17:39.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:39.753+0000 7efc1a3b5700 1 -- 172.21.15.16:0/3298384754 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc14144020 msgr2=0x7efc1415b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:39.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:39.753+0000 7efc1a3b5700 1 --2- 172.21.15.16:0/3298384754 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc14144020 0x7efc1415b350 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7efc0c00efc0 tx=0x7efc0c00c3a0 comp rx=0 tx=0).stop 2023-05-20T01:17:39.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:39.754+0000 7efc1a3b5700 1 -- 172.21.15.16:0/3298384754 shutdown_connections 2023-05-20T01:17:39.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:39.754+0000 7efc1a3b5700 1 --2- 172.21.15.16:0/3298384754 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc14144020 0x7efc1415b350 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:39.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:39.754+0000 7efc1a3b5700 1 --2- 172.21.15.16:0/3298384754 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7efc0006ace0 0x7efc0006d190 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:39.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:39.754+0000 7efc1a3b5700 1 --2- 172.21.15.16:0/3298384754 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efc140b2760 0x7efc1406c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:39.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:39.754+0000 7efc1a3b5700 1 --2- 172.21.15.16:0/3298384754 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efc140cec60 0x7efc141595b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:39.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:39.754+0000 7efc1a3b5700 1 -- 172.21.15.16:0/3298384754 >> 172.21.15.16:0/3298384754 conn(0x7efc1406ae30 msgr2=0x7efc1406fba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:39.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:39.754+0000 7efc1a3b5700 1 -- 172.21.15.16:0/3298384754 shutdown_connections 2023-05-20T01:17:39.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:39.754+0000 7efc1a3b5700 1 -- 172.21.15.16:0/3298384754 wait complete. 2023-05-20T01:17:39.762 INFO:teuthology.orchestra.run.smithi016.stderr: removed all blocklist entries 2023-05-20T01:17:39.773 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-05-20T01:17:39.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:39.901+0000 7f54b8fde700 1 Processor -- start 2023-05-20T01:17:39.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:39.901+0000 7f54b8fde700 1 -- start start 2023-05-20T01:17:39.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:39.902+0000 7f54b8fde700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f54b4157190 0x7f54b40cc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:39.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:39.902+0000 7f54b8fde700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54b4071f20 con 0x7f54b40cccf0 2023-05-20T01:17:39.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:39.902+0000 7f54b8fde700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54b40cb590 con 0x7f54b4071ad0 2023-05-20T01:17:39.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:39.902+0000 7f54b8fde700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54b40ca9a0 con 0x7f54b4157190 2023-05-20T01:17:39.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:39.902+0000 7f54b1d9b700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f54b40cccf0 0x7f54b40713c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:43236/0 (socket says 172.21.15.16:43236) 2023-05-20T01:17:39.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:39.902+0000 7f54b1d9b700 1 -- 172.21.15.16:0/2694110301 learned_addr learned my addr 172.21.15.16:0/2694110301 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:17:39.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:39.902+0000 7f54b259c700 1 --2- 172.21.15.16:0/2694110301 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f54b4157190 0x7f54b40cc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:39.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:39.902+0000 7f54b159a700 1 -- 172.21.15.16:0/2694110301 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3493726385 0 0) 0x7f54a8001110 con 0x7f54b40cccf0 2023-05-20T01:17:39.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:39.902+0000 7f54b159a700 1 -- 172.21.15.16:0/2694110301 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f549c0035b0 con 0x7f54b40cccf0 2023-05-20T01:17:39.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:39.902+0000 7f54b159a700 1 -- 172.21.15.16:0/2694110301 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 363276082 0 0) 0x7f549c0035b0 con 0x7f54b40cccf0 2023-05-20T01:17:39.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:39.902+0000 7f54b159a700 1 -- 172.21.15.16:0/2694110301 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f54b4157190 msgr2=0x7f54b40cc7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:39.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:39.902+0000 7f54b159a700 1 --2- 172.21.15.16:0/2694110301 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f54b4157190 0x7f54b40cc7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:39.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:39.902+0000 7f54b159a700 1 -- 172.21.15.16:0/2694110301 >> v1:172.21.15.132:6789/0 conn(0x7f54b4071ad0 legacy=0x7f54b40ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:39.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:39.902+0000 7f54b159a700 1 -- 172.21.15.16:0/2694110301 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54b40cac60 con 0x7f54b40cccf0 2023-05-20T01:17:39.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:39.902+0000 7f54b159a700 1 -- 172.21.15.16:0/2694110301 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7f54a8001800 con 0x7f54b40cccf0 2023-05-20T01:17:39.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:39.902+0000 7f54b159a700 1 -- 172.21.15.16:0/2694110301 >> v1:172.21.15.16:6789/0 conn(0x7f54b40cccf0 legacy=0x7f54b40713c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:39.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:39.902+0000 7f54b159a700 1 --2- 172.21.15.16:0/2694110301 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f54b4157190 0x7f549c003aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:39.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:39.902+0000 7f54b159a700 1 --2- 172.21.15.16:0/2694110301 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f549c003fe0 0x7f549c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:39.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:39.903+0000 7f54b159a700 1 --2- 172.21.15.16:0/2694110301 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f549c004d90 0x7f549c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:39.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:39.903+0000 7f54b159a700 1 -- 172.21.15.16:0/2694110301 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54b40ca9a0 con 0x7f549c004d90 2023-05-20T01:17:39.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:39.903+0000 7f54b159a700 1 -- 172.21.15.16:0/2694110301 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f549c00bac0 con 0x7f549c003fe0 2023-05-20T01:17:39.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:39.903+0000 7f54b159a700 1 -- 172.21.15.16:0/2694110301 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f549c00bc80 con 0x7f54b4157190 2023-05-20T01:17:39.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:39.903+0000 7f54b2d9d700 1 --2- 172.21.15.16:0/2694110301 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f549c004d90 0x7f549c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:39.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:39.903+0000 7f54b1d9b700 1 --2- 172.21.15.16:0/2694110301 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f549c003fe0 0x7f549c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:39.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:39.903+0000 7f54b259c700 1 --2- 172.21.15.16:0/2694110301 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f54b4157190 0x7f549c003aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:39.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:39.903+0000 7f54b2d9d700 1 -- 172.21.15.16:0/2694110301 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f54b4157190 msgr2=0x7f549c003aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:39.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:39.903+0000 7f54b2d9d700 1 --2- 172.21.15.16:0/2694110301 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f54b4157190 0x7f549c003aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:39.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:39.903+0000 7f54b2d9d700 1 -- 172.21.15.16:0/2694110301 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f549c003fe0 msgr2=0x7f549c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:39.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:39.903+0000 7f54b2d9d700 1 --2- 172.21.15.16:0/2694110301 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f549c003fe0 0x7f549c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:39.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:39.903+0000 7f54b2d9d700 1 -- 172.21.15.16:0/2694110301 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f549c0035b0 con 0x7f549c004d90 2023-05-20T01:17:39.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:39.903+0000 7f54b2d9d700 1 --2- 172.21.15.16:0/2694110301 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f549c004d90 0x7f549c005200 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f54ac00c260 tx=0x7f549c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=248ec828a1b33e01 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:39.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:39.903+0000 7f54b159a700 1 -- 172.21.15.16:0/2694110301 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54ac001e70 con 0x7f549c004d90 2023-05-20T01:17:39.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:39.903+0000 7f54b159a700 1 -- 172.21.15.16:0/2694110301 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54ac00ce20 con 0x7f549c004d90 2023-05-20T01:17:39.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:39.903+0000 7f54b159a700 1 -- 172.21.15.16:0/2694110301 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54ac012660 con 0x7f549c004d90 2023-05-20T01:17:39.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:39.904+0000 7f54b8fde700 1 -- 172.21.15.16:0/2694110301 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f549c004d90 msgr2=0x7f549c005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:39.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:39.904+0000 7f54b8fde700 1 --2- 172.21.15.16:0/2694110301 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f549c004d90 0x7f549c005200 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f54ac00c260 tx=0x7f549c005880 comp rx=0 tx=0).stop 2023-05-20T01:17:39.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:39.904+0000 7f54b2d9d700 1 -- 172.21.15.16:0/2694110301 reap_dead start 2023-05-20T01:17:39.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:39.904+0000 7f54b8fde700 1 -- 172.21.15.16:0/2694110301 shutdown_connections 2023-05-20T01:17:39.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:39.904+0000 7f54b8fde700 1 -- 172.21.15.16:0/2694110301 >> 172.21.15.16:0/2694110301 conn(0x7f54b40cffb0 msgr2=0x7f54b40d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:39.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:39.904+0000 7f54b8fde700 1 -- 172.21.15.16:0/2694110301 shutdown_connections 2023-05-20T01:17:39.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:39.904+0000 7f54b8fde700 1 -- 172.21.15.16:0/2694110301 wait complete. 2023-05-20T01:17:39.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:39.905+0000 7f54b8fde700 1 Processor -- start 2023-05-20T01:17:39.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:39.905+0000 7f54b8fde700 1 -- start start 2023-05-20T01:17:39.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:39.906+0000 7f54b8fde700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f549c004d90 0x7f54b4155a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:39.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:39.906+0000 7f54b8fde700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54b4156160 0x7f54b40c9b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:39.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:39.906+0000 7f54b8fde700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f54b40c8150 0x7f54b41e2620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:39.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:39.906+0000 7f54b8fde700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54b41e4140 con 0x7f54b4156160 2023-05-20T01:17:39.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:39.906+0000 7f54b8fde700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54b40c8aa0 con 0x7f549c004d90 2023-05-20T01:17:39.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:39.906+0000 7f54b8fde700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54b40c8d80 con 0x7f54b40c8150 2023-05-20T01:17:39.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:39.906+0000 7f54b1d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54b4156160 0x7f54b40c9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:39.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:39.906+0000 7f54b1d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54b4156160 0x7f54b40c9b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59952/0 (socket says 172.21.15.16:59952) 2023-05-20T01:17:39.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:39.906+0000 7f54b1d9b700 1 -- 172.21.15.16:0/2037530523 learned_addr learned my addr 172.21.15.16:0/2037530523 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:39.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:39.906+0000 7f54b2d9d700 1 --2- 172.21.15.16:0/2037530523 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f54b40c8150 0x7f54b41e2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:39.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:39.906+0000 7f54b259c700 1 --2- 172.21.15.16:0/2037530523 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f549c004d90 0x7f54b4155a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:39.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:39.907+0000 7f54b1d9b700 1 -- 172.21.15.16:0/2037530523 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f54b40c8150 msgr2=0x7f54b41e2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:39.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:39.907+0000 7f54b1d9b700 1 --2- 172.21.15.16:0/2037530523 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f54b40c8150 0x7f54b41e2620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:39.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:39.907+0000 7f54b1d9b700 1 -- 172.21.15.16:0/2037530523 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f549c004d90 msgr2=0x7f54b4155a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:39.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:39.907+0000 7f54b1d9b700 1 --2- 172.21.15.16:0/2037530523 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f549c004d90 0x7f54b4155a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:39.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:39.907+0000 7f54b1d9b700 1 -- 172.21.15.16:0/2037530523 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54b40cb220 con 0x7f54b4156160 2023-05-20T01:17:39.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:39.907+0000 7f54b1d9b700 1 --2- 172.21.15.16:0/2037530523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54b4156160 0x7f54b40c9b40 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f54a8007290 tx=0x7f54a80075a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:39.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:39.907+0000 7f54a2ffd700 1 -- 172.21.15.16:0/2037530523 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54a800f660 con 0x7f54b4156160 2023-05-20T01:17:39.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:39.907+0000 7f54a2ffd700 1 -- 172.21.15.16:0/2037530523 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54a800f7c0 con 0x7f54b4156160 2023-05-20T01:17:39.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:39.907+0000 7f54b8fde700 1 -- 172.21.15.16:0/2037530523 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54b40caf40 con 0x7f54b4156160 2023-05-20T01:17:39.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:39.907+0000 7f54a2ffd700 1 -- 172.21.15.16:0/2037530523 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54a800fb70 con 0x7f54b4156160 2023-05-20T01:17:39.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:39.907+0000 7f54b8fde700 1 -- 172.21.15.16:0/2037530523 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54b4157560 con 0x7f54b4156160 2023-05-20T01:17:39.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:39.908+0000 7f54b8fde700 1 -- 172.21.15.16:0/2037530523 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54b40747b0 con 0x7f54b4156160 2023-05-20T01:17:39.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:39.909+0000 7f54a2ffd700 1 -- 172.21.15.16:0/2037530523 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f54a800fcd0 con 0x7f54b4156160 2023-05-20T01:17:39.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:39.909+0000 7f54a2ffd700 1 --2- 172.21.15.16:0/2037530523 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f549406ace0 0x7f549406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:39.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:39.909+0000 7f54a2ffd700 1 -- 172.21.15.16:0/2037530523 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f54a8089980 con 0x7f54b4156160 2023-05-20T01:17:39.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:39.909+0000 7f54b259c700 1 --2- 172.21.15.16:0/2037530523 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f549406ace0 0x7f549406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:39.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:39.910+0000 7f54b259c700 1 --2- 172.21.15.16:0/2037530523 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f549406ace0 0x7f549406d190 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f54b40ca360 tx=0x7f54a4009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:39.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:39.913+0000 7f54a2ffd700 1 -- 172.21.15.16:0/2037530523 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f54a8054e70 con 0x7f54b4156160 2023-05-20T01:17:40.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.191+0000 7f54b8fde700 1 -- 172.21.15.16:0/2037530523 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f54b4071f20 con 0x7f54b4156160 2023-05-20T01:17:40.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.192+0000 7f54a2ffd700 1 -- 172.21.15.16:0/2037530523 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v2) v1 ==== 80+0+6645 (secure 0 0 0) 0x7f54b4071f20 con 0x7f54b4156160 2023-05-20T01:17:40.196 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:17:40.197 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-05-20T01:17:40.197 INFO:teuthology.orchestra.run.smithi016.stdout: "auth_dump": [ 2023-05-20T01:17:40.197 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-05-20T01:17:40.198 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.a", 2023-05-20T01:17:40.198 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQA5H2hkq5FEJBAA1qUUPSLmwo43hTQS7GSENA==", 2023-05-20T01:17:40.198 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-05-20T01:17:40.198 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-05-20T01:17:40.198 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2023-05-20T01:17:40.198 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2023-05-20T01:17:40.198 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-05-20T01:17:40.199 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:17:40.199 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-05-20T01:17:40.199 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-05-20T01:17:40.199 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.b", 2023-05-20T01:17:40.199 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQA5H2hk/2XSMBAAnGCIm7TJZsO9HyXwOWxO7w==", 2023-05-20T01:17:40.199 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-05-20T01:17:40.199 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-05-20T01:17:40.199 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2023-05-20T01:17:40.200 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2023-05-20T01:17:40.200 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-05-20T01:17:40.200 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:17:40.200 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-05-20T01:17:40.201 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-05-20T01:17:40.201 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.c", 2023-05-20T01:17:40.201 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQA5H2hkBVK8KxAAgxy9+gbttxZeygwrAX+PHQ==", 2023-05-20T01:17:40.201 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-05-20T01:17:40.201 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-05-20T01:17:40.201 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2023-05-20T01:17:40.201 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2023-05-20T01:17:40.202 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-05-20T01:17:40.202 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:17:40.202 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-05-20T01:17:40.202 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-05-20T01:17:40.202 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.d", 2023-05-20T01:17:40.202 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQA5H2hkFxEMOBAAMPvue7F+/qqOFUD/C//U+g==", 2023-05-20T01:17:40.202 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-05-20T01:17:40.203 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-05-20T01:17:40.203 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2023-05-20T01:17:40.203 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2023-05-20T01:17:40.203 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-05-20T01:17:40.203 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:17:40.203 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-05-20T01:17:40.203 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-05-20T01:17:40.204 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.0", 2023-05-20T01:17:40.204 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQA7H2hkC4WwIRAA5KMQFqldcekzEuH92rZYHA==", 2023-05-20T01:17:40.204 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-05-20T01:17:40.204 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-05-20T01:17:40.205 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-05-20T01:17:40.205 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-05-20T01:17:40.205 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:17:40.205 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-05-20T01:17:40.205 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-05-20T01:17:40.205 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.1", 2023-05-20T01:17:40.205 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQA+H2hkvMH1IBAAhOQOjCiv3r2Pl3HcSS6eAA==", 2023-05-20T01:17:40.206 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-05-20T01:17:40.206 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-05-20T01:17:40.206 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-05-20T01:17:40.206 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-05-20T01:17:40.206 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:17:40.206 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-05-20T01:17:40.206 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-05-20T01:17:40.206 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.2", 2023-05-20T01:17:40.207 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQBBH2hkSJU5IBAAdFd4IhXxISVybFyT24gb0g==", 2023-05-20T01:17:40.207 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-05-20T01:17:40.207 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-05-20T01:17:40.207 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-05-20T01:17:40.207 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-05-20T01:17:40.207 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:17:40.207 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-05-20T01:17:40.208 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-05-20T01:17:40.208 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.3", 2023-05-20T01:17:40.208 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQBEH2hkrcH8HxAANlLWDyIYycLl1TrYCBN1hQ==", 2023-05-20T01:17:40.208 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-05-20T01:17:40.208 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-05-20T01:17:40.208 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-05-20T01:17:40.208 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-05-20T01:17:40.209 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:17:40.209 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-05-20T01:17:40.209 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-05-20T01:17:40.209 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.4", 2023-05-20T01:17:40.209 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQBKH2hkZr+4JBAAzbzKd4ulGI8kVGqXpjCDog==", 2023-05-20T01:17:40.209 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-05-20T01:17:40.209 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-05-20T01:17:40.210 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-05-20T01:17:40.210 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-05-20T01:17:40.210 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:17:40.210 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-05-20T01:17:40.210 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-05-20T01:17:40.210 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.5", 2023-05-20T01:17:40.210 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQBNH2hkNV02JBAALrC7eOTnD1dQM9njk+BgLQ==", 2023-05-20T01:17:40.211 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-05-20T01:17:40.211 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-05-20T01:17:40.211 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-05-20T01:17:40.211 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-05-20T01:17:40.212 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:17:40.212 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-05-20T01:17:40.212 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-05-20T01:17:40.212 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.6", 2023-05-20T01:17:40.212 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQBQH2hkGOfBIxAABCTVPLBi5+drKpqOC8xqdw==", 2023-05-20T01:17:40.212 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-05-20T01:17:40.213 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-05-20T01:17:40.213 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-05-20T01:17:40.213 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-05-20T01:17:40.213 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:17:40.213 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-05-20T01:17:40.213 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-05-20T01:17:40.213 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.7", 2023-05-20T01:17:40.213 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQBTH2hkpwEdIxAAaI3X5ZAVJ9MfVG0MjFJvSQ==", 2023-05-20T01:17:40.214 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-05-20T01:17:40.214 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-05-20T01:17:40.214 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-05-20T01:17:40.214 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-05-20T01:17:40.214 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:17:40.214 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-05-20T01:17:40.214 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-05-20T01:17:40.214 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.0", 2023-05-20T01:17:40.215 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQA6H2hkph3EABAA0277JHCqplw1LklDfidPWQ==", 2023-05-20T01:17:40.215 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-05-20T01:17:40.215 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-05-20T01:17:40.215 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2023-05-20T01:17:40.215 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2023-05-20T01:17:40.215 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2023-05-20T01:17:40.215 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:17:40.216 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-05-20T01:17:40.216 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-05-20T01:17:40.216 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.1", 2023-05-20T01:17:40.216 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQA6H2hk43WQBxAApSfBSu1qEaKSGWaYAvz/Jg==", 2023-05-20T01:17:40.216 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-05-20T01:17:40.216 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-05-20T01:17:40.216 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2023-05-20T01:17:40.217 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2023-05-20T01:17:40.217 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2023-05-20T01:17:40.217 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:17:40.217 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-05-20T01:17:40.217 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-05-20T01:17:40.217 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.2", 2023-05-20T01:17:40.217 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQA6H2hk+TgtBBAAc4+W2eOVgJfNd3xXjGvz4Q==", 2023-05-20T01:17:40.217 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-05-20T01:17:40.218 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-05-20T01:17:40.218 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2023-05-20T01:17:40.218 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2023-05-20T01:17:40.218 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2023-05-20T01:17:40.218 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:17:40.218 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-05-20T01:17:40.218 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-05-20T01:17:40.218 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.3", 2023-05-20T01:17:40.219 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQA6H2hkifVADRAAQP6vtdVNYyeNPV0d+agOCw==", 2023-05-20T01:17:40.219 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-05-20T01:17:40.219 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-05-20T01:17:40.219 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2023-05-20T01:17:40.219 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2023-05-20T01:17:40.219 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2023-05-20T01:17:40.219 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:17:40.220 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-05-20T01:17:40.220 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-05-20T01:17:40.220 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.admin", 2023-05-20T01:17:40.220 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQA5H2hkb4BMBhAAw7zpfDAb7x5szkLY+zTL7Q==", 2023-05-20T01:17:40.220 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-05-20T01:17:40.221 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2023-05-20T01:17:40.221 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2023-05-20T01:17:40.221 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2023-05-20T01:17:40.221 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-05-20T01:17:40.221 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:17:40.221 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-05-20T01:17:40.221 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-05-20T01:17:40.222 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.bootstrap-mds", 2023-05-20T01:17:40.222 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQBZH2hkDTLDIBAAPc/rF6wQJDGA+xIBlUjuAA==", 2023-05-20T01:17:40.222 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-05-20T01:17:40.222 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile bootstrap-mds" 2023-05-20T01:17:40.222 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:17:40.222 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-05-20T01:17:40.222 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-05-20T01:17:40.223 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.bootstrap-mgr", 2023-05-20T01:17:40.223 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQBZH2hkgjvDIBAAhg+1nkFNn8yxZMXZWmobKA==", 2023-05-20T01:17:40.223 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-05-20T01:17:40.223 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile bootstrap-mgr" 2023-05-20T01:17:40.223 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:17:40.223 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-05-20T01:17:40.223 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-05-20T01:17:40.223 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.bootstrap-osd", 2023-05-20T01:17:40.224 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQBZH2hkj0XDIBAAVpovi8chw+yAM4cVKWySvQ==", 2023-05-20T01:17:40.224 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-05-20T01:17:40.224 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile bootstrap-osd" 2023-05-20T01:17:40.224 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:17:40.224 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-05-20T01:17:40.224 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-05-20T01:17:40.224 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.bootstrap-rbd", 2023-05-20T01:17:40.225 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQBZH2hkmE3DIBAAxve5SwdACoVvcV/qRIJCqA==", 2023-05-20T01:17:40.225 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-05-20T01:17:40.225 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile bootstrap-rbd" 2023-05-20T01:17:40.225 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:17:40.225 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-05-20T01:17:40.226 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-05-20T01:17:40.226 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.bootstrap-rbd-mirror", 2023-05-20T01:17:40.226 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQBZH2hk2lXDIBAAufqU/pSRhMkK0qDVbsGazQ==", 2023-05-20T01:17:40.226 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-05-20T01:17:40.226 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2023-05-20T01:17:40.226 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:17:40.226 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-05-20T01:17:40.226 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-05-20T01:17:40.227 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.bootstrap-rgw", 2023-05-20T01:17:40.227 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQBZH2hkol/DIBAAk40DNtJOhhalcqp75pE6yw==", 2023-05-20T01:17:40.227 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-05-20T01:17:40.227 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile bootstrap-rgw" 2023-05-20T01:17:40.227 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:17:40.227 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-05-20T01:17:40.227 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-05-20T01:17:40.227 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mgr.x", 2023-05-20T01:17:40.228 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQA5H2hksQbMIBAAQFm/pY5stJHNudg6zcPBdQ==", 2023-05-20T01:17:40.228 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-05-20T01:17:40.228 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2023-05-20T01:17:40.228 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile mgr", 2023-05-20T01:17:40.228 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-05-20T01:17:40.228 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:17:40.228 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-05-20T01:17:40.229 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-05-20T01:17:40.229 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mgr.y", 2023-05-20T01:17:40.229 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQA5H2hkBQg7HBAAbOJZ+55rRACVBMd2eyjqtg==", 2023-05-20T01:17:40.229 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-05-20T01:17:40.229 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2023-05-20T01:17:40.229 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile mgr", 2023-05-20T01:17:40.229 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-05-20T01:17:40.230 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:17:40.230 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:17:40.230 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2023-05-20T01:17:40.230 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-05-20T01:17:40.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.194+0000 7f54b8fde700 1 -- 172.21.15.16:0/2037530523 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f549406ace0 msgr2=0x7f549406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:40.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.194+0000 7f54b8fde700 1 --2- 172.21.15.16:0/2037530523 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f549406ace0 0x7f549406d190 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f54b40ca360 tx=0x7f54a4009210 comp rx=0 tx=0).stop 2023-05-20T01:17:40.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.194+0000 7f54b8fde700 1 -- 172.21.15.16:0/2037530523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54b4156160 msgr2=0x7f54b40c9b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:40.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.194+0000 7f54b8fde700 1 --2- 172.21.15.16:0/2037530523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54b4156160 0x7f54b40c9b40 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f54a8007290 tx=0x7f54a80075a0 comp rx=0 tx=0).stop 2023-05-20T01:17:40.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.194+0000 7f54b8fde700 1 -- 172.21.15.16:0/2037530523 shutdown_connections 2023-05-20T01:17:40.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.194+0000 7f54b8fde700 1 --2- 172.21.15.16:0/2037530523 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f549406ace0 0x7f549406d190 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:40.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.194+0000 7f54b8fde700 1 --2- 172.21.15.16:0/2037530523 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f54b40c8150 0x7f54b41e2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:40.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.194+0000 7f54b8fde700 1 --2- 172.21.15.16:0/2037530523 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f549c004d90 0x7f54b4155a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:40.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.194+0000 7f54b8fde700 1 --2- 172.21.15.16:0/2037530523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54b4156160 0x7f54b40c9b40 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:40.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.194+0000 7f54b8fde700 1 -- 172.21.15.16:0/2037530523 >> 172.21.15.16:0/2037530523 conn(0x7f54b40cffb0 msgr2=0x7f54b406dc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:40.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.194+0000 7f54b8fde700 1 -- 172.21.15.16:0/2037530523 shutdown_connections 2023-05-20T01:17:40.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.194+0000 7f54b8fde700 1 -- 172.21.15.16:0/2037530523 wait complete. 2023-05-20T01:17:40.233 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2023-05-20T01:17:40.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.357+0000 7f823ba2b700 1 Processor -- start 2023-05-20T01:17:40.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.357+0000 7f823ba2b700 1 -- start start 2023-05-20T01:17:40.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.358+0000 7f823ba2b700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f823406ba90 0x7f82340cbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:40.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.358+0000 7f823ba2b700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82340ce3e0 con 0x7f82340d1000 2023-05-20T01:17:40.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.358+0000 7f823ba2b700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82340cdab0 con 0x7f823406b620 2023-05-20T01:17:40.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.358+0000 7f823ba2b700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82340cdd90 con 0x7f823406ba90 2023-05-20T01:17:40.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.358+0000 7f82397c7700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f82340d1000 0x7f82340d13d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:43254/0 (socket says 172.21.15.16:43254) 2023-05-20T01:17:40.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.358+0000 7f82397c7700 1 -- 172.21.15.16:0/1340771082 learned_addr learned my addr 172.21.15.16:0/1340771082 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:17:40.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.358+0000 7f822bfff700 1 -- 172.21.15.16:0/1340771082 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3532415635 0 0) 0x7f82340ce3e0 con 0x7f82340d1000 2023-05-20T01:17:40.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.358+0000 7f8239fc8700 1 --2- 172.21.15.16:0/1340771082 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f823406ba90 0x7f82340cbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:40.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.358+0000 7f822bfff700 1 -- 172.21.15.16:0/1340771082 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f82200035b0 con 0x7f82340d1000 2023-05-20T01:17:40.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.358+0000 7f822bfff700 1 -- 172.21.15.16:0/1340771082 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 81810374 0 0) 0x7f82200035b0 con 0x7f82340d1000 2023-05-20T01:17:40.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.358+0000 7f822bfff700 1 -- 172.21.15.16:0/1340771082 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f823406ba90 msgr2=0x7f82340cbdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:40.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.358+0000 7f822bfff700 1 --2- 172.21.15.16:0/1340771082 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f823406ba90 0x7f82340cbdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:40.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.358+0000 7f822bfff700 1 -- 172.21.15.16:0/1340771082 >> v1:172.21.15.132:6789/0 conn(0x7f823406b620 legacy=0x7f82340cb6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:40.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.358+0000 7f822bfff700 1 -- 172.21.15.16:0/1340771082 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82340ce070 con 0x7f82340d1000 2023-05-20T01:17:40.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.358+0000 7f822bfff700 1 -- 172.21.15.16:0/1340771082 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7f8230001670 con 0x7f82340d1000 2023-05-20T01:17:40.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.358+0000 7f822bfff700 1 -- 172.21.15.16:0/1340771082 >> v1:172.21.15.16:6789/0 conn(0x7f82340d1000 legacy=0x7f82340d13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:40.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.359+0000 7f822bfff700 1 --2- 172.21.15.16:0/1340771082 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8220003aa0 0x7f8220003eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:40.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.359+0000 7f822bfff700 1 --2- 172.21.15.16:0/1340771082 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f823406ba90 0x7f8220004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:40.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.359+0000 7f822bfff700 1 --2- 172.21.15.16:0/1340771082 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8220004d90 0x7f8220005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:40.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.359+0000 7f822bfff700 1 -- 172.21.15.16:0/1340771082 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82340cdd90 con 0x7f8220004d90 2023-05-20T01:17:40.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.359+0000 7f822bfff700 1 -- 172.21.15.16:0/1340771082 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f822000baf0 con 0x7f8220003aa0 2023-05-20T01:17:40.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.359+0000 7f822bfff700 1 -- 172.21.15.16:0/1340771082 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f822000bcb0 con 0x7f823406ba90 2023-05-20T01:17:40.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.359+0000 7f8239fc8700 1 --2- 172.21.15.16:0/1340771082 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8220004d90 0x7f8220005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:40.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.359+0000 7f82397c7700 1 --2- 172.21.15.16:0/1340771082 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8220003aa0 0x7f8220003eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:40.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.359+0000 7f8238fc6700 1 --2- 172.21.15.16:0/1340771082 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f823406ba90 0x7f8220004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:40.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.359+0000 7f8239fc8700 1 -- 172.21.15.16:0/1340771082 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f823406ba90 msgr2=0x7f8220004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:40.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.359+0000 7f8239fc8700 1 --2- 172.21.15.16:0/1340771082 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f823406ba90 0x7f8220004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:40.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.359+0000 7f8239fc8700 1 -- 172.21.15.16:0/1340771082 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8220003aa0 msgr2=0x7f8220003eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:40.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.359+0000 7f8239fc8700 1 --2- 172.21.15.16:0/1340771082 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8220003aa0 0x7f8220003eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:40.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.359+0000 7f8239fc8700 1 -- 172.21.15.16:0/1340771082 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82200035b0 con 0x7f8220004d90 2023-05-20T01:17:40.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.359+0000 7f8239fc8700 1 --2- 172.21.15.16:0/1340771082 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8220004d90 0x7f8220005200 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f822c009fc0 tx=0x7f8220005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c54ddef9e1c6347 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:40.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.359+0000 7f822bfff700 1 -- 172.21.15.16:0/1340771082 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f822c00ec30 con 0x7f8220004d90 2023-05-20T01:17:40.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.359+0000 7f822bfff700 1 -- 172.21.15.16:0/1340771082 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f822c00f040 con 0x7f8220004d90 2023-05-20T01:17:40.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.359+0000 7f822bfff700 1 -- 172.21.15.16:0/1340771082 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f822c014480 con 0x7f8220004d90 2023-05-20T01:17:40.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.360+0000 7f823ba2b700 1 -- 172.21.15.16:0/1340771082 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8220004d90 msgr2=0x7f8220005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:40.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.360+0000 7f823ba2b700 1 --2- 172.21.15.16:0/1340771082 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8220004d90 0x7f8220005200 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f822c009fc0 tx=0x7f8220005880 comp rx=0 tx=0).stop 2023-05-20T01:17:40.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.360+0000 7f8239fc8700 1 -- 172.21.15.16:0/1340771082 reap_dead start 2023-05-20T01:17:40.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.360+0000 7f823ba2b700 1 -- 172.21.15.16:0/1340771082 shutdown_connections 2023-05-20T01:17:40.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.360+0000 7f823ba2b700 1 -- 172.21.15.16:0/1340771082 >> 172.21.15.16:0/1340771082 conn(0x7f823406ae30 msgr2=0x7f823409a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:40.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.360+0000 7f823ba2b700 1 -- 172.21.15.16:0/1340771082 shutdown_connections 2023-05-20T01:17:40.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.360+0000 7f823ba2b700 1 -- 172.21.15.16:0/1340771082 wait complete. 2023-05-20T01:17:40.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.361+0000 7f823ba2b700 1 Processor -- start 2023-05-20T01:17:40.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.361+0000 7f823ba2b700 1 -- start start 2023-05-20T01:17:40.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.362+0000 7f823ba2b700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f823415a470 0x7f82340c7680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:40.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.362+0000 7f823ba2b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f82340b5db0 0x7f82340c60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:40.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.362+0000 7f823ba2b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82340c6620 0x7f82340b3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:40.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.362+0000 7f823ba2b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82340c6a90 con 0x7f82340c6620 2023-05-20T01:17:40.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.362+0000 7f823ba2b700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f823415a840 con 0x7f82340b5db0 2023-05-20T01:17:40.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.362+0000 7f823ba2b700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82340b6b30 con 0x7f823415a470 2023-05-20T01:17:40.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.362+0000 7f8239fc8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82340c6620 0x7f82340b3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:40.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.362+0000 7f8239fc8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82340c6620 0x7f82340b3ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59974/0 (socket says 172.21.15.16:59974) 2023-05-20T01:17:40.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.362+0000 7f8239fc8700 1 -- 172.21.15.16:0/2442928763 learned_addr learned my addr 172.21.15.16:0/2442928763 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:40.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.362+0000 7f82397c7700 1 --2- 172.21.15.16:0/2442928763 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f823415a470 0x7f82340c7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:40.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.362+0000 7f8238fc6700 1 --2- 172.21.15.16:0/2442928763 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f82340b5db0 0x7f82340c60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:40.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.362+0000 7f8239fc8700 1 -- 172.21.15.16:0/2442928763 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f823415a470 msgr2=0x7f82340c7680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:40.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.362+0000 7f8239fc8700 1 --2- 172.21.15.16:0/2442928763 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f823415a470 0x7f82340c7680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:40.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.362+0000 7f8239fc8700 1 -- 172.21.15.16:0/2442928763 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f82340b5db0 msgr2=0x7f82340c60e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:40.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.362+0000 7f8239fc8700 1 --2- 172.21.15.16:0/2442928763 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f82340b5db0 0x7f82340c60e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:40.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.362+0000 7f8239fc8700 1 -- 172.21.15.16:0/2442928763 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82340ce870 con 0x7f82340c6620 2023-05-20T01:17:40.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.363+0000 7f8239fc8700 1 --2- 172.21.15.16:0/2442928763 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82340c6620 0x7f82340b3ec0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f822c009f60 tx=0x7f822c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:40.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.363+0000 7f8229ffb700 1 -- 172.21.15.16:0/2442928763 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f822c007000 con 0x7f82340c6620 2023-05-20T01:17:40.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.363+0000 7f8229ffb700 1 -- 172.21.15.16:0/2442928763 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f822c003e30 con 0x7f82340c6620 2023-05-20T01:17:40.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.363+0000 7f823ba2b700 1 -- 172.21.15.16:0/2442928763 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82340b28b0 con 0x7f82340c6620 2023-05-20T01:17:40.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.363+0000 7f8229ffb700 1 -- 172.21.15.16:0/2442928763 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f822c015a90 con 0x7f82340c6620 2023-05-20T01:17:40.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.363+0000 7f823ba2b700 1 -- 172.21.15.16:0/2442928763 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82340cd390 con 0x7f82340c6620 2023-05-20T01:17:40.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.364+0000 7f823ba2b700 1 -- 172.21.15.16:0/2442928763 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82340747b0 con 0x7f82340c6620 2023-05-20T01:17:40.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.364+0000 7f8229ffb700 1 -- 172.21.15.16:0/2442928763 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f822c024070 con 0x7f82340c6620 2023-05-20T01:17:40.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.365+0000 7f8229ffb700 1 --2- 172.21.15.16:0/2442928763 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f821806ad10 0x7f821806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:40.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.365+0000 7f8229ffb700 1 -- 172.21.15.16:0/2442928763 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f822c08ae80 con 0x7f82340c6620 2023-05-20T01:17:40.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.365+0000 7f82397c7700 1 --2- 172.21.15.16:0/2442928763 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f821806ad10 0x7f821806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:40.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.365+0000 7f82397c7700 1 --2- 172.21.15.16:0/2442928763 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f821806ad10 0x7f821806d1c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f8230001050 tx=0x7f8230000f70 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:40.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.369+0000 7f8229ffb700 1 -- 172.21.15.16:0/2442928763 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f822c056290 con 0x7f82340c6620 2023-05-20T01:17:40.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.653+0000 7f823ba2b700 1 -- 172.21.15.16:0/2442928763 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f82340b5160 con 0x7f82340c6620 2023-05-20T01:17:40.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.656+0000 7f8229ffb700 1 -- 172.21.15.16:0/2442928763 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v3) v1 ==== 90+0+0 (secure 0 0 0) 0x7f822c0180a0 con 0x7f82340c6620 2023-05-20T01:17:40.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.658+0000 7f823ba2b700 1 -- 172.21.15.16:0/2442928763 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f821806ad10 msgr2=0x7f821806d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:40.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.658+0000 7f823ba2b700 1 --2- 172.21.15.16:0/2442928763 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f821806ad10 0x7f821806d1c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f8230001050 tx=0x7f8230000f70 comp rx=0 tx=0).stop 2023-05-20T01:17:40.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.658+0000 7f823ba2b700 1 -- 172.21.15.16:0/2442928763 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82340c6620 msgr2=0x7f82340b3ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:40.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.658+0000 7f823ba2b700 1 --2- 172.21.15.16:0/2442928763 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82340c6620 0x7f82340b3ec0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f822c009f60 tx=0x7f822c009750 comp rx=0 tx=0).stop 2023-05-20T01:17:40.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.658+0000 7f823ba2b700 1 -- 172.21.15.16:0/2442928763 shutdown_connections 2023-05-20T01:17:40.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.658+0000 7f823ba2b700 1 --2- 172.21.15.16:0/2442928763 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82340c6620 0x7f82340b3ec0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:40.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.658+0000 7f823ba2b700 1 --2- 172.21.15.16:0/2442928763 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f821806ad10 0x7f821806d1c0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:40.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.658+0000 7f823ba2b700 1 --2- 172.21.15.16:0/2442928763 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f823415a470 0x7f82340c7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:40.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.658+0000 7f823ba2b700 1 --2- 172.21.15.16:0/2442928763 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f82340b5db0 0x7f82340c60e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:40.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.658+0000 7f823ba2b700 1 -- 172.21.15.16:0/2442928763 >> 172.21.15.16:0/2442928763 conn(0x7f823406ae30 msgr2=0x7f823406e390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:40.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.659+0000 7f823ba2b700 1 -- 172.21.15.16:0/2442928763 shutdown_connections 2023-05-20T01:17:40.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.659+0000 7f823ba2b700 1 -- 172.21.15.16:0/2442928763 wait complete. 2023-05-20T01:17:40.674 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2023-05-20T01:17:40.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.796+0000 7f95af860700 1 Processor -- start 2023-05-20T01:17:40.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.797+0000 7f95af860700 1 -- start start 2023-05-20T01:17:40.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.797+0000 7f95af860700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f95a80b4750 0x7f95a80d2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:40.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.797+0000 7f95af860700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95a80b3b50 con 0x7f95a80cfc50 2023-05-20T01:17:40.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.797+0000 7f95af860700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95a80d0b90 con 0x7f95a80cd970 2023-05-20T01:17:40.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.797+0000 7f95af860700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95a80cfad0 con 0x7f95a80b4750 2023-05-20T01:17:40.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.797+0000 7f95acdfb700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f95a80b4750 0x7f95a80d2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:40.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.798+0000 7f95ad5fc700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f95a80cfc50 0x7f95a80d0020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:43274/0 (socket says 172.21.15.16:43274) 2023-05-20T01:17:40.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.798+0000 7f95acdfb700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f95a80b4750 0x7f95a80d2c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59978/0 (socket says 172.21.15.16:59978) 2023-05-20T01:17:40.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.798+0000 7f95ad5fc700 1 -- 172.21.15.16:0/2780612359 learned_addr learned my addr 172.21.15.16:0/2780612359 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:17:40.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.798+0000 7f95acdfb700 1 -- 172.21.15.16:0/2780612359 >> v1:172.21.15.132:6789/0 conn(0x7f95a80cd970 legacy=0x7f95a80cdd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:40.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.798+0000 7f95acdfb700 1 -- 172.21.15.16:0/2780612359 >> v1:172.21.15.16:6789/0 conn(0x7f95a80cfc50 legacy=0x7f95a80d0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:40.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.798+0000 7f95acdfb700 1 -- 172.21.15.16:0/2780612359 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95a815d0b0 con 0x7f95a80b4750 2023-05-20T01:17:40.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.798+0000 7f959ffff700 1 -- 172.21.15.16:0/2780612359 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3881466324 0 0) 0x7f95a80b3b50 con 0x7f95a80cfc50 2023-05-20T01:17:40.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.798+0000 7f95acdfb700 1 --2- 172.21.15.16:0/2780612359 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95a80b4750 0x7f95a80d2c10 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f95a4009840 tx=0x7f95a4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62bf1682d3c2f6c1 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:40.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.798+0000 7f959ffff700 1 -- 172.21.15.16:0/2780612359 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a4004230 con 0x7f95a80b4750 2023-05-20T01:17:40.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.798+0000 7f959ffff700 1 -- 172.21.15.16:0/2780612359 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95a4004390 con 0x7f95a80b4750 2023-05-20T01:17:40.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.798+0000 7f959ffff700 1 -- 172.21.15.16:0/2780612359 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a40036a0 con 0x7f95a80b4750 2023-05-20T01:17:40.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.799+0000 7f95af860700 1 -- 172.21.15.16:0/2780612359 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95a80b4750 msgr2=0x7f95a80d2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:40.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.799+0000 7f95af860700 1 --2- 172.21.15.16:0/2780612359 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95a80b4750 0x7f95a80d2c10 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f95a4009840 tx=0x7f95a4009b50 comp rx=0 tx=0).stop 2023-05-20T01:17:40.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.799+0000 7f95af860700 1 -- 172.21.15.16:0/2780612359 shutdown_connections 2023-05-20T01:17:40.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.799+0000 7f95af860700 1 --2- 172.21.15.16:0/2780612359 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95a80b4750 0x7f95a80d2c10 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:40.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.799+0000 7f95af860700 1 -- 172.21.15.16:0/2780612359 >> 172.21.15.16:0/2780612359 conn(0x7f95a809c590 msgr2=0x7f95a80d3240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:40.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.799+0000 7f95af860700 1 -- 172.21.15.16:0/2780612359 shutdown_connections 2023-05-20T01:17:40.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.799+0000 7f95af860700 1 -- 172.21.15.16:0/2780612359 wait complete. 2023-05-20T01:17:40.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.800+0000 7f95af860700 1 Processor -- start 2023-05-20T01:17:40.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.800+0000 7f95af860700 1 -- start start 2023-05-20T01:17:40.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.801+0000 7f95af860700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95a80b4750 0x7f95a81f5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:40.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.801+0000 7f95af860700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f95a80cd970 0x7f95a81588c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:40.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.801+0000 7f95af860700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f95a80cfc50 0x7f95a80b71f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:40.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.801+0000 7f95af860700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95a80ca440 con 0x7f95a80b4750 2023-05-20T01:17:40.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.801+0000 7f95af860700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95a81f6920 con 0x7f95a80cd970 2023-05-20T01:17:40.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.801+0000 7f95af860700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95a81f6080 con 0x7f95a80cfc50 2023-05-20T01:17:40.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.801+0000 7f95ad5fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95a80b4750 0x7f95a81f5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:40.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.801+0000 7f95ad5fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95a80b4750 0x7f95a81f5aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59984/0 (socket says 172.21.15.16:59984) 2023-05-20T01:17:40.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.801+0000 7f95ad5fc700 1 -- 172.21.15.16:0/2335617324 learned_addr learned my addr 172.21.15.16:0/2335617324 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:40.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.801+0000 7f95acdfb700 1 --2- 172.21.15.16:0/2335617324 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f95a80cd970 0x7f95a81588c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:40.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.801+0000 7f95addfd700 1 --2- 172.21.15.16:0/2335617324 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f95a80cfc50 0x7f95a80b71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:40.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.801+0000 7f95ad5fc700 1 -- 172.21.15.16:0/2335617324 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f95a80cfc50 msgr2=0x7f95a80b71f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:40.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.801+0000 7f95ad5fc700 1 --2- 172.21.15.16:0/2335617324 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f95a80cfc50 0x7f95a80b71f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:40.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.801+0000 7f95ad5fc700 1 -- 172.21.15.16:0/2335617324 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f95a80cd970 msgr2=0x7f95a81588c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:40.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.801+0000 7f95ad5fc700 1 --2- 172.21.15.16:0/2335617324 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f95a80cd970 0x7f95a81588c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:40.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.801+0000 7f95ad5fc700 1 -- 172.21.15.16:0/2335617324 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95a40094a0 con 0x7f95a80b4750 2023-05-20T01:17:40.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.801+0000 7f95acdfb700 1 --2- 172.21.15.16:0/2335617324 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f95a80cd970 0x7f95a81588c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:17:40.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.801+0000 7f95addfd700 1 --2- 172.21.15.16:0/2335617324 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f95a80cfc50 0x7f95a80b71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:17:40.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.801+0000 7f95ad5fc700 1 --2- 172.21.15.16:0/2335617324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95a80b4750 0x7f95a81f5aa0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f959800c550 tx=0x7f959800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:40.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.802+0000 7f959dffb700 1 -- 172.21.15.16:0/2335617324 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9598004c80 con 0x7f95a80b4750 2023-05-20T01:17:40.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.802+0000 7f959dffb700 1 -- 172.21.15.16:0/2335617324 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9598004e00 con 0x7f95a80b4750 2023-05-20T01:17:40.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.802+0000 7f95af860700 1 -- 172.21.15.16:0/2335617324 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95a80d4000 con 0x7f95a80b4750 2023-05-20T01:17:40.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.802+0000 7f959dffb700 1 -- 172.21.15.16:0/2335617324 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9598007310 con 0x7f95a80b4750 2023-05-20T01:17:40.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.802+0000 7f95af860700 1 -- 172.21.15.16:0/2335617324 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95a80b4380 con 0x7f95a80b4750 2023-05-20T01:17:40.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.803+0000 7f95af860700 1 -- 172.21.15.16:0/2335617324 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95a80b3da0 con 0x7f95a80b4750 2023-05-20T01:17:40.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.803+0000 7f959dffb700 1 -- 172.21.15.16:0/2335617324 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9598007470 con 0x7f95a80b4750 2023-05-20T01:17:40.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.804+0000 7f959dffb700 1 --2- 172.21.15.16:0/2335617324 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f958c06ace0 0x7f958c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:40.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.804+0000 7f959dffb700 1 -- 172.21.15.16:0/2335617324 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f9598088ff0 con 0x7f95a80b4750 2023-05-20T01:17:40.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.804+0000 7f95acdfb700 1 --2- 172.21.15.16:0/2335617324 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f958c06ace0 0x7f958c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:40.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.804+0000 7f95acdfb700 1 --2- 172.21.15.16:0/2335617324 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f958c06ace0 0x7f958c06d190 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f95a80b6fe0 tx=0x7f95a4013040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:40.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:40.808+0000 7f959dffb700 1 -- 172.21.15.16:0/2335617324 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f9598054210 con 0x7f95a80b4750 2023-05-20T01:17:41.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.087+0000 7f95af860700 1 -- 172.21.15.16:0/2335617324 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f95a80d1cd0 con 0x7f95a80b4750 2023-05-20T01:17:41.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.090+0000 7f959dffb700 1 -- 172.21.15.16:0/2335617324 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v4) v1 ==== 90+0+0 (secure 0 0 0) 0x7f9598057830 con 0x7f95a80b4750 2023-05-20T01:17:41.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.092+0000 7f95af860700 1 -- 172.21.15.16:0/2335617324 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f958c06ace0 msgr2=0x7f958c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:41.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.092+0000 7f95af860700 1 --2- 172.21.15.16:0/2335617324 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f958c06ace0 0x7f958c06d190 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f95a80b6fe0 tx=0x7f95a4013040 comp rx=0 tx=0).stop 2023-05-20T01:17:41.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.092+0000 7f95af860700 1 -- 172.21.15.16:0/2335617324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95a80b4750 msgr2=0x7f95a81f5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:41.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.092+0000 7f95af860700 1 --2- 172.21.15.16:0/2335617324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95a80b4750 0x7f95a81f5aa0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f959800c550 tx=0x7f959800c910 comp rx=0 tx=0).stop 2023-05-20T01:17:41.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.092+0000 7f95af860700 1 -- 172.21.15.16:0/2335617324 shutdown_connections 2023-05-20T01:17:41.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.092+0000 7f95af860700 1 --2- 172.21.15.16:0/2335617324 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f958c06ace0 0x7f958c06d190 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:41.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.092+0000 7f95af860700 1 --2- 172.21.15.16:0/2335617324 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f95a80cfc50 0x7f95a80b71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:41.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.093+0000 7f95af860700 1 --2- 172.21.15.16:0/2335617324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95a80b4750 0x7f95a81f5aa0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:41.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.093+0000 7f95af860700 1 --2- 172.21.15.16:0/2335617324 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f95a80cd970 0x7f95a81588c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:41.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.093+0000 7f95af860700 1 -- 172.21.15.16:0/2335617324 >> 172.21.15.16:0/2335617324 conn(0x7f95a809c590 msgr2=0x7f95a80d0020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:41.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.093+0000 7f95af860700 1 -- 172.21.15.16:0/2335617324 shutdown_connections 2023-05-20T01:17:41.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.093+0000 7f95af860700 1 -- 172.21.15.16:0/2335617324 wait complete. 2023-05-20T01:17:41.108 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2023-05-20T01:17:41.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.230+0000 7f588c3a7700 1 Processor -- start 2023-05-20T01:17:41.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.230+0000 7f588c3a7700 1 -- start start 2023-05-20T01:17:41.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.231+0000 7f588c3a7700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5884144020 0x7f58840d1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:41.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.231+0000 7f588c3a7700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f58840cec60 0x7f58840cf0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:41.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.231+0000 7f588c3a7700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58840cc500 con 0x7f58840b2760 2023-05-20T01:17:41.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.231+0000 7f588c3a7700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58840cb6e0 con 0x7f58840cec60 2023-05-20T01:17:41.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.231+0000 7f588c3a7700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58840cf620 con 0x7f5884144020 2023-05-20T01:17:41.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.231+0000 7f588a944700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f58840cec60 0x7f58840cf0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:41.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.231+0000 7f588a944700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f58840cec60 0x7f58840cf0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59996/0 (socket says 172.21.15.16:59996) 2023-05-20T01:17:41.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.231+0000 7f588a944700 1 -- 172.21.15.16:0/781493784 learned_addr learned my addr 172.21.15.16:0/781493784 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:41.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.231+0000 7f5889942700 1 --2- 172.21.15.16:0/781493784 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5884144020 0x7f58840d1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:41.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.231+0000 7f588a944700 1 -- 172.21.15.16:0/781493784 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5884144020 msgr2=0x7f58840d1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:41.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.231+0000 7f588a944700 1 --2- 172.21.15.16:0/781493784 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5884144020 0x7f58840d1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:41.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.231+0000 7f588a944700 1 -- 172.21.15.16:0/781493784 >> v1:172.21.15.132:6789/0 conn(0x7f58840b2760 legacy=0x7f58840b0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:41.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.231+0000 7f588a944700 1 -- 172.21.15.16:0/781493784 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58840cb8f0 con 0x7f58840cec60 2023-05-20T01:17:41.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.231+0000 7f588a944700 1 --2- 172.21.15.16:0/781493784 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58840cec60 0x7f58840cf0e0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f587c009e50 tx=0x7f587c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9fc55ace2cb16f9 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:41.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.232+0000 7f5889141700 1 -- 172.21.15.16:0/781493784 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f587c00e070 con 0x7f58840cec60 2023-05-20T01:17:41.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.232+0000 7f5889141700 1 -- 172.21.15.16:0/781493784 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f587c00b9c0 con 0x7f58840cec60 2023-05-20T01:17:41.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.232+0000 7f5889141700 1 -- 172.21.15.16:0/781493784 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f587c00bd70 con 0x7f58840cec60 2023-05-20T01:17:41.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.232+0000 7f588c3a7700 1 -- 172.21.15.16:0/781493784 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58840cec60 msgr2=0x7f58840cf0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:41.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.232+0000 7f588c3a7700 1 --2- 172.21.15.16:0/781493784 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58840cec60 0x7f58840cf0e0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f587c009e50 tx=0x7f587c00b330 comp rx=0 tx=0).stop 2023-05-20T01:17:41.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.232+0000 7f588c3a7700 1 -- 172.21.15.16:0/781493784 shutdown_connections 2023-05-20T01:17:41.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.232+0000 7f588c3a7700 1 --2- 172.21.15.16:0/781493784 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58840cec60 0x7f58840cf0e0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:41.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.232+0000 7f588c3a7700 1 --2- 172.21.15.16:0/781493784 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5884144020 0x7f58840d1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:41.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.232+0000 7f588c3a7700 1 -- 172.21.15.16:0/781493784 >> 172.21.15.16:0/781493784 conn(0x7f588406ae30 msgr2=0x7f588409a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:41.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.232+0000 7f588c3a7700 1 -- 172.21.15.16:0/781493784 shutdown_connections 2023-05-20T01:17:41.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.232+0000 7f588c3a7700 1 -- 172.21.15.16:0/781493784 wait complete. 2023-05-20T01:17:41.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.233+0000 7f588c3a7700 1 Processor -- start 2023-05-20T01:17:41.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.233+0000 7f588c3a7700 1 -- start start 2023-05-20T01:17:41.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.234+0000 7f588c3a7700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58840b2760 0x7f5884154960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:41.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.234+0000 7f588c3a7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58840cec60 0x7f58840ca070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:41.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.234+0000 7f588c3a7700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5884144020 0x7f58840c8160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:41.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.234+0000 7f588c3a7700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f588406b620 con 0x7f58840cec60 2023-05-20T01:17:41.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.234+0000 7f588c3a7700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f588406b900 con 0x7f58840b2760 2023-05-20T01:17:41.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.234+0000 7f588c3a7700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58840caa70 con 0x7f5884144020 2023-05-20T01:17:41.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.234+0000 7f5889942700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58840cec60 0x7f58840ca070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:41.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.234+0000 7f5889942700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58840cec60 0x7f58840ca070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59998/0 (socket says 172.21.15.16:59998) 2023-05-20T01:17:41.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.234+0000 7f5889942700 1 -- 172.21.15.16:0/3994360013 learned_addr learned my addr 172.21.15.16:0/3994360013 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:41.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.234+0000 7f588a944700 1 --2- 172.21.15.16:0/3994360013 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5884144020 0x7f58840c8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:41.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.234+0000 7f588a143700 1 --2- 172.21.15.16:0/3994360013 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58840b2760 0x7f5884154960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:41.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.235+0000 7f5889942700 1 -- 172.21.15.16:0/3994360013 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5884144020 msgr2=0x7f58840c8160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:41.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.235+0000 7f5889942700 1 --2- 172.21.15.16:0/3994360013 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5884144020 0x7f58840c8160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:41.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.235+0000 7f5889942700 1 -- 172.21.15.16:0/3994360013 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58840b2760 msgr2=0x7f5884154960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:41.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.235+0000 7f5889942700 1 --2- 172.21.15.16:0/3994360013 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58840b2760 0x7f5884154960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:41.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.235+0000 7f5889942700 1 -- 172.21.15.16:0/3994360013 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f587c009b30 con 0x7f58840cec60 2023-05-20T01:17:41.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.235+0000 7f5889942700 1 --2- 172.21.15.16:0/3994360013 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58840cec60 0x7f58840ca070 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f587400e840 tx=0x7f587400eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:41.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.235+0000 7f587affd700 1 -- 172.21.15.16:0/3994360013 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f587400c070 con 0x7f58840cec60 2023-05-20T01:17:41.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.235+0000 7f587affd700 1 -- 172.21.15.16:0/3994360013 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5874013610 con 0x7f58840cec60 2023-05-20T01:17:41.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.235+0000 7f588c3a7700 1 -- 172.21.15.16:0/3994360013 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58840cbeb0 con 0x7f58840cec60 2023-05-20T01:17:41.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.235+0000 7f587affd700 1 -- 172.21.15.16:0/3994360013 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58740139c0 con 0x7f58840cec60 2023-05-20T01:17:41.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.235+0000 7f588c3a7700 1 -- 172.21.15.16:0/3994360013 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58840cbbd0 con 0x7f58840cec60 2023-05-20T01:17:41.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.236+0000 7f588c3a7700 1 -- 172.21.15.16:0/3994360013 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58840cc190 con 0x7f58840cec60 2023-05-20T01:17:41.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.236+0000 7f587affd700 1 -- 172.21.15.16:0/3994360013 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5874013b20 con 0x7f58840cec60 2023-05-20T01:17:41.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.237+0000 7f587affd700 1 --2- 172.21.15.16:0/3994360013 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f586806ace0 0x7f586806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:41.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.237+0000 7f587affd700 1 -- 172.21.15.16:0/3994360013 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f5874089080 con 0x7f58840cec60 2023-05-20T01:17:41.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.237+0000 7f588a143700 1 --2- 172.21.15.16:0/3994360013 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f586806ace0 0x7f586806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:41.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.237+0000 7f588a143700 1 --2- 172.21.15.16:0/3994360013 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f586806ace0 0x7f586806d190 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f5880007fd0 tx=0x7f5880009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:41.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.241+0000 7f587affd700 1 -- 172.21.15.16:0/3994360013 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f5874057dd0 con 0x7f58840cec60 2023-05-20T01:17:41.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.521+0000 7f588c3a7700 1 -- 172.21.15.16:0/3994360013 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f58840747b0 con 0x7f58840cec60 2023-05-20T01:17:41.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.524+0000 7f587affd700 1 -- 172.21.15.16:0/3994360013 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v5) v1 ==== 90+0+0 (secure 0 0 0) 0x7f5874057960 con 0x7f58840cec60 2023-05-20T01:17:41.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.526+0000 7f588c3a7700 1 -- 172.21.15.16:0/3994360013 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f586806ace0 msgr2=0x7f586806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:41.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.526+0000 7f588c3a7700 1 --2- 172.21.15.16:0/3994360013 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f586806ace0 0x7f586806d190 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f5880007fd0 tx=0x7f5880009040 comp rx=0 tx=0).stop 2023-05-20T01:17:41.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.526+0000 7f588c3a7700 1 -- 172.21.15.16:0/3994360013 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58840cec60 msgr2=0x7f58840ca070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:41.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.526+0000 7f588c3a7700 1 --2- 172.21.15.16:0/3994360013 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58840cec60 0x7f58840ca070 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f587400e840 tx=0x7f587400eb50 comp rx=0 tx=0).stop 2023-05-20T01:17:41.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.527+0000 7f588c3a7700 1 -- 172.21.15.16:0/3994360013 shutdown_connections 2023-05-20T01:17:41.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.527+0000 7f588c3a7700 1 --2- 172.21.15.16:0/3994360013 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f586806ace0 0x7f586806d190 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:41.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.527+0000 7f588c3a7700 1 --2- 172.21.15.16:0/3994360013 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5884144020 0x7f58840c8160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:41.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.527+0000 7f588c3a7700 1 --2- 172.21.15.16:0/3994360013 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58840b2760 0x7f5884154960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:41.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.527+0000 7f588c3a7700 1 --2- 172.21.15.16:0/3994360013 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58840cec60 0x7f58840ca070 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:41.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.527+0000 7f588c3a7700 1 -- 172.21.15.16:0/3994360013 >> 172.21.15.16:0/3994360013 conn(0x7f588406ae30 msgr2=0x7f58840d0b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:41.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.527+0000 7f588c3a7700 1 -- 172.21.15.16:0/3994360013 shutdown_connections 2023-05-20T01:17:41.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.527+0000 7f588c3a7700 1 -- 172.21.15.16:0/3994360013 wait complete. 2023-05-20T01:17:41.542 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2023-05-20T01:17:41.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.664+0000 7f3642ea9700 1 Processor -- start 2023-05-20T01:17:41.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.665+0000 7f3642ea9700 1 -- start start 2023-05-20T01:17:41.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.665+0000 7f3642ea9700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f363c0cc9c0 0x7f363c06a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:41.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.665+0000 7f3642ea9700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f363c0ccf40 con 0x7f363c1585d0 2023-05-20T01:17:41.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.665+0000 7f3642ea9700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f363c0cd220 con 0x7f363c072170 2023-05-20T01:17:41.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.665+0000 7f3642ea9700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f363c15e540 con 0x7f363c0cc9c0 2023-05-20T01:17:41.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.665+0000 7f3640c45700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f363c1585d0 0x7f363c15ae20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:43302/0 (socket says 172.21.15.16:43302) 2023-05-20T01:17:41.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.665+0000 7f3640c45700 1 -- 172.21.15.16:0/2870167435 learned_addr learned my addr 172.21.15.16:0/2870167435 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:17:41.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.666+0000 7f363b7fe700 1 -- 172.21.15.16:0/2870167435 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3613910349 0 0) 0x7f363c0ccf40 con 0x7f363c1585d0 2023-05-20T01:17:41.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.666+0000 7f363b7fe700 1 -- 172.21.15.16:0/2870167435 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f362c0035b0 con 0x7f363c1585d0 2023-05-20T01:17:41.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.666+0000 7f363bfff700 1 --2- 172.21.15.16:0/2870167435 >> v2:172.21.15.132:3300/0 conn(0x7f363c0cc9c0 0x7f363c06a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:41.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.666+0000 7f363b7fe700 1 -- 172.21.15.16:0/2870167435 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 916152178 0 0) 0x7f362c0035b0 con 0x7f363c1585d0 2023-05-20T01:17:41.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.666+0000 7f363b7fe700 1 -- 172.21.15.16:0/2870167435 >> v2:172.21.15.132:3300/0 conn(0x7f363c0cc9c0 msgr2=0x7f363c06a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:41.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.666+0000 7f363b7fe700 1 --2- 172.21.15.16:0/2870167435 >> v2:172.21.15.132:3300/0 conn(0x7f363c0cc9c0 0x7f363c06a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:41.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.666+0000 7f363b7fe700 1 -- 172.21.15.16:0/2870167435 >> v1:172.21.15.132:6789/0 conn(0x7f363c072170 legacy=0x7f363c15e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:41.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.666+0000 7f363b7fe700 1 -- 172.21.15.16:0/2870167435 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f363c0cd500 con 0x7f363c1585d0 2023-05-20T01:17:41.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.666+0000 7f363b7fe700 1 -- 172.21.15.16:0/2870167435 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7f3630001670 con 0x7f363c1585d0 2023-05-20T01:17:41.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.666+0000 7f363b7fe700 1 -- 172.21.15.16:0/2870167435 >> v1:172.21.15.16:6789/0 conn(0x7f363c1585d0 legacy=0x7f363c15ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:41.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.666+0000 7f363b7fe700 1 --2- 172.21.15.16:0/2870167435 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f362c003a70 0x7f362c003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:41.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.666+0000 7f363b7fe700 1 --2- 172.21.15.16:0/2870167435 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f362c004850 0x7f362c004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:41.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.666+0000 7f363b7fe700 1 --2- 172.21.15.16:0/2870167435 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f362c005200 0x7f362c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:41.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.666+0000 7f363b7fe700 1 -- 172.21.15.16:0/2870167435 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f363c15e540 con 0x7f362c005200 2023-05-20T01:17:41.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.666+0000 7f363b7fe700 1 -- 172.21.15.16:0/2870167435 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f362c00c0e0 con 0x7f362c004850 2023-05-20T01:17:41.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.666+0000 7f363b7fe700 1 -- 172.21.15.16:0/2870167435 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f362c00c2a0 con 0x7f362c003a70 2023-05-20T01:17:41.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.666+0000 7f3641446700 1 --2- 172.21.15.16:0/2870167435 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f362c005200 0x7f362c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:41.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.666+0000 7f3640c45700 1 --2- 172.21.15.16:0/2870167435 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f362c003a70 0x7f362c003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:41.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.666+0000 7f363bfff700 1 --2- 172.21.15.16:0/2870167435 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f362c004850 0x7f362c004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:41.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.667+0000 7f3641446700 1 -- 172.21.15.16:0/2870167435 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f362c003a70 msgr2=0x7f362c003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:41.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.667+0000 7f3641446700 1 --2- 172.21.15.16:0/2870167435 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f362c003a70 0x7f362c003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:41.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.667+0000 7f3641446700 1 -- 172.21.15.16:0/2870167435 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f362c004850 msgr2=0x7f362c004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:41.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.667+0000 7f3641446700 1 --2- 172.21.15.16:0/2870167435 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f362c004850 0x7f362c004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:41.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.667+0000 7f3641446700 1 -- 172.21.15.16:0/2870167435 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f362c0035b0 con 0x7f362c005200 2023-05-20T01:17:41.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.667+0000 7f3641446700 1 -- 172.21.15.16:0/2870167435 reap_dead start 2023-05-20T01:17:41.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.667+0000 7f3641446700 1 --2- 172.21.15.16:0/2870167435 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f362c005200 0x7f362c00ba10 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f363400c190 tx=0x7f362c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aea7d1c45cdcf0a5 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:41.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.667+0000 7f363b7fe700 1 -- 172.21.15.16:0/2870167435 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3634001db0 con 0x7f362c005200 2023-05-20T01:17:41.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.667+0000 7f363b7fe700 1 -- 172.21.15.16:0/2870167435 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f363400ccf0 con 0x7f362c005200 2023-05-20T01:17:41.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.667+0000 7f363b7fe700 1 -- 172.21.15.16:0/2870167435 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3634003c40 con 0x7f362c005200 2023-05-20T01:17:41.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.667+0000 7f3642ea9700 1 -- 172.21.15.16:0/2870167435 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f362c005200 msgr2=0x7f362c00ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:41.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.667+0000 7f3642ea9700 1 --2- 172.21.15.16:0/2870167435 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f362c005200 0x7f362c00ba10 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f363400c190 tx=0x7f362c005880 comp rx=0 tx=0).stop 2023-05-20T01:17:41.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.667+0000 7f3642ea9700 1 -- 172.21.15.16:0/2870167435 shutdown_connections 2023-05-20T01:17:41.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.667+0000 7f3642ea9700 1 --2- 172.21.15.16:0/2870167435 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f362c005200 0x7f362c00ba10 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:41.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.667+0000 7f3642ea9700 1 -- 172.21.15.16:0/2870167435 >> 172.21.15.16:0/2870167435 conn(0x7f363c0cffb0 msgr2=0x7f363c0d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:41.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.667+0000 7f3642ea9700 1 -- 172.21.15.16:0/2870167435 shutdown_connections 2023-05-20T01:17:41.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.667+0000 7f3642ea9700 1 -- 172.21.15.16:0/2870167435 wait complete. 2023-05-20T01:17:41.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.669+0000 7f3642ea9700 1 Processor -- start 2023-05-20T01:17:41.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.669+0000 7f3642ea9700 1 -- start start 2023-05-20T01:17:41.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.670+0000 7f3642ea9700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f362c005200 0x7f363c14ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:41.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.670+0000 7f3642ea9700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f363c1576a0 0x7f363c0ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:41.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.670+0000 7f3642ea9700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f363c1e4900 0x7f363c1e4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:41.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.670+0000 7f3642ea9700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f363c1e52b0 con 0x7f363c1576a0 2023-05-20T01:17:41.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.670+0000 7f3642ea9700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f363c157210 con 0x7f363c1e4900 2023-05-20T01:17:41.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.670+0000 7f3642ea9700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f363c14b5b0 con 0x7f362c005200 2023-05-20T01:17:41.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.670+0000 7f363bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f363c1576a0 0x7f363c0ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:41.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.670+0000 7f363bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f363c1576a0 0x7f363c0ceb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60020/0 (socket says 172.21.15.16:60020) 2023-05-20T01:17:41.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.670+0000 7f363bfff700 1 -- 172.21.15.16:0/354782650 learned_addr learned my addr 172.21.15.16:0/354782650 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:41.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.670+0000 7f3640c45700 1 --2- 172.21.15.16:0/354782650 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f362c005200 0x7f363c14ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:41.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.670+0000 7f3641446700 1 --2- 172.21.15.16:0/354782650 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f363c1e4900 0x7f363c1e4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:41.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.670+0000 7f363bfff700 1 -- 172.21.15.16:0/354782650 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f362c005200 msgr2=0x7f363c14ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:41.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.670+0000 7f363bfff700 1 --2- 172.21.15.16:0/354782650 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f362c005200 0x7f363c14ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:41.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.670+0000 7f363bfff700 1 -- 172.21.15.16:0/354782650 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f363c1e4900 msgr2=0x7f363c1e4d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:41.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.670+0000 7f363bfff700 1 --2- 172.21.15.16:0/354782650 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f363c1e4900 0x7f363c1e4d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:41.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.670+0000 7f363bfff700 1 -- 172.21.15.16:0/354782650 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f363c0719a0 con 0x7f363c1576a0 2023-05-20T01:17:41.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.670+0000 7f3641446700 1 --2- 172.21.15.16:0/354782650 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f363c1e4900 0x7f363c1e4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:17:41.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.670+0000 7f3640c45700 1 --2- 172.21.15.16:0/354782650 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f362c005200 0x7f363c14ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:17:41.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.670+0000 7f363bfff700 1 --2- 172.21.15.16:0/354782650 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f363c1576a0 0x7f363c0ceb80 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f36280119d0 tx=0x7f3628011d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:41.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.670+0000 7f36397fa700 1 -- 172.21.15.16:0/354782650 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f362800f940 con 0x7f363c1576a0 2023-05-20T01:17:41.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.670+0000 7f36397fa700 1 -- 172.21.15.16:0/354782650 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f362800faa0 con 0x7f363c1576a0 2023-05-20T01:17:41.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.670+0000 7f3642ea9700 1 -- 172.21.15.16:0/354782650 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f363c0cd7e0 con 0x7f363c1576a0 2023-05-20T01:17:41.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.670+0000 7f36397fa700 1 -- 172.21.15.16:0/354782650 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f362800fe30 con 0x7f363c1576a0 2023-05-20T01:17:41.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.671+0000 7f3642ea9700 1 -- 172.21.15.16:0/354782650 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f363c15ae20 con 0x7f363c1576a0 2023-05-20T01:17:41.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.671+0000 7f3642ea9700 1 -- 172.21.15.16:0/354782650 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f363c0747b0 con 0x7f363c1576a0 2023-05-20T01:17:41.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.672+0000 7f36397fa700 1 -- 172.21.15.16:0/354782650 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3628014440 con 0x7f363c1576a0 2023-05-20T01:17:41.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.672+0000 7f36397fa700 1 --2- 172.21.15.16:0/354782650 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f362006ace0 0x7f362006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:41.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.672+0000 7f36397fa700 1 -- 172.21.15.16:0/354782650 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f3628089960 con 0x7f363c1576a0 2023-05-20T01:17:41.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.672+0000 7f3640c45700 1 --2- 172.21.15.16:0/354782650 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f362006ace0 0x7f362006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:41.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.673+0000 7f3640c45700 1 --2- 172.21.15.16:0/354782650 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f362006ace0 0x7f362006d190 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3630001eb0 tx=0x7f3630009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:41.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.677+0000 7f36397fa700 1 -- 172.21.15.16:0/354782650 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f3628054c20 con 0x7f363c1576a0 2023-05-20T01:17:41.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.955+0000 7f3642ea9700 1 -- 172.21.15.16:0/354782650 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f363c157350 con 0x7f363c1576a0 2023-05-20T01:17:41.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.959+0000 7f36397fa700 1 -- 172.21.15.16:0/354782650 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v6) v1 ==== 90+0+0 (secure 0 0 0) 0x7f363c157350 con 0x7f363c1576a0 2023-05-20T01:17:41.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.961+0000 7f3642ea9700 1 -- 172.21.15.16:0/354782650 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f362006ace0 msgr2=0x7f362006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:41.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.961+0000 7f3642ea9700 1 --2- 172.21.15.16:0/354782650 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f362006ace0 0x7f362006d190 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3630001eb0 tx=0x7f3630009040 comp rx=0 tx=0).stop 2023-05-20T01:17:41.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.961+0000 7f3642ea9700 1 -- 172.21.15.16:0/354782650 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f363c1576a0 msgr2=0x7f363c0ceb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:41.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.961+0000 7f3642ea9700 1 --2- 172.21.15.16:0/354782650 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f363c1576a0 0x7f363c0ceb80 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f36280119d0 tx=0x7f3628011d90 comp rx=0 tx=0).stop 2023-05-20T01:17:41.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.961+0000 7f3642ea9700 1 -- 172.21.15.16:0/354782650 shutdown_connections 2023-05-20T01:17:41.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.961+0000 7f3642ea9700 1 --2- 172.21.15.16:0/354782650 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f363c1e4900 0x7f363c1e4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:41.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.961+0000 7f3642ea9700 1 --2- 172.21.15.16:0/354782650 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f362006ace0 0x7f362006d190 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:41.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.961+0000 7f3642ea9700 1 --2- 172.21.15.16:0/354782650 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f362c005200 0x7f363c14ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:41.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.961+0000 7f3642ea9700 1 --2- 172.21.15.16:0/354782650 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f363c1576a0 0x7f363c0ceb80 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:41.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.961+0000 7f3642ea9700 1 -- 172.21.15.16:0/354782650 >> 172.21.15.16:0/354782650 conn(0x7f363c0cffb0 msgr2=0x7f363c14d330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:41.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.961+0000 7f3642ea9700 1 -- 172.21.15.16:0/354782650 shutdown_connections 2023-05-20T01:17:41.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:41.961+0000 7f3642ea9700 1 -- 172.21.15.16:0/354782650 wait complete. 2023-05-20T01:17:41.977 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2023-05-20T01:17:42.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.099+0000 7f379a1e1700 1 Processor -- start 2023-05-20T01:17:42.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.099+0000 7f379a1e1700 1 -- start start 2023-05-20T01:17:42.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.099+0000 7f379a1e1700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f379406a510 0x7f379415e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:42.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.099+0000 7f379a1e1700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f379406a8e0 0x7f37940cf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:42.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.099+0000 7f379a1e1700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f37940cfb30 0x7f3794071340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:42.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.099+0000 7f379a1e1700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f379415e060 con 0x7f379406a510 2023-05-20T01:17:42.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.099+0000 7f379a1e1700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f379415d450 con 0x7f37940cfb30 2023-05-20T01:17:42.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.099+0000 7f379a1e1700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f379415d730 con 0x7f379406a8e0 2023-05-20T01:17:42.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.100+0000 7f37937fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f379406a510 0x7f379415e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:42.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.100+0000 7f37937fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f379406a510 0x7f379415e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60026/0 (socket says 172.21.15.16:60026) 2023-05-20T01:17:42.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.100+0000 7f37937fe700 1 -- 172.21.15.16:0/50647407 learned_addr learned my addr 172.21.15.16:0/50647407 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:42.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.100+0000 7f3792ffd700 1 --2- 172.21.15.16:0/50647407 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f379406a8e0 0x7f37940cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:42.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.100+0000 7f3793fff700 1 --2- 172.21.15.16:0/50647407 >> v2:172.21.15.132:3300/0 conn(0x7f37940cfb30 0x7f3794071340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:42.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.100+0000 7f37937fe700 1 -- 172.21.15.16:0/50647407 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f379406a8e0 msgr2=0x7f37940cf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:42.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.100+0000 7f37937fe700 1 --2- 172.21.15.16:0/50647407 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f379406a8e0 0x7f37940cf5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:42.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.100+0000 7f37937fe700 1 -- 172.21.15.16:0/50647407 >> v2:172.21.15.132:3300/0 conn(0x7f37940cfb30 msgr2=0x7f3794071340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:42.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.100+0000 7f37937fe700 1 --2- 172.21.15.16:0/50647407 >> v2:172.21.15.132:3300/0 conn(0x7f37940cfb30 0x7f3794071340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:42.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.100+0000 7f37937fe700 1 -- 172.21.15.16:0/50647407 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f379415da10 con 0x7f379406a510 2023-05-20T01:17:42.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.100+0000 7f37937fe700 1 --2- 172.21.15.16:0/50647407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f379406a510 0x7f379415e2a0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f37940b24b0 tx=0x7f3784009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba22b96692ebd73f server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:42.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.100+0000 7f37927fc700 1 -- 172.21.15.16:0/50647407 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f378400b870 con 0x7f379406a510 2023-05-20T01:17:42.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.100+0000 7f37927fc700 1 -- 172.21.15.16:0/50647407 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f378400b9d0 con 0x7f379406a510 2023-05-20T01:17:42.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.100+0000 7f37927fc700 1 -- 172.21.15.16:0/50647407 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f378400bd60 con 0x7f379406a510 2023-05-20T01:17:42.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.101+0000 7f379a1e1700 1 -- 172.21.15.16:0/50647407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f379406a510 msgr2=0x7f379415e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:42.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.101+0000 7f379a1e1700 1 --2- 172.21.15.16:0/50647407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f379406a510 0x7f379415e2a0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f37940b24b0 tx=0x7f3784009bd0 comp rx=0 tx=0).stop 2023-05-20T01:17:42.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.101+0000 7f379a1e1700 1 -- 172.21.15.16:0/50647407 shutdown_connections 2023-05-20T01:17:42.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.101+0000 7f379a1e1700 1 --2- 172.21.15.16:0/50647407 >> v2:172.21.15.132:3300/0 conn(0x7f37940cfb30 0x7f3794071340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:42.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.101+0000 7f379a1e1700 1 --2- 172.21.15.16:0/50647407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f379406a510 0x7f379415e2a0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:42.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.101+0000 7f379a1e1700 1 --2- 172.21.15.16:0/50647407 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f379406a8e0 0x7f37940cf5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:42.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.101+0000 7f379a1e1700 1 -- 172.21.15.16:0/50647407 >> 172.21.15.16:0/50647407 conn(0x7f37940d0c30 msgr2=0x7f37940b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:42.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.101+0000 7f379a1e1700 1 -- 172.21.15.16:0/50647407 shutdown_connections 2023-05-20T01:17:42.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.101+0000 7f379a1e1700 1 -- 172.21.15.16:0/50647407 wait complete. 2023-05-20T01:17:42.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.102+0000 7f379a1e1700 1 Processor -- start 2023-05-20T01:17:42.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.102+0000 7f379a1e1700 1 -- start start 2023-05-20T01:17:42.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.103+0000 7f379a1e1700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f379406a510 0x7f3794153830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:42.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.103+0000 7f379a1e1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f379406a8e0 0x7f37940b42a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:42.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.103+0000 7f379a1e1700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f37940cfb30 0x7f37940b57f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:42.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.103+0000 7f379a1e1700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37940b47e0 con 0x7f379406a8e0 2023-05-20T01:17:42.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.103+0000 7f379a1e1700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37940b3200 con 0x7f37940cfb30 2023-05-20T01:17:42.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.103+0000 7f379a1e1700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37940b34e0 con 0x7f379406a510 2023-05-20T01:17:42.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.103+0000 7f3792ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f379406a8e0 0x7f37940b42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:42.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.103+0000 7f3792ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f379406a8e0 0x7f37940b42a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60034/0 (socket says 172.21.15.16:60034) 2023-05-20T01:17:42.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.103+0000 7f3792ffd700 1 -- 172.21.15.16:0/3235294434 learned_addr learned my addr 172.21.15.16:0/3235294434 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:42.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.103+0000 7f3793fff700 1 --2- 172.21.15.16:0/3235294434 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f37940cfb30 0x7f37940b57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:42.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.103+0000 7f37937fe700 1 --2- 172.21.15.16:0/3235294434 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f379406a510 0x7f3794153830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:42.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.103+0000 7f3792ffd700 1 -- 172.21.15.16:0/3235294434 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f379406a510 msgr2=0x7f3794153830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:42.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.103+0000 7f3792ffd700 1 --2- 172.21.15.16:0/3235294434 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f379406a510 0x7f3794153830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:42.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.103+0000 7f3792ffd700 1 -- 172.21.15.16:0/3235294434 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f37940cfb30 msgr2=0x7f37940b57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:42.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.103+0000 7f3792ffd700 1 --2- 172.21.15.16:0/3235294434 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f37940cfb30 0x7f37940b57f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:42.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.103+0000 7f3792ffd700 1 -- 172.21.15.16:0/3235294434 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3784009550 con 0x7f379406a8e0 2023-05-20T01:17:42.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.103+0000 7f3793fff700 1 --2- 172.21.15.16:0/3235294434 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f37940cfb30 0x7f37940b57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:17:42.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.104+0000 7f3792ffd700 1 --2- 172.21.15.16:0/3235294434 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f379406a8e0 0x7f37940b42a0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f378800e810 tx=0x7f378800ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:42.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.104+0000 7f377bfff700 1 -- 172.21.15.16:0/3235294434 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3788013610 con 0x7f379406a8e0 2023-05-20T01:17:42.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.104+0000 7f377bfff700 1 -- 172.21.15.16:0/3235294434 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3788013770 con 0x7f379406a8e0 2023-05-20T01:17:42.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.104+0000 7f379a1e1700 1 -- 172.21.15.16:0/3235294434 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f379406ad50 con 0x7f379406a8e0 2023-05-20T01:17:42.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.104+0000 7f377bfff700 1 -- 172.21.15.16:0/3235294434 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3788013b20 con 0x7f379406a8e0 2023-05-20T01:17:42.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.104+0000 7f379a1e1700 1 -- 172.21.15.16:0/3235294434 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f379415dcf0 con 0x7f379406a8e0 2023-05-20T01:17:42.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.105+0000 7f379a1e1700 1 -- 172.21.15.16:0/3235294434 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f379415d080 con 0x7f379406a8e0 2023-05-20T01:17:42.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.105+0000 7f377bfff700 1 -- 172.21.15.16:0/3235294434 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3788013c80 con 0x7f379406a8e0 2023-05-20T01:17:42.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.105+0000 7f377bfff700 1 --2- 172.21.15.16:0/3235294434 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f378006ace0 0x7f378006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:42.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.106+0000 7f377bfff700 1 -- 172.21.15.16:0/3235294434 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f3788088ea0 con 0x7f379406a8e0 2023-05-20T01:17:42.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.106+0000 7f37937fe700 1 --2- 172.21.15.16:0/3235294434 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f378006ace0 0x7f378006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:42.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.106+0000 7f37937fe700 1 --2- 172.21.15.16:0/3235294434 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f378006ace0 0x7f378006d190 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3784009c10 tx=0x7f3784008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:42.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.110+0000 7f377bfff700 1 -- 172.21.15.16:0/3235294434 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f3788057c00 con 0x7f379406a8e0 2023-05-20T01:17:42.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.391+0000 7f379a1e1700 1 -- 172.21.15.16:0/3235294434 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f37940747b0 con 0x7f379406a8e0 2023-05-20T01:17:42.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.395+0000 7f377bfff700 1 -- 172.21.15.16:0/3235294434 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v7) v1 ==== 97+0+0 (secure 0 0 0) 0x7f3788057790 con 0x7f379406a8e0 2023-05-20T01:17:42.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.397+0000 7f379a1e1700 1 -- 172.21.15.16:0/3235294434 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f378006ace0 msgr2=0x7f378006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:42.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.397+0000 7f379a1e1700 1 --2- 172.21.15.16:0/3235294434 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f378006ace0 0x7f378006d190 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3784009c10 tx=0x7f3784008030 comp rx=0 tx=0).stop 2023-05-20T01:17:42.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.397+0000 7f379a1e1700 1 -- 172.21.15.16:0/3235294434 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f379406a8e0 msgr2=0x7f37940b42a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:42.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.397+0000 7f379a1e1700 1 --2- 172.21.15.16:0/3235294434 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f379406a8e0 0x7f37940b42a0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f378800e810 tx=0x7f378800ebd0 comp rx=0 tx=0).stop 2023-05-20T01:17:42.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.397+0000 7f379a1e1700 1 -- 172.21.15.16:0/3235294434 shutdown_connections 2023-05-20T01:17:42.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.397+0000 7f379a1e1700 1 --2- 172.21.15.16:0/3235294434 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f37940cfb30 0x7f37940b57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:42.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.397+0000 7f379a1e1700 1 --2- 172.21.15.16:0/3235294434 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f378006ace0 0x7f378006d190 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:42.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.397+0000 7f379a1e1700 1 --2- 172.21.15.16:0/3235294434 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f379406a510 0x7f3794153830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:42.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.397+0000 7f379a1e1700 1 --2- 172.21.15.16:0/3235294434 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f379406a8e0 0x7f37940b42a0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:42.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.397+0000 7f379a1e1700 1 -- 172.21.15.16:0/3235294434 >> 172.21.15.16:0/3235294434 conn(0x7f37940d0c30 msgr2=0x7f37940cbeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:42.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.397+0000 7f379a1e1700 1 -- 172.21.15.16:0/3235294434 shutdown_connections 2023-05-20T01:17:42.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.397+0000 7f379a1e1700 1 -- 172.21.15.16:0/3235294434 wait complete. 2023-05-20T01:17:42.413 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2023-05-20T01:17:42.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.536+0000 7f67731da700 1 Processor -- start 2023-05-20T01:17:42.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.537+0000 7f67731da700 1 -- start start 2023-05-20T01:17:42.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.537+0000 7f67731da700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f676c0ccbd0 0x7f676c0cb000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:42.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.537+0000 7f67731da700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f676c0cd050 con 0x7f676c153eb0 2023-05-20T01:17:42.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.537+0000 7f67731da700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f676c0cb750 con 0x7f676c06cb90 2023-05-20T01:17:42.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.537+0000 7f67731da700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f676c159d30 con 0x7f676c0ccbd0 2023-05-20T01:17:42.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.537+0000 7f676bfff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f676c0ccbd0 0x7f676c0cb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:42.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.538+0000 7f6770f76700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f676c153eb0 0x7f676c15a740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:43336/0 (socket says 172.21.15.16:43336) 2023-05-20T01:17:42.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.538+0000 7f676bfff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f676c0ccbd0 0x7f676c0cb000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60042/0 (socket says 172.21.15.16:60042) 2023-05-20T01:17:42.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.538+0000 7f6770f76700 1 -- 172.21.15.16:0/4273974535 learned_addr learned my addr 172.21.15.16:0/4273974535 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:17:42.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.538+0000 7f676bfff700 1 -- 172.21.15.16:0/4273974535 >> v1:172.21.15.132:6789/0 conn(0x7f676c06cb90 legacy=0x7f676c06cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:42.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.538+0000 7f676bfff700 1 -- 172.21.15.16:0/4273974535 >> v1:172.21.15.16:6789/0 conn(0x7f676c153eb0 legacy=0x7f676c15a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:42.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.538+0000 7f676b7fe700 1 -- 172.21.15.16:0/4273974535 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2809349253 0 0) 0x7f676c0cd050 con 0x7f676c153eb0 2023-05-20T01:17:42.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.538+0000 7f676bfff700 1 -- 172.21.15.16:0/4273974535 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f676c0cba30 con 0x7f676c0ccbd0 2023-05-20T01:17:42.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.538+0000 7f676bfff700 1 --2- 172.21.15.16:0/4273974535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f676c0ccbd0 0x7f676c0cb000 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f6758009840 tx=0x7f6758009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4cf0c267714a168 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:42.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.538+0000 7f676b7fe700 1 -- 172.21.15.16:0/4273974535 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6758004230 con 0x7f676c0ccbd0 2023-05-20T01:17:42.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.538+0000 7f676b7fe700 1 -- 172.21.15.16:0/4273974535 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6758004390 con 0x7f676c0ccbd0 2023-05-20T01:17:42.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.538+0000 7f676b7fe700 1 -- 172.21.15.16:0/4273974535 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67580036a0 con 0x7f676c0ccbd0 2023-05-20T01:17:42.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.539+0000 7f67731da700 1 -- 172.21.15.16:0/4273974535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f676c0ccbd0 msgr2=0x7f676c0cb000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:42.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.539+0000 7f67731da700 1 --2- 172.21.15.16:0/4273974535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f676c0ccbd0 0x7f676c0cb000 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f6758009840 tx=0x7f6758009b50 comp rx=0 tx=0).stop 2023-05-20T01:17:42.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.539+0000 7f67731da700 1 -- 172.21.15.16:0/4273974535 shutdown_connections 2023-05-20T01:17:42.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.539+0000 7f67731da700 1 --2- 172.21.15.16:0/4273974535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f676c0ccbd0 0x7f676c0cb000 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:42.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.539+0000 7f67731da700 1 -- 172.21.15.16:0/4273974535 >> 172.21.15.16:0/4273974535 conn(0x7f676c0661b0 msgr2=0x7f676c066580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:42.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.539+0000 7f67731da700 1 -- 172.21.15.16:0/4273974535 shutdown_connections 2023-05-20T01:17:42.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.539+0000 7f67731da700 1 -- 172.21.15.16:0/4273974535 wait complete. 2023-05-20T01:17:42.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.540+0000 7f67731da700 1 Processor -- start 2023-05-20T01:17:42.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.540+0000 7f67731da700 1 -- start start 2023-05-20T01:17:42.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.541+0000 7f67731da700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f676c06cb90 0x7f676c13b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:42.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.541+0000 7f67731da700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f676c0ccbd0 0x7f676c0ce050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:42.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.541+0000 7f67731da700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f676c153eb0 0x7f676c13a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:42.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.541+0000 7f67731da700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f676c13c380 con 0x7f676c0ccbd0 2023-05-20T01:17:42.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.541+0000 7f67731da700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f676c13b770 con 0x7f676c153eb0 2023-05-20T01:17:42.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.541+0000 7f67731da700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f676c13ba50 con 0x7f676c06cb90 2023-05-20T01:17:42.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.541+0000 7f676bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f676c0ccbd0 0x7f676c0ce050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:42.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.541+0000 7f676bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f676c0ccbd0 0x7f676c0ce050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60048/0 (socket says 172.21.15.16:60048) 2023-05-20T01:17:42.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.541+0000 7f676bfff700 1 -- 172.21.15.16:0/1035854129 learned_addr learned my addr 172.21.15.16:0/1035854129 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:42.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.541+0000 7f6770f76700 1 --2- 172.21.15.16:0/1035854129 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f676c06cb90 0x7f676c13b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:42.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.541+0000 7f6771777700 1 --2- 172.21.15.16:0/1035854129 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f676c153eb0 0x7f676c13a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:42.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.541+0000 7f676bfff700 1 -- 172.21.15.16:0/1035854129 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f676c06cb90 msgr2=0x7f676c13b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:42.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.541+0000 7f676bfff700 1 --2- 172.21.15.16:0/1035854129 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f676c06cb90 0x7f676c13b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:42.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.541+0000 7f676bfff700 1 -- 172.21.15.16:0/1035854129 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f676c153eb0 msgr2=0x7f676c13a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:42.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.541+0000 7f676bfff700 1 --2- 172.21.15.16:0/1035854129 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f676c153eb0 0x7f676c13a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:42.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.541+0000 7f676bfff700 1 -- 172.21.15.16:0/1035854129 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67580094a0 con 0x7f676c0ccbd0 2023-05-20T01:17:42.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.541+0000 7f6770f76700 1 --2- 172.21.15.16:0/1035854129 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f676c06cb90 0x7f676c13b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:17:42.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.541+0000 7f6771777700 1 --2- 172.21.15.16:0/1035854129 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f676c153eb0 0x7f676c13a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:17:42.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.542+0000 7f676bfff700 1 --2- 172.21.15.16:0/1035854129 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f676c0ccbd0 0x7f676c0ce050 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f676c06d8f0 tx=0x7f6758004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:42.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.542+0000 7f67697fa700 1 -- 172.21.15.16:0/1035854129 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6758011630 con 0x7f676c0ccbd0 2023-05-20T01:17:42.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.542+0000 7f67697fa700 1 -- 172.21.15.16:0/1035854129 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f675801e070 con 0x7f676c0ccbd0 2023-05-20T01:17:42.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.542+0000 7f67731da700 1 -- 172.21.15.16:0/1035854129 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f676c159f40 con 0x7f676c0ccbd0 2023-05-20T01:17:42.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.542+0000 7f67697fa700 1 -- 172.21.15.16:0/1035854129 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67580119f0 con 0x7f676c0ccbd0 2023-05-20T01:17:42.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.542+0000 7f67731da700 1 -- 172.21.15.16:0/1035854129 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f676c0cbd10 con 0x7f676c0ccbd0 2023-05-20T01:17:42.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.543+0000 7f67731da700 1 -- 172.21.15.16:0/1035854129 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f676c15a220 con 0x7f676c0ccbd0 2023-05-20T01:17:42.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.543+0000 7f67697fa700 1 -- 172.21.15.16:0/1035854129 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6758011b50 con 0x7f676c0ccbd0 2023-05-20T01:17:42.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.543+0000 7f67697fa700 1 --2- 172.21.15.16:0/1035854129 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f675006ace0 0x7f675006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:42.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.544+0000 7f67697fa700 1 -- 172.21.15.16:0/1035854129 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f675808a740 con 0x7f676c0ccbd0 2023-05-20T01:17:42.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.544+0000 7f6770f76700 1 --2- 172.21.15.16:0/1035854129 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f675006ace0 0x7f675006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:42.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.544+0000 7f6770f76700 1 --2- 172.21.15.16:0/1035854129 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f675006ace0 0x7f675006d190 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f6760000f60 tx=0x7f6760009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:42.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.548+0000 7f67697fa700 1 -- 172.21.15.16:0/1035854129 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f6758055960 con 0x7f676c0ccbd0 2023-05-20T01:17:42.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.830+0000 7f67731da700 1 -- 172.21.15.16:0/1035854129 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f676c0747b0 con 0x7f676c0ccbd0 2023-05-20T01:17:42.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.834+0000 7f67697fa700 1 -- 172.21.15.16:0/1035854129 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7f6758058f80 con 0x7f676c0ccbd0 2023-05-20T01:17:42.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.836+0000 7f67731da700 1 -- 172.21.15.16:0/1035854129 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f675006ace0 msgr2=0x7f675006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:42.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.836+0000 7f67731da700 1 --2- 172.21.15.16:0/1035854129 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f675006ace0 0x7f675006d190 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f6760000f60 tx=0x7f6760009040 comp rx=0 tx=0).stop 2023-05-20T01:17:42.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.836+0000 7f67731da700 1 -- 172.21.15.16:0/1035854129 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f676c0ccbd0 msgr2=0x7f676c0ce050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:42.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.836+0000 7f67731da700 1 --2- 172.21.15.16:0/1035854129 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f676c0ccbd0 0x7f676c0ce050 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f676c06d8f0 tx=0x7f6758004750 comp rx=0 tx=0).stop 2023-05-20T01:17:42.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.836+0000 7f67731da700 1 -- 172.21.15.16:0/1035854129 shutdown_connections 2023-05-20T01:17:42.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.836+0000 7f67731da700 1 --2- 172.21.15.16:0/1035854129 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f676c153eb0 0x7f676c13a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:42.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.836+0000 7f67731da700 1 --2- 172.21.15.16:0/1035854129 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f675006ace0 0x7f675006d190 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:42.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.836+0000 7f67731da700 1 --2- 172.21.15.16:0/1035854129 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f676c06cb90 0x7f676c13b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:42.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.836+0000 7f67731da700 1 --2- 172.21.15.16:0/1035854129 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f676c0ccbd0 0x7f676c0ce050 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:42.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.836+0000 7f67731da700 1 -- 172.21.15.16:0/1035854129 >> 172.21.15.16:0/1035854129 conn(0x7f676c0661b0 msgr2=0x7f676c071b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:42.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.836+0000 7f67731da700 1 -- 172.21.15.16:0/1035854129 shutdown_connections 2023-05-20T01:17:42.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.836+0000 7f67731da700 1 -- 172.21.15.16:0/1035854129 wait complete. 2023-05-20T01:17:42.852 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-20T01:17:42.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.973+0000 7f43e9dcd700 1 Processor -- start 2023-05-20T01:17:42.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.973+0000 7f43e9dcd700 1 -- start start 2023-05-20T01:17:42.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.974+0000 7f43e9dcd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f43e40cfc50 0x7f43e40d2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:42.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.974+0000 7f43e9dcd700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f43e40cd970 0x7f43e40cddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:42.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.974+0000 7f43e9dcd700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43e40b3b50 con 0x7f43e40b4750 2023-05-20T01:17:42.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.974+0000 7f43e9dcd700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43e40cfad0 con 0x7f43e40cfc50 2023-05-20T01:17:42.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.974+0000 7f43e9dcd700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43e40d3df0 con 0x7f43e40cd970 2023-05-20T01:17:42.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.974+0000 7f43e37fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f43e40cfc50 0x7f43e40d2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:42.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.974+0000 7f43e37fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f43e40cfc50 0x7f43e40d2c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60054/0 (socket says 172.21.15.16:60054) 2023-05-20T01:17:42.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.974+0000 7f43e37fe700 1 -- 172.21.15.16:0/2787036172 learned_addr learned my addr 172.21.15.16:0/2787036172 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:42.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.975+0000 7f43e3fff700 1 --2- 172.21.15.16:0/2787036172 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f43e40cd970 0x7f43e40cddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:42.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.975+0000 7f43e37fe700 1 -- 172.21.15.16:0/2787036172 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f43e40cd970 msgr2=0x7f43e40cddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:42.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.975+0000 7f43e37fe700 1 --2- 172.21.15.16:0/2787036172 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f43e40cd970 0x7f43e40cddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:42.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.975+0000 7f43e37fe700 1 -- 172.21.15.16:0/2787036172 >> v1:172.21.15.132:6789/0 conn(0x7f43e40b4750 legacy=0x7f43e40d0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:42.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.975+0000 7f43e37fe700 1 -- 172.21.15.16:0/2787036172 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43e40d0b90 con 0x7f43e40cfc50 2023-05-20T01:17:42.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.975+0000 7f43e37fe700 1 --2- 172.21.15.16:0/2787036172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43e40cfc50 0x7f43e40d2c10 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f43cc0098f0 tx=0x7f43cc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f6f691598fc0ada server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:42.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.975+0000 7f43e27fc700 1 -- 172.21.15.16:0/2787036172 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43cc00b8a0 con 0x7f43e40cfc50 2023-05-20T01:17:42.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.975+0000 7f43e27fc700 1 -- 172.21.15.16:0/2787036172 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43cc00ba00 con 0x7f43e40cfc50 2023-05-20T01:17:42.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.975+0000 7f43e27fc700 1 -- 172.21.15.16:0/2787036172 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43cc00bd90 con 0x7f43e40cfc50 2023-05-20T01:17:42.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.976+0000 7f43e9dcd700 1 -- 172.21.15.16:0/2787036172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43e40cfc50 msgr2=0x7f43e40d2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:42.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.976+0000 7f43e9dcd700 1 --2- 172.21.15.16:0/2787036172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43e40cfc50 0x7f43e40d2c10 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f43cc0098f0 tx=0x7f43cc009c00 comp rx=0 tx=0).stop 2023-05-20T01:17:42.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.976+0000 7f43e9dcd700 1 -- 172.21.15.16:0/2787036172 shutdown_connections 2023-05-20T01:17:42.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.976+0000 7f43e9dcd700 1 --2- 172.21.15.16:0/2787036172 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f43e40cd970 0x7f43e40cddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:42.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.976+0000 7f43e9dcd700 1 --2- 172.21.15.16:0/2787036172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43e40cfc50 0x7f43e40d2c10 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:42.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.976+0000 7f43e9dcd700 1 -- 172.21.15.16:0/2787036172 >> 172.21.15.16:0/2787036172 conn(0x7f43e409c590 msgr2=0x7f43e40d3240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:42.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.976+0000 7f43e9dcd700 1 -- 172.21.15.16:0/2787036172 shutdown_connections 2023-05-20T01:17:42.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.976+0000 7f43e9dcd700 1 -- 172.21.15.16:0/2787036172 wait complete. 2023-05-20T01:17:42.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.977+0000 7f43e9dcd700 1 Processor -- start 2023-05-20T01:17:42.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.977+0000 7f43e9dcd700 1 -- start start 2023-05-20T01:17:42.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.978+0000 7f43e9dcd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43e40b4750 0x7f43e414b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:42.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.978+0000 7f43e9dcd700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f43e40cd970 0x7f43e40cbd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:42.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.978+0000 7f43e9dcd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f43e40cfc50 0x7f43e40b6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:42.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.978+0000 7f43e9dcd700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43e40b7060 con 0x7f43e40b4750 2023-05-20T01:17:42.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.978+0000 7f43e9dcd700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43e40b7b80 con 0x7f43e40cfc50 2023-05-20T01:17:42.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.978+0000 7f43e9dcd700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43e40b6aa0 con 0x7f43e40cd970 2023-05-20T01:17:42.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.978+0000 7f43e37fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43e40b4750 0x7f43e414b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:42.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.978+0000 7f43e37fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43e40b4750 0x7f43e414b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60060/0 (socket says 172.21.15.16:60060) 2023-05-20T01:17:42.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.978+0000 7f43e37fe700 1 -- 172.21.15.16:0/4150481882 learned_addr learned my addr 172.21.15.16:0/4150481882 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:42.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.978+0000 7f43e37fe700 1 -- 172.21.15.16:0/4150481882 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f43e40cd970 msgr2=0x7f43e40cbd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:42.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.978+0000 7f43e37fe700 1 --2- 172.21.15.16:0/4150481882 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f43e40cd970 0x7f43e40cbd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:42.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.978+0000 7f43e37fe700 1 -- 172.21.15.16:0/4150481882 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f43e40cfc50 msgr2=0x7f43e40b6070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:42.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.978+0000 7f43e37fe700 1 --2- 172.21.15.16:0/4150481882 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f43e40cfc50 0x7f43e40b6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:42.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.978+0000 7f43e37fe700 1 -- 172.21.15.16:0/4150481882 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43cc009550 con 0x7f43e40b4750 2023-05-20T01:17:42.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.978+0000 7f43e37fe700 1 --2- 172.21.15.16:0/4150481882 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43e40b4750 0x7f43e414b2c0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f43cc014040 tx=0x7f43cc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:42.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.978+0000 7f43dbfff700 1 -- 172.21.15.16:0/4150481882 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43cc0036a0 con 0x7f43e40b4750 2023-05-20T01:17:42.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.979+0000 7f43dbfff700 1 -- 172.21.15.16:0/4150481882 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43cc01e070 con 0x7f43e40b4750 2023-05-20T01:17:42.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.979+0000 7f43e9dcd700 1 -- 172.21.15.16:0/4150481882 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43e40b4380 con 0x7f43e40b4750 2023-05-20T01:17:42.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.979+0000 7f43dbfff700 1 -- 172.21.15.16:0/4150481882 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43cc003a60 con 0x7f43e40b4750 2023-05-20T01:17:42.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.979+0000 7f43e9dcd700 1 -- 172.21.15.16:0/4150481882 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43e415d0b0 con 0x7f43e40b4750 2023-05-20T01:17:42.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.980+0000 7f43e9dcd700 1 -- 172.21.15.16:0/4150481882 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43e40d4000 con 0x7f43e40b4750 2023-05-20T01:17:42.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.980+0000 7f43dbfff700 1 -- 172.21.15.16:0/4150481882 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f43cc003bc0 con 0x7f43e40b4750 2023-05-20T01:17:42.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.980+0000 7f43dbfff700 1 --2- 172.21.15.16:0/4150481882 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f43c406ad10 0x7f43c406d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:42.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.980+0000 7f43dbfff700 1 -- 172.21.15.16:0/4150481882 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f43cc020070 con 0x7f43e40b4750 2023-05-20T01:17:42.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.980+0000 7f43e2ffd700 1 --2- 172.21.15.16:0/4150481882 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f43c406ad10 0x7f43c406d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:42.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.981+0000 7f43e2ffd700 1 --2- 172.21.15.16:0/4150481882 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f43c406ad10 0x7f43c406d1c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f43d4000f60 tx=0x7f43d4007a40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:42.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:42.985+0000 7f43dbfff700 1 -- 172.21.15.16:0/4150481882 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f43cc055aa0 con 0x7f43e40b4750 2023-05-20T01:17:43.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.184+0000 7f43e9dcd700 1 -- 172.21.15.16:0/4150481882 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f43e40747b0 con 0x7f43e40b4750 2023-05-20T01:17:43.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.185+0000 7f43dbfff700 1 -- 172.21.15.16:0/4150481882 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+14750 (secure 0 0 0) 0x7f43cc0590c0 con 0x7f43e40b4750 2023-05-20T01:17:43.189 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:17:43.190 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":36,"fsid":"c2245cc5-ecab-4087-a59b-bc81db901676","created":"2023-05-20T01:16:09.549230+0000","modified":"2023-05-20T01:17:39.748501+0000","last_up_change":"2023-05-20T01:16:19.788624+0000","last_in_change":"2023-05-20T01:16:12.092160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-05-20T01:16:20.934366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-05-20T01:16:21.372294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"317a83cd-19c8-4958-a301-6087ef342289","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6825","nonce":3923634179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6827","nonce":3923634179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6831","nonce":3923634179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6829","nonce":3923634179}]},"public_addr":"172.21.15.16:6825/3923634179","cluster_addr":"172.21.15.16:6827/3923634179","heartbeat_back_addr":"172.21.15.16:6831/3923634179","heartbeat_front_addr":"172.21.15.16:6829/3923634179","state":["exists","up"]},{"osd":1,"uuid":"5581046f-8219-4089-8d7d-b7846399505e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6809","nonce":3860264220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6811","nonce":3860264220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6815","nonce":3860264220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6813","nonce":3860264220}]},"public_addr":"172.21.15.16:6809/3860264220","cluster_addr":"172.21.15.16:6811/3860264220","heartbeat_back_addr":"172.21.15.16:6815/3860264220","heartbeat_front_addr":"172.21.15.16:6813/3860264220","state":["exists","up"]},{"osd":2,"uuid":"072e9e66-4133-478e-8d4a-64d46c2963f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6817","nonce":293187030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6819","nonce":293187030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6823","nonce":293187030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6821","nonce":293187030}]},"public_addr":"172.21.15.16:6817/293187030","cluster_addr":"172.21.15.16:6819/293187030","heartbeat_back_addr":"172.21.15.16:6823/293187030","heartbeat_front_addr":"172.21.15.16:6821/293187030","state":["exists","up"]},{"osd":3,"uuid":"11e1bba6-00f8-48dd-9028-d8d716b7f145","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6801","nonce":3669781890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6803","nonce":3669781890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6807","nonce":3669781890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6805","nonce":3669781890}]},"public_addr":"172.21.15.16:6801/3669781890","cluster_addr":"172.21.15.16:6803/3669781890","heartbeat_back_addr":"172.21.15.16:6807/3669781890","heartbeat_front_addr":"172.21.15.16:6805/3669781890","state":["exists","up"]},{"osd":4,"uuid":"b8909c3a-cf70-46cb-8d0b-009c707cf9d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6819","nonce":506317223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6823","nonce":506317223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6829","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6831","nonce":506317223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6827","nonce":506317223}]},"public_addr":"172.21.15.132:6819/506317223","cluster_addr":"172.21.15.132:6823/506317223","heartbeat_back_addr":"172.21.15.132:6831/506317223","heartbeat_front_addr":"172.21.15.132:6827/506317223","state":["exists","up"]},{"osd":5,"uuid":"b1818304-4583-47bf-a6ec-0bfbb5166233","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6818","nonce":1844689995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6822","nonce":1844689995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6830","nonce":1844689995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6826","nonce":1844689995}]},"public_addr":"172.21.15.132:6818/1844689995","cluster_addr":"172.21.15.132:6822/1844689995","heartbeat_back_addr":"172.21.15.132:6830/1844689995","heartbeat_front_addr":"172.21.15.132:6826/1844689995","state":["exists","up"]},{"osd":6,"uuid":"d37f4691-eed1-4a93-8818-18354f03f4a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6801","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6802","nonce":727581123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6805","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6807","nonce":727581123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6813","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6815","nonce":727581123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6811","nonce":727581123}]},"public_addr":"172.21.15.132:6802/727581123","cluster_addr":"172.21.15.132:6807/727581123","heartbeat_back_addr":"172.21.15.132:6815/727581123","heartbeat_front_addr":"172.21.15.132:6811/727581123","state":["exists","up"]},{"osd":7,"uuid":"4f712f62-5d1d-4816-8042-03209144f8b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6803","nonce":466001149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6806","nonce":466001149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6814","nonce":466001149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6810","nonce":466001149}]},"public_addr":"172.21.15.132:6803/466001149","cluster_addr":"172.21.15.132:6806/466001149","heartbeat_back_addr":"172.21.15.132:6814/466001149","heartbeat_front_addr":"172.21.15.132:6810/466001149","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.338863+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.278680+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305694+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.246835+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.276237+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.304441+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305421+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.267050+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-20T01:17:43.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.187+0000 7f43e9dcd700 1 -- 172.21.15.16:0/4150481882 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f43c406ad10 msgr2=0x7f43c406d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:43.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.187+0000 7f43e9dcd700 1 --2- 172.21.15.16:0/4150481882 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f43c406ad10 0x7f43c406d1c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f43d4000f60 tx=0x7f43d4007a40 comp rx=0 tx=0).stop 2023-05-20T01:17:43.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.187+0000 7f43e9dcd700 1 -- 172.21.15.16:0/4150481882 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43e40b4750 msgr2=0x7f43e414b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:43.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.187+0000 7f43e9dcd700 1 --2- 172.21.15.16:0/4150481882 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43e40b4750 0x7f43e414b2c0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f43cc014040 tx=0x7f43cc004750 comp rx=0 tx=0).stop 2023-05-20T01:17:43.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.187+0000 7f43e9dcd700 1 -- 172.21.15.16:0/4150481882 shutdown_connections 2023-05-20T01:17:43.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.187+0000 7f43e9dcd700 1 --2- 172.21.15.16:0/4150481882 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f43e40cfc50 0x7f43e40b6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:43.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.187+0000 7f43e9dcd700 1 --2- 172.21.15.16:0/4150481882 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43e40b4750 0x7f43e414b2c0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:43.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.187+0000 7f43e9dcd700 1 --2- 172.21.15.16:0/4150481882 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f43c406ad10 0x7f43c406d1c0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:43.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.187+0000 7f43e9dcd700 1 --2- 172.21.15.16:0/4150481882 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f43e40cd970 0x7f43e40cbd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:43.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.187+0000 7f43e9dcd700 1 -- 172.21.15.16:0/4150481882 >> 172.21.15.16:0/4150481882 conn(0x7f43e409c590 msgr2=0x7f43e40b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:43.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.187+0000 7f43e9dcd700 1 -- 172.21.15.16:0/4150481882 shutdown_connections 2023-05-20T01:17:43.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.187+0000 7f43e9dcd700 1 -- 172.21.15.16:0/4150481882 wait complete. 2023-05-20T01:17:43.204 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-05-20T01:16:20.934366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-05-20T01:16:21.372294+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-05-20T01:17:43.205 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-05-20T01:17:43.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.329+0000 7f4a0433e700 1 Processor -- start 2023-05-20T01:17:43.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.329+0000 7f4a0433e700 1 -- start start 2023-05-20T01:17:43.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.330+0000 7f4a0433e700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f49fc159b50 0x7f49fc06e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:43.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.330+0000 7f4a0433e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f49fc0cd1c0 0x7f49fc0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:43.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.330+0000 7f4a0433e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49fc0ccd30 con 0x7f49fc0cd630 2023-05-20T01:17:43.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.330+0000 7f4a0433e700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49fc0ce990 con 0x7f49fc159b50 2023-05-20T01:17:43.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.330+0000 7f4a0433e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49fc0cc6e0 con 0x7f49fc0cd1c0 2023-05-20T01:17:43.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.330+0000 7f4a020da700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f49fc159b50 0x7f49fc06e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:43.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.330+0000 7f4a020da700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f49fc159b50 0x7f49fc06e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60068/0 (socket says 172.21.15.16:60068) 2023-05-20T01:17:43.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.330+0000 7f4a020da700 1 -- 172.21.15.16:0/3319097307 learned_addr learned my addr 172.21.15.16:0/3319097307 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:43.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.331+0000 7f4a018d9700 1 --2- 172.21.15.16:0/3319097307 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f49fc0cd1c0 0x7f49fc0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:43.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.331+0000 7f4a020da700 1 -- 172.21.15.16:0/3319097307 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f49fc0cd1c0 msgr2=0x7f49fc0710b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:43.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.331+0000 7f4a020da700 1 --2- 172.21.15.16:0/3319097307 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f49fc0cd1c0 0x7f49fc0710b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:43.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.331+0000 7f4a020da700 1 -- 172.21.15.16:0/3319097307 >> v1:172.21.15.132:6789/0 conn(0x7f49fc0cd630 legacy=0x7f49fc0ca870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:43.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.331+0000 7f4a020da700 1 -- 172.21.15.16:0/3319097307 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49fc0cc9c0 con 0x7f49fc159b50 2023-05-20T01:17:43.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.331+0000 7f4a020da700 1 --2- 172.21.15.16:0/3319097307 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49fc159b50 0x7f49fc06e0a0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f49ec006fd0 tx=0x7f49ec009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df8a6dcbf43696c1 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:43.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.331+0000 7f4a010d8700 1 -- 172.21.15.16:0/3319097307 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49ec005e30 con 0x7f49fc159b50 2023-05-20T01:17:43.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.331+0000 7f4a010d8700 1 -- 172.21.15.16:0/3319097307 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49ec004030 con 0x7f49fc159b50 2023-05-20T01:17:43.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.331+0000 7f4a010d8700 1 -- 172.21.15.16:0/3319097307 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49ec0043e0 con 0x7f49fc159b50 2023-05-20T01:17:43.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.332+0000 7f4a0433e700 1 -- 172.21.15.16:0/3319097307 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49fc159b50 msgr2=0x7f49fc06e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:43.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.332+0000 7f4a0433e700 1 --2- 172.21.15.16:0/3319097307 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49fc159b50 0x7f49fc06e0a0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f49ec006fd0 tx=0x7f49ec009bd0 comp rx=0 tx=0).stop 2023-05-20T01:17:43.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.332+0000 7f4a0433e700 1 -- 172.21.15.16:0/3319097307 shutdown_connections 2023-05-20T01:17:43.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.332+0000 7f4a0433e700 1 --2- 172.21.15.16:0/3319097307 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49fc159b50 0x7f49fc06e0a0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:43.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.332+0000 7f4a0433e700 1 --2- 172.21.15.16:0/3319097307 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f49fc0cd1c0 0x7f49fc0710b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:43.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.332+0000 7f4a0433e700 1 -- 172.21.15.16:0/3319097307 >> 172.21.15.16:0/3319097307 conn(0x7f49fc0cffd0 msgr2=0x7f49fc0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:43.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.332+0000 7f4a0433e700 1 -- 172.21.15.16:0/3319097307 shutdown_connections 2023-05-20T01:17:43.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.332+0000 7f4a0433e700 1 -- 172.21.15.16:0/3319097307 wait complete. 2023-05-20T01:17:43.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.333+0000 7f4a0433e700 1 Processor -- start 2023-05-20T01:17:43.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.333+0000 7f4a0433e700 1 -- start start 2023-05-20T01:17:43.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.334+0000 7f4a0433e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f49fc0cd1c0 0x7f49fc14a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:43.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.334+0000 7f4a0433e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49fc0cd630 0x7f49fc14bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:43.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.334+0000 7f4a0433e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49fc159b50 0x7f49fc0c53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:43.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.334+0000 7f4a0433e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49fc0c80f0 con 0x7f49fc159b50 2023-05-20T01:17:43.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.334+0000 7f4a0433e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49fc0c8990 con 0x7f49fc0cd630 2023-05-20T01:17:43.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.334+0000 7f4a0433e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49fc14c460 con 0x7f49fc0cd1c0 2023-05-20T01:17:43.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.334+0000 7f4a028db700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49fc159b50 0x7f49fc0c53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:43.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.334+0000 7f4a028db700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49fc159b50 0x7f49fc0c53b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60078/0 (socket says 172.21.15.16:60078) 2023-05-20T01:17:43.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.334+0000 7f4a028db700 1 -- 172.21.15.16:0/2345311779 learned_addr learned my addr 172.21.15.16:0/2345311779 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:43.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.334+0000 7f4a020da700 1 --2- 172.21.15.16:0/2345311779 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f49fc0cd1c0 0x7f49fc14a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:43.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.334+0000 7f4a018d9700 1 --2- 172.21.15.16:0/2345311779 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49fc0cd630 0x7f49fc14bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:43.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.334+0000 7f4a028db700 1 -- 172.21.15.16:0/2345311779 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f49fc0cd1c0 msgr2=0x7f49fc14a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:43.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.334+0000 7f4a028db700 1 --2- 172.21.15.16:0/2345311779 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f49fc0cd1c0 0x7f49fc14a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:43.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.334+0000 7f4a028db700 1 -- 172.21.15.16:0/2345311779 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49fc0cd630 msgr2=0x7f49fc14bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:43.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.334+0000 7f4a028db700 1 --2- 172.21.15.16:0/2345311779 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49fc0cd630 0x7f49fc14bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:43.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.334+0000 7f4a028db700 1 -- 172.21.15.16:0/2345311779 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49ec009500 con 0x7f49fc159b50 2023-05-20T01:17:43.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.334+0000 7f4a018d9700 1 --2- 172.21.15.16:0/2345311779 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49fc0cd630 0x7f49fc14bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:17:43.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.334+0000 7f4a028db700 1 --2- 172.21.15.16:0/2345311779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49fc159b50 0x7f49fc0c53b0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f49f400cc50 tx=0x7f49f400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:43.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.334+0000 7f49f2ffd700 1 -- 172.21.15.16:0/2345311779 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f4003c00 con 0x7f49fc159b50 2023-05-20T01:17:43.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.335+0000 7f49f2ffd700 1 -- 172.21.15.16:0/2345311779 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49f4003d60 con 0x7f49fc159b50 2023-05-20T01:17:43.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.335+0000 7f4a0433e700 1 -- 172.21.15.16:0/2345311779 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49fc071e80 con 0x7f49fc159b50 2023-05-20T01:17:43.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.335+0000 7f49f2ffd700 1 -- 172.21.15.16:0/2345311779 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f400ab60 con 0x7f49fc159b50 2023-05-20T01:17:43.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.335+0000 7f4a0433e700 1 -- 172.21.15.16:0/2345311779 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49fc06e5e0 con 0x7f49fc159b50 2023-05-20T01:17:43.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.336+0000 7f4a0433e700 1 -- 172.21.15.16:0/2345311779 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49fc0cec70 con 0x7f49fc159b50 2023-05-20T01:17:43.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.336+0000 7f49f2ffd700 1 -- 172.21.15.16:0/2345311779 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f49f400acc0 con 0x7f49fc159b50 2023-05-20T01:17:43.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.336+0000 7f49f2ffd700 1 --2- 172.21.15.16:0/2345311779 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f49e006ace0 0x7f49e006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:43.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.337+0000 7f49f2ffd700 1 -- 172.21.15.16:0/2345311779 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f49f40892a0 con 0x7f49fc159b50 2023-05-20T01:17:43.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.337+0000 7f4a020da700 1 --2- 172.21.15.16:0/2345311779 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f49e006ace0 0x7f49e006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:43.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.337+0000 7f4a020da700 1 --2- 172.21.15.16:0/2345311779 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f49e006ace0 0x7f49e006d190 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f49ec006fd0 tx=0x7f49ec008070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:43.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.341+0000 7f49f2ffd700 1 -- 172.21.15.16:0/2345311779 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f49f40545f0 con 0x7f49fc159b50 2023-05-20T01:17:43.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.535+0000 7f4a0433e700 1 -- 172.21.15.16:0/2345311779 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f49fc0747b0 con 0x7f49fc159b50 2023-05-20T01:17:43.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.536+0000 7f49f2ffd700 1 -- 172.21.15.16:0/2345311779 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v36) v1 ==== 92+0+10 (secure 0 0 0) 0x7f49f4057c10 con 0x7f49fc159b50 2023-05-20T01:17:43.541 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 8 2023-05-20T01:17:43.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.537+0000 7f4a0433e700 1 -- 172.21.15.16:0/2345311779 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f49e006ace0 msgr2=0x7f49e006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:43.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.537+0000 7f4a0433e700 1 --2- 172.21.15.16:0/2345311779 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f49e006ace0 0x7f49e006d190 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f49ec006fd0 tx=0x7f49ec008070 comp rx=0 tx=0).stop 2023-05-20T01:17:43.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.537+0000 7f4a0433e700 1 -- 172.21.15.16:0/2345311779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49fc159b50 msgr2=0x7f49fc0c53b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:43.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.537+0000 7f4a0433e700 1 --2- 172.21.15.16:0/2345311779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49fc159b50 0x7f49fc0c53b0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f49f400cc50 tx=0x7f49f400cf60 comp rx=0 tx=0).stop 2023-05-20T01:17:43.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.538+0000 7f4a0433e700 1 -- 172.21.15.16:0/2345311779 shutdown_connections 2023-05-20T01:17:43.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.538+0000 7f4a0433e700 1 --2- 172.21.15.16:0/2345311779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49fc159b50 0x7f49fc0c53b0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:43.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.538+0000 7f4a0433e700 1 --2- 172.21.15.16:0/2345311779 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f49e006ace0 0x7f49e006d190 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:43.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.538+0000 7f4a0433e700 1 --2- 172.21.15.16:0/2345311779 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f49fc0cd1c0 0x7f49fc14a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:43.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.538+0000 7f4a0433e700 1 --2- 172.21.15.16:0/2345311779 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49fc0cd630 0x7f49fc14bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:43.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.538+0000 7f4a0433e700 1 -- 172.21.15.16:0/2345311779 >> 172.21.15.16:0/2345311779 conn(0x7f49fc0cffd0 msgr2=0x7f49fc0cba80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:43.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.538+0000 7f4a0433e700 1 -- 172.21.15.16:0/2345311779 shutdown_connections 2023-05-20T01:17:43.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.538+0000 7f4a0433e700 1 -- 172.21.15.16:0/2345311779 wait complete. 2023-05-20T01:17:43.554 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-05-20T01:17:43.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.678+0000 7f9905280700 1 Processor -- start 2023-05-20T01:17:43.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.678+0000 7f9905280700 1 -- start start 2023-05-20T01:17:43.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.679+0000 7f9905280700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f990006a510 0x7f990006a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:43.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.679+0000 7f9905280700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f990006dd00 con 0x7f99000cd4f0 2023-05-20T01:17:43.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.679+0000 7f9905280700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99000cfa60 con 0x7f990015cec0 2023-05-20T01:17:43.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.679+0000 7f9905280700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99000cd940 con 0x7f990006a510 2023-05-20T01:17:43.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.679+0000 7f98fe7fc700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f99000cd4f0 0x7f99000cf1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:43380/0 (socket says 172.21.15.16:43380) 2023-05-20T01:17:43.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.679+0000 7f98fe7fc700 1 -- 172.21.15.16:0/3772538508 learned_addr learned my addr 172.21.15.16:0/3772538508 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:17:43.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.679+0000 7f98ff7fe700 1 --2- 172.21.15.16:0/3772538508 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f990006a510 0x7f990006a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:43.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.679+0000 7f98fdffb700 1 -- 172.21.15.16:0/3772538508 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3748417291 0 0) 0x7f990006dd00 con 0x7f99000cd4f0 2023-05-20T01:17:43.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.680+0000 7f98fdffb700 1 -- 172.21.15.16:0/3772538508 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f98e80035b0 con 0x7f99000cd4f0 2023-05-20T01:17:43.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.680+0000 7f98fdffb700 1 -- 172.21.15.16:0/3772538508 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 252419525 0 0) 0x7f98e80035b0 con 0x7f99000cd4f0 2023-05-20T01:17:43.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.680+0000 7f98fdffb700 1 -- 172.21.15.16:0/3772538508 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f990006a510 msgr2=0x7f990006a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:43.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.680+0000 7f98fdffb700 1 --2- 172.21.15.16:0/3772538508 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f990006a510 0x7f990006a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:43.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.680+0000 7f98fdffb700 1 -- 172.21.15.16:0/3772538508 >> v1:172.21.15.132:6789/0 conn(0x7f990015cec0 legacy=0x7f990006e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:43.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.680+0000 7f98fdffb700 1 -- 172.21.15.16:0/3772538508 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f990006d6b0 con 0x7f99000cd4f0 2023-05-20T01:17:43.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.680+0000 7f98fdffb700 1 -- 172.21.15.16:0/3772538508 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7f98f0001670 con 0x7f99000cd4f0 2023-05-20T01:17:43.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.680+0000 7f98fdffb700 1 -- 172.21.15.16:0/3772538508 >> v1:172.21.15.16:6789/0 conn(0x7f99000cd4f0 legacy=0x7f99000cf1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:43.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.680+0000 7f98fdffb700 1 --2- 172.21.15.16:0/3772538508 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f98e8003a30 0x7f98e8003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:43.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.680+0000 7f98fdffb700 1 --2- 172.21.15.16:0/3772538508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98e8004850 0x7f98e8004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:43.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.680+0000 7f98fdffb700 1 --2- 172.21.15.16:0/3772538508 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f990006a510 0x7f98e8005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:43.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.680+0000 7f98fdffb700 1 -- 172.21.15.16:0/3772538508 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99000cd940 con 0x7f98e8004850 2023-05-20T01:17:43.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.680+0000 7f98fdffb700 1 -- 172.21.15.16:0/3772538508 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98e800bac0 con 0x7f98e8003a30 2023-05-20T01:17:43.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.680+0000 7f98fdffb700 1 -- 172.21.15.16:0/3772538508 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98e800bc80 con 0x7f990006a510 2023-05-20T01:17:43.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.680+0000 7f98fe7fc700 1 --2- 172.21.15.16:0/3772538508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98e8004850 0x7f98e8004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:43.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.680+0000 7f98ff7fe700 1 --2- 172.21.15.16:0/3772538508 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f990006a510 0x7f98e8005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:43.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.680+0000 7f98feffd700 1 --2- 172.21.15.16:0/3772538508 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f98e8003a30 0x7f98e8003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:43.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.680+0000 7f98fe7fc700 1 -- 172.21.15.16:0/3772538508 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f990006a510 msgr2=0x7f98e8005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:43.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.680+0000 7f98fe7fc700 1 --2- 172.21.15.16:0/3772538508 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f990006a510 0x7f98e8005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:43.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.680+0000 7f98fe7fc700 1 -- 172.21.15.16:0/3772538508 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f98e8003a30 msgr2=0x7f98e8003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:43.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.680+0000 7f98fe7fc700 1 --2- 172.21.15.16:0/3772538508 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f98e8003a30 0x7f98e8003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:43.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.680+0000 7f98fe7fc700 1 -- 172.21.15.16:0/3772538508 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98e80035b0 con 0x7f98e8004850 2023-05-20T01:17:43.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.680+0000 7f98fe7fc700 1 --2- 172.21.15.16:0/3772538508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98e8004850 0x7f98e8004cc0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f98f000c240 tx=0x7f98e8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7aeb54d1f4a2c92 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:43.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.681+0000 7f98fdffb700 1 -- 172.21.15.16:0/3772538508 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f0007c10 con 0x7f98e8004850 2023-05-20T01:17:43.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.681+0000 7f98fdffb700 1 -- 172.21.15.16:0/3772538508 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98f000cde0 con 0x7f98e8004850 2023-05-20T01:17:43.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.681+0000 7f98fdffb700 1 -- 172.21.15.16:0/3772538508 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f0005ba0 con 0x7f98e8004850 2023-05-20T01:17:43.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.681+0000 7f9905280700 1 -- 172.21.15.16:0/3772538508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98e8004850 msgr2=0x7f98e8004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:43.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.681+0000 7f9905280700 1 --2- 172.21.15.16:0/3772538508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98e8004850 0x7f98e8004cc0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f98f000c240 tx=0x7f98e8005880 comp rx=0 tx=0).stop 2023-05-20T01:17:43.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.681+0000 7f98ff7fe700 1 -- 172.21.15.16:0/3772538508 reap_dead start 2023-05-20T01:17:43.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.681+0000 7f9905280700 1 -- 172.21.15.16:0/3772538508 shutdown_connections 2023-05-20T01:17:43.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.681+0000 7f9905280700 1 -- 172.21.15.16:0/3772538508 >> 172.21.15.16:0/3772538508 conn(0x7f990006c260 msgr2=0x7f99000cffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:43.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.681+0000 7f9905280700 1 -- 172.21.15.16:0/3772538508 shutdown_connections 2023-05-20T01:17:43.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.681+0000 7f9905280700 1 -- 172.21.15.16:0/3772538508 wait complete. 2023-05-20T01:17:43.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.683+0000 7f9905280700 1 Processor -- start 2023-05-20T01:17:43.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.683+0000 7f9905280700 1 -- start start 2023-05-20T01:17:43.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.683+0000 7f9905280700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98e8004850 0x7f99001f2340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:43.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.683+0000 7f9905280700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f990014adb0 0x7f99000c8810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:43.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.683+0000 7f9905280700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f99000b2e40 0x7f99000b32b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:43.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.683+0000 7f9905280700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99000b37f0 con 0x7f98e8004850 2023-05-20T01:17:43.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.683+0000 7f9905280700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99000c9d50 con 0x7f99000b2e40 2023-05-20T01:17:43.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.683+0000 7f9905280700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99000c9140 con 0x7f990014adb0 2023-05-20T01:17:43.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.683+0000 7f98feffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98e8004850 0x7f99001f2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:43.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.683+0000 7f98feffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98e8004850 0x7f99001f2340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60094/0 (socket says 172.21.15.16:60094) 2023-05-20T01:17:43.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.683+0000 7f98feffd700 1 -- 172.21.15.16:0/3309857887 learned_addr learned my addr 172.21.15.16:0/3309857887 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:43.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.684+0000 7f98fe7fc700 1 --2- 172.21.15.16:0/3309857887 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f990014adb0 0x7f99000c8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:43.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.684+0000 7f98ff7fe700 1 --2- 172.21.15.16:0/3309857887 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f99000b2e40 0x7f99000b32b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:43.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.684+0000 7f98feffd700 1 -- 172.21.15.16:0/3309857887 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f990014adb0 msgr2=0x7f99000c8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:43.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.684+0000 7f98feffd700 1 --2- 172.21.15.16:0/3309857887 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f990014adb0 0x7f99000c8810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:43.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.684+0000 7f98feffd700 1 -- 172.21.15.16:0/3309857887 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f99000b2e40 msgr2=0x7f99000b32b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:43.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.684+0000 7f98feffd700 1 --2- 172.21.15.16:0/3309857887 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f99000b2e40 0x7f99000b32b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:43.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.684+0000 7f98feffd700 1 -- 172.21.15.16:0/3309857887 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f990015d290 con 0x7f98e8004850 2023-05-20T01:17:43.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.684+0000 7f98feffd700 1 --2- 172.21.15.16:0/3309857887 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98e8004850 0x7f99001f2340 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f98f400f820 tx=0x7f98f400fb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:43.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.684+0000 7f98e77fe700 1 -- 172.21.15.16:0/3309857887 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f4005320 con 0x7f98e8004850 2023-05-20T01:17:43.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.684+0000 7f98e77fe700 1 -- 172.21.15.16:0/3309857887 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98f4005480 con 0x7f98e8004850 2023-05-20T01:17:43.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.684+0000 7f9905280700 1 -- 172.21.15.16:0/3309857887 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f990006d990 con 0x7f98e8004850 2023-05-20T01:17:43.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.684+0000 7f98e77fe700 1 -- 172.21.15.16:0/3309857887 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f4006aa0 con 0x7f98e8004850 2023-05-20T01:17:43.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.684+0000 7f9905280700 1 -- 172.21.15.16:0/3309857887 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99000cfd40 con 0x7f98e8004850 2023-05-20T01:17:43.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.685+0000 7f9905280700 1 -- 172.21.15.16:0/3309857887 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99000ca030 con 0x7f98e8004850 2023-05-20T01:17:43.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.686+0000 7f98e77fe700 1 -- 172.21.15.16:0/3309857887 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f98f4006c00 con 0x7f98e8004850 2023-05-20T01:17:43.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.686+0000 7f98e77fe700 1 --2- 172.21.15.16:0/3309857887 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f98ec06ace0 0x7f98ec06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:43.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.686+0000 7f98e77fe700 1 -- 172.21.15.16:0/3309857887 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f98f4088a10 con 0x7f98e8004850 2023-05-20T01:17:43.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.686+0000 7f98fe7fc700 1 --2- 172.21.15.16:0/3309857887 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f98ec06ace0 0x7f98ec06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:43.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.686+0000 7f98fe7fc700 1 --2- 172.21.15.16:0/3309857887 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f98ec06ace0 0x7f98ec06d190 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f99000c6be0 tx=0x7f98f0007000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:43.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.690+0000 7f98e77fe700 1 -- 172.21.15.16:0/3309857887 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f98f4053cd0 con 0x7f98e8004850 2023-05-20T01:17:43.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.888+0000 7f9905280700 1 -- 172.21.15.16:0/3309857887 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f99000747b0 con 0x7f98e8004850 2023-05-20T01:17:43.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.888+0000 7f98e77fe700 1 -- 172.21.15.16:0/3309857887 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v36) v1 ==== 93+0+10 (secure 0 0 0) 0x7f98f40572f0 con 0x7f98e8004850 2023-05-20T01:17:43.892 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-05-20T01:17:43.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.890+0000 7f9905280700 1 -- 172.21.15.16:0/3309857887 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f98ec06ace0 msgr2=0x7f98ec06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:43.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.890+0000 7f9905280700 1 --2- 172.21.15.16:0/3309857887 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f98ec06ace0 0x7f98ec06d190 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f99000c6be0 tx=0x7f98f0007000 comp rx=0 tx=0).stop 2023-05-20T01:17:43.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.890+0000 7f9905280700 1 -- 172.21.15.16:0/3309857887 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98e8004850 msgr2=0x7f99001f2340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:43.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.890+0000 7f9905280700 1 --2- 172.21.15.16:0/3309857887 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98e8004850 0x7f99001f2340 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f98f400f820 tx=0x7f98f400fb30 comp rx=0 tx=0).stop 2023-05-20T01:17:43.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.890+0000 7f9905280700 1 -- 172.21.15.16:0/3309857887 shutdown_connections 2023-05-20T01:17:43.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.890+0000 7f9905280700 1 --2- 172.21.15.16:0/3309857887 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f99000b2e40 0x7f99000b32b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:43.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.890+0000 7f9905280700 1 --2- 172.21.15.16:0/3309857887 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98e8004850 0x7f99001f2340 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:43.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.890+0000 7f9905280700 1 --2- 172.21.15.16:0/3309857887 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f98ec06ace0 0x7f98ec06d190 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:43.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.890+0000 7f9905280700 1 --2- 172.21.15.16:0/3309857887 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f990014adb0 0x7f99000c8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:43.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.890+0000 7f9905280700 1 -- 172.21.15.16:0/3309857887 >> 172.21.15.16:0/3309857887 conn(0x7f990006c260 msgr2=0x7f990015c2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:43.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.890+0000 7f9905280700 1 -- 172.21.15.16:0/3309857887 shutdown_connections 2023-05-20T01:17:43.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:43.890+0000 7f9905280700 1 -- 172.21.15.16:0/3309857887 wait complete. 2023-05-20T01:17:43.906 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-05-20T01:17:44.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.027+0000 7f55e1243700 1 Processor -- start 2023-05-20T01:17:44.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.027+0000 7f55e1243700 1 -- start start 2023-05-20T01:17:44.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.028+0000 7f55e1243700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f55dc06a510 0x7f55dc15e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:44.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.028+0000 7f55e1243700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55dc15e060 con 0x7f55dc06a8e0 2023-05-20T01:17:44.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.028+0000 7f55e1243700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55dc15d450 con 0x7f55dc071340 2023-05-20T01:17:44.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.028+0000 7f55e1243700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55dc15d730 con 0x7f55dc06a510 2023-05-20T01:17:44.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.028+0000 7f55da59c700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f55dc06a8e0 0x7f55dc0cf5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:43398/0 (socket says 172.21.15.16:43398) 2023-05-20T01:17:44.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.028+0000 7f55da59c700 1 -- 172.21.15.16:0/2025399783 learned_addr learned my addr 172.21.15.16:0/2025399783 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:17:44.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.028+0000 7f55dad9d700 1 --2- 172.21.15.16:0/2025399783 >> v2:172.21.15.132:3300/0 conn(0x7f55dc06a510 0x7f55dc15e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:44.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.028+0000 7f55d9d9b700 1 -- 172.21.15.16:0/2025399783 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1551590454 0 0) 0x7f55dc15e060 con 0x7f55dc06a8e0 2023-05-20T01:17:44.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.028+0000 7f55d9d9b700 1 -- 172.21.15.16:0/2025399783 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f55c40035b0 con 0x7f55dc06a8e0 2023-05-20T01:17:44.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.029+0000 7f55d9d9b700 1 -- 172.21.15.16:0/2025399783 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 636441080 0 0) 0x7f55c40035b0 con 0x7f55dc06a8e0 2023-05-20T01:17:44.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.029+0000 7f55dad9d700 1 -- 172.21.15.16:0/2025399783 >> v1:172.21.15.132:6789/0 conn(0x7f55dc071340 legacy=0x7f55dc071780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:44.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.029+0000 7f55dad9d700 1 -- 172.21.15.16:0/2025399783 >> v1:172.21.15.16:6789/0 conn(0x7f55dc06a8e0 legacy=0x7f55dc0cf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:44.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.029+0000 7f55dad9d700 1 -- 172.21.15.16:0/2025399783 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55dc15da10 con 0x7f55dc06a510 2023-05-20T01:17:44.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.029+0000 7f55dad9d700 1 --2- 172.21.15.16:0/2025399783 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f55dc06a510 0x7f55dc15e2a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f55d0009840 tx=0x7f55d0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b64a50ac38c65f8a server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:44.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.029+0000 7f55d9d9b700 1 -- 172.21.15.16:0/2025399783 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55cc001670 con 0x7f55dc06a510 2023-05-20T01:17:44.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.029+0000 7f55d9d9b700 1 -- 172.21.15.16:0/2025399783 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55d0004030 con 0x7f55dc06a510 2023-05-20T01:17:44.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.029+0000 7f55d9d9b700 1 -- 172.21.15.16:0/2025399783 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55d00043a0 con 0x7f55dc06a510 2023-05-20T01:17:44.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.030+0000 7f55e1243700 1 -- 172.21.15.16:0/2025399783 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f55dc06a510 msgr2=0x7f55dc15e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:44.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.030+0000 7f55e1243700 1 --2- 172.21.15.16:0/2025399783 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f55dc06a510 0x7f55dc15e2a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f55d0009840 tx=0x7f55d0009b50 comp rx=0 tx=0).stop 2023-05-20T01:17:44.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.030+0000 7f55e1243700 1 -- 172.21.15.16:0/2025399783 shutdown_connections 2023-05-20T01:17:44.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.030+0000 7f55e1243700 1 --2- 172.21.15.16:0/2025399783 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f55dc06a510 0x7f55dc15e2a0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:44.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.030+0000 7f55e1243700 1 -- 172.21.15.16:0/2025399783 >> 172.21.15.16:0/2025399783 conn(0x7f55dc0d0c30 msgr2=0x7f55dc0b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:44.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.030+0000 7f55e1243700 1 -- 172.21.15.16:0/2025399783 shutdown_connections 2023-05-20T01:17:44.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.030+0000 7f55e1243700 1 -- 172.21.15.16:0/2025399783 wait complete. 2023-05-20T01:17:44.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.031+0000 7f55e1243700 1 Processor -- start 2023-05-20T01:17:44.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.031+0000 7f55e1243700 1 -- start start 2023-05-20T01:17:44.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.032+0000 7f55e1243700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f55dc06a510 0x7f55dc0c7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:44.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.032+0000 7f55e1243700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55dc06a8e0 0x7f55dc158f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:44.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.032+0000 7f55e1243700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f55dc071340 0x7f55dc0c9770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:44.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.032+0000 7f55e1243700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55dc0c6b00 con 0x7f55dc06a8e0 2023-05-20T01:17:44.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.032+0000 7f55e1243700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55dc0b2fe0 con 0x7f55dc06a510 2023-05-20T01:17:44.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.032+0000 7f55e1243700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55dc0c6820 con 0x7f55dc071340 2023-05-20T01:17:44.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.032+0000 7f55da59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55dc06a8e0 0x7f55dc158f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:44.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.032+0000 7f55da59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55dc06a8e0 0x7f55dc158f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60110/0 (socket says 172.21.15.16:60110) 2023-05-20T01:17:44.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.032+0000 7f55da59c700 1 -- 172.21.15.16:0/2354189137 learned_addr learned my addr 172.21.15.16:0/2354189137 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:44.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.032+0000 7f55dad9d700 1 --2- 172.21.15.16:0/2354189137 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f55dc06a510 0x7f55dc0c7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:44.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.032+0000 7f55db59e700 1 --2- 172.21.15.16:0/2354189137 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f55dc071340 0x7f55dc0c9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:44.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.032+0000 7f55da59c700 1 -- 172.21.15.16:0/2354189137 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f55dc071340 msgr2=0x7f55dc0c9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:44.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.032+0000 7f55da59c700 1 --2- 172.21.15.16:0/2354189137 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f55dc071340 0x7f55dc0c9770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:44.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.032+0000 7f55da59c700 1 -- 172.21.15.16:0/2354189137 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f55dc06a510 msgr2=0x7f55dc0c7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:44.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.032+0000 7f55da59c700 1 --2- 172.21.15.16:0/2354189137 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f55dc06a510 0x7f55dc0c7080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:44.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.032+0000 7f55da59c700 1 -- 172.21.15.16:0/2354189137 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55d00094a0 con 0x7f55dc06a8e0 2023-05-20T01:17:44.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.032+0000 7f55dad9d700 1 --2- 172.21.15.16:0/2354189137 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f55dc06a510 0x7f55dc0c7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:17:44.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.032+0000 7f55db59e700 1 --2- 172.21.15.16:0/2354189137 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f55dc071340 0x7f55dc0c9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:17:44.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.033+0000 7f55da59c700 1 --2- 172.21.15.16:0/2354189137 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55dc06a8e0 0x7f55dc158f40 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f55cc0011e0 tx=0x7f55cc00cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:44.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.033+0000 7f55c37fe700 1 -- 172.21.15.16:0/2354189137 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55cc007560 con 0x7f55dc06a8e0 2023-05-20T01:17:44.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.033+0000 7f55c37fe700 1 -- 172.21.15.16:0/2354189137 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55cc0076c0 con 0x7f55dc06a8e0 2023-05-20T01:17:44.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.033+0000 7f55e1243700 1 -- 172.21.15.16:0/2354189137 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55dc0cfd00 con 0x7f55dc06a8e0 2023-05-20T01:17:44.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.033+0000 7f55c37fe700 1 -- 172.21.15.16:0/2354189137 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55cc004a90 con 0x7f55dc06a8e0 2023-05-20T01:17:44.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.033+0000 7f55e1243700 1 -- 172.21.15.16:0/2354189137 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55dc15dcf0 con 0x7f55dc06a8e0 2023-05-20T01:17:44.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.034+0000 7f55e1243700 1 -- 172.21.15.16:0/2354189137 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55dc06ad50 con 0x7f55dc06a8e0 2023-05-20T01:17:44.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.034+0000 7f55c37fe700 1 -- 172.21.15.16:0/2354189137 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f55cc004bf0 con 0x7f55dc06a8e0 2023-05-20T01:17:44.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.035+0000 7f55c37fe700 1 --2- 172.21.15.16:0/2354189137 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f55c806ace0 0x7f55c806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:44.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.035+0000 7f55c37fe700 1 -- 172.21.15.16:0/2354189137 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f55cc088700 con 0x7f55dc06a8e0 2023-05-20T01:17:44.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.035+0000 7f55dad9d700 1 --2- 172.21.15.16:0/2354189137 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f55c806ace0 0x7f55c806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:44.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.035+0000 7f55dad9d700 1 --2- 172.21.15.16:0/2354189137 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f55c806ace0 0x7f55c806d190 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f55d000a010 tx=0x7f55d0013040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:44.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.039+0000 7f55c37fe700 1 -- 172.21.15.16:0/2354189137 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f55cc0539b0 con 0x7f55dc06a8e0 2023-05-20T01:17:44.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.239+0000 7f55e1243700 1 -- 172.21.15.16:0/2354189137 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f55dc0747b0 con 0x7f55dc06a8e0 2023-05-20T01:17:44.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.239+0000 7f55c37fe700 1 -- 172.21.15.16:0/2354189137 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v36) v1 ==== 84+0+126 (secure 0 0 0) 0x7f55cc056fd0 con 0x7f55dc06a8e0 2023-05-20T01:17:44.243 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:17:44.244 INFO:teuthology.orchestra.run.smithi016.stdout:[ 2023-05-20T01:17:44.245 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-05-20T01:17:44.245 INFO:teuthology.orchestra.run.smithi016.stdout: "poolnum": 1, 2023-05-20T01:17:44.245 INFO:teuthology.orchestra.run.smithi016.stdout: "poolname": "rbd" 2023-05-20T01:17:44.246 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-05-20T01:17:44.247 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-05-20T01:17:44.247 INFO:teuthology.orchestra.run.smithi016.stdout: "poolnum": 2, 2023-05-20T01:17:44.247 INFO:teuthology.orchestra.run.smithi016.stdout: "poolname": ".mgr" 2023-05-20T01:17:44.247 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:17:44.247 INFO:teuthology.orchestra.run.smithi016.stdout:] 2023-05-20T01:17:44.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.241+0000 7f55e1243700 1 -- 172.21.15.16:0/2354189137 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f55c806ace0 msgr2=0x7f55c806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:44.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.241+0000 7f55e1243700 1 --2- 172.21.15.16:0/2354189137 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f55c806ace0 0x7f55c806d190 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f55d000a010 tx=0x7f55d0013040 comp rx=0 tx=0).stop 2023-05-20T01:17:44.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.241+0000 7f55e1243700 1 -- 172.21.15.16:0/2354189137 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55dc06a8e0 msgr2=0x7f55dc158f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:44.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.241+0000 7f55e1243700 1 --2- 172.21.15.16:0/2354189137 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55dc06a8e0 0x7f55dc158f40 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f55cc0011e0 tx=0x7f55cc00cb00 comp rx=0 tx=0).stop 2023-05-20T01:17:44.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.241+0000 7f55e1243700 1 -- 172.21.15.16:0/2354189137 shutdown_connections 2023-05-20T01:17:44.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.241+0000 7f55e1243700 1 --2- 172.21.15.16:0/2354189137 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f55c806ace0 0x7f55c806d190 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:44.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.241+0000 7f55e1243700 1 --2- 172.21.15.16:0/2354189137 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f55dc071340 0x7f55dc0c9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:44.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.241+0000 7f55e1243700 1 --2- 172.21.15.16:0/2354189137 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f55dc06a510 0x7f55dc0c7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:44.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.241+0000 7f55e1243700 1 --2- 172.21.15.16:0/2354189137 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55dc06a8e0 0x7f55dc158f40 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:44.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.241+0000 7f55e1243700 1 -- 172.21.15.16:0/2354189137 >> 172.21.15.16:0/2354189137 conn(0x7f55dc0d0c30 msgr2=0x7f55dc0ccff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:44.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.241+0000 7f55e1243700 1 -- 172.21.15.16:0/2354189137 shutdown_connections 2023-05-20T01:17:44.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.241+0000 7f55e1243700 1 -- 172.21.15.16:0/2354189137 wait complete. 2023-05-20T01:17:44.258 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-05-20T01:17:44.259 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-05-20T01:17:44.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.385+0000 7fc05fe74700 1 Processor -- start 2023-05-20T01:17:44.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.385+0000 7fc05fe74700 1 -- start start 2023-05-20T01:17:44.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.386+0000 7fc05fe74700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fc058159860 0x7fc05815dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:44.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.386+0000 7fc05fe74700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc0580ce260 0x7fc0580ca9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:44.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.386+0000 7fc05fe74700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc05815e2a0 con 0x7fc0580b1a00 2023-05-20T01:17:44.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.386+0000 7fc05fe74700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0580b2a10 con 0x7fc058159860 2023-05-20T01:17:44.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.386+0000 7fc05fe74700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0580cc550 con 0x7fc0580ce260 2023-05-20T01:17:44.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.386+0000 7fc05dc10700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fc058159860 0x7fc05815dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:44.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.386+0000 7fc05e411700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc0580ce260 0x7fc0580ca9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:44.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.386+0000 7fc05dc10700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fc058159860 0x7fc05815dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.16:57138/0 (socket says 172.21.15.16:57138) 2023-05-20T01:17:44.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.386+0000 7fc05dc10700 1 -- 172.21.15.16:0/2169030202 learned_addr learned my addr 172.21.15.16:0/2169030202 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:44.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.387+0000 7fc05dc10700 1 -- 172.21.15.16:0/2169030202 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc0580ce260 msgr2=0x7fc0580ca9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:44.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.387+0000 7fc05dc10700 1 --2- 172.21.15.16:0/2169030202 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc0580ce260 0x7fc0580ca9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:44.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.387+0000 7fc05dc10700 1 -- 172.21.15.16:0/2169030202 >> v1:172.21.15.132:6789/0 conn(0x7fc0580b1a00 legacy=0x7fc0580cdb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:44.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.387+0000 7fc05dc10700 1 -- 172.21.15.16:0/2169030202 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc058159c30 con 0x7fc058159860 2023-05-20T01:17:44.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.387+0000 7fc05e411700 1 --2- 172.21.15.16:0/2169030202 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc0580ce260 0x7fc0580ca9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-05-20T01:17:44.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.387+0000 7fc05dc10700 1 --2- 172.21.15.16:0/2169030202 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc058159860 0x7fc05815dd60 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc054006fd0 tx=0x7fc054009bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bff16fd108055923 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:44.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.387+0000 7fc05cc0e700 1 -- 172.21.15.16:0/2169030202 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc054005e30 con 0x7fc058159860 2023-05-20T01:17:44.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.387+0000 7fc05cc0e700 1 -- 172.21.15.16:0/2169030202 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc054004030 con 0x7fc058159860 2023-05-20T01:17:44.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.387+0000 7fc05cc0e700 1 -- 172.21.15.16:0/2169030202 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0540043e0 con 0x7fc058159860 2023-05-20T01:17:44.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.387+0000 7fc05fe74700 1 -- 172.21.15.16:0/2169030202 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc058159860 msgr2=0x7fc05815dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:44.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.387+0000 7fc05fe74700 1 --2- 172.21.15.16:0/2169030202 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc058159860 0x7fc05815dd60 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc054006fd0 tx=0x7fc054009bd0 comp rx=0 tx=0).stop 2023-05-20T01:17:44.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.388+0000 7fc05fe74700 1 -- 172.21.15.16:0/2169030202 shutdown_connections 2023-05-20T01:17:44.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.388+0000 7fc05fe74700 1 --2- 172.21.15.16:0/2169030202 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc0580ce260 0x7fc0580ca9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:44.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.388+0000 7fc05fe74700 1 --2- 172.21.15.16:0/2169030202 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc058159860 0x7fc05815dd60 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:44.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.388+0000 7fc05fe74700 1 -- 172.21.15.16:0/2169030202 >> 172.21.15.16:0/2169030202 conn(0x7fc0580cff80 msgr2=0x7fc0580d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:44.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.388+0000 7fc05fe74700 1 -- 172.21.15.16:0/2169030202 shutdown_connections 2023-05-20T01:17:44.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.388+0000 7fc05fe74700 1 -- 172.21.15.16:0/2169030202 wait complete. 2023-05-20T01:17:44.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.389+0000 7fc05fe74700 1 Processor -- start 2023-05-20T01:17:44.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.389+0000 7fc05fe74700 1 -- start start 2023-05-20T01:17:44.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.390+0000 7fc05fe74700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc0580b1a00 0x7fc0580cab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:44.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.390+0000 7fc05fe74700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc0580ce260 0x7fc0580c7050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:44.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.390+0000 7fc05fe74700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc058159860 0x7fc0581481a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:44.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.390+0000 7fc05fe74700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc058156ed0 con 0x7fc058159860 2023-05-20T01:17:44.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.390+0000 7fc05fe74700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc058157c70 con 0x7fc0580ce260 2023-05-20T01:17:44.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.390+0000 7fc05fe74700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc058157120 con 0x7fc0580b1a00 2023-05-20T01:17:44.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.390+0000 7fc05e411700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc058159860 0x7fc0581481a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:44.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.390+0000 7fc05e411700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc058159860 0x7fc0581481a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60126/0 (socket says 172.21.15.16:60126) 2023-05-20T01:17:44.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.390+0000 7fc05e411700 1 -- 172.21.15.16:0/738156172 learned_addr learned my addr 172.21.15.16:0/738156172 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:44.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.390+0000 7fc05dc10700 1 --2- 172.21.15.16:0/738156172 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc0580b1a00 0x7fc0580cab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:44.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.390+0000 7fc05d40f700 1 --2- 172.21.15.16:0/738156172 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc0580ce260 0x7fc0580c7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:44.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.390+0000 7fc05e411700 1 -- 172.21.15.16:0/738156172 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc0580b1a00 msgr2=0x7fc0580cab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:44.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.390+0000 7fc05e411700 1 --2- 172.21.15.16:0/738156172 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc0580b1a00 0x7fc0580cab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:44.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.390+0000 7fc05e411700 1 -- 172.21.15.16:0/738156172 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc0580ce260 msgr2=0x7fc0580c7050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:44.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.390+0000 7fc05e411700 1 --2- 172.21.15.16:0/738156172 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc0580ce260 0x7fc0580c7050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:44.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.390+0000 7fc05e411700 1 -- 172.21.15.16:0/738156172 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc05000bc80 con 0x7fc058159860 2023-05-20T01:17:44.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.390+0000 7fc05d40f700 1 --2- 172.21.15.16:0/738156172 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc0580ce260 0x7fc0580c7050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:17:44.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.390+0000 7fc05e411700 1 --2- 172.21.15.16:0/738156172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc058159860 0x7fc0581481a0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fc05000e130 tx=0x7fc05000e440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:44.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.390+0000 7fc04e7fc700 1 -- 172.21.15.16:0/738156172 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc050010050 con 0x7fc058159860 2023-05-20T01:17:44.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.391+0000 7fc04e7fc700 1 -- 172.21.15.16:0/738156172 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc050011040 con 0x7fc058159860 2023-05-20T01:17:44.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.391+0000 7fc05fe74700 1 -- 172.21.15.16:0/738156172 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc054009500 con 0x7fc058159860 2023-05-20T01:17:44.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.391+0000 7fc04e7fc700 1 -- 172.21.15.16:0/738156172 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc05000ecf0 con 0x7fc058159860 2023-05-20T01:17:44.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.391+0000 7fc05fe74700 1 -- 172.21.15.16:0/738156172 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0580b2170 con 0x7fc058159860 2023-05-20T01:17:44.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.391+0000 7fc05fe74700 1 -- 172.21.15.16:0/738156172 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0580b1e90 con 0x7fc058159860 2023-05-20T01:17:44.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.392+0000 7fc04e7fc700 1 -- 172.21.15.16:0/738156172 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc050009540 con 0x7fc058159860 2023-05-20T01:17:44.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.392+0000 7fc04e7fc700 1 --2- 172.21.15.16:0/738156172 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc03c06ace0 0x7fc03c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:44.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.392+0000 7fc04e7fc700 1 -- 172.21.15.16:0/738156172 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5530+0+0 (secure 0 0 0) 0x7fc05008b5d0 con 0x7fc058159860 2023-05-20T01:17:44.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.393+0000 7fc05dc10700 1 --2- 172.21.15.16:0/738156172 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc03c06ace0 0x7fc03c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:44.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.393+0000 7fc05dc10700 1 --2- 172.21.15.16:0/738156172 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc03c06ace0 0x7fc03c06d190 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc054006fd0 tx=0x7fc054008070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:44.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.397+0000 7fc04e7fc700 1 -- 172.21.15.16:0/738156172 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fc050056920 con 0x7fc058159860 2023-05-20T01:17:44.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:44.597+0000 7fc05fe74700 1 -- 172.21.15.16:0/738156172 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fc0580b2450 con 0x7fc058159860 2023-05-20T01:17:45.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:45.566+0000 7fc04e7fc700 1 -- 172.21.15.16:0/738156172 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v37) v1 ==== 138+0+0 (secure 0 0 0) 0x7fc050059f40 con 0x7fc058159860 2023-05-20T01:17:45.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:45.571+0000 7fc05fe74700 1 -- 172.21.15.16:0/738156172 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc03c06ace0 msgr2=0x7fc03c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:45.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:45.571+0000 7fc05fe74700 1 --2- 172.21.15.16:0/738156172 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc03c06ace0 0x7fc03c06d190 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc054006fd0 tx=0x7fc054008070 comp rx=0 tx=0).stop 2023-05-20T01:17:45.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:45.571+0000 7fc05fe74700 1 -- 172.21.15.16:0/738156172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc058159860 msgr2=0x7fc0581481a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:45.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:45.571+0000 7fc05fe74700 1 --2- 172.21.15.16:0/738156172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc058159860 0x7fc0581481a0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fc05000e130 tx=0x7fc05000e440 comp rx=0 tx=0).stop 2023-05-20T01:17:45.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:45.572+0000 7fc05fe74700 1 -- 172.21.15.16:0/738156172 shutdown_connections 2023-05-20T01:17:45.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:45.572+0000 7fc05fe74700 1 --2- 172.21.15.16:0/738156172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc058159860 0x7fc0581481a0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:45.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:45.572+0000 7fc05fe74700 1 --2- 172.21.15.16:0/738156172 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc03c06ace0 0x7fc03c06d190 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:45.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:45.572+0000 7fc05fe74700 1 --2- 172.21.15.16:0/738156172 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc0580b1a00 0x7fc0580cab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:45.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:45.572+0000 7fc05fe74700 1 --2- 172.21.15.16:0/738156172 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc0580ce260 0x7fc0580c7050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:45.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:45.572+0000 7fc05fe74700 1 -- 172.21.15.16:0/738156172 >> 172.21.15.16:0/738156172 conn(0x7fc0580cff80 msgr2=0x7fc0580aed70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:45.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:45.573+0000 7fc05fe74700 1 -- 172.21.15.16:0/738156172 shutdown_connections 2023-05-20T01:17:45.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:45.573+0000 7fc05fe74700 1 -- 172.21.15.16:0/738156172 wait complete. 2023-05-20T01:17:45.578 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' created 2023-05-20T01:17:45.592 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-05-20T01:17:45.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:45.727+0000 7f875a816700 1 Processor -- start 2023-05-20T01:17:45.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:45.727+0000 7f875a816700 1 -- start start 2023-05-20T01:17:45.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:45.728+0000 7f875a816700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87540cf0f0 0x7f87540cc820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:45.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:45.728+0000 7f875a816700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87540ce4d0 con 0x7f875415ab80 2023-05-20T01:17:45.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:45.728+0000 7f875a816700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87540ce7b0 con 0x7f87540cec80 2023-05-20T01:17:45.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:45.728+0000 7f875a816700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87540cb700 con 0x7f87540cf0f0 2023-05-20T01:17:45.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:45.728+0000 7f8753fff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f875415ab80 0x7f87540cda80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:43426/0 (socket says 172.21.15.16:43426) 2023-05-20T01:17:45.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:45.728+0000 7f8753fff700 1 -- 172.21.15.16:0/1816653643 learned_addr learned my addr 172.21.15.16:0/1816653643 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:17:45.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:45.728+0000 7f8752ffd700 1 -- 172.21.15.16:0/1816653643 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 379043739 0 0) 0x7f87540ce4d0 con 0x7f875415ab80 2023-05-20T01:17:45.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:45.728+0000 7f8758db3700 1 --2- 172.21.15.16:0/1816653643 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87540cf0f0 0x7f87540cc820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:45.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:45.728+0000 7f8752ffd700 1 -- 172.21.15.16:0/1816653643 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87400035b0 con 0x7f875415ab80 2023-05-20T01:17:45.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:45.728+0000 7f8752ffd700 1 -- 172.21.15.16:0/1816653643 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1169566237 0 0) 0x7f87400035b0 con 0x7f875415ab80 2023-05-20T01:17:45.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:45.728+0000 7f8752ffd700 1 -- 172.21.15.16:0/1816653643 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87540cf0f0 msgr2=0x7f87540cc820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:45.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:45.728+0000 7f8752ffd700 1 --2- 172.21.15.16:0/1816653643 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87540cf0f0 0x7f87540cc820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:45.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:45.728+0000 7f8752ffd700 1 -- 172.21.15.16:0/1816653643 >> v1:172.21.15.132:6789/0 conn(0x7f87540cec80 legacy=0x7f87540cc110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:45.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:45.728+0000 7f8752ffd700 1 -- 172.21.15.16:0/1816653643 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87540cb910 con 0x7f875415ab80 2023-05-20T01:17:45.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:45.728+0000 7f8752ffd700 1 -- 172.21.15.16:0/1816653643 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7f8744001670 con 0x7f875415ab80 2023-05-20T01:17:45.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:45.728+0000 7f8752ffd700 1 -- 172.21.15.16:0/1816653643 >> v1:172.21.15.16:6789/0 conn(0x7f875415ab80 legacy=0x7f87540cda80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:45.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:45.728+0000 7f8752ffd700 1 --2- 172.21.15.16:0/1816653643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8740003a30 0x7f8740003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:45.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:45.728+0000 7f8752ffd700 1 --2- 172.21.15.16:0/1816653643 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8740004850 0x7f8740004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:45.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:45.728+0000 7f8752ffd700 1 --2- 172.21.15.16:0/1816653643 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87540cf0f0 0x7f8740005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:45.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:45.728+0000 7f8752ffd700 1 -- 172.21.15.16:0/1816653643 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87540cb700 con 0x7f8740003a30 2023-05-20T01:17:45.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:45.729+0000 7f8752ffd700 1 -- 172.21.15.16:0/1816653643 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f874000bac0 con 0x7f8740004850 2023-05-20T01:17:45.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:45.729+0000 7f8752ffd700 1 -- 172.21.15.16:0/1816653643 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f874000bc50 con 0x7f87540cf0f0 2023-05-20T01:17:45.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:45.729+0000 7f8753fff700 1 --2- 172.21.15.16:0/1816653643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8740003a30 0x7f8740003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:45.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:45.729+0000 7f87537fe700 1 --2- 172.21.15.16:0/1816653643 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8740004850 0x7f8740004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:45.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:45.729+0000 7f8758db3700 1 --2- 172.21.15.16:0/1816653643 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87540cf0f0 0x7f8740005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:45.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:45.729+0000 7f8753fff700 1 -- 172.21.15.16:0/1816653643 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87540cf0f0 msgr2=0x7f8740005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:45.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:45.729+0000 7f8753fff700 1 --2- 172.21.15.16:0/1816653643 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87540cf0f0 0x7f8740005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:45.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:45.729+0000 7f8753fff700 1 -- 172.21.15.16:0/1816653643 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8740004850 msgr2=0x7f8740004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:45.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:45.729+0000 7f8753fff700 1 --2- 172.21.15.16:0/1816653643 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8740004850 0x7f8740004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:45.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:45.729+0000 7f8753fff700 1 -- 172.21.15.16:0/1816653643 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87400035b0 con 0x7f8740003a30 2023-05-20T01:17:45.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:45.729+0000 7f87537fe700 1 --2- 172.21.15.16:0/1816653643 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8740004850 0x7f8740004cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-05-20T01:17:45.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:45.729+0000 7f8753fff700 1 --2- 172.21.15.16:0/1816653643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8740003a30 0x7f8740003e40 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f874400c240 tx=0x7f8740005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60a11d6c60a27094 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:45.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:45.729+0000 7f8752ffd700 1 -- 172.21.15.16:0/1816653643 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8744007c10 con 0x7f8740003a30 2023-05-20T01:17:45.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:45.729+0000 7f8752ffd700 1 -- 172.21.15.16:0/1816653643 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f874400cde0 con 0x7f8740003a30 2023-05-20T01:17:45.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:45.729+0000 7f8752ffd700 1 -- 172.21.15.16:0/1816653643 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8744005ba0 con 0x7f8740003a30 2023-05-20T01:17:45.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:45.730+0000 7f875a816700 1 -- 172.21.15.16:0/1816653643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8740003a30 msgr2=0x7f8740003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:45.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:45.730+0000 7f875a816700 1 --2- 172.21.15.16:0/1816653643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8740003a30 0x7f8740003e40 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f874400c240 tx=0x7f8740005880 comp rx=0 tx=0).stop 2023-05-20T01:17:45.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:45.730+0000 7f8758db3700 1 -- 172.21.15.16:0/1816653643 reap_dead start 2023-05-20T01:17:45.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:45.730+0000 7f875a816700 1 -- 172.21.15.16:0/1816653643 shutdown_connections 2023-05-20T01:17:45.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:45.730+0000 7f875a816700 1 -- 172.21.15.16:0/1816653643 >> 172.21.15.16:0/1816653643 conn(0x7f875406ae30 msgr2=0x7f875409a400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:45.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:45.730+0000 7f875a816700 1 -- 172.21.15.16:0/1816653643 shutdown_connections 2023-05-20T01:17:45.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:45.730+0000 7f875a816700 1 -- 172.21.15.16:0/1816653643 wait complete. 2023-05-20T01:17:45.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:45.731+0000 7f875a816700 1 Processor -- start 2023-05-20T01:17:45.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:45.731+0000 7f875a816700 1 -- start start 2023-05-20T01:17:45.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:45.732+0000 7f875a816700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8740003a30 0x7f87540c5f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:45.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:45.732+0000 7f875a816700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8754156c30 0x7f87541547f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:45.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:45.732+0000 7f875a816700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8754155110 0x7f87540cc5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:45.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:45.732+0000 7f875a816700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87540b5400 con 0x7f8754155110 2023-05-20T01:17:45.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:45.732+0000 7f875a816700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87540c7ba0 con 0x7f8740003a30 2023-05-20T01:17:45.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:45.732+0000 7f875a816700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87540b5650 con 0x7f8754156c30 2023-05-20T01:17:45.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:45.732+0000 7f8758db3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8754155110 0x7f87540cc5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:45.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:45.732+0000 7f8758db3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8754155110 0x7f87540cc5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60146/0 (socket says 172.21.15.16:60146) 2023-05-20T01:17:45.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:45.732+0000 7f8758db3700 1 -- 172.21.15.16:0/3559070517 learned_addr learned my addr 172.21.15.16:0/3559070517 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:45.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:45.732+0000 7f87537fe700 1 --2- 172.21.15.16:0/3559070517 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8754156c30 0x7f87541547f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:45.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:45.732+0000 7f8753fff700 1 --2- 172.21.15.16:0/3559070517 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8740003a30 0x7f87540c5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:45.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:45.733+0000 7f8758db3700 1 -- 172.21.15.16:0/3559070517 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8754156c30 msgr2=0x7f87541547f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:45.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:45.733+0000 7f8758db3700 1 --2- 172.21.15.16:0/3559070517 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8754156c30 0x7f87541547f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:45.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:45.733+0000 7f8758db3700 1 -- 172.21.15.16:0/3559070517 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8740003a30 msgr2=0x7f87540c5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:45.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:45.733+0000 7f8758db3700 1 --2- 172.21.15.16:0/3559070517 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8740003a30 0x7f87540c5f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:45.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:45.733+0000 7f8758db3700 1 -- 172.21.15.16:0/3559070517 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87540cf570 con 0x7f8754155110 2023-05-20T01:17:45.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:45.733+0000 7f87537fe700 1 --2- 172.21.15.16:0/3559070517 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8754156c30 0x7f87541547f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:17:45.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:45.733+0000 7f8758db3700 1 --2- 172.21.15.16:0/3559070517 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8754155110 0x7f87540cc5a0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f87540ceba0 tx=0x7f874c00f490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:45.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:45.733+0000 7f8750ff9700 1 -- 172.21.15.16:0/3559070517 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f874c00e040 con 0x7f8754155110 2023-05-20T01:17:45.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:45.733+0000 7f8750ff9700 1 -- 172.21.15.16:0/3559070517 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f874c00f040 con 0x7f8754155110 2023-05-20T01:17:45.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:45.733+0000 7f875a816700 1 -- 172.21.15.16:0/3559070517 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87540cbbf0 con 0x7f8754155110 2023-05-20T01:17:45.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:45.733+0000 7f8750ff9700 1 -- 172.21.15.16:0/3559070517 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f874c015430 con 0x7f8754155110 2023-05-20T01:17:45.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:45.733+0000 7f875a816700 1 -- 172.21.15.16:0/3559070517 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87540ce190 con 0x7f8754155110 2023-05-20T01:17:45.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:45.734+0000 7f875a816700 1 -- 172.21.15.16:0/3559070517 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87540b5f80 con 0x7f8754155110 2023-05-20T01:17:45.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:45.735+0000 7f8750ff9700 1 -- 172.21.15.16:0/3559070517 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f874c015590 con 0x7f8754155110 2023-05-20T01:17:45.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:45.735+0000 7f8750ff9700 1 --2- 172.21.15.16:0/3559070517 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f873406ace0 0x7f873406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:45.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:45.735+0000 7f8750ff9700 1 -- 172.21.15.16:0/3559070517 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5900+0+0 (secure 0 0 0) 0x7f874c089a80 con 0x7f8754155110 2023-05-20T01:17:45.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:45.735+0000 7f8753fff700 1 --2- 172.21.15.16:0/3559070517 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f873406ace0 0x7f873406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:45.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:45.735+0000 7f8753fff700 1 --2- 172.21.15.16:0/3559070517 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f873406ace0 0x7f873406d190 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8744000c00 tx=0x7f8744007000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:45.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:45.740+0000 7f8750ff9700 1 -- 172.21.15.16:0/3559070517 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f874c054c60 con 0x7f8754155110 2023-05-20T01:17:45.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:45.937+0000 7f875a816700 1 -- 172.21.15.16:0/3559070517 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f87540747b0 con 0x7f8754155110 2023-05-20T01:17:46.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:46.572+0000 7f8750ff9700 1 -- 172.21.15.16:0/3559070517 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v38) v1 ==== 170+0+0 (secure 0 0 0) 0x7f874c058280 con 0x7f8754155110 2023-05-20T01:17:46.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:46.578+0000 7f875a816700 1 -- 172.21.15.16:0/3559070517 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f873406ace0 msgr2=0x7f873406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:46.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:46.578+0000 7f875a816700 1 --2- 172.21.15.16:0/3559070517 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f873406ace0 0x7f873406d190 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8744000c00 tx=0x7f8744007000 comp rx=0 tx=0).stop 2023-05-20T01:17:46.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:46.578+0000 7f875a816700 1 -- 172.21.15.16:0/3559070517 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8754155110 msgr2=0x7f87540cc5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:46.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:46.578+0000 7f875a816700 1 --2- 172.21.15.16:0/3559070517 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8754155110 0x7f87540cc5a0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f87540ceba0 tx=0x7f874c00f490 comp rx=0 tx=0).stop 2023-05-20T01:17:46.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:46.583+0000 7f875a816700 1 -- 172.21.15.16:0/3559070517 shutdown_connections 2023-05-20T01:17:46.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:46.583+0000 7f875a816700 1 --2- 172.21.15.16:0/3559070517 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8754155110 0x7f87540cc5a0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:46.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:46.583+0000 7f875a816700 1 --2- 172.21.15.16:0/3559070517 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8740003a30 0x7f87540c5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:46.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:46.583+0000 7f875a816700 1 --2- 172.21.15.16:0/3559070517 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f873406ace0 0x7f873406d190 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:46.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:46.583+0000 7f875a816700 1 --2- 172.21.15.16:0/3559070517 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8754156c30 0x7f87541547f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:46.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:46.583+0000 7f875a816700 1 -- 172.21.15.16:0/3559070517 >> 172.21.15.16:0/3559070517 conn(0x7f875406ae30 msgr2=0x7f87540b0ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:46.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:46.583+0000 7f875a816700 1 -- 172.21.15.16:0/3559070517 shutdown_connections 2023-05-20T01:17:46.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:46.587+0000 7f875a816700 1 -- 172.21.15.16:0/3559070517 wait complete. 2023-05-20T01:17:46.596 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' created 2023-05-20T01:17:46.609 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-05-20T01:17:46.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:46.739+0000 7f964b642700 1 Processor -- start 2023-05-20T01:17:46.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:46.740+0000 7f964b642700 1 -- start start 2023-05-20T01:17:46.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:46.740+0000 7f964b642700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f964415cd20 0x7f96440b1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:46.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:46.740+0000 7f964b642700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f96440b14c0 0x7f964406b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:46.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:46.740+0000 7f964b642700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f964406bb60 0x7f96440cd930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:46.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:46.740+0000 7f964b642700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f964406bfc0 con 0x7f964415cd20 2023-05-20T01:17:46.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:46.740+0000 7f964b642700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96440cb700 con 0x7f96440b14c0 2023-05-20T01:17:46.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:46.740+0000 7f964b642700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96440b2430 con 0x7f964406bb60 2023-05-20T01:17:46.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:46.741+0000 7f96493de700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f964415cd20 0x7f96440b1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:46.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:46.741+0000 7f96493de700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f964415cd20 0x7f96440b1e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60150/0 (socket says 172.21.15.16:60150) 2023-05-20T01:17:46.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:46.741+0000 7f96493de700 1 -- 172.21.15.16:0/642894826 learned_addr learned my addr 172.21.15.16:0/642894826 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:46.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:46.741+0000 7f9649bdf700 1 --2- 172.21.15.16:0/642894826 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f964406bb60 0x7f96440cd930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:46.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:46.741+0000 7f9648bdd700 1 --2- 172.21.15.16:0/642894826 >> v2:172.21.15.132:3300/0 conn(0x7f96440b14c0 0x7f964406b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:46.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:46.741+0000 7f96493de700 1 -- 172.21.15.16:0/642894826 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f964406bb60 msgr2=0x7f96440cd930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:46.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:46.741+0000 7f96493de700 1 --2- 172.21.15.16:0/642894826 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f964406bb60 0x7f96440cd930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:46.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:46.741+0000 7f96493de700 1 -- 172.21.15.16:0/642894826 >> v2:172.21.15.132:3300/0 conn(0x7f96440b14c0 msgr2=0x7f964406b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:46.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:46.741+0000 7f96493de700 1 --2- 172.21.15.16:0/642894826 >> v2:172.21.15.132:3300/0 conn(0x7f96440b14c0 0x7f964406b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:46.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:46.741+0000 7f96493de700 1 -- 172.21.15.16:0/642894826 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96440b2710 con 0x7f964415cd20 2023-05-20T01:17:46.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:46.741+0000 7f9648bdd700 1 --2- 172.21.15.16:0/642894826 >> v2:172.21.15.132:3300/0 conn(0x7f96440b14c0 0x7f964406b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:17:46.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:46.741+0000 7f96493de700 1 --2- 172.21.15.16:0/642894826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f964415cd20 0x7f96440b1e50 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f96440b3bb0 tx=0x7f9634009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49b50bd873668849 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:46.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:46.741+0000 7f963bfff700 1 -- 172.21.15.16:0/642894826 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9634011830 con 0x7f964415cd20 2023-05-20T01:17:46.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:46.741+0000 7f963bfff700 1 -- 172.21.15.16:0/642894826 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9634011990 con 0x7f964415cd20 2023-05-20T01:17:46.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:46.741+0000 7f963bfff700 1 -- 172.21.15.16:0/642894826 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9634011d20 con 0x7f964415cd20 2023-05-20T01:17:46.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:46.742+0000 7f964b642700 1 -- 172.21.15.16:0/642894826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f964415cd20 msgr2=0x7f96440b1e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:46.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:46.742+0000 7f964b642700 1 --2- 172.21.15.16:0/642894826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f964415cd20 0x7f96440b1e50 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f96440b3bb0 tx=0x7f9634009bd0 comp rx=0 tx=0).stop 2023-05-20T01:17:46.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:46.742+0000 7f964b642700 1 -- 172.21.15.16:0/642894826 shutdown_connections 2023-05-20T01:17:46.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:46.742+0000 7f964b642700 1 --2- 172.21.15.16:0/642894826 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f964406bb60 0x7f96440cd930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:46.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:46.742+0000 7f964b642700 1 --2- 172.21.15.16:0/642894826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f964415cd20 0x7f96440b1e50 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:46.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:46.742+0000 7f964b642700 1 --2- 172.21.15.16:0/642894826 >> v2:172.21.15.132:3300/0 conn(0x7f96440b14c0 0x7f964406b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:46.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:46.742+0000 7f964b642700 1 -- 172.21.15.16:0/642894826 >> 172.21.15.16:0/642894826 conn(0x7f964406ae30 msgr2=0x7f964409a400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:46.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:46.742+0000 7f964b642700 1 -- 172.21.15.16:0/642894826 shutdown_connections 2023-05-20T01:17:46.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:46.742+0000 7f964b642700 1 -- 172.21.15.16:0/642894826 wait complete. 2023-05-20T01:17:46.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:46.743+0000 7f964b642700 1 Processor -- start 2023-05-20T01:17:46.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:46.743+0000 7f964b642700 1 -- start start 2023-05-20T01:17:46.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:46.744+0000 7f964b642700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f964406bb60 0x7f96440c8180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:46.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:46.744+0000 7f964b642700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f96440b14c0 0x7f9644155f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:46.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:46.744+0000 7f964b642700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f964415cd20 0x7f964415de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:46.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:46.744+0000 7f964b642700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96440c86c0 con 0x7f964406bb60 2023-05-20T01:17:46.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:46.744+0000 7f964b642700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96441f21e0 con 0x7f964415cd20 2023-05-20T01:17:46.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:46.744+0000 7f964b642700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96440b51c0 con 0x7f96440b14c0 2023-05-20T01:17:46.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:46.744+0000 7f96493de700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f964406bb60 0x7f96440c8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:46.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:46.744+0000 7f96493de700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f964406bb60 0x7f96440c8180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60156/0 (socket says 172.21.15.16:60156) 2023-05-20T01:17:46.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:46.744+0000 7f96493de700 1 -- 172.21.15.16:0/675127286 learned_addr learned my addr 172.21.15.16:0/675127286 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:46.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:46.744+0000 7f9648bdd700 1 --2- 172.21.15.16:0/675127286 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f96440b14c0 0x7f9644155f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:46.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:46.744+0000 7f9649bdf700 1 --2- 172.21.15.16:0/675127286 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f964415cd20 0x7f964415de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:46.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:46.744+0000 7f96493de700 1 -- 172.21.15.16:0/675127286 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f96440b14c0 msgr2=0x7f9644155f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:46.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:46.744+0000 7f96493de700 1 --2- 172.21.15.16:0/675127286 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f96440b14c0 0x7f9644155f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:46.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:46.744+0000 7f96493de700 1 -- 172.21.15.16:0/675127286 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f964415cd20 msgr2=0x7f964415de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:46.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:46.744+0000 7f96493de700 1 --2- 172.21.15.16:0/675127286 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f964415cd20 0x7f964415de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:46.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:46.744+0000 7f96493de700 1 -- 172.21.15.16:0/675127286 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9634009550 con 0x7f964406bb60 2023-05-20T01:17:46.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:46.745+0000 7f96493de700 1 --2- 172.21.15.16:0/675127286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f964406bb60 0x7f96440c8180 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f9634000c00 tx=0x7f9634004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:46.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:46.745+0000 7f9639ffb700 1 -- 172.21.15.16:0/675127286 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9634004380 con 0x7f964406bb60 2023-05-20T01:17:46.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:46.745+0000 7f9639ffb700 1 -- 172.21.15.16:0/675127286 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f963401f070 con 0x7f964406bb60 2023-05-20T01:17:46.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:46.745+0000 7f964b642700 1 -- 172.21.15.16:0/675127286 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96440cbc30 con 0x7f964406bb60 2023-05-20T01:17:46.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:46.745+0000 7f9639ffb700 1 -- 172.21.15.16:0/675127286 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96340036a0 con 0x7f964406bb60 2023-05-20T01:17:46.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:46.745+0000 7f964b642700 1 -- 172.21.15.16:0/675127286 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96440cb950 con 0x7f964406bb60 2023-05-20T01:17:46.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:46.746+0000 7f964b642700 1 -- 172.21.15.16:0/675127286 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96440b1960 con 0x7f964406bb60 2023-05-20T01:17:46.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:46.746+0000 7f9639ffb700 1 -- 172.21.15.16:0/675127286 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9634021030 con 0x7f964406bb60 2023-05-20T01:17:46.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:46.747+0000 7f9639ffb700 1 --2- 172.21.15.16:0/675127286 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f962806ace0 0x7f962806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:46.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:46.747+0000 7f9639ffb700 1 -- 172.21.15.16:0/675127286 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6282+0+0 (secure 0 0 0) 0x7f963408a8f0 con 0x7f964406bb60 2023-05-20T01:17:46.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:46.747+0000 7f9648bdd700 1 --2- 172.21.15.16:0/675127286 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f962806ace0 0x7f962806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:46.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:46.747+0000 7f9648bdd700 1 --2- 172.21.15.16:0/675127286 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f962806ace0 0x7f962806d190 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f96440ce6c0 tx=0x7f9640008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:46.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:46.752+0000 7f9639ffb700 1 -- 172.21.15.16:0/675127286 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f9634055a70 con 0x7f964406bb60 2023-05-20T01:17:46.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:46.994+0000 7f964b642700 1 -- 172.21.15.16:0/675127286 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f96440d1f60 con 0x7f964406bb60 2023-05-20T01:17:47.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:46.999+0000 7f9639ffb700 1 -- 172.21.15.16:0/675127286 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v12) v1 ==== 172+0+0 (secure 0 0 0) 0x7f9634059090 con 0x7f964406bb60 2023-05-20T01:17:47.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.004+0000 7f964b642700 1 -- 172.21.15.16:0/675127286 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f962806ace0 msgr2=0x7f962806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:47.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.004+0000 7f964b642700 1 --2- 172.21.15.16:0/675127286 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f962806ace0 0x7f962806d190 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f96440ce6c0 tx=0x7f9640008040 comp rx=0 tx=0).stop 2023-05-20T01:17:47.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.004+0000 7f964b642700 1 -- 172.21.15.16:0/675127286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f964406bb60 msgr2=0x7f96440c8180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:47.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.004+0000 7f964b642700 1 --2- 172.21.15.16:0/675127286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f964406bb60 0x7f96440c8180 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f9634000c00 tx=0x7f9634004750 comp rx=0 tx=0).stop 2023-05-20T01:17:47.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.008+0000 7f964b642700 1 -- 172.21.15.16:0/675127286 shutdown_connections 2023-05-20T01:17:47.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.008+0000 7f964b642700 1 --2- 172.21.15.16:0/675127286 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f964415cd20 0x7f964415de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:47.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.008+0000 7f964b642700 1 --2- 172.21.15.16:0/675127286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f964406bb60 0x7f96440c8180 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:47.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.008+0000 7f964b642700 1 --2- 172.21.15.16:0/675127286 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f962806ace0 0x7f962806d190 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:47.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.008+0000 7f964b642700 1 --2- 172.21.15.16:0/675127286 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f96440b14c0 0x7f9644155f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:47.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.008+0000 7f964b642700 1 -- 172.21.15.16:0/675127286 >> 172.21.15.16:0/675127286 conn(0x7f964406ae30 msgr2=0x7f96440afd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:47.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.008+0000 7f964b642700 1 -- 172.21.15.16:0/675127286 shutdown_connections 2023-05-20T01:17:47.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.008+0000 7f964b642700 1 -- 172.21.15.16:0/675127286 wait complete. 2023-05-20T01:17:47.016 INFO:teuthology.orchestra.run.smithi016.stderr:new fs with metadata pool 6 and data pool 7 2023-05-20T01:17:47.034 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-20T01:17:47.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.156+0000 7f5bfee40700 1 Processor -- start 2023-05-20T01:17:47.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.156+0000 7f5bfee40700 1 -- start start 2023-05-20T01:17:47.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.157+0000 7f5bfee40700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5bf80cb320 0x7f5bf80b0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:47.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.157+0000 7f5bfee40700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5bf80b0e50 0x7f5bf80c94e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:47.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.157+0000 7f5bfee40700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bf815ace0 con 0x7f5bf815a910 2023-05-20T01:17:47.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.157+0000 7f5bfee40700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bf80723b0 con 0x7f5bf80b0e50 2023-05-20T01:17:47.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.157+0000 7f5bfee40700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf806ba70 con 0x7f5bf80cb320 2023-05-20T01:17:47.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.157+0000 7f5bfd3dd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5bf80b0e50 0x7f5bf80c94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:47.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.157+0000 7f5bfd3dd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5bf80b0e50 0x7f5bf80c94e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60180/0 (socket says 172.21.15.16:60180) 2023-05-20T01:17:47.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.157+0000 7f5bfd3dd700 1 -- 172.21.15.16:0/1901428544 learned_addr learned my addr 172.21.15.16:0/1901428544 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:47.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.157+0000 7f5bf7fff700 1 --2- 172.21.15.16:0/1901428544 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5bf80cb320 0x7f5bf80b0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:47.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.157+0000 7f5bfd3dd700 1 -- 172.21.15.16:0/1901428544 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5bf80cb320 msgr2=0x7f5bf80b0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:47.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.157+0000 7f5bfd3dd700 1 --2- 172.21.15.16:0/1901428544 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5bf80cb320 0x7f5bf80b0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:47.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.157+0000 7f5bfd3dd700 1 -- 172.21.15.16:0/1901428544 >> v1:172.21.15.132:6789/0 conn(0x7f5bf815a910 legacy=0x7f5bf80cac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:47.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.157+0000 7f5bfd3dd700 1 -- 172.21.15.16:0/1901428544 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bf806bd50 con 0x7f5bf80b0e50 2023-05-20T01:17:47.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.157+0000 7f5bf7fff700 1 --2- 172.21.15.16:0/1901428544 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5bf80cb320 0x7f5bf80b0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:17:47.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.158+0000 7f5bfd3dd700 1 --2- 172.21.15.16:0/1901428544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5bf80b0e50 0x7f5bf80c94e0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f5bf0009e50 tx=0x7f5bf000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8a40706cf05679c server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:47.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.158+0000 7f5bf77fe700 1 -- 172.21.15.16:0/1901428544 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bf000e070 con 0x7f5bf80b0e50 2023-05-20T01:17:47.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.158+0000 7f5bf77fe700 1 -- 172.21.15.16:0/1901428544 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bf0003a20 con 0x7f5bf80b0e50 2023-05-20T01:17:47.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.158+0000 7f5bf77fe700 1 -- 172.21.15.16:0/1901428544 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bf0003dd0 con 0x7f5bf80b0e50 2023-05-20T01:17:47.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.158+0000 7f5bfee40700 1 -- 172.21.15.16:0/1901428544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5bf80b0e50 msgr2=0x7f5bf80c94e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:47.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.158+0000 7f5bfee40700 1 --2- 172.21.15.16:0/1901428544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5bf80b0e50 0x7f5bf80c94e0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f5bf0009e50 tx=0x7f5bf000b330 comp rx=0 tx=0).stop 2023-05-20T01:17:47.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.158+0000 7f5bfee40700 1 -- 172.21.15.16:0/1901428544 shutdown_connections 2023-05-20T01:17:47.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.158+0000 7f5bfee40700 1 --2- 172.21.15.16:0/1901428544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5bf80b0e50 0x7f5bf80c94e0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:47.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.158+0000 7f5bfee40700 1 --2- 172.21.15.16:0/1901428544 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5bf80cb320 0x7f5bf80b0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:47.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.158+0000 7f5bfee40700 1 -- 172.21.15.16:0/1901428544 >> 172.21.15.16:0/1901428544 conn(0x7f5bf80cb910 msgr2=0x7f5bf80cbd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:47.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.158+0000 7f5bfee40700 1 -- 172.21.15.16:0/1901428544 shutdown_connections 2023-05-20T01:17:47.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.158+0000 7f5bfee40700 1 -- 172.21.15.16:0/1901428544 wait complete. 2023-05-20T01:17:47.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.160+0000 7f5bfee40700 1 Processor -- start 2023-05-20T01:17:47.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.160+0000 7f5bfee40700 1 -- start start 2023-05-20T01:17:47.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.161+0000 7f5bfee40700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5bf80b0e50 0x7f5bf81f23a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:47.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.161+0000 7f5bfee40700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5bf80cb320 0x7f5bf80b4110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:47.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.161+0000 7f5bfee40700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5bf815a910 0x7f5bf80afe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:47.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.161+0000 7f5bfee40700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf815d650 con 0x7f5bf815a910 2023-05-20T01:17:47.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.161+0000 7f5bfee40700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf815ccb0 con 0x7f5bf80cb320 2023-05-20T01:17:47.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.161+0000 7f5bfee40700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf815cf90 con 0x7f5bf80b0e50 2023-05-20T01:17:47.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.161+0000 7f5bfd3dd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5bf815a910 0x7f5bf80afe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:47.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.161+0000 7f5bfd3dd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5bf815a910 0x7f5bf80afe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60188/0 (socket says 172.21.15.16:60188) 2023-05-20T01:17:47.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.161+0000 7f5bfd3dd700 1 -- 172.21.15.16:0/2905839471 learned_addr learned my addr 172.21.15.16:0/2905839471 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:47.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.161+0000 7f5bf7fff700 1 --2- 172.21.15.16:0/2905839471 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5bf80cb320 0x7f5bf80b4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:47.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.161+0000 7f5bfcbdc700 1 --2- 172.21.15.16:0/2905839471 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5bf80b0e50 0x7f5bf81f23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:47.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.161+0000 7f5bfd3dd700 1 -- 172.21.15.16:0/2905839471 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5bf80b0e50 msgr2=0x7f5bf81f23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:47.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.161+0000 7f5bfd3dd700 1 --2- 172.21.15.16:0/2905839471 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5bf80b0e50 0x7f5bf81f23a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:47.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.161+0000 7f5bfd3dd700 1 -- 172.21.15.16:0/2905839471 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5bf80cb320 msgr2=0x7f5bf80b4110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:47.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.161+0000 7f5bfd3dd700 1 --2- 172.21.15.16:0/2905839471 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5bf80cb320 0x7f5bf80b4110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:47.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.161+0000 7f5bfd3dd700 1 -- 172.21.15.16:0/2905839471 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bf0009b30 con 0x7f5bf815a910 2023-05-20T01:17:47.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.161+0000 7f5bf7fff700 1 --2- 172.21.15.16:0/2905839471 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5bf80cb320 0x7f5bf80b4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:17:47.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.161+0000 7f5bfd3dd700 1 --2- 172.21.15.16:0/2905839471 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5bf815a910 0x7f5bf80afe50 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f5bf0004000 tx=0x7f5bf0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:47.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.161+0000 7f5bf57fa700 1 -- 172.21.15.16:0/2905839471 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bf0004440 con 0x7f5bf815a910 2023-05-20T01:17:47.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.161+0000 7f5bf57fa700 1 -- 172.21.15.16:0/2905839471 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bf00045a0 con 0x7f5bf815a910 2023-05-20T01:17:47.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.161+0000 7f5bfee40700 1 -- 172.21.15.16:0/2905839471 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bf806c310 con 0x7f5bf815a910 2023-05-20T01:17:47.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.161+0000 7f5bf57fa700 1 -- 172.21.15.16:0/2905839471 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bf0004950 con 0x7f5bf815a910 2023-05-20T01:17:47.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.161+0000 7f5bfee40700 1 -- 172.21.15.16:0/2905839471 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bf806c030 con 0x7f5bf815a910 2023-05-20T01:17:47.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.162+0000 7f5bfee40700 1 -- 172.21.15.16:0/2905839471 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bf806c5f0 con 0x7f5bf815a910 2023-05-20T01:17:47.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.163+0000 7f5bf57fa700 1 -- 172.21.15.16:0/2905839471 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5bf0012430 con 0x7f5bf815a910 2023-05-20T01:17:47.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.163+0000 7f5bf57fa700 1 --2- 172.21.15.16:0/2905839471 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f5bdc06ace0 0x7f5bdc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:47.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.163+0000 7f5bf57fa700 1 -- 172.21.15.16:0/2905839471 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f5bf008ab10 con 0x7f5bf815a910 2023-05-20T01:17:47.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.163+0000 7f5bfcbdc700 1 --2- 172.21.15.16:0/2905839471 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f5bdc06ace0 0x7f5bdc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:47.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.164+0000 7f5bfcbdc700 1 --2- 172.21.15.16:0/2905839471 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f5bdc06ace0 0x7f5bdc06d190 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5bec003e60 tx=0x7f5bec009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:47.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.168+0000 7f5bf57fa700 1 -- 172.21.15.16:0/2905839471 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f5bf0055a90 con 0x7f5bf815a910 2023-05-20T01:17:47.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.369+0000 7f5bfee40700 1 -- 172.21.15.16:0/2905839471 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5bf815d790 con 0x7f5bf815a910 2023-05-20T01:17:47.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.371+0000 7f5bf57fa700 1 -- 172.21.15.16:0/2905839471 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+18419 (secure 0 0 0) 0x7f5bf815d790 con 0x7f5bf815a910 2023-05-20T01:17:47.375 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:17:47.375 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":39,"fsid":"c2245cc5-ecab-4087-a59b-bc81db901676","created":"2023-05-20T01:16:09.549230+0000","modified":"2023-05-20T01:17:46.995594+0000","last_up_change":"2023-05-20T01:16:19.788624+0000","last_in_change":"2023-05-20T01:16:12.092160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-05-20T01:16:20.934366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-05-20T01:16:21.372294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-05-20T01:17:44.628755+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-05-20T01:17:45.968584+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"317a83cd-19c8-4958-a301-6087ef342289","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6825","nonce":3923634179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6827","nonce":3923634179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6831","nonce":3923634179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6829","nonce":3923634179}]},"public_addr":"172.21.15.16:6825/3923634179","cluster_addr":"172.21.15.16:6827/3923634179","heartbeat_back_addr":"172.21.15.16:6831/3923634179","heartbeat_front_addr":"172.21.15.16:6829/3923634179","state":["exists","up"]},{"osd":1,"uuid":"5581046f-8219-4089-8d7d-b7846399505e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6809","nonce":3860264220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6811","nonce":3860264220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6815","nonce":3860264220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6813","nonce":3860264220}]},"public_addr":"172.21.15.16:6809/3860264220","cluster_addr":"172.21.15.16:6811/3860264220","heartbeat_back_addr":"172.21.15.16:6815/3860264220","heartbeat_front_addr":"172.21.15.16:6813/3860264220","state":["exists","up"]},{"osd":2,"uuid":"072e9e66-4133-478e-8d4a-64d46c2963f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6817","nonce":293187030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6819","nonce":293187030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6823","nonce":293187030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6821","nonce":293187030}]},"public_addr":"172.21.15.16:6817/293187030","cluster_addr":"172.21.15.16:6819/293187030","heartbeat_back_addr":"172.21.15.16:6823/293187030","heartbeat_front_addr":"172.21.15.16:6821/293187030","state":["exists","up"]},{"osd":3,"uuid":"11e1bba6-00f8-48dd-9028-d8d716b7f145","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6801","nonce":3669781890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6803","nonce":3669781890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6807","nonce":3669781890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6805","nonce":3669781890}]},"public_addr":"172.21.15.16:6801/3669781890","cluster_addr":"172.21.15.16:6803/3669781890","heartbeat_back_addr":"172.21.15.16:6807/3669781890","heartbeat_front_addr":"172.21.15.16:6805/3669781890","state":["exists","up"]},{"osd":4,"uuid":"b8909c3a-cf70-46cb-8d0b-009c707cf9d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6819","nonce":506317223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6823","nonce":506317223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6829","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6831","nonce":506317223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6827","nonce":506317223}]},"public_addr":"172.21.15.132:6819/506317223","cluster_addr":"172.21.15.132:6823/506317223","heartbeat_back_addr":"172.21.15.132:6831/506317223","heartbeat_front_addr":"172.21.15.132:6827/506317223","state":["exists","up"]},{"osd":5,"uuid":"b1818304-4583-47bf-a6ec-0bfbb5166233","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6818","nonce":1844689995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6822","nonce":1844689995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6830","nonce":1844689995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6826","nonce":1844689995}]},"public_addr":"172.21.15.132:6818/1844689995","cluster_addr":"172.21.15.132:6822/1844689995","heartbeat_back_addr":"172.21.15.132:6830/1844689995","heartbeat_front_addr":"172.21.15.132:6826/1844689995","state":["exists","up"]},{"osd":6,"uuid":"d37f4691-eed1-4a93-8818-18354f03f4a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6801","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6802","nonce":727581123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6805","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6807","nonce":727581123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6813","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6815","nonce":727581123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6811","nonce":727581123}]},"public_addr":"172.21.15.132:6802/727581123","cluster_addr":"172.21.15.132:6807/727581123","heartbeat_back_addr":"172.21.15.132:6815/727581123","heartbeat_front_addr":"172.21.15.132:6811/727581123","state":["exists","up"]},{"osd":7,"uuid":"4f712f62-5d1d-4816-8042-03209144f8b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6803","nonce":466001149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6806","nonce":466001149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6814","nonce":466001149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6810","nonce":466001149}]},"public_addr":"172.21.15.132:6803/466001149","cluster_addr":"172.21.15.132:6806/466001149","heartbeat_back_addr":"172.21.15.132:6814/466001149","heartbeat_front_addr":"172.21.15.132:6810/466001149","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.338863+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.278680+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305694+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.246835+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.276237+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.304441+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305421+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.267050+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-20T01:17:47.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.373+0000 7f5bfee40700 1 -- 172.21.15.16:0/2905839471 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f5bdc06ace0 msgr2=0x7f5bdc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:47.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.373+0000 7f5bfee40700 1 --2- 172.21.15.16:0/2905839471 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f5bdc06ace0 0x7f5bdc06d190 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5bec003e60 tx=0x7f5bec009040 comp rx=0 tx=0).stop 2023-05-20T01:17:47.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.373+0000 7f5bfee40700 1 -- 172.21.15.16:0/2905839471 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5bf815a910 msgr2=0x7f5bf80afe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:47.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.373+0000 7f5bfee40700 1 --2- 172.21.15.16:0/2905839471 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5bf815a910 0x7f5bf80afe50 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f5bf0004000 tx=0x7f5bf0004ad0 comp rx=0 tx=0).stop 2023-05-20T01:17:47.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.373+0000 7f5bfee40700 1 -- 172.21.15.16:0/2905839471 shutdown_connections 2023-05-20T01:17:47.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.373+0000 7f5bfee40700 1 --2- 172.21.15.16:0/2905839471 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5bf815a910 0x7f5bf80afe50 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:47.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.373+0000 7f5bfee40700 1 --2- 172.21.15.16:0/2905839471 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f5bdc06ace0 0x7f5bdc06d190 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:47.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.373+0000 7f5bfee40700 1 --2- 172.21.15.16:0/2905839471 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5bf80b0e50 0x7f5bf81f23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:47.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.373+0000 7f5bfee40700 1 --2- 172.21.15.16:0/2905839471 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5bf80cb320 0x7f5bf80b4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:47.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.373+0000 7f5bfee40700 1 -- 172.21.15.16:0/2905839471 >> 172.21.15.16:0/2905839471 conn(0x7f5bf80cb910 msgr2=0x7f5bf8159640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:47.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.373+0000 7f5bfee40700 1 -- 172.21.15.16:0/2905839471 shutdown_connections 2023-05-20T01:17:47.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.373+0000 7f5bfee40700 1 -- 172.21.15.16:0/2905839471 wait complete. 2023-05-20T01:17:47.389 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-20T01:17:47.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.510+0000 7efeeffbd700 1 Processor -- start 2023-05-20T01:17:47.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.510+0000 7efeeffbd700 1 -- start start 2023-05-20T01:17:47.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.511+0000 7efeeffbd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7efee815a9d0 0x7efee815efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:47.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.511+0000 7efeeffbd700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7efee80b2250 0x7efee806a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:47.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.511+0000 7efeeffbd700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efee806f850 0x7efee806aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:47.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.511+0000 7efeeffbd700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efee80cc8d0 con 0x7efee815a9d0 2023-05-20T01:17:47.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.511+0000 7efeeffbd700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efee806d940 con 0x7efee80b2250 2023-05-20T01:17:47.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.511+0000 7efeeffbd700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efee815f500 con 0x7efee806f850 2023-05-20T01:17:47.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.511+0000 7efeedd59700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7efee815a9d0 0x7efee815efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:47.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.511+0000 7efeedd59700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7efee815a9d0 0x7efee815efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60192/0 (socket says 172.21.15.16:60192) 2023-05-20T01:17:47.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.511+0000 7efeedd59700 1 -- 172.21.15.16:0/3938452468 learned_addr learned my addr 172.21.15.16:0/3938452468 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:47.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.511+0000 7efeedd59700 1 -- 172.21.15.16:0/3938452468 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efee806f850 msgr2=0x7efee806aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:47.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.511+0000 7efeedd59700 1 --2- 172.21.15.16:0/3938452468 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efee806f850 0x7efee806aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:47.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.511+0000 7efeedd59700 1 -- 172.21.15.16:0/3938452468 >> v2:172.21.15.132:3300/0 conn(0x7efee80b2250 msgr2=0x7efee806a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:47.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.511+0000 7efeedd59700 1 --2- 172.21.15.16:0/3938452468 >> v2:172.21.15.132:3300/0 conn(0x7efee80b2250 0x7efee806a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:47.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.511+0000 7efeedd59700 1 -- 172.21.15.16:0/3938452468 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efee80cc030 con 0x7efee815a9d0 2023-05-20T01:17:47.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.511+0000 7efeedd59700 1 --2- 172.21.15.16:0/3938452468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efee815a9d0 0x7efee815efc0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7efee80cfe20 tx=0x7efee4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=defa9db3b328273e server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:47.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.512+0000 7efeecd57700 1 -- 172.21.15.16:0/3938452468 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efee400b870 con 0x7efee815a9d0 2023-05-20T01:17:47.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.512+0000 7efeecd57700 1 -- 172.21.15.16:0/3938452468 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efee400b9d0 con 0x7efee815a9d0 2023-05-20T01:17:47.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.512+0000 7efeecd57700 1 -- 172.21.15.16:0/3938452468 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efee400bd60 con 0x7efee815a9d0 2023-05-20T01:17:47.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.512+0000 7efeeffbd700 1 -- 172.21.15.16:0/3938452468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efee815a9d0 msgr2=0x7efee815efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:47.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.512+0000 7efeeffbd700 1 --2- 172.21.15.16:0/3938452468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efee815a9d0 0x7efee815efc0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7efee80cfe20 tx=0x7efee4009bd0 comp rx=0 tx=0).stop 2023-05-20T01:17:47.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.512+0000 7efeeffbd700 1 -- 172.21.15.16:0/3938452468 shutdown_connections 2023-05-20T01:17:47.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.512+0000 7efeeffbd700 1 --2- 172.21.15.16:0/3938452468 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efee806f850 0x7efee806aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:47.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.512+0000 7efeeffbd700 1 --2- 172.21.15.16:0/3938452468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efee815a9d0 0x7efee815efc0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:47.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.512+0000 7efeeffbd700 1 --2- 172.21.15.16:0/3938452468 >> v2:172.21.15.132:3300/0 conn(0x7efee80b2250 0x7efee806a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:47.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.512+0000 7efeeffbd700 1 -- 172.21.15.16:0/3938452468 >> 172.21.15.16:0/3938452468 conn(0x7efee80d0c30 msgr2=0x7efee80b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:47.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.512+0000 7efeeffbd700 1 -- 172.21.15.16:0/3938452468 shutdown_connections 2023-05-20T01:17:47.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.512+0000 7efeeffbd700 1 -- 172.21.15.16:0/3938452468 wait complete. 2023-05-20T01:17:47.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.513+0000 7efeeffbd700 1 Processor -- start 2023-05-20T01:17:47.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.513+0000 7efeeffbd700 1 -- start start 2023-05-20T01:17:47.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.514+0000 7efeeffbd700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efee806f850 0x7efee814b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:47.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.514+0000 7efeeffbd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efee80b2250 0x7efee8157170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:47.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.514+0000 7efeeffbd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efee815a9d0 0x7efee80c6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:47.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.514+0000 7efeeffbd700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efee8158460 con 0x7efee80b2250 2023-05-20T01:17:47.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.514+0000 7efeeffbd700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efee8156ce0 con 0x7efee815a9d0 2023-05-20T01:17:47.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.514+0000 7efeeffbd700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efee81f1580 con 0x7efee806f850 2023-05-20T01:17:47.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.514+0000 7efeed558700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efee80b2250 0x7efee8157170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:47.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.514+0000 7efeed558700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efee80b2250 0x7efee8157170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60200/0 (socket says 172.21.15.16:60200) 2023-05-20T01:17:47.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.514+0000 7efeed558700 1 -- 172.21.15.16:0/1140794870 learned_addr learned my addr 172.21.15.16:0/1140794870 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:47.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.514+0000 7efeedd59700 1 --2- 172.21.15.16:0/1140794870 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efee806f850 0x7efee814b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:47.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.514+0000 7efeee55a700 1 --2- 172.21.15.16:0/1140794870 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efee815a9d0 0x7efee80c6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:47.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.515+0000 7efeed558700 1 -- 172.21.15.16:0/1140794870 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efee806f850 msgr2=0x7efee814b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:47.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.515+0000 7efeed558700 1 --2- 172.21.15.16:0/1140794870 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efee806f850 0x7efee814b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:47.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.515+0000 7efeed558700 1 -- 172.21.15.16:0/1140794870 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efee815a9d0 msgr2=0x7efee80c6230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:47.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.515+0000 7efeed558700 1 --2- 172.21.15.16:0/1140794870 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efee815a9d0 0x7efee80c6230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:47.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.515+0000 7efeed558700 1 -- 172.21.15.16:0/1140794870 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efee4009550 con 0x7efee80b2250 2023-05-20T01:17:47.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.515+0000 7efeed558700 1 --2- 172.21.15.16:0/1140794870 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efee80b2250 0x7efee8157170 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7efed800b530 tx=0x7efed800b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:47.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.515+0000 7efede7fc700 1 -- 172.21.15.16:0/1140794870 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed8011830 con 0x7efee80b2250 2023-05-20T01:17:47.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.515+0000 7efede7fc700 1 -- 172.21.15.16:0/1140794870 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efed80119b0 con 0x7efee80b2250 2023-05-20T01:17:47.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.515+0000 7efeeffbd700 1 -- 172.21.15.16:0/1140794870 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efee80cc5f0 con 0x7efee80b2250 2023-05-20T01:17:47.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.515+0000 7efede7fc700 1 -- 172.21.15.16:0/1140794870 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed8011d60 con 0x7efee80b2250 2023-05-20T01:17:47.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.515+0000 7efeeffbd700 1 -- 172.21.15.16:0/1140794870 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efee80cc310 con 0x7efee80b2250 2023-05-20T01:17:47.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.516+0000 7efeeffbd700 1 -- 172.21.15.16:0/1140794870 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efee815e7c0 con 0x7efee80b2250 2023-05-20T01:17:47.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.517+0000 7efede7fc700 1 -- 172.21.15.16:0/1140794870 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7efed801a400 con 0x7efee80b2250 2023-05-20T01:17:47.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.517+0000 7efede7fc700 1 --2- 172.21.15.16:0/1140794870 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7efecc06ace0 0x7efecc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:47.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.517+0000 7efeedd59700 1 --2- 172.21.15.16:0/1140794870 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7efecc06ace0 0x7efecc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:47.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.517+0000 7efede7fc700 1 -- 172.21.15.16:0/1140794870 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6382+0+0 (secure 0 0 0) 0x7efed8089650 con 0x7efee80b2250 2023-05-20T01:17:47.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.518+0000 7efeedd59700 1 --2- 172.21.15.16:0/1140794870 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7efecc06ace0 0x7efecc06d190 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7efee4014040 tx=0x7efee4008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:47.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.522+0000 7efede7fc700 1 -- 172.21.15.16:0/1140794870 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7efed8054650 con 0x7efee80b2250 2023-05-20T01:17:47.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.721+0000 7efeeffbd700 1 -- 172.21.15.16:0/1140794870 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efee80747b0 con 0x7efee80b2250 2023-05-20T01:17:47.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.722+0000 7efede7fc700 1 -- 172.21.15.16:0/1140794870 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+18419 (secure 0 0 0) 0x7efed8057c70 con 0x7efee80b2250 2023-05-20T01:17:47.726 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:17:47.726 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":39,"fsid":"c2245cc5-ecab-4087-a59b-bc81db901676","created":"2023-05-20T01:16:09.549230+0000","modified":"2023-05-20T01:17:46.995594+0000","last_up_change":"2023-05-20T01:16:19.788624+0000","last_in_change":"2023-05-20T01:16:12.092160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-05-20T01:16:20.934366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-05-20T01:16:21.372294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-05-20T01:17:44.628755+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-05-20T01:17:45.968584+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"317a83cd-19c8-4958-a301-6087ef342289","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6825","nonce":3923634179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6827","nonce":3923634179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6831","nonce":3923634179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6829","nonce":3923634179}]},"public_addr":"172.21.15.16:6825/3923634179","cluster_addr":"172.21.15.16:6827/3923634179","heartbeat_back_addr":"172.21.15.16:6831/3923634179","heartbeat_front_addr":"172.21.15.16:6829/3923634179","state":["exists","up"]},{"osd":1,"uuid":"5581046f-8219-4089-8d7d-b7846399505e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6809","nonce":3860264220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6811","nonce":3860264220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6815","nonce":3860264220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6813","nonce":3860264220}]},"public_addr":"172.21.15.16:6809/3860264220","cluster_addr":"172.21.15.16:6811/3860264220","heartbeat_back_addr":"172.21.15.16:6815/3860264220","heartbeat_front_addr":"172.21.15.16:6813/3860264220","state":["exists","up"]},{"osd":2,"uuid":"072e9e66-4133-478e-8d4a-64d46c2963f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6817","nonce":293187030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6819","nonce":293187030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6823","nonce":293187030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6821","nonce":293187030}]},"public_addr":"172.21.15.16:6817/293187030","cluster_addr":"172.21.15.16:6819/293187030","heartbeat_back_addr":"172.21.15.16:6823/293187030","heartbeat_front_addr":"172.21.15.16:6821/293187030","state":["exists","up"]},{"osd":3,"uuid":"11e1bba6-00f8-48dd-9028-d8d716b7f145","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6801","nonce":3669781890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6803","nonce":3669781890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6807","nonce":3669781890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6805","nonce":3669781890}]},"public_addr":"172.21.15.16:6801/3669781890","cluster_addr":"172.21.15.16:6803/3669781890","heartbeat_back_addr":"172.21.15.16:6807/3669781890","heartbeat_front_addr":"172.21.15.16:6805/3669781890","state":["exists","up"]},{"osd":4,"uuid":"b8909c3a-cf70-46cb-8d0b-009c707cf9d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6819","nonce":506317223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6823","nonce":506317223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6829","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6831","nonce":506317223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6827","nonce":506317223}]},"public_addr":"172.21.15.132:6819/506317223","cluster_addr":"172.21.15.132:6823/506317223","heartbeat_back_addr":"172.21.15.132:6831/506317223","heartbeat_front_addr":"172.21.15.132:6827/506317223","state":["exists","up"]},{"osd":5,"uuid":"b1818304-4583-47bf-a6ec-0bfbb5166233","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6818","nonce":1844689995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6822","nonce":1844689995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6830","nonce":1844689995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6826","nonce":1844689995}]},"public_addr":"172.21.15.132:6818/1844689995","cluster_addr":"172.21.15.132:6822/1844689995","heartbeat_back_addr":"172.21.15.132:6830/1844689995","heartbeat_front_addr":"172.21.15.132:6826/1844689995","state":["exists","up"]},{"osd":6,"uuid":"d37f4691-eed1-4a93-8818-18354f03f4a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6801","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6802","nonce":727581123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6805","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6807","nonce":727581123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6813","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6815","nonce":727581123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6811","nonce":727581123}]},"public_addr":"172.21.15.132:6802/727581123","cluster_addr":"172.21.15.132:6807/727581123","heartbeat_back_addr":"172.21.15.132:6815/727581123","heartbeat_front_addr":"172.21.15.132:6811/727581123","state":["exists","up"]},{"osd":7,"uuid":"4f712f62-5d1d-4816-8042-03209144f8b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6803","nonce":466001149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6806","nonce":466001149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6814","nonce":466001149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6810","nonce":466001149}]},"public_addr":"172.21.15.132:6803/466001149","cluster_addr":"172.21.15.132:6806/466001149","heartbeat_back_addr":"172.21.15.132:6814/466001149","heartbeat_front_addr":"172.21.15.132:6810/466001149","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.338863+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.278680+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305694+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.246835+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.276237+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.304441+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305421+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.267050+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-20T01:17:47.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.724+0000 7efeeffbd700 1 -- 172.21.15.16:0/1140794870 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7efecc06ace0 msgr2=0x7efecc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:47.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.724+0000 7efeeffbd700 1 --2- 172.21.15.16:0/1140794870 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7efecc06ace0 0x7efecc06d190 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7efee4014040 tx=0x7efee4008030 comp rx=0 tx=0).stop 2023-05-20T01:17:47.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.724+0000 7efeeffbd700 1 -- 172.21.15.16:0/1140794870 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efee80b2250 msgr2=0x7efee8157170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:47.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.724+0000 7efeeffbd700 1 --2- 172.21.15.16:0/1140794870 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efee80b2250 0x7efee8157170 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7efed800b530 tx=0x7efed800b8f0 comp rx=0 tx=0).stop 2023-05-20T01:17:47.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.724+0000 7efeeffbd700 1 -- 172.21.15.16:0/1140794870 shutdown_connections 2023-05-20T01:17:47.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.724+0000 7efeeffbd700 1 --2- 172.21.15.16:0/1140794870 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efee815a9d0 0x7efee80c6230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:47.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.724+0000 7efeeffbd700 1 --2- 172.21.15.16:0/1140794870 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7efecc06ace0 0x7efecc06d190 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:47.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.724+0000 7efeeffbd700 1 --2- 172.21.15.16:0/1140794870 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efee806f850 0x7efee814b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:47.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.724+0000 7efeeffbd700 1 --2- 172.21.15.16:0/1140794870 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efee80b2250 0x7efee8157170 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:47.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.724+0000 7efeeffbd700 1 -- 172.21.15.16:0/1140794870 >> 172.21.15.16:0/1140794870 conn(0x7efee80d0c30 msgr2=0x7efee80b3640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:47.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.724+0000 7efeeffbd700 1 -- 172.21.15.16:0/1140794870 shutdown_connections 2023-05-20T01:17:47.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.724+0000 7efeeffbd700 1 -- 172.21.15.16:0/1140794870 wait complete. 2023-05-20T01:17:47.740 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-05-20T01:17:47.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.864+0000 7ffb01f97700 1 Processor -- start 2023-05-20T01:17:47.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.864+0000 7ffb01f97700 1 -- start start 2023-05-20T01:17:47.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.865+0000 7ffb01f97700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7ffafc0cf090 0x7ffafc15bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:47.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.865+0000 7ffb01f97700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ffafc0cdc20 0x7ffafc0ce040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:47.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.865+0000 7ffb01f97700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffafc06a6b0 con 0x7ffafc0d0910 2023-05-20T01:17:47.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.865+0000 7ffb01f97700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffafc06b470 con 0x7ffafc0cdc20 2023-05-20T01:17:47.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.865+0000 7ffb01f97700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffafc06a990 con 0x7ffafc0cf090 2023-05-20T01:17:47.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.865+0000 7ffafbfff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ffafc0cdc20 0x7ffafc0ce040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:47.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.865+0000 7ffafbfff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ffafc0cdc20 0x7ffafc0ce040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60208/0 (socket says 172.21.15.16:60208) 2023-05-20T01:17:47.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.865+0000 7ffafbfff700 1 -- 172.21.15.16:0/3723044812 learned_addr learned my addr 172.21.15.16:0/3723044812 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:47.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.865+0000 7ffafb7fe700 1 --2- 172.21.15.16:0/3723044812 >> v2:172.21.15.132:3300/0 conn(0x7ffafc0cf090 0x7ffafc15bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:47.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.866+0000 7ffafbfff700 1 -- 172.21.15.16:0/3723044812 >> v2:172.21.15.132:3300/0 conn(0x7ffafc0cf090 msgr2=0x7ffafc15bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:47.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.866+0000 7ffafbfff700 1 --2- 172.21.15.16:0/3723044812 >> v2:172.21.15.132:3300/0 conn(0x7ffafc0cf090 0x7ffafc15bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:47.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.866+0000 7ffafbfff700 1 -- 172.21.15.16:0/3723044812 >> v1:172.21.15.132:6789/0 conn(0x7ffafc0d0910 legacy=0x7ffafc0cf460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:47.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.866+0000 7ffafbfff700 1 -- 172.21.15.16:0/3723044812 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffafc06ac70 con 0x7ffafc0cdc20 2023-05-20T01:17:47.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.866+0000 7ffafbfff700 1 --2- 172.21.15.16:0/3723044812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffafc0cdc20 0x7ffafc0ce040 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7ffafc074360 tx=0x7ffaf400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f14aaf3ef76634 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:47.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.866+0000 7ffafa7fc700 1 -- 172.21.15.16:0/3723044812 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf400e070 con 0x7ffafc0cdc20 2023-05-20T01:17:47.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.866+0000 7ffafa7fc700 1 -- 172.21.15.16:0/3723044812 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaf400b9c0 con 0x7ffafc0cdc20 2023-05-20T01:17:47.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.866+0000 7ffafa7fc700 1 -- 172.21.15.16:0/3723044812 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf400bd70 con 0x7ffafc0cdc20 2023-05-20T01:17:47.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.866+0000 7ffb01f97700 1 -- 172.21.15.16:0/3723044812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffafc0cdc20 msgr2=0x7ffafc0ce040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:47.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.866+0000 7ffb01f97700 1 --2- 172.21.15.16:0/3723044812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffafc0cdc20 0x7ffafc0ce040 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7ffafc074360 tx=0x7ffaf400b330 comp rx=0 tx=0).stop 2023-05-20T01:17:47.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.866+0000 7ffb01f97700 1 -- 172.21.15.16:0/3723044812 shutdown_connections 2023-05-20T01:17:47.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.866+0000 7ffb01f97700 1 --2- 172.21.15.16:0/3723044812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffafc0cdc20 0x7ffafc0ce040 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:47.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.867+0000 7ffb01f97700 1 --2- 172.21.15.16:0/3723044812 >> v2:172.21.15.132:3300/0 conn(0x7ffafc0cf090 0x7ffafc15bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:47.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.867+0000 7ffb01f97700 1 -- 172.21.15.16:0/3723044812 >> 172.21.15.16:0/3723044812 conn(0x7ffafc0cd760 msgr2=0x7ffafc0d0050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:47.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.867+0000 7ffb01f97700 1 -- 172.21.15.16:0/3723044812 shutdown_connections 2023-05-20T01:17:47.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.867+0000 7ffb01f97700 1 -- 172.21.15.16:0/3723044812 wait complete. 2023-05-20T01:17:47.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.868+0000 7ffb01f97700 1 Processor -- start 2023-05-20T01:17:47.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.868+0000 7ffb01f97700 1 -- start start 2023-05-20T01:17:47.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.869+0000 7ffb01f97700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffafc0cdc20 0x7ffafc06d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:47.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.869+0000 7ffb01f97700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffafc0cf090 0x7ffafc06e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:47.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.869+0000 7ffb01f97700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffafc0d0910 0x7ffafc1f2860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:47.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.869+0000 7ffb01f97700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffafc06e680 con 0x7ffafc0cf090 2023-05-20T01:17:47.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.869+0000 7ffb01f97700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffafc1f2da0 con 0x7ffafc0cdc20 2023-05-20T01:17:47.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.869+0000 7ffb01f97700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffafc0ad730 con 0x7ffafc0d0910 2023-05-20T01:17:47.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.869+0000 7ffafaffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffafc0cf090 0x7ffafc06e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:47.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.869+0000 7ffafaffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffafc0cf090 0x7ffafc06e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60212/0 (socket says 172.21.15.16:60212) 2023-05-20T01:17:47.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.869+0000 7ffafaffd700 1 -- 172.21.15.16:0/872196985 learned_addr learned my addr 172.21.15.16:0/872196985 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:47.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.869+0000 7ffafbfff700 1 --2- 172.21.15.16:0/872196985 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffafc0d0910 0x7ffafc1f2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:47.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.869+0000 7ffafb7fe700 1 --2- 172.21.15.16:0/872196985 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffafc0cdc20 0x7ffafc06d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:47.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.869+0000 7ffafaffd700 1 -- 172.21.15.16:0/872196985 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffafc0d0910 msgr2=0x7ffafc1f2860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:47.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.869+0000 7ffafaffd700 1 --2- 172.21.15.16:0/872196985 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffafc0d0910 0x7ffafc1f2860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:47.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.869+0000 7ffafaffd700 1 -- 172.21.15.16:0/872196985 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffafc0cdc20 msgr2=0x7ffafc06d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:47.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.869+0000 7ffafaffd700 1 --2- 172.21.15.16:0/872196985 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffafc0cdc20 0x7ffafc06d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:47.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.869+0000 7ffafaffd700 1 -- 172.21.15.16:0/872196985 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaf4009b30 con 0x7ffafc0cf090 2023-05-20T01:17:47.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.869+0000 7ffafbfff700 1 --2- 172.21.15.16:0/872196985 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffafc0d0910 0x7ffafc1f2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:17:47.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.869+0000 7ffafaffd700 1 --2- 172.21.15.16:0/872196985 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffafc0cf090 0x7ffafc06e140 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7ffae400c510 tx=0x7ffae400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:47.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.869+0000 7ffaf3fff700 1 -- 172.21.15.16:0/872196985 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffae40052d0 con 0x7ffafc0cf090 2023-05-20T01:17:47.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.869+0000 7ffaf3fff700 1 -- 172.21.15.16:0/872196985 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffae4005450 con 0x7ffafc0cf090 2023-05-20T01:17:47.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.869+0000 7ffb01f97700 1 -- 172.21.15.16:0/872196985 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffafc0cb7e0 con 0x7ffafc0cf090 2023-05-20T01:17:47.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.869+0000 7ffaf3fff700 1 -- 172.21.15.16:0/872196985 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffae40038c0 con 0x7ffafc0cf090 2023-05-20T01:17:47.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.870+0000 7ffb01f97700 1 -- 172.21.15.16:0/872196985 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffafc06af50 con 0x7ffafc0cf090 2023-05-20T01:17:47.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.870+0000 7ffb01f97700 1 -- 172.21.15.16:0/872196985 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffafc15ae70 con 0x7ffafc0cf090 2023-05-20T01:17:47.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.871+0000 7ffaf3fff700 1 -- 172.21.15.16:0/872196985 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ffae4003a20 con 0x7ffafc0cf090 2023-05-20T01:17:47.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.871+0000 7ffaf3fff700 1 --2- 172.21.15.16:0/872196985 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ffadc06ace0 0x7ffadc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:47.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.871+0000 7ffaf3fff700 1 -- 172.21.15.16:0/872196985 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6382+0+0 (secure 0 0 0) 0x7ffae4088a10 con 0x7ffafc0cf090 2023-05-20T01:17:47.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.871+0000 7ffafb7fe700 1 --2- 172.21.15.16:0/872196985 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ffadc06ace0 0x7ffadc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:47.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.872+0000 7ffafb7fe700 1 --2- 172.21.15.16:0/872196985 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ffadc06ace0 0x7ffadc06d190 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ffaec009950 tx=0x7ffaec008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:47.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:47.876+0000 7ffaf3fff700 1 -- 172.21.15.16:0/872196985 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7ffae4053980 con 0x7ffafc0cf090 2023-05-20T01:17:48.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:48.124+0000 7ffb01f97700 1 -- 172.21.15.16:0/872196985 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7ffafc0747b0 con 0x7ffafc0cf090 2023-05-20T01:17:49.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:49.009+0000 7ffaf3fff700 1 -- 172.21.15.16:0/872196985 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) v1 ==== 118+0+0 (secure 0 0 0) 0x7ffae4056fa0 con 0x7ffafc0cf090 2023-05-20T01:17:49.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:49.012+0000 7ffb01f97700 1 -- 172.21.15.16:0/872196985 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ffadc06ace0 msgr2=0x7ffadc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:49.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:49.012+0000 7ffb01f97700 1 --2- 172.21.15.16:0/872196985 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ffadc06ace0 0x7ffadc06d190 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ffaec009950 tx=0x7ffaec008040 comp rx=0 tx=0).stop 2023-05-20T01:17:49.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:49.012+0000 7ffb01f97700 1 -- 172.21.15.16:0/872196985 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffafc0cf090 msgr2=0x7ffafc06e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:49.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:49.012+0000 7ffb01f97700 1 --2- 172.21.15.16:0/872196985 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffafc0cf090 0x7ffafc06e140 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7ffae400c510 tx=0x7ffae400c8d0 comp rx=0 tx=0).stop 2023-05-20T01:17:49.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:49.012+0000 7ffb01f97700 1 -- 172.21.15.16:0/872196985 shutdown_connections 2023-05-20T01:17:49.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:49.012+0000 7ffb01f97700 1 --2- 172.21.15.16:0/872196985 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ffadc06ace0 0x7ffadc06d190 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:49.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:49.012+0000 7ffb01f97700 1 --2- 172.21.15.16:0/872196985 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffafc0d0910 0x7ffafc1f2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:49.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:49.012+0000 7ffb01f97700 1 --2- 172.21.15.16:0/872196985 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffafc0cdc20 0x7ffafc06d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:49.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:49.012+0000 7ffb01f97700 1 --2- 172.21.15.16:0/872196985 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffafc0cf090 0x7ffafc06e140 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:49.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:49.012+0000 7ffb01f97700 1 -- 172.21.15.16:0/872196985 >> 172.21.15.16:0/872196985 conn(0x7ffafc0cd760 msgr2=0x7ffafc0cbd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:49.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:49.012+0000 7ffb01f97700 1 -- 172.21.15.16:0/872196985 shutdown_connections 2023-05-20T01:17:49.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:49.012+0000 7ffb01f97700 1 -- 172.21.15.16:0/872196985 wait complete. 2023-05-20T01:17:49.030 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-05-20T01:17:49.031 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-05-20T01:17:49.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:49.164+0000 7f9d7a878700 1 Processor -- start 2023-05-20T01:17:49.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:49.164+0000 7f9d7a878700 1 -- start start 2023-05-20T01:17:49.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:49.165+0000 7f9d7a878700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9d7406a510 0x7f9d7406a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:49.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:49.165+0000 7f9d7a878700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d7406dd00 con 0x7f9d7415cec0 2023-05-20T01:17:49.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:49.165+0000 7f9d7a878700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d740cfa60 con 0x7f9d740cd4f0 2023-05-20T01:17:49.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:49.165+0000 7f9d7a878700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d740cd940 con 0x7f9d7406a510 2023-05-20T01:17:49.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:49.165+0000 7f9d73fff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f9d7415cec0 0x7f9d7406e190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:43514/0 (socket says 172.21.15.16:43514) 2023-05-20T01:17:49.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:49.165+0000 7f9d73fff700 1 -- 172.21.15.16:0/3401624299 learned_addr learned my addr 172.21.15.16:0/3401624299 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:17:49.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:49.165+0000 7f9d78e15700 1 --2- 172.21.15.16:0/3401624299 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9d7406a510 0x7f9d7406a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:49.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:49.165+0000 7f9d72ffd700 1 -- 172.21.15.16:0/3401624299 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4272678736 0 0) 0x7f9d7406dd00 con 0x7f9d7415cec0 2023-05-20T01:17:49.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:49.165+0000 7f9d72ffd700 1 -- 172.21.15.16:0/3401624299 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d5c0035b0 con 0x7f9d7415cec0 2023-05-20T01:17:49.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:49.165+0000 7f9d72ffd700 1 -- 172.21.15.16:0/3401624299 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1568684578 0 0) 0x7f9d740cfa60 con 0x7f9d740cd4f0 2023-05-20T01:17:49.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:49.165+0000 7f9d72ffd700 1 -- 172.21.15.16:0/3401624299 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d7406dd00 con 0x7f9d740cd4f0 2023-05-20T01:17:49.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:49.165+0000 7f9d72ffd700 1 -- 172.21.15.16:0/3401624299 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4142989573 0 0) 0x7f9d5c0035b0 con 0x7f9d7415cec0 2023-05-20T01:17:49.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:49.165+0000 7f9d72ffd700 1 -- 172.21.15.16:0/3401624299 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9d7406a510 msgr2=0x7f9d7406a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:49.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:49.165+0000 7f9d72ffd700 1 --2- 172.21.15.16:0/3401624299 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9d7406a510 0x7f9d7406a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:49.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:49.165+0000 7f9d72ffd700 1 -- 172.21.15.16:0/3401624299 >> v1:172.21.15.132:6789/0 conn(0x7f9d740cd4f0 legacy=0x7f9d740cf1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:49.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:49.165+0000 7f9d72ffd700 1 -- 172.21.15.16:0/3401624299 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d7406d6b0 con 0x7f9d7415cec0 2023-05-20T01:17:49.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:49.165+0000 7f9d72ffd700 1 -- 172.21.15.16:0/3401624299 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7f9d64001670 con 0x7f9d7415cec0 2023-05-20T01:17:49.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:49.165+0000 7f9d72ffd700 1 -- 172.21.15.16:0/3401624299 >> v1:172.21.15.16:6789/0 conn(0x7f9d7415cec0 legacy=0x7f9d7406e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:49.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:49.165+0000 7f9d72ffd700 1 --2- 172.21.15.16:0/3401624299 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9d5c003bc0 0x7f9d5c003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:49.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:49.166+0000 7f9d72ffd700 1 --2- 172.21.15.16:0/3401624299 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9d5c004850 0x7f9d5c004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:49.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:49.166+0000 7f9d72ffd700 1 --2- 172.21.15.16:0/3401624299 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d5c005200 0x7f9d5c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:49.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:49.166+0000 7f9d72ffd700 1 -- 172.21.15.16:0/3401624299 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d740cd940 con 0x7f9d5c005200 2023-05-20T01:17:49.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:49.166+0000 7f9d72ffd700 1 -- 172.21.15.16:0/3401624299 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d5c00bfd0 con 0x7f9d5c003bc0 2023-05-20T01:17:49.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:49.166+0000 7f9d72ffd700 1 -- 172.21.15.16:0/3401624299 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d5c00c190 con 0x7f9d5c004850 2023-05-20T01:17:49.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:49.166+0000 7f9d78e15700 1 --2- 172.21.15.16:0/3401624299 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9d7406a510 0x7f9d7406a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-05-20T01:17:49.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:49.166+0000 7f9d73fff700 1 --2- 172.21.15.16:0/3401624299 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9d5c003bc0 0x7f9d5c003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:49.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:49.166+0000 7f9d78e15700 1 --2- 172.21.15.16:0/3401624299 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d5c005200 0x7f9d5c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:49.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:49.166+0000 7f9d737fe700 1 --2- 172.21.15.16:0/3401624299 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9d5c004850 0x7f9d5c004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:49.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:49.166+0000 7f9d78e15700 1 -- 172.21.15.16:0/3401624299 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9d5c004850 msgr2=0x7f9d5c004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:49.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:49.166+0000 7f9d78e15700 1 --2- 172.21.15.16:0/3401624299 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9d5c004850 0x7f9d5c004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:49.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:49.166+0000 7f9d78e15700 1 -- 172.21.15.16:0/3401624299 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9d5c003bc0 msgr2=0x7f9d5c003fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:49.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:49.166+0000 7f9d78e15700 1 --2- 172.21.15.16:0/3401624299 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9d5c003bc0 0x7f9d5c003fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:49.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:49.166+0000 7f9d78e15700 1 -- 172.21.15.16:0/3401624299 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d5c0035b0 con 0x7f9d5c005200 2023-05-20T01:17:49.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:49.166+0000 7f9d78e15700 1 --2- 172.21.15.16:0/3401624299 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d5c005200 0x7f9d5c00ba10 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c00e1f0 tx=0x7f9d5c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5576a386e9fab849 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:49.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:49.166+0000 7f9d72ffd700 1 -- 172.21.15.16:0/3401624299 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d6c00ec90 con 0x7f9d5c005200 2023-05-20T01:17:49.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:49.166+0000 7f9d72ffd700 1 -- 172.21.15.16:0/3401624299 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d6c00f040 con 0x7f9d5c005200 2023-05-20T01:17:49.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:49.166+0000 7f9d72ffd700 1 -- 172.21.15.16:0/3401624299 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d6c003a20 con 0x7f9d5c005200 2023-05-20T01:17:49.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:49.167+0000 7f9d7a878700 1 -- 172.21.15.16:0/3401624299 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d5c005200 msgr2=0x7f9d5c00ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:49.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:49.167+0000 7f9d7a878700 1 --2- 172.21.15.16:0/3401624299 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d5c005200 0x7f9d5c00ba10 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c00e1f0 tx=0x7f9d5c005880 comp rx=0 tx=0).stop 2023-05-20T01:17:49.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:49.167+0000 7f9d78e15700 1 -- 172.21.15.16:0/3401624299 reap_dead start 2023-05-20T01:17:49.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:49.167+0000 7f9d7a878700 1 -- 172.21.15.16:0/3401624299 shutdown_connections 2023-05-20T01:17:49.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:49.167+0000 7f9d7a878700 1 -- 172.21.15.16:0/3401624299 >> 172.21.15.16:0/3401624299 conn(0x7f9d7406c260 msgr2=0x7f9d740cffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:49.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:49.167+0000 7f9d7a878700 1 -- 172.21.15.16:0/3401624299 shutdown_connections 2023-05-20T01:17:49.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:49.167+0000 7f9d7a878700 1 -- 172.21.15.16:0/3401624299 wait complete. 2023-05-20T01:17:49.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:49.168+0000 7f9d7a878700 1 Processor -- start 2023-05-20T01:17:49.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:49.168+0000 7f9d7a878700 1 -- start start 2023-05-20T01:17:49.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:49.169+0000 7f9d7a878700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9d74152e00 0x7f9d7415f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:49.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:49.169+0000 7f9d7a878700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d7415fbc0 0x7f9d7415d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:49.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:49.169+0000 7f9d7a878700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9d7415d960 0x7f9d740c3b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:49.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:49.169+0000 7f9d7a878700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d74160010 con 0x7f9d7415fbc0 2023-05-20T01:17:49.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:49.169+0000 7f9d7a878700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d7415e110 con 0x7f9d74152e00 2023-05-20T01:17:49.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:49.169+0000 7f9d7a878700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d7415e3f0 con 0x7f9d7415d960 2023-05-20T01:17:49.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:49.169+0000 7f9d737fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d7415fbc0 0x7f9d7415d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:49.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:49.169+0000 7f9d737fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d7415fbc0 0x7f9d7415d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60232/0 (socket says 172.21.15.16:60232) 2023-05-20T01:17:49.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:49.169+0000 7f9d737fe700 1 -- 172.21.15.16:0/3027796806 learned_addr learned my addr 172.21.15.16:0/3027796806 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:49.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:49.169+0000 7f9d73fff700 1 --2- 172.21.15.16:0/3027796806 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9d74152e00 0x7f9d7415f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:49.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:49.169+0000 7f9d78e15700 1 --2- 172.21.15.16:0/3027796806 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9d7415d960 0x7f9d740c3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:49.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:49.170+0000 7f9d737fe700 1 -- 172.21.15.16:0/3027796806 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9d7415d960 msgr2=0x7f9d740c3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:49.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:49.170+0000 7f9d737fe700 1 --2- 172.21.15.16:0/3027796806 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9d7415d960 0x7f9d740c3b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:49.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:49.170+0000 7f9d737fe700 1 -- 172.21.15.16:0/3027796806 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9d74152e00 msgr2=0x7f9d7415f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:49.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:49.170+0000 7f9d737fe700 1 --2- 172.21.15.16:0/3027796806 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9d74152e00 0x7f9d7415f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:49.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:49.170+0000 7f9d737fe700 1 -- 172.21.15.16:0/3027796806 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d7415d290 con 0x7f9d7415fbc0 2023-05-20T01:17:49.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:49.170+0000 7f9d737fe700 1 --2- 172.21.15.16:0/3027796806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d7415fbc0 0x7f9d7415d420 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f9d6800f7f0 tx=0x7f9d6800fb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:49.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:49.170+0000 7f9d70ff9700 1 -- 172.21.15.16:0/3027796806 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d680052e0 con 0x7f9d7415fbc0 2023-05-20T01:17:49.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:49.170+0000 7f9d70ff9700 1 -- 172.21.15.16:0/3027796806 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d68005460 con 0x7f9d7415fbc0 2023-05-20T01:17:49.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:49.170+0000 7f9d7a878700 1 -- 172.21.15.16:0/3027796806 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d7406d990 con 0x7f9d7415fbc0 2023-05-20T01:17:49.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:49.170+0000 7f9d70ff9700 1 -- 172.21.15.16:0/3027796806 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d680038c0 con 0x7f9d7415fbc0 2023-05-20T01:17:49.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:49.170+0000 7f9d7a878700 1 -- 172.21.15.16:0/3027796806 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d740cfd40 con 0x7f9d7415fbc0 2023-05-20T01:17:49.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:49.171+0000 7f9d7a878700 1 -- 172.21.15.16:0/3027796806 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d740747b0 con 0x7f9d7415fbc0 2023-05-20T01:17:49.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:49.171+0000 7f9d70ff9700 1 -- 172.21.15.16:0/3027796806 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9d68003a20 con 0x7f9d7415fbc0 2023-05-20T01:17:49.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:49.172+0000 7f9d70ff9700 1 --2- 172.21.15.16:0/3027796806 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f9d6006ace0 0x7f9d6006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:49.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:49.172+0000 7f9d70ff9700 1 -- 172.21.15.16:0/3027796806 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f9d68089400 con 0x7f9d7415fbc0 2023-05-20T01:17:49.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:49.172+0000 7f9d73fff700 1 --2- 172.21.15.16:0/3027796806 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f9d6006ace0 0x7f9d6006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:49.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:49.172+0000 7f9d73fff700 1 --2- 172.21.15.16:0/3027796806 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f9d6006ace0 0x7f9d6006d190 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f9d74071690 tx=0x7f9d64001e00 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:49.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:49.176+0000 7f9d70ff9700 1 -- 172.21.15.16:0/3027796806 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f9d68054370 con 0x7f9d7415fbc0 2023-05-20T01:17:49.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:49.419+0000 7f9d7a878700 1 -- 172.21.15.16:0/3027796806 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f9d740cbca0 con 0x7f9d7415fbc0 2023-05-20T01:17:49.549 INFO:tasks.ceph.mds.c.smithi016.stderr:2023-05-20T01:17:49.546+0000 7f4c92540700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-05-20T01:17:50.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.327+0000 7f9d70ff9700 1 -- 172.21.15.16:0/3027796806 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7f9d68057990 con 0x7f9d7415fbc0 2023-05-20T01:17:50.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.330+0000 7f9d7a878700 1 -- 172.21.15.16:0/3027796806 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f9d6006ace0 msgr2=0x7f9d6006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:50.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.330+0000 7f9d7a878700 1 --2- 172.21.15.16:0/3027796806 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f9d6006ace0 0x7f9d6006d190 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f9d74071690 tx=0x7f9d64001e00 comp rx=0 tx=0).stop 2023-05-20T01:17:50.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.330+0000 7f9d7a878700 1 -- 172.21.15.16:0/3027796806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d7415fbc0 msgr2=0x7f9d7415d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:50.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.330+0000 7f9d7a878700 1 --2- 172.21.15.16:0/3027796806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d7415fbc0 0x7f9d7415d420 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f9d6800f7f0 tx=0x7f9d6800fb00 comp rx=0 tx=0).stop 2023-05-20T01:17:50.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.331+0000 7f9d7a878700 1 -- 172.21.15.16:0/3027796806 shutdown_connections 2023-05-20T01:17:50.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.331+0000 7f9d7a878700 1 --2- 172.21.15.16:0/3027796806 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f9d6006ace0 0x7f9d6006d190 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:50.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.331+0000 7f9d7a878700 1 --2- 172.21.15.16:0/3027796806 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9d7415d960 0x7f9d740c3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:50.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.331+0000 7f9d7a878700 1 --2- 172.21.15.16:0/3027796806 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9d74152e00 0x7f9d7415f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:50.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.331+0000 7f9d7a878700 1 --2- 172.21.15.16:0/3027796806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d7415fbc0 0x7f9d7415d420 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:50.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.331+0000 7f9d7a878700 1 -- 172.21.15.16:0/3027796806 >> 172.21.15.16:0/3027796806 conn(0x7f9d7406c260 msgr2=0x7f9d74072fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:50.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.331+0000 7f9d7a878700 1 -- 172.21.15.16:0/3027796806 shutdown_connections 2023-05-20T01:17:50.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.331+0000 7f9d7a878700 1 -- 172.21.15.16:0/3027796806 wait complete. 2023-05-20T01:17:50.349 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-05-20T01:17:50.350 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-20T01:17:50.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.479+0000 7f21e948f700 1 Processor -- start 2023-05-20T01:17:50.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.479+0000 7f21e948f700 1 -- start start 2023-05-20T01:17:50.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.480+0000 7f21e948f700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f21e4159000 0x7f21e406cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:50.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.480+0000 7f21e948f700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f21e40cb320 0x7f21e406d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:50.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.480+0000 7f21e948f700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f21e40b1b00 0x7f21e40cf6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:50.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.480+0000 7f21e948f700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21e40ca350 con 0x7f21e4159000 2023-05-20T01:17:50.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.480+0000 7f21e948f700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21e415cac0 con 0x7f21e40cb320 2023-05-20T01:17:50.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.480+0000 7f21e948f700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21e415cda0 con 0x7f21e40b1b00 2023-05-20T01:17:50.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.480+0000 7f21e2ffd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f21e4159000 0x7f21e406cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:50.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.480+0000 7f21e2ffd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f21e4159000 0x7f21e406cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60238/0 (socket says 172.21.15.16:60238) 2023-05-20T01:17:50.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.480+0000 7f21e2ffd700 1 -- 172.21.15.16:0/842996425 learned_addr learned my addr 172.21.15.16:0/842996425 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:50.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.481+0000 7f21e27fc700 1 --2- 172.21.15.16:0/842996425 >> v2:172.21.15.132:3300/0 conn(0x7f21e40cb320 0x7f21e406d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:50.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.481+0000 7f21e37fe700 1 --2- 172.21.15.16:0/842996425 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f21e40b1b00 0x7f21e40cf6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:50.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.481+0000 7f21e2ffd700 1 -- 172.21.15.16:0/842996425 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f21e40b1b00 msgr2=0x7f21e40cf6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:50.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.481+0000 7f21e2ffd700 1 --2- 172.21.15.16:0/842996425 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f21e40b1b00 0x7f21e40cf6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:50.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.481+0000 7f21e2ffd700 1 -- 172.21.15.16:0/842996425 >> v2:172.21.15.132:3300/0 conn(0x7f21e40cb320 msgr2=0x7f21e406d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:50.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.481+0000 7f21e2ffd700 1 --2- 172.21.15.16:0/842996425 >> v2:172.21.15.132:3300/0 conn(0x7f21e40cb320 0x7f21e406d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:50.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.481+0000 7f21e2ffd700 1 -- 172.21.15.16:0/842996425 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21e415d080 con 0x7f21e4159000 2023-05-20T01:17:50.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.481+0000 7f21e2ffd700 1 --2- 172.21.15.16:0/842996425 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21e4159000 0x7f21e406cb90 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f21e40cca30 tx=0x7f21cc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=500c3140e0bbd87f server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:50.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.481+0000 7f21e1ffb700 1 -- 172.21.15.16:0/842996425 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21cc00b870 con 0x7f21e4159000 2023-05-20T01:17:50.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.481+0000 7f21e1ffb700 1 -- 172.21.15.16:0/842996425 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21cc00b9d0 con 0x7f21e4159000 2023-05-20T01:17:50.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.481+0000 7f21e1ffb700 1 -- 172.21.15.16:0/842996425 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21cc00bd60 con 0x7f21e4159000 2023-05-20T01:17:50.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.482+0000 7f21e948f700 1 -- 172.21.15.16:0/842996425 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21e4159000 msgr2=0x7f21e406cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:50.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.482+0000 7f21e948f700 1 --2- 172.21.15.16:0/842996425 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21e4159000 0x7f21e406cb90 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f21e40cca30 tx=0x7f21cc009bd0 comp rx=0 tx=0).stop 2023-05-20T01:17:50.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.482+0000 7f21e948f700 1 -- 172.21.15.16:0/842996425 shutdown_connections 2023-05-20T01:17:50.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.482+0000 7f21e948f700 1 --2- 172.21.15.16:0/842996425 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f21e40b1b00 0x7f21e40cf6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:50.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.482+0000 7f21e948f700 1 --2- 172.21.15.16:0/842996425 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21e4159000 0x7f21e406cb90 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:50.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.482+0000 7f21e948f700 1 --2- 172.21.15.16:0/842996425 >> v2:172.21.15.132:3300/0 conn(0x7f21e40cb320 0x7f21e406d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:50.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.482+0000 7f21e948f700 1 -- 172.21.15.16:0/842996425 >> 172.21.15.16:0/842996425 conn(0x7f21e40661b0 msgr2=0x7f21e4066580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:50.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.482+0000 7f21e948f700 1 -- 172.21.15.16:0/842996425 shutdown_connections 2023-05-20T01:17:50.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.482+0000 7f21e948f700 1 -- 172.21.15.16:0/842996425 wait complete. 2023-05-20T01:17:50.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.483+0000 7f21e948f700 1 Processor -- start 2023-05-20T01:17:50.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.483+0000 7f21e948f700 1 -- start start 2023-05-20T01:17:50.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.484+0000 7f21e948f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21e40b1b00 0x7f21e415bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:50.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.484+0000 7f21e948f700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f21e40cb320 0x7f21e4147330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:50.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.484+0000 7f21e948f700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f21e4159000 0x7f21e415af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:50.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.484+0000 7f21e948f700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21e41596b0 con 0x7f21e40b1b00 2023-05-20T01:17:50.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.484+0000 7f21e948f700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21e415aa50 con 0x7f21e40cb320 2023-05-20T01:17:50.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.484+0000 7f21e948f700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21e4159950 con 0x7f21e4159000 2023-05-20T01:17:50.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.484+0000 7f21e2ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21e40b1b00 0x7f21e415bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:50.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.484+0000 7f21e2ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21e40b1b00 0x7f21e415bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60246/0 (socket says 172.21.15.16:60246) 2023-05-20T01:17:50.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.484+0000 7f21e2ffd700 1 -- 172.21.15.16:0/1739838680 learned_addr learned my addr 172.21.15.16:0/1739838680 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:50.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.484+0000 7f21e27fc700 1 --2- 172.21.15.16:0/1739838680 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f21e40cb320 0x7f21e4147330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:50.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.484+0000 7f21e37fe700 1 --2- 172.21.15.16:0/1739838680 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f21e4159000 0x7f21e415af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:50.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.484+0000 7f21e2ffd700 1 -- 172.21.15.16:0/1739838680 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f21e4159000 msgr2=0x7f21e415af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:50.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.484+0000 7f21e2ffd700 1 --2- 172.21.15.16:0/1739838680 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f21e4159000 0x7f21e415af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:50.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.484+0000 7f21e2ffd700 1 -- 172.21.15.16:0/1739838680 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f21e40cb320 msgr2=0x7f21e4147330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:50.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.484+0000 7f21e2ffd700 1 --2- 172.21.15.16:0/1739838680 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f21e40cb320 0x7f21e4147330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:50.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.484+0000 7f21e2ffd700 1 -- 172.21.15.16:0/1739838680 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21cc009550 con 0x7f21e40b1b00 2023-05-20T01:17:50.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.484+0000 7f21e2ffd700 1 --2- 172.21.15.16:0/1739838680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21e40b1b00 0x7f21e415bcc0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f21cc014040 tx=0x7f21cc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:50.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.485+0000 7f21db7fe700 1 -- 172.21.15.16:0/1739838680 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21cc004520 con 0x7f21e40b1b00 2023-05-20T01:17:50.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.485+0000 7f21db7fe700 1 -- 172.21.15.16:0/1739838680 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21cc01e070 con 0x7f21e40b1b00 2023-05-20T01:17:50.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.485+0000 7f21e948f700 1 -- 172.21.15.16:0/1739838680 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21e415d880 con 0x7f21e40b1b00 2023-05-20T01:17:50.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.485+0000 7f21db7fe700 1 -- 172.21.15.16:0/1739838680 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21cc0038e0 con 0x7f21e40b1b00 2023-05-20T01:17:50.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.485+0000 7f21e948f700 1 -- 172.21.15.16:0/1739838680 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21e415d360 con 0x7f21e40b1b00 2023-05-20T01:17:50.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.486+0000 7f21e948f700 1 -- 172.21.15.16:0/1739838680 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21e40c9de0 con 0x7f21e40b1b00 2023-05-20T01:17:50.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.486+0000 7f21db7fe700 1 -- 172.21.15.16:0/1739838680 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f21cc003a40 con 0x7f21e40b1b00 2023-05-20T01:17:50.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.486+0000 7f21db7fe700 1 --2- 172.21.15.16:0/1739838680 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f21c406ace0 0x7f21c406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:50.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.487+0000 7f21db7fe700 1 -- 172.21.15.16:0/1739838680 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f21cc020070 con 0x7f21e40b1b00 2023-05-20T01:17:50.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.487+0000 7f21e27fc700 1 --2- 172.21.15.16:0/1739838680 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f21c406ace0 0x7f21c406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:50.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.487+0000 7f21e27fc700 1 --2- 172.21.15.16:0/1739838680 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f21c406ace0 0x7f21c406d190 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f21e40d0590 tx=0x7f21d4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:50.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.491+0000 7f21db7fe700 1 -- 172.21.15.16:0/1739838680 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f21cc055c60 con 0x7f21e40b1b00 2023-05-20T01:17:50.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.735+0000 7f21e948f700 1 -- 172.21.15.16:0/1739838680 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f21e415ada0 con 0x7f21e40b1b00 2023-05-20T01:17:50.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.735+0000 7f21db7fe700 1 -- 172.21.15.16:0/1739838680 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4674 (secure 0 0 0) 0x7f21cc015090 con 0x7f21e40b1b00 2023-05-20T01:17:50.739 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:17:50.739 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5176,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/4144439890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4144439890},{"type":"v1","addr":"172.21.15.16:6836","nonce":4144439890}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5188,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/2992755084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":2992755084},{"type":"v1","addr":"172.21.15.132:6834","nonce":2992755084}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/1979284857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1979284857},{"type":"v1","addr":"172.21.15.132:6835","nonce":1979284857}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-20T01:17:46.995921+0000","modified":"2023-05-20T01:17:50.320558+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5397},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5397":{"gid":5397,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.16:6837/942011026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":942011026},{"type":"v1","addr":"172.21.15.16:6837","nonce":942011026}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-05-20T01:17:50.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.738+0000 7f21e948f700 1 -- 172.21.15.16:0/1739838680 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f21c406ace0 msgr2=0x7f21c406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:50.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.738+0000 7f21e948f700 1 --2- 172.21.15.16:0/1739838680 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f21c406ace0 0x7f21c406d190 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f21e40d0590 tx=0x7f21d4008040 comp rx=0 tx=0).stop 2023-05-20T01:17:50.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.738+0000 7f21e948f700 1 -- 172.21.15.16:0/1739838680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21e40b1b00 msgr2=0x7f21e415bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:50.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.738+0000 7f21e948f700 1 --2- 172.21.15.16:0/1739838680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21e40b1b00 0x7f21e415bcc0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f21cc014040 tx=0x7f21cc004750 comp rx=0 tx=0).stop 2023-05-20T01:17:50.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.738+0000 7f21e948f700 1 -- 172.21.15.16:0/1739838680 shutdown_connections 2023-05-20T01:17:50.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.738+0000 7f21e948f700 1 --2- 172.21.15.16:0/1739838680 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f21c406ace0 0x7f21c406d190 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:50.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.738+0000 7f21e948f700 1 --2- 172.21.15.16:0/1739838680 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f21e4159000 0x7f21e415af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:50.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.738+0000 7f21e948f700 1 --2- 172.21.15.16:0/1739838680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21e40b1b00 0x7f21e415bcc0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:50.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.738+0000 7f21e948f700 1 --2- 172.21.15.16:0/1739838680 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f21e40cb320 0x7f21e4147330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:50.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.738+0000 7f21e948f700 1 -- 172.21.15.16:0/1739838680 >> 172.21.15.16:0/1739838680 conn(0x7f21e40661b0 msgr2=0x7f21e40cfac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:50.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.738+0000 7f21e948f700 1 -- 172.21.15.16:0/1739838680 shutdown_connections 2023-05-20T01:17:50.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.738+0000 7f21e948f700 1 -- 172.21.15.16:0/1739838680 wait complete. 2023-05-20T01:17:50.743 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 16 2023-05-20T01:17:50.754 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-20T01:17:50.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.875+0000 7f83aea2b700 1 Processor -- start 2023-05-20T01:17:50.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.875+0000 7f83aea2b700 1 -- start start 2023-05-20T01:17:50.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.876+0000 7f83aea2b700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f83a8157190 0x7f83a80cc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:50.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.876+0000 7f83aea2b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f83a80cccf0 0x7f83a80713c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:50.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.876+0000 7f83aea2b700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83a80cb590 con 0x7f83a8071900 2023-05-20T01:17:50.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.876+0000 7f83aea2b700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83a80ca9a0 con 0x7f83a80cccf0 2023-05-20T01:17:50.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.876+0000 7f83aea2b700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83a80cac60 con 0x7f83a8157190 2023-05-20T01:17:50.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.876+0000 7f83a77fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f83a80cccf0 0x7f83a80713c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:50.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.876+0000 7f83acfc8700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f83a8071900 0x7f83a815d220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:43552/0 (socket says 172.21.15.16:43552) 2023-05-20T01:17:50.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.876+0000 7f83a77fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f83a80cccf0 0x7f83a80713c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60254/0 (socket says 172.21.15.16:60254) 2023-05-20T01:17:50.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.876+0000 7f83acfc8700 1 -- 172.21.15.16:0/1549802419 learned_addr learned my addr 172.21.15.16:0/1549802419 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:17:50.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.876+0000 7f83a77fe700 1 -- 172.21.15.16:0/1549802419 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f83a8157190 msgr2=0x7f83a80cc7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:50.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.876+0000 7f83a7fff700 1 --2- 172.21.15.16:0/1549802419 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f83a8157190 0x7f83a80cc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:50.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.876+0000 7f83a77fe700 1 --2- 172.21.15.16:0/1549802419 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f83a8157190 0x7f83a80cc7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:50.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.876+0000 7f83a6ffd700 1 -- 172.21.15.16:0/1549802419 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4249742270 0 0) 0x7f83a80cb590 con 0x7f83a8071900 2023-05-20T01:17:50.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.876+0000 7f83a77fe700 1 -- 172.21.15.16:0/1549802419 >> v1:172.21.15.16:6789/0 conn(0x7f83a8071900 legacy=0x7f83a815d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:50.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.876+0000 7f83a77fe700 1 -- 172.21.15.16:0/1549802419 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83a80caf40 con 0x7f83a80cccf0 2023-05-20T01:17:50.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.876+0000 7f83a7fff700 1 --2- 172.21.15.16:0/1549802419 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f83a8157190 0x7f83a80cc7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:17:50.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.877+0000 7f83a77fe700 1 --2- 172.21.15.16:0/1549802419 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83a80cccf0 0x7f83a80713c0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f8398009840 tx=0x7f8398009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e2b5a82a7dafc88 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:50.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.877+0000 7f83a6ffd700 1 -- 172.21.15.16:0/1549802419 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8398011830 con 0x7f83a80cccf0 2023-05-20T01:17:50.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.877+0000 7f83a6ffd700 1 -- 172.21.15.16:0/1549802419 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8398011990 con 0x7f83a80cccf0 2023-05-20T01:17:50.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.877+0000 7f83a6ffd700 1 -- 172.21.15.16:0/1549802419 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8398011d20 con 0x7f83a80cccf0 2023-05-20T01:17:50.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.877+0000 7f83aea2b700 1 -- 172.21.15.16:0/1549802419 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83a80cccf0 msgr2=0x7f83a80713c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:50.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.877+0000 7f83aea2b700 1 --2- 172.21.15.16:0/1549802419 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83a80cccf0 0x7f83a80713c0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f8398009840 tx=0x7f8398009b50 comp rx=0 tx=0).stop 2023-05-20T01:17:50.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.877+0000 7f83aea2b700 1 -- 172.21.15.16:0/1549802419 shutdown_connections 2023-05-20T01:17:50.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.877+0000 7f83aea2b700 1 --2- 172.21.15.16:0/1549802419 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f83a8157190 0x7f83a80cc7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:50.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.877+0000 7f83aea2b700 1 --2- 172.21.15.16:0/1549802419 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83a80cccf0 0x7f83a80713c0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:50.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.877+0000 7f83aea2b700 1 -- 172.21.15.16:0/1549802419 >> 172.21.15.16:0/1549802419 conn(0x7f83a80cffb0 msgr2=0x7f83a80d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:50.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.877+0000 7f83aea2b700 1 -- 172.21.15.16:0/1549802419 shutdown_connections 2023-05-20T01:17:50.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.877+0000 7f83aea2b700 1 -- 172.21.15.16:0/1549802419 wait complete. 2023-05-20T01:17:50.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.878+0000 7f83aea2b700 1 Processor -- start 2023-05-20T01:17:50.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.879+0000 7f83aea2b700 1 -- start start 2023-05-20T01:17:50.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.879+0000 7f83aea2b700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f83a8071900 0x7f83a81ee950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:50.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.879+0000 7f83aea2b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83a80cccf0 0x7f83a80c60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:50.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.879+0000 7f83aea2b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f83a8157190 0x7f83a80b3d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:50.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.879+0000 7f83aea2b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83a80c6920 con 0x7f83a80cccf0 2023-05-20T01:17:50.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.879+0000 7f83aea2b700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83a80c3c10 con 0x7f83a8157190 2023-05-20T01:17:50.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.879+0000 7f83aea2b700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83a80c3eb0 con 0x7f83a8071900 2023-05-20T01:17:50.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.880+0000 7f83a77fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83a80cccf0 0x7f83a80c60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:50.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.880+0000 7f83a77fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83a80cccf0 0x7f83a80c60d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60260/0 (socket says 172.21.15.16:60260) 2023-05-20T01:17:50.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.880+0000 7f83a77fe700 1 -- 172.21.15.16:0/3368676813 learned_addr learned my addr 172.21.15.16:0/3368676813 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:50.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.880+0000 7f83a7fff700 1 --2- 172.21.15.16:0/3368676813 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f83a8071900 0x7f83a81ee950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:50.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.880+0000 7f83acfc8700 1 --2- 172.21.15.16:0/3368676813 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f83a8157190 0x7f83a80b3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:50.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.880+0000 7f83a77fe700 1 -- 172.21.15.16:0/3368676813 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f83a8071900 msgr2=0x7f83a81ee950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:50.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.880+0000 7f83a77fe700 1 --2- 172.21.15.16:0/3368676813 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f83a8071900 0x7f83a81ee950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:50.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.880+0000 7f83a77fe700 1 -- 172.21.15.16:0/3368676813 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f83a8157190 msgr2=0x7f83a80b3d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:50.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.880+0000 7f83a77fe700 1 --2- 172.21.15.16:0/3368676813 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f83a8157190 0x7f83a80b3d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:50.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.880+0000 7f83a77fe700 1 -- 172.21.15.16:0/3368676813 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83980094a0 con 0x7f83a80cccf0 2023-05-20T01:17:50.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.880+0000 7f83acfc8700 1 --2- 172.21.15.16:0/3368676813 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f83a8157190 0x7f83a80b3d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:17:50.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.880+0000 7f83a77fe700 1 --2- 172.21.15.16:0/3368676813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83a80cccf0 0x7f83a80c60d0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f83980090b0 tx=0x7f8398004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:50.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.880+0000 7f83a4ff9700 1 -- 172.21.15.16:0/3368676813 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83980036a0 con 0x7f83a80cccf0 2023-05-20T01:17:50.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.880+0000 7f83a4ff9700 1 -- 172.21.15.16:0/3368676813 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f839801f070 con 0x7f83a80cccf0 2023-05-20T01:17:50.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.880+0000 7f83aea2b700 1 -- 172.21.15.16:0/3368676813 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83a8157560 con 0x7f83a80cccf0 2023-05-20T01:17:50.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.880+0000 7f83a4ff9700 1 -- 172.21.15.16:0/3368676813 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8398003a40 con 0x7f83a80cccf0 2023-05-20T01:17:50.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.880+0000 7f83aea2b700 1 -- 172.21.15.16:0/3368676813 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83a80cb220 con 0x7f83a80cccf0 2023-05-20T01:17:50.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.881+0000 7f83aea2b700 1 -- 172.21.15.16:0/3368676813 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83a80cd120 con 0x7f83a80cccf0 2023-05-20T01:17:50.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.882+0000 7f83a4ff9700 1 -- 172.21.15.16:0/3368676813 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8398003ba0 con 0x7f83a80cccf0 2023-05-20T01:17:50.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.882+0000 7f83a4ff9700 1 --2- 172.21.15.16:0/3368676813 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f839406ace0 0x7f839406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:50.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.882+0000 7f83a4ff9700 1 -- 172.21.15.16:0/3368676813 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f8398021070 con 0x7f83a80cccf0 2023-05-20T01:17:50.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.882+0000 7f83a7fff700 1 --2- 172.21.15.16:0/3368676813 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f839406ace0 0x7f839406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:50.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.882+0000 7f83a7fff700 1 --2- 172.21.15.16:0/3368676813 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f839406ace0 0x7f839406d190 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f839c006fd0 tx=0x7f839c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:50.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:50.886+0000 7f83a4ff9700 1 -- 172.21.15.16:0/3368676813 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f8398056070 con 0x7f83a80cccf0 2023-05-20T01:17:51.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.085+0000 7f83aea2b700 1 -- 172.21.15.16:0/3368676813 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f83a80747b0 con 0x7f83a80cccf0 2023-05-20T01:17:51.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.086+0000 7f83a4ff9700 1 -- 172.21.15.16:0/3368676813 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+18406 (secure 0 0 0) 0x7f8398059690 con 0x7f83a80cccf0 2023-05-20T01:17:51.090 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:17:51.090 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":40,"fsid":"c2245cc5-ecab-4087-a59b-bc81db901676","created":"2023-05-20T01:16:09.549230+0000","modified":"2023-05-20T01:17:47.997603+0000","last_up_change":"2023-05-20T01:16:19.788624+0000","last_in_change":"2023-05-20T01:16:12.092160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-05-20T01:16:20.934366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-05-20T01:16:21.372294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-05-20T01:17:44.628755+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-05-20T01:17:45.968584+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"317a83cd-19c8-4958-a301-6087ef342289","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6825","nonce":3923634179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6827","nonce":3923634179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6831","nonce":3923634179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6829","nonce":3923634179}]},"public_addr":"172.21.15.16:6825/3923634179","cluster_addr":"172.21.15.16:6827/3923634179","heartbeat_back_addr":"172.21.15.16:6831/3923634179","heartbeat_front_addr":"172.21.15.16:6829/3923634179","state":["exists","up"]},{"osd":1,"uuid":"5581046f-8219-4089-8d7d-b7846399505e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6809","nonce":3860264220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6811","nonce":3860264220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6815","nonce":3860264220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6813","nonce":3860264220}]},"public_addr":"172.21.15.16:6809/3860264220","cluster_addr":"172.21.15.16:6811/3860264220","heartbeat_back_addr":"172.21.15.16:6815/3860264220","heartbeat_front_addr":"172.21.15.16:6813/3860264220","state":["exists","up"]},{"osd":2,"uuid":"072e9e66-4133-478e-8d4a-64d46c2963f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6817","nonce":293187030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6819","nonce":293187030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6823","nonce":293187030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6821","nonce":293187030}]},"public_addr":"172.21.15.16:6817/293187030","cluster_addr":"172.21.15.16:6819/293187030","heartbeat_back_addr":"172.21.15.16:6823/293187030","heartbeat_front_addr":"172.21.15.16:6821/293187030","state":["exists","up"]},{"osd":3,"uuid":"11e1bba6-00f8-48dd-9028-d8d716b7f145","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6801","nonce":3669781890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6803","nonce":3669781890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6807","nonce":3669781890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6805","nonce":3669781890}]},"public_addr":"172.21.15.16:6801/3669781890","cluster_addr":"172.21.15.16:6803/3669781890","heartbeat_back_addr":"172.21.15.16:6807/3669781890","heartbeat_front_addr":"172.21.15.16:6805/3669781890","state":["exists","up"]},{"osd":4,"uuid":"b8909c3a-cf70-46cb-8d0b-009c707cf9d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6819","nonce":506317223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6823","nonce":506317223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6829","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6831","nonce":506317223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6827","nonce":506317223}]},"public_addr":"172.21.15.132:6819/506317223","cluster_addr":"172.21.15.132:6823/506317223","heartbeat_back_addr":"172.21.15.132:6831/506317223","heartbeat_front_addr":"172.21.15.132:6827/506317223","state":["exists","up"]},{"osd":5,"uuid":"b1818304-4583-47bf-a6ec-0bfbb5166233","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6818","nonce":1844689995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6822","nonce":1844689995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6830","nonce":1844689995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6826","nonce":1844689995}]},"public_addr":"172.21.15.132:6818/1844689995","cluster_addr":"172.21.15.132:6822/1844689995","heartbeat_back_addr":"172.21.15.132:6830/1844689995","heartbeat_front_addr":"172.21.15.132:6826/1844689995","state":["exists","up"]},{"osd":6,"uuid":"d37f4691-eed1-4a93-8818-18354f03f4a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6801","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6802","nonce":727581123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6805","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6807","nonce":727581123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6813","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6815","nonce":727581123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6811","nonce":727581123}]},"public_addr":"172.21.15.132:6802/727581123","cluster_addr":"172.21.15.132:6807/727581123","heartbeat_back_addr":"172.21.15.132:6815/727581123","heartbeat_front_addr":"172.21.15.132:6811/727581123","state":["exists","up"]},{"osd":7,"uuid":"4f712f62-5d1d-4816-8042-03209144f8b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6803","nonce":466001149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6806","nonce":466001149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6814","nonce":466001149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6810","nonce":466001149}]},"public_addr":"172.21.15.132:6803/466001149","cluster_addr":"172.21.15.132:6806/466001149","heartbeat_back_addr":"172.21.15.132:6814/466001149","heartbeat_front_addr":"172.21.15.132:6810/466001149","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.338863+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.278680+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305694+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.246835+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.276237+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.304441+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305421+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.267050+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-20T01:17:51.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.088+0000 7f83aea2b700 1 -- 172.21.15.16:0/3368676813 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f839406ace0 msgr2=0x7f839406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:51.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.088+0000 7f83aea2b700 1 --2- 172.21.15.16:0/3368676813 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f839406ace0 0x7f839406d190 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f839c006fd0 tx=0x7f839c008040 comp rx=0 tx=0).stop 2023-05-20T01:17:51.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.088+0000 7f83aea2b700 1 -- 172.21.15.16:0/3368676813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83a80cccf0 msgr2=0x7f83a80c60d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:51.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.088+0000 7f83aea2b700 1 --2- 172.21.15.16:0/3368676813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83a80cccf0 0x7f83a80c60d0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f83980090b0 tx=0x7f8398004750 comp rx=0 tx=0).stop 2023-05-20T01:17:51.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.088+0000 7f83aea2b700 1 -- 172.21.15.16:0/3368676813 shutdown_connections 2023-05-20T01:17:51.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.088+0000 7f83aea2b700 1 --2- 172.21.15.16:0/3368676813 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f83a8157190 0x7f83a80b3d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:51.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.088+0000 7f83aea2b700 1 --2- 172.21.15.16:0/3368676813 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f839406ace0 0x7f839406d190 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:51.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.088+0000 7f83aea2b700 1 --2- 172.21.15.16:0/3368676813 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f83a8071900 0x7f83a81ee950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:51.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.088+0000 7f83aea2b700 1 --2- 172.21.15.16:0/3368676813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83a80cccf0 0x7f83a80c60d0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:51.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.088+0000 7f83aea2b700 1 -- 172.21.15.16:0/3368676813 >> 172.21.15.16:0/3368676813 conn(0x7f83a80cffb0 msgr2=0x7f83a815a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:51.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.088+0000 7f83aea2b700 1 -- 172.21.15.16:0/3368676813 shutdown_connections 2023-05-20T01:17:51.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.088+0000 7f83aea2b700 1 -- 172.21.15.16:0/3368676813 wait complete. 2023-05-20T01:17:51.104 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-05-20T01:17:51.105 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-05-20T01:17:51.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.226+0000 7f35bfdc1700 1 Processor -- start 2023-05-20T01:17:51.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.226+0000 7f35bfdc1700 1 -- start start 2023-05-20T01:17:51.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.227+0000 7f35bfdc1700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f35b815e2a0 0x7f35b80cf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:51.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.227+0000 7f35bfdc1700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f35b80cfb30 0x7f35b8071340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:51.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.227+0000 7f35bfdc1700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35b815e060 con 0x7f35b806a510 2023-05-20T01:17:51.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.227+0000 7f35bfdc1700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35b815d450 con 0x7f35b815e2a0 2023-05-20T01:17:51.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.227+0000 7f35bfdc1700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35b815d730 con 0x7f35b80cfb30 2023-05-20T01:17:51.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.227+0000 7f35bd35c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f35b815e2a0 0x7f35b80cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:51.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.227+0000 7f35bdb5d700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f35b806a510 0x7f35b806a8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:43562/0 (socket says 172.21.15.16:43562) 2023-05-20T01:17:51.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.227+0000 7f35bdb5d700 1 -- 172.21.15.16:0/799184862 learned_addr learned my addr 172.21.15.16:0/799184862 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:17:51.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.227+0000 7f35be35e700 1 --2- 172.21.15.16:0/799184862 >> v2:172.21.15.132:3300/0 conn(0x7f35b80cfb30 0x7f35b8071340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:51.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.227+0000 7f35bd35c700 1 -- 172.21.15.16:0/799184862 >> v2:172.21.15.132:3300/0 conn(0x7f35b80cfb30 msgr2=0x7f35b8071340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:51.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.227+0000 7f35bd35c700 1 --2- 172.21.15.16:0/799184862 >> v2:172.21.15.132:3300/0 conn(0x7f35b80cfb30 0x7f35b8071340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:51.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.227+0000 7f35bd35c700 1 -- 172.21.15.16:0/799184862 >> v1:172.21.15.16:6789/0 conn(0x7f35b806a510 legacy=0x7f35b806a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:51.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.227+0000 7f35bd35c700 1 -- 172.21.15.16:0/799184862 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35b815da10 con 0x7f35b815e2a0 2023-05-20T01:17:51.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.227+0000 7f35bcb5b700 1 -- 172.21.15.16:0/799184862 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 897329303 0 0) 0x7f35b815e060 con 0x7f35b806a510 2023-05-20T01:17:51.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.227+0000 7f35be35e700 1 --2- 172.21.15.16:0/799184862 >> v2:172.21.15.132:3300/0 conn(0x7f35b80cfb30 0x7f35b8071340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:17:51.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.227+0000 7f35bd35c700 1 --2- 172.21.15.16:0/799184862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35b815e2a0 0x7f35b80cf5f0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f35b8066d00 tx=0x7f35a8009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2db63a33797c2 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:51.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.228+0000 7f35bcb5b700 1 -- 172.21.15.16:0/799184862 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a8011830 con 0x7f35b815e2a0 2023-05-20T01:17:51.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.228+0000 7f35bcb5b700 1 -- 172.21.15.16:0/799184862 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35a8011990 con 0x7f35b815e2a0 2023-05-20T01:17:51.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.228+0000 7f35bcb5b700 1 -- 172.21.15.16:0/799184862 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a8011d20 con 0x7f35b815e2a0 2023-05-20T01:17:51.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.228+0000 7f35bfdc1700 1 -- 172.21.15.16:0/799184862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35b815e2a0 msgr2=0x7f35b80cf5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:51.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.228+0000 7f35bfdc1700 1 --2- 172.21.15.16:0/799184862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35b815e2a0 0x7f35b80cf5f0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f35b8066d00 tx=0x7f35a8009b20 comp rx=0 tx=0).stop 2023-05-20T01:17:51.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.228+0000 7f35bfdc1700 1 -- 172.21.15.16:0/799184862 shutdown_connections 2023-05-20T01:17:51.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.228+0000 7f35bfdc1700 1 --2- 172.21.15.16:0/799184862 >> v2:172.21.15.132:3300/0 conn(0x7f35b80cfb30 0x7f35b8071340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:51.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.228+0000 7f35bfdc1700 1 --2- 172.21.15.16:0/799184862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35b815e2a0 0x7f35b80cf5f0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:51.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.228+0000 7f35bfdc1700 1 -- 172.21.15.16:0/799184862 >> 172.21.15.16:0/799184862 conn(0x7f35b80d0c30 msgr2=0x7f35b80b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:51.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.228+0000 7f35bfdc1700 1 -- 172.21.15.16:0/799184862 shutdown_connections 2023-05-20T01:17:51.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.228+0000 7f35bfdc1700 1 -- 172.21.15.16:0/799184862 wait complete. 2023-05-20T01:17:51.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.229+0000 7f35bfdc1700 1 Processor -- start 2023-05-20T01:17:51.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.230+0000 7f35bfdc1700 1 -- start start 2023-05-20T01:17:51.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.230+0000 7f35bfdc1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35b806a510 0x7f35b8153450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:51.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.230+0000 7f35bfdc1700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f35b80cfb30 0x7f35b80b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:51.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.230+0000 7f35bfdc1700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f35b815e2a0 0x7f35b80c6340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:51.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.230+0000 7f35bfdc1700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35b80c50c0 con 0x7f35b806a510 2023-05-20T01:17:51.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.230+0000 7f35bfdc1700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35b80c52a0 con 0x7f35b80cfb30 2023-05-20T01:17:51.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.230+0000 7f35bfdc1700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35b80c5580 con 0x7f35b815e2a0 2023-05-20T01:17:51.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.230+0000 7f35bdb5d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35b806a510 0x7f35b8153450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:51.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.230+0000 7f35bdb5d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35b806a510 0x7f35b8153450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60272/0 (socket says 172.21.15.16:60272) 2023-05-20T01:17:51.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.230+0000 7f35bdb5d700 1 -- 172.21.15.16:0/4107625713 learned_addr learned my addr 172.21.15.16:0/4107625713 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:51.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.230+0000 7f35bd35c700 1 --2- 172.21.15.16:0/4107625713 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f35b80cfb30 0x7f35b80b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:51.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.231+0000 7f35be35e700 1 --2- 172.21.15.16:0/4107625713 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f35b815e2a0 0x7f35b80c6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:51.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.231+0000 7f35bdb5d700 1 -- 172.21.15.16:0/4107625713 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f35b815e2a0 msgr2=0x7f35b80c6340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:51.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.231+0000 7f35bdb5d700 1 --2- 172.21.15.16:0/4107625713 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f35b815e2a0 0x7f35b80c6340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:51.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.231+0000 7f35bdb5d700 1 -- 172.21.15.16:0/4107625713 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f35b80cfb30 msgr2=0x7f35b80b4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:51.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.231+0000 7f35bdb5d700 1 --2- 172.21.15.16:0/4107625713 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f35b80cfb30 0x7f35b80b4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:51.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.231+0000 7f35bdb5d700 1 -- 172.21.15.16:0/4107625713 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35a80094a0 con 0x7f35b806a510 2023-05-20T01:17:51.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.231+0000 7f35be35e700 1 --2- 172.21.15.16:0/4107625713 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f35b815e2a0 0x7f35b80c6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:17:51.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.231+0000 7f35bdb5d700 1 --2- 172.21.15.16:0/4107625713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35b806a510 0x7f35b8153450 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f35b40011e0 tx=0x7f35b400ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:51.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.231+0000 7f35ae7fc700 1 -- 172.21.15.16:0/4107625713 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35b4004c80 con 0x7f35b806a510 2023-05-20T01:17:51.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.231+0000 7f35ae7fc700 1 -- 172.21.15.16:0/4107625713 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35b4004de0 con 0x7f35b806a510 2023-05-20T01:17:51.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.231+0000 7f35bfdc1700 1 -- 172.21.15.16:0/4107625713 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35b815d080 con 0x7f35b806a510 2023-05-20T01:17:51.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.231+0000 7f35ae7fc700 1 -- 172.21.15.16:0/4107625713 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35b4007310 con 0x7f35b806a510 2023-05-20T01:17:51.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.231+0000 7f35bfdc1700 1 -- 172.21.15.16:0/4107625713 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35b815dcf0 con 0x7f35b806a510 2023-05-20T01:17:51.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.232+0000 7f35bfdc1700 1 -- 172.21.15.16:0/4107625713 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35b815d210 con 0x7f35b806a510 2023-05-20T01:17:51.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.233+0000 7f35ae7fc700 1 -- 172.21.15.16:0/4107625713 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f35b4007470 con 0x7f35b806a510 2023-05-20T01:17:51.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.233+0000 7f35ae7fc700 1 --2- 172.21.15.16:0/4107625713 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f359c06ace0 0x7f359c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:51.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.233+0000 7f35ae7fc700 1 -- 172.21.15.16:0/4107625713 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f35b4089210 con 0x7f35b806a510 2023-05-20T01:17:51.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.233+0000 7f35bd35c700 1 --2- 172.21.15.16:0/4107625713 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f359c06ace0 0x7f359c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:51.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.234+0000 7f35bd35c700 1 --2- 172.21.15.16:0/4107625713 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f359c06ace0 0x7f359c06d190 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f35a80090b0 tx=0x7f35a801c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:51.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.238+0000 7f35ae7fc700 1 -- 172.21.15.16:0/4107625713 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f35b815d210 con 0x7f35b806a510 2023-05-20T01:17:51.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.433+0000 7f35bfdc1700 1 -- 172.21.15.16:0/4107625713 --> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f35b80c5740 con 0x7f359c06ace0 2023-05-20T01:17:51.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.443+0000 7f35ae7fc700 1 -- 172.21.15.16:0/4107625713 <== mgr.4104 v2:172.21.15.16:6832/3817237998 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311338 (secure 0 0 0) 0x7f35a8011c00 con 0x7f359c06ace0 2023-05-20T01:17:51.449 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:17:51.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.446+0000 7f35bfdc1700 1 -- 172.21.15.16:0/4107625713 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f359c06ace0 msgr2=0x7f359c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:51.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.446+0000 7f35bfdc1700 1 --2- 172.21.15.16:0/4107625713 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f359c06ace0 0x7f359c06d190 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f35a80090b0 tx=0x7f35a801c040 comp rx=0 tx=0).stop 2023-05-20T01:17:51.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.446+0000 7f35bfdc1700 1 -- 172.21.15.16:0/4107625713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35b806a510 msgr2=0x7f35b8153450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:51.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.446+0000 7f35bfdc1700 1 --2- 172.21.15.16:0/4107625713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35b806a510 0x7f35b8153450 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f35b40011e0 tx=0x7f35b400ca30 comp rx=0 tx=0).stop 2023-05-20T01:17:51.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.446+0000 7f35bfdc1700 1 -- 172.21.15.16:0/4107625713 shutdown_connections 2023-05-20T01:17:51.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.446+0000 7f35bfdc1700 1 --2- 172.21.15.16:0/4107625713 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f359c06ace0 0x7f359c06d190 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:51.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.446+0000 7f35bfdc1700 1 --2- 172.21.15.16:0/4107625713 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f35b815e2a0 0x7f35b80c6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:51.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.446+0000 7f35bfdc1700 1 --2- 172.21.15.16:0/4107625713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35b806a510 0x7f35b8153450 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:51.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.446+0000 7f35bfdc1700 1 --2- 172.21.15.16:0/4107625713 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f35b80cfb30 0x7f35b80b4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:51.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.446+0000 7f35bfdc1700 1 -- 172.21.15.16:0/4107625713 >> 172.21.15.16:0/4107625713 conn(0x7f35b80d0c30 msgr2=0x7f35b815ed50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:51.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.446+0000 7f35bfdc1700 1 -- 172.21.15.16:0/4107625713 shutdown_connections 2023-05-20T01:17:51.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.446+0000 7f35bfdc1700 1 -- 172.21.15.16:0/4107625713 wait complete. 2023-05-20T01:17:51.453 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2023-05-20T01:17:51.462 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":79,"stamp":"2023-05-20T01:17:51.335150+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":179,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":71584,"kb_used_data":4832,"kb_used_omap":0,"kb_used_meta":66688,"kb_avail":754903136,"statfs":{"total":773094113280,"available":773020811264,"internally_reserved":0,"allocated":4947968,"data_stored":2849746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68288512},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.769476"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.621144+0000","last_change":"2023-05-20T01:17:47.027759+0000","last_active":"2023-05-20T01:17:49.621144+0000","last_peered":"2023-05-20T01:17:49.621144+0000","last_clean":"2023-05-20T01:17:49.621144+0000","last_became_active":"2023-05-20T01:17:47.027356+0000","last_became_peered":"2023-05-20T01:17:47.027356+0000","last_unstale":"2023-05-20T01:17:49.621144+0000","last_undegraded":"2023-05-20T01:17:49.621144+0000","last_fullsized":"2023-05-20T01:17:49.621144+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:16:39.017026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.620596+0000","last_change":"2023-05-20T01:17:47.025175+0000","last_active":"2023-05-20T01:17:49.620596+0000","last_peered":"2023-05-20T01:17:49.620596+0000","last_clean":"2023-05-20T01:17:49.620596+0000","last_became_active":"2023-05-20T01:17:47.024861+0000","last_became_peered":"2023-05-20T01:17:47.024861+0000","last_unstale":"2023-05-20T01:17:49.620596+0000","last_undegraded":"2023-05-20T01:17:49.620596+0000","last_fullsized":"2023-05-20T01:17:49.620596+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:06:19.322637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:48.004778+0000","last_change":"2023-05-20T01:17:47.024627+0000","last_active":"2023-05-20T01:17:48.004778+0000","last_peered":"2023-05-20T01:17:48.004778+0000","last_clean":"2023-05-20T01:17:48.004778+0000","last_became_active":"2023-05-20T01:17:47.024330+0000","last_became_peered":"2023-05-20T01:17:47.024330+0000","last_unstale":"2023-05-20T01:17:48.004778+0000","last_undegraded":"2023-05-20T01:17:48.004778+0000","last_fullsized":"2023-05-20T01:17:48.004778+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:55:56.550858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.119708+0000","last_change":"2023-05-20T01:17:47.017256+0000","last_active":"2023-05-20T01:17:49.119708+0000","last_peered":"2023-05-20T01:17:49.119708+0000","last_clean":"2023-05-20T01:17:49.119708+0000","last_became_active":"2023-05-20T01:17:47.014824+0000","last_became_peered":"2023-05-20T01:17:47.014824+0000","last_unstale":"2023-05-20T01:17:49.119708+0000","last_undegraded":"2023-05-20T01:17:49.119708+0000","last_fullsized":"2023-05-20T01:17:49.119708+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:25:43.698565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.620514+0000","last_change":"2023-05-20T01:17:47.025575+0000","last_active":"2023-05-20T01:17:49.620514+0000","last_peered":"2023-05-20T01:17:49.620514+0000","last_clean":"2023-05-20T01:17:49.620514+0000","last_became_active":"2023-05-20T01:17:47.025298+0000","last_became_peered":"2023-05-20T01:17:47.025298+0000","last_unstale":"2023-05-20T01:17:49.620514+0000","last_undegraded":"2023-05-20T01:17:49.620514+0000","last_fullsized":"2023-05-20T01:17:49.620514+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:55:14.133662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:48.004347+0000","last_change":"2023-05-20T01:17:47.019455+0000","last_active":"2023-05-20T01:17:48.004347+0000","last_peered":"2023-05-20T01:17:48.004347+0000","last_clean":"2023-05-20T01:17:48.004347+0000","last_became_active":"2023-05-20T01:17:47.019187+0000","last_became_peered":"2023-05-20T01:17:47.019187+0000","last_unstale":"2023-05-20T01:17:48.004347+0000","last_undegraded":"2023-05-20T01:17:48.004347+0000","last_fullsized":"2023-05-20T01:17:48.004347+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T13:15:38.249014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.621253+0000","last_change":"2023-05-20T01:17:47.030082+0000","last_active":"2023-05-20T01:17:49.621253+0000","last_peered":"2023-05-20T01:17:49.621253+0000","last_clean":"2023-05-20T01:17:49.621253+0000","last_became_active":"2023-05-20T01:17:47.029780+0000","last_became_peered":"2023-05-20T01:17:47.029780+0000","last_unstale":"2023-05-20T01:17:49.621253+0000","last_undegraded":"2023-05-20T01:17:49.621253+0000","last_fullsized":"2023-05-20T01:17:49.621253+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T01:25:43.080234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.119266+0000","last_change":"2023-05-20T01:17:47.023514+0000","last_active":"2023-05-20T01:17:49.119266+0000","last_peered":"2023-05-20T01:17:49.119266+0000","last_clean":"2023-05-20T01:17:49.119266+0000","last_became_active":"2023-05-20T01:17:47.023158+0000","last_became_peered":"2023-05-20T01:17:47.023158+0000","last_unstale":"2023-05-20T01:17:49.119266+0000","last_undegraded":"2023-05-20T01:17:49.119266+0000","last_fullsized":"2023-05-20T01:17:49.119266+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:41:24.500547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.119766+0000","last_change":"2023-05-20T01:17:47.025333+0000","last_active":"2023-05-20T01:17:49.119766+0000","last_peered":"2023-05-20T01:17:49.119766+0000","last_clean":"2023-05-20T01:17:49.119766+0000","last_became_active":"2023-05-20T01:17:47.025084+0000","last_became_peered":"2023-05-20T01:17:47.025084+0000","last_unstale":"2023-05-20T01:17:49.119766+0000","last_undegraded":"2023-05-20T01:17:49.119766+0000","last_fullsized":"2023-05-20T01:17:49.119766+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:45:39.228191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.119814+0000","last_change":"2023-05-20T01:17:47.022289+0000","last_active":"2023-05-20T01:17:49.119814+0000","last_peered":"2023-05-20T01:17:49.119814+0000","last_clean":"2023-05-20T01:17:49.119814+0000","last_became_active":"2023-05-20T01:17:47.021974+0000","last_became_peered":"2023-05-20T01:17:47.021974+0000","last_unstale":"2023-05-20T01:17:49.119814+0000","last_undegraded":"2023-05-20T01:17:49.119814+0000","last_fullsized":"2023-05-20T01:17:49.119814+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:21:10.992631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.119638+0000","last_change":"2023-05-20T01:17:47.026261+0000","last_active":"2023-05-20T01:17:49.119638+0000","last_peered":"2023-05-20T01:17:49.119638+0000","last_clean":"2023-05-20T01:17:49.119638+0000","last_became_active":"2023-05-20T01:17:47.026015+0000","last_became_peered":"2023-05-20T01:17:47.026015+0000","last_unstale":"2023-05-20T01:17:49.119638+0000","last_undegraded":"2023-05-20T01:17:49.119638+0000","last_fullsized":"2023-05-20T01:17:49.119638+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:28:06.439158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.621257+0000","last_change":"2023-05-20T01:17:47.027226+0000","last_active":"2023-05-20T01:17:49.621257+0000","last_peered":"2023-05-20T01:17:49.621257+0000","last_clean":"2023-05-20T01:17:49.621257+0000","last_became_active":"2023-05-20T01:17:47.026658+0000","last_became_peered":"2023-05-20T01:17:47.026658+0000","last_unstale":"2023-05-20T01:17:49.621257+0000","last_undegraded":"2023-05-20T01:17:49.621257+0000","last_fullsized":"2023-05-20T01:17:49.621257+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:37:48.231424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.620215+0000","last_change":"2023-05-20T01:17:47.033484+0000","last_active":"2023-05-20T01:17:49.620215+0000","last_peered":"2023-05-20T01:17:49.620215+0000","last_clean":"2023-05-20T01:17:49.620215+0000","last_became_active":"2023-05-20T01:17:47.033223+0000","last_became_peered":"2023-05-20T01:17:47.033223+0000","last_unstale":"2023-05-20T01:17:49.620215+0000","last_undegraded":"2023-05-20T01:17:49.620215+0000","last_fullsized":"2023-05-20T01:17:49.620215+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:23:44.562996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.119544+0000","last_change":"2023-05-20T01:17:47.025323+0000","last_active":"2023-05-20T01:17:49.119544+0000","last_peered":"2023-05-20T01:17:49.119544+0000","last_clean":"2023-05-20T01:17:49.119544+0000","last_became_active":"2023-05-20T01:17:47.025047+0000","last_became_peered":"2023-05-20T01:17:47.025047+0000","last_unstale":"2023-05-20T01:17:49.119544+0000","last_undegraded":"2023-05-20T01:17:49.119544+0000","last_fullsized":"2023-05-20T01:17:49.119544+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:25:18.769633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:48.004170+0000","last_change":"2023-05-20T01:17:47.010225+0000","last_active":"2023-05-20T01:17:48.004170+0000","last_peered":"2023-05-20T01:17:48.004170+0000","last_clean":"2023-05-20T01:17:48.004170+0000","last_became_active":"2023-05-20T01:17:47.009992+0000","last_became_peered":"2023-05-20T01:17:47.009992+0000","last_unstale":"2023-05-20T01:17:48.004170+0000","last_undegraded":"2023-05-20T01:17:48.004170+0000","last_fullsized":"2023-05-20T01:17:48.004170+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:49:25.854997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:48.007596+0000","last_change":"2023-05-20T01:17:47.024748+0000","last_active":"2023-05-20T01:17:48.007596+0000","last_peered":"2023-05-20T01:17:48.007596+0000","last_clean":"2023-05-20T01:17:48.007596+0000","last_became_active":"2023-05-20T01:17:47.024336+0000","last_became_peered":"2023-05-20T01:17:47.024336+0000","last_unstale":"2023-05-20T01:17:48.007596+0000","last_undegraded":"2023-05-20T01:17:48.007596+0000","last_fullsized":"2023-05-20T01:17:48.007596+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T01:44:58.149648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:48.005193+0000","last_change":"2023-05-20T01:17:47.016869+0000","last_active":"2023-05-20T01:17:48.005193+0000","last_peered":"2023-05-20T01:17:48.005193+0000","last_clean":"2023-05-20T01:17:48.005193+0000","last_became_active":"2023-05-20T01:17:47.016586+0000","last_became_peered":"2023-05-20T01:17:47.016586+0000","last_unstale":"2023-05-20T01:17:48.005193+0000","last_undegraded":"2023-05-20T01:17:48.005193+0000","last_fullsized":"2023-05-20T01:17:48.005193+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:54:49.002366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:48.004857+0000","last_change":"2023-05-20T01:17:47.026994+0000","last_active":"2023-05-20T01:17:48.004857+0000","last_peered":"2023-05-20T01:17:48.004857+0000","last_clean":"2023-05-20T01:17:48.004857+0000","last_became_active":"2023-05-20T01:17:47.026745+0000","last_became_peered":"2023-05-20T01:17:47.026745+0000","last_unstale":"2023-05-20T01:17:48.004857+0000","last_undegraded":"2023-05-20T01:17:48.004857+0000","last_fullsized":"2023-05-20T01:17:48.004857+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:39:06.671661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:48.007492+0000","last_change":"2023-05-20T01:17:47.023040+0000","last_active":"2023-05-20T01:17:48.007492+0000","last_peered":"2023-05-20T01:17:48.007492+0000","last_clean":"2023-05-20T01:17:48.007492+0000","last_became_active":"2023-05-20T01:17:47.022746+0000","last_became_peered":"2023-05-20T01:17:47.022746+0000","last_unstale":"2023-05-20T01:17:48.007492+0000","last_undegraded":"2023-05-20T01:17:48.007492+0000","last_fullsized":"2023-05-20T01:17:48.007492+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:43:28.693854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.620384+0000","last_change":"2023-05-20T01:17:47.023278+0000","last_active":"2023-05-20T01:17:49.620384+0000","last_peered":"2023-05-20T01:17:49.620384+0000","last_clean":"2023-05-20T01:17:49.620384+0000","last_became_active":"2023-05-20T01:17:47.023004+0000","last_became_peered":"2023-05-20T01:17:47.023004+0000","last_unstale":"2023-05-20T01:17:49.620384+0000","last_undegraded":"2023-05-20T01:17:49.620384+0000","last_fullsized":"2023-05-20T01:17:49.620384+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:42:10.169774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:48.004306+0000","last_change":"2023-05-20T01:17:47.011827+0000","last_active":"2023-05-20T01:17:48.004306+0000","last_peered":"2023-05-20T01:17:48.004306+0000","last_clean":"2023-05-20T01:17:48.004306+0000","last_became_active":"2023-05-20T01:17:47.011604+0000","last_became_peered":"2023-05-20T01:17:47.011604+0000","last_unstale":"2023-05-20T01:17:48.004306+0000","last_undegraded":"2023-05-20T01:17:48.004306+0000","last_fullsized":"2023-05-20T01:17:48.004306+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:18:29.739342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.119515+0000","last_change":"2023-05-20T01:17:47.021320+0000","last_active":"2023-05-20T01:17:49.119515+0000","last_peered":"2023-05-20T01:17:49.119515+0000","last_clean":"2023-05-20T01:17:49.119515+0000","last_became_active":"2023-05-20T01:17:47.020983+0000","last_became_peered":"2023-05-20T01:17:47.020983+0000","last_unstale":"2023-05-20T01:17:49.119515+0000","last_undegraded":"2023-05-20T01:17:49.119515+0000","last_fullsized":"2023-05-20T01:17:49.119515+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:10:36.488554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:48.007123+0000","last_change":"2023-05-20T01:17:47.021513+0000","last_active":"2023-05-20T01:17:48.007123+0000","last_peered":"2023-05-20T01:17:48.007123+0000","last_clean":"2023-05-20T01:17:48.007123+0000","last_became_active":"2023-05-20T01:17:47.021002+0000","last_became_peered":"2023-05-20T01:17:47.021002+0000","last_unstale":"2023-05-20T01:17:48.007123+0000","last_undegraded":"2023-05-20T01:17:48.007123+0000","last_fullsized":"2023-05-20T01:17:48.007123+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:39:00.152190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:48.006934+0000","last_change":"2023-05-20T01:17:47.018274+0000","last_active":"2023-05-20T01:17:48.006934+0000","last_peered":"2023-05-20T01:17:48.006934+0000","last_clean":"2023-05-20T01:17:48.006934+0000","last_became_active":"2023-05-20T01:17:47.017902+0000","last_became_peered":"2023-05-20T01:17:47.017902+0000","last_unstale":"2023-05-20T01:17:48.006934+0000","last_undegraded":"2023-05-20T01:17:48.006934+0000","last_fullsized":"2023-05-20T01:17:48.006934+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:03:26.905011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:48.007534+0000","last_change":"2023-05-20T01:17:47.025464+0000","last_active":"2023-05-20T01:17:48.007534+0000","last_peered":"2023-05-20T01:17:48.007534+0000","last_clean":"2023-05-20T01:17:48.007534+0000","last_became_active":"2023-05-20T01:17:47.025153+0000","last_became_peered":"2023-05-20T01:17:47.025153+0000","last_unstale":"2023-05-20T01:17:48.007534+0000","last_undegraded":"2023-05-20T01:17:48.007534+0000","last_fullsized":"2023-05-20T01:17:48.007534+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:43:11.193161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.620691+0000","last_change":"2023-05-20T01:17:47.028617+0000","last_active":"2023-05-20T01:17:49.620691+0000","last_peered":"2023-05-20T01:17:49.620691+0000","last_clean":"2023-05-20T01:17:49.620691+0000","last_became_active":"2023-05-20T01:17:47.028229+0000","last_became_peered":"2023-05-20T01:17:47.028229+0000","last_unstale":"2023-05-20T01:17:49.620691+0000","last_undegraded":"2023-05-20T01:17:49.620691+0000","last_fullsized":"2023-05-20T01:17:49.620691+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:15:27.154301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:48.004877+0000","last_change":"2023-05-20T01:17:47.017010+0000","last_active":"2023-05-20T01:17:48.004877+0000","last_peered":"2023-05-20T01:17:48.004877+0000","last_clean":"2023-05-20T01:17:48.004877+0000","last_became_active":"2023-05-20T01:17:47.016051+0000","last_became_peered":"2023-05-20T01:17:47.016051+0000","last_unstale":"2023-05-20T01:17:48.004877+0000","last_undegraded":"2023-05-20T01:17:48.004877+0000","last_fullsized":"2023-05-20T01:17:48.004877+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:39:11.759347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.620546+0000","last_change":"2023-05-20T01:17:47.027541+0000","last_active":"2023-05-20T01:17:49.620546+0000","last_peered":"2023-05-20T01:17:49.620546+0000","last_clean":"2023-05-20T01:17:49.620546+0000","last_became_active":"2023-05-20T01:17:47.027049+0000","last_became_peered":"2023-05-20T01:17:47.027049+0000","last_unstale":"2023-05-20T01:17:49.620546+0000","last_undegraded":"2023-05-20T01:17:49.620546+0000","last_fullsized":"2023-05-20T01:17:49.620546+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:10:54.768447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:48.007143+0000","last_change":"2023-05-20T01:17:47.019667+0000","last_active":"2023-05-20T01:17:48.007143+0000","last_peered":"2023-05-20T01:17:48.007143+0000","last_clean":"2023-05-20T01:17:48.007143+0000","last_became_active":"2023-05-20T01:17:47.019309+0000","last_became_peered":"2023-05-20T01:17:47.019309+0000","last_unstale":"2023-05-20T01:17:48.007143+0000","last_undegraded":"2023-05-20T01:17:48.007143+0000","last_fullsized":"2023-05-20T01:17:48.007143+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:05:38.391599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:48.004969+0000","last_change":"2023-05-20T01:17:47.019041+0000","last_active":"2023-05-20T01:17:48.004969+0000","last_peered":"2023-05-20T01:17:48.004969+0000","last_clean":"2023-05-20T01:17:48.004969+0000","last_became_active":"2023-05-20T01:17:47.018747+0000","last_became_peered":"2023-05-20T01:17:47.018747+0000","last_unstale":"2023-05-20T01:17:48.004969+0000","last_undegraded":"2023-05-20T01:17:48.004969+0000","last_fullsized":"2023-05-20T01:17:48.004969+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:42:07.468768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:48.004300+0000","last_change":"2023-05-20T01:17:47.016476+0000","last_active":"2023-05-20T01:17:48.004300+0000","last_peered":"2023-05-20T01:17:48.004300+0000","last_clean":"2023-05-20T01:17:48.004300+0000","last_became_active":"2023-05-20T01:17:47.016213+0000","last_became_peered":"2023-05-20T01:17:47.016213+0000","last_unstale":"2023-05-20T01:17:48.004300+0000","last_undegraded":"2023-05-20T01:17:48.004300+0000","last_fullsized":"2023-05-20T01:17:48.004300+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:50:09.284608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.620403+0000","last_change":"2023-05-20T01:17:47.025453+0000","last_active":"2023-05-20T01:17:49.620403+0000","last_peered":"2023-05-20T01:17:49.620403+0000","last_clean":"2023-05-20T01:17:49.620403+0000","last_became_active":"2023-05-20T01:17:47.025175+0000","last_became_peered":"2023-05-20T01:17:47.025175+0000","last_unstale":"2023-05-20T01:17:49.620403+0000","last_undegraded":"2023-05-20T01:17:49.620403+0000","last_fullsized":"2023-05-20T01:17:49.620403+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:15:44.743926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.620613+0000","last_change":"2023-05-20T01:17:47.015774+0000","last_active":"2023-05-20T01:17:49.620613+0000","last_peered":"2023-05-20T01:17:49.620613+0000","last_clean":"2023-05-20T01:17:49.620613+0000","last_became_active":"2023-05-20T01:17:47.015521+0000","last_became_peered":"2023-05-20T01:17:47.015521+0000","last_unstale":"2023-05-20T01:17:49.620613+0000","last_undegraded":"2023-05-20T01:17:49.620613+0000","last_fullsized":"2023-05-20T01:17:49.620613+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T01:55:04.263460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"39'4","reported_seq":17,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:48.005118+0000","last_change":"2023-05-20T01:17:46.588592+0000","last_active":"2023-05-20T01:17:48.005118+0000","last_peered":"2023-05-20T01:17:48.005118+0000","last_clean":"2023-05-20T01:17:48.005118+0000","last_became_active":"2023-05-20T01:17:46.588300+0000","last_became_peered":"2023-05-20T01:17:46.588300+0000","last_unstale":"2023-05-20T01:17:48.005118+0000","last_undegraded":"2023-05-20T01:17:48.005118+0000","last_fullsized":"2023-05-20T01:17:48.005118+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_clean_scrub_stamp":"2023-05-20T01:17:45.555502+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:39:05.661645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:48.007422+0000","last_change":"2023-05-20T01:17:47.014452+0000","last_active":"2023-05-20T01:17:48.007422+0000","last_peered":"2023-05-20T01:17:48.007422+0000","last_clean":"2023-05-20T01:17:48.007422+0000","last_became_active":"2023-05-20T01:17:47.014170+0000","last_became_peered":"2023-05-20T01:17:47.014170+0000","last_unstale":"2023-05-20T01:17:48.007422+0000","last_undegraded":"2023-05-20T01:17:48.007422+0000","last_fullsized":"2023-05-20T01:17:48.007422+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:24:43.149420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"39'1","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:48.005214+0000","last_change":"2023-05-20T01:17:46.585652+0000","last_active":"2023-05-20T01:17:48.005214+0000","last_peered":"2023-05-20T01:17:48.005214+0000","last_clean":"2023-05-20T01:17:48.005214+0000","last_became_active":"2023-05-20T01:17:46.585287+0000","last_became_peered":"2023-05-20T01:17:46.585287+0000","last_unstale":"2023-05-20T01:17:48.005214+0000","last_undegraded":"2023-05-20T01:17:48.005214+0000","last_fullsized":"2023-05-20T01:17:48.005214+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_clean_scrub_stamp":"2023-05-20T01:17:45.555502+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T01:18:01.154090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.620942+0000","last_change":"2023-05-20T01:17:47.016222+0000","last_active":"2023-05-20T01:17:49.620942+0000","last_peered":"2023-05-20T01:17:49.620942+0000","last_clean":"2023-05-20T01:17:49.620942+0000","last_became_active":"2023-05-20T01:17:47.015565+0000","last_became_peered":"2023-05-20T01:17:47.015565+0000","last_unstale":"2023-05-20T01:17:49.620942+0000","last_undegraded":"2023-05-20T01:17:49.620942+0000","last_fullsized":"2023-05-20T01:17:49.620942+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:41:24.500547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"39'2","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.620938+0000","last_change":"2023-05-20T01:17:46.587660+0000","last_active":"2023-05-20T01:17:49.620938+0000","last_peered":"2023-05-20T01:17:49.620938+0000","last_clean":"2023-05-20T01:17:49.620938+0000","last_became_active":"2023-05-20T01:17:46.587382+0000","last_became_peered":"2023-05-20T01:17:46.587382+0000","last_unstale":"2023-05-20T01:17:49.620938+0000","last_undegraded":"2023-05-20T01:17:49.620938+0000","last_fullsized":"2023-05-20T01:17:49.620938+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_clean_scrub_stamp":"2023-05-20T01:17:45.555502+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:20:33.225006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.620726+0000","last_change":"2023-05-20T01:17:47.031074+0000","last_active":"2023-05-20T01:17:49.620726+0000","last_peered":"2023-05-20T01:17:49.620726+0000","last_clean":"2023-05-20T01:17:49.620726+0000","last_became_active":"2023-05-20T01:17:47.030758+0000","last_became_peered":"2023-05-20T01:17:47.030758+0000","last_unstale":"2023-05-20T01:17:49.620726+0000","last_undegraded":"2023-05-20T01:17:49.620726+0000","last_fullsized":"2023-05-20T01:17:49.620726+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:28:43.939677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"39'2","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.620967+0000","last_change":"2023-05-20T01:17:46.593757+0000","last_active":"2023-05-20T01:17:49.620967+0000","last_peered":"2023-05-20T01:17:49.620967+0000","last_clean":"2023-05-20T01:17:49.620967+0000","last_became_active":"2023-05-20T01:17:46.593530+0000","last_became_peered":"2023-05-20T01:17:46.593530+0000","last_unstale":"2023-05-20T01:17:49.620967+0000","last_undegraded":"2023-05-20T01:17:49.620967+0000","last_fullsized":"2023-05-20T01:17:49.620967+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_clean_scrub_stamp":"2023-05-20T01:17:45.555502+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:47:22.733289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:48.004895+0000","last_change":"2023-05-20T01:17:47.012723+0000","last_active":"2023-05-20T01:17:48.004895+0000","last_peered":"2023-05-20T01:17:48.004895+0000","last_clean":"2023-05-20T01:17:48.004895+0000","last_became_active":"2023-05-20T01:17:47.012398+0000","last_became_peered":"2023-05-20T01:17:47.012398+0000","last_unstale":"2023-05-20T01:17:48.004895+0000","last_undegraded":"2023-05-20T01:17:48.004895+0000","last_fullsized":"2023-05-20T01:17:48.004895+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:01:57.140023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"39'1","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:48.004407+0000","last_change":"2023-05-20T01:17:46.589991+0000","last_active":"2023-05-20T01:17:48.004407+0000","last_peered":"2023-05-20T01:17:48.004407+0000","last_clean":"2023-05-20T01:17:48.004407+0000","last_became_active":"2023-05-20T01:17:46.589701+0000","last_became_peered":"2023-05-20T01:17:46.589701+0000","last_unstale":"2023-05-20T01:17:48.004407+0000","last_undegraded":"2023-05-20T01:17:48.004407+0000","last_fullsized":"2023-05-20T01:17:48.004407+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_clean_scrub_stamp":"2023-05-20T01:17:45.555502+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:46:21.391999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.119647+0000","last_change":"2023-05-20T01:17:47.025927+0000","last_active":"2023-05-20T01:17:49.119647+0000","last_peered":"2023-05-20T01:17:49.119647+0000","last_clean":"2023-05-20T01:17:49.119647+0000","last_became_active":"2023-05-20T01:17:47.025542+0000","last_became_peered":"2023-05-20T01:17:47.025542+0000","last_unstale":"2023-05-20T01:17:49.119647+0000","last_undegraded":"2023-05-20T01:17:49.119647+0000","last_fullsized":"2023-05-20T01:17:49.119647+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:43:58.959515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"39'1","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:48.005083+0000","last_change":"2023-05-20T01:17:46.586996+0000","last_active":"2023-05-20T01:17:48.005083+0000","last_peered":"2023-05-20T01:17:48.005083+0000","last_clean":"2023-05-20T01:17:48.005083+0000","last_became_active":"2023-05-20T01:17:46.586405+0000","last_became_peered":"2023-05-20T01:17:46.586405+0000","last_unstale":"2023-05-20T01:17:48.005083+0000","last_undegraded":"2023-05-20T01:17:48.005083+0000","last_fullsized":"2023-05-20T01:17:48.005083+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_clean_scrub_stamp":"2023-05-20T01:17:45.555502+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:43:35.810675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.119326+0000","last_change":"2023-05-20T01:17:47.020361+0000","last_active":"2023-05-20T01:17:49.119326+0000","last_peered":"2023-05-20T01:17:49.119326+0000","last_clean":"2023-05-20T01:17:49.119326+0000","last_became_active":"2023-05-20T01:17:47.020127+0000","last_became_peered":"2023-05-20T01:17:47.020127+0000","last_unstale":"2023-05-20T01:17:49.119326+0000","last_undegraded":"2023-05-20T01:17:49.119326+0000","last_fullsized":"2023-05-20T01:17:49.119326+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:33:18.790612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.620822+0000","last_change":"2023-05-20T01:17:46.594101+0000","last_active":"2023-05-20T01:17:49.620822+0000","last_peered":"2023-05-20T01:17:49.620822+0000","last_clean":"2023-05-20T01:17:49.620822+0000","last_became_active":"2023-05-20T01:17:46.593849+0000","last_became_peered":"2023-05-20T01:17:46.593849+0000","last_unstale":"2023-05-20T01:17:49.620822+0000","last_undegraded":"2023-05-20T01:17:49.620822+0000","last_fullsized":"2023-05-20T01:17:49.620822+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_clean_scrub_stamp":"2023-05-20T01:17:45.555502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:21:42.210970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:48.007206+0000","last_change":"2023-05-20T01:17:47.021295+0000","last_active":"2023-05-20T01:17:48.007206+0000","last_peered":"2023-05-20T01:17:48.007206+0000","last_clean":"2023-05-20T01:17:48.007206+0000","last_became_active":"2023-05-20T01:17:47.020577+0000","last_became_peered":"2023-05-20T01:17:47.020577+0000","last_unstale":"2023-05-20T01:17:48.007206+0000","last_undegraded":"2023-05-20T01:17:48.007206+0000","last_fullsized":"2023-05-20T01:17:48.007206+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:33:35.934450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.119417+0000","last_change":"2023-05-20T01:17:46.592808+0000","last_active":"2023-05-20T01:17:49.119417+0000","last_peered":"2023-05-20T01:17:49.119417+0000","last_clean":"2023-05-20T01:17:49.119417+0000","last_became_active":"2023-05-20T01:17:46.592637+0000","last_became_peered":"2023-05-20T01:17:46.592637+0000","last_unstale":"2023-05-20T01:17:49.119417+0000","last_undegraded":"2023-05-20T01:17:49.119417+0000","last_fullsized":"2023-05-20T01:17:49.119417+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_clean_scrub_stamp":"2023-05-20T01:17:45.555502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:44:07.422862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:48.005170+0000","last_change":"2023-05-20T01:17:47.026921+0000","last_active":"2023-05-20T01:17:48.005170+0000","last_peered":"2023-05-20T01:17:48.005170+0000","last_clean":"2023-05-20T01:17:48.005170+0000","last_became_active":"2023-05-20T01:17:47.026679+0000","last_became_peered":"2023-05-20T01:17:47.026679+0000","last_unstale":"2023-05-20T01:17:48.005170+0000","last_undegraded":"2023-05-20T01:17:48.005170+0000","last_fullsized":"2023-05-20T01:17:48.005170+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:50:03.983484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:48.005149+0000","last_change":"2023-05-20T01:17:46.588313+0000","last_active":"2023-05-20T01:17:48.005149+0000","last_peered":"2023-05-20T01:17:48.005149+0000","last_clean":"2023-05-20T01:17:48.005149+0000","last_became_active":"2023-05-20T01:17:46.588041+0000","last_became_peered":"2023-05-20T01:17:46.588041+0000","last_unstale":"2023-05-20T01:17:48.005149+0000","last_undegraded":"2023-05-20T01:17:48.005149+0000","last_fullsized":"2023-05-20T01:17:48.005149+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_clean_scrub_stamp":"2023-05-20T01:17:45.555502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:00:36.579161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.621217+0000","last_change":"2023-05-20T01:17:47.023661+0000","last_active":"2023-05-20T01:17:49.621217+0000","last_peered":"2023-05-20T01:17:49.621217+0000","last_clean":"2023-05-20T01:17:49.621217+0000","last_became_active":"2023-05-20T01:17:47.023405+0000","last_became_peered":"2023-05-20T01:17:47.023405+0000","last_unstale":"2023-05-20T01:17:49.621217+0000","last_undegraded":"2023-05-20T01:17:49.621217+0000","last_fullsized":"2023-05-20T01:17:49.621217+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:04:06.970847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.119904+0000","last_change":"2023-05-20T01:17:46.591266+0000","last_active":"2023-05-20T01:17:49.119904+0000","last_peered":"2023-05-20T01:17:49.119904+0000","last_clean":"2023-05-20T01:17:49.119904+0000","last_became_active":"2023-05-20T01:17:46.590943+0000","last_became_peered":"2023-05-20T01:17:46.590943+0000","last_unstale":"2023-05-20T01:17:49.119904+0000","last_undegraded":"2023-05-20T01:17:49.119904+0000","last_fullsized":"2023-05-20T01:17:49.119904+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_clean_scrub_stamp":"2023-05-20T01:17:45.555502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:13:28.112841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:48.004722+0000","last_change":"2023-05-20T01:17:47.027760+0000","last_active":"2023-05-20T01:17:48.004722+0000","last_peered":"2023-05-20T01:17:48.004722+0000","last_clean":"2023-05-20T01:17:48.004722+0000","last_became_active":"2023-05-20T01:17:47.027495+0000","last_became_peered":"2023-05-20T01:17:47.027495+0000","last_unstale":"2023-05-20T01:17:48.004722+0000","last_undegraded":"2023-05-20T01:17:48.004722+0000","last_fullsized":"2023-05-20T01:17:48.004722+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:34:31.667495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"39'3","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.621192+0000","last_change":"2023-05-20T01:17:46.596339+0000","last_active":"2023-05-20T01:17:49.621192+0000","last_peered":"2023-05-20T01:17:49.621192+0000","last_clean":"2023-05-20T01:17:49.621192+0000","last_became_active":"2023-05-20T01:17:46.596039+0000","last_became_peered":"2023-05-20T01:17:46.596039+0000","last_unstale":"2023-05-20T01:17:49.621192+0000","last_undegraded":"2023-05-20T01:17:49.621192+0000","last_fullsized":"2023-05-20T01:17:49.621192+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_clean_scrub_stamp":"2023-05-20T01:17:45.555502+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:05:04.021904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.621042+0000","last_change":"2023-05-20T01:17:47.028275+0000","last_active":"2023-05-20T01:17:49.621042+0000","last_peered":"2023-05-20T01:17:49.621042+0000","last_clean":"2023-05-20T01:17:49.621042+0000","last_became_active":"2023-05-20T01:17:47.028076+0000","last_became_peered":"2023-05-20T01:17:47.028076+0000","last_unstale":"2023-05-20T01:17:49.621042+0000","last_undegraded":"2023-05-20T01:17:49.621042+0000","last_fullsized":"2023-05-20T01:17:49.621042+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:47:23.552100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"39'6","reported_seq":19,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.620534+0000","last_change":"2023-05-20T01:17:46.588586+0000","last_active":"2023-05-20T01:17:49.620534+0000","last_peered":"2023-05-20T01:17:49.620534+0000","last_clean":"2023-05-20T01:17:49.620534+0000","last_became_active":"2023-05-20T01:17:46.588295+0000","last_became_peered":"2023-05-20T01:17:46.588295+0000","last_unstale":"2023-05-20T01:17:49.620534+0000","last_undegraded":"2023-05-20T01:17:49.620534+0000","last_fullsized":"2023-05-20T01:17:49.620534+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_clean_scrub_stamp":"2023-05-20T01:17:45.555502+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T01:31:12.253501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:48.007250+0000","last_change":"2023-05-20T01:17:47.025107+0000","last_active":"2023-05-20T01:17:48.007250+0000","last_peered":"2023-05-20T01:17:48.007250+0000","last_clean":"2023-05-20T01:17:48.007250+0000","last_became_active":"2023-05-20T01:17:47.024305+0000","last_became_peered":"2023-05-20T01:17:47.024305+0000","last_unstale":"2023-05-20T01:17:48.007250+0000","last_undegraded":"2023-05-20T01:17:48.007250+0000","last_fullsized":"2023-05-20T01:17:48.007250+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:30:01.995835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"39'2","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:48.004661+0000","last_change":"2023-05-20T01:17:46.585732+0000","last_active":"2023-05-20T01:17:48.004661+0000","last_peered":"2023-05-20T01:17:48.004661+0000","last_clean":"2023-05-20T01:17:48.004661+0000","last_became_active":"2023-05-20T01:17:46.585290+0000","last_became_peered":"2023-05-20T01:17:46.585290+0000","last_unstale":"2023-05-20T01:17:48.004661+0000","last_undegraded":"2023-05-20T01:17:48.004661+0000","last_fullsized":"2023-05-20T01:17:48.004661+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_clean_scrub_stamp":"2023-05-20T01:17:45.555502+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:16:03.579796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.119809+0000","last_change":"2023-05-20T01:17:47.018074+0000","last_active":"2023-05-20T01:17:49.119809+0000","last_peered":"2023-05-20T01:17:49.119809+0000","last_clean":"2023-05-20T01:17:49.119809+0000","last_became_active":"2023-05-20T01:17:47.017809+0000","last_became_peered":"2023-05-20T01:17:47.017809+0000","last_unstale":"2023-05-20T01:17:49.119809+0000","last_undegraded":"2023-05-20T01:17:49.119809+0000","last_fullsized":"2023-05-20T01:17:49.119809+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:22:30.348997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.621379+0000","last_change":"2023-05-20T01:17:46.592389+0000","last_active":"2023-05-20T01:17:49.621379+0000","last_peered":"2023-05-20T01:17:49.621379+0000","last_clean":"2023-05-20T01:17:49.621379+0000","last_became_active":"2023-05-20T01:17:46.592100+0000","last_became_peered":"2023-05-20T01:17:46.592100+0000","last_unstale":"2023-05-20T01:17:49.621379+0000","last_undegraded":"2023-05-20T01:17:49.621379+0000","last_fullsized":"2023-05-20T01:17:49.621379+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_clean_scrub_stamp":"2023-05-20T01:17:45.555502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:27:43.895840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:48.004157+0000","last_change":"2023-05-20T01:17:47.027652+0000","last_active":"2023-05-20T01:17:48.004157+0000","last_peered":"2023-05-20T01:17:48.004157+0000","last_clean":"2023-05-20T01:17:48.004157+0000","last_became_active":"2023-05-20T01:17:47.027380+0000","last_became_peered":"2023-05-20T01:17:47.027380+0000","last_unstale":"2023-05-20T01:17:48.004157+0000","last_undegraded":"2023-05-20T01:17:48.004157+0000","last_fullsized":"2023-05-20T01:17:48.004157+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:08:09.865722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:48.004402+0000","last_change":"2023-05-20T01:17:46.585289+0000","last_active":"2023-05-20T01:17:48.004402+0000","last_peered":"2023-05-20T01:17:48.004402+0000","last_clean":"2023-05-20T01:17:48.004402+0000","last_became_active":"2023-05-20T01:17:46.584948+0000","last_became_peered":"2023-05-20T01:17:46.584948+0000","last_unstale":"2023-05-20T01:17:48.004402+0000","last_undegraded":"2023-05-20T01:17:48.004402+0000","last_fullsized":"2023-05-20T01:17:48.004402+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_clean_scrub_stamp":"2023-05-20T01:17:45.555502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:58:05.407640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.620818+0000","last_change":"2023-05-20T01:17:47.031110+0000","last_active":"2023-05-20T01:17:49.620818+0000","last_peered":"2023-05-20T01:17:49.620818+0000","last_clean":"2023-05-20T01:17:49.620818+0000","last_became_active":"2023-05-20T01:17:47.030822+0000","last_became_peered":"2023-05-20T01:17:47.030822+0000","last_unstale":"2023-05-20T01:17:49.620818+0000","last_undegraded":"2023-05-20T01:17:49.620818+0000","last_fullsized":"2023-05-20T01:17:49.620818+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:39:11.448882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"39'8","reported_seq":21,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.119771+0000","last_change":"2023-05-20T01:17:46.587662+0000","last_active":"2023-05-20T01:17:49.119771+0000","last_peered":"2023-05-20T01:17:49.119771+0000","last_clean":"2023-05-20T01:17:49.119771+0000","last_became_active":"2023-05-20T01:17:46.587204+0000","last_became_peered":"2023-05-20T01:17:46.587204+0000","last_unstale":"2023-05-20T01:17:49.119771+0000","last_undegraded":"2023-05-20T01:17:49.119771+0000","last_fullsized":"2023-05-20T01:17:49.119771+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_clean_scrub_stamp":"2023-05-20T01:17:45.555502+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:55:04.797021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.620715+0000","last_change":"2023-05-20T01:17:47.027554+0000","last_active":"2023-05-20T01:17:49.620715+0000","last_peered":"2023-05-20T01:17:49.620715+0000","last_clean":"2023-05-20T01:17:49.620715+0000","last_became_active":"2023-05-20T01:17:47.027265+0000","last_became_peered":"2023-05-20T01:17:47.027265+0000","last_unstale":"2023-05-20T01:17:49.620715+0000","last_undegraded":"2023-05-20T01:17:49.620715+0000","last_fullsized":"2023-05-20T01:17:49.620715+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:14:38.834924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"39'1","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.620657+0000","last_change":"2023-05-20T01:17:46.590828+0000","last_active":"2023-05-20T01:17:49.620657+0000","last_peered":"2023-05-20T01:17:49.620657+0000","last_clean":"2023-05-20T01:17:49.620657+0000","last_became_active":"2023-05-20T01:17:46.590574+0000","last_became_peered":"2023-05-20T01:17:46.590574+0000","last_unstale":"2023-05-20T01:17:49.620657+0000","last_undegraded":"2023-05-20T01:17:49.620657+0000","last_fullsized":"2023-05-20T01:17:49.620657+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_clean_scrub_stamp":"2023-05-20T01:17:45.555502+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:29:22.635804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.119892+0000","last_change":"2023-05-20T01:17:47.021905+0000","last_active":"2023-05-20T01:17:49.119892+0000","last_peered":"2023-05-20T01:17:49.119892+0000","last_clean":"2023-05-20T01:17:49.119892+0000","last_became_active":"2023-05-20T01:17:47.021199+0000","last_became_peered":"2023-05-20T01:17:47.021199+0000","last_unstale":"2023-05-20T01:17:49.119892+0000","last_undegraded":"2023-05-20T01:17:49.119892+0000","last_fullsized":"2023-05-20T01:17:49.119892+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:38:41.796431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"39'2","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:48.005074+0000","last_change":"2023-05-20T01:17:46.592910+0000","last_active":"2023-05-20T01:17:48.005074+0000","last_peered":"2023-05-20T01:17:48.005074+0000","last_clean":"2023-05-20T01:17:48.005074+0000","last_became_active":"2023-05-20T01:17:46.592485+0000","last_became_peered":"2023-05-20T01:17:46.592485+0000","last_unstale":"2023-05-20T01:17:48.005074+0000","last_undegraded":"2023-05-20T01:17:48.005074+0000","last_fullsized":"2023-05-20T01:17:48.005074+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_clean_scrub_stamp":"2023-05-20T01:17:45.555502+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:29:18.026861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:48.007059+0000","last_change":"2023-05-20T01:17:47.011581+0000","last_active":"2023-05-20T01:17:48.007059+0000","last_peered":"2023-05-20T01:17:48.007059+0000","last_clean":"2023-05-20T01:17:48.007059+0000","last_became_active":"2023-05-20T01:17:47.011267+0000","last_became_peered":"2023-05-20T01:17:47.011267+0000","last_unstale":"2023-05-20T01:17:48.007059+0000","last_undegraded":"2023-05-20T01:17:48.007059+0000","last_fullsized":"2023-05-20T01:17:48.007059+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:45:03.604204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"39'1","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.620818+0000","last_change":"2023-05-20T01:17:46.592754+0000","last_active":"2023-05-20T01:17:49.620818+0000","last_peered":"2023-05-20T01:17:49.620818+0000","last_clean":"2023-05-20T01:17:49.620818+0000","last_became_active":"2023-05-20T01:17:46.592455+0000","last_became_peered":"2023-05-20T01:17:46.592455+0000","last_unstale":"2023-05-20T01:17:49.620818+0000","last_undegraded":"2023-05-20T01:17:49.620818+0000","last_fullsized":"2023-05-20T01:17:49.620818+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_clean_scrub_stamp":"2023-05-20T01:17:45.555502+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:38:23.603837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:48.007751+0000","last_change":"2023-05-20T01:17:47.025197+0000","last_active":"2023-05-20T01:17:48.007751+0000","last_peered":"2023-05-20T01:17:48.007751+0000","last_clean":"2023-05-20T01:17:48.007751+0000","last_became_active":"2023-05-20T01:17:47.024863+0000","last_became_peered":"2023-05-20T01:17:47.024863+0000","last_unstale":"2023-05-20T01:17:48.007751+0000","last_undegraded":"2023-05-20T01:17:48.007751+0000","last_fullsized":"2023-05-20T01:17:48.007751+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T01:45:54.786657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:48.004790+0000","last_change":"2023-05-20T01:17:46.587017+0000","last_active":"2023-05-20T01:17:48.004790+0000","last_peered":"2023-05-20T01:17:48.004790+0000","last_clean":"2023-05-20T01:17:48.004790+0000","last_became_active":"2023-05-20T01:17:46.586745+0000","last_became_peered":"2023-05-20T01:17:46.586745+0000","last_unstale":"2023-05-20T01:17:48.004790+0000","last_undegraded":"2023-05-20T01:17:48.004790+0000","last_fullsized":"2023-05-20T01:17:48.004790+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_clean_scrub_stamp":"2023-05-20T01:17:45.555502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:09:28.675043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:48.004465+0000","last_change":"2023-05-20T01:17:47.031292+0000","last_active":"2023-05-20T01:17:48.004465+0000","last_peered":"2023-05-20T01:17:48.004465+0000","last_clean":"2023-05-20T01:17:48.004465+0000","last_became_active":"2023-05-20T01:17:47.027841+0000","last_became_peered":"2023-05-20T01:17:47.027841+0000","last_unstale":"2023-05-20T01:17:48.004465+0000","last_undegraded":"2023-05-20T01:17:48.004465+0000","last_fullsized":"2023-05-20T01:17:48.004465+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:20:22.032473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"39'2","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:48.004738+0000","last_change":"2023-05-20T01:17:46.588535+0000","last_active":"2023-05-20T01:17:48.004738+0000","last_peered":"2023-05-20T01:17:48.004738+0000","last_clean":"2023-05-20T01:17:48.004738+0000","last_became_active":"2023-05-20T01:17:46.588227+0000","last_became_peered":"2023-05-20T01:17:46.588227+0000","last_unstale":"2023-05-20T01:17:48.004738+0000","last_undegraded":"2023-05-20T01:17:48.004738+0000","last_fullsized":"2023-05-20T01:17:48.004738+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_clean_scrub_stamp":"2023-05-20T01:17:45.555502+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:09:33.571525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.620722+0000","last_change":"2023-05-20T01:17:47.016444+0000","last_active":"2023-05-20T01:17:49.620722+0000","last_peered":"2023-05-20T01:17:49.620722+0000","last_clean":"2023-05-20T01:17:49.620722+0000","last_became_active":"2023-05-20T01:17:47.016170+0000","last_became_peered":"2023-05-20T01:17:47.016170+0000","last_unstale":"2023-05-20T01:17:49.620722+0000","last_undegraded":"2023-05-20T01:17:49.620722+0000","last_fullsized":"2023-05-20T01:17:49.620722+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:52:26.013882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.620616+0000","last_change":"2023-05-20T01:17:46.587258+0000","last_active":"2023-05-20T01:17:49.620616+0000","last_peered":"2023-05-20T01:17:49.620616+0000","last_clean":"2023-05-20T01:17:49.620616+0000","last_became_active":"2023-05-20T01:17:46.586947+0000","last_became_peered":"2023-05-20T01:17:46.586947+0000","last_unstale":"2023-05-20T01:17:49.620616+0000","last_undegraded":"2023-05-20T01:17:49.620616+0000","last_fullsized":"2023-05-20T01:17:49.620616+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_clean_scrub_stamp":"2023-05-20T01:17:45.555502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:06:11.570396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.620402+0000","last_change":"2023-05-20T01:17:47.021891+0000","last_active":"2023-05-20T01:17:49.620402+0000","last_peered":"2023-05-20T01:17:49.620402+0000","last_clean":"2023-05-20T01:17:49.620402+0000","last_became_active":"2023-05-20T01:17:47.021595+0000","last_became_peered":"2023-05-20T01:17:47.021595+0000","last_unstale":"2023-05-20T01:17:49.620402+0000","last_undegraded":"2023-05-20T01:17:49.620402+0000","last_fullsized":"2023-05-20T01:17:49.620402+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:57:31.565290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.119323+0000","last_change":"2023-05-20T01:17:46.591205+0000","last_active":"2023-05-20T01:17:49.119323+0000","last_peered":"2023-05-20T01:17:49.119323+0000","last_clean":"2023-05-20T01:17:49.119323+0000","last_became_active":"2023-05-20T01:17:46.590877+0000","last_became_peered":"2023-05-20T01:17:46.590877+0000","last_unstale":"2023-05-20T01:17:49.119323+0000","last_undegraded":"2023-05-20T01:17:49.119323+0000","last_fullsized":"2023-05-20T01:17:49.119323+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_clean_scrub_stamp":"2023-05-20T01:17:45.555502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:37:47.221408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.119379+0000","last_change":"2023-05-20T01:17:47.029304+0000","last_active":"2023-05-20T01:17:49.119379+0000","last_peered":"2023-05-20T01:17:49.119379+0000","last_clean":"2023-05-20T01:17:49.119379+0000","last_became_active":"2023-05-20T01:17:47.029029+0000","last_became_peered":"2023-05-20T01:17:47.029029+0000","last_unstale":"2023-05-20T01:17:49.119379+0000","last_undegraded":"2023-05-20T01:17:49.119379+0000","last_fullsized":"2023-05-20T01:17:49.119379+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:50:39.321269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.620250+0000","last_change":"2023-05-20T01:17:46.592664+0000","last_active":"2023-05-20T01:17:49.620250+0000","last_peered":"2023-05-20T01:17:49.620250+0000","last_clean":"2023-05-20T01:17:49.620250+0000","last_became_active":"2023-05-20T01:17:46.592384+0000","last_became_peered":"2023-05-20T01:17:46.592384+0000","last_unstale":"2023-05-20T01:17:49.620250+0000","last_undegraded":"2023-05-20T01:17:49.620250+0000","last_fullsized":"2023-05-20T01:17:49.620250+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_clean_scrub_stamp":"2023-05-20T01:17:45.555502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:50:46.378269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:48.004628+0000","last_change":"2023-05-20T01:17:47.031899+0000","last_active":"2023-05-20T01:17:48.004628+0000","last_peered":"2023-05-20T01:17:48.004628+0000","last_clean":"2023-05-20T01:17:48.004628+0000","last_became_active":"2023-05-20T01:17:47.031654+0000","last_became_peered":"2023-05-20T01:17:47.031654+0000","last_unstale":"2023-05-20T01:17:48.004628+0000","last_undegraded":"2023-05-20T01:17:48.004628+0000","last_fullsized":"2023-05-20T01:17:48.004628+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:07:17.632947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:48.005158+0000","last_change":"2023-05-20T01:17:46.589190+0000","last_active":"2023-05-20T01:17:48.005158+0000","last_peered":"2023-05-20T01:17:48.005158+0000","last_clean":"2023-05-20T01:17:48.005158+0000","last_became_active":"2023-05-20T01:17:46.588893+0000","last_became_peered":"2023-05-20T01:17:46.588893+0000","last_unstale":"2023-05-20T01:17:48.005158+0000","last_undegraded":"2023-05-20T01:17:48.005158+0000","last_fullsized":"2023-05-20T01:17:48.005158+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_clean_scrub_stamp":"2023-05-20T01:17:45.555502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:53:34.150992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":53,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:48.004878+0000","last_change":"2023-05-20T01:16:23.289068+0000","last_active":"2023-05-20T01:17:48.004878+0000","last_peered":"2023-05-20T01:17:48.004878+0000","last_clean":"2023-05-20T01:17:48.004878+0000","last_became_active":"2023-05-20T01:16:23.288781+0000","last_became_peered":"2023-05-20T01:16:23.288781+0000","last_unstale":"2023-05-20T01:17:48.004878+0000","last_undegraded":"2023-05-20T01:17:48.004878+0000","last_fullsized":"2023-05-20T01:17:48.004878+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:55:51.655368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:48.004506+0000","last_change":"2023-05-20T01:17:47.015826+0000","last_active":"2023-05-20T01:17:48.004506+0000","last_peered":"2023-05-20T01:17:48.004506+0000","last_clean":"2023-05-20T01:17:48.004506+0000","last_became_active":"2023-05-20T01:17:47.015560+0000","last_became_peered":"2023-05-20T01:17:47.015560+0000","last_unstale":"2023-05-20T01:17:48.004506+0000","last_undegraded":"2023-05-20T01:17:48.004506+0000","last_fullsized":"2023-05-20T01:17:48.004506+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:01:30.844749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.621141+0000","last_change":"2023-05-20T01:17:46.595158+0000","last_active":"2023-05-20T01:17:49.621141+0000","last_peered":"2023-05-20T01:17:49.621141+0000","last_clean":"2023-05-20T01:17:49.621141+0000","last_became_active":"2023-05-20T01:17:46.594892+0000","last_became_peered":"2023-05-20T01:17:46.594892+0000","last_unstale":"2023-05-20T01:17:49.621141+0000","last_undegraded":"2023-05-20T01:17:49.621141+0000","last_fullsized":"2023-05-20T01:17:49.621141+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_clean_scrub_stamp":"2023-05-20T01:17:45.555502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:40:30.936198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":57,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:48.004803+0000","last_change":"2023-05-20T01:16:22.829324+0000","last_active":"2023-05-20T01:17:48.004803+0000","last_peered":"2023-05-20T01:17:48.004803+0000","last_clean":"2023-05-20T01:17:48.004803+0000","last_became_active":"2023-05-20T01:16:22.828964+0000","last_became_peered":"2023-05-20T01:16:22.828964+0000","last_unstale":"2023-05-20T01:17:48.004803+0000","last_undegraded":"2023-05-20T01:17:48.004803+0000","last_fullsized":"2023-05-20T01:17:48.004803+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:54:31.790865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.621246+0000","last_change":"2023-05-20T01:17:47.028251+0000","last_active":"2023-05-20T01:17:49.621246+0000","last_peered":"2023-05-20T01:17:49.621246+0000","last_clean":"2023-05-20T01:17:49.621246+0000","last_became_active":"2023-05-20T01:17:47.028039+0000","last_became_peered":"2023-05-20T01:17:47.028039+0000","last_unstale":"2023-05-20T01:17:49.621246+0000","last_undegraded":"2023-05-20T01:17:49.621246+0000","last_fullsized":"2023-05-20T01:17:49.621246+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:35:56.309239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.620553+0000","last_change":"2023-05-20T01:17:46.592565+0000","last_active":"2023-05-20T01:17:49.620553+0000","last_peered":"2023-05-20T01:17:49.620553+0000","last_clean":"2023-05-20T01:17:49.620553+0000","last_became_active":"2023-05-20T01:17:46.592182+0000","last_became_peered":"2023-05-20T01:17:46.592182+0000","last_unstale":"2023-05-20T01:17:49.620553+0000","last_undegraded":"2023-05-20T01:17:49.620553+0000","last_fullsized":"2023-05-20T01:17:49.620553+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_clean_scrub_stamp":"2023-05-20T01:17:45.555502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:24:05.756976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":53,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.620763+0000","last_change":"2023-05-20T01:16:22.834754+0000","last_active":"2023-05-20T01:17:49.620763+0000","last_peered":"2023-05-20T01:17:49.620763+0000","last_clean":"2023-05-20T01:17:49.620763+0000","last_became_active":"2023-05-20T01:16:22.834437+0000","last_became_peered":"2023-05-20T01:16:22.834437+0000","last_unstale":"2023-05-20T01:17:49.620763+0000","last_undegraded":"2023-05-20T01:17:49.620763+0000","last_fullsized":"2023-05-20T01:17:49.620763+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:16:13.901189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":201,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.620352+0000","last_change":"2023-05-20T01:16:22.834920+0000","last_active":"2023-05-20T01:17:49.620352+0000","last_peered":"2023-05-20T01:17:49.620352+0000","last_clean":"2023-05-20T01:17:49.620352+0000","last_became_active":"2023-05-20T01:16:22.833924+0000","last_became_peered":"2023-05-20T01:16:22.833924+0000","last_unstale":"2023-05-20T01:17:49.620352+0000","last_undegraded":"2023-05-20T01:17:49.620352+0000","last_fullsized":"2023-05-20T01:17:49.620352+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:09:46.900791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.120096+0000","last_change":"2023-05-20T01:17:47.014401+0000","last_active":"2023-05-20T01:17:49.120096+0000","last_peered":"2023-05-20T01:17:49.120096+0000","last_clean":"2023-05-20T01:17:49.120096+0000","last_became_active":"2023-05-20T01:17:47.014181+0000","last_became_peered":"2023-05-20T01:17:47.014181+0000","last_unstale":"2023-05-20T01:17:49.120096+0000","last_undegraded":"2023-05-20T01:17:49.120096+0000","last_fullsized":"2023-05-20T01:17:49.120096+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:20:03.514131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.621316+0000","last_change":"2023-05-20T01:17:46.596401+0000","last_active":"2023-05-20T01:17:49.621316+0000","last_peered":"2023-05-20T01:17:49.621316+0000","last_clean":"2023-05-20T01:17:49.621316+0000","last_became_active":"2023-05-20T01:17:46.596117+0000","last_became_peered":"2023-05-20T01:17:46.596117+0000","last_unstale":"2023-05-20T01:17:49.621316+0000","last_undegraded":"2023-05-20T01:17:49.621316+0000","last_fullsized":"2023-05-20T01:17:49.621316+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_clean_scrub_stamp":"2023-05-20T01:17:45.555502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:49:04.107823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":51,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.620973+0000","last_change":"2023-05-20T01:16:23.290259+0000","last_active":"2023-05-20T01:17:49.620973+0000","last_peered":"2023-05-20T01:17:49.620973+0000","last_clean":"2023-05-20T01:17:49.620973+0000","last_became_active":"2023-05-20T01:16:23.289911+0000","last_became_peered":"2023-05-20T01:16:23.289911+0000","last_unstale":"2023-05-20T01:17:49.620973+0000","last_undegraded":"2023-05-20T01:17:49.620973+0000","last_fullsized":"2023-05-20T01:17:49.620973+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:43:43.702378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.119906+0000","last_change":"2023-05-20T01:17:47.022729+0000","last_active":"2023-05-20T01:17:49.119906+0000","last_peered":"2023-05-20T01:17:49.119906+0000","last_clean":"2023-05-20T01:17:49.119906+0000","last_became_active":"2023-05-20T01:17:47.022396+0000","last_became_peered":"2023-05-20T01:17:47.022396+0000","last_unstale":"2023-05-20T01:17:49.119906+0000","last_undegraded":"2023-05-20T01:17:49.119906+0000","last_fullsized":"2023-05-20T01:17:49.119906+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:50:47.388284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"39'2","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.620726+0000","last_change":"2023-05-20T01:17:46.585226+0000","last_active":"2023-05-20T01:17:49.620726+0000","last_peered":"2023-05-20T01:17:49.620726+0000","last_clean":"2023-05-20T01:17:49.620726+0000","last_became_active":"2023-05-20T01:17:46.584925+0000","last_became_peered":"2023-05-20T01:17:46.584925+0000","last_unstale":"2023-05-20T01:17:49.620726+0000","last_undegraded":"2023-05-20T01:17:49.620726+0000","last_fullsized":"2023-05-20T01:17:49.620726+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_clean_scrub_stamp":"2023-05-20T01:17:45.555502+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:22:22.071660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":51,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.620311+0000","last_change":"2023-05-20T01:16:23.290207+0000","last_active":"2023-05-20T01:17:49.620311+0000","last_peered":"2023-05-20T01:17:49.620311+0000","last_clean":"2023-05-20T01:17:49.620311+0000","last_became_active":"2023-05-20T01:16:23.289928+0000","last_became_peered":"2023-05-20T01:16:23.289928+0000","last_unstale":"2023-05-20T01:17:49.620311+0000","last_undegraded":"2023-05-20T01:17:49.620311+0000","last_fullsized":"2023-05-20T01:17:49.620311+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:18:38.754138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:48.007379+0000","last_change":"2023-05-20T01:17:47.024690+0000","last_active":"2023-05-20T01:17:48.007379+0000","last_peered":"2023-05-20T01:17:48.007379+0000","last_clean":"2023-05-20T01:17:48.007379+0000","last_became_active":"2023-05-20T01:17:47.024420+0000","last_became_peered":"2023-05-20T01:17:47.024420+0000","last_unstale":"2023-05-20T01:17:48.007379+0000","last_undegraded":"2023-05-20T01:17:48.007379+0000","last_fullsized":"2023-05-20T01:17:48.007379+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:31:34.112838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"40'1","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:48.714390+0000","last_change":"2023-05-20T01:17:46.585759+0000","last_active":"2023-05-20T01:17:48.714390+0000","last_peered":"2023-05-20T01:17:48.714390+0000","last_clean":"2023-05-20T01:17:48.714390+0000","last_became_active":"2023-05-20T01:17:46.585463+0000","last_became_peered":"2023-05-20T01:17:46.585463+0000","last_unstale":"2023-05-20T01:17:48.714390+0000","last_undegraded":"2023-05-20T01:17:48.714390+0000","last_fullsized":"2023-05-20T01:17:48.714390+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_clean_scrub_stamp":"2023-05-20T01:17:45.555502+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:39:10.749332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":51,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.620693+0000","last_change":"2023-05-20T01:16:23.290538+0000","last_active":"2023-05-20T01:17:49.620693+0000","last_peered":"2023-05-20T01:17:49.620693+0000","last_clean":"2023-05-20T01:17:49.620693+0000","last_became_active":"2023-05-20T01:16:23.290230+0000","last_became_peered":"2023-05-20T01:16:23.290230+0000","last_unstale":"2023-05-20T01:17:49.620693+0000","last_undegraded":"2023-05-20T01:17:49.620693+0000","last_fullsized":"2023-05-20T01:17:49.620693+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:44:32.467152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.620324+0000","last_change":"2023-05-20T01:17:47.016165+0000","last_active":"2023-05-20T01:17:49.620324+0000","last_peered":"2023-05-20T01:17:49.620324+0000","last_clean":"2023-05-20T01:17:49.620324+0000","last_became_active":"2023-05-20T01:17:47.015852+0000","last_became_peered":"2023-05-20T01:17:47.015852+0000","last_unstale":"2023-05-20T01:17:49.620324+0000","last_undegraded":"2023-05-20T01:17:49.620324+0000","last_fullsized":"2023-05-20T01:17:49.620324+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:53:58.456833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.620167+0000","last_change":"2023-05-20T01:17:46.593003+0000","last_active":"2023-05-20T01:17:49.620167+0000","last_peered":"2023-05-20T01:17:49.620167+0000","last_clean":"2023-05-20T01:17:49.620167+0000","last_became_active":"2023-05-20T01:17:46.592600+0000","last_became_peered":"2023-05-20T01:17:46.592600+0000","last_unstale":"2023-05-20T01:17:49.620167+0000","last_undegraded":"2023-05-20T01:17:49.620167+0000","last_fullsized":"2023-05-20T01:17:49.620167+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_clean_scrub_stamp":"2023-05-20T01:17:45.555502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:27:00.376244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":49,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:48.007522+0000","last_change":"2023-05-20T01:16:23.290341+0000","last_active":"2023-05-20T01:17:48.007522+0000","last_peered":"2023-05-20T01:17:48.007522+0000","last_clean":"2023-05-20T01:17:48.007522+0000","last_became_active":"2023-05-20T01:16:23.290053+0000","last_became_peered":"2023-05-20T01:16:23.290053+0000","last_unstale":"2023-05-20T01:17:48.007522+0000","last_undegraded":"2023-05-20T01:17:48.007522+0000","last_fullsized":"2023-05-20T01:17:48.007522+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:20:58.978210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:48.004662+0000","last_change":"2023-05-20T01:17:47.011981+0000","last_active":"2023-05-20T01:17:48.004662+0000","last_peered":"2023-05-20T01:17:48.004662+0000","last_clean":"2023-05-20T01:17:48.004662+0000","last_became_active":"2023-05-20T01:17:47.011661+0000","last_became_peered":"2023-05-20T01:17:47.011661+0000","last_unstale":"2023-05-20T01:17:48.004662+0000","last_undegraded":"2023-05-20T01:17:48.004662+0000","last_fullsized":"2023-05-20T01:17:48.004662+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:09:29.685059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.119586+0000","last_change":"2023-05-20T01:17:46.587447+0000","last_active":"2023-05-20T01:17:49.119586+0000","last_peered":"2023-05-20T01:17:49.119586+0000","last_clean":"2023-05-20T01:17:49.119586+0000","last_became_active":"2023-05-20T01:17:46.586864+0000","last_became_peered":"2023-05-20T01:17:46.586864+0000","last_unstale":"2023-05-20T01:17:49.119586+0000","last_undegraded":"2023-05-20T01:17:49.119586+0000","last_fullsized":"2023-05-20T01:17:49.119586+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_clean_scrub_stamp":"2023-05-20T01:17:45.555502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:10:27.574182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":49,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:48.007600+0000","last_change":"2023-05-20T01:16:22.834330+0000","last_active":"2023-05-20T01:17:48.007600+0000","last_peered":"2023-05-20T01:17:48.007600+0000","last_clean":"2023-05-20T01:17:48.007600+0000","last_became_active":"2023-05-20T01:16:22.833946+0000","last_became_peered":"2023-05-20T01:16:22.833946+0000","last_unstale":"2023-05-20T01:17:48.007600+0000","last_undegraded":"2023-05-20T01:17:48.007600+0000","last_fullsized":"2023-05-20T01:17:48.007600+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:38:56.203247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574870,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9812,"kb_used_data":1036,"kb_used_omap":0,"kb_used_meta":8768,"kb_avail":94362028,"statfs":{"total":96636764160,"available":96626716672,"internally_reserved":0,"allocated":1060864,"data_stored":798734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8978432},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.53000000000000003}]},{"osd":1,"last update":"Sat May 20 01:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":8.8879999999999999,"5min":8.8879999999999999,"15min":8.8879999999999999},"last":0.45600000000000002},{"interface":"front","average":{"1min":1.01,"5min":1.01,"15min":1.01},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":8.875,"5min":8.875,"15min":8.875},"last":0.48399999999999999}]},{"osd":2,"last update":"Sat May 20 01:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":8.9120000000000008,"5min":8.9120000000000008,"15min":8.9120000000000008},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":8.8520000000000003,"5min":8.8520000000000003,"15min":8.8520000000000003},"last":0.433}]},{"osd":3,"last update":"Sat May 20 01:17:20 2023","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":1.054,"15min":1.054},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":10.884,"5min":10.884,"15min":10.884},"last":0.34899999999999998},{"interface":"front","average":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":10.952,"5min":10.952,"15min":10.952},"last":0.54100000000000004}]},{"osd":4,"last update":"Sat May 20 01:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.51200000000000001}]},{"osd":5,"last update":"Sat May 20 01:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.47199999999999998}]},{"osd":6,"last update":"Sat May 20 01:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.501},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.55600000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574869,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8596,"kb_used_data":460,"kb_used_omap":0,"kb_used_meta":8128,"kb_avail":94363244,"statfs":{"total":96636764160,"available":96627961856,"internally_reserved":0,"allocated":471040,"data_stored":208948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8323072},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.54200000000000004}]},{"osd":1,"last update":"Sat May 20 01:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.58599999999999997}]},{"osd":2,"last update":"Sat May 20 01:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.496},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.39600000000000002}]},{"osd":3,"last update":"Sat May 20 01:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.56200000000000006}]},{"osd":4,"last update":"Sat May 20 01:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.66700000000000004}]},{"osd":5,"last update":"Sat May 20 01:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.61599999999999999}]},{"osd":7,"last update":"Sat May 20 01:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.64500000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574869,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9748,"kb_used_data":1036,"kb_used_omap":0,"kb_used_meta":8704,"kb_avail":94362092,"statfs":{"total":96636764160,"available":96626782208,"internally_reserved":0,"allocated":1060864,"data_stored":798714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8912896},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.23000000000000001},{"interface":"front","average":{"1min":0.315,"5min":0.315,"15min":0.315},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.434,"5min":0.434,"15min":0.434},"last":0.27700000000000002}]},{"osd":2,"last update":"Sat May 20 01:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"last":0.24199999999999999},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.36699999999999999}]},{"osd":3,"last update":"Sat May 20 01:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.372,"15min":0.372},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.34999999999999998}]},{"osd":4,"last update":"Sat May 20 01:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.55700000000000005}]},{"osd":5,"last update":"Sat May 20 01:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.363,"5min":1.363,"15min":1.363},"last":0.59599999999999997}]},{"osd":6,"last update":"Sat May 20 01:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.58299999999999996}]},{"osd":7,"last update":"Sat May 20 01:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.319,"5min":1.319,"15min":1.319},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.68200000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574869,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8716,"kb_used_data":452,"kb_used_omap":0,"kb_used_meta":8256,"kb_avail":94363124,"statfs":{"total":96636764160,"available":96627838976,"internally_reserved":0,"allocated":462848,"data_stored":207349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8454144},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat May 20 01:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.376,"5min":0.376,"15min":0.376},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.52600000000000002}]},{"osd":2,"last update":"Sat May 20 01:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.627}]},{"osd":3,"last update":"Sat May 20 01:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.372,"5min":0.372,"15min":0.372},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"last":0.60099999999999998}]},{"osd":4,"last update":"Sat May 20 01:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.61399999999999999}]},{"osd":5,"last update":"Sat May 20 01:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.82699999999999996}]},{"osd":6,"last update":"Sat May 20 01:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.65300000000000002}]},{"osd":7,"last update":"Sat May 20 01:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.71999999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574869,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8540,"kb_used_data":468,"kb_used_omap":0,"kb_used_meta":8064,"kb_avail":94363300,"statfs":{"total":96636764160,"available":96628019200,"internally_reserved":0,"allocated":479232,"data_stored":209862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8257536},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.372,"5min":0.372,"15min":0.372},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.249}]},{"osd":1,"last update":"Sat May 20 01:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.434,"15min":0.434},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.4950000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.5189999999999999,"5min":1.5189999999999999,"15min":1.5189999999999999},"last":0.31}]},{"osd":3,"last update":"Sat May 20 01:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.317},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.5620000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.433}]},{"osd":4,"last update":"Sat May 20 01:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.319,"5min":1.319,"15min":1.319},"last":0.504},{"interface":"front","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.4319999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.42499999999999999}]},{"osd":5,"last update":"Sat May 20 01:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.758,"5min":1.758,"15min":1.758},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.7110000000000001,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.66300000000000003}]},{"osd":6,"last update":"Sat May 20 01:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.665,"5min":1.665,"15min":1.665},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.819,"5min":1.819,"15min":1.819},"last":0.57399999999999995}]},{"osd":7,"last update":"Sat May 20 01:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.7310000000000001,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.8460000000000001,"5min":1.8460000000000001,"15min":1.8460000000000001},"last":0.61199999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574869,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8728,"kb_used_data":464,"kb_used_omap":0,"kb_used_meta":8256,"kb_avail":94363112,"statfs":{"total":96636764160,"available":96627826688,"internally_reserved":0,"allocated":475136,"data_stored":208914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8454144},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.504,"5min":0.504,"15min":0.504},"last":0.221},{"interface":"front","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":3.1890000000000001,"5min":3.1890000000000001,"15min":3.1890000000000001},"last":0.28299999999999997}]},{"osd":1,"last update":"Sat May 20 01:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":3.4260000000000002,"5min":3.4260000000000002,"15min":3.4260000000000002},"last":0.26400000000000001},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.34599999999999997}]},{"osd":2,"last update":"Sat May 20 01:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.413,"5min":1.413,"15min":1.413},"last":0.40699999999999997}]},{"osd":4,"last update":"Sat May 20 01:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":3.4660000000000002,"5min":3.4660000000000002,"15min":3.4660000000000002},"last":0.37}]},{"osd":5,"last update":"Sat May 20 01:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":3.5379999999999998,"5min":3.5379999999999998,"15min":3.5379999999999998},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.385,"5min":1.385,"15min":1.385},"last":0.441}]},{"osd":6,"last update":"Sat May 20 01:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.347,"5min":1.347,"15min":1.347},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.57399999999999995}]},{"osd":7,"last update":"Sat May 20 01:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":3.5009999999999999,"5min":3.5009999999999999,"15min":3.5009999999999999},"last":0.501}]}]},{"osd":4,"up_from":13,"seq":55834574869,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8792,"kb_used_data":464,"kb_used_omap":0,"kb_used_meta":8320,"kb_avail":94363048,"statfs":{"total":96636764160,"available":96627761152,"internally_reserved":0,"allocated":475136,"data_stored":209840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8519680},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.51300000000000001}]},{"osd":1,"last update":"Sat May 20 01:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.434},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.52700000000000002}]},{"osd":2,"last update":"Sat May 20 01:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.57199999999999995}]},{"osd":3,"last update":"Sat May 20 01:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":3.1749999999999998,"5min":3.1749999999999998,"15min":3.1749999999999998},"last":0.54000000000000004}]},{"osd":5,"last update":"Sat May 20 01:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.499},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"last":0.38800000000000001}]},{"osd":6,"last update":"Sat May 20 01:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.41099999999999998}]},{"osd":7,"last update":"Sat May 20 01:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.48599999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574870,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8652,"kb_used_data":452,"kb_used_omap":0,"kb_used_meta":8192,"kb_avail":94363188,"statfs":{"total":96636764160,"available":96627904512,"internally_reserved":0,"allocated":462848,"data_stored":207385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8388608},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:17:24 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":6.3929999999999998,"5min":6.3929999999999998,"15min":6.3929999999999998},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":6.3339999999999996,"5min":6.3339999999999996,"15min":6.3339999999999996},"last":0.62}]},{"osd":1,"last update":"Sat May 20 01:17:24 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":3.2450000000000001,"5min":3.2450000000000001,"15min":3.2450000000000001},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.58499999999999996}]},{"osd":2,"last update":"Sat May 20 01:17:24 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":0.60699999999999998}]},{"osd":3,"last update":"Sat May 20 01:17:24 2023","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":6.351,"5min":6.351,"15min":6.351},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":6.3730000000000002,"5min":6.3730000000000002,"15min":6.3730000000000002},"last":0.64700000000000002}]},{"osd":4,"last update":"Sat May 20 01:17:24 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.44600000000000001}]},{"osd":6,"last update":"Sat May 20 01:17:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.54900000000000004}]},{"osd":7,"last update":"Sat May 20 01:17:24 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.52800000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-05-20T01:17:51.469 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-05-20T01:17:51.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.624+0000 7f5d8b434700 1 Processor -- start 2023-05-20T01:17:51.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.624+0000 7f5d8b434700 1 -- start start 2023-05-20T01:17:51.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.625+0000 7f5d8b434700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5d8406a510 0x7f5d8415e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:51.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.625+0000 7f5d8b434700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f5d8406a8e0 0x7f5d840cf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:51.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.625+0000 7f5d8b434700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d8415e060 con 0x7f5d840cfb30 2023-05-20T01:17:51.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.625+0000 7f5d8b434700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d8415d450 con 0x7f5d8406a510 2023-05-20T01:17:51.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.625+0000 7f5d8b434700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d8415d730 con 0x7f5d8406a8e0 2023-05-20T01:17:51.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.625+0000 7f5d891d0700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5d8406a510 0x7f5d8415e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:51.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.625+0000 7f5d899d1700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f5d840cfb30 0x7f5d84071340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:43578/0 (socket says 172.21.15.16:43578) 2023-05-20T01:17:51.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.625+0000 7f5d891d0700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5d8406a510 0x7f5d8415e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60280/0 (socket says 172.21.15.16:60280) 2023-05-20T01:17:51.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.625+0000 7f5d899d1700 1 -- 172.21.15.16:0/432011570 learned_addr learned my addr 172.21.15.16:0/432011570 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:17:51.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.625+0000 7f5d891d0700 1 -- 172.21.15.16:0/432011570 >> v2:172.21.15.132:3300/0 conn(0x7f5d8406a8e0 msgr2=0x7f5d840cf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:51.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.625+0000 7f5d889cf700 1 --2- 172.21.15.16:0/432011570 >> v2:172.21.15.132:3300/0 conn(0x7f5d8406a8e0 0x7f5d840cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:51.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.625+0000 7f5d891d0700 1 --2- 172.21.15.16:0/432011570 >> v2:172.21.15.132:3300/0 conn(0x7f5d8406a8e0 0x7f5d840cf5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:51.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.625+0000 7f5d7bfff700 1 -- 172.21.15.16:0/432011570 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1281562194 0 0) 0x7f5d8415e060 con 0x7f5d840cfb30 2023-05-20T01:17:51.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.625+0000 7f5d891d0700 1 -- 172.21.15.16:0/432011570 >> v1:172.21.15.16:6789/0 conn(0x7f5d840cfb30 legacy=0x7f5d84071340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:51.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.625+0000 7f5d891d0700 1 -- 172.21.15.16:0/432011570 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d8415da10 con 0x7f5d8406a510 2023-05-20T01:17:51.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.625+0000 7f5d891d0700 1 --2- 172.21.15.16:0/432011570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d8406a510 0x7f5d8415e2a0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f5d84070fd0 tx=0x7f5d80009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bfb03cf6984a39c server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:51.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.625+0000 7f5d7bfff700 1 -- 172.21.15.16:0/432011570 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d80011830 con 0x7f5d8406a510 2023-05-20T01:17:51.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.626+0000 7f5d7bfff700 1 -- 172.21.15.16:0/432011570 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d80011990 con 0x7f5d8406a510 2023-05-20T01:17:51.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.626+0000 7f5d7bfff700 1 -- 172.21.15.16:0/432011570 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d80011d20 con 0x7f5d8406a510 2023-05-20T01:17:51.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.626+0000 7f5d8b434700 1 -- 172.21.15.16:0/432011570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d8406a510 msgr2=0x7f5d8415e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:51.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.626+0000 7f5d8b434700 1 --2- 172.21.15.16:0/432011570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d8406a510 0x7f5d8415e2a0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f5d84070fd0 tx=0x7f5d80009b20 comp rx=0 tx=0).stop 2023-05-20T01:17:51.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.626+0000 7f5d8b434700 1 -- 172.21.15.16:0/432011570 shutdown_connections 2023-05-20T01:17:51.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.626+0000 7f5d8b434700 1 --2- 172.21.15.16:0/432011570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d8406a510 0x7f5d8415e2a0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:51.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.626+0000 7f5d8b434700 1 --2- 172.21.15.16:0/432011570 >> v2:172.21.15.132:3300/0 conn(0x7f5d8406a8e0 0x7f5d840cf5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:51.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.626+0000 7f5d8b434700 1 -- 172.21.15.16:0/432011570 >> 172.21.15.16:0/432011570 conn(0x7f5d840d0c30 msgr2=0x7f5d840b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:51.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.626+0000 7f5d8b434700 1 -- 172.21.15.16:0/432011570 shutdown_connections 2023-05-20T01:17:51.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.626+0000 7f5d8b434700 1 -- 172.21.15.16:0/432011570 wait complete. 2023-05-20T01:17:51.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.628+0000 7f5d8b434700 1 Processor -- start 2023-05-20T01:17:51.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.628+0000 7f5d8b434700 1 -- start start 2023-05-20T01:17:51.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.629+0000 7f5d8b434700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5d8406a510 0x7f5d840c7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:51.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.629+0000 7f5d8b434700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d8406a8e0 0x7f5d84146ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:51.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.629+0000 7f5d8b434700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5d840cfb30 0x7f5d840c3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:51.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.629+0000 7f5d8b434700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d84147400 con 0x7f5d8406a8e0 2023-05-20T01:17:51.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.629+0000 7f5d8b434700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d840c4530 con 0x7f5d8406a510 2023-05-20T01:17:51.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.629+0000 7f5d8b434700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d840c4810 con 0x7f5d840cfb30 2023-05-20T01:17:51.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.629+0000 7f5d889cf700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d8406a8e0 0x7f5d84146ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:51.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.629+0000 7f5d889cf700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d8406a8e0 0x7f5d84146ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60288/0 (socket says 172.21.15.16:60288) 2023-05-20T01:17:51.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.629+0000 7f5d889cf700 1 -- 172.21.15.16:0/542443409 learned_addr learned my addr 172.21.15.16:0/542443409 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:51.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.629+0000 7f5d899d1700 1 --2- 172.21.15.16:0/542443409 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5d840cfb30 0x7f5d840c3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:51.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.629+0000 7f5d891d0700 1 --2- 172.21.15.16:0/542443409 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5d8406a510 0x7f5d840c7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:51.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.629+0000 7f5d889cf700 1 -- 172.21.15.16:0/542443409 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5d840cfb30 msgr2=0x7f5d840c3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:51.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.629+0000 7f5d889cf700 1 --2- 172.21.15.16:0/542443409 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5d840cfb30 0x7f5d840c3ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:51.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.629+0000 7f5d889cf700 1 -- 172.21.15.16:0/542443409 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5d8406a510 msgr2=0x7f5d840c7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:51.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.629+0000 7f5d889cf700 1 --2- 172.21.15.16:0/542443409 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5d8406a510 0x7f5d840c7080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:51.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.629+0000 7f5d889cf700 1 -- 172.21.15.16:0/542443409 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d800094a0 con 0x7f5d8406a8e0 2023-05-20T01:17:51.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.629+0000 7f5d899d1700 1 --2- 172.21.15.16:0/542443409 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5d840cfb30 0x7f5d840c3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:17:51.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.629+0000 7f5d889cf700 1 --2- 172.21.15.16:0/542443409 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d8406a8e0 0x7f5d84146ec0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f5d7400c7f0 tx=0x7f5d7400cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:51.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.630+0000 7f5d79ffb700 1 -- 172.21.15.16:0/542443409 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d74011800 con 0x7f5d8406a8e0 2023-05-20T01:17:51.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.630+0000 7f5d79ffb700 1 -- 172.21.15.16:0/542443409 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d74011960 con 0x7f5d8406a8e0 2023-05-20T01:17:51.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.630+0000 7f5d8b434700 1 -- 172.21.15.16:0/542443409 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d8406ad50 con 0x7f5d8406a8e0 2023-05-20T01:17:51.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.630+0000 7f5d79ffb700 1 -- 172.21.15.16:0/542443409 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d74011cf0 con 0x7f5d8406a8e0 2023-05-20T01:17:51.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.630+0000 7f5d8b434700 1 -- 172.21.15.16:0/542443409 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d8415dcf0 con 0x7f5d8406a8e0 2023-05-20T01:17:51.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.631+0000 7f5d8b434700 1 -- 172.21.15.16:0/542443409 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d8415d080 con 0x7f5d8406a8e0 2023-05-20T01:17:51.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.631+0000 7f5d79ffb700 1 -- 172.21.15.16:0/542443409 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5d74018360 con 0x7f5d8406a8e0 2023-05-20T01:17:51.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.632+0000 7f5d79ffb700 1 --2- 172.21.15.16:0/542443409 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f5d6806ace0 0x7f5d6806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:51.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.632+0000 7f5d79ffb700 1 -- 172.21.15.16:0/542443409 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f5d74089680 con 0x7f5d8406a8e0 2023-05-20T01:17:51.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.632+0000 7f5d891d0700 1 --2- 172.21.15.16:0/542443409 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f5d6806ace0 0x7f5d6806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:51.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.632+0000 7f5d891d0700 1 --2- 172.21.15.16:0/542443409 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f5d6806ace0 0x7f5d6806d190 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f5d800090b0 tx=0x7f5d80008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:51.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.636+0000 7f5d79ffb700 1 -- 172.21.15.16:0/542443409 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f5d74057ff0 con 0x7f5d8406a8e0 2023-05-20T01:17:51.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.831+0000 7f5d8b434700 1 -- 172.21.15.16:0/542443409 --> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5d840c5350 con 0x7f5d6806ace0 2023-05-20T01:17:51.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.840+0000 7f5d79ffb700 1 -- 172.21.15.16:0/542443409 <== mgr.4104 v2:172.21.15.16:6832/3817237998 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311338 (secure 0 0 0) 0x7f5d840c5350 con 0x7f5d6806ace0 2023-05-20T01:17:51.847 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:17:51.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.843+0000 7f5d8b434700 1 -- 172.21.15.16:0/542443409 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f5d6806ace0 msgr2=0x7f5d6806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:51.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.843+0000 7f5d8b434700 1 --2- 172.21.15.16:0/542443409 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f5d6806ace0 0x7f5d6806d190 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f5d800090b0 tx=0x7f5d80008030 comp rx=0 tx=0).stop 2023-05-20T01:17:51.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.843+0000 7f5d8b434700 1 -- 172.21.15.16:0/542443409 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d8406a8e0 msgr2=0x7f5d84146ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:51.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.843+0000 7f5d8b434700 1 --2- 172.21.15.16:0/542443409 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d8406a8e0 0x7f5d84146ec0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f5d7400c7f0 tx=0x7f5d7400cbb0 comp rx=0 tx=0).stop 2023-05-20T01:17:51.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.843+0000 7f5d8b434700 1 -- 172.21.15.16:0/542443409 shutdown_connections 2023-05-20T01:17:51.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.843+0000 7f5d8b434700 1 --2- 172.21.15.16:0/542443409 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f5d6806ace0 0x7f5d6806d190 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:51.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.843+0000 7f5d8b434700 1 --2- 172.21.15.16:0/542443409 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5d840cfb30 0x7f5d840c3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:51.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.843+0000 7f5d8b434700 1 --2- 172.21.15.16:0/542443409 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5d8406a510 0x7f5d840c7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:51.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.843+0000 7f5d8b434700 1 --2- 172.21.15.16:0/542443409 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d8406a8e0 0x7f5d84146ec0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:51.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.843+0000 7f5d8b434700 1 -- 172.21.15.16:0/542443409 >> 172.21.15.16:0/542443409 conn(0x7f5d840d0c30 msgr2=0x7f5d840cbeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:51.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.843+0000 7f5d8b434700 1 -- 172.21.15.16:0/542443409 shutdown_connections 2023-05-20T01:17:51.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:51.843+0000 7f5d8b434700 1 -- 172.21.15.16:0/542443409 wait complete. 2023-05-20T01:17:51.851 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2023-05-20T01:17:51.859 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":79,"stamp":"2023-05-20T01:17:51.335150+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":179,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":71584,"kb_used_data":4832,"kb_used_omap":0,"kb_used_meta":66688,"kb_avail":754903136,"statfs":{"total":773094113280,"available":773020811264,"internally_reserved":0,"allocated":4947968,"data_stored":2849746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68288512},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.769476"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.621144+0000","last_change":"2023-05-20T01:17:47.027759+0000","last_active":"2023-05-20T01:17:49.621144+0000","last_peered":"2023-05-20T01:17:49.621144+0000","last_clean":"2023-05-20T01:17:49.621144+0000","last_became_active":"2023-05-20T01:17:47.027356+0000","last_became_peered":"2023-05-20T01:17:47.027356+0000","last_unstale":"2023-05-20T01:17:49.621144+0000","last_undegraded":"2023-05-20T01:17:49.621144+0000","last_fullsized":"2023-05-20T01:17:49.621144+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:16:39.017026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.620596+0000","last_change":"2023-05-20T01:17:47.025175+0000","last_active":"2023-05-20T01:17:49.620596+0000","last_peered":"2023-05-20T01:17:49.620596+0000","last_clean":"2023-05-20T01:17:49.620596+0000","last_became_active":"2023-05-20T01:17:47.024861+0000","last_became_peered":"2023-05-20T01:17:47.024861+0000","last_unstale":"2023-05-20T01:17:49.620596+0000","last_undegraded":"2023-05-20T01:17:49.620596+0000","last_fullsized":"2023-05-20T01:17:49.620596+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:06:19.322637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:48.004778+0000","last_change":"2023-05-20T01:17:47.024627+0000","last_active":"2023-05-20T01:17:48.004778+0000","last_peered":"2023-05-20T01:17:48.004778+0000","last_clean":"2023-05-20T01:17:48.004778+0000","last_became_active":"2023-05-20T01:17:47.024330+0000","last_became_peered":"2023-05-20T01:17:47.024330+0000","last_unstale":"2023-05-20T01:17:48.004778+0000","last_undegraded":"2023-05-20T01:17:48.004778+0000","last_fullsized":"2023-05-20T01:17:48.004778+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:55:56.550858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.119708+0000","last_change":"2023-05-20T01:17:47.017256+0000","last_active":"2023-05-20T01:17:49.119708+0000","last_peered":"2023-05-20T01:17:49.119708+0000","last_clean":"2023-05-20T01:17:49.119708+0000","last_became_active":"2023-05-20T01:17:47.014824+0000","last_became_peered":"2023-05-20T01:17:47.014824+0000","last_unstale":"2023-05-20T01:17:49.119708+0000","last_undegraded":"2023-05-20T01:17:49.119708+0000","last_fullsized":"2023-05-20T01:17:49.119708+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:25:43.698565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.620514+0000","last_change":"2023-05-20T01:17:47.025575+0000","last_active":"2023-05-20T01:17:49.620514+0000","last_peered":"2023-05-20T01:17:49.620514+0000","last_clean":"2023-05-20T01:17:49.620514+0000","last_became_active":"2023-05-20T01:17:47.025298+0000","last_became_peered":"2023-05-20T01:17:47.025298+0000","last_unstale":"2023-05-20T01:17:49.620514+0000","last_undegraded":"2023-05-20T01:17:49.620514+0000","last_fullsized":"2023-05-20T01:17:49.620514+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:55:14.133662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:48.004347+0000","last_change":"2023-05-20T01:17:47.019455+0000","last_active":"2023-05-20T01:17:48.004347+0000","last_peered":"2023-05-20T01:17:48.004347+0000","last_clean":"2023-05-20T01:17:48.004347+0000","last_became_active":"2023-05-20T01:17:47.019187+0000","last_became_peered":"2023-05-20T01:17:47.019187+0000","last_unstale":"2023-05-20T01:17:48.004347+0000","last_undegraded":"2023-05-20T01:17:48.004347+0000","last_fullsized":"2023-05-20T01:17:48.004347+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T13:15:38.249014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.621253+0000","last_change":"2023-05-20T01:17:47.030082+0000","last_active":"2023-05-20T01:17:49.621253+0000","last_peered":"2023-05-20T01:17:49.621253+0000","last_clean":"2023-05-20T01:17:49.621253+0000","last_became_active":"2023-05-20T01:17:47.029780+0000","last_became_peered":"2023-05-20T01:17:47.029780+0000","last_unstale":"2023-05-20T01:17:49.621253+0000","last_undegraded":"2023-05-20T01:17:49.621253+0000","last_fullsized":"2023-05-20T01:17:49.621253+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T01:25:43.080234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.119266+0000","last_change":"2023-05-20T01:17:47.023514+0000","last_active":"2023-05-20T01:17:49.119266+0000","last_peered":"2023-05-20T01:17:49.119266+0000","last_clean":"2023-05-20T01:17:49.119266+0000","last_became_active":"2023-05-20T01:17:47.023158+0000","last_became_peered":"2023-05-20T01:17:47.023158+0000","last_unstale":"2023-05-20T01:17:49.119266+0000","last_undegraded":"2023-05-20T01:17:49.119266+0000","last_fullsized":"2023-05-20T01:17:49.119266+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:41:24.500547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.119766+0000","last_change":"2023-05-20T01:17:47.025333+0000","last_active":"2023-05-20T01:17:49.119766+0000","last_peered":"2023-05-20T01:17:49.119766+0000","last_clean":"2023-05-20T01:17:49.119766+0000","last_became_active":"2023-05-20T01:17:47.025084+0000","last_became_peered":"2023-05-20T01:17:47.025084+0000","last_unstale":"2023-05-20T01:17:49.119766+0000","last_undegraded":"2023-05-20T01:17:49.119766+0000","last_fullsized":"2023-05-20T01:17:49.119766+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:45:39.228191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.119814+0000","last_change":"2023-05-20T01:17:47.022289+0000","last_active":"2023-05-20T01:17:49.119814+0000","last_peered":"2023-05-20T01:17:49.119814+0000","last_clean":"2023-05-20T01:17:49.119814+0000","last_became_active":"2023-05-20T01:17:47.021974+0000","last_became_peered":"2023-05-20T01:17:47.021974+0000","last_unstale":"2023-05-20T01:17:49.119814+0000","last_undegraded":"2023-05-20T01:17:49.119814+0000","last_fullsized":"2023-05-20T01:17:49.119814+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:21:10.992631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.119638+0000","last_change":"2023-05-20T01:17:47.026261+0000","last_active":"2023-05-20T01:17:49.119638+0000","last_peered":"2023-05-20T01:17:49.119638+0000","last_clean":"2023-05-20T01:17:49.119638+0000","last_became_active":"2023-05-20T01:17:47.026015+0000","last_became_peered":"2023-05-20T01:17:47.026015+0000","last_unstale":"2023-05-20T01:17:49.119638+0000","last_undegraded":"2023-05-20T01:17:49.119638+0000","last_fullsized":"2023-05-20T01:17:49.119638+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:28:06.439158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.621257+0000","last_change":"2023-05-20T01:17:47.027226+0000","last_active":"2023-05-20T01:17:49.621257+0000","last_peered":"2023-05-20T01:17:49.621257+0000","last_clean":"2023-05-20T01:17:49.621257+0000","last_became_active":"2023-05-20T01:17:47.026658+0000","last_became_peered":"2023-05-20T01:17:47.026658+0000","last_unstale":"2023-05-20T01:17:49.621257+0000","last_undegraded":"2023-05-20T01:17:49.621257+0000","last_fullsized":"2023-05-20T01:17:49.621257+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:37:48.231424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.620215+0000","last_change":"2023-05-20T01:17:47.033484+0000","last_active":"2023-05-20T01:17:49.620215+0000","last_peered":"2023-05-20T01:17:49.620215+0000","last_clean":"2023-05-20T01:17:49.620215+0000","last_became_active":"2023-05-20T01:17:47.033223+0000","last_became_peered":"2023-05-20T01:17:47.033223+0000","last_unstale":"2023-05-20T01:17:49.620215+0000","last_undegraded":"2023-05-20T01:17:49.620215+0000","last_fullsized":"2023-05-20T01:17:49.620215+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:23:44.562996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.119544+0000","last_change":"2023-05-20T01:17:47.025323+0000","last_active":"2023-05-20T01:17:49.119544+0000","last_peered":"2023-05-20T01:17:49.119544+0000","last_clean":"2023-05-20T01:17:49.119544+0000","last_became_active":"2023-05-20T01:17:47.025047+0000","last_became_peered":"2023-05-20T01:17:47.025047+0000","last_unstale":"2023-05-20T01:17:49.119544+0000","last_undegraded":"2023-05-20T01:17:49.119544+0000","last_fullsized":"2023-05-20T01:17:49.119544+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:25:18.769633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:48.004170+0000","last_change":"2023-05-20T01:17:47.010225+0000","last_active":"2023-05-20T01:17:48.004170+0000","last_peered":"2023-05-20T01:17:48.004170+0000","last_clean":"2023-05-20T01:17:48.004170+0000","last_became_active":"2023-05-20T01:17:47.009992+0000","last_became_peered":"2023-05-20T01:17:47.009992+0000","last_unstale":"2023-05-20T01:17:48.004170+0000","last_undegraded":"2023-05-20T01:17:48.004170+0000","last_fullsized":"2023-05-20T01:17:48.004170+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:49:25.854997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:48.007596+0000","last_change":"2023-05-20T01:17:47.024748+0000","last_active":"2023-05-20T01:17:48.007596+0000","last_peered":"2023-05-20T01:17:48.007596+0000","last_clean":"2023-05-20T01:17:48.007596+0000","last_became_active":"2023-05-20T01:17:47.024336+0000","last_became_peered":"2023-05-20T01:17:47.024336+0000","last_unstale":"2023-05-20T01:17:48.007596+0000","last_undegraded":"2023-05-20T01:17:48.007596+0000","last_fullsized":"2023-05-20T01:17:48.007596+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T01:44:58.149648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:48.005193+0000","last_change":"2023-05-20T01:17:47.016869+0000","last_active":"2023-05-20T01:17:48.005193+0000","last_peered":"2023-05-20T01:17:48.005193+0000","last_clean":"2023-05-20T01:17:48.005193+0000","last_became_active":"2023-05-20T01:17:47.016586+0000","last_became_peered":"2023-05-20T01:17:47.016586+0000","last_unstale":"2023-05-20T01:17:48.005193+0000","last_undegraded":"2023-05-20T01:17:48.005193+0000","last_fullsized":"2023-05-20T01:17:48.005193+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:54:49.002366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:48.004857+0000","last_change":"2023-05-20T01:17:47.026994+0000","last_active":"2023-05-20T01:17:48.004857+0000","last_peered":"2023-05-20T01:17:48.004857+0000","last_clean":"2023-05-20T01:17:48.004857+0000","last_became_active":"2023-05-20T01:17:47.026745+0000","last_became_peered":"2023-05-20T01:17:47.026745+0000","last_unstale":"2023-05-20T01:17:48.004857+0000","last_undegraded":"2023-05-20T01:17:48.004857+0000","last_fullsized":"2023-05-20T01:17:48.004857+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:39:06.671661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:48.007492+0000","last_change":"2023-05-20T01:17:47.023040+0000","last_active":"2023-05-20T01:17:48.007492+0000","last_peered":"2023-05-20T01:17:48.007492+0000","last_clean":"2023-05-20T01:17:48.007492+0000","last_became_active":"2023-05-20T01:17:47.022746+0000","last_became_peered":"2023-05-20T01:17:47.022746+0000","last_unstale":"2023-05-20T01:17:48.007492+0000","last_undegraded":"2023-05-20T01:17:48.007492+0000","last_fullsized":"2023-05-20T01:17:48.007492+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:43:28.693854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.620384+0000","last_change":"2023-05-20T01:17:47.023278+0000","last_active":"2023-05-20T01:17:49.620384+0000","last_peered":"2023-05-20T01:17:49.620384+0000","last_clean":"2023-05-20T01:17:49.620384+0000","last_became_active":"2023-05-20T01:17:47.023004+0000","last_became_peered":"2023-05-20T01:17:47.023004+0000","last_unstale":"2023-05-20T01:17:49.620384+0000","last_undegraded":"2023-05-20T01:17:49.620384+0000","last_fullsized":"2023-05-20T01:17:49.620384+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:42:10.169774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:48.004306+0000","last_change":"2023-05-20T01:17:47.011827+0000","last_active":"2023-05-20T01:17:48.004306+0000","last_peered":"2023-05-20T01:17:48.004306+0000","last_clean":"2023-05-20T01:17:48.004306+0000","last_became_active":"2023-05-20T01:17:47.011604+0000","last_became_peered":"2023-05-20T01:17:47.011604+0000","last_unstale":"2023-05-20T01:17:48.004306+0000","last_undegraded":"2023-05-20T01:17:48.004306+0000","last_fullsized":"2023-05-20T01:17:48.004306+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:18:29.739342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.119515+0000","last_change":"2023-05-20T01:17:47.021320+0000","last_active":"2023-05-20T01:17:49.119515+0000","last_peered":"2023-05-20T01:17:49.119515+0000","last_clean":"2023-05-20T01:17:49.119515+0000","last_became_active":"2023-05-20T01:17:47.020983+0000","last_became_peered":"2023-05-20T01:17:47.020983+0000","last_unstale":"2023-05-20T01:17:49.119515+0000","last_undegraded":"2023-05-20T01:17:49.119515+0000","last_fullsized":"2023-05-20T01:17:49.119515+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:10:36.488554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:48.007123+0000","last_change":"2023-05-20T01:17:47.021513+0000","last_active":"2023-05-20T01:17:48.007123+0000","last_peered":"2023-05-20T01:17:48.007123+0000","last_clean":"2023-05-20T01:17:48.007123+0000","last_became_active":"2023-05-20T01:17:47.021002+0000","last_became_peered":"2023-05-20T01:17:47.021002+0000","last_unstale":"2023-05-20T01:17:48.007123+0000","last_undegraded":"2023-05-20T01:17:48.007123+0000","last_fullsized":"2023-05-20T01:17:48.007123+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:39:00.152190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:48.006934+0000","last_change":"2023-05-20T01:17:47.018274+0000","last_active":"2023-05-20T01:17:48.006934+0000","last_peered":"2023-05-20T01:17:48.006934+0000","last_clean":"2023-05-20T01:17:48.006934+0000","last_became_active":"2023-05-20T01:17:47.017902+0000","last_became_peered":"2023-05-20T01:17:47.017902+0000","last_unstale":"2023-05-20T01:17:48.006934+0000","last_undegraded":"2023-05-20T01:17:48.006934+0000","last_fullsized":"2023-05-20T01:17:48.006934+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:03:26.905011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:48.007534+0000","last_change":"2023-05-20T01:17:47.025464+0000","last_active":"2023-05-20T01:17:48.007534+0000","last_peered":"2023-05-20T01:17:48.007534+0000","last_clean":"2023-05-20T01:17:48.007534+0000","last_became_active":"2023-05-20T01:17:47.025153+0000","last_became_peered":"2023-05-20T01:17:47.025153+0000","last_unstale":"2023-05-20T01:17:48.007534+0000","last_undegraded":"2023-05-20T01:17:48.007534+0000","last_fullsized":"2023-05-20T01:17:48.007534+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:43:11.193161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.620691+0000","last_change":"2023-05-20T01:17:47.028617+0000","last_active":"2023-05-20T01:17:49.620691+0000","last_peered":"2023-05-20T01:17:49.620691+0000","last_clean":"2023-05-20T01:17:49.620691+0000","last_became_active":"2023-05-20T01:17:47.028229+0000","last_became_peered":"2023-05-20T01:17:47.028229+0000","last_unstale":"2023-05-20T01:17:49.620691+0000","last_undegraded":"2023-05-20T01:17:49.620691+0000","last_fullsized":"2023-05-20T01:17:49.620691+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:15:27.154301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:48.004877+0000","last_change":"2023-05-20T01:17:47.017010+0000","last_active":"2023-05-20T01:17:48.004877+0000","last_peered":"2023-05-20T01:17:48.004877+0000","last_clean":"2023-05-20T01:17:48.004877+0000","last_became_active":"2023-05-20T01:17:47.016051+0000","last_became_peered":"2023-05-20T01:17:47.016051+0000","last_unstale":"2023-05-20T01:17:48.004877+0000","last_undegraded":"2023-05-20T01:17:48.004877+0000","last_fullsized":"2023-05-20T01:17:48.004877+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:39:11.759347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.620546+0000","last_change":"2023-05-20T01:17:47.027541+0000","last_active":"2023-05-20T01:17:49.620546+0000","last_peered":"2023-05-20T01:17:49.620546+0000","last_clean":"2023-05-20T01:17:49.620546+0000","last_became_active":"2023-05-20T01:17:47.027049+0000","last_became_peered":"2023-05-20T01:17:47.027049+0000","last_unstale":"2023-05-20T01:17:49.620546+0000","last_undegraded":"2023-05-20T01:17:49.620546+0000","last_fullsized":"2023-05-20T01:17:49.620546+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:10:54.768447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:48.007143+0000","last_change":"2023-05-20T01:17:47.019667+0000","last_active":"2023-05-20T01:17:48.007143+0000","last_peered":"2023-05-20T01:17:48.007143+0000","last_clean":"2023-05-20T01:17:48.007143+0000","last_became_active":"2023-05-20T01:17:47.019309+0000","last_became_peered":"2023-05-20T01:17:47.019309+0000","last_unstale":"2023-05-20T01:17:48.007143+0000","last_undegraded":"2023-05-20T01:17:48.007143+0000","last_fullsized":"2023-05-20T01:17:48.007143+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:05:38.391599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:48.004969+0000","last_change":"2023-05-20T01:17:47.019041+0000","last_active":"2023-05-20T01:17:48.004969+0000","last_peered":"2023-05-20T01:17:48.004969+0000","last_clean":"2023-05-20T01:17:48.004969+0000","last_became_active":"2023-05-20T01:17:47.018747+0000","last_became_peered":"2023-05-20T01:17:47.018747+0000","last_unstale":"2023-05-20T01:17:48.004969+0000","last_undegraded":"2023-05-20T01:17:48.004969+0000","last_fullsized":"2023-05-20T01:17:48.004969+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:42:07.468768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:48.004300+0000","last_change":"2023-05-20T01:17:47.016476+0000","last_active":"2023-05-20T01:17:48.004300+0000","last_peered":"2023-05-20T01:17:48.004300+0000","last_clean":"2023-05-20T01:17:48.004300+0000","last_became_active":"2023-05-20T01:17:47.016213+0000","last_became_peered":"2023-05-20T01:17:47.016213+0000","last_unstale":"2023-05-20T01:17:48.004300+0000","last_undegraded":"2023-05-20T01:17:48.004300+0000","last_fullsized":"2023-05-20T01:17:48.004300+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:50:09.284608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.620403+0000","last_change":"2023-05-20T01:17:47.025453+0000","last_active":"2023-05-20T01:17:49.620403+0000","last_peered":"2023-05-20T01:17:49.620403+0000","last_clean":"2023-05-20T01:17:49.620403+0000","last_became_active":"2023-05-20T01:17:47.025175+0000","last_became_peered":"2023-05-20T01:17:47.025175+0000","last_unstale":"2023-05-20T01:17:49.620403+0000","last_undegraded":"2023-05-20T01:17:49.620403+0000","last_fullsized":"2023-05-20T01:17:49.620403+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:15:44.743926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.620613+0000","last_change":"2023-05-20T01:17:47.015774+0000","last_active":"2023-05-20T01:17:49.620613+0000","last_peered":"2023-05-20T01:17:49.620613+0000","last_clean":"2023-05-20T01:17:49.620613+0000","last_became_active":"2023-05-20T01:17:47.015521+0000","last_became_peered":"2023-05-20T01:17:47.015521+0000","last_unstale":"2023-05-20T01:17:49.620613+0000","last_undegraded":"2023-05-20T01:17:49.620613+0000","last_fullsized":"2023-05-20T01:17:49.620613+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T01:55:04.263460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"39'4","reported_seq":17,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:48.005118+0000","last_change":"2023-05-20T01:17:46.588592+0000","last_active":"2023-05-20T01:17:48.005118+0000","last_peered":"2023-05-20T01:17:48.005118+0000","last_clean":"2023-05-20T01:17:48.005118+0000","last_became_active":"2023-05-20T01:17:46.588300+0000","last_became_peered":"2023-05-20T01:17:46.588300+0000","last_unstale":"2023-05-20T01:17:48.005118+0000","last_undegraded":"2023-05-20T01:17:48.005118+0000","last_fullsized":"2023-05-20T01:17:48.005118+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_clean_scrub_stamp":"2023-05-20T01:17:45.555502+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:39:05.661645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:48.007422+0000","last_change":"2023-05-20T01:17:47.014452+0000","last_active":"2023-05-20T01:17:48.007422+0000","last_peered":"2023-05-20T01:17:48.007422+0000","last_clean":"2023-05-20T01:17:48.007422+0000","last_became_active":"2023-05-20T01:17:47.014170+0000","last_became_peered":"2023-05-20T01:17:47.014170+0000","last_unstale":"2023-05-20T01:17:48.007422+0000","last_undegraded":"2023-05-20T01:17:48.007422+0000","last_fullsized":"2023-05-20T01:17:48.007422+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:24:43.149420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"39'1","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:48.005214+0000","last_change":"2023-05-20T01:17:46.585652+0000","last_active":"2023-05-20T01:17:48.005214+0000","last_peered":"2023-05-20T01:17:48.005214+0000","last_clean":"2023-05-20T01:17:48.005214+0000","last_became_active":"2023-05-20T01:17:46.585287+0000","last_became_peered":"2023-05-20T01:17:46.585287+0000","last_unstale":"2023-05-20T01:17:48.005214+0000","last_undegraded":"2023-05-20T01:17:48.005214+0000","last_fullsized":"2023-05-20T01:17:48.005214+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_clean_scrub_stamp":"2023-05-20T01:17:45.555502+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T01:18:01.154090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.620942+0000","last_change":"2023-05-20T01:17:47.016222+0000","last_active":"2023-05-20T01:17:49.620942+0000","last_peered":"2023-05-20T01:17:49.620942+0000","last_clean":"2023-05-20T01:17:49.620942+0000","last_became_active":"2023-05-20T01:17:47.015565+0000","last_became_peered":"2023-05-20T01:17:47.015565+0000","last_unstale":"2023-05-20T01:17:49.620942+0000","last_undegraded":"2023-05-20T01:17:49.620942+0000","last_fullsized":"2023-05-20T01:17:49.620942+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:41:24.500547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"39'2","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.620938+0000","last_change":"2023-05-20T01:17:46.587660+0000","last_active":"2023-05-20T01:17:49.620938+0000","last_peered":"2023-05-20T01:17:49.620938+0000","last_clean":"2023-05-20T01:17:49.620938+0000","last_became_active":"2023-05-20T01:17:46.587382+0000","last_became_peered":"2023-05-20T01:17:46.587382+0000","last_unstale":"2023-05-20T01:17:49.620938+0000","last_undegraded":"2023-05-20T01:17:49.620938+0000","last_fullsized":"2023-05-20T01:17:49.620938+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_clean_scrub_stamp":"2023-05-20T01:17:45.555502+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:20:33.225006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.620726+0000","last_change":"2023-05-20T01:17:47.031074+0000","last_active":"2023-05-20T01:17:49.620726+0000","last_peered":"2023-05-20T01:17:49.620726+0000","last_clean":"2023-05-20T01:17:49.620726+0000","last_became_active":"2023-05-20T01:17:47.030758+0000","last_became_peered":"2023-05-20T01:17:47.030758+0000","last_unstale":"2023-05-20T01:17:49.620726+0000","last_undegraded":"2023-05-20T01:17:49.620726+0000","last_fullsized":"2023-05-20T01:17:49.620726+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:28:43.939677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"39'2","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.620967+0000","last_change":"2023-05-20T01:17:46.593757+0000","last_active":"2023-05-20T01:17:49.620967+0000","last_peered":"2023-05-20T01:17:49.620967+0000","last_clean":"2023-05-20T01:17:49.620967+0000","last_became_active":"2023-05-20T01:17:46.593530+0000","last_became_peered":"2023-05-20T01:17:46.593530+0000","last_unstale":"2023-05-20T01:17:49.620967+0000","last_undegraded":"2023-05-20T01:17:49.620967+0000","last_fullsized":"2023-05-20T01:17:49.620967+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_clean_scrub_stamp":"2023-05-20T01:17:45.555502+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:47:22.733289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:48.004895+0000","last_change":"2023-05-20T01:17:47.012723+0000","last_active":"2023-05-20T01:17:48.004895+0000","last_peered":"2023-05-20T01:17:48.004895+0000","last_clean":"2023-05-20T01:17:48.004895+0000","last_became_active":"2023-05-20T01:17:47.012398+0000","last_became_peered":"2023-05-20T01:17:47.012398+0000","last_unstale":"2023-05-20T01:17:48.004895+0000","last_undegraded":"2023-05-20T01:17:48.004895+0000","last_fullsized":"2023-05-20T01:17:48.004895+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:01:57.140023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"39'1","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:48.004407+0000","last_change":"2023-05-20T01:17:46.589991+0000","last_active":"2023-05-20T01:17:48.004407+0000","last_peered":"2023-05-20T01:17:48.004407+0000","last_clean":"2023-05-20T01:17:48.004407+0000","last_became_active":"2023-05-20T01:17:46.589701+0000","last_became_peered":"2023-05-20T01:17:46.589701+0000","last_unstale":"2023-05-20T01:17:48.004407+0000","last_undegraded":"2023-05-20T01:17:48.004407+0000","last_fullsized":"2023-05-20T01:17:48.004407+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_clean_scrub_stamp":"2023-05-20T01:17:45.555502+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:46:21.391999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.119647+0000","last_change":"2023-05-20T01:17:47.025927+0000","last_active":"2023-05-20T01:17:49.119647+0000","last_peered":"2023-05-20T01:17:49.119647+0000","last_clean":"2023-05-20T01:17:49.119647+0000","last_became_active":"2023-05-20T01:17:47.025542+0000","last_became_peered":"2023-05-20T01:17:47.025542+0000","last_unstale":"2023-05-20T01:17:49.119647+0000","last_undegraded":"2023-05-20T01:17:49.119647+0000","last_fullsized":"2023-05-20T01:17:49.119647+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:43:58.959515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"39'1","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:48.005083+0000","last_change":"2023-05-20T01:17:46.586996+0000","last_active":"2023-05-20T01:17:48.005083+0000","last_peered":"2023-05-20T01:17:48.005083+0000","last_clean":"2023-05-20T01:17:48.005083+0000","last_became_active":"2023-05-20T01:17:46.586405+0000","last_became_peered":"2023-05-20T01:17:46.586405+0000","last_unstale":"2023-05-20T01:17:48.005083+0000","last_undegraded":"2023-05-20T01:17:48.005083+0000","last_fullsized":"2023-05-20T01:17:48.005083+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_clean_scrub_stamp":"2023-05-20T01:17:45.555502+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:43:35.810675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.119326+0000","last_change":"2023-05-20T01:17:47.020361+0000","last_active":"2023-05-20T01:17:49.119326+0000","last_peered":"2023-05-20T01:17:49.119326+0000","last_clean":"2023-05-20T01:17:49.119326+0000","last_became_active":"2023-05-20T01:17:47.020127+0000","last_became_peered":"2023-05-20T01:17:47.020127+0000","last_unstale":"2023-05-20T01:17:49.119326+0000","last_undegraded":"2023-05-20T01:17:49.119326+0000","last_fullsized":"2023-05-20T01:17:49.119326+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:33:18.790612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.620822+0000","last_change":"2023-05-20T01:17:46.594101+0000","last_active":"2023-05-20T01:17:49.620822+0000","last_peered":"2023-05-20T01:17:49.620822+0000","last_clean":"2023-05-20T01:17:49.620822+0000","last_became_active":"2023-05-20T01:17:46.593849+0000","last_became_peered":"2023-05-20T01:17:46.593849+0000","last_unstale":"2023-05-20T01:17:49.620822+0000","last_undegraded":"2023-05-20T01:17:49.620822+0000","last_fullsized":"2023-05-20T01:17:49.620822+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_clean_scrub_stamp":"2023-05-20T01:17:45.555502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:21:42.210970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:48.007206+0000","last_change":"2023-05-20T01:17:47.021295+0000","last_active":"2023-05-20T01:17:48.007206+0000","last_peered":"2023-05-20T01:17:48.007206+0000","last_clean":"2023-05-20T01:17:48.007206+0000","last_became_active":"2023-05-20T01:17:47.020577+0000","last_became_peered":"2023-05-20T01:17:47.020577+0000","last_unstale":"2023-05-20T01:17:48.007206+0000","last_undegraded":"2023-05-20T01:17:48.007206+0000","last_fullsized":"2023-05-20T01:17:48.007206+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:33:35.934450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.119417+0000","last_change":"2023-05-20T01:17:46.592808+0000","last_active":"2023-05-20T01:17:49.119417+0000","last_peered":"2023-05-20T01:17:49.119417+0000","last_clean":"2023-05-20T01:17:49.119417+0000","last_became_active":"2023-05-20T01:17:46.592637+0000","last_became_peered":"2023-05-20T01:17:46.592637+0000","last_unstale":"2023-05-20T01:17:49.119417+0000","last_undegraded":"2023-05-20T01:17:49.119417+0000","last_fullsized":"2023-05-20T01:17:49.119417+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_clean_scrub_stamp":"2023-05-20T01:17:45.555502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:44:07.422862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:48.005170+0000","last_change":"2023-05-20T01:17:47.026921+0000","last_active":"2023-05-20T01:17:48.005170+0000","last_peered":"2023-05-20T01:17:48.005170+0000","last_clean":"2023-05-20T01:17:48.005170+0000","last_became_active":"2023-05-20T01:17:47.026679+0000","last_became_peered":"2023-05-20T01:17:47.026679+0000","last_unstale":"2023-05-20T01:17:48.005170+0000","last_undegraded":"2023-05-20T01:17:48.005170+0000","last_fullsized":"2023-05-20T01:17:48.005170+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:50:03.983484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:48.005149+0000","last_change":"2023-05-20T01:17:46.588313+0000","last_active":"2023-05-20T01:17:48.005149+0000","last_peered":"2023-05-20T01:17:48.005149+0000","last_clean":"2023-05-20T01:17:48.005149+0000","last_became_active":"2023-05-20T01:17:46.588041+0000","last_became_peered":"2023-05-20T01:17:46.588041+0000","last_unstale":"2023-05-20T01:17:48.005149+0000","last_undegraded":"2023-05-20T01:17:48.005149+0000","last_fullsized":"2023-05-20T01:17:48.005149+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_clean_scrub_stamp":"2023-05-20T01:17:45.555502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:00:36.579161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.621217+0000","last_change":"2023-05-20T01:17:47.023661+0000","last_active":"2023-05-20T01:17:49.621217+0000","last_peered":"2023-05-20T01:17:49.621217+0000","last_clean":"2023-05-20T01:17:49.621217+0000","last_became_active":"2023-05-20T01:17:47.023405+0000","last_became_peered":"2023-05-20T01:17:47.023405+0000","last_unstale":"2023-05-20T01:17:49.621217+0000","last_undegraded":"2023-05-20T01:17:49.621217+0000","last_fullsized":"2023-05-20T01:17:49.621217+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:04:06.970847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.119904+0000","last_change":"2023-05-20T01:17:46.591266+0000","last_active":"2023-05-20T01:17:49.119904+0000","last_peered":"2023-05-20T01:17:49.119904+0000","last_clean":"2023-05-20T01:17:49.119904+0000","last_became_active":"2023-05-20T01:17:46.590943+0000","last_became_peered":"2023-05-20T01:17:46.590943+0000","last_unstale":"2023-05-20T01:17:49.119904+0000","last_undegraded":"2023-05-20T01:17:49.119904+0000","last_fullsized":"2023-05-20T01:17:49.119904+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_clean_scrub_stamp":"2023-05-20T01:17:45.555502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:13:28.112841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:48.004722+0000","last_change":"2023-05-20T01:17:47.027760+0000","last_active":"2023-05-20T01:17:48.004722+0000","last_peered":"2023-05-20T01:17:48.004722+0000","last_clean":"2023-05-20T01:17:48.004722+0000","last_became_active":"2023-05-20T01:17:47.027495+0000","last_became_peered":"2023-05-20T01:17:47.027495+0000","last_unstale":"2023-05-20T01:17:48.004722+0000","last_undegraded":"2023-05-20T01:17:48.004722+0000","last_fullsized":"2023-05-20T01:17:48.004722+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:34:31.667495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"39'3","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.621192+0000","last_change":"2023-05-20T01:17:46.596339+0000","last_active":"2023-05-20T01:17:49.621192+0000","last_peered":"2023-05-20T01:17:49.621192+0000","last_clean":"2023-05-20T01:17:49.621192+0000","last_became_active":"2023-05-20T01:17:46.596039+0000","last_became_peered":"2023-05-20T01:17:46.596039+0000","last_unstale":"2023-05-20T01:17:49.621192+0000","last_undegraded":"2023-05-20T01:17:49.621192+0000","last_fullsized":"2023-05-20T01:17:49.621192+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_clean_scrub_stamp":"2023-05-20T01:17:45.555502+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:05:04.021904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.621042+0000","last_change":"2023-05-20T01:17:47.028275+0000","last_active":"2023-05-20T01:17:49.621042+0000","last_peered":"2023-05-20T01:17:49.621042+0000","last_clean":"2023-05-20T01:17:49.621042+0000","last_became_active":"2023-05-20T01:17:47.028076+0000","last_became_peered":"2023-05-20T01:17:47.028076+0000","last_unstale":"2023-05-20T01:17:49.621042+0000","last_undegraded":"2023-05-20T01:17:49.621042+0000","last_fullsized":"2023-05-20T01:17:49.621042+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:47:23.552100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"39'6","reported_seq":19,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.620534+0000","last_change":"2023-05-20T01:17:46.588586+0000","last_active":"2023-05-20T01:17:49.620534+0000","last_peered":"2023-05-20T01:17:49.620534+0000","last_clean":"2023-05-20T01:17:49.620534+0000","last_became_active":"2023-05-20T01:17:46.588295+0000","last_became_peered":"2023-05-20T01:17:46.588295+0000","last_unstale":"2023-05-20T01:17:49.620534+0000","last_undegraded":"2023-05-20T01:17:49.620534+0000","last_fullsized":"2023-05-20T01:17:49.620534+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_clean_scrub_stamp":"2023-05-20T01:17:45.555502+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T01:31:12.253501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:48.007250+0000","last_change":"2023-05-20T01:17:47.025107+0000","last_active":"2023-05-20T01:17:48.007250+0000","last_peered":"2023-05-20T01:17:48.007250+0000","last_clean":"2023-05-20T01:17:48.007250+0000","last_became_active":"2023-05-20T01:17:47.024305+0000","last_became_peered":"2023-05-20T01:17:47.024305+0000","last_unstale":"2023-05-20T01:17:48.007250+0000","last_undegraded":"2023-05-20T01:17:48.007250+0000","last_fullsized":"2023-05-20T01:17:48.007250+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:30:01.995835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"39'2","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:48.004661+0000","last_change":"2023-05-20T01:17:46.585732+0000","last_active":"2023-05-20T01:17:48.004661+0000","last_peered":"2023-05-20T01:17:48.004661+0000","last_clean":"2023-05-20T01:17:48.004661+0000","last_became_active":"2023-05-20T01:17:46.585290+0000","last_became_peered":"2023-05-20T01:17:46.585290+0000","last_unstale":"2023-05-20T01:17:48.004661+0000","last_undegraded":"2023-05-20T01:17:48.004661+0000","last_fullsized":"2023-05-20T01:17:48.004661+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_clean_scrub_stamp":"2023-05-20T01:17:45.555502+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:16:03.579796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.119809+0000","last_change":"2023-05-20T01:17:47.018074+0000","last_active":"2023-05-20T01:17:49.119809+0000","last_peered":"2023-05-20T01:17:49.119809+0000","last_clean":"2023-05-20T01:17:49.119809+0000","last_became_active":"2023-05-20T01:17:47.017809+0000","last_became_peered":"2023-05-20T01:17:47.017809+0000","last_unstale":"2023-05-20T01:17:49.119809+0000","last_undegraded":"2023-05-20T01:17:49.119809+0000","last_fullsized":"2023-05-20T01:17:49.119809+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:22:30.348997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.621379+0000","last_change":"2023-05-20T01:17:46.592389+0000","last_active":"2023-05-20T01:17:49.621379+0000","last_peered":"2023-05-20T01:17:49.621379+0000","last_clean":"2023-05-20T01:17:49.621379+0000","last_became_active":"2023-05-20T01:17:46.592100+0000","last_became_peered":"2023-05-20T01:17:46.592100+0000","last_unstale":"2023-05-20T01:17:49.621379+0000","last_undegraded":"2023-05-20T01:17:49.621379+0000","last_fullsized":"2023-05-20T01:17:49.621379+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_clean_scrub_stamp":"2023-05-20T01:17:45.555502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:27:43.895840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:48.004157+0000","last_change":"2023-05-20T01:17:47.027652+0000","last_active":"2023-05-20T01:17:48.004157+0000","last_peered":"2023-05-20T01:17:48.004157+0000","last_clean":"2023-05-20T01:17:48.004157+0000","last_became_active":"2023-05-20T01:17:47.027380+0000","last_became_peered":"2023-05-20T01:17:47.027380+0000","last_unstale":"2023-05-20T01:17:48.004157+0000","last_undegraded":"2023-05-20T01:17:48.004157+0000","last_fullsized":"2023-05-20T01:17:48.004157+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:08:09.865722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:48.004402+0000","last_change":"2023-05-20T01:17:46.585289+0000","last_active":"2023-05-20T01:17:48.004402+0000","last_peered":"2023-05-20T01:17:48.004402+0000","last_clean":"2023-05-20T01:17:48.004402+0000","last_became_active":"2023-05-20T01:17:46.584948+0000","last_became_peered":"2023-05-20T01:17:46.584948+0000","last_unstale":"2023-05-20T01:17:48.004402+0000","last_undegraded":"2023-05-20T01:17:48.004402+0000","last_fullsized":"2023-05-20T01:17:48.004402+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_clean_scrub_stamp":"2023-05-20T01:17:45.555502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:58:05.407640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.620818+0000","last_change":"2023-05-20T01:17:47.031110+0000","last_active":"2023-05-20T01:17:49.620818+0000","last_peered":"2023-05-20T01:17:49.620818+0000","last_clean":"2023-05-20T01:17:49.620818+0000","last_became_active":"2023-05-20T01:17:47.030822+0000","last_became_peered":"2023-05-20T01:17:47.030822+0000","last_unstale":"2023-05-20T01:17:49.620818+0000","last_undegraded":"2023-05-20T01:17:49.620818+0000","last_fullsized":"2023-05-20T01:17:49.620818+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:39:11.448882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"39'8","reported_seq":21,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.119771+0000","last_change":"2023-05-20T01:17:46.587662+0000","last_active":"2023-05-20T01:17:49.119771+0000","last_peered":"2023-05-20T01:17:49.119771+0000","last_clean":"2023-05-20T01:17:49.119771+0000","last_became_active":"2023-05-20T01:17:46.587204+0000","last_became_peered":"2023-05-20T01:17:46.587204+0000","last_unstale":"2023-05-20T01:17:49.119771+0000","last_undegraded":"2023-05-20T01:17:49.119771+0000","last_fullsized":"2023-05-20T01:17:49.119771+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_clean_scrub_stamp":"2023-05-20T01:17:45.555502+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:55:04.797021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.620715+0000","last_change":"2023-05-20T01:17:47.027554+0000","last_active":"2023-05-20T01:17:49.620715+0000","last_peered":"2023-05-20T01:17:49.620715+0000","last_clean":"2023-05-20T01:17:49.620715+0000","last_became_active":"2023-05-20T01:17:47.027265+0000","last_became_peered":"2023-05-20T01:17:47.027265+0000","last_unstale":"2023-05-20T01:17:49.620715+0000","last_undegraded":"2023-05-20T01:17:49.620715+0000","last_fullsized":"2023-05-20T01:17:49.620715+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:14:38.834924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"39'1","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.620657+0000","last_change":"2023-05-20T01:17:46.590828+0000","last_active":"2023-05-20T01:17:49.620657+0000","last_peered":"2023-05-20T01:17:49.620657+0000","last_clean":"2023-05-20T01:17:49.620657+0000","last_became_active":"2023-05-20T01:17:46.590574+0000","last_became_peered":"2023-05-20T01:17:46.590574+0000","last_unstale":"2023-05-20T01:17:49.620657+0000","last_undegraded":"2023-05-20T01:17:49.620657+0000","last_fullsized":"2023-05-20T01:17:49.620657+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_clean_scrub_stamp":"2023-05-20T01:17:45.555502+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:29:22.635804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.119892+0000","last_change":"2023-05-20T01:17:47.021905+0000","last_active":"2023-05-20T01:17:49.119892+0000","last_peered":"2023-05-20T01:17:49.119892+0000","last_clean":"2023-05-20T01:17:49.119892+0000","last_became_active":"2023-05-20T01:17:47.021199+0000","last_became_peered":"2023-05-20T01:17:47.021199+0000","last_unstale":"2023-05-20T01:17:49.119892+0000","last_undegraded":"2023-05-20T01:17:49.119892+0000","last_fullsized":"2023-05-20T01:17:49.119892+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:38:41.796431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"39'2","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:48.005074+0000","last_change":"2023-05-20T01:17:46.592910+0000","last_active":"2023-05-20T01:17:48.005074+0000","last_peered":"2023-05-20T01:17:48.005074+0000","last_clean":"2023-05-20T01:17:48.005074+0000","last_became_active":"2023-05-20T01:17:46.592485+0000","last_became_peered":"2023-05-20T01:17:46.592485+0000","last_unstale":"2023-05-20T01:17:48.005074+0000","last_undegraded":"2023-05-20T01:17:48.005074+0000","last_fullsized":"2023-05-20T01:17:48.005074+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_clean_scrub_stamp":"2023-05-20T01:17:45.555502+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:29:18.026861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:48.007059+0000","last_change":"2023-05-20T01:17:47.011581+0000","last_active":"2023-05-20T01:17:48.007059+0000","last_peered":"2023-05-20T01:17:48.007059+0000","last_clean":"2023-05-20T01:17:48.007059+0000","last_became_active":"2023-05-20T01:17:47.011267+0000","last_became_peered":"2023-05-20T01:17:47.011267+0000","last_unstale":"2023-05-20T01:17:48.007059+0000","last_undegraded":"2023-05-20T01:17:48.007059+0000","last_fullsized":"2023-05-20T01:17:48.007059+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:45:03.604204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"39'1","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.620818+0000","last_change":"2023-05-20T01:17:46.592754+0000","last_active":"2023-05-20T01:17:49.620818+0000","last_peered":"2023-05-20T01:17:49.620818+0000","last_clean":"2023-05-20T01:17:49.620818+0000","last_became_active":"2023-05-20T01:17:46.592455+0000","last_became_peered":"2023-05-20T01:17:46.592455+0000","last_unstale":"2023-05-20T01:17:49.620818+0000","last_undegraded":"2023-05-20T01:17:49.620818+0000","last_fullsized":"2023-05-20T01:17:49.620818+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_clean_scrub_stamp":"2023-05-20T01:17:45.555502+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:38:23.603837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:48.007751+0000","last_change":"2023-05-20T01:17:47.025197+0000","last_active":"2023-05-20T01:17:48.007751+0000","last_peered":"2023-05-20T01:17:48.007751+0000","last_clean":"2023-05-20T01:17:48.007751+0000","last_became_active":"2023-05-20T01:17:47.024863+0000","last_became_peered":"2023-05-20T01:17:47.024863+0000","last_unstale":"2023-05-20T01:17:48.007751+0000","last_undegraded":"2023-05-20T01:17:48.007751+0000","last_fullsized":"2023-05-20T01:17:48.007751+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T01:45:54.786657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:48.004790+0000","last_change":"2023-05-20T01:17:46.587017+0000","last_active":"2023-05-20T01:17:48.004790+0000","last_peered":"2023-05-20T01:17:48.004790+0000","last_clean":"2023-05-20T01:17:48.004790+0000","last_became_active":"2023-05-20T01:17:46.586745+0000","last_became_peered":"2023-05-20T01:17:46.586745+0000","last_unstale":"2023-05-20T01:17:48.004790+0000","last_undegraded":"2023-05-20T01:17:48.004790+0000","last_fullsized":"2023-05-20T01:17:48.004790+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_clean_scrub_stamp":"2023-05-20T01:17:45.555502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:09:28.675043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:48.004465+0000","last_change":"2023-05-20T01:17:47.031292+0000","last_active":"2023-05-20T01:17:48.004465+0000","last_peered":"2023-05-20T01:17:48.004465+0000","last_clean":"2023-05-20T01:17:48.004465+0000","last_became_active":"2023-05-20T01:17:47.027841+0000","last_became_peered":"2023-05-20T01:17:47.027841+0000","last_unstale":"2023-05-20T01:17:48.004465+0000","last_undegraded":"2023-05-20T01:17:48.004465+0000","last_fullsized":"2023-05-20T01:17:48.004465+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:20:22.032473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"39'2","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:48.004738+0000","last_change":"2023-05-20T01:17:46.588535+0000","last_active":"2023-05-20T01:17:48.004738+0000","last_peered":"2023-05-20T01:17:48.004738+0000","last_clean":"2023-05-20T01:17:48.004738+0000","last_became_active":"2023-05-20T01:17:46.588227+0000","last_became_peered":"2023-05-20T01:17:46.588227+0000","last_unstale":"2023-05-20T01:17:48.004738+0000","last_undegraded":"2023-05-20T01:17:48.004738+0000","last_fullsized":"2023-05-20T01:17:48.004738+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_clean_scrub_stamp":"2023-05-20T01:17:45.555502+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:09:33.571525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.620722+0000","last_change":"2023-05-20T01:17:47.016444+0000","last_active":"2023-05-20T01:17:49.620722+0000","last_peered":"2023-05-20T01:17:49.620722+0000","last_clean":"2023-05-20T01:17:49.620722+0000","last_became_active":"2023-05-20T01:17:47.016170+0000","last_became_peered":"2023-05-20T01:17:47.016170+0000","last_unstale":"2023-05-20T01:17:49.620722+0000","last_undegraded":"2023-05-20T01:17:49.620722+0000","last_fullsized":"2023-05-20T01:17:49.620722+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:52:26.013882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.620616+0000","last_change":"2023-05-20T01:17:46.587258+0000","last_active":"2023-05-20T01:17:49.620616+0000","last_peered":"2023-05-20T01:17:49.620616+0000","last_clean":"2023-05-20T01:17:49.620616+0000","last_became_active":"2023-05-20T01:17:46.586947+0000","last_became_peered":"2023-05-20T01:17:46.586947+0000","last_unstale":"2023-05-20T01:17:49.620616+0000","last_undegraded":"2023-05-20T01:17:49.620616+0000","last_fullsized":"2023-05-20T01:17:49.620616+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_clean_scrub_stamp":"2023-05-20T01:17:45.555502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:06:11.570396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.620402+0000","last_change":"2023-05-20T01:17:47.021891+0000","last_active":"2023-05-20T01:17:49.620402+0000","last_peered":"2023-05-20T01:17:49.620402+0000","last_clean":"2023-05-20T01:17:49.620402+0000","last_became_active":"2023-05-20T01:17:47.021595+0000","last_became_peered":"2023-05-20T01:17:47.021595+0000","last_unstale":"2023-05-20T01:17:49.620402+0000","last_undegraded":"2023-05-20T01:17:49.620402+0000","last_fullsized":"2023-05-20T01:17:49.620402+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:57:31.565290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.119323+0000","last_change":"2023-05-20T01:17:46.591205+0000","last_active":"2023-05-20T01:17:49.119323+0000","last_peered":"2023-05-20T01:17:49.119323+0000","last_clean":"2023-05-20T01:17:49.119323+0000","last_became_active":"2023-05-20T01:17:46.590877+0000","last_became_peered":"2023-05-20T01:17:46.590877+0000","last_unstale":"2023-05-20T01:17:49.119323+0000","last_undegraded":"2023-05-20T01:17:49.119323+0000","last_fullsized":"2023-05-20T01:17:49.119323+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_clean_scrub_stamp":"2023-05-20T01:17:45.555502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:37:47.221408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.119379+0000","last_change":"2023-05-20T01:17:47.029304+0000","last_active":"2023-05-20T01:17:49.119379+0000","last_peered":"2023-05-20T01:17:49.119379+0000","last_clean":"2023-05-20T01:17:49.119379+0000","last_became_active":"2023-05-20T01:17:47.029029+0000","last_became_peered":"2023-05-20T01:17:47.029029+0000","last_unstale":"2023-05-20T01:17:49.119379+0000","last_undegraded":"2023-05-20T01:17:49.119379+0000","last_fullsized":"2023-05-20T01:17:49.119379+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:50:39.321269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.620250+0000","last_change":"2023-05-20T01:17:46.592664+0000","last_active":"2023-05-20T01:17:49.620250+0000","last_peered":"2023-05-20T01:17:49.620250+0000","last_clean":"2023-05-20T01:17:49.620250+0000","last_became_active":"2023-05-20T01:17:46.592384+0000","last_became_peered":"2023-05-20T01:17:46.592384+0000","last_unstale":"2023-05-20T01:17:49.620250+0000","last_undegraded":"2023-05-20T01:17:49.620250+0000","last_fullsized":"2023-05-20T01:17:49.620250+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_clean_scrub_stamp":"2023-05-20T01:17:45.555502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:50:46.378269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:48.004628+0000","last_change":"2023-05-20T01:17:47.031899+0000","last_active":"2023-05-20T01:17:48.004628+0000","last_peered":"2023-05-20T01:17:48.004628+0000","last_clean":"2023-05-20T01:17:48.004628+0000","last_became_active":"2023-05-20T01:17:47.031654+0000","last_became_peered":"2023-05-20T01:17:47.031654+0000","last_unstale":"2023-05-20T01:17:48.004628+0000","last_undegraded":"2023-05-20T01:17:48.004628+0000","last_fullsized":"2023-05-20T01:17:48.004628+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:07:17.632947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:48.005158+0000","last_change":"2023-05-20T01:17:46.589190+0000","last_active":"2023-05-20T01:17:48.005158+0000","last_peered":"2023-05-20T01:17:48.005158+0000","last_clean":"2023-05-20T01:17:48.005158+0000","last_became_active":"2023-05-20T01:17:46.588893+0000","last_became_peered":"2023-05-20T01:17:46.588893+0000","last_unstale":"2023-05-20T01:17:48.005158+0000","last_undegraded":"2023-05-20T01:17:48.005158+0000","last_fullsized":"2023-05-20T01:17:48.005158+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_clean_scrub_stamp":"2023-05-20T01:17:45.555502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:53:34.150992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":53,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:48.004878+0000","last_change":"2023-05-20T01:16:23.289068+0000","last_active":"2023-05-20T01:17:48.004878+0000","last_peered":"2023-05-20T01:17:48.004878+0000","last_clean":"2023-05-20T01:17:48.004878+0000","last_became_active":"2023-05-20T01:16:23.288781+0000","last_became_peered":"2023-05-20T01:16:23.288781+0000","last_unstale":"2023-05-20T01:17:48.004878+0000","last_undegraded":"2023-05-20T01:17:48.004878+0000","last_fullsized":"2023-05-20T01:17:48.004878+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:55:51.655368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:48.004506+0000","last_change":"2023-05-20T01:17:47.015826+0000","last_active":"2023-05-20T01:17:48.004506+0000","last_peered":"2023-05-20T01:17:48.004506+0000","last_clean":"2023-05-20T01:17:48.004506+0000","last_became_active":"2023-05-20T01:17:47.015560+0000","last_became_peered":"2023-05-20T01:17:47.015560+0000","last_unstale":"2023-05-20T01:17:48.004506+0000","last_undegraded":"2023-05-20T01:17:48.004506+0000","last_fullsized":"2023-05-20T01:17:48.004506+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:01:30.844749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.621141+0000","last_change":"2023-05-20T01:17:46.595158+0000","last_active":"2023-05-20T01:17:49.621141+0000","last_peered":"2023-05-20T01:17:49.621141+0000","last_clean":"2023-05-20T01:17:49.621141+0000","last_became_active":"2023-05-20T01:17:46.594892+0000","last_became_peered":"2023-05-20T01:17:46.594892+0000","last_unstale":"2023-05-20T01:17:49.621141+0000","last_undegraded":"2023-05-20T01:17:49.621141+0000","last_fullsized":"2023-05-20T01:17:49.621141+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_clean_scrub_stamp":"2023-05-20T01:17:45.555502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:40:30.936198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":57,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:48.004803+0000","last_change":"2023-05-20T01:16:22.829324+0000","last_active":"2023-05-20T01:17:48.004803+0000","last_peered":"2023-05-20T01:17:48.004803+0000","last_clean":"2023-05-20T01:17:48.004803+0000","last_became_active":"2023-05-20T01:16:22.828964+0000","last_became_peered":"2023-05-20T01:16:22.828964+0000","last_unstale":"2023-05-20T01:17:48.004803+0000","last_undegraded":"2023-05-20T01:17:48.004803+0000","last_fullsized":"2023-05-20T01:17:48.004803+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:54:31.790865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.621246+0000","last_change":"2023-05-20T01:17:47.028251+0000","last_active":"2023-05-20T01:17:49.621246+0000","last_peered":"2023-05-20T01:17:49.621246+0000","last_clean":"2023-05-20T01:17:49.621246+0000","last_became_active":"2023-05-20T01:17:47.028039+0000","last_became_peered":"2023-05-20T01:17:47.028039+0000","last_unstale":"2023-05-20T01:17:49.621246+0000","last_undegraded":"2023-05-20T01:17:49.621246+0000","last_fullsized":"2023-05-20T01:17:49.621246+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:35:56.309239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.620553+0000","last_change":"2023-05-20T01:17:46.592565+0000","last_active":"2023-05-20T01:17:49.620553+0000","last_peered":"2023-05-20T01:17:49.620553+0000","last_clean":"2023-05-20T01:17:49.620553+0000","last_became_active":"2023-05-20T01:17:46.592182+0000","last_became_peered":"2023-05-20T01:17:46.592182+0000","last_unstale":"2023-05-20T01:17:49.620553+0000","last_undegraded":"2023-05-20T01:17:49.620553+0000","last_fullsized":"2023-05-20T01:17:49.620553+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_clean_scrub_stamp":"2023-05-20T01:17:45.555502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:24:05.756976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":53,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.620763+0000","last_change":"2023-05-20T01:16:22.834754+0000","last_active":"2023-05-20T01:17:49.620763+0000","last_peered":"2023-05-20T01:17:49.620763+0000","last_clean":"2023-05-20T01:17:49.620763+0000","last_became_active":"2023-05-20T01:16:22.834437+0000","last_became_peered":"2023-05-20T01:16:22.834437+0000","last_unstale":"2023-05-20T01:17:49.620763+0000","last_undegraded":"2023-05-20T01:17:49.620763+0000","last_fullsized":"2023-05-20T01:17:49.620763+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:16:13.901189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":201,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.620352+0000","last_change":"2023-05-20T01:16:22.834920+0000","last_active":"2023-05-20T01:17:49.620352+0000","last_peered":"2023-05-20T01:17:49.620352+0000","last_clean":"2023-05-20T01:17:49.620352+0000","last_became_active":"2023-05-20T01:16:22.833924+0000","last_became_peered":"2023-05-20T01:16:22.833924+0000","last_unstale":"2023-05-20T01:17:49.620352+0000","last_undegraded":"2023-05-20T01:17:49.620352+0000","last_fullsized":"2023-05-20T01:17:49.620352+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:09:46.900791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.120096+0000","last_change":"2023-05-20T01:17:47.014401+0000","last_active":"2023-05-20T01:17:49.120096+0000","last_peered":"2023-05-20T01:17:49.120096+0000","last_clean":"2023-05-20T01:17:49.120096+0000","last_became_active":"2023-05-20T01:17:47.014181+0000","last_became_peered":"2023-05-20T01:17:47.014181+0000","last_unstale":"2023-05-20T01:17:49.120096+0000","last_undegraded":"2023-05-20T01:17:49.120096+0000","last_fullsized":"2023-05-20T01:17:49.120096+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:20:03.514131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.621316+0000","last_change":"2023-05-20T01:17:46.596401+0000","last_active":"2023-05-20T01:17:49.621316+0000","last_peered":"2023-05-20T01:17:49.621316+0000","last_clean":"2023-05-20T01:17:49.621316+0000","last_became_active":"2023-05-20T01:17:46.596117+0000","last_became_peered":"2023-05-20T01:17:46.596117+0000","last_unstale":"2023-05-20T01:17:49.621316+0000","last_undegraded":"2023-05-20T01:17:49.621316+0000","last_fullsized":"2023-05-20T01:17:49.621316+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_clean_scrub_stamp":"2023-05-20T01:17:45.555502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:49:04.107823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":51,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.620973+0000","last_change":"2023-05-20T01:16:23.290259+0000","last_active":"2023-05-20T01:17:49.620973+0000","last_peered":"2023-05-20T01:17:49.620973+0000","last_clean":"2023-05-20T01:17:49.620973+0000","last_became_active":"2023-05-20T01:16:23.289911+0000","last_became_peered":"2023-05-20T01:16:23.289911+0000","last_unstale":"2023-05-20T01:17:49.620973+0000","last_undegraded":"2023-05-20T01:17:49.620973+0000","last_fullsized":"2023-05-20T01:17:49.620973+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:43:43.702378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.119906+0000","last_change":"2023-05-20T01:17:47.022729+0000","last_active":"2023-05-20T01:17:49.119906+0000","last_peered":"2023-05-20T01:17:49.119906+0000","last_clean":"2023-05-20T01:17:49.119906+0000","last_became_active":"2023-05-20T01:17:47.022396+0000","last_became_peered":"2023-05-20T01:17:47.022396+0000","last_unstale":"2023-05-20T01:17:49.119906+0000","last_undegraded":"2023-05-20T01:17:49.119906+0000","last_fullsized":"2023-05-20T01:17:49.119906+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:50:47.388284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"39'2","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.620726+0000","last_change":"2023-05-20T01:17:46.585226+0000","last_active":"2023-05-20T01:17:49.620726+0000","last_peered":"2023-05-20T01:17:49.620726+0000","last_clean":"2023-05-20T01:17:49.620726+0000","last_became_active":"2023-05-20T01:17:46.584925+0000","last_became_peered":"2023-05-20T01:17:46.584925+0000","last_unstale":"2023-05-20T01:17:49.620726+0000","last_undegraded":"2023-05-20T01:17:49.620726+0000","last_fullsized":"2023-05-20T01:17:49.620726+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_clean_scrub_stamp":"2023-05-20T01:17:45.555502+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:22:22.071660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":51,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.620311+0000","last_change":"2023-05-20T01:16:23.290207+0000","last_active":"2023-05-20T01:17:49.620311+0000","last_peered":"2023-05-20T01:17:49.620311+0000","last_clean":"2023-05-20T01:17:49.620311+0000","last_became_active":"2023-05-20T01:16:23.289928+0000","last_became_peered":"2023-05-20T01:16:23.289928+0000","last_unstale":"2023-05-20T01:17:49.620311+0000","last_undegraded":"2023-05-20T01:17:49.620311+0000","last_fullsized":"2023-05-20T01:17:49.620311+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:18:38.754138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:48.007379+0000","last_change":"2023-05-20T01:17:47.024690+0000","last_active":"2023-05-20T01:17:48.007379+0000","last_peered":"2023-05-20T01:17:48.007379+0000","last_clean":"2023-05-20T01:17:48.007379+0000","last_became_active":"2023-05-20T01:17:47.024420+0000","last_became_peered":"2023-05-20T01:17:47.024420+0000","last_unstale":"2023-05-20T01:17:48.007379+0000","last_undegraded":"2023-05-20T01:17:48.007379+0000","last_fullsized":"2023-05-20T01:17:48.007379+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:31:34.112838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"40'1","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:48.714390+0000","last_change":"2023-05-20T01:17:46.585759+0000","last_active":"2023-05-20T01:17:48.714390+0000","last_peered":"2023-05-20T01:17:48.714390+0000","last_clean":"2023-05-20T01:17:48.714390+0000","last_became_active":"2023-05-20T01:17:46.585463+0000","last_became_peered":"2023-05-20T01:17:46.585463+0000","last_unstale":"2023-05-20T01:17:48.714390+0000","last_undegraded":"2023-05-20T01:17:48.714390+0000","last_fullsized":"2023-05-20T01:17:48.714390+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_clean_scrub_stamp":"2023-05-20T01:17:45.555502+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:39:10.749332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":51,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.620693+0000","last_change":"2023-05-20T01:16:23.290538+0000","last_active":"2023-05-20T01:17:49.620693+0000","last_peered":"2023-05-20T01:17:49.620693+0000","last_clean":"2023-05-20T01:17:49.620693+0000","last_became_active":"2023-05-20T01:16:23.290230+0000","last_became_peered":"2023-05-20T01:16:23.290230+0000","last_unstale":"2023-05-20T01:17:49.620693+0000","last_undegraded":"2023-05-20T01:17:49.620693+0000","last_fullsized":"2023-05-20T01:17:49.620693+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:44:32.467152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.620324+0000","last_change":"2023-05-20T01:17:47.016165+0000","last_active":"2023-05-20T01:17:49.620324+0000","last_peered":"2023-05-20T01:17:49.620324+0000","last_clean":"2023-05-20T01:17:49.620324+0000","last_became_active":"2023-05-20T01:17:47.015852+0000","last_became_peered":"2023-05-20T01:17:47.015852+0000","last_unstale":"2023-05-20T01:17:49.620324+0000","last_undegraded":"2023-05-20T01:17:49.620324+0000","last_fullsized":"2023-05-20T01:17:49.620324+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:53:58.456833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.620167+0000","last_change":"2023-05-20T01:17:46.593003+0000","last_active":"2023-05-20T01:17:49.620167+0000","last_peered":"2023-05-20T01:17:49.620167+0000","last_clean":"2023-05-20T01:17:49.620167+0000","last_became_active":"2023-05-20T01:17:46.592600+0000","last_became_peered":"2023-05-20T01:17:46.592600+0000","last_unstale":"2023-05-20T01:17:49.620167+0000","last_undegraded":"2023-05-20T01:17:49.620167+0000","last_fullsized":"2023-05-20T01:17:49.620167+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_clean_scrub_stamp":"2023-05-20T01:17:45.555502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:27:00.376244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":49,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:48.007522+0000","last_change":"2023-05-20T01:16:23.290341+0000","last_active":"2023-05-20T01:17:48.007522+0000","last_peered":"2023-05-20T01:17:48.007522+0000","last_clean":"2023-05-20T01:17:48.007522+0000","last_became_active":"2023-05-20T01:16:23.290053+0000","last_became_peered":"2023-05-20T01:16:23.290053+0000","last_unstale":"2023-05-20T01:17:48.007522+0000","last_undegraded":"2023-05-20T01:17:48.007522+0000","last_fullsized":"2023-05-20T01:17:48.007522+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:20:58.978210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:48.004662+0000","last_change":"2023-05-20T01:17:47.011981+0000","last_active":"2023-05-20T01:17:48.004662+0000","last_peered":"2023-05-20T01:17:48.004662+0000","last_clean":"2023-05-20T01:17:48.004662+0000","last_became_active":"2023-05-20T01:17:47.011661+0000","last_became_peered":"2023-05-20T01:17:47.011661+0000","last_unstale":"2023-05-20T01:17:48.004662+0000","last_undegraded":"2023-05-20T01:17:48.004662+0000","last_fullsized":"2023-05-20T01:17:48.004662+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:46.565518+0000","last_clean_scrub_stamp":"2023-05-20T01:17:46.565518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:09:29.685059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:49.119586+0000","last_change":"2023-05-20T01:17:46.587447+0000","last_active":"2023-05-20T01:17:49.119586+0000","last_peered":"2023-05-20T01:17:49.119586+0000","last_clean":"2023-05-20T01:17:49.119586+0000","last_became_active":"2023-05-20T01:17:46.586864+0000","last_became_peered":"2023-05-20T01:17:46.586864+0000","last_unstale":"2023-05-20T01:17:49.119586+0000","last_undegraded":"2023-05-20T01:17:49.119586+0000","last_fullsized":"2023-05-20T01:17:49.119586+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:17:45.555502+0000","last_clean_scrub_stamp":"2023-05-20T01:17:45.555502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:10:27.574182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":49,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-05-20T01:17:48.007600+0000","last_change":"2023-05-20T01:16:22.834330+0000","last_active":"2023-05-20T01:17:48.007600+0000","last_peered":"2023-05-20T01:17:48.007600+0000","last_clean":"2023-05-20T01:17:48.007600+0000","last_became_active":"2023-05-20T01:16:22.833946+0000","last_became_peered":"2023-05-20T01:16:22.833946+0000","last_unstale":"2023-05-20T01:17:48.007600+0000","last_undegraded":"2023-05-20T01:17:48.007600+0000","last_fullsized":"2023-05-20T01:17:48.007600+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:38:56.203247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574870,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9812,"kb_used_data":1036,"kb_used_omap":0,"kb_used_meta":8768,"kb_avail":94362028,"statfs":{"total":96636764160,"available":96626716672,"internally_reserved":0,"allocated":1060864,"data_stored":798734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8978432},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.53000000000000003}]},{"osd":1,"last update":"Sat May 20 01:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":8.8879999999999999,"5min":8.8879999999999999,"15min":8.8879999999999999},"last":0.45600000000000002},{"interface":"front","average":{"1min":1.01,"5min":1.01,"15min":1.01},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":8.875,"5min":8.875,"15min":8.875},"last":0.48399999999999999}]},{"osd":2,"last update":"Sat May 20 01:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":8.9120000000000008,"5min":8.9120000000000008,"15min":8.9120000000000008},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":8.8520000000000003,"5min":8.8520000000000003,"15min":8.8520000000000003},"last":0.433}]},{"osd":3,"last update":"Sat May 20 01:17:20 2023","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":1.054,"15min":1.054},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":10.884,"5min":10.884,"15min":10.884},"last":0.34899999999999998},{"interface":"front","average":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":10.952,"5min":10.952,"15min":10.952},"last":0.54100000000000004}]},{"osd":4,"last update":"Sat May 20 01:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.51200000000000001}]},{"osd":5,"last update":"Sat May 20 01:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.47199999999999998}]},{"osd":6,"last update":"Sat May 20 01:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.501},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.55600000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574869,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8596,"kb_used_data":460,"kb_used_omap":0,"kb_used_meta":8128,"kb_avail":94363244,"statfs":{"total":96636764160,"available":96627961856,"internally_reserved":0,"allocated":471040,"data_stored":208948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8323072},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.54200000000000004}]},{"osd":1,"last update":"Sat May 20 01:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.58599999999999997}]},{"osd":2,"last update":"Sat May 20 01:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.496},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.39600000000000002}]},{"osd":3,"last update":"Sat May 20 01:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.56200000000000006}]},{"osd":4,"last update":"Sat May 20 01:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.66700000000000004}]},{"osd":5,"last update":"Sat May 20 01:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.61599999999999999}]},{"osd":7,"last update":"Sat May 20 01:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.64500000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574869,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9748,"kb_used_data":1036,"kb_used_omap":0,"kb_used_meta":8704,"kb_avail":94362092,"statfs":{"total":96636764160,"available":96626782208,"internally_reserved":0,"allocated":1060864,"data_stored":798714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8912896},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.23000000000000001},{"interface":"front","average":{"1min":0.315,"5min":0.315,"15min":0.315},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.434,"5min":0.434,"15min":0.434},"last":0.27700000000000002}]},{"osd":2,"last update":"Sat May 20 01:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"last":0.24199999999999999},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.36699999999999999}]},{"osd":3,"last update":"Sat May 20 01:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.372,"15min":0.372},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.34999999999999998}]},{"osd":4,"last update":"Sat May 20 01:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.55700000000000005}]},{"osd":5,"last update":"Sat May 20 01:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.363,"5min":1.363,"15min":1.363},"last":0.59599999999999997}]},{"osd":6,"last update":"Sat May 20 01:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.58299999999999996}]},{"osd":7,"last update":"Sat May 20 01:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.319,"5min":1.319,"15min":1.319},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.68200000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574869,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8716,"kb_used_data":452,"kb_used_omap":0,"kb_used_meta":8256,"kb_avail":94363124,"statfs":{"total":96636764160,"available":96627838976,"internally_reserved":0,"allocated":462848,"data_stored":207349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8454144},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat May 20 01:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.376,"5min":0.376,"15min":0.376},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.52600000000000002}]},{"osd":2,"last update":"Sat May 20 01:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.627}]},{"osd":3,"last update":"Sat May 20 01:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.372,"5min":0.372,"15min":0.372},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"last":0.60099999999999998}]},{"osd":4,"last update":"Sat May 20 01:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.61399999999999999}]},{"osd":5,"last update":"Sat May 20 01:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.82699999999999996}]},{"osd":6,"last update":"Sat May 20 01:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.65300000000000002}]},{"osd":7,"last update":"Sat May 20 01:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.71999999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574869,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8540,"kb_used_data":468,"kb_used_omap":0,"kb_used_meta":8064,"kb_avail":94363300,"statfs":{"total":96636764160,"available":96628019200,"internally_reserved":0,"allocated":479232,"data_stored":209862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8257536},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.372,"5min":0.372,"15min":0.372},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.249}]},{"osd":1,"last update":"Sat May 20 01:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.434,"15min":0.434},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.4950000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.5189999999999999,"5min":1.5189999999999999,"15min":1.5189999999999999},"last":0.31}]},{"osd":3,"last update":"Sat May 20 01:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.317},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.5620000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.433}]},{"osd":4,"last update":"Sat May 20 01:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.319,"5min":1.319,"15min":1.319},"last":0.504},{"interface":"front","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.4319999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.42499999999999999}]},{"osd":5,"last update":"Sat May 20 01:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.758,"5min":1.758,"15min":1.758},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.7110000000000001,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.66300000000000003}]},{"osd":6,"last update":"Sat May 20 01:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.665,"5min":1.665,"15min":1.665},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.819,"5min":1.819,"15min":1.819},"last":0.57399999999999995}]},{"osd":7,"last update":"Sat May 20 01:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.7310000000000001,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.8460000000000001,"5min":1.8460000000000001,"15min":1.8460000000000001},"last":0.61199999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574869,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8728,"kb_used_data":464,"kb_used_omap":0,"kb_used_meta":8256,"kb_avail":94363112,"statfs":{"total":96636764160,"available":96627826688,"internally_reserved":0,"allocated":475136,"data_stored":208914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8454144},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.504,"5min":0.504,"15min":0.504},"last":0.221},{"interface":"front","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":3.1890000000000001,"5min":3.1890000000000001,"15min":3.1890000000000001},"last":0.28299999999999997}]},{"osd":1,"last update":"Sat May 20 01:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":3.4260000000000002,"5min":3.4260000000000002,"15min":3.4260000000000002},"last":0.26400000000000001},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.34599999999999997}]},{"osd":2,"last update":"Sat May 20 01:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.413,"5min":1.413,"15min":1.413},"last":0.40699999999999997}]},{"osd":4,"last update":"Sat May 20 01:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":3.4660000000000002,"5min":3.4660000000000002,"15min":3.4660000000000002},"last":0.37}]},{"osd":5,"last update":"Sat May 20 01:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":3.5379999999999998,"5min":3.5379999999999998,"15min":3.5379999999999998},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.385,"5min":1.385,"15min":1.385},"last":0.441}]},{"osd":6,"last update":"Sat May 20 01:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.347,"5min":1.347,"15min":1.347},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.57399999999999995}]},{"osd":7,"last update":"Sat May 20 01:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":3.5009999999999999,"5min":3.5009999999999999,"15min":3.5009999999999999},"last":0.501}]}]},{"osd":4,"up_from":13,"seq":55834574869,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8792,"kb_used_data":464,"kb_used_omap":0,"kb_used_meta":8320,"kb_avail":94363048,"statfs":{"total":96636764160,"available":96627761152,"internally_reserved":0,"allocated":475136,"data_stored":209840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8519680},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.51300000000000001}]},{"osd":1,"last update":"Sat May 20 01:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.434},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.52700000000000002}]},{"osd":2,"last update":"Sat May 20 01:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.57199999999999995}]},{"osd":3,"last update":"Sat May 20 01:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":3.1749999999999998,"5min":3.1749999999999998,"15min":3.1749999999999998},"last":0.54000000000000004}]},{"osd":5,"last update":"Sat May 20 01:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.499},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"last":0.38800000000000001}]},{"osd":6,"last update":"Sat May 20 01:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.41099999999999998}]},{"osd":7,"last update":"Sat May 20 01:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.48599999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574870,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8652,"kb_used_data":452,"kb_used_omap":0,"kb_used_meta":8192,"kb_avail":94363188,"statfs":{"total":96636764160,"available":96627904512,"internally_reserved":0,"allocated":462848,"data_stored":207385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8388608},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:17:24 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":6.3929999999999998,"5min":6.3929999999999998,"15min":6.3929999999999998},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":6.3339999999999996,"5min":6.3339999999999996,"15min":6.3339999999999996},"last":0.62}]},{"osd":1,"last update":"Sat May 20 01:17:24 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":3.2450000000000001,"5min":3.2450000000000001,"15min":3.2450000000000001},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.58499999999999996}]},{"osd":2,"last update":"Sat May 20 01:17:24 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":0.60699999999999998}]},{"osd":3,"last update":"Sat May 20 01:17:24 2023","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":6.351,"5min":6.351,"15min":6.351},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":6.3730000000000002,"5min":6.3730000000000002,"15min":6.3730000000000002},"last":0.64700000000000002}]},{"osd":4,"last update":"Sat May 20 01:17:24 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.44600000000000001}]},{"osd":6,"last update":"Sat May 20 01:17:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.54900000000000004}]},{"osd":7,"last update":"Sat May 20 01:17:24 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.52800000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-05-20T01:17:51.866 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-05-20T01:17:51.867 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-05-20T01:17:52.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.028+0000 7f9b89d1c700 1 Processor -- start 2023-05-20T01:17:52.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.028+0000 7f9b89d1c700 1 -- start start 2023-05-20T01:17:52.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.028+0000 7f9b89d1c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9b8415cd20 0x7f9b840b1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:52.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.028+0000 7f9b89d1c700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f9b840b14c0 0x7f9b8406b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:52.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.028+0000 7f9b89d1c700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b840cb700 con 0x7f9b8406bb60 2023-05-20T01:17:52.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.028+0000 7f9b89d1c700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b8406bf80 con 0x7f9b8415cd20 2023-05-20T01:17:52.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.028+0000 7f9b89d1c700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b840b2430 con 0x7f9b840b14c0 2023-05-20T01:17:52.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.029+0000 7f9b837fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9b8415cd20 0x7f9b840b1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:52.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.029+0000 7f9b83fff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f9b8406bb60 0x7f9b840cd930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:43594/0 (socket says 172.21.15.16:43594) 2023-05-20T01:17:52.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.029+0000 7f9b83fff700 1 -- 172.21.15.16:0/984631262 learned_addr learned my addr 172.21.15.16:0/984631262 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:17:52.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.029+0000 7f9b82ffd700 1 --2- 172.21.15.16:0/984631262 >> v2:172.21.15.132:3300/0 conn(0x7f9b840b14c0 0x7f9b8406b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:52.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.029+0000 7f9b837fe700 1 -- 172.21.15.16:0/984631262 >> v2:172.21.15.132:3300/0 conn(0x7f9b840b14c0 msgr2=0x7f9b8406b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:52.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.029+0000 7f9b837fe700 1 --2- 172.21.15.16:0/984631262 >> v2:172.21.15.132:3300/0 conn(0x7f9b840b14c0 0x7f9b8406b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:52.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.029+0000 7f9b837fe700 1 -- 172.21.15.16:0/984631262 >> v1:172.21.15.16:6789/0 conn(0x7f9b8406bb60 legacy=0x7f9b840cd930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:52.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.029+0000 7f9b837fe700 1 -- 172.21.15.16:0/984631262 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b840b2710 con 0x7f9b8415cd20 2023-05-20T01:17:52.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.029+0000 7f9b82ffd700 1 --2- 172.21.15.16:0/984631262 >> v2:172.21.15.132:3300/0 conn(0x7f9b840b14c0 0x7f9b8406b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:17:52.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.029+0000 7f9b827fc700 1 -- 172.21.15.16:0/984631262 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 494725295 0 0) 0x7f9b840cb700 con 0x7f9b8406bb60 2023-05-20T01:17:52.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.029+0000 7f9b837fe700 1 --2- 172.21.15.16:0/984631262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b8415cd20 0x7f9b840b1e50 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f9b840b3bb0 tx=0x7f9b74009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92ec8fa185e5156 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:52.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.029+0000 7f9b827fc700 1 -- 172.21.15.16:0/984631262 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b74011830 con 0x7f9b8415cd20 2023-05-20T01:17:52.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.029+0000 7f9b827fc700 1 -- 172.21.15.16:0/984631262 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b74011990 con 0x7f9b8415cd20 2023-05-20T01:17:52.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.029+0000 7f9b827fc700 1 -- 172.21.15.16:0/984631262 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b74011d20 con 0x7f9b8415cd20 2023-05-20T01:17:52.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.030+0000 7f9b89d1c700 1 -- 172.21.15.16:0/984631262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b8415cd20 msgr2=0x7f9b840b1e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:52.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.030+0000 7f9b89d1c700 1 --2- 172.21.15.16:0/984631262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b8415cd20 0x7f9b840b1e50 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f9b840b3bb0 tx=0x7f9b74009b20 comp rx=0 tx=0).stop 2023-05-20T01:17:52.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.030+0000 7f9b89d1c700 1 -- 172.21.15.16:0/984631262 shutdown_connections 2023-05-20T01:17:52.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.030+0000 7f9b89d1c700 1 --2- 172.21.15.16:0/984631262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b8415cd20 0x7f9b840b1e50 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:52.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.030+0000 7f9b89d1c700 1 --2- 172.21.15.16:0/984631262 >> v2:172.21.15.132:3300/0 conn(0x7f9b840b14c0 0x7f9b8406b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:52.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.030+0000 7f9b89d1c700 1 -- 172.21.15.16:0/984631262 >> 172.21.15.16:0/984631262 conn(0x7f9b8406ae30 msgr2=0x7f9b8409a400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:52.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.030+0000 7f9b89d1c700 1 -- 172.21.15.16:0/984631262 shutdown_connections 2023-05-20T01:17:52.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.030+0000 7f9b89d1c700 1 -- 172.21.15.16:0/984631262 wait complete. 2023-05-20T01:17:52.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.031+0000 7f9b89d1c700 1 Processor -- start 2023-05-20T01:17:52.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.031+0000 7f9b89d1c700 1 -- start start 2023-05-20T01:17:52.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.032+0000 7f9b89d1c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9b8406bb60 0x7f9b8415a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:52.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.032+0000 7f9b89d1c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b840b14c0 0x7f9b840c7140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:52.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.032+0000 7f9b89d1c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9b8415cd20 0x7f9b8406e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:52.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.032+0000 7f9b89d1c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b840c6d20 con 0x7f9b840b14c0 2023-05-20T01:17:52.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.032+0000 7f9b89d1c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b840c7680 con 0x7f9b8415cd20 2023-05-20T01:17:52.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.032+0000 7f9b89d1c700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b8406eda0 con 0x7f9b8406bb60 2023-05-20T01:17:52.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.032+0000 7f9b82ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b840b14c0 0x7f9b840c7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:52.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.032+0000 7f9b82ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b840b14c0 0x7f9b840c7140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60302/0 (socket says 172.21.15.16:60302) 2023-05-20T01:17:52.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.032+0000 7f9b82ffd700 1 -- 172.21.15.16:0/2424067452 learned_addr learned my addr 172.21.15.16:0/2424067452 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:52.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.032+0000 7f9b83fff700 1 --2- 172.21.15.16:0/2424067452 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9b8415cd20 0x7f9b8406e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:52.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.032+0000 7f9b837fe700 1 --2- 172.21.15.16:0/2424067452 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9b8406bb60 0x7f9b8415a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:52.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.032+0000 7f9b82ffd700 1 -- 172.21.15.16:0/2424067452 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9b8406bb60 msgr2=0x7f9b8415a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:52.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.032+0000 7f9b82ffd700 1 --2- 172.21.15.16:0/2424067452 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9b8406bb60 0x7f9b8415a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:52.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.032+0000 7f9b82ffd700 1 -- 172.21.15.16:0/2424067452 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9b8415cd20 msgr2=0x7f9b8406e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:52.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.032+0000 7f9b82ffd700 1 --2- 172.21.15.16:0/2424067452 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9b8415cd20 0x7f9b8406e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:52.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.032+0000 7f9b82ffd700 1 -- 172.21.15.16:0/2424067452 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b740094a0 con 0x7f9b840b14c0 2023-05-20T01:17:52.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.033+0000 7f9b82ffd700 1 --2- 172.21.15.16:0/2424067452 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b840b14c0 0x7f9b840c7140 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f9b6c00e840 tx=0x7f9b6c00ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:52.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.033+0000 7f9b7bfff700 1 -- 172.21.15.16:0/2424067452 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b6c004230 con 0x7f9b840b14c0 2023-05-20T01:17:52.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.033+0000 7f9b7bfff700 1 -- 172.21.15.16:0/2424067452 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b6c004390 con 0x7f9b840b14c0 2023-05-20T01:17:52.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.033+0000 7f9b89d1c700 1 -- 172.21.15.16:0/2424067452 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b840cbc30 con 0x7f9b840b14c0 2023-05-20T01:17:52.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.033+0000 7f9b7bfff700 1 -- 172.21.15.16:0/2424067452 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b6c0036a0 con 0x7f9b840b14c0 2023-05-20T01:17:52.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.033+0000 7f9b89d1c700 1 -- 172.21.15.16:0/2424067452 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b840cb950 con 0x7f9b840b14c0 2023-05-20T01:17:52.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.034+0000 7f9b89d1c700 1 -- 172.21.15.16:0/2424067452 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b840b1960 con 0x7f9b840b14c0 2023-05-20T01:17:52.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.034+0000 7f9b7bfff700 1 -- 172.21.15.16:0/2424067452 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9b6c003800 con 0x7f9b840b14c0 2023-05-20T01:17:52.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.034+0000 7f9b7bfff700 1 --2- 172.21.15.16:0/2424067452 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f9b6406ace0 0x7f9b6406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:52.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.035+0000 7f9b7bfff700 1 -- 172.21.15.16:0/2424067452 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f9b6c089000 con 0x7f9b840b14c0 2023-05-20T01:17:52.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.035+0000 7f9b837fe700 1 --2- 172.21.15.16:0/2424067452 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f9b6406ace0 0x7f9b6406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:52.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.035+0000 7f9b837fe700 1 --2- 172.21.15.16:0/2424067452 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f9b6406ace0 0x7f9b6406d190 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9b740090b0 tx=0x7f9b74008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:52.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.039+0000 7f9b7bfff700 1 -- 172.21.15.16:0/2424067452 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f9b6c057a90 con 0x7f9b840b14c0 2023-05-20T01:17:52.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.316+0000 7f9b89d1c700 1 -- 172.21.15.16:0/2424067452 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f9b840d1f60 con 0x7f9b840b14c0 2023-05-20T01:17:52.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.320+0000 7f9b7bfff700 1 -- 172.21.15.16:0/2424067452 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v9) v1 ==== 190+0+0 (secure 0 0 0) 0x7f9b6c0094c0 con 0x7f9b840b14c0 2023-05-20T01:17:52.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.322+0000 7f9b89d1c700 1 -- 172.21.15.16:0/2424067452 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f9b6406ace0 msgr2=0x7f9b6406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:52.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.322+0000 7f9b89d1c700 1 --2- 172.21.15.16:0/2424067452 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f9b6406ace0 0x7f9b6406d190 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9b740090b0 tx=0x7f9b74008030 comp rx=0 tx=0).stop 2023-05-20T01:17:52.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.322+0000 7f9b89d1c700 1 -- 172.21.15.16:0/2424067452 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b840b14c0 msgr2=0x7f9b840c7140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:52.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.322+0000 7f9b89d1c700 1 --2- 172.21.15.16:0/2424067452 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b840b14c0 0x7f9b840c7140 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f9b6c00e840 tx=0x7f9b6c00ec00 comp rx=0 tx=0).stop 2023-05-20T01:17:52.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.322+0000 7f9b89d1c700 1 -- 172.21.15.16:0/2424067452 shutdown_connections 2023-05-20T01:17:52.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.322+0000 7f9b89d1c700 1 --2- 172.21.15.16:0/2424067452 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9b8415cd20 0x7f9b8406e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:52.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.322+0000 7f9b89d1c700 1 --2- 172.21.15.16:0/2424067452 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f9b6406ace0 0x7f9b6406d190 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:52.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.322+0000 7f9b89d1c700 1 --2- 172.21.15.16:0/2424067452 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9b8406bb60 0x7f9b8415a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:52.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.322+0000 7f9b89d1c700 1 --2- 172.21.15.16:0/2424067452 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b840b14c0 0x7f9b840c7140 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:52.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.322+0000 7f9b89d1c700 1 -- 172.21.15.16:0/2424067452 >> 172.21.15.16:0/2424067452 conn(0x7f9b8406ae30 msgr2=0x7f9b8415c480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:52.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.322+0000 7f9b89d1c700 1 -- 172.21.15.16:0/2424067452 shutdown_connections 2023-05-20T01:17:52.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.322+0000 7f9b89d1c700 1 -- 172.21.15.16:0/2424067452 wait complete. 2023-05-20T01:17:52.328 INFO:teuthology.orchestra.run.smithi016.stderr:updated caps for client.0 2023-05-20T01:17:52.338 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-20T01:17:52.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.459+0000 7f87ca68d700 1 Processor -- start 2023-05-20T01:17:52.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.459+0000 7f87ca68d700 1 -- start start 2023-05-20T01:17:52.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.459+0000 7f87ca68d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f87c406cb90 0x7f87c415ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:52.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.459+0000 7f87ca68d700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f87c406cf60 0x7f87c4070910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:52.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.459+0000 7f87ca68d700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87c40b31a0 con 0x7f87c40b54f0 2023-05-20T01:17:52.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.459+0000 7f87ca68d700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87c40cc990 con 0x7f87c406cb90 2023-05-20T01:17:52.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.459+0000 7f87ca68d700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87c40b3480 con 0x7f87c406cf60 2023-05-20T01:17:52.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.460+0000 7f87c3fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f87c406cb90 0x7f87c415ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:52.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.460+0000 7f87c3fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f87c406cb90 0x7f87c415ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60308/0 (socket says 172.21.15.16:60308) 2023-05-20T01:17:52.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.460+0000 7f87c3fff700 1 -- 172.21.15.16:0/3701442528 learned_addr learned my addr 172.21.15.16:0/3701442528 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:52.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.460+0000 7f87c8c2a700 1 --2- 172.21.15.16:0/3701442528 >> v2:172.21.15.132:3300/0 conn(0x7f87c406cf60 0x7f87c4070910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:52.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.460+0000 7f87c3fff700 1 -- 172.21.15.16:0/3701442528 >> v2:172.21.15.132:3300/0 conn(0x7f87c406cf60 msgr2=0x7f87c4070910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:52.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.460+0000 7f87c3fff700 1 --2- 172.21.15.16:0/3701442528 >> v2:172.21.15.132:3300/0 conn(0x7f87c406cf60 0x7f87c4070910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:52.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.460+0000 7f87c3fff700 1 -- 172.21.15.16:0/3701442528 >> v1:172.21.15.132:6789/0 conn(0x7f87c40b54f0 legacy=0x7f87c40b29e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:52.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.460+0000 7f87c3fff700 1 -- 172.21.15.16:0/3701442528 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87c415af60 con 0x7f87c406cb90 2023-05-20T01:17:52.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.460+0000 7f87c3fff700 1 --2- 172.21.15.16:0/3701442528 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87c406cb90 0x7f87c415ba40 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f87c4158a20 tx=0x7f87b4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=867b5369fb57b021 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:52.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.460+0000 7f87c2ffd700 1 -- 172.21.15.16:0/3701442528 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b4004230 con 0x7f87c406cb90 2023-05-20T01:17:52.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.460+0000 7f87c2ffd700 1 -- 172.21.15.16:0/3701442528 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87b4004390 con 0x7f87c406cb90 2023-05-20T01:17:52.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.460+0000 7f87c2ffd700 1 -- 172.21.15.16:0/3701442528 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b40036a0 con 0x7f87c406cb90 2023-05-20T01:17:52.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.461+0000 7f87ca68d700 1 -- 172.21.15.16:0/3701442528 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87c406cb90 msgr2=0x7f87c415ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:52.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.461+0000 7f87ca68d700 1 --2- 172.21.15.16:0/3701442528 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87c406cb90 0x7f87c415ba40 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f87c4158a20 tx=0x7f87b4009bd0 comp rx=0 tx=0).stop 2023-05-20T01:17:52.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.461+0000 7f87ca68d700 1 -- 172.21.15.16:0/3701442528 shutdown_connections 2023-05-20T01:17:52.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.461+0000 7f87ca68d700 1 --2- 172.21.15.16:0/3701442528 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87c406cb90 0x7f87c415ba40 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:52.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.461+0000 7f87ca68d700 1 --2- 172.21.15.16:0/3701442528 >> v2:172.21.15.132:3300/0 conn(0x7f87c406cf60 0x7f87c4070910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:52.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.461+0000 7f87ca68d700 1 -- 172.21.15.16:0/3701442528 >> 172.21.15.16:0/3701442528 conn(0x7f87c40661b0 msgr2=0x7f87c4066580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:52.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.461+0000 7f87ca68d700 1 -- 172.21.15.16:0/3701442528 shutdown_connections 2023-05-20T01:17:52.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.461+0000 7f87ca68d700 1 -- 172.21.15.16:0/3701442528 wait complete. 2023-05-20T01:17:52.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.462+0000 7f87ca68d700 1 Processor -- start 2023-05-20T01:17:52.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.462+0000 7f87ca68d700 1 -- start start 2023-05-20T01:17:52.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.463+0000 7f87ca68d700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87c406cb90 0x7f87c41480f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:52.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.463+0000 7f87ca68d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87c406cf60 0x7f87c40ca7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:52.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.463+0000 7f87ca68d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f87c40b54f0 0x7f87c40c8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:52.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.463+0000 7f87ca68d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87c40cb4b0 con 0x7f87c406cf60 2023-05-20T01:17:52.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.463+0000 7f87ca68d700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87c40cb1d0 con 0x7f87c40b54f0 2023-05-20T01:17:52.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.463+0000 7f87ca68d700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87c41566c0 con 0x7f87c406cb90 2023-05-20T01:17:52.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.463+0000 7f87c37fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87c406cf60 0x7f87c40ca7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:52.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.463+0000 7f87c37fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87c406cf60 0x7f87c40ca7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60314/0 (socket says 172.21.15.16:60314) 2023-05-20T01:17:52.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.463+0000 7f87c37fe700 1 -- 172.21.15.16:0/1335216137 learned_addr learned my addr 172.21.15.16:0/1335216137 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:52.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.463+0000 7f87c8c2a700 1 --2- 172.21.15.16:0/1335216137 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f87c40b54f0 0x7f87c40c8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:52.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.463+0000 7f87c3fff700 1 --2- 172.21.15.16:0/1335216137 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87c406cb90 0x7f87c41480f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:52.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.463+0000 7f87c37fe700 1 -- 172.21.15.16:0/1335216137 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87c406cb90 msgr2=0x7f87c41480f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:52.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.463+0000 7f87c37fe700 1 --2- 172.21.15.16:0/1335216137 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87c406cb90 0x7f87c41480f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:52.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.463+0000 7f87c37fe700 1 -- 172.21.15.16:0/1335216137 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f87c40b54f0 msgr2=0x7f87c40c8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:52.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.463+0000 7f87c37fe700 1 --2- 172.21.15.16:0/1335216137 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f87c40b54f0 0x7f87c40c8aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:52.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.463+0000 7f87c37fe700 1 -- 172.21.15.16:0/1335216137 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87b4009550 con 0x7f87c406cf60 2023-05-20T01:17:52.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.464+0000 7f87c37fe700 1 --2- 172.21.15.16:0/1335216137 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87c406cf60 0x7f87c40ca7f0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f87b800c4d0 tx=0x7f87b800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:52.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.464+0000 7f87c0ff9700 1 -- 172.21.15.16:0/1335216137 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b8011790 con 0x7f87c406cf60 2023-05-20T01:17:52.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.464+0000 7f87c0ff9700 1 -- 172.21.15.16:0/1335216137 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87b80118f0 con 0x7f87c406cf60 2023-05-20T01:17:52.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.464+0000 7f87c0ff9700 1 -- 172.21.15.16:0/1335216137 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b8011ca0 con 0x7f87c406cf60 2023-05-20T01:17:52.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.464+0000 7f87ca68d700 1 -- 172.21.15.16:0/1335216137 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87c415b520 con 0x7f87c406cf60 2023-05-20T01:17:52.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.464+0000 7f87ca68d700 1 -- 172.21.15.16:0/1335216137 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87c415b240 con 0x7f87c406cf60 2023-05-20T01:17:52.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.465+0000 7f87ca68d700 1 -- 172.21.15.16:0/1335216137 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87c406d360 con 0x7f87c406cf60 2023-05-20T01:17:52.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.465+0000 7f87c0ff9700 1 -- 172.21.15.16:0/1335216137 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f87b801c460 con 0x7f87c406cf60 2023-05-20T01:17:52.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.466+0000 7f87c0ff9700 1 --2- 172.21.15.16:0/1335216137 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f87b006ace0 0x7f87b006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:52.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.466+0000 7f87c0ff9700 1 -- 172.21.15.16:0/1335216137 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f87b80893e0 con 0x7f87c406cf60 2023-05-20T01:17:52.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.466+0000 7f87c3fff700 1 --2- 172.21.15.16:0/1335216137 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f87b006ace0 0x7f87b006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:52.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.466+0000 7f87c3fff700 1 --2- 172.21.15.16:0/1335216137 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f87b006ace0 0x7f87b006d190 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f87b4000c00 tx=0x7f87b4008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:52.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.470+0000 7f87c0ff9700 1 -- 172.21.15.16:0/1335216137 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f87b80543e0 con 0x7f87c406cf60 2023-05-20T01:17:52.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.711+0000 7f87ca68d700 1 -- 172.21.15.16:0/1335216137 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f87c4156520 con 0x7f87c406cf60 2023-05-20T01:17:52.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.711+0000 7f87c0ff9700 1 -- 172.21.15.16:0/1335216137 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4674 (secure 0 0 0) 0x7f87c4156520 con 0x7f87c406cf60 2023-05-20T01:17:52.715 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:17:52.715 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5176,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/4144439890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4144439890},{"type":"v1","addr":"172.21.15.16:6836","nonce":4144439890}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5188,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/2992755084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":2992755084},{"type":"v1","addr":"172.21.15.132:6834","nonce":2992755084}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/1979284857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1979284857},{"type":"v1","addr":"172.21.15.132:6835","nonce":1979284857}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-20T01:17:46.995921+0000","modified":"2023-05-20T01:17:50.320558+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5397},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5397":{"gid":5397,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.16:6837/942011026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":942011026},{"type":"v1","addr":"172.21.15.16:6837","nonce":942011026}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-05-20T01:17:52.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.713+0000 7f87ca68d700 1 -- 172.21.15.16:0/1335216137 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f87b006ace0 msgr2=0x7f87b006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:52.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.713+0000 7f87ca68d700 1 --2- 172.21.15.16:0/1335216137 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f87b006ace0 0x7f87b006d190 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f87b4000c00 tx=0x7f87b4008030 comp rx=0 tx=0).stop 2023-05-20T01:17:52.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.713+0000 7f87ca68d700 1 -- 172.21.15.16:0/1335216137 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87c406cf60 msgr2=0x7f87c40ca7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:52.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.713+0000 7f87ca68d700 1 --2- 172.21.15.16:0/1335216137 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87c406cf60 0x7f87c40ca7f0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f87b800c4d0 tx=0x7f87b800c890 comp rx=0 tx=0).stop 2023-05-20T01:17:52.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.713+0000 7f87ca68d700 1 -- 172.21.15.16:0/1335216137 shutdown_connections 2023-05-20T01:17:52.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.713+0000 7f87ca68d700 1 --2- 172.21.15.16:0/1335216137 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f87c40b54f0 0x7f87c40c8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:52.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.713+0000 7f87ca68d700 1 --2- 172.21.15.16:0/1335216137 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f87b006ace0 0x7f87b006d190 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:52.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.713+0000 7f87ca68d700 1 --2- 172.21.15.16:0/1335216137 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87c406cb90 0x7f87c41480f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:52.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.713+0000 7f87ca68d700 1 --2- 172.21.15.16:0/1335216137 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87c406cf60 0x7f87c40ca7f0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:52.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.713+0000 7f87ca68d700 1 -- 172.21.15.16:0/1335216137 >> 172.21.15.16:0/1335216137 conn(0x7f87c40661b0 msgr2=0x7f87c40ce240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:52.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.713+0000 7f87ca68d700 1 -- 172.21.15.16:0/1335216137 shutdown_connections 2023-05-20T01:17:52.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:52.713+0000 7f87ca68d700 1 -- 172.21.15.16:0/1335216137 wait complete. 2023-05-20T01:17:52.718 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 16 2023-05-20T01:17:53.131 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-05-20T01:17:46.995921+0000', 'modified': '2023-05-20T01:17:50.320558+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5397}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5397': {'gid': 5397, 'name': 'c', 'rank': 0, 'incarnation': 12, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.16:6837/942011026', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6835', 'nonce': 942011026}, {'type': 'v1', 'addr': '172.21.15.16:6837', 'nonce': 942011026}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-05-20T01:17:53.132 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-05-20T01:17:53.132 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-05-20T01:17:53.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.261+0000 7fe187385700 1 Processor -- start 2023-05-20T01:17:53.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.261+0000 7fe187385700 1 -- start start 2023-05-20T01:17:53.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.262+0000 7fe187385700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe1801599c0 0x7fe1800caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:53.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.262+0000 7fe187385700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe1800caaa0 0x7fe1800cb4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:53.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.262+0000 7fe187385700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe180159d90 con 0x7fe18006f3f0 2023-05-20T01:17:53.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.262+0000 7fe187385700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1800cdf20 con 0x7fe1800caaa0 2023-05-20T01:17:53.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.262+0000 7fe187385700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1800cdb80 con 0x7fe1801599c0 2023-05-20T01:17:53.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.262+0000 7fe184920700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe1800caaa0 0x7fe1800cb4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:53.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.262+0000 7fe184920700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe1800caaa0 0x7fe1800cb4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60324/0 (socket says 172.21.15.16:60324) 2023-05-20T01:17:53.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.262+0000 7fe184920700 1 -- 172.21.15.16:0/4154873764 learned_addr learned my addr 172.21.15.16:0/4154873764 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:53.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.262+0000 7fe185121700 1 --2- 172.21.15.16:0/4154873764 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe1801599c0 0x7fe1800caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:53.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.262+0000 7fe184920700 1 -- 172.21.15.16:0/4154873764 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe1801599c0 msgr2=0x7fe1800caf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:53.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.262+0000 7fe184920700 1 --2- 172.21.15.16:0/4154873764 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe1801599c0 0x7fe1800caf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:53.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.262+0000 7fe184920700 1 -- 172.21.15.16:0/4154873764 >> v1:172.21.15.132:6789/0 conn(0x7fe18006f3f0 legacy=0x7fe1800710d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:53.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.262+0000 7fe184920700 1 -- 172.21.15.16:0/4154873764 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe18006eaf0 con 0x7fe1800caaa0 2023-05-20T01:17:53.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.262+0000 7fe184920700 1 --2- 172.21.15.16:0/4154873764 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1800caaa0 0x7fe1800cb4a0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fe1700098f0 tx=0x7fe170009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=decf9a88a41a3ba4 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:53.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.263+0000 7fe177fff700 1 -- 172.21.15.16:0/4154873764 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe17000b8a0 con 0x7fe1800caaa0 2023-05-20T01:17:53.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.263+0000 7fe177fff700 1 -- 172.21.15.16:0/4154873764 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe17000ba00 con 0x7fe1800caaa0 2023-05-20T01:17:53.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.263+0000 7fe177fff700 1 -- 172.21.15.16:0/4154873764 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe17000bd90 con 0x7fe1800caaa0 2023-05-20T01:17:53.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.263+0000 7fe187385700 1 -- 172.21.15.16:0/4154873764 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1800caaa0 msgr2=0x7fe1800cb4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:53.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.263+0000 7fe187385700 1 --2- 172.21.15.16:0/4154873764 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1800caaa0 0x7fe1800cb4a0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fe1700098f0 tx=0x7fe170009c00 comp rx=0 tx=0).stop 2023-05-20T01:17:53.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.263+0000 7fe187385700 1 -- 172.21.15.16:0/4154873764 shutdown_connections 2023-05-20T01:17:53.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.263+0000 7fe187385700 1 --2- 172.21.15.16:0/4154873764 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe1801599c0 0x7fe1800caf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:53.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.263+0000 7fe187385700 1 --2- 172.21.15.16:0/4154873764 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1800caaa0 0x7fe1800cb4a0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:53.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.263+0000 7fe187385700 1 -- 172.21.15.16:0/4154873764 >> 172.21.15.16:0/4154873764 conn(0x7fe1800d0c30 msgr2=0x7fe1800b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:53.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.263+0000 7fe187385700 1 -- 172.21.15.16:0/4154873764 shutdown_connections 2023-05-20T01:17:53.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.263+0000 7fe187385700 1 -- 172.21.15.16:0/4154873764 wait complete. 2023-05-20T01:17:53.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.264+0000 7fe187385700 1 Processor -- start 2023-05-20T01:17:53.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.264+0000 7fe187385700 1 -- start start 2023-05-20T01:17:53.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.265+0000 7fe187385700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe18006f3f0 0x7fe1800c6010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:53.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.265+0000 7fe187385700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1800caaa0 0x7fe180155050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:53.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.265+0000 7fe187385700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe1801599c0 0x7fe1800accc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:53.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.265+0000 7fe187385700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe18015bdc0 con 0x7fe1800caaa0 2023-05-20T01:17:53.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.265+0000 7fe187385700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1800c45e0 con 0x7fe1801599c0 2023-05-20T01:17:53.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.265+0000 7fe187385700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1800c48c0 con 0x7fe18006f3f0 2023-05-20T01:17:53.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.265+0000 7fe184920700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1800caaa0 0x7fe180155050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:53.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.265+0000 7fe184920700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1800caaa0 0x7fe180155050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60328/0 (socket says 172.21.15.16:60328) 2023-05-20T01:17:53.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.265+0000 7fe184920700 1 -- 172.21.15.16:0/1798889490 learned_addr learned my addr 172.21.15.16:0/1798889490 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:53.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.265+0000 7fe185121700 1 --2- 172.21.15.16:0/1798889490 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe18006f3f0 0x7fe1800c6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:53.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.265+0000 7fe185922700 1 --2- 172.21.15.16:0/1798889490 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe1801599c0 0x7fe1800accc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:53.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.266+0000 7fe184920700 1 -- 172.21.15.16:0/1798889490 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe18006f3f0 msgr2=0x7fe1800c6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:53.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.266+0000 7fe184920700 1 --2- 172.21.15.16:0/1798889490 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe18006f3f0 0x7fe1800c6010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:53.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.266+0000 7fe184920700 1 -- 172.21.15.16:0/1798889490 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe1801599c0 msgr2=0x7fe1800accc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:53.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.266+0000 7fe184920700 1 --2- 172.21.15.16:0/1798889490 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe1801599c0 0x7fe1800accc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:53.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.266+0000 7fe184920700 1 -- 172.21.15.16:0/1798889490 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe170009550 con 0x7fe1800caaa0 2023-05-20T01:17:53.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.266+0000 7fe185922700 1 --2- 172.21.15.16:0/1798889490 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe1801599c0 0x7fe1800accc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:17:53.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.266+0000 7fe184920700 1 --2- 172.21.15.16:0/1798889490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1800caaa0 0x7fe180155050 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fe170009c40 tx=0x7fe170004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:53.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.266+0000 7fe175ffb700 1 -- 172.21.15.16:0/1798889490 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1700045d0 con 0x7fe1800caaa0 2023-05-20T01:17:53.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.266+0000 7fe175ffb700 1 -- 172.21.15.16:0/1798889490 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe17001f070 con 0x7fe1800caaa0 2023-05-20T01:17:53.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.266+0000 7fe175ffb700 1 -- 172.21.15.16:0/1798889490 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1700038e0 con 0x7fe1800caaa0 2023-05-20T01:17:53.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.266+0000 7fe187385700 1 -- 172.21.15.16:0/1798889490 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe18006f0b0 con 0x7fe1800caaa0 2023-05-20T01:17:53.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.266+0000 7fe187385700 1 -- 172.21.15.16:0/1798889490 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe18006edd0 con 0x7fe1800caaa0 2023-05-20T01:17:53.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.268+0000 7fe175ffb700 1 -- 172.21.15.16:0/1798889490 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe170003a40 con 0x7fe1800caaa0 2023-05-20T01:17:53.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.268+0000 7fe175ffb700 1 --2- 172.21.15.16:0/1798889490 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe16406ace0 0x7fe16406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:53.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.268+0000 7fe175ffb700 1 -- 172.21.15.16:0/1798889490 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fe170021070 con 0x7fe1800caaa0 2023-05-20T01:17:53.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.268+0000 7fe185121700 1 --2- 172.21.15.16:0/1798889490 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe16406ace0 0x7fe16406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:53.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.269+0000 7fe185121700 1 --2- 172.21.15.16:0/1798889490 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe16406ace0 0x7fe16406d190 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe17c006b20 tx=0x7fe17c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:53.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.270+0000 7fe193511b80 1 Processor -- start 2023-05-20T01:17:53.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.270+0000 7fe193511b80 1 -- start start 2023-05-20T01:17:53.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.271+0000 7fe193511b80 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55fd13e5a400 0x55fd13e5a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:53.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.271+0000 7fe193511b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fd13e56700 0x55fd13e2f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:53.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.271+0000 7fe193511b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fd13e2d870 0x55fd13e2f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:53.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.271+0000 7fe193511b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd13d35010 con 0x55fd13e2d870 2023-05-20T01:17:53.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.271+0000 7fe193511b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd13d87850 con 0x55fd13e56700 2023-05-20T01:17:53.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.271+0000 7fe193511b80 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fd13d80ce0 con 0x55fd13e5a400 2023-05-20T01:17:53.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.271+0000 7fe185922700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fd13e2d870 0x55fd13e2f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:53.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.271+0000 7fe185922700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fd13e2d870 0x55fd13e2f9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60338/0 (socket says 172.21.15.16:60338) 2023-05-20T01:17:53.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.271+0000 7fe185922700 1 -- 172.21.15.16:0/3732579374 learned_addr learned my addr 172.21.15.16:0/3732579374 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:53.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.271+0000 7fe185121700 1 --2- 172.21.15.16:0/3732579374 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55fd13e5a400 0x55fd13e5a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:53.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.271+0000 7fe184920700 1 --2- 172.21.15.16:0/3732579374 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fd13e56700 0x55fd13e2f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:53.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.271+0000 7fe185922700 1 -- 172.21.15.16:0/3732579374 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55fd13e5a400 msgr2=0x55fd13e5a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:53.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.271+0000 7fe185922700 1 --2- 172.21.15.16:0/3732579374 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55fd13e5a400 0x55fd13e5a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:53.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.271+0000 7fe185922700 1 -- 172.21.15.16:0/3732579374 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fd13e56700 msgr2=0x55fd13e2f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:53.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.271+0000 7fe185922700 1 --2- 172.21.15.16:0/3732579374 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fd13e56700 0x55fd13e2f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:53.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.271+0000 7fe185922700 1 -- 172.21.15.16:0/3732579374 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fd13c025e0 con 0x55fd13e2d870 2023-05-20T01:17:53.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.272+0000 7fe185922700 1 --2- 172.21.15.16:0/3732579374 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fd13e2d870 0x55fd13e2f9a0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fe1780021a0 tx=0x7fe17800e440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17602edcf9ca44d server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:53.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.272+0000 7fe162ffd700 1 -- 172.21.15.16:0/3732579374 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe178003c10 con 0x55fd13e2d870 2023-05-20T01:17:53.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.272+0000 7fe162ffd700 1 -- 172.21.15.16:0/3732579374 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe178003d70 con 0x55fd13e2d870 2023-05-20T01:17:53.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.272+0000 7fe162ffd700 1 -- 172.21.15.16:0/3732579374 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe17800eb60 con 0x55fd13e2d870 2023-05-20T01:17:53.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.272+0000 7fe193511b80 1 -- 172.21.15.16:0/3732579374 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fd13e2d870 msgr2=0x55fd13e2f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:53.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.272+0000 7fe193511b80 1 --2- 172.21.15.16:0/3732579374 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fd13e2d870 0x55fd13e2f9a0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fe1780021a0 tx=0x7fe17800e440 comp rx=0 tx=0).stop 2023-05-20T01:17:53.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.272+0000 7fe193511b80 1 -- 172.21.15.16:0/3732579374 shutdown_connections 2023-05-20T01:17:53.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.272+0000 7fe193511b80 1 --2- 172.21.15.16:0/3732579374 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fd13e2d870 0x55fd13e2f9a0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:53.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.272+0000 7fe193511b80 1 --2- 172.21.15.16:0/3732579374 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55fd13e5a400 0x55fd13e5a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:53.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.272+0000 7fe193511b80 1 --2- 172.21.15.16:0/3732579374 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fd13e56700 0x55fd13e2f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:53.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.272+0000 7fe193511b80 1 -- 172.21.15.16:0/3732579374 >> 172.21.15.16:0/3732579374 conn(0x55fd13e24f30 msgr2=0x55fd13e75210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:53.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.273+0000 7fe193511b80 1 -- 172.21.15.16:0/3732579374 shutdown_connections 2023-05-20T01:17:53.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.273+0000 7fe193511b80 1 -- 172.21.15.16:0/3732579374 wait complete. 2023-05-20T01:17:53.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.273+0000 7fe193511b80 1 -- start start 2023-05-20T01:17:53.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.273+0000 7fe193511b80 1 Processor -- start 2023-05-20T01:17:53.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.274+0000 7fe193511b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fd13e2d870 0x55fd13e56e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:53.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.274+0000 7fe193511b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fd13e56700 0x55fd13e573a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:53.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.274+0000 7fe193511b80 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55fd13e5a400 0x55fd13e4f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:53.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.274+0000 7fe193511b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd13d22890 con 0x55fd13e56700 2023-05-20T01:17:53.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.274+0000 7fe193511b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd13d22530 con 0x55fd13e2d870 2023-05-20T01:17:53.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.274+0000 7fe193511b80 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fd13c048d0 con 0x55fd13e5a400 2023-05-20T01:17:53.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.274+0000 7fe193511b80 10 client.0 _ll_register_callbacks cb 0x55fd13d40730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-05-20T01:17:53.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.274+0000 7fe184920700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fd13e56700 0x55fd13e573a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:53.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.274+0000 7fe184920700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fd13e56700 0x55fd13e573a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60342/0 (socket says 172.21.15.16:60342) 2023-05-20T01:17:53.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.274+0000 7fe184920700 1 -- 172.21.15.16:0/2621150200 learned_addr learned my addr 172.21.15.16:0/2621150200 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:53.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.274+0000 7fe185922700 1 --2- 172.21.15.16:0/2621150200 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55fd13e5a400 0x55fd13e4f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:53.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.274+0000 7fe185121700 1 --2- 172.21.15.16:0/2621150200 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fd13e2d870 0x55fd13e56e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:53.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.275+0000 7fe184920700 1 -- 172.21.15.16:0/2621150200 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55fd13e5a400 msgr2=0x55fd13e4f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:53.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.275+0000 7fe184920700 1 --2- 172.21.15.16:0/2621150200 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55fd13e5a400 0x55fd13e4f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:53.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.275+0000 7fe184920700 1 -- 172.21.15.16:0/2621150200 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fd13e2d870 msgr2=0x55fd13e56e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:53.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.275+0000 7fe184920700 1 --2- 172.21.15.16:0/2621150200 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fd13e2d870 0x55fd13e56e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:53.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.275+0000 7fe184920700 1 -- 172.21.15.16:0/2621150200 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe17800f040 con 0x55fd13e56700 2023-05-20T01:17:53.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.275+0000 7fe184920700 1 --2- 172.21.15.16:0/2621150200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fd13e56700 0x55fd13e573a0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fe170000c00 tx=0x7fe17005da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8019e617483c9697 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:53.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.275+0000 7fe15bfff700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-05-20T01:17:53.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.275+0000 7fe15bfff700 1 -- 172.21.15.16:0/2621150200 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe170058f70 con 0x55fd13e56700 2023-05-20T01:17:53.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.275+0000 7fe15bfff700 1 -- 172.21.15.16:0/2621150200 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(40..40 src has 1..40) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fe170088d50 con 0x55fd13e56700 2023-05-20T01:17:53.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.275+0000 7fe193511b80 1 -- 172.21.15.16:0/2621150200 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fd13e226f0 con 0x55fd13e56700 2023-05-20T01:17:53.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.275+0000 7fe15bfff700 1 -- 172.21.15.16:0/2621150200 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe1700590d0 con 0x55fd13e56700 2023-05-20T01:17:53.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.275+0000 7fe193511b80 10 client.5583 fetch_fsmap learned FSMap version 16 2023-05-20T01:17:53.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.275+0000 7fe193511b80 1 -- 172.21.15.16:0/2621150200 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x55fd13d47140 con 0x55fd13e56700 2023-05-20T01:17:53.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.275+0000 7fe15bfff700 1 -- 172.21.15.16:0/2621150200 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55fd13e226f0 con 0x55fd13e56700 2023-05-20T01:17:53.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.275+0000 7fe15bfff700 1 -- 172.21.15.16:0/2621150200 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 16) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fe1700782f0 con 0x55fd13e56700 2023-05-20T01:17:53.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.275+0000 7fe193511b80 10 client.5583 fetch_fsmap finished waiting for FSMap version 16 2023-05-20T01:17:53.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.275+0000 7fe193511b80 10 client.5583 resolve_mds: resolved name 'c' to daemon mds.c 2023-05-20T01:17:53.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.275+0000 7fe193511b80 20 client.5583 populate_metadata read hostname 'smithi016' 2023-05-20T01:17:53.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.275+0000 7fe193511b80 1 --2- 172.21.15.16:0/2621150200 >> [v2:172.21.15.16:6835/942011026,v1:172.21.15.16:6837/942011026] conn(0x55fd13e74400 0x55fd13e747d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:53.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.275+0000 7fe193511b80 4 client.5583 mds_command: new command op to 5397 tid=0[{"prefix": "get_command_descriptions"}] 2023-05-20T01:17:53.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.275+0000 7fe193511b80 1 -- 172.21.15.16:0/2621150200 --> [v2:172.21.15.16:6835/942011026,v1:172.21.15.16:6837/942011026] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fd13d43050 con 0x55fd13e74400 2023-05-20T01:17:53.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.275+0000 7fe185121700 1 --2- 172.21.15.16:0/2621150200 >> [v2:172.21.15.16:6835/942011026,v1:172.21.15.16:6837/942011026] conn(0x55fd13e74400 0x55fd13e747d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:53.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.276+0000 7fe15bfff700 10 client.5583 ms_handle_connect on v2:172.21.15.16:6835/942011026 2023-05-20T01:17:53.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.276+0000 7fe185121700 1 --2- 172.21.15.16:0/2621150200 >> [v2:172.21.15.16:6835/942011026,v1:172.21.15.16:6837/942011026] conn(0x55fd13e74400 0x55fd13e747d0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc1c5dc36431768 server_cookie=e5f9004be524765a in_seq=0 out_seq=0 2023-05-20T01:17:53.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.276+0000 7fe15bfff700 1 -- 172.21.15.16:0/2621150200 <== mds.0 v2:172.21.15.16:6835/942011026 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55fd13d43050 con 0x55fd13e74400 2023-05-20T01:17:53.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.276+0000 7fe15bfff700 10 client.5583 handle_command_reply: tid=0 2023-05-20T01:17:53.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.276+0000 7fe193511b80 1 client.5583 shutdown 2023-05-20T01:17:53.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.276+0000 7fe193511b80 1 -- 172.21.15.16:0/2621150200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fd13e56700 msgr2=0x55fd13e573a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:53.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.276+0000 7fe193511b80 1 --2- 172.21.15.16:0/2621150200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fd13e56700 0x55fd13e573a0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fe170000c00 tx=0x7fe17005da90 comp rx=0 tx=0).stop 2023-05-20T01:17:53.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.277+0000 7fe193511b80 1 -- 172.21.15.16:0/2621150200 shutdown_connections 2023-05-20T01:17:53.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.277+0000 7fe193511b80 1 --2- 172.21.15.16:0/2621150200 >> [v2:172.21.15.16:6835/942011026,v1:172.21.15.16:6837/942011026] conn(0x55fd13e74400 0x55fd13e747d0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:53.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.277+0000 7fe193511b80 1 --2- 172.21.15.16:0/2621150200 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55fd13e5a400 0x55fd13e4f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:53.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.277+0000 7fe15bfff700 0 client.5583 ms_handle_reset on v2:172.21.15.16:6835/942011026 2023-05-20T01:17:53.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.277+0000 7fe193511b80 1 --2- 172.21.15.16:0/2621150200 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fd13e2d870 0x55fd13e56e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:53.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.277+0000 7fe193511b80 1 --2- 172.21.15.16:0/2621150200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fd13e56700 0x55fd13e573a0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:53.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.277+0000 7fe193511b80 1 -- 172.21.15.16:0/2621150200 >> 172.21.15.16:0/2621150200 conn(0x55fd13e24f30 msgr2=0x55fd13e2b710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:53.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.277+0000 7fe193511b80 1 -- 172.21.15.16:0/2621150200 shutdown_connections 2023-05-20T01:17:53.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.277+0000 7fe193511b80 1 -- 172.21.15.16:0/2621150200 wait complete. 2023-05-20T01:17:53.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.277+0000 7fe193511b80 20 client.5583 trim_cache size 0 max 16384 2023-05-20T01:17:53.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.295+0000 7fe193511b80 1 Processor -- start 2023-05-20T01:17:53.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.295+0000 7fe193511b80 1 -- start start 2023-05-20T01:17:53.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.296+0000 7fe193511b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fd13e2d870 0x55fd13e772a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:53.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.296+0000 7fe193511b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fd13e56700 0x55fd13e777e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:53.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.296+0000 7fe193511b80 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55fd13e5a400 0x55fd13e2fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:53.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.296+0000 7fe193511b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd13e79020 con 0x55fd13e56700 2023-05-20T01:17:53.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.296+0000 7fe193511b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd13e2b6d0 con 0x55fd13e2d870 2023-05-20T01:17:53.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.296+0000 7fe193511b80 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fd13e5e470 con 0x55fd13e5a400 2023-05-20T01:17:53.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.296+0000 7fe184920700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fd13e56700 0x55fd13e777e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:53.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.296+0000 7fe184920700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fd13e56700 0x55fd13e777e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60350/0 (socket says 172.21.15.16:60350) 2023-05-20T01:17:53.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.296+0000 7fe184920700 1 -- 172.21.15.16:0/221917807 learned_addr learned my addr 172.21.15.16:0/221917807 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:53.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.296+0000 7fe185121700 1 --2- 172.21.15.16:0/221917807 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fd13e2d870 0x55fd13e772a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:53.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.296+0000 7fe185922700 1 --2- 172.21.15.16:0/221917807 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55fd13e5a400 0x55fd13e2fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:53.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.296+0000 7fe184920700 1 -- 172.21.15.16:0/221917807 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55fd13e5a400 msgr2=0x55fd13e2fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:53.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.296+0000 7fe184920700 1 --2- 172.21.15.16:0/221917807 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55fd13e5a400 0x55fd13e2fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:53.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.296+0000 7fe184920700 1 -- 172.21.15.16:0/221917807 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fd13e2d870 msgr2=0x55fd13e772a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:53.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.296+0000 7fe184920700 1 --2- 172.21.15.16:0/221917807 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fd13e2d870 0x55fd13e772a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:53.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.296+0000 7fe184920700 1 -- 172.21.15.16:0/221917807 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fd13d46e90 con 0x55fd13e56700 2023-05-20T01:17:53.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.296+0000 7fe184920700 1 --2- 172.21.15.16:0/221917807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fd13e56700 0x55fd13e777e0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fe17008aee0 tx=0x7fe17005c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8d9dc2e8aa8cf74 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:53.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.296+0000 7fe1637fe700 1 -- 172.21.15.16:0/221917807 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe17005da90 con 0x55fd13e56700 2023-05-20T01:17:53.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.296+0000 7fe1637fe700 1 -- 172.21.15.16:0/221917807 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1700782f0 con 0x55fd13e56700 2023-05-20T01:17:53.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.296+0000 7fe1637fe700 1 -- 172.21.15.16:0/221917807 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe170078460 con 0x55fd13e56700 2023-05-20T01:17:53.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.297+0000 7fe193511b80 1 -- 172.21.15.16:0/221917807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fd13e56700 msgr2=0x55fd13e777e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:53.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.297+0000 7fe193511b80 1 --2- 172.21.15.16:0/221917807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fd13e56700 0x55fd13e777e0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fe17008aee0 tx=0x7fe17005c000 comp rx=0 tx=0).stop 2023-05-20T01:17:53.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.297+0000 7fe193511b80 1 -- 172.21.15.16:0/221917807 shutdown_connections 2023-05-20T01:17:53.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.297+0000 7fe193511b80 1 --2- 172.21.15.16:0/221917807 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55fd13e5a400 0x55fd13e2fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:53.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.297+0000 7fe193511b80 1 --2- 172.21.15.16:0/221917807 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fd13e2d870 0x55fd13e772a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:53.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.297+0000 7fe193511b80 1 --2- 172.21.15.16:0/221917807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fd13e56700 0x55fd13e777e0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:53.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.297+0000 7fe193511b80 1 -- 172.21.15.16:0/221917807 >> 172.21.15.16:0/221917807 conn(0x55fd13e24f30 msgr2=0x55fd13e49fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:53.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.297+0000 7fe193511b80 1 -- 172.21.15.16:0/221917807 shutdown_connections 2023-05-20T01:17:53.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.297+0000 7fe193511b80 1 -- 172.21.15.16:0/221917807 wait complete. 2023-05-20T01:17:53.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.297+0000 7fe193511b80 1 -- start start 2023-05-20T01:17:53.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.297+0000 7fe193511b80 1 Processor -- start 2023-05-20T01:17:53.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.298+0000 7fe193511b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fd13e2d870 0x55fd13ede860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:53.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.298+0000 7fe193511b80 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55fd13e56700 0x55fd13e302d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:53.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.298+0000 7fe193511b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fd13e5a400 0x55fd13e50b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:53.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.298+0000 7fe193511b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd13ededa0 con 0x55fd13e2d870 2023-05-20T01:17:53.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.298+0000 7fe193511b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd13e51c90 con 0x55fd13e5a400 2023-05-20T01:17:53.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.298+0000 7fe193511b80 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fd13e29220 con 0x55fd13e56700 2023-05-20T01:17:53.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.298+0000 7fe193511b80 10 client.0 _ll_register_callbacks cb 0x55fd13e538e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-05-20T01:17:53.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.298+0000 7fe185121700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fd13e2d870 0x55fd13ede860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:53.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.298+0000 7fe185121700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fd13e2d870 0x55fd13ede860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60354/0 (socket says 172.21.15.16:60354) 2023-05-20T01:17:53.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.298+0000 7fe185121700 1 -- 172.21.15.16:0/3227661439 learned_addr learned my addr 172.21.15.16:0/3227661439 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:53.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.299+0000 7fe184920700 1 --2- 172.21.15.16:0/3227661439 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55fd13e56700 0x55fd13e302d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:53.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.299+0000 7fe185922700 1 --2- 172.21.15.16:0/3227661439 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fd13e5a400 0x55fd13e50b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:53.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.299+0000 7fe185121700 1 -- 172.21.15.16:0/3227661439 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55fd13e56700 msgr2=0x55fd13e302d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:53.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.299+0000 7fe185121700 1 --2- 172.21.15.16:0/3227661439 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55fd13e56700 0x55fd13e302d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:53.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.299+0000 7fe185121700 1 -- 172.21.15.16:0/3227661439 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fd13e5a400 msgr2=0x55fd13e50b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:53.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.299+0000 7fe185121700 1 --2- 172.21.15.16:0/3227661439 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fd13e5a400 0x55fd13e50b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:53.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.299+0000 7fe185121700 1 -- 172.21.15.16:0/3227661439 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fd13d47140 con 0x55fd13e2d870 2023-05-20T01:17:53.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.299+0000 7fe185922700 1 --2- 172.21.15.16:0/3227661439 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fd13e5a400 0x55fd13e50b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:17:53.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.299+0000 7fe185121700 1 --2- 172.21.15.16:0/3227661439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fd13e2d870 0x55fd13ede860 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fe17c00a000 tx=0x7fe17c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e27f41b8977f937 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:53.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.299+0000 7fe15bfff700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-05-20T01:17:53.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.299+0000 7fe15bfff700 1 -- 172.21.15.16:0/3227661439 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe17c028040 con 0x55fd13e2d870 2023-05-20T01:17:53.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.299+0000 7fe15bfff700 1 -- 172.21.15.16:0/3227661439 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(40..40 src has 1..40) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fe17c01b040 con 0x55fd13e2d870 2023-05-20T01:17:53.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.299+0000 7fe193511b80 1 -- 172.21.15.16:0/3227661439 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe178003d70 con 0x55fd13e2d870 2023-05-20T01:17:53.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.299+0000 7fe15bfff700 1 -- 172.21.15.16:0/3227661439 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe17c02ac70 con 0x55fd13e2d870 2023-05-20T01:17:53.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.299+0000 7fe193511b80 10 client.5589 fetch_fsmap learned FSMap version 16 2023-05-20T01:17:53.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.299+0000 7fe193511b80 1 -- 172.21.15.16:0/3227661439 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x55fd13b97c10 con 0x55fd13e2d870 2023-05-20T01:17:53.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.299+0000 7fe15bfff700 1 -- 172.21.15.16:0/3227661439 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe178003d70 con 0x55fd13e2d870 2023-05-20T01:17:53.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.299+0000 7fe15bfff700 1 -- 172.21.15.16:0/3227661439 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 16) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fe17c033450 con 0x55fd13e2d870 2023-05-20T01:17:53.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.299+0000 7fe193511b80 10 client.5589 fetch_fsmap finished waiting for FSMap version 16 2023-05-20T01:17:53.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.299+0000 7fe193511b80 10 client.5589 resolve_mds: resolved name 'c' to daemon mds.c 2023-05-20T01:17:53.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.299+0000 7fe193511b80 20 client.5589 populate_metadata read hostname 'smithi016' 2023-05-20T01:17:53.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.300+0000 7fe193511b80 1 --2- 172.21.15.16:0/3227661439 >> [v2:172.21.15.16:6835/942011026,v1:172.21.15.16:6837/942011026] conn(0x55fd13e772a0 0x55fd13e77670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:53.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.300+0000 7fe193511b80 4 client.5589 mds_command: new command op to 5397 tid=0[{"prefix": "status"}] 2023-05-20T01:17:53.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.300+0000 7fe193511b80 1 -- 172.21.15.16:0/3227661439 --> [v2:172.21.15.16:6835/942011026,v1:172.21.15.16:6837/942011026] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55fd13d4f610 con 0x55fd13e772a0 2023-05-20T01:17:53.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.300+0000 7fe184920700 1 --2- 172.21.15.16:0/3227661439 >> [v2:172.21.15.16:6835/942011026,v1:172.21.15.16:6837/942011026] conn(0x55fd13e772a0 0x55fd13e77670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:53.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.300+0000 7fe15bfff700 10 client.5589 ms_handle_connect on v2:172.21.15.16:6835/942011026 2023-05-20T01:17:53.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.300+0000 7fe184920700 1 --2- 172.21.15.16:0/3227661439 >> [v2:172.21.15.16:6835/942011026,v1:172.21.15.16:6837/942011026] conn(0x55fd13e772a0 0x55fd13e77670 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80518ffa5442db6e server_cookie=83f3716db76c250b in_seq=0 out_seq=0 2023-05-20T01:17:53.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.300+0000 7fe15bfff700 1 -- 172.21.15.16:0/3227661439 <== mds.0 v2:172.21.15.16:6835/942011026 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55fd13d4f610 con 0x55fd13e772a0 2023-05-20T01:17:53.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.300+0000 7fe15bfff700 10 client.5589 handle_command_reply: tid=0 2023-05-20T01:17:53.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.300+0000 7fe193511b80 1 client.5589 shutdown 2023-05-20T01:17:53.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.300+0000 7fe193511b80 1 -- 172.21.15.16:0/3227661439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fd13e2d870 msgr2=0x55fd13ede860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:53.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.300+0000 7fe193511b80 1 --2- 172.21.15.16:0/3227661439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fd13e2d870 0x55fd13ede860 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fe17c00a000 tx=0x7fe17c0047c0 comp rx=0 tx=0).stop 2023-05-20T01:17:53.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.300+0000 7fe193511b80 1 -- 172.21.15.16:0/3227661439 shutdown_connections 2023-05-20T01:17:53.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.300+0000 7fe193511b80 1 --2- 172.21.15.16:0/3227661439 >> [v2:172.21.15.16:6835/942011026,v1:172.21.15.16:6837/942011026] conn(0x55fd13e772a0 0x55fd13e77670 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:53.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.300+0000 7fe193511b80 1 --2- 172.21.15.16:0/3227661439 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fd13e5a400 0x55fd13e50b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:53.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.300+0000 7fe193511b80 1 --2- 172.21.15.16:0/3227661439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fd13e2d870 0x55fd13ede860 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:53.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.300+0000 7fe15bfff700 0 client.5589 ms_handle_reset on v2:172.21.15.16:6835/942011026 2023-05-20T01:17:53.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.300+0000 7fe193511b80 1 --2- 172.21.15.16:0/3227661439 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55fd13e56700 0x55fd13e302d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:53.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.300+0000 7fe193511b80 1 -- 172.21.15.16:0/3227661439 >> 172.21.15.16:0/3227661439 conn(0x55fd13e24f30 msgr2=0x55fd13e786d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:53.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.301+0000 7fe193511b80 1 -- 172.21.15.16:0/3227661439 shutdown_connections 2023-05-20T01:17:53.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.301+0000 7fe193511b80 1 -- 172.21.15.16:0/3227661439 wait complete. 2023-05-20T01:17:53.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.301+0000 7fe193511b80 20 client.5589 trim_cache size 0 max 16384 2023-05-20T01:17:53.318 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-05-20T01:17:53.318 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "c2245cc5-ecab-4087-a59b-bc81db901676", 2023-05-20T01:17:53.318 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2023-05-20T01:17:53.318 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 5397, 2023-05-20T01:17:53.318 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2023-05-20T01:17:53.319 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2023-05-20T01:17:53.319 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2023-05-20T01:17:53.319 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 6.3039998559999999, 2023-05-20T01:17:53.319 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 16, 2023-05-20T01:17:53.319 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 40, 2023-05-20T01:17:53.319 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 40, 2023-05-20T01:17:53.319 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 14.591700087 2023-05-20T01:17:53.319 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-05-20T01:17:53.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.301+0000 7fe187385700 1 -- 172.21.15.16:0/1798889490 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe16406ace0 msgr2=0x7fe16406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:53.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.301+0000 7fe187385700 1 --2- 172.21.15.16:0/1798889490 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe16406ace0 0x7fe16406d190 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe17c006b20 tx=0x7fe17c008040 comp rx=0 tx=0).stop 2023-05-20T01:17:53.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.301+0000 7fe187385700 1 -- 172.21.15.16:0/1798889490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1800caaa0 msgr2=0x7fe180155050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:53.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.301+0000 7fe187385700 1 --2- 172.21.15.16:0/1798889490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1800caaa0 0x7fe180155050 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fe170009c40 tx=0x7fe170004750 comp rx=0 tx=0).stop 2023-05-20T01:17:53.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.301+0000 7fe187385700 1 -- 172.21.15.16:0/1798889490 shutdown_connections 2023-05-20T01:17:53.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.301+0000 7fe187385700 1 --2- 172.21.15.16:0/1798889490 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe1801599c0 0x7fe1800accc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:53.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.301+0000 7fe187385700 1 --2- 172.21.15.16:0/1798889490 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe16406ace0 0x7fe16406d190 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:53.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.301+0000 7fe187385700 1 --2- 172.21.15.16:0/1798889490 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe18006f3f0 0x7fe1800c6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:53.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.301+0000 7fe187385700 1 --2- 172.21.15.16:0/1798889490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1800caaa0 0x7fe180155050 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:53.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.301+0000 7fe187385700 1 -- 172.21.15.16:0/1798889490 >> 172.21.15.16:0/1798889490 conn(0x7fe1800d0c30 msgr2=0x7fe18006a510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:53.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.301+0000 7fe187385700 1 -- 172.21.15.16:0/1798889490 shutdown_connections 2023-05-20T01:17:53.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.301+0000 7fe187385700 1 -- 172.21.15.16:0/1798889490 wait complete. 2023-05-20T01:17:53.322 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-05-20T01:17:53.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.444+0000 7f6efa4f5700 1 Processor -- start 2023-05-20T01:17:53.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.444+0000 7f6efa4f5700 1 -- start start 2023-05-20T01:17:53.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.445+0000 7f6efa4f5700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f6ef40d16c0 0x7f6ef4144330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:53.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.445+0000 7f6efa4f5700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ef40b1dd0 con 0x7f6ef40b1990 2023-05-20T01:17:53.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.445+0000 7f6efa4f5700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ef4144870 con 0x7f6ef40cec20 2023-05-20T01:17:53.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.445+0000 7f6efa4f5700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ef40b2000 con 0x7f6ef40d16c0 2023-05-20T01:17:53.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.445+0000 7f6ef8a92700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f6ef40b1990 0x7f6ef40ae9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:43660/0 (socket says 172.21.15.16:43660) 2023-05-20T01:17:53.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.445+0000 7f6ef8a92700 1 -- 172.21.15.16:0/2071027317 learned_addr learned my addr 172.21.15.16:0/2071027317 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:17:53.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.445+0000 7f6ef2ffd700 1 -- 172.21.15.16:0/2071027317 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3060311157 0 0) 0x7f6ef40b1dd0 con 0x7f6ef40b1990 2023-05-20T01:17:53.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.445+0000 7f6ef2ffd700 1 -- 172.21.15.16:0/2071027317 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6edc0035b0 con 0x7f6ef40b1990 2023-05-20T01:17:53.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.445+0000 7f6ef3fff700 1 --2- 172.21.15.16:0/2071027317 >> v2:172.21.15.132:3300/0 conn(0x7f6ef40d16c0 0x7f6ef4144330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:53.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.445+0000 7f6ef2ffd700 1 -- 172.21.15.16:0/2071027317 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 380801749 0 0) 0x7f6edc0035b0 con 0x7f6ef40b1990 2023-05-20T01:17:53.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.445+0000 7f6ef2ffd700 1 -- 172.21.15.16:0/2071027317 >> v2:172.21.15.132:3300/0 conn(0x7f6ef40d16c0 msgr2=0x7f6ef4144330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:53.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.445+0000 7f6ef2ffd700 1 --2- 172.21.15.16:0/2071027317 >> v2:172.21.15.132:3300/0 conn(0x7f6ef40d16c0 0x7f6ef4144330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:53.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.445+0000 7f6ef2ffd700 1 -- 172.21.15.16:0/2071027317 >> v1:172.21.15.132:6789/0 conn(0x7f6ef40cec20 legacy=0x7f6ef40ae2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:53.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.445+0000 7f6ef2ffd700 1 -- 172.21.15.16:0/2071027317 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ef40b22e0 con 0x7f6ef40b1990 2023-05-20T01:17:53.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.445+0000 7f6ef2ffd700 1 -- 172.21.15.16:0/2071027317 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7f6eec0019f0 con 0x7f6ef40b1990 2023-05-20T01:17:53.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.445+0000 7f6ef2ffd700 1 -- 172.21.15.16:0/2071027317 >> v1:172.21.15.16:6789/0 conn(0x7f6ef40b1990 legacy=0x7f6ef40ae9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:53.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.445+0000 7f6ef2ffd700 1 --2- 172.21.15.16:0/2071027317 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6edc003a70 0x7f6edc003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:53.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.445+0000 7f6ef2ffd700 1 --2- 172.21.15.16:0/2071027317 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6edc004850 0x7f6edc004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:53.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.445+0000 7f6ef2ffd700 1 --2- 172.21.15.16:0/2071027317 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6edc005200 0x7f6edc00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:53.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.445+0000 7f6ef2ffd700 1 -- 172.21.15.16:0/2071027317 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef40b2000 con 0x7f6edc004850 2023-05-20T01:17:53.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.445+0000 7f6ef2ffd700 1 -- 172.21.15.16:0/2071027317 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6edc00c0e0 con 0x7f6edc003a70 2023-05-20T01:17:53.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.445+0000 7f6ef2ffd700 1 -- 172.21.15.16:0/2071027317 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6edc00c2a0 con 0x7f6edc005200 2023-05-20T01:17:53.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.446+0000 7f6ef37fe700 1 --2- 172.21.15.16:0/2071027317 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6edc004850 0x7f6edc004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:53.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.446+0000 7f6ef3fff700 1 --2- 172.21.15.16:0/2071027317 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6edc003a70 0x7f6edc003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:53.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.446+0000 7f6ef8a92700 1 --2- 172.21.15.16:0/2071027317 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6edc005200 0x7f6edc00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:53.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.446+0000 7f6ef37fe700 1 -- 172.21.15.16:0/2071027317 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6edc005200 msgr2=0x7f6edc00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:53.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.446+0000 7f6ef37fe700 1 --2- 172.21.15.16:0/2071027317 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6edc005200 0x7f6edc00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:53.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.446+0000 7f6ef37fe700 1 -- 172.21.15.16:0/2071027317 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6edc003a70 msgr2=0x7f6edc003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:53.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.446+0000 7f6ef37fe700 1 --2- 172.21.15.16:0/2071027317 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6edc003a70 0x7f6edc003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:53.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.446+0000 7f6ef37fe700 1 -- 172.21.15.16:0/2071027317 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6edc0035b0 con 0x7f6edc004850 2023-05-20T01:17:53.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.446+0000 7f6ef8a92700 1 -- 172.21.15.16:0/2071027317 reap_dead start 2023-05-20T01:17:53.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.446+0000 7f6ef3fff700 1 --2- 172.21.15.16:0/2071027317 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6edc003a70 0x7f6edc003e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-05-20T01:17:53.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.446+0000 7f6ef37fe700 1 --2- 172.21.15.16:0/2071027317 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6edc004850 0x7f6edc004cc0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f6ee80017c0 tx=0x7f6edc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da1db51c40475ff6 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:53.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.446+0000 7f6ef2ffd700 1 -- 172.21.15.16:0/2071027317 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee80052d0 con 0x7f6edc004850 2023-05-20T01:17:53.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.446+0000 7f6ef2ffd700 1 -- 172.21.15.16:0/2071027317 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ee8005450 con 0x7f6edc004850 2023-05-20T01:17:53.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.446+0000 7f6ef2ffd700 1 -- 172.21.15.16:0/2071027317 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee80038c0 con 0x7f6edc004850 2023-05-20T01:17:53.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.447+0000 7f6efa4f5700 1 -- 172.21.15.16:0/2071027317 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6edc004850 msgr2=0x7f6edc004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:53.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.447+0000 7f6efa4f5700 1 --2- 172.21.15.16:0/2071027317 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6edc004850 0x7f6edc004cc0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f6ee80017c0 tx=0x7f6edc005880 comp rx=0 tx=0).stop 2023-05-20T01:17:53.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.447+0000 7f6efa4f5700 1 -- 172.21.15.16:0/2071027317 shutdown_connections 2023-05-20T01:17:53.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.447+0000 7f6efa4f5700 1 --2- 172.21.15.16:0/2071027317 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6edc004850 0x7f6edc004cc0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:53.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.447+0000 7f6efa4f5700 1 -- 172.21.15.16:0/2071027317 >> 172.21.15.16:0/2071027317 conn(0x7f6ef406ae30 msgr2=0x7f6ef409a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:53.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.447+0000 7f6efa4f5700 1 -- 172.21.15.16:0/2071027317 shutdown_connections 2023-05-20T01:17:53.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.447+0000 7f6efa4f5700 1 -- 172.21.15.16:0/2071027317 wait complete. 2023-05-20T01:17:53.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.448+0000 7f6efa4f5700 1 Processor -- start 2023-05-20T01:17:53.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.449+0000 7f6efa4f5700 1 -- start start 2023-05-20T01:17:53.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.449+0000 7f6efa4f5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6edc004850 0x7f6ef40ce080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:53.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.449+0000 7f6efa4f5700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ef40c7770 0x7f6ef40b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:53.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.450+0000 7f6efa4f5700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6ef40b3e80 0x7f6ef40b42f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:53.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.450+0000 7f6efa4f5700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef40cba70 con 0x7f6edc004850 2023-05-20T01:17:53.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.450+0000 7f6efa4f5700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef40b4830 con 0x7f6ef40c7770 2023-05-20T01:17:53.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.450+0000 7f6efa4f5700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef40c7be0 con 0x7f6ef40b3e80 2023-05-20T01:17:53.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.450+0000 7f6ef3fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6edc004850 0x7f6ef40ce080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:53.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.450+0000 7f6ef3fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6edc004850 0x7f6ef40ce080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60372/0 (socket says 172.21.15.16:60372) 2023-05-20T01:17:53.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.450+0000 7f6ef3fff700 1 -- 172.21.15.16:0/1342799142 learned_addr learned my addr 172.21.15.16:0/1342799142 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:53.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.450+0000 7f6ef3fff700 1 -- 172.21.15.16:0/1342799142 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6ef40b3e80 msgr2=0x7f6ef40b42f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:53.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.450+0000 7f6ef3fff700 1 --2- 172.21.15.16:0/1342799142 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6ef40b3e80 0x7f6ef40b42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:53.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.450+0000 7f6ef3fff700 1 -- 172.21.15.16:0/1342799142 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ef40c7770 msgr2=0x7f6ef40b5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:53.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.450+0000 7f6ef3fff700 1 --2- 172.21.15.16:0/1342799142 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ef40c7770 0x7f6ef40b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:53.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.450+0000 7f6ef3fff700 1 -- 172.21.15.16:0/1342799142 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ef40adda0 con 0x7f6edc004850 2023-05-20T01:17:53.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.450+0000 7f6ef3fff700 1 --2- 172.21.15.16:0/1342799142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6edc004850 0x7f6ef40ce080 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f6ee4011c30 tx=0x7f6ee4009e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:53.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.450+0000 7f6ef0ff9700 1 -- 172.21.15.16:0/1342799142 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee400f9a0 con 0x7f6edc004850 2023-05-20T01:17:53.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.450+0000 7f6ef0ff9700 1 -- 172.21.15.16:0/1342799142 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ee400fb00 con 0x7f6edc004850 2023-05-20T01:17:53.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.450+0000 7f6efa4f5700 1 -- 172.21.15.16:0/1342799142 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ef40b25c0 con 0x7f6edc004850 2023-05-20T01:17:53.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.450+0000 7f6ef0ff9700 1 -- 172.21.15.16:0/1342799142 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee400feb0 con 0x7f6edc004850 2023-05-20T01:17:53.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.450+0000 7f6efa4f5700 1 -- 172.21.15.16:0/1342799142 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ef40b28a0 con 0x7f6edc004850 2023-05-20T01:17:53.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.451+0000 7f6efa4f5700 1 -- 172.21.15.16:0/1342799142 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ef41edf10 con 0x7f6edc004850 2023-05-20T01:17:53.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.452+0000 7f6ef0ff9700 1 -- 172.21.15.16:0/1342799142 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6ee4004020 con 0x7f6edc004850 2023-05-20T01:17:53.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.452+0000 7f6ef0ff9700 1 --2- 172.21.15.16:0/1342799142 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f6ee006ace0 0x7f6ee006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:53.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.452+0000 7f6ef0ff9700 1 -- 172.21.15.16:0/1342799142 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f6ee4089cd0 con 0x7f6edc004850 2023-05-20T01:17:53.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.452+0000 7f6ef37fe700 1 --2- 172.21.15.16:0/1342799142 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f6ee006ace0 0x7f6ee006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:53.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.453+0000 7f6ef37fe700 1 --2- 172.21.15.16:0/1342799142 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f6ee006ace0 0x7f6ee006d190 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f6ef415dc30 tx=0x7f6ee8011000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:53.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.457+0000 7f6ef0ff9700 1 -- 172.21.15.16:0/1342799142 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f6ee40586d0 con 0x7f6edc004850 2023-05-20T01:17:53.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.699+0000 7f6efa4f5700 1 -- 172.21.15.16:0/1342799142 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f6ef40b5ff0 con 0x7f6edc004850 2023-05-20T01:17:53.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.700+0000 7f6ef0ff9700 1 -- 172.21.15.16:0/1342799142 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) v1 ==== 53+0+73 (secure 0 0 0) 0x7f6ee4022070 con 0x7f6edc004850 2023-05-20T01:17:53.703 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-05-20T01:17:53.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.701+0000 7f6efa4f5700 1 -- 172.21.15.16:0/1342799142 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f6ee006ace0 msgr2=0x7f6ee006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:53.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.701+0000 7f6efa4f5700 1 --2- 172.21.15.16:0/1342799142 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f6ee006ace0 0x7f6ee006d190 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f6ef415dc30 tx=0x7f6ee8011000 comp rx=0 tx=0).stop 2023-05-20T01:17:53.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.701+0000 7f6efa4f5700 1 -- 172.21.15.16:0/1342799142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6edc004850 msgr2=0x7f6ef40ce080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:53.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.701+0000 7f6efa4f5700 1 --2- 172.21.15.16:0/1342799142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6edc004850 0x7f6ef40ce080 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f6ee4011c30 tx=0x7f6ee4009e50 comp rx=0 tx=0).stop 2023-05-20T01:17:53.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.701+0000 7f6efa4f5700 1 -- 172.21.15.16:0/1342799142 shutdown_connections 2023-05-20T01:17:53.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.701+0000 7f6efa4f5700 1 --2- 172.21.15.16:0/1342799142 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f6ee006ace0 0x7f6ee006d190 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:53.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.701+0000 7f6efa4f5700 1 --2- 172.21.15.16:0/1342799142 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6ef40b3e80 0x7f6ef40b42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:53.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.701+0000 7f6efa4f5700 1 --2- 172.21.15.16:0/1342799142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6edc004850 0x7f6ef40ce080 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:53.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.701+0000 7f6efa4f5700 1 --2- 172.21.15.16:0/1342799142 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ef40c7770 0x7f6ef40b5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:53.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.701+0000 7f6efa4f5700 1 -- 172.21.15.16:0/1342799142 >> 172.21.15.16:0/1342799142 conn(0x7f6ef406ae30 msgr2=0x7f6ef40b01a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:53.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.702+0000 7f6efa4f5700 1 -- 172.21.15.16:0/1342799142 shutdown_connections 2023-05-20T01:17:53.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:53.702+0000 7f6efa4f5700 1 -- 172.21.15.16:0/1342799142 wait complete. 2023-05-20T01:17:53.717 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-05-20T01:17:53.718 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-05-20T01:17:53.718 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2023-05-20T01:17:53.719 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-05-20T01:17:53.719 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-05-20T01:17:53.720 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-05-20T01:17:53.720 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-05-20T01:17:53.720 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-05-20T01:17:53.721 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-05-20T01:17:53.721 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:17:53.721 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2023-05-20T01:17:53.739 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-05-20T01:17:53.739 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-05-20T01:17:53.740 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2023-05-20T01:17:53.740 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-05-20T01:17:53.740 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2023-05-20T01:17:53.740 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-05-20T01:17:53.740 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-05-20T01:17:53.740 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2023-05-20T01:17:53.740 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-05-20T01:17:53.741 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2023-05-20T01:17:53.741 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-05-20T01:17:53.741 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2023-05-20T01:17:53.741 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-05-20T01:17:53.741 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 42360sec preferred_lft 42360sec 2023-05-20T01:17:53.742 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link 2023-05-20T01:17:53.742 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-05-20T01:17:53.742 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-05-20T01:17:53.742 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2023-05-20T01:17:53.742 INFO:teuthology.orchestra.run.smithi016.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-05-20T01:17:53.742 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0a:2e:0c:6b:5d:42 brd ff:ff:ff:ff:ff:ff 2023-05-20T01:17:53.743 INFO:teuthology.orchestra.run.smithi016.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-05-20T01:17:53.743 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-05-20T01:17:53.743 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::7c00:b2ff:fef4:946/64 scope link 2023-05-20T01:17:53.743 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-05-20T01:17:53.743 INFO:teuthology.orchestra.run.smithi016.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-05-20T01:17:53.743 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether d2:e4:70:a7:b9:4b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-05-20T01:17:53.744 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::d0e4:70ff:fea7:b94b/64 scope link 2023-05-20T01:17:53.744 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-05-20T01:17:53.744 INFO:teuthology.orchestra.run.smithi016.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-05-20T01:17:53.744 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0a:2e:0c:6b:5d:42 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-05-20T01:17:53.744 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::82e:cff:fe6b:5d42/64 scope link 2023-05-20T01:17:53.744 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-05-20T01:17:53.745 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:17:53.745 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-05-20T01:17:53.805 INFO:teuthology.orchestra.run.smithi016.stdout:1 2023-05-20T01:17:53.806 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:17:53.807 DEBUG:teuthology.orchestra.run.smithi016:> route 2023-05-20T01:17:53.866 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2023-05-20T01:17:53.866 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-05-20T01:17:53.866 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-05-20T01:17:53.866 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-05-20T01:17:53.866 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-05-20T01:17:53.867 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:17:53.867 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2023-05-20T01:17:53.867 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2023-05-20T01:17:53.868 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-05-20T01:17:53.868 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-05-20T01:17:53.869 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-05-20T01:17:53.869 DEBUG:teuthology.orchestra.run.smithi016:> ') 2023-05-20T01:17:53.973 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:17:53.973 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2023-05-20T01:17:53.990 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-05-20T01:17:53.991 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-05-20T01:17:53.991 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-05-20T01:17:53.991 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:17:53.992 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.0 up 2023-05-20T01:17:54.058 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-05-20T01:17:54.058 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-05-20T01:17:54.115 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-05-20T01:17:54.116 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-05-20T01:17:54.116 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-05-20T01:17:54.172 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-05-20T01:17:54.172 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:17:54.173 DEBUG:teuthology.orchestra.run.smithi016:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-05-20T01:17:54.259 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-05-20T01:17:54.259 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-05-20T01:17:54.287 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-05-20T01:17:54.288 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-05-20T01:17:54.288 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.132:3300,172.21.15.132:3301" 2023-05-20T01:17:54.288 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: trying mount with new device syntax: 0@c2245cc5-ecab-4087-a59b-bc81db901676.cephfs=/ 2023-05-20T01:17:54.288 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.16:3300/172.21.15.132:3300/172.21.15.132:3301" will pass to kernel 2023-05-20T01:17:54.288 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: v2 mount support check returned 2 2023-05-20T01:17:54.288 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: kernel does not support v2 syntax 2023-05-20T01:17:54.288 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: switching to using v1 address with old syntax 2023-05-20T01:17:54.330 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-05-20T01:17:54.331 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-05-20T01:17:54.331 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.132:3300,172.21.15.132:3301" 2023-05-20T01:17:54.331 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: trying mount with new device syntax: 0@c2245cc5-ecab-4087-a59b-bc81db901676.cephfs=/ 2023-05-20T01:17:54.331 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.16:3300/172.21.15.132:3300/172.21.15.132:3301" will pass to kernel 2023-05-20T01:17:54.331 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: v2 mount support check returned 2 2023-05-20T01:17:54.331 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: kernel does not support v2 syntax 2023-05-20T01:17:54.331 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: switching to using v1 address with old syntax 2023-05-20T01:17:54.332 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:6789,172.21.15.132:6789,172.21.15.132:6790" 2023-05-20T01:17:54.332 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.16:6789,172.21.15.132:6789,172.21.15.132:6790:/ 2023-05-20T01:17:54.332 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=c2245cc5-ecab-4087-a59b-bc81db901676" will pass to kernel 2023-05-20T01:17:54.337 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-05-20T01:17:54.338 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:17:54.338 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-05-20T01:17:54.370 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:17:54.370 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-05-20T01:17:54.452 INFO:teuthology.orchestra.run.smithi016.stdout:client5598 2023-05-20T01:17:54.453 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-05-20T01:17:54.453 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2023-05-20T01:17:54.510 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_extra --pg_num_min 64 2023-05-20T01:17:54.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:54.676+0000 7ff9a53d4700 1 Processor -- start 2023-05-20T01:17:54.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:54.676+0000 7ff9a53d4700 1 -- start start 2023-05-20T01:17:54.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:54.676+0000 7ff9a53d4700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff9a00b2670 0x7ff9a00b60d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:54.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:54.676+0000 7ff9a53d4700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7ff9a00d1860 0x7ff9a00d3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:54.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:54.677+0000 7ff9a53d4700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9a00d2eb0 con 0x7ff9a00d3880 2023-05-20T01:17:54.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:54.677+0000 7ff9a53d4700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9a00cd970 con 0x7ff9a00b2670 2023-05-20T01:17:54.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:54.677+0000 7ff9a53d4700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9a00cdc50 con 0x7ff9a00d1860 2023-05-20T01:17:54.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:54.677+0000 7ff99effd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff9a00b2670 0x7ff9a00b60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:54.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:54.677+0000 7ff99f7fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ff9a00d3880 0x7ff9a00cf7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:43682/0 (socket says 172.21.15.16:43682) 2023-05-20T01:17:54.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:54.677+0000 7ff99f7fe700 1 -- 172.21.15.16:0/647655876 learned_addr learned my addr 172.21.15.16:0/647655876 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:17:54.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:54.677+0000 7ff99e7fc700 1 --2- 172.21.15.16:0/647655876 >> v2:172.21.15.132:3300/0 conn(0x7ff9a00d1860 0x7ff9a00d3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:54.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:54.677+0000 7ff99effd700 1 -- 172.21.15.16:0/647655876 >> v2:172.21.15.132:3300/0 conn(0x7ff9a00d1860 msgr2=0x7ff9a00d3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:54.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:54.677+0000 7ff99effd700 1 --2- 172.21.15.16:0/647655876 >> v2:172.21.15.132:3300/0 conn(0x7ff9a00d1860 0x7ff9a00d3340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:54.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:54.677+0000 7ff99effd700 1 -- 172.21.15.16:0/647655876 >> v1:172.21.15.16:6789/0 conn(0x7ff9a00d3880 legacy=0x7ff9a00cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:54.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:54.677+0000 7ff99effd700 1 -- 172.21.15.16:0/647655876 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9a00d2b40 con 0x7ff9a00b2670 2023-05-20T01:17:54.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:54.677+0000 7ff99dffb700 1 -- 172.21.15.16:0/647655876 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4048413547 0 0) 0x7ff9a00d2eb0 con 0x7ff9a00d3880 2023-05-20T01:17:54.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:54.677+0000 7ff99e7fc700 1 --2- 172.21.15.16:0/647655876 >> v2:172.21.15.132:3300/0 conn(0x7ff9a00d1860 0x7ff9a00d3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:17:54.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:54.677+0000 7ff99effd700 1 --2- 172.21.15.16:0/647655876 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9a00b2670 0x7ff9a00b60d0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7ff9a00d0bd0 tx=0x7ff994009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29a5c59c5e0ae2c2 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:54.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:54.677+0000 7ff99dffb700 1 -- 172.21.15.16:0/647655876 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff994011830 con 0x7ff9a00b2670 2023-05-20T01:17:54.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:54.677+0000 7ff99dffb700 1 -- 172.21.15.16:0/647655876 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff994011990 con 0x7ff9a00b2670 2023-05-20T01:17:54.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:54.677+0000 7ff99dffb700 1 -- 172.21.15.16:0/647655876 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff994011d20 con 0x7ff9a00b2670 2023-05-20T01:17:54.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:54.678+0000 7ff9a53d4700 1 -- 172.21.15.16:0/647655876 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9a00b2670 msgr2=0x7ff9a00b60d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:54.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:54.678+0000 7ff9a53d4700 1 --2- 172.21.15.16:0/647655876 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9a00b2670 0x7ff9a00b60d0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7ff9a00d0bd0 tx=0x7ff994009b20 comp rx=0 tx=0).stop 2023-05-20T01:17:54.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:54.678+0000 7ff9a53d4700 1 -- 172.21.15.16:0/647655876 shutdown_connections 2023-05-20T01:17:54.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:54.678+0000 7ff9a53d4700 1 --2- 172.21.15.16:0/647655876 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9a00b2670 0x7ff9a00b60d0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:54.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:54.678+0000 7ff9a53d4700 1 --2- 172.21.15.16:0/647655876 >> v2:172.21.15.132:3300/0 conn(0x7ff9a00d1860 0x7ff9a00d3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:54.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:54.678+0000 7ff9a53d4700 1 -- 172.21.15.16:0/647655876 >> 172.21.15.16:0/647655876 conn(0x7ff9a0071c40 msgr2=0x7ff9a0072010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:54.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:54.678+0000 7ff9a53d4700 1 -- 172.21.15.16:0/647655876 shutdown_connections 2023-05-20T01:17:54.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:54.678+0000 7ff9a53d4700 1 -- 172.21.15.16:0/647655876 wait complete. 2023-05-20T01:17:54.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:54.679+0000 7ff9a53d4700 1 Processor -- start 2023-05-20T01:17:54.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:54.679+0000 7ff9a53d4700 1 -- start start 2023-05-20T01:17:54.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:54.680+0000 7ff9a53d4700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff9a00b2670 0x7ff9a00c6db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:54.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:54.680+0000 7ff9a53d4700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff9a00d1860 0x7ff9a00c9010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:54.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:54.680+0000 7ff9a53d4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9a00d3880 0x7ff9a00c9550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:54.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:54.680+0000 7ff9a53d4700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a00c72f0 con 0x7ff9a00d3880 2023-05-20T01:17:54.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:54.680+0000 7ff9a53d4700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a00c9a90 con 0x7ff9a00b2670 2023-05-20T01:17:54.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:54.680+0000 7ff9a53d4700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a015fa60 con 0x7ff9a00d1860 2023-05-20T01:17:54.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:54.680+0000 7ff99f7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9a00d3880 0x7ff9a00c9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:54.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:54.680+0000 7ff99f7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9a00d3880 0x7ff9a00c9550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60392/0 (socket says 172.21.15.16:60392) 2023-05-20T01:17:54.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:54.680+0000 7ff99f7fe700 1 -- 172.21.15.16:0/2840808527 learned_addr learned my addr 172.21.15.16:0/2840808527 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:54.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:54.680+0000 7ff99effd700 1 --2- 172.21.15.16:0/2840808527 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff9a00b2670 0x7ff9a00c6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:54.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:54.680+0000 7ff99e7fc700 1 --2- 172.21.15.16:0/2840808527 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff9a00d1860 0x7ff9a00c9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:54.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:54.680+0000 7ff99f7fe700 1 -- 172.21.15.16:0/2840808527 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff9a00d1860 msgr2=0x7ff9a00c9010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:54.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:54.680+0000 7ff99f7fe700 1 --2- 172.21.15.16:0/2840808527 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff9a00d1860 0x7ff9a00c9010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:54.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:54.680+0000 7ff99f7fe700 1 -- 172.21.15.16:0/2840808527 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff9a00b2670 msgr2=0x7ff9a00c6db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:54.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:54.680+0000 7ff99f7fe700 1 --2- 172.21.15.16:0/2840808527 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff9a00b2670 0x7ff9a00c6db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:54.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:54.681+0000 7ff99f7fe700 1 -- 172.21.15.16:0/2840808527 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9940094a0 con 0x7ff9a00d3880 2023-05-20T01:17:54.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:54.681+0000 7ff99f7fe700 1 --2- 172.21.15.16:0/2840808527 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9a00d3880 0x7ff9a00c9550 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7ff99800cba0 tx=0x7ff99800ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:54.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:54.681+0000 7ff9877fe700 1 -- 172.21.15.16:0/2840808527 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff99800aad0 con 0x7ff9a00d3880 2023-05-20T01:17:54.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:54.681+0000 7ff9877fe700 1 -- 172.21.15.16:0/2840808527 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff99800ac50 con 0x7ff9a00d3880 2023-05-20T01:17:54.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:54.681+0000 7ff9a53d4700 1 -- 172.21.15.16:0/2840808527 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9a00b2a40 con 0x7ff9a00d3880 2023-05-20T01:17:54.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:54.681+0000 7ff9877fe700 1 -- 172.21.15.16:0/2840808527 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9980070d0 con 0x7ff9a00d3880 2023-05-20T01:17:54.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:54.681+0000 7ff9a53d4700 1 -- 172.21.15.16:0/2840808527 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9a00cdf30 con 0x7ff9a00d3880 2023-05-20T01:17:54.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:54.682+0000 7ff9a53d4700 1 -- 172.21.15.16:0/2840808527 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9a00d1cb0 con 0x7ff9a00d3880 2023-05-20T01:17:54.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:54.682+0000 7ff9877fe700 1 -- 172.21.15.16:0/2840808527 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff998007230 con 0x7ff9a00d3880 2023-05-20T01:17:54.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:54.683+0000 7ff9877fe700 1 --2- 172.21.15.16:0/2840808527 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff98c06ad10 0x7ff98c06d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:54.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:54.683+0000 7ff9877fe700 1 -- 172.21.15.16:0/2840808527 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6382+0+0 (secure 0 0 0) 0x7ff9980892c0 con 0x7ff9a00d3880 2023-05-20T01:17:54.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:54.683+0000 7ff99effd700 1 --2- 172.21.15.16:0/2840808527 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff98c06ad10 0x7ff98c06d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:54.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:54.683+0000 7ff99effd700 1 --2- 172.21.15.16:0/2840808527 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff98c06ad10 0x7ff98c06d1c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff9940090b0 tx=0x7ff994008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:54.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:54.687+0000 7ff9877fe700 1 -- 172.21.15.16:0/2840808527 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7ff9980542c0 con 0x7ff9a00d3880 2023-05-20T01:17:54.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:54.884+0000 7ff9a53d4700 1 -- 172.21.15.16:0/2840808527 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_extra", "pg_num_min": 64} v 0) v1 -- 0x7ff9a004e8c0 con 0x7ff9a00d3880 2023-05-20T01:17:55.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.350+0000 7ff9877fe700 1 -- 172.21.15.16:0/2840808527 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_extra", "pg_num_min": 64}]=0 pool 'cephfs_data_extra' created v41) v1 ==== 142+0+0 (secure 0 0 0) 0x7ff9980578e0 con 0x7ff9a00d3880 2023-05-20T01:17:55.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.354+0000 7ff9a53d4700 1 -- 172.21.15.16:0/2840808527 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff98c06ad10 msgr2=0x7ff98c06d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:55.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.354+0000 7ff9a53d4700 1 --2- 172.21.15.16:0/2840808527 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff98c06ad10 0x7ff98c06d1c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff9940090b0 tx=0x7ff994008030 comp rx=0 tx=0).stop 2023-05-20T01:17:55.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.354+0000 7ff9a53d4700 1 -- 172.21.15.16:0/2840808527 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9a00d3880 msgr2=0x7ff9a00c9550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:55.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.354+0000 7ff9a53d4700 1 --2- 172.21.15.16:0/2840808527 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9a00d3880 0x7ff9a00c9550 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7ff99800cba0 tx=0x7ff99800ceb0 comp rx=0 tx=0).stop 2023-05-20T01:17:55.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.354+0000 7ff9a53d4700 1 -- 172.21.15.16:0/2840808527 shutdown_connections 2023-05-20T01:17:55.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.354+0000 7ff9a53d4700 1 --2- 172.21.15.16:0/2840808527 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9a00d3880 0x7ff9a00c9550 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:55.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.354+0000 7ff9a53d4700 1 --2- 172.21.15.16:0/2840808527 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff9a00b2670 0x7ff9a00c6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:55.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.354+0000 7ff9a53d4700 1 --2- 172.21.15.16:0/2840808527 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff98c06ad10 0x7ff98c06d1c0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:55.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.354+0000 7ff9a53d4700 1 --2- 172.21.15.16:0/2840808527 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff9a00d1860 0x7ff9a00c9010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:55.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.354+0000 7ff9a53d4700 1 -- 172.21.15.16:0/2840808527 >> 172.21.15.16:0/2840808527 conn(0x7ff9a0071c40 msgr2=0x7ff9a00b2bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:55.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.354+0000 7ff9a53d4700 1 -- 172.21.15.16:0/2840808527 shutdown_connections 2023-05-20T01:17:55.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.354+0000 7ff9a53d4700 1 -- 172.21.15.16:0/2840808527 wait complete. 2023-05-20T01:17:55.362 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data_extra' created 2023-05-20T01:17:55.374 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_extra 2023-05-20T01:17:55.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.498+0000 7f0fafb23700 1 Processor -- start 2023-05-20T01:17:55.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.498+0000 7f0fafb23700 1 -- start start 2023-05-20T01:17:55.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.499+0000 7f0fafb23700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f0fa8072300 0x7f0fa80cf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:55.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.499+0000 7f0fafb23700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fa80cd630 con 0x7f0fa815cec0 2023-05-20T01:17:55.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.499+0000 7f0fafb23700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fa80cca60 con 0x7f0fa815d290 2023-05-20T01:17:55.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.499+0000 7f0fafb23700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fa80ccd00 con 0x7f0fa8072300 2023-05-20T01:17:55.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.499+0000 7f0fad8bf700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f0fa815cec0 0x7f0fa8071bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:43692/0 (socket says 172.21.15.16:43692) 2023-05-20T01:17:55.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.499+0000 7f0fad8bf700 1 -- 172.21.15.16:0/1820863987 learned_addr learned my addr 172.21.15.16:0/1820863987 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:17:55.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.500+0000 7f0fad0be700 1 --2- 172.21.15.16:0/1820863987 >> v2:172.21.15.132:3300/0 conn(0x7f0fa8072300 0x7f0fa80cf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:55.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.500+0000 7f0fac8bd700 1 -- 172.21.15.16:0/1820863987 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2807050430 0 0) 0x7f0fa80cd630 con 0x7f0fa815cec0 2023-05-20T01:17:55.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.500+0000 7f0fac8bd700 1 -- 172.21.15.16:0/1820863987 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f940035b0 con 0x7f0fa815cec0 2023-05-20T01:17:55.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.500+0000 7f0fad0be700 1 -- 172.21.15.16:0/1820863987 >> v1:172.21.15.132:6789/0 conn(0x7f0fa815d290 legacy=0x7f0fa80cf800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:55.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.500+0000 7f0fac8bd700 1 -- 172.21.15.16:0/1820863987 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3352594510 0 0) 0x7f0f940035b0 con 0x7f0fa815cec0 2023-05-20T01:17:55.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.500+0000 7f0fad0be700 1 -- 172.21.15.16:0/1820863987 >> v1:172.21.15.16:6789/0 conn(0x7f0fa815cec0 legacy=0x7f0fa8071bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:55.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.500+0000 7f0fad0be700 1 -- 172.21.15.16:0/1820863987 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fa80ccfe0 con 0x7f0fa8072300 2023-05-20T01:17:55.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.500+0000 7f0fad0be700 1 --2- 172.21.15.16:0/1820863987 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0fa8072300 0x7f0fa80cf2c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f0fa4009840 tx=0x7f0fa4009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=864ff50abd8389d9 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:55.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.500+0000 7f0fac8bd700 1 -- 172.21.15.16:0/1820863987 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f980017f0 con 0x7f0fa8072300 2023-05-20T01:17:55.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.500+0000 7f0fac8bd700 1 -- 172.21.15.16:0/1820863987 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fa4004030 con 0x7f0fa8072300 2023-05-20T01:17:55.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.500+0000 7f0fac8bd700 1 -- 172.21.15.16:0/1820863987 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fa40043c0 con 0x7f0fa8072300 2023-05-20T01:17:55.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.501+0000 7f0fafb23700 1 -- 172.21.15.16:0/1820863987 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0fa8072300 msgr2=0x7f0fa80cf2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:55.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.501+0000 7f0fafb23700 1 --2- 172.21.15.16:0/1820863987 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0fa8072300 0x7f0fa80cf2c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f0fa4009840 tx=0x7f0fa4009b50 comp rx=0 tx=0).stop 2023-05-20T01:17:55.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.501+0000 7f0fafb23700 1 -- 172.21.15.16:0/1820863987 shutdown_connections 2023-05-20T01:17:55.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.501+0000 7f0fafb23700 1 --2- 172.21.15.16:0/1820863987 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0fa8072300 0x7f0fa80cf2c0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:55.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.501+0000 7f0fafb23700 1 -- 172.21.15.16:0/1820863987 >> 172.21.15.16:0/1820863987 conn(0x7f0fa80cffd0 msgr2=0x7f0fa80d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:55.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.501+0000 7f0fafb23700 1 -- 172.21.15.16:0/1820863987 shutdown_connections 2023-05-20T01:17:55.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.501+0000 7f0fafb23700 1 -- 172.21.15.16:0/1820863987 wait complete. 2023-05-20T01:17:55.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.502+0000 7f0fafb23700 1 Processor -- start 2023-05-20T01:17:55.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.502+0000 7f0fafb23700 1 -- start start 2023-05-20T01:17:55.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.503+0000 7f0fafb23700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0fa8072300 0x7f0fa8151e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:55.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.503+0000 7f0fafb23700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0fa815cec0 0x7f0fa81f6f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:55.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.503+0000 7f0fafb23700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0fa815d290 0x7f0fa815f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:55.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.503+0000 7f0fafb23700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fa8152350 con 0x7f0fa815d290 2023-05-20T01:17:55.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.503+0000 7f0fafb23700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fa81f7470 con 0x7f0fa8072300 2023-05-20T01:17:55.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.503+0000 7f0fafb23700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fa80b2970 con 0x7f0fa815cec0 2023-05-20T01:17:55.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.503+0000 7f0fae0c0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0fa815d290 0x7f0fa815f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:55.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.503+0000 7f0fae0c0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0fa815d290 0x7f0fa815f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60406/0 (socket says 172.21.15.16:60406) 2023-05-20T01:17:55.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.503+0000 7f0fae0c0700 1 -- 172.21.15.16:0/2274737421 learned_addr learned my addr 172.21.15.16:0/2274737421 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:55.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.503+0000 7f0fad8bf700 1 --2- 172.21.15.16:0/2274737421 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0fa8072300 0x7f0fa8151e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:55.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.503+0000 7f0fad0be700 1 --2- 172.21.15.16:0/2274737421 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0fa815cec0 0x7f0fa81f6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:55.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.504+0000 7f0fae0c0700 1 -- 172.21.15.16:0/2274737421 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0fa815cec0 msgr2=0x7f0fa81f6f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:55.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.504+0000 7f0fae0c0700 1 --2- 172.21.15.16:0/2274737421 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0fa815cec0 0x7f0fa81f6f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:55.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.504+0000 7f0fae0c0700 1 -- 172.21.15.16:0/2274737421 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0fa8072300 msgr2=0x7f0fa8151e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:55.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.504+0000 7f0fae0c0700 1 --2- 172.21.15.16:0/2274737421 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0fa8072300 0x7f0fa8151e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:55.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.504+0000 7f0fae0c0700 1 -- 172.21.15.16:0/2274737421 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fa40094a0 con 0x7f0fa815d290 2023-05-20T01:17:55.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.504+0000 7f0fae0c0700 1 --2- 172.21.15.16:0/2274737421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0fa815d290 0x7f0fa815f720 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f0fa000cb50 tx=0x7f0fa000ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:55.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.504+0000 7f0f9e7fc700 1 -- 172.21.15.16:0/2274737421 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fa0003c00 con 0x7f0fa815d290 2023-05-20T01:17:55.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.504+0000 7f0fafb23700 1 -- 172.21.15.16:0/2274737421 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fa8167b30 con 0x7f0fa815d290 2023-05-20T01:17:55.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.504+0000 7f0f9e7fc700 1 -- 172.21.15.16:0/2274737421 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fa0003d60 con 0x7f0fa815d290 2023-05-20T01:17:55.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.504+0000 7f0f9e7fc700 1 -- 172.21.15.16:0/2274737421 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fa000a9c0 con 0x7f0fa815d290 2023-05-20T01:17:55.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.504+0000 7f0fafb23700 1 -- 172.21.15.16:0/2274737421 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fa80cd2c0 con 0x7f0fa815d290 2023-05-20T01:17:55.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.505+0000 7f0fafb23700 1 -- 172.21.15.16:0/2274737421 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fa8167cc0 con 0x7f0fa815d290 2023-05-20T01:17:55.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.505+0000 7f0f9e7fc700 1 -- 172.21.15.16:0/2274737421 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0fa000ab20 con 0x7f0fa815d290 2023-05-20T01:17:55.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.506+0000 7f0f9e7fc700 1 --2- 172.21.15.16:0/2274737421 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f0f8c06ace0 0x7f0f8c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:55.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.506+0000 7f0f9e7fc700 1 -- 172.21.15.16:0/2274737421 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6754+0+0 (secure 0 0 0) 0x7f0fa0089670 con 0x7f0fa815d290 2023-05-20T01:17:55.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.506+0000 7f0fad8bf700 1 --2- 172.21.15.16:0/2274737421 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f0f8c06ace0 0x7f0f8c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:55.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.506+0000 7f0fad8bf700 1 --2- 172.21.15.16:0/2274737421 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f0f8c06ace0 0x7f0f8c06d190 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f0f98000ef0 tx=0x7f0f980012f0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:55.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.511+0000 7f0f9e7fc700 1 -- 172.21.15.16:0/2274737421 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f0fa0054470 con 0x7f0fa815d290 2023-05-20T01:17:55.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.756+0000 7f0fafb23700 1 -- 172.21.15.16:0/2274737421 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_extra"} v 0) v1 -- 0x7f0fa8066d90 con 0x7f0fa815d290 2023-05-20T01:17:55.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.763+0000 7f0f9e7fc700 1 -- 172.21.15.16:0/2274737421 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_extra"}]=0 added data pool 8 to fsmap v17) v1 ==== 140+0+0 (secure 0 0 0) 0x7f0fa0057a90 con 0x7f0fa815d290 2023-05-20T01:17:55.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.769+0000 7f0fafb23700 1 -- 172.21.15.16:0/2274737421 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f0f8c06ace0 msgr2=0x7f0f8c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:55.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.769+0000 7f0fafb23700 1 --2- 172.21.15.16:0/2274737421 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f0f8c06ace0 0x7f0f8c06d190 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f0f98000ef0 tx=0x7f0f980012f0 comp rx=0 tx=0).stop 2023-05-20T01:17:55.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.769+0000 7f0fafb23700 1 -- 172.21.15.16:0/2274737421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0fa815d290 msgr2=0x7f0fa815f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:55.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.769+0000 7f0fafb23700 1 --2- 172.21.15.16:0/2274737421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0fa815d290 0x7f0fa815f720 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f0fa000cb50 tx=0x7f0fa000ce60 comp rx=0 tx=0).stop 2023-05-20T01:17:55.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.769+0000 7f0fafb23700 1 -- 172.21.15.16:0/2274737421 shutdown_connections 2023-05-20T01:17:55.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.769+0000 7f0fafb23700 1 --2- 172.21.15.16:0/2274737421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0fa815d290 0x7f0fa815f720 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:55.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.769+0000 7f0fafb23700 1 --2- 172.21.15.16:0/2274737421 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0fa8072300 0x7f0fa8151e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:55.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.769+0000 7f0fafb23700 1 --2- 172.21.15.16:0/2274737421 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f0f8c06ace0 0x7f0f8c06d190 secure :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f0f98000ef0 tx=0x7f0f980012f0 comp rx=0 tx=0).stop 2023-05-20T01:17:55.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.769+0000 7f0fafb23700 1 --2- 172.21.15.16:0/2274737421 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0fa815cec0 0x7f0fa81f6f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:55.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.769+0000 7f0fafb23700 1 -- 172.21.15.16:0/2274737421 >> 172.21.15.16:0/2274737421 conn(0x7f0fa80cffd0 msgr2=0x7f0fa80cf0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:55.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.769+0000 7f0fafb23700 1 -- 172.21.15.16:0/2274737421 shutdown_connections 2023-05-20T01:17:55.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.769+0000 7f0fafb23700 1 -- 172.21.15.16:0/2274737421 wait complete. 2023-05-20T01:17:55.775 INFO:teuthology.orchestra.run.smithi016.stderr:added data pool 8 to fsmap 2023-05-20T01:17:55.787 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-20T01:17:55.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.908+0000 7f6f6eab1700 1 Processor -- start 2023-05-20T01:17:55.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.909+0000 7f6f6eab1700 1 -- start start 2023-05-20T01:17:55.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.909+0000 7f6f6eab1700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6f6806ac20 0x7f6f680cf890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:55.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.909+0000 7f6f6eab1700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f680cc510 con 0x7f6f6806f450 2023-05-20T01:17:55.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.909+0000 7f6f6eab1700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f68071ed0 con 0x7f6f6806d4b0 2023-05-20T01:17:55.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.909+0000 7f6f6eab1700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f680721b0 con 0x7f6f6806ac20 2023-05-20T01:17:55.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.909+0000 7f6f6c84d700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f6f6806f450 0x7f6f68161080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:43706/0 (socket says 172.21.15.16:43706) 2023-05-20T01:17:55.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.909+0000 7f6f6c84d700 1 -- 172.21.15.16:0/3213870411 learned_addr learned my addr 172.21.15.16:0/3213870411 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:17:55.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.910+0000 7f6f6d04e700 1 --2- 172.21.15.16:0/3213870411 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6f6806ac20 0x7f6f680cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:55.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.910+0000 7f6f677fe700 1 -- 172.21.15.16:0/3213870411 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1012133566 0 0) 0x7f6f680cc510 con 0x7f6f6806f450 2023-05-20T01:17:55.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.910+0000 7f6f677fe700 1 -- 172.21.15.16:0/3213870411 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f580035b0 con 0x7f6f6806f450 2023-05-20T01:17:55.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.910+0000 7f6f677fe700 1 -- 172.21.15.16:0/3213870411 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2146153015 0 0) 0x7f6f580035b0 con 0x7f6f6806f450 2023-05-20T01:17:55.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.910+0000 7f6f677fe700 1 -- 172.21.15.16:0/3213870411 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6f6806ac20 msgr2=0x7f6f680cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:55.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.910+0000 7f6f677fe700 1 --2- 172.21.15.16:0/3213870411 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6f6806ac20 0x7f6f680cf890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:55.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.910+0000 7f6f677fe700 1 -- 172.21.15.16:0/3213870411 >> v1:172.21.15.132:6789/0 conn(0x7f6f6806d4b0 legacy=0x7f6f6806a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:55.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.910+0000 7f6f677fe700 1 -- 172.21.15.16:0/3213870411 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f680cc1a0 con 0x7f6f6806f450 2023-05-20T01:17:55.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.910+0000 7f6f677fe700 1 -- 172.21.15.16:0/3213870411 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7f6f5c001670 con 0x7f6f6806f450 2023-05-20T01:17:55.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.910+0000 7f6f677fe700 1 -- 172.21.15.16:0/3213870411 >> v1:172.21.15.16:6789/0 conn(0x7f6f6806f450 legacy=0x7f6f68161080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:55.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.910+0000 7f6f677fe700 1 --2- 172.21.15.16:0/3213870411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f58003aa0 0x7f6f58003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:55.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.910+0000 7f6f677fe700 1 --2- 172.21.15.16:0/3213870411 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6f6806ac20 0x7f6f58004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:55.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.910+0000 7f6f677fe700 1 --2- 172.21.15.16:0/3213870411 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6f58004d90 0x7f6f58005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:55.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.910+0000 7f6f677fe700 1 -- 172.21.15.16:0/3213870411 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f680721b0 con 0x7f6f58003aa0 2023-05-20T01:17:55.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.910+0000 7f6f677fe700 1 -- 172.21.15.16:0/3213870411 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f5800bb10 con 0x7f6f58004d90 2023-05-20T01:17:55.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.910+0000 7f6f677fe700 1 -- 172.21.15.16:0/3213870411 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f5800bcd0 con 0x7f6f6806ac20 2023-05-20T01:17:55.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.910+0000 7f6f6c84d700 1 --2- 172.21.15.16:0/3213870411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f58003aa0 0x7f6f58003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:55.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.910+0000 7f6f67fff700 1 --2- 172.21.15.16:0/3213870411 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6f6806ac20 0x7f6f58004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:55.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.910+0000 7f6f6d04e700 1 --2- 172.21.15.16:0/3213870411 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6f58004d90 0x7f6f58005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:55.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.910+0000 7f6f6c84d700 1 -- 172.21.15.16:0/3213870411 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6f6806ac20 msgr2=0x7f6f58004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:55.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.910+0000 7f6f6c84d700 1 --2- 172.21.15.16:0/3213870411 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6f6806ac20 0x7f6f58004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:55.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.911+0000 7f6f6c84d700 1 -- 172.21.15.16:0/3213870411 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6f58004d90 msgr2=0x7f6f58005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:55.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.911+0000 7f6f6c84d700 1 --2- 172.21.15.16:0/3213870411 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6f58004d90 0x7f6f58005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:55.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.911+0000 7f6f6c84d700 1 -- 172.21.15.16:0/3213870411 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f580035b0 con 0x7f6f58003aa0 2023-05-20T01:17:55.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.911+0000 7f6f6c84d700 1 --2- 172.21.15.16:0/3213870411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f58003aa0 0x7f6f58003e70 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c00c2f0 tx=0x7f6f58005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81615c6ab871868a server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:55.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.911+0000 7f6f677fe700 1 -- 172.21.15.16:0/3213870411 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f5c00ce30 con 0x7f6f58003aa0 2023-05-20T01:17:55.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.911+0000 7f6f677fe700 1 -- 172.21.15.16:0/3213870411 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f5c005960 con 0x7f6f58003aa0 2023-05-20T01:17:55.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.911+0000 7f6f677fe700 1 -- 172.21.15.16:0/3213870411 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f5c005cf0 con 0x7f6f58003aa0 2023-05-20T01:17:55.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.911+0000 7f6f6eab1700 1 -- 172.21.15.16:0/3213870411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f58003aa0 msgr2=0x7f6f58003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:55.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.911+0000 7f6f6eab1700 1 --2- 172.21.15.16:0/3213870411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f58003aa0 0x7f6f58003e70 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c00c2f0 tx=0x7f6f58005880 comp rx=0 tx=0).stop 2023-05-20T01:17:55.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.911+0000 7f6f6d04e700 1 -- 172.21.15.16:0/3213870411 reap_dead start 2023-05-20T01:17:55.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.911+0000 7f6f6eab1700 1 -- 172.21.15.16:0/3213870411 shutdown_connections 2023-05-20T01:17:55.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.911+0000 7f6f6eab1700 1 -- 172.21.15.16:0/3213870411 >> 172.21.15.16:0/3213870411 conn(0x7f6f680cffb0 msgr2=0x7f6f680d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:55.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.911+0000 7f6f6eab1700 1 -- 172.21.15.16:0/3213870411 shutdown_connections 2023-05-20T01:17:55.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.911+0000 7f6f6eab1700 1 -- 172.21.15.16:0/3213870411 wait complete. 2023-05-20T01:17:55.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.913+0000 7f6f6eab1700 1 Processor -- start 2023-05-20T01:17:55.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.913+0000 7f6f6eab1700 1 -- start start 2023-05-20T01:17:55.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.914+0000 7f6f6eab1700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6f58003aa0 0x7f6f680b42f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:55.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.914+0000 7f6f6eab1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f681e7fe0 0x7f6f680b5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:55.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.914+0000 7f6f6eab1700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6f680b5a70 0x7f6f681f1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:55.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.914+0000 7f6f6eab1700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f680b3850 con 0x7f6f681e7fe0 2023-05-20T01:17:55.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.914+0000 7f6f6eab1700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f681e71b0 con 0x7f6f680b5a70 2023-05-20T01:17:55.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.914+0000 7f6f6eab1700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f681e7490 con 0x7f6f58003aa0 2023-05-20T01:17:55.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.914+0000 7f6f67fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f681e7fe0 0x7f6f680b5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:55.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.914+0000 7f6f67fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f681e7fe0 0x7f6f680b5530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60422/0 (socket says 172.21.15.16:60422) 2023-05-20T01:17:55.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.914+0000 7f6f67fff700 1 -- 172.21.15.16:0/1440706315 learned_addr learned my addr 172.21.15.16:0/1440706315 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:55.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.915+0000 7f6f6c84d700 1 --2- 172.21.15.16:0/1440706315 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6f58003aa0 0x7f6f680b42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:55.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.915+0000 7f6f6d04e700 1 --2- 172.21.15.16:0/1440706315 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6f680b5a70 0x7f6f681f1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:55.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.915+0000 7f6f67fff700 1 -- 172.21.15.16:0/1440706315 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6f58003aa0 msgr2=0x7f6f680b42f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:55.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.915+0000 7f6f67fff700 1 --2- 172.21.15.16:0/1440706315 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6f58003aa0 0x7f6f680b42f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:55.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.915+0000 7f6f67fff700 1 -- 172.21.15.16:0/1440706315 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6f680b5a70 msgr2=0x7f6f681f1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:55.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.915+0000 7f6f67fff700 1 --2- 172.21.15.16:0/1440706315 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6f680b5a70 0x7f6f681f1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:55.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.915+0000 7f6f67fff700 1 -- 172.21.15.16:0/1440706315 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f680cf890 con 0x7f6f681e7fe0 2023-05-20T01:17:55.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.915+0000 7f6f6d04e700 1 --2- 172.21.15.16:0/1440706315 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6f680b5a70 0x7f6f681f1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:17:55.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.915+0000 7f6f67fff700 1 --2- 172.21.15.16:0/1440706315 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f681e7fe0 0x7f6f680b5530 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f6f5400f820 tx=0x7f6f5400fbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:55.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.915+0000 7f6f657fa700 1 -- 172.21.15.16:0/1440706315 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f540052e0 con 0x7f6f681e7fe0 2023-05-20T01:17:55.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.915+0000 7f6f657fa700 1 -- 172.21.15.16:0/1440706315 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f54005440 con 0x7f6f681e7fe0 2023-05-20T01:17:55.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.915+0000 7f6f6eab1700 1 -- 172.21.15.16:0/1440706315 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f680cbe50 con 0x7f6f681e7fe0 2023-05-20T01:17:55.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.915+0000 7f6f657fa700 1 -- 172.21.15.16:0/1440706315 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f540038c0 con 0x7f6f681e7fe0 2023-05-20T01:17:55.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.915+0000 7f6f6eab1700 1 -- 172.21.15.16:0/1440706315 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f680cfa20 con 0x7f6f681e7fe0 2023-05-20T01:17:55.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.916+0000 7f6f6eab1700 1 -- 172.21.15.16:0/1440706315 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f680747b0 con 0x7f6f681e7fe0 2023-05-20T01:17:55.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.917+0000 7f6f657fa700 1 -- 172.21.15.16:0/1440706315 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6f54003a20 con 0x7f6f681e7fe0 2023-05-20T01:17:55.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.917+0000 7f6f657fa700 1 --2- 172.21.15.16:0/1440706315 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f6f4c06ace0 0x7f6f4c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:55.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.917+0000 7f6f657fa700 1 -- 172.21.15.16:0/1440706315 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6786+0+0 (secure 0 0 0) 0x7f6f540896f0 con 0x7f6f681e7fe0 2023-05-20T01:17:55.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.917+0000 7f6f6c84d700 1 --2- 172.21.15.16:0/1440706315 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f6f4c06ace0 0x7f6f4c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:55.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.917+0000 7f6f6c84d700 1 --2- 172.21.15.16:0/1440706315 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f6f4c06ace0 0x7f6f4c06d190 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c008fd0 tx=0x7f6f5c007000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:55.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:55.921+0000 7f6f657fa700 1 -- 172.21.15.16:0/1440706315 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f6f54054550 con 0x7f6f681e7fe0 2023-05-20T01:17:56.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:56.162+0000 7f6f6eab1700 1 -- 172.21.15.16:0/1440706315 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6f68071c10 con 0x7f6f681e7fe0 2023-05-20T01:17:56.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:56.162+0000 7f6f657fa700 1 -- 172.21.15.16:0/1440706315 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4676 (secure 0 0 0) 0x7f6f54057b70 con 0x7f6f681e7fe0 2023-05-20T01:17:56.166 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:17:56.167 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5176,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/4144439890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4144439890},{"type":"v1","addr":"172.21.15.16:6836","nonce":4144439890}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5188,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/2992755084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":2992755084},{"type":"v1","addr":"172.21.15.132:6834","nonce":2992755084}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/1979284857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1979284857},{"type":"v1","addr":"172.21.15.132:6835","nonce":1979284857}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-20T01:17:46.995921+0000","modified":"2023-05-20T01:17:55.758292+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5397},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5397":{"gid":5397,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.16:6837/942011026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":942011026},{"type":"v1","addr":"172.21.15.16:6837","nonce":942011026}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7,8],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-05-20T01:17:56.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:56.164+0000 7f6f6eab1700 1 -- 172.21.15.16:0/1440706315 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f6f4c06ace0 msgr2=0x7f6f4c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:56.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:56.164+0000 7f6f6eab1700 1 --2- 172.21.15.16:0/1440706315 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f6f4c06ace0 0x7f6f4c06d190 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c008fd0 tx=0x7f6f5c007000 comp rx=0 tx=0).stop 2023-05-20T01:17:56.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:56.164+0000 7f6f6eab1700 1 -- 172.21.15.16:0/1440706315 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f681e7fe0 msgr2=0x7f6f680b5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:56.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:56.164+0000 7f6f6eab1700 1 --2- 172.21.15.16:0/1440706315 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f681e7fe0 0x7f6f680b5530 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f6f5400f820 tx=0x7f6f5400fbe0 comp rx=0 tx=0).stop 2023-05-20T01:17:56.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:56.164+0000 7f6f6eab1700 1 -- 172.21.15.16:0/1440706315 shutdown_connections 2023-05-20T01:17:56.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:56.164+0000 7f6f6eab1700 1 --2- 172.21.15.16:0/1440706315 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6f680b5a70 0x7f6f681f1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:56.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:56.164+0000 7f6f6eab1700 1 --2- 172.21.15.16:0/1440706315 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f6f4c06ace0 0x7f6f4c06d190 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:56.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:56.164+0000 7f6f6eab1700 1 --2- 172.21.15.16:0/1440706315 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6f58003aa0 0x7f6f680b42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:56.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:56.164+0000 7f6f6eab1700 1 --2- 172.21.15.16:0/1440706315 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f681e7fe0 0x7f6f680b5530 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:56.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:56.164+0000 7f6f6eab1700 1 -- 172.21.15.16:0/1440706315 >> 172.21.15.16:0/1440706315 conn(0x7f6f680cffb0 msgr2=0x7f6f680c7a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:56.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:56.164+0000 7f6f6eab1700 1 -- 172.21.15.16:0/1440706315 shutdown_connections 2023-05-20T01:17:56.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:56.164+0000 7f6f6eab1700 1 -- 172.21.15.16:0/1440706315 wait complete. 2023-05-20T01:17:56.170 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 17 2023-05-20T01:17:56.182 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-20T01:17:56.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:56.304+0000 7f783bea4700 1 Processor -- start 2023-05-20T01:17:56.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:56.304+0000 7f783bea4700 1 -- start start 2023-05-20T01:17:56.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:56.305+0000 7f783bea4700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f7834144020 0x7f78340d1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:56.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:56.305+0000 7f783bea4700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78340cc500 con 0x7f78340cec60 2023-05-20T01:17:56.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:56.305+0000 7f783bea4700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78340cb8f0 con 0x7f78340b2760 2023-05-20T01:17:56.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:56.305+0000 7f783bea4700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78340cb6e0 con 0x7f7834144020 2023-05-20T01:17:56.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:56.305+0000 7f783a441700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f78340cec60 0x7f78340cc990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:43728/0 (socket says 172.21.15.16:43728) 2023-05-20T01:17:56.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:56.305+0000 7f783a441700 1 -- 172.21.15.16:0/4234767744 learned_addr learned my addr 172.21.15.16:0/4234767744 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:17:56.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:56.305+0000 7f783943f700 1 --2- 172.21.15.16:0/4234767744 >> v2:172.21.15.132:3300/0 conn(0x7f7834144020 0x7f78340d1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:56.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:56.305+0000 7f7838c3e700 1 -- 172.21.15.16:0/4234767744 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 383852272 0 0) 0x7f78340cc500 con 0x7f78340cec60 2023-05-20T01:17:56.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:56.305+0000 7f7838c3e700 1 -- 172.21.15.16:0/4234767744 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f78200035b0 con 0x7f78340cec60 2023-05-20T01:17:56.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:56.306+0000 7f7838c3e700 1 -- 172.21.15.16:0/4234767744 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4166755292 0 0) 0x7f78200035b0 con 0x7f78340cec60 2023-05-20T01:17:56.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:56.306+0000 7f7838c3e700 1 -- 172.21.15.16:0/4234767744 >> v2:172.21.15.132:3300/0 conn(0x7f7834144020 msgr2=0x7f78340d1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:56.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:56.306+0000 7f7838c3e700 1 --2- 172.21.15.16:0/4234767744 >> v2:172.21.15.132:3300/0 conn(0x7f7834144020 0x7f78340d1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:56.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:56.306+0000 7f7838c3e700 1 -- 172.21.15.16:0/4234767744 >> v1:172.21.15.132:6789/0 conn(0x7f78340b2760 legacy=0x7f78340b0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:56.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:56.306+0000 7f7838c3e700 1 -- 172.21.15.16:0/4234767744 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78340cbbd0 con 0x7f78340cec60 2023-05-20T01:17:56.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:56.306+0000 7f7838c3e700 1 -- 172.21.15.16:0/4234767744 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7f782c0019f0 con 0x7f78340cec60 2023-05-20T01:17:56.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:56.306+0000 7f783943f700 1 --2- 172.21.15.16:0/4234767744 >> v2:172.21.15.132:3300/0 conn(0x7f7834144020 0x7f78340d1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-05-20T01:17:56.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:56.306+0000 7f7838c3e700 1 -- 172.21.15.16:0/4234767744 >> v1:172.21.15.16:6789/0 conn(0x7f78340cec60 legacy=0x7f78340cc990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:56.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:56.306+0000 7f7838c3e700 1 --2- 172.21.15.16:0/4234767744 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7820003a70 0x7f7820003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:56.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:56.306+0000 7f7838c3e700 1 --2- 172.21.15.16:0/4234767744 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7820004850 0x7f7820004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:56.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:56.306+0000 7f7838c3e700 1 --2- 172.21.15.16:0/4234767744 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7820005200 0x7f782000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:56.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:56.306+0000 7f7838c3e700 1 -- 172.21.15.16:0/4234767744 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78340cb6e0 con 0x7f7820003a70 2023-05-20T01:17:56.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:56.306+0000 7f7838c3e700 1 -- 172.21.15.16:0/4234767744 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f782000c0e0 con 0x7f7820004850 2023-05-20T01:17:56.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:56.306+0000 7f7838c3e700 1 -- 172.21.15.16:0/4234767744 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f782000c2a0 con 0x7f7820005200 2023-05-20T01:17:56.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:56.306+0000 7f7839c40700 1 --2- 172.21.15.16:0/4234767744 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7820003a70 0x7f7820003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:56.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:56.306+0000 7f783943f700 1 --2- 172.21.15.16:0/4234767744 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7820004850 0x7f7820004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:56.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:56.306+0000 7f783a441700 1 --2- 172.21.15.16:0/4234767744 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7820005200 0x7f782000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:56.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:56.306+0000 7f7839c40700 1 -- 172.21.15.16:0/4234767744 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7820005200 msgr2=0x7f782000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:56.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:56.306+0000 7f7839c40700 1 --2- 172.21.15.16:0/4234767744 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7820005200 0x7f782000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:56.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:56.306+0000 7f7839c40700 1 -- 172.21.15.16:0/4234767744 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7820004850 msgr2=0x7f7820004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:17:56.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:56.306+0000 7f7839c40700 1 --2- 172.21.15.16:0/4234767744 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7820004850 0x7f7820004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:56.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:56.306+0000 7f7839c40700 1 -- 172.21.15.16:0/4234767744 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78200035b0 con 0x7f7820003a70 2023-05-20T01:17:56.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:56.306+0000 7f783a441700 1 -- 172.21.15.16:0/4234767744 reap_dead start 2023-05-20T01:17:56.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:56.306+0000 7f7839c40700 1 --2- 172.21.15.16:0/4234767744 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7820003a70 0x7f7820003e80 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f78240017c0 tx=0x7f7820005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54562e297a89f19a server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:56.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:56.307+0000 7f7838c3e700 1 -- 172.21.15.16:0/4234767744 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78240052d0 con 0x7f7820003a70 2023-05-20T01:17:56.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:56.307+0000 7f7838c3e700 1 -- 172.21.15.16:0/4234767744 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7824005430 con 0x7f7820003a70 2023-05-20T01:17:56.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:56.307+0000 7f7838c3e700 1 -- 172.21.15.16:0/4234767744 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78240038c0 con 0x7f7820003a70 2023-05-20T01:17:56.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:56.307+0000 7f783bea4700 1 -- 172.21.15.16:0/4234767744 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7820003a70 msgr2=0x7f7820003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:56.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:56.307+0000 7f783bea4700 1 --2- 172.21.15.16:0/4234767744 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7820003a70 0x7f7820003e80 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f78240017c0 tx=0x7f7820005880 comp rx=0 tx=0).stop 2023-05-20T01:17:56.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:56.307+0000 7f783bea4700 1 -- 172.21.15.16:0/4234767744 shutdown_connections 2023-05-20T01:17:56.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:56.307+0000 7f783bea4700 1 --2- 172.21.15.16:0/4234767744 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7820003a70 0x7f7820003e80 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:56.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:56.307+0000 7f783bea4700 1 -- 172.21.15.16:0/4234767744 >> 172.21.15.16:0/4234767744 conn(0x7f783406ae30 msgr2=0x7f783409a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:56.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:56.307+0000 7f783bea4700 1 -- 172.21.15.16:0/4234767744 shutdown_connections 2023-05-20T01:17:56.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:56.307+0000 7f783bea4700 1 -- 172.21.15.16:0/4234767744 wait complete. 2023-05-20T01:17:56.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:56.308+0000 7f783bea4700 1 Processor -- start 2023-05-20T01:17:56.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:56.308+0000 7f783bea4700 1 -- start start 2023-05-20T01:17:56.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:56.309+0000 7f783bea4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7820003a70 0x7f78340c8160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:56.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:56.309+0000 7f783bea4700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f783406d2c0 0x7f78341f2360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:56.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:56.309+0000 7f783bea4700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f783406f410 0x7f783406f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:56.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:56.309+0000 7f783bea4700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f783406fdf0 con 0x7f7820003a70 2023-05-20T01:17:56.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:56.309+0000 7f783bea4700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78340c89b0 con 0x7f783406f410 2023-05-20T01:17:56.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:56.309+0000 7f783bea4700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78340c8c90 con 0x7f783406d2c0 2023-05-20T01:17:56.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:56.309+0000 7f7839c40700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7820003a70 0x7f78340c8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:56.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:56.309+0000 7f7839c40700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7820003a70 0x7f78340c8160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60440/0 (socket says 172.21.15.16:60440) 2023-05-20T01:17:56.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:56.309+0000 7f7839c40700 1 -- 172.21.15.16:0/3347905805 learned_addr learned my addr 172.21.15.16:0/3347905805 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:17:56.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:56.309+0000 7f783943f700 1 --2- 172.21.15.16:0/3347905805 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f783406d2c0 0x7f78341f2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:56.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:56.309+0000 7f783a441700 1 --2- 172.21.15.16:0/3347905805 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f783406f410 0x7f783406f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:56.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:56.310+0000 7f7839c40700 1 -- 172.21.15.16:0/3347905805 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f783406d2c0 msgr2=0x7f78341f2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:56.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:56.310+0000 7f7839c40700 1 --2- 172.21.15.16:0/3347905805 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f783406d2c0 0x7f78341f2360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:56.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:56.310+0000 7f7839c40700 1 -- 172.21.15.16:0/3347905805 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f783406f410 msgr2=0x7f783406f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:56.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:56.310+0000 7f7839c40700 1 --2- 172.21.15.16:0/3347905805 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f783406f410 0x7f783406f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:56.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:56.310+0000 7f7839c40700 1 -- 172.21.15.16:0/3347905805 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78340cc190 con 0x7f7820003a70 2023-05-20T01:17:56.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:56.310+0000 7f783943f700 1 --2- 172.21.15.16:0/3347905805 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f783406d2c0 0x7f78341f2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:17:56.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:56.310+0000 7f7839c40700 1 --2- 172.21.15.16:0/3347905805 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7820003a70 0x7f78340c8160 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f7824005040 tx=0x7f7824001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:56.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:56.310+0000 7f782a7fc700 1 -- 172.21.15.16:0/3347905805 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7824001290 con 0x7f7820003a70 2023-05-20T01:17:56.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:56.310+0000 7f782a7fc700 1 -- 172.21.15.16:0/3347905805 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f782401d480 con 0x7f7820003a70 2023-05-20T01:17:56.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:56.310+0000 7f783bea4700 1 -- 172.21.15.16:0/3347905805 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78340cbeb0 con 0x7f7820003a70 2023-05-20T01:17:56.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:56.310+0000 7f782a7fc700 1 -- 172.21.15.16:0/3347905805 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f782401d810 con 0x7f7820003a70 2023-05-20T01:17:56.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:56.310+0000 7f783bea4700 1 -- 172.21.15.16:0/3347905805 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78340d1680 con 0x7f7820003a70 2023-05-20T01:17:56.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:56.311+0000 7f783bea4700 1 -- 172.21.15.16:0/3347905805 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78340747b0 con 0x7f7820003a70 2023-05-20T01:17:56.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:56.311+0000 7f782a7fc700 1 -- 172.21.15.16:0/3347905805 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7824019040 con 0x7f7820003a70 2023-05-20T01:17:56.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:56.312+0000 7f782a7fc700 1 --2- 172.21.15.16:0/3347905805 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f781806ace0 0x7f781806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:17:56.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:56.312+0000 7f782a7fc700 1 -- 172.21.15.16:0/3347905805 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6786+0+0 (secure 0 0 0) 0x7f782408a260 con 0x7f7820003a70 2023-05-20T01:17:56.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:56.312+0000 7f783943f700 1 --2- 172.21.15.16:0/3347905805 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f781806ace0 0x7f781806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:17:56.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:56.312+0000 7f783943f700 1 --2- 172.21.15.16:0/3347905805 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f781806ace0 0x7f781806d190 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f78340aff00 tx=0x7f7830008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:17:56.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:56.316+0000 7f782a7fc700 1 -- 172.21.15.16:0/3347905805 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f78240550c0 con 0x7f7820003a70 2023-05-20T01:17:56.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:56.521+0000 7f783bea4700 1 -- 172.21.15.16:0/3347905805 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f78340ce610 con 0x7f7820003a70 2023-05-20T01:17:56.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:56.522+0000 7f782a7fc700 1 -- 172.21.15.16:0/3347905805 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+20203 (secure 0 0 0) 0x7f78240586e0 con 0x7f7820003a70 2023-05-20T01:17:56.526 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:17:56.526 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":42,"fsid":"c2245cc5-ecab-4087-a59b-bc81db901676","created":"2023-05-20T01:16:09.549230+0000","modified":"2023-05-20T01:17:55.758013+0000","last_up_change":"2023-05-20T01:16:19.788624+0000","last_in_change":"2023-05-20T01:16:12.092160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-05-20T01:16:20.934366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-05-20T01:16:21.372294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-05-20T01:17:44.628755+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-05-20T01:17:45.968584+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data_extra","create_time":"2023-05-20T01:17:54.915781+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"317a83cd-19c8-4958-a301-6087ef342289","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6825","nonce":3923634179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6827","nonce":3923634179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6831","nonce":3923634179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6829","nonce":3923634179}]},"public_addr":"172.21.15.16:6825/3923634179","cluster_addr":"172.21.15.16:6827/3923634179","heartbeat_back_addr":"172.21.15.16:6831/3923634179","heartbeat_front_addr":"172.21.15.16:6829/3923634179","state":["exists","up"]},{"osd":1,"uuid":"5581046f-8219-4089-8d7d-b7846399505e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6809","nonce":3860264220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6811","nonce":3860264220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6815","nonce":3860264220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6813","nonce":3860264220}]},"public_addr":"172.21.15.16:6809/3860264220","cluster_addr":"172.21.15.16:6811/3860264220","heartbeat_back_addr":"172.21.15.16:6815/3860264220","heartbeat_front_addr":"172.21.15.16:6813/3860264220","state":["exists","up"]},{"osd":2,"uuid":"072e9e66-4133-478e-8d4a-64d46c2963f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6817","nonce":293187030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6819","nonce":293187030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6823","nonce":293187030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6821","nonce":293187030}]},"public_addr":"172.21.15.16:6817/293187030","cluster_addr":"172.21.15.16:6819/293187030","heartbeat_back_addr":"172.21.15.16:6823/293187030","heartbeat_front_addr":"172.21.15.16:6821/293187030","state":["exists","up"]},{"osd":3,"uuid":"11e1bba6-00f8-48dd-9028-d8d716b7f145","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6801","nonce":3669781890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6803","nonce":3669781890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6807","nonce":3669781890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6805","nonce":3669781890}]},"public_addr":"172.21.15.16:6801/3669781890","cluster_addr":"172.21.15.16:6803/3669781890","heartbeat_back_addr":"172.21.15.16:6807/3669781890","heartbeat_front_addr":"172.21.15.16:6805/3669781890","state":["exists","up"]},{"osd":4,"uuid":"b8909c3a-cf70-46cb-8d0b-009c707cf9d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6819","nonce":506317223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6823","nonce":506317223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6829","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6831","nonce":506317223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6827","nonce":506317223}]},"public_addr":"172.21.15.132:6819/506317223","cluster_addr":"172.21.15.132:6823/506317223","heartbeat_back_addr":"172.21.15.132:6831/506317223","heartbeat_front_addr":"172.21.15.132:6827/506317223","state":["exists","up"]},{"osd":5,"uuid":"b1818304-4583-47bf-a6ec-0bfbb5166233","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6818","nonce":1844689995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6822","nonce":1844689995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6830","nonce":1844689995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6826","nonce":1844689995}]},"public_addr":"172.21.15.132:6818/1844689995","cluster_addr":"172.21.15.132:6822/1844689995","heartbeat_back_addr":"172.21.15.132:6830/1844689995","heartbeat_front_addr":"172.21.15.132:6826/1844689995","state":["exists","up"]},{"osd":6,"uuid":"d37f4691-eed1-4a93-8818-18354f03f4a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6801","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6802","nonce":727581123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6805","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6807","nonce":727581123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6813","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6815","nonce":727581123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6811","nonce":727581123}]},"public_addr":"172.21.15.132:6802/727581123","cluster_addr":"172.21.15.132:6807/727581123","heartbeat_back_addr":"172.21.15.132:6815/727581123","heartbeat_front_addr":"172.21.15.132:6811/727581123","state":["exists","up"]},{"osd":7,"uuid":"4f712f62-5d1d-4816-8042-03209144f8b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6803","nonce":466001149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6806","nonce":466001149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6814","nonce":466001149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6810","nonce":466001149}]},"public_addr":"172.21.15.132:6803/466001149","cluster_addr":"172.21.15.132:6806/466001149","heartbeat_back_addr":"172.21.15.132:6814/466001149","heartbeat_front_addr":"172.21.15.132:6810/466001149","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.338863+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.278680+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305694+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.246835+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.276237+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.304441+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305421+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.267050+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-20T01:17:56.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:56.524+0000 7f783bea4700 1 -- 172.21.15.16:0/3347905805 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f781806ace0 msgr2=0x7f781806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:56.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:56.524+0000 7f783bea4700 1 --2- 172.21.15.16:0/3347905805 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f781806ace0 0x7f781806d190 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f78340aff00 tx=0x7f7830008040 comp rx=0 tx=0).stop 2023-05-20T01:17:56.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:56.524+0000 7f783bea4700 1 -- 172.21.15.16:0/3347905805 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7820003a70 msgr2=0x7f78340c8160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:17:56.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:56.524+0000 7f783bea4700 1 --2- 172.21.15.16:0/3347905805 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7820003a70 0x7f78340c8160 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f7824005040 tx=0x7f7824001120 comp rx=0 tx=0).stop 2023-05-20T01:17:56.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:56.524+0000 7f783bea4700 1 -- 172.21.15.16:0/3347905805 shutdown_connections 2023-05-20T01:17:56.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:56.524+0000 7f783bea4700 1 --2- 172.21.15.16:0/3347905805 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f783406f410 0x7f783406f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:56.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:56.524+0000 7f783bea4700 1 --2- 172.21.15.16:0/3347905805 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7820003a70 0x7f78340c8160 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:56.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:56.524+0000 7f783bea4700 1 --2- 172.21.15.16:0/3347905805 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f781806ace0 0x7f781806d190 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:56.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:56.524+0000 7f783bea4700 1 --2- 172.21.15.16:0/3347905805 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f783406d2c0 0x7f78341f2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:17:56.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:56.524+0000 7f783bea4700 1 -- 172.21.15.16:0/3347905805 >> 172.21.15.16:0/3347905805 conn(0x7f783406ae30 msgr2=0x7f78340cf8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:17:56.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:56.525+0000 7f783bea4700 1 -- 172.21.15.16:0/3347905805 shutdown_connections 2023-05-20T01:17:56.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:17:56.525+0000 7f783bea4700 1 -- 172.21.15.16:0/3347905805 wait complete. 2023-05-20T01:17:56.540 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:17:56.541 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.layout.pool -v cephfs_data_extra .) 2023-05-20T01:17:56.565 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:17:56.565 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir subdir) 2023-05-20T01:17:56.625 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/0 2023-05-20T01:17:56.625 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits daemon-helper kill python3 -c ' 2023-05-20T01:17:56.626 DEBUG:teuthology.orchestra.run.smithi016:> import zlib 2023-05-20T01:17:56.626 DEBUG:teuthology.orchestra.run.smithi016:> path = "/home/ubuntu/cephtest/mnt.0/subdir/0" 2023-05-20T01:17:56.626 DEBUG:teuthology.orchestra.run.smithi016:> with open(path, '"'"'w'"'"') as f: 2023-05-20T01:17:56.627 DEBUG:teuthology.orchestra.run.smithi016:> for i in range(0, 6291456): 2023-05-20T01:17:56.627 DEBUG:teuthology.orchestra.run.smithi016:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-05-20T01:17:56.627 DEBUG:teuthology.orchestra.run.smithi016:> f.write(chr(val)) 2023-05-20T01:17:56.627 DEBUG:teuthology.orchestra.run.smithi016:> ' 2023-05-20T01:18:01.527 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/1 2023-05-20T01:18:01.528 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits daemon-helper kill python3 -c ' 2023-05-20T01:18:01.529 DEBUG:teuthology.orchestra.run.smithi016:> import zlib 2023-05-20T01:18:01.530 DEBUG:teuthology.orchestra.run.smithi016:> path = "/home/ubuntu/cephtest/mnt.0/subdir/1" 2023-05-20T01:18:01.530 DEBUG:teuthology.orchestra.run.smithi016:> with open(path, '"'"'w'"'"') as f: 2023-05-20T01:18:01.531 DEBUG:teuthology.orchestra.run.smithi016:> for i in range(0, 6291456): 2023-05-20T01:18:01.531 DEBUG:teuthology.orchestra.run.smithi016:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-05-20T01:18:01.531 DEBUG:teuthology.orchestra.run.smithi016:> f.write(chr(val)) 2023-05-20T01:18:01.532 DEBUG:teuthology.orchestra.run.smithi016:> ' 2023-05-20T01:18:05.204 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:18:05.206 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:18:06.430 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/2 2023-05-20T01:18:06.430 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits daemon-helper kill python3 -c ' 2023-05-20T01:18:06.431 DEBUG:teuthology.orchestra.run.smithi016:> import zlib 2023-05-20T01:18:06.431 DEBUG:teuthology.orchestra.run.smithi016:> path = "/home/ubuntu/cephtest/mnt.0/subdir/2" 2023-05-20T01:18:06.431 DEBUG:teuthology.orchestra.run.smithi016:> with open(path, '"'"'w'"'"') as f: 2023-05-20T01:18:06.431 DEBUG:teuthology.orchestra.run.smithi016:> for i in range(0, 6291456): 2023-05-20T01:18:06.432 DEBUG:teuthology.orchestra.run.smithi016:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-05-20T01:18:06.432 DEBUG:teuthology.orchestra.run.smithi016:> f.write(chr(val)) 2023-05-20T01:18:06.432 DEBUG:teuthology.orchestra.run.smithi016:> ' 2023-05-20T01:18:11.094 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/3 2023-05-20T01:18:11.095 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits daemon-helper kill python3 -c ' 2023-05-20T01:18:11.095 DEBUG:teuthology.orchestra.run.smithi016:> import zlib 2023-05-20T01:18:11.096 DEBUG:teuthology.orchestra.run.smithi016:> path = "/home/ubuntu/cephtest/mnt.0/subdir/3" 2023-05-20T01:18:11.096 DEBUG:teuthology.orchestra.run.smithi016:> with open(path, '"'"'w'"'"') as f: 2023-05-20T01:18:11.097 DEBUG:teuthology.orchestra.run.smithi016:> for i in range(0, 6291456): 2023-05-20T01:18:11.097 DEBUG:teuthology.orchestra.run.smithi016:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-05-20T01:18:11.098 DEBUG:teuthology.orchestra.run.smithi016:> f.write(chr(val)) 2023-05-20T01:18:11.098 DEBUG:teuthology.orchestra.run.smithi016:> ' 2023-05-20T01:18:15.996 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/4 2023-05-20T01:18:15.997 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits daemon-helper kill python3 -c ' 2023-05-20T01:18:15.997 DEBUG:teuthology.orchestra.run.smithi016:> import zlib 2023-05-20T01:18:15.997 DEBUG:teuthology.orchestra.run.smithi016:> path = "/home/ubuntu/cephtest/mnt.0/subdir/4" 2023-05-20T01:18:15.998 DEBUG:teuthology.orchestra.run.smithi016:> with open(path, '"'"'w'"'"') as f: 2023-05-20T01:18:15.998 DEBUG:teuthology.orchestra.run.smithi016:> for i in range(0, 6291456): 2023-05-20T01:18:15.998 DEBUG:teuthology.orchestra.run.smithi016:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-05-20T01:18:15.999 DEBUG:teuthology.orchestra.run.smithi016:> f.write(chr(val)) 2023-05-20T01:18:15.999 DEBUG:teuthology.orchestra.run.smithi016:> ' 2023-05-20T01:18:20.898 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-05-20T01:18:20.899 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2023-05-20T01:18:20.956 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-05-20T01:18:20.957 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2023-05-20T01:18:21.014 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-20T01:18:21.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.177+0000 7f865209d700 1 Processor -- start 2023-05-20T01:18:21.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.177+0000 7f865209d700 1 -- start start 2023-05-20T01:18:21.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.177+0000 7f865209d700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f864c06f450 0x7f864c0cf890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:21.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.177+0000 7f865209d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f864c06d4b0 0x7f864c161080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:21.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.178+0000 7f865209d700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f864c06a510 0x7f864c06a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:21.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.178+0000 7f865209d700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f864c06aef0 con 0x7f864c06d4b0 2023-05-20T01:18:21.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.178+0000 7f865209d700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f864c0cc510 con 0x7f864c06a510 2023-05-20T01:18:21.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.178+0000 7f865209d700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f864c1615c0 con 0x7f864c06f450 2023-05-20T01:18:21.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.178+0000 7f864affd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f864c06d4b0 0x7f864c161080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:21.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.178+0000 7f864affd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f864c06d4b0 0x7f864c161080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60454/0 (socket says 172.21.15.16:60454) 2023-05-20T01:18:21.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.178+0000 7f864affd700 1 -- 172.21.15.16:0/2968606765 learned_addr learned my addr 172.21.15.16:0/2968606765 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:18:21.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.178+0000 7f864bfff700 1 --2- 172.21.15.16:0/2968606765 >> v2:172.21.15.132:3300/0 conn(0x7f864c06a510 0x7f864c06a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:21.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.178+0000 7f864b7fe700 1 --2- 172.21.15.16:0/2968606765 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f864c06f450 0x7f864c0cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:21.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.178+0000 7f864affd700 1 -- 172.21.15.16:0/2968606765 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f864c06f450 msgr2=0x7f864c0cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:18:21.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.178+0000 7f864affd700 1 --2- 172.21.15.16:0/2968606765 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f864c06f450 0x7f864c0cf890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:21.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.178+0000 7f864affd700 1 -- 172.21.15.16:0/2968606765 >> v2:172.21.15.132:3300/0 conn(0x7f864c06a510 msgr2=0x7f864c06a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:18:21.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.178+0000 7f864affd700 1 --2- 172.21.15.16:0/2968606765 >> v2:172.21.15.132:3300/0 conn(0x7f864c06a510 0x7f864c06a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:21.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.178+0000 7f864affd700 1 -- 172.21.15.16:0/2968606765 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f864c071ed0 con 0x7f864c06d4b0 2023-05-20T01:18:21.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.178+0000 7f864affd700 1 --2- 172.21.15.16:0/2968606765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f864c06d4b0 0x7f864c161080 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f864c0c7010 tx=0x7f8640009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=873ddc9e3fc8f5f0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:21.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.178+0000 7f864a7fc700 1 -- 172.21.15.16:0/2968606765 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f864000b870 con 0x7f864c06d4b0 2023-05-20T01:18:21.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.178+0000 7f864a7fc700 1 -- 172.21.15.16:0/2968606765 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f864000b9d0 con 0x7f864c06d4b0 2023-05-20T01:18:21.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.178+0000 7f864a7fc700 1 -- 172.21.15.16:0/2968606765 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f864000bd60 con 0x7f864c06d4b0 2023-05-20T01:18:21.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.179+0000 7f865209d700 1 -- 172.21.15.16:0/2968606765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f864c06d4b0 msgr2=0x7f864c161080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:21.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.179+0000 7f865209d700 1 --2- 172.21.15.16:0/2968606765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f864c06d4b0 0x7f864c161080 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f864c0c7010 tx=0x7f8640009bd0 comp rx=0 tx=0).stop 2023-05-20T01:18:21.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.179+0000 7f865209d700 1 -- 172.21.15.16:0/2968606765 shutdown_connections 2023-05-20T01:18:21.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.179+0000 7f865209d700 1 --2- 172.21.15.16:0/2968606765 >> v2:172.21.15.132:3300/0 conn(0x7f864c06a510 0x7f864c06a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:21.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.179+0000 7f865209d700 1 --2- 172.21.15.16:0/2968606765 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f864c06f450 0x7f864c0cf890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:21.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.179+0000 7f865209d700 1 --2- 172.21.15.16:0/2968606765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f864c06d4b0 0x7f864c161080 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:21.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.179+0000 7f865209d700 1 -- 172.21.15.16:0/2968606765 >> 172.21.15.16:0/2968606765 conn(0x7f864c0cffb0 msgr2=0x7f864c0d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:18:21.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.179+0000 7f865209d700 1 -- 172.21.15.16:0/2968606765 shutdown_connections 2023-05-20T01:18:21.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.179+0000 7f865209d700 1 -- 172.21.15.16:0/2968606765 wait complete. 2023-05-20T01:18:21.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.180+0000 7f865209d700 1 Processor -- start 2023-05-20T01:18:21.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.180+0000 7f865209d700 1 -- start start 2023-05-20T01:18:21.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.181+0000 7f865209d700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f864c06a510 0x7f864c154620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:21.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.181+0000 7f865209d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f864c06d4b0 0x7f864c0b3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:21.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.181+0000 7f865209d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f864c06f450 0x7f864c0b1b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:21.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.181+0000 7f865209d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f864c0b42d0 con 0x7f864c06f450 2023-05-20T01:18:21.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.181+0000 7f865209d700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f864c155fb0 con 0x7f864c06d4b0 2023-05-20T01:18:21.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.181+0000 7f865209d700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f864c156290 con 0x7f864c06a510 2023-05-20T01:18:21.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.181+0000 7f864bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f864c06f450 0x7f864c0b1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:21.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.181+0000 7f864bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f864c06f450 0x7f864c0b1b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60462/0 (socket says 172.21.15.16:60462) 2023-05-20T01:18:21.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.181+0000 7f864bfff700 1 -- 172.21.15.16:0/3922425590 learned_addr learned my addr 172.21.15.16:0/3922425590 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:18:21.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.181+0000 7f864affd700 1 --2- 172.21.15.16:0/3922425590 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f864c06d4b0 0x7f864c0b3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:21.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.181+0000 7f864b7fe700 1 --2- 172.21.15.16:0/3922425590 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f864c06a510 0x7f864c154620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:21.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.182+0000 7f864bfff700 1 -- 172.21.15.16:0/3922425590 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f864c06a510 msgr2=0x7f864c154620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:21.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.182+0000 7f864bfff700 1 --2- 172.21.15.16:0/3922425590 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f864c06a510 0x7f864c154620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:21.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.182+0000 7f864bfff700 1 -- 172.21.15.16:0/3922425590 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f864c06d4b0 msgr2=0x7f864c0b3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:21.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.182+0000 7f864bfff700 1 --2- 172.21.15.16:0/3922425590 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f864c06d4b0 0x7f864c0b3800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:21.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.182+0000 7f864bfff700 1 -- 172.21.15.16:0/3922425590 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8640009550 con 0x7f864c06f450 2023-05-20T01:18:21.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.182+0000 7f864bfff700 1 --2- 172.21.15.16:0/3922425590 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f864c06f450 0x7f864c0b1b20 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f8644007e60 tx=0x7f864400e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:21.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.182+0000 7f8633fff700 1 -- 172.21.15.16:0/3922425590 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f864400ed30 con 0x7f864c06f450 2023-05-20T01:18:21.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.182+0000 7f8633fff700 1 -- 172.21.15.16:0/3922425590 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8644014420 con 0x7f864c06f450 2023-05-20T01:18:21.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.182+0000 7f8633fff700 1 -- 172.21.15.16:0/3922425590 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86440147d0 con 0x7f864c06f450 2023-05-20T01:18:21.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.182+0000 7f865209d700 1 -- 172.21.15.16:0/3922425590 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f864c0cc1a0 con 0x7f864c06f450 2023-05-20T01:18:21.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.182+0000 7f865209d700 1 -- 172.21.15.16:0/3922425590 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f864c0721b0 con 0x7f864c06f450 2023-05-20T01:18:21.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.183+0000 7f865209d700 1 -- 172.21.15.16:0/3922425590 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f864c0cbe50 con 0x7f864c06f450 2023-05-20T01:18:21.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.183+0000 7f8633fff700 1 -- 172.21.15.16:0/3922425590 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8644014930 con 0x7f864c06f450 2023-05-20T01:18:21.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.184+0000 7f8633fff700 1 --2- 172.21.15.16:0/3922425590 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f863806ace0 0x7f863806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:21.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.184+0000 7f8633fff700 1 -- 172.21.15.16:0/3922425590 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6844+0+0 (secure 0 0 0) 0x7f8644089ed0 con 0x7f864c06f450 2023-05-20T01:18:21.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.184+0000 7f864b7fe700 1 --2- 172.21.15.16:0/3922425590 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f863806ace0 0x7f863806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:21.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.184+0000 7f864b7fe700 1 --2- 172.21.15.16:0/3922425590 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f863806ace0 0x7f863806d190 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f863c006b20 tx=0x7f863c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:21.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.189+0000 7f8633fff700 1 -- 172.21.15.16:0/3922425590 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f8644054c90 con 0x7f864c06f450 2023-05-20T01:18:21.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.388+0000 7f865209d700 1 -- 172.21.15.16:0/3922425590 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f864c0747b0 con 0x7f864c06f450 2023-05-20T01:18:21.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.390+0000 7f8633fff700 1 -- 172.21.15.16:0/3922425590 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) v1 ==== 74+0+20281 (secure 0 0 0) 0x7f86440582b0 con 0x7f864c06f450 2023-05-20T01:18:21.394 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:18:21.394 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":45,"fsid":"c2245cc5-ecab-4087-a59b-bc81db901676","created":"2023-05-20T01:16:09.549230+0000","modified":"2023-05-20T01:18:17.399869+0000","last_up_change":"2023-05-20T01:16:19.788624+0000","last_in_change":"2023-05-20T01:16:12.092160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-05-20T01:16:20.934366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-05-20T01:16:21.372294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-05-20T01:17:44.628755+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-05-20T01:17:45.968584+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data_extra","create_time":"2023-05-20T01:17:54.915781+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"317a83cd-19c8-4958-a301-6087ef342289","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6825","nonce":3923634179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6827","nonce":3923634179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6831","nonce":3923634179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6829","nonce":3923634179}]},"public_addr":"172.21.15.16:6825/3923634179","cluster_addr":"172.21.15.16:6827/3923634179","heartbeat_back_addr":"172.21.15.16:6831/3923634179","heartbeat_front_addr":"172.21.15.16:6829/3923634179","state":["exists","up"]},{"osd":1,"uuid":"5581046f-8219-4089-8d7d-b7846399505e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6809","nonce":3860264220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6811","nonce":3860264220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6815","nonce":3860264220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6813","nonce":3860264220}]},"public_addr":"172.21.15.16:6809/3860264220","cluster_addr":"172.21.15.16:6811/3860264220","heartbeat_back_addr":"172.21.15.16:6815/3860264220","heartbeat_front_addr":"172.21.15.16:6813/3860264220","state":["exists","up"]},{"osd":2,"uuid":"072e9e66-4133-478e-8d4a-64d46c2963f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6817","nonce":293187030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6819","nonce":293187030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6823","nonce":293187030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6821","nonce":293187030}]},"public_addr":"172.21.15.16:6817/293187030","cluster_addr":"172.21.15.16:6819/293187030","heartbeat_back_addr":"172.21.15.16:6823/293187030","heartbeat_front_addr":"172.21.15.16:6821/293187030","state":["exists","up"]},{"osd":3,"uuid":"11e1bba6-00f8-48dd-9028-d8d716b7f145","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6801","nonce":3669781890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6803","nonce":3669781890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6807","nonce":3669781890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6805","nonce":3669781890}]},"public_addr":"172.21.15.16:6801/3669781890","cluster_addr":"172.21.15.16:6803/3669781890","heartbeat_back_addr":"172.21.15.16:6807/3669781890","heartbeat_front_addr":"172.21.15.16:6805/3669781890","state":["exists","up"]},{"osd":4,"uuid":"b8909c3a-cf70-46cb-8d0b-009c707cf9d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6819","nonce":506317223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6823","nonce":506317223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6829","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6831","nonce":506317223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6827","nonce":506317223}]},"public_addr":"172.21.15.132:6819/506317223","cluster_addr":"172.21.15.132:6823/506317223","heartbeat_back_addr":"172.21.15.132:6831/506317223","heartbeat_front_addr":"172.21.15.132:6827/506317223","state":["exists","up"]},{"osd":5,"uuid":"b1818304-4583-47bf-a6ec-0bfbb5166233","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6818","nonce":1844689995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6822","nonce":1844689995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6830","nonce":1844689995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6826","nonce":1844689995}]},"public_addr":"172.21.15.132:6818/1844689995","cluster_addr":"172.21.15.132:6822/1844689995","heartbeat_back_addr":"172.21.15.132:6830/1844689995","heartbeat_front_addr":"172.21.15.132:6826/1844689995","state":["exists","up"]},{"osd":6,"uuid":"d37f4691-eed1-4a93-8818-18354f03f4a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6801","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6802","nonce":727581123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6805","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6807","nonce":727581123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6813","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6815","nonce":727581123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6811","nonce":727581123}]},"public_addr":"172.21.15.132:6802/727581123","cluster_addr":"172.21.15.132:6807/727581123","heartbeat_back_addr":"172.21.15.132:6815/727581123","heartbeat_front_addr":"172.21.15.132:6811/727581123","state":["exists","up"]},{"osd":7,"uuid":"4f712f62-5d1d-4816-8042-03209144f8b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6803","nonce":466001149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6806","nonce":466001149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6814","nonce":466001149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6810","nonce":466001149}]},"public_addr":"172.21.15.132:6803/466001149","cluster_addr":"172.21.15.132:6806/466001149","heartbeat_back_addr":"172.21.15.132:6814/466001149","heartbeat_front_addr":"172.21.15.132:6810/466001149","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.338863+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.278680+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305694+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.246835+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.276237+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.304441+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305421+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.267050+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.9","mappings":[{"from":0,"to":4}]},{"pgid":"7.1b","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-20T01:18:21.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.392+0000 7f865209d700 1 -- 172.21.15.16:0/3922425590 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f863806ace0 msgr2=0x7f863806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:21.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.392+0000 7f865209d700 1 --2- 172.21.15.16:0/3922425590 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f863806ace0 0x7f863806d190 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f863c006b20 tx=0x7f863c008040 comp rx=0 tx=0).stop 2023-05-20T01:18:21.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.392+0000 7f865209d700 1 -- 172.21.15.16:0/3922425590 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f864c06f450 msgr2=0x7f864c0b1b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:21.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.392+0000 7f865209d700 1 --2- 172.21.15.16:0/3922425590 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f864c06f450 0x7f864c0b1b20 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f8644007e60 tx=0x7f864400e3c0 comp rx=0 tx=0).stop 2023-05-20T01:18:21.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.392+0000 7f865209d700 1 -- 172.21.15.16:0/3922425590 shutdown_connections 2023-05-20T01:18:21.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.392+0000 7f865209d700 1 --2- 172.21.15.16:0/3922425590 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f864c06f450 0x7f864c0b1b20 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:21.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.392+0000 7f865209d700 1 --2- 172.21.15.16:0/3922425590 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f863806ace0 0x7f863806d190 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:21.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.392+0000 7f865209d700 1 --2- 172.21.15.16:0/3922425590 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f864c06a510 0x7f864c154620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:21.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.392+0000 7f865209d700 1 --2- 172.21.15.16:0/3922425590 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f864c06d4b0 0x7f864c0b3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:21.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.392+0000 7f865209d700 1 -- 172.21.15.16:0/3922425590 >> 172.21.15.16:0/3922425590 conn(0x7f864c0cffb0 msgr2=0x7f864c15b740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:18:21.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.392+0000 7f865209d700 1 -- 172.21.15.16:0/3922425590 shutdown_connections 2023-05-20T01:18:21.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.392+0000 7f865209d700 1 -- 172.21.15.16:0/3922425590 wait complete. 2023-05-20T01:18:21.408 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-05-20T01:16:20.934366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-05-20T01:16:21.372294+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2023-05-20T01:17:44.628755+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '39', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2023-05-20T01:17:45.968584+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '40', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data_extra', 'create_time': '2023-05-20T01:17:54.915781+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '43', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-05-20T01:18:21.409 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-05-20T01:18:21.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.532+0000 7f44eebec700 1 Processor -- start 2023-05-20T01:18:21.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.532+0000 7f44eebec700 1 -- start start 2023-05-20T01:18:21.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.533+0000 7f44eebec700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f44e80b1e50 0x7f44e806b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:21.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.533+0000 7f44eebec700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f44e806bb60 0x7f44e80cc150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:21.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.533+0000 7f44eebec700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44e806bfc0 con 0x7f44e815cd20 2023-05-20T01:18:21.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.533+0000 7f44eebec700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44e80cb700 con 0x7f44e806bb60 2023-05-20T01:18:21.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.533+0000 7f44eebec700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44e80b2430 con 0x7f44e80b1e50 2023-05-20T01:18:21.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.533+0000 7f44ec988700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f44e815cd20 0x7f44e80b14c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:43762/0 (socket says 172.21.15.16:43762) 2023-05-20T01:18:21.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.533+0000 7f44ec988700 1 -- 172.21.15.16:0/4031682689 learned_addr learned my addr 172.21.15.16:0/4031682689 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:18:21.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.534+0000 7f44ed189700 1 --2- 172.21.15.16:0/4031682689 >> v2:172.21.15.132:3300/0 conn(0x7f44e806bb60 0x7f44e80cc150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:21.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.534+0000 7f44e7fff700 1 --2- 172.21.15.16:0/4031682689 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f44e80b1e50 0x7f44e806b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:21.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.534+0000 7f44e77fe700 1 -- 172.21.15.16:0/4031682689 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3432573639 0 0) 0x7f44e806bfc0 con 0x7f44e815cd20 2023-05-20T01:18:21.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.534+0000 7f44e77fe700 1 -- 172.21.15.16:0/4031682689 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44d80035b0 con 0x7f44e815cd20 2023-05-20T01:18:21.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.534+0000 7f44e77fe700 1 -- 172.21.15.16:0/4031682689 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4013874683 0 0) 0x7f44d80035b0 con 0x7f44e815cd20 2023-05-20T01:18:21.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.534+0000 7f44e77fe700 1 -- 172.21.15.16:0/4031682689 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f44e80b1e50 msgr2=0x7f44e806b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:18:21.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.534+0000 7f44e77fe700 1 --2- 172.21.15.16:0/4031682689 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f44e80b1e50 0x7f44e806b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:21.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.534+0000 7f44e77fe700 1 -- 172.21.15.16:0/4031682689 >> v2:172.21.15.132:3300/0 conn(0x7f44e806bb60 msgr2=0x7f44e80cc150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:18:21.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.534+0000 7f44e77fe700 1 --2- 172.21.15.16:0/4031682689 >> v2:172.21.15.132:3300/0 conn(0x7f44e806bb60 0x7f44e80cc150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:21.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.534+0000 7f44e77fe700 1 -- 172.21.15.16:0/4031682689 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44e80b2710 con 0x7f44e815cd20 2023-05-20T01:18:21.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.534+0000 7f44e77fe700 1 -- 172.21.15.16:0/4031682689 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7f44dc001670 con 0x7f44e815cd20 2023-05-20T01:18:21.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.534+0000 7f44e77fe700 1 -- 172.21.15.16:0/4031682689 >> v1:172.21.15.16:6789/0 conn(0x7f44e815cd20 legacy=0x7f44e80b14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:21.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.534+0000 7f44e77fe700 1 --2- 172.21.15.16:0/4031682689 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f44d8003a00 0x7f44d8003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:21.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.534+0000 7f44e77fe700 1 --2- 172.21.15.16:0/4031682689 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f44e80b1e50 0x7f44d8004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:21.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.534+0000 7f44e77fe700 1 --2- 172.21.15.16:0/4031682689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f44d8004d90 0x7f44d8005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:21.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.534+0000 7f44e77fe700 1 -- 172.21.15.16:0/4031682689 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44e80cb700 con 0x7f44d8004d90 2023-05-20T01:18:21.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.534+0000 7f44e77fe700 1 -- 172.21.15.16:0/4031682689 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44e80b2430 con 0x7f44d8003a00 2023-05-20T01:18:21.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.534+0000 7f44e77fe700 1 -- 172.21.15.16:0/4031682689 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44d800ba10 con 0x7f44e80b1e50 2023-05-20T01:18:21.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.534+0000 7f44ed189700 1 --2- 172.21.15.16:0/4031682689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f44d8004d90 0x7f44d8005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:21.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.534+0000 7f44e7fff700 1 --2- 172.21.15.16:0/4031682689 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f44e80b1e50 0x7f44d8004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:21.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.534+0000 7f44ec988700 1 --2- 172.21.15.16:0/4031682689 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f44d8003a00 0x7f44d8003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:21.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.534+0000 7f44ed189700 1 -- 172.21.15.16:0/4031682689 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f44e80b1e50 msgr2=0x7f44d8004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:18:21.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.534+0000 7f44ed189700 1 --2- 172.21.15.16:0/4031682689 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f44e80b1e50 0x7f44d8004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:21.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.534+0000 7f44ed189700 1 -- 172.21.15.16:0/4031682689 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f44d8003a00 msgr2=0x7f44d8003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:18:21.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.534+0000 7f44ed189700 1 --2- 172.21.15.16:0/4031682689 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f44d8003a00 0x7f44d8003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:21.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.535+0000 7f44ed189700 1 -- 172.21.15.16:0/4031682689 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44d80035b0 con 0x7f44d8004d90 2023-05-20T01:18:21.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.535+0000 7f44ed189700 1 --2- 172.21.15.16:0/4031682689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f44d8004d90 0x7f44d8005200 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f44e0009fc0 tx=0x7f44d8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c4c2c3968a7fcdd server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:21.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.535+0000 7f44e77fe700 1 -- 172.21.15.16:0/4031682689 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44e000ebd0 con 0x7f44d8004d90 2023-05-20T01:18:21.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.535+0000 7f44e77fe700 1 -- 172.21.15.16:0/4031682689 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44e000f040 con 0x7f44d8004d90 2023-05-20T01:18:21.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.535+0000 7f44e77fe700 1 -- 172.21.15.16:0/4031682689 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44e0014480 con 0x7f44d8004d90 2023-05-20T01:18:21.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.535+0000 7f44eebec700 1 -- 172.21.15.16:0/4031682689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f44d8004d90 msgr2=0x7f44d8005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:21.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.535+0000 7f44eebec700 1 --2- 172.21.15.16:0/4031682689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f44d8004d90 0x7f44d8005200 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f44e0009fc0 tx=0x7f44d8005880 comp rx=0 tx=0).stop 2023-05-20T01:18:21.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.535+0000 7f44ed189700 1 -- 172.21.15.16:0/4031682689 reap_dead start 2023-05-20T01:18:21.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.535+0000 7f44eebec700 1 -- 172.21.15.16:0/4031682689 shutdown_connections 2023-05-20T01:18:21.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.535+0000 7f44eebec700 1 -- 172.21.15.16:0/4031682689 >> 172.21.15.16:0/4031682689 conn(0x7f44e806ae30 msgr2=0x7f44e809a400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:18:21.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.535+0000 7f44eebec700 1 -- 172.21.15.16:0/4031682689 shutdown_connections 2023-05-20T01:18:21.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.535+0000 7f44eebec700 1 -- 172.21.15.16:0/4031682689 wait complete. 2023-05-20T01:18:21.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.537+0000 7f44eebec700 1 Processor -- start 2023-05-20T01:18:21.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.537+0000 7f44eebec700 1 -- start start 2023-05-20T01:18:21.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.538+0000 7f44eebec700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f44e8159070 0x7f44e8159440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:21.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.538+0000 7f44eebec700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f44e80c4df0 0x7f44e80c71c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:21.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.538+0000 7f44eebec700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f44e80c7700 0x7f44e81f5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:21.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.538+0000 7f44eebec700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44e80c7ba0 con 0x7f44e80c4df0 2023-05-20T01:18:21.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.538+0000 7f44eebec700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44e8159af0 con 0x7f44e80c7700 2023-05-20T01:18:21.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.538+0000 7f44eebec700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44e8159dd0 con 0x7f44e8159070 2023-05-20T01:18:21.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.538+0000 7f44e7fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f44e80c4df0 0x7f44e80c71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:21.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.538+0000 7f44e7fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f44e80c4df0 0x7f44e80c71c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60480/0 (socket says 172.21.15.16:60480) 2023-05-20T01:18:21.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.538+0000 7f44e7fff700 1 -- 172.21.15.16:0/2316829715 learned_addr learned my addr 172.21.15.16:0/2316829715 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:18:21.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.538+0000 7f44ed189700 1 --2- 172.21.15.16:0/2316829715 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f44e80c7700 0x7f44e81f5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:21.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.538+0000 7f44ec988700 1 --2- 172.21.15.16:0/2316829715 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f44e8159070 0x7f44e8159440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:21.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.538+0000 7f44e7fff700 1 -- 172.21.15.16:0/2316829715 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f44e8159070 msgr2=0x7f44e8159440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:21.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.538+0000 7f44e7fff700 1 --2- 172.21.15.16:0/2316829715 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f44e8159070 0x7f44e8159440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:21.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.538+0000 7f44e7fff700 1 -- 172.21.15.16:0/2316829715 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f44e80c7700 msgr2=0x7f44e81f5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:21.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.538+0000 7f44e7fff700 1 --2- 172.21.15.16:0/2316829715 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f44e80c7700 0x7f44e81f5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:21.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.538+0000 7f44e7fff700 1 -- 172.21.15.16:0/2316829715 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44e80cbc30 con 0x7f44e80c4df0 2023-05-20T01:18:21.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.538+0000 7f44e7fff700 1 --2- 172.21.15.16:0/2316829715 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f44e80c4df0 0x7f44e80c71c0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f44d4011aa0 tx=0x7f44d4011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:21.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.538+0000 7f44e57fa700 1 -- 172.21.15.16:0/2316829715 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44d400f980 con 0x7f44e80c4df0 2023-05-20T01:18:21.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.538+0000 7f44e57fa700 1 -- 172.21.15.16:0/2316829715 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44d400fae0 con 0x7f44e80c4df0 2023-05-20T01:18:21.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.538+0000 7f44eebec700 1 -- 172.21.15.16:0/2316829715 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44e80cb950 con 0x7f44e80c4df0 2023-05-20T01:18:21.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.538+0000 7f44e57fa700 1 -- 172.21.15.16:0/2316829715 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44d400fe90 con 0x7f44e80c4df0 2023-05-20T01:18:21.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.538+0000 7f44eebec700 1 -- 172.21.15.16:0/2316829715 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44e815d0f0 con 0x7f44e80c4df0 2023-05-20T01:18:21.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.540+0000 7f44eebec700 1 -- 172.21.15.16:0/2316829715 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44e80d1f60 con 0x7f44e80c4df0 2023-05-20T01:18:21.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.540+0000 7f44e57fa700 1 -- 172.21.15.16:0/2316829715 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f44d4014440 con 0x7f44e80c4df0 2023-05-20T01:18:21.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.540+0000 7f44e57fa700 1 --2- 172.21.15.16:0/2316829715 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f44cc06ace0 0x7f44cc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:21.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.540+0000 7f44e57fa700 1 -- 172.21.15.16:0/2316829715 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6844+0+0 (secure 0 0 0) 0x7f44d4089fb0 con 0x7f44e80c4df0 2023-05-20T01:18:21.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.540+0000 7f44ec988700 1 --2- 172.21.15.16:0/2316829715 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f44cc06ace0 0x7f44cc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:21.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.541+0000 7f44ec988700 1 --2- 172.21.15.16:0/2316829715 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f44cc06ace0 0x7f44cc06d190 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f44dc0048d0 tx=0x7f44dc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:21.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.545+0000 7f44e57fa700 1 -- 172.21.15.16:0/2316829715 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f44d40bc2d0 con 0x7f44e80c4df0 2023-05-20T01:18:21.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.742+0000 7f44eebec700 1 -- 172.21.15.16:0/2316829715 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f44e80747b0 con 0x7f44e80c4df0 2023-05-20T01:18:21.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.743+0000 7f44e57fa700 1 -- 172.21.15.16:0/2316829715 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v45) v1 ==== 92+0+10 (secure 0 0 0) 0x7f44d4022020 con 0x7f44e80c4df0 2023-05-20T01:18:21.747 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 8 2023-05-20T01:18:21.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.745+0000 7f44eebec700 1 -- 172.21.15.16:0/2316829715 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f44cc06ace0 msgr2=0x7f44cc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:21.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.745+0000 7f44eebec700 1 --2- 172.21.15.16:0/2316829715 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f44cc06ace0 0x7f44cc06d190 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f44dc0048d0 tx=0x7f44dc009040 comp rx=0 tx=0).stop 2023-05-20T01:18:21.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.745+0000 7f44eebec700 1 -- 172.21.15.16:0/2316829715 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f44e80c4df0 msgr2=0x7f44e80c71c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:21.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.745+0000 7f44eebec700 1 --2- 172.21.15.16:0/2316829715 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f44e80c4df0 0x7f44e80c71c0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f44d4011aa0 tx=0x7f44d4011e60 comp rx=0 tx=0).stop 2023-05-20T01:18:21.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.745+0000 7f44eebec700 1 -- 172.21.15.16:0/2316829715 shutdown_connections 2023-05-20T01:18:21.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.745+0000 7f44eebec700 1 --2- 172.21.15.16:0/2316829715 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f44e80c7700 0x7f44e81f5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:21.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.745+0000 7f44eebec700 1 --2- 172.21.15.16:0/2316829715 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f44cc06ace0 0x7f44cc06d190 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:21.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.745+0000 7f44eebec700 1 --2- 172.21.15.16:0/2316829715 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f44e8159070 0x7f44e8159440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:21.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.745+0000 7f44eebec700 1 --2- 172.21.15.16:0/2316829715 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f44e80c4df0 0x7f44e80c71c0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:21.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.745+0000 7f44eebec700 1 -- 172.21.15.16:0/2316829715 >> 172.21.15.16:0/2316829715 conn(0x7f44e806ae30 msgr2=0x7f44e806fa30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:18:21.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.745+0000 7f44eebec700 1 -- 172.21.15.16:0/2316829715 shutdown_connections 2023-05-20T01:18:21.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.745+0000 7f44eebec700 1 -- 172.21.15.16:0/2316829715 wait complete. 2023-05-20T01:18:21.761 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-05-20T01:18:21.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.885+0000 7fad2f6d2700 1 Processor -- start 2023-05-20T01:18:21.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.885+0000 7fad2f6d2700 1 -- start start 2023-05-20T01:18:21.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.886+0000 7fad2f6d2700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fad28072010 0x7fad280711b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:21.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.886+0000 7fad2f6d2700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fad280d0b80 0x7fad280716f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:21.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.886+0000 7fad2f6d2700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad2806cbd0 con 0x7fad280af4d0 2023-05-20T01:18:21.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.886+0000 7fad2f6d2700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad280cadb0 con 0x7fad280d0b80 2023-05-20T01:18:21.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.886+0000 7fad2f6d2700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad280af8a0 con 0x7fad28072010 2023-05-20T01:18:21.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.886+0000 7fad2dc6f700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fad280d0b80 0x7fad280716f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:21.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.886+0000 7fad2d46e700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fad280af4d0 0x7fad280ca6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:43782/0 (socket says 172.21.15.16:43782) 2023-05-20T01:18:21.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.886+0000 7fad2d46e700 1 -- 172.21.15.16:0/941797330 learned_addr learned my addr 172.21.15.16:0/941797330 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:18:21.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.886+0000 7fad2cc6d700 1 --2- 172.21.15.16:0/941797330 >> v2:172.21.15.132:3300/0 conn(0x7fad28072010 0x7fad280711b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:21.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.886+0000 7fad2dc6f700 1 -- 172.21.15.16:0/941797330 >> v2:172.21.15.132:3300/0 conn(0x7fad28072010 msgr2=0x7fad280711b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:18:21.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.886+0000 7fad2dc6f700 1 --2- 172.21.15.16:0/941797330 >> v2:172.21.15.132:3300/0 conn(0x7fad28072010 0x7fad280711b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:21.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.886+0000 7fad2dc6f700 1 -- 172.21.15.16:0/941797330 >> v1:172.21.15.16:6789/0 conn(0x7fad280af4d0 legacy=0x7fad280ca6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:21.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.886+0000 7fad1ffff700 1 -- 172.21.15.16:0/941797330 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4023218307 0 0) 0x7fad2806cbd0 con 0x7fad280af4d0 2023-05-20T01:18:21.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.887+0000 7fad2dc6f700 1 -- 172.21.15.16:0/941797330 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad2806ce20 con 0x7fad280d0b80 2023-05-20T01:18:21.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.887+0000 7fad2dc6f700 1 --2- 172.21.15.16:0/941797330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad280d0b80 0x7fad280716f0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fad280cf4a0 tx=0x7fad2000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=409bf464fb255b6b server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:21.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.887+0000 7fad1ffff700 1 -- 172.21.15.16:0/941797330 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad2000e070 con 0x7fad280d0b80 2023-05-20T01:18:21.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.887+0000 7fad1ffff700 1 -- 172.21.15.16:0/941797330 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad2000bbd0 con 0x7fad280d0b80 2023-05-20T01:18:21.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.887+0000 7fad1ffff700 1 -- 172.21.15.16:0/941797330 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad20012430 con 0x7fad280d0b80 2023-05-20T01:18:21.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.887+0000 7fad2f6d2700 1 -- 172.21.15.16:0/941797330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad280d0b80 msgr2=0x7fad280716f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:21.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.887+0000 7fad2f6d2700 1 --2- 172.21.15.16:0/941797330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad280d0b80 0x7fad280716f0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fad280cf4a0 tx=0x7fad2000b330 comp rx=0 tx=0).stop 2023-05-20T01:18:21.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.887+0000 7fad2f6d2700 1 -- 172.21.15.16:0/941797330 shutdown_connections 2023-05-20T01:18:21.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.887+0000 7fad2f6d2700 1 --2- 172.21.15.16:0/941797330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad280d0b80 0x7fad280716f0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:21.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.887+0000 7fad2f6d2700 1 --2- 172.21.15.16:0/941797330 >> v2:172.21.15.132:3300/0 conn(0x7fad28072010 0x7fad280711b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:21.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.887+0000 7fad2f6d2700 1 -- 172.21.15.16:0/941797330 >> 172.21.15.16:0/941797330 conn(0x7fad280cd760 msgr2=0x7fad280d0050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:18:21.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.888+0000 7fad2f6d2700 1 -- 172.21.15.16:0/941797330 shutdown_connections 2023-05-20T01:18:21.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.888+0000 7fad2f6d2700 1 -- 172.21.15.16:0/941797330 wait complete. 2023-05-20T01:18:21.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.889+0000 7fad2f6d2700 1 Processor -- start 2023-05-20T01:18:21.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.889+0000 7fad2f6d2700 1 -- start start 2023-05-20T01:18:21.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.889+0000 7fad2f6d2700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fad28072010 0x7fad281f2f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:21.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.889+0000 7fad2f6d2700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fad280af4d0 0x7fad28159130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:21.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.889+0000 7fad2f6d2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad280d0b80 0x7fad2806a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:21.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.889+0000 7fad2f6d2700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad280c7120 con 0x7fad280d0b80 2023-05-20T01:18:21.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.889+0000 7fad2f6d2700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad281f34c0 con 0x7fad28072010 2023-05-20T01:18:21.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.889+0000 7fad2f6d2700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad2815b1f0 con 0x7fad280af4d0 2023-05-20T01:18:21.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.890+0000 7fad2dc6f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad280d0b80 0x7fad2806a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:21.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.890+0000 7fad2dc6f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad280d0b80 0x7fad2806a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60496/0 (socket says 172.21.15.16:60496) 2023-05-20T01:18:21.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.890+0000 7fad2dc6f700 1 -- 172.21.15.16:0/3003465319 learned_addr learned my addr 172.21.15.16:0/3003465319 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:18:21.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.890+0000 7fad2d46e700 1 --2- 172.21.15.16:0/3003465319 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fad28072010 0x7fad281f2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:21.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.890+0000 7fad2dc6f700 1 -- 172.21.15.16:0/3003465319 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fad280af4d0 msgr2=0x7fad28159130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:21.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.890+0000 7fad2dc6f700 1 --2- 172.21.15.16:0/3003465319 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fad280af4d0 0x7fad28159130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:21.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.890+0000 7fad2dc6f700 1 -- 172.21.15.16:0/3003465319 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fad28072010 msgr2=0x7fad281f2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:21.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.890+0000 7fad2dc6f700 1 --2- 172.21.15.16:0/3003465319 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fad28072010 0x7fad281f2f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:21.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.890+0000 7fad2dc6f700 1 -- 172.21.15.16:0/3003465319 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad20009a70 con 0x7fad280d0b80 2023-05-20T01:18:21.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.890+0000 7fad2d46e700 1 --2- 172.21.15.16:0/3003465319 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fad28072010 0x7fad281f2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:18:21.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.890+0000 7fad2dc6f700 1 --2- 172.21.15.16:0/3003465319 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad280d0b80 0x7fad2806a510 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fad20013000 tx=0x7fad20004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:21.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.890+0000 7fad1dffb700 1 -- 172.21.15.16:0/3003465319 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad20012430 con 0x7fad280d0b80 2023-05-20T01:18:21.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.890+0000 7fad1dffb700 1 -- 172.21.15.16:0/3003465319 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad20020070 con 0x7fad280d0b80 2023-05-20T01:18:21.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.890+0000 7fad2f6d2700 1 -- 172.21.15.16:0/3003465319 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad2806d3e0 con 0x7fad280d0b80 2023-05-20T01:18:21.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.890+0000 7fad1dffb700 1 -- 172.21.15.16:0/3003465319 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad20004280 con 0x7fad280d0b80 2023-05-20T01:18:21.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.891+0000 7fad2f6d2700 1 -- 172.21.15.16:0/3003465319 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad2806d100 con 0x7fad280d0b80 2023-05-20T01:18:21.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.892+0000 7fad2f6d2700 1 -- 172.21.15.16:0/3003465319 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad2806d6c0 con 0x7fad280d0b80 2023-05-20T01:18:21.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.892+0000 7fad1dffb700 1 -- 172.21.15.16:0/3003465319 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fad200043e0 con 0x7fad280d0b80 2023-05-20T01:18:21.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.892+0000 7fad1dffb700 1 --2- 172.21.15.16:0/3003465319 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fad0c06ace0 0x7fad0c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:21.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.893+0000 7fad1dffb700 1 -- 172.21.15.16:0/3003465319 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6844+0+0 (secure 0 0 0) 0x7fad20022070 con 0x7fad280d0b80 2023-05-20T01:18:21.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.893+0000 7fad2d46e700 1 --2- 172.21.15.16:0/3003465319 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fad0c06ace0 0x7fad0c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:21.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.893+0000 7fad2d46e700 1 --2- 172.21.15.16:0/3003465319 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fad0c06ace0 0x7fad0c06d190 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fad18007fd0 tx=0x7fad18009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:21.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:21.897+0000 7fad1dffb700 1 -- 172.21.15.16:0/3003465319 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fad20056900 con 0x7fad280d0b80 2023-05-20T01:18:22.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.093+0000 7fad2f6d2700 1 -- 172.21.15.16:0/3003465319 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fad280c7290 con 0x7fad280d0b80 2023-05-20T01:18:22.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.093+0000 7fad1dffb700 1 -- 172.21.15.16:0/3003465319 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v45) v1 ==== 93+0+10 (secure 0 0 0) 0x7fad280c7290 con 0x7fad280d0b80 2023-05-20T01:18:22.097 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-05-20T01:18:22.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.095+0000 7fad2f6d2700 1 -- 172.21.15.16:0/3003465319 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fad0c06ace0 msgr2=0x7fad0c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:22.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.095+0000 7fad2f6d2700 1 --2- 172.21.15.16:0/3003465319 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fad0c06ace0 0x7fad0c06d190 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fad18007fd0 tx=0x7fad18009040 comp rx=0 tx=0).stop 2023-05-20T01:18:22.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.095+0000 7fad2f6d2700 1 -- 172.21.15.16:0/3003465319 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad280d0b80 msgr2=0x7fad2806a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:22.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.095+0000 7fad2f6d2700 1 --2- 172.21.15.16:0/3003465319 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad280d0b80 0x7fad2806a510 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fad20013000 tx=0x7fad20004ad0 comp rx=0 tx=0).stop 2023-05-20T01:18:22.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.095+0000 7fad2f6d2700 1 -- 172.21.15.16:0/3003465319 shutdown_connections 2023-05-20T01:18:22.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.095+0000 7fad2f6d2700 1 --2- 172.21.15.16:0/3003465319 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad280d0b80 0x7fad2806a510 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:22.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.095+0000 7fad2f6d2700 1 --2- 172.21.15.16:0/3003465319 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fad28072010 0x7fad281f2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:22.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.095+0000 7fad2f6d2700 1 --2- 172.21.15.16:0/3003465319 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fad0c06ace0 0x7fad0c06d190 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:22.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.095+0000 7fad2f6d2700 1 --2- 172.21.15.16:0/3003465319 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fad280af4d0 0x7fad28159130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:22.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.095+0000 7fad2f6d2700 1 -- 172.21.15.16:0/3003465319 >> 172.21.15.16:0/3003465319 conn(0x7fad280cd760 msgr2=0x7fad280caef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:18:22.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.095+0000 7fad2f6d2700 1 -- 172.21.15.16:0/3003465319 shutdown_connections 2023-05-20T01:18:22.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.095+0000 7fad2f6d2700 1 -- 172.21.15.16:0/3003465319 wait complete. 2023-05-20T01:18:22.111 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-05-20T01:18:22.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.235+0000 7fa0c0a8f700 1 Processor -- start 2023-05-20T01:18:22.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.235+0000 7fa0c0a8f700 1 -- start start 2023-05-20T01:18:22.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.236+0000 7fa0c0a8f700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa0bc0cd070 0x7fa0bc06e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:22.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.236+0000 7fa0c0a8f700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fa0bc0710d0 0x7fa0bc0cabb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:22.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.236+0000 7fa0c0a8f700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0bc072430 con 0x7fa0bc15cf40 2023-05-20T01:18:22.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.236+0000 7fa0c0a8f700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0bc071570 con 0x7fa0bc0710d0 2023-05-20T01:18:22.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.236+0000 7fa0c0a8f700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0bc071820 con 0x7fa0bc0cd070 2023-05-20T01:18:22.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.236+0000 7fa0ba59c700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fa0bc15cf40 0x7fa0bc0cc960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:43796/0 (socket says 172.21.15.16:43796) 2023-05-20T01:18:22.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.236+0000 7fa0ba59c700 1 -- 172.21.15.16:0/2807140559 learned_addr learned my addr 172.21.15.16:0/2807140559 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:18:22.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.236+0000 7fa0b9d9b700 1 --2- 172.21.15.16:0/2807140559 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa0bc0cd070 0x7fa0bc06e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:22.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.236+0000 7fa0bad9d700 1 --2- 172.21.15.16:0/2807140559 >> v2:172.21.15.132:3300/0 conn(0x7fa0bc0710d0 0x7fa0bc0cabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:22.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.236+0000 7fa0b959a700 1 -- 172.21.15.16:0/2807140559 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 958385840 0 0) 0x7fa0bc072430 con 0x7fa0bc15cf40 2023-05-20T01:18:22.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.236+0000 7fa0b959a700 1 -- 172.21.15.16:0/2807140559 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0a40035b0 con 0x7fa0bc15cf40 2023-05-20T01:18:22.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.236+0000 7fa0b9d9b700 1 -- 172.21.15.16:0/2807140559 >> v2:172.21.15.132:3300/0 conn(0x7fa0bc0710d0 msgr2=0x7fa0bc0cabb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:18:22.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.236+0000 7fa0b9d9b700 1 --2- 172.21.15.16:0/2807140559 >> v2:172.21.15.132:3300/0 conn(0x7fa0bc0710d0 0x7fa0bc0cabb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:22.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.236+0000 7fa0b9d9b700 1 -- 172.21.15.16:0/2807140559 >> v1:172.21.15.16:6789/0 conn(0x7fa0bc15cf40 legacy=0x7fa0bc0cc960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:22.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.236+0000 7fa0b9d9b700 1 -- 172.21.15.16:0/2807140559 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0bc071b00 con 0x7fa0bc0cd070 2023-05-20T01:18:22.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.237+0000 7fa0b9d9b700 1 --2- 172.21.15.16:0/2807140559 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa0bc0cd070 0x7fa0bc06e020 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa0b0009840 tx=0x7fa0b0009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f95ac86308cc6787 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:22.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.237+0000 7fa0b959a700 1 -- 172.21.15.16:0/2807140559 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b0011830 con 0x7fa0bc0cd070 2023-05-20T01:18:22.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.237+0000 7fa0b959a700 1 -- 172.21.15.16:0/2807140559 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0b0011990 con 0x7fa0bc0cd070 2023-05-20T01:18:22.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.237+0000 7fa0b959a700 1 -- 172.21.15.16:0/2807140559 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b0011d00 con 0x7fa0bc0cd070 2023-05-20T01:18:22.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.237+0000 7fa0c0a8f700 1 -- 172.21.15.16:0/2807140559 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa0bc0cd070 msgr2=0x7fa0bc06e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:22.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.237+0000 7fa0c0a8f700 1 --2- 172.21.15.16:0/2807140559 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa0bc0cd070 0x7fa0bc06e020 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa0b0009840 tx=0x7fa0b0009b50 comp rx=0 tx=0).stop 2023-05-20T01:18:22.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.237+0000 7fa0c0a8f700 1 -- 172.21.15.16:0/2807140559 shutdown_connections 2023-05-20T01:18:22.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.237+0000 7fa0c0a8f700 1 --2- 172.21.15.16:0/2807140559 >> v2:172.21.15.132:3300/0 conn(0x7fa0bc0710d0 0x7fa0bc0cabb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:22.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.237+0000 7fa0c0a8f700 1 --2- 172.21.15.16:0/2807140559 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa0bc0cd070 0x7fa0bc06e020 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:22.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.237+0000 7fa0c0a8f700 1 -- 172.21.15.16:0/2807140559 >> 172.21.15.16:0/2807140559 conn(0x7fa0bc06c260 msgr2=0x7fa0bc0cffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:18:22.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.237+0000 7fa0c0a8f700 1 -- 172.21.15.16:0/2807140559 shutdown_connections 2023-05-20T01:18:22.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.237+0000 7fa0c0a8f700 1 -- 172.21.15.16:0/2807140559 wait complete. 2023-05-20T01:18:22.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.239+0000 7fa0c0a8f700 1 Processor -- start 2023-05-20T01:18:22.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.239+0000 7fa0c0a8f700 1 -- start start 2023-05-20T01:18:22.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.240+0000 7fa0c0a8f700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa0bc0710d0 0x7fa0bc1ef510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:22.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.240+0000 7fa0c0a8f700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa0bc0cd070 0x7fa0bc159a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:22.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.240+0000 7fa0c0a8f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0bc15cf40 0x7fa0bc1f0d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:22.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.240+0000 7fa0c0a8f700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0bc1ef0f0 con 0x7fa0bc15cf40 2023-05-20T01:18:22.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.240+0000 7fa0c0a8f700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0bc0c83a0 con 0x7fa0bc0cd070 2023-05-20T01:18:22.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.240+0000 7fa0c0a8f700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0bc0c8680 con 0x7fa0bc0710d0 2023-05-20T01:18:22.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.240+0000 7fa0bad9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0bc15cf40 0x7fa0bc1f0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:22.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.240+0000 7fa0bad9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0bc15cf40 0x7fa0bc1f0d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60510/0 (socket says 172.21.15.16:60510) 2023-05-20T01:18:22.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.240+0000 7fa0bad9d700 1 -- 172.21.15.16:0/3237373245 learned_addr learned my addr 172.21.15.16:0/3237373245 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:18:22.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.240+0000 7fa0b9d9b700 1 --2- 172.21.15.16:0/3237373245 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa0bc0cd070 0x7fa0bc159a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:22.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.240+0000 7fa0ba59c700 1 --2- 172.21.15.16:0/3237373245 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa0bc0710d0 0x7fa0bc1ef510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:22.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.240+0000 7fa0bad9d700 1 -- 172.21.15.16:0/3237373245 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa0bc0710d0 msgr2=0x7fa0bc1ef510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:22.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.240+0000 7fa0bad9d700 1 --2- 172.21.15.16:0/3237373245 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa0bc0710d0 0x7fa0bc1ef510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:22.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.240+0000 7fa0bad9d700 1 -- 172.21.15.16:0/3237373245 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa0bc0cd070 msgr2=0x7fa0bc159a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:22.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.240+0000 7fa0bad9d700 1 --2- 172.21.15.16:0/3237373245 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa0bc0cd070 0x7fa0bc159a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:22.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.240+0000 7fa0bad9d700 1 -- 172.21.15.16:0/3237373245 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0b00094a0 con 0x7fa0bc15cf40 2023-05-20T01:18:22.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.240+0000 7fa0bad9d700 1 --2- 172.21.15.16:0/3237373245 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0bc15cf40 0x7fa0bc1f0d40 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fa0b400efc0 tx=0x7fa0b400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:22.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.240+0000 7fa0aaffd700 1 -- 172.21.15.16:0/3237373245 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b400cd40 con 0x7fa0bc15cf40 2023-05-20T01:18:22.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.240+0000 7fa0aaffd700 1 -- 172.21.15.16:0/3237373245 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0b400f040 con 0x7fa0bc15cf40 2023-05-20T01:18:22.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.240+0000 7fa0c0a8f700 1 -- 172.21.15.16:0/3237373245 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0bc0720c0 con 0x7fa0bc15cf40 2023-05-20T01:18:22.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.240+0000 7fa0aaffd700 1 -- 172.21.15.16:0/3237373245 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b40146c0 con 0x7fa0bc15cf40 2023-05-20T01:18:22.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.241+0000 7fa0c0a8f700 1 -- 172.21.15.16:0/3237373245 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0bc071de0 con 0x7fa0bc15cf40 2023-05-20T01:18:22.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.242+0000 7fa0c0a8f700 1 -- 172.21.15.16:0/3237373245 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0bc0cb0f0 con 0x7fa0bc15cf40 2023-05-20T01:18:22.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.242+0000 7fa0aaffd700 1 -- 172.21.15.16:0/3237373245 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa0b4014820 con 0x7fa0bc15cf40 2023-05-20T01:18:22.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.242+0000 7fa0aaffd700 1 --2- 172.21.15.16:0/3237373245 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa09c06ace0 0x7fa09c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:22.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.242+0000 7fa0aaffd700 1 -- 172.21.15.16:0/3237373245 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6844+0+0 (secure 0 0 0) 0x7fa0b408a1e0 con 0x7fa0bc15cf40 2023-05-20T01:18:22.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.243+0000 7fa0ba59c700 1 --2- 172.21.15.16:0/3237373245 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa09c06ace0 0x7fa09c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:22.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.243+0000 7fa0ba59c700 1 --2- 172.21.15.16:0/3237373245 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa09c06ace0 0x7fa09c06d190 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac005910 tx=0x7fa0ac009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:22.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.247+0000 7fa0aaffd700 1 -- 172.21.15.16:0/3237373245 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fa0b4058990 con 0x7fa0bc15cf40 2023-05-20T01:18:22.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.444+0000 7fa0c0a8f700 1 -- 172.21.15.16:0/3237373245 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa0bc0c8ba0 con 0x7fa0bc15cf40 2023-05-20T01:18:22.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.445+0000 7fa0aaffd700 1 -- 172.21.15.16:0/3237373245 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v45) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa0bc0c8ba0 con 0x7fa0bc15cf40 2023-05-20T01:18:22.448 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-05-20T01:18:22.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.447+0000 7fa0c0a8f700 1 -- 172.21.15.16:0/3237373245 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa09c06ace0 msgr2=0x7fa09c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:22.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.447+0000 7fa0c0a8f700 1 --2- 172.21.15.16:0/3237373245 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa09c06ace0 0x7fa09c06d190 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac005910 tx=0x7fa0ac009040 comp rx=0 tx=0).stop 2023-05-20T01:18:22.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.447+0000 7fa0c0a8f700 1 -- 172.21.15.16:0/3237373245 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0bc15cf40 msgr2=0x7fa0bc1f0d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:22.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.447+0000 7fa0c0a8f700 1 --2- 172.21.15.16:0/3237373245 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0bc15cf40 0x7fa0bc1f0d40 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fa0b400efc0 tx=0x7fa0b400c3a0 comp rx=0 tx=0).stop 2023-05-20T01:18:22.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.447+0000 7fa0c0a8f700 1 -- 172.21.15.16:0/3237373245 shutdown_connections 2023-05-20T01:18:22.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.447+0000 7fa0c0a8f700 1 --2- 172.21.15.16:0/3237373245 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0bc15cf40 0x7fa0bc1f0d40 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:22.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.447+0000 7fa0c0a8f700 1 --2- 172.21.15.16:0/3237373245 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa09c06ace0 0x7fa09c06d190 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:22.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.447+0000 7fa0c0a8f700 1 --2- 172.21.15.16:0/3237373245 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa0bc0710d0 0x7fa0bc1ef510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:22.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.447+0000 7fa0c0a8f700 1 --2- 172.21.15.16:0/3237373245 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa0bc0cd070 0x7fa0bc159a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:22.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.447+0000 7fa0c0a8f700 1 -- 172.21.15.16:0/3237373245 >> 172.21.15.16:0/3237373245 conn(0x7fa0bc06c260 msgr2=0x7fa0bc0afbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:18:22.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.447+0000 7fa0c0a8f700 1 -- 172.21.15.16:0/3237373245 shutdown_connections 2023-05-20T01:18:22.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.447+0000 7fa0c0a8f700 1 -- 172.21.15.16:0/3237373245 wait complete. 2023-05-20T01:18:22.463 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-05-20T01:18:22.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.586+0000 7f9cba93d700 1 Processor -- start 2023-05-20T01:18:22.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.586+0000 7f9cba93d700 1 -- start start 2023-05-20T01:18:22.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.587+0000 7f9cba93d700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9cb40d16e0 0x7f9cb4144350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:22.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.587+0000 7f9cba93d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9cb40cec40 0x7f9cb40cf0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:22.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.587+0000 7f9cba93d700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cb4144890 con 0x7f9cb40ae2e0 2023-05-20T01:18:22.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.587+0000 7f9cba93d700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cb40cf5f0 con 0x7f9cb40cec40 2023-05-20T01:18:22.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.587+0000 7f9cba93d700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cb40b2020 con 0x7f9cb40d16e0 2023-05-20T01:18:22.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.587+0000 7f9cb37fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9cb40cec40 0x7f9cb40cf0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:22.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.587+0000 7f9cb8eda700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f9cb40ae2e0 0x7f9cb40ae760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:43814/0 (socket says 172.21.15.16:43814) 2023-05-20T01:18:22.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.587+0000 7f9cb8eda700 1 -- 172.21.15.16:0/3722920818 learned_addr learned my addr 172.21.15.16:0/3722920818 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:18:22.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.588+0000 7f9cb3fff700 1 --2- 172.21.15.16:0/3722920818 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9cb40d16e0 0x7f9cb4144350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:22.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.588+0000 7f9cb37fe700 1 -- 172.21.15.16:0/3722920818 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9cb40d16e0 msgr2=0x7f9cb4144350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:18:22.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.588+0000 7f9cb37fe700 1 --2- 172.21.15.16:0/3722920818 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9cb40d16e0 0x7f9cb4144350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:22.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.588+0000 7f9cb37fe700 1 -- 172.21.15.16:0/3722920818 >> v1:172.21.15.16:6789/0 conn(0x7f9cb40ae2e0 legacy=0x7f9cb40ae760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:22.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.588+0000 7f9cb37fe700 1 -- 172.21.15.16:0/3722920818 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cb40b2300 con 0x7f9cb40cec40 2023-05-20T01:18:22.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.588+0000 7f9cb2ffd700 1 -- 172.21.15.16:0/3722920818 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1418374440 0 0) 0x7f9cb4144890 con 0x7f9cb40ae2e0 2023-05-20T01:18:22.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.588+0000 7f9cb37fe700 1 --2- 172.21.15.16:0/3722920818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9cb40cec40 0x7f9cb40cf0b0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f9ca8009840 tx=0x7f9ca8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19ba218baef8b283 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:22.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.588+0000 7f9cb2ffd700 1 -- 172.21.15.16:0/3722920818 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ca8011830 con 0x7f9cb40cec40 2023-05-20T01:18:22.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.588+0000 7f9cb2ffd700 1 -- 172.21.15.16:0/3722920818 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ca8011990 con 0x7f9cb40cec40 2023-05-20T01:18:22.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.588+0000 7f9cb2ffd700 1 -- 172.21.15.16:0/3722920818 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ca8011d20 con 0x7f9cb40cec40 2023-05-20T01:18:22.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.588+0000 7f9cba93d700 1 -- 172.21.15.16:0/3722920818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9cb40cec40 msgr2=0x7f9cb40cf0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:22.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.588+0000 7f9cba93d700 1 --2- 172.21.15.16:0/3722920818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9cb40cec40 0x7f9cb40cf0b0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f9ca8009840 tx=0x7f9ca8009b50 comp rx=0 tx=0).stop 2023-05-20T01:18:22.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.589+0000 7f9cba93d700 1 -- 172.21.15.16:0/3722920818 shutdown_connections 2023-05-20T01:18:22.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.589+0000 7f9cba93d700 1 --2- 172.21.15.16:0/3722920818 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9cb40d16e0 0x7f9cb4144350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:22.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.589+0000 7f9cba93d700 1 --2- 172.21.15.16:0/3722920818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9cb40cec40 0x7f9cb40cf0b0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:22.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.589+0000 7f9cba93d700 1 -- 172.21.15.16:0/3722920818 >> 172.21.15.16:0/3722920818 conn(0x7f9cb406ae30 msgr2=0x7f9cb409a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:18:22.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.589+0000 7f9cba93d700 1 -- 172.21.15.16:0/3722920818 shutdown_connections 2023-05-20T01:18:22.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.589+0000 7f9cba93d700 1 -- 172.21.15.16:0/3722920818 wait complete. 2023-05-20T01:18:22.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.590+0000 7f9cba93d700 1 Processor -- start 2023-05-20T01:18:22.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.590+0000 7f9cba93d700 1 -- start start 2023-05-20T01:18:22.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.591+0000 7f9cba93d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9cb40ae2e0 0x7f9cb415a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:22.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.591+0000 7f9cba93d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9cb40cec40 0x7f9cb40cdc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:22.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.591+0000 7f9cba93d700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9cb40d16e0 0x7f9cb40ce1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:22.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.591+0000 7f9cba93d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cb41f0040 con 0x7f9cb40ae2e0 2023-05-20T01:18:22.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.591+0000 7f9cba93d700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cb415a5b0 con 0x7f9cb40cec40 2023-05-20T01:18:22.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.591+0000 7f9cba93d700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cb40cc2d0 con 0x7f9cb40d16e0 2023-05-20T01:18:22.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.591+0000 7f9cb3fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9cb40ae2e0 0x7f9cb415a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:22.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.591+0000 7f9cb3fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9cb40ae2e0 0x7f9cb415a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60520/0 (socket says 172.21.15.16:60520) 2023-05-20T01:18:22.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.591+0000 7f9cb3fff700 1 -- 172.21.15.16:0/1247135199 learned_addr learned my addr 172.21.15.16:0/1247135199 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:18:22.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.591+0000 7f9cb8eda700 1 --2- 172.21.15.16:0/1247135199 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9cb40d16e0 0x7f9cb40ce1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:22.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.591+0000 7f9cb37fe700 1 --2- 172.21.15.16:0/1247135199 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9cb40cec40 0x7f9cb40cdc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:22.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.591+0000 7f9cb3fff700 1 -- 172.21.15.16:0/1247135199 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9cb40d16e0 msgr2=0x7f9cb40ce1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:22.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.591+0000 7f9cb3fff700 1 --2- 172.21.15.16:0/1247135199 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9cb40d16e0 0x7f9cb40ce1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:22.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.591+0000 7f9cb3fff700 1 -- 172.21.15.16:0/1247135199 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9cb40cec40 msgr2=0x7f9cb40cdc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:22.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.591+0000 7f9cb3fff700 1 --2- 172.21.15.16:0/1247135199 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9cb40cec40 0x7f9cb40cdc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:22.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.591+0000 7f9cb3fff700 1 -- 172.21.15.16:0/1247135199 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ca80094a0 con 0x7f9cb40ae2e0 2023-05-20T01:18:22.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.591+0000 7f9cb3fff700 1 --2- 172.21.15.16:0/1247135199 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9cb40ae2e0 0x7f9cb415a070 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f9ca400c7f0 tx=0x7f9ca400cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:22.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.592+0000 7f9cb0ff9700 1 -- 172.21.15.16:0/1247135199 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ca4011800 con 0x7f9cb40ae2e0 2023-05-20T01:18:22.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.592+0000 7f9cb0ff9700 1 -- 172.21.15.16:0/1247135199 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ca4011960 con 0x7f9cb40ae2e0 2023-05-20T01:18:22.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.592+0000 7f9cb0ff9700 1 -- 172.21.15.16:0/1247135199 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ca4011cf0 con 0x7f9cb40ae2e0 2023-05-20T01:18:22.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.592+0000 7f9cba93d700 1 -- 172.21.15.16:0/1247135199 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cb40addc0 con 0x7f9cb40ae2e0 2023-05-20T01:18:22.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.592+0000 7f9cba93d700 1 -- 172.21.15.16:0/1247135199 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cb40b25e0 con 0x7f9cb40ae2e0 2023-05-20T01:18:22.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.593+0000 7f9cba93d700 1 -- 172.21.15.16:0/1247135199 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cb40b28c0 con 0x7f9cb40ae2e0 2023-05-20T01:18:22.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.593+0000 7f9cb0ff9700 1 -- 172.21.15.16:0/1247135199 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9ca4018360 con 0x7f9cb40ae2e0 2023-05-20T01:18:22.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.593+0000 7f9cb0ff9700 1 --2- 172.21.15.16:0/1247135199 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f9ca006ace0 0x7f9ca006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:22.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.594+0000 7f9cb0ff9700 1 -- 172.21.15.16:0/1247135199 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6844+0+0 (secure 0 0 0) 0x7f9ca4089740 con 0x7f9cb40ae2e0 2023-05-20T01:18:22.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.594+0000 7f9cb37fe700 1 --2- 172.21.15.16:0/1247135199 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f9ca006ace0 0x7f9ca006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:22.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.594+0000 7f9cb37fe700 1 --2- 172.21.15.16:0/1247135199 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f9ca006ace0 0x7f9ca006d190 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f9ca80090b0 tx=0x7f9ca8008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:22.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.598+0000 7f9cb0ff9700 1 -- 172.21.15.16:0/1247135199 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f9ca4057f80 con 0x7f9cb40ae2e0 2023-05-20T01:18:22.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.798+0000 7f9cba93d700 1 -- 172.21.15.16:0/1247135199 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9cb40747b0 con 0x7f9cb40ae2e0 2023-05-20T01:18:22.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.798+0000 7f9cb0ff9700 1 -- 172.21.15.16:0/1247135199 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v45) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9ca4057b10 con 0x7f9cb40ae2e0 2023-05-20T01:18:22.802 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2023-05-20T01:18:22.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.800+0000 7f9cba93d700 1 -- 172.21.15.16:0/1247135199 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f9ca006ace0 msgr2=0x7f9ca006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:22.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.800+0000 7f9cba93d700 1 --2- 172.21.15.16:0/1247135199 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f9ca006ace0 0x7f9ca006d190 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f9ca80090b0 tx=0x7f9ca8008030 comp rx=0 tx=0).stop 2023-05-20T01:18:22.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.800+0000 7f9cba93d700 1 -- 172.21.15.16:0/1247135199 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9cb40ae2e0 msgr2=0x7f9cb415a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:22.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.800+0000 7f9cba93d700 1 --2- 172.21.15.16:0/1247135199 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9cb40ae2e0 0x7f9cb415a070 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f9ca400c7f0 tx=0x7f9ca400cbb0 comp rx=0 tx=0).stop 2023-05-20T01:18:22.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.800+0000 7f9cba93d700 1 -- 172.21.15.16:0/1247135199 shutdown_connections 2023-05-20T01:18:22.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.800+0000 7f9cba93d700 1 --2- 172.21.15.16:0/1247135199 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f9ca006ace0 0x7f9ca006d190 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:22.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.800+0000 7f9cba93d700 1 --2- 172.21.15.16:0/1247135199 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9cb40d16e0 0x7f9cb40ce1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:22.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.800+0000 7f9cba93d700 1 --2- 172.21.15.16:0/1247135199 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9cb40ae2e0 0x7f9cb415a070 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:22.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.800+0000 7f9cba93d700 1 --2- 172.21.15.16:0/1247135199 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9cb40cec40 0x7f9cb40cdc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:22.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.800+0000 7f9cba93d700 1 -- 172.21.15.16:0/1247135199 >> 172.21.15.16:0/1247135199 conn(0x7f9cb406ae30 msgr2=0x7f9cb40ae6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:18:22.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.801+0000 7f9cba93d700 1 -- 172.21.15.16:0/1247135199 shutdown_connections 2023-05-20T01:18:22.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.801+0000 7f9cba93d700 1 -- 172.21.15.16:0/1247135199 wait complete. 2023-05-20T01:18:22.816 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_extra pg_num 2023-05-20T01:18:22.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.941+0000 7fd0e8c62700 1 Processor -- start 2023-05-20T01:18:22.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.941+0000 7fd0e8c62700 1 -- start start 2023-05-20T01:18:22.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.942+0000 7fd0e8c62700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fd0e40cd1c0 0x7fd0e40710b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:22.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.942+0000 7fd0e8c62700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd0e40cd630 0x7fd0e40715f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:22.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.942+0000 7fd0e8c62700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0e40ccd30 con 0x7fd0e4159b50 2023-05-20T01:18:22.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.942+0000 7fd0e8c62700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0e40ce990 con 0x7fd0e40cd630 2023-05-20T01:18:22.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.942+0000 7fd0e8c62700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0e40cc6e0 con 0x7fd0e40cd1c0 2023-05-20T01:18:22.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.942+0000 7fd0e2d9d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd0e40cd630 0x7fd0e40715f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:22.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.942+0000 7fd0e2d9d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd0e40cd630 0x7fd0e40715f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60532/0 (socket says 172.21.15.16:60532) 2023-05-20T01:18:22.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.942+0000 7fd0e259c700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fd0e4159b50 0x7fd0e406e0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:43824/0 (socket says 172.21.15.16:43824) 2023-05-20T01:18:22.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.942+0000 7fd0e2d9d700 1 -- 172.21.15.16:0/4207521240 learned_addr learned my addr 172.21.15.16:0/4207521240 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:18:22.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.942+0000 7fd0e1d9b700 1 --2- 172.21.15.16:0/4207521240 >> v2:172.21.15.132:3300/0 conn(0x7fd0e40cd1c0 0x7fd0e40710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:22.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.942+0000 7fd0e2d9d700 1 -- 172.21.15.16:0/4207521240 >> v2:172.21.15.132:3300/0 conn(0x7fd0e40cd1c0 msgr2=0x7fd0e40710b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:18:22.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.942+0000 7fd0e2d9d700 1 --2- 172.21.15.16:0/4207521240 >> v2:172.21.15.132:3300/0 conn(0x7fd0e40cd1c0 0x7fd0e40710b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:22.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.942+0000 7fd0e2d9d700 1 -- 172.21.15.16:0/4207521240 >> v1:172.21.15.16:6789/0 conn(0x7fd0e4159b50 legacy=0x7fd0e406e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:22.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.942+0000 7fd0e2d9d700 1 -- 172.21.15.16:0/4207521240 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0e40cc9c0 con 0x7fd0e40cd630 2023-05-20T01:18:22.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.942+0000 7fd0e159a700 1 -- 172.21.15.16:0/4207521240 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 706520734 0 0) 0x7fd0e40ccd30 con 0x7fd0e4159b50 2023-05-20T01:18:22.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.942+0000 7fd0e2d9d700 1 --2- 172.21.15.16:0/4207521240 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0e40cd630 0x7fd0e40715f0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fd0e40b2bc0 tx=0x7fd0dc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10ca78c2f653043e server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:22.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.943+0000 7fd0e159a700 1 -- 172.21.15.16:0/4207521240 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0dc00e070 con 0x7fd0e40cd630 2023-05-20T01:18:22.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.943+0000 7fd0e159a700 1 -- 172.21.15.16:0/4207521240 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0dc00b9c0 con 0x7fd0e40cd630 2023-05-20T01:18:22.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.943+0000 7fd0e159a700 1 -- 172.21.15.16:0/4207521240 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0dc00bd70 con 0x7fd0e40cd630 2023-05-20T01:18:22.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.943+0000 7fd0e8c62700 1 -- 172.21.15.16:0/4207521240 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0e40cd630 msgr2=0x7fd0e40715f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:22.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.943+0000 7fd0e8c62700 1 --2- 172.21.15.16:0/4207521240 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0e40cd630 0x7fd0e40715f0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fd0e40b2bc0 tx=0x7fd0dc00b330 comp rx=0 tx=0).stop 2023-05-20T01:18:22.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.943+0000 7fd0e8c62700 1 -- 172.21.15.16:0/4207521240 shutdown_connections 2023-05-20T01:18:22.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.943+0000 7fd0e8c62700 1 --2- 172.21.15.16:0/4207521240 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0e40cd630 0x7fd0e40715f0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:22.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.943+0000 7fd0e8c62700 1 --2- 172.21.15.16:0/4207521240 >> v2:172.21.15.132:3300/0 conn(0x7fd0e40cd1c0 0x7fd0e40710b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:22.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.943+0000 7fd0e8c62700 1 -- 172.21.15.16:0/4207521240 >> 172.21.15.16:0/4207521240 conn(0x7fd0e40cffd0 msgr2=0x7fd0e40d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:18:22.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.943+0000 7fd0e8c62700 1 -- 172.21.15.16:0/4207521240 shutdown_connections 2023-05-20T01:18:22.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.943+0000 7fd0e8c62700 1 -- 172.21.15.16:0/4207521240 wait complete. 2023-05-20T01:18:22.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.945+0000 7fd0e8c62700 1 Processor -- start 2023-05-20T01:18:22.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.945+0000 7fd0e8c62700 1 -- start start 2023-05-20T01:18:22.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.946+0000 7fd0e8c62700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd0e40cd1c0 0x7fd0e40b59d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:22.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.946+0000 7fd0e8c62700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd0e40cd630 0x7fd0e41529e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:22.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.946+0000 7fd0e8c62700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0e4159b50 0x7fd0e4157bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:22.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.946+0000 7fd0e8c62700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0e40b5f10 con 0x7fd0e4159b50 2023-05-20T01:18:22.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.946+0000 7fd0e8c62700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0e4152f20 con 0x7fd0e40cd630 2023-05-20T01:18:22.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.946+0000 7fd0e8c62700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0e40c7140 con 0x7fd0e40cd1c0 2023-05-20T01:18:22.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.946+0000 7fd0e2d9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0e4159b50 0x7fd0e4157bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:22.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.946+0000 7fd0e2d9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0e4159b50 0x7fd0e4157bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60538/0 (socket says 172.21.15.16:60538) 2023-05-20T01:18:22.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.946+0000 7fd0e2d9d700 1 -- 172.21.15.16:0/3991239869 learned_addr learned my addr 172.21.15.16:0/3991239869 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:18:22.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.946+0000 7fd0e259c700 1 --2- 172.21.15.16:0/3991239869 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd0e40cd1c0 0x7fd0e40b59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:22.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.946+0000 7fd0e1d9b700 1 --2- 172.21.15.16:0/3991239869 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd0e40cd630 0x7fd0e41529e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:22.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.947+0000 7fd0e2d9d700 1 -- 172.21.15.16:0/3991239869 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd0e40cd1c0 msgr2=0x7fd0e40b59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:22.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.947+0000 7fd0e2d9d700 1 --2- 172.21.15.16:0/3991239869 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd0e40cd1c0 0x7fd0e40b59d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:22.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.947+0000 7fd0e2d9d700 1 -- 172.21.15.16:0/3991239869 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd0e40cd630 msgr2=0x7fd0e41529e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:22.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.947+0000 7fd0e2d9d700 1 --2- 172.21.15.16:0/3991239869 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd0e40cd630 0x7fd0e41529e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:22.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.947+0000 7fd0e2d9d700 1 -- 172.21.15.16:0/3991239869 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0dc009b30 con 0x7fd0e4159b50 2023-05-20T01:18:22.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.947+0000 7fd0e2d9d700 1 --2- 172.21.15.16:0/3991239869 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0e4159b50 0x7fd0e4157bb0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fd0e40675d0 tx=0x7fd0dc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:22.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.947+0000 7fd0d2ffd700 1 -- 172.21.15.16:0/3991239869 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0dc0099b0 con 0x7fd0e4159b50 2023-05-20T01:18:22.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.947+0000 7fd0d2ffd700 1 -- 172.21.15.16:0/3991239869 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0dc0092e0 con 0x7fd0e4159b50 2023-05-20T01:18:22.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.947+0000 7fd0e8c62700 1 -- 172.21.15.16:0/3991239869 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0e40cec70 con 0x7fd0e4159b50 2023-05-20T01:18:22.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.947+0000 7fd0d2ffd700 1 -- 172.21.15.16:0/3991239869 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0dc01c660 con 0x7fd0e4159b50 2023-05-20T01:18:22.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.947+0000 7fd0e8c62700 1 -- 172.21.15.16:0/3991239869 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0e4071e80 con 0x7fd0e4159b50 2023-05-20T01:18:22.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.948+0000 7fd0e8c62700 1 -- 172.21.15.16:0/3991239869 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0e40747b0 con 0x7fd0e4159b50 2023-05-20T01:18:22.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.948+0000 7fd0d2ffd700 1 -- 172.21.15.16:0/3991239869 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd0dc01c7c0 con 0x7fd0e4159b50 2023-05-20T01:18:22.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.949+0000 7fd0d2ffd700 1 --2- 172.21.15.16:0/3991239869 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd0c406ace0 0x7fd0c406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:22.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.949+0000 7fd0d2ffd700 1 -- 172.21.15.16:0/3991239869 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6844+0+0 (secure 0 0 0) 0x7fd0dc08b100 con 0x7fd0e4159b50 2023-05-20T01:18:22.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.949+0000 7fd0e259c700 1 --2- 172.21.15.16:0/3991239869 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd0c406ace0 0x7fd0c406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:22.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.949+0000 7fd0e259c700 1 --2- 172.21.15.16:0/3991239869 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd0c406ace0 0x7fd0c406d190 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd0d4001020 tx=0x7fd0d4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:22.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:22.953+0000 7fd0d2ffd700 1 -- 172.21.15.16:0/3991239869 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fd0dc055f30 con 0x7fd0e4159b50 2023-05-20T01:18:23.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.152+0000 7fd0e8c62700 1 -- 172.21.15.16:0/3991239869 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"} v 0) v1 -- 0x7fd0e40c6180 con 0x7fd0e4159b50 2023-05-20T01:18:23.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.152+0000 7fd0d2ffd700 1 -- 172.21.15.16:0/3991239869 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"}]=0 v45) v1 ==== 106+0+11 (secure 0 0 0) 0x7fd0dc059550 con 0x7fd0e4159b50 2023-05-20T01:18:23.156 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-05-20T01:18:23.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.154+0000 7fd0e8c62700 1 -- 172.21.15.16:0/3991239869 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd0c406ace0 msgr2=0x7fd0c406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:23.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.154+0000 7fd0e8c62700 1 --2- 172.21.15.16:0/3991239869 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd0c406ace0 0x7fd0c406d190 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd0d4001020 tx=0x7fd0d4009040 comp rx=0 tx=0).stop 2023-05-20T01:18:23.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.154+0000 7fd0e8c62700 1 -- 172.21.15.16:0/3991239869 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0e4159b50 msgr2=0x7fd0e4157bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:23.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.154+0000 7fd0e8c62700 1 --2- 172.21.15.16:0/3991239869 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0e4159b50 0x7fd0e4157bb0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fd0e40675d0 tx=0x7fd0dc004ad0 comp rx=0 tx=0).stop 2023-05-20T01:18:23.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.154+0000 7fd0e8c62700 1 -- 172.21.15.16:0/3991239869 shutdown_connections 2023-05-20T01:18:23.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.154+0000 7fd0e8c62700 1 --2- 172.21.15.16:0/3991239869 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0e4159b50 0x7fd0e4157bb0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:23.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.154+0000 7fd0e8c62700 1 --2- 172.21.15.16:0/3991239869 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd0c406ace0 0x7fd0c406d190 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:23.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.154+0000 7fd0e8c62700 1 --2- 172.21.15.16:0/3991239869 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd0e40cd1c0 0x7fd0e40b59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:23.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.154+0000 7fd0e8c62700 1 --2- 172.21.15.16:0/3991239869 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd0e40cd630 0x7fd0e41529e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:23.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.154+0000 7fd0e8c62700 1 -- 172.21.15.16:0/3991239869 >> 172.21.15.16:0/3991239869 conn(0x7fd0e40cffd0 msgr2=0x7fd0e40b3eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:18:23.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.154+0000 7fd0e8c62700 1 -- 172.21.15.16:0/3991239869 shutdown_connections 2023-05-20T01:18:23.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.154+0000 7fd0e8c62700 1 -- 172.21.15.16:0/3991239869 wait complete. 2023-05-20T01:18:23.170 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-05-20T01:18:23.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.291+0000 7fe8cd41f700 1 Processor -- start 2023-05-20T01:18:23.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.291+0000 7fe8cd41f700 1 -- start start 2023-05-20T01:18:23.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.292+0000 7fe8cd41f700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe8c806d2a0 0x7fe8c80cae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:23.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.292+0000 7fe8cd41f700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe8c80ccf00 0x7fe8c80cb360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:23.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.292+0000 7fe8cd41f700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8c815cec0 con 0x7fe8c815caf0 2023-05-20T01:18:23.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.292+0000 7fe8cd41f700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8c80d0490 con 0x7fe8c806d2a0 2023-05-20T01:18:23.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.292+0000 7fe8cd41f700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c8160610 con 0x7fe8c80ccf00 2023-05-20T01:18:23.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.292+0000 7fe8c67fc700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe8c806d2a0 0x7fe8c80cae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:23.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.292+0000 7fe8c6ffd700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fe8c815caf0 0x7fe8c806cb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:43838/0 (socket says 172.21.15.16:43838) 2023-05-20T01:18:23.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.292+0000 7fe8c6ffd700 1 -- 172.21.15.16:0/864173136 learned_addr learned my addr 172.21.15.16:0/864173136 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:18:23.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.292+0000 7fe8c77fe700 1 --2- 172.21.15.16:0/864173136 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe8c80ccf00 0x7fe8c80cb360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:23.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.293+0000 7fe8c67fc700 1 -- 172.21.15.16:0/864173136 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe8c80ccf00 msgr2=0x7fe8c80cb360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:18:23.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.293+0000 7fe8c67fc700 1 --2- 172.21.15.16:0/864173136 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe8c80ccf00 0x7fe8c80cb360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:23.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.293+0000 7fe8c67fc700 1 -- 172.21.15.16:0/864173136 >> v1:172.21.15.16:6789/0 conn(0x7fe8c815caf0 legacy=0x7fe8c806cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:23.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.293+0000 7fe8c67fc700 1 -- 172.21.15.16:0/864173136 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8c81608f0 con 0x7fe8c806d2a0 2023-05-20T01:18:23.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.293+0000 7fe8c67fc700 1 --2- 172.21.15.16:0/864173136 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8c806d2a0 0x7fe8c80cae20 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fe8b8009840 tx=0x7fe8b8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c1556318949f2c server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:23.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.293+0000 7fe8c5ffb700 1 -- 172.21.15.16:0/864173136 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b8011830 con 0x7fe8c806d2a0 2023-05-20T01:18:23.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.293+0000 7fe8c5ffb700 1 -- 172.21.15.16:0/864173136 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8b8011990 con 0x7fe8c806d2a0 2023-05-20T01:18:23.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.293+0000 7fe8c5ffb700 1 -- 172.21.15.16:0/864173136 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b8011d20 con 0x7fe8c806d2a0 2023-05-20T01:18:23.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.293+0000 7fe8cd41f700 1 -- 172.21.15.16:0/864173136 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8c806d2a0 msgr2=0x7fe8c80cae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:23.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.293+0000 7fe8cd41f700 1 --2- 172.21.15.16:0/864173136 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8c806d2a0 0x7fe8c80cae20 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fe8b8009840 tx=0x7fe8b8009b50 comp rx=0 tx=0).stop 2023-05-20T01:18:23.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.293+0000 7fe8cd41f700 1 -- 172.21.15.16:0/864173136 shutdown_connections 2023-05-20T01:18:23.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.293+0000 7fe8cd41f700 1 --2- 172.21.15.16:0/864173136 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe8c80ccf00 0x7fe8c80cb360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:23.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.293+0000 7fe8cd41f700 1 --2- 172.21.15.16:0/864173136 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8c806d2a0 0x7fe8c80cae20 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:23.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.293+0000 7fe8cd41f700 1 -- 172.21.15.16:0/864173136 >> 172.21.15.16:0/864173136 conn(0x7fe8c80661b0 msgr2=0x7fe8c8066580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:18:23.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.294+0000 7fe8cd41f700 1 -- 172.21.15.16:0/864173136 shutdown_connections 2023-05-20T01:18:23.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.294+0000 7fe8cd41f700 1 -- 172.21.15.16:0/864173136 wait complete. 2023-05-20T01:18:23.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.295+0000 7fe8cd41f700 1 Processor -- start 2023-05-20T01:18:23.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.295+0000 7fe8cd41f700 1 -- start start 2023-05-20T01:18:23.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.296+0000 7fe8cd41f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8c806d2a0 0x7fe8c8157c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:23.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.296+0000 7fe8cd41f700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe8c80ccf00 0x7fe8c815b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:23.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.296+0000 7fe8cd41f700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe8c815caf0 0x7fe8c80b38a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:23.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.296+0000 7fe8cd41f700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c8158c30 con 0x7fe8c806d2a0 2023-05-20T01:18:23.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.296+0000 7fe8cd41f700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c8158160 con 0x7fe8c80ccf00 2023-05-20T01:18:23.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.296+0000 7fe8cd41f700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c815bbe0 con 0x7fe8c815caf0 2023-05-20T01:18:23.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.296+0000 7fe8c6ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8c806d2a0 0x7fe8c8157c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:23.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.296+0000 7fe8c6ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8c806d2a0 0x7fe8c8157c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60548/0 (socket says 172.21.15.16:60548) 2023-05-20T01:18:23.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.296+0000 7fe8c6ffd700 1 -- 172.21.15.16:0/3324119419 learned_addr learned my addr 172.21.15.16:0/3324119419 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:18:23.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.296+0000 7fe8c77fe700 1 --2- 172.21.15.16:0/3324119419 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe8c815caf0 0x7fe8c80b38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:23.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.296+0000 7fe8c67fc700 1 --2- 172.21.15.16:0/3324119419 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe8c80ccf00 0x7fe8c815b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:23.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.296+0000 7fe8c6ffd700 1 -- 172.21.15.16:0/3324119419 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe8c815caf0 msgr2=0x7fe8c80b38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:23.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.296+0000 7fe8c6ffd700 1 --2- 172.21.15.16:0/3324119419 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe8c815caf0 0x7fe8c80b38a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:23.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.296+0000 7fe8c6ffd700 1 -- 172.21.15.16:0/3324119419 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe8c80ccf00 msgr2=0x7fe8c815b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:23.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.296+0000 7fe8c6ffd700 1 --2- 172.21.15.16:0/3324119419 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe8c80ccf00 0x7fe8c815b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:23.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.296+0000 7fe8c6ffd700 1 -- 172.21.15.16:0/3324119419 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8b80094a0 con 0x7fe8c806d2a0 2023-05-20T01:18:23.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.296+0000 7fe8c6ffd700 1 --2- 172.21.15.16:0/3324119419 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8c806d2a0 0x7fe8c8157c20 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc00c480 tx=0x7fe8bc00c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:23.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.296+0000 7fe8af7fe700 1 -- 172.21.15.16:0/3324119419 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8bc005d70 con 0x7fe8c806d2a0 2023-05-20T01:18:23.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.296+0000 7fe8af7fe700 1 -- 172.21.15.16:0/3324119419 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8bc0070d0 con 0x7fe8c806d2a0 2023-05-20T01:18:23.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.296+0000 7fe8cd41f700 1 -- 172.21.15.16:0/3324119419 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8c8160eb0 con 0x7fe8c806d2a0 2023-05-20T01:18:23.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.296+0000 7fe8af7fe700 1 -- 172.21.15.16:0/3324119419 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8bc007440 con 0x7fe8c806d2a0 2023-05-20T01:18:23.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.297+0000 7fe8cd41f700 1 -- 172.21.15.16:0/3324119419 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8c8160bd0 con 0x7fe8c806d2a0 2023-05-20T01:18:23.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.297+0000 7fe8cd41f700 1 -- 172.21.15.16:0/3324119419 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8c80d0110 con 0x7fe8c806d2a0 2023-05-20T01:18:23.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.298+0000 7fe8af7fe700 1 -- 172.21.15.16:0/3324119419 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe8bc0075a0 con 0x7fe8c806d2a0 2023-05-20T01:18:23.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.298+0000 7fe8af7fe700 1 --2- 172.21.15.16:0/3324119419 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe8b406ace0 0x7fe8b406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:23.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.298+0000 7fe8af7fe700 1 -- 172.21.15.16:0/3324119419 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6844+0+0 (secure 0 0 0) 0x7fe8bc088bf0 con 0x7fe8c806d2a0 2023-05-20T01:18:23.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.298+0000 7fe8c67fc700 1 --2- 172.21.15.16:0/3324119419 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe8b406ace0 0x7fe8b406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:23.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.299+0000 7fe8c67fc700 1 --2- 172.21.15.16:0/3324119419 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe8b406ace0 0x7fe8b406d190 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe8c806d690 tx=0x7fe8b8008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:23.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.303+0000 7fe8af7fe700 1 -- 172.21.15.16:0/3324119419 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fe8bc053a40 con 0x7fe8c806d2a0 2023-05-20T01:18:23.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.519+0000 7fe8cd41f700 1 -- 172.21.15.16:0/3324119419 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fe8c8066d90 con 0x7fe8c806d2a0 2023-05-20T01:18:23.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.519+0000 7fe8af7fe700 1 -- 172.21.15.16:0/3324119419 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v45) v1 ==== 80+0+0 (secure 0 0 0) 0x7fe8bc057060 con 0x7fe8c806d2a0 2023-05-20T01:18:23.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.521+0000 7fe8cd41f700 1 -- 172.21.15.16:0/3324119419 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe8b406ace0 msgr2=0x7fe8b406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:23.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.521+0000 7fe8cd41f700 1 --2- 172.21.15.16:0/3324119419 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe8b406ace0 0x7fe8b406d190 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe8c806d690 tx=0x7fe8b8008030 comp rx=0 tx=0).stop 2023-05-20T01:18:23.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.521+0000 7fe8cd41f700 1 -- 172.21.15.16:0/3324119419 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8c806d2a0 msgr2=0x7fe8c8157c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:23.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.521+0000 7fe8cd41f700 1 --2- 172.21.15.16:0/3324119419 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8c806d2a0 0x7fe8c8157c20 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc00c480 tx=0x7fe8bc00c840 comp rx=0 tx=0).stop 2023-05-20T01:18:23.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.521+0000 7fe8cd41f700 1 -- 172.21.15.16:0/3324119419 shutdown_connections 2023-05-20T01:18:23.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.521+0000 7fe8cd41f700 1 --2- 172.21.15.16:0/3324119419 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe8b406ace0 0x7fe8b406d190 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:23.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.521+0000 7fe8cd41f700 1 --2- 172.21.15.16:0/3324119419 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe8c815caf0 0x7fe8c80b38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:23.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.521+0000 7fe8cd41f700 1 --2- 172.21.15.16:0/3324119419 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8c806d2a0 0x7fe8c8157c20 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:23.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.521+0000 7fe8cd41f700 1 --2- 172.21.15.16:0/3324119419 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe8c80ccf00 0x7fe8c815b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:23.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.521+0000 7fe8cd41f700 1 -- 172.21.15.16:0/3324119419 >> 172.21.15.16:0/3324119419 conn(0x7fe8c80661b0 msgr2=0x7fe8c8158f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:18:23.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.521+0000 7fe8cd41f700 1 -- 172.21.15.16:0/3324119419 shutdown_connections 2023-05-20T01:18:23.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.521+0000 7fe8cd41f700 1 -- 172.21.15.16:0/3324119419 wait complete. 2023-05-20T01:18:23.527 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2023-05-20T01:18:23.537 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-05-20T01:18:23.537 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:18:23.537 DEBUG:teuthology.orchestra.run.smithi016:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-05-20T01:18:23.652 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2023-05-20T01:18:23.652 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:18:23.653 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-05-20T01:18:23.680 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-20T01:18:23.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.841+0000 7f4c69517700 1 Processor -- start 2023-05-20T01:18:23.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.841+0000 7f4c69517700 1 -- start start 2023-05-20T01:18:23.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.842+0000 7f4c69517700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4c6406a510 0x7f4c6415e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:23.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.842+0000 7f4c69517700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c6415e060 con 0x7f4c64071340 2023-05-20T01:18:23.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.842+0000 7f4c69517700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c6415d450 con 0x7f4c6406a8e0 2023-05-20T01:18:23.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.842+0000 7f4c69517700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c6415d730 con 0x7f4c6406a510 2023-05-20T01:18:23.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.842+0000 7f4c62ffd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4c6406a510 0x7f4c6415e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:23.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.842+0000 7f4c637fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f4c64071340 0x7f4c64071780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:43860/0 (socket says 172.21.15.16:43860) 2023-05-20T01:18:23.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.842+0000 7f4c637fe700 1 -- 172.21.15.16:0/3242890085 learned_addr learned my addr 172.21.15.16:0/3242890085 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:18:23.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.843+0000 7f4c62ffd700 1 -- 172.21.15.16:0/3242890085 >> v1:172.21.15.132:6789/0 conn(0x7f4c6406a8e0 legacy=0x7f4c640cf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:18:23.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.843+0000 7f4c62ffd700 1 -- 172.21.15.16:0/3242890085 >> v1:172.21.15.16:6789/0 conn(0x7f4c64071340 legacy=0x7f4c64071780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:23.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.843+0000 7f4c61ffb700 1 -- 172.21.15.16:0/3242890085 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1002904523 0 0) 0x7f4c6415e060 con 0x7f4c64071340 2023-05-20T01:18:23.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.843+0000 7f4c62ffd700 1 -- 172.21.15.16:0/3242890085 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c6415da10 con 0x7f4c6406a510 2023-05-20T01:18:23.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.843+0000 7f4c62ffd700 1 --2- 172.21.15.16:0/3242890085 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c6406a510 0x7f4c6415e2a0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f4c54009840 tx=0x7f4c54009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6c210956298b87e server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:23.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.843+0000 7f4c61ffb700 1 -- 172.21.15.16:0/3242890085 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c54011830 con 0x7f4c6406a510 2023-05-20T01:18:23.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.843+0000 7f4c61ffb700 1 -- 172.21.15.16:0/3242890085 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c54011990 con 0x7f4c6406a510 2023-05-20T01:18:23.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.843+0000 7f4c61ffb700 1 -- 172.21.15.16:0/3242890085 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c54011d20 con 0x7f4c6406a510 2023-05-20T01:18:23.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.844+0000 7f4c69517700 1 -- 172.21.15.16:0/3242890085 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c6406a510 msgr2=0x7f4c6415e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:23.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.844+0000 7f4c69517700 1 --2- 172.21.15.16:0/3242890085 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c6406a510 0x7f4c6415e2a0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f4c54009840 tx=0x7f4c54009b50 comp rx=0 tx=0).stop 2023-05-20T01:18:23.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.844+0000 7f4c69517700 1 -- 172.21.15.16:0/3242890085 shutdown_connections 2023-05-20T01:18:23.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.844+0000 7f4c69517700 1 --2- 172.21.15.16:0/3242890085 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c6406a510 0x7f4c6415e2a0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:23.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.844+0000 7f4c69517700 1 -- 172.21.15.16:0/3242890085 >> 172.21.15.16:0/3242890085 conn(0x7f4c640d0c30 msgr2=0x7f4c640b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:18:23.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.844+0000 7f4c69517700 1 -- 172.21.15.16:0/3242890085 shutdown_connections 2023-05-20T01:18:23.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.844+0000 7f4c69517700 1 -- 172.21.15.16:0/3242890085 wait complete. 2023-05-20T01:18:23.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.845+0000 7f4c69517700 1 Processor -- start 2023-05-20T01:18:23.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.845+0000 7f4c69517700 1 -- start start 2023-05-20T01:18:23.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.846+0000 7f4c69517700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c6406a510 0x7f4c641f26d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:23.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.846+0000 7f4c69517700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4c6406a8e0 0x7f4c640c7c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:23.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.846+0000 7f4c69517700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4c64071340 0x7f4c640c9900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:23.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.846+0000 7f4c69517700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c640b4120 con 0x7f4c6406a510 2023-05-20T01:18:23.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.846+0000 7f4c69517700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c640b4400 con 0x7f4c6406a8e0 2023-05-20T01:18:23.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.846+0000 7f4c69517700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c640b46e0 con 0x7f4c64071340 2023-05-20T01:18:23.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.846+0000 7f4c62ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c6406a510 0x7f4c641f26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:23.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.846+0000 7f4c62ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c6406a510 0x7f4c641f26d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60566/0 (socket says 172.21.15.16:60566) 2023-05-20T01:18:23.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.846+0000 7f4c62ffd700 1 -- 172.21.15.16:0/2068994136 learned_addr learned my addr 172.21.15.16:0/2068994136 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:18:23.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.846+0000 7f4c637fe700 1 --2- 172.21.15.16:0/2068994136 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4c64071340 0x7f4c640c9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:23.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.846+0000 7f4c627fc700 1 --2- 172.21.15.16:0/2068994136 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4c6406a8e0 0x7f4c640c7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:23.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.846+0000 7f4c62ffd700 1 -- 172.21.15.16:0/2068994136 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4c64071340 msgr2=0x7f4c640c9900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:23.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.846+0000 7f4c62ffd700 1 --2- 172.21.15.16:0/2068994136 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4c64071340 0x7f4c640c9900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:23.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.846+0000 7f4c62ffd700 1 -- 172.21.15.16:0/2068994136 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4c6406a8e0 msgr2=0x7f4c640c7c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:23.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.846+0000 7f4c62ffd700 1 --2- 172.21.15.16:0/2068994136 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4c6406a8e0 0x7f4c640c7c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:23.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.846+0000 7f4c62ffd700 1 -- 172.21.15.16:0/2068994136 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c540094a0 con 0x7f4c6406a510 2023-05-20T01:18:23.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.847+0000 7f4c62ffd700 1 --2- 172.21.15.16:0/2068994136 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c6406a510 0x7f4c641f26d0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f4c54014040 tx=0x7f4c54004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:23.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.847+0000 7f4c4b7fe700 1 -- 172.21.15.16:0/2068994136 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c5401a900 con 0x7f4c6406a510 2023-05-20T01:18:23.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.847+0000 7f4c4b7fe700 1 -- 172.21.15.16:0/2068994136 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c5401aa60 con 0x7f4c6406a510 2023-05-20T01:18:23.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.847+0000 7f4c69517700 1 -- 172.21.15.16:0/2068994136 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c640cfd00 con 0x7f4c6406a510 2023-05-20T01:18:23.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.847+0000 7f4c4b7fe700 1 -- 172.21.15.16:0/2068994136 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c5401add0 con 0x7f4c6406a510 2023-05-20T01:18:23.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.847+0000 7f4c69517700 1 -- 172.21.15.16:0/2068994136 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c6415dcf0 con 0x7f4c6406a510 2023-05-20T01:18:23.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.848+0000 7f4c69517700 1 -- 172.21.15.16:0/2068994136 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c6406ad50 con 0x7f4c6406a510 2023-05-20T01:18:23.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.848+0000 7f4c4b7fe700 1 -- 172.21.15.16:0/2068994136 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4c54022070 con 0x7f4c6406a510 2023-05-20T01:18:23.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.849+0000 7f4c4b7fe700 1 --2- 172.21.15.16:0/2068994136 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f4c5006ace0 0x7f4c5006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:23.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.849+0000 7f4c4b7fe700 1 -- 172.21.15.16:0/2068994136 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6844+0+0 (secure 0 0 0) 0x7f4c5408aa20 con 0x7f4c6406a510 2023-05-20T01:18:23.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.849+0000 7f4c627fc700 1 --2- 172.21.15.16:0/2068994136 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f4c5006ace0 0x7f4c5006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:23.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.849+0000 7f4c627fc700 1 --2- 172.21.15.16:0/2068994136 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f4c5006ace0 0x7f4c5006d190 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f4c640712b0 tx=0x7f4c58009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:23.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:23.853+0000 7f4c4b7fe700 1 -- 172.21.15.16:0/2068994136 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f4c54055970 con 0x7f4c6406a510 2023-05-20T01:18:24.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:24.095+0000 7f4c69517700 1 -- 172.21.15.16:0/2068994136 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4c640cbad0 con 0x7f4c6406a510 2023-05-20T01:18:24.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:24.095+0000 7f4c4b7fe700 1 -- 172.21.15.16:0/2068994136 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4676 (secure 0 0 0) 0x7f4c54058f90 con 0x7f4c6406a510 2023-05-20T01:18:24.099 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:18:24.099 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5176,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/4144439890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4144439890},{"type":"v1","addr":"172.21.15.16:6836","nonce":4144439890}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5188,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/2992755084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":2992755084},{"type":"v1","addr":"172.21.15.132:6834","nonce":2992755084}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/1979284857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1979284857},{"type":"v1","addr":"172.21.15.132:6835","nonce":1979284857}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-20T01:17:46.995921+0000","modified":"2023-05-20T01:17:55.758292+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5397},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5397":{"gid":5397,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.16:6837/942011026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":942011026},{"type":"v1","addr":"172.21.15.16:6837","nonce":942011026}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7,8],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-05-20T01:18:24.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:24.097+0000 7f4c69517700 1 -- 172.21.15.16:0/2068994136 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f4c5006ace0 msgr2=0x7f4c5006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:24.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:24.097+0000 7f4c69517700 1 --2- 172.21.15.16:0/2068994136 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f4c5006ace0 0x7f4c5006d190 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f4c640712b0 tx=0x7f4c58009040 comp rx=0 tx=0).stop 2023-05-20T01:18:24.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:24.097+0000 7f4c69517700 1 -- 172.21.15.16:0/2068994136 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c6406a510 msgr2=0x7f4c641f26d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:24.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:24.097+0000 7f4c69517700 1 --2- 172.21.15.16:0/2068994136 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c6406a510 0x7f4c641f26d0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f4c54014040 tx=0x7f4c54004750 comp rx=0 tx=0).stop 2023-05-20T01:18:24.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:24.097+0000 7f4c69517700 1 -- 172.21.15.16:0/2068994136 shutdown_connections 2023-05-20T01:18:24.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:24.097+0000 7f4c69517700 1 --2- 172.21.15.16:0/2068994136 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f4c5006ace0 0x7f4c5006d190 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:24.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:24.097+0000 7f4c69517700 1 --2- 172.21.15.16:0/2068994136 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4c64071340 0x7f4c640c9900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:24.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:24.097+0000 7f4c69517700 1 --2- 172.21.15.16:0/2068994136 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c6406a510 0x7f4c641f26d0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:24.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:24.097+0000 7f4c69517700 1 --2- 172.21.15.16:0/2068994136 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4c6406a8e0 0x7f4c640c7c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:24.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:24.097+0000 7f4c69517700 1 -- 172.21.15.16:0/2068994136 >> 172.21.15.16:0/2068994136 conn(0x7f4c640d0c30 msgr2=0x7f4c6415f0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:18:24.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:24.097+0000 7f4c69517700 1 -- 172.21.15.16:0/2068994136 shutdown_connections 2023-05-20T01:18:24.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:24.097+0000 7f4c69517700 1 -- 172.21.15.16:0/2068994136 wait complete. 2023-05-20T01:18:24.103 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 17 2023-05-20T01:18:24.113 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2023-05-20T01:18:24.263 INFO:teuthology.orchestra.run.smithi016.stdout:{"message":"","return_code":0} 2023-05-20T01:18:24.264 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2023-05-20T01:18:24.264 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-05-20T01:18:24.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:24.387+0000 7f8897842700 1 Processor -- start 2023-05-20T01:18:24.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:24.387+0000 7f8897842700 1 -- start start 2023-05-20T01:18:24.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:24.388+0000 7f8897842700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f88901599b0 0x7f889015ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:24.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:24.388+0000 7f8897842700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f88900b0910 0x7f88900b0d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:24.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:24.388+0000 7f8897842700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88900b12b0 con 0x7f88900cd0f0 2023-05-20T01:18:24.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:24.388+0000 7f8897842700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88900ced30 con 0x7f88900b0910 2023-05-20T01:18:24.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:24.388+0000 7f8897842700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88900ca1b0 con 0x7f88901599b0 2023-05-20T01:18:24.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:24.388+0000 7f8894ddd700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f88900cd0f0 0x7f88900ce620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:43870/0 (socket says 172.21.15.16:43870) 2023-05-20T01:18:24.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:24.388+0000 7f8894ddd700 1 -- 172.21.15.16:0/3778934120 learned_addr learned my addr 172.21.15.16:0/3778934120 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:18:24.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:24.388+0000 7f8887fff700 1 -- 172.21.15.16:0/3778934120 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 213215918 0 0) 0x7f88900b12b0 con 0x7f88900cd0f0 2023-05-20T01:18:24.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:24.388+0000 7f8887fff700 1 -- 172.21.15.16:0/3778934120 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f887c0035b0 con 0x7f88900cd0f0 2023-05-20T01:18:24.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:24.388+0000 7f8895ddf700 1 --2- 172.21.15.16:0/3778934120 >> v2:172.21.15.132:3300/0 conn(0x7f88900b0910 0x7f88900b0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:24.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:24.388+0000 7f88955de700 1 --2- 172.21.15.16:0/3778934120 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f88901599b0 0x7f889015ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:24.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:24.389+0000 7f8887fff700 1 -- 172.21.15.16:0/3778934120 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2572891744 0 0) 0x7f887c0035b0 con 0x7f88900cd0f0 2023-05-20T01:18:24.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:24.389+0000 7f8887fff700 1 -- 172.21.15.16:0/3778934120 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f88901599b0 msgr2=0x7f889015ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:18:24.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:24.389+0000 7f8887fff700 1 --2- 172.21.15.16:0/3778934120 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f88901599b0 0x7f889015ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:24.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:24.389+0000 7f8887fff700 1 -- 172.21.15.16:0/3778934120 >> v2:172.21.15.132:3300/0 conn(0x7f88900b0910 msgr2=0x7f88900b0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:18:24.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:24.389+0000 7f8887fff700 1 --2- 172.21.15.16:0/3778934120 >> v2:172.21.15.132:3300/0 conn(0x7f88900b0910 0x7f88900b0d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:24.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:24.389+0000 7f8887fff700 1 -- 172.21.15.16:0/3778934120 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f889015e300 con 0x7f88900cd0f0 2023-05-20T01:18:24.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:24.389+0000 7f8887fff700 1 -- 172.21.15.16:0/3778934120 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7f888c001670 con 0x7f88900cd0f0 2023-05-20T01:18:24.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:24.389+0000 7f8887fff700 1 -- 172.21.15.16:0/3778934120 >> v1:172.21.15.16:6789/0 conn(0x7f88900cd0f0 legacy=0x7f88900ce620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:24.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:24.389+0000 7f8887fff700 1 --2- 172.21.15.16:0/3778934120 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f887c003a70 0x7f887c003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:24.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:24.389+0000 7f8887fff700 1 --2- 172.21.15.16:0/3778934120 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f88901599b0 0x7f887c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:24.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:24.389+0000 7f8887fff700 1 --2- 172.21.15.16:0/3778934120 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f887c004d90 0x7f887c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:24.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:24.389+0000 7f8887fff700 1 -- 172.21.15.16:0/3778934120 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88900ced30 con 0x7f887c003a70 2023-05-20T01:18:24.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:24.389+0000 7f8887fff700 1 -- 172.21.15.16:0/3778934120 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88900ca1b0 con 0x7f887c004d90 2023-05-20T01:18:24.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:24.389+0000 7f8887fff700 1 -- 172.21.15.16:0/3778934120 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f887c00ba10 con 0x7f88901599b0 2023-05-20T01:18:24.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:24.389+0000 7f88955de700 1 --2- 172.21.15.16:0/3778934120 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f887c003a70 0x7f887c003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:24.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:24.389+0000 7f8894ddd700 1 --2- 172.21.15.16:0/3778934120 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f88901599b0 0x7f887c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:24.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:24.389+0000 7f8895ddf700 1 --2- 172.21.15.16:0/3778934120 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f887c004d90 0x7f887c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:24.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:24.389+0000 7f88955de700 1 -- 172.21.15.16:0/3778934120 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f88901599b0 msgr2=0x7f887c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:18:24.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:24.389+0000 7f88955de700 1 --2- 172.21.15.16:0/3778934120 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f88901599b0 0x7f887c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:24.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:24.389+0000 7f88955de700 1 -- 172.21.15.16:0/3778934120 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f887c004d90 msgr2=0x7f887c005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:18:24.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:24.389+0000 7f88955de700 1 --2- 172.21.15.16:0/3778934120 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f887c004d90 0x7f887c005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:24.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:24.389+0000 7f88955de700 1 -- 172.21.15.16:0/3778934120 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f887c0035b0 con 0x7f887c003a70 2023-05-20T01:18:24.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:24.389+0000 7f8895ddf700 1 --2- 172.21.15.16:0/3778934120 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f887c004d90 0x7f887c005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:18:24.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:24.389+0000 7f88955de700 1 --2- 172.21.15.16:0/3778934120 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f887c003a70 0x7f887c003e80 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f8880009b90 tx=0x7f887c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1a3433164968436 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:24.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:24.390+0000 7f8887fff700 1 -- 172.21.15.16:0/3778934120 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f888000e660 con 0x7f887c003a70 2023-05-20T01:18:24.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:24.390+0000 7f8887fff700 1 -- 172.21.15.16:0/3778934120 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f888000e7c0 con 0x7f887c003a70 2023-05-20T01:18:24.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:24.390+0000 7f8887fff700 1 -- 172.21.15.16:0/3778934120 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f888000eb30 con 0x7f887c003a70 2023-05-20T01:18:24.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:24.390+0000 7f8897842700 1 -- 172.21.15.16:0/3778934120 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f887c003a70 msgr2=0x7f887c003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:24.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:24.390+0000 7f8897842700 1 --2- 172.21.15.16:0/3778934120 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f887c003a70 0x7f887c003e80 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f8880009b90 tx=0x7f887c005880 comp rx=0 tx=0).stop 2023-05-20T01:18:24.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:24.390+0000 7f8895ddf700 1 -- 172.21.15.16:0/3778934120 reap_dead start 2023-05-20T01:18:24.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:24.390+0000 7f8897842700 1 -- 172.21.15.16:0/3778934120 shutdown_connections 2023-05-20T01:18:24.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:24.390+0000 7f8897842700 1 -- 172.21.15.16:0/3778934120 >> 172.21.15.16:0/3778934120 conn(0x7f88900cb910 msgr2=0x7f88900cbd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:18:24.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:24.390+0000 7f8897842700 1 -- 172.21.15.16:0/3778934120 shutdown_connections 2023-05-20T01:18:24.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:24.390+0000 7f8897842700 1 -- 172.21.15.16:0/3778934120 wait complete. 2023-05-20T01:18:24.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:24.391+0000 7f8897842700 1 Processor -- start 2023-05-20T01:18:24.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:24.392+0000 7f8897842700 1 -- start start 2023-05-20T01:18:24.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:24.392+0000 7f8897842700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f887c003a70 0x7f8890159d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:24.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:24.392+0000 7f8897842700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8890157fd0 0x7f88900c2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:24.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:24.392+0000 7f8897842700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8890149010 0x7f88901f1620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:24.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:24.392+0000 7f8897842700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8890149860 con 0x7f8890157fd0 2023-05-20T01:18:24.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:24.392+0000 7f8897842700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8890151250 con 0x7f887c003a70 2023-05-20T01:18:24.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:24.392+0000 7f8897842700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8890150920 con 0x7f8890149010 2023-05-20T01:18:24.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:24.392+0000 7f8894ddd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8890157fd0 0x7f88900c2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:24.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:24.392+0000 7f8894ddd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8890157fd0 0x7f88900c2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60588/0 (socket says 172.21.15.16:60588) 2023-05-20T01:18:24.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:24.392+0000 7f8894ddd700 1 -- 172.21.15.16:0/588374172 learned_addr learned my addr 172.21.15.16:0/588374172 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:18:24.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:24.392+0000 7f88955de700 1 --2- 172.21.15.16:0/588374172 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f887c003a70 0x7f8890159d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:24.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:24.392+0000 7f8895ddf700 1 --2- 172.21.15.16:0/588374172 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8890149010 0x7f88901f1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:24.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:24.393+0000 7f8894ddd700 1 -- 172.21.15.16:0/588374172 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8890149010 msgr2=0x7f88901f1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:24.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:24.393+0000 7f8894ddd700 1 --2- 172.21.15.16:0/588374172 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8890149010 0x7f88901f1620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:24.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:24.393+0000 7f8894ddd700 1 -- 172.21.15.16:0/588374172 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f887c003a70 msgr2=0x7f8890159d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:24.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:24.393+0000 7f8894ddd700 1 --2- 172.21.15.16:0/588374172 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f887c003a70 0x7f8890159d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:24.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:24.393+0000 7f8894ddd700 1 -- 172.21.15.16:0/588374172 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88900cd860 con 0x7f8890157fd0 2023-05-20T01:18:24.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:24.393+0000 7f8894ddd700 1 --2- 172.21.15.16:0/588374172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8890157fd0 0x7f88900c2cb0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f888c007340 tx=0x7f888c007650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:24.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:24.393+0000 7f8885ffb700 1 -- 172.21.15.16:0/588374172 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f888c011a70 con 0x7f8890157fd0 2023-05-20T01:18:24.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:24.393+0000 7f8885ffb700 1 -- 172.21.15.16:0/588374172 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f888c011bf0 con 0x7f8890157fd0 2023-05-20T01:18:24.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:24.393+0000 7f8897842700 1 -- 172.21.15.16:0/588374172 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88900cd580 con 0x7f8890157fd0 2023-05-20T01:18:24.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:24.393+0000 7f8885ffb700 1 -- 172.21.15.16:0/588374172 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f888c012450 con 0x7f8890157fd0 2023-05-20T01:18:24.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:24.393+0000 7f8897842700 1 -- 172.21.15.16:0/588374172 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88900cdb40 con 0x7f8890157fd0 2023-05-20T01:18:24.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:24.394+0000 7f8897842700 1 -- 172.21.15.16:0/588374172 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8890151530 con 0x7f8890157fd0 2023-05-20T01:18:24.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:24.395+0000 7f8885ffb700 1 -- 172.21.15.16:0/588374172 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f888c0125b0 con 0x7f8890157fd0 2023-05-20T01:18:24.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:24.395+0000 7f8885ffb700 1 --2- 172.21.15.16:0/588374172 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f887406ace0 0x7f887406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:24.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:24.395+0000 7f8885ffb700 1 -- 172.21.15.16:0/588374172 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6844+0+0 (secure 0 0 0) 0x7f888c08c620 con 0x7f8890157fd0 2023-05-20T01:18:24.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:24.395+0000 7f88955de700 1 --2- 172.21.15.16:0/588374172 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f887406ace0 0x7f887406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:24.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:24.395+0000 7f88955de700 1 --2- 172.21.15.16:0/588374172 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f887406ace0 0x7f887406d190 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f88800090b0 tx=0x7f888001f040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:24.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:24.399+0000 7f8885ffb700 1 -- 172.21.15.16:0/588374172 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f888c057450 con 0x7f8890157fd0 2023-05-20T01:18:24.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:24.643+0000 7f8897842700 1 -- 172.21.15.16:0/588374172 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f8890151390 con 0x7f8890157fd0 2023-05-20T01:18:25.453 INFO:tasks.ceph.mon.a.smithi016.stderr:2023-05-20T01:18:25.449+0000 7f49eb8ae700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-05-20T01:18:25.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.458+0000 7f8885ffb700 1 -- 172.21.15.16:0/588374172 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v18) v1 ==== 161+0+0 (secure 0 0 0) 0x7f8890151390 con 0x7f8890157fd0 2023-05-20T01:18:25.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.461+0000 7f8897842700 1 -- 172.21.15.16:0/588374172 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f887406ace0 msgr2=0x7f887406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:25.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.461+0000 7f8897842700 1 --2- 172.21.15.16:0/588374172 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f887406ace0 0x7f887406d190 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f88800090b0 tx=0x7f888001f040 comp rx=0 tx=0).stop 2023-05-20T01:18:25.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.461+0000 7f8897842700 1 -- 172.21.15.16:0/588374172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8890157fd0 msgr2=0x7f88900c2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:25.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.461+0000 7f8897842700 1 --2- 172.21.15.16:0/588374172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8890157fd0 0x7f88900c2cb0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f888c007340 tx=0x7f888c007650 comp rx=0 tx=0).stop 2023-05-20T01:18:25.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.461+0000 7f8897842700 1 -- 172.21.15.16:0/588374172 shutdown_connections 2023-05-20T01:18:25.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.461+0000 7f8897842700 1 --2- 172.21.15.16:0/588374172 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f887406ace0 0x7f887406d190 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:25.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.461+0000 7f8897842700 1 --2- 172.21.15.16:0/588374172 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8890149010 0x7f88901f1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:25.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.461+0000 7f8897842700 1 --2- 172.21.15.16:0/588374172 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f887c003a70 0x7f8890159d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:25.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.461+0000 7f8897842700 1 --2- 172.21.15.16:0/588374172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8890157fd0 0x7f88900c2cb0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:25.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.461+0000 7f8897842700 1 -- 172.21.15.16:0/588374172 >> 172.21.15.16:0/588374172 conn(0x7f88900cb910 msgr2=0x7f8890152c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:18:25.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.461+0000 7f8897842700 1 -- 172.21.15.16:0/588374172 shutdown_connections 2023-05-20T01:18:25.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.461+0000 7f8897842700 1 -- 172.21.15.16:0/588374172 wait complete. 2023-05-20T01:18:25.466 INFO:teuthology.orchestra.run.smithi016.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-05-20T01:18:25.481 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-05-20T01:18:25.483 DEBUG:teuthology.orchestra.run.smithi132:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-05-20T01:18:25.504 INFO:tasks.ceph.mds.c.smithi016.stdout:starting mds.c at 2023-05-20T01:18:25.535 INFO:teuthology.orchestra.run.smithi016.stdout:[global] 2023-05-20T01:18:25.535 INFO:teuthology.orchestra.run.smithi016.stdout: chdir = "" 2023-05-20T01:18:25.535 INFO:teuthology.orchestra.run.smithi016.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-05-20T01:18:25.535 INFO:teuthology.orchestra.run.smithi016.stdout: auth supported = cephx 2023-05-20T01:18:25.535 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:18:25.536 INFO:teuthology.orchestra.run.smithi016.stdout: filestore xattr use omap = true 2023-05-20T01:18:25.536 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:18:25.536 INFO:teuthology.orchestra.run.smithi016.stdout: mon clock drift allowed = 1.000 2023-05-20T01:18:25.536 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:18:25.536 INFO:teuthology.orchestra.run.smithi016.stdout: osd crush chooseleaf type = 0 2023-05-20T01:18:25.536 INFO:teuthology.orchestra.run.smithi016.stdout: auth debug = true 2023-05-20T01:18:25.536 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:18:25.537 INFO:teuthology.orchestra.run.smithi016.stdout: ms die on old message = true 2023-05-20T01:18:25.537 INFO:teuthology.orchestra.run.smithi016.stdout: ms die on bug = true 2023-05-20T01:18:25.537 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:18:25.537 INFO:teuthology.orchestra.run.smithi016.stdout: mon max pg per osd = 10000 # >= luminous 2023-05-20T01:18:25.537 INFO:teuthology.orchestra.run.smithi016.stdout: mon pg warn max object skew = 0 2023-05-20T01:18:25.537 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:18:25.537 INFO:teuthology.orchestra.run.smithi016.stdout: # disable pg_autoscaler by default for new pools 2023-05-20T01:18:25.537 INFO:teuthology.orchestra.run.smithi016.stdout: osd_pool_default_pg_autoscale_mode = off 2023-05-20T01:18:25.538 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:18:25.538 INFO:teuthology.orchestra.run.smithi016.stdout: osd pool default size = 2 2023-05-20T01:18:25.538 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:18:25.538 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd allow primary affinity = true 2023-05-20T01:18:25.538 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd allow pg remap = true 2023-05-20T01:18:25.538 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on legacy crush tunables = false 2023-05-20T01:18:25.538 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on crush straw calc version zero = false 2023-05-20T01:18:25.538 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on no sortbitwise = false 2023-05-20T01:18:25.539 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on osd down out interval zero = false 2023-05-20T01:18:25.539 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on too few osds = false 2023-05-20T01:18:25.539 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-05-20T01:18:25.539 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_pool_no_redundancy = false 2023-05-20T01:18:25.539 INFO:teuthology.orchestra.run.smithi016.stdout: mon_allow_pool_size_one = true 2023-05-20T01:18:25.539 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:18:25.539 INFO:teuthology.orchestra.run.smithi016.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-05-20T01:18:25.539 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:18:25.540 INFO:teuthology.orchestra.run.smithi016.stdout: osd default data pool replay window = 5 2023-05-20T01:18:25.540 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:18:25.540 INFO:teuthology.orchestra.run.smithi016.stdout: mon allow pool delete = true 2023-05-20T01:18:25.540 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:18:25.540 INFO:teuthology.orchestra.run.smithi016.stdout: mon cluster log file level = debug 2023-05-20T01:18:25.541 INFO:teuthology.orchestra.run.smithi016.stdout: debug asserts on shutdown = true 2023-05-20T01:18:25.541 INFO:teuthology.orchestra.run.smithi016.stdout: mon health detail to clog = false 2023-05-20T01:18:25.541 INFO:teuthology.orchestra.run.smithi016.stdout: mon host = "172.21.15.16,172.21.15.132,[v2:172.21.15.132:3301,v1:172.21.15.132:6790]" 2023-05-20T01:18:25.541 INFO:teuthology.orchestra.run.smithi016.stdout: ms die on skipped message = False 2023-05-20T01:18:25.542 INFO:teuthology.orchestra.run.smithi016.stdout: fsid = c2245cc5-ecab-4087-a59b-bc81db901676 2023-05-20T01:18:25.542 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:18:25.542 INFO:teuthology.orchestra.run.smithi016.stdout:[osd] 2023-05-20T01:18:25.543 INFO:teuthology.orchestra.run.smithi016.stdout: osd journal size = 100 2023-05-20T01:18:25.543 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:18:25.543 INFO:teuthology.orchestra.run.smithi016.stdout: osd scrub load threshold = 5.0 2023-05-20T01:18:25.543 INFO:teuthology.orchestra.run.smithi016.stdout: osd scrub max interval = 600 2023-05-20T01:18:25.544 INFO:teuthology.orchestra.run.smithi016.stdout: osd mclock profile = high_recovery_ops 2023-05-20T01:18:25.544 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:18:25.544 INFO:teuthology.orchestra.run.smithi016.stdout: osd recover clone overlap = true 2023-05-20T01:18:25.545 INFO:teuthology.orchestra.run.smithi016.stdout: osd recovery max chunk = 1048576 2023-05-20T01:18:25.545 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:18:25.545 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug shutdown = true 2023-05-20T01:18:25.545 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug op order = true 2023-05-20T01:18:25.546 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug verify stray on activate = true 2023-05-20T01:18:25.546 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:18:25.546 INFO:teuthology.orchestra.run.smithi016.stdout: osd open classes on start = true 2023-05-20T01:18:25.547 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug pg log writeout = true 2023-05-20T01:18:25.547 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:18:25.547 INFO:teuthology.orchestra.run.smithi016.stdout: osd deep scrub update digest min age = 30 2023-05-20T01:18:25.547 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:18:25.548 INFO:teuthology.orchestra.run.smithi016.stdout: osd map max advance = 10 2023-05-20T01:18:25.548 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:18:25.548 INFO:teuthology.orchestra.run.smithi016.stdout: journal zero on create = true 2023-05-20T01:18:25.548 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:18:25.549 INFO:teuthology.orchestra.run.smithi016.stdout: filestore ondisk finisher threads = 3 2023-05-20T01:18:25.549 INFO:teuthology.orchestra.run.smithi016.stdout: filestore apply finisher threads = 3 2023-05-20T01:18:25.549 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:18:25.549 INFO:teuthology.orchestra.run.smithi016.stdout: bdev debug aio = true 2023-05-20T01:18:25.550 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug misdirected ops = true 2023-05-20T01:18:25.550 INFO:teuthology.orchestra.run.smithi016.stdout: bluestore block size = 96636764160 2023-05-20T01:18:25.550 INFO:teuthology.orchestra.run.smithi016.stdout: bluestore fsck on mount = True 2023-05-20T01:18:25.551 INFO:teuthology.orchestra.run.smithi016.stdout: debug bluefs = 20 2023-05-20T01:18:25.551 INFO:teuthology.orchestra.run.smithi016.stdout: debug bluestore = 20 2023-05-20T01:18:25.551 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2023-05-20T01:18:25.551 INFO:teuthology.orchestra.run.smithi016.stdout: debug osd = 20 2023-05-20T01:18:25.552 INFO:teuthology.orchestra.run.smithi016.stdout: debug rocksdb = 10 2023-05-20T01:18:25.552 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd backfillfull_ratio = 0.85 2023-05-20T01:18:25.552 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd full ratio = 0.9 2023-05-20T01:18:25.552 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd nearfull ratio = 0.8 2023-05-20T01:18:25.552 INFO:teuthology.orchestra.run.smithi016.stdout: osd failsafe full ratio = 0.95 2023-05-20T01:18:25.553 INFO:teuthology.orchestra.run.smithi016.stdout: osd objectstore = bluestore 2023-05-20T01:18:25.553 INFO:teuthology.orchestra.run.smithi016.stdout: osd op complaint time = 180 2023-05-20T01:18:25.553 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:18:25.553 INFO:teuthology.orchestra.run.smithi016.stdout:[mgr] 2023-05-20T01:18:25.554 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2023-05-20T01:18:25.554 INFO:teuthology.orchestra.run.smithi016.stdout: debug mgr = 20 2023-05-20T01:18:25.554 INFO:teuthology.orchestra.run.smithi016.stdout: debug mon = 20 2023-05-20T01:18:25.554 INFO:teuthology.orchestra.run.smithi016.stdout: debug auth = 20 2023-05-20T01:18:25.554 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min pgs per osd = 4 2023-05-20T01:18:25.555 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min bytes per osd = 10 2023-05-20T01:18:25.555 INFO:teuthology.orchestra.run.smithi016.stdout: mgr/telemetry/nag = false 2023-05-20T01:18:25.555 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:18:25.556 INFO:teuthology.orchestra.run.smithi016.stdout:[mon] 2023-05-20T01:18:25.556 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2023-05-20T01:18:25.556 INFO:teuthology.orchestra.run.smithi016.stdout: debug mon = 20 2023-05-20T01:18:25.556 INFO:teuthology.orchestra.run.smithi016.stdout: debug paxos = 20 2023-05-20T01:18:25.557 INFO:teuthology.orchestra.run.smithi016.stdout: debug auth = 20 2023-05-20T01:18:25.557 INFO:teuthology.orchestra.run.smithi016.stdout: mon data avail warn = 5 2023-05-20T01:18:25.557 INFO:teuthology.orchestra.run.smithi016.stdout: mon mgr mkfs grace = 240 2023-05-20T01:18:25.558 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min pgs per osd = 4 2023-05-20T01:18:25.558 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd reporter subtree level = osd 2023-05-20T01:18:25.558 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd prime pg temp = true 2023-05-20T01:18:25.558 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min bytes per osd = 10 2023-05-20T01:18:25.559 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:18:25.559 INFO:teuthology.orchestra.run.smithi016.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-05-20T01:18:25.559 INFO:teuthology.orchestra.run.smithi016.stdout: auth mon ticket ttl = 660 # 11m 2023-05-20T01:18:25.559 INFO:teuthology.orchestra.run.smithi016.stdout: auth service ticket ttl = 240 # 4m 2023-05-20T01:18:25.560 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:18:25.560 INFO:teuthology.orchestra.run.smithi016.stdout: # don't complain about insecure global_id in the test suite 2023-05-20T01:18:25.560 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-05-20T01:18:25.561 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-05-20T01:18:25.561 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:18:25.561 INFO:teuthology.orchestra.run.smithi016.stdout: # 1m isn't quite enough 2023-05-20T01:18:25.561 INFO:teuthology.orchestra.run.smithi016.stdout: mon_down_mkfs_grace = 2m 2023-05-20T01:18:25.562 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:18:25.562 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_filestore_osds = false 2023-05-20T01:18:25.562 INFO:teuthology.orchestra.run.smithi016.stdout: mon op complaint time = 120 2023-05-20T01:18:25.563 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:18:25.563 INFO:teuthology.orchestra.run.smithi016.stdout:[client] 2023-05-20T01:18:25.563 INFO:teuthology.orchestra.run.smithi016.stdout: rgw cache enabled = true 2023-05-20T01:18:25.563 INFO:teuthology.orchestra.run.smithi016.stdout: rgw enable ops log = true 2023-05-20T01:18:25.563 INFO:teuthology.orchestra.run.smithi016.stdout: rgw enable usage log = true 2023-05-20T01:18:25.563 INFO:teuthology.orchestra.run.smithi016.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-05-20T01:18:25.564 INFO:teuthology.orchestra.run.smithi016.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-05-20T01:18:25.564 INFO:teuthology.orchestra.run.smithi016.stdout: client mount timeout = 600 2023-05-20T01:18:25.564 INFO:teuthology.orchestra.run.smithi016.stdout: debug client = 20 2023-05-20T01:18:25.564 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2023-05-20T01:18:25.564 INFO:teuthology.orchestra.run.smithi016.stdout: pid file = "" 2023-05-20T01:18:25.564 INFO:teuthology.orchestra.run.smithi016.stdout: rados mon op timeout = 900 2023-05-20T01:18:25.564 INFO:teuthology.orchestra.run.smithi016.stdout: rados osd op timeout = 900 2023-05-20T01:18:25.565 INFO:teuthology.orchestra.run.smithi016.stdout:[mon.a] 2023-05-20T01:18:25.565 INFO:teuthology.orchestra.run.smithi016.stdout:[mon.b] 2023-05-20T01:18:25.565 INFO:teuthology.orchestra.run.smithi016.stdout:[mon.c] 2023-05-20T01:18:25.565 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.a] 2023-05-20T01:18:25.565 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.c] 2023-05-20T01:18:25.565 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.b] 2023-05-20T01:18:25.565 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.d] 2023-05-20T01:18:25.565 INFO:teuthology.orchestra.run.smithi016.stdout:[mds] 2023-05-20T01:18:25.566 INFO:teuthology.orchestra.run.smithi016.stdout: debug mds = 20 2023-05-20T01:18:25.566 INFO:teuthology.orchestra.run.smithi016.stdout: debug mds balancer = 20 2023-05-20T01:18:25.566 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2023-05-20T01:18:25.566 INFO:teuthology.orchestra.run.smithi016.stdout: mds debug frag = True 2023-05-20T01:18:25.566 INFO:teuthology.orchestra.run.smithi016.stdout: mds debug scatterstat = True 2023-05-20T01:18:25.566 INFO:teuthology.orchestra.run.smithi016.stdout: mds op complaint time = 180 2023-05-20T01:18:25.566 INFO:teuthology.orchestra.run.smithi016.stdout: mds verify scatter = False 2023-05-20T01:18:25.567 INFO:teuthology.orchestra.run.smithi016.stdout: osd op complaint time = 180 2023-05-20T01:18:25.567 INFO:teuthology.orchestra.run.smithi016.stdout: rados mon op timeout = 900 2023-05-20T01:18:25.567 INFO:teuthology.orchestra.run.smithi016.stdout: rados osd op timeout = 900 2023-05-20T01:18:25.568 INFO:teuthology.orchestra.run.smithi132.stdout:[global] 2023-05-20T01:18:25.568 INFO:teuthology.orchestra.run.smithi132.stdout: chdir = "" 2023-05-20T01:18:25.568 INFO:teuthology.orchestra.run.smithi132.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-05-20T01:18:25.569 INFO:teuthology.orchestra.run.smithi132.stdout: auth supported = cephx 2023-05-20T01:18:25.569 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:18:25.569 INFO:teuthology.orchestra.run.smithi132.stdout: filestore xattr use omap = true 2023-05-20T01:18:25.569 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:18:25.569 INFO:teuthology.orchestra.run.smithi132.stdout: mon clock drift allowed = 1.000 2023-05-20T01:18:25.569 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:18:25.569 INFO:teuthology.orchestra.run.smithi132.stdout: osd crush chooseleaf type = 0 2023-05-20T01:18:25.570 INFO:teuthology.orchestra.run.smithi132.stdout: auth debug = true 2023-05-20T01:18:25.570 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:18:25.570 INFO:teuthology.orchestra.run.smithi132.stdout: ms die on old message = true 2023-05-20T01:18:25.570 INFO:teuthology.orchestra.run.smithi132.stdout: ms die on bug = true 2023-05-20T01:18:25.570 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:18:25.570 INFO:teuthology.orchestra.run.smithi132.stdout: mon max pg per osd = 10000 # >= luminous 2023-05-20T01:18:25.570 INFO:teuthology.orchestra.run.smithi132.stdout: mon pg warn max object skew = 0 2023-05-20T01:18:25.570 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:18:25.571 INFO:teuthology.orchestra.run.smithi132.stdout: # disable pg_autoscaler by default for new pools 2023-05-20T01:18:25.571 INFO:teuthology.orchestra.run.smithi132.stdout: osd_pool_default_pg_autoscale_mode = off 2023-05-20T01:18:25.571 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:18:25.571 INFO:teuthology.orchestra.run.smithi132.stdout: osd pool default size = 2 2023-05-20T01:18:25.571 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:18:25.571 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd allow primary affinity = true 2023-05-20T01:18:25.571 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd allow pg remap = true 2023-05-20T01:18:25.571 INFO:teuthology.orchestra.run.smithi132.stdout: mon warn on legacy crush tunables = false 2023-05-20T01:18:25.572 INFO:teuthology.orchestra.run.smithi132.stdout: mon warn on crush straw calc version zero = false 2023-05-20T01:18:25.572 INFO:teuthology.orchestra.run.smithi132.stdout: mon warn on no sortbitwise = false 2023-05-20T01:18:25.572 INFO:teuthology.orchestra.run.smithi132.stdout: mon warn on osd down out interval zero = false 2023-05-20T01:18:25.572 INFO:teuthology.orchestra.run.smithi132.stdout: mon warn on too few osds = false 2023-05-20T01:18:25.572 INFO:teuthology.orchestra.run.smithi132.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-05-20T01:18:25.572 INFO:teuthology.orchestra.run.smithi132.stdout: mon_warn_on_pool_no_redundancy = false 2023-05-20T01:18:25.572 INFO:teuthology.orchestra.run.smithi132.stdout: mon_allow_pool_size_one = true 2023-05-20T01:18:25.573 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:18:25.573 INFO:teuthology.orchestra.run.smithi132.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-05-20T01:18:25.573 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:18:25.573 INFO:teuthology.orchestra.run.smithi132.stdout: osd default data pool replay window = 5 2023-05-20T01:18:25.573 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:18:25.573 INFO:teuthology.orchestra.run.smithi132.stdout: mon allow pool delete = true 2023-05-20T01:18:25.574 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:18:25.574 INFO:teuthology.orchestra.run.smithi132.stdout: mon cluster log file level = debug 2023-05-20T01:18:25.574 INFO:teuthology.orchestra.run.smithi132.stdout: debug asserts on shutdown = true 2023-05-20T01:18:25.574 INFO:teuthology.orchestra.run.smithi132.stdout: mon health detail to clog = false 2023-05-20T01:18:25.575 INFO:teuthology.orchestra.run.smithi132.stdout: mon host = "172.21.15.16,172.21.15.132,[v2:172.21.15.132:3301,v1:172.21.15.132:6790]" 2023-05-20T01:18:25.575 INFO:teuthology.orchestra.run.smithi132.stdout: ms die on skipped message = False 2023-05-20T01:18:25.575 INFO:teuthology.orchestra.run.smithi132.stdout: fsid = c2245cc5-ecab-4087-a59b-bc81db901676 2023-05-20T01:18:25.575 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:18:25.576 INFO:teuthology.orchestra.run.smithi132.stdout:[osd] 2023-05-20T01:18:25.576 INFO:teuthology.orchestra.run.smithi132.stdout: osd journal size = 100 2023-05-20T01:18:25.576 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:18:25.576 INFO:teuthology.orchestra.run.smithi132.stdout: osd scrub load threshold = 5.0 2023-05-20T01:18:25.577 INFO:teuthology.orchestra.run.smithi132.stdout: osd scrub max interval = 600 2023-05-20T01:18:25.577 INFO:teuthology.orchestra.run.smithi132.stdout: osd mclock profile = high_recovery_ops 2023-05-20T01:18:25.577 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:18:25.577 INFO:teuthology.orchestra.run.smithi132.stdout: osd recover clone overlap = true 2023-05-20T01:18:25.577 INFO:teuthology.orchestra.run.smithi132.stdout: osd recovery max chunk = 1048576 2023-05-20T01:18:25.577 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:18:25.578 INFO:teuthology.orchestra.run.smithi132.stdout: osd debug shutdown = true 2023-05-20T01:18:25.578 INFO:teuthology.orchestra.run.smithi132.stdout: osd debug op order = true 2023-05-20T01:18:25.578 INFO:teuthology.orchestra.run.smithi132.stdout: osd debug verify stray on activate = true 2023-05-20T01:18:25.578 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:18:25.578 INFO:teuthology.orchestra.run.smithi132.stdout: osd open classes on start = true 2023-05-20T01:18:25.578 INFO:teuthology.orchestra.run.smithi132.stdout: osd debug pg log writeout = true 2023-05-20T01:18:25.578 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:18:25.578 INFO:teuthology.orchestra.run.smithi132.stdout: osd deep scrub update digest min age = 30 2023-05-20T01:18:25.579 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:18:25.579 INFO:teuthology.orchestra.run.smithi132.stdout: osd map max advance = 10 2023-05-20T01:18:25.579 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:18:25.579 INFO:teuthology.orchestra.run.smithi132.stdout: journal zero on create = true 2023-05-20T01:18:25.579 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:18:25.579 INFO:teuthology.orchestra.run.smithi132.stdout: filestore ondisk finisher threads = 3 2023-05-20T01:18:25.579 INFO:teuthology.orchestra.run.smithi132.stdout: filestore apply finisher threads = 3 2023-05-20T01:18:25.579 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:18:25.580 INFO:teuthology.orchestra.run.smithi132.stdout: bdev debug aio = true 2023-05-20T01:18:25.580 INFO:teuthology.orchestra.run.smithi132.stdout: osd debug misdirected ops = true 2023-05-20T01:18:25.580 INFO:teuthology.orchestra.run.smithi132.stdout: bluestore block size = 96636764160 2023-05-20T01:18:25.580 INFO:teuthology.orchestra.run.smithi132.stdout: bluestore fsck on mount = True 2023-05-20T01:18:25.580 INFO:teuthology.orchestra.run.smithi132.stdout: debug bluefs = 20 2023-05-20T01:18:25.580 INFO:teuthology.orchestra.run.smithi132.stdout: debug bluestore = 20 2023-05-20T01:18:25.580 INFO:teuthology.orchestra.run.smithi132.stdout: debug ms = 1 2023-05-20T01:18:25.580 INFO:teuthology.orchestra.run.smithi132.stdout: debug osd = 20 2023-05-20T01:18:25.581 INFO:teuthology.orchestra.run.smithi132.stdout: debug rocksdb = 10 2023-05-20T01:18:25.581 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd backfillfull_ratio = 0.85 2023-05-20T01:18:25.581 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd full ratio = 0.9 2023-05-20T01:18:25.581 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd nearfull ratio = 0.8 2023-05-20T01:18:25.581 INFO:teuthology.orchestra.run.smithi132.stdout: osd failsafe full ratio = 0.95 2023-05-20T01:18:25.581 INFO:teuthology.orchestra.run.smithi132.stdout: osd objectstore = bluestore 2023-05-20T01:18:25.581 INFO:teuthology.orchestra.run.smithi132.stdout: osd op complaint time = 180 2023-05-20T01:18:25.582 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:18:25.582 INFO:teuthology.orchestra.run.smithi132.stdout:[mgr] 2023-05-20T01:18:25.582 INFO:teuthology.orchestra.run.smithi132.stdout: debug ms = 1 2023-05-20T01:18:25.582 INFO:teuthology.orchestra.run.smithi132.stdout: debug mgr = 20 2023-05-20T01:18:25.582 INFO:teuthology.orchestra.run.smithi132.stdout: debug mon = 20 2023-05-20T01:18:25.582 INFO:teuthology.orchestra.run.smithi132.stdout: debug auth = 20 2023-05-20T01:18:25.582 INFO:teuthology.orchestra.run.smithi132.stdout: mon reweight min pgs per osd = 4 2023-05-20T01:18:25.582 INFO:teuthology.orchestra.run.smithi132.stdout: mon reweight min bytes per osd = 10 2023-05-20T01:18:25.583 INFO:teuthology.orchestra.run.smithi132.stdout: mgr/telemetry/nag = false 2023-05-20T01:18:25.583 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:18:25.583 INFO:teuthology.orchestra.run.smithi132.stdout:[mon] 2023-05-20T01:18:25.583 INFO:teuthology.orchestra.run.smithi132.stdout: debug ms = 1 2023-05-20T01:18:25.583 INFO:teuthology.orchestra.run.smithi132.stdout: debug mon = 20 2023-05-20T01:18:25.583 INFO:teuthology.orchestra.run.smithi132.stdout: debug paxos = 20 2023-05-20T01:18:25.583 INFO:teuthology.orchestra.run.smithi132.stdout: debug auth = 20 2023-05-20T01:18:25.583 INFO:teuthology.orchestra.run.smithi132.stdout: mon data avail warn = 5 2023-05-20T01:18:25.584 INFO:teuthology.orchestra.run.smithi132.stdout: mon mgr mkfs grace = 240 2023-05-20T01:18:25.584 INFO:teuthology.orchestra.run.smithi132.stdout: mon reweight min pgs per osd = 4 2023-05-20T01:18:25.584 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd reporter subtree level = osd 2023-05-20T01:18:25.584 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd prime pg temp = true 2023-05-20T01:18:25.584 INFO:teuthology.orchestra.run.smithi132.stdout: mon reweight min bytes per osd = 10 2023-05-20T01:18:25.584 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:18:25.584 INFO:teuthology.orchestra.run.smithi132.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-05-20T01:18:25.585 INFO:teuthology.orchestra.run.smithi132.stdout: auth mon ticket ttl = 660 # 11m 2023-05-20T01:18:25.585 INFO:teuthology.orchestra.run.smithi132.stdout: auth service ticket ttl = 240 # 4m 2023-05-20T01:18:25.585 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:18:25.585 INFO:teuthology.orchestra.run.smithi132.stdout: # don't complain about insecure global_id in the test suite 2023-05-20T01:18:25.585 INFO:teuthology.orchestra.run.smithi132.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-05-20T01:18:25.585 INFO:teuthology.orchestra.run.smithi132.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-05-20T01:18:25.585 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:18:25.585 INFO:teuthology.orchestra.run.smithi132.stdout: # 1m isn't quite enough 2023-05-20T01:18:25.586 INFO:teuthology.orchestra.run.smithi132.stdout: mon_down_mkfs_grace = 2m 2023-05-20T01:18:25.586 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:18:25.586 INFO:teuthology.orchestra.run.smithi132.stdout: mon_warn_on_filestore_osds = false 2023-05-20T01:18:25.586 INFO:teuthology.orchestra.run.smithi132.stdout: mon op complaint time = 120 2023-05-20T01:18:25.586 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:18:25.586 INFO:teuthology.orchestra.run.smithi132.stdout:[client] 2023-05-20T01:18:25.586 INFO:teuthology.orchestra.run.smithi132.stdout: rgw cache enabled = true 2023-05-20T01:18:25.587 INFO:teuthology.orchestra.run.smithi132.stdout: rgw enable ops log = true 2023-05-20T01:18:25.587 INFO:teuthology.orchestra.run.smithi132.stdout: rgw enable usage log = true 2023-05-20T01:18:25.587 INFO:teuthology.orchestra.run.smithi132.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-05-20T01:18:25.587 INFO:teuthology.orchestra.run.smithi132.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-05-20T01:18:25.587 INFO:teuthology.orchestra.run.smithi132.stdout: client mount timeout = 600 2023-05-20T01:18:25.588 INFO:teuthology.orchestra.run.smithi132.stdout: debug client = 20 2023-05-20T01:18:25.588 INFO:teuthology.orchestra.run.smithi132.stdout: debug ms = 1 2023-05-20T01:18:25.588 INFO:teuthology.orchestra.run.smithi132.stdout: pid file = "" 2023-05-20T01:18:25.589 INFO:teuthology.orchestra.run.smithi132.stdout: rados mon op timeout = 900 2023-05-20T01:18:25.589 INFO:teuthology.orchestra.run.smithi132.stdout: rados osd op timeout = 900 2023-05-20T01:18:25.589 INFO:teuthology.orchestra.run.smithi132.stdout:[mon.a] 2023-05-20T01:18:25.589 INFO:teuthology.orchestra.run.smithi132.stdout:[mon.b] 2023-05-20T01:18:25.590 INFO:teuthology.orchestra.run.smithi132.stdout:[mon.c] 2023-05-20T01:18:25.590 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.a] 2023-05-20T01:18:25.590 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.c] 2023-05-20T01:18:25.591 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.b] 2023-05-20T01:18:25.591 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.d] 2023-05-20T01:18:25.591 INFO:teuthology.orchestra.run.smithi132.stdout:[mds] 2023-05-20T01:18:25.591 INFO:teuthology.orchestra.run.smithi132.stdout: debug mds = 20 2023-05-20T01:18:25.592 INFO:teuthology.orchestra.run.smithi132.stdout: debug mds balancer = 20 2023-05-20T01:18:25.592 INFO:teuthology.orchestra.run.smithi132.stdout: debug ms = 1 2023-05-20T01:18:25.592 INFO:teuthology.orchestra.run.smithi132.stdout: mds debug frag = True 2023-05-20T01:18:25.592 INFO:teuthology.orchestra.run.smithi132.stdout: mds debug scatterstat = True 2023-05-20T01:18:25.593 INFO:teuthology.orchestra.run.smithi132.stdout: mds op complaint time = 180 2023-05-20T01:18:25.593 INFO:teuthology.orchestra.run.smithi132.stdout: mds verify scatter = False 2023-05-20T01:18:25.593 INFO:teuthology.orchestra.run.smithi132.stdout: osd op complaint time = 180 2023-05-20T01:18:25.594 INFO:teuthology.orchestra.run.smithi132.stdout: rados mon op timeout = 900 2023-05-20T01:18:25.594 INFO:teuthology.orchestra.run.smithi132.stdout: rados osd op timeout = 900 2023-05-20T01:18:25.596 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-05-20T01:18:25.611 DEBUG:teuthology.orchestra.run.smithi132:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-05-20T01:18:25.658 INFO:teuthology.orchestra.run.smithi016.stdout:[global] 2023-05-20T01:18:25.659 INFO:teuthology.orchestra.run.smithi016.stdout: chdir = "" 2023-05-20T01:18:25.659 INFO:teuthology.orchestra.run.smithi016.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-05-20T01:18:25.659 INFO:teuthology.orchestra.run.smithi016.stdout: auth supported = cephx 2023-05-20T01:18:25.660 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:18:25.660 INFO:teuthology.orchestra.run.smithi016.stdout: filestore xattr use omap = true 2023-05-20T01:18:25.660 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:18:25.661 INFO:teuthology.orchestra.run.smithi016.stdout: mon clock drift allowed = 1.000 2023-05-20T01:18:25.661 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:18:25.661 INFO:teuthology.orchestra.run.smithi016.stdout: osd crush chooseleaf type = 0 2023-05-20T01:18:25.661 INFO:teuthology.orchestra.run.smithi016.stdout: auth debug = true 2023-05-20T01:18:25.661 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:18:25.661 INFO:teuthology.orchestra.run.smithi016.stdout: ms die on old message = true 2023-05-20T01:18:25.662 INFO:teuthology.orchestra.run.smithi016.stdout: ms die on bug = true 2023-05-20T01:18:25.662 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:18:25.662 INFO:teuthology.orchestra.run.smithi016.stdout: mon max pg per osd = 10000 # >= luminous 2023-05-20T01:18:25.662 INFO:teuthology.orchestra.run.smithi016.stdout: mon pg warn max object skew = 0 2023-05-20T01:18:25.662 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:18:25.662 INFO:teuthology.orchestra.run.smithi016.stdout: # disable pg_autoscaler by default for new pools 2023-05-20T01:18:25.662 INFO:teuthology.orchestra.run.smithi016.stdout: osd_pool_default_pg_autoscale_mode = off 2023-05-20T01:18:25.662 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:18:25.663 INFO:teuthology.orchestra.run.smithi016.stdout: osd pool default size = 2 2023-05-20T01:18:25.663 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:18:25.663 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd allow primary affinity = true 2023-05-20T01:18:25.663 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd allow pg remap = true 2023-05-20T01:18:25.663 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on legacy crush tunables = false 2023-05-20T01:18:25.663 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on crush straw calc version zero = false 2023-05-20T01:18:25.663 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on no sortbitwise = false 2023-05-20T01:18:25.663 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on osd down out interval zero = false 2023-05-20T01:18:25.664 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on too few osds = false 2023-05-20T01:18:25.664 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-05-20T01:18:25.664 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_pool_no_redundancy = false 2023-05-20T01:18:25.664 INFO:teuthology.orchestra.run.smithi016.stdout: mon_allow_pool_size_one = true 2023-05-20T01:18:25.664 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:18:25.664 INFO:teuthology.orchestra.run.smithi016.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-05-20T01:18:25.664 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:18:25.665 INFO:teuthology.orchestra.run.smithi016.stdout: osd default data pool replay window = 5 2023-05-20T01:18:25.665 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:18:25.665 INFO:teuthology.orchestra.run.smithi016.stdout: mon allow pool delete = true 2023-05-20T01:18:25.665 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:18:25.665 INFO:teuthology.orchestra.run.smithi016.stdout: mon cluster log file level = debug 2023-05-20T01:18:25.665 INFO:teuthology.orchestra.run.smithi016.stdout: debug asserts on shutdown = true 2023-05-20T01:18:25.665 INFO:teuthology.orchestra.run.smithi016.stdout: mon health detail to clog = false 2023-05-20T01:18:25.665 INFO:teuthology.orchestra.run.smithi016.stdout: mon host = "172.21.15.16,172.21.15.132,[v2:172.21.15.132:3301,v1:172.21.15.132:6790]" 2023-05-20T01:18:25.666 INFO:teuthology.orchestra.run.smithi016.stdout: ms die on skipped message = False 2023-05-20T01:18:25.666 INFO:teuthology.orchestra.run.smithi016.stdout: fsid = c2245cc5-ecab-4087-a59b-bc81db901676 2023-05-20T01:18:25.666 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:18:25.666 INFO:teuthology.orchestra.run.smithi016.stdout:[osd] 2023-05-20T01:18:25.666 INFO:teuthology.orchestra.run.smithi016.stdout: osd journal size = 100 2023-05-20T01:18:25.666 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:18:25.666 INFO:teuthology.orchestra.run.smithi016.stdout: osd scrub load threshold = 5.0 2023-05-20T01:18:25.667 INFO:teuthology.orchestra.run.smithi016.stdout: osd scrub max interval = 600 2023-05-20T01:18:25.667 INFO:teuthology.orchestra.run.smithi016.stdout: osd mclock profile = high_recovery_ops 2023-05-20T01:18:25.667 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:18:25.667 INFO:teuthology.orchestra.run.smithi016.stdout: osd recover clone overlap = true 2023-05-20T01:18:25.667 INFO:teuthology.orchestra.run.smithi016.stdout: osd recovery max chunk = 1048576 2023-05-20T01:18:25.667 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:18:25.667 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug shutdown = true 2023-05-20T01:18:25.667 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug op order = true 2023-05-20T01:18:25.668 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug verify stray on activate = true 2023-05-20T01:18:25.668 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:18:25.668 INFO:teuthology.orchestra.run.smithi016.stdout: osd open classes on start = true 2023-05-20T01:18:25.668 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug pg log writeout = true 2023-05-20T01:18:25.668 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:18:25.668 INFO:teuthology.orchestra.run.smithi016.stdout: osd deep scrub update digest min age = 30 2023-05-20T01:18:25.668 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:18:25.668 INFO:teuthology.orchestra.run.smithi016.stdout: osd map max advance = 10 2023-05-20T01:18:25.669 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:18:25.669 INFO:teuthology.orchestra.run.smithi016.stdout: journal zero on create = true 2023-05-20T01:18:25.669 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:18:25.669 INFO:teuthology.orchestra.run.smithi016.stdout: filestore ondisk finisher threads = 3 2023-05-20T01:18:25.669 INFO:teuthology.orchestra.run.smithi016.stdout: filestore apply finisher threads = 3 2023-05-20T01:18:25.669 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:18:25.669 INFO:teuthology.orchestra.run.smithi016.stdout: bdev debug aio = true 2023-05-20T01:18:25.669 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug misdirected ops = true 2023-05-20T01:18:25.670 INFO:teuthology.orchestra.run.smithi016.stdout: bluestore block size = 96636764160 2023-05-20T01:18:25.670 INFO:teuthology.orchestra.run.smithi016.stdout: bluestore fsck on mount = True 2023-05-20T01:18:25.670 INFO:teuthology.orchestra.run.smithi016.stdout: debug bluefs = 20 2023-05-20T01:18:25.670 INFO:teuthology.orchestra.run.smithi016.stdout: debug bluestore = 20 2023-05-20T01:18:25.670 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2023-05-20T01:18:25.670 INFO:teuthology.orchestra.run.smithi016.stdout: debug osd = 20 2023-05-20T01:18:25.670 INFO:teuthology.orchestra.run.smithi016.stdout: debug rocksdb = 10 2023-05-20T01:18:25.670 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd backfillfull_ratio = 0.85 2023-05-20T01:18:25.671 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd full ratio = 0.9 2023-05-20T01:18:25.671 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd nearfull ratio = 0.8 2023-05-20T01:18:25.671 INFO:teuthology.orchestra.run.smithi016.stdout: osd failsafe full ratio = 0.95 2023-05-20T01:18:25.671 INFO:teuthology.orchestra.run.smithi016.stdout: osd objectstore = bluestore 2023-05-20T01:18:25.671 INFO:teuthology.orchestra.run.smithi016.stdout: osd op complaint time = 180 2023-05-20T01:18:25.672 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:18:25.672 INFO:teuthology.orchestra.run.smithi016.stdout:[mgr] 2023-05-20T01:18:25.672 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2023-05-20T01:18:25.672 INFO:teuthology.orchestra.run.smithi016.stdout: debug mgr = 20 2023-05-20T01:18:25.672 INFO:teuthology.orchestra.run.smithi016.stdout: debug mon = 20 2023-05-20T01:18:25.672 INFO:teuthology.orchestra.run.smithi016.stdout: debug auth = 20 2023-05-20T01:18:25.672 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min pgs per osd = 4 2023-05-20T01:18:25.673 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min bytes per osd = 10 2023-05-20T01:18:25.673 INFO:teuthology.orchestra.run.smithi016.stdout: mgr/telemetry/nag = false 2023-05-20T01:18:25.673 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:18:25.673 INFO:teuthology.orchestra.run.smithi016.stdout:[mon] 2023-05-20T01:18:25.673 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2023-05-20T01:18:25.673 INFO:teuthology.orchestra.run.smithi016.stdout: debug mon = 20 2023-05-20T01:18:25.673 INFO:teuthology.orchestra.run.smithi016.stdout: debug paxos = 20 2023-05-20T01:18:25.673 INFO:teuthology.orchestra.run.smithi016.stdout: debug auth = 20 2023-05-20T01:18:25.674 INFO:teuthology.orchestra.run.smithi016.stdout: mon data avail warn = 5 2023-05-20T01:18:25.674 INFO:teuthology.orchestra.run.smithi016.stdout: mon mgr mkfs grace = 240 2023-05-20T01:18:25.674 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min pgs per osd = 4 2023-05-20T01:18:25.674 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd reporter subtree level = osd 2023-05-20T01:18:25.674 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd prime pg temp = true 2023-05-20T01:18:25.674 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min bytes per osd = 10 2023-05-20T01:18:25.674 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:18:25.674 INFO:teuthology.orchestra.run.smithi016.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-05-20T01:18:25.675 INFO:teuthology.orchestra.run.smithi016.stdout: auth mon ticket ttl = 660 # 11m 2023-05-20T01:18:25.675 INFO:teuthology.orchestra.run.smithi016.stdout: auth service ticket ttl = 240 # 4m 2023-05-20T01:18:25.675 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:18:25.675 INFO:teuthology.orchestra.run.smithi016.stdout: # don't complain about insecure global_id in the test suite 2023-05-20T01:18:25.675 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-05-20T01:18:25.675 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-05-20T01:18:25.675 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:18:25.675 INFO:teuthology.orchestra.run.smithi016.stdout: # 1m isn't quite enough 2023-05-20T01:18:25.676 INFO:teuthology.orchestra.run.smithi016.stdout: mon_down_mkfs_grace = 2m 2023-05-20T01:18:25.676 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:18:25.676 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_filestore_osds = false 2023-05-20T01:18:25.676 INFO:teuthology.orchestra.run.smithi016.stdout: mon op complaint time = 120 2023-05-20T01:18:25.676 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:18:25.676 INFO:teuthology.orchestra.run.smithi016.stdout:[client] 2023-05-20T01:18:25.676 INFO:teuthology.orchestra.run.smithi016.stdout: rgw cache enabled = true 2023-05-20T01:18:25.677 INFO:teuthology.orchestra.run.smithi016.stdout: rgw enable ops log = true 2023-05-20T01:18:25.677 INFO:teuthology.orchestra.run.smithi016.stdout: rgw enable usage log = true 2023-05-20T01:18:25.677 INFO:teuthology.orchestra.run.smithi016.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-05-20T01:18:25.677 INFO:teuthology.orchestra.run.smithi016.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-05-20T01:18:25.677 INFO:teuthology.orchestra.run.smithi016.stdout: client mount timeout = 600 2023-05-20T01:18:25.677 INFO:teuthology.orchestra.run.smithi016.stdout: debug client = 20 2023-05-20T01:18:25.677 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2023-05-20T01:18:25.677 INFO:teuthology.orchestra.run.smithi016.stdout: pid file = "" 2023-05-20T01:18:25.678 INFO:teuthology.orchestra.run.smithi016.stdout: rados mon op timeout = 900 2023-05-20T01:18:25.678 INFO:teuthology.orchestra.run.smithi016.stdout: rados osd op timeout = 900 2023-05-20T01:18:25.678 INFO:teuthology.orchestra.run.smithi016.stdout:[mon.a] 2023-05-20T01:18:25.678 INFO:teuthology.orchestra.run.smithi016.stdout:[mon.b] 2023-05-20T01:18:25.678 INFO:teuthology.orchestra.run.smithi016.stdout:[mon.c] 2023-05-20T01:18:25.678 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.a] 2023-05-20T01:18:25.678 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.c] 2023-05-20T01:18:25.678 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.b] 2023-05-20T01:18:25.679 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.d] 2023-05-20T01:18:25.679 INFO:teuthology.orchestra.run.smithi016.stdout:[mds] 2023-05-20T01:18:25.679 INFO:teuthology.orchestra.run.smithi016.stdout: debug mds = 20 2023-05-20T01:18:25.679 INFO:teuthology.orchestra.run.smithi016.stdout: debug mds balancer = 20 2023-05-20T01:18:25.679 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2023-05-20T01:18:25.679 INFO:teuthology.orchestra.run.smithi016.stdout: mds debug frag = True 2023-05-20T01:18:25.679 INFO:teuthology.orchestra.run.smithi016.stdout: mds debug scatterstat = False 2023-05-20T01:18:25.679 INFO:teuthology.orchestra.run.smithi016.stdout: mds op complaint time = 180 2023-05-20T01:18:25.680 INFO:teuthology.orchestra.run.smithi016.stdout: mds verify scatter = False 2023-05-20T01:18:25.680 INFO:teuthology.orchestra.run.smithi016.stdout: osd op complaint time = 180 2023-05-20T01:18:25.680 INFO:teuthology.orchestra.run.smithi016.stdout: rados mon op timeout = 900 2023-05-20T01:18:25.680 INFO:teuthology.orchestra.run.smithi016.stdout: rados osd op timeout = 900 2023-05-20T01:18:25.681 INFO:teuthology.orchestra.run.smithi132.stdout:[global] 2023-05-20T01:18:25.682 INFO:teuthology.orchestra.run.smithi132.stdout: chdir = "" 2023-05-20T01:18:25.682 INFO:teuthology.orchestra.run.smithi132.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-05-20T01:18:25.682 INFO:teuthology.orchestra.run.smithi132.stdout: auth supported = cephx 2023-05-20T01:18:25.682 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:18:25.682 INFO:teuthology.orchestra.run.smithi132.stdout: filestore xattr use omap = true 2023-05-20T01:18:25.682 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:18:25.683 INFO:teuthology.orchestra.run.smithi132.stdout: mon clock drift allowed = 1.000 2023-05-20T01:18:25.683 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:18:25.683 INFO:teuthology.orchestra.run.smithi132.stdout: osd crush chooseleaf type = 0 2023-05-20T01:18:25.683 INFO:teuthology.orchestra.run.smithi132.stdout: auth debug = true 2023-05-20T01:18:25.683 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:18:25.683 INFO:teuthology.orchestra.run.smithi132.stdout: ms die on old message = true 2023-05-20T01:18:25.683 INFO:teuthology.orchestra.run.smithi132.stdout: ms die on bug = true 2023-05-20T01:18:25.683 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:18:25.684 INFO:teuthology.orchestra.run.smithi132.stdout: mon max pg per osd = 10000 # >= luminous 2023-05-20T01:18:25.684 INFO:teuthology.orchestra.run.smithi132.stdout: mon pg warn max object skew = 0 2023-05-20T01:18:25.684 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:18:25.684 INFO:teuthology.orchestra.run.smithi132.stdout: # disable pg_autoscaler by default for new pools 2023-05-20T01:18:25.684 INFO:teuthology.orchestra.run.smithi132.stdout: osd_pool_default_pg_autoscale_mode = off 2023-05-20T01:18:25.684 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:18:25.684 INFO:teuthology.orchestra.run.smithi132.stdout: osd pool default size = 2 2023-05-20T01:18:25.685 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:18:25.685 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd allow primary affinity = true 2023-05-20T01:18:25.685 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd allow pg remap = true 2023-05-20T01:18:25.685 INFO:teuthology.orchestra.run.smithi132.stdout: mon warn on legacy crush tunables = false 2023-05-20T01:18:25.685 INFO:teuthology.orchestra.run.smithi132.stdout: mon warn on crush straw calc version zero = false 2023-05-20T01:18:25.685 INFO:teuthology.orchestra.run.smithi132.stdout: mon warn on no sortbitwise = false 2023-05-20T01:18:25.685 INFO:teuthology.orchestra.run.smithi132.stdout: mon warn on osd down out interval zero = false 2023-05-20T01:18:25.686 INFO:teuthology.orchestra.run.smithi132.stdout: mon warn on too few osds = false 2023-05-20T01:18:25.686 INFO:teuthology.orchestra.run.smithi132.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-05-20T01:18:25.686 INFO:teuthology.orchestra.run.smithi132.stdout: mon_warn_on_pool_no_redundancy = false 2023-05-20T01:18:25.686 INFO:teuthology.orchestra.run.smithi132.stdout: mon_allow_pool_size_one = true 2023-05-20T01:18:25.686 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:18:25.686 INFO:teuthology.orchestra.run.smithi132.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-05-20T01:18:25.686 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:18:25.686 INFO:teuthology.orchestra.run.smithi132.stdout: osd default data pool replay window = 5 2023-05-20T01:18:25.687 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:18:25.687 INFO:teuthology.orchestra.run.smithi132.stdout: mon allow pool delete = true 2023-05-20T01:18:25.687 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:18:25.687 INFO:teuthology.orchestra.run.smithi132.stdout: mon cluster log file level = debug 2023-05-20T01:18:25.687 INFO:teuthology.orchestra.run.smithi132.stdout: debug asserts on shutdown = true 2023-05-20T01:18:25.687 INFO:teuthology.orchestra.run.smithi132.stdout: mon health detail to clog = false 2023-05-20T01:18:25.687 INFO:teuthology.orchestra.run.smithi132.stdout: mon host = "172.21.15.16,172.21.15.132,[v2:172.21.15.132:3301,v1:172.21.15.132:6790]" 2023-05-20T01:18:25.687 INFO:teuthology.orchestra.run.smithi132.stdout: ms die on skipped message = False 2023-05-20T01:18:25.688 INFO:teuthology.orchestra.run.smithi132.stdout: fsid = c2245cc5-ecab-4087-a59b-bc81db901676 2023-05-20T01:18:25.688 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:18:25.688 INFO:teuthology.orchestra.run.smithi132.stdout:[osd] 2023-05-20T01:18:25.688 INFO:teuthology.orchestra.run.smithi132.stdout: osd journal size = 100 2023-05-20T01:18:25.688 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:18:25.688 INFO:teuthology.orchestra.run.smithi132.stdout: osd scrub load threshold = 5.0 2023-05-20T01:18:25.688 INFO:teuthology.orchestra.run.smithi132.stdout: osd scrub max interval = 600 2023-05-20T01:18:25.688 INFO:teuthology.orchestra.run.smithi132.stdout: osd mclock profile = high_recovery_ops 2023-05-20T01:18:25.689 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:18:25.689 INFO:teuthology.orchestra.run.smithi132.stdout: osd recover clone overlap = true 2023-05-20T01:18:25.689 INFO:teuthology.orchestra.run.smithi132.stdout: osd recovery max chunk = 1048576 2023-05-20T01:18:25.689 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:18:25.689 INFO:teuthology.orchestra.run.smithi132.stdout: osd debug shutdown = true 2023-05-20T01:18:25.689 INFO:teuthology.orchestra.run.smithi132.stdout: osd debug op order = true 2023-05-20T01:18:25.689 INFO:teuthology.orchestra.run.smithi132.stdout: osd debug verify stray on activate = true 2023-05-20T01:18:25.690 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:18:25.690 INFO:teuthology.orchestra.run.smithi132.stdout: osd open classes on start = true 2023-05-20T01:18:25.690 INFO:teuthology.orchestra.run.smithi132.stdout: osd debug pg log writeout = true 2023-05-20T01:18:25.690 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:18:25.691 INFO:teuthology.orchestra.run.smithi132.stdout: osd deep scrub update digest min age = 30 2023-05-20T01:18:25.691 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:18:25.691 INFO:teuthology.orchestra.run.smithi132.stdout: osd map max advance = 10 2023-05-20T01:18:25.692 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:18:25.692 INFO:teuthology.orchestra.run.smithi132.stdout: journal zero on create = true 2023-05-20T01:18:25.692 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:18:25.692 INFO:teuthology.orchestra.run.smithi132.stdout: filestore ondisk finisher threads = 3 2023-05-20T01:18:25.693 INFO:teuthology.orchestra.run.smithi132.stdout: filestore apply finisher threads = 3 2023-05-20T01:18:25.693 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:18:25.693 INFO:teuthology.orchestra.run.smithi132.stdout: bdev debug aio = true 2023-05-20T01:18:25.693 INFO:teuthology.orchestra.run.smithi132.stdout: osd debug misdirected ops = true 2023-05-20T01:18:25.694 INFO:teuthology.orchestra.run.smithi132.stdout: bluestore block size = 96636764160 2023-05-20T01:18:25.694 INFO:teuthology.orchestra.run.smithi132.stdout: bluestore fsck on mount = True 2023-05-20T01:18:25.694 INFO:teuthology.orchestra.run.smithi132.stdout: debug bluefs = 20 2023-05-20T01:18:25.694 INFO:teuthology.orchestra.run.smithi132.stdout: debug bluestore = 20 2023-05-20T01:18:25.695 INFO:teuthology.orchestra.run.smithi132.stdout: debug ms = 1 2023-05-20T01:18:25.695 INFO:teuthology.orchestra.run.smithi132.stdout: debug osd = 20 2023-05-20T01:18:25.695 INFO:teuthology.orchestra.run.smithi132.stdout: debug rocksdb = 10 2023-05-20T01:18:25.696 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd backfillfull_ratio = 0.85 2023-05-20T01:18:25.696 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd full ratio = 0.9 2023-05-20T01:18:25.696 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd nearfull ratio = 0.8 2023-05-20T01:18:25.696 INFO:teuthology.orchestra.run.smithi132.stdout: osd failsafe full ratio = 0.95 2023-05-20T01:18:25.697 INFO:teuthology.orchestra.run.smithi132.stdout: osd objectstore = bluestore 2023-05-20T01:18:25.697 INFO:teuthology.orchestra.run.smithi132.stdout: osd op complaint time = 180 2023-05-20T01:18:25.697 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:18:25.697 INFO:teuthology.orchestra.run.smithi132.stdout:[mgr] 2023-05-20T01:18:25.698 INFO:teuthology.orchestra.run.smithi132.stdout: debug ms = 1 2023-05-20T01:18:25.698 INFO:teuthology.orchestra.run.smithi132.stdout: debug mgr = 20 2023-05-20T01:18:25.698 INFO:teuthology.orchestra.run.smithi132.stdout: debug mon = 20 2023-05-20T01:18:25.699 INFO:teuthology.orchestra.run.smithi132.stdout: debug auth = 20 2023-05-20T01:18:25.699 INFO:teuthology.orchestra.run.smithi132.stdout: mon reweight min pgs per osd = 4 2023-05-20T01:18:25.699 INFO:teuthology.orchestra.run.smithi132.stdout: mon reweight min bytes per osd = 10 2023-05-20T01:18:25.700 INFO:teuthology.orchestra.run.smithi132.stdout: mgr/telemetry/nag = false 2023-05-20T01:18:25.700 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:18:25.700 INFO:teuthology.orchestra.run.smithi132.stdout:[mon] 2023-05-20T01:18:25.700 INFO:teuthology.orchestra.run.smithi132.stdout: debug ms = 1 2023-05-20T01:18:25.701 INFO:teuthology.orchestra.run.smithi132.stdout: debug mon = 20 2023-05-20T01:18:25.701 INFO:teuthology.orchestra.run.smithi132.stdout: debug paxos = 20 2023-05-20T01:18:25.701 INFO:teuthology.orchestra.run.smithi132.stdout: debug auth = 20 2023-05-20T01:18:25.702 INFO:teuthology.orchestra.run.smithi132.stdout: mon data avail warn = 5 2023-05-20T01:18:25.702 INFO:teuthology.orchestra.run.smithi132.stdout: mon mgr mkfs grace = 240 2023-05-20T01:18:25.702 INFO:teuthology.orchestra.run.smithi132.stdout: mon reweight min pgs per osd = 4 2023-05-20T01:18:25.702 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd reporter subtree level = osd 2023-05-20T01:18:25.703 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd prime pg temp = true 2023-05-20T01:18:25.703 INFO:teuthology.orchestra.run.smithi132.stdout: mon reweight min bytes per osd = 10 2023-05-20T01:18:25.703 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:18:25.703 INFO:teuthology.orchestra.run.smithi132.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-05-20T01:18:25.703 INFO:teuthology.orchestra.run.smithi132.stdout: auth mon ticket ttl = 660 # 11m 2023-05-20T01:18:25.704 INFO:teuthology.orchestra.run.smithi132.stdout: auth service ticket ttl = 240 # 4m 2023-05-20T01:18:25.704 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:18:25.704 INFO:teuthology.orchestra.run.smithi132.stdout: # don't complain about insecure global_id in the test suite 2023-05-20T01:18:25.704 INFO:teuthology.orchestra.run.smithi132.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-05-20T01:18:25.705 INFO:teuthology.orchestra.run.smithi132.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-05-20T01:18:25.705 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:18:25.705 INFO:teuthology.orchestra.run.smithi132.stdout: # 1m isn't quite enough 2023-05-20T01:18:25.705 INFO:teuthology.orchestra.run.smithi132.stdout: mon_down_mkfs_grace = 2m 2023-05-20T01:18:25.706 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:18:25.706 INFO:teuthology.orchestra.run.smithi132.stdout: mon_warn_on_filestore_osds = false 2023-05-20T01:18:25.706 INFO:teuthology.orchestra.run.smithi132.stdout: mon op complaint time = 120 2023-05-20T01:18:25.706 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:18:25.706 INFO:teuthology.orchestra.run.smithi132.stdout:[client] 2023-05-20T01:18:25.706 INFO:teuthology.orchestra.run.smithi132.stdout: rgw cache enabled = true 2023-05-20T01:18:25.706 INFO:teuthology.orchestra.run.smithi132.stdout: rgw enable ops log = true 2023-05-20T01:18:25.707 INFO:teuthology.orchestra.run.smithi132.stdout: rgw enable usage log = true 2023-05-20T01:18:25.707 INFO:teuthology.orchestra.run.smithi132.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-05-20T01:18:25.707 INFO:teuthology.orchestra.run.smithi132.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-05-20T01:18:25.707 INFO:teuthology.orchestra.run.smithi132.stdout: client mount timeout = 600 2023-05-20T01:18:25.707 INFO:teuthology.orchestra.run.smithi132.stdout: debug client = 20 2023-05-20T01:18:25.707 INFO:teuthology.orchestra.run.smithi132.stdout: debug ms = 1 2023-05-20T01:18:25.707 INFO:teuthology.orchestra.run.smithi132.stdout: pid file = "" 2023-05-20T01:18:25.708 INFO:teuthology.orchestra.run.smithi132.stdout: rados mon op timeout = 900 2023-05-20T01:18:25.708 INFO:teuthology.orchestra.run.smithi132.stdout: rados osd op timeout = 900 2023-05-20T01:18:25.708 INFO:teuthology.orchestra.run.smithi132.stdout:[mon.a] 2023-05-20T01:18:25.708 INFO:teuthology.orchestra.run.smithi132.stdout:[mon.b] 2023-05-20T01:18:25.708 INFO:teuthology.orchestra.run.smithi132.stdout:[mon.c] 2023-05-20T01:18:25.708 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.a] 2023-05-20T01:18:25.708 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.c] 2023-05-20T01:18:25.708 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.b] 2023-05-20T01:18:25.709 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.d] 2023-05-20T01:18:25.709 INFO:teuthology.orchestra.run.smithi132.stdout:[mds] 2023-05-20T01:18:25.709 INFO:teuthology.orchestra.run.smithi132.stdout: debug mds = 20 2023-05-20T01:18:25.709 INFO:teuthology.orchestra.run.smithi132.stdout: debug mds balancer = 20 2023-05-20T01:18:25.709 INFO:teuthology.orchestra.run.smithi132.stdout: debug ms = 1 2023-05-20T01:18:25.709 INFO:teuthology.orchestra.run.smithi132.stdout: mds debug frag = True 2023-05-20T01:18:25.709 INFO:teuthology.orchestra.run.smithi132.stdout: mds debug scatterstat = False 2023-05-20T01:18:25.710 INFO:teuthology.orchestra.run.smithi132.stdout: mds op complaint time = 180 2023-05-20T01:18:25.710 INFO:teuthology.orchestra.run.smithi132.stdout: mds verify scatter = False 2023-05-20T01:18:25.710 INFO:teuthology.orchestra.run.smithi132.stdout: osd op complaint time = 180 2023-05-20T01:18:25.710 INFO:teuthology.orchestra.run.smithi132.stdout: rados mon op timeout = 900 2023-05-20T01:18:25.710 INFO:teuthology.orchestra.run.smithi132.stdout: rados osd op timeout = 900 2023-05-20T01:18:25.710 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph purge cephfs_metadata --yes-i-really-really-mean-it 2023-05-20T01:18:25.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.750+0000 7fe93da3f080 1 Processor -- start 2023-05-20T01:18:25.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.750+0000 7fe93da3f080 1 -- start start 2023-05-20T01:18:25.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.751+0000 7fe93da3f080 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56543194e880 0x565431950750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:25.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.751+0000 7fe93da3f080 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56543194deb0 0x56543191f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:25.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.751+0000 7fe93da3f080 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x565431950c90 0x56543191fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:25.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.751+0000 7fe93da3f080 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x565431883e80 con 0x56543194deb0 2023-05-20T01:18:25.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.751+0000 7fe93da3f080 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x565431883b20 con 0x565431950c90 2023-05-20T01:18:25.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.751+0000 7fe93da3f080 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x565431859fe0 con 0x56543194e880 2023-05-20T01:18:25.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.751+0000 7fe93333e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56543194deb0 0x56543191f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:25.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.751+0000 7fe93333e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56543194deb0 0x56543191f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60620/0 (socket says 172.21.15.16:60620) 2023-05-20T01:18:25.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.751+0000 7fe93333e700 1 -- 172.21.15.16:0/3587751535 learned_addr learned my addr 172.21.15.16:0/3587751535 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:18:25.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.751+0000 7fe933b3f700 1 --2- 172.21.15.16:0/3587751535 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56543194e880 0x565431950750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:25.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.751+0000 7fe934340700 1 --2- 172.21.15.16:0/3587751535 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x565431950c90 0x56543191fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:25.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.751+0000 7fe93333e700 1 -- 172.21.15.16:0/3587751535 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56543194e880 msgr2=0x565431950750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:18:25.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.751+0000 7fe93333e700 1 --2- 172.21.15.16:0/3587751535 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56543194e880 0x565431950750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:25.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.751+0000 7fe93333e700 1 -- 172.21.15.16:0/3587751535 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x565431950c90 msgr2=0x56543191fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:18:25.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.751+0000 7fe93333e700 1 --2- 172.21.15.16:0/3587751535 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x565431950c90 0x56543191fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:25.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.751+0000 7fe93333e700 1 -- 172.21.15.16:0/3587751535 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565431871a80 con 0x56543194deb0 2023-05-20T01:18:25.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.751+0000 7fe93333e700 1 --2- 172.21.15.16:0/3587751535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56543194deb0 0x56543191f940 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fe9280098e0 tx=0x7fe928005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1155fbb182a6e93e server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:25.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.751+0000 7fe93233c700 1 -- 172.21.15.16:0/3587751535 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe92800f040 con 0x56543194deb0 2023-05-20T01:18:25.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.751+0000 7fe93233c700 1 -- 172.21.15.16:0/3587751535 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9280089b0 con 0x56543194deb0 2023-05-20T01:18:25.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.751+0000 7fe93233c700 1 -- 172.21.15.16:0/3587751535 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe928008d40 con 0x56543194deb0 2023-05-20T01:18:25.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.752+0000 7fe93da3f080 1 -- 172.21.15.16:0/3587751535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56543194deb0 msgr2=0x56543191f940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:25.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.752+0000 7fe93da3f080 1 --2- 172.21.15.16:0/3587751535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56543194deb0 0x56543191f940 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fe9280098e0 tx=0x7fe928005a00 comp rx=0 tx=0).stop 2023-05-20T01:18:25.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.752+0000 7fe93da3f080 1 -- 172.21.15.16:0/3587751535 shutdown_connections 2023-05-20T01:18:25.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.752+0000 7fe93da3f080 1 --2- 172.21.15.16:0/3587751535 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x565431950c90 0x56543191fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:25.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.752+0000 7fe93da3f080 1 --2- 172.21.15.16:0/3587751535 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56543194e880 0x565431950750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:25.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.752+0000 7fe93da3f080 1 --2- 172.21.15.16:0/3587751535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56543194deb0 0x56543191f940 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:25.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.752+0000 7fe93da3f080 1 -- 172.21.15.16:0/3587751535 >> 172.21.15.16:0/3587751535 conn(0x5654319680a0 msgr2=0x5654319f84d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:18:25.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.752+0000 7fe93da3f080 1 -- 172.21.15.16:0/3587751535 shutdown_connections 2023-05-20T01:18:25.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.752+0000 7fe93da3f080 1 -- 172.21.15.16:0/3587751535 wait complete. 2023-05-20T01:18:25.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.752+0000 7fe93da3f080 1 Processor -- start 2023-05-20T01:18:25.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.753+0000 7fe93da3f080 1 -- start start 2023-05-20T01:18:25.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.753+0000 7fe93da3f080 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56543194deb0 0x5654319e5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:25.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.753+0000 7fe93da3f080 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56543194e880 0x56543191f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:25.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.753+0000 7fe93da3f080 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x565431950c90 0x56543191fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:25.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.753+0000 7fe93da3f080 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654319ee710 con 0x56543194e880 2023-05-20T01:18:25.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.753+0000 7fe93da3f080 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654319ee9f0 con 0x56543194deb0 2023-05-20T01:18:25.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.753+0000 7fe93da3f080 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x5654319eecd0 con 0x565431950c90 2023-05-20T01:18:25.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.753+0000 7fe93333e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56543194e880 0x56543191f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:25.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.753+0000 7fe93333e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56543194e880 0x56543191f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60626/0 (socket says 172.21.15.16:60626) 2023-05-20T01:18:25.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.753+0000 7fe93333e700 1 -- 172.21.15.16:0/3156188675 learned_addr learned my addr 172.21.15.16:0/3156188675 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:18:25.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.753+0000 7fe933b3f700 1 --2- 172.21.15.16:0/3156188675 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56543194deb0 0x5654319e5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:25.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.753+0000 7fe934340700 1 --2- 172.21.15.16:0/3156188675 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x565431950c90 0x56543191fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:25.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.754+0000 7fe93333e700 1 -- 172.21.15.16:0/3156188675 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x565431950c90 msgr2=0x56543191fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:25.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.754+0000 7fe93333e700 1 --2- 172.21.15.16:0/3156188675 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x565431950c90 0x56543191fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:25.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.754+0000 7fe93333e700 1 -- 172.21.15.16:0/3156188675 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56543194deb0 msgr2=0x5654319e5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:25.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.754+0000 7fe93333e700 1 --2- 172.21.15.16:0/3156188675 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56543194deb0 0x5654319e5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:25.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.754+0000 7fe93333e700 1 -- 172.21.15.16:0/3156188675 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe928009540 con 0x56543194e880 2023-05-20T01:18:25.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.754+0000 7fe93333e700 1 --2- 172.21.15.16:0/3156188675 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56543194e880 0x56543191f8b0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fe928017000 tx=0x7fe928002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:25.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.754+0000 7fe93133a700 1 -- 172.21.15.16:0/3156188675 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe92800f040 con 0x56543194e880 2023-05-20T01:18:25.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.754+0000 7fe93133a700 1 -- 172.21.15.16:0/3156188675 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9280158a0 con 0x56543194e880 2023-05-20T01:18:25.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.754+0000 7fe93da3f080 1 -- 172.21.15.16:0/3156188675 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x565431865830 con 0x56543194e880 2023-05-20T01:18:25.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.754+0000 7fe93133a700 1 -- 172.21.15.16:0/3156188675 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe928015c50 con 0x56543194e880 2023-05-20T01:18:25.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.754+0000 7fe93da3f080 1 -- 172.21.15.16:0/3156188675 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5654318654a0 con 0x56543194e880 2023-05-20T01:18:25.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.755+0000 7fe93133a700 1 -- 172.21.15.16:0/3156188675 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe92801e420 con 0x56543194e880 2023-05-20T01:18:25.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.756+0000 7fe93133a700 1 --2- 172.21.15.16:0/3156188675 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe920069600 0x7fe92006bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:25.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.756+0000 7fe93133a700 1 -- 172.21.15.16:0/3156188675 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6930+0+0 (secure 0 0 0) 0x7fe92808a450 con 0x56543194e880 2023-05-20T01:18:25.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.756+0000 7fe933b3f700 1 --2- 172.21.15.16:0/3156188675 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe920069600 0x7fe92006bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:25.768 INFO:teuthology.orchestra.run.smithi016.stdout:Warning: using slow linear search 2023-05-20T01:18:25.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.756+0000 7fe93da3f080 1 --2- 172.21.15.16:0/3156188675 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x5654319e9120 0x565431a9b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:25.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.756+0000 7fe93da3f080 1 -- 172.21.15.16:0/3156188675 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:1 6.0 6:00000000:%01:::head [pgnls start_epoch 46 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e46) v8 -- 0x5654319ef7b0 con 0x5654319e9120 2023-05-20T01:18:25.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.756+0000 7fe934340700 1 --2- 172.21.15.16:0/3156188675 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x5654319e9120 0x565431a9b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:25.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.756+0000 7fe933b3f700 1 --2- 172.21.15.16:0/3156188675 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe920069600 0x7fe92006bab0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x5654319f16e0 tx=0x7fe924011040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:25.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.757+0000 7fe934340700 1 --2- 172.21.15.16:0/3156188675 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x5654319e9120 0x565431a9b660 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:25.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.757+0000 7fe934340700 1 -- 172.21.15.16:0/3156188675 <== osd.0 v2:172.21.15.16:6824/3923634179 1 ==== osd_op_reply(1 [pgnls start_epoch 46 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe92c024040 con 0x5654319e9120 2023-05-20T01:18:25.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.757+0000 7fe934340700 1 -- 172.21.15.16:0/3156188675 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:2 6.10 6:08000000:%01:::head [pgnls start_epoch 46 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e46) v8 -- 0x5654319ef7b0 con 0x5654319e9120 2023-05-20T01:18:25.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.757+0000 7fe934340700 1 -- 172.21.15.16:0/3156188675 <== osd.0 v2:172.21.15.16:6824/3923634179 2 ==== osd_op_reply(2 [pgnls start_epoch 46 out=177b] v45'10 uv10 ondisk = 1) v8 ==== 144+0+177 (crc 0 0 0) 0x7fe92c024040 con 0x5654319e9120 2023-05-20T01:18:25.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.757+0000 7fe934340700 1 --2- 172.21.15.16:0/3156188675 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x7fe91c0035f0 0x7fe92c01c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:25.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.757+0000 7fe934340700 1 -- 172.21.15.16:0/3156188675 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:3 6.8 6:10000000:%01:::head [pgnls start_epoch 46 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e46) v8 -- 0x5654319ef7b0 con 0x7fe91c0035f0 2023-05-20T01:18:25.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.758+0000 7fe933b3f700 1 --2- 172.21.15.16:0/3156188675 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x7fe91c0035f0 0x7fe92c01c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:25.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.758+0000 7fe933b3f700 1 --2- 172.21.15.16:0/3156188675 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x7fe91c0035f0 0x7fe92c01c940 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:25.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.758+0000 7fe933b3f700 1 -- 172.21.15.16:0/3156188675 <== osd.7 v2:172.21.15.132:6800/466001149 1 ==== osd_op_reply(3 [pgnls start_epoch 46 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe924017070 con 0x7fe91c0035f0 2023-05-20T01:18:25.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.758+0000 7fe933b3f700 1 -- 172.21.15.16:0/3156188675 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:4 6.18 6:18000000:%01:::head [pgnls start_epoch 46 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e46) v8 -- 0x5654319ef7b0 con 0x5654319e9120 2023-05-20T01:18:25.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.759+0000 7fe934340700 1 -- 172.21.15.16:0/3156188675 <== osd.0 v2:172.21.15.16:6824/3923634179 3 ==== osd_op_reply(4 [pgnls start_epoch 46 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe92c024040 con 0x5654319e9120 2023-05-20T01:18:25.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.759+0000 7fe934340700 1 --2- 172.21.15.16:0/3156188675 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x565431964f30 0x7fe92c029980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:25.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.759+0000 7fe934340700 1 -- 172.21.15.16:0/3156188675 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:5 6.4 6:20000000:%01:::head [pgnls start_epoch 46 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e46) v8 -- 0x5654319ef7b0 con 0x565431964f30 2023-05-20T01:18:25.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.759+0000 7fe93333e700 1 --2- 172.21.15.16:0/3156188675 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x565431964f30 0x7fe92c029980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:25.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.759+0000 7fe93333e700 1 --2- 172.21.15.16:0/3156188675 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x565431964f30 0x7fe92c029980 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:25.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.759+0000 7fe93333e700 1 -- 172.21.15.16:0/3156188675 <== osd.1 v2:172.21.15.16:6808/3860264220 1 ==== osd_op_reply(5 [pgnls start_epoch 46 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe92805e040 con 0x565431964f30 2023-05-20T01:18:25.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.759+0000 7fe93333e700 1 --2- 172.21.15.16:0/3156188675 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x7fe928009040 0x7fe9280590b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:25.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.759+0000 7fe93333e700 1 -- 172.21.15.16:0/3156188675 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:6 6.14 6:28000000:%01:::head [pgnls start_epoch 46 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e46) v8 -- 0x5654319ef7b0 con 0x7fe928009040 2023-05-20T01:18:25.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.759+0000 7fe934340700 1 --2- 172.21.15.16:0/3156188675 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x7fe928009040 0x7fe9280590b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:25.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.760+0000 7fe934340700 1 --2- 172.21.15.16:0/3156188675 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x7fe928009040 0x7fe9280590b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:25.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.760+0000 7fe934340700 1 -- 172.21.15.16:0/3156188675 <== osd.2 v2:172.21.15.16:6816/293187030 1 ==== osd_op_reply(6 [pgnls start_epoch 46 out=149b] v45'41 uv41 ondisk = 1) v8 ==== 144+0+149 (crc 0 0 0) 0x7fe92c024040 con 0x7fe928009040 2023-05-20T01:18:25.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.760+0000 7fe934340700 1 --2- 172.21.15.16:0/3156188675 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x565431965300 0x7fe92c01a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:25.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.760+0000 7fe934340700 1 -- 172.21.15.16:0/3156188675 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:7 6.c 6:30000000:%01:::head [pgnls start_epoch 46 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e46) v8 -- 0x5654319ef7b0 con 0x565431965300 2023-05-20T01:18:25.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.760+0000 7fe933b3f700 1 --2- 172.21.15.16:0/3156188675 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x565431965300 0x7fe92c01a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:25.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.761+0000 7fe933b3f700 1 --2- 172.21.15.16:0/3156188675 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x565431965300 0x7fe92c01a8c0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:25.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.761+0000 7fe933b3f700 1 -- 172.21.15.16:0/3156188675 <== osd.3 v2:172.21.15.16:6800/3669781890 1 ==== osd_op_reply(7 [pgnls start_epoch 46 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe924017070 con 0x565431965300 2023-05-20T01:18:25.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.761+0000 7fe933b3f700 1 -- 172.21.15.16:0/3156188675 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:8 6.1c 6:38000000:%01:::head [pgnls start_epoch 46 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e46) v8 -- 0x5654319ef7b0 con 0x7fe91c0035f0 2023-05-20T01:18:25.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.761+0000 7fe933b3f700 1 -- 172.21.15.16:0/3156188675 <== osd.7 v2:172.21.15.132:6800/466001149 2 ==== osd_op_reply(8 [pgnls start_epoch 46 out=73b] v39'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x7fe924017070 con 0x7fe91c0035f0 2023-05-20T01:18:25.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.761+0000 7fe933b3f700 1 --2- 172.21.15.16:0/3156188675 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x7fe924019400 0x7fe924019830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:25.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.761+0000 7fe933b3f700 1 -- 172.21.15.16:0/3156188675 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:9 6.2 6:40000000:%01:::head [pgnls start_epoch 46 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e46) v8 -- 0x5654319ef7b0 con 0x7fe924019400 2023-05-20T01:18:25.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.762+0000 7fe93333e700 1 --2- 172.21.15.16:0/3156188675 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x7fe924019400 0x7fe924019830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:25.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.762+0000 7fe93333e700 1 --2- 172.21.15.16:0/3156188675 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x7fe924019400 0x7fe924019830 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:25.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.762+0000 7fe93333e700 1 -- 172.21.15.16:0/3156188675 <== osd.4 v2:172.21.15.132:6817/506317223 1 ==== osd_op_reply(9 [pgnls start_epoch 46 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe92805e040 con 0x7fe924019400 2023-05-20T01:18:25.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.762+0000 7fe93333e700 1 -- 172.21.15.16:0/3156188675 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:10 6.12 6:48000000:%01:::head [pgnls start_epoch 46 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e46) v8 -- 0x5654319ef7b0 con 0x7fe91c0035f0 2023-05-20T01:18:25.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.762+0000 7fe933b3f700 1 -- 172.21.15.16:0/3156188675 <== osd.7 v2:172.21.15.132:6800/466001149 3 ==== osd_op_reply(10 [pgnls start_epoch 46 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe924017070 con 0x7fe91c0035f0 2023-05-20T01:18:25.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.762+0000 7fe933b3f700 1 --2- 172.21.15.16:0/3156188675 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x7fe92402e980 0x7fe92402b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:25.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.762+0000 7fe933b3f700 1 -- 172.21.15.16:0/3156188675 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:11 6.a 6:50000000:%01:::head [pgnls start_epoch 46 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e46) v8 -- 0x5654319ef7b0 con 0x7fe92402e980 2023-05-20T01:18:25.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.763+0000 7fe934340700 1 --2- 172.21.15.16:0/3156188675 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x7fe92402e980 0x7fe92402b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:25.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.763+0000 7fe934340700 1 --2- 172.21.15.16:0/3156188675 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x7fe92402e980 0x7fe92402b3d0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:25.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.763+0000 7fe934340700 1 -- 172.21.15.16:0/3156188675 <== osd.5 v2:172.21.15.132:6816/1844689995 1 ==== osd_op_reply(11 [pgnls start_epoch 46 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe92c024040 con 0x7fe92402e980 2023-05-20T01:18:25.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.763+0000 7fe934340700 1 -- 172.21.15.16:0/3156188675 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:12 6.1a 6:58000000:%01:::head [pgnls start_epoch 46 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e46) v8 -- 0x5654319ef7b0 con 0x7fe924019400 2023-05-20T01:18:25.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.764+0000 7fe93333e700 1 -- 172.21.15.16:0/3156188675 <== osd.4 v2:172.21.15.132:6817/506317223 2 ==== osd_op_reply(12 [pgnls start_epoch 46 out=49b] v39'1 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe92805e040 con 0x7fe924019400 2023-05-20T01:18:25.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.764+0000 7fe93333e700 1 -- 172.21.15.16:0/3156188675 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:13 6.6 6:60000000:%01:::head [pgnls start_epoch 46 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e46) v8 -- 0x5654319ef7b0 con 0x565431965300 2023-05-20T01:18:25.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.764+0000 7fe933b3f700 1 -- 172.21.15.16:0/3156188675 <== osd.3 v2:172.21.15.16:6800/3669781890 2 ==== osd_op_reply(13 [pgnls start_epoch 46 out=77b] v45'8 uv8 ondisk = 1) v8 ==== 144+0+77 (crc 0 0 0) 0x7fe924017070 con 0x565431965300 2023-05-20T01:18:25.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.764+0000 7fe933b3f700 1 -- 172.21.15.16:0/3156188675 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:14 6.16 6:68000000:%01:::head [pgnls start_epoch 46 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e46) v8 -- 0x5654319ef7b0 con 0x5654319e9120 2023-05-20T01:18:25.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.764+0000 7fe934340700 1 -- 172.21.15.16:0/3156188675 <== osd.0 v2:172.21.15.16:6824/3923634179 4 ==== osd_op_reply(14 [pgnls start_epoch 46 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe92c024040 con 0x5654319e9120 2023-05-20T01:18:25.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.764+0000 7fe934340700 1 -- 172.21.15.16:0/3156188675 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:15 6.e 6:70000000:%01:::head [pgnls start_epoch 46 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e46) v8 -- 0x5654319ef7b0 con 0x7fe924019400 2023-05-20T01:18:25.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.764+0000 7fe93333e700 1 -- 172.21.15.16:0/3156188675 <== osd.4 v2:172.21.15.132:6817/506317223 3 ==== osd_op_reply(15 [pgnls start_epoch 46 out=73b] v39'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x7fe92805e040 con 0x7fe924019400 2023-05-20T01:18:25.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.764+0000 7fe93333e700 1 -- 172.21.15.16:0/3156188675 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:16 6.1e 6:78000000:%01:::head [pgnls start_epoch 46 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e46) v8 -- 0x5654319ef7b0 con 0x7fe924019400 2023-05-20T01:18:25.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.765+0000 7fe93333e700 1 -- 172.21.15.16:0/3156188675 <== osd.4 v2:172.21.15.132:6817/506317223 4 ==== osd_op_reply(16 [pgnls start_epoch 46 out=49b] v39'1 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe92805e040 con 0x7fe924019400 2023-05-20T01:18:25.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.765+0000 7fe93333e700 1 -- 172.21.15.16:0/3156188675 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:17 6.1 6:80000000:%01:::head [pgnls start_epoch 46 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e46) v8 -- 0x5654319ef7b0 con 0x565431964f30 2023-05-20T01:18:25.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.765+0000 7fe93333e700 1 -- 172.21.15.16:0/3156188675 <== osd.1 v2:172.21.15.16:6808/3860264220 2 ==== osd_op_reply(17 [pgnls start_epoch 46 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe92805e040 con 0x565431964f30 2023-05-20T01:18:25.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.765+0000 7fe93333e700 1 -- 172.21.15.16:0/3156188675 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:18 6.11 6:88000000:%01:::head [pgnls start_epoch 46 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e46) v8 -- 0x5654319ef7b0 con 0x565431965300 2023-05-20T01:18:25.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.765+0000 7fe933b3f700 1 -- 172.21.15.16:0/3156188675 <== osd.3 v2:172.21.15.16:6800/3669781890 3 ==== osd_op_reply(18 [pgnls start_epoch 46 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe924017070 con 0x565431965300 2023-05-20T01:18:25.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.765+0000 7fe933b3f700 1 -- 172.21.15.16:0/3156188675 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:19 6.9 6:90000000:%01:::head [pgnls start_epoch 46 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e46) v8 -- 0x5654319ef7b0 con 0x5654319e9120 2023-05-20T01:18:25.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.766+0000 7fe934340700 1 -- 172.21.15.16:0/3156188675 <== osd.0 v2:172.21.15.16:6824/3923634179 5 ==== osd_op_reply(19 [pgnls start_epoch 46 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe92c024040 con 0x5654319e9120 2023-05-20T01:18:25.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.766+0000 7fe934340700 1 -- 172.21.15.16:0/3156188675 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:20 6.19 6:98000000:%01:::head [pgnls start_epoch 46 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e46) v8 -- 0x5654319ef7b0 con 0x7fe92402e980 2023-05-20T01:18:25.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.766+0000 7fe934340700 1 -- 172.21.15.16:0/3156188675 <== osd.5 v2:172.21.15.132:6816/1844689995 2 ==== osd_op_reply(20 [pgnls start_epoch 46 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe92c024040 con 0x7fe92402e980 2023-05-20T01:18:25.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.766+0000 7fe934340700 1 -- 172.21.15.16:0/3156188675 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:21 6.5 6:a0000000:%01:::head [pgnls start_epoch 46 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e46) v8 -- 0x5654319ef7b0 con 0x7fe91c0035f0 2023-05-20T01:18:25.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.766+0000 7fe933b3f700 1 -- 172.21.15.16:0/3156188675 <== osd.7 v2:172.21.15.132:6800/466001149 4 ==== osd_op_reply(21 [pgnls start_epoch 46 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe924017070 con 0x7fe91c0035f0 2023-05-20T01:18:25.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.766+0000 7fe933b3f700 1 -- 172.21.15.16:0/3156188675 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:22 6.15 6:a8000000:%01:::head [pgnls start_epoch 46 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e46) v8 -- 0x5654319ef7b0 con 0x7fe91c0035f0 2023-05-20T01:18:25.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.767+0000 7fe933b3f700 1 -- 172.21.15.16:0/3156188675 <== osd.7 v2:172.21.15.132:6800/466001149 5 ==== osd_op_reply(22 [pgnls start_epoch 46 out=97b] v39'3 uv3 ondisk = 1) v8 ==== 144+0+97 (crc 0 0 0) 0x7fe924017070 con 0x7fe91c0035f0 2023-05-20T01:18:25.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.767+0000 7fe933b3f700 1 -- 172.21.15.16:0/3156188675 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:23 6.d 6:b0000000:%01:::head [pgnls start_epoch 46 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e46) v8 -- 0x5654319ef7b0 con 0x7fe92402e980 2023-05-20T01:18:25.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.767+0000 7fe934340700 1 -- 172.21.15.16:0/3156188675 <== osd.5 v2:172.21.15.132:6816/1844689995 3 ==== osd_op_reply(23 [pgnls start_epoch 46 out=74b] v39'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x7fe92c024040 con 0x7fe92402e980 2023-05-20T01:18:25.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.767+0000 7fe934340700 1 -- 172.21.15.16:0/3156188675 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:24 6.1d 6:b8000000:%01:::head [pgnls start_epoch 46 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e46) v8 -- 0x5654319ef7b0 con 0x565431964f30 2023-05-20T01:18:25.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.767+0000 7fe93333e700 1 -- 172.21.15.16:0/3156188675 <== osd.1 v2:172.21.15.16:6808/3860264220 3 ==== osd_op_reply(24 [pgnls start_epoch 46 out=73b] v39'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x7fe92805e040 con 0x565431964f30 2023-05-20T01:18:25.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.767+0000 7fe93333e700 1 -- 172.21.15.16:0/3156188675 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:25 6.3 6:c0000000:%01:::head [pgnls start_epoch 46 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e46) v8 -- 0x5654319ef7b0 con 0x7fe91c0035f0 2023-05-20T01:18:25.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.767+0000 7fe933b3f700 1 -- 172.21.15.16:0/3156188675 <== osd.7 v2:172.21.15.132:6800/466001149 6 ==== osd_op_reply(25 [pgnls start_epoch 46 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe924017070 con 0x7fe91c0035f0 2023-05-20T01:18:25.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.767+0000 7fe933b3f700 1 -- 172.21.15.16:0/3156188675 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:26 6.13 6:c8000000:%01:::head [pgnls start_epoch 46 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e46) v8 -- 0x5654319ef7b0 con 0x565431965300 2023-05-20T01:18:25.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.768+0000 7fe933b3f700 1 -- 172.21.15.16:0/3156188675 <== osd.3 v2:172.21.15.16:6800/3669781890 4 ==== osd_op_reply(26 [pgnls start_epoch 46 out=98b] v45'3 uv3 ondisk = 1) v8 ==== 144+0+98 (crc 0 0 0) 0x7fe924017070 con 0x565431965300 2023-05-20T01:18:25.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.768+0000 7fe933b3f700 1 -- 172.21.15.16:0/3156188675 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:27 6.b 6:d0000000:%01:::head [pgnls start_epoch 46 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e46) v8 -- 0x5654319ef7b0 con 0x565431965300 2023-05-20T01:18:25.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.768+0000 7fe933b3f700 1 -- 172.21.15.16:0/3156188675 <== osd.3 v2:172.21.15.16:6800/3669781890 5 ==== osd_op_reply(27 [pgnls start_epoch 46 out=76b] v45'3 uv3 ondisk = 1) v8 ==== 144+0+76 (crc 0 0 0) 0x7fe924017070 con 0x565431965300 2023-05-20T01:18:25.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.768+0000 7fe933b3f700 1 -- 172.21.15.16:0/3156188675 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:28 6.1b 6:d8000000:%01:::head [pgnls start_epoch 46 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e46) v8 -- 0x5654319ef7b0 con 0x565431965300 2023-05-20T01:18:25.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.768+0000 7fe933b3f700 1 -- 172.21.15.16:0/3156188675 <== osd.3 v2:172.21.15.16:6800/3669781890 6 ==== osd_op_reply(28 [pgnls start_epoch 46 out=49b] v39'1 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe924017070 con 0x565431965300 2023-05-20T01:18:25.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.768+0000 7fe933b3f700 1 -- 172.21.15.16:0/3156188675 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:29 6.7 6:e0000000:%01:::head [pgnls start_epoch 46 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e46) v8 -- 0x5654319ef7b0 con 0x7fe92402e980 2023-05-20T01:18:25.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.769+0000 7fe934340700 1 -- 172.21.15.16:0/3156188675 <== osd.5 v2:172.21.15.132:6816/1844689995 4 ==== osd_op_reply(29 [pgnls start_epoch 46 out=73b] v39'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x7fe92c024040 con 0x7fe92402e980 2023-05-20T01:18:25.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.769+0000 7fe934340700 1 -- 172.21.15.16:0/3156188675 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:30 6.17 6:e8000000:%01:::head [pgnls start_epoch 46 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e46) v8 -- 0x5654319ef7b0 con 0x7fe924019400 2023-05-20T01:18:25.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.769+0000 7fe93333e700 1 -- 172.21.15.16:0/3156188675 <== osd.4 v2:172.21.15.132:6817/506317223 5 ==== osd_op_reply(30 [pgnls start_epoch 46 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe92805e040 con 0x7fe924019400 2023-05-20T01:18:25.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.769+0000 7fe93333e700 1 -- 172.21.15.16:0/3156188675 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:31 6.f 6:f0000000:%01:::head [pgnls start_epoch 46 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e46) v8 -- 0x5654319ef7b0 con 0x7fe928009040 2023-05-20T01:18:25.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.769+0000 7fe934340700 1 -- 172.21.15.16:0/3156188675 <== osd.2 v2:172.21.15.16:6816/293187030 2 ==== osd_op_reply(31 [pgnls start_epoch 46 out=73b] v39'1 uv1 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x7fe92c024040 con 0x7fe928009040 2023-05-20T01:18:25.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.769+0000 7fe934340700 1 -- 172.21.15.16:0/3156188675 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:32 6.1f 6:f8000000:%01:::head [pgnls start_epoch 46 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e46) v8 -- 0x5654319ef7b0 con 0x565431965300 2023-05-20T01:18:25.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.769+0000 7fe933b3f700 1 -- 172.21.15.16:0/3156188675 <== osd.3 v2:172.21.15.16:6800/3669781890 7 ==== osd_op_reply(32 [pgnls start_epoch 46 out=101b] v45'6 uv6 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x7fe924017070 con 0x565431965300 2023-05-20T01:18:25.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.769+0000 7fe93da3f080 1 -- 172.21.15.16:0/3156188675 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:33 6.10 6:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e46) v8 -- 0x565431a81ea0 con 0x5654319e9120 2023-05-20T01:18:25.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.770+0000 7fe93da3f080 1 -- 172.21.15.16:0/3156188675 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:34 6.10 6:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e46) v8 -- 0x565431a82940 con 0x5654319e9120 2023-05-20T01:18:25.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.770+0000 7fe93da3f080 1 -- 172.21.15.16:0/3156188675 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:35 6.10 6:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e46) v8 -- 0x565431a9c040 con 0x5654319e9120 2023-05-20T01:18:25.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.770+0000 7fe93da3f080 1 -- 172.21.15.16:0/3156188675 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:36 6.10 6:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e46) v8 -- 0x565431a9c840 con 0x5654319e9120 2023-05-20T01:18:25.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.770+0000 7fe93da3f080 1 -- 172.21.15.16:0/3156188675 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:37 6.10 6:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e46) v8 -- 0x565431a9d040 con 0x5654319e9120 2023-05-20T01:18:25.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.770+0000 7fe93da3f080 1 -- 172.21.15.16:0/3156188675 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:38 6.14 6:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e46) v8 -- 0x565431a9d840 con 0x7fe928009040 2023-05-20T01:18:25.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.770+0000 7fe93da3f080 1 -- 172.21.15.16:0/3156188675 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:39 6.14 6:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e46) v8 -- 0x565431a9e040 con 0x7fe928009040 2023-05-20T01:18:25.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.770+0000 7fe93da3f080 1 -- 172.21.15.16:0/3156188675 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:40 6.14 6:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e46) v8 -- 0x565431a9e840 con 0x7fe928009040 2023-05-20T01:18:25.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.770+0000 7fe93da3f080 1 -- 172.21.15.16:0/3156188675 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:41 6.14 6:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e46) v8 -- 0x565431a9f040 con 0x7fe928009040 2023-05-20T01:18:25.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.770+0000 7fe93da3f080 1 -- 172.21.15.16:0/3156188675 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:42 6.1c 6:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e46) v8 -- 0x565431a9f840 con 0x7fe91c0035f0 2023-05-20T01:18:25.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.770+0000 7fe93da3f080 1 -- 172.21.15.16:0/3156188675 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:43 6.6 6:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e46) v8 -- 0x565431aa0270 con 0x565431965300 2023-05-20T01:18:25.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.770+0000 7fe93da3f080 1 -- 172.21.15.16:0/3156188675 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:44 6.e 6:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e46) v8 -- 0x565431aa0ca0 con 0x7fe924019400 2023-05-20T01:18:25.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.770+0000 7fe93da3f080 1 -- 172.21.15.16:0/3156188675 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:45 6.15 6:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e46) v8 -- 0x565431a9f040 con 0x7fe91c0035f0 2023-05-20T01:18:25.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.770+0000 7fe93da3f080 1 -- 172.21.15.16:0/3156188675 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:46 6.15 6:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e46) v8 -- 0x565431aa1c10 con 0x7fe91c0035f0 2023-05-20T01:18:25.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.770+0000 7fe93da3f080 1 -- 172.21.15.16:0/3156188675 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:47 6.d 6:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e46) v8 -- 0x565431aa2700 con 0x7fe92402e980 2023-05-20T01:18:25.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.770+0000 7fe93da3f080 1 -- 172.21.15.16:0/3156188675 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:48 6.1d 6:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e46) v8 -- 0x565431aa31a0 con 0x565431964f30 2023-05-20T01:18:25.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.771+0000 7fe933b3f700 1 -- 172.21.15.16:0/3156188675 <== osd.7 v2:172.21.15.132:6800/466001149 7 ==== osd_op_reply(45 604.00000000 [delete] v46'4 uv4 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe924017070 con 0x7fe91c0035f0 2023-05-20T01:18:25.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.772+0000 7fe933b3f700 1 -- 172.21.15.16:0/3156188675 <== osd.7 v2:172.21.15.132:6800/466001149 8 ==== osd_op_reply(42 607.00000000 [delete] v46'3 uv3 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe924017070 con 0x7fe91c0035f0 2023-05-20T01:18:25.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.772+0000 7fe93da3f080 1 -- 172.21.15.16:0/3156188675 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:49 6.13 6:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e46) v8 -- 0x565431aa1000 con 0x565431965300 2023-05-20T01:18:25.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.772+0000 7fe93da3f080 1 -- 172.21.15.16:0/3156188675 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:50 6.13 6:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e46) v8 -- 0x565431aa39f0 con 0x565431965300 2023-05-20T01:18:25.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.772+0000 7fe933b3f700 1 -- 172.21.15.16:0/3156188675 <== osd.7 v2:172.21.15.132:6800/466001149 9 ==== osd_op_reply(46 500.00000000 [delete] v46'5 uv5 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe924017070 con 0x7fe91c0035f0 2023-05-20T01:18:25.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.772+0000 7fe93da3f080 1 -- 172.21.15.16:0/3156188675 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:51 6.b 6:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e46) v8 -- 0x565431aa3d50 con 0x565431965300 2023-05-20T01:18:25.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.772+0000 7fe93333e700 1 -- 172.21.15.16:0/3156188675 <== osd.4 v2:172.21.15.132:6817/506317223 6 ==== osd_op_reply(44 608.00000000 [delete] v46'3 uv3 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe92805e040 con 0x7fe924019400 2023-05-20T01:18:25.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.772+0000 7fe934340700 1 -- 172.21.15.16:0/3156188675 <== osd.0 v2:172.21.15.16:6824/3923634179 6 ==== osd_op_reply(33 601.00000000 [delete] v46'11 uv11 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe92c024040 con 0x5654319e9120 2023-05-20T01:18:25.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.772+0000 7fe934340700 1 -- 172.21.15.16:0/3156188675 <== osd.5 v2:172.21.15.132:6816/1844689995 5 ==== osd_op_reply(47 mds_snaptable [delete] v46'2 uv2 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fe92c024040 con 0x7fe92402e980 2023-05-20T01:18:25.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.772+0000 7fe93da3f080 1 -- 172.21.15.16:0/3156188675 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:52 6.7 6:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e46) v8 -- 0x565431aa2140 con 0x7fe92402e980 2023-05-20T01:18:25.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.772+0000 7fe93da3f080 1 -- 172.21.15.16:0/3156188675 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:53 6.f 6:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e46) v8 -- 0x565431aa3d50 con 0x7fe928009040 2023-05-20T01:18:25.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.772+0000 7fe93da3f080 1 -- 172.21.15.16:0/3156188675 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:54 6.1f 6:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e46) v8 -- 0x565431aa4550 con 0x565431965300 2023-05-20T01:18:25.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.773+0000 7fe93333e700 1 -- 172.21.15.16:0/3156188675 <== osd.1 v2:172.21.15.16:6808/3860264220 4 ==== osd_op_reply(48 605.00000000 [delete] v46'3 uv3 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe92805e040 con 0x565431964f30 2023-05-20T01:18:25.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.773+0000 7fe93da3f080 1 -- 172.21.15.16:0/3156188675 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:55 6.1f 6:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e46) v8 -- 0x565431aa3d50 con 0x565431965300 2023-05-20T01:18:25.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.773+0000 7fe933b3f700 1 -- 172.21.15.16:0/3156188675 <== osd.3 v2:172.21.15.16:6800/3669781890 8 ==== osd_op_reply(43 mds0_openfiles.0 [delete] v46'9 uv9 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fe924017070 con 0x565431965300 2023-05-20T01:18:25.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.773+0000 7fe934340700 1 -- 172.21.15.16:0/3156188675 <== osd.2 v2:172.21.15.16:6816/293187030 3 ==== osd_op_reply(38 1.00000000.inode [delete] v46'42 uv42 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fe92c024040 con 0x7fe928009040 2023-05-20T01:18:25.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.773+0000 7fe934340700 1 -- 172.21.15.16:0/3156188675 <== osd.2 v2:172.21.15.16:6816/293187030 4 ==== osd_op_reply(39 200.00000000 [delete] v46'43 uv43 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe92c024040 con 0x7fe928009040 2023-05-20T01:18:25.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.773+0000 7fe934340700 1 -- 172.21.15.16:0/3156188675 <== osd.0 v2:172.21.15.16:6824/3923634179 7 ==== osd_op_reply(34 602.00000000 [delete] v46'12 uv12 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe92c024040 con 0x5654319e9120 2023-05-20T01:18:25.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.774+0000 7fe934340700 1 -- 172.21.15.16:0/3156188675 <== osd.2 v2:172.21.15.16:6816/293187030 5 ==== osd_op_reply(40 200.00000001 [delete] v46'44 uv44 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe92c024040 con 0x7fe928009040 2023-05-20T01:18:25.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.774+0000 7fe934340700 1 -- 172.21.15.16:0/3156188675 <== osd.2 v2:172.21.15.16:6816/293187030 6 ==== osd_op_reply(41 606.00000000 [delete] v46'45 uv45 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe92c024040 con 0x7fe928009040 2023-05-20T01:18:25.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.774+0000 7fe934340700 1 -- 172.21.15.16:0/3156188675 <== osd.0 v2:172.21.15.16:6824/3923634179 8 ==== osd_op_reply(35 10000000000.00000000 [delete] v46'13 uv13 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fe92c024040 con 0x5654319e9120 2023-05-20T01:18:25.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.774+0000 7fe934340700 1 -- 172.21.15.16:0/3156188675 <== osd.0 v2:172.21.15.16:6824/3923634179 9 ==== osd_op_reply(36 600.00000000 [delete] v46'14 uv14 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe92c024040 con 0x5654319e9120 2023-05-20T01:18:25.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.774+0000 7fe934340700 1 -- 172.21.15.16:0/3156188675 <== osd.0 v2:172.21.15.16:6824/3923634179 10 ==== osd_op_reply(37 603.00000000 [delete] v46'15 uv15 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe92c024040 con 0x5654319e9120 2023-05-20T01:18:25.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.774+0000 7fe933b3f700 1 -- 172.21.15.16:0/3156188675 <== osd.3 v2:172.21.15.16:6800/3669781890 9 ==== osd_op_reply(54 100.00000000.inode [delete] v46'7 uv7 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7fe924017070 con 0x565431965300 2023-05-20T01:18:25.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.774+0000 7fe933b3f700 1 -- 172.21.15.16:0/3156188675 <== osd.3 v2:172.21.15.16:6800/3669781890 10 ==== osd_op_reply(49 mds0_inotable [delete] v46'4 uv4 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fe924017070 con 0x565431965300 2023-05-20T01:18:25.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.774+0000 7fe933b3f700 1 -- 172.21.15.16:0/3156188675 <== osd.3 v2:172.21.15.16:6800/3669781890 11 ==== osd_op_reply(50 100.00000000 [delete] v46'5 uv5 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe924017070 con 0x565431965300 2023-05-20T01:18:25.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.774+0000 7fe933b3f700 1 -- 172.21.15.16:0/3156188675 <== osd.3 v2:172.21.15.16:6800/3669781890 12 ==== osd_op_reply(51 mds0_sessionmap [delete] v46'4 uv4 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7fe924017070 con 0x565431965300 2023-05-20T01:18:25.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.774+0000 7fe934340700 1 -- 172.21.15.16:0/3156188675 <== osd.2 v2:172.21.15.16:6816/293187030 7 ==== osd_op_reply(53 400.00000000 [delete] v46'2 uv2 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe92c024040 con 0x7fe928009040 2023-05-20T01:18:25.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.774+0000 7fe934340700 1 -- 172.21.15.16:0/3156188675 <== osd.5 v2:172.21.15.132:6816/1844689995 6 ==== osd_op_reply(52 609.00000000 [delete] v46'3 uv3 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe92c024040 con 0x7fe92402e980 2023-05-20T01:18:25.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.776+0000 7fe933b3f700 1 -- 172.21.15.16:0/3156188675 <== osd.3 v2:172.21.15.16:6800/3669781890 13 ==== osd_op_reply(55 1.00000000 [delete] v46'8 uv8 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fe924017070 con 0x565431965300 2023-05-20T01:18:25.802 INFO:teuthology.orchestra.run.smithi016.stdout:Removed 23 objects 2023-05-20T01:18:25.802 INFO:teuthology.orchestra.run.smithi016.stdout:successfully purged pool cephfs_metadata 2023-05-20T01:18:25.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.777+0000 7fe93da3f080 1 -- 172.21.15.16:0/3156188675 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x5654319e9120 msgr2=0x565431a9b660 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:25.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.777+0000 7fe93da3f080 1 --2- 172.21.15.16:0/3156188675 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x5654319e9120 0x565431a9b660 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:25.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.777+0000 7fe93da3f080 1 -- 172.21.15.16:0/3156188675 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x565431964f30 msgr2=0x7fe92c029980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:25.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.777+0000 7fe93da3f080 1 --2- 172.21.15.16:0/3156188675 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x565431964f30 0x7fe92c029980 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:25.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.777+0000 7fe93da3f080 1 -- 172.21.15.16:0/3156188675 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x7fe928009040 msgr2=0x7fe9280590b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:25.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.777+0000 7fe93da3f080 1 --2- 172.21.15.16:0/3156188675 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x7fe928009040 0x7fe9280590b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:25.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.777+0000 7fe93da3f080 1 -- 172.21.15.16:0/3156188675 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x565431965300 msgr2=0x7fe92c01a8c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:25.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.777+0000 7fe93da3f080 1 --2- 172.21.15.16:0/3156188675 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x565431965300 0x7fe92c01a8c0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:25.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.777+0000 7fe93da3f080 1 -- 172.21.15.16:0/3156188675 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x7fe924019400 msgr2=0x7fe924019830 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:25.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.777+0000 7fe93da3f080 1 --2- 172.21.15.16:0/3156188675 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x7fe924019400 0x7fe924019830 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:25.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.777+0000 7fe934340700 1 -- 172.21.15.16:0/3156188675 reap_dead start 2023-05-20T01:18:25.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.777+0000 7fe93da3f080 1 -- 172.21.15.16:0/3156188675 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x7fe92402e980 msgr2=0x7fe92402b3d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:25.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.777+0000 7fe93da3f080 1 --2- 172.21.15.16:0/3156188675 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x7fe92402e980 0x7fe92402b3d0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:25.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.777+0000 7fe934340700 1 -- 172.21.15.16:0/3156188675 reap_dead start 2023-05-20T01:18:25.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.777+0000 7fe934340700 1 -- 172.21.15.16:0/3156188675 reap_dead start 2023-05-20T01:18:25.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.777+0000 7fe93da3f080 1 -- 172.21.15.16:0/3156188675 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x7fe91c0035f0 msgr2=0x7fe92c01c940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:25.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.777+0000 7fe93da3f080 1 --2- 172.21.15.16:0/3156188675 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x7fe91c0035f0 0x7fe92c01c940 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:25.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.777+0000 7fe93da3f080 1 -- 172.21.15.16:0/3156188675 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe920069600 msgr2=0x7fe92006bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:25.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.777+0000 7fe93da3f080 1 --2- 172.21.15.16:0/3156188675 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe920069600 0x7fe92006bab0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x5654319f16e0 tx=0x7fe924011040 comp rx=0 tx=0).stop 2023-05-20T01:18:25.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.777+0000 7fe93da3f080 1 -- 172.21.15.16:0/3156188675 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56543194e880 msgr2=0x56543191f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:25.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.777+0000 7fe93da3f080 1 --2- 172.21.15.16:0/3156188675 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56543194e880 0x56543191f8b0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fe928017000 tx=0x7fe928002fb0 comp rx=0 tx=0).stop 2023-05-20T01:18:25.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.777+0000 7fe93da3f080 1 -- 172.21.15.16:0/3156188675 shutdown_connections 2023-05-20T01:18:25.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.777+0000 7fe93da3f080 1 --2- 172.21.15.16:0/3156188675 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x7fe92402e980 0x7fe92402b3d0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:25.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.777+0000 7fe93da3f080 1 --2- 172.21.15.16:0/3156188675 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe920069600 0x7fe92006bab0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:25.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.777+0000 7fe93da3f080 1 --2- 172.21.15.16:0/3156188675 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x7fe91c0035f0 0x7fe92c01c940 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:25.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.777+0000 7fe93da3f080 1 --2- 172.21.15.16:0/3156188675 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56543194e880 0x56543191f8b0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:25.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.777+0000 7fe93da3f080 1 -- 172.21.15.16:0/3156188675 >> 172.21.15.16:0/3156188675 conn(0x5654319680a0 msgr2=0x565431920550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:18:25.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.777+0000 7fe93da3f080 1 -- 172.21.15.16:0/3156188675 shutdown_connections 2023-05-20T01:18:25.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.777+0000 7fe93da3f080 1 -- 172.21.15.16:0/3156188675 wait complete. 2023-05-20T01:18:25.809 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs reset cephfs --yes-i-really-mean-it 2023-05-20T01:18:25.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.969+0000 7fbf45520700 1 Processor -- start 2023-05-20T01:18:25.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.969+0000 7fbf45520700 1 -- start start 2023-05-20T01:18:25.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.970+0000 7fbf45520700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fbf4015ca90 0x7fbf400cd890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:25.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.970+0000 7fbf45520700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf400d1820 con 0x7fbf400d29a0 2023-05-20T01:18:25.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.970+0000 7fbf45520700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf400cdf00 con 0x7fbf400d2550 2023-05-20T01:18:25.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.970+0000 7fbf45520700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf400d0c50 con 0x7fbf4015ca90 2023-05-20T01:18:25.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.970+0000 7fbf3f7fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fbf400d29a0 0x7fbf400cf570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:43944/0 (socket says 172.21.15.16:43944) 2023-05-20T01:18:25.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.970+0000 7fbf3f7fe700 1 -- 172.21.15.16:0/913802122 learned_addr learned my addr 172.21.15.16:0/913802122 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:18:25.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.970+0000 7fbf3effd700 1 --2- 172.21.15.16:0/913802122 >> v2:172.21.15.132:3300/0 conn(0x7fbf4015ca90 0x7fbf400cd890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:25.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.970+0000 7fbf3dffb700 1 -- 172.21.15.16:0/913802122 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 732475943 0 0) 0x7fbf400d1820 con 0x7fbf400d29a0 2023-05-20T01:18:25.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.970+0000 7fbf3dffb700 1 -- 172.21.15.16:0/913802122 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf280035b0 con 0x7fbf400d29a0 2023-05-20T01:18:25.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.970+0000 7fbf3effd700 1 -- 172.21.15.16:0/913802122 >> v1:172.21.15.132:6789/0 conn(0x7fbf400d2550 legacy=0x7fbf400b6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:25.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.970+0000 7fbf3effd700 1 -- 172.21.15.16:0/913802122 >> v1:172.21.15.16:6789/0 conn(0x7fbf400d29a0 legacy=0x7fbf400cf570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:25.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.970+0000 7fbf3effd700 1 -- 172.21.15.16:0/913802122 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf400d1a70 con 0x7fbf4015ca90 2023-05-20T01:18:25.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.970+0000 7fbf3effd700 1 --2- 172.21.15.16:0/913802122 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbf4015ca90 0x7fbf400cd890 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fbf30009840 tx=0x7fbf30009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b32e313a6eaa666 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:25.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.971+0000 7fbf3dffb700 1 -- 172.21.15.16:0/913802122 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf30011830 con 0x7fbf4015ca90 2023-05-20T01:18:25.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.971+0000 7fbf3dffb700 1 -- 172.21.15.16:0/913802122 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf30011990 con 0x7fbf4015ca90 2023-05-20T01:18:25.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.971+0000 7fbf3dffb700 1 -- 172.21.15.16:0/913802122 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf30011d20 con 0x7fbf4015ca90 2023-05-20T01:18:25.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.971+0000 7fbf45520700 1 -- 172.21.15.16:0/913802122 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbf4015ca90 msgr2=0x7fbf400cd890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:25.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.971+0000 7fbf45520700 1 --2- 172.21.15.16:0/913802122 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbf4015ca90 0x7fbf400cd890 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fbf30009840 tx=0x7fbf30009b50 comp rx=0 tx=0).stop 2023-05-20T01:18:25.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.971+0000 7fbf45520700 1 -- 172.21.15.16:0/913802122 shutdown_connections 2023-05-20T01:18:25.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.971+0000 7fbf45520700 1 --2- 172.21.15.16:0/913802122 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbf4015ca90 0x7fbf400cd890 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:25.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.971+0000 7fbf45520700 1 -- 172.21.15.16:0/913802122 >> 172.21.15.16:0/913802122 conn(0x7fbf4006b000 msgr2=0x7fbf4006b400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:18:25.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.971+0000 7fbf45520700 1 -- 172.21.15.16:0/913802122 shutdown_connections 2023-05-20T01:18:25.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.971+0000 7fbf45520700 1 -- 172.21.15.16:0/913802122 wait complete. 2023-05-20T01:18:25.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.973+0000 7fbf45520700 1 Processor -- start 2023-05-20T01:18:25.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.973+0000 7fbf45520700 1 -- start start 2023-05-20T01:18:25.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.974+0000 7fbf45520700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbf400d2550 0x7fbf4015d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:25.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.974+0000 7fbf45520700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbf400d29a0 0x7fbf40148ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:25.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.974+0000 7fbf45520700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf4015ca90 0x7fbf400c6560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:25.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.974+0000 7fbf45520700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf401493e0 con 0x7fbf4015ca90 2023-05-20T01:18:25.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.974+0000 7fbf45520700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf4015d8a0 con 0x7fbf400d2550 2023-05-20T01:18:25.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.974+0000 7fbf45520700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf400b35f0 con 0x7fbf400d29a0 2023-05-20T01:18:25.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.974+0000 7fbf3f7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf4015ca90 0x7fbf400c6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:25.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.974+0000 7fbf3f7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf4015ca90 0x7fbf400c6560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60654/0 (socket says 172.21.15.16:60654) 2023-05-20T01:18:25.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.974+0000 7fbf3f7fe700 1 -- 172.21.15.16:0/2133408772 learned_addr learned my addr 172.21.15.16:0/2133408772 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:18:25.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.974+0000 7fbf3effd700 1 --2- 172.21.15.16:0/2133408772 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbf400d2550 0x7fbf4015d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:25.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.974+0000 7fbf3e7fc700 1 --2- 172.21.15.16:0/2133408772 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbf400d29a0 0x7fbf40148ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:25.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.974+0000 7fbf3f7fe700 1 -- 172.21.15.16:0/2133408772 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbf400d29a0 msgr2=0x7fbf40148ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:25.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.974+0000 7fbf3f7fe700 1 --2- 172.21.15.16:0/2133408772 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbf400d29a0 0x7fbf40148ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:25.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.974+0000 7fbf3f7fe700 1 -- 172.21.15.16:0/2133408772 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbf400d2550 msgr2=0x7fbf4015d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:25.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.974+0000 7fbf3f7fe700 1 --2- 172.21.15.16:0/2133408772 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbf400d2550 0x7fbf4015d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:25.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.974+0000 7fbf3f7fe700 1 -- 172.21.15.16:0/2133408772 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf300094a0 con 0x7fbf4015ca90 2023-05-20T01:18:25.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.975+0000 7fbf3f7fe700 1 --2- 172.21.15.16:0/2133408772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf4015ca90 0x7fbf400c6560 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fbf3800cb40 tx=0x7fbf3800ce50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:25.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.975+0000 7fbf277fe700 1 -- 172.21.15.16:0/2133408772 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf3800a9e0 con 0x7fbf4015ca90 2023-05-20T01:18:25.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.975+0000 7fbf277fe700 1 -- 172.21.15.16:0/2133408772 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf3800ab40 con 0x7fbf4015ca90 2023-05-20T01:18:25.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.975+0000 7fbf45520700 1 -- 172.21.15.16:0/2133408772 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf400d2030 con 0x7fbf4015ca90 2023-05-20T01:18:25.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.975+0000 7fbf277fe700 1 -- 172.21.15.16:0/2133408772 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf38012400 con 0x7fbf4015ca90 2023-05-20T01:18:25.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.975+0000 7fbf45520700 1 -- 172.21.15.16:0/2133408772 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf400d1d50 con 0x7fbf4015ca90 2023-05-20T01:18:25.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.976+0000 7fbf45520700 1 -- 172.21.15.16:0/2133408772 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf4015ce60 con 0x7fbf4015ca90 2023-05-20T01:18:25.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.976+0000 7fbf277fe700 1 -- 172.21.15.16:0/2133408772 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fbf3801b3b0 con 0x7fbf4015ca90 2023-05-20T01:18:25.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.977+0000 7fbf277fe700 1 --2- 172.21.15.16:0/2133408772 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fbf2c06ace0 0x7fbf2c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:25.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.977+0000 7fbf277fe700 1 -- 172.21.15.16:0/2133408772 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6930+0+0 (secure 0 0 0) 0x7fbf38089290 con 0x7fbf4015ca90 2023-05-20T01:18:25.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.977+0000 7fbf3effd700 1 --2- 172.21.15.16:0/2133408772 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fbf2c06ace0 0x7fbf2c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:25.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.977+0000 7fbf3effd700 1 --2- 172.21.15.16:0/2133408772 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fbf2c06ace0 0x7fbf2c06d190 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fbf300090b0 tx=0x7fbf3001c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:25.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:25.981+0000 7fbf277fe700 1 -- 172.21.15.16:0/2133408772 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fbf38054110 con 0x7fbf4015ca90 2023-05-20T01:18:26.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:26.226+0000 7fbf45520700 1 -- 172.21.15.16:0/2133408772 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs reset", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fbf4004e8c0 con 0x7fbf4015ca90 2023-05-20T01:18:26.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:26.458+0000 7fbf277fe700 1 -- 172.21.15.16:0/2133408772 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs reset", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v19) v1 ==== 107+0+0 (secure 0 0 0) 0x7fbf38057730 con 0x7fbf4015ca90 2023-05-20T01:18:26.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:26.461+0000 7fbf45520700 1 -- 172.21.15.16:0/2133408772 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fbf2c06ace0 msgr2=0x7fbf2c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:26.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:26.461+0000 7fbf45520700 1 --2- 172.21.15.16:0/2133408772 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fbf2c06ace0 0x7fbf2c06d190 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fbf300090b0 tx=0x7fbf3001c040 comp rx=0 tx=0).stop 2023-05-20T01:18:26.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:26.461+0000 7fbf45520700 1 -- 172.21.15.16:0/2133408772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf4015ca90 msgr2=0x7fbf400c6560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:26.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:26.461+0000 7fbf45520700 1 --2- 172.21.15.16:0/2133408772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf4015ca90 0x7fbf400c6560 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fbf3800cb40 tx=0x7fbf3800ce50 comp rx=0 tx=0).stop 2023-05-20T01:18:26.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:26.461+0000 7fbf45520700 1 -- 172.21.15.16:0/2133408772 shutdown_connections 2023-05-20T01:18:26.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:26.461+0000 7fbf45520700 1 --2- 172.21.15.16:0/2133408772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf4015ca90 0x7fbf400c6560 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:26.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:26.461+0000 7fbf45520700 1 --2- 172.21.15.16:0/2133408772 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbf400d2550 0x7fbf4015d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:26.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:26.461+0000 7fbf45520700 1 --2- 172.21.15.16:0/2133408772 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fbf2c06ace0 0x7fbf2c06d190 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:26.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:26.461+0000 7fbf45520700 1 --2- 172.21.15.16:0/2133408772 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbf400d29a0 0x7fbf40148ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:26.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:26.461+0000 7fbf45520700 1 -- 172.21.15.16:0/2133408772 >> 172.21.15.16:0/2133408772 conn(0x7fbf4006b000 msgr2=0x7fbf40159ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:18:26.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:26.461+0000 7fbf45520700 1 -- 172.21.15.16:0/2133408772 shutdown_connections 2023-05-20T01:18:26.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:26.461+0000 7fbf45520700 1 -- 172.21.15.16:0/2133408772 wait complete. 2023-05-20T01:18:26.469 INFO:tasks.ceph.mds.c.smithi016.stderr:2023-05-20T01:18:26.466+0000 7fe397322700 -1 mds.0.inotable: load_2 could not read table: -2 2023-05-20T01:18:26.469 INFO:tasks.ceph.mds.c.smithi016.stderr:2023-05-20T01:18:26.466+0000 7fe397322700 -1 log_channel(cluster) log [ERR] : error reading table object 'mds0_inotable' -2 ((2) No such file or directory) 2023-05-20T01:18:26.480 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-05-20T01:18:26.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:26.606+0000 7fa19f8f3700 1 Processor -- start 2023-05-20T01:18:26.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:26.606+0000 7fa19f8f3700 1 -- start start 2023-05-20T01:18:26.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:26.607+0000 7fa19f8f3700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa19815cf40 0x7fa19806e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:26.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:26.607+0000 7fa19f8f3700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa198072430 con 0x7fa1980710d0 2023-05-20T01:18:26.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:26.607+0000 7fa19f8f3700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa198071820 con 0x7fa1980cd070 2023-05-20T01:18:26.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:26.607+0000 7fa19f8f3700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa198071b00 con 0x7fa19815cf40 2023-05-20T01:18:26.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:26.607+0000 7fa19ce8e700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fa1980710d0 0x7fa1980cc960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:43964/0 (socket says 172.21.15.16:43964) 2023-05-20T01:18:26.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:26.607+0000 7fa19ce8e700 1 -- 172.21.15.16:0/2583270981 learned_addr learned my addr 172.21.15.16:0/2583270981 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:18:26.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:26.607+0000 7fa18ffff700 1 -- 172.21.15.16:0/2583270981 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2801990652 0 0) 0x7fa198072430 con 0x7fa1980710d0 2023-05-20T01:18:26.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:26.607+0000 7fa18ffff700 1 -- 172.21.15.16:0/2583270981 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa1840035b0 con 0x7fa1980710d0 2023-05-20T01:18:26.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:26.607+0000 7fa19d68f700 1 --2- 172.21.15.16:0/2583270981 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa19815cf40 0x7fa19806e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:26.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:26.607+0000 7fa18ffff700 1 -- 172.21.15.16:0/2583270981 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2069150561 0 0) 0x7fa1840035b0 con 0x7fa1980710d0 2023-05-20T01:18:26.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:26.607+0000 7fa18ffff700 1 -- 172.21.15.16:0/2583270981 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa19815cf40 msgr2=0x7fa19806e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:18:26.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:26.607+0000 7fa18ffff700 1 --2- 172.21.15.16:0/2583270981 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa19815cf40 0x7fa19806e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:26.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:26.608+0000 7fa18ffff700 1 -- 172.21.15.16:0/2583270981 >> v1:172.21.15.132:6789/0 conn(0x7fa1980cd070 legacy=0x7fa1980cabb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:26.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:26.608+0000 7fa18ffff700 1 -- 172.21.15.16:0/2583270981 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa198071de0 con 0x7fa1980710d0 2023-05-20T01:18:26.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:26.608+0000 7fa18ffff700 1 -- 172.21.15.16:0/2583270981 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7fa194001670 con 0x7fa1980710d0 2023-05-20T01:18:26.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:26.608+0000 7fa18ffff700 1 -- 172.21.15.16:0/2583270981 >> v1:172.21.15.16:6789/0 conn(0x7fa1980710d0 legacy=0x7fa1980cc960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:26.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:26.608+0000 7fa18ffff700 1 --2- 172.21.15.16:0/2583270981 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa184003a30 0x7fa184003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:26.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:26.608+0000 7fa18ffff700 1 --2- 172.21.15.16:0/2583270981 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa19815cf40 0x7fa184004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:26.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:26.608+0000 7fa18ffff700 1 --2- 172.21.15.16:0/2583270981 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa184004d90 0x7fa184005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:26.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:26.608+0000 7fa18ffff700 1 -- 172.21.15.16:0/2583270981 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa198071b00 con 0x7fa184004d90 2023-05-20T01:18:26.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:26.608+0000 7fa18ffff700 1 -- 172.21.15.16:0/2583270981 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa18400dcd0 con 0x7fa184003a30 2023-05-20T01:18:26.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:26.608+0000 7fa18ffff700 1 -- 172.21.15.16:0/2583270981 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa18400de60 con 0x7fa19815cf40 2023-05-20T01:18:26.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:26.608+0000 7fa19de90700 1 --2- 172.21.15.16:0/2583270981 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa184004d90 0x7fa184005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:26.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:26.608+0000 7fa19ce8e700 1 --2- 172.21.15.16:0/2583270981 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa19815cf40 0x7fa184004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:26.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:26.608+0000 7fa19d68f700 1 --2- 172.21.15.16:0/2583270981 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa184003a30 0x7fa184003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:26.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:26.608+0000 7fa19de90700 1 -- 172.21.15.16:0/2583270981 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa19815cf40 msgr2=0x7fa184004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:18:26.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:26.608+0000 7fa19de90700 1 --2- 172.21.15.16:0/2583270981 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa19815cf40 0x7fa184004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:26.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:26.608+0000 7fa19de90700 1 -- 172.21.15.16:0/2583270981 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa184003a30 msgr2=0x7fa184003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:18:26.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:26.608+0000 7fa19de90700 1 --2- 172.21.15.16:0/2583270981 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa184003a30 0x7fa184003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:26.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:26.608+0000 7fa19de90700 1 -- 172.21.15.16:0/2583270981 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1840035b0 con 0x7fa184004d90 2023-05-20T01:18:26.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:26.608+0000 7fa19de90700 1 --2- 172.21.15.16:0/2583270981 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa184004d90 0x7fa184005200 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fa19000c190 tx=0x7fa184005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fba91ed049ab6e0c server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:26.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:26.608+0000 7fa18ffff700 1 -- 172.21.15.16:0/2583270981 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa190001db0 con 0x7fa184004d90 2023-05-20T01:18:26.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:26.608+0000 7fa18ffff700 1 -- 172.21.15.16:0/2583270981 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa19000ccd0 con 0x7fa184004d90 2023-05-20T01:18:26.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:26.608+0000 7fa18ffff700 1 -- 172.21.15.16:0/2583270981 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa190003c40 con 0x7fa184004d90 2023-05-20T01:18:26.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:26.609+0000 7fa19f8f3700 1 -- 172.21.15.16:0/2583270981 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa184004d90 msgr2=0x7fa184005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:26.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:26.609+0000 7fa19f8f3700 1 --2- 172.21.15.16:0/2583270981 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa184004d90 0x7fa184005200 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fa19000c190 tx=0x7fa184005880 comp rx=0 tx=0).stop 2023-05-20T01:18:26.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:26.609+0000 7fa19de90700 1 -- 172.21.15.16:0/2583270981 reap_dead start 2023-05-20T01:18:26.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:26.609+0000 7fa19f8f3700 1 -- 172.21.15.16:0/2583270981 shutdown_connections 2023-05-20T01:18:26.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:26.609+0000 7fa19f8f3700 1 -- 172.21.15.16:0/2583270981 >> 172.21.15.16:0/2583270981 conn(0x7fa19806c260 msgr2=0x7fa1980cffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:18:26.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:26.609+0000 7fa19f8f3700 1 -- 172.21.15.16:0/2583270981 shutdown_connections 2023-05-20T01:18:26.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:26.609+0000 7fa19f8f3700 1 -- 172.21.15.16:0/2583270981 wait complete. 2023-05-20T01:18:26.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:26.610+0000 7fa19f8f3700 1 Processor -- start 2023-05-20T01:18:26.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:26.610+0000 7fa19f8f3700 1 -- start start 2023-05-20T01:18:26.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:26.611+0000 7fa19f8f3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa19814e700 0x7fa1980af780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:26.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:26.611+0000 7fa19f8f3700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa19814ead0 0x7fa1980b5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:26.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:26.611+0000 7fa19f8f3700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa1980b57d0 0x7fa1980c3b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:26.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:26.611+0000 7fa19f8f3700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1980b5c40 con 0x7fa19814e700 2023-05-20T01:18:26.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:26.611+0000 7fa19f8f3700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1980b3ba0 con 0x7fa19814ead0 2023-05-20T01:18:26.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:26.611+0000 7fa19f8f3700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1980c7520 con 0x7fa1980b57d0 2023-05-20T01:18:26.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:26.611+0000 7fa19d68f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa19814e700 0x7fa1980af780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:26.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:26.611+0000 7fa19d68f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa19814e700 0x7fa1980af780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60678/0 (socket says 172.21.15.16:60678) 2023-05-20T01:18:26.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:26.611+0000 7fa19d68f700 1 -- 172.21.15.16:0/3540703595 learned_addr learned my addr 172.21.15.16:0/3540703595 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:18:26.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:26.611+0000 7fa19ce8e700 1 --2- 172.21.15.16:0/3540703595 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa19814ead0 0x7fa1980b5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:26.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:26.611+0000 7fa19de90700 1 --2- 172.21.15.16:0/3540703595 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa1980b57d0 0x7fa1980c3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:26.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:26.611+0000 7fa19d68f700 1 -- 172.21.15.16:0/3540703595 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa1980b57d0 msgr2=0x7fa1980c3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:26.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:26.611+0000 7fa19d68f700 1 --2- 172.21.15.16:0/3540703595 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa1980b57d0 0x7fa1980c3b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:26.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:26.611+0000 7fa19d68f700 1 -- 172.21.15.16:0/3540703595 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa19814ead0 msgr2=0x7fa1980b5290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:26.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:26.611+0000 7fa19d68f700 1 --2- 172.21.15.16:0/3540703595 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa19814ead0 0x7fa1980b5290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:26.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:26.611+0000 7fa19d68f700 1 -- 172.21.15.16:0/3540703595 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa198071540 con 0x7fa19814e700 2023-05-20T01:18:26.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:26.611+0000 7fa19ce8e700 1 --2- 172.21.15.16:0/3540703595 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa19814ead0 0x7fa1980b5290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:18:26.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:26.612+0000 7fa19d68f700 1 --2- 172.21.15.16:0/3540703595 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa19814e700 0x7fa1980af780 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fa188011a50 tx=0x7fa188011e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:26.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:26.612+0000 7fa18dffb700 1 -- 172.21.15.16:0/3540703595 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa18800fa00 con 0x7fa19814e700 2023-05-20T01:18:26.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:26.612+0000 7fa18dffb700 1 -- 172.21.15.16:0/3540703595 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa18800fb60 con 0x7fa19814e700 2023-05-20T01:18:26.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:26.612+0000 7fa19f8f3700 1 -- 172.21.15.16:0/3540703595 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1980720c0 con 0x7fa19814e700 2023-05-20T01:18:26.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:26.612+0000 7fa18dffb700 1 -- 172.21.15.16:0/3540703595 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa188014440 con 0x7fa19814e700 2023-05-20T01:18:26.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:26.612+0000 7fa19f8f3700 1 -- 172.21.15.16:0/3540703595 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa19806e020 con 0x7fa19814e700 2023-05-20T01:18:26.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:26.613+0000 7fa19f8f3700 1 -- 172.21.15.16:0/3540703595 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1980747b0 con 0x7fa19814e700 2023-05-20T01:18:26.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:26.613+0000 7fa18dffb700 1 -- 172.21.15.16:0/3540703595 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa18801d400 con 0x7fa19814e700 2023-05-20T01:18:26.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:26.614+0000 7fa18dffb700 1 --2- 172.21.15.16:0/3540703595 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa17c06ace0 0x7fa17c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:26.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:26.614+0000 7fa18dffb700 1 -- 172.21.15.16:0/3540703595 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 7016+0+0 (secure 0 0 0) 0x7fa188089fc0 con 0x7fa19814e700 2023-05-20T01:18:26.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:26.614+0000 7fa19ce8e700 1 --2- 172.21.15.16:0/3540703595 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa17c06ace0 0x7fa17c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:26.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:26.614+0000 7fa19ce8e700 1 --2- 172.21.15.16:0/3540703595 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa17c06ace0 0x7fa17c06d190 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa1980c6af0 tx=0x7fa194009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:26.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:26.618+0000 7fa18dffb700 1 -- 172.21.15.16:0/3540703595 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fa1880bc2d0 con 0x7fa19814e700 2023-05-20T01:18:26.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:26.863+0000 7fa19f8f3700 1 -- 172.21.15.16:0/3540703595 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7fa1980b2980 con 0x7fa19814e700 2023-05-20T01:18:27.465 INFO:tasks.ceph.mon.a.smithi016.stderr:2023-05-20T01:18:27.462+0000 7f49eb8ae700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-05-20T01:18:27.466 INFO:tasks.ceph.mon.a.smithi016.stderr:2023-05-20T01:18:27.462+0000 7f49eb8ae700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2023-05-20T01:18:27.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:27.465+0000 7fa18dffb700 1 -- 172.21.15.16:0/3540703595 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v21) v1 ==== 162+0+0 (secure 0 0 0) 0x7fa188054d40 con 0x7fa19814e700 2023-05-20T01:18:27.469 INFO:tasks.ceph.mds.c.smithi016.stderr: -30> 2023-05-20T01:18:26.466+0000 7fe397322700 -1 mds.0.inotable: load_2 could not read table: -2 2023-05-20T01:18:27.470 INFO:tasks.ceph.mds.c.smithi016.stderr: -28> 2023-05-20T01:18:26.466+0000 7fe397322700 -1 log_channel(cluster) log [ERR] : error reading table object 'mds0_inotable' -2 ((2) No such file or directory) 2023-05-20T01:18:27.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:27.468+0000 7fa19f8f3700 1 -- 172.21.15.16:0/3540703595 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa17c06ace0 msgr2=0x7fa17c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:27.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:27.468+0000 7fa19f8f3700 1 --2- 172.21.15.16:0/3540703595 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa17c06ace0 0x7fa17c06d190 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa1980c6af0 tx=0x7fa194009040 comp rx=0 tx=0).stop 2023-05-20T01:18:27.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:27.468+0000 7fa19f8f3700 1 -- 172.21.15.16:0/3540703595 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa19814e700 msgr2=0x7fa1980af780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:27.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:27.468+0000 7fa19f8f3700 1 --2- 172.21.15.16:0/3540703595 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa19814e700 0x7fa1980af780 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fa188011a50 tx=0x7fa188011e10 comp rx=0 tx=0).stop 2023-05-20T01:18:27.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:27.468+0000 7fa19f8f3700 1 -- 172.21.15.16:0/3540703595 shutdown_connections 2023-05-20T01:18:27.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:27.468+0000 7fa19f8f3700 1 --2- 172.21.15.16:0/3540703595 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa17c06ace0 0x7fa17c06d190 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:27.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:27.468+0000 7fa19f8f3700 1 --2- 172.21.15.16:0/3540703595 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa1980b57d0 0x7fa1980c3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:27.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:27.468+0000 7fa19f8f3700 1 --2- 172.21.15.16:0/3540703595 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa19814e700 0x7fa1980af780 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:27.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:27.468+0000 7fa19f8f3700 1 --2- 172.21.15.16:0/3540703595 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa19814ead0 0x7fa1980b5290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:27.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:27.468+0000 7fa19f8f3700 1 -- 172.21.15.16:0/3540703595 >> 172.21.15.16:0/3540703595 conn(0x7fa19806c260 msgr2=0x7fa1981598b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:18:27.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:27.468+0000 7fa19f8f3700 1 -- 172.21.15.16:0/3540703595 shutdown_connections 2023-05-20T01:18:27.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:27.468+0000 7fa19f8f3700 1 -- 172.21.15.16:0/3540703595 wait complete. 2023-05-20T01:18:27.474 INFO:teuthology.orchestra.run.smithi016.stderr:cephfs marked joinable; MDS may join as newly active. 2023-05-20T01:18:27.488 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-20T01:18:27.498 INFO:tasks.ceph.mds.c.smithi016.stdout:starting mds.c at 2023-05-20T01:18:27.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:27.618+0000 7f77c92f6700 1 Processor -- start 2023-05-20T01:18:27.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:27.618+0000 7f77c92f6700 1 -- start start 2023-05-20T01:18:27.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:27.619+0000 7f77c92f6700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f77c406b730 0x7f77c40b53d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:27.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:27.619+0000 7f77c92f6700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77c415d020 con 0x7f77c40cf500 2023-05-20T01:18:27.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:27.619+0000 7f77c92f6700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77c40d1bf0 con 0x7f77c415cc50 2023-05-20T01:18:27.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:27.619+0000 7f77c92f6700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77c40cd720 con 0x7f77c406b730 2023-05-20T01:18:27.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:27.619+0000 7f77c37fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f77c40cf500 0x7f77c40cf920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:43998/0 (socket says 172.21.15.16:43998) 2023-05-20T01:18:27.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:27.619+0000 7f77c37fe700 1 -- 172.21.15.16:0/2247397938 learned_addr learned my addr 172.21.15.16:0/2247397938 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:18:27.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:27.619+0000 7f77c27fc700 1 --2- 172.21.15.16:0/2247397938 >> v2:172.21.15.132:3300/0 conn(0x7f77c406b730 0x7f77c40b53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:27.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:27.619+0000 7f77c1ffb700 1 -- 172.21.15.16:0/2247397938 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1386555145 0 0) 0x7f77bc001490 con 0x7f77c40cf500 2023-05-20T01:18:27.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:27.619+0000 7f77c1ffb700 1 -- 172.21.15.16:0/2247397938 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f77ac0035b0 con 0x7f77c40cf500 2023-05-20T01:18:27.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:27.619+0000 7f77c1ffb700 1 -- 172.21.15.16:0/2247397938 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1938477045 0 0) 0x7f77ac0035b0 con 0x7f77c40cf500 2023-05-20T01:18:27.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:27.619+0000 7f77c1ffb700 1 -- 172.21.15.16:0/2247397938 >> v2:172.21.15.132:3300/0 conn(0x7f77c406b730 msgr2=0x7f77c40b53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:18:27.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:27.619+0000 7f77c1ffb700 1 --2- 172.21.15.16:0/2247397938 >> v2:172.21.15.132:3300/0 conn(0x7f77c406b730 0x7f77c40b53d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:27.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:27.619+0000 7f77c1ffb700 1 -- 172.21.15.16:0/2247397938 >> v1:172.21.15.132:6789/0 conn(0x7f77c415cc50 legacy=0x7f77c40b4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:27.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:27.619+0000 7f77c1ffb700 1 -- 172.21.15.16:0/2247397938 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77c40d18c0 con 0x7f77c40cf500 2023-05-20T01:18:27.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:27.619+0000 7f77c1ffb700 1 -- 172.21.15.16:0/2247397938 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7f77bc001b80 con 0x7f77c40cf500 2023-05-20T01:18:27.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:27.620+0000 7f77c1ffb700 1 -- 172.21.15.16:0/2247397938 >> v1:172.21.15.16:6789/0 conn(0x7f77c40cf500 legacy=0x7f77c40cf920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:27.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:27.620+0000 7f77c1ffb700 1 --2- 172.21.15.16:0/2247397938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77ac003a70 0x7f77ac003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:27.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:27.620+0000 7f77c1ffb700 1 --2- 172.21.15.16:0/2247397938 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f77ac004850 0x7f77ac004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:27.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:27.620+0000 7f77c1ffb700 1 --2- 172.21.15.16:0/2247397938 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f77ac005250 0x7f77ac00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:27.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:27.620+0000 7f77c1ffb700 1 -- 172.21.15.16:0/2247397938 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77ac00c160 con 0x7f77ac003a70 2023-05-20T01:18:27.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:27.620+0000 7f77c1ffb700 1 -- 172.21.15.16:0/2247397938 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77ac00c2d0 con 0x7f77ac005250 2023-05-20T01:18:27.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:27.620+0000 7f77c1ffb700 1 -- 172.21.15.16:0/2247397938 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77ac00c490 con 0x7f77ac004850 2023-05-20T01:18:27.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:27.620+0000 7f77c2ffd700 1 --2- 172.21.15.16:0/2247397938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77ac003a70 0x7f77ac003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:27.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:27.620+0000 7f77c27fc700 1 --2- 172.21.15.16:0/2247397938 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f77ac004850 0x7f77ac004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:27.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:27.620+0000 7f77c37fe700 1 --2- 172.21.15.16:0/2247397938 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f77ac005250 0x7f77ac00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:27.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:27.620+0000 7f77c2ffd700 1 -- 172.21.15.16:0/2247397938 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f77ac004850 msgr2=0x7f77ac004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:18:27.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:27.620+0000 7f77c2ffd700 1 --2- 172.21.15.16:0/2247397938 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f77ac004850 0x7f77ac004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:27.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:27.620+0000 7f77c2ffd700 1 -- 172.21.15.16:0/2247397938 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f77ac005250 msgr2=0x7f77ac00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:18:27.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:27.620+0000 7f77c2ffd700 1 --2- 172.21.15.16:0/2247397938 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f77ac005250 0x7f77ac00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:27.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:27.620+0000 7f77c2ffd700 1 -- 172.21.15.16:0/2247397938 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77ac0035b0 con 0x7f77ac003a70 2023-05-20T01:18:27.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:27.620+0000 7f77c37fe700 1 -- 172.21.15.16:0/2247397938 reap_dead start 2023-05-20T01:18:27.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:27.620+0000 7f77c2ffd700 1 --2- 172.21.15.16:0/2247397938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77ac003a70 0x7f77ac003e80 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f77b4007fd0 tx=0x7f77ac005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a890ba2ec94c3f server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:27.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:27.620+0000 7f77c1ffb700 1 -- 172.21.15.16:0/2247397938 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77b40052d0 con 0x7f77ac003a70 2023-05-20T01:18:27.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:27.620+0000 7f77c1ffb700 1 -- 172.21.15.16:0/2247397938 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77b4005450 con 0x7f77ac003a70 2023-05-20T01:18:27.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:27.620+0000 7f77c1ffb700 1 -- 172.21.15.16:0/2247397938 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77b40038c0 con 0x7f77ac003a70 2023-05-20T01:18:27.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:27.621+0000 7f77c92f6700 1 -- 172.21.15.16:0/2247397938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77ac003a70 msgr2=0x7f77ac003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:27.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:27.621+0000 7f77c92f6700 1 --2- 172.21.15.16:0/2247397938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77ac003a70 0x7f77ac003e80 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f77b4007fd0 tx=0x7f77ac005880 comp rx=0 tx=0).stop 2023-05-20T01:18:27.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:27.621+0000 7f77c92f6700 1 -- 172.21.15.16:0/2247397938 shutdown_connections 2023-05-20T01:18:27.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:27.621+0000 7f77c92f6700 1 --2- 172.21.15.16:0/2247397938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77ac003a70 0x7f77ac003e80 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:27.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:27.621+0000 7f77c92f6700 1 -- 172.21.15.16:0/2247397938 >> 172.21.15.16:0/2247397938 conn(0x7f77c4072860 msgr2=0x7f77c406b020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:18:27.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:27.621+0000 7f77c92f6700 1 -- 172.21.15.16:0/2247397938 shutdown_connections 2023-05-20T01:18:27.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:27.621+0000 7f77c92f6700 1 -- 172.21.15.16:0/2247397938 wait complete. 2023-05-20T01:18:27.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:27.622+0000 7f77c92f6700 1 Processor -- start 2023-05-20T01:18:27.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:27.622+0000 7f77c92f6700 1 -- start start 2023-05-20T01:18:27.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:27.623+0000 7f77c92f6700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f77ac003a70 0x7f77c40b53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:27.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:27.623+0000 7f77c92f6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77c40b58f0 0x7f77c40b7e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:27.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:27.623+0000 7f77c92f6700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f77c40b8360 0x7f77c413ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:27.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:27.623+0000 7f77c92f6700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77c40cb670 con 0x7f77c40b58f0 2023-05-20T01:18:27.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:27.623+0000 7f77c92f6700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77c40cb360 con 0x7f77c40b8360 2023-05-20T01:18:27.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:27.623+0000 7f77c92f6700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77c40b87b0 con 0x7f77ac003a70 2023-05-20T01:18:27.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:27.623+0000 7f77c27fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77c40b58f0 0x7f77c40b7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:27.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:27.623+0000 7f77c27fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77c40b58f0 0x7f77c40b7e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60712/0 (socket says 172.21.15.16:60712) 2023-05-20T01:18:27.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:27.623+0000 7f77c27fc700 1 -- 172.21.15.16:0/997439157 learned_addr learned my addr 172.21.15.16:0/997439157 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:18:27.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:27.623+0000 7f77c2ffd700 1 --2- 172.21.15.16:0/997439157 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f77ac003a70 0x7f77c40b53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:27.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:27.623+0000 7f77c37fe700 1 --2- 172.21.15.16:0/997439157 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f77c40b8360 0x7f77c413ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:27.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:27.623+0000 7f77c27fc700 1 -- 172.21.15.16:0/997439157 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f77ac003a70 msgr2=0x7f77c40b53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:27.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:27.623+0000 7f77c27fc700 1 --2- 172.21.15.16:0/997439157 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f77ac003a70 0x7f77c40b53b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:27.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:27.623+0000 7f77c27fc700 1 -- 172.21.15.16:0/997439157 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f77c40b8360 msgr2=0x7f77c413ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:27.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:27.623+0000 7f77c27fc700 1 --2- 172.21.15.16:0/997439157 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f77c40b8360 0x7f77c413ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:27.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:27.623+0000 7f77c27fc700 1 -- 172.21.15.16:0/997439157 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77c40cdca0 con 0x7f77c40b58f0 2023-05-20T01:18:27.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:27.623+0000 7f77c37fe700 1 --2- 172.21.15.16:0/997439157 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f77c40b8360 0x7f77c413ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:18:27.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:27.624+0000 7f77c27fc700 1 --2- 172.21.15.16:0/997439157 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77c40b58f0 0x7f77c40b7e20 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f77b8011600 tx=0x7f77b8011910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:27.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:27.624+0000 7f77ab7fe700 1 -- 172.21.15.16:0/997439157 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77b8015870 con 0x7f77c40b58f0 2023-05-20T01:18:27.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:27.624+0000 7f77ab7fe700 1 -- 172.21.15.16:0/997439157 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77b80159d0 con 0x7f77c40b58f0 2023-05-20T01:18:27.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:27.624+0000 7f77c92f6700 1 -- 172.21.15.16:0/997439157 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77c40cd9c0 con 0x7f77c40b58f0 2023-05-20T01:18:27.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:27.624+0000 7f77ab7fe700 1 -- 172.21.15.16:0/997439157 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77b8015d40 con 0x7f77c40b58f0 2023-05-20T01:18:27.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:27.624+0000 7f77c92f6700 1 -- 172.21.15.16:0/997439157 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77c40cdf80 con 0x7f77c40b58f0 2023-05-20T01:18:27.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:27.625+0000 7f77c92f6700 1 -- 172.21.15.16:0/997439157 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77c404e8c0 con 0x7f77c40b58f0 2023-05-20T01:18:27.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:27.625+0000 7f77ab7fe700 1 -- 172.21.15.16:0/997439157 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f77b8013400 con 0x7f77c40b58f0 2023-05-20T01:18:27.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:27.625+0000 7f77ab7fe700 1 --2- 172.21.15.16:0/997439157 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f77b006ace0 0x7f77b006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:27.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:27.626+0000 7f77ab7fe700 1 -- 172.21.15.16:0/997439157 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 7016+0+0 (secure 0 0 0) 0x7f77b8089a10 con 0x7f77c40b58f0 2023-05-20T01:18:27.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:27.626+0000 7f77c2ffd700 1 --2- 172.21.15.16:0/997439157 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f77b006ace0 0x7f77b006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:27.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:27.626+0000 7f77c2ffd700 1 --2- 172.21.15.16:0/997439157 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f77b006ace0 0x7f77b006d190 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f77b4005040 tx=0x7f77b4009030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:27.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:27.630+0000 7f77ab7fe700 1 -- 172.21.15.16:0/997439157 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f77b8054700 con 0x7f77c40b58f0 2023-05-20T01:18:27.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:27.876+0000 7f77c92f6700 1 -- 172.21.15.16:0/997439157 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f77c41576b0 con 0x7f77c40b58f0 2023-05-20T01:18:27.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:27.876+0000 7f77ab7fe700 1 -- 172.21.15.16:0/997439157 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+3914 (secure 0 0 0) 0x7f77b8057d20 con 0x7f77c40b58f0 2023-05-20T01:18:27.881 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:18:27.881 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":21,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5176,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/4144439890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4144439890},{"type":"v1","addr":"172.21.15.16:6836","nonce":4144439890}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5188,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/2992755084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":2992755084},{"type":"v1","addr":"172.21.15.132:6834","nonce":2992755084}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/1979284857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1979284857},{"type":"v1","addr":"172.21.15.132:6835","nonce":1979284857}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-20T01:18:26.227827+0000","modified":"2023-05-20T01:18:27.463059+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[7,8],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-05-20T01:18:27.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:27.878+0000 7f77c92f6700 1 -- 172.21.15.16:0/997439157 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f77b006ace0 msgr2=0x7f77b006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:27.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:27.878+0000 7f77c92f6700 1 --2- 172.21.15.16:0/997439157 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f77b006ace0 0x7f77b006d190 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f77b4005040 tx=0x7f77b4009030 comp rx=0 tx=0).stop 2023-05-20T01:18:27.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:27.878+0000 7f77c92f6700 1 -- 172.21.15.16:0/997439157 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77c40b58f0 msgr2=0x7f77c40b7e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:27.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:27.878+0000 7f77c92f6700 1 --2- 172.21.15.16:0/997439157 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77c40b58f0 0x7f77c40b7e20 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f77b8011600 tx=0x7f77b8011910 comp rx=0 tx=0).stop 2023-05-20T01:18:27.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:27.878+0000 7f77c92f6700 1 -- 172.21.15.16:0/997439157 shutdown_connections 2023-05-20T01:18:27.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:27.878+0000 7f77c92f6700 1 --2- 172.21.15.16:0/997439157 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f77c40b8360 0x7f77c413ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:27.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:27.878+0000 7f77c92f6700 1 --2- 172.21.15.16:0/997439157 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f77b006ace0 0x7f77b006d190 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:27.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:27.878+0000 7f77c92f6700 1 --2- 172.21.15.16:0/997439157 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f77ac003a70 0x7f77c40b53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:27.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:27.878+0000 7f77c92f6700 1 --2- 172.21.15.16:0/997439157 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77c40b58f0 0x7f77c40b7e20 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:27.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:27.878+0000 7f77c92f6700 1 -- 172.21.15.16:0/997439157 >> 172.21.15.16:0/997439157 conn(0x7f77c4072860 msgr2=0x7f77c415f980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:18:27.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:27.878+0000 7f77c92f6700 1 -- 172.21.15.16:0/997439157 shutdown_connections 2023-05-20T01:18:27.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:27.878+0000 7f77c92f6700 1 -- 172.21.15.16:0/997439157 wait complete. 2023-05-20T01:18:27.884 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 21 2023-05-20T01:18:27.894 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-05-20T01:18:27.895 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-20T01:18:28.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.055+0000 7f496f842700 1 Processor -- start 2023-05-20T01:18:28.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.055+0000 7f496f842700 1 -- start start 2023-05-20T01:18:28.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.056+0000 7f496f842700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f496815c940 0x7f4968071470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:28.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.056+0000 7f496f842700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f49680b17b0 0x7f4968161090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:28.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.056+0000 7f496f842700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4968160640 con 0x7f496806cb90 2023-05-20T01:18:28.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.056+0000 7f496f842700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49681615d0 con 0x7f496815c940 2023-05-20T01:18:28.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.056+0000 7f496f842700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4968071ac0 con 0x7f49680b17b0 2023-05-20T01:18:28.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.056+0000 7f496d5de700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f496815c940 0x7f4968071470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:28.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.056+0000 7f496dddf700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f496806cb90 0x7f49680c8aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:44016/0 (socket says 172.21.15.16:44016) 2023-05-20T01:18:28.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.056+0000 7f496dddf700 1 -- 172.21.15.16:0/2339779199 learned_addr learned my addr 172.21.15.16:0/2339779199 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:18:28.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.056+0000 7f496cddd700 1 --2- 172.21.15.16:0/2339779199 >> v2:172.21.15.132:3300/0 conn(0x7f49680b17b0 0x7f4968161090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:28.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.056+0000 7f496d5de700 1 -- 172.21.15.16:0/2339779199 >> v2:172.21.15.132:3300/0 conn(0x7f49680b17b0 msgr2=0x7f4968161090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:18:28.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.056+0000 7f496d5de700 1 --2- 172.21.15.16:0/2339779199 >> v2:172.21.15.132:3300/0 conn(0x7f49680b17b0 0x7f4968161090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:28.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.056+0000 7f495ffff700 1 -- 172.21.15.16:0/2339779199 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2825375224 0 0) 0x7f4968160640 con 0x7f496806cb90 2023-05-20T01:18:28.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.056+0000 7f496d5de700 1 -- 172.21.15.16:0/2339779199 >> v1:172.21.15.16:6789/0 conn(0x7f496806cb90 legacy=0x7f49680c8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:28.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.056+0000 7f496d5de700 1 -- 172.21.15.16:0/2339779199 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4968071da0 con 0x7f496815c940 2023-05-20T01:18:28.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.056+0000 7f496d5de700 1 --2- 172.21.15.16:0/2339779199 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f496815c940 0x7f4968071470 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f4968070780 tx=0x7f4964009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3cb402a3a99b586 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:28.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.057+0000 7f495ffff700 1 -- 172.21.15.16:0/2339779199 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4964011830 con 0x7f496815c940 2023-05-20T01:18:28.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.057+0000 7f495ffff700 1 -- 172.21.15.16:0/2339779199 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4964011990 con 0x7f496815c940 2023-05-20T01:18:28.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.057+0000 7f495ffff700 1 -- 172.21.15.16:0/2339779199 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4964011d20 con 0x7f496815c940 2023-05-20T01:18:28.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.057+0000 7f496f842700 1 -- 172.21.15.16:0/2339779199 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f496815c940 msgr2=0x7f4968071470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:28.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.057+0000 7f496f842700 1 --2- 172.21.15.16:0/2339779199 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f496815c940 0x7f4968071470 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f4968070780 tx=0x7f4964009b20 comp rx=0 tx=0).stop 2023-05-20T01:18:28.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.057+0000 7f496f842700 1 -- 172.21.15.16:0/2339779199 shutdown_connections 2023-05-20T01:18:28.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.057+0000 7f496f842700 1 --2- 172.21.15.16:0/2339779199 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f496815c940 0x7f4968071470 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:28.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.057+0000 7f496f842700 1 --2- 172.21.15.16:0/2339779199 >> v2:172.21.15.132:3300/0 conn(0x7f49680b17b0 0x7f4968161090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:28.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.057+0000 7f496f842700 1 -- 172.21.15.16:0/2339779199 >> 172.21.15.16:0/2339779199 conn(0x7f49680661b0 msgr2=0x7f4968066580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:18:28.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.057+0000 7f496f842700 1 -- 172.21.15.16:0/2339779199 shutdown_connections 2023-05-20T01:18:28.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.057+0000 7f496f842700 1 -- 172.21.15.16:0/2339779199 wait complete. 2023-05-20T01:18:28.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.059+0000 7f496f842700 1 Processor -- start 2023-05-20T01:18:28.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.059+0000 7f496f842700 1 -- start start 2023-05-20T01:18:28.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.059+0000 7f496f842700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f496806cb90 0x7f496815ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:28.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.059+0000 7f496f842700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f49680b17b0 0x7f4968150ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:28.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.059+0000 7f496f842700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f496815c940 0x7f4968151430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:28.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.059+0000 7f496f842700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4968144980 con 0x7f496815c940 2023-05-20T01:18:28.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.059+0000 7f496f842700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4968143d00 con 0x7f496806cb90 2023-05-20T01:18:28.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.059+0000 7f496f842700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4968143fe0 con 0x7f49680b17b0 2023-05-20T01:18:28.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.059+0000 7f496dddf700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f496815c940 0x7f4968151430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:28.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.059+0000 7f496dddf700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f496815c940 0x7f4968151430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60726/0 (socket says 172.21.15.16:60726) 2023-05-20T01:18:28.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.059+0000 7f496dddf700 1 -- 172.21.15.16:0/3603511405 learned_addr learned my addr 172.21.15.16:0/3603511405 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:18:28.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.060+0000 7f496d5de700 1 --2- 172.21.15.16:0/3603511405 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f496806cb90 0x7f496815ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:28.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.060+0000 7f496cddd700 1 --2- 172.21.15.16:0/3603511405 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f49680b17b0 0x7f4968150ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:28.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.060+0000 7f496dddf700 1 -- 172.21.15.16:0/3603511405 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f49680b17b0 msgr2=0x7f4968150ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:28.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.060+0000 7f496dddf700 1 --2- 172.21.15.16:0/3603511405 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f49680b17b0 0x7f4968150ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:28.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.060+0000 7f496dddf700 1 -- 172.21.15.16:0/3603511405 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f496806cb90 msgr2=0x7f496815ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:28.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.060+0000 7f496dddf700 1 --2- 172.21.15.16:0/3603511405 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f496806cb90 0x7f496815ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:28.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.060+0000 7f496dddf700 1 -- 172.21.15.16:0/3603511405 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49640094a0 con 0x7f496815c940 2023-05-20T01:18:28.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.060+0000 7f496d5de700 1 --2- 172.21.15.16:0/3603511405 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f496806cb90 0x7f496815ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:18:28.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.060+0000 7f496dddf700 1 --2- 172.21.15.16:0/3603511405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f496815c940 0x7f4968151430 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f496000cba0 tx=0x7f496000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:28.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.060+0000 7f495dffb700 1 -- 172.21.15.16:0/3603511405 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f496000ab80 con 0x7f496815c940 2023-05-20T01:18:28.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.060+0000 7f495dffb700 1 -- 172.21.15.16:0/3603511405 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f496000ad00 con 0x7f496815c940 2023-05-20T01:18:28.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.060+0000 7f496f842700 1 -- 172.21.15.16:0/3603511405 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4968160b70 con 0x7f496815c940 2023-05-20T01:18:28.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.060+0000 7f495dffb700 1 -- 172.21.15.16:0/3603511405 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4960007310 con 0x7f496815c940 2023-05-20T01:18:28.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.060+0000 7f496f842700 1 -- 172.21.15.16:0/3603511405 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4968160890 con 0x7f496815c940 2023-05-20T01:18:28.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.061+0000 7f496f842700 1 -- 172.21.15.16:0/3603511405 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49680b1c20 con 0x7f496815c940 2023-05-20T01:18:28.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.062+0000 7f495dffb700 1 -- 172.21.15.16:0/3603511405 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4960007470 con 0x7f496815c940 2023-05-20T01:18:28.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.062+0000 7f495dffb700 1 --2- 172.21.15.16:0/3603511405 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f494c06ace0 0x7f494c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:28.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.062+0000 7f495dffb700 1 -- 172.21.15.16:0/3603511405 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 7016+0+0 (secure 0 0 0) 0x7f49600893a0 con 0x7f496815c940 2023-05-20T01:18:28.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.062+0000 7f496d5de700 1 --2- 172.21.15.16:0/3603511405 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f494c06ace0 0x7f494c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:28.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.062+0000 7f496d5de700 1 --2- 172.21.15.16:0/3603511405 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f494c06ace0 0x7f494c06d190 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f49640090b0 tx=0x7f496401c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:28.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.066+0000 7f495dffb700 1 -- 172.21.15.16:0/3603511405 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f4960053ff0 con 0x7f496815c940 2023-05-20T01:18:28.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.307+0000 7f496f842700 1 -- 172.21.15.16:0/3603511405 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4968144ac0 con 0x7f496815c940 2023-05-20T01:18:28.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.307+0000 7f495dffb700 1 -- 172.21.15.16:0/3603511405 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+3914 (secure 0 0 0) 0x7f4968144ac0 con 0x7f496815c940 2023-05-20T01:18:28.311 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:18:28.312 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":21,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5176,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/4144439890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4144439890},{"type":"v1","addr":"172.21.15.16:6836","nonce":4144439890}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5188,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/2992755084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":2992755084},{"type":"v1","addr":"172.21.15.132:6834","nonce":2992755084}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/1979284857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1979284857},{"type":"v1","addr":"172.21.15.132:6835","nonce":1979284857}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-20T01:18:26.227827+0000","modified":"2023-05-20T01:18:27.463059+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[7,8],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-05-20T01:18:28.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.309+0000 7f496f842700 1 -- 172.21.15.16:0/3603511405 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f494c06ace0 msgr2=0x7f494c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:28.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.309+0000 7f496f842700 1 --2- 172.21.15.16:0/3603511405 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f494c06ace0 0x7f494c06d190 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f49640090b0 tx=0x7f496401c040 comp rx=0 tx=0).stop 2023-05-20T01:18:28.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.309+0000 7f496f842700 1 -- 172.21.15.16:0/3603511405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f496815c940 msgr2=0x7f4968151430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:28.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.309+0000 7f496f842700 1 --2- 172.21.15.16:0/3603511405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f496815c940 0x7f4968151430 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f496000cba0 tx=0x7f496000cf60 comp rx=0 tx=0).stop 2023-05-20T01:18:28.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.309+0000 7f496f842700 1 -- 172.21.15.16:0/3603511405 shutdown_connections 2023-05-20T01:18:28.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.309+0000 7f496f842700 1 --2- 172.21.15.16:0/3603511405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f496815c940 0x7f4968151430 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:28.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.309+0000 7f496f842700 1 --2- 172.21.15.16:0/3603511405 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f496806cb90 0x7f496815ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:28.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.309+0000 7f496f842700 1 --2- 172.21.15.16:0/3603511405 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f494c06ace0 0x7f494c06d190 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:28.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.309+0000 7f496f842700 1 --2- 172.21.15.16:0/3603511405 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f49680b17b0 0x7f4968150ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:28.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.309+0000 7f496f842700 1 -- 172.21.15.16:0/3603511405 >> 172.21.15.16:0/3603511405 conn(0x7f49680661b0 msgr2=0x7f4968070810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:18:28.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.309+0000 7f496f842700 1 -- 172.21.15.16:0/3603511405 shutdown_connections 2023-05-20T01:18:28.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.309+0000 7f496f842700 1 -- 172.21.15.16:0/3603511405 wait complete. 2023-05-20T01:18:28.315 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 21 2023-05-20T01:18:28.325 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-20T01:18:28.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.448+0000 7f7f0405e700 1 Processor -- start 2023-05-20T01:18:28.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.448+0000 7f7f0405e700 1 -- start start 2023-05-20T01:18:28.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.448+0000 7f7f0405e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7efc0cfc50 0x7f7efc0d2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:28.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.449+0000 7f7f0405e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f7efc0cd970 0x7f7efc0cddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:28.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.449+0000 7f7f0405e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7efc0b3b50 con 0x7f7efc0b4750 2023-05-20T01:18:28.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.449+0000 7f7f0405e700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7efc0cfad0 con 0x7f7efc0cd970 2023-05-20T01:18:28.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.449+0000 7f7f0405e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7efc0d3df0 con 0x7f7efc0cfc50 2023-05-20T01:18:28.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.449+0000 7f7f025fb700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f7efc0cd970 0x7f7efc0cddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:28.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.449+0000 7f7f01dfa700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7efc0cfc50 0x7f7efc0d2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:28.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.449+0000 7f7f025fb700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f7efc0cd970 0x7f7efc0cddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.16:57754/0 (socket says 172.21.15.16:57754) 2023-05-20T01:18:28.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.449+0000 7f7f025fb700 1 -- 172.21.15.16:0/912579297 learned_addr learned my addr 172.21.15.16:0/912579297 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:18:28.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.449+0000 7f7f015f9700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f7efc0b4750 0x7f7efc0d0020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.16:53740/0 (socket says 172.21.15.16:53740) 2023-05-20T01:18:28.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.449+0000 7f7f025fb700 1 -- 172.21.15.16:0/912579297 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7efc0cfc50 msgr2=0x7f7efc0d2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:18:28.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.449+0000 7f7f025fb700 1 --2- 172.21.15.16:0/912579297 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7efc0cfc50 0x7f7efc0d2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:28.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.449+0000 7f7f025fb700 1 -- 172.21.15.16:0/912579297 >> v1:172.21.15.132:6789/0 conn(0x7f7efc0b4750 legacy=0x7f7efc0d0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:28.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.449+0000 7f7f025fb700 1 -- 172.21.15.16:0/912579297 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7efc0d0b90 con 0x7f7efc0cd970 2023-05-20T01:18:28.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.450+0000 7f7f025fb700 1 --2- 172.21.15.16:0/912579297 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7efc0cd970 0x7f7efc0cddf0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7ef4009e50 tx=0x7f7ef400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=28ee2421315401d2 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:28.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.450+0000 7f7f00df8700 1 -- 172.21.15.16:0/912579297 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ef400e070 con 0x7f7efc0cd970 2023-05-20T01:18:28.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.450+0000 7f7f00df8700 1 -- 172.21.15.16:0/912579297 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ef400b910 con 0x7f7efc0cd970 2023-05-20T01:18:28.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.450+0000 7f7f00df8700 1 -- 172.21.15.16:0/912579297 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ef400bcc0 con 0x7f7efc0cd970 2023-05-20T01:18:28.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.450+0000 7f7f0405e700 1 -- 172.21.15.16:0/912579297 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7efc0cd970 msgr2=0x7f7efc0cddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:28.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.450+0000 7f7f0405e700 1 --2- 172.21.15.16:0/912579297 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7efc0cd970 0x7f7efc0cddf0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7ef4009e50 tx=0x7f7ef400b330 comp rx=0 tx=0).stop 2023-05-20T01:18:28.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.450+0000 7f7f0405e700 1 -- 172.21.15.16:0/912579297 shutdown_connections 2023-05-20T01:18:28.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.450+0000 7f7f0405e700 1 --2- 172.21.15.16:0/912579297 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7efc0cfc50 0x7f7efc0d2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:28.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.450+0000 7f7f0405e700 1 --2- 172.21.15.16:0/912579297 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7efc0cd970 0x7f7efc0cddf0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:28.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.450+0000 7f7f0405e700 1 -- 172.21.15.16:0/912579297 >> 172.21.15.16:0/912579297 conn(0x7f7efc09c590 msgr2=0x7f7efc0d3240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:18:28.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.451+0000 7f7f0405e700 1 -- 172.21.15.16:0/912579297 shutdown_connections 2023-05-20T01:18:28.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.451+0000 7f7f0405e700 1 -- 172.21.15.16:0/912579297 wait complete. 2023-05-20T01:18:28.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.452+0000 7f7f0405e700 1 Processor -- start 2023-05-20T01:18:28.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.452+0000 7f7f0405e700 1 -- start start 2023-05-20T01:18:28.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.452+0000 7f7f0405e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7efc0b4750 0x7f7efc0ccaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:28.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.452+0000 7f7f0405e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7efc0cd970 0x7f7efc0b8800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:28.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.452+0000 7f7f0405e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7efc0cfc50 0x7f7efc1453a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:28.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.452+0000 7f7f0405e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7efc0cbd70 con 0x7f7efc0cfc50 2023-05-20T01:18:28.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.452+0000 7f7f0405e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7efc0cc610 con 0x7f7efc0cd970 2023-05-20T01:18:28.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.452+0000 7f7f0405e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7efc0ccfe0 con 0x7f7efc0b4750 2023-05-20T01:18:28.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.453+0000 7f7f025fb700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7efc0cfc50 0x7f7efc1453a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:28.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.453+0000 7f7f025fb700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7efc0cfc50 0x7f7efc1453a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60738/0 (socket says 172.21.15.16:60738) 2023-05-20T01:18:28.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.453+0000 7f7f025fb700 1 -- 172.21.15.16:0/4257593782 learned_addr learned my addr 172.21.15.16:0/4257593782 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:18:28.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.453+0000 7f7f01dfa700 1 --2- 172.21.15.16:0/4257593782 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7efc0b4750 0x7f7efc0ccaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:28.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.453+0000 7f7f015f9700 1 --2- 172.21.15.16:0/4257593782 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7efc0cd970 0x7f7efc0b8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:28.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.453+0000 7f7f025fb700 1 -- 172.21.15.16:0/4257593782 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7efc0b4750 msgr2=0x7f7efc0ccaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:28.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.453+0000 7f7f025fb700 1 --2- 172.21.15.16:0/4257593782 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7efc0b4750 0x7f7efc0ccaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:28.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.453+0000 7f7f025fb700 1 -- 172.21.15.16:0/4257593782 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7efc0cd970 msgr2=0x7f7efc0b8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:28.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.453+0000 7f7f025fb700 1 --2- 172.21.15.16:0/4257593782 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7efc0cd970 0x7f7efc0b8800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:28.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.453+0000 7f7f025fb700 1 -- 172.21.15.16:0/4257593782 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ef4009b30 con 0x7f7efc0cfc50 2023-05-20T01:18:28.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.453+0000 7f7f025fb700 1 --2- 172.21.15.16:0/4257593782 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7efc0cfc50 0x7f7efc1453a0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f7ef400b680 tx=0x7f7ef4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:28.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.453+0000 7f7eee7fc700 1 -- 172.21.15.16:0/4257593782 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ef40099b0 con 0x7f7efc0cfc50 2023-05-20T01:18:28.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.453+0000 7f7eee7fc700 1 -- 172.21.15.16:0/4257593782 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ef40092e0 con 0x7f7efc0cfc50 2023-05-20T01:18:28.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.453+0000 7f7f0405e700 1 -- 172.21.15.16:0/4257593782 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7efc0b4380 con 0x7f7efc0cfc50 2023-05-20T01:18:28.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.453+0000 7f7eee7fc700 1 -- 172.21.15.16:0/4257593782 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ef401b630 con 0x7f7efc0cfc50 2023-05-20T01:18:28.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.453+0000 7f7f0405e700 1 -- 172.21.15.16:0/4257593782 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7efc15d0b0 con 0x7f7efc0cfc50 2023-05-20T01:18:28.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.454+0000 7f7f0405e700 1 -- 172.21.15.16:0/4257593782 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7efc0d4000 con 0x7f7efc0cfc50 2023-05-20T01:18:28.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.455+0000 7f7eee7fc700 1 -- 172.21.15.16:0/4257593782 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7ef401b790 con 0x7f7efc0cfc50 2023-05-20T01:18:28.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.455+0000 7f7eee7fc700 1 --2- 172.21.15.16:0/4257593782 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f7ee006ace0 0x7f7ee006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:28.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.455+0000 7f7eee7fc700 1 -- 172.21.15.16:0/4257593782 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 7016+0+0 (secure 0 0 0) 0x7f7ef408b1d0 con 0x7f7efc0cfc50 2023-05-20T01:18:28.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.455+0000 7f7f01dfa700 1 --2- 172.21.15.16:0/4257593782 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f7ee006ace0 0x7f7ee006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:28.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.456+0000 7f7f01dfa700 1 --2- 172.21.15.16:0/4257593782 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f7ee006ace0 0x7f7ee006d190 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f7ef0009ad0 tx=0x7f7ef0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:28.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.460+0000 7f7eee7fc700 1 -- 172.21.15.16:0/4257593782 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f7ef4055ed0 con 0x7f7efc0cfc50 2023-05-20T01:18:28.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.704+0000 7f7f0405e700 1 -- 172.21.15.16:0/4257593782 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7efc0cc750 con 0x7f7efc0cfc50 2023-05-20T01:18:28.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.704+0000 7f7eee7fc700 1 -- 172.21.15.16:0/4257593782 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+4671 (secure 0 0 0) 0x7f7ef40594f0 con 0x7f7efc0cfc50 2023-05-20T01:18:28.708 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:18:28.708 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":22,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5176,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/4144439890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4144439890},{"type":"v1","addr":"172.21.15.16:6836","nonce":4144439890}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5188,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/2992755084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":2992755084},{"type":"v1","addr":"172.21.15.132:6834","nonce":2992755084}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/1979284857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1979284857},{"type":"v1","addr":"172.21.15.132:6835","nonce":1979284857}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5730,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/1867776973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":1867776973},{"type":"v1","addr":"172.21.15.16:6837","nonce":1867776973}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-20T01:18:26.227827+0000","modified":"2023-05-20T01:18:27.463059+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[7,8],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-05-20T01:18:28.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.706+0000 7f7f0405e700 1 -- 172.21.15.16:0/4257593782 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f7ee006ace0 msgr2=0x7f7ee006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:28.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.706+0000 7f7f0405e700 1 --2- 172.21.15.16:0/4257593782 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f7ee006ace0 0x7f7ee006d190 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f7ef0009ad0 tx=0x7f7ef0008040 comp rx=0 tx=0).stop 2023-05-20T01:18:28.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.706+0000 7f7f0405e700 1 -- 172.21.15.16:0/4257593782 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7efc0cfc50 msgr2=0x7f7efc1453a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:28.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.706+0000 7f7f0405e700 1 --2- 172.21.15.16:0/4257593782 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7efc0cfc50 0x7f7efc1453a0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f7ef400b680 tx=0x7f7ef4004ad0 comp rx=0 tx=0).stop 2023-05-20T01:18:28.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.706+0000 7f7f0405e700 1 -- 172.21.15.16:0/4257593782 shutdown_connections 2023-05-20T01:18:28.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.706+0000 7f7f0405e700 1 --2- 172.21.15.16:0/4257593782 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7efc0cfc50 0x7f7efc1453a0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:28.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.706+0000 7f7f0405e700 1 --2- 172.21.15.16:0/4257593782 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f7ee006ace0 0x7f7ee006d190 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:28.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.706+0000 7f7f0405e700 1 --2- 172.21.15.16:0/4257593782 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7efc0b4750 0x7f7efc0ccaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:28.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.706+0000 7f7f0405e700 1 --2- 172.21.15.16:0/4257593782 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7efc0cd970 0x7f7efc0b8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:28.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.706+0000 7f7f0405e700 1 -- 172.21.15.16:0/4257593782 >> 172.21.15.16:0/4257593782 conn(0x7f7efc09c590 msgr2=0x7f7efc0cdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:18:28.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.706+0000 7f7f0405e700 1 -- 172.21.15.16:0/4257593782 shutdown_connections 2023-05-20T01:18:28.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.706+0000 7f7f0405e700 1 -- 172.21.15.16:0/4257593782 wait complete. 2023-05-20T01:18:28.712 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 22 2023-05-20T01:18:28.722 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-20T01:18:28.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.843+0000 7fb347a6e700 1 Processor -- start 2023-05-20T01:18:28.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.843+0000 7fb347a6e700 1 -- start start 2023-05-20T01:18:28.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.844+0000 7fb347a6e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb34006cb90 0x7fb34015ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:28.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.844+0000 7fb347a6e700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb3400b54f0 0x7fb3400b29e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:28.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.844+0000 7fb347a6e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3400b2f20 con 0x7fb34006cf60 2023-05-20T01:18:28.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.844+0000 7fb347a6e700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3400cc990 con 0x7fb3400b54f0 2023-05-20T01:18:28.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.844+0000 7fb347a6e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3400b31a0 con 0x7fb34006cb90 2023-05-20T01:18:28.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.844+0000 7fb345009700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb3400b54f0 0x7fb3400b29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:28.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.844+0000 7fb345009700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb3400b54f0 0x7fb3400b29e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60744/0 (socket says 172.21.15.16:60744) 2023-05-20T01:18:28.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.844+0000 7fb345009700 1 -- 172.21.15.16:0/1204919756 learned_addr learned my addr 172.21.15.16:0/1204919756 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:18:28.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.844+0000 7fb34580a700 1 --2- 172.21.15.16:0/1204919756 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb34006cb90 0x7fb34015ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:28.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.844+0000 7fb345009700 1 -- 172.21.15.16:0/1204919756 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb34006cb90 msgr2=0x7fb34015ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:18:28.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.844+0000 7fb345009700 1 --2- 172.21.15.16:0/1204919756 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb34006cb90 0x7fb34015ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:28.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.844+0000 7fb345009700 1 -- 172.21.15.16:0/1204919756 >> v1:172.21.15.132:6789/0 conn(0x7fb34006cf60 legacy=0x7fb340070910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:28.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.844+0000 7fb345009700 1 -- 172.21.15.16:0/1204919756 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3400b3480 con 0x7fb3400b54f0 2023-05-20T01:18:28.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.844+0000 7fb34580a700 1 --2- 172.21.15.16:0/1204919756 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb34006cb90 0x7fb34015ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:18:28.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.844+0000 7fb345009700 1 --2- 172.21.15.16:0/1204919756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3400b54f0 0x7fb3400b29e0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fb3300098f0 tx=0x7fb330009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70928c4308249015 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:28.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.845+0000 7fb344808700 1 -- 172.21.15.16:0/1204919756 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb33000b8a0 con 0x7fb3400b54f0 2023-05-20T01:18:28.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.845+0000 7fb344808700 1 -- 172.21.15.16:0/1204919756 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb33000ba00 con 0x7fb3400b54f0 2023-05-20T01:18:28.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.845+0000 7fb344808700 1 -- 172.21.15.16:0/1204919756 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb33000bd90 con 0x7fb3400b54f0 2023-05-20T01:18:28.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.845+0000 7fb347a6e700 1 -- 172.21.15.16:0/1204919756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3400b54f0 msgr2=0x7fb3400b29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:28.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.845+0000 7fb347a6e700 1 --2- 172.21.15.16:0/1204919756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3400b54f0 0x7fb3400b29e0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fb3300098f0 tx=0x7fb330009c00 comp rx=0 tx=0).stop 2023-05-20T01:18:28.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.845+0000 7fb347a6e700 1 -- 172.21.15.16:0/1204919756 shutdown_connections 2023-05-20T01:18:28.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.845+0000 7fb347a6e700 1 --2- 172.21.15.16:0/1204919756 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb34006cb90 0x7fb34015ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:28.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.845+0000 7fb347a6e700 1 --2- 172.21.15.16:0/1204919756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3400b54f0 0x7fb3400b29e0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:28.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.845+0000 7fb347a6e700 1 -- 172.21.15.16:0/1204919756 >> 172.21.15.16:0/1204919756 conn(0x7fb3400661b0 msgr2=0x7fb340066580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:18:28.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.845+0000 7fb347a6e700 1 -- 172.21.15.16:0/1204919756 shutdown_connections 2023-05-20T01:18:28.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.845+0000 7fb347a6e700 1 -- 172.21.15.16:0/1204919756 wait complete. 2023-05-20T01:18:28.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.847+0000 7fb347a6e700 1 Processor -- start 2023-05-20T01:18:28.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.847+0000 7fb347a6e700 1 -- start start 2023-05-20T01:18:28.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.848+0000 7fb347a6e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb34006cb90 0x7fb340150470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:28.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.848+0000 7fb347a6e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb34006cf60 0x7fb3400c8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:28.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.848+0000 7fb347a6e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3400b54f0 0x7fb3400c8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:28.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.848+0000 7fb347a6e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3401558f0 con 0x7fb3400b54f0 2023-05-20T01:18:28.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.848+0000 7fb347a6e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb340155bd0 con 0x7fb34006cf60 2023-05-20T01:18:28.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.848+0000 7fb347a6e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb340155eb0 con 0x7fb34006cb90 2023-05-20T01:18:28.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.848+0000 7fb34600b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3400b54f0 0x7fb3400c8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:28.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.848+0000 7fb34600b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3400b54f0 0x7fb3400c8fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60752/0 (socket says 172.21.15.16:60752) 2023-05-20T01:18:28.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.848+0000 7fb34600b700 1 -- 172.21.15.16:0/4022379272 learned_addr learned my addr 172.21.15.16:0/4022379272 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:18:28.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.848+0000 7fb34580a700 1 --2- 172.21.15.16:0/4022379272 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb34006cb90 0x7fb340150470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:28.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.848+0000 7fb345009700 1 --2- 172.21.15.16:0/4022379272 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb34006cf60 0x7fb3400c8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:28.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.848+0000 7fb34600b700 1 -- 172.21.15.16:0/4022379272 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb34006cb90 msgr2=0x7fb340150470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:28.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.848+0000 7fb34600b700 1 --2- 172.21.15.16:0/4022379272 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb34006cb90 0x7fb340150470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:28.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.848+0000 7fb34600b700 1 -- 172.21.15.16:0/4022379272 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb34006cf60 msgr2=0x7fb3400c8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:28.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.848+0000 7fb34600b700 1 --2- 172.21.15.16:0/4022379272 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb34006cf60 0x7fb3400c8aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:28.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.848+0000 7fb34600b700 1 -- 172.21.15.16:0/4022379272 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb330009550 con 0x7fb3400b54f0 2023-05-20T01:18:28.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.848+0000 7fb345009700 1 --2- 172.21.15.16:0/4022379272 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb34006cf60 0x7fb3400c8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:18:28.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.849+0000 7fb34600b700 1 --2- 172.21.15.16:0/4022379272 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3400b54f0 0x7fb3400c8fe0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fb33800cb50 tx=0x7fb33800cf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:28.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.849+0000 7fb3367fc700 1 -- 172.21.15.16:0/4022379272 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb338003c00 con 0x7fb3400b54f0 2023-05-20T01:18:28.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.849+0000 7fb3367fc700 1 -- 172.21.15.16:0/4022379272 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb338003d60 con 0x7fb3400b54f0 2023-05-20T01:18:28.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.849+0000 7fb3367fc700 1 -- 172.21.15.16:0/4022379272 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb33800ab70 con 0x7fb3400b54f0 2023-05-20T01:18:28.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.849+0000 7fb347a6e700 1 -- 172.21.15.16:0/4022379272 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb34015b240 con 0x7fb3400b54f0 2023-05-20T01:18:28.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.849+0000 7fb347a6e700 1 -- 172.21.15.16:0/4022379272 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb34015af60 con 0x7fb3400b54f0 2023-05-20T01:18:28.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.850+0000 7fb347a6e700 1 -- 172.21.15.16:0/4022379272 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb34015b520 con 0x7fb3400b54f0 2023-05-20T01:18:28.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.850+0000 7fb3367fc700 1 -- 172.21.15.16:0/4022379272 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb33800acd0 con 0x7fb3400b54f0 2023-05-20T01:18:28.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.851+0000 7fb3367fc700 1 --2- 172.21.15.16:0/4022379272 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb32406ace0 0x7fb32406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:28.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.851+0000 7fb3367fc700 1 -- 172.21.15.16:0/4022379272 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 7016+0+0 (secure 0 0 0) 0x7fb33808a2c0 con 0x7fb3400b54f0 2023-05-20T01:18:28.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.851+0000 7fb34580a700 1 --2- 172.21.15.16:0/4022379272 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb32406ace0 0x7fb32406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:28.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.851+0000 7fb34580a700 1 --2- 172.21.15.16:0/4022379272 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb32406ace0 0x7fb32406d190 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb33c00d8b0 tx=0x7fb33c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:28.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:28.855+0000 7fb3367fc700 1 -- 172.21.15.16:0/4022379272 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fb338055090 con 0x7fb3400b54f0 2023-05-20T01:18:29.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.099+0000 7fb347a6e700 1 -- 172.21.15.16:0/4022379272 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb3400747b0 con 0x7fb3400b54f0 2023-05-20T01:18:29.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.100+0000 7fb3367fc700 1 -- 172.21.15.16:0/4022379272 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+4671 (secure 0 0 0) 0x7fb3380586b0 con 0x7fb3400b54f0 2023-05-20T01:18:29.104 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:18:29.104 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":22,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5176,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/4144439890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4144439890},{"type":"v1","addr":"172.21.15.16:6836","nonce":4144439890}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5188,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/2992755084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":2992755084},{"type":"v1","addr":"172.21.15.132:6834","nonce":2992755084}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/1979284857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1979284857},{"type":"v1","addr":"172.21.15.132:6835","nonce":1979284857}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5730,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/1867776973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":1867776973},{"type":"v1","addr":"172.21.15.16:6837","nonce":1867776973}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-20T01:18:26.227827+0000","modified":"2023-05-20T01:18:27.463059+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[7,8],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-05-20T01:18:29.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.102+0000 7fb347a6e700 1 -- 172.21.15.16:0/4022379272 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb32406ace0 msgr2=0x7fb32406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:29.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.102+0000 7fb347a6e700 1 --2- 172.21.15.16:0/4022379272 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb32406ace0 0x7fb32406d190 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb33c00d8b0 tx=0x7fb33c008040 comp rx=0 tx=0).stop 2023-05-20T01:18:29.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.102+0000 7fb347a6e700 1 -- 172.21.15.16:0/4022379272 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3400b54f0 msgr2=0x7fb3400c8fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:29.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.102+0000 7fb347a6e700 1 --2- 172.21.15.16:0/4022379272 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3400b54f0 0x7fb3400c8fe0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fb33800cb50 tx=0x7fb33800cf10 comp rx=0 tx=0).stop 2023-05-20T01:18:29.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.102+0000 7fb347a6e700 1 -- 172.21.15.16:0/4022379272 shutdown_connections 2023-05-20T01:18:29.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.102+0000 7fb347a6e700 1 --2- 172.21.15.16:0/4022379272 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3400b54f0 0x7fb3400c8fe0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:29.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.102+0000 7fb347a6e700 1 --2- 172.21.15.16:0/4022379272 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb32406ace0 0x7fb32406d190 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:29.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.102+0000 7fb347a6e700 1 --2- 172.21.15.16:0/4022379272 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb34006cb90 0x7fb340150470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:29.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.102+0000 7fb347a6e700 1 --2- 172.21.15.16:0/4022379272 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb34006cf60 0x7fb3400c8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:29.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.102+0000 7fb347a6e700 1 -- 172.21.15.16:0/4022379272 >> 172.21.15.16:0/4022379272 conn(0x7fb3400661b0 msgr2=0x7fb34015da50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:18:29.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.102+0000 7fb347a6e700 1 -- 172.21.15.16:0/4022379272 shutdown_connections 2023-05-20T01:18:29.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.102+0000 7fb347a6e700 1 -- 172.21.15.16:0/4022379272 wait complete. 2023-05-20T01:18:29.107 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 22 2023-05-20T01:18:29.118 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-20T01:18:29.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.239+0000 7f301abb0700 1 Processor -- start 2023-05-20T01:18:29.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.239+0000 7f301abb0700 1 -- start start 2023-05-20T01:18:29.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.239+0000 7f301abb0700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f301406d2a0 0x7f30140cb000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:29.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.239+0000 7f301abb0700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30140cd050 con 0x7f30140ccbd0 2023-05-20T01:18:29.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.239+0000 7f301abb0700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30140cb750 con 0x7f3014153eb0 2023-05-20T01:18:29.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.239+0000 7f301abb0700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3014159d30 con 0x7f301406d2a0 2023-05-20T01:18:29.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.240+0000 7f301914d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f301406d2a0 0x7f30140cb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:29.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.240+0000 7f3013fff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f30140ccbd0 0x7f301406cb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:44054/0 (socket says 172.21.15.16:44054) 2023-05-20T01:18:29.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.240+0000 7f3013fff700 1 -- 172.21.15.16:0/3097322957 learned_addr learned my addr 172.21.15.16:0/3097322957 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:18:29.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.240+0000 7f301914d700 1 -- 172.21.15.16:0/3097322957 >> v1:172.21.15.132:6789/0 conn(0x7f3014153eb0 legacy=0x7f301415a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:29.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.240+0000 7f301914d700 1 -- 172.21.15.16:0/3097322957 >> v1:172.21.15.16:6789/0 conn(0x7f30140ccbd0 legacy=0x7f301406cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:29.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.240+0000 7f301914d700 1 -- 172.21.15.16:0/3097322957 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30140cba30 con 0x7f301406d2a0 2023-05-20T01:18:29.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.240+0000 7f30137fe700 1 -- 172.21.15.16:0/3097322957 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4177423409 0 0) 0x7f30140cd050 con 0x7f30140ccbd0 2023-05-20T01:18:29.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.240+0000 7f301914d700 1 --2- 172.21.15.16:0/3097322957 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f301406d2a0 0x7f30140cb000 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f300c00bfa0 tx=0x7f300c00d330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b9b9c6f9c08e659 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:29.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.240+0000 7f30137fe700 1 -- 172.21.15.16:0/3097322957 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f300c010070 con 0x7f301406d2a0 2023-05-20T01:18:29.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.240+0000 7f30137fe700 1 -- 172.21.15.16:0/3097322957 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f300c00b2e0 con 0x7f301406d2a0 2023-05-20T01:18:29.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.240+0000 7f30137fe700 1 -- 172.21.15.16:0/3097322957 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f300c00dcf0 con 0x7f301406d2a0 2023-05-20T01:18:29.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.241+0000 7f301abb0700 1 -- 172.21.15.16:0/3097322957 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f301406d2a0 msgr2=0x7f30140cb000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:29.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.241+0000 7f301abb0700 1 --2- 172.21.15.16:0/3097322957 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f301406d2a0 0x7f30140cb000 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f300c00bfa0 tx=0x7f300c00d330 comp rx=0 tx=0).stop 2023-05-20T01:18:29.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.241+0000 7f301abb0700 1 -- 172.21.15.16:0/3097322957 shutdown_connections 2023-05-20T01:18:29.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.241+0000 7f301abb0700 1 --2- 172.21.15.16:0/3097322957 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f301406d2a0 0x7f30140cb000 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:29.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.241+0000 7f301abb0700 1 -- 172.21.15.16:0/3097322957 >> 172.21.15.16:0/3097322957 conn(0x7f30140661b0 msgr2=0x7f3014066580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:18:29.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.241+0000 7f301abb0700 1 -- 172.21.15.16:0/3097322957 shutdown_connections 2023-05-20T01:18:29.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.241+0000 7f301abb0700 1 -- 172.21.15.16:0/3097322957 wait complete. 2023-05-20T01:18:29.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.242+0000 7f301abb0700 1 Processor -- start 2023-05-20T01:18:29.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.242+0000 7f301abb0700 1 -- start start 2023-05-20T01:18:29.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.243+0000 7f301abb0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f301406d2a0 0x7f3014147280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:29.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.243+0000 7f301abb0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30140ccbd0 0x7f30141f62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:29.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.243+0000 7f301abb0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3014153eb0 0x7f30141f22a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:29.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.243+0000 7f301abb0700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30141477c0 con 0x7f30140ccbd0 2023-05-20T01:18:29.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.243+0000 7f301abb0700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30141f67e0 con 0x7f301406d2a0 2023-05-20T01:18:29.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.243+0000 7f301abb0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30141f5850 con 0x7f3014153eb0 2023-05-20T01:18:29.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.243+0000 7f3013fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30140ccbd0 0x7f30141f62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:29.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.243+0000 7f3013fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30140ccbd0 0x7f30141f62a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60764/0 (socket says 172.21.15.16:60764) 2023-05-20T01:18:29.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.243+0000 7f3013fff700 1 -- 172.21.15.16:0/1715397440 learned_addr learned my addr 172.21.15.16:0/1715397440 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:18:29.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.243+0000 7f3013fff700 1 -- 172.21.15.16:0/1715397440 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3014153eb0 msgr2=0x7f30141f22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:29.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.243+0000 7f3013fff700 1 --2- 172.21.15.16:0/1715397440 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3014153eb0 0x7f30141f22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:29.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.243+0000 7f3013fff700 1 -- 172.21.15.16:0/1715397440 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f301406d2a0 msgr2=0x7f3014147280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:29.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.243+0000 7f301894c700 1 --2- 172.21.15.16:0/1715397440 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f301406d2a0 0x7f3014147280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:29.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.243+0000 7f3013fff700 1 --2- 172.21.15.16:0/1715397440 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f301406d2a0 0x7f3014147280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:29.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.243+0000 7f3013fff700 1 -- 172.21.15.16:0/1715397440 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f300c00bc80 con 0x7f30140ccbd0 2023-05-20T01:18:29.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.243+0000 7f301894c700 1 --2- 172.21.15.16:0/1715397440 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f301406d2a0 0x7f3014147280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:18:29.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.244+0000 7f3013fff700 1 --2- 172.21.15.16:0/1715397440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30140ccbd0 0x7f30141f62a0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f300000c520 tx=0x7f300000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:29.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.244+0000 7f30117fa700 1 -- 172.21.15.16:0/1715397440 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3000004c80 con 0x7f30140ccbd0 2023-05-20T01:18:29.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.244+0000 7f30117fa700 1 -- 172.21.15.16:0/1715397440 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3000004e00 con 0x7f30140ccbd0 2023-05-20T01:18:29.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.244+0000 7f301abb0700 1 -- 172.21.15.16:0/1715397440 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3014159f40 con 0x7f30140ccbd0 2023-05-20T01:18:29.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.244+0000 7f30117fa700 1 -- 172.21.15.16:0/1715397440 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3000007310 con 0x7f30140ccbd0 2023-05-20T01:18:29.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.244+0000 7f301abb0700 1 -- 172.21.15.16:0/1715397440 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30140cbd10 con 0x7f30140ccbd0 2023-05-20T01:18:29.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.245+0000 7f301abb0700 1 -- 172.21.15.16:0/1715397440 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f301415a220 con 0x7f30140ccbd0 2023-05-20T01:18:29.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.245+0000 7f30117fa700 1 -- 172.21.15.16:0/1715397440 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3000007470 con 0x7f30140ccbd0 2023-05-20T01:18:29.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.246+0000 7f30117fa700 1 --2- 172.21.15.16:0/1715397440 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f2ff806ace0 0x7f2ff806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:29.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.246+0000 7f30117fa700 1 -- 172.21.15.16:0/1715397440 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 7016+0+0 (secure 0 0 0) 0x7f3000089530 con 0x7f30140ccbd0 2023-05-20T01:18:29.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.246+0000 7f301894c700 1 --2- 172.21.15.16:0/1715397440 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f2ff806ace0 0x7f2ff806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:29.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.246+0000 7f301894c700 1 --2- 172.21.15.16:0/1715397440 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f2ff806ace0 0x7f2ff806d190 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f3014153890 tx=0x7f3008009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:29.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.250+0000 7f30117fa700 1 -- 172.21.15.16:0/1715397440 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f30000542b0 con 0x7f30140ccbd0 2023-05-20T01:18:29.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.496+0000 7f301abb0700 1 -- 172.21.15.16:0/1715397440 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f30140747b0 con 0x7f30140ccbd0 2023-05-20T01:18:29.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.496+0000 7f30117fa700 1 -- 172.21.15.16:0/1715397440 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+4671 (secure 0 0 0) 0x7f30000578d0 con 0x7f30140ccbd0 2023-05-20T01:18:29.500 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:18:29.500 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":22,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5176,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/4144439890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4144439890},{"type":"v1","addr":"172.21.15.16:6836","nonce":4144439890}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5188,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/2992755084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":2992755084},{"type":"v1","addr":"172.21.15.132:6834","nonce":2992755084}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/1979284857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1979284857},{"type":"v1","addr":"172.21.15.132:6835","nonce":1979284857}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5730,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/1867776973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":1867776973},{"type":"v1","addr":"172.21.15.16:6837","nonce":1867776973}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-20T01:18:26.227827+0000","modified":"2023-05-20T01:18:27.463059+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[7,8],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-05-20T01:18:29.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.498+0000 7f301abb0700 1 -- 172.21.15.16:0/1715397440 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f2ff806ace0 msgr2=0x7f2ff806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:29.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.498+0000 7f301abb0700 1 --2- 172.21.15.16:0/1715397440 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f2ff806ace0 0x7f2ff806d190 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f3014153890 tx=0x7f3008009040 comp rx=0 tx=0).stop 2023-05-20T01:18:29.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.498+0000 7f301abb0700 1 -- 172.21.15.16:0/1715397440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30140ccbd0 msgr2=0x7f30141f62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:29.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.498+0000 7f301abb0700 1 --2- 172.21.15.16:0/1715397440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30140ccbd0 0x7f30141f62a0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f300000c520 tx=0x7f300000c8e0 comp rx=0 tx=0).stop 2023-05-20T01:18:29.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.498+0000 7f301abb0700 1 -- 172.21.15.16:0/1715397440 shutdown_connections 2023-05-20T01:18:29.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.498+0000 7f301abb0700 1 --2- 172.21.15.16:0/1715397440 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f2ff806ace0 0x7f2ff806d190 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:29.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.498+0000 7f301abb0700 1 --2- 172.21.15.16:0/1715397440 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3014153eb0 0x7f30141f22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:29.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.498+0000 7f301abb0700 1 --2- 172.21.15.16:0/1715397440 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f301406d2a0 0x7f3014147280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:29.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.498+0000 7f301abb0700 1 --2- 172.21.15.16:0/1715397440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30140ccbd0 0x7f30141f62a0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:29.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.498+0000 7f301abb0700 1 -- 172.21.15.16:0/1715397440 >> 172.21.15.16:0/1715397440 conn(0x7f30140661b0 msgr2=0x7f301406cb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:18:29.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.498+0000 7f301abb0700 1 -- 172.21.15.16:0/1715397440 shutdown_connections 2023-05-20T01:18:29.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.499+0000 7f301abb0700 1 -- 172.21.15.16:0/1715397440 wait complete. 2023-05-20T01:18:29.504 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 22 2023-05-20T01:18:29.514 DEBUG:teuthology.orchestra.run.smithi016:> cephfs-table-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 cephfs:0 reset session 2023-05-20T01:18:29.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.553+0000 7f5d2c1402c0 1 -- start start 2023-05-20T01:18:29.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.553+0000 7f5d2c1402c0 1 Processor -- start 2023-05-20T01:18:29.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.553+0000 7f5d2c1402c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x557410ea5000 0x557410f3fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:29.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.553+0000 7f5d2c1402c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557410ea4c00 0x557410f3f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:29.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.553+0000 7f5d2c1402c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557410ea4800 0x557410f3f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:29.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.553+0000 7f5d2c1402c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x557410ee1cc0 con 0x557410ea4800 2023-05-20T01:18:29.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.553+0000 7f5d2c1402c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x557410ee1400 con 0x557410ea4c00 2023-05-20T01:18:29.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.553+0000 7f5d2c1402c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x557410ee0a00 con 0x557410ea5000 2023-05-20T01:18:29.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.553+0000 7f5d2a8ed700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557410ea4800 0x557410f3f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:29.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.553+0000 7f5d2a8ed700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557410ea4800 0x557410f3f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60780/0 (socket says 172.21.15.16:60780) 2023-05-20T01:18:29.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.553+0000 7f5d2a8ed700 1 -- 172.21.15.16:0/573965036 learned_addr learned my addr 172.21.15.16:0/573965036 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:18:29.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.553+0000 7f5d298eb700 1 --2- 172.21.15.16:0/573965036 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557410ea4c00 0x557410f3f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:29.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.553+0000 7f5d2a0ec700 1 --2- 172.21.15.16:0/573965036 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x557410ea5000 0x557410f3fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:29.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.554+0000 7f5d2a8ed700 1 -- 172.21.15.16:0/573965036 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x557410ea5000 msgr2=0x557410f3fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:18:29.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.554+0000 7f5d2a8ed700 1 --2- 172.21.15.16:0/573965036 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x557410ea5000 0x557410f3fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:29.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.554+0000 7f5d2a8ed700 1 -- 172.21.15.16:0/573965036 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557410ea4c00 msgr2=0x557410f3f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:18:29.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.554+0000 7f5d2a8ed700 1 --2- 172.21.15.16:0/573965036 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557410ea4c00 0x557410f3f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:29.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.554+0000 7f5d2a8ed700 1 -- 172.21.15.16:0/573965036 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557410ec8d00 con 0x557410ea4800 2023-05-20T01:18:29.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.554+0000 7f5d2a8ed700 1 --2- 172.21.15.16:0/573965036 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557410ea4800 0x557410f3f080 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x557411d0e6c0 tx=0x557411d104e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0e052d30b3d4053 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:29.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.554+0000 7f5d278e7700 1 -- 172.21.15.16:0/573965036 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x557410ed9600 con 0x557410ea4800 2023-05-20T01:18:29.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.554+0000 7f5d278e7700 1 -- 172.21.15.16:0/573965036 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557410ef6600 con 0x557410ea4800 2023-05-20T01:18:29.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.554+0000 7f5d278e7700 1 -- 172.21.15.16:0/573965036 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x557410ed91e0 con 0x557410ea4800 2023-05-20T01:18:29.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.554+0000 7f5d2c1402c0 1 -- 172.21.15.16:0/573965036 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557410ec96c0 con 0x557410ea4800 2023-05-20T01:18:29.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.554+0000 7f5d2c1402c0 1 -- 172.21.15.16:0/573965036 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x557410ec9520 con 0x557410ea4800 2023-05-20T01:18:29.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.554+0000 7f5d2c1402c0 4 waiting for MDS map... 2023-05-20T01:18:29.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.554+0000 7f5d278e7700 1 -- 172.21.15.16:0/573965036 <== mon.0 v2:172.21.15.16:3300/0 4 ==== osd_map(47..47 src has 1..47) v4 ==== 7016+0+0 (secure 0 0 0) 0x557410ee6c40 con 0x557410ea4800 2023-05-20T01:18:29.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.554+0000 7f5d278e7700 1 -- 172.21.15.16:0/573965036 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 22) v1 ==== 3126+0+0 (secure 0 0 0) 0x557411c75200 con 0x557410ea4800 2023-05-20T01:18:29.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.554+0000 7f5d2c1402c0 4 Got MDS map 22 2023-05-20T01:18:29.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.554+0000 7f5d2c1402c0 10 main: main 2023-05-20T01:18:29.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.555+0000 7f5d2c1402c0 4 main: connecting to RADOS... 2023-05-20T01:18:29.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.555+0000 7f5d2c1402c0 1 Processor -- start 2023-05-20T01:18:29.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.555+0000 7f5d2c1402c0 1 -- start start 2023-05-20T01:18:29.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.556+0000 7f5d2c1402c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557410ea5c00 0x557410f3eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:29.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.556+0000 7f5d2c1402c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5574111ae000 0x557410f3e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:29.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.556+0000 7f5d2c1402c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5574111ae400 0x557410f40100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:29.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.556+0000 7f5d2c1402c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x557410ee0b40 con 0x557410ea5c00 2023-05-20T01:18:29.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.556+0000 7f5d2c1402c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x557410ee1400 con 0x5574111ae400 2023-05-20T01:18:29.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.556+0000 7f5d2c1402c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x557410ee0c80 con 0x5574111ae000 2023-05-20T01:18:29.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.556+0000 7f5d298eb700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557410ea5c00 0x557410f3eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:29.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.556+0000 7f5d298eb700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557410ea5c00 0x557410f3eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60782/0 (socket says 172.21.15.16:60782) 2023-05-20T01:18:29.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.556+0000 7f5d298eb700 1 -- 172.21.15.16:0/3053360788 learned_addr learned my addr 172.21.15.16:0/3053360788 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:18:29.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.556+0000 7f5d298eb700 1 -- 172.21.15.16:0/3053360788 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5574111ae000 msgr2=0x557410f3e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:18:29.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.556+0000 7f5d298eb700 1 --2- 172.21.15.16:0/3053360788 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5574111ae000 0x557410f3e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:29.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.556+0000 7f5d298eb700 1 -- 172.21.15.16:0/3053360788 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5574111ae400 msgr2=0x557410f40100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:18:29.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.556+0000 7f5d298eb700 1 --2- 172.21.15.16:0/3053360788 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5574111ae400 0x557410f40100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:29.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.556+0000 7f5d298eb700 1 -- 172.21.15.16:0/3053360788 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557410ec9380 con 0x557410ea5c00 2023-05-20T01:18:29.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.556+0000 7f5d298eb700 1 --2- 172.21.15.16:0/3053360788 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557410ea5c00 0x557410f3eb00 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x557411d0fbf0 tx=0x557411d34d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b60434e4c450e56 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:29.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.556+0000 7f5d258e3700 1 -- 172.21.15.16:0/3053360788 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x557410ed9080 con 0x557410ea5c00 2023-05-20T01:18:29.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.557+0000 7f5d258e3700 1 -- 172.21.15.16:0/3053360788 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557410ef7e00 con 0x557410ea5c00 2023-05-20T01:18:29.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.557+0000 7f5d258e3700 1 -- 172.21.15.16:0/3053360788 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x557410ed9600 con 0x557410ea5c00 2023-05-20T01:18:29.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.557+0000 7f5d2c1402c0 1 -- 172.21.15.16:0/3053360788 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557410ea5c00 msgr2=0x557410f3eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:29.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.557+0000 7f5d2c1402c0 1 --2- 172.21.15.16:0/3053360788 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557410ea5c00 0x557410f3eb00 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x557411d0fbf0 tx=0x557411d34d20 comp rx=0 tx=0).stop 2023-05-20T01:18:29.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.557+0000 7f5d2c1402c0 1 -- 172.21.15.16:0/3053360788 shutdown_connections 2023-05-20T01:18:29.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.557+0000 7f5d2c1402c0 1 --2- 172.21.15.16:0/3053360788 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5574111ae400 0x557410f40100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:29.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.557+0000 7f5d2c1402c0 1 --2- 172.21.15.16:0/3053360788 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557410ea5c00 0x557410f3eb00 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:29.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.557+0000 7f5d2c1402c0 1 --2- 172.21.15.16:0/3053360788 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5574111ae000 0x557410f3e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:29.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.557+0000 7f5d2c1402c0 1 -- 172.21.15.16:0/3053360788 >> 172.21.15.16:0/3053360788 conn(0x557410ea4400 msgr2=0x557410f75800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:18:29.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.557+0000 7f5d2c1402c0 1 -- 172.21.15.16:0/3053360788 shutdown_connections 2023-05-20T01:18:29.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.557+0000 7f5d2c1402c0 1 -- 172.21.15.16:0/3053360788 wait complete. 2023-05-20T01:18:29.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.558+0000 7f5d2c1402c0 1 Processor -- start 2023-05-20T01:18:29.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.558+0000 7f5d2c1402c0 1 -- start start 2023-05-20T01:18:29.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.558+0000 7f5d2c1402c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557410ea4400 0x557410f3eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:29.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.558+0000 7f5d2c1402c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5574111c4000 0x557410f3e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:29.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.558+0000 7f5d2c1402c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5574111c4400 0x557410f40100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:29.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.558+0000 7f5d2c1402c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x557410ee0000 con 0x557410ea4400 2023-05-20T01:18:29.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.558+0000 7f5d2c1402c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x557410ee1180 con 0x5574111c4000 2023-05-20T01:18:29.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.558+0000 7f5d2c1402c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x557410ee1040 con 0x5574111c4400 2023-05-20T01:18:29.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.558+0000 7f5d298eb700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557410ea4400 0x557410f3eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:29.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.558+0000 7f5d298eb700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557410ea4400 0x557410f3eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60788/0 (socket says 172.21.15.16:60788) 2023-05-20T01:18:29.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.558+0000 7f5d298eb700 1 -- 172.21.15.16:0/2039203165 learned_addr learned my addr 172.21.15.16:0/2039203165 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:18:29.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.559+0000 7f5d2a0ec700 1 --2- 172.21.15.16:0/2039203165 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5574111c4000 0x557410f3e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:29.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.559+0000 7f5d298eb700 1 -- 172.21.15.16:0/2039203165 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5574111c4400 msgr2=0x557410f40100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:29.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.559+0000 7f5d298eb700 1 --2- 172.21.15.16:0/2039203165 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5574111c4400 0x557410f40100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:29.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.559+0000 7f5d298eb700 1 -- 172.21.15.16:0/2039203165 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5574111c4000 msgr2=0x557410f3e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:29.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.559+0000 7f5d298eb700 1 --2- 172.21.15.16:0/2039203165 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5574111c4000 0x557410f3e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:29.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.559+0000 7f5d298eb700 1 -- 172.21.15.16:0/2039203165 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557410ec91e0 con 0x557410ea4400 2023-05-20T01:18:29.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.559+0000 7f5d2a0ec700 1 --2- 172.21.15.16:0/2039203165 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5574111c4000 0x557410f3e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:18:29.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.559+0000 7f5d298eb700 1 --2- 172.21.15.16:0/2039203165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557410ea4400 0x557410f3eb00 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x5574111c26c0 tx=0x557411d34c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:29.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.559+0000 7f5d240e0700 1 -- 172.21.15.16:0/2039203165 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x557410ed91e0 con 0x557410ea4400 2023-05-20T01:18:29.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.559+0000 7f5d240e0700 1 -- 172.21.15.16:0/2039203165 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557410ef7c80 con 0x557410ea4400 2023-05-20T01:18:29.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.559+0000 7f5d2c1402c0 1 -- 172.21.15.16:0/2039203165 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x557410ec9d40 con 0x557410ea4400 2023-05-20T01:18:29.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.559+0000 7f5d240e0700 1 -- 172.21.15.16:0/2039203165 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x557410ed8dc0 con 0x557410ea4400 2023-05-20T01:18:29.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.559+0000 7f5d2c1402c0 1 -- 172.21.15.16:0/2039203165 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557410ec9380 con 0x557410ea4400 2023-05-20T01:18:29.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.559+0000 7f5d2c1402c0 4 main: resolving pool 6 2023-05-20T01:18:29.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.560+0000 7f5d240e0700 1 -- 172.21.15.16:0/2039203165 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x557410f06840 con 0x557410ea4400 2023-05-20T01:18:29.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.560+0000 7f5d240e0700 1 --2- 172.21.15.16:0/2039203165 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x5574111cc400 0x5574112c2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:29.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.560+0000 7f5d240e0700 1 -- 172.21.15.16:0/2039203165 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 7016+0+0 (secure 0 0 0) 0x557410ee6a80 con 0x557410ea4400 2023-05-20T01:18:29.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.561+0000 7f5d2c1402c0 4 main: creating IoCtx.. 2023-05-20T01:18:29.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.561+0000 7f5d2a0ec700 1 --2- 172.21.15.16:0/2039203165 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x5574111cc400 0x5574112c2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:29.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.561+0000 7f5d2c1402c0 1 --2- 172.21.15.16:0/2039203165 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x5574111c4800 0x557410f40680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:29.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.561+0000 7f5d2c1402c0 1 -- 172.21.15.16:0/2039203165 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:1 6.b 6:d0630e4c:::mds0_sessionmap:head [omap-clear,omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47) v8 -- 0x557411d2c700 con 0x5574111c4800 2023-05-20T01:18:29.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.561+0000 7f5d298eb700 1 --2- 172.21.15.16:0/2039203165 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x5574111c4800 0x557410f40680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:29.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.561+0000 7f5d2a0ec700 1 --2- 172.21.15.16:0/2039203165 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x5574111cc400 0x5574112c2000 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x557411c1fc20 tx=0x557411d35e60 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:29.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.561+0000 7f5d298eb700 1 --2- 172.21.15.16:0/2039203165 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x5574111c4800 0x557410f40680 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:29.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.563+0000 7f5d298eb700 1 -- 172.21.15.16:0/2039203165 <== osd.3 v2:172.21.15.16:6800/3669781890 1 ==== osd_op_reply(1 mds0_sessionmap [omap-clear,omap-set-header] v47'5 uv5 ondisk = 0) v8 ==== 201+0+0 (crc 0 0 0) 0x557411c04fc0 con 0x5574111c4800 2023-05-20T01:18:29.575 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-05-20T01:18:29.575 INFO:teuthology.orchestra.run.smithi016.stdout: "0": { 2023-05-20T01:18:29.575 INFO:teuthology.orchestra.run.smithi016.stdout: "data": {}, 2023-05-20T01:18:29.575 INFO:teuthology.orchestra.run.smithi016.stdout: "result": 0 2023-05-20T01:18:29.575 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:18:29.575 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-05-20T01:18:29.576 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:18:29.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.563+0000 7f5d2c1402c0 1 -- 172.21.15.16:0/2039203165 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x5574111c4800 msgr2=0x557410f40680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:29.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.563+0000 7f5d2c1402c0 1 --2- 172.21.15.16:0/2039203165 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x5574111c4800 0x557410f40680 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:29.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.563+0000 7f5d2c1402c0 1 -- 172.21.15.16:0/2039203165 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x5574111cc400 msgr2=0x5574112c2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:29.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.563+0000 7f5d2c1402c0 1 --2- 172.21.15.16:0/2039203165 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x5574111cc400 0x5574112c2000 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x557411c1fc20 tx=0x557411d35e60 comp rx=0 tx=0).stop 2023-05-20T01:18:29.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.563+0000 7f5d2c1402c0 1 -- 172.21.15.16:0/2039203165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557410ea4400 msgr2=0x557410f3eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:29.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.563+0000 7f5d2c1402c0 1 --2- 172.21.15.16:0/2039203165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557410ea4400 0x557410f3eb00 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x5574111c26c0 tx=0x557411d34c00 comp rx=0 tx=0).stop 2023-05-20T01:18:29.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.563+0000 7f5d2a0ec700 1 -- 172.21.15.16:0/2039203165 reap_dead start 2023-05-20T01:18:29.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.563+0000 7f5d2c1402c0 1 -- 172.21.15.16:0/2039203165 shutdown_connections 2023-05-20T01:18:29.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.563+0000 7f5d2c1402c0 1 -- 172.21.15.16:0/2039203165 >> 172.21.15.16:0/2039203165 conn(0x557410ea5c00 msgr2=0x557410f75800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:18:29.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.563+0000 7f5d2c1402c0 1 -- 172.21.15.16:0/2039203165 shutdown_connections 2023-05-20T01:18:29.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.563+0000 7f5d2c1402c0 1 -- 172.21.15.16:0/2039203165 wait complete. 2023-05-20T01:18:29.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.563+0000 7f5d2c1402c0 1 -- 172.21.15.16:0/573965036 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557410ea4800 msgr2=0x557410f3f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:29.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.563+0000 7f5d2c1402c0 1 --2- 172.21.15.16:0/573965036 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557410ea4800 0x557410f3f080 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x557411d0e6c0 tx=0x557411d104e0 comp rx=0 tx=0).stop 2023-05-20T01:18:29.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.563+0000 7f5d2c1402c0 1 -- 172.21.15.16:0/573965036 shutdown_connections 2023-05-20T01:18:29.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.563+0000 7f5d2c1402c0 1 --2- 172.21.15.16:0/573965036 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557410ea4800 0x557410f3f080 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:29.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.563+0000 7f5d2c1402c0 1 --2- 172.21.15.16:0/573965036 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x557410ea5000 0x557410f3fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:29.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.563+0000 7f5d2c1402c0 1 --2- 172.21.15.16:0/573965036 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557410ea4c00 0x557410f3f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:29.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.563+0000 7f5d2c1402c0 1 -- 172.21.15.16:0/573965036 >> 172.21.15.16:0/573965036 conn(0x557410ea5400 msgr2=0x557410f75000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:18:29.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.563+0000 7f5d2c1402c0 1 -- 172.21.15.16:0/573965036 shutdown_connections 2023-05-20T01:18:29.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.563+0000 7f5d2c1402c0 1 -- 172.21.15.16:0/573965036 wait complete. 2023-05-20T01:18:29.580 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-table-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'cephfs:0', 'reset', 'session'] in time 0:00:00.065855, result: { "0": { "data": {}, "result": 0 } } 2023-05-20T01:18:29.581 DEBUG:teuthology.orchestra.run.smithi016:> cephfs-table-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 cephfs:0 reset snap 2023-05-20T01:18:29.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.656+0000 7fa48ea302c0 1 -- start start 2023-05-20T01:18:29.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.656+0000 7fa48ea302c0 1 Processor -- start 2023-05-20T01:18:29.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.656+0000 7fa48ea302c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1fb45f000 0x55f1fb4f8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:29.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.656+0000 7fa48ea302c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f1fb45ec00 0x55f1fb4f8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:29.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.656+0000 7fa48ea302c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55f1fb45e800 0x55f1fb4f9080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:29.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.656+0000 7fa48ea302c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1fb49bcc0 con 0x55f1fb45f000 2023-05-20T01:18:29.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.656+0000 7fa48ea302c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1fb49b400 con 0x55f1fb45ec00 2023-05-20T01:18:29.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.656+0000 7fa48ea302c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f1fb49be00 con 0x55f1fb45e800 2023-05-20T01:18:29.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.656+0000 7fa48c9dc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1fb45f000 0x55f1fb4f8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:29.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.656+0000 7fa48c9dc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1fb45f000 0x55f1fb4f8580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60802/0 (socket says 172.21.15.16:60802) 2023-05-20T01:18:29.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.656+0000 7fa48c9dc700 1 -- 172.21.15.16:0/1218577099 learned_addr learned my addr 172.21.15.16:0/1218577099 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:18:29.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.657+0000 7fa48d1dd700 1 --2- 172.21.15.16:0/1218577099 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55f1fb45e800 0x55f1fb4f9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:29.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.657+0000 7fa48c1db700 1 --2- 172.21.15.16:0/1218577099 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f1fb45ec00 0x55f1fb4f8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:29.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.657+0000 7fa48c9dc700 1 -- 172.21.15.16:0/1218577099 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55f1fb45e800 msgr2=0x55f1fb4f9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:18:29.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.657+0000 7fa48c9dc700 1 --2- 172.21.15.16:0/1218577099 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55f1fb45e800 0x55f1fb4f9080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:29.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.657+0000 7fa48c9dc700 1 -- 172.21.15.16:0/1218577099 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f1fb45ec00 msgr2=0x55f1fb4f8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:18:29.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.657+0000 7fa48c9dc700 1 --2- 172.21.15.16:0/1218577099 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f1fb45ec00 0x55f1fb4f8b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:29.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.657+0000 7fa48c9dc700 1 -- 172.21.15.16:0/1218577099 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f1fb482d00 con 0x55f1fb45f000 2023-05-20T01:18:29.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.657+0000 7fa48c9dc700 1 --2- 172.21.15.16:0/1218577099 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1fb45f000 0x55f1fb4f8580 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x55f1fc230720 tx=0x55f1fc242f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78bc4e9342a6fbe server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:29.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.657+0000 7fa48a1d7700 1 -- 172.21.15.16:0/1218577099 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55f1fb493600 con 0x55f1fb45f000 2023-05-20T01:18:29.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.657+0000 7fa48a1d7700 1 -- 172.21.15.16:0/1218577099 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f1fb4b0c00 con 0x55f1fb45f000 2023-05-20T01:18:29.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.657+0000 7fa48a1d7700 1 -- 172.21.15.16:0/1218577099 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55f1fb4931e0 con 0x55f1fb45f000 2023-05-20T01:18:29.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.657+0000 7fa48ea302c0 1 -- 172.21.15.16:0/1218577099 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f1fb4836c0 con 0x55f1fb45f000 2023-05-20T01:18:29.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.657+0000 7fa48ea302c0 1 -- 172.21.15.16:0/1218577099 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55f1fb483520 con 0x55f1fb45f000 2023-05-20T01:18:29.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.657+0000 7fa48ea302c0 4 waiting for MDS map... 2023-05-20T01:18:29.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.657+0000 7fa48a1d7700 1 -- 172.21.15.16:0/1218577099 <== mon.0 v2:172.21.15.16:3300/0 4 ==== osd_map(47..47 src has 1..47) v4 ==== 7016+0+0 (secure 0 0 0) 0x55f1fb4a0c40 con 0x55f1fb45f000 2023-05-20T01:18:29.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.657+0000 7fa48a1d7700 1 -- 172.21.15.16:0/1218577099 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 22) v1 ==== 3126+0+0 (secure 0 0 0) 0x55f1fc2ca300 con 0x55f1fb45f000 2023-05-20T01:18:29.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.657+0000 7fa48ea302c0 4 Got MDS map 22 2023-05-20T01:18:29.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.658+0000 7fa48ea302c0 10 main: main 2023-05-20T01:18:29.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.658+0000 7fa48ea302c0 4 main: connecting to RADOS... 2023-05-20T01:18:29.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.658+0000 7fa48ea302c0 1 Processor -- start 2023-05-20T01:18:29.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.658+0000 7fa48ea302c0 1 -- start start 2023-05-20T01:18:29.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.659+0000 7fa48ea302c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55f1fb45fc00 0x55f1fb4f9600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:29.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.659+0000 7fa48ea302c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f1fb776000 0x55f1fb4f9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:29.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.659+0000 7fa48ea302c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1fb776400 0x55f1fb4fa100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:29.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.659+0000 7fa48ea302c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1fb49aa00 con 0x55f1fb776400 2023-05-20T01:18:29.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.659+0000 7fa48ea302c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1fb49b400 con 0x55f1fb776000 2023-05-20T01:18:29.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.659+0000 7fa48ea302c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f1fb49ab40 con 0x55f1fb45fc00 2023-05-20T01:18:29.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.659+0000 7fa48c1db700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1fb776400 0x55f1fb4fa100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:29.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.659+0000 7fa48c1db700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1fb776400 0x55f1fb4fa100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60812/0 (socket says 172.21.15.16:60812) 2023-05-20T01:18:29.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.659+0000 7fa48c1db700 1 -- 172.21.15.16:0/1474279126 learned_addr learned my addr 172.21.15.16:0/1474279126 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:18:29.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.659+0000 7fa48d1dd700 1 --2- 172.21.15.16:0/1474279126 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55f1fb45fc00 0x55f1fb4f9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:29.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.659+0000 7fa48c9dc700 1 --2- 172.21.15.16:0/1474279126 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f1fb776000 0x55f1fb4f9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:29.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.659+0000 7fa48c1db700 1 -- 172.21.15.16:0/1474279126 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55f1fb45fc00 msgr2=0x55f1fb4f9600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:18:29.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.659+0000 7fa48c1db700 1 --2- 172.21.15.16:0/1474279126 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55f1fb45fc00 0x55f1fb4f9600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:29.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.659+0000 7fa48c1db700 1 -- 172.21.15.16:0/1474279126 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f1fb776000 msgr2=0x55f1fb4f9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:18:29.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.659+0000 7fa48c1db700 1 --2- 172.21.15.16:0/1474279126 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f1fb776000 0x55f1fb4f9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:29.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.659+0000 7fa48c1db700 1 -- 172.21.15.16:0/1474279126 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f1fb483380 con 0x55f1fb776400 2023-05-20T01:18:29.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.659+0000 7fa48c9dc700 1 --2- 172.21.15.16:0/1474279126 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f1fb776000 0x55f1fb4f9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:18:29.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.659+0000 7fa48c1db700 1 --2- 172.21.15.16:0/1474279126 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1fb776400 0x55f1fb4fa100 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x55f1fc231770 tx=0x55f1fc2f5800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=362fcc131f3f715f server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:29.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.659+0000 7fa4881d3700 1 -- 172.21.15.16:0/1474279126 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55f1fb493600 con 0x55f1fb776400 2023-05-20T01:18:29.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.660+0000 7fa4881d3700 1 -- 172.21.15.16:0/1474279126 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f1fb4b1e00 con 0x55f1fb776400 2023-05-20T01:18:29.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.660+0000 7fa4881d3700 1 -- 172.21.15.16:0/1474279126 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55f1fb4931e0 con 0x55f1fb776400 2023-05-20T01:18:29.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.660+0000 7fa48ea302c0 1 -- 172.21.15.16:0/1474279126 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1fb776400 msgr2=0x55f1fb4fa100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:29.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.660+0000 7fa48ea302c0 1 --2- 172.21.15.16:0/1474279126 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1fb776400 0x55f1fb4fa100 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x55f1fc231770 tx=0x55f1fc2f5800 comp rx=0 tx=0).stop 2023-05-20T01:18:29.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.660+0000 7fa48ea302c0 1 -- 172.21.15.16:0/1474279126 shutdown_connections 2023-05-20T01:18:29.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.660+0000 7fa48ea302c0 1 --2- 172.21.15.16:0/1474279126 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1fb776400 0x55f1fb4fa100 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:29.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.660+0000 7fa48ea302c0 1 --2- 172.21.15.16:0/1474279126 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55f1fb45fc00 0x55f1fb4f9600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:29.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.660+0000 7fa48ea302c0 1 --2- 172.21.15.16:0/1474279126 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f1fb776000 0x55f1fb4f9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:29.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.660+0000 7fa48ea302c0 1 -- 172.21.15.16:0/1474279126 >> 172.21.15.16:0/1474279126 conn(0x55f1fb45e400 msgr2=0x55f1fb52f800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:18:29.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.660+0000 7fa48ea302c0 1 -- 172.21.15.16:0/1474279126 shutdown_connections 2023-05-20T01:18:29.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.660+0000 7fa48ea302c0 1 -- 172.21.15.16:0/1474279126 wait complete. 2023-05-20T01:18:29.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.661+0000 7fa48ea302c0 1 Processor -- start 2023-05-20T01:18:29.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.661+0000 7fa48ea302c0 1 -- start start 2023-05-20T01:18:29.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.661+0000 7fa48ea302c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55f1fb45e400 0x55f1fb4fb180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:29.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.661+0000 7fa48ea302c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1fb78e000 0x55f1fb4fac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:29.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.661+0000 7fa48ea302c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f1fb78e400 0x55f1fb4fa680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:29.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.661+0000 7fa48ea302c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1fb49ac80 con 0x55f1fb78e000 2023-05-20T01:18:29.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.661+0000 7fa48ea302c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1fb49a000 con 0x55f1fb78e400 2023-05-20T01:18:29.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.661+0000 7fa48ea302c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f1fb49b180 con 0x55f1fb45e400 2023-05-20T01:18:29.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.661+0000 7fa48c9dc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1fb78e000 0x55f1fb4fac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:29.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.661+0000 7fa48c9dc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1fb78e000 0x55f1fb4fac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60816/0 (socket says 172.21.15.16:60816) 2023-05-20T01:18:29.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.661+0000 7fa48c9dc700 1 -- 172.21.15.16:0/3155965925 learned_addr learned my addr 172.21.15.16:0/3155965925 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:18:29.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.661+0000 7fa48d1dd700 1 --2- 172.21.15.16:0/3155965925 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55f1fb45e400 0x55f1fb4fb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:29.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.661+0000 7fa48c1db700 1 --2- 172.21.15.16:0/3155965925 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f1fb78e400 0x55f1fb4fa680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:29.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.662+0000 7fa48c9dc700 1 -- 172.21.15.16:0/3155965925 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55f1fb45e400 msgr2=0x55f1fb4fb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:29.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.662+0000 7fa48c9dc700 1 --2- 172.21.15.16:0/3155965925 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55f1fb45e400 0x55f1fb4fb180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:29.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.662+0000 7fa48c9dc700 1 -- 172.21.15.16:0/3155965925 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f1fb78e400 msgr2=0x55f1fb4fa680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:29.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.662+0000 7fa48c9dc700 1 --2- 172.21.15.16:0/3155965925 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f1fb78e400 0x55f1fb4fa680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:29.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.662+0000 7fa48c9dc700 1 -- 172.21.15.16:0/3155965925 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f1fb482d00 con 0x55f1fb78e000 2023-05-20T01:18:29.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.662+0000 7fa48c9dc700 1 --2- 172.21.15.16:0/3155965925 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1fb78e000 0x55f1fb4fac00 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x55f1fc2315f0 tx=0x55f1fc242ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:29.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.662+0000 7fa4869d0700 1 -- 172.21.15.16:0/3155965925 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55f1fb492f20 con 0x55f1fb78e000 2023-05-20T01:18:29.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.662+0000 7fa4869d0700 1 -- 172.21.15.16:0/3155965925 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f1fc2e6300 con 0x55f1fb78e000 2023-05-20T01:18:29.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.662+0000 7fa48ea302c0 1 -- 172.21.15.16:0/3155965925 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55f1fb483d40 con 0x55f1fb78e000 2023-05-20T01:18:29.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.662+0000 7fa4869d0700 1 -- 172.21.15.16:0/3155965925 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55f1fb493600 con 0x55f1fb78e000 2023-05-20T01:18:29.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.662+0000 7fa48ea302c0 1 -- 172.21.15.16:0/3155965925 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f1fb4836c0 con 0x55f1fb78e000 2023-05-20T01:18:29.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.662+0000 7fa48ea302c0 4 main: resolving pool 6 2023-05-20T01:18:29.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.663+0000 7fa4869d0700 1 -- 172.21.15.16:0/3155965925 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x55f1fb4c0840 con 0x55f1fb78e000 2023-05-20T01:18:29.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.664+0000 7fa4869d0700 1 --2- 172.21.15.16:0/3155965925 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x55f1fb790800 0x55f1fb4fb700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:29.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.664+0000 7fa4869d0700 1 -- 172.21.15.16:0/3155965925 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 7016+0+0 (secure 0 0 0) 0x55f1fb4a0a80 con 0x55f1fb78e000 2023-05-20T01:18:29.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.664+0000 7fa48ea302c0 4 main: creating IoCtx.. 2023-05-20T01:18:29.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.664+0000 7fa48d1dd700 1 --2- 172.21.15.16:0/3155965925 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x55f1fb790800 0x55f1fb4fb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:29.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.664+0000 7fa48ea302c0 1 --2- 172.21.15.16:0/3155965925 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x55f1fb78e800 0x55f1fb4fa100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:29.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.664+0000 7fa48ea302c0 1 -- 172.21.15.16:0/3155965925 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:1 6.d 6:b50e409b:::mds_snaptable:head [writefull 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47) v8 -- 0x55f1fc2cc700 con 0x55f1fb78e800 2023-05-20T01:18:29.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.664+0000 7fa48c1db700 1 --2- 172.21.15.16:0/3155965925 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x55f1fb78e800 0x55f1fb4fa100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:29.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.664+0000 7fa48d1dd700 1 --2- 172.21.15.16:0/3155965925 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x55f1fb790800 0x55f1fb4fb700 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x55f1fc1d8e70 tx=0x55f1fc242120 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:29.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.664+0000 7fa48c1db700 1 --2- 172.21.15.16:0/3155965925 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x55f1fb78e800 0x55f1fb4fa100 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:29.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.666+0000 7fa48c1db700 1 -- 172.21.15.16:0/3155965925 <== osd.5 v2:172.21.15.132:6816/1844689995 1 ==== osd_op_reply(1 mds_snaptable [writefull 0~70] v47'3 uv3 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x55f1fc1c0fc0 con 0x55f1fb78e800 2023-05-20T01:18:29.678 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-05-20T01:18:29.679 INFO:teuthology.orchestra.run.smithi016.stdout: "result": 0 2023-05-20T01:18:29.679 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-05-20T01:18:29.679 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:18:29.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.666+0000 7fa48ea302c0 1 -- 172.21.15.16:0/3155965925 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x55f1fb78e800 msgr2=0x55f1fb4fa100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:29.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.666+0000 7fa48ea302c0 1 --2- 172.21.15.16:0/3155965925 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x55f1fb78e800 0x55f1fb4fa100 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:29.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.666+0000 7fa48ea302c0 1 -- 172.21.15.16:0/3155965925 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x55f1fb790800 msgr2=0x55f1fb4fb700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:29.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.666+0000 7fa48ea302c0 1 --2- 172.21.15.16:0/3155965925 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x55f1fb790800 0x55f1fb4fb700 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x55f1fc1d8e70 tx=0x55f1fc242120 comp rx=0 tx=0).stop 2023-05-20T01:18:29.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.666+0000 7fa48ea302c0 1 -- 172.21.15.16:0/3155965925 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1fb78e000 msgr2=0x55f1fb4fac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:29.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.666+0000 7fa48ea302c0 1 --2- 172.21.15.16:0/3155965925 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1fb78e000 0x55f1fb4fac00 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x55f1fc2315f0 tx=0x55f1fc242ea0 comp rx=0 tx=0).stop 2023-05-20T01:18:29.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.666+0000 7fa48c1db700 1 -- 172.21.15.16:0/3155965925 reap_dead start 2023-05-20T01:18:29.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.666+0000 7fa48ea302c0 1 -- 172.21.15.16:0/3155965925 shutdown_connections 2023-05-20T01:18:29.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.666+0000 7fa48ea302c0 1 -- 172.21.15.16:0/3155965925 >> 172.21.15.16:0/3155965925 conn(0x55f1fb45fc00 msgr2=0x55f1fb52f800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:18:29.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.666+0000 7fa48ea302c0 1 -- 172.21.15.16:0/3155965925 shutdown_connections 2023-05-20T01:18:29.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.666+0000 7fa48ea302c0 1 -- 172.21.15.16:0/3155965925 wait complete. 2023-05-20T01:18:29.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.666+0000 7fa48ea302c0 1 -- 172.21.15.16:0/1218577099 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1fb45f000 msgr2=0x55f1fb4f8580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:29.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.666+0000 7fa48ea302c0 1 --2- 172.21.15.16:0/1218577099 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1fb45f000 0x55f1fb4f8580 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x55f1fc230720 tx=0x55f1fc242f60 comp rx=0 tx=0).stop 2023-05-20T01:18:29.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.666+0000 7fa48ea302c0 1 -- 172.21.15.16:0/1218577099 shutdown_connections 2023-05-20T01:18:29.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.666+0000 7fa48ea302c0 1 --2- 172.21.15.16:0/1218577099 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55f1fb45e800 0x55f1fb4f9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:29.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.666+0000 7fa48ea302c0 1 --2- 172.21.15.16:0/1218577099 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1fb45f000 0x55f1fb4f8580 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:29.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.666+0000 7fa48ea302c0 1 --2- 172.21.15.16:0/1218577099 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f1fb45ec00 0x55f1fb4f8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:29.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.667+0000 7fa48ea302c0 1 -- 172.21.15.16:0/1218577099 >> 172.21.15.16:0/1218577099 conn(0x55f1fb45f400 msgr2=0x55f1fb52f000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:18:29.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.667+0000 7fa48ea302c0 1 -- 172.21.15.16:0/1218577099 shutdown_connections 2023-05-20T01:18:29.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.667+0000 7fa48ea302c0 1 -- 172.21.15.16:0/1218577099 wait complete. 2023-05-20T01:18:29.683 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-table-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'cephfs:0', 'reset', 'snap'] in time 0:00:00.102489, result: { "result": 0 } 2023-05-20T01:18:29.683 DEBUG:teuthology.orchestra.run.smithi016:> cephfs-table-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 cephfs:0 reset inode 2023-05-20T01:18:29.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.759+0000 7f48db8362c0 1 -- start start 2023-05-20T01:18:29.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.759+0000 7f48db8362c0 1 Processor -- start 2023-05-20T01:18:29.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.759+0000 7f48db8362c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560956ea7800 0x560956f41080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:29.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.759+0000 7f48db8362c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x560956ea7400 0x560956f40b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:29.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.759+0000 7f48db8362c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560956ea7c00 0x560956f41b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:29.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.759+0000 7f48db8362c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x560956ee3cc0 con 0x560956ea7800 2023-05-20T01:18:29.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.759+0000 7f48db8362c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x560956ee3400 con 0x560956ea7c00 2023-05-20T01:18:29.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.759+0000 7f48db8362c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x560956ee2000 con 0x560956ea7400 2023-05-20T01:18:29.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.759+0000 7f48d97e2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560956ea7800 0x560956f41080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:29.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.759+0000 7f48d97e2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560956ea7800 0x560956f41080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60834/0 (socket says 172.21.15.16:60834) 2023-05-20T01:18:29.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.759+0000 7f48d97e2700 1 -- 172.21.15.16:0/1230635765 learned_addr learned my addr 172.21.15.16:0/1230635765 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:18:29.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.759+0000 7f48d9fe3700 1 --2- 172.21.15.16:0/1230635765 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560956ea7c00 0x560956f41b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:29.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.759+0000 7f48d8fe1700 1 --2- 172.21.15.16:0/1230635765 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x560956ea7400 0x560956f40b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:29.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.760+0000 7f48d97e2700 1 -- 172.21.15.16:0/1230635765 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x560956ea7400 msgr2=0x560956f40b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:18:29.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.760+0000 7f48d97e2700 1 --2- 172.21.15.16:0/1230635765 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x560956ea7400 0x560956f40b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:29.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.760+0000 7f48d97e2700 1 -- 172.21.15.16:0/1230635765 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560956ea7c00 msgr2=0x560956f41b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:18:29.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.760+0000 7f48d97e2700 1 --2- 172.21.15.16:0/1230635765 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560956ea7c00 0x560956f41b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:29.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.760+0000 7f48d97e2700 1 -- 172.21.15.16:0/1230635765 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560956ecb520 con 0x560956ea7800 2023-05-20T01:18:29.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.760+0000 7f48d97e2700 1 --2- 172.21.15.16:0/1230635765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560956ea7800 0x560956f41080 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x560957c82690 tx=0x560957c72000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54fcd0b7efe3daf6 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:29.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.760+0000 7f48d6fdd700 1 -- 172.21.15.16:0/1230635765 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x560956edb080 con 0x560956ea7800 2023-05-20T01:18:29.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.760+0000 7f48d6fdd700 1 -- 172.21.15.16:0/1230635765 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560956ef9800 con 0x560956ea7800 2023-05-20T01:18:29.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.760+0000 7f48d6fdd700 1 -- 172.21.15.16:0/1230635765 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x560956edb600 con 0x560956ea7800 2023-05-20T01:18:29.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.760+0000 7f48db8362c0 1 -- 172.21.15.16:0/1230635765 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560956ecbd40 con 0x560956ea7800 2023-05-20T01:18:29.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.760+0000 7f48db8362c0 1 -- 172.21.15.16:0/1230635765 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x560956ecb6c0 con 0x560956ea7800 2023-05-20T01:18:29.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.760+0000 7f48db8362c0 4 waiting for MDS map... 2023-05-20T01:18:29.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.760+0000 7f48d6fdd700 1 -- 172.21.15.16:0/1230635765 <== mon.0 v2:172.21.15.16:3300/0 4 ==== osd_map(47..47 src has 1..47) v4 ==== 7016+0+0 (secure 0 0 0) 0x560956ee8c40 con 0x560956ea7800 2023-05-20T01:18:29.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.760+0000 7f48d6fdd700 1 -- 172.21.15.16:0/1230635765 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 22) v1 ==== 3126+0+0 (secure 0 0 0) 0x560956eecf00 con 0x560956ea7800 2023-05-20T01:18:29.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.760+0000 7f48db8362c0 4 Got MDS map 22 2023-05-20T01:18:29.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.760+0000 7f48db8362c0 10 main: main 2023-05-20T01:18:29.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.760+0000 7f48db8362c0 4 main: connecting to RADOS... 2023-05-20T01:18:29.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.761+0000 7f48db8362c0 1 Processor -- start 2023-05-20T01:18:29.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.761+0000 7f48db8362c0 1 -- start start 2023-05-20T01:18:29.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.761+0000 7f48db8362c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560957d3c400 0x560956f41600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:29.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.761+0000 7f48db8362c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560957d3c800 0x560956f40580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:29.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.761+0000 7f48db8362c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x560957d3cc00 0x560956f42100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:29.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.761+0000 7f48db8362c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x560956ee2c80 con 0x560957d3c800 2023-05-20T01:18:29.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.761+0000 7f48db8362c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x560956ee2b40 con 0x560957d3c400 2023-05-20T01:18:29.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.761+0000 7f48db8362c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x560956ee2000 con 0x560957d3cc00 2023-05-20T01:18:29.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.762+0000 7f48d97e2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560957d3c800 0x560956f40580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:29.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.762+0000 7f48d97e2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560957d3c800 0x560956f40580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60842/0 (socket says 172.21.15.16:60842) 2023-05-20T01:18:29.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.762+0000 7f48d97e2700 1 -- 172.21.15.16:0/919807921 learned_addr learned my addr 172.21.15.16:0/919807921 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:18:29.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.762+0000 7f48d9fe3700 1 --2- 172.21.15.16:0/919807921 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560957d3c400 0x560956f41600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:29.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.762+0000 7f48d8fe1700 1 --2- 172.21.15.16:0/919807921 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x560957d3cc00 0x560956f42100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:29.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.762+0000 7f48d97e2700 1 -- 172.21.15.16:0/919807921 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x560957d3cc00 msgr2=0x560956f42100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:18:29.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.762+0000 7f48d97e2700 1 --2- 172.21.15.16:0/919807921 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x560957d3cc00 0x560956f42100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:29.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.762+0000 7f48d97e2700 1 -- 172.21.15.16:0/919807921 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560957d3c400 msgr2=0x560956f41600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:18:29.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.762+0000 7f48d97e2700 1 --2- 172.21.15.16:0/919807921 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560957d3c400 0x560956f41600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:29.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.762+0000 7f48d97e2700 1 -- 172.21.15.16:0/919807921 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560956ecb1e0 con 0x560957d3c800 2023-05-20T01:18:29.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.762+0000 7f48d9fe3700 1 --2- 172.21.15.16:0/919807921 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560957d3c400 0x560956f41600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:18:29.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.762+0000 7f48d97e2700 1 --2- 172.21.15.16:0/919807921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560957d3c800 0x560956f40580 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x560957c83560 tx=0x560957c72060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe39500e508c6699 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:29.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.762+0000 7f48d4fd9700 1 -- 172.21.15.16:0/919807921 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x560956edb1e0 con 0x560957d3c800 2023-05-20T01:18:29.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.762+0000 7f48d4fd9700 1 -- 172.21.15.16:0/919807921 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56095718c780 con 0x560957d3c800 2023-05-20T01:18:29.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.762+0000 7f48d4fd9700 1 -- 172.21.15.16:0/919807921 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x560956edb080 con 0x560957d3c800 2023-05-20T01:18:29.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.763+0000 7f48db8362c0 1 -- 172.21.15.16:0/919807921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560957d3c800 msgr2=0x560956f40580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:29.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.763+0000 7f48db8362c0 1 --2- 172.21.15.16:0/919807921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560957d3c800 0x560956f40580 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x560957c83560 tx=0x560957c72060 comp rx=0 tx=0).stop 2023-05-20T01:18:29.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.763+0000 7f48db8362c0 1 -- 172.21.15.16:0/919807921 shutdown_connections 2023-05-20T01:18:29.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.763+0000 7f48db8362c0 1 --2- 172.21.15.16:0/919807921 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x560957d3cc00 0x560956f42100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:29.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.763+0000 7f48db8362c0 1 --2- 172.21.15.16:0/919807921 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560957d3c400 0x560956f41600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:29.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.763+0000 7f48db8362c0 1 --2- 172.21.15.16:0/919807921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560957d3c800 0x560956f40580 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:29.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.763+0000 7f48db8362c0 1 -- 172.21.15.16:0/919807921 >> 172.21.15.16:0/919807921 conn(0x560956ea7000 msgr2=0x560956f77000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:18:29.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.763+0000 7f48db8362c0 1 -- 172.21.15.16:0/919807921 shutdown_connections 2023-05-20T01:18:29.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.763+0000 7f48db8362c0 1 -- 172.21.15.16:0/919807921 wait complete. 2023-05-20T01:18:29.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.763+0000 7f48db8362c0 1 Processor -- start 2023-05-20T01:18:29.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.763+0000 7f48db8362c0 1 -- start start 2023-05-20T01:18:29.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.764+0000 7f48db8362c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560957d3c400 0x560956f43180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:29.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.764+0000 7f48db8362c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x560957d3c800 0x560956f42c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:29.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.764+0000 7f48db8362c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560957d3cc00 0x560956f42680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:29.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.764+0000 7f48db8362c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x560956ee3400 con 0x560957d3cc00 2023-05-20T01:18:29.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.764+0000 7f48db8362c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x560956ee3cc0 con 0x560957d3c400 2023-05-20T01:18:29.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.764+0000 7f48db8362c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x560956ee2a00 con 0x560957d3c800 2023-05-20T01:18:29.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.764+0000 7f48d8fe1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560957d3cc00 0x560956f42680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:29.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.764+0000 7f48d8fe1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560957d3cc00 0x560956f42680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60848/0 (socket says 172.21.15.16:60848) 2023-05-20T01:18:29.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.764+0000 7f48d8fe1700 1 -- 172.21.15.16:0/3774110809 learned_addr learned my addr 172.21.15.16:0/3774110809 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:18:29.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.764+0000 7f48d97e2700 1 --2- 172.21.15.16:0/3774110809 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x560957d3c800 0x560956f42c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:29.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.764+0000 7f48d9fe3700 1 --2- 172.21.15.16:0/3774110809 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560957d3c400 0x560956f43180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:29.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.764+0000 7f48d8fe1700 1 -- 172.21.15.16:0/3774110809 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x560957d3c800 msgr2=0x560956f42c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:29.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.764+0000 7f48d8fe1700 1 --2- 172.21.15.16:0/3774110809 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x560957d3c800 0x560956f42c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:29.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.764+0000 7f48d8fe1700 1 -- 172.21.15.16:0/3774110809 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560957d3c400 msgr2=0x560956f43180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:29.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.764+0000 7f48d8fe1700 1 --2- 172.21.15.16:0/3774110809 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560957d3c400 0x560956f43180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:29.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.764+0000 7f48d8fe1700 1 -- 172.21.15.16:0/3774110809 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560956ecb520 con 0x560957d3cc00 2023-05-20T01:18:29.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.764+0000 7f48d8fe1700 1 --2- 172.21.15.16:0/3774110809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560957d3cc00 0x560956f42680 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x5609571e86f0 tx=0x560957190e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:29.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.764+0000 7f48d37d6700 1 -- 172.21.15.16:0/3774110809 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x560956edb1e0 con 0x560957d3cc00 2023-05-20T01:18:29.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.765+0000 7f48d37d6700 1 -- 172.21.15.16:0/3774110809 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560956ef8d80 con 0x560957d3cc00 2023-05-20T01:18:29.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.765+0000 7f48db8362c0 1 -- 172.21.15.16:0/3774110809 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x560956ecad00 con 0x560957d3cc00 2023-05-20T01:18:29.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.765+0000 7f48d37d6700 1 -- 172.21.15.16:0/3774110809 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x560956edb080 con 0x560957d3cc00 2023-05-20T01:18:29.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.765+0000 7f48db8362c0 1 -- 172.21.15.16:0/3774110809 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560956eca4e0 con 0x560957d3cc00 2023-05-20T01:18:29.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.765+0000 7f48db8362c0 4 main: resolving pool 6 2023-05-20T01:18:29.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.766+0000 7f48d37d6700 1 -- 172.21.15.16:0/3774110809 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x560956f08840 con 0x560957d3cc00 2023-05-20T01:18:29.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.766+0000 7f48d37d6700 1 --2- 172.21.15.16:0/3774110809 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x560957d3dc00 0x560956f43700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:29.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.766+0000 7f48d37d6700 1 -- 172.21.15.16:0/3774110809 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 7016+0+0 (secure 0 0 0) 0x560956ee8a80 con 0x560957d3cc00 2023-05-20T01:18:29.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.766+0000 7f48db8362c0 4 main: creating IoCtx.. 2023-05-20T01:18:29.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.766+0000 7f48d9fe3700 1 --2- 172.21.15.16:0/3774110809 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x560957d3dc00 0x560956f43700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:29.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.766+0000 7f48db8362c0 1 --2- 172.21.15.16:0/3774110809 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x560957d3d000 0x560956f42100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:29.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.766+0000 7f48db8362c0 1 -- 172.21.15.16:0/3774110809 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:1 6.13 6:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47) v8 -- 0x560957186700 con 0x560957d3d000 2023-05-20T01:18:29.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.767+0000 7f48d97e2700 1 --2- 172.21.15.16:0/3774110809 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x560957d3d000 0x560956f42100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:29.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.767+0000 7f48d9fe3700 1 --2- 172.21.15.16:0/3774110809 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x560957d3dc00 0x560956f43700 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x560957c208a0 tx=0x5609572dc480 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:29.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.767+0000 7f48d97e2700 1 --2- 172.21.15.16:0/3774110809 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x560957d3d000 0x560956f42100 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:29.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.768+0000 7f48d97e2700 1 -- 172.21.15.16:0/3774110809 <== osd.3 v2:172.21.15.16:6800/3669781890 1 ==== osd_op_reply(1 mds0_inotable [writefull 0~34] v47'6 uv6 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x560957c6afc0 con 0x560957d3d000 2023-05-20T01:18:29.781 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-05-20T01:18:29.781 INFO:teuthology.orchestra.run.smithi016.stdout: "0": { 2023-05-20T01:18:29.781 INFO:teuthology.orchestra.run.smithi016.stdout: "data": {}, 2023-05-20T01:18:29.781 INFO:teuthology.orchestra.run.smithi016.stdout: "result": 0 2023-05-20T01:18:29.782 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:18:29.782 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-05-20T01:18:29.782 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:18:29.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.768+0000 7f48db8362c0 1 -- 172.21.15.16:0/3774110809 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x560957d3d000 msgr2=0x560956f42100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:29.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.768+0000 7f48db8362c0 1 --2- 172.21.15.16:0/3774110809 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x560957d3d000 0x560956f42100 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:29.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.768+0000 7f48db8362c0 1 -- 172.21.15.16:0/3774110809 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x560957d3dc00 msgr2=0x560956f43700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:29.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.768+0000 7f48db8362c0 1 --2- 172.21.15.16:0/3774110809 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x560957d3dc00 0x560956f43700 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x560957c208a0 tx=0x5609572dc480 comp rx=0 tx=0).stop 2023-05-20T01:18:29.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.768+0000 7f48db8362c0 1 -- 172.21.15.16:0/3774110809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560957d3cc00 msgr2=0x560956f42680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:29.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.768+0000 7f48db8362c0 1 --2- 172.21.15.16:0/3774110809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560957d3cc00 0x560956f42680 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x5609571e86f0 tx=0x560957190e40 comp rx=0 tx=0).stop 2023-05-20T01:18:29.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.768+0000 7f48d8fe1700 1 -- 172.21.15.16:0/3774110809 reap_dead start 2023-05-20T01:18:29.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.768+0000 7f48db8362c0 1 -- 172.21.15.16:0/3774110809 shutdown_connections 2023-05-20T01:18:29.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.768+0000 7f48db8362c0 1 -- 172.21.15.16:0/3774110809 >> 172.21.15.16:0/3774110809 conn(0x560956ea7000 msgr2=0x560956f77000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:18:29.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.769+0000 7f48db8362c0 1 -- 172.21.15.16:0/3774110809 shutdown_connections 2023-05-20T01:18:29.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.769+0000 7f48db8362c0 1 -- 172.21.15.16:0/3774110809 wait complete. 2023-05-20T01:18:29.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.769+0000 7f48db8362c0 1 -- 172.21.15.16:0/1230635765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560956ea7800 msgr2=0x560956f41080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:29.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.769+0000 7f48db8362c0 1 --2- 172.21.15.16:0/1230635765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560956ea7800 0x560956f41080 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x560957c82690 tx=0x560957c72000 comp rx=0 tx=0).stop 2023-05-20T01:18:29.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.769+0000 7f48db8362c0 1 -- 172.21.15.16:0/1230635765 shutdown_connections 2023-05-20T01:18:29.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.769+0000 7f48db8362c0 1 --2- 172.21.15.16:0/1230635765 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560956ea7c00 0x560956f41b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:29.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.769+0000 7f48db8362c0 1 --2- 172.21.15.16:0/1230635765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560956ea7800 0x560956f41080 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:29.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.769+0000 7f48db8362c0 1 --2- 172.21.15.16:0/1230635765 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x560956ea7400 0x560956f40b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:29.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.769+0000 7f48db8362c0 1 -- 172.21.15.16:0/1230635765 >> 172.21.15.16:0/1230635765 conn(0x560956ea6400 msgr2=0x560956f78000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:18:29.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.769+0000 7f48db8362c0 1 -- 172.21.15.16:0/1230635765 shutdown_connections 2023-05-20T01:18:29.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.769+0000 7f48db8362c0 1 -- 172.21.15.16:0/1230635765 wait complete. 2023-05-20T01:18:29.786 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-table-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'cephfs:0', 'reset', 'inode'] in time 0:00:00.102982, result: { "0": { "data": {}, "result": 0 } } 2023-05-20T01:18:29.787 DEBUG:teuthology.orchestra.run.smithi016:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 journal reset --force 2023-05-20T01:18:29.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.864+0000 7fdd918c12c0 1 -- start start 2023-05-20T01:18:29.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.864+0000 7fdd918c12c0 1 Processor -- start 2023-05-20T01:18:29.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.865+0000 7fdd918c12c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556c36447000 0x556c364e0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:29.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.865+0000 7fdd918c12c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c36446c00 0x556c364e0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:29.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.865+0000 7fdd918c12c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556c36446800 0x556c364e1080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:29.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.865+0000 7fdd918c12c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c36483cc0 con 0x556c36446c00 2023-05-20T01:18:29.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.865+0000 7fdd918c12c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c36483400 con 0x556c36446800 2023-05-20T01:18:29.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.865+0000 7fdd918c12c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x556c36483e00 con 0x556c36447000 2023-05-20T01:18:29.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.865+0000 7fdd8f06c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c36446c00 0x556c364e0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:29.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.865+0000 7fdd8f06c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c36446c00 0x556c364e0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60864/0 (socket says 172.21.15.16:60864) 2023-05-20T01:18:29.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.865+0000 7fdd8f06c700 1 -- 172.21.15.16:0/1463667183 learned_addr learned my addr 172.21.15.16:0/1463667183 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:18:29.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.865+0000 7fdd9006e700 1 --2- 172.21.15.16:0/1463667183 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556c36446800 0x556c364e1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:29.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.865+0000 7fdd8f86d700 1 --2- 172.21.15.16:0/1463667183 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556c36447000 0x556c364e0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:29.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.866+0000 7fdd8f06c700 1 -- 172.21.15.16:0/1463667183 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556c36447000 msgr2=0x556c364e0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:18:29.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.866+0000 7fdd8f06c700 1 --2- 172.21.15.16:0/1463667183 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556c36447000 0x556c364e0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:29.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.866+0000 7fdd8f06c700 1 -- 172.21.15.16:0/1463667183 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556c36446800 msgr2=0x556c364e1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:18:29.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.866+0000 7fdd8f06c700 1 --2- 172.21.15.16:0/1463667183 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556c36446800 0x556c364e1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:29.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.866+0000 7fdd8f06c700 1 -- 172.21.15.16:0/1463667183 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556c3646a4e0 con 0x556c36446c00 2023-05-20T01:18:29.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.866+0000 7fdd8f06c700 1 --2- 172.21.15.16:0/1463667183 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c36446c00 0x556c364e0b00 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x556c372b46f0 tx=0x556c37220f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceeffd8f2022dd17 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:29.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.866+0000 7fdd8d068700 1 -- 172.21.15.16:0/1463667183 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x556c3647b600 con 0x556c36446c00 2023-05-20T01:18:29.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.866+0000 7fdd8d068700 1 -- 172.21.15.16:0/1463667183 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556c36498480 con 0x556c36446c00 2023-05-20T01:18:29.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.866+0000 7fdd8d068700 1 -- 172.21.15.16:0/1463667183 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x556c3647b1e0 con 0x556c36446c00 2023-05-20T01:18:29.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.866+0000 7fdd918c12c0 1 -- 172.21.15.16:0/1463667183 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556c3646b520 con 0x556c36446c00 2023-05-20T01:18:29.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.866+0000 7fdd918c12c0 1 -- 172.21.15.16:0/1463667183 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x556c3646b380 con 0x556c36446c00 2023-05-20T01:18:29.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.866+0000 7fdd918c12c0 4 waiting for MDS map... 2023-05-20T01:18:29.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.866+0000 7fdd8d068700 1 -- 172.21.15.16:0/1463667183 <== mon.0 v2:172.21.15.16:3300/0 4 ==== osd_map(47..47 src has 1..47) v4 ==== 7016+0+0 (secure 0 0 0) 0x556c36488c40 con 0x556c36446c00 2023-05-20T01:18:29.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.866+0000 7fdd8d068700 1 -- 172.21.15.16:0/1463667183 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 22) v1 ==== 3126+0+0 (secure 0 0 0) 0x556c37216000 con 0x556c36446c00 2023-05-20T01:18:29.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.866+0000 7fdd918c12c0 4 Got MDS map 22 2023-05-20T01:18:29.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.866+0000 7fdd918c12c0 10 main: JournalTool::main 2023-05-20T01:18:29.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.866+0000 7fdd918c12c0 4 main: JournalTool: connecting to RADOS... 2023-05-20T01:18:29.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.867+0000 7fdd918c12c0 1 Processor -- start 2023-05-20T01:18:29.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.867+0000 7fdd918c12c0 1 -- start start 2023-05-20T01:18:29.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.867+0000 7fdd918c12c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556c36447c00 0x556c364e1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:29.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.867+0000 7fdd918c12c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c36762000 0x556c364e1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:29.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.867+0000 7fdd918c12c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556c36762400 0x556c364e2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:29.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.867+0000 7fdd918c12c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c36482a00 con 0x556c36762000 2023-05-20T01:18:29.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.867+0000 7fdd918c12c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c36483400 con 0x556c36762400 2023-05-20T01:18:29.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.867+0000 7fdd918c12c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x556c36482b40 con 0x556c36447c00 2023-05-20T01:18:29.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.868+0000 7fdd8f86d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c36762000 0x556c364e1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:29.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.868+0000 7fdd8f86d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c36762000 0x556c364e1b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60870/0 (socket says 172.21.15.16:60870) 2023-05-20T01:18:29.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.868+0000 7fdd8f86d700 1 -- 172.21.15.16:0/3953428157 learned_addr learned my addr 172.21.15.16:0/3953428157 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:18:29.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.868+0000 7fdd8f06c700 1 --2- 172.21.15.16:0/3953428157 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556c36762400 0x556c364e2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:29.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.868+0000 7fdd9006e700 1 --2- 172.21.15.16:0/3953428157 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556c36447c00 0x556c364e1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:29.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.868+0000 7fdd8f86d700 1 -- 172.21.15.16:0/3953428157 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556c36447c00 msgr2=0x556c364e1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:18:29.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.868+0000 7fdd8f86d700 1 --2- 172.21.15.16:0/3953428157 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556c36447c00 0x556c364e1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:29.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.868+0000 7fdd8f86d700 1 -- 172.21.15.16:0/3953428157 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556c36762400 msgr2=0x556c364e2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:18:29.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.868+0000 7fdd8f86d700 1 --2- 172.21.15.16:0/3953428157 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556c36762400 0x556c364e2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:29.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.868+0000 7fdd8f86d700 1 -- 172.21.15.16:0/3953428157 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556c3646bd40 con 0x556c36762000 2023-05-20T01:18:29.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.868+0000 7fdd8f86d700 1 --2- 172.21.15.16:0/3953428157 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c36762000 0x556c364e1b80 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x556c372b5c80 tx=0x556c372db800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dbd32b109a2f2f9 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:29.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.868+0000 7fdd8b064700 1 -- 172.21.15.16:0/3953428157 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x556c3647b600 con 0x556c36762000 2023-05-20T01:18:29.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.868+0000 7fdd8b064700 1 -- 172.21.15.16:0/3953428157 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556c36499e00 con 0x556c36762000 2023-05-20T01:18:29.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.868+0000 7fdd8b064700 1 -- 172.21.15.16:0/3953428157 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x556c3647b1e0 con 0x556c36762000 2023-05-20T01:18:29.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.869+0000 7fdd918c12c0 1 -- 172.21.15.16:0/3953428157 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c36762000 msgr2=0x556c364e1b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:29.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.869+0000 7fdd918c12c0 1 --2- 172.21.15.16:0/3953428157 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c36762000 0x556c364e1b80 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x556c372b5c80 tx=0x556c372db800 comp rx=0 tx=0).stop 2023-05-20T01:18:29.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.869+0000 7fdd918c12c0 1 -- 172.21.15.16:0/3953428157 shutdown_connections 2023-05-20T01:18:29.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.869+0000 7fdd918c12c0 1 --2- 172.21.15.16:0/3953428157 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556c36762400 0x556c364e2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:29.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.869+0000 7fdd918c12c0 1 --2- 172.21.15.16:0/3953428157 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556c36447c00 0x556c364e1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:29.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.869+0000 7fdd918c12c0 1 --2- 172.21.15.16:0/3953428157 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c36762000 0x556c364e1b80 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:29.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.869+0000 7fdd918c12c0 1 -- 172.21.15.16:0/3953428157 >> 172.21.15.16:0/3953428157 conn(0x556c36446400 msgr2=0x556c36523800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:18:29.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.869+0000 7fdd918c12c0 1 -- 172.21.15.16:0/3953428157 shutdown_connections 2023-05-20T01:18:29.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.869+0000 7fdd918c12c0 1 -- 172.21.15.16:0/3953428157 wait complete. 2023-05-20T01:18:29.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.869+0000 7fdd918c12c0 1 Processor -- start 2023-05-20T01:18:29.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.869+0000 7fdd918c12c0 1 -- start start 2023-05-20T01:18:29.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.870+0000 7fdd918c12c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c36446400 0x556c364e3180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:29.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.870+0000 7fdd918c12c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556c36780000 0x556c364e2c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:29.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.870+0000 7fdd918c12c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556c36780400 0x556c364e2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:29.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.870+0000 7fdd918c12c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c36482c80 con 0x556c36446400 2023-05-20T01:18:29.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.870+0000 7fdd918c12c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c36482000 con 0x556c36780000 2023-05-20T01:18:29.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.870+0000 7fdd918c12c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x556c36483180 con 0x556c36780400 2023-05-20T01:18:29.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.870+0000 7fdd9006e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c36446400 0x556c364e3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:29.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.870+0000 7fdd9006e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c36446400 0x556c364e3180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60874/0 (socket says 172.21.15.16:60874) 2023-05-20T01:18:29.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.870+0000 7fdd9006e700 1 -- 172.21.15.16:0/2695556387 learned_addr learned my addr 172.21.15.16:0/2695556387 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:18:29.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.870+0000 7fdd8f06c700 1 --2- 172.21.15.16:0/2695556387 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556c36780400 0x556c364e2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:29.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.870+0000 7fdd8f86d700 1 --2- 172.21.15.16:0/2695556387 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556c36780000 0x556c364e2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:29.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.870+0000 7fdd9006e700 1 -- 172.21.15.16:0/2695556387 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556c36780400 msgr2=0x556c364e2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:29.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.870+0000 7fdd9006e700 1 --2- 172.21.15.16:0/2695556387 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556c36780400 0x556c364e2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:29.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.870+0000 7fdd9006e700 1 -- 172.21.15.16:0/2695556387 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556c36780000 msgr2=0x556c364e2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:29.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.870+0000 7fdd9006e700 1 --2- 172.21.15.16:0/2695556387 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556c36780000 0x556c364e2c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:29.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.870+0000 7fdd9006e700 1 -- 172.21.15.16:0/2695556387 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556c367721a0 con 0x556c36446400 2023-05-20T01:18:29.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.870+0000 7fdd9006e700 1 --2- 172.21.15.16:0/2695556387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c36446400 0x556c364e3180 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x556c36762c90 tx=0x556c37220180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:29.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.870+0000 7fdd89861700 1 -- 172.21.15.16:0/2695556387 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x556c3647b080 con 0x556c36446400 2023-05-20T01:18:29.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.870+0000 7fdd89861700 1 -- 172.21.15.16:0/2695556387 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556c36499800 con 0x556c36446400 2023-05-20T01:18:29.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.870+0000 7fdd89861700 1 -- 172.21.15.16:0/2695556387 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x556c3647b600 con 0x556c36446400 2023-05-20T01:18:29.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.870+0000 7fdd918c12c0 1 -- 172.21.15.16:0/2695556387 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x556c36772000 con 0x556c36446400 2023-05-20T01:18:29.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.871+0000 7fdd918c12c0 1 -- 172.21.15.16:0/2695556387 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556c36772340 con 0x556c36446400 2023-05-20T01:18:29.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.871+0000 7fdd918c12c0 4 main: JournalTool: resolving pool 6 2023-05-20T01:18:29.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.872+0000 7fdd89861700 1 -- 172.21.15.16:0/2695556387 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x556c364a8840 con 0x556c36446400 2023-05-20T01:18:29.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.872+0000 7fdd89861700 1 --2- 172.21.15.16:0/2695556387 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x556c36782400 0x556c364e3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:29.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.872+0000 7fdd89861700 1 -- 172.21.15.16:0/2695556387 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 7016+0+0 (secure 0 0 0) 0x556c36488a80 con 0x556c36446400 2023-05-20T01:18:29.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.872+0000 7fdd8f86d700 1 --2- 172.21.15.16:0/2695556387 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x556c36782400 0x556c364e3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:29.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.872+0000 7fdd918c12c0 4 main: JournalTool: creating IoCtx.. 2023-05-20T01:18:29.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.872+0000 7fdd918c12c0 4 main: Executing for rank 0 2023-05-20T01:18:29.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.872+0000 7fdd8f86d700 1 --2- 172.21.15.16:0/2695556387 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x556c36782400 0x556c364e3700 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x556c372b5c80 tx=0x556c37220f00 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:29.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.873+0000 7fdd918c12c0 1 -- start start 2023-05-20T01:18:29.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.873+0000 7fdd918c12c0 1 Processor -- start 2023-05-20T01:18:29.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.873+0000 7fdd918c12c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c36780c00 0x556c364e2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:29.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.873+0000 7fdd918c12c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556c36781000 0x556c364e1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:29.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.873+0000 7fdd918c12c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556c36781400 0x556c364e1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:29.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.873+0000 7fdd918c12c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c36482000 con 0x556c36780c00 2023-05-20T01:18:29.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.874+0000 7fdd918c12c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c36482c80 con 0x556c36781000 2023-05-20T01:18:29.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.874+0000 7fdd918c12c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x556c36482b40 con 0x556c36781400 2023-05-20T01:18:29.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.874+0000 7fdd9006e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c36780c00 0x556c364e2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:29.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.874+0000 7fdd9006e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c36780c00 0x556c364e2100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60882/0 (socket says 172.21.15.16:60882) 2023-05-20T01:18:29.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.874+0000 7fdd9006e700 1 -- 172.21.15.16:0/2061444390 learned_addr learned my addr 172.21.15.16:0/2061444390 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:18:29.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.874+0000 7fdd8f86d700 1 --2- 172.21.15.16:0/2061444390 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556c36781000 0x556c364e1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:29.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.874+0000 7fdd8f06c700 1 --2- 172.21.15.16:0/2061444390 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556c36781400 0x556c364e1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:29.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.874+0000 7fdd9006e700 1 -- 172.21.15.16:0/2061444390 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556c36781400 msgr2=0x556c364e1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:18:29.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.874+0000 7fdd9006e700 1 --2- 172.21.15.16:0/2061444390 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556c36781400 0x556c364e1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:29.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.874+0000 7fdd9006e700 1 -- 172.21.15.16:0/2061444390 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556c36781000 msgr2=0x556c364e1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:18:29.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.874+0000 7fdd9006e700 1 --2- 172.21.15.16:0/2061444390 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556c36781000 0x556c364e1b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:29.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.874+0000 7fdd9006e700 1 -- 172.21.15.16:0/2061444390 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556c36772680 con 0x556c36780c00 2023-05-20T01:18:29.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.874+0000 7fdd8f86d700 1 --2- 172.21.15.16:0/2061444390 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556c36781000 0x556c364e1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:18:29.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.874+0000 7fdd8f06c700 1 --2- 172.21.15.16:0/2061444390 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556c36781400 0x556c364e1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:18:29.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.874+0000 7fdd9006e700 1 --2- 172.21.15.16:0/2061444390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c36780c00 0x556c364e2100 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x556c368509f0 tx=0x556c368d8000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70096e87364ac330 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:29.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.874+0000 7fdd8685b700 1 -- 172.21.15.16:0/2061444390 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x556c3647b1e0 con 0x556c36780c00 2023-05-20T01:18:29.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.874+0000 7fdd8685b700 1 -- 172.21.15.16:0/2061444390 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556c372d6c00 con 0x556c36780c00 2023-05-20T01:18:29.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.874+0000 7fdd8685b700 1 -- 172.21.15.16:0/2061444390 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x556c3647a420 con 0x556c36780c00 2023-05-20T01:18:29.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.874+0000 7fdd918c12c0 1 -- 172.21.15.16:0/2061444390 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556c36772820 con 0x556c36780c00 2023-05-20T01:18:29.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.874+0000 7fdd918c12c0 1 -- 172.21.15.16:0/2061444390 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x556c367729c0 con 0x556c36780c00 2023-05-20T01:18:29.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.874+0000 7fdd918c12c0 4 waiting for MDS map... 2023-05-20T01:18:29.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.874+0000 7fdd8685b700 1 -- 172.21.15.16:0/2061444390 <== mon.0 v2:172.21.15.16:3300/0 4 ==== osd_map(47..47 src has 1..47) v4 ==== 7016+0+0 (secure 0 0 0) 0x556c364888c0 con 0x556c36780c00 2023-05-20T01:18:29.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.875+0000 7fdd8685b700 1 -- 172.21.15.16:0/2061444390 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 22) v1 ==== 3126+0+0 (secure 0 0 0) 0x556c368bef00 con 0x556c36780c00 2023-05-20T01:18:29.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.875+0000 7fdd918c12c0 4 Got MDS map 22 2023-05-20T01:18:29.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.875+0000 7fdd918c12c0 1 --2- 172.21.15.16:0/2061444390 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x556c36781800 0x556c368ee000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:29.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.875+0000 7fdd918c12c0 1 -- 172.21.15.16:0/2061444390 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:1 6.f 6:f1f69726:::400.00000000:head [writefull 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47) v8 -- 0x556c372b6e00 con 0x556c36781800 2023-05-20T01:18:29.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.875+0000 7fdd8f86d700 1 --2- 172.21.15.16:0/2061444390 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x556c36781800 0x556c368ee000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:29.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.875+0000 7fdd8f86d700 1 --2- 172.21.15.16:0/2061444390 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x556c36781800 0x556c368ee000 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=44ac72db18a09158 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:29.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.876+0000 7fdd8f86d700 1 -- 172.21.15.16:0/2061444390 <== osd.2 v2:172.21.15.16:6816/293187030 1 ==== osd_op_reply(1 400.00000000 [writefull 0~22] v47'4 uv4 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x556c372b3200 con 0x556c36781800 2023-05-20T01:18:29.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.876+0000 7fdd918c12c0 1 -- 172.21.15.16:0/2061444390 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:2 6.14 6:292cf221:::200.00000000:head [writefull 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47) v8 -- 0x556c372b7180 con 0x556c36781800 2023-05-20T01:18:29.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.878+0000 7fdd8f86d700 1 -- 172.21.15.16:0/2061444390 <== osd.2 v2:172.21.15.16:6816/293187030 2 ==== osd_op_reply(2 200.00000000 [writefull 0~90] v47'46 uv46 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x556c372b3200 con 0x556c36781800 2023-05-20T01:18:29.893 INFO:teuthology.orchestra.run.smithi016.stdout:writing EResetJournal entry 2023-05-20T01:18:29.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.878+0000 7fdd918c12c0 1 -- 172.21.15.16:0/2061444390 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:3 6.14 6:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47) v8 -- 0x556c372b7500 con 0x556c36781800 2023-05-20T01:18:29.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.878+0000 7fdd918c12c0 1 --2- 172.21.15.16:0/2061444390 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x556c36781c00 0x556c368ee580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:29.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.878+0000 7fdd918c12c0 1 -- 172.21.15.16:0/2061444390 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:4 6.1b 6:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47) v8 -- 0x556c372b6e00 con 0x556c36781c00 2023-05-20T01:18:29.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.878+0000 7fdd918c12c0 1 -- 172.21.15.16:0/2061444390 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:5 6.b 6:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47) v8 -- 0x556c372b7180 con 0x556c36781c00 2023-05-20T01:18:29.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.878+0000 7fdd918c12c0 1 -- 172.21.15.16:0/2061444390 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:6 6.1f 6:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47) v8 -- 0x556c372b7880 con 0x556c36781c00 2023-05-20T01:18:29.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.878+0000 7fdd918c12c0 1 --2- 172.21.15.16:0/2061444390 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x556c36783000 0x556c368eeb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:29.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.878+0000 7fdd918c12c0 1 -- 172.21.15.16:0/2061444390 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:7 6.1a 6:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47) v8 -- 0x556c372b7c00 con 0x556c36783000 2023-05-20T01:18:29.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.878+0000 7fdd918c12c0 1 -- 172.21.15.16:0/2061444390 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:8 6.1e 6:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47) v8 -- 0x556c368fe000 con 0x556c36783000 2023-05-20T01:18:29.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.878+0000 7fdd8f06c700 1 --2- 172.21.15.16:0/2061444390 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x556c36781c00 0x556c368ee580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:29.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.879+0000 7fdd8f06c700 1 --2- 172.21.15.16:0/2061444390 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x556c36781c00 0x556c368ee580 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=d835b805fcb5b200 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:29.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.879+0000 7fdd9006e700 1 --2- 172.21.15.16:0/2061444390 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x556c36783000 0x556c368eeb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:29.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.879+0000 7fdd9006e700 1 --2- 172.21.15.16:0/2061444390 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x556c36783000 0x556c368eeb00 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=f7356c344bd3f463 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:29.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.880+0000 7fdd8f86d700 1 -- 172.21.15.16:0/2061444390 <== osd.2 v2:172.21.15.16:6816/293187030 3 ==== osd_op_reply(3 200.00000001 [delete] v47'47 uv46 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x556c372b3200 con 0x556c36781800 2023-05-20T01:18:29.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.880+0000 7fdd8f06c700 1 -- 172.21.15.16:0/2061444390 <== osd.3 v2:172.21.15.16:6800/3669781890 1 ==== osd_op_reply(4 200.00000002 [delete] v47'2 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x556c372b38c0 con 0x556c36781c00 2023-05-20T01:18:29.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.880+0000 7fdd85058700 1 -- 172.21.15.16:0/2061444390 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:9 6.14 6:2e2fa760:::200.00000001:head [write 0~48 in=48b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47) v8 -- 0x556c372b6e00 con 0x556c36781800 2023-05-20T01:18:29.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.880+0000 7fdd8f06c700 1 -- 172.21.15.16:0/2061444390 <== osd.3 v2:172.21.15.16:6800/3669781890 2 ==== osd_op_reply(6 200.00000004 [delete] v47'9 uv8 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x556c372b38c0 con 0x556c36781c00 2023-05-20T01:18:29.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.881+0000 7fdd8f06c700 1 -- 172.21.15.16:0/2061444390 <== osd.3 v2:172.21.15.16:6800/3669781890 3 ==== osd_op_reply(5 200.00000003 [delete] v47'6 uv5 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x556c372b38c0 con 0x556c36781c00 2023-05-20T01:18:29.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.881+0000 7fdd9006e700 1 -- 172.21.15.16:0/2061444390 <== osd.4 v2:172.21.15.132:6817/506317223 1 ==== osd_op_reply(7 200.00000005 [delete] v47'2 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x556c36908000 con 0x556c36783000 2023-05-20T01:18:29.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.881+0000 7fdd9006e700 1 -- 172.21.15.16:0/2061444390 <== osd.4 v2:172.21.15.132:6817/506317223 2 ==== osd_op_reply(8 200.00000006 [delete] v47'2 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x556c36908000 con 0x556c36783000 2023-05-20T01:18:29.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.882+0000 7fdd8f86d700 1 -- 172.21.15.16:0/2061444390 <== osd.2 v2:172.21.15.16:6816/293187030 4 ==== osd_op_reply(9 200.00000001 [write 0~48] v47'48 uv48 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x556c372b3200 con 0x556c36781800 2023-05-20T01:18:29.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.882+0000 7fdd918c12c0 4 Successfully wrote new journal pointer and header for rank 2:0 2023-05-20T01:18:29.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.882+0000 7fdd918c12c0 1 -- 172.21.15.16:0/2061444390 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x556c36781800 msgr2=0x556c368ee000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:29.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.882+0000 7fdd918c12c0 1 --2- 172.21.15.16:0/2061444390 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x556c36781800 0x556c368ee000 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:29.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.882+0000 7fdd918c12c0 1 -- 172.21.15.16:0/2061444390 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x556c36781c00 msgr2=0x556c368ee580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:29.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.882+0000 7fdd918c12c0 1 --2- 172.21.15.16:0/2061444390 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x556c36781c00 0x556c368ee580 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:29.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.882+0000 7fdd918c12c0 1 -- 172.21.15.16:0/2061444390 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x556c36783000 msgr2=0x556c368eeb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:29.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.882+0000 7fdd918c12c0 1 --2- 172.21.15.16:0/2061444390 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x556c36783000 0x556c368eeb00 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:29.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.882+0000 7fdd8f06c700 1 -- 172.21.15.16:0/2061444390 reap_dead start 2023-05-20T01:18:29.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.882+0000 7fdd918c12c0 1 -- 172.21.15.16:0/2061444390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c36780c00 msgr2=0x556c364e2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:29.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.882+0000 7fdd918c12c0 1 --2- 172.21.15.16:0/2061444390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c36780c00 0x556c364e2100 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x556c368509f0 tx=0x556c368d8000 comp rx=0 tx=0).stop 2023-05-20T01:18:29.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.883+0000 7fdd918c12c0 1 -- 172.21.15.16:0/2061444390 shutdown_connections 2023-05-20T01:18:29.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.883+0000 7fdd918c12c0 1 --2- 172.21.15.16:0/2061444390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c36780c00 0x556c364e2100 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:29.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.883+0000 7fdd918c12c0 1 -- 172.21.15.16:0/2061444390 >> 172.21.15.16:0/2061444390 conn(0x556c36780800 msgr2=0x556c36524000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:18:29.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.883+0000 7fdd918c12c0 1 -- 172.21.15.16:0/2061444390 shutdown_connections 2023-05-20T01:18:29.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.883+0000 7fdd918c12c0 1 -- 172.21.15.16:0/2061444390 wait complete. 2023-05-20T01:18:29.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.883+0000 7fdd918c12c0 1 -- 172.21.15.16:0/2695556387 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x556c36782400 msgr2=0x556c364e3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:29.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.883+0000 7fdd918c12c0 1 --2- 172.21.15.16:0/2695556387 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x556c36782400 0x556c364e3700 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x556c372b5c80 tx=0x556c37220f00 comp rx=0 tx=0).stop 2023-05-20T01:18:29.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.883+0000 7fdd918c12c0 1 -- 172.21.15.16:0/2695556387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c36446400 msgr2=0x556c364e3180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:29.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.883+0000 7fdd918c12c0 1 --2- 172.21.15.16:0/2695556387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c36446400 0x556c364e3180 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x556c36762c90 tx=0x556c37220180 comp rx=0 tx=0).stop 2023-05-20T01:18:29.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.883+0000 7fdd918c12c0 1 -- 172.21.15.16:0/2695556387 shutdown_connections 2023-05-20T01:18:29.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.883+0000 7fdd918c12c0 1 --2- 172.21.15.16:0/2695556387 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x556c36782400 0x556c364e3700 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:29.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.883+0000 7fdd918c12c0 1 --2- 172.21.15.16:0/2695556387 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556c36780400 0x556c364e2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:29.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.883+0000 7fdd918c12c0 1 --2- 172.21.15.16:0/2695556387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c36446400 0x556c364e3180 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:29.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.883+0000 7fdd918c12c0 1 --2- 172.21.15.16:0/2695556387 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556c36780000 0x556c364e2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:29.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.883+0000 7fdd918c12c0 1 -- 172.21.15.16:0/2695556387 >> 172.21.15.16:0/2695556387 conn(0x556c36447c00 msgr2=0x556c36523800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:18:29.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.883+0000 7fdd918c12c0 1 -- 172.21.15.16:0/2695556387 shutdown_connections 2023-05-20T01:18:29.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.883+0000 7fdd918c12c0 1 -- 172.21.15.16:0/2695556387 wait complete. 2023-05-20T01:18:29.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.883+0000 7fdd918c12c0 1 -- 172.21.15.16:0/1463667183 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c36446c00 msgr2=0x556c364e0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:29.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.883+0000 7fdd918c12c0 1 --2- 172.21.15.16:0/1463667183 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c36446c00 0x556c364e0b00 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x556c372b46f0 tx=0x556c37220f60 comp rx=0 tx=0).stop 2023-05-20T01:18:29.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.883+0000 7fdd918c12c0 1 -- 172.21.15.16:0/1463667183 shutdown_connections 2023-05-20T01:18:29.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.883+0000 7fdd918c12c0 1 --2- 172.21.15.16:0/1463667183 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556c36446800 0x556c364e1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:29.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.883+0000 7fdd918c12c0 1 --2- 172.21.15.16:0/1463667183 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556c36447000 0x556c364e0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:29.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.883+0000 7fdd918c12c0 1 --2- 172.21.15.16:0/1463667183 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c36446c00 0x556c364e0b00 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:29.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.883+0000 7fdd918c12c0 1 -- 172.21.15.16:0/1463667183 >> 172.21.15.16:0/1463667183 conn(0x556c36447400 msgr2=0x556c36523000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:18:29.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.883+0000 7fdd918c12c0 1 -- 172.21.15.16:0/1463667183 shutdown_connections 2023-05-20T01:18:29.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.883+0000 7fdd918c12c0 1 -- 172.21.15.16:0/1463667183 wait complete. 2023-05-20T01:18:29.903 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'journal', 'reset', '--force'] in time 0:00:00.116585, result: writing EResetJournal entry 2023-05-20T01:18:29.904 DEBUG:teuthology.orchestra.run.smithi016:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 init 2023-05-20T01:18:29.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.972+0000 7f18d91bd7c0 1 -- start start 2023-05-20T01:18:29.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.972+0000 7f18d91bd7c0 1 Processor -- start 2023-05-20T01:18:29.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.973+0000 7f18d91bd7c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559132397000 0x559132430580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:29.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.973+0000 7f18d91bd7c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559132396c00 0x559132430b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:29.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.973+0000 7f18d91bd7c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559132396800 0x559132431080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:29.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.973+0000 7f18d91bd7c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591323d3cc0 con 0x559132396c00 2023-05-20T01:18:29.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.973+0000 7f18d91bd7c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591323d3400 con 0x559132397000 2023-05-20T01:18:29.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.973+0000 7f18d91bd7c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x5591323d3e00 con 0x559132396800 2023-05-20T01:18:29.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.973+0000 7f18d696a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559132396c00 0x559132430b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:29.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.973+0000 7f18d696a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559132396c00 0x559132430b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60902/0 (socket says 172.21.15.16:60902) 2023-05-20T01:18:29.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.973+0000 7f18d696a700 1 -- 172.21.15.16:0/3572118577 learned_addr learned my addr 172.21.15.16:0/3572118577 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:18:29.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.973+0000 7f18d796c700 1 --2- 172.21.15.16:0/3572118577 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559132396800 0x559132431080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:29.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.973+0000 7f18d716b700 1 --2- 172.21.15.16:0/3572118577 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559132397000 0x559132430580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:29.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.973+0000 7f18d696a700 1 -- 172.21.15.16:0/3572118577 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559132396800 msgr2=0x559132431080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:18:29.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.973+0000 7f18d696a700 1 --2- 172.21.15.16:0/3572118577 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559132396800 0x559132431080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:29.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.973+0000 7f18d696a700 1 -- 172.21.15.16:0/3572118577 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559132397000 msgr2=0x559132430580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:18:29.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.973+0000 7f18d696a700 1 --2- 172.21.15.16:0/3572118577 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559132397000 0x559132430580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:29.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.973+0000 7f18d696a700 1 -- 172.21.15.16:0/3572118577 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5591323bad00 con 0x559132396c00 2023-05-20T01:18:29.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.973+0000 7f18d696a700 1 --2- 172.21.15.16:0/3572118577 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559132396c00 0x559132430b00 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x5591324f6cf0 tx=0x55913312f3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2ad11ea3635f859 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:29.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.974+0000 7f18d4966700 1 -- 172.21.15.16:0/3572118577 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5591323cb080 con 0x559132396c00 2023-05-20T01:18:29.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.974+0000 7f18d4966700 1 -- 172.21.15.16:0/3572118577 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5591323e9980 con 0x559132396c00 2023-05-20T01:18:29.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.974+0000 7f18d4966700 1 -- 172.21.15.16:0/3572118577 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5591323cb600 con 0x559132396c00 2023-05-20T01:18:29.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.974+0000 7f18d91bd7c0 1 -- 172.21.15.16:0/3572118577 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5591323bb6c0 con 0x559132396c00 2023-05-20T01:18:29.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.974+0000 7f18d91bd7c0 1 -- 172.21.15.16:0/3572118577 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x5591323bb520 con 0x559132396c00 2023-05-20T01:18:29.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.974+0000 7f18d91bd7c0 4 waiting for MDS map... 2023-05-20T01:18:29.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.974+0000 7f18d4966700 1 -- 172.21.15.16:0/3572118577 <== mon.0 v2:172.21.15.16:3300/0 4 ==== osd_map(47..47 src has 1..47) v4 ==== 7016+0+0 (secure 0 0 0) 0x5591323d8c40 con 0x559132396c00 2023-05-20T01:18:29.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.974+0000 7f18d4966700 1 -- 172.21.15.16:0/3572118577 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 22) v1 ==== 3126+0+0 (secure 0 0 0) 0x5591323dcf00 con 0x559132396c00 2023-05-20T01:18:29.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.974+0000 7f18d91bd7c0 4 Got MDS map 22 2023-05-20T01:18:29.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.974+0000 7f18d91bd7c0 4 datascan.main: Using metadata pool output 2023-05-20T01:18:29.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.974+0000 7f18d91bd7c0 4 datascan.main: connecting to RADOS... 2023-05-20T01:18:29.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.974+0000 7f18d91bd7c0 1 Processor -- start 2023-05-20T01:18:29.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.974+0000 7f18d91bd7c0 1 -- start start 2023-05-20T01:18:29.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.975+0000 7f18d91bd7c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559132397c00 0x559132431600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:29.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.975+0000 7f18d91bd7c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5591326ae000 0x559132431b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:29.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.975+0000 7f18d91bd7c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5591326ae400 0x559132432100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:29.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.975+0000 7f18d91bd7c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591323d2a00 con 0x559132397c00 2023-05-20T01:18:29.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.975+0000 7f18d91bd7c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591323d3400 con 0x5591326ae400 2023-05-20T01:18:29.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.975+0000 7f18d91bd7c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x5591323d3cc0 con 0x5591326ae000 2023-05-20T01:18:29.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.975+0000 7f18d796c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559132397c00 0x559132431600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:29.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.975+0000 7f18d796c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559132397c00 0x559132431600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60906/0 (socket says 172.21.15.16:60906) 2023-05-20T01:18:29.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.975+0000 7f18d796c700 1 -- 172.21.15.16:0/3682593400 learned_addr learned my addr 172.21.15.16:0/3682593400 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:18:29.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.975+0000 7f18d696a700 1 --2- 172.21.15.16:0/3682593400 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5591326ae400 0x559132432100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:29.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.975+0000 7f18d716b700 1 --2- 172.21.15.16:0/3682593400 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5591326ae000 0x559132431b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:29.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.975+0000 7f18d796c700 1 -- 172.21.15.16:0/3682593400 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5591326ae000 msgr2=0x559132431b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:18:29.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.975+0000 7f18d796c700 1 --2- 172.21.15.16:0/3682593400 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5591326ae000 0x559132431b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:29.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.975+0000 7f18d796c700 1 -- 172.21.15.16:0/3682593400 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5591326ae400 msgr2=0x559132432100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:18:29.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.975+0000 7f18d796c700 1 --2- 172.21.15.16:0/3682593400 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5591326ae400 0x559132432100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:29.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.975+0000 7f18d796c700 1 -- 172.21.15.16:0/3682593400 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5591323bb380 con 0x559132397c00 2023-05-20T01:18:29.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.975+0000 7f18d696a700 1 --2- 172.21.15.16:0/3682593400 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5591326ae400 0x559132432100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:18:29.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.975+0000 7f18d716b700 1 --2- 172.21.15.16:0/3682593400 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5591326ae000 0x559132431b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:18:29.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.976+0000 7f18d796c700 1 --2- 172.21.15.16:0/3682593400 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559132397c00 0x559132431600 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x5591326c0060 tx=0x559133228c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b296515a8b2eee1 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:29.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.976+0000 7f18d2962700 1 -- 172.21.15.16:0/3682593400 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5591323cb080 con 0x559132397c00 2023-05-20T01:18:29.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.976+0000 7f18d2962700 1 -- 172.21.15.16:0/3682593400 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5591323e9e00 con 0x559132397c00 2023-05-20T01:18:29.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.976+0000 7f18d2962700 1 -- 172.21.15.16:0/3682593400 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5591323cb600 con 0x559132397c00 2023-05-20T01:18:29.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.976+0000 7f18d91bd7c0 1 -- 172.21.15.16:0/3682593400 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559132397c00 msgr2=0x559132431600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:29.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.976+0000 7f18d91bd7c0 1 --2- 172.21.15.16:0/3682593400 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559132397c00 0x559132431600 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x5591326c0060 tx=0x559133228c00 comp rx=0 tx=0).stop 2023-05-20T01:18:29.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.976+0000 7f18d91bd7c0 1 -- 172.21.15.16:0/3682593400 shutdown_connections 2023-05-20T01:18:29.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.976+0000 7f18d91bd7c0 1 --2- 172.21.15.16:0/3682593400 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5591326ae400 0x559132432100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:29.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.976+0000 7f18d91bd7c0 1 --2- 172.21.15.16:0/3682593400 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559132397c00 0x559132431600 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:29.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.976+0000 7f18d91bd7c0 1 --2- 172.21.15.16:0/3682593400 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5591326ae000 0x559132431b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:29.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.976+0000 7f18d91bd7c0 1 -- 172.21.15.16:0/3682593400 >> 172.21.15.16:0/3682593400 conn(0x559132396400 msgr2=0x559132473800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:18:29.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.976+0000 7f18d91bd7c0 1 -- 172.21.15.16:0/3682593400 shutdown_connections 2023-05-20T01:18:29.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.976+0000 7f18d91bd7c0 1 -- 172.21.15.16:0/3682593400 wait complete. 2023-05-20T01:18:29.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.977+0000 7f18d91bd7c0 1 Processor -- start 2023-05-20T01:18:29.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.977+0000 7f18d91bd7c0 1 -- start start 2023-05-20T01:18:29.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.977+0000 7f18d91bd7c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559132396400 0x559132433180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:29.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.977+0000 7f18d91bd7c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5591326ae000 0x559132432c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:29.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.977+0000 7f18d91bd7c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5591326ae400 0x559132432680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:29.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.977+0000 7f18d91bd7c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591323d2b40 con 0x5591326ae400 2023-05-20T01:18:29.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.977+0000 7f18d91bd7c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591323d2c80 con 0x5591326ae000 2023-05-20T01:18:29.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.977+0000 7f18d91bd7c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x5591323d2000 con 0x559132396400 2023-05-20T01:18:29.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.977+0000 7f18d696a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5591326ae400 0x559132432680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:29.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.977+0000 7f18d696a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5591326ae400 0x559132432680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60914/0 (socket says 172.21.15.16:60914) 2023-05-20T01:18:29.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.978+0000 7f18d696a700 1 -- 172.21.15.16:0/1938351120 learned_addr learned my addr 172.21.15.16:0/1938351120 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:18:29.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.978+0000 7f18d796c700 1 --2- 172.21.15.16:0/1938351120 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559132396400 0x559132433180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:29.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.978+0000 7f18d716b700 1 --2- 172.21.15.16:0/1938351120 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5591326ae000 0x559132432c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:29.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.978+0000 7f18d696a700 1 -- 172.21.15.16:0/1938351120 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559132396400 msgr2=0x559132433180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:29.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.978+0000 7f18d696a700 1 --2- 172.21.15.16:0/1938351120 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559132396400 0x559132433180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:29.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.978+0000 7f18d696a700 1 -- 172.21.15.16:0/1938351120 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5591326ae000 msgr2=0x559132432c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:29.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.978+0000 7f18d696a700 1 --2- 172.21.15.16:0/1938351120 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5591326ae000 0x559132432c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:29.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.978+0000 7f18d696a700 1 -- 172.21.15.16:0/1938351120 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5591323bb1e0 con 0x5591326ae400 2023-05-20T01:18:29.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.978+0000 7f18d696a700 1 --2- 172.21.15.16:0/1938351120 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5591326ae400 0x559132432680 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x5591326c1410 tx=0x55913312f2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:29.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.978+0000 7f18d115f700 1 -- 172.21.15.16:0/1938351120 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5591323cb1e0 con 0x5591326ae400 2023-05-20T01:18:29.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.978+0000 7f18d115f700 1 -- 172.21.15.16:0/1938351120 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559133220180 con 0x5591326ae400 2023-05-20T01:18:29.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.978+0000 7f18d115f700 1 -- 172.21.15.16:0/1938351120 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5591323caf20 con 0x5591326ae400 2023-05-20T01:18:29.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.978+0000 7f18d91bd7c0 1 -- 172.21.15.16:0/1938351120 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5591323bbd40 con 0x5591326ae400 2023-05-20T01:18:29.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.978+0000 7f18d91bd7c0 1 -- 172.21.15.16:0/1938351120 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5591323bad00 con 0x5591326ae400 2023-05-20T01:18:29.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.978+0000 7f18d91bd7c0 4 datascan.init: resolving metadata pool 6 2023-05-20T01:18:29.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.979+0000 7f18d115f700 1 -- 172.21.15.16:0/1938351120 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x5591323f8840 con 0x5591326ae400 2023-05-20T01:18:29.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.980+0000 7f18d115f700 1 --2- 172.21.15.16:0/1938351120 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x5591326d0800 0x559132433700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:29.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.980+0000 7f18d115f700 1 -- 172.21.15.16:0/1938351120 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 7016+0+0 (secure 0 0 0) 0x5591323d8a80 con 0x5591326ae400 2023-05-20T01:18:29.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.980+0000 7f18d796c700 1 --2- 172.21.15.16:0/1938351120 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x5591326d0800 0x559132433700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:29.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.980+0000 7f18d91bd7c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-05-20T01:18:29.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.980+0000 7f18d91bd7c0 1 --2- 172.21.15.16:0/1938351120 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x5591326ae800 0x559132432100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:29.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.980+0000 7f18d91bd7c0 1 -- 172.21.15.16:0/1938351120 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:1 6.14 6:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47) v8 -- 0x559133212700 con 0x5591326ae800 2023-05-20T01:18:29.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.980+0000 7f18d716b700 1 --2- 172.21.15.16:0/1938351120 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x5591326ae800 0x559132432100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:29.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.980+0000 7f18d796c700 1 --2- 172.21.15.16:0/1938351120 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x5591326d0800 0x559132433700 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x5591326c0060 tx=0x559132500de0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:29.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.980+0000 7f18d716b700 1 --2- 172.21.15.16:0/1938351120 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x5591326ae800 0x559132432100 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:29.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.981+0000 7f18d716b700 1 -- 172.21.15.16:0/1938351120 <== osd.2 v2:172.21.15.16:6816/293187030 1 ==== osd_op_reply(1 1.00000000.inode [stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 160+0+0 (crc 0 0 0) 0x55913269f440 con 0x5591326ae800 2023-05-20T01:18:29.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.981+0000 7f18d91bd7c0 1 -- 172.21.15.16:0/1938351120 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:2 6.14 6:287034c4:::1.00000000.inode:head [writefull 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47) v8 -- 0x559133212a80 con 0x5591326ae800 2023-05-20T01:18:29.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.982+0000 7f18d716b700 1 -- 172.21.15.16:0/1938351120 <== osd.2 v2:172.21.15.16:6816/293187030 2 ==== osd_op_reply(2 1.00000000.inode [writefull 0~582] v47'49 uv49 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x55913269f440 con 0x5591326ae800 2023-05-20T01:18:29.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.982+0000 7f18d91bd7c0 1 --2- 172.21.15.16:0/1938351120 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x5591326aec00 0x559132431b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:29.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.982+0000 7f18d91bd7c0 1 -- 172.21.15.16:0/1938351120 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:3 6.1f 6:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47) v8 -- 0x559133212e00 con 0x5591326aec00 2023-05-20T01:18:29.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.983+0000 7f18d796c700 1 --2- 172.21.15.16:0/1938351120 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x5591326aec00 0x559132431b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:29.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.983+0000 7f18d796c700 1 --2- 172.21.15.16:0/1938351120 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x5591326aec00 0x559132431b80 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:29.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.983+0000 7f18d796c700 1 -- 172.21.15.16:0/1938351120 <== osd.3 v2:172.21.15.16:6800/3669781890 1 ==== osd_op_reply(3 100.00000000.inode [stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 162+0+0 (crc 0 0 0) 0x55913269fb00 con 0x5591326aec00 2023-05-20T01:18:29.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.983+0000 7f18d91bd7c0 1 -- 172.21.15.16:0/1938351120 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:4 6.1f 6:fe07bba1:::100.00000000.inode:head [writefull 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47) v8 -- 0x559133212700 con 0x5591326aec00 2023-05-20T01:18:30.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.984+0000 7f18d796c700 1 -- 172.21.15.16:0/1938351120 <== osd.3 v2:172.21.15.16:6800/3669781890 2 ==== osd_op_reply(4 100.00000000.inode [writefull 0~582] v47'10 uv10 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x55913269fb00 con 0x5591326aec00 2023-05-20T01:18:30.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.985+0000 7f18d91bd7c0 1 -- 172.21.15.16:0/1938351120 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:5 6.13 6:cd5a64a3:::100.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47) v8 -- 0x559133212a80 con 0x5591326aec00 2023-05-20T01:18:30.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.985+0000 7f18d796c700 1 -- 172.21.15.16:0/1938351120 <== osd.3 v2:172.21.15.16:6800/3669781890 3 ==== osd_op_reply(5 100.00000000 [omap-get-header] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x55913269fb00 con 0x5591326aec00 2023-05-20T01:18:30.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.985+0000 7f18d91bd7c0 10 datascan.find_or_create_dirfrag: read_version = 0 2023-05-20T01:18:30.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.985+0000 7f18d91bd7c0 1 -- 172.21.15.16:0/1938351120 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:6 6.13 6:cd5a64a3:::100.00000000:head [create,omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47) v8 -- 0x559133213180 con 0x5591326aec00 2023-05-20T01:18:30.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.986+0000 7f18d796c700 1 -- 172.21.15.16:0/1938351120 <== osd.3 v2:172.21.15.16:6800/3669781890 4 ==== osd_op_reply(6 100.00000000 [create,omap-set-header] v47'7 uv7 ondisk = 0) v8 ==== 198+0+0 (crc 0 0 0) 0x55913269fb00 con 0x5591326aec00 2023-05-20T01:18:30.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.986+0000 7f18d91bd7c0 10 datascan.find_or_create_dirfrag: Created dirfrag: 0x0x100 2023-05-20T01:18:30.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.986+0000 7f18d91bd7c0 1 -- 172.21.15.16:0/1938351120 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x5591326ae800 msgr2=0x559132432100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:30.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.986+0000 7f18d91bd7c0 1 --2- 172.21.15.16:0/1938351120 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x5591326ae800 0x559132432100 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:30.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.986+0000 7f18d91bd7c0 1 -- 172.21.15.16:0/1938351120 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x5591326aec00 msgr2=0x559132431b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:30.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.986+0000 7f18d91bd7c0 1 --2- 172.21.15.16:0/1938351120 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x5591326aec00 0x559132431b80 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:30.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.986+0000 7f18d91bd7c0 1 -- 172.21.15.16:0/1938351120 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x5591326d0800 msgr2=0x559132433700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:30.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.986+0000 7f18d91bd7c0 1 --2- 172.21.15.16:0/1938351120 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x5591326d0800 0x559132433700 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x5591326c0060 tx=0x559132500de0 comp rx=0 tx=0).stop 2023-05-20T01:18:30.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.986+0000 7f18d91bd7c0 1 -- 172.21.15.16:0/1938351120 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5591326ae400 msgr2=0x559132432680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:30.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.986+0000 7f18d91bd7c0 1 --2- 172.21.15.16:0/1938351120 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5591326ae400 0x559132432680 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x5591326c1410 tx=0x55913312f2c0 comp rx=0 tx=0).stop 2023-05-20T01:18:30.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.986+0000 7f18d716b700 1 -- 172.21.15.16:0/1938351120 reap_dead start 2023-05-20T01:18:30.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.986+0000 7f18d91bd7c0 1 -- 172.21.15.16:0/1938351120 shutdown_connections 2023-05-20T01:18:30.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.986+0000 7f18d91bd7c0 1 -- 172.21.15.16:0/1938351120 >> 172.21.15.16:0/1938351120 conn(0x559132397c00 msgr2=0x559132473800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:18:30.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.986+0000 7f18d91bd7c0 1 -- 172.21.15.16:0/1938351120 shutdown_connections 2023-05-20T01:18:30.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.986+0000 7f18d91bd7c0 1 -- 172.21.15.16:0/1938351120 wait complete. 2023-05-20T01:18:30.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.986+0000 7f18d91bd7c0 1 -- 172.21.15.16:0/3572118577 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559132396c00 msgr2=0x559132430b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:30.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.986+0000 7f18d91bd7c0 1 --2- 172.21.15.16:0/3572118577 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559132396c00 0x559132430b00 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x5591324f6cf0 tx=0x55913312f3e0 comp rx=0 tx=0).stop 2023-05-20T01:18:30.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.986+0000 7f18d91bd7c0 1 -- 172.21.15.16:0/3572118577 shutdown_connections 2023-05-20T01:18:30.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.986+0000 7f18d91bd7c0 1 --2- 172.21.15.16:0/3572118577 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559132396800 0x559132431080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:30.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.986+0000 7f18d91bd7c0 1 --2- 172.21.15.16:0/3572118577 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559132397000 0x559132430580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:30.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.986+0000 7f18d91bd7c0 1 --2- 172.21.15.16:0/3572118577 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559132396c00 0x559132430b00 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:30.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.986+0000 7f18d91bd7c0 1 -- 172.21.15.16:0/3572118577 >> 172.21.15.16:0/3572118577 conn(0x559132397400 msgr2=0x559132473000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:18:30.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.987+0000 7f18d91bd7c0 1 -- 172.21.15.16:0/3572118577 shutdown_connections 2023-05-20T01:18:30.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:29.987+0000 7f18d91bd7c0 1 -- 172.21.15.16:0/3572118577 wait complete. 2023-05-20T01:18:30.004 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'init'] in time 0:00:00.100288, result: 2023-05-20T01:18:30.004 DEBUG:teuthology.orchestra.run.smithi016:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_extents 2023-05-20T01:18:30.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.076+0000 7fbd627367c0 1 -- start start 2023-05-20T01:18:30.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.076+0000 7fbd627367c0 1 Processor -- start 2023-05-20T01:18:30.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.077+0000 7fbd627367c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a5f5ad000 0x556a5f646580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:30.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.077+0000 7fbd627367c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556a5f5acc00 0x556a5f646b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:30.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.077+0000 7fbd627367c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556a5f5ac800 0x556a5f647080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:30.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.077+0000 7fbd627367c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a5f5e9cc0 con 0x556a5f5ad000 2023-05-20T01:18:30.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.077+0000 7fbd627367c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a5f5e9400 con 0x556a5f5acc00 2023-05-20T01:18:30.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.077+0000 7fbd627367c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x556a5f5e8a00 con 0x556a5f5ac800 2023-05-20T01:18:30.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.077+0000 7fbd606e4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a5f5ad000 0x556a5f646580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:30.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.077+0000 7fbd606e4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a5f5ad000 0x556a5f646580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60930/0 (socket says 172.21.15.16:60930) 2023-05-20T01:18:30.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.077+0000 7fbd606e4700 1 -- 172.21.15.16:0/3486946117 learned_addr learned my addr 172.21.15.16:0/3486946117 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:18:30.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.077+0000 7fbd60ee5700 1 --2- 172.21.15.16:0/3486946117 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556a5f5ac800 0x556a5f647080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:30.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.077+0000 7fbd5fee3700 1 --2- 172.21.15.16:0/3486946117 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556a5f5acc00 0x556a5f646b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:30.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.077+0000 7fbd606e4700 1 -- 172.21.15.16:0/3486946117 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556a5f5ac800 msgr2=0x556a5f647080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:18:30.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.077+0000 7fbd606e4700 1 --2- 172.21.15.16:0/3486946117 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556a5f5ac800 0x556a5f647080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:30.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.077+0000 7fbd606e4700 1 -- 172.21.15.16:0/3486946117 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556a5f5acc00 msgr2=0x556a5f646b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:18:30.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.077+0000 7fbd606e4700 1 --2- 172.21.15.16:0/3486946117 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556a5f5acc00 0x556a5f646b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:30.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.077+0000 7fbd606e4700 1 -- 172.21.15.16:0/3486946117 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556a5f5d04e0 con 0x556a5f5ad000 2023-05-20T01:18:30.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.077+0000 7fbd5fee3700 1 --2- 172.21.15.16:0/3486946117 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556a5f5acc00 0x556a5f646b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:18:30.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.077+0000 7fbd606e4700 1 --2- 172.21.15.16:0/3486946117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a5f5ad000 0x556a5f646580 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x556a60378cc0 tx=0x556a6041e180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ffb4f087817ed11 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:30.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.078+0000 7fbd5dedf700 1 -- 172.21.15.16:0/3486946117 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x556a5f5e1600 con 0x556a5f5ad000 2023-05-20T01:18:30.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.078+0000 7fbd5dedf700 1 -- 172.21.15.16:0/3486946117 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556a5f5fe600 con 0x556a5f5ad000 2023-05-20T01:18:30.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.078+0000 7fbd5dedf700 1 -- 172.21.15.16:0/3486946117 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x556a5f5e11e0 con 0x556a5f5ad000 2023-05-20T01:18:30.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.078+0000 7fbd627367c0 1 -- 172.21.15.16:0/3486946117 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556a5f5d1520 con 0x556a5f5ad000 2023-05-20T01:18:30.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.078+0000 7fbd627367c0 1 -- 172.21.15.16:0/3486946117 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x556a5f5d1380 con 0x556a5f5ad000 2023-05-20T01:18:30.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.078+0000 7fbd627367c0 4 waiting for MDS map... 2023-05-20T01:18:30.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.078+0000 7fbd5dedf700 1 -- 172.21.15.16:0/3486946117 <== mon.0 v2:172.21.15.16:3300/0 4 ==== osd_map(47..47 src has 1..47) v4 ==== 7016+0+0 (secure 0 0 0) 0x556a5f5eec40 con 0x556a5f5ad000 2023-05-20T01:18:30.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.078+0000 7fbd5dedf700 1 -- 172.21.15.16:0/3486946117 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 22) v1 ==== 3126+0+0 (secure 0 0 0) 0x556a6042f200 con 0x556a5f5ad000 2023-05-20T01:18:30.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.078+0000 7fbd627367c0 4 Got MDS map 22 2023-05-20T01:18:30.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.078+0000 7fbd627367c0 4 datascan.main: Using metadata pool output 2023-05-20T01:18:30.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.078+0000 7fbd627367c0 4 datascan.main: connecting to RADOS... 2023-05-20T01:18:30.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.079+0000 7fbd627367c0 1 Processor -- start 2023-05-20T01:18:30.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.079+0000 7fbd627367c0 1 -- start start 2023-05-20T01:18:30.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.079+0000 7fbd627367c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a5f5adc00 0x556a5f647600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:30.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.079+0000 7fbd627367c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556a5f8c2000 0x556a5f647b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:30.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.079+0000 7fbd627367c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556a5f8c2400 0x556a5f648100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:30.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.079+0000 7fbd627367c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a5f5e8b40 con 0x556a5f5adc00 2023-05-20T01:18:30.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.079+0000 7fbd627367c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a5f5e8c80 con 0x556a5f8c2000 2023-05-20T01:18:30.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.079+0000 7fbd627367c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x556a5f5e9400 con 0x556a5f8c2400 2023-05-20T01:18:30.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.079+0000 7fbd60ee5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a5f5adc00 0x556a5f647600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:30.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.079+0000 7fbd60ee5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a5f5adc00 0x556a5f647600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60936/0 (socket says 172.21.15.16:60936) 2023-05-20T01:18:30.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.079+0000 7fbd60ee5700 1 -- 172.21.15.16:0/4082036752 learned_addr learned my addr 172.21.15.16:0/4082036752 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:18:30.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.079+0000 7fbd606e4700 1 --2- 172.21.15.16:0/4082036752 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556a5f8c2000 0x556a5f647b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:30.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.079+0000 7fbd5fee3700 1 --2- 172.21.15.16:0/4082036752 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556a5f8c2400 0x556a5f648100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:30.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.080+0000 7fbd60ee5700 1 -- 172.21.15.16:0/4082036752 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556a5f8c2400 msgr2=0x556a5f648100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:18:30.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.080+0000 7fbd60ee5700 1 --2- 172.21.15.16:0/4082036752 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556a5f8c2400 0x556a5f648100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:30.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.080+0000 7fbd60ee5700 1 -- 172.21.15.16:0/4082036752 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556a5f8c2000 msgr2=0x556a5f647b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:18:30.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.080+0000 7fbd60ee5700 1 --2- 172.21.15.16:0/4082036752 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556a5f8c2000 0x556a5f647b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:30.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.080+0000 7fbd60ee5700 1 -- 172.21.15.16:0/4082036752 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556a5f5d1d40 con 0x556a5f5adc00 2023-05-20T01:18:30.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.080+0000 7fbd60ee5700 1 --2- 172.21.15.16:0/4082036752 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a5f5adc00 0x556a5f647600 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x556a5f8ca0f0 tx=0x556a60442c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b576767012de60f server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:30.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.080+0000 7fbd5bedb700 1 -- 172.21.15.16:0/4082036752 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x556a5f5e1080 con 0x556a5f5adc00 2023-05-20T01:18:30.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.080+0000 7fbd5bedb700 1 -- 172.21.15.16:0/4082036752 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556a5f5ffc80 con 0x556a5f5adc00 2023-05-20T01:18:30.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.080+0000 7fbd5bedb700 1 -- 172.21.15.16:0/4082036752 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x556a5f5e1600 con 0x556a5f5adc00 2023-05-20T01:18:30.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.080+0000 7fbd627367c0 1 -- 172.21.15.16:0/4082036752 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a5f5adc00 msgr2=0x556a5f647600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:30.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.080+0000 7fbd627367c0 1 --2- 172.21.15.16:0/4082036752 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a5f5adc00 0x556a5f647600 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x556a5f8ca0f0 tx=0x556a60442c60 comp rx=0 tx=0).stop 2023-05-20T01:18:30.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.080+0000 7fbd627367c0 1 -- 172.21.15.16:0/4082036752 shutdown_connections 2023-05-20T01:18:30.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.080+0000 7fbd627367c0 1 --2- 172.21.15.16:0/4082036752 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556a5f8c2400 0x556a5f648100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:30.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.080+0000 7fbd627367c0 1 --2- 172.21.15.16:0/4082036752 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a5f5adc00 0x556a5f647600 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:30.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.080+0000 7fbd627367c0 1 --2- 172.21.15.16:0/4082036752 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556a5f8c2000 0x556a5f647b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:30.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.080+0000 7fbd627367c0 1 -- 172.21.15.16:0/4082036752 >> 172.21.15.16:0/4082036752 conn(0x556a5f5ac400 msgr2=0x556a5f689800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:18:30.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.080+0000 7fbd627367c0 1 -- 172.21.15.16:0/4082036752 shutdown_connections 2023-05-20T01:18:30.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.080+0000 7fbd627367c0 1 -- 172.21.15.16:0/4082036752 wait complete. 2023-05-20T01:18:30.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.081+0000 7fbd627367c0 1 Processor -- start 2023-05-20T01:18:30.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.081+0000 7fbd627367c0 1 -- start start 2023-05-20T01:18:30.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.082+0000 7fbd627367c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556a5f5ac400 0x556a5f649180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:30.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.082+0000 7fbd627367c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a5f8c2000 0x556a5f648c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:30.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.082+0000 7fbd627367c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556a5f8c2400 0x556a5f648680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:30.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.082+0000 7fbd627367c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a5f5e8000 con 0x556a5f8c2000 2023-05-20T01:18:30.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.082+0000 7fbd627367c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a5f5e9180 con 0x556a5f8c2400 2023-05-20T01:18:30.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.082+0000 7fbd627367c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x556a5f5e9040 con 0x556a5f5ac400 2023-05-20T01:18:30.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.082+0000 7fbd606e4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a5f8c2000 0x556a5f648c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:30.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.082+0000 7fbd606e4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a5f8c2000 0x556a5f648c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60944/0 (socket says 172.21.15.16:60944) 2023-05-20T01:18:30.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.082+0000 7fbd606e4700 1 -- 172.21.15.16:0/2447407827 learned_addr learned my addr 172.21.15.16:0/2447407827 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:18:30.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.082+0000 7fbd60ee5700 1 --2- 172.21.15.16:0/2447407827 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556a5f5ac400 0x556a5f649180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:30.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.082+0000 7fbd5fee3700 1 --2- 172.21.15.16:0/2447407827 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556a5f8c2400 0x556a5f648680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:30.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.082+0000 7fbd606e4700 1 -- 172.21.15.16:0/2447407827 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556a5f5ac400 msgr2=0x556a5f649180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:30.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.082+0000 7fbd606e4700 1 --2- 172.21.15.16:0/2447407827 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556a5f5ac400 0x556a5f649180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:30.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.082+0000 7fbd606e4700 1 -- 172.21.15.16:0/2447407827 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556a5f8c2400 msgr2=0x556a5f648680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:30.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.082+0000 7fbd606e4700 1 --2- 172.21.15.16:0/2447407827 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556a5f8c2400 0x556a5f648680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:30.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.082+0000 7fbd606e4700 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556a5f5d11e0 con 0x556a5f8c2000 2023-05-20T01:18:30.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.082+0000 7fbd60ee5700 1 --2- 172.21.15.16:0/2447407827 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556a5f5ac400 0x556a5f649180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:18:30.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.082+0000 7fbd606e4700 1 --2- 172.21.15.16:0/2447407827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a5f8c2000 0x556a5f648c00 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x556a5f8cb410 tx=0x556a6041e060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:30.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.082+0000 7fbd5a6d8700 1 -- 172.21.15.16:0/2447407827 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x556a5f5e0f20 con 0x556a5f8c2000 2023-05-20T01:18:30.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.082+0000 7fbd5a6d8700 1 -- 172.21.15.16:0/2447407827 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556a6043e780 con 0x556a5f8c2000 2023-05-20T01:18:30.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.082+0000 7fbd627367c0 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x556a5f5d1d40 con 0x556a5f8c2000 2023-05-20T01:18:30.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.082+0000 7fbd5a6d8700 1 -- 172.21.15.16:0/2447407827 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x556a5f5e1080 con 0x556a5f8c2000 2023-05-20T01:18:30.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.082+0000 7fbd627367c0 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556a5f5d1380 con 0x556a5f8c2000 2023-05-20T01:18:30.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.082+0000 7fbd627367c0 4 datascan.init: resolving metadata pool 6 2023-05-20T01:18:30.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.083+0000 7fbd5a6d8700 1 -- 172.21.15.16:0/2447407827 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x556a5f60e840 con 0x556a5f8c2000 2023-05-20T01:18:30.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.084+0000 7fbd5a6d8700 1 --2- 172.21.15.16:0/2447407827 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x556a5f8e4800 0x556a5f649700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:30.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.084+0000 7fbd5a6d8700 1 -- 172.21.15.16:0/2447407827 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 7016+0+0 (secure 0 0 0) 0x556a5f5eea80 con 0x556a5f8c2000 2023-05-20T01:18:30.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.084+0000 7fbd627367c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-05-20T01:18:30.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.084+0000 7fbd627367c0 4 datascan.main: data pool 'cephfs_data' has ID 7 2023-05-20T01:18:30.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.084+0000 7fbd627367c0 4 datascan.main: opening data pool 'cephfs_data' 2023-05-20T01:18:30.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.084+0000 7fbd627367c0 4 datascan.main: data pool 'cephfs_data_extra' has ID 8 2023-05-20T01:18:30.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.084+0000 7fbd627367c0 4 datascan.main: opening data pool 'cephfs_data_extra' 2023-05-20T01:18:30.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.084+0000 7fbd60ee5700 1 --2- 172.21.15.16:0/2447407827 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x556a5f8e4800 0x556a5f649700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:30.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.084+0000 7fbd627367c0 1 --2- 172.21.15.16:0/2447407827 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x556a5f8c2800 0x556a5f648100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:30.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.084+0000 7fbd627367c0 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:1 7.0 7:00000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a60438700 con 0x556a5f8c2800 2023-05-20T01:18:30.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.084+0000 7fbd5fee3700 1 --2- 172.21.15.16:0/2447407827 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x556a5f8c2800 0x556a5f648100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:30.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.084+0000 7fbd60ee5700 1 --2- 172.21.15.16:0/2447407827 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x556a5f8e4800 0x556a5f649700 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x556a5f8ca0f0 tx=0x556a60442e40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:30.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.085+0000 7fbd5fee3700 1 --2- 172.21.15.16:0/2447407827 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x556a5f8c2800 0x556a5f648100 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:30.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.085+0000 7fbd5fee3700 1 -- 172.21.15.16:0/2447407827 <== osd.5 v2:172.21.15.132:6816/1844689995 1 ==== osd_op_reply(1 [pgnls start_epoch 47 out=49b] v45'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556a5f8c1440 con 0x556a5f8c2800 2023-05-20T01:18:30.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.085+0000 7fbd5b6da700 1 --2- 172.21.15.16:0/2447407827 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x556a5f8e4c00 0x556a5f9ee000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:30.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.085+0000 7fbd5b6da700 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:2 7.20 7:04000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a60439180 con 0x556a5f8e4c00 2023-05-20T01:18:30.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.085+0000 7fbd60ee5700 1 --2- 172.21.15.16:0/2447407827 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x556a5f8e4c00 0x556a5f9ee000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:30.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.085+0000 7fbd60ee5700 1 --2- 172.21.15.16:0/2447407827 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x556a5f8e4c00 0x556a5f9ee000 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:30.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.086+0000 7fbd60ee5700 1 -- 172.21.15.16:0/2447407827 <== osd.2 v2:172.21.15.16:6816/293187030 1 ==== osd_op_reply(2 [pgnls start_epoch 47 out=49b] v45'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556a5f8c1680 con 0x556a5f8e4c00 2023-05-20T01:18:30.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.086+0000 7fbd5bedb700 1 --2- 172.21.15.16:0/2447407827 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x556a5f8e5000 0x556a5f9ee580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:30.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.086+0000 7fbd5bedb700 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:3 7.10 7:08000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a60439500 con 0x556a5f8e5000 2023-05-20T01:18:30.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.086+0000 7fbd606e4700 1 --2- 172.21.15.16:0/2447407827 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x556a5f8e5000 0x556a5f9ee580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:30.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.086+0000 7fbd606e4700 1 --2- 172.21.15.16:0/2447407827 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x556a5f8e5000 0x556a5f9ee580 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:30.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.086+0000 7fbd606e4700 1 -- 172.21.15.16:0/2447407827 <== osd.3 v2:172.21.15.16:6800/3669781890 1 ==== osd_op_reply(3 [pgnls start_epoch 47 out=49b] v45'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556a5f8c18c0 con 0x556a5f8e5000 2023-05-20T01:18:30.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.086+0000 7fbd5b6da700 1 --2- 172.21.15.16:0/2447407827 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x556a5f8e5400 0x556a5f9eeb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:30.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.086+0000 7fbd5b6da700 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:4 7.30 7:0c000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a60439880 con 0x556a5f8e5400 2023-05-20T01:18:30.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.087+0000 7fbd5fee3700 1 --2- 172.21.15.16:0/2447407827 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x556a5f8e5400 0x556a5f9eeb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:30.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.087+0000 7fbd5fee3700 1 --2- 172.21.15.16:0/2447407827 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x556a5f8e5400 0x556a5f9eeb00 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:30.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.088+0000 7fbd5fee3700 1 -- 172.21.15.16:0/2447407827 <== osd.6 v2:172.21.15.132:6801/727581123 1 ==== osd_op_reply(4 [pgnls start_epoch 47 out=49b] v45'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556a5f8c1440 con 0x556a5f8e5400 2023-05-20T01:18:30.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.088+0000 7fbd5bedb700 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:5 7.8 7:10000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a60438700 con 0x556a5f8e4c00 2023-05-20T01:18:30.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.088+0000 7fbd60ee5700 1 -- 172.21.15.16:0/2447407827 <== osd.2 v2:172.21.15.16:6816/293187030 2 ==== osd_op_reply(5 [pgnls start_epoch 47 out=49b] v45'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556a5f8c1680 con 0x556a5f8e4c00 2023-05-20T01:18:30.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.088+0000 7fbd5b6da700 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:6 7.28 7:14000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a60439c00 con 0x556a5f8e5400 2023-05-20T01:18:30.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.088+0000 7fbd5fee3700 1 -- 172.21.15.16:0/2447407827 <== osd.6 v2:172.21.15.132:6801/727581123 2 ==== osd_op_reply(6 [pgnls start_epoch 47 out=49b] v45'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556a5f8c1440 con 0x556a5f8e5400 2023-05-20T01:18:30.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.088+0000 7fbd5bedb700 1 --2- 172.21.15.16:0/2447407827 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x556a5f8e5c00 0x556a5f9ef600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:30.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.088+0000 7fbd5bedb700 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:7 7.18 7:18000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a60439880 con 0x556a5f8e5c00 2023-05-20T01:18:30.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.088+0000 7fbd60ee5700 1 --2- 172.21.15.16:0/2447407827 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x556a5f8e5c00 0x556a5f9ef600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:30.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.089+0000 7fbd60ee5700 1 --2- 172.21.15.16:0/2447407827 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x556a5f8e5c00 0x556a5f9ef600 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:30.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.089+0000 7fbd60ee5700 1 -- 172.21.15.16:0/2447407827 <== osd.0 v2:172.21.15.16:6824/3923634179 1 ==== osd_op_reply(7 [pgnls start_epoch 47 out=49b] v45'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556a5f8c1680 con 0x556a5f8e5c00 2023-05-20T01:18:30.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.089+0000 7fbd5b6da700 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:8 7.38 7:1c000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a60439180 con 0x556a5f8e5c00 2023-05-20T01:18:30.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.089+0000 7fbd60ee5700 1 -- 172.21.15.16:0/2447407827 <== osd.0 v2:172.21.15.16:6824/3923634179 2 ==== osd_op_reply(8 [pgnls start_epoch 47 out=49b] v45'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556a5f8c1680 con 0x556a5f8e5c00 2023-05-20T01:18:30.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.089+0000 7fbd5bedb700 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:9 7.4 7:20000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a5fa0a000 con 0x556a5f8e5c00 2023-05-20T01:18:30.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.089+0000 7fbd60ee5700 1 -- 172.21.15.16:0/2447407827 <== osd.0 v2:172.21.15.16:6824/3923634179 3 ==== osd_op_reply(9 [pgnls start_epoch 47 out=49b] v45'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556a5f8c1680 con 0x556a5f8e5c00 2023-05-20T01:18:30.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.089+0000 7fbd5b6da700 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:10 7.24 7:24000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a60438700 con 0x556a5f8c2800 2023-05-20T01:18:30.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.090+0000 7fbd5fee3700 1 -- 172.21.15.16:0/2447407827 <== osd.5 v2:172.21.15.132:6816/1844689995 2 ==== osd_op_reply(10 [pgnls start_epoch 47 out=49b] v45'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556a5f8c1440 con 0x556a5f8c2800 2023-05-20T01:18:30.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.090+0000 7fbd5bedb700 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:11 7.14 7:28000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a5fa0a380 con 0x556a5f8e5000 2023-05-20T01:18:30.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.090+0000 7fbd606e4700 1 -- 172.21.15.16:0/2447407827 <== osd.3 v2:172.21.15.16:6800/3669781890 2 ==== osd_op_reply(11 [pgnls start_epoch 47 out=49b] v45'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556a5f8c18c0 con 0x556a5f8e5000 2023-05-20T01:18:30.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.090+0000 7fbd5b6da700 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:12 7.34 7:2c000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a5fa0a000 con 0x556a5f8c2800 2023-05-20T01:18:30.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.090+0000 7fbd5fee3700 1 -- 172.21.15.16:0/2447407827 <== osd.5 v2:172.21.15.132:6816/1844689995 3 ==== osd_op_reply(12 [pgnls start_epoch 47 out=49b] v45'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556a5f8c1440 con 0x556a5f8c2800 2023-05-20T01:18:30.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.090+0000 7fbd5bedb700 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:13 7.c 7:30000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a5fa0a700 con 0x556a5f8e5400 2023-05-20T01:18:30.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.091+0000 7fbd5fee3700 1 -- 172.21.15.16:0/2447407827 <== osd.6 v2:172.21.15.132:6801/727581123 3 ==== osd_op_reply(13 [pgnls start_epoch 47 out=49b] v45'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556a5f8c1440 con 0x556a5f8e5400 2023-05-20T01:18:30.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.091+0000 7fbd5b6da700 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:14 7.2c 7:34000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a5fa0aa80 con 0x556a5f8e4c00 2023-05-20T01:18:30.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.091+0000 7fbd60ee5700 1 -- 172.21.15.16:0/2447407827 <== osd.2 v2:172.21.15.16:6816/293187030 3 ==== osd_op_reply(14 [pgnls start_epoch 47 out=49b] v45'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556a5f8c1680 con 0x556a5f8e4c00 2023-05-20T01:18:30.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.091+0000 7fbd5bedb700 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:15 7.1c 7:38000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a5fa0a000 con 0x556a5f8c2800 2023-05-20T01:18:30.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.091+0000 7fbd5fee3700 1 -- 172.21.15.16:0/2447407827 <== osd.5 v2:172.21.15.132:6816/1844689995 4 ==== osd_op_reply(15 [pgnls start_epoch 47 out=49b] v45'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556a5f8c1440 con 0x556a5f8c2800 2023-05-20T01:18:30.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.091+0000 7fbd5b6da700 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:16 7.3c 7:3c000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a60439880 con 0x556a5f8e5c00 2023-05-20T01:18:30.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.091+0000 7fbd60ee5700 1 -- 172.21.15.16:0/2447407827 <== osd.0 v2:172.21.15.16:6824/3923634179 4 ==== osd_op_reply(16 [pgnls start_epoch 47 out=49b] v45'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556a5f8c1680 con 0x556a5f8e5c00 2023-05-20T01:18:30.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.091+0000 7fbd5bedb700 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:17 7.2 7:40000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a5fa0ae00 con 0x556a5f8e5000 2023-05-20T01:18:30.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.092+0000 7fbd606e4700 1 -- 172.21.15.16:0/2447407827 <== osd.3 v2:172.21.15.16:6800/3669781890 3 ==== osd_op_reply(17 [pgnls start_epoch 47 out=49b] v45'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556a5f8c18c0 con 0x556a5f8e5000 2023-05-20T01:18:30.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.092+0000 7fbd5b6da700 1 --2- 172.21.15.16:0/2447407827 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x556a5f8e5800 0x556a5f9ef080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:30.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.092+0000 7fbd5b6da700 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:18 7.22 7:44000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a60439180 con 0x556a5f8e5800 2023-05-20T01:18:30.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.092+0000 7fbd606e4700 1 --2- 172.21.15.16:0/2447407827 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x556a5f8e5800 0x556a5f9ef080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:30.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.092+0000 7fbd606e4700 1 --2- 172.21.15.16:0/2447407827 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x556a5f8e5800 0x556a5f9ef080 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:30.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.093+0000 7fbd606e4700 1 -- 172.21.15.16:0/2447407827 <== osd.4 v2:172.21.15.132:6817/506317223 1 ==== osd_op_reply(18 [pgnls start_epoch 47 out=49b] v45'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556a5f8c18c0 con 0x556a5f8e5800 2023-05-20T01:18:30.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.093+0000 7fbd5bedb700 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:19 7.12 7:48000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a60439c00 con 0x556a5f8e5400 2023-05-20T01:18:30.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.093+0000 7fbd5fee3700 1 -- 172.21.15.16:0/2447407827 <== osd.6 v2:172.21.15.132:6801/727581123 4 ==== osd_op_reply(19 [pgnls start_epoch 47 out=49b] v45'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556a5f8c1440 con 0x556a5f8e5400 2023-05-20T01:18:30.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.093+0000 7fbd5b6da700 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:20 7.32 7:4c000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a5fa0a380 con 0x556a5f8e5c00 2023-05-20T01:18:30.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.093+0000 7fbd60ee5700 1 -- 172.21.15.16:0/2447407827 <== osd.0 v2:172.21.15.16:6824/3923634179 5 ==== osd_op_reply(20 [pgnls start_epoch 47 out=49b] v45'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556a5f8c1680 con 0x556a5f8e5c00 2023-05-20T01:18:30.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.093+0000 7fbd5bedb700 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:21 7.a 7:50000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a60438700 con 0x556a5f8e5000 2023-05-20T01:18:30.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.094+0000 7fbd606e4700 1 -- 172.21.15.16:0/2447407827 <== osd.3 v2:172.21.15.16:6800/3669781890 4 ==== osd_op_reply(21 [pgnls start_epoch 47 out=49b] v45'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556a5f8c18c0 con 0x556a5f8e5000 2023-05-20T01:18:30.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.094+0000 7fbd5b6da700 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:22 7.2a 7:54000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a5fa0ae00 con 0x556a5f8e5c00 2023-05-20T01:18:30.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.094+0000 7fbd60ee5700 1 -- 172.21.15.16:0/2447407827 <== osd.0 v2:172.21.15.16:6824/3923634179 6 ==== osd_op_reply(22 [pgnls start_epoch 47 out=81b] v45'10 uv10 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x556a5f8c1680 con 0x556a5f8e5c00 2023-05-20T01:18:30.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.094+0000 7fbd627367c0 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:23 7.2a 7:57fe7535:::100000001f6.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a60438a80 con 0x556a5f8e5c00 2023-05-20T01:18:30.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.094+0000 7fbd60ee5700 1 -- 172.21.15.16:0/2447407827 <== osd.0 v2:172.21.15.16:6824/3923634179 7 ==== osd_op_reply(23 100000001f6.00000000 [stat out=16b] v0'0 uv10 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x556a5f8c1680 con 0x556a5f8e5c00 2023-05-20T01:18:30.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.094+0000 7fbd627367c0 10 datascan.operator(): handling object 1099511628278.0 2023-05-20T01:18:30.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.094+0000 7fbd627367c0 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:24 7.2a 7:57fe7535:::100000001f6.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a60438e00 con 0x556a5f8e5c00 2023-05-20T01:18:30.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.095+0000 7fbd60ee5700 1 -- 172.21.15.16:0/2447407827 <== osd.0 v2:172.21.15.16:6824/3923634179 8 ==== osd_op_reply(24 100000001f6.00000000 [call] v47'11 uv11 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x556a5f8c1680 con 0x556a5f8e5c00 2023-05-20T01:18:30.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.096+0000 7fbd627367c0 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:25 7.1a 7:58000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a5fa0aa80 con 0x556a5f8e5800 2023-05-20T01:18:30.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.096+0000 7fbd606e4700 1 -- 172.21.15.16:0/2447407827 <== osd.4 v2:172.21.15.132:6817/506317223 2 ==== osd_op_reply(25 [pgnls start_epoch 47 out=49b] v45'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556a5f8c18c0 con 0x556a5f8e5800 2023-05-20T01:18:30.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.096+0000 7fbd5b6da700 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:26 7.3a 7:5c000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a5fa0b180 con 0x556a5f8e5000 2023-05-20T01:18:30.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.096+0000 7fbd606e4700 1 -- 172.21.15.16:0/2447407827 <== osd.3 v2:172.21.15.16:6800/3669781890 5 ==== osd_op_reply(26 [pgnls start_epoch 47 out=49b] v45'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556a5f8c18c0 con 0x556a5f8e5000 2023-05-20T01:18:30.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.096+0000 7fbd5bedb700 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:27 7.6 7:60000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a5fa0aa80 con 0x556a5f8e5c00 2023-05-20T01:18:30.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.096+0000 7fbd60ee5700 1 -- 172.21.15.16:0/2447407827 <== osd.0 v2:172.21.15.16:6824/3923634179 9 ==== osd_op_reply(27 [pgnls start_epoch 47 out=49b] v45'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556a5f8c1680 con 0x556a5f8e5c00 2023-05-20T01:18:30.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.096+0000 7fbd5b6da700 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:28 7.26 7:64000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a5fa0b500 con 0x556a5f8e4c00 2023-05-20T01:18:30.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.097+0000 7fbd60ee5700 1 -- 172.21.15.16:0/2447407827 <== osd.2 v2:172.21.15.16:6816/293187030 4 ==== osd_op_reply(28 [pgnls start_epoch 47 out=49b] v45'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556a5f8c1680 con 0x556a5f8e4c00 2023-05-20T01:18:30.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.097+0000 7fbd5bedb700 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:29 7.16 7:68000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a5fa0b180 con 0x556a5f8e5800 2023-05-20T01:18:30.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.097+0000 7fbd606e4700 1 -- 172.21.15.16:0/2447407827 <== osd.4 v2:172.21.15.132:6817/506317223 3 ==== osd_op_reply(29 [pgnls start_epoch 47 out=49b] v45'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556a5f8c18c0 con 0x556a5f8e5800 2023-05-20T01:18:30.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.097+0000 7fbd5b6da700 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:30 7.36 7:6c000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a60439500 con 0x556a5f8e5c00 2023-05-20T01:18:30.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.097+0000 7fbd60ee5700 1 -- 172.21.15.16:0/2447407827 <== osd.0 v2:172.21.15.16:6824/3923634179 10 ==== osd_op_reply(30 [pgnls start_epoch 47 out=101b] v45'17 uv17 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x556a5f8c1680 con 0x556a5f8e5c00 2023-05-20T01:18:30.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.097+0000 7fbd627367c0 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:31 7.36 7:6ee5e2c4:::100000001f9.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a5fa0a380 con 0x556a5f8e5c00 2023-05-20T01:18:30.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.098+0000 7fbd60ee5700 1 -- 172.21.15.16:0/2447407827 <== osd.0 v2:172.21.15.16:6824/3923634179 11 ==== osd_op_reply(31 100000001f9.00000000 [stat out=16b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x556a5f8c1680 con 0x556a5f8e5c00 2023-05-20T01:18:30.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.098+0000 7fbd627367c0 10 datascan.operator(): handling object 1099511628281.0 2023-05-20T01:18:30.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.098+0000 7fbd627367c0 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:32 7.36 7:6ee5e2c4:::100000001f9.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a5fa0ae00 con 0x556a5f8e5c00 2023-05-20T01:18:30.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.099+0000 7fbd60ee5700 1 -- 172.21.15.16:0/2447407827 <== osd.0 v2:172.21.15.16:6824/3923634179 12 ==== osd_op_reply(32 100000001f9.00000000 [call] v47'18 uv18 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x556a5f8c1680 con 0x556a5f8e5c00 2023-05-20T01:18:30.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.099+0000 7fbd627367c0 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:33 7.36 7:6fe38dc2::::head [pgnls start_epoch 47 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a60439880 con 0x556a5f8e5c00 2023-05-20T01:18:30.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.099+0000 7fbd60ee5700 1 -- 172.21.15.16:0/2447407827 <== osd.0 v2:172.21.15.16:6824/3923634179 13 ==== osd_op_reply(33 [pgnls start_epoch 47 out=81b] v47'18 uv18 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x556a5f8c1680 con 0x556a5f8e5c00 2023-05-20T01:18:30.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.099+0000 7fbd627367c0 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:34 7.36 7:6fe38dc2:::100000001f7.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a5fa0aa80 con 0x556a5f8e5c00 2023-05-20T01:18:30.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.099+0000 7fbd60ee5700 1 -- 172.21.15.16:0/2447407827 <== osd.0 v2:172.21.15.16:6824/3923634179 14 ==== osd_op_reply(34 100000001f7.00000000 [stat out=16b] v0'0 uv16 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x556a5f8c1680 con 0x556a5f8e5c00 2023-05-20T01:18:30.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.099+0000 7fbd627367c0 10 datascan.operator(): handling object 1099511628279.0 2023-05-20T01:18:30.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.099+0000 7fbd627367c0 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:35 7.36 7:6fe38dc2:::100000001f7.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a5fa0b500 con 0x556a5f8e5c00 2023-05-20T01:18:30.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.101+0000 7fbd60ee5700 1 -- 172.21.15.16:0/2447407827 <== osd.0 v2:172.21.15.16:6824/3923634179 15 ==== osd_op_reply(35 100000001f7.00000000 [call] v47'19 uv19 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x556a5f8c1680 con 0x556a5f8e5c00 2023-05-20T01:18:30.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.101+0000 7fbd627367c0 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:36 7.e 7:70000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a60438a80 con 0x556a5f8e4c00 2023-05-20T01:18:30.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.101+0000 7fbd60ee5700 1 -- 172.21.15.16:0/2447407827 <== osd.2 v2:172.21.15.16:6816/293187030 5 ==== osd_op_reply(36 [pgnls start_epoch 47 out=49b] v45'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556a5f8c1680 con 0x556a5f8e4c00 2023-05-20T01:18:30.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.101+0000 7fbd5bedb700 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:37 7.2e 7:74000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a5fa0b880 con 0x556a5f8e5800 2023-05-20T01:18:30.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.101+0000 7fbd606e4700 1 -- 172.21.15.16:0/2447407827 <== osd.4 v2:172.21.15.132:6817/506317223 4 ==== osd_op_reply(37 [pgnls start_epoch 47 out=49b] v45'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556a5f8c18c0 con 0x556a5f8e5800 2023-05-20T01:18:30.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.101+0000 7fbd5b6da700 1 --2- 172.21.15.16:0/2447407827 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x556a5fa20000 0x556a5f9f0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:30.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.101+0000 7fbd5b6da700 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:38 7.1e 7:78000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a5fa0a380 con 0x556a5fa20000 2023-05-20T01:18:30.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.102+0000 7fbd5fee3700 1 --2- 172.21.15.16:0/2447407827 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x556a5fa20000 0x556a5f9f0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:30.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.102+0000 7fbd5fee3700 1 --2- 172.21.15.16:0/2447407827 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x556a5fa20000 0x556a5f9f0100 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:30.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.102+0000 7fbd5fee3700 1 -- 172.21.15.16:0/2447407827 <== osd.1 v2:172.21.15.16:6808/3860264220 1 ==== osd_op_reply(38 [pgnls start_epoch 47 out=49b] v45'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556a5f8c1440 con 0x556a5fa20000 2023-05-20T01:18:30.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.102+0000 7fbd5bedb700 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:39 7.3e 7:7c000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a60439180 con 0x556a5f8c2800 2023-05-20T01:18:30.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.102+0000 7fbd5fee3700 1 -- 172.21.15.16:0/2447407827 <== osd.5 v2:172.21.15.132:6816/1844689995 5 ==== osd_op_reply(39 [pgnls start_epoch 47 out=49b] v45'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556a5f8c1440 con 0x556a5f8c2800 2023-05-20T01:18:30.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.102+0000 7fbd5b6da700 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:40 7.1 7:80000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a5fa0ae00 con 0x556a5f8e5800 2023-05-20T01:18:30.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.103+0000 7fbd606e4700 1 -- 172.21.15.16:0/2447407827 <== osd.4 v2:172.21.15.132:6817/506317223 5 ==== osd_op_reply(40 [pgnls start_epoch 47 out=49b] v45'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556a5f8c18c0 con 0x556a5f8e5800 2023-05-20T01:18:30.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.103+0000 7fbd5bedb700 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:41 7.21 7:84000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a60438700 con 0x556a5f8e5000 2023-05-20T01:18:30.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.103+0000 7fbd606e4700 1 -- 172.21.15.16:0/2447407827 <== osd.3 v2:172.21.15.16:6800/3669781890 6 ==== osd_op_reply(41 [pgnls start_epoch 47 out=49b] v45'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556a5f8c18c0 con 0x556a5f8e5000 2023-05-20T01:18:30.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.103+0000 7fbd5b6da700 1 --2- 172.21.15.16:0/2447407827 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x556a5fa20400 0x556a5f9f0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:30.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.103+0000 7fbd5b6da700 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:42 7.11 7:88000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a5fa0aa80 con 0x556a5fa20400 2023-05-20T01:18:30.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.103+0000 7fbd60ee5700 1 --2- 172.21.15.16:0/2447407827 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x556a5fa20400 0x556a5f9f0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:30.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.104+0000 7fbd60ee5700 1 --2- 172.21.15.16:0/2447407827 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x556a5fa20400 0x556a5f9f0680 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:30.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.104+0000 7fbd60ee5700 1 -- 172.21.15.16:0/2447407827 <== osd.7 v2:172.21.15.132:6800/466001149 1 ==== osd_op_reply(42 [pgnls start_epoch 47 out=49b] v45'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556a5f8c1680 con 0x556a5fa20400 2023-05-20T01:18:30.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.104+0000 7fbd5bedb700 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:43 7.31 7:8c000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a5fa0a700 con 0x556a5f8e5000 2023-05-20T01:18:30.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.104+0000 7fbd606e4700 1 -- 172.21.15.16:0/2447407827 <== osd.3 v2:172.21.15.16:6800/3669781890 7 ==== osd_op_reply(43 [pgnls start_epoch 47 out=49b] v45'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556a5f8c18c0 con 0x556a5f8e5000 2023-05-20T01:18:30.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.104+0000 7fbd5b6da700 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:44 7.9 7:90000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a5fa0b500 con 0x556a5f8e5800 2023-05-20T01:18:30.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.104+0000 7fbd606e4700 1 -- 172.21.15.16:0/2447407827 <== osd.4 v2:172.21.15.132:6817/506317223 6 ==== osd_op_reply(44 [pgnls start_epoch 47 out=49b] v45'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556a5f8c18c0 con 0x556a5f8e5800 2023-05-20T01:18:30.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.105+0000 7fbd5bedb700 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:45 7.29 7:94000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a5fa0a000 con 0x556a5f8e5400 2023-05-20T01:18:30.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.105+0000 7fbd5fee3700 1 -- 172.21.15.16:0/2447407827 <== osd.6 v2:172.21.15.132:6801/727581123 5 ==== osd_op_reply(45 [pgnls start_epoch 47 out=49b] v45'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556a5f8c1440 con 0x556a5f8e5400 2023-05-20T01:18:30.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.105+0000 7fbd5b6da700 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:46 7.19 7:98000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a60439880 con 0x556a5f8e5400 2023-05-20T01:18:30.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.105+0000 7fbd5fee3700 1 -- 172.21.15.16:0/2447407827 <== osd.6 v2:172.21.15.132:6801/727581123 6 ==== osd_op_reply(46 [pgnls start_epoch 47 out=81b] v45'11 uv11 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x556a5f8c1440 con 0x556a5f8e5400 2023-05-20T01:18:30.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.105+0000 7fbd627367c0 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:47 7.19 7:997b7129:::100000001f8.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a60438e00 con 0x556a5f8e5400 2023-05-20T01:18:30.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.106+0000 7fbd5fee3700 1 -- 172.21.15.16:0/2447407827 <== osd.6 v2:172.21.15.132:6801/727581123 7 ==== osd_op_reply(47 100000001f8.00000000 [stat out=16b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x556a5f8c1440 con 0x556a5f8e5400 2023-05-20T01:18:30.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.106+0000 7fbd627367c0 10 datascan.operator(): handling object 1099511628280.0 2023-05-20T01:18:30.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.106+0000 7fbd627367c0 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:48 7.19 7:997b7129:::100000001f8.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a60439500 con 0x556a5f8e5400 2023-05-20T01:18:30.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.107+0000 7fbd5fee3700 1 -- 172.21.15.16:0/2447407827 <== osd.6 v2:172.21.15.132:6801/727581123 8 ==== osd_op_reply(48 100000001f8.00000000 [call] v47'12 uv12 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x556a5f8c1440 con 0x556a5f8e5400 2023-05-20T01:18:30.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.107+0000 7fbd627367c0 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:49 7.39 7:9c000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a5fa0a000 con 0x556a5fa20400 2023-05-20T01:18:30.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.107+0000 7fbd60ee5700 1 -- 172.21.15.16:0/2447407827 <== osd.7 v2:172.21.15.132:6800/466001149 2 ==== osd_op_reply(49 [pgnls start_epoch 47 out=49b] v45'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556a5f8c1680 con 0x556a5fa20400 2023-05-20T01:18:30.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.107+0000 7fbd5b6da700 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:50 7.5 7:a0000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a60438a80 con 0x556a5fa20400 2023-05-20T01:18:30.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.108+0000 7fbd60ee5700 1 -- 172.21.15.16:0/2447407827 <== osd.7 v2:172.21.15.132:6800/466001149 3 ==== osd_op_reply(50 [pgnls start_epoch 47 out=49b] v45'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556a5f8c1680 con 0x556a5fa20400 2023-05-20T01:18:30.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.108+0000 7fbd5bedb700 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:51 7.25 7:a4000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a5fa0a380 con 0x556a5f8e5800 2023-05-20T01:18:30.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.108+0000 7fbd606e4700 1 -- 172.21.15.16:0/2447407827 <== osd.4 v2:172.21.15.132:6817/506317223 7 ==== osd_op_reply(51 [pgnls start_epoch 47 out=49b] v45'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556a5f8c18c0 con 0x556a5f8e5800 2023-05-20T01:18:30.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.108+0000 7fbd5b6da700 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:52 7.15 7:a8000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a5fa0b180 con 0x556a5fa20400 2023-05-20T01:18:30.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.108+0000 7fbd60ee5700 1 -- 172.21.15.16:0/2447407827 <== osd.7 v2:172.21.15.132:6800/466001149 4 ==== osd_op_reply(52 [pgnls start_epoch 47 out=49b] v45'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556a5f8c1680 con 0x556a5fa20400 2023-05-20T01:18:30.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.108+0000 7fbd5bedb700 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:53 7.35 7:ac000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a5fa0bc00 con 0x556a5f8e5c00 2023-05-20T01:18:30.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.109+0000 7fbd60ee5700 1 -- 172.21.15.16:0/2447407827 <== osd.0 v2:172.21.15.16:6824/3923634179 16 ==== osd_op_reply(53 [pgnls start_epoch 47 out=49b] v45'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556a5f8c1680 con 0x556a5f8e5c00 2023-05-20T01:18:30.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.109+0000 7fbd5b6da700 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:54 7.d 7:b0000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a5fa0b880 con 0x556a5f8e5400 2023-05-20T01:18:30.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.109+0000 7fbd5fee3700 1 -- 172.21.15.16:0/2447407827 <== osd.6 v2:172.21.15.132:6801/727581123 9 ==== osd_op_reply(54 [pgnls start_epoch 47 out=49b] v45'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556a5f8c1440 con 0x556a5f8e5400 2023-05-20T01:18:30.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.109+0000 7fbd5bedb700 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:55 7.2d 7:b4000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a60439c00 con 0x556a5f8e5400 2023-05-20T01:18:30.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.110+0000 7fbd5fee3700 1 -- 172.21.15.16:0/2447407827 <== osd.6 v2:172.21.15.132:6801/727581123 10 ==== osd_op_reply(55 [pgnls start_epoch 47 out=49b] v45'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556a5f8c1440 con 0x556a5f8e5400 2023-05-20T01:18:30.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.110+0000 7fbd5b6da700 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:56 7.1d 7:b8000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a5fa0ae00 con 0x556a5fa20000 2023-05-20T01:18:30.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.110+0000 7fbd5fee3700 1 -- 172.21.15.16:0/2447407827 <== osd.1 v2:172.21.15.16:6808/3860264220 2 ==== osd_op_reply(56 [pgnls start_epoch 47 out=49b] v45'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556a5f8c1440 con 0x556a5fa20000 2023-05-20T01:18:30.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.110+0000 7fbd5bedb700 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:57 7.3d 7:bc000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a5fa26000 con 0x556a5f8e5800 2023-05-20T01:18:30.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.110+0000 7fbd606e4700 1 -- 172.21.15.16:0/2447407827 <== osd.4 v2:172.21.15.132:6817/506317223 8 ==== osd_op_reply(57 [pgnls start_epoch 47 out=49b] v45'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556a5f8c18c0 con 0x556a5f8e5800 2023-05-20T01:18:30.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.110+0000 7fbd5b6da700 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:58 7.3 7:c0000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a5fa0a700 con 0x556a5f8e5000 2023-05-20T01:18:30.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.110+0000 7fbd606e4700 1 -- 172.21.15.16:0/2447407827 <== osd.3 v2:172.21.15.16:6800/3669781890 8 ==== osd_op_reply(58 [pgnls start_epoch 47 out=49b] v45'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556a5f8c18c0 con 0x556a5f8e5000 2023-05-20T01:18:30.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.110+0000 7fbd5bedb700 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:59 7.23 7:c4000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a5fa26380 con 0x556a5f8e5400 2023-05-20T01:18:30.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.111+0000 7fbd5fee3700 1 -- 172.21.15.16:0/2447407827 <== osd.6 v2:172.21.15.132:6801/727581123 11 ==== osd_op_reply(59 [pgnls start_epoch 47 out=49b] v45'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556a5f8c1440 con 0x556a5f8e5400 2023-05-20T01:18:30.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.111+0000 7fbd5b6da700 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:60 7.13 7:c8000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a5fa27880 con 0x556a5f8e5c00 2023-05-20T01:18:30.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.111+0000 7fbd60ee5700 1 -- 172.21.15.16:0/2447407827 <== osd.0 v2:172.21.15.16:6824/3923634179 17 ==== osd_op_reply(60 [pgnls start_epoch 47 out=49b] v45'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556a5f8c1680 con 0x556a5f8e5c00 2023-05-20T01:18:30.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.111+0000 7fbd5bedb700 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:61 7.33 7:cc000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a5fa26700 con 0x556a5f8e4c00 2023-05-20T01:18:30.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.111+0000 7fbd60ee5700 1 -- 172.21.15.16:0/2447407827 <== osd.2 v2:172.21.15.16:6816/293187030 6 ==== osd_op_reply(61 [pgnls start_epoch 47 out=49b] v45'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556a5f8c1680 con 0x556a5f8e4c00 2023-05-20T01:18:30.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.111+0000 7fbd5b6da700 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:62 7.b 7:d0000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a5fa27c00 con 0x556a5fa20000 2023-05-20T01:18:30.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.112+0000 7fbd5fee3700 1 -- 172.21.15.16:0/2447407827 <== osd.1 v2:172.21.15.16:6808/3860264220 3 ==== osd_op_reply(62 [pgnls start_epoch 47 out=49b] v45'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556a5f8c1440 con 0x556a5fa20000 2023-05-20T01:18:30.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.112+0000 7fbd5bedb700 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:63 7.2b 7:d4000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a5fa26a80 con 0x556a5f8e5000 2023-05-20T01:18:30.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.112+0000 7fbd606e4700 1 -- 172.21.15.16:0/2447407827 <== osd.3 v2:172.21.15.16:6800/3669781890 9 ==== osd_op_reply(63 [pgnls start_epoch 47 out=49b] v45'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556a5f8c18c0 con 0x556a5f8e5000 2023-05-20T01:18:30.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.112+0000 7fbd5b6da700 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:64 7.1b 7:d8000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a5fa27880 con 0x556a5f8c2800 2023-05-20T01:18:30.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.112+0000 7fbd5fee3700 1 -- 172.21.15.16:0/2447407827 <== osd.5 v2:172.21.15.132:6816/1844689995 6 ==== osd_op_reply(64 [pgnls start_epoch 47 out=49b] v45'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556a5f8c1440 con 0x556a5f8c2800 2023-05-20T01:18:30.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.112+0000 7fbd5bedb700 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:65 7.3b 7:dc000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a5fa26e00 con 0x556a5f8c2800 2023-05-20T01:18:30.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.113+0000 7fbd5fee3700 1 -- 172.21.15.16:0/2447407827 <== osd.5 v2:172.21.15.132:6816/1844689995 7 ==== osd_op_reply(65 [pgnls start_epoch 47 out=49b] v45'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556a5f8c1440 con 0x556a5f8c2800 2023-05-20T01:18:30.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.113+0000 7fbd5b6da700 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:66 7.7 7:e0000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a5fa26700 con 0x556a5f8e5400 2023-05-20T01:18:30.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.113+0000 7fbd5fee3700 1 -- 172.21.15.16:0/2447407827 <== osd.6 v2:172.21.15.132:6801/727581123 12 ==== osd_op_reply(66 [pgnls start_epoch 47 out=81b] v45'6 uv6 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x556a5f8c1440 con 0x556a5f8e5400 2023-05-20T01:18:30.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.113+0000 7fbd627367c0 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:67 7.7 7:e2e07930:::10000000001.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a60439180 con 0x556a5f8e5400 2023-05-20T01:18:30.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.113+0000 7fbd5fee3700 1 -- 172.21.15.16:0/2447407827 <== osd.6 v2:172.21.15.132:6801/727581123 13 ==== osd_op_reply(67 10000000001.00000000 [stat out=16b] v0'0 uv6 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x556a5f8c1440 con 0x556a5f8e5400 2023-05-20T01:18:30.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.113+0000 7fbd627367c0 10 datascan.operator(): handling object 1099511627777.0 2023-05-20T01:18:30.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.113+0000 7fbd627367c0 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:68 7.7 7:e2e07930:::10000000001.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a60439880 con 0x556a5f8e5400 2023-05-20T01:18:30.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.114+0000 7fbd5fee3700 1 -- 172.21.15.16:0/2447407827 <== osd.6 v2:172.21.15.132:6801/727581123 14 ==== osd_op_reply(68 10000000001.00000000 [call] v47'7 uv7 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x556a5f8c1440 con 0x556a5f8e5400 2023-05-20T01:18:30.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.114+0000 7fbd627367c0 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:69 7.27 7:e4000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a60438e00 con 0x556a5f8e5400 2023-05-20T01:18:30.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.115+0000 7fbd5fee3700 1 -- 172.21.15.16:0/2447407827 <== osd.6 v2:172.21.15.132:6801/727581123 15 ==== osd_op_reply(69 [pgnls start_epoch 47 out=49b] v45'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556a5f8c1440 con 0x556a5f8e5400 2023-05-20T01:18:30.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.115+0000 7fbd5b6da700 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:70 7.17 7:e8000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a5fa0aa80 con 0x556a5fa20400 2023-05-20T01:18:30.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.115+0000 7fbd60ee5700 1 -- 172.21.15.16:0/2447407827 <== osd.7 v2:172.21.15.132:6800/466001149 5 ==== osd_op_reply(70 [pgnls start_epoch 47 out=49b] v45'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556a5f8c1680 con 0x556a5fa20400 2023-05-20T01:18:30.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.115+0000 7fbd5bedb700 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:71 7.37 7:ec000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a5fa27180 con 0x556a5f8e5c00 2023-05-20T01:18:30.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.115+0000 7fbd60ee5700 1 -- 172.21.15.16:0/2447407827 <== osd.0 v2:172.21.15.16:6824/3923634179 18 ==== osd_op_reply(71 [pgnls start_epoch 47 out=49b] v45'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556a5f8c1680 con 0x556a5f8e5c00 2023-05-20T01:18:30.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.115+0000 7fbd5b6da700 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:72 7.f 7:f0000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a5fa0a000 con 0x556a5f8e5c00 2023-05-20T01:18:30.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.116+0000 7fbd60ee5700 1 -- 172.21.15.16:0/2447407827 <== osd.0 v2:172.21.15.16:6824/3923634179 19 ==== osd_op_reply(72 [pgnls start_epoch 47 out=49b] v45'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556a5f8c1680 con 0x556a5f8e5c00 2023-05-20T01:18:30.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.116+0000 7fbd5bedb700 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:73 7.2f 7:f4000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a5fa27500 con 0x556a5f8e5800 2023-05-20T01:18:30.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.116+0000 7fbd606e4700 1 -- 172.21.15.16:0/2447407827 <== osd.4 v2:172.21.15.132:6817/506317223 9 ==== osd_op_reply(73 [pgnls start_epoch 47 out=49b] v45'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556a5f8c18c0 con 0x556a5f8e5800 2023-05-20T01:18:30.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.116+0000 7fbd5b6da700 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:74 7.1f 7:f8000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a5fa0b180 con 0x556a5f8e5400 2023-05-20T01:18:30.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.116+0000 7fbd5fee3700 1 -- 172.21.15.16:0/2447407827 <== osd.6 v2:172.21.15.132:6801/727581123 16 ==== osd_op_reply(74 [pgnls start_epoch 47 out=49b] v45'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556a5f8c1440 con 0x556a5f8e5400 2023-05-20T01:18:30.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.116+0000 7fbd5bedb700 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:75 7.3f 7:fc000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a5fa26000 con 0x556a5fa20400 2023-05-20T01:18:30.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.117+0000 7fbd60ee5700 1 -- 172.21.15.16:0/2447407827 <== osd.7 v2:172.21.15.132:6800/466001149 6 ==== osd_op_reply(75 [pgnls start_epoch 47 out=49b] v45'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556a5f8c1680 con 0x556a5fa20400 2023-05-20T01:18:30.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.117+0000 7fbd627367c0 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:76 8.0 8:00000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a60439500 con 0x556a5f8e5000 2023-05-20T01:18:30.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.117+0000 7fbd606e4700 1 -- 172.21.15.16:0/2447407827 <== osd.3 v2:172.21.15.16:6800/3669781890 10 ==== osd_op_reply(76 [pgnls start_epoch 47 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556a5f8c18c0 con 0x556a5f8e5000 2023-05-20T01:18:30.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.117+0000 7fbd5bedb700 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:77 8.10 8:08000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a5fa26a80 con 0x556a5fa20400 2023-05-20T01:18:30.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.117+0000 7fbd60ee5700 1 -- 172.21.15.16:0/2447407827 <== osd.7 v2:172.21.15.132:6800/466001149 7 ==== osd_op_reply(77 [pgnls start_epoch 47 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556a5f8c1680 con 0x556a5fa20400 2023-05-20T01:18:30.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.117+0000 7fbd5b6da700 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:78 8.8 8:10000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a5fa0bc00 con 0x556a5f8c2800 2023-05-20T01:18:30.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.118+0000 7fbd5fee3700 1 -- 172.21.15.16:0/2447407827 <== osd.5 v2:172.21.15.132:6816/1844689995 8 ==== osd_op_reply(78 [pgnls start_epoch 47 out=81b] v45'1 uv1 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x556a5f8c1440 con 0x556a5f8c2800 2023-05-20T01:18:30.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.118+0000 7fbd627367c0 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:79 8.8 8:1247e94b:::100000001f9.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a60438700 con 0x556a5f8c2800 2023-05-20T01:18:30.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.118+0000 7fbd5fee3700 1 -- 172.21.15.16:0/2447407827 <== osd.5 v2:172.21.15.132:6816/1844689995 9 ==== osd_op_reply(79 100000001f9.00000001 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x556a5f8c1440 con 0x556a5f8c2800 2023-05-20T01:18:30.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.118+0000 7fbd627367c0 10 datascan.operator(): handling object 1099511628281.1 2023-05-20T01:18:30.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.118+0000 7fbd627367c0 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:80 7.36 7:6ee5e2c4:::100000001f9.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a5fa26e00 con 0x556a5f8e5c00 2023-05-20T01:18:30.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.120+0000 7fbd60ee5700 1 -- 172.21.15.16:0/2447407827 <== osd.0 v2:172.21.15.16:6824/3923634179 20 ==== osd_op_reply(80 100000001f9.00000000 [call,setxattr (8)] v47'20 uv20 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x556a5f8c1680 con 0x556a5f8e5c00 2023-05-20T01:18:30.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.120+0000 7fbd627367c0 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:81 8.18 8:18000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a5fa26700 con 0x556a5f8e5800 2023-05-20T01:18:30.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.120+0000 7fbd606e4700 1 -- 172.21.15.16:0/2447407827 <== osd.4 v2:172.21.15.132:6817/506317223 10 ==== osd_op_reply(81 [pgnls start_epoch 47 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556a5f8c18c0 con 0x556a5f8e5800 2023-05-20T01:18:30.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.120+0000 7fbd5b6da700 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:82 8.4 8:20000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a60438a80 con 0x556a5f8c2800 2023-05-20T01:18:30.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.120+0000 7fbd5fee3700 1 -- 172.21.15.16:0/2447407827 <== osd.5 v2:172.21.15.132:6816/1844689995 10 ==== osd_op_reply(82 [pgnls start_epoch 47 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556a5f8c1440 con 0x556a5f8c2800 2023-05-20T01:18:30.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.120+0000 7fbd5bedb700 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:83 8.14 8:28000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a5fa0b500 con 0x556a5f8e5800 2023-05-20T01:18:30.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.121+0000 7fbd606e4700 1 -- 172.21.15.16:0/2447407827 <== osd.4 v2:172.21.15.132:6817/506317223 11 ==== osd_op_reply(83 [pgnls start_epoch 47 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556a5f8c18c0 con 0x556a5f8e5800 2023-05-20T01:18:30.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.121+0000 7fbd5b6da700 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:84 8.c 8:30000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a5fa27500 con 0x556a5f8e5c00 2023-05-20T01:18:30.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.121+0000 7fbd60ee5700 1 -- 172.21.15.16:0/2447407827 <== osd.0 v2:172.21.15.16:6824/3923634179 21 ==== osd_op_reply(84 [pgnls start_epoch 47 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556a5f8c1680 con 0x556a5f8e5c00 2023-05-20T01:18:30.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.121+0000 7fbd5bedb700 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:85 8.1c 8:38000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a5fa0a380 con 0x556a5f8c2800 2023-05-20T01:18:30.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.121+0000 7fbd5fee3700 1 -- 172.21.15.16:0/2447407827 <== osd.5 v2:172.21.15.132:6816/1844689995 11 ==== osd_op_reply(85 [pgnls start_epoch 47 out=81b] v45'1 uv1 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x556a5f8c1440 con 0x556a5f8c2800 2023-05-20T01:18:30.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.121+0000 7fbd627367c0 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:86 8.1c 8:3d6db04b:::100000001f6.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a5fa0b180 con 0x556a5f8c2800 2023-05-20T01:18:30.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.122+0000 7fbd5fee3700 1 -- 172.21.15.16:0/2447407827 <== osd.5 v2:172.21.15.132:6816/1844689995 12 ==== osd_op_reply(86 100000001f6.00000001 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x556a5f8c1440 con 0x556a5f8c2800 2023-05-20T01:18:30.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.122+0000 7fbd627367c0 10 datascan.operator(): handling object 1099511628278.1 2023-05-20T01:18:30.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.122+0000 7fbd627367c0 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:87 7.2a 7:57fe7535:::100000001f6.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a5fa0bc00 con 0x556a5f8e5c00 2023-05-20T01:18:30.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.123+0000 7fbd60ee5700 1 -- 172.21.15.16:0/2447407827 <== osd.0 v2:172.21.15.16:6824/3923634179 22 ==== osd_op_reply(87 100000001f6.00000000 [call,setxattr (8)] v47'12 uv12 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x556a5f8c1680 con 0x556a5f8e5c00 2023-05-20T01:18:30.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.123+0000 7fbd627367c0 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:88 8.2 8:40000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a60439180 con 0x556a5f8e5800 2023-05-20T01:18:30.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.123+0000 7fbd606e4700 1 -- 172.21.15.16:0/2447407827 <== osd.4 v2:172.21.15.132:6817/506317223 12 ==== osd_op_reply(88 [pgnls start_epoch 47 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556a5f8c18c0 con 0x556a5f8e5800 2023-05-20T01:18:30.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.123+0000 7fbd5bedb700 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:89 8.12 8:48000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a5fa0a700 con 0x556a5f8e5400 2023-05-20T01:18:30.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.124+0000 7fbd5fee3700 1 -- 172.21.15.16:0/2447407827 <== osd.6 v2:172.21.15.132:6801/727581123 17 ==== osd_op_reply(89 [pgnls start_epoch 47 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556a5f8c1440 con 0x556a5f8e5400 2023-05-20T01:18:30.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.124+0000 7fbd5b6da700 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:90 8.a 8:50000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a5fa26380 con 0x556a5fa20000 2023-05-20T01:18:30.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.124+0000 7fbd5fee3700 1 -- 172.21.15.16:0/2447407827 <== osd.1 v2:172.21.15.16:6808/3860264220 4 ==== osd_op_reply(90 [pgnls start_epoch 47 out=81b] v45'3 uv3 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x556a5f8c1440 con 0x556a5fa20000 2023-05-20T01:18:30.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.124+0000 7fbd627367c0 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:91 8.a 8:57fe7535:::100000001f6.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a60439880 con 0x556a5fa20000 2023-05-20T01:18:30.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.125+0000 7fbd5fee3700 1 -- 172.21.15.16:0/2447407827 <== osd.1 v2:172.21.15.16:6808/3860264220 5 ==== osd_op_reply(91 100000001f6.00000000 [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x556a5f8c1440 con 0x556a5fa20000 2023-05-20T01:18:30.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.125+0000 7fbd627367c0 10 datascan.operator(): handling object 1099511628278.0 2023-05-20T01:18:30.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.125+0000 7fbd627367c0 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:92 7.2a 7:57fe7535:::100000001f6.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a60438e00 con 0x556a5f8e5c00 2023-05-20T01:18:30.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.126+0000 7fbd60ee5700 1 -- 172.21.15.16:0/2447407827 <== osd.0 v2:172.21.15.16:6824/3923634179 23 ==== osd_op_reply(92 100000001f6.00000000 [call,setxattr (8)] v47'13 uv13 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x556a5f8c1680 con 0x556a5f8e5c00 2023-05-20T01:18:30.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.126+0000 7fbd627367c0 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:93 8.1a 8:58000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a5fa0aa80 con 0x556a5f8e5c00 2023-05-20T01:18:30.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.126+0000 7fbd60ee5700 1 -- 172.21.15.16:0/2447407827 <== osd.0 v2:172.21.15.16:6824/3923634179 24 ==== osd_op_reply(93 [pgnls start_epoch 47 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556a5f8c1680 con 0x556a5f8e5c00 2023-05-20T01:18:30.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.126+0000 7fbd5b6da700 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:94 8.6 8:60000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a5fa27c00 con 0x556a5f8e5c00 2023-05-20T01:18:30.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.126+0000 7fbd60ee5700 1 -- 172.21.15.16:0/2447407827 <== osd.0 v2:172.21.15.16:6824/3923634179 25 ==== osd_op_reply(94 [pgnls start_epoch 47 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556a5f8c1680 con 0x556a5f8e5c00 2023-05-20T01:18:30.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.126+0000 7fbd5bedb700 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:95 8.16 8:68000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a5fa0b880 con 0x556a5f8e5000 2023-05-20T01:18:30.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.127+0000 7fbd606e4700 1 -- 172.21.15.16:0/2447407827 <== osd.3 v2:172.21.15.16:6800/3669781890 11 ==== osd_op_reply(95 [pgnls start_epoch 47 out=101b] v45'4 uv4 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x556a5f8c18c0 con 0x556a5f8e5000 2023-05-20T01:18:30.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.127+0000 7fbd627367c0 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:96 8.16 8:6ee5e2c4:::100000001f9.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a5fa0a000 con 0x556a5f8e5000 2023-05-20T01:18:30.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.127+0000 7fbd606e4700 1 -- 172.21.15.16:0/2447407827 <== osd.3 v2:172.21.15.16:6800/3669781890 12 ==== osd_op_reply(96 100000001f9.00000000 [stat out=16b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x556a5f8c18c0 con 0x556a5f8e5000 2023-05-20T01:18:30.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.127+0000 7fbd627367c0 10 datascan.operator(): handling object 1099511628281.0 2023-05-20T01:18:30.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.127+0000 7fbd627367c0 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:97 7.36 7:6ee5e2c4:::100000001f9.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a5fa0bc00 con 0x556a5f8e5c00 2023-05-20T01:18:30.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.129+0000 7fbd60ee5700 1 -- 172.21.15.16:0/2447407827 <== osd.0 v2:172.21.15.16:6824/3923634179 26 ==== osd_op_reply(97 100000001f9.00000000 [call,setxattr (8)] v47'21 uv21 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x556a5f8c1680 con 0x556a5f8e5c00 2023-05-20T01:18:30.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.129+0000 7fbd627367c0 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:98 8.16 8:6fe38dc2::::head [pgnls start_epoch 47 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a5fa27180 con 0x556a5f8e5000 2023-05-20T01:18:30.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.129+0000 7fbd606e4700 1 -- 172.21.15.16:0/2447407827 <== osd.3 v2:172.21.15.16:6800/3669781890 13 ==== osd_op_reply(98 [pgnls start_epoch 47 out=81b] v45'4 uv4 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x556a5f8c18c0 con 0x556a5f8e5000 2023-05-20T01:18:30.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.129+0000 7fbd627367c0 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:99 8.16 8:6fe38dc2:::100000001f7.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a5fa26000 con 0x556a5f8e5000 2023-05-20T01:18:30.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.129+0000 7fbd606e4700 1 -- 172.21.15.16:0/2447407827 <== osd.3 v2:172.21.15.16:6800/3669781890 14 ==== osd_op_reply(99 100000001f7.00000000 [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x556a5f8c18c0 con 0x556a5f8e5000 2023-05-20T01:18:30.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.129+0000 7fbd627367c0 10 datascan.operator(): handling object 1099511628279.0 2023-05-20T01:18:30.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.129+0000 7fbd627367c0 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:100 7.36 7:6fe38dc2:::100000001f7.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a5fa26a80 con 0x556a5f8e5c00 2023-05-20T01:18:30.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.130+0000 7fbd60ee5700 1 -- 172.21.15.16:0/2447407827 <== osd.0 v2:172.21.15.16:6824/3923634179 27 ==== osd_op_reply(100 100000001f7.00000000 [call,setxattr (8)] v47'22 uv22 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x556a5f8c1680 con 0x556a5f8e5c00 2023-05-20T01:18:30.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.130+0000 7fbd627367c0 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:101 8.e 8:70000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a5fa26e00 con 0x556a5f8e5c00 2023-05-20T01:18:30.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.131+0000 7fbd60ee5700 1 -- 172.21.15.16:0/2447407827 <== osd.0 v2:172.21.15.16:6824/3923634179 28 ==== osd_op_reply(101 [pgnls start_epoch 47 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556a5f8c1680 con 0x556a5f8e5c00 2023-05-20T01:18:30.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.131+0000 7fbd5b6da700 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:102 8.1e 8:78000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a5fa27880 con 0x556a5f8e5000 2023-05-20T01:18:30.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.131+0000 7fbd606e4700 1 -- 172.21.15.16:0/2447407827 <== osd.3 v2:172.21.15.16:6800/3669781890 15 ==== osd_op_reply(102 [pgnls start_epoch 47 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556a5f8c18c0 con 0x556a5f8e5000 2023-05-20T01:18:30.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.131+0000 7fbd5bedb700 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:103 8.1 8:80000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a5fa0ae00 con 0x556a5fa20000 2023-05-20T01:18:30.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.131+0000 7fbd5fee3700 1 -- 172.21.15.16:0/2447407827 <== osd.1 v2:172.21.15.16:6808/3860264220 6 ==== osd_op_reply(103 [pgnls start_epoch 47 out=81b] v45'1 uv1 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x556a5f8c1440 con 0x556a5fa20000 2023-05-20T01:18:30.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.131+0000 7fbd627367c0 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:104 8.1 8:848ebb0b:::100000001f7.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a5fa27500 con 0x556a5fa20000 2023-05-20T01:18:30.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.132+0000 7fbd5fee3700 1 -- 172.21.15.16:0/2447407827 <== osd.1 v2:172.21.15.16:6808/3860264220 7 ==== osd_op_reply(104 100000001f7.00000001 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x556a5f8c1440 con 0x556a5fa20000 2023-05-20T01:18:30.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.132+0000 7fbd627367c0 10 datascan.operator(): handling object 1099511628279.1 2023-05-20T01:18:30.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.132+0000 7fbd627367c0 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:105 7.36 7:6fe38dc2:::100000001f7.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a5fa0a380 con 0x556a5f8e5c00 2023-05-20T01:18:30.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.133+0000 7fbd60ee5700 1 -- 172.21.15.16:0/2447407827 <== osd.0 v2:172.21.15.16:6824/3923634179 29 ==== osd_op_reply(105 100000001f7.00000000 [call,setxattr (8)] v47'23 uv23 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x556a5f8c1680 con 0x556a5f8e5c00 2023-05-20T01:18:30.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.133+0000 7fbd627367c0 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:106 8.11 8:88000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a5fa0b180 con 0x556a5f8c2800 2023-05-20T01:18:30.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.134+0000 7fbd5fee3700 1 -- 172.21.15.16:0/2447407827 <== osd.5 v2:172.21.15.132:6816/1844689995 13 ==== osd_op_reply(106 [pgnls start_epoch 47 out=81b] v45'1 uv1 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x556a5f8c1440 con 0x556a5f8c2800 2023-05-20T01:18:30.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.134+0000 7fbd627367c0 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:107 8.11 8:893be98d:::100000001f8.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a5fa0a700 con 0x556a5f8c2800 2023-05-20T01:18:30.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.134+0000 7fbd5fee3700 1 -- 172.21.15.16:0/2447407827 <== osd.5 v2:172.21.15.132:6816/1844689995 14 ==== osd_op_reply(107 100000001f8.00000001 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x556a5f8c1440 con 0x556a5f8c2800 2023-05-20T01:18:30.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.134+0000 7fbd627367c0 10 datascan.operator(): handling object 1099511628280.1 2023-05-20T01:18:30.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.134+0000 7fbd627367c0 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:108 7.19 7:997b7129:::100000001f8.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a5fa0ae00 con 0x556a5f8e5400 2023-05-20T01:18:30.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.136+0000 7fbd5fee3700 1 -- 172.21.15.16:0/2447407827 <== osd.6 v2:172.21.15.132:6801/727581123 18 ==== osd_op_reply(108 100000001f8.00000000 [call,setxattr (8)] v47'13 uv13 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x556a5f8c1440 con 0x556a5f8e5400 2023-05-20T01:18:30.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.136+0000 7fbd627367c0 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:109 8.9 8:90000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a60438700 con 0x556a5f8e5400 2023-05-20T01:18:30.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.136+0000 7fbd5fee3700 1 -- 172.21.15.16:0/2447407827 <== osd.6 v2:172.21.15.132:6801/727581123 19 ==== osd_op_reply(109 [pgnls start_epoch 47 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556a5f8c1440 con 0x556a5f8e5400 2023-05-20T01:18:30.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.136+0000 7fbd5b6da700 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:110 8.19 8:98000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a5fa26380 con 0x556a5f8c2800 2023-05-20T01:18:30.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.136+0000 7fbd5fee3700 1 -- 172.21.15.16:0/2447407827 <== osd.5 v2:172.21.15.132:6816/1844689995 15 ==== osd_op_reply(110 [pgnls start_epoch 47 out=81b] v45'2 uv2 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x556a5f8c1440 con 0x556a5f8c2800 2023-05-20T01:18:30.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.136+0000 7fbd627367c0 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:111 8.19 8:997b7129:::100000001f8.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a60438a80 con 0x556a5f8c2800 2023-05-20T01:18:30.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.137+0000 7fbd5fee3700 1 -- 172.21.15.16:0/2447407827 <== osd.5 v2:172.21.15.132:6816/1844689995 16 ==== osd_op_reply(111 100000001f8.00000000 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x556a5f8c1440 con 0x556a5f8c2800 2023-05-20T01:18:30.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.137+0000 7fbd627367c0 10 datascan.operator(): handling object 1099511628280.0 2023-05-20T01:18:30.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.137+0000 7fbd627367c0 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:112 7.19 7:997b7129:::100000001f8.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a60439880 con 0x556a5f8e5400 2023-05-20T01:18:30.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.138+0000 7fbd5fee3700 1 -- 172.21.15.16:0/2447407827 <== osd.6 v2:172.21.15.132:6801/727581123 20 ==== osd_op_reply(112 100000001f8.00000000 [call,setxattr (8)] v47'14 uv14 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x556a5f8c1440 con 0x556a5f8e5400 2023-05-20T01:18:30.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.138+0000 7fbd627367c0 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:113 8.5 8:a0000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a60439500 con 0x556a5f8e5400 2023-05-20T01:18:30.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.138+0000 7fbd5fee3700 1 -- 172.21.15.16:0/2447407827 <== osd.6 v2:172.21.15.132:6801/727581123 21 ==== osd_op_reply(113 [pgnls start_epoch 47 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556a5f8c1440 con 0x556a5f8e5400 2023-05-20T01:18:30.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.138+0000 7fbd5b6da700 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:114 8.15 8:a8000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a5fa26700 con 0x556a5fa20400 2023-05-20T01:18:30.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.139+0000 7fbd60ee5700 1 -- 172.21.15.16:0/2447407827 <== osd.7 v2:172.21.15.132:6800/466001149 8 ==== osd_op_reply(114 [pgnls start_epoch 47 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556a5f8c1680 con 0x556a5fa20400 2023-05-20T01:18:30.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.139+0000 7fbd5bedb700 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:115 8.d 8:b0000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a60439c00 con 0x556a5f8c2800 2023-05-20T01:18:30.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.139+0000 7fbd5fee3700 1 -- 172.21.15.16:0/2447407827 <== osd.5 v2:172.21.15.132:6816/1844689995 17 ==== osd_op_reply(115 [pgnls start_epoch 47 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556a5f8c1440 con 0x556a5f8c2800 2023-05-20T01:18:30.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.139+0000 7fbd5b6da700 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:116 8.1d 8:b8000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a5fa27180 con 0x556a5f8e5c00 2023-05-20T01:18:30.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.139+0000 7fbd60ee5700 1 -- 172.21.15.16:0/2447407827 <== osd.0 v2:172.21.15.16:6824/3923634179 30 ==== osd_op_reply(116 [pgnls start_epoch 47 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556a5f8c1680 con 0x556a5f8e5c00 2023-05-20T01:18:30.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.139+0000 7fbd5bedb700 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:117 8.3 8:c0000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a5fa27c00 con 0x556a5fa20000 2023-05-20T01:18:30.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.140+0000 7fbd5fee3700 1 -- 172.21.15.16:0/2447407827 <== osd.1 v2:172.21.15.16:6808/3860264220 8 ==== osd_op_reply(117 [pgnls start_epoch 47 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556a5f8c1440 con 0x556a5fa20000 2023-05-20T01:18:30.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.140+0000 7fbd5b6da700 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:118 8.13 8:c8000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a5fa0b500 con 0x556a5f8e5000 2023-05-20T01:18:30.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.140+0000 7fbd606e4700 1 -- 172.21.15.16:0/2447407827 <== osd.3 v2:172.21.15.16:6800/3669781890 16 ==== osd_op_reply(118 [pgnls start_epoch 47 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556a5f8c18c0 con 0x556a5f8e5000 2023-05-20T01:18:30.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.140+0000 7fbd5bedb700 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:119 8.b 8:d0000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a5fa26a80 con 0x556a5f8e5000 2023-05-20T01:18:30.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.140+0000 7fbd606e4700 1 -- 172.21.15.16:0/2447407827 <== osd.3 v2:172.21.15.16:6800/3669781890 17 ==== osd_op_reply(119 [pgnls start_epoch 47 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556a5f8c18c0 con 0x556a5f8e5000 2023-05-20T01:18:30.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.140+0000 7fbd5b6da700 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:120 8.1b 8:d8000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a5fa0b880 con 0x556a5f8e5800 2023-05-20T01:18:30.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.140+0000 7fbd606e4700 1 -- 172.21.15.16:0/2447407827 <== osd.4 v2:172.21.15.132:6817/506317223 13 ==== osd_op_reply(120 [pgnls start_epoch 47 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556a5f8c18c0 con 0x556a5f8e5800 2023-05-20T01:18:30.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.140+0000 7fbd5bedb700 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:121 8.7 8:e0000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a5fa26e00 con 0x556a5fa20000 2023-05-20T01:18:30.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.141+0000 7fbd5fee3700 1 -- 172.21.15.16:0/2447407827 <== osd.1 v2:172.21.15.16:6808/3860264220 9 ==== osd_op_reply(121 [pgnls start_epoch 47 out=81b] v45'3 uv3 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x556a5f8c1440 con 0x556a5fa20000 2023-05-20T01:18:30.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.141+0000 7fbd627367c0 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:122 8.7 8:e2e07930:::10000000001.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a60439180 con 0x556a5fa20000 2023-05-20T01:18:30.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.141+0000 7fbd5fee3700 1 -- 172.21.15.16:0/2447407827 <== osd.1 v2:172.21.15.16:6808/3860264220 10 ==== osd_op_reply(122 10000000001.00000000 [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x556a5f8c1440 con 0x556a5fa20000 2023-05-20T01:18:30.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.141+0000 7fbd627367c0 10 datascan.operator(): handling object 1099511627777.0 2023-05-20T01:18:30.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.141+0000 7fbd627367c0 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:123 7.7 7:e2e07930:::10000000001.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a60438e00 con 0x556a5f8e5400 2023-05-20T01:18:30.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.142+0000 7fbd5fee3700 1 -- 172.21.15.16:0/2447407827 <== osd.6 v2:172.21.15.132:6801/727581123 22 ==== osd_op_reply(123 10000000001.00000000 [call,setxattr (8)] v47'8 uv8 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x556a5f8c1440 con 0x556a5f8e5400 2023-05-20T01:18:30.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.142+0000 7fbd627367c0 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:124 8.17 8:e8000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a60438700 con 0x556a5fa20400 2023-05-20T01:18:30.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.142+0000 7fbd60ee5700 1 -- 172.21.15.16:0/2447407827 <== osd.7 v2:172.21.15.132:6800/466001149 9 ==== osd_op_reply(124 [pgnls start_epoch 47 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556a5f8c1680 con 0x556a5fa20400 2023-05-20T01:18:30.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.142+0000 7fbd5bedb700 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:125 8.f 8:f0000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a5fa0aa80 con 0x556a5f8e5000 2023-05-20T01:18:30.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.143+0000 7fbd606e4700 1 -- 172.21.15.16:0/2447407827 <== osd.3 v2:172.21.15.16:6800/3669781890 18 ==== osd_op_reply(125 [pgnls start_epoch 47 out=81b] v43'1 uv1 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x556a5f8c18c0 con 0x556a5f8e5000 2023-05-20T01:18:30.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.143+0000 7fbd627367c0 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:126 8.f 8:f526278d:::10000000001.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a60438a80 con 0x556a5f8e5000 2023-05-20T01:18:30.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.143+0000 7fbd606e4700 1 -- 172.21.15.16:0/2447407827 <== osd.3 v2:172.21.15.16:6800/3669781890 19 ==== osd_op_reply(126 10000000001.00000001 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x556a5f8c18c0 con 0x556a5f8e5000 2023-05-20T01:18:30.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.143+0000 7fbd627367c0 10 datascan.operator(): handling object 1099511627777.1 2023-05-20T01:18:30.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.143+0000 7fbd627367c0 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:127 7.7 7:e2e07930:::10000000001.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a60439880 con 0x556a5f8e5400 2023-05-20T01:18:30.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.144+0000 7fbd5fee3700 1 -- 172.21.15.16:0/2447407827 <== osd.6 v2:172.21.15.132:6801/727581123 23 ==== osd_op_reply(127 10000000001.00000000 [call,setxattr (8)] v47'9 uv9 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x556a5f8c1440 con 0x556a5f8e5400 2023-05-20T01:18:30.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.144+0000 7fbd627367c0 1 -- 172.21.15.16:0/2447407827 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:128 8.1f 8:f8000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x556a60439500 con 0x556a5f8e5c00 2023-05-20T01:18:30.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.144+0000 7fbd60ee5700 1 -- 172.21.15.16:0/2447407827 <== osd.0 v2:172.21.15.16:6824/3923634179 31 ==== osd_op_reply(128 [pgnls start_epoch 47 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556a5f8c1680 con 0x556a5f8e5c00 2023-05-20T01:18:30.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.144+0000 7fbd627367c0 1 -- 172.21.15.16:0/2447407827 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x556a5f8e5c00 msgr2=0x556a5f9ef600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:30.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.144+0000 7fbd627367c0 1 --2- 172.21.15.16:0/2447407827 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x556a5f8e5c00 0x556a5f9ef600 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:30.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.144+0000 7fbd627367c0 1 -- 172.21.15.16:0/2447407827 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x556a5fa20000 msgr2=0x556a5f9f0100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:30.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.144+0000 7fbd627367c0 1 --2- 172.21.15.16:0/2447407827 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x556a5fa20000 0x556a5f9f0100 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:30.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.144+0000 7fbd627367c0 1 -- 172.21.15.16:0/2447407827 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x556a5f8e4c00 msgr2=0x556a5f9ee000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:30.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.144+0000 7fbd627367c0 1 --2- 172.21.15.16:0/2447407827 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x556a5f8e4c00 0x556a5f9ee000 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:30.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.144+0000 7fbd627367c0 1 -- 172.21.15.16:0/2447407827 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x556a5f8e5000 msgr2=0x556a5f9ee580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:30.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.144+0000 7fbd627367c0 1 --2- 172.21.15.16:0/2447407827 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x556a5f8e5000 0x556a5f9ee580 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:30.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.144+0000 7fbd627367c0 1 -- 172.21.15.16:0/2447407827 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x556a5f8e5800 msgr2=0x556a5f9ef080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:30.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.144+0000 7fbd627367c0 1 --2- 172.21.15.16:0/2447407827 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x556a5f8e5800 0x556a5f9ef080 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:30.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.144+0000 7fbd627367c0 1 -- 172.21.15.16:0/2447407827 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x556a5f8c2800 msgr2=0x556a5f648100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:30.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.144+0000 7fbd627367c0 1 --2- 172.21.15.16:0/2447407827 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x556a5f8c2800 0x556a5f648100 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:30.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.144+0000 7fbd5fee3700 1 -- 172.21.15.16:0/2447407827 reap_dead start 2023-05-20T01:18:30.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.144+0000 7fbd627367c0 1 -- 172.21.15.16:0/2447407827 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x556a5f8e5400 msgr2=0x556a5f9eeb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:30.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.144+0000 7fbd627367c0 1 --2- 172.21.15.16:0/2447407827 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x556a5f8e5400 0x556a5f9eeb00 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:30.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.145+0000 7fbd5fee3700 1 -- 172.21.15.16:0/2447407827 reap_dead start 2023-05-20T01:18:30.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.145+0000 7fbd627367c0 1 -- 172.21.15.16:0/2447407827 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x556a5fa20400 msgr2=0x556a5f9f0680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:30.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.145+0000 7fbd627367c0 1 --2- 172.21.15.16:0/2447407827 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x556a5fa20400 0x556a5f9f0680 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:30.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.145+0000 7fbd627367c0 1 -- 172.21.15.16:0/2447407827 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x556a5f8e4800 msgr2=0x556a5f649700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:30.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.145+0000 7fbd627367c0 1 --2- 172.21.15.16:0/2447407827 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x556a5f8e4800 0x556a5f649700 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x556a5f8ca0f0 tx=0x556a60442e40 comp rx=0 tx=0).stop 2023-05-20T01:18:30.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.145+0000 7fbd627367c0 1 -- 172.21.15.16:0/2447407827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a5f8c2000 msgr2=0x556a5f648c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:30.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.145+0000 7fbd627367c0 1 --2- 172.21.15.16:0/2447407827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a5f8c2000 0x556a5f648c00 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x556a5f8cb410 tx=0x556a6041e060 comp rx=0 tx=0).stop 2023-05-20T01:18:30.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.145+0000 7fbd627367c0 1 -- 172.21.15.16:0/2447407827 shutdown_connections 2023-05-20T01:18:30.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.145+0000 7fbd627367c0 1 --2- 172.21.15.16:0/2447407827 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x556a5fa20400 0x556a5f9f0680 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:30.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.145+0000 7fbd627367c0 1 --2- 172.21.15.16:0/2447407827 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x556a5f8e5400 0x556a5f9eeb00 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:30.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.145+0000 7fbd627367c0 1 --2- 172.21.15.16:0/2447407827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a5f8c2000 0x556a5f648c00 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:30.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.145+0000 7fbd627367c0 1 --2- 172.21.15.16:0/2447407827 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x556a5f8e4800 0x556a5f649700 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:30.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.145+0000 7fbd627367c0 1 -- 172.21.15.16:0/2447407827 >> 172.21.15.16:0/2447407827 conn(0x556a5f5adc00 msgr2=0x556a5f689800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:18:30.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.145+0000 7fbd627367c0 1 -- 172.21.15.16:0/2447407827 shutdown_connections 2023-05-20T01:18:30.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.145+0000 7fbd627367c0 1 -- 172.21.15.16:0/2447407827 wait complete. 2023-05-20T01:18:30.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.145+0000 7fbd627367c0 1 -- 172.21.15.16:0/3486946117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a5f5ad000 msgr2=0x556a5f646580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:30.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.145+0000 7fbd627367c0 1 --2- 172.21.15.16:0/3486946117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a5f5ad000 0x556a5f646580 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x556a60378cc0 tx=0x556a6041e180 comp rx=0 tx=0).stop 2023-05-20T01:18:30.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.145+0000 7fbd627367c0 1 -- 172.21.15.16:0/3486946117 shutdown_connections 2023-05-20T01:18:30.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.145+0000 7fbd627367c0 1 --2- 172.21.15.16:0/3486946117 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556a5f5ac800 0x556a5f647080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:30.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.145+0000 7fbd627367c0 1 --2- 172.21.15.16:0/3486946117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a5f5ad000 0x556a5f646580 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:30.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.145+0000 7fbd627367c0 1 --2- 172.21.15.16:0/3486946117 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556a5f5acc00 0x556a5f646b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:30.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.145+0000 7fbd627367c0 1 -- 172.21.15.16:0/3486946117 >> 172.21.15.16:0/3486946117 conn(0x556a5f5ad400 msgr2=0x556a5f689000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:18:30.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.145+0000 7fbd627367c0 1 -- 172.21.15.16:0/3486946117 shutdown_connections 2023-05-20T01:18:30.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.145+0000 7fbd627367c0 1 -- 172.21.15.16:0/3486946117 wait complete. 2023-05-20T01:18:30.174 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_extents'] in time 0:00:00.169264, result: 2023-05-20T01:18:30.174 DEBUG:teuthology.orchestra.run.smithi016:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_inodes 2023-05-20T01:18:30.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.237+0000 7efc612697c0 1 -- start start 2023-05-20T01:18:30.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.237+0000 7efc612697c0 1 Processor -- start 2023-05-20T01:18:30.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.238+0000 7efc612697c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5636f6a3d000 0x5636f6ad6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:30.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.238+0000 7efc612697c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5636f6a3cc00 0x5636f6ad6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:30.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.238+0000 7efc612697c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636f6a3c800 0x5636f6ad7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:30.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.238+0000 7efc612697c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636f6a79cc0 con 0x5636f6a3c800 2023-05-20T01:18:30.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.238+0000 7efc612697c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636f6a79400 con 0x5636f6a3cc00 2023-05-20T01:18:30.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.238+0000 7efc612697c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x5636f6a79e00 con 0x5636f6a3d000 2023-05-20T01:18:30.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.238+0000 7efc5fa18700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636f6a3c800 0x5636f6ad7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:30.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.238+0000 7efc5fa18700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636f6a3c800 0x5636f6ad7080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60976/0 (socket says 172.21.15.16:60976) 2023-05-20T01:18:30.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.238+0000 7efc5fa18700 1 -- 172.21.15.16:0/3848076718 learned_addr learned my addr 172.21.15.16:0/3848076718 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:18:30.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.238+0000 7efc5f217700 1 --2- 172.21.15.16:0/3848076718 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5636f6a3d000 0x5636f6ad6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:30.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.238+0000 7efc5fa18700 1 -- 172.21.15.16:0/3848076718 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5636f6a3d000 msgr2=0x5636f6ad6580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:18:30.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.238+0000 7efc5ea16700 1 --2- 172.21.15.16:0/3848076718 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5636f6a3cc00 0x5636f6ad6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:30.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.238+0000 7efc5fa18700 1 --2- 172.21.15.16:0/3848076718 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5636f6a3d000 0x5636f6ad6580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:30.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.238+0000 7efc5fa18700 1 -- 172.21.15.16:0/3848076718 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5636f6a3cc00 msgr2=0x5636f6ad6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:18:30.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.238+0000 7efc5fa18700 1 --2- 172.21.15.16:0/3848076718 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5636f6a3cc00 0x5636f6ad6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:30.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.238+0000 7efc5fa18700 1 -- 172.21.15.16:0/3848076718 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5636f6a60d00 con 0x5636f6a3c800 2023-05-20T01:18:30.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.238+0000 7efc5f217700 1 --2- 172.21.15.16:0/3848076718 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5636f6a3d000 0x5636f6ad6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:18:30.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.238+0000 7efc5ea16700 1 --2- 172.21.15.16:0/3848076718 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5636f6a3cc00 0x5636f6ad6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:18:30.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.238+0000 7efc5fa18700 1 --2- 172.21.15.16:0/3848076718 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636f6a3c800 0x5636f6ad7080 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x5636f6b98de0 tx=0x5636f77d53e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fae8666a7399edca server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:30.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.238+0000 7efc5ca12700 1 -- 172.21.15.16:0/3848076718 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5636f6a71080 con 0x5636f6a3c800 2023-05-20T01:18:30.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.238+0000 7efc5ca12700 1 -- 172.21.15.16:0/3848076718 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5636f6a8e600 con 0x5636f6a3c800 2023-05-20T01:18:30.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.238+0000 7efc5ca12700 1 -- 172.21.15.16:0/3848076718 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5636f6a71600 con 0x5636f6a3c800 2023-05-20T01:18:30.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.238+0000 7efc612697c0 1 -- 172.21.15.16:0/3848076718 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5636f6a616c0 con 0x5636f6a3c800 2023-05-20T01:18:30.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.238+0000 7efc612697c0 1 -- 172.21.15.16:0/3848076718 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x5636f6a61520 con 0x5636f6a3c800 2023-05-20T01:18:30.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.238+0000 7efc612697c0 4 waiting for MDS map... 2023-05-20T01:18:30.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.238+0000 7efc5ca12700 1 -- 172.21.15.16:0/3848076718 <== mon.0 v2:172.21.15.16:3300/0 4 ==== osd_map(47..47 src has 1..47) v4 ==== 7016+0+0 (secure 0 0 0) 0x5636f6a7ec40 con 0x5636f6a3c800 2023-05-20T01:18:30.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.239+0000 7efc5ca12700 1 -- 172.21.15.16:0/3848076718 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 22) v1 ==== 3126+0+0 (secure 0 0 0) 0x5636f6a82f00 con 0x5636f6a3c800 2023-05-20T01:18:30.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.239+0000 7efc612697c0 4 Got MDS map 22 2023-05-20T01:18:30.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.239+0000 7efc612697c0 4 datascan.main: Using metadata pool output 2023-05-20T01:18:30.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.239+0000 7efc612697c0 4 datascan.main: connecting to RADOS... 2023-05-20T01:18:30.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.239+0000 7efc612697c0 1 Processor -- start 2023-05-20T01:18:30.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.239+0000 7efc612697c0 1 -- start start 2023-05-20T01:18:30.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.240+0000 7efc612697c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5636f6a3dc00 0x5636f6ad7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:30.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.240+0000 7efc612697c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5636f6d40000 0x5636f6ad7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:30.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.240+0000 7efc612697c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636f6d40400 0x5636f6ad8100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:30.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.240+0000 7efc612697c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636f6a78a00 con 0x5636f6d40400 2023-05-20T01:18:30.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.240+0000 7efc612697c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636f6a78b40 con 0x5636f6a3dc00 2023-05-20T01:18:30.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.240+0000 7efc612697c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x5636f6a79400 con 0x5636f6d40000 2023-05-20T01:18:30.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.240+0000 7efc5ea16700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636f6d40400 0x5636f6ad8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:30.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.240+0000 7efc5ea16700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636f6d40400 0x5636f6ad8100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60982/0 (socket says 172.21.15.16:60982) 2023-05-20T01:18:30.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.240+0000 7efc5ea16700 1 -- 172.21.15.16:0/99554684 learned_addr learned my addr 172.21.15.16:0/99554684 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:18:30.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.240+0000 7efc5f217700 1 --2- 172.21.15.16:0/99554684 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5636f6d40000 0x5636f6ad7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:30.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.240+0000 7efc5ea16700 1 --2- 172.21.15.16:0/99554684 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5636f6a3dc00 0x5636f6ad7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:30.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.240+0000 7efc5ea16700 1 -- 172.21.15.16:0/99554684 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5636f6d40000 msgr2=0x5636f6ad7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:18:30.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.240+0000 7efc5ea16700 1 --2- 172.21.15.16:0/99554684 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5636f6d40000 0x5636f6ad7b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:30.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.240+0000 7efc5ea16700 1 -- 172.21.15.16:0/99554684 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5636f6a3dc00 msgr2=0x5636f6ad7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:18:30.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.240+0000 7efc5ea16700 1 --2- 172.21.15.16:0/99554684 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5636f6a3dc00 0x5636f6ad7600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:30.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.240+0000 7efc5ea16700 1 -- 172.21.15.16:0/99554684 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5636f6a61380 con 0x5636f6d40400 2023-05-20T01:18:30.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.241+0000 7efc5ea16700 1 --2- 172.21.15.16:0/99554684 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636f6d40400 0x5636f6ad8100 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x5636f6d520f0 tx=0x5636f78b8e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0afbaef7f7860c0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:30.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.241+0000 7efc5aa0e700 1 -- 172.21.15.16:0/99554684 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5636f6a71080 con 0x5636f6d40400 2023-05-20T01:18:30.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.241+0000 7efc5aa0e700 1 -- 172.21.15.16:0/99554684 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5636f6a8fc80 con 0x5636f6d40400 2023-05-20T01:18:30.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.241+0000 7efc5aa0e700 1 -- 172.21.15.16:0/99554684 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5636f6a71600 con 0x5636f6d40400 2023-05-20T01:18:30.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.241+0000 7efc612697c0 1 -- 172.21.15.16:0/99554684 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636f6d40400 msgr2=0x5636f6ad8100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:30.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.241+0000 7efc612697c0 1 --2- 172.21.15.16:0/99554684 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636f6d40400 0x5636f6ad8100 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x5636f6d520f0 tx=0x5636f78b8e40 comp rx=0 tx=0).stop 2023-05-20T01:18:30.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.241+0000 7efc612697c0 1 -- 172.21.15.16:0/99554684 shutdown_connections 2023-05-20T01:18:30.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.241+0000 7efc612697c0 1 --2- 172.21.15.16:0/99554684 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636f6d40400 0x5636f6ad8100 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:30.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.241+0000 7efc612697c0 1 --2- 172.21.15.16:0/99554684 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5636f6a3dc00 0x5636f6ad7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:30.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.241+0000 7efc612697c0 1 --2- 172.21.15.16:0/99554684 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5636f6d40000 0x5636f6ad7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:30.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.241+0000 7efc612697c0 1 -- 172.21.15.16:0/99554684 >> 172.21.15.16:0/99554684 conn(0x5636f6a3c400 msgr2=0x5636f6b0d800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:18:30.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.241+0000 7efc612697c0 1 -- 172.21.15.16:0/99554684 shutdown_connections 2023-05-20T01:18:30.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.241+0000 7efc612697c0 1 -- 172.21.15.16:0/99554684 wait complete. 2023-05-20T01:18:30.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.242+0000 7efc612697c0 1 Processor -- start 2023-05-20T01:18:30.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.242+0000 7efc612697c0 1 -- start start 2023-05-20T01:18:30.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.242+0000 7efc612697c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5636f6a3c400 0x5636f6ad9180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:30.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.242+0000 7efc612697c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636f6d4c000 0x5636f6ad8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:30.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.242+0000 7efc612697c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5636f6d4c400 0x5636f6ad8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:30.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.242+0000 7efc612697c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636f6a79cc0 con 0x5636f6d4c000 2023-05-20T01:18:30.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.242+0000 7efc612697c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636f6a78c80 con 0x5636f6d4c400 2023-05-20T01:18:30.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.242+0000 7efc612697c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x5636f6a78000 con 0x5636f6a3c400 2023-05-20T01:18:30.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.242+0000 7efc5f217700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636f6d4c000 0x5636f6ad8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:30.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.242+0000 7efc5f217700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636f6d4c000 0x5636f6ad8c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60984/0 (socket says 172.21.15.16:60984) 2023-05-20T01:18:30.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.242+0000 7efc5f217700 1 -- 172.21.15.16:0/3368616031 learned_addr learned my addr 172.21.15.16:0/3368616031 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:18:30.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.243+0000 7efc5ea16700 1 --2- 172.21.15.16:0/3368616031 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5636f6a3c400 0x5636f6ad9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:30.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.243+0000 7efc5ea16700 1 --2- 172.21.15.16:0/3368616031 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5636f6d4c400 0x5636f6ad8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:30.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.243+0000 7efc5f217700 1 -- 172.21.15.16:0/3368616031 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5636f6a3c400 msgr2=0x5636f6ad9180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:30.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.243+0000 7efc5f217700 1 --2- 172.21.15.16:0/3368616031 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5636f6a3c400 0x5636f6ad9180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:30.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.243+0000 7efc5f217700 1 -- 172.21.15.16:0/3368616031 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5636f6d4c400 msgr2=0x5636f6ad8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:30.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.243+0000 7efc5f217700 1 --2- 172.21.15.16:0/3368616031 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5636f6d4c400 0x5636f6ad8680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:30.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.243+0000 7efc5f217700 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5636f6a611e0 con 0x5636f6d4c000 2023-05-20T01:18:30.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.243+0000 7efc5f217700 1 --2- 172.21.15.16:0/3368616031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636f6d4c000 0x5636f6ad8c00 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x5636f6d53410 tx=0x5636f78b8ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:30.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.243+0000 7efc5920b700 1 -- 172.21.15.16:0/3368616031 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5636f6a71080 con 0x5636f6d4c000 2023-05-20T01:18:30.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.243+0000 7efc5920b700 1 -- 172.21.15.16:0/3368616031 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5636f7898780 con 0x5636f6d4c000 2023-05-20T01:18:30.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.243+0000 7efc612697c0 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5636f6a61d40 con 0x5636f6d4c000 2023-05-20T01:18:30.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.243+0000 7efc5920b700 1 -- 172.21.15.16:0/3368616031 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5636f6a71600 con 0x5636f6d4c000 2023-05-20T01:18:30.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.243+0000 7efc612697c0 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5636f6a60d00 con 0x5636f6d4c000 2023-05-20T01:18:30.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.243+0000 7efc612697c0 4 datascan.init: resolving metadata pool 6 2023-05-20T01:18:30.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.244+0000 7efc5920b700 1 -- 172.21.15.16:0/3368616031 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x5636f6a9eb00 con 0x5636f6d4c000 2023-05-20T01:18:30.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.245+0000 7efc5920b700 1 --2- 172.21.15.16:0/3368616031 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x5636f6d80400 0x5636f6ad9700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:30.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.245+0000 7efc5920b700 1 -- 172.21.15.16:0/3368616031 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 7016+0+0 (secure 0 0 0) 0x5636f6a7ea80 con 0x5636f6d4c000 2023-05-20T01:18:30.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.245+0000 7efc612697c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-05-20T01:18:30.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.245+0000 7efc5ea16700 1 --2- 172.21.15.16:0/3368616031 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x5636f6d80400 0x5636f6ad9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:30.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.245+0000 7efc612697c0 4 datascan.main: data pool 'cephfs_data' has ID 7 2023-05-20T01:18:30.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.245+0000 7efc612697c0 4 datascan.main: opening data pool 'cephfs_data' 2023-05-20T01:18:30.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.245+0000 7efc612697c0 1 --2- 172.21.15.16:0/3368616031 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x5636f6d4c800 0x5636f6ad8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:30.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.245+0000 7efc612697c0 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:1 6.14 6:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f7802700 con 0x5636f6d4c800 2023-05-20T01:18:30.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.245+0000 7efc5ea16700 1 --2- 172.21.15.16:0/3368616031 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x5636f6d4c800 0x5636f6ad8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:30.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.245+0000 7efc5ea16700 1 --2- 172.21.15.16:0/3368616031 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x5636f6d80400 0x5636f6ad9700 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x5636f6d52240 tx=0x5636f78b8e40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:30.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.245+0000 7efc5ea16700 1 --2- 172.21.15.16:0/3368616031 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x5636f6d4c800 0x5636f6ad8100 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:30.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.246+0000 7efc5ea16700 1 -- 172.21.15.16:0/3368616031 <== osd.2 v2:172.21.15.16:6816/293187030 1 ==== osd_op_reply(1 1.00000000.inode [stat out=16b] v0'0 uv49 ondisk = 0) v8 ==== 160+0+16 (crc 0 0 0) 0x5636f6d3b440 con 0x5636f6d4c800 2023-05-20T01:18:30.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.246+0000 7efc612697c0 1 --2- 172.21.15.16:0/3368616031 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x5636f6d4cc00 0x5636f6ad7b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:30.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.246+0000 7efc612697c0 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:2 6.1f 6:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f7802a80 con 0x5636f6d4cc00 2023-05-20T01:18:30.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.246+0000 7efc5fa18700 1 --2- 172.21.15.16:0/3368616031 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x5636f6d4cc00 0x5636f6ad7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:30.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.246+0000 7efc5fa18700 1 --2- 172.21.15.16:0/3368616031 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x5636f6d4cc00 0x5636f6ad7b80 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:30.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.246+0000 7efc5fa18700 1 -- 172.21.15.16:0/3368616031 <== osd.3 v2:172.21.15.16:6800/3669781890 1 ==== osd_op_reply(2 100.00000000.inode [stat out=16b] v0'0 uv10 ondisk = 0) v8 ==== 162+0+16 (crc 0 0 0) 0x5636f6d3b680 con 0x5636f6d4cc00 2023-05-20T01:18:30.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.246+0000 7efc612697c0 1 --2- 172.21.15.16:0/3368616031 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x5636f6d4d000 0x5636f6ad7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:30.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.246+0000 7efc612697c0 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:3 7.0 7:00000000::::head [pgnls-filter start_epoch 47 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f7802e00 con 0x5636f6d4d000 2023-05-20T01:18:30.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.247+0000 7efc5f217700 1 --2- 172.21.15.16:0/3368616031 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x5636f6d4d000 0x5636f6ad7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:30.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.247+0000 7efc5f217700 1 --2- 172.21.15.16:0/3368616031 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x5636f6d4d000 0x5636f6ad7600 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:30.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.247+0000 7efc5f217700 1 -- 172.21.15.16:0/3368616031 <== osd.5 v2:172.21.15.132:6816/1844689995 1 ==== osd_op_reply(3 [pgnls-filter start_epoch 47 out=49b] v45'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636f6d3b8c0 con 0x5636f6d4d000 2023-05-20T01:18:30.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.247+0000 7efc5a20d700 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:4 7.20 7:04000000::::head [pgnls-filter start_epoch 47 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f7803180 con 0x5636f6d4c800 2023-05-20T01:18:30.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.248+0000 7efc5ea16700 1 -- 172.21.15.16:0/3368616031 <== osd.2 v2:172.21.15.16:6816/293187030 2 ==== osd_op_reply(4 [pgnls-filter start_epoch 47 out=49b] v45'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636f6d3b440 con 0x5636f6d4c800 2023-05-20T01:18:30.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.248+0000 7efc5aa0e700 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:5 7.10 7:08000000::::head [pgnls-filter start_epoch 47 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f7802700 con 0x5636f6d4cc00 2023-05-20T01:18:30.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.248+0000 7efc5fa18700 1 -- 172.21.15.16:0/3368616031 <== osd.3 v2:172.21.15.16:6800/3669781890 2 ==== osd_op_reply(5 [pgnls-filter start_epoch 47 out=49b] v45'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636f6d3b680 con 0x5636f6d4cc00 2023-05-20T01:18:30.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.248+0000 7efc5a20d700 1 --2- 172.21.15.16:0/3368616031 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x5636f6d80800 0x5636f6e9e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:30.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.248+0000 7efc5a20d700 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:6 7.30 7:0c000000::::head [pgnls-filter start_epoch 47 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f7803180 con 0x5636f6d80800 2023-05-20T01:18:30.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.248+0000 7efc5ea16700 1 --2- 172.21.15.16:0/3368616031 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x5636f6d80800 0x5636f6e9e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:30.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.249+0000 7efc5ea16700 1 --2- 172.21.15.16:0/3368616031 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x5636f6d80800 0x5636f6e9e000 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:30.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.249+0000 7efc5ea16700 1 -- 172.21.15.16:0/3368616031 <== osd.6 v2:172.21.15.132:6801/727581123 1 ==== osd_op_reply(6 [pgnls-filter start_epoch 47 out=49b] v45'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636f6d3b440 con 0x5636f6d80800 2023-05-20T01:18:30.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.249+0000 7efc5aa0e700 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:7 7.8 7:10000000::::head [pgnls-filter start_epoch 47 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f7803880 con 0x5636f6d4c800 2023-05-20T01:18:30.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.249+0000 7efc5ea16700 1 -- 172.21.15.16:0/3368616031 <== osd.2 v2:172.21.15.16:6816/293187030 3 ==== osd_op_reply(7 [pgnls-filter start_epoch 47 out=49b] v45'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636f6d3b440 con 0x5636f6d4c800 2023-05-20T01:18:30.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.249+0000 7efc5a20d700 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:8 7.28 7:14000000::::head [pgnls-filter start_epoch 47 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f7803500 con 0x5636f6d80800 2023-05-20T01:18:30.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.250+0000 7efc5ea16700 1 -- 172.21.15.16:0/3368616031 <== osd.6 v2:172.21.15.132:6801/727581123 2 ==== osd_op_reply(8 [pgnls-filter start_epoch 47 out=49b] v45'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636f6d3b440 con 0x5636f6d80800 2023-05-20T01:18:30.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.250+0000 7efc5aa0e700 1 --2- 172.21.15.16:0/3368616031 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x5636f6d80c00 0x5636f6e9e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:30.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.250+0000 7efc5aa0e700 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:9 7.18 7:18000000::::head [pgnls-filter start_epoch 47 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f7803c00 con 0x5636f6d80c00 2023-05-20T01:18:30.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.250+0000 7efc5fa18700 1 --2- 172.21.15.16:0/3368616031 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x5636f6d80c00 0x5636f6e9e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:30.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.250+0000 7efc5fa18700 1 --2- 172.21.15.16:0/3368616031 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x5636f6d80c00 0x5636f6e9e580 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:30.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.250+0000 7efc5fa18700 1 -- 172.21.15.16:0/3368616031 <== osd.0 v2:172.21.15.16:6824/3923634179 1 ==== osd_op_reply(9 [pgnls-filter start_epoch 47 out=49b] v45'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636f6d3b680 con 0x5636f6d80c00 2023-05-20T01:18:30.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.250+0000 7efc5a20d700 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:10 7.38 7:1c000000::::head [pgnls-filter start_epoch 47 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f7802a80 con 0x5636f6d80c00 2023-05-20T01:18:30.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.251+0000 7efc5fa18700 1 -- 172.21.15.16:0/3368616031 <== osd.0 v2:172.21.15.16:6824/3923634179 2 ==== osd_op_reply(10 [pgnls-filter start_epoch 47 out=49b] v45'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636f6d3b680 con 0x5636f6d80c00 2023-05-20T01:18:30.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.251+0000 7efc5aa0e700 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:11 7.4 7:20000000::::head [pgnls-filter start_epoch 47 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f7803180 con 0x5636f6d80c00 2023-05-20T01:18:30.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.251+0000 7efc5fa18700 1 -- 172.21.15.16:0/3368616031 <== osd.0 v2:172.21.15.16:6824/3923634179 3 ==== osd_op_reply(11 [pgnls-filter start_epoch 47 out=49b] v45'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636f6d3b680 con 0x5636f6d80c00 2023-05-20T01:18:30.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.251+0000 7efc5a20d700 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:12 7.24 7:24000000::::head [pgnls-filter start_epoch 47 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f7802700 con 0x5636f6d4d000 2023-05-20T01:18:30.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.251+0000 7efc5f217700 1 -- 172.21.15.16:0/3368616031 <== osd.5 v2:172.21.15.132:6816/1844689995 2 ==== osd_op_reply(12 [pgnls-filter start_epoch 47 out=49b] v45'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636f6d3b8c0 con 0x5636f6d4d000 2023-05-20T01:18:30.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.251+0000 7efc5aa0e700 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:13 7.14 7:28000000::::head [pgnls-filter start_epoch 47 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f7803880 con 0x5636f6d4cc00 2023-05-20T01:18:30.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.251+0000 7efc5fa18700 1 -- 172.21.15.16:0/3368616031 <== osd.3 v2:172.21.15.16:6800/3669781890 3 ==== osd_op_reply(13 [pgnls-filter start_epoch 47 out=49b] v45'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636f6d3b680 con 0x5636f6d4cc00 2023-05-20T01:18:30.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.251+0000 7efc5a20d700 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:14 7.34 7:2c000000::::head [pgnls-filter start_epoch 47 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f7803c00 con 0x5636f6d4d000 2023-05-20T01:18:30.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.252+0000 7efc5f217700 1 -- 172.21.15.16:0/3368616031 <== osd.5 v2:172.21.15.132:6816/1844689995 3 ==== osd_op_reply(14 [pgnls-filter start_epoch 47 out=49b] v45'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636f6d3b8c0 con 0x5636f6d4d000 2023-05-20T01:18:30.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.252+0000 7efc5aa0e700 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:15 7.c 7:30000000::::head [pgnls-filter start_epoch 47 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f7803500 con 0x5636f6d80800 2023-05-20T01:18:30.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.252+0000 7efc5ea16700 1 -- 172.21.15.16:0/3368616031 <== osd.6 v2:172.21.15.132:6801/727581123 3 ==== osd_op_reply(15 [pgnls-filter start_epoch 47 out=49b] v45'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636f6d3b440 con 0x5636f6d80800 2023-05-20T01:18:30.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.252+0000 7efc5a20d700 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:16 7.2c 7:34000000::::head [pgnls-filter start_epoch 47 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f7802e00 con 0x5636f6d4c800 2023-05-20T01:18:30.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.253+0000 7efc5ea16700 1 -- 172.21.15.16:0/3368616031 <== osd.2 v2:172.21.15.16:6816/293187030 4 ==== osd_op_reply(16 [pgnls-filter start_epoch 47 out=49b] v45'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636f6d3b440 con 0x5636f6d4c800 2023-05-20T01:18:30.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.253+0000 7efc5aa0e700 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:17 7.1c 7:38000000::::head [pgnls-filter start_epoch 47 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f6ea2000 con 0x5636f6d4d000 2023-05-20T01:18:30.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.253+0000 7efc5f217700 1 -- 172.21.15.16:0/3368616031 <== osd.5 v2:172.21.15.132:6816/1844689995 4 ==== osd_op_reply(17 [pgnls-filter start_epoch 47 out=49b] v45'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636f6d3b8c0 con 0x5636f6d4d000 2023-05-20T01:18:30.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.253+0000 7efc5a20d700 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:18 7.3c 7:3c000000::::head [pgnls-filter start_epoch 47 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f7802700 con 0x5636f6d80c00 2023-05-20T01:18:30.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.253+0000 7efc5fa18700 1 -- 172.21.15.16:0/3368616031 <== osd.0 v2:172.21.15.16:6824/3923634179 4 ==== osd_op_reply(18 [pgnls-filter start_epoch 47 out=49b] v45'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636f6d3b680 con 0x5636f6d80c00 2023-05-20T01:18:30.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.253+0000 7efc5aa0e700 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:19 7.2 7:40000000::::head [pgnls-filter start_epoch 47 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f6ea2380 con 0x5636f6d4cc00 2023-05-20T01:18:30.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.254+0000 7efc5fa18700 1 -- 172.21.15.16:0/3368616031 <== osd.3 v2:172.21.15.16:6800/3669781890 4 ==== osd_op_reply(19 [pgnls-filter start_epoch 47 out=49b] v45'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636f6d3b680 con 0x5636f6d4cc00 2023-05-20T01:18:30.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.254+0000 7efc5a20d700 1 --2- 172.21.15.16:0/3368616031 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x5636f6d81000 0x5636f6e9eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:30.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.254+0000 7efc5a20d700 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:20 7.22 7:44000000::::head [pgnls-filter start_epoch 47 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f6ea2e00 con 0x5636f6d81000 2023-05-20T01:18:30.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.254+0000 7efc5f217700 1 --2- 172.21.15.16:0/3368616031 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x5636f6d81000 0x5636f6e9eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:30.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.254+0000 7efc5f217700 1 --2- 172.21.15.16:0/3368616031 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x5636f6d81000 0x5636f6e9eb00 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:30.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.255+0000 7efc5f217700 1 -- 172.21.15.16:0/3368616031 <== osd.4 v2:172.21.15.132:6817/506317223 1 ==== osd_op_reply(20 [pgnls-filter start_epoch 47 out=49b] v45'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636f6d3b8c0 con 0x5636f6d81000 2023-05-20T01:18:30.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.255+0000 7efc5aa0e700 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:21 7.12 7:48000000::::head [pgnls-filter start_epoch 47 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f6ea2700 con 0x5636f6d80800 2023-05-20T01:18:30.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.255+0000 7efc5ea16700 1 -- 172.21.15.16:0/3368616031 <== osd.6 v2:172.21.15.132:6801/727581123 4 ==== osd_op_reply(21 [pgnls-filter start_epoch 47 out=49b] v45'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636f6d3b440 con 0x5636f6d80800 2023-05-20T01:18:30.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.255+0000 7efc5a20d700 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:22 7.32 7:4c000000::::head [pgnls-filter start_epoch 47 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f7802a80 con 0x5636f6d80c00 2023-05-20T01:18:30.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.255+0000 7efc5fa18700 1 -- 172.21.15.16:0/3368616031 <== osd.0 v2:172.21.15.16:6824/3923634179 5 ==== osd_op_reply(22 [pgnls-filter start_epoch 47 out=49b] v45'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636f6d3b680 con 0x5636f6d80c00 2023-05-20T01:18:30.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.255+0000 7efc5aa0e700 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:23 7.a 7:50000000::::head [pgnls-filter start_epoch 47 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f6ea2a80 con 0x5636f6d4cc00 2023-05-20T01:18:30.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.256+0000 7efc5fa18700 1 -- 172.21.15.16:0/3368616031 <== osd.3 v2:172.21.15.16:6800/3669781890 5 ==== osd_op_reply(23 [pgnls-filter start_epoch 47 out=49b] v45'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636f6d3b680 con 0x5636f6d4cc00 2023-05-20T01:18:30.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.256+0000 7efc5a20d700 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:24 7.2a 7:54000000::::head [pgnls-filter start_epoch 47 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f7803180 con 0x5636f6d80c00 2023-05-20T01:18:30.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.256+0000 7efc5fa18700 1 -- 172.21.15.16:0/3368616031 <== osd.0 v2:172.21.15.16:6824/3923634179 6 ==== osd_op_reply(24 [pgnls-filter start_epoch 47 out=81b] v47'13 uv13 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5636f6d3b680 con 0x5636f6d80c00 2023-05-20T01:18:30.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.256+0000 7efc612697c0 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:25 7.0 7:00000000::::head [pgnls-filter start_epoch 47 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f6ea2000 con 0x5636f6d4d000 2023-05-20T01:18:30.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.256+0000 7efc5f217700 1 -- 172.21.15.16:0/3368616031 <== osd.5 v2:172.21.15.132:6816/1844689995 5 ==== osd_op_reply(25 [pgnls-filter start_epoch 47 out=49b] v45'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636f6d3b8c0 con 0x5636f6d4d000 2023-05-20T01:18:30.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.256+0000 7efc5a20d700 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:26 7.20 7:04000000::::head [pgnls-filter start_epoch 47 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f7803880 con 0x5636f6d4c800 2023-05-20T01:18:30.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.256+0000 7efc5ea16700 1 -- 172.21.15.16:0/3368616031 <== osd.2 v2:172.21.15.16:6816/293187030 5 ==== osd_op_reply(26 [pgnls-filter start_epoch 47 out=49b] v45'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636f6d3b440 con 0x5636f6d4c800 2023-05-20T01:18:30.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.256+0000 7efc5aa0e700 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:27 7.10 7:08000000::::head [pgnls-filter start_epoch 47 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f6ea2700 con 0x5636f6d4cc00 2023-05-20T01:18:30.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.257+0000 7efc5fa18700 1 -- 172.21.15.16:0/3368616031 <== osd.3 v2:172.21.15.16:6800/3669781890 6 ==== osd_op_reply(27 [pgnls-filter start_epoch 47 out=49b] v45'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636f6d3b680 con 0x5636f6d4cc00 2023-05-20T01:18:30.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.257+0000 7efc5a20d700 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:28 7.30 7:0c000000::::head [pgnls-filter start_epoch 47 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f7802700 con 0x5636f6d80800 2023-05-20T01:18:30.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.257+0000 7efc5ea16700 1 -- 172.21.15.16:0/3368616031 <== osd.6 v2:172.21.15.132:6801/727581123 5 ==== osd_op_reply(28 [pgnls-filter start_epoch 47 out=49b] v45'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636f6d3b440 con 0x5636f6d80800 2023-05-20T01:18:30.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.257+0000 7efc5aa0e700 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:29 7.8 7:10000000::::head [pgnls-filter start_epoch 47 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f6ea3500 con 0x5636f6d4c800 2023-05-20T01:18:30.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.257+0000 7efc5ea16700 1 -- 172.21.15.16:0/3368616031 <== osd.2 v2:172.21.15.16:6816/293187030 6 ==== osd_op_reply(29 [pgnls-filter start_epoch 47 out=49b] v45'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636f6d3b440 con 0x5636f6d4c800 2023-05-20T01:18:30.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.257+0000 7efc5a20d700 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:30 7.28 7:14000000::::head [pgnls-filter start_epoch 47 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f6ea2380 con 0x5636f6d80800 2023-05-20T01:18:30.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.258+0000 7efc5ea16700 1 -- 172.21.15.16:0/3368616031 <== osd.6 v2:172.21.15.132:6801/727581123 6 ==== osd_op_reply(30 [pgnls-filter start_epoch 47 out=49b] v45'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636f6d3b440 con 0x5636f6d80800 2023-05-20T01:18:30.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.258+0000 7efc5aa0e700 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:31 7.18 7:18000000::::head [pgnls-filter start_epoch 47 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f7803500 con 0x5636f6d80c00 2023-05-20T01:18:30.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.258+0000 7efc5fa18700 1 -- 172.21.15.16:0/3368616031 <== osd.0 v2:172.21.15.16:6824/3923634179 7 ==== osd_op_reply(31 [pgnls-filter start_epoch 47 out=49b] v45'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636f6d3b680 con 0x5636f6d80c00 2023-05-20T01:18:30.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.258+0000 7efc5a20d700 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:32 7.38 7:1c000000::::head [pgnls-filter start_epoch 47 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f6ea2a80 con 0x5636f6d80c00 2023-05-20T01:18:30.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.259+0000 7efc5fa18700 1 -- 172.21.15.16:0/3368616031 <== osd.0 v2:172.21.15.16:6824/3923634179 8 ==== osd_op_reply(32 [pgnls-filter start_epoch 47 out=49b] v45'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636f6d3b680 con 0x5636f6d80c00 2023-05-20T01:18:30.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.259+0000 7efc5aa0e700 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:33 7.4 7:20000000::::head [pgnls-filter start_epoch 47 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f7802e00 con 0x5636f6d80c00 2023-05-20T01:18:30.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.259+0000 7efc5fa18700 1 -- 172.21.15.16:0/3368616031 <== osd.0 v2:172.21.15.16:6824/3923634179 9 ==== osd_op_reply(33 [pgnls-filter start_epoch 47 out=49b] v45'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636f6d3b680 con 0x5636f6d80c00 2023-05-20T01:18:30.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.259+0000 7efc5a20d700 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:34 7.24 7:24000000::::head [pgnls-filter start_epoch 47 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f6ea2700 con 0x5636f6d4d000 2023-05-20T01:18:30.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.259+0000 7efc5f217700 1 -- 172.21.15.16:0/3368616031 <== osd.5 v2:172.21.15.132:6816/1844689995 6 ==== osd_op_reply(34 [pgnls-filter start_epoch 47 out=49b] v45'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636f6d3b8c0 con 0x5636f6d4d000 2023-05-20T01:18:30.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.259+0000 7efc5aa0e700 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:35 7.14 7:28000000::::head [pgnls-filter start_epoch 47 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f7803880 con 0x5636f6d4cc00 2023-05-20T01:18:30.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.260+0000 7efc5fa18700 1 -- 172.21.15.16:0/3368616031 <== osd.3 v2:172.21.15.16:6800/3669781890 7 ==== osd_op_reply(35 [pgnls-filter start_epoch 47 out=49b] v45'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636f6d3b680 con 0x5636f6d4cc00 2023-05-20T01:18:30.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.260+0000 7efc5a20d700 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:36 7.34 7:2c000000::::head [pgnls-filter start_epoch 47 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f6ea3880 con 0x5636f6d4d000 2023-05-20T01:18:30.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.260+0000 7efc5f217700 1 -- 172.21.15.16:0/3368616031 <== osd.5 v2:172.21.15.132:6816/1844689995 7 ==== osd_op_reply(36 [pgnls-filter start_epoch 47 out=49b] v45'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636f6d3b8c0 con 0x5636f6d4d000 2023-05-20T01:18:30.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.260+0000 7efc5aa0e700 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:37 7.c 7:30000000::::head [pgnls-filter start_epoch 47 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f6ea3c00 con 0x5636f6d80800 2023-05-20T01:18:30.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.260+0000 7efc5ea16700 1 -- 172.21.15.16:0/3368616031 <== osd.6 v2:172.21.15.132:6801/727581123 7 ==== osd_op_reply(37 [pgnls-filter start_epoch 47 out=49b] v45'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636f6d3b440 con 0x5636f6d80800 2023-05-20T01:18:30.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.260+0000 7efc5a20d700 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:38 7.2c 7:34000000::::head [pgnls-filter start_epoch 47 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f7802a80 con 0x5636f6d4c800 2023-05-20T01:18:30.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.261+0000 7efc5ea16700 1 -- 172.21.15.16:0/3368616031 <== osd.2 v2:172.21.15.16:6816/293187030 7 ==== osd_op_reply(38 [pgnls-filter start_epoch 47 out=49b] v45'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636f6d3b440 con 0x5636f6d4c800 2023-05-20T01:18:30.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.261+0000 7efc5aa0e700 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:39 7.1c 7:38000000::::head [pgnls-filter start_epoch 47 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f7802700 con 0x5636f6d4d000 2023-05-20T01:18:30.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.261+0000 7efc5f217700 1 -- 172.21.15.16:0/3368616031 <== osd.5 v2:172.21.15.132:6816/1844689995 8 ==== osd_op_reply(39 [pgnls-filter start_epoch 47 out=49b] v45'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636f6d3b8c0 con 0x5636f6d4d000 2023-05-20T01:18:30.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.261+0000 7efc5a20d700 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:40 7.3c 7:3c000000::::head [pgnls-filter start_epoch 47 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f7803180 con 0x5636f6d80c00 2023-05-20T01:18:30.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.261+0000 7efc5fa18700 1 -- 172.21.15.16:0/3368616031 <== osd.0 v2:172.21.15.16:6824/3923634179 10 ==== osd_op_reply(40 [pgnls-filter start_epoch 47 out=49b] v45'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636f6d3b680 con 0x5636f6d80c00 2023-05-20T01:18:30.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.261+0000 7efc5aa0e700 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:41 7.2 7:40000000::::head [pgnls-filter start_epoch 47 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f7802a80 con 0x5636f6d4cc00 2023-05-20T01:18:30.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.261+0000 7efc5fa18700 1 -- 172.21.15.16:0/3368616031 <== osd.3 v2:172.21.15.16:6800/3669781890 8 ==== osd_op_reply(41 [pgnls-filter start_epoch 47 out=49b] v45'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636f6d3b680 con 0x5636f6d4cc00 2023-05-20T01:18:30.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.262+0000 7efc5a20d700 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:42 7.22 7:44000000::::head [pgnls-filter start_epoch 47 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f7803500 con 0x5636f6d81000 2023-05-20T01:18:30.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.262+0000 7efc5f217700 1 -- 172.21.15.16:0/3368616031 <== osd.4 v2:172.21.15.132:6817/506317223 2 ==== osd_op_reply(42 [pgnls-filter start_epoch 47 out=49b] v45'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636f6d3b8c0 con 0x5636f6d81000 2023-05-20T01:18:30.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.262+0000 7efc5aa0e700 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:43 7.12 7:48000000::::head [pgnls-filter start_epoch 47 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f6ea3500 con 0x5636f6d80800 2023-05-20T01:18:30.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.262+0000 7efc5ea16700 1 -- 172.21.15.16:0/3368616031 <== osd.6 v2:172.21.15.132:6801/727581123 8 ==== osd_op_reply(43 [pgnls-filter start_epoch 47 out=49b] v45'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636f6d3b440 con 0x5636f6d80800 2023-05-20T01:18:30.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.262+0000 7efc5a20d700 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:44 7.32 7:4c000000::::head [pgnls-filter start_epoch 47 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f7802e00 con 0x5636f6d80c00 2023-05-20T01:18:30.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.262+0000 7efc5fa18700 1 -- 172.21.15.16:0/3368616031 <== osd.0 v2:172.21.15.16:6824/3923634179 11 ==== osd_op_reply(44 [pgnls-filter start_epoch 47 out=49b] v45'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636f6d3b680 con 0x5636f6d80c00 2023-05-20T01:18:30.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.262+0000 7efc5aa0e700 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:45 7.a 7:50000000::::head [pgnls-filter start_epoch 47 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f6ea2380 con 0x5636f6d4cc00 2023-05-20T01:18:30.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.263+0000 7efc5fa18700 1 -- 172.21.15.16:0/3368616031 <== osd.3 v2:172.21.15.16:6800/3669781890 9 ==== osd_op_reply(45 [pgnls-filter start_epoch 47 out=49b] v45'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636f6d3b680 con 0x5636f6d4cc00 2023-05-20T01:18:30.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.263+0000 7efc5a20d700 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:46 7.2a 7:54000000::::head [pgnls-filter start_epoch 47 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f7803880 con 0x5636f6d80c00 2023-05-20T01:18:30.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.263+0000 7efc5fa18700 1 -- 172.21.15.16:0/3368616031 <== osd.0 v2:172.21.15.16:6824/3923634179 12 ==== osd_op_reply(46 [pgnls-filter start_epoch 47 out=81b] v47'13 uv13 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5636f6d3b680 con 0x5636f6d80c00 2023-05-20T01:18:30.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.263+0000 7efc612697c0 20 datascan.forall_objects: OSD matched oid 100000001f6.00000000 2023-05-20T01:18:30.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.263+0000 7efc612697c0 10 datascan.operator(): handling object 100000001f6.0 2023-05-20T01:18:30.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.263+0000 7efc612697c0 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:47 7.2a 7:57fe7535:::100000001f6.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f6ea2e00 con 0x5636f6d80c00 2023-05-20T01:18:30.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.263+0000 7efc5fa18700 1 -- 172.21.15.16:0/3368616031 <== osd.0 v2:172.21.15.16:6824/3923634179 13 ==== osd_op_reply(47 100000001f6.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (97) out=97b,getxattr,getxattr] v0'0 uv13 ondisk = 0) v8 ==== 416+0+143 (crc 0 0 0) 0x5636f6d3b680 con 0x5636f6d80c00 2023-05-20T01:18:30.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.263+0000 7efc612697c0 1 --2- 172.21.15.16:0/3368616031 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x5636f6d4d800 0x5636f6e9f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:30.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.263+0000 7efc612697c0 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:48 8.a 8:57fe7535:::100000001f6.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f7803c00 con 0x5636f6d4d800 2023-05-20T01:18:30.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.263+0000 7efc5ea16700 1 --2- 172.21.15.16:0/3368616031 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x5636f6d4d800 0x5636f6e9f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:30.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.264+0000 7efc5ea16700 1 --2- 172.21.15.16:0/3368616031 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x5636f6d4d800 0x5636f6e9f600 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:30.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.264+0000 7efc5ea16700 1 -- 172.21.15.16:0/3368616031 <== osd.1 v2:172.21.15.16:6808/3860264220 1 ==== osd_op_reply(48 100000001f6.00000000 [getxattr (30) out=30b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x5636f6d3b440 con 0x5636f6d4d800 2023-05-20T01:18:30.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.264+0000 7efc612697c0 20 datascan.operator(): loaded layout from xattr: pi: 8 os: 4194304 sc: 1 su: 4194304 2023-05-20T01:18:30.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.264+0000 7efc612697c0 10 datascan.inject_with_backtrace: inode: 0x0x100000001f6 2023-05-20T01:18:30.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.264+0000 7efc612697c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/1 2023-05-20T01:18:30.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.264+0000 7efc612697c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=1 2023-05-20T01:18:30.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.264+0000 7efc612697c0 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:49 6.10 6:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f7803180 con 0x5636f6d80c00 2023-05-20T01:18:30.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.264+0000 7efc5fa18700 1 -- 172.21.15.16:0/3368616031 <== osd.0 v2:172.21.15.16:6824/3923634179 14 ==== osd_op_reply(49 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 164+0+0 (crc 0 0 0) 0x5636f6d3b680 con 0x5636f6d80c00 2023-05-20T01:18:30.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.264+0000 7efc612697c0 4 datascan.get_frag_of: Unexpected error on '10000000000.00000000': (2) No such file or directory 2023-05-20T01:18:30.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.264+0000 7efc612697c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/1, will insert to root 2023-05-20T01:18:30.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.264+0000 7efc612697c0 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:50 6.10 6:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f7802a80 con 0x5636f6d80c00 2023-05-20T01:18:30.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.264+0000 7efc5fa18700 1 -- 172.21.15.16:0/3368616031 <== osd.0 v2:172.21.15.16:6824/3923634179 15 ==== osd_op_reply(50 10000000000.00000000 [omap-get-header] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 164+0+0 (crc 0 0 0) 0x5636f6d3b680 con 0x5636f6d80c00 2023-05-20T01:18:30.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.264+0000 7efc612697c0 10 datascan.find_or_create_dirfrag: read_version = 0 2023-05-20T01:18:30.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.264+0000 7efc612697c0 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:51 6.10 6:0cf6ad0f:::10000000000.00000000:head [create,omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f7802700 con 0x5636f6d80c00 2023-05-20T01:18:30.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.266+0000 7efc5fa18700 1 -- 172.21.15.16:0/3368616031 <== osd.0 v2:172.21.15.16:6824/3923634179 16 ==== osd_op_reply(51 10000000000.00000000 [create,omap-set-header] v47'16 uv16 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x5636f6d3b680 con 0x5636f6d80c00 2023-05-20T01:18:30.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.266+0000 7efc612697c0 10 datascan.find_or_create_dirfrag: Created dirfrag: 0x0x10000000000 2023-05-20T01:18:30.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.266+0000 7efc612697c0 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:52 6.10 6:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f7802e00 con 0x5636f6d80c00 2023-05-20T01:18:30.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.266+0000 7efc5fa18700 1 -- 172.21.15.16:0/3368616031 <== osd.0 v2:172.21.15.16:6824/3923634179 17 ==== osd_op_reply(52 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv16 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x5636f6d3b680 con 0x5636f6d80c00 2023-05-20T01:18:30.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.266+0000 7efc612697c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=1 frag=*, r=0 2023-05-20T01:18:30.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.266+0000 7efc612697c0 20 datascan.read_dentry: 1_head not found in result 2023-05-20T01:18:30.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.266+0000 7efc612697c0 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001f6 at 0x0x10000000000/1 with size=6291456 bytes 2023-05-20T01:18:30.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.266+0000 7efc612697c0 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:53 6.10 6:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f7803880 con 0x5636f6d80c00 2023-05-20T01:18:30.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.267+0000 7efc5fa18700 1 -- 172.21.15.16:0/3368616031 <== osd.0 v2:172.21.15.16:6824/3923634179 18 ==== osd_op_reply(53 10000000000.00000000 [omap-set-vals] v47'17 uv17 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5636f6d3b680 con 0x5636f6d80c00 2023-05-20T01:18:30.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.267+0000 7efc612697c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/1 pointing to 0x0x100000001f6 2023-05-20T01:18:30.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.267+0000 7efc612697c0 10 datascan.inject_with_backtrace: backptr: 0x0x1/subdir 2023-05-20T01:18:30.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.267+0000 7efc612697c0 20 datascan.get_frag_of: dirino=0x1 target_dname=subdir 2023-05-20T01:18:30.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.267+0000 7efc612697c0 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:54 6.1f 6:ff5b34d6:::1.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f7803180 con 0x5636f6d4cc00 2023-05-20T01:18:30.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.268+0000 7efc5fa18700 1 -- 172.21.15.16:0/3368616031 <== osd.3 v2:172.21.15.16:6800/3669781890 10 ==== osd_op_reply(54 1.00000000 [getxattr] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x5636f6d3b680 con 0x5636f6d4cc00 2023-05-20T01:18:30.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.268+0000 7efc612697c0 4 datascan.get_frag_of: Unexpected error on '1.00000000': (2) No such file or directory 2023-05-20T01:18:30.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.268+0000 7efc612697c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x1/subdir, will insert to root 2023-05-20T01:18:30.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.268+0000 7efc612697c0 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:55 6.1f 6:ff5b34d6:::1.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f6ea2380 con 0x5636f6d4cc00 2023-05-20T01:18:30.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.268+0000 7efc5fa18700 1 -- 172.21.15.16:0/3368616031 <== osd.3 v2:172.21.15.16:6800/3669781890 11 ==== osd_op_reply(55 1.00000000 [omap-get-header] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x5636f6d3b680 con 0x5636f6d4cc00 2023-05-20T01:18:30.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.268+0000 7efc612697c0 10 datascan.find_or_create_dirfrag: read_version = 0 2023-05-20T01:18:30.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.268+0000 7efc612697c0 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:56 6.1f 6:ff5b34d6:::1.00000000:head [create,omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f6ea3180 con 0x5636f6d4cc00 2023-05-20T01:18:30.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.269+0000 7efc5fa18700 1 -- 172.21.15.16:0/3368616031 <== osd.3 v2:172.21.15.16:6800/3669781890 12 ==== osd_op_reply(56 1.00000000 [create,omap-set-header] v47'11 uv11 ondisk = 0) v8 ==== 196+0+0 (crc 0 0 0) 0x5636f6d3b680 con 0x5636f6d4cc00 2023-05-20T01:18:30.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.269+0000 7efc612697c0 10 datascan.find_or_create_dirfrag: Created dirfrag: 0x0x1 2023-05-20T01:18:30.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.269+0000 7efc612697c0 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:57 6.1f 6:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f6ea2e00 con 0x5636f6d4cc00 2023-05-20T01:18:30.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.269+0000 7efc5fa18700 1 -- 172.21.15.16:0/3368616031 <== osd.3 v2:172.21.15.16:6800/3669781890 13 ==== osd_op_reply(57 1.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv11 ondisk = 0) v8 ==== 154+0+4 (crc 0 0 0) 0x5636f6d3b680 con 0x5636f6d4cc00 2023-05-20T01:18:30.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.269+0000 7efc612697c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-05-20T01:18:30.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.269+0000 7efc612697c0 20 datascan.read_dentry: subdir_head not found in result 2023-05-20T01:18:30.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.269+0000 7efc612697c0 10 datascan.inject_with_backtrace: Linking ancestor directory of inode 0x0x10000000000 at 0x0x1:subdir 2023-05-20T01:18:30.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.269+0000 7efc612697c0 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:58 6.1f 6:ff5b34d6:::1.00000000:head [omap-set-vals in=507b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f7802a80 con 0x5636f6d4cc00 2023-05-20T01:18:30.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.270+0000 7efc5fa18700 1 -- 172.21.15.16:0/3368616031 <== osd.3 v2:172.21.15.16:6800/3669781890 14 ==== osd_op_reply(58 1.00000000 [omap-set-vals] v47'12 uv12 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x5636f6d3b680 con 0x5636f6d4cc00 2023-05-20T01:18:30.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.270+0000 7efc612697c0 20 datascan.inject_linkage: Injected dentry 0x0x1/subdir pointing to 0x0x10000000000 2023-05-20T01:18:30.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.270+0000 7efc612697c0 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:59 7.1a 7:58000000::::head [pgnls-filter start_epoch 47 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f7802700 con 0x5636f6d81000 2023-05-20T01:18:30.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.271+0000 7efc5f217700 1 -- 172.21.15.16:0/3368616031 <== osd.4 v2:172.21.15.132:6817/506317223 3 ==== osd_op_reply(59 [pgnls-filter start_epoch 47 out=49b] v45'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636f6d3b8c0 con 0x5636f6d81000 2023-05-20T01:18:30.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.271+0000 7efc5a20d700 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:60 7.3a 7:5c000000::::head [pgnls-filter start_epoch 47 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f6ea2a80 con 0x5636f6d4cc00 2023-05-20T01:18:30.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.271+0000 7efc5fa18700 1 -- 172.21.15.16:0/3368616031 <== osd.3 v2:172.21.15.16:6800/3669781890 15 ==== osd_op_reply(60 [pgnls-filter start_epoch 47 out=49b] v45'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636f6d3b680 con 0x5636f6d4cc00 2023-05-20T01:18:30.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.271+0000 7efc5aa0e700 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:61 7.6 7:60000000::::head [pgnls-filter start_epoch 47 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f6ea3c00 con 0x5636f6d80c00 2023-05-20T01:18:30.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.271+0000 7efc5fa18700 1 -- 172.21.15.16:0/3368616031 <== osd.0 v2:172.21.15.16:6824/3923634179 19 ==== osd_op_reply(61 [pgnls-filter start_epoch 47 out=49b] v45'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636f6d3b680 con 0x5636f6d80c00 2023-05-20T01:18:30.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.271+0000 7efc5a20d700 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:62 7.26 7:64000000::::head [pgnls-filter start_epoch 47 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f6ea2000 con 0x5636f6d4c800 2023-05-20T01:18:30.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.272+0000 7efc5ea16700 1 -- 172.21.15.16:0/3368616031 <== osd.2 v2:172.21.15.16:6816/293187030 8 ==== osd_op_reply(62 [pgnls-filter start_epoch 47 out=49b] v45'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636f6d3b440 con 0x5636f6d4c800 2023-05-20T01:18:30.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.272+0000 7efc5aa0e700 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:63 7.16 7:68000000::::head [pgnls-filter start_epoch 47 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f6ebe000 con 0x5636f6d81000 2023-05-20T01:18:30.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.272+0000 7efc5f217700 1 -- 172.21.15.16:0/3368616031 <== osd.4 v2:172.21.15.132:6817/506317223 4 ==== osd_op_reply(63 [pgnls-filter start_epoch 47 out=49b] v45'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636f6d3b8c0 con 0x5636f6d81000 2023-05-20T01:18:30.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.272+0000 7efc5a20d700 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:64 7.36 7:6c000000::::head [pgnls-filter start_epoch 47 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f6ea2700 con 0x5636f6d80c00 2023-05-20T01:18:30.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.272+0000 7efc5fa18700 1 -- 172.21.15.16:0/3368616031 <== osd.0 v2:172.21.15.16:6824/3923634179 20 ==== osd_op_reply(64 [pgnls-filter start_epoch 47 out=101b] v47'23 uv23 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x5636f6d3b680 con 0x5636f6d80c00 2023-05-20T01:18:30.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.272+0000 7efc612697c0 20 datascan.forall_objects: OSD matched oid 100000001f9.00000000 2023-05-20T01:18:30.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.272+0000 7efc612697c0 10 datascan.operator(): handling object 100000001f9.0 2023-05-20T01:18:30.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.272+0000 7efc612697c0 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:65 7.36 7:6ee5e2c4:::100000001f9.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f7802e00 con 0x5636f6d80c00 2023-05-20T01:18:30.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.272+0000 7efc5fa18700 1 -- 172.21.15.16:0/3368616031 <== osd.0 v2:172.21.15.16:6824/3923634179 21 ==== osd_op_reply(65 100000001f9.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (97) out=97b,getxattr,getxattr] v0'0 uv21 ondisk = 0) v8 ==== 416+0+143 (crc 0 0 0) 0x5636f6d3b680 con 0x5636f6d80c00 2023-05-20T01:18:30.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.272+0000 7efc612697c0 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:66 8.16 8:6ee5e2c4:::100000001f9.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f7803180 con 0x5636f6d4cc00 2023-05-20T01:18:30.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.273+0000 7efc5fa18700 1 -- 172.21.15.16:0/3368616031 <== osd.3 v2:172.21.15.16:6800/3669781890 16 ==== osd_op_reply(66 100000001f9.00000000 [getxattr (30) out=30b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x5636f6d3b680 con 0x5636f6d4cc00 2023-05-20T01:18:30.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.273+0000 7efc612697c0 20 datascan.operator(): loaded layout from xattr: pi: 8 os: 4194304 sc: 1 su: 4194304 2023-05-20T01:18:30.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.273+0000 7efc612697c0 10 datascan.inject_with_backtrace: inode: 0x0x100000001f9 2023-05-20T01:18:30.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.273+0000 7efc612697c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/4 2023-05-20T01:18:30.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.273+0000 7efc612697c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=4 2023-05-20T01:18:30.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.273+0000 7efc612697c0 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:67 6.10 6:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f7802a80 con 0x5636f6d80c00 2023-05-20T01:18:30.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.273+0000 7efc5fa18700 1 -- 172.21.15.16:0/3368616031 <== osd.0 v2:172.21.15.16:6824/3923634179 22 ==== osd_op_reply(67 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x5636f6d3b680 con 0x5636f6d80c00 2023-05-20T01:18:30.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.273+0000 7efc612697c0 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-05-20T01:18:30.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.273+0000 7efc612697c0 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-05-20T01:18:30.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.273+0000 7efc612697c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/4, will insert to root 2023-05-20T01:18:30.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.273+0000 7efc612697c0 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:68 6.10 6:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f6ea2380 con 0x5636f6d80c00 2023-05-20T01:18:30.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.273+0000 7efc5fa18700 1 -- 172.21.15.16:0/3368616031 <== osd.0 v2:172.21.15.16:6824/3923634179 23 ==== osd_op_reply(68 10000000000.00000000 [omap-get-header out=274b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5636f6d3b680 con 0x5636f6d80c00 2023-05-20T01:18:30.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.273+0000 7efc612697c0 10 datascan.find_or_create_dirfrag: read_version = 17 2023-05-20T01:18:30.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.273+0000 7efc612697c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-05-20T01:18:30.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.273+0000 7efc612697c0 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:69 6.10 6:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f6ea3180 con 0x5636f6d80c00 2023-05-20T01:18:30.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.274+0000 7efc5fa18700 1 -- 172.21.15.16:0/3368616031 <== osd.0 v2:172.21.15.16:6824/3923634179 24 ==== osd_op_reply(69 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x5636f6d3b680 con 0x5636f6d80c00 2023-05-20T01:18:30.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.274+0000 7efc612697c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=4 frag=*, r=0 2023-05-20T01:18:30.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.274+0000 7efc612697c0 20 datascan.read_dentry: 4_head not found in result 2023-05-20T01:18:30.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.274+0000 7efc612697c0 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001f9 at 0x0x10000000000/4 with size=6291456 bytes 2023-05-20T01:18:30.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.274+0000 7efc612697c0 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:70 6.10 6:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f6ea2e00 con 0x5636f6d80c00 2023-05-20T01:18:30.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.275+0000 7efc5fa18700 1 -- 172.21.15.16:0/3368616031 <== osd.0 v2:172.21.15.16:6824/3923634179 25 ==== osd_op_reply(70 10000000000.00000000 [omap-set-vals] v47'18 uv18 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5636f6d3b680 con 0x5636f6d80c00 2023-05-20T01:18:30.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.275+0000 7efc612697c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/4 pointing to 0x0x100000001f9 2023-05-20T01:18:30.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.275+0000 7efc612697c0 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:71 7.36 7:6fe38dc2::::head [pgnls-filter start_epoch 47 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f6ea2a80 con 0x5636f6d80c00 2023-05-20T01:18:30.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.275+0000 7efc5fa18700 1 -- 172.21.15.16:0/3368616031 <== osd.0 v2:172.21.15.16:6824/3923634179 26 ==== osd_op_reply(71 [pgnls-filter start_epoch 47 out=81b] v47'23 uv23 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5636f6d3b680 con 0x5636f6d80c00 2023-05-20T01:18:30.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.275+0000 7efc612697c0 20 datascan.forall_objects: OSD matched oid 100000001f7.00000000 2023-05-20T01:18:30.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.275+0000 7efc612697c0 10 datascan.operator(): handling object 100000001f7.0 2023-05-20T01:18:30.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.275+0000 7efc612697c0 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:72 7.36 7:6fe38dc2:::100000001f7.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f6ea3c00 con 0x5636f6d80c00 2023-05-20T01:18:30.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.275+0000 7efc5fa18700 1 -- 172.21.15.16:0/3368616031 <== osd.0 v2:172.21.15.16:6824/3923634179 27 ==== osd_op_reply(72 100000001f7.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (97) out=97b,getxattr,getxattr] v0'0 uv23 ondisk = 0) v8 ==== 416+0+143 (crc 0 0 0) 0x5636f6d3b680 con 0x5636f6d80c00 2023-05-20T01:18:30.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.275+0000 7efc612697c0 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:73 8.16 8:6fe38dc2:::100000001f7.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f6ea2380 con 0x5636f6d4cc00 2023-05-20T01:18:30.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.275+0000 7efc5fa18700 1 -- 172.21.15.16:0/3368616031 <== osd.3 v2:172.21.15.16:6800/3669781890 17 ==== osd_op_reply(73 100000001f7.00000000 [getxattr (30) out=30b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x5636f6d3b680 con 0x5636f6d4cc00 2023-05-20T01:18:30.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.275+0000 7efc612697c0 20 datascan.operator(): loaded layout from xattr: pi: 8 os: 4194304 sc: 1 su: 4194304 2023-05-20T01:18:30.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.275+0000 7efc612697c0 10 datascan.inject_with_backtrace: inode: 0x0x100000001f7 2023-05-20T01:18:30.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.275+0000 7efc612697c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/2 2023-05-20T01:18:30.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.275+0000 7efc612697c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=2 2023-05-20T01:18:30.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.275+0000 7efc612697c0 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:74 6.10 6:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f6ea3180 con 0x5636f6d80c00 2023-05-20T01:18:30.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.276+0000 7efc5fa18700 1 -- 172.21.15.16:0/3368616031 <== osd.0 v2:172.21.15.16:6824/3923634179 28 ==== osd_op_reply(74 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x5636f6d3b680 con 0x5636f6d80c00 2023-05-20T01:18:30.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.276+0000 7efc612697c0 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-05-20T01:18:30.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.276+0000 7efc612697c0 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-05-20T01:18:30.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.276+0000 7efc612697c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/2, will insert to root 2023-05-20T01:18:30.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.276+0000 7efc612697c0 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:75 6.10 6:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f7803880 con 0x5636f6d80c00 2023-05-20T01:18:30.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.276+0000 7efc5fa18700 1 -- 172.21.15.16:0/3368616031 <== osd.0 v2:172.21.15.16:6824/3923634179 29 ==== osd_op_reply(75 10000000000.00000000 [omap-get-header out=274b] v0'0 uv18 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5636f6d3b680 con 0x5636f6d80c00 2023-05-20T01:18:30.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.276+0000 7efc612697c0 10 datascan.find_or_create_dirfrag: read_version = 18 2023-05-20T01:18:30.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.276+0000 7efc612697c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-05-20T01:18:30.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.276+0000 7efc612697c0 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:76 6.10 6:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f7802e00 con 0x5636f6d80c00 2023-05-20T01:18:30.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.276+0000 7efc5fa18700 1 -- 172.21.15.16:0/3368616031 <== osd.0 v2:172.21.15.16:6824/3923634179 30 ==== osd_op_reply(76 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv18 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x5636f6d3b680 con 0x5636f6d80c00 2023-05-20T01:18:30.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.276+0000 7efc612697c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=2 frag=*, r=0 2023-05-20T01:18:30.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.276+0000 7efc612697c0 20 datascan.read_dentry: 2_head not found in result 2023-05-20T01:18:30.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.276+0000 7efc612697c0 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001f7 at 0x0x10000000000/2 with size=6291456 bytes 2023-05-20T01:18:30.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.276+0000 7efc612697c0 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:77 6.10 6:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f7803180 con 0x5636f6d80c00 2023-05-20T01:18:30.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.277+0000 7efc5fa18700 1 -- 172.21.15.16:0/3368616031 <== osd.0 v2:172.21.15.16:6824/3923634179 31 ==== osd_op_reply(77 10000000000.00000000 [omap-set-vals] v47'19 uv19 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5636f6d3b680 con 0x5636f6d80c00 2023-05-20T01:18:30.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.277+0000 7efc612697c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/2 pointing to 0x0x100000001f7 2023-05-20T01:18:30.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.277+0000 7efc612697c0 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:78 7.e 7:70000000::::head [pgnls-filter start_epoch 47 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f7802a80 con 0x5636f6d4c800 2023-05-20T01:18:30.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.277+0000 7efc5ea16700 1 -- 172.21.15.16:0/3368616031 <== osd.2 v2:172.21.15.16:6816/293187030 9 ==== osd_op_reply(78 [pgnls-filter start_epoch 47 out=49b] v45'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636f6d3b440 con 0x5636f6d4c800 2023-05-20T01:18:30.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.277+0000 7efc5aa0e700 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:79 7.2e 7:74000000::::head [pgnls-filter start_epoch 47 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f6ebe380 con 0x5636f6d81000 2023-05-20T01:18:30.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.278+0000 7efc5f217700 1 -- 172.21.15.16:0/3368616031 <== osd.4 v2:172.21.15.132:6817/506317223 5 ==== osd_op_reply(79 [pgnls-filter start_epoch 47 out=49b] v45'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636f6d3b8c0 con 0x5636f6d81000 2023-05-20T01:18:30.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.278+0000 7efc5a20d700 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:80 7.1e 7:78000000::::head [pgnls-filter start_epoch 47 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f6ea3880 con 0x5636f6d4d800 2023-05-20T01:18:30.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.278+0000 7efc5ea16700 1 -- 172.21.15.16:0/3368616031 <== osd.1 v2:172.21.15.16:6808/3860264220 2 ==== osd_op_reply(80 [pgnls-filter start_epoch 47 out=49b] v45'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636f6d3b440 con 0x5636f6d4d800 2023-05-20T01:18:30.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.278+0000 7efc5aa0e700 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:81 7.3e 7:7c000000::::head [pgnls-filter start_epoch 47 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f6ebe700 con 0x5636f6d4d000 2023-05-20T01:18:30.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.278+0000 7efc5f217700 1 -- 172.21.15.16:0/3368616031 <== osd.5 v2:172.21.15.132:6816/1844689995 9 ==== osd_op_reply(81 [pgnls-filter start_epoch 47 out=49b] v45'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636f6d3b8c0 con 0x5636f6d4d000 2023-05-20T01:18:30.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.278+0000 7efc5a20d700 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:82 7.1 7:80000000::::head [pgnls-filter start_epoch 47 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f6ebe000 con 0x5636f6d81000 2023-05-20T01:18:30.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.279+0000 7efc5f217700 1 -- 172.21.15.16:0/3368616031 <== osd.4 v2:172.21.15.132:6817/506317223 6 ==== osd_op_reply(82 [pgnls-filter start_epoch 47 out=49b] v45'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636f6d3b8c0 con 0x5636f6d81000 2023-05-20T01:18:30.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.279+0000 7efc5aa0e700 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:83 7.21 7:84000000::::head [pgnls-filter start_epoch 47 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f6ebea80 con 0x5636f6d4cc00 2023-05-20T01:18:30.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.279+0000 7efc5fa18700 1 -- 172.21.15.16:0/3368616031 <== osd.3 v2:172.21.15.16:6800/3669781890 18 ==== osd_op_reply(83 [pgnls-filter start_epoch 47 out=49b] v45'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636f6d3b680 con 0x5636f6d4cc00 2023-05-20T01:18:30.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.279+0000 7efc5a20d700 1 --2- 172.21.15.16:0/3368616031 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x5636f6d81400 0x5636f6e9f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:30.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.279+0000 7efc5a20d700 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:84 7.11 7:88000000::::head [pgnls-filter start_epoch 47 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f6ebe380 con 0x5636f6d81400 2023-05-20T01:18:30.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.279+0000 7efc5fa18700 1 --2- 172.21.15.16:0/3368616031 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x5636f6d81400 0x5636f6e9f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:30.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.280+0000 7efc5fa18700 1 --2- 172.21.15.16:0/3368616031 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x5636f6d81400 0x5636f6e9f080 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:30.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.280+0000 7efc5fa18700 1 -- 172.21.15.16:0/3368616031 <== osd.7 v2:172.21.15.132:6800/466001149 1 ==== osd_op_reply(84 [pgnls-filter start_epoch 47 out=49b] v45'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636f6d3b680 con 0x5636f6d81400 2023-05-20T01:18:30.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.280+0000 7efc5aa0e700 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:85 7.31 7:8c000000::::head [pgnls-filter start_epoch 47 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f6ebee00 con 0x5636f6d4cc00 2023-05-20T01:18:30.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.280+0000 7efc5fa18700 1 -- 172.21.15.16:0/3368616031 <== osd.3 v2:172.21.15.16:6800/3669781890 19 ==== osd_op_reply(85 [pgnls-filter start_epoch 47 out=49b] v45'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636f6d3b680 con 0x5636f6d4cc00 2023-05-20T01:18:30.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.280+0000 7efc5a20d700 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:86 7.9 7:90000000::::head [pgnls-filter start_epoch 47 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f6ebe700 con 0x5636f6d81000 2023-05-20T01:18:30.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.280+0000 7efc5f217700 1 -- 172.21.15.16:0/3368616031 <== osd.4 v2:172.21.15.132:6817/506317223 7 ==== osd_op_reply(86 [pgnls-filter start_epoch 47 out=49b] v45'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636f6d3b8c0 con 0x5636f6d81000 2023-05-20T01:18:30.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.280+0000 7efc5aa0e700 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:87 7.29 7:94000000::::head [pgnls-filter start_epoch 47 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f6ebf180 con 0x5636f6d80800 2023-05-20T01:18:30.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.281+0000 7efc5ea16700 1 -- 172.21.15.16:0/3368616031 <== osd.6 v2:172.21.15.132:6801/727581123 9 ==== osd_op_reply(87 [pgnls-filter start_epoch 47 out=49b] v45'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636f6d3b440 con 0x5636f6d80800 2023-05-20T01:18:30.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.281+0000 7efc5a20d700 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:88 7.19 7:98000000::::head [pgnls-filter start_epoch 47 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f6ebfc00 con 0x5636f6d80800 2023-05-20T01:18:30.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.281+0000 7efc5ea16700 1 -- 172.21.15.16:0/3368616031 <== osd.6 v2:172.21.15.132:6801/727581123 10 ==== osd_op_reply(88 [pgnls-filter start_epoch 47 out=81b] v47'14 uv14 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5636f6d3b440 con 0x5636f6d80800 2023-05-20T01:18:30.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.281+0000 7efc612697c0 20 datascan.forall_objects: OSD matched oid 100000001f8.00000000 2023-05-20T01:18:30.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.281+0000 7efc612697c0 10 datascan.operator(): handling object 100000001f8.0 2023-05-20T01:18:30.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.281+0000 7efc612697c0 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:89 7.19 7:997b7129:::100000001f8.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f6ebf180 con 0x5636f6d80800 2023-05-20T01:18:30.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.281+0000 7efc5ea16700 1 -- 172.21.15.16:0/3368616031 <== osd.6 v2:172.21.15.132:6801/727581123 11 ==== osd_op_reply(89 100000001f8.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (97) out=97b,getxattr,getxattr] v0'0 uv14 ondisk = 0) v8 ==== 416+0+143 (crc 0 0 0) 0x5636f6d3b440 con 0x5636f6d80800 2023-05-20T01:18:30.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.281+0000 7efc612697c0 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:90 8.19 8:997b7129:::100000001f8.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f7803c00 con 0x5636f6d4d000 2023-05-20T01:18:30.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.282+0000 7efc5f217700 1 -- 172.21.15.16:0/3368616031 <== osd.5 v2:172.21.15.132:6816/1844689995 10 ==== osd_op_reply(90 100000001f8.00000000 [getxattr (30) out=30b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x5636f6d3b8c0 con 0x5636f6d4d000 2023-05-20T01:18:30.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.282+0000 7efc612697c0 20 datascan.operator(): loaded layout from xattr: pi: 8 os: 4194304 sc: 1 su: 4194304 2023-05-20T01:18:30.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.282+0000 7efc612697c0 10 datascan.inject_with_backtrace: inode: 0x0x100000001f8 2023-05-20T01:18:30.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.282+0000 7efc612697c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/3 2023-05-20T01:18:30.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.282+0000 7efc612697c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=3 2023-05-20T01:18:30.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.282+0000 7efc612697c0 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:91 6.10 6:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f7802a80 con 0x5636f6d80c00 2023-05-20T01:18:30.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.282+0000 7efc5fa18700 1 -- 172.21.15.16:0/3368616031 <== osd.0 v2:172.21.15.16:6824/3923634179 32 ==== osd_op_reply(91 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x5636f6d3b680 con 0x5636f6d80c00 2023-05-20T01:18:30.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.282+0000 7efc612697c0 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-05-20T01:18:30.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.282+0000 7efc612697c0 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-05-20T01:18:30.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.282+0000 7efc612697c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/3, will insert to root 2023-05-20T01:18:30.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.282+0000 7efc612697c0 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:92 6.10 6:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f7803880 con 0x5636f6d80c00 2023-05-20T01:18:30.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.282+0000 7efc5fa18700 1 -- 172.21.15.16:0/3368616031 <== osd.0 v2:172.21.15.16:6824/3923634179 33 ==== osd_op_reply(92 10000000000.00000000 [omap-get-header out=274b] v0'0 uv19 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5636f6d3b680 con 0x5636f6d80c00 2023-05-20T01:18:30.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.282+0000 7efc612697c0 10 datascan.find_or_create_dirfrag: read_version = 19 2023-05-20T01:18:30.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.282+0000 7efc612697c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-05-20T01:18:30.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.282+0000 7efc612697c0 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:93 6.10 6:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f7802e00 con 0x5636f6d80c00 2023-05-20T01:18:30.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.283+0000 7efc5fa18700 1 -- 172.21.15.16:0/3368616031 <== osd.0 v2:172.21.15.16:6824/3923634179 34 ==== osd_op_reply(93 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv19 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x5636f6d3b680 con 0x5636f6d80c00 2023-05-20T01:18:30.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.283+0000 7efc612697c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=3 frag=*, r=0 2023-05-20T01:18:30.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.283+0000 7efc612697c0 20 datascan.read_dentry: 3_head not found in result 2023-05-20T01:18:30.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.283+0000 7efc612697c0 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001f8 at 0x0x10000000000/3 with size=6291456 bytes 2023-05-20T01:18:30.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.283+0000 7efc612697c0 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:94 6.10 6:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f7803180 con 0x5636f6d80c00 2023-05-20T01:18:30.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.287+0000 7efc5fa18700 1 -- 172.21.15.16:0/3368616031 <== osd.0 v2:172.21.15.16:6824/3923634179 35 ==== osd_op_reply(94 10000000000.00000000 [omap-set-vals] v47'20 uv20 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5636f6d3b680 con 0x5636f6d80c00 2023-05-20T01:18:30.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.287+0000 7efc612697c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/3 pointing to 0x0x100000001f8 2023-05-20T01:18:30.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.287+0000 7efc612697c0 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:95 7.39 7:9c000000::::head [pgnls-filter start_epoch 47 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f6ea2000 con 0x5636f6d81400 2023-05-20T01:18:30.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.287+0000 7efc5fa18700 1 -- 172.21.15.16:0/3368616031 <== osd.7 v2:172.21.15.132:6800/466001149 2 ==== osd_op_reply(95 [pgnls-filter start_epoch 47 out=49b] v45'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636f6d3b680 con 0x5636f6d81400 2023-05-20T01:18:30.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.287+0000 7efc5a20d700 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:96 7.5 7:a0000000::::head [pgnls-filter start_epoch 47 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f7803500 con 0x5636f6d81400 2023-05-20T01:18:30.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.288+0000 7efc5fa18700 1 -- 172.21.15.16:0/3368616031 <== osd.7 v2:172.21.15.132:6800/466001149 3 ==== osd_op_reply(96 [pgnls-filter start_epoch 47 out=49b] v45'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636f6d3b680 con 0x5636f6d81400 2023-05-20T01:18:30.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.288+0000 7efc5aa0e700 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:97 7.25 7:a4000000::::head [pgnls-filter start_epoch 47 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f6ebf500 con 0x5636f6d81000 2023-05-20T01:18:30.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.288+0000 7efc5f217700 1 -- 172.21.15.16:0/3368616031 <== osd.4 v2:172.21.15.132:6817/506317223 8 ==== osd_op_reply(97 [pgnls-filter start_epoch 47 out=49b] v45'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636f6d3b8c0 con 0x5636f6d81000 2023-05-20T01:18:30.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.288+0000 7efc5a20d700 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:98 7.15 7:a8000000::::head [pgnls-filter start_epoch 47 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f7802700 con 0x5636f6d81400 2023-05-20T01:18:30.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.288+0000 7efc5fa18700 1 -- 172.21.15.16:0/3368616031 <== osd.7 v2:172.21.15.132:6800/466001149 4 ==== osd_op_reply(98 [pgnls-filter start_epoch 47 out=49b] v45'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636f6d3b680 con 0x5636f6d81400 2023-05-20T01:18:30.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.288+0000 7efc5aa0e700 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:99 7.35 7:ac000000::::head [pgnls-filter start_epoch 47 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f6ebf880 con 0x5636f6d80c00 2023-05-20T01:18:30.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.289+0000 7efc5fa18700 1 -- 172.21.15.16:0/3368616031 <== osd.0 v2:172.21.15.16:6824/3923634179 36 ==== osd_op_reply(99 [pgnls-filter start_epoch 47 out=49b] v45'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636f6d3b680 con 0x5636f6d80c00 2023-05-20T01:18:30.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.289+0000 7efc5a20d700 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:100 7.d 7:b0000000::::head [pgnls-filter start_epoch 47 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f6ed6000 con 0x5636f6d80800 2023-05-20T01:18:30.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.289+0000 7efc5ea16700 1 -- 172.21.15.16:0/3368616031 <== osd.6 v2:172.21.15.132:6801/727581123 12 ==== osd_op_reply(100 [pgnls-filter start_epoch 47 out=49b] v45'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636f6d3b440 con 0x5636f6d80800 2023-05-20T01:18:30.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.289+0000 7efc5aa0e700 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:101 7.2d 7:b4000000::::head [pgnls-filter start_epoch 47 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f6ed8000 con 0x5636f6d80800 2023-05-20T01:18:30.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.289+0000 7efc5ea16700 1 -- 172.21.15.16:0/3368616031 <== osd.6 v2:172.21.15.132:6801/727581123 13 ==== osd_op_reply(101 [pgnls-filter start_epoch 47 out=49b] v45'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636f6d3b440 con 0x5636f6d80800 2023-05-20T01:18:30.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.289+0000 7efc5a20d700 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:102 7.1d 7:b8000000::::head [pgnls-filter start_epoch 47 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f6ed6380 con 0x5636f6d4d800 2023-05-20T01:18:30.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.290+0000 7efc5ea16700 1 -- 172.21.15.16:0/3368616031 <== osd.1 v2:172.21.15.16:6808/3860264220 3 ==== osd_op_reply(102 [pgnls-filter start_epoch 47 out=49b] v45'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636f6d3b440 con 0x5636f6d4d800 2023-05-20T01:18:30.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.290+0000 7efc5aa0e700 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:103 7.3d 7:bc000000::::head [pgnls-filter start_epoch 47 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f6ed8380 con 0x5636f6d81000 2023-05-20T01:18:30.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.290+0000 7efc5f217700 1 -- 172.21.15.16:0/3368616031 <== osd.4 v2:172.21.15.132:6817/506317223 9 ==== osd_op_reply(103 [pgnls-filter start_epoch 47 out=49b] v45'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636f6d3b8c0 con 0x5636f6d81000 2023-05-20T01:18:30.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.290+0000 7efc5a20d700 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:104 7.3 7:c0000000::::head [pgnls-filter start_epoch 47 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f6ed6700 con 0x5636f6d4cc00 2023-05-20T01:18:30.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.290+0000 7efc5fa18700 1 -- 172.21.15.16:0/3368616031 <== osd.3 v2:172.21.15.16:6800/3669781890 20 ==== osd_op_reply(104 [pgnls-filter start_epoch 47 out=49b] v45'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636f6d3b680 con 0x5636f6d4cc00 2023-05-20T01:18:30.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.290+0000 7efc5aa0e700 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:105 7.23 7:c4000000::::head [pgnls-filter start_epoch 47 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f6ed8700 con 0x5636f6d80800 2023-05-20T01:18:30.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.291+0000 7efc5ea16700 1 -- 172.21.15.16:0/3368616031 <== osd.6 v2:172.21.15.132:6801/727581123 14 ==== osd_op_reply(105 [pgnls-filter start_epoch 47 out=49b] v45'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636f6d3b440 con 0x5636f6d80800 2023-05-20T01:18:30.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.291+0000 7efc5a20d700 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:106 7.13 7:c8000000::::head [pgnls-filter start_epoch 47 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f6ed6a80 con 0x5636f6d80c00 2023-05-20T01:18:30.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.291+0000 7efc5fa18700 1 -- 172.21.15.16:0/3368616031 <== osd.0 v2:172.21.15.16:6824/3923634179 37 ==== osd_op_reply(106 [pgnls-filter start_epoch 47 out=49b] v45'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636f6d3b680 con 0x5636f6d80c00 2023-05-20T01:18:30.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.291+0000 7efc5aa0e700 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:107 7.33 7:cc000000::::head [pgnls-filter start_epoch 47 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f6ed8a80 con 0x5636f6d4c800 2023-05-20T01:18:30.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.291+0000 7efc5ea16700 1 -- 172.21.15.16:0/3368616031 <== osd.2 v2:172.21.15.16:6816/293187030 10 ==== osd_op_reply(107 [pgnls-filter start_epoch 47 out=49b] v45'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636f6d3b440 con 0x5636f6d4c800 2023-05-20T01:18:30.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.291+0000 7efc5a20d700 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:108 7.b 7:d0000000::::head [pgnls-filter start_epoch 47 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f6ed6e00 con 0x5636f6d4d800 2023-05-20T01:18:30.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.291+0000 7efc5ea16700 1 -- 172.21.15.16:0/3368616031 <== osd.1 v2:172.21.15.16:6808/3860264220 4 ==== osd_op_reply(108 [pgnls-filter start_epoch 47 out=49b] v45'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636f6d3b440 con 0x5636f6d4d800 2023-05-20T01:18:30.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.291+0000 7efc5aa0e700 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:109 7.2b 7:d4000000::::head [pgnls-filter start_epoch 47 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f6ed8e00 con 0x5636f6d4cc00 2023-05-20T01:18:30.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.292+0000 7efc5fa18700 1 -- 172.21.15.16:0/3368616031 <== osd.3 v2:172.21.15.16:6800/3669781890 21 ==== osd_op_reply(109 [pgnls-filter start_epoch 47 out=49b] v45'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636f6d3b680 con 0x5636f6d4cc00 2023-05-20T01:18:30.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.292+0000 7efc5a20d700 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:110 7.1b 7:d8000000::::head [pgnls-filter start_epoch 47 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f6ed7180 con 0x5636f6d4d000 2023-05-20T01:18:30.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.292+0000 7efc5f217700 1 -- 172.21.15.16:0/3368616031 <== osd.5 v2:172.21.15.132:6816/1844689995 11 ==== osd_op_reply(110 [pgnls-filter start_epoch 47 out=49b] v45'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636f6d3b8c0 con 0x5636f6d4d000 2023-05-20T01:18:30.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.292+0000 7efc5aa0e700 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:111 7.3b 7:dc000000::::head [pgnls-filter start_epoch 47 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f6ed9180 con 0x5636f6d4d000 2023-05-20T01:18:30.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.292+0000 7efc5f217700 1 -- 172.21.15.16:0/3368616031 <== osd.5 v2:172.21.15.132:6816/1844689995 12 ==== osd_op_reply(111 [pgnls-filter start_epoch 47 out=49b] v45'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636f6d3b8c0 con 0x5636f6d4d000 2023-05-20T01:18:30.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.292+0000 7efc5a20d700 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:112 7.7 7:e0000000::::head [pgnls-filter start_epoch 47 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f6ed7500 con 0x5636f6d80800 2023-05-20T01:18:30.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.293+0000 7efc5ea16700 1 -- 172.21.15.16:0/3368616031 <== osd.6 v2:172.21.15.132:6801/727581123 15 ==== osd_op_reply(112 [pgnls-filter start_epoch 47 out=81b] v47'9 uv9 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5636f6d3b440 con 0x5636f6d80800 2023-05-20T01:18:30.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.293+0000 7efc612697c0 20 datascan.forall_objects: OSD matched oid 10000000001.00000000 2023-05-20T01:18:30.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.293+0000 7efc612697c0 10 datascan.operator(): handling object 10000000001.0 2023-05-20T01:18:30.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.293+0000 7efc612697c0 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:113 7.7 7:e2e07930:::10000000001.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f6ea2380 con 0x5636f6d80800 2023-05-20T01:18:30.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.293+0000 7efc5ea16700 1 -- 172.21.15.16:0/3368616031 <== osd.6 v2:172.21.15.132:6801/727581123 16 ==== osd_op_reply(113 10000000001.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (97) out=97b,getxattr,getxattr] v0'0 uv9 ondisk = 0) v8 ==== 416+0+143 (crc 0 0 0) 0x5636f6d3b440 con 0x5636f6d80800 2023-05-20T01:18:30.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.293+0000 7efc612697c0 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:114 8.7 8:e2e07930:::10000000001.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f6ea2a80 con 0x5636f6d4d800 2023-05-20T01:18:30.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.293+0000 7efc5ea16700 1 -- 172.21.15.16:0/3368616031 <== osd.1 v2:172.21.15.16:6808/3860264220 5 ==== osd_op_reply(114 10000000001.00000000 [getxattr (30) out=30b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x5636f6d3b440 con 0x5636f6d4d800 2023-05-20T01:18:30.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.293+0000 7efc612697c0 20 datascan.operator(): loaded layout from xattr: pi: 8 os: 4194304 sc: 1 su: 4194304 2023-05-20T01:18:30.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.293+0000 7efc612697c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000001 2023-05-20T01:18:30.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.293+0000 7efc612697c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/0 2023-05-20T01:18:30.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.293+0000 7efc612697c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=0 2023-05-20T01:18:30.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.293+0000 7efc612697c0 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:115 6.10 6:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f6ea2e00 con 0x5636f6d80c00 2023-05-20T01:18:30.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.294+0000 7efc5fa18700 1 -- 172.21.15.16:0/3368616031 <== osd.0 v2:172.21.15.16:6824/3923634179 38 ==== osd_op_reply(115 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x5636f6d3b680 con 0x5636f6d80c00 2023-05-20T01:18:30.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.294+0000 7efc612697c0 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-05-20T01:18:30.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.294+0000 7efc612697c0 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-05-20T01:18:30.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.294+0000 7efc612697c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/0, will insert to root 2023-05-20T01:18:30.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.294+0000 7efc612697c0 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:116 6.10 6:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f6ea3180 con 0x5636f6d80c00 2023-05-20T01:18:30.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.294+0000 7efc5fa18700 1 -- 172.21.15.16:0/3368616031 <== osd.0 v2:172.21.15.16:6824/3923634179 39 ==== osd_op_reply(116 10000000000.00000000 [omap-get-header out=274b] v0'0 uv20 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5636f6d3b680 con 0x5636f6d80c00 2023-05-20T01:18:30.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.294+0000 7efc612697c0 10 datascan.find_or_create_dirfrag: read_version = 20 2023-05-20T01:18:30.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.294+0000 7efc612697c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-05-20T01:18:30.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.294+0000 7efc612697c0 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:117 6.10 6:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f6ea3500 con 0x5636f6d80c00 2023-05-20T01:18:30.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.294+0000 7efc5fa18700 1 -- 172.21.15.16:0/3368616031 <== osd.0 v2:172.21.15.16:6824/3923634179 40 ==== osd_op_reply(117 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv20 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x5636f6d3b680 con 0x5636f6d80c00 2023-05-20T01:18:30.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.294+0000 7efc612697c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=0 frag=*, r=0 2023-05-20T01:18:30.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.294+0000 7efc612697c0 20 datascan.read_dentry: 0_head not found in result 2023-05-20T01:18:30.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.294+0000 7efc612697c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000001 at 0x0x10000000000/0 with size=6291456 bytes 2023-05-20T01:18:30.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.294+0000 7efc612697c0 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:118 6.10 6:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f6ea3880 con 0x5636f6d80c00 2023-05-20T01:18:30.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.296+0000 7efc5fa18700 1 -- 172.21.15.16:0/3368616031 <== osd.0 v2:172.21.15.16:6824/3923634179 41 ==== osd_op_reply(118 10000000000.00000000 [omap-set-vals] v47'21 uv21 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5636f6d3b680 con 0x5636f6d80c00 2023-05-20T01:18:30.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.296+0000 7efc612697c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/0 pointing to 0x0x10000000001 2023-05-20T01:18:30.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.296+0000 7efc612697c0 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:119 7.27 7:e4000000::::head [pgnls-filter start_epoch 47 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f6ea3c00 con 0x5636f6d80800 2023-05-20T01:18:30.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.296+0000 7efc5ea16700 1 -- 172.21.15.16:0/3368616031 <== osd.6 v2:172.21.15.132:6801/727581123 17 ==== osd_op_reply(119 [pgnls-filter start_epoch 47 out=49b] v45'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636f6d3b440 con 0x5636f6d80800 2023-05-20T01:18:30.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.296+0000 7efc5a20d700 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:120 7.17 7:e8000000::::head [pgnls-filter start_epoch 47 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f6ed7880 con 0x5636f6d81400 2023-05-20T01:18:30.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.296+0000 7efc5fa18700 1 -- 172.21.15.16:0/3368616031 <== osd.7 v2:172.21.15.132:6800/466001149 5 ==== osd_op_reply(120 [pgnls-filter start_epoch 47 out=49b] v45'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636f6d3b680 con 0x5636f6d81400 2023-05-20T01:18:30.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.296+0000 7efc5aa0e700 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:121 7.37 7:ec000000::::head [pgnls-filter start_epoch 47 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f6ed9500 con 0x5636f6d80c00 2023-05-20T01:18:30.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.297+0000 7efc5fa18700 1 -- 172.21.15.16:0/3368616031 <== osd.0 v2:172.21.15.16:6824/3923634179 42 ==== osd_op_reply(121 [pgnls-filter start_epoch 47 out=49b] v45'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636f6d3b680 con 0x5636f6d80c00 2023-05-20T01:18:30.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.297+0000 7efc5a20d700 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:122 7.f 7:f0000000::::head [pgnls-filter start_epoch 47 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f6ed7c00 con 0x5636f6d80c00 2023-05-20T01:18:30.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.297+0000 7efc5fa18700 1 -- 172.21.15.16:0/3368616031 <== osd.0 v2:172.21.15.16:6824/3923634179 43 ==== osd_op_reply(122 [pgnls-filter start_epoch 47 out=49b] v45'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636f6d3b680 con 0x5636f6d80c00 2023-05-20T01:18:30.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.297+0000 7efc5aa0e700 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:123 7.2f 7:f4000000::::head [pgnls-filter start_epoch 47 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f6ed9880 con 0x5636f6d81000 2023-05-20T01:18:30.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.297+0000 7efc5f217700 1 -- 172.21.15.16:0/3368616031 <== osd.4 v2:172.21.15.132:6817/506317223 10 ==== osd_op_reply(123 [pgnls-filter start_epoch 47 out=49b] v45'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636f6d3b8c0 con 0x5636f6d81000 2023-05-20T01:18:30.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.297+0000 7efc5a20d700 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:124 7.1f 7:f8000000::::head [pgnls-filter start_epoch 47 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f7803c00 con 0x5636f6d80800 2023-05-20T01:18:30.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.297+0000 7efc5ea16700 1 -- 172.21.15.16:0/3368616031 <== osd.6 v2:172.21.15.132:6801/727581123 18 ==== osd_op_reply(124 [pgnls-filter start_epoch 47 out=49b] v45'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636f6d3b440 con 0x5636f6d80800 2023-05-20T01:18:30.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.297+0000 7efc5aa0e700 1 -- 172.21.15.16:0/3368616031 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:125 7.3f 7:fc000000::::head [pgnls-filter start_epoch 47 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x5636f6ed9c00 con 0x5636f6d81400 2023-05-20T01:18:30.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.298+0000 7efc5fa18700 1 -- 172.21.15.16:0/3368616031 <== osd.7 v2:172.21.15.132:6800/466001149 6 ==== osd_op_reply(125 [pgnls-filter start_epoch 47 out=49b] v45'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636f6d3b680 con 0x5636f6d81400 2023-05-20T01:18:30.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.298+0000 7efc612697c0 1 -- 172.21.15.16:0/3368616031 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x5636f6d80c00 msgr2=0x5636f6e9e580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:30.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.298+0000 7efc612697c0 1 --2- 172.21.15.16:0/3368616031 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x5636f6d80c00 0x5636f6e9e580 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:30.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.298+0000 7efc612697c0 1 -- 172.21.15.16:0/3368616031 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x5636f6d4d800 msgr2=0x5636f6e9f600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:30.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.298+0000 7efc612697c0 1 --2- 172.21.15.16:0/3368616031 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x5636f6d4d800 0x5636f6e9f600 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:30.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.298+0000 7efc612697c0 1 -- 172.21.15.16:0/3368616031 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x5636f6d4c800 msgr2=0x5636f6ad8100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:30.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.298+0000 7efc612697c0 1 --2- 172.21.15.16:0/3368616031 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x5636f6d4c800 0x5636f6ad8100 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:30.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.298+0000 7efc612697c0 1 -- 172.21.15.16:0/3368616031 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x5636f6d4cc00 msgr2=0x5636f6ad7b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:30.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.298+0000 7efc612697c0 1 --2- 172.21.15.16:0/3368616031 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x5636f6d4cc00 0x5636f6ad7b80 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:30.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.298+0000 7efc5f217700 1 -- 172.21.15.16:0/3368616031 reap_dead start 2023-05-20T01:18:30.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.298+0000 7efc612697c0 1 -- 172.21.15.16:0/3368616031 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x5636f6d81000 msgr2=0x5636f6e9eb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:30.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.298+0000 7efc612697c0 1 --2- 172.21.15.16:0/3368616031 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x5636f6d81000 0x5636f6e9eb00 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:30.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.298+0000 7efc5f217700 1 -- 172.21.15.16:0/3368616031 reap_dead start 2023-05-20T01:18:30.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.298+0000 7efc612697c0 1 -- 172.21.15.16:0/3368616031 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x5636f6d4d000 msgr2=0x5636f6ad7600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:30.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.298+0000 7efc612697c0 1 --2- 172.21.15.16:0/3368616031 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x5636f6d4d000 0x5636f6ad7600 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:30.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.298+0000 7efc612697c0 1 -- 172.21.15.16:0/3368616031 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x5636f6d80800 msgr2=0x5636f6e9e000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:30.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.298+0000 7efc612697c0 1 --2- 172.21.15.16:0/3368616031 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x5636f6d80800 0x5636f6e9e000 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:30.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.298+0000 7efc612697c0 1 -- 172.21.15.16:0/3368616031 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x5636f6d81400 msgr2=0x5636f6e9f080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:30.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.298+0000 7efc612697c0 1 --2- 172.21.15.16:0/3368616031 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x5636f6d81400 0x5636f6e9f080 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:30.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.298+0000 7efc612697c0 1 -- 172.21.15.16:0/3368616031 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x5636f6d80400 msgr2=0x5636f6ad9700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:30.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.298+0000 7efc612697c0 1 --2- 172.21.15.16:0/3368616031 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x5636f6d80400 0x5636f6ad9700 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x5636f6d52240 tx=0x5636f78b8e40 comp rx=0 tx=0).stop 2023-05-20T01:18:30.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.298+0000 7efc612697c0 1 -- 172.21.15.16:0/3368616031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636f6d4c000 msgr2=0x5636f6ad8c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:30.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.298+0000 7efc5f217700 1 -- 172.21.15.16:0/3368616031 reap_dead start 2023-05-20T01:18:30.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.298+0000 7efc612697c0 1 --2- 172.21.15.16:0/3368616031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636f6d4c000 0x5636f6ad8c00 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x5636f6d53410 tx=0x5636f78b8ea0 comp rx=0 tx=0).stop 2023-05-20T01:18:30.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.298+0000 7efc612697c0 1 -- 172.21.15.16:0/3368616031 shutdown_connections 2023-05-20T01:18:30.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.298+0000 7efc612697c0 1 --2- 172.21.15.16:0/3368616031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636f6d4c000 0x5636f6ad8c00 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:30.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.298+0000 7efc612697c0 1 -- 172.21.15.16:0/3368616031 >> 172.21.15.16:0/3368616031 conn(0x5636f6a3dc00 msgr2=0x5636f6b0d800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:18:30.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.298+0000 7efc612697c0 1 -- 172.21.15.16:0/3368616031 shutdown_connections 2023-05-20T01:18:30.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.298+0000 7efc612697c0 1 -- 172.21.15.16:0/3368616031 wait complete. 2023-05-20T01:18:30.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.299+0000 7efc612697c0 1 -- 172.21.15.16:0/3848076718 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636f6a3c800 msgr2=0x5636f6ad7080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:30.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.299+0000 7efc612697c0 1 --2- 172.21.15.16:0/3848076718 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636f6a3c800 0x5636f6ad7080 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x5636f6b98de0 tx=0x5636f77d53e0 comp rx=0 tx=0).stop 2023-05-20T01:18:30.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.299+0000 7efc612697c0 1 -- 172.21.15.16:0/3848076718 shutdown_connections 2023-05-20T01:18:30.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.299+0000 7efc612697c0 1 --2- 172.21.15.16:0/3848076718 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636f6a3c800 0x5636f6ad7080 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:30.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.299+0000 7efc612697c0 1 --2- 172.21.15.16:0/3848076718 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5636f6a3d000 0x5636f6ad6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:30.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.299+0000 7efc612697c0 1 --2- 172.21.15.16:0/3848076718 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5636f6a3cc00 0x5636f6ad6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:30.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.299+0000 7efc612697c0 1 -- 172.21.15.16:0/3848076718 >> 172.21.15.16:0/3848076718 conn(0x5636f6a3d400 msgr2=0x5636f6b0d000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:18:30.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.299+0000 7efc612697c0 1 -- 172.21.15.16:0/3848076718 shutdown_connections 2023-05-20T01:18:30.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.299+0000 7efc612697c0 1 -- 172.21.15.16:0/3848076718 wait complete. 2023-05-20T01:18:30.353 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_inodes'] in time 0:00:00.179720, result: 2023-05-20T01:18:30.354 DEBUG:teuthology.orchestra.run.smithi016:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_links 2023-05-20T01:18:30.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.408+0000 7ff5f48a17c0 1 -- start start 2023-05-20T01:18:30.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.408+0000 7ff5f48a17c0 1 Processor -- start 2023-05-20T01:18:30.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.409+0000 7ff5f48a17c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ac18a69400 0x55ac18b02b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:30.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.409+0000 7ff5f48a17c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ac18a69c00 0x55ac18b03b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:30.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.409+0000 7ff5f48a17c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac18a69000 0x55ac18b03600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:30.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.409+0000 7ff5f48a17c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac18aa5cc0 con 0x55ac18a69000 2023-05-20T01:18:30.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.409+0000 7ff5f48a17c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac18aa5400 con 0x55ac18a69400 2023-05-20T01:18:30.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.409+0000 7ff5f48a17c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ac18aa4000 con 0x55ac18a69c00 2023-05-20T01:18:30.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.409+0000 7ff5f3050700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac18a69000 0x55ac18b03600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:30.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.409+0000 7ff5f3050700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac18a69000 0x55ac18b03600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32790/0 (socket says 172.21.15.16:32790) 2023-05-20T01:18:30.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.409+0000 7ff5f3050700 1 -- 172.21.15.16:0/333497519 learned_addr learned my addr 172.21.15.16:0/333497519 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:18:30.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.409+0000 7ff5f284f700 1 --2- 172.21.15.16:0/333497519 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ac18a69400 0x55ac18b02b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:30.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.409+0000 7ff5f204e700 1 --2- 172.21.15.16:0/333497519 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ac18a69c00 0x55ac18b03b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:30.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.409+0000 7ff5f3050700 1 -- 172.21.15.16:0/333497519 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ac18a69c00 msgr2=0x55ac18b03b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:18:30.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.409+0000 7ff5f3050700 1 --2- 172.21.15.16:0/333497519 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ac18a69c00 0x55ac18b03b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:30.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.409+0000 7ff5f3050700 1 -- 172.21.15.16:0/333497519 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ac18a69400 msgr2=0x55ac18b02b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:18:30.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.409+0000 7ff5f3050700 1 --2- 172.21.15.16:0/333497519 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ac18a69400 0x55ac18b02b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:30.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.409+0000 7ff5f3050700 1 -- 172.21.15.16:0/333497519 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ac18a8cd00 con 0x55ac18a69000 2023-05-20T01:18:30.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.410+0000 7ff5f3050700 1 --2- 172.21.15.16:0/333497519 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac18a69000 0x55ac18b03600 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x55ac18bc8b40 tx=0x55ac18bceea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e27b658e9ca90e6 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:30.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.410+0000 7ff5f004a700 1 -- 172.21.15.16:0/333497519 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55ac18a9d080 con 0x55ac18a69000 2023-05-20T01:18:30.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.410+0000 7ff5f004a700 1 -- 172.21.15.16:0/333497519 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ac18abad80 con 0x55ac18a69000 2023-05-20T01:18:30.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.410+0000 7ff5f004a700 1 -- 172.21.15.16:0/333497519 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55ac18a9d600 con 0x55ac18a69000 2023-05-20T01:18:30.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.410+0000 7ff5f48a17c0 1 -- 172.21.15.16:0/333497519 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ac18a8d380 con 0x55ac18a69000 2023-05-20T01:18:30.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.410+0000 7ff5f48a17c0 1 -- 172.21.15.16:0/333497519 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55ac18a8dd40 con 0x55ac18a69000 2023-05-20T01:18:30.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.410+0000 7ff5f48a17c0 4 waiting for MDS map... 2023-05-20T01:18:30.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.410+0000 7ff5f004a700 1 -- 172.21.15.16:0/333497519 <== mon.0 v2:172.21.15.16:3300/0 4 ==== osd_map(47..47 src has 1..47) v4 ==== 7016+0+0 (secure 0 0 0) 0x55ac18aaac40 con 0x55ac18a69000 2023-05-20T01:18:30.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.410+0000 7ff5f004a700 1 -- 172.21.15.16:0/333497519 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 22) v1 ==== 3126+0+0 (secure 0 0 0) 0x55ac19831200 con 0x55ac18a69000 2023-05-20T01:18:30.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.410+0000 7ff5f48a17c0 4 Got MDS map 22 2023-05-20T01:18:30.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.410+0000 7ff5f48a17c0 4 datascan.main: Using metadata pool output 2023-05-20T01:18:30.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.410+0000 7ff5f48a17c0 4 datascan.main: connecting to RADOS... 2023-05-20T01:18:30.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.411+0000 7ff5f48a17c0 1 Processor -- start 2023-05-20T01:18:30.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.411+0000 7ff5f48a17c0 1 -- start start 2023-05-20T01:18:30.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.411+0000 7ff5f48a17c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ac18d84400 0x55ac18b02580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:30.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.411+0000 7ff5f48a17c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ac18d84800 0x55ac18b04100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:30.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.411+0000 7ff5f48a17c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac18d84c00 0x55ac18b04680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:30.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.411+0000 7ff5f48a17c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac18aa4c80 con 0x55ac18d84c00 2023-05-20T01:18:30.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.411+0000 7ff5f48a17c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac18aa4b40 con 0x55ac18d84400 2023-05-20T01:18:30.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.411+0000 7ff5f48a17c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ac18aa5400 con 0x55ac18d84800 2023-05-20T01:18:30.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.412+0000 7ff5f204e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac18d84c00 0x55ac18b04680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:30.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.412+0000 7ff5f204e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac18d84c00 0x55ac18b04680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32794/0 (socket says 172.21.15.16:32794) 2023-05-20T01:18:30.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.412+0000 7ff5f204e700 1 -- 172.21.15.16:0/1110705620 learned_addr learned my addr 172.21.15.16:0/1110705620 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:18:30.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.412+0000 7ff5f204e700 1 --2- 172.21.15.16:0/1110705620 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ac18d84400 0x55ac18b02580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:30.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.412+0000 7ff5f284f700 1 --2- 172.21.15.16:0/1110705620 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ac18d84800 0x55ac18b04100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:30.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.412+0000 7ff5f204e700 1 -- 172.21.15.16:0/1110705620 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ac18d84800 msgr2=0x55ac18b04100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:18:30.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.412+0000 7ff5f204e700 1 --2- 172.21.15.16:0/1110705620 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ac18d84800 0x55ac18b04100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:30.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.412+0000 7ff5f204e700 1 -- 172.21.15.16:0/1110705620 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ac18d84400 msgr2=0x55ac18b02580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:18:30.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.412+0000 7ff5f204e700 1 --2- 172.21.15.16:0/1110705620 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ac18d84400 0x55ac18b02580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:30.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.412+0000 7ff5f204e700 1 -- 172.21.15.16:0/1110705620 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ac18d90000 con 0x55ac18d84c00 2023-05-20T01:18:30.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.412+0000 7ff5f204e700 1 --2- 172.21.15.16:0/1110705620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac18d84c00 0x55ac18b04680 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x55ac18d9a0f0 tx=0x55ac18bce060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2c0a28f9ee226b5 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:30.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.412+0000 7ff5ee046700 1 -- 172.21.15.16:0/1110705620 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55ac18a9d080 con 0x55ac18d84c00 2023-05-20T01:18:30.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.412+0000 7ff5ee046700 1 -- 172.21.15.16:0/1110705620 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ac18abbc80 con 0x55ac18d84c00 2023-05-20T01:18:30.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.412+0000 7ff5ee046700 1 -- 172.21.15.16:0/1110705620 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55ac18a9d600 con 0x55ac18d84c00 2023-05-20T01:18:30.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.413+0000 7ff5f48a17c0 1 -- 172.21.15.16:0/1110705620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac18d84c00 msgr2=0x55ac18b04680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:30.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.413+0000 7ff5f48a17c0 1 --2- 172.21.15.16:0/1110705620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac18d84c00 0x55ac18b04680 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x55ac18d9a0f0 tx=0x55ac18bce060 comp rx=0 tx=0).stop 2023-05-20T01:18:30.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.413+0000 7ff5f48a17c0 1 -- 172.21.15.16:0/1110705620 shutdown_connections 2023-05-20T01:18:30.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.413+0000 7ff5f48a17c0 1 --2- 172.21.15.16:0/1110705620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac18d84c00 0x55ac18b04680 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:30.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.413+0000 7ff5f48a17c0 1 --2- 172.21.15.16:0/1110705620 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ac18d84400 0x55ac18b02580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:30.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.413+0000 7ff5f48a17c0 1 --2- 172.21.15.16:0/1110705620 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ac18d84800 0x55ac18b04100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:30.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.413+0000 7ff5f48a17c0 1 -- 172.21.15.16:0/1110705620 >> 172.21.15.16:0/1110705620 conn(0x55ac18d84000 msgr2=0x55ac18b45000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:18:30.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.413+0000 7ff5f48a17c0 1 -- 172.21.15.16:0/1110705620 shutdown_connections 2023-05-20T01:18:30.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.413+0000 7ff5f48a17c0 1 -- 172.21.15.16:0/1110705620 wait complete. 2023-05-20T01:18:30.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.413+0000 7ff5f48a17c0 1 Processor -- start 2023-05-20T01:18:30.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.413+0000 7ff5f48a17c0 1 -- start start 2023-05-20T01:18:30.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.414+0000 7ff5f48a17c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ac18d84400 0x55ac18b02580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:30.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.414+0000 7ff5f48a17c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac18d84800 0x55ac18b04100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:30.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.414+0000 7ff5f48a17c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ac18d84c00 0x55ac18b04680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:30.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.414+0000 7ff5f48a17c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac18aa4a00 con 0x55ac18d84800 2023-05-20T01:18:30.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.414+0000 7ff5f48a17c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac18aa5180 con 0x55ac18d84c00 2023-05-20T01:18:30.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.414+0000 7ff5f48a17c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ac18aa5040 con 0x55ac18d84400 2023-05-20T01:18:30.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.414+0000 7ff5f284f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac18d84800 0x55ac18b04100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:30.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.414+0000 7ff5f284f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac18d84800 0x55ac18b04100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32800/0 (socket says 172.21.15.16:32800) 2023-05-20T01:18:30.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.414+0000 7ff5f284f700 1 -- 172.21.15.16:0/661169254 learned_addr learned my addr 172.21.15.16:0/661169254 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:18:30.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.414+0000 7ff5f204e700 1 --2- 172.21.15.16:0/661169254 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ac18d84c00 0x55ac18b04680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:30.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.414+0000 7ff5f204e700 1 --2- 172.21.15.16:0/661169254 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ac18d84400 0x55ac18b02580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:30.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.414+0000 7ff5f284f700 1 -- 172.21.15.16:0/661169254 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ac18d84400 msgr2=0x55ac18b02580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:30.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.414+0000 7ff5f284f700 1 --2- 172.21.15.16:0/661169254 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ac18d84400 0x55ac18b02580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:30.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.414+0000 7ff5f284f700 1 -- 172.21.15.16:0/661169254 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ac18d84c00 msgr2=0x55ac18b04680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:30.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.414+0000 7ff5f284f700 1 --2- 172.21.15.16:0/661169254 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ac18d84c00 0x55ac18b04680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:30.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.414+0000 7ff5f284f700 1 -- 172.21.15.16:0/661169254 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ac18d90340 con 0x55ac18d84800 2023-05-20T01:18:30.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.414+0000 7ff5f284f700 1 --2- 172.21.15.16:0/661169254 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac18d84800 0x55ac18b04100 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x55ac18d9b4a0 tx=0x55ac1990ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:30.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.414+0000 7ff5ec843700 1 -- 172.21.15.16:0/661169254 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55ac18a9d080 con 0x55ac18d84800 2023-05-20T01:18:30.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.415+0000 7ff5ec843700 1 -- 172.21.15.16:0/661169254 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ac1990a780 con 0x55ac18d84800 2023-05-20T01:18:30.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.415+0000 7ff5f48a17c0 1 -- 172.21.15.16:0/661169254 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55ac18d901a0 con 0x55ac18d84800 2023-05-20T01:18:30.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.415+0000 7ff5ec843700 1 -- 172.21.15.16:0/661169254 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55ac18a9d600 con 0x55ac18d84800 2023-05-20T01:18:30.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.415+0000 7ff5f48a17c0 1 -- 172.21.15.16:0/661169254 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ac18d904e0 con 0x55ac18d84800 2023-05-20T01:18:30.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.415+0000 7ff5f48a17c0 4 datascan.init: resolving metadata pool 6 2023-05-20T01:18:30.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.416+0000 7ff5ec843700 1 -- 172.21.15.16:0/661169254 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x55ac18aca840 con 0x55ac18d84800 2023-05-20T01:18:30.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.416+0000 7ff5ec843700 1 --2- 172.21.15.16:0/661169254 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x55ac18d85800 0x55ac18b05700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:30.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.416+0000 7ff5ec843700 1 -- 172.21.15.16:0/661169254 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 7016+0+0 (secure 0 0 0) 0x55ac18aaaa80 con 0x55ac18d84800 2023-05-20T01:18:30.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.416+0000 7ff5f48a17c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-05-20T01:18:30.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.416+0000 7ff5f204e700 1 --2- 172.21.15.16:0/661169254 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x55ac18d85800 0x55ac18b05700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:30.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.416+0000 7ff5f48a17c0 4 datascan.main: resolving metadata pool 6 2023-05-20T01:18:30.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.416+0000 7ff5f48a17c0 1 --2- 172.21.15.16:0/661169254 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x55ac18d85000 0x55ac18b04c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:30.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.416+0000 7ff5f48a17c0 1 -- 172.21.15.16:0/661169254 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:1 6.0 6:00000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x55ac19906700 con 0x55ac18d85000 2023-05-20T01:18:30.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.416+0000 7ff5f204e700 1 --2- 172.21.15.16:0/661169254 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x55ac18d85000 0x55ac18b04c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:30.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.417+0000 7ff5f204e700 1 --2- 172.21.15.16:0/661169254 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x55ac18d85800 0x55ac18b05700 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x55ac18d9a240 tx=0x55ac18bce2a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:30.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.417+0000 7ff5f204e700 1 --2- 172.21.15.16:0/661169254 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x55ac18d85000 0x55ac18b04c00 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:30.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.417+0000 7ff5f204e700 1 -- 172.21.15.16:0/661169254 <== osd.0 v2:172.21.15.16:6824/3923634179 1 ==== osd_op_reply(1 [pgnls start_epoch 47 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ac18d8b440 con 0x55ac18d85000 2023-05-20T01:18:30.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.417+0000 7ff5f204e700 1 -- 172.21.15.16:0/661169254 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:2 6.10 6:08000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x55ac19906700 con 0x55ac18d85000 2023-05-20T01:18:30.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.417+0000 7ff5f204e700 1 -- 172.21.15.16:0/661169254 <== osd.0 v2:172.21.15.16:6824/3923634179 2 ==== osd_op_reply(2 [pgnls start_epoch 47 out=81b] v47'21 uv21 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55ac18d8b440 con 0x55ac18d85000 2023-05-20T01:18:30.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.417+0000 7ff5f204e700 1 --2- 172.21.15.16:0/661169254 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x55ac18a69800 0x55ac18b03080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:30.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.417+0000 7ff5f204e700 1 -- 172.21.15.16:0/661169254 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:3 6.8 6:10000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x55ac19906700 con 0x55ac18a69800 2023-05-20T01:18:30.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.418+0000 7ff5f3050700 1 --2- 172.21.15.16:0/661169254 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x55ac18a69800 0x55ac18b03080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:30.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.418+0000 7ff5f3050700 1 --2- 172.21.15.16:0/661169254 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x55ac18a69800 0x55ac18b03080 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:30.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.419+0000 7ff5f3050700 1 -- 172.21.15.16:0/661169254 <== osd.7 v2:172.21.15.132:6800/466001149 1 ==== osd_op_reply(3 [pgnls start_epoch 47 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ac18d8b680 con 0x55ac18a69800 2023-05-20T01:18:30.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.419+0000 7ff5f3050700 1 -- 172.21.15.16:0/661169254 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:4 6.18 6:18000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x55ac19906700 con 0x55ac18d85000 2023-05-20T01:18:30.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.419+0000 7ff5f204e700 1 -- 172.21.15.16:0/661169254 <== osd.0 v2:172.21.15.16:6824/3923634179 3 ==== osd_op_reply(4 [pgnls start_epoch 47 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ac18d8b440 con 0x55ac18d85000 2023-05-20T01:18:30.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.419+0000 7ff5f204e700 1 --2- 172.21.15.16:0/661169254 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x55ac18d85c00 0x55ac18eb0000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:30.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.419+0000 7ff5f204e700 1 -- 172.21.15.16:0/661169254 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:5 6.4 6:20000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x55ac19906700 con 0x55ac18d85c00 2023-05-20T01:18:30.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.419+0000 7ff5f284f700 1 --2- 172.21.15.16:0/661169254 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x55ac18d85c00 0x55ac18eb0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:30.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.419+0000 7ff5f284f700 1 --2- 172.21.15.16:0/661169254 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x55ac18d85c00 0x55ac18eb0000 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:30.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.419+0000 7ff5f284f700 1 -- 172.21.15.16:0/661169254 <== osd.1 v2:172.21.15.16:6808/3860264220 1 ==== osd_op_reply(5 [pgnls start_epoch 47 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ac18d8b8c0 con 0x55ac18d85c00 2023-05-20T01:18:30.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.419+0000 7ff5f284f700 1 --2- 172.21.15.16:0/661169254 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x55ac18ebe000 0x55ac18eb0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:30.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.419+0000 7ff5f284f700 1 -- 172.21.15.16:0/661169254 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:6 6.14 6:28000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x55ac19906700 con 0x55ac18ebe000 2023-05-20T01:18:30.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.420+0000 7ff5f204e700 1 --2- 172.21.15.16:0/661169254 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x55ac18ebe000 0x55ac18eb0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:30.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.420+0000 7ff5f204e700 1 --2- 172.21.15.16:0/661169254 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x55ac18ebe000 0x55ac18eb0580 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:30.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.420+0000 7ff5f204e700 1 -- 172.21.15.16:0/661169254 <== osd.2 v2:172.21.15.16:6816/293187030 1 ==== osd_op_reply(6 [pgnls start_epoch 47 out=125b] v47'49 uv49 ondisk = 1) v8 ==== 144+0+125 (crc 0 0 0) 0x55ac18d8b440 con 0x55ac18ebe000 2023-05-20T01:18:30.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.420+0000 7ff5f204e700 1 --2- 172.21.15.16:0/661169254 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x55ac18ebe800 0x55ac18eb0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:30.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.420+0000 7ff5f204e700 1 -- 172.21.15.16:0/661169254 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:7 6.c 6:30000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x55ac19906700 con 0x55ac18ebe800 2023-05-20T01:18:30.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.420+0000 7ff5f3050700 1 --2- 172.21.15.16:0/661169254 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x55ac18ebe800 0x55ac18eb0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:30.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.421+0000 7ff5f3050700 1 --2- 172.21.15.16:0/661169254 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x55ac18ebe800 0x55ac18eb0b00 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:30.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.421+0000 7ff5f3050700 1 -- 172.21.15.16:0/661169254 <== osd.3 v2:172.21.15.16:6800/3669781890 1 ==== osd_op_reply(7 [pgnls start_epoch 47 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ac18d8b680 con 0x55ac18ebe800 2023-05-20T01:18:30.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.421+0000 7ff5f3050700 1 -- 172.21.15.16:0/661169254 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:8 6.1c 6:38000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x55ac19906700 con 0x55ac18a69800 2023-05-20T01:18:30.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.421+0000 7ff5f3050700 1 -- 172.21.15.16:0/661169254 <== osd.7 v2:172.21.15.132:6800/466001149 2 ==== osd_op_reply(8 [pgnls start_epoch 47 out=49b] v46'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ac18d8b680 con 0x55ac18a69800 2023-05-20T01:18:30.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.421+0000 7ff5f3050700 1 --2- 172.21.15.16:0/661169254 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x55ac18a68c00 0x55ac18eb1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:30.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.421+0000 7ff5f3050700 1 -- 172.21.15.16:0/661169254 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:9 6.2 6:40000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x55ac19906700 con 0x55ac18a68c00 2023-05-20T01:18:30.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.421+0000 7ff5f284f700 1 --2- 172.21.15.16:0/661169254 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x55ac18a68c00 0x55ac18eb1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:30.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.422+0000 7ff5f284f700 1 --2- 172.21.15.16:0/661169254 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x55ac18a68c00 0x55ac18eb1600 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:30.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.422+0000 7ff5f284f700 1 -- 172.21.15.16:0/661169254 <== osd.4 v2:172.21.15.132:6817/506317223 1 ==== osd_op_reply(9 [pgnls start_epoch 47 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ac18d8b8c0 con 0x55ac18a68c00 2023-05-20T01:18:30.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.422+0000 7ff5f284f700 1 -- 172.21.15.16:0/661169254 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:10 6.12 6:48000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x55ac19906700 con 0x55ac18a69800 2023-05-20T01:18:30.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.422+0000 7ff5f3050700 1 -- 172.21.15.16:0/661169254 <== osd.7 v2:172.21.15.132:6800/466001149 3 ==== osd_op_reply(10 [pgnls start_epoch 47 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ac18d8b680 con 0x55ac18a69800 2023-05-20T01:18:30.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.422+0000 7ff5f3050700 1 --2- 172.21.15.16:0/661169254 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x55ac18ebf000 0x55ac18eb1b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:30.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.422+0000 7ff5f3050700 1 -- 172.21.15.16:0/661169254 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:11 6.a 6:50000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x55ac19906700 con 0x55ac18ebf000 2023-05-20T01:18:30.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.422+0000 7ff5f204e700 1 --2- 172.21.15.16:0/661169254 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x55ac18ebf000 0x55ac18eb1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:30.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.423+0000 7ff5f204e700 1 --2- 172.21.15.16:0/661169254 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x55ac18ebf000 0x55ac18eb1b80 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:30.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.423+0000 7ff5f204e700 1 -- 172.21.15.16:0/661169254 <== osd.5 v2:172.21.15.132:6816/1844689995 1 ==== osd_op_reply(11 [pgnls start_epoch 47 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ac18d8b440 con 0x55ac18ebf000 2023-05-20T01:18:30.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.423+0000 7ff5f204e700 1 -- 172.21.15.16:0/661169254 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:12 6.1a 6:58000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x55ac19906700 con 0x55ac18a68c00 2023-05-20T01:18:30.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.423+0000 7ff5f284f700 1 -- 172.21.15.16:0/661169254 <== osd.4 v2:172.21.15.132:6817/506317223 2 ==== osd_op_reply(12 [pgnls start_epoch 47 out=49b] v47'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ac18d8b8c0 con 0x55ac18a68c00 2023-05-20T01:18:30.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.423+0000 7ff5f284f700 1 -- 172.21.15.16:0/661169254 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:13 6.6 6:60000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x55ac19906700 con 0x55ac18ebe800 2023-05-20T01:18:30.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.424+0000 7ff5f3050700 1 -- 172.21.15.16:0/661169254 <== osd.3 v2:172.21.15.16:6800/3669781890 2 ==== osd_op_reply(13 [pgnls start_epoch 47 out=49b] v46'9 uv9 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ac18d8b680 con 0x55ac18ebe800 2023-05-20T01:18:30.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.424+0000 7ff5f3050700 1 -- 172.21.15.16:0/661169254 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:14 6.16 6:68000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x55ac19906700 con 0x55ac18d85000 2023-05-20T01:18:30.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.424+0000 7ff5f204e700 1 -- 172.21.15.16:0/661169254 <== osd.0 v2:172.21.15.16:6824/3923634179 4 ==== osd_op_reply(14 [pgnls start_epoch 47 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ac18d8b440 con 0x55ac18d85000 2023-05-20T01:18:30.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.424+0000 7ff5f204e700 1 -- 172.21.15.16:0/661169254 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:15 6.e 6:70000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x55ac19906700 con 0x55ac18a68c00 2023-05-20T01:18:30.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.424+0000 7ff5f284f700 1 -- 172.21.15.16:0/661169254 <== osd.4 v2:172.21.15.132:6817/506317223 3 ==== osd_op_reply(15 [pgnls start_epoch 47 out=49b] v46'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ac18d8b8c0 con 0x55ac18a68c00 2023-05-20T01:18:30.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.424+0000 7ff5f284f700 1 -- 172.21.15.16:0/661169254 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:16 6.1e 6:78000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x55ac19906700 con 0x55ac18a68c00 2023-05-20T01:18:30.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.425+0000 7ff5f284f700 1 -- 172.21.15.16:0/661169254 <== osd.4 v2:172.21.15.132:6817/506317223 4 ==== osd_op_reply(16 [pgnls start_epoch 47 out=49b] v47'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ac18d8b8c0 con 0x55ac18a68c00 2023-05-20T01:18:30.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.425+0000 7ff5f284f700 1 -- 172.21.15.16:0/661169254 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:17 6.1 6:80000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x55ac19906700 con 0x55ac18d85c00 2023-05-20T01:18:30.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.425+0000 7ff5f284f700 1 -- 172.21.15.16:0/661169254 <== osd.1 v2:172.21.15.16:6808/3860264220 2 ==== osd_op_reply(17 [pgnls start_epoch 47 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ac18d8b8c0 con 0x55ac18d85c00 2023-05-20T01:18:30.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.425+0000 7ff5f284f700 1 -- 172.21.15.16:0/661169254 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:18 6.11 6:88000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x55ac19906700 con 0x55ac18ebe800 2023-05-20T01:18:30.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.425+0000 7ff5f3050700 1 -- 172.21.15.16:0/661169254 <== osd.3 v2:172.21.15.16:6800/3669781890 3 ==== osd_op_reply(18 [pgnls start_epoch 47 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ac18d8b680 con 0x55ac18ebe800 2023-05-20T01:18:30.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.425+0000 7ff5f3050700 1 -- 172.21.15.16:0/661169254 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:19 6.9 6:90000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x55ac19906700 con 0x55ac18d85000 2023-05-20T01:18:30.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.425+0000 7ff5f204e700 1 -- 172.21.15.16:0/661169254 <== osd.0 v2:172.21.15.16:6824/3923634179 5 ==== osd_op_reply(19 [pgnls start_epoch 47 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ac18d8b440 con 0x55ac18d85000 2023-05-20T01:18:30.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.425+0000 7ff5f204e700 1 -- 172.21.15.16:0/661169254 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:20 6.19 6:98000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x55ac19906700 con 0x55ac18ebf000 2023-05-20T01:18:30.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.426+0000 7ff5f204e700 1 -- 172.21.15.16:0/661169254 <== osd.5 v2:172.21.15.132:6816/1844689995 2 ==== osd_op_reply(20 [pgnls start_epoch 47 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ac18d8b440 con 0x55ac18ebf000 2023-05-20T01:18:30.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.426+0000 7ff5f204e700 1 -- 172.21.15.16:0/661169254 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:21 6.5 6:a0000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x55ac19906700 con 0x55ac18a69800 2023-05-20T01:18:30.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.426+0000 7ff5f3050700 1 -- 172.21.15.16:0/661169254 <== osd.7 v2:172.21.15.132:6800/466001149 4 ==== osd_op_reply(21 [pgnls start_epoch 47 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ac18d8b680 con 0x55ac18a69800 2023-05-20T01:18:30.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.426+0000 7ff5f3050700 1 -- 172.21.15.16:0/661169254 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:22 6.15 6:a8000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x55ac19906700 con 0x55ac18a69800 2023-05-20T01:18:30.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.426+0000 7ff5f3050700 1 -- 172.21.15.16:0/661169254 <== osd.7 v2:172.21.15.132:6800/466001149 5 ==== osd_op_reply(22 [pgnls start_epoch 47 out=73b] v46'6 uv6 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55ac18d8b680 con 0x55ac18a69800 2023-05-20T01:18:30.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.426+0000 7ff5f3050700 1 -- 172.21.15.16:0/661169254 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:23 6.d 6:b0000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x55ac19906700 con 0x55ac18ebf000 2023-05-20T01:18:30.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.427+0000 7ff5f204e700 1 -- 172.21.15.16:0/661169254 <== osd.5 v2:172.21.15.132:6816/1844689995 3 ==== osd_op_reply(23 [pgnls start_epoch 47 out=74b] v47'3 uv3 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x55ac18d8b440 con 0x55ac18ebf000 2023-05-20T01:18:30.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.427+0000 7ff5f204e700 1 -- 172.21.15.16:0/661169254 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:24 6.1d 6:b8000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x55ac19906700 con 0x55ac18d85c00 2023-05-20T01:18:30.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.427+0000 7ff5f284f700 1 -- 172.21.15.16:0/661169254 <== osd.1 v2:172.21.15.16:6808/3860264220 3 ==== osd_op_reply(24 [pgnls start_epoch 47 out=49b] v46'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ac18d8b8c0 con 0x55ac18d85c00 2023-05-20T01:18:30.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.427+0000 7ff5f284f700 1 -- 172.21.15.16:0/661169254 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:25 6.3 6:c0000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x55ac19906700 con 0x55ac18a69800 2023-05-20T01:18:30.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.427+0000 7ff5f3050700 1 -- 172.21.15.16:0/661169254 <== osd.7 v2:172.21.15.132:6800/466001149 6 ==== osd_op_reply(25 [pgnls start_epoch 47 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ac18d8b680 con 0x55ac18a69800 2023-05-20T01:18:30.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.427+0000 7ff5f3050700 1 -- 172.21.15.16:0/661169254 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:26 6.13 6:c8000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x55ac19906700 con 0x55ac18ebe800 2023-05-20T01:18:30.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.427+0000 7ff5f3050700 1 -- 172.21.15.16:0/661169254 <== osd.3 v2:172.21.15.16:6800/3669781890 4 ==== osd_op_reply(26 [pgnls start_epoch 47 out=98b] v47'7 uv7 ondisk = 1) v8 ==== 144+0+98 (crc 0 0 0) 0x55ac18d8b680 con 0x55ac18ebe800 2023-05-20T01:18:30.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.427+0000 7ff5f3050700 1 -- 172.21.15.16:0/661169254 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:27 6.b 6:d0000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x55ac19906700 con 0x55ac18ebe800 2023-05-20T01:18:30.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.428+0000 7ff5f3050700 1 -- 172.21.15.16:0/661169254 <== osd.3 v2:172.21.15.16:6800/3669781890 5 ==== osd_op_reply(27 [pgnls start_epoch 47 out=76b] v47'6 uv5 ondisk = 1) v8 ==== 144+0+76 (crc 0 0 0) 0x55ac18d8b680 con 0x55ac18ebe800 2023-05-20T01:18:30.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.428+0000 7ff5f3050700 1 -- 172.21.15.16:0/661169254 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:28 6.1b 6:d8000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x55ac19906700 con 0x55ac18ebe800 2023-05-20T01:18:30.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.428+0000 7ff5f3050700 1 -- 172.21.15.16:0/661169254 <== osd.3 v2:172.21.15.16:6800/3669781890 6 ==== osd_op_reply(28 [pgnls start_epoch 47 out=49b] v47'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ac18d8b680 con 0x55ac18ebe800 2023-05-20T01:18:30.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.428+0000 7ff5f3050700 1 -- 172.21.15.16:0/661169254 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:29 6.7 6:e0000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x55ac19906700 con 0x55ac18ebf000 2023-05-20T01:18:30.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.428+0000 7ff5f204e700 1 -- 172.21.15.16:0/661169254 <== osd.5 v2:172.21.15.132:6816/1844689995 4 ==== osd_op_reply(29 [pgnls start_epoch 47 out=49b] v46'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ac18d8b440 con 0x55ac18ebf000 2023-05-20T01:18:30.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.428+0000 7ff5f204e700 1 -- 172.21.15.16:0/661169254 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:30 6.17 6:e8000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x55ac19906700 con 0x55ac18a68c00 2023-05-20T01:18:30.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.428+0000 7ff5f284f700 1 -- 172.21.15.16:0/661169254 <== osd.4 v2:172.21.15.132:6817/506317223 5 ==== osd_op_reply(30 [pgnls start_epoch 47 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ac18d8b8c0 con 0x55ac18a68c00 2023-05-20T01:18:30.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.429+0000 7ff5f284f700 1 -- 172.21.15.16:0/661169254 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:31 6.f 6:f0000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x55ac19906700 con 0x55ac18ebe000 2023-05-20T01:18:30.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.429+0000 7ff5f204e700 1 -- 172.21.15.16:0/661169254 <== osd.2 v2:172.21.15.16:6816/293187030 2 ==== osd_op_reply(31 [pgnls start_epoch 47 out=73b] v47'4 uv4 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55ac18d8b440 con 0x55ac18ebe000 2023-05-20T01:18:30.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.429+0000 7ff5f204e700 1 -- 172.21.15.16:0/661169254 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:32 6.1f 6:f8000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x55ac19906700 con 0x55ac18ebe800 2023-05-20T01:18:30.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.429+0000 7ff5f3050700 1 -- 172.21.15.16:0/661169254 <== osd.3 v2:172.21.15.16:6800/3669781890 7 ==== osd_op_reply(32 [pgnls start_epoch 47 out=101b] v47'12 uv12 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x55ac18d8b680 con 0x55ac18ebe800 2023-05-20T01:18:30.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.429+0000 7ff5f48a17c0 10 datascan.scan_links: step 1: handling object 10000000000.00000000 2023-05-20T01:18:30.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.429+0000 7ff5f48a17c0 1 -- 172.21.15.16:0/661169254 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:33 6.10 6:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47) v8 -- 0x55ac19906a80 con 0x55ac18d85000 2023-05-20T01:18:30.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.429+0000 7ff5f204e700 1 -- 172.21.15.16:0/661169254 <== osd.0 v2:172.21.15.16:6824/3923634179 6 ==== osd_op_reply(33 10000000000.00000000 [omap-get-vals out=2495b] v0'0 uv21 ondisk = 0) v8 ==== 164+0+2495 (crc 0 0 0) 0x55ac18d8b440 con 0x55ac18d85000 2023-05-20T01:18:30.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.429+0000 7ff5f48a17c0 20 datascan.scan_links: injected ino detected 2023-05-20T01:18:30.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.429+0000 7ff5f48a17c0 20 datascan.scan_links: injected ino detected 2023-05-20T01:18:30.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.429+0000 7ff5f48a17c0 20 datascan.scan_links: injected ino detected 2023-05-20T01:18:30.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.429+0000 7ff5f48a17c0 20 datascan.scan_links: injected ino detected 2023-05-20T01:18:30.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.429+0000 7ff5f48a17c0 20 datascan.scan_links: injected ino detected 2023-05-20T01:18:30.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.429+0000 7ff5f48a17c0 10 datascan.scan_links: step 1: handling object 1.00000000.inode 2023-05-20T01:18:30.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.429+0000 7ff5f48a17c0 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2023-05-20T01:18:30.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.429+0000 7ff5f48a17c0 10 datascan.scan_links: step 1: handling object 200.00000000 2023-05-20T01:18:30.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.429+0000 7ff5f48a17c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2023-05-20T01:18:30.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.429+0000 7ff5f48a17c0 10 datascan.scan_links: step 1: handling object 200.00000001 2023-05-20T01:18:30.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.429+0000 7ff5f48a17c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2023-05-20T01:18:30.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.429+0000 7ff5f48a17c0 10 datascan.scan_links: step 1: handling object 500.00000000 2023-05-20T01:18:30.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.429+0000 7ff5f48a17c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2023-05-20T01:18:30.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.429+0000 7ff5f48a17c0 10 datascan.scan_links: step 1: handling object mds_snaptable 2023-05-20T01:18:30.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.429+0000 7ff5f48a17c0 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2023-05-20T01:18:30.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.429+0000 7ff5f48a17c0 10 datascan.scan_links: step 1: handling object mds0_inotable 2023-05-20T01:18:30.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.429+0000 7ff5f48a17c0 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2023-05-20T01:18:30.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.429+0000 7ff5f48a17c0 10 datascan.scan_links: step 1: handling object 100.00000000 2023-05-20T01:18:30.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.429+0000 7ff5f48a17c0 1 -- 172.21.15.16:0/661169254 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:34 6.13 6:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47) v8 -- 0x55ac19906e00 con 0x55ac18ebe800 2023-05-20T01:18:30.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.430+0000 7ff5f3050700 1 -- 172.21.15.16:0/661169254 <== osd.3 v2:172.21.15.16:6800/3669781890 8 ==== osd_op_reply(34 100.00000000 [omap-get-vals out=5b] v0'0 uv7 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55ac18d8b680 con 0x55ac18ebe800 2023-05-20T01:18:30.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.430+0000 7ff5f48a17c0 10 datascan.scan_links: step 1: handling object mds0_sessionmap 2023-05-20T01:18:30.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.430+0000 7ff5f48a17c0 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2023-05-20T01:18:30.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.430+0000 7ff5f48a17c0 10 datascan.scan_links: step 1: handling object 400.00000000 2023-05-20T01:18:30.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.430+0000 7ff5f48a17c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2023-05-20T01:18:30.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.430+0000 7ff5f48a17c0 10 datascan.scan_links: step 1: handling object 100.00000000.inode 2023-05-20T01:18:30.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.430+0000 7ff5f48a17c0 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2023-05-20T01:18:30.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.430+0000 7ff5f48a17c0 10 datascan.scan_links: step 1: handling object 1.00000000 2023-05-20T01:18:30.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.430+0000 7ff5f48a17c0 1 -- 172.21.15.16:0/661169254 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:35 6.1f 6:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47) v8 -- 0x55ac19907180 con 0x55ac18ebe800 2023-05-20T01:18:30.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.430+0000 7ff5f3050700 1 -- 172.21.15.16:0/661169254 <== osd.3 v2:172.21.15.16:6800/3669781890 9 ==== osd_op_reply(35 1.00000000 [omap-get-vals out=508b] v0'0 uv12 ondisk = 0) v8 ==== 154+0+508 (crc 0 0 0) 0x55ac18d8b680 con 0x55ac18ebe800 2023-05-20T01:18:30.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.430+0000 7ff5f48a17c0 20 datascan.scan_links: injected ino detected 2023-05-20T01:18:30.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.430+0000 7ff5f48a17c0 1 -- 172.21.15.16:0/661169254 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:36 6.0 6:00000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x55ac19907500 con 0x55ac18d85000 2023-05-20T01:18:30.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.430+0000 7ff5f204e700 1 -- 172.21.15.16:0/661169254 <== osd.0 v2:172.21.15.16:6824/3923634179 7 ==== osd_op_reply(36 [pgnls start_epoch 47 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ac18d8b440 con 0x55ac18d85000 2023-05-20T01:18:30.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.430+0000 7ff5f204e700 1 -- 172.21.15.16:0/661169254 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:37 6.10 6:08000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x55ac19906a80 con 0x55ac18d85000 2023-05-20T01:18:30.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.431+0000 7ff5f204e700 1 -- 172.21.15.16:0/661169254 <== osd.0 v2:172.21.15.16:6824/3923634179 8 ==== osd_op_reply(37 [pgnls start_epoch 47 out=81b] v47'21 uv21 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55ac18d8b440 con 0x55ac18d85000 2023-05-20T01:18:30.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.431+0000 7ff5f204e700 1 -- 172.21.15.16:0/661169254 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:38 6.8 6:10000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x55ac19906a80 con 0x55ac18a69800 2023-05-20T01:18:30.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.431+0000 7ff5f3050700 1 -- 172.21.15.16:0/661169254 <== osd.7 v2:172.21.15.132:6800/466001149 7 ==== osd_op_reply(38 [pgnls start_epoch 47 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ac18d8b680 con 0x55ac18a69800 2023-05-20T01:18:30.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.431+0000 7ff5f3050700 1 -- 172.21.15.16:0/661169254 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:39 6.18 6:18000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x55ac19906a80 con 0x55ac18d85000 2023-05-20T01:18:30.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.431+0000 7ff5f204e700 1 -- 172.21.15.16:0/661169254 <== osd.0 v2:172.21.15.16:6824/3923634179 9 ==== osd_op_reply(39 [pgnls start_epoch 47 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ac18d8b440 con 0x55ac18d85000 2023-05-20T01:18:30.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.431+0000 7ff5f204e700 1 -- 172.21.15.16:0/661169254 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:40 6.4 6:20000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x55ac19906a80 con 0x55ac18d85c00 2023-05-20T01:18:30.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.431+0000 7ff5f284f700 1 -- 172.21.15.16:0/661169254 <== osd.1 v2:172.21.15.16:6808/3860264220 4 ==== osd_op_reply(40 [pgnls start_epoch 47 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ac18d8b8c0 con 0x55ac18d85c00 2023-05-20T01:18:30.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.431+0000 7ff5f284f700 1 -- 172.21.15.16:0/661169254 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:41 6.14 6:28000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x55ac19906a80 con 0x55ac18ebe000 2023-05-20T01:18:30.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.432+0000 7ff5f204e700 1 -- 172.21.15.16:0/661169254 <== osd.2 v2:172.21.15.16:6816/293187030 3 ==== osd_op_reply(41 [pgnls start_epoch 47 out=125b] v47'49 uv49 ondisk = 1) v8 ==== 144+0+125 (crc 0 0 0) 0x55ac18d8b440 con 0x55ac18ebe000 2023-05-20T01:18:30.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.432+0000 7ff5f204e700 1 -- 172.21.15.16:0/661169254 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:42 6.c 6:30000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x55ac19906a80 con 0x55ac18ebe800 2023-05-20T01:18:30.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.432+0000 7ff5f3050700 1 -- 172.21.15.16:0/661169254 <== osd.3 v2:172.21.15.16:6800/3669781890 10 ==== osd_op_reply(42 [pgnls start_epoch 47 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ac18d8b680 con 0x55ac18ebe800 2023-05-20T01:18:30.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.432+0000 7ff5f3050700 1 -- 172.21.15.16:0/661169254 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:43 6.1c 6:38000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x55ac19906a80 con 0x55ac18a69800 2023-05-20T01:18:30.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.432+0000 7ff5f3050700 1 -- 172.21.15.16:0/661169254 <== osd.7 v2:172.21.15.132:6800/466001149 8 ==== osd_op_reply(43 [pgnls start_epoch 47 out=49b] v46'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ac18d8b680 con 0x55ac18a69800 2023-05-20T01:18:30.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.432+0000 7ff5f3050700 1 -- 172.21.15.16:0/661169254 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:44 6.2 6:40000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x55ac19906a80 con 0x55ac18a68c00 2023-05-20T01:18:30.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.433+0000 7ff5f284f700 1 -- 172.21.15.16:0/661169254 <== osd.4 v2:172.21.15.132:6817/506317223 6 ==== osd_op_reply(44 [pgnls start_epoch 47 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ac18d8b8c0 con 0x55ac18a68c00 2023-05-20T01:18:30.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.433+0000 7ff5f284f700 1 -- 172.21.15.16:0/661169254 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:45 6.12 6:48000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x55ac19906a80 con 0x55ac18a69800 2023-05-20T01:18:30.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.433+0000 7ff5f3050700 1 -- 172.21.15.16:0/661169254 <== osd.7 v2:172.21.15.132:6800/466001149 9 ==== osd_op_reply(45 [pgnls start_epoch 47 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ac18d8b680 con 0x55ac18a69800 2023-05-20T01:18:30.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.433+0000 7ff5f3050700 1 -- 172.21.15.16:0/661169254 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:46 6.a 6:50000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x55ac19906a80 con 0x55ac18ebf000 2023-05-20T01:18:30.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.433+0000 7ff5f204e700 1 -- 172.21.15.16:0/661169254 <== osd.5 v2:172.21.15.132:6816/1844689995 5 ==== osd_op_reply(46 [pgnls start_epoch 47 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ac18d8b440 con 0x55ac18ebf000 2023-05-20T01:18:30.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.433+0000 7ff5f204e700 1 -- 172.21.15.16:0/661169254 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:47 6.1a 6:58000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x55ac19906a80 con 0x55ac18a68c00 2023-05-20T01:18:30.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.433+0000 7ff5f284f700 1 -- 172.21.15.16:0/661169254 <== osd.4 v2:172.21.15.132:6817/506317223 7 ==== osd_op_reply(47 [pgnls start_epoch 47 out=49b] v47'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ac18d8b8c0 con 0x55ac18a68c00 2023-05-20T01:18:30.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.434+0000 7ff5f284f700 1 -- 172.21.15.16:0/661169254 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:48 6.6 6:60000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x55ac19906a80 con 0x55ac18ebe800 2023-05-20T01:18:30.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.434+0000 7ff5f3050700 1 -- 172.21.15.16:0/661169254 <== osd.3 v2:172.21.15.16:6800/3669781890 11 ==== osd_op_reply(48 [pgnls start_epoch 47 out=49b] v46'9 uv9 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ac18d8b680 con 0x55ac18ebe800 2023-05-20T01:18:30.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.434+0000 7ff5f3050700 1 -- 172.21.15.16:0/661169254 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:49 6.16 6:68000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x55ac19906a80 con 0x55ac18d85000 2023-05-20T01:18:30.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.434+0000 7ff5f204e700 1 -- 172.21.15.16:0/661169254 <== osd.0 v2:172.21.15.16:6824/3923634179 10 ==== osd_op_reply(49 [pgnls start_epoch 47 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ac18d8b440 con 0x55ac18d85000 2023-05-20T01:18:30.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.434+0000 7ff5f204e700 1 -- 172.21.15.16:0/661169254 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:50 6.e 6:70000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x55ac19906a80 con 0x55ac18a68c00 2023-05-20T01:18:30.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.435+0000 7ff5f284f700 1 -- 172.21.15.16:0/661169254 <== osd.4 v2:172.21.15.132:6817/506317223 8 ==== osd_op_reply(50 [pgnls start_epoch 47 out=49b] v46'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ac18d8b8c0 con 0x55ac18a68c00 2023-05-20T01:18:30.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.435+0000 7ff5f284f700 1 -- 172.21.15.16:0/661169254 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:51 6.1e 6:78000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x55ac19906a80 con 0x55ac18a68c00 2023-05-20T01:18:30.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.435+0000 7ff5f284f700 1 -- 172.21.15.16:0/661169254 <== osd.4 v2:172.21.15.132:6817/506317223 9 ==== osd_op_reply(51 [pgnls start_epoch 47 out=49b] v47'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ac18d8b8c0 con 0x55ac18a68c00 2023-05-20T01:18:30.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.435+0000 7ff5f284f700 1 -- 172.21.15.16:0/661169254 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:52 6.1 6:80000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x55ac19906a80 con 0x55ac18d85c00 2023-05-20T01:18:30.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.435+0000 7ff5f284f700 1 -- 172.21.15.16:0/661169254 <== osd.1 v2:172.21.15.16:6808/3860264220 5 ==== osd_op_reply(52 [pgnls start_epoch 47 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ac18d8b8c0 con 0x55ac18d85c00 2023-05-20T01:18:30.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.435+0000 7ff5f284f700 1 -- 172.21.15.16:0/661169254 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:53 6.11 6:88000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x55ac19906a80 con 0x55ac18ebe800 2023-05-20T01:18:30.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.435+0000 7ff5f3050700 1 -- 172.21.15.16:0/661169254 <== osd.3 v2:172.21.15.16:6800/3669781890 12 ==== osd_op_reply(53 [pgnls start_epoch 47 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ac18d8b680 con 0x55ac18ebe800 2023-05-20T01:18:30.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.435+0000 7ff5f3050700 1 -- 172.21.15.16:0/661169254 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:54 6.9 6:90000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x55ac19906a80 con 0x55ac18d85000 2023-05-20T01:18:30.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.436+0000 7ff5f204e700 1 -- 172.21.15.16:0/661169254 <== osd.0 v2:172.21.15.16:6824/3923634179 11 ==== osd_op_reply(54 [pgnls start_epoch 47 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ac18d8b440 con 0x55ac18d85000 2023-05-20T01:18:30.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.436+0000 7ff5f204e700 1 -- 172.21.15.16:0/661169254 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:55 6.19 6:98000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x55ac19906a80 con 0x55ac18ebf000 2023-05-20T01:18:30.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.436+0000 7ff5f204e700 1 -- 172.21.15.16:0/661169254 <== osd.5 v2:172.21.15.132:6816/1844689995 6 ==== osd_op_reply(55 [pgnls start_epoch 47 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ac18d8b440 con 0x55ac18ebf000 2023-05-20T01:18:30.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.436+0000 7ff5f204e700 1 -- 172.21.15.16:0/661169254 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:56 6.5 6:a0000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x55ac19906a80 con 0x55ac18a69800 2023-05-20T01:18:30.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.436+0000 7ff5f3050700 1 -- 172.21.15.16:0/661169254 <== osd.7 v2:172.21.15.132:6800/466001149 10 ==== osd_op_reply(56 [pgnls start_epoch 47 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ac18d8b680 con 0x55ac18a69800 2023-05-20T01:18:30.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.436+0000 7ff5f3050700 1 -- 172.21.15.16:0/661169254 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:57 6.15 6:a8000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x55ac19906a80 con 0x55ac18a69800 2023-05-20T01:18:30.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.436+0000 7ff5f3050700 1 -- 172.21.15.16:0/661169254 <== osd.7 v2:172.21.15.132:6800/466001149 11 ==== osd_op_reply(57 [pgnls start_epoch 47 out=73b] v46'6 uv6 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55ac18d8b680 con 0x55ac18a69800 2023-05-20T01:18:30.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.437+0000 7ff5f3050700 1 -- 172.21.15.16:0/661169254 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:58 6.d 6:b0000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x55ac19906a80 con 0x55ac18ebf000 2023-05-20T01:18:30.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.437+0000 7ff5f204e700 1 -- 172.21.15.16:0/661169254 <== osd.5 v2:172.21.15.132:6816/1844689995 7 ==== osd_op_reply(58 [pgnls start_epoch 47 out=74b] v47'3 uv3 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x55ac18d8b440 con 0x55ac18ebf000 2023-05-20T01:18:30.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.437+0000 7ff5f204e700 1 -- 172.21.15.16:0/661169254 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:59 6.1d 6:b8000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x55ac19906a80 con 0x55ac18d85c00 2023-05-20T01:18:30.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.437+0000 7ff5f284f700 1 -- 172.21.15.16:0/661169254 <== osd.1 v2:172.21.15.16:6808/3860264220 6 ==== osd_op_reply(59 [pgnls start_epoch 47 out=49b] v46'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ac18d8b8c0 con 0x55ac18d85c00 2023-05-20T01:18:30.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.437+0000 7ff5f284f700 1 -- 172.21.15.16:0/661169254 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:60 6.3 6:c0000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x55ac19906a80 con 0x55ac18a69800 2023-05-20T01:18:30.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.437+0000 7ff5f3050700 1 -- 172.21.15.16:0/661169254 <== osd.7 v2:172.21.15.132:6800/466001149 12 ==== osd_op_reply(60 [pgnls start_epoch 47 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ac18d8b680 con 0x55ac18a69800 2023-05-20T01:18:30.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.437+0000 7ff5f3050700 1 -- 172.21.15.16:0/661169254 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:61 6.13 6:c8000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x55ac19906a80 con 0x55ac18ebe800 2023-05-20T01:18:30.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.438+0000 7ff5f3050700 1 -- 172.21.15.16:0/661169254 <== osd.3 v2:172.21.15.16:6800/3669781890 13 ==== osd_op_reply(61 [pgnls start_epoch 47 out=98b] v47'7 uv7 ondisk = 1) v8 ==== 144+0+98 (crc 0 0 0) 0x55ac18d8b680 con 0x55ac18ebe800 2023-05-20T01:18:30.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.438+0000 7ff5f3050700 1 -- 172.21.15.16:0/661169254 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:62 6.b 6:d0000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x55ac19906a80 con 0x55ac18ebe800 2023-05-20T01:18:30.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.438+0000 7ff5f3050700 1 -- 172.21.15.16:0/661169254 <== osd.3 v2:172.21.15.16:6800/3669781890 14 ==== osd_op_reply(62 [pgnls start_epoch 47 out=76b] v47'6 uv5 ondisk = 1) v8 ==== 144+0+76 (crc 0 0 0) 0x55ac18d8b680 con 0x55ac18ebe800 2023-05-20T01:18:30.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.438+0000 7ff5f3050700 1 -- 172.21.15.16:0/661169254 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:63 6.1b 6:d8000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x55ac19906a80 con 0x55ac18ebe800 2023-05-20T01:18:30.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.438+0000 7ff5f3050700 1 -- 172.21.15.16:0/661169254 <== osd.3 v2:172.21.15.16:6800/3669781890 15 ==== osd_op_reply(63 [pgnls start_epoch 47 out=49b] v47'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ac18d8b680 con 0x55ac18ebe800 2023-05-20T01:18:30.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.438+0000 7ff5f3050700 1 -- 172.21.15.16:0/661169254 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:64 6.7 6:e0000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x55ac19906a80 con 0x55ac18ebf000 2023-05-20T01:18:30.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.438+0000 7ff5f204e700 1 -- 172.21.15.16:0/661169254 <== osd.5 v2:172.21.15.132:6816/1844689995 8 ==== osd_op_reply(64 [pgnls start_epoch 47 out=49b] v46'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ac18d8b440 con 0x55ac18ebf000 2023-05-20T01:18:30.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.438+0000 7ff5f204e700 1 -- 172.21.15.16:0/661169254 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:65 6.17 6:e8000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x55ac19906a80 con 0x55ac18a68c00 2023-05-20T01:18:30.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.439+0000 7ff5f284f700 1 -- 172.21.15.16:0/661169254 <== osd.4 v2:172.21.15.132:6817/506317223 10 ==== osd_op_reply(65 [pgnls start_epoch 47 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ac18d8b8c0 con 0x55ac18a68c00 2023-05-20T01:18:30.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.439+0000 7ff5f284f700 1 -- 172.21.15.16:0/661169254 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:66 6.f 6:f0000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x55ac19906a80 con 0x55ac18ebe000 2023-05-20T01:18:30.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.439+0000 7ff5f204e700 1 -- 172.21.15.16:0/661169254 <== osd.2 v2:172.21.15.16:6816/293187030 4 ==== osd_op_reply(66 [pgnls start_epoch 47 out=73b] v47'4 uv4 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55ac18d8b440 con 0x55ac18ebe000 2023-05-20T01:18:30.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.439+0000 7ff5f204e700 1 -- 172.21.15.16:0/661169254 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:67 6.1f 6:f8000000::::head [pgnls start_epoch 47 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e47) v8 -- 0x55ac19906a80 con 0x55ac18ebe800 2023-05-20T01:18:30.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.439+0000 7ff5f3050700 1 -- 172.21.15.16:0/661169254 <== osd.3 v2:172.21.15.16:6800/3669781890 16 ==== osd_op_reply(67 [pgnls start_epoch 47 out=101b] v47'12 uv12 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x55ac18d8b680 con 0x55ac18ebe800 2023-05-20T01:18:30.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.439+0000 7ff5f48a17c0 10 datascan.scan_links: step 2: handling object 10000000000.00000000 2023-05-20T01:18:30.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.439+0000 7ff5f48a17c0 1 -- 172.21.15.16:0/661169254 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:68 6.10 6:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47) v8 -- 0x55ac19907880 con 0x55ac18d85000 2023-05-20T01:18:30.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.439+0000 7ff5f204e700 1 -- 172.21.15.16:0/661169254 <== osd.0 v2:172.21.15.16:6824/3923634179 12 ==== osd_op_reply(68 10000000000.00000000 [omap-get-vals out=2495b] v0'0 uv21 ondisk = 0) v8 ==== 164+0+2495 (crc 0 0 0) 0x55ac18d8b440 con 0x55ac18d85000 2023-05-20T01:18:30.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.439+0000 7ff5f48a17c0 20 datascan.scan_links: injected ino detected 2023-05-20T01:18:30.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.439+0000 7ff5f48a17c0 20 datascan.scan_links: adding 0x10000000001 for future processing to fix dnfirst 2023-05-20T01:18:30.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.439+0000 7ff5f48a17c0 20 datascan.scan_links: injected ino detected 2023-05-20T01:18:30.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.439+0000 7ff5f48a17c0 20 datascan.scan_links: adding 0x100000001f6 for future processing to fix dnfirst 2023-05-20T01:18:30.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.439+0000 7ff5f48a17c0 20 datascan.scan_links: injected ino detected 2023-05-20T01:18:30.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.439+0000 7ff5f48a17c0 20 datascan.scan_links: adding 0x100000001f7 for future processing to fix dnfirst 2023-05-20T01:18:30.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.439+0000 7ff5f48a17c0 20 datascan.scan_links: injected ino detected 2023-05-20T01:18:30.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.439+0000 7ff5f48a17c0 20 datascan.scan_links: adding 0x100000001f8 for future processing to fix dnfirst 2023-05-20T01:18:30.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.439+0000 7ff5f48a17c0 20 datascan.scan_links: injected ino detected 2023-05-20T01:18:30.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.439+0000 7ff5f48a17c0 20 datascan.scan_links: adding 0x100000001f9 for future processing to fix dnfirst 2023-05-20T01:18:30.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.439+0000 7ff5f48a17c0 10 datascan.scan_links: step 2: handling object 1.00000000.inode 2023-05-20T01:18:30.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.439+0000 7ff5f48a17c0 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2023-05-20T01:18:30.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.439+0000 7ff5f48a17c0 10 datascan.scan_links: step 2: handling object 200.00000000 2023-05-20T01:18:30.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.439+0000 7ff5f48a17c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2023-05-20T01:18:30.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.439+0000 7ff5f48a17c0 10 datascan.scan_links: step 2: handling object 200.00000001 2023-05-20T01:18:30.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.439+0000 7ff5f48a17c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2023-05-20T01:18:30.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.439+0000 7ff5f48a17c0 10 datascan.scan_links: step 2: handling object 500.00000000 2023-05-20T01:18:30.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.439+0000 7ff5f48a17c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2023-05-20T01:18:30.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.439+0000 7ff5f48a17c0 10 datascan.scan_links: step 2: handling object mds_snaptable 2023-05-20T01:18:30.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.439+0000 7ff5f48a17c0 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2023-05-20T01:18:30.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.439+0000 7ff5f48a17c0 10 datascan.scan_links: step 2: handling object mds0_inotable 2023-05-20T01:18:30.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.439+0000 7ff5f48a17c0 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2023-05-20T01:18:30.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.439+0000 7ff5f48a17c0 10 datascan.scan_links: step 2: handling object 100.00000000 2023-05-20T01:18:30.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.439+0000 7ff5f48a17c0 1 -- 172.21.15.16:0/661169254 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:69 6.13 6:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47) v8 -- 0x55ac19907c00 con 0x55ac18ebe800 2023-05-20T01:18:30.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.440+0000 7ff5f3050700 1 -- 172.21.15.16:0/661169254 <== osd.3 v2:172.21.15.16:6800/3669781890 17 ==== osd_op_reply(69 100.00000000 [omap-get-vals out=5b] v0'0 uv7 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55ac18d8b680 con 0x55ac18ebe800 2023-05-20T01:18:30.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.440+0000 7ff5f48a17c0 10 datascan.scan_links: step 2: handling object mds0_sessionmap 2023-05-20T01:18:30.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.440+0000 7ff5f48a17c0 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2023-05-20T01:18:30.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.440+0000 7ff5f48a17c0 10 datascan.scan_links: step 2: handling object 400.00000000 2023-05-20T01:18:30.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.440+0000 7ff5f48a17c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2023-05-20T01:18:30.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.440+0000 7ff5f48a17c0 10 datascan.scan_links: step 2: handling object 100.00000000.inode 2023-05-20T01:18:30.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.440+0000 7ff5f48a17c0 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2023-05-20T01:18:30.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.440+0000 7ff5f48a17c0 10 datascan.scan_links: step 2: handling object 1.00000000 2023-05-20T01:18:30.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.440+0000 7ff5f48a17c0 1 -- 172.21.15.16:0/661169254 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:70 6.1f 6:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47) v8 -- 0x55ac19906700 con 0x55ac18ebe800 2023-05-20T01:18:30.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.440+0000 7ff5f3050700 1 -- 172.21.15.16:0/661169254 <== osd.3 v2:172.21.15.16:6800/3669781890 18 ==== osd_op_reply(70 1.00000000 [omap-get-vals out=508b] v0'0 uv12 ondisk = 0) v8 ==== 154+0+508 (crc 0 0 0) 0x55ac18d8b680 con 0x55ac18ebe800 2023-05-20T01:18:30.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.440+0000 7ff5f48a17c0 20 datascan.scan_links: injected ino detected 2023-05-20T01:18:30.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.440+0000 7ff5f48a17c0 20 datascan.scan_links: adding 0x10000000000 for future processing to fix dnfirst 2023-05-20T01:18:30.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.440+0000 7ff5f48a17c0 10 datascan.scan_links: processing 0 dup_primaries, 0 remote_links 2023-05-20T01:18:30.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.440+0000 7ff5f48a17c0 10 datascan.scan_links: removing dup dentries from 0 objects 2023-05-20T01:18:30.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.440+0000 7ff5f48a17c0 10 datascan.scan_links: processing 0 bad_nlink_inos 2023-05-20T01:18:30.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.440+0000 7ff5f48a17c0 10 datascan.scan_links: processing 6 injected_inos 2023-05-20T01:18:30.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.440+0000 7ff5f48a17c0 10 datascan.scan_links: handling injected_ino 0x10000000000 2023-05-20T01:18:30.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.440+0000 7ff5f48a17c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x1:subdir 2023-05-20T01:18:30.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.440+0000 7ff5f48a17c0 1 -- 172.21.15.16:0/661169254 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:71 6.1f 6:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47) v8 -- 0x55ac19906e00 con 0x55ac18ebe800 2023-05-20T01:18:30.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.440+0000 7ff5f3050700 1 -- 172.21.15.16:0/661169254 <== osd.3 v2:172.21.15.16:6800/3669781890 19 ==== osd_op_reply(71 1.00000000 [omap-get-vals-by-keys out=507b] v0'0 uv12 ondisk = 0) v8 ==== 154+0+507 (crc 0 0 0) 0x55ac18d8b680 con 0x55ac18ebe800 2023-05-20T01:18:30.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.440+0000 7ff5f48a17c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-05-20T01:18:30.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.440+0000 7ff5f48a17c0 20 datascan.scan_links: first is now 2 2023-05-20T01:18:30.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.440+0000 7ff5f48a17c0 1 -- 172.21.15.16:0/661169254 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:72 6.1f 6:ff5b34d6:::1.00000000:head [omap-set-vals in=507b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47) v8 -- 0x55ac19907180 con 0x55ac18ebe800 2023-05-20T01:18:30.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.441+0000 7ff5f3050700 1 -- 172.21.15.16:0/661169254 <== osd.3 v2:172.21.15.16:6800/3669781890 20 ==== osd_op_reply(72 1.00000000 [omap-set-vals] v47'13 uv13 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x55ac18d8b680 con 0x55ac18ebe800 2023-05-20T01:18:30.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.441+0000 7ff5f48a17c0 20 datascan.inject_linkage: Injected dentry 0x0x1/subdir pointing to 0x0x10000000000 2023-05-20T01:18:30.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.441+0000 7ff5f48a17c0 10 datascan.scan_links: handling injected_ino 0x10000000001 2023-05-20T01:18:30.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.441+0000 7ff5f48a17c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:0 2023-05-20T01:18:30.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.441+0000 7ff5f48a17c0 1 -- 172.21.15.16:0/661169254 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:73 6.10 6:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47) v8 -- 0x55ac19906a80 con 0x55ac18d85000 2023-05-20T01:18:30.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.442+0000 7ff5f204e700 1 -- 172.21.15.16:0/661169254 <== osd.0 v2:172.21.15.16:6824/3923634179 13 ==== osd_op_reply(73 10000000000.00000000 [omap-get-vals-by-keys out=502b] v0'0 uv21 ondisk = 0) v8 ==== 164+0+502 (crc 0 0 0) 0x55ac18d8b440 con 0x55ac18d85000 2023-05-20T01:18:30.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.442+0000 7ff5f48a17c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=0 frag=*, r=0 2023-05-20T01:18:30.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.442+0000 7ff5f48a17c0 20 datascan.scan_links: first is now 2 2023-05-20T01:18:30.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.442+0000 7ff5f48a17c0 1 -- 172.21.15.16:0/661169254 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:74 6.10 6:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47) v8 -- 0x55ac19907c00 con 0x55ac18d85000 2023-05-20T01:18:30.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.443+0000 7ff5f204e700 1 -- 172.21.15.16:0/661169254 <== osd.0 v2:172.21.15.16:6824/3923634179 14 ==== osd_op_reply(74 10000000000.00000000 [omap-set-vals] v47'22 uv22 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55ac18d8b440 con 0x55ac18d85000 2023-05-20T01:18:30.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.443+0000 7ff5f48a17c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/0 pointing to 0x0x10000000001 2023-05-20T01:18:30.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.443+0000 7ff5f48a17c0 10 datascan.scan_links: handling injected_ino 0x100000001f6 2023-05-20T01:18:30.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.443+0000 7ff5f48a17c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:1 2023-05-20T01:18:30.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.443+0000 7ff5f48a17c0 1 -- 172.21.15.16:0/661169254 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:75 6.10 6:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47) v8 -- 0x55ac19906700 con 0x55ac18d85000 2023-05-20T01:18:30.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.443+0000 7ff5f204e700 1 -- 172.21.15.16:0/661169254 <== osd.0 v2:172.21.15.16:6824/3923634179 15 ==== osd_op_reply(75 10000000000.00000000 [omap-get-vals-by-keys out=502b] v0'0 uv22 ondisk = 0) v8 ==== 164+0+502 (crc 0 0 0) 0x55ac18d8b440 con 0x55ac18d85000 2023-05-20T01:18:30.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.443+0000 7ff5f48a17c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=1 frag=*, r=0 2023-05-20T01:18:30.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.443+0000 7ff5f48a17c0 20 datascan.scan_links: first is now 2 2023-05-20T01:18:30.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.443+0000 7ff5f48a17c0 1 -- 172.21.15.16:0/661169254 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:76 6.10 6:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47) v8 -- 0x55ac19906e00 con 0x55ac18d85000 2023-05-20T01:18:30.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.444+0000 7ff5f204e700 1 -- 172.21.15.16:0/661169254 <== osd.0 v2:172.21.15.16:6824/3923634179 16 ==== osd_op_reply(76 10000000000.00000000 [omap-set-vals] v47'23 uv23 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55ac18d8b440 con 0x55ac18d85000 2023-05-20T01:18:30.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.444+0000 7ff5f48a17c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/1 pointing to 0x0x100000001f6 2023-05-20T01:18:30.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.444+0000 7ff5f48a17c0 10 datascan.scan_links: handling injected_ino 0x100000001f7 2023-05-20T01:18:30.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.444+0000 7ff5f48a17c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:2 2023-05-20T01:18:30.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.444+0000 7ff5f48a17c0 1 -- 172.21.15.16:0/661169254 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:77 6.10 6:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47) v8 -- 0x55ac19907500 con 0x55ac18d85000 2023-05-20T01:18:30.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.444+0000 7ff5f204e700 1 -- 172.21.15.16:0/661169254 <== osd.0 v2:172.21.15.16:6824/3923634179 17 ==== osd_op_reply(77 10000000000.00000000 [omap-get-vals-by-keys out=502b] v0'0 uv23 ondisk = 0) v8 ==== 164+0+502 (crc 0 0 0) 0x55ac18d8b440 con 0x55ac18d85000 2023-05-20T01:18:30.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.444+0000 7ff5f48a17c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=2 frag=*, r=0 2023-05-20T01:18:30.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.444+0000 7ff5f48a17c0 20 datascan.scan_links: first is now 2 2023-05-20T01:18:30.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.444+0000 7ff5f48a17c0 1 -- 172.21.15.16:0/661169254 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:78 6.10 6:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47) v8 -- 0x55ac19907880 con 0x55ac18d85000 2023-05-20T01:18:30.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.447+0000 7ff5f204e700 1 -- 172.21.15.16:0/661169254 <== osd.0 v2:172.21.15.16:6824/3923634179 18 ==== osd_op_reply(78 10000000000.00000000 [omap-set-vals] v47'24 uv24 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55ac18d8b440 con 0x55ac18d85000 2023-05-20T01:18:30.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.447+0000 7ff5f48a17c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/2 pointing to 0x0x100000001f7 2023-05-20T01:18:30.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.447+0000 7ff5f48a17c0 10 datascan.scan_links: handling injected_ino 0x100000001f8 2023-05-20T01:18:30.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.447+0000 7ff5f48a17c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:3 2023-05-20T01:18:30.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.447+0000 7ff5f48a17c0 1 -- 172.21.15.16:0/661169254 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:79 6.10 6:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47) v8 -- 0x55ac19906a80 con 0x55ac18d85000 2023-05-20T01:18:30.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.447+0000 7ff5f204e700 1 -- 172.21.15.16:0/661169254 <== osd.0 v2:172.21.15.16:6824/3923634179 19 ==== osd_op_reply(79 10000000000.00000000 [omap-get-vals-by-keys out=502b] v0'0 uv24 ondisk = 0) v8 ==== 164+0+502 (crc 0 0 0) 0x55ac18d8b440 con 0x55ac18d85000 2023-05-20T01:18:30.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.447+0000 7ff5f48a17c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=3 frag=*, r=0 2023-05-20T01:18:30.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.447+0000 7ff5f48a17c0 20 datascan.scan_links: first is now 2 2023-05-20T01:18:30.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.447+0000 7ff5f48a17c0 1 -- 172.21.15.16:0/661169254 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:80 6.10 6:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47) v8 -- 0x55ac19907c00 con 0x55ac18d85000 2023-05-20T01:18:30.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.452+0000 7ff5f204e700 1 -- 172.21.15.16:0/661169254 <== osd.0 v2:172.21.15.16:6824/3923634179 20 ==== osd_op_reply(80 10000000000.00000000 [omap-set-vals] v47'25 uv25 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55ac18d8b440 con 0x55ac18d85000 2023-05-20T01:18:30.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.452+0000 7ff5f48a17c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/3 pointing to 0x0x100000001f8 2023-05-20T01:18:30.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.452+0000 7ff5f48a17c0 10 datascan.scan_links: handling injected_ino 0x100000001f9 2023-05-20T01:18:30.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.452+0000 7ff5f48a17c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:4 2023-05-20T01:18:30.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.452+0000 7ff5f48a17c0 1 -- 172.21.15.16:0/661169254 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:81 6.10 6:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47) v8 -- 0x55ac19906700 con 0x55ac18d85000 2023-05-20T01:18:30.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.453+0000 7ff5f204e700 1 -- 172.21.15.16:0/661169254 <== osd.0 v2:172.21.15.16:6824/3923634179 21 ==== osd_op_reply(81 10000000000.00000000 [omap-get-vals-by-keys out=502b] v0'0 uv25 ondisk = 0) v8 ==== 164+0+502 (crc 0 0 0) 0x55ac18d8b440 con 0x55ac18d85000 2023-05-20T01:18:30.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.453+0000 7ff5f48a17c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=4 frag=*, r=0 2023-05-20T01:18:30.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.453+0000 7ff5f48a17c0 20 datascan.scan_links: first is now 2 2023-05-20T01:18:30.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.453+0000 7ff5f48a17c0 1 -- 172.21.15.16:0/661169254 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:82 6.10 6:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47) v8 -- 0x55ac19906e00 con 0x55ac18d85000 2023-05-20T01:18:30.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.454+0000 7ff5f204e700 1 -- 172.21.15.16:0/661169254 <== osd.0 v2:172.21.15.16:6824/3923634179 22 ==== osd_op_reply(82 10000000000.00000000 [omap-set-vals] v47'26 uv26 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55ac18d8b440 con 0x55ac18d85000 2023-05-20T01:18:30.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.454+0000 7ff5f48a17c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/4 pointing to 0x0x100000001f9 2023-05-20T01:18:30.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.454+0000 7ff5f48a17c0 10 datascan.scan_links: updating inotable 2023-05-20T01:18:30.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.454+0000 7ff5f48a17c0 1 -- 172.21.15.16:0/661169254 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:83 6.13 6:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47) v8 -- 0x55ac19907500 con 0x55ac18ebe800 2023-05-20T01:18:30.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.454+0000 7ff5f3050700 1 -- 172.21.15.16:0/661169254 <== osd.3 v2:172.21.15.16:6800/3669781890 21 ==== osd_op_reply(83 mds0_inotable [read 0~34 out=34b] v0'0 uv6 ondisk = 0) v8 ==== 157+0+34 (crc 0 0 0) 0x55ac18d8b680 con 0x55ac18ebe800 2023-05-20T01:18:30.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.454+0000 7ff5f48a17c0 10 mds.0.inotable: skip_inos was [0x10000000000~0x10000000000] 2023-05-20T01:18:30.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.454+0000 7ff5f48a17c0 10 mds.0.inotable: skip_inos now [0x100000001fa~0xfffffffe06] 2023-05-20T01:18:30.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.454+0000 7ff5f48a17c0 1 -- 172.21.15.16:0/661169254 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:84 6.13 6:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47) v8 -- 0x55ac19907880 con 0x55ac18ebe800 2023-05-20T01:18:30.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.455+0000 7ff5f3050700 1 -- 172.21.15.16:0/661169254 <== osd.3 v2:172.21.15.16:6800/3669781890 22 ==== osd_op_reply(84 mds0_inotable [writefull 0~34] v47'8 uv8 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x55ac18d8b680 con 0x55ac18ebe800 2023-05-20T01:18:30.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.455+0000 7ff5f48a17c0 10 datascan.scan_links: updating snaptable 2023-05-20T01:18:30.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.455+0000 7ff5f48a17c0 1 -- 172.21.15.16:0/661169254 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:85 6.d 6:b50e409b:::mds_snaptable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47) v8 -- 0x55ac19906a80 con 0x55ac18ebf000 2023-05-20T01:18:30.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.456+0000 7ff5f204e700 1 -- 172.21.15.16:0/661169254 <== osd.5 v2:172.21.15.132:6816/1844689995 9 ==== osd_op_reply(85 mds_snaptable [read 0~70 out=70b] v0'0 uv3 ondisk = 0) v8 ==== 157+0+70 (crc 0 0 0) 0x55ac18d8b440 con 0x55ac18ebf000 2023-05-20T01:18:30.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.456+0000 7ff5f48a17c0 1 -- 172.21.15.16:0/661169254 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x55ac18d85000 msgr2=0x55ac18b04c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:30.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.456+0000 7ff5f48a17c0 1 --2- 172.21.15.16:0/661169254 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x55ac18d85000 0x55ac18b04c00 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:30.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.456+0000 7ff5f48a17c0 1 -- 172.21.15.16:0/661169254 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x55ac18d85c00 msgr2=0x55ac18eb0000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:30.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.456+0000 7ff5f48a17c0 1 --2- 172.21.15.16:0/661169254 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x55ac18d85c00 0x55ac18eb0000 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:30.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.456+0000 7ff5f48a17c0 1 -- 172.21.15.16:0/661169254 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x55ac18ebe000 msgr2=0x55ac18eb0580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:30.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.456+0000 7ff5f48a17c0 1 --2- 172.21.15.16:0/661169254 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x55ac18ebe000 0x55ac18eb0580 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:30.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.456+0000 7ff5f48a17c0 1 -- 172.21.15.16:0/661169254 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x55ac18ebe800 msgr2=0x55ac18eb0b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:30.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.456+0000 7ff5f48a17c0 1 --2- 172.21.15.16:0/661169254 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x55ac18ebe800 0x55ac18eb0b00 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:30.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.456+0000 7ff5f48a17c0 1 -- 172.21.15.16:0/661169254 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x55ac18a68c00 msgr2=0x55ac18eb1600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:30.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.456+0000 7ff5f48a17c0 1 --2- 172.21.15.16:0/661169254 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x55ac18a68c00 0x55ac18eb1600 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:30.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.456+0000 7ff5f48a17c0 1 -- 172.21.15.16:0/661169254 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x55ac18ebf000 msgr2=0x55ac18eb1b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:30.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.456+0000 7ff5f284f700 1 -- 172.21.15.16:0/661169254 reap_dead start 2023-05-20T01:18:30.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.456+0000 7ff5f48a17c0 1 --2- 172.21.15.16:0/661169254 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x55ac18ebf000 0x55ac18eb1b80 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:30.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.456+0000 7ff5f284f700 1 -- 172.21.15.16:0/661169254 reap_dead start 2023-05-20T01:18:30.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.456+0000 7ff5f284f700 1 -- 172.21.15.16:0/661169254 reap_dead start 2023-05-20T01:18:30.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.456+0000 7ff5f48a17c0 1 -- 172.21.15.16:0/661169254 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x55ac18a69800 msgr2=0x55ac18b03080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:30.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.456+0000 7ff5f48a17c0 1 --2- 172.21.15.16:0/661169254 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x55ac18a69800 0x55ac18b03080 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:30.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.456+0000 7ff5f48a17c0 1 -- 172.21.15.16:0/661169254 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x55ac18d85800 msgr2=0x55ac18b05700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:30.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.456+0000 7ff5f48a17c0 1 --2- 172.21.15.16:0/661169254 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x55ac18d85800 0x55ac18b05700 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x55ac18d9a240 tx=0x55ac18bce2a0 comp rx=0 tx=0).stop 2023-05-20T01:18:30.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.456+0000 7ff5f48a17c0 1 -- 172.21.15.16:0/661169254 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac18d84800 msgr2=0x55ac18b04100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:30.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.456+0000 7ff5f48a17c0 1 --2- 172.21.15.16:0/661169254 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac18d84800 0x55ac18b04100 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x55ac18d9b4a0 tx=0x55ac1990ec00 comp rx=0 tx=0).stop 2023-05-20T01:18:30.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.456+0000 7ff5f48a17c0 1 -- 172.21.15.16:0/661169254 shutdown_connections 2023-05-20T01:18:30.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.456+0000 7ff5f48a17c0 1 --2- 172.21.15.16:0/661169254 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x55ac18d85800 0x55ac18b05700 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:30.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.456+0000 7ff5f48a17c0 1 --2- 172.21.15.16:0/661169254 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x55ac18a69800 0x55ac18b03080 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:30.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.456+0000 7ff5f48a17c0 1 --2- 172.21.15.16:0/661169254 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac18d84800 0x55ac18b04100 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:30.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.456+0000 7ff5f48a17c0 1 -- 172.21.15.16:0/661169254 >> 172.21.15.16:0/661169254 conn(0x55ac18d84000 msgr2=0x55ac18b45000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:18:30.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.456+0000 7ff5f48a17c0 1 -- 172.21.15.16:0/661169254 shutdown_connections 2023-05-20T01:18:30.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.456+0000 7ff5f48a17c0 1 -- 172.21.15.16:0/661169254 wait complete. 2023-05-20T01:18:30.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.457+0000 7ff5f48a17c0 1 -- 172.21.15.16:0/333497519 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac18a69000 msgr2=0x55ac18b03600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:30.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.457+0000 7ff5f48a17c0 1 --2- 172.21.15.16:0/333497519 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac18a69000 0x55ac18b03600 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x55ac18bc8b40 tx=0x55ac18bceea0 comp rx=0 tx=0).stop 2023-05-20T01:18:30.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.457+0000 7ff5f48a17c0 1 -- 172.21.15.16:0/333497519 shutdown_connections 2023-05-20T01:18:30.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.457+0000 7ff5f48a17c0 1 --2- 172.21.15.16:0/333497519 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac18a69000 0x55ac18b03600 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:30.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.457+0000 7ff5f48a17c0 1 --2- 172.21.15.16:0/333497519 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ac18a69400 0x55ac18b02b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:30.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.457+0000 7ff5f48a17c0 1 --2- 172.21.15.16:0/333497519 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ac18a69c00 0x55ac18b03b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:30.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.457+0000 7ff5f48a17c0 1 -- 172.21.15.16:0/333497519 >> 172.21.15.16:0/333497519 conn(0x55ac18a68400 msgr2=0x55ac18b46000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:18:30.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.457+0000 7ff5f48a17c0 1 -- 172.21.15.16:0/333497519 shutdown_connections 2023-05-20T01:18:30.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.457+0000 7ff5f48a17c0 1 -- 172.21.15.16:0/333497519 wait complete. 2023-05-20T01:18:30.510 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_links'] in time 0:00:00.155888, result: 2023-05-20T01:18:30.510 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2023-05-20T01:18:30.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.665+0000 7f2c901e1700 1 Processor -- start 2023-05-20T01:18:30.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.665+0000 7f2c901e1700 1 -- start start 2023-05-20T01:18:30.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.666+0000 7f2c901e1700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2c8806d4b0 0x7f2c880cf890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:30.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.666+0000 7f2c901e1700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c880cc510 con 0x7f2c8806f450 2023-05-20T01:18:30.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.666+0000 7f2c901e1700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c88071ed0 con 0x7f2c8806a510 2023-05-20T01:18:30.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.666+0000 7f2c901e1700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c880721b0 con 0x7f2c8806d4b0 2023-05-20T01:18:30.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.666+0000 7f2c8df7d700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f2c8806f450 0x7f2c88161080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:44348/0 (socket says 172.21.15.16:44348) 2023-05-20T01:18:30.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.666+0000 7f2c8df7d700 1 -- 172.21.15.16:0/3738188686 learned_addr learned my addr 172.21.15.16:0/3738188686 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:18:30.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.666+0000 7f2c8d77c700 1 --2- 172.21.15.16:0/3738188686 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2c8806d4b0 0x7f2c880cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:30.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.667+0000 7f2c8cf7b700 1 -- 172.21.15.16:0/3738188686 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1048372988 0 0) 0x7f2c880cc510 con 0x7f2c8806f450 2023-05-20T01:18:30.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.667+0000 7f2c8cf7b700 1 -- 172.21.15.16:0/3738188686 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c740035b0 con 0x7f2c8806f450 2023-05-20T01:18:30.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.667+0000 7f2c8cf7b700 1 -- 172.21.15.16:0/3738188686 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3445871602 0 0) 0x7f2c88071ed0 con 0x7f2c8806a510 2023-05-20T01:18:30.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.667+0000 7f2c8cf7b700 1 -- 172.21.15.16:0/3738188686 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c880cc510 con 0x7f2c8806a510 2023-05-20T01:18:30.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.667+0000 7f2c8cf7b700 1 -- 172.21.15.16:0/3738188686 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 99055931 0 0) 0x7f2c740035b0 con 0x7f2c8806f450 2023-05-20T01:18:30.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.667+0000 7f2c8cf7b700 1 -- 172.21.15.16:0/3738188686 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2c8806d4b0 msgr2=0x7f2c880cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:18:30.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.667+0000 7f2c8cf7b700 1 --2- 172.21.15.16:0/3738188686 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2c8806d4b0 0x7f2c880cf890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:30.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.667+0000 7f2c8cf7b700 1 -- 172.21.15.16:0/3738188686 >> v1:172.21.15.132:6789/0 conn(0x7f2c8806a510 legacy=0x7f2c880cc9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:30.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.667+0000 7f2c8cf7b700 1 -- 172.21.15.16:0/3738188686 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c880cc1a0 con 0x7f2c8806f450 2023-05-20T01:18:30.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.667+0000 7f2c8cf7b700 1 -- 172.21.15.16:0/3738188686 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7f2c78001670 con 0x7f2c8806f450 2023-05-20T01:18:30.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.667+0000 7f2c8d77c700 1 --2- 172.21.15.16:0/3738188686 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2c8806d4b0 0x7f2c880cf890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-05-20T01:18:30.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.667+0000 7f2c8cf7b700 1 -- 172.21.15.16:0/3738188686 >> v1:172.21.15.16:6789/0 conn(0x7f2c8806f450 legacy=0x7f2c88161080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:30.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.667+0000 7f2c8cf7b700 1 --2- 172.21.15.16:0/3738188686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c74003c30 0x7f2c74004020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:30.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.667+0000 7f2c8cf7b700 1 --2- 172.21.15.16:0/3738188686 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2c8806d4b0 0x7f2c74004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:30.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.667+0000 7f2c8cf7b700 1 --2- 172.21.15.16:0/3738188686 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2c74004d90 0x7f2c74005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:30.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.667+0000 7f2c8cf7b700 1 -- 172.21.15.16:0/3738188686 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c880721b0 con 0x7f2c74003c30 2023-05-20T01:18:30.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.667+0000 7f2c8cf7b700 1 -- 172.21.15.16:0/3738188686 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c7400bb20 con 0x7f2c74004d90 2023-05-20T01:18:30.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.667+0000 7f2c8cf7b700 1 -- 172.21.15.16:0/3738188686 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c7400bce0 con 0x7f2c8806d4b0 2023-05-20T01:18:30.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.667+0000 7f2c8df7d700 1 --2- 172.21.15.16:0/3738188686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c74003c30 0x7f2c74004020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:30.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.667+0000 7f2c8d77c700 1 --2- 172.21.15.16:0/3738188686 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2c8806d4b0 0x7f2c74004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:30.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.667+0000 7f2c8e77e700 1 --2- 172.21.15.16:0/3738188686 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2c74004d90 0x7f2c74005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:30.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.667+0000 7f2c8df7d700 1 -- 172.21.15.16:0/3738188686 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2c8806d4b0 msgr2=0x7f2c74004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:18:30.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.667+0000 7f2c8df7d700 1 --2- 172.21.15.16:0/3738188686 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2c8806d4b0 0x7f2c74004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:30.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.667+0000 7f2c8df7d700 1 -- 172.21.15.16:0/3738188686 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2c74004d90 msgr2=0x7f2c74005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:18:30.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.667+0000 7f2c8df7d700 1 --2- 172.21.15.16:0/3738188686 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2c74004d90 0x7f2c74005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:30.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.667+0000 7f2c8df7d700 1 -- 172.21.15.16:0/3738188686 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c840094a0 con 0x7f2c74003c30 2023-05-20T01:18:30.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.667+0000 7f2c8d77c700 1 --2- 172.21.15.16:0/3738188686 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2c8806d4b0 0x7f2c74004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-05-20T01:18:30.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.668+0000 7f2c8df7d700 1 --2- 172.21.15.16:0/3738188686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c74003c30 0x7f2c74004020 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f2c7800c210 tx=0x7f2c74005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=895e7133093c2a31 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:30.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.668+0000 7f2c8cf7b700 1 -- 172.21.15.16:0/3738188686 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c7800cd50 con 0x7f2c74003c30 2023-05-20T01:18:30.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.668+0000 7f2c8cf7b700 1 -- 172.21.15.16:0/3738188686 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c78005960 con 0x7f2c74003c30 2023-05-20T01:18:30.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.668+0000 7f2c8cf7b700 1 -- 172.21.15.16:0/3738188686 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c78005d10 con 0x7f2c74003c30 2023-05-20T01:18:30.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.668+0000 7f2c901e1700 1 -- 172.21.15.16:0/3738188686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c74003c30 msgr2=0x7f2c74004020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:30.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.668+0000 7f2c901e1700 1 --2- 172.21.15.16:0/3738188686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c74003c30 0x7f2c74004020 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f2c7800c210 tx=0x7f2c74005880 comp rx=0 tx=0).stop 2023-05-20T01:18:30.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.668+0000 7f2c8e77e700 1 -- 172.21.15.16:0/3738188686 reap_dead start 2023-05-20T01:18:30.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.668+0000 7f2c901e1700 1 -- 172.21.15.16:0/3738188686 shutdown_connections 2023-05-20T01:18:30.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.668+0000 7f2c901e1700 1 -- 172.21.15.16:0/3738188686 >> 172.21.15.16:0/3738188686 conn(0x7f2c880cffb0 msgr2=0x7f2c880d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:18:30.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.668+0000 7f2c901e1700 1 -- 172.21.15.16:0/3738188686 shutdown_connections 2023-05-20T01:18:30.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.668+0000 7f2c901e1700 1 -- 172.21.15.16:0/3738188686 wait complete. 2023-05-20T01:18:30.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.670+0000 7f2c901e1700 1 Processor -- start 2023-05-20T01:18:30.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.670+0000 7f2c901e1700 1 -- start start 2023-05-20T01:18:30.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.670+0000 7f2c901e1700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2c74003c30 0x7f2c880c66d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:30.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.670+0000 7f2c901e1700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2c88149400 0x7f2c880b1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:30.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.670+0000 7f2c901e1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c880b24e0 0x7f2c88147fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:30.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.670+0000 7f2c901e1700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c880c6c10 con 0x7f2c880b24e0 2023-05-20T01:18:30.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.670+0000 7f2c901e1700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c88146090 con 0x7f2c74003c30 2023-05-20T01:18:30.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.670+0000 7f2c901e1700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c88146370 con 0x7f2c88149400 2023-05-20T01:18:30.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.670+0000 7f2c8e77e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c880b24e0 0x7f2c88147fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:30.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.670+0000 7f2c8e77e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c880b24e0 0x7f2c88147fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32836/0 (socket says 172.21.15.16:32836) 2023-05-20T01:18:30.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.670+0000 7f2c8e77e700 1 -- 172.21.15.16:0/2527111660 learned_addr learned my addr 172.21.15.16:0/2527111660 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:18:30.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.671+0000 7f2c8df7d700 1 --2- 172.21.15.16:0/2527111660 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2c74003c30 0x7f2c880c66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:30.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.671+0000 7f2c8d77c700 1 --2- 172.21.15.16:0/2527111660 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2c88149400 0x7f2c880b1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:30.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.671+0000 7f2c8e77e700 1 -- 172.21.15.16:0/2527111660 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2c88149400 msgr2=0x7f2c880b1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:30.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.671+0000 7f2c8e77e700 1 --2- 172.21.15.16:0/2527111660 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2c88149400 0x7f2c880b1fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:30.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.671+0000 7f2c8e77e700 1 -- 172.21.15.16:0/2527111660 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2c74003c30 msgr2=0x7f2c880c66d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:30.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.671+0000 7f2c8e77e700 1 --2- 172.21.15.16:0/2527111660 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2c74003c30 0x7f2c880c66d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:30.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.671+0000 7f2c8e77e700 1 -- 172.21.15.16:0/2527111660 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c880cf890 con 0x7f2c880b24e0 2023-05-20T01:18:30.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.671+0000 7f2c8e77e700 1 --2- 172.21.15.16:0/2527111660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c880b24e0 0x7f2c88147fa0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f2c880d0ff0 tx=0x7f2c8000d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:30.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.671+0000 7f2c7e7fc700 1 -- 172.21.15.16:0/2527111660 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c80003c10 con 0x7f2c880b24e0 2023-05-20T01:18:30.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.671+0000 7f2c7e7fc700 1 -- 172.21.15.16:0/2527111660 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c80003d70 con 0x7f2c880b24e0 2023-05-20T01:18:30.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.671+0000 7f2c901e1700 1 -- 172.21.15.16:0/2527111660 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c880cbe50 con 0x7f2c880b24e0 2023-05-20T01:18:30.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.671+0000 7f2c7e7fc700 1 -- 172.21.15.16:0/2527111660 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c8000dbd0 con 0x7f2c880b24e0 2023-05-20T01:18:30.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.671+0000 7f2c901e1700 1 -- 172.21.15.16:0/2527111660 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c880cfa20 con 0x7f2c880b24e0 2023-05-20T01:18:30.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.672+0000 7f2c901e1700 1 -- 172.21.15.16:0/2527111660 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c880747b0 con 0x7f2c880b24e0 2023-05-20T01:18:30.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.672+0000 7f2c7e7fc700 1 -- 172.21.15.16:0/2527111660 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2c8000dd30 con 0x7f2c880b24e0 2023-05-20T01:18:30.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.673+0000 7f2c7e7fc700 1 --2- 172.21.15.16:0/2527111660 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f2c6c06ace0 0x7f2c6c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:30.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.673+0000 7f2c8df7d700 1 --2- 172.21.15.16:0/2527111660 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f2c6c06ace0 0x7f2c6c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:30.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.673+0000 7f2c7e7fc700 1 -- 172.21.15.16:0/2527111660 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 7016+0+0 (secure 0 0 0) 0x7f2c800898e0 con 0x7f2c880b24e0 2023-05-20T01:18:30.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.673+0000 7f2c8df7d700 1 --2- 172.21.15.16:0/2527111660 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f2c6c06ace0 0x7f2c6c06d190 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f2c780018b0 tx=0x7f2c78007f90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:30.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.677+0000 7f2c7e7fc700 1 -- 172.21.15.16:0/2527111660 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f2c80054660 con 0x7f2c880b24e0 2023-05-20T01:18:30.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:30.958+0000 7f2c901e1700 1 -- 172.21.15.16:0/2527111660 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) v1 -- 0x7f2c88071c10 con 0x7f2c880b24e0 2023-05-20T01:18:31.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:31.479+0000 7f2c7e7fc700 1 -- 172.21.15.16:0/2527111660 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 2:0 v23) v1 ==== 101+0+0 (secure 0 0 0) 0x7f2c80057c80 con 0x7f2c880b24e0 2023-05-20T01:18:31.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:31.483+0000 7f2c901e1700 1 -- 172.21.15.16:0/2527111660 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f2c6c06ace0 msgr2=0x7f2c6c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:31.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:31.483+0000 7f2c901e1700 1 --2- 172.21.15.16:0/2527111660 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f2c6c06ace0 0x7f2c6c06d190 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f2c780018b0 tx=0x7f2c78007f90 comp rx=0 tx=0).stop 2023-05-20T01:18:31.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:31.483+0000 7f2c901e1700 1 -- 172.21.15.16:0/2527111660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c880b24e0 msgr2=0x7f2c88147fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:31.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:31.483+0000 7f2c901e1700 1 --2- 172.21.15.16:0/2527111660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c880b24e0 0x7f2c88147fa0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f2c880d0ff0 tx=0x7f2c8000d430 comp rx=0 tx=0).stop 2023-05-20T01:18:31.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:31.483+0000 7f2c901e1700 1 -- 172.21.15.16:0/2527111660 shutdown_connections 2023-05-20T01:18:31.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:31.483+0000 7f2c901e1700 1 --2- 172.21.15.16:0/2527111660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c880b24e0 0x7f2c88147fa0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:31.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:31.483+0000 7f2c901e1700 1 --2- 172.21.15.16:0/2527111660 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2c74003c30 0x7f2c880c66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:31.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:31.483+0000 7f2c901e1700 1 --2- 172.21.15.16:0/2527111660 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f2c6c06ace0 0x7f2c6c06d190 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:31.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:31.483+0000 7f2c901e1700 1 --2- 172.21.15.16:0/2527111660 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2c88149400 0x7f2c880b1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:31.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:31.483+0000 7f2c901e1700 1 -- 172.21.15.16:0/2527111660 >> 172.21.15.16:0/2527111660 conn(0x7f2c880cffb0 msgr2=0x7f2c8806d880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:18:31.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:31.483+0000 7f2c901e1700 1 -- 172.21.15.16:0/2527111660 shutdown_connections 2023-05-20T01:18:31.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:31.483+0000 7f2c901e1700 1 -- 172.21.15.16:0/2527111660 wait complete. 2023-05-20T01:18:31.489 INFO:teuthology.orchestra.run.smithi016.stderr:repaired: restoring rank 2:0 2023-05-20T01:18:31.489 INFO:tasks.ceph.mds.c.smithi016.stderr:2023-05-20T01:18:31.486+0000 7fc9c316b700 -1 mds.0.openfiles _load_finish got (2) No such file or directory 2023-05-20T01:18:31.504 INFO:tasks.ceph.mds.a:Restarting daemon 2023-05-20T01:18:31.504 INFO:tasks.ceph.mds.a:Stopping old one... 2023-05-20T01:18:31.504 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-05-20T01:18:31.505 INFO:teuthology.orchestra.run:waiting for 300 2023-05-20T01:18:31.505 INFO:tasks.ceph.mds.c:Restarting daemon 2023-05-20T01:18:31.505 INFO:tasks.ceph.mds.c:Stopping old one... 2023-05-20T01:18:31.505 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-05-20T01:18:31.505 INFO:teuthology.orchestra.run:waiting for 300 2023-05-20T01:18:31.506 INFO:tasks.ceph.mds.b:Restarting daemon 2023-05-20T01:18:31.506 INFO:tasks.ceph.mds.b:Stopping old one... 2023-05-20T01:18:31.506 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-05-20T01:18:31.506 INFO:teuthology.orchestra.run:waiting for 300 2023-05-20T01:18:31.507 INFO:tasks.ceph.mds.d:Restarting daemon 2023-05-20T01:18:31.507 INFO:tasks.ceph.mds.d:Stopping old one... 2023-05-20T01:18:31.507 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-05-20T01:18:31.507 INFO:teuthology.orchestra.run:waiting for 300 2023-05-20T01:18:31.517 INFO:tasks.ceph.mds.a:Stopped 2023-05-20T01:18:31.518 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-05-20T01:18:31.518 INFO:tasks.ceph.mds.b:Stopped 2023-05-20T01:18:31.519 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-05-20T01:18:31.519 INFO:tasks.ceph.mds.c:Stopped 2023-05-20T01:18:31.520 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-05-20T01:18:31.520 INFO:tasks.ceph.mds.d:Stopped 2023-05-20T01:18:31.520 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-05-20T01:18:31.560 INFO:tasks.ceph.mds.b:Started 2023-05-20T01:18:31.561 DEBUG:teuthology.parallel:result is None 2023-05-20T01:18:31.562 INFO:tasks.ceph.mds.a:Started 2023-05-20T01:18:31.562 INFO:tasks.ceph.mds.d:Started 2023-05-20T01:18:31.562 INFO:tasks.ceph.mds.c:Started 2023-05-20T01:18:31.563 DEBUG:teuthology.parallel:result is None 2023-05-20T01:18:31.563 DEBUG:teuthology.parallel:result is None 2023-05-20T01:18:31.563 DEBUG:teuthology.parallel:result is None 2023-05-20T01:18:31.564 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-20T01:18:31.645 INFO:tasks.ceph.mds.d.smithi132.stdout:starting mds.d at 2023-05-20T01:18:31.645 INFO:tasks.ceph.mds.b.smithi132.stdout:starting mds.b at 2023-05-20T01:18:31.650 INFO:tasks.ceph.mds.a.smithi016.stdout:starting mds.a at 2023-05-20T01:18:31.650 INFO:tasks.ceph.mds.c.smithi016.stdout:starting mds.c at 2023-05-20T01:18:31.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:31.703+0000 7f898a8af700 1 Processor -- start 2023-05-20T01:18:31.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:31.703+0000 7f898a8af700 1 -- start start 2023-05-20T01:18:31.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:31.704+0000 7f898a8af700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f89840b2760 0x7f89840d1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:31.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:31.704+0000 7f898a8af700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89840cc500 con 0x7f8984144020 2023-05-20T01:18:31.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:31.704+0000 7f898a8af700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89840cb8f0 con 0x7f89840cec60 2023-05-20T01:18:31.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:31.704+0000 7f898a8af700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89840cb6e0 con 0x7f89840b2760 2023-05-20T01:18:31.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:31.704+0000 7f89837fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f8984144020 0x7f89840b0980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:44406/0 (socket says 172.21.15.16:44406) 2023-05-20T01:18:31.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:31.704+0000 7f89837fe700 1 -- 172.21.15.16:0/1668416375 learned_addr learned my addr 172.21.15.16:0/1668416375 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:18:31.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:31.705+0000 7f8983fff700 1 --2- 172.21.15.16:0/1668416375 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f89840b2760 0x7f89840d1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:31.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:31.705+0000 7f8982ffd700 1 -- 172.21.15.16:0/1668416375 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2277469691 0 0) 0x7f89840cc500 con 0x7f8984144020 2023-05-20T01:18:31.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:31.705+0000 7f8982ffd700 1 -- 172.21.15.16:0/1668416375 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f89700035b0 con 0x7f8984144020 2023-05-20T01:18:31.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:31.705+0000 7f8982ffd700 1 -- 172.21.15.16:0/1668416375 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2290584096 0 0) 0x7f89840cb8f0 con 0x7f89840cec60 2023-05-20T01:18:31.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:31.705+0000 7f8982ffd700 1 -- 172.21.15.16:0/1668416375 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f89840cc500 con 0x7f89840cec60 2023-05-20T01:18:31.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:31.705+0000 7f8982ffd700 1 -- 172.21.15.16:0/1668416375 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3076733665 0 0) 0x7f89700035b0 con 0x7f8984144020 2023-05-20T01:18:31.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:31.705+0000 7f8982ffd700 1 -- 172.21.15.16:0/1668416375 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f89840b2760 msgr2=0x7f89840d1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:18:31.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:31.705+0000 7f8982ffd700 1 --2- 172.21.15.16:0/1668416375 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f89840b2760 0x7f89840d1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:31.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:31.705+0000 7f8982ffd700 1 -- 172.21.15.16:0/1668416375 >> v1:172.21.15.132:6789/0 conn(0x7f89840cec60 legacy=0x7f89840cc990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:31.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:31.705+0000 7f8982ffd700 1 -- 172.21.15.16:0/1668416375 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89840cbbd0 con 0x7f8984144020 2023-05-20T01:18:31.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:31.705+0000 7f8982ffd700 1 -- 172.21.15.16:0/1668416375 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7f896c001670 con 0x7f8984144020 2023-05-20T01:18:31.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:31.705+0000 7f8983fff700 1 --2- 172.21.15.16:0/1668416375 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f89840b2760 0x7f89840d1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-05-20T01:18:31.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:31.705+0000 7f8982ffd700 1 -- 172.21.15.16:0/1668416375 >> v1:172.21.15.16:6789/0 conn(0x7f8984144020 legacy=0x7f89840b0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:31.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:31.705+0000 7f8982ffd700 1 --2- 172.21.15.16:0/1668416375 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f89840b2760 0x7f8970003c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:31.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:31.705+0000 7f8982ffd700 1 --2- 172.21.15.16:0/1668416375 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8970004170 0x7f8970004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:31.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:31.705+0000 7f8982ffd700 1 --2- 172.21.15.16:0/1668416375 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8970004d90 0x7f8970005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:31.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:31.705+0000 7f8982ffd700 1 -- 172.21.15.16:0/1668416375 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89840cb6e0 con 0x7f8970004d90 2023-05-20T01:18:31.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:31.705+0000 7f8982ffd700 1 -- 172.21.15.16:0/1668416375 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f897000bb20 con 0x7f8970004170 2023-05-20T01:18:31.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:31.705+0000 7f8982ffd700 1 -- 172.21.15.16:0/1668416375 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f897000bce0 con 0x7f89840b2760 2023-05-20T01:18:31.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:31.705+0000 7f8983fff700 1 --2- 172.21.15.16:0/1668416375 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f89840b2760 0x7f8970003c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:31.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:31.705+0000 7f8988e4c700 1 --2- 172.21.15.16:0/1668416375 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8970004d90 0x7f8970005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:31.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:31.705+0000 7f89837fe700 1 --2- 172.21.15.16:0/1668416375 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8970004170 0x7f8970004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:31.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:31.705+0000 7f8988e4c700 1 -- 172.21.15.16:0/1668416375 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f89840b2760 msgr2=0x7f8970003c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:18:31.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:31.705+0000 7f8988e4c700 1 --2- 172.21.15.16:0/1668416375 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f89840b2760 0x7f8970003c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:31.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:31.706+0000 7f8988e4c700 1 -- 172.21.15.16:0/1668416375 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8970004170 msgr2=0x7f8970004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:18:31.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:31.706+0000 7f8988e4c700 1 --2- 172.21.15.16:0/1668416375 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8970004170 0x7f8970004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:31.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:31.706+0000 7f8988e4c700 1 -- 172.21.15.16:0/1668416375 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89740094a0 con 0x7f8970004d90 2023-05-20T01:18:31.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:31.706+0000 7f8988e4c700 1 --2- 172.21.15.16:0/1668416375 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8970004d90 0x7f8970005200 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f897c00c2f0 tx=0x7f8970005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd8c9088389a7094 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:31.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:31.706+0000 7f8982ffd700 1 -- 172.21.15.16:0/1668416375 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f897c001f30 con 0x7f8970004d90 2023-05-20T01:18:31.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:31.706+0000 7f8982ffd700 1 -- 172.21.15.16:0/1668416375 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f897c00ce90 con 0x7f8970004d90 2023-05-20T01:18:31.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:31.706+0000 7f8982ffd700 1 -- 172.21.15.16:0/1668416375 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f897c003c40 con 0x7f8970004d90 2023-05-20T01:18:31.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:31.706+0000 7f898a8af700 1 -- 172.21.15.16:0/1668416375 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8970004d90 msgr2=0x7f8970005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:31.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:31.706+0000 7f898a8af700 1 --2- 172.21.15.16:0/1668416375 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8970004d90 0x7f8970005200 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f897c00c2f0 tx=0x7f8970005880 comp rx=0 tx=0).stop 2023-05-20T01:18:31.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:31.706+0000 7f8988e4c700 1 -- 172.21.15.16:0/1668416375 reap_dead start 2023-05-20T01:18:31.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:31.706+0000 7f898a8af700 1 -- 172.21.15.16:0/1668416375 shutdown_connections 2023-05-20T01:18:31.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:31.706+0000 7f898a8af700 1 -- 172.21.15.16:0/1668416375 >> 172.21.15.16:0/1668416375 conn(0x7f898406ae30 msgr2=0x7f898409a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:18:31.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:31.706+0000 7f898a8af700 1 -- 172.21.15.16:0/1668416375 shutdown_connections 2023-05-20T01:18:31.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:31.706+0000 7f898a8af700 1 -- 172.21.15.16:0/1668416375 wait complete. 2023-05-20T01:18:31.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:31.708+0000 7f898a8af700 1 Processor -- start 2023-05-20T01:18:31.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:31.708+0000 7f898a8af700 1 -- start start 2023-05-20T01:18:31.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:31.709+0000 7f898a8af700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f89841f2710 0x7f89840b4f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:31.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:31.709+0000 7f898a8af700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f89840b5ab0 0x7f89840b5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:31.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:31.709+0000 7f898a8af700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8984153d40 0x7f89840b6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:31.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:31.709+0000 7f898a8af700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8984159c30 con 0x7f8984153d40 2023-05-20T01:18:31.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:31.709+0000 7f898a8af700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f898406b620 con 0x7f89841f2710 2023-05-20T01:18:31.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:31.709+0000 7f898a8af700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89840c6ff0 con 0x7f89840b5ab0 2023-05-20T01:18:31.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:31.709+0000 7f8988e4c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8984153d40 0x7f89840b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:31.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:31.709+0000 7f8988e4c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8984153d40 0x7f89840b6130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32892/0 (socket says 172.21.15.16:32892) 2023-05-20T01:18:31.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:31.709+0000 7f8988e4c700 1 -- 172.21.15.16:0/163495957 learned_addr learned my addr 172.21.15.16:0/163495957 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:18:31.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:31.709+0000 7f89837fe700 1 --2- 172.21.15.16:0/163495957 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f89840b5ab0 0x7f89840b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:31.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:31.709+0000 7f8983fff700 1 --2- 172.21.15.16:0/163495957 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f89841f2710 0x7f89840b4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:31.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:31.709+0000 7f8988e4c700 1 -- 172.21.15.16:0/163495957 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f89840b5ab0 msgr2=0x7f89840b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:31.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:31.709+0000 7f8988e4c700 1 --2- 172.21.15.16:0/163495957 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f89840b5ab0 0x7f89840b5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:31.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:31.709+0000 7f8988e4c700 1 -- 172.21.15.16:0/163495957 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f89841f2710 msgr2=0x7f89840b4f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:31.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:31.709+0000 7f8988e4c700 1 --2- 172.21.15.16:0/163495957 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f89841f2710 0x7f89840b4f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:31.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:31.709+0000 7f8988e4c700 1 -- 172.21.15.16:0/163495957 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89840cc190 con 0x7f8984153d40 2023-05-20T01:18:31.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:31.709+0000 7f8988e4c700 1 --2- 172.21.15.16:0/163495957 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8984153d40 0x7f89840b6130 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f897c015040 tx=0x7f897c001650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:31.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:31.709+0000 7f8980ff9700 1 -- 172.21.15.16:0/163495957 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f897c0050d0 con 0x7f8984153d40 2023-05-20T01:18:31.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:31.710+0000 7f8980ff9700 1 -- 172.21.15.16:0/163495957 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f897c005250 con 0x7f8984153d40 2023-05-20T01:18:31.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:31.710+0000 7f8980ff9700 1 -- 172.21.15.16:0/163495957 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f897c005600 con 0x7f8984153d40 2023-05-20T01:18:31.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:31.710+0000 7f898a8af700 1 -- 172.21.15.16:0/163495957 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89840cbeb0 con 0x7f8984153d40 2023-05-20T01:18:31.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:31.710+0000 7f898a8af700 1 -- 172.21.15.16:0/163495957 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89840d1680 con 0x7f8984153d40 2023-05-20T01:18:31.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:31.710+0000 7f898a8af700 1 -- 172.21.15.16:0/163495957 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f898415a590 con 0x7f8984153d40 2023-05-20T01:18:31.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:31.711+0000 7f8980ff9700 1 -- 172.21.15.16:0/163495957 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f897c019040 con 0x7f8984153d40 2023-05-20T01:18:31.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:31.711+0000 7f8980ff9700 1 --2- 172.21.15.16:0/163495957 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f896406ad10 0x7f896406d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:31.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:31.711+0000 7f8980ff9700 1 -- 172.21.15.16:0/163495957 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 7102+0+0 (secure 0 0 0) 0x7f897c08a730 con 0x7f8984153d40 2023-05-20T01:18:31.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:31.711+0000 7f8983fff700 1 --2- 172.21.15.16:0/163495957 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f896406ad10 0x7f896406d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:31.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:31.712+0000 7f8983fff700 1 --2- 172.21.15.16:0/163495957 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f896406ad10 0x7f896406d1c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f8974009de0 tx=0x7f8974009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:31.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:31.716+0000 7f8980ff9700 1 -- 172.21.15.16:0/163495957 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f897c055580 con 0x7f8984153d40 2023-05-20T01:18:31.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:31.962+0000 7f898a8af700 1 -- 172.21.15.16:0/163495957 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f89840747b0 con 0x7f8984153d40 2023-05-20T01:18:31.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:31.963+0000 7f8980ff9700 1 -- 172.21.15.16:0/163495957 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4680 (secure 0 0 0) 0x7f897c0011c0 con 0x7f8984153d40 2023-05-20T01:18:31.967 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:18:31.967 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":24,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5176,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/4144439890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4144439890},{"type":"v1","addr":"172.21.15.16:6836","nonce":4144439890}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5188,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/2992755084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":2992755084},{"type":"v1","addr":"172.21.15.132:6834","nonce":2992755084}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/1979284857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1979284857},{"type":"v1","addr":"172.21.15.132:6835","nonce":1979284857}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-20T01:18:26.227827+0000","modified":"2023-05-20T01:18:31.480926+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5730},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5730":{"gid":5730,"name":"c","rank":0,"incarnation":24,"state":"up:replay","state_seq":1,"addr":"172.21.15.16:6837/1867776973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":1867776973},{"type":"v1","addr":"172.21.15.16:6837","nonce":1867776973}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7,8],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-05-20T01:18:31.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:31.965+0000 7f898a8af700 1 -- 172.21.15.16:0/163495957 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f896406ad10 msgr2=0x7f896406d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:31.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:31.965+0000 7f898a8af700 1 --2- 172.21.15.16:0/163495957 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f896406ad10 0x7f896406d1c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f8974009de0 tx=0x7f8974009210 comp rx=0 tx=0).stop 2023-05-20T01:18:31.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:31.965+0000 7f898a8af700 1 -- 172.21.15.16:0/163495957 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8984153d40 msgr2=0x7f89840b6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:31.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:31.965+0000 7f898a8af700 1 --2- 172.21.15.16:0/163495957 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8984153d40 0x7f89840b6130 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f897c015040 tx=0x7f897c001650 comp rx=0 tx=0).stop 2023-05-20T01:18:31.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:31.965+0000 7f898a8af700 1 -- 172.21.15.16:0/163495957 shutdown_connections 2023-05-20T01:18:31.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:31.965+0000 7f898a8af700 1 --2- 172.21.15.16:0/163495957 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8984153d40 0x7f89840b6130 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:31.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:31.965+0000 7f898a8af700 1 --2- 172.21.15.16:0/163495957 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f89841f2710 0x7f89840b4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:31.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:31.965+0000 7f898a8af700 1 --2- 172.21.15.16:0/163495957 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f896406ad10 0x7f896406d1c0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:31.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:31.965+0000 7f898a8af700 1 --2- 172.21.15.16:0/163495957 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f89840b5ab0 0x7f89840b5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:31.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:31.965+0000 7f898a8af700 1 -- 172.21.15.16:0/163495957 >> 172.21.15.16:0/163495957 conn(0x7f898406ae30 msgr2=0x7f89840add40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:18:31.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:31.965+0000 7f898a8af700 1 -- 172.21.15.16:0/163495957 shutdown_connections 2023-05-20T01:18:31.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:31.965+0000 7f898a8af700 1 -- 172.21.15.16:0/163495957 wait complete. 2023-05-20T01:18:31.971 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 24 2023-05-20T01:18:32.383 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-05-20T01:18:26.227827+0000', 'modified': '2023-05-20T01:18:31.480926+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 47, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5730}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5730': {'gid': 5730, 'name': 'c', 'rank': 0, 'incarnation': 24, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.16:6837/1867776973', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6835', 'nonce': 1867776973}, {'type': 'v1', 'addr': '172.21.15.16:6837', 'nonce': 1867776973}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [7, 8], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-05-20T01:18:32.384 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_5730:up:replay 2023-05-20T01:18:32.485 INFO:tasks.ceph.mon.a.smithi016.stderr:2023-05-20T01:18:32.482+0000 7f49eb8ae700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-05-20T01:18:32.497 INFO:tasks.ceph.mds.c.smithi016.stderr:2023-05-20T01:18:32.494+0000 7f44af702700 -1 mds.0.openfiles _load_finish got (2) No such file or directory 2023-05-20T01:18:33.385 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-20T01:18:33.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:33.521+0000 7f44d52c3700 1 Processor -- start 2023-05-20T01:18:33.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:33.521+0000 7f44d52c3700 1 -- start start 2023-05-20T01:18:33.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:33.522+0000 7f44d52c3700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f44d006a510 0x7f44d015e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:33.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:33.522+0000 7f44d52c3700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f44d0071340 0x7f44d00717c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:33.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:33.522+0000 7f44d52c3700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44d015e060 con 0x7f44d006a8e0 2023-05-20T01:18:33.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:33.522+0000 7f44d52c3700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44d015d450 con 0x7f44d006a510 2023-05-20T01:18:33.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:33.522+0000 7f44d52c3700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44d015d730 con 0x7f44d0071340 2023-05-20T01:18:33.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:33.522+0000 7f44cf7fe700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f44d0071340 0x7f44d00717c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:33.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:33.522+0000 7f44ceffd700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f44d006a510 0x7f44d015e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:33.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:33.522+0000 7f44ce7fc700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f44d006a8e0 0x7f44d00cf5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.16:54156/0 (socket says 172.21.15.16:54156) 2023-05-20T01:18:33.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:33.522+0000 7f44ce7fc700 1 -- 172.21.15.16:0/2820620869 learned_addr learned my addr 172.21.15.16:0/2820620869 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:18:33.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:33.523+0000 7f44cf7fe700 1 -- 172.21.15.16:0/2820620869 >> v2:172.21.15.132:3300/0 conn(0x7f44d006a510 msgr2=0x7f44d015e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:18:33.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:33.523+0000 7f44cf7fe700 1 --2- 172.21.15.16:0/2820620869 >> v2:172.21.15.132:3300/0 conn(0x7f44d006a510 0x7f44d015e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:33.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:33.523+0000 7f44cf7fe700 1 -- 172.21.15.16:0/2820620869 >> v1:172.21.15.132:6789/0 conn(0x7f44d006a8e0 legacy=0x7f44d00cf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:33.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:33.523+0000 7f44cf7fe700 1 -- 172.21.15.16:0/2820620869 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44d015da10 con 0x7f44d0071340 2023-05-20T01:18:33.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:33.523+0000 7f44ceffd700 1 --2- 172.21.15.16:0/2820620869 >> v2:172.21.15.132:3300/0 conn(0x7f44d006a510 0x7f44d015e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-05-20T01:18:33.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:33.523+0000 7f44cf7fe700 1 --2- 172.21.15.16:0/2820620869 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f44d0071340 0x7f44d00717c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f44c8009d90 tx=0x7f44c800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d59f0a61224c0ea6 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:33.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:33.523+0000 7f44cdffb700 1 -- 172.21.15.16:0/2820620869 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44c800e070 con 0x7f44d0071340 2023-05-20T01:18:33.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:33.523+0000 7f44cdffb700 1 -- 172.21.15.16:0/2820620869 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44c800bb20 con 0x7f44d0071340 2023-05-20T01:18:33.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:33.523+0000 7f44cdffb700 1 -- 172.21.15.16:0/2820620869 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44c8012430 con 0x7f44d0071340 2023-05-20T01:18:33.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:33.524+0000 7f44d52c3700 1 -- 172.21.15.16:0/2820620869 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f44d0071340 msgr2=0x7f44d00717c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:33.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:33.524+0000 7f44d52c3700 1 --2- 172.21.15.16:0/2820620869 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f44d0071340 0x7f44d00717c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f44c8009d90 tx=0x7f44c800b330 comp rx=0 tx=0).stop 2023-05-20T01:18:33.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:33.524+0000 7f44d52c3700 1 -- 172.21.15.16:0/2820620869 shutdown_connections 2023-05-20T01:18:33.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:33.524+0000 7f44d52c3700 1 --2- 172.21.15.16:0/2820620869 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f44d0071340 0x7f44d00717c0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:33.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:33.524+0000 7f44d52c3700 1 --2- 172.21.15.16:0/2820620869 >> v2:172.21.15.132:3300/0 conn(0x7f44d006a510 0x7f44d015e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:33.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:33.524+0000 7f44d52c3700 1 -- 172.21.15.16:0/2820620869 >> 172.21.15.16:0/2820620869 conn(0x7f44d00d0c30 msgr2=0x7f44d00b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:18:33.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:33.524+0000 7f44d52c3700 1 -- 172.21.15.16:0/2820620869 shutdown_connections 2023-05-20T01:18:33.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:33.524+0000 7f44d52c3700 1 -- 172.21.15.16:0/2820620869 wait complete. 2023-05-20T01:18:33.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:33.525+0000 7f44d52c3700 1 Processor -- start 2023-05-20T01:18:33.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:33.525+0000 7f44d52c3700 1 -- start start 2023-05-20T01:18:33.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:33.526+0000 7f44d52c3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f44d006a510 0x7f44d00c64a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:33.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:33.526+0000 7f44d52c3700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f44d006a8e0 0x7f44d00b5890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:33.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:33.526+0000 7f44d52c3700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f44d0071340 0x7f44d01f60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:33.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:33.526+0000 7f44d52c3700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44d00c69e0 con 0x7f44d006a510 2023-05-20T01:18:33.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:33.526+0000 7f44d52c3700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44d00b5dd0 con 0x7f44d006a8e0 2023-05-20T01:18:33.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:33.526+0000 7f44d52c3700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44d00c9e10 con 0x7f44d0071340 2023-05-20T01:18:33.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:33.526+0000 7f44ceffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f44d006a510 0x7f44d00c64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:33.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:33.526+0000 7f44ceffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f44d006a510 0x7f44d00c64a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32918/0 (socket says 172.21.15.16:32918) 2023-05-20T01:18:33.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:33.526+0000 7f44ceffd700 1 -- 172.21.15.16:0/4197742965 learned_addr learned my addr 172.21.15.16:0/4197742965 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:18:33.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:33.526+0000 7f44ce7fc700 1 --2- 172.21.15.16:0/4197742965 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f44d006a8e0 0x7f44d00b5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:33.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:33.526+0000 7f44cf7fe700 1 --2- 172.21.15.16:0/4197742965 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f44d0071340 0x7f44d01f60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:33.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:33.527+0000 7f44ceffd700 1 -- 172.21.15.16:0/4197742965 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f44d0071340 msgr2=0x7f44d01f60d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:33.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:33.527+0000 7f44ceffd700 1 --2- 172.21.15.16:0/4197742965 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f44d0071340 0x7f44d01f60d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:33.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:33.527+0000 7f44ceffd700 1 -- 172.21.15.16:0/4197742965 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f44d006a8e0 msgr2=0x7f44d00b5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:33.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:33.527+0000 7f44ceffd700 1 --2- 172.21.15.16:0/4197742965 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f44d006a8e0 0x7f44d00b5890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:33.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:33.527+0000 7f44ceffd700 1 -- 172.21.15.16:0/4197742965 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44c40094a0 con 0x7f44d006a510 2023-05-20T01:18:33.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:33.527+0000 7f44ceffd700 1 --2- 172.21.15.16:0/4197742965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f44d006a510 0x7f44d00c64a0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f44c400ebe0 tx=0x7f44c400eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:33.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:33.527+0000 7f44b77fe700 1 -- 172.21.15.16:0/4197742965 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44c4009760 con 0x7f44d006a510 2023-05-20T01:18:33.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:33.527+0000 7f44b77fe700 1 -- 172.21.15.16:0/4197742965 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44c400c7f0 con 0x7f44d006a510 2023-05-20T01:18:33.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:33.527+0000 7f44d52c3700 1 -- 172.21.15.16:0/4197742965 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44c8009a70 con 0x7f44d006a510 2023-05-20T01:18:33.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:33.527+0000 7f44b77fe700 1 -- 172.21.15.16:0/4197742965 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44c400cb80 con 0x7f44d006a510 2023-05-20T01:18:33.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:33.527+0000 7f44d52c3700 1 -- 172.21.15.16:0/4197742965 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44d00cfd00 con 0x7f44d006a510 2023-05-20T01:18:33.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:33.528+0000 7f44d52c3700 1 -- 172.21.15.16:0/4197742965 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44d015dcf0 con 0x7f44d006a510 2023-05-20T01:18:33.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:33.529+0000 7f44b77fe700 1 -- 172.21.15.16:0/4197742965 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f44c400cce0 con 0x7f44d006a510 2023-05-20T01:18:33.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:33.529+0000 7f44b77fe700 1 --2- 172.21.15.16:0/4197742965 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f44bc06ace0 0x7f44bc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:33.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:33.529+0000 7f44b77fe700 1 -- 172.21.15.16:0/4197742965 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 7102+0+0 (secure 0 0 0) 0x7f44c4089820 con 0x7f44d006a510 2023-05-20T01:18:33.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:33.529+0000 7f44ce7fc700 1 --2- 172.21.15.16:0/4197742965 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f44bc06ace0 0x7f44bc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:33.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:33.529+0000 7f44ce7fc700 1 --2- 172.21.15.16:0/4197742965 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f44bc06ace0 0x7f44bc06d190 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f44c0005910 tx=0x7f44c0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:33.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:33.533+0000 7f44b77fe700 1 -- 172.21.15.16:0/4197742965 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f44c4054550 con 0x7f44d006a510 2023-05-20T01:18:33.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:33.776+0000 7f44d52c3700 1 -- 172.21.15.16:0/4197742965 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f44d006ad50 con 0x7f44d006a510 2023-05-20T01:18:33.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:33.776+0000 7f44b77fe700 1 -- 172.21.15.16:0/4197742965 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+4683 (secure 0 0 0) 0x7f44c4057b70 con 0x7f44d006a510 2023-05-20T01:18:33.780 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:18:33.781 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":27,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5246,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/2629811751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":2629811751},{"type":"v1","addr":"172.21.15.132:6834","nonce":2629811751}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":5249,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2117148777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":2117148777},{"type":"v1","addr":"172.21.15.132:6835","nonce":2117148777}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":5566,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2025052254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":2025052254},{"type":"v1","addr":"172.21.15.16:6837","nonce":2025052254}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-20T01:18:26.227827+0000","modified":"2023-05-20T01:18:33.487686+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5889},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5889":{"gid":5889,"name":"c","rank":0,"incarnation":26,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.16:6836/3284962389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3284962389},{"type":"v1","addr":"172.21.15.16:6836","nonce":3284962389}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7,8],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-05-20T01:18:33.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:33.778+0000 7f44d52c3700 1 -- 172.21.15.16:0/4197742965 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f44bc06ace0 msgr2=0x7f44bc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:33.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:33.778+0000 7f44d52c3700 1 --2- 172.21.15.16:0/4197742965 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f44bc06ace0 0x7f44bc06d190 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f44c0005910 tx=0x7f44c0009040 comp rx=0 tx=0).stop 2023-05-20T01:18:33.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:33.778+0000 7f44d52c3700 1 -- 172.21.15.16:0/4197742965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f44d006a510 msgr2=0x7f44d00c64a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:33.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:33.778+0000 7f44d52c3700 1 --2- 172.21.15.16:0/4197742965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f44d006a510 0x7f44d00c64a0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f44c400ebe0 tx=0x7f44c400eef0 comp rx=0 tx=0).stop 2023-05-20T01:18:33.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:33.778+0000 7f44d52c3700 1 -- 172.21.15.16:0/4197742965 shutdown_connections 2023-05-20T01:18:33.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:33.778+0000 7f44d52c3700 1 --2- 172.21.15.16:0/4197742965 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f44bc06ace0 0x7f44bc06d190 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:33.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:33.778+0000 7f44d52c3700 1 --2- 172.21.15.16:0/4197742965 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f44d0071340 0x7f44d01f60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:33.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:33.778+0000 7f44d52c3700 1 --2- 172.21.15.16:0/4197742965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f44d006a510 0x7f44d00c64a0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:33.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:33.778+0000 7f44d52c3700 1 --2- 172.21.15.16:0/4197742965 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f44d006a8e0 0x7f44d00b5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:33.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:33.778+0000 7f44d52c3700 1 -- 172.21.15.16:0/4197742965 >> 172.21.15.16:0/4197742965 conn(0x7f44d00d0c30 msgr2=0x7f44d015f300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:18:33.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:33.779+0000 7f44d52c3700 1 -- 172.21.15.16:0/4197742965 shutdown_connections 2023-05-20T01:18:33.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:33.779+0000 7f44d52c3700 1 -- 172.21.15.16:0/4197742965 wait complete. 2023-05-20T01:18:33.784 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 27 2023-05-20T01:18:34.197 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-05-20T01:18:26.227827+0000', 'modified': '2023-05-20T01:18:33.487686+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 50, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5889}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5889': {'gid': 5889, 'name': 'c', 'rank': 0, 'incarnation': 26, 'state': 'up:reconnect', 'state_seq': 2, 'addr': '172.21.15.16:6836/3284962389', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6834', 'nonce': 3284962389}, {'type': 'v1', 'addr': '172.21.15.16:6836', 'nonce': 3284962389}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [7, 8], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-05-20T01:18:34.197 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_5889:up:reconnect 2023-05-20T01:18:35.199 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-20T01:18:35.317 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:18:35.322 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:18:35.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:35.339+0000 7f2f2dd77700 1 Processor -- start 2023-05-20T01:18:35.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:35.339+0000 7f2f2dd77700 1 -- start start 2023-05-20T01:18:35.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:35.340+0000 7f2f2dd77700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2f2806a510 0x7f2f2815e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:35.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:35.340+0000 7f2f2dd77700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2f28071340 0x7f2f280717c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:35.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:35.340+0000 7f2f2dd77700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f2815e060 con 0x7f2f2806a8e0 2023-05-20T01:18:35.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:35.340+0000 7f2f2dd77700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f2815d450 con 0x7f2f28071340 2023-05-20T01:18:35.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:35.340+0000 7f2f2dd77700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f2815d730 con 0x7f2f2806a510 2023-05-20T01:18:35.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:35.340+0000 7f2f27fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2f28071340 0x7f2f280717c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:35.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:35.340+0000 7f2f27fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2f28071340 0x7f2f280717c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32928/0 (socket says 172.21.15.16:32928) 2023-05-20T01:18:35.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:35.340+0000 7f2f27fff700 1 -- 172.21.15.16:0/3411870732 learned_addr learned my addr 172.21.15.16:0/3411870732 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:18:35.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:35.340+0000 7f2f277fe700 1 --2- 172.21.15.16:0/3411870732 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2f2806a510 0x7f2f2815e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:35.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:35.341+0000 7f2f27fff700 1 -- 172.21.15.16:0/3411870732 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2f2806a510 msgr2=0x7f2f2815e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:18:35.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:35.341+0000 7f2f27fff700 1 --2- 172.21.15.16:0/3411870732 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2f2806a510 0x7f2f2815e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:35.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:35.341+0000 7f2f27fff700 1 -- 172.21.15.16:0/3411870732 >> v1:172.21.15.132:6789/0 conn(0x7f2f2806a8e0 legacy=0x7f2f280cf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:35.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:35.341+0000 7f2f27fff700 1 -- 172.21.15.16:0/3411870732 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f2815da10 con 0x7f2f28071340 2023-05-20T01:18:35.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:35.341+0000 7f2f27fff700 1 --2- 172.21.15.16:0/3411870732 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f28071340 0x7f2f280717c0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f2f20009e50 tx=0x7f2f2000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4eb9ca8bd493f28 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:35.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:35.341+0000 7f2f267fc700 1 -- 172.21.15.16:0/3411870732 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f2000e070 con 0x7f2f28071340 2023-05-20T01:18:35.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:35.341+0000 7f2f267fc700 1 -- 172.21.15.16:0/3411870732 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f2000b910 con 0x7f2f28071340 2023-05-20T01:18:35.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:35.341+0000 7f2f267fc700 1 -- 172.21.15.16:0/3411870732 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f2000bcc0 con 0x7f2f28071340 2023-05-20T01:18:35.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:35.342+0000 7f2f2dd77700 1 -- 172.21.15.16:0/3411870732 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f28071340 msgr2=0x7f2f280717c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:35.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:35.342+0000 7f2f2dd77700 1 --2- 172.21.15.16:0/3411870732 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f28071340 0x7f2f280717c0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f2f20009e50 tx=0x7f2f2000b330 comp rx=0 tx=0).stop 2023-05-20T01:18:35.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:35.342+0000 7f2f2dd77700 1 -- 172.21.15.16:0/3411870732 shutdown_connections 2023-05-20T01:18:35.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:35.342+0000 7f2f2dd77700 1 --2- 172.21.15.16:0/3411870732 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f28071340 0x7f2f280717c0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:35.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:35.342+0000 7f2f2dd77700 1 --2- 172.21.15.16:0/3411870732 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2f2806a510 0x7f2f2815e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:35.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:35.342+0000 7f2f2dd77700 1 -- 172.21.15.16:0/3411870732 >> 172.21.15.16:0/3411870732 conn(0x7f2f280d0c30 msgr2=0x7f2f280b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:18:35.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:35.342+0000 7f2f2dd77700 1 -- 172.21.15.16:0/3411870732 shutdown_connections 2023-05-20T01:18:35.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:35.342+0000 7f2f2dd77700 1 -- 172.21.15.16:0/3411870732 wait complete. 2023-05-20T01:18:35.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:35.343+0000 7f2f2dd77700 1 Processor -- start 2023-05-20T01:18:35.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:35.343+0000 7f2f2dd77700 1 -- start start 2023-05-20T01:18:35.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:35.344+0000 7f2f2dd77700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2f2806a510 0x7f2f2814b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:35.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:35.344+0000 7f2f2dd77700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2f2806a8e0 0x7f2f281f62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:35.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:35.344+0000 7f2f2dd77700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f28071340 0x7f2f280c3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:35.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:35.344+0000 7f2f2dd77700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f281f5eb0 con 0x7f2f28071340 2023-05-20T01:18:35.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:35.344+0000 7f2f2dd77700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f281f5b10 con 0x7f2f2806a8e0 2023-05-20T01:18:35.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:35.344+0000 7f2f2dd77700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f2814bcc0 con 0x7f2f2806a510 2023-05-20T01:18:35.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:35.344+0000 7f2f27fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f28071340 0x7f2f280c3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:35.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:35.344+0000 7f2f27fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f28071340 0x7f2f280c3ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32936/0 (socket says 172.21.15.16:32936) 2023-05-20T01:18:35.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:35.344+0000 7f2f277fe700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2f2806a510 0x7f2f2814b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:35.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:35.344+0000 7f2f27fff700 1 -- 172.21.15.16:0/3643794158 learned_addr learned my addr 172.21.15.16:0/3643794158 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:18:35.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:35.344+0000 7f2f26ffd700 1 --2- 172.21.15.16:0/3643794158 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2f2806a8e0 0x7f2f281f62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:35.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:35.345+0000 7f2f277fe700 1 -- 172.21.15.16:0/3643794158 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2f2806a8e0 msgr2=0x7f2f281f62d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:35.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:35.345+0000 7f2f277fe700 1 --2- 172.21.15.16:0/3643794158 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2f2806a8e0 0x7f2f281f62d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:35.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:35.345+0000 7f2f277fe700 1 -- 172.21.15.16:0/3643794158 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f28071340 msgr2=0x7f2f280c3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:35.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:35.345+0000 7f2f277fe700 1 --2- 172.21.15.16:0/3643794158 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f28071340 0x7f2f280c3ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:35.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:35.345+0000 7f2f277fe700 1 -- 172.21.15.16:0/3643794158 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f20009b30 con 0x7f2f2806a510 2023-05-20T01:18:35.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:35.345+0000 7f2f27fff700 1 --2- 172.21.15.16:0/3643794158 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f28071340 0x7f2f280c3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-05-20T01:18:35.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:35.345+0000 7f2f277fe700 1 --2- 172.21.15.16:0/3643794158 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2f2806a510 0x7f2f2814b470 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2f1800e900 tx=0x7f2f1800ec10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:35.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:35.345+0000 7f2f0ffff700 1 -- 172.21.15.16:0/3643794158 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f18013680 con 0x7f2f2806a510 2023-05-20T01:18:35.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:35.345+0000 7f2f0ffff700 1 -- 172.21.15.16:0/3643794158 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f180137e0 con 0x7f2f2806a510 2023-05-20T01:18:35.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:35.345+0000 7f2f0ffff700 1 -- 172.21.15.16:0/3643794158 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f18013b50 con 0x7f2f2806a510 2023-05-20T01:18:35.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:35.345+0000 7f2f2dd77700 1 -- 172.21.15.16:0/3643794158 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f280cfd00 con 0x7f2f2806a510 2023-05-20T01:18:35.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:35.345+0000 7f2f2dd77700 1 -- 172.21.15.16:0/3643794158 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f2815dcf0 con 0x7f2f2806a510 2023-05-20T01:18:35.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:35.347+0000 7f2f2dd77700 1 -- 172.21.15.16:0/3643794158 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f2806ad50 con 0x7f2f2806a510 2023-05-20T01:18:35.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:35.347+0000 7f2f0ffff700 1 -- 172.21.15.16:0/3643794158 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2f18013cb0 con 0x7f2f2806a510 2023-05-20T01:18:35.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:35.348+0000 7f2f0ffff700 1 --2- 172.21.15.16:0/3643794158 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f2f1406ace0 0x7f2f1406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:35.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:35.348+0000 7f2f0ffff700 1 -- 172.21.15.16:0/3643794158 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 7102+0+0 (secure 0 0 0) 0x7f2f18089b90 con 0x7f2f2806a510 2023-05-20T01:18:35.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:35.348+0000 7f2f26ffd700 1 --2- 172.21.15.16:0/3643794158 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f2f1406ace0 0x7f2f1406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:35.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:35.348+0000 7f2f26ffd700 1 --2- 172.21.15.16:0/3643794158 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f2f1406ace0 0x7f2f1406d190 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f2f280ca080 tx=0x7f2f1c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:35.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:35.353+0000 7f2f0ffff700 1 -- 172.21.15.16:0/3643794158 <== mon.2 v2:172.21.15.132:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f2f18054790 con 0x7f2f2806a510 2023-05-20T01:18:35.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:35.601+0000 7f2f2dd77700 1 -- 172.21.15.16:0/3643794158 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2f280747b0 con 0x7f2f2806a510 2023-05-20T01:18:35.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:35.601+0000 7f2f0ffff700 1 -- 172.21.15.16:0/3643794158 <== mon.2 v2:172.21.15.132:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+4680 (secure 0 0 0) 0x7f2f18057db0 con 0x7f2f2806a510 2023-05-20T01:18:35.605 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:18:35.606 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":29,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5246,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/2629811751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":2629811751},{"type":"v1","addr":"172.21.15.132:6834","nonce":2629811751}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":5249,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2117148777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":2117148777},{"type":"v1","addr":"172.21.15.132:6835","nonce":2117148777}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":5566,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2025052254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":2025052254},{"type":"v1","addr":"172.21.15.16:6837","nonce":2025052254}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-20T01:18:26.227827+0000","modified":"2023-05-20T01:18:35.503777+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5889},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5889":{"gid":5889,"name":"c","rank":0,"incarnation":26,"state":"up:active","state_seq":4,"addr":"172.21.15.16:6836/3284962389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3284962389},{"type":"v1","addr":"172.21.15.16:6836","nonce":3284962389}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7,8],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-05-20T01:18:35.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:35.604+0000 7f2f2dd77700 1 -- 172.21.15.16:0/3643794158 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f2f1406ace0 msgr2=0x7f2f1406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:35.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:35.604+0000 7f2f2dd77700 1 --2- 172.21.15.16:0/3643794158 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f2f1406ace0 0x7f2f1406d190 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f2f280ca080 tx=0x7f2f1c009040 comp rx=0 tx=0).stop 2023-05-20T01:18:35.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:35.604+0000 7f2f2dd77700 1 -- 172.21.15.16:0/3643794158 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2f2806a510 msgr2=0x7f2f2814b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:35.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:35.604+0000 7f2f2dd77700 1 --2- 172.21.15.16:0/3643794158 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2f2806a510 0x7f2f2814b470 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2f1800e900 tx=0x7f2f1800ec10 comp rx=0 tx=0).stop 2023-05-20T01:18:35.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:35.604+0000 7f2f2dd77700 1 -- 172.21.15.16:0/3643794158 shutdown_connections 2023-05-20T01:18:35.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:35.604+0000 7f2f2dd77700 1 --2- 172.21.15.16:0/3643794158 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f28071340 0x7f2f280c3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:35.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:35.604+0000 7f2f2dd77700 1 --2- 172.21.15.16:0/3643794158 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f2f1406ace0 0x7f2f1406d190 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:35.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:35.604+0000 7f2f2dd77700 1 --2- 172.21.15.16:0/3643794158 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2f2806a510 0x7f2f2814b470 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:35.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:35.604+0000 7f2f2dd77700 1 --2- 172.21.15.16:0/3643794158 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2f2806a8e0 0x7f2f281f62d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:35.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:35.604+0000 7f2f2dd77700 1 -- 172.21.15.16:0/3643794158 >> 172.21.15.16:0/3643794158 conn(0x7f2f280d0c30 msgr2=0x7f2f28159500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:18:35.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:35.604+0000 7f2f2dd77700 1 -- 172.21.15.16:0/3643794158 shutdown_connections 2023-05-20T01:18:35.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:35.604+0000 7f2f2dd77700 1 -- 172.21.15.16:0/3643794158 wait complete. 2023-05-20T01:18:35.610 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 29 2023-05-20T01:18:36.023 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 29, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-05-20T01:18:26.227827+0000', 'modified': '2023-05-20T01:18:35.503777+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 50, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5889}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5889': {'gid': 5889, 'name': 'c', 'rank': 0, 'incarnation': 26, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.16:6836/3284962389', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6834', 'nonce': 3284962389}, {'type': 'v1', 'addr': '172.21.15.16:6836', 'nonce': 3284962389}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [7, 8], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-05-20T01:18:36.024 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-05-20T01:18:36.024 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-05-20T01:18:36.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.150+0000 7fd9e8163700 1 Processor -- start 2023-05-20T01:18:36.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.150+0000 7fd9e8163700 1 -- start start 2023-05-20T01:18:36.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.151+0000 7fd9e8163700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd9e01599b0 0x7fd9e015ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:36.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.151+0000 7fd9e8163700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9e00ca1b0 con 0x7fd9e00b0910 2023-05-20T01:18:36.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.151+0000 7fd9e8163700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9e015e300 con 0x7fd9e00cd0f0 2023-05-20T01:18:36.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.151+0000 7fd9e8163700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9e00ced30 con 0x7fd9e01599b0 2023-05-20T01:18:36.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.151+0000 7fd9e5eff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd9e01599b0 0x7fd9e015ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:36.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.151+0000 7fd9e5eff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd9e01599b0 0x7fd9e015ddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32940/0 (socket says 172.21.15.16:32940) 2023-05-20T01:18:36.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.151+0000 7fd9e6700700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fd9e00b0910 0x7fd9e00b0d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:44472/0 (socket says 172.21.15.16:44472) 2023-05-20T01:18:36.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.151+0000 7fd9e5eff700 1 -- 172.21.15.16:0/1651624820 learned_addr learned my addr 172.21.15.16:0/1651624820 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:18:36.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.151+0000 7fd9e5eff700 1 -- 172.21.15.16:0/1651624820 >> v1:172.21.15.132:6789/0 conn(0x7fd9e00cd0f0 legacy=0x7fd9e00ce620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:36.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.151+0000 7fd9e5eff700 1 -- 172.21.15.16:0/1651624820 >> v1:172.21.15.16:6789/0 conn(0x7fd9e00b0910 legacy=0x7fd9e00b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:36.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.151+0000 7fd9e5eff700 1 -- 172.21.15.16:0/1651624820 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9e00cd580 con 0x7fd9e01599b0 2023-05-20T01:18:36.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.151+0000 7fd9e5eff700 1 --2- 172.21.15.16:0/1651624820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9e01599b0 0x7fd9e015ddc0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc00bb90 tx=0x7fd9dc00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9d3c099f2c83c80 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:36.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.152+0000 7fd9e4efd700 1 -- 172.21.15.16:0/1651624820 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9dc00dc90 con 0x7fd9e01599b0 2023-05-20T01:18:36.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.152+0000 7fd9e4efd700 1 -- 172.21.15.16:0/1651624820 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9dc00ddf0 con 0x7fd9e01599b0 2023-05-20T01:18:36.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.152+0000 7fd9e4efd700 1 -- 172.21.15.16:0/1651624820 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9dc004250 con 0x7fd9e01599b0 2023-05-20T01:18:36.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.152+0000 7fd9e8163700 1 -- 172.21.15.16:0/1651624820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9e01599b0 msgr2=0x7fd9e015ddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:36.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.152+0000 7fd9e8163700 1 --2- 172.21.15.16:0/1651624820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9e01599b0 0x7fd9e015ddc0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc00bb90 tx=0x7fd9dc00bea0 comp rx=0 tx=0).stop 2023-05-20T01:18:36.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.152+0000 7fd9e8163700 1 -- 172.21.15.16:0/1651624820 shutdown_connections 2023-05-20T01:18:36.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.152+0000 7fd9e8163700 1 --2- 172.21.15.16:0/1651624820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9e01599b0 0x7fd9e015ddc0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:36.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.152+0000 7fd9e8163700 1 -- 172.21.15.16:0/1651624820 >> 172.21.15.16:0/1651624820 conn(0x7fd9e00cb910 msgr2=0x7fd9e00cbd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:18:36.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.152+0000 7fd9e8163700 1 -- 172.21.15.16:0/1651624820 shutdown_connections 2023-05-20T01:18:36.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.152+0000 7fd9e8163700 1 -- 172.21.15.16:0/1651624820 wait complete. 2023-05-20T01:18:36.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.153+0000 7fd9e8163700 1 Processor -- start 2023-05-20T01:18:36.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.154+0000 7fd9e8163700 1 -- start start 2023-05-20T01:18:36.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.154+0000 7fd9e8163700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd9e00b0910 0x7fd9e01ecb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:36.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.154+0000 7fd9e8163700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9e00cd0f0 0x7fd9e00c8640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:36.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.154+0000 7fd9e8163700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd9e01599b0 0x7fd9e00c7980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:36.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.154+0000 7fd9e8163700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e00adcf0 con 0x7fd9e00cd0f0 2023-05-20T01:18:36.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.154+0000 7fd9e8163700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e00c7ec0 con 0x7fd9e01599b0 2023-05-20T01:18:36.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.154+0000 7fd9e8163700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e00b3cd0 con 0x7fd9e00b0910 2023-05-20T01:18:36.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.154+0000 7fd9e56fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9e00cd0f0 0x7fd9e00c8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:36.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.154+0000 7fd9e56fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9e00cd0f0 0x7fd9e00c8640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32948/0 (socket says 172.21.15.16:32948) 2023-05-20T01:18:36.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.154+0000 7fd9e56fe700 1 -- 172.21.15.16:0/1611048930 learned_addr learned my addr 172.21.15.16:0/1611048930 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:18:36.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.154+0000 7fd9e5eff700 1 --2- 172.21.15.16:0/1611048930 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd9e00b0910 0x7fd9e01ecb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:36.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.155+0000 7fd9e6700700 1 --2- 172.21.15.16:0/1611048930 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd9e01599b0 0x7fd9e00c7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:36.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.155+0000 7fd9e56fe700 1 -- 172.21.15.16:0/1611048930 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd9e00b0910 msgr2=0x7fd9e01ecb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:36.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.155+0000 7fd9e56fe700 1 --2- 172.21.15.16:0/1611048930 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd9e00b0910 0x7fd9e01ecb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:36.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.155+0000 7fd9e56fe700 1 -- 172.21.15.16:0/1611048930 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd9e01599b0 msgr2=0x7fd9e00c7980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:36.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.155+0000 7fd9e56fe700 1 --2- 172.21.15.16:0/1611048930 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd9e01599b0 0x7fd9e00c7980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:36.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.155+0000 7fd9e56fe700 1 -- 172.21.15.16:0/1611048930 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9dc00b7f0 con 0x7fd9e00cd0f0 2023-05-20T01:18:36.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.155+0000 7fd9e6700700 1 --2- 172.21.15.16:0/1611048930 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd9e01599b0 0x7fd9e00c7980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:18:36.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.155+0000 7fd9e56fe700 1 --2- 172.21.15.16:0/1611048930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9e00cd0f0 0x7fd9e00c8640 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fd9d000c6b0 tx=0x7fd9d000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:36.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.155+0000 7fd9d67fc700 1 -- 172.21.15.16:0/1611048930 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d00052d0 con 0x7fd9e00cd0f0 2023-05-20T01:18:36.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.155+0000 7fd9d67fc700 1 -- 172.21.15.16:0/1611048930 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9d0005430 con 0x7fd9e00cd0f0 2023-05-20T01:18:36.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.155+0000 7fd9e8163700 1 -- 172.21.15.16:0/1611048930 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9e00cdb40 con 0x7fd9e00cd0f0 2023-05-20T01:18:36.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.155+0000 7fd9d67fc700 1 -- 172.21.15.16:0/1611048930 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d00038c0 con 0x7fd9e00cd0f0 2023-05-20T01:18:36.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.155+0000 7fd9e8163700 1 -- 172.21.15.16:0/1611048930 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9e00cd860 con 0x7fd9e00cd0f0 2023-05-20T01:18:36.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.156+0000 7fd9d67fc700 1 -- 172.21.15.16:0/1611048930 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd9d0003a20 con 0x7fd9e00cd0f0 2023-05-20T01:18:36.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.157+0000 7fd9d67fc700 1 --2- 172.21.15.16:0/1611048930 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd9c406ace0 0x7fd9c406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:36.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.157+0000 7fd9d67fc700 1 -- 172.21.15.16:0/1611048930 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 7102+0+0 (secure 0 0 0) 0x7fd9d0088ce0 con 0x7fd9e00cd0f0 2023-05-20T01:18:36.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.157+0000 7fd9e5eff700 1 --2- 172.21.15.16:0/1611048930 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd9c406ace0 0x7fd9c406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:36.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.157+0000 7fd9e5eff700 1 --2- 172.21.15.16:0/1611048930 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd9c406ace0 0x7fd9c406d190 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc00b340 tx=0x7fd9dc01b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:36.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.159+0000 7fd9f42efb80 1 Processor -- start 2023-05-20T01:18:36.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.159+0000 7fd9f42efb80 1 -- start start 2023-05-20T01:18:36.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.160+0000 7fd9f42efb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b4d7551400 0x55b4d75517d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:36.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.160+0000 7fd9f42efb80 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55b4d754d700 0x55b4d7526460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:36.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.160+0000 7fd9f42efb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b4d7524870 0x55b4d75269a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:36.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.160+0000 7fd9f42efb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4d742c010 con 0x55b4d7524870 2023-05-20T01:18:36.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.160+0000 7fd9f42efb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4d747e850 con 0x55b4d7551400 2023-05-20T01:18:36.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.160+0000 7fd9f42efb80 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b4d7477ce0 con 0x55b4d754d700 2023-05-20T01:18:36.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.160+0000 7fd9e6700700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b4d7524870 0x55b4d75269a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:36.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.160+0000 7fd9e6700700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b4d7524870 0x55b4d75269a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32958/0 (socket says 172.21.15.16:32958) 2023-05-20T01:18:36.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.160+0000 7fd9e6700700 1 -- 172.21.15.16:0/484717355 learned_addr learned my addr 172.21.15.16:0/484717355 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:18:36.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.160+0000 7fd9e56fe700 1 --2- 172.21.15.16:0/484717355 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55b4d754d700 0x55b4d7526460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:36.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.160+0000 7fd9e5eff700 1 --2- 172.21.15.16:0/484717355 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b4d7551400 0x55b4d75517d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:36.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.160+0000 7fd9e6700700 1 -- 172.21.15.16:0/484717355 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55b4d754d700 msgr2=0x55b4d7526460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:18:36.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.160+0000 7fd9e6700700 1 --2- 172.21.15.16:0/484717355 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55b4d754d700 0x55b4d7526460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:36.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.160+0000 7fd9e6700700 1 -- 172.21.15.16:0/484717355 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b4d7551400 msgr2=0x55b4d75517d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:18:36.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.160+0000 7fd9e6700700 1 --2- 172.21.15.16:0/484717355 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b4d7551400 0x55b4d75517d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:36.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.160+0000 7fd9e6700700 1 -- 172.21.15.16:0/484717355 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b4d72f95e0 con 0x55b4d7524870 2023-05-20T01:18:36.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.160+0000 7fd9e6700700 1 --2- 172.21.15.16:0/484717355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b4d7524870 0x55b4d75269a0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fd9d8007260 tx=0x7fd9d8007570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84c0ac7277f2c450 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:36.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.160+0000 7fd9c37fe700 1 -- 172.21.15.16:0/484717355 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d80114f0 con 0x55b4d7524870 2023-05-20T01:18:36.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.160+0000 7fd9c37fe700 1 -- 172.21.15.16:0/484717355 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9d8011650 con 0x55b4d7524870 2023-05-20T01:18:36.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.160+0000 7fd9c37fe700 1 -- 172.21.15.16:0/484717355 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d80119e0 con 0x55b4d7524870 2023-05-20T01:18:36.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.161+0000 7fd9f42efb80 1 -- 172.21.15.16:0/484717355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b4d7524870 msgr2=0x55b4d75269a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:36.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.161+0000 7fd9f42efb80 1 --2- 172.21.15.16:0/484717355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b4d7524870 0x55b4d75269a0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fd9d8007260 tx=0x7fd9d8007570 comp rx=0 tx=0).stop 2023-05-20T01:18:36.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.161+0000 7fd9f42efb80 1 -- 172.21.15.16:0/484717355 shutdown_connections 2023-05-20T01:18:36.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.161+0000 7fd9f42efb80 1 --2- 172.21.15.16:0/484717355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b4d7524870 0x55b4d75269a0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:36.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.161+0000 7fd9f42efb80 1 --2- 172.21.15.16:0/484717355 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b4d7551400 0x55b4d75517d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:36.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.161+0000 7fd9f42efb80 1 --2- 172.21.15.16:0/484717355 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55b4d754d700 0x55b4d7526460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:36.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.161+0000 7fd9f42efb80 1 -- 172.21.15.16:0/484717355 >> 172.21.15.16:0/484717355 conn(0x55b4d751bf30 msgr2=0x55b4d756c210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:18:36.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.161+0000 7fd9f42efb80 1 -- 172.21.15.16:0/484717355 shutdown_connections 2023-05-20T01:18:36.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.161+0000 7fd9f42efb80 1 -- 172.21.15.16:0/484717355 wait complete. 2023-05-20T01:18:36.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.162+0000 7fd9f42efb80 1 -- start start 2023-05-20T01:18:36.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.162+0000 7fd9f42efb80 1 Processor -- start 2023-05-20T01:18:36.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.163+0000 7fd9f42efb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b4d7524870 0x55b4d7543360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:36.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.163+0000 7fd9f42efb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b4d754d700 0x55b4d75423c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:36.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.163+0000 7fd9f42efb80 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55b4d7551400 0x55b4d7542900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:36.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.163+0000 7fd9f42efb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4d7419890 con 0x55b4d754d700 2023-05-20T01:18:36.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.163+0000 7fd9f42efb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4d7419530 con 0x55b4d7524870 2023-05-20T01:18:36.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.163+0000 7fd9f42efb80 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b4d72fb8d0 con 0x55b4d7551400 2023-05-20T01:18:36.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.163+0000 7fd9f42efb80 10 client.0 _ll_register_callbacks cb 0x55b4d7437730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-05-20T01:18:36.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.163+0000 7fd9e56fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b4d754d700 0x55b4d75423c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:36.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.163+0000 7fd9e56fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b4d754d700 0x55b4d75423c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32962/0 (socket says 172.21.15.16:32962) 2023-05-20T01:18:36.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.163+0000 7fd9e56fe700 1 -- 172.21.15.16:0/1475511662 learned_addr learned my addr 172.21.15.16:0/1475511662 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:18:36.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.163+0000 7fd9e56fe700 1 -- 172.21.15.16:0/1475511662 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55b4d7551400 msgr2=0x55b4d7542900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:18:36.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.163+0000 7fd9e6700700 1 --2- 172.21.15.16:0/1475511662 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55b4d7551400 0x55b4d7542900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:36.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.163+0000 7fd9e56fe700 1 --2- 172.21.15.16:0/1475511662 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55b4d7551400 0x55b4d7542900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:36.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.163+0000 7fd9e5eff700 1 --2- 172.21.15.16:0/1475511662 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b4d7524870 0x55b4d7543360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:36.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.163+0000 7fd9e56fe700 1 -- 172.21.15.16:0/1475511662 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b4d7524870 msgr2=0x55b4d7543360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:18:36.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.163+0000 7fd9e56fe700 1 --2- 172.21.15.16:0/1475511662 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b4d7524870 0x55b4d7543360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:36.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.163+0000 7fd9e56fe700 1 -- 172.21.15.16:0/1475511662 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9d800f040 con 0x55b4d754d700 2023-05-20T01:18:36.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.163+0000 7fd9e56fe700 1 --2- 172.21.15.16:0/1475511662 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b4d754d700 0x55b4d75423c0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fd9d000a040 tx=0x7fd9d0057060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a76eae300937202 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:36.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.163+0000 7fd9c0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-05-20T01:18:36.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.163+0000 7fd9c0ff9700 1 -- 172.21.15.16:0/1475511662 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d005f9a0 con 0x55b4d754d700 2023-05-20T01:18:36.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.163+0000 7fd9c0ff9700 1 -- 172.21.15.16:0/1475511662 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(51..51 src has 1..51) v4 ==== 7102+0+0 (secure 0 0 0) 0x7fd9d005f510 con 0x55b4d754d700 2023-05-20T01:18:36.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.164+0000 7fd9f42efb80 1 -- 172.21.15.16:0/1475511662 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b4d75196f0 con 0x55b4d754d700 2023-05-20T01:18:36.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.164+0000 7fd9c0ff9700 1 -- 172.21.15.16:0/1475511662 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd9d005f6d0 con 0x55b4d754d700 2023-05-20T01:18:36.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.164+0000 7fd9f42efb80 10 client.5922 fetch_fsmap learned FSMap version 29 2023-05-20T01:18:36.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.164+0000 7fd9f42efb80 1 -- 172.21.15.16:0/1475511662 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) v3 -- 0x55b4d743e140 con 0x55b4d754d700 2023-05-20T01:18:36.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.164+0000 7fd9c0ff9700 1 -- 172.21.15.16:0/1475511662 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b4d75196f0 con 0x55b4d754d700 2023-05-20T01:18:36.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.164+0000 7fd9c0ff9700 1 -- 172.21.15.16:0/1475511662 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 29) v1 ==== 3122+0+0 (secure 0 0 0) 0x7fd9d005b160 con 0x55b4d754d700 2023-05-20T01:18:36.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.164+0000 7fd9f42efb80 10 client.5922 fetch_fsmap finished waiting for FSMap version 29 2023-05-20T01:18:36.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.164+0000 7fd9f42efb80 10 client.5922 resolve_mds: resolved name 'c' to daemon mds.c 2023-05-20T01:18:36.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.164+0000 7fd9f42efb80 20 client.5922 populate_metadata read hostname 'smithi016' 2023-05-20T01:18:36.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.164+0000 7fd9f42efb80 1 --2- 172.21.15.16:0/1475511662 >> [v2:172.21.15.16:6834/3284962389,v1:172.21.15.16:6836/3284962389] conn(0x55b4d7526dd0 0x55b4d756aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:36.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.164+0000 7fd9f42efb80 4 client.5922 mds_command: new command op to 5889 tid=0[{"prefix": "get_command_descriptions"}] 2023-05-20T01:18:36.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.164+0000 7fd9f42efb80 1 -- 172.21.15.16:0/1475511662 --> [v2:172.21.15.16:6834/3284962389,v1:172.21.15.16:6836/3284962389] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b4d743a050 con 0x55b4d7526dd0 2023-05-20T01:18:36.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.164+0000 7fd9e5eff700 1 --2- 172.21.15.16:0/1475511662 >> [v2:172.21.15.16:6834/3284962389,v1:172.21.15.16:6836/3284962389] conn(0x55b4d7526dd0 0x55b4d756aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:36.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.164+0000 7fd9c0ff9700 10 client.5922 ms_handle_connect on v2:172.21.15.16:6834/3284962389 2023-05-20T01:18:36.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.164+0000 7fd9e5eff700 1 --2- 172.21.15.16:0/1475511662 >> [v2:172.21.15.16:6834/3284962389,v1:172.21.15.16:6836/3284962389] conn(0x55b4d7526dd0 0x55b4d756aa10 crc :-1 s=READY pgs=10 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3448e6436ae7d2f1 server_cookie=30bec26f34a3b662 in_seq=0 out_seq=0 2023-05-20T01:18:36.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.165+0000 7fd9c0ff9700 1 -- 172.21.15.16:0/1475511662 <== mds.0 v2:172.21.15.16:6834/3284962389 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55b4d743a050 con 0x55b4d7526dd0 2023-05-20T01:18:36.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.165+0000 7fd9c0ff9700 10 client.5922 handle_command_reply: tid=0 2023-05-20T01:18:36.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.165+0000 7fd9f42efb80 1 client.5922 shutdown 2023-05-20T01:18:36.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.165+0000 7fd9f42efb80 1 -- 172.21.15.16:0/1475511662 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b4d754d700 msgr2=0x55b4d75423c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:36.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.165+0000 7fd9f42efb80 1 --2- 172.21.15.16:0/1475511662 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b4d754d700 0x55b4d75423c0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fd9d000a040 tx=0x7fd9d0057060 comp rx=0 tx=0).stop 2023-05-20T01:18:36.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.165+0000 7fd9f42efb80 1 -- 172.21.15.16:0/1475511662 shutdown_connections 2023-05-20T01:18:36.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.165+0000 7fd9f42efb80 1 --2- 172.21.15.16:0/1475511662 >> [v2:172.21.15.16:6834/3284962389,v1:172.21.15.16:6836/3284962389] conn(0x55b4d7526dd0 0x55b4d756aa10 crc :-1 s=READY pgs=10 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:36.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.165+0000 7fd9f42efb80 1 --2- 172.21.15.16:0/1475511662 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55b4d7551400 0x55b4d7542900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:36.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.165+0000 7fd9f42efb80 1 --2- 172.21.15.16:0/1475511662 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b4d7524870 0x55b4d7543360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:36.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.165+0000 7fd9f42efb80 1 --2- 172.21.15.16:0/1475511662 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b4d754d700 0x55b4d75423c0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:36.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.165+0000 7fd9c0ff9700 0 client.5922 ms_handle_reset on v2:172.21.15.16:6834/3284962389 2023-05-20T01:18:36.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.165+0000 7fd9f42efb80 1 -- 172.21.15.16:0/1475511662 >> 172.21.15.16:0/1475511662 conn(0x55b4d751bf30 msgr2=0x55b4d754ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:18:36.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.165+0000 7fd9f42efb80 1 -- 172.21.15.16:0/1475511662 shutdown_connections 2023-05-20T01:18:36.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.165+0000 7fd9f42efb80 1 -- 172.21.15.16:0/1475511662 wait complete. 2023-05-20T01:18:36.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.166+0000 7fd9f42efb80 20 client.5922 trim_cache size 0 max 16384 2023-05-20T01:18:36.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.183+0000 7fd9f42efb80 1 Processor -- start 2023-05-20T01:18:36.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.183+0000 7fd9f42efb80 1 -- start start 2023-05-20T01:18:36.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.184+0000 7fd9f42efb80 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55b4d7524870 0x55b4d75271b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:36.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.184+0000 7fd9f42efb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b4d754d700 0x55b4d7551a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:36.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.184+0000 7fd9f42efb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b4d7551400 0x55b4d7568dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:36.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.184+0000 7fd9f42efb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4d7548760 con 0x55b4d7551400 2023-05-20T01:18:36.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.184+0000 7fd9f42efb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4d7527b30 con 0x55b4d754d700 2023-05-20T01:18:36.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.184+0000 7fd9f42efb80 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b4d7548af0 con 0x55b4d7524870 2023-05-20T01:18:36.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.184+0000 7fd9e6700700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b4d7551400 0x55b4d7568dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:36.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.184+0000 7fd9e6700700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b4d7551400 0x55b4d7568dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32972/0 (socket says 172.21.15.16:32972) 2023-05-20T01:18:36.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.184+0000 7fd9e6700700 1 -- 172.21.15.16:0/455235931 learned_addr learned my addr 172.21.15.16:0/455235931 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:18:36.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.184+0000 7fd9e56fe700 1 --2- 172.21.15.16:0/455235931 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b4d754d700 0x55b4d7551a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:36.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.184+0000 7fd9e5eff700 1 --2- 172.21.15.16:0/455235931 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55b4d7524870 0x55b4d75271b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:36.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.184+0000 7fd9e6700700 1 -- 172.21.15.16:0/455235931 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55b4d7524870 msgr2=0x55b4d75271b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:18:36.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.184+0000 7fd9e6700700 1 --2- 172.21.15.16:0/455235931 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55b4d7524870 0x55b4d75271b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:36.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.184+0000 7fd9e6700700 1 -- 172.21.15.16:0/455235931 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b4d754d700 msgr2=0x55b4d7551a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:18:36.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.184+0000 7fd9e6700700 1 --2- 172.21.15.16:0/455235931 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b4d754d700 0x55b4d7551a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:36.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.184+0000 7fd9e6700700 1 -- 172.21.15.16:0/455235931 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b4d743de90 con 0x55b4d7551400 2023-05-20T01:18:36.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.184+0000 7fd9e56fe700 1 --2- 172.21.15.16:0/455235931 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b4d754d700 0x55b4d7551a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:18:36.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.184+0000 7fd9e6700700 1 --2- 172.21.15.16:0/455235931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b4d7551400 0x55b4d7568dd0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fd9d80019b0 tx=0x7fd9d800c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4252f3251e678253 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:36.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.185+0000 7fd9c3fff700 1 -- 172.21.15.16:0/455235931 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d80114f0 con 0x55b4d7551400 2023-05-20T01:18:36.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.185+0000 7fd9c3fff700 1 -- 172.21.15.16:0/455235931 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9d800b030 con 0x55b4d7551400 2023-05-20T01:18:36.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.185+0000 7fd9c3fff700 1 -- 172.21.15.16:0/455235931 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d8011a00 con 0x55b4d7551400 2023-05-20T01:18:36.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.185+0000 7fd9f42efb80 1 -- 172.21.15.16:0/455235931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b4d7551400 msgr2=0x55b4d7568dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:36.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.185+0000 7fd9f42efb80 1 --2- 172.21.15.16:0/455235931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b4d7551400 0x55b4d7568dd0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fd9d80019b0 tx=0x7fd9d800c000 comp rx=0 tx=0).stop 2023-05-20T01:18:36.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.185+0000 7fd9f42efb80 1 -- 172.21.15.16:0/455235931 shutdown_connections 2023-05-20T01:18:36.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.185+0000 7fd9f42efb80 1 --2- 172.21.15.16:0/455235931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b4d7551400 0x55b4d7568dd0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:36.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.185+0000 7fd9f42efb80 1 --2- 172.21.15.16:0/455235931 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55b4d7524870 0x55b4d75271b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:36.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.185+0000 7fd9f42efb80 1 --2- 172.21.15.16:0/455235931 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b4d754d700 0x55b4d7551a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:36.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.185+0000 7fd9f42efb80 1 -- 172.21.15.16:0/455235931 >> 172.21.15.16:0/455235931 conn(0x55b4d751bf30 msgr2=0x55b4d751d400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:18:36.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.185+0000 7fd9f42efb80 1 -- 172.21.15.16:0/455235931 shutdown_connections 2023-05-20T01:18:36.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.185+0000 7fd9f42efb80 1 -- 172.21.15.16:0/455235931 wait complete. 2023-05-20T01:18:36.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.186+0000 7fd9f42efb80 1 -- start start 2023-05-20T01:18:36.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.186+0000 7fd9f42efb80 1 Processor -- start 2023-05-20T01:18:36.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.187+0000 7fd9f42efb80 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55b4d7524870 0x55b4d7545c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:36.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.187+0000 7fd9f42efb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b4d754d700 0x55b4d75464e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:36.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.187+0000 7fd9f42efb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b4d7551400 0x55b4d75d2510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:36.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.187+0000 7fd9f42efb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4d75d2a50 con 0x55b4d7551400 2023-05-20T01:18:36.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.187+0000 7fd9f42efb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4d7521400 con 0x55b4d754d700 2023-05-20T01:18:36.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.187+0000 7fd9f42efb80 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b4d754c540 con 0x55b4d7524870 2023-05-20T01:18:36.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.187+0000 7fd9f42efb80 10 client.0 _ll_register_callbacks cb 0x7fd9d0057420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-05-20T01:18:36.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.187+0000 7fd9e6700700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b4d7551400 0x55b4d75d2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:36.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.187+0000 7fd9e6700700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b4d7551400 0x55b4d75d2510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32978/0 (socket says 172.21.15.16:32978) 2023-05-20T01:18:36.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.187+0000 7fd9e6700700 1 -- 172.21.15.16:0/1870212513 learned_addr learned my addr 172.21.15.16:0/1870212513 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:18:36.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.187+0000 7fd9e56fe700 1 --2- 172.21.15.16:0/1870212513 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b4d754d700 0x55b4d75464e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:36.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.187+0000 7fd9e5eff700 1 --2- 172.21.15.16:0/1870212513 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55b4d7524870 0x55b4d7545c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:36.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.187+0000 7fd9e6700700 1 -- 172.21.15.16:0/1870212513 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55b4d7524870 msgr2=0x55b4d7545c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:18:36.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.187+0000 7fd9e6700700 1 --2- 172.21.15.16:0/1870212513 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55b4d7524870 0x55b4d7545c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:36.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.187+0000 7fd9e6700700 1 -- 172.21.15.16:0/1870212513 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b4d754d700 msgr2=0x55b4d75464e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:18:36.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.187+0000 7fd9e6700700 1 --2- 172.21.15.16:0/1870212513 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b4d754d700 0x55b4d75464e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:36.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.187+0000 7fd9e6700700 1 -- 172.21.15.16:0/1870212513 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b4d72f95e0 con 0x55b4d7551400 2023-05-20T01:18:36.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.187+0000 7fd9e6700700 1 --2- 172.21.15.16:0/1870212513 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b4d7551400 0x55b4d75d2510 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fd9d8007970 tx=0x7fd9d8001940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81c308a0cfccf08a server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:36.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.187+0000 7fd9c0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-05-20T01:18:36.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.188+0000 7fd9c0ff9700 1 -- 172.21.15.16:0/1870212513 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d80114f0 con 0x55b4d7551400 2023-05-20T01:18:36.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.188+0000 7fd9c0ff9700 1 -- 172.21.15.16:0/1870212513 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(51..51 src has 1..51) v4 ==== 7102+0+0 (secure 0 0 0) 0x7fd9d8011d10 con 0x55b4d7551400 2023-05-20T01:18:36.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.188+0000 7fd9f42efb80 1 -- 172.21.15.16:0/1870212513 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd9d8011650 con 0x55b4d7551400 2023-05-20T01:18:36.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.188+0000 7fd9c0ff9700 1 -- 172.21.15.16:0/1870212513 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd9d8011ed0 con 0x55b4d7551400 2023-05-20T01:18:36.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.188+0000 7fd9f42efb80 10 client.5928 fetch_fsmap learned FSMap version 29 2023-05-20T01:18:36.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.188+0000 7fd9f42efb80 1 -- 172.21.15.16:0/1870212513 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) v3 -- 0x55b4d728ec10 con 0x55b4d7551400 2023-05-20T01:18:36.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.188+0000 7fd9c0ff9700 1 -- 172.21.15.16:0/1870212513 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9d8011650 con 0x55b4d7551400 2023-05-20T01:18:36.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.188+0000 7fd9c0ff9700 1 -- 172.21.15.16:0/1870212513 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 29) v1 ==== 3122+0+0 (secure 0 0 0) 0x7fd9d8014480 con 0x55b4d7551400 2023-05-20T01:18:36.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.188+0000 7fd9f42efb80 10 client.5928 fetch_fsmap finished waiting for FSMap version 29 2023-05-20T01:18:36.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.188+0000 7fd9f42efb80 10 client.5928 resolve_mds: resolved name 'c' to daemon mds.c 2023-05-20T01:18:36.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.188+0000 7fd9f42efb80 20 client.5928 populate_metadata read hostname 'smithi016' 2023-05-20T01:18:36.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.188+0000 7fd9f42efb80 1 --2- 172.21.15.16:0/1870212513 >> [v2:172.21.15.16:6834/3284962389,v1:172.21.15.16:6836/3284962389] conn(0x55b4d7520a70 0x55b4d7568dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:36.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.188+0000 7fd9f42efb80 4 client.5928 mds_command: new command op to 5889 tid=0[{"prefix": "status"}] 2023-05-20T01:18:36.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.188+0000 7fd9f42efb80 1 -- 172.21.15.16:0/1870212513 --> [v2:172.21.15.16:6834/3284962389,v1:172.21.15.16:6836/3284962389] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55b4d7446610 con 0x55b4d7520a70 2023-05-20T01:18:36.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.188+0000 7fd9e5eff700 1 --2- 172.21.15.16:0/1870212513 >> [v2:172.21.15.16:6834/3284962389,v1:172.21.15.16:6836/3284962389] conn(0x55b4d7520a70 0x55b4d7568dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:36.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.188+0000 7fd9c0ff9700 10 client.5928 ms_handle_connect on v2:172.21.15.16:6834/3284962389 2023-05-20T01:18:36.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.188+0000 7fd9e5eff700 1 --2- 172.21.15.16:0/1870212513 >> [v2:172.21.15.16:6834/3284962389,v1:172.21.15.16:6836/3284962389] conn(0x55b4d7520a70 0x55b4d7568dd0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c17f9aae3b10ae6 server_cookie=df3b154a42fcbc81 in_seq=0 out_seq=0 2023-05-20T01:18:36.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.189+0000 7fd9c0ff9700 1 -- 172.21.15.16:0/1870212513 <== mds.0 v2:172.21.15.16:6834/3284962389 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55b4d7446610 con 0x55b4d7520a70 2023-05-20T01:18:36.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.189+0000 7fd9c0ff9700 10 client.5928 handle_command_reply: tid=0 2023-05-20T01:18:36.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.189+0000 7fd9f42efb80 1 client.5928 shutdown 2023-05-20T01:18:36.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.189+0000 7fd9f42efb80 1 -- 172.21.15.16:0/1870212513 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b4d7551400 msgr2=0x55b4d75d2510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:36.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.189+0000 7fd9f42efb80 1 --2- 172.21.15.16:0/1870212513 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b4d7551400 0x55b4d75d2510 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fd9d8007970 tx=0x7fd9d8001940 comp rx=0 tx=0).stop 2023-05-20T01:18:36.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.189+0000 7fd9f42efb80 1 -- 172.21.15.16:0/1870212513 shutdown_connections 2023-05-20T01:18:36.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.189+0000 7fd9f42efb80 1 --2- 172.21.15.16:0/1870212513 >> [v2:172.21.15.16:6834/3284962389,v1:172.21.15.16:6836/3284962389] conn(0x55b4d7520a70 0x55b4d7568dd0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:36.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.189+0000 7fd9f42efb80 1 --2- 172.21.15.16:0/1870212513 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b4d7551400 0x55b4d75d2510 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:36.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.189+0000 7fd9f42efb80 1 --2- 172.21.15.16:0/1870212513 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55b4d7524870 0x55b4d7545c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:36.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.189+0000 7fd9c0ff9700 0 client.5928 ms_handle_reset on v2:172.21.15.16:6834/3284962389 2023-05-20T01:18:36.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.189+0000 7fd9f42efb80 1 --2- 172.21.15.16:0/1870212513 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b4d754d700 0x55b4d75464e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:36.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.189+0000 7fd9f42efb80 1 -- 172.21.15.16:0/1870212513 >> 172.21.15.16:0/1870212513 conn(0x55b4d751bf30 msgr2=0x55b4d75271b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:18:36.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.189+0000 7fd9f42efb80 1 -- 172.21.15.16:0/1870212513 shutdown_connections 2023-05-20T01:18:36.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.189+0000 7fd9f42efb80 1 -- 172.21.15.16:0/1870212513 wait complete. 2023-05-20T01:18:36.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.189+0000 7fd9f42efb80 20 client.5928 trim_cache size 0 max 16384 2023-05-20T01:18:36.203 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-05-20T01:18:36.203 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "c2245cc5-ecab-4087-a59b-bc81db901676", 2023-05-20T01:18:36.203 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2023-05-20T01:18:36.203 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 5889, 2023-05-20T01:18:36.203 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2023-05-20T01:18:36.203 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2023-05-20T01:18:36.203 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2023-05-20T01:18:36.204 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 3.7022864819999999, 2023-05-20T01:18:36.204 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 29, 2023-05-20T01:18:36.204 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 51, 2023-05-20T01:18:36.204 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 51, 2023-05-20T01:18:36.204 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 4.5394661740000002 2023-05-20T01:18:36.204 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-05-20T01:18:36.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.190+0000 7fd9e8163700 1 -- 172.21.15.16:0/1611048930 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd9c406ace0 msgr2=0x7fd9c406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:36.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.190+0000 7fd9e8163700 1 --2- 172.21.15.16:0/1611048930 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd9c406ace0 0x7fd9c406d190 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc00b340 tx=0x7fd9dc01b040 comp rx=0 tx=0).stop 2023-05-20T01:18:36.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.190+0000 7fd9e8163700 1 -- 172.21.15.16:0/1611048930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9e00cd0f0 msgr2=0x7fd9e00c8640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:36.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.190+0000 7fd9e8163700 1 --2- 172.21.15.16:0/1611048930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9e00cd0f0 0x7fd9e00c8640 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fd9d000c6b0 tx=0x7fd9d000c9c0 comp rx=0 tx=0).stop 2023-05-20T01:18:36.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.190+0000 7fd9e8163700 1 -- 172.21.15.16:0/1611048930 shutdown_connections 2023-05-20T01:18:36.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.190+0000 7fd9e8163700 1 --2- 172.21.15.16:0/1611048930 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd9e01599b0 0x7fd9e00c7980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:36.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.190+0000 7fd9e8163700 1 --2- 172.21.15.16:0/1611048930 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd9c406ace0 0x7fd9c406d190 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:36.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.190+0000 7fd9e8163700 1 --2- 172.21.15.16:0/1611048930 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd9e00b0910 0x7fd9e01ecb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:36.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.190+0000 7fd9e8163700 1 --2- 172.21.15.16:0/1611048930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9e00cd0f0 0x7fd9e00c8640 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:36.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.190+0000 7fd9e8163700 1 -- 172.21.15.16:0/1611048930 >> 172.21.15.16:0/1611048930 conn(0x7fd9e00cb910 msgr2=0x7fd9e0153740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:18:36.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.190+0000 7fd9e8163700 1 -- 172.21.15.16:0/1611048930 shutdown_connections 2023-05-20T01:18:36.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.190+0000 7fd9e8163700 1 -- 172.21.15.16:0/1611048930 wait complete. 2023-05-20T01:18:36.207 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-20T01:18:36.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.328+0000 7fb1455dd700 1 Processor -- start 2023-05-20T01:18:36.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.328+0000 7fb1455dd700 1 -- start start 2023-05-20T01:18:36.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.329+0000 7fb1455dd700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fb1400ccec0 0x7fb1400713c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:36.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.329+0000 7fb1455dd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb140071900 0x7fb1400ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:36.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.329+0000 7fb1455dd700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1400cb590 con 0x7fb140157190 2023-05-20T01:18:36.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.329+0000 7fb1455dd700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1400ca9a0 con 0x7fb140071900 2023-05-20T01:18:36.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.329+0000 7fb1455dd700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1400cac60 con 0x7fb1400ccec0 2023-05-20T01:18:36.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.329+0000 7fb13effd700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fb140157190 0x7fb1400cc7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:44510/0 (socket says 172.21.15.16:44510) 2023-05-20T01:18:36.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.329+0000 7fb13f7fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb140071900 0x7fb1400ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:36.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.329+0000 7fb13effd700 1 -- 172.21.15.16:0/3556817318 learned_addr learned my addr 172.21.15.16:0/3556817318 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:18:36.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.330+0000 7fb13e7fc700 1 --2- 172.21.15.16:0/3556817318 >> v2:172.21.15.132:3300/0 conn(0x7fb1400ccec0 0x7fb1400713c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:36.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.330+0000 7fb13f7fe700 1 -- 172.21.15.16:0/3556817318 >> v2:172.21.15.132:3300/0 conn(0x7fb1400ccec0 msgr2=0x7fb1400713c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:18:36.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.330+0000 7fb13f7fe700 1 --2- 172.21.15.16:0/3556817318 >> v2:172.21.15.132:3300/0 conn(0x7fb1400ccec0 0x7fb1400713c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:36.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.330+0000 7fb13f7fe700 1 -- 172.21.15.16:0/3556817318 >> v1:172.21.15.16:6789/0 conn(0x7fb140157190 legacy=0x7fb1400cc7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:36.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.330+0000 7fb13f7fe700 1 -- 172.21.15.16:0/3556817318 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1400caf40 con 0x7fb140071900 2023-05-20T01:18:36.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.330+0000 7fb13f7fe700 1 --2- 172.21.15.16:0/3556817318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb140071900 0x7fb1400ce740 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fb1400b6120 tx=0x7fb13800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de9de757a70a5c47 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:36.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.330+0000 7fb13dffb700 1 -- 172.21.15.16:0/3556817318 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb13800e070 con 0x7fb140071900 2023-05-20T01:18:36.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.330+0000 7fb13dffb700 1 -- 172.21.15.16:0/3556817318 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb13800bbd0 con 0x7fb140071900 2023-05-20T01:18:36.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.330+0000 7fb13dffb700 1 -- 172.21.15.16:0/3556817318 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb138012430 con 0x7fb140071900 2023-05-20T01:18:36.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.331+0000 7fb1455dd700 1 -- 172.21.15.16:0/3556817318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb140071900 msgr2=0x7fb1400ce740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:36.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.331+0000 7fb1455dd700 1 --2- 172.21.15.16:0/3556817318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb140071900 0x7fb1400ce740 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fb1400b6120 tx=0x7fb13800b330 comp rx=0 tx=0).stop 2023-05-20T01:18:36.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.331+0000 7fb1455dd700 1 -- 172.21.15.16:0/3556817318 shutdown_connections 2023-05-20T01:18:36.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.331+0000 7fb1455dd700 1 --2- 172.21.15.16:0/3556817318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb140071900 0x7fb1400ce740 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:36.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.331+0000 7fb1455dd700 1 --2- 172.21.15.16:0/3556817318 >> v2:172.21.15.132:3300/0 conn(0x7fb1400ccec0 0x7fb1400713c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:36.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.331+0000 7fb1455dd700 1 -- 172.21.15.16:0/3556817318 >> 172.21.15.16:0/3556817318 conn(0x7fb1400cffb0 msgr2=0x7fb1400d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:18:36.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.331+0000 7fb1455dd700 1 -- 172.21.15.16:0/3556817318 shutdown_connections 2023-05-20T01:18:36.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.331+0000 7fb1455dd700 1 -- 172.21.15.16:0/3556817318 wait complete. 2023-05-20T01:18:36.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.332+0000 7fb1455dd700 1 Processor -- start 2023-05-20T01:18:36.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.333+0000 7fb1455dd700 1 -- start start 2023-05-20T01:18:36.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.333+0000 7fb1455dd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb140071900 0x7fb1400c6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:36.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.333+0000 7fb1455dd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1400ccec0 0x7fb1401451e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:36.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.333+0000 7fb1455dd700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb140157190 0x7fb140158740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:36.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.333+0000 7fb1455dd700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1401527b0 con 0x7fb1400ccec0 2023-05-20T01:18:36.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.333+0000 7fb1455dd700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1400b32b0 con 0x7fb140071900 2023-05-20T01:18:36.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.333+0000 7fb1455dd700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1400b3590 con 0x7fb140157190 2023-05-20T01:18:36.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.334+0000 7fb13e7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1400ccec0 0x7fb1401451e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:36.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.334+0000 7fb13e7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1400ccec0 0x7fb1401451e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32990/0 (socket says 172.21.15.16:32990) 2023-05-20T01:18:36.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.334+0000 7fb13e7fc700 1 -- 172.21.15.16:0/2672769780 learned_addr learned my addr 172.21.15.16:0/2672769780 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:18:36.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.334+0000 7fb13effd700 1 --2- 172.21.15.16:0/2672769780 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb140071900 0x7fb1400c6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:36.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.334+0000 7fb13f7fe700 1 --2- 172.21.15.16:0/2672769780 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb140157190 0x7fb140158740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:36.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.334+0000 7fb13e7fc700 1 -- 172.21.15.16:0/2672769780 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb140157190 msgr2=0x7fb140158740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:36.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.334+0000 7fb13e7fc700 1 --2- 172.21.15.16:0/2672769780 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb140157190 0x7fb140158740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:36.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.334+0000 7fb13e7fc700 1 -- 172.21.15.16:0/2672769780 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb140071900 msgr2=0x7fb1400c6430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:36.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.334+0000 7fb13e7fc700 1 --2- 172.21.15.16:0/2672769780 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb140071900 0x7fb1400c6430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:36.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.334+0000 7fb13e7fc700 1 -- 172.21.15.16:0/2672769780 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb138009a70 con 0x7fb1400ccec0 2023-05-20T01:18:36.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.334+0000 7fb13f7fe700 1 --2- 172.21.15.16:0/2672769780 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb140157190 0x7fb140158740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:18:36.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.334+0000 7fb13effd700 1 --2- 172.21.15.16:0/2672769780 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb140071900 0x7fb1400c6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:18:36.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.334+0000 7fb13e7fc700 1 --2- 172.21.15.16:0/2672769780 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1400ccec0 0x7fb1401451e0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fb13400e810 tx=0x7fb13400ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:36.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.334+0000 7fb1277fe700 1 -- 172.21.15.16:0/2672769780 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb134013610 con 0x7fb1400ccec0 2023-05-20T01:18:36.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.334+0000 7fb1277fe700 1 -- 172.21.15.16:0/2672769780 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb134013770 con 0x7fb1400ccec0 2023-05-20T01:18:36.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.334+0000 7fb1455dd700 1 -- 172.21.15.16:0/2672769780 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb140157560 con 0x7fb1400ccec0 2023-05-20T01:18:36.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.334+0000 7fb1277fe700 1 -- 172.21.15.16:0/2672769780 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb134013b20 con 0x7fb1400ccec0 2023-05-20T01:18:36.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.334+0000 7fb1455dd700 1 -- 172.21.15.16:0/2672769780 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1400cb220 con 0x7fb1400ccec0 2023-05-20T01:18:36.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.335+0000 7fb1455dd700 1 -- 172.21.15.16:0/2672769780 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1400cba20 con 0x7fb1400ccec0 2023-05-20T01:18:36.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.336+0000 7fb1277fe700 1 -- 172.21.15.16:0/2672769780 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb134013c80 con 0x7fb1400ccec0 2023-05-20T01:18:36.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.336+0000 7fb1277fe700 1 --2- 172.21.15.16:0/2672769780 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb12c06ace0 0x7fb12c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:36.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.336+0000 7fb1277fe700 1 -- 172.21.15.16:0/2672769780 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 7102+0+0 (secure 0 0 0) 0x7fb134089560 con 0x7fb1400ccec0 2023-05-20T01:18:36.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.336+0000 7fb13effd700 1 --2- 172.21.15.16:0/2672769780 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb12c06ace0 0x7fb12c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:36.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.337+0000 7fb13effd700 1 --2- 172.21.15.16:0/2672769780 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb12c06ace0 0x7fb12c06d190 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fb130000f60 tx=0x7fb130009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:36.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.341+0000 7fb1277fe700 1 -- 172.21.15.16:0/2672769780 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fb134057ca0 con 0x7fb1400ccec0 2023-05-20T01:18:36.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.584+0000 7fb1455dd700 1 -- 172.21.15.16:0/2672769780 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb1400747b0 con 0x7fb1400ccec0 2023-05-20T01:18:36.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.584+0000 7fb1277fe700 1 -- 172.21.15.16:0/2672769780 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+4680 (secure 0 0 0) 0x7fb134057830 con 0x7fb1400ccec0 2023-05-20T01:18:36.589 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:18:36.589 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":29,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5246,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/2629811751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":2629811751},{"type":"v1","addr":"172.21.15.132:6834","nonce":2629811751}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":5249,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2117148777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":2117148777},{"type":"v1","addr":"172.21.15.132:6835","nonce":2117148777}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":5566,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2025052254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":2025052254},{"type":"v1","addr":"172.21.15.16:6837","nonce":2025052254}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-20T01:18:26.227827+0000","modified":"2023-05-20T01:18:35.503777+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5889},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5889":{"gid":5889,"name":"c","rank":0,"incarnation":26,"state":"up:active","state_seq":4,"addr":"172.21.15.16:6836/3284962389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3284962389},{"type":"v1","addr":"172.21.15.16:6836","nonce":3284962389}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7,8],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-05-20T01:18:36.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.587+0000 7fb1455dd700 1 -- 172.21.15.16:0/2672769780 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb12c06ace0 msgr2=0x7fb12c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:36.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.587+0000 7fb1455dd700 1 --2- 172.21.15.16:0/2672769780 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb12c06ace0 0x7fb12c06d190 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fb130000f60 tx=0x7fb130009040 comp rx=0 tx=0).stop 2023-05-20T01:18:36.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.587+0000 7fb1455dd700 1 -- 172.21.15.16:0/2672769780 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1400ccec0 msgr2=0x7fb1401451e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:36.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.587+0000 7fb1455dd700 1 --2- 172.21.15.16:0/2672769780 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1400ccec0 0x7fb1401451e0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fb13400e810 tx=0x7fb13400ebd0 comp rx=0 tx=0).stop 2023-05-20T01:18:36.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.587+0000 7fb1455dd700 1 -- 172.21.15.16:0/2672769780 shutdown_connections 2023-05-20T01:18:36.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.587+0000 7fb1455dd700 1 --2- 172.21.15.16:0/2672769780 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb12c06ace0 0x7fb12c06d190 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:36.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.587+0000 7fb1455dd700 1 --2- 172.21.15.16:0/2672769780 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb140157190 0x7fb140158740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:36.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.587+0000 7fb1455dd700 1 --2- 172.21.15.16:0/2672769780 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb140071900 0x7fb1400c6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:36.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.587+0000 7fb1455dd700 1 --2- 172.21.15.16:0/2672769780 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1400ccec0 0x7fb1401451e0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:36.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.587+0000 7fb1455dd700 1 -- 172.21.15.16:0/2672769780 >> 172.21.15.16:0/2672769780 conn(0x7fb1400cffb0 msgr2=0x7fb1400cc7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:18:36.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.587+0000 7fb1455dd700 1 -- 172.21.15.16:0/2672769780 shutdown_connections 2023-05-20T01:18:36.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.587+0000 7fb1455dd700 1 -- 172.21.15.16:0/2672769780 wait complete. 2023-05-20T01:18:36.594 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 29 2023-05-20T01:18:36.603 INFO:tasks.cephfs.test_data_scan:{ "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 2, "epoch": 29, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 2, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-05-20T01:18:26.227827+0000", "damaged": [], "data_pools": [ 7, 8 ], "enabled": true, "epoch": 29, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_5889": { "addr": "172.21.15.16:6836/3284962389", "addrs": { "addrvec": [ { "addr": "172.21.15.16:6834", "nonce": 3284962389, "type": "v2" }, { "addr": "172.21.15.16:6836", "nonce": 3284962389, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5889, "incarnation": 26, "join_fscid": -1, "name": "c", "rank": 0, "state": "up:active", "state_seq": 4 } }, "last_failure": 0, "last_failure_osd_epoch": 50, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 6, "modified": "2023-05-20T01:18:35.503777+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 60, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5889 } } } ], "standbys": [ { "addr": "172.21.15.132:6834/2629811751", "addrs": { "addrvec": [ { "addr": "172.21.15.132:6832", "nonce": 2629811751, "type": "v2" }, { "addr": "172.21.15.132:6834", "nonce": 2629811751, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 25, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5246, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.132:6835/2117148777", "addrs": { "addrvec": [ { "addr": "172.21.15.132:6833", "nonce": 2117148777, "type": "v2" }, { "addr": "172.21.15.132:6835", "nonce": 2117148777, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 25, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5249, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.16:6837/2025052254", "addrs": { "addrvec": [ { "addr": "172.21.15.16:6835", "nonce": 2025052254, "type": "v2" }, { "addr": "172.21.15.16:6837", "nonce": 2025052254, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 25, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5566, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-05-20T01:18:36.604 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-05-20T01:18:36.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.726+0000 7f81b8c37700 1 Processor -- start 2023-05-20T01:18:36.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.727+0000 7f81b8c37700 1 -- start start 2023-05-20T01:18:36.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.727+0000 7f81b8c37700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f81b406a510 0x7f81b415e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:36.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.727+0000 7f81b8c37700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81b415e060 con 0x7f81b406a8e0 2023-05-20T01:18:36.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.727+0000 7f81b8c37700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81b415d450 con 0x7f81b4071340 2023-05-20T01:18:36.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.727+0000 7f81b8c37700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81b415d730 con 0x7f81b406a510 2023-05-20T01:18:36.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.728+0000 7f81b259c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f81b406a510 0x7f81b415e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:36.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.728+0000 7f81b1d9b700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f81b406a8e0 0x7f81b40cf5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:44526/0 (socket says 172.21.15.16:44526) 2023-05-20T01:18:36.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.728+0000 7f81b1d9b700 1 -- 172.21.15.16:0/1469931481 learned_addr learned my addr 172.21.15.16:0/1469931481 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:18:36.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.728+0000 7f81b259c700 1 -- 172.21.15.16:0/1469931481 >> v1:172.21.15.132:6789/0 conn(0x7f81b4071340 legacy=0x7f81b4071780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:18:36.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.728+0000 7f81b259c700 1 -- 172.21.15.16:0/1469931481 >> v1:172.21.15.16:6789/0 conn(0x7f81b406a8e0 legacy=0x7f81b40cf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:36.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.728+0000 7f81b259c700 1 -- 172.21.15.16:0/1469931481 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81b415da10 con 0x7f81b406a510 2023-05-20T01:18:36.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.728+0000 7f81b159a700 1 -- 172.21.15.16:0/1469931481 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3171488441 0 0) 0x7f81b415e060 con 0x7f81b406a8e0 2023-05-20T01:18:36.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.728+0000 7f81b259c700 1 --2- 172.21.15.16:0/1469931481 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81b406a510 0x7f81b415e2a0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f81a4009840 tx=0x7f81a4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8660ff2c7ad63c server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:36.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.728+0000 7f81b159a700 1 -- 172.21.15.16:0/1469931481 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a4004230 con 0x7f81b406a510 2023-05-20T01:18:36.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.728+0000 7f81b159a700 1 -- 172.21.15.16:0/1469931481 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81a4004390 con 0x7f81b406a510 2023-05-20T01:18:36.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.728+0000 7f81b159a700 1 -- 172.21.15.16:0/1469931481 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a40036a0 con 0x7f81b406a510 2023-05-20T01:18:36.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.729+0000 7f81b8c37700 1 -- 172.21.15.16:0/1469931481 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81b406a510 msgr2=0x7f81b415e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:36.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.729+0000 7f81b8c37700 1 --2- 172.21.15.16:0/1469931481 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81b406a510 0x7f81b415e2a0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f81a4009840 tx=0x7f81a4009b50 comp rx=0 tx=0).stop 2023-05-20T01:18:36.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.729+0000 7f81b8c37700 1 -- 172.21.15.16:0/1469931481 shutdown_connections 2023-05-20T01:18:36.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.729+0000 7f81b8c37700 1 --2- 172.21.15.16:0/1469931481 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81b406a510 0x7f81b415e2a0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:36.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.729+0000 7f81b8c37700 1 -- 172.21.15.16:0/1469931481 >> 172.21.15.16:0/1469931481 conn(0x7f81b40d0c30 msgr2=0x7f81b40b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:18:36.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.729+0000 7f81b8c37700 1 -- 172.21.15.16:0/1469931481 shutdown_connections 2023-05-20T01:18:36.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.729+0000 7f81b8c37700 1 -- 172.21.15.16:0/1469931481 wait complete. 2023-05-20T01:18:36.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.731+0000 7f81b8c37700 1 Processor -- start 2023-05-20T01:18:36.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.731+0000 7f81b8c37700 1 -- start start 2023-05-20T01:18:36.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.732+0000 7f81b8c37700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81b406a510 0x7f81b40c4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:36.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.732+0000 7f81b8c37700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f81b406a8e0 0x7f81b40c5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:36.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.732+0000 7f81b8c37700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f81b4071340 0x7f81b41f5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:36.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.732+0000 7f81b8c37700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81b40c65f0 con 0x7f81b406a510 2023-05-20T01:18:36.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.732+0000 7f81b8c37700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81b40c6a10 con 0x7f81b4071340 2023-05-20T01:18:36.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.732+0000 7f81b8c37700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81b40c5780 con 0x7f81b406a8e0 2023-05-20T01:18:36.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.732+0000 7f81b259c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81b406a510 0x7f81b40c4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:36.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.732+0000 7f81b259c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81b406a510 0x7f81b40c4290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33002/0 (socket says 172.21.15.16:33002) 2023-05-20T01:18:36.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.732+0000 7f81b259c700 1 -- 172.21.15.16:0/3853480565 learned_addr learned my addr 172.21.15.16:0/3853480565 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:18:36.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.732+0000 7f81b1d9b700 1 --2- 172.21.15.16:0/3853480565 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f81b406a8e0 0x7f81b40c5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:36.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.732+0000 7f81b259c700 1 -- 172.21.15.16:0/3853480565 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f81b406a8e0 msgr2=0x7f81b40c5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:36.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.732+0000 7f81b259c700 1 --2- 172.21.15.16:0/3853480565 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f81b406a8e0 0x7f81b40c5240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:36.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.732+0000 7f81b259c700 1 -- 172.21.15.16:0/3853480565 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f81b4071340 msgr2=0x7f81b41f5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:36.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.732+0000 7f81b259c700 1 --2- 172.21.15.16:0/3853480565 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f81b4071340 0x7f81b41f5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:36.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.732+0000 7f81b259c700 1 -- 172.21.15.16:0/3853480565 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81a40094a0 con 0x7f81b406a510 2023-05-20T01:18:36.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.732+0000 7f81b259c700 1 --2- 172.21.15.16:0/3853480565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81b406a510 0x7f81b40c4290 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f81a40090b0 tx=0x7f81a4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:36.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.732+0000 7f81a2ffd700 1 -- 172.21.15.16:0/3853480565 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a4011830 con 0x7f81b406a510 2023-05-20T01:18:36.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.732+0000 7f81a2ffd700 1 -- 172.21.15.16:0/3853480565 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81a401e070 con 0x7f81b406a510 2023-05-20T01:18:36.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.732+0000 7f81b8c37700 1 -- 172.21.15.16:0/3853480565 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81b40cfd00 con 0x7f81b406a510 2023-05-20T01:18:36.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.732+0000 7f81a2ffd700 1 -- 172.21.15.16:0/3853480565 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a4011bb0 con 0x7f81b406a510 2023-05-20T01:18:36.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.733+0000 7f81b8c37700 1 -- 172.21.15.16:0/3853480565 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81b415dcf0 con 0x7f81b406a510 2023-05-20T01:18:36.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.734+0000 7f81b8c37700 1 -- 172.21.15.16:0/3853480565 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81b406ad50 con 0x7f81b406a510 2023-05-20T01:18:36.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.734+0000 7f81a2ffd700 1 -- 172.21.15.16:0/3853480565 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f81a4011d10 con 0x7f81b406a510 2023-05-20T01:18:36.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.734+0000 7f81a2ffd700 1 --2- 172.21.15.16:0/3853480565 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f819406ace0 0x7f819406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:36.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.734+0000 7f81a2ffd700 1 -- 172.21.15.16:0/3853480565 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 7102+0+0 (secure 0 0 0) 0x7f81a408ac20 con 0x7f81b406a510 2023-05-20T01:18:36.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.734+0000 7f81b1d9b700 1 --2- 172.21.15.16:0/3853480565 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f819406ace0 0x7f819406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:36.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.735+0000 7f81b1d9b700 1 --2- 172.21.15.16:0/3853480565 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f819406ace0 0x7f819406d190 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f81b40c8e10 tx=0x7f81a8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:36.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.739+0000 7f81a2ffd700 1 -- 172.21.15.16:0/3853480565 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f81a4055950 con 0x7f81b406a510 2023-05-20T01:18:36.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.982+0000 7f81b8c37700 1 -- 172.21.15.16:0/3853480565 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f81b40747b0 con 0x7f81b406a510 2023-05-20T01:18:37.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.982+0000 7f81a2ffd700 1 -- 172.21.15.16:0/3853480565 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v29) v1 ==== 53+0+91 (secure 0 0 0) 0x7f81a4058f70 con 0x7f81b406a510 2023-05-20T01:18:37.036 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_extra ] 2023-05-20T01:18:37.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.984+0000 7f81b8c37700 1 -- 172.21.15.16:0/3853480565 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f819406ace0 msgr2=0x7f819406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:37.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.984+0000 7f81b8c37700 1 --2- 172.21.15.16:0/3853480565 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f819406ace0 0x7f819406d190 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f81b40c8e10 tx=0x7f81a8009040 comp rx=0 tx=0).stop 2023-05-20T01:18:37.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.984+0000 7f81b8c37700 1 -- 172.21.15.16:0/3853480565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81b406a510 msgr2=0x7f81b40c4290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:37.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.984+0000 7f81b8c37700 1 --2- 172.21.15.16:0/3853480565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81b406a510 0x7f81b40c4290 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f81a40090b0 tx=0x7f81a4004750 comp rx=0 tx=0).stop 2023-05-20T01:18:37.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.984+0000 7f81b8c37700 1 -- 172.21.15.16:0/3853480565 shutdown_connections 2023-05-20T01:18:37.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.984+0000 7f81b8c37700 1 --2- 172.21.15.16:0/3853480565 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f81b4071340 0x7f81b41f5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:37.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.984+0000 7f81b8c37700 1 --2- 172.21.15.16:0/3853480565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81b406a510 0x7f81b40c4290 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:37.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.984+0000 7f81b8c37700 1 --2- 172.21.15.16:0/3853480565 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f819406ace0 0x7f819406d190 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:37.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.984+0000 7f81b8c37700 1 --2- 172.21.15.16:0/3853480565 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f81b406a8e0 0x7f81b40c5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:37.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.984+0000 7f81b8c37700 1 -- 172.21.15.16:0/3853480565 >> 172.21.15.16:0/3853480565 conn(0x7f81b40d0c30 msgr2=0x7f81b40cd040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:18:37.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.984+0000 7f81b8c37700 1 -- 172.21.15.16:0/3853480565 shutdown_connections 2023-05-20T01:18:37.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:36.984+0000 7f81b8c37700 1 -- 172.21.15.16:0/3853480565 wait complete. 2023-05-20T01:18:37.039 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-05-20T01:18:37.040 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-05-20T01:18:37.040 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2023-05-20T01:18:37.040 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-05-20T01:18:37.041 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-05-20T01:18:37.041 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-05-20T01:18:37.041 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-05-20T01:18:37.041 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-05-20T01:18:37.042 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-05-20T01:18:37.042 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:18:37.042 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2023-05-20T01:18:37.060 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-05-20T01:18:37.060 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-05-20T01:18:37.061 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2023-05-20T01:18:37.061 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-05-20T01:18:37.061 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2023-05-20T01:18:37.061 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-05-20T01:18:37.061 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-05-20T01:18:37.061 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2023-05-20T01:18:37.061 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-05-20T01:18:37.062 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2023-05-20T01:18:37.062 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-05-20T01:18:37.062 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2023-05-20T01:18:37.062 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-05-20T01:18:37.062 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 42317sec preferred_lft 42317sec 2023-05-20T01:18:37.062 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link 2023-05-20T01:18:37.062 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-05-20T01:18:37.062 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-05-20T01:18:37.063 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2023-05-20T01:18:37.063 INFO:teuthology.orchestra.run.smithi016.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-05-20T01:18:37.063 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0a:2e:0c:6b:5d:42 brd ff:ff:ff:ff:ff:ff 2023-05-20T01:18:37.063 INFO:teuthology.orchestra.run.smithi016.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-05-20T01:18:37.063 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-05-20T01:18:37.063 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::7c00:b2ff:fef4:946/64 scope link 2023-05-20T01:18:37.063 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-05-20T01:18:37.064 INFO:teuthology.orchestra.run.smithi016.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-05-20T01:18:37.064 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether d2:e4:70:a7:b9:4b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-05-20T01:18:37.064 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::d0e4:70ff:fea7:b94b/64 scope link 2023-05-20T01:18:37.064 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-05-20T01:18:37.064 INFO:teuthology.orchestra.run.smithi016.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-05-20T01:18:37.064 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0a:2e:0c:6b:5d:42 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-05-20T01:18:37.064 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::82e:cff:fe6b:5d42/64 scope link 2023-05-20T01:18:37.065 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-05-20T01:18:37.065 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:18:37.065 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-05-20T01:18:37.126 INFO:teuthology.orchestra.run.smithi016.stdout:1 2023-05-20T01:18:37.127 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:18:37.128 DEBUG:teuthology.orchestra.run.smithi016:> route 2023-05-20T01:18:37.186 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2023-05-20T01:18:37.186 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-05-20T01:18:37.187 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-05-20T01:18:37.187 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-05-20T01:18:37.187 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-05-20T01:18:37.187 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:18:37.188 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2023-05-20T01:18:37.188 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2023-05-20T01:18:37.188 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-05-20T01:18:37.189 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-05-20T01:18:37.189 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-05-20T01:18:37.189 DEBUG:teuthology.orchestra.run.smithi016:> ') 2023-05-20T01:18:37.293 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:18:37.294 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2023-05-20T01:18:37.311 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-05-20T01:18:37.311 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-05-20T01:18:37.312 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-05-20T01:18:37.312 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:18:37.313 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.0 up 2023-05-20T01:18:37.378 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-05-20T01:18:37.379 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-05-20T01:18:37.435 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-05-20T01:18:37.435 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-05-20T01:18:37.436 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-05-20T01:18:37.493 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-05-20T01:18:37.493 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:18:37.494 DEBUG:teuthology.orchestra.run.smithi016:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-05-20T01:18:37.497 INFO:tasks.ceph.mds.c.smithi016.stderr:2023-05-20T01:18:37.493+0000 7f44b2f09700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-05-20T01:18:37.542 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-05-20T01:18:37.543 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-05-20T01:18:37.571 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-05-20T01:18:37.571 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-05-20T01:18:37.572 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.132:3300,172.21.15.132:3301" 2023-05-20T01:18:37.572 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: trying mount with new device syntax: 0@c2245cc5-ecab-4087-a59b-bc81db901676.cephfs=/ 2023-05-20T01:18:37.572 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.16:3300/172.21.15.132:3300/172.21.15.132:3301" will pass to kernel 2023-05-20T01:18:37.572 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: v2 mount support check returned 2 2023-05-20T01:18:37.572 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: kernel does not support v2 syntax 2023-05-20T01:18:37.572 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: switching to using v1 address with old syntax 2023-05-20T01:18:37.609 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-05-20T01:18:37.610 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-05-20T01:18:37.610 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.132:3300,172.21.15.132:3301" 2023-05-20T01:18:37.610 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: trying mount with new device syntax: 0@c2245cc5-ecab-4087-a59b-bc81db901676.cephfs=/ 2023-05-20T01:18:37.610 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.16:3300/172.21.15.132:3300/172.21.15.132:3301" will pass to kernel 2023-05-20T01:18:37.611 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: v2 mount support check returned 2 2023-05-20T01:18:37.611 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: kernel does not support v2 syntax 2023-05-20T01:18:37.611 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: switching to using v1 address with old syntax 2023-05-20T01:18:37.611 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:6789,172.21.15.132:6789,172.21.15.132:6790" 2023-05-20T01:18:37.611 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.16:6789,172.21.15.132:6789,172.21.15.132:6790:/ 2023-05-20T01:18:37.611 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=c2245cc5-ecab-4087-a59b-bc81db901676" will pass to kernel 2023-05-20T01:18:37.618 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-05-20T01:18:37.619 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:18:37.619 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-05-20T01:18:37.653 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:18:37.654 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-05-20T01:18:37.733 INFO:teuthology.orchestra.run.smithi016.stdout:client5949 2023-05-20T01:18:37.734 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-05-20T01:18:37.734 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2023-05-20T01:18:37.790 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 scrub start / repair,recursive 2023-05-20T01:18:37.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.956+0000 7f2e6f232700 1 Processor -- start 2023-05-20T01:18:37.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.956+0000 7f2e6f232700 1 -- start start 2023-05-20T01:18:37.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.957+0000 7f2e6f232700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2e6806aad0 0x7f2e6806dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:37.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.957+0000 7f2e6f232700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2e680b22a0 0x7f2e6806e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:37.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.957+0000 7f2e6f232700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e68161440 con 0x7f2e6806aea0 2023-05-20T01:18:37.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.957+0000 7f2e6f232700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e680b28c0 con 0x7f2e680b22a0 2023-05-20T01:18:37.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.957+0000 7f2e6f232700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e68160680 con 0x7f2e6806aad0 2023-05-20T01:18:37.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.957+0000 7f2e6d7cf700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2e680b22a0 0x7f2e6806e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:37.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.957+0000 7f2e67fff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f2e6806aea0 0x7f2e680b1b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:44544/0 (socket says 172.21.15.16:44544) 2023-05-20T01:18:37.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.957+0000 7f2e67fff700 1 -- 172.21.15.16:0/3012839839 learned_addr learned my addr 172.21.15.16:0/3012839839 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:18:37.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.957+0000 7f2e6cfce700 1 --2- 172.21.15.16:0/3012839839 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2e6806aad0 0x7f2e6806dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:37.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.957+0000 7f2e6d7cf700 1 -- 172.21.15.16:0/3012839839 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2e6806aad0 msgr2=0x7f2e6806dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:18:37.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.957+0000 7f2e6d7cf700 1 --2- 172.21.15.16:0/3012839839 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2e6806aad0 0x7f2e6806dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:37.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.957+0000 7f2e6d7cf700 1 -- 172.21.15.16:0/3012839839 >> v1:172.21.15.16:6789/0 conn(0x7f2e6806aea0 legacy=0x7f2e680b1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:37.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.957+0000 7f2e677fe700 1 -- 172.21.15.16:0/3012839839 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3269800008 0 0) 0x7f2e68161440 con 0x7f2e6806aea0 2023-05-20T01:18:37.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.957+0000 7f2e6d7cf700 1 -- 172.21.15.16:0/3012839839 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e68160960 con 0x7f2e680b22a0 2023-05-20T01:18:37.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.958+0000 7f2e6d7cf700 1 --2- 172.21.15.16:0/3012839839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e680b22a0 0x7f2e6806e1e0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f2e60009d90 tx=0x7f2e6000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2509122f58edf7df server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:37.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.958+0000 7f2e677fe700 1 -- 172.21.15.16:0/3012839839 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e6000e070 con 0x7f2e680b22a0 2023-05-20T01:18:37.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.958+0000 7f2e677fe700 1 -- 172.21.15.16:0/3012839839 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e6000bbd0 con 0x7f2e680b22a0 2023-05-20T01:18:37.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.958+0000 7f2e677fe700 1 -- 172.21.15.16:0/3012839839 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e60012430 con 0x7f2e680b22a0 2023-05-20T01:18:37.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.958+0000 7f2e6f232700 1 -- 172.21.15.16:0/3012839839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e680b22a0 msgr2=0x7f2e6806e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:37.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.958+0000 7f2e6f232700 1 --2- 172.21.15.16:0/3012839839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e680b22a0 0x7f2e6806e1e0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f2e60009d90 tx=0x7f2e6000b330 comp rx=0 tx=0).stop 2023-05-20T01:18:37.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.958+0000 7f2e6f232700 1 -- 172.21.15.16:0/3012839839 shutdown_connections 2023-05-20T01:18:37.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.958+0000 7f2e6f232700 1 --2- 172.21.15.16:0/3012839839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e680b22a0 0x7f2e6806e1e0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:37.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.958+0000 7f2e6f232700 1 --2- 172.21.15.16:0/3012839839 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2e6806aad0 0x7f2e6806dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:37.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.958+0000 7f2e6f232700 1 -- 172.21.15.16:0/3012839839 >> 172.21.15.16:0/3012839839 conn(0x7f2e680ccab0 msgr2=0x7f2e680cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:18:37.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.958+0000 7f2e6f232700 1 -- 172.21.15.16:0/3012839839 shutdown_connections 2023-05-20T01:18:37.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.958+0000 7f2e6f232700 1 -- 172.21.15.16:0/3012839839 wait complete. 2023-05-20T01:18:37.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.959+0000 7f2e6f232700 1 Processor -- start 2023-05-20T01:18:37.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.960+0000 7f2e6f232700 1 -- start start 2023-05-20T01:18:37.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.960+0000 7f2e6f232700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2e6806aad0 0x7f2e680b54a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:37.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.960+0000 7f2e6f232700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2e6806aea0 0x7f2e681f5c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:37.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.960+0000 7f2e6f232700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e680b22a0 0x7f2e681f61a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:37.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.960+0000 7f2e6f232700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e680b2f90 con 0x7f2e680b22a0 2023-05-20T01:18:37.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.960+0000 7f2e6f232700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e680b3270 con 0x7f2e6806aad0 2023-05-20T01:18:37.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.960+0000 7f2e6f232700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e680b3550 con 0x7f2e6806aea0 2023-05-20T01:18:37.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.961+0000 7f2e6d7cf700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e680b22a0 0x7f2e681f61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:37.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.961+0000 7f2e6d7cf700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e680b22a0 0x7f2e681f61a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33024/0 (socket says 172.21.15.16:33024) 2023-05-20T01:18:37.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.961+0000 7f2e6d7cf700 1 -- 172.21.15.16:0/4156555968 learned_addr learned my addr 172.21.15.16:0/4156555968 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:18:37.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.961+0000 7f2e67fff700 1 --2- 172.21.15.16:0/4156555968 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2e6806aea0 0x7f2e681f5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:37.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.961+0000 7f2e6cfce700 1 --2- 172.21.15.16:0/4156555968 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2e6806aad0 0x7f2e680b54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:37.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.961+0000 7f2e6d7cf700 1 -- 172.21.15.16:0/4156555968 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2e6806aea0 msgr2=0x7f2e681f5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:37.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.961+0000 7f2e6d7cf700 1 --2- 172.21.15.16:0/4156555968 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2e6806aea0 0x7f2e681f5c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:37.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.961+0000 7f2e6d7cf700 1 -- 172.21.15.16:0/4156555968 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2e6806aad0 msgr2=0x7f2e680b54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:37.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.961+0000 7f2e6d7cf700 1 --2- 172.21.15.16:0/4156555968 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2e6806aad0 0x7f2e680b54a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:37.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.961+0000 7f2e6d7cf700 1 -- 172.21.15.16:0/4156555968 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e60009a70 con 0x7f2e680b22a0 2023-05-20T01:18:37.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.961+0000 7f2e6cfce700 1 --2- 172.21.15.16:0/4156555968 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2e6806aad0 0x7f2e680b54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:18:37.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.961+0000 7f2e6d7cf700 1 --2- 172.21.15.16:0/4156555968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e680b22a0 0x7f2e681f61a0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f2e68070ee0 tx=0x7f2e60004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:37.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.961+0000 7f2e657fa700 1 -- 172.21.15.16:0/4156555968 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e60012430 con 0x7f2e680b22a0 2023-05-20T01:18:37.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.961+0000 7f2e657fa700 1 -- 172.21.15.16:0/4156555968 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e60020070 con 0x7f2e680b22a0 2023-05-20T01:18:37.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.961+0000 7f2e657fa700 1 -- 172.21.15.16:0/4156555968 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e60004280 con 0x7f2e680b22a0 2023-05-20T01:18:37.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.961+0000 7f2e6f232700 1 -- 172.21.15.16:0/4156555968 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e68160f20 con 0x7f2e680b22a0 2023-05-20T01:18:37.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.961+0000 7f2e6f232700 1 -- 172.21.15.16:0/4156555968 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e68160c40 con 0x7f2e680b22a0 2023-05-20T01:18:37.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.963+0000 7f2e657fa700 1 -- 172.21.15.16:0/4156555968 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2e600043e0 con 0x7f2e680b22a0 2023-05-20T01:18:37.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.963+0000 7f2e657fa700 1 --2- 172.21.15.16:0/4156555968 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f2e4c06ace0 0x7f2e4c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:37.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.963+0000 7f2e657fa700 1 -- 172.21.15.16:0/4156555968 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 7102+0+0 (secure 0 0 0) 0x7f2e60022070 con 0x7f2e680b22a0 2023-05-20T01:18:37.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.963+0000 7f2e6cfce700 1 --2- 172.21.15.16:0/4156555968 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f2e4c06ace0 0x7f2e4c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:37.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.964+0000 7f2e6cfce700 1 --2- 172.21.15.16:0/4156555968 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f2e4c06ace0 0x7f2e4c06d190 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c009a00 tx=0x7f2e5c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:37.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.965+0000 7f2e7b3beb80 1 Processor -- start 2023-05-20T01:18:37.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.965+0000 7f2e7b3beb80 1 -- start start 2023-05-20T01:18:37.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.966+0000 7f2e7b3beb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ed03b6a4d0 0x55ed03b6a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:37.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.966+0000 7f2e7b3beb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed03b667d0 0x55ed03b3f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:37.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.966+0000 7f2e7b3beb80 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ed03b3d940 0x55ed03b3fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:37.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.966+0000 7f2e7b3beb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed03a450e0 con 0x55ed03b667d0 2023-05-20T01:18:37.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.966+0000 7f2e7b3beb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed03a97920 con 0x55ed03b6a4d0 2023-05-20T01:18:37.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.966+0000 7f2e7b3beb80 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ed03a90db0 con 0x55ed03b3d940 2023-05-20T01:18:37.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.966+0000 7f2e67fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed03b667d0 0x55ed03b3f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:37.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.966+0000 7f2e67fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed03b667d0 0x55ed03b3f530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33030/0 (socket says 172.21.15.16:33030) 2023-05-20T01:18:37.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.966+0000 7f2e67fff700 1 -- 172.21.15.16:0/1962396325 learned_addr learned my addr 172.21.15.16:0/1962396325 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:18:37.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.966+0000 7f2e6cfce700 1 --2- 172.21.15.16:0/1962396325 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ed03b6a4d0 0x55ed03b6a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:37.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.966+0000 7f2e67fff700 1 -- 172.21.15.16:0/1962396325 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ed03b3d940 msgr2=0x55ed03b3fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:18:37.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.966+0000 7f2e6d7cf700 1 --2- 172.21.15.16:0/1962396325 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ed03b3d940 0x55ed03b3fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:37.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.966+0000 7f2e67fff700 1 --2- 172.21.15.16:0/1962396325 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ed03b3d940 0x55ed03b3fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:37.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.966+0000 7f2e67fff700 1 -- 172.21.15.16:0/1962396325 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ed03b6a4d0 msgr2=0x55ed03b6a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:18:37.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.966+0000 7f2e67fff700 1 --2- 172.21.15.16:0/1962396325 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ed03b6a4d0 0x55ed03b6a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:37.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.966+0000 7f2e67fff700 1 -- 172.21.15.16:0/1962396325 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ed039126b0 con 0x55ed03b667d0 2023-05-20T01:18:37.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.966+0000 7f2e6cfce700 1 --2- 172.21.15.16:0/1962396325 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ed03b6a4d0 0x55ed03b6a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:18:37.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.966+0000 7f2e6d7cf700 1 --2- 172.21.15.16:0/1962396325 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ed03b3d940 0x55ed03b3fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:18:37.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.966+0000 7f2e67fff700 1 --2- 172.21.15.16:0/1962396325 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed03b667d0 0x55ed03b3f530 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f2e54005cf0 tx=0x7f2e54007050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b02c60bba256aa99 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:37.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.967+0000 7f2e527fc700 1 -- 172.21.15.16:0/1962396325 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e54007720 con 0x55ed03b667d0 2023-05-20T01:18:37.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.967+0000 7f2e527fc700 1 -- 172.21.15.16:0/1962396325 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e540014c0 con 0x55ed03b667d0 2023-05-20T01:18:37.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.967+0000 7f2e527fc700 1 -- 172.21.15.16:0/1962396325 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e54001870 con 0x55ed03b667d0 2023-05-20T01:18:37.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.967+0000 7f2e7b3beb80 1 -- 172.21.15.16:0/1962396325 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed03b667d0 msgr2=0x55ed03b3f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:37.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.967+0000 7f2e7b3beb80 1 --2- 172.21.15.16:0/1962396325 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed03b667d0 0x55ed03b3f530 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f2e54005cf0 tx=0x7f2e54007050 comp rx=0 tx=0).stop 2023-05-20T01:18:37.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.967+0000 7f2e7b3beb80 1 -- 172.21.15.16:0/1962396325 shutdown_connections 2023-05-20T01:18:37.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.967+0000 7f2e7b3beb80 1 --2- 172.21.15.16:0/1962396325 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ed03b3d940 0x55ed03b3fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:37.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.967+0000 7f2e7b3beb80 1 --2- 172.21.15.16:0/1962396325 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ed03b6a4d0 0x55ed03b6a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:37.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.967+0000 7f2e7b3beb80 1 --2- 172.21.15.16:0/1962396325 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed03b667d0 0x55ed03b3f530 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:37.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.967+0000 7f2e7b3beb80 1 -- 172.21.15.16:0/1962396325 >> 172.21.15.16:0/1962396325 conn(0x55ed03b35000 msgr2=0x55ed03b852e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:18:37.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.967+0000 7f2e7b3beb80 1 -- 172.21.15.16:0/1962396325 shutdown_connections 2023-05-20T01:18:37.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.967+0000 7f2e7b3beb80 1 -- 172.21.15.16:0/1962396325 wait complete. 2023-05-20T01:18:37.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.968+0000 7f2e7b3beb80 1 -- start start 2023-05-20T01:18:37.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.968+0000 7f2e7b3beb80 1 Processor -- start 2023-05-20T01:18:37.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.969+0000 7f2e7b3beb80 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ed03b3d940 0x55ed03b3f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:37.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.969+0000 7f2e7b3beb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed03b667d0 0x55ed03b61ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:37.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.969+0000 7f2e7b3beb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ed03b6a4d0 0x55ed03b62530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:37.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.969+0000 7f2e7b3beb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed03a32960 con 0x55ed03b667d0 2023-05-20T01:18:37.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.969+0000 7f2e7b3beb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed03a32600 con 0x55ed03b6a4d0 2023-05-20T01:18:37.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.969+0000 7f2e7b3beb80 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ed039149a0 con 0x55ed03b3d940 2023-05-20T01:18:37.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.969+0000 7f2e7b3beb80 10 client.0 _ll_register_callbacks cb 0x55ed03a50800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-05-20T01:18:37.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.969+0000 7f2e67fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed03b667d0 0x55ed03b61ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:37.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.969+0000 7f2e67fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed03b667d0 0x55ed03b61ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33036/0 (socket says 172.21.15.16:33036) 2023-05-20T01:18:37.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.969+0000 7f2e67fff700 1 -- 172.21.15.16:0/2392974527 learned_addr learned my addr 172.21.15.16:0/2392974527 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:18:37.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.969+0000 7f2e6d7cf700 1 --2- 172.21.15.16:0/2392974527 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ed03b6a4d0 0x55ed03b62530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:37.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.969+0000 7f2e6cfce700 1 --2- 172.21.15.16:0/2392974527 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ed03b3d940 0x55ed03b3f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:37.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.969+0000 7f2e67fff700 1 -- 172.21.15.16:0/2392974527 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ed03b3d940 msgr2=0x55ed03b3f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:18:37.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.969+0000 7f2e67fff700 1 --2- 172.21.15.16:0/2392974527 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ed03b3d940 0x55ed03b3f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:37.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.969+0000 7f2e67fff700 1 -- 172.21.15.16:0/2392974527 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ed03b6a4d0 msgr2=0x55ed03b62530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:18:37.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.969+0000 7f2e67fff700 1 --2- 172.21.15.16:0/2392974527 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ed03b6a4d0 0x55ed03b62530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:37.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.969+0000 7f2e67fff700 1 -- 172.21.15.16:0/2392974527 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e5400f040 con 0x55ed03b667d0 2023-05-20T01:18:37.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.970+0000 7f2e67fff700 1 --2- 172.21.15.16:0/2392974527 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed03b667d0 0x55ed03b61ff0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f2e54007090 tx=0x7f2e54000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca1eccf32fcaf70e server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:37.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.970+0000 7f2e477fe700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-05-20T01:18:37.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.970+0000 7f2e477fe700 1 -- 172.21.15.16:0/2392974527 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e54007720 con 0x55ed03b667d0 2023-05-20T01:18:37.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.970+0000 7f2e477fe700 1 -- 172.21.15.16:0/2392974527 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(51..51 src has 1..51) v4 ==== 7102+0+0 (secure 0 0 0) 0x7f2e54001d20 con 0x55ed03b667d0 2023-05-20T01:18:37.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.970+0000 7f2e7b3beb80 1 -- 172.21.15.16:0/2392974527 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ed03b327c0 con 0x55ed03b667d0 2023-05-20T01:18:37.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.970+0000 7f2e477fe700 1 -- 172.21.15.16:0/2392974527 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2e54014480 con 0x55ed03b667d0 2023-05-20T01:18:37.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.970+0000 7f2e7b3beb80 10 client.5964 fetch_fsmap learned FSMap version 29 2023-05-20T01:18:37.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.970+0000 7f2e7b3beb80 1 -- 172.21.15.16:0/2392974527 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) v3 -- 0x55ed03a57210 con 0x55ed03b667d0 2023-05-20T01:18:37.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.970+0000 7f2e477fe700 1 -- 172.21.15.16:0/2392974527 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ed03b327c0 con 0x55ed03b667d0 2023-05-20T01:18:37.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.970+0000 7f2e477fe700 1 -- 172.21.15.16:0/2392974527 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 29) v1 ==== 3122+0+0 (secure 0 0 0) 0x7f2e540145e0 con 0x55ed03b667d0 2023-05-20T01:18:37.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.970+0000 7f2e7b3beb80 10 client.5964 fetch_fsmap finished waiting for FSMap version 29 2023-05-20T01:18:37.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.970+0000 7f2e7b3beb80 10 client.5964 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2023-05-20T01:18:37.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.970+0000 7f2e7b3beb80 20 client.5964 populate_metadata read hostname 'smithi016' 2023-05-20T01:18:37.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.970+0000 7f2e7b3beb80 1 --2- 172.21.15.16:0/2392974527 >> [v2:172.21.15.16:6834/3284962389,v1:172.21.15.16:6836/3284962389] conn(0x55ed03b3ddb0 0x55ed03b3e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:37.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.970+0000 7f2e7b3beb80 4 client.5964 mds_command: new command op to 5889 tid=0[{"prefix": "get_command_descriptions"}] 2023-05-20T01:18:37.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.970+0000 7f2e7b3beb80 1 -- 172.21.15.16:0/2392974527 --> [v2:172.21.15.16:6834/3284962389,v1:172.21.15.16:6836/3284962389] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ed03a53120 con 0x55ed03b3ddb0 2023-05-20T01:18:37.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.970+0000 7f2e6cfce700 1 --2- 172.21.15.16:0/2392974527 >> [v2:172.21.15.16:6834/3284962389,v1:172.21.15.16:6836/3284962389] conn(0x55ed03b3ddb0 0x55ed03b3e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:37.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.971+0000 7f2e477fe700 10 client.5964 ms_handle_connect on v2:172.21.15.16:6834/3284962389 2023-05-20T01:18:37.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.971+0000 7f2e6cfce700 1 --2- 172.21.15.16:0/2392974527 >> [v2:172.21.15.16:6834/3284962389,v1:172.21.15.16:6836/3284962389] conn(0x55ed03b3ddb0 0x55ed03b3e180 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f0a847cfad7fd47 server_cookie=50087b48f98b56cb in_seq=0 out_seq=0 2023-05-20T01:18:37.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.971+0000 7f2e477fe700 1 -- 172.21.15.16:0/2392974527 <== mds.0 v2:172.21.15.16:6834/3284962389 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ed03a53120 con 0x55ed03b3ddb0 2023-05-20T01:18:37.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.971+0000 7f2e477fe700 10 client.5964 handle_command_reply: tid=0 2023-05-20T01:18:37.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.971+0000 7f2e7b3beb80 1 client.5964 shutdown 2023-05-20T01:18:37.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.971+0000 7f2e7b3beb80 1 -- 172.21.15.16:0/2392974527 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed03b667d0 msgr2=0x55ed03b61ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:37.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.971+0000 7f2e7b3beb80 1 --2- 172.21.15.16:0/2392974527 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed03b667d0 0x55ed03b61ff0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f2e54007090 tx=0x7f2e54000ed0 comp rx=0 tx=0).stop 2023-05-20T01:18:37.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.971+0000 7f2e7b3beb80 1 -- 172.21.15.16:0/2392974527 shutdown_connections 2023-05-20T01:18:37.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.971+0000 7f2e7b3beb80 1 --2- 172.21.15.16:0/2392974527 >> [v2:172.21.15.16:6834/3284962389,v1:172.21.15.16:6836/3284962389] conn(0x55ed03b3ddb0 0x55ed03b3e180 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:37.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.971+0000 7f2e7b3beb80 1 --2- 172.21.15.16:0/2392974527 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ed03b6a4d0 0x55ed03b62530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:37.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.971+0000 7f2e7b3beb80 1 --2- 172.21.15.16:0/2392974527 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ed03b3d940 0x55ed03b3f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:37.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.971+0000 7f2e7b3beb80 1 --2- 172.21.15.16:0/2392974527 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed03b667d0 0x55ed03b61ff0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:37.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.971+0000 7f2e477fe700 0 client.5964 ms_handle_reset on v2:172.21.15.16:6834/3284962389 2023-05-20T01:18:37.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.972+0000 7f2e7b3beb80 1 -- 172.21.15.16:0/2392974527 >> 172.21.15.16:0/2392974527 conn(0x55ed03b35000 msgr2=0x55ed03b3b770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:18:37.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.972+0000 7f2e7b3beb80 1 -- 172.21.15.16:0/2392974527 shutdown_connections 2023-05-20T01:18:37.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.972+0000 7f2e7b3beb80 1 -- 172.21.15.16:0/2392974527 wait complete. 2023-05-20T01:18:37.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.972+0000 7f2e7b3beb80 20 client.5964 trim_cache size 0 max 16384 2023-05-20T01:18:37.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.989+0000 7f2e7b3beb80 1 Processor -- start 2023-05-20T01:18:37.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.989+0000 7f2e7b3beb80 1 -- start start 2023-05-20T01:18:37.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.989+0000 7f2e7b3beb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed03b3d940 0x55ed03b39fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:37.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.989+0000 7f2e7b3beb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ed03b667d0 0x55ed03ba7130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:37.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.989+0000 7f2e7b3beb80 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ed03b6a4d0 0x55ed03b623a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:37.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.989+0000 7f2e7b3beb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed03b3f620 con 0x55ed03b3d940 2023-05-20T01:18:37.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.989+0000 7f2e7b3beb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed03b87390 con 0x55ed03b667d0 2023-05-20T01:18:37.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.989+0000 7f2e7b3beb80 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ed03b3d1f0 con 0x55ed03b6a4d0 2023-05-20T01:18:37.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.989+0000 7f2e6cfce700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed03b3d940 0x55ed03b39fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:37.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.990+0000 7f2e6cfce700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed03b3d940 0x55ed03b39fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33042/0 (socket says 172.21.15.16:33042) 2023-05-20T01:18:37.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.990+0000 7f2e6cfce700 1 -- 172.21.15.16:0/2505434169 learned_addr learned my addr 172.21.15.16:0/2505434169 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:18:37.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.990+0000 7f2e6d7cf700 1 --2- 172.21.15.16:0/2505434169 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ed03b6a4d0 0x55ed03b623a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:37.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.990+0000 7f2e67fff700 1 --2- 172.21.15.16:0/2505434169 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ed03b667d0 0x55ed03ba7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:37.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.990+0000 7f2e6cfce700 1 -- 172.21.15.16:0/2505434169 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ed03b6a4d0 msgr2=0x55ed03b623a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:18:37.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.990+0000 7f2e6cfce700 1 --2- 172.21.15.16:0/2505434169 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ed03b6a4d0 0x55ed03b623a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:37.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.990+0000 7f2e6cfce700 1 -- 172.21.15.16:0/2505434169 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ed03b667d0 msgr2=0x55ed03ba7130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:18:37.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.990+0000 7f2e6cfce700 1 --2- 172.21.15.16:0/2505434169 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ed03b667d0 0x55ed03ba7130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:37.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.990+0000 7f2e6cfce700 1 -- 172.21.15.16:0/2505434169 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ed03a56f60 con 0x55ed03b3d940 2023-05-20T01:18:37.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.990+0000 7f2e67fff700 1 --2- 172.21.15.16:0/2505434169 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ed03b667d0 0x55ed03ba7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:18:37.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.990+0000 7f2e6d7cf700 1 --2- 172.21.15.16:0/2505434169 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ed03b6a4d0 0x55ed03b623a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:18:37.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.990+0000 7f2e6cfce700 1 --2- 172.21.15.16:0/2505434169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed03b3d940 0x55ed03b39fe0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x55ed03b40280 tx=0x7f2e5c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e33e8b8cb1a5e4c server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:37.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.990+0000 7f2e52ffd700 1 -- 172.21.15.16:0/2505434169 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e5c02c070 con 0x55ed03b3d940 2023-05-20T01:18:37.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.990+0000 7f2e52ffd700 1 -- 172.21.15.16:0/2505434169 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e5c01c040 con 0x55ed03b3d940 2023-05-20T01:18:37.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.990+0000 7f2e52ffd700 1 -- 172.21.15.16:0/2505434169 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e5c027150 con 0x55ed03b3d940 2023-05-20T01:18:38.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.991+0000 7f2e7b3beb80 1 -- 172.21.15.16:0/2505434169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed03b3d940 msgr2=0x55ed03b39fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:38.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.991+0000 7f2e7b3beb80 1 --2- 172.21.15.16:0/2505434169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed03b3d940 0x55ed03b39fe0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x55ed03b40280 tx=0x7f2e5c015000 comp rx=0 tx=0).stop 2023-05-20T01:18:38.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.991+0000 7f2e7b3beb80 1 -- 172.21.15.16:0/2505434169 shutdown_connections 2023-05-20T01:18:38.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.991+0000 7f2e7b3beb80 1 --2- 172.21.15.16:0/2505434169 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ed03b6a4d0 0x55ed03b623a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:38.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.991+0000 7f2e7b3beb80 1 --2- 172.21.15.16:0/2505434169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed03b3d940 0x55ed03b39fe0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:38.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.991+0000 7f2e7b3beb80 1 --2- 172.21.15.16:0/2505434169 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ed03b667d0 0x55ed03ba7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:38.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.991+0000 7f2e7b3beb80 1 -- 172.21.15.16:0/2505434169 >> 172.21.15.16:0/2505434169 conn(0x55ed03b35000 msgr2=0x55ed03b85280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:18:38.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.991+0000 7f2e7b3beb80 1 -- 172.21.15.16:0/2505434169 shutdown_connections 2023-05-20T01:18:38.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.991+0000 7f2e7b3beb80 1 -- 172.21.15.16:0/2505434169 wait complete. 2023-05-20T01:18:38.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.991+0000 7f2e7b3beb80 1 -- start start 2023-05-20T01:18:38.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.992+0000 7f2e7b3beb80 1 Processor -- start 2023-05-20T01:18:38.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.992+0000 7f2e7b3beb80 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ed03b3d940 0x55ed03be2890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:38.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.992+0000 7f2e7b3beb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ed03b667d0 0x55ed03b62410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:38.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.992+0000 7f2e7b3beb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed03b6a4d0 0x55ed03b7ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:38.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.992+0000 7f2e7b3beb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed03bed880 con 0x55ed03b6a4d0 2023-05-20T01:18:38.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.992+0000 7f2e7b3beb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed03b3edb0 con 0x55ed03b667d0 2023-05-20T01:18:38.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.992+0000 7f2e7b3beb80 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ed03b6d530 con 0x55ed03b3d940 2023-05-20T01:18:38.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.992+0000 7f2e7b3beb80 10 client.0 _ll_register_callbacks cb 0x7f2e54017070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-05-20T01:18:38.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.992+0000 7f2e6d7cf700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed03b6a4d0 0x55ed03b7ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:38.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.992+0000 7f2e6d7cf700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed03b6a4d0 0x55ed03b7ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33052/0 (socket says 172.21.15.16:33052) 2023-05-20T01:18:38.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.992+0000 7f2e6d7cf700 1 -- 172.21.15.16:0/1991180810 learned_addr learned my addr 172.21.15.16:0/1991180810 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:18:38.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.993+0000 7f2e6cfce700 1 --2- 172.21.15.16:0/1991180810 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ed03b3d940 0x55ed03be2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:38.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.993+0000 7f2e67fff700 1 --2- 172.21.15.16:0/1991180810 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ed03b667d0 0x55ed03b62410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:38.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.993+0000 7f2e6d7cf700 1 -- 172.21.15.16:0/1991180810 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ed03b3d940 msgr2=0x55ed03be2890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:18:38.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.993+0000 7f2e6d7cf700 1 --2- 172.21.15.16:0/1991180810 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ed03b3d940 0x55ed03be2890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:38.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.993+0000 7f2e6d7cf700 1 -- 172.21.15.16:0/1991180810 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ed03b667d0 msgr2=0x55ed03b62410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:18:38.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.993+0000 7f2e6d7cf700 1 --2- 172.21.15.16:0/1991180810 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ed03b667d0 0x55ed03b62410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:38.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.993+0000 7f2e6d7cf700 1 -- 172.21.15.16:0/1991180810 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e5c00e040 con 0x55ed03b6a4d0 2023-05-20T01:18:38.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.993+0000 7f2e67fff700 1 --2- 172.21.15.16:0/1991180810 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ed03b667d0 0x55ed03b62410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:18:38.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.993+0000 7f2e6d7cf700 1 --2- 172.21.15.16:0/1991180810 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed03b6a4d0 0x55ed03b7ac70 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f2e60009a40 tx=0x7f2e6005e490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eec2e98841dbe05b server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:38.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.993+0000 7f2e477fe700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-05-20T01:18:38.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.993+0000 7f2e477fe700 1 -- 172.21.15.16:0/1991180810 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e60059b40 con 0x55ed03b6a4d0 2023-05-20T01:18:38.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.993+0000 7f2e477fe700 1 -- 172.21.15.16:0/1991180810 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(51..51 src has 1..51) v4 ==== 7102+0+0 (secure 0 0 0) 0x7f2e60089e40 con 0x55ed03b6a4d0 2023-05-20T01:18:38.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.993+0000 7f2e7b3beb80 1 -- 172.21.15.16:0/1991180810 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2e540014c0 con 0x55ed03b6a4d0 2023-05-20T01:18:38.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.993+0000 7f2e477fe700 1 -- 172.21.15.16:0/1991180810 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2e600588b0 con 0x55ed03b6a4d0 2023-05-20T01:18:38.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.993+0000 7f2e7b3beb80 10 client.5970 fetch_fsmap learned FSMap version 29 2023-05-20T01:18:38.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.993+0000 7f2e7b3beb80 1 -- 172.21.15.16:0/1991180810 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) v3 -- 0x55ed038a7ce0 con 0x55ed03b6a4d0 2023-05-20T01:18:38.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.993+0000 7f2e477fe700 1 -- 172.21.15.16:0/1991180810 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e540014c0 con 0x55ed03b6a4d0 2023-05-20T01:18:38.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.993+0000 7f2e477fe700 1 -- 172.21.15.16:0/1991180810 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 29) v1 ==== 3122+0+0 (secure 0 0 0) 0x7f2e60062310 con 0x55ed03b6a4d0 2023-05-20T01:18:38.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.993+0000 7f2e7b3beb80 10 client.5970 fetch_fsmap finished waiting for FSMap version 29 2023-05-20T01:18:38.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.994+0000 7f2e7b3beb80 10 client.5970 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2023-05-20T01:18:38.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.994+0000 7f2e7b3beb80 20 client.5970 populate_metadata read hostname 'smithi016' 2023-05-20T01:18:38.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.994+0000 7f2e7b3beb80 1 --2- 172.21.15.16:0/1991180810 >> [v2:172.21.15.16:6834/3284962389,v1:172.21.15.16:6836/3284962389] conn(0x55ed03b3e890 0x55ed03b7b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:38.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.994+0000 7f2e7b3beb80 4 client.5970 mds_command: new command op to 5889 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2023-05-20T01:18:38.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.994+0000 7f2e7b3beb80 1 -- 172.21.15.16:0/1991180810 --> [v2:172.21.15.16:6834/3284962389,v1:172.21.15.16:6836/3284962389] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) v1 -- 0x55ed03a5f6e0 con 0x55ed03b3e890 2023-05-20T01:18:38.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.994+0000 7f2e6cfce700 1 --2- 172.21.15.16:0/1991180810 >> [v2:172.21.15.16:6834/3284962389,v1:172.21.15.16:6836/3284962389] conn(0x55ed03b3e890 0x55ed03b7b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:38.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.994+0000 7f2e6cfce700 1 --2- 172.21.15.16:0/1991180810 >> [v2:172.21.15.16:6834/3284962389,v1:172.21.15.16:6836/3284962389] conn(0x55ed03b3e890 0x55ed03b7b1b0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81cd2bff82d4c70b server_cookie=491d1b3c875b2695 in_seq=0 out_seq=0 2023-05-20T01:18:38.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.994+0000 7f2e477fe700 10 client.5970 ms_handle_connect on v2:172.21.15.16:6834/3284962389 2023-05-20T01:18:38.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.995+0000 7f2e477fe700 1 -- 172.21.15.16:0/1991180810 <== mds.0 v2:172.21.15.16:6834/3284962389 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55ed03a5f6e0 con 0x55ed03b3e890 2023-05-20T01:18:38.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.995+0000 7f2e477fe700 10 client.5970 handle_command_reply: tid=0 2023-05-20T01:18:38.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.995+0000 7f2e7b3beb80 1 client.5970 shutdown 2023-05-20T01:18:38.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.995+0000 7f2e7b3beb80 1 -- 172.21.15.16:0/1991180810 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed03b6a4d0 msgr2=0x55ed03b7ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:38.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.995+0000 7f2e7b3beb80 1 --2- 172.21.15.16:0/1991180810 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed03b6a4d0 0x55ed03b7ac70 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f2e60009a40 tx=0x7f2e6005e490 comp rx=0 tx=0).stop 2023-05-20T01:18:38.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.995+0000 7f2e7b3beb80 1 -- 172.21.15.16:0/1991180810 shutdown_connections 2023-05-20T01:18:38.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.995+0000 7f2e7b3beb80 1 --2- 172.21.15.16:0/1991180810 >> [v2:172.21.15.16:6834/3284962389,v1:172.21.15.16:6836/3284962389] conn(0x55ed03b3e890 0x55ed03b7b1b0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:38.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.995+0000 7f2e7b3beb80 1 --2- 172.21.15.16:0/1991180810 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed03b6a4d0 0x55ed03b7ac70 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:38.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.995+0000 7f2e7b3beb80 1 --2- 172.21.15.16:0/1991180810 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ed03b3d940 0x55ed03be2890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:38.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.995+0000 7f2e477fe700 0 client.5970 ms_handle_reset on v2:172.21.15.16:6834/3284962389 2023-05-20T01:18:38.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.995+0000 7f2e7b3beb80 1 --2- 172.21.15.16:0/1991180810 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ed03b667d0 0x55ed03b62410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:38.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.995+0000 7f2e7b3beb80 1 -- 172.21.15.16:0/1991180810 >> 172.21.15.16:0/1991180810 conn(0x55ed03b35000 msgr2=0x55ed03b3e180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:18:38.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.995+0000 7f2e7b3beb80 1 -- 172.21.15.16:0/1991180810 shutdown_connections 2023-05-20T01:18:38.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.995+0000 7f2e7b3beb80 1 -- 172.21.15.16:0/1991180810 wait complete. 2023-05-20T01:18:38.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.996+0000 7f2e7b3beb80 20 client.5970 trim_cache size 0 max 16384 2023-05-20T01:18:38.008 INFO:tasks.ceph.mds.c.smithi016.stderr:2023-05-20T01:18:37.996+0000 7f44af702700 -1 log_channel(cluster) log [ERR] : bad backtrace on directory inode 0x10000000000 2023-05-20T01:18:38.009 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-05-20T01:18:38.009 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2023-05-20T01:18:38.009 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "610f824c-6e63-4d09-bf34-36eb92bf457a", 2023-05-20T01:18:38.009 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2023-05-20T01:18:38.009 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-05-20T01:18:38.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.996+0000 7f2e6f232700 1 -- 172.21.15.16:0/4156555968 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f2e4c06ace0 msgr2=0x7f2e4c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:38.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.996+0000 7f2e6f232700 1 --2- 172.21.15.16:0/4156555968 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f2e4c06ace0 0x7f2e4c06d190 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c009a00 tx=0x7f2e5c008040 comp rx=0 tx=0).stop 2023-05-20T01:18:38.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.996+0000 7f2e6f232700 1 -- 172.21.15.16:0/4156555968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e680b22a0 msgr2=0x7f2e681f61a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:38.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.996+0000 7f2e6f232700 1 --2- 172.21.15.16:0/4156555968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e680b22a0 0x7f2e681f61a0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f2e68070ee0 tx=0x7f2e60004ad0 comp rx=0 tx=0).stop 2023-05-20T01:18:38.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.997+0000 7f2e6f232700 1 -- 172.21.15.16:0/4156555968 shutdown_connections 2023-05-20T01:18:38.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.997+0000 7f2e6f232700 1 --2- 172.21.15.16:0/4156555968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e680b22a0 0x7f2e681f61a0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:38.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.997+0000 7f2e6f232700 1 --2- 172.21.15.16:0/4156555968 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2e6806aad0 0x7f2e680b54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:38.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.997+0000 7f2e6f232700 1 --2- 172.21.15.16:0/4156555968 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f2e4c06ace0 0x7f2e4c06d190 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:38.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.997+0000 7f2e6f232700 1 --2- 172.21.15.16:0/4156555968 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2e6806aea0 0x7f2e681f5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:38.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.997+0000 7f2e6f232700 1 -- 172.21.15.16:0/4156555968 >> 172.21.15.16:0/4156555968 conn(0x7f2e680ccab0 msgr2=0x7f2e680d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:18:38.011 INFO:tasks.ceph.mds.c.smithi016.stderr:2023-05-20T01:18:37.996+0000 7f44af702700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /subdir/ auth v1 pv4 ap=3 DIRTYPARENT f(v0 1=1+0) n() (inest lock dirty) (ifile lock->sync w=1 flushing) (iversion lock) | dirtyscattered=2 lock=1 dirfrag=1 dirtyparent=1 waiter=1 authpin=1 scrubqueue=0 0x555dabc65600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(6)0x10000000000:[<0x1/subdir v1>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 1=1+0)","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 1=1+0)","memory_value.rstat":"n()","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-61} 2023-05-20T01:18:38.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.997+0000 7f2e6f232700 1 -- 172.21.15.16:0/4156555968 shutdown_connections 2023-05-20T01:18:38.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:37.997+0000 7f2e6f232700 1 -- 172.21.15.16:0/4156555968 wait complete. 2023-05-20T01:18:38.012 INFO:tasks.ceph.mds.c.smithi016.stderr:2023-05-20T01:18:37.997+0000 7f44af702700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001f7 [2,head] /subdir/2 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x555dabd5a580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(8)0x100000001f7:[<0x10000000000/2 v46>,<0x1/subdir v56>]//[7]","memoryvalue":"(8)0x100000001f7:[<0x10000000000/2 v1>,<0x1/subdir v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-05-20T01:18:38.013 INFO:tasks.ceph.mds.c.smithi016.stderr:2023-05-20T01:18:37.997+0000 7f44af702700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001f9 [2,head] /subdir/4 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x555dabcb3700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(8)0x100000001f9:[<0x10000000000/4 v50>,<0x1/subdir v56>]//[7]","memoryvalue":"(8)0x100000001f9:[<0x10000000000/4 v1>,<0x1/subdir v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-05-20T01:18:38.013 INFO:tasks.ceph.mds.c.smithi016.stderr:2023-05-20T01:18:37.997+0000 7f44af702700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001f8 [2,head] /subdir/3 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x555dabd5a000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(8)0x100000001f8:[<0x10000000000/3 v48>,<0x1/subdir v56>]//[7]","memoryvalue":"(8)0x100000001f8:[<0x10000000000/3 v1>,<0x1/subdir v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-05-20T01:18:38.013 INFO:tasks.ceph.mds.c.smithi016.stderr:2023-05-20T01:18:37.998+0000 7f44af702700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001f6 [2,head] /subdir/1 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x555dabd5ab00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(8)0x100000001f6:[<0x10000000000/1 v44>,<0x1/subdir v56>]//[7]","memoryvalue":"(8)0x100000001f6:[<0x10000000000/1 v1>,<0x1/subdir v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-05-20T01:18:38.013 INFO:tasks.ceph.mds.c.smithi016.stderr:2023-05-20T01:18:37.998+0000 7f44af702700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /subdir/0 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x555dabd5b080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(8)0x10000000001:[<0x10000000000/0 v52>,<0x1/subdir v56>]//[7]","memoryvalue":"(8)0x10000000001:[<0x10000000000/0 v1>,<0x1/subdir v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-05-20T01:18:38.013 INFO:tasks.ceph.mds.c.smithi016.stderr:2023-05-20T01:18:38.003+0000 7f44af702700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1 [...2,head] / auth v5 ap=5 snaprealm=0x555dabc5efc0 DIRTYPARENT f(v1 1=0+1) n(v0 rc2023-05-20T01:18:37.648539+0000 1=0+1)/n() (idft sync r=1) (inest sync r=1) (ifile sync->mix r=1) (iversion lock) caps={5949=pLsXs/-@45} | dirtyscattered=0 request=0 lock=3 dirfrag=1 caps=1 openingsnapparents=0 dirtyparent=1 dirty=1 waiter=1 authpin=1 scrubqueue=0 0x555dabc65080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(6)0x1:[]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 1=0+1)","ondisk_value.rstat":"n(v0 2=0+2)","memory_value.dirstat":"f(v1 1=0+1)","memory_value.rstat":"n(v0 rc2023-05-20T01:18:37.648539+0000 1=0+1)","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-61} 2023-05-20T01:18:38.014 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 scrub status 2023-05-20T01:18:38.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.137+0000 7fb7d18b3700 1 Processor -- start 2023-05-20T01:18:38.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.138+0000 7fb7d18b3700 1 -- start start 2023-05-20T01:18:38.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.138+0000 7fb7d18b3700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fb7cc0cec80 0x7fb7cc0cf0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:38.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.138+0000 7fb7d18b3700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb7cc0cc110 0x7fb7cc0cc550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:38.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.138+0000 7fb7d18b3700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7cc0ce4d0 con 0x7fb7cc15ab80 2023-05-20T01:18:38.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.138+0000 7fb7d18b3700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7cc0cb700 con 0x7fb7cc0cc110 2023-05-20T01:18:38.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.138+0000 7fb7d18b3700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7cc0cf630 con 0x7fb7cc0cec80 2023-05-20T01:18:38.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.138+0000 7fb7cb7fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb7cc0cc110 0x7fb7cc0cc550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:38.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.138+0000 7fb7cb7fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb7cc0cc110 0x7fb7cc0cc550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33062/0 (socket says 172.21.15.16:33062) 2023-05-20T01:18:38.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.139+0000 7fb7cb7fe700 1 -- 172.21.15.16:0/4093834898 learned_addr learned my addr 172.21.15.16:0/4093834898 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:18:38.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.139+0000 7fb7ca7fc700 1 --2- 172.21.15.16:0/4093834898 >> v2:172.21.15.132:3300/0 conn(0x7fb7cc0cec80 0x7fb7cc0cf0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:38.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.139+0000 7fb7cb7fe700 1 -- 172.21.15.16:0/4093834898 >> v2:172.21.15.132:3300/0 conn(0x7fb7cc0cec80 msgr2=0x7fb7cc0cf0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:18:38.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.139+0000 7fb7cb7fe700 1 --2- 172.21.15.16:0/4093834898 >> v2:172.21.15.132:3300/0 conn(0x7fb7cc0cec80 0x7fb7cc0cf0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:38.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.139+0000 7fb7cb7fe700 1 -- 172.21.15.16:0/4093834898 >> v1:172.21.15.132:6789/0 conn(0x7fb7cc15ab80 legacy=0x7fb7cc0cda80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:18:38.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.139+0000 7fb7cb7fe700 1 -- 172.21.15.16:0/4093834898 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7cc0ce7b0 con 0x7fb7cc0cc110 2023-05-20T01:18:38.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.139+0000 7fb7cb7fe700 1 --2- 172.21.15.16:0/4093834898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7cc0cc110 0x7fb7cc0cc550 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc0b3bb0 tx=0x7fb7c400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d3c5b0c560a16c1 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:38.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.139+0000 7fb7c9ffb700 1 -- 172.21.15.16:0/4093834898 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7c400e070 con 0x7fb7cc0cc110 2023-05-20T01:18:38.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.139+0000 7fb7c9ffb700 1 -- 172.21.15.16:0/4093834898 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7c4003a20 con 0x7fb7cc0cc110 2023-05-20T01:18:38.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.139+0000 7fb7c9ffb700 1 -- 172.21.15.16:0/4093834898 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7c4003dd0 con 0x7fb7cc0cc110 2023-05-20T01:18:38.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.140+0000 7fb7d18b3700 1 -- 172.21.15.16:0/4093834898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7cc0cc110 msgr2=0x7fb7cc0cc550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:38.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.140+0000 7fb7d18b3700 1 --2- 172.21.15.16:0/4093834898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7cc0cc110 0x7fb7cc0cc550 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc0b3bb0 tx=0x7fb7c400b330 comp rx=0 tx=0).stop 2023-05-20T01:18:38.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.140+0000 7fb7d18b3700 1 -- 172.21.15.16:0/4093834898 shutdown_connections 2023-05-20T01:18:38.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.140+0000 7fb7d18b3700 1 --2- 172.21.15.16:0/4093834898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7cc0cc110 0x7fb7cc0cc550 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:38.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.140+0000 7fb7d18b3700 1 --2- 172.21.15.16:0/4093834898 >> v2:172.21.15.132:3300/0 conn(0x7fb7cc0cec80 0x7fb7cc0cf0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:38.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.140+0000 7fb7d18b3700 1 -- 172.21.15.16:0/4093834898 >> 172.21.15.16:0/4093834898 conn(0x7fb7cc06ae30 msgr2=0x7fb7cc09a400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:18:38.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.140+0000 7fb7d18b3700 1 -- 172.21.15.16:0/4093834898 shutdown_connections 2023-05-20T01:18:38.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.140+0000 7fb7d18b3700 1 -- 172.21.15.16:0/4093834898 wait complete. 2023-05-20T01:18:38.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.141+0000 7fb7d18b3700 1 Processor -- start 2023-05-20T01:18:38.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.142+0000 7fb7d18b3700 1 -- start start 2023-05-20T01:18:38.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.142+0000 7fb7d18b3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7cc0cc110 0x7fb7cc155600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:38.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.142+0000 7fb7d18b3700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb7cc0cec80 0x7fb7cc06cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:38.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.142+0000 7fb7d18b3700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb7cc15ab80 0x7fb7cc0c6db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:38.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.142+0000 7fb7d18b3700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7cc155b40 con 0x7fb7cc0cc110 2023-05-20T01:18:38.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.142+0000 7fb7d18b3700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7cc0b4e80 con 0x7fb7cc0cec80 2023-05-20T01:18:38.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.142+0000 7fb7d18b3700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7cc06d600 con 0x7fb7cc15ab80 2023-05-20T01:18:38.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.143+0000 7fb7caffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7cc0cc110 0x7fb7cc155600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:38.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.143+0000 7fb7caffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7cc0cc110 0x7fb7cc155600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33064/0 (socket says 172.21.15.16:33064) 2023-05-20T01:18:38.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.143+0000 7fb7caffd700 1 -- 172.21.15.16:0/3972297192 learned_addr learned my addr 172.21.15.16:0/3972297192 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:18:38.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.143+0000 7fb7ca7fc700 1 --2- 172.21.15.16:0/3972297192 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb7cc0cec80 0x7fb7cc06cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:38.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.143+0000 7fb7cb7fe700 1 --2- 172.21.15.16:0/3972297192 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb7cc15ab80 0x7fb7cc0c6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:38.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.143+0000 7fb7caffd700 1 -- 172.21.15.16:0/3972297192 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb7cc15ab80 msgr2=0x7fb7cc0c6db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:38.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.143+0000 7fb7caffd700 1 --2- 172.21.15.16:0/3972297192 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb7cc15ab80 0x7fb7cc0c6db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:38.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.143+0000 7fb7caffd700 1 -- 172.21.15.16:0/3972297192 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb7cc0cec80 msgr2=0x7fb7cc06cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:38.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.143+0000 7fb7caffd700 1 --2- 172.21.15.16:0/3972297192 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb7cc0cec80 0x7fb7cc06cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:38.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.143+0000 7fb7caffd700 1 -- 172.21.15.16:0/3972297192 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7c4009b30 con 0x7fb7cc0cc110 2023-05-20T01:18:38.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.143+0000 7fb7cb7fe700 1 --2- 172.21.15.16:0/3972297192 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb7cc15ab80 0x7fb7cc0c6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:18:38.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.143+0000 7fb7caffd700 1 --2- 172.21.15.16:0/3972297192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7cc0cc110 0x7fb7cc155600 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fb7c000c4d0 tx=0x7fb7c000c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:38.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.143+0000 7fb7b3fff700 1 -- 172.21.15.16:0/3972297192 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7c0011790 con 0x7fb7cc0cc110 2023-05-20T01:18:38.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.143+0000 7fb7b3fff700 1 -- 172.21.15.16:0/3972297192 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7c00118f0 con 0x7fb7cc0cc110 2023-05-20T01:18:38.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.143+0000 7fb7d18b3700 1 -- 172.21.15.16:0/3972297192 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7cc0cbbf0 con 0x7fb7cc0cc110 2023-05-20T01:18:38.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.143+0000 7fb7b3fff700 1 -- 172.21.15.16:0/3972297192 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7c0011ca0 con 0x7fb7cc0cc110 2023-05-20T01:18:38.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.143+0000 7fb7d18b3700 1 -- 172.21.15.16:0/3972297192 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7cc0cb910 con 0x7fb7cc0cc110 2023-05-20T01:18:38.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.145+0000 7fb7b3fff700 1 -- 172.21.15.16:0/3972297192 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb7c001c460 con 0x7fb7cc0cc110 2023-05-20T01:18:38.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.145+0000 7fb7b3fff700 1 --2- 172.21.15.16:0/3972297192 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb7b806ace0 0x7fb7b806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:38.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.145+0000 7fb7b3fff700 1 -- 172.21.15.16:0/3972297192 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 7102+0+0 (secure 0 0 0) 0x7fb7c0089650 con 0x7fb7cc0cc110 2023-05-20T01:18:38.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.145+0000 7fb7ca7fc700 1 --2- 172.21.15.16:0/3972297192 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb7b806ace0 0x7fb7b806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:38.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.146+0000 7fb7ca7fc700 1 --2- 172.21.15.16:0/3972297192 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb7b806ace0 0x7fb7b806d190 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc0c7b80 tx=0x7fb7bc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:38.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.147+0000 7fb7dda3fb80 1 Processor -- start 2023-05-20T01:18:38.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.147+0000 7fb7dda3fb80 1 -- start start 2023-05-20T01:18:38.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.148+0000 7fb7dda3fb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5572e7abae50 0x5572e7abb220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:38.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.148+0000 7fb7dda3fb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5572e7ab88c0 0x5572e7aba4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:38.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.148+0000 7fb7dda3fb80 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5572e7aba9f0 0x5572e7ab9950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:38.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.148+0000 7fb7dda3fb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572e79c0060 con 0x5572e7abae50 2023-05-20T01:18:38.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.148+0000 7fb7dda3fb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572e7a128a0 con 0x5572e7ab88c0 2023-05-20T01:18:38.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.148+0000 7fb7dda3fb80 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x5572e7a0bd30 con 0x5572e7aba9f0 2023-05-20T01:18:38.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.148+0000 7fb7caffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5572e7abae50 0x5572e7abb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:38.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.148+0000 7fb7caffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5572e7abae50 0x5572e7abb220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33072/0 (socket says 172.21.15.16:33072) 2023-05-20T01:18:38.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.148+0000 7fb7caffd700 1 -- 172.21.15.16:0/1940089535 learned_addr learned my addr 172.21.15.16:0/1940089535 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:18:38.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.148+0000 7fb7cb7fe700 1 --2- 172.21.15.16:0/1940089535 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5572e7aba9f0 0x5572e7ab9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:38.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.148+0000 7fb7ca7fc700 1 --2- 172.21.15.16:0/1940089535 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5572e7ab88c0 0x5572e7aba4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:38.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.149+0000 7fb7caffd700 1 -- 172.21.15.16:0/1940089535 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5572e7aba9f0 msgr2=0x5572e7ab9950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:18:38.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.149+0000 7fb7caffd700 1 --2- 172.21.15.16:0/1940089535 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5572e7aba9f0 0x5572e7ab9950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:38.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.149+0000 7fb7caffd700 1 -- 172.21.15.16:0/1940089535 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5572e7ab88c0 msgr2=0x5572e7aba4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:18:38.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.149+0000 7fb7caffd700 1 --2- 172.21.15.16:0/1940089535 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5572e7ab88c0 0x5572e7aba4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:38.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.149+0000 7fb7caffd700 1 -- 172.21.15.16:0/1940089535 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5572e788d630 con 0x5572e7abae50 2023-05-20T01:18:38.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.149+0000 7fb7ca7fc700 1 --2- 172.21.15.16:0/1940089535 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5572e7ab88c0 0x5572e7aba4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:18:38.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.149+0000 7fb7caffd700 1 --2- 172.21.15.16:0/1940089535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5572e7abae50 0x5572e7abb220 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fb7c0057f40 tx=0x7fb7c0057c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1058e1b0b52ee0f0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:38.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.149+0000 7fb7b17fa700 1 -- 172.21.15.16:0/1940089535 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7c0011e00 con 0x5572e7abae50 2023-05-20T01:18:38.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.149+0000 7fb7b17fa700 1 -- 172.21.15.16:0/1940089535 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7c00560f0 con 0x5572e7abae50 2023-05-20T01:18:38.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.149+0000 7fb7b17fa700 1 -- 172.21.15.16:0/1940089535 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7c005fe70 con 0x5572e7abae50 2023-05-20T01:18:38.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.149+0000 7fb7dda3fb80 1 -- 172.21.15.16:0/1940089535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5572e7abae50 msgr2=0x5572e7abb220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:38.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.149+0000 7fb7dda3fb80 1 --2- 172.21.15.16:0/1940089535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5572e7abae50 0x5572e7abb220 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fb7c0057f40 tx=0x7fb7c0057c20 comp rx=0 tx=0).stop 2023-05-20T01:18:38.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.150+0000 7fb7dda3fb80 1 -- 172.21.15.16:0/1940089535 shutdown_connections 2023-05-20T01:18:38.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.150+0000 7fb7dda3fb80 1 --2- 172.21.15.16:0/1940089535 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5572e7aba9f0 0x5572e7ab9950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:38.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.150+0000 7fb7dda3fb80 1 --2- 172.21.15.16:0/1940089535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5572e7abae50 0x5572e7abb220 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:38.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.150+0000 7fb7dda3fb80 1 --2- 172.21.15.16:0/1940089535 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5572e7ab88c0 0x5572e7aba4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:38.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.150+0000 7fb7dda3fb80 1 -- 172.21.15.16:0/1940089535 >> 172.21.15.16:0/1940089535 conn(0x5572e7aaff80 msgr2=0x5572e7b00260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:18:38.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.150+0000 7fb7dda3fb80 1 -- 172.21.15.16:0/1940089535 shutdown_connections 2023-05-20T01:18:38.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.150+0000 7fb7dda3fb80 1 -- 172.21.15.16:0/1940089535 wait complete. 2023-05-20T01:18:38.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.150+0000 7fb7dda3fb80 1 -- start start 2023-05-20T01:18:38.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.150+0000 7fb7dda3fb80 1 Processor -- start 2023-05-20T01:18:38.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.151+0000 7fb7dda3fb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5572e7ab88c0 0x5572e7ae3210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:38.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.151+0000 7fb7dda3fb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5572e7aba9f0 0x5572e7af5460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:38.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.151+0000 7fb7dda3fb80 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5572e7abae50 0x5572e7af59a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:38.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.151+0000 7fb7dda3fb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572e79ad8e0 con 0x5572e7aba9f0 2023-05-20T01:18:38.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.151+0000 7fb7dda3fb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572e79ad580 con 0x5572e7ab88c0 2023-05-20T01:18:38.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.151+0000 7fb7dda3fb80 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x5572e788f920 con 0x5572e7abae50 2023-05-20T01:18:38.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.151+0000 7fb7dda3fb80 10 client.0 _ll_register_callbacks cb 0x5572e79cb780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-05-20T01:18:38.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.151+0000 7fb7ca7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5572e7aba9f0 0x5572e7af5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:38.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.151+0000 7fb7ca7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5572e7aba9f0 0x5572e7af5460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33080/0 (socket says 172.21.15.16:33080) 2023-05-20T01:18:38.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.151+0000 7fb7ca7fc700 1 -- 172.21.15.16:0/3585603094 learned_addr learned my addr 172.21.15.16:0/3585603094 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:18:38.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.151+0000 7fb7caffd700 1 --2- 172.21.15.16:0/3585603094 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5572e7ab88c0 0x5572e7ae3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:38.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.151+0000 7fb7cb7fe700 1 --2- 172.21.15.16:0/3585603094 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5572e7abae50 0x5572e7af59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:38.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.151+0000 7fb7ca7fc700 1 -- 172.21.15.16:0/3585603094 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5572e7abae50 msgr2=0x5572e7af59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:18:38.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.151+0000 7fb7ca7fc700 1 --2- 172.21.15.16:0/3585603094 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5572e7abae50 0x5572e7af59a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:38.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.151+0000 7fb7ca7fc700 1 -- 172.21.15.16:0/3585603094 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5572e7ab88c0 msgr2=0x5572e7ae3210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:18:38.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.151+0000 7fb7ca7fc700 1 --2- 172.21.15.16:0/3585603094 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5572e7ab88c0 0x5572e7ae3210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:38.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.151+0000 7fb7ca7fc700 1 -- 172.21.15.16:0/3585603094 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7cc0cb910 con 0x5572e7aba9f0 2023-05-20T01:18:38.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.151+0000 7fb7cb7fe700 1 --2- 172.21.15.16:0/3585603094 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5572e7abae50 0x5572e7af59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:18:38.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.152+0000 7fb7ca7fc700 1 --2- 172.21.15.16:0/3585603094 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5572e7aba9f0 0x5572e7af5460 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc011ed0 tx=0x7fb7bc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c223386ea4123311 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:38.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.152+0000 7fb7aa7fc700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-05-20T01:18:38.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.152+0000 7fb7aa7fc700 1 -- 172.21.15.16:0/3585603094 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7bc022620 con 0x5572e7aba9f0 2023-05-20T01:18:38.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.152+0000 7fb7aa7fc700 1 -- 172.21.15.16:0/3585603094 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(51..51 src has 1..51) v4 ==== 7102+0+0 (secure 0 0 0) 0x7fb7bc022780 con 0x5572e7aba9f0 2023-05-20T01:18:38.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.152+0000 7fb7dda3fb80 1 -- 172.21.15.16:0/3585603094 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5572e7aad740 con 0x5572e7aba9f0 2023-05-20T01:18:38.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.152+0000 7fb7aa7fc700 1 -- 172.21.15.16:0/3585603094 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb7bc015e70 con 0x5572e7aba9f0 2023-05-20T01:18:38.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.152+0000 7fb7dda3fb80 10 client.5982 fetch_fsmap learned FSMap version 29 2023-05-20T01:18:38.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.152+0000 7fb7dda3fb80 1 -- 172.21.15.16:0/3585603094 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) v3 -- 0x5572e79d2190 con 0x5572e7aba9f0 2023-05-20T01:18:38.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.152+0000 7fb7aa7fc700 1 -- 172.21.15.16:0/3585603094 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5572e7aad740 con 0x5572e7aba9f0 2023-05-20T01:18:38.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.152+0000 7fb7aa7fc700 1 -- 172.21.15.16:0/3585603094 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 29) v1 ==== 3122+0+0 (secure 0 0 0) 0x7fb7bc005a20 con 0x5572e7aba9f0 2023-05-20T01:18:38.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.152+0000 7fb7dda3fb80 10 client.5982 fetch_fsmap finished waiting for FSMap version 29 2023-05-20T01:18:38.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.152+0000 7fb7dda3fb80 10 client.5982 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2023-05-20T01:18:38.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.152+0000 7fb7dda3fb80 20 client.5982 populate_metadata read hostname 'smithi016' 2023-05-20T01:18:38.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.152+0000 7fb7dda3fb80 1 --2- 172.21.15.16:0/3585603094 >> [v2:172.21.15.16:6834/3284962389,v1:172.21.15.16:6836/3284962389] conn(0x5572e7adef10 0x5572e7ae6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:38.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.152+0000 7fb7dda3fb80 4 client.5982 mds_command: new command op to 5889 tid=0[{"prefix": "get_command_descriptions"}] 2023-05-20T01:18:38.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.152+0000 7fb7dda3fb80 1 -- 172.21.15.16:0/3585603094 --> [v2:172.21.15.16:6834/3284962389,v1:172.21.15.16:6836/3284962389] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5572e79ce0a0 con 0x5572e7adef10 2023-05-20T01:18:38.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.152+0000 7fb7caffd700 1 --2- 172.21.15.16:0/3585603094 >> [v2:172.21.15.16:6834/3284962389,v1:172.21.15.16:6836/3284962389] conn(0x5572e7adef10 0x5572e7ae6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:38.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.153+0000 7fb7aa7fc700 10 client.5982 ms_handle_connect on v2:172.21.15.16:6834/3284962389 2023-05-20T01:18:38.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.153+0000 7fb7caffd700 1 --2- 172.21.15.16:0/3585603094 >> [v2:172.21.15.16:6834/3284962389,v1:172.21.15.16:6836/3284962389] conn(0x5572e7adef10 0x5572e7ae6bb0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f5b5dbf51638044 server_cookie=a8e654bcaff3ae6d in_seq=0 out_seq=0 2023-05-20T01:18:38.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.153+0000 7fb7aa7fc700 1 -- 172.21.15.16:0/3585603094 <== mds.0 v2:172.21.15.16:6834/3284962389 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5572e79ce0a0 con 0x5572e7adef10 2023-05-20T01:18:38.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.153+0000 7fb7aa7fc700 10 client.5982 handle_command_reply: tid=0 2023-05-20T01:18:38.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.153+0000 7fb7dda3fb80 1 client.5982 shutdown 2023-05-20T01:18:38.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.153+0000 7fb7dda3fb80 1 -- 172.21.15.16:0/3585603094 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5572e7aba9f0 msgr2=0x5572e7af5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:38.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.153+0000 7fb7dda3fb80 1 --2- 172.21.15.16:0/3585603094 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5572e7aba9f0 0x5572e7af5460 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc011ed0 tx=0x7fb7bc004750 comp rx=0 tx=0).stop 2023-05-20T01:18:38.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.153+0000 7fb7dda3fb80 1 -- 172.21.15.16:0/3585603094 shutdown_connections 2023-05-20T01:18:38.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.153+0000 7fb7dda3fb80 1 --2- 172.21.15.16:0/3585603094 >> [v2:172.21.15.16:6834/3284962389,v1:172.21.15.16:6836/3284962389] conn(0x5572e7adef10 0x5572e7ae6bb0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:38.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.153+0000 7fb7dda3fb80 1 --2- 172.21.15.16:0/3585603094 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5572e7abae50 0x5572e7af59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:38.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.153+0000 7fb7dda3fb80 1 --2- 172.21.15.16:0/3585603094 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5572e7ab88c0 0x5572e7ae3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:38.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.153+0000 7fb7aa7fc700 0 client.5982 ms_handle_reset on v2:172.21.15.16:6834/3284962389 2023-05-20T01:18:38.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.153+0000 7fb7dda3fb80 1 --2- 172.21.15.16:0/3585603094 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5572e7aba9f0 0x5572e7af5460 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:38.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.153+0000 7fb7dda3fb80 1 -- 172.21.15.16:0/3585603094 >> 172.21.15.16:0/3585603094 conn(0x5572e7aaff80 msgr2=0x5572e7b037e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:18:38.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.153+0000 7fb7dda3fb80 1 -- 172.21.15.16:0/3585603094 shutdown_connections 2023-05-20T01:18:38.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.153+0000 7fb7dda3fb80 1 -- 172.21.15.16:0/3585603094 wait complete. 2023-05-20T01:18:38.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.154+0000 7fb7dda3fb80 20 client.5982 trim_cache size 0 max 16384 2023-05-20T01:18:38.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.170+0000 7fb7dda3fb80 1 Processor -- start 2023-05-20T01:18:38.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.171+0000 7fb7dda3fb80 1 -- start start 2023-05-20T01:18:38.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.171+0000 7fb7dda3fb80 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5572e7ab88c0 0x5572e7ab6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:38.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.171+0000 7fb7dda3fb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5572e7aba9f0 0x5572e7ab4a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:38.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.171+0000 7fb7dda3fb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5572e7abae50 0x5572e7ae7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:38.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.171+0000 7fb7dda3fb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572e7abc030 con 0x5572e7abae50 2023-05-20T01:18:38.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.171+0000 7fb7dda3fb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572e7af63e0 con 0x5572e7aba9f0 2023-05-20T01:18:38.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.171+0000 7fb7dda3fb80 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x5572e7ad9a60 con 0x5572e7ab88c0 2023-05-20T01:18:38.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.171+0000 7fb7cb7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5572e7abae50 0x5572e7ae7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:38.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.171+0000 7fb7cb7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5572e7abae50 0x5572e7ae7ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33088/0 (socket says 172.21.15.16:33088) 2023-05-20T01:18:38.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.171+0000 7fb7cb7fe700 1 -- 172.21.15.16:0/1179666002 learned_addr learned my addr 172.21.15.16:0/1179666002 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:18:38.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.172+0000 7fb7cb7fe700 1 -- 172.21.15.16:0/1179666002 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5572e7ab88c0 msgr2=0x5572e7ab6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:18:38.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.172+0000 7fb7cb7fe700 1 --2- 172.21.15.16:0/1179666002 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5572e7ab88c0 0x5572e7ab6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:38.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.172+0000 7fb7cb7fe700 1 -- 172.21.15.16:0/1179666002 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5572e7aba9f0 msgr2=0x5572e7ab4a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:18:38.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.172+0000 7fb7cb7fe700 1 --2- 172.21.15.16:0/1179666002 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5572e7aba9f0 0x5572e7ab4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:38.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.172+0000 7fb7cb7fe700 1 -- 172.21.15.16:0/1179666002 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5572e79d1ee0 con 0x5572e7abae50 2023-05-20T01:18:38.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.172+0000 7fb7cb7fe700 1 --2- 172.21.15.16:0/1179666002 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5572e7abae50 0x5572e7ae7ef0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fb7c400b730 tx=0x7fb7c4010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ab64ff3839d13f0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:38.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.172+0000 7fb7b1ffb700 1 -- 172.21.15.16:0/1179666002 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7c400bc40 con 0x5572e7abae50 2023-05-20T01:18:38.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.172+0000 7fb7b1ffb700 1 -- 172.21.15.16:0/1179666002 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7c400bdc0 con 0x5572e7abae50 2023-05-20T01:18:38.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.172+0000 7fb7b1ffb700 1 -- 172.21.15.16:0/1179666002 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7c4004270 con 0x5572e7abae50 2023-05-20T01:18:38.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.173+0000 7fb7dda3fb80 1 -- 172.21.15.16:0/1179666002 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5572e7abae50 msgr2=0x5572e7ae7ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:38.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.173+0000 7fb7dda3fb80 1 --2- 172.21.15.16:0/1179666002 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5572e7abae50 0x5572e7ae7ef0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fb7c400b730 tx=0x7fb7c4010000 comp rx=0 tx=0).stop 2023-05-20T01:18:38.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.173+0000 7fb7dda3fb80 1 -- 172.21.15.16:0/1179666002 shutdown_connections 2023-05-20T01:18:38.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.173+0000 7fb7dda3fb80 1 --2- 172.21.15.16:0/1179666002 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5572e7abae50 0x5572e7ae7ef0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:38.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.173+0000 7fb7dda3fb80 1 --2- 172.21.15.16:0/1179666002 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5572e7ab88c0 0x5572e7ab6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:38.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.173+0000 7fb7dda3fb80 1 --2- 172.21.15.16:0/1179666002 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5572e7aba9f0 0x5572e7ab4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:38.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.173+0000 7fb7dda3fb80 1 -- 172.21.15.16:0/1179666002 >> 172.21.15.16:0/1179666002 conn(0x5572e7aaff80 msgr2=0x5572e7abb220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:18:38.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.173+0000 7fb7dda3fb80 1 -- 172.21.15.16:0/1179666002 shutdown_connections 2023-05-20T01:18:38.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.173+0000 7fb7dda3fb80 1 -- 172.21.15.16:0/1179666002 wait complete. 2023-05-20T01:18:38.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.173+0000 7fb7dda3fb80 1 -- start start 2023-05-20T01:18:38.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.173+0000 7fb7dda3fb80 1 Processor -- start 2023-05-20T01:18:38.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.174+0000 7fb7dda3fb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5572e7ab88c0 0x5572e7af5260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:38.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.174+0000 7fb7dda3fb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5572e7aba9f0 0x5572e7adda60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:38.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.174+0000 7fb7dda3fb80 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5572e7abae50 0x5572e7ad52c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:38.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.174+0000 7fb7dda3fb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572e7af57a0 con 0x5572e7aba9f0 2023-05-20T01:18:38.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.174+0000 7fb7dda3fb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572e7adbdc0 con 0x5572e7ab88c0 2023-05-20T01:18:38.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.174+0000 7fb7dda3fb80 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x5572e7b21700 con 0x5572e7abae50 2023-05-20T01:18:38.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.174+0000 7fb7dda3fb80 10 client.0 _ll_register_callbacks cb 0x7fb7bc011a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-05-20T01:18:38.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.174+0000 7fb7ca7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5572e7aba9f0 0x5572e7adda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:38.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.174+0000 7fb7ca7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5572e7aba9f0 0x5572e7adda60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33092/0 (socket says 172.21.15.16:33092) 2023-05-20T01:18:38.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.174+0000 7fb7ca7fc700 1 -- 172.21.15.16:0/1649549603 learned_addr learned my addr 172.21.15.16:0/1649549603 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:18:38.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.174+0000 7fb7caffd700 1 --2- 172.21.15.16:0/1649549603 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5572e7ab88c0 0x5572e7af5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:38.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.174+0000 7fb7cb7fe700 1 --2- 172.21.15.16:0/1649549603 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5572e7abae50 0x5572e7ad52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:38.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.175+0000 7fb7ca7fc700 1 -- 172.21.15.16:0/1649549603 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5572e7abae50 msgr2=0x5572e7ad52c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:18:38.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.175+0000 7fb7ca7fc700 1 --2- 172.21.15.16:0/1649549603 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5572e7abae50 0x5572e7ad52c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:38.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.175+0000 7fb7ca7fc700 1 -- 172.21.15.16:0/1649549603 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5572e7ab88c0 msgr2=0x5572e7af5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:18:38.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.175+0000 7fb7ca7fc700 1 --2- 172.21.15.16:0/1649549603 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5572e7ab88c0 0x5572e7af5260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:38.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.175+0000 7fb7ca7fc700 1 -- 172.21.15.16:0/1649549603 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7cc0ce7b0 con 0x5572e7aba9f0 2023-05-20T01:18:38.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.175+0000 7fb7caffd700 1 --2- 172.21.15.16:0/1649549603 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5572e7ab88c0 0x5572e7af5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:18:38.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.175+0000 7fb7cb7fe700 1 --2- 172.21.15.16:0/1649549603 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5572e7abae50 0x5572e7ad52c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:18:38.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.175+0000 7fb7ca7fc700 1 --2- 172.21.15.16:0/1649549603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5572e7aba9f0 0x5572e7adda60 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc011ed0 tx=0x7fb7bc016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8cd8f093ba05210 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:18:38.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.175+0000 7fb7aa7fc700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-05-20T01:18:38.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.175+0000 7fb7aa7fc700 1 -- 172.21.15.16:0/1649549603 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7bc006e60 con 0x5572e7aba9f0 2023-05-20T01:18:38.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.175+0000 7fb7aa7fc700 1 -- 172.21.15.16:0/1649549603 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(51..51 src has 1..51) v4 ==== 7102+0+0 (secure 0 0 0) 0x7fb7bc005a20 con 0x5572e7aba9f0 2023-05-20T01:18:38.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.175+0000 7fb7dda3fb80 1 -- 172.21.15.16:0/1649549603 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb7c00560f0 con 0x5572e7aba9f0 2023-05-20T01:18:38.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.175+0000 7fb7aa7fc700 1 -- 172.21.15.16:0/1649549603 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb7bc011540 con 0x5572e7aba9f0 2023-05-20T01:18:38.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.175+0000 7fb7dda3fb80 10 client.5988 fetch_fsmap learned FSMap version 29 2023-05-20T01:18:38.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.175+0000 7fb7dda3fb80 1 -- 172.21.15.16:0/1649549603 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) v3 -- 0x5572e7822c60 con 0x5572e7aba9f0 2023-05-20T01:18:38.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.175+0000 7fb7aa7fc700 1 -- 172.21.15.16:0/1649549603 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7c00560f0 con 0x5572e7aba9f0 2023-05-20T01:18:38.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.175+0000 7fb7aa7fc700 1 -- 172.21.15.16:0/1649549603 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 29) v1 ==== 3122+0+0 (secure 0 0 0) 0x7fb7bc042200 con 0x5572e7aba9f0 2023-05-20T01:18:38.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.175+0000 7fb7dda3fb80 10 client.5988 fetch_fsmap finished waiting for FSMap version 29 2023-05-20T01:18:38.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.175+0000 7fb7dda3fb80 10 client.5988 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2023-05-20T01:18:38.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.175+0000 7fb7dda3fb80 20 client.5988 populate_metadata read hostname 'smithi016' 2023-05-20T01:18:38.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.175+0000 7fb7dda3fb80 1 --2- 172.21.15.16:0/1649549603 >> [v2:172.21.15.16:6834/3284962389,v1:172.21.15.16:6836/3284962389] conn(0x5572e7ad5800 0x5572e7ad5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:18:38.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.175+0000 7fb7dda3fb80 4 client.5988 mds_command: new command op to 5889 tid=0[{"prefix": "scrub status"}] 2023-05-20T01:18:38.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.175+0000 7fb7dda3fb80 1 -- 172.21.15.16:0/1649549603 --> [v2:172.21.15.16:6834/3284962389,v1:172.21.15.16:6836/3284962389] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5572e79da660 con 0x5572e7ad5800 2023-05-20T01:18:38.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.175+0000 7fb7caffd700 1 --2- 172.21.15.16:0/1649549603 >> [v2:172.21.15.16:6834/3284962389,v1:172.21.15.16:6836/3284962389] conn(0x5572e7ad5800 0x5572e7ad5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:18:38.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.176+0000 7fb7aa7fc700 10 client.5988 ms_handle_connect on v2:172.21.15.16:6834/3284962389 2023-05-20T01:18:38.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.176+0000 7fb7caffd700 1 --2- 172.21.15.16:0/1649549603 >> [v2:172.21.15.16:6834/3284962389,v1:172.21.15.16:6836/3284962389] conn(0x5572e7ad5800 0x5572e7ad5bd0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb931dc500c63e22 server_cookie=f169ca176b023b0f in_seq=0 out_seq=0 2023-05-20T01:18:38.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.176+0000 7fb7aa7fc700 1 -- 172.21.15.16:0/1649549603 <== mds.0 v2:172.21.15.16:6834/3284962389 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+260 (crc 0 0 0) 0x5572e79da660 con 0x5572e7ad5800 2023-05-20T01:18:38.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.176+0000 7fb7aa7fc700 10 client.5988 handle_command_reply: tid=0 2023-05-20T01:18:38.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.176+0000 7fb7dda3fb80 1 client.5988 shutdown 2023-05-20T01:18:38.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.176+0000 7fb7dda3fb80 1 -- 172.21.15.16:0/1649549603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5572e7aba9f0 msgr2=0x5572e7adda60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:38.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.176+0000 7fb7dda3fb80 1 --2- 172.21.15.16:0/1649549603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5572e7aba9f0 0x5572e7adda60 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc011ed0 tx=0x7fb7bc016000 comp rx=0 tx=0).stop 2023-05-20T01:18:38.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.176+0000 7fb7dda3fb80 1 -- 172.21.15.16:0/1649549603 shutdown_connections 2023-05-20T01:18:38.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.176+0000 7fb7dda3fb80 1 --2- 172.21.15.16:0/1649549603 >> [v2:172.21.15.16:6834/3284962389,v1:172.21.15.16:6836/3284962389] conn(0x5572e7ad5800 0x5572e7ad5bd0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:38.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.176+0000 7fb7dda3fb80 1 --2- 172.21.15.16:0/1649549603 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5572e7abae50 0x5572e7ad52c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:38.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.176+0000 7fb7aa7fc700 0 client.5988 ms_handle_reset on v2:172.21.15.16:6834/3284962389 2023-05-20T01:18:38.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.176+0000 7fb7dda3fb80 1 --2- 172.21.15.16:0/1649549603 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5572e7ab88c0 0x5572e7af5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:38.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.176+0000 7fb7dda3fb80 1 --2- 172.21.15.16:0/1649549603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5572e7aba9f0 0x5572e7adda60 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:38.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.176+0000 7fb7dda3fb80 1 -- 172.21.15.16:0/1649549603 >> 172.21.15.16:0/1649549603 conn(0x5572e7aaff80 msgr2=0x5572e7abb220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:18:38.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.176+0000 7fb7dda3fb80 1 -- 172.21.15.16:0/1649549603 shutdown_connections 2023-05-20T01:18:38.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.176+0000 7fb7dda3fb80 1 -- 172.21.15.16:0/1649549603 wait complete. 2023-05-20T01:18:38.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.176+0000 7fb7dda3fb80 20 client.5988 trim_cache size 0 max 16384 2023-05-20T01:18:38.190 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-05-20T01:18:38.191 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2023-05-20T01:18:38.191 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2023-05-20T01:18:38.191 INFO:teuthology.orchestra.run.smithi016.stdout: "610f824c-6e63-4d09-bf34-36eb92bf457a": { 2023-05-20T01:18:38.191 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2023-05-20T01:18:38.191 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "610f824c-6e63-4d09-bf34-36eb92bf457a", 2023-05-20T01:18:38.191 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,repair" 2023-05-20T01:18:38.191 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:18:38.192 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:18:38.192 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-05-20T01:18:38.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.177+0000 7fb7d18b3700 1 -- 172.21.15.16:0/3972297192 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb7b806ace0 msgr2=0x7fb7b806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:38.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.177+0000 7fb7d18b3700 1 --2- 172.21.15.16:0/3972297192 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb7b806ace0 0x7fb7b806d190 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc0c7b80 tx=0x7fb7bc008040 comp rx=0 tx=0).stop 2023-05-20T01:18:38.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.177+0000 7fb7d18b3700 1 -- 172.21.15.16:0/3972297192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7cc0cc110 msgr2=0x7fb7cc155600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:18:38.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.177+0000 7fb7d18b3700 1 --2- 172.21.15.16:0/3972297192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7cc0cc110 0x7fb7cc155600 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fb7c000c4d0 tx=0x7fb7c000c7e0 comp rx=0 tx=0).stop 2023-05-20T01:18:38.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.177+0000 7fb7d18b3700 1 -- 172.21.15.16:0/3972297192 shutdown_connections 2023-05-20T01:18:38.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.177+0000 7fb7d18b3700 1 --2- 172.21.15.16:0/3972297192 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb7b806ace0 0x7fb7b806d190 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:38.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.177+0000 7fb7d18b3700 1 --2- 172.21.15.16:0/3972297192 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb7cc15ab80 0x7fb7cc0c6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:38.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.177+0000 7fb7d18b3700 1 --2- 172.21.15.16:0/3972297192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7cc0cc110 0x7fb7cc155600 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:38.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.177+0000 7fb7d18b3700 1 --2- 172.21.15.16:0/3972297192 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb7cc0cec80 0x7fb7cc06cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:18:38.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.177+0000 7fb7d18b3700 1 -- 172.21.15.16:0/3972297192 >> 172.21.15.16:0/3972297192 conn(0x7fb7cc06ae30 msgr2=0x7fb7cc0ae9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:18:38.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.177+0000 7fb7d18b3700 1 -- 172.21.15.16:0/3972297192 shutdown_connections 2023-05-20T01:18:38.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:18:38.177+0000 7fb7d18b3700 1 -- 172.21.15.16:0/3972297192 wait complete. 2023-05-20T01:18:38.195 INFO:tasks.cephfs.filesystem:scrub status for tag:610f824c-6e63-4d09-bf34-36eb92bf457a - {'path': '/', 'tag': '610f824c-6e63-4d09-bf34-36eb92bf457a', 'options': 'recursive,repair'} 2023-05-20T01:19:05.389 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:19:05.393 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:19:08.196 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 scrub status 2023-05-20T01:19:08.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.334+0000 7fc1a18cd700 1 Processor -- start 2023-05-20T01:19:08.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.335+0000 7fc1a18cd700 1 -- start start 2023-05-20T01:19:08.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.335+0000 7fc1a18cd700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fc19c15cd50 0x7fc19c0d20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:08.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.335+0000 7fc1a18cd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc19c0cfcb0 0x7fc19c0d0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:08.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.335+0000 7fc1a18cd700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc19c0d1c10 con 0x7fc19c0cd920 2023-05-20T01:19:08.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.335+0000 7fc1a18cd700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc19c0d0660 con 0x7fc19c0cfcb0 2023-05-20T01:19:08.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.335+0000 7fc1a18cd700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc19c15d120 con 0x7fc19c15cd50 2023-05-20T01:19:08.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.336+0000 7fc19a7fc700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc19c0cfcb0 0x7fc19c0d0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:08.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.336+0000 7fc19a7fc700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc19c0cfcb0 0x7fc19c0d0120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33100/0 (socket says 172.21.15.16:33100) 2023-05-20T01:19:08.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.336+0000 7fc19a7fc700 1 -- 172.21.15.16:0/3660991629 learned_addr learned my addr 172.21.15.16:0/3660991629 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:19:08.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.336+0000 7fc19affd700 1 --2- 172.21.15.16:0/3660991629 >> v2:172.21.15.132:3300/0 conn(0x7fc19c15cd50 0x7fc19c0d20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:08.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.336+0000 7fc19a7fc700 1 -- 172.21.15.16:0/3660991629 >> v2:172.21.15.132:3300/0 conn(0x7fc19c15cd50 msgr2=0x7fc19c0d20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:19:08.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.336+0000 7fc19a7fc700 1 --2- 172.21.15.16:0/3660991629 >> v2:172.21.15.132:3300/0 conn(0x7fc19c15cd50 0x7fc19c0d20a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:08.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.336+0000 7fc19a7fc700 1 -- 172.21.15.16:0/3660991629 >> v1:172.21.15.132:6789/0 conn(0x7fc19c0cd920 legacy=0x7fc19c0b6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:08.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.336+0000 7fc19a7fc700 1 -- 172.21.15.16:0/3660991629 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc19c0d12e0 con 0x7fc19c0cfcb0 2023-05-20T01:19:08.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.336+0000 7fc19affd700 1 --2- 172.21.15.16:0/3660991629 >> v2:172.21.15.132:3300/0 conn(0x7fc19c15cd50 0x7fc19c0d20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:19:08.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.336+0000 7fc19a7fc700 1 --2- 172.21.15.16:0/3660991629 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc19c0cfcb0 0x7fc19c0d0120 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fc19c0d32a0 tx=0x7fc18c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1b90cff312795e9 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:08.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.336+0000 7fc199ffb700 1 -- 172.21.15.16:0/3660991629 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc18c00b870 con 0x7fc19c0cfcb0 2023-05-20T01:19:08.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.336+0000 7fc199ffb700 1 -- 172.21.15.16:0/3660991629 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc18c00b9d0 con 0x7fc19c0cfcb0 2023-05-20T01:19:08.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.336+0000 7fc199ffb700 1 -- 172.21.15.16:0/3660991629 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc18c00bd60 con 0x7fc19c0cfcb0 2023-05-20T01:19:08.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.337+0000 7fc1a18cd700 1 -- 172.21.15.16:0/3660991629 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc19c0cfcb0 msgr2=0x7fc19c0d0120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:08.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.337+0000 7fc1a18cd700 1 --2- 172.21.15.16:0/3660991629 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc19c0cfcb0 0x7fc19c0d0120 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fc19c0d32a0 tx=0x7fc18c009bd0 comp rx=0 tx=0).stop 2023-05-20T01:19:08.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.337+0000 7fc1a18cd700 1 -- 172.21.15.16:0/3660991629 shutdown_connections 2023-05-20T01:19:08.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.337+0000 7fc1a18cd700 1 --2- 172.21.15.16:0/3660991629 >> v2:172.21.15.132:3300/0 conn(0x7fc19c15cd50 0x7fc19c0d20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:08.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.337+0000 7fc1a18cd700 1 --2- 172.21.15.16:0/3660991629 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc19c0cfcb0 0x7fc19c0d0120 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:08.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.337+0000 7fc1a18cd700 1 -- 172.21.15.16:0/3660991629 >> 172.21.15.16:0/3660991629 conn(0x7fc19c06b070 msgr2=0x7fc19c06b470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:19:08.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.337+0000 7fc1a18cd700 1 -- 172.21.15.16:0/3660991629 shutdown_connections 2023-05-20T01:19:08.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.337+0000 7fc1a18cd700 1 -- 172.21.15.16:0/3660991629 wait complete. 2023-05-20T01:19:08.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.338+0000 7fc1a18cd700 1 Processor -- start 2023-05-20T01:19:08.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.338+0000 7fc1a18cd700 1 -- start start 2023-05-20T01:19:08.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.339+0000 7fc1a18cd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc19c0cd920 0x7fc19c0b8160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:08.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.339+0000 7fc1a18cd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc19c0cfcb0 0x7fc19c0cb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:08.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.339+0000 7fc1a18cd700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc19c15cd50 0x7fc19c0cbb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:08.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.339+0000 7fc1a18cd700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc19c0cc070 con 0x7fc19c0cfcb0 2023-05-20T01:19:08.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.339+0000 7fc1a18cd700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc19c0ca3a0 con 0x7fc19c0cd920 2023-05-20T01:19:08.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.339+0000 7fc1a18cd700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc19c0cb3b0 con 0x7fc19c15cd50 2023-05-20T01:19:08.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.339+0000 7fc19a7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc19c0cfcb0 0x7fc19c0cb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:08.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.339+0000 7fc19a7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc19c0cfcb0 0x7fc19c0cb5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33106/0 (socket says 172.21.15.16:33106) 2023-05-20T01:19:08.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.339+0000 7fc19a7fc700 1 -- 172.21.15.16:0/2779229089 learned_addr learned my addr 172.21.15.16:0/2779229089 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:19:08.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.339+0000 7fc19affd700 1 --2- 172.21.15.16:0/2779229089 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc19c0cd920 0x7fc19c0b8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:08.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.339+0000 7fc19a7fc700 1 -- 172.21.15.16:0/2779229089 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc19c15cd50 msgr2=0x7fc19c0cbb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:08.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.339+0000 7fc19a7fc700 1 --2- 172.21.15.16:0/2779229089 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc19c15cd50 0x7fc19c0cbb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:08.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.339+0000 7fc19a7fc700 1 -- 172.21.15.16:0/2779229089 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc19c0cd920 msgr2=0x7fc19c0b8160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:08.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.339+0000 7fc19a7fc700 1 --2- 172.21.15.16:0/2779229089 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc19c0cd920 0x7fc19c0b8160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:08.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.339+0000 7fc19a7fc700 1 -- 172.21.15.16:0/2779229089 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc18c009550 con 0x7fc19c0cfcb0 2023-05-20T01:19:08.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.340+0000 7fc19a7fc700 1 --2- 172.21.15.16:0/2779229089 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc19c0cfcb0 0x7fc19c0cb5f0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fc18c000c00 tx=0x7fc18c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:08.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.340+0000 7fc193fff700 1 -- 172.21.15.16:0/2779229089 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc18c004540 con 0x7fc19c0cfcb0 2023-05-20T01:19:08.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.340+0000 7fc193fff700 1 -- 172.21.15.16:0/2779229089 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc18c0036a0 con 0x7fc19c0cfcb0 2023-05-20T01:19:08.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.340+0000 7fc1a18cd700 1 -- 172.21.15.16:0/2779229089 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc19c0d18a0 con 0x7fc19c0cfcb0 2023-05-20T01:19:08.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.340+0000 7fc193fff700 1 -- 172.21.15.16:0/2779229089 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc18c003a10 con 0x7fc19c0cfcb0 2023-05-20T01:19:08.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.340+0000 7fc1a18cd700 1 -- 172.21.15.16:0/2779229089 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc19c0d15c0 con 0x7fc19c0cfcb0 2023-05-20T01:19:08.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.341+0000 7fc193fff700 1 -- 172.21.15.16:0/2779229089 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc18c021070 con 0x7fc19c0cfcb0 2023-05-20T01:19:08.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.342+0000 7fc193fff700 1 --2- 172.21.15.16:0/2779229089 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc17c06ace0 0x7fc17c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:08.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.342+0000 7fc193fff700 1 -- 172.21.15.16:0/2779229089 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 7102+0+0 (secure 0 0 0) 0x7fc18c08b120 con 0x7fc19c0cfcb0 2023-05-20T01:19:08.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.342+0000 7fc19affd700 1 --2- 172.21.15.16:0/2779229089 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc17c06ace0 0x7fc17c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:08.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.343+0000 7fc19affd700 1 --2- 172.21.15.16:0/2779229089 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc17c06ace0 0x7fc17c06d190 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fc18400d8b0 tx=0x7fc184008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:08.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.344+0000 7fc1ada59b80 1 Processor -- start 2023-05-20T01:19:08.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.344+0000 7fc1ada59b80 1 -- start start 2023-05-20T01:19:08.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.345+0000 7fc1ada59b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602a319a450 0x5602a319a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:08.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.345+0000 7fc1ada59b80 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5602a3196750 0x5602a316f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:08.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.345+0000 7fc1ada59b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5602a316d8c0 0x5602a316f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:08.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.345+0000 7fc1ada59b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602a3075060 con 0x5602a319a450 2023-05-20T01:19:08.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.345+0000 7fc1ada59b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602a30c78a0 con 0x5602a316d8c0 2023-05-20T01:19:08.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.345+0000 7fc1ada59b80 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x5602a30c0d30 con 0x5602a3196750 2023-05-20T01:19:08.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.345+0000 7fc19affd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602a319a450 0x5602a319a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:08.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.345+0000 7fc19affd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602a319a450 0x5602a319a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33112/0 (socket says 172.21.15.16:33112) 2023-05-20T01:19:08.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.345+0000 7fc19affd700 1 -- 172.21.15.16:0/2871119639 learned_addr learned my addr 172.21.15.16:0/2871119639 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:19:08.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.345+0000 7fc19a7fc700 1 --2- 172.21.15.16:0/2871119639 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5602a3196750 0x5602a316f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:08.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.345+0000 7fc19b7fe700 1 --2- 172.21.15.16:0/2871119639 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5602a316d8c0 0x5602a316f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:08.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.345+0000 7fc19affd700 1 -- 172.21.15.16:0/2871119639 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5602a3196750 msgr2=0x5602a316f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:19:08.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.345+0000 7fc19affd700 1 --2- 172.21.15.16:0/2871119639 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5602a3196750 0x5602a316f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:08.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.345+0000 7fc19affd700 1 -- 172.21.15.16:0/2871119639 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5602a316d8c0 msgr2=0x5602a316f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:19:08.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.345+0000 7fc19affd700 1 --2- 172.21.15.16:0/2871119639 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5602a316d8c0 0x5602a316f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:08.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.345+0000 7fc19affd700 1 -- 172.21.15.16:0/2871119639 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5602a2f42630 con 0x5602a319a450 2023-05-20T01:19:08.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.345+0000 7fc19affd700 1 --2- 172.21.15.16:0/2871119639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602a319a450 0x5602a319a820 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fc184014880 tx=0x7fc184012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb2d7422fd94dc1 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:08.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.346+0000 7fc1917fa700 1 -- 172.21.15.16:0/2871119639 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1840157d0 con 0x5602a319a450 2023-05-20T01:19:08.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.346+0000 7fc1917fa700 1 -- 172.21.15.16:0/2871119639 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc184019070 con 0x5602a319a450 2023-05-20T01:19:08.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.346+0000 7fc1917fa700 1 -- 172.21.15.16:0/2871119639 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc184015b30 con 0x5602a319a450 2023-05-20T01:19:08.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.346+0000 7fc1ada59b80 1 -- 172.21.15.16:0/2871119639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602a319a450 msgr2=0x5602a319a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:08.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.346+0000 7fc1ada59b80 1 --2- 172.21.15.16:0/2871119639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602a319a450 0x5602a319a820 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fc184014880 tx=0x7fc184012000 comp rx=0 tx=0).stop 2023-05-20T01:19:08.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.346+0000 7fc1ada59b80 1 -- 172.21.15.16:0/2871119639 shutdown_connections 2023-05-20T01:19:08.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.346+0000 7fc1ada59b80 1 --2- 172.21.15.16:0/2871119639 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5602a316d8c0 0x5602a316f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:08.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.346+0000 7fc1ada59b80 1 --2- 172.21.15.16:0/2871119639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602a319a450 0x5602a319a820 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:08.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.346+0000 7fc1ada59b80 1 --2- 172.21.15.16:0/2871119639 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5602a3196750 0x5602a316f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:08.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.346+0000 7fc1ada59b80 1 -- 172.21.15.16:0/2871119639 >> 172.21.15.16:0/2871119639 conn(0x5602a3164f80 msgr2=0x5602a31b5260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:19:08.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.346+0000 7fc1ada59b80 1 -- 172.21.15.16:0/2871119639 shutdown_connections 2023-05-20T01:19:08.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.346+0000 7fc1ada59b80 1 -- 172.21.15.16:0/2871119639 wait complete. 2023-05-20T01:19:08.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.347+0000 7fc1ada59b80 1 -- start start 2023-05-20T01:19:08.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.347+0000 7fc1ada59b80 1 Processor -- start 2023-05-20T01:19:08.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.348+0000 7fc1ada59b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602a316d8c0 0x5602a3195310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:08.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.348+0000 7fc1ada59b80 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5602a3196750 0x5602a318d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:08.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.348+0000 7fc1ada59b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5602a319a450 0x5602a318dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:08.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.348+0000 7fc1ada59b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602a30628e0 con 0x5602a316d8c0 2023-05-20T01:19:08.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.348+0000 7fc1ada59b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602a3062580 con 0x5602a319a450 2023-05-20T01:19:08.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.348+0000 7fc1ada59b80 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x5602a2f44920 con 0x5602a3196750 2023-05-20T01:19:08.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.348+0000 7fc1ada59b80 10 client.0 _ll_register_callbacks cb 0x5602a3080780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-05-20T01:19:08.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.348+0000 7fc19affd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602a316d8c0 0x5602a3195310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:08.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.348+0000 7fc19affd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602a316d8c0 0x5602a3195310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33118/0 (socket says 172.21.15.16:33118) 2023-05-20T01:19:08.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.348+0000 7fc19affd700 1 -- 172.21.15.16:0/710879948 learned_addr learned my addr 172.21.15.16:0/710879948 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:19:08.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.348+0000 7fc19b7fe700 1 --2- 172.21.15.16:0/710879948 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5602a319a450 0x5602a318dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:08.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.348+0000 7fc19affd700 1 -- 172.21.15.16:0/710879948 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5602a3196750 msgr2=0x5602a318d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:19:08.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.348+0000 7fc19a7fc700 1 --2- 172.21.15.16:0/710879948 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5602a3196750 0x5602a318d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:08.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.348+0000 7fc19affd700 1 --2- 172.21.15.16:0/710879948 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5602a3196750 0x5602a318d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:08.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.348+0000 7fc19affd700 1 -- 172.21.15.16:0/710879948 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5602a319a450 msgr2=0x5602a318dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:19:08.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.348+0000 7fc19affd700 1 --2- 172.21.15.16:0/710879948 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5602a319a450 0x5602a318dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:08.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.348+0000 7fc19affd700 1 -- 172.21.15.16:0/710879948 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc18400e040 con 0x5602a316d8c0 2023-05-20T01:19:08.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.348+0000 7fc19a7fc700 1 --2- 172.21.15.16:0/710879948 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5602a3196750 0x5602a318d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:19:08.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.348+0000 7fc19b7fe700 1 --2- 172.21.15.16:0/710879948 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5602a319a450 0x5602a318dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:19:08.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.349+0000 7fc19affd700 1 --2- 172.21.15.16:0/710879948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602a316d8c0 0x5602a3195310 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fc184022040 tx=0x7fc1840047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb5d9a1aaeed7c40 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:08.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.349+0000 7fc17a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-05-20T01:19:08.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.349+0000 7fc17a7fc700 1 -- 172.21.15.16:0/710879948 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc184003b10 con 0x5602a316d8c0 2023-05-20T01:19:08.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.349+0000 7fc17a7fc700 1 -- 172.21.15.16:0/710879948 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(51..51 src has 1..51) v4 ==== 7102+0+0 (secure 0 0 0) 0x7fc184003c70 con 0x5602a316d8c0 2023-05-20T01:19:08.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.349+0000 7fc1ada59b80 1 -- 172.21.15.16:0/710879948 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5602a3162740 con 0x5602a316d8c0 2023-05-20T01:19:08.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.349+0000 7fc17a7fc700 1 -- 172.21.15.16:0/710879948 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc184003e30 con 0x5602a316d8c0 2023-05-20T01:19:08.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.349+0000 7fc1ada59b80 10 client.6000 fetch_fsmap learned FSMap version 29 2023-05-20T01:19:08.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.349+0000 7fc1ada59b80 1 -- 172.21.15.16:0/710879948 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) v3 -- 0x5602a3087190 con 0x5602a316d8c0 2023-05-20T01:19:08.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.349+0000 7fc17a7fc700 1 -- 172.21.15.16:0/710879948 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc18403b070 con 0x5602a316d8c0 2023-05-20T01:19:08.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.349+0000 7fc17a7fc700 1 -- 172.21.15.16:0/710879948 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 29) v1 ==== 3122+0+0 (secure 0 0 0) 0x7fc1840155d0 con 0x5602a316d8c0 2023-05-20T01:19:08.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.349+0000 7fc1ada59b80 10 client.6000 fetch_fsmap finished waiting for FSMap version 29 2023-05-20T01:19:08.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.349+0000 7fc1ada59b80 10 client.6000 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2023-05-20T01:19:08.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.349+0000 7fc1ada59b80 20 client.6000 populate_metadata read hostname 'smithi016' 2023-05-20T01:19:08.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.349+0000 7fc1ada59b80 1 --2- 172.21.15.16:0/710879948 >> [v2:172.21.15.16:6834/3284962389,v1:172.21.15.16:6836/3284962389] conn(0x5602a31b4e10 0x5602a31b8080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:08.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.349+0000 7fc1ada59b80 4 client.6000 mds_command: new command op to 5889 tid=0[{"prefix": "get_command_descriptions"}] 2023-05-20T01:19:08.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.349+0000 7fc1ada59b80 1 -- 172.21.15.16:0/710879948 --> [v2:172.21.15.16:6834/3284962389,v1:172.21.15.16:6836/3284962389] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5602a30830a0 con 0x5602a31b4e10 2023-05-20T01:19:08.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.349+0000 7fc19a7fc700 1 --2- 172.21.15.16:0/710879948 >> [v2:172.21.15.16:6834/3284962389,v1:172.21.15.16:6836/3284962389] conn(0x5602a31b4e10 0x5602a31b8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:08.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.350+0000 7fc17a7fc700 10 client.6000 ms_handle_connect on v2:172.21.15.16:6834/3284962389 2023-05-20T01:19:08.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.350+0000 7fc19a7fc700 1 --2- 172.21.15.16:0/710879948 >> [v2:172.21.15.16:6834/3284962389,v1:172.21.15.16:6836/3284962389] conn(0x5602a31b4e10 0x5602a31b8080 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=962a6dbf1e6edbc9 server_cookie=879c916651ab9021 in_seq=0 out_seq=0 2023-05-20T01:19:08.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.350+0000 7fc17a7fc700 1 -- 172.21.15.16:0/710879948 <== mds.0 v2:172.21.15.16:6834/3284962389 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5602a30830a0 con 0x5602a31b4e10 2023-05-20T01:19:08.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.350+0000 7fc17a7fc700 10 client.6000 handle_command_reply: tid=0 2023-05-20T01:19:08.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.350+0000 7fc1ada59b80 1 client.6000 shutdown 2023-05-20T01:19:08.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.350+0000 7fc1ada59b80 1 -- 172.21.15.16:0/710879948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602a316d8c0 msgr2=0x5602a3195310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:08.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.350+0000 7fc1ada59b80 1 --2- 172.21.15.16:0/710879948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602a316d8c0 0x5602a3195310 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fc184022040 tx=0x7fc1840047c0 comp rx=0 tx=0).stop 2023-05-20T01:19:08.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.350+0000 7fc1ada59b80 1 -- 172.21.15.16:0/710879948 shutdown_connections 2023-05-20T01:19:08.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.350+0000 7fc1ada59b80 1 --2- 172.21.15.16:0/710879948 >> [v2:172.21.15.16:6834/3284962389,v1:172.21.15.16:6836/3284962389] conn(0x5602a31b4e10 0x5602a31b8080 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:08.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.350+0000 7fc1ada59b80 1 --2- 172.21.15.16:0/710879948 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5602a319a450 0x5602a318dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:08.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.350+0000 7fc1ada59b80 1 --2- 172.21.15.16:0/710879948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602a316d8c0 0x5602a3195310 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:08.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.351+0000 7fc17a7fc700 0 client.6000 ms_handle_reset on v2:172.21.15.16:6834/3284962389 2023-05-20T01:19:08.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.351+0000 7fc1ada59b80 1 --2- 172.21.15.16:0/710879948 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5602a3196750 0x5602a318d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:08.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.351+0000 7fc1ada59b80 1 -- 172.21.15.16:0/710879948 >> 172.21.15.16:0/710879948 conn(0x5602a3164f80 msgr2=0x5602a316a430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:19:08.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.351+0000 7fc1ada59b80 1 -- 172.21.15.16:0/710879948 shutdown_connections 2023-05-20T01:19:08.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.351+0000 7fc1ada59b80 1 -- 172.21.15.16:0/710879948 wait complete. 2023-05-20T01:19:08.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.351+0000 7fc1ada59b80 20 client.6000 trim_cache size 0 max 16384 2023-05-20T01:19:08.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.368+0000 7fc1ada59b80 1 Processor -- start 2023-05-20T01:19:08.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.368+0000 7fc1ada59b80 1 -- start start 2023-05-20T01:19:08.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.369+0000 7fc1ada59b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5602a316d8c0 0x5602a31b5260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:08.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.369+0000 7fc1ada59b80 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5602a3196750 0x5602a31b57a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:08.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.369+0000 7fc1ada59b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602a319a450 0x5602a3190750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:08.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.369+0000 7fc1ada59b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602a316df80 con 0x5602a319a450 2023-05-20T01:19:08.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.369+0000 7fc1ada59b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602a31963c0 con 0x5602a316d8c0 2023-05-20T01:19:08.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.369+0000 7fc1ada59b80 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x5602a31b5fe0 con 0x5602a3196750 2023-05-20T01:19:08.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.369+0000 7fc19b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602a319a450 0x5602a3190750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:08.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.369+0000 7fc19b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602a319a450 0x5602a3190750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33130/0 (socket says 172.21.15.16:33130) 2023-05-20T01:19:08.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.369+0000 7fc19b7fe700 1 -- 172.21.15.16:0/77648605 learned_addr learned my addr 172.21.15.16:0/77648605 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:19:08.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.369+0000 7fc19a7fc700 1 --2- 172.21.15.16:0/77648605 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5602a3196750 0x5602a31b57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:08.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.369+0000 7fc19affd700 1 --2- 172.21.15.16:0/77648605 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5602a316d8c0 0x5602a31b5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:08.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.369+0000 7fc19b7fe700 1 -- 172.21.15.16:0/77648605 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5602a3196750 msgr2=0x5602a31b57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:19:08.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.369+0000 7fc19b7fe700 1 --2- 172.21.15.16:0/77648605 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5602a3196750 0x5602a31b57a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:08.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.369+0000 7fc19b7fe700 1 -- 172.21.15.16:0/77648605 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5602a316d8c0 msgr2=0x5602a31b5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:19:08.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.369+0000 7fc19b7fe700 1 --2- 172.21.15.16:0/77648605 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5602a316d8c0 0x5602a31b5260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:08.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.369+0000 7fc19b7fe700 1 -- 172.21.15.16:0/77648605 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5602a3086ee0 con 0x5602a319a450 2023-05-20T01:19:08.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.369+0000 7fc19affd700 1 --2- 172.21.15.16:0/77648605 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5602a316d8c0 0x5602a31b5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:19:08.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.369+0000 7fc19b7fe700 1 --2- 172.21.15.16:0/77648605 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602a319a450 0x5602a3190750 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fc1940012e0 tx=0x7fc194002170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f4035f5642da1c4 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:08.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.369+0000 7fc191ffb700 1 -- 172.21.15.16:0/77648605 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc194003c10 con 0x5602a319a450 2023-05-20T01:19:08.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.369+0000 7fc191ffb700 1 -- 172.21.15.16:0/77648605 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc19400d050 con 0x5602a319a450 2023-05-20T01:19:08.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.369+0000 7fc191ffb700 1 -- 172.21.15.16:0/77648605 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc19400f7e0 con 0x5602a319a450 2023-05-20T01:19:08.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.370+0000 7fc1ada59b80 1 -- 172.21.15.16:0/77648605 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602a319a450 msgr2=0x5602a3190750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:08.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.370+0000 7fc1ada59b80 1 --2- 172.21.15.16:0/77648605 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602a319a450 0x5602a3190750 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fc1940012e0 tx=0x7fc194002170 comp rx=0 tx=0).stop 2023-05-20T01:19:08.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.370+0000 7fc1ada59b80 1 -- 172.21.15.16:0/77648605 shutdown_connections 2023-05-20T01:19:08.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.370+0000 7fc1ada59b80 1 --2- 172.21.15.16:0/77648605 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602a319a450 0x5602a3190750 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:08.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.370+0000 7fc1ada59b80 1 --2- 172.21.15.16:0/77648605 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5602a316d8c0 0x5602a31b5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:08.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.370+0000 7fc1ada59b80 1 --2- 172.21.15.16:0/77648605 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5602a3196750 0x5602a31b57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:08.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.370+0000 7fc1ada59b80 1 -- 172.21.15.16:0/77648605 >> 172.21.15.16:0/77648605 conn(0x5602a3164f80 msgr2=0x5602a31716f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:19:08.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.370+0000 7fc1ada59b80 1 -- 172.21.15.16:0/77648605 shutdown_connections 2023-05-20T01:19:08.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.370+0000 7fc1ada59b80 1 -- 172.21.15.16:0/77648605 wait complete. 2023-05-20T01:19:08.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.371+0000 7fc1ada59b80 1 -- start start 2023-05-20T01:19:08.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.371+0000 7fc1ada59b80 1 Processor -- start 2023-05-20T01:19:08.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.371+0000 7fc1ada59b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5602a316d8c0 0x5602a316eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:08.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.371+0000 7fc1ada59b80 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5602a3196750 0x5602a316f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:08.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.371+0000 7fc1ada59b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602a319a450 0x5602a316fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:08.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.371+0000 7fc1ada59b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602a3170d80 con 0x5602a319a450 2023-05-20T01:19:08.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.371+0000 7fc1ada59b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602a3196fe0 con 0x5602a316d8c0 2023-05-20T01:19:08.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.371+0000 7fc1ada59b80 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x5602a3197230 con 0x5602a3196750 2023-05-20T01:19:08.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.372+0000 7fc1ada59b80 10 client.0 _ll_register_callbacks cb 0x7fc184004370 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-05-20T01:19:08.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.372+0000 7fc19b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602a319a450 0x5602a316fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:08.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.372+0000 7fc19b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602a319a450 0x5602a316fed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33136/0 (socket says 172.21.15.16:33136) 2023-05-20T01:19:08.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.372+0000 7fc19b7fe700 1 -- 172.21.15.16:0/952225492 learned_addr learned my addr 172.21.15.16:0/952225492 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:19:08.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.372+0000 7fc19affd700 1 --2- 172.21.15.16:0/952225492 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5602a316d8c0 0x5602a316eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:08.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.372+0000 7fc19a7fc700 1 --2- 172.21.15.16:0/952225492 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5602a3196750 0x5602a316f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:08.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.372+0000 7fc19b7fe700 1 -- 172.21.15.16:0/952225492 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5602a3196750 msgr2=0x5602a316f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:19:08.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.372+0000 7fc19b7fe700 1 --2- 172.21.15.16:0/952225492 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5602a3196750 0x5602a316f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:08.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.372+0000 7fc19b7fe700 1 -- 172.21.15.16:0/952225492 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5602a316d8c0 msgr2=0x5602a316eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:19:08.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.372+0000 7fc19b7fe700 1 --2- 172.21.15.16:0/952225492 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5602a316d8c0 0x5602a316eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:08.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.372+0000 7fc19b7fe700 1 -- 172.21.15.16:0/952225492 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc19c0d1c10 con 0x5602a319a450 2023-05-20T01:19:08.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.372+0000 7fc19b7fe700 1 --2- 172.21.15.16:0/952225492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602a319a450 0x5602a316fed0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fc194000f80 tx=0x7fc194001510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c711faabbea669 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:08.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.372+0000 7fc17a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-05-20T01:19:08.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.372+0000 7fc17a7fc700 1 -- 172.21.15.16:0/952225492 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1940050d0 con 0x5602a319a450 2023-05-20T01:19:08.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.372+0000 7fc17a7fc700 1 -- 172.21.15.16:0/952225492 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(51..51 src has 1..51) v4 ==== 7102+0+0 (secure 0 0 0) 0x7fc194005230 con 0x5602a319a450 2023-05-20T01:19:08.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.372+0000 7fc1ada59b80 1 -- 172.21.15.16:0/952225492 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc184019070 con 0x5602a319a450 2023-05-20T01:19:08.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.372+0000 7fc17a7fc700 1 -- 172.21.15.16:0/952225492 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc194005410 con 0x5602a319a450 2023-05-20T01:19:08.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.372+0000 7fc1ada59b80 10 client.6006 fetch_fsmap learned FSMap version 29 2023-05-20T01:19:08.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.372+0000 7fc1ada59b80 1 -- 172.21.15.16:0/952225492 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) v3 -- 0x5602a2ed7c60 con 0x5602a319a450 2023-05-20T01:19:08.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.373+0000 7fc17a7fc700 1 -- 172.21.15.16:0/952225492 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc184019070 con 0x5602a319a450 2023-05-20T01:19:08.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.373+0000 7fc17a7fc700 1 -- 172.21.15.16:0/952225492 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 29) v1 ==== 3122+0+0 (secure 0 0 0) 0x7fc194012070 con 0x5602a319a450 2023-05-20T01:19:08.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.373+0000 7fc1ada59b80 10 client.6006 fetch_fsmap finished waiting for FSMap version 29 2023-05-20T01:19:08.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.373+0000 7fc1ada59b80 10 client.6006 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2023-05-20T01:19:08.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.373+0000 7fc1ada59b80 20 client.6006 populate_metadata read hostname 'smithi016' 2023-05-20T01:19:08.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.373+0000 7fc1ada59b80 1 --2- 172.21.15.16:0/952225492 >> [v2:172.21.15.16:6834/3284962389,v1:172.21.15.16:6836/3284962389] conn(0x5602a3219650 0x5602a3170410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:08.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.373+0000 7fc1ada59b80 4 client.6006 mds_command: new command op to 5889 tid=0[{"prefix": "scrub status"}] 2023-05-20T01:19:08.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.373+0000 7fc1ada59b80 1 -- 172.21.15.16:0/952225492 --> [v2:172.21.15.16:6834/3284962389,v1:172.21.15.16:6836/3284962389] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5602a308f660 con 0x5602a3219650 2023-05-20T01:19:08.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.373+0000 7fc19affd700 1 --2- 172.21.15.16:0/952225492 >> [v2:172.21.15.16:6834/3284962389,v1:172.21.15.16:6836/3284962389] conn(0x5602a3219650 0x5602a3170410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:08.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.373+0000 7fc17a7fc700 10 client.6006 ms_handle_connect on v2:172.21.15.16:6834/3284962389 2023-05-20T01:19:08.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.373+0000 7fc19affd700 1 --2- 172.21.15.16:0/952225492 >> [v2:172.21.15.16:6834/3284962389,v1:172.21.15.16:6836/3284962389] conn(0x5602a3219650 0x5602a3170410 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e864fa46d94839a server_cookie=8e4caa70dd63752c in_seq=0 out_seq=0 2023-05-20T01:19:08.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.373+0000 7fc17a7fc700 1 -- 172.21.15.16:0/952225492 <== mds.0 v2:172.21.15.16:6834/3284962389 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5602a308f660 con 0x5602a3219650 2023-05-20T01:19:08.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.373+0000 7fc17a7fc700 10 client.6006 handle_command_reply: tid=0 2023-05-20T01:19:08.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.373+0000 7fc1ada59b80 1 client.6006 shutdown 2023-05-20T01:19:08.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.374+0000 7fc1ada59b80 1 -- 172.21.15.16:0/952225492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602a319a450 msgr2=0x5602a316fed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:08.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.374+0000 7fc1ada59b80 1 --2- 172.21.15.16:0/952225492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602a319a450 0x5602a316fed0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fc194000f80 tx=0x7fc194001510 comp rx=0 tx=0).stop 2023-05-20T01:19:08.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.374+0000 7fc1ada59b80 1 -- 172.21.15.16:0/952225492 shutdown_connections 2023-05-20T01:19:08.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.374+0000 7fc1ada59b80 1 --2- 172.21.15.16:0/952225492 >> [v2:172.21.15.16:6834/3284962389,v1:172.21.15.16:6836/3284962389] conn(0x5602a3219650 0x5602a3170410 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:08.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.374+0000 7fc1ada59b80 1 --2- 172.21.15.16:0/952225492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602a319a450 0x5602a316fed0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:08.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.374+0000 7fc17a7fc700 0 client.6006 ms_handle_reset on v2:172.21.15.16:6834/3284962389 2023-05-20T01:19:08.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.374+0000 7fc1ada59b80 1 --2- 172.21.15.16:0/952225492 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5602a316d8c0 0x5602a316eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:08.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.374+0000 7fc1ada59b80 1 --2- 172.21.15.16:0/952225492 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5602a3196750 0x5602a316f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:08.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.374+0000 7fc1ada59b80 1 -- 172.21.15.16:0/952225492 >> 172.21.15.16:0/952225492 conn(0x5602a3164f80 msgr2=0x5602a31b5260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:19:08.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.374+0000 7fc1ada59b80 1 -- 172.21.15.16:0/952225492 shutdown_connections 2023-05-20T01:19:08.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.374+0000 7fc1ada59b80 1 -- 172.21.15.16:0/952225492 wait complete. 2023-05-20T01:19:08.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.374+0000 7fc1ada59b80 20 client.6006 trim_cache size 0 max 16384 2023-05-20T01:19:08.386 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-05-20T01:19:08.387 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2023-05-20T01:19:08.387 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2023-05-20T01:19:08.387 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-05-20T01:19:08.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.374+0000 7fc1a18cd700 1 -- 172.21.15.16:0/2779229089 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc17c06ace0 msgr2=0x7fc17c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:08.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.374+0000 7fc1a18cd700 1 --2- 172.21.15.16:0/2779229089 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc17c06ace0 0x7fc17c06d190 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fc18400d8b0 tx=0x7fc184008040 comp rx=0 tx=0).stop 2023-05-20T01:19:08.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.374+0000 7fc1a18cd700 1 -- 172.21.15.16:0/2779229089 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc19c0cfcb0 msgr2=0x7fc19c0cb5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:08.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.374+0000 7fc1a18cd700 1 --2- 172.21.15.16:0/2779229089 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc19c0cfcb0 0x7fc19c0cb5f0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fc18c000c00 tx=0x7fc18c004750 comp rx=0 tx=0).stop 2023-05-20T01:19:08.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.374+0000 7fc1a18cd700 1 -- 172.21.15.16:0/2779229089 shutdown_connections 2023-05-20T01:19:08.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.374+0000 7fc1a18cd700 1 --2- 172.21.15.16:0/2779229089 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc17c06ace0 0x7fc17c06d190 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:08.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.374+0000 7fc1a18cd700 1 --2- 172.21.15.16:0/2779229089 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc19c15cd50 0x7fc19c0cbb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:08.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.374+0000 7fc1a18cd700 1 --2- 172.21.15.16:0/2779229089 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc19c0cd920 0x7fc19c0b8160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:08.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.374+0000 7fc1a18cd700 1 --2- 172.21.15.16:0/2779229089 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc19c0cfcb0 0x7fc19c0cb5f0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:08.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.374+0000 7fc1a18cd700 1 -- 172.21.15.16:0/2779229089 >> 172.21.15.16:0/2779229089 conn(0x7fc19c06b070 msgr2=0x7fc19c160570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:19:08.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.374+0000 7fc1a18cd700 1 -- 172.21.15.16:0/2779229089 shutdown_connections 2023-05-20T01:19:08.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:08.374+0000 7fc1a18cd700 1 -- 172.21.15.16:0/2779229089 wait complete. 2023-05-20T01:19:08.390 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-05-20T01:19:08.390 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/0 2023-05-20T01:19:08.391 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-05-20T01:19:08.391 DEBUG:teuthology.orchestra.run.smithi016:> import zlib 2023-05-20T01:19:08.391 DEBUG:teuthology.orchestra.run.smithi016:> path = "/home/ubuntu/cephtest/mnt.0/subdir/0" 2023-05-20T01:19:08.392 DEBUG:teuthology.orchestra.run.smithi016:> with open(path, '"'"'r'"'"') as f: 2023-05-20T01:19:08.392 DEBUG:teuthology.orchestra.run.smithi016:> bytes = f.read() 2023-05-20T01:19:08.392 DEBUG:teuthology.orchestra.run.smithi016:> if len(bytes) != 6291456: 2023-05-20T01:19:08.392 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-05-20T01:19:08.393 DEBUG:teuthology.orchestra.run.smithi016:> len(bytes), 6291456 2023-05-20T01:19:08.393 DEBUG:teuthology.orchestra.run.smithi016:> )) 2023-05-20T01:19:08.393 DEBUG:teuthology.orchestra.run.smithi016:> for i, b in enumerate(bytes): 2023-05-20T01:19:08.394 DEBUG:teuthology.orchestra.run.smithi016:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-05-20T01:19:08.394 DEBUG:teuthology.orchestra.run.smithi016:> if b != chr(val): 2023-05-20T01:19:08.394 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-05-20T01:19:08.394 DEBUG:teuthology.orchestra.run.smithi016:> ' 2023-05-20T01:19:12.867 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/1 2023-05-20T01:19:12.867 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-05-20T01:19:12.868 DEBUG:teuthology.orchestra.run.smithi016:> import zlib 2023-05-20T01:19:12.869 DEBUG:teuthology.orchestra.run.smithi016:> path = "/home/ubuntu/cephtest/mnt.0/subdir/1" 2023-05-20T01:19:12.869 DEBUG:teuthology.orchestra.run.smithi016:> with open(path, '"'"'r'"'"') as f: 2023-05-20T01:19:12.869 DEBUG:teuthology.orchestra.run.smithi016:> bytes = f.read() 2023-05-20T01:19:12.870 DEBUG:teuthology.orchestra.run.smithi016:> if len(bytes) != 6291456: 2023-05-20T01:19:12.870 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-05-20T01:19:12.870 DEBUG:teuthology.orchestra.run.smithi016:> len(bytes), 6291456 2023-05-20T01:19:12.870 DEBUG:teuthology.orchestra.run.smithi016:> )) 2023-05-20T01:19:12.871 DEBUG:teuthology.orchestra.run.smithi016:> for i, b in enumerate(bytes): 2023-05-20T01:19:12.871 DEBUG:teuthology.orchestra.run.smithi016:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-05-20T01:19:12.871 DEBUG:teuthology.orchestra.run.smithi016:> if b != chr(val): 2023-05-20T01:19:12.872 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-05-20T01:19:12.872 DEBUG:teuthology.orchestra.run.smithi016:> ' 2023-05-20T01:19:17.379 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/2 2023-05-20T01:19:17.379 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-05-20T01:19:17.380 DEBUG:teuthology.orchestra.run.smithi016:> import zlib 2023-05-20T01:19:17.380 DEBUG:teuthology.orchestra.run.smithi016:> path = "/home/ubuntu/cephtest/mnt.0/subdir/2" 2023-05-20T01:19:17.380 DEBUG:teuthology.orchestra.run.smithi016:> with open(path, '"'"'r'"'"') as f: 2023-05-20T01:19:17.381 DEBUG:teuthology.orchestra.run.smithi016:> bytes = f.read() 2023-05-20T01:19:17.381 DEBUG:teuthology.orchestra.run.smithi016:> if len(bytes) != 6291456: 2023-05-20T01:19:17.381 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-05-20T01:19:17.382 DEBUG:teuthology.orchestra.run.smithi016:> len(bytes), 6291456 2023-05-20T01:19:17.382 DEBUG:teuthology.orchestra.run.smithi016:> )) 2023-05-20T01:19:17.382 DEBUG:teuthology.orchestra.run.smithi016:> for i, b in enumerate(bytes): 2023-05-20T01:19:17.383 DEBUG:teuthology.orchestra.run.smithi016:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-05-20T01:19:17.383 DEBUG:teuthology.orchestra.run.smithi016:> if b != chr(val): 2023-05-20T01:19:17.383 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-05-20T01:19:17.383 DEBUG:teuthology.orchestra.run.smithi016:> ' 2023-05-20T01:19:21.890 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/3 2023-05-20T01:19:21.891 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-05-20T01:19:21.891 DEBUG:teuthology.orchestra.run.smithi016:> import zlib 2023-05-20T01:19:21.891 DEBUG:teuthology.orchestra.run.smithi016:> path = "/home/ubuntu/cephtest/mnt.0/subdir/3" 2023-05-20T01:19:21.892 DEBUG:teuthology.orchestra.run.smithi016:> with open(path, '"'"'r'"'"') as f: 2023-05-20T01:19:21.892 DEBUG:teuthology.orchestra.run.smithi016:> bytes = f.read() 2023-05-20T01:19:21.892 DEBUG:teuthology.orchestra.run.smithi016:> if len(bytes) != 6291456: 2023-05-20T01:19:21.893 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-05-20T01:19:21.893 DEBUG:teuthology.orchestra.run.smithi016:> len(bytes), 6291456 2023-05-20T01:19:21.893 DEBUG:teuthology.orchestra.run.smithi016:> )) 2023-05-20T01:19:21.893 DEBUG:teuthology.orchestra.run.smithi016:> for i, b in enumerate(bytes): 2023-05-20T01:19:21.894 DEBUG:teuthology.orchestra.run.smithi016:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-05-20T01:19:21.894 DEBUG:teuthology.orchestra.run.smithi016:> if b != chr(val): 2023-05-20T01:19:21.894 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-05-20T01:19:21.894 DEBUG:teuthology.orchestra.run.smithi016:> ' 2023-05-20T01:19:26.403 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/4 2023-05-20T01:19:26.404 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-05-20T01:19:26.404 DEBUG:teuthology.orchestra.run.smithi016:> import zlib 2023-05-20T01:19:26.404 DEBUG:teuthology.orchestra.run.smithi016:> path = "/home/ubuntu/cephtest/mnt.0/subdir/4" 2023-05-20T01:19:26.404 DEBUG:teuthology.orchestra.run.smithi016:> with open(path, '"'"'r'"'"') as f: 2023-05-20T01:19:26.405 DEBUG:teuthology.orchestra.run.smithi016:> bytes = f.read() 2023-05-20T01:19:26.405 DEBUG:teuthology.orchestra.run.smithi016:> if len(bytes) != 6291456: 2023-05-20T01:19:26.405 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-05-20T01:19:26.406 DEBUG:teuthology.orchestra.run.smithi016:> len(bytes), 6291456 2023-05-20T01:19:26.406 DEBUG:teuthology.orchestra.run.smithi016:> )) 2023-05-20T01:19:26.406 DEBUG:teuthology.orchestra.run.smithi016:> for i, b in enumerate(bytes): 2023-05-20T01:19:26.406 DEBUG:teuthology.orchestra.run.smithi016:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-05-20T01:19:26.407 DEBUG:teuthology.orchestra.run.smithi016:> if b != chr(val): 2023-05-20T01:19:26.407 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-05-20T01:19:26.407 DEBUG:teuthology.orchestra.run.smithi016:> ' 2023-05-20T01:19:30.915 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-05-20T01:19:30.916 DEBUG:teuthology.orchestra.run.smithi016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-05-20T01:19:30.998 DEBUG:teuthology.orchestra.run.smithi132:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-05-20T01:19:31.049 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-05-20T01:19:31.049 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-05-20T01:19:31.049 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2023-05-20T01:19:31.067 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-05-20T01:19:31.068 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2023-05-20T01:19:31.125 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-20T01:19:31.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.286+0000 7f0eae8d3700 1 Processor -- start 2023-05-20T01:19:31.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.286+0000 7f0eae8d3700 1 -- start start 2023-05-20T01:19:31.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.286+0000 7f0eae8d3700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f0ea80d1000 0x7f0ea80d13d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:31.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.286+0000 7f0eae8d3700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0ea806b620 0x7f0ea806ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:31.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.287+0000 7f0eae8d3700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0ea80b09d0 0x7f0ea80b0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:31.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.287+0000 7f0eae8d3700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ea80ce3e0 con 0x7f0ea80b09d0 2023-05-20T01:19:31.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.287+0000 7f0eae8d3700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ea80d1910 con 0x7f0ea80d1000 2023-05-20T01:19:31.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.287+0000 7f0eae8d3700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ea80cdab0 con 0x7f0ea806b620 2023-05-20T01:19:31.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.287+0000 7f0eace70700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0ea80b09d0 0x7f0ea80b0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:31.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.287+0000 7f0eace70700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0ea80b09d0 0x7f0ea80b0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33150/0 (socket says 172.21.15.16:33150) 2023-05-20T01:19:31.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.287+0000 7f0eace70700 1 -- 172.21.15.16:0/3775638340 learned_addr learned my addr 172.21.15.16:0/3775638340 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:19:31.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.287+0000 7f0ea77fe700 1 --2- 172.21.15.16:0/3775638340 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0ea806b620 0x7f0ea806ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:31.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.287+0000 7f0eace70700 1 -- 172.21.15.16:0/3775638340 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0ea806b620 msgr2=0x7f0ea806ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:19:31.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.287+0000 7f0eace70700 1 --2- 172.21.15.16:0/3775638340 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0ea806b620 0x7f0ea806ba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:31.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.287+0000 7f0eace70700 1 -- 172.21.15.16:0/3775638340 >> v2:172.21.15.132:3300/0 conn(0x7f0ea80d1000 msgr2=0x7f0ea80d13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:19:31.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.287+0000 7f0ea7fff700 1 --2- 172.21.15.16:0/3775638340 >> v2:172.21.15.132:3300/0 conn(0x7f0ea80d1000 0x7f0ea80d13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:31.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.287+0000 7f0eace70700 1 --2- 172.21.15.16:0/3775638340 >> v2:172.21.15.132:3300/0 conn(0x7f0ea80d1000 0x7f0ea80d13d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:31.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.287+0000 7f0eace70700 1 -- 172.21.15.16:0/3775638340 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ea80cdd90 con 0x7f0ea80b09d0 2023-05-20T01:19:31.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.287+0000 7f0ea7fff700 1 --2- 172.21.15.16:0/3775638340 >> v2:172.21.15.132:3300/0 conn(0x7f0ea80d1000 0x7f0ea80d13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:19:31.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.287+0000 7f0eace70700 1 --2- 172.21.15.16:0/3775638340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ea80b09d0 0x7f0ea80b0e50 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f0ea8073b10 tx=0x7f0ea000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcf5d151e92e46b8 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:31.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.287+0000 7f0ea6ffd700 1 -- 172.21.15.16:0/3775638340 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ea000e070 con 0x7f0ea80b09d0 2023-05-20T01:19:31.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.287+0000 7f0ea6ffd700 1 -- 172.21.15.16:0/3775638340 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ea000b9c0 con 0x7f0ea80b09d0 2023-05-20T01:19:31.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.287+0000 7f0ea6ffd700 1 -- 172.21.15.16:0/3775638340 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ea000bd70 con 0x7f0ea80b09d0 2023-05-20T01:19:31.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.288+0000 7f0eae8d3700 1 -- 172.21.15.16:0/3775638340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ea80b09d0 msgr2=0x7f0ea80b0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:31.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.288+0000 7f0eae8d3700 1 --2- 172.21.15.16:0/3775638340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ea80b09d0 0x7f0ea80b0e50 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f0ea8073b10 tx=0x7f0ea000b330 comp rx=0 tx=0).stop 2023-05-20T01:19:31.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.288+0000 7f0eae8d3700 1 -- 172.21.15.16:0/3775638340 shutdown_connections 2023-05-20T01:19:31.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.288+0000 7f0eae8d3700 1 --2- 172.21.15.16:0/3775638340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ea80b09d0 0x7f0ea80b0e50 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:31.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.288+0000 7f0eae8d3700 1 --2- 172.21.15.16:0/3775638340 >> v2:172.21.15.132:3300/0 conn(0x7f0ea80d1000 0x7f0ea80d13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:31.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.288+0000 7f0eae8d3700 1 --2- 172.21.15.16:0/3775638340 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0ea806b620 0x7f0ea806ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:31.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.288+0000 7f0eae8d3700 1 -- 172.21.15.16:0/3775638340 >> 172.21.15.16:0/3775638340 conn(0x7f0ea806ae30 msgr2=0x7f0ea809a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:19:31.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.288+0000 7f0eae8d3700 1 -- 172.21.15.16:0/3775638340 shutdown_connections 2023-05-20T01:19:31.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.288+0000 7f0eae8d3700 1 -- 172.21.15.16:0/3775638340 wait complete. 2023-05-20T01:19:31.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.289+0000 7f0eae8d3700 1 Processor -- start 2023-05-20T01:19:31.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.289+0000 7f0eae8d3700 1 -- start start 2023-05-20T01:19:31.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.290+0000 7f0eae8d3700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0ea806b620 0x7f0ea80c65e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:31.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.290+0000 7f0eae8d3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ea80b09d0 0x7f0ea8147620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:31.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.290+0000 7f0eae8d3700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0ea80d1000 0x7f0ea809af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:31.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.290+0000 7f0eae8d3700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ea80c6e10 con 0x7f0ea80b09d0 2023-05-20T01:19:31.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.290+0000 7f0eae8d3700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ea80c70f0 con 0x7f0ea80d1000 2023-05-20T01:19:31.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.290+0000 7f0eae8d3700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ea80c73d0 con 0x7f0ea806b620 2023-05-20T01:19:31.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.290+0000 7f0ea77fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ea80b09d0 0x7f0ea8147620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:31.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.290+0000 7f0ea77fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ea80b09d0 0x7f0ea8147620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33154/0 (socket says 172.21.15.16:33154) 2023-05-20T01:19:31.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.290+0000 7f0ea77fe700 1 -- 172.21.15.16:0/2358759875 learned_addr learned my addr 172.21.15.16:0/2358759875 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:19:31.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.290+0000 7f0ea7fff700 1 --2- 172.21.15.16:0/2358759875 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0ea806b620 0x7f0ea80c65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:31.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.290+0000 7f0eace70700 1 --2- 172.21.15.16:0/2358759875 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0ea80d1000 0x7f0ea809af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:31.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.291+0000 7f0ea77fe700 1 -- 172.21.15.16:0/2358759875 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0ea806b620 msgr2=0x7f0ea80c65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:31.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.291+0000 7f0ea77fe700 1 --2- 172.21.15.16:0/2358759875 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0ea806b620 0x7f0ea80c65e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:31.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.291+0000 7f0ea77fe700 1 -- 172.21.15.16:0/2358759875 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0ea80d1000 msgr2=0x7f0ea809af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:31.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.291+0000 7f0ea77fe700 1 --2- 172.21.15.16:0/2358759875 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0ea80d1000 0x7f0ea809af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:31.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.291+0000 7f0ea77fe700 1 -- 172.21.15.16:0/2358759875 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ea0009b30 con 0x7f0ea80b09d0 2023-05-20T01:19:31.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.291+0000 7f0ea77fe700 1 --2- 172.21.15.16:0/2358759875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ea80b09d0 0x7f0ea8147620 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c00c7f0 tx=0x7f0e9c00cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:31.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.291+0000 7f0ea4ff9700 1 -- 172.21.15.16:0/2358759875 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e9c011800 con 0x7f0ea80b09d0 2023-05-20T01:19:31.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.291+0000 7f0ea4ff9700 1 -- 172.21.15.16:0/2358759875 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e9c011960 con 0x7f0ea80b09d0 2023-05-20T01:19:31.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.291+0000 7f0ea4ff9700 1 -- 172.21.15.16:0/2358759875 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e9c011cf0 con 0x7f0ea80b09d0 2023-05-20T01:19:31.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.291+0000 7f0eae8d3700 1 -- 172.21.15.16:0/2358759875 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ea80b28b0 con 0x7f0ea80b09d0 2023-05-20T01:19:31.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.291+0000 7f0eae8d3700 1 -- 172.21.15.16:0/2358759875 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ea80ce070 con 0x7f0ea80b09d0 2023-05-20T01:19:31.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.292+0000 7f0eae8d3700 1 -- 172.21.15.16:0/2358759875 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ea80ce870 con 0x7f0ea80b09d0 2023-05-20T01:19:31.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.292+0000 7f0ea4ff9700 1 -- 172.21.15.16:0/2358759875 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0e9c018360 con 0x7f0ea80b09d0 2023-05-20T01:19:31.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.293+0000 7f0ea4ff9700 1 --2- 172.21.15.16:0/2358759875 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f0e9406ace0 0x7f0e9406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:31.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.293+0000 7f0ea4ff9700 1 -- 172.21.15.16:0/2358759875 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 7102+0+0 (secure 0 0 0) 0x7f0e9c089630 con 0x7f0ea80b09d0 2023-05-20T01:19:31.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.293+0000 7f0ea7fff700 1 --2- 172.21.15.16:0/2358759875 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f0e9406ace0 0x7f0e9406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:31.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.293+0000 7f0ea7fff700 1 --2- 172.21.15.16:0/2358759875 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f0e9406ace0 0x7f0e9406d190 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f0e98006fd0 tx=0x7f0e98008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:31.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.297+0000 7f0ea4ff9700 1 -- 172.21.15.16:0/2358759875 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f0e9c0542d0 con 0x7f0ea80b09d0 2023-05-20T01:19:31.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.501+0000 7f0eae8d3700 1 -- 172.21.15.16:0/2358759875 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0ea80c7590 con 0x7f0ea80b09d0 2023-05-20T01:19:31.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.502+0000 7f0ea4ff9700 1 -- 172.21.15.16:0/2358759875 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) v1 ==== 74+0+20668 (secure 0 0 0) 0x7f0e9c0578f0 con 0x7f0ea80b09d0 2023-05-20T01:19:31.507 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:19:31.507 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":51,"fsid":"c2245cc5-ecab-4087-a59b-bc81db901676","created":"2023-05-20T01:16:09.549230+0000","modified":"2023-05-20T01:18:31.674809+0000","last_up_change":"2023-05-20T01:16:19.788624+0000","last_in_change":"2023-05-20T01:16:12.092160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-05-20T01:16:20.934366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-05-20T01:16:21.372294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-05-20T01:17:44.628755+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-05-20T01:17:45.968584+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data_extra","create_time":"2023-05-20T01:17:54.915781+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"317a83cd-19c8-4958-a301-6087ef342289","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6825","nonce":3923634179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6827","nonce":3923634179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6831","nonce":3923634179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6829","nonce":3923634179}]},"public_addr":"172.21.15.16:6825/3923634179","cluster_addr":"172.21.15.16:6827/3923634179","heartbeat_back_addr":"172.21.15.16:6831/3923634179","heartbeat_front_addr":"172.21.15.16:6829/3923634179","state":["exists","up"]},{"osd":1,"uuid":"5581046f-8219-4089-8d7d-b7846399505e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6809","nonce":3860264220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6811","nonce":3860264220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6815","nonce":3860264220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6813","nonce":3860264220}]},"public_addr":"172.21.15.16:6809/3860264220","cluster_addr":"172.21.15.16:6811/3860264220","heartbeat_back_addr":"172.21.15.16:6815/3860264220","heartbeat_front_addr":"172.21.15.16:6813/3860264220","state":["exists","up"]},{"osd":2,"uuid":"072e9e66-4133-478e-8d4a-64d46c2963f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6817","nonce":293187030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6819","nonce":293187030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6823","nonce":293187030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6821","nonce":293187030}]},"public_addr":"172.21.15.16:6817/293187030","cluster_addr":"172.21.15.16:6819/293187030","heartbeat_back_addr":"172.21.15.16:6823/293187030","heartbeat_front_addr":"172.21.15.16:6821/293187030","state":["exists","up"]},{"osd":3,"uuid":"11e1bba6-00f8-48dd-9028-d8d716b7f145","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6801","nonce":3669781890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6803","nonce":3669781890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6807","nonce":3669781890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6805","nonce":3669781890}]},"public_addr":"172.21.15.16:6801/3669781890","cluster_addr":"172.21.15.16:6803/3669781890","heartbeat_back_addr":"172.21.15.16:6807/3669781890","heartbeat_front_addr":"172.21.15.16:6805/3669781890","state":["exists","up"]},{"osd":4,"uuid":"b8909c3a-cf70-46cb-8d0b-009c707cf9d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6819","nonce":506317223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6823","nonce":506317223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6829","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6831","nonce":506317223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6827","nonce":506317223}]},"public_addr":"172.21.15.132:6819/506317223","cluster_addr":"172.21.15.132:6823/506317223","heartbeat_back_addr":"172.21.15.132:6831/506317223","heartbeat_front_addr":"172.21.15.132:6827/506317223","state":["exists","up"]},{"osd":5,"uuid":"b1818304-4583-47bf-a6ec-0bfbb5166233","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6818","nonce":1844689995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6822","nonce":1844689995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6830","nonce":1844689995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6826","nonce":1844689995}]},"public_addr":"172.21.15.132:6818/1844689995","cluster_addr":"172.21.15.132:6822/1844689995","heartbeat_back_addr":"172.21.15.132:6830/1844689995","heartbeat_front_addr":"172.21.15.132:6826/1844689995","state":["exists","up"]},{"osd":6,"uuid":"d37f4691-eed1-4a93-8818-18354f03f4a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6801","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6802","nonce":727581123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6805","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6807","nonce":727581123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6813","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6815","nonce":727581123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6811","nonce":727581123}]},"public_addr":"172.21.15.132:6802/727581123","cluster_addr":"172.21.15.132:6807/727581123","heartbeat_back_addr":"172.21.15.132:6815/727581123","heartbeat_front_addr":"172.21.15.132:6811/727581123","state":["exists","up"]},{"osd":7,"uuid":"4f712f62-5d1d-4816-8042-03209144f8b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6803","nonce":466001149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6806","nonce":466001149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6814","nonce":466001149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6810","nonce":466001149}]},"public_addr":"172.21.15.132:6803/466001149","cluster_addr":"172.21.15.132:6806/466001149","heartbeat_back_addr":"172.21.15.132:6814/466001149","heartbeat_front_addr":"172.21.15.132:6810/466001149","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.338863+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.278680+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305694+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.246835+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.276237+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.304441+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305421+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.267050+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.9","mappings":[{"from":0,"to":4}]},{"pgid":"7.1b","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6837/1867776973":"2023-05-21T01:18:31.666399+0000","172.21.15.16:6835/1867776973":"2023-05-21T01:18:31.666399+0000","172.21.15.16:6837/3517262366":"2023-05-21T01:18:26.467438+0000","172.21.15.16:6835/3517262366":"2023-05-21T01:18:26.467438+0000","172.21.15.16:6835/942011026":"2023-05-21T01:18:24.644805+0000","172.21.15.16:6837/942011026":"2023-05-21T01:18:24.644805+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-20T01:19:31.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.505+0000 7f0eae8d3700 1 -- 172.21.15.16:0/2358759875 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f0e9406ace0 msgr2=0x7f0e9406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:31.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.505+0000 7f0eae8d3700 1 --2- 172.21.15.16:0/2358759875 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f0e9406ace0 0x7f0e9406d190 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f0e98006fd0 tx=0x7f0e98008040 comp rx=0 tx=0).stop 2023-05-20T01:19:31.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.505+0000 7f0eae8d3700 1 -- 172.21.15.16:0/2358759875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ea80b09d0 msgr2=0x7f0ea8147620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:31.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.505+0000 7f0eae8d3700 1 --2- 172.21.15.16:0/2358759875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ea80b09d0 0x7f0ea8147620 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c00c7f0 tx=0x7f0e9c00cb00 comp rx=0 tx=0).stop 2023-05-20T01:19:31.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.505+0000 7f0eae8d3700 1 -- 172.21.15.16:0/2358759875 shutdown_connections 2023-05-20T01:19:31.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.505+0000 7f0eae8d3700 1 --2- 172.21.15.16:0/2358759875 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0ea80d1000 0x7f0ea809af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:31.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.505+0000 7f0eae8d3700 1 --2- 172.21.15.16:0/2358759875 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f0e9406ace0 0x7f0e9406d190 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:31.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.505+0000 7f0eae8d3700 1 --2- 172.21.15.16:0/2358759875 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0ea806b620 0x7f0ea80c65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:31.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.505+0000 7f0eae8d3700 1 --2- 172.21.15.16:0/2358759875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ea80b09d0 0x7f0ea8147620 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:31.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.505+0000 7f0eae8d3700 1 -- 172.21.15.16:0/2358759875 >> 172.21.15.16:0/2358759875 conn(0x7f0ea806ae30 msgr2=0x7f0ea80adfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:19:31.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.505+0000 7f0eae8d3700 1 -- 172.21.15.16:0/2358759875 shutdown_connections 2023-05-20T01:19:31.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.505+0000 7f0eae8d3700 1 -- 172.21.15.16:0/2358759875 wait complete. 2023-05-20T01:19:31.521 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-05-20T01:16:20.934366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-05-20T01:16:21.372294+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2023-05-20T01:17:44.628755+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '39', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2023-05-20T01:17:45.968584+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '40', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data_extra', 'create_time': '2023-05-20T01:17:54.915781+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '43', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-05-20T01:19:31.522 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-05-20T01:19:31.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.645+0000 7fc9e3901700 1 Processor -- start 2023-05-20T01:19:31.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.646+0000 7fc9e3901700 1 -- start start 2023-05-20T01:19:31.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.646+0000 7fc9e3901700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc9dc159b50 0x7fc9dc06e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:31.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.646+0000 7fc9e3901700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fc9dc0717c0 0x7fc9dc0ca870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:31.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.646+0000 7fc9e3901700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9dc0ccd30 con 0x7fc9dc0cd1c0 2023-05-20T01:19:31.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.646+0000 7fc9e3901700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9dc0ce990 con 0x7fc9dc0717c0 2023-05-20T01:19:31.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.646+0000 7fc9e3901700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9dc0cc6e0 con 0x7fc9dc159b50 2023-05-20T01:19:31.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.647+0000 7fc9e0e9c700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fc9dc0cd1c0 0x7fc9dc0710b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:44688/0 (socket says 172.21.15.16:44688) 2023-05-20T01:19:31.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.647+0000 7fc9e0e9c700 1 -- 172.21.15.16:0/3408242806 learned_addr learned my addr 172.21.15.16:0/3408242806 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:19:31.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.647+0000 7fc9e169d700 1 --2- 172.21.15.16:0/3408242806 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc9dc159b50 0x7fc9dc06e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:31.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.647+0000 7fc9e1e9e700 1 --2- 172.21.15.16:0/3408242806 >> v2:172.21.15.132:3300/0 conn(0x7fc9dc0717c0 0x7fc9dc0ca870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:31.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.647+0000 7fc9d3fff700 1 -- 172.21.15.16:0/3408242806 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2301022361 0 0) 0x7fc9dc0ccd30 con 0x7fc9dc0cd1c0 2023-05-20T01:19:31.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.647+0000 7fc9d3fff700 1 -- 172.21.15.16:0/3408242806 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc9c80035b0 con 0x7fc9dc0cd1c0 2023-05-20T01:19:31.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.647+0000 7fc9e169d700 1 -- 172.21.15.16:0/3408242806 >> v2:172.21.15.132:3300/0 conn(0x7fc9dc0717c0 msgr2=0x7fc9dc0ca870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:19:31.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.647+0000 7fc9e169d700 1 --2- 172.21.15.16:0/3408242806 >> v2:172.21.15.132:3300/0 conn(0x7fc9dc0717c0 0x7fc9dc0ca870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:31.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.647+0000 7fc9e169d700 1 -- 172.21.15.16:0/3408242806 >> v1:172.21.15.16:6789/0 conn(0x7fc9dc0cd1c0 legacy=0x7fc9dc0710b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:31.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.647+0000 7fc9d3fff700 1 -- 172.21.15.16:0/3408242806 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2830155872 0 0) 0x7fc9c80035b0 con 0x7fc9dc0cd1c0 2023-05-20T01:19:31.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.647+0000 7fc9e169d700 1 -- 172.21.15.16:0/3408242806 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9dc0cc9c0 con 0x7fc9dc159b50 2023-05-20T01:19:31.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.647+0000 7fc9e169d700 1 --2- 172.21.15.16:0/3408242806 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc9dc159b50 0x7fc9dc06e0a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc009820 tx=0x7fc9cc009b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75cc07b974f1f67 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:31.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.647+0000 7fc9d3fff700 1 -- 172.21.15.16:0/3408242806 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9d8001670 con 0x7fc9dc159b50 2023-05-20T01:19:31.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.647+0000 7fc9d3fff700 1 -- 172.21.15.16:0/3408242806 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9cc005c30 con 0x7fc9dc159b50 2023-05-20T01:19:31.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.647+0000 7fc9d3fff700 1 -- 172.21.15.16:0/3408242806 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9cc004030 con 0x7fc9dc159b50 2023-05-20T01:19:31.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.648+0000 7fc9e3901700 1 -- 172.21.15.16:0/3408242806 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc9dc159b50 msgr2=0x7fc9dc06e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:31.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.648+0000 7fc9e3901700 1 --2- 172.21.15.16:0/3408242806 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc9dc159b50 0x7fc9dc06e0a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc009820 tx=0x7fc9cc009b30 comp rx=0 tx=0).stop 2023-05-20T01:19:31.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.648+0000 7fc9e3901700 1 -- 172.21.15.16:0/3408242806 shutdown_connections 2023-05-20T01:19:31.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.648+0000 7fc9e3901700 1 --2- 172.21.15.16:0/3408242806 >> v2:172.21.15.132:3300/0 conn(0x7fc9dc0717c0 0x7fc9dc0ca870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:31.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.648+0000 7fc9e3901700 1 --2- 172.21.15.16:0/3408242806 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc9dc159b50 0x7fc9dc06e0a0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:31.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.648+0000 7fc9e3901700 1 -- 172.21.15.16:0/3408242806 >> 172.21.15.16:0/3408242806 conn(0x7fc9dc0cffd0 msgr2=0x7fc9dc0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:19:31.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.648+0000 7fc9e3901700 1 -- 172.21.15.16:0/3408242806 shutdown_connections 2023-05-20T01:19:31.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.648+0000 7fc9e3901700 1 -- 172.21.15.16:0/3408242806 wait complete. 2023-05-20T01:19:31.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.649+0000 7fc9e3901700 1 Processor -- start 2023-05-20T01:19:31.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.649+0000 7fc9e3901700 1 -- start start 2023-05-20T01:19:31.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.650+0000 7fc9e3901700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc9dc0717c0 0x7fc9dc152970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:31.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.650+0000 7fc9e3901700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9dc0cd1c0 0x7fc9dc1f30e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:31.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.650+0000 7fc9e3901700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc9dc159b50 0x7fc9dc155dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:31.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.650+0000 7fc9e3901700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9dc158c60 con 0x7fc9dc0cd1c0 2023-05-20T01:19:31.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.650+0000 7fc9e3901700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9dc158f40 con 0x7fc9dc0717c0 2023-05-20T01:19:31.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.650+0000 7fc9e3901700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9dc159220 con 0x7fc9dc159b50 2023-05-20T01:19:31.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.650+0000 7fc9e0e9c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9dc0cd1c0 0x7fc9dc1f30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:31.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.650+0000 7fc9e0e9c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9dc0cd1c0 0x7fc9dc1f30e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33166/0 (socket says 172.21.15.16:33166) 2023-05-20T01:19:31.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.650+0000 7fc9e0e9c700 1 -- 172.21.15.16:0/552540823 learned_addr learned my addr 172.21.15.16:0/552540823 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:19:31.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.650+0000 7fc9e169d700 1 --2- 172.21.15.16:0/552540823 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc9dc0717c0 0x7fc9dc152970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:31.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.650+0000 7fc9e1e9e700 1 --2- 172.21.15.16:0/552540823 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc9dc159b50 0x7fc9dc155dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:31.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.651+0000 7fc9e0e9c700 1 -- 172.21.15.16:0/552540823 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc9dc159b50 msgr2=0x7fc9dc155dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:31.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.651+0000 7fc9e0e9c700 1 --2- 172.21.15.16:0/552540823 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc9dc159b50 0x7fc9dc155dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:31.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.651+0000 7fc9e0e9c700 1 -- 172.21.15.16:0/552540823 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc9dc0717c0 msgr2=0x7fc9dc152970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:31.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.651+0000 7fc9e0e9c700 1 --2- 172.21.15.16:0/552540823 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc9dc0717c0 0x7fc9dc152970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:31.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.651+0000 7fc9e0e9c700 1 -- 172.21.15.16:0/552540823 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9cc009430 con 0x7fc9dc0cd1c0 2023-05-20T01:19:31.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.651+0000 7fc9e0e9c700 1 --2- 172.21.15.16:0/552540823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9dc0cd1c0 0x7fc9dc1f30e0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fc9d800c780 tx=0x7fc9d800ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:31.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.651+0000 7fc9d1ffb700 1 -- 172.21.15.16:0/552540823 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9d8004c80 con 0x7fc9dc0cd1c0 2023-05-20T01:19:31.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.651+0000 7fc9d1ffb700 1 -- 172.21.15.16:0/552540823 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9d8004de0 con 0x7fc9dc0cd1c0 2023-05-20T01:19:31.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.651+0000 7fc9e3901700 1 -- 172.21.15.16:0/552540823 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9dc071e80 con 0x7fc9dc0cd1c0 2023-05-20T01:19:31.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.651+0000 7fc9d1ffb700 1 -- 172.21.15.16:0/552540823 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9d80072f0 con 0x7fc9dc0cd1c0 2023-05-20T01:19:31.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.651+0000 7fc9e3901700 1 -- 172.21.15.16:0/552540823 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9dc06e5e0 con 0x7fc9dc0cd1c0 2023-05-20T01:19:31.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.652+0000 7fc9e3901700 1 -- 172.21.15.16:0/552540823 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9dc0cec70 con 0x7fc9dc0cd1c0 2023-05-20T01:19:31.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.653+0000 7fc9d1ffb700 1 -- 172.21.15.16:0/552540823 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc9d8007450 con 0x7fc9dc0cd1c0 2023-05-20T01:19:31.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.653+0000 7fc9d1ffb700 1 --2- 172.21.15.16:0/552540823 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc9c006ace0 0x7fc9c006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:31.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.653+0000 7fc9e169d700 1 --2- 172.21.15.16:0/552540823 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc9c006ace0 0x7fc9c006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:31.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.653+0000 7fc9d1ffb700 1 -- 172.21.15.16:0/552540823 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 7102+0+0 (secure 0 0 0) 0x7fc9d8089560 con 0x7fc9dc0cd1c0 2023-05-20T01:19:31.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.654+0000 7fc9e169d700 1 --2- 172.21.15.16:0/552540823 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc9c006ace0 0x7fc9c006d190 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc000c00 tx=0x7fc9cc008070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:31.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.658+0000 7fc9d1ffb700 1 -- 172.21.15.16:0/552540823 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fc9d8054290 con 0x7fc9dc0cd1c0 2023-05-20T01:19:31.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.854+0000 7fc9e3901700 1 -- 172.21.15.16:0/552540823 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fc9dc067f90 con 0x7fc9dc0cd1c0 2023-05-20T01:19:31.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.854+0000 7fc9d1ffb700 1 -- 172.21.15.16:0/552540823 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v51) v1 ==== 92+0+10 (secure 0 0 0) 0x7fc9d8007700 con 0x7fc9dc0cd1c0 2023-05-20T01:19:31.859 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 8 2023-05-20T01:19:31.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.856+0000 7fc9e3901700 1 -- 172.21.15.16:0/552540823 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc9c006ace0 msgr2=0x7fc9c006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:31.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.856+0000 7fc9e3901700 1 --2- 172.21.15.16:0/552540823 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc9c006ace0 0x7fc9c006d190 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc000c00 tx=0x7fc9cc008070 comp rx=0 tx=0).stop 2023-05-20T01:19:31.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.856+0000 7fc9e3901700 1 -- 172.21.15.16:0/552540823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9dc0cd1c0 msgr2=0x7fc9dc1f30e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:31.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.856+0000 7fc9e3901700 1 --2- 172.21.15.16:0/552540823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9dc0cd1c0 0x7fc9dc1f30e0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fc9d800c780 tx=0x7fc9d800ca90 comp rx=0 tx=0).stop 2023-05-20T01:19:31.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.856+0000 7fc9e3901700 1 -- 172.21.15.16:0/552540823 shutdown_connections 2023-05-20T01:19:31.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.856+0000 7fc9e3901700 1 --2- 172.21.15.16:0/552540823 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc9c006ace0 0x7fc9c006d190 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:31.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.856+0000 7fc9e3901700 1 --2- 172.21.15.16:0/552540823 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc9dc159b50 0x7fc9dc155dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:31.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.856+0000 7fc9e3901700 1 --2- 172.21.15.16:0/552540823 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc9dc0717c0 0x7fc9dc152970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:31.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.856+0000 7fc9e3901700 1 --2- 172.21.15.16:0/552540823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9dc0cd1c0 0x7fc9dc1f30e0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:31.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.857+0000 7fc9e3901700 1 -- 172.21.15.16:0/552540823 >> 172.21.15.16:0/552540823 conn(0x7fc9dc0cffd0 msgr2=0x7fc9dc0710b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:19:31.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.857+0000 7fc9e3901700 1 -- 172.21.15.16:0/552540823 shutdown_connections 2023-05-20T01:19:31.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.857+0000 7fc9e3901700 1 -- 172.21.15.16:0/552540823 wait complete. 2023-05-20T01:19:31.872 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-05-20T01:19:32.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.996+0000 7f5c19f8d700 1 Processor -- start 2023-05-20T01:19:32.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.996+0000 7f5c19f8d700 1 -- start start 2023-05-20T01:19:32.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.997+0000 7f5c19f8d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5c140d16e0 0x7f5c14144350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:32.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.997+0000 7f5c19f8d700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5c140cec40 0x7f5c140cf0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:32.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.997+0000 7f5c19f8d700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c14144890 con 0x7f5c140ae2e0 2023-05-20T01:19:32.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.997+0000 7f5c19f8d700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c140cf5f0 con 0x7f5c140d16e0 2023-05-20T01:19:32.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.997+0000 7f5c19f8d700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c140b2020 con 0x7f5c140cec40 2023-05-20T01:19:32.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.997+0000 7f5c137fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5c140d16e0 0x7f5c14144350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:32.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.997+0000 7f5c137fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5c140d16e0 0x7f5c14144350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33172/0 (socket says 172.21.15.16:33172) 2023-05-20T01:19:32.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.997+0000 7f5c137fe700 1 -- 172.21.15.16:0/225170023 learned_addr learned my addr 172.21.15.16:0/225170023 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:19:32.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.998+0000 7f5c12ffd700 1 --2- 172.21.15.16:0/225170023 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5c140cec40 0x7f5c140cf0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:32.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.998+0000 7f5c137fe700 1 -- 172.21.15.16:0/225170023 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5c140cec40 msgr2=0x7f5c140cf0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:19:32.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.998+0000 7f5c137fe700 1 --2- 172.21.15.16:0/225170023 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5c140cec40 0x7f5c140cf0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:32.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.998+0000 7f5c137fe700 1 -- 172.21.15.16:0/225170023 >> v1:172.21.15.132:6789/0 conn(0x7f5c140ae2e0 legacy=0x7f5c140ae760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:32.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.998+0000 7f5c137fe700 1 -- 172.21.15.16:0/225170023 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c140b2300 con 0x7f5c140d16e0 2023-05-20T01:19:32.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.998+0000 7f5c137fe700 1 --2- 172.21.15.16:0/225170023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c140d16e0 0x7f5c14144350 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f5c040098f0 tx=0x7f5c04009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ac31d11b4470ab8 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:32.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.998+0000 7f5c127fc700 1 -- 172.21.15.16:0/225170023 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c0400b8a0 con 0x7f5c140d16e0 2023-05-20T01:19:32.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.998+0000 7f5c127fc700 1 -- 172.21.15.16:0/225170023 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c0400ba00 con 0x7f5c140d16e0 2023-05-20T01:19:32.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.998+0000 7f5c127fc700 1 -- 172.21.15.16:0/225170023 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c0400bd90 con 0x7f5c140d16e0 2023-05-20T01:19:32.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.998+0000 7f5c19f8d700 1 -- 172.21.15.16:0/225170023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c140d16e0 msgr2=0x7f5c14144350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:32.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.998+0000 7f5c19f8d700 1 --2- 172.21.15.16:0/225170023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c140d16e0 0x7f5c14144350 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f5c040098f0 tx=0x7f5c04009c00 comp rx=0 tx=0).stop 2023-05-20T01:19:32.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.999+0000 7f5c19f8d700 1 -- 172.21.15.16:0/225170023 shutdown_connections 2023-05-20T01:19:32.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.999+0000 7f5c19f8d700 1 --2- 172.21.15.16:0/225170023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c140d16e0 0x7f5c14144350 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:32.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.999+0000 7f5c19f8d700 1 --2- 172.21.15.16:0/225170023 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5c140cec40 0x7f5c140cf0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:32.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.999+0000 7f5c19f8d700 1 -- 172.21.15.16:0/225170023 >> 172.21.15.16:0/225170023 conn(0x7f5c1406ae30 msgr2=0x7f5c1409a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:19:32.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.999+0000 7f5c19f8d700 1 -- 172.21.15.16:0/225170023 shutdown_connections 2023-05-20T01:19:32.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:31.999+0000 7f5c19f8d700 1 -- 172.21.15.16:0/225170023 wait complete. 2023-05-20T01:19:32.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.000+0000 7f5c19f8d700 1 Processor -- start 2023-05-20T01:19:32.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.000+0000 7f5c19f8d700 1 -- start start 2023-05-20T01:19:32.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.001+0000 7f5c19f8d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5c140ae2e0 0x7f5c140cb930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:32.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.001+0000 7f5c19f8d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c140cec40 0x7f5c140cd780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:32.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.001+0000 7f5c19f8d700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5c140d16e0 0x7f5c141f17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:32.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.001+0000 7f5c19f8d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c140c64d0 con 0x7f5c140cec40 2023-05-20T01:19:32.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.001+0000 7f5c19f8d700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c140c5850 con 0x7f5c140ae2e0 2023-05-20T01:19:32.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.001+0000 7f5c19f8d700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c140c5b30 con 0x7f5c140d16e0 2023-05-20T01:19:32.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.001+0000 7f5c12ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c140cec40 0x7f5c140cd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:32.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.001+0000 7f5c12ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c140cec40 0x7f5c140cd780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33180/0 (socket says 172.21.15.16:33180) 2023-05-20T01:19:32.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.001+0000 7f5c12ffd700 1 -- 172.21.15.16:0/2050221658 learned_addr learned my addr 172.21.15.16:0/2050221658 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:19:32.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.001+0000 7f5c13fff700 1 --2- 172.21.15.16:0/2050221658 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5c140d16e0 0x7f5c141f17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:32.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.001+0000 7f5c137fe700 1 --2- 172.21.15.16:0/2050221658 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5c140ae2e0 0x7f5c140cb930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:32.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.001+0000 7f5c12ffd700 1 -- 172.21.15.16:0/2050221658 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5c140d16e0 msgr2=0x7f5c141f17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:32.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.001+0000 7f5c12ffd700 1 --2- 172.21.15.16:0/2050221658 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5c140d16e0 0x7f5c141f17f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:32.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.001+0000 7f5c12ffd700 1 -- 172.21.15.16:0/2050221658 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5c140ae2e0 msgr2=0x7f5c140cb930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:32.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.001+0000 7f5c12ffd700 1 --2- 172.21.15.16:0/2050221658 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5c140ae2e0 0x7f5c140cb930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:32.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.001+0000 7f5c12ffd700 1 -- 172.21.15.16:0/2050221658 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c04009550 con 0x7f5c140cec40 2023-05-20T01:19:32.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.001+0000 7f5c12ffd700 1 --2- 172.21.15.16:0/2050221658 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c140cec40 0x7f5c140cd780 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f5bfc00e810 tx=0x7f5bfc00eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:32.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.001+0000 7f5c0bfff700 1 -- 172.21.15.16:0/2050221658 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bfc00c070 con 0x7f5c140cec40 2023-05-20T01:19:32.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.002+0000 7f5c0bfff700 1 -- 172.21.15.16:0/2050221658 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bfc013610 con 0x7f5c140cec40 2023-05-20T01:19:32.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.002+0000 7f5c19f8d700 1 -- 172.21.15.16:0/2050221658 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c140addc0 con 0x7f5c140cec40 2023-05-20T01:19:32.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.002+0000 7f5c0bfff700 1 -- 172.21.15.16:0/2050221658 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bfc0139c0 con 0x7f5c140cec40 2023-05-20T01:19:32.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.002+0000 7f5c19f8d700 1 -- 172.21.15.16:0/2050221658 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c140b25e0 con 0x7f5c140cec40 2023-05-20T01:19:32.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.003+0000 7f5c19f8d700 1 -- 172.21.15.16:0/2050221658 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c140b28c0 con 0x7f5c140cec40 2023-05-20T01:19:32.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.003+0000 7f5c0bfff700 1 -- 172.21.15.16:0/2050221658 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5bfc013b20 con 0x7f5c140cec40 2023-05-20T01:19:32.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.004+0000 7f5c0bfff700 1 --2- 172.21.15.16:0/2050221658 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f5bf406ace0 0x7f5bf406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:32.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.004+0000 7f5c0bfff700 1 -- 172.21.15.16:0/2050221658 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 7102+0+0 (secure 0 0 0) 0x7f5bfc0892e0 con 0x7f5c140cec40 2023-05-20T01:19:32.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.004+0000 7f5c137fe700 1 --2- 172.21.15.16:0/2050221658 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f5bf406ace0 0x7f5bf406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:32.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.004+0000 7f5c137fe700 1 --2- 172.21.15.16:0/2050221658 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f5bf406ace0 0x7f5bf406d190 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f5c04000c00 tx=0x7f5c04008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:32.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.008+0000 7f5c0bfff700 1 -- 172.21.15.16:0/2050221658 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f5bfc057aa0 con 0x7f5c140cec40 2023-05-20T01:19:32.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.205+0000 7f5c19f8d700 1 -- 172.21.15.16:0/2050221658 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5c140c6610 con 0x7f5c140cec40 2023-05-20T01:19:32.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.206+0000 7f5c0bfff700 1 -- 172.21.15.16:0/2050221658 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v51) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5c140c6610 con 0x7f5c140cec40 2023-05-20T01:19:32.210 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-05-20T01:19:32.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.208+0000 7f5c19f8d700 1 -- 172.21.15.16:0/2050221658 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f5bf406ace0 msgr2=0x7f5bf406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:32.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.208+0000 7f5c19f8d700 1 --2- 172.21.15.16:0/2050221658 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f5bf406ace0 0x7f5bf406d190 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f5c04000c00 tx=0x7f5c04008030 comp rx=0 tx=0).stop 2023-05-20T01:19:32.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.208+0000 7f5c19f8d700 1 -- 172.21.15.16:0/2050221658 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c140cec40 msgr2=0x7f5c140cd780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:32.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.208+0000 7f5c19f8d700 1 --2- 172.21.15.16:0/2050221658 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c140cec40 0x7f5c140cd780 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f5bfc00e810 tx=0x7f5bfc00eb20 comp rx=0 tx=0).stop 2023-05-20T01:19:32.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.208+0000 7f5c19f8d700 1 -- 172.21.15.16:0/2050221658 shutdown_connections 2023-05-20T01:19:32.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.208+0000 7f5c19f8d700 1 --2- 172.21.15.16:0/2050221658 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f5bf406ace0 0x7f5bf406d190 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:32.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.208+0000 7f5c19f8d700 1 --2- 172.21.15.16:0/2050221658 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5c140d16e0 0x7f5c141f17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:32.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.208+0000 7f5c19f8d700 1 --2- 172.21.15.16:0/2050221658 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5c140ae2e0 0x7f5c140cb930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:32.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.208+0000 7f5c19f8d700 1 --2- 172.21.15.16:0/2050221658 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c140cec40 0x7f5c140cd780 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:32.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.208+0000 7f5c19f8d700 1 -- 172.21.15.16:0/2050221658 >> 172.21.15.16:0/2050221658 conn(0x7f5c1406ae30 msgr2=0x7f5c140b5eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:19:32.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.208+0000 7f5c19f8d700 1 -- 172.21.15.16:0/2050221658 shutdown_connections 2023-05-20T01:19:32.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.208+0000 7f5c19f8d700 1 -- 172.21.15.16:0/2050221658 wait complete. 2023-05-20T01:19:32.224 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-05-20T01:19:32.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.348+0000 7fb5ea29e700 1 Processor -- start 2023-05-20T01:19:32.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.348+0000 7fb5ea29e700 1 -- start start 2023-05-20T01:19:32.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.349+0000 7fb5ea29e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb5e4072300 0x7fb5e40cf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:32.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.349+0000 7fb5ea29e700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb5e415d290 0x7fb5e40cf800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:32.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.349+0000 7fb5ea29e700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5e40cd630 con 0x7fb5e415cec0 2023-05-20T01:19:32.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.349+0000 7fb5ea29e700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5e40cca60 con 0x7fb5e415d290 2023-05-20T01:19:32.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.349+0000 7fb5ea29e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e40ccd00 con 0x7fb5e4072300 2023-05-20T01:19:32.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.349+0000 7fb5e883b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb5e415d290 0x7fb5e40cf800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:32.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.349+0000 7fb5e3fff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fb5e415cec0 0x7fb5e4071bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:44714/0 (socket says 172.21.15.16:44714) 2023-05-20T01:19:32.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.349+0000 7fb5e3fff700 1 -- 172.21.15.16:0/1153760147 learned_addr learned my addr 172.21.15.16:0/1153760147 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:19:32.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.349+0000 7fb5e883b700 1 -- 172.21.15.16:0/1153760147 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb5e4072300 msgr2=0x7fb5e40cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:19:32.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.349+0000 7fb5e37fe700 1 --2- 172.21.15.16:0/1153760147 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb5e4072300 0x7fb5e40cf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:32.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.349+0000 7fb5e2ffd700 1 -- 172.21.15.16:0/1153760147 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 520054679 0 0) 0x7fb5e40cd630 con 0x7fb5e415cec0 2023-05-20T01:19:32.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.349+0000 7fb5e883b700 1 --2- 172.21.15.16:0/1153760147 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb5e4072300 0x7fb5e40cf2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:32.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.349+0000 7fb5e883b700 1 -- 172.21.15.16:0/1153760147 >> v1:172.21.15.16:6789/0 conn(0x7fb5e415cec0 legacy=0x7fb5e4071bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:32.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.349+0000 7fb5e883b700 1 -- 172.21.15.16:0/1153760147 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5e40ccfe0 con 0x7fb5e415d290 2023-05-20T01:19:32.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.349+0000 7fb5e37fe700 1 --2- 172.21.15.16:0/1153760147 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb5e4072300 0x7fb5e40cf2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:19:32.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.349+0000 7fb5e883b700 1 --2- 172.21.15.16:0/1153760147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5e415d290 0x7fb5e40cf800 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc009d90 tx=0x7fb5dc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f764f2162b1c894 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:32.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.350+0000 7fb5e2ffd700 1 -- 172.21.15.16:0/1153760147 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5dc00e070 con 0x7fb5e415d290 2023-05-20T01:19:32.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.350+0000 7fb5e2ffd700 1 -- 172.21.15.16:0/1153760147 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5dc00bbd0 con 0x7fb5e415d290 2023-05-20T01:19:32.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.350+0000 7fb5e2ffd700 1 -- 172.21.15.16:0/1153760147 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5dc012430 con 0x7fb5e415d290 2023-05-20T01:19:32.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.350+0000 7fb5ea29e700 1 -- 172.21.15.16:0/1153760147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5e415d290 msgr2=0x7fb5e40cf800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:32.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.350+0000 7fb5ea29e700 1 --2- 172.21.15.16:0/1153760147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5e415d290 0x7fb5e40cf800 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc009d90 tx=0x7fb5dc00b330 comp rx=0 tx=0).stop 2023-05-20T01:19:32.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.350+0000 7fb5ea29e700 1 -- 172.21.15.16:0/1153760147 shutdown_connections 2023-05-20T01:19:32.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.350+0000 7fb5ea29e700 1 --2- 172.21.15.16:0/1153760147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5e415d290 0x7fb5e40cf800 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:32.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.350+0000 7fb5ea29e700 1 --2- 172.21.15.16:0/1153760147 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb5e4072300 0x7fb5e40cf2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:32.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.350+0000 7fb5ea29e700 1 -- 172.21.15.16:0/1153760147 >> 172.21.15.16:0/1153760147 conn(0x7fb5e40cffd0 msgr2=0x7fb5e40d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:19:32.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.350+0000 7fb5ea29e700 1 -- 172.21.15.16:0/1153760147 shutdown_connections 2023-05-20T01:19:32.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.350+0000 7fb5ea29e700 1 -- 172.21.15.16:0/1153760147 wait complete. 2023-05-20T01:19:32.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.351+0000 7fb5ea29e700 1 Processor -- start 2023-05-20T01:19:32.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.352+0000 7fb5ea29e700 1 -- start start 2023-05-20T01:19:32.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.352+0000 7fb5ea29e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb5e4072300 0x7fb5e41f22e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:32.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.352+0000 7fb5ea29e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5e415cec0 0x7fb5e415f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:32.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.352+0000 7fb5ea29e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb5e415d290 0x7fb5e415ff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:32.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.352+0000 7fb5ea29e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e40b2910 con 0x7fb5e415cec0 2023-05-20T01:19:32.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.352+0000 7fb5ea29e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e41f2820 con 0x7fb5e4072300 2023-05-20T01:19:32.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.352+0000 7fb5ea29e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e40b1c90 con 0x7fb5e415d290 2023-05-20T01:19:32.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.352+0000 7fb5e37fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5e415cec0 0x7fb5e415f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:32.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.353+0000 7fb5e37fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5e415cec0 0x7fb5e415f9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33192/0 (socket says 172.21.15.16:33192) 2023-05-20T01:19:32.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.353+0000 7fb5e37fe700 1 -- 172.21.15.16:0/163406258 learned_addr learned my addr 172.21.15.16:0/163406258 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:19:32.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.353+0000 7fb5e883b700 1 --2- 172.21.15.16:0/163406258 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb5e415d290 0x7fb5e415ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:32.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.353+0000 7fb5e3fff700 1 --2- 172.21.15.16:0/163406258 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb5e4072300 0x7fb5e41f22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:32.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.353+0000 7fb5e37fe700 1 -- 172.21.15.16:0/163406258 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb5e415d290 msgr2=0x7fb5e415ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:32.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.353+0000 7fb5e37fe700 1 --2- 172.21.15.16:0/163406258 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb5e415d290 0x7fb5e415ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:32.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.353+0000 7fb5e37fe700 1 -- 172.21.15.16:0/163406258 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb5e4072300 msgr2=0x7fb5e41f22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:32.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.353+0000 7fb5e37fe700 1 --2- 172.21.15.16:0/163406258 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb5e4072300 0x7fb5e41f22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:32.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.353+0000 7fb5e37fe700 1 -- 172.21.15.16:0/163406258 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5dc009a70 con 0x7fb5e415cec0 2023-05-20T01:19:32.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.353+0000 7fb5e37fe700 1 --2- 172.21.15.16:0/163406258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5e415cec0 0x7fb5e415f9f0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fb5d400c820 tx=0x7fb5d400cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:32.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.353+0000 7fb5e0ff9700 1 -- 172.21.15.16:0/163406258 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5d4011800 con 0x7fb5e415cec0 2023-05-20T01:19:32.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.353+0000 7fb5e0ff9700 1 -- 172.21.15.16:0/163406258 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5d4011960 con 0x7fb5e415cec0 2023-05-20T01:19:32.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.353+0000 7fb5e0ff9700 1 -- 172.21.15.16:0/163406258 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5d4011cf0 con 0x7fb5e415cec0 2023-05-20T01:19:32.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.353+0000 7fb5ea29e700 1 -- 172.21.15.16:0/163406258 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5e40cfd40 con 0x7fb5e415cec0 2023-05-20T01:19:32.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.353+0000 7fb5ea29e700 1 -- 172.21.15.16:0/163406258 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5e40cd2c0 con 0x7fb5e415cec0 2023-05-20T01:19:32.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.354+0000 7fb5ea29e700 1 -- 172.21.15.16:0/163406258 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5e4160870 con 0x7fb5e415cec0 2023-05-20T01:19:32.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.355+0000 7fb5e0ff9700 1 -- 172.21.15.16:0/163406258 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb5d4018360 con 0x7fb5e415cec0 2023-05-20T01:19:32.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.355+0000 7fb5e0ff9700 1 --2- 172.21.15.16:0/163406258 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb5d006ace0 0x7fb5d006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:32.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.355+0000 7fb5e0ff9700 1 -- 172.21.15.16:0/163406258 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 7102+0+0 (secure 0 0 0) 0x7fb5d4089760 con 0x7fb5e415cec0 2023-05-20T01:19:32.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.355+0000 7fb5e3fff700 1 --2- 172.21.15.16:0/163406258 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb5d006ace0 0x7fb5d006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:32.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.356+0000 7fb5e3fff700 1 --2- 172.21.15.16:0/163406258 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb5d006ace0 0x7fb5d006d190 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fb5d80010e0 tx=0x7fb5d8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:32.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.360+0000 7fb5e0ff9700 1 -- 172.21.15.16:0/163406258 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fb5d4054400 con 0x7fb5e415cec0 2023-05-20T01:19:32.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.558+0000 7fb5ea29e700 1 -- 172.21.15.16:0/163406258 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb5e40747b0 con 0x7fb5e415cec0 2023-05-20T01:19:32.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.558+0000 7fb5e0ff9700 1 -- 172.21.15.16:0/163406258 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v51) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb5d4057a20 con 0x7fb5e415cec0 2023-05-20T01:19:32.563 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-05-20T01:19:32.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.560+0000 7fb5ea29e700 1 -- 172.21.15.16:0/163406258 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb5d006ace0 msgr2=0x7fb5d006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:32.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.560+0000 7fb5ea29e700 1 --2- 172.21.15.16:0/163406258 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb5d006ace0 0x7fb5d006d190 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fb5d80010e0 tx=0x7fb5d8009040 comp rx=0 tx=0).stop 2023-05-20T01:19:32.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.560+0000 7fb5ea29e700 1 -- 172.21.15.16:0/163406258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5e415cec0 msgr2=0x7fb5e415f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:32.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.560+0000 7fb5ea29e700 1 --2- 172.21.15.16:0/163406258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5e415cec0 0x7fb5e415f9f0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fb5d400c820 tx=0x7fb5d400cb30 comp rx=0 tx=0).stop 2023-05-20T01:19:32.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.560+0000 7fb5ea29e700 1 -- 172.21.15.16:0/163406258 shutdown_connections 2023-05-20T01:19:32.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.560+0000 7fb5ea29e700 1 --2- 172.21.15.16:0/163406258 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb5d006ace0 0x7fb5d006d190 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:32.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.560+0000 7fb5ea29e700 1 --2- 172.21.15.16:0/163406258 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb5e415d290 0x7fb5e415ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:32.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.560+0000 7fb5ea29e700 1 --2- 172.21.15.16:0/163406258 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb5e4072300 0x7fb5e41f22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:32.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.560+0000 7fb5ea29e700 1 --2- 172.21.15.16:0/163406258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5e415cec0 0x7fb5e415f9f0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:32.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.560+0000 7fb5ea29e700 1 -- 172.21.15.16:0/163406258 >> 172.21.15.16:0/163406258 conn(0x7fb5e40cffd0 msgr2=0x7fb5e40cf560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:19:32.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.560+0000 7fb5ea29e700 1 -- 172.21.15.16:0/163406258 shutdown_connections 2023-05-20T01:19:32.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.560+0000 7fb5ea29e700 1 -- 172.21.15.16:0/163406258 wait complete. 2023-05-20T01:19:32.576 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-05-20T01:19:32.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.701+0000 7f616a209700 1 Processor -- start 2023-05-20T01:19:32.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.701+0000 7f616a209700 1 -- start start 2023-05-20T01:19:32.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.702+0000 7f616a209700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6164159b50 0x7f616406e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:32.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.702+0000 7f616a209700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f61640cd1c0 0x7f61640710b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:32.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.702+0000 7f616a209700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61640ccd30 con 0x7f61640cd630 2023-05-20T01:19:32.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.702+0000 7f616a209700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61640ce990 con 0x7f6164159b50 2023-05-20T01:19:32.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.702+0000 7f616a209700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61640cc6e0 con 0x7f61640cd1c0 2023-05-20T01:19:32.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.702+0000 7f61637fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6164159b50 0x7f616406e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:32.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.702+0000 7f61637fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6164159b50 0x7f616406e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33198/0 (socket says 172.21.15.16:33198) 2023-05-20T01:19:32.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.702+0000 7f61637fe700 1 -- 172.21.15.16:0/2079170122 learned_addr learned my addr 172.21.15.16:0/2079170122 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:19:32.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.702+0000 7f6162ffd700 1 --2- 172.21.15.16:0/2079170122 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f61640cd1c0 0x7f61640710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:32.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.703+0000 7f61637fe700 1 -- 172.21.15.16:0/2079170122 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f61640cd1c0 msgr2=0x7f61640710b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:19:32.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.703+0000 7f61637fe700 1 --2- 172.21.15.16:0/2079170122 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f61640cd1c0 0x7f61640710b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:32.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.703+0000 7f61637fe700 1 -- 172.21.15.16:0/2079170122 >> v1:172.21.15.132:6789/0 conn(0x7f61640cd630 legacy=0x7f61640ca870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:19:32.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.703+0000 7f61637fe700 1 -- 172.21.15.16:0/2079170122 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61640cc9c0 con 0x7f6164159b50 2023-05-20T01:19:32.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.703+0000 7f6162ffd700 1 --2- 172.21.15.16:0/2079170122 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f61640cd1c0 0x7f61640710b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:19:32.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.703+0000 7f61637fe700 1 --2- 172.21.15.16:0/2079170122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6164159b50 0x7f616406e0a0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f6158006fd0 tx=0x7f615800be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afd3f24d48447473 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:32.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.703+0000 7f61627fc700 1 -- 172.21.15.16:0/2079170122 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6158005e30 con 0x7f6164159b50 2023-05-20T01:19:32.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.703+0000 7f61627fc700 1 -- 172.21.15.16:0/2079170122 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f615800d8e0 con 0x7f6164159b50 2023-05-20T01:19:32.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.703+0000 7f61627fc700 1 -- 172.21.15.16:0/2079170122 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f615800dc70 con 0x7f6164159b50 2023-05-20T01:19:32.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.703+0000 7f616a209700 1 -- 172.21.15.16:0/2079170122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6164159b50 msgr2=0x7f616406e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:32.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.703+0000 7f616a209700 1 --2- 172.21.15.16:0/2079170122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6164159b50 0x7f616406e0a0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f6158006fd0 tx=0x7f615800be70 comp rx=0 tx=0).stop 2023-05-20T01:19:32.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.703+0000 7f616a209700 1 -- 172.21.15.16:0/2079170122 shutdown_connections 2023-05-20T01:19:32.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.703+0000 7f616a209700 1 --2- 172.21.15.16:0/2079170122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6164159b50 0x7f616406e0a0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:32.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.704+0000 7f616a209700 1 --2- 172.21.15.16:0/2079170122 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f61640cd1c0 0x7f61640710b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:32.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.704+0000 7f616a209700 1 -- 172.21.15.16:0/2079170122 >> 172.21.15.16:0/2079170122 conn(0x7f61640cffd0 msgr2=0x7f61640d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:19:32.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.704+0000 7f616a209700 1 -- 172.21.15.16:0/2079170122 shutdown_connections 2023-05-20T01:19:32.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.704+0000 7f616a209700 1 -- 172.21.15.16:0/2079170122 wait complete. 2023-05-20T01:19:32.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.705+0000 7f616a209700 1 Processor -- start 2023-05-20T01:19:32.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.705+0000 7f616a209700 1 -- start start 2023-05-20T01:19:32.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.706+0000 7f616a209700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f61640cd1c0 0x7f6164150c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:32.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.706+0000 7f616a209700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61640cd630 0x7f6164152910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:32.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.706+0000 7f616a209700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6164159b50 0x7f6164150500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:32.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.706+0000 7f616a209700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61640c4040 con 0x7f61640cd630 2023-05-20T01:19:32.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.706+0000 7f616a209700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6164152e50 con 0x7f6164159b50 2023-05-20T01:19:32.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.706+0000 7f616a209700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6164150a40 con 0x7f61640cd1c0 2023-05-20T01:19:32.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.706+0000 7f6162ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61640cd630 0x7f6164152910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:32.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.706+0000 7f6162ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61640cd630 0x7f6164152910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33206/0 (socket says 172.21.15.16:33206) 2023-05-20T01:19:32.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.706+0000 7f6162ffd700 1 -- 172.21.15.16:0/855696235 learned_addr learned my addr 172.21.15.16:0/855696235 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:19:32.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.706+0000 7f61637fe700 1 --2- 172.21.15.16:0/855696235 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f61640cd1c0 0x7f6164150c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:32.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.706+0000 7f6163fff700 1 --2- 172.21.15.16:0/855696235 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6164159b50 0x7f6164150500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:32.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.706+0000 7f6162ffd700 1 -- 172.21.15.16:0/855696235 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f61640cd1c0 msgr2=0x7f6164150c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:32.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.706+0000 7f6162ffd700 1 --2- 172.21.15.16:0/855696235 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f61640cd1c0 0x7f6164150c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:32.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.706+0000 7f6162ffd700 1 -- 172.21.15.16:0/855696235 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6164159b50 msgr2=0x7f6164150500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:32.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.706+0000 7f6162ffd700 1 --2- 172.21.15.16:0/855696235 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6164159b50 0x7f6164150500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:32.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.706+0000 7f6162ffd700 1 -- 172.21.15.16:0/855696235 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f615800b7a0 con 0x7f61640cd630 2023-05-20T01:19:32.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.706+0000 7f6163fff700 1 --2- 172.21.15.16:0/855696235 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6164159b50 0x7f6164150500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:19:32.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.706+0000 7f6162ffd700 1 --2- 172.21.15.16:0/855696235 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61640cd630 0x7f6164152910 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f615400c750 tx=0x7f615400ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:32.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.707+0000 7f614bfff700 1 -- 172.21.15.16:0/855696235 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6154011800 con 0x7f61640cd630 2023-05-20T01:19:32.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.707+0000 7f614bfff700 1 -- 172.21.15.16:0/855696235 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6154011960 con 0x7f61640cd630 2023-05-20T01:19:32.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.707+0000 7f616a209700 1 -- 172.21.15.16:0/855696235 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6164071e80 con 0x7f61640cd630 2023-05-20T01:19:32.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.707+0000 7f614bfff700 1 -- 172.21.15.16:0/855696235 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6154011cd0 con 0x7f61640cd630 2023-05-20T01:19:32.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.707+0000 7f616a209700 1 -- 172.21.15.16:0/855696235 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f616406e5e0 con 0x7f61640cd630 2023-05-20T01:19:32.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.708+0000 7f616a209700 1 -- 172.21.15.16:0/855696235 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61640cec70 con 0x7f61640cd630 2023-05-20T01:19:32.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.708+0000 7f614bfff700 1 -- 172.21.15.16:0/855696235 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6154021350 con 0x7f61640cd630 2023-05-20T01:19:32.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.708+0000 7f614bfff700 1 --2- 172.21.15.16:0/855696235 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f615006ace0 0x7f615006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:32.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.709+0000 7f614bfff700 1 -- 172.21.15.16:0/855696235 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 7102+0+0 (secure 0 0 0) 0x7f61540897b0 con 0x7f61640cd630 2023-05-20T01:19:32.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.709+0000 7f61637fe700 1 --2- 172.21.15.16:0/855696235 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f615006ace0 0x7f615006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:32.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.709+0000 7f61637fe700 1 --2- 172.21.15.16:0/855696235 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f615006ace0 0x7f615006d190 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f6158000c00 tx=0x7f6158012000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:32.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.713+0000 7f614bfff700 1 -- 172.21.15.16:0/855696235 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f6154057ee0 con 0x7f61640cd630 2023-05-20T01:19:32.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.910+0000 7f616a209700 1 -- 172.21.15.16:0/855696235 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f616406ee10 con 0x7f61640cd630 2023-05-20T01:19:32.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.910+0000 7f614bfff700 1 -- 172.21.15.16:0/855696235 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v51) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6154057a70 con 0x7f61640cd630 2023-05-20T01:19:32.914 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2023-05-20T01:19:32.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.912+0000 7f616a209700 1 -- 172.21.15.16:0/855696235 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f615006ace0 msgr2=0x7f615006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:32.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.912+0000 7f616a209700 1 --2- 172.21.15.16:0/855696235 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f615006ace0 0x7f615006d190 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f6158000c00 tx=0x7f6158012000 comp rx=0 tx=0).stop 2023-05-20T01:19:32.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.912+0000 7f616a209700 1 -- 172.21.15.16:0/855696235 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61640cd630 msgr2=0x7f6164152910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:32.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.912+0000 7f616a209700 1 --2- 172.21.15.16:0/855696235 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61640cd630 0x7f6164152910 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f615400c750 tx=0x7f615400ca60 comp rx=0 tx=0).stop 2023-05-20T01:19:32.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.912+0000 7f616a209700 1 -- 172.21.15.16:0/855696235 shutdown_connections 2023-05-20T01:19:32.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.912+0000 7f616a209700 1 --2- 172.21.15.16:0/855696235 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6164159b50 0x7f6164150500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:32.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.912+0000 7f616a209700 1 --2- 172.21.15.16:0/855696235 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f615006ace0 0x7f615006d190 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:32.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.912+0000 7f616a209700 1 --2- 172.21.15.16:0/855696235 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f61640cd1c0 0x7f6164150c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:32.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.912+0000 7f616a209700 1 --2- 172.21.15.16:0/855696235 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61640cd630 0x7f6164152910 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:32.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.912+0000 7f616a209700 1 -- 172.21.15.16:0/855696235 >> 172.21.15.16:0/855696235 conn(0x7f61640cffd0 msgr2=0x7f6164153070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:19:32.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.912+0000 7f616a209700 1 -- 172.21.15.16:0/855696235 shutdown_connections 2023-05-20T01:19:32.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:32.912+0000 7f616a209700 1 -- 172.21.15.16:0/855696235 wait complete. 2023-05-20T01:19:32.928 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_extra pg_num 2023-05-20T01:19:33.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.052+0000 7f2a01d1d700 1 Processor -- start 2023-05-20T01:19:33.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.052+0000 7f2a01d1d700 1 -- start start 2023-05-20T01:19:33.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.052+0000 7f2a01d1d700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f29fc072300 0x7f29fc0cf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:33.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.053+0000 7f2a01d1d700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29fc0cd630 con 0x7f29fc15cec0 2023-05-20T01:19:33.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.053+0000 7f2a01d1d700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29fc0cca60 con 0x7f29fc15d290 2023-05-20T01:19:33.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.053+0000 7f2a01d1d700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29fc0ccd00 con 0x7f29fc072300 2023-05-20T01:19:33.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.053+0000 7f29fb7fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f29fc15cec0 0x7f29fc071bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:44740/0 (socket says 172.21.15.16:44740) 2023-05-20T01:19:33.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.053+0000 7f29fb7fe700 1 -- 172.21.15.16:0/2377107077 learned_addr learned my addr 172.21.15.16:0/2377107077 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:19:33.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.053+0000 7f29faffd700 1 --2- 172.21.15.16:0/2377107077 >> v2:172.21.15.132:3300/0 conn(0x7f29fc072300 0x7f29fc0cf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:33.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.053+0000 7f29fa7fc700 1 -- 172.21.15.16:0/2377107077 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2478719303 0 0) 0x7f29fc0cd630 con 0x7f29fc15cec0 2023-05-20T01:19:33.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.053+0000 7f29fa7fc700 1 -- 172.21.15.16:0/2377107077 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f29e80035b0 con 0x7f29fc15cec0 2023-05-20T01:19:33.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.053+0000 7f29fa7fc700 1 -- 172.21.15.16:0/2377107077 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1498147536 0 0) 0x7f29e80035b0 con 0x7f29fc15cec0 2023-05-20T01:19:33.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.053+0000 7f29fa7fc700 1 -- 172.21.15.16:0/2377107077 >> v2:172.21.15.132:3300/0 conn(0x7f29fc072300 msgr2=0x7f29fc0cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:19:33.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.053+0000 7f29fa7fc700 1 --2- 172.21.15.16:0/2377107077 >> v2:172.21.15.132:3300/0 conn(0x7f29fc072300 0x7f29fc0cf2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:33.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.053+0000 7f29fa7fc700 1 -- 172.21.15.16:0/2377107077 >> v1:172.21.15.132:6789/0 conn(0x7f29fc15d290 legacy=0x7f29fc0cf800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:33.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.053+0000 7f29fa7fc700 1 -- 172.21.15.16:0/2377107077 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29fc0ccfe0 con 0x7f29fc15cec0 2023-05-20T01:19:33.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.053+0000 7f29fa7fc700 1 -- 172.21.15.16:0/2377107077 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7f29ec001b70 con 0x7f29fc15cec0 2023-05-20T01:19:33.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.053+0000 7f29fa7fc700 1 -- 172.21.15.16:0/2377107077 >> v1:172.21.15.16:6789/0 conn(0x7f29fc15cec0 legacy=0x7f29fc071bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:33.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.053+0000 7f29fa7fc700 1 --2- 172.21.15.16:0/2377107077 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f29e8003a00 0x7f29e8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:33.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.053+0000 7f29fa7fc700 1 --2- 172.21.15.16:0/2377107077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29e80043f0 0x7f29e80048b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:33.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.053+0000 7f29fa7fc700 1 --2- 172.21.15.16:0/2377107077 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f29e8005150 0x7f29e800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:33.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.053+0000 7f29fa7fc700 1 -- 172.21.15.16:0/2377107077 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29fc0ccd00 con 0x7f29e80043f0 2023-05-20T01:19:33.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.053+0000 7f29fa7fc700 1 -- 172.21.15.16:0/2377107077 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29e800c080 con 0x7f29e8005150 2023-05-20T01:19:33.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.053+0000 7f29fa7fc700 1 -- 172.21.15.16:0/2377107077 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29e800c240 con 0x7f29e8003a00 2023-05-20T01:19:33.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.053+0000 7f29faffd700 1 --2- 172.21.15.16:0/2377107077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29e80043f0 0x7f29e80048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:33.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.054+0000 7f29fb7fe700 1 --2- 172.21.15.16:0/2377107077 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f29e8003a00 0x7f29e8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:33.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.054+0000 7f29fbfff700 1 --2- 172.21.15.16:0/2377107077 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f29e8005150 0x7f29e800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:33.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.054+0000 7f29faffd700 1 -- 172.21.15.16:0/2377107077 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f29e8003a00 msgr2=0x7f29e8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:19:33.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.054+0000 7f29faffd700 1 --2- 172.21.15.16:0/2377107077 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f29e8003a00 0x7f29e8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:33.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.054+0000 7f29faffd700 1 -- 172.21.15.16:0/2377107077 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f29e8005150 msgr2=0x7f29e800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:19:33.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.054+0000 7f29faffd700 1 --2- 172.21.15.16:0/2377107077 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f29e8005150 0x7f29e800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:33.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.054+0000 7f29faffd700 1 -- 172.21.15.16:0/2377107077 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29e80035b0 con 0x7f29e80043f0 2023-05-20T01:19:33.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.054+0000 7f29fbfff700 1 --2- 172.21.15.16:0/2377107077 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f29e8005150 0x7f29e800ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:19:33.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.054+0000 7f29fbfff700 1 -- 172.21.15.16:0/2377107077 reap_dead start 2023-05-20T01:19:33.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.054+0000 7f29faffd700 1 --2- 172.21.15.16:0/2377107077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29e80043f0 0x7f29e80048b0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f29e4009b30 tx=0x7f29e8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a77b30a073b5ff server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:33.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.054+0000 7f29fa7fc700 1 -- 172.21.15.16:0/2377107077 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29e400e6c0 con 0x7f29e80043f0 2023-05-20T01:19:33.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.054+0000 7f29fa7fc700 1 -- 172.21.15.16:0/2377107077 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29e400e820 con 0x7f29e80043f0 2023-05-20T01:19:33.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.054+0000 7f29fa7fc700 1 -- 172.21.15.16:0/2377107077 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29e400eb90 con 0x7f29e80043f0 2023-05-20T01:19:33.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.055+0000 7f2a01d1d700 1 -- 172.21.15.16:0/2377107077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29e80043f0 msgr2=0x7f29e80048b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:33.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.055+0000 7f2a01d1d700 1 --2- 172.21.15.16:0/2377107077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29e80043f0 0x7f29e80048b0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f29e4009b30 tx=0x7f29e8005880 comp rx=0 tx=0).stop 2023-05-20T01:19:33.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.055+0000 7f2a01d1d700 1 -- 172.21.15.16:0/2377107077 shutdown_connections 2023-05-20T01:19:33.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.055+0000 7f2a01d1d700 1 --2- 172.21.15.16:0/2377107077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29e80043f0 0x7f29e80048b0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:33.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.055+0000 7f2a01d1d700 1 -- 172.21.15.16:0/2377107077 >> 172.21.15.16:0/2377107077 conn(0x7f29fc0cffd0 msgr2=0x7f29fc0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:19:33.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.055+0000 7f2a01d1d700 1 -- 172.21.15.16:0/2377107077 shutdown_connections 2023-05-20T01:19:33.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.055+0000 7f2a01d1d700 1 -- 172.21.15.16:0/2377107077 wait complete. 2023-05-20T01:19:33.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.056+0000 7f2a01d1d700 1 Processor -- start 2023-05-20T01:19:33.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.056+0000 7f2a01d1d700 1 -- start start 2023-05-20T01:19:33.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.057+0000 7f2a01d1d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f29e80043f0 0x7f29fc1f3010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:33.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.057+0000 7f2a01d1d700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f29fc0b1840 0x7f29fc0b2e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:33.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.057+0000 7f2a01d1d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29fc159a20 0x7f29fc159e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:33.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.057+0000 7f2a01d1d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29fc15a3d0 con 0x7f29fc159a20 2023-05-20T01:19:33.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.057+0000 7f2a01d1d700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29fc1f3570 con 0x7f29e80043f0 2023-05-20T01:19:33.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.057+0000 7f2a01d1d700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29fc0b1cb0 con 0x7f29fc0b1840 2023-05-20T01:19:33.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.057+0000 7f29fbfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29fc159a20 0x7f29fc159e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:33.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.057+0000 7f29fbfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29fc159a20 0x7f29fc159e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33228/0 (socket says 172.21.15.16:33228) 2023-05-20T01:19:33.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.057+0000 7f29fbfff700 1 -- 172.21.15.16:0/536018791 learned_addr learned my addr 172.21.15.16:0/536018791 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:19:33.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.057+0000 7f29faffd700 1 --2- 172.21.15.16:0/536018791 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f29fc0b1840 0x7f29fc0b2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:33.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.057+0000 7f29fb7fe700 1 --2- 172.21.15.16:0/536018791 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f29e80043f0 0x7f29fc1f3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:33.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.057+0000 7f29fbfff700 1 -- 172.21.15.16:0/536018791 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f29fc0b1840 msgr2=0x7f29fc0b2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:33.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.057+0000 7f29fbfff700 1 --2- 172.21.15.16:0/536018791 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f29fc0b1840 0x7f29fc0b2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:33.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.057+0000 7f29fbfff700 1 -- 172.21.15.16:0/536018791 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f29e80043f0 msgr2=0x7f29fc1f3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:33.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.057+0000 7f29fbfff700 1 --2- 172.21.15.16:0/536018791 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f29e80043f0 0x7f29fc1f3010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:33.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.057+0000 7f29fbfff700 1 -- 172.21.15.16:0/536018791 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29fc071bf0 con 0x7f29fc159a20 2023-05-20T01:19:33.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.057+0000 7f29faffd700 1 --2- 172.21.15.16:0/536018791 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f29fc0b1840 0x7f29fc0b2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:19:33.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.057+0000 7f29fbfff700 1 --2- 172.21.15.16:0/536018791 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29fc159a20 0x7f29fc159e90 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f29fc0cec20 tx=0x7f29f400e860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:33.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.058+0000 7f29f3fff700 1 -- 172.21.15.16:0/536018791 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29f400ee80 con 0x7f29fc159a20 2023-05-20T01:19:33.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.058+0000 7f29f3fff700 1 -- 172.21.15.16:0/536018791 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29f40052e0 con 0x7f29fc159a20 2023-05-20T01:19:33.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.058+0000 7f2a01d1d700 1 -- 172.21.15.16:0/536018791 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29fc0cd2c0 con 0x7f29fc159a20 2023-05-20T01:19:33.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.058+0000 7f29f3fff700 1 -- 172.21.15.16:0/536018791 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29f4006860 con 0x7f29fc159a20 2023-05-20T01:19:33.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.058+0000 7f2a01d1d700 1 -- 172.21.15.16:0/536018791 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29fc071d80 con 0x7f29fc159a20 2023-05-20T01:19:33.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.059+0000 7f2a01d1d700 1 -- 172.21.15.16:0/536018791 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29fc0b5ce0 con 0x7f29fc159a20 2023-05-20T01:19:33.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.059+0000 7f29f3fff700 1 -- 172.21.15.16:0/536018791 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f29f40069c0 con 0x7f29fc159a20 2023-05-20T01:19:33.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.059+0000 7f29f3fff700 1 --2- 172.21.15.16:0/536018791 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f29dc06ace0 0x7f29dc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:33.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.060+0000 7f29f3fff700 1 -- 172.21.15.16:0/536018791 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 7102+0+0 (secure 0 0 0) 0x7f29f4088500 con 0x7f29fc159a20 2023-05-20T01:19:33.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.060+0000 7f29fb7fe700 1 --2- 172.21.15.16:0/536018791 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f29dc06ace0 0x7f29dc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:33.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.060+0000 7f29fb7fe700 1 --2- 172.21.15.16:0/536018791 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f29dc06ace0 0x7f29dc06d190 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f29ec000f80 tx=0x7f29ec009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:33.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.064+0000 7f29f3fff700 1 -- 172.21.15.16:0/536018791 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f29f4056d70 con 0x7f29fc159a20 2023-05-20T01:19:33.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.258+0000 7f2a01d1d700 1 -- 172.21.15.16:0/536018791 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"} v 0) v1 -- 0x7f29fc0b3380 con 0x7f29fc159a20 2023-05-20T01:19:33.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.258+0000 7f29f3fff700 1 -- 172.21.15.16:0/536018791 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"}]=0 v51) v1 ==== 106+0+11 (secure 0 0 0) 0x7f29f4056900 con 0x7f29fc159a20 2023-05-20T01:19:33.262 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-05-20T01:19:33.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.260+0000 7f2a01d1d700 1 -- 172.21.15.16:0/536018791 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f29dc06ace0 msgr2=0x7f29dc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:33.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.260+0000 7f2a01d1d700 1 --2- 172.21.15.16:0/536018791 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f29dc06ace0 0x7f29dc06d190 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f29ec000f80 tx=0x7f29ec009040 comp rx=0 tx=0).stop 2023-05-20T01:19:33.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.260+0000 7f2a01d1d700 1 -- 172.21.15.16:0/536018791 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29fc159a20 msgr2=0x7f29fc159e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:33.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.260+0000 7f2a01d1d700 1 --2- 172.21.15.16:0/536018791 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29fc159a20 0x7f29fc159e90 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f29fc0cec20 tx=0x7f29f400e860 comp rx=0 tx=0).stop 2023-05-20T01:19:33.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.260+0000 7f2a01d1d700 1 -- 172.21.15.16:0/536018791 shutdown_connections 2023-05-20T01:19:33.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.260+0000 7f2a01d1d700 1 --2- 172.21.15.16:0/536018791 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29fc159a20 0x7f29fc159e90 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:33.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.260+0000 7f2a01d1d700 1 --2- 172.21.15.16:0/536018791 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f29e80043f0 0x7f29fc1f3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:33.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.260+0000 7f2a01d1d700 1 --2- 172.21.15.16:0/536018791 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f29dc06ace0 0x7f29dc06d190 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:33.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.260+0000 7f2a01d1d700 1 --2- 172.21.15.16:0/536018791 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f29fc0b1840 0x7f29fc0b2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:33.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.260+0000 7f2a01d1d700 1 -- 172.21.15.16:0/536018791 >> 172.21.15.16:0/536018791 conn(0x7f29fc0cffd0 msgr2=0x7f29fc160800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:19:33.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.260+0000 7f2a01d1d700 1 -- 172.21.15.16:0/536018791 shutdown_connections 2023-05-20T01:19:33.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.260+0000 7f2a01d1d700 1 -- 172.21.15.16:0/536018791 wait complete. 2023-05-20T01:19:33.276 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-05-20T01:19:33.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.398+0000 7f50d23cd700 1 Processor -- start 2023-05-20T01:19:33.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.398+0000 7f50d23cd700 1 -- start start 2023-05-20T01:19:33.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.399+0000 7f50d23cd700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f50cc157190 0x7f50cc0cc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:33.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.399+0000 7f50d23cd700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f50cc0cccf0 0x7f50cc0713c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:33.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.399+0000 7f50d23cd700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50cc0cb590 con 0x7f50cc071900 2023-05-20T01:19:33.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.399+0000 7f50d23cd700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50cc0ca9a0 con 0x7f50cc0cccf0 2023-05-20T01:19:33.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.399+0000 7f50d23cd700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50cc0cac60 con 0x7f50cc157190 2023-05-20T01:19:33.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.399+0000 7f50d096a700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f50cc071900 0x7f50cc15d220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:44764/0 (socket says 172.21.15.16:44764) 2023-05-20T01:19:33.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.399+0000 7f50d096a700 1 -- 172.21.15.16:0/2328020693 learned_addr learned my addr 172.21.15.16:0/2328020693 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:19:33.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.400+0000 7f50cbfff700 1 --2- 172.21.15.16:0/2328020693 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f50cc157190 0x7f50cc0cc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:33.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.400+0000 7f50cb7fe700 1 --2- 172.21.15.16:0/2328020693 >> v2:172.21.15.132:3300/0 conn(0x7f50cc0cccf0 0x7f50cc0713c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:33.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.400+0000 7f50caffd700 1 -- 172.21.15.16:0/2328020693 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2431371387 0 0) 0x7f50cc0cb590 con 0x7f50cc071900 2023-05-20T01:19:33.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.400+0000 7f50caffd700 1 -- 172.21.15.16:0/2328020693 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50b80035b0 con 0x7f50cc071900 2023-05-20T01:19:33.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.400+0000 7f50cbfff700 1 -- 172.21.15.16:0/2328020693 >> v2:172.21.15.132:3300/0 conn(0x7f50cc0cccf0 msgr2=0x7f50cc0713c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:19:33.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.400+0000 7f50cbfff700 1 --2- 172.21.15.16:0/2328020693 >> v2:172.21.15.132:3300/0 conn(0x7f50cc0cccf0 0x7f50cc0713c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:33.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.400+0000 7f50cbfff700 1 -- 172.21.15.16:0/2328020693 >> v1:172.21.15.16:6789/0 conn(0x7f50cc071900 legacy=0x7f50cc15d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:33.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.400+0000 7f50cbfff700 1 -- 172.21.15.16:0/2328020693 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50cc0caf40 con 0x7f50cc157190 2023-05-20T01:19:33.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.400+0000 7f50cbfff700 1 --2- 172.21.15.16:0/2328020693 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f50cc157190 0x7f50cc0cc7b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f50bc009840 tx=0x7f50bc009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb55ef0d802ec5ef server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:33.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.400+0000 7f50caffd700 1 -- 172.21.15.16:0/2328020693 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50bc011830 con 0x7f50cc157190 2023-05-20T01:19:33.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.400+0000 7f50caffd700 1 -- 172.21.15.16:0/2328020693 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50bc011990 con 0x7f50cc157190 2023-05-20T01:19:33.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.400+0000 7f50caffd700 1 -- 172.21.15.16:0/2328020693 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50bc011d00 con 0x7f50cc157190 2023-05-20T01:19:33.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.401+0000 7f50d23cd700 1 -- 172.21.15.16:0/2328020693 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f50cc157190 msgr2=0x7f50cc0cc7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:33.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.401+0000 7f50d23cd700 1 --2- 172.21.15.16:0/2328020693 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f50cc157190 0x7f50cc0cc7b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f50bc009840 tx=0x7f50bc009b50 comp rx=0 tx=0).stop 2023-05-20T01:19:33.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.401+0000 7f50d23cd700 1 -- 172.21.15.16:0/2328020693 shutdown_connections 2023-05-20T01:19:33.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.401+0000 7f50d23cd700 1 --2- 172.21.15.16:0/2328020693 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f50cc157190 0x7f50cc0cc7b0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:33.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.401+0000 7f50d23cd700 1 --2- 172.21.15.16:0/2328020693 >> v2:172.21.15.132:3300/0 conn(0x7f50cc0cccf0 0x7f50cc0713c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:33.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.401+0000 7f50d23cd700 1 -- 172.21.15.16:0/2328020693 >> 172.21.15.16:0/2328020693 conn(0x7f50cc0cffb0 msgr2=0x7f50cc0d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:19:33.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.401+0000 7f50d23cd700 1 -- 172.21.15.16:0/2328020693 shutdown_connections 2023-05-20T01:19:33.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.401+0000 7f50d23cd700 1 -- 172.21.15.16:0/2328020693 wait complete. 2023-05-20T01:19:33.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.402+0000 7f50d23cd700 1 Processor -- start 2023-05-20T01:19:33.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.402+0000 7f50d23cd700 1 -- start start 2023-05-20T01:19:33.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.403+0000 7f50d23cd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f50cc071900 0x7f50cc147f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:33.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.403+0000 7f50d23cd700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f50cc0cccf0 0x7f50cc06eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:33.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.403+0000 7f50d23cd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50cc157190 0x7f50cc06f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:33.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.403+0000 7f50d23cd700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50cc14ce10 con 0x7f50cc157190 2023-05-20T01:19:33.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.403+0000 7f50d23cd700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50cc14c190 con 0x7f50cc071900 2023-05-20T01:19:33.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.403+0000 7f50d23cd700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50cc14c470 con 0x7f50cc0cccf0 2023-05-20T01:19:33.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.403+0000 7f50d096a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50cc157190 0x7f50cc06f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:33.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.403+0000 7f50cbfff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f50cc071900 0x7f50cc147f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:33.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.403+0000 7f50d096a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50cc157190 0x7f50cc06f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33242/0 (socket says 172.21.15.16:33242) 2023-05-20T01:19:33.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.403+0000 7f50d096a700 1 -- 172.21.15.16:0/643727329 learned_addr learned my addr 172.21.15.16:0/643727329 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:19:33.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.403+0000 7f50cb7fe700 1 --2- 172.21.15.16:0/643727329 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f50cc0cccf0 0x7f50cc06eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:33.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.404+0000 7f50d096a700 1 -- 172.21.15.16:0/643727329 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f50cc0cccf0 msgr2=0x7f50cc06eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:33.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.404+0000 7f50d096a700 1 --2- 172.21.15.16:0/643727329 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f50cc0cccf0 0x7f50cc06eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:33.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.404+0000 7f50d096a700 1 -- 172.21.15.16:0/643727329 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f50cc071900 msgr2=0x7f50cc147f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:33.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.404+0000 7f50d096a700 1 --2- 172.21.15.16:0/643727329 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f50cc071900 0x7f50cc147f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:33.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.404+0000 7f50d096a700 1 -- 172.21.15.16:0/643727329 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50bc0094a0 con 0x7f50cc157190 2023-05-20T01:19:33.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.404+0000 7f50d096a700 1 --2- 172.21.15.16:0/643727329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50cc157190 0x7f50cc06f0a0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f50c400cb40 tx=0x7f50c400ce50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:33.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.404+0000 7f50c8ff9700 1 -- 172.21.15.16:0/643727329 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c400a9e0 con 0x7f50cc157190 2023-05-20T01:19:33.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.404+0000 7f50c8ff9700 1 -- 172.21.15.16:0/643727329 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50c400ab40 con 0x7f50cc157190 2023-05-20T01:19:33.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.404+0000 7f50d23cd700 1 -- 172.21.15.16:0/643727329 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50cc157560 con 0x7f50cc157190 2023-05-20T01:19:33.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.404+0000 7f50c8ff9700 1 -- 172.21.15.16:0/643727329 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c4012400 con 0x7f50cc157190 2023-05-20T01:19:33.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.404+0000 7f50d23cd700 1 -- 172.21.15.16:0/643727329 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50cc0cb220 con 0x7f50cc157190 2023-05-20T01:19:33.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.405+0000 7f50d23cd700 1 -- 172.21.15.16:0/643727329 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50cc0cd120 con 0x7f50cc157190 2023-05-20T01:19:33.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.405+0000 7f50c8ff9700 1 -- 172.21.15.16:0/643727329 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f50c401b3b0 con 0x7f50cc157190 2023-05-20T01:19:33.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.406+0000 7f50c8ff9700 1 --2- 172.21.15.16:0/643727329 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f50ac06ace0 0x7f50ac06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:33.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.406+0000 7f50c8ff9700 1 -- 172.21.15.16:0/643727329 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 7102+0+0 (secure 0 0 0) 0x7f50c4089480 con 0x7f50cc157190 2023-05-20T01:19:33.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.406+0000 7f50cbfff700 1 --2- 172.21.15.16:0/643727329 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f50ac06ace0 0x7f50ac06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:33.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.406+0000 7f50cbfff700 1 --2- 172.21.15.16:0/643727329 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f50ac06ace0 0x7f50ac06d190 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f50bc000c00 tx=0x7f50bc01c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:33.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.410+0000 7f50c8ff9700 1 -- 172.21.15.16:0/643727329 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f50c4054150 con 0x7f50cc157190 2023-05-20T01:19:33.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.628+0000 7f50d23cd700 1 -- 172.21.15.16:0/643727329 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f50cc14cf50 con 0x7f50cc157190 2023-05-20T01:19:33.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.628+0000 7f50c8ff9700 1 -- 172.21.15.16:0/643727329 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 6 entries v51) v1 ==== 80+0+364 (secure 0 0 0) 0x7f50cc14cf50 con 0x7f50cc157190 2023-05-20T01:19:33.632 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6837/1867776973 2023-05-21T01:18:31.666399+0000 2023-05-20T01:19:33.632 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6835/1867776973 2023-05-21T01:18:31.666399+0000 2023-05-20T01:19:33.633 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6837/3517262366 2023-05-21T01:18:26.467438+0000 2023-05-20T01:19:33.633 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6835/3517262366 2023-05-21T01:18:26.467438+0000 2023-05-20T01:19:33.633 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6835/942011026 2023-05-21T01:18:24.644805+0000 2023-05-20T01:19:33.633 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6837/942011026 2023-05-21T01:18:24.644805+0000 2023-05-20T01:19:33.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.630+0000 7f50d23cd700 1 -- 172.21.15.16:0/643727329 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f50ac06ace0 msgr2=0x7f50ac06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:33.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.630+0000 7f50d23cd700 1 --2- 172.21.15.16:0/643727329 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f50ac06ace0 0x7f50ac06d190 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f50bc000c00 tx=0x7f50bc01c040 comp rx=0 tx=0).stop 2023-05-20T01:19:33.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.630+0000 7f50d23cd700 1 -- 172.21.15.16:0/643727329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50cc157190 msgr2=0x7f50cc06f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:33.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.630+0000 7f50d23cd700 1 --2- 172.21.15.16:0/643727329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50cc157190 0x7f50cc06f0a0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f50c400cb40 tx=0x7f50c400ce50 comp rx=0 tx=0).stop 2023-05-20T01:19:33.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.630+0000 7f50d23cd700 1 -- 172.21.15.16:0/643727329 shutdown_connections 2023-05-20T01:19:33.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.630+0000 7f50d23cd700 1 --2- 172.21.15.16:0/643727329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50cc157190 0x7f50cc06f0a0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:33.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.630+0000 7f50d23cd700 1 --2- 172.21.15.16:0/643727329 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f50cc071900 0x7f50cc147f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:33.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.630+0000 7f50d23cd700 1 --2- 172.21.15.16:0/643727329 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f50ac06ace0 0x7f50ac06d190 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:33.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.630+0000 7f50d23cd700 1 --2- 172.21.15.16:0/643727329 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f50cc0cccf0 0x7f50cc06eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:33.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.630+0000 7f50d23cd700 1 -- 172.21.15.16:0/643727329 >> 172.21.15.16:0/643727329 conn(0x7f50cc0cffb0 msgr2=0x7f50cc0d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:19:33.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.630+0000 7f50d23cd700 1 -- 172.21.15.16:0/643727329 shutdown_connections 2023-05-20T01:19:33.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.630+0000 7f50d23cd700 1 -- 172.21.15.16:0/643727329 wait complete. 2023-05-20T01:19:33.636 INFO:teuthology.orchestra.run.smithi016.stderr:listed 6 entries 2023-05-20T01:19:33.646 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-05-20T01:19:33.646 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:19:33.646 DEBUG:teuthology.orchestra.run.smithi016:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-05-20T01:19:33.690 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2023-05-20T01:19:33.690 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:19:33.690 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-05-20T01:19:33.747 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2023-05-20T01:19:33.748 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/proc/self/mounts of=/dev/stdout 2023-05-20T01:19:33.769 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284284k,nr_inodes=4071071,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=45,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16175 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260556k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-05-20T01:19:33.770 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-05-20T01:19:33.770 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2023-05-20T01:19:33.806 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16287364k,nr_inodes=4071841,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=38,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16696 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3261172k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-05-20T01:19:33.806 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2023-05-20T01:19:33.807 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/proc/self/mounts of=/dev/stdout 2023-05-20T01:19:33.829 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284284k,nr_inodes=4071071,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=45,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16175 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260556k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-05-20T01:19:33.830 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-20T01:19:33.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.965+0000 7ff44f7c3700 1 Processor -- start 2023-05-20T01:19:33.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.966+0000 7ff44f7c3700 1 -- start start 2023-05-20T01:19:33.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.966+0000 7ff44f7c3700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7ff4480b2760 0x7ff4480d1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:33.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.966+0000 7ff44f7c3700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff448144020 0x7ff4480b0980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:33.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.966+0000 7ff44f7c3700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4480cc500 con 0x7ff4480cec60 2023-05-20T01:19:33.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.967+0000 7ff44f7c3700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4480cb6e0 con 0x7ff4480b2760 2023-05-20T01:19:33.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.967+0000 7ff44f7c3700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4481448a0 con 0x7ff448144020 2023-05-20T01:19:33.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.967+0000 7ff44cd5e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff448144020 0x7ff4480b0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:33.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.967+0000 7ff44cd5e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff448144020 0x7ff4480b0980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.16:33286/0 (socket says 172.21.15.16:33286) 2023-05-20T01:19:33.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.967+0000 7ff44cd5e700 1 -- 172.21.15.16:0/202709223 learned_addr learned my addr 172.21.15.16:0/202709223 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:19:33.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.967+0000 7ff44d55f700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7ff4480b2760 0x7ff4480d1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:33.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.967+0000 7ff44cd5e700 1 -- 172.21.15.16:0/202709223 >> v2:172.21.15.132:3300/0 conn(0x7ff4480b2760 msgr2=0x7ff4480d1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:19:33.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.967+0000 7ff44cd5e700 1 --2- 172.21.15.16:0/202709223 >> v2:172.21.15.132:3300/0 conn(0x7ff4480b2760 0x7ff4480d1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:33.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.967+0000 7ff44cd5e700 1 -- 172.21.15.16:0/202709223 >> v1:172.21.15.132:6789/0 conn(0x7ff4480cec60 legacy=0x7ff4480cc990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:33.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.967+0000 7ff44cd5e700 1 -- 172.21.15.16:0/202709223 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4480b0ec0 con 0x7ff448144020 2023-05-20T01:19:33.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.967+0000 7ff44d55f700 1 --2- 172.21.15.16:0/202709223 >> v2:172.21.15.132:3300/0 conn(0x7ff4480b2760 0x7ff4480d1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-05-20T01:19:33.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.968+0000 7ff44cd5e700 1 --2- 172.21.15.16:0/202709223 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff448144020 0x7ff4480b0980 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff43c0098f0 tx=0x7ff43c009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3baf2661e170908 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:33.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.968+0000 7ff43bfff700 1 -- 172.21.15.16:0/202709223 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff43c011830 con 0x7ff448144020 2023-05-20T01:19:33.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.968+0000 7ff43bfff700 1 -- 172.21.15.16:0/202709223 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff43c011990 con 0x7ff448144020 2023-05-20T01:19:33.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.968+0000 7ff43bfff700 1 -- 172.21.15.16:0/202709223 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff43c011d20 con 0x7ff448144020 2023-05-20T01:19:33.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.968+0000 7ff44f7c3700 1 -- 172.21.15.16:0/202709223 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff448144020 msgr2=0x7ff4480b0980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:33.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.968+0000 7ff44f7c3700 1 --2- 172.21.15.16:0/202709223 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff448144020 0x7ff4480b0980 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff43c0098f0 tx=0x7ff43c009c00 comp rx=0 tx=0).stop 2023-05-20T01:19:33.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.968+0000 7ff44f7c3700 1 -- 172.21.15.16:0/202709223 shutdown_connections 2023-05-20T01:19:33.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.968+0000 7ff44f7c3700 1 --2- 172.21.15.16:0/202709223 >> v2:172.21.15.132:3300/0 conn(0x7ff4480b2760 0x7ff4480d1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:33.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.968+0000 7ff44f7c3700 1 --2- 172.21.15.16:0/202709223 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff448144020 0x7ff4480b0980 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:33.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.968+0000 7ff44f7c3700 1 -- 172.21.15.16:0/202709223 >> 172.21.15.16:0/202709223 conn(0x7ff44806ae30 msgr2=0x7ff44809a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:19:33.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.968+0000 7ff44f7c3700 1 -- 172.21.15.16:0/202709223 shutdown_connections 2023-05-20T01:19:33.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.968+0000 7ff44f7c3700 1 -- 172.21.15.16:0/202709223 wait complete. 2023-05-20T01:19:33.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.970+0000 7ff44f7c3700 1 Processor -- start 2023-05-20T01:19:33.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.970+0000 7ff44f7c3700 1 -- start start 2023-05-20T01:19:33.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.970+0000 7ff44f7c3700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff4480b2760 0x7ff448159df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:33.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.970+0000 7ff44f7c3700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff4480cec60 0x7ff4481595d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:33.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.970+0000 7ff44f7c3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff448144020 0x7ff4481f0560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:33.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.970+0000 7ff44f7c3700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff44809af60 con 0x7ff448144020 2023-05-20T01:19:33.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.970+0000 7ff44f7c3700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4480b6190 con 0x7ff4480b2760 2023-05-20T01:19:33.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.970+0000 7ff44f7c3700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4480c8d10 con 0x7ff4480cec60 2023-05-20T01:19:33.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.970+0000 7ff44dd60700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff448144020 0x7ff4481f0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:33.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.970+0000 7ff44dd60700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff448144020 0x7ff4481f0560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33256/0 (socket says 172.21.15.16:33256) 2023-05-20T01:19:33.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.970+0000 7ff44dd60700 1 -- 172.21.15.16:0/2578626687 learned_addr learned my addr 172.21.15.16:0/2578626687 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:19:33.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.970+0000 7ff44cd5e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff4480cec60 0x7ff4481595d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:33.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.971+0000 7ff44d55f700 1 --2- 172.21.15.16:0/2578626687 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff4480b2760 0x7ff448159df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:33.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.971+0000 7ff44dd60700 1 -- 172.21.15.16:0/2578626687 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff4480cec60 msgr2=0x7ff4481595d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:33.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.971+0000 7ff44dd60700 1 --2- 172.21.15.16:0/2578626687 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff4480cec60 0x7ff4481595d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:33.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.971+0000 7ff44dd60700 1 -- 172.21.15.16:0/2578626687 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff4480b2760 msgr2=0x7ff448159df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:33.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.971+0000 7ff44dd60700 1 --2- 172.21.15.16:0/2578626687 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff4480b2760 0x7ff448159df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:33.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.971+0000 7ff44dd60700 1 -- 172.21.15.16:0/2578626687 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4440094a0 con 0x7ff448144020 2023-05-20T01:19:33.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.971+0000 7ff44dd60700 1 --2- 172.21.15.16:0/2578626687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff448144020 0x7ff4481f0560 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7ff44000cb50 tx=0x7ff44000ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:33.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.971+0000 7ff439ffb700 1 -- 172.21.15.16:0/2578626687 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff440003c00 con 0x7ff448144020 2023-05-20T01:19:33.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.971+0000 7ff439ffb700 1 -- 172.21.15.16:0/2578626687 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff440003d60 con 0x7ff448144020 2023-05-20T01:19:33.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.971+0000 7ff44f7c3700 1 -- 172.21.15.16:0/2578626687 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff43c009550 con 0x7ff448144020 2023-05-20T01:19:33.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.971+0000 7ff439ffb700 1 -- 172.21.15.16:0/2578626687 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff44000aa20 con 0x7ff448144020 2023-05-20T01:19:33.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.971+0000 7ff44f7c3700 1 -- 172.21.15.16:0/2578626687 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4480cbbd0 con 0x7ff448144020 2023-05-20T01:19:33.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.972+0000 7ff44f7c3700 1 -- 172.21.15.16:0/2578626687 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4480cb8f0 con 0x7ff448144020 2023-05-20T01:19:33.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.973+0000 7ff439ffb700 1 -- 172.21.15.16:0/2578626687 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff44000ab80 con 0x7ff448144020 2023-05-20T01:19:33.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.973+0000 7ff439ffb700 1 --2- 172.21.15.16:0/2578626687 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff42c06ace0 0x7ff42c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:33.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.973+0000 7ff439ffb700 1 -- 172.21.15.16:0/2578626687 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 7102+0+0 (secure 0 0 0) 0x7ff4400898a0 con 0x7ff448144020 2023-05-20T01:19:33.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.973+0000 7ff44d55f700 1 --2- 172.21.15.16:0/2578626687 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff42c06ace0 0x7ff42c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:33.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.974+0000 7ff44d55f700 1 --2- 172.21.15.16:0/2578626687 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff42c06ace0 0x7ff42c06d190 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7ff444009cd0 tx=0x7ff444009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:33.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:33.977+0000 7ff439ffb700 1 -- 172.21.15.16:0/2578626687 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7ff440054540 con 0x7ff448144020 2023-05-20T01:19:34.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.220+0000 7ff44f7c3700 1 -- 172.21.15.16:0/2578626687 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff4480cbeb0 con 0x7ff448144020 2023-05-20T01:19:34.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.220+0000 7ff439ffb700 1 -- 172.21.15.16:0/2578626687 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+4680 (secure 0 0 0) 0x7ff440057b60 con 0x7ff448144020 2023-05-20T01:19:34.224 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:19:34.225 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":29,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5246,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/2629811751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":2629811751},{"type":"v1","addr":"172.21.15.132:6834","nonce":2629811751}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":5249,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2117148777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":2117148777},{"type":"v1","addr":"172.21.15.132:6835","nonce":2117148777}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":5566,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2025052254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":2025052254},{"type":"v1","addr":"172.21.15.16:6837","nonce":2025052254}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-20T01:18:26.227827+0000","modified":"2023-05-20T01:18:35.503777+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5889},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5889":{"gid":5889,"name":"c","rank":0,"incarnation":26,"state":"up:active","state_seq":4,"addr":"172.21.15.16:6836/3284962389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3284962389},{"type":"v1","addr":"172.21.15.16:6836","nonce":3284962389}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7,8],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-05-20T01:19:34.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.222+0000 7ff44f7c3700 1 -- 172.21.15.16:0/2578626687 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff42c06ace0 msgr2=0x7ff42c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:34.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.222+0000 7ff44f7c3700 1 --2- 172.21.15.16:0/2578626687 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff42c06ace0 0x7ff42c06d190 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7ff444009cd0 tx=0x7ff444009210 comp rx=0 tx=0).stop 2023-05-20T01:19:34.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.222+0000 7ff44f7c3700 1 -- 172.21.15.16:0/2578626687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff448144020 msgr2=0x7ff4481f0560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:34.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.222+0000 7ff44f7c3700 1 --2- 172.21.15.16:0/2578626687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff448144020 0x7ff4481f0560 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7ff44000cb50 tx=0x7ff44000ce60 comp rx=0 tx=0).stop 2023-05-20T01:19:34.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.222+0000 7ff44f7c3700 1 -- 172.21.15.16:0/2578626687 shutdown_connections 2023-05-20T01:19:34.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.222+0000 7ff44f7c3700 1 --2- 172.21.15.16:0/2578626687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff448144020 0x7ff4481f0560 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:34.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.222+0000 7ff44f7c3700 1 --2- 172.21.15.16:0/2578626687 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff4480b2760 0x7ff448159df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:34.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.222+0000 7ff44f7c3700 1 --2- 172.21.15.16:0/2578626687 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff42c06ace0 0x7ff42c06d190 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:34.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.222+0000 7ff44f7c3700 1 --2- 172.21.15.16:0/2578626687 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff4480cec60 0x7ff4481595d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:34.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.222+0000 7ff44f7c3700 1 -- 172.21.15.16:0/2578626687 >> 172.21.15.16:0/2578626687 conn(0x7ff44806ae30 msgr2=0x7ff44806eb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:19:34.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.222+0000 7ff44f7c3700 1 -- 172.21.15.16:0/2578626687 shutdown_connections 2023-05-20T01:19:34.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.222+0000 7ff44f7c3700 1 -- 172.21.15.16:0/2578626687 wait complete. 2023-05-20T01:19:34.231 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 29 2023-05-20T01:19:34.239 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-20T01:19:34.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.362+0000 7fe1def3d700 1 Processor -- start 2023-05-20T01:19:34.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.362+0000 7fe1def3d700 1 -- start start 2023-05-20T01:19:34.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.363+0000 7fe1def3d700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fe1d80caaa0 0x7fe1d80cea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:34.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.363+0000 7fe1def3d700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1d8159d90 con 0x7fe1d80cb670 2023-05-20T01:19:34.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.363+0000 7fe1def3d700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1d80cdb80 con 0x7fe1d81599c0 2023-05-20T01:19:34.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.363+0000 7fe1def3d700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1d80cdf20 con 0x7fe1d80caaa0 2023-05-20T01:19:34.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.363+0000 7fe1d7fff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fe1d80cb670 0x7fe1d80ce340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:44790/0 (socket says 172.21.15.16:44790) 2023-05-20T01:19:34.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.363+0000 7fe1d7fff700 1 -- 172.21.15.16:0/1260797472 learned_addr learned my addr 172.21.15.16:0/1260797472 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:19:34.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.363+0000 7fe1d77fe700 1 -- 172.21.15.16:0/1260797472 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4286380525 0 0) 0x7fe1c4001110 con 0x7fe1d80cb670 2023-05-20T01:19:34.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.363+0000 7fe1dd4da700 1 --2- 172.21.15.16:0/1260797472 >> v2:172.21.15.132:3300/0 conn(0x7fe1d80caaa0 0x7fe1d80cea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:34.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.363+0000 7fe1d77fe700 1 -- 172.21.15.16:0/1260797472 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe1c80035b0 con 0x7fe1d80cb670 2023-05-20T01:19:34.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.364+0000 7fe1d77fe700 1 -- 172.21.15.16:0/1260797472 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1196346234 0 0) 0x7fe1c80035b0 con 0x7fe1d80cb670 2023-05-20T01:19:34.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.364+0000 7fe1d77fe700 1 -- 172.21.15.16:0/1260797472 >> v2:172.21.15.132:3300/0 conn(0x7fe1d80caaa0 msgr2=0x7fe1d80cea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:19:34.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.364+0000 7fe1d77fe700 1 --2- 172.21.15.16:0/1260797472 >> v2:172.21.15.132:3300/0 conn(0x7fe1d80caaa0 0x7fe1d80cea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:34.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.364+0000 7fe1d77fe700 1 -- 172.21.15.16:0/1260797472 >> v1:172.21.15.132:6789/0 conn(0x7fe1d81599c0 legacy=0x7fe1d80caf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:34.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.364+0000 7fe1d77fe700 1 -- 172.21.15.16:0/1260797472 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1d806eaf0 con 0x7fe1d80cb670 2023-05-20T01:19:34.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.364+0000 7fe1d77fe700 1 -- 172.21.15.16:0/1260797472 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7fe1c4001800 con 0x7fe1d80cb670 2023-05-20T01:19:34.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.364+0000 7fe1d77fe700 1 -- 172.21.15.16:0/1260797472 >> v1:172.21.15.16:6789/0 conn(0x7fe1d80cb670 legacy=0x7fe1d80ce340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:34.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.364+0000 7fe1d77fe700 1 --2- 172.21.15.16:0/1260797472 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe1c8003a70 0x7fe1c8003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:34.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.364+0000 7fe1d77fe700 1 --2- 172.21.15.16:0/1260797472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1c8004850 0x7fe1c8004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:34.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.364+0000 7fe1d77fe700 1 --2- 172.21.15.16:0/1260797472 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe1c8005200 0x7fe1c800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:34.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.364+0000 7fe1d77fe700 1 -- 172.21.15.16:0/1260797472 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d80cdf20 con 0x7fe1c8004850 2023-05-20T01:19:34.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.364+0000 7fe1d77fe700 1 -- 172.21.15.16:0/1260797472 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c800c0e0 con 0x7fe1c8003a70 2023-05-20T01:19:34.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.364+0000 7fe1d77fe700 1 -- 172.21.15.16:0/1260797472 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c800c2a0 con 0x7fe1c8005200 2023-05-20T01:19:34.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.364+0000 7fe1d7fff700 1 --2- 172.21.15.16:0/1260797472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1c8004850 0x7fe1c8004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:34.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.364+0000 7fe1dccd9700 1 --2- 172.21.15.16:0/1260797472 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe1c8003a70 0x7fe1c8003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:34.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.364+0000 7fe1dd4da700 1 --2- 172.21.15.16:0/1260797472 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe1c8005200 0x7fe1c800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:34.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.364+0000 7fe1d7fff700 1 -- 172.21.15.16:0/1260797472 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe1c8005200 msgr2=0x7fe1c800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:19:34.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.364+0000 7fe1d7fff700 1 --2- 172.21.15.16:0/1260797472 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe1c8005200 0x7fe1c800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:34.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.364+0000 7fe1d7fff700 1 -- 172.21.15.16:0/1260797472 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe1c8003a70 msgr2=0x7fe1c8003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:19:34.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.364+0000 7fe1d7fff700 1 --2- 172.21.15.16:0/1260797472 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe1c8003a70 0x7fe1c8003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:34.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.364+0000 7fe1dd4da700 1 -- 172.21.15.16:0/1260797472 reap_dead start 2023-05-20T01:19:34.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.364+0000 7fe1d7fff700 1 -- 172.21.15.16:0/1260797472 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1c80035b0 con 0x7fe1c8004850 2023-05-20T01:19:34.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.364+0000 7fe1d7fff700 1 --2- 172.21.15.16:0/1260797472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1c8004850 0x7fe1c8004cc0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fe1c400c240 tx=0x7fe1c8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ad6970b31540003 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:34.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.364+0000 7fe1d77fe700 1 -- 172.21.15.16:0/1260797472 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1c4007db0 con 0x7fe1c8004850 2023-05-20T01:19:34.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.365+0000 7fe1d77fe700 1 -- 172.21.15.16:0/1260797472 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1c400cd60 con 0x7fe1c8004850 2023-05-20T01:19:34.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.365+0000 7fe1d77fe700 1 -- 172.21.15.16:0/1260797472 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1c4005b80 con 0x7fe1c8004850 2023-05-20T01:19:34.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.365+0000 7fe1def3d700 1 -- 172.21.15.16:0/1260797472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1c8004850 msgr2=0x7fe1c8004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:34.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.365+0000 7fe1def3d700 1 --2- 172.21.15.16:0/1260797472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1c8004850 0x7fe1c8004cc0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fe1c400c240 tx=0x7fe1c8005880 comp rx=0 tx=0).stop 2023-05-20T01:19:34.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.365+0000 7fe1def3d700 1 -- 172.21.15.16:0/1260797472 shutdown_connections 2023-05-20T01:19:34.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.365+0000 7fe1def3d700 1 --2- 172.21.15.16:0/1260797472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1c8004850 0x7fe1c8004cc0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:34.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.365+0000 7fe1def3d700 1 -- 172.21.15.16:0/1260797472 >> 172.21.15.16:0/1260797472 conn(0x7fe1d80d0c30 msgr2=0x7fe1d80b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:19:34.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.365+0000 7fe1def3d700 1 -- 172.21.15.16:0/1260797472 shutdown_connections 2023-05-20T01:19:34.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.365+0000 7fe1def3d700 1 -- 172.21.15.16:0/1260797472 wait complete. 2023-05-20T01:19:34.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.367+0000 7fe1def3d700 1 Processor -- start 2023-05-20T01:19:34.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.367+0000 7fe1def3d700 1 -- start start 2023-05-20T01:19:34.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.368+0000 7fe1def3d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1c8004850 0x7fe1d8148f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:34.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.368+0000 7fe1def3d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe1d80b2e40 0x7fe1d80b0030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:34.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.368+0000 7fe1def3d700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe1d80b0570 0x7fe1d80ae540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:34.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.368+0000 7fe1def3d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d80af5f0 con 0x7fe1c8004850 2023-05-20T01:19:34.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.368+0000 7fe1def3d700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d80b09e0 con 0x7fe1d80b2e40 2023-05-20T01:19:34.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.368+0000 7fe1def3d700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d81494d0 con 0x7fe1d80b0570 2023-05-20T01:19:34.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.368+0000 7fe1dccd9700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1c8004850 0x7fe1d8148f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:34.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.368+0000 7fe1dccd9700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1c8004850 0x7fe1d8148f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33272/0 (socket says 172.21.15.16:33272) 2023-05-20T01:19:34.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.368+0000 7fe1dccd9700 1 -- 172.21.15.16:0/3263340752 learned_addr learned my addr 172.21.15.16:0/3263340752 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:19:34.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.368+0000 7fe1dd4da700 1 --2- 172.21.15.16:0/3263340752 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe1d80b0570 0x7fe1d80ae540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:34.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.368+0000 7fe1d7fff700 1 --2- 172.21.15.16:0/3263340752 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe1d80b2e40 0x7fe1d80b0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:34.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.368+0000 7fe1dccd9700 1 -- 172.21.15.16:0/3263340752 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe1d80b0570 msgr2=0x7fe1d80ae540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:34.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.368+0000 7fe1dccd9700 1 --2- 172.21.15.16:0/3263340752 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe1d80b0570 0x7fe1d80ae540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:34.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.368+0000 7fe1dccd9700 1 -- 172.21.15.16:0/3263340752 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe1d80b2e40 msgr2=0x7fe1d80b0030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:34.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.368+0000 7fe1dccd9700 1 --2- 172.21.15.16:0/3263340752 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe1d80b2e40 0x7fe1d80b0030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:34.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.368+0000 7fe1dccd9700 1 -- 172.21.15.16:0/3263340752 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1d806f0b0 con 0x7fe1c8004850 2023-05-20T01:19:34.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.368+0000 7fe1d7fff700 1 --2- 172.21.15.16:0/3263340752 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe1d80b2e40 0x7fe1d80b0030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:19:34.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.368+0000 7fe1dccd9700 1 --2- 172.21.15.16:0/3263340752 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1c8004850 0x7fe1d8148f90 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fe1cc00f7a0 tx=0x7fe1cc00fb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:34.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.369+0000 7fe1d57fa700 1 -- 172.21.15.16:0/3263340752 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1cc0052e0 con 0x7fe1c8004850 2023-05-20T01:19:34.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.369+0000 7fe1d57fa700 1 -- 172.21.15.16:0/3263340752 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1cc005440 con 0x7fe1c8004850 2023-05-20T01:19:34.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.369+0000 7fe1d57fa700 1 -- 172.21.15.16:0/3263340752 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1cc0038a0 con 0x7fe1c8004850 2023-05-20T01:19:34.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.369+0000 7fe1def3d700 1 -- 172.21.15.16:0/3263340752 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1d806edd0 con 0x7fe1c8004850 2023-05-20T01:19:34.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.369+0000 7fe1def3d700 1 -- 172.21.15.16:0/3263340752 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1d806e750 con 0x7fe1c8004850 2023-05-20T01:19:34.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.369+0000 7fe1def3d700 1 -- 172.21.15.16:0/3263340752 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1d80747b0 con 0x7fe1c8004850 2023-05-20T01:19:34.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.370+0000 7fe1d57fa700 1 -- 172.21.15.16:0/3263340752 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe1cc003a00 con 0x7fe1c8004850 2023-05-20T01:19:34.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.370+0000 7fe1d57fa700 1 --2- 172.21.15.16:0/3263340752 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe1bc06ace0 0x7fe1bc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:34.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.370+0000 7fe1d57fa700 1 -- 172.21.15.16:0/3263340752 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 7102+0+0 (secure 0 0 0) 0x7fe1cc089570 con 0x7fe1c8004850 2023-05-20T01:19:34.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.371+0000 7fe1d7fff700 1 --2- 172.21.15.16:0/3263340752 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe1bc06ace0 0x7fe1bc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:34.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.371+0000 7fe1d7fff700 1 --2- 172.21.15.16:0/3263340752 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe1bc06ace0 0x7fe1bc06d190 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fe1d8073340 tx=0x7fe1c4007000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:34.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.375+0000 7fe1d57fa700 1 -- 172.21.15.16:0/3263340752 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fe1cc054200 con 0x7fe1c8004850 2023-05-20T01:19:34.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.577+0000 7fe1def3d700 1 -- 172.21.15.16:0/3263340752 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe1d80af310 con 0x7fe1c8004850 2023-05-20T01:19:34.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.578+0000 7fe1d57fa700 1 -- 172.21.15.16:0/3263340752 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) v1 ==== 74+0+20668 (secure 0 0 0) 0x7fe1cc003cb0 con 0x7fe1c8004850 2023-05-20T01:19:34.582 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:19:34.582 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":51,"fsid":"c2245cc5-ecab-4087-a59b-bc81db901676","created":"2023-05-20T01:16:09.549230+0000","modified":"2023-05-20T01:18:31.674809+0000","last_up_change":"2023-05-20T01:16:19.788624+0000","last_in_change":"2023-05-20T01:16:12.092160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-05-20T01:16:20.934366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-05-20T01:16:21.372294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-05-20T01:17:44.628755+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-05-20T01:17:45.968584+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data_extra","create_time":"2023-05-20T01:17:54.915781+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"317a83cd-19c8-4958-a301-6087ef342289","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6825","nonce":3923634179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6827","nonce":3923634179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6831","nonce":3923634179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6829","nonce":3923634179}]},"public_addr":"172.21.15.16:6825/3923634179","cluster_addr":"172.21.15.16:6827/3923634179","heartbeat_back_addr":"172.21.15.16:6831/3923634179","heartbeat_front_addr":"172.21.15.16:6829/3923634179","state":["exists","up"]},{"osd":1,"uuid":"5581046f-8219-4089-8d7d-b7846399505e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6809","nonce":3860264220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6811","nonce":3860264220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6815","nonce":3860264220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6813","nonce":3860264220}]},"public_addr":"172.21.15.16:6809/3860264220","cluster_addr":"172.21.15.16:6811/3860264220","heartbeat_back_addr":"172.21.15.16:6815/3860264220","heartbeat_front_addr":"172.21.15.16:6813/3860264220","state":["exists","up"]},{"osd":2,"uuid":"072e9e66-4133-478e-8d4a-64d46c2963f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6817","nonce":293187030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6819","nonce":293187030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6823","nonce":293187030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6821","nonce":293187030}]},"public_addr":"172.21.15.16:6817/293187030","cluster_addr":"172.21.15.16:6819/293187030","heartbeat_back_addr":"172.21.15.16:6823/293187030","heartbeat_front_addr":"172.21.15.16:6821/293187030","state":["exists","up"]},{"osd":3,"uuid":"11e1bba6-00f8-48dd-9028-d8d716b7f145","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6801","nonce":3669781890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6803","nonce":3669781890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6807","nonce":3669781890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6805","nonce":3669781890}]},"public_addr":"172.21.15.16:6801/3669781890","cluster_addr":"172.21.15.16:6803/3669781890","heartbeat_back_addr":"172.21.15.16:6807/3669781890","heartbeat_front_addr":"172.21.15.16:6805/3669781890","state":["exists","up"]},{"osd":4,"uuid":"b8909c3a-cf70-46cb-8d0b-009c707cf9d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6819","nonce":506317223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6823","nonce":506317223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6829","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6831","nonce":506317223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6827","nonce":506317223}]},"public_addr":"172.21.15.132:6819/506317223","cluster_addr":"172.21.15.132:6823/506317223","heartbeat_back_addr":"172.21.15.132:6831/506317223","heartbeat_front_addr":"172.21.15.132:6827/506317223","state":["exists","up"]},{"osd":5,"uuid":"b1818304-4583-47bf-a6ec-0bfbb5166233","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6818","nonce":1844689995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6822","nonce":1844689995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6830","nonce":1844689995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6826","nonce":1844689995}]},"public_addr":"172.21.15.132:6818/1844689995","cluster_addr":"172.21.15.132:6822/1844689995","heartbeat_back_addr":"172.21.15.132:6830/1844689995","heartbeat_front_addr":"172.21.15.132:6826/1844689995","state":["exists","up"]},{"osd":6,"uuid":"d37f4691-eed1-4a93-8818-18354f03f4a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6801","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6802","nonce":727581123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6805","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6807","nonce":727581123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6813","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6815","nonce":727581123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6811","nonce":727581123}]},"public_addr":"172.21.15.132:6802/727581123","cluster_addr":"172.21.15.132:6807/727581123","heartbeat_back_addr":"172.21.15.132:6815/727581123","heartbeat_front_addr":"172.21.15.132:6811/727581123","state":["exists","up"]},{"osd":7,"uuid":"4f712f62-5d1d-4816-8042-03209144f8b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6803","nonce":466001149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6806","nonce":466001149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6814","nonce":466001149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6810","nonce":466001149}]},"public_addr":"172.21.15.132:6803/466001149","cluster_addr":"172.21.15.132:6806/466001149","heartbeat_back_addr":"172.21.15.132:6814/466001149","heartbeat_front_addr":"172.21.15.132:6810/466001149","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.338863+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.278680+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305694+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.246835+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.276237+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.304441+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305421+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.267050+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.9","mappings":[{"from":0,"to":4}]},{"pgid":"7.1b","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6837/1867776973":"2023-05-21T01:18:31.666399+0000","172.21.15.16:6835/1867776973":"2023-05-21T01:18:31.666399+0000","172.21.15.16:6837/3517262366":"2023-05-21T01:18:26.467438+0000","172.21.15.16:6835/3517262366":"2023-05-21T01:18:26.467438+0000","172.21.15.16:6835/942011026":"2023-05-21T01:18:24.644805+0000","172.21.15.16:6837/942011026":"2023-05-21T01:18:24.644805+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-20T01:19:34.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.581+0000 7fe1def3d700 1 -- 172.21.15.16:0/3263340752 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe1bc06ace0 msgr2=0x7fe1bc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:34.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.581+0000 7fe1def3d700 1 --2- 172.21.15.16:0/3263340752 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe1bc06ace0 0x7fe1bc06d190 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fe1d8073340 tx=0x7fe1c4007000 comp rx=0 tx=0).stop 2023-05-20T01:19:34.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.581+0000 7fe1def3d700 1 -- 172.21.15.16:0/3263340752 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1c8004850 msgr2=0x7fe1d8148f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:34.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.581+0000 7fe1def3d700 1 --2- 172.21.15.16:0/3263340752 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1c8004850 0x7fe1d8148f90 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fe1cc00f7a0 tx=0x7fe1cc00fb60 comp rx=0 tx=0).stop 2023-05-20T01:19:34.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.581+0000 7fe1def3d700 1 -- 172.21.15.16:0/3263340752 shutdown_connections 2023-05-20T01:19:34.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.581+0000 7fe1def3d700 1 --2- 172.21.15.16:0/3263340752 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe1bc06ace0 0x7fe1bc06d190 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:34.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.581+0000 7fe1def3d700 1 --2- 172.21.15.16:0/3263340752 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe1d80b0570 0x7fe1d80ae540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:34.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.581+0000 7fe1def3d700 1 --2- 172.21.15.16:0/3263340752 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1c8004850 0x7fe1d8148f90 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:34.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.581+0000 7fe1def3d700 1 --2- 172.21.15.16:0/3263340752 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe1d80b2e40 0x7fe1d80b0030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:34.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.581+0000 7fe1def3d700 1 -- 172.21.15.16:0/3263340752 >> 172.21.15.16:0/3263340752 conn(0x7fe1d80d0c30 msgr2=0x7fe1d80b53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:19:34.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.581+0000 7fe1def3d700 1 -- 172.21.15.16:0/3263340752 shutdown_connections 2023-05-20T01:19:34.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.581+0000 7fe1def3d700 1 -- 172.21.15.16:0/3263340752 wait complete. 2023-05-20T01:19:34.597 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-05-20T01:16:20.934366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-05-20T01:16:21.372294+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2023-05-20T01:17:44.628755+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '39', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2023-05-20T01:17:45.968584+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '40', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data_extra', 'create_time': '2023-05-20T01:17:54.915781+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '43', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-05-20T01:19:34.598 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-05-20T01:19:34.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.722+0000 7fd6f1d53700 1 Processor -- start 2023-05-20T01:19:34.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.722+0000 7fd6f1d53700 1 -- start start 2023-05-20T01:19:34.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.722+0000 7fd6f1d53700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd6ec0b57b0 0x7fd6ec071c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:34.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.722+0000 7fd6f1d53700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fd6ec0ccc50 0x7fd6ec072170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:34.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.723+0000 7fd6f1d53700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6ec06a5b0 con 0x7fd6ec15f000 2023-05-20T01:19:34.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.723+0000 7fd6f1d53700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6ec06a850 con 0x7fd6ec0b57b0 2023-05-20T01:19:34.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.723+0000 7fd6f1d53700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6ec06ab30 con 0x7fd6ec0ccc50 2023-05-20T01:19:34.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.723+0000 7fd6eb7fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd6ec0b57b0 0x7fd6ec071c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:34.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.723+0000 7fd6eaffd700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fd6ec15f000 0x7fd6ec06d4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:44808/0 (socket says 172.21.15.16:44808) 2023-05-20T01:19:34.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.723+0000 7fd6eb7fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd6ec0b57b0 0x7fd6ec071c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33280/0 (socket says 172.21.15.16:33280) 2023-05-20T01:19:34.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.723+0000 7fd6eaffd700 1 -- 172.21.15.16:0/4094741342 learned_addr learned my addr 172.21.15.16:0/4094741342 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:19:34.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.723+0000 7fd6ebfff700 1 --2- 172.21.15.16:0/4094741342 >> v2:172.21.15.132:3300/0 conn(0x7fd6ec0ccc50 0x7fd6ec072170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:34.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.723+0000 7fd6eb7fe700 1 -- 172.21.15.16:0/4094741342 >> v2:172.21.15.132:3300/0 conn(0x7fd6ec0ccc50 msgr2=0x7fd6ec072170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:19:34.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.723+0000 7fd6eb7fe700 1 --2- 172.21.15.16:0/4094741342 >> v2:172.21.15.132:3300/0 conn(0x7fd6ec0ccc50 0x7fd6ec072170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:34.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.723+0000 7fd6ea7fc700 1 -- 172.21.15.16:0/4094741342 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1656914406 0 0) 0x7fd6ec06a5b0 con 0x7fd6ec15f000 2023-05-20T01:19:34.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.723+0000 7fd6eb7fe700 1 -- 172.21.15.16:0/4094741342 >> v1:172.21.15.16:6789/0 conn(0x7fd6ec15f000 legacy=0x7fd6ec06d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:34.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.723+0000 7fd6eb7fe700 1 -- 172.21.15.16:0/4094741342 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6ec06ae10 con 0x7fd6ec0b57b0 2023-05-20T01:19:34.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.723+0000 7fd6ebfff700 1 --2- 172.21.15.16:0/4094741342 >> v2:172.21.15.132:3300/0 conn(0x7fd6ec0ccc50 0x7fd6ec072170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:19:34.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.723+0000 7fd6eb7fe700 1 --2- 172.21.15.16:0/4094741342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6ec0b57b0 0x7fd6ec071c30 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec0ca6b0 tx=0x7fd6dc009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bab30f9b98c84b6 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:34.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.723+0000 7fd6ea7fc700 1 -- 172.21.15.16:0/4094741342 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6dc011830 con 0x7fd6ec0b57b0 2023-05-20T01:19:34.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.723+0000 7fd6ea7fc700 1 -- 172.21.15.16:0/4094741342 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6dc011990 con 0x7fd6ec0b57b0 2023-05-20T01:19:34.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.723+0000 7fd6ea7fc700 1 -- 172.21.15.16:0/4094741342 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6dc011d20 con 0x7fd6ec0b57b0 2023-05-20T01:19:34.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.724+0000 7fd6f1d53700 1 -- 172.21.15.16:0/4094741342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6ec0b57b0 msgr2=0x7fd6ec071c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:34.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.724+0000 7fd6f1d53700 1 --2- 172.21.15.16:0/4094741342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6ec0b57b0 0x7fd6ec071c30 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec0ca6b0 tx=0x7fd6dc009b20 comp rx=0 tx=0).stop 2023-05-20T01:19:34.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.724+0000 7fd6f1d53700 1 -- 172.21.15.16:0/4094741342 shutdown_connections 2023-05-20T01:19:34.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.724+0000 7fd6f1d53700 1 --2- 172.21.15.16:0/4094741342 >> v2:172.21.15.132:3300/0 conn(0x7fd6ec0ccc50 0x7fd6ec072170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:34.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.724+0000 7fd6f1d53700 1 --2- 172.21.15.16:0/4094741342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6ec0b57b0 0x7fd6ec071c30 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:34.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.724+0000 7fd6f1d53700 1 -- 172.21.15.16:0/4094741342 >> 172.21.15.16:0/4094741342 conn(0x7fd6ec06c260 msgr2=0x7fd6ec0cffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:19:34.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.724+0000 7fd6f1d53700 1 -- 172.21.15.16:0/4094741342 shutdown_connections 2023-05-20T01:19:34.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.724+0000 7fd6f1d53700 1 -- 172.21.15.16:0/4094741342 wait complete. 2023-05-20T01:19:34.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.725+0000 7fd6f1d53700 1 Processor -- start 2023-05-20T01:19:34.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.725+0000 7fd6f1d53700 1 -- start start 2023-05-20T01:19:34.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.726+0000 7fd6f1d53700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd6ec0b57b0 0x7fd6ec1483f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:34.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.726+0000 7fd6f1d53700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6ec0ccc50 0x7fd6ec0c98c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:34.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.726+0000 7fd6f1d53700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd6ec15f000 0x7fd6ec15cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:34.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.726+0000 7fd6f1d53700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ec148930 con 0x7fd6ec0ccc50 2023-05-20T01:19:34.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.726+0000 7fd6f1d53700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ec0c3660 con 0x7fd6ec0b57b0 2023-05-20T01:19:34.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.726+0000 7fd6f1d53700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ec0c3940 con 0x7fd6ec15f000 2023-05-20T01:19:34.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.726+0000 7fd6eaffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6ec0ccc50 0x7fd6ec0c98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:34.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.726+0000 7fd6eaffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6ec0ccc50 0x7fd6ec0c98c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33288/0 (socket says 172.21.15.16:33288) 2023-05-20T01:19:34.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.726+0000 7fd6eaffd700 1 -- 172.21.15.16:0/2061517466 learned_addr learned my addr 172.21.15.16:0/2061517466 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:19:34.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.726+0000 7fd6ebfff700 1 --2- 172.21.15.16:0/2061517466 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd6ec15f000 0x7fd6ec15cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:34.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.726+0000 7fd6eb7fe700 1 --2- 172.21.15.16:0/2061517466 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd6ec0b57b0 0x7fd6ec1483f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:34.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.726+0000 7fd6eaffd700 1 -- 172.21.15.16:0/2061517466 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd6ec15f000 msgr2=0x7fd6ec15cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:34.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.726+0000 7fd6eaffd700 1 --2- 172.21.15.16:0/2061517466 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd6ec15f000 0x7fd6ec15cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:34.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.726+0000 7fd6eaffd700 1 -- 172.21.15.16:0/2061517466 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd6ec0b57b0 msgr2=0x7fd6ec1483f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:34.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.726+0000 7fd6eaffd700 1 --2- 172.21.15.16:0/2061517466 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd6ec0b57b0 0x7fd6ec1483f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:34.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.726+0000 7fd6eaffd700 1 -- 172.21.15.16:0/2061517466 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6dc0094a0 con 0x7fd6ec0ccc50 2023-05-20T01:19:34.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.727+0000 7fd6eb7fe700 1 --2- 172.21.15.16:0/2061517466 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd6ec0b57b0 0x7fd6ec1483f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:19:34.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.727+0000 7fd6eaffd700 1 --2- 172.21.15.16:0/2061517466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6ec0ccc50 0x7fd6ec0c98c0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fd6e000c520 tx=0x7fd6e000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:34.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.727+0000 7fd6d3fff700 1 -- 172.21.15.16:0/2061517466 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6e0004c80 con 0x7fd6ec0ccc50 2023-05-20T01:19:34.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.727+0000 7fd6d3fff700 1 -- 172.21.15.16:0/2061517466 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6e0004de0 con 0x7fd6ec0ccc50 2023-05-20T01:19:34.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.727+0000 7fd6f1d53700 1 -- 172.21.15.16:0/2061517466 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6ec0af710 con 0x7fd6ec0ccc50 2023-05-20T01:19:34.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.727+0000 7fd6d3fff700 1 -- 172.21.15.16:0/2061517466 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6e00072d0 con 0x7fd6ec0ccc50 2023-05-20T01:19:34.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.727+0000 7fd6f1d53700 1 -- 172.21.15.16:0/2061517466 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6ec15f4e0 con 0x7fd6ec0ccc50 2023-05-20T01:19:34.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.728+0000 7fd6f1d53700 1 -- 172.21.15.16:0/2061517466 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6ec0af8a0 con 0x7fd6ec0ccc50 2023-05-20T01:19:34.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.728+0000 7fd6d3fff700 1 -- 172.21.15.16:0/2061517466 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd6e0007430 con 0x7fd6ec0ccc50 2023-05-20T01:19:34.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.729+0000 7fd6d3fff700 1 --2- 172.21.15.16:0/2061517466 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd6d806ace0 0x7fd6d806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:34.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.729+0000 7fd6d3fff700 1 -- 172.21.15.16:0/2061517466 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 7102+0+0 (secure 0 0 0) 0x7fd6e0089600 con 0x7fd6ec0ccc50 2023-05-20T01:19:34.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.729+0000 7fd6eb7fe700 1 --2- 172.21.15.16:0/2061517466 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd6d806ace0 0x7fd6d806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:34.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.729+0000 7fd6eb7fe700 1 --2- 172.21.15.16:0/2061517466 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd6d806ace0 0x7fd6d806d190 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fd6dc0090b0 tx=0x7fd6dc008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:34.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.733+0000 7fd6d3fff700 1 -- 172.21.15.16:0/2061517466 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fd6e00542a0 con 0x7fd6ec0ccc50 2023-05-20T01:19:34.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.931+0000 7fd6f1d53700 1 -- 172.21.15.16:0/2061517466 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fd6ec0747b0 con 0x7fd6ec0ccc50 2023-05-20T01:19:34.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.931+0000 7fd6d3fff700 1 -- 172.21.15.16:0/2061517466 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v51) v1 ==== 92+0+10 (secure 0 0 0) 0x7fd6e00578c0 con 0x7fd6ec0ccc50 2023-05-20T01:19:34.935 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 8 2023-05-20T01:19:34.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.933+0000 7fd6f1d53700 1 -- 172.21.15.16:0/2061517466 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd6d806ace0 msgr2=0x7fd6d806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:34.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.933+0000 7fd6f1d53700 1 --2- 172.21.15.16:0/2061517466 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd6d806ace0 0x7fd6d806d190 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fd6dc0090b0 tx=0x7fd6dc008030 comp rx=0 tx=0).stop 2023-05-20T01:19:34.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.933+0000 7fd6f1d53700 1 -- 172.21.15.16:0/2061517466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6ec0ccc50 msgr2=0x7fd6ec0c98c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:34.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.933+0000 7fd6f1d53700 1 --2- 172.21.15.16:0/2061517466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6ec0ccc50 0x7fd6ec0c98c0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fd6e000c520 tx=0x7fd6e000c8e0 comp rx=0 tx=0).stop 2023-05-20T01:19:34.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.933+0000 7fd6f1d53700 1 -- 172.21.15.16:0/2061517466 shutdown_connections 2023-05-20T01:19:34.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.933+0000 7fd6f1d53700 1 --2- 172.21.15.16:0/2061517466 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd6d806ace0 0x7fd6d806d190 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:34.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.933+0000 7fd6f1d53700 1 --2- 172.21.15.16:0/2061517466 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd6ec15f000 0x7fd6ec15cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:34.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.933+0000 7fd6f1d53700 1 --2- 172.21.15.16:0/2061517466 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd6ec0b57b0 0x7fd6ec1483f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:34.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.933+0000 7fd6f1d53700 1 --2- 172.21.15.16:0/2061517466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6ec0ccc50 0x7fd6ec0c98c0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:34.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.933+0000 7fd6f1d53700 1 -- 172.21.15.16:0/2061517466 >> 172.21.15.16:0/2061517466 conn(0x7fd6ec06c260 msgr2=0x7fd6ec071c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:19:34.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.933+0000 7fd6f1d53700 1 -- 172.21.15.16:0/2061517466 shutdown_connections 2023-05-20T01:19:34.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:34.933+0000 7fd6f1d53700 1 -- 172.21.15.16:0/2061517466 wait complete. 2023-05-20T01:19:34.949 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-05-20T01:19:35.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.073+0000 7f548f112700 1 Processor -- start 2023-05-20T01:19:35.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.073+0000 7f548f112700 1 -- start start 2023-05-20T01:19:35.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.074+0000 7f548f112700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f548815cec0 0x7f54880cd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:35.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.074+0000 7f548f112700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f548806e190 0x7f54880cf1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:35.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.074+0000 7f548f112700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f548806dd00 con 0x7f548806a510 2023-05-20T01:19:35.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.074+0000 7f548f112700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54880cfa60 con 0x7f548806e190 2023-05-20T01:19:35.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.074+0000 7f548f112700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f548806d6b0 con 0x7f548815cec0 2023-05-20T01:19:35.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.074+0000 7f548d6af700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f548806a510 0x7f54880710d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:44826/0 (socket says 172.21.15.16:44826) 2023-05-20T01:19:35.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.074+0000 7f548d6af700 1 -- 172.21.15.16:0/864008859 learned_addr learned my addr 172.21.15.16:0/864008859 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:19:35.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.074+0000 7f548ceae700 1 --2- 172.21.15.16:0/864008859 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f548815cec0 0x7f54880cd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:35.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.074+0000 7f5487fff700 1 --2- 172.21.15.16:0/864008859 >> v2:172.21.15.132:3300/0 conn(0x7f548806e190 0x7f54880cf1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:35.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.074+0000 7f54877fe700 1 -- 172.21.15.16:0/864008859 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3861534954 0 0) 0x7f548806dd00 con 0x7f548806a510 2023-05-20T01:19:35.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.074+0000 7f54877fe700 1 -- 172.21.15.16:0/864008859 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f54780035b0 con 0x7f548806a510 2023-05-20T01:19:35.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.075+0000 7f54877fe700 1 -- 172.21.15.16:0/864008859 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2528053005 0 0) 0x7f54780035b0 con 0x7f548806a510 2023-05-20T01:19:35.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.075+0000 7f54877fe700 1 -- 172.21.15.16:0/864008859 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f548815cec0 msgr2=0x7f54880cd4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:19:35.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.075+0000 7f54877fe700 1 --2- 172.21.15.16:0/864008859 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f548815cec0 0x7f54880cd4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:35.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.075+0000 7f54877fe700 1 -- 172.21.15.16:0/864008859 >> v2:172.21.15.132:3300/0 conn(0x7f548806e190 msgr2=0x7f54880cf1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:19:35.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.075+0000 7f54877fe700 1 --2- 172.21.15.16:0/864008859 >> v2:172.21.15.132:3300/0 conn(0x7f548806e190 0x7f54880cf1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:35.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.075+0000 7f54877fe700 1 -- 172.21.15.16:0/864008859 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f548806d990 con 0x7f548806a510 2023-05-20T01:19:35.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.075+0000 7f54877fe700 1 -- 172.21.15.16:0/864008859 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7f54800019f0 con 0x7f548806a510 2023-05-20T01:19:35.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.075+0000 7f54877fe700 1 -- 172.21.15.16:0/864008859 >> v1:172.21.15.16:6789/0 conn(0x7f548806a510 legacy=0x7f54880710d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:35.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.075+0000 7f54877fe700 1 --2- 172.21.15.16:0/864008859 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5478003a00 0x7f5478003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:35.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.075+0000 7f54877fe700 1 --2- 172.21.15.16:0/864008859 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f548815cec0 0x7f5478004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:35.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.075+0000 7f54877fe700 1 --2- 172.21.15.16:0/864008859 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5478004d90 0x7f5478005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:35.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.075+0000 7f54877fe700 1 -- 172.21.15.16:0/864008859 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54880cfa60 con 0x7f5478004d90 2023-05-20T01:19:35.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.075+0000 7f54877fe700 1 -- 172.21.15.16:0/864008859 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f548806d6b0 con 0x7f5478003a00 2023-05-20T01:19:35.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.075+0000 7f54877fe700 1 -- 172.21.15.16:0/864008859 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f547800ba10 con 0x7f548815cec0 2023-05-20T01:19:35.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.075+0000 7f548d6af700 1 --2- 172.21.15.16:0/864008859 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5478004d90 0x7f5478005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:35.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.075+0000 7f548ceae700 1 --2- 172.21.15.16:0/864008859 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5478003a00 0x7f5478003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:35.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.075+0000 7f5487fff700 1 --2- 172.21.15.16:0/864008859 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f548815cec0 0x7f5478004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:35.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.075+0000 7f548d6af700 1 -- 172.21.15.16:0/864008859 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f548815cec0 msgr2=0x7f5478004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:19:35.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.075+0000 7f548d6af700 1 --2- 172.21.15.16:0/864008859 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f548815cec0 0x7f5478004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:35.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.075+0000 7f548d6af700 1 -- 172.21.15.16:0/864008859 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5478003a00 msgr2=0x7f5478003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:19:35.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.075+0000 7f548d6af700 1 --2- 172.21.15.16:0/864008859 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5478003a00 0x7f5478003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:35.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.075+0000 7f548d6af700 1 -- 172.21.15.16:0/864008859 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54780035b0 con 0x7f5478004d90 2023-05-20T01:19:35.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.075+0000 7f548d6af700 1 --2- 172.21.15.16:0/864008859 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5478004d90 0x7f5478005200 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f548000a8d0 tx=0x7f5478005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=710404f539b489fd server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:35.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.076+0000 7f54877fe700 1 -- 172.21.15.16:0/864008859 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f548000a340 con 0x7f5478004d90 2023-05-20T01:19:35.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.076+0000 7f54877fe700 1 -- 172.21.15.16:0/864008859 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5480012600 con 0x7f5478004d90 2023-05-20T01:19:35.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.076+0000 7f54877fe700 1 -- 172.21.15.16:0/864008859 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54800129b0 con 0x7f5478004d90 2023-05-20T01:19:35.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.076+0000 7f548f112700 1 -- 172.21.15.16:0/864008859 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5478004d90 msgr2=0x7f5478005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:35.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.076+0000 7f548f112700 1 --2- 172.21.15.16:0/864008859 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5478004d90 0x7f5478005200 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f548000a8d0 tx=0x7f5478005880 comp rx=0 tx=0).stop 2023-05-20T01:19:35.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.076+0000 7f548d6af700 1 -- 172.21.15.16:0/864008859 reap_dead start 2023-05-20T01:19:35.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.076+0000 7f548f112700 1 -- 172.21.15.16:0/864008859 shutdown_connections 2023-05-20T01:19:35.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.076+0000 7f548f112700 1 -- 172.21.15.16:0/864008859 >> 172.21.15.16:0/864008859 conn(0x7f548806c260 msgr2=0x7f54880cffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:19:35.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.076+0000 7f548f112700 1 -- 172.21.15.16:0/864008859 shutdown_connections 2023-05-20T01:19:35.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.076+0000 7f548f112700 1 -- 172.21.15.16:0/864008859 wait complete. 2023-05-20T01:19:35.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.077+0000 7f548f112700 1 Processor -- start 2023-05-20T01:19:35.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.077+0000 7f548f112700 1 -- start start 2023-05-20T01:19:35.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.078+0000 7f548f112700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54881447d0 0x7f5488144f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:35.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.078+0000 7f548f112700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f54880b5100 0x7f54880b5570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:35.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.078+0000 7f548f112700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5488146fb0 0x7f5488147450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:35.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.078+0000 7f548f112700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5488147fa0 con 0x7f54881447d0 2023-05-20T01:19:35.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.078+0000 7f548f112700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54880b5ab0 con 0x7f54880b5100 2023-05-20T01:19:35.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.078+0000 7f548f112700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f548815eb00 con 0x7f5488146fb0 2023-05-20T01:19:35.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.078+0000 7f548ceae700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54881447d0 0x7f5488144f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:35.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.078+0000 7f548ceae700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54881447d0 0x7f5488144f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33306/0 (socket says 172.21.15.16:33306) 2023-05-20T01:19:35.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.078+0000 7f548ceae700 1 -- 172.21.15.16:0/4258666995 learned_addr learned my addr 172.21.15.16:0/4258666995 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:19:35.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.079+0000 7f548d6af700 1 --2- 172.21.15.16:0/4258666995 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5488146fb0 0x7f5488147450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:35.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.079+0000 7f5487fff700 1 --2- 172.21.15.16:0/4258666995 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f54880b5100 0x7f54880b5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:35.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.079+0000 7f548ceae700 1 -- 172.21.15.16:0/4258666995 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5488146fb0 msgr2=0x7f5488147450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:35.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.079+0000 7f548ceae700 1 --2- 172.21.15.16:0/4258666995 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5488146fb0 0x7f5488147450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:35.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.079+0000 7f548ceae700 1 -- 172.21.15.16:0/4258666995 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f54880b5100 msgr2=0x7f54880b5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:35.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.079+0000 7f548ceae700 1 --2- 172.21.15.16:0/4258666995 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f54880b5100 0x7f54880b5570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:35.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.079+0000 7f548ceae700 1 -- 172.21.15.16:0/4258666995 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54880cfd40 con 0x7f54881447d0 2023-05-20T01:19:35.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.079+0000 7f548d6af700 1 --2- 172.21.15.16:0/4258666995 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5488146fb0 0x7f5488147450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:19:35.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.079+0000 7f5487fff700 1 --2- 172.21.15.16:0/4258666995 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f54880b5100 0x7f54880b5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:19:35.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.079+0000 7f548ceae700 1 --2- 172.21.15.16:0/4258666995 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54881447d0 0x7f5488144f20 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f547c011ad0 tx=0x7f547c011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:35.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.079+0000 7f54857fa700 1 -- 172.21.15.16:0/4258666995 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f547c00f9a0 con 0x7f54881447d0 2023-05-20T01:19:35.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.079+0000 7f54857fa700 1 -- 172.21.15.16:0/4258666995 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f547c00fb00 con 0x7f54881447d0 2023-05-20T01:19:35.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.079+0000 7f548f112700 1 -- 172.21.15.16:0/4258666995 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f548815d290 con 0x7f54881447d0 2023-05-20T01:19:35.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.079+0000 7f54857fa700 1 -- 172.21.15.16:0/4258666995 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f547c00feb0 con 0x7f54881447d0 2023-05-20T01:19:35.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.079+0000 7f548f112700 1 -- 172.21.15.16:0/4258666995 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54880cf730 con 0x7f54881447d0 2023-05-20T01:19:35.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.080+0000 7f548f112700 1 -- 172.21.15.16:0/4258666995 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54880747b0 con 0x7f54881447d0 2023-05-20T01:19:35.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.081+0000 7f54857fa700 1 -- 172.21.15.16:0/4258666995 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f547c014440 con 0x7f54881447d0 2023-05-20T01:19:35.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.081+0000 7f54857fa700 1 --2- 172.21.15.16:0/4258666995 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f546c06ace0 0x7f546c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:35.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.081+0000 7f54857fa700 1 -- 172.21.15.16:0/4258666995 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 7102+0+0 (secure 0 0 0) 0x7f547c089ff0 con 0x7f54881447d0 2023-05-20T01:19:35.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.081+0000 7f5487fff700 1 --2- 172.21.15.16:0/4258666995 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f546c06ace0 0x7f546c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:35.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.081+0000 7f5487fff700 1 --2- 172.21.15.16:0/4258666995 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f546c06ace0 0x7f546c06d190 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f548806f130 tx=0x7f5474008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:35.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.086+0000 7f54857fa700 1 -- 172.21.15.16:0/4258666995 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f547c0587e0 con 0x7f54881447d0 2023-05-20T01:19:35.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.280+0000 7f548f112700 1 -- 172.21.15.16:0/4258666995 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f54880cbca0 con 0x7f54881447d0 2023-05-20T01:19:35.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.280+0000 7f54857fa700 1 -- 172.21.15.16:0/4258666995 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v51) v1 ==== 93+0+10 (secure 0 0 0) 0x7f547c058370 con 0x7f54881447d0 2023-05-20T01:19:35.284 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-05-20T01:19:35.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.282+0000 7f548f112700 1 -- 172.21.15.16:0/4258666995 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f546c06ace0 msgr2=0x7f546c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:35.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.282+0000 7f548f112700 1 --2- 172.21.15.16:0/4258666995 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f546c06ace0 0x7f546c06d190 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f548806f130 tx=0x7f5474008040 comp rx=0 tx=0).stop 2023-05-20T01:19:35.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.282+0000 7f548f112700 1 -- 172.21.15.16:0/4258666995 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54881447d0 msgr2=0x7f5488144f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:35.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.282+0000 7f548f112700 1 --2- 172.21.15.16:0/4258666995 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54881447d0 0x7f5488144f20 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f547c011ad0 tx=0x7f547c011e90 comp rx=0 tx=0).stop 2023-05-20T01:19:35.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.282+0000 7f548f112700 1 -- 172.21.15.16:0/4258666995 shutdown_connections 2023-05-20T01:19:35.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.282+0000 7f548f112700 1 --2- 172.21.15.16:0/4258666995 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f546c06ace0 0x7f546c06d190 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:35.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.282+0000 7f548f112700 1 --2- 172.21.15.16:0/4258666995 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5488146fb0 0x7f5488147450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:35.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.282+0000 7f548f112700 1 --2- 172.21.15.16:0/4258666995 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54881447d0 0x7f5488144f20 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:35.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.282+0000 7f548f112700 1 --2- 172.21.15.16:0/4258666995 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f54880b5100 0x7f54880b5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:35.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.282+0000 7f548f112700 1 -- 172.21.15.16:0/4258666995 >> 172.21.15.16:0/4258666995 conn(0x7f548806c260 msgr2=0x7f54880cd100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:19:35.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.282+0000 7f548f112700 1 -- 172.21.15.16:0/4258666995 shutdown_connections 2023-05-20T01:19:35.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.282+0000 7f548f112700 1 -- 172.21.15.16:0/4258666995 wait complete. 2023-05-20T01:19:35.298 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-05-20T01:19:35.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.424+0000 7f945add7700 1 Processor -- start 2023-05-20T01:19:35.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.424+0000 7f945add7700 1 -- start start 2023-05-20T01:19:35.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.425+0000 7f945add7700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f94540af2f0 0x7f94540cec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:35.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.425+0000 7f945add7700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f94540cf180 0x7f94540b09a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:35.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.425+0000 7f945add7700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94540b2020 con 0x7f94540d16e0 2023-05-20T01:19:35.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.425+0000 7f945add7700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94540cf600 con 0x7f94540af2f0 2023-05-20T01:19:35.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.425+0000 7f945add7700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94540b2300 con 0x7f94540cf180 2023-05-20T01:19:35.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.425+0000 7f9453fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f94540af2f0 0x7f94540cec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:35.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.425+0000 7f9453fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f94540af2f0 0x7f94540cec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33316/0 (socket says 172.21.15.16:33316) 2023-05-20T01:19:35.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.425+0000 7f9453fff700 1 -- 172.21.15.16:0/1081835389 learned_addr learned my addr 172.21.15.16:0/1081835389 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:19:35.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.425+0000 7f9459374700 1 --2- 172.21.15.16:0/1081835389 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f94540cf180 0x7f94540b09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:35.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.425+0000 7f9453fff700 1 -- 172.21.15.16:0/1081835389 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f94540cf180 msgr2=0x7f94540b09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:19:35.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.425+0000 7f9453fff700 1 --2- 172.21.15.16:0/1081835389 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f94540cf180 0x7f94540b09a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:35.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.425+0000 7f9453fff700 1 -- 172.21.15.16:0/1081835389 >> v1:172.21.15.132:6789/0 conn(0x7f94540d16e0 legacy=0x7f9454144350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:35.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.425+0000 7f9453fff700 1 -- 172.21.15.16:0/1081835389 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94540b25e0 con 0x7f94540af2f0 2023-05-20T01:19:35.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.426+0000 7f9453fff700 1 --2- 172.21.15.16:0/1081835389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94540af2f0 0x7f94540cec40 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f9440009920 tx=0x7f9440009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e30778b3f2f07965 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:35.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.426+0000 7f94537fe700 1 -- 172.21.15.16:0/1081835389 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9440011830 con 0x7f94540af2f0 2023-05-20T01:19:35.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.426+0000 7f94537fe700 1 -- 172.21.15.16:0/1081835389 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9440011990 con 0x7f94540af2f0 2023-05-20T01:19:35.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.426+0000 7f94537fe700 1 -- 172.21.15.16:0/1081835389 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9440011d20 con 0x7f94540af2f0 2023-05-20T01:19:35.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.426+0000 7f945add7700 1 -- 172.21.15.16:0/1081835389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94540af2f0 msgr2=0x7f94540cec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:35.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.426+0000 7f945add7700 1 --2- 172.21.15.16:0/1081835389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94540af2f0 0x7f94540cec40 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f9440009920 tx=0x7f9440009c30 comp rx=0 tx=0).stop 2023-05-20T01:19:35.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.426+0000 7f945add7700 1 -- 172.21.15.16:0/1081835389 shutdown_connections 2023-05-20T01:19:35.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.426+0000 7f945add7700 1 --2- 172.21.15.16:0/1081835389 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f94540cf180 0x7f94540b09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:35.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.426+0000 7f945add7700 1 --2- 172.21.15.16:0/1081835389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94540af2f0 0x7f94540cec40 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:35.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.426+0000 7f945add7700 1 -- 172.21.15.16:0/1081835389 >> 172.21.15.16:0/1081835389 conn(0x7f945406ae30 msgr2=0x7f945409a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:19:35.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.426+0000 7f945add7700 1 -- 172.21.15.16:0/1081835389 shutdown_connections 2023-05-20T01:19:35.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.426+0000 7f945add7700 1 -- 172.21.15.16:0/1081835389 wait complete. 2023-05-20T01:19:35.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.428+0000 7f945add7700 1 Processor -- start 2023-05-20T01:19:35.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.428+0000 7f945add7700 1 -- start start 2023-05-20T01:19:35.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.429+0000 7f945add7700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f94540af2f0 0x7f94540b6570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:35.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.429+0000 7f945add7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94540cf180 0x7f94540c9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:35.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.429+0000 7f945add7700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f94540d16e0 0x7f94540ca3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:35.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.429+0000 7f945add7700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94540b4040 con 0x7f94540cf180 2023-05-20T01:19:35.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.429+0000 7f945add7700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94540ca920 con 0x7f94540d16e0 2023-05-20T01:19:35.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.429+0000 7f945add7700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94540c8b50 con 0x7f94540af2f0 2023-05-20T01:19:35.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.429+0000 7f9453fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94540cf180 0x7f94540c9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:35.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.429+0000 7f9453fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94540cf180 0x7f94540c9ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33322/0 (socket says 172.21.15.16:33322) 2023-05-20T01:19:35.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.429+0000 7f9453fff700 1 -- 172.21.15.16:0/2586809565 learned_addr learned my addr 172.21.15.16:0/2586809565 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:19:35.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.429+0000 7f9458b73700 1 --2- 172.21.15.16:0/2586809565 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f94540af2f0 0x7f94540b6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:35.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.429+0000 7f9459374700 1 --2- 172.21.15.16:0/2586809565 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f94540d16e0 0x7f94540ca3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:35.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.429+0000 7f9453fff700 1 -- 172.21.15.16:0/2586809565 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f94540af2f0 msgr2=0x7f94540b6570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:35.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.429+0000 7f9453fff700 1 --2- 172.21.15.16:0/2586809565 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f94540af2f0 0x7f94540b6570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:35.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.429+0000 7f9453fff700 1 -- 172.21.15.16:0/2586809565 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f94540d16e0 msgr2=0x7f94540ca3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:35.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.429+0000 7f9453fff700 1 --2- 172.21.15.16:0/2586809565 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f94540d16e0 0x7f94540ca3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:35.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.429+0000 7f9453fff700 1 -- 172.21.15.16:0/2586809565 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9440009550 con 0x7f94540cf180 2023-05-20T01:19:35.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.429+0000 7f9459374700 1 --2- 172.21.15.16:0/2586809565 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f94540d16e0 0x7f94540ca3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:19:35.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.429+0000 7f9453fff700 1 --2- 172.21.15.16:0/2586809565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94540cf180 0x7f94540c9ea0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f94400098f0 tx=0x7f9440004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:35.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.429+0000 7f94517fa700 1 -- 172.21.15.16:0/2586809565 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9440004300 con 0x7f94540cf180 2023-05-20T01:19:35.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.429+0000 7f94517fa700 1 -- 172.21.15.16:0/2586809565 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f944001f070 con 0x7f94540cf180 2023-05-20T01:19:35.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.429+0000 7f94517fa700 1 -- 172.21.15.16:0/2586809565 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94400036a0 con 0x7f94540cf180 2023-05-20T01:19:35.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.429+0000 7f945add7700 1 -- 172.21.15.16:0/2586809565 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94540b28c0 con 0x7f94540cf180 2023-05-20T01:19:35.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.429+0000 7f945add7700 1 -- 172.21.15.16:0/2586809565 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94540addc0 con 0x7f94540cf180 2023-05-20T01:19:35.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.430+0000 7f945add7700 1 -- 172.21.15.16:0/2586809565 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94540b0ee0 con 0x7f94540cf180 2023-05-20T01:19:35.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.431+0000 7f94517fa700 1 -- 172.21.15.16:0/2586809565 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9440021030 con 0x7f94540cf180 2023-05-20T01:19:35.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.431+0000 7f94517fa700 1 --2- 172.21.15.16:0/2586809565 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f943806ace0 0x7f943806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:35.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.431+0000 7f9458b73700 1 --2- 172.21.15.16:0/2586809565 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f943806ace0 0x7f943806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:35.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.431+0000 7f94517fa700 1 -- 172.21.15.16:0/2586809565 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 7102+0+0 (secure 0 0 0) 0x7f944008ab60 con 0x7f94540cf180 2023-05-20T01:19:35.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.432+0000 7f9458b73700 1 --2- 172.21.15.16:0/2586809565 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f943806ace0 0x7f943806d190 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f9448007fd0 tx=0x7f9448009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:35.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.436+0000 7f94517fa700 1 -- 172.21.15.16:0/2586809565 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f9440055890 con 0x7f94540cf180 2023-05-20T01:19:35.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.634+0000 7f945add7700 1 -- 172.21.15.16:0/2586809565 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f94540b4180 con 0x7f94540cf180 2023-05-20T01:19:35.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.634+0000 7f94517fa700 1 -- 172.21.15.16:0/2586809565 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v51) v1 ==== 104+0+11 (secure 0 0 0) 0x7f94540b4180 con 0x7f94540cf180 2023-05-20T01:19:35.639 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-05-20T01:19:35.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.636+0000 7f945add7700 1 -- 172.21.15.16:0/2586809565 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f943806ace0 msgr2=0x7f943806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:35.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.636+0000 7f945add7700 1 --2- 172.21.15.16:0/2586809565 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f943806ace0 0x7f943806d190 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f9448007fd0 tx=0x7f9448009040 comp rx=0 tx=0).stop 2023-05-20T01:19:35.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.636+0000 7f945add7700 1 -- 172.21.15.16:0/2586809565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94540cf180 msgr2=0x7f94540c9ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:35.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.636+0000 7f945add7700 1 --2- 172.21.15.16:0/2586809565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94540cf180 0x7f94540c9ea0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f94400098f0 tx=0x7f9440004750 comp rx=0 tx=0).stop 2023-05-20T01:19:35.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.636+0000 7f945add7700 1 -- 172.21.15.16:0/2586809565 shutdown_connections 2023-05-20T01:19:35.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.636+0000 7f945add7700 1 --2- 172.21.15.16:0/2586809565 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f94540d16e0 0x7f94540ca3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:35.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.636+0000 7f945add7700 1 --2- 172.21.15.16:0/2586809565 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f943806ace0 0x7f943806d190 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:35.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.636+0000 7f945add7700 1 --2- 172.21.15.16:0/2586809565 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f94540af2f0 0x7f94540b6570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:35.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.636+0000 7f945add7700 1 --2- 172.21.15.16:0/2586809565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94540cf180 0x7f94540c9ea0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:35.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.636+0000 7f945add7700 1 -- 172.21.15.16:0/2586809565 >> 172.21.15.16:0/2586809565 conn(0x7f945406ae30 msgr2=0x7f945406e3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:19:35.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.636+0000 7f945add7700 1 -- 172.21.15.16:0/2586809565 shutdown_connections 2023-05-20T01:19:35.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.636+0000 7f945add7700 1 -- 172.21.15.16:0/2586809565 wait complete. 2023-05-20T01:19:35.652 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-05-20T01:19:35.696 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:19:35.699 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:19:35.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.778+0000 7fba4dd37700 1 Processor -- start 2023-05-20T01:19:35.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.778+0000 7fba4dd37700 1 -- start start 2023-05-20T01:19:35.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.779+0000 7fba4dd37700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fba4815ab80 0x7fba480cda80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:35.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.779+0000 7fba4dd37700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fba480cf390 0x7fba480cc110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:35.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.779+0000 7fba4dd37700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba480ce4d0 con 0x7fba480cdfc0 2023-05-20T01:19:35.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.779+0000 7fba4dd37700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba480cb700 con 0x7fba480cf390 2023-05-20T01:19:35.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.779+0000 7fba4dd37700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba480ce7b0 con 0x7fba4815ab80 2023-05-20T01:19:35.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.779+0000 7fba477fe700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fba4815ab80 0x7fba480cda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:35.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.779+0000 7fba47fff700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fba480cf390 0x7fba480cc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:35.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.779+0000 7fba47fff700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fba480cf390 0x7fba480cc110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.16:58586/0 (socket says 172.21.15.16:58586) 2023-05-20T01:19:35.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.779+0000 7fba47fff700 1 -- 172.21.15.16:0/3714196753 learned_addr learned my addr 172.21.15.16:0/3714196753 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:19:35.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.779+0000 7fba46ffd700 1 --1- 172.21.15.16:0/3714196753 >> v1:172.21.15.132:6789/0 conn(0x7fba480cdfc0 0x7fba480cec80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.16:54570/0 (socket says 172.21.15.16:54570) 2023-05-20T01:19:35.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.780+0000 7fba47fff700 1 -- 172.21.15.16:0/3714196753 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fba4815ab80 msgr2=0x7fba480cda80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:19:35.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.780+0000 7fba47fff700 1 --2- 172.21.15.16:0/3714196753 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fba4815ab80 0x7fba480cda80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:35.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.780+0000 7fba47fff700 1 -- 172.21.15.16:0/3714196753 >> v1:172.21.15.132:6789/0 conn(0x7fba480cdfc0 legacy=0x7fba480cec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:35.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.780+0000 7fba47fff700 1 -- 172.21.15.16:0/3714196753 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba480cb910 con 0x7fba480cf390 2023-05-20T01:19:35.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.780+0000 7fba477fe700 1 --2- 172.21.15.16:0/3714196753 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fba4815ab80 0x7fba480cda80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-05-20T01:19:35.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.780+0000 7fba47fff700 1 --2- 172.21.15.16:0/3714196753 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fba480cf390 0x7fba480cc110 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fba40009e50 tx=0x7fba4000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e666502ef4cce1 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:35.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.780+0000 7fba467fc700 1 -- 172.21.15.16:0/3714196753 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba4000e070 con 0x7fba480cf390 2023-05-20T01:19:35.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.780+0000 7fba467fc700 1 -- 172.21.15.16:0/3714196753 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba4000b910 con 0x7fba480cf390 2023-05-20T01:19:35.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.780+0000 7fba467fc700 1 -- 172.21.15.16:0/3714196753 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba4000bcc0 con 0x7fba480cf390 2023-05-20T01:19:35.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.780+0000 7fba4dd37700 1 -- 172.21.15.16:0/3714196753 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fba480cf390 msgr2=0x7fba480cc110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:35.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.780+0000 7fba4dd37700 1 --2- 172.21.15.16:0/3714196753 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fba480cf390 0x7fba480cc110 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fba40009e50 tx=0x7fba4000b330 comp rx=0 tx=0).stop 2023-05-20T01:19:35.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.781+0000 7fba4dd37700 1 -- 172.21.15.16:0/3714196753 shutdown_connections 2023-05-20T01:19:35.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.781+0000 7fba4dd37700 1 --2- 172.21.15.16:0/3714196753 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fba4815ab80 0x7fba480cda80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:35.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.781+0000 7fba4dd37700 1 --2- 172.21.15.16:0/3714196753 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fba480cf390 0x7fba480cc110 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:35.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.781+0000 7fba4dd37700 1 -- 172.21.15.16:0/3714196753 >> 172.21.15.16:0/3714196753 conn(0x7fba4806ae30 msgr2=0x7fba4809a400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:19:35.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.781+0000 7fba4dd37700 1 -- 172.21.15.16:0/3714196753 shutdown_connections 2023-05-20T01:19:35.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.781+0000 7fba4dd37700 1 -- 172.21.15.16:0/3714196753 wait complete. 2023-05-20T01:19:35.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.782+0000 7fba4dd37700 1 Processor -- start 2023-05-20T01:19:35.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.782+0000 7fba4dd37700 1 -- start start 2023-05-20T01:19:35.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.783+0000 7fba4dd37700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba480cdfc0 0x7fba48155d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:35.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.783+0000 7fba4dd37700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fba480cf390 0x7fba48145d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:35.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.783+0000 7fba4dd37700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fba4815ab80 0x7fba48143320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:35.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.783+0000 7fba4dd37700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba480c5320 con 0x7fba480cdfc0 2023-05-20T01:19:35.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.783+0000 7fba4dd37700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba48156250 con 0x7fba480cf390 2023-05-20T01:19:35.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.783+0000 7fba4dd37700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba480b5840 con 0x7fba4815ab80 2023-05-20T01:19:35.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.783+0000 7fba477fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba480cdfc0 0x7fba48155d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:35.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.784+0000 7fba477fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba480cdfc0 0x7fba48155d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33334/0 (socket says 172.21.15.16:33334) 2023-05-20T01:19:35.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.784+0000 7fba477fe700 1 -- 172.21.15.16:0/4250933743 learned_addr learned my addr 172.21.15.16:0/4250933743 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:19:35.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.784+0000 7fba47fff700 1 --2- 172.21.15.16:0/4250933743 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fba4815ab80 0x7fba48143320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:35.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.784+0000 7fba46ffd700 1 --2- 172.21.15.16:0/4250933743 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fba480cf390 0x7fba48145d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:35.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.784+0000 7fba477fe700 1 -- 172.21.15.16:0/4250933743 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fba4815ab80 msgr2=0x7fba48143320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:35.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.784+0000 7fba477fe700 1 --2- 172.21.15.16:0/4250933743 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fba4815ab80 0x7fba48143320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:35.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.784+0000 7fba477fe700 1 -- 172.21.15.16:0/4250933743 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fba480cf390 msgr2=0x7fba48145d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:35.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.784+0000 7fba477fe700 1 --2- 172.21.15.16:0/4250933743 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fba480cf390 0x7fba48145d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:35.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.784+0000 7fba477fe700 1 -- 172.21.15.16:0/4250933743 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba300094a0 con 0x7fba480cdfc0 2023-05-20T01:19:35.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.784+0000 7fba46ffd700 1 --2- 172.21.15.16:0/4250933743 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fba480cf390 0x7fba48145d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:19:35.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.784+0000 7fba477fe700 1 --2- 172.21.15.16:0/4250933743 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba480cdfc0 0x7fba48155d10 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fba3000ebe0 tx=0x7fba3000eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:35.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.784+0000 7fba3ffff700 1 -- 172.21.15.16:0/4250933743 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba3000c850 con 0x7fba480cdfc0 2023-05-20T01:19:35.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.784+0000 7fba3ffff700 1 -- 172.21.15.16:0/4250933743 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba3000c9b0 con 0x7fba480cdfc0 2023-05-20T01:19:35.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.784+0000 7fba4dd37700 1 -- 172.21.15.16:0/4250933743 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba40009b30 con 0x7fba480cdfc0 2023-05-20T01:19:35.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.784+0000 7fba3ffff700 1 -- 172.21.15.16:0/4250933743 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba3000cd40 con 0x7fba480cdfc0 2023-05-20T01:19:35.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.784+0000 7fba4dd37700 1 -- 172.21.15.16:0/4250933743 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba480d0e30 con 0x7fba480cdfc0 2023-05-20T01:19:35.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.785+0000 7fba4dd37700 1 -- 172.21.15.16:0/4250933743 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba480cbbf0 con 0x7fba480cdfc0 2023-05-20T01:19:35.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.786+0000 7fba3ffff700 1 -- 172.21.15.16:0/4250933743 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fba30013470 con 0x7fba480cdfc0 2023-05-20T01:19:35.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.786+0000 7fba3ffff700 1 --2- 172.21.15.16:0/4250933743 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fba2806ace0 0x7fba2806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:35.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.786+0000 7fba3ffff700 1 -- 172.21.15.16:0/4250933743 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 7102+0+0 (secure 0 0 0) 0x7fba300898e0 con 0x7fba480cdfc0 2023-05-20T01:19:35.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.786+0000 7fba46ffd700 1 --2- 172.21.15.16:0/4250933743 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fba2806ace0 0x7fba2806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:35.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.787+0000 7fba46ffd700 1 --2- 172.21.15.16:0/4250933743 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fba2806ace0 0x7fba2806d190 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fba38000f60 tx=0x7fba38007dd0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:35.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.791+0000 7fba3ffff700 1 -- 172.21.15.16:0/4250933743 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fba30054580 con 0x7fba480cdfc0 2023-05-20T01:19:35.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.987+0000 7fba4dd37700 1 -- 172.21.15.16:0/4250933743 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fba480b6130 con 0x7fba480cdfc0 2023-05-20T01:19:35.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.987+0000 7fba3ffff700 1 -- 172.21.15.16:0/4250933743 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v51) v1 ==== 100+0+11 (secure 0 0 0) 0x7fba3000cea0 con 0x7fba480cdfc0 2023-05-20T01:19:35.991 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2023-05-20T01:19:35.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.989+0000 7fba4dd37700 1 -- 172.21.15.16:0/4250933743 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fba2806ace0 msgr2=0x7fba2806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:35.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.989+0000 7fba4dd37700 1 --2- 172.21.15.16:0/4250933743 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fba2806ace0 0x7fba2806d190 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fba38000f60 tx=0x7fba38007dd0 comp rx=0 tx=0).stop 2023-05-20T01:19:35.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.989+0000 7fba4dd37700 1 -- 172.21.15.16:0/4250933743 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba480cdfc0 msgr2=0x7fba48155d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:35.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.989+0000 7fba4dd37700 1 --2- 172.21.15.16:0/4250933743 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba480cdfc0 0x7fba48155d10 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fba3000ebe0 tx=0x7fba3000eef0 comp rx=0 tx=0).stop 2023-05-20T01:19:35.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.989+0000 7fba4dd37700 1 -- 172.21.15.16:0/4250933743 shutdown_connections 2023-05-20T01:19:35.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.989+0000 7fba4dd37700 1 --2- 172.21.15.16:0/4250933743 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fba2806ace0 0x7fba2806d190 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:35.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.989+0000 7fba4dd37700 1 --2- 172.21.15.16:0/4250933743 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fba4815ab80 0x7fba48143320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:35.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.989+0000 7fba4dd37700 1 --2- 172.21.15.16:0/4250933743 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba480cdfc0 0x7fba48155d10 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:35.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.989+0000 7fba4dd37700 1 --2- 172.21.15.16:0/4250933743 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fba480cf390 0x7fba48145d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:35.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.989+0000 7fba4dd37700 1 -- 172.21.15.16:0/4250933743 >> 172.21.15.16:0/4250933743 conn(0x7fba4806ae30 msgr2=0x7fba480cc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:19:35.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.990+0000 7fba4dd37700 1 -- 172.21.15.16:0/4250933743 shutdown_connections 2023-05-20T01:19:35.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:35.990+0000 7fba4dd37700 1 -- 172.21.15.16:0/4250933743 wait complete. 2023-05-20T01:19:36.005 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_extra pg_num 2023-05-20T01:19:36.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.169+0000 7f7f39c72700 1 Processor -- start 2023-05-20T01:19:36.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.169+0000 7f7f39c72700 1 -- start start 2023-05-20T01:19:36.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.170+0000 7f7f39c72700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f7f34072300 0x7f7f340cf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:36.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.170+0000 7f7f39c72700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7f3415d290 0x7f7f340cf800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:36.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.170+0000 7f7f39c72700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f340cd630 con 0x7f7f3415cec0 2023-05-20T01:19:36.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.170+0000 7f7f39c72700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f340cca60 con 0x7f7f34072300 2023-05-20T01:19:36.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.170+0000 7f7f39c72700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f340ccd00 con 0x7f7f3415d290 2023-05-20T01:19:36.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.170+0000 7f7f32ffd700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f7f34072300 0x7f7f340cf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:36.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.170+0000 7f7f33fff700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7f3415d290 0x7f7f340cf800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:36.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.170+0000 7f7f32ffd700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f7f34072300 0x7f7f340cf2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.16:58598/0 (socket says 172.21.15.16:58598) 2023-05-20T01:19:36.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.170+0000 7f7f33fff700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7f3415d290 0x7f7f340cf800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.16:33384/0 (socket says 172.21.15.16:33384) 2023-05-20T01:19:36.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.170+0000 7f7f337fe700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f7f3415cec0 0x7f7f34071bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.16:54584/0 (socket says 172.21.15.16:54584) 2023-05-20T01:19:36.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.170+0000 7f7f32ffd700 1 -- 172.21.15.16:0/2470915953 learned_addr learned my addr 172.21.15.16:0/2470915953 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:19:36.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.171+0000 7f7f33fff700 1 -- 172.21.15.16:0/2470915953 >> v2:172.21.15.132:3300/0 conn(0x7f7f34072300 msgr2=0x7f7f340cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:19:36.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.171+0000 7f7f33fff700 1 --2- 172.21.15.16:0/2470915953 >> v2:172.21.15.132:3300/0 conn(0x7f7f34072300 0x7f7f340cf2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:36.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.171+0000 7f7f33fff700 1 -- 172.21.15.16:0/2470915953 >> v1:172.21.15.132:6789/0 conn(0x7f7f3415cec0 legacy=0x7f7f34071bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:36.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.171+0000 7f7f33fff700 1 -- 172.21.15.16:0/2470915953 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f340ccfe0 con 0x7f7f3415d290 2023-05-20T01:19:36.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.171+0000 7f7f33fff700 1 --2- 172.21.15.16:0/2470915953 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7f3415d290 0x7f7f340cf800 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7f2c009d90 tx=0x7f7f2c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d223b27502cac3b server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:36.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.171+0000 7f7f327fc700 1 -- 172.21.15.16:0/2470915953 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f2c00e070 con 0x7f7f3415d290 2023-05-20T01:19:36.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.171+0000 7f7f327fc700 1 -- 172.21.15.16:0/2470915953 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f2c00bb20 con 0x7f7f3415d290 2023-05-20T01:19:36.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.171+0000 7f7f327fc700 1 -- 172.21.15.16:0/2470915953 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f2c012430 con 0x7f7f3415d290 2023-05-20T01:19:36.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.171+0000 7f7f39c72700 1 -- 172.21.15.16:0/2470915953 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7f3415d290 msgr2=0x7f7f340cf800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:36.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.171+0000 7f7f39c72700 1 --2- 172.21.15.16:0/2470915953 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7f3415d290 0x7f7f340cf800 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7f2c009d90 tx=0x7f7f2c00b330 comp rx=0 tx=0).stop 2023-05-20T01:19:36.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.172+0000 7f7f39c72700 1 -- 172.21.15.16:0/2470915953 shutdown_connections 2023-05-20T01:19:36.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.172+0000 7f7f39c72700 1 --2- 172.21.15.16:0/2470915953 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7f3415d290 0x7f7f340cf800 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:36.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.172+0000 7f7f39c72700 1 --2- 172.21.15.16:0/2470915953 >> v2:172.21.15.132:3300/0 conn(0x7f7f34072300 0x7f7f340cf2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:36.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.172+0000 7f7f39c72700 1 -- 172.21.15.16:0/2470915953 >> 172.21.15.16:0/2470915953 conn(0x7f7f340cffd0 msgr2=0x7f7f340d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:19:36.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.172+0000 7f7f39c72700 1 -- 172.21.15.16:0/2470915953 shutdown_connections 2023-05-20T01:19:36.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.172+0000 7f7f39c72700 1 -- 172.21.15.16:0/2470915953 wait complete. 2023-05-20T01:19:36.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.173+0000 7f7f39c72700 1 Processor -- start 2023-05-20T01:19:36.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.173+0000 7f7f39c72700 1 -- start start 2023-05-20T01:19:36.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.174+0000 7f7f39c72700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f34072300 0x7f7f340b1380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:36.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.174+0000 7f7f39c72700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7f3415cec0 0x7f7f340cb6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:36.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.174+0000 7f7f39c72700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f3415d290 0x7f7f340cbc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:36.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.174+0000 7f7f39c72700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f341f3400 con 0x7f7f3415d290 2023-05-20T01:19:36.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.174+0000 7f7f39c72700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f341f36e0 con 0x7f7f34072300 2023-05-20T01:19:36.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.174+0000 7f7f39c72700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f341f39c0 con 0x7f7f3415cec0 2023-05-20T01:19:36.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.174+0000 7f7f33fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f3415d290 0x7f7f340cbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:36.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.174+0000 7f7f33fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f3415d290 0x7f7f340cbc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33352/0 (socket says 172.21.15.16:33352) 2023-05-20T01:19:36.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.174+0000 7f7f33fff700 1 -- 172.21.15.16:0/3160567736 learned_addr learned my addr 172.21.15.16:0/3160567736 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:19:36.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.174+0000 7f7f32ffd700 1 --2- 172.21.15.16:0/3160567736 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7f3415cec0 0x7f7f340cb6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:36.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.174+0000 7f7f337fe700 1 --2- 172.21.15.16:0/3160567736 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f34072300 0x7f7f340b1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:36.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.174+0000 7f7f33fff700 1 -- 172.21.15.16:0/3160567736 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7f3415cec0 msgr2=0x7f7f340cb6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:36.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.174+0000 7f7f33fff700 1 --2- 172.21.15.16:0/3160567736 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7f3415cec0 0x7f7f340cb6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:36.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.174+0000 7f7f33fff700 1 -- 172.21.15.16:0/3160567736 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f34072300 msgr2=0x7f7f340b1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:36.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.174+0000 7f7f33fff700 1 --2- 172.21.15.16:0/3160567736 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f34072300 0x7f7f340b1380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:36.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.174+0000 7f7f33fff700 1 -- 172.21.15.16:0/3160567736 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f2c009a70 con 0x7f7f3415d290 2023-05-20T01:19:36.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.174+0000 7f7f32ffd700 1 --2- 172.21.15.16:0/3160567736 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7f3415cec0 0x7f7f340cb6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:19:36.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.174+0000 7f7f337fe700 1 --2- 172.21.15.16:0/3160567736 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f34072300 0x7f7f340b1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:19:36.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.174+0000 7f7f33fff700 1 --2- 172.21.15.16:0/3160567736 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f3415d290 0x7f7f340cbc30 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f7f2c014040 tx=0x7f7f2c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:36.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.175+0000 7f7f2bfff700 1 -- 172.21.15.16:0/3160567736 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f2c012430 con 0x7f7f3415d290 2023-05-20T01:19:36.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.175+0000 7f7f2bfff700 1 -- 172.21.15.16:0/3160567736 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f2c020070 con 0x7f7f3415d290 2023-05-20T01:19:36.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.175+0000 7f7f2bfff700 1 -- 172.21.15.16:0/3160567736 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f2c003ea0 con 0x7f7f3415d290 2023-05-20T01:19:36.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.175+0000 7f7f39c72700 1 -- 172.21.15.16:0/3160567736 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f340cfd40 con 0x7f7f3415d290 2023-05-20T01:19:36.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.175+0000 7f7f39c72700 1 -- 172.21.15.16:0/3160567736 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f340cd2c0 con 0x7f7f3415d290 2023-05-20T01:19:36.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.176+0000 7f7f39c72700 1 -- 172.21.15.16:0/3160567736 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f34160890 con 0x7f7f3415d290 2023-05-20T01:19:36.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.176+0000 7f7f2bfff700 1 -- 172.21.15.16:0/3160567736 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7f2c004280 con 0x7f7f3415d290 2023-05-20T01:19:36.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.177+0000 7f7f2bfff700 1 --2- 172.21.15.16:0/3160567736 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f7f1406ace0 0x7f7f1406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:36.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.177+0000 7f7f2bfff700 1 -- 172.21.15.16:0/3160567736 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 7102+0+0 (secure 0 0 0) 0x7f7f2c022070 con 0x7f7f3415d290 2023-05-20T01:19:36.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.177+0000 7f7f337fe700 1 --2- 172.21.15.16:0/3160567736 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f7f1406ace0 0x7f7f1406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:36.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.177+0000 7f7f337fe700 1 --2- 172.21.15.16:0/3160567736 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f7f1406ace0 0x7f7f1406d190 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c007fd0 tx=0x7f7f1c007da0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:36.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.181+0000 7f7f2bfff700 1 -- 172.21.15.16:0/3160567736 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f7f2c056930 con 0x7f7f3415d290 2023-05-20T01:19:36.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.379+0000 7f7f39c72700 1 -- 172.21.15.16:0/3160567736 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"} v 0) v1 -- 0x7f7f340747b0 con 0x7f7f3415d290 2023-05-20T01:19:36.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.379+0000 7f7f2bfff700 1 -- 172.21.15.16:0/3160567736 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"}]=0 v51) v1 ==== 106+0+11 (secure 0 0 0) 0x7f7f2c02c0b0 con 0x7f7f3415d290 2023-05-20T01:19:36.383 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-05-20T01:19:36.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.381+0000 7f7f39c72700 1 -- 172.21.15.16:0/3160567736 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f7f1406ace0 msgr2=0x7f7f1406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:36.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.381+0000 7f7f39c72700 1 --2- 172.21.15.16:0/3160567736 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f7f1406ace0 0x7f7f1406d190 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c007fd0 tx=0x7f7f1c007da0 comp rx=0 tx=0).stop 2023-05-20T01:19:36.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.381+0000 7f7f39c72700 1 -- 172.21.15.16:0/3160567736 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f3415d290 msgr2=0x7f7f340cbc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:36.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.381+0000 7f7f39c72700 1 --2- 172.21.15.16:0/3160567736 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f3415d290 0x7f7f340cbc30 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f7f2c014040 tx=0x7f7f2c004ad0 comp rx=0 tx=0).stop 2023-05-20T01:19:36.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.381+0000 7f7f39c72700 1 -- 172.21.15.16:0/3160567736 shutdown_connections 2023-05-20T01:19:36.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.381+0000 7f7f39c72700 1 --2- 172.21.15.16:0/3160567736 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f3415d290 0x7f7f340cbc30 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:36.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.381+0000 7f7f39c72700 1 --2- 172.21.15.16:0/3160567736 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f34072300 0x7f7f340b1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:36.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.381+0000 7f7f39c72700 1 --2- 172.21.15.16:0/3160567736 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f7f1406ace0 0x7f7f1406d190 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:36.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.381+0000 7f7f39c72700 1 --2- 172.21.15.16:0/3160567736 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7f3415cec0 0x7f7f340cb6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:36.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.381+0000 7f7f39c72700 1 -- 172.21.15.16:0/3160567736 >> 172.21.15.16:0/3160567736 conn(0x7f7f340cffd0 msgr2=0x7f7f3415f9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:19:36.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.381+0000 7f7f39c72700 1 -- 172.21.15.16:0/3160567736 shutdown_connections 2023-05-20T01:19:36.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.381+0000 7f7f39c72700 1 -- 172.21.15.16:0/3160567736 wait complete. 2023-05-20T01:19:36.397 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-20T01:19:36.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.519+0000 7f7a07ebf700 1 Processor -- start 2023-05-20T01:19:36.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.519+0000 7f7a07ebf700 1 -- start start 2023-05-20T01:19:36.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.520+0000 7f7a07ebf700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f7a000ca980 0x7f7a000aef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:36.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.520+0000 7f7a07ebf700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7a000af450 0x7f7a0015dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:36.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.520+0000 7f7a07ebf700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a000ce920 con 0x7f7a000cad50 2023-05-20T01:19:36.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.520+0000 7f7a07ebf700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a000af8f0 con 0x7f7a000af450 2023-05-20T01:19:36.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.520+0000 7f7a07ebf700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a000725d0 con 0x7f7a000ca980 2023-05-20T01:19:36.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.520+0000 7f7a0545a700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7a000af450 0x7f7a0015dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:36.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.520+0000 7f7a0545a700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7a000af450 0x7f7a0015dc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33356/0 (socket says 172.21.15.16:33356) 2023-05-20T01:19:36.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.520+0000 7f7a0545a700 1 -- 172.21.15.16:0/1742814308 learned_addr learned my addr 172.21.15.16:0/1742814308 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:19:36.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.520+0000 7f7a05c5b700 1 --2- 172.21.15.16:0/1742814308 >> v2:172.21.15.132:3300/0 conn(0x7f7a000ca980 0x7f7a000aef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:36.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.520+0000 7f7a0545a700 1 -- 172.21.15.16:0/1742814308 >> v2:172.21.15.132:3300/0 conn(0x7f7a000ca980 msgr2=0x7f7a000aef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:19:36.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.520+0000 7f7a0545a700 1 --2- 172.21.15.16:0/1742814308 >> v2:172.21.15.132:3300/0 conn(0x7f7a000ca980 0x7f7a000aef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:36.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.520+0000 7f7a0545a700 1 -- 172.21.15.16:0/1742814308 >> v1:172.21.15.132:6789/0 conn(0x7f7a000cad50 legacy=0x7f7a0015e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:19:36.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.520+0000 7f7a0545a700 1 -- 172.21.15.16:0/1742814308 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a000722f0 con 0x7f7a000af450 2023-05-20T01:19:36.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.521+0000 7f7a0545a700 1 --2- 172.21.15.16:0/1742814308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a000af450 0x7f7a0015dc70 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f7a0006a0f0 tx=0x7f79fc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2378bfcb0b0cf95d server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:36.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.521+0000 7f7a04c59700 1 -- 172.21.15.16:0/1742814308 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79fc004230 con 0x7f7a000af450 2023-05-20T01:19:36.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.521+0000 7f7a04c59700 1 -- 172.21.15.16:0/1742814308 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79fc004390 con 0x7f7a000af450 2023-05-20T01:19:36.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.521+0000 7f7a04c59700 1 -- 172.21.15.16:0/1742814308 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79fc0036a0 con 0x7f7a000af450 2023-05-20T01:19:36.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.521+0000 7f7a07ebf700 1 -- 172.21.15.16:0/1742814308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a000af450 msgr2=0x7f7a0015dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:36.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.521+0000 7f7a07ebf700 1 --2- 172.21.15.16:0/1742814308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a000af450 0x7f7a0015dc70 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f7a0006a0f0 tx=0x7f79fc009bd0 comp rx=0 tx=0).stop 2023-05-20T01:19:36.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.521+0000 7f7a07ebf700 1 -- 172.21.15.16:0/1742814308 shutdown_connections 2023-05-20T01:19:36.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.521+0000 7f7a07ebf700 1 --2- 172.21.15.16:0/1742814308 >> v2:172.21.15.132:3300/0 conn(0x7f7a000ca980 0x7f7a000aef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:36.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.521+0000 7f7a07ebf700 1 --2- 172.21.15.16:0/1742814308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a000af450 0x7f7a0015dc70 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:36.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.521+0000 7f7a07ebf700 1 -- 172.21.15.16:0/1742814308 >> 172.21.15.16:0/1742814308 conn(0x7f7a000d0c70 msgr2=0x7f7a000b1af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:19:36.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.521+0000 7f7a07ebf700 1 -- 172.21.15.16:0/1742814308 shutdown_connections 2023-05-20T01:19:36.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.521+0000 7f7a07ebf700 1 -- 172.21.15.16:0/1742814308 wait complete. 2023-05-20T01:19:36.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.523+0000 7f7a07ebf700 1 Processor -- start 2023-05-20T01:19:36.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.523+0000 7f7a07ebf700 1 -- start start 2023-05-20T01:19:36.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.523+0000 7f7a07ebf700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7a000af450 0x7f7a001d79a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:36.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.523+0000 7f7a07ebf700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7a000ca980 0x7f7a0006e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:36.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.523+0000 7f7a07ebf700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a000cad50 0x7f7a0006e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:36.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.523+0000 7f7a07ebf700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a001d48d0 con 0x7f7a000cad50 2023-05-20T01:19:36.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.523+0000 7f7a07ebf700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a001d45f0 con 0x7f7a000af450 2023-05-20T01:19:36.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.523+0000 7f7a07ebf700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a000c7230 con 0x7f7a000ca980 2023-05-20T01:19:36.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.524+0000 7f7a0645c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a000cad50 0x7f7a0006e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:36.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.524+0000 7f7a0645c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a000cad50 0x7f7a0006e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33364/0 (socket says 172.21.15.16:33364) 2023-05-20T01:19:36.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.524+0000 7f7a0645c700 1 -- 172.21.15.16:0/2601125044 learned_addr learned my addr 172.21.15.16:0/2601125044 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:19:36.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.524+0000 7f7a0545a700 1 --2- 172.21.15.16:0/2601125044 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7a000ca980 0x7f7a0006e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:36.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.524+0000 7f7a05c5b700 1 --2- 172.21.15.16:0/2601125044 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7a000af450 0x7f7a001d79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:36.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.524+0000 7f7a0645c700 1 -- 172.21.15.16:0/2601125044 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7a000ca980 msgr2=0x7f7a0006e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:36.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.524+0000 7f7a0645c700 1 --2- 172.21.15.16:0/2601125044 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7a000ca980 0x7f7a0006e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:36.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.524+0000 7f7a0645c700 1 -- 172.21.15.16:0/2601125044 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7a000af450 msgr2=0x7f7a001d79a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:36.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.524+0000 7f7a0645c700 1 --2- 172.21.15.16:0/2601125044 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7a000af450 0x7f7a001d79a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:36.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.524+0000 7f7a0645c700 1 -- 172.21.15.16:0/2601125044 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79fc009550 con 0x7f7a000cad50 2023-05-20T01:19:36.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.524+0000 7f7a0645c700 1 --2- 172.21.15.16:0/2601125044 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a000cad50 0x7f7a0006e590 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f79f800b970 tx=0x7f79f800bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:36.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.524+0000 7f79f67fc700 1 -- 172.21.15.16:0/2601125044 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f800d860 con 0x7f7a000cad50 2023-05-20T01:19:36.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.524+0000 7f79f67fc700 1 -- 172.21.15.16:0/2601125044 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79f800d9c0 con 0x7f7a000cad50 2023-05-20T01:19:36.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.524+0000 7f79f67fc700 1 -- 172.21.15.16:0/2601125044 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f800dd70 con 0x7f7a000cad50 2023-05-20T01:19:36.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.524+0000 7f7a07ebf700 1 -- 172.21.15.16:0/2601125044 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a000cc1c0 con 0x7f7a000cad50 2023-05-20T01:19:36.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.524+0000 7f7a07ebf700 1 -- 172.21.15.16:0/2601125044 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a000ce5b0 con 0x7f7a000cad50 2023-05-20T01:19:36.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.525+0000 7f7a07ebf700 1 -- 172.21.15.16:0/2601125044 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a000cc350 con 0x7f7a000cad50 2023-05-20T01:19:36.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.526+0000 7f79f67fc700 1 -- 172.21.15.16:0/2601125044 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f79f800e430 con 0x7f7a000cad50 2023-05-20T01:19:36.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.526+0000 7f79f67fc700 1 --2- 172.21.15.16:0/2601125044 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f79e406ace0 0x7f79e406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:36.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.526+0000 7f7a05c5b700 1 --2- 172.21.15.16:0/2601125044 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f79e406ace0 0x7f79e406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:36.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.526+0000 7f79f67fc700 1 -- 172.21.15.16:0/2601125044 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 7102+0+0 (secure 0 0 0) 0x7f79f80896b0 con 0x7f7a000cad50 2023-05-20T01:19:36.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.526+0000 7f7a05c5b700 1 --2- 172.21.15.16:0/2601125044 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f79e406ace0 0x7f79e406d190 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f7a000af050 tx=0x7f79f0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:36.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.531+0000 7f79f67fc700 1 -- 172.21.15.16:0/2601125044 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f79f8054350 con 0x7f7a000cad50 2023-05-20T01:19:36.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.775+0000 7f7a07ebf700 1 -- 172.21.15.16:0/2601125044 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7a000747b0 con 0x7f7a000cad50 2023-05-20T01:19:36.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.775+0000 7f79f67fc700 1 -- 172.21.15.16:0/2601125044 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+4680 (secure 0 0 0) 0x7f79f8057970 con 0x7f7a000cad50 2023-05-20T01:19:36.780 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:19:36.780 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":29,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5246,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/2629811751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":2629811751},{"type":"v1","addr":"172.21.15.132:6834","nonce":2629811751}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":5249,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2117148777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":2117148777},{"type":"v1","addr":"172.21.15.132:6835","nonce":2117148777}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":5566,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2025052254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":2025052254},{"type":"v1","addr":"172.21.15.16:6837","nonce":2025052254}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-20T01:18:26.227827+0000","modified":"2023-05-20T01:18:35.503777+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5889},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5889":{"gid":5889,"name":"c","rank":0,"incarnation":26,"state":"up:active","state_seq":4,"addr":"172.21.15.16:6836/3284962389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3284962389},{"type":"v1","addr":"172.21.15.16:6836","nonce":3284962389}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7,8],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-05-20T01:19:36.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.777+0000 7f7a07ebf700 1 -- 172.21.15.16:0/2601125044 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f79e406ace0 msgr2=0x7f79e406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:36.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.777+0000 7f7a07ebf700 1 --2- 172.21.15.16:0/2601125044 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f79e406ace0 0x7f79e406d190 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f7a000af050 tx=0x7f79f0008040 comp rx=0 tx=0).stop 2023-05-20T01:19:36.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.777+0000 7f7a07ebf700 1 -- 172.21.15.16:0/2601125044 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a000cad50 msgr2=0x7f7a0006e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:36.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.777+0000 7f7a07ebf700 1 --2- 172.21.15.16:0/2601125044 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a000cad50 0x7f7a0006e590 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f79f800b970 tx=0x7f79f800bc80 comp rx=0 tx=0).stop 2023-05-20T01:19:36.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.777+0000 7f7a07ebf700 1 -- 172.21.15.16:0/2601125044 shutdown_connections 2023-05-20T01:19:36.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.777+0000 7f7a07ebf700 1 --2- 172.21.15.16:0/2601125044 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a000cad50 0x7f7a0006e590 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:36.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.777+0000 7f7a07ebf700 1 --2- 172.21.15.16:0/2601125044 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7a000af450 0x7f7a001d79a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:36.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.777+0000 7f7a07ebf700 1 --2- 172.21.15.16:0/2601125044 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f79e406ace0 0x7f79e406d190 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:36.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.777+0000 7f7a07ebf700 1 --2- 172.21.15.16:0/2601125044 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7a000ca980 0x7f7a0006e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:36.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.777+0000 7f7a07ebf700 1 -- 172.21.15.16:0/2601125044 >> 172.21.15.16:0/2601125044 conn(0x7f7a000d0c70 msgr2=0x7f7a000b5300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:19:36.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.777+0000 7f7a07ebf700 1 -- 172.21.15.16:0/2601125044 shutdown_connections 2023-05-20T01:19:36.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.778+0000 7f7a07ebf700 1 -- 172.21.15.16:0/2601125044 wait complete. 2023-05-20T01:19:36.783 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 29 2023-05-20T01:19:36.793 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-20T01:19:36.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.913+0000 7fc33ec88700 1 Processor -- start 2023-05-20T01:19:36.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.914+0000 7fc33ec88700 1 -- start start 2023-05-20T01:19:36.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.914+0000 7fc33ec88700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc33806d4b0 0x7fc3380cf890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:36.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.914+0000 7fc33ec88700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fc33806a510 0x7fc33806a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:36.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.914+0000 7fc33ec88700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3380cc510 con 0x7fc33806f450 2023-05-20T01:19:36.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.914+0000 7fc33ec88700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc33806aef0 con 0x7fc33806a510 2023-05-20T01:19:36.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.914+0000 7fc33ec88700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc338071ed0 con 0x7fc33806d4b0 2023-05-20T01:19:36.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.914+0000 7fc33ca24700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fc33806f450 0x7fc338161080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:44896/0 (socket says 172.21.15.16:44896) 2023-05-20T01:19:36.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.914+0000 7fc33ca24700 1 -- 172.21.15.16:0/32467353 learned_addr learned my addr 172.21.15.16:0/32467353 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:19:36.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.915+0000 7fc337fff700 1 --2- 172.21.15.16:0/32467353 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc33806d4b0 0x7fc3380cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:36.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.915+0000 7fc3377fe700 1 -- 172.21.15.16:0/32467353 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2450494604 0 0) 0x7fc3380cc510 con 0x7fc33806f450 2023-05-20T01:19:36.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.915+0000 7fc33d225700 1 --2- 172.21.15.16:0/32467353 >> v2:172.21.15.132:3300/0 conn(0x7fc33806a510 0x7fc33806a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:36.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.915+0000 7fc3377fe700 1 -- 172.21.15.16:0/32467353 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc3280035b0 con 0x7fc33806f450 2023-05-20T01:19:36.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.915+0000 7fc3377fe700 1 -- 172.21.15.16:0/32467353 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1700865439 0 0) 0x7fc3280035b0 con 0x7fc33806f450 2023-05-20T01:19:36.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.915+0000 7fc3377fe700 1 -- 172.21.15.16:0/32467353 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc33806d4b0 msgr2=0x7fc3380cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:19:36.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.915+0000 7fc3377fe700 1 --2- 172.21.15.16:0/32467353 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc33806d4b0 0x7fc3380cf890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:36.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.915+0000 7fc3377fe700 1 -- 172.21.15.16:0/32467353 >> v2:172.21.15.132:3300/0 conn(0x7fc33806a510 msgr2=0x7fc33806a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:19:36.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.915+0000 7fc3377fe700 1 --2- 172.21.15.16:0/32467353 >> v2:172.21.15.132:3300/0 conn(0x7fc33806a510 0x7fc33806a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:36.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.915+0000 7fc3377fe700 1 -- 172.21.15.16:0/32467353 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3380721b0 con 0x7fc33806f450 2023-05-20T01:19:36.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.915+0000 7fc3377fe700 1 -- 172.21.15.16:0/32467353 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7fc32c001670 con 0x7fc33806f450 2023-05-20T01:19:36.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.915+0000 7fc3377fe700 1 -- 172.21.15.16:0/32467353 >> v1:172.21.15.16:6789/0 conn(0x7fc33806f450 legacy=0x7fc338161080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:36.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.915+0000 7fc3377fe700 1 --2- 172.21.15.16:0/32467353 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc33806d4b0 0x7fc328003a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:36.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.915+0000 7fc3377fe700 1 --2- 172.21.15.16:0/32467353 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc328003ff0 0x7fc328004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:36.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.915+0000 7fc3377fe700 1 --2- 172.21.15.16:0/32467353 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc328004d90 0x7fc328005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:36.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.915+0000 7fc3377fe700 1 -- 172.21.15.16:0/32467353 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc338071ed0 con 0x7fc328003ff0 2023-05-20T01:19:36.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.915+0000 7fc3377fe700 1 -- 172.21.15.16:0/32467353 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc32800ba40 con 0x7fc328004d90 2023-05-20T01:19:36.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.915+0000 7fc3377fe700 1 -- 172.21.15.16:0/32467353 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc32800bc00 con 0x7fc33806d4b0 2023-05-20T01:19:36.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.915+0000 7fc337fff700 1 --2- 172.21.15.16:0/32467353 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc328003ff0 0x7fc328004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:36.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.915+0000 7fc33ca24700 1 --2- 172.21.15.16:0/32467353 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc33806d4b0 0x7fc328003a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:36.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.915+0000 7fc33d225700 1 --2- 172.21.15.16:0/32467353 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc328004d90 0x7fc328005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:36.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.916+0000 7fc337fff700 1 -- 172.21.15.16:0/32467353 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc33806d4b0 msgr2=0x7fc328003a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:19:36.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.916+0000 7fc337fff700 1 --2- 172.21.15.16:0/32467353 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc33806d4b0 0x7fc328003a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:36.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.916+0000 7fc337fff700 1 -- 172.21.15.16:0/32467353 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc328004d90 msgr2=0x7fc328005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:19:36.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.916+0000 7fc337fff700 1 --2- 172.21.15.16:0/32467353 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc328004d90 0x7fc328005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:36.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.916+0000 7fc337fff700 1 -- 172.21.15.16:0/32467353 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3280035b0 con 0x7fc328003ff0 2023-05-20T01:19:36.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.916+0000 7fc337fff700 1 --2- 172.21.15.16:0/32467353 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc328003ff0 0x7fc328004850 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fc324009b30 tx=0x7fc328005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97c69c830182ae18 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:36.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.916+0000 7fc3377fe700 1 -- 172.21.15.16:0/32467353 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc32400e6c0 con 0x7fc328003ff0 2023-05-20T01:19:36.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.916+0000 7fc3377fe700 1 -- 172.21.15.16:0/32467353 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc32400e820 con 0x7fc328003ff0 2023-05-20T01:19:36.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.916+0000 7fc3377fe700 1 -- 172.21.15.16:0/32467353 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc32400eb90 con 0x7fc328003ff0 2023-05-20T01:19:36.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.916+0000 7fc33ec88700 1 -- 172.21.15.16:0/32467353 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc328003ff0 msgr2=0x7fc328004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:36.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.916+0000 7fc33ec88700 1 --2- 172.21.15.16:0/32467353 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc328003ff0 0x7fc328004850 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fc324009b30 tx=0x7fc328005880 comp rx=0 tx=0).stop 2023-05-20T01:19:36.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.916+0000 7fc33d225700 1 -- 172.21.15.16:0/32467353 reap_dead start 2023-05-20T01:19:36.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.917+0000 7fc33ec88700 1 -- 172.21.15.16:0/32467353 shutdown_connections 2023-05-20T01:19:36.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.917+0000 7fc33ec88700 1 -- 172.21.15.16:0/32467353 >> 172.21.15.16:0/32467353 conn(0x7fc3380cffb0 msgr2=0x7fc3380d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:19:36.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.917+0000 7fc33ec88700 1 -- 172.21.15.16:0/32467353 shutdown_connections 2023-05-20T01:19:36.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.917+0000 7fc33ec88700 1 -- 172.21.15.16:0/32467353 wait complete. 2023-05-20T01:19:36.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.918+0000 7fc33ec88700 1 Processor -- start 2023-05-20T01:19:36.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.918+0000 7fc33ec88700 1 -- start start 2023-05-20T01:19:36.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.919+0000 7fc33ec88700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc328003ff0 0x7fc3380b0120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:36.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.919+0000 7fc33ec88700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3380ac8a0 0x7fc3381582a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:36.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.919+0000 7fc33ec88700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc3380af200 0x7fc3380b3e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:36.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.919+0000 7fc33ec88700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3380b5360 con 0x7fc3380ac8a0 2023-05-20T01:19:36.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.919+0000 7fc33ec88700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3381f4540 con 0x7fc3380af200 2023-05-20T01:19:36.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.919+0000 7fc33ec88700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3381f3930 con 0x7fc328003ff0 2023-05-20T01:19:36.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.919+0000 7fc337fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3380ac8a0 0x7fc3381582a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:36.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.920+0000 7fc337fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3380ac8a0 0x7fc3381582a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33382/0 (socket says 172.21.15.16:33382) 2023-05-20T01:19:36.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.920+0000 7fc337fff700 1 -- 172.21.15.16:0/409638837 learned_addr learned my addr 172.21.15.16:0/409638837 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:19:36.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.920+0000 7fc33ca24700 1 --2- 172.21.15.16:0/409638837 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc328003ff0 0x7fc3380b0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:36.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.920+0000 7fc33d225700 1 --2- 172.21.15.16:0/409638837 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc3380af200 0x7fc3380b3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:36.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.920+0000 7fc337fff700 1 -- 172.21.15.16:0/409638837 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc328003ff0 msgr2=0x7fc3380b0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:36.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.920+0000 7fc337fff700 1 --2- 172.21.15.16:0/409638837 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc328003ff0 0x7fc3380b0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:36.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.920+0000 7fc337fff700 1 -- 172.21.15.16:0/409638837 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc3380af200 msgr2=0x7fc3380b3e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:36.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.920+0000 7fc337fff700 1 --2- 172.21.15.16:0/409638837 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc3380af200 0x7fc3380b3e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:36.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.920+0000 7fc337fff700 1 -- 172.21.15.16:0/409638837 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3380cbe50 con 0x7fc3380ac8a0 2023-05-20T01:19:36.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.920+0000 7fc33d225700 1 --2- 172.21.15.16:0/409638837 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc3380af200 0x7fc3380b3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:19:36.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.920+0000 7fc337fff700 1 --2- 172.21.15.16:0/409638837 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3380ac8a0 0x7fc3381582a0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fc32400e4b0 tx=0x7fc3240047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:36.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.920+0000 7fc3357fa700 1 -- 172.21.15.16:0/409638837 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc32400e6c0 con 0x7fc3380ac8a0 2023-05-20T01:19:36.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.920+0000 7fc3357fa700 1 -- 172.21.15.16:0/409638837 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc324003ad0 con 0x7fc3380ac8a0 2023-05-20T01:19:36.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.920+0000 7fc33ec88700 1 -- 172.21.15.16:0/409638837 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3380cc1a0 con 0x7fc3380ac8a0 2023-05-20T01:19:36.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.920+0000 7fc3357fa700 1 -- 172.21.15.16:0/409638837 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc324003e80 con 0x7fc3380ac8a0 2023-05-20T01:19:36.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.920+0000 7fc33ec88700 1 -- 172.21.15.16:0/409638837 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3380cf890 con 0x7fc3380ac8a0 2023-05-20T01:19:36.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.921+0000 7fc33ec88700 1 -- 172.21.15.16:0/409638837 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3380747b0 con 0x7fc3380ac8a0 2023-05-20T01:19:36.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.922+0000 7fc3357fa700 1 -- 172.21.15.16:0/409638837 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc324013470 con 0x7fc3380ac8a0 2023-05-20T01:19:36.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.922+0000 7fc3357fa700 1 --2- 172.21.15.16:0/409638837 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc31c06ace0 0x7fc31c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:36.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.922+0000 7fc3357fa700 1 -- 172.21.15.16:0/409638837 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 7102+0+0 (secure 0 0 0) 0x7fc32408b240 con 0x7fc3380ac8a0 2023-05-20T01:19:36.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.922+0000 7fc33ca24700 1 --2- 172.21.15.16:0/409638837 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc31c06ace0 0x7fc31c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:36.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.922+0000 7fc33ca24700 1 --2- 172.21.15.16:0/409638837 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc31c06ace0 0x7fc31c06d190 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fc32c001050 tx=0x7fc32c001e10 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:36.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:36.926+0000 7fc3357fa700 1 -- 172.21.15.16:0/409638837 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fc324056090 con 0x7fc3380ac8a0 2023-05-20T01:19:37.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.127+0000 7fc33ec88700 1 -- 172.21.15.16:0/409638837 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc3381f4680 con 0x7fc3380ac8a0 2023-05-20T01:19:37.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.129+0000 7fc3357fa700 1 -- 172.21.15.16:0/409638837 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) v1 ==== 74+0+20668 (secure 0 0 0) 0x7fc32402d090 con 0x7fc3380ac8a0 2023-05-20T01:19:37.133 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:19:37.134 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":51,"fsid":"c2245cc5-ecab-4087-a59b-bc81db901676","created":"2023-05-20T01:16:09.549230+0000","modified":"2023-05-20T01:18:31.674809+0000","last_up_change":"2023-05-20T01:16:19.788624+0000","last_in_change":"2023-05-20T01:16:12.092160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-05-20T01:16:20.934366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-05-20T01:16:21.372294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-05-20T01:17:44.628755+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-05-20T01:17:45.968584+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data_extra","create_time":"2023-05-20T01:17:54.915781+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"317a83cd-19c8-4958-a301-6087ef342289","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6825","nonce":3923634179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6827","nonce":3923634179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6831","nonce":3923634179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6829","nonce":3923634179}]},"public_addr":"172.21.15.16:6825/3923634179","cluster_addr":"172.21.15.16:6827/3923634179","heartbeat_back_addr":"172.21.15.16:6831/3923634179","heartbeat_front_addr":"172.21.15.16:6829/3923634179","state":["exists","up"]},{"osd":1,"uuid":"5581046f-8219-4089-8d7d-b7846399505e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6809","nonce":3860264220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6811","nonce":3860264220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6815","nonce":3860264220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6813","nonce":3860264220}]},"public_addr":"172.21.15.16:6809/3860264220","cluster_addr":"172.21.15.16:6811/3860264220","heartbeat_back_addr":"172.21.15.16:6815/3860264220","heartbeat_front_addr":"172.21.15.16:6813/3860264220","state":["exists","up"]},{"osd":2,"uuid":"072e9e66-4133-478e-8d4a-64d46c2963f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6817","nonce":293187030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6819","nonce":293187030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6823","nonce":293187030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6821","nonce":293187030}]},"public_addr":"172.21.15.16:6817/293187030","cluster_addr":"172.21.15.16:6819/293187030","heartbeat_back_addr":"172.21.15.16:6823/293187030","heartbeat_front_addr":"172.21.15.16:6821/293187030","state":["exists","up"]},{"osd":3,"uuid":"11e1bba6-00f8-48dd-9028-d8d716b7f145","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6801","nonce":3669781890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6803","nonce":3669781890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6807","nonce":3669781890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6805","nonce":3669781890}]},"public_addr":"172.21.15.16:6801/3669781890","cluster_addr":"172.21.15.16:6803/3669781890","heartbeat_back_addr":"172.21.15.16:6807/3669781890","heartbeat_front_addr":"172.21.15.16:6805/3669781890","state":["exists","up"]},{"osd":4,"uuid":"b8909c3a-cf70-46cb-8d0b-009c707cf9d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6819","nonce":506317223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6823","nonce":506317223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6829","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6831","nonce":506317223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6827","nonce":506317223}]},"public_addr":"172.21.15.132:6819/506317223","cluster_addr":"172.21.15.132:6823/506317223","heartbeat_back_addr":"172.21.15.132:6831/506317223","heartbeat_front_addr":"172.21.15.132:6827/506317223","state":["exists","up"]},{"osd":5,"uuid":"b1818304-4583-47bf-a6ec-0bfbb5166233","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6818","nonce":1844689995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6822","nonce":1844689995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6830","nonce":1844689995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6826","nonce":1844689995}]},"public_addr":"172.21.15.132:6818/1844689995","cluster_addr":"172.21.15.132:6822/1844689995","heartbeat_back_addr":"172.21.15.132:6830/1844689995","heartbeat_front_addr":"172.21.15.132:6826/1844689995","state":["exists","up"]},{"osd":6,"uuid":"d37f4691-eed1-4a93-8818-18354f03f4a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6801","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6802","nonce":727581123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6805","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6807","nonce":727581123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6813","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6815","nonce":727581123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6811","nonce":727581123}]},"public_addr":"172.21.15.132:6802/727581123","cluster_addr":"172.21.15.132:6807/727581123","heartbeat_back_addr":"172.21.15.132:6815/727581123","heartbeat_front_addr":"172.21.15.132:6811/727581123","state":["exists","up"]},{"osd":7,"uuid":"4f712f62-5d1d-4816-8042-03209144f8b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6803","nonce":466001149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6806","nonce":466001149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6814","nonce":466001149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6810","nonce":466001149}]},"public_addr":"172.21.15.132:6803/466001149","cluster_addr":"172.21.15.132:6806/466001149","heartbeat_back_addr":"172.21.15.132:6814/466001149","heartbeat_front_addr":"172.21.15.132:6810/466001149","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.338863+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.278680+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305694+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.246835+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.276237+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.304441+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305421+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.267050+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.9","mappings":[{"from":0,"to":4}]},{"pgid":"7.1b","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6837/1867776973":"2023-05-21T01:18:31.666399+0000","172.21.15.16:6835/1867776973":"2023-05-21T01:18:31.666399+0000","172.21.15.16:6837/3517262366":"2023-05-21T01:18:26.467438+0000","172.21.15.16:6835/3517262366":"2023-05-21T01:18:26.467438+0000","172.21.15.16:6835/942011026":"2023-05-21T01:18:24.644805+0000","172.21.15.16:6837/942011026":"2023-05-21T01:18:24.644805+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-20T01:19:37.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.131+0000 7fc33ec88700 1 -- 172.21.15.16:0/409638837 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc31c06ace0 msgr2=0x7fc31c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:37.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.131+0000 7fc33ec88700 1 --2- 172.21.15.16:0/409638837 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc31c06ace0 0x7fc31c06d190 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fc32c001050 tx=0x7fc32c001e10 comp rx=0 tx=0).stop 2023-05-20T01:19:37.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.131+0000 7fc33ec88700 1 -- 172.21.15.16:0/409638837 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3380ac8a0 msgr2=0x7fc3381582a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:37.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.131+0000 7fc33ec88700 1 --2- 172.21.15.16:0/409638837 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3380ac8a0 0x7fc3381582a0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fc32400e4b0 tx=0x7fc3240047c0 comp rx=0 tx=0).stop 2023-05-20T01:19:37.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.131+0000 7fc33ec88700 1 -- 172.21.15.16:0/409638837 shutdown_connections 2023-05-20T01:19:37.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.131+0000 7fc33ec88700 1 --2- 172.21.15.16:0/409638837 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc3380af200 0x7fc3380b3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:37.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.131+0000 7fc33ec88700 1 --2- 172.21.15.16:0/409638837 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc31c06ace0 0x7fc31c06d190 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:37.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.131+0000 7fc33ec88700 1 --2- 172.21.15.16:0/409638837 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc328003ff0 0x7fc3380b0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:37.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.131+0000 7fc33ec88700 1 --2- 172.21.15.16:0/409638837 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3380ac8a0 0x7fc3381582a0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:37.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.131+0000 7fc33ec88700 1 -- 172.21.15.16:0/409638837 >> 172.21.15.16:0/409638837 conn(0x7fc3380cffb0 msgr2=0x7fc3380cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:19:37.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.131+0000 7fc33ec88700 1 -- 172.21.15.16:0/409638837 shutdown_connections 2023-05-20T01:19:37.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.131+0000 7fc33ec88700 1 -- 172.21.15.16:0/409638837 wait complete. 2023-05-20T01:19:37.147 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-05-20T01:19:37.148 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-20T01:19:37.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.269+0000 7fc490282700 1 Processor -- start 2023-05-20T01:19:37.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.269+0000 7fc490282700 1 -- start start 2023-05-20T01:19:37.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.270+0000 7fc490282700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc48806f450 0x7fc4880cf890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:37.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.270+0000 7fc490282700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fc48806d4b0 0x7fc488161080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:37.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.270+0000 7fc490282700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4880cc510 con 0x7fc48806a510 2023-05-20T01:19:37.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.270+0000 7fc490282700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4881615c0 con 0x7fc48806f450 2023-05-20T01:19:37.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.270+0000 7fc490282700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc488071ed0 con 0x7fc48806d4b0 2023-05-20T01:19:37.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.271+0000 7fc48e01e700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc48806f450 0x7fc4880cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:37.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.271+0000 7fc48e81f700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fc48806a510 0x7fc4880cdb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:44916/0 (socket says 172.21.15.16:44916) 2023-05-20T01:19:37.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.271+0000 7fc48e81f700 1 -- 172.21.15.16:0/4146815622 learned_addr learned my addr 172.21.15.16:0/4146815622 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:19:37.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.271+0000 7fc48d81d700 1 --2- 172.21.15.16:0/4146815622 >> v2:172.21.15.132:3300/0 conn(0x7fc48806d4b0 0x7fc488161080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:37.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.271+0000 7fc48e01e700 1 -- 172.21.15.16:0/4146815622 >> v2:172.21.15.132:3300/0 conn(0x7fc48806d4b0 msgr2=0x7fc488161080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:19:37.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.271+0000 7fc48e01e700 1 --2- 172.21.15.16:0/4146815622 >> v2:172.21.15.132:3300/0 conn(0x7fc48806d4b0 0x7fc488161080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:37.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.271+0000 7fc48e01e700 1 -- 172.21.15.16:0/4146815622 >> v1:172.21.15.16:6789/0 conn(0x7fc48806a510 legacy=0x7fc4880cdb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:37.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.271+0000 7fc48e01e700 1 -- 172.21.15.16:0/4146815622 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4880721b0 con 0x7fc48806f450 2023-05-20T01:19:37.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.271+0000 7fc48d01c700 1 -- 172.21.15.16:0/4146815622 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 725646012 0 0) 0x7fc4880cc510 con 0x7fc48806a510 2023-05-20T01:19:37.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.271+0000 7fc48e01e700 1 --2- 172.21.15.16:0/4146815622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc48806f450 0x7fc4880cf890 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fc488070c40 tx=0x7fc478009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1f9b9d21c49e6a2 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:37.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.271+0000 7fc48d01c700 1 -- 172.21.15.16:0/4146815622 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc478011830 con 0x7fc48806f450 2023-05-20T01:19:37.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.271+0000 7fc48d01c700 1 -- 172.21.15.16:0/4146815622 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc478011990 con 0x7fc48806f450 2023-05-20T01:19:37.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.271+0000 7fc48d01c700 1 -- 172.21.15.16:0/4146815622 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc478011d20 con 0x7fc48806f450 2023-05-20T01:19:37.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.272+0000 7fc490282700 1 -- 172.21.15.16:0/4146815622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc48806f450 msgr2=0x7fc4880cf890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:37.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.272+0000 7fc490282700 1 --2- 172.21.15.16:0/4146815622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc48806f450 0x7fc4880cf890 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fc488070c40 tx=0x7fc478009b20 comp rx=0 tx=0).stop 2023-05-20T01:19:37.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.272+0000 7fc490282700 1 -- 172.21.15.16:0/4146815622 shutdown_connections 2023-05-20T01:19:37.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.272+0000 7fc490282700 1 --2- 172.21.15.16:0/4146815622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc48806f450 0x7fc4880cf890 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:37.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.272+0000 7fc490282700 1 --2- 172.21.15.16:0/4146815622 >> v2:172.21.15.132:3300/0 conn(0x7fc48806d4b0 0x7fc488161080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:37.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.272+0000 7fc490282700 1 -- 172.21.15.16:0/4146815622 >> 172.21.15.16:0/4146815622 conn(0x7fc4880cffb0 msgr2=0x7fc4880d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:19:37.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.272+0000 7fc490282700 1 -- 172.21.15.16:0/4146815622 shutdown_connections 2023-05-20T01:19:37.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.272+0000 7fc490282700 1 -- 172.21.15.16:0/4146815622 wait complete. 2023-05-20T01:19:37.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.273+0000 7fc490282700 1 Processor -- start 2023-05-20T01:19:37.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.273+0000 7fc490282700 1 -- start start 2023-05-20T01:19:37.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.274+0000 7fc490282700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc48806a510 0x7fc4880c8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:37.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.274+0000 7fc490282700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc48806d4b0 0x7fc4880c64d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:37.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.274+0000 7fc490282700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc48806f450 0x7fc4880c6a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:37.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.274+0000 7fc490282700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4881f31d0 con 0x7fc48806d4b0 2023-05-20T01:19:37.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.274+0000 7fc490282700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4881f6c40 con 0x7fc48806a510 2023-05-20T01:19:37.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.274+0000 7fc490282700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4881f28a0 con 0x7fc48806f450 2023-05-20T01:19:37.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.274+0000 7fc48d81d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc48806d4b0 0x7fc4880c64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:37.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.274+0000 7fc48d81d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc48806d4b0 0x7fc4880c64d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33396/0 (socket says 172.21.15.16:33396) 2023-05-20T01:19:37.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.274+0000 7fc48d81d700 1 -- 172.21.15.16:0/3936441843 learned_addr learned my addr 172.21.15.16:0/3936441843 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:19:37.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.274+0000 7fc48e01e700 1 --2- 172.21.15.16:0/3936441843 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc48806a510 0x7fc4880c8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:37.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.274+0000 7fc48e81f700 1 --2- 172.21.15.16:0/3936441843 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc48806f450 0x7fc4880c6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:37.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.274+0000 7fc48d81d700 1 -- 172.21.15.16:0/3936441843 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc48806f450 msgr2=0x7fc4880c6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:37.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.274+0000 7fc48d81d700 1 --2- 172.21.15.16:0/3936441843 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc48806f450 0x7fc4880c6a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:37.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.274+0000 7fc48d81d700 1 -- 172.21.15.16:0/3936441843 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc48806a510 msgr2=0x7fc4880c8b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:37.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.274+0000 7fc48d81d700 1 --2- 172.21.15.16:0/3936441843 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc48806a510 0x7fc4880c8b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:37.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.274+0000 7fc48d81d700 1 -- 172.21.15.16:0/3936441843 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4780094a0 con 0x7fc48806d4b0 2023-05-20T01:19:37.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.274+0000 7fc48e01e700 1 --2- 172.21.15.16:0/3936441843 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc48806a510 0x7fc4880c8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:19:37.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.274+0000 7fc48d81d700 1 --2- 172.21.15.16:0/3936441843 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc48806d4b0 0x7fc4880c64d0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fc48400e810 tx=0x7fc48400ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:37.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.275+0000 7fc47effd700 1 -- 172.21.15.16:0/3936441843 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc484013610 con 0x7fc48806d4b0 2023-05-20T01:19:37.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.275+0000 7fc47effd700 1 -- 172.21.15.16:0/3936441843 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc484013770 con 0x7fc48806d4b0 2023-05-20T01:19:37.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.275+0000 7fc490282700 1 -- 172.21.15.16:0/3936441843 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4880cbe50 con 0x7fc48806d4b0 2023-05-20T01:19:37.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.275+0000 7fc47effd700 1 -- 172.21.15.16:0/3936441843 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc484013b20 con 0x7fc48806d4b0 2023-05-20T01:19:37.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.275+0000 7fc490282700 1 -- 172.21.15.16:0/3936441843 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4880cc1a0 con 0x7fc48806d4b0 2023-05-20T01:19:37.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.276+0000 7fc490282700 1 -- 172.21.15.16:0/3936441843 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc48806a950 con 0x7fc48806d4b0 2023-05-20T01:19:37.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.276+0000 7fc47effd700 1 -- 172.21.15.16:0/3936441843 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc484013c80 con 0x7fc48806d4b0 2023-05-20T01:19:37.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.277+0000 7fc47effd700 1 --2- 172.21.15.16:0/3936441843 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc46c06ace0 0x7fc46c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:37.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.277+0000 7fc48e01e700 1 --2- 172.21.15.16:0/3936441843 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc46c06ace0 0x7fc46c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:37.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.277+0000 7fc47effd700 1 -- 172.21.15.16:0/3936441843 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 7102+0+0 (secure 0 0 0) 0x7fc484089510 con 0x7fc48806d4b0 2023-05-20T01:19:37.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.277+0000 7fc48e01e700 1 --2- 172.21.15.16:0/3936441843 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc46c06ace0 0x7fc46c06d190 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fc4780090b0 tx=0x7fc478008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:37.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.281+0000 7fc47effd700 1 -- 172.21.15.16:0/3936441843 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fc4840541c0 con 0x7fc48806d4b0 2023-05-20T01:19:37.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.526+0000 7fc490282700 1 -- 172.21.15.16:0/3936441843 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc4881f3490 con 0x7fc48806d4b0 2023-05-20T01:19:37.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.526+0000 7fc47effd700 1 -- 172.21.15.16:0/3936441843 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+4680 (secure 0 0 0) 0x7fc4840577e0 con 0x7fc48806d4b0 2023-05-20T01:19:37.531 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:19:37.531 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":29,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5246,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/2629811751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":2629811751},{"type":"v1","addr":"172.21.15.132:6834","nonce":2629811751}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":5249,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2117148777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":2117148777},{"type":"v1","addr":"172.21.15.132:6835","nonce":2117148777}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":5566,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2025052254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":2025052254},{"type":"v1","addr":"172.21.15.16:6837","nonce":2025052254}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-20T01:18:26.227827+0000","modified":"2023-05-20T01:18:35.503777+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5889},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5889":{"gid":5889,"name":"c","rank":0,"incarnation":26,"state":"up:active","state_seq":4,"addr":"172.21.15.16:6836/3284962389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3284962389},{"type":"v1","addr":"172.21.15.16:6836","nonce":3284962389}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7,8],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-05-20T01:19:37.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.528+0000 7fc490282700 1 -- 172.21.15.16:0/3936441843 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc46c06ace0 msgr2=0x7fc46c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:37.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.528+0000 7fc490282700 1 --2- 172.21.15.16:0/3936441843 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc46c06ace0 0x7fc46c06d190 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fc4780090b0 tx=0x7fc478008030 comp rx=0 tx=0).stop 2023-05-20T01:19:37.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.528+0000 7fc490282700 1 -- 172.21.15.16:0/3936441843 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc48806d4b0 msgr2=0x7fc4880c64d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:37.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.528+0000 7fc490282700 1 --2- 172.21.15.16:0/3936441843 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc48806d4b0 0x7fc4880c64d0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fc48400e810 tx=0x7fc48400ebd0 comp rx=0 tx=0).stop 2023-05-20T01:19:37.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.528+0000 7fc490282700 1 -- 172.21.15.16:0/3936441843 shutdown_connections 2023-05-20T01:19:37.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.528+0000 7fc490282700 1 --2- 172.21.15.16:0/3936441843 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc46c06ace0 0x7fc46c06d190 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:37.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.528+0000 7fc490282700 1 --2- 172.21.15.16:0/3936441843 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc48806f450 0x7fc4880c6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:37.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.528+0000 7fc490282700 1 --2- 172.21.15.16:0/3936441843 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc48806a510 0x7fc4880c8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:37.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.528+0000 7fc490282700 1 --2- 172.21.15.16:0/3936441843 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc48806d4b0 0x7fc4880c64d0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:37.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.529+0000 7fc490282700 1 -- 172.21.15.16:0/3936441843 >> 172.21.15.16:0/3936441843 conn(0x7fc4880cffb0 msgr2=0x7fc4880cee90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:19:37.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.529+0000 7fc490282700 1 -- 172.21.15.16:0/3936441843 shutdown_connections 2023-05-20T01:19:37.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.529+0000 7fc490282700 1 -- 172.21.15.16:0/3936441843 wait complete. 2023-05-20T01:19:37.535 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 29 2023-05-20T01:19:37.545 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-20T01:19:37.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.666+0000 7fb84fc37700 1 Processor -- start 2023-05-20T01:19:37.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.666+0000 7fb84fc37700 1 -- start start 2023-05-20T01:19:37.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.667+0000 7fb84fc37700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb84806a510 0x7fb84815e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:37.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.667+0000 7fb84fc37700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb848071340 0x7fb8480717c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:37.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.667+0000 7fb84fc37700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb84815e060 con 0x7fb84806a8e0 2023-05-20T01:19:37.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.667+0000 7fb84fc37700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb84815d450 con 0x7fb848071340 2023-05-20T01:19:37.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.667+0000 7fb84fc37700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb84815d730 con 0x7fb84806a510 2023-05-20T01:19:37.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.667+0000 7fb84e1d4700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb848071340 0x7fb8480717c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:37.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.667+0000 7fb84e1d4700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb848071340 0x7fb8480717c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33406/0 (socket says 172.21.15.16:33406) 2023-05-20T01:19:37.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.667+0000 7fb84e1d4700 1 -- 172.21.15.16:0/2885244948 learned_addr learned my addr 172.21.15.16:0/2885244948 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:19:37.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.667+0000 7fb84d9d3700 1 --2- 172.21.15.16:0/2885244948 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb84806a510 0x7fb84815e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:37.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.667+0000 7fb84e1d4700 1 -- 172.21.15.16:0/2885244948 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb84806a510 msgr2=0x7fb84815e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:19:37.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.667+0000 7fb84e1d4700 1 --2- 172.21.15.16:0/2885244948 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb84806a510 0x7fb84815e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:37.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.667+0000 7fb84e1d4700 1 -- 172.21.15.16:0/2885244948 >> v1:172.21.15.132:6789/0 conn(0x7fb84806a8e0 legacy=0x7fb8480cf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:19:37.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.667+0000 7fb84e1d4700 1 -- 172.21.15.16:0/2885244948 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb84815da10 con 0x7fb848071340 2023-05-20T01:19:37.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.667+0000 7fb84d9d3700 1 --2- 172.21.15.16:0/2885244948 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb84806a510 0x7fb84815e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:19:37.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.667+0000 7fb84e1d4700 1 --2- 172.21.15.16:0/2885244948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb848071340 0x7fb8480717c0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fb840009e50 tx=0x7fb84000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e165d070413d650 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:37.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.668+0000 7fb84c9d1700 1 -- 172.21.15.16:0/2885244948 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb84000e070 con 0x7fb848071340 2023-05-20T01:19:37.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.668+0000 7fb84c9d1700 1 -- 172.21.15.16:0/2885244948 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb840003a20 con 0x7fb848071340 2023-05-20T01:19:37.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.668+0000 7fb84c9d1700 1 -- 172.21.15.16:0/2885244948 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb840003dd0 con 0x7fb848071340 2023-05-20T01:19:37.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.668+0000 7fb84fc37700 1 -- 172.21.15.16:0/2885244948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb848071340 msgr2=0x7fb8480717c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:37.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.668+0000 7fb84fc37700 1 --2- 172.21.15.16:0/2885244948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb848071340 0x7fb8480717c0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fb840009e50 tx=0x7fb84000b330 comp rx=0 tx=0).stop 2023-05-20T01:19:37.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.668+0000 7fb84fc37700 1 -- 172.21.15.16:0/2885244948 shutdown_connections 2023-05-20T01:19:37.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.668+0000 7fb84fc37700 1 --2- 172.21.15.16:0/2885244948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb848071340 0x7fb8480717c0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:37.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.668+0000 7fb84fc37700 1 --2- 172.21.15.16:0/2885244948 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb84806a510 0x7fb84815e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:37.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.668+0000 7fb84fc37700 1 -- 172.21.15.16:0/2885244948 >> 172.21.15.16:0/2885244948 conn(0x7fb8480d0c30 msgr2=0x7fb8480b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:19:37.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.668+0000 7fb84fc37700 1 -- 172.21.15.16:0/2885244948 shutdown_connections 2023-05-20T01:19:37.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.668+0000 7fb84fc37700 1 -- 172.21.15.16:0/2885244948 wait complete. 2023-05-20T01:19:37.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.669+0000 7fb84fc37700 1 Processor -- start 2023-05-20T01:19:37.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.670+0000 7fb84fc37700 1 -- start start 2023-05-20T01:19:37.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.670+0000 7fb84fc37700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb84806a510 0x7fb8480b3e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:37.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.670+0000 7fb84fc37700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb84806a8e0 0x7fb8480c6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:37.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.670+0000 7fb84fc37700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb848071340 0x7fb848140fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:37.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.670+0000 7fb84fc37700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8480b48e0 con 0x7fb84806a510 2023-05-20T01:19:37.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.670+0000 7fb84fc37700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8480b4bc0 con 0x7fb848071340 2023-05-20T01:19:37.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.670+0000 7fb84fc37700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8480b4ea0 con 0x7fb84806a8e0 2023-05-20T01:19:37.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.670+0000 7fb84d9d3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb84806a510 0x7fb8480b3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:37.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.670+0000 7fb84d9d3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb84806a510 0x7fb8480b3e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33408/0 (socket says 172.21.15.16:33408) 2023-05-20T01:19:37.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.670+0000 7fb84d9d3700 1 -- 172.21.15.16:0/1026213766 learned_addr learned my addr 172.21.15.16:0/1026213766 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:19:37.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.670+0000 7fb84e1d4700 1 --2- 172.21.15.16:0/1026213766 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb848071340 0x7fb848140fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:37.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.670+0000 7fb84d1d2700 1 --2- 172.21.15.16:0/1026213766 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb84806a8e0 0x7fb8480c6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:37.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.671+0000 7fb84d9d3700 1 -- 172.21.15.16:0/1026213766 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb84806a8e0 msgr2=0x7fb8480c6280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:37.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.671+0000 7fb84d9d3700 1 --2- 172.21.15.16:0/1026213766 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb84806a8e0 0x7fb8480c6280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:37.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.671+0000 7fb84d9d3700 1 -- 172.21.15.16:0/1026213766 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb848071340 msgr2=0x7fb848140fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:37.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.671+0000 7fb84d9d3700 1 --2- 172.21.15.16:0/1026213766 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb848071340 0x7fb848140fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:37.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.671+0000 7fb84d9d3700 1 -- 172.21.15.16:0/1026213766 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb840009b30 con 0x7fb84806a510 2023-05-20T01:19:37.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.671+0000 7fb84e1d4700 1 --2- 172.21.15.16:0/1026213766 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb848071340 0x7fb848140fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:19:37.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.671+0000 7fb84d9d3700 1 --2- 172.21.15.16:0/1026213766 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb84806a510 0x7fb8480b3e90 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fb84400c750 tx=0x7fb84400cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:37.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.671+0000 7fb83e7fc700 1 -- 172.21.15.16:0/1026213766 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb844011800 con 0x7fb84806a510 2023-05-20T01:19:37.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.671+0000 7fb83e7fc700 1 -- 172.21.15.16:0/1026213766 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb844011960 con 0x7fb84806a510 2023-05-20T01:19:37.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.671+0000 7fb84fc37700 1 -- 172.21.15.16:0/1026213766 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8480cfd00 con 0x7fb84806a510 2023-05-20T01:19:37.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.671+0000 7fb83e7fc700 1 -- 172.21.15.16:0/1026213766 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb844011cd0 con 0x7fb84806a510 2023-05-20T01:19:37.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.671+0000 7fb84fc37700 1 -- 172.21.15.16:0/1026213766 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb84815dcf0 con 0x7fb84806a510 2023-05-20T01:19:37.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.672+0000 7fb84fc37700 1 -- 172.21.15.16:0/1026213766 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb84806ad30 con 0x7fb84806a510 2023-05-20T01:19:37.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.672+0000 7fb83e7fc700 1 -- 172.21.15.16:0/1026213766 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb844021350 con 0x7fb84806a510 2023-05-20T01:19:37.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.673+0000 7fb83e7fc700 1 --2- 172.21.15.16:0/1026213766 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb82c06ace0 0x7fb82c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:37.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.673+0000 7fb83e7fc700 1 -- 172.21.15.16:0/1026213766 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 7102+0+0 (secure 0 0 0) 0x7fb844089830 con 0x7fb84806a510 2023-05-20T01:19:37.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.673+0000 7fb84d1d2700 1 --2- 172.21.15.16:0/1026213766 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb82c06ace0 0x7fb82c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:37.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.673+0000 7fb84d1d2700 1 --2- 172.21.15.16:0/1026213766 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb82c06ace0 0x7fb82c06d190 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fb8481f58c0 tx=0x7fb838009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:37.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.677+0000 7fb83e7fc700 1 -- 172.21.15.16:0/1026213766 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fb844057f60 con 0x7fb84806a510 2023-05-20T01:19:37.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.920+0000 7fb84fc37700 1 -- 172.21.15.16:0/1026213766 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb8480747b0 con 0x7fb84806a510 2023-05-20T01:19:37.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.920+0000 7fb83e7fc700 1 -- 172.21.15.16:0/1026213766 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+4680 (secure 0 0 0) 0x7fb844057af0 con 0x7fb84806a510 2023-05-20T01:19:37.924 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:19:37.924 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":29,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5246,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/2629811751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":2629811751},{"type":"v1","addr":"172.21.15.132:6834","nonce":2629811751}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":5249,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2117148777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":2117148777},{"type":"v1","addr":"172.21.15.132:6835","nonce":2117148777}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":5566,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2025052254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":2025052254},{"type":"v1","addr":"172.21.15.16:6837","nonce":2025052254}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-20T01:18:26.227827+0000","modified":"2023-05-20T01:18:35.503777+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5889},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5889":{"gid":5889,"name":"c","rank":0,"incarnation":26,"state":"up:active","state_seq":4,"addr":"172.21.15.16:6836/3284962389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3284962389},{"type":"v1","addr":"172.21.15.16:6836","nonce":3284962389}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7,8],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-05-20T01:19:37.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.922+0000 7fb84fc37700 1 -- 172.21.15.16:0/1026213766 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb82c06ace0 msgr2=0x7fb82c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:37.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.922+0000 7fb84fc37700 1 --2- 172.21.15.16:0/1026213766 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb82c06ace0 0x7fb82c06d190 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fb8481f58c0 tx=0x7fb838009040 comp rx=0 tx=0).stop 2023-05-20T01:19:37.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.922+0000 7fb84fc37700 1 -- 172.21.15.16:0/1026213766 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb84806a510 msgr2=0x7fb8480b3e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:37.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.922+0000 7fb84fc37700 1 --2- 172.21.15.16:0/1026213766 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb84806a510 0x7fb8480b3e90 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fb84400c750 tx=0x7fb84400cb10 comp rx=0 tx=0).stop 2023-05-20T01:19:37.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.922+0000 7fb84fc37700 1 -- 172.21.15.16:0/1026213766 shutdown_connections 2023-05-20T01:19:37.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.922+0000 7fb84fc37700 1 --2- 172.21.15.16:0/1026213766 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb848071340 0x7fb848140fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:37.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.922+0000 7fb84fc37700 1 --2- 172.21.15.16:0/1026213766 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb84806a510 0x7fb8480b3e90 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:37.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.922+0000 7fb84fc37700 1 --2- 172.21.15.16:0/1026213766 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb82c06ace0 0x7fb82c06d190 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:37.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.922+0000 7fb84fc37700 1 --2- 172.21.15.16:0/1026213766 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb84806a8e0 0x7fb8480c6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:37.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.922+0000 7fb84fc37700 1 -- 172.21.15.16:0/1026213766 >> 172.21.15.16:0/1026213766 conn(0x7fb8480d0c30 msgr2=0x7fb84806ed40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:19:37.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.922+0000 7fb84fc37700 1 -- 172.21.15.16:0/1026213766 shutdown_connections 2023-05-20T01:19:37.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:37.923+0000 7fb84fc37700 1 -- 172.21.15.16:0/1026213766 wait complete. 2023-05-20T01:19:37.928 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 29 2023-05-20T01:19:37.938 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-20T01:19:38.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.059+0000 7f28d4ec5700 1 Processor -- start 2023-05-20T01:19:38.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.059+0000 7f28d4ec5700 1 -- start start 2023-05-20T01:19:38.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.060+0000 7f28d4ec5700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f28d0157190 0x7f28d00cc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:38.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.060+0000 7f28d4ec5700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f28d0071ad0 0x7f28d00ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:38.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.060+0000 7f28d4ec5700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28d00cb590 con 0x7f28d00cccf0 2023-05-20T01:19:38.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.060+0000 7f28d4ec5700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28d0071f50 con 0x7f28d0157190 2023-05-20T01:19:38.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.060+0000 7f28d4ec5700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28d00ca9a0 con 0x7f28d0071ad0 2023-05-20T01:19:38.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.060+0000 7f28ced9d700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f28d0071ad0 0x7f28d00ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:38.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.060+0000 7f28ce59c700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f28d0157190 0x7f28d00cc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:38.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.060+0000 7f28ced9d700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f28d0071ad0 0x7f28d00ce740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.16:33458/0 (socket says 172.21.15.16:33458) 2023-05-20T01:19:38.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.060+0000 7f28cdd9b700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f28d00cccf0 0x7f28d00713c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.16:54658/0 (socket says 172.21.15.16:54658) 2023-05-20T01:19:38.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.060+0000 7f28ced9d700 1 -- 172.21.15.16:0/255105897 learned_addr learned my addr 172.21.15.16:0/255105897 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:19:38.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.061+0000 7f28ced9d700 1 -- 172.21.15.16:0/255105897 >> v2:172.21.15.132:3300/0 conn(0x7f28d0157190 msgr2=0x7f28d00cc7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:19:38.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.061+0000 7f28ced9d700 1 --2- 172.21.15.16:0/255105897 >> v2:172.21.15.132:3300/0 conn(0x7f28d0157190 0x7f28d00cc7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:38.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.061+0000 7f28ced9d700 1 -- 172.21.15.16:0/255105897 >> v1:172.21.15.132:6789/0 conn(0x7f28d00cccf0 legacy=0x7f28d00713c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:38.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.061+0000 7f28ced9d700 1 -- 172.21.15.16:0/255105897 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28d00cac60 con 0x7f28d0071ad0 2023-05-20T01:19:38.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.061+0000 7f28ce59c700 1 --2- 172.21.15.16:0/255105897 >> v2:172.21.15.132:3300/0 conn(0x7f28d0157190 0x7f28d00cc7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-05-20T01:19:38.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.061+0000 7f28ced9d700 1 --2- 172.21.15.16:0/255105897 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f28d0071ad0 0x7f28d00ce740 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f28c8009e50 tx=0x7f28c800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=493ad9c7c8755be1 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:38.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.061+0000 7f28cd59a700 1 -- 172.21.15.16:0/255105897 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c800e070 con 0x7f28d0071ad0 2023-05-20T01:19:38.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.061+0000 7f28cd59a700 1 -- 172.21.15.16:0/255105897 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28c800b910 con 0x7f28d0071ad0 2023-05-20T01:19:38.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.061+0000 7f28cd59a700 1 -- 172.21.15.16:0/255105897 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c800bcc0 con 0x7f28d0071ad0 2023-05-20T01:19:38.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.062+0000 7f28d4ec5700 1 -- 172.21.15.16:0/255105897 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f28d0071ad0 msgr2=0x7f28d00ce740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:38.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.062+0000 7f28d4ec5700 1 --2- 172.21.15.16:0/255105897 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f28d0071ad0 0x7f28d00ce740 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f28c8009e50 tx=0x7f28c800b330 comp rx=0 tx=0).stop 2023-05-20T01:19:38.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.062+0000 7f28d4ec5700 1 -- 172.21.15.16:0/255105897 shutdown_connections 2023-05-20T01:19:38.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.062+0000 7f28d4ec5700 1 --2- 172.21.15.16:0/255105897 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f28d0071ad0 0x7f28d00ce740 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:38.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.062+0000 7f28d4ec5700 1 --2- 172.21.15.16:0/255105897 >> v2:172.21.15.132:3300/0 conn(0x7f28d0157190 0x7f28d00cc7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:38.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.062+0000 7f28d4ec5700 1 -- 172.21.15.16:0/255105897 >> 172.21.15.16:0/255105897 conn(0x7f28d00cffb0 msgr2=0x7f28d00d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:19:38.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.062+0000 7f28d4ec5700 1 -- 172.21.15.16:0/255105897 shutdown_connections 2023-05-20T01:19:38.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.062+0000 7f28d4ec5700 1 -- 172.21.15.16:0/255105897 wait complete. 2023-05-20T01:19:38.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.063+0000 7f28d4ec5700 1 Processor -- start 2023-05-20T01:19:38.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.063+0000 7f28d4ec5700 1 -- start start 2023-05-20T01:19:38.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.064+0000 7f28d4ec5700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f28d0071ad0 0x7f28d01f3420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:38.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.064+0000 7f28d4ec5700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f28d00cccf0 0x7f28d00adf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:38.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.064+0000 7f28d4ec5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28d0157190 0x7f28d00b38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:38.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.064+0000 7f28d4ec5700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28d00b5d50 con 0x7f28d0157190 2023-05-20T01:19:38.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.064+0000 7f28d4ec5700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28d00ae480 con 0x7f28d0071ad0 2023-05-20T01:19:38.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.064+0000 7f28d4ec5700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28d00ae700 con 0x7f28d00cccf0 2023-05-20T01:19:38.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.064+0000 7f28ced9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28d0157190 0x7f28d00b38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:38.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.064+0000 7f28ced9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28d0157190 0x7f28d00b38d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33426/0 (socket says 172.21.15.16:33426) 2023-05-20T01:19:38.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.064+0000 7f28ced9d700 1 -- 172.21.15.16:0/4167159259 learned_addr learned my addr 172.21.15.16:0/4167159259 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:19:38.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.064+0000 7f28ce59c700 1 --2- 172.21.15.16:0/4167159259 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f28d0071ad0 0x7f28d01f3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:38.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.064+0000 7f28cdd9b700 1 --2- 172.21.15.16:0/4167159259 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f28d00cccf0 0x7f28d00adf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:38.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.064+0000 7f28ced9d700 1 -- 172.21.15.16:0/4167159259 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f28d00cccf0 msgr2=0x7f28d00adf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:38.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.064+0000 7f28ced9d700 1 --2- 172.21.15.16:0/4167159259 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f28d00cccf0 0x7f28d00adf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:38.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.064+0000 7f28ced9d700 1 -- 172.21.15.16:0/4167159259 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f28d0071ad0 msgr2=0x7f28d01f3420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:38.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.064+0000 7f28ced9d700 1 --2- 172.21.15.16:0/4167159259 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f28d0071ad0 0x7f28d01f3420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:38.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.064+0000 7f28ced9d700 1 -- 172.21.15.16:0/4167159259 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28c40094a0 con 0x7f28d0157190 2023-05-20T01:19:38.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.064+0000 7f28cdd9b700 1 --2- 172.21.15.16:0/4167159259 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f28d00cccf0 0x7f28d00adf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:19:38.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.065+0000 7f28ced9d700 1 --2- 172.21.15.16:0/4167159259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28d0157190 0x7f28d00b38d0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f28d0072090 tx=0x7f28c8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:38.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.065+0000 7f28beffd700 1 -- 172.21.15.16:0/4167159259 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c80048b0 con 0x7f28d0157190 2023-05-20T01:19:38.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.065+0000 7f28beffd700 1 -- 172.21.15.16:0/4167159259 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28c80099b0 con 0x7f28d0157190 2023-05-20T01:19:38.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.065+0000 7f28beffd700 1 -- 172.21.15.16:0/4167159259 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c801c680 con 0x7f28d0157190 2023-05-20T01:19:38.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.065+0000 7f28d4ec5700 1 -- 172.21.15.16:0/4167159259 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28c8009b30 con 0x7f28d0157190 2023-05-20T01:19:38.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.065+0000 7f28d4ec5700 1 -- 172.21.15.16:0/4167159259 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28d00cb220 con 0x7f28d0157190 2023-05-20T01:19:38.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.066+0000 7f28d4ec5700 1 -- 172.21.15.16:0/4167159259 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28d00caf40 con 0x7f28d0157190 2023-05-20T01:19:38.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.066+0000 7f28beffd700 1 -- 172.21.15.16:0/4167159259 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f28c801c7e0 con 0x7f28d0157190 2023-05-20T01:19:38.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.067+0000 7f28beffd700 1 --2- 172.21.15.16:0/4167159259 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f28b006ace0 0x7f28b006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:38.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.067+0000 7f28beffd700 1 -- 172.21.15.16:0/4167159259 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 7102+0+0 (secure 0 0 0) 0x7f28c808b4c0 con 0x7f28d0157190 2023-05-20T01:19:38.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.067+0000 7f28ce59c700 1 --2- 172.21.15.16:0/4167159259 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f28b006ace0 0x7f28b006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:38.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.067+0000 7f28ce59c700 1 --2- 172.21.15.16:0/4167159259 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f28b006ace0 0x7f28b006d190 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f28d00c39f0 tx=0x7f28c4009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:38.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.071+0000 7f28beffd700 1 -- 172.21.15.16:0/4167159259 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f28c80561f0 con 0x7f28d0157190 2023-05-20T01:19:38.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.270+0000 7f28d4ec5700 1 -- 172.21.15.16:0/4167159259 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f28d0157560 con 0x7f28d0157190 2023-05-20T01:19:38.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.271+0000 7f28beffd700 1 -- 172.21.15.16:0/4167159259 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) v1 ==== 74+0+20668 (secure 0 0 0) 0x7f28c8059810 con 0x7f28d0157190 2023-05-20T01:19:38.275 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:19:38.276 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":51,"fsid":"c2245cc5-ecab-4087-a59b-bc81db901676","created":"2023-05-20T01:16:09.549230+0000","modified":"2023-05-20T01:18:31.674809+0000","last_up_change":"2023-05-20T01:16:19.788624+0000","last_in_change":"2023-05-20T01:16:12.092160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-05-20T01:16:20.934366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-05-20T01:16:21.372294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-05-20T01:17:44.628755+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-05-20T01:17:45.968584+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data_extra","create_time":"2023-05-20T01:17:54.915781+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"317a83cd-19c8-4958-a301-6087ef342289","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6825","nonce":3923634179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6827","nonce":3923634179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6831","nonce":3923634179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6829","nonce":3923634179}]},"public_addr":"172.21.15.16:6825/3923634179","cluster_addr":"172.21.15.16:6827/3923634179","heartbeat_back_addr":"172.21.15.16:6831/3923634179","heartbeat_front_addr":"172.21.15.16:6829/3923634179","state":["exists","up"]},{"osd":1,"uuid":"5581046f-8219-4089-8d7d-b7846399505e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6809","nonce":3860264220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6811","nonce":3860264220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6815","nonce":3860264220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6813","nonce":3860264220}]},"public_addr":"172.21.15.16:6809/3860264220","cluster_addr":"172.21.15.16:6811/3860264220","heartbeat_back_addr":"172.21.15.16:6815/3860264220","heartbeat_front_addr":"172.21.15.16:6813/3860264220","state":["exists","up"]},{"osd":2,"uuid":"072e9e66-4133-478e-8d4a-64d46c2963f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6817","nonce":293187030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6819","nonce":293187030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6823","nonce":293187030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6821","nonce":293187030}]},"public_addr":"172.21.15.16:6817/293187030","cluster_addr":"172.21.15.16:6819/293187030","heartbeat_back_addr":"172.21.15.16:6823/293187030","heartbeat_front_addr":"172.21.15.16:6821/293187030","state":["exists","up"]},{"osd":3,"uuid":"11e1bba6-00f8-48dd-9028-d8d716b7f145","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6801","nonce":3669781890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6803","nonce":3669781890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6807","nonce":3669781890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6805","nonce":3669781890}]},"public_addr":"172.21.15.16:6801/3669781890","cluster_addr":"172.21.15.16:6803/3669781890","heartbeat_back_addr":"172.21.15.16:6807/3669781890","heartbeat_front_addr":"172.21.15.16:6805/3669781890","state":["exists","up"]},{"osd":4,"uuid":"b8909c3a-cf70-46cb-8d0b-009c707cf9d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6819","nonce":506317223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6823","nonce":506317223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6829","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6831","nonce":506317223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6827","nonce":506317223}]},"public_addr":"172.21.15.132:6819/506317223","cluster_addr":"172.21.15.132:6823/506317223","heartbeat_back_addr":"172.21.15.132:6831/506317223","heartbeat_front_addr":"172.21.15.132:6827/506317223","state":["exists","up"]},{"osd":5,"uuid":"b1818304-4583-47bf-a6ec-0bfbb5166233","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6818","nonce":1844689995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6822","nonce":1844689995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6830","nonce":1844689995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6826","nonce":1844689995}]},"public_addr":"172.21.15.132:6818/1844689995","cluster_addr":"172.21.15.132:6822/1844689995","heartbeat_back_addr":"172.21.15.132:6830/1844689995","heartbeat_front_addr":"172.21.15.132:6826/1844689995","state":["exists","up"]},{"osd":6,"uuid":"d37f4691-eed1-4a93-8818-18354f03f4a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6801","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6802","nonce":727581123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6805","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6807","nonce":727581123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6813","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6815","nonce":727581123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6811","nonce":727581123}]},"public_addr":"172.21.15.132:6802/727581123","cluster_addr":"172.21.15.132:6807/727581123","heartbeat_back_addr":"172.21.15.132:6815/727581123","heartbeat_front_addr":"172.21.15.132:6811/727581123","state":["exists","up"]},{"osd":7,"uuid":"4f712f62-5d1d-4816-8042-03209144f8b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6803","nonce":466001149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6806","nonce":466001149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6814","nonce":466001149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6810","nonce":466001149}]},"public_addr":"172.21.15.132:6803/466001149","cluster_addr":"172.21.15.132:6806/466001149","heartbeat_back_addr":"172.21.15.132:6814/466001149","heartbeat_front_addr":"172.21.15.132:6810/466001149","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.338863+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.278680+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305694+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.246835+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.276237+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.304441+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305421+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.267050+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.9","mappings":[{"from":0,"to":4}]},{"pgid":"7.1b","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6837/1867776973":"2023-05-21T01:18:31.666399+0000","172.21.15.16:6835/1867776973":"2023-05-21T01:18:31.666399+0000","172.21.15.16:6837/3517262366":"2023-05-21T01:18:26.467438+0000","172.21.15.16:6835/3517262366":"2023-05-21T01:18:26.467438+0000","172.21.15.16:6835/942011026":"2023-05-21T01:18:24.644805+0000","172.21.15.16:6837/942011026":"2023-05-21T01:18:24.644805+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-20T01:19:38.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.273+0000 7f28d4ec5700 1 -- 172.21.15.16:0/4167159259 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f28b006ace0 msgr2=0x7f28b006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:38.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.273+0000 7f28d4ec5700 1 --2- 172.21.15.16:0/4167159259 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f28b006ace0 0x7f28b006d190 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f28d00c39f0 tx=0x7f28c4009210 comp rx=0 tx=0).stop 2023-05-20T01:19:38.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.273+0000 7f28d4ec5700 1 -- 172.21.15.16:0/4167159259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28d0157190 msgr2=0x7f28d00b38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:38.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.273+0000 7f28d4ec5700 1 --2- 172.21.15.16:0/4167159259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28d0157190 0x7f28d00b38d0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f28d0072090 tx=0x7f28c8004ad0 comp rx=0 tx=0).stop 2023-05-20T01:19:38.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.273+0000 7f28d4ec5700 1 -- 172.21.15.16:0/4167159259 shutdown_connections 2023-05-20T01:19:38.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.273+0000 7f28d4ec5700 1 --2- 172.21.15.16:0/4167159259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28d0157190 0x7f28d00b38d0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:38.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.273+0000 7f28d4ec5700 1 --2- 172.21.15.16:0/4167159259 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f28d0071ad0 0x7f28d01f3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:38.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.273+0000 7f28d4ec5700 1 --2- 172.21.15.16:0/4167159259 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f28b006ace0 0x7f28b006d190 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:38.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.273+0000 7f28d4ec5700 1 --2- 172.21.15.16:0/4167159259 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f28d00cccf0 0x7f28d00adf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:38.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.273+0000 7f28d4ec5700 1 -- 172.21.15.16:0/4167159259 >> 172.21.15.16:0/4167159259 conn(0x7f28d00cffb0 msgr2=0x7f28d015ad40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:19:38.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.273+0000 7f28d4ec5700 1 -- 172.21.15.16:0/4167159259 shutdown_connections 2023-05-20T01:19:38.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.273+0000 7f28d4ec5700 1 -- 172.21.15.16:0/4167159259 wait complete. 2023-05-20T01:19:38.290 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-05-20T01:19:38.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.411+0000 7f4c61405700 1 Processor -- start 2023-05-20T01:19:38.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.411+0000 7f4c61405700 1 -- start start 2023-05-20T01:19:38.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.412+0000 7f4c61405700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4c5c0cfc50 0x7f4c5c0d2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:38.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.412+0000 7f4c61405700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4c5c0b4750 0x7f4c5c0d0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:38.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.412+0000 7f4c61405700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f4c5c0b2b40 0x7f4c5c0b2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:38.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.412+0000 7f4c61405700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c5c0cfad0 con 0x7f4c5c0b4750 2023-05-20T01:19:38.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.412+0000 7f4c61405700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c5c0d3df0 con 0x7f4c5c0b2b40 2023-05-20T01:19:38.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.412+0000 7f4c61405700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c5c0d0b90 con 0x7f4c5c0cfc50 2023-05-20T01:19:38.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.412+0000 7f4c5a7fc700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4c5c0b4750 0x7f4c5c0d0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:38.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.412+0000 7f4c5a7fc700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4c5c0b4750 0x7f4c5c0d0020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33430/0 (socket says 172.21.15.16:33430) 2023-05-20T01:19:38.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.412+0000 7f4c5a7fc700 1 -- 172.21.15.16:0/2039262485 learned_addr learned my addr 172.21.15.16:0/2039262485 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:19:38.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.412+0000 7f4c5affd700 1 --2- 172.21.15.16:0/2039262485 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4c5c0cfc50 0x7f4c5c0d2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:38.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.412+0000 7f4c5b7fe700 1 --2- 172.21.15.16:0/2039262485 >> v2:172.21.15.132:3300/0 conn(0x7f4c5c0b2b40 0x7f4c5c0b2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:38.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.413+0000 7f4c5a7fc700 1 -- 172.21.15.16:0/2039262485 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4c5c0cfc50 msgr2=0x7f4c5c0d2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:19:38.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.413+0000 7f4c5a7fc700 1 --2- 172.21.15.16:0/2039262485 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4c5c0cfc50 0x7f4c5c0d2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:38.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.413+0000 7f4c5a7fc700 1 -- 172.21.15.16:0/2039262485 >> v2:172.21.15.132:3300/0 conn(0x7f4c5c0b2b40 msgr2=0x7f4c5c0b2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:19:38.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.413+0000 7f4c5a7fc700 1 --2- 172.21.15.16:0/2039262485 >> v2:172.21.15.132:3300/0 conn(0x7f4c5c0b2b40 0x7f4c5c0b2fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:38.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.413+0000 7f4c5a7fc700 1 -- 172.21.15.16:0/2039262485 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c5c0d0560 con 0x7f4c5c0b4750 2023-05-20T01:19:38.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.413+0000 7f4c5b7fe700 1 --2- 172.21.15.16:0/2039262485 >> v2:172.21.15.132:3300/0 conn(0x7f4c5c0b2b40 0x7f4c5c0b2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:19:38.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.413+0000 7f4c5a7fc700 1 --2- 172.21.15.16:0/2039262485 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c5c0b4750 0x7f4c5c0d0020 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c0b5db0 tx=0x7f4c4c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51e658478bc375c6 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:38.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.413+0000 7f4c59ffb700 1 -- 172.21.15.16:0/2039262485 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c4c011830 con 0x7f4c5c0b4750 2023-05-20T01:19:38.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.413+0000 7f4c59ffb700 1 -- 172.21.15.16:0/2039262485 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c4c011990 con 0x7f4c5c0b4750 2023-05-20T01:19:38.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.413+0000 7f4c59ffb700 1 -- 172.21.15.16:0/2039262485 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c4c011d20 con 0x7f4c5c0b4750 2023-05-20T01:19:38.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.413+0000 7f4c61405700 1 -- 172.21.15.16:0/2039262485 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c5c0b4750 msgr2=0x7f4c5c0d0020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:38.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.413+0000 7f4c61405700 1 --2- 172.21.15.16:0/2039262485 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c5c0b4750 0x7f4c5c0d0020 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c0b5db0 tx=0x7f4c4c009bd0 comp rx=0 tx=0).stop 2023-05-20T01:19:38.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.414+0000 7f4c61405700 1 -- 172.21.15.16:0/2039262485 shutdown_connections 2023-05-20T01:19:38.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.414+0000 7f4c61405700 1 --2- 172.21.15.16:0/2039262485 >> v2:172.21.15.132:3300/0 conn(0x7f4c5c0b2b40 0x7f4c5c0b2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:38.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.414+0000 7f4c61405700 1 --2- 172.21.15.16:0/2039262485 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4c5c0cfc50 0x7f4c5c0d2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:38.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.414+0000 7f4c61405700 1 --2- 172.21.15.16:0/2039262485 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c5c0b4750 0x7f4c5c0d0020 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:38.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.414+0000 7f4c61405700 1 -- 172.21.15.16:0/2039262485 >> 172.21.15.16:0/2039262485 conn(0x7f4c5c09c590 msgr2=0x7f4c5c0d3240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:19:38.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.414+0000 7f4c61405700 1 -- 172.21.15.16:0/2039262485 shutdown_connections 2023-05-20T01:19:38.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.414+0000 7f4c61405700 1 -- 172.21.15.16:0/2039262485 wait complete. 2023-05-20T01:19:38.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.415+0000 7f4c61405700 1 Processor -- start 2023-05-20T01:19:38.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.415+0000 7f4c61405700 1 -- start start 2023-05-20T01:19:38.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.416+0000 7f4c61405700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4c5c0b2b40 0x7f4c5c1f6520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:38.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.416+0000 7f4c61405700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4c5c0b4750 0x7f4c5c1f1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:38.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.416+0000 7f4c61405700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c5c0cfc50 0x7f4c5c06c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:38.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.416+0000 7f4c61405700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c5c0b7610 con 0x7f4c5c0cfc50 2023-05-20T01:19:38.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.416+0000 7f4c61405700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c5c1f6a60 con 0x7f4c5c0b4750 2023-05-20T01:19:38.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.416+0000 7f4c61405700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c5c0cb7b0 con 0x7f4c5c0b2b40 2023-05-20T01:19:38.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.416+0000 7f4c5b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c5c0cfc50 0x7f4c5c06c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:38.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.416+0000 7f4c5b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c5c0cfc50 0x7f4c5c06c130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33438/0 (socket says 172.21.15.16:33438) 2023-05-20T01:19:38.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.416+0000 7f4c5b7fe700 1 -- 172.21.15.16:0/3617627936 learned_addr learned my addr 172.21.15.16:0/3617627936 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:19:38.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.416+0000 7f4c5affd700 1 --2- 172.21.15.16:0/3617627936 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4c5c0b2b40 0x7f4c5c1f6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:38.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.416+0000 7f4c5a7fc700 1 --2- 172.21.15.16:0/3617627936 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4c5c0b4750 0x7f4c5c1f1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:38.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.416+0000 7f4c5b7fe700 1 -- 172.21.15.16:0/3617627936 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4c5c0b2b40 msgr2=0x7f4c5c1f6520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:38.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.416+0000 7f4c5b7fe700 1 --2- 172.21.15.16:0/3617627936 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4c5c0b2b40 0x7f4c5c1f6520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:38.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.416+0000 7f4c5b7fe700 1 -- 172.21.15.16:0/3617627936 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4c5c0b4750 msgr2=0x7f4c5c1f1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:38.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.416+0000 7f4c5b7fe700 1 --2- 172.21.15.16:0/3617627936 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4c5c0b4750 0x7f4c5c1f1ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:38.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.416+0000 7f4c5b7fe700 1 -- 172.21.15.16:0/3617627936 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c4c009550 con 0x7f4c5c0cfc50 2023-05-20T01:19:38.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.416+0000 7f4c5a7fc700 1 --2- 172.21.15.16:0/3617627936 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4c5c0b4750 0x7f4c5c1f1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:19:38.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.416+0000 7f4c5b7fe700 1 --2- 172.21.15.16:0/3617627936 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c5c0cfc50 0x7f4c5c06c130 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f4c54007db0 tx=0x7f4c5400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:38.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.416+0000 7f4c437fe700 1 -- 172.21.15.16:0/3617627936 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c5400ddd0 con 0x7f4c5c0cfc50 2023-05-20T01:19:38.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.416+0000 7f4c437fe700 1 -- 172.21.15.16:0/3617627936 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c5400f040 con 0x7f4c5c0cfc50 2023-05-20T01:19:38.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.417+0000 7f4c61405700 1 -- 172.21.15.16:0/3617627936 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c5c0b4380 con 0x7f4c5c0cfc50 2023-05-20T01:19:38.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.417+0000 7f4c437fe700 1 -- 172.21.15.16:0/3617627936 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c54003c60 con 0x7f4c5c0cfc50 2023-05-20T01:19:38.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.417+0000 7f4c61405700 1 -- 172.21.15.16:0/3617627936 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c5c15d0b0 con 0x7f4c5c0cfc50 2023-05-20T01:19:38.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.418+0000 7f4c61405700 1 -- 172.21.15.16:0/3617627936 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c5c0d4000 con 0x7f4c5c0cfc50 2023-05-20T01:19:38.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.418+0000 7f4c437fe700 1 -- 172.21.15.16:0/3617627936 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4c54004030 con 0x7f4c5c0cfc50 2023-05-20T01:19:38.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.418+0000 7f4c437fe700 1 --2- 172.21.15.16:0/3617627936 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f4c4806ace0 0x7f4c4806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:38.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.419+0000 7f4c437fe700 1 -- 172.21.15.16:0/3617627936 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 7102+0+0 (secure 0 0 0) 0x7f4c54089870 con 0x7f4c5c0cfc50 2023-05-20T01:19:38.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.419+0000 7f4c5affd700 1 --2- 172.21.15.16:0/3617627936 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f4c4806ace0 0x7f4c4806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:38.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.419+0000 7f4c5affd700 1 --2- 172.21.15.16:0/3617627936 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f4c4806ace0 0x7f4c4806d190 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c06aa70 tx=0x7f4c50009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:38.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.423+0000 7f4c437fe700 1 -- 172.21.15.16:0/3617627936 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f4c540545a0 con 0x7f4c5c0cfc50 2023-05-20T01:19:38.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:38.668+0000 7f4c61405700 1 -- 172.21.15.16:0/3617627936 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f4c5c0b7750 con 0x7f4c5c0cfc50 2023-05-20T01:19:39.435 INFO:tasks.ceph.mon.a.smithi016.stderr:2023-05-20T01:19:39.431+0000 7f49eb8ae700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-05-20T01:19:39.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:39.439+0000 7f4c437fe700 1 -- 172.21.15.16:0/3617627936 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v30) v1 ==== 161+0+0 (secure 0 0 0) 0x7f4c54057bc0 con 0x7f4c5c0cfc50 2023-05-20T01:19:39.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:39.442+0000 7f4c61405700 1 -- 172.21.15.16:0/3617627936 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f4c4806ace0 msgr2=0x7f4c4806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:39.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:39.442+0000 7f4c61405700 1 --2- 172.21.15.16:0/3617627936 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f4c4806ace0 0x7f4c4806d190 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c06aa70 tx=0x7f4c50009210 comp rx=0 tx=0).stop 2023-05-20T01:19:39.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:39.442+0000 7f4c61405700 1 -- 172.21.15.16:0/3617627936 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c5c0cfc50 msgr2=0x7f4c5c06c130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:39.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:39.442+0000 7f4c61405700 1 --2- 172.21.15.16:0/3617627936 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c5c0cfc50 0x7f4c5c06c130 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f4c54007db0 tx=0x7f4c5400d3b0 comp rx=0 tx=0).stop 2023-05-20T01:19:39.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:39.442+0000 7f4c61405700 1 -- 172.21.15.16:0/3617627936 shutdown_connections 2023-05-20T01:19:39.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:39.442+0000 7f4c61405700 1 --2- 172.21.15.16:0/3617627936 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c5c0cfc50 0x7f4c5c06c130 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:39.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:39.442+0000 7f4c61405700 1 --2- 172.21.15.16:0/3617627936 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f4c4806ace0 0x7f4c4806d190 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:39.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:39.442+0000 7f4c61405700 1 --2- 172.21.15.16:0/3617627936 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4c5c0b2b40 0x7f4c5c1f6520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:39.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:39.442+0000 7f4c61405700 1 --2- 172.21.15.16:0/3617627936 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4c5c0b4750 0x7f4c5c1f1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:39.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:39.442+0000 7f4c61405700 1 -- 172.21.15.16:0/3617627936 >> 172.21.15.16:0/3617627936 conn(0x7f4c5c09c590 msgr2=0x7f4c5c0cdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:19:39.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:39.442+0000 7f4c61405700 1 -- 172.21.15.16:0/3617627936 shutdown_connections 2023-05-20T01:19:39.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:39.442+0000 7f4c61405700 1 -- 172.21.15.16:0/3617627936 wait complete. 2023-05-20T01:19:39.448 INFO:teuthology.orchestra.run.smithi016.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-05-20T01:19:39.448 INFO:tasks.ceph.mds.c.smithi016.stderr: -8481> 2023-05-20T01:18:32.494+0000 7f44af702700 -1 mds.0.openfiles _load_finish got (2) No such file or directory 2023-05-20T01:19:39.449 INFO:tasks.ceph.mds.c.smithi016.stderr: -7004> 2023-05-20T01:18:37.493+0000 7f44b2f09700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-05-20T01:19:39.453 INFO:tasks.ceph.mds.c.smithi016.stderr: -4463> 2023-05-20T01:18:37.996+0000 7f44af702700 -1 log_channel(cluster) log [ERR] : bad backtrace on directory inode 0x10000000000 2023-05-20T01:19:39.454 INFO:tasks.ceph.mds.c.smithi016.stderr: -4327> 2023-05-20T01:18:37.996+0000 7f44af702700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /subdir/ auth v1 pv4 ap=3 DIRTYPARENT f(v0 1=1+0) n() (inest lock dirty) (ifile lock->sync w=1 flushing) (iversion lock) | dirtyscattered=2 lock=1 dirfrag=1 dirtyparent=1 waiter=1 authpin=1 scrubqueue=0 0x555dabc65600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(6)0x10000000000:[<0x1/subdir v1>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 1=1+0)","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 1=1+0)","memory_value.rstat":"n()","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-61} 2023-05-20T01:19:39.454 INFO:tasks.ceph.mds.c.smithi016.stderr: -4190> 2023-05-20T01:18:37.997+0000 7f44af702700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001f7 [2,head] /subdir/2 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x555dabd5a580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(8)0x100000001f7:[<0x10000000000/2 v46>,<0x1/subdir v56>]//[7]","memoryvalue":"(8)0x100000001f7:[<0x10000000000/2 v1>,<0x1/subdir v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-05-20T01:19:39.454 INFO:tasks.ceph.mds.c.smithi016.stderr: -4167> 2023-05-20T01:18:37.997+0000 7f44af702700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001f9 [2,head] /subdir/4 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x555dabcb3700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(8)0x100000001f9:[<0x10000000000/4 v50>,<0x1/subdir v56>]//[7]","memoryvalue":"(8)0x100000001f9:[<0x10000000000/4 v1>,<0x1/subdir v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-05-20T01:19:39.455 INFO:tasks.ceph.mds.c.smithi016.stderr: -4151> 2023-05-20T01:18:37.997+0000 7f44af702700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001f8 [2,head] /subdir/3 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x555dabd5a000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(8)0x100000001f8:[<0x10000000000/3 v48>,<0x1/subdir v56>]//[7]","memoryvalue":"(8)0x100000001f8:[<0x10000000000/3 v1>,<0x1/subdir v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-05-20T01:19:39.455 INFO:tasks.ceph.mds.c.smithi016.stderr: -4135> 2023-05-20T01:18:37.998+0000 7f44af702700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001f6 [2,head] /subdir/1 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x555dabd5ab00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(8)0x100000001f6:[<0x10000000000/1 v44>,<0x1/subdir v56>]//[7]","memoryvalue":"(8)0x100000001f6:[<0x10000000000/1 v1>,<0x1/subdir v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-05-20T01:19:39.455 INFO:tasks.ceph.mds.c.smithi016.stderr: -4117> 2023-05-20T01:18:37.998+0000 7f44af702700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /subdir/0 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x555dabd5b080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(8)0x10000000001:[<0x10000000000/0 v52>,<0x1/subdir v56>]//[7]","memoryvalue":"(8)0x10000000001:[<0x10000000000/0 v1>,<0x1/subdir v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-05-20T01:19:39.456 INFO:tasks.ceph.mds.c.smithi016.stderr: -3837> 2023-05-20T01:18:38.003+0000 7f44af702700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1 [...2,head] / auth v5 ap=5 snaprealm=0x555dabc5efc0 DIRTYPARENT f(v1 1=0+1) n(v0 rc2023-05-20T01:18:37.648539+0000 1=0+1)/n() (idft sync r=1) (inest sync r=1) (ifile sync->mix r=1) (iversion lock) caps={5949=pLsXs/-@45} | dirtyscattered=0 request=0 lock=3 dirfrag=1 caps=1 openingsnapparents=0 dirtyparent=1 dirty=1 waiter=1 authpin=1 scrubqueue=0 0x555dabc65080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(6)0x1:[]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 1=0+1)","ondisk_value.rstat":"n(v0 2=0+2)","memory_value.dirstat":"f(v1 1=0+1)","memory_value.rstat":"n(v0 rc2023-05-20T01:18:37.648539+0000 1=0+1)","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-61} 2023-05-20T01:19:39.462 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-05-20T01:19:39.486 INFO:tasks.ceph.mds.c.smithi016.stdout:starting mds.c at 2023-05-20T01:19:39.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:39.594+0000 7fc48dd53700 1 Processor -- start 2023-05-20T01:19:39.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:39.595+0000 7fc48dd53700 1 -- start start 2023-05-20T01:19:39.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:39.595+0000 7fc48dd53700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fc4880cd070 0x7fc48806e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:39.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:39.595+0000 7fc48dd53700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc488072430 con 0x7fc48815cf40 2023-05-20T01:19:39.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:39.595+0000 7fc48dd53700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc488071820 con 0x7fc4880710d0 2023-05-20T01:19:39.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:39.595+0000 7fc48dd53700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc488071b00 con 0x7fc4880cd070 2023-05-20T01:19:39.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:39.595+0000 7fc4877fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fc48815cf40 0x7fc4880cc960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:44982/0 (socket says 172.21.15.16:44982) 2023-05-20T01:19:39.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:39.595+0000 7fc4877fe700 1 -- 172.21.15.16:0/408079272 learned_addr learned my addr 172.21.15.16:0/408079272 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:19:39.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:39.596+0000 7fc486ffd700 1 --2- 172.21.15.16:0/408079272 >> v2:172.21.15.132:3300/0 conn(0x7fc4880cd070 0x7fc48806e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:39.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:39.596+0000 7fc4867fc700 1 -- 172.21.15.16:0/408079272 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 76876592 0 0) 0x7fc488072430 con 0x7fc48815cf40 2023-05-20T01:19:39.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:39.596+0000 7fc4867fc700 1 -- 172.21.15.16:0/408079272 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc4700035b0 con 0x7fc48815cf40 2023-05-20T01:19:39.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:39.596+0000 7fc486ffd700 1 -- 172.21.15.16:0/408079272 >> v1:172.21.15.132:6789/0 conn(0x7fc4880710d0 legacy=0x7fc4880cabb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:39.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:39.596+0000 7fc4867fc700 1 -- 172.21.15.16:0/408079272 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 83892484 0 0) 0x7fc4700035b0 con 0x7fc48815cf40 2023-05-20T01:19:39.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:39.596+0000 7fc486ffd700 1 -- 172.21.15.16:0/408079272 >> v1:172.21.15.16:6789/0 conn(0x7fc48815cf40 legacy=0x7fc4880cc960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:39.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:39.596+0000 7fc486ffd700 1 -- 172.21.15.16:0/408079272 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc488071de0 con 0x7fc4880cd070 2023-05-20T01:19:39.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:39.596+0000 7fc486ffd700 1 --2- 172.21.15.16:0/408079272 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4880cd070 0x7fc48806e020 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fc47c009840 tx=0x7fc47c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb37317edc4b18f0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:39.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:39.596+0000 7fc4867fc700 1 -- 172.21.15.16:0/408079272 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc478001670 con 0x7fc4880cd070 2023-05-20T01:19:39.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:39.596+0000 7fc4867fc700 1 -- 172.21.15.16:0/408079272 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc47c004030 con 0x7fc4880cd070 2023-05-20T01:19:39.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:39.596+0000 7fc4867fc700 1 -- 172.21.15.16:0/408079272 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc47c0043c0 con 0x7fc4880cd070 2023-05-20T01:19:39.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:39.597+0000 7fc48dd53700 1 -- 172.21.15.16:0/408079272 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4880cd070 msgr2=0x7fc48806e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:39.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:39.597+0000 7fc48dd53700 1 --2- 172.21.15.16:0/408079272 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4880cd070 0x7fc48806e020 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fc47c009840 tx=0x7fc47c009b50 comp rx=0 tx=0).stop 2023-05-20T01:19:39.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:39.597+0000 7fc48dd53700 1 -- 172.21.15.16:0/408079272 shutdown_connections 2023-05-20T01:19:39.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:39.597+0000 7fc48dd53700 1 --2- 172.21.15.16:0/408079272 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4880cd070 0x7fc48806e020 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:39.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:39.597+0000 7fc48dd53700 1 -- 172.21.15.16:0/408079272 >> 172.21.15.16:0/408079272 conn(0x7fc48806c260 msgr2=0x7fc4880cffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:19:39.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:39.597+0000 7fc48dd53700 1 -- 172.21.15.16:0/408079272 shutdown_connections 2023-05-20T01:19:39.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:39.597+0000 7fc48dd53700 1 -- 172.21.15.16:0/408079272 wait complete. 2023-05-20T01:19:39.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:39.598+0000 7fc48dd53700 1 Processor -- start 2023-05-20T01:19:39.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:39.598+0000 7fc48dd53700 1 -- start start 2023-05-20T01:19:39.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:39.599+0000 7fc48dd53700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4880710d0 0x7fc4881f1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:39.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:39.599+0000 7fc48dd53700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc4880cd070 0x7fc48814e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:39.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:39.599+0000 7fc48dd53700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc48815cf40 0x7fc4880c70a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:39.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:39.599+0000 7fc48dd53700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4880b0350 con 0x7fc4880710d0 2023-05-20T01:19:39.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:39.599+0000 7fc48dd53700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4880af7b0 con 0x7fc48815cf40 2023-05-20T01:19:39.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:39.599+0000 7fc48dd53700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc48814eb90 con 0x7fc4880cd070 2023-05-20T01:19:39.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:39.599+0000 7fc4877fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4880710d0 0x7fc4881f1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:39.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:39.599+0000 7fc4877fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4880710d0 0x7fc4881f1fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33460/0 (socket says 172.21.15.16:33460) 2023-05-20T01:19:39.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:39.599+0000 7fc4877fe700 1 -- 172.21.15.16:0/2672804075 learned_addr learned my addr 172.21.15.16:0/2672804075 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:19:39.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:39.599+0000 7fc486ffd700 1 --2- 172.21.15.16:0/2672804075 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc4880cd070 0x7fc48814e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:39.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:39.599+0000 7fc487fff700 1 --2- 172.21.15.16:0/2672804075 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc48815cf40 0x7fc4880c70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:39.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:39.600+0000 7fc4877fe700 1 -- 172.21.15.16:0/2672804075 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc4880cd070 msgr2=0x7fc48814e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:39.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:39.600+0000 7fc4877fe700 1 --2- 172.21.15.16:0/2672804075 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc4880cd070 0x7fc48814e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:39.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:39.600+0000 7fc4877fe700 1 -- 172.21.15.16:0/2672804075 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc48815cf40 msgr2=0x7fc4880c70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:39.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:39.600+0000 7fc4877fe700 1 --2- 172.21.15.16:0/2672804075 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc48815cf40 0x7fc4880c70a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:39.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:39.600+0000 7fc4877fe700 1 -- 172.21.15.16:0/2672804075 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc47c0094a0 con 0x7fc4880710d0 2023-05-20T01:19:39.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:39.600+0000 7fc4877fe700 1 --2- 172.21.15.16:0/2672804075 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4880710d0 0x7fc4881f1fe0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fc47800c700 tx=0x7fc47800cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:39.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:39.600+0000 7fc46ffff700 1 -- 172.21.15.16:0/2672804075 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc478004c80 con 0x7fc4880710d0 2023-05-20T01:19:39.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:39.600+0000 7fc46ffff700 1 -- 172.21.15.16:0/2672804075 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc478004de0 con 0x7fc4880710d0 2023-05-20T01:19:39.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:39.600+0000 7fc48dd53700 1 -- 172.21.15.16:0/2672804075 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc488071510 con 0x7fc4880710d0 2023-05-20T01:19:39.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:39.600+0000 7fc46ffff700 1 -- 172.21.15.16:0/2672804075 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc478007310 con 0x7fc4880710d0 2023-05-20T01:19:39.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:39.600+0000 7fc48dd53700 1 -- 172.21.15.16:0/2672804075 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4880720c0 con 0x7fc4880710d0 2023-05-20T01:19:39.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:39.601+0000 7fc48dd53700 1 -- 172.21.15.16:0/2672804075 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4880cb2c0 con 0x7fc4880710d0 2023-05-20T01:19:39.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:39.601+0000 7fc46ffff700 1 -- 172.21.15.16:0/2672804075 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc478007470 con 0x7fc4880710d0 2023-05-20T01:19:39.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:39.602+0000 7fc46ffff700 1 --2- 172.21.15.16:0/2672804075 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc47406ace0 0x7fc47406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:39.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:39.602+0000 7fc46ffff700 1 -- 172.21.15.16:0/2672804075 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 7188+0+0 (secure 0 0 0) 0x7fc478089660 con 0x7fc4880710d0 2023-05-20T01:19:39.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:39.602+0000 7fc486ffd700 1 --2- 172.21.15.16:0/2672804075 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc47406ace0 0x7fc47406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:39.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:39.602+0000 7fc486ffd700 1 --2- 172.21.15.16:0/2672804075 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc47406ace0 0x7fc47406d190 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fc47c009840 tx=0x7fc47c008000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:39.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:39.606+0000 7fc46ffff700 1 -- 172.21.15.16:0/2672804075 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fc478054330 con 0x7fc4880710d0 2023-05-20T01:19:39.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:39.854+0000 7fc48dd53700 1 -- 172.21.15.16:0/2672804075 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fc4880af350 con 0x7fc4880710d0 2023-05-20T01:19:40.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:40.440+0000 7fc46ffff700 1 -- 172.21.15.16:0/2672804075 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v31) v1 ==== 104+0+0 (secure 0 0 0) 0x7fc478057950 con 0x7fc4880710d0 2023-05-20T01:19:40.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:40.444+0000 7fc48dd53700 1 -- 172.21.15.16:0/2672804075 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc47406ace0 msgr2=0x7fc47406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:40.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:40.444+0000 7fc48dd53700 1 --2- 172.21.15.16:0/2672804075 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc47406ace0 0x7fc47406d190 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fc47c009840 tx=0x7fc47c008000 comp rx=0 tx=0).stop 2023-05-20T01:19:40.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:40.444+0000 7fc48dd53700 1 -- 172.21.15.16:0/2672804075 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4880710d0 msgr2=0x7fc4881f1fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:40.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:40.444+0000 7fc48dd53700 1 --2- 172.21.15.16:0/2672804075 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4880710d0 0x7fc4881f1fe0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fc47800c700 tx=0x7fc47800cac0 comp rx=0 tx=0).stop 2023-05-20T01:19:40.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:40.444+0000 7fc48dd53700 1 -- 172.21.15.16:0/2672804075 shutdown_connections 2023-05-20T01:19:40.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:40.444+0000 7fc48dd53700 1 --2- 172.21.15.16:0/2672804075 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc48815cf40 0x7fc4880c70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:40.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:40.444+0000 7fc48dd53700 1 --2- 172.21.15.16:0/2672804075 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4880710d0 0x7fc4881f1fe0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:40.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:40.444+0000 7fc48dd53700 1 --2- 172.21.15.16:0/2672804075 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc47406ace0 0x7fc47406d190 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:40.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:40.444+0000 7fc48dd53700 1 --2- 172.21.15.16:0/2672804075 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc4880cd070 0x7fc48814e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:40.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:40.444+0000 7fc48dd53700 1 -- 172.21.15.16:0/2672804075 >> 172.21.15.16:0/2672804075 conn(0x7fc48806c260 msgr2=0x7fc4880ce060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:19:40.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:40.444+0000 7fc48dd53700 1 -- 172.21.15.16:0/2672804075 shutdown_connections 2023-05-20T01:19:40.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:40.444+0000 7fc48dd53700 1 -- 172.21.15.16:0/2672804075 wait complete. 2023-05-20T01:19:40.463 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-05-20T01:19:40.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:40.636+0000 7f53d3fef700 1 Processor -- start 2023-05-20T01:19:40.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:40.637+0000 7f53d3fef700 1 -- start start 2023-05-20T01:19:40.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:40.637+0000 7f53d3fef700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f53cc06aad0 0x7f53cc06dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:40.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:40.637+0000 7f53d3fef700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53cc161440 con 0x7f53cc06aea0 2023-05-20T01:19:40.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:40.637+0000 7f53d3fef700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53cc0b28c0 con 0x7f53cc0b22a0 2023-05-20T01:19:40.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:40.637+0000 7f53d3fef700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53cc160680 con 0x7f53cc06aad0 2023-05-20T01:19:40.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:40.637+0000 7f53d158a700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f53cc06aea0 0x7f53cc0b1b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:45000/0 (socket says 172.21.15.16:45000) 2023-05-20T01:19:40.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:40.638+0000 7f53d158a700 1 -- 172.21.15.16:0/1187425658 learned_addr learned my addr 172.21.15.16:0/1187425658 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:19:40.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:40.638+0000 7f53d1d8b700 1 --2- 172.21.15.16:0/1187425658 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f53cc06aad0 0x7f53cc06dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:40.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:40.638+0000 7f53d0d89700 1 -- 172.21.15.16:0/1187425658 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2697173260 0 0) 0x7f53cc161440 con 0x7f53cc06aea0 2023-05-20T01:19:40.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:40.638+0000 7f53d0d89700 1 -- 172.21.15.16:0/1187425658 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53b80035b0 con 0x7f53cc06aea0 2023-05-20T01:19:40.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:40.638+0000 7f53d0d89700 1 -- 172.21.15.16:0/1187425658 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2702820401 0 0) 0x7f53cc0b28c0 con 0x7f53cc0b22a0 2023-05-20T01:19:40.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:40.638+0000 7f53d0d89700 1 -- 172.21.15.16:0/1187425658 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53cc161440 con 0x7f53cc0b22a0 2023-05-20T01:19:40.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:40.638+0000 7f53d0d89700 1 -- 172.21.15.16:0/1187425658 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 492520755 0 0) 0x7f53b80035b0 con 0x7f53cc06aea0 2023-05-20T01:19:40.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:40.638+0000 7f53d0d89700 1 -- 172.21.15.16:0/1187425658 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f53cc06aad0 msgr2=0x7f53cc06dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:19:40.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:40.638+0000 7f53d0d89700 1 --2- 172.21.15.16:0/1187425658 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f53cc06aad0 0x7f53cc06dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:40.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:40.638+0000 7f53d0d89700 1 -- 172.21.15.16:0/1187425658 >> v1:172.21.15.132:6789/0 conn(0x7f53cc0b22a0 legacy=0x7f53cc06cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:40.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:40.638+0000 7f53d0d89700 1 -- 172.21.15.16:0/1187425658 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53cc160960 con 0x7f53cc06aea0 2023-05-20T01:19:40.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:40.638+0000 7f53d0d89700 1 -- 172.21.15.16:0/1187425658 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7f53c8001670 con 0x7f53cc06aea0 2023-05-20T01:19:40.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:40.638+0000 7f53d1d8b700 1 --2- 172.21.15.16:0/1187425658 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f53cc06aad0 0x7f53cc06dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-05-20T01:19:40.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:40.638+0000 7f53d0d89700 1 -- 172.21.15.16:0/1187425658 >> v1:172.21.15.16:6789/0 conn(0x7f53cc06aea0 legacy=0x7f53cc0b1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:40.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:40.638+0000 7f53d0d89700 1 --2- 172.21.15.16:0/1187425658 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f53b8003bc0 0x7f53b8003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:40.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:40.638+0000 7f53d0d89700 1 --2- 172.21.15.16:0/1187425658 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f53cc06aad0 0x7f53b8004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:40.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:40.638+0000 7f53d0d89700 1 --2- 172.21.15.16:0/1187425658 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53b8004d90 0x7f53b8005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:40.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:40.638+0000 7f53d0d89700 1 -- 172.21.15.16:0/1187425658 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53cc160680 con 0x7f53b8004d90 2023-05-20T01:19:40.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:40.638+0000 7f53d0d89700 1 -- 172.21.15.16:0/1187425658 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53b800bac0 con 0x7f53b8003bc0 2023-05-20T01:19:40.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:40.638+0000 7f53d0d89700 1 -- 172.21.15.16:0/1187425658 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53b800bc00 con 0x7f53cc06aad0 2023-05-20T01:19:40.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:40.638+0000 7f53d258c700 1 --2- 172.21.15.16:0/1187425658 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53b8004d90 0x7f53b8005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:40.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:40.638+0000 7f53d158a700 1 --2- 172.21.15.16:0/1187425658 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f53cc06aad0 0x7f53b8004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:40.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:40.638+0000 7f53d1d8b700 1 --2- 172.21.15.16:0/1187425658 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f53b8003bc0 0x7f53b8003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:40.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:40.639+0000 7f53d258c700 1 -- 172.21.15.16:0/1187425658 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f53cc06aad0 msgr2=0x7f53b8004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:19:40.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:40.639+0000 7f53d258c700 1 --2- 172.21.15.16:0/1187425658 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f53cc06aad0 0x7f53b8004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:40.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:40.639+0000 7f53d258c700 1 -- 172.21.15.16:0/1187425658 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f53b8003bc0 msgr2=0x7f53b8003fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:19:40.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:40.639+0000 7f53d258c700 1 --2- 172.21.15.16:0/1187425658 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f53b8003bc0 0x7f53b8003fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:40.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:40.639+0000 7f53d258c700 1 -- 172.21.15.16:0/1187425658 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53bc0094a0 con 0x7f53b8004d90 2023-05-20T01:19:40.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:40.639+0000 7f53d258c700 1 --2- 172.21.15.16:0/1187425658 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53b8004d90 0x7f53b8005200 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f53c400c2f0 tx=0x7f53b8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12961d95308820c9 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:40.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:40.639+0000 7f53d0d89700 1 -- 172.21.15.16:0/1187425658 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c4001f30 con 0x7f53b8004d90 2023-05-20T01:19:40.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:40.639+0000 7f53d0d89700 1 -- 172.21.15.16:0/1187425658 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53c400ce90 con 0x7f53b8004d90 2023-05-20T01:19:40.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:40.639+0000 7f53d0d89700 1 -- 172.21.15.16:0/1187425658 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c4003c40 con 0x7f53b8004d90 2023-05-20T01:19:40.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:40.639+0000 7f53d3fef700 1 -- 172.21.15.16:0/1187425658 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53b8004d90 msgr2=0x7f53b8005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:40.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:40.639+0000 7f53d3fef700 1 --2- 172.21.15.16:0/1187425658 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53b8004d90 0x7f53b8005200 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f53c400c2f0 tx=0x7f53b8005880 comp rx=0 tx=0).stop 2023-05-20T01:19:40.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:40.639+0000 7f53d258c700 1 -- 172.21.15.16:0/1187425658 reap_dead start 2023-05-20T01:19:40.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:40.639+0000 7f53d3fef700 1 -- 172.21.15.16:0/1187425658 shutdown_connections 2023-05-20T01:19:40.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:40.639+0000 7f53d3fef700 1 -- 172.21.15.16:0/1187425658 >> 172.21.15.16:0/1187425658 conn(0x7f53cc0ccab0 msgr2=0x7f53cc0cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:19:40.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:40.640+0000 7f53d3fef700 1 -- 172.21.15.16:0/1187425658 shutdown_connections 2023-05-20T01:19:40.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:40.640+0000 7f53d3fef700 1 -- 172.21.15.16:0/1187425658 wait complete. 2023-05-20T01:19:40.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:40.641+0000 7f53d3fef700 1 Processor -- start 2023-05-20T01:19:40.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:40.641+0000 7f53d3fef700 1 -- start start 2023-05-20T01:19:40.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:40.642+0000 7f53d3fef700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f53cc151e70 0x7f53cc1523e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:40.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:40.642+0000 7f53d3fef700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53cc0c3b60 0x7f53cc0c6020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:40.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:40.642+0000 7f53d3fef700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f53cc0c6560 0x7f53cc1f5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:40.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:40.642+0000 7f53d3fef700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53cc152240 con 0x7f53cc0c3b60 2023-05-20T01:19:40.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:40.642+0000 7f53d3fef700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53cc0c69d0 con 0x7f53cc151e70 2023-05-20T01:19:40.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:40.642+0000 7f53d3fef700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53cc156600 con 0x7f53cc0c6560 2023-05-20T01:19:40.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:40.642+0000 7f53d158a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53cc0c3b60 0x7f53cc0c6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:40.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:40.642+0000 7f53d158a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53cc0c3b60 0x7f53cc0c6020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33484/0 (socket says 172.21.15.16:33484) 2023-05-20T01:19:40.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:40.642+0000 7f53d158a700 1 -- 172.21.15.16:0/825769082 learned_addr learned my addr 172.21.15.16:0/825769082 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:19:40.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:40.642+0000 7f53d258c700 1 --2- 172.21.15.16:0/825769082 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f53cc0c6560 0x7f53cc1f5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:40.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:40.642+0000 7f53d1d8b700 1 --2- 172.21.15.16:0/825769082 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f53cc151e70 0x7f53cc1523e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:40.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:40.642+0000 7f53d158a700 1 -- 172.21.15.16:0/825769082 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f53cc0c6560 msgr2=0x7f53cc1f5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:40.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:40.642+0000 7f53d158a700 1 --2- 172.21.15.16:0/825769082 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f53cc0c6560 0x7f53cc1f5de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:40.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:40.642+0000 7f53d158a700 1 -- 172.21.15.16:0/825769082 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f53cc151e70 msgr2=0x7f53cc1523e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:40.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:40.642+0000 7f53d158a700 1 --2- 172.21.15.16:0/825769082 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f53cc151e70 0x7f53cc1523e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:40.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:40.642+0000 7f53d158a700 1 -- 172.21.15.16:0/825769082 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53cc160f20 con 0x7f53cc0c3b60 2023-05-20T01:19:40.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:40.642+0000 7f53d258c700 1 --2- 172.21.15.16:0/825769082 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f53cc0c6560 0x7f53cc1f5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:19:40.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:40.642+0000 7f53d1d8b700 1 --2- 172.21.15.16:0/825769082 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f53cc151e70 0x7f53cc1523e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:19:40.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:40.642+0000 7f53d158a700 1 --2- 172.21.15.16:0/825769082 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53cc0c3b60 0x7f53cc0c6020 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f53c8007310 tx=0x7f53c80076d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:40.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:40.642+0000 7f53c27fc700 1 -- 172.21.15.16:0/825769082 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c800f870 con 0x7f53cc0c3b60 2023-05-20T01:19:40.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:40.642+0000 7f53c27fc700 1 -- 172.21.15.16:0/825769082 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53c800f9d0 con 0x7f53cc0c3b60 2023-05-20T01:19:40.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:40.642+0000 7f53d3fef700 1 -- 172.21.15.16:0/825769082 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53cc160c40 con 0x7f53cc0c3b60 2023-05-20T01:19:40.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:40.642+0000 7f53c27fc700 1 -- 172.21.15.16:0/825769082 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c800fd80 con 0x7f53cc0c3b60 2023-05-20T01:19:40.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:40.643+0000 7f53d3fef700 1 -- 172.21.15.16:0/825769082 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53cc06d290 con 0x7f53cc0c3b60 2023-05-20T01:19:40.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:40.643+0000 7f53d3fef700 1 -- 172.21.15.16:0/825769082 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53cc0c5150 con 0x7f53cc0c3b60 2023-05-20T01:19:40.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:40.644+0000 7f53c27fc700 1 -- 172.21.15.16:0/825769082 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f53c801e3b0 con 0x7f53cc0c3b60 2023-05-20T01:19:40.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:40.644+0000 7f53c27fc700 1 --2- 172.21.15.16:0/825769082 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f53b006ace0 0x7f53b006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:40.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:40.644+0000 7f53c27fc700 1 -- 172.21.15.16:0/825769082 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 7188+0+0 (secure 0 0 0) 0x7f53c8089570 con 0x7f53cc0c3b60 2023-05-20T01:19:40.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:40.644+0000 7f53d1d8b700 1 --2- 172.21.15.16:0/825769082 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f53b006ace0 0x7f53b006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:40.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:40.645+0000 7f53d1d8b700 1 --2- 172.21.15.16:0/825769082 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f53b006ace0 0x7f53b006d190 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f53bc009df0 tx=0x7f53bc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:40.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:40.649+0000 7f53c27fc700 1 -- 172.21.15.16:0/825769082 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f53c80541b0 con 0x7f53cc0c3b60 2023-05-20T01:19:40.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:40.847+0000 7f53d3fef700 1 -- 172.21.15.16:0/825769082 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f53cc152ac0 con 0x7f53cc0c3b60 2023-05-20T01:19:41.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:41.445+0000 7f53c27fc700 1 -- 172.21.15.16:0/825769082 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v53) v1 ==== 181+0+0 (secure 0 0 0) 0x7f53c80577d0 con 0x7f53cc0c3b60 2023-05-20T01:19:41.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:41.449+0000 7f53d3fef700 1 -- 172.21.15.16:0/825769082 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f53b006ace0 msgr2=0x7f53b006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:41.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:41.449+0000 7f53d3fef700 1 --2- 172.21.15.16:0/825769082 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f53b006ace0 0x7f53b006d190 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f53bc009df0 tx=0x7f53bc008040 comp rx=0 tx=0).stop 2023-05-20T01:19:41.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:41.449+0000 7f53d3fef700 1 -- 172.21.15.16:0/825769082 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53cc0c3b60 msgr2=0x7f53cc0c6020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:41.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:41.449+0000 7f53d3fef700 1 --2- 172.21.15.16:0/825769082 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53cc0c3b60 0x7f53cc0c6020 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f53c8007310 tx=0x7f53c80076d0 comp rx=0 tx=0).stop 2023-05-20T01:19:41.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:41.449+0000 7f53d3fef700 1 -- 172.21.15.16:0/825769082 shutdown_connections 2023-05-20T01:19:41.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:41.449+0000 7f53d3fef700 1 --2- 172.21.15.16:0/825769082 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f53b006ace0 0x7f53b006d190 secure :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f53bc009df0 tx=0x7f53bc008040 comp rx=0 tx=0).stop 2023-05-20T01:19:41.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:41.449+0000 7f53d3fef700 1 --2- 172.21.15.16:0/825769082 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f53cc0c6560 0x7f53cc1f5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:41.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:41.449+0000 7f53d3fef700 1 --2- 172.21.15.16:0/825769082 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f53cc151e70 0x7f53cc1523e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:41.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:41.449+0000 7f53d3fef700 1 --2- 172.21.15.16:0/825769082 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53cc0c3b60 0x7f53cc0c6020 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:41.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:41.449+0000 7f53d3fef700 1 -- 172.21.15.16:0/825769082 >> 172.21.15.16:0/825769082 conn(0x7f53cc0ccab0 msgr2=0x7f53cc0b2e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:19:41.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:41.450+0000 7f53d3fef700 1 -- 172.21.15.16:0/825769082 shutdown_connections 2023-05-20T01:19:41.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:41.450+0000 7f53d3fef700 1 -- 172.21.15.16:0/825769082 wait complete. 2023-05-20T01:19:41.455 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' removed 2023-05-20T01:19:41.469 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-05-20T01:19:41.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:41.602+0000 7fd7d27b2700 1 Processor -- start 2023-05-20T01:19:41.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:41.602+0000 7fd7d27b2700 1 -- start start 2023-05-20T01:19:41.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:41.603+0000 7fd7d27b2700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd7cc06aad0 0x7fd7cc06dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:41.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:41.603+0000 7fd7d27b2700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7cc161440 con 0x7fd7cc06aea0 2023-05-20T01:19:41.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:41.603+0000 7fd7d27b2700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7cc0b28c0 con 0x7fd7cc0b22a0 2023-05-20T01:19:41.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:41.603+0000 7fd7d27b2700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7cc160680 con 0x7fd7cc06aad0 2023-05-20T01:19:41.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:41.603+0000 7fd7cb7fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fd7cc06aea0 0x7fd7cc0b1b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:45018/0 (socket says 172.21.15.16:45018) 2023-05-20T01:19:41.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:41.603+0000 7fd7cb7fe700 1 -- 172.21.15.16:0/3387059646 learned_addr learned my addr 172.21.15.16:0/3387059646 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:19:41.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:41.603+0000 7fd7cbfff700 1 --2- 172.21.15.16:0/3387059646 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd7cc06aad0 0x7fd7cc06dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:41.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:41.603+0000 7fd7caffd700 1 -- 172.21.15.16:0/3387059646 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 951951120 0 0) 0x7fd7cc161440 con 0x7fd7cc06aea0 2023-05-20T01:19:41.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:41.603+0000 7fd7caffd700 1 -- 172.21.15.16:0/3387059646 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7b80035b0 con 0x7fd7cc06aea0 2023-05-20T01:19:41.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:41.603+0000 7fd7caffd700 1 -- 172.21.15.16:0/3387059646 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 551660647 0 0) 0x7fd7cc0b28c0 con 0x7fd7cc0b22a0 2023-05-20T01:19:41.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:41.603+0000 7fd7caffd700 1 -- 172.21.15.16:0/3387059646 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7cc161440 con 0x7fd7cc0b22a0 2023-05-20T01:19:41.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:41.603+0000 7fd7cbfff700 1 -- 172.21.15.16:0/3387059646 >> v1:172.21.15.132:6789/0 conn(0x7fd7cc0b22a0 legacy=0x7fd7cc06cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:41.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:41.603+0000 7fd7caffd700 1 -- 172.21.15.16:0/3387059646 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 869004851 0 0) 0x7fd7b80035b0 con 0x7fd7cc06aea0 2023-05-20T01:19:41.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:41.603+0000 7fd7cbfff700 1 -- 172.21.15.16:0/3387059646 >> v1:172.21.15.16:6789/0 conn(0x7fd7cc06aea0 legacy=0x7fd7cc0b1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:41.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:41.603+0000 7fd7cbfff700 1 -- 172.21.15.16:0/3387059646 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7cc160960 con 0x7fd7cc06aad0 2023-05-20T01:19:41.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:41.604+0000 7fd7cbfff700 1 --2- 172.21.15.16:0/3387059646 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd7cc06aad0 0x7fd7cc06dca0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc009870 tx=0x7fd7bc009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da68ca5ac1da7fe server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:41.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:41.604+0000 7fd7caffd700 1 -- 172.21.15.16:0/3387059646 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b4001670 con 0x7fd7cc06aad0 2023-05-20T01:19:41.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:41.604+0000 7fd7caffd700 1 -- 172.21.15.16:0/3387059646 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7bc004030 con 0x7fd7cc06aad0 2023-05-20T01:19:41.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:41.604+0000 7fd7caffd700 1 -- 172.21.15.16:0/3387059646 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7bc0043c0 con 0x7fd7cc06aad0 2023-05-20T01:19:41.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:41.604+0000 7fd7d27b2700 1 -- 172.21.15.16:0/3387059646 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd7cc06aad0 msgr2=0x7fd7cc06dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:41.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:41.604+0000 7fd7d27b2700 1 --2- 172.21.15.16:0/3387059646 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd7cc06aad0 0x7fd7cc06dca0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc009870 tx=0x7fd7bc009b80 comp rx=0 tx=0).stop 2023-05-20T01:19:41.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:41.604+0000 7fd7d27b2700 1 -- 172.21.15.16:0/3387059646 shutdown_connections 2023-05-20T01:19:41.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:41.604+0000 7fd7d27b2700 1 --2- 172.21.15.16:0/3387059646 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd7cc06aad0 0x7fd7cc06dca0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:41.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:41.604+0000 7fd7d27b2700 1 -- 172.21.15.16:0/3387059646 >> 172.21.15.16:0/3387059646 conn(0x7fd7cc0ccab0 msgr2=0x7fd7cc0cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:19:41.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:41.604+0000 7fd7d27b2700 1 -- 172.21.15.16:0/3387059646 shutdown_connections 2023-05-20T01:19:41.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:41.605+0000 7fd7d27b2700 1 -- 172.21.15.16:0/3387059646 wait complete. 2023-05-20T01:19:41.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:41.606+0000 7fd7d27b2700 1 Processor -- start 2023-05-20T01:19:41.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:41.606+0000 7fd7d27b2700 1 -- start start 2023-05-20T01:19:41.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:41.607+0000 7fd7d27b2700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd7cc06aad0 0x7fd7cc150920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:41.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:41.607+0000 7fd7d27b2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7cc06aea0 0x7fd7cc0b3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:41.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:41.607+0000 7fd7d27b2700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd7cc0b22a0 0x7fd7cc0c7100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:41.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:41.607+0000 7fd7d27b2700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7cc0c4f40 con 0x7fd7cc06aea0 2023-05-20T01:19:41.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:41.607+0000 7fd7d27b2700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7cc0c5220 con 0x7fd7cc0b22a0 2023-05-20T01:19:41.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:41.607+0000 7fd7d27b2700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7cc0c5500 con 0x7fd7cc06aad0 2023-05-20T01:19:41.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:41.607+0000 7fd7cb7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7cc06aea0 0x7fd7cc0b3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:41.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:41.607+0000 7fd7cb7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7cc06aea0 0x7fd7cc0b3fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33496/0 (socket says 172.21.15.16:33496) 2023-05-20T01:19:41.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:41.607+0000 7fd7cb7fe700 1 -- 172.21.15.16:0/2286603248 learned_addr learned my addr 172.21.15.16:0/2286603248 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:19:41.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:41.607+0000 7fd7d0d4f700 1 --2- 172.21.15.16:0/2286603248 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd7cc0b22a0 0x7fd7cc0c7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:41.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:41.607+0000 7fd7cbfff700 1 --2- 172.21.15.16:0/2286603248 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd7cc06aad0 0x7fd7cc150920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:41.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:41.607+0000 7fd7cb7fe700 1 -- 172.21.15.16:0/2286603248 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd7cc06aad0 msgr2=0x7fd7cc150920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:41.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:41.607+0000 7fd7cb7fe700 1 --2- 172.21.15.16:0/2286603248 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd7cc06aad0 0x7fd7cc150920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:41.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:41.607+0000 7fd7cb7fe700 1 -- 172.21.15.16:0/2286603248 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd7cc0b22a0 msgr2=0x7fd7cc0c7100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:41.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:41.607+0000 7fd7cb7fe700 1 --2- 172.21.15.16:0/2286603248 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd7cc0b22a0 0x7fd7cc0c7100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:41.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:41.607+0000 7fd7cb7fe700 1 -- 172.21.15.16:0/2286603248 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7bc0094a0 con 0x7fd7cc06aea0 2023-05-20T01:19:41.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:41.607+0000 7fd7cbfff700 1 --2- 172.21.15.16:0/2286603248 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd7cc06aad0 0x7fd7cc150920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:19:41.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:41.607+0000 7fd7d0d4f700 1 --2- 172.21.15.16:0/2286603248 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd7cc0b22a0 0x7fd7cc0c7100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:19:41.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:41.607+0000 7fd7cb7fe700 1 --2- 172.21.15.16:0/2286603248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7cc06aea0 0x7fd7cc0b3fa0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fd7b400c770 tx=0x7fd7b400cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:41.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:41.607+0000 7fd7c8ff9700 1 -- 172.21.15.16:0/2286603248 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b4007560 con 0x7fd7cc06aea0 2023-05-20T01:19:41.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:41.607+0000 7fd7c8ff9700 1 -- 172.21.15.16:0/2286603248 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7b40076c0 con 0x7fd7cc06aea0 2023-05-20T01:19:41.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:41.608+0000 7fd7d27b2700 1 -- 172.21.15.16:0/2286603248 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7cc160f20 con 0x7fd7cc06aea0 2023-05-20T01:19:41.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:41.608+0000 7fd7c8ff9700 1 -- 172.21.15.16:0/2286603248 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b4004a70 con 0x7fd7cc06aea0 2023-05-20T01:19:41.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:41.608+0000 7fd7d27b2700 1 -- 172.21.15.16:0/2286603248 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7cc160c40 con 0x7fd7cc06aea0 2023-05-20T01:19:41.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:41.609+0000 7fd7d27b2700 1 -- 172.21.15.16:0/2286603248 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7cc06e1e0 con 0x7fd7cc06aea0 2023-05-20T01:19:41.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:41.610+0000 7fd7c8ff9700 1 -- 172.21.15.16:0/2286603248 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd7b4004bd0 con 0x7fd7cc06aea0 2023-05-20T01:19:41.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:41.610+0000 7fd7c8ff9700 1 --2- 172.21.15.16:0/2286603248 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd7ac06ace0 0x7fd7ac06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:41.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:41.610+0000 7fd7c8ff9700 1 -- 172.21.15.16:0/2286603248 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6750+0+0 (secure 0 0 0) 0x7fd7b40889a0 con 0x7fd7cc06aea0 2023-05-20T01:19:41.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:41.611+0000 7fd7cbfff700 1 --2- 172.21.15.16:0/2286603248 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd7ac06ace0 0x7fd7ac06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:41.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:41.612+0000 7fd7cbfff700 1 --2- 172.21.15.16:0/2286603248 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd7ac06ace0 0x7fd7ac06d190 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc0d0420 tx=0x7fd7bc008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:41.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:41.618+0000 7fd7c8ff9700 1 -- 172.21.15.16:0/2286603248 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fd7b4053790 con 0x7fd7cc06aea0 2023-05-20T01:19:41.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:41.818+0000 7fd7d27b2700 1 -- 172.21.15.16:0/2286603248 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd7cc0747b0 con 0x7fd7cc06aea0 2023-05-20T01:19:42.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:42.448+0000 7fd7c8ff9700 1 -- 172.21.15.16:0/2286603248 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v54) v1 ==== 169+0+0 (secure 0 0 0) 0x7fd7b4056db0 con 0x7fd7cc06aea0 2023-05-20T01:19:42.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:42.453+0000 7fd7d27b2700 1 -- 172.21.15.16:0/2286603248 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd7ac06ace0 msgr2=0x7fd7ac06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:42.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:42.453+0000 7fd7d27b2700 1 --2- 172.21.15.16:0/2286603248 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd7ac06ace0 0x7fd7ac06d190 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc0d0420 tx=0x7fd7bc008030 comp rx=0 tx=0).stop 2023-05-20T01:19:42.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:42.453+0000 7fd7d27b2700 1 -- 172.21.15.16:0/2286603248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7cc06aea0 msgr2=0x7fd7cc0b3fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:42.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:42.453+0000 7fd7d27b2700 1 --2- 172.21.15.16:0/2286603248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7cc06aea0 0x7fd7cc0b3fa0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fd7b400c770 tx=0x7fd7b400cb30 comp rx=0 tx=0).stop 2023-05-20T01:19:42.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:42.454+0000 7fd7d27b2700 1 -- 172.21.15.16:0/2286603248 shutdown_connections 2023-05-20T01:19:42.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:42.454+0000 7fd7d27b2700 1 --2- 172.21.15.16:0/2286603248 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd7cc0b22a0 0x7fd7cc0c7100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:42.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:42.454+0000 7fd7d27b2700 1 --2- 172.21.15.16:0/2286603248 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd7ac06ace0 0x7fd7ac06d190 secure :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc0d0420 tx=0x7fd7bc008030 comp rx=0 tx=0).stop 2023-05-20T01:19:42.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:42.454+0000 7fd7d27b2700 1 --2- 172.21.15.16:0/2286603248 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd7cc06aad0 0x7fd7cc150920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:42.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:42.454+0000 7fd7d27b2700 1 --2- 172.21.15.16:0/2286603248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7cc06aea0 0x7fd7cc0b3fa0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:42.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:42.454+0000 7fd7d27b2700 1 -- 172.21.15.16:0/2286603248 >> 172.21.15.16:0/2286603248 conn(0x7fd7cc0ccab0 msgr2=0x7fd7cc0d06c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:19:42.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:42.455+0000 7fd7d27b2700 1 -- 172.21.15.16:0/2286603248 shutdown_connections 2023-05-20T01:19:42.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:42.455+0000 7fd7d27b2700 1 -- 172.21.15.16:0/2286603248 wait complete. 2023-05-20T01:19:42.460 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' removed 2023-05-20T01:19:42.475 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_extra cephfs_data_extra --yes-i-really-really-mean-it 2023-05-20T01:19:42.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:42.616+0000 7fed8ad38700 1 Processor -- start 2023-05-20T01:19:42.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:42.616+0000 7fed8ad38700 1 -- start start 2023-05-20T01:19:42.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:42.617+0000 7fed8ad38700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fed8415c830 0x7fed84161070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:42.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:42.617+0000 7fed8ad38700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fed840c6b50 0x7fed8406a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:42.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:42.617+0000 7fed8ad38700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fed8415cc00 0x7fed8406aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:42.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:42.617+0000 7fed8ad38700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed840cf750 con 0x7fed840c6b50 2023-05-20T01:19:42.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:42.617+0000 7fed8ad38700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed840d0730 con 0x7fed8415c830 2023-05-20T01:19:42.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:42.617+0000 7fed8ad38700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed841615b0 con 0x7fed8415cc00 2023-05-20T01:19:42.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:42.617+0000 7fed83fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fed840c6b50 0x7fed8406a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:42.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:42.618+0000 7fed83fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fed840c6b50 0x7fed8406a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33502/0 (socket says 172.21.15.16:33502) 2023-05-20T01:19:42.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:42.618+0000 7fed83fff700 1 -- 172.21.15.16:0/4205709712 learned_addr learned my addr 172.21.15.16:0/4205709712 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:19:42.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:42.618+0000 7fed892d5700 1 --2- 172.21.15.16:0/4205709712 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fed8415cc00 0x7fed8406aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:42.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:42.618+0000 7fed88ad4700 1 --2- 172.21.15.16:0/4205709712 >> v2:172.21.15.132:3300/0 conn(0x7fed8415c830 0x7fed84161070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:42.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:42.618+0000 7fed83fff700 1 -- 172.21.15.16:0/4205709712 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fed8415cc00 msgr2=0x7fed8406aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:19:42.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:42.618+0000 7fed83fff700 1 --2- 172.21.15.16:0/4205709712 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fed8415cc00 0x7fed8406aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:42.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:42.618+0000 7fed83fff700 1 -- 172.21.15.16:0/4205709712 >> v2:172.21.15.132:3300/0 conn(0x7fed8415c830 msgr2=0x7fed84161070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:19:42.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:42.618+0000 7fed83fff700 1 --2- 172.21.15.16:0/4205709712 >> v2:172.21.15.132:3300/0 conn(0x7fed8415c830 0x7fed84161070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:42.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:42.618+0000 7fed83fff700 1 -- 172.21.15.16:0/4205709712 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed840cf980 con 0x7fed840c6b50 2023-05-20T01:19:42.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:42.618+0000 7fed83fff700 1 --2- 172.21.15.16:0/4205709712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed840c6b50 0x7fed8406a510 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fed84072650 tx=0x7fed70009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ac7d755c0f7070c server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:42.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:42.618+0000 7fed837fe700 1 -- 172.21.15.16:0/4205709712 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed7000b870 con 0x7fed840c6b50 2023-05-20T01:19:42.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:42.618+0000 7fed837fe700 1 -- 172.21.15.16:0/4205709712 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed7000b9d0 con 0x7fed840c6b50 2023-05-20T01:19:42.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:42.618+0000 7fed837fe700 1 -- 172.21.15.16:0/4205709712 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed7000bd60 con 0x7fed840c6b50 2023-05-20T01:19:42.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:42.619+0000 7fed8ad38700 1 -- 172.21.15.16:0/4205709712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed840c6b50 msgr2=0x7fed8406a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:42.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:42.619+0000 7fed8ad38700 1 --2- 172.21.15.16:0/4205709712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed840c6b50 0x7fed8406a510 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fed84072650 tx=0x7fed70009bd0 comp rx=0 tx=0).stop 2023-05-20T01:19:42.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:42.619+0000 7fed8ad38700 1 -- 172.21.15.16:0/4205709712 shutdown_connections 2023-05-20T01:19:42.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:42.619+0000 7fed8ad38700 1 --2- 172.21.15.16:0/4205709712 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fed8415cc00 0x7fed8406aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:42.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:42.619+0000 7fed8ad38700 1 --2- 172.21.15.16:0/4205709712 >> v2:172.21.15.132:3300/0 conn(0x7fed8415c830 0x7fed84161070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:42.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:42.619+0000 7fed8ad38700 1 --2- 172.21.15.16:0/4205709712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed840c6b50 0x7fed8406a510 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:42.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:42.619+0000 7fed8ad38700 1 -- 172.21.15.16:0/4205709712 >> 172.21.15.16:0/4205709712 conn(0x7fed840b2920 msgr2=0x7fed840d0020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:19:42.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:42.619+0000 7fed8ad38700 1 -- 172.21.15.16:0/4205709712 shutdown_connections 2023-05-20T01:19:42.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:42.619+0000 7fed8ad38700 1 -- 172.21.15.16:0/4205709712 wait complete. 2023-05-20T01:19:42.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:42.620+0000 7fed8ad38700 1 Processor -- start 2023-05-20T01:19:42.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:42.620+0000 7fed8ad38700 1 -- start start 2023-05-20T01:19:42.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:42.621+0000 7fed8ad38700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fed840c6b50 0x7fed84150a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:42.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:42.621+0000 7fed8ad38700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fed8415c830 0x7fed840aed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:42.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:42.621+0000 7fed8ad38700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed8415cc00 0x7fed84158c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:42.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:42.621+0000 7fed8ad38700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed84150f60 con 0x7fed8415cc00 2023-05-20T01:19:42.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:42.621+0000 7fed8ad38700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed84153f30 con 0x7fed840c6b50 2023-05-20T01:19:42.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:42.621+0000 7fed8ad38700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed84153320 con 0x7fed8415c830 2023-05-20T01:19:42.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:42.621+0000 7fed892d5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed8415cc00 0x7fed84158c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:42.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:42.621+0000 7fed892d5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed8415cc00 0x7fed84158c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33510/0 (socket says 172.21.15.16:33510) 2023-05-20T01:19:42.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:42.621+0000 7fed892d5700 1 -- 172.21.15.16:0/1418328277 learned_addr learned my addr 172.21.15.16:0/1418328277 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:19:42.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:42.621+0000 7fed83fff700 1 --2- 172.21.15.16:0/1418328277 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fed8415c830 0x7fed840aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:42.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:42.621+0000 7fed88ad4700 1 --2- 172.21.15.16:0/1418328277 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fed840c6b50 0x7fed84150a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:42.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:42.621+0000 7fed892d5700 1 -- 172.21.15.16:0/1418328277 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fed8415c830 msgr2=0x7fed840aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:42.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:42.621+0000 7fed892d5700 1 --2- 172.21.15.16:0/1418328277 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fed8415c830 0x7fed840aed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:42.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:42.621+0000 7fed892d5700 1 -- 172.21.15.16:0/1418328277 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fed840c6b50 msgr2=0x7fed84150a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:42.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:42.621+0000 7fed892d5700 1 --2- 172.21.15.16:0/1418328277 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fed840c6b50 0x7fed84150a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:42.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:42.621+0000 7fed892d5700 1 -- 172.21.15.16:0/1418328277 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed70009550 con 0x7fed8415cc00 2023-05-20T01:19:42.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:42.622+0000 7fed892d5700 1 --2- 172.21.15.16:0/1418328277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed8415cc00 0x7fed84158c00 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fed7c007e40 tx=0x7fed7c00e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:42.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:42.622+0000 7fed817fa700 1 -- 172.21.15.16:0/1418328277 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed7c00ed30 con 0x7fed8415cc00 2023-05-20T01:19:42.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:42.622+0000 7fed817fa700 1 -- 172.21.15.16:0/1418328277 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed7c014420 con 0x7fed8415cc00 2023-05-20T01:19:42.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:42.622+0000 7fed8ad38700 1 -- 172.21.15.16:0/1418328277 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed840d0980 con 0x7fed8415cc00 2023-05-20T01:19:42.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:42.622+0000 7fed817fa700 1 -- 172.21.15.16:0/1418328277 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed7c0147d0 con 0x7fed8415cc00 2023-05-20T01:19:42.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:42.622+0000 7fed8ad38700 1 -- 172.21.15.16:0/1418328277 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed840cfc60 con 0x7fed8415cc00 2023-05-20T01:19:42.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:42.623+0000 7fed8ad38700 1 -- 172.21.15.16:0/1418328277 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed840d0c60 con 0x7fed8415cc00 2023-05-20T01:19:42.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:42.623+0000 7fed817fa700 1 -- 172.21.15.16:0/1418328277 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fed7c014930 con 0x7fed8415cc00 2023-05-20T01:19:42.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:42.624+0000 7fed817fa700 1 --2- 172.21.15.16:0/1418328277 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fed6806ace0 0x7fed6806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:42.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:42.624+0000 7fed88ad4700 1 --2- 172.21.15.16:0/1418328277 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fed6806ace0 0x7fed6806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:42.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:42.624+0000 7fed817fa700 1 -- 172.21.15.16:0/1418328277 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6278+0+0 (secure 0 0 0) 0x7fed7c089be0 con 0x7fed8415cc00 2023-05-20T01:19:42.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:42.624+0000 7fed88ad4700 1 --2- 172.21.15.16:0/1418328277 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fed6806ace0 0x7fed6806d190 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fed84157210 tx=0x7fed78008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:42.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:42.628+0000 7fed817fa700 1 -- 172.21.15.16:0/1418328277 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fed7c054bb0 con 0x7fed8415cc00 2023-05-20T01:19:42.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:42.825+0000 7fed8ad38700 1 -- 172.21.15.16:0/1418328277 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_extra", "pool2": "cephfs_data_extra", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fed841540a0 con 0x7fed8415cc00 2023-05-20T01:19:43.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.460+0000 7fed817fa700 1 -- 172.21.15.16:0/1418328277 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_extra", "pool2": "cephfs_data_extra", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_extra' removed v55) v1 ==== 187+0+0 (secure 0 0 0) 0x7fed7c0581d0 con 0x7fed8415cc00 2023-05-20T01:19:43.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.464+0000 7fed8ad38700 1 -- 172.21.15.16:0/1418328277 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fed6806ace0 msgr2=0x7fed6806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:43.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.464+0000 7fed8ad38700 1 --2- 172.21.15.16:0/1418328277 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fed6806ace0 0x7fed6806d190 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fed84157210 tx=0x7fed78008040 comp rx=0 tx=0).stop 2023-05-20T01:19:43.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.464+0000 7fed8ad38700 1 -- 172.21.15.16:0/1418328277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed8415cc00 msgr2=0x7fed84158c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:43.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.464+0000 7fed8ad38700 1 --2- 172.21.15.16:0/1418328277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed8415cc00 0x7fed84158c00 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fed7c007e40 tx=0x7fed7c00e3c0 comp rx=0 tx=0).stop 2023-05-20T01:19:43.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.464+0000 7fed8ad38700 1 -- 172.21.15.16:0/1418328277 shutdown_connections 2023-05-20T01:19:43.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.464+0000 7fed8ad38700 1 --2- 172.21.15.16:0/1418328277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed8415cc00 0x7fed84158c00 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:43.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.464+0000 7fed8ad38700 1 --2- 172.21.15.16:0/1418328277 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fed840c6b50 0x7fed84150a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:43.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.464+0000 7fed8ad38700 1 --2- 172.21.15.16:0/1418328277 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fed6806ace0 0x7fed6806d190 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:43.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.464+0000 7fed8ad38700 1 --2- 172.21.15.16:0/1418328277 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fed8415c830 0x7fed840aed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:43.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.464+0000 7fed8ad38700 1 -- 172.21.15.16:0/1418328277 >> 172.21.15.16:0/1418328277 conn(0x7fed840b2920 msgr2=0x7fed840ccaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:19:43.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.464+0000 7fed8ad38700 1 -- 172.21.15.16:0/1418328277 shutdown_connections 2023-05-20T01:19:43.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.464+0000 7fed8ad38700 1 -- 172.21.15.16:0/1418328277 wait complete. 2023-05-20T01:19:43.473 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data_extra' removed 2023-05-20T01:19:43.484 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-05-20T01:19:43.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.616+0000 7f7698350700 1 Processor -- start 2023-05-20T01:19:43.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.616+0000 7f7698350700 1 -- start start 2023-05-20T01:19:43.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.618+0000 7f7698350700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f76900cd070 0x7f769006e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:43.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.618+0000 7f7698350700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7690072430 con 0x7f76900710d0 2023-05-20T01:19:43.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.618+0000 7f7698350700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7690071820 con 0x7f769015cf40 2023-05-20T01:19:43.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.618+0000 7f7698350700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7690071b00 con 0x7f76900cd070 2023-05-20T01:19:43.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.618+0000 7f76968ed700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f76900710d0 0x7f76900cabb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:45046/0 (socket says 172.21.15.16:45046) 2023-05-20T01:19:43.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.618+0000 7f76968ed700 1 -- 172.21.15.16:0/1343765264 learned_addr learned my addr 172.21.15.16:0/1343765264 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:19:43.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.618+0000 7f76958eb700 1 --2- 172.21.15.16:0/1343765264 >> v2:172.21.15.132:3300/0 conn(0x7f76900cd070 0x7f769006e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:43.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.618+0000 7f76950ea700 1 -- 172.21.15.16:0/1343765264 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2920604520 0 0) 0x7f7690072430 con 0x7f76900710d0 2023-05-20T01:19:43.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.618+0000 7f76950ea700 1 -- 172.21.15.16:0/1343765264 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f767c0035b0 con 0x7f76900710d0 2023-05-20T01:19:43.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.618+0000 7f76950ea700 1 -- 172.21.15.16:0/1343765264 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 432487131 0 0) 0x7f767c0035b0 con 0x7f76900710d0 2023-05-20T01:19:43.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.618+0000 7f76950ea700 1 -- 172.21.15.16:0/1343765264 >> v2:172.21.15.132:3300/0 conn(0x7f76900cd070 msgr2=0x7f769006e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:19:43.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.618+0000 7f76950ea700 1 --2- 172.21.15.16:0/1343765264 >> v2:172.21.15.132:3300/0 conn(0x7f76900cd070 0x7f769006e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:43.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.618+0000 7f76950ea700 1 -- 172.21.15.16:0/1343765264 >> v1:172.21.15.132:6789/0 conn(0x7f769015cf40 legacy=0x7f76900cc960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:43.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.618+0000 7f76950ea700 1 -- 172.21.15.16:0/1343765264 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7690071de0 con 0x7f76900710d0 2023-05-20T01:19:43.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.618+0000 7f76950ea700 1 -- 172.21.15.16:0/1343765264 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7f76880019f0 con 0x7f76900710d0 2023-05-20T01:19:43.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.619+0000 7f76950ea700 1 -- 172.21.15.16:0/1343765264 >> v1:172.21.15.16:6789/0 conn(0x7f76900710d0 legacy=0x7f76900cabb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:43.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.619+0000 7f76950ea700 1 --2- 172.21.15.16:0/1343765264 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f767c003a00 0x7f767c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:43.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.619+0000 7f76950ea700 1 --2- 172.21.15.16:0/1343765264 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f767c004850 0x7f767c004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:43.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.619+0000 7f76950ea700 1 --2- 172.21.15.16:0/1343765264 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f767c005200 0x7f767c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:43.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.619+0000 7f76950ea700 1 -- 172.21.15.16:0/1343765264 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7690071b00 con 0x7f767c005200 2023-05-20T01:19:43.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.619+0000 7f76950ea700 1 -- 172.21.15.16:0/1343765264 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f767c00c080 con 0x7f767c003a00 2023-05-20T01:19:43.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.619+0000 7f76950ea700 1 -- 172.21.15.16:0/1343765264 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f767c00c240 con 0x7f767c004850 2023-05-20T01:19:43.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.619+0000 7f76968ed700 1 --2- 172.21.15.16:0/1343765264 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f767c005200 0x7f767c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:43.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.619+0000 7f76960ec700 1 --2- 172.21.15.16:0/1343765264 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f767c003a00 0x7f767c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:43.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.619+0000 7f76958eb700 1 --2- 172.21.15.16:0/1343765264 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f767c004850 0x7f767c004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:43.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.619+0000 7f76968ed700 1 -- 172.21.15.16:0/1343765264 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f767c004850 msgr2=0x7f767c004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:19:43.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.619+0000 7f76968ed700 1 --2- 172.21.15.16:0/1343765264 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f767c004850 0x7f767c004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:43.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.619+0000 7f76968ed700 1 -- 172.21.15.16:0/1343765264 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f767c003a00 msgr2=0x7f767c003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:19:43.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.619+0000 7f76968ed700 1 --2- 172.21.15.16:0/1343765264 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f767c003a00 0x7f767c003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:43.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.619+0000 7f76968ed700 1 -- 172.21.15.16:0/1343765264 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f767c0035b0 con 0x7f767c005200 2023-05-20T01:19:43.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.619+0000 7f76958eb700 1 --2- 172.21.15.16:0/1343765264 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f767c004850 0x7f767c004cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:19:43.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.619+0000 7f76968ed700 1 -- 172.21.15.16:0/1343765264 reap_dead start 2023-05-20T01:19:43.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.619+0000 7f76968ed700 1 --2- 172.21.15.16:0/1343765264 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f767c005200 0x7f767c00ba10 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f768800a830 tx=0x7f767c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b90cf1b31d6f8365 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:43.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.619+0000 7f76950ea700 1 -- 172.21.15.16:0/1343765264 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f768800a2f0 con 0x7f767c005200 2023-05-20T01:19:43.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.619+0000 7f76950ea700 1 -- 172.21.15.16:0/1343765264 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7688012620 con 0x7f767c005200 2023-05-20T01:19:43.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.619+0000 7f76950ea700 1 -- 172.21.15.16:0/1343765264 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76880129d0 con 0x7f767c005200 2023-05-20T01:19:43.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.620+0000 7f7698350700 1 -- 172.21.15.16:0/1343765264 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f767c005200 msgr2=0x7f767c00ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:43.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.620+0000 7f7698350700 1 --2- 172.21.15.16:0/1343765264 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f767c005200 0x7f767c00ba10 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f768800a830 tx=0x7f767c005880 comp rx=0 tx=0).stop 2023-05-20T01:19:43.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.620+0000 7f7698350700 1 -- 172.21.15.16:0/1343765264 shutdown_connections 2023-05-20T01:19:43.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.620+0000 7f7698350700 1 --2- 172.21.15.16:0/1343765264 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f767c005200 0x7f767c00ba10 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:43.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.620+0000 7f7698350700 1 -- 172.21.15.16:0/1343765264 >> 172.21.15.16:0/1343765264 conn(0x7f769006c260 msgr2=0x7f76900cffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:19:43.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.620+0000 7f7698350700 1 -- 172.21.15.16:0/1343765264 shutdown_connections 2023-05-20T01:19:43.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.620+0000 7f7698350700 1 -- 172.21.15.16:0/1343765264 wait complete. 2023-05-20T01:19:43.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.622+0000 7f7698350700 1 Processor -- start 2023-05-20T01:19:43.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.622+0000 7f7698350700 1 -- start start 2023-05-20T01:19:43.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.622+0000 7f7698350700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f767c005200 0x7f76901501e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:43.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.622+0000 7f7698350700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7690150830 0x7f769014fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:43.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.622+0000 7f7698350700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76900b41d0 0x7f76900b5770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:43.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.622+0000 7f7698350700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76900b46d0 con 0x7f76900b41d0 2023-05-20T01:19:43.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.622+0000 7f7698350700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76900b49b0 con 0x7f7690150830 2023-05-20T01:19:43.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.622+0000 7f7698350700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76900b4c90 con 0x7f767c005200 2023-05-20T01:19:43.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.623+0000 7f76968ed700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76900b41d0 0x7f76900b5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:43.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.623+0000 7f76968ed700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76900b41d0 0x7f76900b5770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33530/0 (socket says 172.21.15.16:33530) 2023-05-20T01:19:43.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.623+0000 7f76968ed700 1 -- 172.21.15.16:0/3705659033 learned_addr learned my addr 172.21.15.16:0/3705659033 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:19:43.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.623+0000 7f76960ec700 1 --2- 172.21.15.16:0/3705659033 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f767c005200 0x7f76901501e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:43.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.623+0000 7f76958eb700 1 --2- 172.21.15.16:0/3705659033 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7690150830 0x7f769014fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:43.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.623+0000 7f76968ed700 1 -- 172.21.15.16:0/3705659033 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f767c005200 msgr2=0x7f76901501e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:43.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.623+0000 7f76968ed700 1 --2- 172.21.15.16:0/3705659033 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f767c005200 0x7f76901501e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:43.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.623+0000 7f76968ed700 1 -- 172.21.15.16:0/3705659033 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7690150830 msgr2=0x7f769014fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:43.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.623+0000 7f76968ed700 1 --2- 172.21.15.16:0/3705659033 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7690150830 0x7f769014fa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:43.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.623+0000 7f76968ed700 1 -- 172.21.15.16:0/3705659033 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7690071510 con 0x7f76900b41d0 2023-05-20T01:19:43.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.623+0000 7f76968ed700 1 --2- 172.21.15.16:0/3705659033 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76900b41d0 0x7f76900b5770 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f76900b2b50 tx=0x7f7688001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:43.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.623+0000 7f7686ffd700 1 -- 172.21.15.16:0/3705659033 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7688013ea0 con 0x7f76900b41d0 2023-05-20T01:19:43.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.623+0000 7f7686ffd700 1 -- 172.21.15.16:0/3705659033 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76880019d0 con 0x7f76900b41d0 2023-05-20T01:19:43.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.623+0000 7f7698350700 1 -- 172.21.15.16:0/3705659033 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76900720c0 con 0x7f76900b41d0 2023-05-20T01:19:43.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.623+0000 7f7686ffd700 1 -- 172.21.15.16:0/3705659033 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f768801d660 con 0x7f76900b41d0 2023-05-20T01:19:43.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.623+0000 7f7698350700 1 -- 172.21.15.16:0/3705659033 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f769006e020 con 0x7f76900b41d0 2023-05-20T01:19:43.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.624+0000 7f7698350700 1 -- 172.21.15.16:0/3705659033 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76900747b0 con 0x7f76900b41d0 2023-05-20T01:19:43.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.625+0000 7f7686ffd700 1 -- 172.21.15.16:0/3705659033 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f768801d7c0 con 0x7f76900b41d0 2023-05-20T01:19:43.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.625+0000 7f7686ffd700 1 --2- 172.21.15.16:0/3705659033 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f767406ace0 0x7f767406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:43.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.625+0000 7f7686ffd700 1 -- 172.21.15.16:0/3705659033 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5874+0+0 (secure 0 0 0) 0x7f768808b510 con 0x7f76900b41d0 2023-05-20T01:19:43.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.625+0000 7f76960ec700 1 --2- 172.21.15.16:0/3705659033 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f767406ace0 0x7f767406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:43.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.625+0000 7f76960ec700 1 --2- 172.21.15.16:0/3705659033 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f767406ace0 0x7f767406d190 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f76901e5400 tx=0x7f7680009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:43.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.629+0000 7f7686ffd700 1 -- 172.21.15.16:0/3705659033 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f7688056820 con 0x7f76900b41d0 2023-05-20T01:19:43.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.828+0000 7f7698350700 1 -- 172.21.15.16:0/3705659033 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f769014ff80 con 0x7f76900b41d0 2023-05-20T01:19:43.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.831+0000 7f7686ffd700 1 -- 172.21.15.16:0/3705659033 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f7688059e40 con 0x7f76900b41d0 2023-05-20T01:19:43.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.833+0000 7f7698350700 1 -- 172.21.15.16:0/3705659033 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f767406ace0 msgr2=0x7f767406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:43.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.833+0000 7f7698350700 1 --2- 172.21.15.16:0/3705659033 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f767406ace0 0x7f767406d190 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f76901e5400 tx=0x7f7680009040 comp rx=0 tx=0).stop 2023-05-20T01:19:43.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.833+0000 7f7698350700 1 -- 172.21.15.16:0/3705659033 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76900b41d0 msgr2=0x7f76900b5770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:43.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.833+0000 7f7698350700 1 --2- 172.21.15.16:0/3705659033 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76900b41d0 0x7f76900b5770 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f76900b2b50 tx=0x7f7688001d80 comp rx=0 tx=0).stop 2023-05-20T01:19:43.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.833+0000 7f7698350700 1 -- 172.21.15.16:0/3705659033 shutdown_connections 2023-05-20T01:19:43.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.833+0000 7f7698350700 1 --2- 172.21.15.16:0/3705659033 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76900b41d0 0x7f76900b5770 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:43.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.833+0000 7f7698350700 1 --2- 172.21.15.16:0/3705659033 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f767406ace0 0x7f767406d190 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:43.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.833+0000 7f7698350700 1 --2- 172.21.15.16:0/3705659033 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f767c005200 0x7f76901501e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:43.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.833+0000 7f7698350700 1 --2- 172.21.15.16:0/3705659033 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7690150830 0x7f769014fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:43.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.833+0000 7f7698350700 1 -- 172.21.15.16:0/3705659033 >> 172.21.15.16:0/3705659033 conn(0x7f769006c260 msgr2=0x7f76900cffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:19:43.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.833+0000 7f7698350700 1 -- 172.21.15.16:0/3705659033 shutdown_connections 2023-05-20T01:19:43.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.833+0000 7f7698350700 1 -- 172.21.15.16:0/3705659033 wait complete. 2023-05-20T01:19:43.849 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_data_scan.TestDataScan.test_extra_data_pool_rebuild_few_files' 2023-05-20T01:19:43.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.975+0000 7f00714f6700 1 Processor -- start 2023-05-20T01:19:43.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.975+0000 7f00714f6700 1 -- start start 2023-05-20T01:19:43.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.976+0000 7f00714f6700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f006c06b730 0x7f006c0b53d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:43.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.976+0000 7f00714f6700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f006c0cf500 0x7f006c0cf960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:43.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.976+0000 7f00714f6700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f006c15d020 con 0x7f006c15cc50 2023-05-20T01:19:43.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.976+0000 7f00714f6700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f006c0cd720 con 0x7f006c0cf500 2023-05-20T01:19:43.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.976+0000 7f00714f6700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f006c0d1bf0 con 0x7f006c06b730 2023-05-20T01:19:43.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.976+0000 7f006b7fe700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f006c0cf500 0x7f006c0cf960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:43.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.976+0000 7f006a7fc700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f006c06b730 0x7f006c0b53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:43.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.976+0000 7f006affd700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f006c15cc50 0x7f006c0b4cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.16:54776/0 (socket says 172.21.15.16:54776) 2023-05-20T01:19:43.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.976+0000 7f006affd700 1 -- 172.21.15.16:0/2012620615 learned_addr learned my addr 172.21.15.16:0/2012620615 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:19:43.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.977+0000 7f006a7fc700 1 -- 172.21.15.16:0/2012620615 >> v2:172.21.15.132:3300/0 conn(0x7f006c0cf500 msgr2=0x7f006c0cf960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:19:43.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.977+0000 7f006a7fc700 1 --2- 172.21.15.16:0/2012620615 >> v2:172.21.15.132:3300/0 conn(0x7f006c0cf500 0x7f006c0cf960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:43.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.977+0000 7f006a7fc700 1 -- 172.21.15.16:0/2012620615 >> v1:172.21.15.132:6789/0 conn(0x7f006c15cc50 legacy=0x7f006c0b4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:43.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.977+0000 7f006a7fc700 1 -- 172.21.15.16:0/2012620615 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f006c0d18c0 con 0x7f006c06b730 2023-05-20T01:19:43.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.977+0000 7f006b7fe700 1 --2- 172.21.15.16:0/2012620615 >> v2:172.21.15.132:3300/0 conn(0x7f006c0cf500 0x7f006c0cf960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-05-20T01:19:43.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.977+0000 7f006a7fc700 1 --2- 172.21.15.16:0/2012620615 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f006c06b730 0x7f006c0b53d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f005c009840 tx=0x7f005c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=678d7839d36bdf95 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:43.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.977+0000 7f0069ffb700 1 -- 172.21.15.16:0/2012620615 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f005c011830 con 0x7f006c06b730 2023-05-20T01:19:43.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.977+0000 7f0069ffb700 1 -- 172.21.15.16:0/2012620615 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f005c011990 con 0x7f006c06b730 2023-05-20T01:19:43.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.977+0000 7f0069ffb700 1 -- 172.21.15.16:0/2012620615 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f005c011d20 con 0x7f006c06b730 2023-05-20T01:19:43.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.977+0000 7f00714f6700 1 -- 172.21.15.16:0/2012620615 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f006c06b730 msgr2=0x7f006c0b53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:43.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.977+0000 7f00714f6700 1 --2- 172.21.15.16:0/2012620615 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f006c06b730 0x7f006c0b53d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f005c009840 tx=0x7f005c009b50 comp rx=0 tx=0).stop 2023-05-20T01:19:43.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.978+0000 7f00714f6700 1 -- 172.21.15.16:0/2012620615 shutdown_connections 2023-05-20T01:19:43.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.978+0000 7f00714f6700 1 --2- 172.21.15.16:0/2012620615 >> v2:172.21.15.132:3300/0 conn(0x7f006c0cf500 0x7f006c0cf960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:43.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.978+0000 7f00714f6700 1 --2- 172.21.15.16:0/2012620615 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f006c06b730 0x7f006c0b53d0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:43.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.978+0000 7f00714f6700 1 -- 172.21.15.16:0/2012620615 >> 172.21.15.16:0/2012620615 conn(0x7f006c072860 msgr2=0x7f006c06b020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:19:43.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.978+0000 7f00714f6700 1 -- 172.21.15.16:0/2012620615 shutdown_connections 2023-05-20T01:19:43.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.978+0000 7f00714f6700 1 -- 172.21.15.16:0/2012620615 wait complete. 2023-05-20T01:19:43.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.979+0000 7f00714f6700 1 Processor -- start 2023-05-20T01:19:43.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.979+0000 7f00714f6700 1 -- start start 2023-05-20T01:19:43.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.980+0000 7f00714f6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f006c06b730 0x7f006c0cc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:43.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.980+0000 7f00714f6700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f006c0cf500 0x7f006c0caf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:43.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.980+0000 7f00714f6700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f006c15cc50 0x7f006c0b6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:43.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.980+0000 7f00714f6700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f006c0b7010 con 0x7f006c06b730 2023-05-20T01:19:43.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.980+0000 7f00714f6700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f006c0cb630 con 0x7f006c0cf500 2023-05-20T01:19:43.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.980+0000 7f00714f6700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f006c0cb910 con 0x7f006c15cc50 2023-05-20T01:19:43.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.980+0000 7f006affd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f006c06b730 0x7f006c0cc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:43.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.980+0000 7f006affd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f006c06b730 0x7f006c0cc6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33540/0 (socket says 172.21.15.16:33540) 2023-05-20T01:19:43.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.980+0000 7f006affd700 1 -- 172.21.15.16:0/3596962984 learned_addr learned my addr 172.21.15.16:0/3596962984 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:19:43.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.980+0000 7f006a7fc700 1 --2- 172.21.15.16:0/3596962984 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f006c0cf500 0x7f006c0caf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:43.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.980+0000 7f006b7fe700 1 --2- 172.21.15.16:0/3596962984 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f006c15cc50 0x7f006c0b6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:43.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.980+0000 7f006affd700 1 -- 172.21.15.16:0/3596962984 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f006c15cc50 msgr2=0x7f006c0b6ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:43.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.980+0000 7f006affd700 1 --2- 172.21.15.16:0/3596962984 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f006c15cc50 0x7f006c0b6ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:43.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.980+0000 7f006affd700 1 -- 172.21.15.16:0/3596962984 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f006c0cf500 msgr2=0x7f006c0caf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:43.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.980+0000 7f006affd700 1 --2- 172.21.15.16:0/3596962984 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f006c0cf500 0x7f006c0caf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:43.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.980+0000 7f006affd700 1 -- 172.21.15.16:0/3596962984 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0064009a70 con 0x7f006c06b730 2023-05-20T01:19:43.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.980+0000 7f006a7fc700 1 --2- 172.21.15.16:0/3596962984 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f006c0cf500 0x7f006c0caf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:19:43.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.980+0000 7f006affd700 1 --2- 172.21.15.16:0/3596962984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f006c06b730 0x7f006c0cc6d0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f006000c490 tx=0x7f006000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:43.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.980+0000 7f00537fe700 1 -- 172.21.15.16:0/3596962984 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0060005d70 con 0x7f006c06b730 2023-05-20T01:19:43.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.980+0000 7f00537fe700 1 -- 172.21.15.16:0/3596962984 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00600070d0 con 0x7f006c06b730 2023-05-20T01:19:43.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.980+0000 7f00714f6700 1 -- 172.21.15.16:0/3596962984 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f005c0094a0 con 0x7f006c06b730 2023-05-20T01:19:43.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.980+0000 7f00537fe700 1 -- 172.21.15.16:0/3596962984 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0060007460 con 0x7f006c06b730 2023-05-20T01:19:43.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.981+0000 7f00714f6700 1 -- 172.21.15.16:0/3596962984 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f006c0cdca0 con 0x7f006c06b730 2023-05-20T01:19:43.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.981+0000 7f00714f6700 1 -- 172.21.15.16:0/3596962984 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f006c0cd9c0 con 0x7f006c06b730 2023-05-20T01:19:43.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.982+0000 7f00537fe700 1 -- 172.21.15.16:0/3596962984 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f00600075c0 con 0x7f006c06b730 2023-05-20T01:19:43.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.982+0000 7f00537fe700 1 --2- 172.21.15.16:0/3596962984 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f005806ace0 0x7f005806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:43.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.982+0000 7f006a7fc700 1 --2- 172.21.15.16:0/3596962984 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f005806ace0 0x7f005806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:43.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.982+0000 7f00537fe700 1 -- 172.21.15.16:0/3596962984 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5874+0+0 (secure 0 0 0) 0x7f0060088730 con 0x7f006c06b730 2023-05-20T01:19:43.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.983+0000 7f006a7fc700 1 --2- 172.21.15.16:0/3596962984 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f005806ace0 0x7f005806d190 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f005c000c00 tx=0x7f005c01c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:43.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:43.987+0000 7f00537fe700 1 -- 172.21.15.16:0/3596962984 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f0060053890 con 0x7f006c06b730 2023-05-20T01:19:44.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:44.268+0000 7f00714f6700 1 -- 172.21.15.16:0/3596962984 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_data_scan.TestDataScan.test_extra_data_pool_rebuild_few_files"]} v 0) v1 -- 0x7f006c0cdf80 con 0x7f006c06b730 2023-05-20T01:19:44.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:44.460+0000 7f00537fe700 1 -- 172.21.15.16:0/3596962984 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_data_scan.TestDataScan.test_extra_data_pool_rebuild_few_files"]}]=0 v172) v1 ==== 158+0+0 (secure 0 0 0) 0x7f0060056eb0 con 0x7f006c06b730 2023-05-20T01:19:44.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:44.462+0000 7f00714f6700 1 -- 172.21.15.16:0/3596962984 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f005806ace0 msgr2=0x7f005806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:44.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:44.462+0000 7f00714f6700 1 --2- 172.21.15.16:0/3596962984 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f005806ace0 0x7f005806d190 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f005c000c00 tx=0x7f005c01c040 comp rx=0 tx=0).stop 2023-05-20T01:19:44.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:44.462+0000 7f00714f6700 1 -- 172.21.15.16:0/3596962984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f006c06b730 msgr2=0x7f006c0cc6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:44.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:44.462+0000 7f00714f6700 1 --2- 172.21.15.16:0/3596962984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f006c06b730 0x7f006c0cc6d0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f006000c490 tx=0x7f006000c850 comp rx=0 tx=0).stop 2023-05-20T01:19:44.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:44.462+0000 7f00714f6700 1 -- 172.21.15.16:0/3596962984 shutdown_connections 2023-05-20T01:19:44.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:44.462+0000 7f00714f6700 1 --2- 172.21.15.16:0/3596962984 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f005806ace0 0x7f005806d190 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:44.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:44.462+0000 7f00714f6700 1 --2- 172.21.15.16:0/3596962984 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f006c15cc50 0x7f006c0b6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:44.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:44.462+0000 7f00714f6700 1 --2- 172.21.15.16:0/3596962984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f006c06b730 0x7f006c0cc6d0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:44.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:44.462+0000 7f00714f6700 1 --2- 172.21.15.16:0/3596962984 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f006c0cf500 0x7f006c0caf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:44.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:44.462+0000 7f00714f6700 1 -- 172.21.15.16:0/3596962984 >> 172.21.15.16:0/3596962984 conn(0x7f006c072860 msgr2=0x7f006c06b020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:19:44.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:44.462+0000 7f00714f6700 1 -- 172.21.15.16:0/3596962984 shutdown_connections 2023-05-20T01:19:44.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:44.462+0000 7f00714f6700 1 -- 172.21.15.16:0/3596962984 wait complete. 2023-05-20T01:19:44.478 INFO:tasks.cephfs_test_runner:test_extra_data_pool_rebuild_few_files (tasks.cephfs.test_data_scan.TestDataScan) ... ok 2023-05-20T01:19:44.479 INFO:tasks.cephfs_test_runner:Starting test: test_extra_data_pool_rebuild_many_files_many_workers (tasks.cephfs.test_data_scan.TestDataScan) 2023-05-20T01:19:44.479 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_data_scan.TestDataScan.test_extra_data_pool_rebuild_many_files_many_workers' 2023-05-20T01:19:44.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:44.601+0000 7f5bc2798700 1 Processor -- start 2023-05-20T01:19:44.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:44.601+0000 7f5bc2798700 1 -- start start 2023-05-20T01:19:44.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:44.602+0000 7f5bc2798700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f5bbc0d07f0 0x7f5bbc0cbe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:44.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:44.602+0000 7f5bc2798700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bbc0cfc90 con 0x7f5bbc15cd40 2023-05-20T01:19:44.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:44.602+0000 7f5bc2798700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bbc0cf080 con 0x7f5bbc0d0380 2023-05-20T01:19:44.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:44.602+0000 7f5bc2798700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bbc0cf360 con 0x7f5bbc0d07f0 2023-05-20T01:19:44.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:44.602+0000 7f5bbbfff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f5bbc15cd40 0x7f5bbc0d1380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:45076/0 (socket says 172.21.15.16:45076) 2023-05-20T01:19:44.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:44.602+0000 7f5bbbfff700 1 -- 172.21.15.16:0/3112079558 learned_addr learned my addr 172.21.15.16:0/3112079558 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:19:44.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:44.602+0000 7f5bc0d35700 1 --2- 172.21.15.16:0/3112079558 >> v2:172.21.15.132:3300/0 conn(0x7f5bbc0d07f0 0x7f5bbc0cbe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:44.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:44.603+0000 7f5bbaffd700 1 -- 172.21.15.16:0/3112079558 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2816821397 0 0) 0x7f5bbc0cfc90 con 0x7f5bbc15cd40 2023-05-20T01:19:44.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:44.603+0000 7f5bbaffd700 1 -- 172.21.15.16:0/3112079558 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ba80035b0 con 0x7f5bbc15cd40 2023-05-20T01:19:44.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:44.603+0000 7f5bbaffd700 1 -- 172.21.15.16:0/3112079558 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 744803718 0 0) 0x7f5ba80035b0 con 0x7f5bbc15cd40 2023-05-20T01:19:44.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:44.603+0000 7f5bbaffd700 1 -- 172.21.15.16:0/3112079558 >> v2:172.21.15.132:3300/0 conn(0x7f5bbc0d07f0 msgr2=0x7f5bbc0cbe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:19:44.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:44.603+0000 7f5bbaffd700 1 --2- 172.21.15.16:0/3112079558 >> v2:172.21.15.132:3300/0 conn(0x7f5bbc0d07f0 0x7f5bbc0cbe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:44.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:44.603+0000 7f5bbaffd700 1 -- 172.21.15.16:0/3112079558 >> v1:172.21.15.132:6789/0 conn(0x7f5bbc0d0380 legacy=0x7f5bbc0cb720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:44.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:44.603+0000 7f5bbaffd700 1 -- 172.21.15.16:0/3112079558 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bbc0cf640 con 0x7f5bbc15cd40 2023-05-20T01:19:44.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:44.603+0000 7f5bbaffd700 1 -- 172.21.15.16:0/3112079558 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7f5bac001670 con 0x7f5bbc15cd40 2023-05-20T01:19:44.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:44.603+0000 7f5bbaffd700 1 -- 172.21.15.16:0/3112079558 >> v1:172.21.15.16:6789/0 conn(0x7f5bbc15cd40 legacy=0x7f5bbc0d1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:44.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:44.603+0000 7f5bbaffd700 1 --2- 172.21.15.16:0/3112079558 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5ba8003a70 0x7f5ba8003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:44.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:44.603+0000 7f5bbaffd700 1 --2- 172.21.15.16:0/3112079558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ba8004850 0x7f5ba8004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:44.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:44.603+0000 7f5bbaffd700 1 --2- 172.21.15.16:0/3112079558 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5ba8005200 0x7f5ba800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:44.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:44.603+0000 7f5bbaffd700 1 -- 172.21.15.16:0/3112079558 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bbc0cf360 con 0x7f5ba8004850 2023-05-20T01:19:44.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:44.603+0000 7f5bbaffd700 1 -- 172.21.15.16:0/3112079558 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba800c0e0 con 0x7f5ba8005200 2023-05-20T01:19:44.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:44.603+0000 7f5bbaffd700 1 -- 172.21.15.16:0/3112079558 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba800c2a0 con 0x7f5ba8003a70 2023-05-20T01:19:44.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:44.603+0000 7f5bbb7fe700 1 --2- 172.21.15.16:0/3112079558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ba8004850 0x7f5ba8004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:44.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:44.603+0000 7f5bc0d35700 1 --2- 172.21.15.16:0/3112079558 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5ba8005200 0x7f5ba800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:44.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:44.603+0000 7f5bbbfff700 1 --2- 172.21.15.16:0/3112079558 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5ba8003a70 0x7f5ba8003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:44.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:44.603+0000 7f5bbb7fe700 1 -- 172.21.15.16:0/3112079558 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5ba8003a70 msgr2=0x7f5ba8003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:19:44.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:44.603+0000 7f5bbb7fe700 1 --2- 172.21.15.16:0/3112079558 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5ba8003a70 0x7f5ba8003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:44.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:44.603+0000 7f5bbb7fe700 1 -- 172.21.15.16:0/3112079558 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5ba8005200 msgr2=0x7f5ba800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:19:44.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:44.603+0000 7f5bbb7fe700 1 --2- 172.21.15.16:0/3112079558 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5ba8005200 0x7f5ba800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:44.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:44.603+0000 7f5bbb7fe700 1 -- 172.21.15.16:0/3112079558 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ba80035b0 con 0x7f5ba8004850 2023-05-20T01:19:44.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:44.603+0000 7f5bc0d35700 1 -- 172.21.15.16:0/3112079558 reap_dead start 2023-05-20T01:19:44.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:44.603+0000 7f5bbb7fe700 1 --2- 172.21.15.16:0/3112079558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ba8004850 0x7f5ba8004cc0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f5ba40017c0 tx=0x7f5ba8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2377634c58533fa5 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:44.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:44.604+0000 7f5bbaffd700 1 -- 172.21.15.16:0/3112079558 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ba40052d0 con 0x7f5ba8004850 2023-05-20T01:19:44.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:44.604+0000 7f5bbaffd700 1 -- 172.21.15.16:0/3112079558 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ba4005450 con 0x7f5ba8004850 2023-05-20T01:19:44.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:44.604+0000 7f5bbaffd700 1 -- 172.21.15.16:0/3112079558 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ba40038c0 con 0x7f5ba8004850 2023-05-20T01:19:44.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:44.604+0000 7f5bc2798700 1 -- 172.21.15.16:0/3112079558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ba8004850 msgr2=0x7f5ba8004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:44.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:44.604+0000 7f5bc2798700 1 --2- 172.21.15.16:0/3112079558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ba8004850 0x7f5ba8004cc0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f5ba40017c0 tx=0x7f5ba8005880 comp rx=0 tx=0).stop 2023-05-20T01:19:44.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:44.604+0000 7f5bc2798700 1 -- 172.21.15.16:0/3112079558 shutdown_connections 2023-05-20T01:19:44.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:44.604+0000 7f5bc2798700 1 --2- 172.21.15.16:0/3112079558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ba8004850 0x7f5ba8004cc0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:44.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:44.604+0000 7f5bc2798700 1 -- 172.21.15.16:0/3112079558 >> 172.21.15.16:0/3112079558 conn(0x7f5bbc06ae30 msgr2=0x7f5bbc09a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:19:44.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:44.604+0000 7f5bc2798700 1 -- 172.21.15.16:0/3112079558 shutdown_connections 2023-05-20T01:19:44.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:44.604+0000 7f5bc2798700 1 -- 172.21.15.16:0/3112079558 wait complete. 2023-05-20T01:19:44.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:44.605+0000 7f5bc2798700 1 Processor -- start 2023-05-20T01:19:44.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:44.606+0000 7f5bc2798700 1 -- start start 2023-05-20T01:19:44.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:44.606+0000 7f5bc2798700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5ba8004850 0x7f5bbc0b68e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:44.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:44.606+0000 7f5bc2798700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5bbc15bd30 0x7f5bbc15c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:44.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:44.606+0000 7f5bc2798700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5bbc0c6990 0x7f5bbc0c6e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:44.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:44.606+0000 7f5bc2798700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bbc155860 con 0x7f5bbc0c6990 2023-05-20T01:19:44.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:44.606+0000 7f5bc2798700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bbc0c7cd0 con 0x7f5bbc15bd30 2023-05-20T01:19:44.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:44.606+0000 7f5bc2798700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bbc15c6e0 con 0x7f5ba8004850 2023-05-20T01:19:44.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:44.606+0000 7f5bc0d35700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5bbc0c6990 0x7f5bbc0c6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:44.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:44.607+0000 7f5bc0d35700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5bbc0c6990 0x7f5bbc0c6e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33562/0 (socket says 172.21.15.16:33562) 2023-05-20T01:19:44.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:44.607+0000 7f5bc0d35700 1 -- 172.21.15.16:0/1782259692 learned_addr learned my addr 172.21.15.16:0/1782259692 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:19:44.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:44.607+0000 7f5bbbfff700 1 --2- 172.21.15.16:0/1782259692 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5ba8004850 0x7f5bbc0b68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:44.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:44.607+0000 7f5bc0d35700 1 -- 172.21.15.16:0/1782259692 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5ba8004850 msgr2=0x7f5bbc0b68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:44.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:44.607+0000 7f5bc0d35700 1 --2- 172.21.15.16:0/1782259692 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5ba8004850 0x7f5bbc0b68e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:44.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:44.607+0000 7f5bbb7fe700 1 --2- 172.21.15.16:0/1782259692 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5bbc15bd30 0x7f5bbc15c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:44.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:44.607+0000 7f5bc0d35700 1 -- 172.21.15.16:0/1782259692 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5bbc15bd30 msgr2=0x7f5bbc15c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:44.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:44.607+0000 7f5bc0d35700 1 --2- 172.21.15.16:0/1782259692 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5bbc15bd30 0x7f5bbc15c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:44.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:44.607+0000 7f5bc0d35700 1 -- 172.21.15.16:0/1782259692 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bbc0d0c70 con 0x7f5bbc0c6990 2023-05-20T01:19:44.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:44.607+0000 7f5bc0d35700 1 --2- 172.21.15.16:0/1782259692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5bbc0c6990 0x7f5bbc0c6e00 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f5bb4011bf0 tx=0x7f5bb400f1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:44.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:44.607+0000 7f5bb8ff9700 1 -- 172.21.15.16:0/1782259692 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb400e040 con 0x7f5bbc0c6990 2023-05-20T01:19:44.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:44.607+0000 7f5bb8ff9700 1 -- 172.21.15.16:0/1782259692 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bb400f040 con 0x7f5bbc0c6990 2023-05-20T01:19:44.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:44.607+0000 7f5bc2798700 1 -- 172.21.15.16:0/1782259692 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bbc0cf920 con 0x7f5bbc0c6990 2023-05-20T01:19:44.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:44.607+0000 7f5bb8ff9700 1 -- 172.21.15.16:0/1782259692 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb400fe50 con 0x7f5bbc0c6990 2023-05-20T01:19:44.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:44.607+0000 7f5bc2798700 1 -- 172.21.15.16:0/1782259692 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bbc15d110 con 0x7f5bbc0c6990 2023-05-20T01:19:44.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:44.608+0000 7f5bc2798700 1 -- 172.21.15.16:0/1782259692 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bbc06cd50 con 0x7f5bbc0c6990 2023-05-20T01:19:44.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:44.609+0000 7f5bb8ff9700 1 -- 172.21.15.16:0/1782259692 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5bb4022460 con 0x7f5bbc0c6990 2023-05-20T01:19:44.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:44.609+0000 7f5bb8ff9700 1 --2- 172.21.15.16:0/1782259692 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f5b9c06ace0 0x7f5b9c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:44.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:44.609+0000 7f5bbbfff700 1 --2- 172.21.15.16:0/1782259692 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f5b9c06ace0 0x7f5b9c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:44.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:44.609+0000 7f5bb8ff9700 1 -- 172.21.15.16:0/1782259692 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5874+0+0 (secure 0 0 0) 0x7f5bb40899b0 con 0x7f5bbc0c6990 2023-05-20T01:19:44.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:44.609+0000 7f5bbbfff700 1 --2- 172.21.15.16:0/1782259692 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f5b9c06ace0 0x7f5b9c06d190 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f5bac001050 tx=0x7f5bac009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:44.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:44.614+0000 7f5bb8ff9700 1 -- 172.21.15.16:0/1782259692 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f5bb4058630 con 0x7f5bbc0c6990 2023-05-20T01:19:44.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:44.892+0000 7f5bc2798700 1 -- 172.21.15.16:0/1782259692 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_data_scan.TestDataScan.test_extra_data_pool_rebuild_many_files_many_workers"]} v 0) v1 -- 0x7f5bbc0afd30 con 0x7f5bbc0c6990 2023-05-20T01:19:45.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:45.464+0000 7f5bb8ff9700 1 -- 172.21.15.16:0/1782259692 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_data_scan.TestDataScan.test_extra_data_pool_rebuild_many_files_many_workers"]}]=0 v173) v1 ==== 175+0+0 (secure 0 0 0) 0x7f5bb40581c0 con 0x7f5bbc0c6990 2023-05-20T01:19:45.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:45.467+0000 7f5bc2798700 1 -- 172.21.15.16:0/1782259692 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f5b9c06ace0 msgr2=0x7f5b9c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:45.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:45.467+0000 7f5bc2798700 1 --2- 172.21.15.16:0/1782259692 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f5b9c06ace0 0x7f5b9c06d190 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f5bac001050 tx=0x7f5bac009040 comp rx=0 tx=0).stop 2023-05-20T01:19:45.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:45.467+0000 7f5bc2798700 1 -- 172.21.15.16:0/1782259692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5bbc0c6990 msgr2=0x7f5bbc0c6e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:45.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:45.467+0000 7f5bc2798700 1 --2- 172.21.15.16:0/1782259692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5bbc0c6990 0x7f5bbc0c6e00 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f5bb4011bf0 tx=0x7f5bb400f1b0 comp rx=0 tx=0).stop 2023-05-20T01:19:45.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:45.467+0000 7f5bc2798700 1 -- 172.21.15.16:0/1782259692 shutdown_connections 2023-05-20T01:19:45.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:45.467+0000 7f5bc2798700 1 --2- 172.21.15.16:0/1782259692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5bbc0c6990 0x7f5bbc0c6e00 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:45.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:45.467+0000 7f5bc2798700 1 --2- 172.21.15.16:0/1782259692 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f5b9c06ace0 0x7f5b9c06d190 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:45.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:45.467+0000 7f5bc2798700 1 --2- 172.21.15.16:0/1782259692 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5ba8004850 0x7f5bbc0b68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:45.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:45.467+0000 7f5bc2798700 1 --2- 172.21.15.16:0/1782259692 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5bbc15bd30 0x7f5bbc15c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:45.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:45.467+0000 7f5bc2798700 1 -- 172.21.15.16:0/1782259692 >> 172.21.15.16:0/1782259692 conn(0x7f5bbc06ae30 msgr2=0x7f5bbc0d1020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:19:45.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:45.467+0000 7f5bc2798700 1 -- 172.21.15.16:0/1782259692 shutdown_connections 2023-05-20T01:19:45.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:45.468+0000 7f5bc2798700 1 -- 172.21.15.16:0/1782259692 wait complete. 2023-05-20T01:19:45.486 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-05-20T01:19:45.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:45.617+0000 7feadb006700 1 Processor -- start 2023-05-20T01:19:45.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:45.617+0000 7feadb006700 1 -- start start 2023-05-20T01:19:45.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:45.618+0000 7feadb006700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fead415cec0 0x7fead40cd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:45.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:45.618+0000 7feadb006700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fead406e190 0x7fead40cf1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:45.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:45.618+0000 7feadb006700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fead406a510 0x7fead406a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:45.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:45.618+0000 7feadb006700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fead406aed0 con 0x7fead406a510 2023-05-20T01:19:45.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:45.618+0000 7feadb006700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fead406dd00 con 0x7fead406e190 2023-05-20T01:19:45.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:45.618+0000 7feadb006700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fead40cfa60 con 0x7fead415cec0 2023-05-20T01:19:45.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:45.618+0000 7fead95a3700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fead406a510 0x7fead406a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:45.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:45.618+0000 7fead95a3700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fead406a510 0x7fead406a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33570/0 (socket says 172.21.15.16:33570) 2023-05-20T01:19:45.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:45.618+0000 7fead95a3700 1 -- 172.21.15.16:0/3907653250 learned_addr learned my addr 172.21.15.16:0/3907653250 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:19:45.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:45.618+0000 7fead8da2700 1 --2- 172.21.15.16:0/3907653250 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fead415cec0 0x7fead40cd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:45.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:45.618+0000 7fead3fff700 1 --2- 172.21.15.16:0/3907653250 >> v2:172.21.15.132:3300/0 conn(0x7fead406e190 0x7fead40cf1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:45.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:45.618+0000 7fead95a3700 1 -- 172.21.15.16:0/3907653250 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fead415cec0 msgr2=0x7fead40cd4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:19:45.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:45.618+0000 7fead95a3700 1 --2- 172.21.15.16:0/3907653250 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fead415cec0 0x7fead40cd4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:45.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:45.618+0000 7fead95a3700 1 -- 172.21.15.16:0/3907653250 >> v2:172.21.15.132:3300/0 conn(0x7fead406e190 msgr2=0x7fead40cf1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:19:45.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:45.618+0000 7fead95a3700 1 --2- 172.21.15.16:0/3907653250 >> v2:172.21.15.132:3300/0 conn(0x7fead406e190 0x7fead40cf1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:45.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:45.618+0000 7fead95a3700 1 -- 172.21.15.16:0/3907653250 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fead406d6b0 con 0x7fead406a510 2023-05-20T01:19:45.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:45.618+0000 7fead95a3700 1 --2- 172.21.15.16:0/3907653250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fead406a510 0x7fead406a990 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fead4067250 tx=0x7feacc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37668c22fd5b273e server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:45.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:45.619+0000 7fead37fe700 1 -- 172.21.15.16:0/3907653250 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feacc00e070 con 0x7fead406a510 2023-05-20T01:19:45.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:45.619+0000 7fead37fe700 1 -- 172.21.15.16:0/3907653250 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feacc00b910 con 0x7fead406a510 2023-05-20T01:19:45.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:45.619+0000 7fead37fe700 1 -- 172.21.15.16:0/3907653250 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feacc00bcc0 con 0x7fead406a510 2023-05-20T01:19:45.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:45.619+0000 7feadb006700 1 -- 172.21.15.16:0/3907653250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fead406a510 msgr2=0x7fead406a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:45.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:45.619+0000 7feadb006700 1 --2- 172.21.15.16:0/3907653250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fead406a510 0x7fead406a990 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fead4067250 tx=0x7feacc00b330 comp rx=0 tx=0).stop 2023-05-20T01:19:45.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:45.619+0000 7feadb006700 1 -- 172.21.15.16:0/3907653250 shutdown_connections 2023-05-20T01:19:45.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:45.619+0000 7feadb006700 1 --2- 172.21.15.16:0/3907653250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fead406a510 0x7fead406a990 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:45.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:45.619+0000 7feadb006700 1 --2- 172.21.15.16:0/3907653250 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fead415cec0 0x7fead40cd4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:45.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:45.619+0000 7feadb006700 1 --2- 172.21.15.16:0/3907653250 >> v2:172.21.15.132:3300/0 conn(0x7fead406e190 0x7fead40cf1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:45.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:45.619+0000 7feadb006700 1 -- 172.21.15.16:0/3907653250 >> 172.21.15.16:0/3907653250 conn(0x7fead406c260 msgr2=0x7fead40cffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:19:45.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:45.619+0000 7feadb006700 1 -- 172.21.15.16:0/3907653250 shutdown_connections 2023-05-20T01:19:45.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:45.619+0000 7feadb006700 1 -- 172.21.15.16:0/3907653250 wait complete. 2023-05-20T01:19:45.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:45.620+0000 7feadb006700 1 Processor -- start 2023-05-20T01:19:45.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:45.621+0000 7feadb006700 1 -- start start 2023-05-20T01:19:45.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:45.621+0000 7feadb006700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fead406a510 0x7fead40c6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:45.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:45.621+0000 7feadb006700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fead406e190 0x7fead40af960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:45.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:45.621+0000 7feadb006700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fead415cec0 0x7fead41e6a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:45.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:45.621+0000 7feadb006700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fead40afea0 con 0x7fead406a510 2023-05-20T01:19:45.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:45.621+0000 7feadb006700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fead40c5820 con 0x7fead406e190 2023-05-20T01:19:45.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:45.621+0000 7feadb006700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fead40c5b00 con 0x7fead415cec0 2023-05-20T01:19:45.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:45.622+0000 7fead8da2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fead406a510 0x7fead40c6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:45.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:45.622+0000 7fead8da2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fead406a510 0x7fead40c6680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33572/0 (socket says 172.21.15.16:33572) 2023-05-20T01:19:45.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:45.622+0000 7fead8da2700 1 -- 172.21.15.16:0/2035548539 learned_addr learned my addr 172.21.15.16:0/2035548539 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:19:45.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:45.622+0000 7fead3fff700 1 --2- 172.21.15.16:0/2035548539 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fead406e190 0x7fead40af960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:45.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:45.622+0000 7fead95a3700 1 --2- 172.21.15.16:0/2035548539 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fead415cec0 0x7fead41e6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:45.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:45.622+0000 7fead8da2700 1 -- 172.21.15.16:0/2035548539 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fead415cec0 msgr2=0x7fead41e6a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:45.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:45.622+0000 7fead8da2700 1 --2- 172.21.15.16:0/2035548539 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fead415cec0 0x7fead41e6a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:45.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:45.622+0000 7fead8da2700 1 -- 172.21.15.16:0/2035548539 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fead406e190 msgr2=0x7fead40af960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:45.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:45.622+0000 7fead8da2700 1 --2- 172.21.15.16:0/2035548539 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fead406e190 0x7fead40af960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:45.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:45.622+0000 7fead8da2700 1 -- 172.21.15.16:0/2035548539 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feacc009b30 con 0x7fead406a510 2023-05-20T01:19:45.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:45.622+0000 7fead3fff700 1 --2- 172.21.15.16:0/2035548539 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fead406e190 0x7fead40af960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:19:45.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:45.622+0000 7fead8da2700 1 --2- 172.21.15.16:0/2035548539 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fead406a510 0x7fead40c6680 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7feac800e9b0 tx=0x7feac800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:45.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:45.622+0000 7fead17fa700 1 -- 172.21.15.16:0/2035548539 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feac8009710 con 0x7fead406a510 2023-05-20T01:19:45.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:45.622+0000 7fead17fa700 1 -- 172.21.15.16:0/2035548539 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feac800c850 con 0x7fead406a510 2023-05-20T01:19:45.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:45.622+0000 7feadb006700 1 -- 172.21.15.16:0/2035548539 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fead415d290 con 0x7fead406a510 2023-05-20T01:19:45.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:45.622+0000 7fead17fa700 1 -- 172.21.15.16:0/2035548539 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feac800cbc0 con 0x7fead406a510 2023-05-20T01:19:45.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:45.622+0000 7feadb006700 1 -- 172.21.15.16:0/2035548539 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fead406d990 con 0x7fead406a510 2023-05-20T01:19:45.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:45.623+0000 7feadb006700 1 -- 172.21.15.16:0/2035548539 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fead40cfd40 con 0x7fead406a510 2023-05-20T01:19:45.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:45.624+0000 7fead17fa700 1 -- 172.21.15.16:0/2035548539 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7feac800cd20 con 0x7fead406a510 2023-05-20T01:19:45.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:45.624+0000 7fead17fa700 1 --2- 172.21.15.16:0/2035548539 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7feab806ace0 0x7feab806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:45.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:45.624+0000 7fead3fff700 1 --2- 172.21.15.16:0/2035548539 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7feab806ace0 0x7feab806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:45.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:45.624+0000 7fead17fa700 1 -- 172.21.15.16:0/2035548539 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5874+0+0 (secure 0 0 0) 0x7feac8089450 con 0x7fead406a510 2023-05-20T01:19:45.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:45.624+0000 7fead3fff700 1 --2- 172.21.15.16:0/2035548539 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7feab806ace0 0x7feab806d190 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fead40c6be0 tx=0x7feac0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:45.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:45.628+0000 7fead17fa700 1 -- 172.21.15.16:0/2035548539 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7feac80580d0 con 0x7fead406a510 2023-05-20T01:19:45.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:45.828+0000 7feadb006700 1 -- 172.21.15.16:0/2035548539 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fead40747b0 con 0x7fead406a510 2023-05-20T01:19:45.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:45.831+0000 7fead17fa700 1 -- 172.21.15.16:0/2035548539 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7feac8057c60 con 0x7fead406a510 2023-05-20T01:19:45.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:45.833+0000 7feadb006700 1 -- 172.21.15.16:0/2035548539 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7feab806ace0 msgr2=0x7feab806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:45.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:45.833+0000 7feadb006700 1 --2- 172.21.15.16:0/2035548539 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7feab806ace0 0x7feab806d190 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fead40c6be0 tx=0x7feac0008040 comp rx=0 tx=0).stop 2023-05-20T01:19:45.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:45.833+0000 7feadb006700 1 -- 172.21.15.16:0/2035548539 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fead406a510 msgr2=0x7fead40c6680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:45.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:45.833+0000 7feadb006700 1 --2- 172.21.15.16:0/2035548539 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fead406a510 0x7fead40c6680 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7feac800e9b0 tx=0x7feac800ed70 comp rx=0 tx=0).stop 2023-05-20T01:19:45.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:45.833+0000 7feadb006700 1 -- 172.21.15.16:0/2035548539 shutdown_connections 2023-05-20T01:19:45.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:45.833+0000 7feadb006700 1 --2- 172.21.15.16:0/2035548539 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7feab806ace0 0x7feab806d190 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:45.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:45.833+0000 7feadb006700 1 --2- 172.21.15.16:0/2035548539 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fead415cec0 0x7fead41e6a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:45.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:45.833+0000 7feadb006700 1 --2- 172.21.15.16:0/2035548539 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fead406a510 0x7fead40c6680 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:45.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:45.833+0000 7feadb006700 1 --2- 172.21.15.16:0/2035548539 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fead406e190 0x7fead40af960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:45.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:45.833+0000 7feadb006700 1 -- 172.21.15.16:0/2035548539 >> 172.21.15.16:0/2035548539 conn(0x7fead406c260 msgr2=0x7fead40cc210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:19:45.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:45.833+0000 7feadb006700 1 -- 172.21.15.16:0/2035548539 shutdown_connections 2023-05-20T01:19:45.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:45.833+0000 7feadb006700 1 -- 172.21.15.16:0/2035548539 wait complete. 2023-05-20T01:19:45.849 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-05-20T01:19:45.850 DEBUG:teuthology.orchestra.run.smithi016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-05-20T01:19:45.889 DEBUG:teuthology.orchestra.run.smithi132:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-05-20T01:19:45.939 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-05-20T01:19:45.940 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-05-20T01:19:45.941 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2023-05-20T01:19:45.958 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16287364k,nr_inodes=4071841,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=38,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16696 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3261172k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-05-20T01:19:45.958 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2023-05-20T01:19:45.959 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/proc/self/mounts of=/dev/stdout 2023-05-20T01:19:45.979 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284284k,nr_inodes=4071071,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=45,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16175 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260556k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-05-20T01:19:45.980 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-05-20T01:19:45.980 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2023-05-20T01:19:46.015 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16287364k,nr_inodes=4071841,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=38,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16696 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3261172k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-05-20T01:19:46.016 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2023-05-20T01:19:46.016 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/proc/self/mounts of=/dev/stdout 2023-05-20T01:19:46.039 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284284k,nr_inodes=4071071,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=45,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16175 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260556k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-05-20T01:19:46.040 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-05-20T01:19:46.040 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-20T01:19:46.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.175+0000 7f08c43d0700 1 Processor -- start 2023-05-20T01:19:46.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.175+0000 7f08c43d0700 1 -- start start 2023-05-20T01:19:46.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.176+0000 7f08c43d0700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f08bc0d2c10 0x7f08bc0ce080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:46.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.176+0000 7f08c43d0700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08bc0b3b50 con 0x7f08bc0b4750 2023-05-20T01:19:46.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.176+0000 7f08c43d0700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08bc0d0b90 con 0x7f08bc0cfc50 2023-05-20T01:19:46.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.176+0000 7f08c43d0700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08bc0cfad0 con 0x7f08bc0d2c10 2023-05-20T01:19:46.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.176+0000 7f08c296d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f08bc0d2c10 0x7f08bc0ce080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:46.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.176+0000 7f08c196b700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f08bc0b4750 0x7f08bc0cd970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:45108/0 (socket says 172.21.15.16:45108) 2023-05-20T01:19:46.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.177+0000 7f08c196b700 1 -- 172.21.15.16:0/1384355538 learned_addr learned my addr 172.21.15.16:0/1384355538 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:19:46.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.177+0000 7f08c296d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f08bc0d2c10 0x7f08bc0ce080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33584/0 (socket says 172.21.15.16:33584) 2023-05-20T01:19:46.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.177+0000 7f08c296d700 1 -- 172.21.15.16:0/1384355538 >> v1:172.21.15.132:6789/0 conn(0x7f08bc0cfc50 legacy=0x7f08bc0d0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:19:46.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.177+0000 7f08c296d700 1 -- 172.21.15.16:0/1384355538 >> v1:172.21.15.16:6789/0 conn(0x7f08bc0b4750 legacy=0x7f08bc0cd970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:46.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.177+0000 7f08c116a700 1 -- 172.21.15.16:0/1384355538 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1861179619 0 0) 0x7f08bc0b3b50 con 0x7f08bc0b4750 2023-05-20T01:19:46.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.177+0000 7f08c296d700 1 -- 172.21.15.16:0/1384355538 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08bc0d3030 con 0x7f08bc0d2c10 2023-05-20T01:19:46.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.177+0000 7f08c296d700 1 --2- 172.21.15.16:0/1384355538 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08bc0d2c10 0x7f08bc0ce080 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f08b4009d90 tx=0x7f08b400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cf28ce3fc36cbd8 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:46.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.177+0000 7f08c116a700 1 -- 172.21.15.16:0/1384355538 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b400e070 con 0x7f08bc0d2c10 2023-05-20T01:19:46.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.177+0000 7f08c116a700 1 -- 172.21.15.16:0/1384355538 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08b400bbd0 con 0x7f08bc0d2c10 2023-05-20T01:19:46.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.177+0000 7f08c116a700 1 -- 172.21.15.16:0/1384355538 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b4012430 con 0x7f08bc0d2c10 2023-05-20T01:19:46.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.178+0000 7f08c43d0700 1 -- 172.21.15.16:0/1384355538 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08bc0d2c10 msgr2=0x7f08bc0ce080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:46.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.178+0000 7f08c43d0700 1 --2- 172.21.15.16:0/1384355538 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08bc0d2c10 0x7f08bc0ce080 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f08b4009d90 tx=0x7f08b400b330 comp rx=0 tx=0).stop 2023-05-20T01:19:46.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.178+0000 7f08c43d0700 1 -- 172.21.15.16:0/1384355538 shutdown_connections 2023-05-20T01:19:46.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.178+0000 7f08c43d0700 1 --2- 172.21.15.16:0/1384355538 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08bc0d2c10 0x7f08bc0ce080 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:46.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.178+0000 7f08c43d0700 1 -- 172.21.15.16:0/1384355538 >> 172.21.15.16:0/1384355538 conn(0x7f08bc09c590 msgr2=0x7f08bc0d3240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:19:46.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.178+0000 7f08c43d0700 1 -- 172.21.15.16:0/1384355538 shutdown_connections 2023-05-20T01:19:46.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.178+0000 7f08c43d0700 1 -- 172.21.15.16:0/1384355538 wait complete. 2023-05-20T01:19:46.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.179+0000 7f08c43d0700 1 Processor -- start 2023-05-20T01:19:46.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.179+0000 7f08c43d0700 1 -- start start 2023-05-20T01:19:46.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.180+0000 7f08c43d0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f08bc0b4750 0x7f08bc0b70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:46.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.180+0000 7f08c43d0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f08bc0cfc50 0x7f08bc14e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:46.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.180+0000 7f08c43d0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08bc0d2c10 0x7f08bc0b2b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:46.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.180+0000 7f08c43d0700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08bc0b7600 con 0x7f08bc0d2c10 2023-05-20T01:19:46.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.180+0000 7f08c43d0700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08bc0b88b0 con 0x7f08bc0b4750 2023-05-20T01:19:46.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.180+0000 7f08c43d0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08bc0b7810 con 0x7f08bc0cfc50 2023-05-20T01:19:46.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.181+0000 7f08c296d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08bc0d2c10 0x7f08bc0b2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:46.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.181+0000 7f08c296d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08bc0d2c10 0x7f08bc0b2b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33590/0 (socket says 172.21.15.16:33590) 2023-05-20T01:19:46.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.181+0000 7f08c296d700 1 -- 172.21.15.16:0/2400362577 learned_addr learned my addr 172.21.15.16:0/2400362577 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:19:46.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.181+0000 7f08c196b700 1 --2- 172.21.15.16:0/2400362577 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f08bc0cfc50 0x7f08bc14e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:46.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.181+0000 7f08c216c700 1 --2- 172.21.15.16:0/2400362577 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f08bc0b4750 0x7f08bc0b70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:46.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.181+0000 7f08c296d700 1 -- 172.21.15.16:0/2400362577 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f08bc0cfc50 msgr2=0x7f08bc14e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:46.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.181+0000 7f08c296d700 1 --2- 172.21.15.16:0/2400362577 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f08bc0cfc50 0x7f08bc14e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:46.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.181+0000 7f08c296d700 1 -- 172.21.15.16:0/2400362577 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f08bc0b4750 msgr2=0x7f08bc0b70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:46.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.181+0000 7f08c296d700 1 --2- 172.21.15.16:0/2400362577 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f08bc0b4750 0x7f08bc0b70c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:46.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.181+0000 7f08c296d700 1 -- 172.21.15.16:0/2400362577 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08b4009a70 con 0x7f08bc0d2c10 2023-05-20T01:19:46.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.181+0000 7f08c296d700 1 --2- 172.21.15.16:0/2400362577 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08bc0d2c10 0x7f08bc0b2b20 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f08b4014040 tx=0x7f08b4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:46.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.181+0000 7f08b2ffd700 1 -- 172.21.15.16:0/2400362577 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b401b810 con 0x7f08bc0d2c10 2023-05-20T01:19:46.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.181+0000 7f08b2ffd700 1 -- 172.21.15.16:0/2400362577 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08b4020070 con 0x7f08bc0d2c10 2023-05-20T01:19:46.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.181+0000 7f08c43d0700 1 -- 172.21.15.16:0/2400362577 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08bc0b4380 con 0x7f08bc0d2c10 2023-05-20T01:19:46.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.181+0000 7f08b2ffd700 1 -- 172.21.15.16:0/2400362577 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b401bbb0 con 0x7f08bc0d2c10 2023-05-20T01:19:46.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.181+0000 7f08c43d0700 1 -- 172.21.15.16:0/2400362577 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08bc15d0b0 con 0x7f08bc0d2c10 2023-05-20T01:19:46.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.182+0000 7f08c43d0700 1 -- 172.21.15.16:0/2400362577 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08bc0d4000 con 0x7f08bc0d2c10 2023-05-20T01:19:46.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.183+0000 7f08b2ffd700 1 -- 172.21.15.16:0/2400362577 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f08b401bd10 con 0x7f08bc0d2c10 2023-05-20T01:19:46.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.183+0000 7f08b2ffd700 1 --2- 172.21.15.16:0/2400362577 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f08a006ace0 0x7f08a006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:46.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.183+0000 7f08b2ffd700 1 -- 172.21.15.16:0/2400362577 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5874+0+0 (secure 0 0 0) 0x7f08b4022070 con 0x7f08bc0d2c10 2023-05-20T01:19:46.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.183+0000 7f08c216c700 1 --2- 172.21.15.16:0/2400362577 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f08a006ace0 0x7f08a006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:46.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.184+0000 7f08c216c700 1 --2- 172.21.15.16:0/2400362577 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f08a006ace0 0x7f08a006d190 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f08ac000f60 tx=0x7f08ac009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:46.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.188+0000 7f08b2ffd700 1 -- 172.21.15.16:0/2400362577 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f08b4056a00 con 0x7f08bc0d2c10 2023-05-20T01:19:46.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.427+0000 7f08c43d0700 1 -- 172.21.15.16:0/2400362577 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f08bc0747b0 con 0x7f08bc0d2c10 2023-05-20T01:19:46.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.427+0000 7f08b2ffd700 1 -- 172.21.15.16:0/2400362577 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+3532 (secure 0 0 0) 0x7f08b405a020 con 0x7f08bc0d2c10 2023-05-20T01:19:46.431 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:19:46.432 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":31,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5246,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/2629811751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":2629811751},{"type":"v1","addr":"172.21.15.132:6834","nonce":2629811751}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":5249,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2117148777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":2117148777},{"type":"v1","addr":"172.21.15.132:6835","nonce":2117148777}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":5566,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2025052254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":2025052254},{"type":"v1","addr":"172.21.15.16:6837","nonce":2025052254}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":6129,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/393871247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":393871247},{"type":"v1","addr":"172.21.15.16:6836","nonce":393871247}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31}],"filesystems":[]} 2023-05-20T01:19:46.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.429+0000 7f08c43d0700 1 -- 172.21.15.16:0/2400362577 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f08a006ace0 msgr2=0x7f08a006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:46.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.429+0000 7f08c43d0700 1 --2- 172.21.15.16:0/2400362577 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f08a006ace0 0x7f08a006d190 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f08ac000f60 tx=0x7f08ac009040 comp rx=0 tx=0).stop 2023-05-20T01:19:46.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.429+0000 7f08c43d0700 1 -- 172.21.15.16:0/2400362577 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08bc0d2c10 msgr2=0x7f08bc0b2b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:46.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.429+0000 7f08c43d0700 1 --2- 172.21.15.16:0/2400362577 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08bc0d2c10 0x7f08bc0b2b20 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f08b4014040 tx=0x7f08b4004ad0 comp rx=0 tx=0).stop 2023-05-20T01:19:46.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.429+0000 7f08c43d0700 1 -- 172.21.15.16:0/2400362577 shutdown_connections 2023-05-20T01:19:46.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.429+0000 7f08c43d0700 1 --2- 172.21.15.16:0/2400362577 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08bc0d2c10 0x7f08bc0b2b20 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:46.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.429+0000 7f08c43d0700 1 --2- 172.21.15.16:0/2400362577 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f08bc0b4750 0x7f08bc0b70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:46.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.429+0000 7f08c43d0700 1 --2- 172.21.15.16:0/2400362577 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f08a006ace0 0x7f08a006d190 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:46.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.429+0000 7f08c43d0700 1 --2- 172.21.15.16:0/2400362577 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f08bc0cfc50 0x7f08bc14e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:46.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.429+0000 7f08c43d0700 1 -- 172.21.15.16:0/2400362577 >> 172.21.15.16:0/2400362577 conn(0x7f08bc09c590 msgr2=0x7f08bc0cda50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:19:46.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.429+0000 7f08c43d0700 1 -- 172.21.15.16:0/2400362577 shutdown_connections 2023-05-20T01:19:46.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.429+0000 7f08c43d0700 1 -- 172.21.15.16:0/2400362577 wait complete. 2023-05-20T01:19:46.435 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 31 2023-05-20T01:19:46.445 INFO:tasks.ceph.mds.a:Restarting daemon 2023-05-20T01:19:46.446 INFO:tasks.ceph.mds.a:Stopping old one... 2023-05-20T01:19:46.446 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-05-20T01:19:46.446 INFO:teuthology.orchestra.run:waiting for 300 2023-05-20T01:19:46.446 INFO:tasks.ceph.mds.c:Restarting daemon 2023-05-20T01:19:46.447 INFO:tasks.ceph.mds.c:Stopping old one... 2023-05-20T01:19:46.447 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-05-20T01:19:46.447 INFO:teuthology.orchestra.run:waiting for 300 2023-05-20T01:19:46.447 INFO:tasks.ceph.mds.b:Restarting daemon 2023-05-20T01:19:46.448 INFO:tasks.ceph.mds.b:Stopping old one... 2023-05-20T01:19:46.448 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-05-20T01:19:46.448 INFO:teuthology.orchestra.run:waiting for 300 2023-05-20T01:19:46.448 INFO:tasks.ceph.mds.d:Restarting daemon 2023-05-20T01:19:46.448 INFO:tasks.ceph.mds.d:Stopping old one... 2023-05-20T01:19:46.449 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-05-20T01:19:46.449 INFO:teuthology.orchestra.run:waiting for 300 2023-05-20T01:19:46.455 INFO:tasks.ceph.mds.a:Stopped 2023-05-20T01:19:46.455 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-05-20T01:19:46.456 INFO:tasks.ceph.mds.c:Stopped 2023-05-20T01:19:46.456 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-05-20T01:19:46.459 INFO:tasks.ceph.mds.b:Stopped 2023-05-20T01:19:46.459 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-05-20T01:19:46.460 INFO:tasks.ceph.mds.d:Stopped 2023-05-20T01:19:46.461 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-05-20T01:19:46.498 INFO:tasks.ceph.mds.a:Started 2023-05-20T01:19:46.499 DEBUG:teuthology.parallel:result is None 2023-05-20T01:19:46.500 INFO:tasks.ceph.mds.c:Started 2023-05-20T01:19:46.500 DEBUG:teuthology.parallel:result is None 2023-05-20T01:19:46.502 INFO:tasks.ceph.mds.b:Started 2023-05-20T01:19:46.503 DEBUG:teuthology.parallel:result is None 2023-05-20T01:19:46.503 INFO:tasks.ceph.mds.d:Started 2023-05-20T01:19:46.504 DEBUG:teuthology.parallel:result is None 2023-05-20T01:19:46.504 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-05-20T01:19:46.582 INFO:tasks.ceph.mds.a.smithi016.stdout:starting mds.a at 2023-05-20T01:19:46.582 INFO:tasks.ceph.mds.c.smithi016.stdout:starting mds.c at 2023-05-20T01:19:46.588 INFO:tasks.ceph.mds.d.smithi132.stdout:starting mds.d at 2023-05-20T01:19:46.589 INFO:tasks.ceph.mds.b.smithi132.stdout:starting mds.b at 2023-05-20T01:19:46.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.637+0000 7f00bda35700 1 Processor -- start 2023-05-20T01:19:46.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.637+0000 7f00bda35700 1 -- start start 2023-05-20T01:19:46.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.638+0000 7f00bda35700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f00b80cfc50 0x7f00b80d2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:46.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.638+0000 7f00bda35700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f00b80b4750 0x7f00b80d0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:46.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.638+0000 7f00bda35700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00b80d0b90 con 0x7f00b80b2b40 2023-05-20T01:19:46.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.638+0000 7f00bda35700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00b80cfad0 con 0x7f00b80cfc50 2023-05-20T01:19:46.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.638+0000 7f00bda35700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00b80d3df0 con 0x7f00b80b4750 2023-05-20T01:19:46.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.638+0000 7f00b67fc700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f00b80b4750 0x7f00b80d0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:46.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.638+0000 7f00b6ffd700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f00b80cfc50 0x7f00b80d2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:46.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.638+0000 7f00b77fe700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f00b80b2b40 0x7f00b80b2fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.16:54864/0 (socket says 172.21.15.16:54864) 2023-05-20T01:19:46.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.638+0000 7f00b77fe700 1 -- 172.21.15.16:0/943382072 learned_addr learned my addr 172.21.15.16:0/943382072 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:19:46.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.639+0000 7f00b5ffb700 1 -- 172.21.15.16:0/943382072 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3506615989 0 0) 0x7f00b80d0b90 con 0x7f00b80b2b40 2023-05-20T01:19:46.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.639+0000 7f00b5ffb700 1 -- 172.21.15.16:0/943382072 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00a00035b0 con 0x7f00b80b2b40 2023-05-20T01:19:46.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.639+0000 7f00b6ffd700 1 -- 172.21.15.16:0/943382072 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f00b80b4750 msgr2=0x7f00b80d0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:19:46.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.639+0000 7f00b6ffd700 1 --2- 172.21.15.16:0/943382072 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f00b80b4750 0x7f00b80d0020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:46.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.639+0000 7f00b6ffd700 1 -- 172.21.15.16:0/943382072 >> v1:172.21.15.132:6789/0 conn(0x7f00b80b2b40 legacy=0x7f00b80b2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:46.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.639+0000 7f00b6ffd700 1 -- 172.21.15.16:0/943382072 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00b80d0560 con 0x7f00b80cfc50 2023-05-20T01:19:46.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.639+0000 7f00b67fc700 1 --2- 172.21.15.16:0/943382072 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f00b80b4750 0x7f00b80d0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-05-20T01:19:46.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.639+0000 7f00b6ffd700 1 --2- 172.21.15.16:0/943382072 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f00b80cfc50 0x7f00b80d2c10 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f00ac009840 tx=0x7f00ac009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74a84943d6257e9 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:46.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.639+0000 7f00b5ffb700 1 -- 172.21.15.16:0/943382072 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00ac011830 con 0x7f00b80cfc50 2023-05-20T01:19:46.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.639+0000 7f00b5ffb700 1 -- 172.21.15.16:0/943382072 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00ac011990 con 0x7f00b80cfc50 2023-05-20T01:19:46.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.639+0000 7f00b5ffb700 1 -- 172.21.15.16:0/943382072 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00ac011d20 con 0x7f00b80cfc50 2023-05-20T01:19:46.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.640+0000 7f00bda35700 1 -- 172.21.15.16:0/943382072 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f00b80cfc50 msgr2=0x7f00b80d2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:46.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.640+0000 7f00bda35700 1 --2- 172.21.15.16:0/943382072 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f00b80cfc50 0x7f00b80d2c10 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f00ac009840 tx=0x7f00ac009b50 comp rx=0 tx=0).stop 2023-05-20T01:19:46.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.640+0000 7f00bda35700 1 -- 172.21.15.16:0/943382072 shutdown_connections 2023-05-20T01:19:46.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.640+0000 7f00bda35700 1 --2- 172.21.15.16:0/943382072 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f00b80cfc50 0x7f00b80d2c10 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:46.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.640+0000 7f00bda35700 1 --2- 172.21.15.16:0/943382072 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f00b80b4750 0x7f00b80d0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:46.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.640+0000 7f00bda35700 1 -- 172.21.15.16:0/943382072 >> 172.21.15.16:0/943382072 conn(0x7f00b809c590 msgr2=0x7f00b80d3240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:19:46.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.640+0000 7f00bda35700 1 -- 172.21.15.16:0/943382072 shutdown_connections 2023-05-20T01:19:46.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.640+0000 7f00bda35700 1 -- 172.21.15.16:0/943382072 wait complete. 2023-05-20T01:19:46.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.641+0000 7f00bda35700 1 Processor -- start 2023-05-20T01:19:46.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.641+0000 7f00bda35700 1 -- start start 2023-05-20T01:19:46.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.642+0000 7f00bda35700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f00b80b2b40 0x7f00b80afef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:46.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.642+0000 7f00bda35700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f00b80b4750 0x7f00b80b0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:46.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.642+0000 7f00bda35700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00b80cfc50 0x7f00b80b0970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:46.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.642+0000 7f00bda35700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00b80b0eb0 con 0x7f00b80cfc50 2023-05-20T01:19:46.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.642+0000 7f00bda35700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00b80b7ea0 con 0x7f00b80b4750 2023-05-20T01:19:46.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.642+0000 7f00bda35700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00b80b8180 con 0x7f00b80b2b40 2023-05-20T01:19:46.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.642+0000 7f00b77fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00b80cfc50 0x7f00b80b0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:46.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.642+0000 7f00b77fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00b80cfc50 0x7f00b80b0970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33628/0 (socket says 172.21.15.16:33628) 2023-05-20T01:19:46.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.642+0000 7f00b77fe700 1 -- 172.21.15.16:0/2051635860 learned_addr learned my addr 172.21.15.16:0/2051635860 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:19:46.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.642+0000 7f00b67fc700 1 --2- 172.21.15.16:0/2051635860 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f00b80b4750 0x7f00b80b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:46.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.642+0000 7f00b6ffd700 1 --2- 172.21.15.16:0/2051635860 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f00b80b2b40 0x7f00b80afef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:46.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.642+0000 7f00b77fe700 1 -- 172.21.15.16:0/2051635860 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f00b80b2b40 msgr2=0x7f00b80afef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:46.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.642+0000 7f00b77fe700 1 --2- 172.21.15.16:0/2051635860 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f00b80b2b40 0x7f00b80afef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:46.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.642+0000 7f00b77fe700 1 -- 172.21.15.16:0/2051635860 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f00b80b4750 msgr2=0x7f00b80b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:46.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.642+0000 7f00b77fe700 1 --2- 172.21.15.16:0/2051635860 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f00b80b4750 0x7f00b80b0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:46.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.642+0000 7f00b77fe700 1 -- 172.21.15.16:0/2051635860 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00a80094a0 con 0x7f00b80cfc50 2023-05-20T01:19:46.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.642+0000 7f00b67fc700 1 --2- 172.21.15.16:0/2051635860 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f00b80b4750 0x7f00b80b0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:19:46.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.642+0000 7f00b77fe700 1 --2- 172.21.15.16:0/2051635860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00b80cfc50 0x7f00b80b0970 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f00b000caa0 tx=0x7f00b000ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:46.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.643+0000 7f009ffff700 1 -- 172.21.15.16:0/2051635860 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00b000aa80 con 0x7f00b80cfc50 2023-05-20T01:19:46.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.643+0000 7f009ffff700 1 -- 172.21.15.16:0/2051635860 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00b000abe0 con 0x7f00b80cfc50 2023-05-20T01:19:46.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.643+0000 7f00bda35700 1 -- 172.21.15.16:0/2051635860 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00ac0094a0 con 0x7f00b80cfc50 2023-05-20T01:19:46.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.643+0000 7f009ffff700 1 -- 172.21.15.16:0/2051635860 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00b0012400 con 0x7f00b80cfc50 2023-05-20T01:19:46.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.643+0000 7f00bda35700 1 -- 172.21.15.16:0/2051635860 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00b80b4380 con 0x7f00b80cfc50 2023-05-20T01:19:46.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.644+0000 7f00bda35700 1 -- 172.21.15.16:0/2051635860 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00b815d0b0 con 0x7f00b80cfc50 2023-05-20T01:19:46.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.644+0000 7f009ffff700 1 -- 172.21.15.16:0/2051635860 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f00b001b3b0 con 0x7f00b80cfc50 2023-05-20T01:19:46.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.645+0000 7f009ffff700 1 --2- 172.21.15.16:0/2051635860 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f00a406ace0 0x7f00a406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:46.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.645+0000 7f00b6ffd700 1 --2- 172.21.15.16:0/2051635860 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f00a406ace0 0x7f00a406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:46.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.645+0000 7f009ffff700 1 -- 172.21.15.16:0/2051635860 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5874+0+0 (secure 0 0 0) 0x7f00b00891a0 con 0x7f00b80cfc50 2023-05-20T01:19:46.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.645+0000 7f00b6ffd700 1 --2- 172.21.15.16:0/2051635860 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f00a406ace0 0x7f00a406d190 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f00ac000c00 tx=0x7f00ac01c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:46.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.649+0000 7f009ffff700 1 -- 172.21.15.16:0/2051635860 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f00b0054300 con 0x7f00b80cfc50 2023-05-20T01:19:46.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:46.863+0000 7f00bda35700 1 -- 172.21.15.16:0/2051635860 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f00b80d4000 con 0x7f00b80cfc50 2023-05-20T01:19:47.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:47.623+0000 7f009ffff700 1 -- 172.21.15.16:0/2051635860 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v60) v1 ==== 97+0+0 (secure 0 0 0) 0x7f00b0057920 con 0x7f00b80cfc50 2023-05-20T01:19:47.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:47.626+0000 7f00bda35700 1 -- 172.21.15.16:0/2051635860 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f00a406ace0 msgr2=0x7f00a406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:47.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:47.626+0000 7f00bda35700 1 --2- 172.21.15.16:0/2051635860 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f00a406ace0 0x7f00a406d190 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f00ac000c00 tx=0x7f00ac01c040 comp rx=0 tx=0).stop 2023-05-20T01:19:47.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:47.626+0000 7f00bda35700 1 -- 172.21.15.16:0/2051635860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00b80cfc50 msgr2=0x7f00b80b0970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:47.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:47.626+0000 7f00bda35700 1 --2- 172.21.15.16:0/2051635860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00b80cfc50 0x7f00b80b0970 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f00b000caa0 tx=0x7f00b000ce60 comp rx=0 tx=0).stop 2023-05-20T01:19:47.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:47.626+0000 7f00bda35700 1 -- 172.21.15.16:0/2051635860 shutdown_connections 2023-05-20T01:19:47.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:47.626+0000 7f00bda35700 1 --2- 172.21.15.16:0/2051635860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00b80cfc50 0x7f00b80b0970 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:47.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:47.626+0000 7f00bda35700 1 --2- 172.21.15.16:0/2051635860 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f00a406ace0 0x7f00a406d190 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:47.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:47.626+0000 7f00bda35700 1 --2- 172.21.15.16:0/2051635860 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f00b80b2b40 0x7f00b80afef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:47.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:47.626+0000 7f00bda35700 1 --2- 172.21.15.16:0/2051635860 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f00b80b4750 0x7f00b80b0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:47.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:47.626+0000 7f00bda35700 1 -- 172.21.15.16:0/2051635860 >> 172.21.15.16:0/2051635860 conn(0x7f00b809c590 msgr2=0x7f00b806b710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:19:47.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:47.626+0000 7f00bda35700 1 -- 172.21.15.16:0/2051635860 shutdown_connections 2023-05-20T01:19:47.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:47.626+0000 7f00bda35700 1 -- 172.21.15.16:0/2051635860 wait complete. 2023-05-20T01:19:47.632 INFO:teuthology.orchestra.run.smithi016.stderr: removed all blocklist entries 2023-05-20T01:19:47.645 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-05-20T01:19:47.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:47.773+0000 7f02ba4a2700 1 Processor -- start 2023-05-20T01:19:47.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:47.773+0000 7f02ba4a2700 1 -- start start 2023-05-20T01:19:47.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:47.773+0000 7f02ba4a2700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f02b4157190 0x7f02b40cc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:47.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:47.773+0000 7f02ba4a2700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f02b40cccf0 0x7f02b40713c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:47.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:47.773+0000 7f02ba4a2700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02b40cb590 con 0x7f02b4071900 2023-05-20T01:19:47.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:47.773+0000 7f02ba4a2700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02b40ca9a0 con 0x7f02b4157190 2023-05-20T01:19:47.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:47.773+0000 7f02ba4a2700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02b40cac60 con 0x7f02b40cccf0 2023-05-20T01:19:47.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:47.774+0000 7f02b3fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f02b4157190 0x7f02b40cc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:47.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:47.774+0000 7f02b8a3f700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f02b4071900 0x7f02b415d220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:45168/0 (socket says 172.21.15.16:45168) 2023-05-20T01:19:47.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:47.774+0000 7f02b3fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f02b4157190 0x7f02b40cc7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33636/0 (socket says 172.21.15.16:33636) 2023-05-20T01:19:47.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:47.774+0000 7f02b8a3f700 1 -- 172.21.15.16:0/595400292 learned_addr learned my addr 172.21.15.16:0/595400292 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:19:47.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:47.774+0000 7f02b37fe700 1 --2- 172.21.15.16:0/595400292 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f02b40cccf0 0x7f02b40713c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:47.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:47.774+0000 7f02b3fff700 1 -- 172.21.15.16:0/595400292 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f02b40cccf0 msgr2=0x7f02b40713c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:19:47.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:47.774+0000 7f02b3fff700 1 --2- 172.21.15.16:0/595400292 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f02b40cccf0 0x7f02b40713c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:47.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:47.774+0000 7f02b3fff700 1 -- 172.21.15.16:0/595400292 >> v1:172.21.15.16:6789/0 conn(0x7f02b4071900 legacy=0x7f02b415d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:47.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:47.774+0000 7f02b2ffd700 1 -- 172.21.15.16:0/595400292 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1163258083 0 0) 0x7f02b40cb590 con 0x7f02b4071900 2023-05-20T01:19:47.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:47.774+0000 7f02b3fff700 1 -- 172.21.15.16:0/595400292 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02b40caf40 con 0x7f02b4157190 2023-05-20T01:19:47.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:47.774+0000 7f02b3fff700 1 --2- 172.21.15.16:0/595400292 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02b4157190 0x7f02b40cc7b0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f02a4009840 tx=0x7f02a4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2196bcb95825de3a server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:47.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:47.774+0000 7f02b2ffd700 1 -- 172.21.15.16:0/595400292 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02a4011830 con 0x7f02b4157190 2023-05-20T01:19:47.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:47.774+0000 7f02b2ffd700 1 -- 172.21.15.16:0/595400292 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02a4011990 con 0x7f02b4157190 2023-05-20T01:19:47.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:47.774+0000 7f02b2ffd700 1 -- 172.21.15.16:0/595400292 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02a4011d20 con 0x7f02b4157190 2023-05-20T01:19:47.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:47.775+0000 7f02ba4a2700 1 -- 172.21.15.16:0/595400292 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02b4157190 msgr2=0x7f02b40cc7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:47.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:47.775+0000 7f02ba4a2700 1 --2- 172.21.15.16:0/595400292 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02b4157190 0x7f02b40cc7b0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f02a4009840 tx=0x7f02a4009b50 comp rx=0 tx=0).stop 2023-05-20T01:19:47.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:47.775+0000 7f02ba4a2700 1 -- 172.21.15.16:0/595400292 shutdown_connections 2023-05-20T01:19:47.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:47.775+0000 7f02ba4a2700 1 --2- 172.21.15.16:0/595400292 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02b4157190 0x7f02b40cc7b0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:47.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:47.775+0000 7f02ba4a2700 1 --2- 172.21.15.16:0/595400292 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f02b40cccf0 0x7f02b40713c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:47.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:47.775+0000 7f02ba4a2700 1 -- 172.21.15.16:0/595400292 >> 172.21.15.16:0/595400292 conn(0x7f02b40cffb0 msgr2=0x7f02b40d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:19:47.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:47.775+0000 7f02ba4a2700 1 -- 172.21.15.16:0/595400292 shutdown_connections 2023-05-20T01:19:47.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:47.775+0000 7f02ba4a2700 1 -- 172.21.15.16:0/595400292 wait complete. 2023-05-20T01:19:47.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:47.776+0000 7f02ba4a2700 1 Processor -- start 2023-05-20T01:19:47.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:47.776+0000 7f02ba4a2700 1 -- start start 2023-05-20T01:19:47.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:47.777+0000 7f02ba4a2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02b4071900 0x7f02b4155250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:47.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:47.777+0000 7f02ba4a2700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f02b40cccf0 0x7f02b40b2e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:47.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:47.777+0000 7f02ba4a2700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f02b4157190 0x7f02b40b3380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:47.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:47.777+0000 7f02ba4a2700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02b40b5ae0 con 0x7f02b4071900 2023-05-20T01:19:47.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:47.777+0000 7f02ba4a2700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02b40afc50 con 0x7f02b4157190 2023-05-20T01:19:47.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:47.777+0000 7f02ba4a2700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02b40aff30 con 0x7f02b40cccf0 2023-05-20T01:19:47.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:47.777+0000 7f02b3fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02b4071900 0x7f02b4155250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:47.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:47.777+0000 7f02b3fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02b4071900 0x7f02b4155250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33642/0 (socket says 172.21.15.16:33642) 2023-05-20T01:19:47.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:47.777+0000 7f02b3fff700 1 -- 172.21.15.16:0/4065621110 learned_addr learned my addr 172.21.15.16:0/4065621110 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:19:47.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:47.777+0000 7f02b37fe700 1 --2- 172.21.15.16:0/4065621110 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f02b40cccf0 0x7f02b40b2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:47.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:47.777+0000 7f02b8a3f700 1 --2- 172.21.15.16:0/4065621110 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f02b4157190 0x7f02b40b3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:47.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:47.778+0000 7f02b3fff700 1 -- 172.21.15.16:0/4065621110 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f02b40cccf0 msgr2=0x7f02b40b2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:47.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:47.778+0000 7f02b3fff700 1 --2- 172.21.15.16:0/4065621110 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f02b40cccf0 0x7f02b40b2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:47.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:47.778+0000 7f02b3fff700 1 -- 172.21.15.16:0/4065621110 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f02b4157190 msgr2=0x7f02b40b3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:47.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:47.778+0000 7f02b3fff700 1 --2- 172.21.15.16:0/4065621110 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f02b4157190 0x7f02b40b3380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:47.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:47.778+0000 7f02b3fff700 1 -- 172.21.15.16:0/4065621110 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02a40094a0 con 0x7f02b4071900 2023-05-20T01:19:47.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:47.778+0000 7f02b8a3f700 1 --2- 172.21.15.16:0/4065621110 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f02b4157190 0x7f02b40b3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:19:47.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:47.778+0000 7f02b3fff700 1 --2- 172.21.15.16:0/4065621110 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02b4071900 0x7f02b4155250 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f02a4014040 tx=0x7f02a4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:47.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:47.778+0000 7f02b0ff9700 1 -- 172.21.15.16:0/4065621110 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02a40036a0 con 0x7f02b4071900 2023-05-20T01:19:47.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:47.778+0000 7f02b0ff9700 1 -- 172.21.15.16:0/4065621110 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02a401e070 con 0x7f02b4071900 2023-05-20T01:19:47.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:47.778+0000 7f02ba4a2700 1 -- 172.21.15.16:0/4065621110 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02b4157560 con 0x7f02b4071900 2023-05-20T01:19:47.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:47.778+0000 7f02b0ff9700 1 -- 172.21.15.16:0/4065621110 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02a4003a20 con 0x7f02b4071900 2023-05-20T01:19:47.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:47.778+0000 7f02ba4a2700 1 -- 172.21.15.16:0/4065621110 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02b40cb220 con 0x7f02b4071900 2023-05-20T01:19:47.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:47.779+0000 7f02ba4a2700 1 -- 172.21.15.16:0/4065621110 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02b40cd120 con 0x7f02b4071900 2023-05-20T01:19:47.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:47.779+0000 7f02b0ff9700 1 -- 172.21.15.16:0/4065621110 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f02a4003b80 con 0x7f02b4071900 2023-05-20T01:19:47.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:47.780+0000 7f02b0ff9700 1 --2- 172.21.15.16:0/4065621110 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f02a006ace0 0x7f02a006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:47.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:47.780+0000 7f02b0ff9700 1 -- 172.21.15.16:0/4065621110 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f02a408a1d0 con 0x7f02b4071900 2023-05-20T01:19:47.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:47.780+0000 7f02b37fe700 1 --2- 172.21.15.16:0/4065621110 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f02a006ace0 0x7f02a006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:47.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:47.780+0000 7f02b37fe700 1 --2- 172.21.15.16:0/4065621110 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f02a006ace0 0x7f02a006d190 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f02b40af690 tx=0x7f02a8009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:47.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:47.784+0000 7f02b0ff9700 1 -- 172.21.15.16:0/4065621110 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f02a4055520 con 0x7f02b4071900 2023-05-20T01:19:48.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.065+0000 7f02ba4a2700 1 -- 172.21.15.16:0/4065621110 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f02b40b0070 con 0x7f02b4071900 2023-05-20T01:19:48.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.066+0000 7f02b0ff9700 1 -- 172.21.15.16:0/4065621110 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v9) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f02b40b0070 con 0x7f02b4071900 2023-05-20T01:19:48.070 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:19:48.070 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-05-20T01:19:48.070 INFO:teuthology.orchestra.run.smithi016.stdout: "auth_dump": [ 2023-05-20T01:19:48.070 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-05-20T01:19:48.070 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.a", 2023-05-20T01:19:48.071 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQA5H2hkq5FEJBAA1qUUPSLmwo43hTQS7GSENA==", 2023-05-20T01:19:48.071 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-05-20T01:19:48.071 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-05-20T01:19:48.071 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2023-05-20T01:19:48.071 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2023-05-20T01:19:48.071 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-05-20T01:19:48.071 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:19:48.072 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-05-20T01:19:48.072 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-05-20T01:19:48.072 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.b", 2023-05-20T01:19:48.072 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQA5H2hk/2XSMBAAnGCIm7TJZsO9HyXwOWxO7w==", 2023-05-20T01:19:48.072 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-05-20T01:19:48.072 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-05-20T01:19:48.072 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2023-05-20T01:19:48.073 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2023-05-20T01:19:48.073 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-05-20T01:19:48.073 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:19:48.073 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-05-20T01:19:48.073 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-05-20T01:19:48.073 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.c", 2023-05-20T01:19:48.073 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQA5H2hkBVK8KxAAgxy9+gbttxZeygwrAX+PHQ==", 2023-05-20T01:19:48.074 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-05-20T01:19:48.074 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-05-20T01:19:48.076 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2023-05-20T01:19:48.076 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2023-05-20T01:19:48.076 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-05-20T01:19:48.076 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:19:48.076 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-05-20T01:19:48.077 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-05-20T01:19:48.077 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.d", 2023-05-20T01:19:48.077 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQA5H2hkFxEMOBAAMPvue7F+/qqOFUD/C//U+g==", 2023-05-20T01:19:48.077 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-05-20T01:19:48.077 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-05-20T01:19:48.077 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2023-05-20T01:19:48.077 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2023-05-20T01:19:48.078 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-05-20T01:19:48.078 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:19:48.078 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-05-20T01:19:48.078 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-05-20T01:19:48.078 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.0", 2023-05-20T01:19:48.078 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQA7H2hkC4WwIRAA5KMQFqldcekzEuH92rZYHA==", 2023-05-20T01:19:48.078 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-05-20T01:19:48.079 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-05-20T01:19:48.079 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-05-20T01:19:48.079 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-05-20T01:19:48.079 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:19:48.079 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-05-20T01:19:48.079 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-05-20T01:19:48.079 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.1", 2023-05-20T01:19:48.080 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQA+H2hkvMH1IBAAhOQOjCiv3r2Pl3HcSS6eAA==", 2023-05-20T01:19:48.080 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-05-20T01:19:48.080 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-05-20T01:19:48.080 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-05-20T01:19:48.080 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-05-20T01:19:48.080 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:19:48.080 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-05-20T01:19:48.081 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-05-20T01:19:48.081 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.2", 2023-05-20T01:19:48.081 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQBBH2hkSJU5IBAAdFd4IhXxISVybFyT24gb0g==", 2023-05-20T01:19:48.081 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-05-20T01:19:48.081 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-05-20T01:19:48.081 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-05-20T01:19:48.081 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-05-20T01:19:48.082 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:19:48.082 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-05-20T01:19:48.082 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-05-20T01:19:48.082 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.3", 2023-05-20T01:19:48.082 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQBEH2hkrcH8HxAANlLWDyIYycLl1TrYCBN1hQ==", 2023-05-20T01:19:48.082 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-05-20T01:19:48.082 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-05-20T01:19:48.083 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-05-20T01:19:48.083 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-05-20T01:19:48.083 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:19:48.083 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-05-20T01:19:48.083 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-05-20T01:19:48.083 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.4", 2023-05-20T01:19:48.083 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQBKH2hkZr+4JBAAzbzKd4ulGI8kVGqXpjCDog==", 2023-05-20T01:19:48.084 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-05-20T01:19:48.084 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-05-20T01:19:48.084 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-05-20T01:19:48.084 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-05-20T01:19:48.084 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:19:48.085 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-05-20T01:19:48.085 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-05-20T01:19:48.085 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.5", 2023-05-20T01:19:48.085 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQBNH2hkNV02JBAALrC7eOTnD1dQM9njk+BgLQ==", 2023-05-20T01:19:48.085 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-05-20T01:19:48.085 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-05-20T01:19:48.086 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-05-20T01:19:48.086 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-05-20T01:19:48.086 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:19:48.086 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-05-20T01:19:48.086 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-05-20T01:19:48.086 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.6", 2023-05-20T01:19:48.086 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQBQH2hkGOfBIxAABCTVPLBi5+drKpqOC8xqdw==", 2023-05-20T01:19:48.087 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-05-20T01:19:48.087 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-05-20T01:19:48.087 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-05-20T01:19:48.087 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-05-20T01:19:48.087 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:19:48.087 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-05-20T01:19:48.087 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-05-20T01:19:48.088 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.7", 2023-05-20T01:19:48.088 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQBTH2hkpwEdIxAAaI3X5ZAVJ9MfVG0MjFJvSQ==", 2023-05-20T01:19:48.088 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-05-20T01:19:48.088 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-05-20T01:19:48.088 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-05-20T01:19:48.088 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-05-20T01:19:48.089 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:19:48.089 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-05-20T01:19:48.089 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-05-20T01:19:48.089 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.0", 2023-05-20T01:19:48.089 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQA6H2hkph3EABAA0277JHCqplw1LklDfidPWQ==", 2023-05-20T01:19:48.089 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-05-20T01:19:48.089 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-05-20T01:19:48.089 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow r", 2023-05-20T01:19:48.090 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-05-20T01:19:48.090 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:19:48.090 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-05-20T01:19:48.090 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-05-20T01:19:48.090 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.1", 2023-05-20T01:19:48.090 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQA6H2hk43WQBxAApSfBSu1qEaKSGWaYAvz/Jg==", 2023-05-20T01:19:48.090 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-05-20T01:19:48.091 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-05-20T01:19:48.091 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2023-05-20T01:19:48.091 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2023-05-20T01:19:48.091 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2023-05-20T01:19:48.091 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:19:48.091 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-05-20T01:19:48.092 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-05-20T01:19:48.092 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.2", 2023-05-20T01:19:48.092 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQA6H2hk+TgtBBAAc4+W2eOVgJfNd3xXjGvz4Q==", 2023-05-20T01:19:48.092 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-05-20T01:19:48.092 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-05-20T01:19:48.092 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2023-05-20T01:19:48.092 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2023-05-20T01:19:48.093 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2023-05-20T01:19:48.093 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:19:48.093 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-05-20T01:19:48.093 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-05-20T01:19:48.093 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.3", 2023-05-20T01:19:48.093 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQA6H2hkifVADRAAQP6vtdVNYyeNPV0d+agOCw==", 2023-05-20T01:19:48.093 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-05-20T01:19:48.094 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-05-20T01:19:48.094 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2023-05-20T01:19:48.094 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2023-05-20T01:19:48.094 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2023-05-20T01:19:48.094 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:19:48.094 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-05-20T01:19:48.094 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-05-20T01:19:48.094 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.admin", 2023-05-20T01:19:48.095 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQA5H2hkb4BMBhAAw7zpfDAb7x5szkLY+zTL7Q==", 2023-05-20T01:19:48.095 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-05-20T01:19:48.095 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2023-05-20T01:19:48.095 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2023-05-20T01:19:48.095 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2023-05-20T01:19:48.095 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-05-20T01:19:48.095 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:19:48.096 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-05-20T01:19:48.096 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-05-20T01:19:48.096 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mgr.x", 2023-05-20T01:19:48.096 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQA5H2hksQbMIBAAQFm/pY5stJHNudg6zcPBdQ==", 2023-05-20T01:19:48.096 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-05-20T01:19:48.096 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2023-05-20T01:19:48.096 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile mgr", 2023-05-20T01:19:48.097 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-05-20T01:19:48.097 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:19:48.097 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-05-20T01:19:48.097 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-05-20T01:19:48.097 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mgr.y", 2023-05-20T01:19:48.097 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQA5H2hkBQg7HBAAbOJZ+55rRACVBMd2eyjqtg==", 2023-05-20T01:19:48.097 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-05-20T01:19:48.098 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2023-05-20T01:19:48.098 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile mgr", 2023-05-20T01:19:48.098 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-05-20T01:19:48.098 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:19:48.098 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:19:48.098 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2023-05-20T01:19:48.098 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-05-20T01:19:48.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.068+0000 7f02ba4a2700 1 -- 172.21.15.16:0/4065621110 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f02a006ace0 msgr2=0x7f02a006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:48.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.068+0000 7f02ba4a2700 1 --2- 172.21.15.16:0/4065621110 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f02a006ace0 0x7f02a006d190 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f02b40af690 tx=0x7f02a8009210 comp rx=0 tx=0).stop 2023-05-20T01:19:48.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.068+0000 7f02ba4a2700 1 -- 172.21.15.16:0/4065621110 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02b4071900 msgr2=0x7f02b4155250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:48.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.068+0000 7f02ba4a2700 1 --2- 172.21.15.16:0/4065621110 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02b4071900 0x7f02b4155250 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f02a4014040 tx=0x7f02a4004750 comp rx=0 tx=0).stop 2023-05-20T01:19:48.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.068+0000 7f02ba4a2700 1 -- 172.21.15.16:0/4065621110 shutdown_connections 2023-05-20T01:19:48.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.068+0000 7f02ba4a2700 1 --2- 172.21.15.16:0/4065621110 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f02b4157190 0x7f02b40b3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:48.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.068+0000 7f02ba4a2700 1 --2- 172.21.15.16:0/4065621110 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02b4071900 0x7f02b4155250 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:48.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.068+0000 7f02ba4a2700 1 --2- 172.21.15.16:0/4065621110 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f02a006ace0 0x7f02a006d190 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:48.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.068+0000 7f02ba4a2700 1 --2- 172.21.15.16:0/4065621110 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f02b40cccf0 0x7f02b40b2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:48.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.068+0000 7f02ba4a2700 1 -- 172.21.15.16:0/4065621110 >> 172.21.15.16:0/4065621110 conn(0x7f02b40cffb0 msgr2=0x7f02b415c4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:19:48.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.068+0000 7f02ba4a2700 1 -- 172.21.15.16:0/4065621110 shutdown_connections 2023-05-20T01:19:48.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.068+0000 7f02ba4a2700 1 -- 172.21.15.16:0/4065621110 wait complete. 2023-05-20T01:19:48.102 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-20T01:19:48.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.223+0000 7fb7fcc77700 1 Processor -- start 2023-05-20T01:19:48.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.223+0000 7fb7fcc77700 1 -- start start 2023-05-20T01:19:48.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.224+0000 7fb7fcc77700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb7f815a9d0 0x7fb7f815efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:48.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.224+0000 7fb7fcc77700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fb7f80b2250 0x7fb7f806a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:48.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.224+0000 7fb7fcc77700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb7f806f850 0x7fb7f806aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:48.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.224+0000 7fb7fcc77700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7f80cc8d0 con 0x7fb7f806f850 2023-05-20T01:19:48.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.224+0000 7fb7fcc77700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7f806d940 con 0x7fb7f80b2250 2023-05-20T01:19:48.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.224+0000 7fb7fcc77700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f815f500 con 0x7fb7f815a9d0 2023-05-20T01:19:48.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.224+0000 7fb7f6d9d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb7f806f850 0x7fb7f806aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:48.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.224+0000 7fb7f6d9d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb7f806f850 0x7fb7f806aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33652/0 (socket says 172.21.15.16:33652) 2023-05-20T01:19:48.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.224+0000 7fb7f6d9d700 1 -- 172.21.15.16:0/3126224266 learned_addr learned my addr 172.21.15.16:0/3126224266 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:19:48.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.225+0000 7fb7f5d9b700 1 --2- 172.21.15.16:0/3126224266 >> v2:172.21.15.132:3300/0 conn(0x7fb7f80b2250 0x7fb7f806a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:48.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.225+0000 7fb7f659c700 1 --2- 172.21.15.16:0/3126224266 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb7f815a9d0 0x7fb7f815efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:48.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.225+0000 7fb7f6d9d700 1 -- 172.21.15.16:0/3126224266 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb7f815a9d0 msgr2=0x7fb7f815efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:19:48.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.225+0000 7fb7f6d9d700 1 --2- 172.21.15.16:0/3126224266 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb7f815a9d0 0x7fb7f815efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:48.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.225+0000 7fb7f6d9d700 1 -- 172.21.15.16:0/3126224266 >> v2:172.21.15.132:3300/0 conn(0x7fb7f80b2250 msgr2=0x7fb7f806a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:19:48.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.225+0000 7fb7f6d9d700 1 --2- 172.21.15.16:0/3126224266 >> v2:172.21.15.132:3300/0 conn(0x7fb7f80b2250 0x7fb7f806a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:48.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.225+0000 7fb7f6d9d700 1 -- 172.21.15.16:0/3126224266 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7f80cc030 con 0x7fb7f806f850 2023-05-20T01:19:48.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.225+0000 7fb7f6d9d700 1 --2- 172.21.15.16:0/3126224266 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7f806f850 0x7fb7f806aa50 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fb7f8066d20 tx=0x7fb7f000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dabc913bb8d72034 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:48.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.225+0000 7fb7f559a700 1 -- 172.21.15.16:0/3126224266 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7f000e070 con 0x7fb7f806f850 2023-05-20T01:19:48.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.225+0000 7fb7f559a700 1 -- 172.21.15.16:0/3126224266 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7f000b910 con 0x7fb7f806f850 2023-05-20T01:19:48.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.225+0000 7fb7f559a700 1 -- 172.21.15.16:0/3126224266 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7f000bcc0 con 0x7fb7f806f850 2023-05-20T01:19:48.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.226+0000 7fb7fcc77700 1 -- 172.21.15.16:0/3126224266 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7f806f850 msgr2=0x7fb7f806aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:48.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.226+0000 7fb7fcc77700 1 --2- 172.21.15.16:0/3126224266 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7f806f850 0x7fb7f806aa50 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fb7f8066d20 tx=0x7fb7f000b330 comp rx=0 tx=0).stop 2023-05-20T01:19:48.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.226+0000 7fb7fcc77700 1 -- 172.21.15.16:0/3126224266 shutdown_connections 2023-05-20T01:19:48.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.226+0000 7fb7fcc77700 1 --2- 172.21.15.16:0/3126224266 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7f806f850 0x7fb7f806aa50 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:48.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.226+0000 7fb7fcc77700 1 --2- 172.21.15.16:0/3126224266 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb7f815a9d0 0x7fb7f815efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:48.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.226+0000 7fb7fcc77700 1 --2- 172.21.15.16:0/3126224266 >> v2:172.21.15.132:3300/0 conn(0x7fb7f80b2250 0x7fb7f806a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:48.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.226+0000 7fb7fcc77700 1 -- 172.21.15.16:0/3126224266 >> 172.21.15.16:0/3126224266 conn(0x7fb7f80d0c30 msgr2=0x7fb7f80b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:19:48.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.226+0000 7fb7fcc77700 1 -- 172.21.15.16:0/3126224266 shutdown_connections 2023-05-20T01:19:48.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.226+0000 7fb7fcc77700 1 -- 172.21.15.16:0/3126224266 wait complete. 2023-05-20T01:19:48.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.228+0000 7fb7fcc77700 1 Processor -- start 2023-05-20T01:19:48.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.228+0000 7fb7fcc77700 1 -- start start 2023-05-20T01:19:48.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.229+0000 7fb7fcc77700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb7f806f850 0x7fb7f81eff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:48.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.229+0000 7fb7fcc77700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7f80b2250 0x7fb7f815bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:48.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.229+0000 7fb7fcc77700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb7f815a9d0 0x7fb7f815c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:48.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.229+0000 7fb7fcc77700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f8153dc0 con 0x7fb7f80b2250 2023-05-20T01:19:48.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.229+0000 7fb7fcc77700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f815c6e0 con 0x7fb7f815a9d0 2023-05-20T01:19:48.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.229+0000 7fb7fcc77700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f815c980 con 0x7fb7f806f850 2023-05-20T01:19:48.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.229+0000 7fb7f5d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7f80b2250 0x7fb7f815bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:48.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.229+0000 7fb7f5d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7f80b2250 0x7fb7f815bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33658/0 (socket says 172.21.15.16:33658) 2023-05-20T01:19:48.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.229+0000 7fb7f5d9b700 1 -- 172.21.15.16:0/496561199 learned_addr learned my addr 172.21.15.16:0/496561199 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:19:48.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.229+0000 7fb7f659c700 1 --2- 172.21.15.16:0/496561199 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb7f806f850 0x7fb7f81eff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:48.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.229+0000 7fb7f6d9d700 1 --2- 172.21.15.16:0/496561199 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb7f815a9d0 0x7fb7f815c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:48.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.229+0000 7fb7f5d9b700 1 -- 172.21.15.16:0/496561199 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb7f806f850 msgr2=0x7fb7f81eff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:48.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.229+0000 7fb7f5d9b700 1 --2- 172.21.15.16:0/496561199 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb7f806f850 0x7fb7f81eff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:48.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.229+0000 7fb7f5d9b700 1 -- 172.21.15.16:0/496561199 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb7f815a9d0 msgr2=0x7fb7f815c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:48.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.229+0000 7fb7f5d9b700 1 --2- 172.21.15.16:0/496561199 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb7f815a9d0 0x7fb7f815c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:48.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.229+0000 7fb7f5d9b700 1 -- 172.21.15.16:0/496561199 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7f0009b30 con 0x7fb7f80b2250 2023-05-20T01:19:48.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.229+0000 7fb7f5d9b700 1 --2- 172.21.15.16:0/496561199 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7f80b2250 0x7fb7f815bc60 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fb7e000e930 tx=0x7fb7e000ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:48.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.229+0000 7fb7eeffd700 1 -- 172.21.15.16:0/496561199 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e0013680 con 0x7fb7f80b2250 2023-05-20T01:19:48.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.229+0000 7fb7eeffd700 1 -- 172.21.15.16:0/496561199 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7e00137e0 con 0x7fb7f80b2250 2023-05-20T01:19:48.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.229+0000 7fb7fcc77700 1 -- 172.21.15.16:0/496561199 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7f80cc5f0 con 0x7fb7f80b2250 2023-05-20T01:19:48.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.229+0000 7fb7eeffd700 1 -- 172.21.15.16:0/496561199 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e0013b70 con 0x7fb7f80b2250 2023-05-20T01:19:48.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.229+0000 7fb7fcc77700 1 -- 172.21.15.16:0/496561199 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7f80cc310 con 0x7fb7f80b2250 2023-05-20T01:19:48.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.230+0000 7fb7fcc77700 1 -- 172.21.15.16:0/496561199 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7f815e7c0 con 0x7fb7f80b2250 2023-05-20T01:19:48.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.231+0000 7fb7eeffd700 1 -- 172.21.15.16:0/496561199 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb7e0013cd0 con 0x7fb7f80b2250 2023-05-20T01:19:48.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.231+0000 7fb7eeffd700 1 --2- 172.21.15.16:0/496561199 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb7d806ace0 0x7fb7d806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:48.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.231+0000 7fb7eeffd700 1 -- 172.21.15.16:0/496561199 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5530+0+0 (secure 0 0 0) 0x7fb7e0089120 con 0x7fb7f80b2250 2023-05-20T01:19:48.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.231+0000 7fb7f659c700 1 --2- 172.21.15.16:0/496561199 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb7d806ace0 0x7fb7d806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:48.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.232+0000 7fb7f659c700 1 --2- 172.21.15.16:0/496561199 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb7d806ace0 0x7fb7d806d190 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fb7e8009b40 tx=0x7fb7e8009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:48.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.236+0000 7fb7eeffd700 1 -- 172.21.15.16:0/496561199 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fb7e0054470 con 0x7fb7f80b2250 2023-05-20T01:19:48.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.434+0000 7fb7fcc77700 1 -- 172.21.15.16:0/496561199 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb7f80c40b0 con 0x7fb7f80b2250 2023-05-20T01:19:48.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.435+0000 7fb7eeffd700 1 -- 172.21.15.16:0/496561199 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v60) v1 ==== 74+0+14750 (secure 0 0 0) 0x7fb7f80c40b0 con 0x7fb7f80b2250 2023-05-20T01:19:48.439 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:19:48.440 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":60,"fsid":"c2245cc5-ecab-4087-a59b-bc81db901676","created":"2023-05-20T01:16:09.549230+0000","modified":"2023-05-20T01:19:47.618181+0000","last_up_change":"2023-05-20T01:16:19.788624+0000","last_in_change":"2023-05-20T01:16:12.092160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-05-20T01:16:20.934366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-05-20T01:16:21.372294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"317a83cd-19c8-4958-a301-6087ef342289","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6825","nonce":3923634179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6827","nonce":3923634179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6831","nonce":3923634179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6829","nonce":3923634179}]},"public_addr":"172.21.15.16:6825/3923634179","cluster_addr":"172.21.15.16:6827/3923634179","heartbeat_back_addr":"172.21.15.16:6831/3923634179","heartbeat_front_addr":"172.21.15.16:6829/3923634179","state":["exists","up"]},{"osd":1,"uuid":"5581046f-8219-4089-8d7d-b7846399505e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6809","nonce":3860264220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6811","nonce":3860264220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6815","nonce":3860264220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6813","nonce":3860264220}]},"public_addr":"172.21.15.16:6809/3860264220","cluster_addr":"172.21.15.16:6811/3860264220","heartbeat_back_addr":"172.21.15.16:6815/3860264220","heartbeat_front_addr":"172.21.15.16:6813/3860264220","state":["exists","up"]},{"osd":2,"uuid":"072e9e66-4133-478e-8d4a-64d46c2963f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6817","nonce":293187030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6819","nonce":293187030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6823","nonce":293187030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6821","nonce":293187030}]},"public_addr":"172.21.15.16:6817/293187030","cluster_addr":"172.21.15.16:6819/293187030","heartbeat_back_addr":"172.21.15.16:6823/293187030","heartbeat_front_addr":"172.21.15.16:6821/293187030","state":["exists","up"]},{"osd":3,"uuid":"11e1bba6-00f8-48dd-9028-d8d716b7f145","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6801","nonce":3669781890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6803","nonce":3669781890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6807","nonce":3669781890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6805","nonce":3669781890}]},"public_addr":"172.21.15.16:6801/3669781890","cluster_addr":"172.21.15.16:6803/3669781890","heartbeat_back_addr":"172.21.15.16:6807/3669781890","heartbeat_front_addr":"172.21.15.16:6805/3669781890","state":["exists","up"]},{"osd":4,"uuid":"b8909c3a-cf70-46cb-8d0b-009c707cf9d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6819","nonce":506317223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6823","nonce":506317223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6829","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6831","nonce":506317223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6827","nonce":506317223}]},"public_addr":"172.21.15.132:6819/506317223","cluster_addr":"172.21.15.132:6823/506317223","heartbeat_back_addr":"172.21.15.132:6831/506317223","heartbeat_front_addr":"172.21.15.132:6827/506317223","state":["exists","up"]},{"osd":5,"uuid":"b1818304-4583-47bf-a6ec-0bfbb5166233","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6818","nonce":1844689995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6822","nonce":1844689995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6830","nonce":1844689995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6826","nonce":1844689995}]},"public_addr":"172.21.15.132:6818/1844689995","cluster_addr":"172.21.15.132:6822/1844689995","heartbeat_back_addr":"172.21.15.132:6830/1844689995","heartbeat_front_addr":"172.21.15.132:6826/1844689995","state":["exists","up"]},{"osd":6,"uuid":"d37f4691-eed1-4a93-8818-18354f03f4a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6801","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6802","nonce":727581123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6805","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6807","nonce":727581123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6813","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6815","nonce":727581123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6811","nonce":727581123}]},"public_addr":"172.21.15.132:6802/727581123","cluster_addr":"172.21.15.132:6807/727581123","heartbeat_back_addr":"172.21.15.132:6815/727581123","heartbeat_front_addr":"172.21.15.132:6811/727581123","state":["exists","up"]},{"osd":7,"uuid":"4f712f62-5d1d-4816-8042-03209144f8b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6803","nonce":466001149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6806","nonce":466001149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6814","nonce":466001149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6810","nonce":466001149}]},"public_addr":"172.21.15.132:6803/466001149","cluster_addr":"172.21.15.132:6806/466001149","heartbeat_back_addr":"172.21.15.132:6814/466001149","heartbeat_front_addr":"172.21.15.132:6810/466001149","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.338863+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.278680+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305694+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.246835+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.276237+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.304441+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305421+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.267050+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-20T01:19:48.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.437+0000 7fb7fcc77700 1 -- 172.21.15.16:0/496561199 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb7d806ace0 msgr2=0x7fb7d806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:48.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.437+0000 7fb7fcc77700 1 --2- 172.21.15.16:0/496561199 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb7d806ace0 0x7fb7d806d190 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fb7e8009b40 tx=0x7fb7e8009210 comp rx=0 tx=0).stop 2023-05-20T01:19:48.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.437+0000 7fb7fcc77700 1 -- 172.21.15.16:0/496561199 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7f80b2250 msgr2=0x7fb7f815bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:48.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.437+0000 7fb7fcc77700 1 --2- 172.21.15.16:0/496561199 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7f80b2250 0x7fb7f815bc60 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fb7e000e930 tx=0x7fb7e000ec40 comp rx=0 tx=0).stop 2023-05-20T01:19:48.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.437+0000 7fb7fcc77700 1 -- 172.21.15.16:0/496561199 shutdown_connections 2023-05-20T01:19:48.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.437+0000 7fb7fcc77700 1 --2- 172.21.15.16:0/496561199 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb7f815a9d0 0x7fb7f815c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:48.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.437+0000 7fb7fcc77700 1 --2- 172.21.15.16:0/496561199 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb7d806ace0 0x7fb7d806d190 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:48.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.437+0000 7fb7fcc77700 1 --2- 172.21.15.16:0/496561199 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb7f806f850 0x7fb7f81eff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:48.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.437+0000 7fb7fcc77700 1 --2- 172.21.15.16:0/496561199 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7f80b2250 0x7fb7f815bc60 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:48.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.437+0000 7fb7fcc77700 1 -- 172.21.15.16:0/496561199 >> 172.21.15.16:0/496561199 conn(0x7fb7f80d0c30 msgr2=0x7fb7f80af950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:19:48.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.437+0000 7fb7fcc77700 1 -- 172.21.15.16:0/496561199 shutdown_connections 2023-05-20T01:19:48.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.437+0000 7fb7fcc77700 1 -- 172.21.15.16:0/496561199 wait complete. 2023-05-20T01:19:48.453 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-05-20T01:16:20.934366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-05-20T01:16:21.372294+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-05-20T01:19:48.454 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-05-20T01:19:48.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.580+0000 7ff855dc4700 1 Processor -- start 2023-05-20T01:19:48.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.580+0000 7ff855dc4700 1 -- start start 2023-05-20T01:19:48.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.581+0000 7ff855dc4700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff8500cd070 0x7ff85006e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:48.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.581+0000 7ff855dc4700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7ff8500710d0 0x7ff8500cabb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:48.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.581+0000 7ff855dc4700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff850072430 con 0x7ff85015cf40 2023-05-20T01:19:48.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.581+0000 7ff855dc4700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff850071570 con 0x7ff8500710d0 2023-05-20T01:19:48.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.581+0000 7ff855dc4700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff850071820 con 0x7ff8500cd070 2023-05-20T01:19:48.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.581+0000 7ff84effd700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff8500cd070 0x7ff85006e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:48.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.581+0000 7ff84ffff700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7ff8500710d0 0x7ff8500cabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:48.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.581+0000 7ff84f7fe700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7ff85015cf40 0x7ff8500cc960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.16:54906/0 (socket says 172.21.15.16:54906) 2023-05-20T01:19:48.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.581+0000 7ff84effd700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff8500cd070 0x7ff85006e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.16:33704/0 (socket says 172.21.15.16:33704) 2023-05-20T01:19:48.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.581+0000 7ff84f7fe700 1 -- 172.21.15.16:0/312931259 learned_addr learned my addr 172.21.15.16:0/312931259 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:19:48.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.582+0000 7ff84ffff700 1 -- 172.21.15.16:0/312931259 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff8500cd070 msgr2=0x7ff85006e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:19:48.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.582+0000 7ff84ffff700 1 --2- 172.21.15.16:0/312931259 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff8500cd070 0x7ff85006e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:48.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.582+0000 7ff84ffff700 1 -- 172.21.15.16:0/312931259 >> v1:172.21.15.132:6789/0 conn(0x7ff85015cf40 legacy=0x7ff8500cc960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:48.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.582+0000 7ff84e7fc700 1 -- 172.21.15.16:0/312931259 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 659960624 0 0) 0x7ff850072430 con 0x7ff85015cf40 2023-05-20T01:19:48.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.582+0000 7ff84ffff700 1 -- 172.21.15.16:0/312931259 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff850071b00 con 0x7ff8500710d0 2023-05-20T01:19:48.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.582+0000 7ff84effd700 1 --2- 172.21.15.16:0/312931259 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff8500cd070 0x7ff85006e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-05-20T01:19:48.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.582+0000 7ff84ffff700 1 --2- 172.21.15.16:0/312931259 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff8500710d0 0x7ff8500cabb0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff848009d90 tx=0x7ff84800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d2d98c8c45d64af server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:48.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.582+0000 7ff84e7fc700 1 -- 172.21.15.16:0/312931259 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff84800e070 con 0x7ff8500710d0 2023-05-20T01:19:48.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.582+0000 7ff84e7fc700 1 -- 172.21.15.16:0/312931259 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff84800bb20 con 0x7ff8500710d0 2023-05-20T01:19:48.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.582+0000 7ff84e7fc700 1 -- 172.21.15.16:0/312931259 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff848012430 con 0x7ff8500710d0 2023-05-20T01:19:48.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.582+0000 7ff855dc4700 1 -- 172.21.15.16:0/312931259 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff8500710d0 msgr2=0x7ff8500cabb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:48.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.582+0000 7ff855dc4700 1 --2- 172.21.15.16:0/312931259 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff8500710d0 0x7ff8500cabb0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff848009d90 tx=0x7ff84800b330 comp rx=0 tx=0).stop 2023-05-20T01:19:48.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.583+0000 7ff855dc4700 1 -- 172.21.15.16:0/312931259 shutdown_connections 2023-05-20T01:19:48.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.583+0000 7ff855dc4700 1 --2- 172.21.15.16:0/312931259 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff8500710d0 0x7ff8500cabb0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:48.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.583+0000 7ff855dc4700 1 --2- 172.21.15.16:0/312931259 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff8500cd070 0x7ff85006e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:48.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.583+0000 7ff855dc4700 1 -- 172.21.15.16:0/312931259 >> 172.21.15.16:0/312931259 conn(0x7ff85006c260 msgr2=0x7ff8500cffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:19:48.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.583+0000 7ff855dc4700 1 -- 172.21.15.16:0/312931259 shutdown_connections 2023-05-20T01:19:48.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.583+0000 7ff855dc4700 1 -- 172.21.15.16:0/312931259 wait complete. 2023-05-20T01:19:48.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.584+0000 7ff855dc4700 1 Processor -- start 2023-05-20T01:19:48.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.584+0000 7ff855dc4700 1 -- start start 2023-05-20T01:19:48.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.585+0000 7ff855dc4700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff8500710d0 0x7ff85014a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:48.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.585+0000 7ff855dc4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8500cd070 0x7ff850146560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:48.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.585+0000 7ff855dc4700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff85015cf40 0x7ff85015b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:48.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.585+0000 7ff855dc4700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8500b3e10 con 0x7ff8500cd070 2023-05-20T01:19:48.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.585+0000 7ff855dc4700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8501f41a0 con 0x7ff8500710d0 2023-05-20T01:19:48.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.585+0000 7ff855dc4700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8500b4310 con 0x7ff85015cf40 2023-05-20T01:19:48.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.585+0000 7ff84effd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8500cd070 0x7ff850146560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:48.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.585+0000 7ff84effd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8500cd070 0x7ff850146560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33672/0 (socket says 172.21.15.16:33672) 2023-05-20T01:19:48.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.585+0000 7ff84effd700 1 -- 172.21.15.16:0/2583675984 learned_addr learned my addr 172.21.15.16:0/2583675984 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:19:48.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.585+0000 7ff84f7fe700 1 --2- 172.21.15.16:0/2583675984 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff8500710d0 0x7ff85014a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:48.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.585+0000 7ff84ffff700 1 --2- 172.21.15.16:0/2583675984 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff85015cf40 0x7ff85015b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:48.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.585+0000 7ff84effd700 1 -- 172.21.15.16:0/2583675984 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff85015cf40 msgr2=0x7ff85015b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:48.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.585+0000 7ff84effd700 1 --2- 172.21.15.16:0/2583675984 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff85015cf40 0x7ff85015b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:48.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.585+0000 7ff84effd700 1 -- 172.21.15.16:0/2583675984 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff8500710d0 msgr2=0x7ff85014a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:48.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.585+0000 7ff84effd700 1 --2- 172.21.15.16:0/2583675984 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff8500710d0 0x7ff85014a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:48.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.585+0000 7ff84effd700 1 -- 172.21.15.16:0/2583675984 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8380094a0 con 0x7ff8500cd070 2023-05-20T01:19:48.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.585+0000 7ff84ffff700 1 --2- 172.21.15.16:0/2583675984 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff85015cf40 0x7ff85015b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:19:48.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.585+0000 7ff84effd700 1 --2- 172.21.15.16:0/2583675984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8500cd070 0x7ff850146560 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7ff83800ebb0 tx=0x7ff83800eec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:48.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.585+0000 7ff847fff700 1 -- 172.21.15.16:0/2583675984 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff83800c850 con 0x7ff8500cd070 2023-05-20T01:19:48.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.585+0000 7ff847fff700 1 -- 172.21.15.16:0/2583675984 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff83800c9b0 con 0x7ff8500cd070 2023-05-20T01:19:48.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.586+0000 7ff855dc4700 1 -- 172.21.15.16:0/2583675984 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff848009a70 con 0x7ff8500cd070 2023-05-20T01:19:48.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.586+0000 7ff847fff700 1 -- 172.21.15.16:0/2583675984 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff83800cd40 con 0x7ff8500cd070 2023-05-20T01:19:48.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.586+0000 7ff855dc4700 1 -- 172.21.15.16:0/2583675984 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8500720c0 con 0x7ff8500cd070 2023-05-20T01:19:48.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.586+0000 7ff855dc4700 1 -- 172.21.15.16:0/2583675984 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff850071de0 con 0x7ff8500cd070 2023-05-20T01:19:48.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.587+0000 7ff847fff700 1 -- 172.21.15.16:0/2583675984 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff838013470 con 0x7ff8500cd070 2023-05-20T01:19:48.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.587+0000 7ff847fff700 1 --2- 172.21.15.16:0/2583675984 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff83006ace0 0x7ff83006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:48.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.587+0000 7ff847fff700 1 -- 172.21.15.16:0/2583675984 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5530+0+0 (secure 0 0 0) 0x7ff8380892c0 con 0x7ff8500cd070 2023-05-20T01:19:48.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.587+0000 7ff84f7fe700 1 --2- 172.21.15.16:0/2583675984 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff83006ace0 0x7ff83006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:48.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.588+0000 7ff84f7fe700 1 --2- 172.21.15.16:0/2583675984 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff83006ace0 0x7ff83006d190 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7ff840000f60 tx=0x7ff840007e70 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:48.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.592+0000 7ff847fff700 1 -- 172.21.15.16:0/2583675984 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7ff838054610 con 0x7ff8500cd070 2023-05-20T01:19:48.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.788+0000 7ff855dc4700 1 -- 172.21.15.16:0/2583675984 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ff8500cb0f0 con 0x7ff8500cd070 2023-05-20T01:19:48.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.788+0000 7ff847fff700 1 -- 172.21.15.16:0/2583675984 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v60) v1 ==== 92+0+10 (secure 0 0 0) 0x7ff838057c30 con 0x7ff8500cd070 2023-05-20T01:19:48.794 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 8 2023-05-20T01:19:48.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.790+0000 7ff855dc4700 1 -- 172.21.15.16:0/2583675984 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff83006ace0 msgr2=0x7ff83006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:48.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.790+0000 7ff855dc4700 1 --2- 172.21.15.16:0/2583675984 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff83006ace0 0x7ff83006d190 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7ff840000f60 tx=0x7ff840007e70 comp rx=0 tx=0).stop 2023-05-20T01:19:48.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.790+0000 7ff855dc4700 1 -- 172.21.15.16:0/2583675984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8500cd070 msgr2=0x7ff850146560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:48.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.790+0000 7ff855dc4700 1 --2- 172.21.15.16:0/2583675984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8500cd070 0x7ff850146560 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7ff83800ebb0 tx=0x7ff83800eec0 comp rx=0 tx=0).stop 2023-05-20T01:19:48.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.790+0000 7ff855dc4700 1 -- 172.21.15.16:0/2583675984 shutdown_connections 2023-05-20T01:19:48.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.790+0000 7ff855dc4700 1 --2- 172.21.15.16:0/2583675984 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff83006ace0 0x7ff83006d190 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:48.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.790+0000 7ff855dc4700 1 --2- 172.21.15.16:0/2583675984 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff85015cf40 0x7ff85015b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:48.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.790+0000 7ff855dc4700 1 --2- 172.21.15.16:0/2583675984 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff8500710d0 0x7ff85014a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:48.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.790+0000 7ff855dc4700 1 --2- 172.21.15.16:0/2583675984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8500cd070 0x7ff850146560 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:48.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.790+0000 7ff855dc4700 1 -- 172.21.15.16:0/2583675984 >> 172.21.15.16:0/2583675984 conn(0x7ff85006c260 msgr2=0x7ff850154ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:19:48.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.790+0000 7ff855dc4700 1 -- 172.21.15.16:0/2583675984 shutdown_connections 2023-05-20T01:19:48.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.790+0000 7ff855dc4700 1 -- 172.21.15.16:0/2583675984 wait complete. 2023-05-20T01:19:48.807 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-05-20T01:19:48.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.932+0000 7fc812a0c700 1 Processor -- start 2023-05-20T01:19:48.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.932+0000 7fc812a0c700 1 -- start start 2023-05-20T01:19:48.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.932+0000 7fc812a0c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc80c15cec0 0x7fc80c071bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:48.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.932+0000 7fc812a0c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc80c072130 0x7fc80c0cf9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:48.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.932+0000 7fc812a0c700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc80c072590 con 0x7fc80c15d290 2023-05-20T01:19:48.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.932+0000 7fc812a0c700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc80c0cd630 con 0x7fc80c072130 2023-05-20T01:19:48.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.932+0000 7fc812a0c700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc80c0cca60 con 0x7fc80c15cec0 2023-05-20T01:19:48.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.933+0000 7fc810fa9700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc80c072130 0x7fc80c0cf9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:48.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.933+0000 7fc80b7fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fc80c15d290 0x7fc80c0cf2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:45206/0 (socket says 172.21.15.16:45206) 2023-05-20T01:19:48.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.933+0000 7fc80b7fe700 1 -- 172.21.15.16:0/1209008370 learned_addr learned my addr 172.21.15.16:0/1209008370 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:19:48.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.933+0000 7fc80bfff700 1 --2- 172.21.15.16:0/1209008370 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc80c15cec0 0x7fc80c071bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:48.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.933+0000 7fc810fa9700 1 -- 172.21.15.16:0/1209008370 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc80c15cec0 msgr2=0x7fc80c071bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:19:48.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.933+0000 7fc810fa9700 1 --2- 172.21.15.16:0/1209008370 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc80c15cec0 0x7fc80c071bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:48.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.933+0000 7fc810fa9700 1 -- 172.21.15.16:0/1209008370 >> v1:172.21.15.16:6789/0 conn(0x7fc80c15d290 legacy=0x7fc80c0cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:48.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.933+0000 7fc810fa9700 1 -- 172.21.15.16:0/1209008370 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc80c0ccd00 con 0x7fc80c072130 2023-05-20T01:19:48.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.933+0000 7fc80affd700 1 -- 172.21.15.16:0/1209008370 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 846035632 0 0) 0x7fc80c072590 con 0x7fc80c15d290 2023-05-20T01:19:48.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.933+0000 7fc80bfff700 1 --2- 172.21.15.16:0/1209008370 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc80c15cec0 0x7fc80c071bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:19:48.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.933+0000 7fc810fa9700 1 --2- 172.21.15.16:0/1209008370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc80c072130 0x7fc80c0cf9d0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fc804009d90 tx=0x7fc80400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d32053a8923d89a server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:48.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.933+0000 7fc80affd700 1 -- 172.21.15.16:0/1209008370 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc80400e070 con 0x7fc80c072130 2023-05-20T01:19:48.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.933+0000 7fc80affd700 1 -- 172.21.15.16:0/1209008370 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc80400bbd0 con 0x7fc80c072130 2023-05-20T01:19:48.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.933+0000 7fc80affd700 1 -- 172.21.15.16:0/1209008370 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc804012430 con 0x7fc80c072130 2023-05-20T01:19:48.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.934+0000 7fc812a0c700 1 -- 172.21.15.16:0/1209008370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc80c072130 msgr2=0x7fc80c0cf9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:48.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.934+0000 7fc812a0c700 1 --2- 172.21.15.16:0/1209008370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc80c072130 0x7fc80c0cf9d0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fc804009d90 tx=0x7fc80400b330 comp rx=0 tx=0).stop 2023-05-20T01:19:48.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.934+0000 7fc812a0c700 1 -- 172.21.15.16:0/1209008370 shutdown_connections 2023-05-20T01:19:48.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.934+0000 7fc812a0c700 1 --2- 172.21.15.16:0/1209008370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc80c072130 0x7fc80c0cf9d0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:48.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.934+0000 7fc812a0c700 1 --2- 172.21.15.16:0/1209008370 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc80c15cec0 0x7fc80c071bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:48.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.934+0000 7fc812a0c700 1 -- 172.21.15.16:0/1209008370 >> 172.21.15.16:0/1209008370 conn(0x7fc80c0cffd0 msgr2=0x7fc80c0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:19:48.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.934+0000 7fc812a0c700 1 -- 172.21.15.16:0/1209008370 shutdown_connections 2023-05-20T01:19:48.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.934+0000 7fc812a0c700 1 -- 172.21.15.16:0/1209008370 wait complete. 2023-05-20T01:19:48.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.935+0000 7fc812a0c700 1 Processor -- start 2023-05-20T01:19:48.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.935+0000 7fc812a0c700 1 -- start start 2023-05-20T01:19:48.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.936+0000 7fc812a0c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc80c072130 0x7fc80c1483f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:48.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.936+0000 7fc812a0c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc80c15cec0 0x7fc80c0c9580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:48.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.936+0000 7fc812a0c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc80c15d290 0x7fc80c15d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:48.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.936+0000 7fc812a0c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc80c148930 con 0x7fc80c15d290 2023-05-20T01:19:48.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.936+0000 7fc812a0c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc80c1f2970 con 0x7fc80c072130 2023-05-20T01:19:48.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.936+0000 7fc812a0c700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc80c0ca040 con 0x7fc80c15cec0 2023-05-20T01:19:48.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.936+0000 7fc810fa9700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc80c15d290 0x7fc80c15d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:48.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.936+0000 7fc810fa9700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc80c15d290 0x7fc80c15d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33688/0 (socket says 172.21.15.16:33688) 2023-05-20T01:19:48.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.936+0000 7fc810fa9700 1 -- 172.21.15.16:0/370044367 learned_addr learned my addr 172.21.15.16:0/370044367 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:19:48.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.936+0000 7fc80b7fe700 1 --2- 172.21.15.16:0/370044367 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc80c15cec0 0x7fc80c0c9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:48.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.936+0000 7fc80bfff700 1 --2- 172.21.15.16:0/370044367 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc80c072130 0x7fc80c1483f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:48.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.936+0000 7fc810fa9700 1 -- 172.21.15.16:0/370044367 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc80c15cec0 msgr2=0x7fc80c0c9580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:48.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.936+0000 7fc810fa9700 1 --2- 172.21.15.16:0/370044367 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc80c15cec0 0x7fc80c0c9580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:48.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.936+0000 7fc810fa9700 1 -- 172.21.15.16:0/370044367 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc80c072130 msgr2=0x7fc80c1483f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:48.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.936+0000 7fc810fa9700 1 --2- 172.21.15.16:0/370044367 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc80c072130 0x7fc80c1483f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:48.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.936+0000 7fc810fa9700 1 -- 172.21.15.16:0/370044367 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc804009a70 con 0x7fc80c15d290 2023-05-20T01:19:48.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.937+0000 7fc810fa9700 1 --2- 172.21.15.16:0/370044367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc80c15d290 0x7fc80c15d700 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fc80c0cdb50 tx=0x7fc804004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:48.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.937+0000 7fc808ff9700 1 -- 172.21.15.16:0/370044367 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc804012430 con 0x7fc80c15d290 2023-05-20T01:19:48.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.937+0000 7fc808ff9700 1 -- 172.21.15.16:0/370044367 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc804020070 con 0x7fc80c15d290 2023-05-20T01:19:48.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.937+0000 7fc812a0c700 1 -- 172.21.15.16:0/370044367 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc80c0cd2c0 con 0x7fc80c15d290 2023-05-20T01:19:48.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.937+0000 7fc808ff9700 1 -- 172.21.15.16:0/370044367 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc804004280 con 0x7fc80c15d290 2023-05-20T01:19:48.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.937+0000 7fc812a0c700 1 -- 172.21.15.16:0/370044367 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc80c0ccfe0 con 0x7fc80c15d290 2023-05-20T01:19:48.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.938+0000 7fc812a0c700 1 -- 172.21.15.16:0/370044367 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc80c160890 con 0x7fc80c15d290 2023-05-20T01:19:48.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.938+0000 7fc808ff9700 1 -- 172.21.15.16:0/370044367 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc8040043e0 con 0x7fc80c15d290 2023-05-20T01:19:48.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.939+0000 7fc808ff9700 1 --2- 172.21.15.16:0/370044367 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc7f806ace0 0x7fc7f806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:48.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.939+0000 7fc808ff9700 1 -- 172.21.15.16:0/370044367 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5530+0+0 (secure 0 0 0) 0x7fc804022070 con 0x7fc80c15d290 2023-05-20T01:19:48.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.939+0000 7fc80bfff700 1 --2- 172.21.15.16:0/370044367 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc7f806ace0 0x7fc7f806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:48.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.939+0000 7fc80bfff700 1 --2- 172.21.15.16:0/370044367 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc7f806ace0 0x7fc7f806d190 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fc80c15a730 tx=0x7fc7fc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:48.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:48.943+0000 7fc808ff9700 1 -- 172.21.15.16:0/370044367 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fc8040565a0 con 0x7fc80c15d290 2023-05-20T01:19:49.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.139+0000 7fc812a0c700 1 -- 172.21.15.16:0/370044367 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc80c0747b0 con 0x7fc80c15d290 2023-05-20T01:19:49.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.139+0000 7fc808ff9700 1 -- 172.21.15.16:0/370044367 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v60) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc804059bc0 con 0x7fc80c15d290 2023-05-20T01:19:49.144 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-05-20T01:19:49.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.141+0000 7fc812a0c700 1 -- 172.21.15.16:0/370044367 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc7f806ace0 msgr2=0x7fc7f806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:49.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.141+0000 7fc812a0c700 1 --2- 172.21.15.16:0/370044367 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc7f806ace0 0x7fc7f806d190 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fc80c15a730 tx=0x7fc7fc008040 comp rx=0 tx=0).stop 2023-05-20T01:19:49.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.141+0000 7fc812a0c700 1 -- 172.21.15.16:0/370044367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc80c15d290 msgr2=0x7fc80c15d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:49.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.141+0000 7fc812a0c700 1 --2- 172.21.15.16:0/370044367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc80c15d290 0x7fc80c15d700 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fc80c0cdb50 tx=0x7fc804004ad0 comp rx=0 tx=0).stop 2023-05-20T01:19:49.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.141+0000 7fc812a0c700 1 -- 172.21.15.16:0/370044367 shutdown_connections 2023-05-20T01:19:49.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.141+0000 7fc812a0c700 1 --2- 172.21.15.16:0/370044367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc80c15d290 0x7fc80c15d700 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:49.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.141+0000 7fc812a0c700 1 --2- 172.21.15.16:0/370044367 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc80c072130 0x7fc80c1483f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:49.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.141+0000 7fc812a0c700 1 --2- 172.21.15.16:0/370044367 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc7f806ace0 0x7fc7f806d190 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:49.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.141+0000 7fc812a0c700 1 --2- 172.21.15.16:0/370044367 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc80c15cec0 0x7fc80c0c9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:49.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.141+0000 7fc812a0c700 1 -- 172.21.15.16:0/370044367 >> 172.21.15.16:0/370044367 conn(0x7fc80c0cffd0 msgr2=0x7fc80c15f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:19:49.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.141+0000 7fc812a0c700 1 -- 172.21.15.16:0/370044367 shutdown_connections 2023-05-20T01:19:49.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.141+0000 7fc812a0c700 1 -- 172.21.15.16:0/370044367 wait complete. 2023-05-20T01:19:49.158 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-05-20T01:19:49.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.280+0000 7fc4fd6a7700 1 Processor -- start 2023-05-20T01:19:49.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.280+0000 7fc4fd6a7700 1 -- start start 2023-05-20T01:19:49.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.281+0000 7fc4fd6a7700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc4f815a8d0 0x7fc4f815dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:49.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.281+0000 7fc4fd6a7700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4f80d05c0 con 0x7fc4f80d3880 2023-05-20T01:19:49.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.281+0000 7fc4fd6a7700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4f80cf9b0 con 0x7fc4f80cdec0 2023-05-20T01:19:49.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.281+0000 7fc4fd6a7700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f815e160 con 0x7fc4f815a8d0 2023-05-20T01:19:49.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.281+0000 7fc4f77fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fc4f80d3880 0x7fc4f80d0f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:45222/0 (socket says 172.21.15.16:45222) 2023-05-20T01:19:49.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.281+0000 7fc4f77fe700 1 -- 172.21.15.16:0/1967497921 learned_addr learned my addr 172.21.15.16:0/1967497921 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:19:49.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.281+0000 7fc4f6ffd700 1 --2- 172.21.15.16:0/1967497921 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc4f815a8d0 0x7fc4f815dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:49.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.281+0000 7fc4f5ffb700 1 -- 172.21.15.16:0/1967497921 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2804532878 0 0) 0x7fc4f80d05c0 con 0x7fc4f80d3880 2023-05-20T01:19:49.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.281+0000 7fc4f5ffb700 1 -- 172.21.15.16:0/1967497921 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc4e00035b0 con 0x7fc4f80d3880 2023-05-20T01:19:49.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.281+0000 7fc4f5ffb700 1 -- 172.21.15.16:0/1967497921 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3749357865 0 0) 0x7fc4e00035b0 con 0x7fc4f80d3880 2023-05-20T01:19:49.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.281+0000 7fc4f5ffb700 1 -- 172.21.15.16:0/1967497921 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc4f815a8d0 msgr2=0x7fc4f815dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:19:49.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.281+0000 7fc4f5ffb700 1 --2- 172.21.15.16:0/1967497921 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc4f815a8d0 0x7fc4f815dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:49.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.281+0000 7fc4f5ffb700 1 -- 172.21.15.16:0/1967497921 >> v1:172.21.15.132:6789/0 conn(0x7fc4f80cdec0 legacy=0x7fc4f80d3170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:49.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.281+0000 7fc4f5ffb700 1 -- 172.21.15.16:0/1967497921 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4f80cfc90 con 0x7fc4f80d3880 2023-05-20T01:19:49.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.281+0000 7fc4f5ffb700 1 -- 172.21.15.16:0/1967497921 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7fc4f00019f0 con 0x7fc4f80d3880 2023-05-20T01:19:49.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.282+0000 7fc4f5ffb700 1 -- 172.21.15.16:0/1967497921 >> v1:172.21.15.16:6789/0 conn(0x7fc4f80d3880 legacy=0x7fc4f80d0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:49.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.282+0000 7fc4f5ffb700 1 --2- 172.21.15.16:0/1967497921 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc4f815a8d0 0x7fc4e0003aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:49.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.282+0000 7fc4f5ffb700 1 --2- 172.21.15.16:0/1967497921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4e0004020 0x7fc4e0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:49.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.282+0000 7fc4f5ffb700 1 --2- 172.21.15.16:0/1967497921 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4e0004d90 0x7fc4e0005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:49.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.282+0000 7fc4f5ffb700 1 -- 172.21.15.16:0/1967497921 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f815e160 con 0x7fc4e0004020 2023-05-20T01:19:49.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.282+0000 7fc4f5ffb700 1 -- 172.21.15.16:0/1967497921 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e000bcf0 con 0x7fc4e0004d90 2023-05-20T01:19:49.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.282+0000 7fc4f5ffb700 1 -- 172.21.15.16:0/1967497921 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e000beb0 con 0x7fc4f815a8d0 2023-05-20T01:19:49.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.282+0000 7fc4f67fc700 1 --2- 172.21.15.16:0/1967497921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4e0004020 0x7fc4e0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:49.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.282+0000 7fc4f6ffd700 1 --2- 172.21.15.16:0/1967497921 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc4f815a8d0 0x7fc4e0003aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:49.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.282+0000 7fc4f77fe700 1 --2- 172.21.15.16:0/1967497921 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4e0004d90 0x7fc4e0005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:49.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.282+0000 7fc4f67fc700 1 -- 172.21.15.16:0/1967497921 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc4f815a8d0 msgr2=0x7fc4e0003aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:19:49.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.282+0000 7fc4f67fc700 1 --2- 172.21.15.16:0/1967497921 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc4f815a8d0 0x7fc4e0003aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:49.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.282+0000 7fc4f67fc700 1 -- 172.21.15.16:0/1967497921 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4e0004d90 msgr2=0x7fc4e0005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:19:49.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.282+0000 7fc4f67fc700 1 --2- 172.21.15.16:0/1967497921 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4e0004d90 0x7fc4e0005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:49.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.282+0000 7fc4f67fc700 1 -- 172.21.15.16:0/1967497921 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4e00035b0 con 0x7fc4e0004020 2023-05-20T01:19:49.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.282+0000 7fc4f67fc700 1 --2- 172.21.15.16:0/1967497921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4e0004020 0x7fc4e0004850 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fc4e8007fd0 tx=0x7fc4e0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=334c5494e35969fd server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:49.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.282+0000 7fc4f5ffb700 1 -- 172.21.15.16:0/1967497921 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4e80052d0 con 0x7fc4e0004020 2023-05-20T01:19:49.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.282+0000 7fc4f5ffb700 1 -- 172.21.15.16:0/1967497921 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4e8005470 con 0x7fc4e0004020 2023-05-20T01:19:49.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.282+0000 7fc4f5ffb700 1 -- 172.21.15.16:0/1967497921 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4e80038c0 con 0x7fc4e0004020 2023-05-20T01:19:49.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.283+0000 7fc4fd6a7700 1 -- 172.21.15.16:0/1967497921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4e0004020 msgr2=0x7fc4e0004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:49.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.283+0000 7fc4fd6a7700 1 --2- 172.21.15.16:0/1967497921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4e0004020 0x7fc4e0004850 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fc4e8007fd0 tx=0x7fc4e0005880 comp rx=0 tx=0).stop 2023-05-20T01:19:49.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.283+0000 7fc4f77fe700 1 -- 172.21.15.16:0/1967497921 reap_dead start 2023-05-20T01:19:49.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.283+0000 7fc4fd6a7700 1 -- 172.21.15.16:0/1967497921 shutdown_connections 2023-05-20T01:19:49.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.283+0000 7fc4fd6a7700 1 -- 172.21.15.16:0/1967497921 >> 172.21.15.16:0/1967497921 conn(0x7fc4f806b0d0 msgr2=0x7fc4f806b4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:19:49.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.283+0000 7fc4fd6a7700 1 -- 172.21.15.16:0/1967497921 shutdown_connections 2023-05-20T01:19:49.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.283+0000 7fc4fd6a7700 1 -- 172.21.15.16:0/1967497921 wait complete. 2023-05-20T01:19:49.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.284+0000 7fc4fd6a7700 1 Processor -- start 2023-05-20T01:19:49.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.284+0000 7fc4fd6a7700 1 -- start start 2023-05-20T01:19:49.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.285+0000 7fc4fd6a7700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc4e0004020 0x7fc4f81f4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:49.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.285+0000 7fc4fd6a7700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4f81e8910 0x7fc4f81e8d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:49.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.285+0000 7fc4fd6a7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4f80cb880 0x7fc4f80cbce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:49.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.285+0000 7fc4fd6a7700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f80c9d10 con 0x7fc4f80cb880 2023-05-20T01:19:49.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.285+0000 7fc4fd6a7700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f81e92c0 con 0x7fc4f81e8910 2023-05-20T01:19:49.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.285+0000 7fc4fd6a7700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f80cc5e0 con 0x7fc4e0004020 2023-05-20T01:19:49.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.285+0000 7fc4f77fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4f80cb880 0x7fc4f80cbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:49.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.285+0000 7fc4f77fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4f80cb880 0x7fc4f80cbce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33708/0 (socket says 172.21.15.16:33708) 2023-05-20T01:19:49.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.285+0000 7fc4f77fe700 1 -- 172.21.15.16:0/533536010 learned_addr learned my addr 172.21.15.16:0/533536010 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:19:49.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.285+0000 7fc4f67fc700 1 --2- 172.21.15.16:0/533536010 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4f81e8910 0x7fc4f81e8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:49.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.285+0000 7fc4f6ffd700 1 --2- 172.21.15.16:0/533536010 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc4e0004020 0x7fc4f81f4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:49.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.285+0000 7fc4f77fe700 1 -- 172.21.15.16:0/533536010 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc4e0004020 msgr2=0x7fc4f81f4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:49.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.285+0000 7fc4f77fe700 1 --2- 172.21.15.16:0/533536010 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc4e0004020 0x7fc4f81f4fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:49.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.285+0000 7fc4f77fe700 1 -- 172.21.15.16:0/533536010 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4f81e8910 msgr2=0x7fc4f81e8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:49.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.285+0000 7fc4f77fe700 1 --2- 172.21.15.16:0/533536010 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4f81e8910 0x7fc4f81e8d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:49.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.285+0000 7fc4f77fe700 1 -- 172.21.15.16:0/533536010 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4f80d0250 con 0x7fc4f80cb880 2023-05-20T01:19:49.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.285+0000 7fc4f6ffd700 1 --2- 172.21.15.16:0/533536010 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc4e0004020 0x7fc4f81f4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:19:49.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.286+0000 7fc4f77fe700 1 --2- 172.21.15.16:0/533536010 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4f80cb880 0x7fc4f80cbce0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fc4f80d3080 tx=0x7fc4f000e440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:49.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.286+0000 7fc4df7fe700 1 -- 172.21.15.16:0/533536010 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4f000ebf0 con 0x7fc4f80cb880 2023-05-20T01:19:49.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.286+0000 7fc4df7fe700 1 -- 172.21.15.16:0/533536010 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4f000ed50 con 0x7fc4f80cb880 2023-05-20T01:19:49.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.286+0000 7fc4df7fe700 1 -- 172.21.15.16:0/533536010 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4f0013660 con 0x7fc4f80cb880 2023-05-20T01:19:49.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.286+0000 7fc4fd6a7700 1 -- 172.21.15.16:0/533536010 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4f80cff70 con 0x7fc4f80cb880 2023-05-20T01:19:49.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.286+0000 7fc4fd6a7700 1 -- 172.21.15.16:0/533536010 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4f80d0a50 con 0x7fc4f80cb880 2023-05-20T01:19:49.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.287+0000 7fc4fd6a7700 1 -- 172.21.15.16:0/533536010 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4f81f35f0 con 0x7fc4f80cb880 2023-05-20T01:19:49.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.287+0000 7fc4df7fe700 1 -- 172.21.15.16:0/533536010 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc4f00137c0 con 0x7fc4f80cb880 2023-05-20T01:19:49.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.288+0000 7fc4df7fe700 1 --2- 172.21.15.16:0/533536010 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc4e406ace0 0x7fc4e406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:49.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.288+0000 7fc4f6ffd700 1 --2- 172.21.15.16:0/533536010 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc4e406ace0 0x7fc4e406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:49.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.288+0000 7fc4df7fe700 1 -- 172.21.15.16:0/533536010 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5530+0+0 (secure 0 0 0) 0x7fc4f0089830 con 0x7fc4f80cb880 2023-05-20T01:19:49.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.288+0000 7fc4f6ffd700 1 --2- 172.21.15.16:0/533536010 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc4e406ace0 0x7fc4e406d190 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fc4f81e57c0 tx=0x7fc4ec008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:49.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.292+0000 7fc4df7fe700 1 -- 172.21.15.16:0/533536010 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fc4f0054af0 con 0x7fc4f80cb880 2023-05-20T01:19:49.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.493+0000 7fc4fd6a7700 1 -- 172.21.15.16:0/533536010 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fc4f81f3410 con 0x7fc4f80cb880 2023-05-20T01:19:49.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.493+0000 7fc4df7fe700 1 -- 172.21.15.16:0/533536010 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v60) v1 ==== 84+0+126 (secure 0 0 0) 0x7fc4f0058110 con 0x7fc4f80cb880 2023-05-20T01:19:49.498 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:19:49.498 INFO:teuthology.orchestra.run.smithi016.stdout:[ 2023-05-20T01:19:49.499 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-05-20T01:19:49.500 INFO:teuthology.orchestra.run.smithi016.stdout: "poolnum": 1, 2023-05-20T01:19:49.500 INFO:teuthology.orchestra.run.smithi016.stdout: "poolname": "rbd" 2023-05-20T01:19:49.500 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-05-20T01:19:49.500 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-05-20T01:19:49.500 INFO:teuthology.orchestra.run.smithi016.stdout: "poolnum": 2, 2023-05-20T01:19:49.500 INFO:teuthology.orchestra.run.smithi016.stdout: "poolname": ".mgr" 2023-05-20T01:19:49.500 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:19:49.501 INFO:teuthology.orchestra.run.smithi016.stdout:] 2023-05-20T01:19:49.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.495+0000 7fc4fd6a7700 1 -- 172.21.15.16:0/533536010 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc4e406ace0 msgr2=0x7fc4e406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:49.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.495+0000 7fc4fd6a7700 1 --2- 172.21.15.16:0/533536010 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc4e406ace0 0x7fc4e406d190 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fc4f81e57c0 tx=0x7fc4ec008040 comp rx=0 tx=0).stop 2023-05-20T01:19:49.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.495+0000 7fc4fd6a7700 1 -- 172.21.15.16:0/533536010 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4f80cb880 msgr2=0x7fc4f80cbce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:49.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.495+0000 7fc4fd6a7700 1 --2- 172.21.15.16:0/533536010 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4f80cb880 0x7fc4f80cbce0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fc4f80d3080 tx=0x7fc4f000e440 comp rx=0 tx=0).stop 2023-05-20T01:19:49.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.495+0000 7fc4fd6a7700 1 -- 172.21.15.16:0/533536010 shutdown_connections 2023-05-20T01:19:49.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.495+0000 7fc4fd6a7700 1 --2- 172.21.15.16:0/533536010 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4f80cb880 0x7fc4f80cbce0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:49.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.495+0000 7fc4fd6a7700 1 --2- 172.21.15.16:0/533536010 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc4e406ace0 0x7fc4e406d190 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:49.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.495+0000 7fc4fd6a7700 1 --2- 172.21.15.16:0/533536010 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc4e0004020 0x7fc4f81f4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:49.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.495+0000 7fc4fd6a7700 1 --2- 172.21.15.16:0/533536010 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4f81e8910 0x7fc4f81e8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:49.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.495+0000 7fc4fd6a7700 1 -- 172.21.15.16:0/533536010 >> 172.21.15.16:0/533536010 conn(0x7fc4f806b0d0 msgr2=0x7fc4f815adc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:19:49.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.495+0000 7fc4fd6a7700 1 -- 172.21.15.16:0/533536010 shutdown_connections 2023-05-20T01:19:49.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.496+0000 7fc4fd6a7700 1 -- 172.21.15.16:0/533536010 wait complete. 2023-05-20T01:19:49.512 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-05-20T01:19:49.512 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-05-20T01:19:49.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.636+0000 7f885a833700 1 Processor -- start 2023-05-20T01:19:49.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.636+0000 7f885a833700 1 -- start start 2023-05-20T01:19:49.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.637+0000 7f885a833700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f8854159790 0x7f88540caf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:49.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.637+0000 7f885a833700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f88540719b0 0x7f88540c4f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:49.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.637+0000 7f885a833700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88540cc320 con 0x7f88540cdf90 2023-05-20T01:19:49.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.637+0000 7f885a833700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88540cf410 con 0x7f88540719b0 2023-05-20T01:19:49.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.637+0000 7f885a833700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88540cb450 con 0x7f8854159790 2023-05-20T01:19:49.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.637+0000 7f88537fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f88540cdf90 0x7f88540712a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:45240/0 (socket says 172.21.15.16:45240) 2023-05-20T01:19:49.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.637+0000 7f8858dd0700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f88540719b0 0x7f88540c4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:49.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.637+0000 7f88537fe700 1 -- 172.21.15.16:0/2843998380 learned_addr learned my addr 172.21.15.16:0/2843998380 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:19:49.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.638+0000 7f8853fff700 1 --2- 172.21.15.16:0/2843998380 >> v2:172.21.15.132:3300/0 conn(0x7f8854159790 0x7f88540caf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:49.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.638+0000 7f8858dd0700 1 -- 172.21.15.16:0/2843998380 >> v2:172.21.15.132:3300/0 conn(0x7f8854159790 msgr2=0x7f88540caf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:19:49.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.638+0000 7f8858dd0700 1 --2- 172.21.15.16:0/2843998380 >> v2:172.21.15.132:3300/0 conn(0x7f8854159790 0x7f88540caf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:49.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.638+0000 7f8858dd0700 1 -- 172.21.15.16:0/2843998380 >> v1:172.21.15.16:6789/0 conn(0x7f88540cdf90 legacy=0x7f88540712a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:49.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.638+0000 7f8858dd0700 1 -- 172.21.15.16:0/2843998380 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8854159b60 con 0x7f88540719b0 2023-05-20T01:19:49.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.638+0000 7f8852ffd700 1 -- 172.21.15.16:0/2843998380 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3571402802 0 0) 0x7f88540cc320 con 0x7f88540cdf90 2023-05-20T01:19:49.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.638+0000 7f8853fff700 1 --2- 172.21.15.16:0/2843998380 >> v2:172.21.15.132:3300/0 conn(0x7f8854159790 0x7f88540caf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:19:49.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.638+0000 7f8858dd0700 1 --2- 172.21.15.16:0/2843998380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88540719b0 0x7f88540c4f00 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f88540d0e50 tx=0x7f884c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e39c1cdfadea72b6 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:49.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.638+0000 7f8852ffd700 1 -- 172.21.15.16:0/2843998380 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f884c00e070 con 0x7f88540719b0 2023-05-20T01:19:49.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.638+0000 7f8852ffd700 1 -- 172.21.15.16:0/2843998380 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f884c00bbd0 con 0x7f88540719b0 2023-05-20T01:19:49.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.638+0000 7f8852ffd700 1 -- 172.21.15.16:0/2843998380 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f884c012430 con 0x7f88540719b0 2023-05-20T01:19:49.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.639+0000 7f885a833700 1 -- 172.21.15.16:0/2843998380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88540719b0 msgr2=0x7f88540c4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:49.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.639+0000 7f885a833700 1 --2- 172.21.15.16:0/2843998380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88540719b0 0x7f88540c4f00 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f88540d0e50 tx=0x7f884c00b330 comp rx=0 tx=0).stop 2023-05-20T01:19:49.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.639+0000 7f885a833700 1 -- 172.21.15.16:0/2843998380 shutdown_connections 2023-05-20T01:19:49.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.639+0000 7f885a833700 1 --2- 172.21.15.16:0/2843998380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88540719b0 0x7f88540c4f00 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:49.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.639+0000 7f885a833700 1 --2- 172.21.15.16:0/2843998380 >> v2:172.21.15.132:3300/0 conn(0x7f8854159790 0x7f88540caf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:49.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.639+0000 7f885a833700 1 -- 172.21.15.16:0/2843998380 >> 172.21.15.16:0/2843998380 conn(0x7f88540b2920 msgr2=0x7f88540d0020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:19:49.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.639+0000 7f885a833700 1 -- 172.21.15.16:0/2843998380 shutdown_connections 2023-05-20T01:19:49.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.639+0000 7f885a833700 1 -- 172.21.15.16:0/2843998380 wait complete. 2023-05-20T01:19:49.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.640+0000 7f885a833700 1 Processor -- start 2023-05-20T01:19:49.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.640+0000 7f885a833700 1 -- start start 2023-05-20T01:19:49.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.641+0000 7f885a833700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f88540719b0 0x7f885413f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:49.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.641+0000 7f885a833700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f88540cdf90 0x7f8854159f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:49.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.641+0000 7f885a833700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8854159790 0x7f88540c9300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:49.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.641+0000 7f885a833700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88540b5630 con 0x7f8854159790 2023-05-20T01:19:49.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.641+0000 7f885a833700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88540b4a20 con 0x7f88540cdf90 2023-05-20T01:19:49.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.641+0000 7f885a833700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88540b4d00 con 0x7f88540719b0 2023-05-20T01:19:49.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.641+0000 7f8858dd0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8854159790 0x7f88540c9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:49.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.641+0000 7f8858dd0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8854159790 0x7f88540c9300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33722/0 (socket says 172.21.15.16:33722) 2023-05-20T01:19:49.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.641+0000 7f8858dd0700 1 -- 172.21.15.16:0/4145489554 learned_addr learned my addr 172.21.15.16:0/4145489554 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:19:49.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.641+0000 7f88537fe700 1 --2- 172.21.15.16:0/4145489554 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f88540cdf90 0x7f8854159f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:49.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.641+0000 7f8853fff700 1 --2- 172.21.15.16:0/4145489554 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f88540719b0 0x7f885413f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:49.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.641+0000 7f8858dd0700 1 -- 172.21.15.16:0/4145489554 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f88540719b0 msgr2=0x7f885413f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:49.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.641+0000 7f8858dd0700 1 --2- 172.21.15.16:0/4145489554 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f88540719b0 0x7f885413f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:49.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.641+0000 7f8858dd0700 1 -- 172.21.15.16:0/4145489554 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f88540cdf90 msgr2=0x7f8854159f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:49.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.641+0000 7f8858dd0700 1 --2- 172.21.15.16:0/4145489554 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f88540cdf90 0x7f8854159f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:49.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.641+0000 7f8858dd0700 1 -- 172.21.15.16:0/4145489554 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f884c009a70 con 0x7f8854159790 2023-05-20T01:19:49.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.641+0000 7f88537fe700 1 --2- 172.21.15.16:0/4145489554 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f88540cdf90 0x7f8854159f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:19:49.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.642+0000 7f8858dd0700 1 --2- 172.21.15.16:0/4145489554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8854159790 0x7f88540c9300 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f884c013000 tx=0x7f884c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:49.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.642+0000 7f8850ff9700 1 -- 172.21.15.16:0/4145489554 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f884c012430 con 0x7f8854159790 2023-05-20T01:19:49.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.642+0000 7f8850ff9700 1 -- 172.21.15.16:0/4145489554 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f884c020070 con 0x7f8854159790 2023-05-20T01:19:49.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.642+0000 7f885a833700 1 -- 172.21.15.16:0/4145489554 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88540cb9f0 con 0x7f8854159790 2023-05-20T01:19:49.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.642+0000 7f8850ff9700 1 -- 172.21.15.16:0/4145489554 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f884c004280 con 0x7f8854159790 2023-05-20T01:19:49.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.642+0000 7f885a833700 1 -- 172.21.15.16:0/4145489554 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88540cb710 con 0x7f8854159790 2023-05-20T01:19:49.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.643+0000 7f885a833700 1 -- 172.21.15.16:0/4145489554 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88540cbcd0 con 0x7f8854159790 2023-05-20T01:19:49.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.643+0000 7f8850ff9700 1 -- 172.21.15.16:0/4145489554 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f884c0043e0 con 0x7f8854159790 2023-05-20T01:19:49.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.644+0000 7f8850ff9700 1 --2- 172.21.15.16:0/4145489554 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f884006ace0 0x7f884006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:49.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.644+0000 7f8853fff700 1 --2- 172.21.15.16:0/4145489554 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f884006ace0 0x7f884006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:49.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.644+0000 7f8850ff9700 1 -- 172.21.15.16:0/4145489554 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f884c022070 con 0x7f8854159790 2023-05-20T01:19:49.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.644+0000 7f8853fff700 1 --2- 172.21.15.16:0/4145489554 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f884006ace0 0x7f884006d190 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f884800c900 tx=0x7f8848008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:49.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.648+0000 7f8850ff9700 1 -- 172.21.15.16:0/4145489554 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f884c0566b0 con 0x7f8854159790 2023-05-20T01:19:49.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:49.847+0000 7f885a833700 1 -- 172.21.15.16:0/4145489554 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f88540747b0 con 0x7f8854159790 2023-05-20T01:19:50.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:50.483+0000 7f8850ff9700 1 -- 172.21.15.16:0/4145489554 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v61) v1 ==== 138+0+0 (secure 0 0 0) 0x7f884c059cd0 con 0x7f8854159790 2023-05-20T01:19:50.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:50.488+0000 7f885a833700 1 -- 172.21.15.16:0/4145489554 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f884006ace0 msgr2=0x7f884006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:50.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:50.488+0000 7f885a833700 1 --2- 172.21.15.16:0/4145489554 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f884006ace0 0x7f884006d190 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f884800c900 tx=0x7f8848008040 comp rx=0 tx=0).stop 2023-05-20T01:19:50.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:50.488+0000 7f885a833700 1 -- 172.21.15.16:0/4145489554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8854159790 msgr2=0x7f88540c9300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:50.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:50.488+0000 7f885a833700 1 --2- 172.21.15.16:0/4145489554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8854159790 0x7f88540c9300 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f884c013000 tx=0x7f884c004ad0 comp rx=0 tx=0).stop 2023-05-20T01:19:50.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:50.488+0000 7f885a833700 1 -- 172.21.15.16:0/4145489554 shutdown_connections 2023-05-20T01:19:50.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:50.488+0000 7f885a833700 1 --2- 172.21.15.16:0/4145489554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8854159790 0x7f88540c9300 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:50.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:50.488+0000 7f885a833700 1 --2- 172.21.15.16:0/4145489554 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f884006ace0 0x7f884006d190 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:50.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:50.488+0000 7f885a833700 1 --2- 172.21.15.16:0/4145489554 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f88540719b0 0x7f885413f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:50.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:50.488+0000 7f885a833700 1 --2- 172.21.15.16:0/4145489554 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f88540cdf90 0x7f8854159f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:50.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:50.488+0000 7f885a833700 1 -- 172.21.15.16:0/4145489554 >> 172.21.15.16:0/4145489554 conn(0x7f88540b2920 msgr2=0x7f88540cfeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:19:50.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:50.489+0000 7f885a833700 1 -- 172.21.15.16:0/4145489554 shutdown_connections 2023-05-20T01:19:50.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:50.489+0000 7f885a833700 1 -- 172.21.15.16:0/4145489554 wait complete. 2023-05-20T01:19:50.494 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' created 2023-05-20T01:19:50.509 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-05-20T01:19:50.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:50.644+0000 7fed1d9ef700 1 Processor -- start 2023-05-20T01:19:50.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:50.644+0000 7fed1d9ef700 1 -- start start 2023-05-20T01:19:50.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:50.645+0000 7fed1d9ef700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fed1815ca90 0x7fed180cd890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:50.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:50.645+0000 7fed1d9ef700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fed180d2550 0x7fed180b6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:50.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:50.645+0000 7fed1d9ef700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed180d1820 con 0x7fed180d29a0 2023-05-20T01:19:50.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:50.645+0000 7fed1d9ef700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed180d0c50 con 0x7fed1815ca90 2023-05-20T01:19:50.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:50.645+0000 7fed1d9ef700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed180cdf00 con 0x7fed180d2550 2023-05-20T01:19:50.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:50.645+0000 7fed177fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fed180d29a0 0x7fed180b4cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:45258/0 (socket says 172.21.15.16:45258) 2023-05-20T01:19:50.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:50.645+0000 7fed177fe700 1 -- 172.21.15.16:0/1812058328 learned_addr learned my addr 172.21.15.16:0/1812058328 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:19:50.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:50.645+0000 7fed167fc700 1 --2- 172.21.15.16:0/1812058328 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fed180d2550 0x7fed180b6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:50.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:50.645+0000 7fed16ffd700 1 --2- 172.21.15.16:0/1812058328 >> v2:172.21.15.132:3300/0 conn(0x7fed1815ca90 0x7fed180cd890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:50.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:50.645+0000 7fed15ffb700 1 -- 172.21.15.16:0/1812058328 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2435828319 0 0) 0x7fed180d1820 con 0x7fed180d29a0 2023-05-20T01:19:50.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:50.645+0000 7fed15ffb700 1 -- 172.21.15.16:0/1812058328 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed000035b0 con 0x7fed180d29a0 2023-05-20T01:19:50.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:50.646+0000 7fed15ffb700 1 -- 172.21.15.16:0/1812058328 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2151197573 0 0) 0x7fed000035b0 con 0x7fed180d29a0 2023-05-20T01:19:50.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:50.646+0000 7fed15ffb700 1 -- 172.21.15.16:0/1812058328 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fed180d2550 msgr2=0x7fed180b6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:19:50.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:50.646+0000 7fed15ffb700 1 --2- 172.21.15.16:0/1812058328 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fed180d2550 0x7fed180b6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:50.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:50.646+0000 7fed15ffb700 1 -- 172.21.15.16:0/1812058328 >> v2:172.21.15.132:3300/0 conn(0x7fed1815ca90 msgr2=0x7fed180cd890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:19:50.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:50.646+0000 7fed15ffb700 1 --2- 172.21.15.16:0/1812058328 >> v2:172.21.15.132:3300/0 conn(0x7fed1815ca90 0x7fed180cd890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:50.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:50.646+0000 7fed15ffb700 1 -- 172.21.15.16:0/1812058328 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed180d1a70 con 0x7fed180d29a0 2023-05-20T01:19:50.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:50.646+0000 7fed15ffb700 1 -- 172.21.15.16:0/1812058328 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7fed100019f0 con 0x7fed180d29a0 2023-05-20T01:19:50.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:50.646+0000 7fed167fc700 1 --2- 172.21.15.16:0/1812058328 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fed180d2550 0x7fed180b6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-05-20T01:19:50.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:50.646+0000 7fed16ffd700 1 --2- 172.21.15.16:0/1812058328 >> v2:172.21.15.132:3300/0 conn(0x7fed1815ca90 0x7fed180cd890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-05-20T01:19:50.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:50.646+0000 7fed15ffb700 1 -- 172.21.15.16:0/1812058328 >> v1:172.21.15.16:6789/0 conn(0x7fed180d29a0 legacy=0x7fed180b4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:50.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:50.646+0000 7fed15ffb700 1 --2- 172.21.15.16:0/1812058328 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fed180d2550 0x7fed00003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:50.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:50.646+0000 7fed15ffb700 1 --2- 172.21.15.16:0/1812058328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed00003f40 0x7fed00004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:50.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:50.646+0000 7fed15ffb700 1 --2- 172.21.15.16:0/1812058328 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fed00004d90 0x7fed00005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:50.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:50.646+0000 7fed15ffb700 1 -- 172.21.15.16:0/1812058328 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed180d0c50 con 0x7fed00003f40 2023-05-20T01:19:50.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:50.646+0000 7fed15ffb700 1 -- 172.21.15.16:0/1812058328 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed180cdf00 con 0x7fed00004d90 2023-05-20T01:19:50.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:50.646+0000 7fed15ffb700 1 -- 172.21.15.16:0/1812058328 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed0000ba10 con 0x7fed180d2550 2023-05-20T01:19:50.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:50.646+0000 7fed167fc700 1 --2- 172.21.15.16:0/1812058328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed00003f40 0x7fed00004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:50.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:50.646+0000 7fed16ffd700 1 --2- 172.21.15.16:0/1812058328 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fed180d2550 0x7fed00003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:50.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:50.646+0000 7fed177fe700 1 --2- 172.21.15.16:0/1812058328 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fed00004d90 0x7fed00005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:50.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:50.646+0000 7fed167fc700 1 -- 172.21.15.16:0/1812058328 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fed180d2550 msgr2=0x7fed00003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:19:50.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:50.646+0000 7fed167fc700 1 --2- 172.21.15.16:0/1812058328 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fed180d2550 0x7fed00003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:50.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:50.646+0000 7fed167fc700 1 -- 172.21.15.16:0/1812058328 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fed00004d90 msgr2=0x7fed00005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:19:50.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:50.646+0000 7fed167fc700 1 --2- 172.21.15.16:0/1812058328 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fed00004d90 0x7fed00005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:50.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:50.646+0000 7fed167fc700 1 -- 172.21.15.16:0/1812058328 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed080094a0 con 0x7fed00003f40 2023-05-20T01:19:50.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:50.646+0000 7fed177fe700 1 --2- 172.21.15.16:0/1812058328 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fed00004d90 0x7fed00005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:19:50.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:50.646+0000 7fed167fc700 1 --2- 172.21.15.16:0/1812058328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed00003f40 0x7fed00004850 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fed08009d30 tx=0x7fed00005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb75ebea07d2c3b6 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:50.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:50.646+0000 7fed15ffb700 1 -- 172.21.15.16:0/1812058328 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed08009e80 con 0x7fed00003f40 2023-05-20T01:19:50.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:50.646+0000 7fed15ffb700 1 -- 172.21.15.16:0/1812058328 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed0800e9a0 con 0x7fed00003f40 2023-05-20T01:19:50.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:50.646+0000 7fed15ffb700 1 -- 172.21.15.16:0/1812058328 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed0800ed10 con 0x7fed00003f40 2023-05-20T01:19:50.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:50.647+0000 7fed1d9ef700 1 -- 172.21.15.16:0/1812058328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed00003f40 msgr2=0x7fed00004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:50.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:50.647+0000 7fed1d9ef700 1 --2- 172.21.15.16:0/1812058328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed00003f40 0x7fed00004850 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fed08009d30 tx=0x7fed00005880 comp rx=0 tx=0).stop 2023-05-20T01:19:50.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:50.647+0000 7fed177fe700 1 -- 172.21.15.16:0/1812058328 reap_dead start 2023-05-20T01:19:50.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:50.647+0000 7fed1d9ef700 1 -- 172.21.15.16:0/1812058328 shutdown_connections 2023-05-20T01:19:50.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:50.647+0000 7fed1d9ef700 1 -- 172.21.15.16:0/1812058328 >> 172.21.15.16:0/1812058328 conn(0x7fed1806b000 msgr2=0x7fed1806b400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:19:50.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:50.647+0000 7fed1d9ef700 1 -- 172.21.15.16:0/1812058328 shutdown_connections 2023-05-20T01:19:50.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:50.647+0000 7fed1d9ef700 1 -- 172.21.15.16:0/1812058328 wait complete. 2023-05-20T01:19:50.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:50.648+0000 7fed1d9ef700 1 Processor -- start 2023-05-20T01:19:50.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:50.649+0000 7fed1d9ef700 1 -- start start 2023-05-20T01:19:50.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:50.649+0000 7fed1d9ef700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fed00003f40 0x7fed1815f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:50.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:50.649+0000 7fed1d9ef700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed1815d180 0x7fed18155b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:50.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:50.649+0000 7fed1d9ef700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fed1815d5f0 0x7fed181560b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:50.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:50.649+0000 7fed1d9ef700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed1815ebe0 con 0x7fed1815d180 2023-05-20T01:19:50.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:50.649+0000 7fed1d9ef700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed1815db40 con 0x7fed00003f40 2023-05-20T01:19:50.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:50.649+0000 7fed1d9ef700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed1815de20 con 0x7fed1815d5f0 2023-05-20T01:19:50.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:50.649+0000 7fed167fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed1815d180 0x7fed18155b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:50.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:50.649+0000 7fed167fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed1815d180 0x7fed18155b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33740/0 (socket says 172.21.15.16:33740) 2023-05-20T01:19:50.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:50.649+0000 7fed167fc700 1 -- 172.21.15.16:0/175090017 learned_addr learned my addr 172.21.15.16:0/175090017 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:19:50.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:50.649+0000 7fed16ffd700 1 --2- 172.21.15.16:0/175090017 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fed00003f40 0x7fed1815f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:50.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:50.650+0000 7fed177fe700 1 --2- 172.21.15.16:0/175090017 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fed1815d5f0 0x7fed181560b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:50.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:50.650+0000 7fed167fc700 1 -- 172.21.15.16:0/175090017 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fed1815d5f0 msgr2=0x7fed181560b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:50.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:50.650+0000 7fed167fc700 1 --2- 172.21.15.16:0/175090017 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fed1815d5f0 0x7fed181560b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:50.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:50.650+0000 7fed167fc700 1 -- 172.21.15.16:0/175090017 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fed00003f40 msgr2=0x7fed1815f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:50.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:50.650+0000 7fed167fc700 1 --2- 172.21.15.16:0/175090017 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fed00003f40 0x7fed1815f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:50.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:50.650+0000 7fed167fc700 1 -- 172.21.15.16:0/175090017 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed180d2030 con 0x7fed1815d180 2023-05-20T01:19:50.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:50.650+0000 7fed167fc700 1 --2- 172.21.15.16:0/175090017 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed1815d180 0x7fed18155b70 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fed080090b0 tx=0x7fed080047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:50.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:50.650+0000 7fecfffff700 1 -- 172.21.15.16:0/175090017 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed08003ab0 con 0x7fed1815d180 2023-05-20T01:19:50.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:50.650+0000 7fecfffff700 1 -- 172.21.15.16:0/175090017 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed08003c10 con 0x7fed1815d180 2023-05-20T01:19:50.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:50.650+0000 7fecfffff700 1 -- 172.21.15.16:0/175090017 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed08009e80 con 0x7fed1815d180 2023-05-20T01:19:50.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:50.650+0000 7fed1d9ef700 1 -- 172.21.15.16:0/175090017 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed180d1d50 con 0x7fed1815d180 2023-05-20T01:19:50.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:50.650+0000 7fed1d9ef700 1 -- 172.21.15.16:0/175090017 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed1815ce60 con 0x7fed1815d180 2023-05-20T01:19:50.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:50.651+0000 7fed1d9ef700 1 -- 172.21.15.16:0/175090017 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed1815ea30 con 0x7fed1815d180 2023-05-20T01:19:50.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:50.652+0000 7fecfffff700 1 -- 172.21.15.16:0/175090017 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fed08013470 con 0x7fed1815d180 2023-05-20T01:19:50.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:50.652+0000 7fecfffff700 1 --2- 172.21.15.16:0/175090017 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fed0406ad10 0x7fed0406d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:50.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:50.652+0000 7fecfffff700 1 -- 172.21.15.16:0/175090017 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5900+0+0 (secure 0 0 0) 0x7fed0808ae50 con 0x7fed1815d180 2023-05-20T01:19:50.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:50.652+0000 7fed16ffd700 1 --2- 172.21.15.16:0/175090017 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fed0406ad10 0x7fed0406d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:50.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:50.652+0000 7fed16ffd700 1 --2- 172.21.15.16:0/175090017 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fed0406ad10 0x7fed0406d1c0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fed0c009e60 tx=0x7fed0c009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:50.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:50.657+0000 7fecfffff700 1 -- 172.21.15.16:0/175090017 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fed08056030 con 0x7fed1815d180 2023-05-20T01:19:50.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:50.856+0000 7fed1d9ef700 1 -- 172.21.15.16:0/175090017 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fed1804e8c0 con 0x7fed1815d180 2023-05-20T01:19:51.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:51.490+0000 7fecfffff700 1 -- 172.21.15.16:0/175090017 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v62) v1 ==== 170+0+0 (secure 0 0 0) 0x7fed080170d0 con 0x7fed1815d180 2023-05-20T01:19:51.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:51.494+0000 7fed1d9ef700 1 -- 172.21.15.16:0/175090017 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fed0406ad10 msgr2=0x7fed0406d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:51.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:51.494+0000 7fed1d9ef700 1 --2- 172.21.15.16:0/175090017 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fed0406ad10 0x7fed0406d1c0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fed0c009e60 tx=0x7fed0c009210 comp rx=0 tx=0).stop 2023-05-20T01:19:51.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:51.494+0000 7fed1d9ef700 1 -- 172.21.15.16:0/175090017 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed1815d180 msgr2=0x7fed18155b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:51.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:51.494+0000 7fed1d9ef700 1 --2- 172.21.15.16:0/175090017 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed1815d180 0x7fed18155b70 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fed080090b0 tx=0x7fed080047c0 comp rx=0 tx=0).stop 2023-05-20T01:19:51.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:51.494+0000 7fed1d9ef700 1 -- 172.21.15.16:0/175090017 shutdown_connections 2023-05-20T01:19:51.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:51.494+0000 7fed1d9ef700 1 --2- 172.21.15.16:0/175090017 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fed0406ad10 0x7fed0406d1c0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:51.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:51.494+0000 7fed1d9ef700 1 --2- 172.21.15.16:0/175090017 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fed1815d5f0 0x7fed181560b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:51.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:51.494+0000 7fed1d9ef700 1 --2- 172.21.15.16:0/175090017 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fed00003f40 0x7fed1815f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:51.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:51.494+0000 7fed1d9ef700 1 --2- 172.21.15.16:0/175090017 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed1815d180 0x7fed18155b70 secure :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fed080090b0 tx=0x7fed080047c0 comp rx=0 tx=0).stop 2023-05-20T01:19:51.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:51.494+0000 7fed1d9ef700 1 -- 172.21.15.16:0/175090017 >> 172.21.15.16:0/175090017 conn(0x7fed1806b000 msgr2=0x7fed180b6090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:19:51.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:51.495+0000 7fed1d9ef700 1 -- 172.21.15.16:0/175090017 shutdown_connections 2023-05-20T01:19:51.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:51.495+0000 7fed1d9ef700 1 -- 172.21.15.16:0/175090017 wait complete. 2023-05-20T01:19:51.516 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' created 2023-05-20T01:19:51.526 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-05-20T01:19:51.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:51.656+0000 7f48d6644700 1 Processor -- start 2023-05-20T01:19:51.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:51.656+0000 7f48d6644700 1 -- start start 2023-05-20T01:19:51.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:51.657+0000 7f48d6644700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f48d015cec0 0x7f48d00cd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:51.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:51.657+0000 7f48d6644700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f48d006a510 0x7f48d006a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:51.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:51.657+0000 7f48d6644700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48d006dd00 con 0x7f48d006e190 2023-05-20T01:19:51.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:51.657+0000 7f48d6644700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48d006aed0 con 0x7f48d015cec0 2023-05-20T01:19:51.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:51.657+0000 7f48d6644700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48d00cfa60 con 0x7f48d006a510 2023-05-20T01:19:51.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:51.657+0000 7f48cffff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f48d015cec0 0x7f48d00cd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:51.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:51.657+0000 7f48cffff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f48d015cec0 0x7f48d00cd4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33746/0 (socket says 172.21.15.16:33746) 2023-05-20T01:19:51.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:51.657+0000 7f48cffff700 1 -- 172.21.15.16:0/3986066258 learned_addr learned my addr 172.21.15.16:0/3986066258 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:19:51.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:51.657+0000 7f48d4be1700 1 --2- 172.21.15.16:0/3986066258 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f48d006a510 0x7f48d006a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:51.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:51.657+0000 7f48cffff700 1 -- 172.21.15.16:0/3986066258 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f48d006a510 msgr2=0x7f48d006a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:19:51.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:51.657+0000 7f48cffff700 1 --2- 172.21.15.16:0/3986066258 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f48d006a510 0x7f48d006a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:51.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:51.657+0000 7f48cffff700 1 -- 172.21.15.16:0/3986066258 >> v1:172.21.15.132:6789/0 conn(0x7f48d006e190 legacy=0x7f48d00cf1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:19:51.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:51.657+0000 7f48cffff700 1 -- 172.21.15.16:0/3986066258 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48d006d6b0 con 0x7f48d015cec0 2023-05-20T01:19:51.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:51.657+0000 7f48cffff700 1 --2- 172.21.15.16:0/3986066258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48d015cec0 0x7f48d00cd4f0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f48c00098f0 tx=0x7f48c0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=669e0693870d12a4 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:51.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:51.657+0000 7f48ceffd700 1 -- 172.21.15.16:0/3986066258 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c0004230 con 0x7f48d015cec0 2023-05-20T01:19:51.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:51.658+0000 7f48ceffd700 1 -- 172.21.15.16:0/3986066258 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48c0004390 con 0x7f48d015cec0 2023-05-20T01:19:51.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:51.658+0000 7f48ceffd700 1 -- 172.21.15.16:0/3986066258 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c00036a0 con 0x7f48d015cec0 2023-05-20T01:19:51.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:51.658+0000 7f48d6644700 1 -- 172.21.15.16:0/3986066258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48d015cec0 msgr2=0x7f48d00cd4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:51.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:51.658+0000 7f48d6644700 1 --2- 172.21.15.16:0/3986066258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48d015cec0 0x7f48d00cd4f0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f48c00098f0 tx=0x7f48c0009c00 comp rx=0 tx=0).stop 2023-05-20T01:19:51.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:51.658+0000 7f48d6644700 1 -- 172.21.15.16:0/3986066258 shutdown_connections 2023-05-20T01:19:51.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:51.658+0000 7f48d6644700 1 --2- 172.21.15.16:0/3986066258 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f48d006a510 0x7f48d006a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:51.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:51.658+0000 7f48d6644700 1 --2- 172.21.15.16:0/3986066258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48d015cec0 0x7f48d00cd4f0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:51.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:51.658+0000 7f48d6644700 1 -- 172.21.15.16:0/3986066258 >> 172.21.15.16:0/3986066258 conn(0x7f48d006c260 msgr2=0x7f48d00cffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:19:51.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:51.658+0000 7f48d6644700 1 -- 172.21.15.16:0/3986066258 shutdown_connections 2023-05-20T01:19:51.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:51.658+0000 7f48d6644700 1 -- 172.21.15.16:0/3986066258 wait complete. 2023-05-20T01:19:51.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:51.659+0000 7f48d6644700 1 Processor -- start 2023-05-20T01:19:51.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:51.660+0000 7f48d6644700 1 -- start start 2023-05-20T01:19:51.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:51.660+0000 7f48d6644700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f48d006a510 0x7f48d01f33a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:51.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:51.660+0000 7f48d6644700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48d006e190 0x7f48d0160460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:51.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:51.660+0000 7f48d6644700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f48d015cec0 0x7f48d01609a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:51.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:51.660+0000 7f48d6644700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48d00c5b90 con 0x7f48d006e190 2023-05-20T01:19:51.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:51.660+0000 7f48d6644700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48d00c5e70 con 0x7f48d006a510 2023-05-20T01:19:51.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:51.660+0000 7f48d6644700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48d00c6150 con 0x7f48d015cec0 2023-05-20T01:19:51.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:51.660+0000 7f48cf7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48d006e190 0x7f48d0160460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:51.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:51.660+0000 7f48cf7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48d006e190 0x7f48d0160460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33754/0 (socket says 172.21.15.16:33754) 2023-05-20T01:19:51.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:51.660+0000 7f48cf7fe700 1 -- 172.21.15.16:0/912711499 learned_addr learned my addr 172.21.15.16:0/912711499 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:19:51.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:51.660+0000 7f48cffff700 1 --2- 172.21.15.16:0/912711499 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f48d006a510 0x7f48d01f33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:51.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:51.661+0000 7f48d4be1700 1 --2- 172.21.15.16:0/912711499 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f48d015cec0 0x7f48d01609a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:51.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:51.661+0000 7f48cf7fe700 1 -- 172.21.15.16:0/912711499 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f48d015cec0 msgr2=0x7f48d01609a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:51.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:51.661+0000 7f48cf7fe700 1 --2- 172.21.15.16:0/912711499 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f48d015cec0 0x7f48d01609a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:51.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:51.661+0000 7f48cf7fe700 1 -- 172.21.15.16:0/912711499 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f48d006a510 msgr2=0x7f48d01f33a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:51.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:51.661+0000 7f48cf7fe700 1 --2- 172.21.15.16:0/912711499 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f48d006a510 0x7f48d01f33a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:51.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:51.661+0000 7f48cf7fe700 1 -- 172.21.15.16:0/912711499 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48c0009550 con 0x7f48d006e190 2023-05-20T01:19:51.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:51.661+0000 7f48d4be1700 1 --2- 172.21.15.16:0/912711499 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f48d015cec0 0x7f48d01609a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:19:51.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:51.661+0000 7f48cf7fe700 1 --2- 172.21.15.16:0/912711499 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48d006e190 0x7f48d0160460 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f48c400c4d0 tx=0x7f48c400c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:51.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:51.661+0000 7f48ccff9700 1 -- 172.21.15.16:0/912711499 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c4011790 con 0x7f48d006e190 2023-05-20T01:19:51.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:51.661+0000 7f48ccff9700 1 -- 172.21.15.16:0/912711499 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48c40118f0 con 0x7f48d006e190 2023-05-20T01:19:51.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:51.661+0000 7f48d6644700 1 -- 172.21.15.16:0/912711499 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48d015d290 con 0x7f48d006e190 2023-05-20T01:19:51.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:51.661+0000 7f48ccff9700 1 -- 172.21.15.16:0/912711499 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c4011ca0 con 0x7f48d006e190 2023-05-20T01:19:51.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:51.661+0000 7f48d6644700 1 -- 172.21.15.16:0/912711499 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48d006d990 con 0x7f48d006e190 2023-05-20T01:19:51.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:51.662+0000 7f48d6644700 1 -- 172.21.15.16:0/912711499 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48d00cfd40 con 0x7f48d006e190 2023-05-20T01:19:51.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:51.663+0000 7f48ccff9700 1 -- 172.21.15.16:0/912711499 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f48c401c460 con 0x7f48d006e190 2023-05-20T01:19:51.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:51.663+0000 7f48ccff9700 1 --2- 172.21.15.16:0/912711499 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f48bc06ace0 0x7f48bc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:51.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:51.663+0000 7f48ccff9700 1 -- 172.21.15.16:0/912711499 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6282+0+0 (secure 0 0 0) 0x7f48c4089410 con 0x7f48d006e190 2023-05-20T01:19:51.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:51.663+0000 7f48cffff700 1 --2- 172.21.15.16:0/912711499 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f48bc06ace0 0x7f48bc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:51.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:51.663+0000 7f48cffff700 1 --2- 172.21.15.16:0/912711499 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f48bc06ace0 0x7f48bc06d190 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f48c0009130 tx=0x7f48c0008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:51.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:51.668+0000 7f48ccff9700 1 -- 172.21.15.16:0/912711499 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f48c40542c0 con 0x7f48d006e190 2023-05-20T01:19:51.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:51.913+0000 7f48d6644700 1 -- 172.21.15.16:0/912711499 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f48d01f3cb0 con 0x7f48d006e190 2023-05-20T01:19:51.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:51.919+0000 7f48ccff9700 1 -- 172.21.15.16:0/912711499 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 9 and data pool 10 v33) v1 ==== 173+0+0 (secure 0 0 0) 0x7f48d01f3cb0 con 0x7f48d006e190 2023-05-20T01:19:51.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:51.930+0000 7f48d6644700 1 -- 172.21.15.16:0/912711499 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f48bc06ace0 msgr2=0x7f48bc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:51.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:51.930+0000 7f48d6644700 1 --2- 172.21.15.16:0/912711499 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f48bc06ace0 0x7f48bc06d190 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f48c0009130 tx=0x7f48c0008030 comp rx=0 tx=0).stop 2023-05-20T01:19:51.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:51.930+0000 7f48d6644700 1 -- 172.21.15.16:0/912711499 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48d006e190 msgr2=0x7f48d0160460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:51.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:51.930+0000 7f48d6644700 1 --2- 172.21.15.16:0/912711499 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48d006e190 0x7f48d0160460 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f48c400c4d0 tx=0x7f48c400c7e0 comp rx=0 tx=0).stop 2023-05-20T01:19:51.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:51.930+0000 7f48d6644700 1 -- 172.21.15.16:0/912711499 shutdown_connections 2023-05-20T01:19:51.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:51.930+0000 7f48d6644700 1 --2- 172.21.15.16:0/912711499 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f48bc06ace0 0x7f48bc06d190 secure :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f48c0009130 tx=0x7f48c0008030 comp rx=0 tx=0).stop 2023-05-20T01:19:51.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:51.930+0000 7f48d6644700 1 --2- 172.21.15.16:0/912711499 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f48d015cec0 0x7f48d01609a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:51.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:51.930+0000 7f48d6644700 1 --2- 172.21.15.16:0/912711499 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f48d006a510 0x7f48d01f33a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:51.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:51.930+0000 7f48d6644700 1 --2- 172.21.15.16:0/912711499 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48d006e190 0x7f48d0160460 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:51.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:51.930+0000 7f48d6644700 1 -- 172.21.15.16:0/912711499 >> 172.21.15.16:0/912711499 conn(0x7f48d006c260 msgr2=0x7f48d00b40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:19:51.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:51.934+0000 7f48d6644700 1 -- 172.21.15.16:0/912711499 shutdown_connections 2023-05-20T01:19:51.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:51.934+0000 7f48d6644700 1 -- 172.21.15.16:0/912711499 wait complete. 2023-05-20T01:19:51.946 INFO:teuthology.orchestra.run.smithi016.stderr:new fs with metadata pool 9 and data pool 10 2023-05-20T01:19:51.959 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-20T01:19:52.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.080+0000 7fdafadf4700 1 Processor -- start 2023-05-20T01:19:52.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.080+0000 7fdafadf4700 1 -- start start 2023-05-20T01:19:52.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.081+0000 7fdafadf4700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fdaf41585d0 0x7fdaf415ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:52.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.081+0000 7fdafadf4700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdaf415b360 0x7fdaf406a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:52.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.081+0000 7fdafadf4700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdaf40ccf40 con 0x7fdaf40cc9c0 2023-05-20T01:19:52.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.081+0000 7fdafadf4700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdaf415e540 con 0x7fdaf41585d0 2023-05-20T01:19:52.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.081+0000 7fdafadf4700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf40cd220 con 0x7fdaf415b360 2023-05-20T01:19:52.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.081+0000 7fdaf8b90700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fdaf41585d0 0x7fdaf415ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:52.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.081+0000 7fdaf9391700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fdaf40cc9c0 0x7fdaf415e990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:45304/0 (socket says 172.21.15.16:45304) 2023-05-20T01:19:52.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.081+0000 7fdaf9391700 1 -- 172.21.15.16:0/44363603 learned_addr learned my addr 172.21.15.16:0/44363603 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:19:52.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.082+0000 7fdaf8b90700 1 -- 172.21.15.16:0/44363603 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdaf415b360 msgr2=0x7fdaf406a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:19:52.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.082+0000 7fdaf8b90700 1 --2- 172.21.15.16:0/44363603 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdaf415b360 0x7fdaf406a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:52.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.082+0000 7fdaf8b90700 1 -- 172.21.15.16:0/44363603 >> v1:172.21.15.16:6789/0 conn(0x7fdaf40cc9c0 legacy=0x7fdaf415e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:52.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.082+0000 7fdaf8b90700 1 -- 172.21.15.16:0/44363603 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaf40cd500 con 0x7fdaf41585d0 2023-05-20T01:19:52.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.082+0000 7fdaf37fe700 1 -- 172.21.15.16:0/44363603 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3179961797 0 0) 0x7fdaf40ccf40 con 0x7fdaf40cc9c0 2023-05-20T01:19:52.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.082+0000 7fdaf8b90700 1 --2- 172.21.15.16:0/44363603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdaf41585d0 0x7fdaf415ae20 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fdae8009840 tx=0x7fdae8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf177282dc6387c8 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:52.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.082+0000 7fdaf37fe700 1 -- 172.21.15.16:0/44363603 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae8011830 con 0x7fdaf41585d0 2023-05-20T01:19:52.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.082+0000 7fdaf37fe700 1 -- 172.21.15.16:0/44363603 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdae8011990 con 0x7fdaf41585d0 2023-05-20T01:19:52.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.082+0000 7fdaf37fe700 1 -- 172.21.15.16:0/44363603 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae8011d20 con 0x7fdaf41585d0 2023-05-20T01:19:52.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.083+0000 7fdafadf4700 1 -- 172.21.15.16:0/44363603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdaf41585d0 msgr2=0x7fdaf415ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:52.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.083+0000 7fdafadf4700 1 --2- 172.21.15.16:0/44363603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdaf41585d0 0x7fdaf415ae20 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fdae8009840 tx=0x7fdae8009b50 comp rx=0 tx=0).stop 2023-05-20T01:19:52.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.083+0000 7fdafadf4700 1 -- 172.21.15.16:0/44363603 shutdown_connections 2023-05-20T01:19:52.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.083+0000 7fdafadf4700 1 --2- 172.21.15.16:0/44363603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdaf41585d0 0x7fdaf415ae20 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:52.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.083+0000 7fdafadf4700 1 --2- 172.21.15.16:0/44363603 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdaf415b360 0x7fdaf406a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:52.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.083+0000 7fdafadf4700 1 -- 172.21.15.16:0/44363603 >> 172.21.15.16:0/44363603 conn(0x7fdaf40cffb0 msgr2=0x7fdaf40d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:19:52.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.083+0000 7fdafadf4700 1 -- 172.21.15.16:0/44363603 shutdown_connections 2023-05-20T01:19:52.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.083+0000 7fdafadf4700 1 -- 172.21.15.16:0/44363603 wait complete. 2023-05-20T01:19:52.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.084+0000 7fdafadf4700 1 Processor -- start 2023-05-20T01:19:52.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.084+0000 7fdafadf4700 1 -- start start 2023-05-20T01:19:52.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.085+0000 7fdafadf4700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdaf40cc9c0 0x7fdaf414c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:52.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.085+0000 7fdafadf4700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdaf41585d0 0x7fdaf40cf5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:52.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.085+0000 7fdafadf4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdaf415b360 0x7fdaf4154b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:52.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.085+0000 7fdafadf4700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf41f5e20 con 0x7fdaf415b360 2023-05-20T01:19:52.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.085+0000 7fdafadf4700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf41f6100 con 0x7fdaf40cc9c0 2023-05-20T01:19:52.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.085+0000 7fdafadf4700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf41f63e0 con 0x7fdaf41585d0 2023-05-20T01:19:52.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.085+0000 7fdaf9391700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdaf415b360 0x7fdaf4154b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:52.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.085+0000 7fdaf9391700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdaf415b360 0x7fdaf4154b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33782/0 (socket says 172.21.15.16:33782) 2023-05-20T01:19:52.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.085+0000 7fdaf9391700 1 -- 172.21.15.16:0/10556196 learned_addr learned my addr 172.21.15.16:0/10556196 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:19:52.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.085+0000 7fdaf8b90700 1 --2- 172.21.15.16:0/10556196 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdaf40cc9c0 0x7fdaf414c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:52.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.085+0000 7fdaf3fff700 1 --2- 172.21.15.16:0/10556196 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdaf41585d0 0x7fdaf40cf5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:52.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.085+0000 7fdaf9391700 1 -- 172.21.15.16:0/10556196 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdaf41585d0 msgr2=0x7fdaf40cf5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:52.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.085+0000 7fdaf9391700 1 --2- 172.21.15.16:0/10556196 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdaf41585d0 0x7fdaf40cf5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:52.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.085+0000 7fdaf9391700 1 -- 172.21.15.16:0/10556196 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdaf40cc9c0 msgr2=0x7fdaf414c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:52.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.085+0000 7fdaf9391700 1 --2- 172.21.15.16:0/10556196 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdaf40cc9c0 0x7fdaf414c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:52.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.085+0000 7fdaf9391700 1 -- 172.21.15.16:0/10556196 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdae80094a0 con 0x7fdaf415b360 2023-05-20T01:19:52.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.085+0000 7fdaf9391700 1 --2- 172.21.15.16:0/10556196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdaf415b360 0x7fdaf4154b60 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fdaec002230 tx=0x7fdaec00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:52.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.085+0000 7fdaf17fa700 1 -- 172.21.15.16:0/10556196 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaec00ac20 con 0x7fdaf415b360 2023-05-20T01:19:52.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.085+0000 7fdaf17fa700 1 -- 172.21.15.16:0/10556196 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdaec00ada0 con 0x7fdaf415b360 2023-05-20T01:19:52.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.086+0000 7fdafadf4700 1 -- 172.21.15.16:0/10556196 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdaf40719a0 con 0x7fdaf415b360 2023-05-20T01:19:52.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.086+0000 7fdaf17fa700 1 -- 172.21.15.16:0/10556196 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaec007310 con 0x7fdaf415b360 2023-05-20T01:19:52.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.086+0000 7fdafadf4700 1 -- 172.21.15.16:0/10556196 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdaf40cd7e0 con 0x7fdaf415b360 2023-05-20T01:19:52.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.086+0000 7fdafadf4700 1 -- 172.21.15.16:0/10556196 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdaf415f0a0 con 0x7fdaf415b360 2023-05-20T01:19:52.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.087+0000 7fdaf17fa700 1 -- 172.21.15.16:0/10556196 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fdaec007470 con 0x7fdaf415b360 2023-05-20T01:19:52.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.087+0000 7fdaf17fa700 1 --2- 172.21.15.16:0/10556196 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fdad806ace0 0x7fdad806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:52.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.087+0000 7fdaf17fa700 1 -- 172.21.15.16:0/10556196 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fdaec089450 con 0x7fdaf415b360 2023-05-20T01:19:52.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.088+0000 7fdaf8b90700 1 --2- 172.21.15.16:0/10556196 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fdad806ace0 0x7fdad806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:52.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.088+0000 7fdaf8b90700 1 --2- 172.21.15.16:0/10556196 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fdad806ace0 0x7fdad806d190 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fdae80090b0 tx=0x7fdae8008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:52.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.092+0000 7fdaf17fa700 1 -- 172.21.15.16:0/10556196 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fdaec054450 con 0x7fdaf415b360 2023-05-20T01:19:52.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.290+0000 7fdafadf4700 1 -- 172.21.15.16:0/10556196 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdaf40747b0 con 0x7fdaf415b360 2023-05-20T01:19:52.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.292+0000 7fdaf17fa700 1 -- 172.21.15.16:0/10556196 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) v1 ==== 74+0+18417 (secure 0 0 0) 0x7fdaec057a70 con 0x7fdaf415b360 2023-05-20T01:19:52.296 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:19:52.296 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":63,"fsid":"c2245cc5-ecab-4087-a59b-bc81db901676","created":"2023-05-20T01:16:09.549230+0000","modified":"2023-05-20T01:19:51.915038+0000","last_up_change":"2023-05-20T01:16:19.788624+0000","last_in_change":"2023-05-20T01:16:12.092160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-05-20T01:16:20.934366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-05-20T01:16:21.372294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-05-20T01:19:49.878849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-05-20T01:19:50.887306+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"317a83cd-19c8-4958-a301-6087ef342289","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6825","nonce":3923634179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6827","nonce":3923634179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6831","nonce":3923634179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6829","nonce":3923634179}]},"public_addr":"172.21.15.16:6825/3923634179","cluster_addr":"172.21.15.16:6827/3923634179","heartbeat_back_addr":"172.21.15.16:6831/3923634179","heartbeat_front_addr":"172.21.15.16:6829/3923634179","state":["exists","up"]},{"osd":1,"uuid":"5581046f-8219-4089-8d7d-b7846399505e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6809","nonce":3860264220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6811","nonce":3860264220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6815","nonce":3860264220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6813","nonce":3860264220}]},"public_addr":"172.21.15.16:6809/3860264220","cluster_addr":"172.21.15.16:6811/3860264220","heartbeat_back_addr":"172.21.15.16:6815/3860264220","heartbeat_front_addr":"172.21.15.16:6813/3860264220","state":["exists","up"]},{"osd":2,"uuid":"072e9e66-4133-478e-8d4a-64d46c2963f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6817","nonce":293187030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6819","nonce":293187030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6823","nonce":293187030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6821","nonce":293187030}]},"public_addr":"172.21.15.16:6817/293187030","cluster_addr":"172.21.15.16:6819/293187030","heartbeat_back_addr":"172.21.15.16:6823/293187030","heartbeat_front_addr":"172.21.15.16:6821/293187030","state":["exists","up"]},{"osd":3,"uuid":"11e1bba6-00f8-48dd-9028-d8d716b7f145","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6801","nonce":3669781890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6803","nonce":3669781890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6807","nonce":3669781890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6805","nonce":3669781890}]},"public_addr":"172.21.15.16:6801/3669781890","cluster_addr":"172.21.15.16:6803/3669781890","heartbeat_back_addr":"172.21.15.16:6807/3669781890","heartbeat_front_addr":"172.21.15.16:6805/3669781890","state":["exists","up"]},{"osd":4,"uuid":"b8909c3a-cf70-46cb-8d0b-009c707cf9d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6819","nonce":506317223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6823","nonce":506317223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6829","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6831","nonce":506317223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6827","nonce":506317223}]},"public_addr":"172.21.15.132:6819/506317223","cluster_addr":"172.21.15.132:6823/506317223","heartbeat_back_addr":"172.21.15.132:6831/506317223","heartbeat_front_addr":"172.21.15.132:6827/506317223","state":["exists","up"]},{"osd":5,"uuid":"b1818304-4583-47bf-a6ec-0bfbb5166233","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6818","nonce":1844689995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6822","nonce":1844689995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6830","nonce":1844689995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6826","nonce":1844689995}]},"public_addr":"172.21.15.132:6818/1844689995","cluster_addr":"172.21.15.132:6822/1844689995","heartbeat_back_addr":"172.21.15.132:6830/1844689995","heartbeat_front_addr":"172.21.15.132:6826/1844689995","state":["exists","up"]},{"osd":6,"uuid":"d37f4691-eed1-4a93-8818-18354f03f4a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6801","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6802","nonce":727581123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6805","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6807","nonce":727581123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6813","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6815","nonce":727581123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6811","nonce":727581123}]},"public_addr":"172.21.15.132:6802/727581123","cluster_addr":"172.21.15.132:6807/727581123","heartbeat_back_addr":"172.21.15.132:6815/727581123","heartbeat_front_addr":"172.21.15.132:6811/727581123","state":["exists","up"]},{"osd":7,"uuid":"4f712f62-5d1d-4816-8042-03209144f8b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6803","nonce":466001149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6806","nonce":466001149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6814","nonce":466001149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6810","nonce":466001149}]},"public_addr":"172.21.15.132:6803/466001149","cluster_addr":"172.21.15.132:6806/466001149","heartbeat_back_addr":"172.21.15.132:6814/466001149","heartbeat_front_addr":"172.21.15.132:6810/466001149","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.338863+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.278680+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305694+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.246835+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.276237+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.304441+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305421+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.267050+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-20T01:19:52.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.294+0000 7fdafadf4700 1 -- 172.21.15.16:0/10556196 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fdad806ace0 msgr2=0x7fdad806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:52.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.294+0000 7fdafadf4700 1 --2- 172.21.15.16:0/10556196 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fdad806ace0 0x7fdad806d190 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fdae80090b0 tx=0x7fdae8008030 comp rx=0 tx=0).stop 2023-05-20T01:19:52.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.294+0000 7fdafadf4700 1 -- 172.21.15.16:0/10556196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdaf415b360 msgr2=0x7fdaf4154b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:52.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.294+0000 7fdafadf4700 1 --2- 172.21.15.16:0/10556196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdaf415b360 0x7fdaf4154b60 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fdaec002230 tx=0x7fdaec00a060 comp rx=0 tx=0).stop 2023-05-20T01:19:52.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.294+0000 7fdafadf4700 1 -- 172.21.15.16:0/10556196 shutdown_connections 2023-05-20T01:19:52.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.294+0000 7fdafadf4700 1 --2- 172.21.15.16:0/10556196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdaf415b360 0x7fdaf4154b60 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:52.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.294+0000 7fdafadf4700 1 --2- 172.21.15.16:0/10556196 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdaf40cc9c0 0x7fdaf414c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:52.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.294+0000 7fdafadf4700 1 --2- 172.21.15.16:0/10556196 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fdad806ace0 0x7fdad806d190 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:52.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.294+0000 7fdafadf4700 1 --2- 172.21.15.16:0/10556196 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdaf41585d0 0x7fdaf40cf5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:52.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.294+0000 7fdafadf4700 1 -- 172.21.15.16:0/10556196 >> 172.21.15.16:0/10556196 conn(0x7fdaf40cffb0 msgr2=0x7fdaf4158fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:19:52.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.294+0000 7fdafadf4700 1 -- 172.21.15.16:0/10556196 shutdown_connections 2023-05-20T01:19:52.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.294+0000 7fdafadf4700 1 -- 172.21.15.16:0/10556196 wait complete. 2023-05-20T01:19:52.310 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-20T01:19:52.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.432+0000 7f4632c78700 1 Processor -- start 2023-05-20T01:19:52.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.432+0000 7f4632c78700 1 -- start start 2023-05-20T01:19:52.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.433+0000 7f4632c78700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f462c0ccbd0 0x7f462c0cb000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:52.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.433+0000 7f4632c78700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f462c06cb90 0x7f462c06cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:52.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.433+0000 7f4632c78700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f462c0cd050 con 0x7f462c153eb0 2023-05-20T01:19:52.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.433+0000 7f4632c78700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f462c159d30 con 0x7f462c0ccbd0 2023-05-20T01:19:52.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.433+0000 7f4632c78700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f462c0cb540 con 0x7f462c06cb90 2023-05-20T01:19:52.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.433+0000 7f462bfff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f462c0ccbd0 0x7f462c0cb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:52.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.433+0000 7f462bfff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f462c0ccbd0 0x7f462c0cb000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33788/0 (socket says 172.21.15.16:33788) 2023-05-20T01:19:52.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.433+0000 7f462bfff700 1 -- 172.21.15.16:0/2161362659 learned_addr learned my addr 172.21.15.16:0/2161362659 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:19:52.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.433+0000 7f4631215700 1 --2- 172.21.15.16:0/2161362659 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f462c06cb90 0x7f462c06cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:52.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.433+0000 7f462bfff700 1 -- 172.21.15.16:0/2161362659 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f462c06cb90 msgr2=0x7f462c06cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:19:52.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.433+0000 7f462bfff700 1 --2- 172.21.15.16:0/2161362659 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f462c06cb90 0x7f462c06cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:52.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.433+0000 7f462bfff700 1 -- 172.21.15.16:0/2161362659 >> v1:172.21.15.132:6789/0 conn(0x7f462c153eb0 legacy=0x7f462c15a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:52.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.433+0000 7f462bfff700 1 -- 172.21.15.16:0/2161362659 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f462c06d4f0 con 0x7f462c0ccbd0 2023-05-20T01:19:52.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.433+0000 7f4631215700 1 --2- 172.21.15.16:0/2161362659 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f462c06cb90 0x7f462c06cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:19:52.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.434+0000 7f462bfff700 1 --2- 172.21.15.16:0/2161362659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f462c0ccbd0 0x7f462c0cb000 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f462c159e70 tx=0x7f4618009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b14949c3c001cb4 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:52.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.434+0000 7f462b7fe700 1 -- 172.21.15.16:0/2161362659 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4618004230 con 0x7f462c0ccbd0 2023-05-20T01:19:52.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.434+0000 7f462b7fe700 1 -- 172.21.15.16:0/2161362659 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4618004390 con 0x7f462c0ccbd0 2023-05-20T01:19:52.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.434+0000 7f462b7fe700 1 -- 172.21.15.16:0/2161362659 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46180036a0 con 0x7f462c0ccbd0 2023-05-20T01:19:52.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.434+0000 7f4632c78700 1 -- 172.21.15.16:0/2161362659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f462c0ccbd0 msgr2=0x7f462c0cb000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:52.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.434+0000 7f4632c78700 1 --2- 172.21.15.16:0/2161362659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f462c0ccbd0 0x7f462c0cb000 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f462c159e70 tx=0x7f4618009bd0 comp rx=0 tx=0).stop 2023-05-20T01:19:52.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.434+0000 7f4632c78700 1 -- 172.21.15.16:0/2161362659 shutdown_connections 2023-05-20T01:19:52.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.434+0000 7f4632c78700 1 --2- 172.21.15.16:0/2161362659 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f462c06cb90 0x7f462c06cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:52.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.434+0000 7f4632c78700 1 --2- 172.21.15.16:0/2161362659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f462c0ccbd0 0x7f462c0cb000 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:52.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.434+0000 7f4632c78700 1 -- 172.21.15.16:0/2161362659 >> 172.21.15.16:0/2161362659 conn(0x7f462c0661b0 msgr2=0x7f462c066580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:19:52.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.434+0000 7f4632c78700 1 -- 172.21.15.16:0/2161362659 shutdown_connections 2023-05-20T01:19:52.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.434+0000 7f4632c78700 1 -- 172.21.15.16:0/2161362659 wait complete. 2023-05-20T01:19:52.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.435+0000 7f4632c78700 1 Processor -- start 2023-05-20T01:19:52.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.436+0000 7f4632c78700 1 -- start start 2023-05-20T01:19:52.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.436+0000 7f4632c78700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f462c06cb90 0x7f462c14d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:52.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.436+0000 7f4632c78700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f462c0ccbd0 0x7f462c1f22c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:52.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.436+0000 7f4632c78700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f462c153eb0 0x7f462c1f5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:52.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.436+0000 7f4632c78700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f462c14e230 con 0x7f462c06cb90 2023-05-20T01:19:52.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.436+0000 7f4632c78700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f462c14d970 con 0x7f462c0ccbd0 2023-05-20T01:19:52.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.436+0000 7f4632c78700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f462c1f62d0 con 0x7f462c153eb0 2023-05-20T01:19:52.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.437+0000 7f4630a14700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f462c06cb90 0x7f462c14d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:52.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.437+0000 7f4630a14700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f462c06cb90 0x7f462c14d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33792/0 (socket says 172.21.15.16:33792) 2023-05-20T01:19:52.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.437+0000 7f4630a14700 1 -- 172.21.15.16:0/1622497397 learned_addr learned my addr 172.21.15.16:0/1622497397 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:19:52.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.437+0000 7f462bfff700 1 --2- 172.21.15.16:0/1622497397 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f462c0ccbd0 0x7f462c1f22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:52.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.437+0000 7f4631215700 1 --2- 172.21.15.16:0/1622497397 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f462c153eb0 0x7f462c1f5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:52.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.437+0000 7f4630a14700 1 -- 172.21.15.16:0/1622497397 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f462c153eb0 msgr2=0x7f462c1f5880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:52.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.437+0000 7f4630a14700 1 --2- 172.21.15.16:0/1622497397 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f462c153eb0 0x7f462c1f5880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:52.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.437+0000 7f4630a14700 1 -- 172.21.15.16:0/1622497397 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f462c0ccbd0 msgr2=0x7f462c1f22c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:52.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.437+0000 7f4630a14700 1 --2- 172.21.15.16:0/1622497397 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f462c0ccbd0 0x7f462c1f22c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:52.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.437+0000 7f4630a14700 1 -- 172.21.15.16:0/1622497397 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4618009550 con 0x7f462c06cb90 2023-05-20T01:19:52.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.437+0000 7f462bfff700 1 --2- 172.21.15.16:0/1622497397 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f462c0ccbd0 0x7f462c1f22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:19:52.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.437+0000 7f4630a14700 1 --2- 172.21.15.16:0/1622497397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f462c06cb90 0x7f462c14d430 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f462000c450 tx=0x7f462000c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:52.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.437+0000 7f46297fa700 1 -- 172.21.15.16:0/1622497397 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46200052d0 con 0x7f462c06cb90 2023-05-20T01:19:52.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.437+0000 7f46297fa700 1 -- 172.21.15.16:0/1622497397 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4620005430 con 0x7f462c06cb90 2023-05-20T01:19:52.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.437+0000 7f4632c78700 1 -- 172.21.15.16:0/1622497397 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f462c0cba30 con 0x7f462c06cb90 2023-05-20T01:19:52.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.437+0000 7f46297fa700 1 -- 172.21.15.16:0/1622497397 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46200038c0 con 0x7f462c06cb90 2023-05-20T01:19:52.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.437+0000 7f4632c78700 1 -- 172.21.15.16:0/1622497397 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f462c0cb750 con 0x7f462c06cb90 2023-05-20T01:19:52.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.438+0000 7f4632c78700 1 -- 172.21.15.16:0/1622497397 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f462c0cbd10 con 0x7f462c06cb90 2023-05-20T01:19:52.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.439+0000 7f46297fa700 1 -- 172.21.15.16:0/1622497397 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4620003a20 con 0x7f462c06cb90 2023-05-20T01:19:52.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.439+0000 7f46297fa700 1 --2- 172.21.15.16:0/1622497397 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f461006ace0 0x7f461006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:52.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.439+0000 7f46297fa700 1 -- 172.21.15.16:0/1622497397 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f46200888d0 con 0x7f462c06cb90 2023-05-20T01:19:52.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.439+0000 7f462bfff700 1 --2- 172.21.15.16:0/1622497397 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f461006ace0 0x7f461006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:52.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.440+0000 7f462bfff700 1 --2- 172.21.15.16:0/1622497397 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f461006ace0 0x7f461006d190 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f4618000c00 tx=0x7f4618008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:52.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.444+0000 7f46297fa700 1 -- 172.21.15.16:0/1622497397 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f46200538d0 con 0x7f462c06cb90 2023-05-20T01:19:52.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.644+0000 7f4632c78700 1 -- 172.21.15.16:0/1622497397 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f462c14e370 con 0x7f462c06cb90 2023-05-20T01:19:52.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.645+0000 7f46297fa700 1 -- 172.21.15.16:0/1622497397 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) v1 ==== 74+0+18417 (secure 0 0 0) 0x7f4620056ef0 con 0x7f462c06cb90 2023-05-20T01:19:52.650 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:19:52.650 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":63,"fsid":"c2245cc5-ecab-4087-a59b-bc81db901676","created":"2023-05-20T01:16:09.549230+0000","modified":"2023-05-20T01:19:51.915038+0000","last_up_change":"2023-05-20T01:16:19.788624+0000","last_in_change":"2023-05-20T01:16:12.092160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-05-20T01:16:20.934366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-05-20T01:16:21.372294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-05-20T01:19:49.878849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-05-20T01:19:50.887306+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"317a83cd-19c8-4958-a301-6087ef342289","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6825","nonce":3923634179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6827","nonce":3923634179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6831","nonce":3923634179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6829","nonce":3923634179}]},"public_addr":"172.21.15.16:6825/3923634179","cluster_addr":"172.21.15.16:6827/3923634179","heartbeat_back_addr":"172.21.15.16:6831/3923634179","heartbeat_front_addr":"172.21.15.16:6829/3923634179","state":["exists","up"]},{"osd":1,"uuid":"5581046f-8219-4089-8d7d-b7846399505e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6809","nonce":3860264220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6811","nonce":3860264220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6815","nonce":3860264220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6813","nonce":3860264220}]},"public_addr":"172.21.15.16:6809/3860264220","cluster_addr":"172.21.15.16:6811/3860264220","heartbeat_back_addr":"172.21.15.16:6815/3860264220","heartbeat_front_addr":"172.21.15.16:6813/3860264220","state":["exists","up"]},{"osd":2,"uuid":"072e9e66-4133-478e-8d4a-64d46c2963f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6817","nonce":293187030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6819","nonce":293187030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6823","nonce":293187030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6821","nonce":293187030}]},"public_addr":"172.21.15.16:6817/293187030","cluster_addr":"172.21.15.16:6819/293187030","heartbeat_back_addr":"172.21.15.16:6823/293187030","heartbeat_front_addr":"172.21.15.16:6821/293187030","state":["exists","up"]},{"osd":3,"uuid":"11e1bba6-00f8-48dd-9028-d8d716b7f145","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6801","nonce":3669781890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6803","nonce":3669781890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6807","nonce":3669781890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6805","nonce":3669781890}]},"public_addr":"172.21.15.16:6801/3669781890","cluster_addr":"172.21.15.16:6803/3669781890","heartbeat_back_addr":"172.21.15.16:6807/3669781890","heartbeat_front_addr":"172.21.15.16:6805/3669781890","state":["exists","up"]},{"osd":4,"uuid":"b8909c3a-cf70-46cb-8d0b-009c707cf9d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6819","nonce":506317223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6823","nonce":506317223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6829","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6831","nonce":506317223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6827","nonce":506317223}]},"public_addr":"172.21.15.132:6819/506317223","cluster_addr":"172.21.15.132:6823/506317223","heartbeat_back_addr":"172.21.15.132:6831/506317223","heartbeat_front_addr":"172.21.15.132:6827/506317223","state":["exists","up"]},{"osd":5,"uuid":"b1818304-4583-47bf-a6ec-0bfbb5166233","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6818","nonce":1844689995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6822","nonce":1844689995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6830","nonce":1844689995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6826","nonce":1844689995}]},"public_addr":"172.21.15.132:6818/1844689995","cluster_addr":"172.21.15.132:6822/1844689995","heartbeat_back_addr":"172.21.15.132:6830/1844689995","heartbeat_front_addr":"172.21.15.132:6826/1844689995","state":["exists","up"]},{"osd":6,"uuid":"d37f4691-eed1-4a93-8818-18354f03f4a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6801","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6802","nonce":727581123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6805","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6807","nonce":727581123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6813","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6815","nonce":727581123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6811","nonce":727581123}]},"public_addr":"172.21.15.132:6802/727581123","cluster_addr":"172.21.15.132:6807/727581123","heartbeat_back_addr":"172.21.15.132:6815/727581123","heartbeat_front_addr":"172.21.15.132:6811/727581123","state":["exists","up"]},{"osd":7,"uuid":"4f712f62-5d1d-4816-8042-03209144f8b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6803","nonce":466001149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6806","nonce":466001149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6814","nonce":466001149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6810","nonce":466001149}]},"public_addr":"172.21.15.132:6803/466001149","cluster_addr":"172.21.15.132:6806/466001149","heartbeat_back_addr":"172.21.15.132:6814/466001149","heartbeat_front_addr":"172.21.15.132:6810/466001149","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.338863+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.278680+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305694+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.246835+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.276237+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.304441+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305421+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.267050+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-20T01:19:52.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.647+0000 7f4632c78700 1 -- 172.21.15.16:0/1622497397 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f461006ace0 msgr2=0x7f461006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:52.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.647+0000 7f4632c78700 1 --2- 172.21.15.16:0/1622497397 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f461006ace0 0x7f461006d190 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f4618000c00 tx=0x7f4618008030 comp rx=0 tx=0).stop 2023-05-20T01:19:52.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.647+0000 7f4632c78700 1 -- 172.21.15.16:0/1622497397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f462c06cb90 msgr2=0x7f462c14d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:52.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.647+0000 7f4632c78700 1 --2- 172.21.15.16:0/1622497397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f462c06cb90 0x7f462c14d430 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f462000c450 tx=0x7f462000c810 comp rx=0 tx=0).stop 2023-05-20T01:19:52.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.647+0000 7f4632c78700 1 -- 172.21.15.16:0/1622497397 shutdown_connections 2023-05-20T01:19:52.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.647+0000 7f4632c78700 1 --2- 172.21.15.16:0/1622497397 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f461006ace0 0x7f461006d190 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:52.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.647+0000 7f4632c78700 1 --2- 172.21.15.16:0/1622497397 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f462c153eb0 0x7f462c1f5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:52.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.647+0000 7f4632c78700 1 --2- 172.21.15.16:0/1622497397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f462c06cb90 0x7f462c14d430 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:52.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.647+0000 7f4632c78700 1 --2- 172.21.15.16:0/1622497397 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f462c0ccbd0 0x7f462c1f22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:52.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.647+0000 7f4632c78700 1 -- 172.21.15.16:0/1622497397 >> 172.21.15.16:0/1622497397 conn(0x7f462c0661b0 msgr2=0x7f462c0cd2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:19:52.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.647+0000 7f4632c78700 1 -- 172.21.15.16:0/1622497397 shutdown_connections 2023-05-20T01:19:52.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.647+0000 7f4632c78700 1 -- 172.21.15.16:0/1622497397 wait complete. 2023-05-20T01:19:52.663 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-05-20T01:19:52.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.787+0000 7f59a39d0700 1 Processor -- start 2023-05-20T01:19:52.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.787+0000 7f59a39d0700 1 -- start start 2023-05-20T01:19:52.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.788+0000 7f59a39d0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f599c146710 0x7f599c06b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:52.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.788+0000 7f59a39d0700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f599c0cb6c0 con 0x7f599c06bb60 2023-05-20T01:19:52.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.788+0000 7f59a39d0700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f599c0ce840 con 0x7f599c0ce1e0 2023-05-20T01:19:52.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.788+0000 7f59a39d0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f599c0ce660 con 0x7f599c146710 2023-05-20T01:19:52.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.788+0000 7f59a0f6b700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f599c06bb60 0x7f599c0cdad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:45328/0 (socket says 172.21.15.16:45328) 2023-05-20T01:19:52.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.788+0000 7f59a0f6b700 1 -- 172.21.15.16:0/2593506730 learned_addr learned my addr 172.21.15.16:0/2593506730 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:19:52.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.788+0000 7f59a176c700 1 --2- 172.21.15.16:0/2593506730 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f599c146710 0x7f599c06b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:52.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.788+0000 7f5993fff700 1 -- 172.21.15.16:0/2593506730 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3155005156 0 0) 0x7f599c0cb6c0 con 0x7f599c06bb60 2023-05-20T01:19:52.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.788+0000 7f5993fff700 1 -- 172.21.15.16:0/2593506730 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f59880035b0 con 0x7f599c06bb60 2023-05-20T01:19:52.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.788+0000 7f5993fff700 1 -- 172.21.15.16:0/2593506730 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1605624092 0 0) 0x7f599c0ce840 con 0x7f599c0ce1e0 2023-05-20T01:19:52.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.788+0000 7f5993fff700 1 -- 172.21.15.16:0/2593506730 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f599c0cb6c0 con 0x7f599c0ce1e0 2023-05-20T01:19:52.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.788+0000 7f5993fff700 1 -- 172.21.15.16:0/2593506730 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1959877172 0 0) 0x7f59880035b0 con 0x7f599c06bb60 2023-05-20T01:19:52.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.788+0000 7f5993fff700 1 -- 172.21.15.16:0/2593506730 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f599c146710 msgr2=0x7f599c06b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:19:52.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.788+0000 7f5993fff700 1 --2- 172.21.15.16:0/2593506730 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f599c146710 0x7f599c06b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:52.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.788+0000 7f5993fff700 1 -- 172.21.15.16:0/2593506730 >> v1:172.21.15.132:6789/0 conn(0x7f599c0ce1e0 legacy=0x7f599c0cc3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:52.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.789+0000 7f5993fff700 1 -- 172.21.15.16:0/2593506730 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f599c0cb910 con 0x7f599c06bb60 2023-05-20T01:19:52.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.789+0000 7f5993fff700 1 -- 172.21.15.16:0/2593506730 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7f598c001670 con 0x7f599c06bb60 2023-05-20T01:19:52.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.789+0000 7f5993fff700 1 -- 172.21.15.16:0/2593506730 >> v1:172.21.15.16:6789/0 conn(0x7f599c06bb60 legacy=0x7f599c0cdad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:52.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.789+0000 7f5993fff700 1 --2- 172.21.15.16:0/2593506730 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f599c146710 0x7f5988003bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:52.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.789+0000 7f5993fff700 1 --2- 172.21.15.16:0/2593506730 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5988004120 0x7f5988004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:52.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.789+0000 7f5993fff700 1 --2- 172.21.15.16:0/2593506730 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5988004d90 0x7f5988005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:52.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.789+0000 7f5993fff700 1 -- 172.21.15.16:0/2593506730 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f599c0ce660 con 0x7f5988004120 2023-05-20T01:19:52.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.789+0000 7f5993fff700 1 -- 172.21.15.16:0/2593506730 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f598800bac0 con 0x7f5988004d90 2023-05-20T01:19:52.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.789+0000 7f5993fff700 1 -- 172.21.15.16:0/2593506730 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f598800bc80 con 0x7f599c146710 2023-05-20T01:19:52.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.789+0000 7f59a0f6b700 1 --2- 172.21.15.16:0/2593506730 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5988004120 0x7f5988004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:52.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.789+0000 7f59a176c700 1 --2- 172.21.15.16:0/2593506730 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f599c146710 0x7f5988003bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:52.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.789+0000 7f59a1f6d700 1 --2- 172.21.15.16:0/2593506730 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5988004d90 0x7f5988005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:52.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.789+0000 7f59a0f6b700 1 -- 172.21.15.16:0/2593506730 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f599c146710 msgr2=0x7f5988003bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:19:52.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.789+0000 7f59a0f6b700 1 --2- 172.21.15.16:0/2593506730 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f599c146710 0x7f5988003bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:52.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.789+0000 7f59a0f6b700 1 -- 172.21.15.16:0/2593506730 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5988004d90 msgr2=0x7f5988005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:19:52.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.789+0000 7f59a0f6b700 1 --2- 172.21.15.16:0/2593506730 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5988004d90 0x7f5988005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:52.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.789+0000 7f59a0f6b700 1 -- 172.21.15.16:0/2593506730 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59880035b0 con 0x7f5988004120 2023-05-20T01:19:52.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.789+0000 7f59a1f6d700 1 --2- 172.21.15.16:0/2593506730 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5988004d90 0x7f5988005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:19:52.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.789+0000 7f59a0f6b700 1 --2- 172.21.15.16:0/2593506730 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5988004120 0x7f5988004850 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f598c001eb0 tx=0x7f5988005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2df8996f0060d98a server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:52.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.789+0000 7f5993fff700 1 -- 172.21.15.16:0/2593506730 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f598c00cd20 con 0x7f5988004120 2023-05-20T01:19:52.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.789+0000 7f5993fff700 1 -- 172.21.15.16:0/2593506730 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f598c001af0 con 0x7f5988004120 2023-05-20T01:19:52.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.789+0000 7f5993fff700 1 -- 172.21.15.16:0/2593506730 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f598c00cea0 con 0x7f5988004120 2023-05-20T01:19:52.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.790+0000 7f59a39d0700 1 -- 172.21.15.16:0/2593506730 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5988004120 msgr2=0x7f5988004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:52.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.790+0000 7f59a39d0700 1 --2- 172.21.15.16:0/2593506730 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5988004120 0x7f5988004850 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f598c001eb0 tx=0x7f5988005880 comp rx=0 tx=0).stop 2023-05-20T01:19:52.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.790+0000 7f59a1f6d700 1 -- 172.21.15.16:0/2593506730 reap_dead start 2023-05-20T01:19:52.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.790+0000 7f59a39d0700 1 -- 172.21.15.16:0/2593506730 shutdown_connections 2023-05-20T01:19:52.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.790+0000 7f59a39d0700 1 -- 172.21.15.16:0/2593506730 >> 172.21.15.16:0/2593506730 conn(0x7f599c06ae30 msgr2=0x7f599c09a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:19:52.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.790+0000 7f59a39d0700 1 -- 172.21.15.16:0/2593506730 shutdown_connections 2023-05-20T01:19:52.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.790+0000 7f59a39d0700 1 -- 172.21.15.16:0/2593506730 wait complete. 2023-05-20T01:19:52.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.791+0000 7f59a39d0700 1 Processor -- start 2023-05-20T01:19:52.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.791+0000 7f59a39d0700 1 -- start start 2023-05-20T01:19:52.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.792+0000 7f59a39d0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5988004120 0x7f599c153500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:52.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.792+0000 7f59a39d0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f599c0c9ba0 0x7f599c0c67b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:52.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.792+0000 7f59a39d0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f599c0c5b50 0x7f599c0c6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:52.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.792+0000 7f59a39d0700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f599c0b17c0 con 0x7f599c0c5b50 2023-05-20T01:19:52.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.792+0000 7f59a39d0700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f599c0cabb0 con 0x7f5988004120 2023-05-20T01:19:52.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.792+0000 7f59a39d0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f599c0ca030 con 0x7f599c0c9ba0 2023-05-20T01:19:52.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.792+0000 7f59a1f6d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f599c0c5b50 0x7f599c0c6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:52.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.792+0000 7f59a1f6d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f599c0c5b50 0x7f599c0c6cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33816/0 (socket says 172.21.15.16:33816) 2023-05-20T01:19:52.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.792+0000 7f59a1f6d700 1 -- 172.21.15.16:0/3775625306 learned_addr learned my addr 172.21.15.16:0/3775625306 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:19:52.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.792+0000 7f59a176c700 1 --2- 172.21.15.16:0/3775625306 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5988004120 0x7f599c153500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:52.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.792+0000 7f59a0f6b700 1 --2- 172.21.15.16:0/3775625306 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f599c0c9ba0 0x7f599c0c67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:52.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.793+0000 7f59a1f6d700 1 -- 172.21.15.16:0/3775625306 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f599c0c9ba0 msgr2=0x7f599c0c67b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:52.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.793+0000 7f59a1f6d700 1 --2- 172.21.15.16:0/3775625306 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f599c0c9ba0 0x7f599c0c67b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:52.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.793+0000 7f59a1f6d700 1 -- 172.21.15.16:0/3775625306 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5988004120 msgr2=0x7f599c153500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:52.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.793+0000 7f59a1f6d700 1 --2- 172.21.15.16:0/3775625306 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5988004120 0x7f599c153500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:52.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.793+0000 7f59a1f6d700 1 -- 172.21.15.16:0/3775625306 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f599c0cbed0 con 0x7f599c0c5b50 2023-05-20T01:19:52.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.793+0000 7f59a1f6d700 1 --2- 172.21.15.16:0/3775625306 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f599c0c5b50 0x7f599c0c6cf0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f5994001a50 tx=0x7f599400c0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:52.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.793+0000 7f5991ffb700 1 -- 172.21.15.16:0/3775625306 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5994003c10 con 0x7f599c0c5b50 2023-05-20T01:19:52.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.793+0000 7f5991ffb700 1 -- 172.21.15.16:0/3775625306 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5994003db0 con 0x7f599c0c5b50 2023-05-20T01:19:52.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.793+0000 7f59a39d0700 1 -- 172.21.15.16:0/3775625306 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f599c0cbbf0 con 0x7f599c0c5b50 2023-05-20T01:19:52.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.793+0000 7f5991ffb700 1 -- 172.21.15.16:0/3775625306 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f599400c9a0 con 0x7f599c0c5b50 2023-05-20T01:19:52.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.793+0000 7f59a39d0700 1 -- 172.21.15.16:0/3775625306 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f599c06b620 con 0x7f599c0c5b50 2023-05-20T01:19:52.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.794+0000 7f59a39d0700 1 -- 172.21.15.16:0/3775625306 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f599c0747b0 con 0x7f599c0c5b50 2023-05-20T01:19:52.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.794+0000 7f5991ffb700 1 -- 172.21.15.16:0/3775625306 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f599400cb00 con 0x7f599c0c5b50 2023-05-20T01:19:52.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.795+0000 7f5991ffb700 1 --2- 172.21.15.16:0/3775625306 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f598006ace0 0x7f598006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:52.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.795+0000 7f5991ffb700 1 -- 172.21.15.16:0/3775625306 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f59940897d0 con 0x7f599c0c5b50 2023-05-20T01:19:52.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.795+0000 7f59a176c700 1 --2- 172.21.15.16:0/3775625306 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f598006ace0 0x7f598006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:52.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.795+0000 7f59a176c700 1 --2- 172.21.15.16:0/3775625306 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f598006ace0 0x7f598006d190 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f5998009ac0 tx=0x7f5998008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:52.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:52.799+0000 7f5991ffb700 1 -- 172.21.15.16:0/3775625306 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f59940546a0 con 0x7f599c0c5b50 2023-05-20T01:19:53.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:53.046+0000 7f59a39d0700 1 -- 172.21.15.16:0/3775625306 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f599c0cb4a0 con 0x7f599c0c5b50 2023-05-20T01:19:53.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:53.931+0000 7f5991ffb700 1 -- 172.21.15.16:0/3775625306 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v36) v1 ==== 118+0+0 (secure 0 0 0) 0x7f5994057cc0 con 0x7f599c0c5b50 2023-05-20T01:19:53.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:53.934+0000 7f59a39d0700 1 -- 172.21.15.16:0/3775625306 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f598006ace0 msgr2=0x7f598006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:53.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:53.934+0000 7f59a39d0700 1 --2- 172.21.15.16:0/3775625306 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f598006ace0 0x7f598006d190 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f5998009ac0 tx=0x7f5998008040 comp rx=0 tx=0).stop 2023-05-20T01:19:53.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:53.934+0000 7f59a39d0700 1 -- 172.21.15.16:0/3775625306 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f599c0c5b50 msgr2=0x7f599c0c6cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:53.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:53.934+0000 7f59a39d0700 1 --2- 172.21.15.16:0/3775625306 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f599c0c5b50 0x7f599c0c6cf0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f5994001a50 tx=0x7f599400c0e0 comp rx=0 tx=0).stop 2023-05-20T01:19:53.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:53.934+0000 7f59a39d0700 1 -- 172.21.15.16:0/3775625306 shutdown_connections 2023-05-20T01:19:53.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:53.934+0000 7f59a39d0700 1 --2- 172.21.15.16:0/3775625306 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f599c0c5b50 0x7f599c0c6cf0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:53.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:53.934+0000 7f59a39d0700 1 --2- 172.21.15.16:0/3775625306 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5988004120 0x7f599c153500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:53.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:53.934+0000 7f59a39d0700 1 --2- 172.21.15.16:0/3775625306 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f598006ace0 0x7f598006d190 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:53.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:53.934+0000 7f59a39d0700 1 --2- 172.21.15.16:0/3775625306 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f599c0c9ba0 0x7f599c0c67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:53.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:53.934+0000 7f59a39d0700 1 -- 172.21.15.16:0/3775625306 >> 172.21.15.16:0/3775625306 conn(0x7f599c06ae30 msgr2=0x7f599c06c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:19:53.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:53.934+0000 7f59a39d0700 1 -- 172.21.15.16:0/3775625306 shutdown_connections 2023-05-20T01:19:53.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:53.934+0000 7f59a39d0700 1 -- 172.21.15.16:0/3775625306 wait complete. 2023-05-20T01:19:53.953 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-05-20T01:19:53.954 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-05-20T01:19:54.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:54.085+0000 7f74ee843700 1 Processor -- start 2023-05-20T01:19:54.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:54.085+0000 7f74ee843700 1 -- start start 2023-05-20T01:19:54.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:54.086+0000 7f74ee843700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f74e815cd50 0x7f74e80d20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:54.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:54.086+0000 7f74ee843700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f74e80cfcb0 0x7f74e80d0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:54.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:54.086+0000 7f74ee843700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74e80d1c10 con 0x7f74e80cd920 2023-05-20T01:19:54.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:54.086+0000 7f74ee843700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74e80d0660 con 0x7f74e815cd50 2023-05-20T01:19:54.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:54.086+0000 7f74ee843700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74e815d120 con 0x7f74e80cfcb0 2023-05-20T01:19:54.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:54.086+0000 7f74e7fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f74e815cd50 0x7f74e80d20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:54.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:54.086+0000 7f74ecde0700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f74e80cd920 0x7f74e80b6120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:45352/0 (socket says 172.21.15.16:45352) 2023-05-20T01:19:54.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:54.086+0000 7f74ecde0700 1 -- 172.21.15.16:0/1115277624 learned_addr learned my addr 172.21.15.16:0/1115277624 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:19:54.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:54.087+0000 7f74e77fe700 1 --2- 172.21.15.16:0/1115277624 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f74e80cfcb0 0x7f74e80d0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:54.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:54.087+0000 7f74e7fff700 1 -- 172.21.15.16:0/1115277624 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f74e80cfcb0 msgr2=0x7f74e80d0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:19:54.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:54.087+0000 7f74e7fff700 1 --2- 172.21.15.16:0/1115277624 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f74e80cfcb0 0x7f74e80d0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:54.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:54.087+0000 7f74e7fff700 1 -- 172.21.15.16:0/1115277624 >> v1:172.21.15.16:6789/0 conn(0x7f74e80cd920 legacy=0x7f74e80b6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:54.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:54.087+0000 7f74e7fff700 1 -- 172.21.15.16:0/1115277624 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74e80d12e0 con 0x7f74e815cd50 2023-05-20T01:19:54.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:54.087+0000 7f74e6ffd700 1 -- 172.21.15.16:0/1115277624 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3853906805 0 0) 0x7f74e80d1c10 con 0x7f74e80cd920 2023-05-20T01:19:54.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:54.087+0000 7f74e77fe700 1 --2- 172.21.15.16:0/1115277624 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f74e80cfcb0 0x7f74e80d0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:19:54.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:54.087+0000 7f74e7fff700 1 --2- 172.21.15.16:0/1115277624 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74e815cd50 0x7f74e80d20a0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f74d8009820 tx=0x7f74d8009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=850d56e1c952d5b4 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:54.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:54.087+0000 7f74e6ffd700 1 -- 172.21.15.16:0/1115277624 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74d8005e30 con 0x7f74e815cd50 2023-05-20T01:19:54.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:54.087+0000 7f74e6ffd700 1 -- 172.21.15.16:0/1115277624 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74d8006030 con 0x7f74e815cd50 2023-05-20T01:19:54.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:54.087+0000 7f74e6ffd700 1 -- 172.21.15.16:0/1115277624 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74d80063e0 con 0x7f74e815cd50 2023-05-20T01:19:54.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:54.088+0000 7f74ee843700 1 -- 172.21.15.16:0/1115277624 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74e815cd50 msgr2=0x7f74e80d20a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:54.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:54.088+0000 7f74ee843700 1 --2- 172.21.15.16:0/1115277624 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74e815cd50 0x7f74e80d20a0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f74d8009820 tx=0x7f74d8009b30 comp rx=0 tx=0).stop 2023-05-20T01:19:54.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:54.088+0000 7f74ee843700 1 -- 172.21.15.16:0/1115277624 shutdown_connections 2023-05-20T01:19:54.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:54.088+0000 7f74ee843700 1 --2- 172.21.15.16:0/1115277624 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74e815cd50 0x7f74e80d20a0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:54.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:54.088+0000 7f74ee843700 1 --2- 172.21.15.16:0/1115277624 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f74e80cfcb0 0x7f74e80d0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:54.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:54.088+0000 7f74ee843700 1 -- 172.21.15.16:0/1115277624 >> 172.21.15.16:0/1115277624 conn(0x7f74e806b070 msgr2=0x7f74e806b470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:19:54.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:54.088+0000 7f74ee843700 1 -- 172.21.15.16:0/1115277624 shutdown_connections 2023-05-20T01:19:54.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:54.088+0000 7f74ee843700 1 -- 172.21.15.16:0/1115277624 wait complete. 2023-05-20T01:19:54.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:54.089+0000 7f74ee843700 1 Processor -- start 2023-05-20T01:19:54.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:54.089+0000 7f74ee843700 1 -- start start 2023-05-20T01:19:54.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:54.090+0000 7f74ee843700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f74e80cd920 0x7f74e80b6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:54.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:54.090+0000 7f74ee843700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f74e80cfcb0 0x7f74e81f1e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:54.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:54.090+0000 7f74ee843700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74e815cd50 0x7f74e80c8410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:54.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:54.090+0000 7f74ee843700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74e80b7500 con 0x7f74e815cd50 2023-05-20T01:19:54.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:54.090+0000 7f74ee843700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74e80b71d0 con 0x7f74e80cfcb0 2023-05-20T01:19:54.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:54.090+0000 7f74ee843700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74e80b6930 con 0x7f74e80cd920 2023-05-20T01:19:54.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:54.090+0000 7f74ecde0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74e815cd50 0x7f74e80c8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:54.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:54.090+0000 7f74ecde0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74e815cd50 0x7f74e80c8410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33830/0 (socket says 172.21.15.16:33830) 2023-05-20T01:19:54.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:54.090+0000 7f74ecde0700 1 -- 172.21.15.16:0/4049722923 learned_addr learned my addr 172.21.15.16:0/4049722923 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:19:54.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:54.090+0000 7f74e7fff700 1 --2- 172.21.15.16:0/4049722923 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f74e80cd920 0x7f74e80b6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:54.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:54.090+0000 7f74e77fe700 1 --2- 172.21.15.16:0/4049722923 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f74e80cfcb0 0x7f74e81f1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:54.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:54.090+0000 7f74ecde0700 1 -- 172.21.15.16:0/4049722923 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f74e80cd920 msgr2=0x7f74e80b6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:54.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:54.090+0000 7f74ecde0700 1 --2- 172.21.15.16:0/4049722923 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f74e80cd920 0x7f74e80b6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:54.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:54.090+0000 7f74ecde0700 1 -- 172.21.15.16:0/4049722923 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f74e80cfcb0 msgr2=0x7f74e81f1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:54.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:54.090+0000 7f74ecde0700 1 --2- 172.21.15.16:0/4049722923 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f74e80cfcb0 0x7f74e81f1e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:54.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:54.090+0000 7f74ecde0700 1 -- 172.21.15.16:0/4049722923 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74d8009430 con 0x7f74e815cd50 2023-05-20T01:19:54.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:54.091+0000 7f74ecde0700 1 --2- 172.21.15.16:0/4049722923 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74e815cd50 0x7f74e80c8410 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f74e0002230 tx=0x7f74e000cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:54.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:54.091+0000 7f74e4ff9700 1 -- 172.21.15.16:0/4049722923 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e000aac0 con 0x7f74e815cd50 2023-05-20T01:19:54.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:54.091+0000 7f74e4ff9700 1 -- 172.21.15.16:0/4049722923 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74e000ac40 con 0x7f74e815cd50 2023-05-20T01:19:54.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:54.091+0000 7f74ee843700 1 -- 172.21.15.16:0/4049722923 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74e80d18a0 con 0x7f74e815cd50 2023-05-20T01:19:54.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:54.091+0000 7f74e4ff9700 1 -- 172.21.15.16:0/4049722923 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e00070d0 con 0x7f74e815cd50 2023-05-20T01:19:54.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:54.091+0000 7f74ee843700 1 -- 172.21.15.16:0/4049722923 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74e80d15c0 con 0x7f74e815cd50 2023-05-20T01:19:54.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:54.092+0000 7f74ee843700 1 -- 172.21.15.16:0/4049722923 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74e8161680 con 0x7f74e815cd50 2023-05-20T01:19:54.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:54.092+0000 7f74e4ff9700 1 -- 172.21.15.16:0/4049722923 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f74e0007250 con 0x7f74e815cd50 2023-05-20T01:19:54.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:54.093+0000 7f74e4ff9700 1 --2- 172.21.15.16:0/4049722923 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f74c806ace0 0x7f74c806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:54.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:54.093+0000 7f74e4ff9700 1 -- 172.21.15.16:0/4049722923 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f74e0088f90 con 0x7f74e815cd50 2023-05-20T01:19:54.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:54.093+0000 7f74e7fff700 1 --2- 172.21.15.16:0/4049722923 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f74c806ace0 0x7f74c806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:54.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:54.093+0000 7f74e7fff700 1 --2- 172.21.15.16:0/4049722923 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f74c806ace0 0x7f74c806d190 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f74d8013040 tx=0x7f74d8008070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:54.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:54.097+0000 7f74e4ff9700 1 -- 172.21.15.16:0/4049722923 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f74e00bb2d0 con 0x7f74e815cd50 2023-05-20T01:19:54.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:54.342+0000 7f74ee843700 1 -- 172.21.15.16:0/4049722923 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f74e80cd6e0 con 0x7f74e815cd50 2023-05-20T01:19:55.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.345+0000 7f74e4ff9700 1 -- 172.21.15.16:0/4049722923 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v37) v1 ==== 122+0+0 (secure 0 0 0) 0x7f74e00180c0 con 0x7f74e815cd50 2023-05-20T01:19:55.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.348+0000 7f74ee843700 1 -- 172.21.15.16:0/4049722923 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f74c806ace0 msgr2=0x7f74c806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:55.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.348+0000 7f74ee843700 1 --2- 172.21.15.16:0/4049722923 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f74c806ace0 0x7f74c806d190 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f74d8013040 tx=0x7f74d8008070 comp rx=0 tx=0).stop 2023-05-20T01:19:55.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.348+0000 7f74ee843700 1 -- 172.21.15.16:0/4049722923 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74e815cd50 msgr2=0x7f74e80c8410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:55.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.348+0000 7f74ee843700 1 --2- 172.21.15.16:0/4049722923 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74e815cd50 0x7f74e80c8410 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f74e0002230 tx=0x7f74e000cf50 comp rx=0 tx=0).stop 2023-05-20T01:19:55.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.348+0000 7f74ee843700 1 -- 172.21.15.16:0/4049722923 shutdown_connections 2023-05-20T01:19:55.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.348+0000 7f74ee843700 1 --2- 172.21.15.16:0/4049722923 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74e815cd50 0x7f74e80c8410 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:55.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.348+0000 7f74ee843700 1 --2- 172.21.15.16:0/4049722923 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f74c806ace0 0x7f74c806d190 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:55.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.348+0000 7f74ee843700 1 --2- 172.21.15.16:0/4049722923 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f74e80cd920 0x7f74e80b6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:55.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.348+0000 7f74ee843700 1 --2- 172.21.15.16:0/4049722923 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f74e80cfcb0 0x7f74e81f1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:55.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.348+0000 7f74ee843700 1 -- 172.21.15.16:0/4049722923 >> 172.21.15.16:0/4049722923 conn(0x7f74e806b070 msgr2=0x7f74e8157260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:19:55.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.348+0000 7f74ee843700 1 -- 172.21.15.16:0/4049722923 shutdown_connections 2023-05-20T01:19:55.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.348+0000 7f74ee843700 1 -- 172.21.15.16:0/4049722923 wait complete. 2023-05-20T01:19:55.366 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-05-20T01:19:55.367 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-20T01:19:55.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.495+0000 7f85c510c700 1 Processor -- start 2023-05-20T01:19:55.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.495+0000 7f85c510c700 1 -- start start 2023-05-20T01:19:55.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.496+0000 7f85c510c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f85c00cfc50 0x7f85c00d2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:55.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.496+0000 7f85c510c700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f85c00b4750 0x7f85c00d0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:55.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.496+0000 7f85c510c700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85c00d0b90 con 0x7f85c00b2b40 2023-05-20T01:19:55.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.496+0000 7f85c510c700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85c00cfad0 con 0x7f85c00cfc50 2023-05-20T01:19:55.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.496+0000 7f85c510c700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85c00d3df0 con 0x7f85c00b4750 2023-05-20T01:19:55.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.496+0000 7f85bed9d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f85c00cfc50 0x7f85c00d2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:55.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.496+0000 7f85bf59e700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f85c00b2b40 0x7f85c00b2f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:45366/0 (socket says 172.21.15.16:45366) 2023-05-20T01:19:55.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.496+0000 7f85bf59e700 1 -- 172.21.15.16:0/2410283192 learned_addr learned my addr 172.21.15.16:0/2410283192 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:19:55.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.496+0000 7f85be59c700 1 --2- 172.21.15.16:0/2410283192 >> v2:172.21.15.132:3300/0 conn(0x7f85c00b4750 0x7f85c00d0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:55.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.496+0000 7f85bed9d700 1 -- 172.21.15.16:0/2410283192 >> v2:172.21.15.132:3300/0 conn(0x7f85c00b4750 msgr2=0x7f85c00d0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:19:55.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.496+0000 7f85bed9d700 1 --2- 172.21.15.16:0/2410283192 >> v2:172.21.15.132:3300/0 conn(0x7f85c00b4750 0x7f85c00d0020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:55.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.496+0000 7f85bed9d700 1 -- 172.21.15.16:0/2410283192 >> v1:172.21.15.16:6789/0 conn(0x7f85c00b2b40 legacy=0x7f85c00b2f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:55.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.497+0000 7f85bed9d700 1 -- 172.21.15.16:0/2410283192 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85c00d0560 con 0x7f85c00cfc50 2023-05-20T01:19:55.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.497+0000 7f85be59c700 1 --2- 172.21.15.16:0/2410283192 >> v2:172.21.15.132:3300/0 conn(0x7f85c00b4750 0x7f85c00d0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:19:55.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.497+0000 7f85bed9d700 1 --2- 172.21.15.16:0/2410283192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85c00cfc50 0x7f85c00d2c10 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f85c008b9b0 tx=0x7f85b000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18470cc3e0125736 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:55.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.497+0000 7f85bdd9b700 1 -- 172.21.15.16:0/2410283192 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b000e070 con 0x7f85c00cfc50 2023-05-20T01:19:55.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.497+0000 7f85bdd9b700 1 -- 172.21.15.16:0/2410283192 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85b000bbd0 con 0x7f85c00cfc50 2023-05-20T01:19:55.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.497+0000 7f85bdd9b700 1 -- 172.21.15.16:0/2410283192 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b0012430 con 0x7f85c00cfc50 2023-05-20T01:19:55.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.497+0000 7f85c510c700 1 -- 172.21.15.16:0/2410283192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85c00cfc50 msgr2=0x7f85c00d2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:55.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.497+0000 7f85c510c700 1 --2- 172.21.15.16:0/2410283192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85c00cfc50 0x7f85c00d2c10 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f85c008b9b0 tx=0x7f85b000b330 comp rx=0 tx=0).stop 2023-05-20T01:19:55.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.497+0000 7f85c510c700 1 -- 172.21.15.16:0/2410283192 shutdown_connections 2023-05-20T01:19:55.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.497+0000 7f85c510c700 1 --2- 172.21.15.16:0/2410283192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85c00cfc50 0x7f85c00d2c10 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:55.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.497+0000 7f85c510c700 1 --2- 172.21.15.16:0/2410283192 >> v2:172.21.15.132:3300/0 conn(0x7f85c00b4750 0x7f85c00d0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:55.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.497+0000 7f85c510c700 1 -- 172.21.15.16:0/2410283192 >> 172.21.15.16:0/2410283192 conn(0x7f85c009c590 msgr2=0x7f85c00d3240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:19:55.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.497+0000 7f85c510c700 1 -- 172.21.15.16:0/2410283192 shutdown_connections 2023-05-20T01:19:55.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.497+0000 7f85c510c700 1 -- 172.21.15.16:0/2410283192 wait complete. 2023-05-20T01:19:55.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.499+0000 7f85c510c700 1 Processor -- start 2023-05-20T01:19:55.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.499+0000 7f85c510c700 1 -- start start 2023-05-20T01:19:55.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.500+0000 7f85c510c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85c00b2b40 0x7f85c00cb360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:55.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.500+0000 7f85c510c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f85c00b4750 0x7f85c00cc9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:55.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.500+0000 7f85c510c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f85c00cfc50 0x7f85c00c6d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:55.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.500+0000 7f85c510c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85c00c7290 con 0x7f85c00b2b40 2023-05-20T01:19:55.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.500+0000 7f85c510c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85c00cc550 con 0x7f85c00cfc50 2023-05-20T01:19:55.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.500+0000 7f85c510c700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85c00cb940 con 0x7f85c00b4750 2023-05-20T01:19:55.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.500+0000 7f85bed9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85c00b2b40 0x7f85c00cb360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:55.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.500+0000 7f85bed9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85c00b2b40 0x7f85c00cb360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33840/0 (socket says 172.21.15.16:33840) 2023-05-20T01:19:55.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.500+0000 7f85bed9d700 1 -- 172.21.15.16:0/3698794563 learned_addr learned my addr 172.21.15.16:0/3698794563 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:19:55.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.500+0000 7f85bf59e700 1 --2- 172.21.15.16:0/3698794563 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f85c00cfc50 0x7f85c00c6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:55.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.500+0000 7f85bed9d700 1 -- 172.21.15.16:0/3698794563 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f85c00b4750 msgr2=0x7f85c00cc9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:55.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.500+0000 7f85bed9d700 1 --2- 172.21.15.16:0/3698794563 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f85c00b4750 0x7f85c00cc9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:55.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.500+0000 7f85bed9d700 1 -- 172.21.15.16:0/3698794563 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f85c00cfc50 msgr2=0x7f85c00c6d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:55.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.500+0000 7f85bed9d700 1 --2- 172.21.15.16:0/3698794563 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f85c00cfc50 0x7f85c00c6d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:55.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.500+0000 7f85bed9d700 1 -- 172.21.15.16:0/3698794563 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85b0009a70 con 0x7f85c00b2b40 2023-05-20T01:19:55.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.500+0000 7f85bed9d700 1 --2- 172.21.15.16:0/3698794563 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85c00b2b40 0x7f85c00cb360 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f85b0014040 tx=0x7f85b0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:55.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.500+0000 7f85b77fe700 1 -- 172.21.15.16:0/3698794563 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b00098f0 con 0x7f85c00b2b40 2023-05-20T01:19:55.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.501+0000 7f85b77fe700 1 -- 172.21.15.16:0/3698794563 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85b0020070 con 0x7f85c00b2b40 2023-05-20T01:19:55.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.501+0000 7f85c510c700 1 -- 172.21.15.16:0/3698794563 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85c00b4380 con 0x7f85c00b2b40 2023-05-20T01:19:55.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.501+0000 7f85b77fe700 1 -- 172.21.15.16:0/3698794563 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b0004260 con 0x7f85c00b2b40 2023-05-20T01:19:55.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.501+0000 7f85c510c700 1 -- 172.21.15.16:0/3698794563 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85c015d0b0 con 0x7f85c00b2b40 2023-05-20T01:19:55.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.501+0000 7f85c510c700 1 -- 172.21.15.16:0/3698794563 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85c00d4000 con 0x7f85c00b2b40 2023-05-20T01:19:55.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.502+0000 7f85b77fe700 1 -- 172.21.15.16:0/3698794563 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f85b00043c0 con 0x7f85c00b2b40 2023-05-20T01:19:55.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.502+0000 7f85b77fe700 1 --2- 172.21.15.16:0/3698794563 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f85a006ace0 0x7f85a006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:55.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.502+0000 7f85b77fe700 1 -- 172.21.15.16:0/3698794563 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f85b0022070 con 0x7f85c00b2b40 2023-05-20T01:19:55.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.503+0000 7f85be59c700 1 --2- 172.21.15.16:0/3698794563 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f85a006ace0 0x7f85a006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:55.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.503+0000 7f85be59c700 1 --2- 172.21.15.16:0/3698794563 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f85a006ace0 0x7f85a006d190 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f85c014bd30 tx=0x7f85a8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:55.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.507+0000 7f85b77fe700 1 -- 172.21.15.16:0/3698794563 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f85b0056c20 con 0x7f85c00b2b40 2023-05-20T01:19:55.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.750+0000 7f85c510c700 1 -- 172.21.15.16:0/3698794563 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f85c004e8c0 con 0x7f85c00b2b40 2023-05-20T01:19:55.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.751+0000 7f85b77fe700 1 -- 172.21.15.16:0/3698794563 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+4672 (secure 0 0 0) 0x7f85b005a240 con 0x7f85c00b2b40 2023-05-20T01:19:55.755 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:19:55.755 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":37,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5824,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/2609276800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":2609276800},{"type":"v1","addr":"172.21.15.132:6834","nonce":2609276800}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32},{"gid":5827,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/29554453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":29554453},{"type":"v1","addr":"172.21.15.132:6835","nonce":29554453}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32},{"gid":6192,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/4090303296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4090303296},{"type":"v1","addr":"172.21.15.16:6836","nonce":4090303296}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-20T01:19:51.915360+0000","modified":"2023-05-20T01:19:55.342773+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6195},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6195":{"gid":6195,"name":"c","rank":0,"incarnation":33,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6837/1427908556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":1427908556},{"type":"v1","addr":"172.21.15.16:6837","nonce":1427908556}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-05-20T01:19:55.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.753+0000 7f85c510c700 1 -- 172.21.15.16:0/3698794563 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f85a006ace0 msgr2=0x7f85a006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:55.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.753+0000 7f85c510c700 1 --2- 172.21.15.16:0/3698794563 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f85a006ace0 0x7f85a006d190 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f85c014bd30 tx=0x7f85a8008040 comp rx=0 tx=0).stop 2023-05-20T01:19:55.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.753+0000 7f85c510c700 1 -- 172.21.15.16:0/3698794563 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85c00b2b40 msgr2=0x7f85c00cb360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:55.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.753+0000 7f85c510c700 1 --2- 172.21.15.16:0/3698794563 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85c00b2b40 0x7f85c00cb360 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f85b0014040 tx=0x7f85b0004ad0 comp rx=0 tx=0).stop 2023-05-20T01:19:55.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.753+0000 7f85c510c700 1 -- 172.21.15.16:0/3698794563 shutdown_connections 2023-05-20T01:19:55.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.753+0000 7f85c510c700 1 --2- 172.21.15.16:0/3698794563 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f85c00cfc50 0x7f85c00c6d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:55.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.753+0000 7f85c510c700 1 --2- 172.21.15.16:0/3698794563 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85c00b2b40 0x7f85c00cb360 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:55.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.753+0000 7f85c510c700 1 --2- 172.21.15.16:0/3698794563 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f85a006ace0 0x7f85a006d190 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:55.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.753+0000 7f85c510c700 1 --2- 172.21.15.16:0/3698794563 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f85c00b4750 0x7f85c00cc9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:55.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.753+0000 7f85c510c700 1 -- 172.21.15.16:0/3698794563 >> 172.21.15.16:0/3698794563 conn(0x7f85c009c590 msgr2=0x7f85c00ce5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:19:55.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.753+0000 7f85c510c700 1 -- 172.21.15.16:0/3698794563 shutdown_connections 2023-05-20T01:19:55.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.753+0000 7f85c510c700 1 -- 172.21.15.16:0/3698794563 wait complete. 2023-05-20T01:19:55.760 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 37 2023-05-20T01:19:55.769 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-20T01:19:55.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.889+0000 7f912da48700 1 Processor -- start 2023-05-20T01:19:55.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.890+0000 7f912da48700 1 -- start start 2023-05-20T01:19:55.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.890+0000 7f912da48700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f912815c8f0 0x7f91280d0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:55.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.890+0000 7f912da48700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f912806fd00 0x7f91280cda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:55.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.890+0000 7f912da48700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9128160870 con 0x7f91280cdf60 2023-05-20T01:19:55.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.890+0000 7f912da48700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91280ce3e0 con 0x7f912815c8f0 2023-05-20T01:19:55.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.890+0000 7f912da48700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9128160b50 con 0x7f912806fd00 2023-05-20T01:19:55.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.891+0000 7f9126ffd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f912815c8f0 0x7f91280d0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:55.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.891+0000 7f9126ffd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f912815c8f0 0x7f91280d0a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33848/0 (socket says 172.21.15.16:33848) 2023-05-20T01:19:55.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.891+0000 7f9126ffd700 1 -- 172.21.15.16:0/3671883358 learned_addr learned my addr 172.21.15.16:0/3671883358 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:19:55.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.891+0000 7f91267fc700 1 --2- 172.21.15.16:0/3671883358 >> v2:172.21.15.132:3300/0 conn(0x7f912806fd00 0x7f91280cda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:55.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.891+0000 7f9126ffd700 1 -- 172.21.15.16:0/3671883358 >> v2:172.21.15.132:3300/0 conn(0x7f912806fd00 msgr2=0x7f91280cda20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:19:55.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.891+0000 7f9126ffd700 1 --2- 172.21.15.16:0/3671883358 >> v2:172.21.15.132:3300/0 conn(0x7f912806fd00 0x7f91280cda20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:55.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.891+0000 7f9126ffd700 1 -- 172.21.15.16:0/3671883358 >> v1:172.21.15.132:6789/0 conn(0x7f91280cdf60 legacy=0x7f91280b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:55.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.891+0000 7f9126ffd700 1 -- 172.21.15.16:0/3671883358 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9128160e30 con 0x7f912815c8f0 2023-05-20T01:19:55.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.891+0000 7f9126ffd700 1 --2- 172.21.15.16:0/3671883358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f912815c8f0 0x7f91280d0a60 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f91280ceb70 tx=0x7f911c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e2c50d2e62dc3c8 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:55.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.891+0000 7f9125ffb700 1 -- 172.21.15.16:0/3671883358 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f911c00b870 con 0x7f912815c8f0 2023-05-20T01:19:55.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.891+0000 7f9125ffb700 1 -- 172.21.15.16:0/3671883358 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f911c00b9d0 con 0x7f912815c8f0 2023-05-20T01:19:55.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.891+0000 7f9125ffb700 1 -- 172.21.15.16:0/3671883358 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f911c00bd60 con 0x7f912815c8f0 2023-05-20T01:19:55.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.892+0000 7f912da48700 1 -- 172.21.15.16:0/3671883358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f912815c8f0 msgr2=0x7f91280d0a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:55.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.892+0000 7f912da48700 1 --2- 172.21.15.16:0/3671883358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f912815c8f0 0x7f91280d0a60 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f91280ceb70 tx=0x7f911c009bd0 comp rx=0 tx=0).stop 2023-05-20T01:19:55.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.892+0000 7f912da48700 1 -- 172.21.15.16:0/3671883358 shutdown_connections 2023-05-20T01:19:55.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.892+0000 7f912da48700 1 --2- 172.21.15.16:0/3671883358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f912815c8f0 0x7f91280d0a60 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:55.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.892+0000 7f912da48700 1 --2- 172.21.15.16:0/3671883358 >> v2:172.21.15.132:3300/0 conn(0x7f912806fd00 0x7f91280cda20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:55.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.892+0000 7f912da48700 1 -- 172.21.15.16:0/3671883358 >> 172.21.15.16:0/3671883358 conn(0x7f912806e280 msgr2=0x7f912806e680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:19:55.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.892+0000 7f912da48700 1 -- 172.21.15.16:0/3671883358 shutdown_connections 2023-05-20T01:19:55.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.892+0000 7f912da48700 1 -- 172.21.15.16:0/3671883358 wait complete. 2023-05-20T01:19:55.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.893+0000 7f912da48700 1 Processor -- start 2023-05-20T01:19:55.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.893+0000 7f912da48700 1 -- start start 2023-05-20T01:19:55.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.894+0000 7f912da48700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f912806fd00 0x7f912815d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:55.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.894+0000 7f912da48700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f91280cdf60 0x7f91281e8330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:55.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.894+0000 7f912da48700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f912815c8f0 0x7f91281e8870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:55.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.894+0000 7f912da48700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9128159520 con 0x7f912815c8f0 2023-05-20T01:19:55.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.894+0000 7f912da48700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f912815dbe0 con 0x7f912806fd00 2023-05-20T01:19:55.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.894+0000 7f912da48700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f912815dec0 con 0x7f91280cdf60 2023-05-20T01:19:55.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.894+0000 7f91277fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f912815c8f0 0x7f91281e8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:55.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.894+0000 7f91277fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f912815c8f0 0x7f91281e8870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33858/0 (socket says 172.21.15.16:33858) 2023-05-20T01:19:55.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.894+0000 7f91277fe700 1 -- 172.21.15.16:0/4039286911 learned_addr learned my addr 172.21.15.16:0/4039286911 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:19:55.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.894+0000 7f9126ffd700 1 --2- 172.21.15.16:0/4039286911 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f912806fd00 0x7f912815d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:55.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.894+0000 7f91267fc700 1 --2- 172.21.15.16:0/4039286911 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f91280cdf60 0x7f91281e8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:55.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.894+0000 7f91277fe700 1 -- 172.21.15.16:0/4039286911 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f91280cdf60 msgr2=0x7f91281e8330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:55.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.894+0000 7f91277fe700 1 --2- 172.21.15.16:0/4039286911 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f91280cdf60 0x7f91281e8330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:55.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.894+0000 7f91277fe700 1 -- 172.21.15.16:0/4039286911 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f912806fd00 msgr2=0x7f912815d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:55.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.894+0000 7f91277fe700 1 --2- 172.21.15.16:0/4039286911 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f912806fd00 0x7f912815d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:55.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.894+0000 7f91277fe700 1 -- 172.21.15.16:0/4039286911 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f911c009550 con 0x7f912815c8f0 2023-05-20T01:19:55.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.895+0000 7f91277fe700 1 --2- 172.21.15.16:0/4039286911 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f912815c8f0 0x7f91281e8870 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f912000cc50 tx=0x7f912000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:55.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.895+0000 7f910ffff700 1 -- 172.21.15.16:0/4039286911 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9120003c00 con 0x7f912815c8f0 2023-05-20T01:19:55.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.895+0000 7f910ffff700 1 -- 172.21.15.16:0/4039286911 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9120003d60 con 0x7f912815c8f0 2023-05-20T01:19:55.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.895+0000 7f912da48700 1 -- 172.21.15.16:0/4039286911 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91281613f0 con 0x7f912815c8f0 2023-05-20T01:19:55.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.895+0000 7f910ffff700 1 -- 172.21.15.16:0/4039286911 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f912000ab60 con 0x7f912815c8f0 2023-05-20T01:19:55.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.895+0000 7f912da48700 1 -- 172.21.15.16:0/4039286911 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9128161110 con 0x7f912815c8f0 2023-05-20T01:19:55.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.896+0000 7f912da48700 1 -- 172.21.15.16:0/4039286911 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9128160540 con 0x7f912815c8f0 2023-05-20T01:19:55.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.896+0000 7f910ffff700 1 -- 172.21.15.16:0/4039286911 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f912000acc0 con 0x7f912815c8f0 2023-05-20T01:19:55.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.897+0000 7f910ffff700 1 --2- 172.21.15.16:0/4039286911 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f911406ace0 0x7f911406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:55.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.897+0000 7f910ffff700 1 -- 172.21.15.16:0/4039286911 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f91200895f0 con 0x7f912815c8f0 2023-05-20T01:19:55.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.897+0000 7f9126ffd700 1 --2- 172.21.15.16:0/4039286911 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f911406ace0 0x7f911406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:55.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.897+0000 7f9126ffd700 1 --2- 172.21.15.16:0/4039286911 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f911406ace0 0x7f911406d190 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f911c014040 tx=0x7f911c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:55.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:55.901+0000 7f910ffff700 1 -- 172.21.15.16:0/4039286911 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f91200545f0 con 0x7f912815c8f0 2023-05-20T01:19:56.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.099+0000 7f912da48700 1 -- 172.21.15.16:0/4039286911 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f912804e8c0 con 0x7f912815c8f0 2023-05-20T01:19:56.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.100+0000 7f910ffff700 1 -- 172.21.15.16:0/4039286911 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v64) v1 ==== 74+0+18404 (secure 0 0 0) 0x7f9120057c10 con 0x7f912815c8f0 2023-05-20T01:19:56.104 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:19:56.105 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":64,"fsid":"c2245cc5-ecab-4087-a59b-bc81db901676","created":"2023-05-20T01:16:09.549230+0000","modified":"2023-05-20T01:19:52.917201+0000","last_up_change":"2023-05-20T01:16:19.788624+0000","last_in_change":"2023-05-20T01:16:12.092160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-05-20T01:16:20.934366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-05-20T01:16:21.372294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-05-20T01:19:49.878849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-05-20T01:19:50.887306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"317a83cd-19c8-4958-a301-6087ef342289","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6825","nonce":3923634179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6827","nonce":3923634179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6831","nonce":3923634179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6829","nonce":3923634179}]},"public_addr":"172.21.15.16:6825/3923634179","cluster_addr":"172.21.15.16:6827/3923634179","heartbeat_back_addr":"172.21.15.16:6831/3923634179","heartbeat_front_addr":"172.21.15.16:6829/3923634179","state":["exists","up"]},{"osd":1,"uuid":"5581046f-8219-4089-8d7d-b7846399505e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6809","nonce":3860264220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6811","nonce":3860264220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6815","nonce":3860264220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6813","nonce":3860264220}]},"public_addr":"172.21.15.16:6809/3860264220","cluster_addr":"172.21.15.16:6811/3860264220","heartbeat_back_addr":"172.21.15.16:6815/3860264220","heartbeat_front_addr":"172.21.15.16:6813/3860264220","state":["exists","up"]},{"osd":2,"uuid":"072e9e66-4133-478e-8d4a-64d46c2963f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6817","nonce":293187030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6819","nonce":293187030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6823","nonce":293187030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6821","nonce":293187030}]},"public_addr":"172.21.15.16:6817/293187030","cluster_addr":"172.21.15.16:6819/293187030","heartbeat_back_addr":"172.21.15.16:6823/293187030","heartbeat_front_addr":"172.21.15.16:6821/293187030","state":["exists","up"]},{"osd":3,"uuid":"11e1bba6-00f8-48dd-9028-d8d716b7f145","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6801","nonce":3669781890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6803","nonce":3669781890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6807","nonce":3669781890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6805","nonce":3669781890}]},"public_addr":"172.21.15.16:6801/3669781890","cluster_addr":"172.21.15.16:6803/3669781890","heartbeat_back_addr":"172.21.15.16:6807/3669781890","heartbeat_front_addr":"172.21.15.16:6805/3669781890","state":["exists","up"]},{"osd":4,"uuid":"b8909c3a-cf70-46cb-8d0b-009c707cf9d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6819","nonce":506317223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6823","nonce":506317223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6829","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6831","nonce":506317223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6827","nonce":506317223}]},"public_addr":"172.21.15.132:6819/506317223","cluster_addr":"172.21.15.132:6823/506317223","heartbeat_back_addr":"172.21.15.132:6831/506317223","heartbeat_front_addr":"172.21.15.132:6827/506317223","state":["exists","up"]},{"osd":5,"uuid":"b1818304-4583-47bf-a6ec-0bfbb5166233","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6818","nonce":1844689995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6822","nonce":1844689995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6830","nonce":1844689995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6826","nonce":1844689995}]},"public_addr":"172.21.15.132:6818/1844689995","cluster_addr":"172.21.15.132:6822/1844689995","heartbeat_back_addr":"172.21.15.132:6830/1844689995","heartbeat_front_addr":"172.21.15.132:6826/1844689995","state":["exists","up"]},{"osd":6,"uuid":"d37f4691-eed1-4a93-8818-18354f03f4a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6801","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6802","nonce":727581123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6805","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6807","nonce":727581123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6813","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6815","nonce":727581123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6811","nonce":727581123}]},"public_addr":"172.21.15.132:6802/727581123","cluster_addr":"172.21.15.132:6807/727581123","heartbeat_back_addr":"172.21.15.132:6815/727581123","heartbeat_front_addr":"172.21.15.132:6811/727581123","state":["exists","up"]},{"osd":7,"uuid":"4f712f62-5d1d-4816-8042-03209144f8b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6803","nonce":466001149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6806","nonce":466001149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6814","nonce":466001149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6810","nonce":466001149}]},"public_addr":"172.21.15.132:6803/466001149","cluster_addr":"172.21.15.132:6806/466001149","heartbeat_back_addr":"172.21.15.132:6814/466001149","heartbeat_front_addr":"172.21.15.132:6810/466001149","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.338863+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.278680+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305694+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.246835+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.276237+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.304441+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305421+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.267050+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-20T01:19:56.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.102+0000 7f912da48700 1 -- 172.21.15.16:0/4039286911 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f911406ace0 msgr2=0x7f911406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:56.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.102+0000 7f912da48700 1 --2- 172.21.15.16:0/4039286911 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f911406ace0 0x7f911406d190 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f911c014040 tx=0x7f911c008030 comp rx=0 tx=0).stop 2023-05-20T01:19:56.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.102+0000 7f912da48700 1 -- 172.21.15.16:0/4039286911 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f912815c8f0 msgr2=0x7f91281e8870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:56.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.102+0000 7f912da48700 1 --2- 172.21.15.16:0/4039286911 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f912815c8f0 0x7f91281e8870 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f912000cc50 tx=0x7f912000cf60 comp rx=0 tx=0).stop 2023-05-20T01:19:56.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.102+0000 7f912da48700 1 -- 172.21.15.16:0/4039286911 shutdown_connections 2023-05-20T01:19:56.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.102+0000 7f912da48700 1 --2- 172.21.15.16:0/4039286911 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f912815c8f0 0x7f91281e8870 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:56.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.102+0000 7f912da48700 1 --2- 172.21.15.16:0/4039286911 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f912806fd00 0x7f912815d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:56.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.102+0000 7f912da48700 1 --2- 172.21.15.16:0/4039286911 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f911406ace0 0x7f911406d190 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:56.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.102+0000 7f912da48700 1 --2- 172.21.15.16:0/4039286911 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f91280cdf60 0x7f91281e8330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:56.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.102+0000 7f912da48700 1 -- 172.21.15.16:0/4039286911 >> 172.21.15.16:0/4039286911 conn(0x7f912806e280 msgr2=0x7f91280c8cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:19:56.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.102+0000 7f912da48700 1 -- 172.21.15.16:0/4039286911 shutdown_connections 2023-05-20T01:19:56.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.102+0000 7f912da48700 1 -- 172.21.15.16:0/4039286911 wait complete. 2023-05-20T01:19:56.119 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-05-20T01:19:56.119 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-05-20T01:19:56.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.240+0000 7ff76d568700 1 Processor -- start 2023-05-20T01:19:56.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.240+0000 7ff76d568700 1 -- start start 2023-05-20T01:19:56.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.241+0000 7ff76d568700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff7680b2760 0x7ff7680d1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:56.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.241+0000 7ff76d568700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff7680cec60 0x7ff7680cf0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:56.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.241+0000 7ff76d568700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7680cc500 con 0x7ff768144020 2023-05-20T01:19:56.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.241+0000 7ff76d568700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7680cb6e0 con 0x7ff7680b2760 2023-05-20T01:19:56.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.241+0000 7ff76d568700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7680cf620 con 0x7ff7680cec60 2023-05-20T01:19:56.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.241+0000 7ff766ffd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff7680b2760 0x7ff7680d1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:56.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.241+0000 7ff7667fc700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ff768144020 0x7ff7680b0980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:45392/0 (socket says 172.21.15.16:45392) 2023-05-20T01:19:56.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.241+0000 7ff766ffd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff7680b2760 0x7ff7680d1680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33862/0 (socket says 172.21.15.16:33862) 2023-05-20T01:19:56.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.241+0000 7ff7667fc700 1 -- 172.21.15.16:0/2823649577 learned_addr learned my addr 172.21.15.16:0/2823649577 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:19:56.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.242+0000 7ff7677fe700 1 --2- 172.21.15.16:0/2823649577 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff7680cec60 0x7ff7680cf0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:56.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.242+0000 7ff766ffd700 1 -- 172.21.15.16:0/2823649577 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff7680cec60 msgr2=0x7ff7680cf0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:19:56.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.242+0000 7ff766ffd700 1 --2- 172.21.15.16:0/2823649577 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff7680cec60 0x7ff7680cf0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:56.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.242+0000 7ff766ffd700 1 -- 172.21.15.16:0/2823649577 >> v1:172.21.15.16:6789/0 conn(0x7ff768144020 legacy=0x7ff7680b0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:56.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.242+0000 7ff766ffd700 1 -- 172.21.15.16:0/2823649577 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7680cb8f0 con 0x7ff7680b2760 2023-05-20T01:19:56.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.242+0000 7ff765ffb700 1 -- 172.21.15.16:0/2823649577 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 671399665 0 0) 0x7ff7680cc500 con 0x7ff768144020 2023-05-20T01:19:56.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.242+0000 7ff766ffd700 1 --2- 172.21.15.16:0/2823649577 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7680b2760 0x7ff7680d1680 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7ff75c009840 tx=0x7ff75c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=551e50751df07eba server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:56.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.242+0000 7ff765ffb700 1 -- 172.21.15.16:0/2823649577 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff75c011830 con 0x7ff7680b2760 2023-05-20T01:19:56.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.242+0000 7ff765ffb700 1 -- 172.21.15.16:0/2823649577 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff75c011990 con 0x7ff7680b2760 2023-05-20T01:19:56.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.242+0000 7ff765ffb700 1 -- 172.21.15.16:0/2823649577 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff75c011d20 con 0x7ff7680b2760 2023-05-20T01:19:56.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.243+0000 7ff76d568700 1 -- 172.21.15.16:0/2823649577 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7680b2760 msgr2=0x7ff7680d1680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:56.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.243+0000 7ff76d568700 1 --2- 172.21.15.16:0/2823649577 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7680b2760 0x7ff7680d1680 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7ff75c009840 tx=0x7ff75c009b50 comp rx=0 tx=0).stop 2023-05-20T01:19:56.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.243+0000 7ff76d568700 1 -- 172.21.15.16:0/2823649577 shutdown_connections 2023-05-20T01:19:56.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.243+0000 7ff76d568700 1 --2- 172.21.15.16:0/2823649577 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff7680cec60 0x7ff7680cf0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:56.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.243+0000 7ff76d568700 1 --2- 172.21.15.16:0/2823649577 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7680b2760 0x7ff7680d1680 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:56.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.243+0000 7ff76d568700 1 -- 172.21.15.16:0/2823649577 >> 172.21.15.16:0/2823649577 conn(0x7ff76806ae30 msgr2=0x7ff76809a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:19:56.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.243+0000 7ff76d568700 1 -- 172.21.15.16:0/2823649577 shutdown_connections 2023-05-20T01:19:56.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.243+0000 7ff76d568700 1 -- 172.21.15.16:0/2823649577 wait complete. 2023-05-20T01:19:56.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.244+0000 7ff76d568700 1 Processor -- start 2023-05-20T01:19:56.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.244+0000 7ff76d568700 1 -- start start 2023-05-20T01:19:56.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.245+0000 7ff76d568700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff7680b2760 0x7ff76809ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:56.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.245+0000 7ff76d568700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7680cec60 0x7ff7680b0490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:56.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.245+0000 7ff76d568700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff768144020 0x7ff76815b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:56.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.245+0000 7ff76d568700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7680b5440 con 0x7ff7680cec60 2023-05-20T01:19:56.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.245+0000 7ff76d568700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7680b5720 con 0x7ff768144020 2023-05-20T01:19:56.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.245+0000 7ff76d568700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7680b5a00 con 0x7ff7680b2760 2023-05-20T01:19:56.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.245+0000 7ff7667fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7680cec60 0x7ff7680b0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:56.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.245+0000 7ff7667fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7680cec60 0x7ff7680b0490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33870/0 (socket says 172.21.15.16:33870) 2023-05-20T01:19:56.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.245+0000 7ff7667fc700 1 -- 172.21.15.16:0/130726900 learned_addr learned my addr 172.21.15.16:0/130726900 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:19:56.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.245+0000 7ff7677fe700 1 --2- 172.21.15.16:0/130726900 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff768144020 0x7ff76815b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:56.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.245+0000 7ff766ffd700 1 --2- 172.21.15.16:0/130726900 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff7680b2760 0x7ff76809ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:56.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.245+0000 7ff7667fc700 1 -- 172.21.15.16:0/130726900 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff7680b2760 msgr2=0x7ff76809ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:56.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.245+0000 7ff7667fc700 1 --2- 172.21.15.16:0/130726900 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff7680b2760 0x7ff76809ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:56.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.245+0000 7ff7667fc700 1 -- 172.21.15.16:0/130726900 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff768144020 msgr2=0x7ff76815b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:56.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.245+0000 7ff7667fc700 1 --2- 172.21.15.16:0/130726900 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff768144020 0x7ff76815b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:56.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.245+0000 7ff7667fc700 1 -- 172.21.15.16:0/130726900 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff75c0094a0 con 0x7ff7680cec60 2023-05-20T01:19:56.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.245+0000 7ff7677fe700 1 --2- 172.21.15.16:0/130726900 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff768144020 0x7ff76815b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:19:56.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.245+0000 7ff766ffd700 1 --2- 172.21.15.16:0/130726900 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff7680b2760 0x7ff76809ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:19:56.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.245+0000 7ff7667fc700 1 --2- 172.21.15.16:0/130726900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7680cec60 0x7ff7680b0490 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7ff7580011e0 tx=0x7ff75800cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:56.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.246+0000 7ff74f7fe700 1 -- 172.21.15.16:0/130726900 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff758004c80 con 0x7ff7680cec60 2023-05-20T01:19:56.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.246+0000 7ff74f7fe700 1 -- 172.21.15.16:0/130726900 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff758004e00 con 0x7ff7680cec60 2023-05-20T01:19:56.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.246+0000 7ff74f7fe700 1 -- 172.21.15.16:0/130726900 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff758007310 con 0x7ff7680cec60 2023-05-20T01:19:56.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.246+0000 7ff76d568700 1 -- 172.21.15.16:0/130726900 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7680cbeb0 con 0x7ff7680cec60 2023-05-20T01:19:56.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.246+0000 7ff76d568700 1 -- 172.21.15.16:0/130726900 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7680cbbd0 con 0x7ff7680cec60 2023-05-20T01:19:56.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.246+0000 7ff76d568700 1 -- 172.21.15.16:0/130726900 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7680cc190 con 0x7ff7680cec60 2023-05-20T01:19:56.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.247+0000 7ff74f7fe700 1 -- 172.21.15.16:0/130726900 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff758007470 con 0x7ff7680cec60 2023-05-20T01:19:56.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.247+0000 7ff74f7fe700 1 --2- 172.21.15.16:0/130726900 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff75406ace0 0x7ff75406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:56.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.248+0000 7ff74f7fe700 1 -- 172.21.15.16:0/130726900 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6382+0+0 (secure 0 0 0) 0x7ff758089300 con 0x7ff7680cec60 2023-05-20T01:19:56.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.248+0000 7ff766ffd700 1 --2- 172.21.15.16:0/130726900 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff75406ace0 0x7ff75406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:56.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.248+0000 7ff766ffd700 1 --2- 172.21.15.16:0/130726900 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff75406ace0 0x7ff75406d190 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7ff75c009b90 tx=0x7ff75c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:56.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.252+0000 7ff74f7fe700 1 -- 172.21.15.16:0/130726900 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7ff758054300 con 0x7ff7680cec60 2023-05-20T01:19:56.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.448+0000 7ff76d568700 1 -- 172.21.15.16:0/130726900 --> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff7680747b0 con 0x7ff75406ace0 2023-05-20T01:19:56.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.457+0000 7ff74f7fe700 1 -- 172.21.15.16:0/130726900 <== mgr.4104 v2:172.21.15.16:6832/3817237998 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+296970 (secure 0 0 0) 0x7ff7680747b0 con 0x7ff75406ace0 2023-05-20T01:19:56.466 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:19:56.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.460+0000 7ff76d568700 1 -- 172.21.15.16:0/130726900 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff75406ace0 msgr2=0x7ff75406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:56.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.460+0000 7ff76d568700 1 --2- 172.21.15.16:0/130726900 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff75406ace0 0x7ff75406d190 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7ff75c009b90 tx=0x7ff75c008030 comp rx=0 tx=0).stop 2023-05-20T01:19:56.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.460+0000 7ff76d568700 1 -- 172.21.15.16:0/130726900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7680cec60 msgr2=0x7ff7680b0490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:56.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.460+0000 7ff76d568700 1 --2- 172.21.15.16:0/130726900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7680cec60 0x7ff7680b0490 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7ff7580011e0 tx=0x7ff75800cae0 comp rx=0 tx=0).stop 2023-05-20T01:19:56.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.461+0000 7ff76d568700 1 -- 172.21.15.16:0/130726900 shutdown_connections 2023-05-20T01:19:56.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.461+0000 7ff76d568700 1 --2- 172.21.15.16:0/130726900 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff768144020 0x7ff76815b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:56.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.461+0000 7ff76d568700 1 --2- 172.21.15.16:0/130726900 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff75406ace0 0x7ff75406d190 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:56.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.461+0000 7ff76d568700 1 --2- 172.21.15.16:0/130726900 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff7680b2760 0x7ff76809ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:56.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.461+0000 7ff76d568700 1 --2- 172.21.15.16:0/130726900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7680cec60 0x7ff7680b0490 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:56.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.461+0000 7ff76d568700 1 -- 172.21.15.16:0/130726900 >> 172.21.15.16:0/130726900 conn(0x7ff76806ae30 msgr2=0x7ff7680addf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:19:56.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.461+0000 7ff76d568700 1 -- 172.21.15.16:0/130726900 shutdown_connections 2023-05-20T01:19:56.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.461+0000 7ff76d568700 1 -- 172.21.15.16:0/130726900 wait complete. 2023-05-20T01:19:56.473 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2023-05-20T01:19:56.477 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":165,"stamp":"2023-05-20T01:19:55.364291+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":77184,"kb_used_data":6784,"kb_used_omap":0,"kb_used_meta":70336,"kb_avail":754897536,"statfs":{"total":773094113280,"available":773015076864,"internally_reserved":0,"allocated":6946816,"data_stored":3957374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":72024064},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.881826"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:50.473280+0000","last_change":"2023-05-20T01:19:50.473280+0000","last_active":"2023-05-20T01:19:50.473280+0000","last_peered":"2023-05-20T01:19:50.473280+0000","last_clean":"2023-05-20T01:19:50.473280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:50.473280+0000","last_undegraded":"2023-05-20T01:19:50.473280+0000","last_fullsized":"2023-05-20T01:19:50.473280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:50.473280+0000","last_change":"2023-05-20T01:19:50.473280+0000","last_active":"2023-05-20T01:19:50.473280+0000","last_peered":"2023-05-20T01:19:50.473280+0000","last_clean":"2023-05-20T01:19:50.473280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:50.473280+0000","last_undegraded":"2023-05-20T01:19:50.473280+0000","last_fullsized":"2023-05-20T01:19:50.473280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:50.473280+0000","last_change":"2023-05-20T01:19:50.473280+0000","last_active":"2023-05-20T01:19:50.473280+0000","last_peered":"2023-05-20T01:19:50.473280+0000","last_clean":"2023-05-20T01:19:50.473280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:50.473280+0000","last_undegraded":"2023-05-20T01:19:50.473280+0000","last_fullsized":"2023-05-20T01:19:50.473280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:50.473280+0000","last_change":"2023-05-20T01:19:50.473280+0000","last_active":"2023-05-20T01:19:50.473280+0000","last_peered":"2023-05-20T01:19:50.473280+0000","last_clean":"2023-05-20T01:19:50.473280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:50.473280+0000","last_undegraded":"2023-05-20T01:19:50.473280+0000","last_fullsized":"2023-05-20T01:19:50.473280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:50.473280+0000","last_change":"2023-05-20T01:19:50.473280+0000","last_active":"2023-05-20T01:19:50.473280+0000","last_peered":"2023-05-20T01:19:50.473280+0000","last_clean":"2023-05-20T01:19:50.473280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:50.473280+0000","last_undegraded":"2023-05-20T01:19:50.473280+0000","last_fullsized":"2023-05-20T01:19:50.473280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:50.473280+0000","last_change":"2023-05-20T01:19:50.473280+0000","last_active":"2023-05-20T01:19:50.473280+0000","last_peered":"2023-05-20T01:19:50.473280+0000","last_clean":"2023-05-20T01:19:50.473280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:50.473280+0000","last_undegraded":"2023-05-20T01:19:50.473280+0000","last_fullsized":"2023-05-20T01:19:50.473280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:50.473280+0000","last_change":"2023-05-20T01:19:50.473280+0000","last_active":"2023-05-20T01:19:50.473280+0000","last_peered":"2023-05-20T01:19:50.473280+0000","last_clean":"2023-05-20T01:19:50.473280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:50.473280+0000","last_undegraded":"2023-05-20T01:19:50.473280+0000","last_fullsized":"2023-05-20T01:19:50.473280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:50.473280+0000","last_change":"2023-05-20T01:19:50.473280+0000","last_active":"2023-05-20T01:19:50.473280+0000","last_peered":"2023-05-20T01:19:50.473280+0000","last_clean":"2023-05-20T01:19:50.473280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:50.473280+0000","last_undegraded":"2023-05-20T01:19:50.473280+0000","last_fullsized":"2023-05-20T01:19:50.473280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:50.473280+0000","last_change":"2023-05-20T01:19:50.473280+0000","last_active":"2023-05-20T01:19:50.473280+0000","last_peered":"2023-05-20T01:19:50.473280+0000","last_clean":"2023-05-20T01:19:50.473280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:50.473280+0000","last_undegraded":"2023-05-20T01:19:50.473280+0000","last_fullsized":"2023-05-20T01:19:50.473280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:50.473280+0000","last_change":"2023-05-20T01:19:50.473280+0000","last_active":"2023-05-20T01:19:50.473280+0000","last_peered":"2023-05-20T01:19:50.473280+0000","last_clean":"2023-05-20T01:19:50.473280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:50.473280+0000","last_undegraded":"2023-05-20T01:19:50.473280+0000","last_fullsized":"2023-05-20T01:19:50.473280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:50.473280+0000","last_change":"2023-05-20T01:19:50.473280+0000","last_active":"2023-05-20T01:19:50.473280+0000","last_peered":"2023-05-20T01:19:50.473280+0000","last_clean":"2023-05-20T01:19:50.473280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:50.473280+0000","last_undegraded":"2023-05-20T01:19:50.473280+0000","last_fullsized":"2023-05-20T01:19:50.473280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:50.473280+0000","last_change":"2023-05-20T01:19:50.473280+0000","last_active":"2023-05-20T01:19:50.473280+0000","last_peered":"2023-05-20T01:19:50.473280+0000","last_clean":"2023-05-20T01:19:50.473280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:50.473280+0000","last_undegraded":"2023-05-20T01:19:50.473280+0000","last_fullsized":"2023-05-20T01:19:50.473280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:50.473280+0000","last_change":"2023-05-20T01:19:50.473280+0000","last_active":"2023-05-20T01:19:50.473280+0000","last_peered":"2023-05-20T01:19:50.473280+0000","last_clean":"2023-05-20T01:19:50.473280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:50.473280+0000","last_undegraded":"2023-05-20T01:19:50.473280+0000","last_fullsized":"2023-05-20T01:19:50.473280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:50.473280+0000","last_change":"2023-05-20T01:19:50.473280+0000","last_active":"2023-05-20T01:19:50.473280+0000","last_peered":"2023-05-20T01:19:50.473280+0000","last_clean":"2023-05-20T01:19:50.473280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:50.473280+0000","last_undegraded":"2023-05-20T01:19:50.473280+0000","last_fullsized":"2023-05-20T01:19:50.473280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:50.473280+0000","last_change":"2023-05-20T01:19:50.473280+0000","last_active":"2023-05-20T01:19:50.473280+0000","last_peered":"2023-05-20T01:19:50.473280+0000","last_clean":"2023-05-20T01:19:50.473280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:50.473280+0000","last_undegraded":"2023-05-20T01:19:50.473280+0000","last_fullsized":"2023-05-20T01:19:50.473280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:50.473280+0000","last_change":"2023-05-20T01:19:50.473280+0000","last_active":"2023-05-20T01:19:50.473280+0000","last_peered":"2023-05-20T01:19:50.473280+0000","last_clean":"2023-05-20T01:19:50.473280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:50.473280+0000","last_undegraded":"2023-05-20T01:19:50.473280+0000","last_fullsized":"2023-05-20T01:19:50.473280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:50.473280+0000","last_change":"2023-05-20T01:19:50.473280+0000","last_active":"2023-05-20T01:19:50.473280+0000","last_peered":"2023-05-20T01:19:50.473280+0000","last_clean":"2023-05-20T01:19:50.473280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:50.473280+0000","last_undegraded":"2023-05-20T01:19:50.473280+0000","last_fullsized":"2023-05-20T01:19:50.473280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:50.473280+0000","last_change":"2023-05-20T01:19:50.473280+0000","last_active":"2023-05-20T01:19:50.473280+0000","last_peered":"2023-05-20T01:19:50.473280+0000","last_clean":"2023-05-20T01:19:50.473280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:50.473280+0000","last_undegraded":"2023-05-20T01:19:50.473280+0000","last_fullsized":"2023-05-20T01:19:50.473280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:50.473280+0000","last_change":"2023-05-20T01:19:50.473280+0000","last_active":"2023-05-20T01:19:50.473280+0000","last_peered":"2023-05-20T01:19:50.473280+0000","last_clean":"2023-05-20T01:19:50.473280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:50.473280+0000","last_undegraded":"2023-05-20T01:19:50.473280+0000","last_fullsized":"2023-05-20T01:19:50.473280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:50.473280+0000","last_change":"2023-05-20T01:19:50.473280+0000","last_active":"2023-05-20T01:19:50.473280+0000","last_peered":"2023-05-20T01:19:50.473280+0000","last_clean":"2023-05-20T01:19:50.473280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:50.473280+0000","last_undegraded":"2023-05-20T01:19:50.473280+0000","last_fullsized":"2023-05-20T01:19:50.473280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:50.473280+0000","last_change":"2023-05-20T01:19:50.473280+0000","last_active":"2023-05-20T01:19:50.473280+0000","last_peered":"2023-05-20T01:19:50.473280+0000","last_clean":"2023-05-20T01:19:50.473280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:50.473280+0000","last_undegraded":"2023-05-20T01:19:50.473280+0000","last_fullsized":"2023-05-20T01:19:50.473280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:50.473280+0000","last_change":"2023-05-20T01:19:50.473280+0000","last_active":"2023-05-20T01:19:50.473280+0000","last_peered":"2023-05-20T01:19:50.473280+0000","last_clean":"2023-05-20T01:19:50.473280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:50.473280+0000","last_undegraded":"2023-05-20T01:19:50.473280+0000","last_fullsized":"2023-05-20T01:19:50.473280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:50.473280+0000","last_change":"2023-05-20T01:19:50.473280+0000","last_active":"2023-05-20T01:19:50.473280+0000","last_peered":"2023-05-20T01:19:50.473280+0000","last_clean":"2023-05-20T01:19:50.473280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:50.473280+0000","last_undegraded":"2023-05-20T01:19:50.473280+0000","last_fullsized":"2023-05-20T01:19:50.473280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:50.473280+0000","last_change":"2023-05-20T01:19:50.473280+0000","last_active":"2023-05-20T01:19:50.473280+0000","last_peered":"2023-05-20T01:19:50.473280+0000","last_clean":"2023-05-20T01:19:50.473280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:50.473280+0000","last_undegraded":"2023-05-20T01:19:50.473280+0000","last_fullsized":"2023-05-20T01:19:50.473280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:50.473280+0000","last_change":"2023-05-20T01:19:50.473280+0000","last_active":"2023-05-20T01:19:50.473280+0000","last_peered":"2023-05-20T01:19:50.473280+0000","last_clean":"2023-05-20T01:19:50.473280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:50.473280+0000","last_undegraded":"2023-05-20T01:19:50.473280+0000","last_fullsized":"2023-05-20T01:19:50.473280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-20T01:19:49.160921+0000","last_change":"2023-05-20T01:16:23.289068+0000","last_active":"2023-05-20T01:19:49.160921+0000","last_peered":"2023-05-20T01:19:49.160921+0000","last_clean":"2023-05-20T01:19:49.160921+0000","last_became_active":"2023-05-20T01:16:23.288781+0000","last_became_peered":"2023-05-20T01:16:23.288781+0000","last_unstale":"2023-05-20T01:19:49.160921+0000","last_undegraded":"2023-05-20T01:19:49.160921+0000","last_fullsized":"2023-05-20T01:19:49.160921+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:22:31.777331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:50.473280+0000","last_change":"2023-05-20T01:19:50.473280+0000","last_active":"2023-05-20T01:19:50.473280+0000","last_peered":"2023-05-20T01:19:50.473280+0000","last_clean":"2023-05-20T01:19:50.473280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:50.473280+0000","last_undegraded":"2023-05-20T01:19:50.473280+0000","last_fullsized":"2023-05-20T01:19:50.473280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":98,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-20T01:19:47.623794+0000","last_change":"2023-05-20T01:16:22.829324+0000","last_active":"2023-05-20T01:19:47.623794+0000","last_peered":"2023-05-20T01:19:47.623794+0000","last_clean":"2023-05-20T01:19:47.623794+0000","last_became_active":"2023-05-20T01:16:22.828964+0000","last_became_peered":"2023-05-20T01:16:22.828964+0000","last_unstale":"2023-05-20T01:19:47.623794+0000","last_undegraded":"2023-05-20T01:19:47.623794+0000","last_fullsized":"2023-05-20T01:19:47.623794+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:37:06.370724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:50.473280+0000","last_change":"2023-05-20T01:19:50.473280+0000","last_active":"2023-05-20T01:19:50.473280+0000","last_peered":"2023-05-20T01:19:50.473280+0000","last_clean":"2023-05-20T01:19:50.473280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:50.473280+0000","last_undegraded":"2023-05-20T01:19:50.473280+0000","last_fullsized":"2023-05-20T01:19:50.473280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":90,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-20T01:19:48.831080+0000","last_change":"2023-05-20T01:16:22.834754+0000","last_active":"2023-05-20T01:19:48.831080+0000","last_peered":"2023-05-20T01:19:48.831080+0000","last_clean":"2023-05-20T01:19:48.831080+0000","last_became_active":"2023-05-20T01:16:22.834437+0000","last_became_peered":"2023-05-20T01:16:22.834437+0000","last_unstale":"2023-05-20T01:19:48.831080+0000","last_undegraded":"2023-05-20T01:19:48.831080+0000","last_fullsized":"2023-05-20T01:19:48.831080+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:11:59.543234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":230,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-20T01:19:49.160859+0000","last_change":"2023-05-20T01:16:22.834920+0000","last_active":"2023-05-20T01:19:49.160859+0000","last_peered":"2023-05-20T01:19:49.160859+0000","last_clean":"2023-05-20T01:19:49.160859+0000","last_became_active":"2023-05-20T01:16:22.833924+0000","last_became_peered":"2023-05-20T01:16:22.833924+0000","last_unstale":"2023-05-20T01:19:49.160859+0000","last_undegraded":"2023-05-20T01:19:49.160859+0000","last_fullsized":"2023-05-20T01:19:49.160859+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:08:49.674910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:50.473280+0000","last_change":"2023-05-20T01:19:50.473280+0000","last_active":"2023-05-20T01:19:50.473280+0000","last_peered":"2023-05-20T01:19:50.473280+0000","last_clean":"2023-05-20T01:19:50.473280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:50.473280+0000","last_undegraded":"2023-05-20T01:19:50.473280+0000","last_fullsized":"2023-05-20T01:19:50.473280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":80,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-20T01:19:49.160858+0000","last_change":"2023-05-20T01:16:23.290259+0000","last_active":"2023-05-20T01:19:49.160858+0000","last_peered":"2023-05-20T01:19:49.160858+0000","last_clean":"2023-05-20T01:19:49.160858+0000","last_became_active":"2023-05-20T01:16:23.289911+0000","last_became_peered":"2023-05-20T01:16:23.289911+0000","last_unstale":"2023-05-20T01:19:49.160858+0000","last_undegraded":"2023-05-20T01:19:49.160858+0000","last_fullsized":"2023-05-20T01:19:49.160858+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:39:38.298562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:50.473280+0000","last_change":"2023-05-20T01:19:50.473280+0000","last_active":"2023-05-20T01:19:50.473280+0000","last_peered":"2023-05-20T01:19:50.473280+0000","last_clean":"2023-05-20T01:19:50.473280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:50.473280+0000","last_undegraded":"2023-05-20T01:19:50.473280+0000","last_fullsized":"2023-05-20T01:19:50.473280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-20T01:19:49.160859+0000","last_change":"2023-05-20T01:16:23.290207+0000","last_active":"2023-05-20T01:19:49.160859+0000","last_peered":"2023-05-20T01:19:49.160859+0000","last_clean":"2023-05-20T01:19:49.160859+0000","last_became_active":"2023-05-20T01:16:23.289928+0000","last_became_peered":"2023-05-20T01:16:23.289928+0000","last_unstale":"2023-05-20T01:19:49.160859+0000","last_undegraded":"2023-05-20T01:19:49.160859+0000","last_fullsized":"2023-05-20T01:19:49.160859+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:11:42.924171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:50.473280+0000","last_change":"2023-05-20T01:19:50.473280+0000","last_active":"2023-05-20T01:19:50.473280+0000","last_peered":"2023-05-20T01:19:50.473280+0000","last_clean":"2023-05-20T01:19:50.473280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:50.473280+0000","last_undegraded":"2023-05-20T01:19:50.473280+0000","last_fullsized":"2023-05-20T01:19:50.473280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-20T01:19:47.624189+0000","last_change":"2023-05-20T01:16:23.290538+0000","last_active":"2023-05-20T01:19:47.624189+0000","last_peered":"2023-05-20T01:19:47.624189+0000","last_clean":"2023-05-20T01:19:47.624189+0000","last_became_active":"2023-05-20T01:16:23.290230+0000","last_became_peered":"2023-05-20T01:16:23.290230+0000","last_unstale":"2023-05-20T01:19:47.624189+0000","last_undegraded":"2023-05-20T01:19:47.624189+0000","last_fullsized":"2023-05-20T01:19:47.624189+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:49:18.136308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:50.473280+0000","last_change":"2023-05-20T01:19:50.473280+0000","last_active":"2023-05-20T01:19:50.473280+0000","last_peered":"2023-05-20T01:19:50.473280+0000","last_clean":"2023-05-20T01:19:50.473280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:50.473280+0000","last_undegraded":"2023-05-20T01:19:50.473280+0000","last_fullsized":"2023-05-20T01:19:50.473280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":82,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-20T01:19:47.624338+0000","last_change":"2023-05-20T01:16:23.290341+0000","last_active":"2023-05-20T01:19:47.624338+0000","last_peered":"2023-05-20T01:19:47.624338+0000","last_clean":"2023-05-20T01:19:47.624338+0000","last_became_active":"2023-05-20T01:16:23.290053+0000","last_became_peered":"2023-05-20T01:16:23.290053+0000","last_unstale":"2023-05-20T01:19:47.624338+0000","last_undegraded":"2023-05-20T01:19:47.624338+0000","last_fullsized":"2023-05-20T01:19:47.624338+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:45:41.557083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:50.473280+0000","last_change":"2023-05-20T01:19:50.473280+0000","last_active":"2023-05-20T01:19:50.473280+0000","last_peered":"2023-05-20T01:19:50.473280+0000","last_clean":"2023-05-20T01:19:50.473280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:50.473280+0000","last_undegraded":"2023-05-20T01:19:50.473280+0000","last_fullsized":"2023-05-20T01:19:50.473280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":82,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-20T01:19:47.624309+0000","last_change":"2023-05-20T01:16:22.834330+0000","last_active":"2023-05-20T01:19:47.624309+0000","last_peered":"2023-05-20T01:19:47.624309+0000","last_clean":"2023-05-20T01:19:47.624309+0000","last_became_active":"2023-05-20T01:16:22.833946+0000","last_became_peered":"2023-05-20T01:16:22.833946+0000","last_unstale":"2023-05-20T01:19:47.624309+0000","last_undegraded":"2023-05-20T01:19:47.624309+0000","last_fullsized":"2023-05-20T01:19:47.624309+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:01:18.093853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574894,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10380,"kb_used_data":1284,"kb_used_omap":0,"kb_used_meta":9088,"kb_avail":94361460,"statfs":{"total":96636764160,"available":96626135040,"internally_reserved":0,"allocated":1314816,"data_stored":937688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9306112},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.52800000000000002,"15min":0.53400000000000003},"min":{"1min":0.29699999999999999,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.71799999999999997,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.56599999999999995,"15min":0.57099999999999995},"min":{"1min":0.38700000000000001,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.70099999999999996,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.498}]},{"osd":1,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.79500000000000004,"15min":0.90700000000000003},"min":{"1min":0.435,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.78600000000000003,"5min":8.8879999999999999,"15min":8.8879999999999999},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.83399999999999996,"15min":0.95099999999999996},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.73299999999999998,"5min":8.875,"15min":8.875},"last":0.378}]},{"osd":2,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.77200000000000002,"15min":0.88},"min":{"1min":0.371,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.73299999999999998,"5min":8.9120000000000008,"15min":8.9120000000000008},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.77600000000000002,"15min":0.88400000000000001},"min":{"1min":0.35699999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.751,"5min":8.8520000000000003,"15min":8.8520000000000003},"last":0.443}]},{"osd":3,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.84299999999999997,"15min":0.98299999999999998},"min":{"1min":0.35299999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.74299999999999999,"5min":10.884,"15min":10.884},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.89900000000000002,"15min":1.0349999999999999},"min":{"1min":0.433,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.72199999999999998,"5min":10.952,"15min":10.952},"last":0.51300000000000001}]},{"osd":4,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.71499999999999997,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.53200000000000003,"15min":0.52900000000000003},"min":{"1min":0.38800000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.67300000000000004,"5min":0.875,"15min":0.875},"last":0.53100000000000003}]},{"osd":5,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54300000000000004,"15min":0.52800000000000002},"min":{"1min":0.39600000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.69899999999999995,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.58199999999999996,"15min":0.57799999999999996},"min":{"1min":0.42699999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.745,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.59699999999999998}]},{"osd":6,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59299999999999997},"min":{"1min":0.47699999999999998,"5min":0.378,"15min":0.378},"max":{"1min":0.77000000000000002,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.59299999999999997,"15min":0.58499999999999996},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.83299999999999996,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.73599999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574893,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9224,"kb_used_data":704,"kb_used_omap":0,"kb_used_meta":8512,"kb_avail":94362616,"statfs":{"total":96636764160,"available":96627318784,"internally_reserved":0,"allocated":720896,"data_stored":347320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8716288},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.495,"15min":0.48699999999999999},"min":{"1min":0.34000000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.71699999999999997,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.54300000000000004,"15min":0.54100000000000004},"min":{"1min":0.36699999999999999,"5min":0.307,"15min":0.307},"max":{"1min":0.71199999999999997,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.52900000000000003}]},{"osd":1,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53500000000000003,"15min":0.53400000000000003},"min":{"1min":0.35899999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.68200000000000005,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53300000000000003,"15min":0.53200000000000003},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.78600000000000003,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.65500000000000003}]},{"osd":2,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.51700000000000002,"15min":0.51500000000000001},"min":{"1min":0.35499999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.68700000000000006,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.56299999999999994,"15min":0.56699999999999995},"min":{"1min":0.44500000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.64000000000000001}]},{"osd":3,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.75900000000000001,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.54900000000000004,"15min":0.55000000000000004},"min":{"1min":0.39400000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.747,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.54100000000000004}]},{"osd":4,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.54000000000000004,"15min":0.56000000000000005},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.71799999999999997,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.56499999999999995,"15min":0.58399999999999996},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.72199999999999998,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.55300000000000005}]},{"osd":5,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.54000000000000004,"15min":0.54800000000000004},"min":{"1min":0.373,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.70699999999999996,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.60599999999999998,"15min":0.621},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.80200000000000005,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.67200000000000004}]},{"osd":7,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.59799999999999998,"15min":0.61199999999999999},"min":{"1min":0.38600000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.77800000000000002,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.61499999999999999,"15min":0.628},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.749,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.63}]}]},{"osd":1,"up_from":13,"seq":55834574893,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10316,"kb_used_data":1284,"kb_used_omap":0,"kb_used_meta":9024,"kb_avail":94361524,"statfs":{"total":96636764160,"available":96626200576,"internally_reserved":0,"allocated":1314816,"data_stored":937688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9240576},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:19:24 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.32700000000000001,"15min":0.32900000000000001},"min":{"1min":0.193,"5min":0.193,"15min":0.193},"max":{"1min":1.194,"5min":1.194,"15min":1.194},"last":0.252},{"interface":"front","average":{"1min":0.35699999999999998,"5min":0.32000000000000001,"15min":0.316},"min":{"1min":0.20599999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.314}]},{"osd":2,"last update":"Sat May 20 01:19:24 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.33000000000000002,"15min":0.32400000000000001},"min":{"1min":0.253,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.27500000000000002},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.42199999999999999,"15min":0.42599999999999999},"min":{"1min":0.29599999999999999,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.3580000000000001,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":0.35399999999999998}]},{"osd":3,"last update":"Sat May 20 01:19:24 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.38500000000000001,"15min":0.376},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":1.619,"5min":1.619,"15min":1.619},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.39800000000000002,"15min":0.39400000000000002},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":1.8400000000000001,"5min":1.8400000000000001,"15min":1.8400000000000001},"last":0.36599999999999999}]},{"osd":4,"last update":"Sat May 20 01:19:24 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.59599999999999997,"15min":0.59099999999999997},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.59099999999999997,"15min":0.58899999999999997},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.61099999999999999}]},{"osd":5,"last update":"Sat May 20 01:19:24 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.59999999999999998,"15min":0.58899999999999997},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.61599999999999999,"15min":0.61199999999999999},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.409,"5min":1.409,"15min":1.409},"last":0.59899999999999998}]},{"osd":6,"last update":"Sat May 20 01:19:24 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.61699999999999999,"15min":0.60599999999999998},"min":{"1min":0.47099999999999997,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.575,"5min":1.575,"15min":1.575},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.60199999999999998,"15min":0.59699999999999998},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.58899999999999997}]},{"osd":7,"last update":"Sat May 20 01:19:24 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.61299999999999999,"15min":0.60899999999999999},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.496,"5min":1.496,"15min":1.496},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.65100000000000002,"15min":0.64400000000000002},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.5169999999999999,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.61899999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574893,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9412,"kb_used_data":700,"kb_used_omap":0,"kb_used_meta":8704,"kb_avail":94362428,"statfs":{"total":96636764160,"available":96627126272,"internally_reserved":0,"allocated":716800,"data_stored":346359,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8912896},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.31,"5min":0.32500000000000001,"15min":0.33300000000000002},"min":{"1min":0.188,"5min":0.188,"15min":0.188},"max":{"1min":0.47599999999999998,"5min":0.56599999999999995,"15min":0.56599999999999995},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.33400000000000002,"5min":0.35399999999999998,"15min":0.36799999999999999},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.53400000000000003,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.33400000000000002}]},{"osd":2,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.34300000000000003,"15min":0.34100000000000003},"min":{"1min":0.19400000000000001,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.39600000000000002,"15min":0.39900000000000002},"min":{"1min":0.29399999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.67000000000000004,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.378}]},{"osd":3,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.36799999999999999,"15min":0.376},"min":{"1min":0.26100000000000001,"5min":0.245,"15min":0.245},"max":{"1min":0.51400000000000001,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.375,"5min":0.372,"15min":0.372},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.50600000000000001,"5min":0.60099999999999998,"15min":0.60099999999999998},"last":0.436}]},{"osd":4,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.54200000000000004,"15min":0.54000000000000004},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.76000000000000001,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.56599999999999995,"15min":0.56399999999999995},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.72299999999999998,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.495}]},{"osd":5,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.57299999999999995,"15min":0.57699999999999996},"min":{"1min":0.40300000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.68799999999999994,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56799999999999995,"15min":0.56499999999999995},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.78900000000000003,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.56200000000000006}]},{"osd":6,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.57199999999999995,"15min":0.57399999999999995},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.69799999999999995,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.63},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.61799999999999999,"15min":0.623},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.68500000000000005}]},{"osd":7,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.59499999999999997,"15min":0.59099999999999997},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.78300000000000003,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.61799999999999999,"15min":0.624},"min":{"1min":0.43099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.66300000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574893,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9416,"kb_used_data":704,"kb_used_omap":0,"kb_used_meta":8704,"kb_avail":94362424,"statfs":{"total":96636764160,"available":96627122176,"internally_reserved":0,"allocated":720896,"data_stored":347320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8912896},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.32000000000000001,"5min":0.33900000000000002,"15min":0.35299999999999998},"min":{"1min":0.20100000000000001,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.58999999999999997,"5min":1.038,"15min":1.038},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.35699999999999998,"5min":0.35799999999999998,"15min":0.36699999999999999},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.52900000000000003,"5min":1.079,"15min":1.079},"last":0.45200000000000001}]},{"osd":1,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.39600000000000002,"15min":0.42099999999999999},"min":{"1min":0.26000000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.50900000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.44},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.436,"15min":0.45500000000000002},"min":{"1min":0.28399999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.64500000000000002,"5min":1.5189999999999999,"15min":1.5189999999999999},"last":0.49299999999999999}]},{"osd":3,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.39700000000000002,"15min":0.41499999999999998},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.55000000000000004,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.45000000000000001,"15min":0.46999999999999997},"min":{"1min":0.32600000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.63100000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.51300000000000001}]},{"osd":4,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.58399999999999996,"15min":0.58999999999999997},"min":{"1min":0.42699999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.86499999999999999,"5min":1.319,"15min":1.319},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.625,"15min":0.626},"min":{"1min":0.48699999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.83899999999999997,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.64100000000000001}]},{"osd":5,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.64000000000000001,"15min":0.65500000000000003},"min":{"1min":0.44800000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.80100000000000005,"5min":1.758,"15min":1.758},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.625,"5min":0.65000000000000002,"15min":0.67000000000000004},"min":{"1min":0.48099999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.80800000000000005,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.80600000000000005}]},{"osd":6,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65100000000000002,"15min":0.65700000000000003},"min":{"1min":0.45500000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.88,"5min":1.665,"15min":1.665},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.626,"5min":0.64800000000000002,"15min":0.65900000000000003},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.86599999999999999,"5min":1.819,"15min":1.819},"last":0.70299999999999996}]},{"osd":7,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.68000000000000005,"15min":0.69199999999999995},"min":{"1min":0.45500000000000002,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.83199999999999996,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.66600000000000004,"15min":0.68400000000000005},"min":{"1min":0.49299999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.89300000000000002,"5min":1.8460000000000001,"15min":1.8460000000000001},"last":0.81899999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574893,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9480,"kb_used_data":704,"kb_used_omap":0,"kb_used_meta":8768,"kb_avail":94362360,"statfs":{"total":96636764160,"available":96627056640,"internally_reserved":0,"allocated":720896,"data_stored":347320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8978432},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:19:23 2023","interfaces":[{"interface":"back","average":{"1min":0.33100000000000002,"5min":0.318,"15min":0.32400000000000001},"min":{"1min":0.219,"5min":0.192,"15min":0.192},"max":{"1min":0.46100000000000002,"5min":0.504,"15min":0.504},"last":0.29799999999999999},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.42699999999999999,"15min":0.47599999999999998},"min":{"1min":0.247,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.52800000000000002,"5min":3.1890000000000001,"15min":3.1890000000000001},"last":0.317}]},{"osd":1,"last update":"Sat May 20 01:19:23 2023","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.42099999999999999,"15min":0.46700000000000003},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.501,"5min":3.4260000000000002,"15min":3.4260000000000002},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.36199999999999999,"15min":0.36399999999999999},"min":{"1min":0.26100000000000001,"5min":0.221,"15min":0.221},"max":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"last":0.372}]},{"osd":2,"last update":"Sat May 20 01:19:23 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.39800000000000002,"15min":0.41299999999999998},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.62,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.41999999999999998,"15min":0.44},"min":{"1min":0.28699999999999998,"5min":0.253,"15min":0.253},"max":{"1min":0.58899999999999997,"5min":1.413,"15min":1.413},"last":0.39600000000000002}]},{"osd":4,"last update":"Sat May 20 01:19:23 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.59599999999999997,"15min":0.60799999999999998},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.80100000000000005,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.68300000000000005,"15min":0.72499999999999998},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.83599999999999997,"5min":3.4660000000000002,"15min":3.4660000000000002},"last":0.434}]},{"osd":5,"last update":"Sat May 20 01:19:23 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.69399999999999995,"15min":0.73999999999999999},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.77900000000000003,"5min":3.5379999999999998,"15min":3.5379999999999998},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.63300000000000001,"15min":0.65300000000000002},"min":{"1min":0.42599999999999999,"5min":0.374,"15min":0.374},"max":{"1min":0.81699999999999995,"5min":1.385,"15min":1.385},"last":0.58599999999999997}]},{"osd":6,"last update":"Sat May 20 01:19:23 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.63800000000000001,"15min":0.65800000000000003},"min":{"1min":0.40400000000000003,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.84299999999999997,"5min":1.347,"15min":1.347},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64200000000000002,"15min":0.65700000000000003},"min":{"1min":0.434,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.77400000000000002,"5min":1.107,"15min":1.107},"last":0.45700000000000002}]},{"osd":7,"last update":"Sat May 20 01:19:23 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.64600000000000002,"15min":0.66000000000000003},"min":{"1min":0.46600000000000003,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.83199999999999996,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.72899999999999998,"15min":0.77400000000000002},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.82199999999999995,"5min":3.5009999999999999,"15min":3.5009999999999999},"last":0.59399999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574893,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9736,"kb_used_data":704,"kb_used_omap":0,"kb_used_meta":9024,"kb_avail":94362104,"statfs":{"total":96636764160,"available":96626794496,"internally_reserved":0,"allocated":720896,"data_stored":347320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9240576},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:19:24 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.495,"15min":0.48099999999999998},"min":{"1min":0.379,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.72399999999999998,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.497,"15min":0.49299999999999999},"min":{"1min":0.35499999999999998,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.55200000000000005}]},{"osd":1,"last update":"Sat May 20 01:19:24 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.53400000000000003,"15min":0.53200000000000003},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.73299999999999998,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.622},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.54600000000000004,"15min":0.53200000000000003},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":2.0739999999999998,"5min":2.0739999999999998,"15min":2.0739999999999998},"last":0.498}]},{"osd":2,"last update":"Sat May 20 01:19:24 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.51400000000000001,"15min":0.499},"min":{"1min":0.439,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.745,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.55400000000000005,"15min":0.55500000000000005},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.751,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.60899999999999999}]},{"osd":3,"last update":"Sat May 20 01:19:24 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.53300000000000003,"15min":0.51900000000000002},"min":{"1min":0.42699999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.827,"5min":1.827,"15min":1.827},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.66700000000000004,"15min":0.70699999999999996},"min":{"1min":0.40400000000000003,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.77400000000000002,"5min":3.1749999999999998,"15min":3.1749999999999998},"last":0.59699999999999998}]},{"osd":5,"last update":"Sat May 20 01:19:24 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.47199999999999998,"15min":0.46000000000000002},"min":{"1min":0.33800000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.63100000000000001,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.49199999999999999,"15min":0.49299999999999999},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.67300000000000004,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.35399999999999998}]},{"osd":6,"last update":"Sat May 20 01:19:24 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52900000000000003,"15min":0.52500000000000002},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.64700000000000002,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.58199999999999996,"15min":0.57799999999999996},"min":{"1min":0.46200000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.65500000000000003,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.58299999999999996}]},{"osd":7,"last update":"Sat May 20 01:19:24 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.54800000000000004,"15min":0.54500000000000004},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.443},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.58599999999999997,"15min":0.58999999999999997},"min":{"1min":0.39100000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.67200000000000004,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.47099999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574894,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9220,"kb_used_data":700,"kb_used_omap":0,"kb_used_meta":8512,"kb_avail":94362620,"statfs":{"total":96636764160,"available":96627322880,"internally_reserved":0,"allocated":716800,"data_stored":346359,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8716288},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:19:26 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.68799999999999994,"15min":0.77400000000000002},"min":{"1min":0.33700000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.71899999999999997,"5min":6.3929999999999998,"15min":6.3929999999999998},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.72199999999999998,"15min":0.81200000000000006},"min":{"1min":0.434,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.745,"5min":6.3339999999999996,"15min":6.3339999999999996},"last":0.35899999999999999}]},{"osd":1,"last update":"Sat May 20 01:19:26 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.60999999999999999,"15min":0.66500000000000004},"min":{"1min":0.38100000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.73899999999999999,"5min":3.2450000000000001,"15min":3.2450000000000001},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54500000000000004,"15min":0.55500000000000005},"min":{"1min":0.41199999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.71899999999999997,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.46999999999999997}]},{"osd":2,"last update":"Sat May 20 01:19:26 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.55500000000000005,"15min":0.57599999999999996},"min":{"1min":0.34100000000000003,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.76300000000000001,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.56100000000000005,"15min":0.57899999999999996},"min":{"1min":0.375,"5min":0.314,"15min":0.314},"max":{"1min":0.72299999999999998,"5min":1.258,"15min":1.258},"last":0.55500000000000005}]},{"osd":3,"last update":"Sat May 20 01:19:26 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.751,"15min":0.85699999999999998},"min":{"1min":0.38200000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.68600000000000005,"5min":6.351,"15min":6.351},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.79300000000000004,"15min":0.89800000000000002},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.72599999999999998,"5min":6.3730000000000002,"15min":6.3730000000000002},"last":0.57899999999999996}]},{"osd":4,"last update":"Sat May 20 01:19:26 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.48599999999999999,"15min":0.48799999999999999},"min":{"1min":0.32400000000000001,"5min":0.316,"15min":0.316},"max":{"1min":0.65300000000000002,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.54800000000000004,"15min":0.56000000000000005},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.53100000000000003}]},{"osd":6,"last update":"Sat May 20 01:19:26 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.54300000000000004,"15min":0.55900000000000005},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.67600000000000005,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.56000000000000005,"15min":0.56599999999999995},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.747,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.59399999999999997}]},{"osd":7,"last update":"Sat May 20 01:19:26 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.57899999999999996,"15min":0.59199999999999997},"min":{"1min":0.36199999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.76900000000000002,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.61299999999999999,"15min":0.627},"min":{"1min":0.41399999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.73399999999999999,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.625}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-05-20T01:19:56.486 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-05-20T01:19:56.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.638+0000 7f1fb6cbe700 1 Processor -- start 2023-05-20T01:19:56.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.638+0000 7f1fb6cbe700 1 -- start start 2023-05-20T01:19:56.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.639+0000 7f1fb6cbe700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f1fb01585d0 0x7f1fb015ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:56.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.639+0000 7f1fb6cbe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1fb015b360 0x7f1fb006a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:56.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.639+0000 7f1fb6cbe700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fb00ccf40 con 0x7f1fb00cc9c0 2023-05-20T01:19:56.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.639+0000 7f1fb6cbe700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fb015e540 con 0x7f1fb015b360 2023-05-20T01:19:56.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.639+0000 7f1fb6cbe700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fb00cd220 con 0x7f1fb01585d0 2023-05-20T01:19:56.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.639+0000 7f1faffff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1fb015b360 0x7f1fb006a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:56.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.639+0000 7f1faffff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1fb015b360 0x7f1fb006a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33878/0 (socket says 172.21.15.16:33878) 2023-05-20T01:19:56.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.639+0000 7f1fb525b700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f1fb00cc9c0 0x7f1fb015e990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:45408/0 (socket says 172.21.15.16:45408) 2023-05-20T01:19:56.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.639+0000 7f1faffff700 1 -- 172.21.15.16:0/1789603913 learned_addr learned my addr 172.21.15.16:0/1789603913 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:19:56.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.639+0000 7f1fb4a5a700 1 --2- 172.21.15.16:0/1789603913 >> v2:172.21.15.132:3300/0 conn(0x7f1fb01585d0 0x7f1fb015ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:56.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.640+0000 7f1faffff700 1 -- 172.21.15.16:0/1789603913 >> v2:172.21.15.132:3300/0 conn(0x7f1fb01585d0 msgr2=0x7f1fb015ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:19:56.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.640+0000 7f1faffff700 1 --2- 172.21.15.16:0/1789603913 >> v2:172.21.15.132:3300/0 conn(0x7f1fb01585d0 0x7f1fb015ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:56.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.640+0000 7f1faffff700 1 -- 172.21.15.16:0/1789603913 >> v1:172.21.15.16:6789/0 conn(0x7f1fb00cc9c0 legacy=0x7f1fb015e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:56.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.640+0000 7f1faffff700 1 -- 172.21.15.16:0/1789603913 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fb00cd500 con 0x7f1fb015b360 2023-05-20T01:19:56.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.640+0000 7f1fb4a5a700 1 --2- 172.21.15.16:0/1789603913 >> v2:172.21.15.132:3300/0 conn(0x7f1fb01585d0 0x7f1fb015ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:19:56.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.640+0000 7f1faf7fe700 1 -- 172.21.15.16:0/1789603913 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 728355673 0 0) 0x7f1fb00ccf40 con 0x7f1fb00cc9c0 2023-05-20T01:19:56.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.640+0000 7f1faffff700 1 --2- 172.21.15.16:0/1789603913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1fb015b360 0x7f1fb006a510 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f1fb00cfde0 tx=0x7f1f9c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16dfc21a9d020446 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:56.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.640+0000 7f1faf7fe700 1 -- 172.21.15.16:0/1789603913 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f9c00b870 con 0x7f1fb015b360 2023-05-20T01:19:56.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.640+0000 7f1faf7fe700 1 -- 172.21.15.16:0/1789603913 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f9c00b9d0 con 0x7f1fb015b360 2023-05-20T01:19:56.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.640+0000 7f1faf7fe700 1 -- 172.21.15.16:0/1789603913 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f9c00bd60 con 0x7f1fb015b360 2023-05-20T01:19:56.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.640+0000 7f1fb6cbe700 1 -- 172.21.15.16:0/1789603913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1fb015b360 msgr2=0x7f1fb006a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:56.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.640+0000 7f1fb6cbe700 1 --2- 172.21.15.16:0/1789603913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1fb015b360 0x7f1fb006a510 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f1fb00cfde0 tx=0x7f1f9c009bd0 comp rx=0 tx=0).stop 2023-05-20T01:19:56.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.641+0000 7f1fb6cbe700 1 -- 172.21.15.16:0/1789603913 shutdown_connections 2023-05-20T01:19:56.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.641+0000 7f1fb6cbe700 1 --2- 172.21.15.16:0/1789603913 >> v2:172.21.15.132:3300/0 conn(0x7f1fb01585d0 0x7f1fb015ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:56.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.641+0000 7f1fb6cbe700 1 --2- 172.21.15.16:0/1789603913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1fb015b360 0x7f1fb006a510 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:56.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.641+0000 7f1fb6cbe700 1 -- 172.21.15.16:0/1789603913 >> 172.21.15.16:0/1789603913 conn(0x7f1fb00cffb0 msgr2=0x7f1fb00d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:19:56.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.641+0000 7f1fb6cbe700 1 -- 172.21.15.16:0/1789603913 shutdown_connections 2023-05-20T01:19:56.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.641+0000 7f1fb6cbe700 1 -- 172.21.15.16:0/1789603913 wait complete. 2023-05-20T01:19:56.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.642+0000 7f1fb6cbe700 1 Processor -- start 2023-05-20T01:19:56.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.642+0000 7f1fb6cbe700 1 -- start start 2023-05-20T01:19:56.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.643+0000 7f1fb6cbe700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1fb00cc9c0 0x7f1fb00c79b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:56.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.643+0000 7f1fb6cbe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1fb01585d0 0x7f1fb01f36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:56.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.643+0000 7f1fb6cbe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1fb015b360 0x7f1fb0159a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:56.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.643+0000 7f1fb6cbe700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb00c7590 con 0x7f1fb01585d0 2023-05-20T01:19:56.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.643+0000 7f1fb6cbe700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb01f4580 con 0x7f1fb015b360 2023-05-20T01:19:56.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.643+0000 7f1fb6cbe700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb01f3fc0 con 0x7f1fb00cc9c0 2023-05-20T01:19:56.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.643+0000 7f1faffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1fb01585d0 0x7f1fb01f36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:56.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.643+0000 7f1faffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1fb01585d0 0x7f1fb01f36e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33884/0 (socket says 172.21.15.16:33884) 2023-05-20T01:19:56.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.643+0000 7f1faffff700 1 -- 172.21.15.16:0/1659245880 learned_addr learned my addr 172.21.15.16:0/1659245880 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:19:56.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.643+0000 7f1fb525b700 1 --2- 172.21.15.16:0/1659245880 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1fb015b360 0x7f1fb0159a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:56.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.643+0000 7f1faffff700 1 -- 172.21.15.16:0/1659245880 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1fb00cc9c0 msgr2=0x7f1fb00c79b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:56.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.643+0000 7f1fb4a5a700 1 --2- 172.21.15.16:0/1659245880 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1fb00cc9c0 0x7f1fb00c79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:56.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.643+0000 7f1faffff700 1 --2- 172.21.15.16:0/1659245880 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1fb00cc9c0 0x7f1fb00c79b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:56.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.643+0000 7f1faffff700 1 -- 172.21.15.16:0/1659245880 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1fb015b360 msgr2=0x7f1fb0159a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:56.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.643+0000 7f1faffff700 1 --2- 172.21.15.16:0/1659245880 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1fb015b360 0x7f1fb0159a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:56.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.643+0000 7f1faffff700 1 -- 172.21.15.16:0/1659245880 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f9c009550 con 0x7f1fb01585d0 2023-05-20T01:19:56.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.643+0000 7f1fb525b700 1 --2- 172.21.15.16:0/1659245880 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1fb015b360 0x7f1fb0159a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:19:56.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.643+0000 7f1faffff700 1 --2- 172.21.15.16:0/1659245880 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1fb01585d0 0x7f1fb01f36e0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c009130 tx=0x7f1f9c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:56.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.643+0000 7f1fad7fa700 1 -- 172.21.15.16:0/1659245880 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f9c004520 con 0x7f1fb01585d0 2023-05-20T01:19:56.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.643+0000 7f1fad7fa700 1 -- 172.21.15.16:0/1659245880 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f9c01e070 con 0x7f1fb01585d0 2023-05-20T01:19:56.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.643+0000 7f1fb6cbe700 1 -- 172.21.15.16:0/1659245880 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fb00719a0 con 0x7f1fb01585d0 2023-05-20T01:19:56.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.644+0000 7f1fad7fa700 1 -- 172.21.15.16:0/1659245880 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f9c0038e0 con 0x7f1fb01585d0 2023-05-20T01:19:56.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.644+0000 7f1fb6cbe700 1 -- 172.21.15.16:0/1659245880 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fb00cd7e0 con 0x7f1fb01585d0 2023-05-20T01:19:56.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.644+0000 7f1fb6cbe700 1 -- 172.21.15.16:0/1659245880 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fb015f0a0 con 0x7f1fb01585d0 2023-05-20T01:19:56.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.645+0000 7f1fad7fa700 1 -- 172.21.15.16:0/1659245880 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1f9c003a40 con 0x7f1fb01585d0 2023-05-20T01:19:56.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.645+0000 7f1fad7fa700 1 --2- 172.21.15.16:0/1659245880 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f1f9406ace0 0x7f1f9406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:56.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.645+0000 7f1fad7fa700 1 -- 172.21.15.16:0/1659245880 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f1f9c020070 con 0x7f1fb01585d0 2023-05-20T01:19:56.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.645+0000 7f1fb4a5a700 1 --2- 172.21.15.16:0/1659245880 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f1f9406ace0 0x7f1f9406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:56.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.646+0000 7f1fb4a5a700 1 --2- 172.21.15.16:0/1659245880 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f1f9406ace0 0x7f1f9406d190 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f1fa4009930 tx=0x7f1fa4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:56.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.650+0000 7f1fad7fa700 1 -- 172.21.15.16:0/1659245880 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f1f9c055bd0 con 0x7f1fb01585d0 2023-05-20T01:19:56.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.846+0000 7f1fb6cbe700 1 -- 172.21.15.16:0/1659245880 --> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1fb00c76d0 con 0x7f1f9406ace0 2023-05-20T01:19:56.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.855+0000 7f1fad7fa700 1 -- 172.21.15.16:0/1659245880 <== mgr.4104 v2:172.21.15.16:6832/3817237998 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+296970 (secure 0 0 0) 0x7f1fb00c76d0 con 0x7f1f9406ace0 2023-05-20T01:19:56.861 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:19:56.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.858+0000 7f1fb6cbe700 1 -- 172.21.15.16:0/1659245880 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f1f9406ace0 msgr2=0x7f1f9406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:56.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.858+0000 7f1fb6cbe700 1 --2- 172.21.15.16:0/1659245880 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f1f9406ace0 0x7f1f9406d190 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f1fa4009930 tx=0x7f1fa4008040 comp rx=0 tx=0).stop 2023-05-20T01:19:56.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.858+0000 7f1fb6cbe700 1 -- 172.21.15.16:0/1659245880 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1fb01585d0 msgr2=0x7f1fb01f36e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:56.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.858+0000 7f1fb6cbe700 1 --2- 172.21.15.16:0/1659245880 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1fb01585d0 0x7f1fb01f36e0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c009130 tx=0x7f1f9c004750 comp rx=0 tx=0).stop 2023-05-20T01:19:56.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.858+0000 7f1fb6cbe700 1 -- 172.21.15.16:0/1659245880 shutdown_connections 2023-05-20T01:19:56.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.858+0000 7f1fb6cbe700 1 --2- 172.21.15.16:0/1659245880 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1fb015b360 0x7f1fb0159a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:56.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.858+0000 7f1fb6cbe700 1 --2- 172.21.15.16:0/1659245880 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f1f9406ace0 0x7f1f9406d190 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:56.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.858+0000 7f1fb6cbe700 1 --2- 172.21.15.16:0/1659245880 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1fb00cc9c0 0x7f1fb00c79b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:56.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.858+0000 7f1fb6cbe700 1 --2- 172.21.15.16:0/1659245880 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1fb01585d0 0x7f1fb01f36e0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:56.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.858+0000 7f1fb6cbe700 1 -- 172.21.15.16:0/1659245880 >> 172.21.15.16:0/1659245880 conn(0x7f1fb00cffb0 msgr2=0x7f1fb006d960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:19:56.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.858+0000 7f1fb6cbe700 1 -- 172.21.15.16:0/1659245880 shutdown_connections 2023-05-20T01:19:56.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:56.858+0000 7f1fb6cbe700 1 -- 172.21.15.16:0/1659245880 wait complete. 2023-05-20T01:19:56.865 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2023-05-20T01:19:56.874 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":165,"stamp":"2023-05-20T01:19:55.364291+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":77184,"kb_used_data":6784,"kb_used_omap":0,"kb_used_meta":70336,"kb_avail":754897536,"statfs":{"total":773094113280,"available":773015076864,"internally_reserved":0,"allocated":6946816,"data_stored":3957374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":72024064},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.881826"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:50.473280+0000","last_change":"2023-05-20T01:19:50.473280+0000","last_active":"2023-05-20T01:19:50.473280+0000","last_peered":"2023-05-20T01:19:50.473280+0000","last_clean":"2023-05-20T01:19:50.473280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:50.473280+0000","last_undegraded":"2023-05-20T01:19:50.473280+0000","last_fullsized":"2023-05-20T01:19:50.473280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:50.473280+0000","last_change":"2023-05-20T01:19:50.473280+0000","last_active":"2023-05-20T01:19:50.473280+0000","last_peered":"2023-05-20T01:19:50.473280+0000","last_clean":"2023-05-20T01:19:50.473280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:50.473280+0000","last_undegraded":"2023-05-20T01:19:50.473280+0000","last_fullsized":"2023-05-20T01:19:50.473280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:50.473280+0000","last_change":"2023-05-20T01:19:50.473280+0000","last_active":"2023-05-20T01:19:50.473280+0000","last_peered":"2023-05-20T01:19:50.473280+0000","last_clean":"2023-05-20T01:19:50.473280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:50.473280+0000","last_undegraded":"2023-05-20T01:19:50.473280+0000","last_fullsized":"2023-05-20T01:19:50.473280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:50.473280+0000","last_change":"2023-05-20T01:19:50.473280+0000","last_active":"2023-05-20T01:19:50.473280+0000","last_peered":"2023-05-20T01:19:50.473280+0000","last_clean":"2023-05-20T01:19:50.473280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:50.473280+0000","last_undegraded":"2023-05-20T01:19:50.473280+0000","last_fullsized":"2023-05-20T01:19:50.473280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:50.473280+0000","last_change":"2023-05-20T01:19:50.473280+0000","last_active":"2023-05-20T01:19:50.473280+0000","last_peered":"2023-05-20T01:19:50.473280+0000","last_clean":"2023-05-20T01:19:50.473280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:50.473280+0000","last_undegraded":"2023-05-20T01:19:50.473280+0000","last_fullsized":"2023-05-20T01:19:50.473280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:50.473280+0000","last_change":"2023-05-20T01:19:50.473280+0000","last_active":"2023-05-20T01:19:50.473280+0000","last_peered":"2023-05-20T01:19:50.473280+0000","last_clean":"2023-05-20T01:19:50.473280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:50.473280+0000","last_undegraded":"2023-05-20T01:19:50.473280+0000","last_fullsized":"2023-05-20T01:19:50.473280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:50.473280+0000","last_change":"2023-05-20T01:19:50.473280+0000","last_active":"2023-05-20T01:19:50.473280+0000","last_peered":"2023-05-20T01:19:50.473280+0000","last_clean":"2023-05-20T01:19:50.473280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:50.473280+0000","last_undegraded":"2023-05-20T01:19:50.473280+0000","last_fullsized":"2023-05-20T01:19:50.473280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:50.473280+0000","last_change":"2023-05-20T01:19:50.473280+0000","last_active":"2023-05-20T01:19:50.473280+0000","last_peered":"2023-05-20T01:19:50.473280+0000","last_clean":"2023-05-20T01:19:50.473280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:50.473280+0000","last_undegraded":"2023-05-20T01:19:50.473280+0000","last_fullsized":"2023-05-20T01:19:50.473280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:50.473280+0000","last_change":"2023-05-20T01:19:50.473280+0000","last_active":"2023-05-20T01:19:50.473280+0000","last_peered":"2023-05-20T01:19:50.473280+0000","last_clean":"2023-05-20T01:19:50.473280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:50.473280+0000","last_undegraded":"2023-05-20T01:19:50.473280+0000","last_fullsized":"2023-05-20T01:19:50.473280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:50.473280+0000","last_change":"2023-05-20T01:19:50.473280+0000","last_active":"2023-05-20T01:19:50.473280+0000","last_peered":"2023-05-20T01:19:50.473280+0000","last_clean":"2023-05-20T01:19:50.473280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:50.473280+0000","last_undegraded":"2023-05-20T01:19:50.473280+0000","last_fullsized":"2023-05-20T01:19:50.473280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:50.473280+0000","last_change":"2023-05-20T01:19:50.473280+0000","last_active":"2023-05-20T01:19:50.473280+0000","last_peered":"2023-05-20T01:19:50.473280+0000","last_clean":"2023-05-20T01:19:50.473280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:50.473280+0000","last_undegraded":"2023-05-20T01:19:50.473280+0000","last_fullsized":"2023-05-20T01:19:50.473280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:50.473280+0000","last_change":"2023-05-20T01:19:50.473280+0000","last_active":"2023-05-20T01:19:50.473280+0000","last_peered":"2023-05-20T01:19:50.473280+0000","last_clean":"2023-05-20T01:19:50.473280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:50.473280+0000","last_undegraded":"2023-05-20T01:19:50.473280+0000","last_fullsized":"2023-05-20T01:19:50.473280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:50.473280+0000","last_change":"2023-05-20T01:19:50.473280+0000","last_active":"2023-05-20T01:19:50.473280+0000","last_peered":"2023-05-20T01:19:50.473280+0000","last_clean":"2023-05-20T01:19:50.473280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:50.473280+0000","last_undegraded":"2023-05-20T01:19:50.473280+0000","last_fullsized":"2023-05-20T01:19:50.473280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:50.473280+0000","last_change":"2023-05-20T01:19:50.473280+0000","last_active":"2023-05-20T01:19:50.473280+0000","last_peered":"2023-05-20T01:19:50.473280+0000","last_clean":"2023-05-20T01:19:50.473280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:50.473280+0000","last_undegraded":"2023-05-20T01:19:50.473280+0000","last_fullsized":"2023-05-20T01:19:50.473280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:50.473280+0000","last_change":"2023-05-20T01:19:50.473280+0000","last_active":"2023-05-20T01:19:50.473280+0000","last_peered":"2023-05-20T01:19:50.473280+0000","last_clean":"2023-05-20T01:19:50.473280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:50.473280+0000","last_undegraded":"2023-05-20T01:19:50.473280+0000","last_fullsized":"2023-05-20T01:19:50.473280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:50.473280+0000","last_change":"2023-05-20T01:19:50.473280+0000","last_active":"2023-05-20T01:19:50.473280+0000","last_peered":"2023-05-20T01:19:50.473280+0000","last_clean":"2023-05-20T01:19:50.473280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:50.473280+0000","last_undegraded":"2023-05-20T01:19:50.473280+0000","last_fullsized":"2023-05-20T01:19:50.473280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:50.473280+0000","last_change":"2023-05-20T01:19:50.473280+0000","last_active":"2023-05-20T01:19:50.473280+0000","last_peered":"2023-05-20T01:19:50.473280+0000","last_clean":"2023-05-20T01:19:50.473280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:50.473280+0000","last_undegraded":"2023-05-20T01:19:50.473280+0000","last_fullsized":"2023-05-20T01:19:50.473280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:50.473280+0000","last_change":"2023-05-20T01:19:50.473280+0000","last_active":"2023-05-20T01:19:50.473280+0000","last_peered":"2023-05-20T01:19:50.473280+0000","last_clean":"2023-05-20T01:19:50.473280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:50.473280+0000","last_undegraded":"2023-05-20T01:19:50.473280+0000","last_fullsized":"2023-05-20T01:19:50.473280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:50.473280+0000","last_change":"2023-05-20T01:19:50.473280+0000","last_active":"2023-05-20T01:19:50.473280+0000","last_peered":"2023-05-20T01:19:50.473280+0000","last_clean":"2023-05-20T01:19:50.473280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:50.473280+0000","last_undegraded":"2023-05-20T01:19:50.473280+0000","last_fullsized":"2023-05-20T01:19:50.473280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:50.473280+0000","last_change":"2023-05-20T01:19:50.473280+0000","last_active":"2023-05-20T01:19:50.473280+0000","last_peered":"2023-05-20T01:19:50.473280+0000","last_clean":"2023-05-20T01:19:50.473280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:50.473280+0000","last_undegraded":"2023-05-20T01:19:50.473280+0000","last_fullsized":"2023-05-20T01:19:50.473280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:50.473280+0000","last_change":"2023-05-20T01:19:50.473280+0000","last_active":"2023-05-20T01:19:50.473280+0000","last_peered":"2023-05-20T01:19:50.473280+0000","last_clean":"2023-05-20T01:19:50.473280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:50.473280+0000","last_undegraded":"2023-05-20T01:19:50.473280+0000","last_fullsized":"2023-05-20T01:19:50.473280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:50.473280+0000","last_change":"2023-05-20T01:19:50.473280+0000","last_active":"2023-05-20T01:19:50.473280+0000","last_peered":"2023-05-20T01:19:50.473280+0000","last_clean":"2023-05-20T01:19:50.473280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:50.473280+0000","last_undegraded":"2023-05-20T01:19:50.473280+0000","last_fullsized":"2023-05-20T01:19:50.473280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:50.473280+0000","last_change":"2023-05-20T01:19:50.473280+0000","last_active":"2023-05-20T01:19:50.473280+0000","last_peered":"2023-05-20T01:19:50.473280+0000","last_clean":"2023-05-20T01:19:50.473280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:50.473280+0000","last_undegraded":"2023-05-20T01:19:50.473280+0000","last_fullsized":"2023-05-20T01:19:50.473280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:50.473280+0000","last_change":"2023-05-20T01:19:50.473280+0000","last_active":"2023-05-20T01:19:50.473280+0000","last_peered":"2023-05-20T01:19:50.473280+0000","last_clean":"2023-05-20T01:19:50.473280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:50.473280+0000","last_undegraded":"2023-05-20T01:19:50.473280+0000","last_fullsized":"2023-05-20T01:19:50.473280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:50.473280+0000","last_change":"2023-05-20T01:19:50.473280+0000","last_active":"2023-05-20T01:19:50.473280+0000","last_peered":"2023-05-20T01:19:50.473280+0000","last_clean":"2023-05-20T01:19:50.473280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:50.473280+0000","last_undegraded":"2023-05-20T01:19:50.473280+0000","last_fullsized":"2023-05-20T01:19:50.473280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-20T01:19:49.160921+0000","last_change":"2023-05-20T01:16:23.289068+0000","last_active":"2023-05-20T01:19:49.160921+0000","last_peered":"2023-05-20T01:19:49.160921+0000","last_clean":"2023-05-20T01:19:49.160921+0000","last_became_active":"2023-05-20T01:16:23.288781+0000","last_became_peered":"2023-05-20T01:16:23.288781+0000","last_unstale":"2023-05-20T01:19:49.160921+0000","last_undegraded":"2023-05-20T01:19:49.160921+0000","last_fullsized":"2023-05-20T01:19:49.160921+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:22:31.777331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:50.473280+0000","last_change":"2023-05-20T01:19:50.473280+0000","last_active":"2023-05-20T01:19:50.473280+0000","last_peered":"2023-05-20T01:19:50.473280+0000","last_clean":"2023-05-20T01:19:50.473280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:50.473280+0000","last_undegraded":"2023-05-20T01:19:50.473280+0000","last_fullsized":"2023-05-20T01:19:50.473280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":98,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-20T01:19:47.623794+0000","last_change":"2023-05-20T01:16:22.829324+0000","last_active":"2023-05-20T01:19:47.623794+0000","last_peered":"2023-05-20T01:19:47.623794+0000","last_clean":"2023-05-20T01:19:47.623794+0000","last_became_active":"2023-05-20T01:16:22.828964+0000","last_became_peered":"2023-05-20T01:16:22.828964+0000","last_unstale":"2023-05-20T01:19:47.623794+0000","last_undegraded":"2023-05-20T01:19:47.623794+0000","last_fullsized":"2023-05-20T01:19:47.623794+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:37:06.370724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:50.473280+0000","last_change":"2023-05-20T01:19:50.473280+0000","last_active":"2023-05-20T01:19:50.473280+0000","last_peered":"2023-05-20T01:19:50.473280+0000","last_clean":"2023-05-20T01:19:50.473280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:50.473280+0000","last_undegraded":"2023-05-20T01:19:50.473280+0000","last_fullsized":"2023-05-20T01:19:50.473280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":90,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-20T01:19:48.831080+0000","last_change":"2023-05-20T01:16:22.834754+0000","last_active":"2023-05-20T01:19:48.831080+0000","last_peered":"2023-05-20T01:19:48.831080+0000","last_clean":"2023-05-20T01:19:48.831080+0000","last_became_active":"2023-05-20T01:16:22.834437+0000","last_became_peered":"2023-05-20T01:16:22.834437+0000","last_unstale":"2023-05-20T01:19:48.831080+0000","last_undegraded":"2023-05-20T01:19:48.831080+0000","last_fullsized":"2023-05-20T01:19:48.831080+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:11:59.543234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":230,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-20T01:19:49.160859+0000","last_change":"2023-05-20T01:16:22.834920+0000","last_active":"2023-05-20T01:19:49.160859+0000","last_peered":"2023-05-20T01:19:49.160859+0000","last_clean":"2023-05-20T01:19:49.160859+0000","last_became_active":"2023-05-20T01:16:22.833924+0000","last_became_peered":"2023-05-20T01:16:22.833924+0000","last_unstale":"2023-05-20T01:19:49.160859+0000","last_undegraded":"2023-05-20T01:19:49.160859+0000","last_fullsized":"2023-05-20T01:19:49.160859+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:08:49.674910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:50.473280+0000","last_change":"2023-05-20T01:19:50.473280+0000","last_active":"2023-05-20T01:19:50.473280+0000","last_peered":"2023-05-20T01:19:50.473280+0000","last_clean":"2023-05-20T01:19:50.473280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:50.473280+0000","last_undegraded":"2023-05-20T01:19:50.473280+0000","last_fullsized":"2023-05-20T01:19:50.473280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":80,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-20T01:19:49.160858+0000","last_change":"2023-05-20T01:16:23.290259+0000","last_active":"2023-05-20T01:19:49.160858+0000","last_peered":"2023-05-20T01:19:49.160858+0000","last_clean":"2023-05-20T01:19:49.160858+0000","last_became_active":"2023-05-20T01:16:23.289911+0000","last_became_peered":"2023-05-20T01:16:23.289911+0000","last_unstale":"2023-05-20T01:19:49.160858+0000","last_undegraded":"2023-05-20T01:19:49.160858+0000","last_fullsized":"2023-05-20T01:19:49.160858+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:39:38.298562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:50.473280+0000","last_change":"2023-05-20T01:19:50.473280+0000","last_active":"2023-05-20T01:19:50.473280+0000","last_peered":"2023-05-20T01:19:50.473280+0000","last_clean":"2023-05-20T01:19:50.473280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:50.473280+0000","last_undegraded":"2023-05-20T01:19:50.473280+0000","last_fullsized":"2023-05-20T01:19:50.473280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-20T01:19:49.160859+0000","last_change":"2023-05-20T01:16:23.290207+0000","last_active":"2023-05-20T01:19:49.160859+0000","last_peered":"2023-05-20T01:19:49.160859+0000","last_clean":"2023-05-20T01:19:49.160859+0000","last_became_active":"2023-05-20T01:16:23.289928+0000","last_became_peered":"2023-05-20T01:16:23.289928+0000","last_unstale":"2023-05-20T01:19:49.160859+0000","last_undegraded":"2023-05-20T01:19:49.160859+0000","last_fullsized":"2023-05-20T01:19:49.160859+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:11:42.924171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:50.473280+0000","last_change":"2023-05-20T01:19:50.473280+0000","last_active":"2023-05-20T01:19:50.473280+0000","last_peered":"2023-05-20T01:19:50.473280+0000","last_clean":"2023-05-20T01:19:50.473280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:50.473280+0000","last_undegraded":"2023-05-20T01:19:50.473280+0000","last_fullsized":"2023-05-20T01:19:50.473280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-20T01:19:47.624189+0000","last_change":"2023-05-20T01:16:23.290538+0000","last_active":"2023-05-20T01:19:47.624189+0000","last_peered":"2023-05-20T01:19:47.624189+0000","last_clean":"2023-05-20T01:19:47.624189+0000","last_became_active":"2023-05-20T01:16:23.290230+0000","last_became_peered":"2023-05-20T01:16:23.290230+0000","last_unstale":"2023-05-20T01:19:47.624189+0000","last_undegraded":"2023-05-20T01:19:47.624189+0000","last_fullsized":"2023-05-20T01:19:47.624189+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:49:18.136308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:50.473280+0000","last_change":"2023-05-20T01:19:50.473280+0000","last_active":"2023-05-20T01:19:50.473280+0000","last_peered":"2023-05-20T01:19:50.473280+0000","last_clean":"2023-05-20T01:19:50.473280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:50.473280+0000","last_undegraded":"2023-05-20T01:19:50.473280+0000","last_fullsized":"2023-05-20T01:19:50.473280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":82,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-20T01:19:47.624338+0000","last_change":"2023-05-20T01:16:23.290341+0000","last_active":"2023-05-20T01:19:47.624338+0000","last_peered":"2023-05-20T01:19:47.624338+0000","last_clean":"2023-05-20T01:19:47.624338+0000","last_became_active":"2023-05-20T01:16:23.290053+0000","last_became_peered":"2023-05-20T01:16:23.290053+0000","last_unstale":"2023-05-20T01:19:47.624338+0000","last_undegraded":"2023-05-20T01:19:47.624338+0000","last_fullsized":"2023-05-20T01:19:47.624338+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:45:41.557083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:50.473280+0000","last_change":"2023-05-20T01:19:50.473280+0000","last_active":"2023-05-20T01:19:50.473280+0000","last_peered":"2023-05-20T01:19:50.473280+0000","last_clean":"2023-05-20T01:19:50.473280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:50.473280+0000","last_undegraded":"2023-05-20T01:19:50.473280+0000","last_fullsized":"2023-05-20T01:19:50.473280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":82,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-20T01:19:47.624309+0000","last_change":"2023-05-20T01:16:22.834330+0000","last_active":"2023-05-20T01:19:47.624309+0000","last_peered":"2023-05-20T01:19:47.624309+0000","last_clean":"2023-05-20T01:19:47.624309+0000","last_became_active":"2023-05-20T01:16:22.833946+0000","last_became_peered":"2023-05-20T01:16:22.833946+0000","last_unstale":"2023-05-20T01:19:47.624309+0000","last_undegraded":"2023-05-20T01:19:47.624309+0000","last_fullsized":"2023-05-20T01:19:47.624309+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:01:18.093853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574894,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10380,"kb_used_data":1284,"kb_used_omap":0,"kb_used_meta":9088,"kb_avail":94361460,"statfs":{"total":96636764160,"available":96626135040,"internally_reserved":0,"allocated":1314816,"data_stored":937688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9306112},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.52800000000000002,"15min":0.53400000000000003},"min":{"1min":0.29699999999999999,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.71799999999999997,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.56599999999999995,"15min":0.57099999999999995},"min":{"1min":0.38700000000000001,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.70099999999999996,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.498}]},{"osd":1,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.79500000000000004,"15min":0.90700000000000003},"min":{"1min":0.435,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.78600000000000003,"5min":8.8879999999999999,"15min":8.8879999999999999},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.83399999999999996,"15min":0.95099999999999996},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.73299999999999998,"5min":8.875,"15min":8.875},"last":0.378}]},{"osd":2,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.77200000000000002,"15min":0.88},"min":{"1min":0.371,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.73299999999999998,"5min":8.9120000000000008,"15min":8.9120000000000008},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.77600000000000002,"15min":0.88400000000000001},"min":{"1min":0.35699999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.751,"5min":8.8520000000000003,"15min":8.8520000000000003},"last":0.443}]},{"osd":3,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.84299999999999997,"15min":0.98299999999999998},"min":{"1min":0.35299999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.74299999999999999,"5min":10.884,"15min":10.884},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.89900000000000002,"15min":1.0349999999999999},"min":{"1min":0.433,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.72199999999999998,"5min":10.952,"15min":10.952},"last":0.51300000000000001}]},{"osd":4,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.71499999999999997,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.53200000000000003,"15min":0.52900000000000003},"min":{"1min":0.38800000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.67300000000000004,"5min":0.875,"15min":0.875},"last":0.53100000000000003}]},{"osd":5,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54300000000000004,"15min":0.52800000000000002},"min":{"1min":0.39600000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.69899999999999995,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.58199999999999996,"15min":0.57799999999999996},"min":{"1min":0.42699999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.745,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.59699999999999998}]},{"osd":6,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59299999999999997},"min":{"1min":0.47699999999999998,"5min":0.378,"15min":0.378},"max":{"1min":0.77000000000000002,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.59299999999999997,"15min":0.58499999999999996},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.83299999999999996,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.73599999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574893,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9224,"kb_used_data":704,"kb_used_omap":0,"kb_used_meta":8512,"kb_avail":94362616,"statfs":{"total":96636764160,"available":96627318784,"internally_reserved":0,"allocated":720896,"data_stored":347320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8716288},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.495,"15min":0.48699999999999999},"min":{"1min":0.34000000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.71699999999999997,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.54300000000000004,"15min":0.54100000000000004},"min":{"1min":0.36699999999999999,"5min":0.307,"15min":0.307},"max":{"1min":0.71199999999999997,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.52900000000000003}]},{"osd":1,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53500000000000003,"15min":0.53400000000000003},"min":{"1min":0.35899999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.68200000000000005,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53300000000000003,"15min":0.53200000000000003},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.78600000000000003,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.65500000000000003}]},{"osd":2,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.51700000000000002,"15min":0.51500000000000001},"min":{"1min":0.35499999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.68700000000000006,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.56299999999999994,"15min":0.56699999999999995},"min":{"1min":0.44500000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.64000000000000001}]},{"osd":3,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.75900000000000001,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.54900000000000004,"15min":0.55000000000000004},"min":{"1min":0.39400000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.747,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.54100000000000004}]},{"osd":4,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.54000000000000004,"15min":0.56000000000000005},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.71799999999999997,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.56499999999999995,"15min":0.58399999999999996},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.72199999999999998,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.55300000000000005}]},{"osd":5,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.54000000000000004,"15min":0.54800000000000004},"min":{"1min":0.373,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.70699999999999996,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.60599999999999998,"15min":0.621},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.80200000000000005,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.67200000000000004}]},{"osd":7,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.59799999999999998,"15min":0.61199999999999999},"min":{"1min":0.38600000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.77800000000000002,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.61499999999999999,"15min":0.628},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.749,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.63}]}]},{"osd":1,"up_from":13,"seq":55834574893,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10316,"kb_used_data":1284,"kb_used_omap":0,"kb_used_meta":9024,"kb_avail":94361524,"statfs":{"total":96636764160,"available":96626200576,"internally_reserved":0,"allocated":1314816,"data_stored":937688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9240576},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:19:24 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.32700000000000001,"15min":0.32900000000000001},"min":{"1min":0.193,"5min":0.193,"15min":0.193},"max":{"1min":1.194,"5min":1.194,"15min":1.194},"last":0.252},{"interface":"front","average":{"1min":0.35699999999999998,"5min":0.32000000000000001,"15min":0.316},"min":{"1min":0.20599999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.314}]},{"osd":2,"last update":"Sat May 20 01:19:24 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.33000000000000002,"15min":0.32400000000000001},"min":{"1min":0.253,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.27500000000000002},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.42199999999999999,"15min":0.42599999999999999},"min":{"1min":0.29599999999999999,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.3580000000000001,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":0.35399999999999998}]},{"osd":3,"last update":"Sat May 20 01:19:24 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.38500000000000001,"15min":0.376},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":1.619,"5min":1.619,"15min":1.619},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.39800000000000002,"15min":0.39400000000000002},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":1.8400000000000001,"5min":1.8400000000000001,"15min":1.8400000000000001},"last":0.36599999999999999}]},{"osd":4,"last update":"Sat May 20 01:19:24 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.59599999999999997,"15min":0.59099999999999997},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.59099999999999997,"15min":0.58899999999999997},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.61099999999999999}]},{"osd":5,"last update":"Sat May 20 01:19:24 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.59999999999999998,"15min":0.58899999999999997},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.61599999999999999,"15min":0.61199999999999999},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.409,"5min":1.409,"15min":1.409},"last":0.59899999999999998}]},{"osd":6,"last update":"Sat May 20 01:19:24 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.61699999999999999,"15min":0.60599999999999998},"min":{"1min":0.47099999999999997,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.575,"5min":1.575,"15min":1.575},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.60199999999999998,"15min":0.59699999999999998},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.58899999999999997}]},{"osd":7,"last update":"Sat May 20 01:19:24 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.61299999999999999,"15min":0.60899999999999999},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.496,"5min":1.496,"15min":1.496},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.65100000000000002,"15min":0.64400000000000002},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.5169999999999999,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.61899999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574893,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9412,"kb_used_data":700,"kb_used_omap":0,"kb_used_meta":8704,"kb_avail":94362428,"statfs":{"total":96636764160,"available":96627126272,"internally_reserved":0,"allocated":716800,"data_stored":346359,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8912896},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.31,"5min":0.32500000000000001,"15min":0.33300000000000002},"min":{"1min":0.188,"5min":0.188,"15min":0.188},"max":{"1min":0.47599999999999998,"5min":0.56599999999999995,"15min":0.56599999999999995},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.33400000000000002,"5min":0.35399999999999998,"15min":0.36799999999999999},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.53400000000000003,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.33400000000000002}]},{"osd":2,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.34300000000000003,"15min":0.34100000000000003},"min":{"1min":0.19400000000000001,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.39600000000000002,"15min":0.39900000000000002},"min":{"1min":0.29399999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.67000000000000004,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.378}]},{"osd":3,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.36799999999999999,"15min":0.376},"min":{"1min":0.26100000000000001,"5min":0.245,"15min":0.245},"max":{"1min":0.51400000000000001,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.375,"5min":0.372,"15min":0.372},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.50600000000000001,"5min":0.60099999999999998,"15min":0.60099999999999998},"last":0.436}]},{"osd":4,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.54200000000000004,"15min":0.54000000000000004},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.76000000000000001,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.56599999999999995,"15min":0.56399999999999995},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.72299999999999998,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.495}]},{"osd":5,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.57299999999999995,"15min":0.57699999999999996},"min":{"1min":0.40300000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.68799999999999994,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56799999999999995,"15min":0.56499999999999995},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.78900000000000003,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.56200000000000006}]},{"osd":6,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.57199999999999995,"15min":0.57399999999999995},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.69799999999999995,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.63},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.61799999999999999,"15min":0.623},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.68500000000000005}]},{"osd":7,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.59499999999999997,"15min":0.59099999999999997},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.78300000000000003,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.61799999999999999,"15min":0.624},"min":{"1min":0.43099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.66300000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574893,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9416,"kb_used_data":704,"kb_used_omap":0,"kb_used_meta":8704,"kb_avail":94362424,"statfs":{"total":96636764160,"available":96627122176,"internally_reserved":0,"allocated":720896,"data_stored":347320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8912896},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.32000000000000001,"5min":0.33900000000000002,"15min":0.35299999999999998},"min":{"1min":0.20100000000000001,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.58999999999999997,"5min":1.038,"15min":1.038},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.35699999999999998,"5min":0.35799999999999998,"15min":0.36699999999999999},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.52900000000000003,"5min":1.079,"15min":1.079},"last":0.45200000000000001}]},{"osd":1,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.39600000000000002,"15min":0.42099999999999999},"min":{"1min":0.26000000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.50900000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.44},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.436,"15min":0.45500000000000002},"min":{"1min":0.28399999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.64500000000000002,"5min":1.5189999999999999,"15min":1.5189999999999999},"last":0.49299999999999999}]},{"osd":3,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.39700000000000002,"15min":0.41499999999999998},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.55000000000000004,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.45000000000000001,"15min":0.46999999999999997},"min":{"1min":0.32600000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.63100000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.51300000000000001}]},{"osd":4,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.58399999999999996,"15min":0.58999999999999997},"min":{"1min":0.42699999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.86499999999999999,"5min":1.319,"15min":1.319},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.625,"15min":0.626},"min":{"1min":0.48699999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.83899999999999997,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.64100000000000001}]},{"osd":5,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.64000000000000001,"15min":0.65500000000000003},"min":{"1min":0.44800000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.80100000000000005,"5min":1.758,"15min":1.758},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.625,"5min":0.65000000000000002,"15min":0.67000000000000004},"min":{"1min":0.48099999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.80800000000000005,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.80600000000000005}]},{"osd":6,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65100000000000002,"15min":0.65700000000000003},"min":{"1min":0.45500000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.88,"5min":1.665,"15min":1.665},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.626,"5min":0.64800000000000002,"15min":0.65900000000000003},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.86599999999999999,"5min":1.819,"15min":1.819},"last":0.70299999999999996}]},{"osd":7,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.68000000000000005,"15min":0.69199999999999995},"min":{"1min":0.45500000000000002,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.83199999999999996,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.66600000000000004,"15min":0.68400000000000005},"min":{"1min":0.49299999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.89300000000000002,"5min":1.8460000000000001,"15min":1.8460000000000001},"last":0.81899999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574893,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9480,"kb_used_data":704,"kb_used_omap":0,"kb_used_meta":8768,"kb_avail":94362360,"statfs":{"total":96636764160,"available":96627056640,"internally_reserved":0,"allocated":720896,"data_stored":347320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8978432},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:19:23 2023","interfaces":[{"interface":"back","average":{"1min":0.33100000000000002,"5min":0.318,"15min":0.32400000000000001},"min":{"1min":0.219,"5min":0.192,"15min":0.192},"max":{"1min":0.46100000000000002,"5min":0.504,"15min":0.504},"last":0.29799999999999999},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.42699999999999999,"15min":0.47599999999999998},"min":{"1min":0.247,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.52800000000000002,"5min":3.1890000000000001,"15min":3.1890000000000001},"last":0.317}]},{"osd":1,"last update":"Sat May 20 01:19:23 2023","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.42099999999999999,"15min":0.46700000000000003},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.501,"5min":3.4260000000000002,"15min":3.4260000000000002},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.36199999999999999,"15min":0.36399999999999999},"min":{"1min":0.26100000000000001,"5min":0.221,"15min":0.221},"max":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"last":0.372}]},{"osd":2,"last update":"Sat May 20 01:19:23 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.39800000000000002,"15min":0.41299999999999998},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.62,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.41999999999999998,"15min":0.44},"min":{"1min":0.28699999999999998,"5min":0.253,"15min":0.253},"max":{"1min":0.58899999999999997,"5min":1.413,"15min":1.413},"last":0.39600000000000002}]},{"osd":4,"last update":"Sat May 20 01:19:23 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.59599999999999997,"15min":0.60799999999999998},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.80100000000000005,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.68300000000000005,"15min":0.72499999999999998},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.83599999999999997,"5min":3.4660000000000002,"15min":3.4660000000000002},"last":0.434}]},{"osd":5,"last update":"Sat May 20 01:19:23 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.69399999999999995,"15min":0.73999999999999999},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.77900000000000003,"5min":3.5379999999999998,"15min":3.5379999999999998},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.63300000000000001,"15min":0.65300000000000002},"min":{"1min":0.42599999999999999,"5min":0.374,"15min":0.374},"max":{"1min":0.81699999999999995,"5min":1.385,"15min":1.385},"last":0.58599999999999997}]},{"osd":6,"last update":"Sat May 20 01:19:23 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.63800000000000001,"15min":0.65800000000000003},"min":{"1min":0.40400000000000003,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.84299999999999997,"5min":1.347,"15min":1.347},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64200000000000002,"15min":0.65700000000000003},"min":{"1min":0.434,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.77400000000000002,"5min":1.107,"15min":1.107},"last":0.45700000000000002}]},{"osd":7,"last update":"Sat May 20 01:19:23 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.64600000000000002,"15min":0.66000000000000003},"min":{"1min":0.46600000000000003,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.83199999999999996,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.72899999999999998,"15min":0.77400000000000002},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.82199999999999995,"5min":3.5009999999999999,"15min":3.5009999999999999},"last":0.59399999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574893,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9736,"kb_used_data":704,"kb_used_omap":0,"kb_used_meta":9024,"kb_avail":94362104,"statfs":{"total":96636764160,"available":96626794496,"internally_reserved":0,"allocated":720896,"data_stored":347320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9240576},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:19:24 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.495,"15min":0.48099999999999998},"min":{"1min":0.379,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.72399999999999998,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.497,"15min":0.49299999999999999},"min":{"1min":0.35499999999999998,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.55200000000000005}]},{"osd":1,"last update":"Sat May 20 01:19:24 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.53400000000000003,"15min":0.53200000000000003},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.73299999999999998,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.622},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.54600000000000004,"15min":0.53200000000000003},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":2.0739999999999998,"5min":2.0739999999999998,"15min":2.0739999999999998},"last":0.498}]},{"osd":2,"last update":"Sat May 20 01:19:24 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.51400000000000001,"15min":0.499},"min":{"1min":0.439,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.745,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.55400000000000005,"15min":0.55500000000000005},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.751,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.60899999999999999}]},{"osd":3,"last update":"Sat May 20 01:19:24 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.53300000000000003,"15min":0.51900000000000002},"min":{"1min":0.42699999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.827,"5min":1.827,"15min":1.827},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.66700000000000004,"15min":0.70699999999999996},"min":{"1min":0.40400000000000003,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.77400000000000002,"5min":3.1749999999999998,"15min":3.1749999999999998},"last":0.59699999999999998}]},{"osd":5,"last update":"Sat May 20 01:19:24 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.47199999999999998,"15min":0.46000000000000002},"min":{"1min":0.33800000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.63100000000000001,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.49199999999999999,"15min":0.49299999999999999},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.67300000000000004,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.35399999999999998}]},{"osd":6,"last update":"Sat May 20 01:19:24 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52900000000000003,"15min":0.52500000000000002},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.64700000000000002,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.58199999999999996,"15min":0.57799999999999996},"min":{"1min":0.46200000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.65500000000000003,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.58299999999999996}]},{"osd":7,"last update":"Sat May 20 01:19:24 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.54800000000000004,"15min":0.54500000000000004},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.443},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.58599999999999997,"15min":0.58999999999999997},"min":{"1min":0.39100000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.67200000000000004,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.47099999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574894,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9220,"kb_used_data":700,"kb_used_omap":0,"kb_used_meta":8512,"kb_avail":94362620,"statfs":{"total":96636764160,"available":96627322880,"internally_reserved":0,"allocated":716800,"data_stored":346359,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8716288},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:19:26 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.68799999999999994,"15min":0.77400000000000002},"min":{"1min":0.33700000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.71899999999999997,"5min":6.3929999999999998,"15min":6.3929999999999998},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.72199999999999998,"15min":0.81200000000000006},"min":{"1min":0.434,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.745,"5min":6.3339999999999996,"15min":6.3339999999999996},"last":0.35899999999999999}]},{"osd":1,"last update":"Sat May 20 01:19:26 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.60999999999999999,"15min":0.66500000000000004},"min":{"1min":0.38100000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.73899999999999999,"5min":3.2450000000000001,"15min":3.2450000000000001},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54500000000000004,"15min":0.55500000000000005},"min":{"1min":0.41199999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.71899999999999997,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.46999999999999997}]},{"osd":2,"last update":"Sat May 20 01:19:26 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.55500000000000005,"15min":0.57599999999999996},"min":{"1min":0.34100000000000003,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.76300000000000001,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.56100000000000005,"15min":0.57899999999999996},"min":{"1min":0.375,"5min":0.314,"15min":0.314},"max":{"1min":0.72299999999999998,"5min":1.258,"15min":1.258},"last":0.55500000000000005}]},{"osd":3,"last update":"Sat May 20 01:19:26 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.751,"15min":0.85699999999999998},"min":{"1min":0.38200000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.68600000000000005,"5min":6.351,"15min":6.351},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.79300000000000004,"15min":0.89800000000000002},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.72599999999999998,"5min":6.3730000000000002,"15min":6.3730000000000002},"last":0.57899999999999996}]},{"osd":4,"last update":"Sat May 20 01:19:26 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.48599999999999999,"15min":0.48799999999999999},"min":{"1min":0.32400000000000001,"5min":0.316,"15min":0.316},"max":{"1min":0.65300000000000002,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.54800000000000004,"15min":0.56000000000000005},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.53100000000000003}]},{"osd":6,"last update":"Sat May 20 01:19:26 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.54300000000000004,"15min":0.55900000000000005},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.67600000000000005,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.56000000000000005,"15min":0.56599999999999995},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.747,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.59399999999999997}]},{"osd":7,"last update":"Sat May 20 01:19:26 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.57899999999999996,"15min":0.59199999999999997},"min":{"1min":0.36199999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.76900000000000002,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.61299999999999999,"15min":0.627},"min":{"1min":0.41399999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.73399999999999999,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.625}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-05-20T01:19:56.881 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-05-20T01:19:57.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.036+0000 7f88f1ae7700 1 Processor -- start 2023-05-20T01:19:57.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.036+0000 7f88f1ae7700 1 -- start start 2023-05-20T01:19:57.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.036+0000 7f88f1ae7700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f88ec06f450 0x7f88ec0cf890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:57.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.036+0000 7f88f1ae7700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f88ec06a510 0x7f88ec06a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:57.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.036+0000 7f88f1ae7700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88ec0cc510 con 0x7f88ec06d4b0 2023-05-20T01:19:57.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.036+0000 7f88f1ae7700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88ec06aed0 con 0x7f88ec06f450 2023-05-20T01:19:57.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.036+0000 7f88f1ae7700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88ec071ed0 con 0x7f88ec06a510 2023-05-20T01:19:57.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.037+0000 7f88eaffd700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f88ec06d4b0 0x7f88ec161080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:45420/0 (socket says 172.21.15.16:45420) 2023-05-20T01:19:57.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.037+0000 7f88eaffd700 1 -- 172.21.15.16:0/1223593017 learned_addr learned my addr 172.21.15.16:0/1223593017 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:19:57.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.037+0000 7f88eb7fe700 1 --2- 172.21.15.16:0/1223593017 >> v2:172.21.15.132:3300/0 conn(0x7f88ec06f450 0x7f88ec0cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:57.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.037+0000 7f88ebfff700 1 --2- 172.21.15.16:0/1223593017 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f88ec06a510 0x7f88ec06a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:57.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.037+0000 7f88ea7fc700 1 -- 172.21.15.16:0/1223593017 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 164151285 0 0) 0x7f88ec0cc510 con 0x7f88ec06d4b0 2023-05-20T01:19:57.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.037+0000 7f88ea7fc700 1 -- 172.21.15.16:0/1223593017 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f88d40035b0 con 0x7f88ec06d4b0 2023-05-20T01:19:57.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.037+0000 7f88ea7fc700 1 -- 172.21.15.16:0/1223593017 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2499382251 0 0) 0x7f88d40035b0 con 0x7f88ec06d4b0 2023-05-20T01:19:57.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.037+0000 7f88ea7fc700 1 -- 172.21.15.16:0/1223593017 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f88ec06a510 msgr2=0x7f88ec06a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:19:57.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.037+0000 7f88ea7fc700 1 --2- 172.21.15.16:0/1223593017 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f88ec06a510 0x7f88ec06a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:57.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.037+0000 7f88ea7fc700 1 -- 172.21.15.16:0/1223593017 >> v2:172.21.15.132:3300/0 conn(0x7f88ec06f450 msgr2=0x7f88ec0cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:19:57.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.037+0000 7f88ea7fc700 1 --2- 172.21.15.16:0/1223593017 >> v2:172.21.15.132:3300/0 conn(0x7f88ec06f450 0x7f88ec0cf890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:57.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.037+0000 7f88ea7fc700 1 -- 172.21.15.16:0/1223593017 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88ec0721b0 con 0x7f88ec06d4b0 2023-05-20T01:19:57.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.037+0000 7f88ea7fc700 1 -- 172.21.15.16:0/1223593017 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7f88e0001670 con 0x7f88ec06d4b0 2023-05-20T01:19:57.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.037+0000 7f88ea7fc700 1 -- 172.21.15.16:0/1223593017 >> v1:172.21.15.16:6789/0 conn(0x7f88ec06d4b0 legacy=0x7f88ec161080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:57.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.037+0000 7f88ea7fc700 1 --2- 172.21.15.16:0/1223593017 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f88ec06a510 0x7f88d4003a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:57.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.037+0000 7f88ea7fc700 1 --2- 172.21.15.16:0/1223593017 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f88d4003ff0 0x7f88d4004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:57.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.037+0000 7f88ea7fc700 1 --2- 172.21.15.16:0/1223593017 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88d4004d90 0x7f88d4005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:57.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.037+0000 7f88ea7fc700 1 -- 172.21.15.16:0/1223593017 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88ec06aed0 con 0x7f88d4004d90 2023-05-20T01:19:57.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.037+0000 7f88ea7fc700 1 -- 172.21.15.16:0/1223593017 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88ec071ed0 con 0x7f88d4003ff0 2023-05-20T01:19:57.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.037+0000 7f88ea7fc700 1 -- 172.21.15.16:0/1223593017 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88d400bac0 con 0x7f88ec06a510 2023-05-20T01:19:57.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.038+0000 7f88ebfff700 1 --2- 172.21.15.16:0/1223593017 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88d4004d90 0x7f88d4005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:57.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.038+0000 7f88eb7fe700 1 --2- 172.21.15.16:0/1223593017 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f88ec06a510 0x7f88d4003a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:57.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.038+0000 7f88eaffd700 1 --2- 172.21.15.16:0/1223593017 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f88d4003ff0 0x7f88d4004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:57.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.038+0000 7f88ebfff700 1 -- 172.21.15.16:0/1223593017 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f88ec06a510 msgr2=0x7f88d4003a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:19:57.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.038+0000 7f88ebfff700 1 --2- 172.21.15.16:0/1223593017 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f88ec06a510 0x7f88d4003a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:57.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.038+0000 7f88ebfff700 1 -- 172.21.15.16:0/1223593017 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f88d4003ff0 msgr2=0x7f88d4004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:19:57.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.038+0000 7f88ebfff700 1 --2- 172.21.15.16:0/1223593017 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f88d4003ff0 0x7f88d4004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:57.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.038+0000 7f88ebfff700 1 -- 172.21.15.16:0/1223593017 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88d40035b0 con 0x7f88d4004d90 2023-05-20T01:19:57.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.038+0000 7f88ebfff700 1 --2- 172.21.15.16:0/1223593017 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88d4004d90 0x7f88d4005200 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f88e4009f90 tx=0x7f88d4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cef966cb73b53e1e server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:57.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.038+0000 7f88ea7fc700 1 -- 172.21.15.16:0/1223593017 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88e400ec30 con 0x7f88d4004d90 2023-05-20T01:19:57.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.038+0000 7f88ea7fc700 1 -- 172.21.15.16:0/1223593017 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88e400f040 con 0x7f88d4004d90 2023-05-20T01:19:57.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.038+0000 7f88ea7fc700 1 -- 172.21.15.16:0/1223593017 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88e4014480 con 0x7f88d4004d90 2023-05-20T01:19:57.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.039+0000 7f88f1ae7700 1 -- 172.21.15.16:0/1223593017 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88d4004d90 msgr2=0x7f88d4005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:57.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.039+0000 7f88f1ae7700 1 --2- 172.21.15.16:0/1223593017 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88d4004d90 0x7f88d4005200 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f88e4009f90 tx=0x7f88d4005880 comp rx=0 tx=0).stop 2023-05-20T01:19:57.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.039+0000 7f88ebfff700 1 -- 172.21.15.16:0/1223593017 reap_dead start 2023-05-20T01:19:57.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.039+0000 7f88f1ae7700 1 -- 172.21.15.16:0/1223593017 shutdown_connections 2023-05-20T01:19:57.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.039+0000 7f88f1ae7700 1 -- 172.21.15.16:0/1223593017 >> 172.21.15.16:0/1223593017 conn(0x7f88ec0cffb0 msgr2=0x7f88ec0d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:19:57.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.039+0000 7f88f1ae7700 1 -- 172.21.15.16:0/1223593017 shutdown_connections 2023-05-20T01:19:57.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.039+0000 7f88f1ae7700 1 -- 172.21.15.16:0/1223593017 wait complete. 2023-05-20T01:19:57.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.040+0000 7f88f1ae7700 1 Processor -- start 2023-05-20T01:19:57.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.040+0000 7f88f1ae7700 1 -- start start 2023-05-20T01:19:57.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.041+0000 7f88f1ae7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88d4004d90 0x7f88ec0c58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:57.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.041+0000 7f88f1ae7700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f88ec1f6af0 0x7f88ec1f61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:57.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.041+0000 7f88f1ae7700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f88ec0b52e0 0x7f88ec0b5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:57.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.041+0000 7f88f1ae7700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88ec0c4510 con 0x7f88d4004d90 2023-05-20T01:19:57.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.041+0000 7f88f1ae7700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88ec159db0 con 0x7f88ec0b52e0 2023-05-20T01:19:57.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.041+0000 7f88f1ae7700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88ec0c61f0 con 0x7f88ec1f6af0 2023-05-20T01:19:57.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.041+0000 7f88eb7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88d4004d90 0x7f88ec0c58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:57.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.041+0000 7f88eb7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88d4004d90 0x7f88ec0c58e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33902/0 (socket says 172.21.15.16:33902) 2023-05-20T01:19:57.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.041+0000 7f88eb7fe700 1 -- 172.21.15.16:0/683709266 learned_addr learned my addr 172.21.15.16:0/683709266 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:19:57.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.041+0000 7f88eb7fe700 1 -- 172.21.15.16:0/683709266 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f88ec1f6af0 msgr2=0x7f88ec1f61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:57.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.041+0000 7f88eaffd700 1 --2- 172.21.15.16:0/683709266 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f88ec1f6af0 0x7f88ec1f61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:57.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.041+0000 7f88ebfff700 1 --2- 172.21.15.16:0/683709266 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f88ec0b52e0 0x7f88ec0b5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:57.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.041+0000 7f88eb7fe700 1 --2- 172.21.15.16:0/683709266 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f88ec1f6af0 0x7f88ec1f61c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:57.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.041+0000 7f88eb7fe700 1 -- 172.21.15.16:0/683709266 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f88ec0b52e0 msgr2=0x7f88ec0b5780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:57.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.041+0000 7f88eb7fe700 1 --2- 172.21.15.16:0/683709266 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f88ec0b52e0 0x7f88ec0b5780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:57.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.041+0000 7f88eb7fe700 1 -- 172.21.15.16:0/683709266 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88ec0cbe50 con 0x7f88d4004d90 2023-05-20T01:19:57.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.041+0000 7f88ebfff700 1 --2- 172.21.15.16:0/683709266 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f88ec0b52e0 0x7f88ec0b5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:19:57.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.041+0000 7f88eb7fe700 1 --2- 172.21.15.16:0/683709266 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88d4004d90 0x7f88ec0c58e0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f88dc011ca0 tx=0x7f88dc00f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:57.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.041+0000 7f88d3fff700 1 -- 172.21.15.16:0/683709266 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88dc009740 con 0x7f88d4004d90 2023-05-20T01:19:57.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.042+0000 7f88d3fff700 1 -- 172.21.15.16:0/683709266 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88dc00fb30 con 0x7f88d4004d90 2023-05-20T01:19:57.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.042+0000 7f88d3fff700 1 -- 172.21.15.16:0/683709266 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88dc00fea0 con 0x7f88d4004d90 2023-05-20T01:19:57.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.042+0000 7f88f1ae7700 1 -- 172.21.15.16:0/683709266 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88ec0cc1a0 con 0x7f88d4004d90 2023-05-20T01:19:57.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.042+0000 7f88f1ae7700 1 -- 172.21.15.16:0/683709266 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88ec06a990 con 0x7f88d4004d90 2023-05-20T01:19:57.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.042+0000 7f88f1ae7700 1 -- 172.21.15.16:0/683709266 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88ec0747b0 con 0x7f88d4004d90 2023-05-20T01:19:57.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.043+0000 7f88d3fff700 1 -- 172.21.15.16:0/683709266 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f88dc014440 con 0x7f88d4004d90 2023-05-20T01:19:57.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.043+0000 7f88d3fff700 1 --2- 172.21.15.16:0/683709266 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f88d806ace0 0x7f88d806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:57.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.043+0000 7f88d3fff700 1 -- 172.21.15.16:0/683709266 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f88dc089d20 con 0x7f88d4004d90 2023-05-20T01:19:57.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.043+0000 7f88eaffd700 1 --2- 172.21.15.16:0/683709266 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f88d806ace0 0x7f88d806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:57.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.044+0000 7f88eaffd700 1 --2- 172.21.15.16:0/683709266 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f88d806ace0 0x7f88d806d190 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f88e0000fd0 tx=0x7f88e0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:57.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.048+0000 7f88d3fff700 1 -- 172.21.15.16:0/683709266 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f88dc054d20 con 0x7f88d4004d90 2023-05-20T01:19:57.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.238+0000 7f88f1ae7700 1 -- 172.21.15.16:0/683709266 --> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f88ec071c10 con 0x7f88d806ace0 2023-05-20T01:19:57.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.247+0000 7f88d3fff700 1 -- 172.21.15.16:0/683709266 <== mgr.4104 v2:172.21.15.16:6832/3817237998 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+296970 (secure 0 0 0) 0x7f88ec071c10 con 0x7f88d806ace0 2023-05-20T01:19:57.253 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:19:57.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.250+0000 7f88f1ae7700 1 -- 172.21.15.16:0/683709266 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f88d806ace0 msgr2=0x7f88d806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:57.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.250+0000 7f88f1ae7700 1 --2- 172.21.15.16:0/683709266 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f88d806ace0 0x7f88d806d190 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f88e0000fd0 tx=0x7f88e0009040 comp rx=0 tx=0).stop 2023-05-20T01:19:57.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.250+0000 7f88f1ae7700 1 -- 172.21.15.16:0/683709266 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88d4004d90 msgr2=0x7f88ec0c58e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:57.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.250+0000 7f88f1ae7700 1 --2- 172.21.15.16:0/683709266 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88d4004d90 0x7f88ec0c58e0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f88dc011ca0 tx=0x7f88dc00f520 comp rx=0 tx=0).stop 2023-05-20T01:19:57.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.250+0000 7f88f1ae7700 1 -- 172.21.15.16:0/683709266 shutdown_connections 2023-05-20T01:19:57.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.250+0000 7f88f1ae7700 1 --2- 172.21.15.16:0/683709266 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f88ec0b52e0 0x7f88ec0b5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:57.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.250+0000 7f88f1ae7700 1 --2- 172.21.15.16:0/683709266 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88d4004d90 0x7f88ec0c58e0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:57.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.250+0000 7f88f1ae7700 1 --2- 172.21.15.16:0/683709266 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f88d806ace0 0x7f88d806d190 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:57.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.250+0000 7f88f1ae7700 1 --2- 172.21.15.16:0/683709266 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f88ec1f6af0 0x7f88ec1f61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:57.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.250+0000 7f88f1ae7700 1 -- 172.21.15.16:0/683709266 >> 172.21.15.16:0/683709266 conn(0x7f88ec0cffb0 msgr2=0x7f88ec0cc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:19:57.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.250+0000 7f88f1ae7700 1 -- 172.21.15.16:0/683709266 shutdown_connections 2023-05-20T01:19:57.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.250+0000 7f88f1ae7700 1 -- 172.21.15.16:0/683709266 wait complete. 2023-05-20T01:19:57.257 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2023-05-20T01:19:57.266 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":165,"stamp":"2023-05-20T01:19:55.364291+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":77184,"kb_used_data":6784,"kb_used_omap":0,"kb_used_meta":70336,"kb_avail":754897536,"statfs":{"total":773094113280,"available":773015076864,"internally_reserved":0,"allocated":6946816,"data_stored":3957374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":72024064},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.881826"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:50.473280+0000","last_change":"2023-05-20T01:19:50.473280+0000","last_active":"2023-05-20T01:19:50.473280+0000","last_peered":"2023-05-20T01:19:50.473280+0000","last_clean":"2023-05-20T01:19:50.473280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:50.473280+0000","last_undegraded":"2023-05-20T01:19:50.473280+0000","last_fullsized":"2023-05-20T01:19:50.473280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:50.473280+0000","last_change":"2023-05-20T01:19:50.473280+0000","last_active":"2023-05-20T01:19:50.473280+0000","last_peered":"2023-05-20T01:19:50.473280+0000","last_clean":"2023-05-20T01:19:50.473280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:50.473280+0000","last_undegraded":"2023-05-20T01:19:50.473280+0000","last_fullsized":"2023-05-20T01:19:50.473280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:50.473280+0000","last_change":"2023-05-20T01:19:50.473280+0000","last_active":"2023-05-20T01:19:50.473280+0000","last_peered":"2023-05-20T01:19:50.473280+0000","last_clean":"2023-05-20T01:19:50.473280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:50.473280+0000","last_undegraded":"2023-05-20T01:19:50.473280+0000","last_fullsized":"2023-05-20T01:19:50.473280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:50.473280+0000","last_change":"2023-05-20T01:19:50.473280+0000","last_active":"2023-05-20T01:19:50.473280+0000","last_peered":"2023-05-20T01:19:50.473280+0000","last_clean":"2023-05-20T01:19:50.473280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:50.473280+0000","last_undegraded":"2023-05-20T01:19:50.473280+0000","last_fullsized":"2023-05-20T01:19:50.473280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:50.473280+0000","last_change":"2023-05-20T01:19:50.473280+0000","last_active":"2023-05-20T01:19:50.473280+0000","last_peered":"2023-05-20T01:19:50.473280+0000","last_clean":"2023-05-20T01:19:50.473280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:50.473280+0000","last_undegraded":"2023-05-20T01:19:50.473280+0000","last_fullsized":"2023-05-20T01:19:50.473280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:50.473280+0000","last_change":"2023-05-20T01:19:50.473280+0000","last_active":"2023-05-20T01:19:50.473280+0000","last_peered":"2023-05-20T01:19:50.473280+0000","last_clean":"2023-05-20T01:19:50.473280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:50.473280+0000","last_undegraded":"2023-05-20T01:19:50.473280+0000","last_fullsized":"2023-05-20T01:19:50.473280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:50.473280+0000","last_change":"2023-05-20T01:19:50.473280+0000","last_active":"2023-05-20T01:19:50.473280+0000","last_peered":"2023-05-20T01:19:50.473280+0000","last_clean":"2023-05-20T01:19:50.473280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:50.473280+0000","last_undegraded":"2023-05-20T01:19:50.473280+0000","last_fullsized":"2023-05-20T01:19:50.473280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:50.473280+0000","last_change":"2023-05-20T01:19:50.473280+0000","last_active":"2023-05-20T01:19:50.473280+0000","last_peered":"2023-05-20T01:19:50.473280+0000","last_clean":"2023-05-20T01:19:50.473280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:50.473280+0000","last_undegraded":"2023-05-20T01:19:50.473280+0000","last_fullsized":"2023-05-20T01:19:50.473280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:50.473280+0000","last_change":"2023-05-20T01:19:50.473280+0000","last_active":"2023-05-20T01:19:50.473280+0000","last_peered":"2023-05-20T01:19:50.473280+0000","last_clean":"2023-05-20T01:19:50.473280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:50.473280+0000","last_undegraded":"2023-05-20T01:19:50.473280+0000","last_fullsized":"2023-05-20T01:19:50.473280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:50.473280+0000","last_change":"2023-05-20T01:19:50.473280+0000","last_active":"2023-05-20T01:19:50.473280+0000","last_peered":"2023-05-20T01:19:50.473280+0000","last_clean":"2023-05-20T01:19:50.473280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:50.473280+0000","last_undegraded":"2023-05-20T01:19:50.473280+0000","last_fullsized":"2023-05-20T01:19:50.473280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:50.473280+0000","last_change":"2023-05-20T01:19:50.473280+0000","last_active":"2023-05-20T01:19:50.473280+0000","last_peered":"2023-05-20T01:19:50.473280+0000","last_clean":"2023-05-20T01:19:50.473280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:50.473280+0000","last_undegraded":"2023-05-20T01:19:50.473280+0000","last_fullsized":"2023-05-20T01:19:50.473280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:50.473280+0000","last_change":"2023-05-20T01:19:50.473280+0000","last_active":"2023-05-20T01:19:50.473280+0000","last_peered":"2023-05-20T01:19:50.473280+0000","last_clean":"2023-05-20T01:19:50.473280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:50.473280+0000","last_undegraded":"2023-05-20T01:19:50.473280+0000","last_fullsized":"2023-05-20T01:19:50.473280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:50.473280+0000","last_change":"2023-05-20T01:19:50.473280+0000","last_active":"2023-05-20T01:19:50.473280+0000","last_peered":"2023-05-20T01:19:50.473280+0000","last_clean":"2023-05-20T01:19:50.473280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:50.473280+0000","last_undegraded":"2023-05-20T01:19:50.473280+0000","last_fullsized":"2023-05-20T01:19:50.473280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:50.473280+0000","last_change":"2023-05-20T01:19:50.473280+0000","last_active":"2023-05-20T01:19:50.473280+0000","last_peered":"2023-05-20T01:19:50.473280+0000","last_clean":"2023-05-20T01:19:50.473280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:50.473280+0000","last_undegraded":"2023-05-20T01:19:50.473280+0000","last_fullsized":"2023-05-20T01:19:50.473280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:50.473280+0000","last_change":"2023-05-20T01:19:50.473280+0000","last_active":"2023-05-20T01:19:50.473280+0000","last_peered":"2023-05-20T01:19:50.473280+0000","last_clean":"2023-05-20T01:19:50.473280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:50.473280+0000","last_undegraded":"2023-05-20T01:19:50.473280+0000","last_fullsized":"2023-05-20T01:19:50.473280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:50.473280+0000","last_change":"2023-05-20T01:19:50.473280+0000","last_active":"2023-05-20T01:19:50.473280+0000","last_peered":"2023-05-20T01:19:50.473280+0000","last_clean":"2023-05-20T01:19:50.473280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:50.473280+0000","last_undegraded":"2023-05-20T01:19:50.473280+0000","last_fullsized":"2023-05-20T01:19:50.473280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:50.473280+0000","last_change":"2023-05-20T01:19:50.473280+0000","last_active":"2023-05-20T01:19:50.473280+0000","last_peered":"2023-05-20T01:19:50.473280+0000","last_clean":"2023-05-20T01:19:50.473280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:50.473280+0000","last_undegraded":"2023-05-20T01:19:50.473280+0000","last_fullsized":"2023-05-20T01:19:50.473280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:50.473280+0000","last_change":"2023-05-20T01:19:50.473280+0000","last_active":"2023-05-20T01:19:50.473280+0000","last_peered":"2023-05-20T01:19:50.473280+0000","last_clean":"2023-05-20T01:19:50.473280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:50.473280+0000","last_undegraded":"2023-05-20T01:19:50.473280+0000","last_fullsized":"2023-05-20T01:19:50.473280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:50.473280+0000","last_change":"2023-05-20T01:19:50.473280+0000","last_active":"2023-05-20T01:19:50.473280+0000","last_peered":"2023-05-20T01:19:50.473280+0000","last_clean":"2023-05-20T01:19:50.473280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:50.473280+0000","last_undegraded":"2023-05-20T01:19:50.473280+0000","last_fullsized":"2023-05-20T01:19:50.473280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:50.473280+0000","last_change":"2023-05-20T01:19:50.473280+0000","last_active":"2023-05-20T01:19:50.473280+0000","last_peered":"2023-05-20T01:19:50.473280+0000","last_clean":"2023-05-20T01:19:50.473280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:50.473280+0000","last_undegraded":"2023-05-20T01:19:50.473280+0000","last_fullsized":"2023-05-20T01:19:50.473280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:50.473280+0000","last_change":"2023-05-20T01:19:50.473280+0000","last_active":"2023-05-20T01:19:50.473280+0000","last_peered":"2023-05-20T01:19:50.473280+0000","last_clean":"2023-05-20T01:19:50.473280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:50.473280+0000","last_undegraded":"2023-05-20T01:19:50.473280+0000","last_fullsized":"2023-05-20T01:19:50.473280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:50.473280+0000","last_change":"2023-05-20T01:19:50.473280+0000","last_active":"2023-05-20T01:19:50.473280+0000","last_peered":"2023-05-20T01:19:50.473280+0000","last_clean":"2023-05-20T01:19:50.473280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:50.473280+0000","last_undegraded":"2023-05-20T01:19:50.473280+0000","last_fullsized":"2023-05-20T01:19:50.473280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:50.473280+0000","last_change":"2023-05-20T01:19:50.473280+0000","last_active":"2023-05-20T01:19:50.473280+0000","last_peered":"2023-05-20T01:19:50.473280+0000","last_clean":"2023-05-20T01:19:50.473280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:50.473280+0000","last_undegraded":"2023-05-20T01:19:50.473280+0000","last_fullsized":"2023-05-20T01:19:50.473280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:50.473280+0000","last_change":"2023-05-20T01:19:50.473280+0000","last_active":"2023-05-20T01:19:50.473280+0000","last_peered":"2023-05-20T01:19:50.473280+0000","last_clean":"2023-05-20T01:19:50.473280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:50.473280+0000","last_undegraded":"2023-05-20T01:19:50.473280+0000","last_fullsized":"2023-05-20T01:19:50.473280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:50.473280+0000","last_change":"2023-05-20T01:19:50.473280+0000","last_active":"2023-05-20T01:19:50.473280+0000","last_peered":"2023-05-20T01:19:50.473280+0000","last_clean":"2023-05-20T01:19:50.473280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:50.473280+0000","last_undegraded":"2023-05-20T01:19:50.473280+0000","last_fullsized":"2023-05-20T01:19:50.473280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-20T01:19:49.160921+0000","last_change":"2023-05-20T01:16:23.289068+0000","last_active":"2023-05-20T01:19:49.160921+0000","last_peered":"2023-05-20T01:19:49.160921+0000","last_clean":"2023-05-20T01:19:49.160921+0000","last_became_active":"2023-05-20T01:16:23.288781+0000","last_became_peered":"2023-05-20T01:16:23.288781+0000","last_unstale":"2023-05-20T01:19:49.160921+0000","last_undegraded":"2023-05-20T01:19:49.160921+0000","last_fullsized":"2023-05-20T01:19:49.160921+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:22:31.777331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:50.473280+0000","last_change":"2023-05-20T01:19:50.473280+0000","last_active":"2023-05-20T01:19:50.473280+0000","last_peered":"2023-05-20T01:19:50.473280+0000","last_clean":"2023-05-20T01:19:50.473280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:50.473280+0000","last_undegraded":"2023-05-20T01:19:50.473280+0000","last_fullsized":"2023-05-20T01:19:50.473280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":98,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-20T01:19:47.623794+0000","last_change":"2023-05-20T01:16:22.829324+0000","last_active":"2023-05-20T01:19:47.623794+0000","last_peered":"2023-05-20T01:19:47.623794+0000","last_clean":"2023-05-20T01:19:47.623794+0000","last_became_active":"2023-05-20T01:16:22.828964+0000","last_became_peered":"2023-05-20T01:16:22.828964+0000","last_unstale":"2023-05-20T01:19:47.623794+0000","last_undegraded":"2023-05-20T01:19:47.623794+0000","last_fullsized":"2023-05-20T01:19:47.623794+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:37:06.370724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:50.473280+0000","last_change":"2023-05-20T01:19:50.473280+0000","last_active":"2023-05-20T01:19:50.473280+0000","last_peered":"2023-05-20T01:19:50.473280+0000","last_clean":"2023-05-20T01:19:50.473280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:50.473280+0000","last_undegraded":"2023-05-20T01:19:50.473280+0000","last_fullsized":"2023-05-20T01:19:50.473280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":90,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-20T01:19:48.831080+0000","last_change":"2023-05-20T01:16:22.834754+0000","last_active":"2023-05-20T01:19:48.831080+0000","last_peered":"2023-05-20T01:19:48.831080+0000","last_clean":"2023-05-20T01:19:48.831080+0000","last_became_active":"2023-05-20T01:16:22.834437+0000","last_became_peered":"2023-05-20T01:16:22.834437+0000","last_unstale":"2023-05-20T01:19:48.831080+0000","last_undegraded":"2023-05-20T01:19:48.831080+0000","last_fullsized":"2023-05-20T01:19:48.831080+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:11:59.543234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":230,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-20T01:19:49.160859+0000","last_change":"2023-05-20T01:16:22.834920+0000","last_active":"2023-05-20T01:19:49.160859+0000","last_peered":"2023-05-20T01:19:49.160859+0000","last_clean":"2023-05-20T01:19:49.160859+0000","last_became_active":"2023-05-20T01:16:22.833924+0000","last_became_peered":"2023-05-20T01:16:22.833924+0000","last_unstale":"2023-05-20T01:19:49.160859+0000","last_undegraded":"2023-05-20T01:19:49.160859+0000","last_fullsized":"2023-05-20T01:19:49.160859+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:08:49.674910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:50.473280+0000","last_change":"2023-05-20T01:19:50.473280+0000","last_active":"2023-05-20T01:19:50.473280+0000","last_peered":"2023-05-20T01:19:50.473280+0000","last_clean":"2023-05-20T01:19:50.473280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:50.473280+0000","last_undegraded":"2023-05-20T01:19:50.473280+0000","last_fullsized":"2023-05-20T01:19:50.473280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":80,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-20T01:19:49.160858+0000","last_change":"2023-05-20T01:16:23.290259+0000","last_active":"2023-05-20T01:19:49.160858+0000","last_peered":"2023-05-20T01:19:49.160858+0000","last_clean":"2023-05-20T01:19:49.160858+0000","last_became_active":"2023-05-20T01:16:23.289911+0000","last_became_peered":"2023-05-20T01:16:23.289911+0000","last_unstale":"2023-05-20T01:19:49.160858+0000","last_undegraded":"2023-05-20T01:19:49.160858+0000","last_fullsized":"2023-05-20T01:19:49.160858+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:39:38.298562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:50.473280+0000","last_change":"2023-05-20T01:19:50.473280+0000","last_active":"2023-05-20T01:19:50.473280+0000","last_peered":"2023-05-20T01:19:50.473280+0000","last_clean":"2023-05-20T01:19:50.473280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:50.473280+0000","last_undegraded":"2023-05-20T01:19:50.473280+0000","last_fullsized":"2023-05-20T01:19:50.473280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-20T01:19:49.160859+0000","last_change":"2023-05-20T01:16:23.290207+0000","last_active":"2023-05-20T01:19:49.160859+0000","last_peered":"2023-05-20T01:19:49.160859+0000","last_clean":"2023-05-20T01:19:49.160859+0000","last_became_active":"2023-05-20T01:16:23.289928+0000","last_became_peered":"2023-05-20T01:16:23.289928+0000","last_unstale":"2023-05-20T01:19:49.160859+0000","last_undegraded":"2023-05-20T01:19:49.160859+0000","last_fullsized":"2023-05-20T01:19:49.160859+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:11:42.924171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:50.473280+0000","last_change":"2023-05-20T01:19:50.473280+0000","last_active":"2023-05-20T01:19:50.473280+0000","last_peered":"2023-05-20T01:19:50.473280+0000","last_clean":"2023-05-20T01:19:50.473280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:50.473280+0000","last_undegraded":"2023-05-20T01:19:50.473280+0000","last_fullsized":"2023-05-20T01:19:50.473280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-20T01:19:47.624189+0000","last_change":"2023-05-20T01:16:23.290538+0000","last_active":"2023-05-20T01:19:47.624189+0000","last_peered":"2023-05-20T01:19:47.624189+0000","last_clean":"2023-05-20T01:19:47.624189+0000","last_became_active":"2023-05-20T01:16:23.290230+0000","last_became_peered":"2023-05-20T01:16:23.290230+0000","last_unstale":"2023-05-20T01:19:47.624189+0000","last_undegraded":"2023-05-20T01:19:47.624189+0000","last_fullsized":"2023-05-20T01:19:47.624189+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:49:18.136308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:50.473280+0000","last_change":"2023-05-20T01:19:50.473280+0000","last_active":"2023-05-20T01:19:50.473280+0000","last_peered":"2023-05-20T01:19:50.473280+0000","last_clean":"2023-05-20T01:19:50.473280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:50.473280+0000","last_undegraded":"2023-05-20T01:19:50.473280+0000","last_fullsized":"2023-05-20T01:19:50.473280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":82,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-20T01:19:47.624338+0000","last_change":"2023-05-20T01:16:23.290341+0000","last_active":"2023-05-20T01:19:47.624338+0000","last_peered":"2023-05-20T01:19:47.624338+0000","last_clean":"2023-05-20T01:19:47.624338+0000","last_became_active":"2023-05-20T01:16:23.290053+0000","last_became_peered":"2023-05-20T01:16:23.290053+0000","last_unstale":"2023-05-20T01:19:47.624338+0000","last_undegraded":"2023-05-20T01:19:47.624338+0000","last_fullsized":"2023-05-20T01:19:47.624338+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:45:41.557083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:51.482477+0000","last_change":"2023-05-20T01:19:51.482477+0000","last_active":"2023-05-20T01:19:51.482477+0000","last_peered":"2023-05-20T01:19:51.482477+0000","last_clean":"2023-05-20T01:19:51.482477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:51.482477+0000","last_undegraded":"2023-05-20T01:19:51.482477+0000","last_fullsized":"2023-05-20T01:19:51.482477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:19:50.473280+0000","last_change":"2023-05-20T01:19:50.473280+0000","last_active":"2023-05-20T01:19:50.473280+0000","last_peered":"2023-05-20T01:19:50.473280+0000","last_clean":"2023-05-20T01:19:50.473280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:19:50.473280+0000","last_undegraded":"2023-05-20T01:19:50.473280+0000","last_fullsized":"2023-05-20T01:19:50.473280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":82,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-05-20T01:19:47.624309+0000","last_change":"2023-05-20T01:16:22.834330+0000","last_active":"2023-05-20T01:19:47.624309+0000","last_peered":"2023-05-20T01:19:47.624309+0000","last_clean":"2023-05-20T01:19:47.624309+0000","last_became_active":"2023-05-20T01:16:22.833946+0000","last_became_peered":"2023-05-20T01:16:22.833946+0000","last_unstale":"2023-05-20T01:19:47.624309+0000","last_undegraded":"2023-05-20T01:19:47.624309+0000","last_fullsized":"2023-05-20T01:19:47.624309+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:01:18.093853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574894,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10380,"kb_used_data":1284,"kb_used_omap":0,"kb_used_meta":9088,"kb_avail":94361460,"statfs":{"total":96636764160,"available":96626135040,"internally_reserved":0,"allocated":1314816,"data_stored":937688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9306112},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.52800000000000002,"15min":0.53400000000000003},"min":{"1min":0.29699999999999999,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.71799999999999997,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.56599999999999995,"15min":0.57099999999999995},"min":{"1min":0.38700000000000001,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.70099999999999996,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.498}]},{"osd":1,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.79500000000000004,"15min":0.90700000000000003},"min":{"1min":0.435,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.78600000000000003,"5min":8.8879999999999999,"15min":8.8879999999999999},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.83399999999999996,"15min":0.95099999999999996},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.73299999999999998,"5min":8.875,"15min":8.875},"last":0.378}]},{"osd":2,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.77200000000000002,"15min":0.88},"min":{"1min":0.371,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.73299999999999998,"5min":8.9120000000000008,"15min":8.9120000000000008},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.77600000000000002,"15min":0.88400000000000001},"min":{"1min":0.35699999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.751,"5min":8.8520000000000003,"15min":8.8520000000000003},"last":0.443}]},{"osd":3,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.84299999999999997,"15min":0.98299999999999998},"min":{"1min":0.35299999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.74299999999999999,"5min":10.884,"15min":10.884},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.89900000000000002,"15min":1.0349999999999999},"min":{"1min":0.433,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.72199999999999998,"5min":10.952,"15min":10.952},"last":0.51300000000000001}]},{"osd":4,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.71499999999999997,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.53200000000000003,"15min":0.52900000000000003},"min":{"1min":0.38800000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.67300000000000004,"5min":0.875,"15min":0.875},"last":0.53100000000000003}]},{"osd":5,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54300000000000004,"15min":0.52800000000000002},"min":{"1min":0.39600000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.69899999999999995,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.58199999999999996,"15min":0.57799999999999996},"min":{"1min":0.42699999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.745,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.59699999999999998}]},{"osd":6,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59299999999999997},"min":{"1min":0.47699999999999998,"5min":0.378,"15min":0.378},"max":{"1min":0.77000000000000002,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.59299999999999997,"15min":0.58499999999999996},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.83299999999999996,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.73599999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574893,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9224,"kb_used_data":704,"kb_used_omap":0,"kb_used_meta":8512,"kb_avail":94362616,"statfs":{"total":96636764160,"available":96627318784,"internally_reserved":0,"allocated":720896,"data_stored":347320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8716288},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.495,"15min":0.48699999999999999},"min":{"1min":0.34000000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.71699999999999997,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.54300000000000004,"15min":0.54100000000000004},"min":{"1min":0.36699999999999999,"5min":0.307,"15min":0.307},"max":{"1min":0.71199999999999997,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.52900000000000003}]},{"osd":1,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53500000000000003,"15min":0.53400000000000003},"min":{"1min":0.35899999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.68200000000000005,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53300000000000003,"15min":0.53200000000000003},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.78600000000000003,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.65500000000000003}]},{"osd":2,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.51700000000000002,"15min":0.51500000000000001},"min":{"1min":0.35499999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.68700000000000006,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.56299999999999994,"15min":0.56699999999999995},"min":{"1min":0.44500000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.64000000000000001}]},{"osd":3,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.75900000000000001,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.54900000000000004,"15min":0.55000000000000004},"min":{"1min":0.39400000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.747,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.54100000000000004}]},{"osd":4,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.54000000000000004,"15min":0.56000000000000005},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.71799999999999997,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.56499999999999995,"15min":0.58399999999999996},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.72199999999999998,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.55300000000000005}]},{"osd":5,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.54000000000000004,"15min":0.54800000000000004},"min":{"1min":0.373,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.70699999999999996,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.60599999999999998,"15min":0.621},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.80200000000000005,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.67200000000000004}]},{"osd":7,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.59799999999999998,"15min":0.61199999999999999},"min":{"1min":0.38600000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.77800000000000002,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.61499999999999999,"15min":0.628},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.749,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.63}]}]},{"osd":1,"up_from":13,"seq":55834574893,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10316,"kb_used_data":1284,"kb_used_omap":0,"kb_used_meta":9024,"kb_avail":94361524,"statfs":{"total":96636764160,"available":96626200576,"internally_reserved":0,"allocated":1314816,"data_stored":937688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9240576},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:19:24 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.32700000000000001,"15min":0.32900000000000001},"min":{"1min":0.193,"5min":0.193,"15min":0.193},"max":{"1min":1.194,"5min":1.194,"15min":1.194},"last":0.252},{"interface":"front","average":{"1min":0.35699999999999998,"5min":0.32000000000000001,"15min":0.316},"min":{"1min":0.20599999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.314}]},{"osd":2,"last update":"Sat May 20 01:19:24 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.33000000000000002,"15min":0.32400000000000001},"min":{"1min":0.253,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.27500000000000002},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.42199999999999999,"15min":0.42599999999999999},"min":{"1min":0.29599999999999999,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.3580000000000001,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":0.35399999999999998}]},{"osd":3,"last update":"Sat May 20 01:19:24 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.38500000000000001,"15min":0.376},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":1.619,"5min":1.619,"15min":1.619},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.39800000000000002,"15min":0.39400000000000002},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":1.8400000000000001,"5min":1.8400000000000001,"15min":1.8400000000000001},"last":0.36599999999999999}]},{"osd":4,"last update":"Sat May 20 01:19:24 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.59599999999999997,"15min":0.59099999999999997},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.59099999999999997,"15min":0.58899999999999997},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.61099999999999999}]},{"osd":5,"last update":"Sat May 20 01:19:24 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.59999999999999998,"15min":0.58899999999999997},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.61599999999999999,"15min":0.61199999999999999},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.409,"5min":1.409,"15min":1.409},"last":0.59899999999999998}]},{"osd":6,"last update":"Sat May 20 01:19:24 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.61699999999999999,"15min":0.60599999999999998},"min":{"1min":0.47099999999999997,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.575,"5min":1.575,"15min":1.575},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.60199999999999998,"15min":0.59699999999999998},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.58899999999999997}]},{"osd":7,"last update":"Sat May 20 01:19:24 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.61299999999999999,"15min":0.60899999999999999},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.496,"5min":1.496,"15min":1.496},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.65100000000000002,"15min":0.64400000000000002},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.5169999999999999,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.61899999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574893,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9412,"kb_used_data":700,"kb_used_omap":0,"kb_used_meta":8704,"kb_avail":94362428,"statfs":{"total":96636764160,"available":96627126272,"internally_reserved":0,"allocated":716800,"data_stored":346359,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8912896},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.31,"5min":0.32500000000000001,"15min":0.33300000000000002},"min":{"1min":0.188,"5min":0.188,"15min":0.188},"max":{"1min":0.47599999999999998,"5min":0.56599999999999995,"15min":0.56599999999999995},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.33400000000000002,"5min":0.35399999999999998,"15min":0.36799999999999999},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.53400000000000003,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.33400000000000002}]},{"osd":2,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.34300000000000003,"15min":0.34100000000000003},"min":{"1min":0.19400000000000001,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.39600000000000002,"15min":0.39900000000000002},"min":{"1min":0.29399999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.67000000000000004,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.378}]},{"osd":3,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.36799999999999999,"15min":0.376},"min":{"1min":0.26100000000000001,"5min":0.245,"15min":0.245},"max":{"1min":0.51400000000000001,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.375,"5min":0.372,"15min":0.372},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.50600000000000001,"5min":0.60099999999999998,"15min":0.60099999999999998},"last":0.436}]},{"osd":4,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.54200000000000004,"15min":0.54000000000000004},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.76000000000000001,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.56599999999999995,"15min":0.56399999999999995},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.72299999999999998,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.495}]},{"osd":5,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.57299999999999995,"15min":0.57699999999999996},"min":{"1min":0.40300000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.68799999999999994,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56799999999999995,"15min":0.56499999999999995},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.78900000000000003,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.56200000000000006}]},{"osd":6,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.57199999999999995,"15min":0.57399999999999995},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.69799999999999995,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.63},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.61799999999999999,"15min":0.623},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.68500000000000005}]},{"osd":7,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.59499999999999997,"15min":0.59099999999999997},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.78300000000000003,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.61799999999999999,"15min":0.624},"min":{"1min":0.43099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.66300000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574893,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9416,"kb_used_data":704,"kb_used_omap":0,"kb_used_meta":8704,"kb_avail":94362424,"statfs":{"total":96636764160,"available":96627122176,"internally_reserved":0,"allocated":720896,"data_stored":347320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8912896},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.32000000000000001,"5min":0.33900000000000002,"15min":0.35299999999999998},"min":{"1min":0.20100000000000001,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.58999999999999997,"5min":1.038,"15min":1.038},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.35699999999999998,"5min":0.35799999999999998,"15min":0.36699999999999999},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.52900000000000003,"5min":1.079,"15min":1.079},"last":0.45200000000000001}]},{"osd":1,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.39600000000000002,"15min":0.42099999999999999},"min":{"1min":0.26000000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.50900000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.44},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.436,"15min":0.45500000000000002},"min":{"1min":0.28399999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.64500000000000002,"5min":1.5189999999999999,"15min":1.5189999999999999},"last":0.49299999999999999}]},{"osd":3,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.39700000000000002,"15min":0.41499999999999998},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.55000000000000004,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.45000000000000001,"15min":0.46999999999999997},"min":{"1min":0.32600000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.63100000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.51300000000000001}]},{"osd":4,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.58399999999999996,"15min":0.58999999999999997},"min":{"1min":0.42699999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.86499999999999999,"5min":1.319,"15min":1.319},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.625,"15min":0.626},"min":{"1min":0.48699999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.83899999999999997,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.64100000000000001}]},{"osd":5,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.64000000000000001,"15min":0.65500000000000003},"min":{"1min":0.44800000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.80100000000000005,"5min":1.758,"15min":1.758},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.625,"5min":0.65000000000000002,"15min":0.67000000000000004},"min":{"1min":0.48099999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.80800000000000005,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.80600000000000005}]},{"osd":6,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65100000000000002,"15min":0.65700000000000003},"min":{"1min":0.45500000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.88,"5min":1.665,"15min":1.665},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.626,"5min":0.64800000000000002,"15min":0.65900000000000003},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.86599999999999999,"5min":1.819,"15min":1.819},"last":0.70299999999999996}]},{"osd":7,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.68000000000000005,"15min":0.69199999999999995},"min":{"1min":0.45500000000000002,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.83199999999999996,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.66600000000000004,"15min":0.68400000000000005},"min":{"1min":0.49299999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.89300000000000002,"5min":1.8460000000000001,"15min":1.8460000000000001},"last":0.81899999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574893,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9480,"kb_used_data":704,"kb_used_omap":0,"kb_used_meta":8768,"kb_avail":94362360,"statfs":{"total":96636764160,"available":96627056640,"internally_reserved":0,"allocated":720896,"data_stored":347320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8978432},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:19:23 2023","interfaces":[{"interface":"back","average":{"1min":0.33100000000000002,"5min":0.318,"15min":0.32400000000000001},"min":{"1min":0.219,"5min":0.192,"15min":0.192},"max":{"1min":0.46100000000000002,"5min":0.504,"15min":0.504},"last":0.29799999999999999},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.42699999999999999,"15min":0.47599999999999998},"min":{"1min":0.247,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.52800000000000002,"5min":3.1890000000000001,"15min":3.1890000000000001},"last":0.317}]},{"osd":1,"last update":"Sat May 20 01:19:23 2023","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.42099999999999999,"15min":0.46700000000000003},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.501,"5min":3.4260000000000002,"15min":3.4260000000000002},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.36199999999999999,"15min":0.36399999999999999},"min":{"1min":0.26100000000000001,"5min":0.221,"15min":0.221},"max":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"last":0.372}]},{"osd":2,"last update":"Sat May 20 01:19:23 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.39800000000000002,"15min":0.41299999999999998},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.62,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.41999999999999998,"15min":0.44},"min":{"1min":0.28699999999999998,"5min":0.253,"15min":0.253},"max":{"1min":0.58899999999999997,"5min":1.413,"15min":1.413},"last":0.39600000000000002}]},{"osd":4,"last update":"Sat May 20 01:19:23 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.59599999999999997,"15min":0.60799999999999998},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.80100000000000005,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.68300000000000005,"15min":0.72499999999999998},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.83599999999999997,"5min":3.4660000000000002,"15min":3.4660000000000002},"last":0.434}]},{"osd":5,"last update":"Sat May 20 01:19:23 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.69399999999999995,"15min":0.73999999999999999},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.77900000000000003,"5min":3.5379999999999998,"15min":3.5379999999999998},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.63300000000000001,"15min":0.65300000000000002},"min":{"1min":0.42599999999999999,"5min":0.374,"15min":0.374},"max":{"1min":0.81699999999999995,"5min":1.385,"15min":1.385},"last":0.58599999999999997}]},{"osd":6,"last update":"Sat May 20 01:19:23 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.63800000000000001,"15min":0.65800000000000003},"min":{"1min":0.40400000000000003,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.84299999999999997,"5min":1.347,"15min":1.347},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64200000000000002,"15min":0.65700000000000003},"min":{"1min":0.434,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.77400000000000002,"5min":1.107,"15min":1.107},"last":0.45700000000000002}]},{"osd":7,"last update":"Sat May 20 01:19:23 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.64600000000000002,"15min":0.66000000000000003},"min":{"1min":0.46600000000000003,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.83199999999999996,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.72899999999999998,"15min":0.77400000000000002},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.82199999999999995,"5min":3.5009999999999999,"15min":3.5009999999999999},"last":0.59399999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574893,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9736,"kb_used_data":704,"kb_used_omap":0,"kb_used_meta":9024,"kb_avail":94362104,"statfs":{"total":96636764160,"available":96626794496,"internally_reserved":0,"allocated":720896,"data_stored":347320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9240576},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:19:24 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.495,"15min":0.48099999999999998},"min":{"1min":0.379,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.72399999999999998,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.497,"15min":0.49299999999999999},"min":{"1min":0.35499999999999998,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.55200000000000005}]},{"osd":1,"last update":"Sat May 20 01:19:24 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.53400000000000003,"15min":0.53200000000000003},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.73299999999999998,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.622},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.54600000000000004,"15min":0.53200000000000003},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":2.0739999999999998,"5min":2.0739999999999998,"15min":2.0739999999999998},"last":0.498}]},{"osd":2,"last update":"Sat May 20 01:19:24 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.51400000000000001,"15min":0.499},"min":{"1min":0.439,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.745,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.55400000000000005,"15min":0.55500000000000005},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.751,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.60899999999999999}]},{"osd":3,"last update":"Sat May 20 01:19:24 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.53300000000000003,"15min":0.51900000000000002},"min":{"1min":0.42699999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.827,"5min":1.827,"15min":1.827},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.66700000000000004,"15min":0.70699999999999996},"min":{"1min":0.40400000000000003,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.77400000000000002,"5min":3.1749999999999998,"15min":3.1749999999999998},"last":0.59699999999999998}]},{"osd":5,"last update":"Sat May 20 01:19:24 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.47199999999999998,"15min":0.46000000000000002},"min":{"1min":0.33800000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.63100000000000001,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.49199999999999999,"15min":0.49299999999999999},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.67300000000000004,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.35399999999999998}]},{"osd":6,"last update":"Sat May 20 01:19:24 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52900000000000003,"15min":0.52500000000000002},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.64700000000000002,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.58199999999999996,"15min":0.57799999999999996},"min":{"1min":0.46200000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.65500000000000003,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.58299999999999996}]},{"osd":7,"last update":"Sat May 20 01:19:24 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.54800000000000004,"15min":0.54500000000000004},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.443},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.58599999999999997,"15min":0.58999999999999997},"min":{"1min":0.39100000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.67200000000000004,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.47099999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574894,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9220,"kb_used_data":700,"kb_used_omap":0,"kb_used_meta":8512,"kb_avail":94362620,"statfs":{"total":96636764160,"available":96627322880,"internally_reserved":0,"allocated":716800,"data_stored":346359,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8716288},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:19:26 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.68799999999999994,"15min":0.77400000000000002},"min":{"1min":0.33700000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.71899999999999997,"5min":6.3929999999999998,"15min":6.3929999999999998},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.72199999999999998,"15min":0.81200000000000006},"min":{"1min":0.434,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.745,"5min":6.3339999999999996,"15min":6.3339999999999996},"last":0.35899999999999999}]},{"osd":1,"last update":"Sat May 20 01:19:26 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.60999999999999999,"15min":0.66500000000000004},"min":{"1min":0.38100000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.73899999999999999,"5min":3.2450000000000001,"15min":3.2450000000000001},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54500000000000004,"15min":0.55500000000000005},"min":{"1min":0.41199999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.71899999999999997,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.46999999999999997}]},{"osd":2,"last update":"Sat May 20 01:19:26 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.55500000000000005,"15min":0.57599999999999996},"min":{"1min":0.34100000000000003,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.76300000000000001,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.56100000000000005,"15min":0.57899999999999996},"min":{"1min":0.375,"5min":0.314,"15min":0.314},"max":{"1min":0.72299999999999998,"5min":1.258,"15min":1.258},"last":0.55500000000000005}]},{"osd":3,"last update":"Sat May 20 01:19:26 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.751,"15min":0.85699999999999998},"min":{"1min":0.38200000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.68600000000000005,"5min":6.351,"15min":6.351},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.79300000000000004,"15min":0.89800000000000002},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.72599999999999998,"5min":6.3730000000000002,"15min":6.3730000000000002},"last":0.57899999999999996}]},{"osd":4,"last update":"Sat May 20 01:19:26 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.48599999999999999,"15min":0.48799999999999999},"min":{"1min":0.32400000000000001,"5min":0.316,"15min":0.316},"max":{"1min":0.65300000000000002,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.54800000000000004,"15min":0.56000000000000005},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.53100000000000003}]},{"osd":6,"last update":"Sat May 20 01:19:26 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.54300000000000004,"15min":0.55900000000000005},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.67600000000000005,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.56000000000000005,"15min":0.56599999999999995},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.747,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.59399999999999997}]},{"osd":7,"last update":"Sat May 20 01:19:26 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.57899999999999996,"15min":0.59199999999999997},"min":{"1min":0.36199999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.76900000000000002,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.61299999999999999,"15min":0.627},"min":{"1min":0.41399999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.73399999999999999,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.625}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-05-20T01:19:57.273 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3f is not active+clean 2023-05-20T01:19:57.274 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:19:51.482477+0000', 'last_change': '2023-05-20T01:19:51.482477+0000', 'last_active': '2023-05-20T01:19:51.482477+0000', 'last_peered': '2023-05-20T01:19:51.482477+0000', 'last_clean': '2023-05-20T01:19:51.482477+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:19:51.482477+0000', 'last_undegraded': '2023-05-20T01:19:51.482477+0000', 'last_fullsized': '2023-05-20T01:19:51.482477+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_clean_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:19:57.274 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3e is not active+clean 2023-05-20T01:19:57.274 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:19:51.482477+0000', 'last_change': '2023-05-20T01:19:51.482477+0000', 'last_active': '2023-05-20T01:19:51.482477+0000', 'last_peered': '2023-05-20T01:19:51.482477+0000', 'last_clean': '2023-05-20T01:19:51.482477+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:19:51.482477+0000', 'last_undegraded': '2023-05-20T01:19:51.482477+0000', 'last_fullsized': '2023-05-20T01:19:51.482477+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_clean_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:19:57.275 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3d is not active+clean 2023-05-20T01:19:57.275 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:19:51.482477+0000', 'last_change': '2023-05-20T01:19:51.482477+0000', 'last_active': '2023-05-20T01:19:51.482477+0000', 'last_peered': '2023-05-20T01:19:51.482477+0000', 'last_clean': '2023-05-20T01:19:51.482477+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:19:51.482477+0000', 'last_undegraded': '2023-05-20T01:19:51.482477+0000', 'last_fullsized': '2023-05-20T01:19:51.482477+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_clean_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:19:57.275 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3c is not active+clean 2023-05-20T01:19:57.276 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:19:51.482477+0000', 'last_change': '2023-05-20T01:19:51.482477+0000', 'last_active': '2023-05-20T01:19:51.482477+0000', 'last_peered': '2023-05-20T01:19:51.482477+0000', 'last_clean': '2023-05-20T01:19:51.482477+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:19:51.482477+0000', 'last_undegraded': '2023-05-20T01:19:51.482477+0000', 'last_fullsized': '2023-05-20T01:19:51.482477+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_clean_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:19:57.276 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3b is not active+clean 2023-05-20T01:19:57.276 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:19:51.482477+0000', 'last_change': '2023-05-20T01:19:51.482477+0000', 'last_active': '2023-05-20T01:19:51.482477+0000', 'last_peered': '2023-05-20T01:19:51.482477+0000', 'last_clean': '2023-05-20T01:19:51.482477+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:19:51.482477+0000', 'last_undegraded': '2023-05-20T01:19:51.482477+0000', 'last_fullsized': '2023-05-20T01:19:51.482477+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_clean_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:19:57.277 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3a is not active+clean 2023-05-20T01:19:57.277 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:19:51.482477+0000', 'last_change': '2023-05-20T01:19:51.482477+0000', 'last_active': '2023-05-20T01:19:51.482477+0000', 'last_peered': '2023-05-20T01:19:51.482477+0000', 'last_clean': '2023-05-20T01:19:51.482477+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:19:51.482477+0000', 'last_undegraded': '2023-05-20T01:19:51.482477+0000', 'last_fullsized': '2023-05-20T01:19:51.482477+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_clean_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:19:57.277 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.39 is not active+clean 2023-05-20T01:19:57.277 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:19:51.482477+0000', 'last_change': '2023-05-20T01:19:51.482477+0000', 'last_active': '2023-05-20T01:19:51.482477+0000', 'last_peered': '2023-05-20T01:19:51.482477+0000', 'last_clean': '2023-05-20T01:19:51.482477+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:19:51.482477+0000', 'last_undegraded': '2023-05-20T01:19:51.482477+0000', 'last_fullsized': '2023-05-20T01:19:51.482477+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_clean_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:19:57.278 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.38 is not active+clean 2023-05-20T01:19:57.278 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:19:51.482477+0000', 'last_change': '2023-05-20T01:19:51.482477+0000', 'last_active': '2023-05-20T01:19:51.482477+0000', 'last_peered': '2023-05-20T01:19:51.482477+0000', 'last_clean': '2023-05-20T01:19:51.482477+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:19:51.482477+0000', 'last_undegraded': '2023-05-20T01:19:51.482477+0000', 'last_fullsized': '2023-05-20T01:19:51.482477+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_clean_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:19:57.278 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.37 is not active+clean 2023-05-20T01:19:57.279 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:19:51.482477+0000', 'last_change': '2023-05-20T01:19:51.482477+0000', 'last_active': '2023-05-20T01:19:51.482477+0000', 'last_peered': '2023-05-20T01:19:51.482477+0000', 'last_clean': '2023-05-20T01:19:51.482477+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:19:51.482477+0000', 'last_undegraded': '2023-05-20T01:19:51.482477+0000', 'last_fullsized': '2023-05-20T01:19:51.482477+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_clean_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:19:57.279 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.36 is not active+clean 2023-05-20T01:19:57.279 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:19:51.482477+0000', 'last_change': '2023-05-20T01:19:51.482477+0000', 'last_active': '2023-05-20T01:19:51.482477+0000', 'last_peered': '2023-05-20T01:19:51.482477+0000', 'last_clean': '2023-05-20T01:19:51.482477+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:19:51.482477+0000', 'last_undegraded': '2023-05-20T01:19:51.482477+0000', 'last_fullsized': '2023-05-20T01:19:51.482477+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_clean_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:19:57.280 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.35 is not active+clean 2023-05-20T01:19:57.280 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:19:51.482477+0000', 'last_change': '2023-05-20T01:19:51.482477+0000', 'last_active': '2023-05-20T01:19:51.482477+0000', 'last_peered': '2023-05-20T01:19:51.482477+0000', 'last_clean': '2023-05-20T01:19:51.482477+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:19:51.482477+0000', 'last_undegraded': '2023-05-20T01:19:51.482477+0000', 'last_fullsized': '2023-05-20T01:19:51.482477+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_clean_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:19:57.280 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.34 is not active+clean 2023-05-20T01:19:57.281 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:19:51.482477+0000', 'last_change': '2023-05-20T01:19:51.482477+0000', 'last_active': '2023-05-20T01:19:51.482477+0000', 'last_peered': '2023-05-20T01:19:51.482477+0000', 'last_clean': '2023-05-20T01:19:51.482477+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:19:51.482477+0000', 'last_undegraded': '2023-05-20T01:19:51.482477+0000', 'last_fullsized': '2023-05-20T01:19:51.482477+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_clean_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:19:57.281 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.33 is not active+clean 2023-05-20T01:19:57.281 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:19:51.482477+0000', 'last_change': '2023-05-20T01:19:51.482477+0000', 'last_active': '2023-05-20T01:19:51.482477+0000', 'last_peered': '2023-05-20T01:19:51.482477+0000', 'last_clean': '2023-05-20T01:19:51.482477+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:19:51.482477+0000', 'last_undegraded': '2023-05-20T01:19:51.482477+0000', 'last_fullsized': '2023-05-20T01:19:51.482477+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_clean_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:19:57.282 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.32 is not active+clean 2023-05-20T01:19:57.282 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:19:51.482477+0000', 'last_change': '2023-05-20T01:19:51.482477+0000', 'last_active': '2023-05-20T01:19:51.482477+0000', 'last_peered': '2023-05-20T01:19:51.482477+0000', 'last_clean': '2023-05-20T01:19:51.482477+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:19:51.482477+0000', 'last_undegraded': '2023-05-20T01:19:51.482477+0000', 'last_fullsized': '2023-05-20T01:19:51.482477+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_clean_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:19:57.282 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.31 is not active+clean 2023-05-20T01:19:57.282 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:19:51.482477+0000', 'last_change': '2023-05-20T01:19:51.482477+0000', 'last_active': '2023-05-20T01:19:51.482477+0000', 'last_peered': '2023-05-20T01:19:51.482477+0000', 'last_clean': '2023-05-20T01:19:51.482477+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:19:51.482477+0000', 'last_undegraded': '2023-05-20T01:19:51.482477+0000', 'last_fullsized': '2023-05-20T01:19:51.482477+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_clean_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:19:57.283 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.30 is not active+clean 2023-05-20T01:19:57.283 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:19:51.482477+0000', 'last_change': '2023-05-20T01:19:51.482477+0000', 'last_active': '2023-05-20T01:19:51.482477+0000', 'last_peered': '2023-05-20T01:19:51.482477+0000', 'last_clean': '2023-05-20T01:19:51.482477+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:19:51.482477+0000', 'last_undegraded': '2023-05-20T01:19:51.482477+0000', 'last_fullsized': '2023-05-20T01:19:51.482477+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_clean_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:19:57.284 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2f is not active+clean 2023-05-20T01:19:57.284 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:19:51.482477+0000', 'last_change': '2023-05-20T01:19:51.482477+0000', 'last_active': '2023-05-20T01:19:51.482477+0000', 'last_peered': '2023-05-20T01:19:51.482477+0000', 'last_clean': '2023-05-20T01:19:51.482477+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:19:51.482477+0000', 'last_undegraded': '2023-05-20T01:19:51.482477+0000', 'last_fullsized': '2023-05-20T01:19:51.482477+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_clean_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:19:57.285 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2e is not active+clean 2023-05-20T01:19:57.286 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:19:51.482477+0000', 'last_change': '2023-05-20T01:19:51.482477+0000', 'last_active': '2023-05-20T01:19:51.482477+0000', 'last_peered': '2023-05-20T01:19:51.482477+0000', 'last_clean': '2023-05-20T01:19:51.482477+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:19:51.482477+0000', 'last_undegraded': '2023-05-20T01:19:51.482477+0000', 'last_fullsized': '2023-05-20T01:19:51.482477+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_clean_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:19:57.287 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2d is not active+clean 2023-05-20T01:19:57.288 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:19:51.482477+0000', 'last_change': '2023-05-20T01:19:51.482477+0000', 'last_active': '2023-05-20T01:19:51.482477+0000', 'last_peered': '2023-05-20T01:19:51.482477+0000', 'last_clean': '2023-05-20T01:19:51.482477+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:19:51.482477+0000', 'last_undegraded': '2023-05-20T01:19:51.482477+0000', 'last_fullsized': '2023-05-20T01:19:51.482477+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_clean_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:19:57.288 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2c is not active+clean 2023-05-20T01:19:57.289 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:19:51.482477+0000', 'last_change': '2023-05-20T01:19:51.482477+0000', 'last_active': '2023-05-20T01:19:51.482477+0000', 'last_peered': '2023-05-20T01:19:51.482477+0000', 'last_clean': '2023-05-20T01:19:51.482477+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:19:51.482477+0000', 'last_undegraded': '2023-05-20T01:19:51.482477+0000', 'last_fullsized': '2023-05-20T01:19:51.482477+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_clean_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:19:57.290 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2b is not active+clean 2023-05-20T01:19:57.291 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:19:51.482477+0000', 'last_change': '2023-05-20T01:19:51.482477+0000', 'last_active': '2023-05-20T01:19:51.482477+0000', 'last_peered': '2023-05-20T01:19:51.482477+0000', 'last_clean': '2023-05-20T01:19:51.482477+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:19:51.482477+0000', 'last_undegraded': '2023-05-20T01:19:51.482477+0000', 'last_fullsized': '2023-05-20T01:19:51.482477+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_clean_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:19:57.291 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2a is not active+clean 2023-05-20T01:19:57.292 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:19:51.482477+0000', 'last_change': '2023-05-20T01:19:51.482477+0000', 'last_active': '2023-05-20T01:19:51.482477+0000', 'last_peered': '2023-05-20T01:19:51.482477+0000', 'last_clean': '2023-05-20T01:19:51.482477+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:19:51.482477+0000', 'last_undegraded': '2023-05-20T01:19:51.482477+0000', 'last_fullsized': '2023-05-20T01:19:51.482477+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_clean_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:19:57.292 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.29 is not active+clean 2023-05-20T01:19:57.293 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:19:51.482477+0000', 'last_change': '2023-05-20T01:19:51.482477+0000', 'last_active': '2023-05-20T01:19:51.482477+0000', 'last_peered': '2023-05-20T01:19:51.482477+0000', 'last_clean': '2023-05-20T01:19:51.482477+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:19:51.482477+0000', 'last_undegraded': '2023-05-20T01:19:51.482477+0000', 'last_fullsized': '2023-05-20T01:19:51.482477+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_clean_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:19:57.294 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.28 is not active+clean 2023-05-20T01:19:57.294 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:19:51.482477+0000', 'last_change': '2023-05-20T01:19:51.482477+0000', 'last_active': '2023-05-20T01:19:51.482477+0000', 'last_peered': '2023-05-20T01:19:51.482477+0000', 'last_clean': '2023-05-20T01:19:51.482477+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:19:51.482477+0000', 'last_undegraded': '2023-05-20T01:19:51.482477+0000', 'last_fullsized': '2023-05-20T01:19:51.482477+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_clean_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:19:57.295 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.27 is not active+clean 2023-05-20T01:19:57.295 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:19:51.482477+0000', 'last_change': '2023-05-20T01:19:51.482477+0000', 'last_active': '2023-05-20T01:19:51.482477+0000', 'last_peered': '2023-05-20T01:19:51.482477+0000', 'last_clean': '2023-05-20T01:19:51.482477+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:19:51.482477+0000', 'last_undegraded': '2023-05-20T01:19:51.482477+0000', 'last_fullsized': '2023-05-20T01:19:51.482477+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_clean_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:19:57.295 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.26 is not active+clean 2023-05-20T01:19:57.295 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:19:51.482477+0000', 'last_change': '2023-05-20T01:19:51.482477+0000', 'last_active': '2023-05-20T01:19:51.482477+0000', 'last_peered': '2023-05-20T01:19:51.482477+0000', 'last_clean': '2023-05-20T01:19:51.482477+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:19:51.482477+0000', 'last_undegraded': '2023-05-20T01:19:51.482477+0000', 'last_fullsized': '2023-05-20T01:19:51.482477+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_clean_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:19:57.296 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.25 is not active+clean 2023-05-20T01:19:57.296 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:19:51.482477+0000', 'last_change': '2023-05-20T01:19:51.482477+0000', 'last_active': '2023-05-20T01:19:51.482477+0000', 'last_peered': '2023-05-20T01:19:51.482477+0000', 'last_clean': '2023-05-20T01:19:51.482477+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:19:51.482477+0000', 'last_undegraded': '2023-05-20T01:19:51.482477+0000', 'last_fullsized': '2023-05-20T01:19:51.482477+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_clean_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:19:57.297 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.24 is not active+clean 2023-05-20T01:19:57.297 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:19:51.482477+0000', 'last_change': '2023-05-20T01:19:51.482477+0000', 'last_active': '2023-05-20T01:19:51.482477+0000', 'last_peered': '2023-05-20T01:19:51.482477+0000', 'last_clean': '2023-05-20T01:19:51.482477+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:19:51.482477+0000', 'last_undegraded': '2023-05-20T01:19:51.482477+0000', 'last_fullsized': '2023-05-20T01:19:51.482477+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_clean_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:19:57.297 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.23 is not active+clean 2023-05-20T01:19:57.298 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:19:51.482477+0000', 'last_change': '2023-05-20T01:19:51.482477+0000', 'last_active': '2023-05-20T01:19:51.482477+0000', 'last_peered': '2023-05-20T01:19:51.482477+0000', 'last_clean': '2023-05-20T01:19:51.482477+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:19:51.482477+0000', 'last_undegraded': '2023-05-20T01:19:51.482477+0000', 'last_fullsized': '2023-05-20T01:19:51.482477+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_clean_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:19:57.298 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.22 is not active+clean 2023-05-20T01:19:57.298 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:19:51.482477+0000', 'last_change': '2023-05-20T01:19:51.482477+0000', 'last_active': '2023-05-20T01:19:51.482477+0000', 'last_peered': '2023-05-20T01:19:51.482477+0000', 'last_clean': '2023-05-20T01:19:51.482477+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:19:51.482477+0000', 'last_undegraded': '2023-05-20T01:19:51.482477+0000', 'last_fullsized': '2023-05-20T01:19:51.482477+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_clean_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:19:57.299 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.21 is not active+clean 2023-05-20T01:19:57.299 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:19:51.482477+0000', 'last_change': '2023-05-20T01:19:51.482477+0000', 'last_active': '2023-05-20T01:19:51.482477+0000', 'last_peered': '2023-05-20T01:19:51.482477+0000', 'last_clean': '2023-05-20T01:19:51.482477+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:19:51.482477+0000', 'last_undegraded': '2023-05-20T01:19:51.482477+0000', 'last_fullsized': '2023-05-20T01:19:51.482477+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_clean_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:19:57.299 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.20 is not active+clean 2023-05-20T01:19:57.299 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:19:51.482477+0000', 'last_change': '2023-05-20T01:19:51.482477+0000', 'last_active': '2023-05-20T01:19:51.482477+0000', 'last_peered': '2023-05-20T01:19:51.482477+0000', 'last_clean': '2023-05-20T01:19:51.482477+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:19:51.482477+0000', 'last_undegraded': '2023-05-20T01:19:51.482477+0000', 'last_fullsized': '2023-05-20T01:19:51.482477+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_clean_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:19:57.300 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1c is not active+clean 2023-05-20T01:19:57.300 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:19:51.482477+0000', 'last_change': '2023-05-20T01:19:51.482477+0000', 'last_active': '2023-05-20T01:19:51.482477+0000', 'last_peered': '2023-05-20T01:19:51.482477+0000', 'last_clean': '2023-05-20T01:19:51.482477+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:19:51.482477+0000', 'last_undegraded': '2023-05-20T01:19:51.482477+0000', 'last_fullsized': '2023-05-20T01:19:51.482477+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_clean_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:19:57.300 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1f is not active+clean 2023-05-20T01:19:57.301 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:19:50.473280+0000', 'last_change': '2023-05-20T01:19:50.473280+0000', 'last_active': '2023-05-20T01:19:50.473280+0000', 'last_peered': '2023-05-20T01:19:50.473280+0000', 'last_clean': '2023-05-20T01:19:50.473280+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:19:50.473280+0000', 'last_undegraded': '2023-05-20T01:19:50.473280+0000', 'last_fullsized': '2023-05-20T01:19:50.473280+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:19:50.473280+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:19:50.473280+0000', 'last_clean_scrub_stamp': '2023-05-20T01:19:50.473280+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:19:57.301 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1d is not active+clean 2023-05-20T01:19:57.301 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:19:51.482477+0000', 'last_change': '2023-05-20T01:19:51.482477+0000', 'last_active': '2023-05-20T01:19:51.482477+0000', 'last_peered': '2023-05-20T01:19:51.482477+0000', 'last_clean': '2023-05-20T01:19:51.482477+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:19:51.482477+0000', 'last_undegraded': '2023-05-20T01:19:51.482477+0000', 'last_fullsized': '2023-05-20T01:19:51.482477+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_clean_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:19:57.302 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1e is not active+clean 2023-05-20T01:19:57.302 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:19:50.473280+0000', 'last_change': '2023-05-20T01:19:50.473280+0000', 'last_active': '2023-05-20T01:19:50.473280+0000', 'last_peered': '2023-05-20T01:19:50.473280+0000', 'last_clean': '2023-05-20T01:19:50.473280+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:19:50.473280+0000', 'last_undegraded': '2023-05-20T01:19:50.473280+0000', 'last_fullsized': '2023-05-20T01:19:50.473280+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:19:50.473280+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:19:50.473280+0000', 'last_clean_scrub_stamp': '2023-05-20T01:19:50.473280+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:19:57.302 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1e is not active+clean 2023-05-20T01:19:57.303 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:19:51.482477+0000', 'last_change': '2023-05-20T01:19:51.482477+0000', 'last_active': '2023-05-20T01:19:51.482477+0000', 'last_peered': '2023-05-20T01:19:51.482477+0000', 'last_clean': '2023-05-20T01:19:51.482477+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:19:51.482477+0000', 'last_undegraded': '2023-05-20T01:19:51.482477+0000', 'last_fullsized': '2023-05-20T01:19:51.482477+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_clean_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:19:57.303 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1d is not active+clean 2023-05-20T01:19:57.303 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:19:50.473280+0000', 'last_change': '2023-05-20T01:19:50.473280+0000', 'last_active': '2023-05-20T01:19:50.473280+0000', 'last_peered': '2023-05-20T01:19:50.473280+0000', 'last_clean': '2023-05-20T01:19:50.473280+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:19:50.473280+0000', 'last_undegraded': '2023-05-20T01:19:50.473280+0000', 'last_fullsized': '2023-05-20T01:19:50.473280+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:19:50.473280+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:19:50.473280+0000', 'last_clean_scrub_stamp': '2023-05-20T01:19:50.473280+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:19:57.304 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1f is not active+clean 2023-05-20T01:19:57.304 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:19:51.482477+0000', 'last_change': '2023-05-20T01:19:51.482477+0000', 'last_active': '2023-05-20T01:19:51.482477+0000', 'last_peered': '2023-05-20T01:19:51.482477+0000', 'last_clean': '2023-05-20T01:19:51.482477+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:19:51.482477+0000', 'last_undegraded': '2023-05-20T01:19:51.482477+0000', 'last_fullsized': '2023-05-20T01:19:51.482477+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_clean_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:19:57.304 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1c is not active+clean 2023-05-20T01:19:57.305 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:19:50.473280+0000', 'last_change': '2023-05-20T01:19:50.473280+0000', 'last_active': '2023-05-20T01:19:50.473280+0000', 'last_peered': '2023-05-20T01:19:50.473280+0000', 'last_clean': '2023-05-20T01:19:50.473280+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:19:50.473280+0000', 'last_undegraded': '2023-05-20T01:19:50.473280+0000', 'last_fullsized': '2023-05-20T01:19:50.473280+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:19:50.473280+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:19:50.473280+0000', 'last_clean_scrub_stamp': '2023-05-20T01:19:50.473280+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:19:57.305 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.18 is not active+clean 2023-05-20T01:19:57.305 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:19:51.482477+0000', 'last_change': '2023-05-20T01:19:51.482477+0000', 'last_active': '2023-05-20T01:19:51.482477+0000', 'last_peered': '2023-05-20T01:19:51.482477+0000', 'last_clean': '2023-05-20T01:19:51.482477+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:19:51.482477+0000', 'last_undegraded': '2023-05-20T01:19:51.482477+0000', 'last_fullsized': '2023-05-20T01:19:51.482477+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_clean_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:19:57.306 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1b is not active+clean 2023-05-20T01:19:57.306 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:19:50.473280+0000', 'last_change': '2023-05-20T01:19:50.473280+0000', 'last_active': '2023-05-20T01:19:50.473280+0000', 'last_peered': '2023-05-20T01:19:50.473280+0000', 'last_clean': '2023-05-20T01:19:50.473280+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:19:50.473280+0000', 'last_undegraded': '2023-05-20T01:19:50.473280+0000', 'last_fullsized': '2023-05-20T01:19:50.473280+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:19:50.473280+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:19:50.473280+0000', 'last_clean_scrub_stamp': '2023-05-20T01:19:50.473280+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:19:57.306 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.19 is not active+clean 2023-05-20T01:19:57.306 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:19:51.482477+0000', 'last_change': '2023-05-20T01:19:51.482477+0000', 'last_active': '2023-05-20T01:19:51.482477+0000', 'last_peered': '2023-05-20T01:19:51.482477+0000', 'last_clean': '2023-05-20T01:19:51.482477+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:19:51.482477+0000', 'last_undegraded': '2023-05-20T01:19:51.482477+0000', 'last_fullsized': '2023-05-20T01:19:51.482477+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_clean_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:19:57.307 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1a is not active+clean 2023-05-20T01:19:57.307 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:19:50.473280+0000', 'last_change': '2023-05-20T01:19:50.473280+0000', 'last_active': '2023-05-20T01:19:50.473280+0000', 'last_peered': '2023-05-20T01:19:50.473280+0000', 'last_clean': '2023-05-20T01:19:50.473280+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:19:50.473280+0000', 'last_undegraded': '2023-05-20T01:19:50.473280+0000', 'last_fullsized': '2023-05-20T01:19:50.473280+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:19:50.473280+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:19:50.473280+0000', 'last_clean_scrub_stamp': '2023-05-20T01:19:50.473280+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:19:57.308 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1a is not active+clean 2023-05-20T01:19:57.308 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:19:51.482477+0000', 'last_change': '2023-05-20T01:19:51.482477+0000', 'last_active': '2023-05-20T01:19:51.482477+0000', 'last_peered': '2023-05-20T01:19:51.482477+0000', 'last_clean': '2023-05-20T01:19:51.482477+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:19:51.482477+0000', 'last_undegraded': '2023-05-20T01:19:51.482477+0000', 'last_fullsized': '2023-05-20T01:19:51.482477+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_clean_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:19:57.308 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.19 is not active+clean 2023-05-20T01:19:57.309 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:19:50.473280+0000', 'last_change': '2023-05-20T01:19:50.473280+0000', 'last_active': '2023-05-20T01:19:50.473280+0000', 'last_peered': '2023-05-20T01:19:50.473280+0000', 'last_clean': '2023-05-20T01:19:50.473280+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:19:50.473280+0000', 'last_undegraded': '2023-05-20T01:19:50.473280+0000', 'last_fullsized': '2023-05-20T01:19:50.473280+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:19:50.473280+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:19:50.473280+0000', 'last_clean_scrub_stamp': '2023-05-20T01:19:50.473280+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:19:57.309 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1b is not active+clean 2023-05-20T01:19:57.309 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:19:51.482477+0000', 'last_change': '2023-05-20T01:19:51.482477+0000', 'last_active': '2023-05-20T01:19:51.482477+0000', 'last_peered': '2023-05-20T01:19:51.482477+0000', 'last_clean': '2023-05-20T01:19:51.482477+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:19:51.482477+0000', 'last_undegraded': '2023-05-20T01:19:51.482477+0000', 'last_fullsized': '2023-05-20T01:19:51.482477+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_clean_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:19:57.309 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.18 is not active+clean 2023-05-20T01:19:57.310 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:19:50.473280+0000', 'last_change': '2023-05-20T01:19:50.473280+0000', 'last_active': '2023-05-20T01:19:50.473280+0000', 'last_peered': '2023-05-20T01:19:50.473280+0000', 'last_clean': '2023-05-20T01:19:50.473280+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:19:50.473280+0000', 'last_undegraded': '2023-05-20T01:19:50.473280+0000', 'last_fullsized': '2023-05-20T01:19:50.473280+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:19:50.473280+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:19:50.473280+0000', 'last_clean_scrub_stamp': '2023-05-20T01:19:50.473280+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:19:57.310 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.14 is not active+clean 2023-05-20T01:19:57.310 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:19:51.482477+0000', 'last_change': '2023-05-20T01:19:51.482477+0000', 'last_active': '2023-05-20T01:19:51.482477+0000', 'last_peered': '2023-05-20T01:19:51.482477+0000', 'last_clean': '2023-05-20T01:19:51.482477+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:19:51.482477+0000', 'last_undegraded': '2023-05-20T01:19:51.482477+0000', 'last_fullsized': '2023-05-20T01:19:51.482477+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_clean_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:19:57.311 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.17 is not active+clean 2023-05-20T01:19:57.311 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:19:50.473280+0000', 'last_change': '2023-05-20T01:19:50.473280+0000', 'last_active': '2023-05-20T01:19:50.473280+0000', 'last_peered': '2023-05-20T01:19:50.473280+0000', 'last_clean': '2023-05-20T01:19:50.473280+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:19:50.473280+0000', 'last_undegraded': '2023-05-20T01:19:50.473280+0000', 'last_fullsized': '2023-05-20T01:19:50.473280+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:19:50.473280+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:19:50.473280+0000', 'last_clean_scrub_stamp': '2023-05-20T01:19:50.473280+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:19:57.311 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.15 is not active+clean 2023-05-20T01:19:57.312 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:19:51.482477+0000', 'last_change': '2023-05-20T01:19:51.482477+0000', 'last_active': '2023-05-20T01:19:51.482477+0000', 'last_peered': '2023-05-20T01:19:51.482477+0000', 'last_clean': '2023-05-20T01:19:51.482477+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:19:51.482477+0000', 'last_undegraded': '2023-05-20T01:19:51.482477+0000', 'last_fullsized': '2023-05-20T01:19:51.482477+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_clean_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:19:57.312 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.16 is not active+clean 2023-05-20T01:19:57.312 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:19:50.473280+0000', 'last_change': '2023-05-20T01:19:50.473280+0000', 'last_active': '2023-05-20T01:19:50.473280+0000', 'last_peered': '2023-05-20T01:19:50.473280+0000', 'last_clean': '2023-05-20T01:19:50.473280+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:19:50.473280+0000', 'last_undegraded': '2023-05-20T01:19:50.473280+0000', 'last_fullsized': '2023-05-20T01:19:50.473280+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:19:50.473280+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:19:50.473280+0000', 'last_clean_scrub_stamp': '2023-05-20T01:19:50.473280+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:19:57.313 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.16 is not active+clean 2023-05-20T01:19:57.313 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:19:51.482477+0000', 'last_change': '2023-05-20T01:19:51.482477+0000', 'last_active': '2023-05-20T01:19:51.482477+0000', 'last_peered': '2023-05-20T01:19:51.482477+0000', 'last_clean': '2023-05-20T01:19:51.482477+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:19:51.482477+0000', 'last_undegraded': '2023-05-20T01:19:51.482477+0000', 'last_fullsized': '2023-05-20T01:19:51.482477+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_clean_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:19:57.313 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.15 is not active+clean 2023-05-20T01:19:57.314 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:19:50.473280+0000', 'last_change': '2023-05-20T01:19:50.473280+0000', 'last_active': '2023-05-20T01:19:50.473280+0000', 'last_peered': '2023-05-20T01:19:50.473280+0000', 'last_clean': '2023-05-20T01:19:50.473280+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:19:50.473280+0000', 'last_undegraded': '2023-05-20T01:19:50.473280+0000', 'last_fullsized': '2023-05-20T01:19:50.473280+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:19:50.473280+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:19:50.473280+0000', 'last_clean_scrub_stamp': '2023-05-20T01:19:50.473280+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:19:57.314 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.17 is not active+clean 2023-05-20T01:19:57.315 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:19:51.482477+0000', 'last_change': '2023-05-20T01:19:51.482477+0000', 'last_active': '2023-05-20T01:19:51.482477+0000', 'last_peered': '2023-05-20T01:19:51.482477+0000', 'last_clean': '2023-05-20T01:19:51.482477+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:19:51.482477+0000', 'last_undegraded': '2023-05-20T01:19:51.482477+0000', 'last_fullsized': '2023-05-20T01:19:51.482477+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_clean_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:19:57.315 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.14 is not active+clean 2023-05-20T01:19:57.315 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:19:50.473280+0000', 'last_change': '2023-05-20T01:19:50.473280+0000', 'last_active': '2023-05-20T01:19:50.473280+0000', 'last_peered': '2023-05-20T01:19:50.473280+0000', 'last_clean': '2023-05-20T01:19:50.473280+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:19:50.473280+0000', 'last_undegraded': '2023-05-20T01:19:50.473280+0000', 'last_fullsized': '2023-05-20T01:19:50.473280+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:19:50.473280+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:19:50.473280+0000', 'last_clean_scrub_stamp': '2023-05-20T01:19:50.473280+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:19:57.316 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.10 is not active+clean 2023-05-20T01:19:57.316 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:19:51.482477+0000', 'last_change': '2023-05-20T01:19:51.482477+0000', 'last_active': '2023-05-20T01:19:51.482477+0000', 'last_peered': '2023-05-20T01:19:51.482477+0000', 'last_clean': '2023-05-20T01:19:51.482477+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:19:51.482477+0000', 'last_undegraded': '2023-05-20T01:19:51.482477+0000', 'last_fullsized': '2023-05-20T01:19:51.482477+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_clean_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:19:57.316 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.13 is not active+clean 2023-05-20T01:19:57.317 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:19:50.473280+0000', 'last_change': '2023-05-20T01:19:50.473280+0000', 'last_active': '2023-05-20T01:19:50.473280+0000', 'last_peered': '2023-05-20T01:19:50.473280+0000', 'last_clean': '2023-05-20T01:19:50.473280+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:19:50.473280+0000', 'last_undegraded': '2023-05-20T01:19:50.473280+0000', 'last_fullsized': '2023-05-20T01:19:50.473280+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:19:50.473280+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:19:50.473280+0000', 'last_clean_scrub_stamp': '2023-05-20T01:19:50.473280+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:19:57.317 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.11 is not active+clean 2023-05-20T01:19:57.317 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:19:51.482477+0000', 'last_change': '2023-05-20T01:19:51.482477+0000', 'last_active': '2023-05-20T01:19:51.482477+0000', 'last_peered': '2023-05-20T01:19:51.482477+0000', 'last_clean': '2023-05-20T01:19:51.482477+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:19:51.482477+0000', 'last_undegraded': '2023-05-20T01:19:51.482477+0000', 'last_fullsized': '2023-05-20T01:19:51.482477+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_clean_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:19:57.318 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.12 is not active+clean 2023-05-20T01:19:57.318 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:19:50.473280+0000', 'last_change': '2023-05-20T01:19:50.473280+0000', 'last_active': '2023-05-20T01:19:50.473280+0000', 'last_peered': '2023-05-20T01:19:50.473280+0000', 'last_clean': '2023-05-20T01:19:50.473280+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:19:50.473280+0000', 'last_undegraded': '2023-05-20T01:19:50.473280+0000', 'last_fullsized': '2023-05-20T01:19:50.473280+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:19:50.473280+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:19:50.473280+0000', 'last_clean_scrub_stamp': '2023-05-20T01:19:50.473280+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:19:57.318 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.12 is not active+clean 2023-05-20T01:19:57.318 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:19:51.482477+0000', 'last_change': '2023-05-20T01:19:51.482477+0000', 'last_active': '2023-05-20T01:19:51.482477+0000', 'last_peered': '2023-05-20T01:19:51.482477+0000', 'last_clean': '2023-05-20T01:19:51.482477+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:19:51.482477+0000', 'last_undegraded': '2023-05-20T01:19:51.482477+0000', 'last_fullsized': '2023-05-20T01:19:51.482477+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_clean_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:19:57.319 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.11 is not active+clean 2023-05-20T01:19:57.319 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:19:50.473280+0000', 'last_change': '2023-05-20T01:19:50.473280+0000', 'last_active': '2023-05-20T01:19:50.473280+0000', 'last_peered': '2023-05-20T01:19:50.473280+0000', 'last_clean': '2023-05-20T01:19:50.473280+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:19:50.473280+0000', 'last_undegraded': '2023-05-20T01:19:50.473280+0000', 'last_fullsized': '2023-05-20T01:19:50.473280+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:19:50.473280+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:19:50.473280+0000', 'last_clean_scrub_stamp': '2023-05-20T01:19:50.473280+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:19:57.319 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.13 is not active+clean 2023-05-20T01:19:57.320 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:19:51.482477+0000', 'last_change': '2023-05-20T01:19:51.482477+0000', 'last_active': '2023-05-20T01:19:51.482477+0000', 'last_peered': '2023-05-20T01:19:51.482477+0000', 'last_clean': '2023-05-20T01:19:51.482477+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:19:51.482477+0000', 'last_undegraded': '2023-05-20T01:19:51.482477+0000', 'last_fullsized': '2023-05-20T01:19:51.482477+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_clean_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:19:57.320 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.10 is not active+clean 2023-05-20T01:19:57.320 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:19:50.473280+0000', 'last_change': '2023-05-20T01:19:50.473280+0000', 'last_active': '2023-05-20T01:19:50.473280+0000', 'last_peered': '2023-05-20T01:19:50.473280+0000', 'last_clean': '2023-05-20T01:19:50.473280+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:19:50.473280+0000', 'last_undegraded': '2023-05-20T01:19:50.473280+0000', 'last_fullsized': '2023-05-20T01:19:50.473280+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:19:50.473280+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:19:50.473280+0000', 'last_clean_scrub_stamp': '2023-05-20T01:19:50.473280+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:19:57.321 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.4 is not active+clean 2023-05-20T01:19:57.321 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:19:51.482477+0000', 'last_change': '2023-05-20T01:19:51.482477+0000', 'last_active': '2023-05-20T01:19:51.482477+0000', 'last_peered': '2023-05-20T01:19:51.482477+0000', 'last_clean': '2023-05-20T01:19:51.482477+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:19:51.482477+0000', 'last_undegraded': '2023-05-20T01:19:51.482477+0000', 'last_fullsized': '2023-05-20T01:19:51.482477+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_clean_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:19:57.322 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.7 is not active+clean 2023-05-20T01:19:57.322 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:19:50.473280+0000', 'last_change': '2023-05-20T01:19:50.473280+0000', 'last_active': '2023-05-20T01:19:50.473280+0000', 'last_peered': '2023-05-20T01:19:50.473280+0000', 'last_clean': '2023-05-20T01:19:50.473280+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:19:50.473280+0000', 'last_undegraded': '2023-05-20T01:19:50.473280+0000', 'last_fullsized': '2023-05-20T01:19:50.473280+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:19:50.473280+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:19:50.473280+0000', 'last_clean_scrub_stamp': '2023-05-20T01:19:50.473280+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:19:57.322 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.5 is not active+clean 2023-05-20T01:19:57.322 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:19:51.482477+0000', 'last_change': '2023-05-20T01:19:51.482477+0000', 'last_active': '2023-05-20T01:19:51.482477+0000', 'last_peered': '2023-05-20T01:19:51.482477+0000', 'last_clean': '2023-05-20T01:19:51.482477+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:19:51.482477+0000', 'last_undegraded': '2023-05-20T01:19:51.482477+0000', 'last_fullsized': '2023-05-20T01:19:51.482477+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_clean_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:19:57.323 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.6 is not active+clean 2023-05-20T01:19:57.323 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:19:50.473280+0000', 'last_change': '2023-05-20T01:19:50.473280+0000', 'last_active': '2023-05-20T01:19:50.473280+0000', 'last_peered': '2023-05-20T01:19:50.473280+0000', 'last_clean': '2023-05-20T01:19:50.473280+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:19:50.473280+0000', 'last_undegraded': '2023-05-20T01:19:50.473280+0000', 'last_fullsized': '2023-05-20T01:19:50.473280+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:19:50.473280+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:19:50.473280+0000', 'last_clean_scrub_stamp': '2023-05-20T01:19:50.473280+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:19:57.323 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.6 is not active+clean 2023-05-20T01:19:57.324 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:19:51.482477+0000', 'last_change': '2023-05-20T01:19:51.482477+0000', 'last_active': '2023-05-20T01:19:51.482477+0000', 'last_peered': '2023-05-20T01:19:51.482477+0000', 'last_clean': '2023-05-20T01:19:51.482477+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:19:51.482477+0000', 'last_undegraded': '2023-05-20T01:19:51.482477+0000', 'last_fullsized': '2023-05-20T01:19:51.482477+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_clean_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:19:57.324 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.5 is not active+clean 2023-05-20T01:19:57.324 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:19:50.473280+0000', 'last_change': '2023-05-20T01:19:50.473280+0000', 'last_active': '2023-05-20T01:19:50.473280+0000', 'last_peered': '2023-05-20T01:19:50.473280+0000', 'last_clean': '2023-05-20T01:19:50.473280+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:19:50.473280+0000', 'last_undegraded': '2023-05-20T01:19:50.473280+0000', 'last_fullsized': '2023-05-20T01:19:50.473280+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:19:50.473280+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:19:50.473280+0000', 'last_clean_scrub_stamp': '2023-05-20T01:19:50.473280+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:19:57.325 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.7 is not active+clean 2023-05-20T01:19:57.326 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:19:51.482477+0000', 'last_change': '2023-05-20T01:19:51.482477+0000', 'last_active': '2023-05-20T01:19:51.482477+0000', 'last_peered': '2023-05-20T01:19:51.482477+0000', 'last_clean': '2023-05-20T01:19:51.482477+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:19:51.482477+0000', 'last_undegraded': '2023-05-20T01:19:51.482477+0000', 'last_fullsized': '2023-05-20T01:19:51.482477+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_clean_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:19:57.326 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.4 is not active+clean 2023-05-20T01:19:57.327 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:19:50.473280+0000', 'last_change': '2023-05-20T01:19:50.473280+0000', 'last_active': '2023-05-20T01:19:50.473280+0000', 'last_peered': '2023-05-20T01:19:50.473280+0000', 'last_clean': '2023-05-20T01:19:50.473280+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:19:50.473280+0000', 'last_undegraded': '2023-05-20T01:19:50.473280+0000', 'last_fullsized': '2023-05-20T01:19:50.473280+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:19:50.473280+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:19:50.473280+0000', 'last_clean_scrub_stamp': '2023-05-20T01:19:50.473280+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:19:57.328 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.0 is not active+clean 2023-05-20T01:19:57.328 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:19:51.482477+0000', 'last_change': '2023-05-20T01:19:51.482477+0000', 'last_active': '2023-05-20T01:19:51.482477+0000', 'last_peered': '2023-05-20T01:19:51.482477+0000', 'last_clean': '2023-05-20T01:19:51.482477+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:19:51.482477+0000', 'last_undegraded': '2023-05-20T01:19:51.482477+0000', 'last_fullsized': '2023-05-20T01:19:51.482477+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_clean_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:19:57.329 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3 is not active+clean 2023-05-20T01:19:57.329 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:19:50.473280+0000', 'last_change': '2023-05-20T01:19:50.473280+0000', 'last_active': '2023-05-20T01:19:50.473280+0000', 'last_peered': '2023-05-20T01:19:50.473280+0000', 'last_clean': '2023-05-20T01:19:50.473280+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:19:50.473280+0000', 'last_undegraded': '2023-05-20T01:19:50.473280+0000', 'last_fullsized': '2023-05-20T01:19:50.473280+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:19:50.473280+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:19:50.473280+0000', 'last_clean_scrub_stamp': '2023-05-20T01:19:50.473280+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:19:57.330 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1 is not active+clean 2023-05-20T01:19:57.331 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:19:51.482477+0000', 'last_change': '2023-05-20T01:19:51.482477+0000', 'last_active': '2023-05-20T01:19:51.482477+0000', 'last_peered': '2023-05-20T01:19:51.482477+0000', 'last_clean': '2023-05-20T01:19:51.482477+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:19:51.482477+0000', 'last_undegraded': '2023-05-20T01:19:51.482477+0000', 'last_fullsized': '2023-05-20T01:19:51.482477+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_clean_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:19:57.331 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.2 is not active+clean 2023-05-20T01:19:57.332 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:19:50.473280+0000', 'last_change': '2023-05-20T01:19:50.473280+0000', 'last_active': '2023-05-20T01:19:50.473280+0000', 'last_peered': '2023-05-20T01:19:50.473280+0000', 'last_clean': '2023-05-20T01:19:50.473280+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:19:50.473280+0000', 'last_undegraded': '2023-05-20T01:19:50.473280+0000', 'last_fullsized': '2023-05-20T01:19:50.473280+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:19:50.473280+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:19:50.473280+0000', 'last_clean_scrub_stamp': '2023-05-20T01:19:50.473280+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:19:57.332 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2 is not active+clean 2023-05-20T01:19:57.333 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:19:51.482477+0000', 'last_change': '2023-05-20T01:19:51.482477+0000', 'last_active': '2023-05-20T01:19:51.482477+0000', 'last_peered': '2023-05-20T01:19:51.482477+0000', 'last_clean': '2023-05-20T01:19:51.482477+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:19:51.482477+0000', 'last_undegraded': '2023-05-20T01:19:51.482477+0000', 'last_fullsized': '2023-05-20T01:19:51.482477+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_clean_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:19:57.333 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1 is not active+clean 2023-05-20T01:19:57.333 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:19:50.473280+0000', 'last_change': '2023-05-20T01:19:50.473280+0000', 'last_active': '2023-05-20T01:19:50.473280+0000', 'last_peered': '2023-05-20T01:19:50.473280+0000', 'last_clean': '2023-05-20T01:19:50.473280+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:19:50.473280+0000', 'last_undegraded': '2023-05-20T01:19:50.473280+0000', 'last_fullsized': '2023-05-20T01:19:50.473280+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:19:50.473280+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:19:50.473280+0000', 'last_clean_scrub_stamp': '2023-05-20T01:19:50.473280+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:19:57.334 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3 is not active+clean 2023-05-20T01:19:57.334 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:19:51.482477+0000', 'last_change': '2023-05-20T01:19:51.482477+0000', 'last_active': '2023-05-20T01:19:51.482477+0000', 'last_peered': '2023-05-20T01:19:51.482477+0000', 'last_clean': '2023-05-20T01:19:51.482477+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:19:51.482477+0000', 'last_undegraded': '2023-05-20T01:19:51.482477+0000', 'last_fullsized': '2023-05-20T01:19:51.482477+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_clean_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:19:57.334 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.0 is not active+clean 2023-05-20T01:19:57.335 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:19:50.473280+0000', 'last_change': '2023-05-20T01:19:50.473280+0000', 'last_active': '2023-05-20T01:19:50.473280+0000', 'last_peered': '2023-05-20T01:19:50.473280+0000', 'last_clean': '2023-05-20T01:19:50.473280+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:19:50.473280+0000', 'last_undegraded': '2023-05-20T01:19:50.473280+0000', 'last_fullsized': '2023-05-20T01:19:50.473280+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:19:50.473280+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:19:50.473280+0000', 'last_clean_scrub_stamp': '2023-05-20T01:19:50.473280+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:19:57.335 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.e is not active+clean 2023-05-20T01:19:57.335 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:19:51.482477+0000', 'last_change': '2023-05-20T01:19:51.482477+0000', 'last_active': '2023-05-20T01:19:51.482477+0000', 'last_peered': '2023-05-20T01:19:51.482477+0000', 'last_clean': '2023-05-20T01:19:51.482477+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:19:51.482477+0000', 'last_undegraded': '2023-05-20T01:19:51.482477+0000', 'last_fullsized': '2023-05-20T01:19:51.482477+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_clean_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:19:57.336 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.d is not active+clean 2023-05-20T01:19:57.336 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:19:50.473280+0000', 'last_change': '2023-05-20T01:19:50.473280+0000', 'last_active': '2023-05-20T01:19:50.473280+0000', 'last_peered': '2023-05-20T01:19:50.473280+0000', 'last_clean': '2023-05-20T01:19:50.473280+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:19:50.473280+0000', 'last_undegraded': '2023-05-20T01:19:50.473280+0000', 'last_fullsized': '2023-05-20T01:19:50.473280+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:19:50.473280+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:19:50.473280+0000', 'last_clean_scrub_stamp': '2023-05-20T01:19:50.473280+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:19:57.336 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.f is not active+clean 2023-05-20T01:19:57.337 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:19:51.482477+0000', 'last_change': '2023-05-20T01:19:51.482477+0000', 'last_active': '2023-05-20T01:19:51.482477+0000', 'last_peered': '2023-05-20T01:19:51.482477+0000', 'last_clean': '2023-05-20T01:19:51.482477+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:19:51.482477+0000', 'last_undegraded': '2023-05-20T01:19:51.482477+0000', 'last_fullsized': '2023-05-20T01:19:51.482477+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_clean_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:19:57.337 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.c is not active+clean 2023-05-20T01:19:57.337 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:19:50.473280+0000', 'last_change': '2023-05-20T01:19:50.473280+0000', 'last_active': '2023-05-20T01:19:50.473280+0000', 'last_peered': '2023-05-20T01:19:50.473280+0000', 'last_clean': '2023-05-20T01:19:50.473280+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:19:50.473280+0000', 'last_undegraded': '2023-05-20T01:19:50.473280+0000', 'last_fullsized': '2023-05-20T01:19:50.473280+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:19:50.473280+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:19:50.473280+0000', 'last_clean_scrub_stamp': '2023-05-20T01:19:50.473280+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:19:57.338 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.8 is not active+clean 2023-05-20T01:19:57.338 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:19:51.482477+0000', 'last_change': '2023-05-20T01:19:51.482477+0000', 'last_active': '2023-05-20T01:19:51.482477+0000', 'last_peered': '2023-05-20T01:19:51.482477+0000', 'last_clean': '2023-05-20T01:19:51.482477+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:19:51.482477+0000', 'last_undegraded': '2023-05-20T01:19:51.482477+0000', 'last_fullsized': '2023-05-20T01:19:51.482477+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_clean_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:19:57.338 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.b is not active+clean 2023-05-20T01:19:57.339 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:19:50.473280+0000', 'last_change': '2023-05-20T01:19:50.473280+0000', 'last_active': '2023-05-20T01:19:50.473280+0000', 'last_peered': '2023-05-20T01:19:50.473280+0000', 'last_clean': '2023-05-20T01:19:50.473280+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:19:50.473280+0000', 'last_undegraded': '2023-05-20T01:19:50.473280+0000', 'last_fullsized': '2023-05-20T01:19:50.473280+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:19:50.473280+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:19:50.473280+0000', 'last_clean_scrub_stamp': '2023-05-20T01:19:50.473280+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:19:57.340 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.9 is not active+clean 2023-05-20T01:19:57.341 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:19:51.482477+0000', 'last_change': '2023-05-20T01:19:51.482477+0000', 'last_active': '2023-05-20T01:19:51.482477+0000', 'last_peered': '2023-05-20T01:19:51.482477+0000', 'last_clean': '2023-05-20T01:19:51.482477+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:19:51.482477+0000', 'last_undegraded': '2023-05-20T01:19:51.482477+0000', 'last_fullsized': '2023-05-20T01:19:51.482477+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_clean_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:19:57.341 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.a is not active+clean 2023-05-20T01:19:57.342 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:19:50.473280+0000', 'last_change': '2023-05-20T01:19:50.473280+0000', 'last_active': '2023-05-20T01:19:50.473280+0000', 'last_peered': '2023-05-20T01:19:50.473280+0000', 'last_clean': '2023-05-20T01:19:50.473280+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:19:50.473280+0000', 'last_undegraded': '2023-05-20T01:19:50.473280+0000', 'last_fullsized': '2023-05-20T01:19:50.473280+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:19:50.473280+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:19:50.473280+0000', 'last_clean_scrub_stamp': '2023-05-20T01:19:50.473280+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:19:57.343 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.b is not active+clean 2023-05-20T01:19:57.344 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:19:51.482477+0000', 'last_change': '2023-05-20T01:19:51.482477+0000', 'last_active': '2023-05-20T01:19:51.482477+0000', 'last_peered': '2023-05-20T01:19:51.482477+0000', 'last_clean': '2023-05-20T01:19:51.482477+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:19:51.482477+0000', 'last_undegraded': '2023-05-20T01:19:51.482477+0000', 'last_fullsized': '2023-05-20T01:19:51.482477+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_clean_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:19:57.344 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.8 is not active+clean 2023-05-20T01:19:57.345 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:19:50.473280+0000', 'last_change': '2023-05-20T01:19:50.473280+0000', 'last_active': '2023-05-20T01:19:50.473280+0000', 'last_peered': '2023-05-20T01:19:50.473280+0000', 'last_clean': '2023-05-20T01:19:50.473280+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:19:50.473280+0000', 'last_undegraded': '2023-05-20T01:19:50.473280+0000', 'last_fullsized': '2023-05-20T01:19:50.473280+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:19:50.473280+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:19:50.473280+0000', 'last_clean_scrub_stamp': '2023-05-20T01:19:50.473280+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:19:57.346 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.a is not active+clean 2023-05-20T01:19:57.346 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:19:51.482477+0000', 'last_change': '2023-05-20T01:19:51.482477+0000', 'last_active': '2023-05-20T01:19:51.482477+0000', 'last_peered': '2023-05-20T01:19:51.482477+0000', 'last_clean': '2023-05-20T01:19:51.482477+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:19:51.482477+0000', 'last_undegraded': '2023-05-20T01:19:51.482477+0000', 'last_fullsized': '2023-05-20T01:19:51.482477+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_clean_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:19:57.347 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.9 is not active+clean 2023-05-20T01:19:57.347 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:19:50.473280+0000', 'last_change': '2023-05-20T01:19:50.473280+0000', 'last_active': '2023-05-20T01:19:50.473280+0000', 'last_peered': '2023-05-20T01:19:50.473280+0000', 'last_clean': '2023-05-20T01:19:50.473280+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:19:50.473280+0000', 'last_undegraded': '2023-05-20T01:19:50.473280+0000', 'last_fullsized': '2023-05-20T01:19:50.473280+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:19:50.473280+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:19:50.473280+0000', 'last_clean_scrub_stamp': '2023-05-20T01:19:50.473280+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:19:57.348 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.d is not active+clean 2023-05-20T01:19:57.349 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:19:51.482477+0000', 'last_change': '2023-05-20T01:19:51.482477+0000', 'last_active': '2023-05-20T01:19:51.482477+0000', 'last_peered': '2023-05-20T01:19:51.482477+0000', 'last_clean': '2023-05-20T01:19:51.482477+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:19:51.482477+0000', 'last_undegraded': '2023-05-20T01:19:51.482477+0000', 'last_fullsized': '2023-05-20T01:19:51.482477+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_clean_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:19:57.349 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.e is not active+clean 2023-05-20T01:19:57.349 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:19:50.473280+0000', 'last_change': '2023-05-20T01:19:50.473280+0000', 'last_active': '2023-05-20T01:19:50.473280+0000', 'last_peered': '2023-05-20T01:19:50.473280+0000', 'last_clean': '2023-05-20T01:19:50.473280+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:19:50.473280+0000', 'last_undegraded': '2023-05-20T01:19:50.473280+0000', 'last_fullsized': '2023-05-20T01:19:50.473280+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:19:50.473280+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:19:50.473280+0000', 'last_clean_scrub_stamp': '2023-05-20T01:19:50.473280+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:19:57.350 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.c is not active+clean 2023-05-20T01:19:57.350 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:19:51.482477+0000', 'last_change': '2023-05-20T01:19:51.482477+0000', 'last_active': '2023-05-20T01:19:51.482477+0000', 'last_peered': '2023-05-20T01:19:51.482477+0000', 'last_clean': '2023-05-20T01:19:51.482477+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:19:51.482477+0000', 'last_undegraded': '2023-05-20T01:19:51.482477+0000', 'last_fullsized': '2023-05-20T01:19:51.482477+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'last_clean_scrub_stamp': '2023-05-20T01:19:51.482477+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:19:57.350 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.f is not active+clean 2023-05-20T01:19:57.351 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:19:50.473280+0000', 'last_change': '2023-05-20T01:19:50.473280+0000', 'last_active': '2023-05-20T01:19:50.473280+0000', 'last_peered': '2023-05-20T01:19:50.473280+0000', 'last_clean': '2023-05-20T01:19:50.473280+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:19:50.473280+0000', 'last_undegraded': '2023-05-20T01:19:50.473280+0000', 'last_fullsized': '2023-05-20T01:19:50.473280+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:19:50.473280+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:19:50.473280+0000', 'last_clean_scrub_stamp': '2023-05-20T01:19:50.473280+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:19:57.351 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-05-20T01:19:57.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.472+0000 7f519651a700 1 Processor -- start 2023-05-20T01:19:57.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.472+0000 7f519651a700 1 -- start start 2023-05-20T01:19:57.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.473+0000 7f519651a700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f51900cb320 0x7f5190070910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:57.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.473+0000 7f519651a700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51900ca350 con 0x7f519006d2a0 2023-05-20T01:19:57.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.473+0000 7f519651a700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f519015cac0 con 0x7f5190159000 2023-05-20T01:19:57.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.473+0000 7f519651a700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f519015cda0 con 0x7f51900cb320 2023-05-20T01:19:57.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.473+0000 7f5194ab7700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f519006d2a0 0x7f5190070e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:45442/0 (socket says 172.21.15.16:45442) 2023-05-20T01:19:57.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.473+0000 7f5194ab7700 1 -- 172.21.15.16:0/4214207072 learned_addr learned my addr 172.21.15.16:0/4214207072 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:19:57.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.474+0000 7f518f7fe700 1 --2- 172.21.15.16:0/4214207072 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f51900cb320 0x7f5190070910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:57.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.474+0000 7f518effd700 1 -- 172.21.15.16:0/4214207072 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4102319282 0 0) 0x7f51900ca350 con 0x7f519006d2a0 2023-05-20T01:19:57.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.474+0000 7f518effd700 1 -- 172.21.15.16:0/4214207072 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51780035b0 con 0x7f519006d2a0 2023-05-20T01:19:57.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.474+0000 7f518effd700 1 -- 172.21.15.16:0/4214207072 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3167740948 0 0) 0x7f519015cac0 con 0x7f5190159000 2023-05-20T01:19:57.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.474+0000 7f518effd700 1 -- 172.21.15.16:0/4214207072 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51900ca350 con 0x7f5190159000 2023-05-20T01:19:57.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.474+0000 7f518effd700 1 -- 172.21.15.16:0/4214207072 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1271263809 0 0) 0x7f51780035b0 con 0x7f519006d2a0 2023-05-20T01:19:57.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.474+0000 7f518effd700 1 -- 172.21.15.16:0/4214207072 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f51900cb320 msgr2=0x7f5190070910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:19:57.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.474+0000 7f518effd700 1 --2- 172.21.15.16:0/4214207072 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f51900cb320 0x7f5190070910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:57.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.474+0000 7f518effd700 1 -- 172.21.15.16:0/4214207072 >> v1:172.21.15.132:6789/0 conn(0x7f5190159000 legacy=0x7f519006cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:57.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.474+0000 7f518effd700 1 -- 172.21.15.16:0/4214207072 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f519015d080 con 0x7f519006d2a0 2023-05-20T01:19:57.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.474+0000 7f518effd700 1 -- 172.21.15.16:0/4214207072 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7f51880019f0 con 0x7f519006d2a0 2023-05-20T01:19:57.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.474+0000 7f518effd700 1 -- 172.21.15.16:0/4214207072 >> v1:172.21.15.16:6789/0 conn(0x7f519006d2a0 legacy=0x7f5190070e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:57.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.474+0000 7f518effd700 1 --2- 172.21.15.16:0/4214207072 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5178003c30 0x7f5178004020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:57.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.474+0000 7f518effd700 1 --2- 172.21.15.16:0/4214207072 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f51900cb320 0x7f5178004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:57.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.474+0000 7f518effd700 1 --2- 172.21.15.16:0/4214207072 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5178004d90 0x7f5178005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:57.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.474+0000 7f518effd700 1 -- 172.21.15.16:0/4214207072 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f519015cda0 con 0x7f5178004d90 2023-05-20T01:19:57.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.474+0000 7f518effd700 1 -- 172.21.15.16:0/4214207072 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f517800dd30 con 0x7f5178003c30 2023-05-20T01:19:57.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.474+0000 7f518effd700 1 -- 172.21.15.16:0/4214207072 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f517800def0 con 0x7f51900cb320 2023-05-20T01:19:57.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.474+0000 7f5194ab7700 1 --2- 172.21.15.16:0/4214207072 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5178004d90 0x7f5178005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:57.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.474+0000 7f518ffff700 1 --2- 172.21.15.16:0/4214207072 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5178003c30 0x7f5178004020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:57.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.474+0000 7f518f7fe700 1 --2- 172.21.15.16:0/4214207072 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f51900cb320 0x7f5178004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:57.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.474+0000 7f5194ab7700 1 -- 172.21.15.16:0/4214207072 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f51900cb320 msgr2=0x7f5178004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:19:57.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.474+0000 7f5194ab7700 1 --2- 172.21.15.16:0/4214207072 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f51900cb320 0x7f5178004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:57.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.474+0000 7f5194ab7700 1 -- 172.21.15.16:0/4214207072 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5178003c30 msgr2=0x7f5178004020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:19:57.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.474+0000 7f5194ab7700 1 --2- 172.21.15.16:0/4214207072 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5178003c30 0x7f5178004020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:57.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.474+0000 7f5194ab7700 1 -- 172.21.15.16:0/4214207072 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51780035b0 con 0x7f5178004d90 2023-05-20T01:19:57.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.475+0000 7f5194ab7700 1 --2- 172.21.15.16:0/4214207072 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5178004d90 0x7f5178005200 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f5190072c00 tx=0x7f5178005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fd360e1bdafc4c1 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:57.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.475+0000 7f518effd700 1 -- 172.21.15.16:0/4214207072 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51880072d0 con 0x7f5178004d90 2023-05-20T01:19:57.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.475+0000 7f518effd700 1 -- 172.21.15.16:0/4214207072 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5188007430 con 0x7f5178004d90 2023-05-20T01:19:57.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.475+0000 7f518effd700 1 -- 172.21.15.16:0/4214207072 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51880077e0 con 0x7f5178004d90 2023-05-20T01:19:57.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.475+0000 7f519651a700 1 -- 172.21.15.16:0/4214207072 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5178004d90 msgr2=0x7f5178005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:57.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.475+0000 7f519651a700 1 --2- 172.21.15.16:0/4214207072 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5178004d90 0x7f5178005200 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f5190072c00 tx=0x7f5178005880 comp rx=0 tx=0).stop 2023-05-20T01:19:57.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.475+0000 7f5194ab7700 1 -- 172.21.15.16:0/4214207072 reap_dead start 2023-05-20T01:19:57.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.475+0000 7f519651a700 1 -- 172.21.15.16:0/4214207072 shutdown_connections 2023-05-20T01:19:57.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.475+0000 7f519651a700 1 -- 172.21.15.16:0/4214207072 >> 172.21.15.16:0/4214207072 conn(0x7f51900661b0 msgr2=0x7f5190066580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:19:57.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.475+0000 7f519651a700 1 -- 172.21.15.16:0/4214207072 shutdown_connections 2023-05-20T01:19:57.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.475+0000 7f519651a700 1 -- 172.21.15.16:0/4214207072 wait complete. 2023-05-20T01:19:57.486 INFO:tasks.ceph.mds.c.smithi016.stderr:2023-05-20T01:19:57.476+0000 7f1c5ee44700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-05-20T01:19:57.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.477+0000 7f519651a700 1 Processor -- start 2023-05-20T01:19:57.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.477+0000 7f519651a700 1 -- start start 2023-05-20T01:19:57.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.477+0000 7f519651a700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f51901556c0 0x7f5190143e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:57.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.478+0000 7f519651a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5190155a90 0x7f51901443a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:57.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.478+0000 7f519651a700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f51901ef6d0 0x7f51901f2040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:57.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.478+0000 7f519651a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51901eea00 con 0x7f5190155a90 2023-05-20T01:19:57.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.478+0000 7f519651a700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5190147c80 con 0x7f51901ef6d0 2023-05-20T01:19:57.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.478+0000 7f519651a700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51901f2f70 con 0x7f51901556c0 2023-05-20T01:19:57.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.478+0000 7f518f7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5190155a90 0x7f51901443a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:57.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.478+0000 7f518f7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5190155a90 0x7f51901443a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33924/0 (socket says 172.21.15.16:33924) 2023-05-20T01:19:57.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.478+0000 7f518f7fe700 1 -- 172.21.15.16:0/3033806639 learned_addr learned my addr 172.21.15.16:0/3033806639 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:19:57.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.478+0000 7f5194ab7700 1 --2- 172.21.15.16:0/3033806639 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f51901ef6d0 0x7f51901f2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:57.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.478+0000 7f518ffff700 1 --2- 172.21.15.16:0/3033806639 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f51901556c0 0x7f5190143e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:57.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.478+0000 7f518f7fe700 1 -- 172.21.15.16:0/3033806639 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f51901556c0 msgr2=0x7f5190143e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:57.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.478+0000 7f518f7fe700 1 --2- 172.21.15.16:0/3033806639 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f51901556c0 0x7f5190143e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:57.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.478+0000 7f518f7fe700 1 -- 172.21.15.16:0/3033806639 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f51901ef6d0 msgr2=0x7f51901f2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:57.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.478+0000 7f518f7fe700 1 --2- 172.21.15.16:0/3033806639 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f51901ef6d0 0x7f51901f2040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:57.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.478+0000 7f518f7fe700 1 -- 172.21.15.16:0/3033806639 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f519015d880 con 0x7f5190155a90 2023-05-20T01:19:57.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.478+0000 7f518ffff700 1 --2- 172.21.15.16:0/3033806639 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f51901556c0 0x7f5190143e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:19:57.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.478+0000 7f518f7fe700 1 --2- 172.21.15.16:0/3033806639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5190155a90 0x7f51901443a0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f51840129f0 tx=0x7f5184012d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:57.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.478+0000 7f518cff9700 1 -- 172.21.15.16:0/3033806639 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5184012ee0 con 0x7f5190155a90 2023-05-20T01:19:57.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.478+0000 7f518cff9700 1 -- 172.21.15.16:0/3033806639 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5184004440 con 0x7f5190155a90 2023-05-20T01:19:57.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.478+0000 7f519651a700 1 -- 172.21.15.16:0/3033806639 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f519015d360 con 0x7f5190155a90 2023-05-20T01:19:57.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.478+0000 7f518cff9700 1 -- 172.21.15.16:0/3033806639 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51840045b0 con 0x7f5190155a90 2023-05-20T01:19:57.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.478+0000 7f519651a700 1 -- 172.21.15.16:0/3033806639 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51900b1b00 con 0x7f5190155a90 2023-05-20T01:19:57.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.479+0000 7f519651a700 1 -- 172.21.15.16:0/3033806639 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51900747b0 con 0x7f5190155a90 2023-05-20T01:19:57.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.480+0000 7f518cff9700 1 -- 172.21.15.16:0/3033806639 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5184019070 con 0x7f5190155a90 2023-05-20T01:19:57.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.480+0000 7f518cff9700 1 --2- 172.21.15.16:0/3033806639 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f517c06ace0 0x7f517c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:57.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.480+0000 7f518cff9700 1 -- 172.21.15.16:0/3033806639 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f5184089e20 con 0x7f5190155a90 2023-05-20T01:19:57.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.480+0000 7f518ffff700 1 --2- 172.21.15.16:0/3033806639 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f517c06ace0 0x7f517c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:57.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.481+0000 7f518ffff700 1 --2- 172.21.15.16:0/3033806639 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f517c06ace0 0x7f517c06d190 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f5180007fd0 tx=0x7f5180009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:57.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.485+0000 7f518cff9700 1 -- 172.21.15.16:0/3033806639 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f5184054e20 con 0x7f5190155a90 2023-05-20T01:19:57.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.765+0000 7f519651a700 1 -- 172.21.15.16:0/3033806639 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f5190072230 con 0x7f5190155a90 2023-05-20T01:19:57.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.765+0000 7f518cff9700 1 -- 172.21.15.16:0/3033806639 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1183 (secure 0 0 0) 0x7f51840214b0 con 0x7f5190155a90 2023-05-20T01:19:57.769 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:19:57.769 INFO:teuthology.orchestra.run.smithi016.stdout:{"fsid":"c2245cc5-ecab-4087-a59b-bc81db901676","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":228,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":64,"num_osds":8,"num_up_osds":8,"osd_up_since":1684545379,"num_in_osds":8,"osd_in_since":1684545372,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":96},{"state_name":"active+clean","count":9}],"num_pgs":105,"num_pools":4,"num_objects":2,"data_bytes":590368,"bytes_used":79036416,"bytes_avail":773015076864,"bytes_total":773094113280,"unknown_pgs_ratio":0.91428571939468384},"fsmap":{"epoch":37,"id":3,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":3,"rank":0,"name":"c","status":"up:active","gid":6195}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":9,"modified":"2023-05-20T01:19:33.358917+0000","services":{"mds":{"daemons":{"summary":"","c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-05-20T01:19:57.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.767+0000 7f519651a700 1 -- 172.21.15.16:0/3033806639 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f517c06ace0 msgr2=0x7f517c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:57.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.767+0000 7f519651a700 1 --2- 172.21.15.16:0/3033806639 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f517c06ace0 0x7f517c06d190 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f5180007fd0 tx=0x7f5180009040 comp rx=0 tx=0).stop 2023-05-20T01:19:57.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.767+0000 7f519651a700 1 -- 172.21.15.16:0/3033806639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5190155a90 msgr2=0x7f51901443a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:57.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.767+0000 7f519651a700 1 --2- 172.21.15.16:0/3033806639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5190155a90 0x7f51901443a0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f51840129f0 tx=0x7f5184012d00 comp rx=0 tx=0).stop 2023-05-20T01:19:57.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.767+0000 7f519651a700 1 -- 172.21.15.16:0/3033806639 shutdown_connections 2023-05-20T01:19:57.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.767+0000 7f519651a700 1 --2- 172.21.15.16:0/3033806639 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f51901ef6d0 0x7f51901f2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:57.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.767+0000 7f519651a700 1 --2- 172.21.15.16:0/3033806639 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f517c06ace0 0x7f517c06d190 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:57.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.767+0000 7f519651a700 1 --2- 172.21.15.16:0/3033806639 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f51901556c0 0x7f5190143e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:57.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.767+0000 7f519651a700 1 --2- 172.21.15.16:0/3033806639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5190155a90 0x7f51901443a0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:57.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.767+0000 7f519651a700 1 -- 172.21.15.16:0/3033806639 >> 172.21.15.16:0/3033806639 conn(0x7f51900661b0 msgr2=0x7f51900ca4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:19:57.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.767+0000 7f519651a700 1 -- 172.21.15.16:0/3033806639 shutdown_connections 2023-05-20T01:19:57.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.767+0000 7f519651a700 1 -- 172.21.15.16:0/3033806639 wait complete. 2023-05-20T01:19:57.783 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-05-20T01:19:57.784 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-05-20T01:19:57.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.905+0000 7f1d5906e700 1 Processor -- start 2023-05-20T01:19:57.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.905+0000 7f1d5906e700 1 -- start start 2023-05-20T01:19:57.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.906+0000 7f1d5906e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1d5406ac20 0x7f1d540cf890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:57.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.906+0000 7f1d5906e700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d540cc510 con 0x7f1d5406d4b0 2023-05-20T01:19:57.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.906+0000 7f1d5906e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d54071ed0 con 0x7f1d5406f450 2023-05-20T01:19:57.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.906+0000 7f1d5906e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d540721b0 con 0x7f1d5406ac20 2023-05-20T01:19:57.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.906+0000 7f1d5259c700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f1d5406d4b0 0x7f1d5406a510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:45458/0 (socket says 172.21.15.16:45458) 2023-05-20T01:19:57.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.906+0000 7f1d5259c700 1 -- 172.21.15.16:0/1702565659 learned_addr learned my addr 172.21.15.16:0/1702565659 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:19:57.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.907+0000 7f1d5359e700 1 --2- 172.21.15.16:0/1702565659 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1d5406ac20 0x7f1d540cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:57.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.907+0000 7f1d51d9b700 1 -- 172.21.15.16:0/1702565659 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2677656560 0 0) 0x7f1d540cc510 con 0x7f1d5406d4b0 2023-05-20T01:19:57.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.907+0000 7f1d51d9b700 1 -- 172.21.15.16:0/1702565659 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d400035b0 con 0x7f1d5406d4b0 2023-05-20T01:19:57.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.907+0000 7f1d51d9b700 1 -- 172.21.15.16:0/1702565659 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2627070257 0 0) 0x7f1d400035b0 con 0x7f1d5406d4b0 2023-05-20T01:19:57.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.907+0000 7f1d51d9b700 1 -- 172.21.15.16:0/1702565659 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1d5406ac20 msgr2=0x7f1d540cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:19:57.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.907+0000 7f1d51d9b700 1 --2- 172.21.15.16:0/1702565659 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1d5406ac20 0x7f1d540cf890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:57.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.907+0000 7f1d51d9b700 1 -- 172.21.15.16:0/1702565659 >> v1:172.21.15.132:6789/0 conn(0x7f1d5406f450 legacy=0x7f1d54161080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:57.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.907+0000 7f1d51d9b700 1 -- 172.21.15.16:0/1702565659 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d540cc1a0 con 0x7f1d5406d4b0 2023-05-20T01:19:57.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.907+0000 7f1d51d9b700 1 -- 172.21.15.16:0/1702565659 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7f1d3c001670 con 0x7f1d5406d4b0 2023-05-20T01:19:57.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.907+0000 7f1d51d9b700 1 -- 172.21.15.16:0/1702565659 >> v1:172.21.15.16:6789/0 conn(0x7f1d5406d4b0 legacy=0x7f1d5406a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:57.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.907+0000 7f1d51d9b700 1 --2- 172.21.15.16:0/1702565659 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d40003aa0 0x7f1d40003eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:57.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.907+0000 7f1d51d9b700 1 --2- 172.21.15.16:0/1702565659 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1d5406ac20 0x7f1d40004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:57.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.907+0000 7f1d51d9b700 1 --2- 172.21.15.16:0/1702565659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d40004d90 0x7f1d40005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:57.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.907+0000 7f1d51d9b700 1 -- 172.21.15.16:0/1702565659 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d540721b0 con 0x7f1d40004d90 2023-05-20T01:19:57.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.907+0000 7f1d51d9b700 1 -- 172.21.15.16:0/1702565659 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d4000bbf0 con 0x7f1d40003aa0 2023-05-20T01:19:57.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.907+0000 7f1d51d9b700 1 -- 172.21.15.16:0/1702565659 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d4000bdb0 con 0x7f1d5406ac20 2023-05-20T01:19:57.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.907+0000 7f1d5359e700 1 --2- 172.21.15.16:0/1702565659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d40004d90 0x7f1d40005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:57.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.907+0000 7f1d5259c700 1 --2- 172.21.15.16:0/1702565659 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1d5406ac20 0x7f1d40004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:57.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.907+0000 7f1d52d9d700 1 --2- 172.21.15.16:0/1702565659 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d40003aa0 0x7f1d40003eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:57.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.907+0000 7f1d5359e700 1 -- 172.21.15.16:0/1702565659 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1d5406ac20 msgr2=0x7f1d40004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:19:57.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.907+0000 7f1d5359e700 1 --2- 172.21.15.16:0/1702565659 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1d5406ac20 0x7f1d40004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:57.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.907+0000 7f1d5359e700 1 -- 172.21.15.16:0/1702565659 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d40003aa0 msgr2=0x7f1d40003eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:19:57.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.907+0000 7f1d5359e700 1 --2- 172.21.15.16:0/1702565659 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d40003aa0 0x7f1d40003eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:57.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.908+0000 7f1d5359e700 1 -- 172.21.15.16:0/1702565659 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d400035b0 con 0x7f1d40004d90 2023-05-20T01:19:57.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.908+0000 7f1d52d9d700 1 --2- 172.21.15.16:0/1702565659 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d40003aa0 0x7f1d40003eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:19:57.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.908+0000 7f1d5359e700 1 --2- 172.21.15.16:0/1702565659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d40004d90 0x7f1d40005200 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f1d540c7030 tx=0x7f1d40005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31ec0dfcb96538a4 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:57.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.908+0000 7f1d51d9b700 1 -- 172.21.15.16:0/1702565659 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d4c00ebd0 con 0x7f1d40004d90 2023-05-20T01:19:57.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.908+0000 7f1d51d9b700 1 -- 172.21.15.16:0/1702565659 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d4c00f040 con 0x7f1d40004d90 2023-05-20T01:19:57.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.908+0000 7f1d51d9b700 1 -- 172.21.15.16:0/1702565659 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d4c014480 con 0x7f1d40004d90 2023-05-20T01:19:57.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.908+0000 7f1d5906e700 1 -- 172.21.15.16:0/1702565659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d40004d90 msgr2=0x7f1d40005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:57.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.908+0000 7f1d5906e700 1 --2- 172.21.15.16:0/1702565659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d40004d90 0x7f1d40005200 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f1d540c7030 tx=0x7f1d40005880 comp rx=0 tx=0).stop 2023-05-20T01:19:57.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.908+0000 7f1d5359e700 1 -- 172.21.15.16:0/1702565659 reap_dead start 2023-05-20T01:19:57.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.908+0000 7f1d5906e700 1 -- 172.21.15.16:0/1702565659 shutdown_connections 2023-05-20T01:19:57.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.908+0000 7f1d5906e700 1 -- 172.21.15.16:0/1702565659 >> 172.21.15.16:0/1702565659 conn(0x7f1d540cffb0 msgr2=0x7f1d540d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:19:57.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.908+0000 7f1d5906e700 1 -- 172.21.15.16:0/1702565659 shutdown_connections 2023-05-20T01:19:57.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.908+0000 7f1d5906e700 1 -- 172.21.15.16:0/1702565659 wait complete. 2023-05-20T01:19:57.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.910+0000 7f1d5906e700 1 Processor -- start 2023-05-20T01:19:57.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.910+0000 7f1d5906e700 1 -- start start 2023-05-20T01:19:57.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.911+0000 7f1d5906e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d54155510 0x7f1d540c7a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:57.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.911+0000 7f1d5906e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1d541558e0 0x7f1d540c7f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:57.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.911+0000 7f1d5906e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d540b3cc0 0x7f1d540b4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:57.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.911+0000 7f1d5906e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d540b5b50 con 0x7f1d540b3cc0 2023-05-20T01:19:57.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.911+0000 7f1d5906e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d540c4890 con 0x7f1d54155510 2023-05-20T01:19:57.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.911+0000 7f1d5906e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d540c3c80 con 0x7f1d541558e0 2023-05-20T01:19:57.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.911+0000 7f1d5359e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d540b3cc0 0x7f1d540b4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:57.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.911+0000 7f1d5359e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d540b3cc0 0x7f1d540b4140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33946/0 (socket says 172.21.15.16:33946) 2023-05-20T01:19:57.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.911+0000 7f1d5359e700 1 -- 172.21.15.16:0/4279535995 learned_addr learned my addr 172.21.15.16:0/4279535995 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:19:57.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.911+0000 7f1d52d9d700 1 --2- 172.21.15.16:0/4279535995 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d54155510 0x7f1d540c7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:57.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.911+0000 7f1d5359e700 1 -- 172.21.15.16:0/4279535995 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1d541558e0 msgr2=0x7f1d540c7f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:57.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.911+0000 7f1d5359e700 1 --2- 172.21.15.16:0/4279535995 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1d541558e0 0x7f1d540c7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:57.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.911+0000 7f1d5359e700 1 -- 172.21.15.16:0/4279535995 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d54155510 msgr2=0x7f1d540c7a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:57.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.911+0000 7f1d5359e700 1 --2- 172.21.15.16:0/4279535995 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d54155510 0x7f1d540c7a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:57.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.911+0000 7f1d5359e700 1 -- 172.21.15.16:0/4279535995 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d540cf890 con 0x7f1d540b3cc0 2023-05-20T01:19:57.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.911+0000 7f1d52d9d700 1 --2- 172.21.15.16:0/4279535995 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d54155510 0x7f1d540c7a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:19:57.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.911+0000 7f1d5359e700 1 --2- 172.21.15.16:0/4279535995 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d540b3cc0 0x7f1d540b4140 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f1d4000bbc0 tx=0x7f1d4c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:57.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.911+0000 7f1d4b7fe700 1 -- 172.21.15.16:0/4279535995 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d4c007000 con 0x7f1d540b3cc0 2023-05-20T01:19:57.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.911+0000 7f1d4b7fe700 1 -- 172.21.15.16:0/4279535995 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d4c022070 con 0x7f1d540b3cc0 2023-05-20T01:19:57.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.911+0000 7f1d5906e700 1 -- 172.21.15.16:0/4279535995 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d540cbe50 con 0x7f1d540b3cc0 2023-05-20T01:19:57.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.911+0000 7f1d4b7fe700 1 -- 172.21.15.16:0/4279535995 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d4c015920 con 0x7f1d540b3cc0 2023-05-20T01:19:57.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.912+0000 7f1d5906e700 1 -- 172.21.15.16:0/4279535995 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d540cfa20 con 0x7f1d540b3cc0 2023-05-20T01:19:57.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.913+0000 7f1d5906e700 1 -- 172.21.15.16:0/4279535995 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d540747b0 con 0x7f1d540b3cc0 2023-05-20T01:19:57.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.913+0000 7f1d4b7fe700 1 -- 172.21.15.16:0/4279535995 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1d4c024030 con 0x7f1d540b3cc0 2023-05-20T01:19:57.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.913+0000 7f1d4b7fe700 1 --2- 172.21.15.16:0/4279535995 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f1d3406ace0 0x7f1d3406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:19:57.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.913+0000 7f1d4b7fe700 1 -- 172.21.15.16:0/4279535995 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f1d4c08ae60 con 0x7f1d540b3cc0 2023-05-20T01:19:57.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.913+0000 7f1d52d9d700 1 --2- 172.21.15.16:0/4279535995 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f1d3406ace0 0x7f1d3406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:19:57.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.914+0000 7f1d52d9d700 1 --2- 172.21.15.16:0/4279535995 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f1d3406ace0 0x7f1d3406d190 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f1d44000c00 tx=0x7f1d44009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:19:57.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:57.918+0000 7f1d4b7fe700 1 -- 172.21.15.16:0/4279535995 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f1d4c055f80 con 0x7f1d540b3cc0 2023-05-20T01:19:58.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:58.109+0000 7f1d5906e700 1 -- 172.21.15.16:0/4279535995 --> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1d540c49d0 con 0x7f1d3406ace0 2023-05-20T01:19:58.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:58.118+0000 7f1d4b7fe700 1 -- 172.21.15.16:0/4279535995 <== mgr.4104 v2:172.21.15.16:6832/3817237998 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311841 (secure 0 0 0) 0x7f1d540c49d0 con 0x7f1d3406ace0 2023-05-20T01:19:58.126 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:19:58.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:58.121+0000 7f1d5906e700 1 -- 172.21.15.16:0/4279535995 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f1d3406ace0 msgr2=0x7f1d3406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:58.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:58.121+0000 7f1d5906e700 1 --2- 172.21.15.16:0/4279535995 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f1d3406ace0 0x7f1d3406d190 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f1d44000c00 tx=0x7f1d44009040 comp rx=0 tx=0).stop 2023-05-20T01:19:58.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:58.121+0000 7f1d5906e700 1 -- 172.21.15.16:0/4279535995 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d540b3cc0 msgr2=0x7f1d540b4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:19:58.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:58.121+0000 7f1d5906e700 1 --2- 172.21.15.16:0/4279535995 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d540b3cc0 0x7f1d540b4140 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f1d4000bbc0 tx=0x7f1d4c009750 comp rx=0 tx=0).stop 2023-05-20T01:19:58.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:58.121+0000 7f1d5906e700 1 -- 172.21.15.16:0/4279535995 shutdown_connections 2023-05-20T01:19:58.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:58.121+0000 7f1d5906e700 1 --2- 172.21.15.16:0/4279535995 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d540b3cc0 0x7f1d540b4140 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:58.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:58.121+0000 7f1d5906e700 1 --2- 172.21.15.16:0/4279535995 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d54155510 0x7f1d540c7a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:58.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:58.121+0000 7f1d5906e700 1 --2- 172.21.15.16:0/4279535995 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f1d3406ace0 0x7f1d3406d190 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:58.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:58.121+0000 7f1d5906e700 1 --2- 172.21.15.16:0/4279535995 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1d541558e0 0x7f1d540c7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:19:58.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:58.121+0000 7f1d5906e700 1 -- 172.21.15.16:0/4279535995 >> 172.21.15.16:0/4279535995 conn(0x7f1d540cffb0 msgr2=0x7f1d540c8f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:19:58.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:58.121+0000 7f1d5906e700 1 -- 172.21.15.16:0/4279535995 shutdown_connections 2023-05-20T01:19:58.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:19:58.121+0000 7f1d5906e700 1 -- 172.21.15.16:0/4279535995 wait complete. 2023-05-20T01:19:58.133 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2023-05-20T01:19:58.138 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":166,"stamp":"2023-05-20T01:19:57.364629+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":183,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":78912,"kb_used_data":7232,"kb_used_omap":0,"kb_used_meta":71616,"kb_avail":754895808,"statfs":{"total":773094113280,"available":773013307392,"internally_reserved":0,"allocated":7405568,"data_stored":4184906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":73334784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001209"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:54.137250+0000","last_change":"2023-05-20T01:19:51.945445+0000","last_active":"2023-05-20T01:19:54.137250+0000","last_peered":"2023-05-20T01:19:54.137250+0000","last_clean":"2023-05-20T01:19:54.137250+0000","last_became_active":"2023-05-20T01:19:51.945160+0000","last_became_peered":"2023-05-20T01:19:51.945160+0000","last_unstale":"2023-05-20T01:19:54.137250+0000","last_undegraded":"2023-05-20T01:19:54.137250+0000","last_fullsized":"2023-05-20T01:19:54.137250+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:04:47.770805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:54.137442+0000","last_change":"2023-05-20T01:19:51.933445+0000","last_active":"2023-05-20T01:19:54.137442+0000","last_peered":"2023-05-20T01:19:54.137442+0000","last_clean":"2023-05-20T01:19:54.137442+0000","last_became_active":"2023-05-20T01:19:51.933189+0000","last_became_peered":"2023-05-20T01:19:51.933189+0000","last_unstale":"2023-05-20T01:19:54.137442+0000","last_undegraded":"2023-05-20T01:19:54.137442+0000","last_fullsized":"2023-05-20T01:19:54.137442+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:51:19.406691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:54.138009+0000","last_change":"2023-05-20T01:19:51.944350+0000","last_active":"2023-05-20T01:19:54.138009+0000","last_peered":"2023-05-20T01:19:54.138009+0000","last_clean":"2023-05-20T01:19:54.138009+0000","last_became_active":"2023-05-20T01:19:51.941812+0000","last_became_peered":"2023-05-20T01:19:51.941812+0000","last_unstale":"2023-05-20T01:19:54.138009+0000","last_undegraded":"2023-05-20T01:19:54.138009+0000","last_fullsized":"2023-05-20T01:19:54.138009+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:46:21.957714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:52.934128+0000","last_change":"2023-05-20T01:19:51.931092+0000","last_active":"2023-05-20T01:19:52.934128+0000","last_peered":"2023-05-20T01:19:52.934128+0000","last_clean":"2023-05-20T01:19:52.934128+0000","last_became_active":"2023-05-20T01:19:51.929780+0000","last_became_peered":"2023-05-20T01:19:51.929780+0000","last_unstale":"2023-05-20T01:19:52.934128+0000","last_undegraded":"2023-05-20T01:19:52.934128+0000","last_fullsized":"2023-05-20T01:19:52.934128+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T13:02:26.622523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:52.934254+0000","last_change":"2023-05-20T01:19:51.945244+0000","last_active":"2023-05-20T01:19:52.934254+0000","last_peered":"2023-05-20T01:19:52.934254+0000","last_clean":"2023-05-20T01:19:52.934254+0000","last_became_active":"2023-05-20T01:19:51.944571+0000","last_became_peered":"2023-05-20T01:19:51.944571+0000","last_unstale":"2023-05-20T01:19:52.934254+0000","last_undegraded":"2023-05-20T01:19:52.934254+0000","last_fullsized":"2023-05-20T01:19:52.934254+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:33:23.171998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:53.436345+0000","last_change":"2023-05-20T01:19:51.938758+0000","last_active":"2023-05-20T01:19:53.436345+0000","last_peered":"2023-05-20T01:19:53.436345+0000","last_clean":"2023-05-20T01:19:53.436345+0000","last_became_active":"2023-05-20T01:19:51.938496+0000","last_became_peered":"2023-05-20T01:19:51.938496+0000","last_unstale":"2023-05-20T01:19:53.436345+0000","last_undegraded":"2023-05-20T01:19:53.436345+0000","last_fullsized":"2023-05-20T01:19:53.436345+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:17:05.433211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:53.636999+0000","last_change":"2023-05-20T01:19:51.942231+0000","last_active":"2023-05-20T01:19:53.636999+0000","last_peered":"2023-05-20T01:19:53.636999+0000","last_clean":"2023-05-20T01:19:53.636999+0000","last_became_active":"2023-05-20T01:19:51.941916+0000","last_became_peered":"2023-05-20T01:19:51.941916+0000","last_unstale":"2023-05-20T01:19:53.636999+0000","last_undegraded":"2023-05-20T01:19:53.636999+0000","last_fullsized":"2023-05-20T01:19:53.636999+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T01:48:59.400838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:54.137212+0000","last_change":"2023-05-20T01:19:51.944138+0000","last_active":"2023-05-20T01:19:54.137212+0000","last_peered":"2023-05-20T01:19:54.137212+0000","last_clean":"2023-05-20T01:19:54.137212+0000","last_became_active":"2023-05-20T01:19:51.943872+0000","last_became_peered":"2023-05-20T01:19:51.943872+0000","last_unstale":"2023-05-20T01:19:54.137212+0000","last_undegraded":"2023-05-20T01:19:54.137212+0000","last_fullsized":"2023-05-20T01:19:54.137212+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:43:07.975515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:54.138201+0000","last_change":"2023-05-20T01:19:51.947064+0000","last_active":"2023-05-20T01:19:54.138201+0000","last_peered":"2023-05-20T01:19:54.138201+0000","last_clean":"2023-05-20T01:19:54.138201+0000","last_became_active":"2023-05-20T01:19:51.946725+0000","last_became_peered":"2023-05-20T01:19:51.946725+0000","last_unstale":"2023-05-20T01:19:54.138201+0000","last_undegraded":"2023-05-20T01:19:54.138201+0000","last_fullsized":"2023-05-20T01:19:54.138201+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:38:01.198660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:54.138056+0000","last_change":"2023-05-20T01:19:51.938942+0000","last_active":"2023-05-20T01:19:54.138056+0000","last_peered":"2023-05-20T01:19:54.138056+0000","last_clean":"2023-05-20T01:19:54.138056+0000","last_became_active":"2023-05-20T01:19:51.938510+0000","last_became_peered":"2023-05-20T01:19:51.938510+0000","last_unstale":"2023-05-20T01:19:54.138056+0000","last_undegraded":"2023-05-20T01:19:54.138056+0000","last_fullsized":"2023-05-20T01:19:54.138056+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:08:52.099992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:52.934501+0000","last_change":"2023-05-20T01:19:51.934286+0000","last_active":"2023-05-20T01:19:52.934501+0000","last_peered":"2023-05-20T01:19:52.934501+0000","last_clean":"2023-05-20T01:19:52.934501+0000","last_became_active":"2023-05-20T01:19:51.933953+0000","last_became_peered":"2023-05-20T01:19:51.933953+0000","last_unstale":"2023-05-20T01:19:52.934501+0000","last_undegraded":"2023-05-20T01:19:52.934501+0000","last_fullsized":"2023-05-20T01:19:52.934501+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:14:27.490210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:54.137891+0000","last_change":"2023-05-20T01:19:51.938231+0000","last_active":"2023-05-20T01:19:54.137891+0000","last_peered":"2023-05-20T01:19:54.137891+0000","last_clean":"2023-05-20T01:19:54.137891+0000","last_became_active":"2023-05-20T01:19:51.937947+0000","last_became_peered":"2023-05-20T01:19:51.937947+0000","last_unstale":"2023-05-20T01:19:54.137891+0000","last_undegraded":"2023-05-20T01:19:54.137891+0000","last_fullsized":"2023-05-20T01:19:54.137891+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:55:42.050629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:54.137657+0000","last_change":"2023-05-20T01:19:51.947945+0000","last_active":"2023-05-20T01:19:54.137657+0000","last_peered":"2023-05-20T01:19:54.137657+0000","last_clean":"2023-05-20T01:19:54.137657+0000","last_became_active":"2023-05-20T01:19:51.947696+0000","last_became_peered":"2023-05-20T01:19:51.947696+0000","last_unstale":"2023-05-20T01:19:54.137657+0000","last_undegraded":"2023-05-20T01:19:54.137657+0000","last_fullsized":"2023-05-20T01:19:54.137657+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:40:27.165242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:54.137495+0000","last_change":"2023-05-20T01:19:51.943523+0000","last_active":"2023-05-20T01:19:54.137495+0000","last_peered":"2023-05-20T01:19:54.137495+0000","last_clean":"2023-05-20T01:19:54.137495+0000","last_became_active":"2023-05-20T01:19:51.941751+0000","last_became_peered":"2023-05-20T01:19:51.941751+0000","last_unstale":"2023-05-20T01:19:54.137495+0000","last_undegraded":"2023-05-20T01:19:54.137495+0000","last_fullsized":"2023-05-20T01:19:54.137495+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:13:38.799198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:54.137966+0000","last_change":"2023-05-20T01:19:51.941319+0000","last_active":"2023-05-20T01:19:54.137966+0000","last_peered":"2023-05-20T01:19:54.137966+0000","last_clean":"2023-05-20T01:19:54.137966+0000","last_became_active":"2023-05-20T01:19:51.940969+0000","last_became_peered":"2023-05-20T01:19:51.940969+0000","last_unstale":"2023-05-20T01:19:54.137966+0000","last_undegraded":"2023-05-20T01:19:54.137966+0000","last_fullsized":"2023-05-20T01:19:54.137966+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:34:27.000278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:53.637000+0000","last_change":"2023-05-20T01:19:51.935543+0000","last_active":"2023-05-20T01:19:53.637000+0000","last_peered":"2023-05-20T01:19:53.637000+0000","last_clean":"2023-05-20T01:19:53.637000+0000","last_became_active":"2023-05-20T01:19:51.935287+0000","last_became_peered":"2023-05-20T01:19:51.935287+0000","last_unstale":"2023-05-20T01:19:53.637000+0000","last_undegraded":"2023-05-20T01:19:53.637000+0000","last_fullsized":"2023-05-20T01:19:53.637000+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:31:05.916596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:54.137372+0000","last_change":"2023-05-20T01:19:51.947938+0000","last_active":"2023-05-20T01:19:54.137372+0000","last_peered":"2023-05-20T01:19:54.137372+0000","last_clean":"2023-05-20T01:19:54.137372+0000","last_became_active":"2023-05-20T01:19:51.947650+0000","last_became_peered":"2023-05-20T01:19:51.947650+0000","last_unstale":"2023-05-20T01:19:54.137372+0000","last_undegraded":"2023-05-20T01:19:54.137372+0000","last_fullsized":"2023-05-20T01:19:54.137372+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:47:04.749262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:52.934606+0000","last_change":"2023-05-20T01:19:51.932362+0000","last_active":"2023-05-20T01:19:52.934606+0000","last_peered":"2023-05-20T01:19:52.934606+0000","last_clean":"2023-05-20T01:19:52.934606+0000","last_became_active":"2023-05-20T01:19:51.932135+0000","last_became_peered":"2023-05-20T01:19:51.932135+0000","last_unstale":"2023-05-20T01:19:52.934606+0000","last_undegraded":"2023-05-20T01:19:52.934606+0000","last_fullsized":"2023-05-20T01:19:52.934606+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:24:56.004774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:52.934130+0000","last_change":"2023-05-20T01:19:51.936075+0000","last_active":"2023-05-20T01:19:52.934130+0000","last_peered":"2023-05-20T01:19:52.934130+0000","last_clean":"2023-05-20T01:19:52.934130+0000","last_became_active":"2023-05-20T01:19:51.935677+0000","last_became_peered":"2023-05-20T01:19:51.935677+0000","last_unstale":"2023-05-20T01:19:52.934130+0000","last_undegraded":"2023-05-20T01:19:52.934130+0000","last_fullsized":"2023-05-20T01:19:52.934130+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:52:47.813260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:52.928122+0000","last_change":"2023-05-20T01:19:51.930152+0000","last_active":"2023-05-20T01:19:52.928122+0000","last_peered":"2023-05-20T01:19:52.928122+0000","last_clean":"2023-05-20T01:19:52.928122+0000","last_became_active":"2023-05-20T01:19:51.929859+0000","last_became_peered":"2023-05-20T01:19:51.929859+0000","last_unstale":"2023-05-20T01:19:52.928122+0000","last_undegraded":"2023-05-20T01:19:52.928122+0000","last_fullsized":"2023-05-20T01:19:52.928122+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:31:56.703552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:54.137807+0000","last_change":"2023-05-20T01:19:51.938573+0000","last_active":"2023-05-20T01:19:54.137807+0000","last_peered":"2023-05-20T01:19:54.137807+0000","last_clean":"2023-05-20T01:19:54.137807+0000","last_became_active":"2023-05-20T01:19:51.938294+0000","last_became_peered":"2023-05-20T01:19:51.938294+0000","last_unstale":"2023-05-20T01:19:54.137807+0000","last_undegraded":"2023-05-20T01:19:54.137807+0000","last_fullsized":"2023-05-20T01:19:54.137807+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:10:13.440024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:52.928021+0000","last_change":"2023-05-20T01:19:51.932413+0000","last_active":"2023-05-20T01:19:52.928021+0000","last_peered":"2023-05-20T01:19:52.928021+0000","last_clean":"2023-05-20T01:19:52.928021+0000","last_became_active":"2023-05-20T01:19:51.932152+0000","last_became_peered":"2023-05-20T01:19:51.932152+0000","last_unstale":"2023-05-20T01:19:52.928021+0000","last_undegraded":"2023-05-20T01:19:52.928021+0000","last_fullsized":"2023-05-20T01:19:52.928021+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:41:44.890671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:52.927824+0000","last_change":"2023-05-20T01:19:51.932416+0000","last_active":"2023-05-20T01:19:52.927824+0000","last_peered":"2023-05-20T01:19:52.927824+0000","last_clean":"2023-05-20T01:19:52.927824+0000","last_became_active":"2023-05-20T01:19:51.932156+0000","last_became_peered":"2023-05-20T01:19:51.932156+0000","last_unstale":"2023-05-20T01:19:52.927824+0000","last_undegraded":"2023-05-20T01:19:52.927824+0000","last_fullsized":"2023-05-20T01:19:52.927824+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:36:38.819476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:54.138287+0000","last_change":"2023-05-20T01:19:51.943006+0000","last_active":"2023-05-20T01:19:54.138287+0000","last_peered":"2023-05-20T01:19:54.138287+0000","last_clean":"2023-05-20T01:19:54.138287+0000","last_became_active":"2023-05-20T01:19:51.942685+0000","last_became_peered":"2023-05-20T01:19:51.942685+0000","last_unstale":"2023-05-20T01:19:54.138287+0000","last_undegraded":"2023-05-20T01:19:54.138287+0000","last_fullsized":"2023-05-20T01:19:54.138287+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:18:26.867598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:54.138067+0000","last_change":"2023-05-20T01:19:51.940251+0000","last_active":"2023-05-20T01:19:54.138067+0000","last_peered":"2023-05-20T01:19:54.138067+0000","last_clean":"2023-05-20T01:19:54.138067+0000","last_became_active":"2023-05-20T01:19:51.937972+0000","last_became_peered":"2023-05-20T01:19:51.937972+0000","last_unstale":"2023-05-20T01:19:54.138067+0000","last_undegraded":"2023-05-20T01:19:54.138067+0000","last_fullsized":"2023-05-20T01:19:54.138067+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:03:53.250451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:54.137420+0000","last_change":"2023-05-20T01:19:51.943182+0000","last_active":"2023-05-20T01:19:54.137420+0000","last_peered":"2023-05-20T01:19:54.137420+0000","last_clean":"2023-05-20T01:19:54.137420+0000","last_became_active":"2023-05-20T01:19:51.937980+0000","last_became_peered":"2023-05-20T01:19:51.937980+0000","last_unstale":"2023-05-20T01:19:54.137420+0000","last_undegraded":"2023-05-20T01:19:54.137420+0000","last_fullsized":"2023-05-20T01:19:54.137420+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:32:33.635086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:52.934626+0000","last_change":"2023-05-20T01:19:51.935765+0000","last_active":"2023-05-20T01:19:52.934626+0000","last_peered":"2023-05-20T01:19:52.934626+0000","last_clean":"2023-05-20T01:19:52.934626+0000","last_became_active":"2023-05-20T01:19:51.935275+0000","last_became_peered":"2023-05-20T01:19:51.935275+0000","last_unstale":"2023-05-20T01:19:52.934626+0000","last_undegraded":"2023-05-20T01:19:52.934626+0000","last_fullsized":"2023-05-20T01:19:52.934626+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:10:48.987365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:54.137174+0000","last_change":"2023-05-20T01:19:51.938126+0000","last_active":"2023-05-20T01:19:54.137174+0000","last_peered":"2023-05-20T01:19:54.137174+0000","last_clean":"2023-05-20T01:19:54.137174+0000","last_became_active":"2023-05-20T01:19:51.937493+0000","last_became_peered":"2023-05-20T01:19:51.937493+0000","last_unstale":"2023-05-20T01:19:54.137174+0000","last_undegraded":"2023-05-20T01:19:54.137174+0000","last_fullsized":"2023-05-20T01:19:54.137174+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:09:17.812439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:54.137855+0000","last_change":"2023-05-20T01:19:51.943999+0000","last_active":"2023-05-20T01:19:54.137855+0000","last_peered":"2023-05-20T01:19:54.137855+0000","last_clean":"2023-05-20T01:19:54.137855+0000","last_became_active":"2023-05-20T01:19:51.943730+0000","last_became_peered":"2023-05-20T01:19:51.943730+0000","last_unstale":"2023-05-20T01:19:54.137855+0000","last_undegraded":"2023-05-20T01:19:54.137855+0000","last_fullsized":"2023-05-20T01:19:54.137855+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:13:16.951441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:54.137344+0000","last_change":"2023-05-20T01:19:51.943585+0000","last_active":"2023-05-20T01:19:54.137344+0000","last_peered":"2023-05-20T01:19:54.137344+0000","last_clean":"2023-05-20T01:19:54.137344+0000","last_became_active":"2023-05-20T01:19:51.943314+0000","last_became_peered":"2023-05-20T01:19:51.943314+0000","last_unstale":"2023-05-20T01:19:54.137344+0000","last_undegraded":"2023-05-20T01:19:54.137344+0000","last_fullsized":"2023-05-20T01:19:54.137344+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:12:19.351862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:52.933859+0000","last_change":"2023-05-20T01:19:51.941063+0000","last_active":"2023-05-20T01:19:52.933859+0000","last_peered":"2023-05-20T01:19:52.933859+0000","last_clean":"2023-05-20T01:19:52.933859+0000","last_became_active":"2023-05-20T01:19:51.937055+0000","last_became_peered":"2023-05-20T01:19:51.937055+0000","last_unstale":"2023-05-20T01:19:52.933859+0000","last_undegraded":"2023-05-20T01:19:52.933859+0000","last_fullsized":"2023-05-20T01:19:52.933859+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:05:12.671768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:53.436081+0000","last_change":"2023-05-20T01:19:51.936450+0000","last_active":"2023-05-20T01:19:53.436081+0000","last_peered":"2023-05-20T01:19:53.436081+0000","last_clean":"2023-05-20T01:19:53.436081+0000","last_became_active":"2023-05-20T01:19:51.936047+0000","last_became_peered":"2023-05-20T01:19:51.936047+0000","last_unstale":"2023-05-20T01:19:53.436081+0000","last_undegraded":"2023-05-20T01:19:53.436081+0000","last_fullsized":"2023-05-20T01:19:53.436081+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:41:11.309142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:52.933848+0000","last_change":"2023-05-20T01:19:51.939431+0000","last_active":"2023-05-20T01:19:52.933848+0000","last_peered":"2023-05-20T01:19:52.933848+0000","last_clean":"2023-05-20T01:19:52.933848+0000","last_became_active":"2023-05-20T01:19:51.939042+0000","last_became_peered":"2023-05-20T01:19:51.939042+0000","last_unstale":"2023-05-20T01:19:52.933848+0000","last_undegraded":"2023-05-20T01:19:52.933848+0000","last_fullsized":"2023-05-20T01:19:52.933848+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:25:34.546455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"63'4","reported_seq":17,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:54.137447+0000","last_change":"2023-05-20T01:19:51.508203+0000","last_active":"2023-05-20T01:19:54.137447+0000","last_peered":"2023-05-20T01:19:54.137447+0000","last_clean":"2023-05-20T01:19:54.137447+0000","last_became_active":"2023-05-20T01:19:51.507889+0000","last_became_peered":"2023-05-20T01:19:51.507889+0000","last_unstale":"2023-05-20T01:19:54.137447+0000","last_undegraded":"2023-05-20T01:19:54.137447+0000","last_fullsized":"2023-05-20T01:19:54.137447+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:52:37.956564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:52.934687+0000","last_change":"2023-05-20T01:19:51.930423+0000","last_active":"2023-05-20T01:19:52.934687+0000","last_peered":"2023-05-20T01:19:52.934687+0000","last_clean":"2023-05-20T01:19:52.934687+0000","last_became_active":"2023-05-20T01:19:51.930138+0000","last_became_peered":"2023-05-20T01:19:51.930138+0000","last_unstale":"2023-05-20T01:19:52.934687+0000","last_undegraded":"2023-05-20T01:19:52.934687+0000","last_fullsized":"2023-05-20T01:19:52.934687+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:20:37.462017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"63'1","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:53.436611+0000","last_change":"2023-05-20T01:19:51.510878+0000","last_active":"2023-05-20T01:19:53.436611+0000","last_peered":"2023-05-20T01:19:53.436611+0000","last_clean":"2023-05-20T01:19:53.436611+0000","last_became_active":"2023-05-20T01:19:51.510409+0000","last_became_peered":"2023-05-20T01:19:51.510409+0000","last_unstale":"2023-05-20T01:19:53.436611+0000","last_undegraded":"2023-05-20T01:19:53.436611+0000","last_fullsized":"2023-05-20T01:19:53.436611+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:31:51.936693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:53.436553+0000","last_change":"2023-05-20T01:19:51.935814+0000","last_active":"2023-05-20T01:19:53.436553+0000","last_peered":"2023-05-20T01:19:53.436553+0000","last_clean":"2023-05-20T01:19:53.436553+0000","last_became_active":"2023-05-20T01:19:51.935132+0000","last_became_peered":"2023-05-20T01:19:51.935132+0000","last_unstale":"2023-05-20T01:19:53.436553+0000","last_undegraded":"2023-05-20T01:19:53.436553+0000","last_fullsized":"2023-05-20T01:19:53.436553+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:38:10.169861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"63'2","reported_seq":15,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:53.436471+0000","last_change":"2023-05-20T01:19:51.513176+0000","last_active":"2023-05-20T01:19:53.436471+0000","last_peered":"2023-05-20T01:19:53.436471+0000","last_clean":"2023-05-20T01:19:53.436471+0000","last_became_active":"2023-05-20T01:19:51.512562+0000","last_became_peered":"2023-05-20T01:19:51.512562+0000","last_unstale":"2023-05-20T01:19:53.436471+0000","last_undegraded":"2023-05-20T01:19:53.436471+0000","last_fullsized":"2023-05-20T01:19:53.436471+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:28:12.838142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:54.138161+0000","last_change":"2023-05-20T01:19:51.938549+0000","last_active":"2023-05-20T01:19:54.138161+0000","last_peered":"2023-05-20T01:19:54.138161+0000","last_clean":"2023-05-20T01:19:54.138161+0000","last_became_active":"2023-05-20T01:19:51.938310+0000","last_became_peered":"2023-05-20T01:19:51.938310+0000","last_unstale":"2023-05-20T01:19:54.138161+0000","last_undegraded":"2023-05-20T01:19:54.138161+0000","last_fullsized":"2023-05-20T01:19:54.138161+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:33:43.077695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"63'2","reported_seq":15,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:53.436448+0000","last_change":"2023-05-20T01:19:51.511427+0000","last_active":"2023-05-20T01:19:53.436448+0000","last_peered":"2023-05-20T01:19:53.436448+0000","last_clean":"2023-05-20T01:19:53.436448+0000","last_became_active":"2023-05-20T01:19:51.511141+0000","last_became_peered":"2023-05-20T01:19:51.511141+0000","last_unstale":"2023-05-20T01:19:53.436448+0000","last_undegraded":"2023-05-20T01:19:53.436448+0000","last_fullsized":"2023-05-20T01:19:53.436448+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:59:15.391262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:53.636823+0000","last_change":"2023-05-20T01:19:51.935529+0000","last_active":"2023-05-20T01:19:53.636823+0000","last_peered":"2023-05-20T01:19:53.636823+0000","last_clean":"2023-05-20T01:19:53.636823+0000","last_became_active":"2023-05-20T01:19:51.934957+0000","last_became_peered":"2023-05-20T01:19:51.934957+0000","last_unstale":"2023-05-20T01:19:53.636823+0000","last_undegraded":"2023-05-20T01:19:53.636823+0000","last_fullsized":"2023-05-20T01:19:53.636823+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:45:34.257820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"63'1","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:52.928293+0000","last_change":"2023-05-20T01:19:51.500164+0000","last_active":"2023-05-20T01:19:52.928293+0000","last_peered":"2023-05-20T01:19:52.928293+0000","last_clean":"2023-05-20T01:19:52.928293+0000","last_became_active":"2023-05-20T01:19:51.499814+0000","last_became_peered":"2023-05-20T01:19:51.499814+0000","last_unstale":"2023-05-20T01:19:52.928293+0000","last_undegraded":"2023-05-20T01:19:52.928293+0000","last_fullsized":"2023-05-20T01:19:52.928293+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T01:26:53.281106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:54.136877+0000","last_change":"2023-05-20T01:19:51.946579+0000","last_active":"2023-05-20T01:19:54.136877+0000","last_peered":"2023-05-20T01:19:54.136877+0000","last_clean":"2023-05-20T01:19:54.136877+0000","last_became_active":"2023-05-20T01:19:51.946362+0000","last_became_peered":"2023-05-20T01:19:51.946362+0000","last_unstale":"2023-05-20T01:19:54.136877+0000","last_undegraded":"2023-05-20T01:19:54.136877+0000","last_fullsized":"2023-05-20T01:19:54.136877+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:52:47.849782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"63'1","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:52.927880+0000","last_change":"2023-05-20T01:19:51.506384+0000","last_active":"2023-05-20T01:19:52.927880+0000","last_peered":"2023-05-20T01:19:52.927880+0000","last_clean":"2023-05-20T01:19:52.927880+0000","last_became_active":"2023-05-20T01:19:51.506097+0000","last_became_peered":"2023-05-20T01:19:51.506097+0000","last_unstale":"2023-05-20T01:19:52.927880+0000","last_undegraded":"2023-05-20T01:19:52.927880+0000","last_fullsized":"2023-05-20T01:19:52.927880+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:53:23.268040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:52.933870+0000","last_change":"2023-05-20T01:19:51.930152+0000","last_active":"2023-05-20T01:19:52.933870+0000","last_peered":"2023-05-20T01:19:52.933870+0000","last_clean":"2023-05-20T01:19:52.933870+0000","last_became_active":"2023-05-20T01:19:51.929734+0000","last_became_peered":"2023-05-20T01:19:51.929734+0000","last_unstale":"2023-05-20T01:19:52.933870+0000","last_undegraded":"2023-05-20T01:19:52.933870+0000","last_fullsized":"2023-05-20T01:19:52.933870+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:46:37.721293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:52.927896+0000","last_change":"2023-05-20T01:19:51.501875+0000","last_active":"2023-05-20T01:19:52.927896+0000","last_peered":"2023-05-20T01:19:52.927896+0000","last_clean":"2023-05-20T01:19:52.927896+0000","last_became_active":"2023-05-20T01:19:51.500887+0000","last_became_peered":"2023-05-20T01:19:51.500887+0000","last_unstale":"2023-05-20T01:19:52.927896+0000","last_undegraded":"2023-05-20T01:19:52.927896+0000","last_fullsized":"2023-05-20T01:19:52.927896+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:03:28.388879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:52.928185+0000","last_change":"2023-05-20T01:19:51.930315+0000","last_active":"2023-05-20T01:19:52.928185+0000","last_peered":"2023-05-20T01:19:52.928185+0000","last_clean":"2023-05-20T01:19:52.928185+0000","last_became_active":"2023-05-20T01:19:51.930025+0000","last_became_peered":"2023-05-20T01:19:51.930025+0000","last_unstale":"2023-05-20T01:19:52.928185+0000","last_undegraded":"2023-05-20T01:19:52.928185+0000","last_fullsized":"2023-05-20T01:19:52.928185+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:23:08.275931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:52.933523+0000","last_change":"2023-05-20T01:19:51.506858+0000","last_active":"2023-05-20T01:19:52.933523+0000","last_peered":"2023-05-20T01:19:52.933523+0000","last_clean":"2023-05-20T01:19:52.933523+0000","last_became_active":"2023-05-20T01:19:51.506514+0000","last_became_peered":"2023-05-20T01:19:51.506514+0000","last_unstale":"2023-05-20T01:19:52.933523+0000","last_undegraded":"2023-05-20T01:19:52.933523+0000","last_fullsized":"2023-05-20T01:19:52.933523+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:58:06.309817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:54.137366+0000","last_change":"2023-05-20T01:19:51.938315+0000","last_active":"2023-05-20T01:19:54.137366+0000","last_peered":"2023-05-20T01:19:54.137366+0000","last_clean":"2023-05-20T01:19:54.137366+0000","last_became_active":"2023-05-20T01:19:51.937857+0000","last_became_peered":"2023-05-20T01:19:51.937857+0000","last_unstale":"2023-05-20T01:19:54.137366+0000","last_undegraded":"2023-05-20T01:19:54.137366+0000","last_fullsized":"2023-05-20T01:19:54.137366+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:25:49.473325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:54.138345+0000","last_change":"2023-05-20T01:19:51.510993+0000","last_active":"2023-05-20T01:19:54.138345+0000","last_peered":"2023-05-20T01:19:54.138345+0000","last_clean":"2023-05-20T01:19:54.138345+0000","last_became_active":"2023-05-20T01:19:51.510657+0000","last_became_peered":"2023-05-20T01:19:51.510657+0000","last_unstale":"2023-05-20T01:19:54.138345+0000","last_undegraded":"2023-05-20T01:19:54.138345+0000","last_fullsized":"2023-05-20T01:19:54.138345+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:55:44.597369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:52.927986+0000","last_change":"2023-05-20T01:19:51.931020+0000","last_active":"2023-05-20T01:19:52.927986+0000","last_peered":"2023-05-20T01:19:52.927986+0000","last_clean":"2023-05-20T01:19:52.927986+0000","last_became_active":"2023-05-20T01:19:51.930750+0000","last_became_peered":"2023-05-20T01:19:51.930750+0000","last_unstale":"2023-05-20T01:19:52.927986+0000","last_undegraded":"2023-05-20T01:19:52.927986+0000","last_fullsized":"2023-05-20T01:19:52.927986+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:13:26.310064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:52.934749+0000","last_change":"2023-05-20T01:19:51.510988+0000","last_active":"2023-05-20T01:19:52.934749+0000","last_peered":"2023-05-20T01:19:52.934749+0000","last_clean":"2023-05-20T01:19:52.934749+0000","last_became_active":"2023-05-20T01:19:51.510728+0000","last_became_peered":"2023-05-20T01:19:51.510728+0000","last_unstale":"2023-05-20T01:19:52.934749+0000","last_undegraded":"2023-05-20T01:19:52.934749+0000","last_fullsized":"2023-05-20T01:19:52.934749+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:32:37.916213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:52.934144+0000","last_change":"2023-05-20T01:19:51.934976+0000","last_active":"2023-05-20T01:19:52.934144+0000","last_peered":"2023-05-20T01:19:52.934144+0000","last_clean":"2023-05-20T01:19:52.934144+0000","last_became_active":"2023-05-20T01:19:51.934662+0000","last_became_peered":"2023-05-20T01:19:51.934662+0000","last_unstale":"2023-05-20T01:19:52.934144+0000","last_undegraded":"2023-05-20T01:19:52.934144+0000","last_fullsized":"2023-05-20T01:19:52.934144+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:18:05.645506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"63'3","reported_seq":16,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:54.138284+0000","last_change":"2023-05-20T01:19:51.511325+0000","last_active":"2023-05-20T01:19:54.138284+0000","last_peered":"2023-05-20T01:19:54.138284+0000","last_clean":"2023-05-20T01:19:54.138284+0000","last_became_active":"2023-05-20T01:19:51.510525+0000","last_became_peered":"2023-05-20T01:19:51.510525+0000","last_unstale":"2023-05-20T01:19:54.138284+0000","last_undegraded":"2023-05-20T01:19:54.138284+0000","last_fullsized":"2023-05-20T01:19:54.138284+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:12:58.176737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:52.934729+0000","last_change":"2023-05-20T01:19:51.941822+0000","last_active":"2023-05-20T01:19:52.934729+0000","last_peered":"2023-05-20T01:19:52.934729+0000","last_clean":"2023-05-20T01:19:52.934729+0000","last_became_active":"2023-05-20T01:19:51.941545+0000","last_became_peered":"2023-05-20T01:19:51.941545+0000","last_unstale":"2023-05-20T01:19:52.934729+0000","last_undegraded":"2023-05-20T01:19:52.934729+0000","last_fullsized":"2023-05-20T01:19:52.934729+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:29:07.674750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"63'6","reported_seq":19,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:54.137182+0000","last_change":"2023-05-20T01:19:51.513591+0000","last_active":"2023-05-20T01:19:54.137182+0000","last_peered":"2023-05-20T01:19:54.137182+0000","last_clean":"2023-05-20T01:19:54.137182+0000","last_became_active":"2023-05-20T01:19:51.513292+0000","last_became_peered":"2023-05-20T01:19:51.513292+0000","last_unstale":"2023-05-20T01:19:54.137182+0000","last_undegraded":"2023-05-20T01:19:54.137182+0000","last_fullsized":"2023-05-20T01:19:54.137182+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:36:24.693283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:52.933740+0000","last_change":"2023-05-20T01:19:51.942847+0000","last_active":"2023-05-20T01:19:52.933740+0000","last_peered":"2023-05-20T01:19:52.933740+0000","last_clean":"2023-05-20T01:19:52.933740+0000","last_became_active":"2023-05-20T01:19:51.942485+0000","last_became_peered":"2023-05-20T01:19:51.942485+0000","last_unstale":"2023-05-20T01:19:52.933740+0000","last_undegraded":"2023-05-20T01:19:52.933740+0000","last_fullsized":"2023-05-20T01:19:52.933740+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:42:40.123929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"63'2","reported_seq":15,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:54.138002+0000","last_change":"2023-05-20T01:19:51.506740+0000","last_active":"2023-05-20T01:19:54.138002+0000","last_peered":"2023-05-20T01:19:54.138002+0000","last_clean":"2023-05-20T01:19:54.138002+0000","last_became_active":"2023-05-20T01:19:51.506453+0000","last_became_peered":"2023-05-20T01:19:51.506453+0000","last_unstale":"2023-05-20T01:19:54.138002+0000","last_undegraded":"2023-05-20T01:19:54.138002+0000","last_fullsized":"2023-05-20T01:19:54.138002+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:18:40.151218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:54.137358+0000","last_change":"2023-05-20T01:19:51.933327+0000","last_active":"2023-05-20T01:19:54.137358+0000","last_peered":"2023-05-20T01:19:54.137358+0000","last_clean":"2023-05-20T01:19:54.137358+0000","last_became_active":"2023-05-20T01:19:51.933061+0000","last_became_peered":"2023-05-20T01:19:51.933061+0000","last_unstale":"2023-05-20T01:19:54.137358+0000","last_undegraded":"2023-05-20T01:19:54.137358+0000","last_fullsized":"2023-05-20T01:19:54.137358+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:28:44.290645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:54.137489+0000","last_change":"2023-05-20T01:19:51.508126+0000","last_active":"2023-05-20T01:19:54.137489+0000","last_peered":"2023-05-20T01:19:54.137489+0000","last_clean":"2023-05-20T01:19:54.137489+0000","last_became_active":"2023-05-20T01:19:51.507837+0000","last_became_peered":"2023-05-20T01:19:51.507837+0000","last_unstale":"2023-05-20T01:19:54.137489+0000","last_undegraded":"2023-05-20T01:19:54.137489+0000","last_fullsized":"2023-05-20T01:19:54.137489+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:43:24.091676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:52.934036+0000","last_change":"2023-05-20T01:19:51.936792+0000","last_active":"2023-05-20T01:19:52.934036+0000","last_peered":"2023-05-20T01:19:52.934036+0000","last_clean":"2023-05-20T01:19:52.934036+0000","last_became_active":"2023-05-20T01:19:51.936524+0000","last_became_peered":"2023-05-20T01:19:51.936524+0000","last_unstale":"2023-05-20T01:19:52.934036+0000","last_undegraded":"2023-05-20T01:19:52.934036+0000","last_fullsized":"2023-05-20T01:19:52.934036+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:04:05.309842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:54.138280+0000","last_change":"2023-05-20T01:19:51.507794+0000","last_active":"2023-05-20T01:19:54.138280+0000","last_peered":"2023-05-20T01:19:54.138280+0000","last_clean":"2023-05-20T01:19:54.138280+0000","last_became_active":"2023-05-20T01:19:51.507547+0000","last_became_peered":"2023-05-20T01:19:51.507547+0000","last_unstale":"2023-05-20T01:19:54.138280+0000","last_undegraded":"2023-05-20T01:19:54.138280+0000","last_fullsized":"2023-05-20T01:19:54.138280+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:56:49.137434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:54.137013+0000","last_change":"2023-05-20T01:19:51.949014+0000","last_active":"2023-05-20T01:19:54.137013+0000","last_peered":"2023-05-20T01:19:54.137013+0000","last_clean":"2023-05-20T01:19:54.137013+0000","last_became_active":"2023-05-20T01:19:51.948813+0000","last_became_peered":"2023-05-20T01:19:51.948813+0000","last_unstale":"2023-05-20T01:19:54.137013+0000","last_undegraded":"2023-05-20T01:19:54.137013+0000","last_fullsized":"2023-05-20T01:19:54.137013+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:48:25.587471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"63'8","reported_seq":21,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:52.933892+0000","last_change":"2023-05-20T01:19:51.509281+0000","last_active":"2023-05-20T01:19:52.933892+0000","last_peered":"2023-05-20T01:19:52.933892+0000","last_clean":"2023-05-20T01:19:52.933892+0000","last_became_active":"2023-05-20T01:19:51.508971+0000","last_became_peered":"2023-05-20T01:19:51.508971+0000","last_unstale":"2023-05-20T01:19:52.933892+0000","last_undegraded":"2023-05-20T01:19:52.933892+0000","last_fullsized":"2023-05-20T01:19:52.933892+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:27:18.277647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:53.436544+0000","last_change":"2023-05-20T01:19:51.947213+0000","last_active":"2023-05-20T01:19:53.436544+0000","last_peered":"2023-05-20T01:19:53.436544+0000","last_clean":"2023-05-20T01:19:53.436544+0000","last_became_active":"2023-05-20T01:19:51.946895+0000","last_became_peered":"2023-05-20T01:19:51.946895+0000","last_unstale":"2023-05-20T01:19:53.436544+0000","last_undegraded":"2023-05-20T01:19:53.436544+0000","last_fullsized":"2023-05-20T01:19:53.436544+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:26:01.454283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"63'2","reported_seq":15,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:54.138270+0000","last_change":"2023-05-20T01:19:51.505066+0000","last_active":"2023-05-20T01:19:54.138270+0000","last_peered":"2023-05-20T01:19:54.138270+0000","last_clean":"2023-05-20T01:19:54.138270+0000","last_became_active":"2023-05-20T01:19:51.504740+0000","last_became_peered":"2023-05-20T01:19:51.504740+0000","last_unstale":"2023-05-20T01:19:54.138270+0000","last_undegraded":"2023-05-20T01:19:54.138270+0000","last_fullsized":"2023-05-20T01:19:54.138270+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:33:37.663977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:53.636992+0000","last_change":"2023-05-20T01:19:51.936622+0000","last_active":"2023-05-20T01:19:53.636992+0000","last_peered":"2023-05-20T01:19:53.636992+0000","last_clean":"2023-05-20T01:19:53.636992+0000","last_became_active":"2023-05-20T01:19:51.935663+0000","last_became_peered":"2023-05-20T01:19:51.935663+0000","last_unstale":"2023-05-20T01:19:53.636992+0000","last_undegraded":"2023-05-20T01:19:53.636992+0000","last_fullsized":"2023-05-20T01:19:53.636992+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:40:41.636194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:54.137778+0000","last_change":"2023-05-20T01:19:51.505943+0000","last_active":"2023-05-20T01:19:54.137778+0000","last_peered":"2023-05-20T01:19:54.137778+0000","last_clean":"2023-05-20T01:19:54.137778+0000","last_became_active":"2023-05-20T01:19:51.505483+0000","last_became_peered":"2023-05-20T01:19:51.505483+0000","last_unstale":"2023-05-20T01:19:54.137778+0000","last_undegraded":"2023-05-20T01:19:54.137778+0000","last_fullsized":"2023-05-20T01:19:54.137778+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:16:35.292935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:54.137693+0000","last_change":"2023-05-20T01:19:51.944056+0000","last_active":"2023-05-20T01:19:54.137693+0000","last_peered":"2023-05-20T01:19:54.137693+0000","last_clean":"2023-05-20T01:19:54.137693+0000","last_became_active":"2023-05-20T01:19:51.943768+0000","last_became_peered":"2023-05-20T01:19:51.943768+0000","last_unstale":"2023-05-20T01:19:54.137693+0000","last_undegraded":"2023-05-20T01:19:54.137693+0000","last_fullsized":"2023-05-20T01:19:54.137693+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:57:59.518282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:53.436552+0000","last_change":"2023-05-20T01:19:51.512499+0000","last_active":"2023-05-20T01:19:53.436552+0000","last_peered":"2023-05-20T01:19:53.436552+0000","last_clean":"2023-05-20T01:19:53.436552+0000","last_became_active":"2023-05-20T01:19:51.512222+0000","last_became_peered":"2023-05-20T01:19:51.512222+0000","last_unstale":"2023-05-20T01:19:53.436552+0000","last_undegraded":"2023-05-20T01:19:53.436552+0000","last_fullsized":"2023-05-20T01:19:53.436552+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:59:57.390952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:52.934383+0000","last_change":"2023-05-20T01:19:51.936248+0000","last_active":"2023-05-20T01:19:52.934383+0000","last_peered":"2023-05-20T01:19:52.934383+0000","last_clean":"2023-05-20T01:19:52.934383+0000","last_became_active":"2023-05-20T01:19:51.935918+0000","last_became_peered":"2023-05-20T01:19:51.935918+0000","last_unstale":"2023-05-20T01:19:52.934383+0000","last_undegraded":"2023-05-20T01:19:52.934383+0000","last_fullsized":"2023-05-20T01:19:52.934383+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:06:15.619927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:54.137335+0000","last_change":"2023-05-20T01:19:51.513458+0000","last_active":"2023-05-20T01:19:54.137335+0000","last_peered":"2023-05-20T01:19:54.137335+0000","last_clean":"2023-05-20T01:19:54.137335+0000","last_became_active":"2023-05-20T01:19:51.513156+0000","last_became_peered":"2023-05-20T01:19:51.513156+0000","last_unstale":"2023-05-20T01:19:54.137335+0000","last_undegraded":"2023-05-20T01:19:54.137335+0000","last_fullsized":"2023-05-20T01:19:54.137335+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:49:10.224839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:52.933470+0000","last_change":"2023-05-20T01:19:51.934151+0000","last_active":"2023-05-20T01:19:52.933470+0000","last_peered":"2023-05-20T01:19:52.933470+0000","last_clean":"2023-05-20T01:19:52.933470+0000","last_became_active":"2023-05-20T01:19:51.933844+0000","last_became_peered":"2023-05-20T01:19:51.933844+0000","last_unstale":"2023-05-20T01:19:52.933470+0000","last_undegraded":"2023-05-20T01:19:52.933470+0000","last_fullsized":"2023-05-20T01:19:52.933470+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:17:17.969734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:54.138081+0000","last_change":"2023-05-20T01:19:51.510976+0000","last_active":"2023-05-20T01:19:54.138081+0000","last_peered":"2023-05-20T01:19:54.138081+0000","last_clean":"2023-05-20T01:19:54.138081+0000","last_became_active":"2023-05-20T01:19:51.510244+0000","last_became_peered":"2023-05-20T01:19:51.510244+0000","last_unstale":"2023-05-20T01:19:54.138081+0000","last_undegraded":"2023-05-20T01:19:54.138081+0000","last_fullsized":"2023-05-20T01:19:54.138081+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:49:31.184252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:52.934032+0000","last_change":"2023-05-20T01:19:51.932566+0000","last_active":"2023-05-20T01:19:52.934032+0000","last_peered":"2023-05-20T01:19:52.934032+0000","last_clean":"2023-05-20T01:19:52.934032+0000","last_became_active":"2023-05-20T01:19:51.932268+0000","last_became_peered":"2023-05-20T01:19:51.932268+0000","last_unstale":"2023-05-20T01:19:52.934032+0000","last_undegraded":"2023-05-20T01:19:52.934032+0000","last_fullsized":"2023-05-20T01:19:52.934032+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:45:31.086812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:52.928012+0000","last_change":"2023-05-20T01:19:51.508441+0000","last_active":"2023-05-20T01:19:52.928012+0000","last_peered":"2023-05-20T01:19:52.928012+0000","last_clean":"2023-05-20T01:19:52.928012+0000","last_became_active":"2023-05-20T01:19:51.508110+0000","last_became_peered":"2023-05-20T01:19:51.508110+0000","last_unstale":"2023-05-20T01:19:52.928012+0000","last_undegraded":"2023-05-20T01:19:52.928012+0000","last_fullsized":"2023-05-20T01:19:52.928012+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:45:56.940973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:54.136976+0000","last_change":"2023-05-20T01:19:51.946612+0000","last_active":"2023-05-20T01:19:54.136976+0000","last_peered":"2023-05-20T01:19:54.136976+0000","last_clean":"2023-05-20T01:19:54.136976+0000","last_became_active":"2023-05-20T01:19:51.946370+0000","last_became_peered":"2023-05-20T01:19:51.946370+0000","last_unstale":"2023-05-20T01:19:54.136976+0000","last_undegraded":"2023-05-20T01:19:54.136976+0000","last_fullsized":"2023-05-20T01:19:54.136976+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:22:12.352183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:52.928136+0000","last_change":"2023-05-20T01:19:51.501240+0000","last_active":"2023-05-20T01:19:52.928136+0000","last_peered":"2023-05-20T01:19:52.928136+0000","last_clean":"2023-05-20T01:19:52.928136+0000","last_became_active":"2023-05-20T01:19:51.500879+0000","last_became_peered":"2023-05-20T01:19:51.500879+0000","last_unstale":"2023-05-20T01:19:52.928136+0000","last_undegraded":"2023-05-20T01:19:52.928136+0000","last_fullsized":"2023-05-20T01:19:52.928136+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:57:28.696361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:53.636828+0000","last_change":"2023-05-20T01:19:51.938554+0000","last_active":"2023-05-20T01:19:53.636828+0000","last_peered":"2023-05-20T01:19:53.636828+0000","last_clean":"2023-05-20T01:19:53.636828+0000","last_became_active":"2023-05-20T01:19:51.938278+0000","last_became_peered":"2023-05-20T01:19:51.938278+0000","last_unstale":"2023-05-20T01:19:53.636828+0000","last_undegraded":"2023-05-20T01:19:53.636828+0000","last_fullsized":"2023-05-20T01:19:53.636828+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:57:57.525533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:54.136973+0000","last_change":"2023-05-20T01:19:51.503966+0000","last_active":"2023-05-20T01:19:54.136973+0000","last_peered":"2023-05-20T01:19:54.136973+0000","last_clean":"2023-05-20T01:19:54.136973+0000","last_became_active":"2023-05-20T01:19:51.503685+0000","last_became_peered":"2023-05-20T01:19:51.503685+0000","last_unstale":"2023-05-20T01:19:54.136973+0000","last_undegraded":"2023-05-20T01:19:54.136973+0000","last_fullsized":"2023-05-20T01:19:54.136973+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:46:26.729837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:52.934704+0000","last_change":"2023-05-20T01:19:51.931614+0000","last_active":"2023-05-20T01:19:52.934704+0000","last_peered":"2023-05-20T01:19:52.934704+0000","last_clean":"2023-05-20T01:19:52.934704+0000","last_became_active":"2023-05-20T01:19:51.931331+0000","last_became_peered":"2023-05-20T01:19:51.931331+0000","last_unstale":"2023-05-20T01:19:52.934704+0000","last_undegraded":"2023-05-20T01:19:52.934704+0000","last_fullsized":"2023-05-20T01:19:52.934704+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:05:02.922467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"63'1","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:54.138289+0000","last_change":"2023-05-20T01:19:51.511507+0000","last_active":"2023-05-20T01:19:54.138289+0000","last_peered":"2023-05-20T01:19:54.138289+0000","last_clean":"2023-05-20T01:19:54.138289+0000","last_became_active":"2023-05-20T01:19:51.511246+0000","last_became_peered":"2023-05-20T01:19:51.511246+0000","last_unstale":"2023-05-20T01:19:54.138289+0000","last_undegraded":"2023-05-20T01:19:54.138289+0000","last_fullsized":"2023-05-20T01:19:54.138289+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:44:58.536398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":102,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:52.934729+0000","last_change":"2023-05-20T01:16:23.289068+0000","last_active":"2023-05-20T01:19:52.934729+0000","last_peered":"2023-05-20T01:19:52.934729+0000","last_clean":"2023-05-20T01:19:52.934729+0000","last_became_active":"2023-05-20T01:16:23.288781+0000","last_became_peered":"2023-05-20T01:16:23.288781+0000","last_unstale":"2023-05-20T01:19:52.934729+0000","last_undegraded":"2023-05-20T01:19:52.934729+0000","last_fullsized":"2023-05-20T01:19:52.934729+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:05:36.497526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:52.934379+0000","last_change":"2023-05-20T01:19:51.935964+0000","last_active":"2023-05-20T01:19:52.934379+0000","last_peered":"2023-05-20T01:19:52.934379+0000","last_clean":"2023-05-20T01:19:52.934379+0000","last_became_active":"2023-05-20T01:19:51.935500+0000","last_became_peered":"2023-05-20T01:19:51.935500+0000","last_unstale":"2023-05-20T01:19:52.934379+0000","last_undegraded":"2023-05-20T01:19:52.934379+0000","last_fullsized":"2023-05-20T01:19:52.934379+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:13:08.801742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:52.928248+0000","last_change":"2023-05-20T01:19:51.512659+0000","last_active":"2023-05-20T01:19:52.928248+0000","last_peered":"2023-05-20T01:19:52.928248+0000","last_clean":"2023-05-20T01:19:52.928248+0000","last_became_active":"2023-05-20T01:19:51.512356+0000","last_became_peered":"2023-05-20T01:19:51.512356+0000","last_unstale":"2023-05-20T01:19:52.928248+0000","last_undegraded":"2023-05-20T01:19:52.928248+0000","last_fullsized":"2023-05-20T01:19:52.928248+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:44:56.148628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":106,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:52.928327+0000","last_change":"2023-05-20T01:16:22.829324+0000","last_active":"2023-05-20T01:19:52.928327+0000","last_peered":"2023-05-20T01:19:52.928327+0000","last_clean":"2023-05-20T01:19:52.928327+0000","last_became_active":"2023-05-20T01:16:22.828964+0000","last_became_peered":"2023-05-20T01:16:22.828964+0000","last_unstale":"2023-05-20T01:19:52.928327+0000","last_undegraded":"2023-05-20T01:19:52.928327+0000","last_fullsized":"2023-05-20T01:19:52.928327+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:04:16.729108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:53.436271+0000","last_change":"2023-05-20T01:19:51.936292+0000","last_active":"2023-05-20T01:19:53.436271+0000","last_peered":"2023-05-20T01:19:53.436271+0000","last_clean":"2023-05-20T01:19:53.436271+0000","last_became_active":"2023-05-20T01:19:51.935807+0000","last_became_peered":"2023-05-20T01:19:51.935807+0000","last_unstale":"2023-05-20T01:19:53.436271+0000","last_undegraded":"2023-05-20T01:19:53.436271+0000","last_fullsized":"2023-05-20T01:19:53.436271+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T01:22:42.445360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"63'2","reported_seq":15,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:53.436374+0000","last_change":"2023-05-20T01:19:51.508354+0000","last_active":"2023-05-20T01:19:53.436374+0000","last_peered":"2023-05-20T01:19:53.436374+0000","last_clean":"2023-05-20T01:19:53.436374+0000","last_became_active":"2023-05-20T01:19:51.508042+0000","last_became_peered":"2023-05-20T01:19:51.508042+0000","last_unstale":"2023-05-20T01:19:53.436374+0000","last_undegraded":"2023-05-20T01:19:53.436374+0000","last_fullsized":"2023-05-20T01:19:53.436374+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:06:58.676646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":98,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:54.137097+0000","last_change":"2023-05-20T01:16:22.834754+0000","last_active":"2023-05-20T01:19:54.137097+0000","last_peered":"2023-05-20T01:19:54.137097+0000","last_clean":"2023-05-20T01:19:54.137097+0000","last_became_active":"2023-05-20T01:16:22.834437+0000","last_became_peered":"2023-05-20T01:16:22.834437+0000","last_unstale":"2023-05-20T01:19:54.137097+0000","last_undegraded":"2023-05-20T01:19:54.137097+0000","last_fullsized":"2023-05-20T01:19:54.137097+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:12:53.831710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":238,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:54.137194+0000","last_change":"2023-05-20T01:16:22.834920+0000","last_active":"2023-05-20T01:19:54.137194+0000","last_peered":"2023-05-20T01:19:54.137194+0000","last_clean":"2023-05-20T01:19:54.137194+0000","last_became_active":"2023-05-20T01:16:22.833924+0000","last_became_peered":"2023-05-20T01:16:22.833924+0000","last_unstale":"2023-05-20T01:19:54.137194+0000","last_undegraded":"2023-05-20T01:19:54.137194+0000","last_fullsized":"2023-05-20T01:19:54.137194+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:17:10.132648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:54.137530+0000","last_change":"2023-05-20T01:19:51.937376+0000","last_active":"2023-05-20T01:19:54.137530+0000","last_peered":"2023-05-20T01:19:54.137530+0000","last_clean":"2023-05-20T01:19:54.137530+0000","last_became_active":"2023-05-20T01:19:51.937076+0000","last_became_peered":"2023-05-20T01:19:51.937076+0000","last_unstale":"2023-05-20T01:19:54.137530+0000","last_undegraded":"2023-05-20T01:19:54.137530+0000","last_fullsized":"2023-05-20T01:19:54.137530+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:22:01.609909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:54.137631+0000","last_change":"2023-05-20T01:19:51.508215+0000","last_active":"2023-05-20T01:19:54.137631+0000","last_peered":"2023-05-20T01:19:54.137631+0000","last_clean":"2023-05-20T01:19:54.137631+0000","last_became_active":"2023-05-20T01:19:51.507945+0000","last_became_peered":"2023-05-20T01:19:51.507945+0000","last_unstale":"2023-05-20T01:19:54.137631+0000","last_undegraded":"2023-05-20T01:19:54.137631+0000","last_fullsized":"2023-05-20T01:19:54.137631+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:18:56.763339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":88,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:54.137901+0000","last_change":"2023-05-20T01:16:23.290259+0000","last_active":"2023-05-20T01:19:54.137901+0000","last_peered":"2023-05-20T01:19:54.137901+0000","last_clean":"2023-05-20T01:19:54.137901+0000","last_became_active":"2023-05-20T01:16:23.289911+0000","last_became_peered":"2023-05-20T01:16:23.289911+0000","last_unstale":"2023-05-20T01:19:54.137901+0000","last_undegraded":"2023-05-20T01:19:54.137901+0000","last_fullsized":"2023-05-20T01:19:54.137901+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:02:09.647064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:52.934417+0000","last_change":"2023-05-20T01:19:51.928915+0000","last_active":"2023-05-20T01:19:52.934417+0000","last_peered":"2023-05-20T01:19:52.934417+0000","last_clean":"2023-05-20T01:19:52.934417+0000","last_became_active":"2023-05-20T01:19:51.928616+0000","last_became_peered":"2023-05-20T01:19:51.928616+0000","last_unstale":"2023-05-20T01:19:52.934417+0000","last_undegraded":"2023-05-20T01:19:52.934417+0000","last_fullsized":"2023-05-20T01:19:52.934417+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:16:53.546866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:52.927912+0000","last_change":"2023-05-20T01:19:51.506498+0000","last_active":"2023-05-20T01:19:52.927912+0000","last_peered":"2023-05-20T01:19:52.927912+0000","last_clean":"2023-05-20T01:19:52.927912+0000","last_became_active":"2023-05-20T01:19:51.506214+0000","last_became_peered":"2023-05-20T01:19:51.506214+0000","last_unstale":"2023-05-20T01:19:52.927912+0000","last_undegraded":"2023-05-20T01:19:52.927912+0000","last_fullsized":"2023-05-20T01:19:52.927912+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:01:30.964917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":88,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:54.137322+0000","last_change":"2023-05-20T01:16:23.290207+0000","last_active":"2023-05-20T01:19:54.137322+0000","last_peered":"2023-05-20T01:19:54.137322+0000","last_clean":"2023-05-20T01:19:54.137322+0000","last_became_active":"2023-05-20T01:16:23.289928+0000","last_became_peered":"2023-05-20T01:16:23.289928+0000","last_unstale":"2023-05-20T01:19:54.137322+0000","last_undegraded":"2023-05-20T01:19:54.137322+0000","last_fullsized":"2023-05-20T01:19:54.137322+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:36:03.264266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:52.934166+0000","last_change":"2023-05-20T01:19:51.929982+0000","last_active":"2023-05-20T01:19:52.934166+0000","last_peered":"2023-05-20T01:19:52.934166+0000","last_clean":"2023-05-20T01:19:52.934166+0000","last_became_active":"2023-05-20T01:19:51.929407+0000","last_became_peered":"2023-05-20T01:19:51.929407+0000","last_unstale":"2023-05-20T01:19:52.934166+0000","last_undegraded":"2023-05-20T01:19:52.934166+0000","last_fullsized":"2023-05-20T01:19:52.934166+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:36:52.793147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:53.436106+0000","last_change":"2023-05-20T01:19:51.510373+0000","last_active":"2023-05-20T01:19:53.436106+0000","last_peered":"2023-05-20T01:19:53.436106+0000","last_clean":"2023-05-20T01:19:53.436106+0000","last_became_active":"2023-05-20T01:19:51.510077+0000","last_became_peered":"2023-05-20T01:19:51.510077+0000","last_unstale":"2023-05-20T01:19:53.436106+0000","last_undegraded":"2023-05-20T01:19:53.436106+0000","last_fullsized":"2023-05-20T01:19:53.436106+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:32:36.503979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:54.137941+0000","last_change":"2023-05-20T01:16:23.290538+0000","last_active":"2023-05-20T01:19:54.137941+0000","last_peered":"2023-05-20T01:19:54.137941+0000","last_clean":"2023-05-20T01:19:54.137941+0000","last_became_active":"2023-05-20T01:16:23.290230+0000","last_became_peered":"2023-05-20T01:16:23.290230+0000","last_unstale":"2023-05-20T01:19:54.137941+0000","last_undegraded":"2023-05-20T01:19:54.137941+0000","last_fullsized":"2023-05-20T01:19:54.137941+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:16:04.109942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:52.934304+0000","last_change":"2023-05-20T01:19:51.934758+0000","last_active":"2023-05-20T01:19:52.934304+0000","last_peered":"2023-05-20T01:19:52.934304+0000","last_clean":"2023-05-20T01:19:52.934304+0000","last_became_active":"2023-05-20T01:19:51.933066+0000","last_became_peered":"2023-05-20T01:19:51.933066+0000","last_unstale":"2023-05-20T01:19:52.934304+0000","last_undegraded":"2023-05-20T01:19:52.934304+0000","last_fullsized":"2023-05-20T01:19:52.934304+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:09:17.142028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"63'2","reported_seq":15,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:54.138095+0000","last_change":"2023-05-20T01:19:51.513212+0000","last_active":"2023-05-20T01:19:54.138095+0000","last_peered":"2023-05-20T01:19:54.138095+0000","last_clean":"2023-05-20T01:19:54.138095+0000","last_became_active":"2023-05-20T01:19:51.512908+0000","last_became_peered":"2023-05-20T01:19:51.512908+0000","last_unstale":"2023-05-20T01:19:54.138095+0000","last_undegraded":"2023-05-20T01:19:54.138095+0000","last_fullsized":"2023-05-20T01:19:54.138095+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:19:59.308408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":90,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:52.934229+0000","last_change":"2023-05-20T01:16:23.290341+0000","last_active":"2023-05-20T01:19:52.934229+0000","last_peered":"2023-05-20T01:19:52.934229+0000","last_clean":"2023-05-20T01:19:52.934229+0000","last_became_active":"2023-05-20T01:16:23.290053+0000","last_became_peered":"2023-05-20T01:16:23.290053+0000","last_unstale":"2023-05-20T01:19:52.934229+0000","last_undegraded":"2023-05-20T01:19:52.934229+0000","last_fullsized":"2023-05-20T01:19:52.934229+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:47:27.554535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:54.137889+0000","last_change":"2023-05-20T01:19:51.944004+0000","last_active":"2023-05-20T01:19:54.137889+0000","last_peered":"2023-05-20T01:19:54.137889+0000","last_clean":"2023-05-20T01:19:54.137889+0000","last_became_active":"2023-05-20T01:19:51.943782+0000","last_became_peered":"2023-05-20T01:19:51.943782+0000","last_unstale":"2023-05-20T01:19:54.137889+0000","last_undegraded":"2023-05-20T01:19:54.137889+0000","last_fullsized":"2023-05-20T01:19:54.137889+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:44:50.726245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"63'1","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:54.138222+0000","last_change":"2023-05-20T01:19:51.512080+0000","last_active":"2023-05-20T01:19:54.138222+0000","last_peered":"2023-05-20T01:19:54.138222+0000","last_clean":"2023-05-20T01:19:54.138222+0000","last_became_active":"2023-05-20T01:19:51.511785+0000","last_became_peered":"2023-05-20T01:19:51.511785+0000","last_unstale":"2023-05-20T01:19:54.138222+0000","last_undegraded":"2023-05-20T01:19:54.138222+0000","last_fullsized":"2023-05-20T01:19:54.138222+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:53:34.483502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":90,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:52.934482+0000","last_change":"2023-05-20T01:16:22.834330+0000","last_active":"2023-05-20T01:19:52.934482+0000","last_peered":"2023-05-20T01:19:52.934482+0000","last_clean":"2023-05-20T01:19:52.934482+0000","last_became_active":"2023-05-20T01:16:22.833946+0000","last_became_peered":"2023-05-20T01:16:22.833946+0000","last_unstale":"2023-05-20T01:19:52.934482+0000","last_undegraded":"2023-05-20T01:19:52.934482+0000","last_fullsized":"2023-05-20T01:19:52.934482+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T01:52:19.713897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574895,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10636,"kb_used_data":1348,"kb_used_omap":0,"kb_used_meta":9280,"kb_avail":94361204,"statfs":{"total":96636764160,"available":96625872896,"internally_reserved":0,"allocated":1380352,"data_stored":966315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9502720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.52800000000000002,"15min":0.53400000000000003},"min":{"1min":0.29699999999999999,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.71799999999999997,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.56599999999999995,"15min":0.57099999999999995},"min":{"1min":0.38700000000000001,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.70099999999999996,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.61599999999999999}]},{"osd":1,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.79500000000000004,"15min":0.90700000000000003},"min":{"1min":0.435,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.78600000000000003,"5min":8.8879999999999999,"15min":8.8879999999999999},"last":0.503},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.83399999999999996,"15min":0.95099999999999996},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.73299999999999998,"5min":8.875,"15min":8.875},"last":0.59299999999999997}]},{"osd":2,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.77200000000000002,"15min":0.88},"min":{"1min":0.371,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.73299999999999998,"5min":8.9120000000000008,"15min":8.9120000000000008},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.77600000000000002,"15min":0.88400000000000001},"min":{"1min":0.35699999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.751,"5min":8.8520000000000003,"15min":8.8520000000000003},"last":0.52400000000000002}]},{"osd":3,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.84299999999999997,"15min":0.98299999999999998},"min":{"1min":0.35299999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.74299999999999999,"5min":10.884,"15min":10.884},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.89900000000000002,"15min":1.0349999999999999},"min":{"1min":0.433,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.72199999999999998,"5min":10.952,"15min":10.952},"last":0.63500000000000001}]},{"osd":4,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.71499999999999997,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.53200000000000003,"15min":0.52900000000000003},"min":{"1min":0.38800000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.67300000000000004,"5min":0.875,"15min":0.875},"last":0.60799999999999998}]},{"osd":5,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54300000000000004,"15min":0.52800000000000002},"min":{"1min":0.39600000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.69899999999999995,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.58199999999999996,"15min":0.57799999999999996},"min":{"1min":0.42699999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.745,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.70099999999999996}]},{"osd":6,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59299999999999997},"min":{"1min":0.47699999999999998,"5min":0.378,"15min":0.378},"max":{"1min":0.77000000000000002,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.59299999999999997,"15min":0.58499999999999996},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.83299999999999996,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.67800000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574894,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9404,"kb_used_data":756,"kb_used_omap":0,"kb_used_meta":8640,"kb_avail":94362436,"statfs":{"total":96636764160,"available":96627134464,"internally_reserved":0,"allocated":774144,"data_stored":375205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8847360},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.495,"15min":0.48699999999999999},"min":{"1min":0.34000000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.71699999999999997,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.54300000000000004,"15min":0.54100000000000004},"min":{"1min":0.36699999999999999,"5min":0.307,"15min":0.307},"max":{"1min":0.71199999999999997,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.40699999999999997}]},{"osd":1,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53500000000000003,"15min":0.53400000000000003},"min":{"1min":0.35899999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.68200000000000005,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53300000000000003,"15min":0.53200000000000003},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.78600000000000003,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.46999999999999997}]},{"osd":2,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.51700000000000002,"15min":0.51500000000000001},"min":{"1min":0.35499999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.68700000000000006,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.56299999999999994,"15min":0.56699999999999995},"min":{"1min":0.44500000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.439}]},{"osd":3,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.75900000000000001,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.54900000000000004,"15min":0.55000000000000004},"min":{"1min":0.39400000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.747,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.42599999999999999}]},{"osd":4,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.54000000000000004,"15min":0.56000000000000005},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.71799999999999997,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.56499999999999995,"15min":0.58399999999999996},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.72199999999999998,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.502}]},{"osd":5,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.54000000000000004,"15min":0.54800000000000004},"min":{"1min":0.373,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.70699999999999996,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.60599999999999998,"15min":0.621},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.80200000000000005,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.49099999999999999}]},{"osd":7,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.59799999999999998,"15min":0.61199999999999999},"min":{"1min":0.38600000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.77800000000000002,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.61499999999999999,"15min":0.628},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.749,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.47899999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574894,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10584,"kb_used_data":1360,"kb_used_omap":0,"kb_used_meta":9216,"kb_avail":94361256,"statfs":{"total":96636764160,"available":96625926144,"internally_reserved":0,"allocated":1392640,"data_stored":967867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9437184},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:19:24 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.32700000000000001,"15min":0.32900000000000001},"min":{"1min":0.193,"5min":0.193,"15min":0.193},"max":{"1min":1.194,"5min":1.194,"15min":1.194},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.35699999999999998,"5min":0.32000000000000001,"15min":0.316},"min":{"1min":0.20599999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.42599999999999999}]},{"osd":2,"last update":"Sat May 20 01:19:24 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.33000000000000002,"15min":0.32400000000000001},"min":{"1min":0.253,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.42199999999999999,"15min":0.42599999999999999},"min":{"1min":0.29599999999999999,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.3580000000000001,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":0.54700000000000004}]},{"osd":3,"last update":"Sat May 20 01:19:24 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.38500000000000001,"15min":0.376},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":1.619,"5min":1.619,"15min":1.619},"last":0.505},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.39800000000000002,"15min":0.39400000000000002},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":1.8400000000000001,"5min":1.8400000000000001,"15min":1.8400000000000001},"last":0.52500000000000002}]},{"osd":4,"last update":"Sat May 20 01:19:24 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.59599999999999997,"15min":0.59099999999999997},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.59099999999999997,"15min":0.58899999999999997},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.70399999999999996}]},{"osd":5,"last update":"Sat May 20 01:19:24 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.59999999999999998,"15min":0.58899999999999997},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.61599999999999999,"15min":0.61199999999999999},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.409,"5min":1.409,"15min":1.409},"last":0.79900000000000004}]},{"osd":6,"last update":"Sat May 20 01:19:24 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.61699999999999999,"15min":0.60599999999999998},"min":{"1min":0.47099999999999997,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.575,"5min":1.575,"15min":1.575},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.60199999999999998,"15min":0.59699999999999998},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.83099999999999996}]},{"osd":7,"last update":"Sat May 20 01:19:24 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.61299999999999999,"15min":0.60899999999999999},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.496,"5min":1.496,"15min":1.496},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.65100000000000002,"15min":0.64400000000000002},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.5169999999999999,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.87}]}]},{"osd":0,"up_from":13,"seq":55834574894,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9588,"kb_used_data":748,"kb_used_omap":0,"kb_used_meta":8832,"kb_avail":94362252,"statfs":{"total":96636764160,"available":96626946048,"internally_reserved":0,"allocated":765952,"data_stored":374210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9043968},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.31,"5min":0.32500000000000001,"15min":0.33300000000000002},"min":{"1min":0.188,"5min":0.188,"15min":0.188},"max":{"1min":0.47599999999999998,"5min":0.56599999999999995,"15min":0.56599999999999995},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.33400000000000002,"5min":0.35399999999999998,"15min":0.36799999999999999},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.53400000000000003,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.48799999999999999}]},{"osd":2,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.34300000000000003,"15min":0.34100000000000003},"min":{"1min":0.19400000000000001,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.39600000000000002,"15min":0.39900000000000002},"min":{"1min":0.29399999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.67000000000000004,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.56899999999999995}]},{"osd":3,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.36799999999999999,"15min":0.376},"min":{"1min":0.26100000000000001,"5min":0.245,"15min":0.245},"max":{"1min":0.51400000000000001,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.623},{"interface":"front","average":{"1min":0.375,"5min":0.372,"15min":0.372},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.50600000000000001,"5min":0.60099999999999998,"15min":0.60099999999999998},"last":0.60899999999999999}]},{"osd":4,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.54200000000000004,"15min":0.54000000000000004},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.76000000000000001,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.56599999999999995,"15min":0.56399999999999995},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.72299999999999998,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.85099999999999998}]},{"osd":5,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.57299999999999995,"15min":0.57699999999999996},"min":{"1min":0.40300000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.68799999999999994,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56799999999999995,"15min":0.56499999999999995},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.78900000000000003,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.69199999999999995}]},{"osd":6,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.57199999999999995,"15min":0.57399999999999995},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.69799999999999995,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.61799999999999999,"15min":0.623},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.82299999999999995}]},{"osd":7,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.59499999999999997,"15min":0.59099999999999997},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.78300000000000003,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.61799999999999999,"15min":0.624},"min":{"1min":0.43099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.72299999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574894,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9592,"kb_used_data":752,"kb_used_omap":0,"kb_used_meta":8832,"kb_avail":94362248,"statfs":{"total":96636764160,"available":96626941952,"internally_reserved":0,"allocated":770048,"data_stored":375171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9043968},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.32000000000000001,"5min":0.33900000000000002,"15min":0.35299999999999998},"min":{"1min":0.20100000000000001,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.58999999999999997,"5min":1.038,"15min":1.038},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.35699999999999998,"5min":0.35799999999999998,"15min":0.36699999999999999},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.52900000000000003,"5min":1.079,"15min":1.079},"last":0.32300000000000001}]},{"osd":1,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.39600000000000002,"15min":0.42099999999999999},"min":{"1min":0.26000000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.50900000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.436,"15min":0.45500000000000002},"min":{"1min":0.28399999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.64500000000000002,"5min":1.5189999999999999,"15min":1.5189999999999999},"last":0.37}]},{"osd":3,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.39700000000000002,"15min":0.41499999999999998},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.55000000000000004,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.379},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.45000000000000001,"15min":0.46999999999999997},"min":{"1min":0.32600000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.63100000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.39100000000000001}]},{"osd":4,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.58399999999999996,"15min":0.58999999999999997},"min":{"1min":0.42699999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.86499999999999999,"5min":1.319,"15min":1.319},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.625,"15min":0.626},"min":{"1min":0.48699999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.83899999999999997,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.63700000000000001}]},{"osd":5,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.64000000000000001,"15min":0.65500000000000003},"min":{"1min":0.44800000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.80100000000000005,"5min":1.758,"15min":1.758},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.625,"5min":0.65000000000000002,"15min":0.67000000000000004},"min":{"1min":0.48099999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.80800000000000005,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.76800000000000002}]},{"osd":6,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65100000000000002,"15min":0.65700000000000003},"min":{"1min":0.45500000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.88,"5min":1.665,"15min":1.665},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.626,"5min":0.64800000000000002,"15min":0.65900000000000003},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.86599999999999999,"5min":1.819,"15min":1.819},"last":0.69099999999999995}]},{"osd":7,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.68000000000000005,"15min":0.69199999999999995},"min":{"1min":0.45500000000000002,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.83199999999999996,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.66600000000000004,"15min":0.68400000000000005},"min":{"1min":0.49299999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.89300000000000002,"5min":1.8460000000000001,"15min":1.8460000000000001},"last":0.629}]}]},{"osd":3,"up_from":13,"seq":55834574894,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9656,"kb_used_data":752,"kb_used_omap":0,"kb_used_meta":8896,"kb_avail":94362184,"statfs":{"total":96636764160,"available":96626876416,"internally_reserved":0,"allocated":770048,"data_stored":375171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9109504},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:19:23 2023","interfaces":[{"interface":"back","average":{"1min":0.33100000000000002,"5min":0.318,"15min":0.32400000000000001},"min":{"1min":0.219,"5min":0.192,"15min":0.192},"max":{"1min":0.46100000000000002,"5min":0.504,"15min":0.504},"last":0.25600000000000001},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.42699999999999999,"15min":0.47599999999999998},"min":{"1min":0.247,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.52800000000000002,"5min":3.1890000000000001,"15min":3.1890000000000001},"last":0.34399999999999997}]},{"osd":1,"last update":"Sat May 20 01:19:23 2023","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.42099999999999999,"15min":0.46700000000000003},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.501,"5min":3.4260000000000002,"15min":3.4260000000000002},"last":0.378},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.36199999999999999,"15min":0.36399999999999999},"min":{"1min":0.26100000000000001,"5min":0.221,"15min":0.221},"max":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"last":0.29999999999999999}]},{"osd":2,"last update":"Sat May 20 01:19:23 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.39800000000000002,"15min":0.41299999999999998},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.62,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.41999999999999998,"15min":0.44},"min":{"1min":0.28699999999999998,"5min":0.253,"15min":0.253},"max":{"1min":0.58899999999999997,"5min":1.413,"15min":1.413},"last":0.36799999999999999}]},{"osd":4,"last update":"Sat May 20 01:19:23 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.59599999999999997,"15min":0.60799999999999998},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.80100000000000005,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.68300000000000005,"15min":0.72499999999999998},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.83599999999999997,"5min":3.4660000000000002,"15min":3.4660000000000002},"last":0.60399999999999998}]},{"osd":5,"last update":"Sat May 20 01:19:23 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.69399999999999995,"15min":0.73999999999999999},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.77900000000000003,"5min":3.5379999999999998,"15min":3.5379999999999998},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.63300000000000001,"15min":0.65300000000000002},"min":{"1min":0.42599999999999999,"5min":0.374,"15min":0.374},"max":{"1min":0.81699999999999995,"5min":1.385,"15min":1.385},"last":0.78200000000000003}]},{"osd":6,"last update":"Sat May 20 01:19:23 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.63800000000000001,"15min":0.65800000000000003},"min":{"1min":0.40400000000000003,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.84299999999999997,"5min":1.347,"15min":1.347},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64200000000000002,"15min":0.65700000000000003},"min":{"1min":0.434,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.77400000000000002,"5min":1.107,"15min":1.107},"last":0.56699999999999995}]},{"osd":7,"last update":"Sat May 20 01:19:23 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.64600000000000002,"15min":0.66000000000000003},"min":{"1min":0.46600000000000003,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.83199999999999996,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.72899999999999998,"15min":0.77400000000000002},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.82199999999999995,"5min":3.5009999999999999,"15min":3.5009999999999999},"last":0.69699999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574894,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9988,"kb_used_data":764,"kb_used_omap":0,"kb_used_meta":9216,"kb_avail":94361852,"statfs":{"total":96636764160,"available":96626536448,"internally_reserved":0,"allocated":782336,"data_stored":376735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9437184},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:19:24 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.495,"15min":0.48099999999999998},"min":{"1min":0.379,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.72399999999999998,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.497,"15min":0.49299999999999999},"min":{"1min":0.35499999999999998,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.55200000000000005}]},{"osd":1,"last update":"Sat May 20 01:19:24 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.53400000000000003,"15min":0.53200000000000003},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.73299999999999998,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.622},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.54600000000000004,"15min":0.53200000000000003},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":2.0739999999999998,"5min":2.0739999999999998,"15min":2.0739999999999998},"last":0.498}]},{"osd":2,"last update":"Sat May 20 01:19:24 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.51400000000000001,"15min":0.499},"min":{"1min":0.439,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.745,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.55400000000000005,"15min":0.55500000000000005},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.751,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.60899999999999999}]},{"osd":3,"last update":"Sat May 20 01:19:24 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.53300000000000003,"15min":0.51900000000000002},"min":{"1min":0.42699999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.827,"5min":1.827,"15min":1.827},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.66700000000000004,"15min":0.70699999999999996},"min":{"1min":0.40400000000000003,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.77400000000000002,"5min":3.1749999999999998,"15min":3.1749999999999998},"last":0.59699999999999998}]},{"osd":5,"last update":"Sat May 20 01:19:24 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.47199999999999998,"15min":0.46000000000000002},"min":{"1min":0.33800000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.63100000000000001,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.49199999999999999,"15min":0.49299999999999999},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.67300000000000004,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.35399999999999998}]},{"osd":6,"last update":"Sat May 20 01:19:24 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52900000000000003,"15min":0.52500000000000002},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.64700000000000002,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.58199999999999996,"15min":0.57799999999999996},"min":{"1min":0.46200000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.65500000000000003,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.58299999999999996}]},{"osd":7,"last update":"Sat May 20 01:19:24 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.54800000000000004,"15min":0.54500000000000004},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.443},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.58599999999999997,"15min":0.58999999999999997},"min":{"1min":0.39100000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.67200000000000004,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.47099999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574895,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9464,"kb_used_data":752,"kb_used_omap":0,"kb_used_meta":8704,"kb_avail":94362376,"statfs":{"total":96636764160,"available":96627073024,"internally_reserved":0,"allocated":770048,"data_stored":374232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8912896},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:19:26 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.68799999999999994,"15min":0.77400000000000002},"min":{"1min":0.33700000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.71899999999999997,"5min":6.3929999999999998,"15min":6.3929999999999998},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.72199999999999998,"15min":0.81200000000000006},"min":{"1min":0.434,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.745,"5min":6.3339999999999996,"15min":6.3339999999999996},"last":0.503}]},{"osd":1,"last update":"Sat May 20 01:19:26 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.60999999999999999,"15min":0.66500000000000004},"min":{"1min":0.38100000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.73899999999999999,"5min":3.2450000000000001,"15min":3.2450000000000001},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54500000000000004,"15min":0.55500000000000005},"min":{"1min":0.41199999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.71899999999999997,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.48099999999999998}]},{"osd":2,"last update":"Sat May 20 01:19:26 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.55500000000000005,"15min":0.57599999999999996},"min":{"1min":0.34100000000000003,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.76300000000000001,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.56100000000000005,"15min":0.57899999999999996},"min":{"1min":0.375,"5min":0.314,"15min":0.314},"max":{"1min":0.72299999999999998,"5min":1.258,"15min":1.258},"last":0.65500000000000003}]},{"osd":3,"last update":"Sat May 20 01:19:26 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.751,"15min":0.85699999999999998},"min":{"1min":0.38200000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.68600000000000005,"5min":6.351,"15min":6.351},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.79300000000000004,"15min":0.89800000000000002},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.72599999999999998,"5min":6.3730000000000002,"15min":6.3730000000000002},"last":0.68200000000000005}]},{"osd":4,"last update":"Sat May 20 01:19:26 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.48599999999999999,"15min":0.48799999999999999},"min":{"1min":0.32400000000000001,"5min":0.316,"15min":0.316},"max":{"1min":0.65300000000000002,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.54800000000000004,"15min":0.56000000000000005},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.73099999999999998}]},{"osd":6,"last update":"Sat May 20 01:19:26 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.54300000000000004,"15min":0.55900000000000005},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.67600000000000005,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.56000000000000005,"15min":0.56599999999999995},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.747,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.69599999999999995}]},{"osd":7,"last update":"Sat May 20 01:19:26 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.57899999999999996,"15min":0.59199999999999997},"min":{"1min":0.36199999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.76900000000000002,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.61299999999999999,"15min":0.627},"min":{"1min":0.41399999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.73399999999999999,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.71499999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-05-20T01:20:01.147 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-05-20T01:20:01.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.283+0000 7f3217390700 1 Processor -- start 2023-05-20T01:20:01.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.283+0000 7f3217390700 1 -- start start 2023-05-20T01:20:01.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.284+0000 7f3217390700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f32101599b0 0x7f321015ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:20:01.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.284+0000 7f3217390700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f32100b0910 0x7f32100b0d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:20:01.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.284+0000 7f3217390700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32100b12b0 con 0x7f32100cd0f0 2023-05-20T01:20:01.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.284+0000 7f3217390700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32100ced30 con 0x7f32100b0910 2023-05-20T01:20:01.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.284+0000 7f3217390700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32100ca1b0 con 0x7f32101599b0 2023-05-20T01:20:01.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.284+0000 7f321492b700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f32100cd0f0 0x7f32100ce620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:45478/0 (socket says 172.21.15.16:45478) 2023-05-20T01:20:01.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.284+0000 7f321492b700 1 -- 172.21.15.16:0/2800571490 learned_addr learned my addr 172.21.15.16:0/2800571490 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:20:01.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.285+0000 7f321592d700 1 --2- 172.21.15.16:0/2800571490 >> v2:172.21.15.132:3300/0 conn(0x7f32100b0910 0x7f32100b0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:20:01.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.285+0000 7f3207fff700 1 -- 172.21.15.16:0/2800571490 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1606643898 0 0) 0x7f32100b12b0 con 0x7f32100cd0f0 2023-05-20T01:20:01.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.285+0000 7f321512c700 1 --2- 172.21.15.16:0/2800571490 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f32101599b0 0x7f321015ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:20:01.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.285+0000 7f3207fff700 1 -- 172.21.15.16:0/2800571490 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f31fc0057c0 con 0x7f32100cd0f0 2023-05-20T01:20:01.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.285+0000 7f3207fff700 1 -- 172.21.15.16:0/2800571490 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3463250007 0 0) 0x7f31fc0057c0 con 0x7f32100cd0f0 2023-05-20T01:20:01.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.285+0000 7f3207fff700 1 -- 172.21.15.16:0/2800571490 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f32101599b0 msgr2=0x7f321015ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:20:01.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.285+0000 7f3207fff700 1 --2- 172.21.15.16:0/2800571490 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f32101599b0 0x7f321015ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:20:01.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.285+0000 7f3207fff700 1 -- 172.21.15.16:0/2800571490 >> v2:172.21.15.132:3300/0 conn(0x7f32100b0910 msgr2=0x7f32100b0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:20:01.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.285+0000 7f3207fff700 1 --2- 172.21.15.16:0/2800571490 >> v2:172.21.15.132:3300/0 conn(0x7f32100b0910 0x7f32100b0d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:20:01.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.285+0000 7f3207fff700 1 -- 172.21.15.16:0/2800571490 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f321015e300 con 0x7f32100cd0f0 2023-05-20T01:20:01.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.285+0000 7f3207fff700 1 -- 172.21.15.16:0/2800571490 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7f320c001670 con 0x7f32100cd0f0 2023-05-20T01:20:01.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.285+0000 7f3207fff700 1 -- 172.21.15.16:0/2800571490 >> v1:172.21.15.16:6789/0 conn(0x7f32100cd0f0 legacy=0x7f32100ce620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:20:01.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.285+0000 7f3207fff700 1 --2- 172.21.15.16:0/2800571490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31fc005c80 0x7f31fc006090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:20:01.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.285+0000 7f3207fff700 1 --2- 172.21.15.16:0/2800571490 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f31fc006a60 0x7f31fc006ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:20:01.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.285+0000 7f3207fff700 1 --2- 172.21.15.16:0/2800571490 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f32101599b0 0x7f31fc007410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:20:01.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.285+0000 7f3207fff700 1 -- 172.21.15.16:0/2800571490 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32100ced30 con 0x7f31fc005c80 2023-05-20T01:20:01.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.285+0000 7f3207fff700 1 -- 172.21.15.16:0/2800571490 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32100ca1b0 con 0x7f31fc006a60 2023-05-20T01:20:01.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.285+0000 7f3207fff700 1 -- 172.21.15.16:0/2800571490 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31fc00dc20 con 0x7f32101599b0 2023-05-20T01:20:01.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.285+0000 7f321512c700 1 --2- 172.21.15.16:0/2800571490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31fc005c80 0x7f31fc006090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:20:01.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.285+0000 7f321492b700 1 --2- 172.21.15.16:0/2800571490 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f31fc006a60 0x7f31fc006ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:20:01.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.285+0000 7f321592d700 1 --2- 172.21.15.16:0/2800571490 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f32101599b0 0x7f31fc007410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:20:01.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.285+0000 7f321512c700 1 -- 172.21.15.16:0/2800571490 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f32101599b0 msgr2=0x7f31fc007410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:20:01.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.285+0000 7f321512c700 1 --2- 172.21.15.16:0/2800571490 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f32101599b0 0x7f31fc007410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:20:01.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.285+0000 7f321512c700 1 -- 172.21.15.16:0/2800571490 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f31fc006a60 msgr2=0x7f31fc006ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:20:01.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.285+0000 7f321512c700 1 --2- 172.21.15.16:0/2800571490 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f31fc006a60 0x7f31fc006ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:20:01.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.285+0000 7f321512c700 1 -- 172.21.15.16:0/2800571490 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31fc0057c0 con 0x7f31fc005c80 2023-05-20T01:20:01.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.286+0000 7f321492b700 1 --2- 172.21.15.16:0/2800571490 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f31fc006a60 0x7f31fc006ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:20:01.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.286+0000 7f321512c700 1 --2- 172.21.15.16:0/2800571490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31fc005c80 0x7f31fc006090 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f3200009b60 tx=0x7f31fc007a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11737a1522da5c9f server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:20:01.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.286+0000 7f3207fff700 1 -- 172.21.15.16:0/2800571490 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f320000e6c0 con 0x7f31fc005c80 2023-05-20T01:20:01.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.286+0000 7f3207fff700 1 -- 172.21.15.16:0/2800571490 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f320000e820 con 0x7f31fc005c80 2023-05-20T01:20:01.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.286+0000 7f3207fff700 1 -- 172.21.15.16:0/2800571490 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f320000eb90 con 0x7f31fc005c80 2023-05-20T01:20:01.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.286+0000 7f3217390700 1 -- 172.21.15.16:0/2800571490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31fc005c80 msgr2=0x7f31fc006090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:20:01.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.286+0000 7f3217390700 1 --2- 172.21.15.16:0/2800571490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31fc005c80 0x7f31fc006090 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f3200009b60 tx=0x7f31fc007a90 comp rx=0 tx=0).stop 2023-05-20T01:20:01.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.286+0000 7f321592d700 1 -- 172.21.15.16:0/2800571490 reap_dead start 2023-05-20T01:20:01.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.286+0000 7f3217390700 1 -- 172.21.15.16:0/2800571490 shutdown_connections 2023-05-20T01:20:01.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.286+0000 7f3217390700 1 -- 172.21.15.16:0/2800571490 >> 172.21.15.16:0/2800571490 conn(0x7f32100cb910 msgr2=0x7f32100cbd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:20:01.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.287+0000 7f3217390700 1 -- 172.21.15.16:0/2800571490 shutdown_connections 2023-05-20T01:20:01.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.287+0000 7f3217390700 1 -- 172.21.15.16:0/2800571490 wait complete. 2023-05-20T01:20:01.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.288+0000 7f3217390700 1 Processor -- start 2023-05-20T01:20:01.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.288+0000 7f3217390700 1 -- start start 2023-05-20T01:20:01.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.289+0000 7f3217390700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3210149240 0x7f32100cf260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:20:01.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.289+0000 7f3217390700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3210149610 0x7f32100cf7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:20:01.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.289+0000 7f3217390700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3210151940 0x7f3210151de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:20:01.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.289+0000 7f3217390700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3210152870 con 0x7f3210149610 2023-05-20T01:20:01.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.289+0000 7f3217390700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3210152320 con 0x7f3210149240 2023-05-20T01:20:01.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.289+0000 7f3217390700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3210152590 con 0x7f3210151940 2023-05-20T01:20:01.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.289+0000 7f321492b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3210149610 0x7f32100cf7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:20:01.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.289+0000 7f321492b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3210149610 0x7f32100cf7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33962/0 (socket says 172.21.15.16:33962) 2023-05-20T01:20:01.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.289+0000 7f321492b700 1 -- 172.21.15.16:0/3645556309 learned_addr learned my addr 172.21.15.16:0/3645556309 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:20:01.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.289+0000 7f321512c700 1 --2- 172.21.15.16:0/3645556309 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3210149240 0x7f32100cf260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:20:01.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.290+0000 7f321592d700 1 --2- 172.21.15.16:0/3645556309 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3210151940 0x7f3210151de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:20:01.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.290+0000 7f321492b700 1 -- 172.21.15.16:0/3645556309 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3210151940 msgr2=0x7f3210151de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:20:01.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.290+0000 7f321492b700 1 --2- 172.21.15.16:0/3645556309 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3210151940 0x7f3210151de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:20:01.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.290+0000 7f321492b700 1 -- 172.21.15.16:0/3645556309 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3210149240 msgr2=0x7f32100cf260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:20:01.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.290+0000 7f321492b700 1 --2- 172.21.15.16:0/3645556309 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3210149240 0x7f32100cf260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:20:01.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.290+0000 7f321492b700 1 -- 172.21.15.16:0/3645556309 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32100cd860 con 0x7f3210149610 2023-05-20T01:20:01.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.290+0000 7f321592d700 1 --2- 172.21.15.16:0/3645556309 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3210151940 0x7f3210151de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:20:01.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.290+0000 7f321492b700 1 --2- 172.21.15.16:0/3645556309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3210149610 0x7f32100cf7a0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f320c00e780 tx=0x7f320c00ea90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:20:01.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.290+0000 7f3205ffb700 1 -- 172.21.15.16:0/3645556309 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f320c012650 con 0x7f3210149610 2023-05-20T01:20:01.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.290+0000 7f3205ffb700 1 -- 172.21.15.16:0/3645556309 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f320c0127b0 con 0x7f3210149610 2023-05-20T01:20:01.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.290+0000 7f3205ffb700 1 -- 172.21.15.16:0/3645556309 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f320c012b20 con 0x7f3210149610 2023-05-20T01:20:01.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.290+0000 7f3217390700 1 -- 172.21.15.16:0/3645556309 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32100cd580 con 0x7f3210149610 2023-05-20T01:20:01.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.290+0000 7f3217390700 1 -- 172.21.15.16:0/3645556309 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32100cdb40 con 0x7f3210149610 2023-05-20T01:20:01.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.291+0000 7f3217390700 1 -- 172.21.15.16:0/3645556309 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3210159d80 con 0x7f3210149610 2023-05-20T01:20:01.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.292+0000 7f3205ffb700 1 -- 172.21.15.16:0/3645556309 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f320c012c80 con 0x7f3210149610 2023-05-20T01:20:01.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.292+0000 7f3205ffb700 1 --2- 172.21.15.16:0/3645556309 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f31f406ace0 0x7f31f406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:20:01.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.292+0000 7f3205ffb700 1 -- 172.21.15.16:0/3645556309 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f320c089290 con 0x7f3210149610 2023-05-20T01:20:01.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.292+0000 7f321512c700 1 --2- 172.21.15.16:0/3645556309 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f31f406ace0 0x7f31f406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:20:01.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.293+0000 7f321512c700 1 --2- 172.21.15.16:0/3645556309 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f31f406ace0 0x7f31f406d190 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f32100cc960 tx=0x7f320001f040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:20:01.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.297+0000 7f3205ffb700 1 -- 172.21.15.16:0/3645556309 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f3210159d80 con 0x7f3210149610 2023-05-20T01:20:01.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.490+0000 7f3217390700 1 -- 172.21.15.16:0/3645556309 --> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3210159d80 con 0x7f31f406ace0 2023-05-20T01:20:01.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.499+0000 7f3205ffb700 1 -- 172.21.15.16:0/3645556309 <== mgr.4104 v2:172.21.15.16:6832/3817237998 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311760 (secure 0 0 0) 0x7f3210159d80 con 0x7f31f406ace0 2023-05-20T01:20:01.508 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:20:01.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.503+0000 7f3217390700 1 -- 172.21.15.16:0/3645556309 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f31f406ace0 msgr2=0x7f31f406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:20:01.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.503+0000 7f3217390700 1 --2- 172.21.15.16:0/3645556309 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f31f406ace0 0x7f31f406d190 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f32100cc960 tx=0x7f320001f040 comp rx=0 tx=0).stop 2023-05-20T01:20:01.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.503+0000 7f3217390700 1 -- 172.21.15.16:0/3645556309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3210149610 msgr2=0x7f32100cf7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:20:01.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.503+0000 7f3217390700 1 --2- 172.21.15.16:0/3645556309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3210149610 0x7f32100cf7a0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f320c00e780 tx=0x7f320c00ea90 comp rx=0 tx=0).stop 2023-05-20T01:20:01.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.503+0000 7f3217390700 1 -- 172.21.15.16:0/3645556309 shutdown_connections 2023-05-20T01:20:01.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.503+0000 7f3217390700 1 --2- 172.21.15.16:0/3645556309 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f31f406ace0 0x7f31f406d190 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:20:01.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.503+0000 7f3217390700 1 --2- 172.21.15.16:0/3645556309 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3210151940 0x7f3210151de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:20:01.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.503+0000 7f3217390700 1 --2- 172.21.15.16:0/3645556309 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3210149240 0x7f32100cf260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:20:01.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.503+0000 7f3217390700 1 --2- 172.21.15.16:0/3645556309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3210149610 0x7f32100cf7a0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:20:01.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.503+0000 7f3217390700 1 -- 172.21.15.16:0/3645556309 >> 172.21.15.16:0/3645556309 conn(0x7f32100cb910 msgr2=0x7f321006c340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:20:01.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.503+0000 7f3217390700 1 -- 172.21.15.16:0/3645556309 shutdown_connections 2023-05-20T01:20:01.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.503+0000 7f3217390700 1 -- 172.21.15.16:0/3645556309 wait complete. 2023-05-20T01:20:01.513 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2023-05-20T01:20:01.520 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":168,"stamp":"2023-05-20T01:20:01.365736+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":78912,"kb_used_data":7232,"kb_used_omap":0,"kb_used_meta":71616,"kb_avail":754895808,"statfs":{"total":773094113280,"available":773013307392,"internally_reserved":0,"allocated":7405568,"data_stored":4184906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":73334784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.448254"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:54.137250+0000","last_change":"2023-05-20T01:19:51.945445+0000","last_active":"2023-05-20T01:19:54.137250+0000","last_peered":"2023-05-20T01:19:54.137250+0000","last_clean":"2023-05-20T01:19:54.137250+0000","last_became_active":"2023-05-20T01:19:51.945160+0000","last_became_peered":"2023-05-20T01:19:51.945160+0000","last_unstale":"2023-05-20T01:19:54.137250+0000","last_undegraded":"2023-05-20T01:19:54.137250+0000","last_fullsized":"2023-05-20T01:19:54.137250+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:04:47.770805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:54.137442+0000","last_change":"2023-05-20T01:19:51.933445+0000","last_active":"2023-05-20T01:19:54.137442+0000","last_peered":"2023-05-20T01:19:54.137442+0000","last_clean":"2023-05-20T01:19:54.137442+0000","last_became_active":"2023-05-20T01:19:51.933189+0000","last_became_peered":"2023-05-20T01:19:51.933189+0000","last_unstale":"2023-05-20T01:19:54.137442+0000","last_undegraded":"2023-05-20T01:19:54.137442+0000","last_fullsized":"2023-05-20T01:19:54.137442+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:51:19.406691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:54.138009+0000","last_change":"2023-05-20T01:19:51.944350+0000","last_active":"2023-05-20T01:19:54.138009+0000","last_peered":"2023-05-20T01:19:54.138009+0000","last_clean":"2023-05-20T01:19:54.138009+0000","last_became_active":"2023-05-20T01:19:51.941812+0000","last_became_peered":"2023-05-20T01:19:51.941812+0000","last_unstale":"2023-05-20T01:19:54.138009+0000","last_undegraded":"2023-05-20T01:19:54.138009+0000","last_fullsized":"2023-05-20T01:19:54.138009+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:46:21.957714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:52.934128+0000","last_change":"2023-05-20T01:19:51.931092+0000","last_active":"2023-05-20T01:19:52.934128+0000","last_peered":"2023-05-20T01:19:52.934128+0000","last_clean":"2023-05-20T01:19:52.934128+0000","last_became_active":"2023-05-20T01:19:51.929780+0000","last_became_peered":"2023-05-20T01:19:51.929780+0000","last_unstale":"2023-05-20T01:19:52.934128+0000","last_undegraded":"2023-05-20T01:19:52.934128+0000","last_fullsized":"2023-05-20T01:19:52.934128+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T13:02:26.622523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:52.934254+0000","last_change":"2023-05-20T01:19:51.945244+0000","last_active":"2023-05-20T01:19:52.934254+0000","last_peered":"2023-05-20T01:19:52.934254+0000","last_clean":"2023-05-20T01:19:52.934254+0000","last_became_active":"2023-05-20T01:19:51.944571+0000","last_became_peered":"2023-05-20T01:19:51.944571+0000","last_unstale":"2023-05-20T01:19:52.934254+0000","last_undegraded":"2023-05-20T01:19:52.934254+0000","last_fullsized":"2023-05-20T01:19:52.934254+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:33:23.171998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:53.436345+0000","last_change":"2023-05-20T01:19:51.938758+0000","last_active":"2023-05-20T01:19:53.436345+0000","last_peered":"2023-05-20T01:19:53.436345+0000","last_clean":"2023-05-20T01:19:53.436345+0000","last_became_active":"2023-05-20T01:19:51.938496+0000","last_became_peered":"2023-05-20T01:19:51.938496+0000","last_unstale":"2023-05-20T01:19:53.436345+0000","last_undegraded":"2023-05-20T01:19:53.436345+0000","last_fullsized":"2023-05-20T01:19:53.436345+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:17:05.433211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:53.636999+0000","last_change":"2023-05-20T01:19:51.942231+0000","last_active":"2023-05-20T01:19:53.636999+0000","last_peered":"2023-05-20T01:19:53.636999+0000","last_clean":"2023-05-20T01:19:53.636999+0000","last_became_active":"2023-05-20T01:19:51.941916+0000","last_became_peered":"2023-05-20T01:19:51.941916+0000","last_unstale":"2023-05-20T01:19:53.636999+0000","last_undegraded":"2023-05-20T01:19:53.636999+0000","last_fullsized":"2023-05-20T01:19:53.636999+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T01:48:59.400838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:54.137212+0000","last_change":"2023-05-20T01:19:51.944138+0000","last_active":"2023-05-20T01:19:54.137212+0000","last_peered":"2023-05-20T01:19:54.137212+0000","last_clean":"2023-05-20T01:19:54.137212+0000","last_became_active":"2023-05-20T01:19:51.943872+0000","last_became_peered":"2023-05-20T01:19:51.943872+0000","last_unstale":"2023-05-20T01:19:54.137212+0000","last_undegraded":"2023-05-20T01:19:54.137212+0000","last_fullsized":"2023-05-20T01:19:54.137212+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:43:07.975515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:54.138201+0000","last_change":"2023-05-20T01:19:51.947064+0000","last_active":"2023-05-20T01:19:54.138201+0000","last_peered":"2023-05-20T01:19:54.138201+0000","last_clean":"2023-05-20T01:19:54.138201+0000","last_became_active":"2023-05-20T01:19:51.946725+0000","last_became_peered":"2023-05-20T01:19:51.946725+0000","last_unstale":"2023-05-20T01:19:54.138201+0000","last_undegraded":"2023-05-20T01:19:54.138201+0000","last_fullsized":"2023-05-20T01:19:54.138201+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:38:01.198660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:54.138056+0000","last_change":"2023-05-20T01:19:51.938942+0000","last_active":"2023-05-20T01:19:54.138056+0000","last_peered":"2023-05-20T01:19:54.138056+0000","last_clean":"2023-05-20T01:19:54.138056+0000","last_became_active":"2023-05-20T01:19:51.938510+0000","last_became_peered":"2023-05-20T01:19:51.938510+0000","last_unstale":"2023-05-20T01:19:54.138056+0000","last_undegraded":"2023-05-20T01:19:54.138056+0000","last_fullsized":"2023-05-20T01:19:54.138056+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:08:52.099992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:52.934501+0000","last_change":"2023-05-20T01:19:51.934286+0000","last_active":"2023-05-20T01:19:52.934501+0000","last_peered":"2023-05-20T01:19:52.934501+0000","last_clean":"2023-05-20T01:19:52.934501+0000","last_became_active":"2023-05-20T01:19:51.933953+0000","last_became_peered":"2023-05-20T01:19:51.933953+0000","last_unstale":"2023-05-20T01:19:52.934501+0000","last_undegraded":"2023-05-20T01:19:52.934501+0000","last_fullsized":"2023-05-20T01:19:52.934501+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:14:27.490210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:54.137891+0000","last_change":"2023-05-20T01:19:51.938231+0000","last_active":"2023-05-20T01:19:54.137891+0000","last_peered":"2023-05-20T01:19:54.137891+0000","last_clean":"2023-05-20T01:19:54.137891+0000","last_became_active":"2023-05-20T01:19:51.937947+0000","last_became_peered":"2023-05-20T01:19:51.937947+0000","last_unstale":"2023-05-20T01:19:54.137891+0000","last_undegraded":"2023-05-20T01:19:54.137891+0000","last_fullsized":"2023-05-20T01:19:54.137891+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:55:42.050629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:54.137657+0000","last_change":"2023-05-20T01:19:51.947945+0000","last_active":"2023-05-20T01:19:54.137657+0000","last_peered":"2023-05-20T01:19:54.137657+0000","last_clean":"2023-05-20T01:19:54.137657+0000","last_became_active":"2023-05-20T01:19:51.947696+0000","last_became_peered":"2023-05-20T01:19:51.947696+0000","last_unstale":"2023-05-20T01:19:54.137657+0000","last_undegraded":"2023-05-20T01:19:54.137657+0000","last_fullsized":"2023-05-20T01:19:54.137657+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:40:27.165242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:54.137495+0000","last_change":"2023-05-20T01:19:51.943523+0000","last_active":"2023-05-20T01:19:54.137495+0000","last_peered":"2023-05-20T01:19:54.137495+0000","last_clean":"2023-05-20T01:19:54.137495+0000","last_became_active":"2023-05-20T01:19:51.941751+0000","last_became_peered":"2023-05-20T01:19:51.941751+0000","last_unstale":"2023-05-20T01:19:54.137495+0000","last_undegraded":"2023-05-20T01:19:54.137495+0000","last_fullsized":"2023-05-20T01:19:54.137495+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:13:38.799198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:54.137966+0000","last_change":"2023-05-20T01:19:51.941319+0000","last_active":"2023-05-20T01:19:54.137966+0000","last_peered":"2023-05-20T01:19:54.137966+0000","last_clean":"2023-05-20T01:19:54.137966+0000","last_became_active":"2023-05-20T01:19:51.940969+0000","last_became_peered":"2023-05-20T01:19:51.940969+0000","last_unstale":"2023-05-20T01:19:54.137966+0000","last_undegraded":"2023-05-20T01:19:54.137966+0000","last_fullsized":"2023-05-20T01:19:54.137966+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:34:27.000278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:53.637000+0000","last_change":"2023-05-20T01:19:51.935543+0000","last_active":"2023-05-20T01:19:53.637000+0000","last_peered":"2023-05-20T01:19:53.637000+0000","last_clean":"2023-05-20T01:19:53.637000+0000","last_became_active":"2023-05-20T01:19:51.935287+0000","last_became_peered":"2023-05-20T01:19:51.935287+0000","last_unstale":"2023-05-20T01:19:53.637000+0000","last_undegraded":"2023-05-20T01:19:53.637000+0000","last_fullsized":"2023-05-20T01:19:53.637000+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:31:05.916596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:54.137372+0000","last_change":"2023-05-20T01:19:51.947938+0000","last_active":"2023-05-20T01:19:54.137372+0000","last_peered":"2023-05-20T01:19:54.137372+0000","last_clean":"2023-05-20T01:19:54.137372+0000","last_became_active":"2023-05-20T01:19:51.947650+0000","last_became_peered":"2023-05-20T01:19:51.947650+0000","last_unstale":"2023-05-20T01:19:54.137372+0000","last_undegraded":"2023-05-20T01:19:54.137372+0000","last_fullsized":"2023-05-20T01:19:54.137372+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:47:04.749262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:52.934606+0000","last_change":"2023-05-20T01:19:51.932362+0000","last_active":"2023-05-20T01:19:52.934606+0000","last_peered":"2023-05-20T01:19:52.934606+0000","last_clean":"2023-05-20T01:19:52.934606+0000","last_became_active":"2023-05-20T01:19:51.932135+0000","last_became_peered":"2023-05-20T01:19:51.932135+0000","last_unstale":"2023-05-20T01:19:52.934606+0000","last_undegraded":"2023-05-20T01:19:52.934606+0000","last_fullsized":"2023-05-20T01:19:52.934606+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:24:56.004774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:52.934130+0000","last_change":"2023-05-20T01:19:51.936075+0000","last_active":"2023-05-20T01:19:52.934130+0000","last_peered":"2023-05-20T01:19:52.934130+0000","last_clean":"2023-05-20T01:19:52.934130+0000","last_became_active":"2023-05-20T01:19:51.935677+0000","last_became_peered":"2023-05-20T01:19:51.935677+0000","last_unstale":"2023-05-20T01:19:52.934130+0000","last_undegraded":"2023-05-20T01:19:52.934130+0000","last_fullsized":"2023-05-20T01:19:52.934130+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:52:47.813260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:52.928122+0000","last_change":"2023-05-20T01:19:51.930152+0000","last_active":"2023-05-20T01:19:52.928122+0000","last_peered":"2023-05-20T01:19:52.928122+0000","last_clean":"2023-05-20T01:19:52.928122+0000","last_became_active":"2023-05-20T01:19:51.929859+0000","last_became_peered":"2023-05-20T01:19:51.929859+0000","last_unstale":"2023-05-20T01:19:52.928122+0000","last_undegraded":"2023-05-20T01:19:52.928122+0000","last_fullsized":"2023-05-20T01:19:52.928122+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:31:56.703552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:54.137807+0000","last_change":"2023-05-20T01:19:51.938573+0000","last_active":"2023-05-20T01:19:54.137807+0000","last_peered":"2023-05-20T01:19:54.137807+0000","last_clean":"2023-05-20T01:19:54.137807+0000","last_became_active":"2023-05-20T01:19:51.938294+0000","last_became_peered":"2023-05-20T01:19:51.938294+0000","last_unstale":"2023-05-20T01:19:54.137807+0000","last_undegraded":"2023-05-20T01:19:54.137807+0000","last_fullsized":"2023-05-20T01:19:54.137807+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:10:13.440024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:52.928021+0000","last_change":"2023-05-20T01:19:51.932413+0000","last_active":"2023-05-20T01:19:52.928021+0000","last_peered":"2023-05-20T01:19:52.928021+0000","last_clean":"2023-05-20T01:19:52.928021+0000","last_became_active":"2023-05-20T01:19:51.932152+0000","last_became_peered":"2023-05-20T01:19:51.932152+0000","last_unstale":"2023-05-20T01:19:52.928021+0000","last_undegraded":"2023-05-20T01:19:52.928021+0000","last_fullsized":"2023-05-20T01:19:52.928021+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:41:44.890671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:52.927824+0000","last_change":"2023-05-20T01:19:51.932416+0000","last_active":"2023-05-20T01:19:52.927824+0000","last_peered":"2023-05-20T01:19:52.927824+0000","last_clean":"2023-05-20T01:19:52.927824+0000","last_became_active":"2023-05-20T01:19:51.932156+0000","last_became_peered":"2023-05-20T01:19:51.932156+0000","last_unstale":"2023-05-20T01:19:52.927824+0000","last_undegraded":"2023-05-20T01:19:52.927824+0000","last_fullsized":"2023-05-20T01:19:52.927824+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:36:38.819476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:54.138287+0000","last_change":"2023-05-20T01:19:51.943006+0000","last_active":"2023-05-20T01:19:54.138287+0000","last_peered":"2023-05-20T01:19:54.138287+0000","last_clean":"2023-05-20T01:19:54.138287+0000","last_became_active":"2023-05-20T01:19:51.942685+0000","last_became_peered":"2023-05-20T01:19:51.942685+0000","last_unstale":"2023-05-20T01:19:54.138287+0000","last_undegraded":"2023-05-20T01:19:54.138287+0000","last_fullsized":"2023-05-20T01:19:54.138287+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:18:26.867598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:54.138067+0000","last_change":"2023-05-20T01:19:51.940251+0000","last_active":"2023-05-20T01:19:54.138067+0000","last_peered":"2023-05-20T01:19:54.138067+0000","last_clean":"2023-05-20T01:19:54.138067+0000","last_became_active":"2023-05-20T01:19:51.937972+0000","last_became_peered":"2023-05-20T01:19:51.937972+0000","last_unstale":"2023-05-20T01:19:54.138067+0000","last_undegraded":"2023-05-20T01:19:54.138067+0000","last_fullsized":"2023-05-20T01:19:54.138067+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:03:53.250451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:54.137420+0000","last_change":"2023-05-20T01:19:51.943182+0000","last_active":"2023-05-20T01:19:54.137420+0000","last_peered":"2023-05-20T01:19:54.137420+0000","last_clean":"2023-05-20T01:19:54.137420+0000","last_became_active":"2023-05-20T01:19:51.937980+0000","last_became_peered":"2023-05-20T01:19:51.937980+0000","last_unstale":"2023-05-20T01:19:54.137420+0000","last_undegraded":"2023-05-20T01:19:54.137420+0000","last_fullsized":"2023-05-20T01:19:54.137420+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:32:33.635086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:52.934626+0000","last_change":"2023-05-20T01:19:51.935765+0000","last_active":"2023-05-20T01:19:52.934626+0000","last_peered":"2023-05-20T01:19:52.934626+0000","last_clean":"2023-05-20T01:19:52.934626+0000","last_became_active":"2023-05-20T01:19:51.935275+0000","last_became_peered":"2023-05-20T01:19:51.935275+0000","last_unstale":"2023-05-20T01:19:52.934626+0000","last_undegraded":"2023-05-20T01:19:52.934626+0000","last_fullsized":"2023-05-20T01:19:52.934626+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:10:48.987365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:54.137174+0000","last_change":"2023-05-20T01:19:51.938126+0000","last_active":"2023-05-20T01:19:54.137174+0000","last_peered":"2023-05-20T01:19:54.137174+0000","last_clean":"2023-05-20T01:19:54.137174+0000","last_became_active":"2023-05-20T01:19:51.937493+0000","last_became_peered":"2023-05-20T01:19:51.937493+0000","last_unstale":"2023-05-20T01:19:54.137174+0000","last_undegraded":"2023-05-20T01:19:54.137174+0000","last_fullsized":"2023-05-20T01:19:54.137174+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:09:17.812439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:54.137855+0000","last_change":"2023-05-20T01:19:51.943999+0000","last_active":"2023-05-20T01:19:54.137855+0000","last_peered":"2023-05-20T01:19:54.137855+0000","last_clean":"2023-05-20T01:19:54.137855+0000","last_became_active":"2023-05-20T01:19:51.943730+0000","last_became_peered":"2023-05-20T01:19:51.943730+0000","last_unstale":"2023-05-20T01:19:54.137855+0000","last_undegraded":"2023-05-20T01:19:54.137855+0000","last_fullsized":"2023-05-20T01:19:54.137855+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:13:16.951441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:54.137344+0000","last_change":"2023-05-20T01:19:51.943585+0000","last_active":"2023-05-20T01:19:54.137344+0000","last_peered":"2023-05-20T01:19:54.137344+0000","last_clean":"2023-05-20T01:19:54.137344+0000","last_became_active":"2023-05-20T01:19:51.943314+0000","last_became_peered":"2023-05-20T01:19:51.943314+0000","last_unstale":"2023-05-20T01:19:54.137344+0000","last_undegraded":"2023-05-20T01:19:54.137344+0000","last_fullsized":"2023-05-20T01:19:54.137344+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:12:19.351862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:52.933859+0000","last_change":"2023-05-20T01:19:51.941063+0000","last_active":"2023-05-20T01:19:52.933859+0000","last_peered":"2023-05-20T01:19:52.933859+0000","last_clean":"2023-05-20T01:19:52.933859+0000","last_became_active":"2023-05-20T01:19:51.937055+0000","last_became_peered":"2023-05-20T01:19:51.937055+0000","last_unstale":"2023-05-20T01:19:52.933859+0000","last_undegraded":"2023-05-20T01:19:52.933859+0000","last_fullsized":"2023-05-20T01:19:52.933859+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:05:12.671768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:53.436081+0000","last_change":"2023-05-20T01:19:51.936450+0000","last_active":"2023-05-20T01:19:53.436081+0000","last_peered":"2023-05-20T01:19:53.436081+0000","last_clean":"2023-05-20T01:19:53.436081+0000","last_became_active":"2023-05-20T01:19:51.936047+0000","last_became_peered":"2023-05-20T01:19:51.936047+0000","last_unstale":"2023-05-20T01:19:53.436081+0000","last_undegraded":"2023-05-20T01:19:53.436081+0000","last_fullsized":"2023-05-20T01:19:53.436081+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:41:11.309142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:52.933848+0000","last_change":"2023-05-20T01:19:51.939431+0000","last_active":"2023-05-20T01:19:52.933848+0000","last_peered":"2023-05-20T01:19:52.933848+0000","last_clean":"2023-05-20T01:19:52.933848+0000","last_became_active":"2023-05-20T01:19:51.939042+0000","last_became_peered":"2023-05-20T01:19:51.939042+0000","last_unstale":"2023-05-20T01:19:52.933848+0000","last_undegraded":"2023-05-20T01:19:52.933848+0000","last_fullsized":"2023-05-20T01:19:52.933848+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:25:34.546455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"63'4","reported_seq":17,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:54.137447+0000","last_change":"2023-05-20T01:19:51.508203+0000","last_active":"2023-05-20T01:19:54.137447+0000","last_peered":"2023-05-20T01:19:54.137447+0000","last_clean":"2023-05-20T01:19:54.137447+0000","last_became_active":"2023-05-20T01:19:51.507889+0000","last_became_peered":"2023-05-20T01:19:51.507889+0000","last_unstale":"2023-05-20T01:19:54.137447+0000","last_undegraded":"2023-05-20T01:19:54.137447+0000","last_fullsized":"2023-05-20T01:19:54.137447+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:52:37.956564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:52.934687+0000","last_change":"2023-05-20T01:19:51.930423+0000","last_active":"2023-05-20T01:19:52.934687+0000","last_peered":"2023-05-20T01:19:52.934687+0000","last_clean":"2023-05-20T01:19:52.934687+0000","last_became_active":"2023-05-20T01:19:51.930138+0000","last_became_peered":"2023-05-20T01:19:51.930138+0000","last_unstale":"2023-05-20T01:19:52.934687+0000","last_undegraded":"2023-05-20T01:19:52.934687+0000","last_fullsized":"2023-05-20T01:19:52.934687+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:20:37.462017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"63'1","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:53.436611+0000","last_change":"2023-05-20T01:19:51.510878+0000","last_active":"2023-05-20T01:19:53.436611+0000","last_peered":"2023-05-20T01:19:53.436611+0000","last_clean":"2023-05-20T01:19:53.436611+0000","last_became_active":"2023-05-20T01:19:51.510409+0000","last_became_peered":"2023-05-20T01:19:51.510409+0000","last_unstale":"2023-05-20T01:19:53.436611+0000","last_undegraded":"2023-05-20T01:19:53.436611+0000","last_fullsized":"2023-05-20T01:19:53.436611+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:31:51.936693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:53.436553+0000","last_change":"2023-05-20T01:19:51.935814+0000","last_active":"2023-05-20T01:19:53.436553+0000","last_peered":"2023-05-20T01:19:53.436553+0000","last_clean":"2023-05-20T01:19:53.436553+0000","last_became_active":"2023-05-20T01:19:51.935132+0000","last_became_peered":"2023-05-20T01:19:51.935132+0000","last_unstale":"2023-05-20T01:19:53.436553+0000","last_undegraded":"2023-05-20T01:19:53.436553+0000","last_fullsized":"2023-05-20T01:19:53.436553+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:38:10.169861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"63'2","reported_seq":15,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:53.436471+0000","last_change":"2023-05-20T01:19:51.513176+0000","last_active":"2023-05-20T01:19:53.436471+0000","last_peered":"2023-05-20T01:19:53.436471+0000","last_clean":"2023-05-20T01:19:53.436471+0000","last_became_active":"2023-05-20T01:19:51.512562+0000","last_became_peered":"2023-05-20T01:19:51.512562+0000","last_unstale":"2023-05-20T01:19:53.436471+0000","last_undegraded":"2023-05-20T01:19:53.436471+0000","last_fullsized":"2023-05-20T01:19:53.436471+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:28:12.838142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:54.138161+0000","last_change":"2023-05-20T01:19:51.938549+0000","last_active":"2023-05-20T01:19:54.138161+0000","last_peered":"2023-05-20T01:19:54.138161+0000","last_clean":"2023-05-20T01:19:54.138161+0000","last_became_active":"2023-05-20T01:19:51.938310+0000","last_became_peered":"2023-05-20T01:19:51.938310+0000","last_unstale":"2023-05-20T01:19:54.138161+0000","last_undegraded":"2023-05-20T01:19:54.138161+0000","last_fullsized":"2023-05-20T01:19:54.138161+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:33:43.077695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"63'2","reported_seq":15,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:53.436448+0000","last_change":"2023-05-20T01:19:51.511427+0000","last_active":"2023-05-20T01:19:53.436448+0000","last_peered":"2023-05-20T01:19:53.436448+0000","last_clean":"2023-05-20T01:19:53.436448+0000","last_became_active":"2023-05-20T01:19:51.511141+0000","last_became_peered":"2023-05-20T01:19:51.511141+0000","last_unstale":"2023-05-20T01:19:53.436448+0000","last_undegraded":"2023-05-20T01:19:53.436448+0000","last_fullsized":"2023-05-20T01:19:53.436448+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:59:15.391262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:53.636823+0000","last_change":"2023-05-20T01:19:51.935529+0000","last_active":"2023-05-20T01:19:53.636823+0000","last_peered":"2023-05-20T01:19:53.636823+0000","last_clean":"2023-05-20T01:19:53.636823+0000","last_became_active":"2023-05-20T01:19:51.934957+0000","last_became_peered":"2023-05-20T01:19:51.934957+0000","last_unstale":"2023-05-20T01:19:53.636823+0000","last_undegraded":"2023-05-20T01:19:53.636823+0000","last_fullsized":"2023-05-20T01:19:53.636823+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:45:34.257820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"63'1","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:52.928293+0000","last_change":"2023-05-20T01:19:51.500164+0000","last_active":"2023-05-20T01:19:52.928293+0000","last_peered":"2023-05-20T01:19:52.928293+0000","last_clean":"2023-05-20T01:19:52.928293+0000","last_became_active":"2023-05-20T01:19:51.499814+0000","last_became_peered":"2023-05-20T01:19:51.499814+0000","last_unstale":"2023-05-20T01:19:52.928293+0000","last_undegraded":"2023-05-20T01:19:52.928293+0000","last_fullsized":"2023-05-20T01:19:52.928293+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T01:26:53.281106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:54.136877+0000","last_change":"2023-05-20T01:19:51.946579+0000","last_active":"2023-05-20T01:19:54.136877+0000","last_peered":"2023-05-20T01:19:54.136877+0000","last_clean":"2023-05-20T01:19:54.136877+0000","last_became_active":"2023-05-20T01:19:51.946362+0000","last_became_peered":"2023-05-20T01:19:51.946362+0000","last_unstale":"2023-05-20T01:19:54.136877+0000","last_undegraded":"2023-05-20T01:19:54.136877+0000","last_fullsized":"2023-05-20T01:19:54.136877+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:52:47.849782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"63'1","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:52.927880+0000","last_change":"2023-05-20T01:19:51.506384+0000","last_active":"2023-05-20T01:19:52.927880+0000","last_peered":"2023-05-20T01:19:52.927880+0000","last_clean":"2023-05-20T01:19:52.927880+0000","last_became_active":"2023-05-20T01:19:51.506097+0000","last_became_peered":"2023-05-20T01:19:51.506097+0000","last_unstale":"2023-05-20T01:19:52.927880+0000","last_undegraded":"2023-05-20T01:19:52.927880+0000","last_fullsized":"2023-05-20T01:19:52.927880+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:53:23.268040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:52.933870+0000","last_change":"2023-05-20T01:19:51.930152+0000","last_active":"2023-05-20T01:19:52.933870+0000","last_peered":"2023-05-20T01:19:52.933870+0000","last_clean":"2023-05-20T01:19:52.933870+0000","last_became_active":"2023-05-20T01:19:51.929734+0000","last_became_peered":"2023-05-20T01:19:51.929734+0000","last_unstale":"2023-05-20T01:19:52.933870+0000","last_undegraded":"2023-05-20T01:19:52.933870+0000","last_fullsized":"2023-05-20T01:19:52.933870+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:46:37.721293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:52.927896+0000","last_change":"2023-05-20T01:19:51.501875+0000","last_active":"2023-05-20T01:19:52.927896+0000","last_peered":"2023-05-20T01:19:52.927896+0000","last_clean":"2023-05-20T01:19:52.927896+0000","last_became_active":"2023-05-20T01:19:51.500887+0000","last_became_peered":"2023-05-20T01:19:51.500887+0000","last_unstale":"2023-05-20T01:19:52.927896+0000","last_undegraded":"2023-05-20T01:19:52.927896+0000","last_fullsized":"2023-05-20T01:19:52.927896+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:03:28.388879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:52.928185+0000","last_change":"2023-05-20T01:19:51.930315+0000","last_active":"2023-05-20T01:19:52.928185+0000","last_peered":"2023-05-20T01:19:52.928185+0000","last_clean":"2023-05-20T01:19:52.928185+0000","last_became_active":"2023-05-20T01:19:51.930025+0000","last_became_peered":"2023-05-20T01:19:51.930025+0000","last_unstale":"2023-05-20T01:19:52.928185+0000","last_undegraded":"2023-05-20T01:19:52.928185+0000","last_fullsized":"2023-05-20T01:19:52.928185+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:23:08.275931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:52.933523+0000","last_change":"2023-05-20T01:19:51.506858+0000","last_active":"2023-05-20T01:19:52.933523+0000","last_peered":"2023-05-20T01:19:52.933523+0000","last_clean":"2023-05-20T01:19:52.933523+0000","last_became_active":"2023-05-20T01:19:51.506514+0000","last_became_peered":"2023-05-20T01:19:51.506514+0000","last_unstale":"2023-05-20T01:19:52.933523+0000","last_undegraded":"2023-05-20T01:19:52.933523+0000","last_fullsized":"2023-05-20T01:19:52.933523+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:58:06.309817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:54.137366+0000","last_change":"2023-05-20T01:19:51.938315+0000","last_active":"2023-05-20T01:19:54.137366+0000","last_peered":"2023-05-20T01:19:54.137366+0000","last_clean":"2023-05-20T01:19:54.137366+0000","last_became_active":"2023-05-20T01:19:51.937857+0000","last_became_peered":"2023-05-20T01:19:51.937857+0000","last_unstale":"2023-05-20T01:19:54.137366+0000","last_undegraded":"2023-05-20T01:19:54.137366+0000","last_fullsized":"2023-05-20T01:19:54.137366+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:25:49.473325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:54.138345+0000","last_change":"2023-05-20T01:19:51.510993+0000","last_active":"2023-05-20T01:19:54.138345+0000","last_peered":"2023-05-20T01:19:54.138345+0000","last_clean":"2023-05-20T01:19:54.138345+0000","last_became_active":"2023-05-20T01:19:51.510657+0000","last_became_peered":"2023-05-20T01:19:51.510657+0000","last_unstale":"2023-05-20T01:19:54.138345+0000","last_undegraded":"2023-05-20T01:19:54.138345+0000","last_fullsized":"2023-05-20T01:19:54.138345+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:55:44.597369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:52.927986+0000","last_change":"2023-05-20T01:19:51.931020+0000","last_active":"2023-05-20T01:19:52.927986+0000","last_peered":"2023-05-20T01:19:52.927986+0000","last_clean":"2023-05-20T01:19:52.927986+0000","last_became_active":"2023-05-20T01:19:51.930750+0000","last_became_peered":"2023-05-20T01:19:51.930750+0000","last_unstale":"2023-05-20T01:19:52.927986+0000","last_undegraded":"2023-05-20T01:19:52.927986+0000","last_fullsized":"2023-05-20T01:19:52.927986+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:13:26.310064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:52.934749+0000","last_change":"2023-05-20T01:19:51.510988+0000","last_active":"2023-05-20T01:19:52.934749+0000","last_peered":"2023-05-20T01:19:52.934749+0000","last_clean":"2023-05-20T01:19:52.934749+0000","last_became_active":"2023-05-20T01:19:51.510728+0000","last_became_peered":"2023-05-20T01:19:51.510728+0000","last_unstale":"2023-05-20T01:19:52.934749+0000","last_undegraded":"2023-05-20T01:19:52.934749+0000","last_fullsized":"2023-05-20T01:19:52.934749+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:32:37.916213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:52.934144+0000","last_change":"2023-05-20T01:19:51.934976+0000","last_active":"2023-05-20T01:19:52.934144+0000","last_peered":"2023-05-20T01:19:52.934144+0000","last_clean":"2023-05-20T01:19:52.934144+0000","last_became_active":"2023-05-20T01:19:51.934662+0000","last_became_peered":"2023-05-20T01:19:51.934662+0000","last_unstale":"2023-05-20T01:19:52.934144+0000","last_undegraded":"2023-05-20T01:19:52.934144+0000","last_fullsized":"2023-05-20T01:19:52.934144+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:18:05.645506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"63'3","reported_seq":16,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:54.138284+0000","last_change":"2023-05-20T01:19:51.511325+0000","last_active":"2023-05-20T01:19:54.138284+0000","last_peered":"2023-05-20T01:19:54.138284+0000","last_clean":"2023-05-20T01:19:54.138284+0000","last_became_active":"2023-05-20T01:19:51.510525+0000","last_became_peered":"2023-05-20T01:19:51.510525+0000","last_unstale":"2023-05-20T01:19:54.138284+0000","last_undegraded":"2023-05-20T01:19:54.138284+0000","last_fullsized":"2023-05-20T01:19:54.138284+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:12:58.176737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:52.934729+0000","last_change":"2023-05-20T01:19:51.941822+0000","last_active":"2023-05-20T01:19:52.934729+0000","last_peered":"2023-05-20T01:19:52.934729+0000","last_clean":"2023-05-20T01:19:52.934729+0000","last_became_active":"2023-05-20T01:19:51.941545+0000","last_became_peered":"2023-05-20T01:19:51.941545+0000","last_unstale":"2023-05-20T01:19:52.934729+0000","last_undegraded":"2023-05-20T01:19:52.934729+0000","last_fullsized":"2023-05-20T01:19:52.934729+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:29:07.674750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"63'6","reported_seq":19,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:54.137182+0000","last_change":"2023-05-20T01:19:51.513591+0000","last_active":"2023-05-20T01:19:54.137182+0000","last_peered":"2023-05-20T01:19:54.137182+0000","last_clean":"2023-05-20T01:19:54.137182+0000","last_became_active":"2023-05-20T01:19:51.513292+0000","last_became_peered":"2023-05-20T01:19:51.513292+0000","last_unstale":"2023-05-20T01:19:54.137182+0000","last_undegraded":"2023-05-20T01:19:54.137182+0000","last_fullsized":"2023-05-20T01:19:54.137182+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:36:24.693283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:52.933740+0000","last_change":"2023-05-20T01:19:51.942847+0000","last_active":"2023-05-20T01:19:52.933740+0000","last_peered":"2023-05-20T01:19:52.933740+0000","last_clean":"2023-05-20T01:19:52.933740+0000","last_became_active":"2023-05-20T01:19:51.942485+0000","last_became_peered":"2023-05-20T01:19:51.942485+0000","last_unstale":"2023-05-20T01:19:52.933740+0000","last_undegraded":"2023-05-20T01:19:52.933740+0000","last_fullsized":"2023-05-20T01:19:52.933740+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:42:40.123929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"63'2","reported_seq":15,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:54.138002+0000","last_change":"2023-05-20T01:19:51.506740+0000","last_active":"2023-05-20T01:19:54.138002+0000","last_peered":"2023-05-20T01:19:54.138002+0000","last_clean":"2023-05-20T01:19:54.138002+0000","last_became_active":"2023-05-20T01:19:51.506453+0000","last_became_peered":"2023-05-20T01:19:51.506453+0000","last_unstale":"2023-05-20T01:19:54.138002+0000","last_undegraded":"2023-05-20T01:19:54.138002+0000","last_fullsized":"2023-05-20T01:19:54.138002+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:18:40.151218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:54.137358+0000","last_change":"2023-05-20T01:19:51.933327+0000","last_active":"2023-05-20T01:19:54.137358+0000","last_peered":"2023-05-20T01:19:54.137358+0000","last_clean":"2023-05-20T01:19:54.137358+0000","last_became_active":"2023-05-20T01:19:51.933061+0000","last_became_peered":"2023-05-20T01:19:51.933061+0000","last_unstale":"2023-05-20T01:19:54.137358+0000","last_undegraded":"2023-05-20T01:19:54.137358+0000","last_fullsized":"2023-05-20T01:19:54.137358+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:28:44.290645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:54.137489+0000","last_change":"2023-05-20T01:19:51.508126+0000","last_active":"2023-05-20T01:19:54.137489+0000","last_peered":"2023-05-20T01:19:54.137489+0000","last_clean":"2023-05-20T01:19:54.137489+0000","last_became_active":"2023-05-20T01:19:51.507837+0000","last_became_peered":"2023-05-20T01:19:51.507837+0000","last_unstale":"2023-05-20T01:19:54.137489+0000","last_undegraded":"2023-05-20T01:19:54.137489+0000","last_fullsized":"2023-05-20T01:19:54.137489+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:43:24.091676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:52.934036+0000","last_change":"2023-05-20T01:19:51.936792+0000","last_active":"2023-05-20T01:19:52.934036+0000","last_peered":"2023-05-20T01:19:52.934036+0000","last_clean":"2023-05-20T01:19:52.934036+0000","last_became_active":"2023-05-20T01:19:51.936524+0000","last_became_peered":"2023-05-20T01:19:51.936524+0000","last_unstale":"2023-05-20T01:19:52.934036+0000","last_undegraded":"2023-05-20T01:19:52.934036+0000","last_fullsized":"2023-05-20T01:19:52.934036+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:04:05.309842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:54.138280+0000","last_change":"2023-05-20T01:19:51.507794+0000","last_active":"2023-05-20T01:19:54.138280+0000","last_peered":"2023-05-20T01:19:54.138280+0000","last_clean":"2023-05-20T01:19:54.138280+0000","last_became_active":"2023-05-20T01:19:51.507547+0000","last_became_peered":"2023-05-20T01:19:51.507547+0000","last_unstale":"2023-05-20T01:19:54.138280+0000","last_undegraded":"2023-05-20T01:19:54.138280+0000","last_fullsized":"2023-05-20T01:19:54.138280+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:56:49.137434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:54.137013+0000","last_change":"2023-05-20T01:19:51.949014+0000","last_active":"2023-05-20T01:19:54.137013+0000","last_peered":"2023-05-20T01:19:54.137013+0000","last_clean":"2023-05-20T01:19:54.137013+0000","last_became_active":"2023-05-20T01:19:51.948813+0000","last_became_peered":"2023-05-20T01:19:51.948813+0000","last_unstale":"2023-05-20T01:19:54.137013+0000","last_undegraded":"2023-05-20T01:19:54.137013+0000","last_fullsized":"2023-05-20T01:19:54.137013+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:48:25.587471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"63'8","reported_seq":21,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:52.933892+0000","last_change":"2023-05-20T01:19:51.509281+0000","last_active":"2023-05-20T01:19:52.933892+0000","last_peered":"2023-05-20T01:19:52.933892+0000","last_clean":"2023-05-20T01:19:52.933892+0000","last_became_active":"2023-05-20T01:19:51.508971+0000","last_became_peered":"2023-05-20T01:19:51.508971+0000","last_unstale":"2023-05-20T01:19:52.933892+0000","last_undegraded":"2023-05-20T01:19:52.933892+0000","last_fullsized":"2023-05-20T01:19:52.933892+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:27:18.277647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:53.436544+0000","last_change":"2023-05-20T01:19:51.947213+0000","last_active":"2023-05-20T01:19:53.436544+0000","last_peered":"2023-05-20T01:19:53.436544+0000","last_clean":"2023-05-20T01:19:53.436544+0000","last_became_active":"2023-05-20T01:19:51.946895+0000","last_became_peered":"2023-05-20T01:19:51.946895+0000","last_unstale":"2023-05-20T01:19:53.436544+0000","last_undegraded":"2023-05-20T01:19:53.436544+0000","last_fullsized":"2023-05-20T01:19:53.436544+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:26:01.454283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"63'2","reported_seq":15,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:54.138270+0000","last_change":"2023-05-20T01:19:51.505066+0000","last_active":"2023-05-20T01:19:54.138270+0000","last_peered":"2023-05-20T01:19:54.138270+0000","last_clean":"2023-05-20T01:19:54.138270+0000","last_became_active":"2023-05-20T01:19:51.504740+0000","last_became_peered":"2023-05-20T01:19:51.504740+0000","last_unstale":"2023-05-20T01:19:54.138270+0000","last_undegraded":"2023-05-20T01:19:54.138270+0000","last_fullsized":"2023-05-20T01:19:54.138270+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:33:37.663977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:53.636992+0000","last_change":"2023-05-20T01:19:51.936622+0000","last_active":"2023-05-20T01:19:53.636992+0000","last_peered":"2023-05-20T01:19:53.636992+0000","last_clean":"2023-05-20T01:19:53.636992+0000","last_became_active":"2023-05-20T01:19:51.935663+0000","last_became_peered":"2023-05-20T01:19:51.935663+0000","last_unstale":"2023-05-20T01:19:53.636992+0000","last_undegraded":"2023-05-20T01:19:53.636992+0000","last_fullsized":"2023-05-20T01:19:53.636992+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:40:41.636194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"64'1","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:55.403471+0000","last_change":"2023-05-20T01:19:51.505943+0000","last_active":"2023-05-20T01:19:55.403471+0000","last_peered":"2023-05-20T01:19:55.403471+0000","last_clean":"2023-05-20T01:19:55.403471+0000","last_became_active":"2023-05-20T01:19:51.505483+0000","last_became_peered":"2023-05-20T01:19:51.505483+0000","last_unstale":"2023-05-20T01:19:55.403471+0000","last_undegraded":"2023-05-20T01:19:55.403471+0000","last_fullsized":"2023-05-20T01:19:55.403471+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:16:35.292935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:54.137693+0000","last_change":"2023-05-20T01:19:51.944056+0000","last_active":"2023-05-20T01:19:54.137693+0000","last_peered":"2023-05-20T01:19:54.137693+0000","last_clean":"2023-05-20T01:19:54.137693+0000","last_became_active":"2023-05-20T01:19:51.943768+0000","last_became_peered":"2023-05-20T01:19:51.943768+0000","last_unstale":"2023-05-20T01:19:54.137693+0000","last_undegraded":"2023-05-20T01:19:54.137693+0000","last_fullsized":"2023-05-20T01:19:54.137693+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:57:59.518282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:53.436552+0000","last_change":"2023-05-20T01:19:51.512499+0000","last_active":"2023-05-20T01:19:53.436552+0000","last_peered":"2023-05-20T01:19:53.436552+0000","last_clean":"2023-05-20T01:19:53.436552+0000","last_became_active":"2023-05-20T01:19:51.512222+0000","last_became_peered":"2023-05-20T01:19:51.512222+0000","last_unstale":"2023-05-20T01:19:53.436552+0000","last_undegraded":"2023-05-20T01:19:53.436552+0000","last_fullsized":"2023-05-20T01:19:53.436552+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:59:57.390952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:52.934383+0000","last_change":"2023-05-20T01:19:51.936248+0000","last_active":"2023-05-20T01:19:52.934383+0000","last_peered":"2023-05-20T01:19:52.934383+0000","last_clean":"2023-05-20T01:19:52.934383+0000","last_became_active":"2023-05-20T01:19:51.935918+0000","last_became_peered":"2023-05-20T01:19:51.935918+0000","last_unstale":"2023-05-20T01:19:52.934383+0000","last_undegraded":"2023-05-20T01:19:52.934383+0000","last_fullsized":"2023-05-20T01:19:52.934383+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:06:15.619927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:54.137335+0000","last_change":"2023-05-20T01:19:51.513458+0000","last_active":"2023-05-20T01:19:54.137335+0000","last_peered":"2023-05-20T01:19:54.137335+0000","last_clean":"2023-05-20T01:19:54.137335+0000","last_became_active":"2023-05-20T01:19:51.513156+0000","last_became_peered":"2023-05-20T01:19:51.513156+0000","last_unstale":"2023-05-20T01:19:54.137335+0000","last_undegraded":"2023-05-20T01:19:54.137335+0000","last_fullsized":"2023-05-20T01:19:54.137335+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:49:10.224839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:52.933470+0000","last_change":"2023-05-20T01:19:51.934151+0000","last_active":"2023-05-20T01:19:52.933470+0000","last_peered":"2023-05-20T01:19:52.933470+0000","last_clean":"2023-05-20T01:19:52.933470+0000","last_became_active":"2023-05-20T01:19:51.933844+0000","last_became_peered":"2023-05-20T01:19:51.933844+0000","last_unstale":"2023-05-20T01:19:52.933470+0000","last_undegraded":"2023-05-20T01:19:52.933470+0000","last_fullsized":"2023-05-20T01:19:52.933470+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:17:17.969734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:54.138081+0000","last_change":"2023-05-20T01:19:51.510976+0000","last_active":"2023-05-20T01:19:54.138081+0000","last_peered":"2023-05-20T01:19:54.138081+0000","last_clean":"2023-05-20T01:19:54.138081+0000","last_became_active":"2023-05-20T01:19:51.510244+0000","last_became_peered":"2023-05-20T01:19:51.510244+0000","last_unstale":"2023-05-20T01:19:54.138081+0000","last_undegraded":"2023-05-20T01:19:54.138081+0000","last_fullsized":"2023-05-20T01:19:54.138081+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:49:31.184252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:52.934032+0000","last_change":"2023-05-20T01:19:51.932566+0000","last_active":"2023-05-20T01:19:52.934032+0000","last_peered":"2023-05-20T01:19:52.934032+0000","last_clean":"2023-05-20T01:19:52.934032+0000","last_became_active":"2023-05-20T01:19:51.932268+0000","last_became_peered":"2023-05-20T01:19:51.932268+0000","last_unstale":"2023-05-20T01:19:52.934032+0000","last_undegraded":"2023-05-20T01:19:52.934032+0000","last_fullsized":"2023-05-20T01:19:52.934032+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:45:31.086812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:52.928012+0000","last_change":"2023-05-20T01:19:51.508441+0000","last_active":"2023-05-20T01:19:52.928012+0000","last_peered":"2023-05-20T01:19:52.928012+0000","last_clean":"2023-05-20T01:19:52.928012+0000","last_became_active":"2023-05-20T01:19:51.508110+0000","last_became_peered":"2023-05-20T01:19:51.508110+0000","last_unstale":"2023-05-20T01:19:52.928012+0000","last_undegraded":"2023-05-20T01:19:52.928012+0000","last_fullsized":"2023-05-20T01:19:52.928012+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:45:56.940973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:54.136976+0000","last_change":"2023-05-20T01:19:51.946612+0000","last_active":"2023-05-20T01:19:54.136976+0000","last_peered":"2023-05-20T01:19:54.136976+0000","last_clean":"2023-05-20T01:19:54.136976+0000","last_became_active":"2023-05-20T01:19:51.946370+0000","last_became_peered":"2023-05-20T01:19:51.946370+0000","last_unstale":"2023-05-20T01:19:54.136976+0000","last_undegraded":"2023-05-20T01:19:54.136976+0000","last_fullsized":"2023-05-20T01:19:54.136976+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:22:12.352183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:52.928136+0000","last_change":"2023-05-20T01:19:51.501240+0000","last_active":"2023-05-20T01:19:52.928136+0000","last_peered":"2023-05-20T01:19:52.928136+0000","last_clean":"2023-05-20T01:19:52.928136+0000","last_became_active":"2023-05-20T01:19:51.500879+0000","last_became_peered":"2023-05-20T01:19:51.500879+0000","last_unstale":"2023-05-20T01:19:52.928136+0000","last_undegraded":"2023-05-20T01:19:52.928136+0000","last_fullsized":"2023-05-20T01:19:52.928136+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:57:28.696361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:53.636828+0000","last_change":"2023-05-20T01:19:51.938554+0000","last_active":"2023-05-20T01:19:53.636828+0000","last_peered":"2023-05-20T01:19:53.636828+0000","last_clean":"2023-05-20T01:19:53.636828+0000","last_became_active":"2023-05-20T01:19:51.938278+0000","last_became_peered":"2023-05-20T01:19:51.938278+0000","last_unstale":"2023-05-20T01:19:53.636828+0000","last_undegraded":"2023-05-20T01:19:53.636828+0000","last_fullsized":"2023-05-20T01:19:53.636828+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:57:57.525533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:54.136973+0000","last_change":"2023-05-20T01:19:51.503966+0000","last_active":"2023-05-20T01:19:54.136973+0000","last_peered":"2023-05-20T01:19:54.136973+0000","last_clean":"2023-05-20T01:19:54.136973+0000","last_became_active":"2023-05-20T01:19:51.503685+0000","last_became_peered":"2023-05-20T01:19:51.503685+0000","last_unstale":"2023-05-20T01:19:54.136973+0000","last_undegraded":"2023-05-20T01:19:54.136973+0000","last_fullsized":"2023-05-20T01:19:54.136973+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:46:26.729837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:52.934704+0000","last_change":"2023-05-20T01:19:51.931614+0000","last_active":"2023-05-20T01:19:52.934704+0000","last_peered":"2023-05-20T01:19:52.934704+0000","last_clean":"2023-05-20T01:19:52.934704+0000","last_became_active":"2023-05-20T01:19:51.931331+0000","last_became_peered":"2023-05-20T01:19:51.931331+0000","last_unstale":"2023-05-20T01:19:52.934704+0000","last_undegraded":"2023-05-20T01:19:52.934704+0000","last_fullsized":"2023-05-20T01:19:52.934704+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:05:02.922467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"63'1","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:54.138289+0000","last_change":"2023-05-20T01:19:51.511507+0000","last_active":"2023-05-20T01:19:54.138289+0000","last_peered":"2023-05-20T01:19:54.138289+0000","last_clean":"2023-05-20T01:19:54.138289+0000","last_became_active":"2023-05-20T01:19:51.511246+0000","last_became_peered":"2023-05-20T01:19:51.511246+0000","last_unstale":"2023-05-20T01:19:54.138289+0000","last_undegraded":"2023-05-20T01:19:54.138289+0000","last_fullsized":"2023-05-20T01:19:54.138289+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:44:58.536398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":102,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:52.934729+0000","last_change":"2023-05-20T01:16:23.289068+0000","last_active":"2023-05-20T01:19:52.934729+0000","last_peered":"2023-05-20T01:19:52.934729+0000","last_clean":"2023-05-20T01:19:52.934729+0000","last_became_active":"2023-05-20T01:16:23.288781+0000","last_became_peered":"2023-05-20T01:16:23.288781+0000","last_unstale":"2023-05-20T01:19:52.934729+0000","last_undegraded":"2023-05-20T01:19:52.934729+0000","last_fullsized":"2023-05-20T01:19:52.934729+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:05:36.497526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:52.934379+0000","last_change":"2023-05-20T01:19:51.935964+0000","last_active":"2023-05-20T01:19:52.934379+0000","last_peered":"2023-05-20T01:19:52.934379+0000","last_clean":"2023-05-20T01:19:52.934379+0000","last_became_active":"2023-05-20T01:19:51.935500+0000","last_became_peered":"2023-05-20T01:19:51.935500+0000","last_unstale":"2023-05-20T01:19:52.934379+0000","last_undegraded":"2023-05-20T01:19:52.934379+0000","last_fullsized":"2023-05-20T01:19:52.934379+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:13:08.801742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:52.928248+0000","last_change":"2023-05-20T01:19:51.512659+0000","last_active":"2023-05-20T01:19:52.928248+0000","last_peered":"2023-05-20T01:19:52.928248+0000","last_clean":"2023-05-20T01:19:52.928248+0000","last_became_active":"2023-05-20T01:19:51.512356+0000","last_became_peered":"2023-05-20T01:19:51.512356+0000","last_unstale":"2023-05-20T01:19:52.928248+0000","last_undegraded":"2023-05-20T01:19:52.928248+0000","last_fullsized":"2023-05-20T01:19:52.928248+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:44:56.148628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":106,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:52.928327+0000","last_change":"2023-05-20T01:16:22.829324+0000","last_active":"2023-05-20T01:19:52.928327+0000","last_peered":"2023-05-20T01:19:52.928327+0000","last_clean":"2023-05-20T01:19:52.928327+0000","last_became_active":"2023-05-20T01:16:22.828964+0000","last_became_peered":"2023-05-20T01:16:22.828964+0000","last_unstale":"2023-05-20T01:19:52.928327+0000","last_undegraded":"2023-05-20T01:19:52.928327+0000","last_fullsized":"2023-05-20T01:19:52.928327+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:04:16.729108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:53.436271+0000","last_change":"2023-05-20T01:19:51.936292+0000","last_active":"2023-05-20T01:19:53.436271+0000","last_peered":"2023-05-20T01:19:53.436271+0000","last_clean":"2023-05-20T01:19:53.436271+0000","last_became_active":"2023-05-20T01:19:51.935807+0000","last_became_peered":"2023-05-20T01:19:51.935807+0000","last_unstale":"2023-05-20T01:19:53.436271+0000","last_undegraded":"2023-05-20T01:19:53.436271+0000","last_fullsized":"2023-05-20T01:19:53.436271+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T01:22:42.445360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"63'2","reported_seq":15,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:53.436374+0000","last_change":"2023-05-20T01:19:51.508354+0000","last_active":"2023-05-20T01:19:53.436374+0000","last_peered":"2023-05-20T01:19:53.436374+0000","last_clean":"2023-05-20T01:19:53.436374+0000","last_became_active":"2023-05-20T01:19:51.508042+0000","last_became_peered":"2023-05-20T01:19:51.508042+0000","last_unstale":"2023-05-20T01:19:53.436374+0000","last_undegraded":"2023-05-20T01:19:53.436374+0000","last_fullsized":"2023-05-20T01:19:53.436374+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:06:58.676646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":98,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:54.137097+0000","last_change":"2023-05-20T01:16:22.834754+0000","last_active":"2023-05-20T01:19:54.137097+0000","last_peered":"2023-05-20T01:19:54.137097+0000","last_clean":"2023-05-20T01:19:54.137097+0000","last_became_active":"2023-05-20T01:16:22.834437+0000","last_became_peered":"2023-05-20T01:16:22.834437+0000","last_unstale":"2023-05-20T01:19:54.137097+0000","last_undegraded":"2023-05-20T01:19:54.137097+0000","last_fullsized":"2023-05-20T01:19:54.137097+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:12:53.831710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":238,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:54.137194+0000","last_change":"2023-05-20T01:16:22.834920+0000","last_active":"2023-05-20T01:19:54.137194+0000","last_peered":"2023-05-20T01:19:54.137194+0000","last_clean":"2023-05-20T01:19:54.137194+0000","last_became_active":"2023-05-20T01:16:22.833924+0000","last_became_peered":"2023-05-20T01:16:22.833924+0000","last_unstale":"2023-05-20T01:19:54.137194+0000","last_undegraded":"2023-05-20T01:19:54.137194+0000","last_fullsized":"2023-05-20T01:19:54.137194+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:17:10.132648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:54.137530+0000","last_change":"2023-05-20T01:19:51.937376+0000","last_active":"2023-05-20T01:19:54.137530+0000","last_peered":"2023-05-20T01:19:54.137530+0000","last_clean":"2023-05-20T01:19:54.137530+0000","last_became_active":"2023-05-20T01:19:51.937076+0000","last_became_peered":"2023-05-20T01:19:51.937076+0000","last_unstale":"2023-05-20T01:19:54.137530+0000","last_undegraded":"2023-05-20T01:19:54.137530+0000","last_fullsized":"2023-05-20T01:19:54.137530+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:22:01.609909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:54.137631+0000","last_change":"2023-05-20T01:19:51.508215+0000","last_active":"2023-05-20T01:19:54.137631+0000","last_peered":"2023-05-20T01:19:54.137631+0000","last_clean":"2023-05-20T01:19:54.137631+0000","last_became_active":"2023-05-20T01:19:51.507945+0000","last_became_peered":"2023-05-20T01:19:51.507945+0000","last_unstale":"2023-05-20T01:19:54.137631+0000","last_undegraded":"2023-05-20T01:19:54.137631+0000","last_fullsized":"2023-05-20T01:19:54.137631+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:18:56.763339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":88,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:54.137901+0000","last_change":"2023-05-20T01:16:23.290259+0000","last_active":"2023-05-20T01:19:54.137901+0000","last_peered":"2023-05-20T01:19:54.137901+0000","last_clean":"2023-05-20T01:19:54.137901+0000","last_became_active":"2023-05-20T01:16:23.289911+0000","last_became_peered":"2023-05-20T01:16:23.289911+0000","last_unstale":"2023-05-20T01:19:54.137901+0000","last_undegraded":"2023-05-20T01:19:54.137901+0000","last_fullsized":"2023-05-20T01:19:54.137901+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:02:09.647064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:52.934417+0000","last_change":"2023-05-20T01:19:51.928915+0000","last_active":"2023-05-20T01:19:52.934417+0000","last_peered":"2023-05-20T01:19:52.934417+0000","last_clean":"2023-05-20T01:19:52.934417+0000","last_became_active":"2023-05-20T01:19:51.928616+0000","last_became_peered":"2023-05-20T01:19:51.928616+0000","last_unstale":"2023-05-20T01:19:52.934417+0000","last_undegraded":"2023-05-20T01:19:52.934417+0000","last_fullsized":"2023-05-20T01:19:52.934417+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:16:53.546866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:52.927912+0000","last_change":"2023-05-20T01:19:51.506498+0000","last_active":"2023-05-20T01:19:52.927912+0000","last_peered":"2023-05-20T01:19:52.927912+0000","last_clean":"2023-05-20T01:19:52.927912+0000","last_became_active":"2023-05-20T01:19:51.506214+0000","last_became_peered":"2023-05-20T01:19:51.506214+0000","last_unstale":"2023-05-20T01:19:52.927912+0000","last_undegraded":"2023-05-20T01:19:52.927912+0000","last_fullsized":"2023-05-20T01:19:52.927912+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:01:30.964917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":88,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:54.137322+0000","last_change":"2023-05-20T01:16:23.290207+0000","last_active":"2023-05-20T01:19:54.137322+0000","last_peered":"2023-05-20T01:19:54.137322+0000","last_clean":"2023-05-20T01:19:54.137322+0000","last_became_active":"2023-05-20T01:16:23.289928+0000","last_became_peered":"2023-05-20T01:16:23.289928+0000","last_unstale":"2023-05-20T01:19:54.137322+0000","last_undegraded":"2023-05-20T01:19:54.137322+0000","last_fullsized":"2023-05-20T01:19:54.137322+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:36:03.264266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:52.934166+0000","last_change":"2023-05-20T01:19:51.929982+0000","last_active":"2023-05-20T01:19:52.934166+0000","last_peered":"2023-05-20T01:19:52.934166+0000","last_clean":"2023-05-20T01:19:52.934166+0000","last_became_active":"2023-05-20T01:19:51.929407+0000","last_became_peered":"2023-05-20T01:19:51.929407+0000","last_unstale":"2023-05-20T01:19:52.934166+0000","last_undegraded":"2023-05-20T01:19:52.934166+0000","last_fullsized":"2023-05-20T01:19:52.934166+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:36:52.793147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:53.436106+0000","last_change":"2023-05-20T01:19:51.510373+0000","last_active":"2023-05-20T01:19:53.436106+0000","last_peered":"2023-05-20T01:19:53.436106+0000","last_clean":"2023-05-20T01:19:53.436106+0000","last_became_active":"2023-05-20T01:19:51.510077+0000","last_became_peered":"2023-05-20T01:19:51.510077+0000","last_unstale":"2023-05-20T01:19:53.436106+0000","last_undegraded":"2023-05-20T01:19:53.436106+0000","last_fullsized":"2023-05-20T01:19:53.436106+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:32:36.503979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:54.137941+0000","last_change":"2023-05-20T01:16:23.290538+0000","last_active":"2023-05-20T01:19:54.137941+0000","last_peered":"2023-05-20T01:19:54.137941+0000","last_clean":"2023-05-20T01:19:54.137941+0000","last_became_active":"2023-05-20T01:16:23.290230+0000","last_became_peered":"2023-05-20T01:16:23.290230+0000","last_unstale":"2023-05-20T01:19:54.137941+0000","last_undegraded":"2023-05-20T01:19:54.137941+0000","last_fullsized":"2023-05-20T01:19:54.137941+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:16:04.109942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:52.934304+0000","last_change":"2023-05-20T01:19:51.934758+0000","last_active":"2023-05-20T01:19:52.934304+0000","last_peered":"2023-05-20T01:19:52.934304+0000","last_clean":"2023-05-20T01:19:52.934304+0000","last_became_active":"2023-05-20T01:19:51.933066+0000","last_became_peered":"2023-05-20T01:19:51.933066+0000","last_unstale":"2023-05-20T01:19:52.934304+0000","last_undegraded":"2023-05-20T01:19:52.934304+0000","last_fullsized":"2023-05-20T01:19:52.934304+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:09:17.142028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"63'2","reported_seq":15,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:54.138095+0000","last_change":"2023-05-20T01:19:51.513212+0000","last_active":"2023-05-20T01:19:54.138095+0000","last_peered":"2023-05-20T01:19:54.138095+0000","last_clean":"2023-05-20T01:19:54.138095+0000","last_became_active":"2023-05-20T01:19:51.512908+0000","last_became_peered":"2023-05-20T01:19:51.512908+0000","last_unstale":"2023-05-20T01:19:54.138095+0000","last_undegraded":"2023-05-20T01:19:54.138095+0000","last_fullsized":"2023-05-20T01:19:54.138095+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:19:59.308408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":90,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:52.934229+0000","last_change":"2023-05-20T01:16:23.290341+0000","last_active":"2023-05-20T01:19:52.934229+0000","last_peered":"2023-05-20T01:19:52.934229+0000","last_clean":"2023-05-20T01:19:52.934229+0000","last_became_active":"2023-05-20T01:16:23.290053+0000","last_became_peered":"2023-05-20T01:16:23.290053+0000","last_unstale":"2023-05-20T01:19:52.934229+0000","last_undegraded":"2023-05-20T01:19:52.934229+0000","last_fullsized":"2023-05-20T01:19:52.934229+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:47:27.554535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:54.137889+0000","last_change":"2023-05-20T01:19:51.944004+0000","last_active":"2023-05-20T01:19:54.137889+0000","last_peered":"2023-05-20T01:19:54.137889+0000","last_clean":"2023-05-20T01:19:54.137889+0000","last_became_active":"2023-05-20T01:19:51.943782+0000","last_became_peered":"2023-05-20T01:19:51.943782+0000","last_unstale":"2023-05-20T01:19:54.137889+0000","last_undegraded":"2023-05-20T01:19:54.137889+0000","last_fullsized":"2023-05-20T01:19:54.137889+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:51.482477+0000","last_clean_scrub_stamp":"2023-05-20T01:19:51.482477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:44:50.726245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"63'1","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:54.138222+0000","last_change":"2023-05-20T01:19:51.512080+0000","last_active":"2023-05-20T01:19:54.138222+0000","last_peered":"2023-05-20T01:19:54.138222+0000","last_clean":"2023-05-20T01:19:54.138222+0000","last_became_active":"2023-05-20T01:19:51.511785+0000","last_became_peered":"2023-05-20T01:19:51.511785+0000","last_unstale":"2023-05-20T01:19:54.138222+0000","last_undegraded":"2023-05-20T01:19:54.138222+0000","last_fullsized":"2023-05-20T01:19:54.138222+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:19:50.473280+0000","last_clean_scrub_stamp":"2023-05-20T01:19:50.473280+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:53:34.483502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":90,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-05-20T01:19:52.934482+0000","last_change":"2023-05-20T01:16:22.834330+0000","last_active":"2023-05-20T01:19:52.934482+0000","last_peered":"2023-05-20T01:19:52.934482+0000","last_clean":"2023-05-20T01:19:52.934482+0000","last_became_active":"2023-05-20T01:16:22.833946+0000","last_became_peered":"2023-05-20T01:16:22.833946+0000","last_unstale":"2023-05-20T01:19:52.934482+0000","last_undegraded":"2023-05-20T01:19:52.934482+0000","last_fullsized":"2023-05-20T01:19:52.934482+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T01:52:19.713897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574896,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10636,"kb_used_data":1348,"kb_used_omap":0,"kb_used_meta":9280,"kb_avail":94361204,"statfs":{"total":96636764160,"available":96625872896,"internally_reserved":0,"allocated":1380352,"data_stored":966315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9502720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.52800000000000002,"15min":0.53400000000000003},"min":{"1min":0.29699999999999999,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.71799999999999997,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.56599999999999995,"15min":0.57099999999999995},"min":{"1min":0.38700000000000001,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.70099999999999996,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.752}]},{"osd":1,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.79500000000000004,"15min":0.90700000000000003},"min":{"1min":0.435,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.78600000000000003,"5min":8.8879999999999999,"15min":8.8879999999999999},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.83399999999999996,"15min":0.95099999999999996},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.73299999999999998,"5min":8.875,"15min":8.875},"last":0.70699999999999996}]},{"osd":2,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.77200000000000002,"15min":0.88},"min":{"1min":0.371,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.73299999999999998,"5min":8.9120000000000008,"15min":8.9120000000000008},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.77600000000000002,"15min":0.88400000000000001},"min":{"1min":0.35699999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.751,"5min":8.8520000000000003,"15min":8.8520000000000003},"last":0.72399999999999998}]},{"osd":3,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.84299999999999997,"15min":0.98299999999999998},"min":{"1min":0.35299999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.74299999999999999,"5min":10.884,"15min":10.884},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.89900000000000002,"15min":1.0349999999999999},"min":{"1min":0.433,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.72199999999999998,"5min":10.952,"15min":10.952},"last":0.63500000000000001}]},{"osd":4,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.71499999999999997,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.53200000000000003,"15min":0.52900000000000003},"min":{"1min":0.38800000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.67300000000000004,"5min":0.875,"15min":0.875},"last":0.51700000000000002}]},{"osd":5,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54300000000000004,"15min":0.52800000000000002},"min":{"1min":0.39600000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.69899999999999995,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.58199999999999996,"15min":0.57799999999999996},"min":{"1min":0.42699999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.745,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.48899999999999999}]},{"osd":6,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59299999999999997},"min":{"1min":0.47699999999999998,"5min":0.378,"15min":0.378},"max":{"1min":0.77000000000000002,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.59299999999999997,"15min":0.58499999999999996},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.83299999999999996,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.58099999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574895,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9404,"kb_used_data":756,"kb_used_omap":0,"kb_used_meta":8640,"kb_avail":94362436,"statfs":{"total":96636764160,"available":96627134464,"internally_reserved":0,"allocated":774144,"data_stored":375205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8847360},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.495,"15min":0.48699999999999999},"min":{"1min":0.34000000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.71699999999999997,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.54300000000000004,"15min":0.54100000000000004},"min":{"1min":0.36699999999999999,"5min":0.307,"15min":0.307},"max":{"1min":0.71199999999999997,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.53400000000000003}]},{"osd":1,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53500000000000003,"15min":0.53400000000000003},"min":{"1min":0.35899999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.68200000000000005,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53300000000000003,"15min":0.53200000000000003},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.78600000000000003,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.68500000000000005}]},{"osd":2,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.51700000000000002,"15min":0.51500000000000001},"min":{"1min":0.35499999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.68700000000000006,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.56299999999999994,"15min":0.56699999999999995},"min":{"1min":0.44500000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.51700000000000002}]},{"osd":3,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.75900000000000001,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.54900000000000004,"15min":0.55000000000000004},"min":{"1min":0.39400000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.747,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.65000000000000002}]},{"osd":4,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.54000000000000004,"15min":0.56000000000000005},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.71799999999999997,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.56499999999999995,"15min":0.58399999999999996},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.72199999999999998,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.72899999999999998}]},{"osd":5,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.54000000000000004,"15min":0.54800000000000004},"min":{"1min":0.373,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.70699999999999996,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.60599999999999998,"15min":0.621},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.80200000000000005,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.71599999999999997}]},{"osd":7,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.59799999999999998,"15min":0.61199999999999999},"min":{"1min":0.38600000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.77800000000000002,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.753},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.61499999999999999,"15min":0.628},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.749,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.73999999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574895,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10584,"kb_used_data":1360,"kb_used_omap":0,"kb_used_meta":9216,"kb_avail":94361256,"statfs":{"total":96636764160,"available":96625926144,"internally_reserved":0,"allocated":1392640,"data_stored":967867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9437184},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:19:24 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.32700000000000001,"15min":0.32900000000000001},"min":{"1min":0.193,"5min":0.193,"15min":0.193},"max":{"1min":1.194,"5min":1.194,"15min":1.194},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.35699999999999998,"5min":0.32000000000000001,"15min":0.316},"min":{"1min":0.20599999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.42599999999999999}]},{"osd":2,"last update":"Sat May 20 01:19:24 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.33000000000000002,"15min":0.32400000000000001},"min":{"1min":0.253,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.42199999999999999,"15min":0.42599999999999999},"min":{"1min":0.29599999999999999,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.3580000000000001,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":0.53600000000000003}]},{"osd":3,"last update":"Sat May 20 01:19:24 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.38500000000000001,"15min":0.376},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":1.619,"5min":1.619,"15min":1.619},"last":0.497},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.39800000000000002,"15min":0.39400000000000002},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":1.8400000000000001,"5min":1.8400000000000001,"15min":1.8400000000000001},"last":0.52300000000000002}]},{"osd":4,"last update":"Sat May 20 01:19:24 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.59599999999999997,"15min":0.59099999999999997},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.59099999999999997,"15min":0.58899999999999997},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.60399999999999998}]},{"osd":5,"last update":"Sat May 20 01:19:24 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.59999999999999998,"15min":0.58899999999999997},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.61599999999999999,"15min":0.61199999999999999},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.409,"5min":1.409,"15min":1.409},"last":0.57799999999999996}]},{"osd":6,"last update":"Sat May 20 01:19:24 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.61699999999999999,"15min":0.60599999999999998},"min":{"1min":0.47099999999999997,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.575,"5min":1.575,"15min":1.575},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.60199999999999998,"15min":0.59699999999999998},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.56599999999999995}]},{"osd":7,"last update":"Sat May 20 01:19:24 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.61299999999999999,"15min":0.60899999999999999},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.496,"5min":1.496,"15min":1.496},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.65100000000000002,"15min":0.64400000000000002},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.5169999999999999,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.67400000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574895,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9588,"kb_used_data":748,"kb_used_omap":0,"kb_used_meta":8832,"kb_avail":94362252,"statfs":{"total":96636764160,"available":96626946048,"internally_reserved":0,"allocated":765952,"data_stored":374210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9043968},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.31,"5min":0.32500000000000001,"15min":0.33300000000000002},"min":{"1min":0.188,"5min":0.188,"15min":0.188},"max":{"1min":0.47599999999999998,"5min":0.56599999999999995,"15min":0.56599999999999995},"last":0.496},{"interface":"front","average":{"1min":0.33400000000000002,"5min":0.35399999999999998,"15min":0.36799999999999999},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.53400000000000003,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.38400000000000001}]},{"osd":2,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.34300000000000003,"15min":0.34100000000000003},"min":{"1min":0.19400000000000001,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.39600000000000002,"15min":0.39900000000000002},"min":{"1min":0.29399999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.67000000000000004,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.47799999999999998}]},{"osd":3,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.36799999999999999,"15min":0.376},"min":{"1min":0.26100000000000001,"5min":0.245,"15min":0.245},"max":{"1min":0.51400000000000001,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.44},{"interface":"front","average":{"1min":0.375,"5min":0.372,"15min":0.372},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.50600000000000001,"5min":0.60099999999999998,"15min":0.60099999999999998},"last":0.52500000000000002}]},{"osd":4,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.54200000000000004,"15min":0.54000000000000004},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.76000000000000001,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.56599999999999995,"15min":0.56399999999999995},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.72299999999999998,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.58199999999999996}]},{"osd":5,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.57299999999999995,"15min":0.57699999999999996},"min":{"1min":0.40300000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.68799999999999994,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56799999999999995,"15min":0.56499999999999995},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.78900000000000003,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.60799999999999998}]},{"osd":6,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.57199999999999995,"15min":0.57399999999999995},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.69799999999999995,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.61799999999999999,"15min":0.623},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.73099999999999998}]},{"osd":7,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.59499999999999997,"15min":0.59099999999999997},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.78300000000000003,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.61799999999999999,"15min":0.624},"min":{"1min":0.43099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.76200000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574895,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9592,"kb_used_data":752,"kb_used_omap":0,"kb_used_meta":8832,"kb_avail":94362248,"statfs":{"total":96636764160,"available":96626941952,"internally_reserved":0,"allocated":770048,"data_stored":375171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9043968},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.32000000000000001,"5min":0.33900000000000002,"15min":0.35299999999999998},"min":{"1min":0.20100000000000001,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.58999999999999997,"5min":1.038,"15min":1.038},"last":0.23999999999999999},{"interface":"front","average":{"1min":0.35699999999999998,"5min":0.35799999999999998,"15min":0.36699999999999999},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.52900000000000003,"5min":1.079,"15min":1.079},"last":0.38100000000000001}]},{"osd":1,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.39600000000000002,"15min":0.42099999999999999},"min":{"1min":0.26000000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.50900000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.436,"15min":0.45500000000000002},"min":{"1min":0.28399999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.64500000000000002,"5min":1.5189999999999999,"15min":1.5189999999999999},"last":0.32800000000000001}]},{"osd":3,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.39700000000000002,"15min":0.41499999999999998},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.55000000000000004,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.26800000000000002},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.45000000000000001,"15min":0.46999999999999997},"min":{"1min":0.32600000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.63100000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.34399999999999997}]},{"osd":4,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.58399999999999996,"15min":0.58999999999999997},"min":{"1min":0.42699999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.86499999999999999,"5min":1.319,"15min":1.319},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.625,"15min":0.626},"min":{"1min":0.48699999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.83899999999999997,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.61199999999999999}]},{"osd":5,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.64000000000000001,"15min":0.65500000000000003},"min":{"1min":0.44800000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.80100000000000005,"5min":1.758,"15min":1.758},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.625,"5min":0.65000000000000002,"15min":0.67000000000000004},"min":{"1min":0.48099999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.80800000000000005,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.59899999999999998}]},{"osd":6,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65100000000000002,"15min":0.65700000000000003},"min":{"1min":0.45500000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.88,"5min":1.665,"15min":1.665},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.626,"5min":0.64800000000000002,"15min":0.65900000000000003},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.86599999999999999,"5min":1.819,"15min":1.819},"last":0.74199999999999999}]},{"osd":7,"last update":"Sat May 20 01:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.68000000000000005,"15min":0.69199999999999995},"min":{"1min":0.45500000000000002,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.83199999999999996,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.66600000000000004,"15min":0.68400000000000005},"min":{"1min":0.49299999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.89300000000000002,"5min":1.8460000000000001,"15min":1.8460000000000001},"last":0.70799999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574895,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9656,"kb_used_data":752,"kb_used_omap":0,"kb_used_meta":8896,"kb_avail":94362184,"statfs":{"total":96636764160,"available":96626876416,"internally_reserved":0,"allocated":770048,"data_stored":375171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9109504},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:19:23 2023","interfaces":[{"interface":"back","average":{"1min":0.33100000000000002,"5min":0.318,"15min":0.32400000000000001},"min":{"1min":0.219,"5min":0.192,"15min":0.192},"max":{"1min":0.46100000000000002,"5min":0.504,"15min":0.504},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.42699999999999999,"15min":0.47599999999999998},"min":{"1min":0.247,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.52800000000000002,"5min":3.1890000000000001,"15min":3.1890000000000001},"last":0.34799999999999998}]},{"osd":1,"last update":"Sat May 20 01:19:23 2023","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.42099999999999999,"15min":0.46700000000000003},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.501,"5min":3.4260000000000002,"15min":3.4260000000000002},"last":0.316},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.36199999999999999,"15min":0.36399999999999999},"min":{"1min":0.26100000000000001,"5min":0.221,"15min":0.221},"max":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"last":0.48699999999999999}]},{"osd":2,"last update":"Sat May 20 01:19:23 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.39800000000000002,"15min":0.41299999999999998},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.62,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.41999999999999998,"15min":0.44},"min":{"1min":0.28699999999999998,"5min":0.253,"15min":0.253},"max":{"1min":0.58899999999999997,"5min":1.413,"15min":1.413},"last":0.52900000000000003}]},{"osd":4,"last update":"Sat May 20 01:19:23 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.59599999999999997,"15min":0.60799999999999998},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.80100000000000005,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.503},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.68300000000000005,"15min":0.72499999999999998},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.83599999999999997,"5min":3.4660000000000002,"15min":3.4660000000000002},"last":0.56000000000000005}]},{"osd":5,"last update":"Sat May 20 01:19:23 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.69399999999999995,"15min":0.73999999999999999},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.77900000000000003,"5min":3.5379999999999998,"15min":3.5379999999999998},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.63300000000000001,"15min":0.65300000000000002},"min":{"1min":0.42599999999999999,"5min":0.374,"15min":0.374},"max":{"1min":0.81699999999999995,"5min":1.385,"15min":1.385},"last":0.58699999999999997}]},{"osd":6,"last update":"Sat May 20 01:19:23 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.63800000000000001,"15min":0.65800000000000003},"min":{"1min":0.40400000000000003,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.84299999999999997,"5min":1.347,"15min":1.347},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64200000000000002,"15min":0.65700000000000003},"min":{"1min":0.434,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.77400000000000002,"5min":1.107,"15min":1.107},"last":0.60999999999999999}]},{"osd":7,"last update":"Sat May 20 01:19:23 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.64600000000000002,"15min":0.66000000000000003},"min":{"1min":0.46600000000000003,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.83199999999999996,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.72899999999999998,"15min":0.77400000000000002},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.82199999999999995,"5min":3.5009999999999999,"15min":3.5009999999999999},"last":0.59699999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574895,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9988,"kb_used_data":764,"kb_used_omap":0,"kb_used_meta":9216,"kb_avail":94361852,"statfs":{"total":96636764160,"available":96626536448,"internally_reserved":0,"allocated":782336,"data_stored":376735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9437184},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:19:24 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.495,"15min":0.48099999999999998},"min":{"1min":0.379,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.72399999999999998,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.379},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.497,"15min":0.49299999999999999},"min":{"1min":0.35499999999999998,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.53900000000000003}]},{"osd":1,"last update":"Sat May 20 01:19:24 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.53400000000000003,"15min":0.53200000000000003},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.73299999999999998,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.54600000000000004,"15min":0.53200000000000003},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":2.0739999999999998,"5min":2.0739999999999998,"15min":2.0739999999999998},"last":0.46200000000000002}]},{"osd":2,"last update":"Sat May 20 01:19:24 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.51400000000000001,"15min":0.499},"min":{"1min":0.439,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.745,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.55400000000000005,"15min":0.55500000000000005},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.751,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.56699999999999995}]},{"osd":3,"last update":"Sat May 20 01:19:24 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.53300000000000003,"15min":0.51900000000000002},"min":{"1min":0.42699999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.827,"5min":1.827,"15min":1.827},"last":0.439},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.66700000000000004,"15min":0.70699999999999996},"min":{"1min":0.40400000000000003,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.77400000000000002,"5min":3.1749999999999998,"15min":3.1749999999999998},"last":0.51300000000000001}]},{"osd":5,"last update":"Sat May 20 01:19:24 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.47199999999999998,"15min":0.46000000000000002},"min":{"1min":0.33800000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.63100000000000001,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.49199999999999999,"15min":0.49299999999999999},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.67300000000000004,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.58099999999999996}]},{"osd":6,"last update":"Sat May 20 01:19:24 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52900000000000003,"15min":0.52500000000000002},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.64700000000000002,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.58199999999999996,"15min":0.57799999999999996},"min":{"1min":0.46200000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.65500000000000003,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.59599999999999997}]},{"osd":7,"last update":"Sat May 20 01:19:24 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.54800000000000004,"15min":0.54500000000000004},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.499},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.58599999999999997,"15min":0.58999999999999997},"min":{"1min":0.39100000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.67200000000000004,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.55600000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574896,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9464,"kb_used_data":752,"kb_used_omap":0,"kb_used_meta":8704,"kb_avail":94362376,"statfs":{"total":96636764160,"available":96627073024,"internally_reserved":0,"allocated":770048,"data_stored":374232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8912896},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:19:26 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.68799999999999994,"15min":0.77400000000000002},"min":{"1min":0.33700000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.71899999999999997,"5min":6.3929999999999998,"15min":6.3929999999999998},"last":0.311},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.72199999999999998,"15min":0.81200000000000006},"min":{"1min":0.434,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.745,"5min":6.3339999999999996,"15min":6.3339999999999996},"last":0.33700000000000002}]},{"osd":1,"last update":"Sat May 20 01:19:26 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.60999999999999999,"15min":0.66500000000000004},"min":{"1min":0.38100000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.73899999999999999,"5min":3.2450000000000001,"15min":3.2450000000000001},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54500000000000004,"15min":0.55500000000000005},"min":{"1min":0.41199999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.71899999999999997,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.41899999999999998}]},{"osd":2,"last update":"Sat May 20 01:19:26 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.55500000000000005,"15min":0.57599999999999996},"min":{"1min":0.34100000000000003,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.76300000000000001,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.56100000000000005,"15min":0.57899999999999996},"min":{"1min":0.375,"5min":0.314,"15min":0.314},"max":{"1min":0.72299999999999998,"5min":1.258,"15min":1.258},"last":0.56200000000000006}]},{"osd":3,"last update":"Sat May 20 01:19:26 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.751,"15min":0.85699999999999998},"min":{"1min":0.38200000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.68600000000000005,"5min":6.351,"15min":6.351},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.79300000000000004,"15min":0.89800000000000002},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.72599999999999998,"5min":6.3730000000000002,"15min":6.3730000000000002},"last":0.47899999999999998}]},{"osd":4,"last update":"Sat May 20 01:19:26 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.48599999999999999,"15min":0.48799999999999999},"min":{"1min":0.32400000000000001,"5min":0.316,"15min":0.316},"max":{"1min":0.65300000000000002,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.54800000000000004,"15min":0.56000000000000005},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.497}]},{"osd":6,"last update":"Sat May 20 01:19:26 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.54300000000000004,"15min":0.55900000000000005},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.67600000000000005,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.56000000000000005,"15min":0.56599999999999995},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.747,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.60599999999999998}]},{"osd":7,"last update":"Sat May 20 01:19:26 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.57899999999999996,"15min":0.59199999999999997},"min":{"1min":0.36199999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.76900000000000002,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.61299999999999999,"15min":0.627},"min":{"1min":0.41399999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.73399999999999999,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.57299999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-05-20T01:20:01.529 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-05-20T01:20:01.529 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-05-20T01:20:01.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.689+0000 7f8041c22700 1 Processor -- start 2023-05-20T01:20:01.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.689+0000 7f8041c22700 1 -- start start 2023-05-20T01:20:01.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.690+0000 7f8041c22700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f803c06bd30 0x7f803c0cdad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:20:01.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.690+0000 7f8041c22700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f803c0ce010 0x7f803c0cc3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:20:01.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.690+0000 7f8041c22700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f803c0cb6c0 con 0x7f803c146710 2023-05-20T01:20:01.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.690+0000 7f8041c22700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f803c0ce840 con 0x7f803c0ce010 2023-05-20T01:20:01.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.690+0000 7f8041c22700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f803c0cb910 con 0x7f803c06bd30 2023-05-20T01:20:01.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.690+0000 7f803bfff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f803c0ce010 0x7f803c0cc3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:20:01.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.690+0000 7f803bfff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f803c0ce010 0x7f803c0cc3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33972/0 (socket says 172.21.15.16:33972) 2023-05-20T01:20:01.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.690+0000 7f803bfff700 1 -- 172.21.15.16:0/3979850301 learned_addr learned my addr 172.21.15.16:0/3979850301 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:20:01.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.690+0000 7f803affd700 1 --2- 172.21.15.16:0/3979850301 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f803c06bd30 0x7f803c0cdad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:20:01.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.690+0000 7f803bfff700 1 -- 172.21.15.16:0/3979850301 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f803c06bd30 msgr2=0x7f803c0cdad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:20:01.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.690+0000 7f803bfff700 1 --2- 172.21.15.16:0/3979850301 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f803c06bd30 0x7f803c0cdad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:20:01.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.690+0000 7f803bfff700 1 -- 172.21.15.16:0/3979850301 >> v1:172.21.15.132:6789/0 conn(0x7f803c146710 legacy=0x7f803c06b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:20:01.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.690+0000 7f803bfff700 1 -- 172.21.15.16:0/3979850301 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f803c0cbbf0 con 0x7f803c0ce010 2023-05-20T01:20:01.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.690+0000 7f803bfff700 1 --2- 172.21.15.16:0/3979850301 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f803c0ce010 0x7f803c0cc3f0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f8034009e50 tx=0x7f803400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a2560218f01476e server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:20:01.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.690+0000 7f803a7fc700 1 -- 172.21.15.16:0/3979850301 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f803400e070 con 0x7f803c0ce010 2023-05-20T01:20:01.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.690+0000 7f803a7fc700 1 -- 172.21.15.16:0/3979850301 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f803400b910 con 0x7f803c0ce010 2023-05-20T01:20:01.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.690+0000 7f803a7fc700 1 -- 172.21.15.16:0/3979850301 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f803400bcc0 con 0x7f803c0ce010 2023-05-20T01:20:01.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.691+0000 7f8041c22700 1 -- 172.21.15.16:0/3979850301 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f803c0ce010 msgr2=0x7f803c0cc3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:20:01.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.691+0000 7f8041c22700 1 --2- 172.21.15.16:0/3979850301 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f803c0ce010 0x7f803c0cc3f0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f8034009e50 tx=0x7f803400b330 comp rx=0 tx=0).stop 2023-05-20T01:20:01.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.691+0000 7f8041c22700 1 -- 172.21.15.16:0/3979850301 shutdown_connections 2023-05-20T01:20:01.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.691+0000 7f8041c22700 1 --2- 172.21.15.16:0/3979850301 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f803c0ce010 0x7f803c0cc3f0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:20:01.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.691+0000 7f8041c22700 1 --2- 172.21.15.16:0/3979850301 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f803c06bd30 0x7f803c0cdad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:20:01.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.691+0000 7f8041c22700 1 -- 172.21.15.16:0/3979850301 >> 172.21.15.16:0/3979850301 conn(0x7f803c06ae30 msgr2=0x7f803c09a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:20:01.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.691+0000 7f8041c22700 1 -- 172.21.15.16:0/3979850301 shutdown_connections 2023-05-20T01:20:01.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.691+0000 7f8041c22700 1 -- 172.21.15.16:0/3979850301 wait complete. 2023-05-20T01:20:01.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.692+0000 7f8041c22700 1 Processor -- start 2023-05-20T01:20:01.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.692+0000 7f8041c22700 1 -- start start 2023-05-20T01:20:01.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.693+0000 7f8041c22700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f803c06bd30 0x7f803c1f67c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:20:01.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.693+0000 7f8041c22700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f803c0ce010 0x7f803c155f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:20:01.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.693+0000 7f8041c22700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f803c146710 0x7f803c1f2f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:20:01.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.693+0000 7f8041c22700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f803c1f6da0 con 0x7f803c06bd30 2023-05-20T01:20:01.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.693+0000 7f8041c22700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f803c1f2ad0 con 0x7f803c146710 2023-05-20T01:20:01.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.693+0000 7f8041c22700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f803c1f27c0 con 0x7f803c0ce010 2023-05-20T01:20:01.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.693+0000 7f803b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f803c06bd30 0x7f803c1f67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:20:01.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.693+0000 7f803b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f803c06bd30 0x7f803c1f67c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33974/0 (socket says 172.21.15.16:33974) 2023-05-20T01:20:01.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.693+0000 7f803b7fe700 1 -- 172.21.15.16:0/1949573938 learned_addr learned my addr 172.21.15.16:0/1949573938 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:20:01.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.693+0000 7f803affd700 1 --2- 172.21.15.16:0/1949573938 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f803c0ce010 0x7f803c155f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:20:01.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.693+0000 7f803bfff700 1 --2- 172.21.15.16:0/1949573938 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f803c146710 0x7f803c1f2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:20:01.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.694+0000 7f803b7fe700 1 -- 172.21.15.16:0/1949573938 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f803c0ce010 msgr2=0x7f803c155f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:20:01.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.694+0000 7f803b7fe700 1 --2- 172.21.15.16:0/1949573938 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f803c0ce010 0x7f803c155f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:20:01.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.694+0000 7f803b7fe700 1 -- 172.21.15.16:0/1949573938 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f803c146710 msgr2=0x7f803c1f2f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:20:01.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.694+0000 7f803b7fe700 1 --2- 172.21.15.16:0/1949573938 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f803c146710 0x7f803c1f2f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:20:01.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.694+0000 7f803b7fe700 1 -- 172.21.15.16:0/1949573938 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8034009b30 con 0x7f803c06bd30 2023-05-20T01:20:01.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.694+0000 7f803b7fe700 1 --2- 172.21.15.16:0/1949573938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f803c06bd30 0x7f803c1f67c0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f803000c450 tx=0x7f803000c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:20:01.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.694+0000 7f8023fff700 1 -- 172.21.15.16:0/1949573938 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80300052d0 con 0x7f803c06bd30 2023-05-20T01:20:01.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.694+0000 7f8023fff700 1 -- 172.21.15.16:0/1949573938 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8030005430 con 0x7f803c06bd30 2023-05-20T01:20:01.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.694+0000 7f8041c22700 1 -- 172.21.15.16:0/1949573938 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f803c0ce490 con 0x7f803c06bd30 2023-05-20T01:20:01.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.694+0000 7f8023fff700 1 -- 172.21.15.16:0/1949573938 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80300038c0 con 0x7f803c06bd30 2023-05-20T01:20:01.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.694+0000 7f8041c22700 1 -- 172.21.15.16:0/1949573938 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f803c0cbed0 con 0x7f803c06bd30 2023-05-20T01:20:01.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.695+0000 7f8041c22700 1 -- 172.21.15.16:0/1949573938 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f803c0cc930 con 0x7f803c06bd30 2023-05-20T01:20:01.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.695+0000 7f8023fff700 1 -- 172.21.15.16:0/1949573938 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8030003a20 con 0x7f803c06bd30 2023-05-20T01:20:01.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.696+0000 7f8023fff700 1 --2- 172.21.15.16:0/1949573938 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f802806ace0 0x7f802806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:20:01.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.696+0000 7f8023fff700 1 -- 172.21.15.16:0/1949573938 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f8030088850 con 0x7f803c06bd30 2023-05-20T01:20:01.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.696+0000 7f803affd700 1 --2- 172.21.15.16:0/1949573938 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f802806ace0 0x7f802806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:20:01.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.696+0000 7f803affd700 1 --2- 172.21.15.16:0/1949573938 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f802806ace0 0x7f802806d190 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f803c06ff10 tx=0x7f802c009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:20:01.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.700+0000 7f8023fff700 1 -- 172.21.15.16:0/1949573938 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f8030053850 con 0x7f803c06bd30 2023-05-20T01:20:01.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.982+0000 7f8041c22700 1 -- 172.21.15.16:0/1949573938 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f803c1f2db0 con 0x7f803c06bd30 2023-05-20T01:20:01.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.986+0000 7f8023fff700 1 -- 172.21.15.16:0/1949573938 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v10) v1 ==== 190+0+0 (secure 0 0 0) 0x7f803000ce60 con 0x7f803c06bd30 2023-05-20T01:20:01.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.988+0000 7f8041c22700 1 -- 172.21.15.16:0/1949573938 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f802806ace0 msgr2=0x7f802806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:20:01.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.988+0000 7f8041c22700 1 --2- 172.21.15.16:0/1949573938 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f802806ace0 0x7f802806d190 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f803c06ff10 tx=0x7f802c009210 comp rx=0 tx=0).stop 2023-05-20T01:20:01.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.988+0000 7f8041c22700 1 -- 172.21.15.16:0/1949573938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f803c06bd30 msgr2=0x7f803c1f67c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:20:01.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.988+0000 7f8041c22700 1 --2- 172.21.15.16:0/1949573938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f803c06bd30 0x7f803c1f67c0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f803000c450 tx=0x7f803000c760 comp rx=0 tx=0).stop 2023-05-20T01:20:01.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.989+0000 7f8041c22700 1 -- 172.21.15.16:0/1949573938 shutdown_connections 2023-05-20T01:20:01.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.989+0000 7f8041c22700 1 --2- 172.21.15.16:0/1949573938 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f803c146710 0x7f803c1f2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:20:01.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.989+0000 7f8041c22700 1 --2- 172.21.15.16:0/1949573938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f803c06bd30 0x7f803c1f67c0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:20:01.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.989+0000 7f8041c22700 1 --2- 172.21.15.16:0/1949573938 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f802806ace0 0x7f802806d190 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:20:01.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.989+0000 7f8041c22700 1 --2- 172.21.15.16:0/1949573938 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f803c0ce010 0x7f803c155f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:20:01.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.989+0000 7f8041c22700 1 -- 172.21.15.16:0/1949573938 >> 172.21.15.16:0/1949573938 conn(0x7f803c06ae30 msgr2=0x7f803c06c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:20:01.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.989+0000 7f8041c22700 1 -- 172.21.15.16:0/1949573938 shutdown_connections 2023-05-20T01:20:01.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:01.989+0000 7f8041c22700 1 -- 172.21.15.16:0/1949573938 wait complete. 2023-05-20T01:20:01.994 INFO:teuthology.orchestra.run.smithi016.stderr:updated caps for client.0 2023-05-20T01:20:02.004 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-20T01:20:02.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.125+0000 7fd61e92c700 1 Processor -- start 2023-05-20T01:20:02.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.125+0000 7fd61e92c700 1 -- start start 2023-05-20T01:20:02.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.126+0000 7fd61e92c700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fd61806d2a0 0x7fd6180b1720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:20:02.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.126+0000 7fd61e92c700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd61815aaa0 con 0x7fd61815a6d0 2023-05-20T01:20:02.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.126+0000 7fd61e92c700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd618160610 con 0x7fd6180b1c60 2023-05-20T01:20:02.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.126+0000 7fd61e92c700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd618161190 con 0x7fd61806d2a0 2023-05-20T01:20:02.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.126+0000 7fd617fff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fd61815a6d0 0x7fd61806cb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:45510/0 (socket says 172.21.15.16:45510) 2023-05-20T01:20:02.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.126+0000 7fd617fff700 1 -- 172.21.15.16:0/3282252817 learned_addr learned my addr 172.21.15.16:0/3282252817 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:20:02.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.127+0000 7fd6177fe700 1 --2- 172.21.15.16:0/3282252817 >> v2:172.21.15.132:3300/0 conn(0x7fd61806d2a0 0x7fd6180b1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:20:02.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.127+0000 7fd616ffd700 1 -- 172.21.15.16:0/3282252817 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2862722966 0 0) 0x7fd608001110 con 0x7fd61815a6d0 2023-05-20T01:20:02.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.127+0000 7fd616ffd700 1 -- 172.21.15.16:0/3282252817 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6000035b0 con 0x7fd61815a6d0 2023-05-20T01:20:02.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.127+0000 7fd616ffd700 1 -- 172.21.15.16:0/3282252817 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3654009663 0 0) 0x7fd618160610 con 0x7fd6180b1c60 2023-05-20T01:20:02.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.127+0000 7fd616ffd700 1 -- 172.21.15.16:0/3282252817 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd608001110 con 0x7fd6180b1c60 2023-05-20T01:20:02.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.127+0000 7fd616ffd700 1 -- 172.21.15.16:0/3282252817 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1450943216 0 0) 0x7fd6000035b0 con 0x7fd61815a6d0 2023-05-20T01:20:02.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.127+0000 7fd616ffd700 1 -- 172.21.15.16:0/3282252817 >> v2:172.21.15.132:3300/0 conn(0x7fd61806d2a0 msgr2=0x7fd6180b1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:20:02.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.127+0000 7fd616ffd700 1 --2- 172.21.15.16:0/3282252817 >> v2:172.21.15.132:3300/0 conn(0x7fd61806d2a0 0x7fd6180b1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:20:02.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.127+0000 7fd616ffd700 1 -- 172.21.15.16:0/3282252817 >> v1:172.21.15.132:6789/0 conn(0x7fd6180b1c60 legacy=0x7fd6180c8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:20:02.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.127+0000 7fd616ffd700 1 -- 172.21.15.16:0/3282252817 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6180cae20 con 0x7fd61815a6d0 2023-05-20T01:20:02.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.127+0000 7fd616ffd700 1 -- 172.21.15.16:0/3282252817 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7fd608001800 con 0x7fd61815a6d0 2023-05-20T01:20:02.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.127+0000 7fd616ffd700 1 -- 172.21.15.16:0/3282252817 >> v1:172.21.15.16:6789/0 conn(0x7fd61815a6d0 legacy=0x7fd61806cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:20:02.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.127+0000 7fd616ffd700 1 --2- 172.21.15.16:0/3282252817 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd600003c00 0x7fd600003ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:20:02.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.127+0000 7fd616ffd700 1 --2- 172.21.15.16:0/3282252817 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd600004850 0x7fd600004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:20:02.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.127+0000 7fd616ffd700 1 --2- 172.21.15.16:0/3282252817 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd600005310 0x7fd60000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:20:02.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.127+0000 7fd616ffd700 1 -- 172.21.15.16:0/3282252817 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd618161190 con 0x7fd600004850 2023-05-20T01:20:02.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.127+0000 7fd616ffd700 1 -- 172.21.15.16:0/3282252817 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd60000c210 con 0x7fd600003c00 2023-05-20T01:20:02.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.127+0000 7fd616ffd700 1 -- 172.21.15.16:0/3282252817 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd60000c3d0 con 0x7fd600005310 2023-05-20T01:20:02.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.127+0000 7fd6177fe700 1 --2- 172.21.15.16:0/3282252817 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd600004850 0x7fd600004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:20:02.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.127+0000 7fd617fff700 1 --2- 172.21.15.16:0/3282252817 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd600003c00 0x7fd600003ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:20:02.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.127+0000 7fd61cec9700 1 --2- 172.21.15.16:0/3282252817 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd600005310 0x7fd60000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:20:02.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.127+0000 7fd6177fe700 1 -- 172.21.15.16:0/3282252817 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd600005310 msgr2=0x7fd60000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:20:02.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.127+0000 7fd6177fe700 1 --2- 172.21.15.16:0/3282252817 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd600005310 0x7fd60000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:20:02.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.127+0000 7fd6177fe700 1 -- 172.21.15.16:0/3282252817 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd600003c00 msgr2=0x7fd600003ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:20:02.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.127+0000 7fd6177fe700 1 --2- 172.21.15.16:0/3282252817 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd600003c00 0x7fd600003ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:20:02.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.128+0000 7fd6177fe700 1 -- 172.21.15.16:0/3282252817 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6000035b0 con 0x7fd600004850 2023-05-20T01:20:02.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.128+0000 7fd61cec9700 1 -- 172.21.15.16:0/3282252817 reap_dead start 2023-05-20T01:20:02.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.128+0000 7fd6177fe700 1 --2- 172.21.15.16:0/3282252817 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd600004850 0x7fd600004cc0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fd60c009b30 tx=0x7fd600005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bcbde901bd307e8 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:20:02.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.128+0000 7fd616ffd700 1 -- 172.21.15.16:0/3282252817 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd60c00e6c0 con 0x7fd600004850 2023-05-20T01:20:02.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.128+0000 7fd616ffd700 1 -- 172.21.15.16:0/3282252817 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd60c00e820 con 0x7fd600004850 2023-05-20T01:20:02.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.128+0000 7fd616ffd700 1 -- 172.21.15.16:0/3282252817 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd60c00eb90 con 0x7fd600004850 2023-05-20T01:20:02.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.128+0000 7fd61e92c700 1 -- 172.21.15.16:0/3282252817 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd600004850 msgr2=0x7fd600004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:20:02.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.128+0000 7fd61e92c700 1 --2- 172.21.15.16:0/3282252817 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd600004850 0x7fd600004cc0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fd60c009b30 tx=0x7fd600005880 comp rx=0 tx=0).stop 2023-05-20T01:20:02.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.128+0000 7fd61e92c700 1 -- 172.21.15.16:0/3282252817 shutdown_connections 2023-05-20T01:20:02.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.128+0000 7fd61e92c700 1 --2- 172.21.15.16:0/3282252817 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd600004850 0x7fd600004cc0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:20:02.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.128+0000 7fd61e92c700 1 -- 172.21.15.16:0/3282252817 >> 172.21.15.16:0/3282252817 conn(0x7fd6180661b0 msgr2=0x7fd618066580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:20:02.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.128+0000 7fd61e92c700 1 -- 172.21.15.16:0/3282252817 shutdown_connections 2023-05-20T01:20:02.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.128+0000 7fd61e92c700 1 -- 172.21.15.16:0/3282252817 wait complete. 2023-05-20T01:20:02.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.130+0000 7fd61e92c700 1 Processor -- start 2023-05-20T01:20:02.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.130+0000 7fd61e92c700 1 -- start start 2023-05-20T01:20:02.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.131+0000 7fd61e92c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd600004850 0x7fd61814b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:20:02.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.131+0000 7fd61e92c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd61815f910 0x7fd61814c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:20:02.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.131+0000 7fd61e92c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6180b2e80 0x7fd61815bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:20:02.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.131+0000 7fd61e92c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd61814bad0 con 0x7fd6180b2e80 2023-05-20T01:20:02.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.131+0000 7fd61e92c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd618152ee0 con 0x7fd61815f910 2023-05-20T01:20:02.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.131+0000 7fd61e92c700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6181531c0 con 0x7fd600004850 2023-05-20T01:20:02.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.131+0000 7fd61cec9700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6180b2e80 0x7fd61815bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:20:02.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.131+0000 7fd61cec9700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6180b2e80 0x7fd61815bdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33998/0 (socket says 172.21.15.16:33998) 2023-05-20T01:20:02.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.131+0000 7fd61cec9700 1 -- 172.21.15.16:0/1725781832 learned_addr learned my addr 172.21.15.16:0/1725781832 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:20:02.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.131+0000 7fd617fff700 1 --2- 172.21.15.16:0/1725781832 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd600004850 0x7fd61814b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:20:02.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.131+0000 7fd6177fe700 1 --2- 172.21.15.16:0/1725781832 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd61815f910 0x7fd61814c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:20:02.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.132+0000 7fd61cec9700 1 -- 172.21.15.16:0/1725781832 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd600004850 msgr2=0x7fd61814b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:20:02.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.132+0000 7fd61cec9700 1 --2- 172.21.15.16:0/1725781832 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd600004850 0x7fd61814b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:20:02.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.132+0000 7fd61cec9700 1 -- 172.21.15.16:0/1725781832 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd61815f910 msgr2=0x7fd61814c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:20:02.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.132+0000 7fd61cec9700 1 --2- 172.21.15.16:0/1725781832 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd61815f910 0x7fd61814c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:20:02.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.132+0000 7fd61cec9700 1 -- 172.21.15.16:0/1725781832 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd618160b40 con 0x7fd6180b2e80 2023-05-20T01:20:02.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.132+0000 7fd617fff700 1 --2- 172.21.15.16:0/1725781832 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd600004850 0x7fd61814b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:20:02.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.132+0000 7fd6177fe700 1 --2- 172.21.15.16:0/1725781832 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd61815f910 0x7fd61814c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:20:02.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.132+0000 7fd61cec9700 1 --2- 172.21.15.16:0/1725781832 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6180b2e80 0x7fd61815bdc0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fd6180b5200 tx=0x7fd61000ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:20:02.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.132+0000 7fd614ff9700 1 -- 172.21.15.16:0/1725781832 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd610003c10 con 0x7fd6180b2e80 2023-05-20T01:20:02.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.132+0000 7fd614ff9700 1 -- 172.21.15.16:0/1725781832 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd610003d70 con 0x7fd6180b2e80 2023-05-20T01:20:02.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.132+0000 7fd61e92c700 1 -- 172.21.15.16:0/1725781832 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd618160860 con 0x7fd6180b2e80 2023-05-20T01:20:02.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.132+0000 7fd614ff9700 1 -- 172.21.15.16:0/1725781832 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd610005310 con 0x7fd6180b2e80 2023-05-20T01:20:02.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.132+0000 7fd61e92c700 1 -- 172.21.15.16:0/1725781832 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd618160e20 con 0x7fd6180b2e80 2023-05-20T01:20:02.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.133+0000 7fd61e92c700 1 -- 172.21.15.16:0/1725781832 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd618153dd0 con 0x7fd6180b2e80 2023-05-20T01:20:02.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.134+0000 7fd614ff9700 1 -- 172.21.15.16:0/1725781832 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd610005470 con 0x7fd6180b2e80 2023-05-20T01:20:02.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.134+0000 7fd614ff9700 1 --2- 172.21.15.16:0/1725781832 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd60406ace0 0x7fd60406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:20:02.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.134+0000 7fd614ff9700 1 -- 172.21.15.16:0/1725781832 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fd610089560 con 0x7fd6180b2e80 2023-05-20T01:20:02.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.134+0000 7fd617fff700 1 --2- 172.21.15.16:0/1725781832 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd60406ace0 0x7fd60406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:20:02.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.134+0000 7fd617fff700 1 --2- 172.21.15.16:0/1725781832 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd60406ace0 0x7fd60406d190 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fd6181560a0 tx=0x7fd608009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:20:02.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.138+0000 7fd614ff9700 1 -- 172.21.15.16:0/1725781832 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fd6100544c0 con 0x7fd6180b2e80 2023-05-20T01:20:02.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.378+0000 7fd61e92c700 1 -- 172.21.15.16:0/1725781832 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd6180747b0 con 0x7fd6180b2e80 2023-05-20T01:20:02.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.379+0000 7fd614ff9700 1 -- 172.21.15.16:0/1725781832 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+4672 (secure 0 0 0) 0x7fd610057ae0 con 0x7fd6180b2e80 2023-05-20T01:20:02.383 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:20:02.383 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":37,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5824,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/2609276800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":2609276800},{"type":"v1","addr":"172.21.15.132:6834","nonce":2609276800}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32},{"gid":5827,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/29554453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":29554453},{"type":"v1","addr":"172.21.15.132:6835","nonce":29554453}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32},{"gid":6192,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/4090303296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4090303296},{"type":"v1","addr":"172.21.15.16:6836","nonce":4090303296}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-20T01:19:51.915360+0000","modified":"2023-05-20T01:19:55.342773+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6195},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6195":{"gid":6195,"name":"c","rank":0,"incarnation":33,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6837/1427908556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":1427908556},{"type":"v1","addr":"172.21.15.16:6837","nonce":1427908556}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-05-20T01:20:02.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.381+0000 7fd61e92c700 1 -- 172.21.15.16:0/1725781832 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd60406ace0 msgr2=0x7fd60406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:20:02.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.381+0000 7fd61e92c700 1 --2- 172.21.15.16:0/1725781832 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd60406ace0 0x7fd60406d190 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fd6181560a0 tx=0x7fd608009040 comp rx=0 tx=0).stop 2023-05-20T01:20:02.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.381+0000 7fd61e92c700 1 -- 172.21.15.16:0/1725781832 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6180b2e80 msgr2=0x7fd61815bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:20:02.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.381+0000 7fd61e92c700 1 --2- 172.21.15.16:0/1725781832 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6180b2e80 0x7fd61815bdc0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fd6180b5200 tx=0x7fd61000ee00 comp rx=0 tx=0).stop 2023-05-20T01:20:02.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.381+0000 7fd61e92c700 1 -- 172.21.15.16:0/1725781832 shutdown_connections 2023-05-20T01:20:02.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.381+0000 7fd61e92c700 1 --2- 172.21.15.16:0/1725781832 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6180b2e80 0x7fd61815bdc0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:20:02.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.381+0000 7fd61e92c700 1 --2- 172.21.15.16:0/1725781832 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd60406ace0 0x7fd60406d190 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:20:02.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.381+0000 7fd61e92c700 1 --2- 172.21.15.16:0/1725781832 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd600004850 0x7fd61814b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:20:02.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.381+0000 7fd61e92c700 1 --2- 172.21.15.16:0/1725781832 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd61815f910 0x7fd61814c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:20:02.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.381+0000 7fd61e92c700 1 -- 172.21.15.16:0/1725781832 >> 172.21.15.16:0/1725781832 conn(0x7fd6180661b0 msgr2=0x7fd6180cf670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:20:02.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.381+0000 7fd61e92c700 1 -- 172.21.15.16:0/1725781832 shutdown_connections 2023-05-20T01:20:02.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.381+0000 7fd61e92c700 1 -- 172.21.15.16:0/1725781832 wait complete. 2023-05-20T01:20:02.386 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 37 2023-05-20T01:20:02.799 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 37, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-05-20T01:19:51.915360+0000', 'modified': '2023-05-20T01:19:55.342773+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6195}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6195': {'gid': 6195, 'name': 'c', 'rank': 0, 'incarnation': 33, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.16:6837/1427908556', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6835', 'nonce': 1427908556}, {'type': 'v1', 'addr': '172.21.15.16:6837', 'nonce': 1427908556}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-05-20T01:20:02.800 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-05-20T01:20:02.801 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-05-20T01:20:02.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.928+0000 7fb750fd1700 1 Processor -- start 2023-05-20T01:20:02.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.928+0000 7fb750fd1700 1 -- start start 2023-05-20T01:20:02.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.929+0000 7fb750fd1700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb74c0ccec0 0x7fb74c0713c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:20:02.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.929+0000 7fb750fd1700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb74c071900 0x7fb74c0ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:20:02.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.929+0000 7fb750fd1700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb74c0cb590 con 0x7fb74c157190 2023-05-20T01:20:02.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.929+0000 7fb750fd1700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb74c0ca9a0 con 0x7fb74c071900 2023-05-20T01:20:02.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.929+0000 7fb750fd1700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb74c0cac60 con 0x7fb74c0ccec0 2023-05-20T01:20:02.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.929+0000 7fb74ad9d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb74c071900 0x7fb74c0ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:20:02.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.929+0000 7fb74ad9d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb74c071900 0x7fb74c0ce740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34006/0 (socket says 172.21.15.16:34006) 2023-05-20T01:20:02.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.929+0000 7fb74ad9d700 1 -- 172.21.15.16:0/1771531752 learned_addr learned my addr 172.21.15.16:0/1771531752 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:20:02.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.929+0000 7fb74ad9d700 1 -- 172.21.15.16:0/1771531752 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb74c0ccec0 msgr2=0x7fb74c0713c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:20:02.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.929+0000 7fb749d9b700 1 --2- 172.21.15.16:0/1771531752 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb74c0ccec0 0x7fb74c0713c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:20:02.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.929+0000 7fb74ad9d700 1 --2- 172.21.15.16:0/1771531752 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb74c0ccec0 0x7fb74c0713c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:20:02.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.929+0000 7fb74ad9d700 1 -- 172.21.15.16:0/1771531752 >> v1:172.21.15.132:6789/0 conn(0x7fb74c157190 legacy=0x7fb74c0cc7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:20:02.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.929+0000 7fb74ad9d700 1 -- 172.21.15.16:0/1771531752 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb74c0caf40 con 0x7fb74c071900 2023-05-20T01:20:02.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.929+0000 7fb74ad9d700 1 --2- 172.21.15.16:0/1771531752 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb74c071900 0x7fb74c0ce740 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fb744009e50 tx=0x7fb74400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be8fcdb82bd9b17e server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:20:02.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.929+0000 7fb74959a700 1 -- 172.21.15.16:0/1771531752 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb74400e070 con 0x7fb74c071900 2023-05-20T01:20:02.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.929+0000 7fb74959a700 1 -- 172.21.15.16:0/1771531752 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb744003a20 con 0x7fb74c071900 2023-05-20T01:20:02.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.929+0000 7fb74959a700 1 -- 172.21.15.16:0/1771531752 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb744003dd0 con 0x7fb74c071900 2023-05-20T01:20:02.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.930+0000 7fb750fd1700 1 -- 172.21.15.16:0/1771531752 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb74c071900 msgr2=0x7fb74c0ce740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:20:02.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.930+0000 7fb750fd1700 1 --2- 172.21.15.16:0/1771531752 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb74c071900 0x7fb74c0ce740 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fb744009e50 tx=0x7fb74400b330 comp rx=0 tx=0).stop 2023-05-20T01:20:02.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.930+0000 7fb750fd1700 1 -- 172.21.15.16:0/1771531752 shutdown_connections 2023-05-20T01:20:02.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.930+0000 7fb750fd1700 1 --2- 172.21.15.16:0/1771531752 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb74c071900 0x7fb74c0ce740 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:20:02.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.930+0000 7fb750fd1700 1 --2- 172.21.15.16:0/1771531752 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb74c0ccec0 0x7fb74c0713c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:20:02.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.930+0000 7fb750fd1700 1 -- 172.21.15.16:0/1771531752 >> 172.21.15.16:0/1771531752 conn(0x7fb74c0cffb0 msgr2=0x7fb74c0d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:20:02.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.930+0000 7fb750fd1700 1 -- 172.21.15.16:0/1771531752 shutdown_connections 2023-05-20T01:20:02.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.930+0000 7fb750fd1700 1 -- 172.21.15.16:0/1771531752 wait complete. 2023-05-20T01:20:02.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.931+0000 7fb750fd1700 1 Processor -- start 2023-05-20T01:20:02.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.931+0000 7fb750fd1700 1 -- start start 2023-05-20T01:20:02.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.932+0000 7fb750fd1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb74c071900 0x7fb74c1f1c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:20:02.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.932+0000 7fb750fd1700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb74c0ccec0 0x7fb74c0ac8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:20:02.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.932+0000 7fb750fd1700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb74c157190 0x7fb74c14e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:20:02.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.932+0000 7fb750fd1700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb74c0c5570 con 0x7fb74c071900 2023-05-20T01:20:02.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.932+0000 7fb750fd1700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb74c0b2f50 con 0x7fb74c157190 2023-05-20T01:20:02.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.932+0000 7fb750fd1700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb74c1f2170 con 0x7fb74c0ccec0 2023-05-20T01:20:02.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.932+0000 7fb74a59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb74c071900 0x7fb74c1f1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:20:02.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.932+0000 7fb74a59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb74c071900 0x7fb74c1f1c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34008/0 (socket says 172.21.15.16:34008) 2023-05-20T01:20:02.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.932+0000 7fb74a59c700 1 -- 172.21.15.16:0/4256045728 learned_addr learned my addr 172.21.15.16:0/4256045728 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:20:02.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.932+0000 7fb749d9b700 1 --2- 172.21.15.16:0/4256045728 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb74c0ccec0 0x7fb74c0ac8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:20:02.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.932+0000 7fb74ad9d700 1 --2- 172.21.15.16:0/4256045728 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb74c157190 0x7fb74c14e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:20:02.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.933+0000 7fb74a59c700 1 -- 172.21.15.16:0/4256045728 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb74c0ccec0 msgr2=0x7fb74c0ac8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:20:02.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.933+0000 7fb74a59c700 1 --2- 172.21.15.16:0/4256045728 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb74c0ccec0 0x7fb74c0ac8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:20:02.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.933+0000 7fb74a59c700 1 -- 172.21.15.16:0/4256045728 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb74c157190 msgr2=0x7fb74c14e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:20:02.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.933+0000 7fb74a59c700 1 --2- 172.21.15.16:0/4256045728 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb74c157190 0x7fb74c14e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:20:02.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.933+0000 7fb74a59c700 1 -- 172.21.15.16:0/4256045728 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb744009b30 con 0x7fb74c071900 2023-05-20T01:20:02.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.933+0000 7fb74a59c700 1 --2- 172.21.15.16:0/4256045728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb74c071900 0x7fb74c1f1c30 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fb73c00b4a0 tx=0x7fb73c00b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:20:02.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.933+0000 7fb73affd700 1 -- 172.21.15.16:0/4256045728 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb73c011830 con 0x7fb74c071900 2023-05-20T01:20:02.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.933+0000 7fb73affd700 1 -- 172.21.15.16:0/4256045728 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb73c011990 con 0x7fb74c071900 2023-05-20T01:20:02.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.933+0000 7fb750fd1700 1 -- 172.21.15.16:0/4256045728 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb74c157560 con 0x7fb74c071900 2023-05-20T01:20:02.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.933+0000 7fb73affd700 1 -- 172.21.15.16:0/4256045728 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb73c011d40 con 0x7fb74c071900 2023-05-20T01:20:02.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.933+0000 7fb750fd1700 1 -- 172.21.15.16:0/4256045728 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb74c0cb220 con 0x7fb74c071900 2023-05-20T01:20:02.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.935+0000 7fb73affd700 1 -- 172.21.15.16:0/4256045728 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb73c01a400 con 0x7fb74c071900 2023-05-20T01:20:02.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.935+0000 7fb73affd700 1 --2- 172.21.15.16:0/4256045728 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb72c06ace0 0x7fb72c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:20:02.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.935+0000 7fb73affd700 1 -- 172.21.15.16:0/4256045728 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fb73c089380 con 0x7fb74c071900 2023-05-20T01:20:02.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.935+0000 7fb749d9b700 1 --2- 172.21.15.16:0/4256045728 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb72c06ace0 0x7fb72c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:20:02.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.936+0000 7fb749d9b700 1 --2- 172.21.15.16:0/4256045728 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb72c06ace0 0x7fb72c06d190 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb74c0b5a10 tx=0x7fb740008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:20:02.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.937+0000 7fb75d15db80 1 Processor -- start 2023-05-20T01:20:02.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.937+0000 7fb75d15db80 1 -- start start 2023-05-20T01:20:02.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.938+0000 7fb75d15db80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55adb1838400 0x55adb18387d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:20:02.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.938+0000 7fb75d15db80 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55adb1834700 0x55adb180d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:20:02.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.938+0000 7fb75d15db80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55adb180b870 0x55adb180d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:20:02.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.938+0000 7fb75d15db80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adb1713010 con 0x55adb1838400 2023-05-20T01:20:02.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.938+0000 7fb75d15db80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adb1765850 con 0x55adb180b870 2023-05-20T01:20:02.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.938+0000 7fb75d15db80 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55adb175ece0 con 0x55adb1834700 2023-05-20T01:20:02.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.938+0000 7fb74a59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55adb1838400 0x55adb18387d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:20:02.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.938+0000 7fb74a59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55adb1838400 0x55adb18387d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34016/0 (socket says 172.21.15.16:34016) 2023-05-20T01:20:02.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.938+0000 7fb74a59c700 1 -- 172.21.15.16:0/304685848 learned_addr learned my addr 172.21.15.16:0/304685848 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:20:02.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.938+0000 7fb74ad9d700 1 --2- 172.21.15.16:0/304685848 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55adb180b870 0x55adb180d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:20:02.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.938+0000 7fb749d9b700 1 --2- 172.21.15.16:0/304685848 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55adb1834700 0x55adb180d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:20:02.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.938+0000 7fb74a59c700 1 -- 172.21.15.16:0/304685848 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55adb1834700 msgr2=0x55adb180d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:20:02.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.939+0000 7fb74a59c700 1 --2- 172.21.15.16:0/304685848 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55adb1834700 0x55adb180d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:20:02.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.939+0000 7fb74a59c700 1 -- 172.21.15.16:0/304685848 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55adb180b870 msgr2=0x55adb180d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:20:02.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.939+0000 7fb74a59c700 1 --2- 172.21.15.16:0/304685848 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55adb180b870 0x55adb180d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:20:02.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.939+0000 7fb74a59c700 1 -- 172.21.15.16:0/304685848 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55adb15e05e0 con 0x55adb1838400 2023-05-20T01:20:02.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.939+0000 7fb74ad9d700 1 --2- 172.21.15.16:0/304685848 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55adb180b870 0x55adb180d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:20:02.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.939+0000 7fb74a59c700 1 --2- 172.21.15.16:0/304685848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55adb1838400 0x55adb18387d0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fb73c0874b0 tx=0x7fb73c057940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81c06fa7fac58f33 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:20:02.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.939+0000 7fb72bfff700 1 -- 172.21.15.16:0/304685848 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb73c05d040 con 0x55adb1838400 2023-05-20T01:20:02.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.939+0000 7fb72bfff700 1 -- 172.21.15.16:0/304685848 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb73c0890c0 con 0x55adb1838400 2023-05-20T01:20:02.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.939+0000 7fb72bfff700 1 -- 172.21.15.16:0/304685848 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb73c057400 con 0x55adb1838400 2023-05-20T01:20:02.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.939+0000 7fb75d15db80 1 -- 172.21.15.16:0/304685848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55adb1838400 msgr2=0x55adb18387d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:20:02.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.939+0000 7fb75d15db80 1 --2- 172.21.15.16:0/304685848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55adb1838400 0x55adb18387d0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fb73c0874b0 tx=0x7fb73c057940 comp rx=0 tx=0).stop 2023-05-20T01:20:02.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.939+0000 7fb75d15db80 1 -- 172.21.15.16:0/304685848 shutdown_connections 2023-05-20T01:20:02.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.939+0000 7fb75d15db80 1 --2- 172.21.15.16:0/304685848 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55adb180b870 0x55adb180d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:20:02.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.939+0000 7fb75d15db80 1 --2- 172.21.15.16:0/304685848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55adb1838400 0x55adb18387d0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:20:02.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.940+0000 7fb75d15db80 1 --2- 172.21.15.16:0/304685848 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55adb1834700 0x55adb180d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:20:02.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.940+0000 7fb75d15db80 1 -- 172.21.15.16:0/304685848 >> 172.21.15.16:0/304685848 conn(0x55adb1802f30 msgr2=0x55adb1853210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:20:02.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.940+0000 7fb75d15db80 1 -- 172.21.15.16:0/304685848 shutdown_connections 2023-05-20T01:20:02.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.940+0000 7fb75d15db80 1 -- 172.21.15.16:0/304685848 wait complete. 2023-05-20T01:20:02.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.940+0000 7fb75d15db80 1 -- start start 2023-05-20T01:20:02.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.940+0000 7fb75d15db80 1 Processor -- start 2023-05-20T01:20:02.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.941+0000 7fb75d15db80 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55adb180b870 0x55adb1852830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:20:02.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.941+0000 7fb75d15db80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55adb1834700 0x55adb1856060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:20:02.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.941+0000 7fb75d15db80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55adb1838400 0x55adb180ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:20:02.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.941+0000 7fb75d15db80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adb1700890 con 0x55adb1838400 2023-05-20T01:20:02.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.941+0000 7fb75d15db80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adb1700530 con 0x55adb1834700 2023-05-20T01:20:02.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.941+0000 7fb75d15db80 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55adb15e28d0 con 0x55adb180b870 2023-05-20T01:20:02.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.941+0000 7fb75d15db80 10 client.0 _ll_register_callbacks cb 0x55adb171e730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-05-20T01:20:02.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.941+0000 7fb74ad9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55adb1838400 0x55adb180ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:20:02.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.941+0000 7fb74ad9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55adb1838400 0x55adb180ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34026/0 (socket says 172.21.15.16:34026) 2023-05-20T01:20:02.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.941+0000 7fb74ad9d700 1 -- 172.21.15.16:0/1793355598 learned_addr learned my addr 172.21.15.16:0/1793355598 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:20:02.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.941+0000 7fb74a59c700 1 --2- 172.21.15.16:0/1793355598 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55adb180b870 0x55adb1852830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:20:02.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.942+0000 7fb749d9b700 1 --2- 172.21.15.16:0/1793355598 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55adb1834700 0x55adb1856060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:20:02.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.942+0000 7fb74ad9d700 1 -- 172.21.15.16:0/1793355598 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55adb180b870 msgr2=0x55adb1852830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:20:02.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.942+0000 7fb74ad9d700 1 --2- 172.21.15.16:0/1793355598 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55adb180b870 0x55adb1852830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:20:02.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.942+0000 7fb74ad9d700 1 -- 172.21.15.16:0/1793355598 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55adb1834700 msgr2=0x55adb1856060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:20:02.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.942+0000 7fb74ad9d700 1 --2- 172.21.15.16:0/1793355598 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55adb1834700 0x55adb1856060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:20:02.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.942+0000 7fb74ad9d700 1 -- 172.21.15.16:0/1793355598 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb74c0cb220 con 0x55adb1838400 2023-05-20T01:20:02.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.942+0000 7fb74ad9d700 1 --2- 172.21.15.16:0/1793355598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55adb1838400 0x55adb180ccf0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fb744004000 tx=0x7fb744004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efafe9b24d73ad18 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:20:02.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.942+0000 7fb7297fa700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-05-20T01:20:02.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.942+0000 7fb7297fa700 1 -- 172.21.15.16:0/1793355598 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb744004440 con 0x55adb1838400 2023-05-20T01:20:02.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.942+0000 7fb7297fa700 1 -- 172.21.15.16:0/1793355598 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(64..64 src has 1..64) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fb7440094d0 con 0x55adb1838400 2023-05-20T01:20:02.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.942+0000 7fb75d15db80 1 -- 172.21.15.16:0/1793355598 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55adb18006f0 con 0x55adb1838400 2023-05-20T01:20:02.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.942+0000 7fb7297fa700 1 -- 172.21.15.16:0/1793355598 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb7440045a0 con 0x55adb1838400 2023-05-20T01:20:02.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.942+0000 7fb75d15db80 10 client.6372 fetch_fsmap learned FSMap version 37 2023-05-20T01:20:02.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.942+0000 7fb75d15db80 1 -- 172.21.15.16:0/1793355598 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=2+}) v3 -- 0x55adb1725140 con 0x55adb1838400 2023-05-20T01:20:02.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.942+0000 7fb7297fa700 1 -- 172.21.15.16:0/1793355598 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55adb18006f0 con 0x55adb1838400 2023-05-20T01:20:02.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.942+0000 7fb7297fa700 1 -- 172.21.15.16:0/1793355598 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 37) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fb744004700 con 0x55adb1838400 2023-05-20T01:20:02.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.942+0000 7fb75d15db80 10 client.6372 fetch_fsmap finished waiting for FSMap version 37 2023-05-20T01:20:02.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.942+0000 7fb75d15db80 10 client.6372 resolve_mds: resolved name 'c' to daemon mds.c 2023-05-20T01:20:02.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.942+0000 7fb75d15db80 20 client.6372 populate_metadata read hostname 'smithi016' 2023-05-20T01:20:02.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.942+0000 7fb75d15db80 1 --2- 172.21.15.16:0/1793355598 >> [v2:172.21.15.16:6835/1427908556,v1:172.21.15.16:6837/1427908556] conn(0x55adb18398e0 0x55adb1839cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:20:02.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.942+0000 7fb75d15db80 4 client.6372 mds_command: new command op to 6195 tid=0[{"prefix": "get_command_descriptions"}] 2023-05-20T01:20:02.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.942+0000 7fb75d15db80 1 -- 172.21.15.16:0/1793355598 --> [v2:172.21.15.16:6835/1427908556,v1:172.21.15.16:6837/1427908556] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55adb1721050 con 0x55adb18398e0 2023-05-20T01:20:02.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.943+0000 7fb74a59c700 1 --2- 172.21.15.16:0/1793355598 >> [v2:172.21.15.16:6835/1427908556,v1:172.21.15.16:6837/1427908556] conn(0x55adb18398e0 0x55adb1839cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:20:02.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.943+0000 7fb7297fa700 10 client.6372 ms_handle_connect on v2:172.21.15.16:6835/1427908556 2023-05-20T01:20:02.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.943+0000 7fb74a59c700 1 --2- 172.21.15.16:0/1793355598 >> [v2:172.21.15.16:6835/1427908556,v1:172.21.15.16:6837/1427908556] conn(0x55adb18398e0 0x55adb1839cb0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8cc8e6728001908d server_cookie=f0f37362ba6e8b43 in_seq=0 out_seq=0 2023-05-20T01:20:02.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.943+0000 7fb7297fa700 1 -- 172.21.15.16:0/1793355598 <== mds.0 v2:172.21.15.16:6835/1427908556 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55adb1721050 con 0x55adb18398e0 2023-05-20T01:20:02.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.943+0000 7fb7297fa700 10 client.6372 handle_command_reply: tid=0 2023-05-20T01:20:02.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.943+0000 7fb75d15db80 1 client.6372 shutdown 2023-05-20T01:20:02.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.944+0000 7fb75d15db80 1 -- 172.21.15.16:0/1793355598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55adb1838400 msgr2=0x55adb180ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:20:02.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.944+0000 7fb75d15db80 1 --2- 172.21.15.16:0/1793355598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55adb1838400 0x55adb180ccf0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fb744004000 tx=0x7fb744004b40 comp rx=0 tx=0).stop 2023-05-20T01:20:02.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.944+0000 7fb75d15db80 1 -- 172.21.15.16:0/1793355598 shutdown_connections 2023-05-20T01:20:02.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.944+0000 7fb75d15db80 1 --2- 172.21.15.16:0/1793355598 >> [v2:172.21.15.16:6835/1427908556,v1:172.21.15.16:6837/1427908556] conn(0x55adb18398e0 0x55adb1839cb0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:20:02.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.944+0000 7fb75d15db80 1 --2- 172.21.15.16:0/1793355598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55adb1838400 0x55adb180ccf0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:20:02.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.944+0000 7fb7297fa700 0 client.6372 ms_handle_reset on v2:172.21.15.16:6835/1427908556 2023-05-20T01:20:02.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.944+0000 7fb75d15db80 1 --2- 172.21.15.16:0/1793355598 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55adb180b870 0x55adb1852830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:20:02.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.944+0000 7fb75d15db80 1 --2- 172.21.15.16:0/1793355598 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55adb1834700 0x55adb1856060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:20:02.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.944+0000 7fb75d15db80 1 -- 172.21.15.16:0/1793355598 >> 172.21.15.16:0/1793355598 conn(0x55adb1802f30 msgr2=0x55adb1808440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:20:02.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.944+0000 7fb75d15db80 1 -- 172.21.15.16:0/1793355598 shutdown_connections 2023-05-20T01:20:02.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.944+0000 7fb75d15db80 1 -- 172.21.15.16:0/1793355598 wait complete. 2023-05-20T01:20:02.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.944+0000 7fb75d15db80 20 client.6372 trim_cache size 0 max 16384 2023-05-20T01:20:02.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.962+0000 7fb75d15db80 1 Processor -- start 2023-05-20T01:20:02.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.962+0000 7fb75d15db80 1 -- start start 2023-05-20T01:20:02.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.962+0000 7fb75d15db80 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55adb180b870 0x55adb1832750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:20:02.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.962+0000 7fb75d15db80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55adb1834700 0x55adb1808e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:20:02.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.962+0000 7fb75d15db80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55adb1838400 0x55adb18078e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:20:02.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.962+0000 7fb75d15db80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adb183ab70 con 0x55adb1838400 2023-05-20T01:20:02.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.962+0000 7fb75d15db80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adb1855b00 con 0x55adb1834700 2023-05-20T01:20:02.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.962+0000 7fb75d15db80 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55adb18558b0 con 0x55adb180b870 2023-05-20T01:20:02.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.962+0000 7fb74ad9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55adb1838400 0x55adb18078e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:20:02.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.963+0000 7fb74ad9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55adb1838400 0x55adb18078e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34034/0 (socket says 172.21.15.16:34034) 2023-05-20T01:20:02.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.963+0000 7fb74ad9d700 1 -- 172.21.15.16:0/3389962349 learned_addr learned my addr 172.21.15.16:0/3389962349 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:20:02.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.963+0000 7fb749d9b700 1 --2- 172.21.15.16:0/3389962349 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55adb1834700 0x55adb1808e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:20:02.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.963+0000 7fb74a59c700 1 --2- 172.21.15.16:0/3389962349 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55adb180b870 0x55adb1832750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:20:02.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.963+0000 7fb74ad9d700 1 -- 172.21.15.16:0/3389962349 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55adb180b870 msgr2=0x55adb1832750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:20:02.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.963+0000 7fb74ad9d700 1 --2- 172.21.15.16:0/3389962349 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55adb180b870 0x55adb1832750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:20:02.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.963+0000 7fb74ad9d700 1 -- 172.21.15.16:0/3389962349 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55adb1834700 msgr2=0x55adb1808e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:20:02.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.963+0000 7fb74ad9d700 1 --2- 172.21.15.16:0/3389962349 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55adb1834700 0x55adb1808e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:20:02.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.963+0000 7fb74ad9d700 1 -- 172.21.15.16:0/3389962349 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55adb1724e90 con 0x55adb1838400 2023-05-20T01:20:02.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.963+0000 7fb749d9b700 1 --2- 172.21.15.16:0/3389962349 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55adb1834700 0x55adb1808e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:20:02.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.963+0000 7fb74a59c700 1 --2- 172.21.15.16:0/3389962349 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55adb180b870 0x55adb1832750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:20:02.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.963+0000 7fb74ad9d700 1 --2- 172.21.15.16:0/3389962349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55adb1838400 0x55adb18078e0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fb74400bfd0 tx=0x7fb744021070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7978d9107403f51d server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:20:02.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.963+0000 7fb738ff9700 1 -- 172.21.15.16:0/3389962349 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7440093a0 con 0x55adb1838400 2023-05-20T01:20:02.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.963+0000 7fb738ff9700 1 -- 172.21.15.16:0/3389962349 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb744009500 con 0x55adb1838400 2023-05-20T01:20:02.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.963+0000 7fb738ff9700 1 -- 172.21.15.16:0/3389962349 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7440126c0 con 0x55adb1838400 2023-05-20T01:20:02.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.964+0000 7fb75d15db80 1 -- 172.21.15.16:0/3389962349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55adb1838400 msgr2=0x55adb18078e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:20:02.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.964+0000 7fb75d15db80 1 --2- 172.21.15.16:0/3389962349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55adb1838400 0x55adb18078e0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fb74400bfd0 tx=0x7fb744021070 comp rx=0 tx=0).stop 2023-05-20T01:20:02.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.964+0000 7fb75d15db80 1 -- 172.21.15.16:0/3389962349 shutdown_connections 2023-05-20T01:20:02.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.964+0000 7fb75d15db80 1 --2- 172.21.15.16:0/3389962349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55adb1838400 0x55adb18078e0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:20:02.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.964+0000 7fb75d15db80 1 --2- 172.21.15.16:0/3389962349 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55adb180b870 0x55adb1832750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:20:02.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.964+0000 7fb75d15db80 1 --2- 172.21.15.16:0/3389962349 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55adb1834700 0x55adb1808e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:20:02.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.964+0000 7fb75d15db80 1 -- 172.21.15.16:0/3389962349 >> 172.21.15.16:0/3389962349 conn(0x55adb1802f30 msgr2=0x55adb180f6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:20:02.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.964+0000 7fb75d15db80 1 -- 172.21.15.16:0/3389962349 shutdown_connections 2023-05-20T01:20:02.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.964+0000 7fb75d15db80 1 -- 172.21.15.16:0/3389962349 wait complete. 2023-05-20T01:20:02.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.964+0000 7fb75d15db80 1 -- start start 2023-05-20T01:20:02.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.964+0000 7fb75d15db80 1 Processor -- start 2023-05-20T01:20:02.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.965+0000 7fb75d15db80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55adb180b870 0x55adb1833df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:20:02.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.965+0000 7fb75d15db80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55adb1834700 0x55adb18b97a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:20:02.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.965+0000 7fb75d15db80 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55adb1838400 0x55adb1834ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:20:02.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.965+0000 7fb75d15db80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adb184cd60 con 0x55adb180b870 2023-05-20T01:20:02.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.965+0000 7fb75d15db80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adb1832750 con 0x55adb1834700 2023-05-20T01:20:02.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.965+0000 7fb75d15db80 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55adb18b9ce0 con 0x55adb1838400 2023-05-20T01:20:02.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.965+0000 7fb75d15db80 10 client.0 _ll_register_callbacks cb 0x7fb744015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-05-20T01:20:02.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.965+0000 7fb74a59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55adb180b870 0x55adb1833df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:20:02.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.965+0000 7fb74a59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55adb180b870 0x55adb1833df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34036/0 (socket says 172.21.15.16:34036) 2023-05-20T01:20:02.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.965+0000 7fb74a59c700 1 -- 172.21.15.16:0/4129085764 learned_addr learned my addr 172.21.15.16:0/4129085764 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:20:02.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.965+0000 7fb749d9b700 1 --2- 172.21.15.16:0/4129085764 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55adb1834700 0x55adb18b97a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:20:02.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.966+0000 7fb74ad9d700 1 --2- 172.21.15.16:0/4129085764 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55adb1838400 0x55adb1834ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:20:02.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.966+0000 7fb74a59c700 1 -- 172.21.15.16:0/4129085764 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55adb1838400 msgr2=0x55adb1834ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:20:02.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.966+0000 7fb74a59c700 1 --2- 172.21.15.16:0/4129085764 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55adb1838400 0x55adb1834ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:20:02.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.966+0000 7fb74a59c700 1 -- 172.21.15.16:0/4129085764 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55adb1834700 msgr2=0x55adb18b97a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:20:02.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.966+0000 7fb74a59c700 1 --2- 172.21.15.16:0/4129085764 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55adb1834700 0x55adb18b97a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:20:02.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.966+0000 7fb74a59c700 1 -- 172.21.15.16:0/4129085764 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55adb1725140 con 0x55adb180b870 2023-05-20T01:20:02.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.966+0000 7fb749d9b700 1 --2- 172.21.15.16:0/4129085764 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55adb1834700 0x55adb18b97a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:20:02.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.966+0000 7fb74a59c700 1 --2- 172.21.15.16:0/4129085764 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55adb180b870 0x55adb1833df0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fb73c000c00 tx=0x7fb73c01b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=becc04a10d5acc16 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:20:02.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.966+0000 7fb7297fa700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-05-20T01:20:02.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.966+0000 7fb7297fa700 1 -- 172.21.15.16:0/4129085764 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb73c07a070 con 0x55adb180b870 2023-05-20T01:20:02.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.966+0000 7fb7297fa700 1 -- 172.21.15.16:0/4129085764 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(64..64 src has 1..64) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fb73c057400 con 0x55adb180b870 2023-05-20T01:20:02.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.966+0000 7fb75d15db80 1 -- 172.21.15.16:0/4129085764 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb73c0890c0 con 0x55adb180b870 2023-05-20T01:20:02.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.966+0000 7fb7297fa700 1 -- 172.21.15.16:0/4129085764 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb73c05d040 con 0x55adb180b870 2023-05-20T01:20:02.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.966+0000 7fb75d15db80 10 client.6378 fetch_fsmap learned FSMap version 37 2023-05-20T01:20:02.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.966+0000 7fb75d15db80 1 -- 172.21.15.16:0/4129085764 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=2+}) v3 -- 0x55adb1575c10 con 0x55adb180b870 2023-05-20T01:20:02.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.966+0000 7fb7297fa700 1 -- 172.21.15.16:0/4129085764 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb73c0890c0 con 0x55adb180b870 2023-05-20T01:20:02.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.966+0000 7fb7297fa700 1 -- 172.21.15.16:0/4129085764 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 37) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fb73c07d020 con 0x55adb180b870 2023-05-20T01:20:02.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.966+0000 7fb75d15db80 10 client.6378 fetch_fsmap finished waiting for FSMap version 37 2023-05-20T01:20:02.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.966+0000 7fb75d15db80 10 client.6378 resolve_mds: resolved name 'c' to daemon mds.c 2023-05-20T01:20:02.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.966+0000 7fb75d15db80 20 client.6378 populate_metadata read hostname 'smithi016' 2023-05-20T01:20:02.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.966+0000 7fb75d15db80 1 --2- 172.21.15.16:0/4129085764 >> [v2:172.21.15.16:6835/1427908556,v1:172.21.15.16:6837/1427908556] conn(0x55adb1854690 0x55adb1854a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:20:02.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.966+0000 7fb75d15db80 4 client.6378 mds_command: new command op to 6195 tid=0[{"prefix": "status"}] 2023-05-20T01:20:02.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.966+0000 7fb75d15db80 1 -- 172.21.15.16:0/4129085764 --> [v2:172.21.15.16:6835/1427908556,v1:172.21.15.16:6837/1427908556] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55adb172d610 con 0x55adb1854690 2023-05-20T01:20:02.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.967+0000 7fb749d9b700 1 --2- 172.21.15.16:0/4129085764 >> [v2:172.21.15.16:6835/1427908556,v1:172.21.15.16:6837/1427908556] conn(0x55adb1854690 0x55adb1854a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:20:02.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.967+0000 7fb7297fa700 10 client.6378 ms_handle_connect on v2:172.21.15.16:6835/1427908556 2023-05-20T01:20:02.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.967+0000 7fb749d9b700 1 --2- 172.21.15.16:0/4129085764 >> [v2:172.21.15.16:6835/1427908556,v1:172.21.15.16:6837/1427908556] conn(0x55adb1854690 0x55adb1854a60 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1611013f148df17f server_cookie=454f0b237b8db074 in_seq=0 out_seq=0 2023-05-20T01:20:02.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.967+0000 7fb7297fa700 1 -- 172.21.15.16:0/4129085764 <== mds.0 v2:172.21.15.16:6835/1427908556 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55adb172d610 con 0x55adb1854690 2023-05-20T01:20:02.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.967+0000 7fb7297fa700 10 client.6378 handle_command_reply: tid=0 2023-05-20T01:20:02.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.967+0000 7fb75d15db80 1 client.6378 shutdown 2023-05-20T01:20:02.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.967+0000 7fb75d15db80 1 -- 172.21.15.16:0/4129085764 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55adb180b870 msgr2=0x55adb1833df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:20:02.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.967+0000 7fb75d15db80 1 --2- 172.21.15.16:0/4129085764 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55adb180b870 0x55adb1833df0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fb73c000c00 tx=0x7fb73c01b000 comp rx=0 tx=0).stop 2023-05-20T01:20:02.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.967+0000 7fb75d15db80 1 -- 172.21.15.16:0/4129085764 shutdown_connections 2023-05-20T01:20:02.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.967+0000 7fb75d15db80 1 --2- 172.21.15.16:0/4129085764 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55adb1838400 0x55adb1834ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:20:02.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.967+0000 7fb75d15db80 1 --2- 172.21.15.16:0/4129085764 >> [v2:172.21.15.16:6835/1427908556,v1:172.21.15.16:6837/1427908556] conn(0x55adb1854690 0x55adb1854a60 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:20:02.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.967+0000 7fb75d15db80 1 --2- 172.21.15.16:0/4129085764 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55adb180b870 0x55adb1833df0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:20:02.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.967+0000 7fb7297fa700 0 client.6378 ms_handle_reset on v2:172.21.15.16:6835/1427908556 2023-05-20T01:20:02.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.967+0000 7fb75d15db80 1 --2- 172.21.15.16:0/4129085764 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55adb1834700 0x55adb18b97a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:20:02.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.967+0000 7fb75d15db80 1 -- 172.21.15.16:0/4129085764 >> 172.21.15.16:0/4129085764 conn(0x55adb1802f30 msgr2=0x55adb1853210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:20:02.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.967+0000 7fb75d15db80 1 -- 172.21.15.16:0/4129085764 shutdown_connections 2023-05-20T01:20:02.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.967+0000 7fb75d15db80 1 -- 172.21.15.16:0/4129085764 wait complete. 2023-05-20T01:20:02.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.967+0000 7fb75d15db80 20 client.6378 trim_cache size 0 max 16384 2023-05-20T01:20:02.989 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-05-20T01:20:02.990 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "c2245cc5-ecab-4087-a59b-bc81db901676", 2023-05-20T01:20:02.990 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2023-05-20T01:20:02.990 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 6195, 2023-05-20T01:20:02.990 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2023-05-20T01:20:02.990 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2023-05-20T01:20:02.991 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2023-05-20T01:20:02.991 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 11.051132458, 2023-05-20T01:20:02.991 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 37, 2023-05-20T01:20:02.991 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 64, 2023-05-20T01:20:02.991 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 64, 2023-05-20T01:20:02.991 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 16.387177309999998 2023-05-20T01:20:02.992 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-05-20T01:20:02.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.968+0000 7fb750fd1700 1 -- 172.21.15.16:0/4256045728 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb72c06ace0 msgr2=0x7fb72c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:20:02.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.968+0000 7fb750fd1700 1 --2- 172.21.15.16:0/4256045728 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb72c06ace0 0x7fb72c06d190 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb74c0b5a10 tx=0x7fb740008040 comp rx=0 tx=0).stop 2023-05-20T01:20:02.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.968+0000 7fb750fd1700 1 -- 172.21.15.16:0/4256045728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb74c071900 msgr2=0x7fb74c1f1c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:20:02.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.968+0000 7fb750fd1700 1 --2- 172.21.15.16:0/4256045728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb74c071900 0x7fb74c1f1c30 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fb73c00b4a0 tx=0x7fb73c00b860 comp rx=0 tx=0).stop 2023-05-20T01:20:02.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.968+0000 7fb750fd1700 1 -- 172.21.15.16:0/4256045728 shutdown_connections 2023-05-20T01:20:02.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.968+0000 7fb750fd1700 1 --2- 172.21.15.16:0/4256045728 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb74c157190 0x7fb74c14e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:20:02.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.968+0000 7fb750fd1700 1 --2- 172.21.15.16:0/4256045728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb74c071900 0x7fb74c1f1c30 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:20:02.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.968+0000 7fb750fd1700 1 --2- 172.21.15.16:0/4256045728 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb72c06ace0 0x7fb72c06d190 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:20:02.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.968+0000 7fb750fd1700 1 --2- 172.21.15.16:0/4256045728 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb74c0ccec0 0x7fb74c0ac8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:20:02.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.968+0000 7fb750fd1700 1 -- 172.21.15.16:0/4256045728 >> 172.21.15.16:0/4256045728 conn(0x7fb74c0cffb0 msgr2=0x7fb74c0cc7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:20:02.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.968+0000 7fb750fd1700 1 -- 172.21.15.16:0/4256045728 shutdown_connections 2023-05-20T01:20:02.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:02.968+0000 7fb750fd1700 1 -- 172.21.15.16:0/4256045728 wait complete. 2023-05-20T01:20:02.996 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-05-20T01:20:03.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:03.117+0000 7f293f4e0700 1 Processor -- start 2023-05-20T01:20:03.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:03.117+0000 7f293f4e0700 1 -- start start 2023-05-20T01:20:03.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:03.118+0000 7f293f4e0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f293806f450 0x7f29380cf890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:20:03.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:03.118+0000 7f293f4e0700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f293806d4b0 0x7f2938161080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:20:03.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:03.118+0000 7f293f4e0700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29380cc510 con 0x7f293806a510 2023-05-20T01:20:03.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:03.118+0000 7f293f4e0700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29381615c0 con 0x7f293806d4b0 2023-05-20T01:20:03.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:03.118+0000 7f293f4e0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2938071ed0 con 0x7f293806f450 2023-05-20T01:20:03.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:03.118+0000 7f293da7d700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f293806a510 0x7f29380cdb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:45576/0 (socket says 172.21.15.16:45576) 2023-05-20T01:20:03.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:03.118+0000 7f293ca7b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f293806d4b0 0x7f2938161080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:20:03.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:03.118+0000 7f293da7d700 1 -- 172.21.15.16:0/553370617 learned_addr learned my addr 172.21.15.16:0/553370617 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:20:03.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:03.119+0000 7f293d27c700 1 --2- 172.21.15.16:0/553370617 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f293806f450 0x7f29380cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:20:03.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:03.119+0000 7f292ffff700 1 -- 172.21.15.16:0/553370617 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1986923540 0 0) 0x7f29380cc510 con 0x7f293806a510 2023-05-20T01:20:03.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:03.119+0000 7f292ffff700 1 -- 172.21.15.16:0/553370617 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f29240035b0 con 0x7f293806a510 2023-05-20T01:20:03.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:03.119+0000 7f293ca7b700 1 -- 172.21.15.16:0/553370617 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f293806f450 msgr2=0x7f29380cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:20:03.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:03.119+0000 7f293ca7b700 1 --2- 172.21.15.16:0/553370617 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f293806f450 0x7f29380cf890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:20:03.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:03.119+0000 7f293ca7b700 1 -- 172.21.15.16:0/553370617 >> v1:172.21.15.16:6789/0 conn(0x7f293806a510 legacy=0x7f29380cdb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:20:03.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:03.119+0000 7f293ca7b700 1 -- 172.21.15.16:0/553370617 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29380721b0 con 0x7f293806d4b0 2023-05-20T01:20:03.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:03.119+0000 7f293ca7b700 1 --2- 172.21.15.16:0/553370617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f293806d4b0 0x7f2938161080 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f2934009840 tx=0x7f2934009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88bb06ba3a83778f server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:20:03.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:03.119+0000 7f292ffff700 1 -- 172.21.15.16:0/553370617 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2934011830 con 0x7f293806d4b0 2023-05-20T01:20:03.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:03.119+0000 7f292ffff700 1 -- 172.21.15.16:0/553370617 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2934011990 con 0x7f293806d4b0 2023-05-20T01:20:03.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:03.119+0000 7f292ffff700 1 -- 172.21.15.16:0/553370617 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2934011d20 con 0x7f293806d4b0 2023-05-20T01:20:03.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:03.120+0000 7f293f4e0700 1 -- 172.21.15.16:0/553370617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f293806d4b0 msgr2=0x7f2938161080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:20:03.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:03.120+0000 7f293f4e0700 1 --2- 172.21.15.16:0/553370617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f293806d4b0 0x7f2938161080 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f2934009840 tx=0x7f2934009b50 comp rx=0 tx=0).stop 2023-05-20T01:20:03.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:03.120+0000 7f293f4e0700 1 -- 172.21.15.16:0/553370617 shutdown_connections 2023-05-20T01:20:03.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:03.120+0000 7f293f4e0700 1 --2- 172.21.15.16:0/553370617 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f293806f450 0x7f29380cf890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:20:03.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:03.120+0000 7f293f4e0700 1 --2- 172.21.15.16:0/553370617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f293806d4b0 0x7f2938161080 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:20:03.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:03.120+0000 7f293f4e0700 1 -- 172.21.15.16:0/553370617 >> 172.21.15.16:0/553370617 conn(0x7f29380cffb0 msgr2=0x7f29380d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:20:03.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:03.120+0000 7f293f4e0700 1 -- 172.21.15.16:0/553370617 shutdown_connections 2023-05-20T01:20:03.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:03.120+0000 7f293f4e0700 1 -- 172.21.15.16:0/553370617 wait complete. 2023-05-20T01:20:03.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:03.121+0000 7f293f4e0700 1 Processor -- start 2023-05-20T01:20:03.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:03.121+0000 7f293f4e0700 1 -- start start 2023-05-20T01:20:03.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:03.122+0000 7f293f4e0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f293806a510 0x7f29381f68d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:20:03.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:03.122+0000 7f293f4e0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f293806d4b0 0x7f293815fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:20:03.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:03.122+0000 7f293f4e0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f293806f450 0x7f29380bfc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:20:03.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:03.122+0000 7f293f4e0700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2938160e60 con 0x7f293806a510 2023-05-20T01:20:03.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:03.122+0000 7f293f4e0700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f293815ff40 con 0x7f293806f450 2023-05-20T01:20:03.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:03.122+0000 7f293f4e0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29381601e0 con 0x7f293806d4b0 2023-05-20T01:20:03.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:03.122+0000 7f293d27c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f293806a510 0x7f29381f68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:20:03.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:03.122+0000 7f293d27c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f293806a510 0x7f29381f68d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34050/0 (socket says 172.21.15.16:34050) 2023-05-20T01:20:03.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:03.122+0000 7f293d27c700 1 -- 172.21.15.16:0/1756464247 learned_addr learned my addr 172.21.15.16:0/1756464247 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:20:03.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:03.122+0000 7f293da7d700 1 --2- 172.21.15.16:0/1756464247 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f293806f450 0x7f29380bfc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:20:03.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:03.122+0000 7f293ca7b700 1 --2- 172.21.15.16:0/1756464247 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f293806d4b0 0x7f293815fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:20:03.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:03.122+0000 7f293d27c700 1 -- 172.21.15.16:0/1756464247 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f293806d4b0 msgr2=0x7f293815fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:20:03.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:03.122+0000 7f293d27c700 1 --2- 172.21.15.16:0/1756464247 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f293806d4b0 0x7f293815fa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:20:03.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:03.122+0000 7f293d27c700 1 -- 172.21.15.16:0/1756464247 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f293806f450 msgr2=0x7f29380bfc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:20:03.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:03.122+0000 7f293d27c700 1 --2- 172.21.15.16:0/1756464247 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f293806f450 0x7f29380bfc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:20:03.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:03.122+0000 7f293d27c700 1 -- 172.21.15.16:0/1756464247 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29340094a0 con 0x7f293806a510 2023-05-20T01:20:03.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:03.122+0000 7f293da7d700 1 --2- 172.21.15.16:0/1756464247 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f293806f450 0x7f29380bfc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:20:03.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:03.123+0000 7f293d27c700 1 --2- 172.21.15.16:0/1756464247 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f293806a510 0x7f29381f68d0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f292800e870 tx=0x7f292800ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:20:03.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:03.123+0000 7f292dffb700 1 -- 172.21.15.16:0/1756464247 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2928009590 con 0x7f293806a510 2023-05-20T01:20:03.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:03.123+0000 7f292dffb700 1 -- 172.21.15.16:0/1756464247 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f292800ee70 con 0x7f293806a510 2023-05-20T01:20:03.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:03.123+0000 7f293f4e0700 1 -- 172.21.15.16:0/1756464247 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29380cbe50 con 0x7f293806a510 2023-05-20T01:20:03.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:03.123+0000 7f292dffb700 1 -- 172.21.15.16:0/1756464247 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f292800ca50 con 0x7f293806a510 2023-05-20T01:20:03.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:03.123+0000 7f293f4e0700 1 -- 172.21.15.16:0/1756464247 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29380cc1a0 con 0x7f293806a510 2023-05-20T01:20:03.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:03.124+0000 7f293f4e0700 1 -- 172.21.15.16:0/1756464247 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f293806a9b0 con 0x7f293806a510 2023-05-20T01:20:03.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:03.124+0000 7f292dffb700 1 -- 172.21.15.16:0/1756464247 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f292800cbb0 con 0x7f293806a510 2023-05-20T01:20:03.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:03.125+0000 7f292dffb700 1 --2- 172.21.15.16:0/1756464247 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f291c06ace0 0x7f291c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:20:03.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:03.125+0000 7f292dffb700 1 -- 172.21.15.16:0/1756464247 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f2928089540 con 0x7f293806a510 2023-05-20T01:20:03.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:03.125+0000 7f293ca7b700 1 --2- 172.21.15.16:0/1756464247 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f291c06ace0 0x7f291c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:20:03.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:03.125+0000 7f293ca7b700 1 --2- 172.21.15.16:0/1756464247 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f291c06ace0 0x7f291c06d190 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f29380b2c00 tx=0x7f2934008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:20:03.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:03.129+0000 7f292dffb700 1 -- 172.21.15.16:0/1756464247 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f2928057fd0 con 0x7f293806a510 2023-05-20T01:20:03.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:03.371+0000 7f293f4e0700 1 -- 172.21.15.16:0/1756464247 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f29380747b0 con 0x7f293806a510 2023-05-20T01:20:03.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:03.372+0000 7f292dffb700 1 -- 172.21.15.16:0/1756464247 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v37) v1 ==== 53+0+73 (secure 0 0 0) 0x7f292800ce60 con 0x7f293806a510 2023-05-20T01:20:03.376 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-05-20T01:20:03.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:03.374+0000 7f293f4e0700 1 -- 172.21.15.16:0/1756464247 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f291c06ace0 msgr2=0x7f291c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:20:03.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:03.374+0000 7f293f4e0700 1 --2- 172.21.15.16:0/1756464247 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f291c06ace0 0x7f291c06d190 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f29380b2c00 tx=0x7f2934008030 comp rx=0 tx=0).stop 2023-05-20T01:20:03.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:03.374+0000 7f293f4e0700 1 -- 172.21.15.16:0/1756464247 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f293806a510 msgr2=0x7f29381f68d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:20:03.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:03.374+0000 7f293f4e0700 1 --2- 172.21.15.16:0/1756464247 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f293806a510 0x7f29381f68d0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f292800e870 tx=0x7f292800ec30 comp rx=0 tx=0).stop 2023-05-20T01:20:03.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:03.374+0000 7f293f4e0700 1 -- 172.21.15.16:0/1756464247 shutdown_connections 2023-05-20T01:20:03.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:03.374+0000 7f293f4e0700 1 --2- 172.21.15.16:0/1756464247 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f293806f450 0x7f29380bfc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:20:03.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:03.374+0000 7f293f4e0700 1 --2- 172.21.15.16:0/1756464247 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f293806a510 0x7f29381f68d0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:20:03.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:03.374+0000 7f293f4e0700 1 --2- 172.21.15.16:0/1756464247 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f291c06ace0 0x7f291c06d190 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:20:03.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:03.374+0000 7f293f4e0700 1 --2- 172.21.15.16:0/1756464247 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f293806d4b0 0x7f293815fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:20:03.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:03.374+0000 7f293f4e0700 1 -- 172.21.15.16:0/1756464247 >> 172.21.15.16:0/1756464247 conn(0x7f29380cffb0 msgr2=0x7f29380c95e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:20:03.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:03.374+0000 7f293f4e0700 1 -- 172.21.15.16:0/1756464247 shutdown_connections 2023-05-20T01:20:03.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:03.374+0000 7f293f4e0700 1 -- 172.21.15.16:0/1756464247 wait complete. 2023-05-20T01:20:03.390 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-05-20T01:20:03.390 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-05-20T01:20:03.390 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2023-05-20T01:20:03.391 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-05-20T01:20:03.391 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-05-20T01:20:03.391 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-05-20T01:20:03.391 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-05-20T01:20:03.392 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-05-20T01:20:03.392 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-05-20T01:20:03.392 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:20:03.393 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2023-05-20T01:20:03.410 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-05-20T01:20:03.410 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-05-20T01:20:03.411 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2023-05-20T01:20:03.411 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-05-20T01:20:03.411 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2023-05-20T01:20:03.411 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-05-20T01:20:03.411 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-05-20T01:20:03.411 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2023-05-20T01:20:03.411 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-05-20T01:20:03.412 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2023-05-20T01:20:03.412 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-05-20T01:20:03.412 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2023-05-20T01:20:03.412 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-05-20T01:20:03.412 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 42230sec preferred_lft 42230sec 2023-05-20T01:20:03.412 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link 2023-05-20T01:20:03.413 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-05-20T01:20:03.413 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-05-20T01:20:03.413 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2023-05-20T01:20:03.413 INFO:teuthology.orchestra.run.smithi016.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-05-20T01:20:03.413 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0a:2e:0c:6b:5d:42 brd ff:ff:ff:ff:ff:ff 2023-05-20T01:20:03.413 INFO:teuthology.orchestra.run.smithi016.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-05-20T01:20:03.413 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-05-20T01:20:03.414 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::7c00:b2ff:fef4:946/64 scope link 2023-05-20T01:20:03.414 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-05-20T01:20:03.414 INFO:teuthology.orchestra.run.smithi016.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-05-20T01:20:03.414 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether d2:e4:70:a7:b9:4b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-05-20T01:20:03.414 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::d0e4:70ff:fea7:b94b/64 scope link 2023-05-20T01:20:03.414 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-05-20T01:20:03.414 INFO:teuthology.orchestra.run.smithi016.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-05-20T01:20:03.415 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0a:2e:0c:6b:5d:42 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-05-20T01:20:03.415 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::82e:cff:fe6b:5d42/64 scope link 2023-05-20T01:20:03.415 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-05-20T01:20:03.415 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:20:03.416 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-05-20T01:20:03.477 INFO:teuthology.orchestra.run.smithi016.stdout:1 2023-05-20T01:20:03.478 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:20:03.479 DEBUG:teuthology.orchestra.run.smithi016:> route 2023-05-20T01:20:03.539 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2023-05-20T01:20:03.540 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-05-20T01:20:03.540 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-05-20T01:20:03.540 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-05-20T01:20:03.541 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-05-20T01:20:03.542 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:20:03.543 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2023-05-20T01:20:03.543 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2023-05-20T01:20:03.544 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-05-20T01:20:03.544 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-05-20T01:20:03.544 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-05-20T01:20:03.545 DEBUG:teuthology.orchestra.run.smithi016:> ') 2023-05-20T01:20:03.647 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:20:03.647 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2023-05-20T01:20:03.664 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-05-20T01:20:03.664 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-05-20T01:20:03.665 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-05-20T01:20:03.665 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:20:03.665 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.0 up 2023-05-20T01:20:03.732 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-05-20T01:20:03.732 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-05-20T01:20:03.789 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-05-20T01:20:03.789 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-05-20T01:20:03.789 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-05-20T01:20:03.845 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-05-20T01:20:03.846 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:20:03.846 DEBUG:teuthology.orchestra.run.smithi016:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-05-20T01:20:03.932 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-05-20T01:20:03.932 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-05-20T01:20:03.964 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-05-20T01:20:03.964 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-05-20T01:20:03.965 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.132:3300,172.21.15.132:3301" 2023-05-20T01:20:03.965 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: trying mount with new device syntax: 0@c2245cc5-ecab-4087-a59b-bc81db901676.cephfs=/ 2023-05-20T01:20:03.965 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.16:3300/172.21.15.132:3300/172.21.15.132:3301" will pass to kernel 2023-05-20T01:20:03.965 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: v2 mount support check returned 2 2023-05-20T01:20:03.965 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: kernel does not support v2 syntax 2023-05-20T01:20:03.965 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: switching to using v1 address with old syntax 2023-05-20T01:20:04.038 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-05-20T01:20:04.038 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-05-20T01:20:04.038 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.132:3300,172.21.15.132:3301" 2023-05-20T01:20:04.038 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: trying mount with new device syntax: 0@c2245cc5-ecab-4087-a59b-bc81db901676.cephfs=/ 2023-05-20T01:20:04.039 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.16:3300/172.21.15.132:3300/172.21.15.132:3301" will pass to kernel 2023-05-20T01:20:04.039 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: v2 mount support check returned 2 2023-05-20T01:20:04.039 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: kernel does not support v2 syntax 2023-05-20T01:20:04.039 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: switching to using v1 address with old syntax 2023-05-20T01:20:04.039 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:6789,172.21.15.132:6789,172.21.15.132:6790" 2023-05-20T01:20:04.039 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.16:6789,172.21.15.132:6789,172.21.15.132:6790:/ 2023-05-20T01:20:04.040 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=c2245cc5-ecab-4087-a59b-bc81db901676" will pass to kernel 2023-05-20T01:20:04.041 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-05-20T01:20:04.041 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:20:04.042 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-05-20T01:20:04.069 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:20:04.070 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-05-20T01:20:04.155 INFO:teuthology.orchestra.run.smithi016.stdout:client5594 2023-05-20T01:20:04.156 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-05-20T01:20:04.156 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2023-05-20T01:20:04.173 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_extra --pg_num_min 64 2023-05-20T01:20:04.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:04.338+0000 7efdb4683700 1 Processor -- start 2023-05-20T01:20:04.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:04.338+0000 7efdb4683700 1 -- start start 2023-05-20T01:20:04.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:04.340+0000 7efdb4683700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7efdac0c95c0 0x7efdac15cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:20:04.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:04.340+0000 7efdb4683700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdac0c9d30 con 0x7efdac15d180 2023-05-20T01:20:04.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:04.340+0000 7efdb4683700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdac0c9a50 con 0x7efdac1591f0 2023-05-20T01:20:04.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:04.340+0000 7efdb4683700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdac15d600 con 0x7efdac0c95c0 2023-05-20T01:20:04.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:04.340+0000 7efdb2c20700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7efdac15d180 0x7efdac0702e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:45594/0 (socket says 172.21.15.16:45594) 2023-05-20T01:20:04.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:04.340+0000 7efdb2c20700 1 -- 172.21.15.16:0/3976604174 learned_addr learned my addr 172.21.15.16:0/3976604174 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:20:04.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:04.340+0000 7efdb1c1e700 1 --2- 172.21.15.16:0/3976604174 >> v2:172.21.15.132:3300/0 conn(0x7efdac0c95c0 0x7efdac15cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:20:04.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:04.340+0000 7efdb141d700 1 -- 172.21.15.16:0/3976604174 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3542056378 0 0) 0x7efdac0c9d30 con 0x7efdac15d180 2023-05-20T01:20:04.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:04.340+0000 7efdb141d700 1 -- 172.21.15.16:0/3976604174 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd980035b0 con 0x7efdac15d180 2023-05-20T01:20:04.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:04.340+0000 7efdb141d700 1 -- 172.21.15.16:0/3976604174 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2709496907 0 0) 0x7efd980035b0 con 0x7efdac15d180 2023-05-20T01:20:04.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:04.340+0000 7efdb141d700 1 -- 172.21.15.16:0/3976604174 >> v2:172.21.15.132:3300/0 conn(0x7efdac0c95c0 msgr2=0x7efdac15cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:20:04.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:04.340+0000 7efdb141d700 1 --2- 172.21.15.16:0/3976604174 >> v2:172.21.15.132:3300/0 conn(0x7efdac0c95c0 0x7efdac15cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:20:04.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:04.340+0000 7efdb141d700 1 -- 172.21.15.16:0/3976604174 >> v1:172.21.15.132:6789/0 conn(0x7efdac1591f0 legacy=0x7efdac0cb1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:20:04.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:04.340+0000 7efdb141d700 1 -- 172.21.15.16:0/3976604174 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdac06cdb0 con 0x7efdac15d180 2023-05-20T01:20:04.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:04.340+0000 7efdb141d700 1 -- 172.21.15.16:0/3976604174 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7efda40019f0 con 0x7efdac15d180 2023-05-20T01:20:04.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:04.340+0000 7efdb141d700 1 -- 172.21.15.16:0/3976604174 >> v1:172.21.15.16:6789/0 conn(0x7efdac15d180 legacy=0x7efdac0702e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:20:04.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:04.341+0000 7efdb141d700 1 --2- 172.21.15.16:0/3976604174 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efd98003a00 0x7efd98003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:20:04.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:04.341+0000 7efdb141d700 1 --2- 172.21.15.16:0/3976604174 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efd980043f0 0x7efd980048b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:20:04.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:04.341+0000 7efdb141d700 1 --2- 172.21.15.16:0/3976604174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd980050b0 0x7efd9800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:20:04.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:04.341+0000 7efdb141d700 1 -- 172.21.15.16:0/3976604174 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdac15d600 con 0x7efd980050b0 2023-05-20T01:20:04.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:04.341+0000 7efdb141d700 1 -- 172.21.15.16:0/3976604174 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd9800bf50 con 0x7efd98003a00 2023-05-20T01:20:04.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:04.341+0000 7efdb141d700 1 -- 172.21.15.16:0/3976604174 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd9800c110 con 0x7efd980043f0 2023-05-20T01:20:04.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:04.341+0000 7efdb2c20700 1 --2- 172.21.15.16:0/3976604174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd980050b0 0x7efd9800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:20:04.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:04.341+0000 7efdb241f700 1 --2- 172.21.15.16:0/3976604174 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efd98003a00 0x7efd98003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:20:04.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:04.341+0000 7efdb1c1e700 1 --2- 172.21.15.16:0/3976604174 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efd980043f0 0x7efd980048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:20:04.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:04.341+0000 7efdb2c20700 1 -- 172.21.15.16:0/3976604174 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efd980043f0 msgr2=0x7efd980048b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:20:04.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:04.341+0000 7efdb2c20700 1 --2- 172.21.15.16:0/3976604174 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efd980043f0 0x7efd980048b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:20:04.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:04.341+0000 7efdb2c20700 1 -- 172.21.15.16:0/3976604174 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efd98003a00 msgr2=0x7efd98003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:20:04.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:04.341+0000 7efdb2c20700 1 --2- 172.21.15.16:0/3976604174 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efd98003a00 0x7efd98003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:20:04.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:04.341+0000 7efdb2c20700 1 -- 172.21.15.16:0/3976604174 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd980035b0 con 0x7efd980050b0 2023-05-20T01:20:04.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:04.341+0000 7efdb1c1e700 1 --2- 172.21.15.16:0/3976604174 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efd980043f0 0x7efd980048b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:20:04.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:04.341+0000 7efdb2c20700 1 -- 172.21.15.16:0/3976604174 reap_dead start 2023-05-20T01:20:04.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:04.341+0000 7efdb2c20700 1 --2- 172.21.15.16:0/3976604174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd980050b0 0x7efd9800ba10 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7efda400a830 tx=0x7efd98005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b559b843553c9539 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:20:04.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:04.341+0000 7efdb141d700 1 -- 172.21.15.16:0/3976604174 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efda4012600 con 0x7efd980050b0 2023-05-20T01:20:04.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:04.341+0000 7efdb141d700 1 -- 172.21.15.16:0/3976604174 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efda4012780 con 0x7efd980050b0 2023-05-20T01:20:04.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:04.341+0000 7efdb141d700 1 -- 172.21.15.16:0/3976604174 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efda4012b30 con 0x7efd980050b0 2023-05-20T01:20:04.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:04.342+0000 7efdb4683700 1 -- 172.21.15.16:0/3976604174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd980050b0 msgr2=0x7efd9800ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:20:04.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:04.342+0000 7efdb4683700 1 --2- 172.21.15.16:0/3976604174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd980050b0 0x7efd9800ba10 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7efda400a830 tx=0x7efd98005880 comp rx=0 tx=0).stop 2023-05-20T01:20:04.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:04.342+0000 7efdb4683700 1 -- 172.21.15.16:0/3976604174 shutdown_connections 2023-05-20T01:20:04.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:04.342+0000 7efdb4683700 1 --2- 172.21.15.16:0/3976604174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd980050b0 0x7efd9800ba10 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:20:04.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:04.342+0000 7efdb4683700 1 -- 172.21.15.16:0/3976604174 >> 172.21.15.16:0/3976604174 conn(0x7efdac0cb9f0 msgr2=0x7efdac0cbdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:20:04.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:04.342+0000 7efdb4683700 1 -- 172.21.15.16:0/3976604174 shutdown_connections 2023-05-20T01:20:04.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:04.342+0000 7efdb4683700 1 -- 172.21.15.16:0/3976604174 wait complete. 2023-05-20T01:20:04.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:04.344+0000 7efdb4683700 1 Processor -- start 2023-05-20T01:20:04.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:04.344+0000 7efdb4683700 1 -- start start 2023-05-20T01:20:04.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:04.344+0000 7efdb4683700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efd980050b0 0x7efdac15bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:20:04.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:04.344+0000 7efdb4683700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efdac099a20 0x7efdac09b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:20:04.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:04.344+0000 7efdb4683700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efdac09ba70 0x7efdac0cef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:20:04.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:04.344+0000 7efdb4683700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdac09bf80 con 0x7efdac099a20 2023-05-20T01:20:04.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:04.344+0000 7efdb4683700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdac09c260 con 0x7efd980050b0 2023-05-20T01:20:04.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:04.344+0000 7efdb4683700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdac09c540 con 0x7efdac09ba70 2023-05-20T01:20:04.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:04.344+0000 7efdb1c1e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efdac099a20 0x7efdac09b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:20:04.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:04.344+0000 7efdb1c1e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efdac099a20 0x7efdac09b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34076/0 (socket says 172.21.15.16:34076) 2023-05-20T01:20:04.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:04.344+0000 7efdb1c1e700 1 -- 172.21.15.16:0/3492180634 learned_addr learned my addr 172.21.15.16:0/3492180634 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:20:04.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:04.345+0000 7efdb241f700 1 --2- 172.21.15.16:0/3492180634 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efd980050b0 0x7efdac15bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:20:04.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:04.345+0000 7efdb2c20700 1 --2- 172.21.15.16:0/3492180634 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efdac09ba70 0x7efdac0cef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:20:04.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:04.345+0000 7efdb1c1e700 1 -- 172.21.15.16:0/3492180634 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efdac09ba70 msgr2=0x7efdac0cef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:20:04.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:04.345+0000 7efdb1c1e700 1 --2- 172.21.15.16:0/3492180634 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efdac09ba70 0x7efdac0cef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:20:04.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:04.345+0000 7efdb1c1e700 1 -- 172.21.15.16:0/3492180634 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efd980050b0 msgr2=0x7efdac15bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:20:04.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:04.345+0000 7efdb1c1e700 1 --2- 172.21.15.16:0/3492180634 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efd980050b0 0x7efdac15bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:20:04.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:04.345+0000 7efdb1c1e700 1 -- 172.21.15.16:0/3492180634 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdac06d370 con 0x7efdac099a20 2023-05-20T01:20:04.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:04.345+0000 7efdb2c20700 1 --2- 172.21.15.16:0/3492180634 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efdac09ba70 0x7efdac0cef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:20:04.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:04.345+0000 7efdb1c1e700 1 --2- 172.21.15.16:0/3492180634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efdac099a20 0x7efdac09b530 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7efda800a2f0 tx=0x7efda8010a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:20:04.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:04.346+0000 7efda2ffd700 1 -- 172.21.15.16:0/3492180634 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efda8004220 con 0x7efdac099a20 2023-05-20T01:20:04.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:04.346+0000 7efda2ffd700 1 -- 172.21.15.16:0/3492180634 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efda8004380 con 0x7efdac099a20 2023-05-20T01:20:04.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:04.346+0000 7efdb4683700 1 -- 172.21.15.16:0/3492180634 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdac06d090 con 0x7efdac099a20 2023-05-20T01:20:04.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:04.346+0000 7efda2ffd700 1 -- 172.21.15.16:0/3492180634 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efda80036a0 con 0x7efdac099a20 2023-05-20T01:20:04.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:04.346+0000 7efdb4683700 1 -- 172.21.15.16:0/3492180634 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdac15da30 con 0x7efdac099a20 2023-05-20T01:20:04.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:04.347+0000 7efdb4683700 1 -- 172.21.15.16:0/3492180634 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdac04e8c0 con 0x7efdac099a20 2023-05-20T01:20:04.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:04.347+0000 7efda2ffd700 1 -- 172.21.15.16:0/3492180634 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7efda8003800 con 0x7efdac099a20 2023-05-20T01:20:04.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:04.347+0000 7efda2ffd700 1 --2- 172.21.15.16:0/3492180634 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7efd9006ace0 0x7efd9006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:20:04.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:04.348+0000 7efda2ffd700 1 -- 172.21.15.16:0/3492180634 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6382+0+0 (secure 0 0 0) 0x7efda8089420 con 0x7efdac099a20 2023-05-20T01:20:04.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:04.348+0000 7efdb241f700 1 --2- 172.21.15.16:0/3492180634 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7efd9006ace0 0x7efd9006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:20:04.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:04.348+0000 7efdb241f700 1 --2- 172.21.15.16:0/3492180634 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7efd9006ace0 0x7efd9006d190 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7efd9c007fd0 tx=0x7efd9c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:20:04.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:04.352+0000 7efda2ffd700 1 -- 172.21.15.16:0/3492180634 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7efda8057eb0 con 0x7efdac099a20 2023-05-20T01:20:04.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:04.551+0000 7efdb4683700 1 -- 172.21.15.16:0/3492180634 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_extra", "pg_num_min": 64} v 0) v1 -- 0x7efdac0c93a0 con 0x7efdac099a20 2023-05-20T01:20:05.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:05.433+0000 7efda2ffd700 1 -- 172.21.15.16:0/3492180634 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_extra", "pg_num_min": 64}]=0 pool 'cephfs_data_extra' created v65) v1 ==== 142+0+0 (secure 0 0 0) 0x7efda80220b0 con 0x7efdac099a20 2023-05-20T01:20:05.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:05.438+0000 7efdb4683700 1 -- 172.21.15.16:0/3492180634 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7efd9006ace0 msgr2=0x7efd9006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:20:05.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:05.438+0000 7efdb4683700 1 --2- 172.21.15.16:0/3492180634 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7efd9006ace0 0x7efd9006d190 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7efd9c007fd0 tx=0x7efd9c009040 comp rx=0 tx=0).stop 2023-05-20T01:20:05.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:05.438+0000 7efdb4683700 1 -- 172.21.15.16:0/3492180634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efdac099a20 msgr2=0x7efdac09b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:20:05.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:05.438+0000 7efdb4683700 1 --2- 172.21.15.16:0/3492180634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efdac099a20 0x7efdac09b530 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7efda800a2f0 tx=0x7efda8010a10 comp rx=0 tx=0).stop 2023-05-20T01:20:05.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:05.438+0000 7efdb4683700 1 -- 172.21.15.16:0/3492180634 shutdown_connections 2023-05-20T01:20:05.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:05.438+0000 7efdb4683700 1 --2- 172.21.15.16:0/3492180634 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7efd9006ace0 0x7efd9006d190 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:20:05.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:05.438+0000 7efdb4683700 1 --2- 172.21.15.16:0/3492180634 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efdac09ba70 0x7efdac0cef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:20:05.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:05.438+0000 7efdb4683700 1 --2- 172.21.15.16:0/3492180634 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efd980050b0 0x7efdac15bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:20:05.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:05.438+0000 7efdb4683700 1 --2- 172.21.15.16:0/3492180634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efdac099a20 0x7efdac09b530 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:20:05.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:05.438+0000 7efdb4683700 1 -- 172.21.15.16:0/3492180634 >> 172.21.15.16:0/3492180634 conn(0x7efdac0cb9f0 msgr2=0x7efdac0add20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:20:05.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:05.439+0000 7efdb4683700 1 -- 172.21.15.16:0/3492180634 shutdown_connections 2023-05-20T01:20:05.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:05.441+0000 7efdb4683700 1 -- 172.21.15.16:0/3492180634 wait complete. 2023-05-20T01:20:05.446 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data_extra' created 2023-05-20T01:20:05.460 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_extra 2023-05-20T01:20:05.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:05.590+0000 7f390f1d0700 1 Processor -- start 2023-05-20T01:20:05.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:05.591+0000 7f390f1d0700 1 -- start start 2023-05-20T01:20:05.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:05.591+0000 7f390f1d0700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f39080cec80 0x7f39080cf0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:20:05.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:05.591+0000 7f390f1d0700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f39080cc110 0x7f39080cc550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:20:05.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:05.591+0000 7f390f1d0700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39080ce4d0 con 0x7f390815ab80 2023-05-20T01:20:05.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:05.591+0000 7f390f1d0700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39080cb700 con 0x7f39080cc110 2023-05-20T01:20:05.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:05.591+0000 7f390f1d0700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39080cf630 con 0x7f39080cec80 2023-05-20T01:20:05.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:05.592+0000 7f390d76d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f39080cc110 0x7f39080cc550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:20:05.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:05.592+0000 7f390d76d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f39080cc110 0x7f39080cc550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34086/0 (socket says 172.21.15.16:34086) 2023-05-20T01:20:05.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:05.592+0000 7f390d76d700 1 -- 172.21.15.16:0/1753904333 learned_addr learned my addr 172.21.15.16:0/1753904333 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:20:05.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:05.592+0000 7f3907fff700 1 --2- 172.21.15.16:0/1753904333 >> v2:172.21.15.132:3300/0 conn(0x7f39080cec80 0x7f39080cf0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:20:05.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:05.592+0000 7f390d76d700 1 -- 172.21.15.16:0/1753904333 >> v2:172.21.15.132:3300/0 conn(0x7f39080cec80 msgr2=0x7f39080cf0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:20:05.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:05.592+0000 7f390d76d700 1 --2- 172.21.15.16:0/1753904333 >> v2:172.21.15.132:3300/0 conn(0x7f39080cec80 0x7f39080cf0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:20:05.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:05.592+0000 7f390d76d700 1 -- 172.21.15.16:0/1753904333 >> v1:172.21.15.132:6789/0 conn(0x7f390815ab80 legacy=0x7f39080cda80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:20:05.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:05.592+0000 7f390d76d700 1 -- 172.21.15.16:0/1753904333 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39080ce7b0 con 0x7f39080cc110 2023-05-20T01:20:05.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:05.592+0000 7f3907fff700 1 --2- 172.21.15.16:0/1753904333 >> v2:172.21.15.132:3300/0 conn(0x7f39080cec80 0x7f39080cf0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:20:05.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:05.592+0000 7f390d76d700 1 --2- 172.21.15.16:0/1753904333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39080cc110 0x7f39080cc550 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f39080b3bb0 tx=0x7f390000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4823664bf795c690 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:20:05.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:05.592+0000 7f39077fe700 1 -- 172.21.15.16:0/1753904333 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f390000e070 con 0x7f39080cc110 2023-05-20T01:20:05.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:05.592+0000 7f39077fe700 1 -- 172.21.15.16:0/1753904333 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f390000b9c0 con 0x7f39080cc110 2023-05-20T01:20:05.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:05.592+0000 7f39077fe700 1 -- 172.21.15.16:0/1753904333 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f390000bd70 con 0x7f39080cc110 2023-05-20T01:20:05.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:05.593+0000 7f390f1d0700 1 -- 172.21.15.16:0/1753904333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39080cc110 msgr2=0x7f39080cc550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:20:05.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:05.593+0000 7f390f1d0700 1 --2- 172.21.15.16:0/1753904333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39080cc110 0x7f39080cc550 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f39080b3bb0 tx=0x7f390000b330 comp rx=0 tx=0).stop 2023-05-20T01:20:05.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:05.593+0000 7f390f1d0700 1 -- 172.21.15.16:0/1753904333 shutdown_connections 2023-05-20T01:20:05.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:05.593+0000 7f390f1d0700 1 --2- 172.21.15.16:0/1753904333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39080cc110 0x7f39080cc550 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:20:05.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:05.593+0000 7f390f1d0700 1 --2- 172.21.15.16:0/1753904333 >> v2:172.21.15.132:3300/0 conn(0x7f39080cec80 0x7f39080cf0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:20:05.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:05.593+0000 7f390f1d0700 1 -- 172.21.15.16:0/1753904333 >> 172.21.15.16:0/1753904333 conn(0x7f390806ae30 msgr2=0x7f390809a400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:20:05.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:05.593+0000 7f390f1d0700 1 -- 172.21.15.16:0/1753904333 shutdown_connections 2023-05-20T01:20:05.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:05.593+0000 7f390f1d0700 1 -- 172.21.15.16:0/1753904333 wait complete. 2023-05-20T01:20:05.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:05.594+0000 7f390f1d0700 1 Processor -- start 2023-05-20T01:20:05.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:05.594+0000 7f390f1d0700 1 -- start start 2023-05-20T01:20:05.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:05.595+0000 7f390f1d0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f39080cc110 0x7f39080ca5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:20:05.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:05.595+0000 7f390f1d0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f39080cec80 0x7f39080b0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:20:05.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:05.595+0000 7f390f1d0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f390815ab80 0x7f39080b11d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:20:05.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:05.595+0000 7f390f1d0700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39080b21f0 con 0x7f390815ab80 2023-05-20T01:20:05.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:05.595+0000 7f390f1d0700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39080b2440 con 0x7f39080cec80 2023-05-20T01:20:05.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:05.595+0000 7f390f1d0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39080b2720 con 0x7f39080cc110 2023-05-20T01:20:05.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:05.595+0000 7f390d76d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f390815ab80 0x7f39080b11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:20:05.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:05.595+0000 7f390d76d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f390815ab80 0x7f39080b11d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34092/0 (socket says 172.21.15.16:34092) 2023-05-20T01:20:05.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:05.595+0000 7f390d76d700 1 -- 172.21.15.16:0/647788480 learned_addr learned my addr 172.21.15.16:0/647788480 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:20:05.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:05.595+0000 7f3907fff700 1 --2- 172.21.15.16:0/647788480 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f39080cec80 0x7f39080b0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:20:05.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:05.595+0000 7f390cf6c700 1 --2- 172.21.15.16:0/647788480 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f39080cc110 0x7f39080ca5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:20:05.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:05.595+0000 7f390d76d700 1 -- 172.21.15.16:0/647788480 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f39080cc110 msgr2=0x7f39080ca5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:20:05.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:05.595+0000 7f390d76d700 1 --2- 172.21.15.16:0/647788480 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f39080cc110 0x7f39080ca5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:20:05.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:05.595+0000 7f390d76d700 1 -- 172.21.15.16:0/647788480 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f39080cec80 msgr2=0x7f39080b0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:20:05.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:05.595+0000 7f390d76d700 1 --2- 172.21.15.16:0/647788480 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f39080cec80 0x7f39080b0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:20:05.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:05.595+0000 7f390d76d700 1 -- 172.21.15.16:0/647788480 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3900009b30 con 0x7f390815ab80 2023-05-20T01:20:05.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:05.595+0000 7f3907fff700 1 --2- 172.21.15.16:0/647788480 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f39080cec80 0x7f39080b0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:20:05.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:05.596+0000 7f390d76d700 1 --2- 172.21.15.16:0/647788480 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f390815ab80 0x7f39080b11d0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f39080cec30 tx=0x7f3900004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:20:05.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:05.596+0000 7f39057fa700 1 -- 172.21.15.16:0/647788480 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39000099b0 con 0x7f390815ab80 2023-05-20T01:20:05.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:05.596+0000 7f39057fa700 1 -- 172.21.15.16:0/647788480 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39000092e0 con 0x7f390815ab80 2023-05-20T01:20:05.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:05.596+0000 7f390f1d0700 1 -- 172.21.15.16:0/647788480 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39080cbbf0 con 0x7f390815ab80 2023-05-20T01:20:05.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:05.596+0000 7f39057fa700 1 -- 172.21.15.16:0/647788480 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f390001b650 con 0x7f390815ab80 2023-05-20T01:20:05.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:05.596+0000 7f390f1d0700 1 -- 172.21.15.16:0/647788480 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39080cb910 con 0x7f390815ab80 2023-05-20T01:20:05.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:05.597+0000 7f390f1d0700 1 -- 172.21.15.16:0/647788480 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39080ce190 con 0x7f390815ab80 2023-05-20T01:20:05.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:05.598+0000 7f39057fa700 1 -- 172.21.15.16:0/647788480 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f390001b7b0 con 0x7f390815ab80 2023-05-20T01:20:05.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:05.598+0000 7f39057fa700 1 --2- 172.21.15.16:0/647788480 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f38ec06ace0 0x7f38ec06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:20:05.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:05.598+0000 7f39057fa700 1 -- 172.21.15.16:0/647788480 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6754+0+0 (secure 0 0 0) 0x7f390008afd0 con 0x7f390815ab80 2023-05-20T01:20:05.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:05.598+0000 7f390cf6c700 1 --2- 172.21.15.16:0/647788480 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f38ec06ace0 0x7f38ec06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:20:05.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:05.599+0000 7f390cf6c700 1 --2- 172.21.15.16:0/647788480 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f38ec06ace0 0x7f38ec06d190 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f38fc000f60 tx=0x7f38fc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:20:05.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:05.602+0000 7f39057fa700 1 -- 172.21.15.16:0/647788480 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f3900056050 con 0x7f390815ab80 2023-05-20T01:20:05.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:05.848+0000 7f390f1d0700 1 -- 172.21.15.16:0/647788480 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_extra"} v 0) v1 -- 0x7f39080747b0 con 0x7f390815ab80 2023-05-20T01:20:05.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:05.853+0000 7f39057fa700 1 -- 172.21.15.16:0/647788480 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_extra"}]=0 added data pool 11 to fsmap v38) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3900059670 con 0x7f390815ab80 2023-05-20T01:20:05.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:05.856+0000 7f390f1d0700 1 -- 172.21.15.16:0/647788480 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f38ec06ace0 msgr2=0x7f38ec06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:20:05.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:05.856+0000 7f390f1d0700 1 --2- 172.21.15.16:0/647788480 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f38ec06ace0 0x7f38ec06d190 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f38fc000f60 tx=0x7f38fc009040 comp rx=0 tx=0).stop 2023-05-20T01:20:05.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:05.856+0000 7f390f1d0700 1 -- 172.21.15.16:0/647788480 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f390815ab80 msgr2=0x7f39080b11d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:20:05.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:05.856+0000 7f390f1d0700 1 --2- 172.21.15.16:0/647788480 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f390815ab80 0x7f39080b11d0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f39080cec30 tx=0x7f3900004ad0 comp rx=0 tx=0).stop 2023-05-20T01:20:05.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:05.856+0000 7f390f1d0700 1 -- 172.21.15.16:0/647788480 shutdown_connections 2023-05-20T01:20:05.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:05.856+0000 7f390f1d0700 1 --2- 172.21.15.16:0/647788480 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f390815ab80 0x7f39080b11d0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:20:05.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:05.856+0000 7f390f1d0700 1 --2- 172.21.15.16:0/647788480 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f38ec06ace0 0x7f38ec06d190 secure :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f38fc000f60 tx=0x7f38fc009040 comp rx=0 tx=0).stop 2023-05-20T01:20:05.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:05.856+0000 7f390f1d0700 1 --2- 172.21.15.16:0/647788480 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f39080cc110 0x7f39080ca5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:20:05.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:05.856+0000 7f390f1d0700 1 --2- 172.21.15.16:0/647788480 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f39080cec80 0x7f39080b0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:20:05.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:05.856+0000 7f390f1d0700 1 -- 172.21.15.16:0/647788480 >> 172.21.15.16:0/647788480 conn(0x7f390806ae30 msgr2=0x7f39080ae4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:20:05.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:05.856+0000 7f390f1d0700 1 -- 172.21.15.16:0/647788480 shutdown_connections 2023-05-20T01:20:05.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:05.856+0000 7f390f1d0700 1 -- 172.21.15.16:0/647788480 wait complete. 2023-05-20T01:20:05.863 INFO:teuthology.orchestra.run.smithi016.stderr:added data pool 11 to fsmap 2023-05-20T01:20:05.877 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:20:05.878 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-20T01:20:05.921 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:20:06.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.043+0000 7fe9c7546700 1 Processor -- start 2023-05-20T01:20:06.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.043+0000 7fe9c7546700 1 -- start start 2023-05-20T01:20:06.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.044+0000 7fe9c7546700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe9c00b2760 0x7fe9c00d1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:20:06.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.044+0000 7fe9c7546700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fe9c0144020 0x7fe9c00b0980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:20:06.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.044+0000 7fe9c7546700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9c00cc500 con 0x7fe9c00cec60 2023-05-20T01:20:06.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.044+0000 7fe9c7546700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9c00cb6e0 con 0x7fe9c0144020 2023-05-20T01:20:06.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.044+0000 7fe9c7546700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c01448a0 con 0x7fe9c00b2760 2023-05-20T01:20:06.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.044+0000 7fe9c4ae1700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fe9c0144020 0x7fe9c00b0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:20:06.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.044+0000 7fe9c5ae3700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7fe9c00cec60 0x7fe9c00cc990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.16:55342/0 (socket says 172.21.15.16:55342) 2023-05-20T01:20:06.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.044+0000 7fe9c5ae3700 1 -- 172.21.15.16:0/1233285337 learned_addr learned my addr 172.21.15.16:0/1233285337 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:20:06.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.044+0000 7fe9c52e2700 1 --2- 172.21.15.16:0/1233285337 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe9c00b2760 0x7fe9c00d1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:20:06.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.044+0000 7fe9c52e2700 1 -- 172.21.15.16:0/1233285337 >> v2:172.21.15.132:3300/0 conn(0x7fe9c0144020 msgr2=0x7fe9c00b0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:20:06.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.044+0000 7fe9c52e2700 1 --2- 172.21.15.16:0/1233285337 >> v2:172.21.15.132:3300/0 conn(0x7fe9c0144020 0x7fe9c00b0980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:20:06.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.044+0000 7fe9c52e2700 1 -- 172.21.15.16:0/1233285337 >> v1:172.21.15.132:6789/0 conn(0x7fe9c00cec60 legacy=0x7fe9c00cc990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:20:06.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.044+0000 7fe9c52e2700 1 -- 172.21.15.16:0/1233285337 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9c00b0ec0 con 0x7fe9c00b2760 2023-05-20T01:20:06.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.044+0000 7fe9c4ae1700 1 --2- 172.21.15.16:0/1233285337 >> v2:172.21.15.132:3300/0 conn(0x7fe9c0144020 0x7fe9c00b0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-05-20T01:20:06.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.045+0000 7fe9c52e2700 1 --2- 172.21.15.16:0/1233285337 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe9c00b2760 0x7fe9c00d1680 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc009840 tx=0x7fe9bc009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e828f89e9314b997 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:20:06.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.045+0000 7fe9b7fff700 1 -- 172.21.15.16:0/1233285337 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9bc011830 con 0x7fe9c00b2760 2023-05-20T01:20:06.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.045+0000 7fe9b7fff700 1 -- 172.21.15.16:0/1233285337 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9bc011990 con 0x7fe9c00b2760 2023-05-20T01:20:06.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.045+0000 7fe9b7fff700 1 -- 172.21.15.16:0/1233285337 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9bc011d20 con 0x7fe9c00b2760 2023-05-20T01:20:06.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.045+0000 7fe9c7546700 1 -- 172.21.15.16:0/1233285337 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe9c00b2760 msgr2=0x7fe9c00d1680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:20:06.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.045+0000 7fe9c7546700 1 --2- 172.21.15.16:0/1233285337 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe9c00b2760 0x7fe9c00d1680 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc009840 tx=0x7fe9bc009b50 comp rx=0 tx=0).stop 2023-05-20T01:20:06.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.045+0000 7fe9c7546700 1 -- 172.21.15.16:0/1233285337 shutdown_connections 2023-05-20T01:20:06.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.045+0000 7fe9c7546700 1 --2- 172.21.15.16:0/1233285337 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe9c00b2760 0x7fe9c00d1680 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:20:06.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.045+0000 7fe9c7546700 1 --2- 172.21.15.16:0/1233285337 >> v2:172.21.15.132:3300/0 conn(0x7fe9c0144020 0x7fe9c00b0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:20:06.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.045+0000 7fe9c7546700 1 -- 172.21.15.16:0/1233285337 >> 172.21.15.16:0/1233285337 conn(0x7fe9c006ae30 msgr2=0x7fe9c009a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:20:06.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.045+0000 7fe9c7546700 1 -- 172.21.15.16:0/1233285337 shutdown_connections 2023-05-20T01:20:06.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.045+0000 7fe9c7546700 1 -- 172.21.15.16:0/1233285337 wait complete. 2023-05-20T01:20:06.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.047+0000 7fe9c7546700 1 Processor -- start 2023-05-20T01:20:06.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.047+0000 7fe9c7546700 1 -- start start 2023-05-20T01:20:06.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.047+0000 7fe9c7546700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe9c00b2760 0x7fe9c00b5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:20:06.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.047+0000 7fe9c7546700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe9c00cec60 0x7fe9c01f04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:20:06.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.047+0000 7fe9c7546700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9c0144020 0x7fe9c01f09f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:20:06.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.047+0000 7fe9c7546700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c00b6220 con 0x7fe9c0144020 2023-05-20T01:20:06.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.047+0000 7fe9c7546700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c00c9fd0 con 0x7fe9c00cec60 2023-05-20T01:20:06.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.047+0000 7fe9c7546700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c00b4c40 con 0x7fe9c00b2760 2023-05-20T01:20:06.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.047+0000 7fe9c5ae3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9c0144020 0x7fe9c01f09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:20:06.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.047+0000 7fe9c5ae3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9c0144020 0x7fe9c01f09f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34106/0 (socket says 172.21.15.16:34106) 2023-05-20T01:20:06.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.047+0000 7fe9c5ae3700 1 -- 172.21.15.16:0/1287257420 learned_addr learned my addr 172.21.15.16:0/1287257420 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:20:06.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.048+0000 7fe9c52e2700 1 --2- 172.21.15.16:0/1287257420 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe9c00b2760 0x7fe9c00b5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:20:06.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.048+0000 7fe9c4ae1700 1 --2- 172.21.15.16:0/1287257420 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe9c00cec60 0x7fe9c01f04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:20:06.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.048+0000 7fe9c5ae3700 1 -- 172.21.15.16:0/1287257420 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe9c00b2760 msgr2=0x7fe9c00b5ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:20:06.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.048+0000 7fe9c5ae3700 1 --2- 172.21.15.16:0/1287257420 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe9c00b2760 0x7fe9c00b5ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:20:06.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.048+0000 7fe9c5ae3700 1 -- 172.21.15.16:0/1287257420 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe9c00cec60 msgr2=0x7fe9c01f04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:20:06.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.048+0000 7fe9c5ae3700 1 --2- 172.21.15.16:0/1287257420 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe9c00cec60 0x7fe9c01f04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:20:06.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.048+0000 7fe9c5ae3700 1 -- 172.21.15.16:0/1287257420 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9b00094a0 con 0x7fe9c0144020 2023-05-20T01:20:06.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.048+0000 7fe9c5ae3700 1 --2- 172.21.15.16:0/1287257420 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9c0144020 0x7fe9c01f09f0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fe9b800ca80 tx=0x7fe9b800cd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:20:06.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.048+0000 7fe9b5ffb700 1 -- 172.21.15.16:0/1287257420 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b800a9e0 con 0x7fe9c0144020 2023-05-20T01:20:06.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.048+0000 7fe9b5ffb700 1 -- 172.21.15.16:0/1287257420 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9b800ab40 con 0x7fe9c0144020 2023-05-20T01:20:06.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.048+0000 7fe9c7546700 1 -- 172.21.15.16:0/1287257420 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9bc0094a0 con 0x7fe9c0144020 2023-05-20T01:20:06.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.048+0000 7fe9b5ffb700 1 -- 172.21.15.16:0/1287257420 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b8012400 con 0x7fe9c0144020 2023-05-20T01:20:06.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.048+0000 7fe9c7546700 1 -- 172.21.15.16:0/1287257420 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9c00cbbd0 con 0x7fe9c0144020 2023-05-20T01:20:06.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.049+0000 7fe9c7546700 1 -- 172.21.15.16:0/1287257420 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9c00cb8f0 con 0x7fe9c0144020 2023-05-20T01:20:06.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.050+0000 7fe9b5ffb700 1 -- 172.21.15.16:0/1287257420 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe9b801b3b0 con 0x7fe9c0144020 2023-05-20T01:20:06.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.050+0000 7fe9b5ffb700 1 --2- 172.21.15.16:0/1287257420 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe9a406ace0 0x7fe9a406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:20:06.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.051+0000 7fe9b5ffb700 1 -- 172.21.15.16:0/1287257420 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6786+0+0 (secure 0 0 0) 0x7fe9b8089270 con 0x7fe9c0144020 2023-05-20T01:20:06.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.051+0000 7fe9c52e2700 1 --2- 172.21.15.16:0/1287257420 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe9a406ace0 0x7fe9a406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:20:06.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.051+0000 7fe9c52e2700 1 --2- 172.21.15.16:0/1287257420 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe9a406ace0 0x7fe9a406d190 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc009fd0 tx=0x7fe9bc01c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:20:06.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.055+0000 7fe9b5ffb700 1 -- 172.21.15.16:0/1287257420 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fe9b8054050 con 0x7fe9c0144020 2023-05-20T01:20:06.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.298+0000 7fe9c7546700 1 -- 172.21.15.16:0/1287257420 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe9c00cbeb0 con 0x7fe9c0144020 2023-05-20T01:20:06.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.298+0000 7fe9b5ffb700 1 -- 172.21.15.16:0/1287257420 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4675 (secure 0 0 0) 0x7fe9b8057670 con 0x7fe9c0144020 2023-05-20T01:20:06.302 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:20:06.302 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":38,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5824,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/2609276800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":2609276800},{"type":"v1","addr":"172.21.15.132:6834","nonce":2609276800}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32},{"gid":5827,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/29554453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":29554453},{"type":"v1","addr":"172.21.15.132:6835","nonce":29554453}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32},{"gid":6192,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/4090303296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4090303296},{"type":"v1","addr":"172.21.15.16:6836","nonce":4090303296}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-20T01:19:51.915360+0000","modified":"2023-05-20T01:20:05.849669+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6195},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6195":{"gid":6195,"name":"c","rank":0,"incarnation":33,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6837/1427908556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":1427908556},{"type":"v1","addr":"172.21.15.16:6837","nonce":1427908556}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10,11],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-05-20T01:20:06.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.300+0000 7fe9c7546700 1 -- 172.21.15.16:0/1287257420 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe9a406ace0 msgr2=0x7fe9a406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:20:06.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.300+0000 7fe9c7546700 1 --2- 172.21.15.16:0/1287257420 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe9a406ace0 0x7fe9a406d190 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc009fd0 tx=0x7fe9bc01c040 comp rx=0 tx=0).stop 2023-05-20T01:20:06.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.300+0000 7fe9c7546700 1 -- 172.21.15.16:0/1287257420 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9c0144020 msgr2=0x7fe9c01f09f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:20:06.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.300+0000 7fe9c7546700 1 --2- 172.21.15.16:0/1287257420 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9c0144020 0x7fe9c01f09f0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fe9b800ca80 tx=0x7fe9b800cd90 comp rx=0 tx=0).stop 2023-05-20T01:20:06.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.300+0000 7fe9c7546700 1 -- 172.21.15.16:0/1287257420 shutdown_connections 2023-05-20T01:20:06.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.300+0000 7fe9c7546700 1 --2- 172.21.15.16:0/1287257420 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9c0144020 0x7fe9c01f09f0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:20:06.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.300+0000 7fe9c7546700 1 --2- 172.21.15.16:0/1287257420 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe9a406ace0 0x7fe9a406d190 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:20:06.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.300+0000 7fe9c7546700 1 --2- 172.21.15.16:0/1287257420 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe9c00b2760 0x7fe9c00b5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:20:06.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.300+0000 7fe9c7546700 1 --2- 172.21.15.16:0/1287257420 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe9c00cec60 0x7fe9c01f04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:20:06.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.300+0000 7fe9c7546700 1 -- 172.21.15.16:0/1287257420 >> 172.21.15.16:0/1287257420 conn(0x7fe9c006ae30 msgr2=0x7fe9c00b0220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:20:06.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.300+0000 7fe9c7546700 1 -- 172.21.15.16:0/1287257420 shutdown_connections 2023-05-20T01:20:06.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.300+0000 7fe9c7546700 1 -- 172.21.15.16:0/1287257420 wait complete. 2023-05-20T01:20:06.308 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 38 2023-05-20T01:20:06.316 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-20T01:20:06.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.439+0000 7f7b84996700 1 Processor -- start 2023-05-20T01:20:06.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.439+0000 7f7b84996700 1 -- start start 2023-05-20T01:20:06.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.440+0000 7f7b84996700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7b800ae860 0x7f7b8015d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:20:06.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.440+0000 7f7b84996700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7b8015d780 0x7f7b800b1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:20:06.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.440+0000 7f7b84996700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f7b800b1eb0 0x7f7b80071090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:20:06.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.440+0000 7f7b84996700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b800aec30 con 0x7f7b800ae860 2023-05-20T01:20:06.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.440+0000 7f7b84996700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b8006d660 con 0x7f7b800b1eb0 2023-05-20T01:20:06.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.440+0000 7f7b84996700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b800b22f0 con 0x7f7b8015d780 2023-05-20T01:20:06.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.440+0000 7f7b7e59c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7b800ae860 0x7f7b8015d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:20:06.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.440+0000 7f7b7e59c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7b800ae860 0x7f7b8015d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34110/0 (socket says 172.21.15.16:34110) 2023-05-20T01:20:06.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.440+0000 7f7b7e59c700 1 -- 172.21.15.16:0/3465101176 learned_addr learned my addr 172.21.15.16:0/3465101176 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:20:06.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.440+0000 7f7b7ed9d700 1 --2- 172.21.15.16:0/3465101176 >> v2:172.21.15.132:3300/0 conn(0x7f7b800b1eb0 0x7f7b80071090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:20:06.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.440+0000 7f7b7dd9b700 1 --2- 172.21.15.16:0/3465101176 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7b8015d780 0x7f7b800b1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:20:06.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.440+0000 7f7b7e59c700 1 -- 172.21.15.16:0/3465101176 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7b8015d780 msgr2=0x7f7b800b1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:20:06.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.440+0000 7f7b7e59c700 1 --2- 172.21.15.16:0/3465101176 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7b8015d780 0x7f7b800b1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:20:06.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.440+0000 7f7b7e59c700 1 -- 172.21.15.16:0/3465101176 >> v2:172.21.15.132:3300/0 conn(0x7f7b800b1eb0 msgr2=0x7f7b80071090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:20:06.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.440+0000 7f7b7e59c700 1 --2- 172.21.15.16:0/3465101176 >> v2:172.21.15.132:3300/0 conn(0x7f7b800b1eb0 0x7f7b80071090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:20:06.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.440+0000 7f7b7e59c700 1 -- 172.21.15.16:0/3465101176 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b800acd60 con 0x7f7b800ae860 2023-05-20T01:20:06.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.440+0000 7f7b7e59c700 1 --2- 172.21.15.16:0/3465101176 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b800ae860 0x7f7b8015d240 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f7b8009e090 tx=0x7f7b7000be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81e72b755811bf9c server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:20:06.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.441+0000 7f7b7d59a700 1 -- 172.21.15.16:0/3465101176 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b7000dd30 con 0x7f7b800ae860 2023-05-20T01:20:06.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.441+0000 7f7b7d59a700 1 -- 172.21.15.16:0/3465101176 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b7000de90 con 0x7f7b800ae860 2023-05-20T01:20:06.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.441+0000 7f7b7d59a700 1 -- 172.21.15.16:0/3465101176 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b70010690 con 0x7f7b800ae860 2023-05-20T01:20:06.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.441+0000 7f7b84996700 1 -- 172.21.15.16:0/3465101176 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b800ae860 msgr2=0x7f7b8015d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:20:06.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.441+0000 7f7b84996700 1 --2- 172.21.15.16:0/3465101176 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b800ae860 0x7f7b8015d240 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f7b8009e090 tx=0x7f7b7000be70 comp rx=0 tx=0).stop 2023-05-20T01:20:06.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.441+0000 7f7b84996700 1 -- 172.21.15.16:0/3465101176 shutdown_connections 2023-05-20T01:20:06.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.441+0000 7f7b84996700 1 --2- 172.21.15.16:0/3465101176 >> v2:172.21.15.132:3300/0 conn(0x7f7b800b1eb0 0x7f7b80071090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:20:06.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.441+0000 7f7b84996700 1 --2- 172.21.15.16:0/3465101176 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b800ae860 0x7f7b8015d240 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:20:06.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.441+0000 7f7b84996700 1 --2- 172.21.15.16:0/3465101176 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7b8015d780 0x7f7b800b1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:20:06.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.441+0000 7f7b84996700 1 -- 172.21.15.16:0/3465101176 >> 172.21.15.16:0/3465101176 conn(0x7f7b800d0b80 msgr2=0x7f7b800cc970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:20:06.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.441+0000 7f7b84996700 1 -- 172.21.15.16:0/3465101176 shutdown_connections 2023-05-20T01:20:06.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.441+0000 7f7b84996700 1 -- 172.21.15.16:0/3465101176 wait complete. 2023-05-20T01:20:06.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.443+0000 7f7b84996700 1 Processor -- start 2023-05-20T01:20:06.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.443+0000 7f7b84996700 1 -- start start 2023-05-20T01:20:06.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.443+0000 7f7b84996700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b800ae860 0x7f7b801f2780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:20:06.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.443+0000 7f7b84996700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7b800b1eb0 0x7f7b800b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:20:06.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.443+0000 7f7b84996700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7b8015d780 0x7f7b801562c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:20:06.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.443+0000 7f7b84996700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b800c6fd0 con 0x7f7b800ae860 2023-05-20T01:20:06.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.443+0000 7f7b84996700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b801f2de0 con 0x7f7b8015d780 2023-05-20T01:20:06.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.443+0000 7f7b84996700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b801f30c0 con 0x7f7b800b1eb0 2023-05-20T01:20:06.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.443+0000 7f7b7e59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b800ae860 0x7f7b801f2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:20:06.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.443+0000 7f7b7e59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b800ae860 0x7f7b801f2780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34116/0 (socket says 172.21.15.16:34116) 2023-05-20T01:20:06.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.443+0000 7f7b7e59c700 1 -- 172.21.15.16:0/928652340 learned_addr learned my addr 172.21.15.16:0/928652340 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:20:06.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.444+0000 7f7b7dd9b700 1 --2- 172.21.15.16:0/928652340 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7b800b1eb0 0x7f7b800b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:20:06.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.444+0000 7f7b7ed9d700 1 --2- 172.21.15.16:0/928652340 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7b8015d780 0x7f7b801562c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:20:06.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.444+0000 7f7b7e59c700 1 -- 172.21.15.16:0/928652340 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7b800b1eb0 msgr2=0x7f7b800b4ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:20:06.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.444+0000 7f7b7e59c700 1 --2- 172.21.15.16:0/928652340 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7b800b1eb0 0x7f7b800b4ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:20:06.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.444+0000 7f7b7e59c700 1 -- 172.21.15.16:0/928652340 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7b8015d780 msgr2=0x7f7b801562c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:20:06.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.444+0000 7f7b7e59c700 1 --2- 172.21.15.16:0/928652340 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7b8015d780 0x7f7b801562c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:20:06.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.444+0000 7f7b7e59c700 1 -- 172.21.15.16:0/928652340 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b7000b7f0 con 0x7f7b800ae860 2023-05-20T01:20:06.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.444+0000 7f7b7e59c700 1 --2- 172.21.15.16:0/928652340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b800ae860 0x7f7b801f2780 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f7b7000beb0 tx=0x7f7b70004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:20:06.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.444+0000 7f7b6effd700 1 -- 172.21.15.16:0/928652340 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b70003da0 con 0x7f7b800ae860 2023-05-20T01:20:06.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.444+0000 7f7b6effd700 1 -- 172.21.15.16:0/928652340 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b70010470 con 0x7f7b800ae860 2023-05-20T01:20:06.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.444+0000 7f7b84996700 1 -- 172.21.15.16:0/928652340 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b8015e090 con 0x7f7b800ae860 2023-05-20T01:20:06.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.444+0000 7f7b6effd700 1 -- 172.21.15.16:0/928652340 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b70010820 con 0x7f7b800ae860 2023-05-20T01:20:06.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.444+0000 7f7b84996700 1 -- 172.21.15.16:0/928652340 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b8015ddb0 con 0x7f7b800ae860 2023-05-20T01:20:06.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.445+0000 7f7b84996700 1 -- 172.21.15.16:0/928652340 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b800747b0 con 0x7f7b800ae860 2023-05-20T01:20:06.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.446+0000 7f7b6effd700 1 -- 172.21.15.16:0/928652340 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7b70019400 con 0x7f7b800ae860 2023-05-20T01:20:06.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.446+0000 7f7b6effd700 1 --2- 172.21.15.16:0/928652340 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f7b6006ace0 0x7f7b6006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:20:06.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.446+0000 7f7b6effd700 1 -- 172.21.15.16:0/928652340 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6786+0+0 (secure 0 0 0) 0x7f7b7008abf0 con 0x7f7b800ae860 2023-05-20T01:20:06.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.446+0000 7f7b7dd9b700 1 --2- 172.21.15.16:0/928652340 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f7b6006ace0 0x7f7b6006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:20:06.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.446+0000 7f7b7dd9b700 1 --2- 172.21.15.16:0/928652340 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f7b6006ace0 0x7f7b6006d190 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f7b7400b9b0 tx=0x7f7b74008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:20:06.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.450+0000 7f7b6effd700 1 -- 172.21.15.16:0/928652340 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f7b70055a50 con 0x7f7b800ae860 2023-05-20T01:20:06.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.648+0000 7f7b84996700 1 -- 172.21.15.16:0/928652340 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7b8006cf90 con 0x7f7b800ae860 2023-05-20T01:20:06.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.649+0000 7f7b6effd700 1 -- 172.21.15.16:0/928652340 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) v1 ==== 74+0+20206 (secure 0 0 0) 0x7f7b70059070 con 0x7f7b800ae860 2023-05-20T01:20:06.653 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:20:06.653 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":66,"fsid":"c2245cc5-ecab-4087-a59b-bc81db901676","created":"2023-05-20T01:16:09.549230+0000","modified":"2023-05-20T01:20:05.849400+0000","last_up_change":"2023-05-20T01:16:19.788624+0000","last_in_change":"2023-05-20T01:16:12.092160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-05-20T01:16:20.934366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-05-20T01:16:21.372294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-05-20T01:19:49.878849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-05-20T01:19:50.887306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data_extra","create_time":"2023-05-20T01:20:04.581773+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"317a83cd-19c8-4958-a301-6087ef342289","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6825","nonce":3923634179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6827","nonce":3923634179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6831","nonce":3923634179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6829","nonce":3923634179}]},"public_addr":"172.21.15.16:6825/3923634179","cluster_addr":"172.21.15.16:6827/3923634179","heartbeat_back_addr":"172.21.15.16:6831/3923634179","heartbeat_front_addr":"172.21.15.16:6829/3923634179","state":["exists","up"]},{"osd":1,"uuid":"5581046f-8219-4089-8d7d-b7846399505e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6809","nonce":3860264220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6811","nonce":3860264220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6815","nonce":3860264220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6813","nonce":3860264220}]},"public_addr":"172.21.15.16:6809/3860264220","cluster_addr":"172.21.15.16:6811/3860264220","heartbeat_back_addr":"172.21.15.16:6815/3860264220","heartbeat_front_addr":"172.21.15.16:6813/3860264220","state":["exists","up"]},{"osd":2,"uuid":"072e9e66-4133-478e-8d4a-64d46c2963f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6817","nonce":293187030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6819","nonce":293187030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6823","nonce":293187030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6821","nonce":293187030}]},"public_addr":"172.21.15.16:6817/293187030","cluster_addr":"172.21.15.16:6819/293187030","heartbeat_back_addr":"172.21.15.16:6823/293187030","heartbeat_front_addr":"172.21.15.16:6821/293187030","state":["exists","up"]},{"osd":3,"uuid":"11e1bba6-00f8-48dd-9028-d8d716b7f145","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6801","nonce":3669781890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6803","nonce":3669781890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6807","nonce":3669781890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6805","nonce":3669781890}]},"public_addr":"172.21.15.16:6801/3669781890","cluster_addr":"172.21.15.16:6803/3669781890","heartbeat_back_addr":"172.21.15.16:6807/3669781890","heartbeat_front_addr":"172.21.15.16:6805/3669781890","state":["exists","up"]},{"osd":4,"uuid":"b8909c3a-cf70-46cb-8d0b-009c707cf9d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6819","nonce":506317223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6823","nonce":506317223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6829","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6831","nonce":506317223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6827","nonce":506317223}]},"public_addr":"172.21.15.132:6819/506317223","cluster_addr":"172.21.15.132:6823/506317223","heartbeat_back_addr":"172.21.15.132:6831/506317223","heartbeat_front_addr":"172.21.15.132:6827/506317223","state":["exists","up"]},{"osd":5,"uuid":"b1818304-4583-47bf-a6ec-0bfbb5166233","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6818","nonce":1844689995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6822","nonce":1844689995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6830","nonce":1844689995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6826","nonce":1844689995}]},"public_addr":"172.21.15.132:6818/1844689995","cluster_addr":"172.21.15.132:6822/1844689995","heartbeat_back_addr":"172.21.15.132:6830/1844689995","heartbeat_front_addr":"172.21.15.132:6826/1844689995","state":["exists","up"]},{"osd":6,"uuid":"d37f4691-eed1-4a93-8818-18354f03f4a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6801","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6802","nonce":727581123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6805","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6807","nonce":727581123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6813","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6815","nonce":727581123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6811","nonce":727581123}]},"public_addr":"172.21.15.132:6802/727581123","cluster_addr":"172.21.15.132:6807/727581123","heartbeat_back_addr":"172.21.15.132:6815/727581123","heartbeat_front_addr":"172.21.15.132:6811/727581123","state":["exists","up"]},{"osd":7,"uuid":"4f712f62-5d1d-4816-8042-03209144f8b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6803","nonce":466001149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6806","nonce":466001149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6814","nonce":466001149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6810","nonce":466001149}]},"public_addr":"172.21.15.132:6803/466001149","cluster_addr":"172.21.15.132:6806/466001149","heartbeat_back_addr":"172.21.15.132:6814/466001149","heartbeat_front_addr":"172.21.15.132:6810/466001149","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.338863+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.278680+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305694+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.246835+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.276237+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.304441+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305421+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.267050+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-20T01:20:06.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.651+0000 7f7b84996700 1 -- 172.21.15.16:0/928652340 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f7b6006ace0 msgr2=0x7f7b6006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:20:06.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.651+0000 7f7b84996700 1 --2- 172.21.15.16:0/928652340 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f7b6006ace0 0x7f7b6006d190 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f7b7400b9b0 tx=0x7f7b74008040 comp rx=0 tx=0).stop 2023-05-20T01:20:06.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.652+0000 7f7b84996700 1 -- 172.21.15.16:0/928652340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b800ae860 msgr2=0x7f7b801f2780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:20:06.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.652+0000 7f7b84996700 1 --2- 172.21.15.16:0/928652340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b800ae860 0x7f7b801f2780 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f7b7000beb0 tx=0x7f7b70004750 comp rx=0 tx=0).stop 2023-05-20T01:20:06.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.652+0000 7f7b84996700 1 -- 172.21.15.16:0/928652340 shutdown_connections 2023-05-20T01:20:06.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.652+0000 7f7b84996700 1 --2- 172.21.15.16:0/928652340 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7b8015d780 0x7f7b801562c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:20:06.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.652+0000 7f7b84996700 1 --2- 172.21.15.16:0/928652340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b800ae860 0x7f7b801f2780 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:20:06.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.652+0000 7f7b84996700 1 --2- 172.21.15.16:0/928652340 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f7b6006ace0 0x7f7b6006d190 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:20:06.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.652+0000 7f7b84996700 1 --2- 172.21.15.16:0/928652340 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7b800b1eb0 0x7f7b800b4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:20:06.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.652+0000 7f7b84996700 1 -- 172.21.15.16:0/928652340 >> 172.21.15.16:0/928652340 conn(0x7f7b800d0b80 msgr2=0x7f7b800afbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:20:06.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.652+0000 7f7b84996700 1 -- 172.21.15.16:0/928652340 shutdown_connections 2023-05-20T01:20:06.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:20:06.652+0000 7f7b84996700 1 -- 172.21.15.16:0/928652340 wait complete. 2023-05-20T01:20:06.667 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:20:06.667 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.layout.pool -v cephfs_data_extra .) 2023-05-20T01:20:06.687 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:20:06.688 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir subdir) 2023-05-20T01:20:06.749 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/0 2023-05-20T01:20:06.749 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits daemon-helper kill python3 -c ' 2023-05-20T01:20:06.749 DEBUG:teuthology.orchestra.run.smithi016:> import zlib 2023-05-20T01:20:06.750 DEBUG:teuthology.orchestra.run.smithi016:> path = "/home/ubuntu/cephtest/mnt.0/subdir/0" 2023-05-20T01:20:06.750 DEBUG:teuthology.orchestra.run.smithi016:> with open(path, '"'"'w'"'"') as f: 2023-05-20T01:20:06.750 DEBUG:teuthology.orchestra.run.smithi016:> for i in range(0, 6291456): 2023-05-20T01:20:06.751 DEBUG:teuthology.orchestra.run.smithi016:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-05-20T01:20:06.751 DEBUG:teuthology.orchestra.run.smithi016:> f.write(chr(val)) 2023-05-20T01:20:06.751 DEBUG:teuthology.orchestra.run.smithi016:> ' 2023-05-20T01:20:11.651 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/1 2023-05-20T01:20:11.652 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits daemon-helper kill python3 -c ' 2023-05-20T01:20:11.653 DEBUG:teuthology.orchestra.run.smithi016:> import zlib 2023-05-20T01:20:11.653 DEBUG:teuthology.orchestra.run.smithi016:> path = "/home/ubuntu/cephtest/mnt.0/subdir/1" 2023-05-20T01:20:11.654 DEBUG:teuthology.orchestra.run.smithi016:> with open(path, '"'"'w'"'"') as f: 2023-05-20T01:20:11.654 DEBUG:teuthology.orchestra.run.smithi016:> for i in range(0, 6291456): 2023-05-20T01:20:11.655 DEBUG:teuthology.orchestra.run.smithi016:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-05-20T01:20:11.656 DEBUG:teuthology.orchestra.run.smithi016:> f.write(chr(val)) 2023-05-20T01:20:11.656 DEBUG:teuthology.orchestra.run.smithi016:> ' 2023-05-20T01:20:16.354 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/2 2023-05-20T01:20:16.354 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits daemon-helper kill python3 -c ' 2023-05-20T01:20:16.355 DEBUG:teuthology.orchestra.run.smithi016:> import zlib 2023-05-20T01:20:16.355 DEBUG:teuthology.orchestra.run.smithi016:> path = "/home/ubuntu/cephtest/mnt.0/subdir/2" 2023-05-20T01:20:16.356 DEBUG:teuthology.orchestra.run.smithi016:> with open(path, '"'"'w'"'"') as f: 2023-05-20T01:20:16.357 DEBUG:teuthology.orchestra.run.smithi016:> for i in range(0, 6291456): 2023-05-20T01:20:16.357 DEBUG:teuthology.orchestra.run.smithi016:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-05-20T01:20:16.358 DEBUG:teuthology.orchestra.run.smithi016:> f.write(chr(val)) 2023-05-20T01:20:16.358 DEBUG:teuthology.orchestra.run.smithi016:> ' 2023-05-20T01:20:21.260 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/3 2023-05-20T01:20:21.261 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits daemon-helper kill python3 -c ' 2023-05-20T01:20:21.261 DEBUG:teuthology.orchestra.run.smithi016:> import zlib 2023-05-20T01:20:21.261 DEBUG:teuthology.orchestra.run.smithi016:> path = "/home/ubuntu/cephtest/mnt.0/subdir/3" 2023-05-20T01:20:21.262 DEBUG:teuthology.orchestra.run.smithi016:> with open(path, '"'"'w'"'"') as f: 2023-05-20T01:20:21.262 DEBUG:teuthology.orchestra.run.smithi016:> for i in range(0, 6291456): 2023-05-20T01:20:21.262 DEBUG:teuthology.orchestra.run.smithi016:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-05-20T01:20:21.263 DEBUG:teuthology.orchestra.run.smithi016:> f.write(chr(val)) 2023-05-20T01:20:21.263 DEBUG:teuthology.orchestra.run.smithi016:> ' 2023-05-20T01:20:26.163 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/4 2023-05-20T01:20:26.164 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits daemon-helper kill python3 -c ' 2023-05-20T01:20:26.164 DEBUG:teuthology.orchestra.run.smithi016:> import zlib 2023-05-20T01:20:26.165 DEBUG:teuthology.orchestra.run.smithi016:> path = "/home/ubuntu/cephtest/mnt.0/subdir/4" 2023-05-20T01:20:26.165 DEBUG:teuthology.orchestra.run.smithi016:> with open(path, '"'"'w'"'"') as f: 2023-05-20T01:20:26.165 DEBUG:teuthology.orchestra.run.smithi016:> for i in range(0, 6291456): 2023-05-20T01:20:26.166 DEBUG:teuthology.orchestra.run.smithi016:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-05-20T01:20:26.166 DEBUG:teuthology.orchestra.run.smithi016:> f.write(chr(val)) 2023-05-20T01:20:26.166 DEBUG:teuthology.orchestra.run.smithi016:> ' 2023-05-20T01:20:31.066 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/5 2023-05-20T01:20:31.067 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits daemon-helper kill python3 -c ' 2023-05-20T01:20:31.067 DEBUG:teuthology.orchestra.run.smithi016:> import zlib 2023-05-20T01:20:31.068 DEBUG:teuthology.orchestra.run.smithi016:> path = "/home/ubuntu/cephtest/mnt.0/subdir/5" 2023-05-20T01:20:31.068 DEBUG:teuthology.orchestra.run.smithi016:> with open(path, '"'"'w'"'"') as f: 2023-05-20T01:20:31.068 DEBUG:teuthology.orchestra.run.smithi016:> for i in range(0, 6291456): 2023-05-20T01:20:31.068 DEBUG:teuthology.orchestra.run.smithi016:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-05-20T01:20:31.069 DEBUG:teuthology.orchestra.run.smithi016:> f.write(chr(val)) 2023-05-20T01:20:31.069 DEBUG:teuthology.orchestra.run.smithi016:> ' 2023-05-20T01:20:35.969 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/6 2023-05-20T01:20:35.969 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits daemon-helper kill python3 -c ' 2023-05-20T01:20:35.970 DEBUG:teuthology.orchestra.run.smithi016:> import zlib 2023-05-20T01:20:35.970 DEBUG:teuthology.orchestra.run.smithi016:> path = "/home/ubuntu/cephtest/mnt.0/subdir/6" 2023-05-20T01:20:35.970 DEBUG:teuthology.orchestra.run.smithi016:> with open(path, '"'"'w'"'"') as f: 2023-05-20T01:20:35.970 DEBUG:teuthology.orchestra.run.smithi016:> for i in range(0, 6291456): 2023-05-20T01:20:35.971 DEBUG:teuthology.orchestra.run.smithi016:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-05-20T01:20:35.971 DEBUG:teuthology.orchestra.run.smithi016:> f.write(chr(val)) 2023-05-20T01:20:35.972 DEBUG:teuthology.orchestra.run.smithi016:> ' 2023-05-20T01:20:36.104 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:20:36.107 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:20:40.871 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/7 2023-05-20T01:20:40.872 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits daemon-helper kill python3 -c ' 2023-05-20T01:20:40.872 DEBUG:teuthology.orchestra.run.smithi016:> import zlib 2023-05-20T01:20:40.873 DEBUG:teuthology.orchestra.run.smithi016:> path = "/home/ubuntu/cephtest/mnt.0/subdir/7" 2023-05-20T01:20:40.873 DEBUG:teuthology.orchestra.run.smithi016:> with open(path, '"'"'w'"'"') as f: 2023-05-20T01:20:40.873 DEBUG:teuthology.orchestra.run.smithi016:> for i in range(0, 6291456): 2023-05-20T01:20:40.873 DEBUG:teuthology.orchestra.run.smithi016:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-05-20T01:20:40.874 DEBUG:teuthology.orchestra.run.smithi016:> f.write(chr(val)) 2023-05-20T01:20:40.874 DEBUG:teuthology.orchestra.run.smithi016:> ' 2023-05-20T01:20:45.737 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/8 2023-05-20T01:20:45.737 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits daemon-helper kill python3 -c ' 2023-05-20T01:20:45.738 DEBUG:teuthology.orchestra.run.smithi016:> import zlib 2023-05-20T01:20:45.739 DEBUG:teuthology.orchestra.run.smithi016:> path = "/home/ubuntu/cephtest/mnt.0/subdir/8" 2023-05-20T01:20:45.739 DEBUG:teuthology.orchestra.run.smithi016:> with open(path, '"'"'w'"'"') as f: 2023-05-20T01:20:45.740 DEBUG:teuthology.orchestra.run.smithi016:> for i in range(0, 6291456): 2023-05-20T01:20:45.740 DEBUG:teuthology.orchestra.run.smithi016:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-05-20T01:20:45.740 DEBUG:teuthology.orchestra.run.smithi016:> f.write(chr(val)) 2023-05-20T01:20:45.742 DEBUG:teuthology.orchestra.run.smithi016:> ' 2023-05-20T01:20:50.638 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/9 2023-05-20T01:20:50.639 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits daemon-helper kill python3 -c ' 2023-05-20T01:20:50.640 DEBUG:teuthology.orchestra.run.smithi016:> import zlib 2023-05-20T01:20:50.640 DEBUG:teuthology.orchestra.run.smithi016:> path = "/home/ubuntu/cephtest/mnt.0/subdir/9" 2023-05-20T01:20:50.641 DEBUG:teuthology.orchestra.run.smithi016:> with open(path, '"'"'w'"'"') as f: 2023-05-20T01:20:50.642 DEBUG:teuthology.orchestra.run.smithi016:> for i in range(0, 6291456): 2023-05-20T01:20:50.642 DEBUG:teuthology.orchestra.run.smithi016:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-05-20T01:20:50.643 DEBUG:teuthology.orchestra.run.smithi016:> f.write(chr(val)) 2023-05-20T01:20:50.643 DEBUG:teuthology.orchestra.run.smithi016:> ' 2023-05-20T01:20:55.541 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/10 2023-05-20T01:20:55.542 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits daemon-helper kill python3 -c ' 2023-05-20T01:20:55.542 DEBUG:teuthology.orchestra.run.smithi016:> import zlib 2023-05-20T01:20:55.543 DEBUG:teuthology.orchestra.run.smithi016:> path = "/home/ubuntu/cephtest/mnt.0/subdir/10" 2023-05-20T01:20:55.543 DEBUG:teuthology.orchestra.run.smithi016:> with open(path, '"'"'w'"'"') as f: 2023-05-20T01:20:55.543 DEBUG:teuthology.orchestra.run.smithi016:> for i in range(0, 6291456): 2023-05-20T01:20:55.544 DEBUG:teuthology.orchestra.run.smithi016:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-05-20T01:20:55.544 DEBUG:teuthology.orchestra.run.smithi016:> f.write(chr(val)) 2023-05-20T01:20:55.544 DEBUG:teuthology.orchestra.run.smithi016:> ' 2023-05-20T01:21:00.443 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/11 2023-05-20T01:21:00.444 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits daemon-helper kill python3 -c ' 2023-05-20T01:21:00.444 DEBUG:teuthology.orchestra.run.smithi016:> import zlib 2023-05-20T01:21:00.444 DEBUG:teuthology.orchestra.run.smithi016:> path = "/home/ubuntu/cephtest/mnt.0/subdir/11" 2023-05-20T01:21:00.445 DEBUG:teuthology.orchestra.run.smithi016:> with open(path, '"'"'w'"'"') as f: 2023-05-20T01:21:00.445 DEBUG:teuthology.orchestra.run.smithi016:> for i in range(0, 6291456): 2023-05-20T01:21:00.445 DEBUG:teuthology.orchestra.run.smithi016:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-05-20T01:21:00.446 DEBUG:teuthology.orchestra.run.smithi016:> f.write(chr(val)) 2023-05-20T01:21:00.446 DEBUG:teuthology.orchestra.run.smithi016:> ' 2023-05-20T01:21:05.344 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/12 2023-05-20T01:21:05.345 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits daemon-helper kill python3 -c ' 2023-05-20T01:21:05.345 DEBUG:teuthology.orchestra.run.smithi016:> import zlib 2023-05-20T01:21:05.345 DEBUG:teuthology.orchestra.run.smithi016:> path = "/home/ubuntu/cephtest/mnt.0/subdir/12" 2023-05-20T01:21:05.346 DEBUG:teuthology.orchestra.run.smithi016:> with open(path, '"'"'w'"'"') as f: 2023-05-20T01:21:05.346 DEBUG:teuthology.orchestra.run.smithi016:> for i in range(0, 6291456): 2023-05-20T01:21:05.346 DEBUG:teuthology.orchestra.run.smithi016:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-05-20T01:21:05.347 DEBUG:teuthology.orchestra.run.smithi016:> f.write(chr(val)) 2023-05-20T01:21:05.347 DEBUG:teuthology.orchestra.run.smithi016:> ' 2023-05-20T01:21:06.188 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:21:06.190 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:21:10.248 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/13 2023-05-20T01:21:10.249 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits daemon-helper kill python3 -c ' 2023-05-20T01:21:10.250 DEBUG:teuthology.orchestra.run.smithi016:> import zlib 2023-05-20T01:21:10.250 DEBUG:teuthology.orchestra.run.smithi016:> path = "/home/ubuntu/cephtest/mnt.0/subdir/13" 2023-05-20T01:21:10.250 DEBUG:teuthology.orchestra.run.smithi016:> with open(path, '"'"'w'"'"') as f: 2023-05-20T01:21:10.251 DEBUG:teuthology.orchestra.run.smithi016:> for i in range(0, 6291456): 2023-05-20T01:21:10.251 DEBUG:teuthology.orchestra.run.smithi016:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-05-20T01:21:10.251 DEBUG:teuthology.orchestra.run.smithi016:> f.write(chr(val)) 2023-05-20T01:21:10.252 DEBUG:teuthology.orchestra.run.smithi016:> ' 2023-05-20T01:21:15.117 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/14 2023-05-20T01:21:15.118 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits daemon-helper kill python3 -c ' 2023-05-20T01:21:15.118 DEBUG:teuthology.orchestra.run.smithi016:> import zlib 2023-05-20T01:21:15.119 DEBUG:teuthology.orchestra.run.smithi016:> path = "/home/ubuntu/cephtest/mnt.0/subdir/14" 2023-05-20T01:21:15.120 DEBUG:teuthology.orchestra.run.smithi016:> with open(path, '"'"'w'"'"') as f: 2023-05-20T01:21:15.120 DEBUG:teuthology.orchestra.run.smithi016:> for i in range(0, 6291456): 2023-05-20T01:21:15.121 DEBUG:teuthology.orchestra.run.smithi016:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-05-20T01:21:15.121 DEBUG:teuthology.orchestra.run.smithi016:> f.write(chr(val)) 2023-05-20T01:21:15.122 DEBUG:teuthology.orchestra.run.smithi016:> ' 2023-05-20T01:21:20.019 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/15 2023-05-20T01:21:20.020 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits daemon-helper kill python3 -c ' 2023-05-20T01:21:20.020 DEBUG:teuthology.orchestra.run.smithi016:> import zlib 2023-05-20T01:21:20.021 DEBUG:teuthology.orchestra.run.smithi016:> path = "/home/ubuntu/cephtest/mnt.0/subdir/15" 2023-05-20T01:21:20.021 DEBUG:teuthology.orchestra.run.smithi016:> with open(path, '"'"'w'"'"') as f: 2023-05-20T01:21:20.021 DEBUG:teuthology.orchestra.run.smithi016:> for i in range(0, 6291456): 2023-05-20T01:21:20.021 DEBUG:teuthology.orchestra.run.smithi016:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-05-20T01:21:20.022 DEBUG:teuthology.orchestra.run.smithi016:> f.write(chr(val)) 2023-05-20T01:21:20.022 DEBUG:teuthology.orchestra.run.smithi016:> ' 2023-05-20T01:21:24.923 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/16 2023-05-20T01:21:24.924 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits daemon-helper kill python3 -c ' 2023-05-20T01:21:24.924 DEBUG:teuthology.orchestra.run.smithi016:> import zlib 2023-05-20T01:21:24.925 DEBUG:teuthology.orchestra.run.smithi016:> path = "/home/ubuntu/cephtest/mnt.0/subdir/16" 2023-05-20T01:21:24.925 DEBUG:teuthology.orchestra.run.smithi016:> with open(path, '"'"'w'"'"') as f: 2023-05-20T01:21:24.925 DEBUG:teuthology.orchestra.run.smithi016:> for i in range(0, 6291456): 2023-05-20T01:21:24.925 DEBUG:teuthology.orchestra.run.smithi016:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-05-20T01:21:24.926 DEBUG:teuthology.orchestra.run.smithi016:> f.write(chr(val)) 2023-05-20T01:21:24.926 DEBUG:teuthology.orchestra.run.smithi016:> ' 2023-05-20T01:21:29.826 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/17 2023-05-20T01:21:29.827 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits daemon-helper kill python3 -c ' 2023-05-20T01:21:29.828 DEBUG:teuthology.orchestra.run.smithi016:> import zlib 2023-05-20T01:21:29.828 DEBUG:teuthology.orchestra.run.smithi016:> path = "/home/ubuntu/cephtest/mnt.0/subdir/17" 2023-05-20T01:21:29.829 DEBUG:teuthology.orchestra.run.smithi016:> with open(path, '"'"'w'"'"') as f: 2023-05-20T01:21:29.830 DEBUG:teuthology.orchestra.run.smithi016:> for i in range(0, 6291456): 2023-05-20T01:21:29.830 DEBUG:teuthology.orchestra.run.smithi016:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-05-20T01:21:29.830 DEBUG:teuthology.orchestra.run.smithi016:> f.write(chr(val)) 2023-05-20T01:21:29.831 DEBUG:teuthology.orchestra.run.smithi016:> ' 2023-05-20T01:21:34.729 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/18 2023-05-20T01:21:34.730 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits daemon-helper kill python3 -c ' 2023-05-20T01:21:34.730 DEBUG:teuthology.orchestra.run.smithi016:> import zlib 2023-05-20T01:21:34.731 DEBUG:teuthology.orchestra.run.smithi016:> path = "/home/ubuntu/cephtest/mnt.0/subdir/18" 2023-05-20T01:21:34.731 DEBUG:teuthology.orchestra.run.smithi016:> with open(path, '"'"'w'"'"') as f: 2023-05-20T01:21:34.731 DEBUG:teuthology.orchestra.run.smithi016:> for i in range(0, 6291456): 2023-05-20T01:21:34.731 DEBUG:teuthology.orchestra.run.smithi016:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-05-20T01:21:34.732 DEBUG:teuthology.orchestra.run.smithi016:> f.write(chr(val)) 2023-05-20T01:21:34.732 DEBUG:teuthology.orchestra.run.smithi016:> ' 2023-05-20T01:21:36.259 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:21:36.262 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:21:39.632 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/19 2023-05-20T01:21:39.632 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits daemon-helper kill python3 -c ' 2023-05-20T01:21:39.633 DEBUG:teuthology.orchestra.run.smithi016:> import zlib 2023-05-20T01:21:39.633 DEBUG:teuthology.orchestra.run.smithi016:> path = "/home/ubuntu/cephtest/mnt.0/subdir/19" 2023-05-20T01:21:39.633 DEBUG:teuthology.orchestra.run.smithi016:> with open(path, '"'"'w'"'"') as f: 2023-05-20T01:21:39.634 DEBUG:teuthology.orchestra.run.smithi016:> for i in range(0, 6291456): 2023-05-20T01:21:39.634 DEBUG:teuthology.orchestra.run.smithi016:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-05-20T01:21:39.634 DEBUG:teuthology.orchestra.run.smithi016:> f.write(chr(val)) 2023-05-20T01:21:39.634 DEBUG:teuthology.orchestra.run.smithi016:> ' 2023-05-20T01:21:44.496 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/20 2023-05-20T01:21:44.497 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits daemon-helper kill python3 -c ' 2023-05-20T01:21:44.497 DEBUG:teuthology.orchestra.run.smithi016:> import zlib 2023-05-20T01:21:44.498 DEBUG:teuthology.orchestra.run.smithi016:> path = "/home/ubuntu/cephtest/mnt.0/subdir/20" 2023-05-20T01:21:44.498 DEBUG:teuthology.orchestra.run.smithi016:> with open(path, '"'"'w'"'"') as f: 2023-05-20T01:21:44.498 DEBUG:teuthology.orchestra.run.smithi016:> for i in range(0, 6291456): 2023-05-20T01:21:44.498 DEBUG:teuthology.orchestra.run.smithi016:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-05-20T01:21:44.499 DEBUG:teuthology.orchestra.run.smithi016:> f.write(chr(val)) 2023-05-20T01:21:44.499 DEBUG:teuthology.orchestra.run.smithi016:> ' 2023-05-20T01:21:49.399 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/21 2023-05-20T01:21:49.400 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits daemon-helper kill python3 -c ' 2023-05-20T01:21:49.400 DEBUG:teuthology.orchestra.run.smithi016:> import zlib 2023-05-20T01:21:49.401 DEBUG:teuthology.orchestra.run.smithi016:> path = "/home/ubuntu/cephtest/mnt.0/subdir/21" 2023-05-20T01:21:49.401 DEBUG:teuthology.orchestra.run.smithi016:> with open(path, '"'"'w'"'"') as f: 2023-05-20T01:21:49.402 DEBUG:teuthology.orchestra.run.smithi016:> for i in range(0, 6291456): 2023-05-20T01:21:49.402 DEBUG:teuthology.orchestra.run.smithi016:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-05-20T01:21:49.402 DEBUG:teuthology.orchestra.run.smithi016:> f.write(chr(val)) 2023-05-20T01:21:49.402 DEBUG:teuthology.orchestra.run.smithi016:> ' 2023-05-20T01:21:54.303 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/22 2023-05-20T01:21:54.304 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits daemon-helper kill python3 -c ' 2023-05-20T01:21:54.304 DEBUG:teuthology.orchestra.run.smithi016:> import zlib 2023-05-20T01:21:54.305 DEBUG:teuthology.orchestra.run.smithi016:> path = "/home/ubuntu/cephtest/mnt.0/subdir/22" 2023-05-20T01:21:54.305 DEBUG:teuthology.orchestra.run.smithi016:> with open(path, '"'"'w'"'"') as f: 2023-05-20T01:21:54.306 DEBUG:teuthology.orchestra.run.smithi016:> for i in range(0, 6291456): 2023-05-20T01:21:54.306 DEBUG:teuthology.orchestra.run.smithi016:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-05-20T01:21:54.306 DEBUG:teuthology.orchestra.run.smithi016:> f.write(chr(val)) 2023-05-20T01:21:54.306 DEBUG:teuthology.orchestra.run.smithi016:> ' 2023-05-20T01:21:59.204 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/23 2023-05-20T01:21:59.205 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits daemon-helper kill python3 -c ' 2023-05-20T01:21:59.205 DEBUG:teuthology.orchestra.run.smithi016:> import zlib 2023-05-20T01:21:59.206 DEBUG:teuthology.orchestra.run.smithi016:> path = "/home/ubuntu/cephtest/mnt.0/subdir/23" 2023-05-20T01:21:59.206 DEBUG:teuthology.orchestra.run.smithi016:> with open(path, '"'"'w'"'"') as f: 2023-05-20T01:21:59.206 DEBUG:teuthology.orchestra.run.smithi016:> for i in range(0, 6291456): 2023-05-20T01:21:59.207 DEBUG:teuthology.orchestra.run.smithi016:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-05-20T01:21:59.207 DEBUG:teuthology.orchestra.run.smithi016:> f.write(chr(val)) 2023-05-20T01:21:59.207 DEBUG:teuthology.orchestra.run.smithi016:> ' 2023-05-20T01:22:04.107 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/24 2023-05-20T01:22:04.108 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits daemon-helper kill python3 -c ' 2023-05-20T01:22:04.108 DEBUG:teuthology.orchestra.run.smithi016:> import zlib 2023-05-20T01:22:04.109 DEBUG:teuthology.orchestra.run.smithi016:> path = "/home/ubuntu/cephtest/mnt.0/subdir/24" 2023-05-20T01:22:04.109 DEBUG:teuthology.orchestra.run.smithi016:> with open(path, '"'"'w'"'"') as f: 2023-05-20T01:22:04.109 DEBUG:teuthology.orchestra.run.smithi016:> for i in range(0, 6291456): 2023-05-20T01:22:04.110 DEBUG:teuthology.orchestra.run.smithi016:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-05-20T01:22:04.110 DEBUG:teuthology.orchestra.run.smithi016:> f.write(chr(val)) 2023-05-20T01:22:04.110 DEBUG:teuthology.orchestra.run.smithi016:> ' 2023-05-20T01:22:06.320 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:22:06.323 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:22:09.009 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-05-20T01:22:09.010 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2023-05-20T01:22:09.028 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-05-20T01:22:09.028 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2023-05-20T01:22:09.085 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-20T01:22:09.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.246+0000 7f122aa17700 1 Processor -- start 2023-05-20T01:22:09.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.246+0000 7f122aa17700 1 -- start start 2023-05-20T01:22:09.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.247+0000 7f122aa17700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1224144020 0x7f12240d1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:09.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.247+0000 7f122aa17700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12240cc500 con 0x7f12240b2760 2023-05-20T01:22:09.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.247+0000 7f122aa17700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12240cb8f0 con 0x7f12240cec60 2023-05-20T01:22:09.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.247+0000 7f122aa17700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12240cb6e0 con 0x7f1224144020 2023-05-20T01:22:09.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.247+0000 7f12237fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1224144020 0x7f12240d1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:09.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.247+0000 7f1223fff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f12240b2760 0x7f12240b0980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:45668/0 (socket says 172.21.15.16:45668) 2023-05-20T01:22:09.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.247+0000 7f1223fff700 1 -- 172.21.15.16:0/1613620354 learned_addr learned my addr 172.21.15.16:0/1613620354 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:22:09.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.247+0000 7f12237fe700 1 -- 172.21.15.16:0/1613620354 >> v1:172.21.15.132:6789/0 conn(0x7f12240cec60 legacy=0x7f12240cc990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:09.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.247+0000 7f12237fe700 1 -- 172.21.15.16:0/1613620354 >> v1:172.21.15.16:6789/0 conn(0x7f12240b2760 legacy=0x7f12240b0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:09.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.247+0000 7f12237fe700 1 -- 172.21.15.16:0/1613620354 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12240cbbd0 con 0x7f1224144020 2023-05-20T01:22:09.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.247+0000 7f1222ffd700 1 -- 172.21.15.16:0/1613620354 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2055138623 0 0) 0x7f12240cc500 con 0x7f12240b2760 2023-05-20T01:22:09.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.247+0000 7f12237fe700 1 --2- 172.21.15.16:0/1613620354 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1224144020 0x7f12240d1680 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f1218009840 tx=0x7f1218009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5bbb596f6340a6 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:09.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.248+0000 7f1222ffd700 1 -- 172.21.15.16:0/1613620354 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1218011830 con 0x7f1224144020 2023-05-20T01:22:09.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.248+0000 7f1222ffd700 1 -- 172.21.15.16:0/1613620354 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1218011990 con 0x7f1224144020 2023-05-20T01:22:09.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.248+0000 7f1222ffd700 1 -- 172.21.15.16:0/1613620354 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1218011d20 con 0x7f1224144020 2023-05-20T01:22:09.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.248+0000 7f122aa17700 1 -- 172.21.15.16:0/1613620354 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1224144020 msgr2=0x7f12240d1680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:09.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.248+0000 7f122aa17700 1 --2- 172.21.15.16:0/1613620354 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1224144020 0x7f12240d1680 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f1218009840 tx=0x7f1218009b50 comp rx=0 tx=0).stop 2023-05-20T01:22:09.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.248+0000 7f122aa17700 1 -- 172.21.15.16:0/1613620354 shutdown_connections 2023-05-20T01:22:09.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.248+0000 7f122aa17700 1 --2- 172.21.15.16:0/1613620354 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1224144020 0x7f12240d1680 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:09.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.248+0000 7f122aa17700 1 -- 172.21.15.16:0/1613620354 >> 172.21.15.16:0/1613620354 conn(0x7f122406ae30 msgr2=0x7f122409a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:09.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.248+0000 7f122aa17700 1 -- 172.21.15.16:0/1613620354 shutdown_connections 2023-05-20T01:22:09.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.248+0000 7f122aa17700 1 -- 172.21.15.16:0/1613620354 wait complete. 2023-05-20T01:22:09.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.249+0000 7f122aa17700 1 Processor -- start 2023-05-20T01:22:09.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.250+0000 7f122aa17700 1 -- start start 2023-05-20T01:22:09.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.250+0000 7f122aa17700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f12240b2760 0x7f12241f0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:09.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.250+0000 7f122aa17700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12240cec60 0x7f12241efa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:09.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.250+0000 7f122aa17700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1224144020 0x7f12241f3820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:09.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.250+0000 7f122aa17700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12241f11a0 con 0x7f12240cec60 2023-05-20T01:22:09.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.250+0000 7f122aa17700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12241f4270 con 0x7f1224144020 2023-05-20T01:22:09.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.250+0000 7f122aa17700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12241f4550 con 0x7f12240b2760 2023-05-20T01:22:09.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.250+0000 7f12237fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12240cec60 0x7f12241efa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:09.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.250+0000 7f12237fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12240cec60 0x7f12241efa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34148/0 (socket says 172.21.15.16:34148) 2023-05-20T01:22:09.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.250+0000 7f12237fe700 1 -- 172.21.15.16:0/4093445423 learned_addr learned my addr 172.21.15.16:0/4093445423 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:22:09.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.250+0000 7f1223fff700 1 --2- 172.21.15.16:0/4093445423 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f12240b2760 0x7f12241f0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:09.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.250+0000 7f1228fb4700 1 --2- 172.21.15.16:0/4093445423 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1224144020 0x7f12241f3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:09.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.251+0000 7f12237fe700 1 -- 172.21.15.16:0/4093445423 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f12240b2760 msgr2=0x7f12241f0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:09.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.251+0000 7f12237fe700 1 --2- 172.21.15.16:0/4093445423 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f12240b2760 0x7f12241f0c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:09.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.251+0000 7f12237fe700 1 -- 172.21.15.16:0/4093445423 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1224144020 msgr2=0x7f12241f3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:09.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.251+0000 7f12237fe700 1 --2- 172.21.15.16:0/4093445423 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1224144020 0x7f12241f3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:09.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.251+0000 7f12237fe700 1 -- 172.21.15.16:0/4093445423 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12180094a0 con 0x7f12240cec60 2023-05-20T01:22:09.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.251+0000 7f12237fe700 1 --2- 172.21.15.16:0/4093445423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12240cec60 0x7f12241efa60 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f1218014040 tx=0x7f1218004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:09.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.251+0000 7f1220ff9700 1 -- 172.21.15.16:0/4093445423 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f121801a7e0 con 0x7f12240cec60 2023-05-20T01:22:09.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.251+0000 7f1220ff9700 1 -- 172.21.15.16:0/4093445423 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f121801a940 con 0x7f12240cec60 2023-05-20T01:22:09.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.251+0000 7f122aa17700 1 -- 172.21.15.16:0/4093445423 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12240cc190 con 0x7f12240cec60 2023-05-20T01:22:09.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.251+0000 7f1220ff9700 1 -- 172.21.15.16:0/4093445423 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f121801acb0 con 0x7f12240cec60 2023-05-20T01:22:09.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.251+0000 7f122aa17700 1 -- 172.21.15.16:0/4093445423 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12240cbeb0 con 0x7f12240cec60 2023-05-20T01:22:09.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.252+0000 7f122aa17700 1 -- 172.21.15.16:0/4093445423 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12240cf0c0 con 0x7f12240cec60 2023-05-20T01:22:09.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.253+0000 7f1220ff9700 1 -- 172.21.15.16:0/4093445423 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1218022070 con 0x7f12240cec60 2023-05-20T01:22:09.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.253+0000 7f1220ff9700 1 --2- 172.21.15.16:0/4093445423 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f121006ace0 0x7f121006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:09.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.253+0000 7f1220ff9700 1 -- 172.21.15.16:0/4093445423 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6931+0+0 (secure 0 0 0) 0x7f121808a980 con 0x7f12240cec60 2023-05-20T01:22:09.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.253+0000 7f1223fff700 1 --2- 172.21.15.16:0/4093445423 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f121006ace0 0x7f121006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:09.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.254+0000 7f1223fff700 1 --2- 172.21.15.16:0/4093445423 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f121006ace0 0x7f121006d190 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f1214007b10 tx=0x7f1214007a60 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:09.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.258+0000 7f1220ff9700 1 -- 172.21.15.16:0/4093445423 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f1218055730 con 0x7f12240cec60 2023-05-20T01:22:09.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.458+0000 7f122aa17700 1 -- 172.21.15.16:0/4093445423 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f12240747b0 con 0x7f12240cec60 2023-05-20T01:22:09.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.459+0000 7f1220ff9700 1 -- 172.21.15.16:0/4093445423 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v69) v1 ==== 74+0+20433 (secure 0 0 0) 0x7f1218058d50 con 0x7f12240cec60 2023-05-20T01:22:09.463 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:22:09.464 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":69,"fsid":"c2245cc5-ecab-4087-a59b-bc81db901676","created":"2023-05-20T01:16:09.549230+0000","modified":"2023-05-20T01:20:17.425794+0000","last_up_change":"2023-05-20T01:16:19.788624+0000","last_in_change":"2023-05-20T01:16:12.092160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-05-20T01:16:20.934366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-05-20T01:16:21.372294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-05-20T01:19:49.878849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-05-20T01:19:50.887306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data_extra","create_time":"2023-05-20T01:20:04.581773+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"317a83cd-19c8-4958-a301-6087ef342289","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6825","nonce":3923634179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6827","nonce":3923634179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6831","nonce":3923634179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6829","nonce":3923634179}]},"public_addr":"172.21.15.16:6825/3923634179","cluster_addr":"172.21.15.16:6827/3923634179","heartbeat_back_addr":"172.21.15.16:6831/3923634179","heartbeat_front_addr":"172.21.15.16:6829/3923634179","state":["exists","up"]},{"osd":1,"uuid":"5581046f-8219-4089-8d7d-b7846399505e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6809","nonce":3860264220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6811","nonce":3860264220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6815","nonce":3860264220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6813","nonce":3860264220}]},"public_addr":"172.21.15.16:6809/3860264220","cluster_addr":"172.21.15.16:6811/3860264220","heartbeat_back_addr":"172.21.15.16:6815/3860264220","heartbeat_front_addr":"172.21.15.16:6813/3860264220","state":["exists","up"]},{"osd":2,"uuid":"072e9e66-4133-478e-8d4a-64d46c2963f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6817","nonce":293187030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6819","nonce":293187030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6823","nonce":293187030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6821","nonce":293187030}]},"public_addr":"172.21.15.16:6817/293187030","cluster_addr":"172.21.15.16:6819/293187030","heartbeat_back_addr":"172.21.15.16:6823/293187030","heartbeat_front_addr":"172.21.15.16:6821/293187030","state":["exists","up"]},{"osd":3,"uuid":"11e1bba6-00f8-48dd-9028-d8d716b7f145","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6801","nonce":3669781890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6803","nonce":3669781890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6807","nonce":3669781890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6805","nonce":3669781890}]},"public_addr":"172.21.15.16:6801/3669781890","cluster_addr":"172.21.15.16:6803/3669781890","heartbeat_back_addr":"172.21.15.16:6807/3669781890","heartbeat_front_addr":"172.21.15.16:6805/3669781890","state":["exists","up"]},{"osd":4,"uuid":"b8909c3a-cf70-46cb-8d0b-009c707cf9d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6819","nonce":506317223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6823","nonce":506317223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6829","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6831","nonce":506317223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6827","nonce":506317223}]},"public_addr":"172.21.15.132:6819/506317223","cluster_addr":"172.21.15.132:6823/506317223","heartbeat_back_addr":"172.21.15.132:6831/506317223","heartbeat_front_addr":"172.21.15.132:6827/506317223","state":["exists","up"]},{"osd":5,"uuid":"b1818304-4583-47bf-a6ec-0bfbb5166233","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6818","nonce":1844689995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6822","nonce":1844689995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6830","nonce":1844689995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6826","nonce":1844689995}]},"public_addr":"172.21.15.132:6818/1844689995","cluster_addr":"172.21.15.132:6822/1844689995","heartbeat_back_addr":"172.21.15.132:6830/1844689995","heartbeat_front_addr":"172.21.15.132:6826/1844689995","state":["exists","up"]},{"osd":6,"uuid":"d37f4691-eed1-4a93-8818-18354f03f4a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6801","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6802","nonce":727581123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6805","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6807","nonce":727581123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6813","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6815","nonce":727581123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6811","nonce":727581123}]},"public_addr":"172.21.15.132:6802/727581123","cluster_addr":"172.21.15.132:6807/727581123","heartbeat_back_addr":"172.21.15.132:6815/727581123","heartbeat_front_addr":"172.21.15.132:6811/727581123","state":["exists","up"]},{"osd":7,"uuid":"4f712f62-5d1d-4816-8042-03209144f8b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6803","nonce":466001149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6806","nonce":466001149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6814","nonce":466001149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6810","nonce":466001149}]},"public_addr":"172.21.15.132:6803/466001149","cluster_addr":"172.21.15.132:6806/466001149","heartbeat_back_addr":"172.21.15.132:6814/466001149","heartbeat_front_addr":"172.21.15.132:6810/466001149","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.338863+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.278680+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305694+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.246835+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.276237+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.304441+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305421+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.267050+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.d","mappings":[{"from":7,"to":2}]},{"pgid":"10.10","mappings":[{"from":4,"to":3}]},{"pgid":"10.2e","mappings":[{"from":4,"to":3}]},{"pgid":"10.34","mappings":[{"from":7,"to":3}]},{"pgid":"10.37","mappings":[{"from":4,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-20T01:22:09.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.461+0000 7f122aa17700 1 -- 172.21.15.16:0/4093445423 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f121006ace0 msgr2=0x7f121006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:09.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.461+0000 7f122aa17700 1 --2- 172.21.15.16:0/4093445423 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f121006ace0 0x7f121006d190 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f1214007b10 tx=0x7f1214007a60 comp rx=0 tx=0).stop 2023-05-20T01:22:09.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.461+0000 7f122aa17700 1 -- 172.21.15.16:0/4093445423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12240cec60 msgr2=0x7f12241efa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:09.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.461+0000 7f122aa17700 1 --2- 172.21.15.16:0/4093445423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12240cec60 0x7f12241efa60 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f1218014040 tx=0x7f1218004750 comp rx=0 tx=0).stop 2023-05-20T01:22:09.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.461+0000 7f122aa17700 1 -- 172.21.15.16:0/4093445423 shutdown_connections 2023-05-20T01:22:09.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.461+0000 7f122aa17700 1 --2- 172.21.15.16:0/4093445423 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1224144020 0x7f12241f3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:09.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.461+0000 7f122aa17700 1 --2- 172.21.15.16:0/4093445423 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f121006ace0 0x7f121006d190 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:09.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.461+0000 7f122aa17700 1 --2- 172.21.15.16:0/4093445423 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f12240b2760 0x7f12241f0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:09.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.461+0000 7f122aa17700 1 --2- 172.21.15.16:0/4093445423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12240cec60 0x7f12241efa60 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:09.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.461+0000 7f122aa17700 1 -- 172.21.15.16:0/4093445423 >> 172.21.15.16:0/4093445423 conn(0x7f122406ae30 msgr2=0x7f12240add40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:09.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.461+0000 7f122aa17700 1 -- 172.21.15.16:0/4093445423 shutdown_connections 2023-05-20T01:22:09.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.462+0000 7f122aa17700 1 -- 172.21.15.16:0/4093445423 wait complete. 2023-05-20T01:22:09.478 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-05-20T01:16:20.934366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-05-20T01:16:21.372294+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2023-05-20T01:19:49.878849+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '63', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2023-05-20T01:19:50.887306+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '64', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data_extra', 'create_time': '2023-05-20T01:20:04.581773+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '67', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-05-20T01:22:09.478 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-05-20T01:22:09.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.603+0000 7f4186d90700 1 Processor -- start 2023-05-20T01:22:09.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.603+0000 7f4186d90700 1 -- start start 2023-05-20T01:22:09.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.604+0000 7f4186d90700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f41800cd070 0x7f418006e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:09.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.604+0000 7f4186d90700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4180072430 con 0x7f41800710d0 2023-05-20T01:22:09.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.604+0000 7f4186d90700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4180071570 con 0x7f418015cf40 2023-05-20T01:22:09.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.604+0000 7f4186d90700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4180071820 con 0x7f41800cd070 2023-05-20T01:22:09.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.604+0000 7f418532d700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f41800710d0 0x7f41800cabb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:45686/0 (socket says 172.21.15.16:45686) 2023-05-20T01:22:09.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.604+0000 7f418532d700 1 -- 172.21.15.16:0/3176890162 learned_addr learned my addr 172.21.15.16:0/3176890162 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:22:09.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.604+0000 7f417ffff700 1 --2- 172.21.15.16:0/3176890162 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f41800cd070 0x7f418006e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:09.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.604+0000 7f417f7fe700 1 -- 172.21.15.16:0/3176890162 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4244275107 0 0) 0x7f4180072430 con 0x7f41800710d0 2023-05-20T01:22:09.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.604+0000 7f417f7fe700 1 -- 172.21.15.16:0/3176890162 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41700035b0 con 0x7f41800710d0 2023-05-20T01:22:09.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.604+0000 7f417f7fe700 1 -- 172.21.15.16:0/3176890162 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4032217423 0 0) 0x7f41700035b0 con 0x7f41800710d0 2023-05-20T01:22:09.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.604+0000 7f417f7fe700 1 -- 172.21.15.16:0/3176890162 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f41800cd070 msgr2=0x7f418006e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:09.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.604+0000 7f417f7fe700 1 --2- 172.21.15.16:0/3176890162 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f41800cd070 0x7f418006e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:09.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.604+0000 7f417f7fe700 1 -- 172.21.15.16:0/3176890162 >> v1:172.21.15.132:6789/0 conn(0x7f418015cf40 legacy=0x7f41800cc960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:09.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.604+0000 7f417f7fe700 1 -- 172.21.15.16:0/3176890162 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4180071b00 con 0x7f41800710d0 2023-05-20T01:22:09.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.604+0000 7f417f7fe700 1 -- 172.21.15.16:0/3176890162 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7f41780019f0 con 0x7f41800710d0 2023-05-20T01:22:09.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.605+0000 7f417f7fe700 1 -- 172.21.15.16:0/3176890162 >> v1:172.21.15.16:6789/0 conn(0x7f41800710d0 legacy=0x7f41800cabb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:09.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.605+0000 7f417f7fe700 1 --2- 172.21.15.16:0/3176890162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4170003a30 0x7f4170003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:09.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.605+0000 7f417f7fe700 1 --2- 172.21.15.16:0/3176890162 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f41800cd070 0x7f4170004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:09.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.605+0000 7f417f7fe700 1 --2- 172.21.15.16:0/3176890162 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4170004d90 0x7f4170005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:09.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.605+0000 7f417f7fe700 1 -- 172.21.15.16:0/3176890162 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4180071820 con 0x7f4170003a30 2023-05-20T01:22:09.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.605+0000 7f417f7fe700 1 -- 172.21.15.16:0/3176890162 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f417000bac0 con 0x7f4170004d90 2023-05-20T01:22:09.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.605+0000 7f417f7fe700 1 -- 172.21.15.16:0/3176890162 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f417000de60 con 0x7f41800cd070 2023-05-20T01:22:09.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.605+0000 7f4184b2c700 1 --2- 172.21.15.16:0/3176890162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4170003a30 0x7f4170003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:09.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.605+0000 7f417ffff700 1 --2- 172.21.15.16:0/3176890162 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f41800cd070 0x7f4170004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:09.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.605+0000 7f418532d700 1 --2- 172.21.15.16:0/3176890162 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4170004d90 0x7f4170005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:09.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.605+0000 7f4184b2c700 1 -- 172.21.15.16:0/3176890162 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f41800cd070 msgr2=0x7f4170004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:09.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.605+0000 7f4184b2c700 1 --2- 172.21.15.16:0/3176890162 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f41800cd070 0x7f4170004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:09.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.605+0000 7f4184b2c700 1 -- 172.21.15.16:0/3176890162 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4170004d90 msgr2=0x7f4170005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:09.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.605+0000 7f4184b2c700 1 --2- 172.21.15.16:0/3176890162 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4170004d90 0x7f4170005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:09.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.605+0000 7f4184b2c700 1 -- 172.21.15.16:0/3176890162 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41700035b0 con 0x7f4170003a30 2023-05-20T01:22:09.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.605+0000 7f4184b2c700 1 --2- 172.21.15.16:0/3176890162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4170003a30 0x7f4170003e40 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f41740017c0 tx=0x7f4170005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1019194b7f6794d server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:09.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.605+0000 7f417f7fe700 1 -- 172.21.15.16:0/3176890162 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41740052d0 con 0x7f4170003a30 2023-05-20T01:22:09.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.605+0000 7f417f7fe700 1 -- 172.21.15.16:0/3176890162 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4174005430 con 0x7f4170003a30 2023-05-20T01:22:09.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.605+0000 7f417f7fe700 1 -- 172.21.15.16:0/3176890162 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41740038c0 con 0x7f4170003a30 2023-05-20T01:22:09.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.606+0000 7f4186d90700 1 -- 172.21.15.16:0/3176890162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4170003a30 msgr2=0x7f4170003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:09.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.606+0000 7f4186d90700 1 --2- 172.21.15.16:0/3176890162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4170003a30 0x7f4170003e40 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f41740017c0 tx=0x7f4170005880 comp rx=0 tx=0).stop 2023-05-20T01:22:09.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.606+0000 7f418532d700 1 -- 172.21.15.16:0/3176890162 reap_dead start 2023-05-20T01:22:09.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.606+0000 7f4186d90700 1 -- 172.21.15.16:0/3176890162 shutdown_connections 2023-05-20T01:22:09.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.606+0000 7f4186d90700 1 -- 172.21.15.16:0/3176890162 >> 172.21.15.16:0/3176890162 conn(0x7f418006c260 msgr2=0x7f41800cffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:09.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.606+0000 7f4186d90700 1 -- 172.21.15.16:0/3176890162 shutdown_connections 2023-05-20T01:22:09.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.606+0000 7f4186d90700 1 -- 172.21.15.16:0/3176890162 wait complete. 2023-05-20T01:22:09.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.607+0000 7f4186d90700 1 Processor -- start 2023-05-20T01:22:09.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.607+0000 7f4186d90700 1 -- start start 2023-05-20T01:22:09.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.608+0000 7f4186d90700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4170003a30 0x7f41800b57f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:09.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.608+0000 7f4186d90700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f41800b1d20 0x7f41800cbc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:09.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.608+0000 7f4186d90700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f41800b2190 0x7f41800cc150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:09.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.608+0000 7f4186d90700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41800b0dd0 con 0x7f4170003a30 2023-05-20T01:22:09.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.608+0000 7f4186d90700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41800b5d30 con 0x7f41800b2190 2023-05-20T01:22:09.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.608+0000 7f4186d90700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41800c5040 con 0x7f41800b1d20 2023-05-20T01:22:09.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.608+0000 7f4184b2c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4170003a30 0x7f41800b57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:09.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.608+0000 7f4184b2c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4170003a30 0x7f41800b57f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34166/0 (socket says 172.21.15.16:34166) 2023-05-20T01:22:09.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.608+0000 7f4184b2c700 1 -- 172.21.15.16:0/3376568057 learned_addr learned my addr 172.21.15.16:0/3376568057 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:22:09.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.608+0000 7f417ffff700 1 --2- 172.21.15.16:0/3376568057 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f41800b1d20 0x7f41800cbc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:09.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.608+0000 7f418532d700 1 --2- 172.21.15.16:0/3376568057 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f41800b2190 0x7f41800cc150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:09.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.608+0000 7f4184b2c700 1 -- 172.21.15.16:0/3376568057 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f41800b1d20 msgr2=0x7f41800cbc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:09.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.608+0000 7f4184b2c700 1 --2- 172.21.15.16:0/3376568057 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f41800b1d20 0x7f41800cbc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:09.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.608+0000 7f4184b2c700 1 -- 172.21.15.16:0/3376568057 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f41800b2190 msgr2=0x7f41800cc150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:09.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.608+0000 7f4184b2c700 1 --2- 172.21.15.16:0/3376568057 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f41800b2190 0x7f41800cc150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:09.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.608+0000 7f4184b2c700 1 -- 172.21.15.16:0/3376568057 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41800720c0 con 0x7f4170003a30 2023-05-20T01:22:09.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.609+0000 7f4184b2c700 1 --2- 172.21.15.16:0/3376568057 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4170003a30 0x7f41800b57f0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f41700047a0 tx=0x7f4174001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:09.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.609+0000 7f417d7fa700 1 -- 172.21.15.16:0/3376568057 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4174001290 con 0x7f4170003a30 2023-05-20T01:22:09.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.609+0000 7f417d7fa700 1 -- 172.21.15.16:0/3376568057 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4174026460 con 0x7f4170003a30 2023-05-20T01:22:09.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.609+0000 7f4186d90700 1 -- 172.21.15.16:0/3376568057 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4180071de0 con 0x7f4170003a30 2023-05-20T01:22:09.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.609+0000 7f417d7fa700 1 -- 172.21.15.16:0/3376568057 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4174026810 con 0x7f4170003a30 2023-05-20T01:22:09.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.609+0000 7f4186d90700 1 -- 172.21.15.16:0/3376568057 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f418006e020 con 0x7f4170003a30 2023-05-20T01:22:09.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.610+0000 7f4186d90700 1 -- 172.21.15.16:0/3376568057 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41800747b0 con 0x7f4170003a30 2023-05-20T01:22:09.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.610+0000 7f417d7fa700 1 -- 172.21.15.16:0/3376568057 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4174019070 con 0x7f4170003a30 2023-05-20T01:22:09.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.611+0000 7f417d7fa700 1 --2- 172.21.15.16:0/3376568057 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f416406ad10 0x7f416406d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:09.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.611+0000 7f417ffff700 1 --2- 172.21.15.16:0/3376568057 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f416406ad10 0x7f416406d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:09.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.611+0000 7f417d7fa700 1 -- 172.21.15.16:0/3376568057 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6931+0+0 (secure 0 0 0) 0x7f417408a580 con 0x7f4170003a30 2023-05-20T01:22:09.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.611+0000 7f417ffff700 1 --2- 172.21.15.16:0/3376568057 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f416406ad10 0x7f416406d1c0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f41800cca30 tx=0x7f416c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:09.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.615+0000 7f417d7fa700 1 -- 172.21.15.16:0/3376568057 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f41740552b0 con 0x7f4170003a30 2023-05-20T01:22:09.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.810+0000 7f4186d90700 1 -- 172.21.15.16:0/3376568057 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f4180158900 con 0x7f4170003a30 2023-05-20T01:22:09.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.810+0000 7f417d7fa700 1 -- 172.21.15.16:0/3376568057 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v69) v1 ==== 92+0+10 (secure 0 0 0) 0x7f41740170a0 con 0x7f4170003a30 2023-05-20T01:22:09.815 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 8 2023-05-20T01:22:09.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.812+0000 7f4186d90700 1 -- 172.21.15.16:0/3376568057 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f416406ad10 msgr2=0x7f416406d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:09.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.812+0000 7f4186d90700 1 --2- 172.21.15.16:0/3376568057 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f416406ad10 0x7f416406d1c0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f41800cca30 tx=0x7f416c008040 comp rx=0 tx=0).stop 2023-05-20T01:22:09.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.812+0000 7f4186d90700 1 -- 172.21.15.16:0/3376568057 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4170003a30 msgr2=0x7f41800b57f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:09.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.812+0000 7f4186d90700 1 --2- 172.21.15.16:0/3376568057 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4170003a30 0x7f41800b57f0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f41700047a0 tx=0x7f4174001120 comp rx=0 tx=0).stop 2023-05-20T01:22:09.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.812+0000 7f4186d90700 1 -- 172.21.15.16:0/3376568057 shutdown_connections 2023-05-20T01:22:09.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.812+0000 7f4186d90700 1 --2- 172.21.15.16:0/3376568057 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f41800b2190 0x7f41800cc150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:09.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.812+0000 7f4186d90700 1 --2- 172.21.15.16:0/3376568057 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4170003a30 0x7f41800b57f0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:09.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.812+0000 7f4186d90700 1 --2- 172.21.15.16:0/3376568057 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f416406ad10 0x7f416406d1c0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:09.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.812+0000 7f4186d90700 1 --2- 172.21.15.16:0/3376568057 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f41800b1d20 0x7f41800cbc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:09.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.812+0000 7f4186d90700 1 -- 172.21.15.16:0/3376568057 >> 172.21.15.16:0/3376568057 conn(0x7f418006c260 msgr2=0x7f418015a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:09.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.812+0000 7f4186d90700 1 -- 172.21.15.16:0/3376568057 shutdown_connections 2023-05-20T01:22:09.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.812+0000 7f4186d90700 1 -- 172.21.15.16:0/3376568057 wait complete. 2023-05-20T01:22:09.828 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-05-20T01:22:09.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.952+0000 7f08505da700 1 Processor -- start 2023-05-20T01:22:09.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.952+0000 7f08505da700 1 -- start start 2023-05-20T01:22:09.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.953+0000 7f08505da700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f08480cd620 0x7f08480cb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:09.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.953+0000 7f08505da700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f08480cbde0 0x7f08480b2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:09.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.953+0000 7f08505da700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08480d0e10 con 0x7f0848071880 2023-05-20T01:22:09.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.953+0000 7f08505da700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08480cd200 con 0x7f08480cd620 2023-05-20T01:22:09.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.953+0000 7f08505da700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08480cc580 con 0x7f08480cbde0 2023-05-20T01:22:09.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.953+0000 7f084e376700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f0848071880 0x7f0848071c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:45702/0 (socket says 172.21.15.16:45702) 2023-05-20T01:22:09.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.953+0000 7f084e376700 1 -- 172.21.15.16:0/1113016110 learned_addr learned my addr 172.21.15.16:0/1113016110 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:22:09.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.953+0000 7f084db75700 1 --2- 172.21.15.16:0/1113016110 >> v2:172.21.15.132:3300/0 conn(0x7f08480cd620 0x7f08480cb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:09.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.953+0000 7f084eb77700 1 --2- 172.21.15.16:0/1113016110 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f08480cbde0 0x7f08480b2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:09.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.953+0000 7f084d374700 1 -- 172.21.15.16:0/1113016110 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 998141233 0 0) 0x7f08480d0e10 con 0x7f0848071880 2023-05-20T01:22:09.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.953+0000 7f084d374700 1 -- 172.21.15.16:0/1113016110 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f08340035b0 con 0x7f0848071880 2023-05-20T01:22:09.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.953+0000 7f084db75700 1 -- 172.21.15.16:0/1113016110 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f08480cbde0 msgr2=0x7f08480b2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:09.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.953+0000 7f084db75700 1 --2- 172.21.15.16:0/1113016110 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f08480cbde0 0x7f08480b2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:09.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.953+0000 7f084db75700 1 -- 172.21.15.16:0/1113016110 >> v1:172.21.15.16:6789/0 conn(0x7f0848071880 legacy=0x7f0848071c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:09.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.953+0000 7f084db75700 1 -- 172.21.15.16:0/1113016110 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08480cc860 con 0x7f08480cd620 2023-05-20T01:22:09.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.953+0000 7f084db75700 1 --2- 172.21.15.16:0/1113016110 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f08480cd620 0x7f08480cb8a0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f0844009840 tx=0x7f0844009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d31269616f42c3ae server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:09.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.954+0000 7f084d374700 1 -- 172.21.15.16:0/1113016110 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0844011830 con 0x7f08480cd620 2023-05-20T01:22:09.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.954+0000 7f084d374700 1 -- 172.21.15.16:0/1113016110 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0844011990 con 0x7f08480cd620 2023-05-20T01:22:09.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.954+0000 7f084d374700 1 -- 172.21.15.16:0/1113016110 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0844011d20 con 0x7f08480cd620 2023-05-20T01:22:09.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.954+0000 7f08505da700 1 -- 172.21.15.16:0/1113016110 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f08480cd620 msgr2=0x7f08480cb8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:09.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.954+0000 7f08505da700 1 --2- 172.21.15.16:0/1113016110 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f08480cd620 0x7f08480cb8a0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f0844009840 tx=0x7f0844009b50 comp rx=0 tx=0).stop 2023-05-20T01:22:09.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.954+0000 7f08505da700 1 -- 172.21.15.16:0/1113016110 shutdown_connections 2023-05-20T01:22:09.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.954+0000 7f08505da700 1 --2- 172.21.15.16:0/1113016110 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f08480cbde0 0x7f08480b2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:09.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.954+0000 7f08505da700 1 --2- 172.21.15.16:0/1113016110 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f08480cd620 0x7f08480cb8a0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:09.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.954+0000 7f08505da700 1 -- 172.21.15.16:0/1113016110 >> 172.21.15.16:0/1113016110 conn(0x7f08480cffd0 msgr2=0x7f08480d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:09.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.954+0000 7f08505da700 1 -- 172.21.15.16:0/1113016110 shutdown_connections 2023-05-20T01:22:09.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.954+0000 7f08505da700 1 -- 172.21.15.16:0/1113016110 wait complete. 2023-05-20T01:22:09.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.955+0000 7f08505da700 1 Processor -- start 2023-05-20T01:22:09.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.955+0000 7f08505da700 1 -- start start 2023-05-20T01:22:09.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.956+0000 7f08505da700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0848071880 0x7f084814edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:09.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.956+0000 7f08505da700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08480cbde0 0x7f08480b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:09.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.956+0000 7f08505da700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f08480cd620 0x7f08480c88f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:09.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.956+0000 7f08505da700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f084814f300 con 0x7f08480cbde0 2023-05-20T01:22:09.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.956+0000 7f08505da700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08480afc50 con 0x7f08480cd620 2023-05-20T01:22:09.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.956+0000 7f08505da700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08480aff30 con 0x7f0848071880 2023-05-20T01:22:09.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.956+0000 7f084db75700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08480cbde0 0x7f08480b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:09.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.956+0000 7f084db75700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08480cbde0 0x7f08480b2120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34182/0 (socket says 172.21.15.16:34182) 2023-05-20T01:22:09.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.956+0000 7f084db75700 1 -- 172.21.15.16:0/2395679699 learned_addr learned my addr 172.21.15.16:0/2395679699 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:22:09.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.956+0000 7f084eb77700 1 --2- 172.21.15.16:0/2395679699 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f08480cd620 0x7f08480c88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:09.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.956+0000 7f084e376700 1 --2- 172.21.15.16:0/2395679699 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0848071880 0x7f084814edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:09.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.957+0000 7f084db75700 1 -- 172.21.15.16:0/2395679699 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0848071880 msgr2=0x7f084814edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:09.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.957+0000 7f084db75700 1 --2- 172.21.15.16:0/2395679699 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0848071880 0x7f084814edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:09.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.957+0000 7f084db75700 1 -- 172.21.15.16:0/2395679699 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f08480cd620 msgr2=0x7f08480c88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:09.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.957+0000 7f084db75700 1 --2- 172.21.15.16:0/2395679699 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f08480cd620 0x7f08480c88f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:09.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.957+0000 7f084db75700 1 -- 172.21.15.16:0/2395679699 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08440094a0 con 0x7f08480cbde0 2023-05-20T01:22:09.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.957+0000 7f084e376700 1 --2- 172.21.15.16:0/2395679699 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0848071880 0x7f084814edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:22:09.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.957+0000 7f084db75700 1 --2- 172.21.15.16:0/2395679699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08480cbde0 0x7f08480b2120 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f08440090b0 tx=0x7f0844004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:09.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.957+0000 7f083affd700 1 -- 172.21.15.16:0/2395679699 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08440036a0 con 0x7f08480cbde0 2023-05-20T01:22:09.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.957+0000 7f083affd700 1 -- 172.21.15.16:0/2395679699 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f084401f070 con 0x7f08480cbde0 2023-05-20T01:22:09.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.957+0000 7f08505da700 1 -- 172.21.15.16:0/2395679699 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08480cce20 con 0x7f08480cbde0 2023-05-20T01:22:09.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.957+0000 7f083affd700 1 -- 172.21.15.16:0/2395679699 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0844003a60 con 0x7f08480cbde0 2023-05-20T01:22:09.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.957+0000 7f08505da700 1 -- 172.21.15.16:0/2395679699 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08480ccb40 con 0x7f08480cbde0 2023-05-20T01:22:09.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.958+0000 7f08505da700 1 -- 172.21.15.16:0/2395679699 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08480cc260 con 0x7f08480cbde0 2023-05-20T01:22:09.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.958+0000 7f083affd700 1 -- 172.21.15.16:0/2395679699 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0844003bc0 con 0x7f08480cbde0 2023-05-20T01:22:09.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.959+0000 7f083affd700 1 --2- 172.21.15.16:0/2395679699 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f082c06ad10 0x7f082c06d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:09.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.959+0000 7f083affd700 1 -- 172.21.15.16:0/2395679699 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6931+0+0 (secure 0 0 0) 0x7f0844021070 con 0x7f08480cbde0 2023-05-20T01:22:09.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.959+0000 7f084e376700 1 --2- 172.21.15.16:0/2395679699 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f082c06ad10 0x7f082c06d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:09.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.959+0000 7f084e376700 1 --2- 172.21.15.16:0/2395679699 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f082c06ad10 0x7f082c06d1c0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f083c0010e0 tx=0x7f083c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:09.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:09.963+0000 7f083affd700 1 -- 172.21.15.16:0/2395679699 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f0844055d30 con 0x7f08480cbde0 2023-05-20T01:22:10.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.159+0000 7f08505da700 1 -- 172.21.15.16:0/2395679699 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f08480747b0 con 0x7f08480cbde0 2023-05-20T01:22:10.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.159+0000 7f083affd700 1 -- 172.21.15.16:0/2395679699 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v69) v1 ==== 93+0+10 (secure 0 0 0) 0x7f08440150a0 con 0x7f08480cbde0 2023-05-20T01:22:10.164 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-05-20T01:22:10.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.161+0000 7f08505da700 1 -- 172.21.15.16:0/2395679699 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f082c06ad10 msgr2=0x7f082c06d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:10.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.161+0000 7f08505da700 1 --2- 172.21.15.16:0/2395679699 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f082c06ad10 0x7f082c06d1c0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f083c0010e0 tx=0x7f083c009040 comp rx=0 tx=0).stop 2023-05-20T01:22:10.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.161+0000 7f08505da700 1 -- 172.21.15.16:0/2395679699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08480cbde0 msgr2=0x7f08480b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:10.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.161+0000 7f08505da700 1 --2- 172.21.15.16:0/2395679699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08480cbde0 0x7f08480b2120 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f08440090b0 tx=0x7f0844004750 comp rx=0 tx=0).stop 2023-05-20T01:22:10.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.162+0000 7f08505da700 1 -- 172.21.15.16:0/2395679699 shutdown_connections 2023-05-20T01:22:10.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.162+0000 7f08505da700 1 --2- 172.21.15.16:0/2395679699 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f08480cd620 0x7f08480c88f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:10.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.162+0000 7f08505da700 1 --2- 172.21.15.16:0/2395679699 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f082c06ad10 0x7f082c06d1c0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:10.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.162+0000 7f08505da700 1 --2- 172.21.15.16:0/2395679699 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0848071880 0x7f084814edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:10.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.162+0000 7f08505da700 1 --2- 172.21.15.16:0/2395679699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08480cbde0 0x7f08480b2120 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:10.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.162+0000 7f08505da700 1 -- 172.21.15.16:0/2395679699 >> 172.21.15.16:0/2395679699 conn(0x7f08480cffd0 msgr2=0x7f0848071c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:10.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.162+0000 7f08505da700 1 -- 172.21.15.16:0/2395679699 shutdown_connections 2023-05-20T01:22:10.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.162+0000 7f08505da700 1 -- 172.21.15.16:0/2395679699 wait complete. 2023-05-20T01:22:10.177 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-05-20T01:22:10.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.301+0000 7f29026d3700 1 Processor -- start 2023-05-20T01:22:10.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.301+0000 7f29026d3700 1 -- start start 2023-05-20T01:22:10.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.301+0000 7f29026d3700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f28fc15ab80 0x7f28fc0cda80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:10.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.301+0000 7f29026d3700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f28fc0cf390 0x7f28fc0cc110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:10.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.301+0000 7f29026d3700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28fc0ce4d0 con 0x7f28fc0cdfc0 2023-05-20T01:22:10.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.302+0000 7f29026d3700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28fc0cb700 con 0x7f28fc15ab80 2023-05-20T01:22:10.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.302+0000 7f29026d3700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28fc0ce7b0 con 0x7f28fc0cf390 2023-05-20T01:22:10.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.302+0000 7f28fbfff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f28fc15ab80 0x7f28fc0cda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:10.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.302+0000 7f28fbfff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f28fc15ab80 0x7f28fc0cda80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34188/0 (socket says 172.21.15.16:34188) 2023-05-20T01:22:10.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.302+0000 7f28fbfff700 1 -- 172.21.15.16:0/3622862515 learned_addr learned my addr 172.21.15.16:0/3622862515 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:22:10.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.302+0000 7f28fb7fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f28fc0cdfc0 0x7f28fc0cec80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:45716/0 (socket says 172.21.15.16:45716) 2023-05-20T01:22:10.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.302+0000 7f2900c70700 1 --2- 172.21.15.16:0/3622862515 >> v2:172.21.15.132:3300/0 conn(0x7f28fc0cf390 0x7f28fc0cc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:10.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.302+0000 7f28fbfff700 1 -- 172.21.15.16:0/3622862515 >> v2:172.21.15.132:3300/0 conn(0x7f28fc0cf390 msgr2=0x7f28fc0cc110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:10.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.302+0000 7f28fbfff700 1 --2- 172.21.15.16:0/3622862515 >> v2:172.21.15.132:3300/0 conn(0x7f28fc0cf390 0x7f28fc0cc110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:10.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.302+0000 7f28fbfff700 1 -- 172.21.15.16:0/3622862515 >> v1:172.21.15.16:6789/0 conn(0x7f28fc0cdfc0 legacy=0x7f28fc0cec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:10.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.302+0000 7f28fbfff700 1 -- 172.21.15.16:0/3622862515 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28fc0cb910 con 0x7f28fc15ab80 2023-05-20T01:22:10.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.302+0000 7f28faffd700 1 -- 172.21.15.16:0/3622862515 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 914699605 0 0) 0x7f28fc0ce4d0 con 0x7f28fc0cdfc0 2023-05-20T01:22:10.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.302+0000 7f28fbfff700 1 --2- 172.21.15.16:0/3622862515 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28fc15ab80 0x7f28fc0cda80 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f28fc0b2b50 tx=0x7f28ec009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ec426d4049278f6 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:10.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.302+0000 7f28faffd700 1 -- 172.21.15.16:0/3622862515 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28ec00b870 con 0x7f28fc15ab80 2023-05-20T01:22:10.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.302+0000 7f28faffd700 1 -- 172.21.15.16:0/3622862515 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28ec00b9d0 con 0x7f28fc15ab80 2023-05-20T01:22:10.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.302+0000 7f28faffd700 1 -- 172.21.15.16:0/3622862515 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28ec00bd60 con 0x7f28fc15ab80 2023-05-20T01:22:10.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.303+0000 7f29026d3700 1 -- 172.21.15.16:0/3622862515 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28fc15ab80 msgr2=0x7f28fc0cda80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:10.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.303+0000 7f29026d3700 1 --2- 172.21.15.16:0/3622862515 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28fc15ab80 0x7f28fc0cda80 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f28fc0b2b50 tx=0x7f28ec009bd0 comp rx=0 tx=0).stop 2023-05-20T01:22:10.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.303+0000 7f29026d3700 1 -- 172.21.15.16:0/3622862515 shutdown_connections 2023-05-20T01:22:10.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.303+0000 7f29026d3700 1 --2- 172.21.15.16:0/3622862515 >> v2:172.21.15.132:3300/0 conn(0x7f28fc0cf390 0x7f28fc0cc110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:10.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.303+0000 7f29026d3700 1 --2- 172.21.15.16:0/3622862515 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28fc15ab80 0x7f28fc0cda80 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:10.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.303+0000 7f29026d3700 1 -- 172.21.15.16:0/3622862515 >> 172.21.15.16:0/3622862515 conn(0x7f28fc06ae30 msgr2=0x7f28fc09a400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:10.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.303+0000 7f29026d3700 1 -- 172.21.15.16:0/3622862515 shutdown_connections 2023-05-20T01:22:10.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.303+0000 7f29026d3700 1 -- 172.21.15.16:0/3622862515 wait complete. 2023-05-20T01:22:10.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.304+0000 7f29026d3700 1 Processor -- start 2023-05-20T01:22:10.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.304+0000 7f29026d3700 1 -- start start 2023-05-20T01:22:10.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.305+0000 7f29026d3700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f28fc0cdfc0 0x7f28fc0caa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:10.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.305+0000 7f29026d3700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f28fc0cf390 0x7f28fc0b1b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:10.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.305+0000 7f29026d3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28fc15ab80 0x7f28fc06cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:10.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.305+0000 7f29026d3700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28fc06c8c0 con 0x7f28fc15ab80 2023-05-20T01:22:10.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.305+0000 7f29026d3700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28fc06bcb0 con 0x7f28fc0cf390 2023-05-20T01:22:10.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.305+0000 7f29026d3700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28fc06bf90 con 0x7f28fc0cdfc0 2023-05-20T01:22:10.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.305+0000 7f2900c70700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28fc15ab80 0x7f28fc06cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:10.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.305+0000 7f2900c70700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28fc15ab80 0x7f28fc06cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34198/0 (socket says 172.21.15.16:34198) 2023-05-20T01:22:10.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.305+0000 7f2900c70700 1 -- 172.21.15.16:0/3442899170 learned_addr learned my addr 172.21.15.16:0/3442899170 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:22:10.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.305+0000 7f28fb7fe700 1 --2- 172.21.15.16:0/3442899170 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f28fc0cf390 0x7f28fc0b1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:10.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.305+0000 7f28fbfff700 1 --2- 172.21.15.16:0/3442899170 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f28fc0cdfc0 0x7f28fc0caa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:10.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.306+0000 7f2900c70700 1 -- 172.21.15.16:0/3442899170 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f28fc0cdfc0 msgr2=0x7f28fc0caa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:10.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.306+0000 7f2900c70700 1 --2- 172.21.15.16:0/3442899170 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f28fc0cdfc0 0x7f28fc0caa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:10.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.306+0000 7f2900c70700 1 -- 172.21.15.16:0/3442899170 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f28fc0cf390 msgr2=0x7f28fc0b1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:10.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.306+0000 7f2900c70700 1 --2- 172.21.15.16:0/3442899170 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f28fc0cf390 0x7f28fc0b1b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:10.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.306+0000 7f2900c70700 1 -- 172.21.15.16:0/3442899170 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28ec009550 con 0x7f28fc15ab80 2023-05-20T01:22:10.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.306+0000 7f28fb7fe700 1 --2- 172.21.15.16:0/3442899170 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f28fc0cf390 0x7f28fc0b1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:22:10.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.306+0000 7f2900c70700 1 --2- 172.21.15.16:0/3442899170 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28fc15ab80 0x7f28fc06cd50 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f28f4007e30 tx=0x7f28f400e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:10.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.306+0000 7f28f8ff9700 1 -- 172.21.15.16:0/3442899170 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f400eea0 con 0x7f28fc15ab80 2023-05-20T01:22:10.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.306+0000 7f28f8ff9700 1 -- 172.21.15.16:0/3442899170 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28f4014420 con 0x7f28fc15ab80 2023-05-20T01:22:10.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.306+0000 7f29026d3700 1 -- 172.21.15.16:0/3442899170 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28fc0d0e30 con 0x7f28fc15ab80 2023-05-20T01:22:10.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.306+0000 7f28f8ff9700 1 -- 172.21.15.16:0/3442899170 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f40147d0 con 0x7f28fc15ab80 2023-05-20T01:22:10.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.306+0000 7f29026d3700 1 -- 172.21.15.16:0/3442899170 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28fc0cbbf0 con 0x7f28fc15ab80 2023-05-20T01:22:10.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.307+0000 7f29026d3700 1 -- 172.21.15.16:0/3442899170 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28fc0d0fc0 con 0x7f28fc15ab80 2023-05-20T01:22:10.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.307+0000 7f28f8ff9700 1 -- 172.21.15.16:0/3442899170 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f28f4014930 con 0x7f28fc15ab80 2023-05-20T01:22:10.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.308+0000 7f28f8ff9700 1 --2- 172.21.15.16:0/3442899170 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f28e806ace0 0x7f28e806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:10.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.308+0000 7f28f8ff9700 1 -- 172.21.15.16:0/3442899170 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6931+0+0 (secure 0 0 0) 0x7f28f408a000 con 0x7f28fc15ab80 2023-05-20T01:22:10.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.308+0000 7f28fbfff700 1 --2- 172.21.15.16:0/3442899170 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f28e806ace0 0x7f28e806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:10.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.308+0000 7f28fbfff700 1 --2- 172.21.15.16:0/3442899170 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f28e806ace0 0x7f28e806d190 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f28ec014040 tx=0x7f28ec008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:10.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.312+0000 7f28f8ff9700 1 -- 172.21.15.16:0/3442899170 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f28f4054d40 con 0x7f28fc15ab80 2023-05-20T01:22:10.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.514+0000 7f29026d3700 1 -- 172.21.15.16:0/3442899170 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f28fc0747b0 con 0x7f28fc15ab80 2023-05-20T01:22:10.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.515+0000 7f28f8ff9700 1 -- 172.21.15.16:0/3442899170 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v69) v1 ==== 104+0+11 (secure 0 0 0) 0x7f28f4058360 con 0x7f28fc15ab80 2023-05-20T01:22:10.520 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-05-20T01:22:10.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.517+0000 7f29026d3700 1 -- 172.21.15.16:0/3442899170 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f28e806ace0 msgr2=0x7f28e806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:10.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.517+0000 7f29026d3700 1 --2- 172.21.15.16:0/3442899170 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f28e806ace0 0x7f28e806d190 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f28ec014040 tx=0x7f28ec008030 comp rx=0 tx=0).stop 2023-05-20T01:22:10.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.517+0000 7f29026d3700 1 -- 172.21.15.16:0/3442899170 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28fc15ab80 msgr2=0x7f28fc06cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:10.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.517+0000 7f29026d3700 1 --2- 172.21.15.16:0/3442899170 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28fc15ab80 0x7f28fc06cd50 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f28f4007e30 tx=0x7f28f400e3c0 comp rx=0 tx=0).stop 2023-05-20T01:22:10.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.517+0000 7f29026d3700 1 -- 172.21.15.16:0/3442899170 shutdown_connections 2023-05-20T01:22:10.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.517+0000 7f29026d3700 1 --2- 172.21.15.16:0/3442899170 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28fc15ab80 0x7f28fc06cd50 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:10.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.517+0000 7f29026d3700 1 --2- 172.21.15.16:0/3442899170 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f28e806ace0 0x7f28e806d190 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:10.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.517+0000 7f29026d3700 1 --2- 172.21.15.16:0/3442899170 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f28fc0cdfc0 0x7f28fc0caa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:10.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.517+0000 7f29026d3700 1 --2- 172.21.15.16:0/3442899170 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f28fc0cf390 0x7f28fc0b1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:10.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.517+0000 7f29026d3700 1 -- 172.21.15.16:0/3442899170 >> 172.21.15.16:0/3442899170 conn(0x7f28fc06ae30 msgr2=0x7f28fc0b0a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:10.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.517+0000 7f29026d3700 1 -- 172.21.15.16:0/3442899170 shutdown_connections 2023-05-20T01:22:10.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.517+0000 7f29026d3700 1 -- 172.21.15.16:0/3442899170 wait complete. 2023-05-20T01:22:10.533 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-05-20T01:22:10.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.657+0000 7f10a1296700 1 Processor -- start 2023-05-20T01:22:10.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.657+0000 7f10a1296700 1 -- start start 2023-05-20T01:22:10.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.658+0000 7f10a1296700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f109c0cec80 0x7f109c0cf0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:10.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.658+0000 7f10a1296700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f109c0cc110 0x7f109c0cc590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:10.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.658+0000 7f10a1296700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f109c0ce4d0 con 0x7f109c15ab80 2023-05-20T01:22:10.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.658+0000 7f10a1296700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f109c0cb700 con 0x7f109c0cc110 2023-05-20T01:22:10.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.658+0000 7f10a1296700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f109c0cf630 con 0x7f109c0cec80 2023-05-20T01:22:10.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.658+0000 7f109b7fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f109c0cc110 0x7f109c0cc590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:10.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.658+0000 7f109affd700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f109c15ab80 0x7f109c0cda80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:45730/0 (socket says 172.21.15.16:45730) 2023-05-20T01:22:10.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.658+0000 7f109affd700 1 -- 172.21.15.16:0/3219752468 learned_addr learned my addr 172.21.15.16:0/3219752468 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:22:10.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.658+0000 7f109a7fc700 1 --2- 172.21.15.16:0/3219752468 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f109c0cec80 0x7f109c0cf0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:10.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.659+0000 7f1099ffb700 1 -- 172.21.15.16:0/3219752468 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3003361759 0 0) 0x7f109c0ce4d0 con 0x7f109c15ab80 2023-05-20T01:22:10.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.659+0000 7f1099ffb700 1 -- 172.21.15.16:0/3219752468 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f10840035b0 con 0x7f109c15ab80 2023-05-20T01:22:10.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.659+0000 7f109b7fe700 1 -- 172.21.15.16:0/3219752468 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f109c0cec80 msgr2=0x7f109c0cf0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:10.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.659+0000 7f109b7fe700 1 --2- 172.21.15.16:0/3219752468 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f109c0cec80 0x7f109c0cf0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:10.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.659+0000 7f109b7fe700 1 -- 172.21.15.16:0/3219752468 >> v1:172.21.15.16:6789/0 conn(0x7f109c15ab80 legacy=0x7f109c0cda80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:10.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.659+0000 7f109b7fe700 1 -- 172.21.15.16:0/3219752468 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f109c0ce7b0 con 0x7f109c0cc110 2023-05-20T01:22:10.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.659+0000 7f109b7fe700 1 --2- 172.21.15.16:0/3219752468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f109c0cc110 0x7f109c0cc590 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f1094009d90 tx=0x7f109400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bcc6096de82f7e1 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:10.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.659+0000 7f1099ffb700 1 -- 172.21.15.16:0/3219752468 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f109400e070 con 0x7f109c0cc110 2023-05-20T01:22:10.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.659+0000 7f1099ffb700 1 -- 172.21.15.16:0/3219752468 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f109400bb20 con 0x7f109c0cc110 2023-05-20T01:22:10.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.659+0000 7f1099ffb700 1 -- 172.21.15.16:0/3219752468 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1094012430 con 0x7f109c0cc110 2023-05-20T01:22:10.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.659+0000 7f10a1296700 1 -- 172.21.15.16:0/3219752468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f109c0cc110 msgr2=0x7f109c0cc590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:10.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.659+0000 7f10a1296700 1 --2- 172.21.15.16:0/3219752468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f109c0cc110 0x7f109c0cc590 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f1094009d90 tx=0x7f109400b330 comp rx=0 tx=0).stop 2023-05-20T01:22:10.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.659+0000 7f10a1296700 1 -- 172.21.15.16:0/3219752468 shutdown_connections 2023-05-20T01:22:10.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.659+0000 7f10a1296700 1 --2- 172.21.15.16:0/3219752468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f109c0cc110 0x7f109c0cc590 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:10.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.659+0000 7f10a1296700 1 --2- 172.21.15.16:0/3219752468 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f109c0cec80 0x7f109c0cf0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:10.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.660+0000 7f10a1296700 1 -- 172.21.15.16:0/3219752468 >> 172.21.15.16:0/3219752468 conn(0x7f109c06ae30 msgr2=0x7f109c09a400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:10.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.660+0000 7f10a1296700 1 -- 172.21.15.16:0/3219752468 shutdown_connections 2023-05-20T01:22:10.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.660+0000 7f10a1296700 1 -- 172.21.15.16:0/3219752468 wait complete. 2023-05-20T01:22:10.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.661+0000 7f10a1296700 1 Processor -- start 2023-05-20T01:22:10.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.661+0000 7f10a1296700 1 -- start start 2023-05-20T01:22:10.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.662+0000 7f10a1296700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f109c0cc110 0x7f109c154130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:10.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.662+0000 7f10a1296700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f109c0cec80 0x7f109c1e3e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:10.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.662+0000 7f10a1296700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f109c15ab80 0x7f109c1e12c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:10.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.662+0000 7f10a1296700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f109c0c6ee0 con 0x7f109c0cec80 2023-05-20T01:22:10.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.662+0000 7f10a1296700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f109c0b4550 con 0x7f109c15ab80 2023-05-20T01:22:10.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.662+0000 7f10a1296700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f109c1e2110 con 0x7f109c0cc110 2023-05-20T01:22:10.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.662+0000 7f109a7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f109c0cec80 0x7f109c1e3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:10.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.662+0000 7f109a7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f109c0cec80 0x7f109c1e3e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34210/0 (socket says 172.21.15.16:34210) 2023-05-20T01:22:10.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.662+0000 7f109a7fc700 1 -- 172.21.15.16:0/1416769850 learned_addr learned my addr 172.21.15.16:0/1416769850 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:22:10.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.662+0000 7f109b7fe700 1 --2- 172.21.15.16:0/1416769850 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f109c15ab80 0x7f109c1e12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:10.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.662+0000 7f109affd700 1 --2- 172.21.15.16:0/1416769850 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f109c0cc110 0x7f109c154130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:10.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.662+0000 7f109a7fc700 1 -- 172.21.15.16:0/1416769850 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f109c0cc110 msgr2=0x7f109c154130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:10.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.662+0000 7f109a7fc700 1 --2- 172.21.15.16:0/1416769850 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f109c0cc110 0x7f109c154130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:10.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.662+0000 7f109a7fc700 1 -- 172.21.15.16:0/1416769850 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f109c15ab80 msgr2=0x7f109c1e12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:10.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.662+0000 7f109a7fc700 1 --2- 172.21.15.16:0/1416769850 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f109c15ab80 0x7f109c1e12c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:10.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.662+0000 7f109a7fc700 1 -- 172.21.15.16:0/1416769850 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1094009a70 con 0x7f109c0cec80 2023-05-20T01:22:10.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.662+0000 7f109b7fe700 1 --2- 172.21.15.16:0/1416769850 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f109c15ab80 0x7f109c1e12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:22:10.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.662+0000 7f109a7fc700 1 --2- 172.21.15.16:0/1416769850 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f109c0cec80 0x7f109c1e3e10 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f108c00e810 tx=0x7f108c00ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:10.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.662+0000 7f10837fe700 1 -- 172.21.15.16:0/1416769850 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f108c013610 con 0x7f109c0cec80 2023-05-20T01:22:10.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.662+0000 7f10837fe700 1 -- 172.21.15.16:0/1416769850 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f108c013770 con 0x7f109c0cec80 2023-05-20T01:22:10.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.662+0000 7f10837fe700 1 -- 172.21.15.16:0/1416769850 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f108c013b20 con 0x7f109c0cec80 2023-05-20T01:22:10.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.662+0000 7f10a1296700 1 -- 172.21.15.16:0/1416769850 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f109c0cbbf0 con 0x7f109c0cec80 2023-05-20T01:22:10.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.663+0000 7f10a1296700 1 -- 172.21.15.16:0/1416769850 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f109c0cb910 con 0x7f109c0cec80 2023-05-20T01:22:10.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.663+0000 7f10a1296700 1 -- 172.21.15.16:0/1416769850 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f109c0ce190 con 0x7f109c0cec80 2023-05-20T01:22:10.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.664+0000 7f10837fe700 1 -- 172.21.15.16:0/1416769850 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f108c013c80 con 0x7f109c0cec80 2023-05-20T01:22:10.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.664+0000 7f10837fe700 1 --2- 172.21.15.16:0/1416769850 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f108806ace0 0x7f108806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:10.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.665+0000 7f109affd700 1 --2- 172.21.15.16:0/1416769850 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f108806ace0 0x7f108806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:10.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.665+0000 7f10837fe700 1 -- 172.21.15.16:0/1416769850 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6931+0+0 (secure 0 0 0) 0x7f108c089520 con 0x7f109c0cec80 2023-05-20T01:22:10.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.665+0000 7f109affd700 1 --2- 172.21.15.16:0/1416769850 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f108806ace0 0x7f108806d190 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f10900011e0 tx=0x7f1090009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:10.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.669+0000 7f10837fe700 1 -- 172.21.15.16:0/1416769850 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f108c054270 con 0x7f109c0cec80 2023-05-20T01:22:10.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.864+0000 7f10a1296700 1 -- 172.21.15.16:0/1416769850 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f109c0747b0 con 0x7f109c0cec80 2023-05-20T01:22:10.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.864+0000 7f10837fe700 1 -- 172.21.15.16:0/1416769850 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v69) v1 ==== 100+0+11 (secure 0 0 0) 0x7f108c057890 con 0x7f109c0cec80 2023-05-20T01:22:10.869 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2023-05-20T01:22:10.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.866+0000 7f10a1296700 1 -- 172.21.15.16:0/1416769850 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f108806ace0 msgr2=0x7f108806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:10.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.866+0000 7f10a1296700 1 --2- 172.21.15.16:0/1416769850 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f108806ace0 0x7f108806d190 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f10900011e0 tx=0x7f1090009040 comp rx=0 tx=0).stop 2023-05-20T01:22:10.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.866+0000 7f10a1296700 1 -- 172.21.15.16:0/1416769850 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f109c0cec80 msgr2=0x7f109c1e3e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:10.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.866+0000 7f10a1296700 1 --2- 172.21.15.16:0/1416769850 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f109c0cec80 0x7f109c1e3e10 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f108c00e810 tx=0x7f108c00ebd0 comp rx=0 tx=0).stop 2023-05-20T01:22:10.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.866+0000 7f10a1296700 1 -- 172.21.15.16:0/1416769850 shutdown_connections 2023-05-20T01:22:10.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.866+0000 7f10a1296700 1 --2- 172.21.15.16:0/1416769850 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f109c15ab80 0x7f109c1e12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:10.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.866+0000 7f10a1296700 1 --2- 172.21.15.16:0/1416769850 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f108806ace0 0x7f108806d190 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:10.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.866+0000 7f10a1296700 1 --2- 172.21.15.16:0/1416769850 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f109c0cc110 0x7f109c154130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:10.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.866+0000 7f10a1296700 1 --2- 172.21.15.16:0/1416769850 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f109c0cec80 0x7f109c1e3e10 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:10.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.866+0000 7f10a1296700 1 -- 172.21.15.16:0/1416769850 >> 172.21.15.16:0/1416769850 conn(0x7f109c06ae30 msgr2=0x7f109c156750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:10.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.866+0000 7f10a1296700 1 -- 172.21.15.16:0/1416769850 shutdown_connections 2023-05-20T01:22:10.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:10.866+0000 7f10a1296700 1 -- 172.21.15.16:0/1416769850 wait complete. 2023-05-20T01:22:10.882 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_extra pg_num 2023-05-20T01:22:11.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.006+0000 7eff31d41700 1 Processor -- start 2023-05-20T01:22:11.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.006+0000 7eff31d41700 1 -- start start 2023-05-20T01:22:11.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.007+0000 7eff31d41700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7eff2c0cec80 0x7eff2c0cf0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:11.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.007+0000 7eff31d41700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff2c0ce4d0 con 0x7eff2c0cc110 2023-05-20T01:22:11.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.007+0000 7eff31d41700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff2c0ce7b0 con 0x7eff2c15ab80 2023-05-20T01:22:11.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.007+0000 7eff31d41700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff2c0cb700 con 0x7eff2c0cec80 2023-05-20T01:22:11.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.007+0000 7eff2affd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7eff2c0cec80 0x7eff2c0cf0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:11.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.007+0000 7eff2bfff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7eff2c0cc110 0x7eff2c0cc550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:45748/0 (socket says 172.21.15.16:45748) 2023-05-20T01:22:11.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.007+0000 7eff2affd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7eff2c0cec80 0x7eff2c0cf0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34216/0 (socket says 172.21.15.16:34216) 2023-05-20T01:22:11.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.007+0000 7eff2bfff700 1 -- 172.21.15.16:0/959992533 learned_addr learned my addr 172.21.15.16:0/959992533 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:22:11.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.007+0000 7eff2affd700 1 -- 172.21.15.16:0/959992533 >> v1:172.21.15.132:6789/0 conn(0x7eff2c15ab80 legacy=0x7eff2c0cda80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:11.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.007+0000 7eff2affd700 1 -- 172.21.15.16:0/959992533 >> v1:172.21.15.16:6789/0 conn(0x7eff2c0cc110 legacy=0x7eff2c0cc550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:11.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.007+0000 7eff2affd700 1 -- 172.21.15.16:0/959992533 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff2c0cb910 con 0x7eff2c0cec80 2023-05-20T01:22:11.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.007+0000 7eff2affd700 1 --2- 172.21.15.16:0/959992533 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff2c0cec80 0x7eff2c0cf0f0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7eff1c009840 tx=0x7eff1c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27bfecbfb3a119ed server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:11.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.008+0000 7eff2a7fc700 1 -- 172.21.15.16:0/959992533 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff1c004230 con 0x7eff2c0cec80 2023-05-20T01:22:11.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.008+0000 7eff2a7fc700 1 -- 172.21.15.16:0/959992533 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff1c004390 con 0x7eff2c0cec80 2023-05-20T01:22:11.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.008+0000 7eff2a7fc700 1 -- 172.21.15.16:0/959992533 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff1c0036a0 con 0x7eff2c0cec80 2023-05-20T01:22:11.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.008+0000 7eff31d41700 1 -- 172.21.15.16:0/959992533 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff2c0cec80 msgr2=0x7eff2c0cf0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:11.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.008+0000 7eff31d41700 1 --2- 172.21.15.16:0/959992533 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff2c0cec80 0x7eff2c0cf0f0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7eff1c009840 tx=0x7eff1c009b50 comp rx=0 tx=0).stop 2023-05-20T01:22:11.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.008+0000 7eff31d41700 1 -- 172.21.15.16:0/959992533 shutdown_connections 2023-05-20T01:22:11.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.008+0000 7eff31d41700 1 --2- 172.21.15.16:0/959992533 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff2c0cec80 0x7eff2c0cf0f0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:11.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.008+0000 7eff31d41700 1 -- 172.21.15.16:0/959992533 >> 172.21.15.16:0/959992533 conn(0x7eff2c06ae30 msgr2=0x7eff2c09a400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:11.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.008+0000 7eff31d41700 1 -- 172.21.15.16:0/959992533 shutdown_connections 2023-05-20T01:22:11.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.008+0000 7eff31d41700 1 -- 172.21.15.16:0/959992533 wait complete. 2023-05-20T01:22:11.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.009+0000 7eff31d41700 1 Processor -- start 2023-05-20T01:22:11.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.010+0000 7eff31d41700 1 -- start start 2023-05-20T01:22:11.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.010+0000 7eff31d41700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7eff2c0cc110 0x7eff2c1f6630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:11.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.010+0000 7eff31d41700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff2c0cec80 0x7eff2c06c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:11.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.010+0000 7eff31d41700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7eff2c15ab80 0x7eff2c159430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:11.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.010+0000 7eff31d41700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff2c06d660 con 0x7eff2c0cec80 2023-05-20T01:22:11.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.010+0000 7eff31d41700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff2c1f6b70 con 0x7eff2c15ab80 2023-05-20T01:22:11.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.010+0000 7eff31d41700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff2c06d0a0 con 0x7eff2c0cc110 2023-05-20T01:22:11.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.010+0000 7eff2affd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff2c0cec80 0x7eff2c06c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:11.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.011+0000 7eff2affd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff2c0cec80 0x7eff2c06c750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34224/0 (socket says 172.21.15.16:34224) 2023-05-20T01:22:11.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.011+0000 7eff2affd700 1 -- 172.21.15.16:0/1912618781 learned_addr learned my addr 172.21.15.16:0/1912618781 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:22:11.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.011+0000 7eff2bfff700 1 --2- 172.21.15.16:0/1912618781 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7eff2c15ab80 0x7eff2c159430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:11.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.011+0000 7eff2b7fe700 1 --2- 172.21.15.16:0/1912618781 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7eff2c0cc110 0x7eff2c1f6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:11.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.011+0000 7eff2affd700 1 -- 172.21.15.16:0/1912618781 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7eff2c0cc110 msgr2=0x7eff2c1f6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:11.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.011+0000 7eff2affd700 1 --2- 172.21.15.16:0/1912618781 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7eff2c0cc110 0x7eff2c1f6630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:11.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.011+0000 7eff2affd700 1 -- 172.21.15.16:0/1912618781 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7eff2c15ab80 msgr2=0x7eff2c159430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:11.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.011+0000 7eff2affd700 1 --2- 172.21.15.16:0/1912618781 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7eff2c15ab80 0x7eff2c159430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:11.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.011+0000 7eff2affd700 1 -- 172.21.15.16:0/1912618781 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff1c0094a0 con 0x7eff2c0cec80 2023-05-20T01:22:11.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.011+0000 7eff2affd700 1 --2- 172.21.15.16:0/1912618781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff2c0cec80 0x7eff2c06c750 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7eff1c0090b0 tx=0x7eff1c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:11.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.011+0000 7eff23fff700 1 -- 172.21.15.16:0/1912618781 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff1c011830 con 0x7eff2c0cec80 2023-05-20T01:22:11.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.011+0000 7eff23fff700 1 -- 172.21.15.16:0/1912618781 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff1c01e070 con 0x7eff2c0cec80 2023-05-20T01:22:11.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.011+0000 7eff31d41700 1 -- 172.21.15.16:0/1912618781 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff2c0ce190 con 0x7eff2c0cec80 2023-05-20T01:22:11.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.011+0000 7eff23fff700 1 -- 172.21.15.16:0/1912618781 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff1c011bf0 con 0x7eff2c0cec80 2023-05-20T01:22:11.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.011+0000 7eff31d41700 1 -- 172.21.15.16:0/1912618781 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff2c0cbbf0 con 0x7eff2c0cec80 2023-05-20T01:22:11.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.012+0000 7eff31d41700 1 -- 172.21.15.16:0/1912618781 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff2c0d0e30 con 0x7eff2c0cec80 2023-05-20T01:22:11.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.013+0000 7eff23fff700 1 -- 172.21.15.16:0/1912618781 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7eff1c011d50 con 0x7eff2c0cec80 2023-05-20T01:22:11.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.014+0000 7eff23fff700 1 --2- 172.21.15.16:0/1912618781 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7eff0c06ace0 0x7eff0c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:11.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.014+0000 7eff23fff700 1 -- 172.21.15.16:0/1912618781 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6931+0+0 (secure 0 0 0) 0x7eff1c08b4b0 con 0x7eff2c0cec80 2023-05-20T01:22:11.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.014+0000 7eff2b7fe700 1 --2- 172.21.15.16:0/1912618781 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7eff0c06ace0 0x7eff0c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:11.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.014+0000 7eff2b7fe700 1 --2- 172.21.15.16:0/1912618781 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7eff0c06ace0 0x7eff0c06d190 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7eff14004f30 tx=0x7eff14009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:11.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.017+0000 7eff23fff700 1 -- 172.21.15.16:0/1912618781 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7eff1c056260 con 0x7eff2c0cec80 2023-05-20T01:22:11.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.214+0000 7eff31d41700 1 -- 172.21.15.16:0/1912618781 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"} v 0) v1 -- 0x7eff2c0747b0 con 0x7eff2c0cec80 2023-05-20T01:22:11.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.214+0000 7eff23fff700 1 -- 172.21.15.16:0/1912618781 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"}]=0 v69) v1 ==== 106+0+11 (secure 0 0 0) 0x7eff1c020020 con 0x7eff2c0cec80 2023-05-20T01:22:11.219 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-05-20T01:22:11.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.216+0000 7eff31d41700 1 -- 172.21.15.16:0/1912618781 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7eff0c06ace0 msgr2=0x7eff0c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:11.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.216+0000 7eff31d41700 1 --2- 172.21.15.16:0/1912618781 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7eff0c06ace0 0x7eff0c06d190 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7eff14004f30 tx=0x7eff14009040 comp rx=0 tx=0).stop 2023-05-20T01:22:11.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.216+0000 7eff31d41700 1 -- 172.21.15.16:0/1912618781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff2c0cec80 msgr2=0x7eff2c06c750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:11.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.216+0000 7eff31d41700 1 --2- 172.21.15.16:0/1912618781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff2c0cec80 0x7eff2c06c750 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7eff1c0090b0 tx=0x7eff1c004750 comp rx=0 tx=0).stop 2023-05-20T01:22:11.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.216+0000 7eff31d41700 1 -- 172.21.15.16:0/1912618781 shutdown_connections 2023-05-20T01:22:11.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.216+0000 7eff31d41700 1 --2- 172.21.15.16:0/1912618781 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7eff2c15ab80 0x7eff2c159430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:11.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.216+0000 7eff31d41700 1 --2- 172.21.15.16:0/1912618781 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7eff0c06ace0 0x7eff0c06d190 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:11.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.216+0000 7eff31d41700 1 --2- 172.21.15.16:0/1912618781 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7eff2c0cc110 0x7eff2c1f6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:11.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.216+0000 7eff31d41700 1 --2- 172.21.15.16:0/1912618781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff2c0cec80 0x7eff2c06c750 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:11.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.216+0000 7eff31d41700 1 -- 172.21.15.16:0/1912618781 >> 172.21.15.16:0/1912618781 conn(0x7eff2c06ae30 msgr2=0x7eff2c06ed40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:11.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.216+0000 7eff31d41700 1 -- 172.21.15.16:0/1912618781 shutdown_connections 2023-05-20T01:22:11.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.216+0000 7eff31d41700 1 -- 172.21.15.16:0/1912618781 wait complete. 2023-05-20T01:22:11.233 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-05-20T01:22:11.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.356+0000 7f12aa90a700 1 Processor -- start 2023-05-20T01:22:11.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.356+0000 7f12aa90a700 1 -- start start 2023-05-20T01:22:11.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.357+0000 7f12aa90a700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f12a406a510 0x7f12a415e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:11.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.357+0000 7f12aa90a700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f12a406a8e0 0x7f12a40cf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:11.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.357+0000 7f12aa90a700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12a415e060 con 0x7f12a40cfb30 2023-05-20T01:22:11.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.357+0000 7f12aa90a700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12a415d450 con 0x7f12a406a8e0 2023-05-20T01:22:11.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.357+0000 7f12aa90a700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12a415d730 con 0x7f12a406a510 2023-05-20T01:22:11.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.357+0000 7f12a3fff700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f12a406a510 0x7f12a415e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:11.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.357+0000 7f129bfff700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f12a406a8e0 0x7f12a40cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:11.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.357+0000 7f12a3fff700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f12a406a510 0x7f12a415e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.16:34268/0 (socket says 172.21.15.16:34268) 2023-05-20T01:22:11.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.357+0000 7f12a8ea7700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f12a40cfb30 0x7f12a4071340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.16:55476/0 (socket says 172.21.15.16:55476) 2023-05-20T01:22:11.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.357+0000 7f12a3fff700 1 -- 172.21.15.16:0/3108540614 learned_addr learned my addr 172.21.15.16:0/3108540614 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:22:11.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.357+0000 7f12a3fff700 1 -- 172.21.15.16:0/3108540614 >> v2:172.21.15.132:3300/0 conn(0x7f12a406a8e0 msgr2=0x7f12a40cf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:11.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.357+0000 7f12a3fff700 1 --2- 172.21.15.16:0/3108540614 >> v2:172.21.15.132:3300/0 conn(0x7f12a406a8e0 0x7f12a40cf5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:11.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.357+0000 7f12a3fff700 1 -- 172.21.15.16:0/3108540614 >> v1:172.21.15.132:6789/0 conn(0x7f12a40cfb30 legacy=0x7f12a4071340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:11.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.357+0000 7f12a3fff700 1 -- 172.21.15.16:0/3108540614 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12a415da10 con 0x7f12a406a510 2023-05-20T01:22:11.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.357+0000 7f129bfff700 1 --2- 172.21.15.16:0/3108540614 >> v2:172.21.15.132:3300/0 conn(0x7f12a406a8e0 0x7f12a40cf5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-05-20T01:22:11.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.357+0000 7f12a37fe700 1 -- 172.21.15.16:0/3108540614 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 177196203 0 0) 0x7f12a415e060 con 0x7f12a40cfb30 2023-05-20T01:22:11.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.358+0000 7f12a3fff700 1 --2- 172.21.15.16:0/3108540614 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f12a406a510 0x7f12a415e2a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f12940098f0 tx=0x7f1294009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c49958a9ccb5b7a5 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:11.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.358+0000 7f12a37fe700 1 -- 172.21.15.16:0/3108540614 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1294011830 con 0x7f12a406a510 2023-05-20T01:22:11.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.358+0000 7f12a37fe700 1 -- 172.21.15.16:0/3108540614 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1294011990 con 0x7f12a406a510 2023-05-20T01:22:11.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.358+0000 7f12a37fe700 1 -- 172.21.15.16:0/3108540614 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1294011d20 con 0x7f12a406a510 2023-05-20T01:22:11.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.358+0000 7f12aa90a700 1 -- 172.21.15.16:0/3108540614 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f12a406a510 msgr2=0x7f12a415e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:11.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.358+0000 7f12aa90a700 1 --2- 172.21.15.16:0/3108540614 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f12a406a510 0x7f12a415e2a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f12940098f0 tx=0x7f1294009c00 comp rx=0 tx=0).stop 2023-05-20T01:22:11.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.358+0000 7f12aa90a700 1 -- 172.21.15.16:0/3108540614 shutdown_connections 2023-05-20T01:22:11.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.358+0000 7f12aa90a700 1 --2- 172.21.15.16:0/3108540614 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f12a406a510 0x7f12a415e2a0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:11.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.358+0000 7f12aa90a700 1 --2- 172.21.15.16:0/3108540614 >> v2:172.21.15.132:3300/0 conn(0x7f12a406a8e0 0x7f12a40cf5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:11.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.358+0000 7f12aa90a700 1 -- 172.21.15.16:0/3108540614 >> 172.21.15.16:0/3108540614 conn(0x7f12a40d0c30 msgr2=0x7f12a40b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:11.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.358+0000 7f12aa90a700 1 -- 172.21.15.16:0/3108540614 shutdown_connections 2023-05-20T01:22:11.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.358+0000 7f12aa90a700 1 -- 172.21.15.16:0/3108540614 wait complete. 2023-05-20T01:22:11.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.360+0000 7f12aa90a700 1 Processor -- start 2023-05-20T01:22:11.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.360+0000 7f12aa90a700 1 -- start start 2023-05-20T01:22:11.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.360+0000 7f12aa90a700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f12a406a510 0x7f12a40c4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:11.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.360+0000 7f12aa90a700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f12a406a8e0 0x7f12a40b5860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:11.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.360+0000 7f12aa90a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12a40cfb30 0x7f12a414e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:11.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.360+0000 7f12aa90a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12a40b5da0 con 0x7f12a40cfb30 2023-05-20T01:22:11.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.360+0000 7f12aa90a700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12a40c3ae0 con 0x7f12a406a510 2023-05-20T01:22:11.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.360+0000 7f12aa90a700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12a40c3dc0 con 0x7f12a406a8e0 2023-05-20T01:22:11.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.361+0000 7f12a8ea7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12a40cfb30 0x7f12a414e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:11.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.361+0000 7f12a8ea7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12a40cfb30 0x7f12a414e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34240/0 (socket says 172.21.15.16:34240) 2023-05-20T01:22:11.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.361+0000 7f12a8ea7700 1 -- 172.21.15.16:0/3382152786 learned_addr learned my addr 172.21.15.16:0/3382152786 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:22:11.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.361+0000 7f12a3fff700 1 --2- 172.21.15.16:0/3382152786 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f12a406a510 0x7f12a40c4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:11.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.361+0000 7f129bfff700 1 --2- 172.21.15.16:0/3382152786 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f12a406a8e0 0x7f12a40b5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:11.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.361+0000 7f12a8ea7700 1 -- 172.21.15.16:0/3382152786 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f12a406a8e0 msgr2=0x7f12a40b5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:11.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.361+0000 7f12a8ea7700 1 --2- 172.21.15.16:0/3382152786 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f12a406a8e0 0x7f12a40b5860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:11.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.361+0000 7f12a8ea7700 1 -- 172.21.15.16:0/3382152786 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f12a406a510 msgr2=0x7f12a40c4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:11.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.361+0000 7f12a8ea7700 1 --2- 172.21.15.16:0/3382152786 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f12a406a510 0x7f12a40c4e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:11.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.361+0000 7f12a8ea7700 1 -- 172.21.15.16:0/3382152786 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f128c0094a0 con 0x7f12a40cfb30 2023-05-20T01:22:11.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.361+0000 7f12a8ea7700 1 --2- 172.21.15.16:0/3382152786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12a40cfb30 0x7f12a414e410 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f129c002230 tx=0x7f129c00cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:11.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.361+0000 7f12a17fa700 1 -- 172.21.15.16:0/3382152786 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f129c00aac0 con 0x7f12a40cfb30 2023-05-20T01:22:11.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.361+0000 7f12a17fa700 1 -- 172.21.15.16:0/3382152786 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f129c00ac40 con 0x7f12a40cfb30 2023-05-20T01:22:11.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.361+0000 7f12aa90a700 1 -- 172.21.15.16:0/3382152786 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1294009550 con 0x7f12a40cfb30 2023-05-20T01:22:11.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.361+0000 7f12a17fa700 1 -- 172.21.15.16:0/3382152786 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f129c0070d0 con 0x7f12a40cfb30 2023-05-20T01:22:11.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.361+0000 7f12aa90a700 1 -- 172.21.15.16:0/3382152786 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12a406ad50 con 0x7f12a40cfb30 2023-05-20T01:22:11.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.362+0000 7f12aa90a700 1 -- 172.21.15.16:0/3382152786 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12a415dcf0 con 0x7f12a40cfb30 2023-05-20T01:22:11.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.363+0000 7f12a17fa700 1 -- 172.21.15.16:0/3382152786 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f129c007250 con 0x7f12a40cfb30 2023-05-20T01:22:11.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.363+0000 7f12a17fa700 1 --2- 172.21.15.16:0/3382152786 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f128406ace0 0x7f128406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:11.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.363+0000 7f12a17fa700 1 -- 172.21.15.16:0/3382152786 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6931+0+0 (secure 0 0 0) 0x7f129c0892d0 con 0x7f12a40cfb30 2023-05-20T01:22:11.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.363+0000 7f12a3fff700 1 --2- 172.21.15.16:0/3382152786 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f128406ace0 0x7f128406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:11.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.364+0000 7f12a3fff700 1 --2- 172.21.15.16:0/3382152786 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f128406ace0 0x7f128406d190 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f1294014040 tx=0x7f1294008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:11.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.368+0000 7f12a17fa700 1 -- 172.21.15.16:0/3382152786 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f129c053f90 con 0x7f12a40cfb30 2023-05-20T01:22:11.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.582+0000 7f12aa90a700 1 -- 172.21.15.16:0/3382152786 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f12a415d080 con 0x7f12a40cfb30 2023-05-20T01:22:11.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.582+0000 7f12a17fa700 1 -- 172.21.15.16:0/3382152786 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v69) v1 ==== 80+0+0 (secure 0 0 0) 0x7f129c0575b0 con 0x7f12a40cfb30 2023-05-20T01:22:11.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.584+0000 7f12aa90a700 1 -- 172.21.15.16:0/3382152786 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f128406ace0 msgr2=0x7f128406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:11.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.584+0000 7f12aa90a700 1 --2- 172.21.15.16:0/3382152786 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f128406ace0 0x7f128406d190 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f1294014040 tx=0x7f1294008030 comp rx=0 tx=0).stop 2023-05-20T01:22:11.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.584+0000 7f12aa90a700 1 -- 172.21.15.16:0/3382152786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12a40cfb30 msgr2=0x7f12a414e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:11.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.584+0000 7f12aa90a700 1 --2- 172.21.15.16:0/3382152786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12a40cfb30 0x7f12a414e410 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f129c002230 tx=0x7f129c00cf50 comp rx=0 tx=0).stop 2023-05-20T01:22:11.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.584+0000 7f12aa90a700 1 -- 172.21.15.16:0/3382152786 shutdown_connections 2023-05-20T01:22:11.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.584+0000 7f12aa90a700 1 --2- 172.21.15.16:0/3382152786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12a40cfb30 0x7f12a414e410 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:11.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.584+0000 7f12aa90a700 1 --2- 172.21.15.16:0/3382152786 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f12a406a510 0x7f12a40c4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:11.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.584+0000 7f12aa90a700 1 --2- 172.21.15.16:0/3382152786 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f128406ace0 0x7f128406d190 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:11.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.584+0000 7f12aa90a700 1 --2- 172.21.15.16:0/3382152786 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f12a406a8e0 0x7f12a40b5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:11.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.584+0000 7f12aa90a700 1 -- 172.21.15.16:0/3382152786 >> 172.21.15.16:0/3382152786 conn(0x7f12a40d0c30 msgr2=0x7f12a40cee90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:11.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.584+0000 7f12aa90a700 1 -- 172.21.15.16:0/3382152786 shutdown_connections 2023-05-20T01:22:11.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.584+0000 7f12aa90a700 1 -- 172.21.15.16:0/3382152786 wait complete. 2023-05-20T01:22:11.592 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2023-05-20T01:22:11.601 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-05-20T01:22:11.601 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:22:11.601 DEBUG:teuthology.orchestra.run.smithi016:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-05-20T01:22:11.734 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2023-05-20T01:22:11.735 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:22:11.735 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-05-20T01:22:11.762 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-20T01:22:11.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.922+0000 7f9cef953700 1 Processor -- start 2023-05-20T01:22:11.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.923+0000 7f9cef953700 1 -- start start 2023-05-20T01:22:11.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.924+0000 7f9cef953700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9ce8157190 0x7f9ce80cc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:11.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.924+0000 7f9cef953700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9ce8071ad0 0x7f9ce80ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:11.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.924+0000 7f9cef953700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ce80cb590 con 0x7f9ce80cccf0 2023-05-20T01:22:11.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.924+0000 7f9cef953700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ce8071f50 con 0x7f9ce8157190 2023-05-20T01:22:11.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.924+0000 7f9cef953700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce80ca9a0 con 0x7f9ce8071ad0 2023-05-20T01:22:11.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.924+0000 7f9ced6ef700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9ce8157190 0x7f9ce80cc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:11.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.924+0000 7f9ceceee700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f9ce80cccf0 0x7f9ce80713c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:45776/0 (socket says 172.21.15.16:45776) 2023-05-20T01:22:11.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.924+0000 7f9ceceee700 1 -- 172.21.15.16:0/3130316243 learned_addr learned my addr 172.21.15.16:0/3130316243 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:22:11.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.924+0000 7f9ced6ef700 1 -- 172.21.15.16:0/3130316243 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9ce8071ad0 msgr2=0x7f9ce80ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:11.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.924+0000 7f9ced6ef700 1 --2- 172.21.15.16:0/3130316243 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9ce8071ad0 0x7f9ce80ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:11.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.924+0000 7f9ced6ef700 1 -- 172.21.15.16:0/3130316243 >> v1:172.21.15.16:6789/0 conn(0x7f9ce80cccf0 legacy=0x7f9ce80713c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:11.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.924+0000 7f9ced6ef700 1 -- 172.21.15.16:0/3130316243 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ce80cac60 con 0x7f9ce8157190 2023-05-20T01:22:11.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.924+0000 7f9cdffff700 1 -- 172.21.15.16:0/3130316243 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1538113468 0 0) 0x7f9ce80cb590 con 0x7f9ce80cccf0 2023-05-20T01:22:11.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.924+0000 7f9ced6ef700 1 --2- 172.21.15.16:0/3130316243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ce8157190 0x7f9ce80cc7b0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f9ce4009840 tx=0x7f9ce4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93a719eab092e048 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:11.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.925+0000 7f9cdffff700 1 -- 172.21.15.16:0/3130316243 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce4011830 con 0x7f9ce8157190 2023-05-20T01:22:11.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.925+0000 7f9cdffff700 1 -- 172.21.15.16:0/3130316243 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ce4011990 con 0x7f9ce8157190 2023-05-20T01:22:11.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.925+0000 7f9cdffff700 1 -- 172.21.15.16:0/3130316243 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce4011d20 con 0x7f9ce8157190 2023-05-20T01:22:11.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.925+0000 7f9cef953700 1 -- 172.21.15.16:0/3130316243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ce8157190 msgr2=0x7f9ce80cc7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:11.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.925+0000 7f9cef953700 1 --2- 172.21.15.16:0/3130316243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ce8157190 0x7f9ce80cc7b0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f9ce4009840 tx=0x7f9ce4009b50 comp rx=0 tx=0).stop 2023-05-20T01:22:11.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.925+0000 7f9cef953700 1 -- 172.21.15.16:0/3130316243 shutdown_connections 2023-05-20T01:22:11.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.925+0000 7f9cef953700 1 --2- 172.21.15.16:0/3130316243 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9ce8071ad0 0x7f9ce80ce740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:11.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.925+0000 7f9cef953700 1 --2- 172.21.15.16:0/3130316243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ce8157190 0x7f9ce80cc7b0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:11.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.925+0000 7f9cef953700 1 -- 172.21.15.16:0/3130316243 >> 172.21.15.16:0/3130316243 conn(0x7f9ce80cffb0 msgr2=0x7f9ce80d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:11.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.925+0000 7f9cef953700 1 -- 172.21.15.16:0/3130316243 shutdown_connections 2023-05-20T01:22:11.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.925+0000 7f9cef953700 1 -- 172.21.15.16:0/3130316243 wait complete. 2023-05-20T01:22:11.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.927+0000 7f9cef953700 1 Processor -- start 2023-05-20T01:22:11.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.927+0000 7f9cef953700 1 -- start start 2023-05-20T01:22:11.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.927+0000 7f9cef953700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9ce8071ad0 0x7f9ce81f6a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:11.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.927+0000 7f9cef953700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9ce80cccf0 0x7f9ce8149240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:11.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.927+0000 7f9cef953700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ce8157190 0x7f9ce80b3e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:11.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.927+0000 7f9cef953700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce81f6f90 con 0x7f9ce8157190 2023-05-20T01:22:11.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.927+0000 7f9cef953700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce80aff50 con 0x7f9ce8071ad0 2023-05-20T01:22:11.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.927+0000 7f9cef953700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce80b0230 con 0x7f9ce80cccf0 2023-05-20T01:22:11.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.927+0000 7f9cedef0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ce8157190 0x7f9ce80b3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:11.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.927+0000 7f9cedef0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ce8157190 0x7f9ce80b3e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34256/0 (socket says 172.21.15.16:34256) 2023-05-20T01:22:11.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.927+0000 7f9cedef0700 1 -- 172.21.15.16:0/2308810501 learned_addr learned my addr 172.21.15.16:0/2308810501 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:22:11.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.927+0000 7f9ced6ef700 1 --2- 172.21.15.16:0/2308810501 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9ce8071ad0 0x7f9ce81f6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:11.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.928+0000 7f9ceceee700 1 --2- 172.21.15.16:0/2308810501 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9ce80cccf0 0x7f9ce8149240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:11.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.928+0000 7f9cedef0700 1 -- 172.21.15.16:0/2308810501 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9ce80cccf0 msgr2=0x7f9ce8149240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:11.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.928+0000 7f9cedef0700 1 --2- 172.21.15.16:0/2308810501 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9ce80cccf0 0x7f9ce8149240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:11.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.928+0000 7f9cedef0700 1 -- 172.21.15.16:0/2308810501 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9ce8071ad0 msgr2=0x7f9ce81f6a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:11.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.928+0000 7f9cedef0700 1 --2- 172.21.15.16:0/2308810501 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9ce8071ad0 0x7f9ce81f6a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:11.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.928+0000 7f9cedef0700 1 -- 172.21.15.16:0/2308810501 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ce40094a0 con 0x7f9ce8157190 2023-05-20T01:22:11.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.928+0000 7f9cedef0700 1 --2- 172.21.15.16:0/2308810501 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ce8157190 0x7f9ce80b3e80 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f9ce000b970 tx=0x7f9ce000bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:11.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.928+0000 7f9cddffb700 1 -- 172.21.15.16:0/2308810501 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce000d860 con 0x7f9ce8157190 2023-05-20T01:22:11.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.928+0000 7f9cddffb700 1 -- 172.21.15.16:0/2308810501 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ce000d9e0 con 0x7f9ce8157190 2023-05-20T01:22:11.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.928+0000 7f9cddffb700 1 -- 172.21.15.16:0/2308810501 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce000dd90 con 0x7f9ce8157190 2023-05-20T01:22:11.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.928+0000 7f9cef953700 1 -- 172.21.15.16:0/2308810501 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ce80cb220 con 0x7f9ce8157190 2023-05-20T01:22:11.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.928+0000 7f9cef953700 1 -- 172.21.15.16:0/2308810501 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ce80caf40 con 0x7f9ce8157190 2023-05-20T01:22:11.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.929+0000 7f9cef953700 1 -- 172.21.15.16:0/2308810501 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ce8157560 con 0x7f9ce8157190 2023-05-20T01:22:11.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.930+0000 7f9cddffb700 1 -- 172.21.15.16:0/2308810501 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9ce000e430 con 0x7f9ce8157190 2023-05-20T01:22:11.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.930+0000 7f9cddffb700 1 --2- 172.21.15.16:0/2308810501 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f9ccc06ace0 0x7f9ccc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:11.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.930+0000 7f9cddffb700 1 -- 172.21.15.16:0/2308810501 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6931+0+0 (secure 0 0 0) 0x7f9ce0006e40 con 0x7f9ce8157190 2023-05-20T01:22:11.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.930+0000 7f9ced6ef700 1 --2- 172.21.15.16:0/2308810501 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f9ccc06ace0 0x7f9ccc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:11.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.931+0000 7f9ced6ef700 1 --2- 172.21.15.16:0/2308810501 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f9ccc06ace0 0x7f9ccc06d190 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f9ce40090b0 tx=0x7f9ce401c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:11.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:11.935+0000 7f9cddffb700 1 -- 172.21.15.16:0/2308810501 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f9ce0054680 con 0x7f9ce8157190 2023-05-20T01:22:12.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:12.177+0000 7f9cef953700 1 -- 172.21.15.16:0/2308810501 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9ce80747b0 con 0x7f9ce8157190 2023-05-20T01:22:12.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:12.178+0000 7f9cddffb700 1 -- 172.21.15.16:0/2308810501 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4675 (secure 0 0 0) 0x7f9ce0057ca0 con 0x7f9ce8157190 2023-05-20T01:22:12.182 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:22:12.182 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":38,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5824,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/2609276800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":2609276800},{"type":"v1","addr":"172.21.15.132:6834","nonce":2609276800}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32},{"gid":5827,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/29554453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":29554453},{"type":"v1","addr":"172.21.15.132:6835","nonce":29554453}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32},{"gid":6192,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/4090303296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4090303296},{"type":"v1","addr":"172.21.15.16:6836","nonce":4090303296}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-20T01:19:51.915360+0000","modified":"2023-05-20T01:20:05.849669+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6195},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6195":{"gid":6195,"name":"c","rank":0,"incarnation":33,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6837/1427908556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":1427908556},{"type":"v1","addr":"172.21.15.16:6837","nonce":1427908556}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10,11],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-05-20T01:22:12.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:12.180+0000 7f9cef953700 1 -- 172.21.15.16:0/2308810501 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f9ccc06ace0 msgr2=0x7f9ccc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:12.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:12.180+0000 7f9cef953700 1 --2- 172.21.15.16:0/2308810501 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f9ccc06ace0 0x7f9ccc06d190 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f9ce40090b0 tx=0x7f9ce401c040 comp rx=0 tx=0).stop 2023-05-20T01:22:12.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:12.180+0000 7f9cef953700 1 -- 172.21.15.16:0/2308810501 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ce8157190 msgr2=0x7f9ce80b3e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:12.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:12.180+0000 7f9cef953700 1 --2- 172.21.15.16:0/2308810501 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ce8157190 0x7f9ce80b3e80 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f9ce000b970 tx=0x7f9ce000bc80 comp rx=0 tx=0).stop 2023-05-20T01:22:12.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:12.180+0000 7f9cef953700 1 -- 172.21.15.16:0/2308810501 shutdown_connections 2023-05-20T01:22:12.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:12.180+0000 7f9cef953700 1 --2- 172.21.15.16:0/2308810501 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ce8157190 0x7f9ce80b3e80 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:12.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:12.180+0000 7f9cef953700 1 --2- 172.21.15.16:0/2308810501 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9ce8071ad0 0x7f9ce81f6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:12.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:12.180+0000 7f9cef953700 1 --2- 172.21.15.16:0/2308810501 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f9ccc06ace0 0x7f9ccc06d190 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:12.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:12.180+0000 7f9cef953700 1 --2- 172.21.15.16:0/2308810501 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9ce80cccf0 0x7f9ce8149240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:12.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:12.180+0000 7f9cef953700 1 -- 172.21.15.16:0/2308810501 >> 172.21.15.16:0/2308810501 conn(0x7f9ce80cffb0 msgr2=0x7f9ce815bf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:12.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:12.180+0000 7f9cef953700 1 -- 172.21.15.16:0/2308810501 shutdown_connections 2023-05-20T01:22:12.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:12.180+0000 7f9cef953700 1 -- 172.21.15.16:0/2308810501 wait complete. 2023-05-20T01:22:12.186 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 38 2023-05-20T01:22:12.196 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2023-05-20T01:22:12.356 INFO:teuthology.orchestra.run.smithi016.stdout:{"message":"","return_code":0} 2023-05-20T01:22:12.356 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2023-05-20T01:22:12.357 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-05-20T01:22:12.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:12.480+0000 7f45a1c52700 1 Processor -- start 2023-05-20T01:22:12.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:12.480+0000 7f45a1c52700 1 -- start start 2023-05-20T01:22:12.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:12.481+0000 7f45a1c52700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f459c0cb320 0x7f459c070910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:12.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:12.481+0000 7f45a1c52700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f459c06d2a0 0x7f459c070e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:12.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:12.481+0000 7f45a1c52700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f459c0ca350 con 0x7f459c159000 2023-05-20T01:22:12.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:12.481+0000 7f45a1c52700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f459c15cac0 con 0x7f459c06d2a0 2023-05-20T01:22:12.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:12.481+0000 7f45a1c52700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f459c15cda0 con 0x7f459c0cb320 2023-05-20T01:22:12.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:12.481+0000 7f459bfff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f459c06d2a0 0x7f459c070e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:12.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:12.481+0000 7f459bfff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f459c06d2a0 0x7f459c070e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34264/0 (socket says 172.21.15.16:34264) 2023-05-20T01:22:12.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:12.481+0000 7f459bfff700 1 -- 172.21.15.16:0/781743633 learned_addr learned my addr 172.21.15.16:0/781743633 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:22:12.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:12.481+0000 7f459affd700 1 --2- 172.21.15.16:0/781743633 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f459c0cb320 0x7f459c070910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:12.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:12.481+0000 7f459bfff700 1 -- 172.21.15.16:0/781743633 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f459c0cb320 msgr2=0x7f459c070910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:12.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:12.481+0000 7f459bfff700 1 --2- 172.21.15.16:0/781743633 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f459c0cb320 0x7f459c070910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:12.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:12.481+0000 7f459bfff700 1 -- 172.21.15.16:0/781743633 >> v1:172.21.15.132:6789/0 conn(0x7f459c159000 legacy=0x7f459c06cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:12.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:12.481+0000 7f459bfff700 1 -- 172.21.15.16:0/781743633 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f459c15d080 con 0x7f459c06d2a0 2023-05-20T01:22:12.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:12.481+0000 7f459bfff700 1 --2- 172.21.15.16:0/781743633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f459c06d2a0 0x7f459c070e50 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f459400d150 tx=0x7f459400d460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc22ccf3f0f38cd3 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:12.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:12.481+0000 7f459a7fc700 1 -- 172.21.15.16:0/781743633 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4594010070 con 0x7f459c06d2a0 2023-05-20T01:22:12.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:12.481+0000 7f459a7fc700 1 -- 172.21.15.16:0/781743633 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f459400db70 con 0x7f459c06d2a0 2023-05-20T01:22:12.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:12.481+0000 7f459a7fc700 1 -- 172.21.15.16:0/781743633 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4594003a20 con 0x7f459c06d2a0 2023-05-20T01:22:12.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:12.482+0000 7f45a1c52700 1 -- 172.21.15.16:0/781743633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f459c06d2a0 msgr2=0x7f459c070e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:12.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:12.482+0000 7f45a1c52700 1 --2- 172.21.15.16:0/781743633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f459c06d2a0 0x7f459c070e50 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f459400d150 tx=0x7f459400d460 comp rx=0 tx=0).stop 2023-05-20T01:22:12.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:12.482+0000 7f45a1c52700 1 -- 172.21.15.16:0/781743633 shutdown_connections 2023-05-20T01:22:12.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:12.482+0000 7f45a1c52700 1 --2- 172.21.15.16:0/781743633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f459c06d2a0 0x7f459c070e50 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:12.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:12.482+0000 7f45a1c52700 1 --2- 172.21.15.16:0/781743633 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f459c0cb320 0x7f459c070910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:12.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:12.482+0000 7f45a1c52700 1 -- 172.21.15.16:0/781743633 >> 172.21.15.16:0/781743633 conn(0x7f459c0661b0 msgr2=0x7f459c066580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:12.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:12.482+0000 7f45a1c52700 1 -- 172.21.15.16:0/781743633 shutdown_connections 2023-05-20T01:22:12.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:12.482+0000 7f45a1c52700 1 -- 172.21.15.16:0/781743633 wait complete. 2023-05-20T01:22:12.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:12.484+0000 7f45a1c52700 1 Processor -- start 2023-05-20T01:22:12.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:12.484+0000 7f45a1c52700 1 -- start start 2023-05-20T01:22:12.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:12.485+0000 7f45a1c52700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f459c06d2a0 0x7f459c15bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:12.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:12.485+0000 7f45a1c52700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f459c0cb320 0x7f459c14d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:12.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:12.485+0000 7f45a1c52700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f459c159000 0x7f459c0b3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:12.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:12.485+0000 7f45a1c52700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f459c0b4370 con 0x7f459c06d2a0 2023-05-20T01:22:12.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:12.485+0000 7f45a1c52700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f459c0b4650 con 0x7f459c0cb320 2023-05-20T01:22:12.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:12.485+0000 7f45a1c52700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f459c0b4930 con 0x7f459c159000 2023-05-20T01:22:12.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:12.485+0000 7f459b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f459c06d2a0 0x7f459c15bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:12.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:12.485+0000 7f459b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f459c06d2a0 0x7f459c15bb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34266/0 (socket says 172.21.15.16:34266) 2023-05-20T01:22:12.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:12.485+0000 7f459b7fe700 1 -- 172.21.15.16:0/4002934485 learned_addr learned my addr 172.21.15.16:0/4002934485 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:22:12.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:12.485+0000 7f459bfff700 1 --2- 172.21.15.16:0/4002934485 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f459c159000 0x7f459c0b3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:12.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:12.485+0000 7f459affd700 1 --2- 172.21.15.16:0/4002934485 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f459c0cb320 0x7f459c14d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:12.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:12.485+0000 7f459b7fe700 1 -- 172.21.15.16:0/4002934485 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f459c159000 msgr2=0x7f459c0b3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:12.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:12.485+0000 7f459b7fe700 1 --2- 172.21.15.16:0/4002934485 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f459c159000 0x7f459c0b3ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:12.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:12.485+0000 7f459b7fe700 1 -- 172.21.15.16:0/4002934485 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f459c0cb320 msgr2=0x7f459c14d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:12.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:12.485+0000 7f459b7fe700 1 --2- 172.21.15.16:0/4002934485 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f459c0cb320 0x7f459c14d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:12.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:12.485+0000 7f459b7fe700 1 -- 172.21.15.16:0/4002934485 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f459400bd40 con 0x7f459c06d2a0 2023-05-20T01:22:12.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:12.485+0000 7f459b7fe700 1 --2- 172.21.15.16:0/4002934485 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f459c06d2a0 0x7f459c15bb70 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f458c00b4a0 tx=0x7f458c00b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:12.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:12.486+0000 7f4583fff700 1 -- 172.21.15.16:0/4002934485 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f458c011830 con 0x7f459c06d2a0 2023-05-20T01:22:12.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:12.486+0000 7f4583fff700 1 -- 172.21.15.16:0/4002934485 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f458c011990 con 0x7f459c06d2a0 2023-05-20T01:22:12.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:12.486+0000 7f45a1c52700 1 -- 172.21.15.16:0/4002934485 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f459c15d880 con 0x7f459c06d2a0 2023-05-20T01:22:12.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:12.486+0000 7f4583fff700 1 -- 172.21.15.16:0/4002934485 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f458c011d40 con 0x7f459c06d2a0 2023-05-20T01:22:12.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:12.486+0000 7f45a1c52700 1 -- 172.21.15.16:0/4002934485 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f459c15d360 con 0x7f459c06d2a0 2023-05-20T01:22:12.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:12.487+0000 7f45a1c52700 1 -- 172.21.15.16:0/4002934485 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f459c0b1b00 con 0x7f459c06d2a0 2023-05-20T01:22:12.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:12.487+0000 7f4583fff700 1 -- 172.21.15.16:0/4002934485 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f458c01a400 con 0x7f459c06d2a0 2023-05-20T01:22:12.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:12.487+0000 7f4583fff700 1 --2- 172.21.15.16:0/4002934485 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f458806ace0 0x7f458806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:12.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:12.488+0000 7f4583fff700 1 -- 172.21.15.16:0/4002934485 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6931+0+0 (secure 0 0 0) 0x7f458c089570 con 0x7f459c06d2a0 2023-05-20T01:22:12.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:12.488+0000 7f459affd700 1 --2- 172.21.15.16:0/4002934485 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f458806ace0 0x7f458806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:12.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:12.488+0000 7f459affd700 1 --2- 172.21.15.16:0/4002934485 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f458806ace0 0x7f458806d190 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f459c15b960 tx=0x7f4590008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:12.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:12.492+0000 7f4583fff700 1 -- 172.21.15.16:0/4002934485 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f458c0542b0 con 0x7f459c06d2a0 2023-05-20T01:22:12.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:12.735+0000 7f45a1c52700 1 -- 172.21.15.16:0/4002934485 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f459c0747b0 con 0x7f459c06d2a0 2023-05-20T01:22:13.381 INFO:tasks.ceph.mon.a.smithi016.stderr:2023-05-20T01:22:13.377+0000 7f49eb8ae700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-05-20T01:22:13.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.383+0000 7f4583fff700 1 -- 172.21.15.16:0/4002934485 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v39) v1 ==== 161+0+0 (secure 0 0 0) 0x7f458c0578d0 con 0x7f459c06d2a0 2023-05-20T01:22:13.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.388+0000 7f45a1c52700 1 -- 172.21.15.16:0/4002934485 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f458806ace0 msgr2=0x7f458806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:13.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.388+0000 7f45a1c52700 1 --2- 172.21.15.16:0/4002934485 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f458806ace0 0x7f458806d190 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f459c15b960 tx=0x7f4590008040 comp rx=0 tx=0).stop 2023-05-20T01:22:13.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.388+0000 7f45a1c52700 1 -- 172.21.15.16:0/4002934485 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f459c06d2a0 msgr2=0x7f459c15bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:13.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.388+0000 7f45a1c52700 1 --2- 172.21.15.16:0/4002934485 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f459c06d2a0 0x7f459c15bb70 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f458c00b4a0 tx=0x7f458c00b860 comp rx=0 tx=0).stop 2023-05-20T01:22:13.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.388+0000 7f45a1c52700 1 -- 172.21.15.16:0/4002934485 shutdown_connections 2023-05-20T01:22:13.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.388+0000 7f45a1c52700 1 --2- 172.21.15.16:0/4002934485 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f458806ace0 0x7f458806d190 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:13.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.388+0000 7f45a1c52700 1 --2- 172.21.15.16:0/4002934485 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f459c159000 0x7f459c0b3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:13.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.388+0000 7f45a1c52700 1 --2- 172.21.15.16:0/4002934485 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f459c06d2a0 0x7f459c15bb70 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:13.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.388+0000 7f45a1c52700 1 --2- 172.21.15.16:0/4002934485 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f459c0cb320 0x7f459c14d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:13.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.388+0000 7f45a1c52700 1 -- 172.21.15.16:0/4002934485 >> 172.21.15.16:0/4002934485 conn(0x7f459c0661b0 msgr2=0x7f459c1e2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:13.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.388+0000 7f45a1c52700 1 -- 172.21.15.16:0/4002934485 shutdown_connections 2023-05-20T01:22:13.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.388+0000 7f45a1c52700 1 -- 172.21.15.16:0/4002934485 wait complete. 2023-05-20T01:22:13.394 INFO:teuthology.orchestra.run.smithi016.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-05-20T01:22:13.408 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-05-20T01:22:13.411 DEBUG:teuthology.orchestra.run.smithi132:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-05-20T01:22:13.432 INFO:tasks.ceph.mds.c.smithi016.stdout:starting mds.c at 2023-05-20T01:22:13.464 INFO:teuthology.orchestra.run.smithi016.stdout:[global] 2023-05-20T01:22:13.465 INFO:teuthology.orchestra.run.smithi016.stdout: chdir = "" 2023-05-20T01:22:13.465 INFO:teuthology.orchestra.run.smithi016.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-05-20T01:22:13.465 INFO:teuthology.orchestra.run.smithi016.stdout: auth supported = cephx 2023-05-20T01:22:13.465 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:22:13.465 INFO:teuthology.orchestra.run.smithi016.stdout: filestore xattr use omap = true 2023-05-20T01:22:13.465 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:22:13.466 INFO:teuthology.orchestra.run.smithi016.stdout: mon clock drift allowed = 1.000 2023-05-20T01:22:13.466 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:22:13.466 INFO:teuthology.orchestra.run.smithi016.stdout: osd crush chooseleaf type = 0 2023-05-20T01:22:13.466 INFO:teuthology.orchestra.run.smithi016.stdout: auth debug = true 2023-05-20T01:22:13.466 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:22:13.466 INFO:teuthology.orchestra.run.smithi016.stdout: ms die on old message = true 2023-05-20T01:22:13.466 INFO:teuthology.orchestra.run.smithi016.stdout: ms die on bug = true 2023-05-20T01:22:13.467 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:22:13.467 INFO:teuthology.orchestra.run.smithi016.stdout: mon max pg per osd = 10000 # >= luminous 2023-05-20T01:22:13.467 INFO:teuthology.orchestra.run.smithi016.stdout: mon pg warn max object skew = 0 2023-05-20T01:22:13.467 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:22:13.467 INFO:teuthology.orchestra.run.smithi016.stdout: # disable pg_autoscaler by default for new pools 2023-05-20T01:22:13.467 INFO:teuthology.orchestra.run.smithi016.stdout: osd_pool_default_pg_autoscale_mode = off 2023-05-20T01:22:13.467 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:22:13.468 INFO:teuthology.orchestra.run.smithi016.stdout: osd pool default size = 2 2023-05-20T01:22:13.468 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:22:13.468 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd allow primary affinity = true 2023-05-20T01:22:13.468 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd allow pg remap = true 2023-05-20T01:22:13.468 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on legacy crush tunables = false 2023-05-20T01:22:13.468 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on crush straw calc version zero = false 2023-05-20T01:22:13.468 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on no sortbitwise = false 2023-05-20T01:22:13.469 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on osd down out interval zero = false 2023-05-20T01:22:13.469 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on too few osds = false 2023-05-20T01:22:13.469 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-05-20T01:22:13.469 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_pool_no_redundancy = false 2023-05-20T01:22:13.469 INFO:teuthology.orchestra.run.smithi016.stdout: mon_allow_pool_size_one = true 2023-05-20T01:22:13.469 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:22:13.469 INFO:teuthology.orchestra.run.smithi016.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-05-20T01:22:13.470 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:22:13.470 INFO:teuthology.orchestra.run.smithi016.stdout: osd default data pool replay window = 5 2023-05-20T01:22:13.470 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:22:13.470 INFO:teuthology.orchestra.run.smithi016.stdout: mon allow pool delete = true 2023-05-20T01:22:13.470 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:22:13.470 INFO:teuthology.orchestra.run.smithi016.stdout: mon cluster log file level = debug 2023-05-20T01:22:13.470 INFO:teuthology.orchestra.run.smithi016.stdout: debug asserts on shutdown = true 2023-05-20T01:22:13.470 INFO:teuthology.orchestra.run.smithi016.stdout: mon health detail to clog = false 2023-05-20T01:22:13.471 INFO:teuthology.orchestra.run.smithi016.stdout: mon host = "172.21.15.16,172.21.15.132,[v2:172.21.15.132:3301,v1:172.21.15.132:6790]" 2023-05-20T01:22:13.471 INFO:teuthology.orchestra.run.smithi016.stdout: ms die on skipped message = False 2023-05-20T01:22:13.471 INFO:teuthology.orchestra.run.smithi016.stdout: fsid = c2245cc5-ecab-4087-a59b-bc81db901676 2023-05-20T01:22:13.471 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:22:13.471 INFO:teuthology.orchestra.run.smithi016.stdout:[osd] 2023-05-20T01:22:13.471 INFO:teuthology.orchestra.run.smithi016.stdout: osd journal size = 100 2023-05-20T01:22:13.472 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:22:13.472 INFO:teuthology.orchestra.run.smithi016.stdout: osd scrub load threshold = 5.0 2023-05-20T01:22:13.472 INFO:teuthology.orchestra.run.smithi016.stdout: osd scrub max interval = 600 2023-05-20T01:22:13.472 INFO:teuthology.orchestra.run.smithi016.stdout: osd mclock profile = high_recovery_ops 2023-05-20T01:22:13.472 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:22:13.472 INFO:teuthology.orchestra.run.smithi016.stdout: osd recover clone overlap = true 2023-05-20T01:22:13.472 INFO:teuthology.orchestra.run.smithi016.stdout: osd recovery max chunk = 1048576 2023-05-20T01:22:13.473 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:22:13.473 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug shutdown = true 2023-05-20T01:22:13.473 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug op order = true 2023-05-20T01:22:13.473 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug verify stray on activate = true 2023-05-20T01:22:13.473 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:22:13.473 INFO:teuthology.orchestra.run.smithi016.stdout: osd open classes on start = true 2023-05-20T01:22:13.473 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug pg log writeout = true 2023-05-20T01:22:13.473 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:22:13.474 INFO:teuthology.orchestra.run.smithi016.stdout: osd deep scrub update digest min age = 30 2023-05-20T01:22:13.474 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:22:13.474 INFO:teuthology.orchestra.run.smithi016.stdout: osd map max advance = 10 2023-05-20T01:22:13.474 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:22:13.474 INFO:teuthology.orchestra.run.smithi016.stdout: journal zero on create = true 2023-05-20T01:22:13.474 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:22:13.474 INFO:teuthology.orchestra.run.smithi016.stdout: filestore ondisk finisher threads = 3 2023-05-20T01:22:13.475 INFO:teuthology.orchestra.run.smithi016.stdout: filestore apply finisher threads = 3 2023-05-20T01:22:13.475 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:22:13.475 INFO:teuthology.orchestra.run.smithi016.stdout: bdev debug aio = true 2023-05-20T01:22:13.475 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug misdirected ops = true 2023-05-20T01:22:13.475 INFO:teuthology.orchestra.run.smithi016.stdout: bluestore block size = 96636764160 2023-05-20T01:22:13.475 INFO:teuthology.orchestra.run.smithi016.stdout: bluestore fsck on mount = True 2023-05-20T01:22:13.475 INFO:teuthology.orchestra.run.smithi016.stdout: debug bluefs = 20 2023-05-20T01:22:13.476 INFO:teuthology.orchestra.run.smithi016.stdout: debug bluestore = 20 2023-05-20T01:22:13.476 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2023-05-20T01:22:13.476 INFO:teuthology.orchestra.run.smithi016.stdout: debug osd = 20 2023-05-20T01:22:13.476 INFO:teuthology.orchestra.run.smithi016.stdout: debug rocksdb = 10 2023-05-20T01:22:13.476 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd backfillfull_ratio = 0.85 2023-05-20T01:22:13.476 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd full ratio = 0.9 2023-05-20T01:22:13.476 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd nearfull ratio = 0.8 2023-05-20T01:22:13.477 INFO:teuthology.orchestra.run.smithi016.stdout: osd failsafe full ratio = 0.95 2023-05-20T01:22:13.477 INFO:teuthology.orchestra.run.smithi016.stdout: osd objectstore = bluestore 2023-05-20T01:22:13.477 INFO:teuthology.orchestra.run.smithi016.stdout: osd op complaint time = 180 2023-05-20T01:22:13.477 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:22:13.477 INFO:teuthology.orchestra.run.smithi016.stdout:[mgr] 2023-05-20T01:22:13.477 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2023-05-20T01:22:13.477 INFO:teuthology.orchestra.run.smithi016.stdout: debug mgr = 20 2023-05-20T01:22:13.478 INFO:teuthology.orchestra.run.smithi016.stdout: debug mon = 20 2023-05-20T01:22:13.478 INFO:teuthology.orchestra.run.smithi016.stdout: debug auth = 20 2023-05-20T01:22:13.478 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min pgs per osd = 4 2023-05-20T01:22:13.478 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min bytes per osd = 10 2023-05-20T01:22:13.478 INFO:teuthology.orchestra.run.smithi016.stdout: mgr/telemetry/nag = false 2023-05-20T01:22:13.478 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:22:13.478 INFO:teuthology.orchestra.run.smithi016.stdout:[mon] 2023-05-20T01:22:13.479 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2023-05-20T01:22:13.479 INFO:teuthology.orchestra.run.smithi016.stdout: debug mon = 20 2023-05-20T01:22:13.479 INFO:teuthology.orchestra.run.smithi016.stdout: debug paxos = 20 2023-05-20T01:22:13.479 INFO:teuthology.orchestra.run.smithi016.stdout: debug auth = 20 2023-05-20T01:22:13.479 INFO:teuthology.orchestra.run.smithi016.stdout: mon data avail warn = 5 2023-05-20T01:22:13.479 INFO:teuthology.orchestra.run.smithi016.stdout: mon mgr mkfs grace = 240 2023-05-20T01:22:13.479 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min pgs per osd = 4 2023-05-20T01:22:13.479 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd reporter subtree level = osd 2023-05-20T01:22:13.480 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd prime pg temp = true 2023-05-20T01:22:13.480 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min bytes per osd = 10 2023-05-20T01:22:13.480 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:22:13.480 INFO:teuthology.orchestra.run.smithi016.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-05-20T01:22:13.480 INFO:teuthology.orchestra.run.smithi016.stdout: auth mon ticket ttl = 660 # 11m 2023-05-20T01:22:13.480 INFO:teuthology.orchestra.run.smithi016.stdout: auth service ticket ttl = 240 # 4m 2023-05-20T01:22:13.480 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:22:13.481 INFO:teuthology.orchestra.run.smithi016.stdout: # don't complain about insecure global_id in the test suite 2023-05-20T01:22:13.481 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-05-20T01:22:13.481 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-05-20T01:22:13.481 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:22:13.481 INFO:teuthology.orchestra.run.smithi016.stdout: # 1m isn't quite enough 2023-05-20T01:22:13.481 INFO:teuthology.orchestra.run.smithi016.stdout: mon_down_mkfs_grace = 2m 2023-05-20T01:22:13.481 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:22:13.482 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_filestore_osds = false 2023-05-20T01:22:13.482 INFO:teuthology.orchestra.run.smithi016.stdout: mon op complaint time = 120 2023-05-20T01:22:13.482 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:22:13.482 INFO:teuthology.orchestra.run.smithi016.stdout:[client] 2023-05-20T01:22:13.482 INFO:teuthology.orchestra.run.smithi016.stdout: rgw cache enabled = true 2023-05-20T01:22:13.482 INFO:teuthology.orchestra.run.smithi016.stdout: rgw enable ops log = true 2023-05-20T01:22:13.482 INFO:teuthology.orchestra.run.smithi016.stdout: rgw enable usage log = true 2023-05-20T01:22:13.483 INFO:teuthology.orchestra.run.smithi016.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-05-20T01:22:13.483 INFO:teuthology.orchestra.run.smithi016.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-05-20T01:22:13.483 INFO:teuthology.orchestra.run.smithi016.stdout: client mount timeout = 600 2023-05-20T01:22:13.483 INFO:teuthology.orchestra.run.smithi016.stdout: debug client = 20 2023-05-20T01:22:13.483 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2023-05-20T01:22:13.483 INFO:teuthology.orchestra.run.smithi016.stdout: pid file = "" 2023-05-20T01:22:13.483 INFO:teuthology.orchestra.run.smithi016.stdout: rados mon op timeout = 900 2023-05-20T01:22:13.484 INFO:teuthology.orchestra.run.smithi016.stdout: rados osd op timeout = 900 2023-05-20T01:22:13.484 INFO:teuthology.orchestra.run.smithi016.stdout:[mon.a] 2023-05-20T01:22:13.484 INFO:teuthology.orchestra.run.smithi016.stdout:[mon.b] 2023-05-20T01:22:13.484 INFO:teuthology.orchestra.run.smithi016.stdout:[mon.c] 2023-05-20T01:22:13.484 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.a] 2023-05-20T01:22:13.484 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.c] 2023-05-20T01:22:13.484 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.b] 2023-05-20T01:22:13.484 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.d] 2023-05-20T01:22:13.485 INFO:teuthology.orchestra.run.smithi016.stdout:[mds] 2023-05-20T01:22:13.485 INFO:teuthology.orchestra.run.smithi016.stdout: debug mds = 20 2023-05-20T01:22:13.485 INFO:teuthology.orchestra.run.smithi016.stdout: debug mds balancer = 20 2023-05-20T01:22:13.485 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2023-05-20T01:22:13.485 INFO:teuthology.orchestra.run.smithi016.stdout: mds debug frag = True 2023-05-20T01:22:13.485 INFO:teuthology.orchestra.run.smithi016.stdout: mds debug scatterstat = False 2023-05-20T01:22:13.485 INFO:teuthology.orchestra.run.smithi016.stdout: mds op complaint time = 180 2023-05-20T01:22:13.486 INFO:teuthology.orchestra.run.smithi016.stdout: mds verify scatter = False 2023-05-20T01:22:13.486 INFO:teuthology.orchestra.run.smithi016.stdout: osd op complaint time = 180 2023-05-20T01:22:13.486 INFO:teuthology.orchestra.run.smithi016.stdout: rados mon op timeout = 900 2023-05-20T01:22:13.486 INFO:teuthology.orchestra.run.smithi016.stdout: rados osd op timeout = 900 2023-05-20T01:22:13.487 INFO:teuthology.orchestra.run.smithi132.stdout:[global] 2023-05-20T01:22:13.488 INFO:teuthology.orchestra.run.smithi132.stdout: chdir = "" 2023-05-20T01:22:13.488 INFO:teuthology.orchestra.run.smithi132.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-05-20T01:22:13.488 INFO:teuthology.orchestra.run.smithi132.stdout: auth supported = cephx 2023-05-20T01:22:13.488 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:22:13.488 INFO:teuthology.orchestra.run.smithi132.stdout: filestore xattr use omap = true 2023-05-20T01:22:13.488 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:22:13.488 INFO:teuthology.orchestra.run.smithi132.stdout: mon clock drift allowed = 1.000 2023-05-20T01:22:13.489 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:22:13.489 INFO:teuthology.orchestra.run.smithi132.stdout: osd crush chooseleaf type = 0 2023-05-20T01:22:13.489 INFO:teuthology.orchestra.run.smithi132.stdout: auth debug = true 2023-05-20T01:22:13.489 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:22:13.489 INFO:teuthology.orchestra.run.smithi132.stdout: ms die on old message = true 2023-05-20T01:22:13.489 INFO:teuthology.orchestra.run.smithi132.stdout: ms die on bug = true 2023-05-20T01:22:13.489 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:22:13.490 INFO:teuthology.orchestra.run.smithi132.stdout: mon max pg per osd = 10000 # >= luminous 2023-05-20T01:22:13.490 INFO:teuthology.orchestra.run.smithi132.stdout: mon pg warn max object skew = 0 2023-05-20T01:22:13.490 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:22:13.490 INFO:teuthology.orchestra.run.smithi132.stdout: # disable pg_autoscaler by default for new pools 2023-05-20T01:22:13.490 INFO:teuthology.orchestra.run.smithi132.stdout: osd_pool_default_pg_autoscale_mode = off 2023-05-20T01:22:13.490 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:22:13.490 INFO:teuthology.orchestra.run.smithi132.stdout: osd pool default size = 2 2023-05-20T01:22:13.491 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:22:13.491 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd allow primary affinity = true 2023-05-20T01:22:13.491 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd allow pg remap = true 2023-05-20T01:22:13.491 INFO:teuthology.orchestra.run.smithi132.stdout: mon warn on legacy crush tunables = false 2023-05-20T01:22:13.491 INFO:teuthology.orchestra.run.smithi132.stdout: mon warn on crush straw calc version zero = false 2023-05-20T01:22:13.491 INFO:teuthology.orchestra.run.smithi132.stdout: mon warn on no sortbitwise = false 2023-05-20T01:22:13.491 INFO:teuthology.orchestra.run.smithi132.stdout: mon warn on osd down out interval zero = false 2023-05-20T01:22:13.492 INFO:teuthology.orchestra.run.smithi132.stdout: mon warn on too few osds = false 2023-05-20T01:22:13.492 INFO:teuthology.orchestra.run.smithi132.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-05-20T01:22:13.492 INFO:teuthology.orchestra.run.smithi132.stdout: mon_warn_on_pool_no_redundancy = false 2023-05-20T01:22:13.492 INFO:teuthology.orchestra.run.smithi132.stdout: mon_allow_pool_size_one = true 2023-05-20T01:22:13.492 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:22:13.492 INFO:teuthology.orchestra.run.smithi132.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-05-20T01:22:13.493 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:22:13.493 INFO:teuthology.orchestra.run.smithi132.stdout: osd default data pool replay window = 5 2023-05-20T01:22:13.493 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:22:13.494 INFO:teuthology.orchestra.run.smithi132.stdout: mon allow pool delete = true 2023-05-20T01:22:13.494 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:22:13.494 INFO:teuthology.orchestra.run.smithi132.stdout: mon cluster log file level = debug 2023-05-20T01:22:13.494 INFO:teuthology.orchestra.run.smithi132.stdout: debug asserts on shutdown = true 2023-05-20T01:22:13.494 INFO:teuthology.orchestra.run.smithi132.stdout: mon health detail to clog = false 2023-05-20T01:22:13.494 INFO:teuthology.orchestra.run.smithi132.stdout: mon host = "172.21.15.16,172.21.15.132,[v2:172.21.15.132:3301,v1:172.21.15.132:6790]" 2023-05-20T01:22:13.494 INFO:teuthology.orchestra.run.smithi132.stdout: ms die on skipped message = False 2023-05-20T01:22:13.495 INFO:teuthology.orchestra.run.smithi132.stdout: fsid = c2245cc5-ecab-4087-a59b-bc81db901676 2023-05-20T01:22:13.495 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:22:13.495 INFO:teuthology.orchestra.run.smithi132.stdout:[osd] 2023-05-20T01:22:13.495 INFO:teuthology.orchestra.run.smithi132.stdout: osd journal size = 100 2023-05-20T01:22:13.495 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:22:13.495 INFO:teuthology.orchestra.run.smithi132.stdout: osd scrub load threshold = 5.0 2023-05-20T01:22:13.496 INFO:teuthology.orchestra.run.smithi132.stdout: osd scrub max interval = 600 2023-05-20T01:22:13.496 INFO:teuthology.orchestra.run.smithi132.stdout: osd mclock profile = high_recovery_ops 2023-05-20T01:22:13.496 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:22:13.496 INFO:teuthology.orchestra.run.smithi132.stdout: osd recover clone overlap = true 2023-05-20T01:22:13.496 INFO:teuthology.orchestra.run.smithi132.stdout: osd recovery max chunk = 1048576 2023-05-20T01:22:13.496 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:22:13.497 INFO:teuthology.orchestra.run.smithi132.stdout: osd debug shutdown = true 2023-05-20T01:22:13.497 INFO:teuthology.orchestra.run.smithi132.stdout: osd debug op order = true 2023-05-20T01:22:13.497 INFO:teuthology.orchestra.run.smithi132.stdout: osd debug verify stray on activate = true 2023-05-20T01:22:13.497 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:22:13.497 INFO:teuthology.orchestra.run.smithi132.stdout: osd open classes on start = true 2023-05-20T01:22:13.498 INFO:teuthology.orchestra.run.smithi132.stdout: osd debug pg log writeout = true 2023-05-20T01:22:13.498 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:22:13.498 INFO:teuthology.orchestra.run.smithi132.stdout: osd deep scrub update digest min age = 30 2023-05-20T01:22:13.498 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:22:13.498 INFO:teuthology.orchestra.run.smithi132.stdout: osd map max advance = 10 2023-05-20T01:22:13.498 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:22:13.499 INFO:teuthology.orchestra.run.smithi132.stdout: journal zero on create = true 2023-05-20T01:22:13.499 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:22:13.499 INFO:teuthology.orchestra.run.smithi132.stdout: filestore ondisk finisher threads = 3 2023-05-20T01:22:13.499 INFO:teuthology.orchestra.run.smithi132.stdout: filestore apply finisher threads = 3 2023-05-20T01:22:13.499 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:22:13.499 INFO:teuthology.orchestra.run.smithi132.stdout: bdev debug aio = true 2023-05-20T01:22:13.499 INFO:teuthology.orchestra.run.smithi132.stdout: osd debug misdirected ops = true 2023-05-20T01:22:13.500 INFO:teuthology.orchestra.run.smithi132.stdout: bluestore block size = 96636764160 2023-05-20T01:22:13.500 INFO:teuthology.orchestra.run.smithi132.stdout: bluestore fsck on mount = True 2023-05-20T01:22:13.500 INFO:teuthology.orchestra.run.smithi132.stdout: debug bluefs = 20 2023-05-20T01:22:13.500 INFO:teuthology.orchestra.run.smithi132.stdout: debug bluestore = 20 2023-05-20T01:22:13.500 INFO:teuthology.orchestra.run.smithi132.stdout: debug ms = 1 2023-05-20T01:22:13.500 INFO:teuthology.orchestra.run.smithi132.stdout: debug osd = 20 2023-05-20T01:22:13.500 INFO:teuthology.orchestra.run.smithi132.stdout: debug rocksdb = 10 2023-05-20T01:22:13.501 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd backfillfull_ratio = 0.85 2023-05-20T01:22:13.501 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd full ratio = 0.9 2023-05-20T01:22:13.501 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd nearfull ratio = 0.8 2023-05-20T01:22:13.501 INFO:teuthology.orchestra.run.smithi132.stdout: osd failsafe full ratio = 0.95 2023-05-20T01:22:13.501 INFO:teuthology.orchestra.run.smithi132.stdout: osd objectstore = bluestore 2023-05-20T01:22:13.501 INFO:teuthology.orchestra.run.smithi132.stdout: osd op complaint time = 180 2023-05-20T01:22:13.501 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:22:13.502 INFO:teuthology.orchestra.run.smithi132.stdout:[mgr] 2023-05-20T01:22:13.502 INFO:teuthology.orchestra.run.smithi132.stdout: debug ms = 1 2023-05-20T01:22:13.502 INFO:teuthology.orchestra.run.smithi132.stdout: debug mgr = 20 2023-05-20T01:22:13.502 INFO:teuthology.orchestra.run.smithi132.stdout: debug mon = 20 2023-05-20T01:22:13.502 INFO:teuthology.orchestra.run.smithi132.stdout: debug auth = 20 2023-05-20T01:22:13.502 INFO:teuthology.orchestra.run.smithi132.stdout: mon reweight min pgs per osd = 4 2023-05-20T01:22:13.502 INFO:teuthology.orchestra.run.smithi132.stdout: mon reweight min bytes per osd = 10 2023-05-20T01:22:13.502 INFO:teuthology.orchestra.run.smithi132.stdout: mgr/telemetry/nag = false 2023-05-20T01:22:13.503 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:22:13.503 INFO:teuthology.orchestra.run.smithi132.stdout:[mon] 2023-05-20T01:22:13.503 INFO:teuthology.orchestra.run.smithi132.stdout: debug ms = 1 2023-05-20T01:22:13.503 INFO:teuthology.orchestra.run.smithi132.stdout: debug mon = 20 2023-05-20T01:22:13.503 INFO:teuthology.orchestra.run.smithi132.stdout: debug paxos = 20 2023-05-20T01:22:13.503 INFO:teuthology.orchestra.run.smithi132.stdout: debug auth = 20 2023-05-20T01:22:13.503 INFO:teuthology.orchestra.run.smithi132.stdout: mon data avail warn = 5 2023-05-20T01:22:13.503 INFO:teuthology.orchestra.run.smithi132.stdout: mon mgr mkfs grace = 240 2023-05-20T01:22:13.504 INFO:teuthology.orchestra.run.smithi132.stdout: mon reweight min pgs per osd = 4 2023-05-20T01:22:13.504 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd reporter subtree level = osd 2023-05-20T01:22:13.504 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd prime pg temp = true 2023-05-20T01:22:13.504 INFO:teuthology.orchestra.run.smithi132.stdout: mon reweight min bytes per osd = 10 2023-05-20T01:22:13.504 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:22:13.504 INFO:teuthology.orchestra.run.smithi132.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-05-20T01:22:13.504 INFO:teuthology.orchestra.run.smithi132.stdout: auth mon ticket ttl = 660 # 11m 2023-05-20T01:22:13.505 INFO:teuthology.orchestra.run.smithi132.stdout: auth service ticket ttl = 240 # 4m 2023-05-20T01:22:13.505 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:22:13.505 INFO:teuthology.orchestra.run.smithi132.stdout: # don't complain about insecure global_id in the test suite 2023-05-20T01:22:13.505 INFO:teuthology.orchestra.run.smithi132.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-05-20T01:22:13.505 INFO:teuthology.orchestra.run.smithi132.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-05-20T01:22:13.505 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:22:13.505 INFO:teuthology.orchestra.run.smithi132.stdout: # 1m isn't quite enough 2023-05-20T01:22:13.506 INFO:teuthology.orchestra.run.smithi132.stdout: mon_down_mkfs_grace = 2m 2023-05-20T01:22:13.506 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:22:13.506 INFO:teuthology.orchestra.run.smithi132.stdout: mon_warn_on_filestore_osds = false 2023-05-20T01:22:13.506 INFO:teuthology.orchestra.run.smithi132.stdout: mon op complaint time = 120 2023-05-20T01:22:13.506 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:22:13.506 INFO:teuthology.orchestra.run.smithi132.stdout:[client] 2023-05-20T01:22:13.506 INFO:teuthology.orchestra.run.smithi132.stdout: rgw cache enabled = true 2023-05-20T01:22:13.506 INFO:teuthology.orchestra.run.smithi132.stdout: rgw enable ops log = true 2023-05-20T01:22:13.507 INFO:teuthology.orchestra.run.smithi132.stdout: rgw enable usage log = true 2023-05-20T01:22:13.507 INFO:teuthology.orchestra.run.smithi132.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-05-20T01:22:13.507 INFO:teuthology.orchestra.run.smithi132.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-05-20T01:22:13.507 INFO:teuthology.orchestra.run.smithi132.stdout: client mount timeout = 600 2023-05-20T01:22:13.507 INFO:teuthology.orchestra.run.smithi132.stdout: debug client = 20 2023-05-20T01:22:13.507 INFO:teuthology.orchestra.run.smithi132.stdout: debug ms = 1 2023-05-20T01:22:13.507 INFO:teuthology.orchestra.run.smithi132.stdout: pid file = "" 2023-05-20T01:22:13.508 INFO:teuthology.orchestra.run.smithi132.stdout: rados mon op timeout = 900 2023-05-20T01:22:13.508 INFO:teuthology.orchestra.run.smithi132.stdout: rados osd op timeout = 900 2023-05-20T01:22:13.508 INFO:teuthology.orchestra.run.smithi132.stdout:[mon.a] 2023-05-20T01:22:13.508 INFO:teuthology.orchestra.run.smithi132.stdout:[mon.b] 2023-05-20T01:22:13.508 INFO:teuthology.orchestra.run.smithi132.stdout:[mon.c] 2023-05-20T01:22:13.508 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.a] 2023-05-20T01:22:13.508 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.c] 2023-05-20T01:22:13.509 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.b] 2023-05-20T01:22:13.509 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.d] 2023-05-20T01:22:13.509 INFO:teuthology.orchestra.run.smithi132.stdout:[mds] 2023-05-20T01:22:13.509 INFO:teuthology.orchestra.run.smithi132.stdout: debug mds = 20 2023-05-20T01:22:13.509 INFO:teuthology.orchestra.run.smithi132.stdout: debug mds balancer = 20 2023-05-20T01:22:13.509 INFO:teuthology.orchestra.run.smithi132.stdout: debug ms = 1 2023-05-20T01:22:13.509 INFO:teuthology.orchestra.run.smithi132.stdout: mds debug frag = True 2023-05-20T01:22:13.509 INFO:teuthology.orchestra.run.smithi132.stdout: mds debug scatterstat = False 2023-05-20T01:22:13.510 INFO:teuthology.orchestra.run.smithi132.stdout: mds op complaint time = 180 2023-05-20T01:22:13.510 INFO:teuthology.orchestra.run.smithi132.stdout: mds verify scatter = False 2023-05-20T01:22:13.510 INFO:teuthology.orchestra.run.smithi132.stdout: osd op complaint time = 180 2023-05-20T01:22:13.510 INFO:teuthology.orchestra.run.smithi132.stdout: rados mon op timeout = 900 2023-05-20T01:22:13.510 INFO:teuthology.orchestra.run.smithi132.stdout: rados osd op timeout = 900 2023-05-20T01:22:13.511 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-05-20T01:22:13.530 DEBUG:teuthology.orchestra.run.smithi132:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-05-20T01:22:13.576 INFO:teuthology.orchestra.run.smithi016.stdout:[global] 2023-05-20T01:22:13.576 INFO:teuthology.orchestra.run.smithi016.stdout: chdir = "" 2023-05-20T01:22:13.577 INFO:teuthology.orchestra.run.smithi016.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-05-20T01:22:13.577 INFO:teuthology.orchestra.run.smithi016.stdout: auth supported = cephx 2023-05-20T01:22:13.577 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:22:13.577 INFO:teuthology.orchestra.run.smithi016.stdout: filestore xattr use omap = true 2023-05-20T01:22:13.577 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:22:13.577 INFO:teuthology.orchestra.run.smithi016.stdout: mon clock drift allowed = 1.000 2023-05-20T01:22:13.577 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:22:13.578 INFO:teuthology.orchestra.run.smithi016.stdout: osd crush chooseleaf type = 0 2023-05-20T01:22:13.578 INFO:teuthology.orchestra.run.smithi016.stdout: auth debug = true 2023-05-20T01:22:13.578 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:22:13.578 INFO:teuthology.orchestra.run.smithi016.stdout: ms die on old message = true 2023-05-20T01:22:13.578 INFO:teuthology.orchestra.run.smithi016.stdout: ms die on bug = true 2023-05-20T01:22:13.578 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:22:13.578 INFO:teuthology.orchestra.run.smithi016.stdout: mon max pg per osd = 10000 # >= luminous 2023-05-20T01:22:13.578 INFO:teuthology.orchestra.run.smithi016.stdout: mon pg warn max object skew = 0 2023-05-20T01:22:13.579 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:22:13.579 INFO:teuthology.orchestra.run.smithi016.stdout: # disable pg_autoscaler by default for new pools 2023-05-20T01:22:13.579 INFO:teuthology.orchestra.run.smithi016.stdout: osd_pool_default_pg_autoscale_mode = off 2023-05-20T01:22:13.579 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:22:13.579 INFO:teuthology.orchestra.run.smithi016.stdout: osd pool default size = 2 2023-05-20T01:22:13.579 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:22:13.579 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd allow primary affinity = true 2023-05-20T01:22:13.579 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd allow pg remap = true 2023-05-20T01:22:13.580 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on legacy crush tunables = false 2023-05-20T01:22:13.580 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on crush straw calc version zero = false 2023-05-20T01:22:13.580 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on no sortbitwise = false 2023-05-20T01:22:13.580 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on osd down out interval zero = false 2023-05-20T01:22:13.580 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on too few osds = false 2023-05-20T01:22:13.580 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-05-20T01:22:13.580 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_pool_no_redundancy = false 2023-05-20T01:22:13.581 INFO:teuthology.orchestra.run.smithi016.stdout: mon_allow_pool_size_one = true 2023-05-20T01:22:13.581 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:22:13.581 INFO:teuthology.orchestra.run.smithi016.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-05-20T01:22:13.581 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:22:13.581 INFO:teuthology.orchestra.run.smithi016.stdout: osd default data pool replay window = 5 2023-05-20T01:22:13.581 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:22:13.581 INFO:teuthology.orchestra.run.smithi016.stdout: mon allow pool delete = true 2023-05-20T01:22:13.581 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:22:13.582 INFO:teuthology.orchestra.run.smithi016.stdout: mon cluster log file level = debug 2023-05-20T01:22:13.582 INFO:teuthology.orchestra.run.smithi016.stdout: debug asserts on shutdown = true 2023-05-20T01:22:13.582 INFO:teuthology.orchestra.run.smithi016.stdout: mon health detail to clog = false 2023-05-20T01:22:13.582 INFO:teuthology.orchestra.run.smithi016.stdout: mon host = "172.21.15.16,172.21.15.132,[v2:172.21.15.132:3301,v1:172.21.15.132:6790]" 2023-05-20T01:22:13.582 INFO:teuthology.orchestra.run.smithi016.stdout: ms die on skipped message = False 2023-05-20T01:22:13.582 INFO:teuthology.orchestra.run.smithi016.stdout: fsid = c2245cc5-ecab-4087-a59b-bc81db901676 2023-05-20T01:22:13.582 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:22:13.582 INFO:teuthology.orchestra.run.smithi016.stdout:[osd] 2023-05-20T01:22:13.583 INFO:teuthology.orchestra.run.smithi016.stdout: osd journal size = 100 2023-05-20T01:22:13.583 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:22:13.583 INFO:teuthology.orchestra.run.smithi016.stdout: osd scrub load threshold = 5.0 2023-05-20T01:22:13.583 INFO:teuthology.orchestra.run.smithi016.stdout: osd scrub max interval = 600 2023-05-20T01:22:13.583 INFO:teuthology.orchestra.run.smithi016.stdout: osd mclock profile = high_recovery_ops 2023-05-20T01:22:13.583 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:22:13.583 INFO:teuthology.orchestra.run.smithi016.stdout: osd recover clone overlap = true 2023-05-20T01:22:13.583 INFO:teuthology.orchestra.run.smithi016.stdout: osd recovery max chunk = 1048576 2023-05-20T01:22:13.584 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:22:13.584 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug shutdown = true 2023-05-20T01:22:13.584 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug op order = true 2023-05-20T01:22:13.584 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug verify stray on activate = true 2023-05-20T01:22:13.584 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:22:13.584 INFO:teuthology.orchestra.run.smithi016.stdout: osd open classes on start = true 2023-05-20T01:22:13.584 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug pg log writeout = true 2023-05-20T01:22:13.584 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:22:13.585 INFO:teuthology.orchestra.run.smithi016.stdout: osd deep scrub update digest min age = 30 2023-05-20T01:22:13.585 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:22:13.585 INFO:teuthology.orchestra.run.smithi016.stdout: osd map max advance = 10 2023-05-20T01:22:13.585 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:22:13.585 INFO:teuthology.orchestra.run.smithi016.stdout: journal zero on create = true 2023-05-20T01:22:13.585 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:22:13.585 INFO:teuthology.orchestra.run.smithi016.stdout: filestore ondisk finisher threads = 3 2023-05-20T01:22:13.585 INFO:teuthology.orchestra.run.smithi016.stdout: filestore apply finisher threads = 3 2023-05-20T01:22:13.586 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:22:13.586 INFO:teuthology.orchestra.run.smithi016.stdout: bdev debug aio = true 2023-05-20T01:22:13.586 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug misdirected ops = true 2023-05-20T01:22:13.586 INFO:teuthology.orchestra.run.smithi016.stdout: bluestore block size = 96636764160 2023-05-20T01:22:13.586 INFO:teuthology.orchestra.run.smithi016.stdout: bluestore fsck on mount = True 2023-05-20T01:22:13.586 INFO:teuthology.orchestra.run.smithi016.stdout: debug bluefs = 20 2023-05-20T01:22:13.586 INFO:teuthology.orchestra.run.smithi016.stdout: debug bluestore = 20 2023-05-20T01:22:13.587 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2023-05-20T01:22:13.587 INFO:teuthology.orchestra.run.smithi016.stdout: debug osd = 20 2023-05-20T01:22:13.587 INFO:teuthology.orchestra.run.smithi016.stdout: debug rocksdb = 10 2023-05-20T01:22:13.587 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd backfillfull_ratio = 0.85 2023-05-20T01:22:13.587 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd full ratio = 0.9 2023-05-20T01:22:13.587 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd nearfull ratio = 0.8 2023-05-20T01:22:13.587 INFO:teuthology.orchestra.run.smithi016.stdout: osd failsafe full ratio = 0.95 2023-05-20T01:22:13.587 INFO:teuthology.orchestra.run.smithi016.stdout: osd objectstore = bluestore 2023-05-20T01:22:13.588 INFO:teuthology.orchestra.run.smithi016.stdout: osd op complaint time = 180 2023-05-20T01:22:13.588 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:22:13.588 INFO:teuthology.orchestra.run.smithi016.stdout:[mgr] 2023-05-20T01:22:13.588 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2023-05-20T01:22:13.588 INFO:teuthology.orchestra.run.smithi016.stdout: debug mgr = 20 2023-05-20T01:22:13.588 INFO:teuthology.orchestra.run.smithi016.stdout: debug mon = 20 2023-05-20T01:22:13.588 INFO:teuthology.orchestra.run.smithi016.stdout: debug auth = 20 2023-05-20T01:22:13.588 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min pgs per osd = 4 2023-05-20T01:22:13.589 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min bytes per osd = 10 2023-05-20T01:22:13.589 INFO:teuthology.orchestra.run.smithi016.stdout: mgr/telemetry/nag = false 2023-05-20T01:22:13.589 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:22:13.589 INFO:teuthology.orchestra.run.smithi016.stdout:[mon] 2023-05-20T01:22:13.589 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2023-05-20T01:22:13.589 INFO:teuthology.orchestra.run.smithi016.stdout: debug mon = 20 2023-05-20T01:22:13.589 INFO:teuthology.orchestra.run.smithi016.stdout: debug paxos = 20 2023-05-20T01:22:13.589 INFO:teuthology.orchestra.run.smithi016.stdout: debug auth = 20 2023-05-20T01:22:13.590 INFO:teuthology.orchestra.run.smithi016.stdout: mon data avail warn = 5 2023-05-20T01:22:13.590 INFO:teuthology.orchestra.run.smithi016.stdout: mon mgr mkfs grace = 240 2023-05-20T01:22:13.590 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min pgs per osd = 4 2023-05-20T01:22:13.590 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd reporter subtree level = osd 2023-05-20T01:22:13.590 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd prime pg temp = true 2023-05-20T01:22:13.590 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min bytes per osd = 10 2023-05-20T01:22:13.590 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:22:13.590 INFO:teuthology.orchestra.run.smithi016.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-05-20T01:22:13.591 INFO:teuthology.orchestra.run.smithi016.stdout: auth mon ticket ttl = 660 # 11m 2023-05-20T01:22:13.591 INFO:teuthology.orchestra.run.smithi016.stdout: auth service ticket ttl = 240 # 4m 2023-05-20T01:22:13.591 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:22:13.591 INFO:teuthology.orchestra.run.smithi016.stdout: # don't complain about insecure global_id in the test suite 2023-05-20T01:22:13.591 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-05-20T01:22:13.591 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-05-20T01:22:13.591 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:22:13.592 INFO:teuthology.orchestra.run.smithi016.stdout: # 1m isn't quite enough 2023-05-20T01:22:13.592 INFO:teuthology.orchestra.run.smithi016.stdout: mon_down_mkfs_grace = 2m 2023-05-20T01:22:13.592 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:22:13.592 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_filestore_osds = false 2023-05-20T01:22:13.592 INFO:teuthology.orchestra.run.smithi016.stdout: mon op complaint time = 120 2023-05-20T01:22:13.592 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:22:13.592 INFO:teuthology.orchestra.run.smithi016.stdout:[client] 2023-05-20T01:22:13.592 INFO:teuthology.orchestra.run.smithi016.stdout: rgw cache enabled = true 2023-05-20T01:22:13.593 INFO:teuthology.orchestra.run.smithi016.stdout: rgw enable ops log = true 2023-05-20T01:22:13.593 INFO:teuthology.orchestra.run.smithi016.stdout: rgw enable usage log = true 2023-05-20T01:22:13.593 INFO:teuthology.orchestra.run.smithi016.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-05-20T01:22:13.593 INFO:teuthology.orchestra.run.smithi016.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-05-20T01:22:13.593 INFO:teuthology.orchestra.run.smithi016.stdout: client mount timeout = 600 2023-05-20T01:22:13.593 INFO:teuthology.orchestra.run.smithi016.stdout: debug client = 20 2023-05-20T01:22:13.593 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2023-05-20T01:22:13.593 INFO:teuthology.orchestra.run.smithi016.stdout: pid file = "" 2023-05-20T01:22:13.594 INFO:teuthology.orchestra.run.smithi016.stdout: rados mon op timeout = 900 2023-05-20T01:22:13.594 INFO:teuthology.orchestra.run.smithi016.stdout: rados osd op timeout = 900 2023-05-20T01:22:13.594 INFO:teuthology.orchestra.run.smithi016.stdout:[mon.a] 2023-05-20T01:22:13.594 INFO:teuthology.orchestra.run.smithi016.stdout:[mon.b] 2023-05-20T01:22:13.594 INFO:teuthology.orchestra.run.smithi016.stdout:[mon.c] 2023-05-20T01:22:13.594 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.a] 2023-05-20T01:22:13.594 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.c] 2023-05-20T01:22:13.594 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.b] 2023-05-20T01:22:13.595 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.d] 2023-05-20T01:22:13.595 INFO:teuthology.orchestra.run.smithi016.stdout:[mds] 2023-05-20T01:22:13.595 INFO:teuthology.orchestra.run.smithi016.stdout: debug mds = 20 2023-05-20T01:22:13.595 INFO:teuthology.orchestra.run.smithi016.stdout: debug mds balancer = 20 2023-05-20T01:22:13.595 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2023-05-20T01:22:13.595 INFO:teuthology.orchestra.run.smithi016.stdout: mds debug frag = True 2023-05-20T01:22:13.595 INFO:teuthology.orchestra.run.smithi016.stdout: mds debug scatterstat = False 2023-05-20T01:22:13.595 INFO:teuthology.orchestra.run.smithi016.stdout: mds op complaint time = 180 2023-05-20T01:22:13.596 INFO:teuthology.orchestra.run.smithi016.stdout: mds verify scatter = False 2023-05-20T01:22:13.596 INFO:teuthology.orchestra.run.smithi016.stdout: osd op complaint time = 180 2023-05-20T01:22:13.596 INFO:teuthology.orchestra.run.smithi016.stdout: rados mon op timeout = 900 2023-05-20T01:22:13.596 INFO:teuthology.orchestra.run.smithi016.stdout: rados osd op timeout = 900 2023-05-20T01:22:13.597 INFO:teuthology.orchestra.run.smithi132.stdout:[global] 2023-05-20T01:22:13.597 INFO:teuthology.orchestra.run.smithi132.stdout: chdir = "" 2023-05-20T01:22:13.598 INFO:teuthology.orchestra.run.smithi132.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-05-20T01:22:13.598 INFO:teuthology.orchestra.run.smithi132.stdout: auth supported = cephx 2023-05-20T01:22:13.598 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:22:13.598 INFO:teuthology.orchestra.run.smithi132.stdout: filestore xattr use omap = true 2023-05-20T01:22:13.598 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:22:13.598 INFO:teuthology.orchestra.run.smithi132.stdout: mon clock drift allowed = 1.000 2023-05-20T01:22:13.598 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:22:13.599 INFO:teuthology.orchestra.run.smithi132.stdout: osd crush chooseleaf type = 0 2023-05-20T01:22:13.599 INFO:teuthology.orchestra.run.smithi132.stdout: auth debug = true 2023-05-20T01:22:13.599 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:22:13.599 INFO:teuthology.orchestra.run.smithi132.stdout: ms die on old message = true 2023-05-20T01:22:13.599 INFO:teuthology.orchestra.run.smithi132.stdout: ms die on bug = true 2023-05-20T01:22:13.599 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:22:13.599 INFO:teuthology.orchestra.run.smithi132.stdout: mon max pg per osd = 10000 # >= luminous 2023-05-20T01:22:13.599 INFO:teuthology.orchestra.run.smithi132.stdout: mon pg warn max object skew = 0 2023-05-20T01:22:13.600 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:22:13.600 INFO:teuthology.orchestra.run.smithi132.stdout: # disable pg_autoscaler by default for new pools 2023-05-20T01:22:13.600 INFO:teuthology.orchestra.run.smithi132.stdout: osd_pool_default_pg_autoscale_mode = off 2023-05-20T01:22:13.600 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:22:13.600 INFO:teuthology.orchestra.run.smithi132.stdout: osd pool default size = 2 2023-05-20T01:22:13.600 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:22:13.600 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd allow primary affinity = true 2023-05-20T01:22:13.600 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd allow pg remap = true 2023-05-20T01:22:13.601 INFO:teuthology.orchestra.run.smithi132.stdout: mon warn on legacy crush tunables = false 2023-05-20T01:22:13.601 INFO:teuthology.orchestra.run.smithi132.stdout: mon warn on crush straw calc version zero = false 2023-05-20T01:22:13.601 INFO:teuthology.orchestra.run.smithi132.stdout: mon warn on no sortbitwise = false 2023-05-20T01:22:13.601 INFO:teuthology.orchestra.run.smithi132.stdout: mon warn on osd down out interval zero = false 2023-05-20T01:22:13.601 INFO:teuthology.orchestra.run.smithi132.stdout: mon warn on too few osds = false 2023-05-20T01:22:13.601 INFO:teuthology.orchestra.run.smithi132.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-05-20T01:22:13.601 INFO:teuthology.orchestra.run.smithi132.stdout: mon_warn_on_pool_no_redundancy = false 2023-05-20T01:22:13.602 INFO:teuthology.orchestra.run.smithi132.stdout: mon_allow_pool_size_one = true 2023-05-20T01:22:13.602 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:22:13.602 INFO:teuthology.orchestra.run.smithi132.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-05-20T01:22:13.602 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:22:13.602 INFO:teuthology.orchestra.run.smithi132.stdout: osd default data pool replay window = 5 2023-05-20T01:22:13.602 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:22:13.602 INFO:teuthology.orchestra.run.smithi132.stdout: mon allow pool delete = true 2023-05-20T01:22:13.602 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:22:13.603 INFO:teuthology.orchestra.run.smithi132.stdout: mon cluster log file level = debug 2023-05-20T01:22:13.603 INFO:teuthology.orchestra.run.smithi132.stdout: debug asserts on shutdown = true 2023-05-20T01:22:13.603 INFO:teuthology.orchestra.run.smithi132.stdout: mon health detail to clog = false 2023-05-20T01:22:13.603 INFO:teuthology.orchestra.run.smithi132.stdout: mon host = "172.21.15.16,172.21.15.132,[v2:172.21.15.132:3301,v1:172.21.15.132:6790]" 2023-05-20T01:22:13.603 INFO:teuthology.orchestra.run.smithi132.stdout: ms die on skipped message = False 2023-05-20T01:22:13.603 INFO:teuthology.orchestra.run.smithi132.stdout: fsid = c2245cc5-ecab-4087-a59b-bc81db901676 2023-05-20T01:22:13.603 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:22:13.603 INFO:teuthology.orchestra.run.smithi132.stdout:[osd] 2023-05-20T01:22:13.604 INFO:teuthology.orchestra.run.smithi132.stdout: osd journal size = 100 2023-05-20T01:22:13.604 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:22:13.604 INFO:teuthology.orchestra.run.smithi132.stdout: osd scrub load threshold = 5.0 2023-05-20T01:22:13.604 INFO:teuthology.orchestra.run.smithi132.stdout: osd scrub max interval = 600 2023-05-20T01:22:13.604 INFO:teuthology.orchestra.run.smithi132.stdout: osd mclock profile = high_recovery_ops 2023-05-20T01:22:13.604 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:22:13.604 INFO:teuthology.orchestra.run.smithi132.stdout: osd recover clone overlap = true 2023-05-20T01:22:13.604 INFO:teuthology.orchestra.run.smithi132.stdout: osd recovery max chunk = 1048576 2023-05-20T01:22:13.605 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:22:13.605 INFO:teuthology.orchestra.run.smithi132.stdout: osd debug shutdown = true 2023-05-20T01:22:13.605 INFO:teuthology.orchestra.run.smithi132.stdout: osd debug op order = true 2023-05-20T01:22:13.605 INFO:teuthology.orchestra.run.smithi132.stdout: osd debug verify stray on activate = true 2023-05-20T01:22:13.605 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:22:13.605 INFO:teuthology.orchestra.run.smithi132.stdout: osd open classes on start = true 2023-05-20T01:22:13.605 INFO:teuthology.orchestra.run.smithi132.stdout: osd debug pg log writeout = true 2023-05-20T01:22:13.605 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:22:13.606 INFO:teuthology.orchestra.run.smithi132.stdout: osd deep scrub update digest min age = 30 2023-05-20T01:22:13.606 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:22:13.606 INFO:teuthology.orchestra.run.smithi132.stdout: osd map max advance = 10 2023-05-20T01:22:13.606 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:22:13.606 INFO:teuthology.orchestra.run.smithi132.stdout: journal zero on create = true 2023-05-20T01:22:13.606 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:22:13.606 INFO:teuthology.orchestra.run.smithi132.stdout: filestore ondisk finisher threads = 3 2023-05-20T01:22:13.606 INFO:teuthology.orchestra.run.smithi132.stdout: filestore apply finisher threads = 3 2023-05-20T01:22:13.607 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:22:13.607 INFO:teuthology.orchestra.run.smithi132.stdout: bdev debug aio = true 2023-05-20T01:22:13.607 INFO:teuthology.orchestra.run.smithi132.stdout: osd debug misdirected ops = true 2023-05-20T01:22:13.607 INFO:teuthology.orchestra.run.smithi132.stdout: bluestore block size = 96636764160 2023-05-20T01:22:13.607 INFO:teuthology.orchestra.run.smithi132.stdout: bluestore fsck on mount = True 2023-05-20T01:22:13.607 INFO:teuthology.orchestra.run.smithi132.stdout: debug bluefs = 20 2023-05-20T01:22:13.607 INFO:teuthology.orchestra.run.smithi132.stdout: debug bluestore = 20 2023-05-20T01:22:13.607 INFO:teuthology.orchestra.run.smithi132.stdout: debug ms = 1 2023-05-20T01:22:13.608 INFO:teuthology.orchestra.run.smithi132.stdout: debug osd = 20 2023-05-20T01:22:13.608 INFO:teuthology.orchestra.run.smithi132.stdout: debug rocksdb = 10 2023-05-20T01:22:13.608 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd backfillfull_ratio = 0.85 2023-05-20T01:22:13.608 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd full ratio = 0.9 2023-05-20T01:22:13.608 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd nearfull ratio = 0.8 2023-05-20T01:22:13.608 INFO:teuthology.orchestra.run.smithi132.stdout: osd failsafe full ratio = 0.95 2023-05-20T01:22:13.608 INFO:teuthology.orchestra.run.smithi132.stdout: osd objectstore = bluestore 2023-05-20T01:22:13.608 INFO:teuthology.orchestra.run.smithi132.stdout: osd op complaint time = 180 2023-05-20T01:22:13.609 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:22:13.609 INFO:teuthology.orchestra.run.smithi132.stdout:[mgr] 2023-05-20T01:22:13.609 INFO:teuthology.orchestra.run.smithi132.stdout: debug ms = 1 2023-05-20T01:22:13.609 INFO:teuthology.orchestra.run.smithi132.stdout: debug mgr = 20 2023-05-20T01:22:13.609 INFO:teuthology.orchestra.run.smithi132.stdout: debug mon = 20 2023-05-20T01:22:13.609 INFO:teuthology.orchestra.run.smithi132.stdout: debug auth = 20 2023-05-20T01:22:13.609 INFO:teuthology.orchestra.run.smithi132.stdout: mon reweight min pgs per osd = 4 2023-05-20T01:22:13.609 INFO:teuthology.orchestra.run.smithi132.stdout: mon reweight min bytes per osd = 10 2023-05-20T01:22:13.610 INFO:teuthology.orchestra.run.smithi132.stdout: mgr/telemetry/nag = false 2023-05-20T01:22:13.610 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:22:13.610 INFO:teuthology.orchestra.run.smithi132.stdout:[mon] 2023-05-20T01:22:13.610 INFO:teuthology.orchestra.run.smithi132.stdout: debug ms = 1 2023-05-20T01:22:13.610 INFO:teuthology.orchestra.run.smithi132.stdout: debug mon = 20 2023-05-20T01:22:13.610 INFO:teuthology.orchestra.run.smithi132.stdout: debug paxos = 20 2023-05-20T01:22:13.610 INFO:teuthology.orchestra.run.smithi132.stdout: debug auth = 20 2023-05-20T01:22:13.610 INFO:teuthology.orchestra.run.smithi132.stdout: mon data avail warn = 5 2023-05-20T01:22:13.611 INFO:teuthology.orchestra.run.smithi132.stdout: mon mgr mkfs grace = 240 2023-05-20T01:22:13.611 INFO:teuthology.orchestra.run.smithi132.stdout: mon reweight min pgs per osd = 4 2023-05-20T01:22:13.611 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd reporter subtree level = osd 2023-05-20T01:22:13.611 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd prime pg temp = true 2023-05-20T01:22:13.611 INFO:teuthology.orchestra.run.smithi132.stdout: mon reweight min bytes per osd = 10 2023-05-20T01:22:13.611 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:22:13.611 INFO:teuthology.orchestra.run.smithi132.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-05-20T01:22:13.612 INFO:teuthology.orchestra.run.smithi132.stdout: auth mon ticket ttl = 660 # 11m 2023-05-20T01:22:13.612 INFO:teuthology.orchestra.run.smithi132.stdout: auth service ticket ttl = 240 # 4m 2023-05-20T01:22:13.612 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:22:13.612 INFO:teuthology.orchestra.run.smithi132.stdout: # don't complain about insecure global_id in the test suite 2023-05-20T01:22:13.612 INFO:teuthology.orchestra.run.smithi132.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-05-20T01:22:13.612 INFO:teuthology.orchestra.run.smithi132.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-05-20T01:22:13.612 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:22:13.612 INFO:teuthology.orchestra.run.smithi132.stdout: # 1m isn't quite enough 2023-05-20T01:22:13.613 INFO:teuthology.orchestra.run.smithi132.stdout: mon_down_mkfs_grace = 2m 2023-05-20T01:22:13.613 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:22:13.613 INFO:teuthology.orchestra.run.smithi132.stdout: mon_warn_on_filestore_osds = false 2023-05-20T01:22:13.613 INFO:teuthology.orchestra.run.smithi132.stdout: mon op complaint time = 120 2023-05-20T01:22:13.613 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:22:13.613 INFO:teuthology.orchestra.run.smithi132.stdout:[client] 2023-05-20T01:22:13.613 INFO:teuthology.orchestra.run.smithi132.stdout: rgw cache enabled = true 2023-05-20T01:22:13.613 INFO:teuthology.orchestra.run.smithi132.stdout: rgw enable ops log = true 2023-05-20T01:22:13.614 INFO:teuthology.orchestra.run.smithi132.stdout: rgw enable usage log = true 2023-05-20T01:22:13.614 INFO:teuthology.orchestra.run.smithi132.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-05-20T01:22:13.614 INFO:teuthology.orchestra.run.smithi132.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-05-20T01:22:13.614 INFO:teuthology.orchestra.run.smithi132.stdout: client mount timeout = 600 2023-05-20T01:22:13.614 INFO:teuthology.orchestra.run.smithi132.stdout: debug client = 20 2023-05-20T01:22:13.614 INFO:teuthology.orchestra.run.smithi132.stdout: debug ms = 1 2023-05-20T01:22:13.614 INFO:teuthology.orchestra.run.smithi132.stdout: pid file = "" 2023-05-20T01:22:13.614 INFO:teuthology.orchestra.run.smithi132.stdout: rados mon op timeout = 900 2023-05-20T01:22:13.615 INFO:teuthology.orchestra.run.smithi132.stdout: rados osd op timeout = 900 2023-05-20T01:22:13.615 INFO:teuthology.orchestra.run.smithi132.stdout:[mon.a] 2023-05-20T01:22:13.615 INFO:teuthology.orchestra.run.smithi132.stdout:[mon.b] 2023-05-20T01:22:13.615 INFO:teuthology.orchestra.run.smithi132.stdout:[mon.c] 2023-05-20T01:22:13.615 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.a] 2023-05-20T01:22:13.615 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.c] 2023-05-20T01:22:13.615 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.b] 2023-05-20T01:22:13.615 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.d] 2023-05-20T01:22:13.616 INFO:teuthology.orchestra.run.smithi132.stdout:[mds] 2023-05-20T01:22:13.616 INFO:teuthology.orchestra.run.smithi132.stdout: debug mds = 20 2023-05-20T01:22:13.616 INFO:teuthology.orchestra.run.smithi132.stdout: debug mds balancer = 20 2023-05-20T01:22:13.616 INFO:teuthology.orchestra.run.smithi132.stdout: debug ms = 1 2023-05-20T01:22:13.616 INFO:teuthology.orchestra.run.smithi132.stdout: mds debug frag = True 2023-05-20T01:22:13.616 INFO:teuthology.orchestra.run.smithi132.stdout: mds debug scatterstat = False 2023-05-20T01:22:13.616 INFO:teuthology.orchestra.run.smithi132.stdout: mds op complaint time = 180 2023-05-20T01:22:13.616 INFO:teuthology.orchestra.run.smithi132.stdout: mds verify scatter = False 2023-05-20T01:22:13.617 INFO:teuthology.orchestra.run.smithi132.stdout: osd op complaint time = 180 2023-05-20T01:22:13.617 INFO:teuthology.orchestra.run.smithi132.stdout: rados mon op timeout = 900 2023-05-20T01:22:13.617 INFO:teuthology.orchestra.run.smithi132.stdout: rados osd op timeout = 900 2023-05-20T01:22:13.617 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph purge cephfs_metadata --yes-i-really-really-mean-it 2023-05-20T01:22:13.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.656+0000 7fe67c467080 1 Processor -- start 2023-05-20T01:22:13.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.656+0000 7fe67c467080 1 -- start start 2023-05-20T01:22:13.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.656+0000 7fe67c467080 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56302a9c3210 0x56302aa4b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:13.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.656+0000 7fe67c467080 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56302aa4e960 0x56302aa4b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:13.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.656+0000 7fe67c467080 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56302aa534d0 0x56302a978ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:13.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.656+0000 7fe67c467080 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x56302a8ddb20 con 0x56302aa534d0 2023-05-20T01:22:13.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.656+0000 7fe67c467080 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x56302a8b3fe0 con 0x56302a9c3210 2023-05-20T01:22:13.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.656+0000 7fe67c467080 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x56302aad8660 con 0x56302aa4e960 2023-05-20T01:22:13.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.657+0000 7fe672d68700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56302aa534d0 0x56302a978ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:13.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.657+0000 7fe672d68700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56302aa534d0 0x56302a978ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34296/0 (socket says 172.21.15.16:34296) 2023-05-20T01:22:13.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.657+0000 7fe672d68700 1 -- 172.21.15.16:0/3084484383 learned_addr learned my addr 172.21.15.16:0/3084484383 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:22:13.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.657+0000 7fe672567700 1 --2- 172.21.15.16:0/3084484383 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56302a9c3210 0x56302aa4b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:13.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.657+0000 7fe671d66700 1 --2- 172.21.15.16:0/3084484383 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56302aa4e960 0x56302aa4b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:13.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.657+0000 7fe672d68700 1 -- 172.21.15.16:0/3084484383 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56302aa4e960 msgr2=0x56302aa4b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:13.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.657+0000 7fe672d68700 1 --2- 172.21.15.16:0/3084484383 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56302aa4e960 0x56302aa4b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:13.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.657+0000 7fe672d68700 1 -- 172.21.15.16:0/3084484383 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56302a9c3210 msgr2=0x56302aa4b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:13.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.657+0000 7fe672d68700 1 --2- 172.21.15.16:0/3084484383 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56302a9c3210 0x56302aa4b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:13.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.657+0000 7fe672d68700 1 -- 172.21.15.16:0/3084484383 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe664005de0 con 0x56302aa534d0 2023-05-20T01:22:13.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.657+0000 7fe672d68700 1 --2- 172.21.15.16:0/3084484383 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56302aa534d0 0x56302a978ad0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fe66c00b580 tx=0x7fe66c00b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fc01e125cee87d8 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:13.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.657+0000 7fe670d64700 1 -- 172.21.15.16:0/3084484383 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe66c014070 con 0x56302aa534d0 2023-05-20T01:22:13.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.657+0000 7fe670d64700 1 -- 172.21.15.16:0/3084484383 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe66c00f560 con 0x56302aa534d0 2023-05-20T01:22:13.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.657+0000 7fe670d64700 1 -- 172.21.15.16:0/3084484383 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe66c00f910 con 0x56302aa534d0 2023-05-20T01:22:13.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.657+0000 7fe67c467080 1 -- 172.21.15.16:0/3084484383 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56302aa534d0 msgr2=0x56302a978ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:13.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.657+0000 7fe67c467080 1 --2- 172.21.15.16:0/3084484383 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56302aa534d0 0x56302a978ad0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fe66c00b580 tx=0x7fe66c00b890 comp rx=0 tx=0).stop 2023-05-20T01:22:13.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.657+0000 7fe67c467080 1 -- 172.21.15.16:0/3084484383 shutdown_connections 2023-05-20T01:22:13.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.657+0000 7fe67c467080 1 --2- 172.21.15.16:0/3084484383 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56302aa534d0 0x56302a978ad0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:13.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.657+0000 7fe67c467080 1 --2- 172.21.15.16:0/3084484383 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56302a9c3210 0x56302aa4b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:13.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.657+0000 7fe67c467080 1 --2- 172.21.15.16:0/3084484383 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56302aa4e960 0x56302aa4b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:13.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.657+0000 7fe67c467080 1 -- 172.21.15.16:0/3084484383 >> 172.21.15.16:0/3084484383 conn(0x56302a9a7350 msgr2=0x56302a9c5e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:13.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.658+0000 7fe67c467080 1 -- 172.21.15.16:0/3084484383 shutdown_connections 2023-05-20T01:22:13.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.658+0000 7fe67c467080 1 -- 172.21.15.16:0/3084484383 wait complete. 2023-05-20T01:22:13.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.658+0000 7fe67c467080 1 Processor -- start 2023-05-20T01:22:13.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.658+0000 7fe67c467080 1 -- start start 2023-05-20T01:22:13.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.659+0000 7fe67c467080 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56302a9c3210 0x56302a9a6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:13.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.659+0000 7fe67c467080 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56302aa4e960 0x56302a9a2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:13.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.659+0000 7fe67c467080 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56302aa534d0 0x56302aa43e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:13.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.659+0000 7fe67c467080 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x56302a9a2ab0 con 0x56302a9c3210 2023-05-20T01:22:13.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.659+0000 7fe67c467080 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x56302a9a2840 con 0x56302aa534d0 2023-05-20T01:22:13.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.659+0000 7fe67c467080 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x56302a9a7130 con 0x56302aa4e960 2023-05-20T01:22:13.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.659+0000 7fe672567700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56302a9c3210 0x56302a9a6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:13.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.659+0000 7fe672567700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56302a9c3210 0x56302a9a6bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34298/0 (socket says 172.21.15.16:34298) 2023-05-20T01:22:13.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.660+0000 7fe672567700 1 -- 172.21.15.16:0/1001178467 learned_addr learned my addr 172.21.15.16:0/1001178467 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:22:13.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.660+0000 7fe671d66700 1 --2- 172.21.15.16:0/1001178467 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56302aa4e960 0x56302a9a2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:13.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.660+0000 7fe672d68700 1 --2- 172.21.15.16:0/1001178467 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56302aa534d0 0x56302aa43e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:13.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.660+0000 7fe672567700 1 -- 172.21.15.16:0/1001178467 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56302aa4e960 msgr2=0x56302a9a2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:13.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.660+0000 7fe672567700 1 --2- 172.21.15.16:0/1001178467 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56302aa4e960 0x56302a9a2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:13.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.660+0000 7fe672567700 1 -- 172.21.15.16:0/1001178467 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56302aa534d0 msgr2=0x56302aa43e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:13.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.660+0000 7fe672567700 1 --2- 172.21.15.16:0/1001178467 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56302aa534d0 0x56302aa43e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:13.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.660+0000 7fe672567700 1 -- 172.21.15.16:0/1001178467 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe66c00b1b0 con 0x56302a9c3210 2023-05-20T01:22:13.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.660+0000 7fe672d68700 1 --2- 172.21.15.16:0/1001178467 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56302aa534d0 0x56302aa43e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:22:13.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.660+0000 7fe671d66700 1 --2- 172.21.15.16:0/1001178467 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56302aa4e960 0x56302a9a2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:22:13.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.660+0000 7fe672567700 1 --2- 172.21.15.16:0/1001178467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56302a9c3210 0x56302a9a6bf0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fe664015040 tx=0x7fe664002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:13.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.660+0000 7fe6637fe700 1 -- 172.21.15.16:0/1001178467 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe66401d630 con 0x56302a9c3210 2023-05-20T01:22:13.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.660+0000 7fe6637fe700 1 -- 172.21.15.16:0/1001178467 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe66401d790 con 0x56302a9c3210 2023-05-20T01:22:13.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.660+0000 7fe67c467080 1 -- 172.21.15.16:0/1001178467 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56302a8e6240 con 0x56302a9c3210 2023-05-20T01:22:13.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.660+0000 7fe6637fe700 1 -- 172.21.15.16:0/1001178467 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe66401db40 con 0x56302a9c3210 2023-05-20T01:22:13.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.660+0000 7fe67c467080 1 -- 172.21.15.16:0/1001178467 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56302a8bf4a0 con 0x56302a9c3210 2023-05-20T01:22:13.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.661+0000 7fe6637fe700 1 -- 172.21.15.16:0/1001178467 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe66401dca0 con 0x56302a9c3210 2023-05-20T01:22:13.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.662+0000 7fe6637fe700 1 --2- 172.21.15.16:0/1001178467 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe654069630 0x7fe65406bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:13.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.662+0000 7fe6637fe700 1 -- 172.21.15.16:0/1001178467 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 7017+0+0 (secure 0 0 0) 0x7fe664089800 con 0x56302a9c3210 2023-05-20T01:22:13.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.662+0000 7fe671d66700 1 --2- 172.21.15.16:0/1001178467 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe654069630 0x7fe65406bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:13.674 INFO:teuthology.orchestra.run.smithi016.stdout:Warning: using slow linear search 2023-05-20T01:22:13.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.662+0000 7fe67c467080 1 --2- 172.21.15.16:0/1001178467 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x56302a9a3430 0x56302a9a10e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:13.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.662+0000 7fe67c467080 1 -- 172.21.15.16:0/1001178467 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:1 9.0 9:00000000:%01:::head [pgnls start_epoch 70 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e70) v8 -- 0x56302a9c2b00 con 0x56302a9a3430 2023-05-20T01:22:13.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.662+0000 7fe672d68700 1 --2- 172.21.15.16:0/1001178467 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x56302a9a3430 0x56302a9a10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:13.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.662+0000 7fe671d66700 1 --2- 172.21.15.16:0/1001178467 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe654069630 0x7fe65406bae0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x56302aadab60 tx=0x7fe668005310 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:13.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.663+0000 7fe672d68700 1 --2- 172.21.15.16:0/1001178467 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x56302a9a3430 0x56302a9a10e0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:13.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.663+0000 7fe672d68700 1 -- 172.21.15.16:0/1001178467 <== osd.1 v2:172.21.15.16:6808/3860264220 1 ==== osd_op_reply(1 [pgnls start_epoch 70 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe66c016070 con 0x56302a9a3430 2023-05-20T01:22:13.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.663+0000 7fe672d68700 1 --2- 172.21.15.16:0/1001178467 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x7fe66c01bab0 0x7fe66c0103f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:13.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.663+0000 7fe672d68700 1 -- 172.21.15.16:0/1001178467 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:2 9.10 9:08000000:%01:::head [pgnls start_epoch 70 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e70) v8 -- 0x56302a9c2b00 con 0x7fe66c01bab0 2023-05-20T01:22:13.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.663+0000 7fe672567700 1 --2- 172.21.15.16:0/1001178467 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x7fe66c01bab0 0x7fe66c0103f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:13.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.664+0000 7fe672567700 1 --2- 172.21.15.16:0/1001178467 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x7fe66c01bab0 0x7fe66c0103f0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:13.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.664+0000 7fe672567700 1 -- 172.21.15.16:0/1001178467 <== osd.4 v2:172.21.15.132:6817/506317223 1 ==== osd_op_reply(2 [pgnls start_epoch 70 out=177b] v69'10 uv10 ondisk = 1) v8 ==== 144+0+177 (crc 0 0 0) 0x7fe664019040 con 0x7fe66c01bab0 2023-05-20T01:22:13.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.664+0000 7fe672567700 1 --2- 172.21.15.16:0/1001178467 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x7fe664009040 0x7fe664058fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:13.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.664+0000 7fe672567700 1 -- 172.21.15.16:0/1001178467 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:3 9.8 9:10000000:%01:::head [pgnls start_epoch 70 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e70) v8 -- 0x56302a9c2b00 con 0x7fe664009040 2023-05-20T01:22:13.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.664+0000 7fe671d66700 1 --2- 172.21.15.16:0/1001178467 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x7fe664009040 0x7fe664058fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:13.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.664+0000 7fe671d66700 1 --2- 172.21.15.16:0/1001178467 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x7fe664009040 0x7fe664058fb0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:13.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.665+0000 7fe671d66700 1 -- 172.21.15.16:0/1001178467 <== osd.3 v2:172.21.15.16:6800/3669781890 1 ==== osd_op_reply(3 [pgnls start_epoch 70 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe6680074f0 con 0x7fe664009040 2023-05-20T01:22:13.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.665+0000 7fe671d66700 1 --2- 172.21.15.16:0/1001178467 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x7fe668009040 0x7fe668018940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:13.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.665+0000 7fe671d66700 1 -- 172.21.15.16:0/1001178467 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:4 9.18 9:18000000:%01:::head [pgnls start_epoch 70 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e70) v8 -- 0x56302a9c2b00 con 0x7fe668009040 2023-05-20T01:22:13.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.665+0000 7fe672d68700 1 --2- 172.21.15.16:0/1001178467 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x7fe668009040 0x7fe668018940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:13.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.665+0000 7fe672d68700 1 --2- 172.21.15.16:0/1001178467 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x7fe668009040 0x7fe668018940 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:13.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.666+0000 7fe672d68700 1 -- 172.21.15.16:0/1001178467 <== osd.6 v2:172.21.15.132:6801/727581123 1 ==== osd_op_reply(4 [pgnls start_epoch 70 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe66c016070 con 0x7fe668009040 2023-05-20T01:22:13.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.666+0000 7fe672d68700 1 -- 172.21.15.16:0/1001178467 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:5 9.4 9:20000000:%01:::head [pgnls start_epoch 70 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e70) v8 -- 0x7fe66c029040 con 0x56302a9a3430 2023-05-20T01:22:13.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.666+0000 7fe672d68700 1 -- 172.21.15.16:0/1001178467 <== osd.1 v2:172.21.15.16:6808/3860264220 2 ==== osd_op_reply(5 [pgnls start_epoch 70 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe66c016070 con 0x56302a9a3430 2023-05-20T01:22:13.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.666+0000 7fe672d68700 1 -- 172.21.15.16:0/1001178467 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:6 9.14 9:28000000:%01:::head [pgnls start_epoch 70 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e70) v8 -- 0x7fe66c029040 con 0x56302a9a3430 2023-05-20T01:22:13.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.666+0000 7fe672d68700 1 -- 172.21.15.16:0/1001178467 <== osd.1 v2:172.21.15.16:6808/3860264220 3 ==== osd_op_reply(6 [pgnls start_epoch 70 out=149b] v69'125 uv125 ondisk = 1) v8 ==== 144+0+149 (crc 0 0 0) 0x7fe66c016070 con 0x56302a9a3430 2023-05-20T01:22:13.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.666+0000 7fe672d68700 1 -- 172.21.15.16:0/1001178467 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:7 9.c 9:30000000:%01:::head [pgnls start_epoch 70 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e70) v8 -- 0x7fe66c029040 con 0x7fe664009040 2023-05-20T01:22:13.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.666+0000 7fe671d66700 1 -- 172.21.15.16:0/1001178467 <== osd.3 v2:172.21.15.16:6800/3669781890 2 ==== osd_op_reply(7 [pgnls start_epoch 70 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe668022040 con 0x7fe664009040 2023-05-20T01:22:13.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.667+0000 7fe671d66700 1 --2- 172.21.15.16:0/1001178467 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x7fe6680158e0 0x7fe668016380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:13.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.667+0000 7fe671d66700 1 -- 172.21.15.16:0/1001178467 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:8 9.1c 9:38000000:%01:::head [pgnls start_epoch 70 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e70) v8 -- 0x7fe66c029040 con 0x7fe6680158e0 2023-05-20T01:22:13.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.667+0000 7fe672567700 1 --2- 172.21.15.16:0/1001178467 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x7fe6680158e0 0x7fe668016380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:13.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.667+0000 7fe672567700 1 --2- 172.21.15.16:0/1001178467 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x7fe6680158e0 0x7fe668016380 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:13.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.667+0000 7fe672567700 1 -- 172.21.15.16:0/1001178467 <== osd.0 v2:172.21.15.16:6824/3923634179 1 ==== osd_op_reply(8 [pgnls start_epoch 70 out=73b] v63'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x7fe664019040 con 0x7fe6680158e0 2023-05-20T01:22:13.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.667+0000 7fe672567700 1 -- 172.21.15.16:0/1001178467 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:9 9.2 9:40000000:%01:::head [pgnls start_epoch 70 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e70) v8 -- 0x7fe66c029040 con 0x7fe664009040 2023-05-20T01:22:13.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.668+0000 7fe671d66700 1 -- 172.21.15.16:0/1001178467 <== osd.3 v2:172.21.15.16:6800/3669781890 3 ==== osd_op_reply(9 [pgnls start_epoch 70 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe668022040 con 0x7fe664009040 2023-05-20T01:22:13.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.668+0000 7fe671d66700 1 --2- 172.21.15.16:0/1001178467 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x7fe668006380 0x7fe6680067f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:13.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.668+0000 7fe671d66700 1 -- 172.21.15.16:0/1001178467 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:10 9.12 9:48000000:%01:::head [pgnls start_epoch 70 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e70) v8 -- 0x7fe66c029040 con 0x7fe668006380 2023-05-20T01:22:13.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.668+0000 7fe671d66700 1 --2- 172.21.15.16:0/1001178467 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x7fe668006380 0x7fe6680067f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:13.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.668+0000 7fe671d66700 1 --2- 172.21.15.16:0/1001178467 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x7fe668006380 0x7fe6680067f0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:13.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.668+0000 7fe671d66700 1 -- 172.21.15.16:0/1001178467 <== osd.5 v2:172.21.15.132:6816/1844689995 1 ==== osd_op_reply(10 [pgnls start_epoch 70 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe668022040 con 0x7fe668006380 2023-05-20T01:22:13.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.668+0000 7fe671d66700 1 -- 172.21.15.16:0/1001178467 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:11 9.a 9:50000000:%01:::head [pgnls start_epoch 70 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e70) v8 -- 0x7fe66c029040 con 0x7fe668006380 2023-05-20T01:22:13.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.669+0000 7fe671d66700 1 -- 172.21.15.16:0/1001178467 <== osd.5 v2:172.21.15.132:6816/1844689995 2 ==== osd_op_reply(11 [pgnls start_epoch 70 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe668022040 con 0x7fe668006380 2023-05-20T01:22:13.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.669+0000 7fe671d66700 1 -- 172.21.15.16:0/1001178467 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:12 9.1a 9:58000000:%01:::head [pgnls start_epoch 70 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e70) v8 -- 0x7fe66c029040 con 0x7fe664009040 2023-05-20T01:22:13.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.669+0000 7fe671d66700 1 -- 172.21.15.16:0/1001178467 <== osd.3 v2:172.21.15.16:6800/3669781890 4 ==== osd_op_reply(12 [pgnls start_epoch 70 out=49b] v63'1 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe668022040 con 0x7fe664009040 2023-05-20T01:22:13.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.669+0000 7fe671d66700 1 -- 172.21.15.16:0/1001178467 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:13 9.6 9:60000000:%01:::head [pgnls start_epoch 70 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e70) v8 -- 0x7fe66c029040 con 0x7fe668006380 2023-05-20T01:22:13.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.669+0000 7fe671d66700 1 -- 172.21.15.16:0/1001178467 <== osd.5 v2:172.21.15.132:6816/1844689995 3 ==== osd_op_reply(13 [pgnls start_epoch 70 out=77b] v69'27 uv27 ondisk = 1) v8 ==== 144+0+77 (crc 0 0 0) 0x7fe668022040 con 0x7fe668006380 2023-05-20T01:22:13.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.669+0000 7fe671d66700 1 -- 172.21.15.16:0/1001178467 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:14 9.16 9:68000000:%01:::head [pgnls start_epoch 70 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e70) v8 -- 0x7fe66c029040 con 0x7fe66c01bab0 2023-05-20T01:22:13.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.670+0000 7fe672567700 1 -- 172.21.15.16:0/1001178467 <== osd.4 v2:172.21.15.132:6817/506317223 2 ==== osd_op_reply(14 [pgnls start_epoch 70 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe664019040 con 0x7fe66c01bab0 2023-05-20T01:22:13.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.670+0000 7fe672567700 1 --2- 172.21.15.16:0/1001178467 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x7fe664057730 0x7fe6640871c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:13.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.670+0000 7fe672567700 1 -- 172.21.15.16:0/1001178467 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:15 9.e 9:70000000:%01:::head [pgnls start_epoch 70 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e70) v8 -- 0x7fe66c029040 con 0x7fe664057730 2023-05-20T01:22:13.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.670+0000 7fe672d68700 1 --2- 172.21.15.16:0/1001178467 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x7fe664057730 0x7fe6640871c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:13.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.670+0000 7fe672d68700 1 --2- 172.21.15.16:0/1001178467 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x7fe664057730 0x7fe6640871c0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:13.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.670+0000 7fe672d68700 1 -- 172.21.15.16:0/1001178467 <== osd.7 v2:172.21.15.132:6800/466001149 1 ==== osd_op_reply(15 [pgnls start_epoch 70 out=73b] v63'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x7fe66c016070 con 0x7fe664057730 2023-05-20T01:22:13.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.670+0000 7fe672d68700 1 -- 172.21.15.16:0/1001178467 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:16 9.1e 9:78000000:%01:::head [pgnls start_epoch 70 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e70) v8 -- 0x7fe66c029040 con 0x7fe6680158e0 2023-05-20T01:22:13.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.671+0000 7fe672567700 1 -- 172.21.15.16:0/1001178467 <== osd.0 v2:172.21.15.16:6824/3923634179 2 ==== osd_op_reply(16 [pgnls start_epoch 70 out=49b] v63'1 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe664019040 con 0x7fe6680158e0 2023-05-20T01:22:13.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.671+0000 7fe672567700 1 -- 172.21.15.16:0/1001178467 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:17 9.1 9:80000000:%01:::head [pgnls start_epoch 70 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e70) v8 -- 0x7fe66c029040 con 0x7fe664009040 2023-05-20T01:22:13.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.671+0000 7fe671d66700 1 -- 172.21.15.16:0/1001178467 <== osd.3 v2:172.21.15.16:6800/3669781890 5 ==== osd_op_reply(17 [pgnls start_epoch 70 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe668022040 con 0x7fe664009040 2023-05-20T01:22:13.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.671+0000 7fe671d66700 1 -- 172.21.15.16:0/1001178467 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:18 9.11 9:88000000:%01:::head [pgnls start_epoch 70 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e70) v8 -- 0x7fe66c029040 con 0x7fe664057730 2023-05-20T01:22:13.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.671+0000 7fe672d68700 1 -- 172.21.15.16:0/1001178467 <== osd.7 v2:172.21.15.132:6800/466001149 2 ==== osd_op_reply(18 [pgnls start_epoch 70 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe66c016070 con 0x7fe664057730 2023-05-20T01:22:13.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.671+0000 7fe672d68700 1 -- 172.21.15.16:0/1001178467 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:19 9.9 9:90000000:%01:::head [pgnls start_epoch 70 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e70) v8 -- 0x7fe66c029040 con 0x7fe6680158e0 2023-05-20T01:22:13.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.672+0000 7fe672567700 1 -- 172.21.15.16:0/1001178467 <== osd.0 v2:172.21.15.16:6824/3923634179 3 ==== osd_op_reply(19 [pgnls start_epoch 70 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe664019040 con 0x7fe6680158e0 2023-05-20T01:22:13.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.672+0000 7fe672567700 1 -- 172.21.15.16:0/1001178467 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:20 9.19 9:98000000:%01:::head [pgnls start_epoch 70 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e70) v8 -- 0x7fe66c029040 con 0x7fe664009040 2023-05-20T01:22:13.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.672+0000 7fe671d66700 1 -- 172.21.15.16:0/1001178467 <== osd.3 v2:172.21.15.16:6800/3669781890 6 ==== osd_op_reply(20 [pgnls start_epoch 70 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe668022040 con 0x7fe664009040 2023-05-20T01:22:13.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.672+0000 7fe671d66700 1 -- 172.21.15.16:0/1001178467 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:21 9.5 9:a0000000:%01:::head [pgnls start_epoch 70 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e70) v8 -- 0x7fe66c029040 con 0x7fe6680158e0 2023-05-20T01:22:13.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.672+0000 7fe672567700 1 -- 172.21.15.16:0/1001178467 <== osd.0 v2:172.21.15.16:6824/3923634179 4 ==== osd_op_reply(21 [pgnls start_epoch 70 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe664019040 con 0x7fe6680158e0 2023-05-20T01:22:13.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.672+0000 7fe672567700 1 -- 172.21.15.16:0/1001178467 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:22 9.15 9:a8000000:%01:::head [pgnls start_epoch 70 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e70) v8 -- 0x7fe66c029040 con 0x7fe664057730 2023-05-20T01:22:13.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.672+0000 7fe672d68700 1 -- 172.21.15.16:0/1001178467 <== osd.7 v2:172.21.15.132:6800/466001149 3 ==== osd_op_reply(22 [pgnls start_epoch 70 out=97b] v63'3 uv3 ondisk = 1) v8 ==== 144+0+97 (crc 0 0 0) 0x7fe66c016070 con 0x7fe664057730 2023-05-20T01:22:13.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.672+0000 7fe672d68700 1 --2- 172.21.15.16:0/1001178467 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x7fe66c037040 0x7fe66c037410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:13.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.672+0000 7fe672d68700 1 -- 172.21.15.16:0/1001178467 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:23 9.d 9:b0000000:%01:::head [pgnls start_epoch 70 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e70) v8 -- 0x7fe66c029040 con 0x7fe66c037040 2023-05-20T01:22:13.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.673+0000 7fe672567700 1 --2- 172.21.15.16:0/1001178467 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x7fe66c037040 0x7fe66c037410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:13.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.673+0000 7fe672567700 1 --2- 172.21.15.16:0/1001178467 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x7fe66c037040 0x7fe66c037410 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:13.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.673+0000 7fe672567700 1 -- 172.21.15.16:0/1001178467 <== osd.2 v2:172.21.15.16:6816/293187030 1 ==== osd_op_reply(23 [pgnls start_epoch 70 out=74b] v63'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x7fe664019040 con 0x7fe66c037040 2023-05-20T01:22:13.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.673+0000 7fe672567700 1 -- 172.21.15.16:0/1001178467 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:24 9.1d 9:b8000000:%01:::head [pgnls start_epoch 70 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e70) v8 -- 0x7fe66c029040 con 0x7fe6680158e0 2023-05-20T01:22:13.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.673+0000 7fe672567700 1 -- 172.21.15.16:0/1001178467 <== osd.0 v2:172.21.15.16:6824/3923634179 5 ==== osd_op_reply(24 [pgnls start_epoch 70 out=73b] v63'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x7fe664019040 con 0x7fe6680158e0 2023-05-20T01:22:13.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.673+0000 7fe672567700 1 -- 172.21.15.16:0/1001178467 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:25 9.3 9:c0000000:%01:::head [pgnls start_epoch 70 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e70) v8 -- 0x7fe66c029040 con 0x7fe664057730 2023-05-20T01:22:13.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.674+0000 7fe672d68700 1 -- 172.21.15.16:0/1001178467 <== osd.7 v2:172.21.15.132:6800/466001149 4 ==== osd_op_reply(25 [pgnls start_epoch 70 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe66c016070 con 0x7fe664057730 2023-05-20T01:22:13.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.674+0000 7fe672d68700 1 -- 172.21.15.16:0/1001178467 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:26 9.13 9:c8000000:%01:::head [pgnls start_epoch 70 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e70) v8 -- 0x7fe66c029040 con 0x7fe664057730 2023-05-20T01:22:13.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.674+0000 7fe672d68700 1 -- 172.21.15.16:0/1001178467 <== osd.7 v2:172.21.15.132:6800/466001149 5 ==== osd_op_reply(26 [pgnls start_epoch 70 out=98b] v69'3 uv3 ondisk = 1) v8 ==== 144+0+98 (crc 0 0 0) 0x7fe66c016070 con 0x7fe664057730 2023-05-20T01:22:13.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.674+0000 7fe672d68700 1 -- 172.21.15.16:0/1001178467 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:27 9.b 9:d0000000:%01:::head [pgnls start_epoch 70 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e70) v8 -- 0x7fe66c029040 con 0x7fe6680158e0 2023-05-20T01:22:13.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.674+0000 7fe672567700 1 -- 172.21.15.16:0/1001178467 <== osd.0 v2:172.21.15.16:6824/3923634179 6 ==== osd_op_reply(27 [pgnls start_epoch 70 out=76b] v69'3 uv3 ondisk = 1) v8 ==== 144+0+76 (crc 0 0 0) 0x7fe664019040 con 0x7fe6680158e0 2023-05-20T01:22:13.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.674+0000 7fe672567700 1 -- 172.21.15.16:0/1001178467 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:28 9.1b 9:d8000000:%01:::head [pgnls start_epoch 70 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e70) v8 -- 0x7fe66c029040 con 0x7fe664009040 2023-05-20T01:22:13.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.675+0000 7fe671d66700 1 -- 172.21.15.16:0/1001178467 <== osd.3 v2:172.21.15.16:6800/3669781890 7 ==== osd_op_reply(28 [pgnls start_epoch 70 out=49b] v63'1 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe66803d040 con 0x7fe664009040 2023-05-20T01:22:13.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.675+0000 7fe671d66700 1 -- 172.21.15.16:0/1001178467 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:29 9.7 9:e0000000:%01:::head [pgnls start_epoch 70 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e70) v8 -- 0x7fe66c029040 con 0x7fe668006380 2023-05-20T01:22:13.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.675+0000 7fe671d66700 1 -- 172.21.15.16:0/1001178467 <== osd.5 v2:172.21.15.132:6816/1844689995 4 ==== osd_op_reply(29 [pgnls start_epoch 70 out=73b] v63'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x7fe66803d040 con 0x7fe668006380 2023-05-20T01:22:13.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.675+0000 7fe671d66700 1 -- 172.21.15.16:0/1001178467 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:30 9.17 9:e8000000:%01:::head [pgnls start_epoch 70 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e70) v8 -- 0x7fe66c029040 con 0x7fe664057730 2023-05-20T01:22:13.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.675+0000 7fe672d68700 1 -- 172.21.15.16:0/1001178467 <== osd.7 v2:172.21.15.132:6800/466001149 6 ==== osd_op_reply(30 [pgnls start_epoch 70 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe66c016070 con 0x7fe664057730 2023-05-20T01:22:13.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.675+0000 7fe672d68700 1 -- 172.21.15.16:0/1001178467 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:31 9.f 9:f0000000:%01:::head [pgnls start_epoch 70 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e70) v8 -- 0x7fe66c029040 con 0x7fe668006380 2023-05-20T01:22:13.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.676+0000 7fe671d66700 1 -- 172.21.15.16:0/1001178467 <== osd.5 v2:172.21.15.132:6816/1844689995 5 ==== osd_op_reply(31 [pgnls start_epoch 70 out=73b] v63'1 uv1 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x7fe66803d040 con 0x7fe668006380 2023-05-20T01:22:13.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.676+0000 7fe671d66700 1 -- 172.21.15.16:0/1001178467 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:32 9.1f 9:f8000000:%01:::head [pgnls start_epoch 70 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e70) v8 -- 0x7fe66c029040 con 0x56302a9a3430 2023-05-20T01:22:13.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.676+0000 7fe672d68700 1 -- 172.21.15.16:0/1001178467 <== osd.1 v2:172.21.15.16:6808/3860264220 4 ==== osd_op_reply(32 [pgnls start_epoch 70 out=101b] v69'6 uv6 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x7fe66c016070 con 0x56302a9a3430 2023-05-20T01:22:13.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.676+0000 7fe67c467080 1 -- 172.21.15.16:0/1001178467 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:33 9.10 9:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e70) v8 -- 0x56302a97d830 con 0x7fe66c01bab0 2023-05-20T01:22:13.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.676+0000 7fe67c467080 1 -- 172.21.15.16:0/1001178467 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:34 9.10 9:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e70) v8 -- 0x56302aaf2ed0 con 0x7fe66c01bab0 2023-05-20T01:22:13.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.676+0000 7fe67c467080 1 -- 172.21.15.16:0/1001178467 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:35 9.10 9:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e70) v8 -- 0x56302aaf5a50 con 0x7fe66c01bab0 2023-05-20T01:22:13.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.676+0000 7fe67c467080 1 -- 172.21.15.16:0/1001178467 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:36 9.10 9:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e70) v8 -- 0x56302aaf6250 con 0x7fe66c01bab0 2023-05-20T01:22:13.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.676+0000 7fe67c467080 1 -- 172.21.15.16:0/1001178467 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:37 9.10 9:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e70) v8 -- 0x56302aaf6a50 con 0x7fe66c01bab0 2023-05-20T01:22:13.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.676+0000 7fe67c467080 1 -- 172.21.15.16:0/1001178467 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:38 9.14 9:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e70) v8 -- 0x56302aaf7250 con 0x56302a9a3430 2023-05-20T01:22:13.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.676+0000 7fe67c467080 1 -- 172.21.15.16:0/1001178467 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:39 9.14 9:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e70) v8 -- 0x56302aaf7a50 con 0x56302a9a3430 2023-05-20T01:22:13.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.676+0000 7fe67c467080 1 -- 172.21.15.16:0/1001178467 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:40 9.14 9:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e70) v8 -- 0x56302aaf8250 con 0x56302a9a3430 2023-05-20T01:22:13.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.676+0000 7fe67c467080 1 -- 172.21.15.16:0/1001178467 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:41 9.14 9:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e70) v8 -- 0x56302aaf8a50 con 0x56302a9a3430 2023-05-20T01:22:13.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.676+0000 7fe67c467080 1 -- 172.21.15.16:0/1001178467 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:42 9.1c 9:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e70) v8 -- 0x56302aaf9250 con 0x7fe6680158e0 2023-05-20T01:22:13.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.676+0000 7fe67c467080 1 -- 172.21.15.16:0/1001178467 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:43 9.6 9:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e70) v8 -- 0x56302aaf9a50 con 0x7fe668006380 2023-05-20T01:22:13.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.676+0000 7fe67c467080 1 -- 172.21.15.16:0/1001178467 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:44 9.e 9:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e70) v8 -- 0x56302aafa250 con 0x7fe664057730 2023-05-20T01:22:13.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.676+0000 7fe67c467080 1 -- 172.21.15.16:0/1001178467 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:45 9.15 9:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e70) v8 -- 0x56302aafaa50 con 0x7fe664057730 2023-05-20T01:22:13.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.676+0000 7fe67c467080 1 -- 172.21.15.16:0/1001178467 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:46 9.15 9:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e70) v8 -- 0x56302aafb440 con 0x7fe664057730 2023-05-20T01:22:13.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.676+0000 7fe67c467080 1 -- 172.21.15.16:0/1001178467 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:47 9.d 9:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e70) v8 -- 0x56302aafbee0 con 0x7fe66c037040 2023-05-20T01:22:13.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.676+0000 7fe67c467080 1 -- 172.21.15.16:0/1001178467 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:48 9.1d 9:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e70) v8 -- 0x56302aafc6e0 con 0x7fe6680158e0 2023-05-20T01:22:13.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.678+0000 7fe671d66700 1 -- 172.21.15.16:0/1001178467 <== osd.5 v2:172.21.15.132:6816/1844689995 6 ==== osd_op_reply(43 mds0_openfiles.0 [delete] v70'28 uv28 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fe66803d040 con 0x7fe668006380 2023-05-20T01:22:13.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.678+0000 7fe67c467080 1 -- 172.21.15.16:0/1001178467 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:49 9.13 9:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e70) v8 -- 0x56302aafaa50 con 0x7fe664057730 2023-05-20T01:22:13.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.678+0000 7fe672567700 1 -- 172.21.15.16:0/1001178467 <== osd.0 v2:172.21.15.16:6824/3923634179 7 ==== osd_op_reply(42 607.00000000 [delete] v70'3 uv3 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe664066040 con 0x7fe6680158e0 2023-05-20T01:22:13.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.678+0000 7fe67c467080 1 -- 172.21.15.16:0/1001178467 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:50 9.13 9:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e70) v8 -- 0x56302aafaa50 con 0x7fe664057730 2023-05-20T01:22:13.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.679+0000 7fe672567700 1 -- 172.21.15.16:0/1001178467 <== osd.2 v2:172.21.15.16:6816/293187030 2 ==== osd_op_reply(47 mds_snaptable [delete] v70'2 uv2 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fe664066040 con 0x7fe66c037040 2023-05-20T01:22:13.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.679+0000 7fe67c467080 1 -- 172.21.15.16:0/1001178467 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:51 9.b 9:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e70) v8 -- 0x56302aafaa50 con 0x7fe6680158e0 2023-05-20T01:22:13.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.680+0000 7fe672d68700 1 -- 172.21.15.16:0/1001178467 <== osd.1 v2:172.21.15.16:6808/3860264220 5 ==== osd_op_reply(38 1.00000000.inode [delete] v70'126 uv126 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fe66c016070 con 0x56302a9a3430 2023-05-20T01:22:13.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.680+0000 7fe67c467080 1 -- 172.21.15.16:0/1001178467 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:52 9.7 9:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e70) v8 -- 0x56302aafaa50 con 0x7fe668006380 2023-05-20T01:22:13.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.680+0000 7fe672d68700 1 -- 172.21.15.16:0/1001178467 <== osd.7 v2:172.21.15.132:6800/466001149 7 ==== osd_op_reply(45 604.00000000 [delete] v70'4 uv4 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe66c016070 con 0x7fe664057730 2023-05-20T01:22:13.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.680+0000 7fe672d68700 1 -- 172.21.15.16:0/1001178467 <== osd.7 v2:172.21.15.132:6800/466001149 8 ==== osd_op_reply(46 500.00000000 [delete] v70'5 uv5 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe66c016070 con 0x7fe664057730 2023-05-20T01:22:13.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.680+0000 7fe67c467080 1 -- 172.21.15.16:0/1001178467 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:53 9.f 9:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e70) v8 -- 0x56302aafa5b0 con 0x7fe668006380 2023-05-20T01:22:13.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.680+0000 7fe67c467080 1 -- 172.21.15.16:0/1001178467 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:54 9.1f 9:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e70) v8 -- 0x56302aafcb80 con 0x56302a9a3430 2023-05-20T01:22:13.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.681+0000 7fe672567700 1 -- 172.21.15.16:0/1001178467 <== osd.4 v2:172.21.15.132:6817/506317223 3 ==== osd_op_reply(33 601.00000000 [delete] v70'11 uv11 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe664066040 con 0x7fe66c01bab0 2023-05-20T01:22:13.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.681+0000 7fe67c467080 1 -- 172.21.15.16:0/1001178467 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:55 9.1f 9:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e70) v8 -- 0x56302aafcb80 con 0x56302a9a3430 2023-05-20T01:22:13.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.681+0000 7fe672567700 1 -- 172.21.15.16:0/1001178467 <== osd.0 v2:172.21.15.16:6824/3923634179 8 ==== osd_op_reply(48 605.00000000 [delete] v70'3 uv3 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe664066040 con 0x7fe6680158e0 2023-05-20T01:22:13.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.683+0000 7fe672d68700 1 -- 172.21.15.16:0/1001178467 <== osd.7 v2:172.21.15.132:6800/466001149 9 ==== osd_op_reply(44 608.00000000 [delete] v70'3 uv3 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe66c016070 con 0x7fe664057730 2023-05-20T01:22:13.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.683+0000 7fe672567700 1 -- 172.21.15.16:0/1001178467 <== osd.4 v2:172.21.15.132:6817/506317223 4 ==== osd_op_reply(34 602.00000000 [delete] v70'12 uv12 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe664066040 con 0x7fe66c01bab0 2023-05-20T01:22:13.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.683+0000 7fe672d68700 1 -- 172.21.15.16:0/1001178467 <== osd.1 v2:172.21.15.16:6808/3860264220 6 ==== osd_op_reply(39 200.00000000 [delete] v70'127 uv127 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe66c016070 con 0x56302a9a3430 2023-05-20T01:22:13.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.684+0000 7fe672d68700 1 -- 172.21.15.16:0/1001178467 <== osd.1 v2:172.21.15.16:6808/3860264220 7 ==== osd_op_reply(40 200.00000001 [delete] v70'128 uv128 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe66c016070 con 0x56302a9a3430 2023-05-20T01:22:13.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.684+0000 7fe672d68700 1 -- 172.21.15.16:0/1001178467 <== osd.7 v2:172.21.15.132:6800/466001149 10 ==== osd_op_reply(49 mds0_inotable [delete] v70'4 uv4 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fe66c016070 con 0x7fe664057730 2023-05-20T01:22:13.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.684+0000 7fe672567700 1 -- 172.21.15.16:0/1001178467 <== osd.4 v2:172.21.15.132:6817/506317223 5 ==== osd_op_reply(35 10000000000.00000000 [delete] v70'13 uv13 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fe664066040 con 0x7fe66c01bab0 2023-05-20T01:22:13.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.684+0000 7fe672567700 1 -- 172.21.15.16:0/1001178467 <== osd.0 v2:172.21.15.16:6824/3923634179 9 ==== osd_op_reply(51 mds0_sessionmap [delete] v70'4 uv4 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7fe664066040 con 0x7fe6680158e0 2023-05-20T01:22:13.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.684+0000 7fe672d68700 1 -- 172.21.15.16:0/1001178467 <== osd.1 v2:172.21.15.16:6808/3860264220 8 ==== osd_op_reply(41 606.00000000 [delete] v70'129 uv129 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe66c016070 con 0x56302a9a3430 2023-05-20T01:22:13.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.684+0000 7fe672d68700 1 -- 172.21.15.16:0/1001178467 <== osd.7 v2:172.21.15.132:6800/466001149 11 ==== osd_op_reply(50 100.00000000 [delete] v70'5 uv5 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe66c016070 con 0x7fe664057730 2023-05-20T01:22:13.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.684+0000 7fe672567700 1 -- 172.21.15.16:0/1001178467 <== osd.4 v2:172.21.15.132:6817/506317223 6 ==== osd_op_reply(36 600.00000000 [delete] v70'14 uv14 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe664066040 con 0x7fe66c01bab0 2023-05-20T01:22:13.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.684+0000 7fe672567700 1 -- 172.21.15.16:0/1001178467 <== osd.4 v2:172.21.15.132:6817/506317223 7 ==== osd_op_reply(37 603.00000000 [delete] v70'15 uv15 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe664066040 con 0x7fe66c01bab0 2023-05-20T01:22:13.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.686+0000 7fe671d66700 1 -- 172.21.15.16:0/1001178467 <== osd.5 v2:172.21.15.132:6816/1844689995 7 ==== osd_op_reply(53 400.00000000 [delete] v70'2 uv2 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe66803d040 con 0x7fe668006380 2023-05-20T01:22:13.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.686+0000 7fe672d68700 1 -- 172.21.15.16:0/1001178467 <== osd.1 v2:172.21.15.16:6808/3860264220 9 ==== osd_op_reply(54 100.00000000.inode [delete] v70'7 uv7 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7fe66c016070 con 0x56302a9a3430 2023-05-20T01:22:13.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.686+0000 7fe671d66700 1 -- 172.21.15.16:0/1001178467 <== osd.5 v2:172.21.15.132:6816/1844689995 8 ==== osd_op_reply(52 609.00000000 [delete] v70'3 uv3 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe66803d040 con 0x7fe668006380 2023-05-20T01:22:13.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.686+0000 7fe672d68700 1 -- 172.21.15.16:0/1001178467 <== osd.1 v2:172.21.15.16:6808/3860264220 10 ==== osd_op_reply(55 1.00000000 [delete] v70'8 uv8 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fe66c016070 con 0x56302a9a3430 2023-05-20T01:22:13.707 INFO:teuthology.orchestra.run.smithi016.stdout:Removed 23 objects 2023-05-20T01:22:13.707 INFO:teuthology.orchestra.run.smithi016.stdout:successfully purged pool cephfs_metadata 2023-05-20T01:22:13.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.686+0000 7fe67c467080 1 -- 172.21.15.16:0/1001178467 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x7fe6680158e0 msgr2=0x7fe668016380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:13.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.686+0000 7fe67c467080 1 --2- 172.21.15.16:0/1001178467 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x7fe6680158e0 0x7fe668016380 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:13.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.686+0000 7fe67c467080 1 -- 172.21.15.16:0/1001178467 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x56302a9a3430 msgr2=0x56302a9a10e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:13.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.686+0000 7fe67c467080 1 --2- 172.21.15.16:0/1001178467 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x56302a9a3430 0x56302a9a10e0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:13.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.686+0000 7fe67c467080 1 -- 172.21.15.16:0/1001178467 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x7fe66c037040 msgr2=0x7fe66c037410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:13.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.686+0000 7fe67c467080 1 --2- 172.21.15.16:0/1001178467 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x7fe66c037040 0x7fe66c037410 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:13.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.686+0000 7fe67c467080 1 -- 172.21.15.16:0/1001178467 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x7fe664009040 msgr2=0x7fe664058fb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:13.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.686+0000 7fe67c467080 1 --2- 172.21.15.16:0/1001178467 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x7fe664009040 0x7fe664058fb0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:13.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.686+0000 7fe67c467080 1 -- 172.21.15.16:0/1001178467 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x7fe66c01bab0 msgr2=0x7fe66c0103f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:13.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.686+0000 7fe67c467080 1 --2- 172.21.15.16:0/1001178467 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x7fe66c01bab0 0x7fe66c0103f0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:13.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.686+0000 7fe67c467080 1 -- 172.21.15.16:0/1001178467 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x7fe668006380 msgr2=0x7fe6680067f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:13.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.686+0000 7fe67c467080 1 --2- 172.21.15.16:0/1001178467 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x7fe668006380 0x7fe6680067f0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:13.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.686+0000 7fe672d68700 1 -- 172.21.15.16:0/1001178467 reap_dead start 2023-05-20T01:22:13.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.686+0000 7fe67c467080 1 -- 172.21.15.16:0/1001178467 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x7fe668009040 msgr2=0x7fe668018940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:13.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.686+0000 7fe67c467080 1 --2- 172.21.15.16:0/1001178467 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x7fe668009040 0x7fe668018940 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:13.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.686+0000 7fe672d68700 1 -- 172.21.15.16:0/1001178467 reap_dead start 2023-05-20T01:22:13.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.686+0000 7fe67c467080 1 -- 172.21.15.16:0/1001178467 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x7fe664057730 msgr2=0x7fe6640871c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:13.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.686+0000 7fe67c467080 1 --2- 172.21.15.16:0/1001178467 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x7fe664057730 0x7fe6640871c0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:13.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.686+0000 7fe67c467080 1 -- 172.21.15.16:0/1001178467 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe654069630 msgr2=0x7fe65406bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:13.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.686+0000 7fe67c467080 1 --2- 172.21.15.16:0/1001178467 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe654069630 0x7fe65406bae0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x56302aadab60 tx=0x7fe668005310 comp rx=0 tx=0).stop 2023-05-20T01:22:13.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.686+0000 7fe67c467080 1 -- 172.21.15.16:0/1001178467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56302a9c3210 msgr2=0x56302a9a6bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:13.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.686+0000 7fe67c467080 1 --2- 172.21.15.16:0/1001178467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56302a9c3210 0x56302a9a6bf0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fe664015040 tx=0x7fe664002fb0 comp rx=0 tx=0).stop 2023-05-20T01:22:13.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.686+0000 7fe67c467080 1 -- 172.21.15.16:0/1001178467 shutdown_connections 2023-05-20T01:22:13.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.686+0000 7fe67c467080 1 --2- 172.21.15.16:0/1001178467 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe654069630 0x7fe65406bae0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:13.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.686+0000 7fe67c467080 1 --2- 172.21.15.16:0/1001178467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56302a9c3210 0x56302a9a6bf0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:13.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.686+0000 7fe67c467080 1 --2- 172.21.15.16:0/1001178467 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x7fe664057730 0x7fe6640871c0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:13.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.686+0000 7fe67c467080 1 -- 172.21.15.16:0/1001178467 >> 172.21.15.16:0/1001178467 conn(0x56302a9a7350 msgr2=0x56302a9c5df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:13.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.686+0000 7fe67c467080 1 -- 172.21.15.16:0/1001178467 shutdown_connections 2023-05-20T01:22:13.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.686+0000 7fe67c467080 1 -- 172.21.15.16:0/1001178467 wait complete. 2023-05-20T01:22:13.713 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs reset cephfs --yes-i-really-mean-it 2023-05-20T01:22:13.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.872+0000 7f57efb40700 1 Processor -- start 2023-05-20T01:22:13.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.872+0000 7f57efb40700 1 -- start start 2023-05-20T01:22:13.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.873+0000 7f57efb40700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f57e80b57b0 0x7f57e8071c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:13.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.873+0000 7f57efb40700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f57e815f000 0x7f57e8072170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:13.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.873+0000 7f57efb40700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57e806a5b0 con 0x7f57e80d09d0 2023-05-20T01:22:13.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.873+0000 7f57efb40700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57e806a850 con 0x7f57e80b57b0 2023-05-20T01:22:13.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.873+0000 7f57efb40700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57e806ab30 con 0x7f57e815f000 2023-05-20T01:22:13.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.873+0000 7f57ee0dd700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f57e80d09d0 0x7f57e806d4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:45854/0 (socket says 172.21.15.16:45854) 2023-05-20T01:22:13.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.873+0000 7f57ee0dd700 1 -- 172.21.15.16:0/3240069384 learned_addr learned my addr 172.21.15.16:0/3240069384 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:22:13.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.874+0000 7f57ed8dc700 1 --2- 172.21.15.16:0/3240069384 >> v2:172.21.15.132:3300/0 conn(0x7f57e80b57b0 0x7f57e8071c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:13.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.874+0000 7f57ed0db700 1 --2- 172.21.15.16:0/3240069384 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f57e815f000 0x7f57e8072170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:13.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.874+0000 7f57ec8da700 1 -- 172.21.15.16:0/3240069384 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 81220058 0 0) 0x7f57e806a5b0 con 0x7f57e80d09d0 2023-05-20T01:22:13.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.874+0000 7f57ec8da700 1 -- 172.21.15.16:0/3240069384 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57d40035b0 con 0x7f57e80d09d0 2023-05-20T01:22:13.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.874+0000 7f57ed8dc700 1 -- 172.21.15.16:0/3240069384 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f57e815f000 msgr2=0x7f57e8072170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:13.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.874+0000 7f57ed8dc700 1 --2- 172.21.15.16:0/3240069384 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f57e815f000 0x7f57e8072170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:13.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.874+0000 7f57ed8dc700 1 -- 172.21.15.16:0/3240069384 >> v1:172.21.15.16:6789/0 conn(0x7f57e80d09d0 legacy=0x7f57e806d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:13.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.874+0000 7f57ed8dc700 1 -- 172.21.15.16:0/3240069384 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57e806ae10 con 0x7f57e80b57b0 2023-05-20T01:22:13.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.874+0000 7f57ed8dc700 1 --2- 172.21.15.16:0/3240069384 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f57e80b57b0 0x7f57e8071c30 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f57e4009840 tx=0x7f57e4009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7464c12133f7c51 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:13.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.874+0000 7f57ec8da700 1 -- 172.21.15.16:0/3240069384 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e4011830 con 0x7f57e80b57b0 2023-05-20T01:22:13.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.874+0000 7f57ec8da700 1 -- 172.21.15.16:0/3240069384 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57e4011990 con 0x7f57e80b57b0 2023-05-20T01:22:13.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.874+0000 7f57ec8da700 1 -- 172.21.15.16:0/3240069384 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e4011d20 con 0x7f57e80b57b0 2023-05-20T01:22:13.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.875+0000 7f57efb40700 1 -- 172.21.15.16:0/3240069384 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f57e80b57b0 msgr2=0x7f57e8071c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:13.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.875+0000 7f57efb40700 1 --2- 172.21.15.16:0/3240069384 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f57e80b57b0 0x7f57e8071c30 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f57e4009840 tx=0x7f57e4009b50 comp rx=0 tx=0).stop 2023-05-20T01:22:13.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.875+0000 7f57efb40700 1 -- 172.21.15.16:0/3240069384 shutdown_connections 2023-05-20T01:22:13.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.875+0000 7f57efb40700 1 --2- 172.21.15.16:0/3240069384 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f57e80b57b0 0x7f57e8071c30 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:13.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.875+0000 7f57efb40700 1 --2- 172.21.15.16:0/3240069384 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f57e815f000 0x7f57e8072170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:13.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.875+0000 7f57efb40700 1 -- 172.21.15.16:0/3240069384 >> 172.21.15.16:0/3240069384 conn(0x7f57e806c260 msgr2=0x7f57e80cffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:13.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.875+0000 7f57efb40700 1 -- 172.21.15.16:0/3240069384 shutdown_connections 2023-05-20T01:22:13.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.875+0000 7f57efb40700 1 -- 172.21.15.16:0/3240069384 wait complete. 2023-05-20T01:22:13.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.876+0000 7f57efb40700 1 Processor -- start 2023-05-20T01:22:13.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.876+0000 7f57efb40700 1 -- start start 2023-05-20T01:22:13.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.877+0000 7f57efb40700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f57e80b57b0 0x7f57e81589d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:13.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.877+0000 7f57efb40700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f57e80d09d0 0x7f57e80c7110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:13.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.877+0000 7f57efb40700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57e815f000 0x7f57e815cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:13.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.877+0000 7f57efb40700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57e80c4df0 con 0x7f57e815f000 2023-05-20T01:22:13.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.877+0000 7f57efb40700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57e80c7ec0 con 0x7f57e80d09d0 2023-05-20T01:22:13.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.877+0000 7f57efb40700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57e80c47a0 con 0x7f57e80b57b0 2023-05-20T01:22:13.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.877+0000 7f57ee0dd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57e815f000 0x7f57e815cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:13.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.877+0000 7f57ee0dd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57e815f000 0x7f57e815cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34332/0 (socket says 172.21.15.16:34332) 2023-05-20T01:22:13.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.877+0000 7f57ee0dd700 1 -- 172.21.15.16:0/1068379578 learned_addr learned my addr 172.21.15.16:0/1068379578 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:22:13.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.877+0000 7f57ed0db700 1 --2- 172.21.15.16:0/1068379578 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f57e80d09d0 0x7f57e80c7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:13.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.877+0000 7f57ed8dc700 1 --2- 172.21.15.16:0/1068379578 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f57e80b57b0 0x7f57e81589d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:13.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.877+0000 7f57ee0dd700 1 -- 172.21.15.16:0/1068379578 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f57e80b57b0 msgr2=0x7f57e81589d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:13.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.877+0000 7f57ee0dd700 1 --2- 172.21.15.16:0/1068379578 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f57e80b57b0 0x7f57e81589d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:13.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.877+0000 7f57ee0dd700 1 -- 172.21.15.16:0/1068379578 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f57e80d09d0 msgr2=0x7f57e80c7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:13.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.877+0000 7f57ee0dd700 1 --2- 172.21.15.16:0/1068379578 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f57e80d09d0 0x7f57e80c7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:13.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.877+0000 7f57ee0dd700 1 -- 172.21.15.16:0/1068379578 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57e40094a0 con 0x7f57e815f000 2023-05-20T01:22:13.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.878+0000 7f57ee0dd700 1 --2- 172.21.15.16:0/1068379578 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57e815f000 0x7f57e815cf90 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f57e000cb40 tx=0x7f57e000cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:13.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.878+0000 7f57de7fc700 1 -- 172.21.15.16:0/1068379578 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e000ab50 con 0x7f57e815f000 2023-05-20T01:22:13.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.878+0000 7f57de7fc700 1 -- 172.21.15.16:0/1068379578 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57e000acb0 con 0x7f57e815f000 2023-05-20T01:22:13.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.878+0000 7f57efb40700 1 -- 172.21.15.16:0/1068379578 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57e80af710 con 0x7f57e815f000 2023-05-20T01:22:13.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.878+0000 7f57de7fc700 1 -- 172.21.15.16:0/1068379578 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e0012620 con 0x7f57e815f000 2023-05-20T01:22:13.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.878+0000 7f57efb40700 1 -- 172.21.15.16:0/1068379578 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57e815f4e0 con 0x7f57e815f000 2023-05-20T01:22:13.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.879+0000 7f57efb40700 1 -- 172.21.15.16:0/1068379578 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57e80af8a0 con 0x7f57e815f000 2023-05-20T01:22:13.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.879+0000 7f57de7fc700 1 -- 172.21.15.16:0/1068379578 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f57e001b3b0 con 0x7f57e815f000 2023-05-20T01:22:13.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.879+0000 7f57de7fc700 1 --2- 172.21.15.16:0/1068379578 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f57cc06ad10 0x7f57cc06d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:13.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.880+0000 7f57de7fc700 1 -- 172.21.15.16:0/1068379578 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 7017+0+0 (secure 0 0 0) 0x7f57e0089660 con 0x7f57e815f000 2023-05-20T01:22:13.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.880+0000 7f57ed8dc700 1 --2- 172.21.15.16:0/1068379578 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f57cc06ad10 0x7f57cc06d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:13.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.880+0000 7f57ed8dc700 1 --2- 172.21.15.16:0/1068379578 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f57cc06ad10 0x7f57cc06d1c0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f57e4000c00 tx=0x7f57e4008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:13.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:13.884+0000 7f57de7fc700 1 -- 172.21.15.16:0/1068379578 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f57e0054350 con 0x7f57e815f000 2023-05-20T01:22:14.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:14.126+0000 7f57efb40700 1 -- 172.21.15.16:0/1068379578 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs reset", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f57e80747b0 con 0x7f57e815f000 2023-05-20T01:22:14.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:14.365+0000 7f57de7fc700 1 -- 172.21.15.16:0/1068379578 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs reset", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v40) v1 ==== 107+0+0 (secure 0 0 0) 0x7f57e0057970 con 0x7f57e815f000 2023-05-20T01:22:14.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:14.367+0000 7f57efb40700 1 -- 172.21.15.16:0/1068379578 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f57cc06ad10 msgr2=0x7f57cc06d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:14.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:14.367+0000 7f57efb40700 1 --2- 172.21.15.16:0/1068379578 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f57cc06ad10 0x7f57cc06d1c0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f57e4000c00 tx=0x7f57e4008030 comp rx=0 tx=0).stop 2023-05-20T01:22:14.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:14.367+0000 7f57efb40700 1 -- 172.21.15.16:0/1068379578 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57e815f000 msgr2=0x7f57e815cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:14.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:14.367+0000 7f57efb40700 1 --2- 172.21.15.16:0/1068379578 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57e815f000 0x7f57e815cf90 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f57e000cb40 tx=0x7f57e000cf00 comp rx=0 tx=0).stop 2023-05-20T01:22:14.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:14.367+0000 7f57efb40700 1 -- 172.21.15.16:0/1068379578 shutdown_connections 2023-05-20T01:22:14.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:14.367+0000 7f57efb40700 1 --2- 172.21.15.16:0/1068379578 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57e815f000 0x7f57e815cf90 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:14.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:14.367+0000 7f57efb40700 1 --2- 172.21.15.16:0/1068379578 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f57cc06ad10 0x7f57cc06d1c0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:14.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:14.367+0000 7f57efb40700 1 --2- 172.21.15.16:0/1068379578 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f57e80b57b0 0x7f57e81589d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:14.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:14.367+0000 7f57efb40700 1 --2- 172.21.15.16:0/1068379578 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f57e80d09d0 0x7f57e80c7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:14.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:14.368+0000 7f57efb40700 1 -- 172.21.15.16:0/1068379578 >> 172.21.15.16:0/1068379578 conn(0x7f57e806c260 msgr2=0x7f57e80b48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:14.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:14.368+0000 7f57efb40700 1 -- 172.21.15.16:0/1068379578 shutdown_connections 2023-05-20T01:22:14.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:14.368+0000 7f57efb40700 1 -- 172.21.15.16:0/1068379578 wait complete. 2023-05-20T01:22:14.374 INFO:tasks.ceph.mds.c.smithi016.stderr:2023-05-20T01:22:14.370+0000 7f993048a700 -1 mds.0.snaptable: load_2 could not read table: -2 2023-05-20T01:22:14.374 INFO:tasks.ceph.mds.c.smithi016.stderr:2023-05-20T01:22:14.370+0000 7f993048a700 -1 log_channel(cluster) log [ERR] : error reading table object 'mds_snaptable' -2 ((2) No such file or directory) 2023-05-20T01:22:14.389 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-05-20T01:22:14.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:14.515+0000 7fa16cd7b700 1 Processor -- start 2023-05-20T01:22:14.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:14.515+0000 7fa16cd7b700 1 -- start start 2023-05-20T01:22:14.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:14.516+0000 7fa16cd7b700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa16815aad0 0x7fa1680b27d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:14.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:14.516+0000 7fa16cd7b700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fa168073dc0 0x7fa1680cc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:14.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:14.516+0000 7fa16cd7b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa1680b1e20 0x7fa1680cc600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:14.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:14.516+0000 7fa16cd7b700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa16815aea0 con 0x7fa1680b1e20 2023-05-20T01:22:14.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:14.516+0000 7fa16cd7b700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa168070930 con 0x7fa168073dc0 2023-05-20T01:22:14.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:14.516+0000 7fa16cd7b700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1680b1b00 con 0x7fa16815aad0 2023-05-20T01:22:14.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:14.516+0000 7fa166d9d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa1680b1e20 0x7fa1680cc600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:14.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:14.516+0000 7fa166d9d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa1680b1e20 0x7fa1680cc600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34350/0 (socket says 172.21.15.16:34350) 2023-05-20T01:22:14.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:14.516+0000 7fa166d9d700 1 -- 172.21.15.16:0/2349032092 learned_addr learned my addr 172.21.15.16:0/2349032092 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:22:14.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:14.516+0000 7fa15ffff700 1 --2- 172.21.15.16:0/2349032092 >> v2:172.21.15.132:3300/0 conn(0x7fa168073dc0 0x7fa1680cc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:14.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:14.516+0000 7fa16659c700 1 --2- 172.21.15.16:0/2349032092 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa16815aad0 0x7fa1680b27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:14.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:14.516+0000 7fa166d9d700 1 -- 172.21.15.16:0/2349032092 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa16815aad0 msgr2=0x7fa1680b27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:14.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:14.516+0000 7fa166d9d700 1 --2- 172.21.15.16:0/2349032092 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa16815aad0 0x7fa1680b27d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:14.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:14.516+0000 7fa166d9d700 1 -- 172.21.15.16:0/2349032092 >> v2:172.21.15.132:3300/0 conn(0x7fa168073dc0 msgr2=0x7fa1680cc0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:14.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:14.516+0000 7fa166d9d700 1 --2- 172.21.15.16:0/2349032092 >> v2:172.21.15.132:3300/0 conn(0x7fa168073dc0 0x7fa1680cc0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:14.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:14.516+0000 7fa166d9d700 1 -- 172.21.15.16:0/2349032092 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa168070bf0 con 0x7fa1680b1e20 2023-05-20T01:22:14.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:14.516+0000 7fa166d9d700 1 --2- 172.21.15.16:0/2349032092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1680b1e20 0x7fa1680cc600 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fa1680cf710 tx=0x7fa16000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51cc4f5631a94b5c server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:14.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:14.516+0000 7fa165d9b700 1 -- 172.21.15.16:0/2349032092 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa16000e070 con 0x7fa1680b1e20 2023-05-20T01:22:14.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:14.516+0000 7fa165d9b700 1 -- 172.21.15.16:0/2349032092 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa16000b910 con 0x7fa1680b1e20 2023-05-20T01:22:14.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:14.516+0000 7fa165d9b700 1 -- 172.21.15.16:0/2349032092 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa16000bcc0 con 0x7fa1680b1e20 2023-05-20T01:22:14.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:14.517+0000 7fa16cd7b700 1 -- 172.21.15.16:0/2349032092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1680b1e20 msgr2=0x7fa1680cc600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:14.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:14.517+0000 7fa16cd7b700 1 --2- 172.21.15.16:0/2349032092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1680b1e20 0x7fa1680cc600 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fa1680cf710 tx=0x7fa16000b330 comp rx=0 tx=0).stop 2023-05-20T01:22:14.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:14.517+0000 7fa16cd7b700 1 -- 172.21.15.16:0/2349032092 shutdown_connections 2023-05-20T01:22:14.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:14.517+0000 7fa16cd7b700 1 --2- 172.21.15.16:0/2349032092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1680b1e20 0x7fa1680cc600 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:14.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:14.517+0000 7fa16cd7b700 1 --2- 172.21.15.16:0/2349032092 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa16815aad0 0x7fa1680b27d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:14.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:14.517+0000 7fa16cd7b700 1 --2- 172.21.15.16:0/2349032092 >> v2:172.21.15.132:3300/0 conn(0x7fa168073dc0 0x7fa1680cc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:14.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:14.517+0000 7fa16cd7b700 1 -- 172.21.15.16:0/2349032092 >> 172.21.15.16:0/2349032092 conn(0x7fa16806cbb0 msgr2=0x7fa16806cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:14.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:14.517+0000 7fa16cd7b700 1 -- 172.21.15.16:0/2349032092 shutdown_connections 2023-05-20T01:22:14.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:14.517+0000 7fa16cd7b700 1 -- 172.21.15.16:0/2349032092 wait complete. 2023-05-20T01:22:14.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:14.518+0000 7fa16cd7b700 1 Processor -- start 2023-05-20T01:22:14.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:14.518+0000 7fa16cd7b700 1 -- start start 2023-05-20T01:22:14.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:14.519+0000 7fa16cd7b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa168073dc0 0x7fa1680cdf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:14.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:14.519+0000 7fa16cd7b700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa1680b1e20 0x7fa168157790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:14.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:14.519+0000 7fa16cd7b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa16815aad0 0x7fa1681ef5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:14.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:14.519+0000 7fa16cd7b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1680cef70 con 0x7fa16815aad0 2023-05-20T01:22:14.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:14.519+0000 7fa16cd7b700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1681f34e0 con 0x7fa168073dc0 2023-05-20T01:22:14.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:14.519+0000 7fa16cd7b700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1681f3200 con 0x7fa1680b1e20 2023-05-20T01:22:14.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:14.519+0000 7fa166d9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa16815aad0 0x7fa1681ef5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:14.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:14.519+0000 7fa166d9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa16815aad0 0x7fa1681ef5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34356/0 (socket says 172.21.15.16:34356) 2023-05-20T01:22:14.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:14.519+0000 7fa166d9d700 1 -- 172.21.15.16:0/929636400 learned_addr learned my addr 172.21.15.16:0/929636400 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:22:14.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:14.519+0000 7fa16659c700 1 --2- 172.21.15.16:0/929636400 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa168073dc0 0x7fa1680cdf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:14.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:14.519+0000 7fa15ffff700 1 --2- 172.21.15.16:0/929636400 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa1680b1e20 0x7fa168157790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:14.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:14.520+0000 7fa166d9d700 1 -- 172.21.15.16:0/929636400 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa1680b1e20 msgr2=0x7fa168157790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:14.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:14.520+0000 7fa166d9d700 1 --2- 172.21.15.16:0/929636400 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa1680b1e20 0x7fa168157790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:14.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:14.520+0000 7fa166d9d700 1 -- 172.21.15.16:0/929636400 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa168073dc0 msgr2=0x7fa1680cdf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:14.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:14.520+0000 7fa166d9d700 1 --2- 172.21.15.16:0/929636400 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa168073dc0 0x7fa1680cdf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:14.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:14.520+0000 7fa166d9d700 1 -- 172.21.15.16:0/929636400 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa160009b30 con 0x7fa16815aad0 2023-05-20T01:22:14.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:14.520+0000 7fa166d9d700 1 --2- 172.21.15.16:0/929636400 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa16815aad0 0x7fa1681ef5e0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fa1680c6830 tx=0x7fa160004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:14.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:14.520+0000 7fa15effd700 1 -- 172.21.15.16:0/929636400 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1600099b0 con 0x7fa16815aad0 2023-05-20T01:22:14.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:14.520+0000 7fa15effd700 1 -- 172.21.15.16:0/929636400 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa16000ce80 con 0x7fa16815aad0 2023-05-20T01:22:14.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:14.520+0000 7fa16cd7b700 1 -- 172.21.15.16:0/929636400 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1680711b0 con 0x7fa16815aad0 2023-05-20T01:22:14.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:14.520+0000 7fa15effd700 1 -- 172.21.15.16:0/929636400 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa160004490 con 0x7fa16815aad0 2023-05-20T01:22:14.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:14.520+0000 7fa16cd7b700 1 -- 172.21.15.16:0/929636400 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa168070ed0 con 0x7fa16815aad0 2023-05-20T01:22:14.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:14.521+0000 7fa16cd7b700 1 -- 172.21.15.16:0/929636400 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1680b1720 con 0x7fa16815aad0 2023-05-20T01:22:14.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:14.521+0000 7fa15effd700 1 -- 172.21.15.16:0/929636400 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa1600045f0 con 0x7fa16815aad0 2023-05-20T01:22:14.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:14.522+0000 7fa15effd700 1 --2- 172.21.15.16:0/929636400 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa14806ace0 0x7fa14806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:14.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:14.522+0000 7fa15effd700 1 -- 172.21.15.16:0/929636400 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 7103+0+0 (secure 0 0 0) 0x7fa16008ab30 con 0x7fa16815aad0 2023-05-20T01:22:14.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:14.522+0000 7fa16659c700 1 --2- 172.21.15.16:0/929636400 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa14806ace0 0x7fa14806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:14.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:14.522+0000 7fa16659c700 1 --2- 172.21.15.16:0/929636400 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa14806ace0 0x7fa14806d190 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fa158009b40 tx=0x7fa158009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:14.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:14.526+0000 7fa15effd700 1 -- 172.21.15.16:0/929636400 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fa160055910 con 0x7fa16815aad0 2023-05-20T01:22:14.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:14.768+0000 7fa16cd7b700 1 -- 172.21.15.16:0/929636400 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7fa1680747b0 con 0x7fa16815aad0 2023-05-20T01:22:15.370 INFO:tasks.ceph.mon.a.smithi016.stderr:2023-05-20T01:22:15.366+0000 7f49eb8ae700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-05-20T01:22:15.370 INFO:tasks.ceph.mon.a.smithi016.stderr:2023-05-20T01:22:15.366+0000 7f49eb8ae700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2023-05-20T01:22:15.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.371+0000 7fa15effd700 1 -- 172.21.15.16:0/929636400 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v41) v1 ==== 162+0+0 (secure 0 0 0) 0x7fa160058f30 con 0x7fa16815aad0 2023-05-20T01:22:15.375 INFO:tasks.ceph.mds.c.smithi016.stderr: -30> 2023-05-20T01:22:14.370+0000 7f993048a700 -1 mds.0.snaptable: load_2 could not read table: -2 2023-05-20T01:22:15.376 INFO:tasks.ceph.mds.c.smithi016.stderr: -29> 2023-05-20T01:22:14.370+0000 7f993048a700 -1 log_channel(cluster) log [ERR] : error reading table object 'mds_snaptable' -2 ((2) No such file or directory) 2023-05-20T01:22:15.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.373+0000 7fa16cd7b700 1 -- 172.21.15.16:0/929636400 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa14806ace0 msgr2=0x7fa14806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:15.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.374+0000 7fa16cd7b700 1 --2- 172.21.15.16:0/929636400 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa14806ace0 0x7fa14806d190 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fa158009b40 tx=0x7fa158009210 comp rx=0 tx=0).stop 2023-05-20T01:22:15.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.374+0000 7fa16cd7b700 1 -- 172.21.15.16:0/929636400 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa16815aad0 msgr2=0x7fa1681ef5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:15.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.374+0000 7fa16cd7b700 1 --2- 172.21.15.16:0/929636400 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa16815aad0 0x7fa1681ef5e0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fa1680c6830 tx=0x7fa160004ad0 comp rx=0 tx=0).stop 2023-05-20T01:22:15.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.374+0000 7fa16cd7b700 1 -- 172.21.15.16:0/929636400 shutdown_connections 2023-05-20T01:22:15.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.374+0000 7fa16cd7b700 1 --2- 172.21.15.16:0/929636400 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa16815aad0 0x7fa1681ef5e0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:15.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.374+0000 7fa16cd7b700 1 --2- 172.21.15.16:0/929636400 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa168073dc0 0x7fa1680cdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:15.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.374+0000 7fa16cd7b700 1 --2- 172.21.15.16:0/929636400 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa14806ace0 0x7fa14806d190 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:15.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.374+0000 7fa16cd7b700 1 --2- 172.21.15.16:0/929636400 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa1680b1e20 0x7fa168157790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:15.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.374+0000 7fa16cd7b700 1 -- 172.21.15.16:0/929636400 >> 172.21.15.16:0/929636400 conn(0x7fa16806cbb0 msgr2=0x7fa1680c9de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:15.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.374+0000 7fa16cd7b700 1 -- 172.21.15.16:0/929636400 shutdown_connections 2023-05-20T01:22:15.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.374+0000 7fa16cd7b700 1 -- 172.21.15.16:0/929636400 wait complete. 2023-05-20T01:22:15.381 INFO:teuthology.orchestra.run.smithi016.stderr:cephfs marked joinable; MDS may join as newly active. 2023-05-20T01:22:15.395 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-20T01:22:15.403 INFO:tasks.ceph.mds.c.smithi016.stdout:starting mds.c at 2023-05-20T01:22:15.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.525+0000 7f7ea2ff7700 1 Processor -- start 2023-05-20T01:22:15.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.525+0000 7f7ea2ff7700 1 -- start start 2023-05-20T01:22:15.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.526+0000 7f7ea2ff7700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7e9c0b2760 0x7f7e9c0d1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:15.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.526+0000 7f7ea2ff7700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7e9c144020 0x7f7e9c0b0980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:15.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.526+0000 7f7ea2ff7700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e9c0cc500 con 0x7f7e9c0cec60 2023-05-20T01:22:15.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.526+0000 7f7ea2ff7700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e9c0cb6e0 con 0x7f7e9c144020 2023-05-20T01:22:15.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.526+0000 7f7ea2ff7700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9c1448a0 con 0x7f7e9c0b2760 2023-05-20T01:22:15.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.526+0000 7f7e9bfff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7e9c144020 0x7f7e9c0b0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:15.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.526+0000 7f7ea1594700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f7e9c0cec60 0x7f7e9c0cc990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:45904/0 (socket says 172.21.15.16:45904) 2023-05-20T01:22:15.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.526+0000 7f7ea1594700 1 -- 172.21.15.16:0/2269001868 learned_addr learned my addr 172.21.15.16:0/2269001868 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:22:15.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.526+0000 7f7ea0d93700 1 --2- 172.21.15.16:0/2269001868 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7e9c0b2760 0x7f7e9c0d1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:15.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.526+0000 7f7e9bfff700 1 -- 172.21.15.16:0/2269001868 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7e9c0b2760 msgr2=0x7f7e9c0d1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:15.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.526+0000 7f7e9bfff700 1 --2- 172.21.15.16:0/2269001868 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7e9c0b2760 0x7f7e9c0d1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:15.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.526+0000 7f7e9bfff700 1 -- 172.21.15.16:0/2269001868 >> v1:172.21.15.16:6789/0 conn(0x7f7e9c0cec60 legacy=0x7f7e9c0cc990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:15.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.526+0000 7f7e9b7fe700 1 -- 172.21.15.16:0/2269001868 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2370929122 0 0) 0x7f7e9c0cc500 con 0x7f7e9c0cec60 2023-05-20T01:22:15.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.526+0000 7f7e9bfff700 1 -- 172.21.15.16:0/2269001868 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e9c0b0ec0 con 0x7f7e9c144020 2023-05-20T01:22:15.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.526+0000 7f7ea0d93700 1 --2- 172.21.15.16:0/2269001868 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7e9c0b2760 0x7f7e9c0d1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:22:15.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.527+0000 7f7e9bfff700 1 --2- 172.21.15.16:0/2269001868 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e9c144020 0x7f7e9c0b0980 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f7e88009840 tx=0x7f7e88009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b7a6885f4e01cc1 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:15.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.527+0000 7f7e9b7fe700 1 -- 172.21.15.16:0/2269001868 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e88011830 con 0x7f7e9c144020 2023-05-20T01:22:15.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.527+0000 7f7e9b7fe700 1 -- 172.21.15.16:0/2269001868 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e88011990 con 0x7f7e9c144020 2023-05-20T01:22:15.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.527+0000 7f7e9b7fe700 1 -- 172.21.15.16:0/2269001868 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e88011d20 con 0x7f7e9c144020 2023-05-20T01:22:15.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.527+0000 7f7ea2ff7700 1 -- 172.21.15.16:0/2269001868 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e9c144020 msgr2=0x7f7e9c0b0980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:15.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.527+0000 7f7ea2ff7700 1 --2- 172.21.15.16:0/2269001868 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e9c144020 0x7f7e9c0b0980 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f7e88009840 tx=0x7f7e88009b50 comp rx=0 tx=0).stop 2023-05-20T01:22:15.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.527+0000 7f7ea2ff7700 1 -- 172.21.15.16:0/2269001868 shutdown_connections 2023-05-20T01:22:15.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.527+0000 7f7ea2ff7700 1 --2- 172.21.15.16:0/2269001868 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7e9c0b2760 0x7f7e9c0d1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:15.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.527+0000 7f7ea2ff7700 1 --2- 172.21.15.16:0/2269001868 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e9c144020 0x7f7e9c0b0980 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:15.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.527+0000 7f7ea2ff7700 1 -- 172.21.15.16:0/2269001868 >> 172.21.15.16:0/2269001868 conn(0x7f7e9c06ae30 msgr2=0x7f7e9c09a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:15.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.527+0000 7f7ea2ff7700 1 -- 172.21.15.16:0/2269001868 shutdown_connections 2023-05-20T01:22:15.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.527+0000 7f7ea2ff7700 1 -- 172.21.15.16:0/2269001868 wait complete. 2023-05-20T01:22:15.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.528+0000 7f7ea2ff7700 1 Processor -- start 2023-05-20T01:22:15.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.529+0000 7f7ea2ff7700 1 -- start start 2023-05-20T01:22:15.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.529+0000 7f7ea2ff7700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7e9c0b2760 0x7f7e9c159250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:15.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.529+0000 7f7ea2ff7700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7e9c0cec60 0x7f7e9c1f5f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:15.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.529+0000 7f7ea2ff7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e9c144020 0x7f7e9c0c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:15.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.529+0000 7f7ea2ff7700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9c0c5120 con 0x7f7e9c144020 2023-05-20T01:22:15.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.529+0000 7f7ea2ff7700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9c1f6450 con 0x7f7e9c0b2760 2023-05-20T01:22:15.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.529+0000 7f7ea2ff7700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9c0cada0 con 0x7f7e9c0cec60 2023-05-20T01:22:15.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.529+0000 7f7ea1594700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e9c144020 0x7f7e9c0c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:15.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.530+0000 7f7ea1594700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e9c144020 0x7f7e9c0c6dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34382/0 (socket says 172.21.15.16:34382) 2023-05-20T01:22:15.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.530+0000 7f7ea1594700 1 -- 172.21.15.16:0/2190565630 learned_addr learned my addr 172.21.15.16:0/2190565630 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:22:15.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.530+0000 7f7e9bfff700 1 --2- 172.21.15.16:0/2190565630 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7e9c0cec60 0x7f7e9c1f5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:15.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.530+0000 7f7ea0d93700 1 --2- 172.21.15.16:0/2190565630 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7e9c0b2760 0x7f7e9c159250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:15.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.530+0000 7f7ea1594700 1 -- 172.21.15.16:0/2190565630 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7e9c0cec60 msgr2=0x7f7e9c1f5f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:15.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.530+0000 7f7ea1594700 1 --2- 172.21.15.16:0/2190565630 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7e9c0cec60 0x7f7e9c1f5f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:15.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.530+0000 7f7ea1594700 1 -- 172.21.15.16:0/2190565630 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7e9c0b2760 msgr2=0x7f7e9c159250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:15.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.530+0000 7f7ea1594700 1 --2- 172.21.15.16:0/2190565630 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7e9c0b2760 0x7f7e9c159250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:15.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.530+0000 7f7ea1594700 1 -- 172.21.15.16:0/2190565630 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e880094a0 con 0x7f7e9c144020 2023-05-20T01:22:15.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.530+0000 7f7ea1594700 1 --2- 172.21.15.16:0/2190565630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e9c144020 0x7f7e9c0c6dd0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f7e9400cb60 tx=0x7f7e9400cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:15.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.530+0000 7f7e997fa700 1 -- 172.21.15.16:0/2190565630 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e9400ab50 con 0x7f7e9c144020 2023-05-20T01:22:15.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.530+0000 7f7e997fa700 1 -- 172.21.15.16:0/2190565630 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e9400acb0 con 0x7f7e9c144020 2023-05-20T01:22:15.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.530+0000 7f7ea2ff7700 1 -- 172.21.15.16:0/2190565630 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e9c0cbbd0 con 0x7f7e9c144020 2023-05-20T01:22:15.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.530+0000 7f7e997fa700 1 -- 172.21.15.16:0/2190565630 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e94007310 con 0x7f7e9c144020 2023-05-20T01:22:15.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.530+0000 7f7ea2ff7700 1 -- 172.21.15.16:0/2190565630 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e9c0cb8f0 con 0x7f7e9c144020 2023-05-20T01:22:15.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.531+0000 7f7ea2ff7700 1 -- 172.21.15.16:0/2190565630 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e9c0cbeb0 con 0x7f7e9c144020 2023-05-20T01:22:15.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.532+0000 7f7e997fa700 1 -- 172.21.15.16:0/2190565630 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7e94007470 con 0x7f7e9c144020 2023-05-20T01:22:15.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.532+0000 7f7e997fa700 1 --2- 172.21.15.16:0/2190565630 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f7e8006ace0 0x7f7e8006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:15.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.532+0000 7f7e997fa700 1 -- 172.21.15.16:0/2190565630 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 7103+0+0 (secure 0 0 0) 0x7f7e94089610 con 0x7f7e9c144020 2023-05-20T01:22:15.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.532+0000 7f7ea0d93700 1 --2- 172.21.15.16:0/2190565630 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f7e8006ace0 0x7f7e8006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:15.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.533+0000 7f7ea0d93700 1 --2- 172.21.15.16:0/2190565630 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f7e8006ace0 0x7f7e8006d190 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f7e90006fd0 tx=0x7f7e90008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:15.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.536+0000 7f7e997fa700 1 -- 172.21.15.16:0/2190565630 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f7e94054340 con 0x7f7e9c144020 2023-05-20T01:22:15.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.777+0000 7f7ea2ff7700 1 -- 172.21.15.16:0/2190565630 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7e9c0c4ba0 con 0x7f7e9c144020 2023-05-20T01:22:15.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.778+0000 7f7e997fa700 1 -- 172.21.15.16:0/2190565630 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+3910 (secure 0 0 0) 0x7f7e9c0c4ba0 con 0x7f7e9c144020 2023-05-20T01:22:15.782 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:22:15.782 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":41,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5824,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/2609276800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":2609276800},{"type":"v1","addr":"172.21.15.132:6834","nonce":2609276800}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32},{"gid":5827,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/29554453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":29554453},{"type":"v1","addr":"172.21.15.132:6835","nonce":29554453}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32},{"gid":6192,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/4090303296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4090303296},{"type":"v1","addr":"172.21.15.16:6836","nonce":4090303296}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-20T01:22:14.127671+0000","modified":"2023-05-20T01:22:15.367610+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":71,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[10,11],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-05-20T01:22:15.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.780+0000 7f7ea2ff7700 1 -- 172.21.15.16:0/2190565630 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f7e8006ace0 msgr2=0x7f7e8006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:15.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.780+0000 7f7ea2ff7700 1 --2- 172.21.15.16:0/2190565630 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f7e8006ace0 0x7f7e8006d190 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f7e90006fd0 tx=0x7f7e90008040 comp rx=0 tx=0).stop 2023-05-20T01:22:15.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.780+0000 7f7ea2ff7700 1 -- 172.21.15.16:0/2190565630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e9c144020 msgr2=0x7f7e9c0c6dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:15.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.780+0000 7f7ea2ff7700 1 --2- 172.21.15.16:0/2190565630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e9c144020 0x7f7e9c0c6dd0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f7e9400cb60 tx=0x7f7e9400cf20 comp rx=0 tx=0).stop 2023-05-20T01:22:15.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.780+0000 7f7ea2ff7700 1 -- 172.21.15.16:0/2190565630 shutdown_connections 2023-05-20T01:22:15.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.780+0000 7f7ea2ff7700 1 --2- 172.21.15.16:0/2190565630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e9c144020 0x7f7e9c0c6dd0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:15.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.780+0000 7f7ea2ff7700 1 --2- 172.21.15.16:0/2190565630 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7e9c0b2760 0x7f7e9c159250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:15.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.780+0000 7f7ea2ff7700 1 --2- 172.21.15.16:0/2190565630 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f7e8006ace0 0x7f7e8006d190 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:15.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.780+0000 7f7ea2ff7700 1 --2- 172.21.15.16:0/2190565630 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7e9c0cec60 0x7f7e9c1f5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:15.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.780+0000 7f7ea2ff7700 1 -- 172.21.15.16:0/2190565630 >> 172.21.15.16:0/2190565630 conn(0x7f7e9c06ae30 msgr2=0x7f7e9c0d0b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:15.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.780+0000 7f7ea2ff7700 1 -- 172.21.15.16:0/2190565630 shutdown_connections 2023-05-20T01:22:15.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.780+0000 7f7ea2ff7700 1 -- 172.21.15.16:0/2190565630 wait complete. 2023-05-20T01:22:15.786 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 41 2023-05-20T01:22:15.796 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-05-20T01:22:15.797 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-20T01:22:15.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.959+0000 7fc88a9ea700 1 Processor -- start 2023-05-20T01:22:15.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.959+0000 7fc88a9ea700 1 -- start start 2023-05-20T01:22:15.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.960+0000 7fc88a9ea700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc8840cfc50 0x7fc8840d2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:15.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.960+0000 7fc88a9ea700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fc8840cd970 0x7fc8840cddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:15.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.960+0000 7fc88a9ea700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8840b3b50 con 0x7fc8840b4750 2023-05-20T01:22:15.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.960+0000 7fc88a9ea700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8840cfad0 con 0x7fc8840cd970 2023-05-20T01:22:15.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.960+0000 7fc88a9ea700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8840d3df0 con 0x7fc8840cfc50 2023-05-20T01:22:15.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.960+0000 7fc8837fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fc8840b4750 0x7fc8840d0020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:45914/0 (socket says 172.21.15.16:45914) 2023-05-20T01:22:15.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.960+0000 7fc8837fe700 1 -- 172.21.15.16:0/1476251823 learned_addr learned my addr 172.21.15.16:0/1476251823 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:22:15.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.960+0000 7fc888f87700 1 --2- 172.21.15.16:0/1476251823 >> v2:172.21.15.132:3300/0 conn(0x7fc8840cd970 0x7fc8840cddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:15.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.960+0000 7fc883fff700 1 --2- 172.21.15.16:0/1476251823 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc8840cfc50 0x7fc8840d2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:15.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.960+0000 7fc882ffd700 1 -- 172.21.15.16:0/1476251823 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3359894856 0 0) 0x7fc8840b3b50 con 0x7fc8840b4750 2023-05-20T01:22:15.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.960+0000 7fc882ffd700 1 -- 172.21.15.16:0/1476251823 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc8700035b0 con 0x7fc8840b4750 2023-05-20T01:22:15.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.960+0000 7fc888f87700 1 -- 172.21.15.16:0/1476251823 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc8840cfc50 msgr2=0x7fc8840d2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:15.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.960+0000 7fc888f87700 1 --2- 172.21.15.16:0/1476251823 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc8840cfc50 0x7fc8840d2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:15.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.960+0000 7fc888f87700 1 -- 172.21.15.16:0/1476251823 >> v1:172.21.15.16:6789/0 conn(0x7fc8840b4750 legacy=0x7fc8840d0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:15.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.960+0000 7fc882ffd700 1 -- 172.21.15.16:0/1476251823 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3771975439 0 0) 0x7fc8700035b0 con 0x7fc8840b4750 2023-05-20T01:22:15.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.960+0000 7fc888f87700 1 -- 172.21.15.16:0/1476251823 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8840d0b90 con 0x7fc8840cd970 2023-05-20T01:22:15.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.961+0000 7fc888f87700 1 --2- 172.21.15.16:0/1476251823 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc8840cd970 0x7fc8840cddf0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fc87c009d90 tx=0x7fc87c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95f5b55a94d62106 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:15.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.961+0000 7fc882ffd700 1 -- 172.21.15.16:0/1476251823 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc87c00e070 con 0x7fc8840cd970 2023-05-20T01:22:15.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.961+0000 7fc882ffd700 1 -- 172.21.15.16:0/1476251823 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc87c003a20 con 0x7fc8840cd970 2023-05-20T01:22:15.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.961+0000 7fc882ffd700 1 -- 172.21.15.16:0/1476251823 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc874001670 con 0x7fc8840cd970 2023-05-20T01:22:15.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.961+0000 7fc88a9ea700 1 -- 172.21.15.16:0/1476251823 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc8840cd970 msgr2=0x7fc8840cddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:15.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.961+0000 7fc88a9ea700 1 --2- 172.21.15.16:0/1476251823 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc8840cd970 0x7fc8840cddf0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fc87c009d90 tx=0x7fc87c00b330 comp rx=0 tx=0).stop 2023-05-20T01:22:15.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.961+0000 7fc88a9ea700 1 -- 172.21.15.16:0/1476251823 shutdown_connections 2023-05-20T01:22:15.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.961+0000 7fc88a9ea700 1 --2- 172.21.15.16:0/1476251823 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc8840cd970 0x7fc8840cddf0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:15.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.961+0000 7fc88a9ea700 1 --2- 172.21.15.16:0/1476251823 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc8840cfc50 0x7fc8840d2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:15.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.961+0000 7fc88a9ea700 1 -- 172.21.15.16:0/1476251823 >> 172.21.15.16:0/1476251823 conn(0x7fc88409c590 msgr2=0x7fc8840d3240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:15.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.961+0000 7fc88a9ea700 1 -- 172.21.15.16:0/1476251823 shutdown_connections 2023-05-20T01:22:15.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.961+0000 7fc88a9ea700 1 -- 172.21.15.16:0/1476251823 wait complete. 2023-05-20T01:22:15.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.963+0000 7fc88a9ea700 1 Processor -- start 2023-05-20T01:22:15.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.963+0000 7fc88a9ea700 1 -- start start 2023-05-20T01:22:15.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.963+0000 7fc88a9ea700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc8840b4750 0x7fc88414f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:15.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.963+0000 7fc88a9ea700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8840cd970 0x7fc8840cc730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:15.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.964+0000 7fc88a9ea700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc8840cfc50 0x7fc8840ccc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:15.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.964+0000 7fc88a9ea700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8840cbef0 con 0x7fc8840cd970 2023-05-20T01:22:15.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.964+0000 7fc88a9ea700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8840cb2e0 con 0x7fc8840b4750 2023-05-20T01:22:15.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.964+0000 7fc88a9ea700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8840cb5c0 con 0x7fc8840cfc50 2023-05-20T01:22:15.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.964+0000 7fc8837fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8840cd970 0x7fc8840cc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:15.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.964+0000 7fc8837fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8840cd970 0x7fc8840cc730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34394/0 (socket says 172.21.15.16:34394) 2023-05-20T01:22:15.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.964+0000 7fc8837fe700 1 -- 172.21.15.16:0/4147821138 learned_addr learned my addr 172.21.15.16:0/4147821138 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:22:15.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.964+0000 7fc883fff700 1 --2- 172.21.15.16:0/4147821138 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc8840b4750 0x7fc88414f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:15.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.964+0000 7fc888f87700 1 --2- 172.21.15.16:0/4147821138 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc8840cfc50 0x7fc8840ccc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:15.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.964+0000 7fc8837fe700 1 -- 172.21.15.16:0/4147821138 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc8840cfc50 msgr2=0x7fc8840ccc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:15.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.964+0000 7fc8837fe700 1 --2- 172.21.15.16:0/4147821138 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc8840cfc50 0x7fc8840ccc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:15.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.964+0000 7fc8837fe700 1 -- 172.21.15.16:0/4147821138 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc8840b4750 msgr2=0x7fc88414f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:15.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.964+0000 7fc8837fe700 1 --2- 172.21.15.16:0/4147821138 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc8840b4750 0x7fc88414f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:15.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.964+0000 7fc8837fe700 1 -- 172.21.15.16:0/4147821138 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc87c009a70 con 0x7fc8840cd970 2023-05-20T01:22:15.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.964+0000 7fc8837fe700 1 --2- 172.21.15.16:0/4147821138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8840cd970 0x7fc8840cc730 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fc87400c520 tx=0x7fc87400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:15.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.964+0000 7fc880ff9700 1 -- 172.21.15.16:0/4147821138 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc874004c80 con 0x7fc8840cd970 2023-05-20T01:22:15.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.964+0000 7fc880ff9700 1 -- 172.21.15.16:0/4147821138 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc874004de0 con 0x7fc8840cd970 2023-05-20T01:22:15.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.964+0000 7fc88a9ea700 1 -- 172.21.15.16:0/4147821138 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8840b4380 con 0x7fc8840cd970 2023-05-20T01:22:15.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.964+0000 7fc880ff9700 1 -- 172.21.15.16:0/4147821138 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc874007310 con 0x7fc8840cd970 2023-05-20T01:22:15.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.964+0000 7fc88a9ea700 1 -- 172.21.15.16:0/4147821138 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc88415d0b0 con 0x7fc8840cd970 2023-05-20T01:22:15.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.965+0000 7fc88a9ea700 1 -- 172.21.15.16:0/4147821138 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8840d4000 con 0x7fc8840cd970 2023-05-20T01:22:15.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.966+0000 7fc880ff9700 1 -- 172.21.15.16:0/4147821138 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc874007470 con 0x7fc8840cd970 2023-05-20T01:22:15.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.966+0000 7fc880ff9700 1 --2- 172.21.15.16:0/4147821138 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc86406ad10 0x7fc86406d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:15.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.966+0000 7fc880ff9700 1 -- 172.21.15.16:0/4147821138 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 7103+0+0 (secure 0 0 0) 0x7fc874089590 con 0x7fc8840cd970 2023-05-20T01:22:15.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.966+0000 7fc883fff700 1 --2- 172.21.15.16:0/4147821138 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc86406ad10 0x7fc86406d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:15.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.967+0000 7fc883fff700 1 --2- 172.21.15.16:0/4147821138 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc86406ad10 0x7fc86406d1c0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fc86c009b40 tx=0x7fc86c009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:15.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:15.971+0000 7fc880ff9700 1 -- 172.21.15.16:0/4147821138 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fc8740541a0 con 0x7fc8840cd970 2023-05-20T01:22:16.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.216+0000 7fc88a9ea700 1 -- 172.21.15.16:0/4147821138 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc8841e4f00 con 0x7fc8840cd970 2023-05-20T01:22:16.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.216+0000 7fc880ff9700 1 -- 172.21.15.16:0/4147821138 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+3910 (secure 0 0 0) 0x7fc8740577c0 con 0x7fc8840cd970 2023-05-20T01:22:16.221 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:22:16.221 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":41,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5824,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/2609276800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":2609276800},{"type":"v1","addr":"172.21.15.132:6834","nonce":2609276800}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32},{"gid":5827,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/29554453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":29554453},{"type":"v1","addr":"172.21.15.132:6835","nonce":29554453}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32},{"gid":6192,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/4090303296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4090303296},{"type":"v1","addr":"172.21.15.16:6836","nonce":4090303296}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-20T01:22:14.127671+0000","modified":"2023-05-20T01:22:15.367610+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":71,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[10,11],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-05-20T01:22:16.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.218+0000 7fc88a9ea700 1 -- 172.21.15.16:0/4147821138 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc86406ad10 msgr2=0x7fc86406d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:16.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.218+0000 7fc88a9ea700 1 --2- 172.21.15.16:0/4147821138 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc86406ad10 0x7fc86406d1c0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fc86c009b40 tx=0x7fc86c009210 comp rx=0 tx=0).stop 2023-05-20T01:22:16.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.218+0000 7fc88a9ea700 1 -- 172.21.15.16:0/4147821138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8840cd970 msgr2=0x7fc8840cc730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:16.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.218+0000 7fc88a9ea700 1 --2- 172.21.15.16:0/4147821138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8840cd970 0x7fc8840cc730 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fc87400c520 tx=0x7fc87400c830 comp rx=0 tx=0).stop 2023-05-20T01:22:16.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.218+0000 7fc88a9ea700 1 -- 172.21.15.16:0/4147821138 shutdown_connections 2023-05-20T01:22:16.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.218+0000 7fc88a9ea700 1 --2- 172.21.15.16:0/4147821138 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc86406ad10 0x7fc86406d1c0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:16.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.218+0000 7fc88a9ea700 1 --2- 172.21.15.16:0/4147821138 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc8840cfc50 0x7fc8840ccc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:16.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.218+0000 7fc88a9ea700 1 --2- 172.21.15.16:0/4147821138 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc8840b4750 0x7fc88414f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:16.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.218+0000 7fc88a9ea700 1 --2- 172.21.15.16:0/4147821138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8840cd970 0x7fc8840cc730 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:16.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.218+0000 7fc88a9ea700 1 -- 172.21.15.16:0/4147821138 >> 172.21.15.16:0/4147821138 conn(0x7fc88409c590 msgr2=0x7fc8840d0020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:16.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.218+0000 7fc88a9ea700 1 -- 172.21.15.16:0/4147821138 shutdown_connections 2023-05-20T01:22:16.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.219+0000 7fc88a9ea700 1 -- 172.21.15.16:0/4147821138 wait complete. 2023-05-20T01:22:16.227 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 41 2023-05-20T01:22:16.235 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-20T01:22:16.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.356+0000 7f72dcb1d700 1 Processor -- start 2023-05-20T01:22:16.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.356+0000 7f72dcb1d700 1 -- start start 2023-05-20T01:22:16.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.357+0000 7f72dcb1d700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f72d8144020 0x7f72d80d1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:16.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.357+0000 7f72dcb1d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f72d80cec60 0x7f72d80cf0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:16.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.357+0000 7f72dcb1d700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72d80cc500 con 0x7f72d80b2760 2023-05-20T01:22:16.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.357+0000 7f72dcb1d700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72d80cb6e0 con 0x7f72d80cec60 2023-05-20T01:22:16.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.357+0000 7f72dcb1d700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72d80cf620 con 0x7f72d8144020 2023-05-20T01:22:16.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.357+0000 7f72d6d9d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f72d80cec60 0x7f72d80cf0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:16.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.357+0000 7f72d659c700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f72d80b2760 0x7f72d80b0980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:45928/0 (socket says 172.21.15.16:45928) 2023-05-20T01:22:16.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.357+0000 7f72d659c700 1 -- 172.21.15.16:0/2743121032 learned_addr learned my addr 172.21.15.16:0/2743121032 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:22:16.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.358+0000 7f72d5d9b700 1 --2- 172.21.15.16:0/2743121032 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f72d8144020 0x7f72d80d1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:16.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.358+0000 7f72d6d9d700 1 -- 172.21.15.16:0/2743121032 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f72d8144020 msgr2=0x7f72d80d1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:16.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.358+0000 7f72d6d9d700 1 --2- 172.21.15.16:0/2743121032 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f72d8144020 0x7f72d80d1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:16.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.358+0000 7f72d6d9d700 1 -- 172.21.15.16:0/2743121032 >> v1:172.21.15.16:6789/0 conn(0x7f72d80b2760 legacy=0x7f72d80b0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:16.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.358+0000 7f72d559a700 1 -- 172.21.15.16:0/2743121032 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1659515847 0 0) 0x7f72d80cc500 con 0x7f72d80b2760 2023-05-20T01:22:16.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.358+0000 7f72d6d9d700 1 -- 172.21.15.16:0/2743121032 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72d80cb8f0 con 0x7f72d80cec60 2023-05-20T01:22:16.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.358+0000 7f72d6d9d700 1 --2- 172.21.15.16:0/2743121032 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72d80cec60 0x7f72d80cf0e0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f72d0009d90 tx=0x7f72d000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dcbe53bd33f4dbe server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:16.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.358+0000 7f72d559a700 1 -- 172.21.15.16:0/2743121032 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72d000e070 con 0x7f72d80cec60 2023-05-20T01:22:16.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.358+0000 7f72d559a700 1 -- 172.21.15.16:0/2743121032 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72d000bbd0 con 0x7f72d80cec60 2023-05-20T01:22:16.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.358+0000 7f72d559a700 1 -- 172.21.15.16:0/2743121032 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72d0012430 con 0x7f72d80cec60 2023-05-20T01:22:16.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.359+0000 7f72dcb1d700 1 -- 172.21.15.16:0/2743121032 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72d80cec60 msgr2=0x7f72d80cf0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:16.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.359+0000 7f72dcb1d700 1 --2- 172.21.15.16:0/2743121032 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72d80cec60 0x7f72d80cf0e0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f72d0009d90 tx=0x7f72d000b330 comp rx=0 tx=0).stop 2023-05-20T01:22:16.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.359+0000 7f72dcb1d700 1 -- 172.21.15.16:0/2743121032 shutdown_connections 2023-05-20T01:22:16.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.359+0000 7f72dcb1d700 1 --2- 172.21.15.16:0/2743121032 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72d80cec60 0x7f72d80cf0e0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:16.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.359+0000 7f72dcb1d700 1 --2- 172.21.15.16:0/2743121032 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f72d8144020 0x7f72d80d1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:16.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.359+0000 7f72dcb1d700 1 -- 172.21.15.16:0/2743121032 >> 172.21.15.16:0/2743121032 conn(0x7f72d806ae30 msgr2=0x7f72d809a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:16.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.359+0000 7f72dcb1d700 1 -- 172.21.15.16:0/2743121032 shutdown_connections 2023-05-20T01:22:16.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.359+0000 7f72dcb1d700 1 -- 172.21.15.16:0/2743121032 wait complete. 2023-05-20T01:22:16.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.360+0000 7f72dcb1d700 1 Processor -- start 2023-05-20T01:22:16.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.360+0000 7f72dcb1d700 1 -- start start 2023-05-20T01:22:16.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.361+0000 7f72dcb1d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f72d80b2760 0x7f72d8157660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:16.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.361+0000 7f72dcb1d700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f72d80cec60 0x7f72d80b5630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:16.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.361+0000 7f72dcb1d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72d8144020 0x7f72d81efe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:16.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.361+0000 7f72dcb1d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72d81f03a0 con 0x7f72d8144020 2023-05-20T01:22:16.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.361+0000 7f72dcb1d700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72d80b5d30 con 0x7f72d80b2760 2023-05-20T01:22:16.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.361+0000 7f72dcb1d700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72d80b6010 con 0x7f72d80cec60 2023-05-20T01:22:16.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.361+0000 7f72d6d9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72d8144020 0x7f72d81efe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:16.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.361+0000 7f72d6d9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72d8144020 0x7f72d81efe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34410/0 (socket says 172.21.15.16:34410) 2023-05-20T01:22:16.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.361+0000 7f72d6d9d700 1 -- 172.21.15.16:0/3237453868 learned_addr learned my addr 172.21.15.16:0/3237453868 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:22:16.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.361+0000 7f72d659c700 1 --2- 172.21.15.16:0/3237453868 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f72d80b2760 0x7f72d8157660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:16.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.361+0000 7f72d5d9b700 1 --2- 172.21.15.16:0/3237453868 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f72d80cec60 0x7f72d80b5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:16.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.361+0000 7f72d6d9d700 1 -- 172.21.15.16:0/3237453868 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f72d80cec60 msgr2=0x7f72d80b5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:16.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.361+0000 7f72d6d9d700 1 --2- 172.21.15.16:0/3237453868 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f72d80cec60 0x7f72d80b5630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:16.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.361+0000 7f72d6d9d700 1 -- 172.21.15.16:0/3237453868 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f72d80b2760 msgr2=0x7f72d8157660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:16.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.361+0000 7f72d6d9d700 1 --2- 172.21.15.16:0/3237453868 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f72d80b2760 0x7f72d8157660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:16.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.361+0000 7f72d6d9d700 1 -- 172.21.15.16:0/3237453868 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72d0009a70 con 0x7f72d8144020 2023-05-20T01:22:16.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.361+0000 7f72d659c700 1 --2- 172.21.15.16:0/3237453868 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f72d80b2760 0x7f72d8157660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:22:16.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.362+0000 7f72d6d9d700 1 --2- 172.21.15.16:0/3237453868 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72d8144020 0x7f72d81efe60 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f72d80cec10 tx=0x7f72d0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:16.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.362+0000 7f72ceffd700 1 -- 172.21.15.16:0/3237453868 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72d0012430 con 0x7f72d8144020 2023-05-20T01:22:16.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.362+0000 7f72ceffd700 1 -- 172.21.15.16:0/3237453868 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72d0020070 con 0x7f72d8144020 2023-05-20T01:22:16.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.362+0000 7f72dcb1d700 1 -- 172.21.15.16:0/3237453868 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72d80cbeb0 con 0x7f72d8144020 2023-05-20T01:22:16.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.362+0000 7f72ceffd700 1 -- 172.21.15.16:0/3237453868 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72d0004280 con 0x7f72d8144020 2023-05-20T01:22:16.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.362+0000 7f72dcb1d700 1 -- 172.21.15.16:0/3237453868 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72d80cbbd0 con 0x7f72d8144020 2023-05-20T01:22:16.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.363+0000 7f72dcb1d700 1 -- 172.21.15.16:0/3237453868 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72d80cc190 con 0x7f72d8144020 2023-05-20T01:22:16.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.363+0000 7f72ceffd700 1 -- 172.21.15.16:0/3237453868 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f72d00043e0 con 0x7f72d8144020 2023-05-20T01:22:16.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.363+0000 7f72ceffd700 1 --2- 172.21.15.16:0/3237453868 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f72b806ace0 0x7f72b806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:16.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.364+0000 7f72ceffd700 1 -- 172.21.15.16:0/3237453868 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 7103+0+0 (secure 0 0 0) 0x7f72d0022070 con 0x7f72d8144020 2023-05-20T01:22:16.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.364+0000 7f72d659c700 1 --2- 172.21.15.16:0/3237453868 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f72b806ace0 0x7f72b806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:16.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.364+0000 7f72d659c700 1 --2- 172.21.15.16:0/3237453868 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f72b806ace0 0x7f72b806d190 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f72c8000f60 tx=0x7f72c8007d30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:16.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.368+0000 7f72ceffd700 1 -- 172.21.15.16:0/3237453868 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f72d00563b0 con 0x7f72d8144020 2023-05-20T01:22:16.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.613+0000 7f72dcb1d700 1 -- 172.21.15.16:0/3237453868 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f72d80747b0 con 0x7f72d8144020 2023-05-20T01:22:16.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.613+0000 7f72ceffd700 1 -- 172.21.15.16:0/3237453868 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+4667 (secure 0 0 0) 0x7f72d00599d0 con 0x7f72d8144020 2023-05-20T01:22:16.618 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:22:16.618 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":42,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5824,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/2609276800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":2609276800},{"type":"v1","addr":"172.21.15.132:6834","nonce":2609276800}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32},{"gid":5827,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/29554453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":29554453},{"type":"v1","addr":"172.21.15.132:6835","nonce":29554453}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32},{"gid":6192,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/4090303296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4090303296},{"type":"v1","addr":"172.21.15.16:6836","nonce":4090303296}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32},{"gid":6507,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/3291457857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":3291457857},{"type":"v1","addr":"172.21.15.16:6837","nonce":3291457857}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-20T01:22:14.127671+0000","modified":"2023-05-20T01:22:15.367610+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":71,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[10,11],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-05-20T01:22:16.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.615+0000 7f72dcb1d700 1 -- 172.21.15.16:0/3237453868 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f72b806ace0 msgr2=0x7f72b806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:16.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.615+0000 7f72dcb1d700 1 --2- 172.21.15.16:0/3237453868 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f72b806ace0 0x7f72b806d190 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f72c8000f60 tx=0x7f72c8007d30 comp rx=0 tx=0).stop 2023-05-20T01:22:16.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.615+0000 7f72dcb1d700 1 -- 172.21.15.16:0/3237453868 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72d8144020 msgr2=0x7f72d81efe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:16.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.615+0000 7f72dcb1d700 1 --2- 172.21.15.16:0/3237453868 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72d8144020 0x7f72d81efe60 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f72d80cec10 tx=0x7f72d0004ad0 comp rx=0 tx=0).stop 2023-05-20T01:22:16.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.615+0000 7f72dcb1d700 1 -- 172.21.15.16:0/3237453868 shutdown_connections 2023-05-20T01:22:16.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.615+0000 7f72dcb1d700 1 --2- 172.21.15.16:0/3237453868 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72d8144020 0x7f72d81efe60 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:16.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.615+0000 7f72dcb1d700 1 --2- 172.21.15.16:0/3237453868 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f72d80b2760 0x7f72d8157660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:16.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.615+0000 7f72dcb1d700 1 --2- 172.21.15.16:0/3237453868 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f72b806ace0 0x7f72b806d190 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:16.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.615+0000 7f72dcb1d700 1 --2- 172.21.15.16:0/3237453868 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f72d80cec60 0x7f72d80b5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:16.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.615+0000 7f72dcb1d700 1 -- 172.21.15.16:0/3237453868 >> 172.21.15.16:0/3237453868 conn(0x7f72d806ae30 msgr2=0x7f72d80cdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:16.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.616+0000 7f72dcb1d700 1 -- 172.21.15.16:0/3237453868 shutdown_connections 2023-05-20T01:22:16.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.616+0000 7f72dcb1d700 1 -- 172.21.15.16:0/3237453868 wait complete. 2023-05-20T01:22:16.622 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 42 2023-05-20T01:22:16.632 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-20T01:22:16.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.752+0000 7fb824b95700 1 Processor -- start 2023-05-20T01:22:16.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.752+0000 7fb824b95700 1 -- start start 2023-05-20T01:22:16.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.753+0000 7fb824b95700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb8200d2c10 0x7fb8200ce080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:16.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.753+0000 7fb824b95700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8200b3b50 con 0x7fb8200b4750 2023-05-20T01:22:16.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.753+0000 7fb824b95700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8200d0b90 con 0x7fb8200cfc50 2023-05-20T01:22:16.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.753+0000 7fb824b95700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8200cfad0 con 0x7fb8200d2c10 2023-05-20T01:22:16.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.753+0000 7fb81ed9d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb8200d2c10 0x7fb8200ce080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:16.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.753+0000 7fb81dd9b700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fb8200b4750 0x7fb8200cd970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:45946/0 (socket says 172.21.15.16:45946) 2023-05-20T01:22:16.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.753+0000 7fb81dd9b700 1 -- 172.21.15.16:0/1027298043 learned_addr learned my addr 172.21.15.16:0/1027298043 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:22:16.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.753+0000 7fb81ed9d700 1 -- 172.21.15.16:0/1027298043 >> v1:172.21.15.132:6789/0 conn(0x7fb8200cfc50 legacy=0x7fb8200d0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:16.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.753+0000 7fb81ed9d700 1 -- 172.21.15.16:0/1027298043 >> v1:172.21.15.16:6789/0 conn(0x7fb8200b4750 legacy=0x7fb8200cd970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:16.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.753+0000 7fb81d59a700 1 -- 172.21.15.16:0/1027298043 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2562583489 0 0) 0x7fb8200b3b50 con 0x7fb8200b4750 2023-05-20T01:22:16.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.753+0000 7fb81ed9d700 1 -- 172.21.15.16:0/1027298043 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8200d3030 con 0x7fb8200d2c10 2023-05-20T01:22:16.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.753+0000 7fb81ed9d700 1 --2- 172.21.15.16:0/1027298043 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8200d2c10 0x7fb8200ce080 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fb818009d90 tx=0x7fb81800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45a412ac16fcb7c0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:16.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.754+0000 7fb81d59a700 1 -- 172.21.15.16:0/1027298043 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb81800e070 con 0x7fb8200d2c10 2023-05-20T01:22:16.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.754+0000 7fb81d59a700 1 -- 172.21.15.16:0/1027298043 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb81800bbd0 con 0x7fb8200d2c10 2023-05-20T01:22:16.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.754+0000 7fb81d59a700 1 -- 172.21.15.16:0/1027298043 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb818003a20 con 0x7fb8200d2c10 2023-05-20T01:22:16.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.754+0000 7fb824b95700 1 -- 172.21.15.16:0/1027298043 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8200d2c10 msgr2=0x7fb8200ce080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:16.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.754+0000 7fb824b95700 1 --2- 172.21.15.16:0/1027298043 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8200d2c10 0x7fb8200ce080 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fb818009d90 tx=0x7fb81800b330 comp rx=0 tx=0).stop 2023-05-20T01:22:16.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.754+0000 7fb824b95700 1 -- 172.21.15.16:0/1027298043 shutdown_connections 2023-05-20T01:22:16.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.754+0000 7fb824b95700 1 --2- 172.21.15.16:0/1027298043 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8200d2c10 0x7fb8200ce080 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:16.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.754+0000 7fb824b95700 1 -- 172.21.15.16:0/1027298043 >> 172.21.15.16:0/1027298043 conn(0x7fb82009c590 msgr2=0x7fb8200d3240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:16.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.754+0000 7fb824b95700 1 -- 172.21.15.16:0/1027298043 shutdown_connections 2023-05-20T01:22:16.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.754+0000 7fb824b95700 1 -- 172.21.15.16:0/1027298043 wait complete. 2023-05-20T01:22:16.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.756+0000 7fb824b95700 1 Processor -- start 2023-05-20T01:22:16.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.756+0000 7fb824b95700 1 -- start start 2023-05-20T01:22:16.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.757+0000 7fb824b95700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb8200b4750 0x7fb8200c9100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:16.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.757+0000 7fb824b95700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8200cfc50 0x7fb8200b0f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:16.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.757+0000 7fb824b95700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb8200d2c10 0x7fb820159b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:16.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.757+0000 7fb824b95700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8200b1770 con 0x7fb8200cfc50 2023-05-20T01:22:16.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.757+0000 7fb824b95700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8200c8c70 con 0x7fb8200d2c10 2023-05-20T01:22:16.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.757+0000 7fb824b95700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8200b8840 con 0x7fb8200b4750 2023-05-20T01:22:16.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.757+0000 7fb81dd9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8200cfc50 0x7fb8200b0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:16.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.757+0000 7fb81dd9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8200cfc50 0x7fb8200b0f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34424/0 (socket says 172.21.15.16:34424) 2023-05-20T01:22:16.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.757+0000 7fb81dd9b700 1 -- 172.21.15.16:0/462329348 learned_addr learned my addr 172.21.15.16:0/462329348 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:22:16.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.757+0000 7fb81e59c700 1 --2- 172.21.15.16:0/462329348 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb8200b4750 0x7fb8200c9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:16.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.757+0000 7fb81ed9d700 1 --2- 172.21.15.16:0/462329348 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb8200d2c10 0x7fb820159b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:16.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.757+0000 7fb81dd9b700 1 -- 172.21.15.16:0/462329348 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb8200b4750 msgr2=0x7fb8200c9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:16.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.757+0000 7fb81dd9b700 1 --2- 172.21.15.16:0/462329348 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb8200b4750 0x7fb8200c9100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:16.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.757+0000 7fb81dd9b700 1 -- 172.21.15.16:0/462329348 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb8200d2c10 msgr2=0x7fb820159b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:16.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.757+0000 7fb81dd9b700 1 --2- 172.21.15.16:0/462329348 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb8200d2c10 0x7fb820159b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:16.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.757+0000 7fb81dd9b700 1 -- 172.21.15.16:0/462329348 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb818009a70 con 0x7fb8200cfc50 2023-05-20T01:22:16.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.757+0000 7fb81ed9d700 1 --2- 172.21.15.16:0/462329348 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb8200d2c10 0x7fb820159b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:22:16.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.758+0000 7fb81dd9b700 1 --2- 172.21.15.16:0/462329348 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8200cfc50 0x7fb8200b0f40 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fb81400c550 tx=0x7fb81400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:16.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.758+0000 7fb80effd700 1 -- 172.21.15.16:0/462329348 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb814004c80 con 0x7fb8200cfc50 2023-05-20T01:22:16.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.758+0000 7fb80effd700 1 -- 172.21.15.16:0/462329348 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb814004e00 con 0x7fb8200cfc50 2023-05-20T01:22:16.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.758+0000 7fb824b95700 1 -- 172.21.15.16:0/462329348 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8200b4380 con 0x7fb8200cfc50 2023-05-20T01:22:16.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.758+0000 7fb80effd700 1 -- 172.21.15.16:0/462329348 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb814007310 con 0x7fb8200cfc50 2023-05-20T01:22:16.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.758+0000 7fb824b95700 1 -- 172.21.15.16:0/462329348 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb82015d0b0 con 0x7fb8200cfc50 2023-05-20T01:22:16.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.759+0000 7fb824b95700 1 -- 172.21.15.16:0/462329348 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8200d4000 con 0x7fb8200cfc50 2023-05-20T01:22:16.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.759+0000 7fb80effd700 1 -- 172.21.15.16:0/462329348 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb814007470 con 0x7fb8200cfc50 2023-05-20T01:22:16.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.760+0000 7fb80effd700 1 --2- 172.21.15.16:0/462329348 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb80006ace0 0x7fb80006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:16.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.760+0000 7fb80effd700 1 -- 172.21.15.16:0/462329348 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 7103+0+0 (secure 0 0 0) 0x7fb8140895a0 con 0x7fb8200cfc50 2023-05-20T01:22:16.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.760+0000 7fb81e59c700 1 --2- 172.21.15.16:0/462329348 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb80006ace0 0x7fb80006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:16.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.760+0000 7fb81e59c700 1 --2- 172.21.15.16:0/462329348 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb80006ace0 0x7fb80006d190 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fb81000a7d0 tx=0x7fb810008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:16.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:16.764+0000 7fb80effd700 1 -- 172.21.15.16:0/462329348 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fb814054240 con 0x7fb8200cfc50 2023-05-20T01:22:17.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.004+0000 7fb824b95700 1 -- 172.21.15.16:0/462329348 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb8200c8de0 con 0x7fb8200cfc50 2023-05-20T01:22:17.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.005+0000 7fb80effd700 1 -- 172.21.15.16:0/462329348 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+4667 (secure 0 0 0) 0x7fb814057860 con 0x7fb8200cfc50 2023-05-20T01:22:17.009 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:22:17.009 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":42,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5824,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/2609276800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":2609276800},{"type":"v1","addr":"172.21.15.132:6834","nonce":2609276800}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32},{"gid":5827,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/29554453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":29554453},{"type":"v1","addr":"172.21.15.132:6835","nonce":29554453}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32},{"gid":6192,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/4090303296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4090303296},{"type":"v1","addr":"172.21.15.16:6836","nonce":4090303296}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32},{"gid":6507,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/3291457857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":3291457857},{"type":"v1","addr":"172.21.15.16:6837","nonce":3291457857}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-20T01:22:14.127671+0000","modified":"2023-05-20T01:22:15.367610+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":71,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[10,11],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-05-20T01:22:17.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.007+0000 7fb824b95700 1 -- 172.21.15.16:0/462329348 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb80006ace0 msgr2=0x7fb80006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:17.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.007+0000 7fb824b95700 1 --2- 172.21.15.16:0/462329348 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb80006ace0 0x7fb80006d190 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fb81000a7d0 tx=0x7fb810008040 comp rx=0 tx=0).stop 2023-05-20T01:22:17.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.007+0000 7fb824b95700 1 -- 172.21.15.16:0/462329348 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8200cfc50 msgr2=0x7fb8200b0f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:17.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.007+0000 7fb824b95700 1 --2- 172.21.15.16:0/462329348 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8200cfc50 0x7fb8200b0f40 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fb81400c550 tx=0x7fb81400c910 comp rx=0 tx=0).stop 2023-05-20T01:22:17.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.007+0000 7fb824b95700 1 -- 172.21.15.16:0/462329348 shutdown_connections 2023-05-20T01:22:17.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.007+0000 7fb824b95700 1 --2- 172.21.15.16:0/462329348 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb8200d2c10 0x7fb820159b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:17.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.007+0000 7fb824b95700 1 --2- 172.21.15.16:0/462329348 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb80006ace0 0x7fb80006d190 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:17.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.007+0000 7fb824b95700 1 --2- 172.21.15.16:0/462329348 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb8200b4750 0x7fb8200c9100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:17.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.007+0000 7fb824b95700 1 --2- 172.21.15.16:0/462329348 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8200cfc50 0x7fb8200b0f40 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:17.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.007+0000 7fb824b95700 1 -- 172.21.15.16:0/462329348 >> 172.21.15.16:0/462329348 conn(0x7fb82009c590 msgr2=0x7fb8200d1f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:17.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.007+0000 7fb824b95700 1 -- 172.21.15.16:0/462329348 shutdown_connections 2023-05-20T01:22:17.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.007+0000 7fb824b95700 1 -- 172.21.15.16:0/462329348 wait complete. 2023-05-20T01:22:17.013 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 42 2023-05-20T01:22:17.023 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-20T01:22:17.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.144+0000 7f67673af700 1 Processor -- start 2023-05-20T01:22:17.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.144+0000 7f67673af700 1 -- start start 2023-05-20T01:22:17.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.145+0000 7f67673af700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f676006b620 0x7f676006ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:17.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.145+0000 7f67673af700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f67600cb6e0 0x7f67600cbb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:17.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.145+0000 7f67673af700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67600ce3e0 con 0x7f67600d1000 2023-05-20T01:22:17.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.145+0000 7f67673af700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67600cdab0 con 0x7f676006b620 2023-05-20T01:22:17.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.145+0000 7f67673af700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67600cdd90 con 0x7f67600cb6e0 2023-05-20T01:22:17.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.145+0000 7f676514b700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f67600d1000 0x7f67600d13d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:45958/0 (socket says 172.21.15.16:45958) 2023-05-20T01:22:17.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.145+0000 7f676514b700 1 -- 172.21.15.16:0/620716640 learned_addr learned my addr 172.21.15.16:0/620716640 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:22:17.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.145+0000 7f676594c700 1 --2- 172.21.15.16:0/620716640 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f67600cb6e0 0x7f67600cbb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:17.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.145+0000 7f676494a700 1 --2- 172.21.15.16:0/620716640 >> v2:172.21.15.132:3300/0 conn(0x7f676006b620 0x7f676006ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:17.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.145+0000 7f6757fff700 1 -- 172.21.15.16:0/620716640 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 193411323 0 0) 0x7f67600ce3e0 con 0x7f67600d1000 2023-05-20T01:22:17.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.145+0000 7f6757fff700 1 -- 172.21.15.16:0/620716640 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f674c0035b0 con 0x7f67600d1000 2023-05-20T01:22:17.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.146+0000 7f6757fff700 1 -- 172.21.15.16:0/620716640 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3145340170 0 0) 0x7f674c0035b0 con 0x7f67600d1000 2023-05-20T01:22:17.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.146+0000 7f6757fff700 1 -- 172.21.15.16:0/620716640 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f67600cb6e0 msgr2=0x7f67600cbb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:17.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.146+0000 7f6757fff700 1 --2- 172.21.15.16:0/620716640 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f67600cb6e0 0x7f67600cbb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:17.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.146+0000 7f6757fff700 1 -- 172.21.15.16:0/620716640 >> v2:172.21.15.132:3300/0 conn(0x7f676006b620 msgr2=0x7f676006ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:17.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.146+0000 7f6757fff700 1 --2- 172.21.15.16:0/620716640 >> v2:172.21.15.132:3300/0 conn(0x7f676006b620 0x7f676006ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:17.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.146+0000 7f6757fff700 1 -- 172.21.15.16:0/620716640 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67600ce070 con 0x7f67600d1000 2023-05-20T01:22:17.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.146+0000 7f6757fff700 1 -- 172.21.15.16:0/620716640 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7f6750001670 con 0x7f67600d1000 2023-05-20T01:22:17.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.146+0000 7f6757fff700 1 -- 172.21.15.16:0/620716640 >> v1:172.21.15.16:6789/0 conn(0x7f67600d1000 legacy=0x7f67600d13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:17.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.146+0000 7f6757fff700 1 --2- 172.21.15.16:0/620716640 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f67600cb6e0 0x7f674c003a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:17.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.146+0000 7f6757fff700 1 --2- 172.21.15.16:0/620716640 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f674c003ff0 0x7f674c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:17.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.146+0000 7f6757fff700 1 --2- 172.21.15.16:0/620716640 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f674c004d90 0x7f674c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:17.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.146+0000 7f6757fff700 1 -- 172.21.15.16:0/620716640 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67600cdab0 con 0x7f674c003ff0 2023-05-20T01:22:17.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.146+0000 7f6757fff700 1 -- 172.21.15.16:0/620716640 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67600cdd90 con 0x7f674c004d90 2023-05-20T01:22:17.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.146+0000 7f6757fff700 1 -- 172.21.15.16:0/620716640 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f674c00ba10 con 0x7f67600cb6e0 2023-05-20T01:22:17.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.146+0000 7f676494a700 1 --2- 172.21.15.16:0/620716640 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f674c003ff0 0x7f674c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:17.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.146+0000 7f676514b700 1 --2- 172.21.15.16:0/620716640 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f67600cb6e0 0x7f674c003a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:17.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.146+0000 7f676594c700 1 --2- 172.21.15.16:0/620716640 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f674c004d90 0x7f674c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:17.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.146+0000 7f676494a700 1 -- 172.21.15.16:0/620716640 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f67600cb6e0 msgr2=0x7f674c003a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:17.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.146+0000 7f676494a700 1 --2- 172.21.15.16:0/620716640 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f67600cb6e0 0x7f674c003a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:17.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.146+0000 7f676494a700 1 -- 172.21.15.16:0/620716640 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f674c004d90 msgr2=0x7f674c005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:17.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.146+0000 7f676494a700 1 --2- 172.21.15.16:0/620716640 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f674c004d90 0x7f674c005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:17.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.146+0000 7f676494a700 1 -- 172.21.15.16:0/620716640 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f674c0035b0 con 0x7f674c003ff0 2023-05-20T01:22:17.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.146+0000 7f676594c700 1 --2- 172.21.15.16:0/620716640 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f674c004d90 0x7f674c005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:22:17.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.146+0000 7f676494a700 1 --2- 172.21.15.16:0/620716640 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f674c003ff0 0x7f674c004850 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f675c009b90 tx=0x7f674c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6565f42c7ba4f20 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:17.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.147+0000 7f6757fff700 1 -- 172.21.15.16:0/620716640 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f675c00e6c0 con 0x7f674c003ff0 2023-05-20T01:22:17.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.147+0000 7f6757fff700 1 -- 172.21.15.16:0/620716640 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f675c00e820 con 0x7f674c003ff0 2023-05-20T01:22:17.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.147+0000 7f6757fff700 1 -- 172.21.15.16:0/620716640 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f675c00eb90 con 0x7f674c003ff0 2023-05-20T01:22:17.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.147+0000 7f67673af700 1 -- 172.21.15.16:0/620716640 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f674c003ff0 msgr2=0x7f674c004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:17.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.147+0000 7f67673af700 1 --2- 172.21.15.16:0/620716640 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f674c003ff0 0x7f674c004850 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f675c009b90 tx=0x7f674c005880 comp rx=0 tx=0).stop 2023-05-20T01:22:17.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.147+0000 7f676594c700 1 -- 172.21.15.16:0/620716640 reap_dead start 2023-05-20T01:22:17.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.147+0000 7f67673af700 1 -- 172.21.15.16:0/620716640 shutdown_connections 2023-05-20T01:22:17.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.147+0000 7f67673af700 1 -- 172.21.15.16:0/620716640 >> 172.21.15.16:0/620716640 conn(0x7f676006ae30 msgr2=0x7f676009a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:17.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.147+0000 7f67673af700 1 -- 172.21.15.16:0/620716640 shutdown_connections 2023-05-20T01:22:17.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.147+0000 7f67673af700 1 -- 172.21.15.16:0/620716640 wait complete. 2023-05-20T01:22:17.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.148+0000 7f67673af700 1 Processor -- start 2023-05-20T01:22:17.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.148+0000 7f67673af700 1 -- start start 2023-05-20T01:22:17.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.149+0000 7f67673af700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f674c003ff0 0x7f676015a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:17.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.149+0000 7f67673af700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f67600c8a50 0x7f67600c8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:17.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.149+0000 7f67673af700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67600cf750 0x7f67600cfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:17.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.149+0000 7f67673af700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67600d0130 con 0x7f67600cf750 2023-05-20T01:22:17.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.149+0000 7f67673af700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f676015ad30 con 0x7f674c003ff0 2023-05-20T01:22:17.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.149+0000 7f67673af700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f676015c450 con 0x7f67600c8a50 2023-05-20T01:22:17.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.149+0000 7f676594c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67600cf750 0x7f67600cfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:17.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.149+0000 7f676594c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67600cf750 0x7f67600cfbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34446/0 (socket says 172.21.15.16:34446) 2023-05-20T01:22:17.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.149+0000 7f676594c700 1 -- 172.21.15.16:0/2535198805 learned_addr learned my addr 172.21.15.16:0/2535198805 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:22:17.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.149+0000 7f676514b700 1 --2- 172.21.15.16:0/2535198805 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f674c003ff0 0x7f676015a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:17.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.149+0000 7f676494a700 1 --2- 172.21.15.16:0/2535198805 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f67600c8a50 0x7f67600c8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:17.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.150+0000 7f676594c700 1 -- 172.21.15.16:0/2535198805 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f67600c8a50 msgr2=0x7f67600c8ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:17.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.150+0000 7f676594c700 1 --2- 172.21.15.16:0/2535198805 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f67600c8a50 0x7f67600c8ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:17.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.150+0000 7f676594c700 1 -- 172.21.15.16:0/2535198805 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f674c003ff0 msgr2=0x7f676015a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:17.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.150+0000 7f676594c700 1 --2- 172.21.15.16:0/2535198805 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f674c003ff0 0x7f676015a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:17.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.150+0000 7f676594c700 1 -- 172.21.15.16:0/2535198805 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67600ce870 con 0x7f67600cf750 2023-05-20T01:22:17.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.150+0000 7f676594c700 1 --2- 172.21.15.16:0/2535198805 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67600cf750 0x7f67600cfbf0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f6760071fe0 tx=0x7f67580104a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:17.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.150+0000 7f6755ffb700 1 -- 172.21.15.16:0/2535198805 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f675801b070 con 0x7f67600cf750 2023-05-20T01:22:17.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.150+0000 7f6755ffb700 1 -- 172.21.15.16:0/2535198805 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f675800f040 con 0x7f67600cf750 2023-05-20T01:22:17.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.150+0000 7f6755ffb700 1 -- 172.21.15.16:0/2535198805 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f675800e050 con 0x7f67600cf750 2023-05-20T01:22:17.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.150+0000 7f67673af700 1 -- 172.21.15.16:0/2535198805 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67600b28b0 con 0x7f67600cf750 2023-05-20T01:22:17.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.150+0000 7f67673af700 1 -- 172.21.15.16:0/2535198805 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67600cd390 con 0x7f67600cf750 2023-05-20T01:22:17.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.151+0000 7f67673af700 1 -- 172.21.15.16:0/2535198805 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67600747b0 con 0x7f67600cf750 2023-05-20T01:22:17.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.152+0000 7f6755ffb700 1 -- 172.21.15.16:0/2535198805 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6758010d20 con 0x7f67600cf750 2023-05-20T01:22:17.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.152+0000 7f6755ffb700 1 --2- 172.21.15.16:0/2535198805 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f674406ace0 0x7f674406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:17.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.152+0000 7f6755ffb700 1 -- 172.21.15.16:0/2535198805 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 7103+0+0 (secure 0 0 0) 0x7f6758089770 con 0x7f67600cf750 2023-05-20T01:22:17.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.152+0000 7f676514b700 1 --2- 172.21.15.16:0/2535198805 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f674406ace0 0x7f674406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:17.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.152+0000 7f676514b700 1 --2- 172.21.15.16:0/2535198805 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f674406ace0 0x7f674406d190 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f6750001cf0 tx=0x7f6750007980 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:17.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.156+0000 7f6755ffb700 1 -- 172.21.15.16:0/2535198805 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f67580544a0 con 0x7f67600cf750 2023-05-20T01:22:17.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.398+0000 7f67673af700 1 -- 172.21.15.16:0/2535198805 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f67600008d0 con 0x7f67600cf750 2023-05-20T01:22:17.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.398+0000 7f6755ffb700 1 -- 172.21.15.16:0/2535198805 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+4667 (secure 0 0 0) 0x7f6758057ac0 con 0x7f67600cf750 2023-05-20T01:22:17.403 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:22:17.403 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":42,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5824,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/2609276800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":2609276800},{"type":"v1","addr":"172.21.15.132:6834","nonce":2609276800}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32},{"gid":5827,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/29554453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":29554453},{"type":"v1","addr":"172.21.15.132:6835","nonce":29554453}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32},{"gid":6192,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/4090303296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4090303296},{"type":"v1","addr":"172.21.15.16:6836","nonce":4090303296}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32},{"gid":6507,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/3291457857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":3291457857},{"type":"v1","addr":"172.21.15.16:6837","nonce":3291457857}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-20T01:22:14.127671+0000","modified":"2023-05-20T01:22:15.367610+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":71,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[10,11],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-05-20T01:22:17.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.400+0000 7f67673af700 1 -- 172.21.15.16:0/2535198805 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f674406ace0 msgr2=0x7f674406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:17.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.400+0000 7f67673af700 1 --2- 172.21.15.16:0/2535198805 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f674406ace0 0x7f674406d190 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f6750001cf0 tx=0x7f6750007980 comp rx=0 tx=0).stop 2023-05-20T01:22:17.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.400+0000 7f67673af700 1 -- 172.21.15.16:0/2535198805 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67600cf750 msgr2=0x7f67600cfbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:17.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.400+0000 7f67673af700 1 --2- 172.21.15.16:0/2535198805 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67600cf750 0x7f67600cfbf0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f6760071fe0 tx=0x7f67580104a0 comp rx=0 tx=0).stop 2023-05-20T01:22:17.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.400+0000 7f67673af700 1 -- 172.21.15.16:0/2535198805 shutdown_connections 2023-05-20T01:22:17.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.400+0000 7f67673af700 1 --2- 172.21.15.16:0/2535198805 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67600cf750 0x7f67600cfbf0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:17.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.400+0000 7f67673af700 1 --2- 172.21.15.16:0/2535198805 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f674c003ff0 0x7f676015a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:17.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.400+0000 7f67673af700 1 --2- 172.21.15.16:0/2535198805 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f674406ace0 0x7f674406d190 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:17.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.400+0000 7f67673af700 1 --2- 172.21.15.16:0/2535198805 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f67600c8a50 0x7f67600c8ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:17.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.400+0000 7f67673af700 1 -- 172.21.15.16:0/2535198805 >> 172.21.15.16:0/2535198805 conn(0x7f676006ae30 msgr2=0x7f676006fbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:17.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.401+0000 7f67673af700 1 -- 172.21.15.16:0/2535198805 shutdown_connections 2023-05-20T01:22:17.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.401+0000 7f67673af700 1 -- 172.21.15.16:0/2535198805 wait complete. 2023-05-20T01:22:17.407 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 42 2023-05-20T01:22:17.417 DEBUG:teuthology.orchestra.run.smithi016:> cephfs-table-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 cephfs:0 reset session 2023-05-20T01:22:17.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.456+0000 7f470a36a2c0 1 -- start start 2023-05-20T01:22:17.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.456+0000 7f470a36a2c0 1 Processor -- start 2023-05-20T01:22:17.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.457+0000 7f470a36a2c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55dc8c993000 0x55dc8ca2c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:17.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.457+0000 7f470a36a2c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dc8c992c00 0x55dc8ca2cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:17.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.457+0000 7f470a36a2c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc8c992800 0x55dc8ca2d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:17.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.457+0000 7f470a36a2c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc8c9cfcc0 con 0x55dc8c992800 2023-05-20T01:22:17.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.457+0000 7f470a36a2c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc8c9cf400 con 0x55dc8c992c00 2023-05-20T01:22:17.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.457+0000 7f470a36a2c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55dc8c9cfe00 con 0x55dc8c993000 2023-05-20T01:22:17.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.457+0000 7f4708b17700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc8c992800 0x55dc8ca2d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:17.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.457+0000 7f4708b17700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc8c992800 0x55dc8ca2d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34458/0 (socket says 172.21.15.16:34458) 2023-05-20T01:22:17.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.457+0000 7f4708b17700 1 -- 172.21.15.16:0/387524056 learned_addr learned my addr 172.21.15.16:0/387524056 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:22:17.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.457+0000 7f4707b15700 1 --2- 172.21.15.16:0/387524056 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dc8c992c00 0x55dc8ca2cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:17.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.457+0000 7f4708316700 1 --2- 172.21.15.16:0/387524056 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55dc8c993000 0x55dc8ca2c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:17.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.457+0000 7f4708b17700 1 -- 172.21.15.16:0/387524056 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55dc8c993000 msgr2=0x55dc8ca2c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:17.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.457+0000 7f4708b17700 1 --2- 172.21.15.16:0/387524056 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55dc8c993000 0x55dc8ca2c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:17.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.457+0000 7f4708b17700 1 -- 172.21.15.16:0/387524056 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dc8c992c00 msgr2=0x55dc8ca2cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:17.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.457+0000 7f4708b17700 1 --2- 172.21.15.16:0/387524056 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dc8c992c00 0x55dc8ca2cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:17.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.457+0000 7f4708b17700 1 -- 172.21.15.16:0/387524056 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dc8c9b64e0 con 0x55dc8c992800 2023-05-20T01:22:17.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.457+0000 7f4708316700 1 --2- 172.21.15.16:0/387524056 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55dc8c993000 0x55dc8ca2c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:22:17.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.457+0000 7f4708b17700 1 --2- 172.21.15.16:0/387524056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc8c992800 0x55dc8ca2d080 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x55dc8d8060f0 tx=0x55dc8d80c060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4460dbccd47a7b59 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:17.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.457+0000 7f4705b11700 1 -- 172.21.15.16:0/387524056 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55dc8c9c7080 con 0x55dc8c992800 2023-05-20T01:22:17.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.457+0000 7f4705b11700 1 -- 172.21.15.16:0/387524056 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55dc8c9e5980 con 0x55dc8c992800 2023-05-20T01:22:17.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.457+0000 7f4705b11700 1 -- 172.21.15.16:0/387524056 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55dc8c9c7600 con 0x55dc8c992800 2023-05-20T01:22:17.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.457+0000 7f470a36a2c0 1 -- 172.21.15.16:0/387524056 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dc8c9b7520 con 0x55dc8c992800 2023-05-20T01:22:17.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.457+0000 7f470a36a2c0 1 -- 172.21.15.16:0/387524056 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55dc8c9b7380 con 0x55dc8c992800 2023-05-20T01:22:17.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.457+0000 7f470a36a2c0 4 waiting for MDS map... 2023-05-20T01:22:17.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.458+0000 7f4705b11700 1 -- 172.21.15.16:0/387524056 <== mon.0 v2:172.21.15.16:3300/0 4 ==== osd_map(71..71 src has 1..71) v4 ==== 7103+0+0 (secure 0 0 0) 0x55dc8c9d4c40 con 0x55dc8c992800 2023-05-20T01:22:17.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.458+0000 7f4705b11700 1 -- 172.21.15.16:0/387524056 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 42) v1 ==== 3126+0+0 (secure 0 0 0) 0x55dc8c9d8f00 con 0x55dc8c992800 2023-05-20T01:22:17.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.458+0000 7f470a36a2c0 4 Got MDS map 42 2023-05-20T01:22:17.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.458+0000 7f470a36a2c0 10 main: main 2023-05-20T01:22:17.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.458+0000 7f470a36a2c0 4 main: connecting to RADOS... 2023-05-20T01:22:17.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.458+0000 7f470a36a2c0 1 Processor -- start 2023-05-20T01:22:17.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.458+0000 7f470a36a2c0 1 -- start start 2023-05-20T01:22:17.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.459+0000 7f470a36a2c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dc8c993c00 0x55dc8ca2d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:17.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.459+0000 7f470a36a2c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc8cca2000 0x55dc8ca2db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:17.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.459+0000 7f470a36a2c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55dc8cca2400 0x55dc8ca2e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:17.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.459+0000 7f470a36a2c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc8c9cea00 con 0x55dc8cca2000 2023-05-20T01:22:17.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.459+0000 7f470a36a2c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc8c9cf400 con 0x55dc8c993c00 2023-05-20T01:22:17.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.459+0000 7f470a36a2c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55dc8c9cfcc0 con 0x55dc8cca2400 2023-05-20T01:22:17.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.459+0000 7f4708316700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc8cca2000 0x55dc8ca2db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:17.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.459+0000 7f4708316700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc8cca2000 0x55dc8ca2db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34460/0 (socket says 172.21.15.16:34460) 2023-05-20T01:22:17.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.459+0000 7f4708316700 1 -- 172.21.15.16:0/1366127903 learned_addr learned my addr 172.21.15.16:0/1366127903 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:22:17.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.459+0000 7f4707b15700 1 --2- 172.21.15.16:0/1366127903 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dc8c993c00 0x55dc8ca2d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:17.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.459+0000 7f4708316700 1 -- 172.21.15.16:0/1366127903 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55dc8cca2400 msgr2=0x55dc8ca2e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:17.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.459+0000 7f4708316700 1 --2- 172.21.15.16:0/1366127903 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55dc8cca2400 0x55dc8ca2e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:17.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.459+0000 7f4708316700 1 -- 172.21.15.16:0/1366127903 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dc8c993c00 msgr2=0x55dc8ca2d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:17.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.459+0000 7f4708316700 1 --2- 172.21.15.16:0/1366127903 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dc8c993c00 0x55dc8ca2d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:17.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.459+0000 7f4708316700 1 -- 172.21.15.16:0/1366127903 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dc8c9b7d40 con 0x55dc8cca2000 2023-05-20T01:22:17.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.459+0000 7f4707b15700 1 --2- 172.21.15.16:0/1366127903 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dc8c993c00 0x55dc8ca2d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:22:17.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.459+0000 7f4708316700 1 --2- 172.21.15.16:0/1366127903 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc8cca2000 0x55dc8ca2db80 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x55dc8d807650 tx=0x55dc8d830cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33415200bdd8ef2a server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:17.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.459+0000 7f4703b0d700 1 -- 172.21.15.16:0/1366127903 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55dc8c9c7080 con 0x55dc8cca2000 2023-05-20T01:22:17.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.460+0000 7f4703b0d700 1 -- 172.21.15.16:0/1366127903 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55dc8d82cd80 con 0x55dc8cca2000 2023-05-20T01:22:17.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.460+0000 7f4703b0d700 1 -- 172.21.15.16:0/1366127903 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55dc8c9c7600 con 0x55dc8cca2000 2023-05-20T01:22:17.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.460+0000 7f470a36a2c0 1 -- 172.21.15.16:0/1366127903 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc8cca2000 msgr2=0x55dc8ca2db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:17.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.460+0000 7f470a36a2c0 1 --2- 172.21.15.16:0/1366127903 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc8cca2000 0x55dc8ca2db80 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x55dc8d807650 tx=0x55dc8d830cc0 comp rx=0 tx=0).stop 2023-05-20T01:22:17.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.460+0000 7f470a36a2c0 1 -- 172.21.15.16:0/1366127903 shutdown_connections 2023-05-20T01:22:17.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.460+0000 7f470a36a2c0 1 --2- 172.21.15.16:0/1366127903 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55dc8cca2400 0x55dc8ca2e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:17.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.460+0000 7f470a36a2c0 1 --2- 172.21.15.16:0/1366127903 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dc8c993c00 0x55dc8ca2d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:17.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.460+0000 7f470a36a2c0 1 --2- 172.21.15.16:0/1366127903 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc8cca2000 0x55dc8ca2db80 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:17.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.460+0000 7f470a36a2c0 1 -- 172.21.15.16:0/1366127903 >> 172.21.15.16:0/1366127903 conn(0x55dc8c992400 msgr2=0x55dc8ca6f800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:17.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.460+0000 7f470a36a2c0 1 -- 172.21.15.16:0/1366127903 shutdown_connections 2023-05-20T01:22:17.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.460+0000 7f470a36a2c0 1 -- 172.21.15.16:0/1366127903 wait complete. 2023-05-20T01:22:17.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.460+0000 7f470a36a2c0 1 Processor -- start 2023-05-20T01:22:17.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.461+0000 7f470a36a2c0 1 -- start start 2023-05-20T01:22:17.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.461+0000 7f470a36a2c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc8c992400 0x55dc8ca2f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:17.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.461+0000 7f470a36a2c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55dc8ccc0000 0x55dc8ca2ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:17.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.461+0000 7f470a36a2c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dc8ccc0400 0x55dc8ca2e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:17.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.461+0000 7f470a36a2c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc8c9ceb40 con 0x55dc8c992400 2023-05-20T01:22:17.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.461+0000 7f470a36a2c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc8c9cec80 con 0x55dc8ccc0400 2023-05-20T01:22:17.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.461+0000 7f470a36a2c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55dc8c9ce000 con 0x55dc8ccc0000 2023-05-20T01:22:17.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.461+0000 7f4707b15700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc8c992400 0x55dc8ca2f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:17.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.461+0000 7f4707b15700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc8c992400 0x55dc8ca2f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34466/0 (socket says 172.21.15.16:34466) 2023-05-20T01:22:17.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.461+0000 7f4707b15700 1 -- 172.21.15.16:0/3854152159 learned_addr learned my addr 172.21.15.16:0/3854152159 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:22:17.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.461+0000 7f4708316700 1 --2- 172.21.15.16:0/3854152159 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55dc8ccc0000 0x55dc8ca2ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:17.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.461+0000 7f4707b15700 1 --2- 172.21.15.16:0/3854152159 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dc8ccc0400 0x55dc8ca2e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:17.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.462+0000 7f4707b15700 1 -- 172.21.15.16:0/3854152159 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55dc8ccc0000 msgr2=0x55dc8ca2ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:17.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.462+0000 7f4707b15700 1 --2- 172.21.15.16:0/3854152159 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55dc8ccc0000 0x55dc8ca2ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:17.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.462+0000 7f4707b15700 1 -- 172.21.15.16:0/3854152159 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dc8ccc0400 msgr2=0x55dc8ca2e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:17.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.462+0000 7f4707b15700 1 --2- 172.21.15.16:0/3854152159 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dc8ccc0400 0x55dc8ca2e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:17.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.462+0000 7f4707b15700 1 -- 172.21.15.16:0/3854152159 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dc8ccae1a0 con 0x55dc8c992400 2023-05-20T01:22:17.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.462+0000 7f4707b15700 1 --2- 172.21.15.16:0/3854152159 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc8c992400 0x55dc8ca2f180 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x55dc8cca2780 tx=0x55dc8d830e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:17.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.462+0000 7f470230a700 1 -- 172.21.15.16:0/3854152159 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55dc8c9c7080 con 0x55dc8c992400 2023-05-20T01:22:17.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.462+0000 7f470230a700 1 -- 172.21.15.16:0/3854152159 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55dc8c9e4c00 con 0x55dc8c992400 2023-05-20T01:22:17.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.462+0000 7f470a36a2c0 1 -- 172.21.15.16:0/3854152159 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55dc8ccae000 con 0x55dc8c992400 2023-05-20T01:22:17.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.462+0000 7f470230a700 1 -- 172.21.15.16:0/3854152159 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55dc8c9c7600 con 0x55dc8c992400 2023-05-20T01:22:17.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.462+0000 7f470a36a2c0 1 -- 172.21.15.16:0/3854152159 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dc8ccae340 con 0x55dc8c992400 2023-05-20T01:22:17.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.462+0000 7f470a36a2c0 4 main: resolving pool 9 2023-05-20T01:22:17.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.463+0000 7f470230a700 1 -- 172.21.15.16:0/3854152159 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x55dc8c9f4840 con 0x55dc8c992400 2023-05-20T01:22:17.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.463+0000 7f470230a700 1 --2- 172.21.15.16:0/3854152159 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x55dc8ccd2400 0x55dc8ca2f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:17.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.464+0000 7f470230a700 1 -- 172.21.15.16:0/3854152159 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 7103+0+0 (secure 0 0 0) 0x55dc8c9d4a80 con 0x55dc8c992400 2023-05-20T01:22:17.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.464+0000 7f4708316700 1 --2- 172.21.15.16:0/3854152159 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x55dc8ccd2400 0x55dc8ca2f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:17.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.464+0000 7f470a36a2c0 4 main: creating IoCtx.. 2023-05-20T01:22:17.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.464+0000 7f470a36a2c0 1 --2- 172.21.15.16:0/3854152159 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x55dc8ccc0800 0x55dc8ca2e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:17.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.464+0000 7f470a36a2c0 1 -- 172.21.15.16:0/3854152159 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:1 9.b 9:d0630e4c:::mds0_sessionmap:head [omap-clear,omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x55dc8d828700 con 0x55dc8ccc0800 2023-05-20T01:22:17.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.464+0000 7f4707b15700 1 --2- 172.21.15.16:0/3854152159 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x55dc8ccc0800 0x55dc8ca2e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:17.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.464+0000 7f4708316700 1 --2- 172.21.15.16:0/3854152159 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x55dc8ccd2400 0x55dc8ca2f700 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x55dc8d807650 tx=0x55dc8d830c00 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:17.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.464+0000 7f4707b15700 1 --2- 172.21.15.16:0/3854152159 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x55dc8ccc0800 0x55dc8ca2e100 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:17.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.466+0000 7f4707b15700 1 -- 172.21.15.16:0/3854152159 <== osd.0 v2:172.21.15.16:6824/3923634179 1 ==== osd_op_reply(1 mds0_sessionmap [omap-clear,omap-set-header] v71'5 uv5 ondisk = 0) v8 ==== 201+0+0 (crc 0 0 0) 0x55dc8d7fefc0 con 0x55dc8ccc0800 2023-05-20T01:22:17.484 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-05-20T01:22:17.485 INFO:teuthology.orchestra.run.smithi016.stdout: "0": { 2023-05-20T01:22:17.485 INFO:teuthology.orchestra.run.smithi016.stdout: "data": {}, 2023-05-20T01:22:17.485 INFO:teuthology.orchestra.run.smithi016.stdout: "result": 0 2023-05-20T01:22:17.485 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:22:17.486 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-05-20T01:22:17.486 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:22:17.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.466+0000 7f470a36a2c0 1 -- 172.21.15.16:0/3854152159 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x55dc8ccc0800 msgr2=0x55dc8ca2e100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:17.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.466+0000 7f470a36a2c0 1 --2- 172.21.15.16:0/3854152159 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x55dc8ccc0800 0x55dc8ca2e100 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:17.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.466+0000 7f470a36a2c0 1 -- 172.21.15.16:0/3854152159 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x55dc8ccd2400 msgr2=0x55dc8ca2f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:17.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.466+0000 7f470a36a2c0 1 --2- 172.21.15.16:0/3854152159 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x55dc8ccd2400 0x55dc8ca2f700 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x55dc8d807650 tx=0x55dc8d830c00 comp rx=0 tx=0).stop 2023-05-20T01:22:17.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.466+0000 7f470a36a2c0 1 -- 172.21.15.16:0/3854152159 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc8c992400 msgr2=0x55dc8ca2f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:17.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.466+0000 7f470a36a2c0 1 --2- 172.21.15.16:0/3854152159 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc8c992400 0x55dc8ca2f180 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x55dc8cca2780 tx=0x55dc8d830e40 comp rx=0 tx=0).stop 2023-05-20T01:22:17.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.466+0000 7f4708316700 1 -- 172.21.15.16:0/3854152159 reap_dead start 2023-05-20T01:22:17.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.466+0000 7f470a36a2c0 1 -- 172.21.15.16:0/3854152159 shutdown_connections 2023-05-20T01:22:17.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.466+0000 7f470a36a2c0 1 -- 172.21.15.16:0/3854152159 >> 172.21.15.16:0/3854152159 conn(0x55dc8c993c00 msgr2=0x55dc8ca6f800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:17.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.466+0000 7f470a36a2c0 1 -- 172.21.15.16:0/3854152159 shutdown_connections 2023-05-20T01:22:17.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.466+0000 7f470a36a2c0 1 -- 172.21.15.16:0/3854152159 wait complete. 2023-05-20T01:22:17.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.466+0000 7f470a36a2c0 1 -- 172.21.15.16:0/387524056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc8c992800 msgr2=0x55dc8ca2d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:17.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.466+0000 7f470a36a2c0 1 --2- 172.21.15.16:0/387524056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc8c992800 0x55dc8ca2d080 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x55dc8d8060f0 tx=0x55dc8d80c060 comp rx=0 tx=0).stop 2023-05-20T01:22:17.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.466+0000 7f470a36a2c0 1 -- 172.21.15.16:0/387524056 shutdown_connections 2023-05-20T01:22:17.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.466+0000 7f470a36a2c0 1 --2- 172.21.15.16:0/387524056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc8c992800 0x55dc8ca2d080 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:17.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.466+0000 7f470a36a2c0 1 --2- 172.21.15.16:0/387524056 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55dc8c993000 0x55dc8ca2c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:17.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.466+0000 7f470a36a2c0 1 --2- 172.21.15.16:0/387524056 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dc8c992c00 0x55dc8ca2cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:17.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.466+0000 7f470a36a2c0 1 -- 172.21.15.16:0/387524056 >> 172.21.15.16:0/387524056 conn(0x55dc8c993400 msgr2=0x55dc8ca6f000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:17.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.466+0000 7f470a36a2c0 1 -- 172.21.15.16:0/387524056 shutdown_connections 2023-05-20T01:22:17.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.466+0000 7f470a36a2c0 1 -- 172.21.15.16:0/387524056 wait complete. 2023-05-20T01:22:17.491 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-table-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'cephfs:0', 'reset', 'session'] in time 0:00:00.074131, result: { "0": { "data": {}, "result": 0 } } 2023-05-20T01:22:17.491 DEBUG:teuthology.orchestra.run.smithi016:> cephfs-table-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 cephfs:0 reset snap 2023-05-20T01:22:17.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.565+0000 7f1d644fd2c0 1 -- start start 2023-05-20T01:22:17.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.565+0000 7f1d644fd2c0 1 Processor -- start 2023-05-20T01:22:17.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.566+0000 7f1d644fd2c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55be6df19000 0x55be6dfb2b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:17.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.566+0000 7f1d644fd2c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be6df18c00 0x55be6dfb2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:17.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.566+0000 7f1d644fd2c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55be6df18800 0x55be6dfb3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:17.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.566+0000 7f1d644fd2c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be6df55cc0 con 0x55be6df18c00 2023-05-20T01:22:17.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.566+0000 7f1d644fd2c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be6df55400 con 0x55be6df18800 2023-05-20T01:22:17.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.566+0000 7f1d644fd2c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55be6df55e00 con 0x55be6df19000 2023-05-20T01:22:17.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.566+0000 7f1d61ca8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be6df18c00 0x55be6dfb2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:17.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.566+0000 7f1d61ca8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be6df18c00 0x55be6dfb2580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34484/0 (socket says 172.21.15.16:34484) 2023-05-20T01:22:17.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.566+0000 7f1d61ca8700 1 -- 172.21.15.16:0/1797155793 learned_addr learned my addr 172.21.15.16:0/1797155793 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:22:17.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.566+0000 7f1d62caa700 1 --2- 172.21.15.16:0/1797155793 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55be6df18800 0x55be6dfb3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:17.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.566+0000 7f1d624a9700 1 --2- 172.21.15.16:0/1797155793 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55be6df19000 0x55be6dfb2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:17.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.566+0000 7f1d61ca8700 1 -- 172.21.15.16:0/1797155793 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55be6df19000 msgr2=0x55be6dfb2b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:17.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.566+0000 7f1d61ca8700 1 --2- 172.21.15.16:0/1797155793 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55be6df19000 0x55be6dfb2b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:17.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.566+0000 7f1d61ca8700 1 -- 172.21.15.16:0/1797155793 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55be6df18800 msgr2=0x55be6dfb3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:17.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.566+0000 7f1d61ca8700 1 --2- 172.21.15.16:0/1797155793 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55be6df18800 0x55be6dfb3080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:17.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.566+0000 7f1d61ca8700 1 -- 172.21.15.16:0/1797155793 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be6df3d520 con 0x55be6df18c00 2023-05-20T01:22:17.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.566+0000 7f1d62caa700 1 --2- 172.21.15.16:0/1797155793 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55be6df18800 0x55be6dfb3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:22:17.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.567+0000 7f1d61ca8700 1 --2- 172.21.15.16:0/1797155793 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be6df18c00 0x55be6dfb2580 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x55be6ed886c0 tx=0x55be6ece4ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ed4a40fd96c4ff6 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:17.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.567+0000 7f1d5fca4700 1 -- 172.21.15.16:0/1797155793 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55be6df4d600 con 0x55be6df18c00 2023-05-20T01:22:17.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.567+0000 7f1d5fca4700 1 -- 172.21.15.16:0/1797155793 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55be6df6b800 con 0x55be6df18c00 2023-05-20T01:22:17.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.567+0000 7f1d5fca4700 1 -- 172.21.15.16:0/1797155793 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55be6df4d1e0 con 0x55be6df18c00 2023-05-20T01:22:17.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.567+0000 7f1d644fd2c0 1 -- 172.21.15.16:0/1797155793 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55be6df3d380 con 0x55be6df18c00 2023-05-20T01:22:17.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.567+0000 7f1d644fd2c0 1 -- 172.21.15.16:0/1797155793 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55be6df3dd40 con 0x55be6df18c00 2023-05-20T01:22:17.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.567+0000 7f1d644fd2c0 4 waiting for MDS map... 2023-05-20T01:22:17.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.567+0000 7f1d5fca4700 1 -- 172.21.15.16:0/1797155793 <== mon.0 v2:172.21.15.16:3300/0 4 ==== osd_map(71..71 src has 1..71) v4 ==== 7103+0+0 (secure 0 0 0) 0x55be6df5ac40 con 0x55be6df18c00 2023-05-20T01:22:17.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.567+0000 7f1d5fca4700 1 -- 172.21.15.16:0/1797155793 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 42) v1 ==== 3126+0+0 (secure 0 0 0) 0x55be6edb0000 con 0x55be6df18c00 2023-05-20T01:22:17.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.567+0000 7f1d644fd2c0 4 Got MDS map 42 2023-05-20T01:22:17.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.567+0000 7f1d644fd2c0 10 main: main 2023-05-20T01:22:17.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.567+0000 7f1d644fd2c0 4 main: connecting to RADOS... 2023-05-20T01:22:17.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.568+0000 7f1d644fd2c0 1 Processor -- start 2023-05-20T01:22:17.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.568+0000 7f1d644fd2c0 1 -- start start 2023-05-20T01:22:17.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.568+0000 7f1d644fd2c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be6df19c00 0x55be6dfb3600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:17.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.568+0000 7f1d644fd2c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55be6e230000 0x55be6dfb3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:17.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.568+0000 7f1d644fd2c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55be6e230400 0x55be6dfb4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:17.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.568+0000 7f1d644fd2c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be6df54a00 con 0x55be6df19c00 2023-05-20T01:22:17.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.568+0000 7f1d644fd2c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be6df55400 con 0x55be6e230000 2023-05-20T01:22:17.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.568+0000 7f1d644fd2c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55be6df54b40 con 0x55be6e230400 2023-05-20T01:22:17.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.568+0000 7f1d62caa700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be6df19c00 0x55be6dfb3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:17.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.568+0000 7f1d62caa700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be6df19c00 0x55be6dfb3600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34488/0 (socket says 172.21.15.16:34488) 2023-05-20T01:22:17.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.568+0000 7f1d62caa700 1 -- 172.21.15.16:0/3587061366 learned_addr learned my addr 172.21.15.16:0/3587061366 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:22:17.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.569+0000 7f1d624a9700 1 --2- 172.21.15.16:0/3587061366 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55be6e230000 0x55be6dfb3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:17.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.569+0000 7f1d61ca8700 1 --2- 172.21.15.16:0/3587061366 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55be6e230400 0x55be6dfb4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:17.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.569+0000 7f1d62caa700 1 -- 172.21.15.16:0/3587061366 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55be6e230400 msgr2=0x55be6dfb4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:17.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.569+0000 7f1d62caa700 1 --2- 172.21.15.16:0/3587061366 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55be6e230400 0x55be6dfb4100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:17.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.569+0000 7f1d62caa700 1 -- 172.21.15.16:0/3587061366 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55be6e230000 msgr2=0x55be6dfb3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:17.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.569+0000 7f1d62caa700 1 --2- 172.21.15.16:0/3587061366 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55be6e230000 0x55be6dfb3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:17.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.569+0000 7f1d62caa700 1 -- 172.21.15.16:0/3587061366 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be6df3d6c0 con 0x55be6df19c00 2023-05-20T01:22:17.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.569+0000 7f1d624a9700 1 --2- 172.21.15.16:0/3587061366 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55be6e230000 0x55be6dfb3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:22:17.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.569+0000 7f1d62caa700 1 --2- 172.21.15.16:0/3587061366 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be6df19c00 0x55be6dfb3600 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x55be6ed89680 tx=0x55be6edbcc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fbbe8c0546c2b59 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:17.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.569+0000 7f1d5dca0700 1 -- 172.21.15.16:0/3587061366 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55be6df4d600 con 0x55be6df19c00 2023-05-20T01:22:17.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.569+0000 7f1d5dca0700 1 -- 172.21.15.16:0/3587061366 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55be6df6be00 con 0x55be6df19c00 2023-05-20T01:22:17.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.569+0000 7f1d5dca0700 1 -- 172.21.15.16:0/3587061366 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55be6df4d1e0 con 0x55be6df19c00 2023-05-20T01:22:17.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.569+0000 7f1d644fd2c0 1 -- 172.21.15.16:0/3587061366 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be6df19c00 msgr2=0x55be6dfb3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:17.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.569+0000 7f1d644fd2c0 1 --2- 172.21.15.16:0/3587061366 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be6df19c00 0x55be6dfb3600 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x55be6ed89680 tx=0x55be6edbcc00 comp rx=0 tx=0).stop 2023-05-20T01:22:17.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.569+0000 7f1d644fd2c0 1 -- 172.21.15.16:0/3587061366 shutdown_connections 2023-05-20T01:22:17.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.569+0000 7f1d644fd2c0 1 --2- 172.21.15.16:0/3587061366 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55be6e230400 0x55be6dfb4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:17.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.569+0000 7f1d644fd2c0 1 --2- 172.21.15.16:0/3587061366 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be6df19c00 0x55be6dfb3600 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:17.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.569+0000 7f1d644fd2c0 1 --2- 172.21.15.16:0/3587061366 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55be6e230000 0x55be6dfb3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:17.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.569+0000 7f1d644fd2c0 1 -- 172.21.15.16:0/3587061366 >> 172.21.15.16:0/3587061366 conn(0x55be6df18400 msgr2=0x55be6dff5800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:17.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.569+0000 7f1d644fd2c0 1 -- 172.21.15.16:0/3587061366 shutdown_connections 2023-05-20T01:22:17.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.569+0000 7f1d644fd2c0 1 -- 172.21.15.16:0/3587061366 wait complete. 2023-05-20T01:22:17.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.570+0000 7f1d644fd2c0 1 Processor -- start 2023-05-20T01:22:17.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.570+0000 7f1d644fd2c0 1 -- start start 2023-05-20T01:22:17.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.571+0000 7f1d644fd2c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be6df18400 0x55be6dfb3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:17.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.571+0000 7f1d644fd2c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55be6e248000 0x55be6dfb3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:17.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.571+0000 7f1d644fd2c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55be6e248400 0x55be6dfb4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:17.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.571+0000 7f1d644fd2c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be6df54c80 con 0x55be6df18400 2023-05-20T01:22:17.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.571+0000 7f1d644fd2c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be6df54000 con 0x55be6e248400 2023-05-20T01:22:17.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.571+0000 7f1d644fd2c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55be6df55180 con 0x55be6e248000 2023-05-20T01:22:17.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.571+0000 7f1d62caa700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be6df18400 0x55be6dfb3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:17.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.571+0000 7f1d62caa700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be6df18400 0x55be6dfb3600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34494/0 (socket says 172.21.15.16:34494) 2023-05-20T01:22:17.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.571+0000 7f1d62caa700 1 -- 172.21.15.16:0/1837615028 learned_addr learned my addr 172.21.15.16:0/1837615028 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:22:17.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.571+0000 7f1d61ca8700 1 --2- 172.21.15.16:0/1837615028 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55be6e248400 0x55be6dfb4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:17.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.571+0000 7f1d624a9700 1 --2- 172.21.15.16:0/1837615028 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55be6e248000 0x55be6dfb3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:17.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.571+0000 7f1d62caa700 1 -- 172.21.15.16:0/1837615028 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55be6e248000 msgr2=0x55be6dfb3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:17.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.571+0000 7f1d62caa700 1 --2- 172.21.15.16:0/1837615028 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55be6e248000 0x55be6dfb3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:17.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.571+0000 7f1d62caa700 1 -- 172.21.15.16:0/1837615028 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55be6e248400 msgr2=0x55be6dfb4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:17.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.571+0000 7f1d62caa700 1 --2- 172.21.15.16:0/1837615028 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55be6e248400 0x55be6dfb4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:17.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.571+0000 7f1d62caa700 1 -- 172.21.15.16:0/1837615028 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be6df3d520 con 0x55be6df18400 2023-05-20T01:22:17.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.571+0000 7f1d62caa700 1 --2- 172.21.15.16:0/1837615028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be6df18400 0x55be6dfb3600 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x55be6e2306c0 tx=0x55be6ece4de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:17.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.571+0000 7f1d5c49d700 1 -- 172.21.15.16:0/1837615028 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55be6df4cdc0 con 0x55be6df18400 2023-05-20T01:22:17.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.571+0000 7f1d5c49d700 1 -- 172.21.15.16:0/1837615028 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55be6edb6d80 con 0x55be6df18400 2023-05-20T01:22:17.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.571+0000 7f1d5c49d700 1 -- 172.21.15.16:0/1837615028 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55be6df4d600 con 0x55be6df18400 2023-05-20T01:22:17.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.571+0000 7f1d644fd2c0 1 -- 172.21.15.16:0/1837615028 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55be6df3cd00 con 0x55be6df18400 2023-05-20T01:22:17.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.572+0000 7f1d644fd2c0 1 -- 172.21.15.16:0/1837615028 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55be6df3d380 con 0x55be6df18400 2023-05-20T01:22:17.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.572+0000 7f1d644fd2c0 4 main: resolving pool 9 2023-05-20T01:22:17.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.573+0000 7f1d5c49d700 1 -- 172.21.15.16:0/1837615028 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x55be6df7b600 con 0x55be6df18400 2023-05-20T01:22:17.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.573+0000 7f1d5c49d700 1 --2- 172.21.15.16:0/1837615028 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x55be6e252400 0x55be6dfb5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:17.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.573+0000 7f1d5c49d700 1 -- 172.21.15.16:0/1837615028 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 7103+0+0 (secure 0 0 0) 0x55be6df5aa80 con 0x55be6df18400 2023-05-20T01:22:17.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.573+0000 7f1d644fd2c0 4 main: creating IoCtx.. 2023-05-20T01:22:17.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.573+0000 7f1d624a9700 1 --2- 172.21.15.16:0/1837615028 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x55be6e252400 0x55be6dfb5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:17.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.573+0000 7f1d644fd2c0 1 --2- 172.21.15.16:0/1837615028 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x55be6e248800 0x55be6dfb4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:17.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.573+0000 7f1d644fd2c0 1 -- 172.21.15.16:0/1837615028 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:1 9.d 9:b50e409b:::mds_snaptable:head [writefull 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x55be6edb4700 con 0x55be6e248800 2023-05-20T01:22:17.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.573+0000 7f1d61ca8700 1 --2- 172.21.15.16:0/1837615028 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x55be6e248800 0x55be6dfb4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:17.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.573+0000 7f1d624a9700 1 --2- 172.21.15.16:0/1837615028 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x55be6e252400 0x55be6dfb5180 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x55be6ec92780 tx=0x55be6ecb13e0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:17.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.574+0000 7f1d61ca8700 1 --2- 172.21.15.16:0/1837615028 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x55be6e248800 0x55be6dfb4680 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:17.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.575+0000 7f1d61ca8700 1 -- 172.21.15.16:0/1837615028 <== osd.2 v2:172.21.15.16:6816/293187030 1 ==== osd_op_reply(1 mds_snaptable [writefull 0~70] v71'3 uv3 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x55be6ed7cfc0 con 0x55be6e248800 2023-05-20T01:22:17.594 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-05-20T01:22:17.594 INFO:teuthology.orchestra.run.smithi016.stdout: "result": 0 2023-05-20T01:22:17.594 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-05-20T01:22:17.595 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:22:17.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.575+0000 7f1d644fd2c0 1 -- 172.21.15.16:0/1837615028 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x55be6e248800 msgr2=0x55be6dfb4680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:17.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.575+0000 7f1d644fd2c0 1 --2- 172.21.15.16:0/1837615028 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x55be6e248800 0x55be6dfb4680 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:17.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.575+0000 7f1d644fd2c0 1 -- 172.21.15.16:0/1837615028 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x55be6e252400 msgr2=0x55be6dfb5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:17.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.575+0000 7f1d644fd2c0 1 --2- 172.21.15.16:0/1837615028 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x55be6e252400 0x55be6dfb5180 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x55be6ec92780 tx=0x55be6ecb13e0 comp rx=0 tx=0).stop 2023-05-20T01:22:17.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.575+0000 7f1d644fd2c0 1 -- 172.21.15.16:0/1837615028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be6df18400 msgr2=0x55be6dfb3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:17.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.575+0000 7f1d644fd2c0 1 --2- 172.21.15.16:0/1837615028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be6df18400 0x55be6dfb3600 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x55be6e2306c0 tx=0x55be6ece4de0 comp rx=0 tx=0).stop 2023-05-20T01:22:17.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.575+0000 7f1d624a9700 1 -- 172.21.15.16:0/1837615028 reap_dead start 2023-05-20T01:22:17.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.575+0000 7f1d644fd2c0 1 -- 172.21.15.16:0/1837615028 shutdown_connections 2023-05-20T01:22:17.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.575+0000 7f1d644fd2c0 1 -- 172.21.15.16:0/1837615028 >> 172.21.15.16:0/1837615028 conn(0x55be6df19c00 msgr2=0x55be6dff5800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:17.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.575+0000 7f1d644fd2c0 1 -- 172.21.15.16:0/1837615028 shutdown_connections 2023-05-20T01:22:17.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.575+0000 7f1d644fd2c0 1 -- 172.21.15.16:0/1837615028 wait complete. 2023-05-20T01:22:17.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.575+0000 7f1d644fd2c0 1 -- 172.21.15.16:0/1797155793 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be6df18c00 msgr2=0x55be6dfb2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:17.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.575+0000 7f1d644fd2c0 1 --2- 172.21.15.16:0/1797155793 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be6df18c00 0x55be6dfb2580 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x55be6ed886c0 tx=0x55be6ece4ea0 comp rx=0 tx=0).stop 2023-05-20T01:22:17.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.576+0000 7f1d644fd2c0 1 -- 172.21.15.16:0/1797155793 shutdown_connections 2023-05-20T01:22:17.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.576+0000 7f1d644fd2c0 1 --2- 172.21.15.16:0/1797155793 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55be6df18800 0x55be6dfb3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:17.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.576+0000 7f1d644fd2c0 1 --2- 172.21.15.16:0/1797155793 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55be6df19000 0x55be6dfb2b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:17.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.576+0000 7f1d644fd2c0 1 --2- 172.21.15.16:0/1797155793 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be6df18c00 0x55be6dfb2580 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:17.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.576+0000 7f1d644fd2c0 1 -- 172.21.15.16:0/1797155793 >> 172.21.15.16:0/1797155793 conn(0x55be6df19400 msgr2=0x55be6dff5000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:17.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.576+0000 7f1d644fd2c0 1 -- 172.21.15.16:0/1797155793 shutdown_connections 2023-05-20T01:22:17.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.576+0000 7f1d644fd2c0 1 -- 172.21.15.16:0/1797155793 wait complete. 2023-05-20T01:22:17.600 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-table-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'cephfs:0', 'reset', 'snap'] in time 0:00:00.108588, result: { "result": 0 } 2023-05-20T01:22:17.600 DEBUG:teuthology.orchestra.run.smithi016:> cephfs-table-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 cephfs:0 reset inode 2023-05-20T01:22:17.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.676+0000 7fc83b87c2c0 1 -- start start 2023-05-20T01:22:17.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.676+0000 7fc83b87c2c0 1 Processor -- start 2023-05-20T01:22:17.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.677+0000 7fc83b87c2c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f734ddf400 0x55f734e78b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:17.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.677+0000 7fc83b87c2c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f734ddec00 0x55f734e79b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:17.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.677+0000 7fc83b87c2c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55f734dde800 0x55f734e79080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:17.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.677+0000 7fc83b87c2c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f734e1bcc0 con 0x55f734ddf400 2023-05-20T01:22:17.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.677+0000 7fc83b87c2c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f734e1b400 con 0x55f734ddec00 2023-05-20T01:22:17.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.677+0000 7fc83b87c2c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f734e1a000 con 0x55f734dde800 2023-05-20T01:22:17.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.677+0000 7fc839828700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f734ddf400 0x55f734e78b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:17.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.677+0000 7fc839828700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f734ddf400 0x55f734e78b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34516/0 (socket says 172.21.15.16:34516) 2023-05-20T01:22:17.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.677+0000 7fc839828700 1 -- 172.21.15.16:0/3374166128 learned_addr learned my addr 172.21.15.16:0/3374166128 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:22:17.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.677+0000 7fc83a029700 1 --2- 172.21.15.16:0/3374166128 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55f734dde800 0x55f734e79080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:17.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.677+0000 7fc839027700 1 --2- 172.21.15.16:0/3374166128 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f734ddec00 0x55f734e79b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:17.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.678+0000 7fc839828700 1 -- 172.21.15.16:0/3374166128 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55f734dde800 msgr2=0x55f734e79080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:17.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.678+0000 7fc839828700 1 --2- 172.21.15.16:0/3374166128 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55f734dde800 0x55f734e79080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:17.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.678+0000 7fc839828700 1 -- 172.21.15.16:0/3374166128 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f734ddec00 msgr2=0x55f734e79b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:17.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.678+0000 7fc839828700 1 --2- 172.21.15.16:0/3374166128 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f734ddec00 0x55f734e79b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:17.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.678+0000 7fc839828700 1 -- 172.21.15.16:0/3374166128 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f734e03520 con 0x55f734ddf400 2023-05-20T01:22:17.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.678+0000 7fc839027700 1 --2- 172.21.15.16:0/3374166128 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f734ddec00 0x55f734e79b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:22:17.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.678+0000 7fc839828700 1 --2- 172.21.15.16:0/3374166128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f734ddf400 0x55f734e78b00 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x55f735bce630 tx=0x55f735bbaf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1034708d84fecf5 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:17.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.678+0000 7fc837023700 1 -- 172.21.15.16:0/3374166128 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55f734e13080 con 0x55f734ddf400 2023-05-20T01:22:17.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.678+0000 7fc837023700 1 -- 172.21.15.16:0/3374166128 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f734e31800 con 0x55f734ddf400 2023-05-20T01:22:17.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.678+0000 7fc837023700 1 -- 172.21.15.16:0/3374166128 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55f734e13600 con 0x55f734ddf400 2023-05-20T01:22:17.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.678+0000 7fc83b87c2c0 1 -- 172.21.15.16:0/3374166128 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f734e03d40 con 0x55f734ddf400 2023-05-20T01:22:17.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.678+0000 7fc83b87c2c0 1 -- 172.21.15.16:0/3374166128 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55f734e036c0 con 0x55f734ddf400 2023-05-20T01:22:17.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.678+0000 7fc83b87c2c0 4 waiting for MDS map... 2023-05-20T01:22:17.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.678+0000 7fc837023700 1 -- 172.21.15.16:0/3374166128 <== mon.0 v2:172.21.15.16:3300/0 4 ==== osd_map(71..71 src has 1..71) v4 ==== 7103+0+0 (secure 0 0 0) 0x55f734e20c40 con 0x55f734ddf400 2023-05-20T01:22:17.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.678+0000 7fc837023700 1 -- 172.21.15.16:0/3374166128 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 42) v1 ==== 3126+0+0 (secure 0 0 0) 0x55f735c6a900 con 0x55f734ddf400 2023-05-20T01:22:17.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.678+0000 7fc83b87c2c0 4 Got MDS map 42 2023-05-20T01:22:17.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.678+0000 7fc83b87c2c0 10 main: main 2023-05-20T01:22:17.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.678+0000 7fc83b87c2c0 4 main: connecting to RADOS... 2023-05-20T01:22:17.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.679+0000 7fc83b87c2c0 1 Processor -- start 2023-05-20T01:22:17.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.679+0000 7fc83b87c2c0 1 -- start start 2023-05-20T01:22:17.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.679+0000 7fc83b87c2c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f734ddf800 0x55f734e78580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:17.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.679+0000 7fc83b87c2c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f7350fc000 0x55f734e7a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:17.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.679+0000 7fc83b87c2c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55f7350fc400 0x55f734e7a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:17.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.679+0000 7fc83b87c2c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f734e1ac80 con 0x55f7350fc000 2023-05-20T01:22:17.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.679+0000 7fc83b87c2c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f734e1ab40 con 0x55f734ddf800 2023-05-20T01:22:17.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.679+0000 7fc83b87c2c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f734e1b400 con 0x55f7350fc400 2023-05-20T01:22:17.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.679+0000 7fc839828700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f7350fc000 0x55f734e7a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:17.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.680+0000 7fc839828700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f7350fc000 0x55f734e7a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34524/0 (socket says 172.21.15.16:34524) 2023-05-20T01:22:17.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.680+0000 7fc839828700 1 -- 172.21.15.16:0/4193880720 learned_addr learned my addr 172.21.15.16:0/4193880720 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:22:17.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.680+0000 7fc839027700 1 --2- 172.21.15.16:0/4193880720 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55f7350fc400 0x55f734e7a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:17.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.680+0000 7fc83a029700 1 --2- 172.21.15.16:0/4193880720 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f734ddf800 0x55f734e78580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:17.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.680+0000 7fc839828700 1 -- 172.21.15.16:0/4193880720 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55f7350fc400 msgr2=0x55f734e7a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:17.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.680+0000 7fc839828700 1 --2- 172.21.15.16:0/4193880720 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55f7350fc400 0x55f734e7a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:17.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.680+0000 7fc839828700 1 -- 172.21.15.16:0/4193880720 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f734ddf800 msgr2=0x55f734e78580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:17.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.680+0000 7fc839828700 1 --2- 172.21.15.16:0/4193880720 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f734ddf800 0x55f734e78580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:17.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.680+0000 7fc839828700 1 -- 172.21.15.16:0/4193880720 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f734e031e0 con 0x55f7350fc000 2023-05-20T01:22:17.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.680+0000 7fc839027700 1 --2- 172.21.15.16:0/4193880720 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55f7350fc400 0x55f734e7a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:22:17.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.680+0000 7fc83a029700 1 --2- 172.21.15.16:0/4193880720 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f734ddf800 0x55f734e78580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:22:17.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.680+0000 7fc839828700 1 --2- 172.21.15.16:0/4193880720 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f7350fc000 0x55f734e7a100 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x55f735bcfb00 tx=0x55f735bba0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca2e7bf2bae04b79 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:17.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.680+0000 7fc83501f700 1 -- 172.21.15.16:0/4193880720 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55f734e13080 con 0x55f7350fc000 2023-05-20T01:22:17.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.680+0000 7fc83501f700 1 -- 172.21.15.16:0/4193880720 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f734e31c80 con 0x55f7350fc000 2023-05-20T01:22:17.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.680+0000 7fc83501f700 1 -- 172.21.15.16:0/4193880720 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55f734e13600 con 0x55f7350fc000 2023-05-20T01:22:17.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.680+0000 7fc83b87c2c0 1 -- 172.21.15.16:0/4193880720 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f7350fc000 msgr2=0x55f734e7a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:17.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.680+0000 7fc83b87c2c0 1 --2- 172.21.15.16:0/4193880720 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f7350fc000 0x55f734e7a100 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x55f735bcfb00 tx=0x55f735bba0c0 comp rx=0 tx=0).stop 2023-05-20T01:22:17.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.680+0000 7fc83b87c2c0 1 -- 172.21.15.16:0/4193880720 shutdown_connections 2023-05-20T01:22:17.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.680+0000 7fc83b87c2c0 1 --2- 172.21.15.16:0/4193880720 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55f7350fc400 0x55f734e7a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:17.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.680+0000 7fc83b87c2c0 1 --2- 172.21.15.16:0/4193880720 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f734ddf800 0x55f734e78580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:17.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.680+0000 7fc83b87c2c0 1 --2- 172.21.15.16:0/4193880720 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f7350fc000 0x55f734e7a100 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:17.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.680+0000 7fc83b87c2c0 1 -- 172.21.15.16:0/4193880720 >> 172.21.15.16:0/4193880720 conn(0x55f734ddfc00 msgr2=0x55f734ebb000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:17.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.681+0000 7fc83b87c2c0 1 -- 172.21.15.16:0/4193880720 shutdown_connections 2023-05-20T01:22:17.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.681+0000 7fc83b87c2c0 1 -- 172.21.15.16:0/4193880720 wait complete. 2023-05-20T01:22:17.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.681+0000 7fc83b87c2c0 1 Processor -- start 2023-05-20T01:22:17.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.681+0000 7fc83b87c2c0 1 -- start start 2023-05-20T01:22:17.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.682+0000 7fc83b87c2c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f734ddfc00 0x55f734e78580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:17.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.682+0000 7fc83b87c2c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f73510c000 0x55f734e7a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:17.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.682+0000 7fc83b87c2c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55f73510c400 0x55f734e7a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:17.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.682+0000 7fc83b87c2c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f734e1bcc0 con 0x55f73510c000 2023-05-20T01:22:17.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.682+0000 7fc83b87c2c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f734e1b180 con 0x55f734ddfc00 2023-05-20T01:22:17.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.682+0000 7fc83b87c2c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f734e1b040 con 0x55f73510c400 2023-05-20T01:22:17.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.682+0000 7fc839828700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f73510c000 0x55f734e7a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:17.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.682+0000 7fc839828700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f73510c000 0x55f734e7a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34530/0 (socket says 172.21.15.16:34530) 2023-05-20T01:22:17.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.682+0000 7fc839828700 1 -- 172.21.15.16:0/3518212495 learned_addr learned my addr 172.21.15.16:0/3518212495 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:22:17.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.682+0000 7fc83a029700 1 --2- 172.21.15.16:0/3518212495 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f734ddfc00 0x55f734e78580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:17.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.682+0000 7fc839027700 1 --2- 172.21.15.16:0/3518212495 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55f73510c400 0x55f734e7a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:17.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.682+0000 7fc839828700 1 -- 172.21.15.16:0/3518212495 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55f73510c400 msgr2=0x55f734e7a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:17.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.682+0000 7fc839828700 1 --2- 172.21.15.16:0/3518212495 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55f73510c400 0x55f734e7a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:17.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.682+0000 7fc839828700 1 -- 172.21.15.16:0/3518212495 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f734ddfc00 msgr2=0x55f734e78580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:17.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.682+0000 7fc839828700 1 --2- 172.21.15.16:0/3518212495 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f734ddfc00 0x55f734e78580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:17.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.682+0000 7fc839828700 1 -- 172.21.15.16:0/3518212495 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f735c67040 con 0x55f73510c000 2023-05-20T01:22:17.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.682+0000 7fc839828700 1 --2- 172.21.15.16:0/3518212495 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f73510c000 0x55f734e7a100 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x55f735bcfd40 tx=0x55f735c82c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:17.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.682+0000 7fc83381c700 1 -- 172.21.15.16:0/3518212495 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55f734e131e0 con 0x55f73510c000 2023-05-20T01:22:17.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.682+0000 7fc83381c700 1 -- 172.21.15.16:0/3518212495 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f734e30480 con 0x55f73510c000 2023-05-20T01:22:17.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.682+0000 7fc83381c700 1 -- 172.21.15.16:0/3518212495 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55f734e13080 con 0x55f73510c000 2023-05-20T01:22:17.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.682+0000 7fc83b87c2c0 1 -- 172.21.15.16:0/3518212495 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55f735c66ea0 con 0x55f73510c000 2023-05-20T01:22:17.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.682+0000 7fc83b87c2c0 1 -- 172.21.15.16:0/3518212495 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f735c66d00 con 0x55f73510c000 2023-05-20T01:22:17.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.682+0000 7fc83b87c2c0 4 main: resolving pool 9 2023-05-20T01:22:17.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.684+0000 7fc83381c700 1 -- 172.21.15.16:0/3518212495 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x55f734e40840 con 0x55f73510c000 2023-05-20T01:22:17.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.684+0000 7fc83381c700 1 --2- 172.21.15.16:0/3518212495 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x55f73510e800 0x55f734e7b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:17.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.684+0000 7fc83381c700 1 -- 172.21.15.16:0/3518212495 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 7103+0+0 (secure 0 0 0) 0x55f734e20a80 con 0x55f73510c000 2023-05-20T01:22:17.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.684+0000 7fc83b87c2c0 4 main: creating IoCtx.. 2023-05-20T01:22:17.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.684+0000 7fc83a029700 1 --2- 172.21.15.16:0/3518212495 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x55f73510e800 0x55f734e7b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:17.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.684+0000 7fc83b87c2c0 1 --2- 172.21.15.16:0/3518212495 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x55f73510c800 0x55f734e7ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:17.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.684+0000 7fc83b87c2c0 1 -- 172.21.15.16:0/3518212495 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:1 9.13 9:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x55f735c76700 con 0x55f73510c800 2023-05-20T01:22:17.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.684+0000 7fc839027700 1 --2- 172.21.15.16:0/3518212495 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x55f73510c800 0x55f734e7ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:17.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.684+0000 7fc83a029700 1 --2- 172.21.15.16:0/3518212495 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x55f73510e800 0x55f734e7b700 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x55f734e57fb0 tx=0x55f735b771a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:17.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.685+0000 7fc839027700 1 --2- 172.21.15.16:0/3518212495 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x55f73510c800 0x55f734e7ac00 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:17.706 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-05-20T01:22:17.706 INFO:teuthology.orchestra.run.smithi016.stdout: "0": { 2023-05-20T01:22:17.707 INFO:teuthology.orchestra.run.smithi016.stdout: "data": {}, 2023-05-20T01:22:17.707 INFO:teuthology.orchestra.run.smithi016.stdout: "result": 0 2023-05-20T01:22:17.707 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:22:17.707 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-05-20T01:22:17.707 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:22:17.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.686+0000 7fc839027700 1 -- 172.21.15.16:0/3518212495 <== osd.7 v2:172.21.15.132:6800/466001149 1 ==== osd_op_reply(1 mds0_inotable [writefull 0~34] v71'6 uv6 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x55f734f48fc0 con 0x55f73510c800 2023-05-20T01:22:17.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.687+0000 7fc83b87c2c0 1 -- 172.21.15.16:0/3518212495 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x55f73510c800 msgr2=0x55f734e7ac00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:17.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.687+0000 7fc83b87c2c0 1 --2- 172.21.15.16:0/3518212495 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x55f73510c800 0x55f734e7ac00 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:17.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.687+0000 7fc83b87c2c0 1 -- 172.21.15.16:0/3518212495 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x55f73510e800 msgr2=0x55f734e7b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:17.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.687+0000 7fc83b87c2c0 1 --2- 172.21.15.16:0/3518212495 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x55f73510e800 0x55f734e7b700 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x55f734e57fb0 tx=0x55f735b771a0 comp rx=0 tx=0).stop 2023-05-20T01:22:17.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.687+0000 7fc83b87c2c0 1 -- 172.21.15.16:0/3518212495 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f73510c000 msgr2=0x55f734e7a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:17.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.687+0000 7fc83b87c2c0 1 --2- 172.21.15.16:0/3518212495 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f73510c000 0x55f734e7a100 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x55f735bcfd40 tx=0x55f735c82c00 comp rx=0 tx=0).stop 2023-05-20T01:22:17.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.687+0000 7fc839027700 1 -- 172.21.15.16:0/3518212495 reap_dead start 2023-05-20T01:22:17.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.687+0000 7fc83b87c2c0 1 -- 172.21.15.16:0/3518212495 shutdown_connections 2023-05-20T01:22:17.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.687+0000 7fc83b87c2c0 1 -- 172.21.15.16:0/3518212495 >> 172.21.15.16:0/3518212495 conn(0x55f734ddf800 msgr2=0x55f734ebb000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:17.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.687+0000 7fc83b87c2c0 1 -- 172.21.15.16:0/3518212495 shutdown_connections 2023-05-20T01:22:17.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.687+0000 7fc83b87c2c0 1 -- 172.21.15.16:0/3518212495 wait complete. 2023-05-20T01:22:17.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.687+0000 7fc83b87c2c0 1 -- 172.21.15.16:0/3374166128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f734ddf400 msgr2=0x55f734e78b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:17.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.687+0000 7fc83b87c2c0 1 --2- 172.21.15.16:0/3374166128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f734ddf400 0x55f734e78b00 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x55f735bce630 tx=0x55f735bbaf00 comp rx=0 tx=0).stop 2023-05-20T01:22:17.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.687+0000 7fc83b87c2c0 1 -- 172.21.15.16:0/3374166128 shutdown_connections 2023-05-20T01:22:17.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.687+0000 7fc83b87c2c0 1 --2- 172.21.15.16:0/3374166128 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55f734dde800 0x55f734e79080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:17.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.687+0000 7fc83b87c2c0 1 --2- 172.21.15.16:0/3374166128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f734ddf400 0x55f734e78b00 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:17.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.687+0000 7fc83b87c2c0 1 --2- 172.21.15.16:0/3374166128 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f734ddec00 0x55f734e79b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:17.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.687+0000 7fc83b87c2c0 1 -- 172.21.15.16:0/3374166128 >> 172.21.15.16:0/3374166128 conn(0x55f734dde400 msgr2=0x55f734ebc000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:17.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.687+0000 7fc83b87c2c0 1 -- 172.21.15.16:0/3374166128 shutdown_connections 2023-05-20T01:22:17.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.687+0000 7fc83b87c2c0 1 -- 172.21.15.16:0/3374166128 wait complete. 2023-05-20T01:22:17.713 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-table-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'cephfs:0', 'reset', 'inode'] in time 0:00:00.113121, result: { "0": { "data": {}, "result": 0 } } 2023-05-20T01:22:17.714 DEBUG:teuthology.orchestra.run.smithi016:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 journal reset --force 2023-05-20T01:22:17.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.790+0000 7fbd5d2f62c0 1 -- start start 2023-05-20T01:22:17.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.790+0000 7fbd5d2f62c0 1 Processor -- start 2023-05-20T01:22:17.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.791+0000 7fbd5d2f62c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a7e2c73000 0x55a7e2d0c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:17.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.791+0000 7fbd5d2f62c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a7e2c72c00 0x55a7e2d0cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:17.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.791+0000 7fbd5d2f62c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a7e2c72800 0x55a7e2d0d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:17.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.791+0000 7fbd5d2f62c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7e2cafcc0 con 0x55a7e2c72c00 2023-05-20T01:22:17.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.791+0000 7fbd5d2f62c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7e2caf400 con 0x55a7e2c73000 2023-05-20T01:22:17.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.791+0000 7fbd5d2f62c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a7e2cafe00 con 0x55a7e2c72800 2023-05-20T01:22:17.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.791+0000 7fbd5aaa1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a7e2c72c00 0x55a7e2d0cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:17.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.791+0000 7fbd5aaa1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a7e2c72c00 0x55a7e2d0cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34544/0 (socket says 172.21.15.16:34544) 2023-05-20T01:22:17.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.791+0000 7fbd5aaa1700 1 -- 172.21.15.16:0/2303762132 learned_addr learned my addr 172.21.15.16:0/2303762132 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:22:17.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.791+0000 7fbd5b2a2700 1 --2- 172.21.15.16:0/2303762132 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a7e2c73000 0x55a7e2d0c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:17.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.791+0000 7fbd5baa3700 1 --2- 172.21.15.16:0/2303762132 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a7e2c72800 0x55a7e2d0d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:17.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.791+0000 7fbd5aaa1700 1 -- 172.21.15.16:0/2303762132 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a7e2c72800 msgr2=0x55a7e2d0d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:17.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.791+0000 7fbd5aaa1700 1 --2- 172.21.15.16:0/2303762132 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a7e2c72800 0x55a7e2d0d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:17.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.791+0000 7fbd5aaa1700 1 -- 172.21.15.16:0/2303762132 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a7e2c73000 msgr2=0x55a7e2d0c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:17.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.791+0000 7fbd5aaa1700 1 --2- 172.21.15.16:0/2303762132 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a7e2c73000 0x55a7e2d0c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:17.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.791+0000 7fbd5aaa1700 1 -- 172.21.15.16:0/2303762132 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a7e2c964e0 con 0x55a7e2c72c00 2023-05-20T01:22:17.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.791+0000 7fbd5aaa1700 1 --2- 172.21.15.16:0/2303762132 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a7e2c72c00 0x55a7e2d0cb00 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x55a7e3aea780 tx=0x55a7e3a0b2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27784133f22120d1 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:17.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.792+0000 7fbd58a9d700 1 -- 172.21.15.16:0/2303762132 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55a7e2ca7080 con 0x55a7e2c72c00 2023-05-20T01:22:17.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.792+0000 7fbd58a9d700 1 -- 172.21.15.16:0/2303762132 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a7e2cc5980 con 0x55a7e2c72c00 2023-05-20T01:22:17.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.792+0000 7fbd58a9d700 1 -- 172.21.15.16:0/2303762132 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55a7e2ca7600 con 0x55a7e2c72c00 2023-05-20T01:22:17.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.792+0000 7fbd5d2f62c0 1 -- 172.21.15.16:0/2303762132 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a7e2c97520 con 0x55a7e2c72c00 2023-05-20T01:22:17.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.792+0000 7fbd5d2f62c0 1 -- 172.21.15.16:0/2303762132 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55a7e2c97380 con 0x55a7e2c72c00 2023-05-20T01:22:17.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.792+0000 7fbd5d2f62c0 4 waiting for MDS map... 2023-05-20T01:22:17.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.792+0000 7fbd58a9d700 1 -- 172.21.15.16:0/2303762132 <== mon.0 v2:172.21.15.16:3300/0 4 ==== osd_map(71..71 src has 1..71) v4 ==== 7103+0+0 (secure 0 0 0) 0x55a7e2cb4c40 con 0x55a7e2c72c00 2023-05-20T01:22:17.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.792+0000 7fbd58a9d700 1 -- 172.21.15.16:0/2303762132 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 42) v1 ==== 3126+0+0 (secure 0 0 0) 0x55a7e2cb8f00 con 0x55a7e2c72c00 2023-05-20T01:22:17.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.792+0000 7fbd5d2f62c0 4 Got MDS map 42 2023-05-20T01:22:17.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.792+0000 7fbd5d2f62c0 10 main: JournalTool::main 2023-05-20T01:22:17.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.792+0000 7fbd5d2f62c0 4 main: JournalTool: connecting to RADOS... 2023-05-20T01:22:17.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.792+0000 7fbd5d2f62c0 1 Processor -- start 2023-05-20T01:22:17.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.793+0000 7fbd5d2f62c0 1 -- start start 2023-05-20T01:22:17.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.793+0000 7fbd5d2f62c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a7e2c73c00 0x55a7e2d0d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:17.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.793+0000 7fbd5d2f62c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a7e2f8a000 0x55a7e2d0db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:17.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.793+0000 7fbd5d2f62c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a7e2f8a400 0x55a7e2d0e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:17.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.793+0000 7fbd5d2f62c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7e2caea00 con 0x55a7e2c73c00 2023-05-20T01:22:17.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.793+0000 7fbd5d2f62c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7e2caf400 con 0x55a7e2f8a000 2023-05-20T01:22:17.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.793+0000 7fbd5d2f62c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a7e2cafcc0 con 0x55a7e2f8a400 2023-05-20T01:22:17.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.793+0000 7fbd5baa3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a7e2c73c00 0x55a7e2d0d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:17.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.793+0000 7fbd5baa3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a7e2c73c00 0x55a7e2d0d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34548/0 (socket says 172.21.15.16:34548) 2023-05-20T01:22:17.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.793+0000 7fbd5baa3700 1 -- 172.21.15.16:0/3648398891 learned_addr learned my addr 172.21.15.16:0/3648398891 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:22:17.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.793+0000 7fbd5b2a2700 1 --2- 172.21.15.16:0/3648398891 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a7e2f8a000 0x55a7e2d0db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:17.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.793+0000 7fbd5aaa1700 1 --2- 172.21.15.16:0/3648398891 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a7e2f8a400 0x55a7e2d0e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:17.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.793+0000 7fbd5baa3700 1 -- 172.21.15.16:0/3648398891 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a7e2f8a400 msgr2=0x55a7e2d0e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:17.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.793+0000 7fbd5baa3700 1 --2- 172.21.15.16:0/3648398891 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a7e2f8a400 0x55a7e2d0e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:17.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.793+0000 7fbd5baa3700 1 -- 172.21.15.16:0/3648398891 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a7e2f8a000 msgr2=0x55a7e2d0db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:17.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.793+0000 7fbd5baa3700 1 --2- 172.21.15.16:0/3648398891 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a7e2f8a000 0x55a7e2d0db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:17.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.793+0000 7fbd5baa3700 1 -- 172.21.15.16:0/3648398891 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a7e2c97d40 con 0x55a7e2c73c00 2023-05-20T01:22:17.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.793+0000 7fbd5b2a2700 1 --2- 172.21.15.16:0/3648398891 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a7e2f8a000 0x55a7e2d0db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:22:17.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.794+0000 7fbd5aaa1700 1 --2- 172.21.15.16:0/3648398891 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a7e2f8a400 0x55a7e2d0e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:22:17.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.794+0000 7fbd5baa3700 1 --2- 172.21.15.16:0/3648398891 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a7e2c73c00 0x55a7e2d0d600 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x55a7e3aebc50 tx=0x55a7e3b0ac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86d6e456f0f8dc01 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:17.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.794+0000 7fbd56a99700 1 -- 172.21.15.16:0/3648398891 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55a7e2ca7080 con 0x55a7e2c73c00 2023-05-20T01:22:17.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.794+0000 7fbd56a99700 1 -- 172.21.15.16:0/3648398891 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a7e3b00d80 con 0x55a7e2c73c00 2023-05-20T01:22:17.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.794+0000 7fbd56a99700 1 -- 172.21.15.16:0/3648398891 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55a7e2ca7600 con 0x55a7e2c73c00 2023-05-20T01:22:17.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.794+0000 7fbd5d2f62c0 1 -- 172.21.15.16:0/3648398891 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a7e2c73c00 msgr2=0x55a7e2d0d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:17.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.794+0000 7fbd5d2f62c0 1 --2- 172.21.15.16:0/3648398891 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a7e2c73c00 0x55a7e2d0d600 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x55a7e3aebc50 tx=0x55a7e3b0ac00 comp rx=0 tx=0).stop 2023-05-20T01:22:17.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.794+0000 7fbd5d2f62c0 1 -- 172.21.15.16:0/3648398891 shutdown_connections 2023-05-20T01:22:17.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.794+0000 7fbd5d2f62c0 1 --2- 172.21.15.16:0/3648398891 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a7e2f8a400 0x55a7e2d0e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:17.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.794+0000 7fbd5d2f62c0 1 --2- 172.21.15.16:0/3648398891 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a7e2c73c00 0x55a7e2d0d600 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:17.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.794+0000 7fbd5d2f62c0 1 --2- 172.21.15.16:0/3648398891 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a7e2f8a000 0x55a7e2d0db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:17.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.794+0000 7fbd5d2f62c0 1 -- 172.21.15.16:0/3648398891 >> 172.21.15.16:0/3648398891 conn(0x55a7e2c72400 msgr2=0x55a7e2d43800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:17.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.794+0000 7fbd5d2f62c0 1 -- 172.21.15.16:0/3648398891 shutdown_connections 2023-05-20T01:22:17.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.794+0000 7fbd5d2f62c0 1 -- 172.21.15.16:0/3648398891 wait complete. 2023-05-20T01:22:17.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.795+0000 7fbd5d2f62c0 1 Processor -- start 2023-05-20T01:22:17.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.795+0000 7fbd5d2f62c0 1 -- start start 2023-05-20T01:22:17.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.795+0000 7fbd5d2f62c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a7e2c72400 0x55a7e2d0f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:17.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.795+0000 7fbd5d2f62c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a7e2faa000 0x55a7e2d0ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:17.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.795+0000 7fbd5d2f62c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a7e2faa400 0x55a7e2d0e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:17.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.795+0000 7fbd5d2f62c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7e2caeb40 con 0x55a7e2faa000 2023-05-20T01:22:17.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.795+0000 7fbd5d2f62c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7e2caec80 con 0x55a7e2faa400 2023-05-20T01:22:17.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.795+0000 7fbd5d2f62c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a7e2cae000 con 0x55a7e2c72400 2023-05-20T01:22:17.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.796+0000 7fbd5b2a2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a7e2faa000 0x55a7e2d0ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:17.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.796+0000 7fbd5b2a2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a7e2faa000 0x55a7e2d0ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34554/0 (socket says 172.21.15.16:34554) 2023-05-20T01:22:17.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.796+0000 7fbd5b2a2700 1 -- 172.21.15.16:0/1090139213 learned_addr learned my addr 172.21.15.16:0/1090139213 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:22:17.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.796+0000 7fbd5aaa1700 1 --2- 172.21.15.16:0/1090139213 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a7e2faa400 0x55a7e2d0e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:17.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.796+0000 7fbd5baa3700 1 --2- 172.21.15.16:0/1090139213 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a7e2c72400 0x55a7e2d0f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:17.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.796+0000 7fbd5b2a2700 1 -- 172.21.15.16:0/1090139213 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a7e2c72400 msgr2=0x55a7e2d0f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:17.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.796+0000 7fbd5b2a2700 1 --2- 172.21.15.16:0/1090139213 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a7e2c72400 0x55a7e2d0f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:17.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.796+0000 7fbd5b2a2700 1 -- 172.21.15.16:0/1090139213 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a7e2faa400 msgr2=0x55a7e2d0e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:17.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.796+0000 7fbd5b2a2700 1 --2- 172.21.15.16:0/1090139213 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a7e2faa400 0x55a7e2d0e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:17.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.796+0000 7fbd5b2a2700 1 -- 172.21.15.16:0/1090139213 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a7e2fa61a0 con 0x55a7e2faa000 2023-05-20T01:22:17.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.796+0000 7fbd5aaa1700 1 --2- 172.21.15.16:0/1090139213 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a7e2faa400 0x55a7e2d0e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:22:17.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.796+0000 7fbd5b2a2700 1 --2- 172.21.15.16:0/1090139213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a7e2faa000 0x55a7e2d0ec00 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x55a7e2f8ac30 tx=0x55a7e3b0ad20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:17.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.796+0000 7fbd55296700 1 -- 172.21.15.16:0/1090139213 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55a7e2ca7080 con 0x55a7e2faa000 2023-05-20T01:22:17.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.796+0000 7fbd55296700 1 -- 172.21.15.16:0/1090139213 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a7e2cc4c00 con 0x55a7e2faa000 2023-05-20T01:22:17.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.796+0000 7fbd55296700 1 -- 172.21.15.16:0/1090139213 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55a7e2ca7600 con 0x55a7e2faa000 2023-05-20T01:22:17.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.796+0000 7fbd5d2f62c0 1 -- 172.21.15.16:0/1090139213 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55a7e2fa6000 con 0x55a7e2faa000 2023-05-20T01:22:17.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.796+0000 7fbd5d2f62c0 1 -- 172.21.15.16:0/1090139213 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a7e2fa6340 con 0x55a7e2faa000 2023-05-20T01:22:17.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.796+0000 7fbd5d2f62c0 4 main: JournalTool: resolving pool 9 2023-05-20T01:22:17.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.797+0000 7fbd55296700 1 -- 172.21.15.16:0/1090139213 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x55a7e2cd4840 con 0x55a7e2faa000 2023-05-20T01:22:17.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.798+0000 7fbd55296700 1 --2- 172.21.15.16:0/1090139213 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x55a7e2fb0400 0x55a7e2d0f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:17.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.798+0000 7fbd55296700 1 -- 172.21.15.16:0/1090139213 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 7103+0+0 (secure 0 0 0) 0x55a7e2cb4a80 con 0x55a7e2faa000 2023-05-20T01:22:17.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.798+0000 7fbd5d2f62c0 4 main: JournalTool: creating IoCtx.. 2023-05-20T01:22:17.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.798+0000 7fbd5d2f62c0 4 main: Executing for rank 0 2023-05-20T01:22:17.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.798+0000 7fbd5baa3700 1 --2- 172.21.15.16:0/1090139213 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x55a7e2fb0400 0x55a7e2d0f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:17.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.798+0000 7fbd5baa3700 1 --2- 172.21.15.16:0/1090139213 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x55a7e2fb0400 0x55a7e2d0f700 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x55a7e3aebc50 tx=0x55a7e3a0b140 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:17.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.799+0000 7fbd5d2f62c0 1 -- start start 2023-05-20T01:22:17.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.799+0000 7fbd5d2f62c0 1 Processor -- start 2023-05-20T01:22:17.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.799+0000 7fbd5d2f62c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a7e2faac00 0x55a7e2d0e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:17.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.799+0000 7fbd5d2f62c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a7e2fab000 0x55a7e2d0db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:17.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.799+0000 7fbd5d2f62c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a7e2fab400 0x55a7e2d0d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:17.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.799+0000 7fbd5d2f62c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7e2caec80 con 0x55a7e2fab400 2023-05-20T01:22:17.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.799+0000 7fbd5d2f62c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7e2cafcc0 con 0x55a7e2faac00 2023-05-20T01:22:17.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.799+0000 7fbd5d2f62c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a7e2caf400 con 0x55a7e2fab000 2023-05-20T01:22:17.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.799+0000 7fbd5aaa1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a7e2fab400 0x55a7e2d0d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:17.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.799+0000 7fbd5aaa1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a7e2fab400 0x55a7e2d0d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34564/0 (socket says 172.21.15.16:34564) 2023-05-20T01:22:17.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.799+0000 7fbd5aaa1700 1 -- 172.21.15.16:0/1498579469 learned_addr learned my addr 172.21.15.16:0/1498579469 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:22:17.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.799+0000 7fbd5baa3700 1 --2- 172.21.15.16:0/1498579469 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a7e2faac00 0x55a7e2d0e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:17.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.799+0000 7fbd5b2a2700 1 --2- 172.21.15.16:0/1498579469 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a7e2fab000 0x55a7e2d0db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:17.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.800+0000 7fbd5aaa1700 1 -- 172.21.15.16:0/1498579469 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a7e2fab000 msgr2=0x55a7e2d0db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:17.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.800+0000 7fbd5aaa1700 1 --2- 172.21.15.16:0/1498579469 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a7e2fab000 0x55a7e2d0db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:17.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.800+0000 7fbd5aaa1700 1 -- 172.21.15.16:0/1498579469 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a7e2faac00 msgr2=0x55a7e2d0e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:17.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.800+0000 7fbd5aaa1700 1 --2- 172.21.15.16:0/1498579469 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a7e2faac00 0x55a7e2d0e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:17.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.800+0000 7fbd5aaa1700 1 -- 172.21.15.16:0/1498579469 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a7e2fa6680 con 0x55a7e2fab400 2023-05-20T01:22:17.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.800+0000 7fbd5aaa1700 1 --2- 172.21.15.16:0/1498579469 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a7e2fab400 0x55a7e2d0d600 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x55a7e307a870 tx=0x55a7e3a0b440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20cffb9e69e13aa0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:17.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.800+0000 7fbd52290700 1 -- 172.21.15.16:0/1498579469 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55a7e2ca71e0 con 0x55a7e2fab400 2023-05-20T01:22:17.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.800+0000 7fbd52290700 1 -- 172.21.15.16:0/1498579469 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a7e3b00180 con 0x55a7e2fab400 2023-05-20T01:22:17.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.800+0000 7fbd52290700 1 -- 172.21.15.16:0/1498579469 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55a7e2ca6f20 con 0x55a7e2fab400 2023-05-20T01:22:17.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.800+0000 7fbd5d2f62c0 1 -- 172.21.15.16:0/1498579469 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a7e2fa6820 con 0x55a7e2fab400 2023-05-20T01:22:17.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.800+0000 7fbd5d2f62c0 1 -- 172.21.15.16:0/1498579469 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55a7e2fa69c0 con 0x55a7e2fab400 2023-05-20T01:22:17.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.800+0000 7fbd5d2f62c0 4 waiting for MDS map... 2023-05-20T01:22:17.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.800+0000 7fbd52290700 1 -- 172.21.15.16:0/1498579469 <== mon.0 v2:172.21.15.16:3300/0 4 ==== osd_map(71..71 src has 1..71) v4 ==== 7103+0+0 (secure 0 0 0) 0x55a7e2cb48c0 con 0x55a7e2fab400 2023-05-20T01:22:17.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.800+0000 7fbd52290700 1 -- 172.21.15.16:0/1498579469 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 42) v1 ==== 3126+0+0 (secure 0 0 0) 0x55a7e30eb500 con 0x55a7e2fab400 2023-05-20T01:22:17.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.800+0000 7fbd5d2f62c0 4 Got MDS map 42 2023-05-20T01:22:17.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.800+0000 7fbd5d2f62c0 1 --2- 172.21.15.16:0/1498579469 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x55a7e2fab800 0x55a7e311c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:17.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.800+0000 7fbd5d2f62c0 1 -- 172.21.15.16:0/1498579469 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:1 9.f 9:f1f69726:::400.00000000:head [writefull 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x55a7e3afd180 con 0x55a7e2fab800 2023-05-20T01:22:17.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.801+0000 7fbd5baa3700 1 --2- 172.21.15.16:0/1498579469 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x55a7e2fab800 0x55a7e311c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:17.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.801+0000 7fbd5baa3700 1 --2- 172.21.15.16:0/1498579469 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x55a7e2fab800 0x55a7e311c000 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=5ebbe8b08a653df0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:17.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.803+0000 7fbd5baa3700 1 -- 172.21.15.16:0/1498579469 <== osd.5 v2:172.21.15.132:6816/1844689995 1 ==== osd_op_reply(1 400.00000000 [writefull 0~22] v71'4 uv4 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x55a7e3ae9200 con 0x55a7e2fab800 2023-05-20T01:22:17.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.803+0000 7fbd5d2f62c0 1 --2- 172.21.15.16:0/1498579469 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x55a7e2fabc00 0x55a7e311c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:17.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.803+0000 7fbd5d2f62c0 1 -- 172.21.15.16:0/1498579469 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:2 9.14 9:292cf221:::200.00000000:head [writefull 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x55a7e3afd500 con 0x55a7e2fabc00 2023-05-20T01:22:17.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.803+0000 7fbd5b2a2700 1 --2- 172.21.15.16:0/1498579469 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x55a7e2fabc00 0x55a7e311c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:17.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.803+0000 7fbd5b2a2700 1 --2- 172.21.15.16:0/1498579469 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x55a7e2fabc00 0x55a7e311c580 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=fa38f45af985061b server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:17.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.805+0000 7fbd5b2a2700 1 -- 172.21.15.16:0/1498579469 <== osd.1 v2:172.21.15.16:6808/3860264220 1 ==== osd_op_reply(2 200.00000000 [writefull 0~90] v71'130 uv130 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x55a7e3ae9440 con 0x55a7e2fabc00 2023-05-20T01:22:17.830 INFO:teuthology.orchestra.run.smithi016.stdout:writing EResetJournal entry 2023-05-20T01:22:17.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.805+0000 7fbd5d2f62c0 1 -- 172.21.15.16:0/1498579469 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:3 9.14 9:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x55a7e3afd880 con 0x55a7e2fabc00 2023-05-20T01:22:17.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.805+0000 7fbd5d2f62c0 1 --2- 172.21.15.16:0/1498579469 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x55a7e2fb1000 0x55a7e311cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:17.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.805+0000 7fbd5d2f62c0 1 -- 172.21.15.16:0/1498579469 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:4 9.1b 9:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x55a7e3afdc00 con 0x55a7e2fb1000 2023-05-20T01:22:17.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.805+0000 7fbd5d2f62c0 1 --2- 172.21.15.16:0/1498579469 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x55a7e2fb1400 0x55a7e311d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:17.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.805+0000 7fbd5d2f62c0 1 -- 172.21.15.16:0/1498579469 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:5 9.b 9:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x55a7e3afd500 con 0x55a7e2fb1400 2023-05-20T01:22:17.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.805+0000 7fbd5d2f62c0 1 -- 172.21.15.16:0/1498579469 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:6 9.1f 9:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x55a7e3afd880 con 0x55a7e2fabc00 2023-05-20T01:22:17.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.805+0000 7fbd5d2f62c0 1 -- 172.21.15.16:0/1498579469 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:7 9.1a 9:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x55a7e3130000 con 0x55a7e2fb1000 2023-05-20T01:22:17.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.805+0000 7fbd5d2f62c0 1 -- 172.21.15.16:0/1498579469 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:8 9.1e 9:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x55a7e3130380 con 0x55a7e2fb1400 2023-05-20T01:22:17.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.805+0000 7fbd5b2a2700 1 --2- 172.21.15.16:0/1498579469 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x55a7e2fb1400 0x55a7e311d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:17.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.805+0000 7fbd5baa3700 1 --2- 172.21.15.16:0/1498579469 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x55a7e2fb1000 0x55a7e311cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:17.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.805+0000 7fbd5b2a2700 1 --2- 172.21.15.16:0/1498579469 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x55a7e2fb1400 0x55a7e311d080 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=fc854260af80a833 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:17.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.805+0000 7fbd5baa3700 1 --2- 172.21.15.16:0/1498579469 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x55a7e2fb1000 0x55a7e311cb00 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=d2a62d4fed3dfba server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:17.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.806+0000 7fbd5b2a2700 1 -- 172.21.15.16:0/1498579469 <== osd.1 v2:172.21.15.16:6808/3860264220 2 ==== osd_op_reply(3 200.00000001 [delete] v71'131 uv130 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x55a7e3ae9440 con 0x55a7e2fabc00 2023-05-20T01:22:17.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.806+0000 7fbd5b2a2700 1 -- 172.21.15.16:0/1498579469 <== osd.1 v2:172.21.15.16:6808/3860264220 3 ==== osd_op_reply(6 200.00000004 [delete] v71'9 uv8 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x55a7e3ae9440 con 0x55a7e2fabc00 2023-05-20T01:22:17.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.806+0000 7fbd5b2a2700 1 -- 172.21.15.16:0/1498579469 <== osd.0 v2:172.21.15.16:6824/3923634179 1 ==== osd_op_reply(5 200.00000003 [delete] v71'6 uv5 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x55a7e3ae9440 con 0x55a7e2fb1400 2023-05-20T01:22:17.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.807+0000 7fbd5baa3700 1 -- 172.21.15.16:0/1498579469 <== osd.3 v2:172.21.15.16:6800/3669781890 1 ==== osd_op_reply(7 200.00000005 [delete] v71'2 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x55a7e3ae9200 con 0x55a7e2fb1000 2023-05-20T01:22:17.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.807+0000 7fbd5baa3700 1 -- 172.21.15.16:0/1498579469 <== osd.3 v2:172.21.15.16:6800/3669781890 2 ==== osd_op_reply(4 200.00000002 [delete] v71'2 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x55a7e3ae9200 con 0x55a7e2fb1000 2023-05-20T01:22:17.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.807+0000 7fbd50a8d700 1 -- 172.21.15.16:0/1498579469 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:9 9.14 9:2e2fa760:::200.00000001:head [write 0~48 in=48b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x55a7e3afd180 con 0x55a7e2fabc00 2023-05-20T01:22:17.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.807+0000 7fbd5b2a2700 1 -- 172.21.15.16:0/1498579469 <== osd.0 v2:172.21.15.16:6824/3923634179 2 ==== osd_op_reply(8 200.00000006 [delete] v71'2 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x55a7e3ae9440 con 0x55a7e2fb1400 2023-05-20T01:22:17.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.808+0000 7fbd5b2a2700 1 -- 172.21.15.16:0/1498579469 <== osd.1 v2:172.21.15.16:6808/3860264220 4 ==== osd_op_reply(9 200.00000001 [write 0~48] v71'132 uv132 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x55a7e3ae9440 con 0x55a7e2fabc00 2023-05-20T01:22:17.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.808+0000 7fbd5d2f62c0 4 Successfully wrote new journal pointer and header for rank 3:0 2023-05-20T01:22:17.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.808+0000 7fbd5d2f62c0 1 -- 172.21.15.16:0/1498579469 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x55a7e2fb1400 msgr2=0x55a7e311d080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:17.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.808+0000 7fbd5d2f62c0 1 --2- 172.21.15.16:0/1498579469 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x55a7e2fb1400 0x55a7e311d080 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:17.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.808+0000 7fbd5d2f62c0 1 -- 172.21.15.16:0/1498579469 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x55a7e2fabc00 msgr2=0x55a7e311c580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:17.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.808+0000 7fbd5d2f62c0 1 --2- 172.21.15.16:0/1498579469 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x55a7e2fabc00 0x55a7e311c580 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:17.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.808+0000 7fbd5d2f62c0 1 -- 172.21.15.16:0/1498579469 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x55a7e2fb1000 msgr2=0x55a7e311cb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:17.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.808+0000 7fbd5d2f62c0 1 --2- 172.21.15.16:0/1498579469 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x55a7e2fb1000 0x55a7e311cb00 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:17.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.808+0000 7fbd5d2f62c0 1 -- 172.21.15.16:0/1498579469 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x55a7e2fab800 msgr2=0x55a7e311c000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:17.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.808+0000 7fbd5d2f62c0 1 --2- 172.21.15.16:0/1498579469 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x55a7e2fab800 0x55a7e311c000 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:17.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.808+0000 7fbd5aaa1700 1 -- 172.21.15.16:0/1498579469 reap_dead start 2023-05-20T01:22:17.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.808+0000 7fbd5aaa1700 1 -- 172.21.15.16:0/1498579469 reap_dead start 2023-05-20T01:22:17.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.808+0000 7fbd5d2f62c0 1 -- 172.21.15.16:0/1498579469 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a7e2fab400 msgr2=0x55a7e2d0d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:17.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.808+0000 7fbd5d2f62c0 1 --2- 172.21.15.16:0/1498579469 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a7e2fab400 0x55a7e2d0d600 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x55a7e307a870 tx=0x55a7e3a0b440 comp rx=0 tx=0).stop 2023-05-20T01:22:17.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.808+0000 7fbd5d2f62c0 1 -- 172.21.15.16:0/1498579469 shutdown_connections 2023-05-20T01:22:17.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.808+0000 7fbd5d2f62c0 1 --2- 172.21.15.16:0/1498579469 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a7e2fab400 0x55a7e2d0d600 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:17.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.808+0000 7fbd5d2f62c0 1 -- 172.21.15.16:0/1498579469 >> 172.21.15.16:0/1498579469 conn(0x55a7e2faa800 msgr2=0x55a7e2d44000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:17.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.808+0000 7fbd5d2f62c0 1 -- 172.21.15.16:0/1498579469 shutdown_connections 2023-05-20T01:22:17.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.808+0000 7fbd5d2f62c0 1 -- 172.21.15.16:0/1498579469 wait complete. 2023-05-20T01:22:17.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.809+0000 7fbd5d2f62c0 1 -- 172.21.15.16:0/1090139213 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x55a7e2fb0400 msgr2=0x55a7e2d0f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:17.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.809+0000 7fbd5d2f62c0 1 --2- 172.21.15.16:0/1090139213 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x55a7e2fb0400 0x55a7e2d0f700 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x55a7e3aebc50 tx=0x55a7e3a0b140 comp rx=0 tx=0).stop 2023-05-20T01:22:17.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.809+0000 7fbd5d2f62c0 1 -- 172.21.15.16:0/1090139213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a7e2faa000 msgr2=0x55a7e2d0ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:17.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.809+0000 7fbd5d2f62c0 1 --2- 172.21.15.16:0/1090139213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a7e2faa000 0x55a7e2d0ec00 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x55a7e2f8ac30 tx=0x55a7e3b0ad20 comp rx=0 tx=0).stop 2023-05-20T01:22:17.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.809+0000 7fbd5d2f62c0 1 -- 172.21.15.16:0/1090139213 shutdown_connections 2023-05-20T01:22:17.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.809+0000 7fbd5d2f62c0 1 --2- 172.21.15.16:0/1090139213 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a7e2faa400 0x55a7e2d0e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:17.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.809+0000 7fbd5d2f62c0 1 --2- 172.21.15.16:0/1090139213 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x55a7e2fb0400 0x55a7e2d0f700 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:17.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.809+0000 7fbd5d2f62c0 1 --2- 172.21.15.16:0/1090139213 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a7e2c72400 0x55a7e2d0f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:17.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.809+0000 7fbd5d2f62c0 1 --2- 172.21.15.16:0/1090139213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a7e2faa000 0x55a7e2d0ec00 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:17.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.809+0000 7fbd5d2f62c0 1 -- 172.21.15.16:0/1090139213 >> 172.21.15.16:0/1090139213 conn(0x55a7e2c73c00 msgr2=0x55a7e2d43800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:17.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.809+0000 7fbd5d2f62c0 1 -- 172.21.15.16:0/1090139213 shutdown_connections 2023-05-20T01:22:17.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.809+0000 7fbd5d2f62c0 1 -- 172.21.15.16:0/1090139213 wait complete. 2023-05-20T01:22:17.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.809+0000 7fbd5d2f62c0 1 -- 172.21.15.16:0/2303762132 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a7e2c72c00 msgr2=0x55a7e2d0cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:17.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.809+0000 7fbd5d2f62c0 1 --2- 172.21.15.16:0/2303762132 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a7e2c72c00 0x55a7e2d0cb00 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x55a7e3aea780 tx=0x55a7e3a0b2c0 comp rx=0 tx=0).stop 2023-05-20T01:22:17.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.809+0000 7fbd5d2f62c0 1 -- 172.21.15.16:0/2303762132 shutdown_connections 2023-05-20T01:22:17.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.809+0000 7fbd5d2f62c0 1 --2- 172.21.15.16:0/2303762132 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a7e2c72800 0x55a7e2d0d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:17.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.809+0000 7fbd5d2f62c0 1 --2- 172.21.15.16:0/2303762132 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a7e2c73000 0x55a7e2d0c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:17.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.809+0000 7fbd5d2f62c0 1 --2- 172.21.15.16:0/2303762132 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a7e2c72c00 0x55a7e2d0cb00 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:17.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.809+0000 7fbd5d2f62c0 1 -- 172.21.15.16:0/2303762132 >> 172.21.15.16:0/2303762132 conn(0x55a7e2c73400 msgr2=0x55a7e2d43000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:17.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.809+0000 7fbd5d2f62c0 1 -- 172.21.15.16:0/2303762132 shutdown_connections 2023-05-20T01:22:17.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.809+0000 7fbd5d2f62c0 1 -- 172.21.15.16:0/2303762132 wait complete. 2023-05-20T01:22:17.845 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'journal', 'reset', '--force'] in time 0:00:00.131067, result: writing EResetJournal entry 2023-05-20T01:22:17.846 DEBUG:teuthology.orchestra.run.smithi016:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 init 2023-05-20T01:22:17.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.909+0000 7f56e4af67c0 1 -- start start 2023-05-20T01:22:17.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.909+0000 7f56e4af67c0 1 Processor -- start 2023-05-20T01:22:17.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.910+0000 7f56e4af67c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563e6489f000 0x563e64938580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:17.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.910+0000 7f56e4af67c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563e6489ec00 0x563e64938b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:17.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.910+0000 7f56e4af67c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e6489e800 0x563e64939080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:17.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.910+0000 7f56e4af67c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e648dbcc0 con 0x563e6489e800 2023-05-20T01:22:17.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.910+0000 7f56e4af67c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e648db400 con 0x563e6489f000 2023-05-20T01:22:17.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.910+0000 7f56e4af67c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x563e648dbe00 con 0x563e6489ec00 2023-05-20T01:22:17.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.911+0000 7f56e32a5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e6489e800 0x563e64939080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:17.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.911+0000 7f56e32a5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e6489e800 0x563e64939080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34584/0 (socket says 172.21.15.16:34584) 2023-05-20T01:22:17.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.911+0000 7f56e32a5700 1 -- 172.21.15.16:0/345073892 learned_addr learned my addr 172.21.15.16:0/345073892 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:22:17.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.911+0000 7f56e22a3700 1 --2- 172.21.15.16:0/345073892 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563e6489ec00 0x563e64938b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:17.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.911+0000 7f56e2aa4700 1 --2- 172.21.15.16:0/345073892 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563e6489f000 0x563e64938580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:17.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.911+0000 7f56e32a5700 1 -- 172.21.15.16:0/345073892 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563e6489ec00 msgr2=0x563e64938b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:17.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.911+0000 7f56e32a5700 1 --2- 172.21.15.16:0/345073892 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563e6489ec00 0x563e64938b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:17.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.911+0000 7f56e32a5700 1 -- 172.21.15.16:0/345073892 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563e6489f000 msgr2=0x563e64938580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:17.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.911+0000 7f56e32a5700 1 --2- 172.21.15.16:0/345073892 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563e6489f000 0x563e64938580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:17.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.911+0000 7f56e32a5700 1 -- 172.21.15.16:0/345073892 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563e648c2d00 con 0x563e6489e800 2023-05-20T01:22:17.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.911+0000 7f56e22a3700 1 --2- 172.21.15.16:0/345073892 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563e6489ec00 0x563e64938b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:22:17.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.911+0000 7f56e32a5700 1 --2- 172.21.15.16:0/345073892 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e6489e800 0x563e64939080 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x563e65666d50 tx=0x563e656373e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28d25569ee829bd0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:17.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.911+0000 7f56e029f700 1 -- 172.21.15.16:0/345073892 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x563e648d3080 con 0x563e6489e800 2023-05-20T01:22:17.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.911+0000 7f56e029f700 1 -- 172.21.15.16:0/345073892 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x563e648f0600 con 0x563e6489e800 2023-05-20T01:22:17.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.911+0000 7f56e029f700 1 -- 172.21.15.16:0/345073892 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x563e648d3600 con 0x563e6489e800 2023-05-20T01:22:17.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.911+0000 7f56e4af67c0 1 -- 172.21.15.16:0/345073892 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563e648c36c0 con 0x563e6489e800 2023-05-20T01:22:17.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.911+0000 7f56e4af67c0 1 -- 172.21.15.16:0/345073892 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x563e648c3520 con 0x563e6489e800 2023-05-20T01:22:17.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.911+0000 7f56e4af67c0 4 waiting for MDS map... 2023-05-20T01:22:17.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.911+0000 7f56e029f700 1 -- 172.21.15.16:0/345073892 <== mon.0 v2:172.21.15.16:3300/0 4 ==== osd_map(71..71 src has 1..71) v4 ==== 7103+0+0 (secure 0 0 0) 0x563e648e0c40 con 0x563e6489e800 2023-05-20T01:22:17.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.911+0000 7f56e029f700 1 -- 172.21.15.16:0/345073892 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 42) v1 ==== 3126+0+0 (secure 0 0 0) 0x563e648e4f00 con 0x563e6489e800 2023-05-20T01:22:17.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.912+0000 7f56e4af67c0 4 Got MDS map 42 2023-05-20T01:22:17.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.912+0000 7f56e4af67c0 4 datascan.main: Using metadata pool output 2023-05-20T01:22:17.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.912+0000 7f56e4af67c0 4 datascan.main: connecting to RADOS... 2023-05-20T01:22:17.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.912+0000 7f56e4af67c0 1 Processor -- start 2023-05-20T01:22:17.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.912+0000 7f56e4af67c0 1 -- start start 2023-05-20T01:22:17.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.913+0000 7f56e4af67c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563e6489fc00 0x563e64939600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:17.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.913+0000 7f56e4af67c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e64bb4000 0x563e64939b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:17.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.913+0000 7f56e4af67c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563e64bb4400 0x563e6493a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:17.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.913+0000 7f56e4af67c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e648daa00 con 0x563e64bb4000 2023-05-20T01:22:17.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.913+0000 7f56e4af67c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e648db400 con 0x563e6489fc00 2023-05-20T01:22:17.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.913+0000 7f56e4af67c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x563e648dbcc0 con 0x563e64bb4400 2023-05-20T01:22:17.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.913+0000 7f56e2aa4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e64bb4000 0x563e64939b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:17.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.913+0000 7f56e2aa4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e64bb4000 0x563e64939b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34588/0 (socket says 172.21.15.16:34588) 2023-05-20T01:22:17.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.913+0000 7f56e2aa4700 1 -- 172.21.15.16:0/25070309 learned_addr learned my addr 172.21.15.16:0/25070309 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:22:17.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.913+0000 7f56e2aa4700 1 -- 172.21.15.16:0/25070309 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563e64bb4400 msgr2=0x563e6493a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:17.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.913+0000 7f56e2aa4700 1 --2- 172.21.15.16:0/25070309 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563e64bb4400 0x563e6493a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:17.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.913+0000 7f56e22a3700 1 --2- 172.21.15.16:0/25070309 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563e6489fc00 0x563e64939600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:17.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.913+0000 7f56e2aa4700 1 -- 172.21.15.16:0/25070309 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563e6489fc00 msgr2=0x563e64939600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:17.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.913+0000 7f56e2aa4700 1 --2- 172.21.15.16:0/25070309 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563e6489fc00 0x563e64939600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:17.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.913+0000 7f56e2aa4700 1 -- 172.21.15.16:0/25070309 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563e648c3380 con 0x563e64bb4000 2023-05-20T01:22:17.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.913+0000 7f56e2aa4700 1 --2- 172.21.15.16:0/25070309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e64bb4000 0x563e64939b80 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x563e64bbe0c0 tx=0x563e65734c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec10b627ca0ea232 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:17.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.913+0000 7f56de29b700 1 -- 172.21.15.16:0/25070309 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x563e648d3080 con 0x563e64bb4000 2023-05-20T01:22:17.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.913+0000 7f56de29b700 1 -- 172.21.15.16:0/25070309 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x563e648f1c80 con 0x563e64bb4000 2023-05-20T01:22:17.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.913+0000 7f56de29b700 1 -- 172.21.15.16:0/25070309 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x563e648d3600 con 0x563e64bb4000 2023-05-20T01:22:17.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.914+0000 7f56e4af67c0 1 -- 172.21.15.16:0/25070309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e64bb4000 msgr2=0x563e64939b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:17.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.914+0000 7f56e4af67c0 1 --2- 172.21.15.16:0/25070309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e64bb4000 0x563e64939b80 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x563e64bbe0c0 tx=0x563e65734c00 comp rx=0 tx=0).stop 2023-05-20T01:22:17.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.914+0000 7f56e4af67c0 1 -- 172.21.15.16:0/25070309 shutdown_connections 2023-05-20T01:22:17.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.914+0000 7f56e4af67c0 1 --2- 172.21.15.16:0/25070309 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563e64bb4400 0x563e6493a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:17.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.914+0000 7f56e4af67c0 1 --2- 172.21.15.16:0/25070309 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563e6489fc00 0x563e64939600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:17.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.914+0000 7f56e4af67c0 1 --2- 172.21.15.16:0/25070309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e64bb4000 0x563e64939b80 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:17.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.914+0000 7f56e4af67c0 1 -- 172.21.15.16:0/25070309 >> 172.21.15.16:0/25070309 conn(0x563e6489e400 msgr2=0x563e6497b800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:17.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.914+0000 7f56e4af67c0 1 -- 172.21.15.16:0/25070309 shutdown_connections 2023-05-20T01:22:17.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.914+0000 7f56e4af67c0 1 -- 172.21.15.16:0/25070309 wait complete. 2023-05-20T01:22:17.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.914+0000 7f56e4af67c0 1 Processor -- start 2023-05-20T01:22:17.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.914+0000 7f56e4af67c0 1 -- start start 2023-05-20T01:22:17.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.915+0000 7f56e4af67c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563e6489e400 0x563e6493b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:17.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.915+0000 7f56e4af67c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e64bb4000 0x563e6493ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:17.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.915+0000 7f56e4af67c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563e64bb4400 0x563e6493a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:17.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.915+0000 7f56e4af67c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e648dab40 con 0x563e64bb4000 2023-05-20T01:22:17.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.915+0000 7f56e4af67c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e648dac80 con 0x563e6489e400 2023-05-20T01:22:17.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.915+0000 7f56e4af67c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x563e648da000 con 0x563e64bb4400 2023-05-20T01:22:17.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.915+0000 7f56e2aa4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e64bb4000 0x563e6493ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:17.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.915+0000 7f56e2aa4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e64bb4000 0x563e6493ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34594/0 (socket says 172.21.15.16:34594) 2023-05-20T01:22:17.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.915+0000 7f56e2aa4700 1 -- 172.21.15.16:0/3883204751 learned_addr learned my addr 172.21.15.16:0/3883204751 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:22:17.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.915+0000 7f56e22a3700 1 --2- 172.21.15.16:0/3883204751 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563e6489e400 0x563e6493b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:17.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.915+0000 7f56e22a3700 1 --2- 172.21.15.16:0/3883204751 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563e64bb4400 0x563e6493a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:17.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.915+0000 7f56e2aa4700 1 -- 172.21.15.16:0/3883204751 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563e64bb4400 msgr2=0x563e6493a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:17.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.915+0000 7f56e2aa4700 1 --2- 172.21.15.16:0/3883204751 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563e64bb4400 0x563e6493a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:17.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.916+0000 7f56e2aa4700 1 -- 172.21.15.16:0/3883204751 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563e6489e400 msgr2=0x563e6493b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:17.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.916+0000 7f56e2aa4700 1 --2- 172.21.15.16:0/3883204751 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563e6489e400 0x563e6493b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:17.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.916+0000 7f56e2aa4700 1 -- 172.21.15.16:0/3883204751 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563e648c31e0 con 0x563e64bb4000 2023-05-20T01:22:17.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.916+0000 7f56e22a3700 1 --2- 172.21.15.16:0/3883204751 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563e64bb4400 0x563e6493a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:22:17.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.916+0000 7f56e2aa4700 1 --2- 172.21.15.16:0/3883204751 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e64bb4000 0x563e6493ac00 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x563e64bbf380 tx=0x563e65680de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:17.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.916+0000 7f56dca98700 1 -- 172.21.15.16:0/3883204751 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x563e648d2dc0 con 0x563e64bb4000 2023-05-20T01:22:17.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.916+0000 7f56dca98700 1 -- 172.21.15.16:0/3883204751 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x563e648f0480 con 0x563e64bb4000 2023-05-20T01:22:17.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.916+0000 7f56e4af67c0 1 -- 172.21.15.16:0/3883204751 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x563e648c3d40 con 0x563e64bb4000 2023-05-20T01:22:17.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.916+0000 7f56dca98700 1 -- 172.21.15.16:0/3883204751 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x563e648d3080 con 0x563e64bb4000 2023-05-20T01:22:17.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.916+0000 7f56e4af67c0 1 -- 172.21.15.16:0/3883204751 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563e648c2d00 con 0x563e64bb4000 2023-05-20T01:22:17.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.916+0000 7f56e4af67c0 4 datascan.init: resolving metadata pool 9 2023-05-20T01:22:17.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.917+0000 7f56dca98700 1 -- 172.21.15.16:0/3883204751 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x563e64900840 con 0x563e64bb4000 2023-05-20T01:22:17.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.917+0000 7f56dca98700 1 --2- 172.21.15.16:0/3883204751 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x563e64be2400 0x563e6493b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:17.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.917+0000 7f56dca98700 1 -- 172.21.15.16:0/3883204751 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 7103+0+0 (secure 0 0 0) 0x563e648e0a80 con 0x563e64bb4000 2023-05-20T01:22:17.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.918+0000 7f56e4af67c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-05-20T01:22:17.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.918+0000 7f56e22a3700 1 --2- 172.21.15.16:0/3883204751 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x563e64be2400 0x563e6493b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:17.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.918+0000 7f56e4af67c0 1 --2- 172.21.15.16:0/3883204751 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x563e64bb4800 0x563e6493a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:17.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.918+0000 7f56e4af67c0 1 -- 172.21.15.16:0/3883204751 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:1 9.14 9:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x563e65726700 con 0x563e64bb4800 2023-05-20T01:22:17.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.918+0000 7f56e22a3700 1 --2- 172.21.15.16:0/3883204751 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x563e64bb4800 0x563e6493a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:17.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.918+0000 7f56e22a3700 1 --2- 172.21.15.16:0/3883204751 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x563e64be2400 0x563e6493b700 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x563e65666120 tx=0x563e64cce180 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:17.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.918+0000 7f56e22a3700 1 --2- 172.21.15.16:0/3883204751 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x563e64bb4800 0x563e6493a100 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:17.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.918+0000 7f56e22a3700 1 -- 172.21.15.16:0/3883204751 <== osd.1 v2:172.21.15.16:6808/3860264220 1 ==== osd_op_reply(1 1.00000000.inode [stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 160+0+0 (crc 0 0 0) 0x563e64baf440 con 0x563e64bb4800 2023-05-20T01:22:17.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.918+0000 7f56e4af67c0 1 -- 172.21.15.16:0/3883204751 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:2 9.14 9:287034c4:::1.00000000.inode:head [writefull 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x563e65726a80 con 0x563e64bb4800 2023-05-20T01:22:17.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.920+0000 7f56e22a3700 1 -- 172.21.15.16:0/3883204751 <== osd.1 v2:172.21.15.16:6808/3860264220 2 ==== osd_op_reply(2 1.00000000.inode [writefull 0~582] v71'133 uv133 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x563e64baf440 con 0x563e64bb4800 2023-05-20T01:22:17.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.920+0000 7f56e4af67c0 1 -- 172.21.15.16:0/3883204751 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:3 9.1f 9:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x563e65726e00 con 0x563e64bb4800 2023-05-20T01:22:17.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.920+0000 7f56e22a3700 1 -- 172.21.15.16:0/3883204751 <== osd.1 v2:172.21.15.16:6808/3860264220 3 ==== osd_op_reply(3 100.00000000.inode [stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 162+0+0 (crc 0 0 0) 0x563e64baf440 con 0x563e64bb4800 2023-05-20T01:22:17.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.920+0000 7f56e4af67c0 1 -- 172.21.15.16:0/3883204751 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:4 9.1f 9:fe07bba1:::100.00000000.inode:head [writefull 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x563e65726700 con 0x563e64bb4800 2023-05-20T01:22:17.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.922+0000 7f56e22a3700 1 -- 172.21.15.16:0/3883204751 <== osd.1 v2:172.21.15.16:6808/3860264220 4 ==== osd_op_reply(4 100.00000000.inode [writefull 0~582] v71'10 uv10 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x563e64baf440 con 0x563e64bb4800 2023-05-20T01:22:17.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.922+0000 7f56e4af67c0 1 --2- 172.21.15.16:0/3883204751 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x563e64bb4c00 0x563e64939b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:17.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.922+0000 7f56e4af67c0 1 -- 172.21.15.16:0/3883204751 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:5 9.13 9:cd5a64a3:::100.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x563e65726a80 con 0x563e64bb4c00 2023-05-20T01:22:17.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.922+0000 7f56e32a5700 1 --2- 172.21.15.16:0/3883204751 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x563e64bb4c00 0x563e64939b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:17.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.922+0000 7f56e32a5700 1 --2- 172.21.15.16:0/3883204751 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x563e64bb4c00 0x563e64939b80 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:17.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.923+0000 7f56e32a5700 1 -- 172.21.15.16:0/3883204751 <== osd.7 v2:172.21.15.132:6800/466001149 1 ==== osd_op_reply(5 100.00000000 [omap-get-header] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x563e64bafb00 con 0x563e64bb4c00 2023-05-20T01:22:17.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.923+0000 7f56e4af67c0 10 datascan.find_or_create_dirfrag: read_version = 0 2023-05-20T01:22:17.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.923+0000 7f56e4af67c0 1 -- 172.21.15.16:0/3883204751 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:6 9.13 9:cd5a64a3:::100.00000000:head [create,omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x563e65727180 con 0x563e64bb4c00 2023-05-20T01:22:17.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.924+0000 7f56e32a5700 1 -- 172.21.15.16:0/3883204751 <== osd.7 v2:172.21.15.132:6800/466001149 2 ==== osd_op_reply(6 100.00000000 [create,omap-set-header] v71'7 uv7 ondisk = 0) v8 ==== 198+0+0 (crc 0 0 0) 0x563e64bafb00 con 0x563e64bb4c00 2023-05-20T01:22:17.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.924+0000 7f56e4af67c0 10 datascan.find_or_create_dirfrag: Created dirfrag: 0x0x100 2023-05-20T01:22:17.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.924+0000 7f56e4af67c0 1 -- 172.21.15.16:0/3883204751 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x563e64bb4800 msgr2=0x563e6493a100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:17.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.924+0000 7f56e4af67c0 1 --2- 172.21.15.16:0/3883204751 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x563e64bb4800 0x563e6493a100 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:17.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.924+0000 7f56e4af67c0 1 -- 172.21.15.16:0/3883204751 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x563e64bb4c00 msgr2=0x563e64939b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:17.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.924+0000 7f56e4af67c0 1 --2- 172.21.15.16:0/3883204751 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x563e64bb4c00 0x563e64939b80 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:17.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.924+0000 7f56e4af67c0 1 -- 172.21.15.16:0/3883204751 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x563e64be2400 msgr2=0x563e6493b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:17.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.924+0000 7f56e4af67c0 1 --2- 172.21.15.16:0/3883204751 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x563e64be2400 0x563e6493b700 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x563e65666120 tx=0x563e64cce180 comp rx=0 tx=0).stop 2023-05-20T01:22:17.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.924+0000 7f56e4af67c0 1 -- 172.21.15.16:0/3883204751 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e64bb4000 msgr2=0x563e6493ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:17.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.924+0000 7f56e2aa4700 1 -- 172.21.15.16:0/3883204751 reap_dead start 2023-05-20T01:22:17.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.924+0000 7f56e4af67c0 1 --2- 172.21.15.16:0/3883204751 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e64bb4000 0x563e6493ac00 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x563e64bbf380 tx=0x563e65680de0 comp rx=0 tx=0).stop 2023-05-20T01:22:17.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.924+0000 7f56e4af67c0 1 -- 172.21.15.16:0/3883204751 shutdown_connections 2023-05-20T01:22:17.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.924+0000 7f56e4af67c0 1 --2- 172.21.15.16:0/3883204751 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e64bb4000 0x563e6493ac00 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:17.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.924+0000 7f56e4af67c0 1 -- 172.21.15.16:0/3883204751 >> 172.21.15.16:0/3883204751 conn(0x563e6489fc00 msgr2=0x563e6497b800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:17.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.924+0000 7f56e4af67c0 1 -- 172.21.15.16:0/3883204751 shutdown_connections 2023-05-20T01:22:17.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.924+0000 7f56e4af67c0 1 -- 172.21.15.16:0/3883204751 wait complete. 2023-05-20T01:22:17.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.925+0000 7f56e4af67c0 1 -- 172.21.15.16:0/345073892 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e6489e800 msgr2=0x563e64939080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:17.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.925+0000 7f56e4af67c0 1 --2- 172.21.15.16:0/345073892 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e6489e800 0x563e64939080 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x563e65666d50 tx=0x563e656373e0 comp rx=0 tx=0).stop 2023-05-20T01:22:17.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.925+0000 7f56e4af67c0 1 -- 172.21.15.16:0/345073892 shutdown_connections 2023-05-20T01:22:17.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.925+0000 7f56e4af67c0 1 --2- 172.21.15.16:0/345073892 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e6489e800 0x563e64939080 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:17.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.925+0000 7f56e4af67c0 1 --2- 172.21.15.16:0/345073892 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563e6489f000 0x563e64938580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:17.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.925+0000 7f56e4af67c0 1 --2- 172.21.15.16:0/345073892 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563e6489ec00 0x563e64938b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:17.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.925+0000 7f56e4af67c0 1 -- 172.21.15.16:0/345073892 >> 172.21.15.16:0/345073892 conn(0x563e6489f400 msgr2=0x563e6497b000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:17.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.925+0000 7f56e4af67c0 1 -- 172.21.15.16:0/345073892 shutdown_connections 2023-05-20T01:22:17.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:17.925+0000 7f56e4af67c0 1 -- 172.21.15.16:0/345073892 wait complete. 2023-05-20T01:22:17.949 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'init'] in time 0:00:00.103678, result: 2023-05-20T01:22:17.949 DEBUG:teuthology.orchestra.run.smithi016:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_extents --worker_n 0 --worker_m 7 2023-05-20T01:22:17.950 DEBUG:teuthology.orchestra.run.smithi016:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_extents --worker_n 1 --worker_m 7 2023-05-20T01:22:17.951 DEBUG:teuthology.orchestra.run.smithi016:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_extents --worker_n 2 --worker_m 7 2023-05-20T01:22:17.951 DEBUG:teuthology.orchestra.run.smithi016:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_extents --worker_n 3 --worker_m 7 2023-05-20T01:22:17.951 DEBUG:teuthology.orchestra.run.smithi016:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_extents --worker_n 4 --worker_m 7 2023-05-20T01:22:17.952 DEBUG:teuthology.orchestra.run.smithi016:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_extents --worker_n 5 --worker_m 7 2023-05-20T01:22:17.952 DEBUG:teuthology.orchestra.run.smithi016:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_extents --worker_n 6 --worker_m 7 2023-05-20T01:22:18.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.043+0000 7fe20140e7c0 1 -- start start 2023-05-20T01:22:18.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.043+0000 7fe20140e7c0 1 Processor -- start 2023-05-20T01:22:18.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.044+0000 7fe20140e7c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5624b6a38c00 0x5624b6ad2b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.045+0000 7fe20140e7c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5624b6a39000 0x5624b6ad3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.045+0000 7fe20140e7c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624b6a39400 0x5624b6ad3600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.045+0000 7fe20140e7c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624b6a75400 con 0x5624b6a39400 2023-05-20T01:22:18.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.045+0000 7fe20140e7c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624b6a75cc0 con 0x5624b6a38c00 2023-05-20T01:22:18.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.045+0000 7fe20140e7c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x5624b6a74a00 con 0x5624b6a39000 2023-05-20T01:22:18.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.045+0000 7fe1ff3bc700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5624b6a38c00 0x5624b6ad2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.045+0000 7fe1febbb700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5624b6a39000 0x5624b6ad3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.045+0000 7fe1febbb700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5624b6a39000 0x5624b6ad3080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.16:34656/0 (socket says 172.21.15.16:34656) 2023-05-20T01:22:18.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.045+0000 7fe1febbb700 1 -- 172.21.15.16:0/652359026 learned_addr learned my addr 172.21.15.16:0/652359026 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:22:18.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.045+0000 7fe1ffbbd700 1 --2- 172.21.15.16:0/652359026 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624b6a39400 0x5624b6ad3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.045+0000 7fe1febbb700 1 -- 172.21.15.16:0/652359026 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5624b6a38c00 msgr2=0x5624b6ad2b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:18.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.045+0000 7fe1febbb700 1 --2- 172.21.15.16:0/652359026 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5624b6a38c00 0x5624b6ad2b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.045+0000 7fe1febbb700 1 -- 172.21.15.16:0/652359026 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624b6a39400 msgr2=0x5624b6ad3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:18.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.045+0000 7fe1febbb700 1 --2- 172.21.15.16:0/652359026 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624b6a39400 0x5624b6ad3600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.045+0000 7fe1febbb700 1 -- 172.21.15.16:0/652359026 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5624b6a5cd00 con 0x5624b6a39000 2023-05-20T01:22:18.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.046+0000 7fe1febbb700 1 --2- 172.21.15.16:0/652359026 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5624b6a39000 0x5624b6ad3080 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x5624b7814e10 tx=0x5624b781e480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e56e3293b5d4ddd server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.046+0000 7fe1fcbb7700 1 -- 172.21.15.16:0/652359026 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5624b6a6d600 con 0x5624b6a39000 2023-05-20T01:22:18.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.046+0000 7fe1fcbb7700 1 -- 172.21.15.16:0/652359026 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5624b6a8b980 con 0x5624b6a39000 2023-05-20T01:22:18.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.046+0000 7fe1fcbb7700 1 -- 172.21.15.16:0/652359026 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5624b6a6d1e0 con 0x5624b6a39000 2023-05-20T01:22:18.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.046+0000 7fe20140e7c0 1 -- 172.21.15.16:0/652359026 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5624b6a5d6c0 con 0x5624b6a39000 2023-05-20T01:22:18.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.046+0000 7fe20140e7c0 1 -- 172.21.15.16:0/652359026 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({fsmap=0}) v3 -- 0x5624b6a5d520 con 0x5624b6a39000 2023-05-20T01:22:18.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.046+0000 7fe20140e7c0 4 waiting for MDS map... 2023-05-20T01:22:18.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.047+0000 7ffaa34017c0 1 -- start start 2023-05-20T01:22:18.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.047+0000 7ffaa34017c0 1 Processor -- start 2023-05-20T01:22:18.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.047+0000 7fe1fcbb7700 1 -- 172.21.15.16:0/652359026 <== mon.2 v2:172.21.15.132:3301/0 4 ==== osd_map(71..71 src has 1..71) v4 ==== 7103+0+0 (secure 0 0 0) 0x5624b6a7ac40 con 0x5624b6a39000 2023-05-20T01:22:18.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.047+0000 7fe1fcbb7700 1 -- 172.21.15.16:0/652359026 <== mon.2 v2:172.21.15.132:3301/0 5 ==== fsmap(e 42) v1 ==== 3126+0+0 (secure 0 0 0) 0x5624b78de000 con 0x5624b6a39000 2023-05-20T01:22:18.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.047+0000 7fe20140e7c0 4 Got MDS map 42 2023-05-20T01:22:18.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.047+0000 7fe20140e7c0 4 datascan.main: Using metadata pool output 2023-05-20T01:22:18.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.047+0000 7fe20140e7c0 4 datascan.main: connecting to RADOS... 2023-05-20T01:22:18.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.048+0000 7fe20140e7c0 1 Processor -- start 2023-05-20T01:22:18.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.048+0000 7fe20140e7c0 1 -- start start 2023-05-20T01:22:18.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.048+0000 7ffaa34017c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560888f3b000 0x560888fd4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.048+0000 7ffaa34017c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x560888f3ac00 0x560888fd4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.048+0000 7ffaa34017c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560888f3a800 0x560888fd5080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.048+0000 7ffaa34017c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x560888f77cc0 con 0x560888f3a800 2023-05-20T01:22:18.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.048+0000 7ffaa34017c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x560888f77400 con 0x560888f3b000 2023-05-20T01:22:18.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.048+0000 7ffaa34017c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x560888f77e00 con 0x560888f3ac00 2023-05-20T01:22:18.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.048+0000 7ffaa1bb0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560888f3a800 0x560888fd5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.048+0000 7ffaa1bb0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560888f3a800 0x560888fd5080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34634/0 (socket says 172.21.15.16:34634) 2023-05-20T01:22:18.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.048+0000 7ffaa1bb0700 1 -- 172.21.15.16:0/590433309 learned_addr learned my addr 172.21.15.16:0/590433309 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:22:18.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.048+0000 7ffaa13af700 1 --2- 172.21.15.16:0/590433309 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560888f3b000 0x560888fd4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.048+0000 7ffaa0bae700 1 --2- 172.21.15.16:0/590433309 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x560888f3ac00 0x560888fd4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.049+0000 7ffaa1bb0700 1 -- 172.21.15.16:0/590433309 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x560888f3ac00 msgr2=0x560888fd4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:18.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.049+0000 7ffaa1bb0700 1 --2- 172.21.15.16:0/590433309 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x560888f3ac00 0x560888fd4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.049+0000 7ffaa1bb0700 1 -- 172.21.15.16:0/590433309 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560888f3b000 msgr2=0x560888fd4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:18.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.049+0000 7ffaa1bb0700 1 --2- 172.21.15.16:0/590433309 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560888f3b000 0x560888fd4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.049+0000 7ffaa1bb0700 1 -- 172.21.15.16:0/590433309 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560888f5ed00 con 0x560888f3a800 2023-05-20T01:22:18.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.049+0000 7ffaa1bb0700 1 --2- 172.21.15.16:0/590433309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560888f3a800 0x560888fd5080 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x56088909adb0 tx=0x5608890a4ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ac97a16b7b82150 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.049+0000 7fe20140e7c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624b6a39c00 0x5624b6ad3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.049+0000 7fe20140e7c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5624b78d2400 0x5624b6ad4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.049+0000 7fe20140e7c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5624b78d2800 0x5624b6ad4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.049+0000 7fe20140e7c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624b6a74b40 con 0x5624b6a39c00 2023-05-20T01:22:18.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.049+0000 7fe20140e7c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624b6a74c80 con 0x5624b78d2400 2023-05-20T01:22:18.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.049+0000 7fe20140e7c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x5624b6a74000 con 0x5624b78d2800 2023-05-20T01:22:18.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.049+0000 7fe1febbb700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5624b78d2800 0x5624b6ad4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.049+0000 7fe1febbb700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5624b78d2800 0x5624b6ad4680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.16:34676/0 (socket says 172.21.15.16:34676) 2023-05-20T01:22:18.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.049+0000 7fe1febbb700 1 -- 172.21.15.16:0/584035537 learned_addr learned my addr 172.21.15.16:0/584035537 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:22:18.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.050+0000 7fe1febbb700 1 -- 172.21.15.16:0/584035537 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5624b78d2400 msgr2=0x5624b6ad4100 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-05-20T01:22:18.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.050+0000 7fe1febbb700 1 --2- 172.21.15.16:0/584035537 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5624b78d2400 0x5624b6ad4100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.050+0000 7fe1febbb700 1 -- 172.21.15.16:0/584035537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624b6a39c00 msgr2=0x5624b6ad3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:18.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.050+0000 7fe1febbb700 1 --2- 172.21.15.16:0/584035537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624b6a39c00 0x5624b6ad3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.050+0000 7fe1febbb700 1 -- 172.21.15.16:0/584035537 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5624b6a5d380 con 0x5624b78d2800 2023-05-20T01:22:18.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.050+0000 7fe1febbb700 1 --2- 172.21.15.16:0/584035537 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5624b78d2800 0x5624b6ad4680 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x5624b6d70030 tx=0x5624b781e300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b0193fe9d30a211 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.050+0000 7fe1fabb3700 1 -- 172.21.15.16:0/584035537 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5624b6a6cf20 con 0x5624b78d2800 2023-05-20T01:22:18.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.050+0000 7fe1fabb3700 1 -- 172.21.15.16:0/584035537 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5624b78c8300 con 0x5624b78d2800 2023-05-20T01:22:18.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.050+0000 7fe1fabb3700 1 -- 172.21.15.16:0/584035537 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5624b6a6d600 con 0x5624b78d2800 2023-05-20T01:22:18.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.051+0000 7fe20140e7c0 1 -- 172.21.15.16:0/584035537 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5624b78d2800 msgr2=0x5624b6ad4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.051+0000 7fe20140e7c0 1 --2- 172.21.15.16:0/584035537 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5624b78d2800 0x5624b6ad4680 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x5624b6d70030 tx=0x5624b781e300 comp rx=0 tx=0).stop 2023-05-20T01:22:18.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.051+0000 7f7d9df997c0 1 -- start start 2023-05-20T01:22:18.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.051+0000 7f7d9df997c0 1 Processor -- start 2023-05-20T01:22:18.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.051+0000 7fe20140e7c0 1 -- 172.21.15.16:0/584035537 shutdown_connections 2023-05-20T01:22:18.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.051+0000 7fe20140e7c0 1 --2- 172.21.15.16:0/584035537 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5624b78d2800 0x5624b6ad4680 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.051+0000 7fe20140e7c0 1 --2- 172.21.15.16:0/584035537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624b6a39c00 0x5624b6ad3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.051+0000 7fe20140e7c0 1 --2- 172.21.15.16:0/584035537 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5624b78d2400 0x5624b6ad4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.051+0000 7fe20140e7c0 1 -- 172.21.15.16:0/584035537 >> 172.21.15.16:0/584035537 conn(0x5624b6a39800 msgr2=0x5624b6af9800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:18.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.051+0000 7ffa9ebaa700 1 -- 172.21.15.16:0/590433309 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x560888f6f600 con 0x560888f3a800 2023-05-20T01:22:18.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.051+0000 7ffa9ebaa700 1 -- 172.21.15.16:0/590433309 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560888f8c480 con 0x560888f3a800 2023-05-20T01:22:18.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.051+0000 7ffa9ebaa700 1 -- 172.21.15.16:0/590433309 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x560888f6f1e0 con 0x560888f3a800 2023-05-20T01:22:18.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.051+0000 7fe20140e7c0 1 -- 172.21.15.16:0/584035537 shutdown_connections 2023-05-20T01:22:18.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.051+0000 7fe20140e7c0 1 -- 172.21.15.16:0/584035537 wait complete. 2023-05-20T01:22:18.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.051+0000 7ffaa34017c0 1 -- 172.21.15.16:0/590433309 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560888f5f6c0 con 0x560888f3a800 2023-05-20T01:22:18.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.051+0000 7ffaa34017c0 1 -- 172.21.15.16:0/590433309 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x560888f5f520 con 0x560888f3a800 2023-05-20T01:22:18.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.051+0000 7ffaa34017c0 4 waiting for MDS map... 2023-05-20T01:22:18.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.052+0000 7fe20140e7c0 1 Processor -- start 2023-05-20T01:22:18.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.052+0000 7fe20140e7c0 1 -- start start 2023-05-20T01:22:18.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.052+0000 7f7d9df997c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55687ad32800 0x55687adcc580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.053+0000 7f7d9df997c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55687ad32c00 0x55687adccb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.053+0000 7f7d9df997c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55687ad33000 0x55687adcd080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.053+0000 7f7d9df997c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55687ad6fcc0 con 0x55687ad32c00 2023-05-20T01:22:18.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.053+0000 7f7d9df997c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55687ad6f400 con 0x55687ad32800 2023-05-20T01:22:18.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.053+0000 7f7d9df997c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55687ad6fe00 con 0x55687ad33000 2023-05-20T01:22:18.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.053+0000 7f7d9b746700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55687ad32c00 0x55687adccb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.053+0000 7f7d9b746700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55687ad32c00 0x55687adccb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34658/0 (socket says 172.21.15.16:34658) 2023-05-20T01:22:18.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.053+0000 7f7d9b746700 1 -- 172.21.15.16:0/3140357676 learned_addr learned my addr 172.21.15.16:0/3140357676 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:22:18.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.053+0000 7f7d9c748700 1 --2- 172.21.15.16:0/3140357676 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55687ad33000 0x55687adcd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.053+0000 7f7d9b746700 1 -- 172.21.15.16:0/3140357676 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55687ad33000 msgr2=0x55687adcd080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:18.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.053+0000 7f7d9b746700 1 --2- 172.21.15.16:0/3140357676 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55687ad33000 0x55687adcd080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.053+0000 7f7d9b746700 1 -- 172.21.15.16:0/3140357676 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55687ad32800 msgr2=0x55687adcc580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:18.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.053+0000 7f7d9b746700 1 --2- 172.21.15.16:0/3140357676 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55687ad32800 0x55687adcc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.053+0000 7f7d9b746700 1 -- 172.21.15.16:0/3140357676 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55687ad56d00 con 0x55687ad32c00 2023-05-20T01:22:18.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.053+0000 7fe20140e7c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5624b78d3000 0x5624b6ad3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.053+0000 7fe20140e7c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5624b78d2c00 0x5624b6ad4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.053+0000 7fe20140e7c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624b78d2800 0x5624b6ad4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.053+0000 7fe20140e7c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624b6a74a00 con 0x5624b78d2800 2023-05-20T01:22:18.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.053+0000 7fe20140e7c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624b6a74c80 con 0x5624b78d2c00 2023-05-20T01:22:18.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.053+0000 7fe20140e7c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x5624b6a74b40 con 0x5624b78d3000 2023-05-20T01:22:18.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.053+0000 7f7d9b746700 1 --2- 172.21.15.16:0/3140357676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55687ad32c00 0x55687adccb00 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x55687bb1aea0 tx=0x55687bb24360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8111d3316e6229 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.053+0000 7fe1febbb700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624b78d2800 0x5624b6ad4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.053+0000 7fe1febbb700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624b78d2800 0x5624b6ad4680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34662/0 (socket says 172.21.15.16:34662) 2023-05-20T01:22:18.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.053+0000 7fe1ff3bc700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5624b78d2c00 0x5624b6ad4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.053+0000 7fe1febbb700 1 -- 172.21.15.16:0/2808449806 learned_addr learned my addr 172.21.15.16:0/2808449806 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:22:18.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.054+0000 7fe1ffbbd700 1 --2- 172.21.15.16:0/2808449806 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5624b78d3000 0x5624b6ad3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.054+0000 7ffa9ebaa700 1 -- 172.21.15.16:0/590433309 <== mon.0 v2:172.21.15.16:3300/0 4 ==== osd_map(71..71 src has 1..71) v4 ==== 7103+0+0 (secure 0 0 0) 0x560888f7cc40 con 0x560888f3a800 2023-05-20T01:22:18.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.053+0000 7fee7eecb7c0 1 -- start start 2023-05-20T01:22:18.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.053+0000 7fee7eecb7c0 1 Processor -- start 2023-05-20T01:22:18.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.054+0000 7fe1ff3bc700 1 -- 172.21.15.16:0/2808449806 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5624b78d3000 msgr2=0x5624b6ad3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.054+0000 7fe1ff3bc700 1 --2- 172.21.15.16:0/2808449806 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5624b78d3000 0x5624b6ad3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.054+0000 7fe1ff3bc700 1 -- 172.21.15.16:0/2808449806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624b78d2800 msgr2=0x5624b6ad4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.054+0000 7fe1ff3bc700 1 --2- 172.21.15.16:0/2808449806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624b78d2800 0x5624b6ad4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.054+0000 7fe1ff3bc700 1 -- 172.21.15.16:0/2808449806 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5624b78bc4e0 con 0x5624b78d2c00 2023-05-20T01:22:18.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.054+0000 7fe1febbb700 1 --2- 172.21.15.16:0/2808449806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624b78d2800 0x5624b6ad4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-05-20T01:22:18.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.054+0000 7fee7eecb7c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e8d17d7000 0x55e8d1870580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.054+0000 7fee7eecb7c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8d17d6c00 0x55e8d1870b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.054+0000 7fee7eecb7c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e8d17d6800 0x55e8d1871080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.054+0000 7fee7eecb7c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8d1813cc0 con 0x55e8d17d6c00 2023-05-20T01:22:18.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.054+0000 7fee7eecb7c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8d1813400 con 0x55e8d17d6800 2023-05-20T01:22:18.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.054+0000 7fee7eecb7c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e8d1813e00 con 0x55e8d17d7000 2023-05-20T01:22:18.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.055+0000 7fe1ff3bc700 1 --2- 172.21.15.16:0/2808449806 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5624b78d2c00 0x5624b6ad4100 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x5624b6a935c0 tx=0x5624b781e000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.054+0000 7ffa9ebaa700 1 -- 172.21.15.16:0/590433309 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 42) v1 ==== 3126+0+0 (secure 0 0 0) 0x56088921a000 con 0x560888f3a800 2023-05-20T01:22:18.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.054+0000 7ffaa34017c0 4 Got MDS map 42 2023-05-20T01:22:18.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.054+0000 7ffaa34017c0 4 datascan.main: Using metadata pool output 2023-05-20T01:22:18.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.054+0000 7ffaa34017c0 4 datascan.main: connecting to RADOS... 2023-05-20T01:22:18.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.055+0000 7ffaa34017c0 1 Processor -- start 2023-05-20T01:22:18.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.054+0000 7fee7d67a700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e8d17d6800 0x55e8d1871080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.054+0000 7fee7ce79700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e8d17d7000 0x55e8d1870580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.054+0000 7fee7ce79700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e8d17d7000 0x55e8d1870580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.16:34712/0 (socket says 172.21.15.16:34712) 2023-05-20T01:22:18.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.054+0000 7fee7ce79700 1 -- 172.21.15.16:0/709612565 learned_addr learned my addr 172.21.15.16:0/709612565 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:22:18.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.054+0000 7fee7c678700 1 --2- 172.21.15.16:0/709612565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8d17d6c00 0x55e8d1870b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.054+0000 7f7d99742700 1 -- 172.21.15.16:0/3140357676 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55687ad67600 con 0x55687ad32c00 2023-05-20T01:22:18.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.054+0000 7f7d99742700 1 -- 172.21.15.16:0/3140357676 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55687ad84c00 con 0x55687ad32c00 2023-05-20T01:22:18.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.055+0000 7fee7ce79700 1 -- 172.21.15.16:0/709612565 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e8d17d6800 msgr2=0x55e8d1871080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:18.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.055+0000 7fee7ce79700 1 --2- 172.21.15.16:0/709612565 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e8d17d6800 0x55e8d1871080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.055+0000 7fee7ce79700 1 -- 172.21.15.16:0/709612565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8d17d6c00 msgr2=0x55e8d1870b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:18.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.054+0000 7f7d99742700 1 -- 172.21.15.16:0/3140357676 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55687ad671e0 con 0x55687ad32c00 2023-05-20T01:22:18.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.054+0000 7f7d9df997c0 1 -- 172.21.15.16:0/3140357676 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55687ad576c0 con 0x55687ad32c00 2023-05-20T01:22:18.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.054+0000 7f7d9df997c0 1 -- 172.21.15.16:0/3140357676 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55687ad57520 con 0x55687ad32c00 2023-05-20T01:22:18.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.054+0000 7f7d9df997c0 4 waiting for MDS map... 2023-05-20T01:22:18.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.055+0000 7fee7ce79700 1 --2- 172.21.15.16:0/709612565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8d17d6c00 0x55e8d1870b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.055+0000 7fee7ce79700 1 -- 172.21.15.16:0/709612565 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e8d17fad00 con 0x55e8d17d7000 2023-05-20T01:22:18.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.055+0000 7fe1f93b0700 1 -- 172.21.15.16:0/2808449806 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5624b6a6d080 con 0x5624b78d2c00 2023-05-20T01:22:18.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.055+0000 7fe1f93b0700 1 -- 172.21.15.16:0/2808449806 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5624b6a8a600 con 0x5624b78d2c00 2023-05-20T01:22:18.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.055+0000 7fe20140e7c0 1 -- 172.21.15.16:0/2808449806 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5624b78bc340 con 0x5624b78d2c00 2023-05-20T01:22:18.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.055+0000 7fee7ce79700 1 --2- 172.21.15.16:0/709612565 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e8d17d7000 0x55e8d1870580 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x55e8d1936cf0 tx=0x55e8d256f3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c215300971b7d52b server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.055+0000 7ffaa34017c0 1 -- start start 2023-05-20T01:22:18.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.055+0000 7fee7a674700 1 -- 172.21.15.16:0/709612565 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55e8d180b080 con 0x55e8d17d7000 2023-05-20T01:22:18.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.055+0000 7fee7a674700 1 -- 172.21.15.16:0/709612565 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e8d1829980 con 0x55e8d17d7000 2023-05-20T01:22:18.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.055+0000 7fee7a674700 1 -- 172.21.15.16:0/709612565 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55e8d180b600 con 0x55e8d17d7000 2023-05-20T01:22:18.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.055+0000 7fee7eecb7c0 1 -- 172.21.15.16:0/709612565 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e8d17fb6c0 con 0x55e8d17d7000 2023-05-20T01:22:18.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.055+0000 7fee7eecb7c0 1 -- 172.21.15.16:0/709612565 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55e8d17fb520 con 0x55e8d17d7000 2023-05-20T01:22:18.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.055+0000 7fee7eecb7c0 4 waiting for MDS map... 2023-05-20T01:22:18.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.056+0000 7fee7a674700 1 -- 172.21.15.16:0/709612565 <== mon.2 v2:172.21.15.132:3301/0 4 ==== osd_map(71..71 src has 1..71) v4 ==== 7103+0+0 (secure 0 0 0) 0x55e8d1818c40 con 0x55e8d17d7000 2023-05-20T01:22:18.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.055+0000 7fe1f93b0700 1 -- 172.21.15.16:0/2808449806 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5624b6a6cf20 con 0x5624b78d2c00 2023-05-20T01:22:18.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.055+0000 7fe20140e7c0 1 -- 172.21.15.16:0/2808449806 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5624b78bc680 con 0x5624b78d2c00 2023-05-20T01:22:18.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.055+0000 7fe20140e7c0 4 datascan.init: resolving metadata pool 9 2023-05-20T01:22:18.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.056+0000 7fee7a674700 1 -- 172.21.15.16:0/709612565 <== mon.2 v2:172.21.15.132:3301/0 5 ==== fsmap(e 42) v1 ==== 3126+0+0 (secure 0 0 0) 0x55e8d181cf00 con 0x55e8d17d7000 2023-05-20T01:22:18.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.056+0000 7fee7eecb7c0 4 Got MDS map 42 2023-05-20T01:22:18.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.056+0000 7ffaa34017c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560888f3bc00 0x560888fd5600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.056+0000 7ffaa34017c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56088926c000 0x560888fd5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.056+0000 7ffaa34017c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56088926c400 0x560888fd6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.056+0000 7ffaa34017c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x560888f76a00 con 0x560888f3bc00 2023-05-20T01:22:18.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.056+0000 7ffaa34017c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x560888f77400 con 0x56088926c400 2023-05-20T01:22:18.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.056+0000 7ffaa34017c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x560888f76b40 con 0x56088926c000 2023-05-20T01:22:18.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.056+0000 7f7d99742700 1 -- 172.21.15.16:0/3140357676 <== mon.0 v2:172.21.15.16:3300/0 4 ==== osd_map(71..71 src has 1..71) v4 ==== 7103+0+0 (secure 0 0 0) 0x55687ad74c40 con 0x55687ad32c00 2023-05-20T01:22:18.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.056+0000 7f7d99742700 1 -- 172.21.15.16:0/3140357676 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 42) v1 ==== 3126+0+0 (secure 0 0 0) 0x55687ae98000 con 0x55687ad32c00 2023-05-20T01:22:18.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.056+0000 7f7d9df997c0 4 Got MDS map 42 2023-05-20T01:22:18.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.056+0000 7f70438bd7c0 1 -- start start 2023-05-20T01:22:18.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.056+0000 7f70438bd7c0 1 Processor -- start 2023-05-20T01:22:18.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.056+0000 7fee7eecb7c0 4 datascan.main: Using metadata pool output 2023-05-20T01:22:18.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.056+0000 7fee7eecb7c0 4 datascan.main: connecting to RADOS... 2023-05-20T01:22:18.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.056+0000 7ffaa13af700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56088926c000 0x560888fd5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.056+0000 7ffaa13af700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56088926c000 0x560888fd5b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.16:34718/0 (socket says 172.21.15.16:34718) 2023-05-20T01:22:18.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.056+0000 7ffaa13af700 1 -- 172.21.15.16:0/1940461941 learned_addr learned my addr 172.21.15.16:0/1940461941 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:22:18.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.057+0000 7ffaa0bae700 1 --2- 172.21.15.16:0/1940461941 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56088926c400 0x560888fd6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.057+0000 7fee7eecb7c0 1 Processor -- start 2023-05-20T01:22:18.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.057+0000 7ffaa13af700 1 -- 172.21.15.16:0/1940461941 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56088926c400 msgr2=0x560888fd6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:18.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.057+0000 7ffaa13af700 1 --2- 172.21.15.16:0/1940461941 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56088926c400 0x560888fd6100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.057+0000 7ffaa13af700 1 -- 172.21.15.16:0/1940461941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560888f3bc00 msgr2=0x560888fd5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:18.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.057+0000 7ffaa13af700 1 --2- 172.21.15.16:0/1940461941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560888f3bc00 0x560888fd5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.057+0000 7ffaa13af700 1 -- 172.21.15.16:0/1940461941 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560888f5f380 con 0x56088926c000 2023-05-20T01:22:18.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.057+0000 7fee7eecb7c0 1 -- start start 2023-05-20T01:22:18.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.057+0000 7ffaa13af700 1 --2- 172.21.15.16:0/1940461941 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56088926c000 0x560888fd5b80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x56088927a0c0 tx=0x560889cd33e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c532bc2df6f94a10 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.057+0000 7ffa9cba6700 1 -- 172.21.15.16:0/1940461941 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x560888f6f080 con 0x56088926c000 2023-05-20T01:22:18.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.057+0000 7ffa9cba6700 1 -- 172.21.15.16:0/1940461941 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560889220c00 con 0x56088926c000 2023-05-20T01:22:18.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.057+0000 7ffa9cba6700 1 -- 172.21.15.16:0/1940461941 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x560888f6f600 con 0x56088926c000 2023-05-20T01:22:18.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.056+0000 7f7d9df997c0 4 datascan.main: Using metadata pool output 2023-05-20T01:22:18.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.056+0000 7f7d9df997c0 4 datascan.main: connecting to RADOS... 2023-05-20T01:22:18.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.057+0000 7f7d9df997c0 1 Processor -- start 2023-05-20T01:22:18.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.057+0000 7f5a55a4d7c0 1 -- start start 2023-05-20T01:22:18.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.057+0000 7f5a55a4d7c0 1 Processor -- start 2023-05-20T01:22:18.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.057+0000 7f70438bd7c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f909993000 0x55f909a2c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.057+0000 7f70438bd7c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55f909992c00 0x55f909a2cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.057+0000 7f70438bd7c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f909992800 0x55f909a2d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.057+0000 7f70438bd7c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9099cfcc0 con 0x55f909993000 2023-05-20T01:22:18.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.057+0000 7f70438bd7c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9099cfe00 con 0x55f909992800 2023-05-20T01:22:18.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.057+0000 7f70438bd7c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f9099cf400 con 0x55f909992c00 2023-05-20T01:22:18.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.058+0000 7f7d9df997c0 1 -- start start 2023-05-20T01:22:18.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.058+0000 7f704186b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f909993000 0x55f909a2c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.058+0000 7f704186b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f909993000 0x55f909a2c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34686/0 (socket says 172.21.15.16:34686) 2023-05-20T01:22:18.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.058+0000 7f704186b700 1 -- 172.21.15.16:0/1197830548 learned_addr learned my addr 172.21.15.16:0/1197830548 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:22:18.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.058+0000 7fee7eecb7c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e8d17d7c00 0x55e8d1871600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.058+0000 7fee7eecb7c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e8d1afe000 0x55e8d1871b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.058+0000 7fee7eecb7c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8d1afe400 0x55e8d1872100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.058+0000 7fee7eecb7c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8d1812a00 con 0x55e8d1afe400 2023-05-20T01:22:18.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.058+0000 7fee7eecb7c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8d1813cc0 con 0x55e8d17d7c00 2023-05-20T01:22:18.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.058+0000 7fee7eecb7c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e8d1813e00 con 0x55e8d1afe000 2023-05-20T01:22:18.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.058+0000 7fee7c678700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8d1afe400 0x55e8d1872100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.058+0000 7fee7d67a700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e8d17d7c00 0x55e8d1871600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.058+0000 7fee7c678700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8d1afe400 0x55e8d1872100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34696/0 (socket says 172.21.15.16:34696) 2023-05-20T01:22:18.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.058+0000 7fee7c678700 1 -- 172.21.15.16:0/546353635 learned_addr learned my addr 172.21.15.16:0/546353635 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:22:18.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.058+0000 7fe1f93b0700 1 -- 172.21.15.16:0/2808449806 <== mon.1 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x5624b6a9a840 con 0x5624b78d2c00 2023-05-20T01:22:18.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.058+0000 7fe1f93b0700 1 --2- 172.21.15.16:0/2808449806 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x5624b6e76000 0x5624b6ad5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.058+0000 7fe1f93b0700 1 -- 172.21.15.16:0/2808449806 <== mon.1 v2:172.21.15.132:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 7103+0+0 (secure 0 0 0) 0x5624b6a7aa80 con 0x5624b78d2c00 2023-05-20T01:22:18.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.058+0000 7ffaa34017c0 1 -- 172.21.15.16:0/1940461941 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56088926c000 msgr2=0x560888fd5b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.058+0000 7ffaa34017c0 1 --2- 172.21.15.16:0/1940461941 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56088926c000 0x560888fd5b80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x56088927a0c0 tx=0x560889cd33e0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.058+0000 7fe20140e7c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-05-20T01:22:18.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.058+0000 7fe20140e7c0 4 datascan.main: data pool 'cephfs_data' has ID 10 2023-05-20T01:22:18.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.058+0000 7fe20140e7c0 4 datascan.main: opening data pool 'cephfs_data' 2023-05-20T01:22:18.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.058+0000 7fe20140e7c0 4 datascan.main: data pool 'cephfs_data_extra' has ID 11 2023-05-20T01:22:18.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.058+0000 7fe20140e7c0 4 datascan.main: opening data pool 'cephfs_data_extra' 2023-05-20T01:22:18.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.058+0000 7fe20140e7c0 1 --2- 172.21.15.16:0/2808449806 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x5624b78d2400 0x5624b6ad4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.058+0000 7fe20140e7c0 1 -- 172.21.15.16:0/2808449806 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:1 10.0 10:00000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x5624b78c6700 con 0x5624b78d2400 2023-05-20T01:22:18.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.058+0000 7f704106a700 1 --2- 172.21.15.16:0/1197830548 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55f909992c00 0x55f909a2cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.058+0000 7f704186b700 1 -- 172.21.15.16:0/1197830548 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55f909992c00 msgr2=0x55f909a2cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:18.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.058+0000 7f704186b700 1 --2- 172.21.15.16:0/1197830548 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55f909992c00 0x55f909a2cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.058+0000 7f704186b700 1 -- 172.21.15.16:0/1197830548 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f909992800 msgr2=0x55f909a2d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:18.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.058+0000 7f704186b700 1 --2- 172.21.15.16:0/1197830548 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f909992800 0x55f909a2d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.058+0000 7f704186b700 1 -- 172.21.15.16:0/1197830548 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f9099b64e0 con 0x55f909993000 2023-05-20T01:22:18.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.059+0000 7fe1ffbbd700 1 --2- 172.21.15.16:0/2808449806 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x5624b6e76000 0x5624b6ad5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.059+0000 7fe1febbb700 1 --2- 172.21.15.16:0/2808449806 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x5624b78d2400 0x5624b6ad4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.059+0000 7fe1ffbbd700 1 --2- 172.21.15.16:0/2808449806 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x5624b6e76000 0x5624b6ad5700 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x5624b7814c60 tx=0x5624b6e7c420 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.060+0000 7fe1febbb700 1 --2- 172.21.15.16:0/2808449806 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x5624b78d2400 0x5624b6ad4c00 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.060+0000 7ffaa34017c0 1 -- 172.21.15.16:0/1940461941 shutdown_connections 2023-05-20T01:22:18.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.060+0000 7ffaa34017c0 1 --2- 172.21.15.16:0/1940461941 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56088926c400 0x560888fd6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.060+0000 7ffaa34017c0 1 --2- 172.21.15.16:0/1940461941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560888f3bc00 0x560888fd5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.060+0000 7ffaa34017c0 1 --2- 172.21.15.16:0/1940461941 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56088926c000 0x560888fd5b80 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.060+0000 7ffaa34017c0 1 -- 172.21.15.16:0/1940461941 >> 172.21.15.16:0/1940461941 conn(0x560888f3a400 msgr2=0x560889017800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:18.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.060+0000 7ffaa34017c0 1 -- 172.21.15.16:0/1940461941 shutdown_connections 2023-05-20T01:22:18.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.060+0000 7ffaa34017c0 1 -- 172.21.15.16:0/1940461941 wait complete. 2023-05-20T01:22:18.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.058+0000 7fee7ce79700 1 --2- 172.21.15.16:0/546353635 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e8d1afe000 0x55e8d1871b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.058+0000 7fee7c678700 1 -- 172.21.15.16:0/546353635 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e8d1afe000 msgr2=0x55e8d1871b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:18.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.058+0000 7fee7c678700 1 --2- 172.21.15.16:0/546353635 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e8d1afe000 0x55e8d1871b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.058+0000 7fee7c678700 1 -- 172.21.15.16:0/546353635 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e8d17d7c00 msgr2=0x55e8d1871600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:18.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.058+0000 7fee7c678700 1 --2- 172.21.15.16:0/546353635 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e8d17d7c00 0x55e8d1871600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.059+0000 7fee7c678700 1 -- 172.21.15.16:0/546353635 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e8d17fb380 con 0x55e8d1afe400 2023-05-20T01:22:18.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.059+0000 7fee7c678700 1 --2- 172.21.15.16:0/546353635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8d1afe400 0x55e8d1872100 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x55e8d1b080f0 tx=0x55e8d267ac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99cb5de085d56db1 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.059+0000 7fee7d67a700 1 --2- 172.21.15.16:0/546353635 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e8d17d7c00 0x55e8d1871600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:22:18.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.059+0000 7fee78670700 1 -- 172.21.15.16:0/546353635 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55e8d180b080 con 0x55e8d1afe400 2023-05-20T01:22:18.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.059+0000 7fee78670700 1 -- 172.21.15.16:0/546353635 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e8d1829e00 con 0x55e8d1afe400 2023-05-20T01:22:18.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.059+0000 7fee78670700 1 -- 172.21.15.16:0/546353635 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55e8d180b600 con 0x55e8d1afe400 2023-05-20T01:22:18.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.058+0000 7f7d9df997c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55687ad33c00 0x55687adcd600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.058+0000 7f7d9df997c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55687b050000 0x55687adcdb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.059+0000 7f7d9df997c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55687b050400 0x55687adce100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.059+0000 7f7d9df997c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55687ad6ea00 con 0x55687ad33c00 2023-05-20T01:22:18.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.059+0000 7f7d9df997c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55687ad6f400 con 0x55687b050000 2023-05-20T01:22:18.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.059+0000 7f7d9df997c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55687ad6eb40 con 0x55687b050400 2023-05-20T01:22:18.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.059+0000 7f7d9c748700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55687ad33c00 0x55687adcd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.059+0000 7f7d9c748700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55687ad33c00 0x55687adcd600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34700/0 (socket says 172.21.15.16:34700) 2023-05-20T01:22:18.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.059+0000 7f7d9c748700 1 -- 172.21.15.16:0/54916626 learned_addr learned my addr 172.21.15.16:0/54916626 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:22:18.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.059+0000 7f7d9bf47700 1 --2- 172.21.15.16:0/54916626 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55687b050000 0x55687adcdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.059+0000 7f7d9b746700 1 --2- 172.21.15.16:0/54916626 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55687b050400 0x55687adce100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.059+0000 7f7d9bf47700 1 -- 172.21.15.16:0/54916626 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55687b050400 msgr2=0x55687adce100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:18.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.059+0000 7f7d9bf47700 1 --2- 172.21.15.16:0/54916626 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55687b050400 0x55687adce100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.059+0000 7f7d9bf47700 1 -- 172.21.15.16:0/54916626 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55687ad33c00 msgr2=0x55687adcd600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:18.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.059+0000 7f7d9bf47700 1 --2- 172.21.15.16:0/54916626 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55687ad33c00 0x55687adcd600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.059+0000 7f7d9bf47700 1 -- 172.21.15.16:0/54916626 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55687ad57380 con 0x55687b050000 2023-05-20T01:22:18.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.060+0000 7f7d9b746700 1 --2- 172.21.15.16:0/54916626 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55687b050400 0x55687adce100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:22:18.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.059+0000 7f5a55a4d7c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559726c94800 0x559726d2e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.059+0000 7f5a55a4d7c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559726c94c00 0x559726d2eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.059+0000 7f5a55a4d7c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559726c95000 0x559726d2f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.059+0000 7f5a55a4d7c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x559726cd1cc0 con 0x559726c94c00 2023-05-20T01:22:18.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.059+0000 7f5a55a4d7c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x559726cd1400 con 0x559726c95000 2023-05-20T01:22:18.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.059+0000 7f5a55a4d7c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x559726cd1e00 con 0x559726c94800 2023-05-20T01:22:18.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.060+0000 7f5a539fb700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559726c94800 0x559726d2e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.060+0000 7f5a539fb700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559726c94800 0x559726d2e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.16:34746/0 (socket says 172.21.15.16:34746) 2023-05-20T01:22:18.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.060+0000 7f5a539fb700 1 -- 172.21.15.16:0/544605894 learned_addr learned my addr 172.21.15.16:0/544605894 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:22:18.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.060+0000 7f5a541fc700 1 --2- 172.21.15.16:0/544605894 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559726c95000 0x559726d2f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.060+0000 7f5a541fc700 1 -- 172.21.15.16:0/544605894 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559726c94800 msgr2=0x559726d2e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:18.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.060+0000 7f5a541fc700 1 --2- 172.21.15.16:0/544605894 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559726c94800 0x559726d2e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.060+0000 7f5a541fc700 1 -- 172.21.15.16:0/544605894 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559726c94c00 msgr2=0x559726d2eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:18.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.060+0000 7f5a541fc700 1 --2- 172.21.15.16:0/544605894 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559726c94c00 0x559726d2eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.060+0000 7f5a541fc700 1 -- 172.21.15.16:0/544605894 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559726cb8d00 con 0x559726c95000 2023-05-20T01:22:18.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.058+0000 7f704186b700 1 --2- 172.21.15.16:0/1197830548 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f909993000 0x55f909a2c580 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x55f909aeed20 tx=0x55f90a72b3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=195c80f3aaba7e33 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.059+0000 7f703f066700 1 -- 172.21.15.16:0/1197830548 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55f9099c7080 con 0x55f909993000 2023-05-20T01:22:18.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.059+0000 7f703f066700 1 -- 172.21.15.16:0/1197830548 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f9099e5980 con 0x55f909993000 2023-05-20T01:22:18.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.059+0000 7f703f066700 1 -- 172.21.15.16:0/1197830548 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55f9099c7600 con 0x55f909993000 2023-05-20T01:22:18.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.059+0000 7f70438bd7c0 1 -- 172.21.15.16:0/1197830548 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f9099b7520 con 0x55f909993000 2023-05-20T01:22:18.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.059+0000 7f70438bd7c0 1 -- 172.21.15.16:0/1197830548 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55f9099b7380 con 0x55f909993000 2023-05-20T01:22:18.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.059+0000 7f70438bd7c0 4 waiting for MDS map... 2023-05-20T01:22:18.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.060+0000 7f703f066700 1 -- 172.21.15.16:0/1197830548 <== mon.0 v2:172.21.15.16:3300/0 4 ==== osd_map(71..71 src has 1..71) v4 ==== 7103+0+0 (secure 0 0 0) 0x55f9099d4c40 con 0x55f909993000 2023-05-20T01:22:18.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.060+0000 7f703f066700 1 -- 172.21.15.16:0/1197830548 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 42) v1 ==== 3126+0+0 (secure 0 0 0) 0x55f9099d8f00 con 0x55f909993000 2023-05-20T01:22:18.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.060+0000 7f70438bd7c0 4 Got MDS map 42 2023-05-20T01:22:18.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.060+0000 7f70438bd7c0 4 datascan.main: Using metadata pool output 2023-05-20T01:22:18.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.060+0000 7f70438bd7c0 4 datascan.main: connecting to RADOS... 2023-05-20T01:22:18.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.060+0000 7f61961d87c0 1 -- start start 2023-05-20T01:22:18.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.060+0000 7f61961d87c0 1 Processor -- start 2023-05-20T01:22:18.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.060+0000 7fee7eecb7c0 1 -- 172.21.15.16:0/546353635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8d1afe400 msgr2=0x55e8d1872100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.060+0000 7fee7eecb7c0 1 --2- 172.21.15.16:0/546353635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8d1afe400 0x55e8d1872100 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x55e8d1b080f0 tx=0x55e8d267ac00 comp rx=0 tx=0).stop 2023-05-20T01:22:18.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.060+0000 7f70438bd7c0 1 Processor -- start 2023-05-20T01:22:18.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.060+0000 7f5a541fc700 1 --2- 172.21.15.16:0/544605894 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559726c95000 0x559726d2f080 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x559727a6ae40 tx=0x559727a86480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad6cdf1357a23538 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.061+0000 7f7d9bf47700 1 --2- 172.21.15.16:0/54916626 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55687b050000 0x55687adcdb80 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x55687b066000 tx=0x55687bbc8d20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e87337de270ba2a server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.061+0000 7fee7eecb7c0 1 -- 172.21.15.16:0/546353635 shutdown_connections 2023-05-20T01:22:18.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.061+0000 7fee7eecb7c0 1 --2- 172.21.15.16:0/546353635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8d1afe400 0x55e8d1872100 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.061+0000 7fee7eecb7c0 1 --2- 172.21.15.16:0/546353635 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e8d17d7c00 0x55e8d1871600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.061+0000 7fee7eecb7c0 1 --2- 172.21.15.16:0/546353635 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e8d1afe000 0x55e8d1871b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.061+0000 7fee7eecb7c0 1 -- 172.21.15.16:0/546353635 >> 172.21.15.16:0/546353635 conn(0x55e8d17d6400 msgr2=0x55e8d18b3800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:18.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.061+0000 7ffaa34017c0 1 Processor -- start 2023-05-20T01:22:18.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.061+0000 7f7d9773e700 1 -- 172.21.15.16:0/54916626 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55687ad67600 con 0x55687b050000 2023-05-20T01:22:18.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.061+0000 7f5a511f6700 1 -- 172.21.15.16:0/544605894 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x559726cc9600 con 0x559726c95000 2023-05-20T01:22:18.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.061+0000 7ffaa34017c0 1 -- start start 2023-05-20T01:22:18.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.061+0000 7f7d9773e700 1 -- 172.21.15.16:0/54916626 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55687bbbec00 con 0x55687b050000 2023-05-20T01:22:18.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.061+0000 7f7d9773e700 1 -- 172.21.15.16:0/54916626 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55687ad671e0 con 0x55687b050000 2023-05-20T01:22:18.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.061+0000 7f61961d87c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x564461af3000 0x564461b8cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.061+0000 7f61961d87c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564461af2c00 0x564461b8c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.061+0000 7f61961d87c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564461af2800 0x564461b8d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.061+0000 7f61961d87c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x564461b2fcc0 con 0x564461af2800 2023-05-20T01:22:18.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.061+0000 7f61961d87c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x564461b2fe00 con 0x564461af2c00 2023-05-20T01:22:18.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.061+0000 7f61961d87c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x564461b2f400 con 0x564461af3000 2023-05-20T01:22:18.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.061+0000 7fee7eecb7c0 1 -- 172.21.15.16:0/546353635 shutdown_connections 2023-05-20T01:22:18.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.061+0000 7f6194186700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x564461af3000 0x564461b8cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.061+0000 7f6193985700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564461af2c00 0x564461b8c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.061+0000 7f6193985700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564461af2c00 0x564461b8c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.16:59970/0 (socket says 172.21.15.16:59970) 2023-05-20T01:22:18.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.061+0000 7f6193985700 1 -- 172.21.15.16:0/3109423973 learned_addr learned my addr 172.21.15.16:0/3109423973 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:22:18.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.062+0000 7fe1febbb700 1 -- 172.21.15.16:0/2808449806 <== osd.6 v2:172.21.15.132:6801/727581123 1 ==== osd_op_reply(1 [pgnls start_epoch 71 out=81b] v69'9 uv9 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5624b6d67440 con 0x5624b78d2400 2023-05-20T01:22:18.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.062+0000 7fe20140e7c0 1 -- 172.21.15.16:0/2808449806 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:2 10.0 10:012cdbc7:::10000000206.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x5624b78c6a80 con 0x5624b78d2400 2023-05-20T01:22:18.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.061+0000 7f5a511f6700 1 -- 172.21.15.16:0/544605894 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559726ce7980 con 0x559726c95000 2023-05-20T01:22:18.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.061+0000 7f5a511f6700 1 -- 172.21.15.16:0/544605894 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x559726cc91e0 con 0x559726c95000 2023-05-20T01:22:18.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.061+0000 7f5a55a4d7c0 1 -- 172.21.15.16:0/544605894 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559726cb96c0 con 0x559726c95000 2023-05-20T01:22:18.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.061+0000 7f5a55a4d7c0 1 -- 172.21.15.16:0/544605894 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x559726cb9520 con 0x559726c95000 2023-05-20T01:22:18.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.061+0000 7f5a55a4d7c0 4 waiting for MDS map... 2023-05-20T01:22:18.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.061+0000 7f5a511f6700 1 -- 172.21.15.16:0/544605894 <== mon.1 v2:172.21.15.132:3300/0 4 ==== osd_map(71..71 src has 1..71) v4 ==== 7103+0+0 (secure 0 0 0) 0x559726cd6c40 con 0x559726c95000 2023-05-20T01:22:18.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.061+0000 7f5a511f6700 1 -- 172.21.15.16:0/544605894 <== mon.1 v2:172.21.15.132:3300/0 5 ==== fsmap(e 42) v1 ==== 3126+0+0 (secure 0 0 0) 0x559727a8c300 con 0x559726c95000 2023-05-20T01:22:18.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.061+0000 7f5a55a4d7c0 4 Got MDS map 42 2023-05-20T01:22:18.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.062+0000 7f5a55a4d7c0 4 datascan.main: Using metadata pool output 2023-05-20T01:22:18.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.062+0000 7f5a55a4d7c0 4 datascan.main: connecting to RADOS... 2023-05-20T01:22:18.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.061+0000 7f70438bd7c0 1 -- start start 2023-05-20T01:22:18.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.061+0000 7f70438bd7c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f909993c00 0x55f909a2d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.061+0000 7f70438bd7c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55f909cc8000 0x55f909a2db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.062+0000 7f70438bd7c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f909cc8400 0x55f909a2e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.062+0000 7f70438bd7c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9099cea00 con 0x55f909993c00 2023-05-20T01:22:18.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.062+0000 7f70438bd7c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9099cfe00 con 0x55f909cc8400 2023-05-20T01:22:18.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.062+0000 7f70438bd7c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f9099cfcc0 con 0x55f909cc8000 2023-05-20T01:22:18.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.062+0000 7f704106a700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f909cc8400 0x55f909a2e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.062+0000 7f7d9df997c0 1 -- 172.21.15.16:0/54916626 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55687b050000 msgr2=0x55687adcdb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.062+0000 7f7d9df997c0 1 --2- 172.21.15.16:0/54916626 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55687b050000 0x55687adcdb80 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x55687b066000 tx=0x55687bbc8d20 comp rx=0 tx=0).stop 2023-05-20T01:22:18.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.062+0000 7f704106a700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f909cc8400 0x55f909a2e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.16:59974/0 (socket says 172.21.15.16:59974) 2023-05-20T01:22:18.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.062+0000 7f704106a700 1 -- 172.21.15.16:0/2471666982 learned_addr learned my addr 172.21.15.16:0/2471666982 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:22:18.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.062+0000 7fe1febbb700 1 -- 172.21.15.16:0/2808449806 <== osd.6 v2:172.21.15.132:6801/727581123 2 ==== osd_op_reply(2 10000000206.00000000 [stat out=16b] v0'0 uv9 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5624b6d67440 con 0x5624b78d2400 2023-05-20T01:22:18.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.062+0000 7fe20140e7c0 10 datascan.operator(): handling object 1099511628294.0 2023-05-20T01:22:18.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.062+0000 7fe20140e7c0 1 -- 172.21.15.16:0/2808449806 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:3 10.0 10:012cdbc7:::10000000206.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x5624b78c6e00 con 0x5624b78d2400 2023-05-20T01:22:18.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.062+0000 7ffaa34017c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560888f3a400 0x560888fd7180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.062+0000 7ffaa34017c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560889282000 0x560888fd6c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.062+0000 7ffaa34017c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x560889282400 0x560888fd6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.062+0000 7ffaa34017c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x560888f76c80 con 0x560889282000 2023-05-20T01:22:18.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.062+0000 7ffaa34017c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x560888f76000 con 0x560888f3a400 2023-05-20T01:22:18.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.062+0000 7ffaa34017c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x560888f77180 con 0x560889282400 2023-05-20T01:22:18.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.062+0000 7ffaa0bae700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560888f3a400 0x560888fd7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.062+0000 7ffaa0bae700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560888f3a400 0x560888fd7180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.16:59978/0 (socket says 172.21.15.16:59978) 2023-05-20T01:22:18.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.062+0000 7ffaa0bae700 1 -- 172.21.15.16:0/1771343338 learned_addr learned my addr 172.21.15.16:0/1771343338 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:22:18.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.062+0000 7ffaa13af700 1 --2- 172.21.15.16:0/1771343338 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560889282000 0x560888fd6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.062+0000 7ffaa0bae700 1 --2- 172.21.15.16:0/1771343338 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x560889282400 0x560888fd6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.063+0000 7ffaa13af700 1 -- 172.21.15.16:0/1771343338 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x560889282400 msgr2=0x560888fd6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.063+0000 7ffaa13af700 1 --2- 172.21.15.16:0/1771343338 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x560889282400 0x560888fd6680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.063+0000 7ffaa13af700 1 -- 172.21.15.16:0/1771343338 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560888f3a400 msgr2=0x560888fd7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.063+0000 7ffaa13af700 1 --2- 172.21.15.16:0/1771343338 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560888f3a400 0x560888fd7180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.063+0000 7ffaa13af700 1 -- 172.21.15.16:0/1771343338 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560888f5f1e0 con 0x560889282000 2023-05-20T01:22:18.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.063+0000 7ffaa0bae700 1 --2- 172.21.15.16:0/1771343338 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560888f3a400 0x560888fd7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-05-20T01:22:18.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.062+0000 7fee7eecb7c0 1 -- 172.21.15.16:0/546353635 wait complete. 2023-05-20T01:22:18.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.062+0000 7fee7eecb7c0 1 Processor -- start 2023-05-20T01:22:18.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.062+0000 7f7d9df997c0 1 -- 172.21.15.16:0/54916626 shutdown_connections 2023-05-20T01:22:18.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.062+0000 7f7d9df997c0 1 --2- 172.21.15.16:0/54916626 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55687b050400 0x55687adce100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.062+0000 7f7d9df997c0 1 --2- 172.21.15.16:0/54916626 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55687ad33c00 0x55687adcd600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.062+0000 7f7d9df997c0 1 --2- 172.21.15.16:0/54916626 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55687b050000 0x55687adcdb80 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.062+0000 7f7d9df997c0 1 -- 172.21.15.16:0/54916626 >> 172.21.15.16:0/54916626 conn(0x55687ad33800 msgr2=0x55687ae03800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:18.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.062+0000 7f5a55a4d7c0 1 Processor -- start 2023-05-20T01:22:18.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.063+0000 7f5a55a4d7c0 1 -- start start 2023-05-20T01:22:18.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.062+0000 7f704206c700 1 --2- 172.21.15.16:0/2471666982 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f909993c00 0x55f909a2d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.062+0000 7f704106a700 1 -- 172.21.15.16:0/2471666982 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55f909cc8000 msgr2=0x55f909a2db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:18.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.062+0000 7f704106a700 1 --2- 172.21.15.16:0/2471666982 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55f909cc8000 0x55f909a2db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.062+0000 7f704106a700 1 -- 172.21.15.16:0/2471666982 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f909993c00 msgr2=0x55f909a2d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:18.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.062+0000 7f704106a700 1 --2- 172.21.15.16:0/2471666982 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f909993c00 0x55f909a2d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.062+0000 7f704106a700 1 -- 172.21.15.16:0/2471666982 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f9099b7d40 con 0x55f909cc8400 2023-05-20T01:22:18.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.062+0000 7f704206c700 1 --2- 172.21.15.16:0/2471666982 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f909993c00 0x55f909a2d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:22:18.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.063+0000 7f704106a700 1 --2- 172.21.15.16:0/2471666982 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f909cc8400 0x55f909a2e100 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x55f909cd20f0 tx=0x55f909c84c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d9c9c34d335ff00 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.062+0000 7f6194186700 1 -- 172.21.15.16:0/3109423973 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564461af2c00 msgr2=0x564461b8c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:18.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.062+0000 7f6194186700 1 --2- 172.21.15.16:0/3109423973 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564461af2c00 0x564461b8c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.062+0000 7f6194186700 1 -- 172.21.15.16:0/3109423973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564461af2800 msgr2=0x564461b8d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:18.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.062+0000 7f6194186700 1 --2- 172.21.15.16:0/3109423973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564461af2800 0x564461b8d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.062+0000 7f6194186700 1 -- 172.21.15.16:0/3109423973 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564461b17380 con 0x564461af3000 2023-05-20T01:22:18.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.062+0000 7f6194186700 1 --2- 172.21.15.16:0/3109423973 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x564461af3000 0x564461b8cb00 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x564461c52d20 tx=0x564461c5cea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8061ce43a8d4793e server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.063+0000 7f6191981700 1 -- 172.21.15.16:0/3109423973 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x564461b27600 con 0x564461af3000 2023-05-20T01:22:18.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.063+0000 7ffaa13af700 1 --2- 172.21.15.16:0/1771343338 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560889282000 0x560888fd6c00 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x56088909a060 tx=0x560889cd3440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.063+0000 7f6191981700 1 -- 172.21.15.16:0/3109423973 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564461b44600 con 0x564461af3000 2023-05-20T01:22:18.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.063+0000 7f6191981700 1 -- 172.21.15.16:0/3109423973 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x564461b271e0 con 0x564461af3000 2023-05-20T01:22:18.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.063+0000 7f703d062700 1 -- 172.21.15.16:0/2471666982 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55f9099c7080 con 0x55f909cc8400 2023-05-20T01:22:18.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.063+0000 7f61961d87c0 1 -- 172.21.15.16:0/3109423973 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564461b17d40 con 0x564461af3000 2023-05-20T01:22:18.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.063+0000 7f61961d87c0 1 -- 172.21.15.16:0/3109423973 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({fsmap=0}) v3 -- 0x564461b17520 con 0x564461af3000 2023-05-20T01:22:18.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.063+0000 7f61961d87c0 4 waiting for MDS map... 2023-05-20T01:22:18.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.063+0000 7f7d9df997c0 1 -- 172.21.15.16:0/54916626 shutdown_connections 2023-05-20T01:22:18.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.063+0000 7f7d9df997c0 1 -- 172.21.15.16:0/54916626 wait complete. 2023-05-20T01:22:18.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.063+0000 7f5a55a4d7c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559726c95c00 0x559726d2f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.063+0000 7f5a55a4d7c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559726fc2000 0x559726d2fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.063+0000 7f5a55a4d7c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559726fc2400 0x559726d30100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.063+0000 7f5a55a4d7c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x559726cd0a00 con 0x559726fc2000 2023-05-20T01:22:18.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.063+0000 7f5a55a4d7c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x559726cd1cc0 con 0x559726fc2400 2023-05-20T01:22:18.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.063+0000 7f5a55a4d7c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x559726cd0b40 con 0x559726c95c00 2023-05-20T01:22:18.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.063+0000 7f6191981700 1 -- 172.21.15.16:0/3109423973 <== mon.2 v2:172.21.15.132:3301/0 4 ==== osd_map(71..71 src has 1..71) v4 ==== 7103+0+0 (secure 0 0 0) 0x564461b34c40 con 0x564461af3000 2023-05-20T01:22:18.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.063+0000 7f6191981700 1 -- 172.21.15.16:0/3109423973 <== mon.2 v2:172.21.15.132:3301/0 5 ==== fsmap(e 42) v1 ==== 3126+0+0 (secure 0 0 0) 0x56446286c000 con 0x564461af3000 2023-05-20T01:22:18.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.063+0000 7f61961d87c0 4 Got MDS map 42 2023-05-20T01:22:18.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.063+0000 7f61961d87c0 4 datascan.main: Using metadata pool output 2023-05-20T01:22:18.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.063+0000 7f61961d87c0 4 datascan.main: connecting to RADOS... 2023-05-20T01:22:18.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.064+0000 7f5a539fb700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559726fc2000 0x559726d2fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.064+0000 7f5a539fb700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559726fc2000 0x559726d2fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34732/0 (socket says 172.21.15.16:34732) 2023-05-20T01:22:18.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.064+0000 7f5a539fb700 1 -- 172.21.15.16:0/964501258 learned_addr learned my addr 172.21.15.16:0/964501258 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:22:18.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.063+0000 7f703d062700 1 -- 172.21.15.16:0/2471666982 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f9099e5e00 con 0x55f909cc8400 2023-05-20T01:22:18.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.063+0000 7f703d062700 1 -- 172.21.15.16:0/2471666982 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55f9099c7600 con 0x55f909cc8400 2023-05-20T01:22:18.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.064+0000 7f70438bd7c0 1 -- 172.21.15.16:0/2471666982 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f909cc8400 msgr2=0x55f909a2e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.064+0000 7f70438bd7c0 1 --2- 172.21.15.16:0/2471666982 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f909cc8400 0x55f909a2e100 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x55f909cd20f0 tx=0x55f909c84c00 comp rx=0 tx=0).stop 2023-05-20T01:22:18.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.064+0000 7f7d9df997c0 1 Processor -- start 2023-05-20T01:22:18.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.064+0000 7f7d9df997c0 1 -- start start 2023-05-20T01:22:18.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.064+0000 7f70438bd7c0 1 -- 172.21.15.16:0/2471666982 shutdown_connections 2023-05-20T01:22:18.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.064+0000 7f70438bd7c0 1 --2- 172.21.15.16:0/2471666982 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f909cc8400 0x55f909a2e100 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.064+0000 7f70438bd7c0 1 --2- 172.21.15.16:0/2471666982 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f909993c00 0x55f909a2d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.064+0000 7f70438bd7c0 1 --2- 172.21.15.16:0/2471666982 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55f909cc8000 0x55f909a2db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.064+0000 7f70438bd7c0 1 -- 172.21.15.16:0/2471666982 >> 172.21.15.16:0/2471666982 conn(0x55f909992400 msgr2=0x55f909a6f800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:18.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.064+0000 7ffa9b3a3700 1 -- 172.21.15.16:0/1771343338 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x560888f6f1e0 con 0x560889282000 2023-05-20T01:22:18.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.064+0000 7ffa9b3a3700 1 -- 172.21.15.16:0/1771343338 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560889220a80 con 0x560889282000 2023-05-20T01:22:18.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.064+0000 7ffa9b3a3700 1 -- 172.21.15.16:0/1771343338 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x560888f6edc0 con 0x560889282000 2023-05-20T01:22:18.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.064+0000 7f70438bd7c0 1 -- 172.21.15.16:0/2471666982 shutdown_connections 2023-05-20T01:22:18.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.064+0000 7f5a539fb700 1 -- 172.21.15.16:0/964501258 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559726c95c00 msgr2=0x559726d2f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:18.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.064+0000 7f5a539fb700 1 --2- 172.21.15.16:0/964501258 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559726c95c00 0x559726d2f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.064+0000 7f5a539fb700 1 -- 172.21.15.16:0/964501258 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559726fc2400 msgr2=0x559726d30100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:18.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.064+0000 7f5a539fb700 1 --2- 172.21.15.16:0/964501258 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559726fc2400 0x559726d30100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.064+0000 7f5a539fb700 1 -- 172.21.15.16:0/964501258 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559726cb9380 con 0x559726fc2000 2023-05-20T01:22:18.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.063+0000 7fee7eecb7c0 1 -- start start 2023-05-20T01:22:18.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.065+0000 7f70438bd7c0 1 -- 172.21.15.16:0/2471666982 wait complete. 2023-05-20T01:22:18.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.065+0000 7f61961d87c0 1 Processor -- start 2023-05-20T01:22:18.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.063+0000 7fee7eecb7c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e8d17d6400 0x55e8d1873180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.064+0000 7fee7eecb7c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8d2656000 0x55e8d1872c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.064+0000 7fee7eecb7c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e8d2656400 0x55e8d1872680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.064+0000 7fee7eecb7c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8d1812b40 con 0x55e8d2656000 2023-05-20T01:22:18.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.065+0000 7f5a539fb700 1 --2- 172.21.15.16:0/964501258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559726fc2000 0x559726d2fb80 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x559726fcc0c0 tx=0x559727b3ac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b943103a5eb36ba2 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.064+0000 7fee7eecb7c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8d1812c80 con 0x55e8d17d6400 2023-05-20T01:22:18.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.064+0000 7fee7eecb7c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e8d1812000 con 0x55e8d2656400 2023-05-20T01:22:18.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.064+0000 7fee7ce79700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8d2656000 0x55e8d1872c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.064+0000 7fee7ce79700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8d2656000 0x55e8d1872c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34734/0 (socket says 172.21.15.16:34734) 2023-05-20T01:22:18.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.064+0000 7fee7ce79700 1 -- 172.21.15.16:0/1398005986 learned_addr learned my addr 172.21.15.16:0/1398005986 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:22:18.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.064+0000 7fee7ce79700 1 -- 172.21.15.16:0/1398005986 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e8d2656400 msgr2=0x55e8d1872680 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-20T01:22:18.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.064+0000 7fee7ce79700 1 --2- 172.21.15.16:0/1398005986 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e8d2656400 0x55e8d1872680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.064+0000 7fee7ce79700 1 -- 172.21.15.16:0/1398005986 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e8d17d6400 msgr2=0x55e8d1873180 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-20T01:22:18.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.064+0000 7fee7ce79700 1 --2- 172.21.15.16:0/1398005986 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e8d17d6400 0x55e8d1873180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.064+0000 7fee7ce79700 1 -- 172.21.15.16:0/1398005986 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e8d17fb1e0 con 0x55e8d2656000 2023-05-20T01:22:18.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.065+0000 7f5a4f1f2700 1 -- 172.21.15.16:0/964501258 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x559726cc9600 con 0x559726fc2000 2023-05-20T01:22:18.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.065+0000 7f7d9df997c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55687ad33800 0x55687adcf180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.065+0000 7f7d9df997c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55687b062000 0x55687adcec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.065+0000 7f7d9df997c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55687b062400 0x55687adce680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.065+0000 7f7d9df997c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55687ad6ec80 con 0x55687b062400 2023-05-20T01:22:18.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.065+0000 7f7d9df997c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55687ad6e000 con 0x55687b062000 2023-05-20T01:22:18.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.065+0000 7f7d9df997c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55687ad6f180 con 0x55687ad33800 2023-05-20T01:22:18.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.065+0000 7f5a4f1f2700 1 -- 172.21.15.16:0/964501258 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559726ce7c80 con 0x559726fc2000 2023-05-20T01:22:18.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.065+0000 7f5a4f1f2700 1 -- 172.21.15.16:0/964501258 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x559726cc91e0 con 0x559726fc2000 2023-05-20T01:22:18.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.065+0000 7fe1febbb700 1 -- 172.21.15.16:0/2808449806 <== osd.6 v2:172.21.15.132:6801/727581123 3 ==== osd_op_reply(3 10000000206.00000000 [call] v71'10 uv10 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5624b6d67440 con 0x5624b78d2400 2023-05-20T01:22:18.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.065+0000 7fe20140e7c0 1 --2- 172.21.15.16:0/2808449806 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x5624b6a39c00 0x5624b6ad5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.065+0000 7fe20140e7c0 1 -- 172.21.15.16:0/2808449806 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:4 10.20 10:04000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x5624b78c6700 con 0x5624b6a39c00 2023-05-20T01:22:18.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.066+0000 7fe1ffbbd700 1 --2- 172.21.15.16:0/2808449806 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x5624b6a39c00 0x5624b6ad5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.065+0000 7ffaa34017c0 1 -- 172.21.15.16:0/1771343338 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x560888f5fd40 con 0x560889282000 2023-05-20T01:22:18.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.065+0000 7ffaa34017c0 1 -- 172.21.15.16:0/1771343338 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560888f5f380 con 0x560889282000 2023-05-20T01:22:18.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.065+0000 7ffaa34017c0 4 datascan.init: resolving metadata pool 9 2023-05-20T01:22:18.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.065+0000 7f7d9c748700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55687ad33800 0x55687adcf180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.065+0000 7f7d9c748700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55687ad33800 0x55687adcf180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.16:34778/0 (socket says 172.21.15.16:34778) 2023-05-20T01:22:18.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.065+0000 7f7d9c748700 1 -- 172.21.15.16:0/1705842756 learned_addr learned my addr 172.21.15.16:0/1705842756 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:22:18.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.065+0000 7f7d9b746700 1 --2- 172.21.15.16:0/1705842756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55687b062400 0x55687adce680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.065+0000 7f7d9bf47700 1 --2- 172.21.15.16:0/1705842756 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55687b062000 0x55687adcec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.066+0000 7f7d9b746700 1 -- 172.21.15.16:0/1705842756 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55687ad33800 msgr2=0x55687adcf180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.066+0000 7f7d9b746700 1 --2- 172.21.15.16:0/1705842756 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55687ad33800 0x55687adcf180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.066+0000 7f7d9b746700 1 -- 172.21.15.16:0/1705842756 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55687b062000 msgr2=0x55687adcec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.066+0000 7f7d9b746700 1 --2- 172.21.15.16:0/1705842756 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55687b062000 0x55687adcec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.066+0000 7f7d9b746700 1 -- 172.21.15.16:0/1705842756 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55687ad576c0 con 0x55687b062400 2023-05-20T01:22:18.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.065+0000 7f70438bd7c0 1 Processor -- start 2023-05-20T01:22:18.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.065+0000 7f70438bd7c0 1 -- start start 2023-05-20T01:22:18.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.065+0000 7f61961d87c0 1 -- start start 2023-05-20T01:22:18.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.066+0000 7f61961d87c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x564461af3c00 0x564461b8d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.066+0000 7f61961d87c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564461e20000 0x564461b8db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.066+0000 7f61961d87c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564461e20400 0x564461b8e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.066+0000 7f61961d87c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x564461b2ea00 con 0x564461e20000 2023-05-20T01:22:18.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.066+0000 7f61961d87c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x564461b2fcc0 con 0x564461e20400 2023-05-20T01:22:18.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.066+0000 7f61961d87c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x564461b2eb40 con 0x564461af3c00 2023-05-20T01:22:18.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.066+0000 7f5a55a4d7c0 1 -- 172.21.15.16:0/964501258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559726fc2000 msgr2=0x559726d2fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.066+0000 7f5a55a4d7c0 1 --2- 172.21.15.16:0/964501258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559726fc2000 0x559726d2fb80 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x559726fcc0c0 tx=0x559727b3ac00 comp rx=0 tx=0).stop 2023-05-20T01:22:18.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.066+0000 7f6194987700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x564461af3c00 0x564461b8d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.066+0000 7f6194987700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x564461af3c00 0x564461b8d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.16:34786/0 (socket says 172.21.15.16:34786) 2023-05-20T01:22:18.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.066+0000 7f6194987700 1 -- 172.21.15.16:0/4015665882 learned_addr learned my addr 172.21.15.16:0/4015665882 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:22:18.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.066+0000 7f6193985700 1 --2- 172.21.15.16:0/4015665882 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564461e20400 0x564461b8e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.066+0000 7fee7ce79700 1 --2- 172.21.15.16:0/1398005986 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8d2656000 0x55e8d1872c00 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x55e8d1b09320 tx=0x55e8d256f2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.066+0000 7f7d9b746700 1 --2- 172.21.15.16:0/1705842756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55687b062400 0x55687adce680 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x55687b0673b0 tx=0x55687bb242a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.066+0000 7fe1ffbbd700 1 --2- 172.21.15.16:0/2808449806 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x5624b6a39c00 0x5624b6ad5180 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.066+0000 7f6193985700 1 -- 172.21.15.16:0/4015665882 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x564461af3c00 msgr2=0x564461b8d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:18.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.066+0000 7f6193985700 1 --2- 172.21.15.16:0/4015665882 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x564461af3c00 0x564461b8d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.066+0000 7f6193985700 1 -- 172.21.15.16:0/4015665882 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564461e20000 msgr2=0x564461b8db80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-05-20T01:22:18.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.066+0000 7f6193985700 1 --2- 172.21.15.16:0/4015665882 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564461e20000 0x564461b8db80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.066+0000 7f6193985700 1 -- 172.21.15.16:0/4015665882 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564461b164e0 con 0x564461e20400 2023-05-20T01:22:18.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.067+0000 7fe1ffbbd700 1 -- 172.21.15.16:0/2808449806 <== osd.0 v2:172.21.15.16:6824/3923634179 1 ==== osd_op_reply(4 [pgnls start_epoch 71 out=81b] v69'10 uv10 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5624b6d67680 con 0x5624b6a39c00 2023-05-20T01:22:18.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.067+0000 7fe20140e7c0 1 -- 172.21.15.16:0/2808449806 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:5 10.20 10:05552577:::1000000020b.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x5624b78c6a80 con 0x5624b6a39c00 2023-05-20T01:22:18.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.067+0000 7fee76e6d700 1 -- 172.21.15.16:0/1398005986 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55e8d180b1e0 con 0x55e8d2656000 2023-05-20T01:22:18.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.067+0000 7fee76e6d700 1 -- 172.21.15.16:0/1398005986 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e8d2674180 con 0x55e8d2656000 2023-05-20T01:22:18.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.067+0000 7fee76e6d700 1 -- 172.21.15.16:0/1398005986 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55e8d180af20 con 0x55e8d2656000 2023-05-20T01:22:18.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.067+0000 7fee7eecb7c0 1 -- 172.21.15.16:0/1398005986 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55e8d17fbd40 con 0x55e8d2656000 2023-05-20T01:22:18.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.067+0000 7fee7eecb7c0 1 -- 172.21.15.16:0/1398005986 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e8d17fad00 con 0x55e8d2656000 2023-05-20T01:22:18.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.067+0000 7fee7eecb7c0 4 datascan.init: resolving metadata pool 9 2023-05-20T01:22:18.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.067+0000 7f6193985700 1 --2- 172.21.15.16:0/4015665882 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564461e20400 0x564461b8e100 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x564461e26090 tx=0x56446288b3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b26d1c036908dd6 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.066+0000 7f70438bd7c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f909992400 0x55f909a2f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.066+0000 7f70438bd7c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f909ce0000 0x55f909a2ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.066+0000 7f70438bd7c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55f909ce0400 0x55f909a2e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.066+0000 7f70438bd7c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9099ceb40 con 0x55f909992400 2023-05-20T01:22:18.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.066+0000 7f70438bd7c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9099cec80 con 0x55f909ce0000 2023-05-20T01:22:18.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.066+0000 7f70438bd7c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f9099ce000 con 0x55f909ce0400 2023-05-20T01:22:18.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.067+0000 7f704106a700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55f909ce0400 0x55f909a2e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.067+0000 7f704186b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f909ce0000 0x55f909a2ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.067+0000 7f704106a700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55f909ce0400 0x55f909a2e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.16:34792/0 (socket says 172.21.15.16:34792) 2023-05-20T01:22:18.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.067+0000 7f704106a700 1 -- 172.21.15.16:0/739319487 learned_addr learned my addr 172.21.15.16:0/739319487 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:22:18.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.067+0000 7f704206c700 1 --2- 172.21.15.16:0/739319487 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f909992400 0x55f909a2f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.067+0000 7f618f97d700 1 -- 172.21.15.16:0/4015665882 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x564461b27080 con 0x564461e20400 2023-05-20T01:22:18.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.067+0000 7f618f97d700 1 -- 172.21.15.16:0/4015665882 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564461b45e00 con 0x564461e20400 2023-05-20T01:22:18.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.067+0000 7f618f97d700 1 -- 172.21.15.16:0/4015665882 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x564461b27600 con 0x564461e20400 2023-05-20T01:22:18.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.068+0000 7fe1ffbbd700 1 -- 172.21.15.16:0/2808449806 <== osd.0 v2:172.21.15.16:6824/3923634179 2 ==== osd_op_reply(5 1000000020b.00000000 [stat out=16b] v0'0 uv10 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5624b6d67680 con 0x5624b6a39c00 2023-05-20T01:22:18.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.068+0000 7fe20140e7c0 10 datascan.operator(): handling object 1099511628299.0 2023-05-20T01:22:18.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.068+0000 7fe20140e7c0 1 -- 172.21.15.16:0/2808449806 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:6 10.20 10:05552577:::1000000020b.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x5624b78c6e00 con 0x5624b6a39c00 2023-05-20T01:22:18.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.067+0000 7ffa9b3a3700 1 -- 172.21.15.16:0/1771343338 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x560888f9c840 con 0x560889282000 2023-05-20T01:22:18.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.068+0000 7ffa9b3a3700 1 --2- 172.21.15.16:0/1771343338 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x560889290800 0x560888fd7700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.068+0000 7f7d95f3b700 1 -- 172.21.15.16:0/1705842756 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55687ad66f20 con 0x55687b062400 2023-05-20T01:22:18.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.068+0000 7f7d95f3b700 1 -- 172.21.15.16:0/1705842756 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55687bbbe180 con 0x55687b062400 2023-05-20T01:22:18.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.068+0000 7f7d9df997c0 1 -- 172.21.15.16:0/1705842756 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55687ad56d00 con 0x55687b062400 2023-05-20T01:22:18.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.068+0000 7f7d95f3b700 1 -- 172.21.15.16:0/1705842756 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55687ad67600 con 0x55687b062400 2023-05-20T01:22:18.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.068+0000 7f7d9df997c0 1 -- 172.21.15.16:0/1705842756 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55687ad57520 con 0x55687b062400 2023-05-20T01:22:18.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.068+0000 7f7d9df997c0 4 datascan.init: resolving metadata pool 9 2023-05-20T01:22:18.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.068+0000 7f5a55a4d7c0 1 -- 172.21.15.16:0/964501258 shutdown_connections 2023-05-20T01:22:18.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.068+0000 7f5a55a4d7c0 1 --2- 172.21.15.16:0/964501258 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559726fc2400 0x559726d30100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.068+0000 7f5a55a4d7c0 1 --2- 172.21.15.16:0/964501258 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559726c95c00 0x559726d2f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.068+0000 7f5a55a4d7c0 1 --2- 172.21.15.16:0/964501258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559726fc2000 0x559726d2fb80 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.068+0000 7f5a55a4d7c0 1 -- 172.21.15.16:0/964501258 >> 172.21.15.16:0/964501258 conn(0x559726c95800 msgr2=0x559726d71800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:18.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.067+0000 7f704106a700 1 -- 172.21.15.16:0/739319487 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f909ce0000 msgr2=0x55f909a2ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.067+0000 7f704106a700 1 --2- 172.21.15.16:0/739319487 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f909ce0000 0x55f909a2ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.067+0000 7f704106a700 1 -- 172.21.15.16:0/739319487 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f909992400 msgr2=0x55f909a2f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.067+0000 7f704106a700 1 --2- 172.21.15.16:0/739319487 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f909992400 0x55f909a2f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.067+0000 7f704106a700 1 -- 172.21.15.16:0/739319487 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f909cde1a0 con 0x55f909ce0400 2023-05-20T01:22:18.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.067+0000 7f704186b700 1 --2- 172.21.15.16:0/739319487 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f909ce0000 0x55f909a2ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-05-20T01:22:18.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.068+0000 7f704106a700 1 --2- 172.21.15.16:0/739319487 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55f909ce0400 0x55f909a2e680 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x55f909cd33b0 tx=0x55f909afcde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.068+0000 7f703b85f700 1 -- 172.21.15.16:0/739319487 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55f9099c6dc0 con 0x55f909ce0400 2023-05-20T01:22:18.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.068+0000 7f703b85f700 1 -- 172.21.15.16:0/739319487 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f909c78d80 con 0x55f909ce0400 2023-05-20T01:22:18.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.068+0000 7f703b85f700 1 -- 172.21.15.16:0/739319487 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55f9099c7080 con 0x55f909ce0400 2023-05-20T01:22:18.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.068+0000 7f70438bd7c0 1 -- 172.21.15.16:0/739319487 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55f909cde000 con 0x55f909ce0400 2023-05-20T01:22:18.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.068+0000 7f70438bd7c0 1 -- 172.21.15.16:0/739319487 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f909cde340 con 0x55f909ce0400 2023-05-20T01:22:18.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.068+0000 7f70438bd7c0 4 datascan.init: resolving metadata pool 9 2023-05-20T01:22:18.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.068+0000 7f61961d87c0 1 -- 172.21.15.16:0/4015665882 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564461e20400 msgr2=0x564461b8e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.068+0000 7f61961d87c0 1 --2- 172.21.15.16:0/4015665882 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564461e20400 0x564461b8e100 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x564461e26090 tx=0x56446288b3e0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.068+0000 7f61961d87c0 1 -- 172.21.15.16:0/4015665882 shutdown_connections 2023-05-20T01:22:18.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.068+0000 7f61961d87c0 1 --2- 172.21.15.16:0/4015665882 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564461e20400 0x564461b8e100 secure :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0x564461e26090 tx=0x56446288b3e0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.068+0000 7f61961d87c0 1 --2- 172.21.15.16:0/4015665882 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x564461af3c00 0x564461b8d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.068+0000 7f61961d87c0 1 --2- 172.21.15.16:0/4015665882 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564461e20000 0x564461b8db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.068+0000 7f61961d87c0 1 -- 172.21.15.16:0/4015665882 >> 172.21.15.16:0/4015665882 conn(0x564461af2400 msgr2=0x564461bcf800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:18.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.068+0000 7ffa9b3a3700 1 -- 172.21.15.16:0/1771343338 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 7103+0+0 (secure 0 0 0) 0x560888f7ca80 con 0x560889282000 2023-05-20T01:22:18.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.068+0000 7ffaa34017c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-05-20T01:22:18.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.068+0000 7ffaa34017c0 4 datascan.main: data pool 'cephfs_data' has ID 10 2023-05-20T01:22:18.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.068+0000 7ffaa34017c0 4 datascan.main: opening data pool 'cephfs_data' 2023-05-20T01:22:18.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.068+0000 7ffaa34017c0 4 datascan.main: data pool 'cephfs_data_extra' has ID 11 2023-05-20T01:22:18.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.068+0000 7ffaa34017c0 4 datascan.main: opening data pool 'cephfs_data_extra' 2023-05-20T01:22:18.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.068+0000 7ffaa34017c0 1 --2- 172.21.15.16:0/1771343338 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x560889282800 0x560888fd6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.069+0000 7f61961d87c0 1 -- 172.21.15.16:0/4015665882 shutdown_connections 2023-05-20T01:22:18.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.068+0000 7ffaa34017c0 1 -- 172.21.15.16:0/1771343338 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:1 10.24 10:24924924::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x56088921e700 con 0x560889282800 2023-05-20T01:22:18.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.068+0000 7ffaa0bae700 1 --2- 172.21.15.16:0/1771343338 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x560889290800 0x560888fd7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.068+0000 7ffaa0bae700 1 --2- 172.21.15.16:0/1771343338 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x560889282800 0x560888fd6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.069+0000 7ffaa0bae700 1 --2- 172.21.15.16:0/1771343338 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x560889282800 0x560888fd6100 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.069+0000 7f5a55a4d7c0 1 -- 172.21.15.16:0/964501258 shutdown_connections 2023-05-20T01:22:18.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.069+0000 7ffaa0bae700 1 -- 172.21.15.16:0/1771343338 <== osd.1 v2:172.21.15.16:6808/3860264220 1 ==== osd_op_reply(1 [pgnls start_epoch 71 out=49b] v69'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560889267440 con 0x560889282800 2023-05-20T01:22:18.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.069+0000 7ffa9c3a5700 1 -- 172.21.15.16:0/1771343338 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:2 10.14 10:28000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x56088921f180 con 0x560889282800 2023-05-20T01:22:18.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.070+0000 7ffaa0bae700 1 -- 172.21.15.16:0/1771343338 <== osd.1 v2:172.21.15.16:6808/3860264220 2 ==== osd_op_reply(2 [pgnls start_epoch 71 out=49b] v69'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560889267440 con 0x560889282800 2023-05-20T01:22:18.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.070+0000 7f5a55a4d7c0 1 -- 172.21.15.16:0/964501258 wait complete. 2023-05-20T01:22:18.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.070+0000 7ffa9cba6700 1 --2- 172.21.15.16:0/1771343338 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x560889290c00 0x5608893a2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.070+0000 7ffa9cba6700 1 -- 172.21.15.16:0/1771343338 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:3 10.34 10:2c000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x56088921e700 con 0x560889290c00 2023-05-20T01:22:18.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.070+0000 7ffaa0bae700 1 --2- 172.21.15.16:0/1771343338 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x560889290800 0x560888fd7700 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x56088927b320 tx=0x5608893881e0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.069+0000 7fee76e6d700 1 -- 172.21.15.16:0/1398005986 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x55e8d1838840 con 0x55e8d2656000 2023-05-20T01:22:18.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.070+0000 7fee76e6d700 1 --2- 172.21.15.16:0/1398005986 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x55e8d2658800 0x55e8d1873700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.070+0000 7fee76e6d700 1 -- 172.21.15.16:0/1398005986 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 7103+0+0 (secure 0 0 0) 0x55e8d1818a80 con 0x55e8d2656000 2023-05-20T01:22:18.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.070+0000 7fee7d67a700 1 --2- 172.21.15.16:0/1398005986 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x55e8d2658800 0x55e8d1873700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.070+0000 7fee7eecb7c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-05-20T01:22:18.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.070+0000 7fee7eecb7c0 4 datascan.main: data pool 'cephfs_data' has ID 10 2023-05-20T01:22:18.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.070+0000 7fee7eecb7c0 4 datascan.main: opening data pool 'cephfs_data' 2023-05-20T01:22:18.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.070+0000 7fee7eecb7c0 4 datascan.main: data pool 'cephfs_data_extra' has ID 11 2023-05-20T01:22:18.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.070+0000 7fee7eecb7c0 4 datascan.main: opening data pool 'cephfs_data_extra' 2023-05-20T01:22:18.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.070+0000 7fee7eecb7c0 1 --2- 172.21.15.16:0/1398005986 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x55e8d2656800 0x55e8d1872100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.070+0000 7fee7eecb7c0 1 -- 172.21.15.16:0/1398005986 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:1 10.12 10:49249249::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x55e8d2672700 con 0x55e8d2656800 2023-05-20T01:22:18.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.070+0000 7f703b85f700 1 -- 172.21.15.16:0/739319487 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x55f9099f4840 con 0x55f909ce0400 2023-05-20T01:22:18.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.069+0000 7f61961d87c0 1 -- 172.21.15.16:0/4015665882 wait complete. 2023-05-20T01:22:18.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.070+0000 7f61961d87c0 1 Processor -- start 2023-05-20T01:22:18.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.070+0000 7f61961d87c0 1 -- start start 2023-05-20T01:22:18.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.070+0000 7fee7c678700 1 --2- 172.21.15.16:0/1398005986 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x55e8d2656800 0x55e8d1872100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.071+0000 7ffaa1bb0700 1 --2- 172.21.15.16:0/1771343338 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x560889290c00 0x5608893a2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.071+0000 7ffaa1bb0700 1 --2- 172.21.15.16:0/1771343338 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x560889290c00 0x5608893a2000 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.071+0000 7fee7c678700 1 --2- 172.21.15.16:0/1398005986 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x55e8d2656800 0x55e8d1872100 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.071+0000 7fee7d67a700 1 --2- 172.21.15.16:0/1398005986 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x55e8d2658800 0x55e8d1873700 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x55e8d1936c90 tx=0x55e8d1c1a000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.071+0000 7fee7c678700 1 -- 172.21.15.16:0/1398005986 <== osd.6 v2:172.21.15.132:6801/727581123 1 ==== osd_op_reply(1 [pgnls start_epoch 71 out=49b] v69'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e8d1af5440 con 0x55e8d2656800 2023-05-20T01:22:18.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.071+0000 7fee77e6f700 1 --2- 172.21.15.16:0/1398005986 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x55e8d2658c00 0x55e8d1c26000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.071+0000 7fee77e6f700 1 -- 172.21.15.16:0/1398005986 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:2 10.32 10:4c000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x55e8d2673180 con 0x55e8d2658c00 2023-05-20T01:22:18.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.070+0000 7f5a55a4d7c0 1 Processor -- start 2023-05-20T01:22:18.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.071+0000 7f5a55a4d7c0 1 -- start start 2023-05-20T01:22:18.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.070+0000 7f703b85f700 1 --2- 172.21.15.16:0/739319487 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x55f909ce6c00 0x55f909a2f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.070+0000 7f703b85f700 1 -- 172.21.15.16:0/739319487 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 7103+0+0 (secure 0 0 0) 0x55f9099d4a80 con 0x55f909ce0400 2023-05-20T01:22:18.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.071+0000 7f70438bd7c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-05-20T01:22:18.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.071+0000 7f70438bd7c0 4 datascan.main: data pool 'cephfs_data' has ID 10 2023-05-20T01:22:18.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.071+0000 7f70438bd7c0 4 datascan.main: opening data pool 'cephfs_data' 2023-05-20T01:22:18.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.071+0000 7f70438bd7c0 4 datascan.main: data pool 'cephfs_data_extra' has ID 11 2023-05-20T01:22:18.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.071+0000 7f70438bd7c0 4 datascan.main: opening data pool 'cephfs_data_extra' 2023-05-20T01:22:18.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.071+0000 7f70438bd7c0 1 --2- 172.21.15.16:0/739319487 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x55f909ce0800 0x55f909a2e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.071+0000 7f70438bd7c0 1 -- 172.21.15.16:0/739319487 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:1 10.2d 10:b6db6db6::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x55f90a82e700 con 0x55f909ce0800 2023-05-20T01:22:18.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.071+0000 7f704206c700 1 --2- 172.21.15.16:0/739319487 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x55f909ce6c00 0x55f909a2f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.071+0000 7f704186b700 1 --2- 172.21.15.16:0/739319487 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x55f909ce0800 0x55f909a2e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.071+0000 7f704206c700 1 --2- 172.21.15.16:0/739319487 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x55f909ce6c00 0x55f909a2f700 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x55f909aee120 tx=0x55f909de4000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.071+0000 7f704186b700 1 --2- 172.21.15.16:0/739319487 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x55f909ce0800 0x55f909a2e100 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.072+0000 7fee7d67a700 1 --2- 172.21.15.16:0/1398005986 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x55e8d2658c00 0x55e8d1c26000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.072+0000 7f5a55a4d7c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559726c95800 0x559726d31180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.072+0000 7f5a55a4d7c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559726fc2000 0x559726d30c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.072+0000 7f5a55a4d7c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559726fc2400 0x559726d30680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.072+0000 7ffaa1bb0700 1 -- 172.21.15.16:0/1771343338 <== osd.5 v2:172.21.15.132:6816/1844689995 1 ==== osd_op_reply(3 [pgnls start_epoch 71 out=49b] v69'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560889267680 con 0x560889290c00 2023-05-20T01:22:18.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.072+0000 7f5a55a4d7c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x559726cd0c80 con 0x559726fc2000 2023-05-20T01:22:18.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.072+0000 7f5a55a4d7c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x559726cd0000 con 0x559726fc2400 2023-05-20T01:22:18.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.072+0000 7f5a55a4d7c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x559726cd1180 con 0x559726c95800 2023-05-20T01:22:18.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.072+0000 7ffa9c3a5700 1 --2- 172.21.15.16:0/1771343338 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x560889291000 0x5608893a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.072+0000 7ffa9c3a5700 1 -- 172.21.15.16:0/1771343338 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:4 10.c 10:30000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x56088921f180 con 0x560889291000 2023-05-20T01:22:18.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.072+0000 7f704186b700 1 -- 172.21.15.16:0/739319487 <== osd.6 v2:172.21.15.132:6801/727581123 1 ==== osd_op_reply(1 [pgnls start_epoch 71 out=49b] v69'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f909cc3440 con 0x55f909ce0800 2023-05-20T01:22:18.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.072+0000 7f703d062700 1 --2- 172.21.15.16:0/739319487 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x55f909ce7000 0x55f909df2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.072+0000 7f703d062700 1 -- 172.21.15.16:0/739319487 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:2 10.1d 10:b8000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x55f90a82f180 con 0x55f909ce7000 2023-05-20T01:22:18.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.072+0000 7f5a531fa700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559726c95800 0x559726d31180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.072+0000 7ffaa13af700 1 --2- 172.21.15.16:0/1771343338 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x560889291000 0x5608893a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.072+0000 7fee7d67a700 1 --2- 172.21.15.16:0/1398005986 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x55e8d2658c00 0x55e8d1c26000 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.072+0000 7f5a531fa700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559726c95800 0x559726d31180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.16:34816/0 (socket says 172.21.15.16:34816) 2023-05-20T01:22:18.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.072+0000 7f5a531fa700 1 -- 172.21.15.16:0/494201187 learned_addr learned my addr 172.21.15.16:0/494201187 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:22:18.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.072+0000 7f5a531fa700 1 --2- 172.21.15.16:0/494201187 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559726fc2400 0x559726d30680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.071+0000 7f61961d87c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564461af3c00 0x564461b8d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.071+0000 7f61961d87c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564461e20000 0x564461b8db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.071+0000 7f61961d87c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x564461e20800 0x564461b8e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.071+0000 7f61961d87c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x564461b2ec80 con 0x564461e20000 2023-05-20T01:22:18.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.071+0000 7f61961d87c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x564461b2e000 con 0x564461af3c00 2023-05-20T01:22:18.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.071+0000 7f61961d87c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x564461b2f180 con 0x564461e20800 2023-05-20T01:22:18.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.071+0000 7f6194186700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564461e20000 0x564461b8db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.071+0000 7f6194186700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564461e20000 0x564461b8db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34772/0 (socket says 172.21.15.16:34772) 2023-05-20T01:22:18.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.071+0000 7f6194186700 1 -- 172.21.15.16:0/3379690421 learned_addr learned my addr 172.21.15.16:0/3379690421 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:22:18.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.072+0000 7f6194186700 1 -- 172.21.15.16:0/3379690421 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x564461e20800 msgr2=0x564461b8e680 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-20T01:22:18.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.072+0000 7f6194186700 1 --2- 172.21.15.16:0/3379690421 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x564461e20800 0x564461b8e680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.072+0000 7f6194186700 1 -- 172.21.15.16:0/3379690421 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564461af3c00 msgr2=0x564461b8d600 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-05-20T01:22:18.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.072+0000 7f6194186700 1 --2- 172.21.15.16:0/3379690421 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564461af3c00 0x564461b8d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.072+0000 7f6194186700 1 -- 172.21.15.16:0/3379690421 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564461b171e0 con 0x564461e20000 2023-05-20T01:22:18.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.072+0000 7f6194186700 1 --2- 172.21.15.16:0/3379690421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564461e20000 0x564461b8db80 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x564461e272f0 tx=0x564461c5cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.072+0000 7f704206c700 1 --2- 172.21.15.16:0/739319487 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x55f909ce7000 0x55f909df2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.072+0000 7f5a539fb700 1 --2- 172.21.15.16:0/494201187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559726fc2000 0x559726d30c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.072+0000 7f618e17a700 1 -- 172.21.15.16:0/3379690421 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x564461b26f20 con 0x564461e20000 2023-05-20T01:22:18.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.072+0000 7f618e17a700 1 -- 172.21.15.16:0/3379690421 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564461dd8780 con 0x564461e20000 2023-05-20T01:22:18.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.072+0000 7f618e17a700 1 -- 172.21.15.16:0/3379690421 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x564461b27080 con 0x564461e20000 2023-05-20T01:22:18.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.072+0000 7f61961d87c0 1 -- 172.21.15.16:0/3379690421 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x564461b16d00 con 0x564461e20000 2023-05-20T01:22:18.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.072+0000 7f61961d87c0 1 -- 172.21.15.16:0/3379690421 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564461b164e0 con 0x564461e20000 2023-05-20T01:22:18.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.072+0000 7f61961d87c0 4 datascan.init: resolving metadata pool 9 2023-05-20T01:22:18.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.073+0000 7ffaa13af700 1 --2- 172.21.15.16:0/1771343338 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x560889291000 0x5608893a2580 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.073+0000 7f5a531fa700 1 -- 172.21.15.16:0/494201187 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559726fc2400 msgr2=0x559726d30680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.073+0000 7f5a531fa700 1 --2- 172.21.15.16:0/494201187 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559726fc2400 0x559726d30680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.073+0000 7f5a531fa700 1 -- 172.21.15.16:0/494201187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559726fc2000 msgr2=0x559726d30c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.073+0000 7f5a531fa700 1 --2- 172.21.15.16:0/494201187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559726fc2000 0x559726d30c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.073+0000 7f5a531fa700 1 -- 172.21.15.16:0/494201187 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559726dfc4e0 con 0x559726c95800 2023-05-20T01:22:18.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.072+0000 7f7d95f3b700 1 -- 172.21.15.16:0/1705842756 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x55687ad95600 con 0x55687b062400 2023-05-20T01:22:18.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.073+0000 7f7d95f3b700 1 --2- 172.21.15.16:0/1705842756 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x55687b064800 0x55687adcf700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.073+0000 7f7d95f3b700 1 -- 172.21.15.16:0/1705842756 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 7103+0+0 (secure 0 0 0) 0x55687ad74a80 con 0x55687b062400 2023-05-20T01:22:18.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.073+0000 7f7d9df997c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-05-20T01:22:18.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.073+0000 7f7d9df997c0 4 datascan.main: data pool 'cephfs_data' has ID 10 2023-05-20T01:22:18.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.073+0000 7f7d9df997c0 4 datascan.main: opening data pool 'cephfs_data' 2023-05-20T01:22:18.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.073+0000 7f7d9df997c0 4 datascan.main: data pool 'cephfs_data_extra' has ID 11 2023-05-20T01:22:18.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.073+0000 7f7d9df997c0 4 datascan.main: opening data pool 'cephfs_data_extra' 2023-05-20T01:22:18.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.073+0000 7f7d9df997c0 1 --2- 172.21.15.16:0/1705842756 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x55687b062800 0x55687adce100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.073+0000 7f7d9df997c0 1 -- 172.21.15.16:0/1705842756 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:1 10.36 10:6db6db6d::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x55687bbbc700 con 0x55687b062800 2023-05-20T01:22:18.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.073+0000 7f5a531fa700 1 --2- 172.21.15.16:0/494201187 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559726c95800 0x559726d31180 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x559726fcd440 tx=0x559727a86060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.073+0000 7f704206c700 1 --2- 172.21.15.16:0/739319487 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x55f909ce7000 0x55f909df2000 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.073+0000 7ffaa13af700 1 -- 172.21.15.16:0/1771343338 <== osd.7 v2:172.21.15.132:6800/466001149 1 ==== osd_op_reply(4 [pgnls start_epoch 71 out=81b] v69'7 uv7 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5608892678c0 con 0x560889291000 2023-05-20T01:22:18.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.073+0000 7f7d9bf47700 1 --2- 172.21.15.16:0/1705842756 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x55687b062800 0x55687adce100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.073+0000 7f5a4d9ef700 1 -- 172.21.15.16:0/494201187 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x559726cc9080 con 0x559726c95800 2023-05-20T01:22:18.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.073+0000 7fe1ffbbd700 1 -- 172.21.15.16:0/2808449806 <== osd.0 v2:172.21.15.16:6824/3923634179 3 ==== osd_op_reply(6 1000000020b.00000000 [call] v71'11 uv11 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5624b6d67680 con 0x5624b6a39c00 2023-05-20T01:22:18.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.073+0000 7fe20140e7c0 1 --2- 172.21.15.16:0/2808449806 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x5624b6a39800 0x5624b6e9c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.073+0000 7fe20140e7c0 1 -- 172.21.15.16:0/2808449806 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:7 10.10 10:08000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x5624b78c7180 con 0x5624b6a39800 2023-05-20T01:22:18.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.074+0000 7fe1ff3bc700 1 --2- 172.21.15.16:0/2808449806 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x5624b6a39800 0x5624b6e9c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.073+0000 7ffaa34017c0 1 -- 172.21.15.16:0/1771343338 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:5 10.c 10:30604137:::100000001fc.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x56088921ea80 con 0x560889291000 2023-05-20T01:22:18.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.073+0000 7f7d9c748700 1 --2- 172.21.15.16:0/1705842756 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x55687b064800 0x55687adcf700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.073+0000 7f5a55a4d7c0 1 -- 172.21.15.16:0/494201187 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x559726dfc340 con 0x559726c95800 2023-05-20T01:22:18.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.073+0000 7f5a4d9ef700 1 -- 172.21.15.16:0/494201187 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559726ce6600 con 0x559726c95800 2023-05-20T01:22:18.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.073+0000 7f5a4d9ef700 1 -- 172.21.15.16:0/494201187 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x559726cc9600 con 0x559726c95800 2023-05-20T01:22:18.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.073+0000 7f5a55a4d7c0 1 -- 172.21.15.16:0/494201187 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559726dfc680 con 0x559726c95800 2023-05-20T01:22:18.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.073+0000 7f5a55a4d7c0 4 datascan.init: resolving metadata pool 9 2023-05-20T01:22:18.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.074+0000 7fee7d67a700 1 -- 172.21.15.16:0/1398005986 <== osd.7 v2:172.21.15.132:6800/466001149 1 ==== osd_op_reply(2 [pgnls start_epoch 71 out=49b] v69'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e8d1af5680 con 0x55e8d2658c00 2023-05-20T01:22:18.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.074+0000 7f704206c700 1 -- 172.21.15.16:0/739319487 <== osd.4 v2:172.21.15.132:6817/506317223 1 ==== osd_op_reply(2 [pgnls start_epoch 71 out=49b] v69'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f909cc3680 con 0x55f909ce7000 2023-05-20T01:22:18.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.074+0000 7f703c861700 1 --2- 172.21.15.16:0/739319487 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x55f909ce7400 0x55f909df2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.074+0000 7fee78670700 1 -- 172.21.15.16:0/1398005986 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:3 10.a 10:50000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x55e8d2673500 con 0x55e8d2656800 2023-05-20T01:22:18.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.074+0000 7f7d9c748700 1 --2- 172.21.15.16:0/1705842756 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x55687b064800 0x55687adcf700 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x55687b067290 tx=0x55687bb24000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.074+0000 7f7d9bf47700 1 --2- 172.21.15.16:0/1705842756 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x55687b062800 0x55687adce100 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.074+0000 7f703c861700 1 -- 172.21.15.16:0/739319487 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:3 10.3d 10:bc000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x55f90a82f500 con 0x55f909ce7400 2023-05-20T01:22:18.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.074+0000 7ffaa13af700 1 -- 172.21.15.16:0/1771343338 <== osd.7 v2:172.21.15.132:6800/466001149 2 ==== osd_op_reply(5 100000001fc.00000000 [stat out=16b] v0'0 uv7 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5608892678c0 con 0x560889291000 2023-05-20T01:22:18.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.074+0000 7f704186b700 1 --2- 172.21.15.16:0/739319487 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x55f909ce7400 0x55f909df2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.074+0000 7fe1ff3bc700 1 --2- 172.21.15.16:0/2808449806 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x5624b6a39800 0x5624b6e9c000 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.074+0000 7ffaa34017c0 10 datascan.operator(): handling object 1099511628284.0 2023-05-20T01:22:18.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.074+0000 7ffaa34017c0 1 -- 172.21.15.16:0/1771343338 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:6 10.c 10:30604137:::100000001fc.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x56088921ee00 con 0x560889291000 2023-05-20T01:22:18.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.074+0000 7fee7c678700 1 -- 172.21.15.16:0/1398005986 <== osd.6 v2:172.21.15.132:6801/727581123 2 ==== osd_op_reply(3 [pgnls start_epoch 71 out=49b] v69'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e8d1af5440 con 0x55e8d2656800 2023-05-20T01:22:18.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.074+0000 7fee77e6f700 1 --2- 172.21.15.16:0/1398005986 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x55e8d2659000 0x55e8d1c26580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.074+0000 7fee77e6f700 1 -- 172.21.15.16:0/1398005986 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:4 10.2a 10:54000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x55e8d2672700 con 0x55e8d2659000 2023-05-20T01:22:18.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.074+0000 7f618e17a700 1 -- 172.21.15.16:0/3379690421 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x564461b55600 con 0x564461e20000 2023-05-20T01:22:18.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.074+0000 7f7d9bf47700 1 -- 172.21.15.16:0/1705842756 <== osd.7 v2:172.21.15.132:6800/466001149 1 ==== osd_op_reply(1 [pgnls start_epoch 71 out=101b] v69'17 uv17 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55687b04f440 con 0x55687b062800 2023-05-20T01:22:18.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.074+0000 7f7d9df997c0 1 -- 172.21.15.16:0/1705842756 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:2 10.36 10:6ee5e2c4:::100000001f9.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x55687bbbca80 con 0x55687b062800 2023-05-20T01:22:18.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.075+0000 7f704186b700 1 --2- 172.21.15.16:0/739319487 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x55f909ce7400 0x55f909df2580 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.075+0000 7fee7ce79700 1 --2- 172.21.15.16:0/1398005986 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x55e8d2659000 0x55e8d1c26580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.075+0000 7fe1ff3bc700 1 -- 172.21.15.16:0/2808449806 <== osd.3 v2:172.21.15.16:6800/3669781890 1 ==== osd_op_reply(7 [pgnls start_epoch 71 out=81b] v69'7 uv7 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5624b6d678c0 con 0x5624b6a39800 2023-05-20T01:22:18.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.075+0000 7fe20140e7c0 1 -- 172.21.15.16:0/2808449806 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:8 10.10 10:08b99fea:::100000001fd.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x5624b78c6700 con 0x5624b6a39800 2023-05-20T01:22:18.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.075+0000 7f7d9bf47700 1 -- 172.21.15.16:0/1705842756 <== osd.7 v2:172.21.15.132:6800/466001149 2 ==== osd_op_reply(2 100000001f9.00000000 [stat out=16b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55687b04f440 con 0x55687b062800 2023-05-20T01:22:18.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.075+0000 7f7d9df997c0 10 datascan.operator(): handling object 1099511628281.0 2023-05-20T01:22:18.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.075+0000 7f7d9df997c0 1 -- 172.21.15.16:0/1705842756 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:3 10.36 10:6ee5e2c4:::100000001f9.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x55687bbbce00 con 0x55687b062800 2023-05-20T01:22:18.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.075+0000 7f618e17a700 1 --2- 172.21.15.16:0/3379690421 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x564461f2e000 0x564461b8f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.075+0000 7f618e17a700 1 -- 172.21.15.16:0/3379690421 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 7103+0+0 (secure 0 0 0) 0x564461b34a80 con 0x564461e20000 2023-05-20T01:22:18.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.075+0000 7f6194987700 1 --2- 172.21.15.16:0/3379690421 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x564461f2e000 0x564461b8f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.075+0000 7f61961d87c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-05-20T01:22:18.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.075+0000 7f61961d87c0 4 datascan.main: data pool 'cephfs_data' has ID 10 2023-05-20T01:22:18.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.075+0000 7f61961d87c0 4 datascan.main: opening data pool 'cephfs_data' 2023-05-20T01:22:18.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.075+0000 7f61961d87c0 4 datascan.main: data pool 'cephfs_data_extra' has ID 11 2023-05-20T01:22:18.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.075+0000 7f61961d87c0 4 datascan.main: opening data pool 'cephfs_data_extra' 2023-05-20T01:22:18.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.075+0000 7f61961d87c0 1 --2- 172.21.15.16:0/3379690421 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x564461e20c00 0x564461b8ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.075+0000 7fe1ff3bc700 1 -- 172.21.15.16:0/2808449806 <== osd.3 v2:172.21.15.16:6800/3669781890 2 ==== osd_op_reply(8 100000001fd.00000000 [stat out=16b] v0'0 uv7 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5624b6d678c0 con 0x5624b6a39800 2023-05-20T01:22:18.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.075+0000 7fe20140e7c0 10 datascan.operator(): handling object 1099511628285.0 2023-05-20T01:22:18.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.075+0000 7f61961d87c0 1 -- 172.21.15.16:0/3379690421 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:1 10.1b 10:db6db6db::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x564462998700 con 0x564461e20c00 2023-05-20T01:22:18.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.075+0000 7fee7ce79700 1 --2- 172.21.15.16:0/1398005986 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x55e8d2659000 0x55e8d1c26580 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.075+0000 7fe20140e7c0 1 -- 172.21.15.16:0/2808449806 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:9 10.10 10:08b99fea:::100000001fd.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x5624b78c6a80 con 0x5624b6a39800 2023-05-20T01:22:18.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.075+0000 7f6193985700 1 --2- 172.21.15.16:0/3379690421 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x564461e20c00 0x564461b8ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.075+0000 7fee7ce79700 1 -- 172.21.15.16:0/1398005986 <== osd.3 v2:172.21.15.16:6800/3669781890 1 ==== osd_op_reply(4 [pgnls start_epoch 71 out=101b] v69'11 uv11 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55e8d1af58c0 con 0x55e8d2659000 2023-05-20T01:22:18.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.075+0000 7fee7eecb7c0 1 -- 172.21.15.16:0/1398005986 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:5 10.2a 10:55a6cd83:::1000000020a.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x55e8d2672a80 con 0x55e8d2659000 2023-05-20T01:22:18.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.075+0000 7f6194987700 1 --2- 172.21.15.16:0/3379690421 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x564461f2e000 0x564461b8f180 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x564461e26060 tx=0x564461f42000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.076+0000 7f6193985700 1 --2- 172.21.15.16:0/3379690421 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x564461e20c00 0x564461b8ec00 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.076+0000 7ffaa13af700 1 -- 172.21.15.16:0/1771343338 <== osd.7 v2:172.21.15.132:6800/466001149 3 ==== osd_op_reply(6 100000001fc.00000000 [call] v71'8 uv8 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5608892678c0 con 0x560889291000 2023-05-20T01:22:18.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.075+0000 7f704186b700 1 -- 172.21.15.16:0/739319487 <== osd.7 v2:172.21.15.132:6800/466001149 1 ==== osd_op_reply(3 [pgnls start_epoch 71 out=81b] v69'8 uv8 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55f909cc3440 con 0x55f909ce7400 2023-05-20T01:22:18.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.075+0000 7f70438bd7c0 1 -- 172.21.15.16:0/739319487 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:4 10.3d 10:bfcd3380:::10000000205.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x55f90a82ea80 con 0x55f909ce7400 2023-05-20T01:22:18.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.076+0000 7f704186b700 1 -- 172.21.15.16:0/739319487 <== osd.7 v2:172.21.15.132:6800/466001149 2 ==== osd_op_reply(4 10000000205.00000000 [stat out=16b] v0'0 uv8 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55f909cc3440 con 0x55f909ce7400 2023-05-20T01:22:18.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.076+0000 7f70438bd7c0 10 datascan.operator(): handling object 1099511628293.0 2023-05-20T01:22:18.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.076+0000 7f70438bd7c0 1 -- 172.21.15.16:0/739319487 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:5 10.3d 10:bfcd3380:::10000000205.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x55f90a82ee00 con 0x55f909ce7400 2023-05-20T01:22:18.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.076+0000 7fee7ce79700 1 -- 172.21.15.16:0/1398005986 <== osd.3 v2:172.21.15.16:6800/3669781890 2 ==== osd_op_reply(5 1000000020a.00000000 [stat out=16b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55e8d1af58c0 con 0x55e8d2659000 2023-05-20T01:22:18.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.076+0000 7fee7eecb7c0 10 datascan.operator(): handling object 1099511628298.0 2023-05-20T01:22:18.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.076+0000 7fee7eecb7c0 1 -- 172.21.15.16:0/1398005986 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:6 10.2a 10:55a6cd83:::1000000020a.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x55e8d2672e00 con 0x55e8d2659000 2023-05-20T01:22:18.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.076+0000 7f6193985700 1 -- 172.21.15.16:0/3379690421 <== osd.3 v2:172.21.15.16:6800/3669781890 1 ==== osd_op_reply(1 [pgnls start_epoch 71 out=49b] v69'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564461dd7680 con 0x564461e20c00 2023-05-20T01:22:18.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.076+0000 7f618f97d700 1 --2- 172.21.15.16:0/3379690421 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x564461f2e400 0x564461b8f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.076+0000 7f618f97d700 1 -- 172.21.15.16:0/3379690421 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:2 10.3b 10:dc000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x564462999180 con 0x564461f2e400 2023-05-20T01:22:18.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.076+0000 7f6194987700 1 --2- 172.21.15.16:0/3379690421 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x564461f2e400 0x564461b8f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.076+0000 7ffaa34017c0 1 --2- 172.21.15.16:0/1771343338 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x560889282c00 0x560888fd5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.076+0000 7ffaa34017c0 1 -- 172.21.15.16:0/1771343338 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:7 10.2c 10:34000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x56088921f180 con 0x560889282c00 2023-05-20T01:22:18.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.077+0000 7f7d9bf47700 1 -- 172.21.15.16:0/1705842756 <== osd.7 v2:172.21.15.132:6800/466001149 3 ==== osd_op_reply(3 100000001f9.00000000 [call] v71'18 uv18 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55687b04f440 con 0x55687b062800 2023-05-20T01:22:18.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.077+0000 7f7d9df997c0 1 -- 172.21.15.16:0/1705842756 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:4 10.36 10:6fe38dc2::::head [pgnls start_epoch 71 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x55687bbbc700 con 0x55687b062800 2023-05-20T01:22:18.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.076+0000 7f5a4d9ef700 1 -- 172.21.15.16:0/494201187 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x559726cf7600 con 0x559726c95800 2023-05-20T01:22:18.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.077+0000 7ffaa0bae700 1 --2- 172.21.15.16:0/1771343338 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x560889282c00 0x560888fd5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.077+0000 7f5a4d9ef700 1 --2- 172.21.15.16:0/494201187 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x559726ff0400 0x559726d31700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.077+0000 7f5a4d9ef700 1 -- 172.21.15.16:0/494201187 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 7103+0+0 (secure 0 0 0) 0x559726cd6a80 con 0x559726c95800 2023-05-20T01:22:18.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.077+0000 7ffaa0bae700 1 --2- 172.21.15.16:0/1771343338 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x560889282c00 0x560888fd5b80 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.077+0000 7f7d9bf47700 1 -- 172.21.15.16:0/1705842756 <== osd.7 v2:172.21.15.132:6800/466001149 4 ==== osd_op_reply(4 [pgnls start_epoch 71 out=81b] v71'18 uv18 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55687b04f440 con 0x55687b062800 2023-05-20T01:22:18.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.077+0000 7f7d9df997c0 1 -- 172.21.15.16:0/1705842756 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:5 10.36 10:6fe38dc2:::100000001f7.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x55687bbbca80 con 0x55687b062800 2023-05-20T01:22:18.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.077+0000 7f6194987700 1 --2- 172.21.15.16:0/3379690421 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x564461f2e400 0x564461b8f700 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.077+0000 7f5a539fb700 1 --2- 172.21.15.16:0/494201187 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x559726ff0400 0x559726d31700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.077+0000 7f5a55a4d7c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-05-20T01:22:18.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.077+0000 7f5a55a4d7c0 4 datascan.main: data pool 'cephfs_data' has ID 10 2023-05-20T01:22:18.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.077+0000 7f5a55a4d7c0 4 datascan.main: opening data pool 'cephfs_data' 2023-05-20T01:22:18.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.077+0000 7f5a55a4d7c0 4 datascan.main: data pool 'cephfs_data_extra' has ID 11 2023-05-20T01:22:18.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.077+0000 7f5a55a4d7c0 4 datascan.main: opening data pool 'cephfs_data_extra' 2023-05-20T01:22:18.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.077+0000 7f5a55a4d7c0 1 --2- 172.21.15.16:0/494201187 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x559726fc2800 0x559726d30100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.077+0000 7f5a55a4d7c0 1 -- 172.21.15.16:0/494201187 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:1 10.9 10:92492492::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x559727b1c700 con 0x559726fc2800 2023-05-20T01:22:18.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.077+0000 7fe1ff3bc700 1 -- 172.21.15.16:0/2808449806 <== osd.3 v2:172.21.15.16:6800/3669781890 3 ==== osd_op_reply(9 100000001fd.00000000 [call] v71'8 uv8 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5624b6d678c0 con 0x5624b6a39800 2023-05-20T01:22:18.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.077+0000 7fe20140e7c0 1 --2- 172.21.15.16:0/2808449806 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x5624b6e76400 0x5624b6e9c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.077+0000 7fe20140e7c0 1 -- 172.21.15.16:0/2808449806 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:10 10.30 10:0c000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x5624b78c7500 con 0x5624b6e76400 2023-05-20T01:22:18.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.077+0000 7ffaa0bae700 1 -- 172.21.15.16:0/1771343338 <== osd.3 v2:172.21.15.16:6800/3669781890 1 ==== osd_op_reply(7 [pgnls start_epoch 71 out=81b] v69'9 uv9 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x560889267440 con 0x560889282c00 2023-05-20T01:22:18.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.077+0000 7f7d9bf47700 1 -- 172.21.15.16:0/1705842756 <== osd.7 v2:172.21.15.132:6800/466001149 5 ==== osd_op_reply(5 100000001f7.00000000 [stat out=16b] v0'0 uv16 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55687b04f440 con 0x55687b062800 2023-05-20T01:22:18.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.077+0000 7f5a531fa700 1 --2- 172.21.15.16:0/494201187 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x559726fc2800 0x559726d30100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.077+0000 7ffaa34017c0 1 -- 172.21.15.16:0/1771343338 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:8 10.2c 10:37389ade:::10000000209.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x56088921ea80 con 0x560889282c00 2023-05-20T01:22:18.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.077+0000 7fe1febbb700 1 --2- 172.21.15.16:0/2808449806 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x5624b6e76400 0x5624b6e9c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.077+0000 7f7d9df997c0 10 datascan.operator(): handling object 1099511628279.0 2023-05-20T01:22:18.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.077+0000 7f7d9df997c0 1 -- 172.21.15.16:0/1705842756 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:6 10.36 10:6fe38dc2:::100000001f7.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x55687bbbd180 con 0x55687b062800 2023-05-20T01:22:18.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.078+0000 7f5a539fb700 1 --2- 172.21.15.16:0/494201187 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x559726ff0400 0x559726d31700 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x559726fcc0c0 tx=0x559727a86300 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.078+0000 7f6194987700 1 -- 172.21.15.16:0/3379690421 <== osd.6 v2:172.21.15.132:6801/727581123 1 ==== osd_op_reply(2 [pgnls start_epoch 71 out=81b] v69'5 uv5 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x564461dd78c0 con 0x564461f2e400 2023-05-20T01:22:18.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.078+0000 7f61961d87c0 1 -- 172.21.15.16:0/3379690421 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:3 10.3b 10:deccbd37:::100000001fe.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x564462998a80 con 0x564461f2e400 2023-05-20T01:22:18.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.078+0000 7ffaa0bae700 1 -- 172.21.15.16:0/1771343338 <== osd.3 v2:172.21.15.16:6800/3669781890 2 ==== osd_op_reply(8 10000000209.00000000 [stat out=16b] v0'0 uv9 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x560889267440 con 0x560889282c00 2023-05-20T01:22:18.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.078+0000 7ffaa34017c0 10 datascan.operator(): handling object 1099511628297.0 2023-05-20T01:22:18.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.078+0000 7ffaa34017c0 1 -- 172.21.15.16:0/1771343338 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:9 10.2c 10:37389ade:::10000000209.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x56088921f880 con 0x560889282c00 2023-05-20T01:22:18.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.078+0000 7fe1febbb700 1 --2- 172.21.15.16:0/2808449806 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x5624b6e76400 0x5624b6e9c580 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.078+0000 7f704186b700 1 -- 172.21.15.16:0/739319487 <== osd.7 v2:172.21.15.132:6800/466001149 3 ==== osd_op_reply(5 10000000205.00000000 [call] v71'9 uv9 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55f909cc3440 con 0x55f909ce7400 2023-05-20T01:22:18.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.078+0000 7f5a531fa700 1 --2- 172.21.15.16:0/494201187 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x559726fc2800 0x559726d30100 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.078+0000 7f6194987700 1 -- 172.21.15.16:0/3379690421 <== osd.6 v2:172.21.15.132:6801/727581123 2 ==== osd_op_reply(3 100000001fe.00000000 [stat out=16b] v0'0 uv5 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x564461dd78c0 con 0x564461f2e400 2023-05-20T01:22:18.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.078+0000 7f61961d87c0 10 datascan.operator(): handling object 1099511628286.0 2023-05-20T01:22:18.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.078+0000 7f61961d87c0 1 -- 172.21.15.16:0/3379690421 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:4 10.3b 10:deccbd37:::100000001fe.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x564462998e00 con 0x564461f2e400 2023-05-20T01:22:18.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.078+0000 7f70438bd7c0 1 --2- 172.21.15.16:0/739319487 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x55f909ce0c00 0x55f909a2db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.078+0000 7f70438bd7c0 1 -- 172.21.15.16:0/739319487 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:6 10.3 10:c0000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x55f90a82e700 con 0x55f909ce0c00 2023-05-20T01:22:18.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.078+0000 7f704106a700 1 --2- 172.21.15.16:0/739319487 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x55f909ce0c00 0x55f909a2db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.078+0000 7fee7ce79700 1 -- 172.21.15.16:0/1398005986 <== osd.3 v2:172.21.15.16:6800/3669781890 3 ==== osd_op_reply(6 1000000020a.00000000 [call] v71'12 uv12 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55e8d1af58c0 con 0x55e8d2659000 2023-05-20T01:22:18.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.078+0000 7fe1febbb700 1 -- 172.21.15.16:0/2808449806 <== osd.2 v2:172.21.15.16:6816/293187030 1 ==== osd_op_reply(10 [pgnls start_epoch 71 out=49b] v69'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5624b6d67440 con 0x5624b6e76400 2023-05-20T01:22:18.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.078+0000 7fee7eecb7c0 1 -- 172.21.15.16:0/1398005986 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:7 10.2a 10:57fe7535::::head [pgnls start_epoch 71 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x55e8d2672700 con 0x55e8d2659000 2023-05-20T01:22:18.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.078+0000 7f5a531fa700 1 -- 172.21.15.16:0/494201187 <== osd.5 v2:172.21.15.132:6816/1844689995 1 ==== osd_op_reply(1 [pgnls start_epoch 71 out=49b] v69'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559726fc1440 con 0x559726fc2800 2023-05-20T01:22:18.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.078+0000 7f5a4f1f2700 1 --2- 172.21.15.16:0/494201187 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x559726ff0800 0x5597270f6000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.078+0000 7f5a4f1f2700 1 -- 172.21.15.16:0/494201187 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:2 10.29 10:94000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x559727b1d180 con 0x559726ff0800 2023-05-20T01:22:18.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.078+0000 7fe1fabb3700 1 -- 172.21.15.16:0/2808449806 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:11 10.8 10:10000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x5624b78c7180 con 0x5624b6a39c00 2023-05-20T01:22:18.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.079+0000 7f704106a700 1 --2- 172.21.15.16:0/739319487 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x55f909ce0c00 0x55f909a2db80 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.079+0000 7fee7ce79700 1 -- 172.21.15.16:0/1398005986 <== osd.3 v2:172.21.15.16:6800/3669781890 4 ==== osd_op_reply(7 [pgnls start_epoch 71 out=81b] v71'12 uv12 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55e8d1af58c0 con 0x55e8d2659000 2023-05-20T01:22:18.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.079+0000 7f5a541fc700 1 --2- 172.21.15.16:0/494201187 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x559726ff0800 0x5597270f6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.079+0000 7f704106a700 1 -- 172.21.15.16:0/739319487 <== osd.2 v2:172.21.15.16:6816/293187030 1 ==== osd_op_reply(6 [pgnls start_epoch 71 out=49b] v69'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f909cc38c0 con 0x55f909ce0c00 2023-05-20T01:22:18.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.079+0000 7fee7eecb7c0 1 -- 172.21.15.16:0/1398005986 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:8 10.2a 10:57fe7535:::100000001f6.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x55e8d2672a80 con 0x55e8d2659000 2023-05-20T01:22:18.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.079+0000 7f703c861700 1 -- 172.21.15.16:0/739319487 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:7 10.23 10:c4000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x55f90a82fc00 con 0x55f909ce7400 2023-05-20T01:22:18.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.079+0000 7fe1ffbbd700 1 -- 172.21.15.16:0/2808449806 <== osd.0 v2:172.21.15.16:6824/3923634179 4 ==== osd_op_reply(11 [pgnls start_epoch 71 out=49b] v69'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5624b6d67680 con 0x5624b6a39c00 2023-05-20T01:22:18.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.079+0000 7fe1fa3b2700 1 --2- 172.21.15.16:0/2808449806 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x5624b6eae000 0x5624b6e9db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.079+0000 7fe1fa3b2700 1 -- 172.21.15.16:0/2808449806 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:12 10.28 10:14000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x5624b78c6700 con 0x5624b6eae000 2023-05-20T01:22:18.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.079+0000 7f5a541fc700 1 --2- 172.21.15.16:0/494201187 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x559726ff0800 0x5597270f6000 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.079+0000 7fe1ffbbd700 1 --2- 172.21.15.16:0/2808449806 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x5624b6eae000 0x5624b6e9db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.079+0000 7fee7ce79700 1 -- 172.21.15.16:0/1398005986 <== osd.3 v2:172.21.15.16:6800/3669781890 5 ==== osd_op_reply(8 100000001f6.00000000 [stat out=16b] v0'0 uv10 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55e8d1af58c0 con 0x55e8d2659000 2023-05-20T01:22:18.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.079+0000 7f704186b700 1 -- 172.21.15.16:0/739319487 <== osd.7 v2:172.21.15.132:6800/466001149 4 ==== osd_op_reply(7 [pgnls start_epoch 71 out=49b] v69'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f909cc3440 con 0x55f909ce7400 2023-05-20T01:22:18.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.079+0000 7f703d062700 1 --2- 172.21.15.16:0/739319487 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x55f909ce7800 0x55f909df2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.079+0000 7f703d062700 1 -- 172.21.15.16:0/739319487 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:8 10.13 10:c8000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x55f909e0e000 con 0x55f909ce7800 2023-05-20T01:22:18.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.079+0000 7fee7eecb7c0 10 datascan.operator(): handling object 1099511628278.0 2023-05-20T01:22:18.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.079+0000 7fee7eecb7c0 1 -- 172.21.15.16:0/1398005986 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:9 10.2a 10:57fe7535:::100000001f6.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x55e8d2672e00 con 0x55e8d2659000 2023-05-20T01:22:18.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.079+0000 7f5a541fc700 1 -- 172.21.15.16:0/494201187 <== osd.3 v2:172.21.15.16:6800/3669781890 1 ==== osd_op_reply(2 [pgnls start_epoch 71 out=49b] v69'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559726fc1680 con 0x559726ff0800 2023-05-20T01:22:18.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.079+0000 7f5a4e9f1700 1 --2- 172.21.15.16:0/494201187 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x559726ff0c00 0x5597270f6580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.079+0000 7f704206c700 1 --2- 172.21.15.16:0/739319487 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x55f909ce7800 0x55f909df2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.080+0000 7f5a4e9f1700 1 -- 172.21.15.16:0/494201187 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:3 10.19 10:98000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x559727b1d500 con 0x559726ff0c00 2023-05-20T01:22:18.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.080+0000 7f7d9bf47700 1 -- 172.21.15.16:0/1705842756 <== osd.7 v2:172.21.15.132:6800/466001149 6 ==== osd_op_reply(6 100000001f7.00000000 [call] v71'19 uv19 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55687b04f440 con 0x55687b062800 2023-05-20T01:22:18.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.080+0000 7f7d9df997c0 1 --2- 172.21.15.16:0/1705842756 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x55687b062c00 0x55687adcdb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.080+0000 7f7d9df997c0 1 -- 172.21.15.16:0/1705842756 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:7 10.e 10:70000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x55687bbbd500 con 0x55687b062c00 2023-05-20T01:22:18.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.080+0000 7f5a539fb700 1 --2- 172.21.15.16:0/494201187 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x559726ff0c00 0x5597270f6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.080+0000 7fe1ffbbd700 1 --2- 172.21.15.16:0/2808449806 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x5624b6eae000 0x5624b6e9db80 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.080+0000 7ffaa0bae700 1 -- 172.21.15.16:0/1771343338 <== osd.3 v2:172.21.15.16:6800/3669781890 3 ==== osd_op_reply(9 10000000209.00000000 [call] v71'10 uv10 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x560889267440 con 0x560889282c00 2023-05-20T01:22:18.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.080+0000 7ffaa34017c0 1 --2- 172.21.15.16:0/1771343338 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x560889283000 0x560888fd5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.080+0000 7ffaa34017c0 1 -- 172.21.15.16:0/1771343338 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:10 10.1c 10:38000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x56088921fc00 con 0x560889283000 2023-05-20T01:22:18.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.080+0000 7f7d9c748700 1 --2- 172.21.15.16:0/1705842756 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x55687b062c00 0x55687adcdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.080+0000 7f704206c700 1 --2- 172.21.15.16:0/739319487 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x55f909ce7800 0x55f909df2b00 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.080+0000 7fe1ffbbd700 1 -- 172.21.15.16:0/2808449806 <== osd.7 v2:172.21.15.132:6800/466001149 1 ==== osd_op_reply(12 [pgnls start_epoch 71 out=101b] v69'8 uv8 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x5624b6d67680 con 0x5624b6eae000 2023-05-20T01:22:18.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.080+0000 7f5a539fb700 1 --2- 172.21.15.16:0/494201187 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x559726ff0c00 0x5597270f6580 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.080+0000 7f6194987700 1 -- 172.21.15.16:0/3379690421 <== osd.6 v2:172.21.15.132:6801/727581123 3 ==== osd_op_reply(4 100000001fe.00000000 [call] v71'6 uv6 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564461dd78c0 con 0x564461f2e400 2023-05-20T01:22:18.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.080+0000 7f61961d87c0 1 -- 172.21.15.16:0/3379690421 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:5 10.7 10:e0000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x564462999180 con 0x564461f2e400 2023-05-20T01:22:18.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.080+0000 7fe20140e7c0 1 -- 172.21.15.16:0/2808449806 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:13 10.28 10:15246602:::10000000202.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x5624b78c7880 con 0x5624b6eae000 2023-05-20T01:22:18.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.080+0000 7ffaa1bb0700 1 --2- 172.21.15.16:0/1771343338 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x560889283000 0x560888fd5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.080+0000 7f704206c700 1 -- 172.21.15.16:0/739319487 <== osd.1 v2:172.21.15.16:6808/3860264220 1 ==== osd_op_reply(8 [pgnls start_epoch 71 out=81b] v69'16 uv16 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55f909cc3680 con 0x55f909ce7800 2023-05-20T01:22:18.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.080+0000 7f70438bd7c0 1 -- 172.21.15.16:0/739319487 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:9 10.13 10:ca8fdbba:::100000001fb.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x55f90a82f500 con 0x55f909ce7800 2023-05-20T01:22:18.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.080+0000 7f7d9c748700 1 --2- 172.21.15.16:0/1705842756 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x55687b062c00 0x55687adcdb80 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.081+0000 7f5a539fb700 1 -- 172.21.15.16:0/494201187 <== osd.1 v2:172.21.15.16:6808/3860264220 1 ==== osd_op_reply(3 [pgnls start_epoch 71 out=101b] v69'12 uv12 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x559726fc18c0 con 0x559726ff0c00 2023-05-20T01:22:18.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.081+0000 7f5a55a4d7c0 1 -- 172.21.15.16:0/494201187 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:4 10.19 10:992a6246:::10000000200.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x559727b1ca80 con 0x559726ff0c00 2023-05-20T01:22:18.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.081+0000 7f704206c700 1 -- 172.21.15.16:0/739319487 <== osd.1 v2:172.21.15.16:6808/3860264220 2 ==== osd_op_reply(9 100000001fb.00000000 [stat out=16b] v0'0 uv16 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55f909cc3680 con 0x55f909ce7800 2023-05-20T01:22:18.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.081+0000 7fe1ffbbd700 1 -- 172.21.15.16:0/2808449806 <== osd.7 v2:172.21.15.132:6800/466001149 2 ==== osd_op_reply(13 10000000202.00000000 [stat out=16b] v0'0 uv8 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5624b6d67680 con 0x5624b6eae000 2023-05-20T01:22:18.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.081+0000 7f70438bd7c0 10 datascan.operator(): handling object 1099511628283.0 2023-05-20T01:22:18.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.081+0000 7fe20140e7c0 10 datascan.operator(): handling object 1099511628290.0 2023-05-20T01:22:18.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.081+0000 7fe20140e7c0 1 -- 172.21.15.16:0/2808449806 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:14 10.28 10:15246602:::10000000202.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x5624b78c7c00 con 0x5624b6eae000 2023-05-20T01:22:18.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.081+0000 7f70438bd7c0 1 -- 172.21.15.16:0/739319487 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:10 10.13 10:ca8fdbba:::100000001fb.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x55f90a82ea80 con 0x55f909ce7800 2023-05-20T01:22:18.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.081+0000 7f6194987700 1 -- 172.21.15.16:0/3379690421 <== osd.6 v2:172.21.15.132:6801/727581123 4 ==== osd_op_reply(5 [pgnls start_epoch 71 out=101b] v69'7 uv7 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x564461dd78c0 con 0x564461f2e400 2023-05-20T01:22:18.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.081+0000 7ffaa1bb0700 1 --2- 172.21.15.16:0/1771343338 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x560889283000 0x560888fd5600 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.081+0000 7f7d9c748700 1 -- 172.21.15.16:0/1705842756 <== osd.4 v2:172.21.15.132:6817/506317223 1 ==== osd_op_reply(7 [pgnls start_epoch 71 out=49b] v69'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55687b04f680 con 0x55687b062c00 2023-05-20T01:22:18.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.081+0000 7f7d9773e700 1 --2- 172.21.15.16:0/1705842756 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x55687b064c00 0x55687b188000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.081+0000 7f7d9773e700 1 -- 172.21.15.16:0/1705842756 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:8 10.2e 10:74000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x55687bbbce00 con 0x55687b064c00 2023-05-20T01:22:18.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.081+0000 7f5a539fb700 1 -- 172.21.15.16:0/494201187 <== osd.1 v2:172.21.15.16:6808/3860264220 2 ==== osd_op_reply(4 10000000200.00000000 [stat out=16b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x559726fc18c0 con 0x559726ff0c00 2023-05-20T01:22:18.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.081+0000 7f5a55a4d7c0 10 datascan.operator(): handling object 1099511628288.0 2023-05-20T01:22:18.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.081+0000 7f5a55a4d7c0 1 -- 172.21.15.16:0/494201187 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:5 10.19 10:992a6246:::10000000200.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x559727b1ce00 con 0x559726ff0c00 2023-05-20T01:22:18.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.081+0000 7f61961d87c0 1 -- 172.21.15.16:0/3379690421 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:6 10.7 10:e15a914f:::10000000207.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x564462998a80 con 0x564461f2e400 2023-05-20T01:22:18.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.081+0000 7f7d9bf47700 1 --2- 172.21.15.16:0/1705842756 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x55687b064c00 0x55687b188000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.081+0000 7ffaa1bb0700 1 -- 172.21.15.16:0/1771343338 <== osd.6 v2:172.21.15.132:6801/727581123 1 ==== osd_op_reply(10 [pgnls start_epoch 71 out=49b] v69'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560889267680 con 0x560889283000 2023-05-20T01:22:18.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.081+0000 7fee7ce79700 1 -- 172.21.15.16:0/1398005986 <== osd.3 v2:172.21.15.16:6800/3669781890 6 ==== osd_op_reply(9 100000001f6.00000000 [call] v71'13 uv13 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55e8d1af58c0 con 0x55e8d2659000 2023-05-20T01:22:18.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.081+0000 7fee7eecb7c0 1 -- 172.21.15.16:0/1398005986 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:10 10.1a 10:58000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x55e8d2673880 con 0x55e8d2656800 2023-05-20T01:22:18.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.081+0000 7ffa9cba6700 1 --2- 172.21.15.16:0/1771343338 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x560889291400 0x5608893a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.081+0000 7ffa9cba6700 1 -- 172.21.15.16:0/1771343338 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:11 10.3c 10:3c000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x56088921f180 con 0x560889291400 2023-05-20T01:22:18.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.081+0000 7f6194987700 1 -- 172.21.15.16:0/3379690421 <== osd.6 v2:172.21.15.132:6801/727581123 5 ==== osd_op_reply(6 10000000207.00000000 [stat out=16b] v0'0 uv7 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x564461dd78c0 con 0x564461f2e400 2023-05-20T01:22:18.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.081+0000 7f61961d87c0 10 datascan.operator(): handling object 1099511628295.0 2023-05-20T01:22:18.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.081+0000 7f61961d87c0 1 -- 172.21.15.16:0/3379690421 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:7 10.7 10:e15a914f:::10000000207.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x564462999500 con 0x564461f2e400 2023-05-20T01:22:18.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.081+0000 7ffaa13af700 1 --2- 172.21.15.16:0/1771343338 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x560889291400 0x5608893a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.082+0000 7fee7c678700 1 -- 172.21.15.16:0/1398005986 <== osd.6 v2:172.21.15.132:6801/727581123 3 ==== osd_op_reply(10 [pgnls start_epoch 71 out=49b] v69'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e8d1af5440 con 0x55e8d2656800 2023-05-20T01:22:18.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.082+0000 7fee78670700 1 --2- 172.21.15.16:0/1398005986 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x55e8d2659800 0x55e8d1c27080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.082+0000 7fee78670700 1 -- 172.21.15.16:0/1398005986 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:11 10.3a 10:5c000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x55e8d2673c00 con 0x55e8d2659800 2023-05-20T01:22:18.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.082+0000 7f7d9bf47700 1 --2- 172.21.15.16:0/1705842756 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x55687b064c00 0x55687b188000 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.082+0000 7fee7c678700 1 --2- 172.21.15.16:0/1398005986 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x55e8d2659800 0x55e8d1c27080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.082+0000 7f7d9bf47700 1 -- 172.21.15.16:0/1705842756 <== osd.3 v2:172.21.15.16:6800/3669781890 1 ==== osd_op_reply(8 [pgnls start_epoch 71 out=49b] v69'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55687b04f440 con 0x55687b064c00 2023-05-20T01:22:18.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.082+0000 7ffaa13af700 1 --2- 172.21.15.16:0/1771343338 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x560889291400 0x5608893a2b00 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.082+0000 7f7d96f3d700 1 --2- 172.21.15.16:0/1705842756 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x55687b065000 0x55687b188580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.082+0000 7f7d96f3d700 1 -- 172.21.15.16:0/1705842756 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:9 10.1e 10:78000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x55687bbbc700 con 0x55687b065000 2023-05-20T01:22:18.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.082+0000 7fee7c678700 1 --2- 172.21.15.16:0/1398005986 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x55e8d2659800 0x55e8d1c27080 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.082+0000 7f7d9b746700 1 --2- 172.21.15.16:0/1705842756 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x55687b065000 0x55687b188580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.082+0000 7fe1ffbbd700 1 -- 172.21.15.16:0/2808449806 <== osd.7 v2:172.21.15.132:6800/466001149 3 ==== osd_op_reply(14 10000000202.00000000 [call] v71'9 uv9 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5624b6d67680 con 0x5624b6eae000 2023-05-20T01:22:18.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.083+0000 7fe20140e7c0 1 -- 172.21.15.16:0/2808449806 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:15 10.28 10:177a762d::::head [pgnls start_epoch 71 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x5624b78c6e00 con 0x5624b6eae000 2023-05-20T01:22:18.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.082+0000 7ffaa13af700 1 -- 172.21.15.16:0/1771343338 <== osd.4 v2:172.21.15.132:6817/506317223 1 ==== osd_op_reply(11 [pgnls start_epoch 71 out=49b] v69'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5608892678c0 con 0x560889291400 2023-05-20T01:22:18.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.083+0000 7ffa9c3a5700 1 -- 172.21.15.16:0/1771343338 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:12 10.2 10:40000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x56088921f500 con 0x560889282800 2023-05-20T01:22:18.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.083+0000 7fee7c678700 1 -- 172.21.15.16:0/1398005986 <== osd.0 v2:172.21.15.16:6824/3923634179 1 ==== osd_op_reply(11 [pgnls start_epoch 71 out=49b] v69'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e8d1af5440 con 0x55e8d2659800 2023-05-20T01:22:18.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.083+0000 7fee77e6f700 1 --2- 172.21.15.16:0/1398005986 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x55e8d2659400 0x55e8d1c26b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.083+0000 7fee77e6f700 1 -- 172.21.15.16:0/1398005986 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:12 10.6 10:60000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x55e8d2673500 con 0x55e8d2659400 2023-05-20T01:22:18.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.083+0000 7f704206c700 1 -- 172.21.15.16:0/739319487 <== osd.1 v2:172.21.15.16:6808/3860264220 3 ==== osd_op_reply(10 100000001fb.00000000 [call] v71'17 uv17 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55f909cc3680 con 0x55f909ce7800 2023-05-20T01:22:18.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.083+0000 7f70438bd7c0 1 --2- 172.21.15.16:0/739319487 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x55f909ce1000 0x55f909a2d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.083+0000 7f70438bd7c0 1 -- 172.21.15.16:0/739319487 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:11 10.33 10:cc000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x55f90a82f880 con 0x55f909ce1000 2023-05-20T01:22:18.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.083+0000 7f5a539fb700 1 -- 172.21.15.16:0/494201187 <== osd.1 v2:172.21.15.16:6808/3860264220 3 ==== osd_op_reply(5 10000000200.00000000 [call] v71'13 uv13 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x559726fc18c0 con 0x559726ff0c00 2023-05-20T01:22:18.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.083+0000 7fee7d67a700 1 --2- 172.21.15.16:0/1398005986 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x55e8d2659400 0x55e8d1c26b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.083+0000 7f7d9b746700 1 --2- 172.21.15.16:0/1705842756 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x55687b065000 0x55687b188580 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.083+0000 7f5a55a4d7c0 1 -- 172.21.15.16:0/494201187 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:6 10.19 10:997b7129::::head [pgnls start_epoch 71 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x559727b1d500 con 0x559726ff0c00 2023-05-20T01:22:18.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.083+0000 7ffaa0bae700 1 -- 172.21.15.16:0/1771343338 <== osd.1 v2:172.21.15.16:6808/3860264220 3 ==== osd_op_reply(12 [pgnls start_epoch 71 out=49b] v69'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560889267440 con 0x560889282800 2023-05-20T01:22:18.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.083+0000 7ffa9cba6700 1 -- 172.21.15.16:0/1771343338 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:13 10.22 10:44000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x56088921ea80 con 0x560889290c00 2023-05-20T01:22:18.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.083+0000 7fe1ffbbd700 1 -- 172.21.15.16:0/2808449806 <== osd.7 v2:172.21.15.132:6800/466001149 4 ==== osd_op_reply(15 [pgnls start_epoch 71 out=81b] v71'9 uv9 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5624b6d67680 con 0x5624b6eae000 2023-05-20T01:22:18.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.083+0000 7fe20140e7c0 1 -- 172.21.15.16:0/2808449806 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:16 10.28 10:177a762d:::10000000201.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x5624b78c7180 con 0x5624b6eae000 2023-05-20T01:22:18.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.083+0000 7f7d9b746700 1 -- 172.21.15.16:0/1705842756 <== osd.0 v2:172.21.15.16:6824/3923634179 1 ==== osd_op_reply(9 [pgnls start_epoch 71 out=81b] v69'6 uv6 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55687b04f8c0 con 0x55687b065000 2023-05-20T01:22:18.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.083+0000 7f704186b700 1 --2- 172.21.15.16:0/739319487 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x55f909ce1000 0x55f909a2d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.083+0000 7f7d9df997c0 1 -- 172.21.15.16:0/1705842756 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:10 10.1e 10:789b801b:::1000000020c.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x55687bbbca80 con 0x55687b065000 2023-05-20T01:22:18.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.083+0000 7f6194987700 1 -- 172.21.15.16:0/3379690421 <== osd.6 v2:172.21.15.132:6801/727581123 6 ==== osd_op_reply(7 10000000207.00000000 [call] v71'8 uv8 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564461dd78c0 con 0x564461f2e400 2023-05-20T01:22:18.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.083+0000 7f5a539fb700 1 -- 172.21.15.16:0/494201187 <== osd.1 v2:172.21.15.16:6808/3860264220 4 ==== osd_op_reply(6 [pgnls start_epoch 71 out=81b] v71'13 uv13 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x559726fc18c0 con 0x559726ff0c00 2023-05-20T01:22:18.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.083+0000 7f5a55a4d7c0 1 -- 172.21.15.16:0/494201187 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:7 10.19 10:997b7129:::100000001f8.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x559727b1ca80 con 0x559726ff0c00 2023-05-20T01:22:18.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.083+0000 7f61961d87c0 1 -- 172.21.15.16:0/3379690421 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:8 10.7 10:e2e07930::::head [pgnls start_epoch 71 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x564462999880 con 0x564461f2e400 2023-05-20T01:22:18.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.083+0000 7ffaa1bb0700 1 -- 172.21.15.16:0/1771343338 <== osd.5 v2:172.21.15.132:6816/1844689995 2 ==== osd_op_reply(13 [pgnls start_epoch 71 out=49b] v69'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560889267680 con 0x560889290c00 2023-05-20T01:22:18.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.083+0000 7ffa9c3a5700 1 -- 172.21.15.16:0/1771343338 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:14 10.12 10:48000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x56088921e700 con 0x560889283000 2023-05-20T01:22:18.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.083+0000 7f7d9b746700 1 -- 172.21.15.16:0/1705842756 <== osd.0 v2:172.21.15.16:6824/3923634179 2 ==== osd_op_reply(10 1000000020c.00000000 [stat out=16b] v0'0 uv6 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55687b04f8c0 con 0x55687b065000 2023-05-20T01:22:18.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.083+0000 7f7d9df997c0 10 datascan.operator(): handling object 1099511628300.0 2023-05-20T01:22:18.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.083+0000 7fee7d67a700 1 --2- 172.21.15.16:0/1398005986 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x55e8d2659400 0x55e8d1c26b00 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.083+0000 7f7d9df997c0 1 -- 172.21.15.16:0/1705842756 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:11 10.1e 10:789b801b:::1000000020c.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x55687bbbd880 con 0x55687b065000 2023-05-20T01:22:18.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.084+0000 7fe1ffbbd700 1 -- 172.21.15.16:0/2808449806 <== osd.7 v2:172.21.15.132:6800/466001149 5 ==== osd_op_reply(16 10000000201.00000000 [stat out=16b] v0'0 uv7 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5624b6d67680 con 0x5624b6eae000 2023-05-20T01:22:18.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.084+0000 7f5a539fb700 1 -- 172.21.15.16:0/494201187 <== osd.1 v2:172.21.15.16:6808/3860264220 5 ==== osd_op_reply(7 100000001f8.00000000 [stat out=16b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x559726fc18c0 con 0x559726ff0c00 2023-05-20T01:22:18.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.084+0000 7f5a55a4d7c0 10 datascan.operator(): handling object 1099511628280.0 2023-05-20T01:22:18.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.084+0000 7fe20140e7c0 10 datascan.operator(): handling object 1099511628289.0 2023-05-20T01:22:18.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.084+0000 7fe20140e7c0 1 -- 172.21.15.16:0/2808449806 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:17 10.28 10:177a762d:::10000000201.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x5624b78c6700 con 0x5624b6eae000 2023-05-20T01:22:18.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.084+0000 7f5a55a4d7c0 1 -- 172.21.15.16:0/494201187 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:8 10.19 10:997b7129:::100000001f8.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x559727b1d880 con 0x559726ff0c00 2023-05-20T01:22:18.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.084+0000 7f704186b700 1 --2- 172.21.15.16:0/739319487 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x55f909ce1000 0x55f909a2d600 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.084+0000 7f6194987700 1 -- 172.21.15.16:0/3379690421 <== osd.6 v2:172.21.15.132:6801/727581123 7 ==== osd_op_reply(8 [pgnls start_epoch 71 out=81b] v71'8 uv8 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x564461dd78c0 con 0x564461f2e400 2023-05-20T01:22:18.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.084+0000 7f61961d87c0 1 -- 172.21.15.16:0/3379690421 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:9 10.7 10:e2e07930:::10000000001.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x564462998e00 con 0x564461f2e400 2023-05-20T01:22:18.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.084+0000 7ffaa1bb0700 1 -- 172.21.15.16:0/1771343338 <== osd.6 v2:172.21.15.132:6801/727581123 2 ==== osd_op_reply(14 [pgnls start_epoch 71 out=49b] v69'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560889267680 con 0x560889283000 2023-05-20T01:22:18.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.084+0000 7ffaa34017c0 1 -- 172.21.15.16:0/1771343338 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:15 11.4 11:24924924::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x56088921f880 con 0x560889291000 2023-05-20T01:22:18.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.084+0000 7fee7d67a700 1 -- 172.21.15.16:0/1398005986 <== osd.5 v2:172.21.15.132:6816/1844689995 1 ==== osd_op_reply(12 [pgnls start_epoch 71 out=49b] v69'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e8d1af5680 con 0x55e8d2659400 2023-05-20T01:22:18.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.084+0000 7fee78670700 1 -- 172.21.15.16:0/1398005986 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:13 10.26 10:64000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x55e8d2673180 con 0x55e8d2659400 2023-05-20T01:22:18.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.084+0000 7f704186b700 1 -- 172.21.15.16:0/739319487 <== osd.5 v2:172.21.15.132:6816/1844689995 1 ==== osd_op_reply(11 [pgnls start_epoch 71 out=49b] v69'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f909cc3440 con 0x55f909ce1000 2023-05-20T01:22:18.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.084+0000 7f703d062700 1 -- 172.21.15.16:0/739319487 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:12 10.b 10:d0000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x55f909e0e380 con 0x55f909ce0800 2023-05-20T01:22:18.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.084+0000 7f6194987700 1 -- 172.21.15.16:0/3379690421 <== osd.6 v2:172.21.15.132:6801/727581123 8 ==== osd_op_reply(9 10000000001.00000000 [stat out=16b] v0'0 uv6 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x564461dd78c0 con 0x564461f2e400 2023-05-20T01:22:18.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.084+0000 7f61961d87c0 10 datascan.operator(): handling object 1099511627777.0 2023-05-20T01:22:18.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.084+0000 7f61961d87c0 1 -- 172.21.15.16:0/3379690421 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:10 10.7 10:e2e07930:::10000000001.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x564462999180 con 0x564461f2e400 2023-05-20T01:22:18.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.084+0000 7ffaa13af700 1 -- 172.21.15.16:0/1771343338 <== osd.7 v2:172.21.15.132:6800/466001149 4 ==== osd_op_reply(15 [pgnls start_epoch 71 out=81b] v69'3 uv3 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5608892678c0 con 0x560889291000 2023-05-20T01:22:18.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.084+0000 7ffaa34017c0 1 -- 172.21.15.16:0/1771343338 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:16 11.4 11:24ae92ec:::100000001fb.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x56088921ee00 con 0x560889291000 2023-05-20T01:22:18.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.084+0000 7fee7d67a700 1 -- 172.21.15.16:0/1398005986 <== osd.5 v2:172.21.15.132:6816/1844689995 2 ==== osd_op_reply(13 [pgnls start_epoch 71 out=81b] v69'16 uv16 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55e8d1af5680 con 0x55e8d2659400 2023-05-20T01:22:18.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.084+0000 7fee7eecb7c0 1 -- 172.21.15.16:0/1398005986 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:14 10.26 10:658f2455:::100000001ff.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x55e8d1c3e000 con 0x55e8d2659400 2023-05-20T01:22:18.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.084+0000 7f704186b700 1 -- 172.21.15.16:0/739319487 <== osd.6 v2:172.21.15.132:6801/727581123 2 ==== osd_op_reply(12 [pgnls start_epoch 71 out=49b] v69'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f909cc3440 con 0x55f909ce0800 2023-05-20T01:22:18.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.084+0000 7f703c861700 1 -- 172.21.15.16:0/739319487 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:13 10.2b 10:d4000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x55f90a82ee00 con 0x55f909ce1000 2023-05-20T01:22:18.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.085+0000 7fee7d67a700 1 -- 172.21.15.16:0/1398005986 <== osd.5 v2:172.21.15.132:6816/1844689995 3 ==== osd_op_reply(14 100000001ff.00000000 [stat out=16b] v0'0 uv16 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55e8d1af5680 con 0x55e8d2659400 2023-05-20T01:22:18.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.085+0000 7fee7eecb7c0 10 datascan.operator(): handling object 1099511628287.0 2023-05-20T01:22:18.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.085+0000 7fee7eecb7c0 1 -- 172.21.15.16:0/1398005986 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:15 10.26 10:658f2455:::100000001ff.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x55e8d1c3e380 con 0x55e8d2659400 2023-05-20T01:22:18.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.085+0000 7ffaa13af700 1 -- 172.21.15.16:0/1771343338 <== osd.7 v2:172.21.15.132:6800/466001149 5 ==== osd_op_reply(16 100000001fb.00000001 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5608892678c0 con 0x560889291000 2023-05-20T01:22:18.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.085+0000 7ffaa34017c0 10 datascan.operator(): handling object 1099511628283.1 2023-05-20T01:22:18.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.085+0000 7ffaa34017c0 1 -- 172.21.15.16:0/1771343338 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:17 10.13 10:ca8fdbba:::100000001fb.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x56088921f180 con 0x560889282800 2023-05-20T01:22:18.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.085+0000 7f704186b700 1 -- 172.21.15.16:0/739319487 <== osd.5 v2:172.21.15.132:6816/1844689995 2 ==== osd_op_reply(13 [pgnls start_epoch 71 out=81b] v69'12 uv12 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55f909cc3440 con 0x55f909ce1000 2023-05-20T01:22:18.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.085+0000 7f70438bd7c0 1 -- 172.21.15.16:0/739319487 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:14 10.2b 10:d4a58307:::10000000203.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x55f90a82f180 con 0x55f909ce1000 2023-05-20T01:22:18.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.085+0000 7fe1ffbbd700 1 -- 172.21.15.16:0/2808449806 <== osd.7 v2:172.21.15.132:6800/466001149 6 ==== osd_op_reply(17 10000000201.00000000 [call] v71'10 uv10 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5624b6d67680 con 0x5624b6eae000 2023-05-20T01:22:18.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.085+0000 7fe20140e7c0 1 -- 172.21.15.16:0/2808449806 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:18 10.18 10:18000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x5624b78c7880 con 0x5624b6e76400 2023-05-20T01:22:18.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.086+0000 7fe1febbb700 1 -- 172.21.15.16:0/2808449806 <== osd.2 v2:172.21.15.16:6816/293187030 2 ==== osd_op_reply(18 [pgnls start_epoch 71 out=49b] v69'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5624b6d67440 con 0x5624b6e76400 2023-05-20T01:22:18.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.086+0000 7fe1fabb3700 1 --2- 172.21.15.16:0/2808449806 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x5624b6eae400 0x5624b6e9e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.086+0000 7fe1fabb3700 1 -- 172.21.15.16:0/2808449806 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:19 10.38 10:1c000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x5624b6eb8000 con 0x5624b6eae400 2023-05-20T01:22:18.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.085+0000 7f7d9b746700 1 -- 172.21.15.16:0/1705842756 <== osd.0 v2:172.21.15.16:6824/3923634179 3 ==== osd_op_reply(11 1000000020c.00000000 [call] v71'7 uv7 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55687b04f8c0 con 0x55687b065000 2023-05-20T01:22:18.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.086+0000 7f7d9df997c0 1 --2- 172.21.15.16:0/1705842756 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x55687b063000 0x55687adcd600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.086+0000 7f7d9df997c0 1 -- 172.21.15.16:0/1705842756 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:12 10.3e 10:7c000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x55687bbbdc00 con 0x55687b063000 2023-05-20T01:22:18.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.085+0000 7f5a539fb700 1 -- 172.21.15.16:0/494201187 <== osd.1 v2:172.21.15.16:6808/3860264220 6 ==== osd_op_reply(8 100000001f8.00000000 [call] v71'14 uv14 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x559726fc18c0 con 0x559726ff0c00 2023-05-20T01:22:18.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.085+0000 7f5a55a4d7c0 1 --2- 172.21.15.16:0/494201187 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x559726fc2c00 0x559726d2fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.085+0000 7f5a55a4d7c0 1 -- 172.21.15.16:0/494201187 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:9 10.39 10:9c000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x559727b1dc00 con 0x559726fc2c00 2023-05-20T01:22:18.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.086+0000 7f5a531fa700 1 --2- 172.21.15.16:0/494201187 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x559726fc2c00 0x559726d2fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.086+0000 7f7d9c748700 1 --2- 172.21.15.16:0/1705842756 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x55687b063000 0x55687adcd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.086+0000 7f704186b700 1 -- 172.21.15.16:0/739319487 <== osd.5 v2:172.21.15.132:6816/1844689995 3 ==== osd_op_reply(14 10000000203.00000000 [stat out=16b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55f909cc3440 con 0x55f909ce1000 2023-05-20T01:22:18.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.086+0000 7f70438bd7c0 10 datascan.operator(): handling object 1099511628291.0 2023-05-20T01:22:18.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.086+0000 7f70438bd7c0 1 -- 172.21.15.16:0/739319487 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:15 10.2b 10:d4a58307:::10000000203.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x55f909e0e000 con 0x55f909ce1000 2023-05-20T01:22:18.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.086+0000 7fe1ff3bc700 1 --2- 172.21.15.16:0/2808449806 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x5624b6eae400 0x5624b6e9e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.086+0000 7f7d9c748700 1 --2- 172.21.15.16:0/1705842756 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x55687b063000 0x55687adcd600 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.086+0000 7f5a531fa700 1 --2- 172.21.15.16:0/494201187 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x559726fc2c00 0x559726d2fb80 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.086+0000 7ffaa0bae700 1 -- 172.21.15.16:0/1771343338 <== osd.1 v2:172.21.15.16:6808/3860264220 4 ==== osd_op_reply(17 100000001fb.00000000 [call,setxattr (8)] v71'18 uv18 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x560889267440 con 0x560889282800 2023-05-20T01:22:18.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.086+0000 7fe1ff3bc700 1 --2- 172.21.15.16:0/2808449806 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x5624b6eae400 0x5624b6e9e100 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.086+0000 7ffaa34017c0 1 -- 172.21.15.16:0/1771343338 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:18 11.14 11:28000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x56088921f880 con 0x560889283000 2023-05-20T01:22:18.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.086+0000 7fee7d67a700 1 -- 172.21.15.16:0/1398005986 <== osd.5 v2:172.21.15.132:6816/1844689995 4 ==== osd_op_reply(15 100000001ff.00000000 [call] v71'17 uv17 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55e8d1af5680 con 0x55e8d2659400 2023-05-20T01:22:18.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.086+0000 7fee7eecb7c0 1 -- 172.21.15.16:0/1398005986 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:16 10.16 10:68000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x55e8d1c3e700 con 0x55e8d2656800 2023-05-20T01:22:18.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.087+0000 7f7d9c748700 1 -- 172.21.15.16:0/1705842756 <== osd.1 v2:172.21.15.16:6808/3860264220 1 ==== osd_op_reply(12 [pgnls start_epoch 71 out=49b] v69'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55687b04f680 con 0x55687b063000 2023-05-20T01:22:18.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.086+0000 7f6194987700 1 -- 172.21.15.16:0/3379690421 <== osd.6 v2:172.21.15.132:6801/727581123 9 ==== osd_op_reply(10 10000000001.00000000 [call] v71'9 uv9 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564461dd78c0 con 0x564461f2e400 2023-05-20T01:22:18.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.087+0000 7f61961d87c0 1 --2- 172.21.15.16:0/3379690421 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x564461e21000 0x564461f5c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.087+0000 7f7d96f3d700 1 -- 172.21.15.16:0/1705842756 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:13 10.1 10:80000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x55687bbbd500 con 0x55687b062c00 2023-05-20T01:22:18.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.087+0000 7f5a531fa700 1 -- 172.21.15.16:0/494201187 <== osd.2 v2:172.21.15.16:6816/293187030 1 ==== osd_op_reply(9 [pgnls start_epoch 71 out=49b] v69'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559726fc1440 con 0x559726fc2c00 2023-05-20T01:22:18.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.087+0000 7f61961d87c0 1 -- 172.21.15.16:0/3379690421 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:11 10.27 10:e4000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x564462998a80 con 0x564461e21000 2023-05-20T01:22:18.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.087+0000 7f5a4f1f2700 1 -- 172.21.15.16:0/494201187 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:10 10.5 10:a0000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x559727b1c700 con 0x559726fc2c00 2023-05-20T01:22:18.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.087+0000 7f6194186700 1 --2- 172.21.15.16:0/3379690421 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x564461e21000 0x564461f5c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.087+0000 7fe1ff3bc700 1 -- 172.21.15.16:0/2808449806 <== osd.5 v2:172.21.15.132:6816/1844689995 1 ==== osd_op_reply(19 [pgnls start_epoch 71 out=49b] v69'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5624b6d678c0 con 0x5624b6eae400 2023-05-20T01:22:18.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.087+0000 7fe1fa3b2700 1 -- 172.21.15.16:0/2808449806 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:20 10.4 10:20000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x5624b6eb8700 con 0x5624b6a39c00 2023-05-20T01:22:18.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.087+0000 7ffaa1bb0700 1 -- 172.21.15.16:0/1771343338 <== osd.6 v2:172.21.15.132:6801/727581123 3 ==== osd_op_reply(18 [pgnls start_epoch 71 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560889267680 con 0x560889283000 2023-05-20T01:22:18.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.087+0000 7ffa9cba6700 1 -- 172.21.15.16:0/1771343338 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:19 11.c 11:30000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x5608893bc000 con 0x560889282800 2023-05-20T01:22:18.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.087+0000 7fee7c678700 1 -- 172.21.15.16:0/1398005986 <== osd.6 v2:172.21.15.132:6801/727581123 4 ==== osd_op_reply(16 [pgnls start_epoch 71 out=49b] v69'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e8d1af5440 con 0x55e8d2656800 2023-05-20T01:22:18.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.087+0000 7f5a531fa700 1 -- 172.21.15.16:0/494201187 <== osd.2 v2:172.21.15.16:6816/293187030 2 ==== osd_op_reply(10 [pgnls start_epoch 71 out=81b] v69'6 uv6 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x559726fc1440 con 0x559726fc2c00 2023-05-20T01:22:18.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.087+0000 7fee78670700 1 -- 172.21.15.16:0/1398005986 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:17 10.36 10:6c000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x55e8d2673500 con 0x55e8d2658c00 2023-05-20T01:22:18.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.087+0000 7f5a55a4d7c0 1 -- 172.21.15.16:0/494201187 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:11 10.5 10:a0beda95:::10000000204.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x559727b1ce00 con 0x559726fc2c00 2023-05-20T01:22:18.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.087+0000 7f7d9c748700 1 -- 172.21.15.16:0/1705842756 <== osd.4 v2:172.21.15.132:6817/506317223 2 ==== osd_op_reply(13 [pgnls start_epoch 71 out=49b] v69'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55687b04f680 con 0x55687b062c00 2023-05-20T01:22:18.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.087+0000 7fe1ffbbd700 1 -- 172.21.15.16:0/2808449806 <== osd.0 v2:172.21.15.16:6824/3923634179 5 ==== osd_op_reply(20 [pgnls start_epoch 71 out=49b] v69'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5624b6d67680 con 0x5624b6a39c00 2023-05-20T01:22:18.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.087+0000 7fe1fabb3700 1 --2- 172.21.15.16:0/2808449806 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x5624b6eae800 0x5624b6e9e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.087+0000 7fe1fabb3700 1 -- 172.21.15.16:0/2808449806 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:21 10.24 10:24000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x5624b6eb8380 con 0x5624b6eae800 2023-05-20T01:22:18.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.087+0000 7fe1febbb700 1 --2- 172.21.15.16:0/2808449806 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x5624b6eae800 0x5624b6e9e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.087+0000 7ffaa0bae700 1 -- 172.21.15.16:0/1771343338 <== osd.1 v2:172.21.15.16:6808/3860264220 5 ==== osd_op_reply(19 [pgnls start_epoch 71 out=101b] v69'4 uv4 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x560889267440 con 0x560889282800 2023-05-20T01:22:18.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.087+0000 7ffaa34017c0 1 -- 172.21.15.16:0/1771343338 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:20 11.c 11:30604137:::100000001fc.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x5608893bca80 con 0x560889282800 2023-05-20T01:22:18.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.087+0000 7fee7d67a700 1 -- 172.21.15.16:0/1398005986 <== osd.7 v2:172.21.15.132:6800/466001149 2 ==== osd_op_reply(17 [pgnls start_epoch 71 out=101b] v71'19 uv19 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55e8d1af5680 con 0x55e8d2658c00 2023-05-20T01:22:18.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.087+0000 7fee7eecb7c0 1 -- 172.21.15.16:0/1398005986 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:18 11.12 11:49249249::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x55e8d1c3ea80 con 0x55e8d2658c00 2023-05-20T01:22:18.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.087+0000 7f7d9773e700 1 -- 172.21.15.16:0/1705842756 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:14 10.21 10:84000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x55687bbbc700 con 0x55687b062c00 2023-05-20T01:22:18.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.087+0000 7f5a531fa700 1 -- 172.21.15.16:0/494201187 <== osd.2 v2:172.21.15.16:6816/293187030 3 ==== osd_op_reply(11 10000000204.00000000 [stat out=16b] v0'0 uv6 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x559726fc1440 con 0x559726fc2c00 2023-05-20T01:22:18.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.087+0000 7f5a55a4d7c0 10 datascan.operator(): handling object 1099511628292.0 2023-05-20T01:22:18.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.087+0000 7f5a55a4d7c0 1 -- 172.21.15.16:0/494201187 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:12 10.5 10:a0beda95:::10000000204.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x559727b1d500 con 0x559726fc2c00 2023-05-20T01:22:18.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.087+0000 7f6194186700 1 --2- 172.21.15.16:0/3379690421 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x564461e21000 0x564461f5c000 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.088+0000 7f7d9c748700 1 -- 172.21.15.16:0/1705842756 <== osd.4 v2:172.21.15.132:6817/506317223 3 ==== osd_op_reply(14 [pgnls start_epoch 71 out=49b] v69'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55687b04f680 con 0x55687b062c00 2023-05-20T01:22:18.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.088+0000 7f6194186700 1 -- 172.21.15.16:0/3379690421 <== osd.5 v2:172.21.15.132:6816/1844689995 1 ==== osd_op_reply(11 [pgnls start_epoch 71 out=81b] v69'3 uv3 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x564461dd7b00 con 0x564461e21000 2023-05-20T01:22:18.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.088+0000 7f7d96f3d700 1 --2- 172.21.15.16:0/1705842756 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x55687b065400 0x55687b188b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.088+0000 7f7d96f3d700 1 -- 172.21.15.16:0/1705842756 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:15 10.11 10:88000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x55687bbbdc00 con 0x55687b065400 2023-05-20T01:22:18.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.088+0000 7f61961d87c0 1 -- 172.21.15.16:0/3379690421 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:12 10.27 10:e5993619:::10000000208.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x564462999c00 con 0x564461e21000 2023-05-20T01:22:18.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.088+0000 7ffaa0bae700 1 -- 172.21.15.16:0/1771343338 <== osd.1 v2:172.21.15.16:6808/3860264220 6 ==== osd_op_reply(20 100000001fc.00000000 [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x560889267440 con 0x560889282800 2023-05-20T01:22:18.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.088+0000 7ffaa34017c0 10 datascan.operator(): handling object 1099511628284.0 2023-05-20T01:22:18.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.088+0000 7ffaa34017c0 1 -- 172.21.15.16:0/1771343338 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:21 10.c 10:30604137:::100000001fc.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x5608893bce00 con 0x560889291000 2023-05-20T01:22:18.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.088+0000 7f704186b700 1 -- 172.21.15.16:0/739319487 <== osd.5 v2:172.21.15.132:6816/1844689995 4 ==== osd_op_reply(15 10000000203.00000000 [call] v71'13 uv13 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55f909cc3440 con 0x55f909ce1000 2023-05-20T01:22:18.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.088+0000 7f70438bd7c0 1 --2- 172.21.15.16:0/739319487 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x55f909ce1400 0x55f909df3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.088+0000 7f70438bd7c0 1 -- 172.21.15.16:0/739319487 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:16 10.1b 10:d8000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x55f909e0e380 con 0x55f909ce1400 2023-05-20T01:22:18.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.088+0000 7fe1febbb700 1 --2- 172.21.15.16:0/2808449806 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x5624b6eae800 0x5624b6e9e680 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.088+0000 7fee7d67a700 1 -- 172.21.15.16:0/1398005986 <== osd.7 v2:172.21.15.132:6800/466001149 3 ==== osd_op_reply(18 [pgnls start_epoch 71 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e8d1af5680 con 0x55e8d2658c00 2023-05-20T01:22:18.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.088+0000 7fee78670700 1 -- 172.21.15.16:0/1398005986 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:19 11.a 11:50000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x55e8d1c3e700 con 0x55e8d2659000 2023-05-20T01:22:18.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.088+0000 7f7d9bf47700 1 --2- 172.21.15.16:0/1705842756 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x55687b065400 0x55687b188b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.088+0000 7f704106a700 1 --2- 172.21.15.16:0/739319487 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x55f909ce1400 0x55f909df3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.088+0000 7f6194186700 1 -- 172.21.15.16:0/3379690421 <== osd.5 v2:172.21.15.132:6816/1844689995 2 ==== osd_op_reply(12 10000000208.00000000 [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x564461dd7b00 con 0x564461e21000 2023-05-20T01:22:18.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.088+0000 7f61961d87c0 10 datascan.operator(): handling object 1099511628296.0 2023-05-20T01:22:18.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.088+0000 7f61961d87c0 1 -- 172.21.15.16:0/3379690421 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:13 10.27 10:e5993619:::10000000208.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x564462999500 con 0x564461e21000 2023-05-20T01:22:18.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.088+0000 7fe1febbb700 1 -- 172.21.15.16:0/2808449806 <== osd.1 v2:172.21.15.16:6808/3860264220 1 ==== osd_op_reply(21 [pgnls start_epoch 71 out=49b] v69'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5624b6d67440 con 0x5624b6eae800 2023-05-20T01:22:18.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.088+0000 7fe20140e7c0 1 -- 172.21.15.16:0/2808449806 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:22 11.0 11:00000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x5624b78c7c00 con 0x5624b6eae800 2023-05-20T01:22:18.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.088+0000 7fee7ce79700 1 -- 172.21.15.16:0/1398005986 <== osd.3 v2:172.21.15.16:6800/3669781890 7 ==== osd_op_reply(19 [pgnls start_epoch 71 out=101b] v69'6 uv6 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55e8d1af58c0 con 0x55e8d2659000 2023-05-20T01:22:18.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.088+0000 7fee7eecb7c0 1 -- 172.21.15.16:0/1398005986 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:20 11.a 11:5097359c:::100000001fd.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x55e8d1c3ee00 con 0x55e8d2659000 2023-05-20T01:22:18.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.089+0000 7f704106a700 1 --2- 172.21.15.16:0/739319487 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x55f909ce1400 0x55f909df3600 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.089+0000 7fe1febbb700 1 -- 172.21.15.16:0/2808449806 <== osd.1 v2:172.21.15.16:6808/3860264220 2 ==== osd_op_reply(22 [pgnls start_epoch 71 out=101b] v69'5 uv5 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x5624b6d67440 con 0x5624b6eae800 2023-05-20T01:22:18.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.089+0000 7f7d9bf47700 1 --2- 172.21.15.16:0/1705842756 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x55687b065400 0x55687b188b00 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.089+0000 7fe20140e7c0 1 -- 172.21.15.16:0/2808449806 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:23 11.0 11:012cdbc7:::10000000206.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x5624b78c6e00 con 0x5624b6eae800 2023-05-20T01:22:18.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.089+0000 7fee7ce79700 1 -- 172.21.15.16:0/1398005986 <== osd.3 v2:172.21.15.16:6800/3669781890 8 ==== osd_op_reply(20 100000001fd.00000001 [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55e8d1af58c0 con 0x55e8d2659000 2023-05-20T01:22:18.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.089+0000 7fee7eecb7c0 10 datascan.operator(): handling object 1099511628285.1 2023-05-20T01:22:18.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.089+0000 7fee7eecb7c0 1 -- 172.21.15.16:0/1398005986 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:21 10.10 10:08b99fea:::100000001fd.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x55e8d1c3e700 con 0x55e8d2659000 2023-05-20T01:22:18.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.089+0000 7f5a531fa700 1 -- 172.21.15.16:0/494201187 <== osd.2 v2:172.21.15.16:6816/293187030 4 ==== osd_op_reply(12 10000000204.00000000 [call] v71'7 uv7 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x559726fc1440 con 0x559726fc2c00 2023-05-20T01:22:18.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.089+0000 7f7d9bf47700 1 -- 172.21.15.16:0/1705842756 <== osd.5 v2:172.21.15.132:6816/1844689995 1 ==== osd_op_reply(15 [pgnls start_epoch 71 out=49b] v69'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55687b04f440 con 0x55687b065400 2023-05-20T01:22:18.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.089+0000 7f7d9773e700 1 -- 172.21.15.16:0/1705842756 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:16 10.31 10:8c000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x55687bbbca80 con 0x55687b062800 2023-05-20T01:22:18.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.089+0000 7f5a55a4d7c0 1 --2- 172.21.15.16:0/494201187 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x559726fc3000 0x559726d2f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.089+0000 7fe1febbb700 1 -- 172.21.15.16:0/2808449806 <== osd.1 v2:172.21.15.16:6808/3860264220 3 ==== osd_op_reply(23 10000000206.00000000 [stat out=16b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5624b6d67440 con 0x5624b6eae800 2023-05-20T01:22:18.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.089+0000 7fe20140e7c0 10 datascan.operator(): handling object 1099511628294.0 2023-05-20T01:22:18.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.089+0000 7f5a55a4d7c0 1 -- 172.21.15.16:0/494201187 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:13 10.25 10:a4000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x559727b1ca80 con 0x559726fc3000 2023-05-20T01:22:18.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.089+0000 7ffaa13af700 1 -- 172.21.15.16:0/1771343338 <== osd.7 v2:172.21.15.132:6800/466001149 6 ==== osd_op_reply(21 100000001fc.00000000 [call,setxattr (8)] v71'9 uv9 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x5608892678c0 con 0x560889291000 2023-05-20T01:22:18.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.089+0000 7ffaa34017c0 1 -- 172.21.15.16:0/1771343338 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:22 11.c 11:37389ade::::head [pgnls start_epoch 71 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x5608893bd180 con 0x560889282800 2023-05-20T01:22:18.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.089+0000 7f704106a700 1 -- 172.21.15.16:0/739319487 <== osd.3 v2:172.21.15.16:6800/3669781890 1 ==== osd_op_reply(16 [pgnls start_epoch 71 out=49b] v69'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f909cc38c0 con 0x55f909ce1400 2023-05-20T01:22:18.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.089+0000 7f70438bd7c0 1 -- 172.21.15.16:0/739319487 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:17 11.d 11:b6db6db6::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x55f909e0e700 con 0x55f909ce0c00 2023-05-20T01:22:18.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.089+0000 7fe20140e7c0 1 -- 172.21.15.16:0/2808449806 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:24 10.0 10:012cdbc7:::10000000206.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x5624b78c7180 con 0x5624b78d2400 2023-05-20T01:22:18.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.089+0000 7f5a541fc700 1 --2- 172.21.15.16:0/494201187 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x559726fc3000 0x559726d2f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.089+0000 7f704106a700 1 -- 172.21.15.16:0/739319487 <== osd.2 v2:172.21.15.16:6816/293187030 2 ==== osd_op_reply(17 [pgnls start_epoch 71 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f909cc38c0 con 0x55f909ce0c00 2023-05-20T01:22:18.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.089+0000 7f7d9bf47700 1 -- 172.21.15.16:0/1705842756 <== osd.7 v2:172.21.15.132:6800/466001149 7 ==== osd_op_reply(16 [pgnls start_epoch 71 out=49b] v69'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55687b04f440 con 0x55687b062800 2023-05-20T01:22:18.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.089+0000 7f703d062700 1 -- 172.21.15.16:0/739319487 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:18 11.1d 11:b8000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x55f909e0e380 con 0x55f909ce7400 2023-05-20T01:22:18.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.090+0000 7ffaa0bae700 1 -- 172.21.15.16:0/1771343338 <== osd.1 v2:172.21.15.16:6808/3860264220 7 ==== osd_op_reply(22 [pgnls start_epoch 71 out=81b] v69'4 uv4 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x560889267440 con 0x560889282800 2023-05-20T01:22:18.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.090+0000 7f7d96f3d700 1 -- 172.21.15.16:0/1705842756 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:17 10.9 10:90000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x55687bbbd180 con 0x55687b065400 2023-05-20T01:22:18.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.090+0000 7ffaa34017c0 1 -- 172.21.15.16:0/1771343338 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:23 11.c 11:37389ade:::10000000209.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x5608893bd500 con 0x560889282800 2023-05-20T01:22:18.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.090+0000 7f7d9bf47700 1 -- 172.21.15.16:0/1705842756 <== osd.5 v2:172.21.15.132:6816/1844689995 2 ==== osd_op_reply(17 [pgnls start_epoch 71 out=49b] v69'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55687b04f440 con 0x55687b065400 2023-05-20T01:22:18.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.090+0000 7f5a541fc700 1 --2- 172.21.15.16:0/494201187 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x559726fc3000 0x559726d2f600 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.090+0000 7ffaa0bae700 1 -- 172.21.15.16:0/1771343338 <== osd.1 v2:172.21.15.16:6808/3860264220 8 ==== osd_op_reply(23 10000000209.00000000 [stat out=16b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x560889267440 con 0x560889282800 2023-05-20T01:22:18.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.090+0000 7ffaa34017c0 10 datascan.operator(): handling object 1099511628297.0 2023-05-20T01:22:18.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.090+0000 7ffaa34017c0 1 -- 172.21.15.16:0/1771343338 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:24 10.2c 10:37389ade:::10000000209.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x5608893bd880 con 0x560889282c00 2023-05-20T01:22:18.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.090+0000 7f7d9df997c0 1 -- 172.21.15.16:0/1705842756 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:18 11.16 11:6db6db6d::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x55687bbbca80 con 0x55687b065400 2023-05-20T01:22:18.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.090+0000 7f6194186700 1 -- 172.21.15.16:0/3379690421 <== osd.5 v2:172.21.15.132:6816/1844689995 3 ==== osd_op_reply(13 10000000208.00000000 [call] v71'4 uv4 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564461dd7b00 con 0x564461e21000 2023-05-20T01:22:18.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.090+0000 7f704186b700 1 -- 172.21.15.16:0/739319487 <== osd.7 v2:172.21.15.132:6800/466001149 5 ==== osd_op_reply(18 [pgnls start_epoch 71 out=101b] v69'4 uv4 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55f909cc3440 con 0x55f909ce7400 2023-05-20T01:22:18.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.090+0000 7f70438bd7c0 1 -- 172.21.15.16:0/739319487 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:19 11.1d 11:b9a363c8:::1000000020d.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x55f909e0ea80 con 0x55f909ce7400 2023-05-20T01:22:18.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.090+0000 7f61961d87c0 1 --2- 172.21.15.16:0/3379690421 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x564461e21400 0x564461f5c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.090+0000 7f61961d87c0 1 -- 172.21.15.16:0/3379690421 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:14 10.17 10:e8000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x564462999880 con 0x564461e21400 2023-05-20T01:22:18.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.090+0000 7fee7ce79700 1 -- 172.21.15.16:0/1398005986 <== osd.3 v2:172.21.15.16:6800/3669781890 9 ==== osd_op_reply(21 100000001fd.00000000 [call,setxattr (8)] v71'9 uv9 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x55e8d1af58c0 con 0x55e8d2659000 2023-05-20T01:22:18.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.090+0000 7f5a541fc700 1 -- 172.21.15.16:0/494201187 <== osd.4 v2:172.21.15.132:6817/506317223 1 ==== osd_op_reply(13 [pgnls start_epoch 71 out=49b] v69'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559726fc1680 con 0x559726fc3000 2023-05-20T01:22:18.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.090+0000 7f7d9bf47700 1 -- 172.21.15.16:0/1705842756 <== osd.5 v2:172.21.15.132:6816/1844689995 3 ==== osd_op_reply(18 [pgnls start_epoch 71 out=101b] v69'5 uv5 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55687b04f440 con 0x55687b065400 2023-05-20T01:22:18.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.090+0000 7fee7eecb7c0 1 -- 172.21.15.16:0/1398005986 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:22 11.a 11:55a6cd83::::head [pgnls start_epoch 71 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x55e8d1c3e000 con 0x55e8d2659000 2023-05-20T01:22:18.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.090+0000 7f5a4f1f2700 1 -- 172.21.15.16:0/494201187 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:14 10.15 10:a8000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x559727b1dc00 con 0x559726ff0800 2023-05-20T01:22:18.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.090+0000 7f704186b700 1 -- 172.21.15.16:0/739319487 <== osd.7 v2:172.21.15.132:6800/466001149 6 ==== osd_op_reply(19 1000000020d.00000001 [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55f909cc3440 con 0x55f909ce7400 2023-05-20T01:22:18.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.091+0000 7f70438bd7c0 10 datascan.operator(): handling object 1099511628301.1 2023-05-20T01:22:18.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.090+0000 7f6193985700 1 --2- 172.21.15.16:0/3379690421 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x564461e21400 0x564461f5c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.091+0000 7f70438bd7c0 1 -- 172.21.15.16:0/739319487 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:20 10.15 10:a8e69ea8:::1000000020d.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x55f909e0ee00 con 0x55f909ce1400 2023-05-20T01:22:18.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.091+0000 7f7d9df997c0 1 -- 172.21.15.16:0/1705842756 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:19 11.16 11:6ee5e2c4:::100000001f9.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x55687bbbd880 con 0x55687b065400 2023-05-20T01:22:18.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.091+0000 7fee7ce79700 1 -- 172.21.15.16:0/1398005986 <== osd.3 v2:172.21.15.16:6800/3669781890 10 ==== osd_op_reply(22 [pgnls start_epoch 71 out=101b] v69'6 uv6 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55e8d1af58c0 con 0x55e8d2659000 2023-05-20T01:22:18.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.091+0000 7fe1febbb700 1 -- 172.21.15.16:0/2808449806 <== osd.6 v2:172.21.15.132:6801/727581123 4 ==== osd_op_reply(24 10000000206.00000000 [call,setxattr (8)] v71'11 uv11 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x5624b6d67440 con 0x5624b78d2400 2023-05-20T01:22:18.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.091+0000 7fee7eecb7c0 1 -- 172.21.15.16:0/1398005986 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:23 11.a 11:55a6cd83:::1000000020a.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x55e8d1c3e380 con 0x55e8d2659000 2023-05-20T01:22:18.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.091+0000 7fe20140e7c0 1 -- 172.21.15.16:0/2808449806 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:25 11.0 11:04f22c1d::::head [pgnls start_epoch 71 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x5624b6eb8380 con 0x5624b6eae800 2023-05-20T01:22:18.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.091+0000 7f7d9bf47700 1 -- 172.21.15.16:0/1705842756 <== osd.5 v2:172.21.15.132:6816/1844689995 4 ==== osd_op_reply(19 100000001f9.00000000 [stat out=16b] v0'0 uv5 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55687b04f440 con 0x55687b065400 2023-05-20T01:22:18.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.091+0000 7f7d9df997c0 10 datascan.operator(): handling object 1099511628281.0 2023-05-20T01:22:18.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.091+0000 7f6193985700 1 --2- 172.21.15.16:0/3379690421 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x564461e21400 0x564461f5c580 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.091+0000 7f7d9df997c0 1 -- 172.21.15.16:0/1705842756 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:20 10.36 10:6ee5e2c4:::100000001f9.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x55687b1a6000 con 0x55687b062800 2023-05-20T01:22:18.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.091+0000 7fe1febbb700 1 -- 172.21.15.16:0/2808449806 <== osd.1 v2:172.21.15.16:6808/3860264220 4 ==== osd_op_reply(25 [pgnls start_epoch 71 out=101b] v69'5 uv5 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x5624b6d67440 con 0x5624b6eae800 2023-05-20T01:22:18.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.091+0000 7fe20140e7c0 1 -- 172.21.15.16:0/2808449806 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:26 11.0 11:04f22c1d:::1000000020a.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x5624b6eb8e00 con 0x5624b6eae800 2023-05-20T01:22:18.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.091+0000 7f6193985700 1 -- 172.21.15.16:0/3379690421 <== osd.4 v2:172.21.15.132:6817/506317223 1 ==== osd_op_reply(14 [pgnls start_epoch 71 out=49b] v69'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564461dd7680 con 0x564461e21400 2023-05-20T01:22:18.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.091+0000 7fee7ce79700 1 -- 172.21.15.16:0/1398005986 <== osd.3 v2:172.21.15.16:6800/3669781890 11 ==== osd_op_reply(23 1000000020a.00000000 [stat out=16b] v0'0 uv6 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55e8d1af58c0 con 0x55e8d2659000 2023-05-20T01:22:18.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.091+0000 7fee7eecb7c0 10 datascan.operator(): handling object 1099511628298.0 2023-05-20T01:22:18.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.091+0000 7fee7eecb7c0 1 -- 172.21.15.16:0/1398005986 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:24 10.2a 10:55a6cd83:::1000000020a.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x55e8d2672700 con 0x55e8d2659000 2023-05-20T01:22:18.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.091+0000 7f5a541fc700 1 -- 172.21.15.16:0/494201187 <== osd.3 v2:172.21.15.16:6800/3669781890 2 ==== osd_op_reply(14 [pgnls start_epoch 71 out=81b] v71'8 uv8 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x559726fc1680 con 0x559726ff0800 2023-05-20T01:22:18.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.091+0000 7f618f17c700 1 --2- 172.21.15.16:0/3379690421 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x564461f2e800 0x564461f5db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.091+0000 7f618f17c700 1 -- 172.21.15.16:0/3379690421 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:15 10.37 10:ec000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x564462998700 con 0x564461f2e800 2023-05-20T01:22:18.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.091+0000 7f5a55a4d7c0 1 -- 172.21.15.16:0/494201187 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:15 10.15 10:a8e69ea8:::1000000020d.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x559727b1d180 con 0x559726ff0800 2023-05-20T01:22:18.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.092+0000 7ffaa0bae700 1 -- 172.21.15.16:0/1771343338 <== osd.3 v2:172.21.15.16:6800/3669781890 4 ==== osd_op_reply(24 10000000209.00000000 [call,setxattr (8)] v71'11 uv11 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x560889267440 con 0x560889282c00 2023-05-20T01:22:18.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.092+0000 7ffaa34017c0 1 -- 172.21.15.16:0/1771343338 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:25 11.1c 11:38000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x5608893bdc00 con 0x560889283000 2023-05-20T01:22:18.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.092+0000 7fe1febbb700 1 -- 172.21.15.16:0/2808449806 <== osd.1 v2:172.21.15.16:6808/3860264220 5 ==== osd_op_reply(26 1000000020a.00000001 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5624b6d67440 con 0x5624b6eae800 2023-05-20T01:22:18.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.092+0000 7fe20140e7c0 10 datascan.operator(): handling object 1099511628298.1 2023-05-20T01:22:18.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.092+0000 7fe20140e7c0 1 -- 172.21.15.16:0/2808449806 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:27 10.2a 10:55a6cd83:::1000000020a.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x5624b6eb9180 con 0x5624b6a39800 2023-05-20T01:22:18.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.092+0000 7f6194987700 1 --2- 172.21.15.16:0/3379690421 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x564461f2e800 0x564461f5db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.092+0000 7f704106a700 1 -- 172.21.15.16:0/739319487 <== osd.3 v2:172.21.15.16:6800/3669781890 2 ==== osd_op_reply(20 1000000020d.00000000 [call,setxattr (8)] v71'8 uv8 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x55f909cc38c0 con 0x55f909ce1400 2023-05-20T01:22:18.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.092+0000 7ffaa1bb0700 1 -- 172.21.15.16:0/1771343338 <== osd.6 v2:172.21.15.132:6801/727581123 4 ==== osd_op_reply(25 [pgnls start_epoch 71 out=81b] v69'1 uv1 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x560889267680 con 0x560889283000 2023-05-20T01:22:18.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.092+0000 7f70438bd7c0 1 -- 172.21.15.16:0/739319487 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:21 11.1d 11:bbb18289::::head [pgnls start_epoch 71 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x55f909e0f180 con 0x55f909ce7400 2023-05-20T01:22:18.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.092+0000 7ffaa34017c0 1 -- 172.21.15.16:0/1771343338 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:26 11.1c 11:3d6db04b:::100000001f6.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x5608893bc000 con 0x560889283000 2023-05-20T01:22:18.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.092+0000 7f5a541fc700 1 -- 172.21.15.16:0/494201187 <== osd.3 v2:172.21.15.16:6800/3669781890 3 ==== osd_op_reply(15 1000000020d.00000000 [stat out=16b] v0'0 uv8 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x559726fc1680 con 0x559726ff0800 2023-05-20T01:22:18.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.092+0000 7f5a55a4d7c0 10 datascan.operator(): handling object 1099511628301.0 2023-05-20T01:22:18.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.092+0000 7f5a55a4d7c0 1 -- 172.21.15.16:0/494201187 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:16 10.15 10:a8e69ea8:::1000000020d.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x559727b1ca80 con 0x559726ff0800 2023-05-20T01:22:18.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.092+0000 7f6194987700 1 --2- 172.21.15.16:0/3379690421 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x564461f2e800 0x564461f5db80 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.092+0000 7f704186b700 1 -- 172.21.15.16:0/739319487 <== osd.7 v2:172.21.15.132:6800/466001149 7 ==== osd_op_reply(21 [pgnls start_epoch 71 out=101b] v69'4 uv4 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55f909cc3440 con 0x55f909ce7400 2023-05-20T01:22:18.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.092+0000 7f70438bd7c0 1 -- 172.21.15.16:0/739319487 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:22 11.1d 11:bbb18289:::10000000204.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x55f909e0e000 con 0x55f909ce7400 2023-05-20T01:22:18.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.093+0000 7ffaa1bb0700 1 -- 172.21.15.16:0/1771343338 <== osd.6 v2:172.21.15.132:6801/727581123 5 ==== osd_op_reply(26 100000001f6.00000001 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x560889267680 con 0x560889283000 2023-05-20T01:22:18.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.093+0000 7ffaa34017c0 10 datascan.operator(): handling object 1099511628278.1 2023-05-20T01:22:18.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.093+0000 7ffaa34017c0 1 -- 172.21.15.16:0/1771343338 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:27 10.2a 10:57fe7535:::100000001f6.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x5608893bca80 con 0x560889282c00 2023-05-20T01:22:18.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.093+0000 7f6194987700 1 -- 172.21.15.16:0/3379690421 <== osd.7 v2:172.21.15.132:6800/466001149 1 ==== osd_op_reply(15 [pgnls start_epoch 71 out=49b] v69'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564461dd78c0 con 0x564461f2e800 2023-05-20T01:22:18.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.093+0000 7f618f97d700 1 -- 172.21.15.16:0/3379690421 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:16 10.f 10:f0000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x564462999880 con 0x564461f2e400 2023-05-20T01:22:18.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.093+0000 7f704186b700 1 -- 172.21.15.16:0/739319487 <== osd.7 v2:172.21.15.132:6800/466001149 8 ==== osd_op_reply(22 10000000204.00000001 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55f909cc3440 con 0x55f909ce7400 2023-05-20T01:22:18.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.093+0000 7f70438bd7c0 10 datascan.operator(): handling object 1099511628292.1 2023-05-20T01:22:18.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.093+0000 7f70438bd7c0 1 -- 172.21.15.16:0/739319487 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:23 10.5 10:a0beda95:::10000000204.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x55f909e0e380 con 0x55f909ce0c00 2023-05-20T01:22:18.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.093+0000 7f6194987700 1 -- 172.21.15.16:0/3379690421 <== osd.6 v2:172.21.15.132:6801/727581123 10 ==== osd_op_reply(16 [pgnls start_epoch 71 out=49b] v69'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564461dd78c0 con 0x564461f2e400 2023-05-20T01:22:18.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.093+0000 7f618f17c700 1 --2- 172.21.15.16:0/3379690421 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x564461f2ec00 0x564461f5e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.093+0000 7f618f17c700 1 -- 172.21.15.16:0/3379690421 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:17 10.2f 10:f4000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x564462999c00 con 0x564461f2ec00 2023-05-20T01:22:18.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.093+0000 7f6194186700 1 --2- 172.21.15.16:0/3379690421 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x564461f2ec00 0x564461f5e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.093+0000 7fee7ce79700 1 -- 172.21.15.16:0/1398005986 <== osd.3 v2:172.21.15.16:6800/3669781890 12 ==== osd_op_reply(24 1000000020a.00000000 [call,setxattr (8)] v71'14 uv14 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x55e8d1af58c0 con 0x55e8d2659000 2023-05-20T01:22:18.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.093+0000 7fee7eecb7c0 1 -- 172.21.15.16:0/1398005986 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:25 11.a 11:57fe7535::::head [pgnls start_epoch 71 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x55e8d2672a80 con 0x55e8d2659000 2023-05-20T01:22:18.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.094+0000 7f6194186700 1 --2- 172.21.15.16:0/3379690421 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x564461f2ec00 0x564461f5e100 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.093+0000 7f7d9bf47700 1 -- 172.21.15.16:0/1705842756 <== osd.7 v2:172.21.15.132:6800/466001149 8 ==== osd_op_reply(20 100000001f9.00000000 [call,setxattr (8)] v71'20 uv20 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x55687b04f440 con 0x55687b062800 2023-05-20T01:22:18.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.093+0000 7f7d9df997c0 1 -- 172.21.15.16:0/1705842756 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:21 11.16 11:6fe38dc2::::head [pgnls start_epoch 71 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x55687b1a6380 con 0x55687b065400 2023-05-20T01:22:18.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.094+0000 7fee7ce79700 1 -- 172.21.15.16:0/1398005986 <== osd.3 v2:172.21.15.16:6800/3669781890 13 ==== osd_op_reply(25 [pgnls start_epoch 71 out=81b] v69'6 uv6 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55e8d1af58c0 con 0x55e8d2659000 2023-05-20T01:22:18.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.094+0000 7fee7eecb7c0 1 -- 172.21.15.16:0/1398005986 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:26 11.a 11:57fe7535:::100000001f6.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x55e8d2672e00 con 0x55e8d2659000 2023-05-20T01:22:18.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.094+0000 7f5a541fc700 1 -- 172.21.15.16:0/494201187 <== osd.3 v2:172.21.15.16:6800/3669781890 4 ==== osd_op_reply(16 1000000020d.00000000 [call] v71'9 uv9 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x559726fc1680 con 0x559726ff0800 2023-05-20T01:22:18.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.094+0000 7f5a55a4d7c0 1 -- 172.21.15.16:0/494201187 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:17 10.35 10:ac000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x559727b1dc00 con 0x559726fc3000 2023-05-20T01:22:18.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.094+0000 7f6194186700 1 -- 172.21.15.16:0/3379690421 <== osd.1 v2:172.21.15.16:6808/3860264220 1 ==== osd_op_reply(17 [pgnls start_epoch 71 out=49b] v69'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564461dd7b00 con 0x564461f2ec00 2023-05-20T01:22:18.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.094+0000 7f618f97d700 1 -- 172.21.15.16:0/3379690421 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:18 10.1f 10:f8000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x564462998a80 con 0x564461f2e800 2023-05-20T01:22:18.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.094+0000 7f7d9bf47700 1 -- 172.21.15.16:0/1705842756 <== osd.5 v2:172.21.15.132:6816/1844689995 5 ==== osd_op_reply(21 [pgnls start_epoch 71 out=81b] v69'5 uv5 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55687b04f440 con 0x55687b065400 2023-05-20T01:22:18.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.094+0000 7f7d9df997c0 1 -- 172.21.15.16:0/1705842756 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:22 11.16 11:6fe38dc2:::100000001f7.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x55687b1a6700 con 0x55687b065400 2023-05-20T01:22:18.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.094+0000 7fee7ce79700 1 -- 172.21.15.16:0/1398005986 <== osd.3 v2:172.21.15.16:6800/3669781890 14 ==== osd_op_reply(26 100000001f6.00000000 [stat out=16b] v0'0 uv5 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55e8d1af58c0 con 0x55e8d2659000 2023-05-20T01:22:18.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.094+0000 7ffaa0bae700 1 -- 172.21.15.16:0/1771343338 <== osd.3 v2:172.21.15.16:6800/3669781890 5 ==== osd_op_reply(27 100000001f6.00000000 [call,setxattr (8)] v71'15 uv15 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x560889267440 con 0x560889282c00 2023-05-20T01:22:18.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.094+0000 7fee7eecb7c0 10 datascan.operator(): handling object 1099511628278.0 2023-05-20T01:22:18.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.094+0000 7fee7eecb7c0 1 -- 172.21.15.16:0/1398005986 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:27 10.2a 10:57fe7535:::100000001f6.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x55e8d1c3ee00 con 0x55e8d2659000 2023-05-20T01:22:18.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.094+0000 7ffaa34017c0 1 --2- 172.21.15.16:0/1771343338 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x560889283400 0x5608893a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.094+0000 7f5a541fc700 1 -- 172.21.15.16:0/494201187 <== osd.4 v2:172.21.15.132:6817/506317223 2 ==== osd_op_reply(17 [pgnls start_epoch 71 out=49b] v69'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559726fc1680 con 0x559726fc3000 2023-05-20T01:22:18.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.094+0000 7f704106a700 1 -- 172.21.15.16:0/739319487 <== osd.2 v2:172.21.15.16:6816/293187030 3 ==== osd_op_reply(23 10000000204.00000000 [call,setxattr (8)] v71'8 uv8 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x55f909cc38c0 con 0x55f909ce0c00 2023-05-20T01:22:18.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.094+0000 7ffaa34017c0 1 -- 172.21.15.16:0/1771343338 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:28 11.2 11:40000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x56088921f500 con 0x560889283400 2023-05-20T01:22:18.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.094+0000 7f70438bd7c0 1 -- 172.21.15.16:0/739319487 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:24 11.1d 11:bfcd3380::::head [pgnls start_epoch 71 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x55f909e0ea80 con 0x55f909ce7400 2023-05-20T01:22:18.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.095+0000 7f5a4f1f2700 1 --2- 172.21.15.16:0/494201187 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x559726ff1000 0x5597270f6b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.095+0000 7ffaa0bae700 1 --2- 172.21.15.16:0/1771343338 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x560889283400 0x5608893a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.095+0000 7f7d9bf47700 1 -- 172.21.15.16:0/1705842756 <== osd.5 v2:172.21.15.132:6816/1844689995 6 ==== osd_op_reply(22 100000001f7.00000000 [stat out=16b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55687b04f440 con 0x55687b065400 2023-05-20T01:22:18.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.095+0000 7f5a4f1f2700 1 -- 172.21.15.16:0/494201187 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:18 10.d 10:b0000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x559727118000 con 0x559726ff1000 2023-05-20T01:22:18.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.095+0000 7f6194987700 1 -- 172.21.15.16:0/3379690421 <== osd.7 v2:172.21.15.132:6800/466001149 2 ==== osd_op_reply(18 [pgnls start_epoch 71 out=49b] v69'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564461dd78c0 con 0x564461f2e800 2023-05-20T01:22:18.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.095+0000 7f618f17c700 1 -- 172.21.15.16:0/3379690421 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:19 10.3f 10:fc000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x564462999500 con 0x564461f2ec00 2023-05-20T01:22:18.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.095+0000 7f7d9df997c0 10 datascan.operator(): handling object 1099511628279.0 2023-05-20T01:22:18.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.095+0000 7f7d9df997c0 1 -- 172.21.15.16:0/1705842756 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:23 10.36 10:6fe38dc2:::100000001f7.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x55687b1a6a80 con 0x55687b062800 2023-05-20T01:22:18.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.095+0000 7f704186b700 1 -- 172.21.15.16:0/739319487 <== osd.7 v2:172.21.15.132:6800/466001149 9 ==== osd_op_reply(24 [pgnls start_epoch 71 out=81b] v69'4 uv4 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55f909cc3440 con 0x55f909ce7400 2023-05-20T01:22:18.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.095+0000 7f70438bd7c0 1 -- 172.21.15.16:0/739319487 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:25 11.1d 11:bfcd3380:::10000000205.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x55f909e0f880 con 0x55f909ce7400 2023-05-20T01:22:18.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.095+0000 7f5a539fb700 1 --2- 172.21.15.16:0/494201187 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x559726ff1000 0x5597270f6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.095+0000 7ffaa0bae700 1 --2- 172.21.15.16:0/1771343338 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x560889283400 0x5608893a3600 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.095+0000 7f6194186700 1 -- 172.21.15.16:0/3379690421 <== osd.1 v2:172.21.15.16:6808/3860264220 2 ==== osd_op_reply(19 [pgnls start_epoch 71 out=81b] v69'9 uv9 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x564461dd7b00 con 0x564461f2ec00 2023-05-20T01:22:18.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.095+0000 7f704186b700 1 -- 172.21.15.16:0/739319487 <== osd.7 v2:172.21.15.132:6800/466001149 10 ==== osd_op_reply(25 10000000205.00000000 [stat out=16b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55f909cc3440 con 0x55f909ce7400 2023-05-20T01:22:18.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.095+0000 7f70438bd7c0 10 datascan.operator(): handling object 1099511628293.0 2023-05-20T01:22:18.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.095+0000 7f70438bd7c0 1 -- 172.21.15.16:0/739319487 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:26 10.3d 10:bfcd3380:::10000000205.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x55f909e0fc00 con 0x55f909ce7400 2023-05-20T01:22:18.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.095+0000 7f61961d87c0 1 -- 172.21.15.16:0/3379690421 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:20 10.3f 10:feb1c803:::100000001fa.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x564462998e00 con 0x564461f2ec00 2023-05-20T01:22:18.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.095+0000 7f5a539fb700 1 --2- 172.21.15.16:0/494201187 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x559726ff1000 0x5597270f6b00 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.095+0000 7fe1ff3bc700 1 -- 172.21.15.16:0/2808449806 <== osd.3 v2:172.21.15.16:6800/3669781890 4 ==== osd_op_reply(27 1000000020a.00000000 [call,setxattr (8)] v71'16 uv16 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x5624b6d678c0 con 0x5624b6a39800 2023-05-20T01:22:18.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.095+0000 7ffaa0bae700 1 -- 172.21.15.16:0/1771343338 <== osd.2 v2:172.21.15.16:6816/293187030 1 ==== osd_op_reply(28 [pgnls start_epoch 71 out=81b] v69'1 uv1 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x560889267440 con 0x560889283400 2023-05-20T01:22:18.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.095+0000 7ffaa34017c0 1 -- 172.21.15.16:0/1771343338 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:29 11.2 11:40d56c48:::100000001fc.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x56088921f180 con 0x560889283400 2023-05-20T01:22:18.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.096+0000 7fe20140e7c0 1 -- 172.21.15.16:0/2808449806 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:28 11.0 11:05552577::::head [pgnls start_epoch 71 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x5624b6eb9500 con 0x5624b6eae800 2023-05-20T01:22:18.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.096+0000 7f6194186700 1 -- 172.21.15.16:0/3379690421 <== osd.1 v2:172.21.15.16:6808/3860264220 3 ==== osd_op_reply(20 100000001fa.00000000 [stat out=16b] v0'0 uv9 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x564461dd7b00 con 0x564461f2ec00 2023-05-20T01:22:18.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.096+0000 7f61961d87c0 10 datascan.operator(): handling object 1099511628282.0 2023-05-20T01:22:18.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.096+0000 7f61961d87c0 1 -- 172.21.15.16:0/3379690421 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:21 10.3f 10:feb1c803:::100000001fa.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x564462999180 con 0x564461f2ec00 2023-05-20T01:22:18.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.096+0000 7f5a539fb700 1 -- 172.21.15.16:0/494201187 <== osd.6 v2:172.21.15.132:6801/727581123 1 ==== osd_op_reply(18 [pgnls start_epoch 71 out=49b] v69'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559726fc18c0 con 0x559726ff1000 2023-05-20T01:22:18.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.096+0000 7f5a4e9f1700 1 -- 172.21.15.16:0/494201187 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:19 10.2d 10:b4000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x559727118a80 con 0x559726ff1000 2023-05-20T01:22:18.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.096+0000 7fe1febbb700 1 -- 172.21.15.16:0/2808449806 <== osd.1 v2:172.21.15.16:6808/3860264220 6 ==== osd_op_reply(28 [pgnls start_epoch 71 out=81b] v69'5 uv5 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5624b6d67440 con 0x5624b6eae800 2023-05-20T01:22:18.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.096+0000 7ffaa0bae700 1 -- 172.21.15.16:0/1771343338 <== osd.2 v2:172.21.15.16:6816/293187030 2 ==== osd_op_reply(29 100000001fc.00000001 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x560889267440 con 0x560889283400 2023-05-20T01:22:18.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.096+0000 7ffaa34017c0 10 datascan.operator(): handling object 1099511628284.1 2023-05-20T01:22:18.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.096+0000 7fe20140e7c0 1 -- 172.21.15.16:0/2808449806 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:29 11.0 11:05552577:::1000000020b.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x5624b6eb9880 con 0x5624b6eae800 2023-05-20T01:22:18.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.096+0000 7ffaa34017c0 1 -- 172.21.15.16:0/1771343338 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:30 10.c 10:30604137:::100000001fc.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x5608893bd180 con 0x560889291000 2023-05-20T01:22:18.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.096+0000 7f5a539fb700 1 -- 172.21.15.16:0/494201187 <== osd.6 v2:172.21.15.132:6801/727581123 2 ==== osd_op_reply(19 [pgnls start_epoch 71 out=49b] v69'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559726fc18c0 con 0x559726ff1000 2023-05-20T01:22:18.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.096+0000 7f5a55a4d7c0 1 -- 172.21.15.16:0/494201187 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:20 11.9 11:92492492::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x559727b1c700 con 0x559726ff0800 2023-05-20T01:22:18.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.096+0000 7fe1febbb700 1 -- 172.21.15.16:0/2808449806 <== osd.1 v2:172.21.15.16:6808/3860264220 7 ==== osd_op_reply(29 1000000020b.00000000 [stat out=16b] v0'0 uv5 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5624b6d67440 con 0x5624b6eae800 2023-05-20T01:22:18.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.096+0000 7fee7ce79700 1 -- 172.21.15.16:0/1398005986 <== osd.3 v2:172.21.15.16:6800/3669781890 15 ==== osd_op_reply(27 100000001f6.00000000 [call,setxattr (8)] v71'17 uv17 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x55e8d1af58c0 con 0x55e8d2659000 2023-05-20T01:22:18.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.096+0000 7fe20140e7c0 10 datascan.operator(): handling object 1099511628299.0 2023-05-20T01:22:18.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.096+0000 7fe20140e7c0 1 -- 172.21.15.16:0/2808449806 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:30 10.20 10:05552577:::1000000020b.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x5624b78c7500 con 0x5624b6a39c00 2023-05-20T01:22:18.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.096+0000 7fee7eecb7c0 1 -- 172.21.15.16:0/1398005986 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:28 11.1a 11:58000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x55e8d1c3e700 con 0x55e8d2659400 2023-05-20T01:22:18.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.096+0000 7f7d9bf47700 1 -- 172.21.15.16:0/1705842756 <== osd.7 v2:172.21.15.132:6800/466001149 9 ==== osd_op_reply(23 100000001f7.00000000 [call,setxattr (8)] v71'21 uv21 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x55687b04f440 con 0x55687b062800 2023-05-20T01:22:18.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.096+0000 7f7d9df997c0 1 -- 172.21.15.16:0/1705842756 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:24 11.e 11:70000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x55687b1a6e00 con 0x55687b062c00 2023-05-20T01:22:18.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.096+0000 7f5a541fc700 1 -- 172.21.15.16:0/494201187 <== osd.3 v2:172.21.15.16:6800/3669781890 5 ==== osd_op_reply(20 [pgnls start_epoch 71 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559726fc1680 con 0x559726ff0800 2023-05-20T01:22:18.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.096+0000 7f5a4e9f1700 1 --2- 172.21.15.16:0/494201187 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x559726ff1800 0x5597270f7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.096+0000 7f5a4e9f1700 1 -- 172.21.15.16:0/494201187 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:21 11.19 11:98000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x559727118e00 con 0x559726ff1800 2023-05-20T01:22:18.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.097+0000 7f5a531fa700 1 --2- 172.21.15.16:0/494201187 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x559726ff1800 0x5597270f7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.097+0000 7f7d9c748700 1 -- 172.21.15.16:0/1705842756 <== osd.4 v2:172.21.15.132:6817/506317223 4 ==== osd_op_reply(24 [pgnls start_epoch 71 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55687b04f680 con 0x55687b062c00 2023-05-20T01:22:18.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.097+0000 7f7d96f3d700 1 -- 172.21.15.16:0/1705842756 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:25 11.1e 11:78000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x55687bbbce00 con 0x55687b062800 2023-05-20T01:22:18.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.097+0000 7f704186b700 1 -- 172.21.15.16:0/739319487 <== osd.7 v2:172.21.15.132:6800/466001149 11 ==== osd_op_reply(26 10000000205.00000000 [call,setxattr (8)] v71'10 uv10 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x55f909cc3440 con 0x55f909ce7400 2023-05-20T01:22:18.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.097+0000 7f70438bd7c0 1 -- 172.21.15.16:0/739319487 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:27 11.3 11:c0000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x55f90a82ee00 con 0x55f909ce0800 2023-05-20T01:22:18.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.097+0000 7fee7d67a700 1 -- 172.21.15.16:0/1398005986 <== osd.5 v2:172.21.15.132:6816/1844689995 5 ==== osd_op_reply(28 [pgnls start_epoch 71 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e8d1af5680 con 0x55e8d2659400 2023-05-20T01:22:18.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.097+0000 7fee78670700 1 -- 172.21.15.16:0/1398005986 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:29 11.6 11:60000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x55e8d2673880 con 0x55e8d2659800 2023-05-20T01:22:18.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.097+0000 7f5a531fa700 1 --2- 172.21.15.16:0/494201187 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x559726ff1800 0x5597270f7600 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.097+0000 7f7d9bf47700 1 -- 172.21.15.16:0/1705842756 <== osd.7 v2:172.21.15.132:6800/466001149 10 ==== osd_op_reply(25 [pgnls start_epoch 71 out=101b] v69'3 uv3 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55687b04f440 con 0x55687b062800 2023-05-20T01:22:18.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.097+0000 7f704186b700 1 -- 172.21.15.16:0/739319487 <== osd.6 v2:172.21.15.132:6801/727581123 3 ==== osd_op_reply(27 [pgnls start_epoch 71 out=81b] v69'1 uv1 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55f909cc3440 con 0x55f909ce0800 2023-05-20T01:22:18.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.097+0000 7f7d9df997c0 1 -- 172.21.15.16:0/1705842756 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:26 11.1e 11:789b801b:::1000000020c.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x55687b1a7180 con 0x55687b062800 2023-05-20T01:22:18.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.097+0000 7ffaa13af700 1 -- 172.21.15.16:0/1771343338 <== osd.7 v2:172.21.15.132:6800/466001149 7 ==== osd_op_reply(30 100000001fc.00000000 [call,setxattr (8)] v71'10 uv10 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x5608892678c0 con 0x560889291000 2023-05-20T01:22:18.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.097+0000 7ffaa34017c0 1 -- 172.21.15.16:0/1771343338 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:31 11.12 11:48000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x5608893bd500 con 0x560889291000 2023-05-20T01:22:18.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.097+0000 7fee7c678700 1 -- 172.21.15.16:0/1398005986 <== osd.0 v2:172.21.15.16:6824/3923634179 2 ==== osd_op_reply(29 [pgnls start_epoch 71 out=81b] v69'2 uv2 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55e8d1af5440 con 0x55e8d2659800 2023-05-20T01:22:18.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.097+0000 7fee7eecb7c0 1 -- 172.21.15.16:0/1398005986 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:30 11.6 11:658f2455:::100000001ff.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x55e8d1c3e000 con 0x55e8d2659800 2023-05-20T01:22:18.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.097+0000 7f70438bd7c0 1 -- 172.21.15.16:0/739319487 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:28 11.3 11:c1028715:::10000000202.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x55f90a82f180 con 0x55f909ce0800 2023-05-20T01:22:18.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.097+0000 7f5a531fa700 1 -- 172.21.15.16:0/494201187 <== osd.0 v2:172.21.15.16:6824/3923634179 1 ==== osd_op_reply(21 [pgnls start_epoch 71 out=101b] v69'5 uv5 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x559726fc1440 con 0x559726ff1800 2023-05-20T01:22:18.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.098+0000 7f5a55a4d7c0 1 -- 172.21.15.16:0/494201187 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:22 11.19 11:992a6246:::10000000200.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x559727b1ce00 con 0x559726ff1800 2023-05-20T01:22:18.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.098+0000 7f6194186700 1 -- 172.21.15.16:0/3379690421 <== osd.1 v2:172.21.15.16:6808/3860264220 4 ==== osd_op_reply(21 100000001fa.00000000 [call] v71'10 uv10 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564461dd7b00 con 0x564461f2ec00 2023-05-20T01:22:18.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.098+0000 7f61961d87c0 1 -- 172.21.15.16:0/3379690421 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:22 11.1b 11:db6db6db::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x564462999c00 con 0x564461e21400 2023-05-20T01:22:18.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.098+0000 7f7d9bf47700 1 -- 172.21.15.16:0/1705842756 <== osd.7 v2:172.21.15.132:6800/466001149 11 ==== osd_op_reply(26 1000000020c.00000000 [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55687b04f440 con 0x55687b062800 2023-05-20T01:22:18.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.098+0000 7f7d9df997c0 10 datascan.operator(): handling object 1099511628300.0 2023-05-20T01:22:18.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.098+0000 7f7d9df997c0 1 -- 172.21.15.16:0/1705842756 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:27 10.1e 10:789b801b:::1000000020c.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x55687b1a7500 con 0x55687b065000 2023-05-20T01:22:18.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.098+0000 7ffaa13af700 1 -- 172.21.15.16:0/1771343338 <== osd.7 v2:172.21.15.132:6800/466001149 8 ==== osd_op_reply(31 [pgnls start_epoch 71 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5608892678c0 con 0x560889291000 2023-05-20T01:22:18.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.098+0000 7fee7c678700 1 -- 172.21.15.16:0/1398005986 <== osd.0 v2:172.21.15.16:6824/3923634179 3 ==== osd_op_reply(30 100000001ff.00000000 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55e8d1af5440 con 0x55e8d2659800 2023-05-20T01:22:18.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.098+0000 7fee7eecb7c0 10 datascan.operator(): handling object 1099511628287.0 2023-05-20T01:22:18.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.098+0000 7fee7eecb7c0 1 -- 172.21.15.16:0/1398005986 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:31 10.26 10:658f2455:::100000001ff.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x55e8d1c3e380 con 0x55e8d2659400 2023-05-20T01:22:18.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.098+0000 7fe1ffbbd700 1 -- 172.21.15.16:0/2808449806 <== osd.0 v2:172.21.15.16:6824/3923634179 6 ==== osd_op_reply(30 1000000020b.00000000 [call,setxattr (8)] v71'12 uv12 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x5624b6d67680 con 0x5624b6a39c00 2023-05-20T01:22:18.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.098+0000 7fe20140e7c0 1 -- 172.21.15.16:0/2808449806 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:31 11.10 11:08000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x5624b78c7880 con 0x5624b6a39c00 2023-05-20T01:22:18.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.098+0000 7ffaa34017c0 1 -- 172.21.15.16:0/1771343338 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x560889282800 msgr2=0x560888fd6100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.098+0000 7ffaa34017c0 1 --2- 172.21.15.16:0/1771343338 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x560889282800 0x560888fd6100 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.098+0000 7ffaa34017c0 1 -- 172.21.15.16:0/1771343338 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x560889283400 msgr2=0x5608893a3600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.098+0000 7ffaa34017c0 1 --2- 172.21.15.16:0/1771343338 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x560889283400 0x5608893a3600 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.098+0000 7ffaa34017c0 1 -- 172.21.15.16:0/1771343338 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x560889282c00 msgr2=0x560888fd5b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.098+0000 7ffaa34017c0 1 --2- 172.21.15.16:0/1771343338 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x560889282c00 0x560888fd5b80 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.098+0000 7ffaa34017c0 1 -- 172.21.15.16:0/1771343338 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x560889291400 msgr2=0x5608893a2b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.098+0000 7ffaa34017c0 1 --2- 172.21.15.16:0/1771343338 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x560889291400 0x5608893a2b00 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.098+0000 7ffaa13af700 1 -- 172.21.15.16:0/1771343338 reap_dead start 2023-05-20T01:22:18.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.098+0000 7ffaa34017c0 1 -- 172.21.15.16:0/1771343338 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x560889290c00 msgr2=0x5608893a2000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.098+0000 7ffaa34017c0 1 --2- 172.21.15.16:0/1771343338 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x560889290c00 0x5608893a2000 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.098+0000 7ffaa13af700 1 -- 172.21.15.16:0/1771343338 reap_dead start 2023-05-20T01:22:18.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.098+0000 7ffaa34017c0 1 -- 172.21.15.16:0/1771343338 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x560889283000 msgr2=0x560888fd5600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.098+0000 7ffaa34017c0 1 --2- 172.21.15.16:0/1771343338 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x560889283000 0x560888fd5600 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.098+0000 7ffaa34017c0 1 -- 172.21.15.16:0/1771343338 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x560889291000 msgr2=0x5608893a2580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.098+0000 7ffaa34017c0 1 --2- 172.21.15.16:0/1771343338 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x560889291000 0x5608893a2580 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.098+0000 7ffaa34017c0 1 -- 172.21.15.16:0/1771343338 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x560889290800 msgr2=0x560888fd7700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.098+0000 7ffaa34017c0 1 --2- 172.21.15.16:0/1771343338 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x560889290800 0x560888fd7700 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x56088927b320 tx=0x5608893881e0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.098+0000 7ffaa34017c0 1 -- 172.21.15.16:0/1771343338 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560889282000 msgr2=0x560888fd6c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.098+0000 7ffaa34017c0 1 --2- 172.21.15.16:0/1771343338 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560889282000 0x560888fd6c00 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x56088909a060 tx=0x560889cd3440 comp rx=0 tx=0).stop 2023-05-20T01:22:18.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.098+0000 7f5a531fa700 1 -- 172.21.15.16:0/494201187 <== osd.0 v2:172.21.15.16:6824/3923634179 2 ==== osd_op_reply(22 10000000200.00000000 [stat out=16b] v0'0 uv5 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x559726fc1440 con 0x559726ff1800 2023-05-20T01:22:18.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.098+0000 7f5a55a4d7c0 10 datascan.operator(): handling object 1099511628288.0 2023-05-20T01:22:18.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.098+0000 7f5a55a4d7c0 1 -- 172.21.15.16:0/494201187 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:23 10.19 10:992a6246:::10000000200.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x559727b1d500 con 0x559726ff0c00 2023-05-20T01:22:18.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.098+0000 7f704186b700 1 -- 172.21.15.16:0/739319487 <== osd.6 v2:172.21.15.132:6801/727581123 4 ==== osd_op_reply(28 10000000202.00000001 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55f909cc3440 con 0x55f909ce0800 2023-05-20T01:22:18.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.098+0000 7f70438bd7c0 10 datascan.operator(): handling object 1099511628290.1 2023-05-20T01:22:18.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.098+0000 7f70438bd7c0 1 -- 172.21.15.16:0/739319487 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:29 10.28 10:15246602:::10000000202.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x55f909e0f180 con 0x55f909ce7400 2023-05-20T01:22:18.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.098+0000 7f6193985700 1 -- 172.21.15.16:0/3379690421 <== osd.4 v2:172.21.15.132:6817/506317223 2 ==== osd_op_reply(22 [pgnls start_epoch 71 out=81b] v69'2 uv2 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x564461dd7680 con 0x564461e21400 2023-05-20T01:22:18.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.098+0000 7f61961d87c0 1 -- 172.21.15.16:0/3379690421 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:23 11.1b 11:deccbd37:::100000001fe.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x564462999500 con 0x564461e21400 2023-05-20T01:22:18.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.099+0000 7ffaa34017c0 1 -- 172.21.15.16:0/1771343338 shutdown_connections 2023-05-20T01:22:18.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.099+0000 7ffaa34017c0 1 --2- 172.21.15.16:0/1771343338 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x560889283000 0x560888fd5600 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.099+0000 7ffaa34017c0 1 --2- 172.21.15.16:0/1771343338 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560889282000 0x560888fd6c00 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.099+0000 7ffaa34017c0 1 --2- 172.21.15.16:0/1771343338 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x560889291000 0x5608893a2580 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.099+0000 7ffaa34017c0 1 --2- 172.21.15.16:0/1771343338 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x560889290800 0x560888fd7700 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.099+0000 7ffaa34017c0 1 -- 172.21.15.16:0/1771343338 >> 172.21.15.16:0/1771343338 conn(0x560888f3bc00 msgr2=0x560889017800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:18.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.098+0000 7f6193985700 1 -- 172.21.15.16:0/3379690421 <== osd.4 v2:172.21.15.132:6817/506317223 3 ==== osd_op_reply(23 100000001fe.00000000 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x564461dd7680 con 0x564461e21400 2023-05-20T01:22:18.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.098+0000 7f61961d87c0 10 datascan.operator(): handling object 1099511628286.0 2023-05-20T01:22:18.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.099+0000 7f61961d87c0 1 -- 172.21.15.16:0/3379690421 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:24 10.3b 10:deccbd37:::100000001fe.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x564462998e00 con 0x564461f2e400 2023-05-20T01:22:18.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.099+0000 7fe1ffbbd700 1 -- 172.21.15.16:0/2808449806 <== osd.0 v2:172.21.15.16:6824/3923634179 7 ==== osd_op_reply(31 [pgnls start_epoch 71 out=81b] v69'2 uv2 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5624b6d67680 con 0x5624b6a39c00 2023-05-20T01:22:18.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.099+0000 7ffaa34017c0 1 -- 172.21.15.16:0/1771343338 shutdown_connections 2023-05-20T01:22:18.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.099+0000 7ffaa34017c0 1 -- 172.21.15.16:0/1771343338 wait complete. 2023-05-20T01:22:18.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.099+0000 7fe20140e7c0 1 -- 172.21.15.16:0/2808449806 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:32 11.10 11:08b99fea:::100000001fd.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x5624b78c7c00 con 0x5624b6a39c00 2023-05-20T01:22:18.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.099+0000 7fe1ffbbd700 1 -- 172.21.15.16:0/2808449806 <== osd.0 v2:172.21.15.16:6824/3923634179 8 ==== osd_op_reply(32 100000001fd.00000000 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5624b6d67680 con 0x5624b6a39c00 2023-05-20T01:22:18.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.099+0000 7fe20140e7c0 10 datascan.operator(): handling object 1099511628285.0 2023-05-20T01:22:18.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.099+0000 7ffaa34017c0 1 -- 172.21.15.16:0/590433309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560888f3a800 msgr2=0x560888fd5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.099+0000 7ffaa34017c0 1 --2- 172.21.15.16:0/590433309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560888f3a800 0x560888fd5080 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x56088909adb0 tx=0x5608890a4ea0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.099+0000 7fe20140e7c0 1 -- 172.21.15.16:0/2808449806 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:33 10.10 10:08b99fea:::100000001fd.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x5624b6eb8700 con 0x5624b6a39800 2023-05-20T01:22:18.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.099+0000 7fee7d67a700 1 -- 172.21.15.16:0/1398005986 <== osd.5 v2:172.21.15.132:6816/1844689995 6 ==== osd_op_reply(31 100000001ff.00000000 [call,setxattr (8)] v71'18 uv18 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x55e8d1af5680 con 0x55e8d2659400 2023-05-20T01:22:18.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.099+0000 7f5a539fb700 1 -- 172.21.15.16:0/494201187 <== osd.1 v2:172.21.15.16:6808/3860264220 7 ==== osd_op_reply(23 10000000200.00000000 [call,setxattr (8)] v71'15 uv15 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x559726fc18c0 con 0x559726ff0c00 2023-05-20T01:22:18.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.099+0000 7f5a55a4d7c0 1 -- 172.21.15.16:0/494201187 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:24 11.19 11:997b7129::::head [pgnls start_epoch 71 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x559727118000 con 0x559726ff1800 2023-05-20T01:22:18.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.100+0000 7f7d9b746700 1 -- 172.21.15.16:0/1705842756 <== osd.0 v2:172.21.15.16:6824/3923634179 4 ==== osd_op_reply(27 1000000020c.00000000 [call,setxattr (8)] v71'8 uv8 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x55687b04f8c0 con 0x55687b065000 2023-05-20T01:22:18.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.100+0000 7f7d9df997c0 1 -- 172.21.15.16:0/1705842756 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:28 11.1e 11:7ac44a97::::head [pgnls start_epoch 71 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x55687b1a6e00 con 0x55687b062800 2023-05-20T01:22:18.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.100+0000 7f704186b700 1 -- 172.21.15.16:0/739319487 <== osd.7 v2:172.21.15.132:6800/466001149 12 ==== osd_op_reply(29 10000000202.00000000 [call,setxattr (8)] v71'11 uv11 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x55f909cc3440 con 0x55f909ce7400 2023-05-20T01:22:18.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.100+0000 7f70438bd7c0 1 -- 172.21.15.16:0/739319487 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:30 11.13 11:c8000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x55f909e0e000 con 0x55f909ce1000 2023-05-20T01:22:18.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.100+0000 7ffaa34017c0 1 -- 172.21.15.16:0/590433309 shutdown_connections 2023-05-20T01:22:18.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.100+0000 7ffaa34017c0 1 --2- 172.21.15.16:0/590433309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560888f3a800 0x560888fd5080 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.100+0000 7ffaa34017c0 1 --2- 172.21.15.16:0/590433309 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560888f3b000 0x560888fd4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.100+0000 7ffaa34017c0 1 --2- 172.21.15.16:0/590433309 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x560888f3ac00 0x560888fd4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.100+0000 7ffaa34017c0 1 -- 172.21.15.16:0/590433309 >> 172.21.15.16:0/590433309 conn(0x560888f3b400 msgr2=0x560889017000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:18.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.100+0000 7f5a531fa700 1 -- 172.21.15.16:0/494201187 <== osd.0 v2:172.21.15.16:6824/3923634179 3 ==== osd_op_reply(24 [pgnls start_epoch 71 out=81b] v69'5 uv5 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x559726fc1440 con 0x559726ff1800 2023-05-20T01:22:18.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.100+0000 7f7d9bf47700 1 -- 172.21.15.16:0/1705842756 <== osd.7 v2:172.21.15.132:6800/466001149 12 ==== osd_op_reply(28 [pgnls start_epoch 71 out=81b] v69'3 uv3 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55687b04f440 con 0x55687b062800 2023-05-20T01:22:18.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.100+0000 7ffaa34017c0 1 -- 172.21.15.16:0/590433309 shutdown_connections 2023-05-20T01:22:18.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.100+0000 7ffaa34017c0 1 -- 172.21.15.16:0/590433309 wait complete. 2023-05-20T01:22:18.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.100+0000 7f7d9df997c0 1 -- 172.21.15.16:0/1705842756 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:29 11.1e 11:7ac44a97:::10000000207.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x55687b1a6000 con 0x55687b062800 2023-05-20T01:22:18.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.100+0000 7f5a55a4d7c0 1 -- 172.21.15.16:0/494201187 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:25 11.19 11:997b7129:::100000001f8.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x559727b1d880 con 0x559726ff1800 2023-05-20T01:22:18.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.099+0000 7fee7eecb7c0 1 -- 172.21.15.16:0/1398005986 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:32 11.16 11:68000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x55e8d1c3f180 con 0x55e8d2659400 2023-05-20T01:22:18.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.100+0000 7fee7d67a700 1 -- 172.21.15.16:0/1398005986 <== osd.5 v2:172.21.15.132:6816/1844689995 7 ==== osd_op_reply(32 [pgnls start_epoch 71 out=101b] v69'5 uv5 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55e8d1af5680 con 0x55e8d2659400 2023-05-20T01:22:18.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.100+0000 7fee7eecb7c0 1 -- 172.21.15.16:0/1398005986 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:33 11.16 11:6bb58479:::10000000206.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x55e8d1c3f500 con 0x55e8d2659400 2023-05-20T01:22:18.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.100+0000 7f704186b700 1 -- 172.21.15.16:0/739319487 <== osd.5 v2:172.21.15.132:6816/1844689995 5 ==== osd_op_reply(30 [pgnls start_epoch 71 out=101b] v69'4 uv4 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55f909cc3440 con 0x55f909ce1000 2023-05-20T01:22:18.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.100+0000 7f70438bd7c0 1 -- 172.21.15.16:0/739319487 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:31 11.13 11:ca8fdbba:::100000001fb.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x55f909e0ea80 con 0x55f909ce1000 2023-05-20T01:22:18.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.100+0000 7f5a531fa700 1 -- 172.21.15.16:0/494201187 <== osd.0 v2:172.21.15.16:6824/3923634179 4 ==== osd_op_reply(25 100000001f8.00000000 [stat out=16b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x559726fc1440 con 0x559726ff1800 2023-05-20T01:22:18.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.100+0000 7f5a55a4d7c0 10 datascan.operator(): handling object 1099511628280.0 2023-05-20T01:22:18.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.100+0000 7fee7d67a700 1 -- 172.21.15.16:0/1398005986 <== osd.5 v2:172.21.15.132:6816/1844689995 8 ==== osd_op_reply(33 10000000206.00000001 [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55e8d1af5680 con 0x55e8d2659400 2023-05-20T01:22:18.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.100+0000 7f5a55a4d7c0 1 -- 172.21.15.16:0/494201187 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:26 10.19 10:997b7129:::100000001f8.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x559727b1d500 con 0x559726ff0c00 2023-05-20T01:22:18.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.100+0000 7f7d9bf47700 1 -- 172.21.15.16:0/1705842756 <== osd.7 v2:172.21.15.132:6800/466001149 13 ==== osd_op_reply(29 10000000207.00000001 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55687b04f440 con 0x55687b062800 2023-05-20T01:22:18.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.100+0000 7f7d9df997c0 10 datascan.operator(): handling object 1099511628295.1 2023-05-20T01:22:18.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.100+0000 7fee7eecb7c0 10 datascan.operator(): handling object 1099511628294.1 2023-05-20T01:22:18.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.100+0000 7fee7eecb7c0 1 -- 172.21.15.16:0/1398005986 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:34 10.0 10:012cdbc7:::10000000206.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x55e8d2672700 con 0x55e8d2656800 2023-05-20T01:22:18.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.100+0000 7f7d9df997c0 1 --2- 172.21.15.16:0/1705842756 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x55687b063400 0x55687b189600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.100+0000 7f7d9df997c0 1 -- 172.21.15.16:0/1705842756 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:30 10.7 10:e15a914f:::10000000207.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x55687b1a6380 con 0x55687b063400 2023-05-20T01:22:18.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.100+0000 7f704186b700 1 -- 172.21.15.16:0/739319487 <== osd.5 v2:172.21.15.132:6816/1844689995 6 ==== osd_op_reply(31 100000001fb.00000000 [stat out=16b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55f909cc3440 con 0x55f909ce1000 2023-05-20T01:22:18.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.101+0000 7f70438bd7c0 10 datascan.operator(): handling object 1099511628283.0 2023-05-20T01:22:18.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.101+0000 7f70438bd7c0 1 -- 172.21.15.16:0/739319487 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:32 10.13 10:ca8fdbba:::100000001fb.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x55f909e0f880 con 0x55f909ce7800 2023-05-20T01:22:18.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.101+0000 7f7d9b746700 1 --2- 172.21.15.16:0/1705842756 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x55687b063400 0x55687b189600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.101+0000 7f7d9b746700 1 --2- 172.21.15.16:0/1705842756 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x55687b063400 0x55687b189600 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.102+0000 7fee7c678700 1 -- 172.21.15.16:0/1398005986 <== osd.6 v2:172.21.15.132:6801/727581123 5 ==== osd_op_reply(34 10000000206.00000000 [call,setxattr (8)] v71'12 uv12 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x55e8d1af5440 con 0x55e8d2656800 2023-05-20T01:22:18.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.102+0000 7f6194987700 1 -- 172.21.15.16:0/3379690421 <== osd.6 v2:172.21.15.132:6801/727581123 11 ==== osd_op_reply(24 100000001fe.00000000 [call,setxattr (8)] v71'7 uv7 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x564461dd78c0 con 0x564461f2e400 2023-05-20T01:22:18.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.102+0000 7fee7eecb7c0 1 -- 172.21.15.16:0/1398005986 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x55e8d2659800 msgr2=0x55e8d1c27080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.102+0000 7fee7eecb7c0 1 --2- 172.21.15.16:0/1398005986 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x55e8d2659800 0x55e8d1c27080 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.102+0000 7fee7eecb7c0 1 -- 172.21.15.16:0/1398005986 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x55e8d2659000 msgr2=0x55e8d1c26580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.102+0000 7fee7eecb7c0 1 --2- 172.21.15.16:0/1398005986 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x55e8d2659000 0x55e8d1c26580 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.102+0000 7f61961d87c0 1 -- 172.21.15.16:0/3379690421 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:25 11.7 11:e0000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x564462999180 con 0x564461e21000 2023-05-20T01:22:18.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.102+0000 7fee7eecb7c0 1 -- 172.21.15.16:0/1398005986 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x55e8d2659400 msgr2=0x55e8d1c26b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.102+0000 7fee7eecb7c0 1 --2- 172.21.15.16:0/1398005986 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x55e8d2659400 0x55e8d1c26b00 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.102+0000 7fee7eecb7c0 1 -- 172.21.15.16:0/1398005986 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x55e8d2656800 msgr2=0x55e8d1872100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.102+0000 7fee7eecb7c0 1 --2- 172.21.15.16:0/1398005986 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x55e8d2656800 0x55e8d1872100 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.102+0000 7fee7c678700 1 -- 172.21.15.16:0/1398005986 reap_dead start 2023-05-20T01:22:18.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.102+0000 7fee7eecb7c0 1 -- 172.21.15.16:0/1398005986 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x55e8d2658c00 msgr2=0x55e8d1c26000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.102+0000 7fee7eecb7c0 1 --2- 172.21.15.16:0/1398005986 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x55e8d2658c00 0x55e8d1c26000 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.102+0000 7fee7c678700 1 -- 172.21.15.16:0/1398005986 reap_dead start 2023-05-20T01:22:18.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.102+0000 7fee7eecb7c0 1 -- 172.21.15.16:0/1398005986 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x55e8d2658800 msgr2=0x55e8d1873700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.102+0000 7fee7eecb7c0 1 --2- 172.21.15.16:0/1398005986 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x55e8d2658800 0x55e8d1873700 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x55e8d1936c90 tx=0x55e8d1c1a000 comp rx=0 tx=0).stop 2023-05-20T01:22:18.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.102+0000 7fee7eecb7c0 1 -- 172.21.15.16:0/1398005986 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8d2656000 msgr2=0x55e8d1872c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.102+0000 7fee7eecb7c0 1 --2- 172.21.15.16:0/1398005986 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8d2656000 0x55e8d1872c00 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x55e8d1b09320 tx=0x55e8d256f2c0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.102+0000 7fee7eecb7c0 1 -- 172.21.15.16:0/1398005986 shutdown_connections 2023-05-20T01:22:18.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.102+0000 7fee7eecb7c0 1 --2- 172.21.15.16:0/1398005986 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8d2656000 0x55e8d1872c00 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.102+0000 7fee7eecb7c0 1 --2- 172.21.15.16:0/1398005986 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x55e8d2658800 0x55e8d1873700 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.102+0000 7fee7eecb7c0 1 -- 172.21.15.16:0/1398005986 >> 172.21.15.16:0/1398005986 conn(0x55e8d17d7c00 msgr2=0x55e8d18b3800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:18.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.102+0000 7f5a539fb700 1 -- 172.21.15.16:0/494201187 <== osd.1 v2:172.21.15.16:6808/3860264220 8 ==== osd_op_reply(26 100000001f8.00000000 [call,setxattr (8)] v71'16 uv16 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x559726fc18c0 con 0x559726ff0c00 2023-05-20T01:22:18.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.102+0000 7f6194186700 1 -- 172.21.15.16:0/3379690421 <== osd.5 v2:172.21.15.132:6816/1844689995 4 ==== osd_op_reply(25 [pgnls start_epoch 71 out=101b] v69'7 uv7 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x564461dd7b00 con 0x564461e21000 2023-05-20T01:22:18.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.102+0000 7f61961d87c0 1 -- 172.21.15.16:0/3379690421 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:26 11.7 11:e15a914f:::10000000207.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x564461f7c000 con 0x564461e21000 2023-05-20T01:22:18.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.103+0000 7fee7eecb7c0 1 -- 172.21.15.16:0/1398005986 shutdown_connections 2023-05-20T01:22:18.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.102+0000 7f5a55a4d7c0 1 --2- 172.21.15.16:0/494201187 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x559726fc3400 0x5597270f8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.102+0000 7f5a55a4d7c0 1 -- 172.21.15.16:0/494201187 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:27 11.5 11:a0000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x559727b1d180 con 0x559726fc3400 2023-05-20T01:22:18.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.103+0000 7fee7eecb7c0 1 -- 172.21.15.16:0/1398005986 wait complete. 2023-05-20T01:22:18.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.103+0000 7f704206c700 1 -- 172.21.15.16:0/739319487 <== osd.1 v2:172.21.15.16:6808/3860264220 4 ==== osd_op_reply(32 100000001fb.00000000 [call,setxattr (8)] v71'19 uv19 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x55f909cc3680 con 0x55f909ce7800 2023-05-20T01:22:18.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.103+0000 7f70438bd7c0 1 -- 172.21.15.16:0/739319487 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:33 11.13 11:cb24bee2::::head [pgnls start_epoch 71 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x55f909e0fc00 con 0x55f909ce1000 2023-05-20T01:22:18.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.103+0000 7f6194186700 1 -- 172.21.15.16:0/3379690421 <== osd.5 v2:172.21.15.132:6816/1844689995 5 ==== osd_op_reply(26 10000000207.00000000 [stat out=16b] v0'0 uv6 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x564461dd7b00 con 0x564461e21000 2023-05-20T01:22:18.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.103+0000 7f61961d87c0 10 datascan.operator(): handling object 1099511628295.0 2023-05-20T01:22:18.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.103+0000 7f61961d87c0 1 -- 172.21.15.16:0/3379690421 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:27 10.7 10:e15a914f:::10000000207.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x564461f7c380 con 0x564461f2e400 2023-05-20T01:22:18.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.103+0000 7f7d9b746700 1 -- 172.21.15.16:0/1705842756 <== osd.6 v2:172.21.15.132:6801/727581123 1 ==== osd_op_reply(30 10000000207.00000000 [call,setxattr (8)] v71'10 uv10 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x55687b04f8c0 con 0x55687b063400 2023-05-20T01:22:18.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.103+0000 7f5a541fc700 1 --2- 172.21.15.16:0/494201187 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x559726fc3400 0x5597270f8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.103+0000 7f7d9df997c0 1 -- 172.21.15.16:0/1705842756 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:31 11.1 11:80000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x55687bbbd500 con 0x55687b063400 2023-05-20T01:22:18.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.103+0000 7fee7eecb7c0 1 -- 172.21.15.16:0/709612565 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e8d17d7000 msgr2=0x55e8d1870580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.103+0000 7fee7eecb7c0 1 --2- 172.21.15.16:0/709612565 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e8d17d7000 0x55e8d1870580 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x55e8d1936cf0 tx=0x55e8d256f3e0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.399 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_extents', '--worker_n', '1', '--worker_m', '7'] in time 0:00:00.449235, result: 2023-05-20T01:22:18.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.102+0000 7fe1ff3bc700 1 -- 172.21.15.16:0/2808449806 <== osd.3 v2:172.21.15.16:6800/3669781890 5 ==== osd_op_reply(33 100000001fd.00000000 [call,setxattr (8)] v71'10 uv10 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x5624b6d678c0 con 0x5624b6a39800 2023-05-20T01:22:18.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.102+0000 7fe20140e7c0 1 -- 172.21.15.16:0/2808449806 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:34 11.8 11:10000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x5624b6eb8380 con 0x5624b6a39800 2023-05-20T01:22:18.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.103+0000 7fe1ff3bc700 1 -- 172.21.15.16:0/2808449806 <== osd.3 v2:172.21.15.16:6800/3669781890 6 ==== osd_op_reply(34 [pgnls start_epoch 71 out=101b] v69'6 uv6 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x5624b6d678c0 con 0x5624b6a39800 2023-05-20T01:22:18.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.103+0000 7fe20140e7c0 1 -- 172.21.15.16:0/2808449806 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:35 11.8 11:101c2845:::100000001fe.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x5624b78c6700 con 0x5624b6a39800 2023-05-20T01:22:18.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.103+0000 7fee7eecb7c0 1 -- 172.21.15.16:0/709612565 shutdown_connections 2023-05-20T01:22:18.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.103+0000 7fee7eecb7c0 1 --2- 172.21.15.16:0/709612565 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e8d17d6800 0x55e8d1871080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.103+0000 7fee7eecb7c0 1 --2- 172.21.15.16:0/709612565 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e8d17d7000 0x55e8d1870580 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.103+0000 7fee7eecb7c0 1 --2- 172.21.15.16:0/709612565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8d17d6c00 0x55e8d1870b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.103+0000 7fee7eecb7c0 1 -- 172.21.15.16:0/709612565 >> 172.21.15.16:0/709612565 conn(0x55e8d17d7400 msgr2=0x55e8d18b3000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:18.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.103+0000 7f704186b700 1 -- 172.21.15.16:0/739319487 <== osd.5 v2:172.21.15.132:6816/1844689995 7 ==== osd_op_reply(33 [pgnls start_epoch 71 out=101b] v69'4 uv4 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55f909cc3440 con 0x55f909ce1000 2023-05-20T01:22:18.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.103+0000 7f70438bd7c0 1 -- 172.21.15.16:0/739319487 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:34 11.13 11:cb24bee2:::100000001fa.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x55f909e0e700 con 0x55f909ce1000 2023-05-20T01:22:18.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.103+0000 7fee7eecb7c0 1 -- 172.21.15.16:0/709612565 shutdown_connections 2023-05-20T01:22:18.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.103+0000 7f7d9b746700 1 -- 172.21.15.16:0/1705842756 <== osd.6 v2:172.21.15.132:6801/727581123 2 ==== osd_op_reply(31 [pgnls start_epoch 71 out=101b] v69'2 uv2 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55687b04f8c0 con 0x55687b063400 2023-05-20T01:22:18.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.103+0000 7fe1ff3bc700 1 -- 172.21.15.16:0/2808449806 <== osd.3 v2:172.21.15.16:6800/3669781890 7 ==== osd_op_reply(35 100000001fe.00000001 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5624b6d678c0 con 0x5624b6a39800 2023-05-20T01:22:18.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.103+0000 7fe20140e7c0 10 datascan.operator(): handling object 1099511628286.1 2023-05-20T01:22:18.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.103+0000 7f7d9df997c0 1 -- 172.21.15.16:0/1705842756 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:32 11.1 11:8410d73e:::10000000201.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x55687bbbc700 con 0x55687b063400 2023-05-20T01:22:18.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.103+0000 7fe20140e7c0 1 -- 172.21.15.16:0/2808449806 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:36 10.3b 10:deccbd37:::100000001fe.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x5624b78c7500 con 0x5624b78d2400 2023-05-20T01:22:18.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.103+0000 7fee7eecb7c0 1 -- 172.21.15.16:0/709612565 wait complete. 2023-05-20T01:22:18.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.103+0000 7f5a541fc700 1 --2- 172.21.15.16:0/494201187 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x559726fc3400 0x5597270f8100 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.104+0000 7f704186b700 1 -- 172.21.15.16:0/739319487 <== osd.5 v2:172.21.15.132:6816/1844689995 8 ==== osd_op_reply(34 100000001fa.00000001 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55f909cc3440 con 0x55f909ce1000 2023-05-20T01:22:18.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.104+0000 7f70438bd7c0 10 datascan.operator(): handling object 1099511628282.1 2023-05-20T01:22:18.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.104+0000 7f7d9b746700 1 -- 172.21.15.16:0/1705842756 <== osd.6 v2:172.21.15.132:6801/727581123 3 ==== osd_op_reply(32 10000000201.00000001 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55687b04f8c0 con 0x55687b063400 2023-05-20T01:22:18.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.104+0000 7f7d9df997c0 10 datascan.operator(): handling object 1099511628289.1 2023-05-20T01:22:18.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.104+0000 7f70438bd7c0 1 -- 172.21.15.16:0/739319487 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:35 10.3f 10:feb1c803:::100000001fa.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x55f909e0ee00 con 0x55f909ce7800 2023-05-20T01:22:18.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.104+0000 7f7d9df997c0 1 -- 172.21.15.16:0/1705842756 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:33 10.28 10:177a762d:::10000000201.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x55687bbbd180 con 0x55687b062800 2023-05-20T01:22:18.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.104+0000 7f5a541fc700 1 -- 172.21.15.16:0/494201187 <== osd.7 v2:172.21.15.132:6800/466001149 1 ==== osd_op_reply(27 [pgnls start_epoch 71 out=81b] v69'2 uv2 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x559726fc1680 con 0x559726fc3400 2023-05-20T01:22:18.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.104+0000 7f5a55a4d7c0 1 -- 172.21.15.16:0/494201187 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:28 11.5 11:a0beda95:::10000000204.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x559727b1ca80 con 0x559726fc3400 2023-05-20T01:22:18.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.104+0000 7f5a541fc700 1 -- 172.21.15.16:0/494201187 <== osd.7 v2:172.21.15.132:6800/466001149 2 ==== osd_op_reply(28 10000000204.00000000 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x559726fc1680 con 0x559726fc3400 2023-05-20T01:22:18.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.104+0000 7f5a55a4d7c0 10 datascan.operator(): handling object 1099511628292.0 2023-05-20T01:22:18.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.104+0000 7f5a55a4d7c0 1 -- 172.21.15.16:0/494201187 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:29 10.5 10:a0beda95:::10000000204.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x559727b1dc00 con 0x559726fc2c00 2023-05-20T01:22:18.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.105+0000 7f6194987700 1 -- 172.21.15.16:0/3379690421 <== osd.6 v2:172.21.15.132:6801/727581123 12 ==== osd_op_reply(27 10000000207.00000000 [call,setxattr (8)] v71'11 uv11 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x564461dd78c0 con 0x564461f2e400 2023-05-20T01:22:18.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.105+0000 7f61961d87c0 1 -- 172.21.15.16:0/3379690421 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:28 11.7 11:e2e07930::::head [pgnls start_epoch 71 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x564461f7c700 con 0x564461e21000 2023-05-20T01:22:18.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.105+0000 7fe1febbb700 1 -- 172.21.15.16:0/2808449806 <== osd.6 v2:172.21.15.132:6801/727581123 5 ==== osd_op_reply(36 100000001fe.00000000 [call,setxattr (8)] v71'8 uv8 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x5624b6d67440 con 0x5624b78d2400 2023-05-20T01:22:18.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.105+0000 7fe20140e7c0 1 -- 172.21.15.16:0/2808449806 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:37 11.8 11:1247e94b::::head [pgnls start_epoch 71 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x5624b78c7880 con 0x5624b6a39800 2023-05-20T01:22:18.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.105+0000 7f704206c700 1 -- 172.21.15.16:0/739319487 <== osd.1 v2:172.21.15.16:6808/3860264220 5 ==== osd_op_reply(35 100000001fa.00000000 [call,setxattr (8)] v71'11 uv11 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x55f909cc3680 con 0x55f909ce7800 2023-05-20T01:22:18.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.105+0000 7f70438bd7c0 1 -- 172.21.15.16:0/739319487 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:36 11.13 11:cc6bf2c4::::head [pgnls start_epoch 71 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x55f90a82ee00 con 0x55f909ce1000 2023-05-20T01:22:18.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.105+0000 7f6194186700 1 -- 172.21.15.16:0/3379690421 <== osd.5 v2:172.21.15.132:6816/1844689995 6 ==== osd_op_reply(28 [pgnls start_epoch 71 out=101b] v69'7 uv7 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x564461dd7b00 con 0x564461e21000 2023-05-20T01:22:18.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.105+0000 7f61961d87c0 1 -- 172.21.15.16:0/3379690421 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:29 11.7 11:e2e07930:::10000000001.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x564461f7ca80 con 0x564461e21000 2023-05-20T01:22:18.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.105+0000 7fe1ff3bc700 1 -- 172.21.15.16:0/2808449806 <== osd.3 v2:172.21.15.16:6800/3669781890 8 ==== osd_op_reply(37 [pgnls start_epoch 71 out=101b] v69'6 uv6 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x5624b6d678c0 con 0x5624b6a39800 2023-05-20T01:22:18.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.105+0000 7fe20140e7c0 1 -- 172.21.15.16:0/2808449806 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:38 11.8 11:1247e94b:::100000001f9.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x5624b78c7c00 con 0x5624b6a39800 2023-05-20T01:22:18.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.105+0000 7f704186b700 1 -- 172.21.15.16:0/739319487 <== osd.5 v2:172.21.15.132:6816/1844689995 9 ==== osd_op_reply(36 [pgnls start_epoch 71 out=81b] v69'4 uv4 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55f909cc3440 con 0x55f909ce1000 2023-05-20T01:22:18.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.106+0000 7f70438bd7c0 1 -- 172.21.15.16:0/739319487 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:37 11.13 11:cc6bf2c4:::1000000020b.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x55f909e0f180 con 0x55f909ce1000 2023-05-20T01:22:18.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.106+0000 7f7d9bf47700 1 -- 172.21.15.16:0/1705842756 <== osd.7 v2:172.21.15.132:6800/466001149 14 ==== osd_op_reply(33 10000000201.00000000 [call,setxattr (8)] v71'12 uv12 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x55687b04f440 con 0x55687b062800 2023-05-20T01:22:18.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.106+0000 7f7d9df997c0 1 -- 172.21.15.16:0/1705842756 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:34 11.1 11:848ebb0b::::head [pgnls start_epoch 71 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x55687bbbca80 con 0x55687b063400 2023-05-20T01:22:18.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.106+0000 7f6194186700 1 -- 172.21.15.16:0/3379690421 <== osd.5 v2:172.21.15.132:6816/1844689995 7 ==== osd_op_reply(29 10000000001.00000000 [stat out=16b] v0'0 uv5 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x564461dd7b00 con 0x564461e21000 2023-05-20T01:22:18.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.106+0000 7f61961d87c0 10 datascan.operator(): handling object 1099511627777.0 2023-05-20T01:22:18.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.106+0000 7f61961d87c0 1 -- 172.21.15.16:0/3379690421 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:30 10.7 10:e2e07930:::10000000001.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x564461f7ce00 con 0x564461f2e400 2023-05-20T01:22:18.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.106+0000 7fe1ff3bc700 1 -- 172.21.15.16:0/2808449806 <== osd.3 v2:172.21.15.16:6800/3669781890 9 ==== osd_op_reply(38 100000001f9.00000001 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5624b6d678c0 con 0x5624b6a39800 2023-05-20T01:22:18.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.106+0000 7fe20140e7c0 10 datascan.operator(): handling object 1099511628281.1 2023-05-20T01:22:18.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.106+0000 7fe20140e7c0 1 -- 172.21.15.16:0/2808449806 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:39 10.36 10:6ee5e2c4:::100000001f9.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x5624b78c6e00 con 0x5624b6eae000 2023-05-20T01:22:18.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.106+0000 7f704186b700 1 -- 172.21.15.16:0/739319487 <== osd.5 v2:172.21.15.132:6816/1844689995 10 ==== osd_op_reply(37 1000000020b.00000001 [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55f909cc3440 con 0x55f909ce1000 2023-05-20T01:22:18.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.106+0000 7f70438bd7c0 10 datascan.operator(): handling object 1099511628299.1 2023-05-20T01:22:18.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.106+0000 7f7d9b746700 1 -- 172.21.15.16:0/1705842756 <== osd.6 v2:172.21.15.132:6801/727581123 4 ==== osd_op_reply(34 [pgnls start_epoch 71 out=81b] v69'2 uv2 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55687b04f8c0 con 0x55687b063400 2023-05-20T01:22:18.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.106+0000 7f7d9df997c0 1 -- 172.21.15.16:0/1705842756 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:35 11.1 11:848ebb0b:::100000001f7.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x55687bbbd880 con 0x55687b063400 2023-05-20T01:22:18.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.106+0000 7f70438bd7c0 1 --2- 172.21.15.16:0/739319487 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x55f909ce1800 0x55f909df3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.106+0000 7f70438bd7c0 1 -- 172.21.15.16:0/739319487 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:38 10.20 10:05552577:::1000000020b.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x55f909e0e000 con 0x55f909ce1800 2023-05-20T01:22:18.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.106+0000 7f5a531fa700 1 -- 172.21.15.16:0/494201187 <== osd.2 v2:172.21.15.16:6816/293187030 5 ==== osd_op_reply(29 10000000204.00000000 [call,setxattr (8)] v71'9 uv9 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x559726fc1440 con 0x559726fc2c00 2023-05-20T01:22:18.423 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_extents', '--worker_n', '2', '--worker_m', '7'] in time 0:00:00.472397, result: 2023-05-20T01:22:18.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.106+0000 7f5a55a4d7c0 1 -- 172.21.15.16:0/494201187 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:30 11.15 11:a8000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x559727b1c700 con 0x559726fc2c00 2023-05-20T01:22:18.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.106+0000 7f704206c700 1 --2- 172.21.15.16:0/739319487 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x55f909ce1800 0x55f909df3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.106+0000 7f7d9b746700 1 -- 172.21.15.16:0/1705842756 <== osd.6 v2:172.21.15.132:6801/727581123 5 ==== osd_op_reply(35 100000001f7.00000001 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55687b04f8c0 con 0x55687b063400 2023-05-20T01:22:18.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.106+0000 7f7d9df997c0 10 datascan.operator(): handling object 1099511628279.1 2023-05-20T01:22:18.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.106+0000 7f7d9df997c0 1 -- 172.21.15.16:0/1705842756 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:36 10.36 10:6fe38dc2:::100000001f7.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x55687bbbd500 con 0x55687b062800 2023-05-20T01:22:18.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.107+0000 7f5a531fa700 1 -- 172.21.15.16:0/494201187 <== osd.2 v2:172.21.15.16:6816/293187030 6 ==== osd_op_reply(30 [pgnls start_epoch 71 out=101b] v69'4 uv4 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x559726fc1440 con 0x559726fc2c00 2023-05-20T01:22:18.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.107+0000 7f704206c700 1 --2- 172.21.15.16:0/739319487 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x55f909ce1800 0x55f909df3b80 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.107+0000 7f5a55a4d7c0 1 -- 172.21.15.16:0/494201187 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:31 11.15 11:a8e69ea8:::1000000020d.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x559727118e00 con 0x559726fc2c00 2023-05-20T01:22:18.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.107+0000 7f5a531fa700 1 -- 172.21.15.16:0/494201187 <== osd.2 v2:172.21.15.16:6816/293187030 7 ==== osd_op_reply(31 1000000020d.00000000 [stat out=16b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x559726fc1440 con 0x559726fc2c00 2023-05-20T01:22:18.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.107+0000 7f5a55a4d7c0 10 datascan.operator(): handling object 1099511628301.0 2023-05-20T01:22:18.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.107+0000 7f6194987700 1 -- 172.21.15.16:0/3379690421 <== osd.6 v2:172.21.15.132:6801/727581123 13 ==== osd_op_reply(30 10000000001.00000000 [call,setxattr (8)] v71'12 uv12 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x564461dd78c0 con 0x564461f2e400 2023-05-20T01:22:18.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.107+0000 7f5a55a4d7c0 1 -- 172.21.15.16:0/494201187 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:32 10.15 10:a8e69ea8:::1000000020d.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x559727118000 con 0x559726ff0800 2023-05-20T01:22:18.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.107+0000 7f61961d87c0 1 -- 172.21.15.16:0/3379690421 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:31 11.7 11:e5993619::::head [pgnls start_epoch 71 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x564461f7d180 con 0x564461e21000 2023-05-20T01:22:18.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.107+0000 7fe1ffbbd700 1 -- 172.21.15.16:0/2808449806 <== osd.7 v2:172.21.15.132:6800/466001149 7 ==== osd_op_reply(39 100000001f9.00000000 [call,setxattr (8)] v71'22 uv22 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x5624b6d67680 con 0x5624b6eae000 2023-05-20T01:22:18.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.107+0000 7fe20140e7c0 1 -- 172.21.15.16:0/2808449806 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:40 11.8 11:15246602::::head [pgnls start_epoch 71 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x5624b78c7180 con 0x5624b6a39800 2023-05-20T01:22:18.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.107+0000 7f6194186700 1 -- 172.21.15.16:0/3379690421 <== osd.5 v2:172.21.15.132:6816/1844689995 8 ==== osd_op_reply(31 [pgnls start_epoch 71 out=81b] v69'7 uv7 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x564461dd7b00 con 0x564461e21000 2023-05-20T01:22:18.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.108+0000 7f61961d87c0 1 -- 172.21.15.16:0/3379690421 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:32 11.7 11:e5993619:::10000000208.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x564461f7d500 con 0x564461e21000 2023-05-20T01:22:18.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.108+0000 7fe1ff3bc700 1 -- 172.21.15.16:0/2808449806 <== osd.3 v2:172.21.15.16:6800/3669781890 10 ==== osd_op_reply(40 [pgnls start_epoch 71 out=101b] v69'6 uv6 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x5624b6d678c0 con 0x5624b6a39800 2023-05-20T01:22:18.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.108+0000 7f6194186700 1 -- 172.21.15.16:0/3379690421 <== osd.5 v2:172.21.15.132:6816/1844689995 9 ==== osd_op_reply(32 10000000208.00000000 [stat out=16b] v0'0 uv7 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x564461dd7b00 con 0x564461e21000 2023-05-20T01:22:18.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.108+0000 7f7d9bf47700 1 -- 172.21.15.16:0/1705842756 <== osd.7 v2:172.21.15.132:6800/466001149 15 ==== osd_op_reply(36 100000001f7.00000000 [call,setxattr (8)] v71'23 uv23 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x55687b04f440 con 0x55687b062800 2023-05-20T01:22:18.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.108+0000 7f61961d87c0 10 datascan.operator(): handling object 1099511628296.0 2023-05-20T01:22:18.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.108+0000 7f61961d87c0 1 -- 172.21.15.16:0/3379690421 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:33 10.27 10:e5993619:::10000000208.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x564461f7d880 con 0x564461e21000 2023-05-20T01:22:18.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.108+0000 7fe20140e7c0 1 -- 172.21.15.16:0/2808449806 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:41 11.8 11:15246602:::10000000202.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x5624b78c6a80 con 0x5624b6a39800 2023-05-20T01:22:18.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.108+0000 7f7d9df997c0 1 --2- 172.21.15.16:0/1705842756 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x55687b063800 0x55687b189b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.108+0000 7f7d9df997c0 1 -- 172.21.15.16:0/1705842756 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:37 11.11 11:88000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x55687bbbc700 con 0x55687b063800 2023-05-20T01:22:18.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.108+0000 7f704206c700 1 -- 172.21.15.16:0/739319487 <== osd.0 v2:172.21.15.16:6824/3923634179 1 ==== osd_op_reply(38 1000000020b.00000000 [call,setxattr (8)] v71'13 uv13 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x55f909cc3680 con 0x55f909ce1800 2023-05-20T01:22:18.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.108+0000 7f7d9c748700 1 --2- 172.21.15.16:0/1705842756 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x55687b063800 0x55687b189b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.108+0000 7fe1ff3bc700 1 -- 172.21.15.16:0/2808449806 <== osd.3 v2:172.21.15.16:6800/3669781890 11 ==== osd_op_reply(41 10000000202.00000000 [stat out=16b] v0'0 uv6 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5624b6d678c0 con 0x5624b6a39800 2023-05-20T01:22:18.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.108+0000 7fe20140e7c0 10 datascan.operator(): handling object 1099511628290.0 2023-05-20T01:22:18.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.108+0000 7f70438bd7c0 1 -- 172.21.15.16:0/739319487 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:39 11.b 11:d0000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x55f909e0ea80 con 0x55f909ce1400 2023-05-20T01:22:18.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.108+0000 7fe20140e7c0 1 -- 172.21.15.16:0/2808449806 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:42 10.28 10:15246602:::10000000202.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x5624b6eb8700 con 0x5624b6eae000 2023-05-20T01:22:18.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.109+0000 7f7d9c748700 1 --2- 172.21.15.16:0/1705842756 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x55687b063800 0x55687b189b80 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.109+0000 7f704106a700 1 -- 172.21.15.16:0/739319487 <== osd.3 v2:172.21.15.16:6800/3669781890 3 ==== osd_op_reply(39 [pgnls start_epoch 71 out=101b] v69'3 uv3 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55f909cc38c0 con 0x55f909ce1400 2023-05-20T01:22:18.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.109+0000 7f5a541fc700 1 -- 172.21.15.16:0/494201187 <== osd.3 v2:172.21.15.16:6800/3669781890 6 ==== osd_op_reply(32 1000000020d.00000000 [call,setxattr (8)] v71'10 uv10 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x559726fc1680 con 0x559726ff0800 2023-05-20T01:22:18.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.109+0000 7f5a55a4d7c0 1 -- 172.21.15.16:0/494201187 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:33 11.15 11:ab3d427d::::head [pgnls start_epoch 71 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x559727118a80 con 0x559726fc2c00 2023-05-20T01:22:18.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.109+0000 7f70438bd7c0 1 -- 172.21.15.16:0/739319487 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:40 11.b 11:d2952c40:::10000000208.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x55f909e0fc00 con 0x55f909ce1400 2023-05-20T01:22:18.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.109+0000 7f7d9c748700 1 -- 172.21.15.16:0/1705842756 <== osd.2 v2:172.21.15.16:6816/293187030 1 ==== osd_op_reply(37 [pgnls start_epoch 71 out=101b] v69'2 uv2 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55687b04f680 con 0x55687b063800 2023-05-20T01:22:18.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.109+0000 7f704106a700 1 -- 172.21.15.16:0/739319487 <== osd.3 v2:172.21.15.16:6800/3669781890 4 ==== osd_op_reply(40 10000000208.00000001 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55f909cc38c0 con 0x55f909ce1400 2023-05-20T01:22:18.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.109+0000 7f7d9df997c0 1 -- 172.21.15.16:0/1705842756 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:38 11.11 11:893be98d:::100000001f8.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x55687bbbce00 con 0x55687b063800 2023-05-20T01:22:18.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.109+0000 7f70438bd7c0 10 datascan.operator(): handling object 1099511628296.1 2023-05-20T01:22:18.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.109+0000 7f70438bd7c0 1 -- 172.21.15.16:0/739319487 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:41 10.27 10:e5993619:::10000000208.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x55f909e0e700 con 0x55f909ce1000 2023-05-20T01:22:18.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.109+0000 7f5a531fa700 1 -- 172.21.15.16:0/494201187 <== osd.2 v2:172.21.15.16:6816/293187030 8 ==== osd_op_reply(33 [pgnls start_epoch 71 out=101b] v69'4 uv4 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x559726fc1440 con 0x559726fc2c00 2023-05-20T01:22:18.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.109+0000 7f5a55a4d7c0 1 -- 172.21.15.16:0/494201187 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:34 11.15 11:ab3d427d:::10000000203.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x559727119180 con 0x559726fc2c00 2023-05-20T01:22:18.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.109+0000 7f7d9c748700 1 -- 172.21.15.16:0/1705842756 <== osd.2 v2:172.21.15.16:6816/293187030 2 ==== osd_op_reply(38 100000001f8.00000001 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55687b04f680 con 0x55687b063800 2023-05-20T01:22:18.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.109+0000 7f7d9df997c0 10 datascan.operator(): handling object 1099511628280.1 2023-05-20T01:22:18.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.109+0000 7f7d9df997c0 1 -- 172.21.15.16:0/1705842756 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:39 10.19 10:997b7129:::100000001f8.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x55687b1a6000 con 0x55687b063000 2023-05-20T01:22:18.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.109+0000 7f5a531fa700 1 -- 172.21.15.16:0/494201187 <== osd.2 v2:172.21.15.16:6816/293187030 9 ==== osd_op_reply(34 10000000203.00000001 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x559726fc1440 con 0x559726fc2c00 2023-05-20T01:22:18.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.109+0000 7f5a55a4d7c0 10 datascan.operator(): handling object 1099511628291.1 2023-05-20T01:22:18.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.109+0000 7f6194186700 1 -- 172.21.15.16:0/3379690421 <== osd.5 v2:172.21.15.132:6816/1844689995 10 ==== osd_op_reply(33 10000000208.00000000 [call,setxattr (8)] v71'5 uv5 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x564461dd7b00 con 0x564461e21000 2023-05-20T01:22:18.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.109+0000 7f5a55a4d7c0 1 -- 172.21.15.16:0/494201187 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:35 10.2b 10:d4a58307:::10000000203.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x559727119500 con 0x559726fc2800 2023-05-20T01:22:18.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.109+0000 7f61961d87c0 1 -- 172.21.15.16:0/3379690421 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:34 11.17 11:e8000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x564461f7c000 con 0x564461f2e800 2023-05-20T01:22:18.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.110+0000 7fe1ffbbd700 1 -- 172.21.15.16:0/2808449806 <== osd.7 v2:172.21.15.132:6800/466001149 8 ==== osd_op_reply(42 10000000202.00000000 [call,setxattr (8)] v71'13 uv13 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x5624b6d67680 con 0x5624b6eae000 2023-05-20T01:22:18.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.110+0000 7fe20140e7c0 1 -- 172.21.15.16:0/2808449806 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:43 11.8 11:177a762d::::head [pgnls start_epoch 71 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x5624b6eb8380 con 0x5624b6a39800 2023-05-20T01:22:18.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.110+0000 7f6194987700 1 -- 172.21.15.16:0/3379690421 <== osd.7 v2:172.21.15.132:6800/466001149 3 ==== osd_op_reply(34 [pgnls start_epoch 71 out=81b] v69'1 uv1 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x564461dd78c0 con 0x564461f2e800 2023-05-20T01:22:18.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.110+0000 7f61961d87c0 1 -- 172.21.15.16:0/3379690421 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:35 11.17 11:e88218fc:::10000000209.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x564461f7c700 con 0x564461f2e800 2023-05-20T01:22:18.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.110+0000 7fe1ff3bc700 1 -- 172.21.15.16:0/2808449806 <== osd.3 v2:172.21.15.16:6800/3669781890 12 ==== osd_op_reply(43 [pgnls start_epoch 71 out=81b] v69'6 uv6 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5624b6d678c0 con 0x5624b6a39800 2023-05-20T01:22:18.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.110+0000 7fe20140e7c0 1 -- 172.21.15.16:0/2808449806 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:44 11.8 11:177a762d:::10000000201.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x5624b6eb8e00 con 0x5624b6a39800 2023-05-20T01:22:18.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.110+0000 7f6194987700 1 -- 172.21.15.16:0/3379690421 <== osd.7 v2:172.21.15.132:6800/466001149 4 ==== osd_op_reply(35 10000000209.00000001 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x564461dd78c0 con 0x564461f2e800 2023-05-20T01:22:18.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.110+0000 7f61961d87c0 10 datascan.operator(): handling object 1099511628297.1 2023-05-20T01:22:18.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.110+0000 7f61961d87c0 1 -- 172.21.15.16:0/3379690421 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:36 10.2c 10:37389ade:::10000000209.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x564461f7ca80 con 0x564461e20c00 2023-05-20T01:22:18.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.110+0000 7fe1ff3bc700 1 -- 172.21.15.16:0/2808449806 <== osd.3 v2:172.21.15.16:6800/3669781890 13 ==== osd_op_reply(44 10000000201.00000000 [stat out=16b] v0'0 uv5 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5624b6d678c0 con 0x5624b6a39800 2023-05-20T01:22:18.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.110+0000 7fe20140e7c0 10 datascan.operator(): handling object 1099511628289.0 2023-05-20T01:22:18.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.110+0000 7fe20140e7c0 1 -- 172.21.15.16:0/2808449806 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:45 10.28 10:177a762d:::10000000201.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x5624b6eb9500 con 0x5624b6eae000 2023-05-20T01:22:18.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.111+0000 7f704186b700 1 -- 172.21.15.16:0/739319487 <== osd.5 v2:172.21.15.132:6816/1844689995 11 ==== osd_op_reply(41 10000000208.00000000 [call,setxattr (8)] v71'6 uv6 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x55f909cc3440 con 0x55f909ce1000 2023-05-20T01:22:18.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.111+0000 7f70438bd7c0 1 -- 172.21.15.16:0/739319487 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:42 11.b 11:d4a58307::::head [pgnls start_epoch 71 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x55f90a82f180 con 0x55f909ce1400 2023-05-20T01:22:18.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.111+0000 7f7d9c748700 1 -- 172.21.15.16:0/1705842756 <== osd.1 v2:172.21.15.16:6808/3860264220 2 ==== osd_op_reply(39 100000001f8.00000000 [call,setxattr (8)] v71'17 uv17 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x55687b04f680 con 0x55687b063000 2023-05-20T01:22:18.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.111+0000 7f7d9df997c0 1 -- 172.21.15.16:0/1705842756 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:40 11.11 11:8f3fbbd0::::head [pgnls start_epoch 71 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x55687b1a6380 con 0x55687b063800 2023-05-20T01:22:18.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.111+0000 7f704106a700 1 -- 172.21.15.16:0/739319487 <== osd.3 v2:172.21.15.16:6800/3669781890 5 ==== osd_op_reply(42 [pgnls start_epoch 71 out=81b] v69'3 uv3 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55f909cc38c0 con 0x55f909ce1400 2023-05-20T01:22:18.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.111+0000 7f5a531fa700 1 -- 172.21.15.16:0/494201187 <== osd.5 v2:172.21.15.132:6816/1844689995 2 ==== osd_op_reply(35 10000000203.00000000 [call,setxattr (8)] v71'14 uv14 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x559726fc1440 con 0x559726fc2800 2023-05-20T01:22:18.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.111+0000 7f70438bd7c0 1 -- 172.21.15.16:0/739319487 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:43 11.b 11:d4a58307:::10000000203.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x55f90a82ee00 con 0x55f909ce1400 2023-05-20T01:22:18.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.111+0000 7f5a55a4d7c0 1 -- 172.21.15.16:0/494201187 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:36 11.15 11:ab9be7ec::::head [pgnls start_epoch 71 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x559727119880 con 0x559726fc2c00 2023-05-20T01:22:18.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.111+0000 7f7d9c748700 1 -- 172.21.15.16:0/1705842756 <== osd.2 v2:172.21.15.16:6816/293187030 3 ==== osd_op_reply(40 [pgnls start_epoch 71 out=81b] v69'2 uv2 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55687b04f680 con 0x55687b063800 2023-05-20T01:22:18.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.111+0000 7f7d9df997c0 1 -- 172.21.15.16:0/1705842756 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:41 11.11 11:8f3fbbd0:::1000000020c.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x55687b1a6700 con 0x55687b063800 2023-05-20T01:22:18.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.111+0000 7f704106a700 1 -- 172.21.15.16:0/739319487 <== osd.3 v2:172.21.15.16:6800/3669781890 6 ==== osd_op_reply(43 10000000203.00000000 [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55f909cc38c0 con 0x55f909ce1400 2023-05-20T01:22:18.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.111+0000 7f70438bd7c0 10 datascan.operator(): handling object 1099511628291.0 2023-05-20T01:22:18.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.111+0000 7f70438bd7c0 1 -- 172.21.15.16:0/739319487 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:44 10.2b 10:d4a58307:::10000000203.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x55f90a82f500 con 0x55f909ce1000 2023-05-20T01:22:18.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.112+0000 7f7d9c748700 1 -- 172.21.15.16:0/1705842756 <== osd.2 v2:172.21.15.16:6816/293187030 4 ==== osd_op_reply(41 1000000020c.00000001 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55687b04f680 con 0x55687b063800 2023-05-20T01:22:18.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.112+0000 7f7d9df997c0 10 datascan.operator(): handling object 1099511628300.1 2023-05-20T01:22:18.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.112+0000 7f7d9df997c0 1 -- 172.21.15.16:0/1705842756 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:42 10.1e 10:789b801b:::1000000020c.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x55687b1a6a80 con 0x55687b065000 2023-05-20T01:22:18.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.112+0000 7f6193985700 1 -- 172.21.15.16:0/3379690421 <== osd.3 v2:172.21.15.16:6800/3669781890 2 ==== osd_op_reply(36 10000000209.00000000 [call,setxattr (8)] v71'12 uv12 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x564461dd7680 con 0x564461e20c00 2023-05-20T01:22:18.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.112+0000 7f61961d87c0 1 -- 172.21.15.16:0/3379690421 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:37 11.f 11:f0000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x564461f7d180 con 0x564461f2e800 2023-05-20T01:22:18.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.112+0000 7f5a531fa700 1 -- 172.21.15.16:0/494201187 <== osd.2 v2:172.21.15.16:6816/293187030 10 ==== osd_op_reply(36 [pgnls start_epoch 71 out=81b] v69'4 uv4 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x559726fc1440 con 0x559726fc2c00 2023-05-20T01:22:18.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.112+0000 7fe1ffbbd700 1 -- 172.21.15.16:0/2808449806 <== osd.7 v2:172.21.15.132:6800/466001149 9 ==== osd_op_reply(45 10000000201.00000000 [call,setxattr (8)] v71'14 uv14 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x5624b6d67680 con 0x5624b6eae000 2023-05-20T01:22:18.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.112+0000 7f5a55a4d7c0 1 -- 172.21.15.16:0/494201187 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:37 11.15 11:ab9be7ec:::10000000205.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x559727b1ce00 con 0x559726fc2c00 2023-05-20T01:22:18.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.112+0000 7fe20140e7c0 1 -- 172.21.15.16:0/2808449806 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:46 11.18 11:18000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x5624b6eb8000 con 0x5624b6e76400 2023-05-20T01:22:18.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.112+0000 7f6194987700 1 -- 172.21.15.16:0/3379690421 <== osd.7 v2:172.21.15.132:6800/466001149 5 ==== osd_op_reply(37 [pgnls start_epoch 71 out=81b] v67'1 uv1 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x564461dd78c0 con 0x564461f2e800 2023-05-20T01:22:18.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.112+0000 7f5a531fa700 1 -- 172.21.15.16:0/494201187 <== osd.2 v2:172.21.15.16:6816/293187030 11 ==== osd_op_reply(37 10000000205.00000001 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x559726fc1440 con 0x559726fc2c00 2023-05-20T01:22:18.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.112+0000 7f5a55a4d7c0 10 datascan.operator(): handling object 1099511628293.1 2023-05-20T01:22:18.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.112+0000 7fe1febbb700 1 -- 172.21.15.16:0/2808449806 <== osd.2 v2:172.21.15.16:6816/293187030 3 ==== osd_op_reply(46 [pgnls start_epoch 71 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5624b6d67440 con 0x5624b6e76400 2023-05-20T01:22:18.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.112+0000 7f5a55a4d7c0 1 -- 172.21.15.16:0/494201187 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:38 10.3d 10:bfcd3380:::10000000205.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x559727b1d880 con 0x559726fc3400 2023-05-20T01:22:18.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.112+0000 7f61961d87c0 1 -- 172.21.15.16:0/3379690421 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:38 11.f 11:f526278d:::10000000001.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x564462999180 con 0x564461f2e800 2023-05-20T01:22:18.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.112+0000 7fe1fabb3700 1 -- 172.21.15.16:0/2808449806 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:47 11.4 11:20000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x5624b78c7c00 con 0x5624b6eae000 2023-05-20T01:22:18.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.113+0000 7f6194987700 1 -- 172.21.15.16:0/3379690421 <== osd.7 v2:172.21.15.132:6800/466001149 6 ==== osd_op_reply(38 10000000001.00000001 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x564461dd78c0 con 0x564461f2e800 2023-05-20T01:22:18.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.113+0000 7f61961d87c0 10 datascan.operator(): handling object 1099511627777.1 2023-05-20T01:22:18.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.113+0000 7f61961d87c0 1 -- 172.21.15.16:0/3379690421 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:39 10.7 10:e2e07930:::10000000001.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x564462998700 con 0x564461f2e400 2023-05-20T01:22:18.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.113+0000 7fe1ffbbd700 1 -- 172.21.15.16:0/2808449806 <== osd.7 v2:172.21.15.132:6800/466001149 10 ==== osd_op_reply(47 [pgnls start_epoch 71 out=101b] v69'3 uv3 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x5624b6d67680 con 0x5624b6eae000 2023-05-20T01:22:18.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.113+0000 7fe20140e7c0 1 -- 172.21.15.16:0/2808449806 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:48 11.4 11:2057972a:::100000001ff.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x5624b6eb9180 con 0x5624b6eae000 2023-05-20T01:22:18.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.113+0000 7f704186b700 1 -- 172.21.15.16:0/739319487 <== osd.5 v2:172.21.15.132:6816/1844689995 12 ==== osd_op_reply(44 10000000203.00000000 [call,setxattr (8)] v71'15 uv15 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x55f909cc3440 con 0x55f909ce1000 2023-05-20T01:22:18.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.113+0000 7f70438bd7c0 1 -- 172.21.15.16:0/739319487 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:45 11.1b 11:d8000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x55f90a82ea80 con 0x55f909ce7000 2023-05-20T01:22:18.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.113+0000 7f7d9b746700 1 -- 172.21.15.16:0/1705842756 <== osd.0 v2:172.21.15.16:6824/3923634179 5 ==== osd_op_reply(42 1000000020c.00000000 [call,setxattr (8)] v71'9 uv9 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x55687b04f8c0 con 0x55687b065000 2023-05-20T01:22:18.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.113+0000 7f7d9df997c0 1 -- 172.21.15.16:0/1705842756 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:43 11.9 11:90000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x55687b1a6e00 con 0x55687b064c00 2023-05-20T01:22:18.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.113+0000 7fe1ffbbd700 1 -- 172.21.15.16:0/2808449806 <== osd.7 v2:172.21.15.132:6800/466001149 11 ==== osd_op_reply(48 100000001ff.00000001 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5624b6d67680 con 0x5624b6eae000 2023-05-20T01:22:18.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.113+0000 7fe20140e7c0 10 datascan.operator(): handling object 1099511628287.1 2023-05-20T01:22:18.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.113+0000 7fe20140e7c0 1 -- 172.21.15.16:0/2808449806 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:49 10.26 10:658f2455:::100000001ff.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x5624b6eb9c00 con 0x5624b6eae400 2023-05-20T01:22:18.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.113+0000 7f7d9bf47700 1 -- 172.21.15.16:0/1705842756 <== osd.3 v2:172.21.15.16:6800/3669781890 2 ==== osd_op_reply(43 [pgnls start_epoch 71 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55687b04f440 con 0x55687b064c00 2023-05-20T01:22:18.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.113+0000 7f704206c700 1 -- 172.21.15.16:0/739319487 <== osd.4 v2:172.21.15.132:6817/506317223 2 ==== osd_op_reply(45 [pgnls start_epoch 71 out=81b] v69'2 uv2 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55f909cc3680 con 0x55f909ce7000 2023-05-20T01:22:18.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.113+0000 7f7d9df997c0 1 -- 172.21.15.16:0/1705842756 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x55687b065000 msgr2=0x55687b188580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.113+0000 7f70438bd7c0 1 -- 172.21.15.16:0/739319487 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x55f909ce1800 msgr2=0x55f909df3b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.113+0000 7f70438bd7c0 1 --2- 172.21.15.16:0/739319487 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x55f909ce1800 0x55f909df3b80 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.113+0000 7f7d9df997c0 1 --2- 172.21.15.16:0/1705842756 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x55687b065000 0x55687b188580 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.113+0000 7f7d9df997c0 1 -- 172.21.15.16:0/1705842756 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x55687b063000 msgr2=0x55687adcd600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.113+0000 7f7d9df997c0 1 --2- 172.21.15.16:0/1705842756 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x55687b063000 0x55687adcd600 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.113+0000 7f7d9df997c0 1 -- 172.21.15.16:0/1705842756 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x55687b063800 msgr2=0x55687b189b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.113+0000 7f70438bd7c0 1 -- 172.21.15.16:0/739319487 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x55f909ce7800 msgr2=0x55f909df2b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.113+0000 7f70438bd7c0 1 --2- 172.21.15.16:0/739319487 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x55f909ce7800 0x55f909df2b00 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.113+0000 7f70438bd7c0 1 -- 172.21.15.16:0/739319487 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x55f909ce0c00 msgr2=0x55f909a2db80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.113+0000 7f70438bd7c0 1 --2- 172.21.15.16:0/739319487 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x55f909ce0c00 0x55f909a2db80 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.114+0000 7f7d9df997c0 1 --2- 172.21.15.16:0/1705842756 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x55687b063800 0x55687b189b80 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.114+0000 7f7d9df997c0 1 -- 172.21.15.16:0/1705842756 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x55687b064c00 msgr2=0x55687b188000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.114+0000 7f7d9df997c0 1 --2- 172.21.15.16:0/1705842756 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x55687b064c00 0x55687b188000 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.114+0000 7f7d9df997c0 1 -- 172.21.15.16:0/1705842756 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x55687b062c00 msgr2=0x55687adcdb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.114+0000 7f7d9df997c0 1 --2- 172.21.15.16:0/1705842756 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x55687b062c00 0x55687adcdb80 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.114+0000 7f7d9df997c0 1 -- 172.21.15.16:0/1705842756 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x55687b065400 msgr2=0x55687b188b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.114+0000 7f7d9df997c0 1 --2- 172.21.15.16:0/1705842756 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x55687b065400 0x55687b188b00 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.114+0000 7f70438bd7c0 1 -- 172.21.15.16:0/739319487 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x55f909ce1400 msgr2=0x55f909df3600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.114+0000 7f70438bd7c0 1 --2- 172.21.15.16:0/739319487 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x55f909ce1400 0x55f909df3600 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.114+0000 7f704106a700 1 -- 172.21.15.16:0/739319487 reap_dead start 2023-05-20T01:22:18.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.114+0000 7f70438bd7c0 1 -- 172.21.15.16:0/739319487 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x55f909ce7000 msgr2=0x55f909df2000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.114+0000 7f70438bd7c0 1 --2- 172.21.15.16:0/739319487 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x55f909ce7000 0x55f909df2000 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.114+0000 7f7d9df997c0 1 -- 172.21.15.16:0/1705842756 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x55687b063400 msgr2=0x55687b189600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.114+0000 7f7d9df997c0 1 --2- 172.21.15.16:0/1705842756 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x55687b063400 0x55687b189600 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.114+0000 7f7d9df997c0 1 -- 172.21.15.16:0/1705842756 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x55687b062800 msgr2=0x55687adce100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.114+0000 7f7d9df997c0 1 --2- 172.21.15.16:0/1705842756 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x55687b062800 0x55687adce100 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.114+0000 7f7d9df997c0 1 -- 172.21.15.16:0/1705842756 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x55687b064800 msgr2=0x55687adcf700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.114+0000 7f7d9df997c0 1 --2- 172.21.15.16:0/1705842756 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x55687b064800 0x55687adcf700 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x55687b067290 tx=0x55687bb24000 comp rx=0 tx=0).stop 2023-05-20T01:22:18.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.114+0000 7f7d9df997c0 1 -- 172.21.15.16:0/1705842756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55687b062400 msgr2=0x55687adce680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.114+0000 7f7d9df997c0 1 --2- 172.21.15.16:0/1705842756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55687b062400 0x55687adce680 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x55687b0673b0 tx=0x55687bb242a0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.114+0000 7f5a541fc700 1 -- 172.21.15.16:0/494201187 <== osd.7 v2:172.21.15.132:6800/466001149 3 ==== osd_op_reply(38 10000000205.00000000 [call,setxattr (8)] v71'11 uv11 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x559726fc1680 con 0x559726fc3400 2023-05-20T01:22:18.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.114+0000 7f5a55a4d7c0 1 -- 172.21.15.16:0/494201187 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:39 11.d 11:b0000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x559727118e00 con 0x559726fc2c00 2023-05-20T01:22:18.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.114+0000 7f704106a700 1 -- 172.21.15.16:0/739319487 reap_dead start 2023-05-20T01:22:18.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.114+0000 7f70438bd7c0 1 -- 172.21.15.16:0/739319487 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x55f909ce1000 msgr2=0x55f909a2d600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.114+0000 7f70438bd7c0 1 --2- 172.21.15.16:0/739319487 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x55f909ce1000 0x55f909a2d600 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.114+0000 7f70438bd7c0 1 -- 172.21.15.16:0/739319487 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x55f909ce0800 msgr2=0x55f909a2e100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.114+0000 7f70438bd7c0 1 --2- 172.21.15.16:0/739319487 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x55f909ce0800 0x55f909a2e100 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.114+0000 7f70438bd7c0 1 -- 172.21.15.16:0/739319487 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x55f909ce7400 msgr2=0x55f909df2580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.114+0000 7f70438bd7c0 1 --2- 172.21.15.16:0/739319487 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x55f909ce7400 0x55f909df2580 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.114+0000 7f70438bd7c0 1 -- 172.21.15.16:0/739319487 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x55f909ce6c00 msgr2=0x55f909a2f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.114+0000 7f70438bd7c0 1 --2- 172.21.15.16:0/739319487 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x55f909ce6c00 0x55f909a2f700 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x55f909aee120 tx=0x55f909de4000 comp rx=0 tx=0).stop 2023-05-20T01:22:18.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.114+0000 7f70438bd7c0 1 -- 172.21.15.16:0/739319487 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55f909ce0400 msgr2=0x55f909a2e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.114+0000 7f70438bd7c0 1 --2- 172.21.15.16:0/739319487 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55f909ce0400 0x55f909a2e680 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x55f909cd33b0 tx=0x55f909afcde0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.114+0000 7f704106a700 1 -- 172.21.15.16:0/739319487 reap_dead start 2023-05-20T01:22:18.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.114+0000 7f704106a700 1 -- 172.21.15.16:0/739319487 reap_dead start 2023-05-20T01:22:18.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.114+0000 7f70438bd7c0 1 -- 172.21.15.16:0/739319487 shutdown_connections 2023-05-20T01:22:18.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.114+0000 7f70438bd7c0 1 -- 172.21.15.16:0/739319487 >> 172.21.15.16:0/739319487 conn(0x55f909993c00 msgr2=0x55f909a6f800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:18.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.114+0000 7f7d9bf47700 1 -- 172.21.15.16:0/1705842756 reap_dead start 2023-05-20T01:22:18.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.114+0000 7f7d9bf47700 1 -- 172.21.15.16:0/1705842756 reap_dead start 2023-05-20T01:22:18.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.114+0000 7f7d9bf47700 1 -- 172.21.15.16:0/1705842756 reap_dead start 2023-05-20T01:22:18.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.114+0000 7f7d9bf47700 1 -- 172.21.15.16:0/1705842756 reap_dead start 2023-05-20T01:22:18.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.114+0000 7f7d9df997c0 1 -- 172.21.15.16:0/1705842756 shutdown_connections 2023-05-20T01:22:18.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.114+0000 7f7d9df997c0 1 -- 172.21.15.16:0/1705842756 >> 172.21.15.16:0/1705842756 conn(0x55687ad33c00 msgr2=0x55687ae03800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:18.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.114+0000 7f7d9bf47700 1 -- 172.21.15.16:0/1705842756 reap_dead start 2023-05-20T01:22:18.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.114+0000 7f7d9bf47700 1 -- 172.21.15.16:0/1705842756 reap_dead start 2023-05-20T01:22:18.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.114+0000 7f7d9bf47700 1 -- 172.21.15.16:0/1705842756 reap_dead start 2023-05-20T01:22:18.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.114+0000 7f7d9df997c0 1 -- 172.21.15.16:0/1705842756 shutdown_connections 2023-05-20T01:22:18.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.114+0000 7f7d9df997c0 1 -- 172.21.15.16:0/1705842756 wait complete. 2023-05-20T01:22:18.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.114+0000 7f5a531fa700 1 -- 172.21.15.16:0/494201187 <== osd.2 v2:172.21.15.16:6816/293187030 12 ==== osd_op_reply(39 [pgnls start_epoch 71 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559726fc1440 con 0x559726fc2c00 2023-05-20T01:22:18.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.114+0000 7f70438bd7c0 1 -- 172.21.15.16:0/739319487 shutdown_connections 2023-05-20T01:22:18.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.114+0000 7f70438bd7c0 1 -- 172.21.15.16:0/739319487 wait complete. 2023-05-20T01:22:18.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.114+0000 7f5a55a4d7c0 1 -- 172.21.15.16:0/494201187 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x559726ff1800 msgr2=0x5597270f7600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.114+0000 7f5a55a4d7c0 1 --2- 172.21.15.16:0/494201187 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x559726ff1800 0x5597270f7600 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.114+0000 7f5a55a4d7c0 1 -- 172.21.15.16:0/494201187 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x559726ff0c00 msgr2=0x5597270f6580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.114+0000 7f5a55a4d7c0 1 --2- 172.21.15.16:0/494201187 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x559726ff0c00 0x5597270f6580 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.114+0000 7f5a55a4d7c0 1 -- 172.21.15.16:0/494201187 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x559726fc2c00 msgr2=0x559726d2fb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.114+0000 7f5a55a4d7c0 1 --2- 172.21.15.16:0/494201187 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x559726fc2c00 0x559726d2fb80 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.114+0000 7f5a55a4d7c0 1 -- 172.21.15.16:0/494201187 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x559726ff0800 msgr2=0x5597270f6000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.114+0000 7f5a55a4d7c0 1 --2- 172.21.15.16:0/494201187 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x559726ff0800 0x5597270f6000 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.115+0000 7f5a55a4d7c0 1 -- 172.21.15.16:0/494201187 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x559726fc3000 msgr2=0x559726d2f600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.115+0000 7f5a55a4d7c0 1 --2- 172.21.15.16:0/494201187 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x559726fc3000 0x559726d2f600 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.115+0000 7f5a55a4d7c0 1 -- 172.21.15.16:0/494201187 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x559726fc2800 msgr2=0x559726d30100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.115+0000 7f5a55a4d7c0 1 --2- 172.21.15.16:0/494201187 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x559726fc2800 0x559726d30100 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.115+0000 7f5a55a4d7c0 1 -- 172.21.15.16:0/494201187 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x559726ff1000 msgr2=0x5597270f6b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.115+0000 7f5a539fb700 1 -- 172.21.15.16:0/494201187 reap_dead start 2023-05-20T01:22:18.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.115+0000 7f5a55a4d7c0 1 --2- 172.21.15.16:0/494201187 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x559726ff1000 0x5597270f6b00 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.115+0000 7f6194987700 1 -- 172.21.15.16:0/3379690421 <== osd.6 v2:172.21.15.132:6801/727581123 14 ==== osd_op_reply(39 10000000001.00000000 [call,setxattr (8)] v71'13 uv13 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x564461dd78c0 con 0x564461f2e400 2023-05-20T01:22:18.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.115+0000 7f7d9df997c0 1 -- 172.21.15.16:0/3140357676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55687ad32c00 msgr2=0x55687adccb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.115+0000 7f7d9df997c0 1 --2- 172.21.15.16:0/3140357676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55687ad32c00 0x55687adccb00 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x55687bb1aea0 tx=0x55687bb24360 comp rx=0 tx=0).stop 2023-05-20T01:22:18.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.115+0000 7f5a55a4d7c0 1 -- 172.21.15.16:0/494201187 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x559726fc3400 msgr2=0x5597270f8100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.115+0000 7f5a55a4d7c0 1 --2- 172.21.15.16:0/494201187 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x559726fc3400 0x5597270f8100 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.115+0000 7f7d9df997c0 1 -- 172.21.15.16:0/3140357676 shutdown_connections 2023-05-20T01:22:18.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.115+0000 7f7d9df997c0 1 --2- 172.21.15.16:0/3140357676 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55687ad33000 0x55687adcd080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.115+0000 7f7d9df997c0 1 --2- 172.21.15.16:0/3140357676 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55687ad32800 0x55687adcc580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.115+0000 7f7d9df997c0 1 --2- 172.21.15.16:0/3140357676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55687ad32c00 0x55687adccb00 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.115+0000 7f5a55a4d7c0 1 -- 172.21.15.16:0/494201187 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x559726ff0400 msgr2=0x559726d31700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.115+0000 7f5a55a4d7c0 1 --2- 172.21.15.16:0/494201187 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x559726ff0400 0x559726d31700 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x559726fcc0c0 tx=0x559727a86300 comp rx=0 tx=0).stop 2023-05-20T01:22:18.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.115+0000 7f5a55a4d7c0 1 -- 172.21.15.16:0/494201187 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559726c95800 msgr2=0x559726d31180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.115+0000 7f5a55a4d7c0 1 --2- 172.21.15.16:0/494201187 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559726c95800 0x559726d31180 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x559726fcd440 tx=0x559727a86060 comp rx=0 tx=0).stop 2023-05-20T01:22:18.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.115+0000 7f7d9df997c0 1 -- 172.21.15.16:0/3140357676 >> 172.21.15.16:0/3140357676 conn(0x55687ad32400 msgr2=0x55687ae03000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:18.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.115+0000 7fe1ff3bc700 1 -- 172.21.15.16:0/2808449806 <== osd.5 v2:172.21.15.132:6816/1844689995 2 ==== osd_op_reply(49 100000001ff.00000000 [call,setxattr (8)] v71'19 uv19 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x5624b6d678c0 con 0x5624b6eae400 2023-05-20T01:22:18.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.115+0000 7f7d9df997c0 1 -- 172.21.15.16:0/3140357676 shutdown_connections 2023-05-20T01:22:18.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.115+0000 7fe20140e7c0 1 -- 172.21.15.16:0/2808449806 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:50 11.4 11:218dfa3a::::head [pgnls start_epoch 71 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x5624b78c6700 con 0x5624b6eae000 2023-05-20T01:22:18.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.115+0000 7f61961d87c0 1 --2- 172.21.15.16:0/3379690421 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x564461e21800 0x564461f5cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.115+0000 7f61961d87c0 1 -- 172.21.15.16:0/3379690421 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:40 11.1f 11:f8000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x564462999880 con 0x564461e21800 2023-05-20T01:22:18.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.115+0000 7f7d9df997c0 1 -- 172.21.15.16:0/3140357676 wait complete. 2023-05-20T01:22:18.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.115+0000 7f5a55a4d7c0 1 -- 172.21.15.16:0/494201187 shutdown_connections 2023-05-20T01:22:18.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.115+0000 7f5a55a4d7c0 1 --2- 172.21.15.16:0/494201187 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x559726ff1000 0x5597270f6b00 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.115+0000 7f5a55a4d7c0 1 --2- 172.21.15.16:0/494201187 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x559726fc3400 0x5597270f8100 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.115+0000 7f5a55a4d7c0 1 --2- 172.21.15.16:0/494201187 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559726c95800 0x559726d31180 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.115+0000 7f5a55a4d7c0 1 --2- 172.21.15.16:0/494201187 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x559726ff0400 0x559726d31700 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.115+0000 7f5a55a4d7c0 1 -- 172.21.15.16:0/494201187 >> 172.21.15.16:0/494201187 conn(0x559726c95c00 msgr2=0x559726d71800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:18.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.115+0000 7f6193985700 1 --2- 172.21.15.16:0/3379690421 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x564461e21800 0x564461f5cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.115+0000 7f5a55a4d7c0 1 -- 172.21.15.16:0/494201187 shutdown_connections 2023-05-20T01:22:18.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.115+0000 7f70438bd7c0 1 -- 172.21.15.16:0/1197830548 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f909993000 msgr2=0x55f909a2c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.115+0000 7f70438bd7c0 1 --2- 172.21.15.16:0/1197830548 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f909993000 0x55f909a2c580 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x55f909aeed20 tx=0x55f90a72b3e0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.115+0000 7fe1ffbbd700 1 -- 172.21.15.16:0/2808449806 <== osd.7 v2:172.21.15.132:6800/466001149 12 ==== osd_op_reply(50 [pgnls start_epoch 71 out=101b] v69'3 uv3 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x5624b6d67680 con 0x5624b6eae000 2023-05-20T01:22:18.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.115+0000 7f70438bd7c0 1 -- 172.21.15.16:0/1197830548 shutdown_connections 2023-05-20T01:22:18.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.115+0000 7f70438bd7c0 1 --2- 172.21.15.16:0/1197830548 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f909992800 0x55f909a2d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.115+0000 7f70438bd7c0 1 --2- 172.21.15.16:0/1197830548 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f909993000 0x55f909a2c580 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.115+0000 7f70438bd7c0 1 --2- 172.21.15.16:0/1197830548 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55f909992c00 0x55f909a2cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.115+0000 7f70438bd7c0 1 -- 172.21.15.16:0/1197830548 >> 172.21.15.16:0/1197830548 conn(0x55f909993400 msgr2=0x55f909a6f000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:18.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.115+0000 7f5a55a4d7c0 1 -- 172.21.15.16:0/494201187 wait complete. 2023-05-20T01:22:18.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.115+0000 7fe20140e7c0 1 -- 172.21.15.16:0/2808449806 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:51 11.4 11:218dfa3a:::10000000200.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x5624b78c7880 con 0x5624b6eae000 2023-05-20T01:22:18.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.115+0000 7f6193985700 1 --2- 172.21.15.16:0/3379690421 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x564461e21800 0x564461f5cb00 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.115+0000 7f70438bd7c0 1 -- 172.21.15.16:0/1197830548 shutdown_connections 2023-05-20T01:22:18.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.116+0000 7f70438bd7c0 1 -- 172.21.15.16:0/1197830548 wait complete. 2023-05-20T01:22:18.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.116+0000 7fe1ffbbd700 1 -- 172.21.15.16:0/2808449806 <== osd.7 v2:172.21.15.132:6800/466001149 13 ==== osd_op_reply(51 10000000200.00000001 [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5624b6d67680 con 0x5624b6eae000 2023-05-20T01:22:18.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.116+0000 7fe20140e7c0 10 datascan.operator(): handling object 1099511628288.1 2023-05-20T01:22:18.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.116+0000 7fe20140e7c0 1 -- 172.21.15.16:0/2808449806 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:52 10.19 10:992a6246:::10000000200.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x5624b6eb8700 con 0x5624b6eae800 2023-05-20T01:22:18.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.116+0000 7f5a55a4d7c0 1 -- 172.21.15.16:0/544605894 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559726c95000 msgr2=0x559726d2f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.116+0000 7f5a55a4d7c0 1 --2- 172.21.15.16:0/544605894 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559726c95000 0x559726d2f080 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x559727a6ae40 tx=0x559727a86480 comp rx=0 tx=0).stop 2023-05-20T01:22:18.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.116+0000 7f6193985700 1 -- 172.21.15.16:0/3379690421 <== osd.2 v2:172.21.15.16:6816/293187030 1 ==== osd_op_reply(40 [pgnls start_epoch 71 out=81b] v69'2 uv2 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x564461dd7680 con 0x564461e21800 2023-05-20T01:22:18.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.116+0000 7f61961d87c0 1 -- 172.21.15.16:0/3379690421 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:41 11.1f 11:feb1c803:::100000001fa.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x564462998a80 con 0x564461e21800 2023-05-20T01:22:18.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.116+0000 7f6193985700 1 -- 172.21.15.16:0/3379690421 <== osd.2 v2:172.21.15.16:6816/293187030 2 ==== osd_op_reply(41 100000001fa.00000000 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x564461dd7680 con 0x564461e21800 2023-05-20T01:22:18.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.116+0000 7f61961d87c0 10 datascan.operator(): handling object 1099511628282.0 2023-05-20T01:22:18.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.116+0000 7f61961d87c0 1 -- 172.21.15.16:0/3379690421 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:42 10.3f 10:feb1c803:::100000001fa.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x564462998e00 con 0x564461f2ec00 2023-05-20T01:22:18.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.117+0000 7fe1febbb700 1 -- 172.21.15.16:0/2808449806 <== osd.1 v2:172.21.15.16:6808/3860264220 8 ==== osd_op_reply(52 10000000200.00000000 [call,setxattr (8)] v71'18 uv18 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x5624b6d67440 con 0x5624b6eae800 2023-05-20T01:22:18.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.117+0000 7fe20140e7c0 1 -- 172.21.15.16:0/2808449806 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x5624b6a39c00 msgr2=0x5624b6ad5180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.117+0000 7fe20140e7c0 1 --2- 172.21.15.16:0/2808449806 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x5624b6a39c00 0x5624b6ad5180 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.117+0000 7fe20140e7c0 1 -- 172.21.15.16:0/2808449806 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x5624b6eae800 msgr2=0x5624b6e9e680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.117+0000 7fe20140e7c0 1 --2- 172.21.15.16:0/2808449806 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x5624b6eae800 0x5624b6e9e680 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.117+0000 7fe20140e7c0 1 -- 172.21.15.16:0/2808449806 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x5624b6e76400 msgr2=0x5624b6e9c580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.117+0000 7fe20140e7c0 1 --2- 172.21.15.16:0/2808449806 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x5624b6e76400 0x5624b6e9c580 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.117+0000 7fe20140e7c0 1 -- 172.21.15.16:0/2808449806 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x5624b6a39800 msgr2=0x5624b6e9c000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.117+0000 7fe20140e7c0 1 --2- 172.21.15.16:0/2808449806 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x5624b6a39800 0x5624b6e9c000 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.117+0000 7fe20140e7c0 1 -- 172.21.15.16:0/2808449806 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x5624b6eae400 msgr2=0x5624b6e9e100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.117+0000 7fe20140e7c0 1 --2- 172.21.15.16:0/2808449806 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x5624b6eae400 0x5624b6e9e100 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.117+0000 7fe20140e7c0 1 -- 172.21.15.16:0/2808449806 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x5624b78d2400 msgr2=0x5624b6ad4c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.117+0000 7fe20140e7c0 1 --2- 172.21.15.16:0/2808449806 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x5624b78d2400 0x5624b6ad4c00 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.117+0000 7fe20140e7c0 1 -- 172.21.15.16:0/2808449806 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x5624b6eae000 msgr2=0x5624b6e9db80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.117+0000 7fe20140e7c0 1 --2- 172.21.15.16:0/2808449806 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x5624b6eae000 0x5624b6e9db80 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.117+0000 7fe1ff3bc700 1 -- 172.21.15.16:0/2808449806 reap_dead start 2023-05-20T01:22:18.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.117+0000 7fe1ff3bc700 1 -- 172.21.15.16:0/2808449806 reap_dead start 2023-05-20T01:22:18.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.117+0000 7fe20140e7c0 1 -- 172.21.15.16:0/2808449806 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x5624b6e76000 msgr2=0x5624b6ad5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.117+0000 7fe20140e7c0 1 --2- 172.21.15.16:0/2808449806 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x5624b6e76000 0x5624b6ad5700 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x5624b7814c60 tx=0x5624b6e7c420 comp rx=0 tx=0).stop 2023-05-20T01:22:18.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.117+0000 7fe20140e7c0 1 -- 172.21.15.16:0/2808449806 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5624b78d2c00 msgr2=0x5624b6ad4100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.117+0000 7fe20140e7c0 1 --2- 172.21.15.16:0/2808449806 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5624b78d2c00 0x5624b6ad4100 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x5624b6a935c0 tx=0x5624b781e000 comp rx=0 tx=0).stop 2023-05-20T01:22:18.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.117+0000 7fe1ff3bc700 1 -- 172.21.15.16:0/2808449806 reap_dead start 2023-05-20T01:22:18.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.117+0000 7fe1ff3bc700 1 -- 172.21.15.16:0/2808449806 reap_dead start 2023-05-20T01:22:18.506 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_extents', '--worker_n', '3', '--worker_m', '7'] in time 0:00:00.554511, result: 2023-05-20T01:22:18.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.118+0000 7fe20140e7c0 1 -- 172.21.15.16:0/2808449806 shutdown_connections 2023-05-20T01:22:18.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.118+0000 7fe20140e7c0 1 -- 172.21.15.16:0/2808449806 >> 172.21.15.16:0/2808449806 conn(0x5624b78d3400 msgr2=0x5624b6af9800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:18.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.118+0000 7f5a55a4d7c0 1 -- 172.21.15.16:0/544605894 shutdown_connections 2023-05-20T01:22:18.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.118+0000 7f5a55a4d7c0 1 --2- 172.21.15.16:0/544605894 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559726c95000 0x559726d2f080 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.118+0000 7f5a55a4d7c0 1 --2- 172.21.15.16:0/544605894 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559726c94800 0x559726d2e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.118+0000 7f5a55a4d7c0 1 --2- 172.21.15.16:0/544605894 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559726c94c00 0x559726d2eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.118+0000 7f5a55a4d7c0 1 -- 172.21.15.16:0/544605894 >> 172.21.15.16:0/544605894 conn(0x559726c94400 msgr2=0x559726d71000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:18.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.118+0000 7f5a55a4d7c0 1 -- 172.21.15.16:0/544605894 shutdown_connections 2023-05-20T01:22:18.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.118+0000 7f5a55a4d7c0 1 -- 172.21.15.16:0/544605894 wait complete. 2023-05-20T01:22:18.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.118+0000 7fe20140e7c0 1 -- 172.21.15.16:0/2808449806 shutdown_connections 2023-05-20T01:22:18.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.118+0000 7fe20140e7c0 1 -- 172.21.15.16:0/2808449806 wait complete. 2023-05-20T01:22:18.509 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_extents', '--worker_n', '5', '--worker_m', '7'] in time 0:00:00.556939, result: 2023-05-20T01:22:18.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.118+0000 7fe20140e7c0 1 -- 172.21.15.16:0/652359026 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5624b6a39000 msgr2=0x5624b6ad3080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.118+0000 7fe20140e7c0 1 --2- 172.21.15.16:0/652359026 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5624b6a39000 0x5624b6ad3080 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x5624b7814e10 tx=0x5624b781e480 comp rx=0 tx=0).stop 2023-05-20T01:22:18.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.118+0000 7fe20140e7c0 1 -- 172.21.15.16:0/652359026 shutdown_connections 2023-05-20T01:22:18.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.118+0000 7fe20140e7c0 1 --2- 172.21.15.16:0/652359026 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624b6a39400 0x5624b6ad3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.118+0000 7fe20140e7c0 1 --2- 172.21.15.16:0/652359026 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5624b6a38c00 0x5624b6ad2b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.118+0000 7fe20140e7c0 1 --2- 172.21.15.16:0/652359026 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5624b6a39000 0x5624b6ad3080 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.118+0000 7fe20140e7c0 1 -- 172.21.15.16:0/652359026 >> 172.21.15.16:0/652359026 conn(0x5624b6a38400 msgr2=0x5624b6af9000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:18.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.118+0000 7fe20140e7c0 1 -- 172.21.15.16:0/652359026 shutdown_connections 2023-05-20T01:22:18.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.118+0000 7fe20140e7c0 1 -- 172.21.15.16:0/652359026 wait complete. 2023-05-20T01:22:18.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.119+0000 7f6194186700 1 -- 172.21.15.16:0/3379690421 <== osd.1 v2:172.21.15.16:6808/3860264220 5 ==== osd_op_reply(42 100000001fa.00000000 [call,setxattr (8)] v71'12 uv12 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x564461dd7b00 con 0x564461f2ec00 2023-05-20T01:22:18.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.119+0000 7f61961d87c0 1 -- 172.21.15.16:0/3379690421 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x564461f2ec00 msgr2=0x564461f5e100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.119+0000 7f61961d87c0 1 --2- 172.21.15.16:0/3379690421 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x564461f2ec00 0x564461f5e100 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.119+0000 7f61961d87c0 1 -- 172.21.15.16:0/3379690421 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x564461e21800 msgr2=0x564461f5cb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.119+0000 7f61961d87c0 1 --2- 172.21.15.16:0/3379690421 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x564461e21800 0x564461f5cb00 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.119+0000 7f61961d87c0 1 -- 172.21.15.16:0/3379690421 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x564461e20c00 msgr2=0x564461b8ec00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.119+0000 7f61961d87c0 1 --2- 172.21.15.16:0/3379690421 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x564461e20c00 0x564461b8ec00 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.119+0000 7f61961d87c0 1 -- 172.21.15.16:0/3379690421 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x564461e21400 msgr2=0x564461f5c580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.119+0000 7f61961d87c0 1 --2- 172.21.15.16:0/3379690421 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x564461e21400 0x564461f5c580 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.119+0000 7f61961d87c0 1 -- 172.21.15.16:0/3379690421 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x564461e21000 msgr2=0x564461f5c000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.119+0000 7f61961d87c0 1 --2- 172.21.15.16:0/3379690421 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x564461e21000 0x564461f5c000 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.119+0000 7f61961d87c0 1 -- 172.21.15.16:0/3379690421 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x564461f2e400 msgr2=0x564461b8f700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.119+0000 7f6193985700 1 -- 172.21.15.16:0/3379690421 reap_dead start 2023-05-20T01:22:18.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.119+0000 7f61961d87c0 1 --2- 172.21.15.16:0/3379690421 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x564461f2e400 0x564461b8f700 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.119+0000 7f61961d87c0 1 -- 172.21.15.16:0/3379690421 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x564461f2e800 msgr2=0x564461f5db80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.119+0000 7f61961d87c0 1 --2- 172.21.15.16:0/3379690421 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x564461f2e800 0x564461f5db80 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.119+0000 7f6193985700 1 -- 172.21.15.16:0/3379690421 reap_dead start 2023-05-20T01:22:18.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.119+0000 7f6193985700 1 -- 172.21.15.16:0/3379690421 reap_dead start 2023-05-20T01:22:18.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.119+0000 7f61961d87c0 1 -- 172.21.15.16:0/3379690421 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x564461f2e000 msgr2=0x564461b8f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.119+0000 7f61961d87c0 1 --2- 172.21.15.16:0/3379690421 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x564461f2e000 0x564461b8f180 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x564461e26060 tx=0x564461f42000 comp rx=0 tx=0).stop 2023-05-20T01:22:18.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.119+0000 7f61961d87c0 1 -- 172.21.15.16:0/3379690421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564461e20000 msgr2=0x564461b8db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.119+0000 7f61961d87c0 1 --2- 172.21.15.16:0/3379690421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564461e20000 0x564461b8db80 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x564461e272f0 tx=0x564461c5cde0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.119+0000 7f61961d87c0 1 -- 172.21.15.16:0/3379690421 shutdown_connections 2023-05-20T01:22:18.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.119+0000 7f61961d87c0 1 --2- 172.21.15.16:0/3379690421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564461e20000 0x564461b8db80 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.119+0000 7f61961d87c0 1 --2- 172.21.15.16:0/3379690421 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x564461f2e000 0x564461b8f180 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.119+0000 7f61961d87c0 1 -- 172.21.15.16:0/3379690421 >> 172.21.15.16:0/3379690421 conn(0x564461af2400 msgr2=0x564461bcf800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:18.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.119+0000 7f61961d87c0 1 -- 172.21.15.16:0/3379690421 shutdown_connections 2023-05-20T01:22:18.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.119+0000 7f61961d87c0 1 -- 172.21.15.16:0/3379690421 wait complete. 2023-05-20T01:22:18.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.120+0000 7f61961d87c0 1 -- 172.21.15.16:0/3109423973 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x564461af3000 msgr2=0x564461b8cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.120+0000 7f61961d87c0 1 --2- 172.21.15.16:0/3109423973 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x564461af3000 0x564461b8cb00 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x564461c52d20 tx=0x564461c5cea0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.120+0000 7f61961d87c0 1 -- 172.21.15.16:0/3109423973 shutdown_connections 2023-05-20T01:22:18.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.120+0000 7f61961d87c0 1 --2- 172.21.15.16:0/3109423973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564461af2800 0x564461b8d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.120+0000 7f61961d87c0 1 --2- 172.21.15.16:0/3109423973 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x564461af3000 0x564461b8cb00 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.120+0000 7f61961d87c0 1 --2- 172.21.15.16:0/3109423973 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564461af2c00 0x564461b8c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.120+0000 7f61961d87c0 1 -- 172.21.15.16:0/3109423973 >> 172.21.15.16:0/3109423973 conn(0x564461af3400 msgr2=0x564461bcf000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:18.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.120+0000 7f61961d87c0 1 -- 172.21.15.16:0/3109423973 shutdown_connections 2023-05-20T01:22:18.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.120+0000 7f61961d87c0 1 -- 172.21.15.16:0/3109423973 wait complete. 2023-05-20T01:22:18.521 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_extents', '--worker_n', '4', '--worker_m', '7'] in time 0:00:00.569644, result: 2023-05-20T01:22:18.522 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_extents', '--worker_n', '0', '--worker_m', '7'] in time 0:00:00.572668, result: 2023-05-20T01:22:18.523 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_extents', '--worker_n', '6', '--worker_m', '7'] in time 0:00:00.570697, result: 2023-05-20T01:22:18.524 DEBUG:teuthology.orchestra.run.smithi016:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_inodes --worker_n 0 --worker_m 7 2023-05-20T01:22:18.524 DEBUG:teuthology.orchestra.run.smithi016:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_inodes --worker_n 1 --worker_m 7 2023-05-20T01:22:18.525 DEBUG:teuthology.orchestra.run.smithi016:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_inodes --worker_n 2 --worker_m 7 2023-05-20T01:22:18.526 DEBUG:teuthology.orchestra.run.smithi016:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_inodes --worker_n 3 --worker_m 7 2023-05-20T01:22:18.526 DEBUG:teuthology.orchestra.run.smithi016:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_inodes --worker_n 4 --worker_m 7 2023-05-20T01:22:18.527 DEBUG:teuthology.orchestra.run.smithi016:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_inodes --worker_n 5 --worker_m 7 2023-05-20T01:22:18.528 DEBUG:teuthology.orchestra.run.smithi016:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_inodes --worker_n 6 --worker_m 7 2023-05-20T01:22:18.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.590+0000 7fabbddea7c0 1 -- start start 2023-05-20T01:22:18.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.590+0000 7fabbddea7c0 1 Processor -- start 2023-05-20T01:22:18.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.591+0000 7fabbddea7c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614bda12800 0x5614bdaac580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.591+0000 7fabbddea7c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5614bda12c00 0x5614bdaacb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.591+0000 7fabbddea7c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5614bda13000 0x5614bdaad080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.591+0000 7fabbddea7c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614bda4fcc0 con 0x5614bda12800 2023-05-20T01:22:18.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.591+0000 7fabbddea7c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614bda4f400 con 0x5614bda12c00 2023-05-20T01:22:18.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.591+0000 7fabbddea7c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x5614bda4fe00 con 0x5614bda13000 2023-05-20T01:22:18.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.592+0000 7fabbb597700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5614bda12c00 0x5614bdaacb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.592+0000 7fabbb597700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5614bda12c00 0x5614bdaacb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.16:60136/0 (socket says 172.21.15.16:60136) 2023-05-20T01:22:18.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.592+0000 7fabbb597700 1 -- 172.21.15.16:0/782650732 learned_addr learned my addr 172.21.15.16:0/782650732 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:22:18.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.592+0000 7fabbc599700 1 --2- 172.21.15.16:0/782650732 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5614bda13000 0x5614bdaad080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.592+0000 7fabbbd98700 1 --2- 172.21.15.16:0/782650732 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614bda12800 0x5614bdaac580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.592+0000 7fabbb597700 1 -- 172.21.15.16:0/782650732 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5614bda13000 msgr2=0x5614bdaad080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:18.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.592+0000 7fabbb597700 1 --2- 172.21.15.16:0/782650732 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5614bda13000 0x5614bdaad080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.592+0000 7fabbb597700 1 -- 172.21.15.16:0/782650732 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614bda12800 msgr2=0x5614bdaac580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:18.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.592+0000 7fabbb597700 1 --2- 172.21.15.16:0/782650732 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614bda12800 0x5614bdaac580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.592+0000 7fabbb597700 1 -- 172.21.15.16:0/782650732 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5614bda36d00 con 0x5614bda12c00 2023-05-20T01:22:18.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.592+0000 7fabbc599700 1 --2- 172.21.15.16:0/782650732 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5614bda13000 0x5614bdaad080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-05-20T01:22:18.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.593+0000 7fabbb597700 1 --2- 172.21.15.16:0/782650732 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5614bda12c00 0x5614bdaacb00 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x5614be7eedb0 tx=0x5614be7f8480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=233d024067ca0df1 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.593+0000 7fabb9593700 1 -- 172.21.15.16:0/782650732 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5614bda47600 con 0x5614bda12c00 2023-05-20T01:22:18.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.593+0000 7fabb9593700 1 -- 172.21.15.16:0/782650732 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5614bda64c00 con 0x5614bda12c00 2023-05-20T01:22:18.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.593+0000 7fabb9593700 1 -- 172.21.15.16:0/782650732 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5614bda471e0 con 0x5614bda12c00 2023-05-20T01:22:18.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.593+0000 7fabbddea7c0 1 -- 172.21.15.16:0/782650732 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5614bda376c0 con 0x5614bda12c00 2023-05-20T01:22:18.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.593+0000 7fabbddea7c0 1 -- 172.21.15.16:0/782650732 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x5614bda37520 con 0x5614bda12c00 2023-05-20T01:22:18.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.593+0000 7fabbddea7c0 4 waiting for MDS map... 2023-05-20T01:22:18.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.593+0000 7fabb9593700 1 -- 172.21.15.16:0/782650732 <== mon.1 v2:172.21.15.132:3300/0 4 ==== osd_map(71..71 src has 1..71) v4 ==== 7103+0+0 (secure 0 0 0) 0x5614bda54c40 con 0x5614bda12c00 2023-05-20T01:22:18.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.594+0000 7fabb9593700 1 -- 172.21.15.16:0/782650732 <== mon.1 v2:172.21.15.132:3300/0 5 ==== fsmap(e 42) v1 ==== 3126+0+0 (secure 0 0 0) 0x5614be8a7200 con 0x5614bda12c00 2023-05-20T01:22:18.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.594+0000 7fabbddea7c0 4 Got MDS map 42 2023-05-20T01:22:18.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.594+0000 7fabbddea7c0 4 datascan.main: Using metadata pool output 2023-05-20T01:22:18.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.594+0000 7fabbddea7c0 4 datascan.main: connecting to RADOS... 2023-05-20T01:22:18.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.594+0000 7fabbddea7c0 1 Processor -- start 2023-05-20T01:22:18.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.595+0000 7fabbddea7c0 1 -- start start 2023-05-20T01:22:18.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.595+0000 7fabbddea7c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5614bdd38000 0x5614bdaad600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.595+0000 7fabbddea7c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614bdd38400 0x5614bdaadb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.595+0000 7fabbddea7c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5614bdd38800 0x5614bdaae100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.595+0000 7fabbddea7c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614bda4ea00 con 0x5614bdd38400 2023-05-20T01:22:18.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.595+0000 7fabbddea7c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614bda4eb40 con 0x5614bdd38000 2023-05-20T01:22:18.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.595+0000 7fabbddea7c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x5614bda4fcc0 con 0x5614bdd38800 2023-05-20T01:22:18.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.595+0000 7fabbc599700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5614bdd38000 0x5614bdaad600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.595+0000 7fabbc599700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5614bdd38000 0x5614bdaad600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.16:60142/0 (socket says 172.21.15.16:60142) 2023-05-20T01:22:18.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.595+0000 7fabbc599700 1 -- 172.21.15.16:0/2067995370 learned_addr learned my addr 172.21.15.16:0/2067995370 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:22:18.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.596+0000 7fabbb597700 1 --2- 172.21.15.16:0/2067995370 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5614bdd38800 0x5614bdaae100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.596+0000 7fabbc599700 1 -- 172.21.15.16:0/2067995370 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5614bdd38800 msgr2=0x5614bdaae100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:18.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.596+0000 7fabbc599700 1 --2- 172.21.15.16:0/2067995370 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5614bdd38800 0x5614bdaae100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.596+0000 7fabbc599700 1 -- 172.21.15.16:0/2067995370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614bdd38400 msgr2=0x5614bdaadb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:18.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.596+0000 7fabbc599700 1 --2- 172.21.15.16:0/2067995370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614bdd38400 0x5614bdaadb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.596+0000 7fabbc599700 1 -- 172.21.15.16:0/2067995370 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5614bda37380 con 0x5614bdd38000 2023-05-20T01:22:18.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.596+0000 7fabbc599700 1 --2- 172.21.15.16:0/2067995370 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5614bdd38000 0x5614bdaad600 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x5614bdd3e2d0 tx=0x5614be8b0c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb7cb49f027ef10 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.597+0000 7fabb758f700 1 -- 172.21.15.16:0/2067995370 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5614bda47600 con 0x5614bdd38000 2023-05-20T01:22:18.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.597+0000 7fabb758f700 1 -- 172.21.15.16:0/2067995370 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5614be7fcc00 con 0x5614bdd38000 2023-05-20T01:22:18.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.597+0000 7fabb758f700 1 -- 172.21.15.16:0/2067995370 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5614bda471e0 con 0x5614bdd38000 2023-05-20T01:22:18.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.597+0000 7fabbddea7c0 1 -- 172.21.15.16:0/2067995370 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5614bdd38000 msgr2=0x5614bdaad600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.597+0000 7fabbddea7c0 1 --2- 172.21.15.16:0/2067995370 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5614bdd38000 0x5614bdaad600 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x5614bdd3e2d0 tx=0x5614be8b0c00 comp rx=0 tx=0).stop 2023-05-20T01:22:18.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.599+0000 7fabbddea7c0 1 -- 172.21.15.16:0/2067995370 shutdown_connections 2023-05-20T01:22:18.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.599+0000 7fabbddea7c0 1 --2- 172.21.15.16:0/2067995370 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5614bdd38800 0x5614bdaae100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.599+0000 7fabbddea7c0 1 --2- 172.21.15.16:0/2067995370 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5614bdd38000 0x5614bdaad600 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.599+0000 7fabbddea7c0 1 --2- 172.21.15.16:0/2067995370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614bdd38400 0x5614bdaadb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.599+0000 7fabbddea7c0 1 -- 172.21.15.16:0/2067995370 >> 172.21.15.16:0/2067995370 conn(0x5614bda13c00 msgr2=0x5614bdad3800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:18.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.599+0000 7fabbddea7c0 1 -- 172.21.15.16:0/2067995370 shutdown_connections 2023-05-20T01:22:18.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.599+0000 7fabbddea7c0 1 -- 172.21.15.16:0/2067995370 wait complete. 2023-05-20T01:22:18.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.600+0000 7fabbddea7c0 1 Processor -- start 2023-05-20T01:22:18.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.601+0000 7fabbddea7c0 1 -- start start 2023-05-20T01:22:18.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.602+0000 7fabbddea7c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614bdd38000 0x5614bdaaf180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.602+0000 7fabbddea7c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5614bdd38400 0x5614bdaaec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.602+0000 7fabbddea7c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5614bdd38800 0x5614bdaae680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.602+0000 7fabbddea7c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614bda4ec80 con 0x5614bdd38000 2023-05-20T01:22:18.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.602+0000 7fabbddea7c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614bda4e000 con 0x5614bdd38800 2023-05-20T01:22:18.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.602+0000 7fabbddea7c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x5614bda4f180 con 0x5614bdd38400 2023-05-20T01:22:18.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.602+0000 7fabbb597700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5614bdd38800 0x5614bdaae680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.602+0000 7fabbc599700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614bdd38000 0x5614bdaaf180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.602+0000 7fabbc599700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614bdd38000 0x5614bdaaf180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34912/0 (socket says 172.21.15.16:34912) 2023-05-20T01:22:18.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.602+0000 7fabbc599700 1 -- 172.21.15.16:0/2723465486 learned_addr learned my addr 172.21.15.16:0/2723465486 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:22:18.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.602+0000 7fabbbd98700 1 --2- 172.21.15.16:0/2723465486 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5614bdd38400 0x5614bdaaec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.603+0000 7fabbc599700 1 -- 172.21.15.16:0/2723465486 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5614bdd38400 msgr2=0x5614bdaaec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.603+0000 7fabbc599700 1 --2- 172.21.15.16:0/2723465486 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5614bdd38400 0x5614bdaaec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.603+0000 7fabbc599700 1 -- 172.21.15.16:0/2723465486 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5614bdd38800 msgr2=0x5614bdaae680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.603+0000 7fabbc599700 1 --2- 172.21.15.16:0/2723465486 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5614bdd38800 0x5614bdaae680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.603+0000 7fabbc599700 1 -- 172.21.15.16:0/2723465486 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5614be8a44e0 con 0x5614bdd38000 2023-05-20T01:22:18.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.604+0000 7fabbc599700 1 --2- 172.21.15.16:0/2723465486 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614bdd38000 0x5614bdaaf180 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x5614bdd3ef90 tx=0x5614be7f81e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.602+0000 7f7782aee7c0 1 -- 172.21.15.16:0/1827096106 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563d8a4a2c00 msgr2=0x563d8a53c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.602+0000 7f7782aee7c0 1 --2- 172.21.15.16:0/1827096106 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563d8a4a2c00 0x563d8a53c580 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x563d8a4fd320 tx=0x563d8b236240 comp rx=0 tx=0).stop 2023-05-20T01:22:18.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.602+0000 7f7782aee7c0 1 -- 172.21.15.16:0/1827096106 shutdown_connections 2023-05-20T01:22:18.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.602+0000 7f7782aee7c0 1 --2- 172.21.15.16:0/1827096106 >> v2:172.21.15.16:3300/0 conn(0x563d8a4a3000 0x563d8a53cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.602+0000 7f7782aee7c0 1 --2- 172.21.15.16:0/1827096106 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563d8a4a2c00 0x563d8a53c580 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.602+0000 7f7782aee7c0 1 -- 172.21.15.16:0/1827096106 >> 172.21.15.16:0/1827096106 conn(0x563d8a4a2400 msgr2=0x563d8a562000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:18.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.602+0000 7f7782aee7c0 1 -- 172.21.15.16:0/1827096106 shutdown_connections 2023-05-20T01:22:18.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.602+0000 7f7782aee7c0 1 -- 172.21.15.16:0/1827096106 wait complete. 2023-05-20T01:22:18.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.603+0000 7f70ce3ee7c0 1 -- 172.21.15.16:0/2256577254 shutdown_connections 2023-05-20T01:22:18.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.603+0000 7f70ce3ee7c0 1 -- 172.21.15.16:0/2256577254 wait complete. 2023-05-20T01:22:18.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.604+0000 7fabb5d8c700 1 -- 172.21.15.16:0/2723465486 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5614bda46dc0 con 0x5614bdd38000 2023-05-20T01:22:18.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.604+0000 7fabb5d8c700 1 -- 172.21.15.16:0/2723465486 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5614bda65c80 con 0x5614bdd38000 2023-05-20T01:22:18.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.604+0000 7fabb5d8c700 1 -- 172.21.15.16:0/2723465486 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5614bda47600 con 0x5614bdd38000 2023-05-20T01:22:18.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.604+0000 7fabbddea7c0 1 -- 172.21.15.16:0/2723465486 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5614be8a4340 con 0x5614bdd38000 2023-05-20T01:22:18.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.604+0000 7fabbddea7c0 1 -- 172.21.15.16:0/2723465486 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5614be8a4680 con 0x5614bdd38000 2023-05-20T01:22:18.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.604+0000 7fabbddea7c0 4 datascan.init: resolving metadata pool 9 2023-05-20T01:22:18.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.604+0000 7f70ce3ee7c0 1 -- start start 2023-05-20T01:22:18.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.605+0000 7f70ce3ee7c0 1 Processor -- start 2023-05-20T01:22:18.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.604+0000 7f7782aee7c0 1 -- start start 2023-05-20T01:22:18.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.604+0000 7f7782aee7c0 1 Processor -- start 2023-05-20T01:22:18.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.605+0000 7f7782aee7c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d8a4a2800 0x563d8a53c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.605+0000 7f7782aee7c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563d8a4a2c00 0x563d8a53cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.605+0000 7f7782aee7c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563d8a4a3000 0x563d8a53d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.605+0000 7f7782aee7c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d8a4df400 con 0x563d8a4a2800 2023-05-20T01:22:18.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.605+0000 7f7782aee7c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d8a4dfcc0 con 0x563d8a4a2c00 2023-05-20T01:22:18.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.605+0000 7f7782aee7c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x563d8a4dfe00 con 0x563d8a4a3000 2023-05-20T01:22:18.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.605+0000 7f7780a9c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d8a4a2800 0x563d8a53c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.605+0000 7f7780a9c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d8a4a2800 0x563d8a53c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34936/0 (socket says 172.21.15.16:34936) 2023-05-20T01:22:18.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.605+0000 7f7780a9c700 1 -- 172.21.15.16:0/691035864 learned_addr learned my addr 172.21.15.16:0/691035864 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:22:18.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.605+0000 7f778029b700 1 --2- 172.21.15.16:0/691035864 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563d8a4a2c00 0x563d8a53cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.605+0000 7f778129d700 1 --2- 172.21.15.16:0/691035864 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563d8a4a3000 0x563d8a53d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.606+0000 7f7780a9c700 1 -- 172.21.15.16:0/691035864 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563d8a4a3000 msgr2=0x563d8a53d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:18.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.606+0000 7f7780a9c700 1 --2- 172.21.15.16:0/691035864 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563d8a4a3000 0x563d8a53d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.606+0000 7f7780a9c700 1 -- 172.21.15.16:0/691035864 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563d8a4a2c00 msgr2=0x563d8a53cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:18.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.606+0000 7f7780a9c700 1 --2- 172.21.15.16:0/691035864 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563d8a4a2c00 0x563d8a53cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.606+0000 7f7780a9c700 1 -- 172.21.15.16:0/691035864 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563d8a4c6d00 con 0x563d8a4a2800 2023-05-20T01:22:18.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.606+0000 7f70ce3ee7c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56364168cc00 0x563641726b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.606+0000 7f70ce3ee7c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56364168d000 0x563641727080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.606+0000 7f70ce3ee7c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56364168d400 0x563641727600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.606+0000 7f70ce3ee7c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636416c9cc0 con 0x56364168d400 2023-05-20T01:22:18.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.606+0000 7f70ce3ee7c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636416c9e00 con 0x56364168cc00 2023-05-20T01:22:18.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.606+0000 7f70ce3ee7c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x5636416c9400 con 0x56364168d000 2023-05-20T01:22:18.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.606+0000 7f70cc39c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56364168cc00 0x563641726b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.606+0000 7f70cc39c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56364168cc00 0x563641726b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.16:60196/0 (socket says 172.21.15.16:60196) 2023-05-20T01:22:18.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.606+0000 7f70cc39c700 1 -- 172.21.15.16:0/3195839697 learned_addr learned my addr 172.21.15.16:0/3195839697 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:22:18.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.606+0000 7f70ccb9d700 1 --2- 172.21.15.16:0/3195839697 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56364168d400 0x563641727600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.606+0000 7f90cfff67c0 1 -- 172.21.15.16:0/3872588619 wait complete. 2023-05-20T01:22:18.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.606+0000 7f70cbb9b700 1 --2- 172.21.15.16:0/3195839697 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56364168d000 0x563641727080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.606+0000 7f7780a9c700 1 --2- 172.21.15.16:0/691035864 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d8a4a2800 0x563d8a53c580 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x563d8b21ce10 tx=0x563d8b236480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81322478812a8466 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.606+0000 7f70cc39c700 1 -- 172.21.15.16:0/3195839697 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56364168d000 msgr2=0x563641727080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:18.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.606+0000 7f70cc39c700 1 --2- 172.21.15.16:0/3195839697 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56364168d000 0x563641727080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.606+0000 7f70cc39c700 1 -- 172.21.15.16:0/3195839697 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56364168d400 msgr2=0x563641727600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:18.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.606+0000 7f70cc39c700 1 --2- 172.21.15.16:0/3195839697 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56364168d400 0x563641727600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.606+0000 7f70cc39c700 1 -- 172.21.15.16:0/3195839697 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5636416b0d00 con 0x56364168cc00 2023-05-20T01:22:18.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.606+0000 7f777e297700 1 -- 172.21.15.16:0/691035864 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x563d8a4d7600 con 0x563d8a4a2800 2023-05-20T01:22:18.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.606+0000 7f777e297700 1 -- 172.21.15.16:0/691035864 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x563d8a4f4c00 con 0x563d8a4a2800 2023-05-20T01:22:18.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.606+0000 7f777e297700 1 -- 172.21.15.16:0/691035864 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x563d8a4d71e0 con 0x563d8a4a2800 2023-05-20T01:22:18.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.606+0000 7f7782aee7c0 1 -- 172.21.15.16:0/691035864 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563d8a4c76c0 con 0x563d8a4a2800 2023-05-20T01:22:18.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.606+0000 7f7782aee7c0 1 -- 172.21.15.16:0/691035864 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x563d8a4c7520 con 0x563d8a4a2800 2023-05-20T01:22:18.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.606+0000 7f7782aee7c0 4 waiting for MDS map... 2023-05-20T01:22:18.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.607+0000 7f90cfff67c0 1 -- start start 2023-05-20T01:22:18.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.607+0000 7f70cc39c700 1 --2- 172.21.15.16:0/3195839697 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56364168cc00 0x563641726b00 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x563642468d50 tx=0x563642474000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66621876cae16704 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.607+0000 7f90cfff67c0 1 Processor -- start 2023-05-20T01:22:18.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.607+0000 7f70c9b97700 1 -- 172.21.15.16:0/3195839697 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5636416c1080 con 0x56364168cc00 2023-05-20T01:22:18.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.607+0000 7f70c9b97700 1 -- 172.21.15.16:0/3195839697 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5636416df980 con 0x56364168cc00 2023-05-20T01:22:18.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.607+0000 7f70c9b97700 1 -- 172.21.15.16:0/3195839697 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5636416c1600 con 0x56364168cc00 2023-05-20T01:22:18.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.607+0000 7f70ce3ee7c0 1 -- 172.21.15.16:0/3195839697 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5636416b16c0 con 0x56364168cc00 2023-05-20T01:22:18.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.607+0000 7f70ce3ee7c0 1 -- 172.21.15.16:0/3195839697 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x5636416b1520 con 0x56364168cc00 2023-05-20T01:22:18.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.607+0000 7f70ce3ee7c0 4 waiting for MDS map... 2023-05-20T01:22:18.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.607+0000 7f37780767c0 1 -- start start 2023-05-20T01:22:18.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.607+0000 7f37780767c0 1 Processor -- start 2023-05-20T01:22:18.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.608+0000 7fabb5d8c700 1 -- 172.21.15.16:0/2723465486 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x5614bda74840 con 0x5614bdd38000 2023-05-20T01:22:18.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.608+0000 7f90cfff67c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556b16dba800 0x556b16e54580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.608+0000 7f90cfff67c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556b16dbac00 0x556b16e54b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.608+0000 7f90cfff67c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556b16dbb000 0x556b16e55080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.608+0000 7f90cfff67c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b16df7400 con 0x556b16dbac00 2023-05-20T01:22:18.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.608+0000 7f90cfff67c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b16df7cc0 con 0x556b16dbb000 2023-05-20T01:22:18.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.608+0000 7f90cfff67c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x556b16df6a00 con 0x556b16dba800 2023-05-20T01:22:18.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.608+0000 7f777e297700 1 -- 172.21.15.16:0/691035864 <== mon.0 v2:172.21.15.16:3300/0 4 ==== osd_map(71..71 src has 1..71) v4 ==== 7103+0+0 (secure 0 0 0) 0x563d8a4e4c40 con 0x563d8a4a2800 2023-05-20T01:22:18.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.608+0000 7fa3b53a97c0 1 -- start start 2023-05-20T01:22:18.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.608+0000 7fa3b53a97c0 1 Processor -- start 2023-05-20T01:22:18.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.608+0000 7f70c9b97700 1 -- 172.21.15.16:0/3195839697 <== mon.1 v2:172.21.15.132:3300/0 4 ==== osd_map(71..71 src has 1..71) v4 ==== 7103+0+0 (secure 0 0 0) 0x5636416cec40 con 0x56364168cc00 2023-05-20T01:22:18.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.608+0000 7f777e297700 1 -- 172.21.15.16:0/691035864 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 42) v1 ==== 3126+0+0 (secure 0 0 0) 0x563d8b348000 con 0x563d8a4a2800 2023-05-20T01:22:18.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.608+0000 7fabb5d8c700 1 --2- 172.21.15.16:0/2723465486 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x5614bdd39c00 0x5614bdaaf700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.608+0000 7fabb5d8c700 1 -- 172.21.15.16:0/2723465486 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 7103+0+0 (secure 0 0 0) 0x5614bda54a80 con 0x5614bdd38000 2023-05-20T01:22:18.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.608+0000 7f37780767c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce83b85000 0x55ce83c1eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.608+0000 7f37780767c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ce83b84c00 0x55ce83c1e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.608+0000 7f37780767c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ce83b84800 0x55ce83c1f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.608+0000 7f37780767c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce83bc1cc0 con 0x55ce83b85000 2023-05-20T01:22:18.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.608+0000 7f37780767c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce83bc1e00 con 0x55ce83b84800 2023-05-20T01:22:18.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.608+0000 7f90ce7a5700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556b16dbb000 0x556b16e55080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.608+0000 7f37780767c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ce83bc1400 con 0x55ce83b84c00 2023-05-20T01:22:18.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.609+0000 7f7782aee7c0 4 Got MDS map 42 2023-05-20T01:22:18.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.609+0000 7fabbddea7c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-05-20T01:22:18.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.609+0000 7fabbddea7c0 4 datascan.main: data pool 'cephfs_data' has ID 10 2023-05-20T01:22:18.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.609+0000 7fabbddea7c0 4 datascan.main: opening data pool 'cephfs_data' 2023-05-20T01:22:18.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.609+0000 7fabbddea7c0 1 --2- 172.21.15.16:0/2723465486 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x5614bdd38c00 0x5614bdaae100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.609+0000 7fabbddea7c0 1 -- 172.21.15.16:0/2723465486 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:1 9.14 9:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x5614be7fa700 con 0x5614bdd38c00 2023-05-20T01:22:18.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.609+0000 7fabbbd98700 1 --2- 172.21.15.16:0/2723465486 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x5614bdd39c00 0x5614bdaaf700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.608+0000 7f70c9b97700 1 -- 172.21.15.16:0/3195839697 <== mon.1 v2:172.21.15.132:3300/0 5 ==== fsmap(e 42) v1 ==== 3126+0+0 (secure 0 0 0) 0x5636416d2f00 con 0x56364168cc00 2023-05-20T01:22:18.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.609+0000 7f7782aee7c0 4 datascan.main: Using metadata pool output 2023-05-20T01:22:18.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.609+0000 7f7782aee7c0 4 datascan.main: connecting to RADOS... 2023-05-20T01:22:18.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.609+0000 7f90cdfa4700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556b16dba800 0x556b16e54580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.609+0000 7f90cdfa4700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556b16dba800 0x556b16e54580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.16:34990/0 (socket says 172.21.15.16:34990) 2023-05-20T01:22:18.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.609+0000 7f90cdfa4700 1 -- 172.21.15.16:0/2063914123 learned_addr learned my addr 172.21.15.16:0/2063914123 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:22:18.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.609+0000 7f90ce7a5700 1 --2- 172.21.15.16:0/2063914123 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556b16dbb000 0x556b16e55080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.16:60204/0 (socket says 172.21.15.16:60204) 2023-05-20T01:22:18.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.609+0000 7f90cd7a3700 1 --2- 172.21.15.16:0/2063914123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556b16dbac00 0x556b16e54b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.609+0000 7f3776825700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ce83b84800 0x55ce83c1f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.609+0000 7f3775823700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ce83b84c00 0x55ce83c1e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.609+0000 7f3775823700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ce83b84c00 0x55ce83c1e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.16:34998/0 (socket says 172.21.15.16:34998) 2023-05-20T01:22:18.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.609+0000 7f3775823700 1 -- 172.21.15.16:0/3796352817 learned_addr learned my addr 172.21.15.16:0/3796352817 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:22:18.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.609+0000 7f3776024700 1 --2- 172.21.15.16:0/3796352817 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce83b85000 0x55ce83c1eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.609+0000 7f3776024700 1 -- 172.21.15.16:0/3796352817 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ce83b84c00 msgr2=0x55ce83c1e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:18.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.609+0000 7f3776024700 1 --2- 172.21.15.16:0/3796352817 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ce83b84c00 0x55ce83c1e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.609+0000 7f3776024700 1 -- 172.21.15.16:0/3796352817 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ce83b84800 msgr2=0x55ce83c1f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:18.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.609+0000 7f3776024700 1 --2- 172.21.15.16:0/3796352817 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ce83b84800 0x55ce83c1f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.609+0000 7f3776024700 1 -- 172.21.15.16:0/3796352817 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ce83ba9380 con 0x55ce83b85000 2023-05-20T01:22:18.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.609+0000 7f7782aee7c0 1 Processor -- start 2023-05-20T01:22:18.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.609+0000 7f3775823700 1 --2- 172.21.15.16:0/3796352817 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ce83b84c00 0x55ce83c1e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-05-20T01:22:18.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.609+0000 7f3776825700 1 --2- 172.21.15.16:0/3796352817 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ce83b84800 0x55ce83c1f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:22:18.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.610+0000 7fabbbd98700 1 --2- 172.21.15.16:0/2723465486 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x5614bdd39c00 0x5614bdaaf700 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x5614bda6d620 tx=0x5614be7f8000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.610+0000 7fabbb597700 1 --2- 172.21.15.16:0/2723465486 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x5614bdd38c00 0x5614bdaae100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.610+0000 7f3776024700 1 --2- 172.21.15.16:0/3796352817 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce83b85000 0x55ce83c1eb00 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x55ce8494e3c0 tx=0x55ce8496c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61b0ef99c7c27b52 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.610+0000 7f377381f700 1 -- 172.21.15.16:0/3796352817 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55ce83bb9080 con 0x55ce83b85000 2023-05-20T01:22:18.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.610+0000 7f377381f700 1 -- 172.21.15.16:0/3796352817 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ce83bd6c00 con 0x55ce83b85000 2023-05-20T01:22:18.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.610+0000 7f377381f700 1 -- 172.21.15.16:0/3796352817 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55ce83bb9600 con 0x55ce83b85000 2023-05-20T01:22:18.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.610+0000 7f37780767c0 1 -- 172.21.15.16:0/3796352817 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ce83ba9d40 con 0x55ce83b85000 2023-05-20T01:22:18.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.610+0000 7f37780767c0 1 -- 172.21.15.16:0/3796352817 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55ce83ba91e0 con 0x55ce83b85000 2023-05-20T01:22:18.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.610+0000 7f37780767c0 4 waiting for MDS map... 2023-05-20T01:22:18.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.609+0000 7fa3b53a97c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x555b4a4fc800 0x555b4a596580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.609+0000 7fa3b53a97c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555b4a4fd000 0x555b4a596b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.609+0000 7fa3b53a97c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b4a4fd400 0x555b4a597080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.609+0000 7fa3b53a97c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b4a539cc0 con 0x555b4a4fd400 2023-05-20T01:22:18.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.609+0000 7fa3b53a97c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b4a539400 con 0x555b4a4fd000 2023-05-20T01:22:18.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.609+0000 7fa3b53a97c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x555b4a538a00 con 0x555b4a4fc800 2023-05-20T01:22:18.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.610+0000 7fa3b2b56700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555b4a4fd000 0x555b4a596b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.610+0000 7fa3b2b56700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555b4a4fd000 0x555b4a596b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.16:60220/0 (socket says 172.21.15.16:60220) 2023-05-20T01:22:18.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.610+0000 7fa3b2b56700 1 -- 172.21.15.16:0/128094146 learned_addr learned my addr 172.21.15.16:0/128094146 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:22:18.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.610+0000 7fa3b3357700 1 --2- 172.21.15.16:0/128094146 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x555b4a4fc800 0x555b4a596580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.610+0000 7fa3b2b56700 1 -- 172.21.15.16:0/128094146 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x555b4a4fc800 msgr2=0x555b4a596580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:18.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.610+0000 7fa3b2b56700 1 --2- 172.21.15.16:0/128094146 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x555b4a4fc800 0x555b4a596580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.610+0000 7fa3b2b56700 1 -- 172.21.15.16:0/128094146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b4a4fd400 msgr2=0x555b4a597080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:18.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.610+0000 7fa3b2b56700 1 --2- 172.21.15.16:0/128094146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b4a4fd400 0x555b4a597080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.610+0000 7fa3b2b56700 1 -- 172.21.15.16:0/128094146 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555b4a520d00 con 0x555b4a4fd000 2023-05-20T01:22:18.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.610+0000 7f3b5beef7c0 1 -- start start 2023-05-20T01:22:18.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.609+0000 7f90cdfa4700 1 -- 172.21.15.16:0/2063914123 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556b16dbb000 msgr2=0x556b16e55080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:18.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.609+0000 7f90cdfa4700 1 --2- 172.21.15.16:0/2063914123 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556b16dbb000 0x556b16e55080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.609+0000 7f90cdfa4700 1 -- 172.21.15.16:0/2063914123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556b16dbac00 msgr2=0x556b16e54b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:18.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.609+0000 7f90cdfa4700 1 --2- 172.21.15.16:0/2063914123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556b16dbac00 0x556b16e54b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.610+0000 7f3b5beef7c0 1 Processor -- start 2023-05-20T01:22:18.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.609+0000 7f90cdfa4700 1 -- 172.21.15.16:0/2063914123 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556b16dded00 con 0x556b16dba800 2023-05-20T01:22:18.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.610+0000 7fa3b3357700 1 --2- 172.21.15.16:0/128094146 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x555b4a4fc800 0x555b4a596580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-05-20T01:22:18.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.610+0000 7f7782aee7c0 1 -- start start 2023-05-20T01:22:18.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.611+0000 7f7782aee7c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563d8a4a3c00 0x563d8a53d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.611+0000 7f7782aee7c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563d8a7f0000 0x563d8a53db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.611+0000 7f7782aee7c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d8a7f0400 0x563d8a53e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.611+0000 7f7782aee7c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d8a4dea00 con 0x563d8a7f0400 2023-05-20T01:22:18.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.611+0000 7f7782aee7c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d8a4dfcc0 con 0x563d8a7f0000 2023-05-20T01:22:18.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.611+0000 7f7782aee7c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x563d8a4deb40 con 0x563d8a4a3c00 2023-05-20T01:22:18.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.611+0000 7f778129d700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563d8a4a3c00 0x563d8a53d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.611+0000 7f778029b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d8a7f0400 0x563d8a53e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.611+0000 7f778129d700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563d8a4a3c00 0x563d8a53d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.16:35008/0 (socket says 172.21.15.16:35008) 2023-05-20T01:22:18.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.611+0000 7f778129d700 1 -- 172.21.15.16:0/3004817534 learned_addr learned my addr 172.21.15.16:0/3004817534 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:22:18.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.611+0000 7f7780a9c700 1 --2- 172.21.15.16:0/3004817534 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563d8a7f0000 0x563d8a53db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.611+0000 7f377381f700 1 -- 172.21.15.16:0/3796352817 <== mon.0 v2:172.21.15.16:3300/0 4 ==== osd_map(71..71 src has 1..71) v4 ==== 7103+0+0 (secure 0 0 0) 0x55ce83bc6c40 con 0x55ce83b85000 2023-05-20T01:22:18.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.611+0000 7fabbb597700 1 --2- 172.21.15.16:0/2723465486 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x5614bdd38c00 0x5614bdaae100 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.611+0000 7f778029b700 1 -- 172.21.15.16:0/3004817534 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563d8a4a3c00 msgr2=0x563d8a53d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:18.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.611+0000 7f778029b700 1 --2- 172.21.15.16:0/3004817534 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563d8a4a3c00 0x563d8a53d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.611+0000 7f778029b700 1 -- 172.21.15.16:0/3004817534 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563d8a7f0000 msgr2=0x563d8a53db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:18.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.611+0000 7f778029b700 1 --2- 172.21.15.16:0/3004817534 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563d8a7f0000 0x563d8a53db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.611+0000 7f778029b700 1 -- 172.21.15.16:0/3004817534 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563d8a4c7380 con 0x563d8a7f0400 2023-05-20T01:22:18.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.611+0000 7f377381f700 1 -- 172.21.15.16:0/3796352817 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 42) v1 ==== 3126+0+0 (secure 0 0 0) 0x55ce83bcaf00 con 0x55ce83b85000 2023-05-20T01:22:18.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.611+0000 7f37780767c0 4 Got MDS map 42 2023-05-20T01:22:18.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.611+0000 7f37780767c0 4 datascan.main: Using metadata pool output 2023-05-20T01:22:18.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.611+0000 7f37780767c0 4 datascan.main: connecting to RADOS... 2023-05-20T01:22:18.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.611+0000 7f778029b700 1 --2- 172.21.15.16:0/3004817534 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d8a7f0400 0x563d8a53e100 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x563d8a4fcbd0 tx=0x563d8b236060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=312928fbc5df8292 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.612+0000 7fabbb597700 1 -- 172.21.15.16:0/2723465486 <== osd.1 v2:172.21.15.16:6808/3860264220 1 ==== osd_op_reply(1 1.00000000.inode [stat out=16b] v0'0 uv133 ondisk = 0) v8 ==== 160+0+16 (crc 0 0 0) 0x5614bdd37440 con 0x5614bdd38c00 2023-05-20T01:22:18.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.612+0000 7fabbddea7c0 1 -- 172.21.15.16:0/2723465486 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:2 9.1f 9:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x5614be7faa80 con 0x5614bdd38c00 2023-05-20T01:22:18.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.612+0000 7fabbb597700 1 -- 172.21.15.16:0/2723465486 <== osd.1 v2:172.21.15.16:6808/3860264220 2 ==== osd_op_reply(2 100.00000000.inode [stat out=16b] v0'0 uv10 ondisk = 0) v8 ==== 162+0+16 (crc 0 0 0) 0x5614bdd37440 con 0x5614bdd38c00 2023-05-20T01:22:18.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.612+0000 7fabbddea7c0 1 --2- 172.21.15.16:0/2723465486 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x5614bdd39000 0x5614bdaadb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.612+0000 7fabbddea7c0 1 -- 172.21.15.16:0/2723465486 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:3 10.0 10:00000000::::head [pgnls-filter start_epoch 71 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x5614be7fae00 con 0x5614bdd39000 2023-05-20T01:22:18.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.612+0000 7f37780767c0 1 Processor -- start 2023-05-20T01:22:18.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.612+0000 7f37780767c0 1 -- start start 2023-05-20T01:22:18.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.612+0000 7fabbc599700 1 --2- 172.21.15.16:0/2723465486 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x5614bdd39000 0x5614bdaadb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.612+0000 7f3b5beef7c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56013b31d000 0x56013b3b6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.612+0000 7f3b5beef7c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56013b31cc00 0x56013b3b6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.612+0000 7f3b5beef7c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56013b31c800 0x56013b3b7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.612+0000 7f3b5beef7c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x56013b359cc0 con 0x56013b31d000 2023-05-20T01:22:18.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.612+0000 7f3b5beef7c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x56013b359400 con 0x56013b31c800 2023-05-20T01:22:18.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.612+0000 7f3b5beef7c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x56013b358a00 con 0x56013b31cc00 2023-05-20T01:22:18.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.611+0000 7f90cdfa4700 1 --2- 172.21.15.16:0/2063914123 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556b16dba800 0x556b16e54580 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x556b17b96ed0 tx=0x556b17b9a480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da04d7ff7ff19e7a server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.612+0000 7f3b59e9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56013b31d000 0x56013b3b6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.612+0000 7f3b59e9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56013b31d000 0x56013b3b6580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34978/0 (socket says 172.21.15.16:34978) 2023-05-20T01:22:18.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.612+0000 7f3b59e9d700 1 -- 172.21.15.16:0/1833980693 learned_addr learned my addr 172.21.15.16:0/1833980693 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:22:18.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.612+0000 7f3b5969c700 1 --2- 172.21.15.16:0/1833980693 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56013b31cc00 0x56013b3b6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.613+0000 7f777c293700 1 -- 172.21.15.16:0/3004817534 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x563d8a4d7080 con 0x563d8a7f0400 2023-05-20T01:22:18.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.613+0000 7f777c293700 1 -- 172.21.15.16:0/3004817534 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x563d8a796c00 con 0x563d8a7f0400 2023-05-20T01:22:18.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.613+0000 7f777c293700 1 -- 172.21.15.16:0/3004817534 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x563d8a4d7600 con 0x563d8a7f0400 2023-05-20T01:22:18.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.614+0000 7f7782aee7c0 1 -- 172.21.15.16:0/3004817534 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d8a7f0400 msgr2=0x563d8a53e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.614+0000 7f7782aee7c0 1 --2- 172.21.15.16:0/3004817534 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d8a7f0400 0x563d8a53e100 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x563d8a4fcbd0 tx=0x563d8b236060 comp rx=0 tx=0).stop 2023-05-20T01:22:18.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.614+0000 7f7782aee7c0 1 -- 172.21.15.16:0/3004817534 shutdown_connections 2023-05-20T01:22:18.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.614+0000 7f7782aee7c0 1 --2- 172.21.15.16:0/3004817534 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d8a7f0400 0x563d8a53e100 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.614+0000 7f7782aee7c0 1 --2- 172.21.15.16:0/3004817534 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563d8a4a3c00 0x563d8a53d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.614+0000 7f7782aee7c0 1 --2- 172.21.15.16:0/3004817534 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563d8a7f0000 0x563d8a53db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.614+0000 7f7782aee7c0 1 -- 172.21.15.16:0/3004817534 >> 172.21.15.16:0/3004817534 conn(0x563d8a4a3800 msgr2=0x563d8a563800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:18.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.613+0000 7f90cb79f700 1 -- 172.21.15.16:0/2063914123 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x556b16def600 con 0x556b16dba800 2023-05-20T01:22:18.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.613+0000 7f90cb79f700 1 -- 172.21.15.16:0/2063914123 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556b16e0d980 con 0x556b16dba800 2023-05-20T01:22:18.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.613+0000 7f90cb79f700 1 -- 172.21.15.16:0/2063914123 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x556b16def1e0 con 0x556b16dba800 2023-05-20T01:22:18.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.613+0000 7f90cfff67c0 1 -- 172.21.15.16:0/2063914123 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556b16ddf6c0 con 0x556b16dba800 2023-05-20T01:22:18.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.613+0000 7f90cfff67c0 1 -- 172.21.15.16:0/2063914123 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({fsmap=0}) v3 -- 0x556b16ddf520 con 0x556b16dba800 2023-05-20T01:22:18.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.613+0000 7f90cfff67c0 4 waiting for MDS map... 2023-05-20T01:22:18.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.613+0000 7f37780767c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ce83eae000 0x55ce83c1f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.613+0000 7f37780767c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce83eae400 0x55ce83c1fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.613+0000 7f37780767c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ce83eae800 0x55ce83c20100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.613+0000 7f37780767c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce83bc0a00 con 0x55ce83eae400 2023-05-20T01:22:18.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.613+0000 7f37780767c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce83bc1e00 con 0x55ce83eae000 2023-05-20T01:22:18.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.613+0000 7f37780767c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ce83bc1cc0 con 0x55ce83eae800 2023-05-20T01:22:18.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.613+0000 7f3776825700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ce83eae000 0x55ce83c1f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.613+0000 7f3776825700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ce83eae000 0x55ce83c1f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.16:60238/0 (socket says 172.21.15.16:60238) 2023-05-20T01:22:18.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.613+0000 7f3776825700 1 -- 172.21.15.16:0/4201969874 learned_addr learned my addr 172.21.15.16:0/4201969874 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:22:18.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.613+0000 7f3775823700 1 --2- 172.21.15.16:0/4201969874 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ce83eae800 0x55ce83c20100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.614+0000 7f3776024700 1 --2- 172.21.15.16:0/4201969874 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce83eae400 0x55ce83c1fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.614+0000 7f3776825700 1 -- 172.21.15.16:0/4201969874 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ce83eae800 msgr2=0x55ce83c20100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:18.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.614+0000 7f3776825700 1 --2- 172.21.15.16:0/4201969874 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ce83eae800 0x55ce83c20100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.614+0000 7f3776825700 1 -- 172.21.15.16:0/4201969874 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce83eae400 msgr2=0x55ce83c1fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:18.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.614+0000 7f3776825700 1 --2- 172.21.15.16:0/4201969874 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce83eae400 0x55ce83c1fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.614+0000 7f3776825700 1 -- 172.21.15.16:0/4201969874 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ce83ba9380 con 0x55ce83eae000 2023-05-20T01:22:18.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.614+0000 7f3776825700 1 --2- 172.21.15.16:0/4201969874 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ce83eae000 0x55ce83c1f600 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x55ce83eb82a0 tx=0x55ce84a2ac00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb451ee400cd6dc8 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.614+0000 7f377181b700 1 -- 172.21.15.16:0/4201969874 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55ce83bb9080 con 0x55ce83eae000 2023-05-20T01:22:18.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.614+0000 7f377181b700 1 -- 172.21.15.16:0/4201969874 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ce83bd6480 con 0x55ce83eae000 2023-05-20T01:22:18.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.614+0000 7f377181b700 1 -- 172.21.15.16:0/4201969874 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55ce83bb9600 con 0x55ce83eae000 2023-05-20T01:22:18.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.613+0000 7f3b5a69e700 1 --2- 172.21.15.16:0/1833980693 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56013b31c800 0x56013b3b7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.613+0000 7f3b59e9d700 1 -- 172.21.15.16:0/1833980693 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56013b31cc00 msgr2=0x56013b3b6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:18.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.613+0000 7f3b59e9d700 1 --2- 172.21.15.16:0/1833980693 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56013b31cc00 0x56013b3b6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.613+0000 7f3b59e9d700 1 -- 172.21.15.16:0/1833980693 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56013b31c800 msgr2=0x56013b3b7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:18.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.613+0000 7f3b59e9d700 1 --2- 172.21.15.16:0/1833980693 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56013b31c800 0x56013b3b7080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.613+0000 7f3b59e9d700 1 -- 172.21.15.16:0/1833980693 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56013b3404e0 con 0x56013b31d000 2023-05-20T01:22:18.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.613+0000 7f3b59e9d700 1 --2- 172.21.15.16:0/1833980693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56013b31d000 0x56013b3b6580 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x56013c0e8d20 tx=0x56013c0fe360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c701b6b967031abe server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.613+0000 7f3b57698700 1 -- 172.21.15.16:0/1833980693 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56013b351600 con 0x56013b31d000 2023-05-20T01:22:18.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.613+0000 7f3b57698700 1 -- 172.21.15.16:0/1833980693 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56013b36f980 con 0x56013b31d000 2023-05-20T01:22:18.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.613+0000 7f3b57698700 1 -- 172.21.15.16:0/1833980693 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56013b3511e0 con 0x56013b31d000 2023-05-20T01:22:18.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.613+0000 7f3b5beef7c0 1 -- 172.21.15.16:0/1833980693 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56013b341520 con 0x56013b31d000 2023-05-20T01:22:18.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.613+0000 7f3b5beef7c0 1 -- 172.21.15.16:0/1833980693 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x56013b341380 con 0x56013b31d000 2023-05-20T01:22:18.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.613+0000 7f3b5beef7c0 4 waiting for MDS map... 2023-05-20T01:22:18.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.614+0000 7f3b57698700 1 -- 172.21.15.16:0/1833980693 <== mon.0 v2:172.21.15.16:3300/0 4 ==== osd_map(71..71 src has 1..71) v4 ==== 7103+0+0 (secure 0 0 0) 0x56013b35ec40 con 0x56013b31d000 2023-05-20T01:22:18.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.614+0000 7f3b57698700 1 -- 172.21.15.16:0/1833980693 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 42) v1 ==== 3126+0+0 (secure 0 0 0) 0x56013c106000 con 0x56013b31d000 2023-05-20T01:22:18.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.614+0000 7f3b5beef7c0 4 Got MDS map 42 2023-05-20T01:22:18.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.614+0000 7f3b5beef7c0 4 datascan.main: Using metadata pool output 2023-05-20T01:22:18.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.614+0000 7f3b5beef7c0 4 datascan.main: connecting to RADOS... 2023-05-20T01:22:18.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.615+0000 7f90cb79f700 1 -- 172.21.15.16:0/2063914123 <== mon.2 v2:172.21.15.132:3301/0 4 ==== osd_map(71..71 src has 1..71) v4 ==== 7103+0+0 (secure 0 0 0) 0x556b16dfcc40 con 0x556b16dba800 2023-05-20T01:22:18.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.615+0000 7fabbc599700 1 --2- 172.21.15.16:0/2723465486 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x5614bdd39000 0x5614bdaadb80 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.615+0000 7f3b5beef7c0 1 Processor -- start 2023-05-20T01:22:18.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.613+0000 7f70ce3ee7c0 4 Got MDS map 42 2023-05-20T01:22:18.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.615+0000 7f37780767c0 1 -- 172.21.15.16:0/4201969874 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ce83eae000 msgr2=0x55ce83c1f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.615+0000 7f37780767c0 1 --2- 172.21.15.16:0/4201969874 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ce83eae000 0x55ce83c1f600 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x55ce83eb82a0 tx=0x55ce84a2ac00 comp rx=0 tx=0).stop 2023-05-20T01:22:18.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.613+0000 7f70ce3ee7c0 4 datascan.main: Using metadata pool output 2023-05-20T01:22:18.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.613+0000 7f70ce3ee7c0 4 datascan.main: connecting to RADOS... 2023-05-20T01:22:18.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.615+0000 7f7782aee7c0 1 -- 172.21.15.16:0/3004817534 shutdown_connections 2023-05-20T01:22:18.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.613+0000 7f70ce3ee7c0 1 Processor -- start 2023-05-20T01:22:18.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.614+0000 7f70ce3ee7c0 1 -- start start 2023-05-20T01:22:18.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.614+0000 7f70ce3ee7c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56364168dc00 0x563641727b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.614+0000 7f70ce3ee7c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636419b4000 0x563641728100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.614+0000 7f70ce3ee7c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5636419b4400 0x563641728680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.615+0000 7f70ce3ee7c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636416c8a00 con 0x5636419b4000 2023-05-20T01:22:18.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.615+0000 7f70ce3ee7c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636416c8b40 con 0x5636419b4400 2023-05-20T01:22:18.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.615+0000 7f70ce3ee7c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x5636416c9cc0 con 0x56364168dc00 2023-05-20T01:22:18.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.615+0000 7f70cbb9b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5636419b4400 0x563641728680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.615+0000 7f70cbb9b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5636419b4400 0x563641728680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.16:60244/0 (socket says 172.21.15.16:60244) 2023-05-20T01:22:18.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.615+0000 7f70cbb9b700 1 -- 172.21.15.16:0/766410834 learned_addr learned my addr 172.21.15.16:0/766410834 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:22:18.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.615+0000 7f37780767c0 1 -- 172.21.15.16:0/4201969874 shutdown_connections 2023-05-20T01:22:18.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.615+0000 7f37780767c0 1 --2- 172.21.15.16:0/4201969874 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ce83eae800 0x55ce83c20100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.615+0000 7f37780767c0 1 --2- 172.21.15.16:0/4201969874 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ce83eae000 0x55ce83c1f600 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.615+0000 7f37780767c0 1 --2- 172.21.15.16:0/4201969874 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce83eae400 0x55ce83c1fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.615+0000 7f37780767c0 1 -- 172.21.15.16:0/4201969874 >> 172.21.15.16:0/4201969874 conn(0x55ce83b84400 msgr2=0x55ce83c45800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:18.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.615+0000 7f70cc39c700 1 --2- 172.21.15.16:0/766410834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636419b4000 0x563641728100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.615+0000 7f3b5beef7c0 1 -- start start 2023-05-20T01:22:18.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.615+0000 7f70ccb9d700 1 --2- 172.21.15.16:0/766410834 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56364168dc00 0x563641727b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.615+0000 7f70cc39c700 1 -- 172.21.15.16:0/766410834 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56364168dc00 msgr2=0x563641727b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:18.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.615+0000 7f90cb79f700 1 -- 172.21.15.16:0/2063914123 <== mon.2 v2:172.21.15.132:3301/0 5 ==== fsmap(e 42) v1 ==== 3126+0+0 (secure 0 0 0) 0x556b17c47200 con 0x556b16dba800 2023-05-20T01:22:18.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.615+0000 7f70cc39c700 1 --2- 172.21.15.16:0/766410834 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56364168dc00 0x563641727b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.615+0000 7f70cc39c700 1 -- 172.21.15.16:0/766410834 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5636419b4400 msgr2=0x563641728680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:18.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.615+0000 7f70cc39c700 1 --2- 172.21.15.16:0/766410834 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5636419b4400 0x563641728680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.615+0000 7f70cc39c700 1 -- 172.21.15.16:0/766410834 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5636416b1380 con 0x5636419b4000 2023-05-20T01:22:18.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.615+0000 7f70ccb9d700 1 --2- 172.21.15.16:0/766410834 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56364168dc00 0x563641727b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:22:18.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.615+0000 7fabbc599700 1 -- 172.21.15.16:0/2723465486 <== osd.6 v2:172.21.15.132:6801/727581123 1 ==== osd_op_reply(3 [pgnls-filter start_epoch 71 out=81b] v71'12 uv12 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5614bdd37680 con 0x5614bdd39000 2023-05-20T01:22:18.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.615+0000 7fabbddea7c0 1 -- 172.21.15.16:0/2723465486 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:4 10.0 10:00000000::::head [pgnls-filter start_epoch 71 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x5614be7fb180 con 0x5614bdd39000 2023-05-20T01:22:18.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.615+0000 7f7782aee7c0 1 -- 172.21.15.16:0/3004817534 wait complete. 2023-05-20T01:22:18.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.616+0000 7f70cc39c700 1 --2- 172.21.15.16:0/766410834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636419b4000 0x563641728100 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x563642468e40 tx=0x563642474060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eabb1973981f38cc server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.616+0000 7f70c7b93700 1 -- 172.21.15.16:0/766410834 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5636416c11e0 con 0x5636419b4000 2023-05-20T01:22:18.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.616+0000 7fabbc599700 1 -- 172.21.15.16:0/2723465486 <== osd.6 v2:172.21.15.132:6801/727581123 2 ==== osd_op_reply(4 [pgnls-filter start_epoch 71 out=81b] v71'12 uv12 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5614bdd37680 con 0x5614bdd39000 2023-05-20T01:22:18.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.616+0000 7f70c7b93700 1 -- 172.21.15.16:0/766410834 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56364252a300 con 0x5636419b4000 2023-05-20T01:22:18.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.616+0000 7f70c7b93700 1 -- 172.21.15.16:0/766410834 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5636416c0f20 con 0x5636419b4000 2023-05-20T01:22:18.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.616+0000 7f7782aee7c0 1 Processor -- start 2023-05-20T01:22:18.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.616+0000 7fabbddea7c0 20 datascan.forall_objects: OSD matched oid 10000000206.00000000 2023-05-20T01:22:18.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.616+0000 7fabbddea7c0 10 datascan.operator(): handling object 10000000206.0 2023-05-20T01:22:18.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.616+0000 7f3b5beef7c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56013b31dc00 0x56013b3b7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.616+0000 7fabbddea7c0 1 -- 172.21.15.16:0/2723465486 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:5 10.0 10:012cdbc7:::10000000206.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x5614be7fb500 con 0x5614bdd39000 2023-05-20T01:22:18.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.616+0000 7f3b5beef7c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56013b658000 0x56013b3b7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.616+0000 7f3b5beef7c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56013b658400 0x56013b3b8100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.616+0000 7f3b5beef7c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x56013b358b40 con 0x56013b31dc00 2023-05-20T01:22:18.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.616+0000 7f3b5beef7c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x56013b359400 con 0x56013b658000 2023-05-20T01:22:18.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.616+0000 7f3b5beef7c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x56013b359cc0 con 0x56013b658400 2023-05-20T01:22:18.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.616+0000 7f90cfff67c0 4 Got MDS map 42 2023-05-20T01:22:18.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.616+0000 7f90cfff67c0 4 datascan.main: Using metadata pool output 2023-05-20T01:22:18.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.616+0000 7f90cfff67c0 4 datascan.main: connecting to RADOS... 2023-05-20T01:22:18.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.616+0000 7f7782aee7c0 1 -- start start 2023-05-20T01:22:18.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.616+0000 7f3b5969c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56013b658400 0x56013b3b8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.616+0000 7f3b59e9d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56013b658000 0x56013b3b7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.616+0000 7f3b59e9d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56013b658000 0x56013b3b7b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.16:60254/0 (socket says 172.21.15.16:60254) 2023-05-20T01:22:18.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.616+0000 7f3b5969c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56013b658400 0x56013b3b8100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.16:35036/0 (socket says 172.21.15.16:35036) 2023-05-20T01:22:18.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.616+0000 7f3b59e9d700 1 -- 172.21.15.16:0/1981106667 learned_addr learned my addr 172.21.15.16:0/1981106667 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:22:18.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.617+0000 7fabbc599700 1 -- 172.21.15.16:0/2723465486 <== osd.6 v2:172.21.15.132:6801/727581123 3 ==== osd_op_reply(5 10000000206.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (98) out=98b,getxattr,getxattr] v0'0 uv12 ondisk = 0) v8 ==== 416+0+144 (crc 0 0 0) 0x5614bdd37680 con 0x5614bdd39000 2023-05-20T01:22:18.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.617+0000 7fabbddea7c0 1 -- 172.21.15.16:0/2723465486 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:6 11.0 11:012cdbc7:::10000000206.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x5614be7fbc00 con 0x5614bdd38c00 2023-05-20T01:22:18.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.617+0000 7f37780767c0 1 -- 172.21.15.16:0/4201969874 shutdown_connections 2023-05-20T01:22:18.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.617+0000 7f3b5969c700 1 -- 172.21.15.16:0/1981106667 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56013b658000 msgr2=0x56013b3b7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:18.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.617+0000 7f3b5969c700 1 --2- 172.21.15.16:0/1981106667 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56013b658000 0x56013b3b7b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.617+0000 7f3b5969c700 1 -- 172.21.15.16:0/1981106667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56013b31dc00 msgr2=0x56013b3b7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:18.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.617+0000 7f3b5969c700 1 --2- 172.21.15.16:0/1981106667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56013b31dc00 0x56013b3b7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.617+0000 7f3b5969c700 1 -- 172.21.15.16:0/1981106667 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56013b341d40 con 0x56013b658400 2023-05-20T01:22:18.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.617+0000 7f70ce3ee7c0 1 -- 172.21.15.16:0/766410834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636419b4000 msgr2=0x563641728100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.617+0000 7f70ce3ee7c0 1 --2- 172.21.15.16:0/766410834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636419b4000 0x563641728100 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x563642468e40 tx=0x563642474060 comp rx=0 tx=0).stop 2023-05-20T01:22:18.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.617+0000 7f90cfff67c0 1 Processor -- start 2023-05-20T01:22:18.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.617+0000 7f37780767c0 1 -- 172.21.15.16:0/4201969874 wait complete. 2023-05-20T01:22:18.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.617+0000 7f3b5969c700 1 --2- 172.21.15.16:0/1981106667 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56013b658400 0x56013b3b8100 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x56013c0e6210 tx=0x56013b60cc00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18437afa8b701673 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.617+0000 7f7782aee7c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563d8a4a3800 0x563d8a53f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.617+0000 7f3b55694700 1 -- 172.21.15.16:0/1981106667 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56013b351600 con 0x56013b658400 2023-05-20T01:22:18.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.617+0000 7f3b55694700 1 -- 172.21.15.16:0/1981106667 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56013b5fed80 con 0x56013b658400 2023-05-20T01:22:18.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.617+0000 7f3b55694700 1 -- 172.21.15.16:0/1981106667 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56013b3511e0 con 0x56013b658400 2023-05-20T01:22:18.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.617+0000 7f7782aee7c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563d8a80c000 0x563d8a53ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.617+0000 7f7782aee7c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d8a80c400 0x563d8a53e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.617+0000 7f7782aee7c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d8a4dec80 con 0x563d8a80c400 2023-05-20T01:22:18.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.617+0000 7f7782aee7c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d8a4de000 con 0x563d8a80c000 2023-05-20T01:22:18.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.617+0000 7f7782aee7c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x563d8a4df180 con 0x563d8a4a3800 2023-05-20T01:22:18.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.617+0000 7f70ce3ee7c0 1 -- 172.21.15.16:0/766410834 shutdown_connections 2023-05-20T01:22:18.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.617+0000 7f70ce3ee7c0 1 --2- 172.21.15.16:0/766410834 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5636419b4400 0x563641728680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.617+0000 7f70ce3ee7c0 1 --2- 172.21.15.16:0/766410834 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56364168dc00 0x563641727b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.617+0000 7f70ce3ee7c0 1 --2- 172.21.15.16:0/766410834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636419b4000 0x563641728100 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.617+0000 7f70ce3ee7c0 1 -- 172.21.15.16:0/766410834 >> 172.21.15.16:0/766410834 conn(0x56364168d800 msgr2=0x56364174d800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:18.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.617+0000 7f70ce3ee7c0 1 -- 172.21.15.16:0/766410834 shutdown_connections 2023-05-20T01:22:18.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.617+0000 7f778129d700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563d8a4a3800 0x563d8a53f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.617+0000 7f7780a9c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563d8a80c000 0x563d8a53ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.617+0000 7f778129d700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563d8a4a3800 0x563d8a53f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.16:35040/0 (socket says 172.21.15.16:35040) 2023-05-20T01:22:18.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.617+0000 7f778129d700 1 -- 172.21.15.16:0/251369889 learned_addr learned my addr 172.21.15.16:0/251369889 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:22:18.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.617+0000 7f70ce3ee7c0 1 -- 172.21.15.16:0/766410834 wait complete. 2023-05-20T01:22:18.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.617+0000 7fabbb597700 1 -- 172.21.15.16:0/2723465486 <== osd.1 v2:172.21.15.16:6808/3860264220 3 ==== osd_op_reply(6 10000000206.00000000 [getxattr (30) out=30b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x5614bdd37440 con 0x5614bdd38c00 2023-05-20T01:22:18.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.618+0000 7fabbddea7c0 20 datascan.operator(): loaded layout from xattr: pi: 11 os: 4194304 sc: 1 su: 4194304 2023-05-20T01:22:18.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.618+0000 7fabbddea7c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000206 2023-05-20T01:22:18.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.618+0000 7fabbddea7c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/17 2023-05-20T01:22:18.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.618+0000 7fabbddea7c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=17 2023-05-20T01:22:18.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.617+0000 7f90cfff67c0 1 -- start start 2023-05-20T01:22:18.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.618+0000 7fabbddea7c0 1 --2- 172.21.15.16:0/2723465486 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x5614bde80400 0x5614bdaad600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.618+0000 7fabbddea7c0 1 -- 172.21.15.16:0/2723465486 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:7 9.10 9:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x5614be7fa700 con 0x5614bde80400 2023-05-20T01:22:18.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.618+0000 7f37780767c0 1 Processor -- start 2023-05-20T01:22:18.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.618+0000 7f778129d700 1 -- 172.21.15.16:0/251369889 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563d8a80c000 msgr2=0x563d8a53ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.618+0000 7f778129d700 1 --2- 172.21.15.16:0/251369889 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563d8a80c000 0x563d8a53ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.618+0000 7f778129d700 1 -- 172.21.15.16:0/251369889 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d8a80c400 msgr2=0x563d8a53e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.618+0000 7f778129d700 1 --2- 172.21.15.16:0/251369889 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d8a80c400 0x563d8a53e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.618+0000 7f778129d700 1 -- 172.21.15.16:0/251369889 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563d8a4c71e0 con 0x563d8a4a3800 2023-05-20T01:22:18.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.618+0000 7f7780a9c700 1 --2- 172.21.15.16:0/251369889 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563d8a80c000 0x563d8a53ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-05-20T01:22:18.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.618+0000 7f90cfff67c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556b16dbbc00 0x556b16e55600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.618+0000 7f90cfff67c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556b170ec000 0x556b16e55b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.618+0000 7f90cfff67c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556b170ec400 0x556b16e56100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.618+0000 7f90cfff67c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b16df6b40 con 0x556b170ec000 2023-05-20T01:22:18.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.618+0000 7f90cfff67c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b16df7e00 con 0x556b170ec400 2023-05-20T01:22:18.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.618+0000 7f90cfff67c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x556b16df6c80 con 0x556b16dbbc00 2023-05-20T01:22:18.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.618+0000 7f37780767c0 1 -- start start 2023-05-20T01:22:18.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.618+0000 7f90ce7a5700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556b16dbbc00 0x556b16e55600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.618+0000 7f90ce7a5700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556b16dbbc00 0x556b16e55600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.16:35050/0 (socket says 172.21.15.16:35050) 2023-05-20T01:22:18.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.618+0000 7f90ce7a5700 1 -- 172.21.15.16:0/1295055451 learned_addr learned my addr 172.21.15.16:0/1295055451 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:22:18.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.618+0000 7f778129d700 1 --2- 172.21.15.16:0/251369889 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563d8a4a3800 0x563d8a53f180 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x563d8a7f6f60 tx=0x563d8a7a0c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.618+0000 7f70ce3ee7c0 1 Processor -- start 2023-05-20T01:22:18.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.618+0000 7f90cdfa4700 1 --2- 172.21.15.16:0/1295055451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556b170ec000 0x556b16e55b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.618+0000 7f777aa90700 1 -- 172.21.15.16:0/251369889 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x563d8a4d7080 con 0x563d8a4a3800 2023-05-20T01:22:18.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.618+0000 7f777aa90700 1 -- 172.21.15.16:0/251369889 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x563d8a796180 con 0x563d8a4a3800 2023-05-20T01:22:18.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.618+0000 7f777aa90700 1 -- 172.21.15.16:0/251369889 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x563d8a4d7600 con 0x563d8a4a3800 2023-05-20T01:22:18.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.619+0000 7fabbbd98700 1 --2- 172.21.15.16:0/2723465486 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x5614bde80400 0x5614bdaad600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.619+0000 7f70ce3ee7c0 1 -- start start 2023-05-20T01:22:18.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.618+0000 7f3b5beef7c0 1 -- 172.21.15.16:0/1981106667 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56013b658400 msgr2=0x56013b3b8100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.618+0000 7f3b5beef7c0 1 --2- 172.21.15.16:0/1981106667 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56013b658400 0x56013b3b8100 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x56013c0e6210 tx=0x56013b60cc00 comp rx=0 tx=0).stop 2023-05-20T01:22:18.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.619+0000 7f90ce7a5700 1 -- 172.21.15.16:0/1295055451 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556b170ec400 msgr2=0x556b16e56100 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-05-20T01:22:18.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.619+0000 7f90ce7a5700 1 --2- 172.21.15.16:0/1295055451 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556b170ec400 0x556b16e56100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.619+0000 7f90ce7a5700 1 -- 172.21.15.16:0/1295055451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556b170ec000 msgr2=0x556b16e55b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:18.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.619+0000 7f90ce7a5700 1 --2- 172.21.15.16:0/1295055451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556b170ec000 0x556b16e55b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.619+0000 7f90ce7a5700 1 -- 172.21.15.16:0/1295055451 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556b16ddf380 con 0x556b16dbbc00 2023-05-20T01:22:18.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.619+0000 7f37780767c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce83eae000 0x55ce83c1f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.619+0000 7f37780767c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ce83eae400 0x55ce83c1fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.619+0000 7f37780767c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ce83eae800 0x55ce83c20100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.619+0000 7f37780767c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce83bc0b40 con 0x55ce83eae000 2023-05-20T01:22:18.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.619+0000 7f37780767c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce83bc0c80 con 0x55ce83eae800 2023-05-20T01:22:18.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.619+0000 7f37780767c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ce83bc0000 con 0x55ce83eae400 2023-05-20T01:22:18.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.619+0000 7f3b5beef7c0 1 -- 172.21.15.16:0/1981106667 shutdown_connections 2023-05-20T01:22:18.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.619+0000 7f3b5beef7c0 1 --2- 172.21.15.16:0/1981106667 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56013b658400 0x56013b3b8100 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.619+0000 7f3b5beef7c0 1 --2- 172.21.15.16:0/1981106667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56013b31dc00 0x56013b3b7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.619+0000 7f3b5beef7c0 1 --2- 172.21.15.16:0/1981106667 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56013b658000 0x56013b3b7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.619+0000 7f3b5beef7c0 1 -- 172.21.15.16:0/1981106667 >> 172.21.15.16:0/1981106667 conn(0x56013b31c400 msgr2=0x56013b3dd800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:18.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.619+0000 7f3775823700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ce83eae800 0x55ce83c20100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.619+0000 7f3775823700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ce83eae800 0x55ce83c20100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.16:60268/0 (socket says 172.21.15.16:60268) 2023-05-20T01:22:18.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.619+0000 7f3775823700 1 -- 172.21.15.16:0/1569316897 learned_addr learned my addr 172.21.15.16:0/1569316897 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:22:18.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.619+0000 7fabbbd98700 1 --2- 172.21.15.16:0/2723465486 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x5614bde80400 0x5614bdaad600 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.619+0000 7f90ce7a5700 1 --2- 172.21.15.16:0/1295055451 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556b16dbbc00 0x556b16e55600 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x556b170f20c0 tx=0x556b17b9a000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1d06bfbed84a2f6 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.619+0000 7f7782aee7c0 1 -- 172.21.15.16:0/251369889 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x563d8a4c7380 con 0x563d8a4a3800 2023-05-20T01:22:18.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.619+0000 7f7782aee7c0 1 -- 172.21.15.16:0/251369889 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563d8a8184e0 con 0x563d8a4a3800 2023-05-20T01:22:18.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.619+0000 7f7782aee7c0 4 datascan.init: resolving metadata pool 9 2023-05-20T01:22:18.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.619+0000 7f90c979b700 1 -- 172.21.15.16:0/1295055451 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x556b16def600 con 0x556b16dbbc00 2023-05-20T01:22:18.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.619+0000 7f90c979b700 1 -- 172.21.15.16:0/1295055451 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556b16e0de00 con 0x556b16dbbc00 2023-05-20T01:22:18.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.619+0000 7f90c979b700 1 -- 172.21.15.16:0/1295055451 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x556b16def1e0 con 0x556b16dbbc00 2023-05-20T01:22:18.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.619+0000 7f3b5beef7c0 1 -- 172.21.15.16:0/1981106667 shutdown_connections 2023-05-20T01:22:18.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.619+0000 7f3775823700 1 -- 172.21.15.16:0/1569316897 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ce83eae400 msgr2=0x55ce83c1fb80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-05-20T01:22:18.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.619+0000 7f3775823700 1 --2- 172.21.15.16:0/1569316897 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ce83eae400 0x55ce83c1fb80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.619+0000 7f3775823700 1 -- 172.21.15.16:0/1569316897 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce83eae000 msgr2=0x55ce83c1f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.619+0000 7f3775823700 1 --2- 172.21.15.16:0/1569316897 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce83eae000 0x55ce83c1f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.619+0000 7f3775823700 1 -- 172.21.15.16:0/1569316897 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ce83ba8d00 con 0x55ce83eae800 2023-05-20T01:22:18.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.619+0000 7fabbbd98700 1 -- 172.21.15.16:0/2723465486 <== osd.4 v2:172.21.15.132:6817/506317223 1 ==== osd_op_reply(7 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 164+0+0 (crc 0 0 0) 0x5614bdd378c0 con 0x5614bde80400 2023-05-20T01:22:18.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.619+0000 7fabbddea7c0 4 datascan.get_frag_of: Unexpected error on '10000000000.00000000': (2) No such file or directory 2023-05-20T01:22:18.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.619+0000 7fabbddea7c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/17, will insert to root 2023-05-20T01:22:18.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.619+0000 7fabbddea7c0 1 -- 172.21.15.16:0/2723465486 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:8 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x5614be7faa80 con 0x5614bde80400 2023-05-20T01:22:18.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.620+0000 7f70ce3ee7c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56364168d800 0x563641727b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.620+0000 7f70ce3ee7c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5636419bc000 0x563641728100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.620+0000 7f70ce3ee7c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5636419bc400 0x563641728680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.620+0000 7f70ce3ee7c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636416c9e00 con 0x56364168d800 2023-05-20T01:22:18.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.620+0000 7f70ce3ee7c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636416c8c80 con 0x5636419bc400 2023-05-20T01:22:18.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.620+0000 7f70ce3ee7c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x5636416c8000 con 0x5636419bc000 2023-05-20T01:22:18.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.620+0000 7f3775823700 1 --2- 172.21.15.16:0/1569316897 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ce83eae800 0x55ce83c20100 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x55ce83eb9020 tx=0x55ce84a2ad20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.620+0000 7fabbbd98700 1 -- 172.21.15.16:0/2723465486 <== osd.4 v2:172.21.15.132:6817/506317223 2 ==== osd_op_reply(8 10000000000.00000000 [omap-get-header] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 164+0+0 (crc 0 0 0) 0x5614bdd378c0 con 0x5614bde80400 2023-05-20T01:22:18.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.620+0000 7fabbddea7c0 10 datascan.find_or_create_dirfrag: read_version = 0 2023-05-20T01:22:18.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.620+0000 7fabbddea7c0 1 -- 172.21.15.16:0/2723465486 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:9 9.10 9:0cf6ad0f:::10000000000.00000000:head [create,omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x5614be7fbc00 con 0x5614bde80400 2023-05-20T01:22:18.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.620+0000 7f70ccb9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56364168d800 0x563641727b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.620+0000 7f70ccb9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56364168d800 0x563641727b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35018/0 (socket says 172.21.15.16:35018) 2023-05-20T01:22:18.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.620+0000 7f70ccb9d700 1 -- 172.21.15.16:0/3489021394 learned_addr learned my addr 172.21.15.16:0/3489021394 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:22:18.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.620+0000 7f3770018700 1 -- 172.21.15.16:0/1569316897 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55ce83bb9080 con 0x55ce83eae800 2023-05-20T01:22:18.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.620+0000 7f37780767c0 1 -- 172.21.15.16:0/1569316897 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55ce83ba96c0 con 0x55ce83eae800 2023-05-20T01:22:18.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.620+0000 7f3770018700 1 -- 172.21.15.16:0/1569316897 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ce84a26600 con 0x55ce83eae800 2023-05-20T01:22:18.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.620+0000 7f3770018700 1 -- 172.21.15.16:0/1569316897 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55ce83bb9600 con 0x55ce83eae800 2023-05-20T01:22:18.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.620+0000 7f37780767c0 1 -- 172.21.15.16:0/1569316897 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ce83ba84e0 con 0x55ce83eae800 2023-05-20T01:22:18.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.620+0000 7f37780767c0 4 datascan.init: resolving metadata pool 9 2023-05-20T01:22:18.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.620+0000 7f90cfff67c0 1 -- 172.21.15.16:0/1295055451 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556b16dbbc00 msgr2=0x556b16e55600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.620+0000 7f90cfff67c0 1 --2- 172.21.15.16:0/1295055451 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556b16dbbc00 0x556b16e55600 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x556b170f20c0 tx=0x556b17b9a000 comp rx=0 tx=0).stop 2023-05-20T01:22:18.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.620+0000 7f3b5beef7c0 1 -- 172.21.15.16:0/1981106667 wait complete. 2023-05-20T01:22:18.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.621+0000 7f90cfff67c0 1 -- 172.21.15.16:0/1295055451 shutdown_connections 2023-05-20T01:22:18.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.621+0000 7f90cfff67c0 1 --2- 172.21.15.16:0/1295055451 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556b170ec400 0x556b16e56100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.621+0000 7f90cfff67c0 1 --2- 172.21.15.16:0/1295055451 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556b16dbbc00 0x556b16e55600 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.621+0000 7f90cfff67c0 1 --2- 172.21.15.16:0/1295055451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556b170ec000 0x556b16e55b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.621+0000 7f90cfff67c0 1 -- 172.21.15.16:0/1295055451 >> 172.21.15.16:0/1295055451 conn(0x556b16dbb800 msgr2=0x556b16e7b800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:18.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.621+0000 7f90cfff67c0 1 -- 172.21.15.16:0/1295055451 shutdown_connections 2023-05-20T01:22:18.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.620+0000 7f70cbb9b700 1 --2- 172.21.15.16:0/3489021394 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5636419bc400 0x563641728680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.620+0000 7f70ccb9d700 1 -- 172.21.15.16:0/3489021394 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5636419bc000 msgr2=0x563641728100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.620+0000 7f70ccb9d700 1 --2- 172.21.15.16:0/3489021394 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5636419bc000 0x563641728100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.620+0000 7f70ccb9d700 1 -- 172.21.15.16:0/3489021394 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5636419bc400 msgr2=0x563641728680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.620+0000 7f70ccb9d700 1 --2- 172.21.15.16:0/3489021394 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5636419bc400 0x563641728680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.620+0000 7f70ccb9d700 1 -- 172.21.15.16:0/3489021394 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5636416b1d40 con 0x56364168d800 2023-05-20T01:22:18.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.620+0000 7f70ccb9d700 1 --2- 172.21.15.16:0/3489021394 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56364168d800 0x563641727b80 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x5636419bf350 tx=0x56364196cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.621+0000 7f70c6390700 1 -- 172.21.15.16:0/3489021394 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5636416c11e0 con 0x56364168d800 2023-05-20T01:22:18.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.621+0000 7f70c6390700 1 -- 172.21.15.16:0/3489021394 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5636416dec00 con 0x56364168d800 2023-05-20T01:22:18.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.621+0000 7f70c6390700 1 -- 172.21.15.16:0/3489021394 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5636416c0f20 con 0x56364168d800 2023-05-20T01:22:18.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.621+0000 7f70ce3ee7c0 1 -- 172.21.15.16:0/3489021394 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5636416b11e0 con 0x56364168d800 2023-05-20T01:22:18.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.621+0000 7f70ce3ee7c0 1 -- 172.21.15.16:0/3489021394 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5636419f0000 con 0x56364168d800 2023-05-20T01:22:18.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.621+0000 7f70ce3ee7c0 4 datascan.init: resolving metadata pool 9 2023-05-20T01:22:18.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.621+0000 7f777aa90700 1 -- 172.21.15.16:0/251369889 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x563d8a504840 con 0x563d8a4a3800 2023-05-20T01:22:18.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.621+0000 7f3b5beef7c0 1 Processor -- start 2023-05-20T01:22:18.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.621+0000 7f3b5beef7c0 1 -- start start 2023-05-20T01:22:18.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.621+0000 7f90cfff67c0 1 -- 172.21.15.16:0/1295055451 wait complete. 2023-05-20T01:22:18.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.621+0000 7f90cfff67c0 1 Processor -- start 2023-05-20T01:22:18.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.622+0000 7f90cfff67c0 1 -- start start 2023-05-20T01:22:18.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.622+0000 7f777aa90700 1 --2- 172.21.15.16:0/251369889 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x563d8a80ec00 0x563d8a53f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.622+0000 7f777aa90700 1 -- 172.21.15.16:0/251369889 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 7103+0+0 (secure 0 0 0) 0x563d8a4e4a80 con 0x563d8a4a3800 2023-05-20T01:22:18.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.622+0000 7f7780a9c700 1 --2- 172.21.15.16:0/251369889 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x563d8a80ec00 0x563d8a53f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.622+0000 7f3b5beef7c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56013b31c400 0x56013b3b9180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.622+0000 7f3b5beef7c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56013b664000 0x56013b3b8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.622+0000 7f3b5beef7c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56013b664400 0x56013b3b8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.622+0000 7f3b5beef7c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x56013b358c80 con 0x56013b31c400 2023-05-20T01:22:18.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.622+0000 7f3b5beef7c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x56013b358000 con 0x56013b664400 2023-05-20T01:22:18.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.622+0000 7f3b5beef7c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x56013b359180 con 0x56013b664000 2023-05-20T01:22:18.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.622+0000 7fabbbd98700 1 -- 172.21.15.16:0/2723465486 <== osd.4 v2:172.21.15.132:6817/506317223 3 ==== osd_op_reply(9 10000000000.00000000 [create,omap-set-header] v71'16 uv16 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x5614bdd378c0 con 0x5614bde80400 2023-05-20T01:22:18.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.622+0000 7fabbddea7c0 10 datascan.find_or_create_dirfrag: Created dirfrag: 0x0x10000000000 2023-05-20T01:22:18.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.622+0000 7f3b59e9d700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56013b664000 0x56013b3b8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.622+0000 7f3b59e9d700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56013b664000 0x56013b3b8c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.16:35066/0 (socket says 172.21.15.16:35066) 2023-05-20T01:22:18.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.622+0000 7f3b59e9d700 1 -- 172.21.15.16:0/3957359014 learned_addr learned my addr 172.21.15.16:0/3957359014 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:22:18.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.622+0000 7f3b5969c700 1 --2- 172.21.15.16:0/3957359014 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56013b664400 0x56013b3b8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.622+0000 7fabbddea7c0 1 -- 172.21.15.16:0/2723465486 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:10 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x5614be7fae00 con 0x5614bde80400 2023-05-20T01:22:18.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.623+0000 7f7780a9c700 1 --2- 172.21.15.16:0/251369889 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x563d8a80ec00 0x563d8a53f700 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x563d8a7f6030 tx=0x563d8b2363c0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.623+0000 7f90cfff67c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556b16dbb800 0x556b16e57180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.623+0000 7f90cfff67c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556b170fe000 0x556b16e56c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.623+0000 7f90cfff67c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556b170fe400 0x556b16e56680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.623+0000 7fa3b2b56700 1 --2- 172.21.15.16:0/128094146 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555b4a4fd000 0x555b4a596b00 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x555b4b2d8cf0 tx=0x555b4b2de360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97999544e9c9dbfe server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.623+0000 7f3b5969c700 1 -- 172.21.15.16:0/3957359014 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56013b664000 msgr2=0x56013b3b8c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.623+0000 7f3b5969c700 1 --2- 172.21.15.16:0/3957359014 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56013b664000 0x56013b3b8c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.623+0000 7f3b5969c700 1 -- 172.21.15.16:0/3957359014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56013b31c400 msgr2=0x56013b3b9180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.623+0000 7f3b5969c700 1 --2- 172.21.15.16:0/3957359014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56013b31c400 0x56013b3b9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.623+0000 7f3b5969c700 1 -- 172.21.15.16:0/3957359014 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56013b341380 con 0x56013b664400 2023-05-20T01:22:18.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.623+0000 7f90cfff67c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b16df7e00 con 0x556b170fe000 2023-05-20T01:22:18.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.623+0000 7f90cfff67c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b16df6b40 con 0x556b170fe400 2023-05-20T01:22:18.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.623+0000 7fabbbd98700 1 -- 172.21.15.16:0/2723465486 <== osd.4 v2:172.21.15.132:6817/506317223 4 ==== osd_op_reply(10 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv16 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x5614bdd378c0 con 0x5614bde80400 2023-05-20T01:22:18.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.623+0000 7f90cfff67c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x556b16df6c80 con 0x556b16dbb800 2023-05-20T01:22:18.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.623+0000 7f90ce7a5700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556b16dbb800 0x556b16e57180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.623+0000 7fabbddea7c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=17 frag=*, r=0 2023-05-20T01:22:18.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.623+0000 7fabbddea7c0 20 datascan.read_dentry: 17_head not found in result 2023-05-20T01:22:18.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.623+0000 7fabbddea7c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000206 at 0x0x10000000000/17 with size=6291456 bytes 2023-05-20T01:22:18.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.623+0000 7fabbddea7c0 1 -- 172.21.15.16:0/2723465486 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:11 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x5614be7fb180 con 0x5614bde80400 2023-05-20T01:22:18.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.623+0000 7fa3b0b52700 1 -- 172.21.15.16:0/128094146 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x555b4a531600 con 0x555b4a4fd000 2023-05-20T01:22:18.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.623+0000 7f90cd7a3700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556b170fe400 0x556b16e56680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.623+0000 7f90cd7a3700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556b170fe400 0x556b16e56680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.16:60290/0 (socket says 172.21.15.16:60290) 2023-05-20T01:22:18.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.623+0000 7f90cd7a3700 1 -- 172.21.15.16:0/3200652326 learned_addr learned my addr 172.21.15.16:0/3200652326 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:22:18.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.624+0000 7fabbbd98700 1 -- 172.21.15.16:0/2723465486 <== osd.4 v2:172.21.15.132:6817/506317223 5 ==== osd_op_reply(11 10000000000.00000000 [omap-set-vals] v71'17 uv17 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5614bdd378c0 con 0x5614bde80400 2023-05-20T01:22:18.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.623+0000 7f70c6390700 1 -- 172.21.15.16:0/3489021394 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x5636416ee840 con 0x56364168d800 2023-05-20T01:22:18.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.623+0000 7f70c6390700 1 --2- 172.21.15.16:0/3489021394 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x5636419f8400 0x563641729700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.623+0000 7f70c6390700 1 -- 172.21.15.16:0/3489021394 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 7103+0+0 (secure 0 0 0) 0x5636416cea80 con 0x56364168d800 2023-05-20T01:22:18.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.623+0000 7f70ce3ee7c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-05-20T01:22:18.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.623+0000 7f70cc39c700 1 --2- 172.21.15.16:0/3489021394 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x5636419f8400 0x563641729700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.623+0000 7f70ce3ee7c0 4 datascan.main: data pool 'cephfs_data' has ID 10 2023-05-20T01:22:18.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.623+0000 7f70ce3ee7c0 4 datascan.main: opening data pool 'cephfs_data' 2023-05-20T01:22:18.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.623+0000 7f70ce3ee7c0 1 --2- 172.21.15.16:0/3489021394 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x5636419bc800 0x563641728c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.623+0000 7f70ce3ee7c0 1 -- 172.21.15.16:0/3489021394 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:1 9.14 9:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x56364246c700 con 0x5636419bc800 2023-05-20T01:22:18.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.622+0000 7f7782aee7c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-05-20T01:22:18.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.623+0000 7f7782aee7c0 4 datascan.main: data pool 'cephfs_data' has ID 10 2023-05-20T01:22:18.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.623+0000 7f7782aee7c0 4 datascan.main: opening data pool 'cephfs_data' 2023-05-20T01:22:18.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.623+0000 7f7782aee7c0 1 --2- 172.21.15.16:0/251369889 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x563d8a80c800 0x563d8a53e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.623+0000 7f7782aee7c0 1 -- 172.21.15.16:0/251369889 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:1 9.14 9:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x563d8a794700 con 0x563d8a80c800 2023-05-20T01:22:18.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.624+0000 7f778029b700 1 --2- 172.21.15.16:0/251369889 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x563d8a80c800 0x563d8a53e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.624+0000 7f90cd7a3700 1 -- 172.21.15.16:0/3200652326 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556b16dbb800 msgr2=0x556b16e57180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.624+0000 7f90cd7a3700 1 --2- 172.21.15.16:0/3200652326 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556b16dbb800 0x556b16e57180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.624+0000 7f90cd7a3700 1 -- 172.21.15.16:0/3200652326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556b170fe000 msgr2=0x556b16e56c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.624+0000 7f90cd7a3700 1 --2- 172.21.15.16:0/3200652326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556b170fe000 0x556b16e56c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.624+0000 7f90cd7a3700 1 -- 172.21.15.16:0/3200652326 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556b17b9c340 con 0x556b170fe400 2023-05-20T01:22:18.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.624+0000 7f90ce7a5700 1 --2- 172.21.15.16:0/3200652326 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556b16dbb800 0x556b16e57180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-05-20T01:22:18.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.623+0000 7fa3b0b52700 1 -- 172.21.15.16:0/128094146 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555b4a54f800 con 0x555b4a4fd000 2023-05-20T01:22:18.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.623+0000 7fa3b0b52700 1 -- 172.21.15.16:0/128094146 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x555b4a5311e0 con 0x555b4a4fd000 2023-05-20T01:22:18.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.623+0000 7fa3b53a97c0 1 -- 172.21.15.16:0/128094146 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555b4a5216c0 con 0x555b4a4fd000 2023-05-20T01:22:18.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.623+0000 7fa3b53a97c0 1 -- 172.21.15.16:0/128094146 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x555b4a521520 con 0x555b4a4fd000 2023-05-20T01:22:18.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.623+0000 7fa3b53a97c0 4 waiting for MDS map... 2023-05-20T01:22:18.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.624+0000 7fa3b0b52700 1 -- 172.21.15.16:0/128094146 <== mon.1 v2:172.21.15.132:3300/0 4 ==== osd_map(71..71 src has 1..71) v4 ==== 7103+0+0 (secure 0 0 0) 0x555b4a53ec40 con 0x555b4a4fd000 2023-05-20T01:22:18.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.624+0000 7fa3b0b52700 1 -- 172.21.15.16:0/128094146 <== mon.1 v2:172.21.15.132:3300/0 5 ==== fsmap(e 42) v1 ==== 3126+0+0 (secure 0 0 0) 0x555b4b2ee300 con 0x555b4a4fd000 2023-05-20T01:22:18.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.624+0000 7fa3b53a97c0 4 Got MDS map 42 2023-05-20T01:22:18.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.624+0000 7fa3b53a97c0 4 datascan.main: Using metadata pool output 2023-05-20T01:22:18.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.624+0000 7fa3b53a97c0 4 datascan.main: connecting to RADOS... 2023-05-20T01:22:18.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.623+0000 7f3770018700 1 -- 172.21.15.16:0/1569316897 <== mon.1 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x55ce83be7600 con 0x55ce83eae800 2023-05-20T01:22:18.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.623+0000 7f3770018700 1 --2- 172.21.15.16:0/1569316897 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x55ce83fbe000 0x55ce83c21180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.624+0000 7f3770018700 1 -- 172.21.15.16:0/1569316897 <== mon.1 v2:172.21.15.132:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 7103+0+0 (secure 0 0 0) 0x55ce83bc6a80 con 0x55ce83eae800 2023-05-20T01:22:18.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.624+0000 7f37780767c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-05-20T01:22:18.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.624+0000 7f37780767c0 4 datascan.main: data pool 'cephfs_data' has ID 10 2023-05-20T01:22:18.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.624+0000 7f37780767c0 4 datascan.main: opening data pool 'cephfs_data' 2023-05-20T01:22:18.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.624+0000 7f37780767c0 1 --2- 172.21.15.16:0/1569316897 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x55ce83eaec00 0x55ce83c20680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.624+0000 7f37780767c0 1 -- 172.21.15.16:0/1569316897 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:1 9.14 9:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x55ce84a1c700 con 0x55ce83eaec00 2023-05-20T01:22:18.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.624+0000 7f3776825700 1 --2- 172.21.15.16:0/1569316897 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x55ce83fbe000 0x55ce83c21180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.624+0000 7f3776024700 1 --2- 172.21.15.16:0/1569316897 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x55ce83eaec00 0x55ce83c20680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.623+0000 7f3b5969c700 1 --2- 172.21.15.16:0/3957359014 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56013b664400 0x56013b3b8680 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x56013c0e63c0 tx=0x56013c0fe300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.623+0000 7f3b53e91700 1 -- 172.21.15.16:0/3957359014 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56013b351080 con 0x56013b664400 2023-05-20T01:22:18.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.623+0000 7f3b53e91700 1 -- 172.21.15.16:0/3957359014 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56013b36fc80 con 0x56013b664400 2023-05-20T01:22:18.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.623+0000 7f3b53e91700 1 -- 172.21.15.16:0/3957359014 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56013b351600 con 0x56013b664400 2023-05-20T01:22:18.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.624+0000 7fabbddea7c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/17 pointing to 0x0x10000000206 2023-05-20T01:22:18.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.624+0000 7fabbddea7c0 10 datascan.inject_with_backtrace: backptr: 0x0x1/subdir 2023-05-20T01:22:18.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.624+0000 7fabbddea7c0 20 datascan.get_frag_of: dirino=0x1 target_dname=subdir 2023-05-20T01:22:18.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.624+0000 7fabbddea7c0 1 -- 172.21.15.16:0/2723465486 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:12 9.1f 9:ff5b34d6:::1.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x5614be7fbc00 con 0x5614bdd38c00 2023-05-20T01:22:18.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.624+0000 7f3776825700 1 --2- 172.21.15.16:0/1569316897 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x55ce83fbe000 0x55ce83c21180 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x55ce83eb82a0 tx=0x55ce8496c180 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.624+0000 7f3b5beef7c0 1 -- 172.21.15.16:0/3957359014 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56013b340d00 con 0x56013b664400 2023-05-20T01:22:18.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.624+0000 7f3b5beef7c0 1 -- 172.21.15.16:0/3957359014 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56013b341d40 con 0x56013b664400 2023-05-20T01:22:18.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.624+0000 7f3b5beef7c0 4 datascan.init: resolving metadata pool 9 2023-05-20T01:22:18.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.625+0000 7f70cc39c700 1 --2- 172.21.15.16:0/3489021394 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x5636419f8400 0x563641729700 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x563642468e40 tx=0x56364196ccc0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.625+0000 7f90cd7a3700 1 --2- 172.21.15.16:0/3200652326 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556b170fe400 0x556b16e56680 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x556b170f3380 tx=0x556b1709cc00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.625+0000 7f70cbb9b700 1 --2- 172.21.15.16:0/3489021394 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x5636419bc800 0x563641728c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.625+0000 7f778029b700 1 --2- 172.21.15.16:0/251369889 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x563d8a80c800 0x563d8a53e100 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.625+0000 7fa3b53a97c0 1 Processor -- start 2023-05-20T01:22:18.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.625+0000 7fabbb597700 1 -- 172.21.15.16:0/2723465486 <== osd.1 v2:172.21.15.16:6808/3860264220 4 ==== osd_op_reply(12 1.00000000 [getxattr] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x5614bdd37440 con 0x5614bdd38c00 2023-05-20T01:22:18.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.625+0000 7f3776024700 1 --2- 172.21.15.16:0/1569316897 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x55ce83eaec00 0x55ce83c20680 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.625+0000 7fabbddea7c0 4 datascan.get_frag_of: Unexpected error on '1.00000000': (2) No such file or directory 2023-05-20T01:22:18.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.625+0000 7fabbddea7c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x1/subdir, will insert to root 2023-05-20T01:22:18.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.625+0000 7fabbddea7c0 1 -- 172.21.15.16:0/2723465486 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:13 9.1f 9:ff5b34d6:::1.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x5614be7fae00 con 0x5614bdd38c00 2023-05-20T01:22:18.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.625+0000 7f90c7f98700 1 -- 172.21.15.16:0/3200652326 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x556b16def600 con 0x556b170fe400 2023-05-20T01:22:18.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.625+0000 7f90c7f98700 1 -- 172.21.15.16:0/3200652326 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556b17c5e780 con 0x556b170fe400 2023-05-20T01:22:18.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.625+0000 7f90c7f98700 1 -- 172.21.15.16:0/3200652326 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x556b16def1e0 con 0x556b170fe400 2023-05-20T01:22:18.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.625+0000 7f90cfff67c0 1 -- 172.21.15.16:0/3200652326 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x556b17b9c1a0 con 0x556b170fe400 2023-05-20T01:22:18.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.625+0000 7f70cbb9b700 1 --2- 172.21.15.16:0/3489021394 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x5636419bc800 0x563641728c00 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.625+0000 7f90cfff67c0 1 -- 172.21.15.16:0/3200652326 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556b17b9c4e0 con 0x556b170fe400 2023-05-20T01:22:18.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.625+0000 7f90cfff67c0 4 datascan.init: resolving metadata pool 9 2023-05-20T01:22:18.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.625+0000 7fabbb597700 1 -- 172.21.15.16:0/2723465486 <== osd.1 v2:172.21.15.16:6808/3860264220 5 ==== osd_op_reply(13 1.00000000 [omap-get-header] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x5614bdd37440 con 0x5614bdd38c00 2023-05-20T01:22:18.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.625+0000 7fabbddea7c0 10 datascan.find_or_create_dirfrag: read_version = 0 2023-05-20T01:22:18.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.625+0000 7f3776024700 1 -- 172.21.15.16:0/1569316897 <== osd.1 v2:172.21.15.16:6808/3860264220 1 ==== osd_op_reply(1 1.00000000.inode [stat out=16b] v0'0 uv133 ondisk = 0) v8 ==== 160+0+16 (crc 0 0 0) 0x55ce84a25680 con 0x55ce83eaec00 2023-05-20T01:22:18.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.625+0000 7fabbddea7c0 1 -- 172.21.15.16:0/2723465486 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:14 9.1f 9:ff5b34d6:::1.00000000:head [create,omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x5614be7fb180 con 0x5614bdd38c00 2023-05-20T01:22:18.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.625+0000 7f37780767c0 1 -- 172.21.15.16:0/1569316897 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:2 9.1f 9:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x55ce84a1ca80 con 0x55ce83eaec00 2023-05-20T01:22:18.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.626+0000 7f778029b700 1 -- 172.21.15.16:0/251369889 <== osd.1 v2:172.21.15.16:6808/3860264220 1 ==== osd_op_reply(1 1.00000000.inode [stat out=16b] v0'0 uv133 ondisk = 0) v8 ==== 160+0+16 (crc 0 0 0) 0x563d8a7ef440 con 0x563d8a80c800 2023-05-20T01:22:18.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.626+0000 7f7782aee7c0 1 -- 172.21.15.16:0/251369889 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:2 9.1f 9:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x563d8a794a80 con 0x563d8a80c800 2023-05-20T01:22:18.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.625+0000 7fa3b53a97c0 1 -- start start 2023-05-20T01:22:18.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.626+0000 7fa3b53a97c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x555b4a4fdc00 0x555b4a597600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.626+0000 7fa3b53a97c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b4a5dc400 0x555b4a597b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.626+0000 7fa3b53a97c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555b4a5dc800 0x555b4a598100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.626+0000 7fa3b53a97c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b4a538b40 con 0x555b4a5dc400 2023-05-20T01:22:18.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.626+0000 7fa3b53a97c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b4a538c80 con 0x555b4a5dc800 2023-05-20T01:22:18.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.626+0000 7fa3b53a97c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x555b4a539cc0 con 0x555b4a4fdc00 2023-05-20T01:22:18.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.626+0000 7fa3b3357700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b4a5dc400 0x555b4a597b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.626+0000 7fa3b3b58700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x555b4a4fdc00 0x555b4a597600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.626+0000 7fa3b2b56700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555b4a5dc800 0x555b4a598100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.626+0000 7fa3b3b58700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x555b4a4fdc00 0x555b4a597600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.16:35088/0 (socket says 172.21.15.16:35088) 2023-05-20T01:22:18.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.626+0000 7fa3b3357700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b4a5dc400 0x555b4a597b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35052/0 (socket says 172.21.15.16:35052) 2023-05-20T01:22:18.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.626+0000 7fa3b3b58700 1 -- 172.21.15.16:0/2935861361 learned_addr learned my addr 172.21.15.16:0/2935861361 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:22:18.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.626+0000 7f3776024700 1 -- 172.21.15.16:0/1569316897 <== osd.1 v2:172.21.15.16:6808/3860264220 2 ==== osd_op_reply(2 100.00000000.inode [stat out=16b] v0'0 uv10 ondisk = 0) v8 ==== 162+0+16 (crc 0 0 0) 0x55ce84a25680 con 0x55ce83eaec00 2023-05-20T01:22:18.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.626+0000 7f37780767c0 1 --2- 172.21.15.16:0/1569316897 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x55ce83eaf000 0x55ce83c20c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.626+0000 7f37780767c0 1 -- 172.21.15.16:0/1569316897 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:3 10.0 10:00000000::::head [pgnls-filter start_epoch 71 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x55ce84a1ce00 con 0x55ce83eaf000 2023-05-20T01:22:18.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.626+0000 7f3b53e91700 1 -- 172.21.15.16:0/3957359014 <== mon.1 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x56013b37e840 con 0x56013b664400 2023-05-20T01:22:18.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.627+0000 7f778029b700 1 -- 172.21.15.16:0/251369889 <== osd.1 v2:172.21.15.16:6808/3860264220 2 ==== osd_op_reply(2 100.00000000.inode [stat out=16b] v0'0 uv10 ondisk = 0) v8 ==== 162+0+16 (crc 0 0 0) 0x563d8a7ef440 con 0x563d8a80c800 2023-05-20T01:22:18.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.627+0000 7f7782aee7c0 1 --2- 172.21.15.16:0/251369889 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x563d8a80cc00 0x563d8a53db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.627+0000 7f7782aee7c0 1 -- 172.21.15.16:0/251369889 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:3 10.0 10:00000000::::head [pgnls-filter start_epoch 71 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x563d8a794e00 con 0x563d8a80cc00 2023-05-20T01:22:18.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.627+0000 7f3776825700 1 --2- 172.21.15.16:0/1569316897 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x55ce83eaf000 0x55ce83c20c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.626+0000 7fa3b3357700 1 -- 172.21.15.16:0/2935861361 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x555b4a4fdc00 msgr2=0x555b4a597600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:18.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.626+0000 7fa3b3357700 1 --2- 172.21.15.16:0/2935861361 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x555b4a4fdc00 0x555b4a597600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.627+0000 7fa3b3357700 1 -- 172.21.15.16:0/2935861361 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555b4a5dc800 msgr2=0x555b4a598100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:18.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.627+0000 7fa3b3357700 1 --2- 172.21.15.16:0/2935861361 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555b4a5dc800 0x555b4a598100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.627+0000 7fa3b3357700 1 -- 172.21.15.16:0/2935861361 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555b4a521380 con 0x555b4a5dc400 2023-05-20T01:22:18.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.627+0000 7fa3b3b58700 1 --2- 172.21.15.16:0/2935861361 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x555b4a4fdc00 0x555b4a597600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:22:18.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.627+0000 7f70cbb9b700 1 -- 172.21.15.16:0/3489021394 <== osd.1 v2:172.21.15.16:6808/3860264220 1 ==== osd_op_reply(1 1.00000000.inode [stat out=16b] v0'0 uv133 ondisk = 0) v8 ==== 160+0+16 (crc 0 0 0) 0x5636419b3440 con 0x5636419bc800 2023-05-20T01:22:18.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.627+0000 7f70ce3ee7c0 1 -- 172.21.15.16:0/3489021394 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:2 9.1f 9:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x56364246ca80 con 0x5636419bc800 2023-05-20T01:22:18.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.627+0000 7f3b53e91700 1 --2- 172.21.15.16:0/3957359014 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x56013b66c400 0x56013b3b9700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.627+0000 7f3b53e91700 1 -- 172.21.15.16:0/3957359014 <== mon.1 v2:172.21.15.132:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 7103+0+0 (secure 0 0 0) 0x56013b35ea80 con 0x56013b664400 2023-05-20T01:22:18.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.627+0000 7f3b5beef7c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-05-20T01:22:18.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.627+0000 7f3b5beef7c0 4 datascan.main: data pool 'cephfs_data' has ID 10 2023-05-20T01:22:18.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.627+0000 7f3b5beef7c0 4 datascan.main: opening data pool 'cephfs_data' 2023-05-20T01:22:18.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.627+0000 7f3b5beef7c0 1 --2- 172.21.15.16:0/3957359014 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x56013b664800 0x56013b3b8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.627+0000 7f3b5beef7c0 1 -- 172.21.15.16:0/3957359014 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:1 9.14 9:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x56013b5fc700 con 0x56013b664800 2023-05-20T01:22:18.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.627+0000 7fa3b3357700 1 --2- 172.21.15.16:0/2935861361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b4a5dc400 0x555b4a597b80 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x555b4a82a180 tx=0x555b4b396c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47ead893f6b7a726 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.627+0000 7f3b5a69e700 1 --2- 172.21.15.16:0/3957359014 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x56013b66c400 0x56013b3b9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.627+0000 7f778129d700 1 --2- 172.21.15.16:0/251369889 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x563d8a80cc00 0x563d8a53db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.627+0000 7f3b59e9d700 1 --2- 172.21.15.16:0/3957359014 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x56013b664800 0x56013b3b8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.627+0000 7f3776825700 1 --2- 172.21.15.16:0/1569316897 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x55ce83eaf000 0x55ce83c20c00 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.627+0000 7f70cbb9b700 1 -- 172.21.15.16:0/3489021394 <== osd.1 v2:172.21.15.16:6808/3860264220 2 ==== osd_op_reply(2 100.00000000.inode [stat out=16b] v0'0 uv10 ondisk = 0) v8 ==== 162+0+16 (crc 0 0 0) 0x5636419b3440 con 0x5636419bc800 2023-05-20T01:22:18.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.627+0000 7f70ce3ee7c0 1 --2- 172.21.15.16:0/3489021394 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x5636419bcc00 0x563641729180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.627+0000 7f70ce3ee7c0 1 -- 172.21.15.16:0/3489021394 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:3 10.0 10:00000000::::head [pgnls-filter start_epoch 71 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x56364246ce00 con 0x5636419bcc00 2023-05-20T01:22:18.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.627+0000 7fabbb597700 1 -- 172.21.15.16:0/2723465486 <== osd.1 v2:172.21.15.16:6808/3860264220 6 ==== osd_op_reply(14 1.00000000 [create,omap-set-header] v71'11 uv11 ondisk = 0) v8 ==== 196+0+0 (crc 0 0 0) 0x5614bdd37440 con 0x5614bdd38c00 2023-05-20T01:22:18.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.628+0000 7fabbddea7c0 10 datascan.find_or_create_dirfrag: Created dirfrag: 0x0x1 2023-05-20T01:22:18.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.628+0000 7fabbddea7c0 1 -- 172.21.15.16:0/2723465486 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:15 9.1f 9:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x5614be7fa700 con 0x5614bdd38c00 2023-05-20T01:22:18.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.628+0000 7f3b59e9d700 1 --2- 172.21.15.16:0/3957359014 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x56013b664800 0x56013b3b8100 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.628+0000 7f3b5a69e700 1 --2- 172.21.15.16:0/3957359014 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x56013b66c400 0x56013b3b9700 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x56013b377590 tx=0x56013c0fe060 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.628+0000 7f70ccb9d700 1 --2- 172.21.15.16:0/3489021394 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x5636419bcc00 0x563641729180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.628+0000 7f3776825700 1 -- 172.21.15.16:0/1569316897 <== osd.6 v2:172.21.15.132:6801/727581123 1 ==== osd_op_reply(3 [pgnls-filter start_epoch 71 out=81b] v71'12 uv12 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55ce84a258c0 con 0x55ce83eaf000 2023-05-20T01:22:18.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.628+0000 7fabbb597700 1 -- 172.21.15.16:0/2723465486 <== osd.1 v2:172.21.15.16:6808/3860264220 7 ==== osd_op_reply(15 1.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv11 ondisk = 0) v8 ==== 154+0+4 (crc 0 0 0) 0x5614bdd37440 con 0x5614bdd38c00 2023-05-20T01:22:18.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.628+0000 7fabbddea7c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-05-20T01:22:18.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.628+0000 7fabbddea7c0 20 datascan.read_dentry: subdir_head not found in result 2023-05-20T01:22:18.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.628+0000 7fabbddea7c0 10 datascan.inject_with_backtrace: Linking ancestor directory of inode 0x0x10000000000 at 0x0x1:subdir 2023-05-20T01:22:18.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.628+0000 7fabbddea7c0 1 -- 172.21.15.16:0/2723465486 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:16 9.1f 9:ff5b34d6:::1.00000000:head [omap-set-vals in=507b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x5614be7faa80 con 0x5614bdd38c00 2023-05-20T01:22:18.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.628+0000 7f778129d700 1 --2- 172.21.15.16:0/251369889 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x563d8a80cc00 0x563d8a53db80 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.628+0000 7f37780767c0 1 -- 172.21.15.16:0/1569316897 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:4 10.2d 10:b6db6db6::::head [pgnls-filter start_epoch 71 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x55ce84a1d180 con 0x55ce83eaf000 2023-05-20T01:22:18.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.628+0000 7f3b59e9d700 1 -- 172.21.15.16:0/3957359014 <== osd.1 v2:172.21.15.16:6808/3860264220 1 ==== osd_op_reply(1 1.00000000.inode [stat out=16b] v0'0 uv133 ondisk = 0) v8 ==== 160+0+16 (crc 0 0 0) 0x56013b657440 con 0x56013b664800 2023-05-20T01:22:18.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.628+0000 7f3b5beef7c0 1 -- 172.21.15.16:0/3957359014 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:2 9.1f 9:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x56013b5fca80 con 0x56013b664800 2023-05-20T01:22:18.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.628+0000 7f70ccb9d700 1 --2- 172.21.15.16:0/3489021394 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x5636419bcc00 0x563641729180 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.628+0000 7f90c7f98700 1 -- 172.21.15.16:0/3200652326 <== mon.1 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x556b16e1c840 con 0x556b170fe400 2023-05-20T01:22:18.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.628+0000 7f3776825700 1 -- 172.21.15.16:0/1569316897 <== osd.6 v2:172.21.15.132:6801/727581123 2 ==== osd_op_reply(4 [pgnls-filter start_epoch 71 out=49b] v69'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ce84a258c0 con 0x55ce83eaf000 2023-05-20T01:22:18.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.628+0000 7f377181b700 1 --2- 172.21.15.16:0/1569316897 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x55ce83fbe400 0x55ce83c21700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.628+0000 7f377181b700 1 -- 172.21.15.16:0/1569316897 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:5 10.1d 10:b8000000::::head [pgnls-filter start_epoch 71 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x55ce84a1ce00 con 0x55ce83fbe400 2023-05-20T01:22:18.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.628+0000 7f778129d700 1 -- 172.21.15.16:0/251369889 <== osd.6 v2:172.21.15.132:6801/727581123 1 ==== osd_op_reply(3 [pgnls-filter start_epoch 71 out=81b] v71'12 uv12 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x563d8a7ef680 con 0x563d8a80cc00 2023-05-20T01:22:18.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.628+0000 7f70ccb9d700 1 -- 172.21.15.16:0/3489021394 <== osd.6 v2:172.21.15.132:6801/727581123 1 ==== osd_op_reply(3 [pgnls-filter start_epoch 71 out=81b] v71'12 uv12 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5636419b3680 con 0x5636419bcc00 2023-05-20T01:22:18.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.628+0000 7f7782aee7c0 1 -- 172.21.15.16:0/251369889 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:4 10.12 10:49249249::::head [pgnls-filter start_epoch 71 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x563d8a794700 con 0x563d8a80cc00 2023-05-20T01:22:18.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.629+0000 7f70ce3ee7c0 1 -- 172.21.15.16:0/3489021394 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:4 10.24 10:24924924::::head [pgnls-filter start_epoch 71 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x56364246c700 con 0x5636419bc800 2023-05-20T01:22:18.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.629+0000 7f3776024700 1 --2- 172.21.15.16:0/1569316897 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x55ce83fbe400 0x55ce83c21700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.629+0000 7f90c7f98700 1 --2- 172.21.15.16:0/3200652326 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x556b17108800 0x556b16e57700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.629+0000 7f90c7f98700 1 -- 172.21.15.16:0/3200652326 <== mon.1 v2:172.21.15.132:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 7103+0+0 (secure 0 0 0) 0x556b16dfca80 con 0x556b170fe400 2023-05-20T01:22:18.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.629+0000 7f3b59e9d700 1 -- 172.21.15.16:0/3957359014 <== osd.1 v2:172.21.15.16:6808/3860264220 2 ==== osd_op_reply(2 100.00000000.inode [stat out=16b] v0'0 uv10 ondisk = 0) v8 ==== 162+0+16 (crc 0 0 0) 0x56013b657440 con 0x56013b664800 2023-05-20T01:22:18.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.629+0000 7f3b5beef7c0 1 --2- 172.21.15.16:0/3957359014 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x56013b664c00 0x56013b3b7b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.629+0000 7f3b5beef7c0 1 -- 172.21.15.16:0/3957359014 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:3 10.0 10:00000000::::head [pgnls-filter start_epoch 71 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x56013b5fce00 con 0x56013b664c00 2023-05-20T01:22:18.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.629+0000 7f90cfff67c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-05-20T01:22:18.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.629+0000 7f90cfff67c0 4 datascan.main: data pool 'cephfs_data' has ID 10 2023-05-20T01:22:18.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.629+0000 7f90cfff67c0 4 datascan.main: opening data pool 'cephfs_data' 2023-05-20T01:22:18.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.629+0000 7f778129d700 1 -- 172.21.15.16:0/251369889 <== osd.6 v2:172.21.15.132:6801/727581123 2 ==== osd_op_reply(4 [pgnls-filter start_epoch 71 out=49b] v69'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563d8a7ef680 con 0x563d8a80cc00 2023-05-20T01:22:18.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.629+0000 7f90cfff67c0 1 --2- 172.21.15.16:0/3200652326 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x556b170fe800 0x556b16e56100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.629+0000 7f90cfff67c0 1 -- 172.21.15.16:0/3200652326 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:1 9.14 9:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x556b17c5c700 con 0x556b170fe800 2023-05-20T01:22:18.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.629+0000 7f777ba92700 1 --2- 172.21.15.16:0/251369889 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x563d8a80f000 0x563d8a924000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.629+0000 7f777ba92700 1 -- 172.21.15.16:0/251369889 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:5 10.32 10:4c000000::::head [pgnls-filter start_epoch 71 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x563d8a794e00 con 0x563d8a80f000 2023-05-20T01:22:18.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.629+0000 7f3b5a69e700 1 --2- 172.21.15.16:0/3957359014 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x56013b664c00 0x56013b3b7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.629+0000 7f70cbb9b700 1 -- 172.21.15.16:0/3489021394 <== osd.1 v2:172.21.15.16:6808/3860264220 3 ==== osd_op_reply(4 [pgnls-filter start_epoch 71 out=49b] v69'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636419b3440 con 0x5636419bc800 2023-05-20T01:22:18.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.629+0000 7f70c7b93700 1 -- 172.21.15.16:0/3489021394 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:5 10.14 10:28000000::::head [pgnls-filter start_epoch 71 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x56364246d880 con 0x5636419bc800 2023-05-20T01:22:18.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.629+0000 7f3776024700 1 --2- 172.21.15.16:0/1569316897 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x55ce83fbe400 0x55ce83c21700 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.629+0000 7f90cdfa4700 1 --2- 172.21.15.16:0/3200652326 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x556b170fe800 0x556b16e56100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.629+0000 7f7780a9c700 1 --2- 172.21.15.16:0/251369889 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x563d8a80f000 0x563d8a924000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.629+0000 7f90ce7a5700 1 --2- 172.21.15.16:0/3200652326 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x556b17108800 0x556b16e57700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.629+0000 7f70cbb9b700 1 -- 172.21.15.16:0/3489021394 <== osd.1 v2:172.21.15.16:6808/3860264220 4 ==== osd_op_reply(5 [pgnls-filter start_epoch 71 out=49b] v69'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636419b3440 con 0x5636419bc800 2023-05-20T01:22:18.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.630+0000 7f70c7392700 1 --2- 172.21.15.16:0/3489021394 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x5636419f8800 0x563641afe000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.630+0000 7f70c7392700 1 -- 172.21.15.16:0/3489021394 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:6 10.34 10:2c000000::::head [pgnls-filter start_epoch 71 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x56364246dc00 con 0x5636419f8800 2023-05-20T01:22:18.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.629+0000 7fa3aeb4e700 1 -- 172.21.15.16:0/2935861361 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x555b4a531600 con 0x555b4a5dc400 2023-05-20T01:22:18.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.629+0000 7fa3aeb4e700 1 -- 172.21.15.16:0/2935861361 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555b4a54e480 con 0x555b4a5dc400 2023-05-20T01:22:18.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.630+0000 7fa3aeb4e700 1 -- 172.21.15.16:0/2935861361 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x555b4a5311e0 con 0x555b4a5dc400 2023-05-20T01:22:18.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.630+0000 7f3776024700 1 -- 172.21.15.16:0/1569316897 <== osd.4 v2:172.21.15.132:6817/506317223 1 ==== osd_op_reply(5 [pgnls-filter start_epoch 71 out=49b] v69'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ce84a25680 con 0x55ce83fbe400 2023-05-20T01:22:18.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.630+0000 7f3b5a69e700 1 --2- 172.21.15.16:0/3957359014 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x56013b664c00 0x56013b3b7b80 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.630+0000 7f377101a700 1 --2- 172.21.15.16:0/1569316897 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x55ce83fbe800 0x55ce83fe2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.630+0000 7f90cdfa4700 1 --2- 172.21.15.16:0/3200652326 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x556b170fe800 0x556b16e56100 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.630+0000 7f377101a700 1 -- 172.21.15.16:0/1569316897 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:6 10.3d 10:bc000000::::head [pgnls-filter start_epoch 71 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x55ce84a1c700 con 0x55ce83fbe800 2023-05-20T01:22:18.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.630+0000 7f70cc39c700 1 --2- 172.21.15.16:0/3489021394 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x5636419f8800 0x563641afe000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.630+0000 7f3775823700 1 --2- 172.21.15.16:0/1569316897 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x55ce83fbe800 0x55ce83fe2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.630+0000 7f7780a9c700 1 --2- 172.21.15.16:0/251369889 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x563d8a80f000 0x563d8a924000 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.630+0000 7f90ce7a5700 1 --2- 172.21.15.16:0/3200652326 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x556b17108800 0x556b16e57700 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x556b170f20c0 tx=0x556b17b9a0c0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.630+0000 7fabbb597700 1 -- 172.21.15.16:0/2723465486 <== osd.1 v2:172.21.15.16:6808/3860264220 8 ==== osd_op_reply(16 1.00000000 [omap-set-vals] v71'12 uv12 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x5614bdd37440 con 0x5614bdd38c00 2023-05-20T01:22:18.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.630+0000 7f90cdfa4700 1 -- 172.21.15.16:0/3200652326 <== osd.1 v2:172.21.15.16:6808/3860264220 1 ==== osd_op_reply(1 1.00000000.inode [stat out=16b] v0'0 uv133 ondisk = 0) v8 ==== 160+0+16 (crc 0 0 0) 0x556b170eb440 con 0x556b170fe800 2023-05-20T01:22:18.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.630+0000 7fabbddea7c0 20 datascan.inject_linkage: Injected dentry 0x0x1/subdir pointing to 0x0x10000000000 2023-05-20T01:22:18.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.630+0000 7f90cfff67c0 1 -- 172.21.15.16:0/3200652326 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:2 9.1f 9:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x556b17c5ca80 con 0x556b170fe800 2023-05-20T01:22:18.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.630+0000 7f3b5a69e700 1 -- 172.21.15.16:0/3957359014 <== osd.6 v2:172.21.15.132:6801/727581123 1 ==== osd_op_reply(3 [pgnls-filter start_epoch 71 out=81b] v71'12 uv12 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x56013b657680 con 0x56013b664c00 2023-05-20T01:22:18.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.630+0000 7fabbddea7c0 1 --2- 172.21.15.16:0/2723465486 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x5614bde80800 0x5614bde90000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.630+0000 7fabbddea7c0 1 -- 172.21.15.16:0/2723465486 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:17 10.20 10:04000000::::head [pgnls-filter start_epoch 71 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x5614be7fbc00 con 0x5614bde80800 2023-05-20T01:22:18.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.630+0000 7f3b5beef7c0 1 --2- 172.21.15.16:0/3957359014 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x56013b665000 0x56013b3b7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.630+0000 7f3b5beef7c0 1 -- 172.21.15.16:0/3957359014 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:4 10.1b 10:db6db6db::::head [pgnls-filter start_epoch 71 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x56013b5fd180 con 0x56013b665000 2023-05-20T01:22:18.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.630+0000 7f7780a9c700 1 -- 172.21.15.16:0/251369889 <== osd.7 v2:172.21.15.132:6800/466001149 1 ==== osd_op_reply(5 [pgnls-filter start_epoch 71 out=49b] v69'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563d8a7ef8c0 con 0x563d8a80f000 2023-05-20T01:22:18.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.630+0000 7f70cc39c700 1 --2- 172.21.15.16:0/3489021394 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x5636419f8800 0x563641afe000 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.630+0000 7f777c293700 1 -- 172.21.15.16:0/251369889 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:6 10.a 10:50000000::::head [pgnls-filter start_epoch 71 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x563d8a794700 con 0x563d8a80cc00 2023-05-20T01:22:18.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.630+0000 7f3775823700 1 --2- 172.21.15.16:0/1569316897 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x55ce83fbe800 0x55ce83fe2000 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.630+0000 7fa3b53a97c0 1 -- 172.21.15.16:0/2935861361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b4a5dc400 msgr2=0x555b4a597b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.630+0000 7fa3b53a97c0 1 --2- 172.21.15.16:0/2935861361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b4a5dc400 0x555b4a597b80 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x555b4a82a180 tx=0x555b4b396c00 comp rx=0 tx=0).stop 2023-05-20T01:22:18.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.631+0000 7f90cdfa4700 1 -- 172.21.15.16:0/3200652326 <== osd.1 v2:172.21.15.16:6808/3860264220 2 ==== osd_op_reply(2 100.00000000.inode [stat out=16b] v0'0 uv10 ondisk = 0) v8 ==== 162+0+16 (crc 0 0 0) 0x556b170eb440 con 0x556b170fe800 2023-05-20T01:22:18.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.631+0000 7fabbb597700 1 --2- 172.21.15.16:0/2723465486 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x5614bde80800 0x5614bde90000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.631+0000 7f90cfff67c0 1 --2- 172.21.15.16:0/3200652326 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x556b170fec00 0x556b16e55b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.631+0000 7f90cfff67c0 1 -- 172.21.15.16:0/3200652326 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:3 10.0 10:00000000::::head [pgnls-filter start_epoch 71 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x556b17c5ce00 con 0x556b170fec00 2023-05-20T01:22:18.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.631+0000 7f3b59e9d700 1 --2- 172.21.15.16:0/3957359014 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x56013b665000 0x56013b3b7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.631+0000 7f70cc39c700 1 -- 172.21.15.16:0/3489021394 <== osd.5 v2:172.21.15.132:6816/1844689995 1 ==== osd_op_reply(6 [pgnls-filter start_epoch 71 out=49b] v69'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636419b38c0 con 0x5636419f8800 2023-05-20T01:22:18.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.631+0000 7fa3b53a97c0 1 -- 172.21.15.16:0/2935861361 shutdown_connections 2023-05-20T01:22:18.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.631+0000 7fa3b53a97c0 1 --2- 172.21.15.16:0/2935861361 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555b4a5dc800 0x555b4a598100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.631+0000 7fa3b53a97c0 1 --2- 172.21.15.16:0/2935861361 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x555b4a4fdc00 0x555b4a597600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.631+0000 7fa3b53a97c0 1 --2- 172.21.15.16:0/2935861361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b4a5dc400 0x555b4a597b80 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.631+0000 7fa3b53a97c0 1 -- 172.21.15.16:0/2935861361 >> 172.21.15.16:0/2935861361 conn(0x555b4a4fd800 msgr2=0x555b4a5be000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:18.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.631+0000 7f70c7b93700 1 --2- 172.21.15.16:0/3489021394 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x5636419f8c00 0x563641afe580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.631+0000 7f70c7b93700 1 -- 172.21.15.16:0/3489021394 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:7 10.c 10:30000000::::head [pgnls-filter start_epoch 71 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x563641b04000 con 0x5636419f8c00 2023-05-20T01:22:18.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.631+0000 7f778129d700 1 -- 172.21.15.16:0/251369889 <== osd.6 v2:172.21.15.132:6801/727581123 3 ==== osd_op_reply(6 [pgnls-filter start_epoch 71 out=49b] v69'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563d8a7ef680 con 0x563d8a80cc00 2023-05-20T01:22:18.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.631+0000 7f777ba92700 1 --2- 172.21.15.16:0/251369889 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x563d8a80f400 0x563d8a924580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.631+0000 7f3775823700 1 -- 172.21.15.16:0/1569316897 <== osd.7 v2:172.21.15.132:6800/466001149 1 ==== osd_op_reply(6 [pgnls-filter start_epoch 71 out=81b] v71'11 uv11 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55ce84a25b00 con 0x55ce83fbe800 2023-05-20T01:22:18.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.631+0000 7f777ba92700 1 -- 172.21.15.16:0/251369889 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:7 10.2a 10:54000000::::head [pgnls-filter start_epoch 71 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x563d8a795880 con 0x563d8a80f400 2023-05-20T01:22:18.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.631+0000 7f90ce7a5700 1 --2- 172.21.15.16:0/3200652326 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x556b170fec00 0x556b16e55b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.631+0000 7f37780767c0 20 datascan.forall_objects: OSD matched oid 10000000205.00000000 2023-05-20T01:22:18.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.631+0000 7f37780767c0 10 datascan.operator(): handling object 10000000205.0 2023-05-20T01:22:18.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.631+0000 7f37780767c0 1 -- 172.21.15.16:0/1569316897 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:7 10.3d 10:bfcd3380:::10000000205.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x55ce84a1d500 con 0x55ce83fbe800 2023-05-20T01:22:18.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.631+0000 7f70cbb9b700 1 --2- 172.21.15.16:0/3489021394 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x5636419f8c00 0x563641afe580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.631+0000 7fabbb597700 1 --2- 172.21.15.16:0/2723465486 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x5614bde80800 0x5614bde90000 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.631+0000 7f778029b700 1 --2- 172.21.15.16:0/251369889 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x563d8a80f400 0x563d8a924580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.631+0000 7fa3b53a97c0 1 -- 172.21.15.16:0/2935861361 shutdown_connections 2023-05-20T01:22:18.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.631+0000 7f3775823700 1 -- 172.21.15.16:0/1569316897 <== osd.7 v2:172.21.15.132:6800/466001149 2 ==== osd_op_reply(7 10000000205.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (98) out=98b,getxattr,getxattr] v0'0 uv11 ondisk = 0) v8 ==== 416+0+144 (crc 0 0 0) 0x55ce84a25b00 con 0x55ce83fbe800 2023-05-20T01:22:18.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.631+0000 7f37780767c0 1 -- 172.21.15.16:0/1569316897 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:8 11.1d 11:bfcd3380:::10000000205.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x55ce84a1dc00 con 0x55ce83fbe800 2023-05-20T01:22:18.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.631+0000 7f3b59e9d700 1 --2- 172.21.15.16:0/3957359014 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x56013b665000 0x56013b3b7600 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.631+0000 7f90ce7a5700 1 --2- 172.21.15.16:0/3200652326 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x556b170fec00 0x556b16e55b80 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.632+0000 7fabbb597700 1 -- 172.21.15.16:0/2723465486 <== osd.0 v2:172.21.15.16:6824/3923634179 1 ==== osd_op_reply(17 [pgnls-filter start_epoch 71 out=81b] v71'13 uv13 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5614bdd37440 con 0x5614bde80800 2023-05-20T01:22:18.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.632+0000 7fabbddea7c0 20 datascan.forall_objects: OSD matched oid 1000000020b.00000000 2023-05-20T01:22:18.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.632+0000 7fabbddea7c0 10 datascan.operator(): handling object 1000000020b.0 2023-05-20T01:22:18.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.632+0000 7fabbddea7c0 1 -- 172.21.15.16:0/2723465486 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:18 10.20 10:05552577:::1000000020b.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x5614be7fae00 con 0x5614bde80800 2023-05-20T01:22:18.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.632+0000 7f3b59e9d700 1 -- 172.21.15.16:0/3957359014 <== osd.3 v2:172.21.15.16:6800/3669781890 1 ==== osd_op_reply(4 [pgnls-filter start_epoch 71 out=49b] v69'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56013b657440 con 0x56013b665000 2023-05-20T01:22:18.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.632+0000 7f778029b700 1 --2- 172.21.15.16:0/251369889 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x563d8a80f400 0x563d8a924580 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.632+0000 7f70cbb9b700 1 --2- 172.21.15.16:0/3489021394 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x5636419f8c00 0x563641afe580 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.632+0000 7f3b55694700 1 -- 172.21.15.16:0/3957359014 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:5 10.3b 10:dc000000::::head [pgnls-filter start_epoch 71 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x56013b5fc700 con 0x56013b664c00 2023-05-20T01:22:18.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.632+0000 7f90ce7a5700 1 -- 172.21.15.16:0/3200652326 <== osd.6 v2:172.21.15.132:6801/727581123 1 ==== osd_op_reply(3 [pgnls-filter start_epoch 71 out=81b] v71'12 uv12 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x556b170eb680 con 0x556b170fec00 2023-05-20T01:22:18.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.632+0000 7f90cfff67c0 1 --2- 172.21.15.16:0/3200652326 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x556b170ff000 0x556b16e55600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.632+0000 7f90cfff67c0 1 -- 172.21.15.16:0/3200652326 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:4 10.36 10:6db6db6d::::head [pgnls-filter start_epoch 71 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x556b17c5d180 con 0x556b170ff000 2023-05-20T01:22:18.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.631+0000 7fa3b53a97c0 1 -- 172.21.15.16:0/2935861361 wait complete. 2023-05-20T01:22:18.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.632+0000 7fa3b53a97c0 1 Processor -- start 2023-05-20T01:22:18.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.632+0000 7f70cbb9b700 1 -- 172.21.15.16:0/3489021394 <== osd.7 v2:172.21.15.132:6800/466001149 1 ==== osd_op_reply(7 [pgnls-filter start_epoch 71 out=81b] v71'10 uv10 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5636419b3440 con 0x5636419f8c00 2023-05-20T01:22:18.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.632+0000 7f70ce3ee7c0 20 datascan.forall_objects: OSD matched oid 100000001fc.00000000 2023-05-20T01:22:18.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.632+0000 7f70ce3ee7c0 10 datascan.operator(): handling object 100000001fc.0 2023-05-20T01:22:18.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.632+0000 7f70ce3ee7c0 1 -- 172.21.15.16:0/3489021394 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:8 10.c 10:30604137:::100000001fc.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x56364246ca80 con 0x5636419f8c00 2023-05-20T01:22:18.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.632+0000 7f3b5a69e700 1 -- 172.21.15.16:0/3957359014 <== osd.6 v2:172.21.15.132:6801/727581123 2 ==== osd_op_reply(5 [pgnls-filter start_epoch 71 out=81b] v71'8 uv8 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x56013b657680 con 0x56013b664c00 2023-05-20T01:22:18.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.632+0000 7f3775823700 1 -- 172.21.15.16:0/1569316897 <== osd.7 v2:172.21.15.132:6800/466001149 3 ==== osd_op_reply(8 10000000205.00000000 [getxattr (30) out=30b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x55ce84a25b00 con 0x55ce83fbe800 2023-05-20T01:22:18.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.632+0000 7f37780767c0 20 datascan.operator(): loaded layout from xattr: pi: 11 os: 4194304 sc: 1 su: 4194304 2023-05-20T01:22:18.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.632+0000 7f37780767c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000205 2023-05-20T01:22:18.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.632+0000 7f37780767c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/16 2023-05-20T01:22:18.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.632+0000 7f37780767c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=16 2023-05-20T01:22:18.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.632+0000 7f3b5beef7c0 20 datascan.forall_objects: OSD matched oid 100000001fe.00000000 2023-05-20T01:22:18.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.632+0000 7f3b5beef7c0 10 datascan.operator(): handling object 100000001fe.0 2023-05-20T01:22:18.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.632+0000 7f778029b700 1 -- 172.21.15.16:0/251369889 <== osd.3 v2:172.21.15.16:6800/3669781890 1 ==== osd_op_reply(7 [pgnls-filter start_epoch 71 out=101b] v71'17 uv17 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x563d8a7ef440 con 0x563d8a80f400 2023-05-20T01:22:18.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.632+0000 7f7782aee7c0 20 datascan.forall_objects: OSD matched oid 1000000020a.00000000 2023-05-20T01:22:18.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.632+0000 7f7782aee7c0 10 datascan.operator(): handling object 1000000020a.0 2023-05-20T01:22:18.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.632+0000 7f7782aee7c0 1 -- 172.21.15.16:0/251369889 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:8 10.2a 10:55a6cd83:::1000000020a.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x563d8a794a80 con 0x563d8a80f400 2023-05-20T01:22:18.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.632+0000 7f37780767c0 1 -- 172.21.15.16:0/1569316897 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:9 9.10 9:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x55ce84a1c700 con 0x55ce83fbe400 2023-05-20T01:22:18.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.632+0000 7fabbb597700 1 -- 172.21.15.16:0/2723465486 <== osd.0 v2:172.21.15.16:6824/3923634179 2 ==== osd_op_reply(18 1000000020b.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (98) out=98b,getxattr,getxattr] v0'0 uv13 ondisk = 0) v8 ==== 416+0+144 (crc 0 0 0) 0x5614bdd37440 con 0x5614bde80800 2023-05-20T01:22:18.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.632+0000 7fabbddea7c0 1 -- 172.21.15.16:0/2723465486 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:19 11.0 11:05552577:::1000000020b.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x5614be7fa700 con 0x5614bdd38c00 2023-05-20T01:22:18.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.632+0000 7f90cdfa4700 1 --2- 172.21.15.16:0/3200652326 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x556b170ff000 0x556b16e55600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.632+0000 7f3b5beef7c0 1 -- 172.21.15.16:0/3957359014 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:6 10.3b 10:deccbd37:::100000001fe.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x56013b5fd500 con 0x56013b664c00 2023-05-20T01:22:18.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.632+0000 7fa3b53a97c0 1 -- start start 2023-05-20T01:22:18.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.633+0000 7f70cbb9b700 1 -- 172.21.15.16:0/3489021394 <== osd.7 v2:172.21.15.132:6800/466001149 2 ==== osd_op_reply(8 100000001fc.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (97) out=97b,getxattr,getxattr] v0'0 uv10 ondisk = 0) v8 ==== 416+0+143 (crc 0 0 0) 0x5636419b3440 con 0x5636419f8c00 2023-05-20T01:22:18.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.633+0000 7f70ce3ee7c0 1 -- 172.21.15.16:0/3489021394 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:9 11.c 11:30604137:::100000001fc.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x56364246d500 con 0x5636419bc800 2023-05-20T01:22:18.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.633+0000 7f3776024700 1 -- 172.21.15.16:0/1569316897 <== osd.4 v2:172.21.15.132:6817/506317223 2 ==== osd_op_reply(9 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x55ce84a25680 con 0x55ce83fbe400 2023-05-20T01:22:18.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.633+0000 7f37780767c0 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-05-20T01:22:18.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.633+0000 7f37780767c0 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-05-20T01:22:18.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.633+0000 7f37780767c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/16, will insert to root 2023-05-20T01:22:18.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.633+0000 7f37780767c0 1 -- 172.21.15.16:0/1569316897 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:10 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x55ce84a1d880 con 0x55ce83fbe400 2023-05-20T01:22:18.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.633+0000 7fabbb597700 1 -- 172.21.15.16:0/2723465486 <== osd.1 v2:172.21.15.16:6808/3860264220 9 ==== osd_op_reply(19 1000000020b.00000000 [getxattr (30) out=30b] v0'0 uv5 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x5614bdd37440 con 0x5614bdd38c00 2023-05-20T01:22:18.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.633+0000 7fabbddea7c0 20 datascan.operator(): loaded layout from xattr: pi: 11 os: 4194304 sc: 1 su: 4194304 2023-05-20T01:22:18.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.633+0000 7fabbddea7c0 10 datascan.inject_with_backtrace: inode: 0x0x1000000020b 2023-05-20T01:22:18.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.633+0000 7fabbddea7c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/22 2023-05-20T01:22:18.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.633+0000 7fabbddea7c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=22 2023-05-20T01:22:18.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.633+0000 7fabbddea7c0 1 -- 172.21.15.16:0/2723465486 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:20 9.10 9:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x5614be7faa80 con 0x5614bde80400 2023-05-20T01:22:18.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.633+0000 7f3b5a69e700 1 -- 172.21.15.16:0/3957359014 <== osd.6 v2:172.21.15.132:6801/727581123 3 ==== osd_op_reply(6 100000001fe.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (97) out=97b,getxattr,getxattr] v0'0 uv8 ondisk = 0) v8 ==== 416+0+143 (crc 0 0 0) 0x56013b657680 con 0x56013b664c00 2023-05-20T01:22:18.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.633+0000 7f778029b700 1 -- 172.21.15.16:0/251369889 <== osd.3 v2:172.21.15.16:6800/3669781890 2 ==== osd_op_reply(8 1000000020a.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (98) out=98b,getxattr,getxattr] v0'0 uv16 ondisk = 0) v8 ==== 416+0+144 (crc 0 0 0) 0x563d8a7ef440 con 0x563d8a80f400 2023-05-20T01:22:18.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.633+0000 7f7782aee7c0 1 -- 172.21.15.16:0/251369889 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:9 11.a 11:55a6cd83:::1000000020a.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x563d8a795500 con 0x563d8a80f400 2023-05-20T01:22:18.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.633+0000 7f3b5beef7c0 1 --2- 172.21.15.16:0/3957359014 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x56013b665800 0x56013b78c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.633+0000 7f3b5beef7c0 1 -- 172.21.15.16:0/3957359014 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:7 11.1b 11:deccbd37:::100000001fe.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x56013b5fdc00 con 0x56013b665800 2023-05-20T01:22:18.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.633+0000 7f90cdfa4700 1 --2- 172.21.15.16:0/3200652326 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x556b170ff000 0x556b16e55600 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.633+0000 7f3776024700 1 -- 172.21.15.16:0/1569316897 <== osd.4 v2:172.21.15.132:6817/506317223 3 ==== osd_op_reply(10 10000000000.00000000 [omap-get-header out=274b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x55ce84a25680 con 0x55ce83fbe400 2023-05-20T01:22:18.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.633+0000 7f37780767c0 10 datascan.find_or_create_dirfrag: read_version = 17 2023-05-20T01:22:18.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.633+0000 7f37780767c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-05-20T01:22:18.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.633+0000 7f37780767c0 1 -- 172.21.15.16:0/1569316897 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:11 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x55ce84a1ca80 con 0x55ce83fbe400 2023-05-20T01:22:18.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.633+0000 7f3b5969c700 1 --2- 172.21.15.16:0/3957359014 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x56013b665800 0x56013b78c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.633+0000 7f90cdfa4700 1 -- 172.21.15.16:0/3200652326 <== osd.7 v2:172.21.15.132:6800/466001149 1 ==== osd_op_reply(4 [pgnls-filter start_epoch 71 out=101b] v71'23 uv23 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x556b170eb440 con 0x556b170ff000 2023-05-20T01:22:18.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.633+0000 7f90cfff67c0 20 datascan.forall_objects: OSD matched oid 100000001f9.00000000 2023-05-20T01:22:18.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.633+0000 7f90cfff67c0 10 datascan.operator(): handling object 100000001f9.0 2023-05-20T01:22:18.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.633+0000 7f90cfff67c0 1 -- 172.21.15.16:0/3200652326 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:5 10.36 10:6ee5e2c4:::100000001f9.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x556b17c5d500 con 0x556b170ff000 2023-05-20T01:22:18.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.633+0000 7f70cbb9b700 1 -- 172.21.15.16:0/3489021394 <== osd.1 v2:172.21.15.16:6808/3860264220 5 ==== osd_op_reply(9 100000001fc.00000000 [getxattr (30) out=30b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x5636419b3440 con 0x5636419bc800 2023-05-20T01:22:18.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.633+0000 7f778029b700 1 -- 172.21.15.16:0/251369889 <== osd.3 v2:172.21.15.16:6800/3669781890 3 ==== osd_op_reply(9 1000000020a.00000000 [getxattr (30) out=30b] v0'0 uv6 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x563d8a7ef440 con 0x563d8a80f400 2023-05-20T01:22:18.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.633+0000 7f70ce3ee7c0 20 datascan.operator(): loaded layout from xattr: pi: 11 os: 4194304 sc: 1 su: 4194304 2023-05-20T01:22:18.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.633+0000 7f70ce3ee7c0 10 datascan.inject_with_backtrace: inode: 0x0x100000001fc 2023-05-20T01:22:18.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.633+0000 7f70ce3ee7c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/7 2023-05-20T01:22:18.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.633+0000 7f70ce3ee7c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=7 2023-05-20T01:22:18.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.633+0000 7f70ce3ee7c0 1 --2- 172.21.15.16:0/3489021394 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x5636419bd400 0x563641afeb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.633+0000 7f70ce3ee7c0 1 -- 172.21.15.16:0/3489021394 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:10 9.10 9:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x563641b04000 con 0x5636419bd400 2023-05-20T01:22:18.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.633+0000 7f7782aee7c0 20 datascan.operator(): loaded layout from xattr: pi: 11 os: 4194304 sc: 1 su: 4194304 2023-05-20T01:22:18.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.633+0000 7f7782aee7c0 10 datascan.inject_with_backtrace: inode: 0x0x1000000020a 2023-05-20T01:22:18.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.633+0000 7f7782aee7c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/21 2023-05-20T01:22:18.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.633+0000 7f7782aee7c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=21 2023-05-20T01:22:18.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.633+0000 7f3776024700 1 -- 172.21.15.16:0/1569316897 <== osd.4 v2:172.21.15.132:6817/506317223 4 ==== osd_op_reply(11 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x55ce84a25680 con 0x55ce83fbe400 2023-05-20T01:22:18.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.633+0000 7f37780767c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=16 frag=*, r=0 2023-05-20T01:22:18.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.633+0000 7f37780767c0 20 datascan.read_dentry: 16_head not found in result 2023-05-20T01:22:18.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.633+0000 7f37780767c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000205 at 0x0x10000000000/16 with size=6291456 bytes 2023-05-20T01:22:18.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.634+0000 7f70ccb9d700 1 --2- 172.21.15.16:0/3489021394 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x5636419bd400 0x563641afeb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.633+0000 7f37780767c0 1 -- 172.21.15.16:0/1569316897 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:12 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x55ce84a1ce00 con 0x55ce83fbe400 2023-05-20T01:22:18.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.633+0000 7f7782aee7c0 1 --2- 172.21.15.16:0/251369889 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x563d8a80d400 0x563d8a53d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.633+0000 7f7782aee7c0 1 -- 172.21.15.16:0/251369889 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:10 9.10 9:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x563d8a93a000 con 0x563d8a80d400 2023-05-20T01:22:18.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.634+0000 7f3b5969c700 1 --2- 172.21.15.16:0/3957359014 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x56013b665800 0x56013b78c000 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.634+0000 7f90cdfa4700 1 -- 172.21.15.16:0/3200652326 <== osd.7 v2:172.21.15.132:6800/466001149 2 ==== osd_op_reply(5 100000001f9.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (97) out=97b,getxattr,getxattr] v0'0 uv22 ondisk = 0) v8 ==== 416+0+143 (crc 0 0 0) 0x556b170eb440 con 0x556b170ff000 2023-05-20T01:22:18.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.634+0000 7f90cfff67c0 1 --2- 172.21.15.16:0/3200652326 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x556b170ff800 0x556b1721c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.634+0000 7f90cfff67c0 1 -- 172.21.15.16:0/3200652326 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:6 11.16 11:6ee5e2c4:::100000001f9.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x556b17c5dc00 con 0x556b170ff800 2023-05-20T01:22:18.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.633+0000 7fa3b53a97c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555b4a5dd000 0x555b4a599180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.633+0000 7fa3b53a97c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x555b4a5dcc00 0x555b4a598c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.633+0000 7fa3b53a97c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b4a5dc800 0x555b4a598680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.633+0000 7fa3b53a97c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b4a538000 con 0x555b4a5dc800 2023-05-20T01:22:18.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.633+0000 7fa3b53a97c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b4a539180 con 0x555b4a5dd000 2023-05-20T01:22:18.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.633+0000 7fa3b53a97c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x555b4a539040 con 0x555b4a5dcc00 2023-05-20T01:22:18.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.634+0000 7fa3b2b56700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b4a5dc800 0x555b4a598680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.634+0000 7fa3b3357700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x555b4a5dcc00 0x555b4a598c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.634+0000 7fa3b2b56700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b4a5dc800 0x555b4a598680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35100/0 (socket says 172.21.15.16:35100) 2023-05-20T01:22:18.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.634+0000 7fa3b2b56700 1 -- 172.21.15.16:0/2859455639 learned_addr learned my addr 172.21.15.16:0/2859455639 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:22:18.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.634+0000 7fa3b3b58700 1 --2- 172.21.15.16:0/2859455639 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555b4a5dd000 0x555b4a599180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.634+0000 7f778129d700 1 --2- 172.21.15.16:0/251369889 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x563d8a80d400 0x563d8a53d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.634+0000 7f90cd7a3700 1 --2- 172.21.15.16:0/3200652326 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x556b170ff800 0x556b1721c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.634+0000 7fabbbd98700 1 -- 172.21.15.16:0/2723465486 <== osd.4 v2:172.21.15.132:6817/506317223 6 ==== osd_op_reply(20 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x5614bdd378c0 con 0x5614bde80400 2023-05-20T01:22:18.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.634+0000 7fabbddea7c0 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-05-20T01:22:18.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.634+0000 7fabbddea7c0 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-05-20T01:22:18.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.634+0000 7fabbddea7c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/22, will insert to root 2023-05-20T01:22:18.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.634+0000 7fabbddea7c0 1 -- 172.21.15.16:0/2723465486 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:21 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x5614be7fbc00 con 0x5614bde80400 2023-05-20T01:22:18.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.634+0000 7fa3b2b56700 1 -- 172.21.15.16:0/2859455639 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x555b4a5dcc00 msgr2=0x555b4a598c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.634+0000 7fa3b2b56700 1 --2- 172.21.15.16:0/2859455639 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x555b4a5dcc00 0x555b4a598c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.634+0000 7f70ccb9d700 1 --2- 172.21.15.16:0/3489021394 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x5636419bd400 0x563641afeb00 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.634+0000 7fa3b2b56700 1 -- 172.21.15.16:0/2859455639 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555b4a5dd000 msgr2=0x555b4a599180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.634+0000 7fa3b2b56700 1 --2- 172.21.15.16:0/2859455639 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555b4a5dd000 0x555b4a599180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.634+0000 7fa3b2b56700 1 -- 172.21.15.16:0/2859455639 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555b4b2761a0 con 0x555b4a5dc800 2023-05-20T01:22:18.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.634+0000 7fa3b3357700 1 --2- 172.21.15.16:0/2859455639 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x555b4a5dcc00 0x555b4a598c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-05-20T01:22:18.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.634+0000 7f3b5969c700 1 -- 172.21.15.16:0/3957359014 <== osd.4 v2:172.21.15.132:6817/506317223 1 ==== osd_op_reply(7 100000001fe.00000000 [getxattr (30) out=30b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x56013b6578c0 con 0x56013b665800 2023-05-20T01:22:18.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.634+0000 7f3b5beef7c0 20 datascan.operator(): loaded layout from xattr: pi: 11 os: 4194304 sc: 1 su: 4194304 2023-05-20T01:22:18.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.634+0000 7f3b5beef7c0 10 datascan.inject_with_backtrace: inode: 0x0x100000001fe 2023-05-20T01:22:18.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.634+0000 7f3b5beef7c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/9 2023-05-20T01:22:18.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.634+0000 7f3b5beef7c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=9 2023-05-20T01:22:18.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.634+0000 7f3b5beef7c0 1 -- 172.21.15.16:0/3957359014 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:8 9.10 9:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x56013b5fce00 con 0x56013b665800 2023-05-20T01:22:18.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.634+0000 7fa3b2b56700 1 --2- 172.21.15.16:0/2859455639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b4a5dc800 0x555b4a598680 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x555b4b2d9350 tx=0x555b4a56d620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.634+0000 7f778129d700 1 --2- 172.21.15.16:0/251369889 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x563d8a80d400 0x563d8a53d600 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.634+0000 7fa3ad34b700 1 -- 172.21.15.16:0/2859455639 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x555b4a530f20 con 0x555b4a5dc800 2023-05-20T01:22:18.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.634+0000 7fa3ad34b700 1 -- 172.21.15.16:0/2859455639 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555b4a54fe00 con 0x555b4a5dc800 2023-05-20T01:22:18.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.634+0000 7fa3b53a97c0 1 -- 172.21.15.16:0/2859455639 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x555b4b276000 con 0x555b4a5dc800 2023-05-20T01:22:18.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.634+0000 7fa3ad34b700 1 -- 172.21.15.16:0/2859455639 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x555b4a531600 con 0x555b4a5dc800 2023-05-20T01:22:18.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.635+0000 7fa3b53a97c0 1 -- 172.21.15.16:0/2859455639 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555b4b276340 con 0x555b4a5dc800 2023-05-20T01:22:18.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.635+0000 7fa3b53a97c0 4 datascan.init: resolving metadata pool 9 2023-05-20T01:22:18.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.635+0000 7f90cd7a3700 1 --2- 172.21.15.16:0/3200652326 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x556b170ff800 0x556b1721c000 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.635+0000 7f3776024700 1 -- 172.21.15.16:0/1569316897 <== osd.4 v2:172.21.15.132:6817/506317223 5 ==== osd_op_reply(12 10000000000.00000000 [omap-set-vals] v71'18 uv18 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55ce84a25680 con 0x55ce83fbe400 2023-05-20T01:22:18.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.635+0000 7f37780767c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/16 pointing to 0x0x10000000205 2023-05-20T01:22:18.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.635+0000 7f37780767c0 1 --2- 172.21.15.16:0/1569316897 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x55ce83fbec00 0x55ce83fe2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.635+0000 7f37780767c0 1 -- 172.21.15.16:0/1569316897 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:13 10.3 10:c0000000::::head [pgnls-filter start_epoch 71 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x55ce84a1d180 con 0x55ce83fbec00 2023-05-20T01:22:18.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.635+0000 7f90cd7a3700 1 -- 172.21.15.16:0/3200652326 <== osd.5 v2:172.21.15.132:6816/1844689995 1 ==== osd_op_reply(6 100000001f9.00000000 [getxattr (30) out=30b] v0'0 uv5 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x556b170eb8c0 con 0x556b170ff800 2023-05-20T01:22:18.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.635+0000 7f90cfff67c0 20 datascan.operator(): loaded layout from xattr: pi: 11 os: 4194304 sc: 1 su: 4194304 2023-05-20T01:22:18.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.635+0000 7f90cfff67c0 10 datascan.inject_with_backtrace: inode: 0x0x100000001f9 2023-05-20T01:22:18.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.635+0000 7f90cfff67c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/4 2023-05-20T01:22:18.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.635+0000 7f90cfff67c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=4 2023-05-20T01:22:18.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.635+0000 7f3776825700 1 --2- 172.21.15.16:0/1569316897 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x55ce83fbec00 0x55ce83fe2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.635+0000 7f90cfff67c0 1 --2- 172.21.15.16:0/3200652326 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x556b170ffc00 0x556b1721c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.635+0000 7f90cfff67c0 1 -- 172.21.15.16:0/3200652326 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:7 9.10 9:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x556b17c5c700 con 0x556b170ffc00 2023-05-20T01:22:18.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.636+0000 7fabbbd98700 1 -- 172.21.15.16:0/2723465486 <== osd.4 v2:172.21.15.132:6817/506317223 7 ==== osd_op_reply(21 10000000000.00000000 [omap-get-header out=274b] v0'0 uv18 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5614bdd378c0 con 0x5614bde80400 2023-05-20T01:22:18.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.636+0000 7fabbddea7c0 10 datascan.find_or_create_dirfrag: read_version = 18 2023-05-20T01:22:18.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.636+0000 7fabbddea7c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-05-20T01:22:18.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.636+0000 7fabbddea7c0 1 -- 172.21.15.16:0/2723465486 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:22 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x5614be7fb180 con 0x5614bde80400 2023-05-20T01:22:18.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.636+0000 7f90ce7a5700 1 --2- 172.21.15.16:0/3200652326 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x556b170ffc00 0x556b1721c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.636+0000 7f70ccb9d700 1 -- 172.21.15.16:0/3489021394 <== osd.4 v2:172.21.15.132:6817/506317223 1 ==== osd_op_reply(10 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x5636419b3680 con 0x5636419bd400 2023-05-20T01:22:18.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.636+0000 7f70ce3ee7c0 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-05-20T01:22:18.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.636+0000 7f70ce3ee7c0 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-05-20T01:22:18.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.636+0000 7f70ce3ee7c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/7, will insert to root 2023-05-20T01:22:18.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.636+0000 7f70ce3ee7c0 1 -- 172.21.15.16:0/3489021394 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:11 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x563641b04700 con 0x5636419bd400 2023-05-20T01:22:18.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.636+0000 7f3776825700 1 --2- 172.21.15.16:0/1569316897 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x55ce83fbec00 0x55ce83fe2580 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.636+0000 7f3b5969c700 1 -- 172.21.15.16:0/3957359014 <== osd.4 v2:172.21.15.132:6817/506317223 2 ==== osd_op_reply(8 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x56013b6578c0 con 0x56013b665800 2023-05-20T01:22:18.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.636+0000 7f3b5beef7c0 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-05-20T01:22:18.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.636+0000 7f3b5beef7c0 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-05-20T01:22:18.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.636+0000 7f3b5beef7c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/9, will insert to root 2023-05-20T01:22:18.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.636+0000 7f3b5beef7c0 1 -- 172.21.15.16:0/3957359014 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:9 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x56013b5fc700 con 0x56013b665800 2023-05-20T01:22:18.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.636+0000 7fa3ad34b700 1 -- 172.21.15.16:0/2859455639 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x555b4a55f600 con 0x555b4a5dc800 2023-05-20T01:22:18.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.636+0000 7f778129d700 1 -- 172.21.15.16:0/251369889 <== osd.4 v2:172.21.15.132:6817/506317223 1 ==== osd_op_reply(10 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x563d8a7ef680 con 0x563d8a80d400 2023-05-20T01:22:18.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.636+0000 7f7782aee7c0 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-05-20T01:22:18.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.636+0000 7f7782aee7c0 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-05-20T01:22:18.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.636+0000 7f7782aee7c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/21, will insert to root 2023-05-20T01:22:18.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.636+0000 7f90ce7a5700 1 --2- 172.21.15.16:0/3200652326 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x556b170ffc00 0x556b1721c580 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.636+0000 7f7782aee7c0 1 -- 172.21.15.16:0/251369889 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:11 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x563d8a93a380 con 0x563d8a80d400 2023-05-20T01:22:18.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.636+0000 7fabbbd98700 1 -- 172.21.15.16:0/2723465486 <== osd.4 v2:172.21.15.132:6817/506317223 8 ==== osd_op_reply(22 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv18 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x5614bdd378c0 con 0x5614bde80400 2023-05-20T01:22:18.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.636+0000 7f3776825700 1 -- 172.21.15.16:0/1569316897 <== osd.2 v2:172.21.15.16:6816/293187030 1 ==== osd_op_reply(13 [pgnls-filter start_epoch 71 out=49b] v69'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ce84a258c0 con 0x55ce83fbec00 2023-05-20T01:22:18.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.636+0000 7fabbddea7c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=22 frag=*, r=0 2023-05-20T01:22:18.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.636+0000 7fabbddea7c0 20 datascan.read_dentry: 22_head not found in result 2023-05-20T01:22:18.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.636+0000 7fabbddea7c0 10 datascan.inject_with_backtrace: Linking inode 0x0x1000000020b at 0x0x10000000000/22 with size=6291456 bytes 2023-05-20T01:22:18.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.636+0000 7fabbddea7c0 1 -- 172.21.15.16:0/2723465486 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:23 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x5614be7fae00 con 0x5614bde80400 2023-05-20T01:22:18.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.636+0000 7f377101a700 1 -- 172.21.15.16:0/1569316897 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:14 10.23 10:c4000000::::head [pgnls-filter start_epoch 71 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x55ce84a1c700 con 0x55ce83fbe800 2023-05-20T01:22:18.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.636+0000 7f70ccb9d700 1 -- 172.21.15.16:0/3489021394 <== osd.4 v2:172.21.15.132:6817/506317223 2 ==== osd_op_reply(11 10000000000.00000000 [omap-get-header out=274b] v0'0 uv18 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5636419b3680 con 0x5636419bd400 2023-05-20T01:22:18.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.636+0000 7f70ce3ee7c0 10 datascan.find_or_create_dirfrag: read_version = 18 2023-05-20T01:22:18.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.636+0000 7f70ce3ee7c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-05-20T01:22:18.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.636+0000 7f70ce3ee7c0 1 -- 172.21.15.16:0/3489021394 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:12 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x563641b04a80 con 0x5636419bd400 2023-05-20T01:22:18.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.636+0000 7fa3ad34b700 1 --2- 172.21.15.16:0/2859455639 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x555b4a930000 0x555b4a599700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.636+0000 7fa3ad34b700 1 -- 172.21.15.16:0/2859455639 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 7103+0+0 (secure 0 0 0) 0x555b4a53ea80 con 0x555b4a5dc800 2023-05-20T01:22:18.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.637+0000 7fa3b53a97c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-05-20T01:22:18.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.637+0000 7fa3b53a97c0 4 datascan.main: data pool 'cephfs_data' has ID 10 2023-05-20T01:22:18.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.637+0000 7fa3b53a97c0 4 datascan.main: opening data pool 'cephfs_data' 2023-05-20T01:22:18.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.637+0000 7fa3b3b58700 1 --2- 172.21.15.16:0/2859455639 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x555b4a930000 0x555b4a599700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.637+0000 7fa3b53a97c0 1 --2- 172.21.15.16:0/2859455639 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x555b4a5dc400 0x555b4a598100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.637+0000 7f3b5969c700 1 -- 172.21.15.16:0/3957359014 <== osd.4 v2:172.21.15.132:6817/506317223 3 ==== osd_op_reply(9 10000000000.00000000 [omap-get-header out=274b] v0'0 uv18 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x56013b6578c0 con 0x56013b665800 2023-05-20T01:22:18.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.637+0000 7fa3b53a97c0 1 -- 172.21.15.16:0/2859455639 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:1 9.14 9:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x555b4b38a700 con 0x555b4a5dc400 2023-05-20T01:22:18.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.637+0000 7f3b5beef7c0 10 datascan.find_or_create_dirfrag: read_version = 18 2023-05-20T01:22:18.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.637+0000 7f3b5beef7c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-05-20T01:22:18.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.637+0000 7f3775823700 1 -- 172.21.15.16:0/1569316897 <== osd.7 v2:172.21.15.132:6800/466001149 4 ==== osd_op_reply(14 [pgnls-filter start_epoch 71 out=49b] v69'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ce84a25b00 con 0x55ce83fbe800 2023-05-20T01:22:18.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.637+0000 7f3b5beef7c0 1 -- 172.21.15.16:0/3957359014 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:10 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x56013b5fca80 con 0x56013b665800 2023-05-20T01:22:18.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.637+0000 7f377181b700 1 -- 172.21.15.16:0/1569316897 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:15 10.13 10:c8000000::::head [pgnls-filter start_epoch 71 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x55ce84a1d500 con 0x55ce83eaec00 2023-05-20T01:22:18.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.637+0000 7f90ce7a5700 1 -- 172.21.15.16:0/3200652326 <== osd.4 v2:172.21.15.132:6817/506317223 1 ==== osd_op_reply(7 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x556b170eb680 con 0x556b170ffc00 2023-05-20T01:22:18.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.637+0000 7f90cfff67c0 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-05-20T01:22:18.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.637+0000 7f90cfff67c0 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-05-20T01:22:18.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.637+0000 7f90cfff67c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/4, will insert to root 2023-05-20T01:22:18.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.637+0000 7f90cfff67c0 1 -- 172.21.15.16:0/3200652326 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:8 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x556b17c5ca80 con 0x556b170ffc00 2023-05-20T01:22:18.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.637+0000 7fa3b3357700 1 --2- 172.21.15.16:0/2859455639 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x555b4a5dc400 0x555b4a598100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.637+0000 7f778129d700 1 -- 172.21.15.16:0/251369889 <== osd.4 v2:172.21.15.132:6817/506317223 2 ==== osd_op_reply(11 10000000000.00000000 [omap-get-header out=274b] v0'0 uv18 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x563d8a7ef680 con 0x563d8a80d400 2023-05-20T01:22:18.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.637+0000 7f7782aee7c0 10 datascan.find_or_create_dirfrag: read_version = 18 2023-05-20T01:22:18.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.637+0000 7f7782aee7c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-05-20T01:22:18.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.637+0000 7f7782aee7c0 1 -- 172.21.15.16:0/251369889 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:12 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x563d8a93a700 con 0x563d8a80d400 2023-05-20T01:22:18.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.637+0000 7f70ccb9d700 1 -- 172.21.15.16:0/3489021394 <== osd.4 v2:172.21.15.132:6817/506317223 3 ==== osd_op_reply(12 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv18 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x5636419b3680 con 0x5636419bd400 2023-05-20T01:22:18.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.637+0000 7f70ce3ee7c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=7 frag=*, r=0 2023-05-20T01:22:18.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.637+0000 7f70ce3ee7c0 20 datascan.read_dentry: 7_head not found in result 2023-05-20T01:22:18.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.637+0000 7f70ce3ee7c0 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001fc at 0x0x10000000000/7 with size=6291456 bytes 2023-05-20T01:22:18.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.637+0000 7f70ce3ee7c0 1 -- 172.21.15.16:0/3489021394 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:13 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x56364246c700 con 0x5636419bd400 2023-05-20T01:22:18.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.637+0000 7f3776024700 1 -- 172.21.15.16:0/1569316897 <== osd.1 v2:172.21.15.16:6808/3860264220 3 ==== osd_op_reply(15 [pgnls-filter start_epoch 71 out=81b] v71'19 uv19 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55ce84a25680 con 0x55ce83eaec00 2023-05-20T01:22:18.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.637+0000 7fa3b3b58700 1 --2- 172.21.15.16:0/2859455639 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x555b4a930000 0x555b4a599700 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x555b4a82a090 tx=0x555b4b2de000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.637+0000 7f37780767c0 20 datascan.forall_objects: OSD matched oid 100000001fb.00000000 2023-05-20T01:22:18.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.637+0000 7f37780767c0 10 datascan.operator(): handling object 100000001fb.0 2023-05-20T01:22:18.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.637+0000 7f37780767c0 1 -- 172.21.15.16:0/1569316897 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:16 10.13 10:ca8fdbba:::100000001fb.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x55ce84a1c700 con 0x55ce83eaec00 2023-05-20T01:22:18.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.637+0000 7f3b5969c700 1 -- 172.21.15.16:0/3957359014 <== osd.4 v2:172.21.15.132:6817/506317223 4 ==== osd_op_reply(10 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv18 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x56013b6578c0 con 0x56013b665800 2023-05-20T01:22:18.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.637+0000 7f3b5beef7c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=9 frag=*, r=0 2023-05-20T01:22:18.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.637+0000 7f3b5beef7c0 20 datascan.read_dentry: 9_head not found in result 2023-05-20T01:22:18.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.637+0000 7f3b5beef7c0 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001fe at 0x0x10000000000/9 with size=6291456 bytes 2023-05-20T01:22:18.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.637+0000 7f3b5beef7c0 1 -- 172.21.15.16:0/3957359014 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:11 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x56013b5fd180 con 0x56013b665800 2023-05-20T01:22:18.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.637+0000 7fa3b3357700 1 --2- 172.21.15.16:0/2859455639 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x555b4a5dc400 0x555b4a598100 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.637+0000 7f90ce7a5700 1 -- 172.21.15.16:0/3200652326 <== osd.4 v2:172.21.15.132:6817/506317223 2 ==== osd_op_reply(8 10000000000.00000000 [omap-get-header out=274b] v0'0 uv18 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x556b170eb680 con 0x556b170ffc00 2023-05-20T01:22:18.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.637+0000 7f90cfff67c0 10 datascan.find_or_create_dirfrag: read_version = 18 2023-05-20T01:22:18.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.637+0000 7f90cfff67c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-05-20T01:22:18.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.637+0000 7f90cfff67c0 1 -- 172.21.15.16:0/3200652326 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:9 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x556b17c5d180 con 0x556b170ffc00 2023-05-20T01:22:18.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.637+0000 7f778129d700 1 -- 172.21.15.16:0/251369889 <== osd.4 v2:172.21.15.132:6817/506317223 3 ==== osd_op_reply(12 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv18 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x563d8a7ef680 con 0x563d8a80d400 2023-05-20T01:22:18.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.637+0000 7f7782aee7c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=21 frag=*, r=0 2023-05-20T01:22:18.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.637+0000 7f7782aee7c0 20 datascan.read_dentry: 21_head not found in result 2023-05-20T01:22:18.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.637+0000 7fa3b3357700 1 -- 172.21.15.16:0/2859455639 <== osd.1 v2:172.21.15.16:6808/3860264220 1 ==== osd_op_reply(1 1.00000000.inode [stat out=16b] v0'0 uv133 ondisk = 0) v8 ==== 160+0+16 (crc 0 0 0) 0x555b4a80d440 con 0x555b4a5dc400 2023-05-20T01:22:18.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.637+0000 7f7782aee7c0 10 datascan.inject_with_backtrace: Linking inode 0x0x1000000020a at 0x0x10000000000/21 with size=6291456 bytes 2023-05-20T01:22:18.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.638+0000 7f7782aee7c0 1 -- 172.21.15.16:0/251369889 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:13 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x563d8a93aa80 con 0x563d8a80d400 2023-05-20T01:22:18.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.637+0000 7fa3b53a97c0 1 -- 172.21.15.16:0/2859455639 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:2 9.1f 9:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x555b4b38aa80 con 0x555b4a5dc400 2023-05-20T01:22:18.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.638+0000 7f3776024700 1 -- 172.21.15.16:0/1569316897 <== osd.1 v2:172.21.15.16:6808/3860264220 4 ==== osd_op_reply(16 100000001fb.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (97) out=97b,getxattr,getxattr] v0'0 uv19 ondisk = 0) v8 ==== 416+0+143 (crc 0 0 0) 0x55ce84a25680 con 0x55ce83eaec00 2023-05-20T01:22:18.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.638+0000 7f37780767c0 1 --2- 172.21.15.16:0/1569316897 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x55ce83fbf400 0x55ce83fe2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.638+0000 7f37780767c0 1 -- 172.21.15.16:0/1569316897 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:17 11.13 11:ca8fdbba:::100000001fb.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x55ce84a1ce00 con 0x55ce83fbf400 2023-05-20T01:22:18.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.638+0000 7fa3b3357700 1 -- 172.21.15.16:0/2859455639 <== osd.1 v2:172.21.15.16:6808/3860264220 2 ==== osd_op_reply(2 100.00000000.inode [stat out=16b] v0'0 uv10 ondisk = 0) v8 ==== 162+0+16 (crc 0 0 0) 0x555b4a80d440 con 0x555b4a5dc400 2023-05-20T01:22:18.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.638+0000 7f3776024700 1 --2- 172.21.15.16:0/1569316897 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x55ce83fbf400 0x55ce83fe2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.638+0000 7fa3b53a97c0 1 --2- 172.21.15.16:0/2859455639 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x555b4a4fdc00 0x555b4a597b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.638+0000 7fa3b53a97c0 1 -- 172.21.15.16:0/2859455639 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:3 10.0 10:00000000::::head [pgnls-filter start_epoch 71 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x555b4b38ae00 con 0x555b4a4fdc00 2023-05-20T01:22:18.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.638+0000 7fa3b3b58700 1 --2- 172.21.15.16:0/2859455639 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x555b4a4fdc00 0x555b4a597b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.638+0000 7f3776024700 1 --2- 172.21.15.16:0/1569316897 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x55ce83fbf400 0x55ce83fe2b00 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.639+0000 7fa3b3b58700 1 --2- 172.21.15.16:0/2859455639 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x555b4a4fdc00 0x555b4a597b80 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.639+0000 7f70ccb9d700 1 -- 172.21.15.16:0/3489021394 <== osd.4 v2:172.21.15.132:6817/506317223 4 ==== osd_op_reply(13 10000000000.00000000 [omap-set-vals] v71'19 uv19 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5636419b3680 con 0x5636419bd400 2023-05-20T01:22:18.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.639+0000 7f70ce3ee7c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/7 pointing to 0x0x100000001fc 2023-05-20T01:22:18.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.639+0000 7f70ce3ee7c0 1 --2- 172.21.15.16:0/3489021394 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x5636419bd800 0x563641aff080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.639+0000 7f70ce3ee7c0 1 -- 172.21.15.16:0/3489021394 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:14 10.2c 10:34000000::::head [pgnls-filter start_epoch 71 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x56364246d880 con 0x5636419bd800 2023-05-20T01:22:18.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.639+0000 7f3776024700 1 -- 172.21.15.16:0/1569316897 <== osd.5 v2:172.21.15.132:6816/1844689995 1 ==== osd_op_reply(17 100000001fb.00000000 [getxattr (30) out=30b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x55ce84a25680 con 0x55ce83fbf400 2023-05-20T01:22:18.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.639+0000 7f37780767c0 20 datascan.operator(): loaded layout from xattr: pi: 11 os: 4194304 sc: 1 su: 4194304 2023-05-20T01:22:18.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.639+0000 7f37780767c0 10 datascan.inject_with_backtrace: inode: 0x0x100000001fb 2023-05-20T01:22:18.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.639+0000 7f37780767c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/6 2023-05-20T01:22:18.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.639+0000 7f37780767c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=6 2023-05-20T01:22:18.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.639+0000 7f37780767c0 1 -- 172.21.15.16:0/1569316897 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:18 9.10 9:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x55ce83ffc000 con 0x55ce83fbe400 2023-05-20T01:22:18.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.639+0000 7f70cc39c700 1 --2- 172.21.15.16:0/3489021394 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x5636419bd800 0x563641aff080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.639+0000 7f90ce7a5700 1 -- 172.21.15.16:0/3200652326 <== osd.4 v2:172.21.15.132:6817/506317223 3 ==== osd_op_reply(9 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv19 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x556b170eb680 con 0x556b170ffc00 2023-05-20T01:22:18.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.639+0000 7f90cfff67c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=4 frag=*, r=0 2023-05-20T01:22:18.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.639+0000 7f90cfff67c0 20 datascan.read_dentry: 4_head not found in result 2023-05-20T01:22:18.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.639+0000 7f90cfff67c0 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001f9 at 0x0x10000000000/4 with size=6291456 bytes 2023-05-20T01:22:18.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.639+0000 7f90cfff67c0 1 -- 172.21.15.16:0/3200652326 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:10 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x556b17c5ce00 con 0x556b170ffc00 2023-05-20T01:22:18.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.639+0000 7fa3b3b58700 1 -- 172.21.15.16:0/2859455639 <== osd.6 v2:172.21.15.132:6801/727581123 1 ==== osd_op_reply(3 [pgnls-filter start_epoch 71 out=81b] v71'12 uv12 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x555b4a80d680 con 0x555b4a4fdc00 2023-05-20T01:22:18.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.639+0000 7fa3b53a97c0 1 --2- 172.21.15.16:0/2859455639 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x555b4a4fd800 0x555b4a597600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.639+0000 7fa3b53a97c0 1 -- 172.21.15.16:0/2859455639 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:4 10.9 10:92492492::::head [pgnls-filter start_epoch 71 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x555b4b38a700 con 0x555b4a4fd800 2023-05-20T01:22:18.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.639+0000 7f70cc39c700 1 --2- 172.21.15.16:0/3489021394 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x5636419bd800 0x563641aff080 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.639+0000 7fa3b3357700 1 --2- 172.21.15.16:0/2859455639 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x555b4a4fd800 0x555b4a597600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.639+0000 7f70cc39c700 1 -- 172.21.15.16:0/3489021394 <== osd.3 v2:172.21.15.16:6800/3669781890 1 ==== osd_op_reply(14 [pgnls-filter start_epoch 71 out=81b] v71'12 uv12 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5636419b38c0 con 0x5636419bd800 2023-05-20T01:22:18.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.639+0000 7f70ce3ee7c0 20 datascan.forall_objects: OSD matched oid 10000000209.00000000 2023-05-20T01:22:18.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.639+0000 7f70ce3ee7c0 10 datascan.operator(): handling object 10000000209.0 2023-05-20T01:22:18.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.640+0000 7f70ce3ee7c0 1 -- 172.21.15.16:0/3489021394 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:15 10.2c 10:37389ade:::10000000209.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x563641b04700 con 0x5636419bd800 2023-05-20T01:22:18.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.640+0000 7fa3b3357700 1 --2- 172.21.15.16:0/2859455639 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x555b4a4fd800 0x555b4a597600 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.640+0000 7f70cc39c700 1 -- 172.21.15.16:0/3489021394 <== osd.3 v2:172.21.15.16:6800/3669781890 2 ==== osd_op_reply(15 10000000209.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (98) out=98b,getxattr,getxattr] v0'0 uv12 ondisk = 0) v8 ==== 416+0+144 (crc 0 0 0) 0x5636419b38c0 con 0x5636419bd800 2023-05-20T01:22:18.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.640+0000 7f70ce3ee7c0 1 -- 172.21.15.16:0/3489021394 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:16 11.c 11:37389ade:::10000000209.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x563641b04000 con 0x5636419bc800 2023-05-20T01:22:18.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.640+0000 7f70cbb9b700 1 -- 172.21.15.16:0/3489021394 <== osd.1 v2:172.21.15.16:6808/3860264220 6 ==== osd_op_reply(16 10000000209.00000000 [getxattr (30) out=30b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x5636419b3440 con 0x5636419bc800 2023-05-20T01:22:18.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.640+0000 7f70ce3ee7c0 20 datascan.operator(): loaded layout from xattr: pi: 11 os: 4194304 sc: 1 su: 4194304 2023-05-20T01:22:18.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.640+0000 7f70ce3ee7c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000209 2023-05-20T01:22:18.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.640+0000 7f70ce3ee7c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/20 2023-05-20T01:22:18.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.640+0000 7f70ce3ee7c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=20 2023-05-20T01:22:18.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.640+0000 7fa3b3357700 1 -- 172.21.15.16:0/2859455639 <== osd.5 v2:172.21.15.132:6816/1844689995 1 ==== osd_op_reply(4 [pgnls-filter start_epoch 71 out=49b] v69'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555b4a80d440 con 0x555b4a4fd800 2023-05-20T01:22:18.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.640+0000 7f70ce3ee7c0 1 -- 172.21.15.16:0/3489021394 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:17 9.10 9:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x563641b04e00 con 0x5636419bd400 2023-05-20T01:22:18.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.640+0000 7fa3ae34d700 1 --2- 172.21.15.16:0/2859455639 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x555b4a930400 0x555b4a966000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.640+0000 7fa3ae34d700 1 -- 172.21.15.16:0/2859455639 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:5 10.29 10:94000000::::head [pgnls-filter start_epoch 71 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x555b4b38b880 con 0x555b4a930400 2023-05-20T01:22:18.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.640+0000 7fa3b2b56700 1 --2- 172.21.15.16:0/2859455639 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x555b4a930400 0x555b4a966000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.640+0000 7f3b5969c700 1 -- 172.21.15.16:0/3957359014 <== osd.4 v2:172.21.15.132:6817/506317223 5 ==== osd_op_reply(11 10000000000.00000000 [omap-set-vals] v71'20 uv20 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56013b6578c0 con 0x56013b665800 2023-05-20T01:22:18.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.640+0000 7f3b5beef7c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/9 pointing to 0x0x100000001fe 2023-05-20T01:22:18.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.640+0000 7f3b5beef7c0 1 -- 172.21.15.16:0/3957359014 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:12 10.7 10:e0000000::::head [pgnls-filter start_epoch 71 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x56013b5fc700 con 0x56013b664c00 2023-05-20T01:22:18.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.641+0000 7fa3b2b56700 1 --2- 172.21.15.16:0/2859455639 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x555b4a930400 0x555b4a966000 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.641+0000 7f778129d700 1 -- 172.21.15.16:0/251369889 <== osd.4 v2:172.21.15.132:6817/506317223 4 ==== osd_op_reply(13 10000000000.00000000 [omap-set-vals] v71'21 uv21 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563d8a7ef680 con 0x563d8a80d400 2023-05-20T01:22:18.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.641+0000 7f7782aee7c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/21 pointing to 0x0x1000000020a 2023-05-20T01:22:18.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.641+0000 7f7782aee7c0 1 -- 172.21.15.16:0/251369889 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:14 10.2a 10:57fe7535::::head [pgnls-filter start_epoch 71 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x563d8a93ae00 con 0x563d8a80f400 2023-05-20T01:22:18.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.641+0000 7fabbbd98700 1 -- 172.21.15.16:0/2723465486 <== osd.4 v2:172.21.15.132:6817/506317223 9 ==== osd_op_reply(23 10000000000.00000000 [omap-set-vals] v71'22 uv22 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5614bdd378c0 con 0x5614bde80400 2023-05-20T01:22:18.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.641+0000 7fabbddea7c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/22 pointing to 0x0x1000000020b 2023-05-20T01:22:18.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.641+0000 7fa3b2b56700 1 -- 172.21.15.16:0/2859455639 <== osd.3 v2:172.21.15.16:6800/3669781890 1 ==== osd_op_reply(5 [pgnls-filter start_epoch 71 out=49b] v69'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555b4a80d8c0 con 0x555b4a930400 2023-05-20T01:22:18.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.641+0000 7fabbddea7c0 1 --2- 172.21.15.16:0/2723465486 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x5614bde81000 0x5614bde90580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.641+0000 7fabbddea7c0 1 -- 172.21.15.16:0/2723465486 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:24 10.10 10:08000000::::head [pgnls-filter start_epoch 71 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x5614be7fa700 con 0x5614bde81000 2023-05-20T01:22:18.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.641+0000 7f3b5a69e700 1 -- 172.21.15.16:0/3957359014 <== osd.6 v2:172.21.15.132:6801/727581123 4 ==== osd_op_reply(12 [pgnls-filter start_epoch 71 out=101b] v71'13 uv13 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x56013b657680 con 0x56013b664c00 2023-05-20T01:22:18.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.641+0000 7fa3aeb4e700 1 -- 172.21.15.16:0/2859455639 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:6 10.19 10:98000000::::head [pgnls-filter start_epoch 71 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x555b4b38bc00 con 0x555b4a5dc400 2023-05-20T01:22:18.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.641+0000 7f778029b700 1 -- 172.21.15.16:0/251369889 <== osd.3 v2:172.21.15.16:6800/3669781890 4 ==== osd_op_reply(14 [pgnls-filter start_epoch 71 out=81b] v71'17 uv17 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x563d8a7ef440 con 0x563d8a80f400 2023-05-20T01:22:18.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.641+0000 7f3b5beef7c0 20 datascan.forall_objects: OSD matched oid 10000000207.00000000 2023-05-20T01:22:18.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.641+0000 7f3b5beef7c0 10 datascan.operator(): handling object 10000000207.0 2023-05-20T01:22:18.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.641+0000 7fabbc599700 1 --2- 172.21.15.16:0/2723465486 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x5614bde81000 0x5614bde90580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.641+0000 7f3776024700 1 -- 172.21.15.16:0/1569316897 <== osd.4 v2:172.21.15.132:6817/506317223 6 ==== osd_op_reply(18 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x55ce84a25680 con 0x55ce83fbe400 2023-05-20T01:22:18.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.641+0000 7f37780767c0 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-05-20T01:22:18.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.641+0000 7f37780767c0 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-05-20T01:22:18.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.641+0000 7f37780767c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/6, will insert to root 2023-05-20T01:22:18.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.641+0000 7f37780767c0 1 -- 172.21.15.16:0/1569316897 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:19 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x55ce83ffc380 con 0x55ce83fbe400 2023-05-20T01:22:18.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.641+0000 7f3b5beef7c0 1 -- 172.21.15.16:0/3957359014 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:13 10.7 10:e15a914f:::10000000207.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x56013b5fca80 con 0x56013b664c00 2023-05-20T01:22:18.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.641+0000 7f7782aee7c0 20 datascan.forall_objects: OSD matched oid 100000001f6.00000000 2023-05-20T01:22:18.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.641+0000 7f7782aee7c0 10 datascan.operator(): handling object 100000001f6.0 2023-05-20T01:22:18.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.641+0000 7f7782aee7c0 1 -- 172.21.15.16:0/251369889 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:15 10.2a 10:57fe7535:::100000001f6.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x563d8a93ae00 con 0x563d8a80f400 2023-05-20T01:22:18.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.641+0000 7fa3b3357700 1 -- 172.21.15.16:0/2859455639 <== osd.1 v2:172.21.15.16:6808/3860264220 3 ==== osd_op_reply(6 [pgnls-filter start_epoch 71 out=101b] v71'18 uv18 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x555b4a80d440 con 0x555b4a5dc400 2023-05-20T01:22:18.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.641+0000 7fa3b53a97c0 20 datascan.forall_objects: OSD matched oid 10000000200.00000000 2023-05-20T01:22:18.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.641+0000 7fa3b53a97c0 10 datascan.operator(): handling object 10000000200.0 2023-05-20T01:22:18.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.641+0000 7fa3b53a97c0 1 -- 172.21.15.16:0/2859455639 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:7 10.19 10:992a6246:::10000000200.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x555b4b38aa80 con 0x555b4a5dc400 2023-05-20T01:22:18.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.641+0000 7fabbc599700 1 --2- 172.21.15.16:0/2723465486 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x5614bde81000 0x5614bde90580 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.642+0000 7f778029b700 1 -- 172.21.15.16:0/251369889 <== osd.3 v2:172.21.15.16:6800/3669781890 5 ==== osd_op_reply(15 100000001f6.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (97) out=97b,getxattr,getxattr] v0'0 uv17 ondisk = 0) v8 ==== 416+0+143 (crc 0 0 0) 0x563d8a7ef440 con 0x563d8a80f400 2023-05-20T01:22:18.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.642+0000 7f3b5a69e700 1 -- 172.21.15.16:0/3957359014 <== osd.6 v2:172.21.15.132:6801/727581123 5 ==== osd_op_reply(13 10000000207.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (98) out=98b,getxattr,getxattr] v0'0 uv11 ondisk = 0) v8 ==== 416+0+144 (crc 0 0 0) 0x56013b657680 con 0x56013b664c00 2023-05-20T01:22:18.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.642+0000 7f7782aee7c0 1 -- 172.21.15.16:0/251369889 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:16 11.a 11:57fe7535:::100000001f6.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x563d8a795180 con 0x563d8a80f400 2023-05-20T01:22:18.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.642+0000 7fa3b3357700 1 -- 172.21.15.16:0/2859455639 <== osd.1 v2:172.21.15.16:6808/3860264220 4 ==== osd_op_reply(7 10000000200.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (98) out=98b,getxattr,getxattr] v0'0 uv18 ondisk = 0) v8 ==== 416+0+144 (crc 0 0 0) 0x555b4a80d440 con 0x555b4a5dc400 2023-05-20T01:22:18.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.642+0000 7fa3b53a97c0 1 --2- 172.21.15.16:0/2859455639 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x555b4a930c00 0x555b4a966580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.642+0000 7f3b5beef7c0 1 --2- 172.21.15.16:0/3957359014 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x56013b66c800 0x56013b78c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.642+0000 7f3b5beef7c0 1 -- 172.21.15.16:0/3957359014 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:14 11.7 11:e15a914f:::10000000207.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x56013b5fdc00 con 0x56013b66c800 2023-05-20T01:22:18.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.642+0000 7fa3b53a97c0 1 -- 172.21.15.16:0/2859455639 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:8 11.19 11:992a6246:::10000000200.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x555b4b38b500 con 0x555b4a930c00 2023-05-20T01:22:18.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.642+0000 7fabbc599700 1 -- 172.21.15.16:0/2723465486 <== osd.3 v2:172.21.15.16:6800/3669781890 1 ==== osd_op_reply(24 [pgnls-filter start_epoch 71 out=81b] v71'10 uv10 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5614bdd37680 con 0x5614bde81000 2023-05-20T01:22:18.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.642+0000 7fa3b3b58700 1 --2- 172.21.15.16:0/2859455639 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x555b4a930c00 0x555b4a966580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.642+0000 7f3b5a69e700 1 --2- 172.21.15.16:0/3957359014 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x56013b66c800 0x56013b78c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.642+0000 7f778029b700 1 -- 172.21.15.16:0/251369889 <== osd.3 v2:172.21.15.16:6800/3669781890 6 ==== osd_op_reply(16 100000001f6.00000000 [getxattr (30) out=30b] v0'0 uv5 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x563d8a7ef440 con 0x563d8a80f400 2023-05-20T01:22:18.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.642+0000 7f7782aee7c0 20 datascan.operator(): loaded layout from xattr: pi: 11 os: 4194304 sc: 1 su: 4194304 2023-05-20T01:22:18.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.642+0000 7f7782aee7c0 10 datascan.inject_with_backtrace: inode: 0x0x100000001f6 2023-05-20T01:22:18.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.642+0000 7f7782aee7c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/1 2023-05-20T01:22:18.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.642+0000 7f7782aee7c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=1 2023-05-20T01:22:18.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.642+0000 7f7782aee7c0 1 -- 172.21.15.16:0/251369889 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:17 9.10 9:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x563d8a795880 con 0x563d8a80d400 2023-05-20T01:22:18.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.642+0000 7fabbddea7c0 20 datascan.forall_objects: OSD matched oid 100000001fd.00000000 2023-05-20T01:22:18.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.642+0000 7fabbddea7c0 10 datascan.operator(): handling object 100000001fd.0 2023-05-20T01:22:18.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.642+0000 7fabbddea7c0 1 -- 172.21.15.16:0/2723465486 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:25 10.10 10:08b99fea:::100000001fd.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x5614be7fb880 con 0x5614bde81000 2023-05-20T01:22:18.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.642+0000 7f90ce7a5700 1 -- 172.21.15.16:0/3200652326 <== osd.4 v2:172.21.15.132:6817/506317223 4 ==== osd_op_reply(10 10000000000.00000000 [omap-set-vals] v71'23 uv23 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x556b170eb680 con 0x556b170ffc00 2023-05-20T01:22:18.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.642+0000 7f90cfff67c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/4 pointing to 0x0x100000001f9 2023-05-20T01:22:18.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.642+0000 7f90cfff67c0 1 -- 172.21.15.16:0/3200652326 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:11 10.36 10:6fe38dc2::::head [pgnls-filter start_epoch 71 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x556b17c5c700 con 0x556b170ff000 2023-05-20T01:22:18.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.642+0000 7fa3b3b58700 1 --2- 172.21.15.16:0/2859455639 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x555b4a930c00 0x555b4a966580 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.642+0000 7f3b5a69e700 1 --2- 172.21.15.16:0/3957359014 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x56013b66c800 0x56013b78c580 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.642+0000 7f70ccb9d700 1 -- 172.21.15.16:0/3489021394 <== osd.4 v2:172.21.15.132:6817/506317223 5 ==== osd_op_reply(17 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x5636419b3680 con 0x5636419bd400 2023-05-20T01:22:18.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.642+0000 7f70ce3ee7c0 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-05-20T01:22:18.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.642+0000 7f70ce3ee7c0 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-05-20T01:22:18.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.642+0000 7f70ce3ee7c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/20, will insert to root 2023-05-20T01:22:18.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.642+0000 7fabbc599700 1 -- 172.21.15.16:0/2723465486 <== osd.3 v2:172.21.15.16:6800/3669781890 2 ==== osd_op_reply(25 100000001fd.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (97) out=97b,getxattr,getxattr] v0'0 uv10 ondisk = 0) v8 ==== 416+0+143 (crc 0 0 0) 0x5614bdd37680 con 0x5614bde81000 2023-05-20T01:22:18.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.642+0000 7fabbddea7c0 1 -- 172.21.15.16:0/2723465486 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:26 11.10 11:08b99fea:::100000001fd.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x5614be7fa700 con 0x5614bde80800 2023-05-20T01:22:18.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.642+0000 7f70ce3ee7c0 1 -- 172.21.15.16:0/3489021394 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:18 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x56364246c700 con 0x5636419bd400 2023-05-20T01:22:18.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.643+0000 7f90cdfa4700 1 -- 172.21.15.16:0/3200652326 <== osd.7 v2:172.21.15.132:6800/466001149 3 ==== osd_op_reply(11 [pgnls-filter start_epoch 71 out=81b] v71'23 uv23 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x556b170eb440 con 0x556b170ff000 2023-05-20T01:22:18.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.643+0000 7f3776024700 1 -- 172.21.15.16:0/1569316897 <== osd.4 v2:172.21.15.132:6817/506317223 7 ==== osd_op_reply(19 10000000000.00000000 [omap-get-header out=274b] v0'0 uv23 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x55ce84a25680 con 0x55ce83fbe400 2023-05-20T01:22:18.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.643+0000 7f37780767c0 10 datascan.find_or_create_dirfrag: read_version = 23 2023-05-20T01:22:18.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.643+0000 7f37780767c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-05-20T01:22:18.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.643+0000 7f37780767c0 1 -- 172.21.15.16:0/1569316897 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:20 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x55ce83ffc700 con 0x55ce83fbe400 2023-05-20T01:22:18.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.643+0000 7f778129d700 1 -- 172.21.15.16:0/251369889 <== osd.4 v2:172.21.15.132:6817/506317223 5 ==== osd_op_reply(17 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x563d8a7ef680 con 0x563d8a80d400 2023-05-20T01:22:18.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.643+0000 7fa3b3b58700 1 -- 172.21.15.16:0/2859455639 <== osd.0 v2:172.21.15.16:6824/3923634179 1 ==== osd_op_reply(8 10000000200.00000000 [getxattr (30) out=30b] v0'0 uv5 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x555b4a80d680 con 0x555b4a930c00 2023-05-20T01:22:18.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.643+0000 7fa3b53a97c0 20 datascan.operator(): loaded layout from xattr: pi: 11 os: 4194304 sc: 1 su: 4194304 2023-05-20T01:22:18.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.643+0000 7fa3b53a97c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000200 2023-05-20T01:22:18.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.643+0000 7fa3b53a97c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/11 2023-05-20T01:22:18.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.643+0000 7fa3b53a97c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=11 2023-05-20T01:22:18.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.643+0000 7fa3b53a97c0 1 --2- 172.21.15.16:0/2859455639 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x555b4a931000 0x555b4a966b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.643+0000 7f90cfff67c0 20 datascan.forall_objects: OSD matched oid 100000001f7.00000000 2023-05-20T01:22:18.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.643+0000 7f90cfff67c0 10 datascan.operator(): handling object 100000001f7.0 2023-05-20T01:22:18.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.643+0000 7f90cfff67c0 1 -- 172.21.15.16:0/3200652326 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:12 10.36 10:6fe38dc2:::100000001f7.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x556b17c5ca80 con 0x556b170ff000 2023-05-20T01:22:18.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.643+0000 7f3b5a69e700 1 -- 172.21.15.16:0/3957359014 <== osd.5 v2:172.21.15.132:6816/1844689995 1 ==== osd_op_reply(14 10000000207.00000000 [getxattr (30) out=30b] v0'0 uv6 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x56013b657680 con 0x56013b66c800 2023-05-20T01:22:18.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.643+0000 7f3b5beef7c0 20 datascan.operator(): loaded layout from xattr: pi: 11 os: 4194304 sc: 1 su: 4194304 2023-05-20T01:22:18.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.643+0000 7f3b5beef7c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000207 2023-05-20T01:22:18.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.643+0000 7f3b5beef7c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/18 2023-05-20T01:22:18.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.643+0000 7f3b5beef7c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=18 2023-05-20T01:22:18.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.643+0000 7fabbb597700 1 -- 172.21.15.16:0/2723465486 <== osd.0 v2:172.21.15.16:6824/3923634179 3 ==== osd_op_reply(26 100000001fd.00000000 [getxattr (30) out=30b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x5614bdd37440 con 0x5614bde80800 2023-05-20T01:22:18.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.643+0000 7fabbddea7c0 20 datascan.operator(): loaded layout from xattr: pi: 11 os: 4194304 sc: 1 su: 4194304 2023-05-20T01:22:18.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.643+0000 7fabbddea7c0 10 datascan.inject_with_backtrace: inode: 0x0x100000001fd 2023-05-20T01:22:18.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.643+0000 7fabbddea7c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/8 2023-05-20T01:22:18.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.643+0000 7fabbddea7c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=8 2023-05-20T01:22:18.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.643+0000 7f7782aee7c0 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-05-20T01:22:18.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.643+0000 7f7782aee7c0 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-05-20T01:22:18.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.643+0000 7f7782aee7c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/1, will insert to root 2023-05-20T01:22:18.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.643+0000 7f7782aee7c0 1 -- 172.21.15.16:0/251369889 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:18 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x563d8a795500 con 0x563d8a80d400 2023-05-20T01:22:18.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.643+0000 7fa3b53a97c0 1 -- 172.21.15.16:0/2859455639 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:9 9.10 9:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x555b4b38ae00 con 0x555b4a931000 2023-05-20T01:22:18.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.643+0000 7f3b5beef7c0 1 -- 172.21.15.16:0/3957359014 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:15 9.10 9:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x56013b5fce00 con 0x56013b665800 2023-05-20T01:22:18.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.643+0000 7fabbddea7c0 1 -- 172.21.15.16:0/2723465486 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:27 9.10 9:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x5614be7faa80 con 0x5614bde80400 2023-05-20T01:22:18.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.643+0000 7f70ccb9d700 1 -- 172.21.15.16:0/3489021394 <== osd.4 v2:172.21.15.132:6817/506317223 6 ==== osd_op_reply(18 10000000000.00000000 [omap-get-header out=274b] v0'0 uv23 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5636419b3680 con 0x5636419bd400 2023-05-20T01:22:18.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.643+0000 7f70ce3ee7c0 10 datascan.find_or_create_dirfrag: read_version = 23 2023-05-20T01:22:18.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.643+0000 7f70ce3ee7c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-05-20T01:22:18.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.643+0000 7f70ce3ee7c0 1 -- 172.21.15.16:0/3489021394 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:19 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x56364246ce00 con 0x5636419bd400 2023-05-20T01:22:18.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.643+0000 7fa3b3357700 1 --2- 172.21.15.16:0/2859455639 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x555b4a931000 0x555b4a966b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.643+0000 7f3776024700 1 -- 172.21.15.16:0/1569316897 <== osd.4 v2:172.21.15.132:6817/506317223 8 ==== osd_op_reply(20 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv23 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x55ce84a25680 con 0x55ce83fbe400 2023-05-20T01:22:18.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.643+0000 7f37780767c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=6 frag=*, r=0 2023-05-20T01:22:18.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.643+0000 7f37780767c0 20 datascan.read_dentry: 6_head not found in result 2023-05-20T01:22:18.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.643+0000 7f37780767c0 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001fb at 0x0x10000000000/6 with size=6291456 bytes 2023-05-20T01:22:18.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.643+0000 7f37780767c0 1 -- 172.21.15.16:0/1569316897 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:21 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x55ce83ffca80 con 0x55ce83fbe400 2023-05-20T01:22:18.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.643+0000 7f90cdfa4700 1 -- 172.21.15.16:0/3200652326 <== osd.7 v2:172.21.15.132:6800/466001149 4 ==== osd_op_reply(12 100000001f7.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (97) out=97b,getxattr,getxattr] v0'0 uv23 ondisk = 0) v8 ==== 416+0+143 (crc 0 0 0) 0x556b170eb440 con 0x556b170ff000 2023-05-20T01:22:18.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.643+0000 7f90cfff67c0 1 -- 172.21.15.16:0/3200652326 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:13 11.16 11:6fe38dc2:::100000001f7.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x556b17c5ce00 con 0x556b170ff800 2023-05-20T01:22:18.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.643+0000 7f3b5969c700 1 -- 172.21.15.16:0/3957359014 <== osd.4 v2:172.21.15.132:6817/506317223 6 ==== osd_op_reply(15 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x56013b6578c0 con 0x56013b665800 2023-05-20T01:22:18.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.643+0000 7f3b5beef7c0 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-05-20T01:22:18.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.643+0000 7f3b5beef7c0 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-05-20T01:22:18.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.643+0000 7f3b5beef7c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/18, will insert to root 2023-05-20T01:22:18.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.643+0000 7f3b5beef7c0 1 -- 172.21.15.16:0/3957359014 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:16 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x56013b5fd880 con 0x56013b665800 2023-05-20T01:22:18.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.643+0000 7f778129d700 1 -- 172.21.15.16:0/251369889 <== osd.4 v2:172.21.15.132:6817/506317223 6 ==== osd_op_reply(18 10000000000.00000000 [omap-get-header out=274b] v0'0 uv23 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x563d8a7ef680 con 0x563d8a80d400 2023-05-20T01:22:18.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.643+0000 7f7782aee7c0 10 datascan.find_or_create_dirfrag: read_version = 23 2023-05-20T01:22:18.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.643+0000 7f7782aee7c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-05-20T01:22:18.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.643+0000 7fa3b3357700 1 --2- 172.21.15.16:0/2859455639 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x555b4a931000 0x555b4a966b00 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.643+0000 7f7782aee7c0 1 -- 172.21.15.16:0/251369889 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:19 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x563d8a794700 con 0x563d8a80d400 2023-05-20T01:22:18.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.644+0000 7f90cd7a3700 1 -- 172.21.15.16:0/3200652326 <== osd.5 v2:172.21.15.132:6816/1844689995 2 ==== osd_op_reply(13 100000001f7.00000000 [getxattr (30) out=30b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x556b170eb8c0 con 0x556b170ff800 2023-05-20T01:22:18.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.644+0000 7f90cfff67c0 20 datascan.operator(): loaded layout from xattr: pi: 11 os: 4194304 sc: 1 su: 4194304 2023-05-20T01:22:18.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.644+0000 7fabbbd98700 1 -- 172.21.15.16:0/2723465486 <== osd.4 v2:172.21.15.132:6817/506317223 10 ==== osd_op_reply(27 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x5614bdd378c0 con 0x5614bde80400 2023-05-20T01:22:18.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.644+0000 7fabbddea7c0 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-05-20T01:22:18.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.644+0000 7fabbddea7c0 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-05-20T01:22:18.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.644+0000 7fabbddea7c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/8, will insert to root 2023-05-20T01:22:18.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.644+0000 7f90cfff67c0 10 datascan.inject_with_backtrace: inode: 0x0x100000001f7 2023-05-20T01:22:18.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.644+0000 7f90cfff67c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/2 2023-05-20T01:22:18.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.644+0000 7f90cfff67c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=2 2023-05-20T01:22:18.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.644+0000 7f90cfff67c0 1 -- 172.21.15.16:0/3200652326 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:14 9.10 9:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x556b17c5dc00 con 0x556b170ffc00 2023-05-20T01:22:18.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.644+0000 7fabbddea7c0 1 -- 172.21.15.16:0/2723465486 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:28 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x5614be7fbc00 con 0x5614bde80400 2023-05-20T01:22:18.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.644+0000 7f3b5969c700 1 -- 172.21.15.16:0/3957359014 <== osd.4 v2:172.21.15.132:6817/506317223 7 ==== osd_op_reply(16 10000000000.00000000 [omap-get-header out=274b] v0'0 uv23 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x56013b6578c0 con 0x56013b665800 2023-05-20T01:22:18.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.644+0000 7f3b5beef7c0 10 datascan.find_or_create_dirfrag: read_version = 23 2023-05-20T01:22:18.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.644+0000 7f3b5beef7c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-05-20T01:22:18.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.644+0000 7f3b5beef7c0 1 -- 172.21.15.16:0/3957359014 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:17 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x56013b5fd500 con 0x56013b665800 2023-05-20T01:22:18.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.644+0000 7fa3b3357700 1 -- 172.21.15.16:0/2859455639 <== osd.4 v2:172.21.15.132:6817/506317223 1 ==== osd_op_reply(9 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x555b4a80d440 con 0x555b4a931000 2023-05-20T01:22:18.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.644+0000 7fa3b53a97c0 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-05-20T01:22:18.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.644+0000 7fa3b53a97c0 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-05-20T01:22:18.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.644+0000 7fa3b53a97c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/11, will insert to root 2023-05-20T01:22:18.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.644+0000 7fa3b53a97c0 1 -- 172.21.15.16:0/2859455639 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:10 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x555b4b38b500 con 0x555b4a931000 2023-05-20T01:22:18.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.644+0000 7f70ccb9d700 1 -- 172.21.15.16:0/3489021394 <== osd.4 v2:172.21.15.132:6817/506317223 7 ==== osd_op_reply(19 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv23 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x5636419b3680 con 0x5636419bd400 2023-05-20T01:22:18.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.644+0000 7f70ce3ee7c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=20 frag=*, r=0 2023-05-20T01:22:18.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.644+0000 7f70ce3ee7c0 20 datascan.read_dentry: 20_head not found in result 2023-05-20T01:22:18.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.644+0000 7f70ce3ee7c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000209 at 0x0x10000000000/20 with size=6291456 bytes 2023-05-20T01:22:18.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.644+0000 7f70ce3ee7c0 1 -- 172.21.15.16:0/3489021394 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:20 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x563641b04000 con 0x5636419bd400 2023-05-20T01:22:18.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.645+0000 7f3776024700 1 -- 172.21.15.16:0/1569316897 <== osd.4 v2:172.21.15.132:6817/506317223 9 ==== osd_op_reply(21 10000000000.00000000 [omap-set-vals] v71'24 uv24 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55ce84a25680 con 0x55ce83fbe400 2023-05-20T01:22:18.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.645+0000 7f37780767c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/6 pointing to 0x0x100000001fb 2023-05-20T01:22:18.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.645+0000 7f37780767c0 1 -- 172.21.15.16:0/1569316897 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:22 10.33 10:cc000000::::head [pgnls-filter start_epoch 71 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x55ce83ffce00 con 0x55ce83fbf400 2023-05-20T01:22:18.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.645+0000 7f90ce7a5700 1 -- 172.21.15.16:0/3200652326 <== osd.4 v2:172.21.15.132:6817/506317223 5 ==== osd_op_reply(14 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x556b170eb680 con 0x556b170ffc00 2023-05-20T01:22:18.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.645+0000 7f90cfff67c0 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-05-20T01:22:18.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.645+0000 7f90cfff67c0 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-05-20T01:22:18.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.645+0000 7f90cfff67c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/2, will insert to root 2023-05-20T01:22:18.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.645+0000 7f90cfff67c0 1 -- 172.21.15.16:0/3200652326 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:15 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x556b17c5ce00 con 0x556b170ffc00 2023-05-20T01:22:18.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.645+0000 7f3776024700 1 -- 172.21.15.16:0/1569316897 <== osd.5 v2:172.21.15.132:6816/1844689995 2 ==== osd_op_reply(22 [pgnls-filter start_epoch 71 out=49b] v69'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ce84a25680 con 0x55ce83fbf400 2023-05-20T01:22:18.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.645+0000 7f377181b700 1 -- 172.21.15.16:0/1569316897 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:23 10.b 10:d0000000::::head [pgnls-filter start_epoch 71 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x55ce84a1dc00 con 0x55ce83eaf000 2023-05-20T01:22:18.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.646+0000 7fa3b3357700 1 -- 172.21.15.16:0/2859455639 <== osd.4 v2:172.21.15.132:6817/506317223 2 ==== osd_op_reply(10 10000000000.00000000 [omap-get-header out=274b] v0'0 uv24 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x555b4a80d440 con 0x555b4a931000 2023-05-20T01:22:18.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.646+0000 7fa3b53a97c0 10 datascan.find_or_create_dirfrag: read_version = 24 2023-05-20T01:22:18.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.646+0000 7fa3b53a97c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-05-20T01:22:18.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.646+0000 7fa3b53a97c0 1 -- 172.21.15.16:0/2859455639 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:11 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x555b4b38a700 con 0x555b4a931000 2023-05-20T01:22:18.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.646+0000 7f3b5969c700 1 -- 172.21.15.16:0/3957359014 <== osd.4 v2:172.21.15.132:6817/506317223 8 ==== osd_op_reply(17 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv24 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x56013b6578c0 con 0x56013b665800 2023-05-20T01:22:18.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.646+0000 7f3b5beef7c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=18 frag=*, r=0 2023-05-20T01:22:18.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.646+0000 7f3b5beef7c0 20 datascan.read_dentry: 18_head not found in result 2023-05-20T01:22:18.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.646+0000 7f3b5beef7c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000207 at 0x0x10000000000/18 with size=6291456 bytes 2023-05-20T01:22:18.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.646+0000 7f3b5beef7c0 1 -- 172.21.15.16:0/3957359014 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:18 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x56013b5fc700 con 0x56013b665800 2023-05-20T01:22:18.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.646+0000 7f3776825700 1 -- 172.21.15.16:0/1569316897 <== osd.6 v2:172.21.15.132:6801/727581123 3 ==== osd_op_reply(23 [pgnls-filter start_epoch 71 out=49b] v69'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ce84a258c0 con 0x55ce83eaf000 2023-05-20T01:22:18.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.646+0000 7f377101a700 1 -- 172.21.15.16:0/1569316897 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:24 10.2b 10:d4000000::::head [pgnls-filter start_epoch 71 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x55ce84a1d880 con 0x55ce83fbf400 2023-05-20T01:22:18.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.646+0000 7f778129d700 1 -- 172.21.15.16:0/251369889 <== osd.4 v2:172.21.15.132:6817/506317223 7 ==== osd_op_reply(19 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv24 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x563d8a7ef680 con 0x563d8a80d400 2023-05-20T01:22:18.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.646+0000 7f7782aee7c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=1 frag=*, r=0 2023-05-20T01:22:18.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.646+0000 7f7782aee7c0 20 datascan.read_dentry: 1_head not found in result 2023-05-20T01:22:18.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.646+0000 7f7782aee7c0 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001f6 at 0x0x10000000000/1 with size=6291456 bytes 2023-05-20T01:22:18.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.646+0000 7f7782aee7c0 1 -- 172.21.15.16:0/251369889 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:20 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x563d8a93a700 con 0x563d8a80d400 2023-05-20T01:22:18.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.646+0000 7f3776024700 1 -- 172.21.15.16:0/1569316897 <== osd.5 v2:172.21.15.132:6816/1844689995 3 ==== osd_op_reply(24 [pgnls-filter start_epoch 71 out=81b] v71'15 uv15 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55ce84a25680 con 0x55ce83fbf400 2023-05-20T01:22:18.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.646+0000 7f37780767c0 20 datascan.forall_objects: OSD matched oid 10000000203.00000000 2023-05-20T01:22:18.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.646+0000 7f37780767c0 10 datascan.operator(): handling object 10000000203.0 2023-05-20T01:22:18.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.646+0000 7f37780767c0 1 -- 172.21.15.16:0/1569316897 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:25 10.2b 10:d4a58307:::10000000203.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x55ce83ffd180 con 0x55ce83fbf400 2023-05-20T01:22:18.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.647+0000 7f3776024700 1 -- 172.21.15.16:0/1569316897 <== osd.5 v2:172.21.15.132:6816/1844689995 4 ==== osd_op_reply(25 10000000203.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (98) out=98b,getxattr,getxattr] v0'0 uv15 ondisk = 0) v8 ==== 416+0+144 (crc 0 0 0) 0x55ce84a25680 con 0x55ce83fbf400 2023-05-20T01:22:18.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.647+0000 7f37780767c0 1 --2- 172.21.15.16:0/1569316897 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x55ce83fbfc00 0x55ce83fe3080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.647+0000 7f37780767c0 1 -- 172.21.15.16:0/1569316897 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:26 11.b 11:d4a58307:::10000000203.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x55ce83ffc380 con 0x55ce83fbfc00 2023-05-20T01:22:18.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.647+0000 7f3775823700 1 --2- 172.21.15.16:0/1569316897 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x55ce83fbfc00 0x55ce83fe3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.647+0000 7f70ccb9d700 1 -- 172.21.15.16:0/3489021394 <== osd.4 v2:172.21.15.132:6817/506317223 8 ==== osd_op_reply(20 10000000000.00000000 [omap-set-vals] v71'25 uv25 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5636419b3680 con 0x5636419bd400 2023-05-20T01:22:18.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.647+0000 7f70ce3ee7c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/20 pointing to 0x0x10000000209 2023-05-20T01:22:18.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.647+0000 7f70ce3ee7c0 1 -- 172.21.15.16:0/3489021394 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:21 10.1c 10:38000000::::head [pgnls-filter start_epoch 71 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x563641b04a80 con 0x5636419bcc00 2023-05-20T01:22:18.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.647+0000 7f3775823700 1 --2- 172.21.15.16:0/1569316897 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x55ce83fbfc00 0x55ce83fe3080 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.647+0000 7fabbbd98700 1 -- 172.21.15.16:0/2723465486 <== osd.4 v2:172.21.15.132:6817/506317223 11 ==== osd_op_reply(28 10000000000.00000000 [omap-get-header out=274b] v0'0 uv25 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5614bdd378c0 con 0x5614bde80400 2023-05-20T01:22:18.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.647+0000 7fabbddea7c0 10 datascan.find_or_create_dirfrag: read_version = 25 2023-05-20T01:22:18.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.647+0000 7fabbddea7c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-05-20T01:22:18.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.647+0000 7fabbddea7c0 1 -- 172.21.15.16:0/2723465486 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:29 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x5614be7fb180 con 0x5614bde80400 2023-05-20T01:22:18.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.647+0000 7f70ccb9d700 1 -- 172.21.15.16:0/3489021394 <== osd.6 v2:172.21.15.132:6801/727581123 2 ==== osd_op_reply(21 [pgnls-filter start_epoch 71 out=49b] v69'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636419b3680 con 0x5636419bcc00 2023-05-20T01:22:18.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.647+0000 7f3775823700 1 -- 172.21.15.16:0/1569316897 <== osd.3 v2:172.21.15.16:6800/3669781890 1 ==== osd_op_reply(26 10000000203.00000000 [getxattr (30) out=30b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x55ce84a25b00 con 0x55ce83fbfc00 2023-05-20T01:22:18.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.647+0000 7f70c7392700 1 -- 172.21.15.16:0/3489021394 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:22 10.3c 10:3c000000::::head [pgnls-filter start_epoch 71 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x56364246c700 con 0x5636419bd400 2023-05-20T01:22:18.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.647+0000 7f37780767c0 20 datascan.operator(): loaded layout from xattr: pi: 11 os: 4194304 sc: 1 su: 4194304 2023-05-20T01:22:18.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.647+0000 7f37780767c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000203 2023-05-20T01:22:18.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.647+0000 7f37780767c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/14 2023-05-20T01:22:18.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.647+0000 7f37780767c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=14 2023-05-20T01:22:18.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.647+0000 7f37780767c0 1 -- 172.21.15.16:0/1569316897 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:27 9.10 9:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x55ce83ffc700 con 0x55ce83fbe400 2023-05-20T01:22:18.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.647+0000 7fa3b3357700 1 -- 172.21.15.16:0/2859455639 <== osd.4 v2:172.21.15.132:6817/506317223 3 ==== osd_op_reply(11 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv25 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x555b4a80d440 con 0x555b4a931000 2023-05-20T01:22:18.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.648+0000 7fa3b53a97c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=11 frag=*, r=0 2023-05-20T01:22:18.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.648+0000 7fa3b53a97c0 20 datascan.read_dentry: 11_head not found in result 2023-05-20T01:22:18.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.648+0000 7fa3b53a97c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000200 at 0x0x10000000000/11 with size=6291456 bytes 2023-05-20T01:22:18.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.648+0000 7fa3b53a97c0 1 -- 172.21.15.16:0/2859455639 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:12 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x555b4b38bc00 con 0x555b4a931000 2023-05-20T01:22:18.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.648+0000 7f90ce7a5700 1 -- 172.21.15.16:0/3200652326 <== osd.4 v2:172.21.15.132:6817/506317223 6 ==== osd_op_reply(15 10000000000.00000000 [omap-get-header out=274b] v0'0 uv25 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x556b170eb680 con 0x556b170ffc00 2023-05-20T01:22:18.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.648+0000 7f90cfff67c0 10 datascan.find_or_create_dirfrag: read_version = 25 2023-05-20T01:22:18.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.648+0000 7f90cfff67c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-05-20T01:22:18.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.648+0000 7f90cfff67c0 1 -- 172.21.15.16:0/3200652326 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:16 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x556b17c5d880 con 0x556b170ffc00 2023-05-20T01:22:18.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.648+0000 7f70ccb9d700 1 -- 172.21.15.16:0/3489021394 <== osd.4 v2:172.21.15.132:6817/506317223 9 ==== osd_op_reply(22 [pgnls-filter start_epoch 71 out=49b] v69'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636419b3680 con 0x5636419bd400 2023-05-20T01:22:18.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.648+0000 7f70c7b93700 1 -- 172.21.15.16:0/3489021394 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:23 10.2 10:40000000::::head [pgnls-filter start_epoch 71 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x563641b04380 con 0x5636419bc800 2023-05-20T01:22:18.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.648+0000 7f70cbb9b700 1 -- 172.21.15.16:0/3489021394 <== osd.1 v2:172.21.15.16:6808/3860264220 7 ==== osd_op_reply(23 [pgnls-filter start_epoch 71 out=49b] v69'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636419b3440 con 0x5636419bc800 2023-05-20T01:22:18.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.648+0000 7f70c7392700 1 -- 172.21.15.16:0/3489021394 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:24 10.22 10:44000000::::head [pgnls-filter start_epoch 71 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x56364246ce00 con 0x5636419f8800 2023-05-20T01:22:18.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.648+0000 7f70cc39c700 1 -- 172.21.15.16:0/3489021394 <== osd.5 v2:172.21.15.132:6816/1844689995 2 ==== osd_op_reply(24 [pgnls-filter start_epoch 71 out=49b] v69'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636419b38c0 con 0x5636419f8800 2023-05-20T01:22:18.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.649+0000 7f70c7b93700 1 -- 172.21.15.16:0/3489021394 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:25 10.12 10:48000000::::head [pgnls-filter start_epoch 71 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x563641b04e00 con 0x5636419bcc00 2023-05-20T01:22:18.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.649+0000 7f70ccb9d700 1 -- 172.21.15.16:0/3489021394 <== osd.6 v2:172.21.15.132:6801/727581123 3 ==== osd_op_reply(25 [pgnls-filter start_epoch 71 out=49b] v69'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636419b3680 con 0x5636419bcc00 2023-05-20T01:22:18.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.649+0000 7f70ce3ee7c0 1 -- 172.21.15.16:0/3489021394 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x5636419bc800 msgr2=0x563641728c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.649+0000 7f70ce3ee7c0 1 --2- 172.21.15.16:0/3489021394 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x5636419bc800 0x563641728c00 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.649+0000 7f70ce3ee7c0 1 -- 172.21.15.16:0/3489021394 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x5636419bd800 msgr2=0x563641aff080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.649+0000 7f70ce3ee7c0 1 --2- 172.21.15.16:0/3489021394 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x5636419bd800 0x563641aff080 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.649+0000 7f70ce3ee7c0 1 -- 172.21.15.16:0/3489021394 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x5636419bd400 msgr2=0x563641afeb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.649+0000 7f70ce3ee7c0 1 --2- 172.21.15.16:0/3489021394 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x5636419bd400 0x563641afeb00 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.649+0000 7f70ce3ee7c0 1 -- 172.21.15.16:0/3489021394 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x5636419f8800 msgr2=0x563641afe000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.649+0000 7f70ce3ee7c0 1 --2- 172.21.15.16:0/3489021394 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x5636419f8800 0x563641afe000 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.649+0000 7f70ce3ee7c0 1 -- 172.21.15.16:0/3489021394 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x5636419bcc00 msgr2=0x563641729180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.649+0000 7f70ce3ee7c0 1 --2- 172.21.15.16:0/3489021394 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x5636419bcc00 0x563641729180 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.649+0000 7f70cbb9b700 1 -- 172.21.15.16:0/3489021394 reap_dead start 2023-05-20T01:22:18.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.649+0000 7f70ce3ee7c0 1 -- 172.21.15.16:0/3489021394 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x5636419f8c00 msgr2=0x563641afe580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.649+0000 7f70ce3ee7c0 1 --2- 172.21.15.16:0/3489021394 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x5636419f8c00 0x563641afe580 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.649+0000 7f70cbb9b700 1 -- 172.21.15.16:0/3489021394 reap_dead start 2023-05-20T01:22:18.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.649+0000 7f70ce3ee7c0 1 -- 172.21.15.16:0/3489021394 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x5636419f8400 msgr2=0x563641729700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.649+0000 7f70ce3ee7c0 1 --2- 172.21.15.16:0/3489021394 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x5636419f8400 0x563641729700 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x563642468e40 tx=0x56364196ccc0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.649+0000 7f70ce3ee7c0 1 -- 172.21.15.16:0/3489021394 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56364168d800 msgr2=0x563641727b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.649+0000 7f70ce3ee7c0 1 --2- 172.21.15.16:0/3489021394 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56364168d800 0x563641727b80 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x5636419bf350 tx=0x56364196cf00 comp rx=0 tx=0).stop 2023-05-20T01:22:18.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.649+0000 7f3b5969c700 1 -- 172.21.15.16:0/3957359014 <== osd.4 v2:172.21.15.132:6817/506317223 9 ==== osd_op_reply(18 10000000000.00000000 [omap-set-vals] v71'26 uv26 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56013b6578c0 con 0x56013b665800 2023-05-20T01:22:18.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.649+0000 7f3b5beef7c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/18 pointing to 0x0x10000000207 2023-05-20T01:22:18.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.649+0000 7f3b5beef7c0 1 -- 172.21.15.16:0/3957359014 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:19 10.7 10:e2e07930::::head [pgnls-filter start_epoch 71 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x56013b5fce00 con 0x56013b664c00 2023-05-20T01:22:18.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.649+0000 7f778129d700 1 -- 172.21.15.16:0/251369889 <== osd.4 v2:172.21.15.132:6817/506317223 8 ==== osd_op_reply(20 10000000000.00000000 [omap-set-vals] v71'27 uv27 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563d8a7ef680 con 0x563d8a80d400 2023-05-20T01:22:18.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.649+0000 7f7782aee7c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/1 pointing to 0x0x100000001f6 2023-05-20T01:22:18.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.649+0000 7f7782aee7c0 1 -- 172.21.15.16:0/251369889 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:21 10.1a 10:58000000::::head [pgnls-filter start_epoch 71 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x563d8a93aa80 con 0x563d8a80cc00 2023-05-20T01:22:18.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.649+0000 7f70ce3ee7c0 1 -- 172.21.15.16:0/3489021394 shutdown_connections 2023-05-20T01:22:18.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.649+0000 7f70ce3ee7c0 1 --2- 172.21.15.16:0/3489021394 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x5636419f8400 0x563641729700 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.649+0000 7f70ce3ee7c0 1 --2- 172.21.15.16:0/3489021394 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56364168d800 0x563641727b80 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.649+0000 7f70ce3ee7c0 1 --2- 172.21.15.16:0/3489021394 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x5636419f8c00 0x563641afe580 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.649+0000 7f70ce3ee7c0 1 -- 172.21.15.16:0/3489021394 >> 172.21.15.16:0/3489021394 conn(0x56364168dc00 msgr2=0x56364174d800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:18.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.649+0000 7f70ce3ee7c0 1 -- 172.21.15.16:0/3489021394 shutdown_connections 2023-05-20T01:22:18.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.649+0000 7f70ce3ee7c0 1 -- 172.21.15.16:0/3489021394 wait complete. 2023-05-20T01:22:18.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.649+0000 7fabbbd98700 1 -- 172.21.15.16:0/2723465486 <== osd.4 v2:172.21.15.132:6817/506317223 12 ==== osd_op_reply(29 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv27 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x5614bdd378c0 con 0x5614bde80400 2023-05-20T01:22:18.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.649+0000 7fabbddea7c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=8 frag=*, r=0 2023-05-20T01:22:18.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.649+0000 7fabbddea7c0 20 datascan.read_dentry: 8_head not found in result 2023-05-20T01:22:18.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.649+0000 7fabbddea7c0 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001fd at 0x0x10000000000/8 with size=6291456 bytes 2023-05-20T01:22:18.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.649+0000 7f3b5a69e700 1 -- 172.21.15.16:0/3957359014 <== osd.6 v2:172.21.15.132:6801/727581123 6 ==== osd_op_reply(19 [pgnls-filter start_epoch 71 out=81b] v71'13 uv13 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x56013b657680 con 0x56013b664c00 2023-05-20T01:22:18.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.649+0000 7fabbddea7c0 1 -- 172.21.15.16:0/2723465486 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:30 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x5614be7fae00 con 0x5614bde80400 2023-05-20T01:22:18.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.650+0000 7f3b5beef7c0 20 datascan.forall_objects: OSD matched oid 10000000001.00000000 2023-05-20T01:22:18.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.650+0000 7f3b5beef7c0 10 datascan.operator(): handling object 10000000001.0 2023-05-20T01:22:18.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.650+0000 7f3b5beef7c0 1 -- 172.21.15.16:0/3957359014 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:20 10.7 10:e2e07930:::10000000001.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x56013b5fd880 con 0x56013b664c00 2023-05-20T01:22:18.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.650+0000 7f778129d700 1 -- 172.21.15.16:0/251369889 <== osd.6 v2:172.21.15.132:6801/727581123 4 ==== osd_op_reply(21 [pgnls-filter start_epoch 71 out=49b] v69'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563d8a7ef680 con 0x563d8a80cc00 2023-05-20T01:22:18.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.650+0000 7f777c293700 1 --2- 172.21.15.16:0/251369889 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x563d8a80d000 0x563d8a925080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.650+0000 7f777c293700 1 -- 172.21.15.16:0/251369889 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:22 10.3a 10:5c000000::::head [pgnls-filter start_epoch 71 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x563d8a93b500 con 0x563d8a80d000 2023-05-20T01:22:18.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.650+0000 7f70ce3ee7c0 1 -- 172.21.15.16:0/3195839697 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56364168cc00 msgr2=0x563641726b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.650+0000 7f70ce3ee7c0 1 --2- 172.21.15.16:0/3195839697 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56364168cc00 0x563641726b00 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x563642468d50 tx=0x563642474000 comp rx=0 tx=0).stop 2023-05-20T01:22:18.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.650+0000 7f7780a9c700 1 --2- 172.21.15.16:0/251369889 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x563d8a80d000 0x563d8a925080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.650+0000 7f70ce3ee7c0 1 -- 172.21.15.16:0/3195839697 shutdown_connections 2023-05-20T01:22:18.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.650+0000 7f70ce3ee7c0 1 --2- 172.21.15.16:0/3195839697 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56364168d400 0x563641727600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.650+0000 7f70ce3ee7c0 1 --2- 172.21.15.16:0/3195839697 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56364168cc00 0x563641726b00 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.650+0000 7f70ce3ee7c0 1 --2- 172.21.15.16:0/3195839697 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56364168d000 0x563641727080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.650+0000 7f70ce3ee7c0 1 -- 172.21.15.16:0/3195839697 >> 172.21.15.16:0/3195839697 conn(0x56364168c400 msgr2=0x56364174d000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:18.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.650+0000 7f70ce3ee7c0 1 -- 172.21.15.16:0/3195839697 shutdown_connections 2023-05-20T01:22:18.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.650+0000 7f70ce3ee7c0 1 -- 172.21.15.16:0/3195839697 wait complete. 2023-05-20T01:22:18.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.650+0000 7f3b5a69e700 1 -- 172.21.15.16:0/3957359014 <== osd.6 v2:172.21.15.132:6801/727581123 7 ==== osd_op_reply(20 10000000001.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (97) out=97b,getxattr,getxattr] v0'0 uv13 ondisk = 0) v8 ==== 416+0+143 (crc 0 0 0) 0x56013b657680 con 0x56013b664c00 2023-05-20T01:22:18.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.650+0000 7f3b5beef7c0 1 -- 172.21.15.16:0/3957359014 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:21 11.7 11:e2e07930:::10000000001.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x56013b5fc700 con 0x56013b66c800 2023-05-20T01:22:18.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.650+0000 7f7780a9c700 1 --2- 172.21.15.16:0/251369889 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x563d8a80d000 0x563d8a925080 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.650+0000 7f3b5a69e700 1 -- 172.21.15.16:0/3957359014 <== osd.5 v2:172.21.15.132:6816/1844689995 2 ==== osd_op_reply(21 10000000001.00000000 [getxattr (30) out=30b] v0'0 uv5 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x56013b657680 con 0x56013b66c800 2023-05-20T01:22:18.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.650+0000 7f3b5beef7c0 20 datascan.operator(): loaded layout from xattr: pi: 11 os: 4194304 sc: 1 su: 4194304 2023-05-20T01:22:18.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.650+0000 7f3b5beef7c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000001 2023-05-20T01:22:18.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.650+0000 7f3b5beef7c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/0 2023-05-20T01:22:18.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.650+0000 7f3b5beef7c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=0 2023-05-20T01:22:18.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.650+0000 7f3b5beef7c0 1 -- 172.21.15.16:0/3957359014 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:22 9.10 9:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x56013b5fd180 con 0x56013b665800 2023-05-20T01:22:18.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.650+0000 7f7780a9c700 1 -- 172.21.15.16:0/251369889 <== osd.0 v2:172.21.15.16:6824/3923634179 1 ==== osd_op_reply(22 [pgnls-filter start_epoch 71 out=49b] v69'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563d8a7ef8c0 con 0x563d8a80d000 2023-05-20T01:22:18.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.650+0000 7f777ba92700 1 --2- 172.21.15.16:0/251369889 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x563d8a80f800 0x563d8a924b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.650+0000 7f777ba92700 1 -- 172.21.15.16:0/251369889 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:23 10.6 10:60000000::::head [pgnls-filter start_epoch 71 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x563d8a795c00 con 0x563d8a80f800 2023-05-20T01:22:18.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.651+0000 7fa3b3357700 1 -- 172.21.15.16:0/2859455639 <== osd.4 v2:172.21.15.132:6817/506317223 4 ==== osd_op_reply(12 10000000000.00000000 [omap-set-vals] v71'28 uv28 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x555b4a80d440 con 0x555b4a931000 2023-05-20T01:22:18.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.651+0000 7fa3b53a97c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/11 pointing to 0x0x10000000200 2023-05-20T01:22:18.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.651+0000 7fa3b53a97c0 1 -- 172.21.15.16:0/2859455639 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:13 10.19 10:997b7129::::head [pgnls-filter start_epoch 71 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x555b4b38b180 con 0x555b4a5dc400 2023-05-20T01:22:18.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.651+0000 7f778029b700 1 --2- 172.21.15.16:0/251369889 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x563d8a80f800 0x563d8a924b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.651+0000 7f3776024700 1 -- 172.21.15.16:0/1569316897 <== osd.4 v2:172.21.15.132:6817/506317223 10 ==== osd_op_reply(27 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x55ce84a25680 con 0x55ce83fbe400 2023-05-20T01:22:18.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.651+0000 7f37780767c0 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-05-20T01:22:18.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.651+0000 7f37780767c0 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-05-20T01:22:18.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.651+0000 7f37780767c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/14, will insert to root 2023-05-20T01:22:18.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.651+0000 7f37780767c0 1 -- 172.21.15.16:0/1569316897 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:28 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x55ce83ffca80 con 0x55ce83fbe400 2023-05-20T01:22:18.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.651+0000 7f90ce7a5700 1 -- 172.21.15.16:0/3200652326 <== osd.4 v2:172.21.15.132:6817/506317223 7 ==== osd_op_reply(16 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv28 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x556b170eb680 con 0x556b170ffc00 2023-05-20T01:22:18.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.651+0000 7f90cfff67c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=2 frag=*, r=0 2023-05-20T01:22:18.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.651+0000 7f90cfff67c0 20 datascan.read_dentry: 2_head not found in result 2023-05-20T01:22:18.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.651+0000 7f90cfff67c0 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001f7 at 0x0x10000000000/2 with size=6291456 bytes 2023-05-20T01:22:18.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.651+0000 7fa3b3357700 1 -- 172.21.15.16:0/2859455639 <== osd.1 v2:172.21.15.16:6808/3860264220 5 ==== osd_op_reply(13 [pgnls-filter start_epoch 71 out=81b] v71'18 uv18 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x555b4a80d440 con 0x555b4a5dc400 2023-05-20T01:22:18.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.651+0000 7f90cfff67c0 1 -- 172.21.15.16:0/3200652326 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:17 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x556b17c5d500 con 0x556b170ffc00 2023-05-20T01:22:18.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.651+0000 7fa3b53a97c0 20 datascan.forall_objects: OSD matched oid 100000001f8.00000000 2023-05-20T01:22:18.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.651+0000 7fa3b53a97c0 10 datascan.operator(): handling object 100000001f8.0 2023-05-20T01:22:18.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.651+0000 7fa3b53a97c0 1 -- 172.21.15.16:0/2859455639 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:14 10.19 10:997b7129:::100000001f8.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x555b4b38aa80 con 0x555b4a5dc400 2023-05-20T01:22:18.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.651+0000 7f778029b700 1 --2- 172.21.15.16:0/251369889 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x563d8a80f800 0x563d8a924b00 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.651+0000 7fa3b3357700 1 -- 172.21.15.16:0/2859455639 <== osd.1 v2:172.21.15.16:6808/3860264220 6 ==== osd_op_reply(14 100000001f8.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (97) out=97b,getxattr,getxattr] v0'0 uv17 ondisk = 0) v8 ==== 416+0+143 (crc 0 0 0) 0x555b4a80d440 con 0x555b4a5dc400 2023-05-20T01:22:18.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.651+0000 7fa3b53a97c0 1 -- 172.21.15.16:0/2859455639 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:15 11.19 11:997b7129:::100000001f8.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x555b4b38b500 con 0x555b4a930c00 2023-05-20T01:22:18.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.651+0000 7f778029b700 1 -- 172.21.15.16:0/251369889 <== osd.5 v2:172.21.15.132:6816/1844689995 1 ==== osd_op_reply(23 [pgnls-filter start_epoch 71 out=49b] v69'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563d8a7ef440 con 0x563d8a80f800 2023-05-20T01:22:18.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.652+0000 7f777c293700 1 -- 172.21.15.16:0/251369889 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:24 10.26 10:64000000::::head [pgnls-filter start_epoch 71 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x563d8a93b880 con 0x563d8a80f800 2023-05-20T01:22:18.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.652+0000 7fa3b3b58700 1 -- 172.21.15.16:0/2859455639 <== osd.0 v2:172.21.15.16:6824/3923634179 2 ==== osd_op_reply(15 100000001f8.00000000 [getxattr (30) out=30b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x555b4a80d680 con 0x555b4a930c00 2023-05-20T01:22:18.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.652+0000 7fa3b53a97c0 20 datascan.operator(): loaded layout from xattr: pi: 11 os: 4194304 sc: 1 su: 4194304 2023-05-20T01:22:18.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.652+0000 7fa3b53a97c0 10 datascan.inject_with_backtrace: inode: 0x0x100000001f8 2023-05-20T01:22:18.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.652+0000 7fa3b53a97c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/3 2023-05-20T01:22:18.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.652+0000 7fa3b53a97c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=3 2023-05-20T01:22:18.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.652+0000 7fa3b53a97c0 1 -- 172.21.15.16:0/2859455639 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:16 9.10 9:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x555b4b38a700 con 0x555b4a931000 2023-05-20T01:22:18.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.652+0000 7f778029b700 1 -- 172.21.15.16:0/251369889 <== osd.5 v2:172.21.15.132:6816/1844689995 2 ==== osd_op_reply(24 [pgnls-filter start_epoch 71 out=81b] v71'19 uv19 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x563d8a7ef440 con 0x563d8a80f800 2023-05-20T01:22:18.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.652+0000 7f7782aee7c0 20 datascan.forall_objects: OSD matched oid 100000001ff.00000000 2023-05-20T01:22:18.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.652+0000 7f7782aee7c0 10 datascan.operator(): handling object 100000001ff.0 2023-05-20T01:22:18.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.652+0000 7f7782aee7c0 1 -- 172.21.15.16:0/251369889 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:25 10.26 10:658f2455:::100000001ff.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x563d8a93a000 con 0x563d8a80f800 2023-05-20T01:22:18.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.652+0000 7fabbbd98700 1 -- 172.21.15.16:0/2723465486 <== osd.4 v2:172.21.15.132:6817/506317223 13 ==== osd_op_reply(30 10000000000.00000000 [omap-set-vals] v71'29 uv29 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5614bdd378c0 con 0x5614bde80400 2023-05-20T01:22:18.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.652+0000 7fabbddea7c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/8 pointing to 0x0x100000001fd 2023-05-20T01:22:18.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.652+0000 7fabbddea7c0 1 --2- 172.21.15.16:0/2723465486 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x5614bde81800 0x5614bde90b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.652+0000 7fabbddea7c0 1 -- 172.21.15.16:0/2723465486 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:31 10.30 10:0c000000::::head [pgnls-filter start_epoch 71 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x5614bdea6000 con 0x5614bde81800 2023-05-20T01:22:18.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.652+0000 7f3b5969c700 1 -- 172.21.15.16:0/3957359014 <== osd.4 v2:172.21.15.132:6817/506317223 10 ==== osd_op_reply(22 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x56013b6578c0 con 0x56013b665800 2023-05-20T01:22:18.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.652+0000 7f3b5beef7c0 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-05-20T01:22:18.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.652+0000 7f3b5beef7c0 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-05-20T01:22:18.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.652+0000 7fabbbd98700 1 --2- 172.21.15.16:0/2723465486 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x5614bde81800 0x5614bde90b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.652+0000 7f3b5beef7c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/0, will insert to root 2023-05-20T01:22:18.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.652+0000 7f3b5beef7c0 1 -- 172.21.15.16:0/3957359014 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:23 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x56013b5fca80 con 0x56013b665800 2023-05-20T01:22:18.913 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_inodes', '--worker_n', '1', '--worker_m', '7'] in time 0:00:00.389039, result: 2023-05-20T01:22:18.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.652+0000 7f778029b700 1 -- 172.21.15.16:0/251369889 <== osd.5 v2:172.21.15.132:6816/1844689995 3 ==== osd_op_reply(25 100000001ff.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (98) out=98b,getxattr,getxattr] v0'0 uv19 ondisk = 0) v8 ==== 416+0+144 (crc 0 0 0) 0x563d8a7ef440 con 0x563d8a80f800 2023-05-20T01:22:18.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.652+0000 7f7782aee7c0 1 -- 172.21.15.16:0/251369889 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:26 11.6 11:658f2455:::100000001ff.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x563d8a93b180 con 0x563d8a80d000 2023-05-20T01:22:18.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.653+0000 7fabbbd98700 1 --2- 172.21.15.16:0/2723465486 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x5614bde81800 0x5614bde90b00 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.653+0000 7f7780a9c700 1 -- 172.21.15.16:0/251369889 <== osd.0 v2:172.21.15.16:6824/3923634179 2 ==== osd_op_reply(26 100000001ff.00000000 [getxattr (30) out=30b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x563d8a7ef8c0 con 0x563d8a80d000 2023-05-20T01:22:18.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.653+0000 7f7782aee7c0 20 datascan.operator(): loaded layout from xattr: pi: 11 os: 4194304 sc: 1 su: 4194304 2023-05-20T01:22:18.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.653+0000 7f7782aee7c0 10 datascan.inject_with_backtrace: inode: 0x0x100000001ff 2023-05-20T01:22:18.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.653+0000 7f7782aee7c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/10 2023-05-20T01:22:18.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.653+0000 7f7782aee7c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=10 2023-05-20T01:22:18.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.653+0000 7f7782aee7c0 1 -- 172.21.15.16:0/251369889 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:27 9.10 9:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x563d8a795880 con 0x563d8a80d400 2023-05-20T01:22:18.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.653+0000 7fabbbd98700 1 -- 172.21.15.16:0/2723465486 <== osd.2 v2:172.21.15.16:6816/293187030 1 ==== osd_op_reply(31 [pgnls-filter start_epoch 71 out=49b] v69'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5614bdd378c0 con 0x5614bde81800 2023-05-20T01:22:18.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.653+0000 7fabb758f700 1 -- 172.21.15.16:0/2723465486 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:32 10.8 10:10000000::::head [pgnls-filter start_epoch 71 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x5614be7faa80 con 0x5614bde80800 2023-05-20T01:22:18.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.653+0000 7fabbb597700 1 -- 172.21.15.16:0/2723465486 <== osd.0 v2:172.21.15.16:6824/3923634179 4 ==== osd_op_reply(32 [pgnls-filter start_epoch 71 out=49b] v69'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5614bdd37440 con 0x5614bde80800 2023-05-20T01:22:18.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.653+0000 7fabb6d8e700 1 --2- 172.21.15.16:0/2723465486 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x5614bdeb0000 0x5614bde92680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.653+0000 7fabb6d8e700 1 -- 172.21.15.16:0/2723465486 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:33 10.28 10:14000000::::head [pgnls-filter start_epoch 71 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x5614be7fbc00 con 0x5614bdeb0000 2023-05-20T01:22:18.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.653+0000 7fabbb597700 1 --2- 172.21.15.16:0/2723465486 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x5614bdeb0000 0x5614bde92680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.654+0000 7fabbb597700 1 --2- 172.21.15.16:0/2723465486 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x5614bdeb0000 0x5614bde92680 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.654+0000 7fabbb597700 1 -- 172.21.15.16:0/2723465486 <== osd.7 v2:172.21.15.132:6800/466001149 1 ==== osd_op_reply(33 [pgnls-filter start_epoch 71 out=101b] v71'14 uv14 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x5614bdd37440 con 0x5614bdeb0000 2023-05-20T01:22:18.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.654+0000 7fabbddea7c0 20 datascan.forall_objects: OSD matched oid 10000000202.00000000 2023-05-20T01:22:18.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.654+0000 7fabbddea7c0 10 datascan.operator(): handling object 10000000202.0 2023-05-20T01:22:18.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.654+0000 7fabbddea7c0 1 -- 172.21.15.16:0/2723465486 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:34 10.28 10:15246602:::10000000202.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x5614bdea6380 con 0x5614bdeb0000 2023-05-20T01:22:18.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.654+0000 7f90ce7a5700 1 -- 172.21.15.16:0/3200652326 <== osd.4 v2:172.21.15.132:6817/506317223 8 ==== osd_op_reply(17 10000000000.00000000 [omap-set-vals] v71'30 uv30 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x556b170eb680 con 0x556b170ffc00 2023-05-20T01:22:18.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.654+0000 7f90cfff67c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/2 pointing to 0x0x100000001f7 2023-05-20T01:22:18.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.654+0000 7f90cfff67c0 1 -- 172.21.15.16:0/3200652326 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:18 10.e 10:70000000::::head [pgnls-filter start_epoch 71 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x556b17c5c700 con 0x556b170ffc00 2023-05-20T01:22:18.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.655+0000 7fabbb597700 1 -- 172.21.15.16:0/2723465486 <== osd.7 v2:172.21.15.132:6800/466001149 2 ==== osd_op_reply(34 10000000202.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (98) out=98b,getxattr,getxattr] v0'0 uv13 ondisk = 0) v8 ==== 416+0+144 (crc 0 0 0) 0x5614bdd37440 con 0x5614bdeb0000 2023-05-20T01:22:18.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.655+0000 7fabbddea7c0 1 -- 172.21.15.16:0/2723465486 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:35 11.8 11:15246602:::10000000202.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x5614bdea6a80 con 0x5614bde81000 2023-05-20T01:22:18.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.655+0000 7f3776024700 1 -- 172.21.15.16:0/1569316897 <== osd.4 v2:172.21.15.132:6817/506317223 11 ==== osd_op_reply(28 10000000000.00000000 [omap-get-header out=274b] v0'0 uv30 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x55ce84a25680 con 0x55ce83fbe400 2023-05-20T01:22:18.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.655+0000 7f37780767c0 10 datascan.find_or_create_dirfrag: read_version = 30 2023-05-20T01:22:18.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.655+0000 7f37780767c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-05-20T01:22:18.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.655+0000 7f37780767c0 1 -- 172.21.15.16:0/1569316897 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:29 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x55ce84a1d880 con 0x55ce83fbe400 2023-05-20T01:22:18.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.655+0000 7f90ce7a5700 1 -- 172.21.15.16:0/3200652326 <== osd.4 v2:172.21.15.132:6817/506317223 9 ==== osd_op_reply(18 [pgnls-filter start_epoch 71 out=49b] v69'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556b170eb680 con 0x556b170ffc00 2023-05-20T01:22:18.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.655+0000 7f90c979b700 1 --2- 172.21.15.16:0/3200652326 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x556b16e9c000 0x556b1721e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.655+0000 7f90c979b700 1 -- 172.21.15.16:0/3200652326 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:19 10.2e 10:74000000::::head [pgnls-filter start_epoch 71 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x556b17c5dc00 con 0x556b16e9c000 2023-05-20T01:22:18.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.655+0000 7fa3b3357700 1 -- 172.21.15.16:0/2859455639 <== osd.4 v2:172.21.15.132:6817/506317223 5 ==== osd_op_reply(16 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x555b4a80d440 con 0x555b4a931000 2023-05-20T01:22:18.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.655+0000 7fa3b53a97c0 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-05-20T01:22:18.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.655+0000 7fa3b53a97c0 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-05-20T01:22:18.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.655+0000 7fabbc599700 1 -- 172.21.15.16:0/2723465486 <== osd.3 v2:172.21.15.16:6800/3669781890 3 ==== osd_op_reply(35 10000000202.00000000 [getxattr (30) out=30b] v0'0 uv6 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x5614bdd37680 con 0x5614bde81000 2023-05-20T01:22:18.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.655+0000 7fa3b53a97c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/3, will insert to root 2023-05-20T01:22:18.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.655+0000 7fa3b53a97c0 1 -- 172.21.15.16:0/2859455639 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:17 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x555b4b38bc00 con 0x555b4a931000 2023-05-20T01:22:18.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.655+0000 7f90cdfa4700 1 --2- 172.21.15.16:0/3200652326 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x556b16e9c000 0x556b1721e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.655+0000 7f3b5969c700 1 -- 172.21.15.16:0/3957359014 <== osd.4 v2:172.21.15.132:6817/506317223 11 ==== osd_op_reply(23 10000000000.00000000 [omap-get-header out=274b] v0'0 uv30 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x56013b6578c0 con 0x56013b665800 2023-05-20T01:22:18.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.655+0000 7f3b5beef7c0 10 datascan.find_or_create_dirfrag: read_version = 30 2023-05-20T01:22:18.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.655+0000 7f3b5beef7c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-05-20T01:22:18.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.655+0000 7fabbddea7c0 20 datascan.operator(): loaded layout from xattr: pi: 11 os: 4194304 sc: 1 su: 4194304 2023-05-20T01:22:18.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.655+0000 7fabbddea7c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000202 2023-05-20T01:22:18.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.655+0000 7fabbddea7c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/13 2023-05-20T01:22:18.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.655+0000 7fabbddea7c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=13 2023-05-20T01:22:18.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.655+0000 7f3b5beef7c0 1 -- 172.21.15.16:0/3957359014 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:24 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x56013b5fdc00 con 0x56013b665800 2023-05-20T01:22:18.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.655+0000 7fabbddea7c0 1 -- 172.21.15.16:0/2723465486 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:36 9.10 9:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x5614bdea6e00 con 0x5614bde80400 2023-05-20T01:22:18.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.655+0000 7f778129d700 1 -- 172.21.15.16:0/251369889 <== osd.4 v2:172.21.15.132:6817/506317223 9 ==== osd_op_reply(27 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x563d8a7ef680 con 0x563d8a80d400 2023-05-20T01:22:18.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.655+0000 7f7782aee7c0 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-05-20T01:22:18.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.655+0000 7f7782aee7c0 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-05-20T01:22:18.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.655+0000 7f7782aee7c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/10, will insert to root 2023-05-20T01:22:18.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.655+0000 7f7782aee7c0 1 -- 172.21.15.16:0/251369889 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:28 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x563d8a795500 con 0x563d8a80d400 2023-05-20T01:22:18.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.655+0000 7f3776024700 1 -- 172.21.15.16:0/1569316897 <== osd.4 v2:172.21.15.132:6817/506317223 12 ==== osd_op_reply(29 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv30 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x55ce84a25680 con 0x55ce83fbe400 2023-05-20T01:22:18.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.655+0000 7f37780767c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=14 frag=*, r=0 2023-05-20T01:22:18.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.655+0000 7f37780767c0 20 datascan.read_dentry: 14_head not found in result 2023-05-20T01:22:18.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.655+0000 7f37780767c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000203 at 0x0x10000000000/14 with size=6291456 bytes 2023-05-20T01:22:18.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.655+0000 7f37780767c0 1 -- 172.21.15.16:0/1569316897 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:30 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x55ce84a1d500 con 0x55ce83fbe400 2023-05-20T01:22:18.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.655+0000 7f90cdfa4700 1 --2- 172.21.15.16:0/3200652326 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x556b16e9c000 0x556b1721e680 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.655+0000 7fa3b3357700 1 -- 172.21.15.16:0/2859455639 <== osd.4 v2:172.21.15.132:6817/506317223 6 ==== osd_op_reply(17 10000000000.00000000 [omap-get-header out=274b] v0'0 uv30 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x555b4a80d440 con 0x555b4a931000 2023-05-20T01:22:18.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.655+0000 7fa3b53a97c0 10 datascan.find_or_create_dirfrag: read_version = 30 2023-05-20T01:22:18.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.655+0000 7fa3b53a97c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-05-20T01:22:18.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.655+0000 7fa3b53a97c0 1 -- 172.21.15.16:0/2859455639 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:18 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x555b4b38b180 con 0x555b4a931000 2023-05-20T01:22:18.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.656+0000 7fabbbd98700 1 -- 172.21.15.16:0/2723465486 <== osd.4 v2:172.21.15.132:6817/506317223 14 ==== osd_op_reply(36 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x5614bdd378c0 con 0x5614bde80400 2023-05-20T01:22:18.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.656+0000 7fabbddea7c0 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-05-20T01:22:18.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.656+0000 7fabbddea7c0 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-05-20T01:22:18.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.656+0000 7fabbddea7c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/13, will insert to root 2023-05-20T01:22:18.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.656+0000 7fabbddea7c0 1 -- 172.21.15.16:0/2723465486 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:37 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x5614bdea7180 con 0x5614bde80400 2023-05-20T01:22:18.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.656+0000 7f90cdfa4700 1 -- 172.21.15.16:0/3200652326 <== osd.3 v2:172.21.15.16:6800/3669781890 1 ==== osd_op_reply(19 [pgnls-filter start_epoch 71 out=49b] v69'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556b170eb440 con 0x556b16e9c000 2023-05-20T01:22:18.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.656+0000 7f3b5969c700 1 -- 172.21.15.16:0/3957359014 <== osd.4 v2:172.21.15.132:6817/506317223 12 ==== osd_op_reply(24 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv30 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x56013b6578c0 con 0x56013b665800 2023-05-20T01:22:18.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.656+0000 7f3b5beef7c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=0 frag=*, r=0 2023-05-20T01:22:18.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.656+0000 7f3b5beef7c0 20 datascan.read_dentry: 0_head not found in result 2023-05-20T01:22:18.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.656+0000 7f3b5beef7c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000001 at 0x0x10000000000/0 with size=6291456 bytes 2023-05-20T01:22:18.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.656+0000 7f3b5beef7c0 1 -- 172.21.15.16:0/3957359014 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:25 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x56013b5fce00 con 0x56013b665800 2023-05-20T01:22:18.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.656+0000 7f90c8f9a700 1 --2- 172.21.15.16:0/3200652326 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x556b16e9c400 0x556b1721ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.656+0000 7f90c8f9a700 1 -- 172.21.15.16:0/3200652326 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:20 10.1e 10:78000000::::head [pgnls-filter start_epoch 71 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x556b17c5ce00 con 0x556b16e9c400 2023-05-20T01:22:18.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.656+0000 7f778129d700 1 -- 172.21.15.16:0/251369889 <== osd.4 v2:172.21.15.132:6817/506317223 10 ==== osd_op_reply(28 10000000000.00000000 [omap-get-header out=274b] v0'0 uv30 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x563d8a7ef680 con 0x563d8a80d400 2023-05-20T01:22:18.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.656+0000 7f7782aee7c0 10 datascan.find_or_create_dirfrag: read_version = 30 2023-05-20T01:22:18.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.656+0000 7f7782aee7c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-05-20T01:22:18.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.656+0000 7f7782aee7c0 1 -- 172.21.15.16:0/251369889 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:29 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x563d8a93a700 con 0x563d8a80d400 2023-05-20T01:22:18.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.656+0000 7f90cd7a3700 1 --2- 172.21.15.16:0/3200652326 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x556b16e9c400 0x556b1721ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.656+0000 7fa3b3357700 1 -- 172.21.15.16:0/2859455639 <== osd.4 v2:172.21.15.132:6817/506317223 7 ==== osd_op_reply(18 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv30 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x555b4a80d440 con 0x555b4a931000 2023-05-20T01:22:18.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.656+0000 7fa3b53a97c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=3 frag=*, r=0 2023-05-20T01:22:18.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.656+0000 7fa3b53a97c0 20 datascan.read_dentry: 3_head not found in result 2023-05-20T01:22:18.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.656+0000 7fa3b53a97c0 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001f8 at 0x0x10000000000/3 with size=6291456 bytes 2023-05-20T01:22:18.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.656+0000 7fa3b53a97c0 1 -- 172.21.15.16:0/2859455639 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:19 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x555b4b38ae00 con 0x555b4a931000 2023-05-20T01:22:18.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.656+0000 7f90cd7a3700 1 --2- 172.21.15.16:0/3200652326 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x556b16e9c400 0x556b1721ec00 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.656+0000 7f90cd7a3700 1 -- 172.21.15.16:0/3200652326 <== osd.0 v2:172.21.15.16:6824/3923634179 1 ==== osd_op_reply(20 [pgnls-filter start_epoch 71 out=81b] v71'9 uv9 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x556b170eb8c0 con 0x556b16e9c400 2023-05-20T01:22:18.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.656+0000 7f90cfff67c0 20 datascan.forall_objects: OSD matched oid 1000000020c.00000000 2023-05-20T01:22:18.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.656+0000 7f90cfff67c0 10 datascan.operator(): handling object 1000000020c.0 2023-05-20T01:22:18.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.656+0000 7f90cfff67c0 1 -- 172.21.15.16:0/3200652326 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:21 10.1e 10:789b801b:::1000000020c.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x556b17c5d180 con 0x556b16e9c400 2023-05-20T01:22:18.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.657+0000 7f90cd7a3700 1 -- 172.21.15.16:0/3200652326 <== osd.0 v2:172.21.15.16:6824/3923634179 2 ==== osd_op_reply(21 1000000020c.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (98) out=98b,getxattr,getxattr] v0'0 uv9 ondisk = 0) v8 ==== 416+0+144 (crc 0 0 0) 0x556b170eb8c0 con 0x556b16e9c400 2023-05-20T01:22:18.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.657+0000 7f90cfff67c0 1 -- 172.21.15.16:0/3200652326 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:22 11.1e 11:789b801b:::1000000020c.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x556b17c5d500 con 0x556b170ff000 2023-05-20T01:22:18.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.657+0000 7f90cdfa4700 1 -- 172.21.15.16:0/3200652326 <== osd.7 v2:172.21.15.132:6800/466001149 5 ==== osd_op_reply(22 1000000020c.00000000 [getxattr (30) out=30b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x556b170eb440 con 0x556b170ff000 2023-05-20T01:22:18.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.657+0000 7f90cfff67c0 20 datascan.operator(): loaded layout from xattr: pi: 11 os: 4194304 sc: 1 su: 4194304 2023-05-20T01:22:18.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.657+0000 7f90cfff67c0 10 datascan.inject_with_backtrace: inode: 0x0x1000000020c 2023-05-20T01:22:18.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.657+0000 7f90cfff67c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/23 2023-05-20T01:22:18.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.657+0000 7f90cfff67c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=23 2023-05-20T01:22:18.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.657+0000 7f90cfff67c0 1 -- 172.21.15.16:0/3200652326 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:23 9.10 9:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x556b1723c000 con 0x556b170ffc00 2023-05-20T01:22:18.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.657+0000 7f3776024700 1 -- 172.21.15.16:0/1569316897 <== osd.4 v2:172.21.15.132:6817/506317223 13 ==== osd_op_reply(30 10000000000.00000000 [omap-set-vals] v71'31 uv31 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55ce84a25680 con 0x55ce83fbe400 2023-05-20T01:22:18.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.657+0000 7f37780767c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/14 pointing to 0x0x10000000203 2023-05-20T01:22:18.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.657+0000 7f37780767c0 1 -- 172.21.15.16:0/1569316897 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:31 10.1b 10:d8000000::::head [pgnls-filter start_epoch 71 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x55ce84a1ca80 con 0x55ce83fbfc00 2023-05-20T01:22:18.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.658+0000 7f3775823700 1 -- 172.21.15.16:0/1569316897 <== osd.3 v2:172.21.15.16:6800/3669781890 2 ==== osd_op_reply(31 [pgnls-filter start_epoch 71 out=49b] v69'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ce84a25b00 con 0x55ce83fbfc00 2023-05-20T01:22:18.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.658+0000 7f37780767c0 1 -- 172.21.15.16:0/1569316897 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x55ce83eaec00 msgr2=0x55ce83c20680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.658+0000 7f37780767c0 1 --2- 172.21.15.16:0/1569316897 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x55ce83eaec00 0x55ce83c20680 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.658+0000 7f37780767c0 1 -- 172.21.15.16:0/1569316897 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x55ce83fbec00 msgr2=0x55ce83fe2580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.658+0000 7f37780767c0 1 --2- 172.21.15.16:0/1569316897 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x55ce83fbec00 0x55ce83fe2580 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.658+0000 7f37780767c0 1 -- 172.21.15.16:0/1569316897 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x55ce83fbfc00 msgr2=0x55ce83fe3080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.658+0000 7f37780767c0 1 --2- 172.21.15.16:0/1569316897 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x55ce83fbfc00 0x55ce83fe3080 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.658+0000 7f37780767c0 1 -- 172.21.15.16:0/1569316897 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x55ce83fbe400 msgr2=0x55ce83c21700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.658+0000 7f37780767c0 1 --2- 172.21.15.16:0/1569316897 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x55ce83fbe400 0x55ce83c21700 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.658+0000 7f3775823700 1 -- 172.21.15.16:0/1569316897 reap_dead start 2023-05-20T01:22:18.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.658+0000 7f37780767c0 1 -- 172.21.15.16:0/1569316897 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x55ce83fbf400 msgr2=0x55ce83fe2b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.658+0000 7f37780767c0 1 --2- 172.21.15.16:0/1569316897 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x55ce83fbf400 0x55ce83fe2b00 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.658+0000 7f37780767c0 1 -- 172.21.15.16:0/1569316897 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x55ce83eaf000 msgr2=0x55ce83c20c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.658+0000 7f37780767c0 1 --2- 172.21.15.16:0/1569316897 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x55ce83eaf000 0x55ce83c20c00 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.658+0000 7f37780767c0 1 -- 172.21.15.16:0/1569316897 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x55ce83fbe800 msgr2=0x55ce83fe2000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.658+0000 7f37780767c0 1 --2- 172.21.15.16:0/1569316897 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x55ce83fbe800 0x55ce83fe2000 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.658+0000 7f3775823700 1 -- 172.21.15.16:0/1569316897 reap_dead start 2023-05-20T01:22:18.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.658+0000 7f37780767c0 1 -- 172.21.15.16:0/1569316897 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x55ce83fbe000 msgr2=0x55ce83c21180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.658+0000 7f37780767c0 1 --2- 172.21.15.16:0/1569316897 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x55ce83fbe000 0x55ce83c21180 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x55ce83eb82a0 tx=0x55ce8496c180 comp rx=0 tx=0).stop 2023-05-20T01:22:18.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.658+0000 7f37780767c0 1 -- 172.21.15.16:0/1569316897 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ce83eae800 msgr2=0x55ce83c20100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.658+0000 7f37780767c0 1 --2- 172.21.15.16:0/1569316897 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ce83eae800 0x55ce83c20100 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x55ce83eb9020 tx=0x55ce84a2ad20 comp rx=0 tx=0).stop 2023-05-20T01:22:18.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.658+0000 7f37780767c0 1 -- 172.21.15.16:0/1569316897 shutdown_connections 2023-05-20T01:22:18.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.658+0000 7f37780767c0 1 --2- 172.21.15.16:0/1569316897 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x55ce83fbe000 0x55ce83c21180 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.658+0000 7f37780767c0 1 --2- 172.21.15.16:0/1569316897 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ce83eae800 0x55ce83c20100 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.658+0000 7f37780767c0 1 -- 172.21.15.16:0/1569316897 >> 172.21.15.16:0/1569316897 conn(0x55ce83b84400 msgr2=0x55ce83c45800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:18.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.658+0000 7f37780767c0 1 -- 172.21.15.16:0/1569316897 shutdown_connections 2023-05-20T01:22:18.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.658+0000 7f37780767c0 1 -- 172.21.15.16:0/1569316897 wait complete. 2023-05-20T01:22:18.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.658+0000 7f37780767c0 1 -- 172.21.15.16:0/3796352817 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce83b85000 msgr2=0x55ce83c1eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.658+0000 7f37780767c0 1 --2- 172.21.15.16:0/3796352817 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce83b85000 0x55ce83c1eb00 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x55ce8494e3c0 tx=0x55ce8496c000 comp rx=0 tx=0).stop 2023-05-20T01:22:18.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.658+0000 7f37780767c0 1 -- 172.21.15.16:0/3796352817 shutdown_connections 2023-05-20T01:22:18.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.658+0000 7f37780767c0 1 --2- 172.21.15.16:0/3796352817 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ce83b84800 0x55ce83c1f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.658+0000 7f37780767c0 1 --2- 172.21.15.16:0/3796352817 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce83b85000 0x55ce83c1eb00 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.658+0000 7f37780767c0 1 --2- 172.21.15.16:0/3796352817 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ce83b84c00 0x55ce83c1e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.658+0000 7f37780767c0 1 -- 172.21.15.16:0/3796352817 >> 172.21.15.16:0/3796352817 conn(0x55ce83b85400 msgr2=0x55ce83c45000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:18.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.658+0000 7f37780767c0 1 -- 172.21.15.16:0/3796352817 shutdown_connections 2023-05-20T01:22:18.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.658+0000 7f37780767c0 1 -- 172.21.15.16:0/3796352817 wait complete. 2023-05-20T01:22:18.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.660+0000 7f3b5969c700 1 -- 172.21.15.16:0/3957359014 <== osd.4 v2:172.21.15.132:6817/506317223 13 ==== osd_op_reply(25 10000000000.00000000 [omap-set-vals] v71'32 uv32 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56013b6578c0 con 0x56013b665800 2023-05-20T01:22:18.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.660+0000 7f3b5beef7c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/0 pointing to 0x0x10000000001 2023-05-20T01:22:18.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.660+0000 7f3b5beef7c0 1 -- 172.21.15.16:0/3957359014 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:26 10.27 10:e4000000::::head [pgnls-filter start_epoch 71 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x56013b7a6000 con 0x56013b66c800 2023-05-20T01:22:18.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.660+0000 7f3b5a69e700 1 -- 172.21.15.16:0/3957359014 <== osd.5 v2:172.21.15.132:6816/1844689995 3 ==== osd_op_reply(26 [pgnls-filter start_epoch 71 out=81b] v71'6 uv6 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x56013b657680 con 0x56013b66c800 2023-05-20T01:22:18.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.660+0000 7f3b5beef7c0 20 datascan.forall_objects: OSD matched oid 10000000208.00000000 2023-05-20T01:22:18.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.660+0000 7f3b5beef7c0 10 datascan.operator(): handling object 10000000208.0 2023-05-20T01:22:18.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.660+0000 7f3b5beef7c0 1 -- 172.21.15.16:0/3957359014 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:27 10.27 10:e5993619:::10000000208.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x56013b7a6380 con 0x56013b66c800 2023-05-20T01:22:18.947 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_inodes', '--worker_n', '5', '--worker_m', '7'] in time 0:00:00.419952, result: 2023-05-20T01:22:18.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.661+0000 7f3b5a69e700 1 -- 172.21.15.16:0/3957359014 <== osd.5 v2:172.21.15.132:6816/1844689995 4 ==== osd_op_reply(27 10000000208.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (98) out=98b,getxattr,getxattr] v0'0 uv6 ondisk = 0) v8 ==== 416+0+144 (crc 0 0 0) 0x56013b657680 con 0x56013b66c800 2023-05-20T01:22:18.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.661+0000 7f3b5beef7c0 1 -- 172.21.15.16:0/3957359014 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:28 11.7 11:e5993619:::10000000208.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x56013b7a6a80 con 0x56013b66c800 2023-05-20T01:22:18.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.661+0000 7f3b5a69e700 1 -- 172.21.15.16:0/3957359014 <== osd.5 v2:172.21.15.132:6816/1844689995 5 ==== osd_op_reply(28 10000000208.00000000 [getxattr (30) out=30b] v0'0 uv7 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x56013b657680 con 0x56013b66c800 2023-05-20T01:22:18.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.661+0000 7f3b5beef7c0 20 datascan.operator(): loaded layout from xattr: pi: 11 os: 4194304 sc: 1 su: 4194304 2023-05-20T01:22:18.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.661+0000 7f3b5beef7c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000208 2023-05-20T01:22:18.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.661+0000 7f3b5beef7c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/19 2023-05-20T01:22:18.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.661+0000 7f3b5beef7c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=19 2023-05-20T01:22:18.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.661+0000 7f3b5beef7c0 1 -- 172.21.15.16:0/3957359014 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:29 9.10 9:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x56013b7a6e00 con 0x56013b665800 2023-05-20T01:22:18.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.662+0000 7fa3b3357700 1 -- 172.21.15.16:0/2859455639 <== osd.4 v2:172.21.15.132:6817/506317223 8 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v71'33 uv33 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x555b4a80d440 con 0x555b4a931000 2023-05-20T01:22:18.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.662+0000 7fa3b53a97c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/3 pointing to 0x0x100000001f8 2023-05-20T01:22:18.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.663+0000 7fa3b53a97c0 1 --2- 172.21.15.16:0/2859455639 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x555b4a931800 0x555b4a967080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.663+0000 7fa3b53a97c0 1 -- 172.21.15.16:0/2859455639 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:20 10.39 10:9c000000::::head [pgnls-filter start_epoch 71 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x555b4b38aa80 con 0x555b4a931800 2023-05-20T01:22:18.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.663+0000 7f778129d700 1 -- 172.21.15.16:0/251369889 <== osd.4 v2:172.21.15.132:6817/506317223 11 ==== osd_op_reply(29 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv33 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x563d8a7ef680 con 0x563d8a80d400 2023-05-20T01:22:18.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.663+0000 7f7782aee7c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=10 frag=*, r=0 2023-05-20T01:22:18.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.663+0000 7f7782aee7c0 20 datascan.read_dentry: 10_head not found in result 2023-05-20T01:22:18.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.663+0000 7f7782aee7c0 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001ff at 0x0x10000000000/10 with size=6291456 bytes 2023-05-20T01:22:18.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.663+0000 7fabbbd98700 1 -- 172.21.15.16:0/2723465486 <== osd.4 v2:172.21.15.132:6817/506317223 15 ==== osd_op_reply(37 10000000000.00000000 [omap-get-header out=274b] v0'0 uv33 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5614bdd378c0 con 0x5614bde80400 2023-05-20T01:22:18.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.663+0000 7f7782aee7c0 1 -- 172.21.15.16:0/251369889 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:30 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x563d8a794700 con 0x563d8a80d400 2023-05-20T01:22:18.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.663+0000 7f90ce7a5700 1 -- 172.21.15.16:0/3200652326 <== osd.4 v2:172.21.15.132:6817/506317223 10 ==== osd_op_reply(23 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x556b170eb680 con 0x556b170ffc00 2023-05-20T01:22:18.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.663+0000 7fa3b2b56700 1 --2- 172.21.15.16:0/2859455639 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x555b4a931800 0x555b4a967080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.663+0000 7f90cfff67c0 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-05-20T01:22:18.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.663+0000 7f90cfff67c0 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-05-20T01:22:18.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.663+0000 7f90cfff67c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/23, will insert to root 2023-05-20T01:22:18.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.663+0000 7fabbddea7c0 10 datascan.find_or_create_dirfrag: read_version = 33 2023-05-20T01:22:18.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.663+0000 7fabbddea7c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-05-20T01:22:18.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.663+0000 7f3b5969c700 1 -- 172.21.15.16:0/3957359014 <== osd.4 v2:172.21.15.132:6817/506317223 14 ==== osd_op_reply(29 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x56013b6578c0 con 0x56013b665800 2023-05-20T01:22:18.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.663+0000 7fabbddea7c0 1 -- 172.21.15.16:0/2723465486 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:38 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x5614bdea7500 con 0x5614bde80400 2023-05-20T01:22:18.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.663+0000 7f90cfff67c0 1 -- 172.21.15.16:0/3200652326 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:24 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x556b1723c380 con 0x556b170ffc00 2023-05-20T01:22:18.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.663+0000 7f3b5beef7c0 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-05-20T01:22:18.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.663+0000 7f3b5beef7c0 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-05-20T01:22:18.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.663+0000 7f3b5beef7c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/19, will insert to root 2023-05-20T01:22:18.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.663+0000 7f3b5beef7c0 1 -- 172.21.15.16:0/3957359014 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:30 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x56013b7a7180 con 0x56013b665800 2023-05-20T01:22:18.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.663+0000 7fa3b2b56700 1 --2- 172.21.15.16:0/2859455639 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x555b4a931800 0x555b4a967080 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.663+0000 7fa3b2b56700 1 -- 172.21.15.16:0/2859455639 <== osd.2 v2:172.21.15.16:6816/293187030 1 ==== osd_op_reply(20 [pgnls-filter start_epoch 71 out=49b] v69'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555b4a80d8c0 con 0x555b4a931800 2023-05-20T01:22:18.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.663+0000 7fa3ae34d700 1 -- 172.21.15.16:0/2859455639 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:21 10.5 10:a0000000::::head [pgnls-filter start_epoch 71 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x555b4a97c000 con 0x555b4a931800 2023-05-20T01:22:18.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.664+0000 7fa3b2b56700 1 -- 172.21.15.16:0/2859455639 <== osd.2 v2:172.21.15.16:6816/293187030 2 ==== osd_op_reply(21 [pgnls-filter start_epoch 71 out=81b] v71'9 uv9 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x555b4a80d8c0 con 0x555b4a931800 2023-05-20T01:22:18.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.664+0000 7fa3b53a97c0 20 datascan.forall_objects: OSD matched oid 10000000204.00000000 2023-05-20T01:22:18.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.664+0000 7fa3b53a97c0 10 datascan.operator(): handling object 10000000204.0 2023-05-20T01:22:18.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.664+0000 7fa3b53a97c0 1 -- 172.21.15.16:0/2859455639 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:22 10.5 10:a0beda95:::10000000204.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x555b4b38a700 con 0x555b4a931800 2023-05-20T01:22:18.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.664+0000 7fa3b2b56700 1 -- 172.21.15.16:0/2859455639 <== osd.2 v2:172.21.15.16:6816/293187030 3 ==== osd_op_reply(22 10000000204.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (98) out=98b,getxattr,getxattr] v0'0 uv9 ondisk = 0) v8 ==== 416+0+144 (crc 0 0 0) 0x555b4a80d8c0 con 0x555b4a931800 2023-05-20T01:22:18.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.664+0000 7fa3b53a97c0 1 --2- 172.21.15.16:0/2859455639 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x555b4a986000 0x555b4a967600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.664+0000 7fa3b53a97c0 1 -- 172.21.15.16:0/2859455639 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:23 11.5 11:a0beda95:::10000000204.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x555b4a97c700 con 0x555b4a986000 2023-05-20T01:22:18.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.665+0000 7fa3b3b58700 1 --2- 172.21.15.16:0/2859455639 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x555b4a986000 0x555b4a967600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.665+0000 7fa3b3b58700 1 --2- 172.21.15.16:0/2859455639 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x555b4a986000 0x555b4a967600 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.665+0000 7f778129d700 1 -- 172.21.15.16:0/251369889 <== osd.4 v2:172.21.15.132:6817/506317223 12 ==== osd_op_reply(30 10000000000.00000000 [omap-set-vals] v71'34 uv34 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563d8a7ef680 con 0x563d8a80d400 2023-05-20T01:22:18.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.665+0000 7f7782aee7c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/10 pointing to 0x0x100000001ff 2023-05-20T01:22:18.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.665+0000 7f7782aee7c0 1 -- 172.21.15.16:0/251369889 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:31 10.16 10:68000000::::head [pgnls-filter start_epoch 71 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x563d8a795880 con 0x563d8a80cc00 2023-05-20T01:22:18.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.665+0000 7fa3b3b58700 1 -- 172.21.15.16:0/2859455639 <== osd.7 v2:172.21.15.132:6800/466001149 1 ==== osd_op_reply(23 10000000204.00000000 [getxattr (30) out=30b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x555b4a80d680 con 0x555b4a986000 2023-05-20T01:22:18.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.665+0000 7fa3b53a97c0 20 datascan.operator(): loaded layout from xattr: pi: 11 os: 4194304 sc: 1 su: 4194304 2023-05-20T01:22:18.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.665+0000 7fa3b53a97c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000204 2023-05-20T01:22:18.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.665+0000 7fa3b53a97c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/15 2023-05-20T01:22:18.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.665+0000 7fa3b53a97c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=15 2023-05-20T01:22:18.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.665+0000 7fa3b53a97c0 1 -- 172.21.15.16:0/2859455639 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:24 9.10 9:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x555b4a97ca80 con 0x555b4a931000 2023-05-20T01:22:18.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.665+0000 7fabbbd98700 1 -- 172.21.15.16:0/2723465486 <== osd.4 v2:172.21.15.132:6817/506317223 16 ==== osd_op_reply(38 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv34 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x5614bdd378c0 con 0x5614bde80400 2023-05-20T01:22:18.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.665+0000 7fabbddea7c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=13 frag=*, r=0 2023-05-20T01:22:18.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.665+0000 7fabbddea7c0 20 datascan.read_dentry: 13_head not found in result 2023-05-20T01:22:18.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.665+0000 7fabbddea7c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000202 at 0x0x10000000000/13 with size=6291456 bytes 2023-05-20T01:22:18.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.665+0000 7fabbddea7c0 1 -- 172.21.15.16:0/2723465486 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:39 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x5614bdea7880 con 0x5614bde80400 2023-05-20T01:22:18.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.665+0000 7f90ce7a5700 1 -- 172.21.15.16:0/3200652326 <== osd.4 v2:172.21.15.132:6817/506317223 11 ==== osd_op_reply(24 10000000000.00000000 [omap-get-header out=274b] v0'0 uv34 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x556b170eb680 con 0x556b170ffc00 2023-05-20T01:22:18.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.665+0000 7f90cfff67c0 10 datascan.find_or_create_dirfrag: read_version = 34 2023-05-20T01:22:18.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.665+0000 7f90cfff67c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-05-20T01:22:18.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.665+0000 7f90cfff67c0 1 -- 172.21.15.16:0/3200652326 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:25 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x556b1723c700 con 0x556b170ffc00 2023-05-20T01:22:18.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.665+0000 7f778129d700 1 -- 172.21.15.16:0/251369889 <== osd.6 v2:172.21.15.132:6801/727581123 5 ==== osd_op_reply(31 [pgnls-filter start_epoch 71 out=49b] v69'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563d8a7ef680 con 0x563d8a80cc00 2023-05-20T01:22:18.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.666+0000 7f777c293700 1 -- 172.21.15.16:0/251369889 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:32 10.36 10:6c000000::::head [pgnls-filter start_epoch 71 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x563d8a93aa80 con 0x563d8a80f000 2023-05-20T01:22:18.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.666+0000 7f3b5969c700 1 -- 172.21.15.16:0/3957359014 <== osd.4 v2:172.21.15.132:6817/506317223 15 ==== osd_op_reply(30 10000000000.00000000 [omap-get-header out=274b] v0'0 uv34 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x56013b6578c0 con 0x56013b665800 2023-05-20T01:22:18.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.666+0000 7f3b5beef7c0 10 datascan.find_or_create_dirfrag: read_version = 34 2023-05-20T01:22:18.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.666+0000 7f3b5beef7c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-05-20T01:22:18.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.666+0000 7f3b5beef7c0 1 -- 172.21.15.16:0/3957359014 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:31 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x56013b7a7500 con 0x56013b665800 2023-05-20T01:22:18.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.666+0000 7fa3b3357700 1 -- 172.21.15.16:0/2859455639 <== osd.4 v2:172.21.15.132:6817/506317223 9 ==== osd_op_reply(24 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x555b4a80d440 con 0x555b4a931000 2023-05-20T01:22:18.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.666+0000 7fa3b53a97c0 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-05-20T01:22:18.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.666+0000 7fa3b53a97c0 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-05-20T01:22:18.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.666+0000 7fa3b53a97c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/15, will insert to root 2023-05-20T01:22:18.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.666+0000 7fa3b53a97c0 1 -- 172.21.15.16:0/2859455639 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:25 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x555b4a97ce00 con 0x555b4a931000 2023-05-20T01:22:18.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.666+0000 7f7780a9c700 1 -- 172.21.15.16:0/251369889 <== osd.7 v2:172.21.15.132:6800/466001149 2 ==== osd_op_reply(32 [pgnls-filter start_epoch 71 out=101b] v71'23 uv23 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x563d8a7ef8c0 con 0x563d8a80f000 2023-05-20T01:22:18.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.666+0000 7f7782aee7c0 1 -- 172.21.15.16:0/251369889 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x563d8a80d000 msgr2=0x563d8a925080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.666+0000 7f7782aee7c0 1 --2- 172.21.15.16:0/251369889 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x563d8a80d000 0x563d8a925080 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.666+0000 7f7782aee7c0 1 -- 172.21.15.16:0/251369889 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x563d8a80c800 msgr2=0x563d8a53e100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.666+0000 7f7782aee7c0 1 --2- 172.21.15.16:0/251369889 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x563d8a80c800 0x563d8a53e100 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.666+0000 7f7782aee7c0 1 -- 172.21.15.16:0/251369889 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x563d8a80f400 msgr2=0x563d8a924580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.666+0000 7f7782aee7c0 1 --2- 172.21.15.16:0/251369889 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x563d8a80f400 0x563d8a924580 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.666+0000 7f7782aee7c0 1 -- 172.21.15.16:0/251369889 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x563d8a80d400 msgr2=0x563d8a53d600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.666+0000 7f7782aee7c0 1 --2- 172.21.15.16:0/251369889 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x563d8a80d400 0x563d8a53d600 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.666+0000 7f7782aee7c0 1 -- 172.21.15.16:0/251369889 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x563d8a80f800 msgr2=0x563d8a924b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.666+0000 7f7782aee7c0 1 --2- 172.21.15.16:0/251369889 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x563d8a80f800 0x563d8a924b00 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.666+0000 7f7782aee7c0 1 -- 172.21.15.16:0/251369889 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x563d8a80cc00 msgr2=0x563d8a53db80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.666+0000 7f778029b700 1 -- 172.21.15.16:0/251369889 reap_dead start 2023-05-20T01:22:18.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.666+0000 7f7782aee7c0 1 --2- 172.21.15.16:0/251369889 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x563d8a80cc00 0x563d8a53db80 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.666+0000 7f778029b700 1 -- 172.21.15.16:0/251369889 reap_dead start 2023-05-20T01:22:18.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.666+0000 7f7782aee7c0 1 -- 172.21.15.16:0/251369889 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x563d8a80f000 msgr2=0x563d8a924000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.666+0000 7f778029b700 1 -- 172.21.15.16:0/251369889 reap_dead start 2023-05-20T01:22:18.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.666+0000 7f7782aee7c0 1 --2- 172.21.15.16:0/251369889 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x563d8a80f000 0x563d8a924000 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.666+0000 7f7782aee7c0 1 -- 172.21.15.16:0/251369889 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x563d8a80ec00 msgr2=0x563d8a53f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.666+0000 7f7782aee7c0 1 --2- 172.21.15.16:0/251369889 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x563d8a80ec00 0x563d8a53f700 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x563d8a7f6030 tx=0x563d8b2363c0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.666+0000 7f7782aee7c0 1 -- 172.21.15.16:0/251369889 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563d8a4a3800 msgr2=0x563d8a53f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.666+0000 7f7782aee7c0 1 --2- 172.21.15.16:0/251369889 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563d8a4a3800 0x563d8a53f180 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x563d8a7f6f60 tx=0x563d8a7a0c00 comp rx=0 tx=0).stop 2023-05-20T01:22:18.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.666+0000 7f7782aee7c0 1 -- 172.21.15.16:0/251369889 shutdown_connections 2023-05-20T01:22:18.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.666+0000 7f7782aee7c0 1 --2- 172.21.15.16:0/251369889 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x563d8a80f000 0x563d8a924000 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.666+0000 7f7782aee7c0 1 --2- 172.21.15.16:0/251369889 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563d8a4a3800 0x563d8a53f180 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.666+0000 7f7782aee7c0 1 --2- 172.21.15.16:0/251369889 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x563d8a80ec00 0x563d8a53f700 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.666+0000 7f7782aee7c0 1 -- 172.21.15.16:0/251369889 >> 172.21.15.16:0/251369889 conn(0x563d8a4a3c00 msgr2=0x563d8a563800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:18.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.666+0000 7f7782aee7c0 1 -- 172.21.15.16:0/251369889 shutdown_connections 2023-05-20T01:22:18.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.666+0000 7f7782aee7c0 1 -- 172.21.15.16:0/251369889 wait complete. 2023-05-20T01:22:18.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.667+0000 7f7782aee7c0 1 -- 172.21.15.16:0/691035864 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d8a4a2800 msgr2=0x563d8a53c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.667+0000 7f7782aee7c0 1 --2- 172.21.15.16:0/691035864 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d8a4a2800 0x563d8a53c580 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x563d8b21ce10 tx=0x563d8b236480 comp rx=0 tx=0).stop 2023-05-20T01:22:18.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.667+0000 7f7782aee7c0 1 -- 172.21.15.16:0/691035864 shutdown_connections 2023-05-20T01:22:18.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.667+0000 7f7782aee7c0 1 --2- 172.21.15.16:0/691035864 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563d8a4a3000 0x563d8a53d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.667+0000 7f7782aee7c0 1 --2- 172.21.15.16:0/691035864 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d8a4a2800 0x563d8a53c580 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.667+0000 7f7782aee7c0 1 --2- 172.21.15.16:0/691035864 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563d8a4a2c00 0x563d8a53cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.667+0000 7f7782aee7c0 1 -- 172.21.15.16:0/691035864 >> 172.21.15.16:0/691035864 conn(0x563d8a4a2400 msgr2=0x563d8a563000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:18.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.667+0000 7f7782aee7c0 1 -- 172.21.15.16:0/691035864 shutdown_connections 2023-05-20T01:22:18.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.667+0000 7f7782aee7c0 1 -- 172.21.15.16:0/691035864 wait complete. 2023-05-20T01:22:18.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.668+0000 7fabbbd98700 1 -- 172.21.15.16:0/2723465486 <== osd.4 v2:172.21.15.132:6817/506317223 17 ==== osd_op_reply(39 10000000000.00000000 [omap-set-vals] v71'35 uv35 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5614bdd378c0 con 0x5614bde80400 2023-05-20T01:22:18.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.668+0000 7fabbddea7c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/13 pointing to 0x0x10000000202 2023-05-20T01:22:18.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.668+0000 7fabbddea7c0 1 -- 172.21.15.16:0/2723465486 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:40 10.28 10:177a762d::::head [pgnls-filter start_epoch 71 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x5614bdea7c00 con 0x5614bdeb0000 2023-05-20T01:22:18.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.668+0000 7f90ce7a5700 1 -- 172.21.15.16:0/3200652326 <== osd.4 v2:172.21.15.132:6817/506317223 12 ==== osd_op_reply(25 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv35 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x556b170eb680 con 0x556b170ffc00 2023-05-20T01:22:18.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.668+0000 7f90cfff67c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=23 frag=*, r=0 2023-05-20T01:22:18.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.668+0000 7f90cfff67c0 20 datascan.read_dentry: 23_head not found in result 2023-05-20T01:22:18.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.668+0000 7f90cfff67c0 10 datascan.inject_with_backtrace: Linking inode 0x0x1000000020c at 0x0x10000000000/23 with size=6291456 bytes 2023-05-20T01:22:18.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.668+0000 7f90cfff67c0 1 -- 172.21.15.16:0/3200652326 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:26 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x556b1723ca80 con 0x556b170ffc00 2023-05-20T01:22:18.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.668+0000 7fabbb597700 1 -- 172.21.15.16:0/2723465486 <== osd.7 v2:172.21.15.132:6800/466001149 3 ==== osd_op_reply(40 [pgnls-filter start_epoch 71 out=81b] v71'14 uv14 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5614bdd37440 con 0x5614bdeb0000 2023-05-20T01:22:18.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.668+0000 7f3b5969c700 1 -- 172.21.15.16:0/3957359014 <== osd.4 v2:172.21.15.132:6817/506317223 16 ==== osd_op_reply(31 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv35 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x56013b6578c0 con 0x56013b665800 2023-05-20T01:22:18.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.668+0000 7fabbddea7c0 20 datascan.forall_objects: OSD matched oid 10000000201.00000000 2023-05-20T01:22:18.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.668+0000 7fabbddea7c0 10 datascan.operator(): handling object 10000000201.0 2023-05-20T01:22:18.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.668+0000 7f3b5beef7c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=19 frag=*, r=0 2023-05-20T01:22:18.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.668+0000 7f3b5beef7c0 20 datascan.read_dentry: 19_head not found in result 2023-05-20T01:22:18.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.668+0000 7f3b5beef7c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000208 at 0x0x10000000000/19 with size=6291456 bytes 2023-05-20T01:22:18.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.668+0000 7f3b5beef7c0 1 -- 172.21.15.16:0/3957359014 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:32 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x56013b7a7880 con 0x56013b665800 2023-05-20T01:22:18.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.668+0000 7fabbddea7c0 1 -- 172.21.15.16:0/2723465486 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:41 10.28 10:177a762d:::10000000201.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x5614bdea6700 con 0x5614bdeb0000 2023-05-20T01:22:18.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.668+0000 7fa3b3357700 1 -- 172.21.15.16:0/2859455639 <== osd.4 v2:172.21.15.132:6817/506317223 10 ==== osd_op_reply(25 10000000000.00000000 [omap-get-header out=274b] v0'0 uv35 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x555b4a80d440 con 0x555b4a931000 2023-05-20T01:22:18.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.668+0000 7fa3b53a97c0 10 datascan.find_or_create_dirfrag: read_version = 35 2023-05-20T01:22:18.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.668+0000 7fa3b53a97c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-05-20T01:22:18.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.668+0000 7fa3b53a97c0 1 -- 172.21.15.16:0/2859455639 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:26 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x555b4a97d180 con 0x555b4a931000 2023-05-20T01:22:18.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.669+0000 7fabbb597700 1 -- 172.21.15.16:0/2723465486 <== osd.7 v2:172.21.15.132:6800/466001149 4 ==== osd_op_reply(41 10000000201.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (98) out=98b,getxattr,getxattr] v0'0 uv14 ondisk = 0) v8 ==== 416+0+144 (crc 0 0 0) 0x5614bdd37440 con 0x5614bdeb0000 2023-05-20T01:22:18.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.669+0000 7fabbddea7c0 1 -- 172.21.15.16:0/2723465486 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:42 11.8 11:177a762d:::10000000201.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x5614bdea7c00 con 0x5614bde81000 2023-05-20T01:22:18.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.669+0000 7fabbc599700 1 -- 172.21.15.16:0/2723465486 <== osd.3 v2:172.21.15.16:6800/3669781890 4 ==== osd_op_reply(42 10000000201.00000000 [getxattr (30) out=30b] v0'0 uv5 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x5614bdd37680 con 0x5614bde81000 2023-05-20T01:22:18.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.669+0000 7fabbddea7c0 20 datascan.operator(): loaded layout from xattr: pi: 11 os: 4194304 sc: 1 su: 4194304 2023-05-20T01:22:18.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.669+0000 7fabbddea7c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000201 2023-05-20T01:22:18.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.669+0000 7fabbddea7c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/12 2023-05-20T01:22:18.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.669+0000 7fabbddea7c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=12 2023-05-20T01:22:18.979 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_inodes', '--worker_n', '2', '--worker_m', '7'] in time 0:00:00.453892, result: 2023-05-20T01:22:18.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.669+0000 7fabbddea7c0 1 -- 172.21.15.16:0/2723465486 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:43 9.10 9:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x5614bdea6000 con 0x5614bde80400 2023-05-20T01:22:18.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.670+0000 7f90ce7a5700 1 -- 172.21.15.16:0/3200652326 <== osd.4 v2:172.21.15.132:6817/506317223 13 ==== osd_op_reply(26 10000000000.00000000 [omap-set-vals] v71'36 uv36 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x556b170eb680 con 0x556b170ffc00 2023-05-20T01:22:18.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.670+0000 7f90cfff67c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/23 pointing to 0x0x1000000020c 2023-05-20T01:22:18.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.670+0000 7f90cfff67c0 1 -- 172.21.15.16:0/3200652326 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:27 10.3e 10:7c000000::::head [pgnls-filter start_epoch 71 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x556b1723ce00 con 0x556b170fe800 2023-05-20T01:22:18.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.671+0000 7f90cdfa4700 1 -- 172.21.15.16:0/3200652326 <== osd.1 v2:172.21.15.16:6808/3860264220 3 ==== osd_op_reply(27 [pgnls-filter start_epoch 71 out=49b] v69'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556b170eb440 con 0x556b170fe800 2023-05-20T01:22:18.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.671+0000 7f90c8f9a700 1 -- 172.21.15.16:0/3200652326 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:28 10.1 10:80000000::::head [pgnls-filter start_epoch 71 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x556b1723c000 con 0x556b170ffc00 2023-05-20T01:22:18.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.671+0000 7f90ce7a5700 1 -- 172.21.15.16:0/3200652326 <== osd.4 v2:172.21.15.132:6817/506317223 14 ==== osd_op_reply(28 [pgnls-filter start_epoch 71 out=49b] v69'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556b170eb680 con 0x556b170ffc00 2023-05-20T01:22:18.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.671+0000 7f90c979b700 1 -- 172.21.15.16:0/3200652326 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:29 10.21 10:84000000::::head [pgnls-filter start_epoch 71 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x556b1723c380 con 0x556b170ffc00 2023-05-20T01:22:18.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.671+0000 7f90ce7a5700 1 -- 172.21.15.16:0/3200652326 <== osd.4 v2:172.21.15.132:6817/506317223 15 ==== osd_op_reply(29 [pgnls-filter start_epoch 71 out=49b] v69'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556b170eb680 con 0x556b170ffc00 2023-05-20T01:22:18.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.671+0000 7f90c8f9a700 1 -- 172.21.15.16:0/3200652326 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:30 10.11 10:88000000::::head [pgnls-filter start_epoch 71 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x556b17c5c700 con 0x556b170ff800 2023-05-20T01:22:18.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.672+0000 7f90cd7a3700 1 -- 172.21.15.16:0/3200652326 <== osd.5 v2:172.21.15.132:6816/1844689995 3 ==== osd_op_reply(30 [pgnls-filter start_epoch 71 out=49b] v69'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556b170eb8c0 con 0x556b170ff800 2023-05-20T01:22:18.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.672+0000 7f90c979b700 1 -- 172.21.15.16:0/3200652326 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:31 10.31 10:8c000000::::head [pgnls-filter start_epoch 71 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x556b1723c700 con 0x556b170ff000 2023-05-20T01:22:18.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.672+0000 7f90cdfa4700 1 -- 172.21.15.16:0/3200652326 <== osd.7 v2:172.21.15.132:6800/466001149 6 ==== osd_op_reply(31 [pgnls-filter start_epoch 71 out=49b] v69'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556b170eb440 con 0x556b170ff000 2023-05-20T01:22:18.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.672+0000 7f90c8f9a700 1 -- 172.21.15.16:0/3200652326 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:32 10.9 10:90000000::::head [pgnls-filter start_epoch 71 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x556b17c5ca80 con 0x556b170ff800 2023-05-20T01:22:18.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.673+0000 7f90cd7a3700 1 -- 172.21.15.16:0/3200652326 <== osd.5 v2:172.21.15.132:6816/1844689995 4 ==== osd_op_reply(32 [pgnls-filter start_epoch 71 out=49b] v69'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556b170eb8c0 con 0x556b170ff800 2023-05-20T01:22:18.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.673+0000 7f90cfff67c0 1 -- 172.21.15.16:0/3200652326 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x556b16e9c400 msgr2=0x556b1721ec00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.673+0000 7f90cfff67c0 1 --2- 172.21.15.16:0/3200652326 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x556b16e9c400 0x556b1721ec00 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.673+0000 7f90cfff67c0 1 -- 172.21.15.16:0/3200652326 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x556b170fe800 msgr2=0x556b16e56100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.673+0000 7f90cfff67c0 1 --2- 172.21.15.16:0/3200652326 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x556b170fe800 0x556b16e56100 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.673+0000 7f90cfff67c0 1 -- 172.21.15.16:0/3200652326 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x556b16e9c000 msgr2=0x556b1721e680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.673+0000 7f90cfff67c0 1 --2- 172.21.15.16:0/3200652326 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x556b16e9c000 0x556b1721e680 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.673+0000 7f90cfff67c0 1 -- 172.21.15.16:0/3200652326 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x556b170ffc00 msgr2=0x556b1721c580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.673+0000 7f90cfff67c0 1 --2- 172.21.15.16:0/3200652326 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x556b170ffc00 0x556b1721c580 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.673+0000 7f90cd7a3700 1 -- 172.21.15.16:0/3200652326 reap_dead start 2023-05-20T01:22:18.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.673+0000 7f90cfff67c0 1 -- 172.21.15.16:0/3200652326 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x556b170ff800 msgr2=0x556b1721c000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.673+0000 7f90cfff67c0 1 --2- 172.21.15.16:0/3200652326 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x556b170ff800 0x556b1721c000 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.673+0000 7f90cd7a3700 1 -- 172.21.15.16:0/3200652326 reap_dead start 2023-05-20T01:22:18.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.673+0000 7f90cfff67c0 1 -- 172.21.15.16:0/3200652326 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x556b170fec00 msgr2=0x556b16e55b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.673+0000 7f90cfff67c0 1 --2- 172.21.15.16:0/3200652326 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x556b170fec00 0x556b16e55b80 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.673+0000 7f90cfff67c0 1 -- 172.21.15.16:0/3200652326 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x556b170ff000 msgr2=0x556b16e55600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.673+0000 7f90cfff67c0 1 --2- 172.21.15.16:0/3200652326 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x556b170ff000 0x556b16e55600 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.673+0000 7f90cfff67c0 1 -- 172.21.15.16:0/3200652326 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x556b17108800 msgr2=0x556b16e57700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.673+0000 7f90cfff67c0 1 --2- 172.21.15.16:0/3200652326 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x556b17108800 0x556b16e57700 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x556b170f20c0 tx=0x556b17b9a0c0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.673+0000 7f90cfff67c0 1 -- 172.21.15.16:0/3200652326 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556b170fe400 msgr2=0x556b16e56680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.673+0000 7f3b5969c700 1 -- 172.21.15.16:0/3957359014 <== osd.4 v2:172.21.15.132:6817/506317223 17 ==== osd_op_reply(32 10000000000.00000000 [omap-set-vals] v71'37 uv37 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56013b6578c0 con 0x56013b665800 2023-05-20T01:22:18.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.673+0000 7f3b5beef7c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/19 pointing to 0x0x10000000208 2023-05-20T01:22:18.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.673+0000 7f90cfff67c0 1 --2- 172.21.15.16:0/3200652326 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556b170fe400 0x556b16e56680 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x556b170f3380 tx=0x556b1709cc00 comp rx=0 tx=0).stop 2023-05-20T01:22:18.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.673+0000 7f90cd7a3700 1 -- 172.21.15.16:0/3200652326 reap_dead start 2023-05-20T01:22:18.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.673+0000 7f3b5beef7c0 1 -- 172.21.15.16:0/3957359014 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:33 10.17 10:e8000000::::head [pgnls-filter start_epoch 71 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x56013b7a7c00 con 0x56013b665800 2023-05-20T01:22:18.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.673+0000 7f90cfff67c0 1 -- 172.21.15.16:0/3200652326 shutdown_connections 2023-05-20T01:22:18.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.673+0000 7f90cfff67c0 1 -- 172.21.15.16:0/3200652326 >> 172.21.15.16:0/3200652326 conn(0x556b16dbbc00 msgr2=0x556b16e7b800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:18.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.673+0000 7f90cfff67c0 1 -- 172.21.15.16:0/3200652326 shutdown_connections 2023-05-20T01:22:18.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.673+0000 7f90cfff67c0 1 -- 172.21.15.16:0/3200652326 wait complete. 2023-05-20T01:22:18.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.673+0000 7fa3b3357700 1 -- 172.21.15.16:0/2859455639 <== osd.4 v2:172.21.15.132:6817/506317223 11 ==== osd_op_reply(26 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv37 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x555b4a80d440 con 0x555b4a931000 2023-05-20T01:22:18.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.673+0000 7fa3b53a97c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=15 frag=*, r=0 2023-05-20T01:22:18.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.673+0000 7fa3b53a97c0 20 datascan.read_dentry: 15_head not found in result 2023-05-20T01:22:18.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.673+0000 7fa3b53a97c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000204 at 0x0x10000000000/15 with size=6291456 bytes 2023-05-20T01:22:18.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.673+0000 7fa3b53a97c0 1 -- 172.21.15.16:0/2859455639 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:27 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x555b4a97d500 con 0x555b4a931000 2023-05-20T01:22:18.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.673+0000 7f3b5969c700 1 -- 172.21.15.16:0/3957359014 <== osd.4 v2:172.21.15.132:6817/506317223 18 ==== osd_op_reply(33 [pgnls-filter start_epoch 71 out=49b] v69'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56013b6578c0 con 0x56013b665800 2023-05-20T01:22:18.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.673+0000 7fabbbd98700 1 -- 172.21.15.16:0/2723465486 <== osd.4 v2:172.21.15.132:6817/506317223 18 ==== osd_op_reply(43 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x5614bdd378c0 con 0x5614bde80400 2023-05-20T01:22:18.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.673+0000 7fabbddea7c0 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-05-20T01:22:18.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.673+0000 7fabbddea7c0 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-05-20T01:22:18.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.673+0000 7fabbddea7c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/12, will insert to root 2023-05-20T01:22:18.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.673+0000 7fabbddea7c0 1 -- 172.21.15.16:0/2723465486 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:44 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x5614be7fa700 con 0x5614bde80400 2023-05-20T01:22:18.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.673+0000 7f3b54e93700 1 --2- 172.21.15.16:0/3957359014 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x56013b7b0000 0x56013b78e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.673+0000 7f3b54e93700 1 -- 172.21.15.16:0/3957359014 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:34 10.37 10:ec000000::::head [pgnls-filter start_epoch 71 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x56013b5fd180 con 0x56013b7b0000 2023-05-20T01:22:18.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.673+0000 7f90cfff67c0 1 -- 172.21.15.16:0/2063914123 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556b16dba800 msgr2=0x556b16e54580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:18.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.673+0000 7f90cfff67c0 1 --2- 172.21.15.16:0/2063914123 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556b16dba800 0x556b16e54580 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x556b17b96ed0 tx=0x556b17b9a480 comp rx=0 tx=0).stop 2023-05-20T01:22:18.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.674+0000 7f90cfff67c0 1 -- 172.21.15.16:0/2063914123 shutdown_connections 2023-05-20T01:22:18.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.674+0000 7f3b59e9d700 1 --2- 172.21.15.16:0/3957359014 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x56013b7b0000 0x56013b78e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:18.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.674+0000 7f90cfff67c0 1 --2- 172.21.15.16:0/2063914123 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556b16dbb000 0x556b16e55080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.674+0000 7f90cfff67c0 1 --2- 172.21.15.16:0/2063914123 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556b16dba800 0x556b16e54580 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.674+0000 7f90cfff67c0 1 --2- 172.21.15.16:0/2063914123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556b16dbac00 0x556b16e54b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:18.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.674+0000 7f90cfff67c0 1 -- 172.21.15.16:0/2063914123 >> 172.21.15.16:0/2063914123 conn(0x556b16dba400 msgr2=0x556b16e7b000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:18.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.674+0000 7f90cfff67c0 1 -- 172.21.15.16:0/2063914123 shutdown_connections 2023-05-20T01:22:18.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.674+0000 7f90cfff67c0 1 -- 172.21.15.16:0/2063914123 wait complete. 2023-05-20T01:22:18.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.674+0000 7f3b59e9d700 1 --2- 172.21.15.16:0/3957359014 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x56013b7b0000 0x56013b78e680 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:18.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.674+0000 7f3b59e9d700 1 -- 172.21.15.16:0/3957359014 <== osd.7 v2:172.21.15.132:6800/466001149 1 ==== osd_op_reply(34 [pgnls-filter start_epoch 71 out=49b] v69'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56013b657440 con 0x56013b7b0000 2023-05-20T01:22:18.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.674+0000 7f3b55694700 1 -- 172.21.15.16:0/3957359014 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:35 10.f 10:f0000000::::head [pgnls-filter start_epoch 71 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x56013b5fca80 con 0x56013b664c00 2023-05-20T01:22:18.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.675+0000 7f3b5a69e700 1 -- 172.21.15.16:0/3957359014 <== osd.6 v2:172.21.15.132:6801/727581123 8 ==== osd_op_reply(35 [pgnls-filter start_epoch 71 out=49b] v69'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56013b657680 con 0x56013b664c00 2023-05-20T01:22:18.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.675+0000 7f3b54e93700 1 -- 172.21.15.16:0/3957359014 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:36 10.2f 10:f4000000::::head [pgnls-filter start_epoch 71 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x56013b5fce00 con 0x56013b664800 2023-05-20T01:22:18.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.675+0000 7f3b59e9d700 1 -- 172.21.15.16:0/3957359014 <== osd.1 v2:172.21.15.16:6808/3860264220 3 ==== osd_op_reply(36 [pgnls-filter start_epoch 71 out=49b] v69'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56013b657440 con 0x56013b664800 2023-05-20T01:22:18.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.675+0000 7f3b55694700 1 -- 172.21.15.16:0/3957359014 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:37 10.1f 10:f8000000::::head [pgnls-filter start_epoch 71 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x56013b5fdc00 con 0x56013b7b0000 2023-05-20T01:22:18.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.675+0000 7f3b59e9d700 1 -- 172.21.15.16:0/3957359014 <== osd.7 v2:172.21.15.132:6800/466001149 2 ==== osd_op_reply(37 [pgnls-filter start_epoch 71 out=49b] v69'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56013b657440 con 0x56013b7b0000 2023-05-20T01:22:18.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.675+0000 7f3b54e93700 1 -- 172.21.15.16:0/3957359014 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:38 10.3f 10:fc000000::::head [pgnls-filter start_epoch 71 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x56013b5fd500 con 0x56013b664800 2023-05-20T01:22:18.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.675+0000 7f3b59e9d700 1 -- 172.21.15.16:0/3957359014 <== osd.1 v2:172.21.15.16:6808/3860264220 4 ==== osd_op_reply(38 [pgnls-filter start_epoch 71 out=81b] v71'12 uv12 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x56013b657440 con 0x56013b664800 2023-05-20T01:22:18.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.675+0000 7f3b5beef7c0 20 datascan.forall_objects: OSD matched oid 100000001fa.00000000 2023-05-20T01:22:18.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.675+0000 7f3b5beef7c0 10 datascan.operator(): handling object 100000001fa.0 2023-05-20T01:22:18.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.675+0000 7f3b5beef7c0 1 -- 172.21.15.16:0/3957359014 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:39 10.3f 10:feb1c803:::100000001fa.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x56013b7a6e00 con 0x56013b664800 2023-05-20T01:22:18.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.676+0000 7fa3b3357700 1 -- 172.21.15.16:0/2859455639 <== osd.4 v2:172.21.15.132:6817/506317223 12 ==== osd_op_reply(27 10000000000.00000000 [omap-set-vals] v71'38 uv38 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x555b4a80d440 con 0x555b4a931000 2023-05-20T01:22:18.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.676+0000 7fa3b53a97c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/15 pointing to 0x0x10000000204 2023-05-20T01:22:18.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.676+0000 7fa3b53a97c0 1 -- 172.21.15.16:0/2859455639 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:28 10.25 10:a4000000::::head [pgnls-filter start_epoch 71 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x555b4b38b880 con 0x555b4a931000 2023-05-20T01:22:18.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.676+0000 7fabbbd98700 1 -- 172.21.15.16:0/2723465486 <== osd.4 v2:172.21.15.132:6817/506317223 19 ==== osd_op_reply(44 10000000000.00000000 [omap-get-header out=274b] v0'0 uv38 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5614bdd378c0 con 0x5614bde80400 2023-05-20T01:22:18.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.676+0000 7fabbddea7c0 10 datascan.find_or_create_dirfrag: read_version = 38 2023-05-20T01:22:18.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.676+0000 7fabbddea7c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-05-20T01:22:18.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.676+0000 7fabbddea7c0 1 -- 172.21.15.16:0/2723465486 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:45 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x5614be7faa80 con 0x5614bde80400 2023-05-20T01:22:18.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.676+0000 7f3b59e9d700 1 -- 172.21.15.16:0/3957359014 <== osd.1 v2:172.21.15.16:6808/3860264220 5 ==== osd_op_reply(39 100000001fa.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (97) out=97b,getxattr,getxattr] v0'0 uv12 ondisk = 0) v8 ==== 416+0+143 (crc 0 0 0) 0x56013b657440 con 0x56013b664800 2023-05-20T01:22:18.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.676+0000 7f3b5beef7c0 1 --2- 172.21.15.16:0/3957359014 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x56013b66d800 0x56013b78cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:18.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.676+0000 7f3b5beef7c0 1 -- 172.21.15.16:0/3957359014 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:40 11.1f 11:feb1c803:::100000001fa.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x56013b7a6700 con 0x56013b66d800 2023-05-20T01:22:18.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.676+0000 7fa3b3357700 1 -- 172.21.15.16:0/2859455639 <== osd.4 v2:172.21.15.132:6817/506317223 13 ==== osd_op_reply(28 [pgnls-filter start_epoch 71 out=49b] v69'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555b4a80d440 con 0x555b4a931000 2023-05-20T01:22:18.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.676+0000 7f3b5969c700 1 --2- 172.21.15.16:0/3957359014 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x56013b66d800 0x56013b78cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:19.000 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_inodes', '--worker_n', '3', '--worker_m', '7'] in time 0:00:00.474213, result: 2023-05-20T01:22:19.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.676+0000 7fa3ae34d700 1 -- 172.21.15.16:0/2859455639 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:29 10.15 10:a8000000::::head [pgnls-filter start_epoch 71 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x555b4a97c380 con 0x555b4a930400 2023-05-20T01:22:19.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.676+0000 7fabbbd98700 1 -- 172.21.15.16:0/2723465486 <== osd.4 v2:172.21.15.132:6817/506317223 20 ==== osd_op_reply(45 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv38 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x5614bdd378c0 con 0x5614bde80400 2023-05-20T01:22:19.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.676+0000 7fabbddea7c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=12 frag=*, r=0 2023-05-20T01:22:19.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.676+0000 7fabbddea7c0 20 datascan.read_dentry: 12_head not found in result 2023-05-20T01:22:19.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.676+0000 7fabbddea7c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000201 at 0x0x10000000000/12 with size=6291456 bytes 2023-05-20T01:22:19.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.676+0000 7fabbddea7c0 1 -- 172.21.15.16:0/2723465486 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:46 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x5614be7fbc00 con 0x5614bde80400 2023-05-20T01:22:19.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.676+0000 7f3b5969c700 1 --2- 172.21.15.16:0/3957359014 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x56013b66d800 0x56013b78cb00 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:19.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.676+0000 7fa3b2b56700 1 -- 172.21.15.16:0/2859455639 <== osd.3 v2:172.21.15.16:6800/3669781890 2 ==== osd_op_reply(29 [pgnls-filter start_epoch 71 out=81b] v71'10 uv10 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x555b4a80d8c0 con 0x555b4a930400 2023-05-20T01:22:19.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.676+0000 7fa3b53a97c0 20 datascan.forall_objects: OSD matched oid 1000000020d.00000000 2023-05-20T01:22:19.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.676+0000 7fa3b53a97c0 10 datascan.operator(): handling object 1000000020d.0 2023-05-20T01:22:19.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.676+0000 7fa3b53a97c0 1 -- 172.21.15.16:0/2859455639 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:30 10.15 10:a8e69ea8:::1000000020d.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x555b4b38aa80 con 0x555b4a930400 2023-05-20T01:22:19.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.677+0000 7f3b5969c700 1 -- 172.21.15.16:0/3957359014 <== osd.2 v2:172.21.15.16:6816/293187030 1 ==== osd_op_reply(40 100000001fa.00000000 [getxattr (30) out=30b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x56013b6578c0 con 0x56013b66d800 2023-05-20T01:22:19.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.677+0000 7f3b5beef7c0 20 datascan.operator(): loaded layout from xattr: pi: 11 os: 4194304 sc: 1 su: 4194304 2023-05-20T01:22:19.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.677+0000 7f3b5beef7c0 10 datascan.inject_with_backtrace: inode: 0x0x100000001fa 2023-05-20T01:22:19.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.677+0000 7f3b5beef7c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/5 2023-05-20T01:22:19.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.677+0000 7f3b5beef7c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=5 2023-05-20T01:22:19.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.677+0000 7f3b5beef7c0 1 -- 172.21.15.16:0/3957359014 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:41 9.10 9:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x56013b5fd180 con 0x56013b665800 2023-05-20T01:22:19.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.677+0000 7fa3b2b56700 1 -- 172.21.15.16:0/2859455639 <== osd.3 v2:172.21.15.16:6800/3669781890 3 ==== osd_op_reply(30 1000000020d.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (98) out=98b,getxattr,getxattr] v0'0 uv10 ondisk = 0) v8 ==== 416+0+144 (crc 0 0 0) 0x555b4a80d8c0 con 0x555b4a930400 2023-05-20T01:22:19.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.677+0000 7fa3b53a97c0 1 -- 172.21.15.16:0/2859455639 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:31 11.15 11:a8e69ea8:::1000000020d.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x555b4a97ce00 con 0x555b4a931800 2023-05-20T01:22:19.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.677+0000 7fa3b2b56700 1 -- 172.21.15.16:0/2859455639 <== osd.2 v2:172.21.15.16:6816/293187030 4 ==== osd_op_reply(31 1000000020d.00000000 [getxattr (30) out=30b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x555b4a80d8c0 con 0x555b4a931800 2023-05-20T01:22:19.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.677+0000 7fa3b53a97c0 20 datascan.operator(): loaded layout from xattr: pi: 11 os: 4194304 sc: 1 su: 4194304 2023-05-20T01:22:19.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.677+0000 7fa3b53a97c0 10 datascan.inject_with_backtrace: inode: 0x0x1000000020d 2023-05-20T01:22:19.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.677+0000 7fa3b53a97c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/24 2023-05-20T01:22:19.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.677+0000 7fa3b53a97c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=24 2023-05-20T01:22:19.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.677+0000 7fa3b53a97c0 1 -- 172.21.15.16:0/2859455639 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:32 9.10 9:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x555b4a97d180 con 0x555b4a931000 2023-05-20T01:22:19.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.677+0000 7fabbbd98700 1 -- 172.21.15.16:0/2723465486 <== osd.4 v2:172.21.15.132:6817/506317223 21 ==== osd_op_reply(46 10000000000.00000000 [omap-set-vals] v71'39 uv39 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5614bdd378c0 con 0x5614bde80400 2023-05-20T01:22:19.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.677+0000 7fabbddea7c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/12 pointing to 0x0x10000000201 2023-05-20T01:22:19.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.677+0000 7fabbddea7c0 1 -- 172.21.15.16:0/2723465486 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:47 10.18 10:18000000::::head [pgnls-filter start_epoch 71 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x5614be7fb180 con 0x5614bde81800 2023-05-20T01:22:19.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.677+0000 7f3b5969c700 1 -- 172.21.15.16:0/3957359014 <== osd.4 v2:172.21.15.132:6817/506317223 19 ==== osd_op_reply(41 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x56013b6578c0 con 0x56013b665800 2023-05-20T01:22:19.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.677+0000 7f3b5beef7c0 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-05-20T01:22:19.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.677+0000 7f3b5beef7c0 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-05-20T01:22:19.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.677+0000 7f3b5beef7c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/5, will insert to root 2023-05-20T01:22:19.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.677+0000 7f3b5beef7c0 1 -- 172.21.15.16:0/3957359014 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:42 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x56013b5fce00 con 0x56013b665800 2023-05-20T01:22:19.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.677+0000 7fa3b3357700 1 -- 172.21.15.16:0/2859455639 <== osd.4 v2:172.21.15.132:6817/506317223 14 ==== osd_op_reply(32 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x555b4a80d440 con 0x555b4a931000 2023-05-20T01:22:19.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.677+0000 7fa3b53a97c0 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-05-20T01:22:19.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.677+0000 7fa3b53a97c0 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-05-20T01:22:19.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.677+0000 7fa3b53a97c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/24, will insert to root 2023-05-20T01:22:19.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.677+0000 7fa3b53a97c0 1 -- 172.21.15.16:0/2859455639 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:33 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x555b4a97d880 con 0x555b4a931000 2023-05-20T01:22:19.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.678+0000 7fabbbd98700 1 -- 172.21.15.16:0/2723465486 <== osd.2 v2:172.21.15.16:6816/293187030 2 ==== osd_op_reply(47 [pgnls-filter start_epoch 71 out=49b] v69'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5614bdd378c0 con 0x5614bde81800 2023-05-20T01:22:19.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.678+0000 7fabb758f700 1 --2- 172.21.15.16:0/2723465486 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x5614bde80c00 0x5614bde92c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:19.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.678+0000 7fabb758f700 1 -- 172.21.15.16:0/2723465486 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:48 10.38 10:1c000000::::head [pgnls-filter start_epoch 71 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x5614be7fa700 con 0x5614bde80c00 2023-05-20T01:22:19.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.678+0000 7f3b5969c700 1 -- 172.21.15.16:0/3957359014 <== osd.4 v2:172.21.15.132:6817/506317223 20 ==== osd_op_reply(42 10000000000.00000000 [omap-get-header out=274b] v0'0 uv39 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x56013b6578c0 con 0x56013b665800 2023-05-20T01:22:19.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.678+0000 7f3b5beef7c0 10 datascan.find_or_create_dirfrag: read_version = 39 2023-05-20T01:22:19.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.678+0000 7f3b5beef7c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-05-20T01:22:19.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.678+0000 7f3b5beef7c0 1 -- 172.21.15.16:0/3957359014 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:43 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x56013b5fdc00 con 0x56013b665800 2023-05-20T01:22:19.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.678+0000 7fa3b3357700 1 -- 172.21.15.16:0/2859455639 <== osd.4 v2:172.21.15.132:6817/506317223 15 ==== osd_op_reply(33 10000000000.00000000 [omap-get-header out=274b] v0'0 uv39 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x555b4a80d440 con 0x555b4a931000 2023-05-20T01:22:19.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.678+0000 7fa3b53a97c0 10 datascan.find_or_create_dirfrag: read_version = 39 2023-05-20T01:22:19.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.678+0000 7fa3b53a97c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-05-20T01:22:19.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.678+0000 7fa3b53a97c0 1 -- 172.21.15.16:0/2859455639 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:34 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x555b4a97dc00 con 0x555b4a931000 2023-05-20T01:22:19.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.678+0000 7fabbc599700 1 --2- 172.21.15.16:0/2723465486 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x5614bde80c00 0x5614bde92c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:19.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.678+0000 7f3b5969c700 1 -- 172.21.15.16:0/3957359014 <== osd.4 v2:172.21.15.132:6817/506317223 21 ==== osd_op_reply(43 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv39 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x56013b6578c0 con 0x56013b665800 2023-05-20T01:22:19.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.678+0000 7f3b5beef7c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=5 frag=*, r=0 2023-05-20T01:22:19.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.678+0000 7f3b5beef7c0 20 datascan.read_dentry: 5_head not found in result 2023-05-20T01:22:19.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.678+0000 7f3b5beef7c0 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001fa at 0x0x10000000000/5 with size=6291456 bytes 2023-05-20T01:22:19.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.678+0000 7f3b5beef7c0 1 -- 172.21.15.16:0/3957359014 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:44 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x56013b5fd500 con 0x56013b665800 2023-05-20T01:22:19.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.678+0000 7fa3b3357700 1 -- 172.21.15.16:0/2859455639 <== osd.4 v2:172.21.15.132:6817/506317223 16 ==== osd_op_reply(34 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv39 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x555b4a80d440 con 0x555b4a931000 2023-05-20T01:22:19.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.678+0000 7fa3b53a97c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=24 frag=*, r=0 2023-05-20T01:22:19.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.678+0000 7fa3b53a97c0 20 datascan.read_dentry: 24_head not found in result 2023-05-20T01:22:19.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.678+0000 7fabbc599700 1 --2- 172.21.15.16:0/2723465486 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x5614bde80c00 0x5614bde92c00 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:19.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.678+0000 7fa3b53a97c0 10 datascan.inject_with_backtrace: Linking inode 0x0x1000000020d at 0x0x10000000000/24 with size=6291456 bytes 2023-05-20T01:22:19.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.678+0000 7fa3b53a97c0 1 -- 172.21.15.16:0/2859455639 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:35 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x555b4b38bc00 con 0x555b4a931000 2023-05-20T01:22:19.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.678+0000 7fabbc599700 1 -- 172.21.15.16:0/2723465486 <== osd.5 v2:172.21.15.132:6816/1844689995 1 ==== osd_op_reply(48 [pgnls-filter start_epoch 71 out=49b] v69'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5614bdd37680 con 0x5614bde80c00 2023-05-20T01:22:19.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.679+0000 7fabb6d8e700 1 -- 172.21.15.16:0/2723465486 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:49 10.4 10:20000000::::head [pgnls-filter start_epoch 71 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x5614bdea6e00 con 0x5614bde80800 2023-05-20T01:22:19.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.679+0000 7fabbb597700 1 -- 172.21.15.16:0/2723465486 <== osd.0 v2:172.21.15.16:6824/3923634179 5 ==== osd_op_reply(49 [pgnls-filter start_epoch 71 out=49b] v69'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5614bdd37440 con 0x5614bde80800 2023-05-20T01:22:19.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.679+0000 7fabb758f700 1 -- 172.21.15.16:0/2723465486 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:50 10.24 10:24000000::::head [pgnls-filter start_epoch 71 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x5614be7fb500 con 0x5614bdd38c00 2023-05-20T01:22:19.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.679+0000 7fabbb597700 1 -- 172.21.15.16:0/2723465486 <== osd.1 v2:172.21.15.16:6808/3860264220 10 ==== osd_op_reply(50 [pgnls-filter start_epoch 71 out=49b] v69'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5614bdd37440 con 0x5614bdd38c00 2023-05-20T01:22:19.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.679+0000 7fabbddea7c0 1 -- 172.21.15.16:0/2723465486 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x5614bde80800 msgr2=0x5614bde90000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:19.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.679+0000 7fabbddea7c0 1 --2- 172.21.15.16:0/2723465486 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x5614bde80800 0x5614bde90000 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:19.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.679+0000 7fabbddea7c0 1 -- 172.21.15.16:0/2723465486 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x5614bdd38c00 msgr2=0x5614bdaae100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:19.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.679+0000 7fabbddea7c0 1 --2- 172.21.15.16:0/2723465486 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x5614bdd38c00 0x5614bdaae100 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:19.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.679+0000 7fabbddea7c0 1 -- 172.21.15.16:0/2723465486 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x5614bde81800 msgr2=0x5614bde90b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:19.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.679+0000 7fabbddea7c0 1 --2- 172.21.15.16:0/2723465486 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x5614bde81800 0x5614bde90b00 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:19.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.679+0000 7fabbddea7c0 1 -- 172.21.15.16:0/2723465486 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x5614bde81000 msgr2=0x5614bde90580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:19.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.679+0000 7fabbddea7c0 1 --2- 172.21.15.16:0/2723465486 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x5614bde81000 0x5614bde90580 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:19.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.679+0000 7fabbddea7c0 1 -- 172.21.15.16:0/2723465486 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x5614bde80400 msgr2=0x5614bdaad600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:19.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.679+0000 7fabbbd98700 1 -- 172.21.15.16:0/2723465486 reap_dead start 2023-05-20T01:22:19.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.679+0000 7fabbddea7c0 1 --2- 172.21.15.16:0/2723465486 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x5614bde80400 0x5614bdaad600 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:19.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.679+0000 7fabbddea7c0 1 -- 172.21.15.16:0/2723465486 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x5614bde80c00 msgr2=0x5614bde92c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:19.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.679+0000 7fabbddea7c0 1 --2- 172.21.15.16:0/2723465486 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x5614bde80c00 0x5614bde92c00 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:19.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.679+0000 7fabbbd98700 1 -- 172.21.15.16:0/2723465486 reap_dead start 2023-05-20T01:22:19.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.679+0000 7fabbddea7c0 1 -- 172.21.15.16:0/2723465486 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x5614bdd39000 msgr2=0x5614bdaadb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:19.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.679+0000 7fabbddea7c0 1 --2- 172.21.15.16:0/2723465486 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x5614bdd39000 0x5614bdaadb80 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:19.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.679+0000 7fabbddea7c0 1 -- 172.21.15.16:0/2723465486 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x5614bdeb0000 msgr2=0x5614bde92680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:19.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.679+0000 7fabbddea7c0 1 --2- 172.21.15.16:0/2723465486 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x5614bdeb0000 0x5614bde92680 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:19.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.679+0000 7f3b5969c700 1 -- 172.21.15.16:0/3957359014 <== osd.4 v2:172.21.15.132:6817/506317223 22 ==== osd_op_reply(44 10000000000.00000000 [omap-set-vals] v71'40 uv40 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56013b6578c0 con 0x56013b665800 2023-05-20T01:22:19.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.679+0000 7f3b5beef7c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/5 pointing to 0x0x100000001fa 2023-05-20T01:22:19.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.679+0000 7fabbddea7c0 1 -- 172.21.15.16:0/2723465486 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x5614bdd39c00 msgr2=0x5614bdaaf700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:19.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.679+0000 7fabbddea7c0 1 --2- 172.21.15.16:0/2723465486 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x5614bdd39c00 0x5614bdaaf700 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x5614bda6d620 tx=0x5614be7f8000 comp rx=0 tx=0).stop 2023-05-20T01:22:19.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.679+0000 7fabbddea7c0 1 -- 172.21.15.16:0/2723465486 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614bdd38000 msgr2=0x5614bdaaf180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:19.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.679+0000 7fabbddea7c0 1 --2- 172.21.15.16:0/2723465486 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614bdd38000 0x5614bdaaf180 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x5614bdd3ef90 tx=0x5614be7f81e0 comp rx=0 tx=0).stop 2023-05-20T01:22:19.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.679+0000 7f3b5beef7c0 1 -- 172.21.15.16:0/3957359014 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x56013b664800 msgr2=0x56013b3b8100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:19.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.679+0000 7f3b5beef7c0 1 --2- 172.21.15.16:0/3957359014 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x56013b664800 0x56013b3b8100 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:19.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.679+0000 7f3b5beef7c0 1 -- 172.21.15.16:0/3957359014 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x56013b66d800 msgr2=0x56013b78cb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:19.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.679+0000 7f3b5beef7c0 1 --2- 172.21.15.16:0/3957359014 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x56013b66d800 0x56013b78cb00 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:19.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.679+0000 7f3b5beef7c0 1 -- 172.21.15.16:0/3957359014 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x56013b665000 msgr2=0x56013b3b7600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:19.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.679+0000 7f3b5beef7c0 1 --2- 172.21.15.16:0/3957359014 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x56013b665000 0x56013b3b7600 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:19.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.679+0000 7f3b5beef7c0 1 -- 172.21.15.16:0/3957359014 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x56013b665800 msgr2=0x56013b78c000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:19.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.679+0000 7f3b5beef7c0 1 --2- 172.21.15.16:0/3957359014 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x56013b665800 0x56013b78c000 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:19.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.679+0000 7f3b5beef7c0 1 -- 172.21.15.16:0/3957359014 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x56013b66c800 msgr2=0x56013b78c580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:19.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.679+0000 7f3b5beef7c0 1 --2- 172.21.15.16:0/3957359014 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x56013b66c800 0x56013b78c580 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:19.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.679+0000 7f3b5beef7c0 1 -- 172.21.15.16:0/3957359014 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x56013b664c00 msgr2=0x56013b3b7b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:19.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.679+0000 7f3b5beef7c0 1 --2- 172.21.15.16:0/3957359014 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x56013b664c00 0x56013b3b7b80 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:19.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.679+0000 7f3b5beef7c0 1 -- 172.21.15.16:0/3957359014 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x56013b7b0000 msgr2=0x56013b78e680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:19.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.679+0000 7f3b5beef7c0 1 --2- 172.21.15.16:0/3957359014 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x56013b7b0000 0x56013b78e680 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:19.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.679+0000 7fabbddea7c0 1 -- 172.21.15.16:0/2723465486 shutdown_connections 2023-05-20T01:22:19.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.679+0000 7fabbddea7c0 1 --2- 172.21.15.16:0/2723465486 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x5614bdd39c00 0x5614bdaaf700 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:19.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.680+0000 7fabbddea7c0 1 --2- 172.21.15.16:0/2723465486 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614bdd38000 0x5614bdaaf180 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:19.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.680+0000 7fabbddea7c0 1 --2- 172.21.15.16:0/2723465486 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x5614bdeb0000 0x5614bde92680 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:19.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.680+0000 7fabbddea7c0 1 --2- 172.21.15.16:0/2723465486 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x5614bdd39000 0x5614bdaadb80 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:19.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.680+0000 7f3b5969c700 1 -- 172.21.15.16:0/3957359014 reap_dead start 2023-05-20T01:22:19.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.680+0000 7f3b5969c700 1 -- 172.21.15.16:0/3957359014 reap_dead start 2023-05-20T01:22:19.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.680+0000 7fabbddea7c0 1 -- 172.21.15.16:0/2723465486 >> 172.21.15.16:0/2723465486 conn(0x5614bda13c00 msgr2=0x5614bdad3800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:19.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.680+0000 7f3b5969c700 1 -- 172.21.15.16:0/3957359014 reap_dead start 2023-05-20T01:22:19.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.680+0000 7f3b5beef7c0 1 -- 172.21.15.16:0/3957359014 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x56013b66c400 msgr2=0x56013b3b9700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:19.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.680+0000 7f3b5beef7c0 1 --2- 172.21.15.16:0/3957359014 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x56013b66c400 0x56013b3b9700 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x56013b377590 tx=0x56013c0fe060 comp rx=0 tx=0).stop 2023-05-20T01:22:19.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.680+0000 7f3b5beef7c0 1 -- 172.21.15.16:0/3957359014 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56013b664400 msgr2=0x56013b3b8680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:19.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.680+0000 7f3b5beef7c0 1 --2- 172.21.15.16:0/3957359014 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56013b664400 0x56013b3b8680 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x56013c0e63c0 tx=0x56013c0fe300 comp rx=0 tx=0).stop 2023-05-20T01:22:19.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.680+0000 7f3b5beef7c0 1 -- 172.21.15.16:0/3957359014 shutdown_connections 2023-05-20T01:22:19.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.680+0000 7f3b5beef7c0 1 --2- 172.21.15.16:0/3957359014 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x56013b66c400 0x56013b3b9700 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:19.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.680+0000 7f3b5beef7c0 1 --2- 172.21.15.16:0/3957359014 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56013b664400 0x56013b3b8680 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:19.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.680+0000 7f3b5beef7c0 1 -- 172.21.15.16:0/3957359014 >> 172.21.15.16:0/3957359014 conn(0x56013b31dc00 msgr2=0x56013b3dd800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:19.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.680+0000 7fabbddea7c0 1 -- 172.21.15.16:0/2723465486 shutdown_connections 2023-05-20T01:22:19.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.680+0000 7fabbddea7c0 1 -- 172.21.15.16:0/2723465486 wait complete. 2023-05-20T01:22:19.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.680+0000 7f3b5beef7c0 1 -- 172.21.15.16:0/3957359014 shutdown_connections 2023-05-20T01:22:19.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.680+0000 7f3b5beef7c0 1 -- 172.21.15.16:0/3957359014 wait complete. 2023-05-20T01:22:19.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.680+0000 7fa3b3357700 1 -- 172.21.15.16:0/2859455639 <== osd.4 v2:172.21.15.132:6817/506317223 17 ==== osd_op_reply(35 10000000000.00000000 [omap-set-vals] v71'41 uv41 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x555b4a80d440 con 0x555b4a931000 2023-05-20T01:22:19.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.680+0000 7fa3b53a97c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/24 pointing to 0x0x1000000020d 2023-05-20T01:22:19.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.680+0000 7fa3b53a97c0 1 -- 172.21.15.16:0/2859455639 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:36 10.35 10:ac000000::::head [pgnls-filter start_epoch 71 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x555b4b38b180 con 0x555b4a931000 2023-05-20T01:22:19.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.680+0000 7fabbddea7c0 1 -- 172.21.15.16:0/782650732 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5614bda12c00 msgr2=0x5614bdaacb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:19.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.680+0000 7fabbddea7c0 1 --2- 172.21.15.16:0/782650732 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5614bda12c00 0x5614bdaacb00 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x5614be7eedb0 tx=0x5614be7f8480 comp rx=0 tx=0).stop 2023-05-20T01:22:19.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.680+0000 7f3b5beef7c0 1 -- 172.21.15.16:0/1833980693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56013b31d000 msgr2=0x56013b3b6580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:19.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.680+0000 7f3b5beef7c0 1 --2- 172.21.15.16:0/1833980693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56013b31d000 0x56013b3b6580 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x56013c0e8d20 tx=0x56013c0fe360 comp rx=0 tx=0).stop 2023-05-20T01:22:19.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.680+0000 7fabbddea7c0 1 -- 172.21.15.16:0/782650732 shutdown_connections 2023-05-20T01:22:19.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.680+0000 7fabbddea7c0 1 --2- 172.21.15.16:0/782650732 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5614bda13000 0x5614bdaad080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:19.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.680+0000 7fabbddea7c0 1 --2- 172.21.15.16:0/782650732 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614bda12800 0x5614bdaac580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:19.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.680+0000 7fabbddea7c0 1 --2- 172.21.15.16:0/782650732 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5614bda12c00 0x5614bdaacb00 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:19.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.680+0000 7f3b5beef7c0 1 -- 172.21.15.16:0/1833980693 shutdown_connections 2023-05-20T01:22:19.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.680+0000 7f3b5beef7c0 1 --2- 172.21.15.16:0/1833980693 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56013b31c800 0x56013b3b7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:19.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.680+0000 7fabbddea7c0 1 -- 172.21.15.16:0/782650732 >> 172.21.15.16:0/782650732 conn(0x5614bda12400 msgr2=0x5614bdad3000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:19.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.680+0000 7f3b5beef7c0 1 --2- 172.21.15.16:0/1833980693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56013b31d000 0x56013b3b6580 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:19.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.680+0000 7f3b5beef7c0 1 --2- 172.21.15.16:0/1833980693 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56013b31cc00 0x56013b3b6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:19.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.680+0000 7f3b5beef7c0 1 -- 172.21.15.16:0/1833980693 >> 172.21.15.16:0/1833980693 conn(0x56013b31d400 msgr2=0x56013b3dd000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:19.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.680+0000 7fabbddea7c0 1 -- 172.21.15.16:0/782650732 shutdown_connections 2023-05-20T01:22:19.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.680+0000 7f3b5beef7c0 1 -- 172.21.15.16:0/1833980693 shutdown_connections 2023-05-20T01:22:19.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.680+0000 7f3b5beef7c0 1 -- 172.21.15.16:0/1833980693 wait complete. 2023-05-20T01:22:19.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.680+0000 7fabbddea7c0 1 -- 172.21.15.16:0/782650732 wait complete. 2023-05-20T01:22:19.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.680+0000 7fa3b3357700 1 -- 172.21.15.16:0/2859455639 <== osd.4 v2:172.21.15.132:6817/506317223 18 ==== osd_op_reply(36 [pgnls-filter start_epoch 71 out=49b] v69'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555b4a80d440 con 0x555b4a931000 2023-05-20T01:22:19.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.680+0000 7fa3ae34d700 1 -- 172.21.15.16:0/2859455639 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:37 10.d 10:b0000000::::head [pgnls-filter start_epoch 71 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x555b4b38b880 con 0x555b4a4fdc00 2023-05-20T01:22:19.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.681+0000 7fa3b3b58700 1 -- 172.21.15.16:0/2859455639 <== osd.6 v2:172.21.15.132:6801/727581123 2 ==== osd_op_reply(37 [pgnls-filter start_epoch 71 out=49b] v69'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555b4a80d680 con 0x555b4a4fdc00 2023-05-20T01:22:19.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.681+0000 7fa3aeb4e700 1 -- 172.21.15.16:0/2859455639 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:38 10.2d 10:b4000000::::head [pgnls-filter start_epoch 71 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x555b4b38b500 con 0x555b4a4fdc00 2023-05-20T01:22:19.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.681+0000 7fa3b3b58700 1 -- 172.21.15.16:0/2859455639 <== osd.6 v2:172.21.15.132:6801/727581123 3 ==== osd_op_reply(38 [pgnls-filter start_epoch 71 out=49b] v69'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555b4a80d680 con 0x555b4a4fdc00 2023-05-20T01:22:19.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.681+0000 7fa3b53a97c0 1 -- 172.21.15.16:0/2859455639 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x555b4a930c00 msgr2=0x555b4a966580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:19.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.681+0000 7fa3b53a97c0 1 --2- 172.21.15.16:0/2859455639 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x555b4a930c00 0x555b4a966580 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:19.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.681+0000 7fa3b53a97c0 1 -- 172.21.15.16:0/2859455639 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x555b4a5dc400 msgr2=0x555b4a598100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:19.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.681+0000 7fa3b53a97c0 1 --2- 172.21.15.16:0/2859455639 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x555b4a5dc400 0x555b4a598100 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:19.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.681+0000 7fa3b53a97c0 1 -- 172.21.15.16:0/2859455639 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x555b4a931800 msgr2=0x555b4a967080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:19.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.681+0000 7fa3b53a97c0 1 --2- 172.21.15.16:0/2859455639 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x555b4a931800 0x555b4a967080 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:19.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.681+0000 7fa3b53a97c0 1 -- 172.21.15.16:0/2859455639 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x555b4a930400 msgr2=0x555b4a966000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:19.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.681+0000 7fa3b53a97c0 1 --2- 172.21.15.16:0/2859455639 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x555b4a930400 0x555b4a966000 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:19.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.681+0000 7fa3b53a97c0 1 -- 172.21.15.16:0/2859455639 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x555b4a931000 msgr2=0x555b4a966b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:19.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.681+0000 7fa3b53a97c0 1 --2- 172.21.15.16:0/2859455639 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x555b4a931000 0x555b4a966b00 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:19.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.681+0000 7fa3b53a97c0 1 -- 172.21.15.16:0/2859455639 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x555b4a4fd800 msgr2=0x555b4a597600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:19.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.681+0000 7fa3b53a97c0 1 --2- 172.21.15.16:0/2859455639 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x555b4a4fd800 0x555b4a597600 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:19.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.681+0000 7fa3b53a97c0 1 -- 172.21.15.16:0/2859455639 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x555b4a4fdc00 msgr2=0x555b4a597b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:19.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.681+0000 7fa3b3357700 1 -- 172.21.15.16:0/2859455639 reap_dead start 2023-05-20T01:22:19.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.681+0000 7fa3b53a97c0 1 --2- 172.21.15.16:0/2859455639 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x555b4a4fdc00 0x555b4a597b80 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:19.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.681+0000 7fa3b3357700 1 -- 172.21.15.16:0/2859455639 reap_dead start 2023-05-20T01:22:19.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.681+0000 7fa3b3357700 1 -- 172.21.15.16:0/2859455639 reap_dead start 2023-05-20T01:22:19.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.681+0000 7fa3b53a97c0 1 -- 172.21.15.16:0/2859455639 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x555b4a986000 msgr2=0x555b4a967600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:19.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.681+0000 7fa3b53a97c0 1 --2- 172.21.15.16:0/2859455639 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x555b4a986000 0x555b4a967600 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:19.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.681+0000 7fa3b53a97c0 1 -- 172.21.15.16:0/2859455639 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x555b4a930000 msgr2=0x555b4a599700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:19.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.681+0000 7fa3b53a97c0 1 --2- 172.21.15.16:0/2859455639 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x555b4a930000 0x555b4a599700 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x555b4a82a090 tx=0x555b4b2de000 comp rx=0 tx=0).stop 2023-05-20T01:22:19.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.682+0000 7fa3b53a97c0 1 -- 172.21.15.16:0/2859455639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b4a5dc800 msgr2=0x555b4a598680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:19.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.682+0000 7fa3b53a97c0 1 --2- 172.21.15.16:0/2859455639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b4a5dc800 0x555b4a598680 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x555b4b2d9350 tx=0x555b4a56d620 comp rx=0 tx=0).stop 2023-05-20T01:22:19.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.682+0000 7fa3b53a97c0 1 -- 172.21.15.16:0/2859455639 shutdown_connections 2023-05-20T01:22:19.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.682+0000 7fa3b53a97c0 1 --2- 172.21.15.16:0/2859455639 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x555b4a930000 0x555b4a599700 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:19.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.682+0000 7fa3b53a97c0 1 --2- 172.21.15.16:0/2859455639 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x555b4a986000 0x555b4a967600 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:19.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.682+0000 7fa3b53a97c0 1 --2- 172.21.15.16:0/2859455639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b4a5dc800 0x555b4a598680 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:19.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.682+0000 7fa3b53a97c0 1 --2- 172.21.15.16:0/2859455639 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x555b4a4fdc00 0x555b4a597b80 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:19.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.682+0000 7fa3b53a97c0 1 -- 172.21.15.16:0/2859455639 >> 172.21.15.16:0/2859455639 conn(0x555b4a5dd400 msgr2=0x555b4a5be000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:19.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.682+0000 7fa3b53a97c0 1 -- 172.21.15.16:0/2859455639 shutdown_connections 2023-05-20T01:22:19.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.682+0000 7fa3b53a97c0 1 -- 172.21.15.16:0/2859455639 wait complete. 2023-05-20T01:22:19.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.682+0000 7fa3b53a97c0 1 -- 172.21.15.16:0/128094146 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555b4a4fd000 msgr2=0x555b4a596b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:19.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.682+0000 7fa3b53a97c0 1 --2- 172.21.15.16:0/128094146 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555b4a4fd000 0x555b4a596b00 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x555b4b2d8cf0 tx=0x555b4b2de360 comp rx=0 tx=0).stop 2023-05-20T01:22:19.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.682+0000 7fa3b53a97c0 1 -- 172.21.15.16:0/128094146 shutdown_connections 2023-05-20T01:22:19.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.682+0000 7fa3b53a97c0 1 --2- 172.21.15.16:0/128094146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b4a4fd400 0x555b4a597080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:19.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.682+0000 7fa3b53a97c0 1 --2- 172.21.15.16:0/128094146 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x555b4a4fc800 0x555b4a596580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:19.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.682+0000 7fa3b53a97c0 1 --2- 172.21.15.16:0/128094146 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555b4a4fd000 0x555b4a596b00 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:19.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.682+0000 7fa3b53a97c0 1 -- 172.21.15.16:0/128094146 >> 172.21.15.16:0/128094146 conn(0x555b4a4fc400 msgr2=0x555b4a5bd800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:19.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.682+0000 7fa3b53a97c0 1 -- 172.21.15.16:0/128094146 shutdown_connections 2023-05-20T01:22:19.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:18.682+0000 7fa3b53a97c0 1 -- 172.21.15.16:0/128094146 wait complete. 2023-05-20T01:22:19.041 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_inodes', '--worker_n', '0', '--worker_m', '7'] in time 0:00:00.517582, result: 2023-05-20T01:22:19.042 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_inodes', '--worker_n', '6', '--worker_m', '7'] in time 0:00:00.514205, result: 2023-05-20T01:22:19.043 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_inodes', '--worker_n', '4', '--worker_m', '7'] in time 0:00:00.516229, result: 2023-05-20T01:22:19.043 DEBUG:teuthology.orchestra.run.smithi016:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_links 2023-05-20T01:22:19.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.084+0000 7f0789d227c0 1 -- start start 2023-05-20T01:22:19.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.084+0000 7f0789d227c0 1 Processor -- start 2023-05-20T01:22:19.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.085+0000 7f0789d227c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556acdc1f000 0x556acdcb8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:19.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.085+0000 7f0789d227c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556acdc1ec00 0x556acdcb8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:19.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.085+0000 7f0789d227c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556acdc1e800 0x556acdcb9080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:19.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.085+0000 7f0789d227c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x556acdc5bcc0 con 0x556acdc1f000 2023-05-20T01:22:19.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.085+0000 7f0789d227c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x556acdc5b400 con 0x556acdc1ec00 2023-05-20T01:22:19.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.085+0000 7f0789d227c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x556acdc5aa00 con 0x556acdc1e800 2023-05-20T01:22:19.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.085+0000 7f0787cd0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556acdc1f000 0x556acdcb8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:19.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.085+0000 7f0787cd0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556acdc1f000 0x556acdcb8580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35160/0 (socket says 172.21.15.16:35160) 2023-05-20T01:22:19.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.085+0000 7f0787cd0700 1 -- 172.21.15.16:0/3039714127 learned_addr learned my addr 172.21.15.16:0/3039714127 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:22:19.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.085+0000 7f07874cf700 1 --2- 172.21.15.16:0/3039714127 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556acdc1ec00 0x556acdcb8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:19.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.085+0000 7f07884d1700 1 --2- 172.21.15.16:0/3039714127 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556acdc1e800 0x556acdcb9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:19.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.085+0000 7f0787cd0700 1 -- 172.21.15.16:0/3039714127 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556acdc1e800 msgr2=0x556acdcb9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:19.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.085+0000 7f0787cd0700 1 --2- 172.21.15.16:0/3039714127 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556acdc1e800 0x556acdcb9080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:19.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.085+0000 7f0787cd0700 1 -- 172.21.15.16:0/3039714127 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556acdc1ec00 msgr2=0x556acdcb8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:19.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.085+0000 7f0787cd0700 1 --2- 172.21.15.16:0/3039714127 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556acdc1ec00 0x556acdcb8b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:19.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.085+0000 7f0787cd0700 1 -- 172.21.15.16:0/3039714127 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556acdc436c0 con 0x556acdc1f000 2023-05-20T01:22:19.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.085+0000 7f07884d1700 1 --2- 172.21.15.16:0/3039714127 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556acdc1e800 0x556acdcb9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:22:19.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.085+0000 7f07874cf700 1 --2- 172.21.15.16:0/3039714127 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556acdc1ec00 0x556acdcb8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:22:19.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.085+0000 7f0787cd0700 1 --2- 172.21.15.16:0/3039714127 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556acdc1f000 0x556acdcb8580 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x556acdd88db0 tx=0x556ace9f6ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ecdc04030dbfbfd server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:19.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.085+0000 7f07854cb700 1 -- 172.21.15.16:0/3039714127 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x556acdc53600 con 0x556acdc1f000 2023-05-20T01:22:19.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.085+0000 7f07854cb700 1 -- 172.21.15.16:0/3039714127 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556acdc71800 con 0x556acdc1f000 2023-05-20T01:22:19.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.085+0000 7f0789d227c0 1 -- 172.21.15.16:0/3039714127 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556acdc43380 con 0x556acdc1f000 2023-05-20T01:22:19.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.085+0000 7f0789d227c0 1 -- 172.21.15.16:0/3039714127 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x556acdc43d40 con 0x556acdc1f000 2023-05-20T01:22:19.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.085+0000 7f0789d227c0 4 waiting for MDS map... 2023-05-20T01:22:19.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.085+0000 7f07854cb700 1 -- 172.21.15.16:0/3039714127 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x556acdc531e0 con 0x556acdc1f000 2023-05-20T01:22:19.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.086+0000 7f07854cb700 1 -- 172.21.15.16:0/3039714127 <== mon.0 v2:172.21.15.16:3300/0 4 ==== osd_map(71..71 src has 1..71) v4 ==== 7103+0+0 (secure 0 0 0) 0x556acdc60c40 con 0x556acdc1f000 2023-05-20T01:22:19.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.086+0000 7f07854cb700 1 -- 172.21.15.16:0/3039714127 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 42) v1 ==== 3126+0+0 (secure 0 0 0) 0x556acea85200 con 0x556acdc1f000 2023-05-20T01:22:19.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.086+0000 7f0789d227c0 4 Got MDS map 42 2023-05-20T01:22:19.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.086+0000 7f0789d227c0 4 datascan.main: Using metadata pool output 2023-05-20T01:22:19.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.086+0000 7f0789d227c0 4 datascan.main: connecting to RADOS... 2023-05-20T01:22:19.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.086+0000 7f0789d227c0 1 Processor -- start 2023-05-20T01:22:19.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.086+0000 7f0789d227c0 1 -- start start 2023-05-20T01:22:19.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.087+0000 7f0789d227c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556acdc1fc00 0x556acdcb9600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:19.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.087+0000 7f0789d227c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556acdf2e000 0x556acdcb9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:19.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.087+0000 7f0789d227c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556acdf2e400 0x556acdcba100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:19.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.087+0000 7f0789d227c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x556acdc5ab40 con 0x556acdc1fc00 2023-05-20T01:22:19.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.087+0000 7f0789d227c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x556acdc5ac80 con 0x556acdf2e400 2023-05-20T01:22:19.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.087+0000 7f0789d227c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x556acdc5b400 con 0x556acdf2e000 2023-05-20T01:22:19.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.087+0000 7f07884d1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556acdc1fc00 0x556acdcb9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:19.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.087+0000 7f07884d1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556acdc1fc00 0x556acdcb9600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35166/0 (socket says 172.21.15.16:35166) 2023-05-20T01:22:19.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.087+0000 7f07884d1700 1 -- 172.21.15.16:0/478768593 learned_addr learned my addr 172.21.15.16:0/478768593 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:22:19.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.087+0000 7f07874cf700 1 --2- 172.21.15.16:0/478768593 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556acdf2e400 0x556acdcba100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:19.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.087+0000 7f0787cd0700 1 --2- 172.21.15.16:0/478768593 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556acdf2e000 0x556acdcb9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:19.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.087+0000 7f07884d1700 1 -- 172.21.15.16:0/478768593 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556acdf2e000 msgr2=0x556acdcb9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:19.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.087+0000 7f07884d1700 1 --2- 172.21.15.16:0/478768593 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556acdf2e000 0x556acdcb9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:19.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.087+0000 7f07884d1700 1 -- 172.21.15.16:0/478768593 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556acdf2e400 msgr2=0x556acdcba100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:19.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.087+0000 7f07884d1700 1 --2- 172.21.15.16:0/478768593 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556acdf2e400 0x556acdcba100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:19.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.087+0000 7f07884d1700 1 -- 172.21.15.16:0/478768593 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556acdc43d40 con 0x556acdc1fc00 2023-05-20T01:22:19.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.087+0000 7f07874cf700 1 --2- 172.21.15.16:0/478768593 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556acdf2e400 0x556acdcba100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:22:19.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.087+0000 7f07884d1700 1 --2- 172.21.15.16:0/478768593 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556acdc1fc00 0x556acdcb9600 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x556acdf3a0f0 tx=0x556aceaaac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=629fdddd50bfe64a server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:19.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.088+0000 7f07834c7700 1 -- 172.21.15.16:0/478768593 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x556acdc53600 con 0x556acdc1fc00 2023-05-20T01:22:19.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.088+0000 7f07834c7700 1 -- 172.21.15.16:0/478768593 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556acdc71c80 con 0x556acdc1fc00 2023-05-20T01:22:19.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.088+0000 7f07834c7700 1 -- 172.21.15.16:0/478768593 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x556acdc531e0 con 0x556acdc1fc00 2023-05-20T01:22:19.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.088+0000 7f0789d227c0 1 -- 172.21.15.16:0/478768593 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556acdc1fc00 msgr2=0x556acdcb9600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:19.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.088+0000 7f0789d227c0 1 --2- 172.21.15.16:0/478768593 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556acdc1fc00 0x556acdcb9600 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x556acdf3a0f0 tx=0x556aceaaac00 comp rx=0 tx=0).stop 2023-05-20T01:22:19.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.088+0000 7f0789d227c0 1 -- 172.21.15.16:0/478768593 shutdown_connections 2023-05-20T01:22:19.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.088+0000 7f0789d227c0 1 --2- 172.21.15.16:0/478768593 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556acdf2e400 0x556acdcba100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:19.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.088+0000 7f0789d227c0 1 --2- 172.21.15.16:0/478768593 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556acdc1fc00 0x556acdcb9600 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:19.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.088+0000 7f0789d227c0 1 --2- 172.21.15.16:0/478768593 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556acdf2e000 0x556acdcb9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:19.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.088+0000 7f0789d227c0 1 -- 172.21.15.16:0/478768593 >> 172.21.15.16:0/478768593 conn(0x556acdc1e400 msgr2=0x556acdcfb800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:19.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.088+0000 7f0789d227c0 1 -- 172.21.15.16:0/478768593 shutdown_connections 2023-05-20T01:22:19.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.088+0000 7f0789d227c0 1 -- 172.21.15.16:0/478768593 wait complete. 2023-05-20T01:22:19.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.089+0000 7f0789d227c0 1 Processor -- start 2023-05-20T01:22:19.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.089+0000 7f0789d227c0 1 -- start start 2023-05-20T01:22:19.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.089+0000 7f0789d227c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556acdc1e400 0x556acdcbb180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:19.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.089+0000 7f0789d227c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556acdf2e000 0x556acdcbac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:19.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.089+0000 7f0789d227c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556acdf2e400 0x556acdcba680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:19.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.089+0000 7f0789d227c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x556acdc5bcc0 con 0x556acdc1e400 2023-05-20T01:22:19.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.089+0000 7f0789d227c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x556acdc5a000 con 0x556acdf2e400 2023-05-20T01:22:19.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.089+0000 7f0789d227c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x556acdc5b180 con 0x556acdf2e000 2023-05-20T01:22:19.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.090+0000 7f07884d1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556acdc1e400 0x556acdcbb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:19.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.090+0000 7f07884d1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556acdc1e400 0x556acdcbb180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35172/0 (socket says 172.21.15.16:35172) 2023-05-20T01:22:19.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.090+0000 7f07884d1700 1 -- 172.21.15.16:0/3037775639 learned_addr learned my addr 172.21.15.16:0/3037775639 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:22:19.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.090+0000 7f07874cf700 1 --2- 172.21.15.16:0/3037775639 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556acdf2e400 0x556acdcba680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:19.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.090+0000 7f0787cd0700 1 --2- 172.21.15.16:0/3037775639 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556acdf2e000 0x556acdcbac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:19.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.090+0000 7f07884d1700 1 -- 172.21.15.16:0/3037775639 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556acdf2e000 msgr2=0x556acdcbac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:19.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.090+0000 7f07884d1700 1 --2- 172.21.15.16:0/3037775639 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556acdf2e000 0x556acdcbac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:19.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.090+0000 7f07884d1700 1 -- 172.21.15.16:0/3037775639 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556acdf2e400 msgr2=0x556acdcba680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:19.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.090+0000 7f07884d1700 1 --2- 172.21.15.16:0/3037775639 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556acdf2e400 0x556acdcba680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:19.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.090+0000 7f07884d1700 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556acdf4c000 con 0x556acdc1e400 2023-05-20T01:22:19.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.090+0000 7f07884d1700 1 --2- 172.21.15.16:0/3037775639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556acdc1e400 0x556acdcbb180 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x556acdf3b3b0 tx=0x556ace9f6de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:19.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.090+0000 7f0781cc4700 1 -- 172.21.15.16:0/3037775639 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x556acdc53080 con 0x556acdc1e400 2023-05-20T01:22:19.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.090+0000 7f0781cc4700 1 -- 172.21.15.16:0/3037775639 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556aceaa4780 con 0x556acdc1e400 2023-05-20T01:22:19.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.090+0000 7f0789d227c0 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x556acdc42d00 con 0x556acdc1e400 2023-05-20T01:22:19.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.090+0000 7f0781cc4700 1 -- 172.21.15.16:0/3037775639 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x556acdc53600 con 0x556acdc1e400 2023-05-20T01:22:19.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.090+0000 7f0789d227c0 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556acdf4c1a0 con 0x556acdc1e400 2023-05-20T01:22:19.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.090+0000 7f0789d227c0 4 datascan.init: resolving metadata pool 9 2023-05-20T01:22:19.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.091+0000 7f0781cc4700 1 -- 172.21.15.16:0/3037775639 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x556acdc80840 con 0x556acdc1e400 2023-05-20T01:22:19.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.092+0000 7f0781cc4700 1 --2- 172.21.15.16:0/3037775639 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x556acdf4e400 0x556acdcbb700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:19.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.092+0000 7f0781cc4700 1 -- 172.21.15.16:0/3037775639 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 7103+0+0 (secure 0 0 0) 0x556acdc60a80 con 0x556acdc1e400 2023-05-20T01:22:19.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.092+0000 7f0787cd0700 1 --2- 172.21.15.16:0/3037775639 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x556acdf4e400 0x556acdcbb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:19.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.092+0000 7f0789d227c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-05-20T01:22:19.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.092+0000 7f0789d227c0 4 datascan.main: resolving metadata pool 9 2023-05-20T01:22:19.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.092+0000 7f0789d227c0 1 --2- 172.21.15.16:0/3037775639 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x556acdf2e800 0x556acdcba100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:19.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.092+0000 7f0789d227c0 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:1 9.0 9:00000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x556acea88700 con 0x556acdf2e800 2023-05-20T01:22:19.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.092+0000 7f07874cf700 1 --2- 172.21.15.16:0/3037775639 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x556acdf2e800 0x556acdcba100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:19.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.092+0000 7f0787cd0700 1 --2- 172.21.15.16:0/3037775639 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x556acdf4e400 0x556acdcbb700 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x556acdd89770 tx=0x556ace9f6cc0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:19.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.092+0000 7f07874cf700 1 --2- 172.21.15.16:0/3037775639 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x556acdf2e800 0x556acdcba100 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:19.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.093+0000 7f07874cf700 1 -- 172.21.15.16:0/3037775639 <== osd.1 v2:172.21.15.16:6808/3860264220 1 ==== osd_op_reply(1 [pgnls start_epoch 71 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556acdf2d440 con 0x556acdf2e800 2023-05-20T01:22:19.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.093+0000 7f07874cf700 1 --2- 172.21.15.16:0/3037775639 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x556acdf4e800 0x556ace054000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:19.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.093+0000 7f07874cf700 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:2 9.10 9:08000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x556acea88700 con 0x556acdf4e800 2023-05-20T01:22:19.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.093+0000 7f07884d1700 1 --2- 172.21.15.16:0/3037775639 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x556acdf4e800 0x556ace054000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:19.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.094+0000 7f07884d1700 1 --2- 172.21.15.16:0/3037775639 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x556acdf4e800 0x556ace054000 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:19.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.094+0000 7f07884d1700 1 -- 172.21.15.16:0/3037775639 <== osd.4 v2:172.21.15.132:6817/506317223 1 ==== osd_op_reply(2 [pgnls start_epoch 71 out=81b] v71'41 uv41 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x556acdf2d680 con 0x556acdf4e800 2023-05-20T01:22:19.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.094+0000 7f07884d1700 1 --2- 172.21.15.16:0/3037775639 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x556acdf4ec00 0x556ace054580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:19.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.094+0000 7f07884d1700 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:3 9.8 9:10000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x556acea88700 con 0x556acdf4ec00 2023-05-20T01:22:19.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.094+0000 7f0787cd0700 1 --2- 172.21.15.16:0/3037775639 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x556acdf4ec00 0x556ace054580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:19.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.094+0000 7f0787cd0700 1 --2- 172.21.15.16:0/3037775639 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x556acdf4ec00 0x556ace054580 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:19.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.095+0000 7f0787cd0700 1 -- 172.21.15.16:0/3037775639 <== osd.3 v2:172.21.15.16:6800/3669781890 1 ==== osd_op_reply(3 [pgnls start_epoch 71 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556acdf2d8c0 con 0x556acdf4ec00 2023-05-20T01:22:19.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.095+0000 7f0787cd0700 1 --2- 172.21.15.16:0/3037775639 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x556acdf4f400 0x556ace054b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:19.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.095+0000 7f0787cd0700 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:4 9.18 9:18000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x556acea88700 con 0x556acdf4f400 2023-05-20T01:22:19.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.095+0000 7f07874cf700 1 --2- 172.21.15.16:0/3037775639 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x556acdf4f400 0x556ace054b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:19.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.096+0000 7f07874cf700 1 --2- 172.21.15.16:0/3037775639 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x556acdf4f400 0x556ace054b00 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:19.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.096+0000 7f07874cf700 1 -- 172.21.15.16:0/3037775639 <== osd.6 v2:172.21.15.132:6801/727581123 1 ==== osd_op_reply(4 [pgnls start_epoch 71 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556acdf2d440 con 0x556acdf4f400 2023-05-20T01:22:19.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.096+0000 7f07874cf700 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:5 9.4 9:20000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x556acea88700 con 0x556acdf2e800 2023-05-20T01:22:19.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.096+0000 7f07874cf700 1 -- 172.21.15.16:0/3037775639 <== osd.1 v2:172.21.15.16:6808/3860264220 2 ==== osd_op_reply(5 [pgnls start_epoch 71 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556acdf2d440 con 0x556acdf2e800 2023-05-20T01:22:19.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.096+0000 7f07874cf700 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:6 9.14 9:28000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x556acea88700 con 0x556acdf2e800 2023-05-20T01:22:19.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.096+0000 7f07874cf700 1 -- 172.21.15.16:0/3037775639 <== osd.1 v2:172.21.15.16:6808/3860264220 3 ==== osd_op_reply(6 [pgnls start_epoch 71 out=125b] v71'133 uv133 ondisk = 1) v8 ==== 144+0+125 (crc 0 0 0) 0x556acdf2d440 con 0x556acdf2e800 2023-05-20T01:22:19.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.096+0000 7f07874cf700 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:7 9.c 9:30000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x556acea88700 con 0x556acdf4ec00 2023-05-20T01:22:19.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.097+0000 7f0787cd0700 1 -- 172.21.15.16:0/3037775639 <== osd.3 v2:172.21.15.16:6800/3669781890 2 ==== osd_op_reply(7 [pgnls start_epoch 71 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556acdf2d8c0 con 0x556acdf4ec00 2023-05-20T01:22:19.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.097+0000 7f0787cd0700 1 --2- 172.21.15.16:0/3037775639 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x556acdf4f800 0x556ace055080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:19.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.097+0000 7f0787cd0700 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:8 9.1c 9:38000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x556acea88700 con 0x556acdf4f800 2023-05-20T01:22:19.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.097+0000 7f07884d1700 1 --2- 172.21.15.16:0/3037775639 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x556acdf4f800 0x556ace055080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:19.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.097+0000 7f07884d1700 1 --2- 172.21.15.16:0/3037775639 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x556acdf4f800 0x556ace055080 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:19.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.097+0000 7f07884d1700 1 -- 172.21.15.16:0/3037775639 <== osd.0 v2:172.21.15.16:6824/3923634179 1 ==== osd_op_reply(8 [pgnls start_epoch 71 out=49b] v70'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556acdf2d680 con 0x556acdf4f800 2023-05-20T01:22:19.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.097+0000 7f07884d1700 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:9 9.2 9:40000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x556acea88700 con 0x556acdf4ec00 2023-05-20T01:22:19.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.098+0000 7f0787cd0700 1 -- 172.21.15.16:0/3037775639 <== osd.3 v2:172.21.15.16:6800/3669781890 3 ==== osd_op_reply(9 [pgnls start_epoch 71 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556acdf2d8c0 con 0x556acdf4ec00 2023-05-20T01:22:19.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.098+0000 7f0787cd0700 1 --2- 172.21.15.16:0/3037775639 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x556acdf4fc00 0x556ace055600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:19.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.098+0000 7f0787cd0700 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:10 9.12 9:48000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x556acea88700 con 0x556acdf4fc00 2023-05-20T01:22:19.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.098+0000 7f0787cd0700 1 --2- 172.21.15.16:0/3037775639 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x556acdf4fc00 0x556ace055600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:19.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.098+0000 7f0787cd0700 1 --2- 172.21.15.16:0/3037775639 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x556acdf4fc00 0x556ace055600 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:19.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.099+0000 7f0787cd0700 1 -- 172.21.15.16:0/3037775639 <== osd.5 v2:172.21.15.132:6816/1844689995 1 ==== osd_op_reply(10 [pgnls start_epoch 71 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556acdf2d8c0 con 0x556acdf4fc00 2023-05-20T01:22:19.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.099+0000 7f0787cd0700 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:11 9.a 9:50000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x556acea88700 con 0x556acdf4fc00 2023-05-20T01:22:19.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.099+0000 7f0787cd0700 1 -- 172.21.15.16:0/3037775639 <== osd.5 v2:172.21.15.132:6816/1844689995 2 ==== osd_op_reply(11 [pgnls start_epoch 71 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556acdf2d8c0 con 0x556acdf4fc00 2023-05-20T01:22:19.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.099+0000 7f0787cd0700 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:12 9.1a 9:58000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x556acea88700 con 0x556acdf4ec00 2023-05-20T01:22:19.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.099+0000 7f0787cd0700 1 -- 172.21.15.16:0/3037775639 <== osd.3 v2:172.21.15.16:6800/3669781890 4 ==== osd_op_reply(12 [pgnls start_epoch 71 out=49b] v71'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556acdf2d8c0 con 0x556acdf4ec00 2023-05-20T01:22:19.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.099+0000 7f0787cd0700 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:13 9.6 9:60000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x556acea88700 con 0x556acdf4fc00 2023-05-20T01:22:19.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.099+0000 7f0787cd0700 1 -- 172.21.15.16:0/3037775639 <== osd.5 v2:172.21.15.132:6816/1844689995 3 ==== osd_op_reply(13 [pgnls start_epoch 71 out=49b] v70'28 uv28 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556acdf2d8c0 con 0x556acdf4fc00 2023-05-20T01:22:19.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.099+0000 7f0787cd0700 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:14 9.16 9:68000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x556acea88700 con 0x556acdf4e800 2023-05-20T01:22:19.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.100+0000 7f07884d1700 1 -- 172.21.15.16:0/3037775639 <== osd.4 v2:172.21.15.132:6817/506317223 2 ==== osd_op_reply(14 [pgnls start_epoch 71 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556acdf2d680 con 0x556acdf4e800 2023-05-20T01:22:19.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.100+0000 7f07884d1700 1 --2- 172.21.15.16:0/3037775639 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x556acdf4f000 0x556ace055b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:19.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.100+0000 7f07884d1700 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:15 9.e 9:70000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x556acea88700 con 0x556acdf4f000 2023-05-20T01:22:19.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.100+0000 7f07874cf700 1 --2- 172.21.15.16:0/3037775639 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x556acdf4f000 0x556ace055b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:19.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.100+0000 7f07874cf700 1 --2- 172.21.15.16:0/3037775639 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x556acdf4f000 0x556ace055b80 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:19.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.101+0000 7f07874cf700 1 -- 172.21.15.16:0/3037775639 <== osd.7 v2:172.21.15.132:6800/466001149 1 ==== osd_op_reply(15 [pgnls start_epoch 71 out=49b] v70'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556acdf2d440 con 0x556acdf4f000 2023-05-20T01:22:19.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.101+0000 7f07874cf700 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:16 9.1e 9:78000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x556acea88700 con 0x556acdf4f800 2023-05-20T01:22:19.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.101+0000 7f07884d1700 1 -- 172.21.15.16:0/3037775639 <== osd.0 v2:172.21.15.16:6824/3923634179 2 ==== osd_op_reply(16 [pgnls start_epoch 71 out=49b] v71'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556acdf2d680 con 0x556acdf4f800 2023-05-20T01:22:19.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.101+0000 7f07884d1700 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:17 9.1 9:80000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x556acea88700 con 0x556acdf4ec00 2023-05-20T01:22:19.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.101+0000 7f0787cd0700 1 -- 172.21.15.16:0/3037775639 <== osd.3 v2:172.21.15.16:6800/3669781890 5 ==== osd_op_reply(17 [pgnls start_epoch 71 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556acdf2d8c0 con 0x556acdf4ec00 2023-05-20T01:22:19.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.101+0000 7f0787cd0700 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:18 9.11 9:88000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x556acea88700 con 0x556acdf4f000 2023-05-20T01:22:19.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.102+0000 7f07874cf700 1 -- 172.21.15.16:0/3037775639 <== osd.7 v2:172.21.15.132:6800/466001149 2 ==== osd_op_reply(18 [pgnls start_epoch 71 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556acdf2d440 con 0x556acdf4f000 2023-05-20T01:22:19.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.102+0000 7f07874cf700 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:19 9.9 9:90000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x556acea88700 con 0x556acdf4f800 2023-05-20T01:22:19.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.102+0000 7f07884d1700 1 -- 172.21.15.16:0/3037775639 <== osd.0 v2:172.21.15.16:6824/3923634179 3 ==== osd_op_reply(19 [pgnls start_epoch 71 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556acdf2d680 con 0x556acdf4f800 2023-05-20T01:22:19.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.102+0000 7f07884d1700 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:20 9.19 9:98000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x556acea88700 con 0x556acdf4ec00 2023-05-20T01:22:19.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.102+0000 7f0787cd0700 1 -- 172.21.15.16:0/3037775639 <== osd.3 v2:172.21.15.16:6800/3669781890 6 ==== osd_op_reply(20 [pgnls start_epoch 71 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556acdf2d8c0 con 0x556acdf4ec00 2023-05-20T01:22:19.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.102+0000 7f0787cd0700 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:21 9.5 9:a0000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x556acea88700 con 0x556acdf4f800 2023-05-20T01:22:19.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.102+0000 7f07884d1700 1 -- 172.21.15.16:0/3037775639 <== osd.0 v2:172.21.15.16:6824/3923634179 4 ==== osd_op_reply(21 [pgnls start_epoch 71 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556acdf2d680 con 0x556acdf4f800 2023-05-20T01:22:19.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.102+0000 7f07884d1700 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:22 9.15 9:a8000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x556acea88700 con 0x556acdf4f000 2023-05-20T01:22:19.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.103+0000 7f07874cf700 1 -- 172.21.15.16:0/3037775639 <== osd.7 v2:172.21.15.132:6800/466001149 3 ==== osd_op_reply(22 [pgnls start_epoch 71 out=73b] v70'6 uv6 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x556acdf2d440 con 0x556acdf4f000 2023-05-20T01:22:19.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.103+0000 7f07874cf700 1 --2- 172.21.15.16:0/3037775639 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x556ace076000 0x556ace056680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:19.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.103+0000 7f07874cf700 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:23 9.d 9:b0000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x556acea88700 con 0x556ace076000 2023-05-20T01:22:19.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.103+0000 7f07884d1700 1 --2- 172.21.15.16:0/3037775639 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x556ace076000 0x556ace056680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:19.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.103+0000 7f07884d1700 1 --2- 172.21.15.16:0/3037775639 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x556ace076000 0x556ace056680 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:19.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.103+0000 7f07884d1700 1 -- 172.21.15.16:0/3037775639 <== osd.2 v2:172.21.15.16:6816/293187030 1 ==== osd_op_reply(23 [pgnls start_epoch 71 out=74b] v71'3 uv3 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x556acdf2d680 con 0x556ace076000 2023-05-20T01:22:19.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.103+0000 7f07884d1700 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:24 9.1d 9:b8000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x556acea88700 con 0x556acdf4f800 2023-05-20T01:22:19.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.104+0000 7f07884d1700 1 -- 172.21.15.16:0/3037775639 <== osd.0 v2:172.21.15.16:6824/3923634179 5 ==== osd_op_reply(24 [pgnls start_epoch 71 out=49b] v70'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556acdf2d680 con 0x556acdf4f800 2023-05-20T01:22:19.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.104+0000 7f07884d1700 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:25 9.3 9:c0000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x556acea88700 con 0x556acdf4f000 2023-05-20T01:22:19.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.104+0000 7f07874cf700 1 -- 172.21.15.16:0/3037775639 <== osd.7 v2:172.21.15.132:6800/466001149 4 ==== osd_op_reply(25 [pgnls start_epoch 71 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556acdf2d440 con 0x556acdf4f000 2023-05-20T01:22:19.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.104+0000 7f07874cf700 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:26 9.13 9:c8000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x556acea88700 con 0x556acdf4f000 2023-05-20T01:22:19.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.104+0000 7f07874cf700 1 -- 172.21.15.16:0/3037775639 <== osd.7 v2:172.21.15.132:6800/466001149 5 ==== osd_op_reply(26 [pgnls start_epoch 71 out=98b] v71'7 uv7 ondisk = 1) v8 ==== 144+0+98 (crc 0 0 0) 0x556acdf2d440 con 0x556acdf4f000 2023-05-20T01:22:19.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.104+0000 7f07874cf700 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:27 9.b 9:d0000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x556acea88700 con 0x556acdf4f800 2023-05-20T01:22:19.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.105+0000 7f07884d1700 1 -- 172.21.15.16:0/3037775639 <== osd.0 v2:172.21.15.16:6824/3923634179 6 ==== osd_op_reply(27 [pgnls start_epoch 71 out=76b] v71'6 uv5 ondisk = 1) v8 ==== 144+0+76 (crc 0 0 0) 0x556acdf2d680 con 0x556acdf4f800 2023-05-20T01:22:19.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.105+0000 7f07884d1700 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:28 9.1b 9:d8000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x556acea88700 con 0x556acdf4ec00 2023-05-20T01:22:19.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.105+0000 7f0787cd0700 1 -- 172.21.15.16:0/3037775639 <== osd.3 v2:172.21.15.16:6800/3669781890 7 ==== osd_op_reply(28 [pgnls start_epoch 71 out=49b] v71'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556acdf2d8c0 con 0x556acdf4ec00 2023-05-20T01:22:19.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.105+0000 7f0787cd0700 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:29 9.7 9:e0000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x556acea88700 con 0x556acdf4fc00 2023-05-20T01:22:19.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.106+0000 7f0787cd0700 1 -- 172.21.15.16:0/3037775639 <== osd.5 v2:172.21.15.132:6816/1844689995 4 ==== osd_op_reply(29 [pgnls start_epoch 71 out=49b] v70'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556acdf2d8c0 con 0x556acdf4fc00 2023-05-20T01:22:19.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.106+0000 7f0787cd0700 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:30 9.17 9:e8000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x556acea88700 con 0x556acdf4f000 2023-05-20T01:22:19.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.106+0000 7f07874cf700 1 -- 172.21.15.16:0/3037775639 <== osd.7 v2:172.21.15.132:6800/466001149 6 ==== osd_op_reply(30 [pgnls start_epoch 71 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556acdf2d440 con 0x556acdf4f000 2023-05-20T01:22:19.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.106+0000 7f07874cf700 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:31 9.f 9:f0000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x556acea88700 con 0x556acdf4fc00 2023-05-20T01:22:19.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.106+0000 7f0787cd0700 1 -- 172.21.15.16:0/3037775639 <== osd.5 v2:172.21.15.132:6816/1844689995 5 ==== osd_op_reply(31 [pgnls start_epoch 71 out=73b] v71'4 uv4 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x556acdf2d8c0 con 0x556acdf4fc00 2023-05-20T01:22:19.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.106+0000 7f0787cd0700 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:32 9.1f 9:f8000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x556acea88700 con 0x556acdf2e800 2023-05-20T01:22:19.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.106+0000 7f07874cf700 1 -- 172.21.15.16:0/3037775639 <== osd.1 v2:172.21.15.16:6808/3860264220 4 ==== osd_op_reply(32 [pgnls start_epoch 71 out=101b] v71'12 uv12 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x556acdf2d440 con 0x556acdf2e800 2023-05-20T01:22:19.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.107+0000 7f0789d227c0 10 datascan.scan_links: step 1: handling object 10000000000.00000000 2023-05-20T01:22:19.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.107+0000 7f0789d227c0 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:33 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x556acea88a80 con 0x556acdf4e800 2023-05-20T01:22:19.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.107+0000 7f07884d1700 1 -- 172.21.15.16:0/3037775639 <== osd.4 v2:172.21.15.132:6817/506317223 3 ==== osd_op_reply(33 10000000000.00000000 [omap-get-vals out=12470b] v0'0 uv41 ondisk = 0) v8 ==== 164+0+12470 (crc 0 0 0) 0x556acdf2d680 con 0x556acdf4e800 2023-05-20T01:22:19.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.107+0000 7f0789d227c0 20 datascan.scan_links: injected ino detected 2023-05-20T01:22:19.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.107+0000 7f0789d227c0 20 datascan.scan_links: injected ino detected 2023-05-20T01:22:19.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.107+0000 7f0789d227c0 20 datascan.scan_links: injected ino detected 2023-05-20T01:22:19.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.107+0000 7f0789d227c0 20 datascan.scan_links: injected ino detected 2023-05-20T01:22:19.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.107+0000 7f0789d227c0 20 datascan.scan_links: injected ino detected 2023-05-20T01:22:19.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.107+0000 7f0789d227c0 20 datascan.scan_links: injected ino detected 2023-05-20T01:22:19.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.107+0000 7f0789d227c0 20 datascan.scan_links: injected ino detected 2023-05-20T01:22:19.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.107+0000 7f0789d227c0 20 datascan.scan_links: injected ino detected 2023-05-20T01:22:19.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.107+0000 7f0789d227c0 20 datascan.scan_links: injected ino detected 2023-05-20T01:22:19.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.107+0000 7f0789d227c0 20 datascan.scan_links: injected ino detected 2023-05-20T01:22:19.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.107+0000 7f0789d227c0 20 datascan.scan_links: injected ino detected 2023-05-20T01:22:19.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.107+0000 7f0789d227c0 20 datascan.scan_links: injected ino detected 2023-05-20T01:22:19.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.107+0000 7f0789d227c0 20 datascan.scan_links: injected ino detected 2023-05-20T01:22:19.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.107+0000 7f0789d227c0 20 datascan.scan_links: injected ino detected 2023-05-20T01:22:19.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.107+0000 7f0789d227c0 20 datascan.scan_links: injected ino detected 2023-05-20T01:22:19.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.107+0000 7f0789d227c0 20 datascan.scan_links: injected ino detected 2023-05-20T01:22:19.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.107+0000 7f0789d227c0 20 datascan.scan_links: injected ino detected 2023-05-20T01:22:19.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.107+0000 7f0789d227c0 20 datascan.scan_links: injected ino detected 2023-05-20T01:22:19.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.107+0000 7f0789d227c0 20 datascan.scan_links: injected ino detected 2023-05-20T01:22:19.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.107+0000 7f0789d227c0 20 datascan.scan_links: injected ino detected 2023-05-20T01:22:19.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.107+0000 7f0789d227c0 20 datascan.scan_links: injected ino detected 2023-05-20T01:22:19.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.107+0000 7f0789d227c0 20 datascan.scan_links: injected ino detected 2023-05-20T01:22:19.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.107+0000 7f0789d227c0 20 datascan.scan_links: injected ino detected 2023-05-20T01:22:19.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.107+0000 7f0789d227c0 20 datascan.scan_links: injected ino detected 2023-05-20T01:22:19.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.107+0000 7f0789d227c0 20 datascan.scan_links: injected ino detected 2023-05-20T01:22:19.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.107+0000 7f0789d227c0 10 datascan.scan_links: step 1: handling object 1.00000000.inode 2023-05-20T01:22:19.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.107+0000 7f0789d227c0 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2023-05-20T01:22:19.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.107+0000 7f0789d227c0 10 datascan.scan_links: step 1: handling object 200.00000000 2023-05-20T01:22:19.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.107+0000 7f0789d227c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2023-05-20T01:22:19.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.107+0000 7f0789d227c0 10 datascan.scan_links: step 1: handling object 200.00000001 2023-05-20T01:22:19.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.107+0000 7f0789d227c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2023-05-20T01:22:19.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.107+0000 7f0789d227c0 10 datascan.scan_links: step 1: handling object 500.00000000 2023-05-20T01:22:19.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.107+0000 7f0789d227c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2023-05-20T01:22:19.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.107+0000 7f0789d227c0 10 datascan.scan_links: step 1: handling object mds_snaptable 2023-05-20T01:22:19.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.107+0000 7f0789d227c0 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2023-05-20T01:22:19.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.107+0000 7f0789d227c0 10 datascan.scan_links: step 1: handling object mds0_inotable 2023-05-20T01:22:19.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.107+0000 7f0789d227c0 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2023-05-20T01:22:19.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.107+0000 7f0789d227c0 10 datascan.scan_links: step 1: handling object 100.00000000 2023-05-20T01:22:19.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.107+0000 7f0789d227c0 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:34 9.13 9:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x556acea88e00 con 0x556acdf4f000 2023-05-20T01:22:19.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.108+0000 7f07874cf700 1 -- 172.21.15.16:0/3037775639 <== osd.7 v2:172.21.15.132:6800/466001149 7 ==== osd_op_reply(34 100.00000000 [omap-get-vals out=5b] v0'0 uv7 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x556acdf2d440 con 0x556acdf4f000 2023-05-20T01:22:19.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.108+0000 7f0789d227c0 10 datascan.scan_links: step 1: handling object mds0_sessionmap 2023-05-20T01:22:19.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.108+0000 7f0789d227c0 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2023-05-20T01:22:19.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.108+0000 7f0789d227c0 10 datascan.scan_links: step 1: handling object 400.00000000 2023-05-20T01:22:19.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.108+0000 7f0789d227c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2023-05-20T01:22:19.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.108+0000 7f0789d227c0 10 datascan.scan_links: step 1: handling object 100.00000000.inode 2023-05-20T01:22:19.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.108+0000 7f0789d227c0 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2023-05-20T01:22:19.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.108+0000 7f0789d227c0 10 datascan.scan_links: step 1: handling object 1.00000000 2023-05-20T01:22:19.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.108+0000 7f0789d227c0 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:35 9.1f 9:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x556acea88700 con 0x556acdf2e800 2023-05-20T01:22:19.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.108+0000 7f07874cf700 1 -- 172.21.15.16:0/3037775639 <== osd.1 v2:172.21.15.16:6808/3860264220 5 ==== osd_op_reply(35 1.00000000 [omap-get-vals out=508b] v0'0 uv12 ondisk = 0) v8 ==== 154+0+508 (crc 0 0 0) 0x556acdf2d440 con 0x556acdf2e800 2023-05-20T01:22:19.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.108+0000 7f0789d227c0 20 datascan.scan_links: injected ino detected 2023-05-20T01:22:19.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.108+0000 7f0789d227c0 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:36 9.0 9:00000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x556acea88e00 con 0x556acdf2e800 2023-05-20T01:22:19.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.108+0000 7f07874cf700 1 -- 172.21.15.16:0/3037775639 <== osd.1 v2:172.21.15.16:6808/3860264220 6 ==== osd_op_reply(36 [pgnls start_epoch 71 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556acdf2d440 con 0x556acdf2e800 2023-05-20T01:22:19.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.108+0000 7f07874cf700 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:37 9.10 9:08000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x556acea88e00 con 0x556acdf4e800 2023-05-20T01:22:19.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.109+0000 7f07884d1700 1 -- 172.21.15.16:0/3037775639 <== osd.4 v2:172.21.15.132:6817/506317223 4 ==== osd_op_reply(37 [pgnls start_epoch 71 out=81b] v71'41 uv41 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x556acdf2d680 con 0x556acdf4e800 2023-05-20T01:22:19.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.109+0000 7f07884d1700 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:38 9.8 9:10000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x556acea88a80 con 0x556acdf4ec00 2023-05-20T01:22:19.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.109+0000 7f0787cd0700 1 -- 172.21.15.16:0/3037775639 <== osd.3 v2:172.21.15.16:6800/3669781890 8 ==== osd_op_reply(38 [pgnls start_epoch 71 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556acdf2d8c0 con 0x556acdf4ec00 2023-05-20T01:22:19.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.109+0000 7f0787cd0700 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:39 9.18 9:18000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x556acea88a80 con 0x556acdf4f400 2023-05-20T01:22:19.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.109+0000 7f07874cf700 1 -- 172.21.15.16:0/3037775639 <== osd.6 v2:172.21.15.132:6801/727581123 2 ==== osd_op_reply(39 [pgnls start_epoch 71 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556acdf2d440 con 0x556acdf4f400 2023-05-20T01:22:19.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.109+0000 7f07874cf700 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:40 9.4 9:20000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x556acea88a80 con 0x556acdf2e800 2023-05-20T01:22:19.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.109+0000 7f07874cf700 1 -- 172.21.15.16:0/3037775639 <== osd.1 v2:172.21.15.16:6808/3860264220 7 ==== osd_op_reply(40 [pgnls start_epoch 71 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556acdf2d440 con 0x556acdf2e800 2023-05-20T01:22:19.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.109+0000 7f07874cf700 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:41 9.14 9:28000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x556acea88a80 con 0x556acdf2e800 2023-05-20T01:22:19.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.110+0000 7f07874cf700 1 -- 172.21.15.16:0/3037775639 <== osd.1 v2:172.21.15.16:6808/3860264220 8 ==== osd_op_reply(41 [pgnls start_epoch 71 out=125b] v71'133 uv133 ondisk = 1) v8 ==== 144+0+125 (crc 0 0 0) 0x556acdf2d440 con 0x556acdf2e800 2023-05-20T01:22:19.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.110+0000 7f07874cf700 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:42 9.c 9:30000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x556acea88a80 con 0x556acdf4ec00 2023-05-20T01:22:19.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.110+0000 7f0787cd0700 1 -- 172.21.15.16:0/3037775639 <== osd.3 v2:172.21.15.16:6800/3669781890 9 ==== osd_op_reply(42 [pgnls start_epoch 71 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556acdf2d8c0 con 0x556acdf4ec00 2023-05-20T01:22:19.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.110+0000 7f0787cd0700 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:43 9.1c 9:38000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x556acea88a80 con 0x556acdf4f800 2023-05-20T01:22:19.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.110+0000 7f07884d1700 1 -- 172.21.15.16:0/3037775639 <== osd.0 v2:172.21.15.16:6824/3923634179 7 ==== osd_op_reply(43 [pgnls start_epoch 71 out=49b] v70'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556acdf2d680 con 0x556acdf4f800 2023-05-20T01:22:19.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.110+0000 7f07884d1700 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:44 9.2 9:40000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x556acea88a80 con 0x556acdf4ec00 2023-05-20T01:22:19.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.111+0000 7f0787cd0700 1 -- 172.21.15.16:0/3037775639 <== osd.3 v2:172.21.15.16:6800/3669781890 10 ==== osd_op_reply(44 [pgnls start_epoch 71 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556acdf2d8c0 con 0x556acdf4ec00 2023-05-20T01:22:19.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.111+0000 7f0787cd0700 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:45 9.12 9:48000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x556acea88a80 con 0x556acdf4fc00 2023-05-20T01:22:19.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.111+0000 7f0787cd0700 1 -- 172.21.15.16:0/3037775639 <== osd.5 v2:172.21.15.132:6816/1844689995 6 ==== osd_op_reply(45 [pgnls start_epoch 71 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556acdf2d8c0 con 0x556acdf4fc00 2023-05-20T01:22:19.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.111+0000 7f0787cd0700 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:46 9.a 9:50000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x556acea88a80 con 0x556acdf4fc00 2023-05-20T01:22:19.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.111+0000 7f0787cd0700 1 -- 172.21.15.16:0/3037775639 <== osd.5 v2:172.21.15.132:6816/1844689995 7 ==== osd_op_reply(46 [pgnls start_epoch 71 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556acdf2d8c0 con 0x556acdf4fc00 2023-05-20T01:22:19.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.111+0000 7f0787cd0700 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:47 9.1a 9:58000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x556acea88a80 con 0x556acdf4ec00 2023-05-20T01:22:19.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.111+0000 7f0787cd0700 1 -- 172.21.15.16:0/3037775639 <== osd.3 v2:172.21.15.16:6800/3669781890 11 ==== osd_op_reply(47 [pgnls start_epoch 71 out=49b] v71'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556acdf2d8c0 con 0x556acdf4ec00 2023-05-20T01:22:19.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.112+0000 7f0787cd0700 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:48 9.6 9:60000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x556acea88a80 con 0x556acdf4fc00 2023-05-20T01:22:19.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.112+0000 7f0787cd0700 1 -- 172.21.15.16:0/3037775639 <== osd.5 v2:172.21.15.132:6816/1844689995 8 ==== osd_op_reply(48 [pgnls start_epoch 71 out=49b] v70'28 uv28 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556acdf2d8c0 con 0x556acdf4fc00 2023-05-20T01:22:19.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.112+0000 7f0787cd0700 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:49 9.16 9:68000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x556acea88a80 con 0x556acdf4e800 2023-05-20T01:22:19.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.112+0000 7f07884d1700 1 -- 172.21.15.16:0/3037775639 <== osd.4 v2:172.21.15.132:6817/506317223 5 ==== osd_op_reply(49 [pgnls start_epoch 71 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556acdf2d680 con 0x556acdf4e800 2023-05-20T01:22:19.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.112+0000 7f07884d1700 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:50 9.e 9:70000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x556acea88a80 con 0x556acdf4f000 2023-05-20T01:22:19.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.113+0000 7f07874cf700 1 -- 172.21.15.16:0/3037775639 <== osd.7 v2:172.21.15.132:6800/466001149 8 ==== osd_op_reply(50 [pgnls start_epoch 71 out=49b] v70'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556acdf2d440 con 0x556acdf4f000 2023-05-20T01:22:19.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.113+0000 7f07874cf700 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:51 9.1e 9:78000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x556acea88a80 con 0x556acdf4f800 2023-05-20T01:22:19.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.113+0000 7f07884d1700 1 -- 172.21.15.16:0/3037775639 <== osd.0 v2:172.21.15.16:6824/3923634179 8 ==== osd_op_reply(51 [pgnls start_epoch 71 out=49b] v71'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556acdf2d680 con 0x556acdf4f800 2023-05-20T01:22:19.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.113+0000 7f07884d1700 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:52 9.1 9:80000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x556acea88a80 con 0x556acdf4ec00 2023-05-20T01:22:19.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.113+0000 7f0787cd0700 1 -- 172.21.15.16:0/3037775639 <== osd.3 v2:172.21.15.16:6800/3669781890 12 ==== osd_op_reply(52 [pgnls start_epoch 71 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556acdf2d8c0 con 0x556acdf4ec00 2023-05-20T01:22:19.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.113+0000 7f0787cd0700 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:53 9.11 9:88000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x556acea88a80 con 0x556acdf4f000 2023-05-20T01:22:19.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.113+0000 7f07874cf700 1 -- 172.21.15.16:0/3037775639 <== osd.7 v2:172.21.15.132:6800/466001149 9 ==== osd_op_reply(53 [pgnls start_epoch 71 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556acdf2d440 con 0x556acdf4f000 2023-05-20T01:22:19.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.113+0000 7f07874cf700 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:54 9.9 9:90000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x556acea88a80 con 0x556acdf4f800 2023-05-20T01:22:19.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.114+0000 7f07884d1700 1 -- 172.21.15.16:0/3037775639 <== osd.0 v2:172.21.15.16:6824/3923634179 9 ==== osd_op_reply(54 [pgnls start_epoch 71 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556acdf2d680 con 0x556acdf4f800 2023-05-20T01:22:19.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.114+0000 7f07884d1700 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:55 9.19 9:98000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x556acea88a80 con 0x556acdf4ec00 2023-05-20T01:22:19.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.114+0000 7f0787cd0700 1 -- 172.21.15.16:0/3037775639 <== osd.3 v2:172.21.15.16:6800/3669781890 13 ==== osd_op_reply(55 [pgnls start_epoch 71 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556acdf2d8c0 con 0x556acdf4ec00 2023-05-20T01:22:19.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.114+0000 7f0787cd0700 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:56 9.5 9:a0000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x556acea88a80 con 0x556acdf4f800 2023-05-20T01:22:19.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.114+0000 7f07884d1700 1 -- 172.21.15.16:0/3037775639 <== osd.0 v2:172.21.15.16:6824/3923634179 10 ==== osd_op_reply(56 [pgnls start_epoch 71 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556acdf2d680 con 0x556acdf4f800 2023-05-20T01:22:19.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.114+0000 7f07884d1700 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:57 9.15 9:a8000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x556acea88a80 con 0x556acdf4f000 2023-05-20T01:22:19.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.114+0000 7f07874cf700 1 -- 172.21.15.16:0/3037775639 <== osd.7 v2:172.21.15.132:6800/466001149 10 ==== osd_op_reply(57 [pgnls start_epoch 71 out=73b] v70'6 uv6 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x556acdf2d440 con 0x556acdf4f000 2023-05-20T01:22:19.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.114+0000 7f07874cf700 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:58 9.d 9:b0000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x556acea88a80 con 0x556ace076000 2023-05-20T01:22:19.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.115+0000 7f07884d1700 1 -- 172.21.15.16:0/3037775639 <== osd.2 v2:172.21.15.16:6816/293187030 2 ==== osd_op_reply(58 [pgnls start_epoch 71 out=74b] v71'3 uv3 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x556acdf2d680 con 0x556ace076000 2023-05-20T01:22:19.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.115+0000 7f07884d1700 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:59 9.1d 9:b8000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x556acea88a80 con 0x556acdf4f800 2023-05-20T01:22:19.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.115+0000 7f07884d1700 1 -- 172.21.15.16:0/3037775639 <== osd.0 v2:172.21.15.16:6824/3923634179 11 ==== osd_op_reply(59 [pgnls start_epoch 71 out=49b] v70'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556acdf2d680 con 0x556acdf4f800 2023-05-20T01:22:19.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.115+0000 7f07884d1700 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:60 9.3 9:c0000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x556acea88a80 con 0x556acdf4f000 2023-05-20T01:22:19.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.115+0000 7f07874cf700 1 -- 172.21.15.16:0/3037775639 <== osd.7 v2:172.21.15.132:6800/466001149 11 ==== osd_op_reply(60 [pgnls start_epoch 71 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556acdf2d440 con 0x556acdf4f000 2023-05-20T01:22:19.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.115+0000 7f07874cf700 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:61 9.13 9:c8000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x556acea88a80 con 0x556acdf4f000 2023-05-20T01:22:19.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.116+0000 7f07874cf700 1 -- 172.21.15.16:0/3037775639 <== osd.7 v2:172.21.15.132:6800/466001149 12 ==== osd_op_reply(61 [pgnls start_epoch 71 out=98b] v71'7 uv7 ondisk = 1) v8 ==== 144+0+98 (crc 0 0 0) 0x556acdf2d440 con 0x556acdf4f000 2023-05-20T01:22:19.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.116+0000 7f07874cf700 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:62 9.b 9:d0000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x556acea88a80 con 0x556acdf4f800 2023-05-20T01:22:19.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.116+0000 7f07884d1700 1 -- 172.21.15.16:0/3037775639 <== osd.0 v2:172.21.15.16:6824/3923634179 12 ==== osd_op_reply(62 [pgnls start_epoch 71 out=76b] v71'6 uv5 ondisk = 1) v8 ==== 144+0+76 (crc 0 0 0) 0x556acdf2d680 con 0x556acdf4f800 2023-05-20T01:22:19.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.116+0000 7f07884d1700 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:63 9.1b 9:d8000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x556acea88a80 con 0x556acdf4ec00 2023-05-20T01:22:19.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.116+0000 7f0787cd0700 1 -- 172.21.15.16:0/3037775639 <== osd.3 v2:172.21.15.16:6800/3669781890 14 ==== osd_op_reply(63 [pgnls start_epoch 71 out=49b] v71'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556acdf2d8c0 con 0x556acdf4ec00 2023-05-20T01:22:19.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.116+0000 7f0787cd0700 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:64 9.7 9:e0000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x556acea88a80 con 0x556acdf4fc00 2023-05-20T01:22:19.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.116+0000 7f0787cd0700 1 -- 172.21.15.16:0/3037775639 <== osd.5 v2:172.21.15.132:6816/1844689995 9 ==== osd_op_reply(64 [pgnls start_epoch 71 out=49b] v70'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556acdf2d8c0 con 0x556acdf4fc00 2023-05-20T01:22:19.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.116+0000 7f0787cd0700 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:65 9.17 9:e8000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x556acea88a80 con 0x556acdf4f000 2023-05-20T01:22:19.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.117+0000 7f07874cf700 1 -- 172.21.15.16:0/3037775639 <== osd.7 v2:172.21.15.132:6800/466001149 13 ==== osd_op_reply(65 [pgnls start_epoch 71 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556acdf2d440 con 0x556acdf4f000 2023-05-20T01:22:19.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.117+0000 7f07874cf700 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:66 9.f 9:f0000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x556acea88a80 con 0x556acdf4fc00 2023-05-20T01:22:19.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.117+0000 7f0787cd0700 1 -- 172.21.15.16:0/3037775639 <== osd.5 v2:172.21.15.132:6816/1844689995 10 ==== osd_op_reply(66 [pgnls start_epoch 71 out=73b] v71'4 uv4 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x556acdf2d8c0 con 0x556acdf4fc00 2023-05-20T01:22:19.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.117+0000 7f0787cd0700 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:67 9.1f 9:f8000000::::head [pgnls start_epoch 71 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e71) v8 -- 0x556acea88a80 con 0x556acdf2e800 2023-05-20T01:22:19.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.117+0000 7f07874cf700 1 -- 172.21.15.16:0/3037775639 <== osd.1 v2:172.21.15.16:6808/3860264220 9 ==== osd_op_reply(67 [pgnls start_epoch 71 out=101b] v71'12 uv12 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x556acdf2d440 con 0x556acdf2e800 2023-05-20T01:22:19.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.117+0000 7f0789d227c0 10 datascan.scan_links: step 2: handling object 10000000000.00000000 2023-05-20T01:22:19.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.117+0000 7f0789d227c0 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:68 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x556acea89180 con 0x556acdf4e800 2023-05-20T01:22:19.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.118+0000 7f07884d1700 1 -- 172.21.15.16:0/3037775639 <== osd.4 v2:172.21.15.132:6817/506317223 6 ==== osd_op_reply(68 10000000000.00000000 [omap-get-vals out=12470b] v0'0 uv41 ondisk = 0) v8 ==== 164+0+12470 (crc 0 0 0) 0x556acdf2d680 con 0x556acdf4e800 2023-05-20T01:22:19.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.118+0000 7f0789d227c0 20 datascan.scan_links: injected ino detected 2023-05-20T01:22:19.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.118+0000 7f0789d227c0 20 datascan.scan_links: adding 0x10000000001 for future processing to fix dnfirst 2023-05-20T01:22:19.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.118+0000 7f0789d227c0 20 datascan.scan_links: injected ino detected 2023-05-20T01:22:19.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.118+0000 7f0789d227c0 20 datascan.scan_links: adding 0x100000001ff for future processing to fix dnfirst 2023-05-20T01:22:19.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.118+0000 7f0789d227c0 20 datascan.scan_links: injected ino detected 2023-05-20T01:22:19.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.118+0000 7f0789d227c0 20 datascan.scan_links: adding 0x10000000200 for future processing to fix dnfirst 2023-05-20T01:22:19.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.118+0000 7f0789d227c0 20 datascan.scan_links: injected ino detected 2023-05-20T01:22:19.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.118+0000 7f0789d227c0 20 datascan.scan_links: adding 0x10000000201 for future processing to fix dnfirst 2023-05-20T01:22:19.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.118+0000 7f0789d227c0 20 datascan.scan_links: injected ino detected 2023-05-20T01:22:19.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.118+0000 7f0789d227c0 20 datascan.scan_links: adding 0x10000000202 for future processing to fix dnfirst 2023-05-20T01:22:19.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.118+0000 7f0789d227c0 20 datascan.scan_links: injected ino detected 2023-05-20T01:22:19.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.118+0000 7f0789d227c0 20 datascan.scan_links: adding 0x10000000203 for future processing to fix dnfirst 2023-05-20T01:22:19.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.118+0000 7f0789d227c0 20 datascan.scan_links: injected ino detected 2023-05-20T01:22:19.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.118+0000 7f0789d227c0 20 datascan.scan_links: adding 0x10000000204 for future processing to fix dnfirst 2023-05-20T01:22:19.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.118+0000 7f0789d227c0 20 datascan.scan_links: injected ino detected 2023-05-20T01:22:19.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.118+0000 7f0789d227c0 20 datascan.scan_links: adding 0x10000000205 for future processing to fix dnfirst 2023-05-20T01:22:19.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.118+0000 7f0789d227c0 20 datascan.scan_links: injected ino detected 2023-05-20T01:22:19.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.118+0000 7f0789d227c0 20 datascan.scan_links: adding 0x10000000206 for future processing to fix dnfirst 2023-05-20T01:22:19.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.118+0000 7f0789d227c0 20 datascan.scan_links: injected ino detected 2023-05-20T01:22:19.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.118+0000 7f0789d227c0 20 datascan.scan_links: adding 0x10000000207 for future processing to fix dnfirst 2023-05-20T01:22:19.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.118+0000 7f0789d227c0 20 datascan.scan_links: injected ino detected 2023-05-20T01:22:19.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.118+0000 7f0789d227c0 20 datascan.scan_links: adding 0x10000000208 for future processing to fix dnfirst 2023-05-20T01:22:19.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.118+0000 7f0789d227c0 20 datascan.scan_links: injected ino detected 2023-05-20T01:22:19.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.118+0000 7f0789d227c0 20 datascan.scan_links: adding 0x100000001f6 for future processing to fix dnfirst 2023-05-20T01:22:19.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.118+0000 7f0789d227c0 20 datascan.scan_links: injected ino detected 2023-05-20T01:22:19.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.118+0000 7f0789d227c0 20 datascan.scan_links: adding 0x10000000209 for future processing to fix dnfirst 2023-05-20T01:22:19.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.118+0000 7f0789d227c0 20 datascan.scan_links: injected ino detected 2023-05-20T01:22:19.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.118+0000 7f0789d227c0 20 datascan.scan_links: adding 0x1000000020a for future processing to fix dnfirst 2023-05-20T01:22:19.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.118+0000 7f0789d227c0 20 datascan.scan_links: injected ino detected 2023-05-20T01:22:19.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.118+0000 7f0789d227c0 20 datascan.scan_links: adding 0x1000000020b for future processing to fix dnfirst 2023-05-20T01:22:19.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.118+0000 7f0789d227c0 20 datascan.scan_links: injected ino detected 2023-05-20T01:22:19.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.118+0000 7f0789d227c0 20 datascan.scan_links: adding 0x1000000020c for future processing to fix dnfirst 2023-05-20T01:22:19.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.118+0000 7f0789d227c0 20 datascan.scan_links: injected ino detected 2023-05-20T01:22:19.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.118+0000 7f0789d227c0 20 datascan.scan_links: adding 0x1000000020d for future processing to fix dnfirst 2023-05-20T01:22:19.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.118+0000 7f0789d227c0 20 datascan.scan_links: injected ino detected 2023-05-20T01:22:19.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.118+0000 7f0789d227c0 20 datascan.scan_links: adding 0x100000001f7 for future processing to fix dnfirst 2023-05-20T01:22:19.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.118+0000 7f0789d227c0 20 datascan.scan_links: injected ino detected 2023-05-20T01:22:19.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.118+0000 7f0789d227c0 20 datascan.scan_links: adding 0x100000001f8 for future processing to fix dnfirst 2023-05-20T01:22:19.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.118+0000 7f0789d227c0 20 datascan.scan_links: injected ino detected 2023-05-20T01:22:19.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.118+0000 7f0789d227c0 20 datascan.scan_links: adding 0x100000001f9 for future processing to fix dnfirst 2023-05-20T01:22:19.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.118+0000 7f0789d227c0 20 datascan.scan_links: injected ino detected 2023-05-20T01:22:19.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.118+0000 7f0789d227c0 20 datascan.scan_links: adding 0x100000001fa for future processing to fix dnfirst 2023-05-20T01:22:19.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.118+0000 7f0789d227c0 20 datascan.scan_links: injected ino detected 2023-05-20T01:22:19.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.118+0000 7f0789d227c0 20 datascan.scan_links: adding 0x100000001fb for future processing to fix dnfirst 2023-05-20T01:22:19.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.118+0000 7f0789d227c0 20 datascan.scan_links: injected ino detected 2023-05-20T01:22:19.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.118+0000 7f0789d227c0 20 datascan.scan_links: adding 0x100000001fc for future processing to fix dnfirst 2023-05-20T01:22:19.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.118+0000 7f0789d227c0 20 datascan.scan_links: injected ino detected 2023-05-20T01:22:19.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.118+0000 7f0789d227c0 20 datascan.scan_links: adding 0x100000001fd for future processing to fix dnfirst 2023-05-20T01:22:19.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.118+0000 7f0789d227c0 20 datascan.scan_links: injected ino detected 2023-05-20T01:22:19.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.118+0000 7f0789d227c0 20 datascan.scan_links: adding 0x100000001fe for future processing to fix dnfirst 2023-05-20T01:22:19.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.118+0000 7f0789d227c0 10 datascan.scan_links: step 2: handling object 1.00000000.inode 2023-05-20T01:22:19.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.118+0000 7f0789d227c0 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2023-05-20T01:22:19.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.118+0000 7f0789d227c0 10 datascan.scan_links: step 2: handling object 200.00000000 2023-05-20T01:22:19.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.118+0000 7f0789d227c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2023-05-20T01:22:19.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.118+0000 7f0789d227c0 10 datascan.scan_links: step 2: handling object 200.00000001 2023-05-20T01:22:19.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.118+0000 7f0789d227c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2023-05-20T01:22:19.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.118+0000 7f0789d227c0 10 datascan.scan_links: step 2: handling object 500.00000000 2023-05-20T01:22:19.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.118+0000 7f0789d227c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2023-05-20T01:22:19.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.118+0000 7f0789d227c0 10 datascan.scan_links: step 2: handling object mds_snaptable 2023-05-20T01:22:19.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.118+0000 7f0789d227c0 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2023-05-20T01:22:19.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.118+0000 7f0789d227c0 10 datascan.scan_links: step 2: handling object mds0_inotable 2023-05-20T01:22:19.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.118+0000 7f0789d227c0 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2023-05-20T01:22:19.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.118+0000 7f0789d227c0 10 datascan.scan_links: step 2: handling object 100.00000000 2023-05-20T01:22:19.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.118+0000 7f0789d227c0 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:69 9.13 9:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x556acea89500 con 0x556acdf4f000 2023-05-20T01:22:19.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.118+0000 7f07874cf700 1 -- 172.21.15.16:0/3037775639 <== osd.7 v2:172.21.15.132:6800/466001149 14 ==== osd_op_reply(69 100.00000000 [omap-get-vals out=5b] v0'0 uv7 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x556acdf2d440 con 0x556acdf4f000 2023-05-20T01:22:19.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.118+0000 7f0789d227c0 10 datascan.scan_links: step 2: handling object mds0_sessionmap 2023-05-20T01:22:19.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.118+0000 7f0789d227c0 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2023-05-20T01:22:19.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.118+0000 7f0789d227c0 10 datascan.scan_links: step 2: handling object 400.00000000 2023-05-20T01:22:19.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.118+0000 7f0789d227c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2023-05-20T01:22:19.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.118+0000 7f0789d227c0 10 datascan.scan_links: step 2: handling object 100.00000000.inode 2023-05-20T01:22:19.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.118+0000 7f0789d227c0 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2023-05-20T01:22:19.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.118+0000 7f0789d227c0 10 datascan.scan_links: step 2: handling object 1.00000000 2023-05-20T01:22:19.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.118+0000 7f0789d227c0 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:70 9.1f 9:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x556acea88700 con 0x556acdf2e800 2023-05-20T01:22:19.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.119+0000 7f07874cf700 1 -- 172.21.15.16:0/3037775639 <== osd.1 v2:172.21.15.16:6808/3860264220 10 ==== osd_op_reply(70 1.00000000 [omap-get-vals out=508b] v0'0 uv12 ondisk = 0) v8 ==== 154+0+508 (crc 0 0 0) 0x556acdf2d440 con 0x556acdf2e800 2023-05-20T01:22:19.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.119+0000 7f0789d227c0 20 datascan.scan_links: injected ino detected 2023-05-20T01:22:19.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.119+0000 7f0789d227c0 20 datascan.scan_links: adding 0x10000000000 for future processing to fix dnfirst 2023-05-20T01:22:19.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.119+0000 7f0789d227c0 10 datascan.scan_links: processing 0 dup_primaries, 0 remote_links 2023-05-20T01:22:19.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.119+0000 7f0789d227c0 10 datascan.scan_links: removing dup dentries from 0 objects 2023-05-20T01:22:19.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.119+0000 7f0789d227c0 10 datascan.scan_links: processing 0 bad_nlink_inos 2023-05-20T01:22:19.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.119+0000 7f0789d227c0 10 datascan.scan_links: processing 26 injected_inos 2023-05-20T01:22:19.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.119+0000 7f0789d227c0 10 datascan.scan_links: handling injected_ino 0x10000000000 2023-05-20T01:22:19.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.119+0000 7f0789d227c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x1:subdir 2023-05-20T01:22:19.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.119+0000 7f0789d227c0 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:71 9.1f 9:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x556acea88a80 con 0x556acdf2e800 2023-05-20T01:22:19.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.119+0000 7f07874cf700 1 -- 172.21.15.16:0/3037775639 <== osd.1 v2:172.21.15.16:6808/3860264220 11 ==== osd_op_reply(71 1.00000000 [omap-get-vals-by-keys out=507b] v0'0 uv12 ondisk = 0) v8 ==== 154+0+507 (crc 0 0 0) 0x556acdf2d440 con 0x556acdf2e800 2023-05-20T01:22:19.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.119+0000 7f0789d227c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-05-20T01:22:19.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.119+0000 7f0789d227c0 20 datascan.scan_links: first is now 2 2023-05-20T01:22:19.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.119+0000 7f0789d227c0 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:72 9.1f 9:ff5b34d6:::1.00000000:head [omap-set-vals in=507b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x556acea89500 con 0x556acdf2e800 2023-05-20T01:22:19.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.120+0000 7f07874cf700 1 -- 172.21.15.16:0/3037775639 <== osd.1 v2:172.21.15.16:6808/3860264220 12 ==== osd_op_reply(72 1.00000000 [omap-set-vals] v71'13 uv13 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x556acdf2d440 con 0x556acdf2e800 2023-05-20T01:22:19.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.120+0000 7f0789d227c0 20 datascan.inject_linkage: Injected dentry 0x0x1/subdir pointing to 0x0x10000000000 2023-05-20T01:22:19.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.120+0000 7f0789d227c0 10 datascan.scan_links: handling injected_ino 0x10000000001 2023-05-20T01:22:19.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.120+0000 7f0789d227c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:0 2023-05-20T01:22:19.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.120+0000 7f0789d227c0 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:73 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x556acea89880 con 0x556acdf4e800 2023-05-20T01:22:19.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.120+0000 7f07884d1700 1 -- 172.21.15.16:0/3037775639 <== osd.4 v2:172.21.15.132:6817/506317223 7 ==== osd_op_reply(73 10000000000.00000000 [omap-get-vals-by-keys out=502b] v0'0 uv41 ondisk = 0) v8 ==== 164+0+502 (crc 0 0 0) 0x556acdf2d680 con 0x556acdf4e800 2023-05-20T01:22:19.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.120+0000 7f0789d227c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=0 frag=*, r=0 2023-05-20T01:22:19.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.120+0000 7f0789d227c0 20 datascan.scan_links: first is now 2 2023-05-20T01:22:19.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.121+0000 7f0789d227c0 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:74 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x556acea89c00 con 0x556acdf4e800 2023-05-20T01:22:19.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.122+0000 7f07884d1700 1 -- 172.21.15.16:0/3037775639 <== osd.4 v2:172.21.15.132:6817/506317223 8 ==== osd_op_reply(74 10000000000.00000000 [omap-set-vals] v71'42 uv42 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x556acdf2d680 con 0x556acdf4e800 2023-05-20T01:22:19.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.122+0000 7f0789d227c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/0 pointing to 0x0x10000000001 2023-05-20T01:22:19.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.122+0000 7f0789d227c0 10 datascan.scan_links: handling injected_ino 0x100000001f6 2023-05-20T01:22:19.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.122+0000 7f0789d227c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:1 2023-05-20T01:22:19.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.122+0000 7f0789d227c0 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:75 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x556acea88e00 con 0x556acdf4e800 2023-05-20T01:22:19.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.122+0000 7f07884d1700 1 -- 172.21.15.16:0/3037775639 <== osd.4 v2:172.21.15.132:6817/506317223 9 ==== osd_op_reply(75 10000000000.00000000 [omap-get-vals-by-keys out=502b] v0'0 uv42 ondisk = 0) v8 ==== 164+0+502 (crc 0 0 0) 0x556acdf2d680 con 0x556acdf4e800 2023-05-20T01:22:19.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.122+0000 7f0789d227c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=1 frag=*, r=0 2023-05-20T01:22:19.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.122+0000 7f0789d227c0 20 datascan.scan_links: first is now 2 2023-05-20T01:22:19.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.122+0000 7f0789d227c0 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:76 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x556acea89180 con 0x556acdf4e800 2023-05-20T01:22:19.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.123+0000 7f07884d1700 1 -- 172.21.15.16:0/3037775639 <== osd.4 v2:172.21.15.132:6817/506317223 10 ==== osd_op_reply(76 10000000000.00000000 [omap-set-vals] v71'43 uv43 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x556acdf2d680 con 0x556acdf4e800 2023-05-20T01:22:19.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.123+0000 7f0789d227c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/1 pointing to 0x0x100000001f6 2023-05-20T01:22:19.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.123+0000 7f0789d227c0 10 datascan.scan_links: handling injected_ino 0x100000001f7 2023-05-20T01:22:19.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.123+0000 7f0789d227c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:2 2023-05-20T01:22:19.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.123+0000 7f0789d227c0 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:77 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x556acea89880 con 0x556acdf4e800 2023-05-20T01:22:19.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.124+0000 7f07884d1700 1 -- 172.21.15.16:0/3037775639 <== osd.4 v2:172.21.15.132:6817/506317223 11 ==== osd_op_reply(77 10000000000.00000000 [omap-get-vals-by-keys out=502b] v0'0 uv43 ondisk = 0) v8 ==== 164+0+502 (crc 0 0 0) 0x556acdf2d680 con 0x556acdf4e800 2023-05-20T01:22:19.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.124+0000 7f0789d227c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=2 frag=*, r=0 2023-05-20T01:22:19.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.124+0000 7f0789d227c0 20 datascan.scan_links: first is now 2 2023-05-20T01:22:19.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.124+0000 7f0789d227c0 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:78 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x556acea89c00 con 0x556acdf4e800 2023-05-20T01:22:19.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.125+0000 7f07884d1700 1 -- 172.21.15.16:0/3037775639 <== osd.4 v2:172.21.15.132:6817/506317223 12 ==== osd_op_reply(78 10000000000.00000000 [omap-set-vals] v71'44 uv44 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x556acdf2d680 con 0x556acdf4e800 2023-05-20T01:22:19.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.125+0000 7f0789d227c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/2 pointing to 0x0x100000001f7 2023-05-20T01:22:19.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.125+0000 7f0789d227c0 10 datascan.scan_links: handling injected_ino 0x100000001f8 2023-05-20T01:22:19.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.125+0000 7f0789d227c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:3 2023-05-20T01:22:19.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.125+0000 7f0789d227c0 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:79 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x556ace0a2000 con 0x556acdf4e800 2023-05-20T01:22:19.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.125+0000 7f07884d1700 1 -- 172.21.15.16:0/3037775639 <== osd.4 v2:172.21.15.132:6817/506317223 13 ==== osd_op_reply(79 10000000000.00000000 [omap-get-vals-by-keys out=502b] v0'0 uv44 ondisk = 0) v8 ==== 164+0+502 (crc 0 0 0) 0x556acdf2d680 con 0x556acdf4e800 2023-05-20T01:22:19.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.125+0000 7f0789d227c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=3 frag=*, r=0 2023-05-20T01:22:19.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.125+0000 7f0789d227c0 20 datascan.scan_links: first is now 2 2023-05-20T01:22:19.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.125+0000 7f0789d227c0 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:80 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x556ace0a2380 con 0x556acdf4e800 2023-05-20T01:22:19.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.127+0000 7f07884d1700 1 -- 172.21.15.16:0/3037775639 <== osd.4 v2:172.21.15.132:6817/506317223 14 ==== osd_op_reply(80 10000000000.00000000 [omap-set-vals] v71'45 uv45 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x556acdf2d680 con 0x556acdf4e800 2023-05-20T01:22:19.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.127+0000 7f0789d227c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/3 pointing to 0x0x100000001f8 2023-05-20T01:22:19.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.127+0000 7f0789d227c0 10 datascan.scan_links: handling injected_ino 0x100000001f9 2023-05-20T01:22:19.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.127+0000 7f0789d227c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:4 2023-05-20T01:22:19.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.127+0000 7f0789d227c0 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:81 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x556ace0a2700 con 0x556acdf4e800 2023-05-20T01:22:19.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.127+0000 7f07884d1700 1 -- 172.21.15.16:0/3037775639 <== osd.4 v2:172.21.15.132:6817/506317223 15 ==== osd_op_reply(81 10000000000.00000000 [omap-get-vals-by-keys out=502b] v0'0 uv45 ondisk = 0) v8 ==== 164+0+502 (crc 0 0 0) 0x556acdf2d680 con 0x556acdf4e800 2023-05-20T01:22:19.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.127+0000 7f0789d227c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=4 frag=*, r=0 2023-05-20T01:22:19.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.127+0000 7f0789d227c0 20 datascan.scan_links: first is now 2 2023-05-20T01:22:19.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.127+0000 7f0789d227c0 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:82 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x556ace0a2a80 con 0x556acdf4e800 2023-05-20T01:22:19.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.128+0000 7f07884d1700 1 -- 172.21.15.16:0/3037775639 <== osd.4 v2:172.21.15.132:6817/506317223 16 ==== osd_op_reply(82 10000000000.00000000 [omap-set-vals] v71'46 uv46 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x556acdf2d680 con 0x556acdf4e800 2023-05-20T01:22:19.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.128+0000 7f0789d227c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/4 pointing to 0x0x100000001f9 2023-05-20T01:22:19.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.128+0000 7f0789d227c0 10 datascan.scan_links: handling injected_ino 0x100000001fa 2023-05-20T01:22:19.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.128+0000 7f0789d227c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:5 2023-05-20T01:22:19.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.128+0000 7f0789d227c0 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:83 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x556ace0a2e00 con 0x556acdf4e800 2023-05-20T01:22:19.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.128+0000 7f07884d1700 1 -- 172.21.15.16:0/3037775639 <== osd.4 v2:172.21.15.132:6817/506317223 17 ==== osd_op_reply(83 10000000000.00000000 [omap-get-vals-by-keys out=502b] v0'0 uv46 ondisk = 0) v8 ==== 164+0+502 (crc 0 0 0) 0x556acdf2d680 con 0x556acdf4e800 2023-05-20T01:22:19.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.129+0000 7f0789d227c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=5 frag=*, r=0 2023-05-20T01:22:19.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.129+0000 7f0789d227c0 20 datascan.scan_links: first is now 2 2023-05-20T01:22:19.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.129+0000 7f0789d227c0 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:84 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x556ace0a3180 con 0x556acdf4e800 2023-05-20T01:22:19.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.130+0000 7f07884d1700 1 -- 172.21.15.16:0/3037775639 <== osd.4 v2:172.21.15.132:6817/506317223 18 ==== osd_op_reply(84 10000000000.00000000 [omap-set-vals] v71'47 uv47 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x556acdf2d680 con 0x556acdf4e800 2023-05-20T01:22:19.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.130+0000 7f0789d227c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/5 pointing to 0x0x100000001fa 2023-05-20T01:22:19.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.130+0000 7f0789d227c0 10 datascan.scan_links: handling injected_ino 0x100000001fb 2023-05-20T01:22:19.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.130+0000 7f0789d227c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:6 2023-05-20T01:22:19.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.130+0000 7f0789d227c0 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:85 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x556ace0a3500 con 0x556acdf4e800 2023-05-20T01:22:19.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.130+0000 7f07884d1700 1 -- 172.21.15.16:0/3037775639 <== osd.4 v2:172.21.15.132:6817/506317223 19 ==== osd_op_reply(85 10000000000.00000000 [omap-get-vals-by-keys out=502b] v0'0 uv47 ondisk = 0) v8 ==== 164+0+502 (crc 0 0 0) 0x556acdf2d680 con 0x556acdf4e800 2023-05-20T01:22:19.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.130+0000 7f0789d227c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=6 frag=*, r=0 2023-05-20T01:22:19.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.130+0000 7f0789d227c0 20 datascan.scan_links: first is now 2 2023-05-20T01:22:19.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.130+0000 7f0789d227c0 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:86 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x556acea88e00 con 0x556acdf4e800 2023-05-20T01:22:19.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.131+0000 7f07884d1700 1 -- 172.21.15.16:0/3037775639 <== osd.4 v2:172.21.15.132:6817/506317223 20 ==== osd_op_reply(86 10000000000.00000000 [omap-set-vals] v71'48 uv48 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x556acdf2d680 con 0x556acdf4e800 2023-05-20T01:22:19.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.131+0000 7f0789d227c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/6 pointing to 0x0x100000001fb 2023-05-20T01:22:19.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.131+0000 7f0789d227c0 10 datascan.scan_links: handling injected_ino 0x100000001fc 2023-05-20T01:22:19.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.131+0000 7f0789d227c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:7 2023-05-20T01:22:19.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.131+0000 7f0789d227c0 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:87 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x556acea89180 con 0x556acdf4e800 2023-05-20T01:22:19.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.132+0000 7f07884d1700 1 -- 172.21.15.16:0/3037775639 <== osd.4 v2:172.21.15.132:6817/506317223 21 ==== osd_op_reply(87 10000000000.00000000 [omap-get-vals-by-keys out=502b] v0'0 uv48 ondisk = 0) v8 ==== 164+0+502 (crc 0 0 0) 0x556acdf2d680 con 0x556acdf4e800 2023-05-20T01:22:19.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.132+0000 7f0789d227c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=7 frag=*, r=0 2023-05-20T01:22:19.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.132+0000 7f0789d227c0 20 datascan.scan_links: first is now 2 2023-05-20T01:22:19.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.132+0000 7f0789d227c0 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:88 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x556acea89880 con 0x556acdf4e800 2023-05-20T01:22:19.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.133+0000 7f07884d1700 1 -- 172.21.15.16:0/3037775639 <== osd.4 v2:172.21.15.132:6817/506317223 22 ==== osd_op_reply(88 10000000000.00000000 [omap-set-vals] v71'49 uv49 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x556acdf2d680 con 0x556acdf4e800 2023-05-20T01:22:19.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.133+0000 7f0789d227c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/7 pointing to 0x0x100000001fc 2023-05-20T01:22:19.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.133+0000 7f0789d227c0 10 datascan.scan_links: handling injected_ino 0x100000001fd 2023-05-20T01:22:19.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.133+0000 7f0789d227c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:8 2023-05-20T01:22:19.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.133+0000 7f0789d227c0 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:89 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x556acea89c00 con 0x556acdf4e800 2023-05-20T01:22:19.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.133+0000 7f07884d1700 1 -- 172.21.15.16:0/3037775639 <== osd.4 v2:172.21.15.132:6817/506317223 23 ==== osd_op_reply(89 10000000000.00000000 [omap-get-vals-by-keys out=502b] v0'0 uv49 ondisk = 0) v8 ==== 164+0+502 (crc 0 0 0) 0x556acdf2d680 con 0x556acdf4e800 2023-05-20T01:22:19.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.133+0000 7f0789d227c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=8 frag=*, r=0 2023-05-20T01:22:19.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.133+0000 7f0789d227c0 20 datascan.scan_links: first is now 2 2023-05-20T01:22:19.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.133+0000 7f0789d227c0 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:90 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x556ace0a2000 con 0x556acdf4e800 2023-05-20T01:22:19.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.134+0000 7f07884d1700 1 -- 172.21.15.16:0/3037775639 <== osd.4 v2:172.21.15.132:6817/506317223 24 ==== osd_op_reply(90 10000000000.00000000 [omap-set-vals] v71'50 uv50 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x556acdf2d680 con 0x556acdf4e800 2023-05-20T01:22:19.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.134+0000 7f0789d227c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/8 pointing to 0x0x100000001fd 2023-05-20T01:22:19.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.134+0000 7f0789d227c0 10 datascan.scan_links: handling injected_ino 0x100000001fe 2023-05-20T01:22:19.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.134+0000 7f0789d227c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:9 2023-05-20T01:22:19.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.134+0000 7f0789d227c0 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:91 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x556ace0a2380 con 0x556acdf4e800 2023-05-20T01:22:19.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.135+0000 7f07884d1700 1 -- 172.21.15.16:0/3037775639 <== osd.4 v2:172.21.15.132:6817/506317223 25 ==== osd_op_reply(91 10000000000.00000000 [omap-get-vals-by-keys out=502b] v0'0 uv50 ondisk = 0) v8 ==== 164+0+502 (crc 0 0 0) 0x556acdf2d680 con 0x556acdf4e800 2023-05-20T01:22:19.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.135+0000 7f0789d227c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=9 frag=*, r=0 2023-05-20T01:22:19.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.135+0000 7f0789d227c0 20 datascan.scan_links: first is now 2 2023-05-20T01:22:19.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.135+0000 7f0789d227c0 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:92 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x556ace0a2700 con 0x556acdf4e800 2023-05-20T01:22:19.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.136+0000 7f07884d1700 1 -- 172.21.15.16:0/3037775639 <== osd.4 v2:172.21.15.132:6817/506317223 26 ==== osd_op_reply(92 10000000000.00000000 [omap-set-vals] v71'51 uv51 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x556acdf2d680 con 0x556acdf4e800 2023-05-20T01:22:19.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.136+0000 7f0789d227c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/9 pointing to 0x0x100000001fe 2023-05-20T01:22:19.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.136+0000 7f0789d227c0 10 datascan.scan_links: handling injected_ino 0x100000001ff 2023-05-20T01:22:19.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.136+0000 7f0789d227c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:10 2023-05-20T01:22:19.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.136+0000 7f0789d227c0 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:93 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x556ace0a2a80 con 0x556acdf4e800 2023-05-20T01:22:19.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.136+0000 7f07884d1700 1 -- 172.21.15.16:0/3037775639 <== osd.4 v2:172.21.15.132:6817/506317223 27 ==== osd_op_reply(93 10000000000.00000000 [omap-get-vals-by-keys out=503b] v0'0 uv51 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x556acdf2d680 con 0x556acdf4e800 2023-05-20T01:22:19.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.136+0000 7f0789d227c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=10 frag=*, r=0 2023-05-20T01:22:19.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.136+0000 7f0789d227c0 20 datascan.scan_links: first is now 2 2023-05-20T01:22:19.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.136+0000 7f0789d227c0 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:94 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x556ace0a2e00 con 0x556acdf4e800 2023-05-20T01:22:19.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.138+0000 7f07884d1700 1 -- 172.21.15.16:0/3037775639 <== osd.4 v2:172.21.15.132:6817/506317223 28 ==== osd_op_reply(94 10000000000.00000000 [omap-set-vals] v71'52 uv52 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x556acdf2d680 con 0x556acdf4e800 2023-05-20T01:22:19.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.138+0000 7f0789d227c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/10 pointing to 0x0x100000001ff 2023-05-20T01:22:19.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.138+0000 7f0789d227c0 10 datascan.scan_links: handling injected_ino 0x10000000200 2023-05-20T01:22:19.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.138+0000 7f0789d227c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:11 2023-05-20T01:22:19.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.138+0000 7f0789d227c0 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:95 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x556ace0a3180 con 0x556acdf4e800 2023-05-20T01:22:19.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.139+0000 7f07884d1700 1 -- 172.21.15.16:0/3037775639 <== osd.4 v2:172.21.15.132:6817/506317223 29 ==== osd_op_reply(95 10000000000.00000000 [omap-get-vals-by-keys out=503b] v0'0 uv52 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x556acdf2d680 con 0x556acdf4e800 2023-05-20T01:22:19.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.139+0000 7f0789d227c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=11 frag=*, r=0 2023-05-20T01:22:19.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.139+0000 7f0789d227c0 20 datascan.scan_links: first is now 2 2023-05-20T01:22:19.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.139+0000 7f0789d227c0 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:96 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x556ace0a3500 con 0x556acdf4e800 2023-05-20T01:22:19.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.141+0000 7f07884d1700 1 -- 172.21.15.16:0/3037775639 <== osd.4 v2:172.21.15.132:6817/506317223 30 ==== osd_op_reply(96 10000000000.00000000 [omap-set-vals] v71'53 uv53 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x556acdf2d680 con 0x556acdf4e800 2023-05-20T01:22:19.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.141+0000 7f0789d227c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/11 pointing to 0x0x10000000200 2023-05-20T01:22:19.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.141+0000 7f0789d227c0 10 datascan.scan_links: handling injected_ino 0x10000000201 2023-05-20T01:22:19.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.141+0000 7f0789d227c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:12 2023-05-20T01:22:19.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.141+0000 7f0789d227c0 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:97 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x556ace0a3880 con 0x556acdf4e800 2023-05-20T01:22:19.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.142+0000 7f07884d1700 1 -- 172.21.15.16:0/3037775639 <== osd.4 v2:172.21.15.132:6817/506317223 31 ==== osd_op_reply(97 10000000000.00000000 [omap-get-vals-by-keys out=503b] v0'0 uv53 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x556acdf2d680 con 0x556acdf4e800 2023-05-20T01:22:19.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.142+0000 7f0789d227c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=12 frag=*, r=0 2023-05-20T01:22:19.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.142+0000 7f0789d227c0 20 datascan.scan_links: first is now 2 2023-05-20T01:22:19.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.142+0000 7f0789d227c0 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:98 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x556ace0a3c00 con 0x556acdf4e800 2023-05-20T01:22:19.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.144+0000 7f07884d1700 1 -- 172.21.15.16:0/3037775639 <== osd.4 v2:172.21.15.132:6817/506317223 32 ==== osd_op_reply(98 10000000000.00000000 [omap-set-vals] v71'54 uv54 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x556acdf2d680 con 0x556acdf4e800 2023-05-20T01:22:19.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.144+0000 7f0789d227c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/12 pointing to 0x0x10000000201 2023-05-20T01:22:19.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.144+0000 7f0789d227c0 10 datascan.scan_links: handling injected_ino 0x10000000202 2023-05-20T01:22:19.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.144+0000 7f0789d227c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:13 2023-05-20T01:22:19.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.144+0000 7f0789d227c0 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:99 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x556ace0a2000 con 0x556acdf4e800 2023-05-20T01:22:19.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.144+0000 7f07884d1700 1 -- 172.21.15.16:0/3037775639 <== osd.4 v2:172.21.15.132:6817/506317223 33 ==== osd_op_reply(99 10000000000.00000000 [omap-get-vals-by-keys out=503b] v0'0 uv54 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x556acdf2d680 con 0x556acdf4e800 2023-05-20T01:22:19.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.144+0000 7f0789d227c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=13 frag=*, r=0 2023-05-20T01:22:19.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.144+0000 7f0789d227c0 20 datascan.scan_links: first is now 2 2023-05-20T01:22:19.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.144+0000 7f0789d227c0 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:100 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x556ace0a2380 con 0x556acdf4e800 2023-05-20T01:22:19.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.147+0000 7f07884d1700 1 -- 172.21.15.16:0/3037775639 <== osd.4 v2:172.21.15.132:6817/506317223 34 ==== osd_op_reply(100 10000000000.00000000 [omap-set-vals] v71'55 uv55 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x556acdf2d680 con 0x556acdf4e800 2023-05-20T01:22:19.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.147+0000 7f0789d227c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/13 pointing to 0x0x10000000202 2023-05-20T01:22:19.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.147+0000 7f0789d227c0 10 datascan.scan_links: handling injected_ino 0x10000000203 2023-05-20T01:22:19.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.147+0000 7f0789d227c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:14 2023-05-20T01:22:19.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.147+0000 7f0789d227c0 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:101 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x556ace0a2700 con 0x556acdf4e800 2023-05-20T01:22:19.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.147+0000 7f07884d1700 1 -- 172.21.15.16:0/3037775639 <== osd.4 v2:172.21.15.132:6817/506317223 35 ==== osd_op_reply(101 10000000000.00000000 [omap-get-vals-by-keys out=503b] v0'0 uv55 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x556acdf2d680 con 0x556acdf4e800 2023-05-20T01:22:19.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.147+0000 7f0789d227c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=14 frag=*, r=0 2023-05-20T01:22:19.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.147+0000 7f0789d227c0 20 datascan.scan_links: first is now 2 2023-05-20T01:22:19.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.147+0000 7f0789d227c0 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:102 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x556acea88e00 con 0x556acdf4e800 2023-05-20T01:22:19.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.150+0000 7f07884d1700 1 -- 172.21.15.16:0/3037775639 <== osd.4 v2:172.21.15.132:6817/506317223 36 ==== osd_op_reply(102 10000000000.00000000 [omap-set-vals] v71'56 uv56 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x556acdf2d680 con 0x556acdf4e800 2023-05-20T01:22:19.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.150+0000 7f0789d227c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/14 pointing to 0x0x10000000203 2023-05-20T01:22:19.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.150+0000 7f0789d227c0 10 datascan.scan_links: handling injected_ino 0x10000000204 2023-05-20T01:22:19.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.150+0000 7f0789d227c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:15 2023-05-20T01:22:19.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.150+0000 7f0789d227c0 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:103 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x556acea89180 con 0x556acdf4e800 2023-05-20T01:22:19.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.150+0000 7f07884d1700 1 -- 172.21.15.16:0/3037775639 <== osd.4 v2:172.21.15.132:6817/506317223 37 ==== osd_op_reply(103 10000000000.00000000 [omap-get-vals-by-keys out=503b] v0'0 uv56 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x556acdf2d680 con 0x556acdf4e800 2023-05-20T01:22:19.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.150+0000 7f0789d227c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=15 frag=*, r=0 2023-05-20T01:22:19.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.150+0000 7f0789d227c0 20 datascan.scan_links: first is now 2 2023-05-20T01:22:19.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.150+0000 7f0789d227c0 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:104 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x556acea89880 con 0x556acdf4e800 2023-05-20T01:22:19.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.153+0000 7f07884d1700 1 -- 172.21.15.16:0/3037775639 <== osd.4 v2:172.21.15.132:6817/506317223 38 ==== osd_op_reply(104 10000000000.00000000 [omap-set-vals] v71'57 uv57 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x556acdf2d680 con 0x556acdf4e800 2023-05-20T01:22:19.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.153+0000 7f0789d227c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/15 pointing to 0x0x10000000204 2023-05-20T01:22:19.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.153+0000 7f0789d227c0 10 datascan.scan_links: handling injected_ino 0x10000000205 2023-05-20T01:22:19.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.153+0000 7f0789d227c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:16 2023-05-20T01:22:19.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.153+0000 7f0789d227c0 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:105 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x556acea89c00 con 0x556acdf4e800 2023-05-20T01:22:19.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.153+0000 7f07884d1700 1 -- 172.21.15.16:0/3037775639 <== osd.4 v2:172.21.15.132:6817/506317223 39 ==== osd_op_reply(105 10000000000.00000000 [omap-get-vals-by-keys out=503b] v0'0 uv57 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x556acdf2d680 con 0x556acdf4e800 2023-05-20T01:22:19.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.153+0000 7f0789d227c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=16 frag=*, r=0 2023-05-20T01:22:19.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.153+0000 7f0789d227c0 20 datascan.scan_links: first is now 2 2023-05-20T01:22:19.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.153+0000 7f0789d227c0 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:106 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x556ace0a2a80 con 0x556acdf4e800 2023-05-20T01:22:19.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.155+0000 7f07884d1700 1 -- 172.21.15.16:0/3037775639 <== osd.4 v2:172.21.15.132:6817/506317223 40 ==== osd_op_reply(106 10000000000.00000000 [omap-set-vals] v71'58 uv58 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x556acdf2d680 con 0x556acdf4e800 2023-05-20T01:22:19.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.155+0000 7f0789d227c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/16 pointing to 0x0x10000000205 2023-05-20T01:22:19.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.155+0000 7f0789d227c0 10 datascan.scan_links: handling injected_ino 0x10000000206 2023-05-20T01:22:19.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.155+0000 7f0789d227c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:17 2023-05-20T01:22:19.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.155+0000 7f0789d227c0 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:107 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x556ace0a2e00 con 0x556acdf4e800 2023-05-20T01:22:19.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.156+0000 7f07884d1700 1 -- 172.21.15.16:0/3037775639 <== osd.4 v2:172.21.15.132:6817/506317223 41 ==== osd_op_reply(107 10000000000.00000000 [omap-get-vals-by-keys out=503b] v0'0 uv58 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x556acdf2d680 con 0x556acdf4e800 2023-05-20T01:22:19.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.156+0000 7f0789d227c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=17 frag=*, r=0 2023-05-20T01:22:19.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.156+0000 7f0789d227c0 20 datascan.scan_links: first is now 2 2023-05-20T01:22:19.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.156+0000 7f0789d227c0 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:108 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x556ace0a3180 con 0x556acdf4e800 2023-05-20T01:22:19.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.158+0000 7f07884d1700 1 -- 172.21.15.16:0/3037775639 <== osd.4 v2:172.21.15.132:6817/506317223 42 ==== osd_op_reply(108 10000000000.00000000 [omap-set-vals] v71'59 uv59 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x556acdf2d680 con 0x556acdf4e800 2023-05-20T01:22:19.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.158+0000 7f0789d227c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/17 pointing to 0x0x10000000206 2023-05-20T01:22:19.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.158+0000 7f0789d227c0 10 datascan.scan_links: handling injected_ino 0x10000000207 2023-05-20T01:22:19.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.158+0000 7f0789d227c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:18 2023-05-20T01:22:19.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.158+0000 7f0789d227c0 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:109 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x556ace0a3500 con 0x556acdf4e800 2023-05-20T01:22:19.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.158+0000 7f07884d1700 1 -- 172.21.15.16:0/3037775639 <== osd.4 v2:172.21.15.132:6817/506317223 43 ==== osd_op_reply(109 10000000000.00000000 [omap-get-vals-by-keys out=503b] v0'0 uv59 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x556acdf2d680 con 0x556acdf4e800 2023-05-20T01:22:19.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.158+0000 7f0789d227c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=18 frag=*, r=0 2023-05-20T01:22:19.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.158+0000 7f0789d227c0 20 datascan.scan_links: first is now 2 2023-05-20T01:22:19.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.158+0000 7f0789d227c0 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:110 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x556ace0a3880 con 0x556acdf4e800 2023-05-20T01:22:19.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.160+0000 7f07884d1700 1 -- 172.21.15.16:0/3037775639 <== osd.4 v2:172.21.15.132:6817/506317223 44 ==== osd_op_reply(110 10000000000.00000000 [omap-set-vals] v71'60 uv60 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x556acdf2d680 con 0x556acdf4e800 2023-05-20T01:22:19.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.161+0000 7f0789d227c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/18 pointing to 0x0x10000000207 2023-05-20T01:22:19.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.161+0000 7f0789d227c0 10 datascan.scan_links: handling injected_ino 0x10000000208 2023-05-20T01:22:19.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.161+0000 7f0789d227c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:19 2023-05-20T01:22:19.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.161+0000 7f0789d227c0 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:111 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x556ace0a3c00 con 0x556acdf4e800 2023-05-20T01:22:19.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.161+0000 7f07884d1700 1 -- 172.21.15.16:0/3037775639 <== osd.4 v2:172.21.15.132:6817/506317223 45 ==== osd_op_reply(111 10000000000.00000000 [omap-get-vals-by-keys out=503b] v0'0 uv60 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x556acdf2d680 con 0x556acdf4e800 2023-05-20T01:22:19.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.161+0000 7f0789d227c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=19 frag=*, r=0 2023-05-20T01:22:19.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.161+0000 7f0789d227c0 20 datascan.scan_links: first is now 2 2023-05-20T01:22:19.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.161+0000 7f0789d227c0 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:112 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x556ace0a2000 con 0x556acdf4e800 2023-05-20T01:22:19.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.164+0000 7f07884d1700 1 -- 172.21.15.16:0/3037775639 <== osd.4 v2:172.21.15.132:6817/506317223 46 ==== osd_op_reply(112 10000000000.00000000 [omap-set-vals] v71'61 uv61 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x556acdf2d680 con 0x556acdf4e800 2023-05-20T01:22:19.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.164+0000 7f0789d227c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/19 pointing to 0x0x10000000208 2023-05-20T01:22:19.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.164+0000 7f0789d227c0 10 datascan.scan_links: handling injected_ino 0x10000000209 2023-05-20T01:22:19.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.164+0000 7f0789d227c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:20 2023-05-20T01:22:19.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.164+0000 7f0789d227c0 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:113 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x556ace0a2380 con 0x556acdf4e800 2023-05-20T01:22:19.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.164+0000 7f07884d1700 1 -- 172.21.15.16:0/3037775639 <== osd.4 v2:172.21.15.132:6817/506317223 47 ==== osd_op_reply(113 10000000000.00000000 [omap-get-vals-by-keys out=503b] v0'0 uv61 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x556acdf2d680 con 0x556acdf4e800 2023-05-20T01:22:19.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.164+0000 7f0789d227c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=20 frag=*, r=0 2023-05-20T01:22:19.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.164+0000 7f0789d227c0 20 datascan.scan_links: first is now 2 2023-05-20T01:22:19.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.164+0000 7f0789d227c0 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:114 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x556ace0a2700 con 0x556acdf4e800 2023-05-20T01:22:19.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.166+0000 7f07884d1700 1 -- 172.21.15.16:0/3037775639 <== osd.4 v2:172.21.15.132:6817/506317223 48 ==== osd_op_reply(114 10000000000.00000000 [omap-set-vals] v71'62 uv62 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x556acdf2d680 con 0x556acdf4e800 2023-05-20T01:22:19.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.166+0000 7f0789d227c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/20 pointing to 0x0x10000000209 2023-05-20T01:22:19.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.166+0000 7f0789d227c0 10 datascan.scan_links: handling injected_ino 0x1000000020a 2023-05-20T01:22:19.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.166+0000 7f0789d227c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:21 2023-05-20T01:22:19.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.166+0000 7f0789d227c0 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:115 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x556ace0a2a80 con 0x556acdf4e800 2023-05-20T01:22:19.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.167+0000 7f07884d1700 1 -- 172.21.15.16:0/3037775639 <== osd.4 v2:172.21.15.132:6817/506317223 49 ==== osd_op_reply(115 10000000000.00000000 [omap-get-vals-by-keys out=503b] v0'0 uv62 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x556acdf2d680 con 0x556acdf4e800 2023-05-20T01:22:19.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.167+0000 7f0789d227c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=21 frag=*, r=0 2023-05-20T01:22:19.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.167+0000 7f0789d227c0 20 datascan.scan_links: first is now 2 2023-05-20T01:22:19.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.167+0000 7f0789d227c0 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:116 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x556ace0a2e00 con 0x556acdf4e800 2023-05-20T01:22:19.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.169+0000 7f07884d1700 1 -- 172.21.15.16:0/3037775639 <== osd.4 v2:172.21.15.132:6817/506317223 50 ==== osd_op_reply(116 10000000000.00000000 [omap-set-vals] v71'63 uv63 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x556acdf2d680 con 0x556acdf4e800 2023-05-20T01:22:19.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.169+0000 7f0789d227c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/21 pointing to 0x0x1000000020a 2023-05-20T01:22:19.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.169+0000 7f0789d227c0 10 datascan.scan_links: handling injected_ino 0x1000000020b 2023-05-20T01:22:19.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.169+0000 7f0789d227c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:22 2023-05-20T01:22:19.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.169+0000 7f0789d227c0 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:117 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x556ace0a3180 con 0x556acdf4e800 2023-05-20T01:22:19.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.169+0000 7f07884d1700 1 -- 172.21.15.16:0/3037775639 <== osd.4 v2:172.21.15.132:6817/506317223 51 ==== osd_op_reply(117 10000000000.00000000 [omap-get-vals-by-keys out=503b] v0'0 uv63 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x556acdf2d680 con 0x556acdf4e800 2023-05-20T01:22:19.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.169+0000 7f0789d227c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=22 frag=*, r=0 2023-05-20T01:22:19.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.169+0000 7f0789d227c0 20 datascan.scan_links: first is now 2 2023-05-20T01:22:19.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.169+0000 7f0789d227c0 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:118 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x556ace0a3500 con 0x556acdf4e800 2023-05-20T01:22:19.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.170+0000 7f07884d1700 1 -- 172.21.15.16:0/3037775639 <== osd.4 v2:172.21.15.132:6817/506317223 52 ==== osd_op_reply(118 10000000000.00000000 [omap-set-vals] v71'64 uv64 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x556acdf2d680 con 0x556acdf4e800 2023-05-20T01:22:19.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.170+0000 7f0789d227c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/22 pointing to 0x0x1000000020b 2023-05-20T01:22:19.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.170+0000 7f0789d227c0 10 datascan.scan_links: handling injected_ino 0x1000000020c 2023-05-20T01:22:19.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.170+0000 7f0789d227c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:23 2023-05-20T01:22:19.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.170+0000 7f0789d227c0 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:119 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x556acea88e00 con 0x556acdf4e800 2023-05-20T01:22:19.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.170+0000 7f07884d1700 1 -- 172.21.15.16:0/3037775639 <== osd.4 v2:172.21.15.132:6817/506317223 53 ==== osd_op_reply(119 10000000000.00000000 [omap-get-vals-by-keys out=503b] v0'0 uv64 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x556acdf2d680 con 0x556acdf4e800 2023-05-20T01:22:19.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.170+0000 7f0789d227c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=23 frag=*, r=0 2023-05-20T01:22:19.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.170+0000 7f0789d227c0 20 datascan.scan_links: first is now 2 2023-05-20T01:22:19.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.170+0000 7f0789d227c0 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:120 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x556acea89180 con 0x556acdf4e800 2023-05-20T01:22:19.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.172+0000 7f07884d1700 1 -- 172.21.15.16:0/3037775639 <== osd.4 v2:172.21.15.132:6817/506317223 54 ==== osd_op_reply(120 10000000000.00000000 [omap-set-vals] v71'65 uv65 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x556acdf2d680 con 0x556acdf4e800 2023-05-20T01:22:19.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.172+0000 7f0789d227c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/23 pointing to 0x0x1000000020c 2023-05-20T01:22:19.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.172+0000 7f0789d227c0 10 datascan.scan_links: handling injected_ino 0x1000000020d 2023-05-20T01:22:19.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.172+0000 7f0789d227c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:24 2023-05-20T01:22:19.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.172+0000 7f0789d227c0 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:121 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x556acea89880 con 0x556acdf4e800 2023-05-20T01:22:19.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.172+0000 7f07884d1700 1 -- 172.21.15.16:0/3037775639 <== osd.4 v2:172.21.15.132:6817/506317223 55 ==== osd_op_reply(121 10000000000.00000000 [omap-get-vals-by-keys out=503b] v0'0 uv65 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x556acdf2d680 con 0x556acdf4e800 2023-05-20T01:22:19.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.172+0000 7f0789d227c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=24 frag=*, r=0 2023-05-20T01:22:19.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.172+0000 7f0789d227c0 20 datascan.scan_links: first is now 2 2023-05-20T01:22:19.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.172+0000 7f0789d227c0 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:122 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x556acea89c00 con 0x556acdf4e800 2023-05-20T01:22:19.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.173+0000 7f07884d1700 1 -- 172.21.15.16:0/3037775639 <== osd.4 v2:172.21.15.132:6817/506317223 56 ==== osd_op_reply(122 10000000000.00000000 [omap-set-vals] v71'66 uv66 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x556acdf2d680 con 0x556acdf4e800 2023-05-20T01:22:19.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.173+0000 7f0789d227c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/24 pointing to 0x0x1000000020d 2023-05-20T01:22:19.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.173+0000 7f0789d227c0 10 datascan.scan_links: updating inotable 2023-05-20T01:22:19.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.173+0000 7f0789d227c0 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:123 9.13 9:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x556acea88e00 con 0x556acdf4f000 2023-05-20T01:22:19.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.174+0000 7f07874cf700 1 -- 172.21.15.16:0/3037775639 <== osd.7 v2:172.21.15.132:6800/466001149 15 ==== osd_op_reply(123 mds0_inotable [read 0~34 out=34b] v0'0 uv6 ondisk = 0) v8 ==== 157+0+34 (crc 0 0 0) 0x556acdf2d440 con 0x556acdf4f000 2023-05-20T01:22:19.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.174+0000 7f0789d227c0 10 mds.0.inotable: skip_inos was [0x10000000000~0x10000000000] 2023-05-20T01:22:19.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.174+0000 7f0789d227c0 10 mds.0.inotable: skip_inos now [0x1000000020e~0xfffffffdf2] 2023-05-20T01:22:19.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.174+0000 7f0789d227c0 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:124 9.13 9:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v8 -- 0x556acea88700 con 0x556acdf4f000 2023-05-20T01:22:19.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.175+0000 7f07874cf700 1 -- 172.21.15.16:0/3037775639 <== osd.7 v2:172.21.15.132:6800/466001149 16 ==== osd_op_reply(124 mds0_inotable [writefull 0~34] v71'8 uv8 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x556acdf2d440 con 0x556acdf4f000 2023-05-20T01:22:19.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.175+0000 7f0789d227c0 10 datascan.scan_links: updating snaptable 2023-05-20T01:22:19.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.175+0000 7f0789d227c0 1 -- 172.21.15.16:0/3037775639 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:125 9.d 9:b50e409b:::mds_snaptable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) v8 -- 0x556acea88a80 con 0x556ace076000 2023-05-20T01:22:19.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.175+0000 7f07884d1700 1 -- 172.21.15.16:0/3037775639 <== osd.2 v2:172.21.15.16:6816/293187030 3 ==== osd_op_reply(125 mds_snaptable [read 0~70 out=70b] v0'0 uv3 ondisk = 0) v8 ==== 157+0+70 (crc 0 0 0) 0x556acdf2d680 con 0x556ace076000 2023-05-20T01:22:19.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.175+0000 7f0789d227c0 1 -- 172.21.15.16:0/3037775639 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x556acdf4f800 msgr2=0x556ace055080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:19.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.175+0000 7f0789d227c0 1 --2- 172.21.15.16:0/3037775639 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x556acdf4f800 0x556ace055080 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:19.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.175+0000 7f0789d227c0 1 -- 172.21.15.16:0/3037775639 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x556acdf2e800 msgr2=0x556acdcba100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:19.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.175+0000 7f0789d227c0 1 --2- 172.21.15.16:0/3037775639 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x556acdf2e800 0x556acdcba100 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:19.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.175+0000 7f0789d227c0 1 -- 172.21.15.16:0/3037775639 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x556ace076000 msgr2=0x556ace056680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:19.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.175+0000 7f0789d227c0 1 --2- 172.21.15.16:0/3037775639 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x556ace076000 0x556ace056680 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:19.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.175+0000 7f0789d227c0 1 -- 172.21.15.16:0/3037775639 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x556acdf4ec00 msgr2=0x556ace054580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:19.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.175+0000 7f0789d227c0 1 --2- 172.21.15.16:0/3037775639 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x556acdf4ec00 0x556ace054580 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:19.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.175+0000 7f0789d227c0 1 -- 172.21.15.16:0/3037775639 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x556acdf4e800 msgr2=0x556ace054000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:19.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.175+0000 7f0789d227c0 1 --2- 172.21.15.16:0/3037775639 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x556acdf4e800 0x556ace054000 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:19.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.175+0000 7f0789d227c0 1 -- 172.21.15.16:0/3037775639 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x556acdf4fc00 msgr2=0x556ace055600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:19.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.175+0000 7f0789d227c0 1 --2- 172.21.15.16:0/3037775639 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x556acdf4fc00 0x556ace055600 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:19.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.175+0000 7f0789d227c0 1 -- 172.21.15.16:0/3037775639 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x556acdf4f400 msgr2=0x556ace054b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:19.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.175+0000 7f0789d227c0 1 --2- 172.21.15.16:0/3037775639 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x556acdf4f400 0x556ace054b00 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:19.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.175+0000 7f0789d227c0 1 -- 172.21.15.16:0/3037775639 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x556acdf4f000 msgr2=0x556ace055b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:19.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.175+0000 7f0789d227c0 1 --2- 172.21.15.16:0/3037775639 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x556acdf4f000 0x556ace055b80 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:19.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.175+0000 7f07874cf700 1 -- 172.21.15.16:0/3037775639 reap_dead start 2023-05-20T01:22:19.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.175+0000 7f07874cf700 1 -- 172.21.15.16:0/3037775639 reap_dead start 2023-05-20T01:22:19.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.175+0000 7f07874cf700 1 -- 172.21.15.16:0/3037775639 reap_dead start 2023-05-20T01:22:19.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.175+0000 7f07874cf700 1 -- 172.21.15.16:0/3037775639 reap_dead start 2023-05-20T01:22:19.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.175+0000 7f0789d227c0 1 -- 172.21.15.16:0/3037775639 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x556acdf4e400 msgr2=0x556acdcbb700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:19.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.175+0000 7f0789d227c0 1 --2- 172.21.15.16:0/3037775639 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x556acdf4e400 0x556acdcbb700 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x556acdd89770 tx=0x556ace9f6cc0 comp rx=0 tx=0).stop 2023-05-20T01:22:19.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.175+0000 7f0789d227c0 1 -- 172.21.15.16:0/3037775639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556acdc1e400 msgr2=0x556acdcbb180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:19.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.176+0000 7f0789d227c0 1 --2- 172.21.15.16:0/3037775639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556acdc1e400 0x556acdcbb180 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x556acdf3b3b0 tx=0x556ace9f6de0 comp rx=0 tx=0).stop 2023-05-20T01:22:19.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.176+0000 7f0789d227c0 1 -- 172.21.15.16:0/3037775639 shutdown_connections 2023-05-20T01:22:19.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.176+0000 7f0789d227c0 1 --2- 172.21.15.16:0/3037775639 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x556acdf4e400 0x556acdcbb700 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:19.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.176+0000 7f0789d227c0 1 --2- 172.21.15.16:0/3037775639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556acdc1e400 0x556acdcbb180 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:19.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.176+0000 7f0789d227c0 1 -- 172.21.15.16:0/3037775639 >> 172.21.15.16:0/3037775639 conn(0x556acdc1fc00 msgr2=0x556acdcfb800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:19.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.176+0000 7f0789d227c0 1 -- 172.21.15.16:0/3037775639 shutdown_connections 2023-05-20T01:22:19.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.176+0000 7f0789d227c0 1 -- 172.21.15.16:0/3037775639 wait complete. 2023-05-20T01:22:19.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.176+0000 7f0789d227c0 1 -- 172.21.15.16:0/3039714127 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556acdc1f000 msgr2=0x556acdcb8580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:19.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.176+0000 7f0789d227c0 1 --2- 172.21.15.16:0/3039714127 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556acdc1f000 0x556acdcb8580 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x556acdd88db0 tx=0x556ace9f6ea0 comp rx=0 tx=0).stop 2023-05-20T01:22:19.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.176+0000 7f0789d227c0 1 -- 172.21.15.16:0/3039714127 shutdown_connections 2023-05-20T01:22:19.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.176+0000 7f0789d227c0 1 --2- 172.21.15.16:0/3039714127 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556acdc1e800 0x556acdcb9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:19.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.176+0000 7f0789d227c0 1 --2- 172.21.15.16:0/3039714127 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556acdc1f000 0x556acdcb8580 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:19.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.176+0000 7f0789d227c0 1 --2- 172.21.15.16:0/3039714127 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556acdc1ec00 0x556acdcb8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:19.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.176+0000 7f0789d227c0 1 -- 172.21.15.16:0/3039714127 >> 172.21.15.16:0/3039714127 conn(0x556acdc1f400 msgr2=0x556acdcfb000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:19.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.176+0000 7f0789d227c0 1 -- 172.21.15.16:0/3039714127 shutdown_connections 2023-05-20T01:22:19.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.176+0000 7f0789d227c0 1 -- 172.21.15.16:0/3039714127 wait complete. 2023-05-20T01:22:19.211 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_links'] in time 0:00:00.167658, result: 2023-05-20T01:22:19.211 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2023-05-20T01:22:19.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.334+0000 7f94d9796700 1 Processor -- start 2023-05-20T01:22:19.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.334+0000 7f94d9796700 1 -- start start 2023-05-20T01:22:19.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.335+0000 7f94d9796700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f94d40cb670 0x7f94d40ce340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:19.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.335+0000 7f94d9796700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94d4159d90 con 0x7f94d41599c0 2023-05-20T01:22:19.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.335+0000 7f94d9796700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94d40cdb80 con 0x7f94d40caaa0 2023-05-20T01:22:19.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.335+0000 7f94d9796700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94d40cdf20 con 0x7f94d40cb670 2023-05-20T01:22:19.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.335+0000 7f94d2ffd700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f94d41599c0 0x7f94d40caf60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:46724/0 (socket says 172.21.15.16:46724) 2023-05-20T01:22:19.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.335+0000 7f94d2ffd700 1 -- 172.21.15.16:0/2722017164 learned_addr learned my addr 172.21.15.16:0/2722017164 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:22:19.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.335+0000 7f94d27fc700 1 --2- 172.21.15.16:0/2722017164 >> v2:172.21.15.132:3300/0 conn(0x7f94d40cb670 0x7f94d40ce340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:19.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.336+0000 7f94d1ffb700 1 -- 172.21.15.16:0/2722017164 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3271292996 0 0) 0x7f94c8001110 con 0x7f94d41599c0 2023-05-20T01:22:19.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.336+0000 7f94d1ffb700 1 -- 172.21.15.16:0/2722017164 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94bc0035b0 con 0x7f94d41599c0 2023-05-20T01:22:19.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.336+0000 7f94d1ffb700 1 -- 172.21.15.16:0/2722017164 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3889163192 0 0) 0x7f94d40cdb80 con 0x7f94d40caaa0 2023-05-20T01:22:19.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.336+0000 7f94d1ffb700 1 -- 172.21.15.16:0/2722017164 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94c8001110 con 0x7f94d40caaa0 2023-05-20T01:22:19.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.336+0000 7f94d27fc700 1 -- 172.21.15.16:0/2722017164 >> v1:172.21.15.132:6789/0 conn(0x7f94d40caaa0 legacy=0x7f94d40ce880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:19.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.336+0000 7f94d27fc700 1 -- 172.21.15.16:0/2722017164 >> v1:172.21.15.16:6789/0 conn(0x7f94d41599c0 legacy=0x7f94d40caf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:19.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.336+0000 7f94d27fc700 1 -- 172.21.15.16:0/2722017164 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94d406eaf0 con 0x7f94d40cb670 2023-05-20T01:22:19.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.336+0000 7f94d27fc700 1 --2- 172.21.15.16:0/2722017164 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f94d40cb670 0x7f94d40ce340 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f94c4009840 tx=0x7f94c4009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13f8521357c80a46 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:19.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.336+0000 7f94d1ffb700 1 -- 172.21.15.16:0/2722017164 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94c4011830 con 0x7f94d40cb670 2023-05-20T01:22:19.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.336+0000 7f94d1ffb700 1 -- 172.21.15.16:0/2722017164 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94c4011990 con 0x7f94d40cb670 2023-05-20T01:22:19.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.336+0000 7f94d1ffb700 1 -- 172.21.15.16:0/2722017164 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94c4011d20 con 0x7f94d40cb670 2023-05-20T01:22:19.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.337+0000 7f94d9796700 1 -- 172.21.15.16:0/2722017164 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f94d40cb670 msgr2=0x7f94d40ce340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:19.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.337+0000 7f94d9796700 1 --2- 172.21.15.16:0/2722017164 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f94d40cb670 0x7f94d40ce340 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f94c4009840 tx=0x7f94c4009b50 comp rx=0 tx=0).stop 2023-05-20T01:22:19.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.337+0000 7f94d9796700 1 -- 172.21.15.16:0/2722017164 shutdown_connections 2023-05-20T01:22:19.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.337+0000 7f94d9796700 1 --2- 172.21.15.16:0/2722017164 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f94d40cb670 0x7f94d40ce340 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:19.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.337+0000 7f94d9796700 1 -- 172.21.15.16:0/2722017164 >> 172.21.15.16:0/2722017164 conn(0x7f94d40d0c30 msgr2=0x7f94d40b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:19.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.337+0000 7f94d9796700 1 -- 172.21.15.16:0/2722017164 shutdown_connections 2023-05-20T01:22:19.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.337+0000 7f94d9796700 1 -- 172.21.15.16:0/2722017164 wait complete. 2023-05-20T01:22:19.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.338+0000 7f94d9796700 1 Processor -- start 2023-05-20T01:22:19.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.338+0000 7f94d9796700 1 -- start start 2023-05-20T01:22:19.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.339+0000 7f94d9796700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f94d40caaa0 0x7f94d40c4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:19.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.339+0000 7f94d9796700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f94d40cb670 0x7f94d40c9290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:19.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.339+0000 7f94d9796700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94d41599c0 0x7f94d40c97d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:19.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.339+0000 7f94d9796700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94d4156df0 con 0x7f94d41599c0 2023-05-20T01:22:19.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.339+0000 7f94d9796700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94d41f0360 con 0x7f94d40caaa0 2023-05-20T01:22:19.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.339+0000 7f94d9796700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94d41f0640 con 0x7f94d40cb670 2023-05-20T01:22:19.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.339+0000 7f94d37fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94d41599c0 0x7f94d40c97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:19.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.339+0000 7f94d37fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94d41599c0 0x7f94d40c97d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35206/0 (socket says 172.21.15.16:35206) 2023-05-20T01:22:19.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.339+0000 7f94d37fe700 1 -- 172.21.15.16:0/2581724851 learned_addr learned my addr 172.21.15.16:0/2581724851 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:22:19.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.339+0000 7f94d2ffd700 1 --2- 172.21.15.16:0/2581724851 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f94d40caaa0 0x7f94d40c4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:19.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.339+0000 7f94d27fc700 1 --2- 172.21.15.16:0/2581724851 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f94d40cb670 0x7f94d40c9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:19.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.339+0000 7f94d37fe700 1 -- 172.21.15.16:0/2581724851 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f94d40cb670 msgr2=0x7f94d40c9290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:19.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.339+0000 7f94d37fe700 1 --2- 172.21.15.16:0/2581724851 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f94d40cb670 0x7f94d40c9290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:19.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.339+0000 7f94d37fe700 1 -- 172.21.15.16:0/2581724851 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f94d40caaa0 msgr2=0x7f94d40c4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:19.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.339+0000 7f94d37fe700 1 --2- 172.21.15.16:0/2581724851 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f94d40caaa0 0x7f94d40c4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:19.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.339+0000 7f94d37fe700 1 -- 172.21.15.16:0/2581724851 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94c40094a0 con 0x7f94d41599c0 2023-05-20T01:22:19.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.340+0000 7f94d37fe700 1 --2- 172.21.15.16:0/2581724851 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94d41599c0 0x7f94d40c97d0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f94cc00cc20 tx=0x7f94cc00cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:19.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.340+0000 7f94bb7fe700 1 -- 172.21.15.16:0/2581724851 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94cc003c00 con 0x7f94d41599c0 2023-05-20T01:22:19.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.340+0000 7f94bb7fe700 1 -- 172.21.15.16:0/2581724851 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94cc003d60 con 0x7f94d41599c0 2023-05-20T01:22:19.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.340+0000 7f94d9796700 1 -- 172.21.15.16:0/2581724851 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94d406f0b0 con 0x7f94d41599c0 2023-05-20T01:22:19.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.340+0000 7f94bb7fe700 1 -- 172.21.15.16:0/2581724851 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94cc00aae0 con 0x7f94d41599c0 2023-05-20T01:22:19.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.340+0000 7f94d9796700 1 -- 172.21.15.16:0/2581724851 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94d406edd0 con 0x7f94d41599c0 2023-05-20T01:22:19.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.341+0000 7f94d9796700 1 -- 172.21.15.16:0/2581724851 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94d406e750 con 0x7f94d41599c0 2023-05-20T01:22:19.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.341+0000 7f94bb7fe700 1 -- 172.21.15.16:0/2581724851 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f94cc00ac40 con 0x7f94d41599c0 2023-05-20T01:22:19.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.342+0000 7f94bb7fe700 1 --2- 172.21.15.16:0/2581724851 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f94c006ace0 0x7f94c006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:19.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.342+0000 7f94bb7fe700 1 -- 172.21.15.16:0/2581724851 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 7103+0+0 (secure 0 0 0) 0x7f94cc0898f0 con 0x7f94d41599c0 2023-05-20T01:22:19.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.342+0000 7f94d2ffd700 1 --2- 172.21.15.16:0/2581724851 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f94c006ace0 0x7f94c006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:19.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.342+0000 7f94d2ffd700 1 --2- 172.21.15.16:0/2581724851 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f94c006ace0 0x7f94c006d190 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f94c8007fd0 tx=0x7f94c8007dc0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:19.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.346+0000 7f94bb7fe700 1 -- 172.21.15.16:0/2581724851 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f94cc054590 con 0x7f94d41599c0 2023-05-20T01:22:19.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:19.632+0000 7f94d9796700 1 -- 172.21.15.16:0/2581724851 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) v1 -- 0x7f94d40747b0 con 0x7f94d41599c0 2023-05-20T01:22:20.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:20.373+0000 7f94bb7fe700 1 -- 172.21.15.16:0/2581724851 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 3:0 v43) v1 ==== 101+0+0 (secure 0 0 0) 0x7f94cc057bb0 con 0x7f94d41599c0 2023-05-20T01:22:20.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:20.377+0000 7f94d9796700 1 -- 172.21.15.16:0/2581724851 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f94c006ace0 msgr2=0x7f94c006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:20.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:20.377+0000 7f94d9796700 1 --2- 172.21.15.16:0/2581724851 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f94c006ace0 0x7f94c006d190 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f94c8007fd0 tx=0x7f94c8007dc0 comp rx=0 tx=0).stop 2023-05-20T01:22:20.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:20.377+0000 7f94d9796700 1 -- 172.21.15.16:0/2581724851 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94d41599c0 msgr2=0x7f94d40c97d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:20.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:20.377+0000 7f94d9796700 1 --2- 172.21.15.16:0/2581724851 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94d41599c0 0x7f94d40c97d0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f94cc00cc20 tx=0x7f94cc00cf30 comp rx=0 tx=0).stop 2023-05-20T01:22:20.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:20.377+0000 7f94d9796700 1 -- 172.21.15.16:0/2581724851 shutdown_connections 2023-05-20T01:22:20.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:20.377+0000 7f94d9796700 1 --2- 172.21.15.16:0/2581724851 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94d41599c0 0x7f94d40c97d0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:20.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:20.377+0000 7f94d9796700 1 --2- 172.21.15.16:0/2581724851 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f94d40caaa0 0x7f94d40c4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:20.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:20.377+0000 7f94d9796700 1 --2- 172.21.15.16:0/2581724851 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f94c006ace0 0x7f94c006d190 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:20.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:20.377+0000 7f94d9796700 1 --2- 172.21.15.16:0/2581724851 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f94d40cb670 0x7f94d40c9290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:20.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:20.377+0000 7f94d9796700 1 -- 172.21.15.16:0/2581724851 >> 172.21.15.16:0/2581724851 conn(0x7f94d40d0c30 msgr2=0x7f94d40b4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:20.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:20.377+0000 7f94d9796700 1 -- 172.21.15.16:0/2581724851 shutdown_connections 2023-05-20T01:22:20.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:20.377+0000 7f94d9796700 1 -- 172.21.15.16:0/2581724851 wait complete. 2023-05-20T01:22:20.385 INFO:teuthology.orchestra.run.smithi016.stderr:repaired: restoring rank 3:0 2023-05-20T01:22:20.386 INFO:tasks.ceph.mds.c.smithi016.stderr:2023-05-20T01:22:20.382+0000 7fa1a2894700 -1 mds.0.openfiles _load_finish got (2) No such file or directory 2023-05-20T01:22:20.400 INFO:tasks.ceph.mds.a:Restarting daemon 2023-05-20T01:22:20.400 INFO:tasks.ceph.mds.a:Stopping old one... 2023-05-20T01:22:20.400 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-05-20T01:22:20.401 INFO:teuthology.orchestra.run:waiting for 300 2023-05-20T01:22:20.401 INFO:tasks.ceph.mds.c:Restarting daemon 2023-05-20T01:22:20.401 INFO:tasks.ceph.mds.c:Stopping old one... 2023-05-20T01:22:20.401 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-05-20T01:22:20.402 INFO:teuthology.orchestra.run:waiting for 300 2023-05-20T01:22:20.402 INFO:tasks.ceph.mds.b:Restarting daemon 2023-05-20T01:22:20.402 INFO:tasks.ceph.mds.b:Stopping old one... 2023-05-20T01:22:20.403 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-05-20T01:22:20.403 INFO:teuthology.orchestra.run:waiting for 300 2023-05-20T01:22:20.404 INFO:tasks.ceph.mds.d:Restarting daemon 2023-05-20T01:22:20.404 INFO:tasks.ceph.mds.d:Stopping old one... 2023-05-20T01:22:20.404 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-05-20T01:22:20.405 INFO:teuthology.orchestra.run:waiting for 300 2023-05-20T01:22:20.414 INFO:tasks.ceph.mds.a:Stopped 2023-05-20T01:22:20.414 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-05-20T01:22:20.416 INFO:tasks.ceph.mds.c:Stopped 2023-05-20T01:22:20.416 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-05-20T01:22:20.419 INFO:tasks.ceph.mds.b:Stopped 2023-05-20T01:22:20.419 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-05-20T01:22:20.421 INFO:tasks.ceph.mds.d:Stopped 2023-05-20T01:22:20.421 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-05-20T01:22:20.456 INFO:tasks.ceph.mds.a:Started 2023-05-20T01:22:20.457 DEBUG:teuthology.parallel:result is None 2023-05-20T01:22:20.458 INFO:tasks.ceph.mds.c:Started 2023-05-20T01:22:20.458 DEBUG:teuthology.parallel:result is None 2023-05-20T01:22:20.460 INFO:tasks.ceph.mds.b:Started 2023-05-20T01:22:20.461 DEBUG:teuthology.parallel:result is None 2023-05-20T01:22:20.461 INFO:tasks.ceph.mds.d:Started 2023-05-20T01:22:20.462 DEBUG:teuthology.parallel:result is None 2023-05-20T01:22:20.462 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-20T01:22:20.544 INFO:tasks.ceph.mds.a.smithi016.stdout:starting mds.a at 2023-05-20T01:22:20.545 INFO:tasks.ceph.mds.c.smithi016.stdout:starting mds.c at 2023-05-20T01:22:20.546 INFO:tasks.ceph.mds.b.smithi132.stdout:starting mds.b at 2023-05-20T01:22:20.547 INFO:tasks.ceph.mds.d.smithi132.stdout:starting mds.d at 2023-05-20T01:22:20.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:20.601+0000 7f7866604700 1 Processor -- start 2023-05-20T01:22:20.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:20.601+0000 7f7866604700 1 -- start start 2023-05-20T01:22:20.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:20.601+0000 7f7866604700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f78600cfc50 0x7f78600d2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:20.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:20.601+0000 7f7866604700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f78600cd970 0x7f78600cddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:20.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:20.601+0000 7f7866604700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78600b3b50 con 0x7f78600b4750 2023-05-20T01:22:20.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:20.601+0000 7f7866604700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78600cfad0 con 0x7f78600cd970 2023-05-20T01:22:20.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:20.601+0000 7f7866604700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78600d3df0 con 0x7f78600cfc50 2023-05-20T01:22:20.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:20.602+0000 7f785f7fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f78600b4750 0x7f78600d0020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:46774/0 (socket says 172.21.15.16:46774) 2023-05-20T01:22:20.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:20.602+0000 7f785f7fe700 1 -- 172.21.15.16:0/2176845327 learned_addr learned my addr 172.21.15.16:0/2176845327 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:22:20.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:20.602+0000 7f7864ba1700 1 --2- 172.21.15.16:0/2176845327 >> v2:172.21.15.132:3300/0 conn(0x7f78600cd970 0x7f78600cddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:20.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:20.602+0000 7f785ffff700 1 --2- 172.21.15.16:0/2176845327 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f78600cfc50 0x7f78600d2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:20.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:20.602+0000 7f785effd700 1 -- 172.21.15.16:0/2176845327 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1053107599 0 0) 0x7f78600b3b50 con 0x7f78600b4750 2023-05-20T01:22:20.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:20.602+0000 7f785effd700 1 -- 172.21.15.16:0/2176845327 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f78480035b0 con 0x7f78600b4750 2023-05-20T01:22:20.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:20.602+0000 7f785effd700 1 -- 172.21.15.16:0/2176845327 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2163654640 0 0) 0x7f78480035b0 con 0x7f78600b4750 2023-05-20T01:22:20.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:20.602+0000 7f7864ba1700 1 -- 172.21.15.16:0/2176845327 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f78600cfc50 msgr2=0x7f78600d2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:20.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:20.602+0000 7f7864ba1700 1 --2- 172.21.15.16:0/2176845327 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f78600cfc50 0x7f78600d2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:20.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:20.602+0000 7f7864ba1700 1 -- 172.21.15.16:0/2176845327 >> v1:172.21.15.16:6789/0 conn(0x7f78600b4750 legacy=0x7f78600d0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:20.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:20.602+0000 7f7864ba1700 1 -- 172.21.15.16:0/2176845327 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78600d0b90 con 0x7f78600cd970 2023-05-20T01:22:20.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:20.602+0000 7f785ffff700 1 --2- 172.21.15.16:0/2176845327 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f78600cfc50 0x7f78600d2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-05-20T01:22:20.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:20.602+0000 7f7864ba1700 1 --2- 172.21.15.16:0/2176845327 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f78600cd970 0x7f78600cddf0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f7858009d90 tx=0x7f785800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27d78e1ed2095c7b server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:20.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:20.602+0000 7f785effd700 1 -- 172.21.15.16:0/2176845327 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f785800e070 con 0x7f78600cd970 2023-05-20T01:22:20.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:20.602+0000 7f785effd700 1 -- 172.21.15.16:0/2176845327 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7858003a20 con 0x7f78600cd970 2023-05-20T01:22:20.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:20.602+0000 7f785effd700 1 -- 172.21.15.16:0/2176845327 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7854001670 con 0x7f78600cd970 2023-05-20T01:22:20.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:20.603+0000 7f7866604700 1 -- 172.21.15.16:0/2176845327 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f78600cd970 msgr2=0x7f78600cddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:20.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:20.603+0000 7f7866604700 1 --2- 172.21.15.16:0/2176845327 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f78600cd970 0x7f78600cddf0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f7858009d90 tx=0x7f785800b330 comp rx=0 tx=0).stop 2023-05-20T01:22:20.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:20.603+0000 7f7866604700 1 -- 172.21.15.16:0/2176845327 shutdown_connections 2023-05-20T01:22:20.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:20.603+0000 7f7866604700 1 --2- 172.21.15.16:0/2176845327 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f78600cd970 0x7f78600cddf0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:20.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:20.603+0000 7f7866604700 1 --2- 172.21.15.16:0/2176845327 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f78600cfc50 0x7f78600d2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:20.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:20.603+0000 7f7866604700 1 -- 172.21.15.16:0/2176845327 >> 172.21.15.16:0/2176845327 conn(0x7f786009c590 msgr2=0x7f78600d3240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:20.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:20.603+0000 7f7866604700 1 -- 172.21.15.16:0/2176845327 shutdown_connections 2023-05-20T01:22:20.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:20.603+0000 7f7866604700 1 -- 172.21.15.16:0/2176845327 wait complete. 2023-05-20T01:22:20.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:20.605+0000 7f7866604700 1 Processor -- start 2023-05-20T01:22:20.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:20.605+0000 7f7866604700 1 -- start start 2023-05-20T01:22:20.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:20.606+0000 7f7866604700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f78600b4750 0x7f78600c93c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:20.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:20.606+0000 7f7866604700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f78600cd970 0x7f78600cad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:20.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:20.606+0000 7f7866604700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78600cfc50 0x7f78600cb280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:20.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:20.606+0000 7f7866604700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f786015aa50 con 0x7f78600cfc50 2023-05-20T01:22:20.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:20.606+0000 7f7866604700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78600c8320 con 0x7f78600b4750 2023-05-20T01:22:20.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:20.606+0000 7f7866604700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78600c8600 con 0x7f78600cd970 2023-05-20T01:22:20.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:20.606+0000 7f7864ba1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78600cfc50 0x7f78600cb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:20.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:20.606+0000 7f7864ba1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78600cfc50 0x7f78600cb280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35254/0 (socket says 172.21.15.16:35254) 2023-05-20T01:22:20.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:20.606+0000 7f7864ba1700 1 -- 172.21.15.16:0/2454478313 learned_addr learned my addr 172.21.15.16:0/2454478313 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:22:20.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:20.606+0000 7f785f7fe700 1 --2- 172.21.15.16:0/2454478313 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f78600cd970 0x7f78600cad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:20.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:20.606+0000 7f785ffff700 1 --2- 172.21.15.16:0/2454478313 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f78600b4750 0x7f78600c93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:20.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:20.606+0000 7f7864ba1700 1 -- 172.21.15.16:0/2454478313 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f78600cd970 msgr2=0x7f78600cad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:20.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:20.606+0000 7f7864ba1700 1 --2- 172.21.15.16:0/2454478313 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f78600cd970 0x7f78600cad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:20.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:20.606+0000 7f7864ba1700 1 -- 172.21.15.16:0/2454478313 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f78600b4750 msgr2=0x7f78600c93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:20.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:20.606+0000 7f7864ba1700 1 --2- 172.21.15.16:0/2454478313 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f78600b4750 0x7f78600c93c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:20.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:20.606+0000 7f7864ba1700 1 -- 172.21.15.16:0/2454478313 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78500094a0 con 0x7f78600cfc50 2023-05-20T01:22:20.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:20.606+0000 7f785f7fe700 1 --2- 172.21.15.16:0/2454478313 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f78600cd970 0x7f78600cad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:22:20.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:20.606+0000 7f785ffff700 1 --2- 172.21.15.16:0/2454478313 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f78600b4750 0x7f78600c93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:22:20.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:20.606+0000 7f7864ba1700 1 --2- 172.21.15.16:0/2454478313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78600cfc50 0x7f78600cb280 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f7860073aa0 tx=0x7f7858004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:20.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:20.607+0000 7f785cff9700 1 -- 172.21.15.16:0/2454478313 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7858009410 con 0x7f78600cfc50 2023-05-20T01:22:20.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:20.607+0000 7f785cff9700 1 -- 172.21.15.16:0/2454478313 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7858004590 con 0x7f78600cfc50 2023-05-20T01:22:20.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:20.607+0000 7f7866604700 1 -- 172.21.15.16:0/2454478313 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7858009a70 con 0x7f78600cfc50 2023-05-20T01:22:20.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:20.607+0000 7f785cff9700 1 -- 172.21.15.16:0/2454478313 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7858004940 con 0x7f78600cfc50 2023-05-20T01:22:20.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:20.607+0000 7f7866604700 1 -- 172.21.15.16:0/2454478313 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78600b4380 con 0x7f78600cfc50 2023-05-20T01:22:20.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:20.608+0000 7f7866604700 1 -- 172.21.15.16:0/2454478313 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f786015d0b0 con 0x7f78600cfc50 2023-05-20T01:22:20.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:20.608+0000 7f785cff9700 1 -- 172.21.15.16:0/2454478313 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7858012450 con 0x7f78600cfc50 2023-05-20T01:22:20.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:20.608+0000 7f785cff9700 1 --2- 172.21.15.16:0/2454478313 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f784c06ace0 0x7f784c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:20.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:20.609+0000 7f785cff9700 1 -- 172.21.15.16:0/2454478313 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7189+0+0 (secure 0 0 0) 0x7f785808b4f0 con 0x7f78600cfc50 2023-05-20T01:22:20.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:20.609+0000 7f785ffff700 1 --2- 172.21.15.16:0/2454478313 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f784c06ace0 0x7f784c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:20.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:20.609+0000 7f785ffff700 1 --2- 172.21.15.16:0/2454478313 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f784c06ace0 0x7f784c06d190 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f7850009c50 tx=0x7f7850008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:20.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:20.613+0000 7f785cff9700 1 -- 172.21.15.16:0/2454478313 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f7858059c50 con 0x7f78600cfc50 2023-05-20T01:22:20.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:20.855+0000 7f7866604700 1 -- 172.21.15.16:0/2454478313 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f78600d4000 con 0x7f78600cfc50 2023-05-20T01:22:20.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:20.856+0000 7f785cff9700 1 -- 172.21.15.16:0/2454478313 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+4676 (secure 0 0 0) 0x7f78580597e0 con 0x7f78600cfc50 2023-05-20T01:22:20.860 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:22:20.860 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":44,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5824,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/2609276800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":2609276800},{"type":"v1","addr":"172.21.15.132:6834","nonce":2609276800}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32},{"gid":5827,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/29554453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":29554453},{"type":"v1","addr":"172.21.15.132:6835","nonce":29554453}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32},{"gid":6192,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/4090303296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4090303296},{"type":"v1","addr":"172.21.15.16:6836","nonce":4090303296}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-20T01:22:14.127671+0000","modified":"2023-05-20T01:22:20.374967+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":71,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6507},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6507":{"gid":6507,"name":"c","rank":0,"incarnation":44,"state":"up:replay","state_seq":1,"addr":"172.21.15.16:6837/3291457857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":3291457857},{"type":"v1","addr":"172.21.15.16:6837","nonce":3291457857}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10,11],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-05-20T01:22:20.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:20.858+0000 7f7866604700 1 -- 172.21.15.16:0/2454478313 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f784c06ace0 msgr2=0x7f784c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:20.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:20.858+0000 7f7866604700 1 --2- 172.21.15.16:0/2454478313 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f784c06ace0 0x7f784c06d190 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f7850009c50 tx=0x7f7850008040 comp rx=0 tx=0).stop 2023-05-20T01:22:20.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:20.858+0000 7f7866604700 1 -- 172.21.15.16:0/2454478313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78600cfc50 msgr2=0x7f78600cb280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:20.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:20.858+0000 7f7866604700 1 --2- 172.21.15.16:0/2454478313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78600cfc50 0x7f78600cb280 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f7860073aa0 tx=0x7f7858004ad0 comp rx=0 tx=0).stop 2023-05-20T01:22:20.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:20.858+0000 7f7866604700 1 -- 172.21.15.16:0/2454478313 shutdown_connections 2023-05-20T01:22:20.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:20.858+0000 7f7866604700 1 --2- 172.21.15.16:0/2454478313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78600cfc50 0x7f78600cb280 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:20.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:20.858+0000 7f7866604700 1 --2- 172.21.15.16:0/2454478313 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f78600b4750 0x7f78600c93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:20.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:20.858+0000 7f7866604700 1 --2- 172.21.15.16:0/2454478313 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f784c06ace0 0x7f784c06d190 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:20.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:20.858+0000 7f7866604700 1 --2- 172.21.15.16:0/2454478313 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f78600cd970 0x7f78600cad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:20.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:20.858+0000 7f7866604700 1 -- 172.21.15.16:0/2454478313 >> 172.21.15.16:0/2454478313 conn(0x7f786009c590 msgr2=0x7f786006b090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:20.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:20.858+0000 7f7866604700 1 -- 172.21.15.16:0/2454478313 shutdown_connections 2023-05-20T01:22:20.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:20.858+0000 7f7866604700 1 -- 172.21.15.16:0/2454478313 wait complete. 2023-05-20T01:22:20.864 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 44 2023-05-20T01:22:21.276 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 44, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-05-20T01:22:14.127671+0000', 'modified': '2023-05-20T01:22:20.374967+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 71, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6507}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6507': {'gid': 6507, 'name': 'c', 'rank': 0, 'incarnation': 44, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.16:6837/3291457857', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6835', 'nonce': 3291457857}, {'type': 'v1', 'addr': '172.21.15.16:6837', 'nonce': 3291457857}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [10, 11], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-05-20T01:22:21.277 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6507:up:replay 2023-05-20T01:22:21.381 INFO:tasks.ceph.mon.a.smithi016.stderr:2023-05-20T01:22:21.377+0000 7f49eb8ae700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-05-20T01:22:21.394 INFO:tasks.ceph.mds.c.smithi016.stderr:2023-05-20T01:22:21.390+0000 7fd0f0e15700 -1 mds.0.openfiles _load_finish got (2) No such file or directory 2023-05-20T01:22:22.278 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-20T01:22:22.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:22.415+0000 7fea5fcdf700 1 Processor -- start 2023-05-20T01:22:22.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:22.415+0000 7fea5fcdf700 1 -- start start 2023-05-20T01:22:22.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:22.416+0000 7fea5fcdf700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fea58071340 0x7fea58071760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:22.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:22.416+0000 7fea5fcdf700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea5815e060 con 0x7fea5806a510 2023-05-20T01:22:22.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:22.416+0000 7fea5fcdf700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea5815d450 con 0x7fea5815e2a0 2023-05-20T01:22:22.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:22.416+0000 7fea5fcdf700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea5815d730 con 0x7fea58071340 2023-05-20T01:22:22.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:22.416+0000 7fea5e27c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fea58071340 0x7fea58071760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:22.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:22.416+0000 7fea5e27c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fea58071340 0x7fea58071760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35278/0 (socket says 172.21.15.16:35278) 2023-05-20T01:22:22.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:22.416+0000 7fea5da7b700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fea5806a510 0x7fea5806a8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:46802/0 (socket says 172.21.15.16:46802) 2023-05-20T01:22:22.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:22.416+0000 7fea5e27c700 1 -- 172.21.15.16:0/1468813737 learned_addr learned my addr 172.21.15.16:0/1468813737 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:22:22.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:22.417+0000 7fea5e27c700 1 -- 172.21.15.16:0/1468813737 >> v1:172.21.15.132:6789/0 conn(0x7fea5815e2a0 legacy=0x7fea580cf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:22.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:22.417+0000 7fea5e27c700 1 -- 172.21.15.16:0/1468813737 >> v1:172.21.15.16:6789/0 conn(0x7fea5806a510 legacy=0x7fea5806a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:22.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:22.417+0000 7fea5e27c700 1 -- 172.21.15.16:0/1468813737 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea5815da10 con 0x7fea58071340 2023-05-20T01:22:22.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:22.417+0000 7fea5ca79700 1 -- 172.21.15.16:0/1468813737 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 117343401 0 0) 0x7fea5815e060 con 0x7fea5806a510 2023-05-20T01:22:22.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:22.417+0000 7fea5e27c700 1 --2- 172.21.15.16:0/1468813737 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea58071340 0x7fea58071760 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fea50009e50 tx=0x7fea5000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0295dc5d6f4d33f server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:22.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:22.417+0000 7fea5ca79700 1 -- 172.21.15.16:0/1468813737 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea5000e070 con 0x7fea58071340 2023-05-20T01:22:22.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:22.417+0000 7fea5ca79700 1 -- 172.21.15.16:0/1468813737 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea5000b9c0 con 0x7fea58071340 2023-05-20T01:22:22.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:22.417+0000 7fea5ca79700 1 -- 172.21.15.16:0/1468813737 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea5000bd70 con 0x7fea58071340 2023-05-20T01:22:22.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:22.418+0000 7fea5fcdf700 1 -- 172.21.15.16:0/1468813737 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea58071340 msgr2=0x7fea58071760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:22.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:22.418+0000 7fea5fcdf700 1 --2- 172.21.15.16:0/1468813737 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea58071340 0x7fea58071760 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fea50009e50 tx=0x7fea5000b330 comp rx=0 tx=0).stop 2023-05-20T01:22:22.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:22.418+0000 7fea5fcdf700 1 -- 172.21.15.16:0/1468813737 shutdown_connections 2023-05-20T01:22:22.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:22.418+0000 7fea5fcdf700 1 --2- 172.21.15.16:0/1468813737 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea58071340 0x7fea58071760 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:22.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:22.418+0000 7fea5fcdf700 1 -- 172.21.15.16:0/1468813737 >> 172.21.15.16:0/1468813737 conn(0x7fea580d0c30 msgr2=0x7fea580b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:22.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:22.418+0000 7fea5fcdf700 1 -- 172.21.15.16:0/1468813737 shutdown_connections 2023-05-20T01:22:22.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:22.418+0000 7fea5fcdf700 1 -- 172.21.15.16:0/1468813737 wait complete. 2023-05-20T01:22:22.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:22.419+0000 7fea5fcdf700 1 Processor -- start 2023-05-20T01:22:22.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:22.419+0000 7fea5fcdf700 1 -- start start 2023-05-20T01:22:22.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:22.420+0000 7fea5fcdf700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fea5806a510 0x7fea5814d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:22.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:22.420+0000 7fea5fcdf700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fea58071340 0x7fea580c49e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:22.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:22.420+0000 7fea5fcdf700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea5815e2a0 0x7fea580c83a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:22.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:22.420+0000 7fea5fcdf700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea580c5fd0 con 0x7fea5815e2a0 2023-05-20T01:22:22.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:22.420+0000 7fea5fcdf700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea580c53c0 con 0x7fea58071340 2023-05-20T01:22:22.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:22.420+0000 7fea5fcdf700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea580c56a0 con 0x7fea5806a510 2023-05-20T01:22:22.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:22.420+0000 7fea5e27c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea5815e2a0 0x7fea580c83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:22.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:22.420+0000 7fea5e27c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea5815e2a0 0x7fea580c83a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35284/0 (socket says 172.21.15.16:35284) 2023-05-20T01:22:22.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:22.420+0000 7fea5e27c700 1 -- 172.21.15.16:0/3701316566 learned_addr learned my addr 172.21.15.16:0/3701316566 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:22:22.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:22.420+0000 7fea5da7b700 1 --2- 172.21.15.16:0/3701316566 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fea5806a510 0x7fea5814d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:22.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:22.420+0000 7fea5d27a700 1 --2- 172.21.15.16:0/3701316566 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fea58071340 0x7fea580c49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:22.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:22.420+0000 7fea5e27c700 1 -- 172.21.15.16:0/3701316566 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fea5806a510 msgr2=0x7fea5814d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:22.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:22.420+0000 7fea5e27c700 1 --2- 172.21.15.16:0/3701316566 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fea5806a510 0x7fea5814d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:22.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:22.420+0000 7fea5e27c700 1 -- 172.21.15.16:0/3701316566 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fea58071340 msgr2=0x7fea580c49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:22.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:22.420+0000 7fea5e27c700 1 --2- 172.21.15.16:0/3701316566 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fea58071340 0x7fea580c49e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:22.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:22.420+0000 7fea5e27c700 1 -- 172.21.15.16:0/3701316566 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea50009b30 con 0x7fea5815e2a0 2023-05-20T01:22:22.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:22.420+0000 7fea5d27a700 1 --2- 172.21.15.16:0/3701316566 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fea58071340 0x7fea580c49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:22:22.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:22.420+0000 7fea5e27c700 1 --2- 172.21.15.16:0/3701316566 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea5815e2a0 0x7fea580c83a0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fea5000b730 tx=0x7fea50004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:22.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:22.420+0000 7fea4e7fc700 1 -- 172.21.15.16:0/3701316566 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea5001c640 con 0x7fea5815e2a0 2023-05-20T01:22:22.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:22.421+0000 7fea4e7fc700 1 -- 172.21.15.16:0/3701316566 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea5001c7c0 con 0x7fea5815e2a0 2023-05-20T01:22:22.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:22.421+0000 7fea5fcdf700 1 -- 172.21.15.16:0/3701316566 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea580cfd00 con 0x7fea5815e2a0 2023-05-20T01:22:22.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:22.421+0000 7fea4e7fc700 1 -- 172.21.15.16:0/3701316566 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea5001cb70 con 0x7fea5815e2a0 2023-05-20T01:22:22.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:22.421+0000 7fea5fcdf700 1 -- 172.21.15.16:0/3701316566 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea5815dcf0 con 0x7fea5815e2a0 2023-05-20T01:22:22.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:22.422+0000 7fea5fcdf700 1 -- 172.21.15.16:0/3701316566 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea5815d080 con 0x7fea5815e2a0 2023-05-20T01:22:22.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:22.422+0000 7fea4e7fc700 1 -- 172.21.15.16:0/3701316566 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fea5001ccd0 con 0x7fea5815e2a0 2023-05-20T01:22:22.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:22.422+0000 7fea4e7fc700 1 --2- 172.21.15.16:0/3701316566 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fea3c06ace0 0x7fea3c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:22.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:22.423+0000 7fea5da7b700 1 --2- 172.21.15.16:0/3701316566 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fea3c06ace0 0x7fea3c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:22.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:22.423+0000 7fea4e7fc700 1 -- 172.21.15.16:0/3701316566 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7189+0+0 (secure 0 0 0) 0x7fea5008b7a0 con 0x7fea5815e2a0 2023-05-20T01:22:22.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:22.423+0000 7fea5da7b700 1 --2- 172.21.15.16:0/3701316566 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fea3c06ace0 0x7fea3c06d190 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fea48000f60 tx=0x7fea48009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:22.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:22.427+0000 7fea4e7fc700 1 -- 172.21.15.16:0/3701316566 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fea50056590 con 0x7fea5815e2a0 2023-05-20T01:22:22.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:22.669+0000 7fea5fcdf700 1 -- 172.21.15.16:0/3701316566 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fea580b22d0 con 0x7fea5815e2a0 2023-05-20T01:22:22.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:22.669+0000 7fea4e7fc700 1 -- 172.21.15.16:0/3701316566 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4682 (secure 0 0 0) 0x7fea50059bb0 con 0x7fea5815e2a0 2023-05-20T01:22:22.674 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:22:22.674 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":47,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5873,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3769502841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3769502841},{"type":"v1","addr":"172.21.15.16:6835","nonce":3769502841}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45},{"gid":5879,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/688624373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":688624373},{"type":"v1","addr":"172.21.15.132:6834","nonce":688624373}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45},{"gid":6307,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/1655712081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":1655712081},{"type":"v1","addr":"172.21.15.132:6835","nonce":1655712081}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-20T01:22:14.127671+0000","modified":"2023-05-20T01:22:22.383541+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":72,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6768},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6768":{"gid":6768,"name":"c","rank":0,"incarnation":46,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.16:6837/2351975329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2351975329},{"type":"v1","addr":"172.21.15.16:6837","nonce":2351975329}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10,11],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-05-20T01:22:22.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:22.671+0000 7fea5fcdf700 1 -- 172.21.15.16:0/3701316566 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fea3c06ace0 msgr2=0x7fea3c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:22.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:22.671+0000 7fea5fcdf700 1 --2- 172.21.15.16:0/3701316566 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fea3c06ace0 0x7fea3c06d190 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fea48000f60 tx=0x7fea48009040 comp rx=0 tx=0).stop 2023-05-20T01:22:22.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:22.671+0000 7fea5fcdf700 1 -- 172.21.15.16:0/3701316566 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea5815e2a0 msgr2=0x7fea580c83a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:22.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:22.671+0000 7fea5fcdf700 1 --2- 172.21.15.16:0/3701316566 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea5815e2a0 0x7fea580c83a0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fea5000b730 tx=0x7fea50004ad0 comp rx=0 tx=0).stop 2023-05-20T01:22:22.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:22.671+0000 7fea5fcdf700 1 -- 172.21.15.16:0/3701316566 shutdown_connections 2023-05-20T01:22:22.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:22.671+0000 7fea5fcdf700 1 --2- 172.21.15.16:0/3701316566 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea5815e2a0 0x7fea580c83a0 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:22.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:22.671+0000 7fea5fcdf700 1 --2- 172.21.15.16:0/3701316566 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fea3c06ace0 0x7fea3c06d190 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:22.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:22.671+0000 7fea5fcdf700 1 --2- 172.21.15.16:0/3701316566 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fea5806a510 0x7fea5814d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:22.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:22.671+0000 7fea5fcdf700 1 --2- 172.21.15.16:0/3701316566 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fea58071340 0x7fea580c49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:22.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:22.671+0000 7fea5fcdf700 1 -- 172.21.15.16:0/3701316566 >> 172.21.15.16:0/3701316566 conn(0x7fea580d0c30 msgr2=0x7fea5806a8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:22.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:22.672+0000 7fea5fcdf700 1 -- 172.21.15.16:0/3701316566 shutdown_connections 2023-05-20T01:22:22.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:22.672+0000 7fea5fcdf700 1 -- 172.21.15.16:0/3701316566 wait complete. 2023-05-20T01:22:22.678 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 47 2023-05-20T01:22:23.091 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 47, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-05-20T01:22:14.127671+0000', 'modified': '2023-05-20T01:22:22.383541+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 72, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6768}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6768': {'gid': 6768, 'name': 'c', 'rank': 0, 'incarnation': 46, 'state': 'up:reconnect', 'state_seq': 2, 'addr': '172.21.15.16:6837/2351975329', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6836', 'nonce': 2351975329}, {'type': 'v1', 'addr': '172.21.15.16:6837', 'nonce': 2351975329}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [10, 11], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-05-20T01:22:23.091 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6768:up:reconnect 2023-05-20T01:22:24.092 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-20T01:22:24.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:24.228+0000 7f8653ddb700 1 Processor -- start 2023-05-20T01:22:24.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:24.228+0000 7f8653ddb700 1 -- start start 2023-05-20T01:22:24.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:24.229+0000 7f8653ddb700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f864c0b2200 0x7f864c06dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:24.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:24.229+0000 7f8653ddb700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f864c0b2680 0x7f864c06e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:24.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:24.229+0000 7f8653ddb700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f864c15f650 con 0x7f864c15e110 2023-05-20T01:22:24.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:24.229+0000 7f8653ddb700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f864c15e4e0 con 0x7f864c0b2200 2023-05-20T01:22:24.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:24.229+0000 7f8653ddb700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f864c15ead0 con 0x7f864c0b2680 2023-05-20T01:22:24.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:24.229+0000 7f8651376700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f864c0b2200 0x7f864c06dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:24.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:24.229+0000 7f8651b77700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f864c15e110 0x7f864c071140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:46816/0 (socket says 172.21.15.16:46816) 2023-05-20T01:22:24.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:24.229+0000 7f8651b77700 1 -- 172.21.15.16:0/2620803753 learned_addr learned my addr 172.21.15.16:0/2620803753 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:22:24.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:24.229+0000 7f8652378700 1 --2- 172.21.15.16:0/2620803753 >> v2:172.21.15.132:3300/0 conn(0x7f864c0b2680 0x7f864c06e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:24.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:24.229+0000 7f8651376700 1 -- 172.21.15.16:0/2620803753 >> v2:172.21.15.132:3300/0 conn(0x7f864c0b2680 msgr2=0x7f864c06e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:24.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:24.229+0000 7f8651376700 1 --2- 172.21.15.16:0/2620803753 >> v2:172.21.15.132:3300/0 conn(0x7f864c0b2680 0x7f864c06e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:24.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:24.229+0000 7f8651376700 1 -- 172.21.15.16:0/2620803753 >> v1:172.21.15.16:6789/0 conn(0x7f864c15e110 legacy=0x7f864c071140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:24.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:24.229+0000 7f8651376700 1 -- 172.21.15.16:0/2620803753 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f864c15edb0 con 0x7f864c0b2200 2023-05-20T01:22:24.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:24.229+0000 7f8650b75700 1 -- 172.21.15.16:0/2620803753 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3358703165 0 0) 0x7f864c15f650 con 0x7f864c15e110 2023-05-20T01:22:24.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:24.229+0000 7f8652378700 1 --2- 172.21.15.16:0/2620803753 >> v2:172.21.15.132:3300/0 conn(0x7f864c0b2680 0x7f864c06e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:22:24.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:24.230+0000 7f8651376700 1 --2- 172.21.15.16:0/2620803753 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f864c0b2200 0x7f864c06dfc0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f864c066c20 tx=0x7f8648009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d63f9349f6c2dda5 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:24.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:24.230+0000 7f8650b75700 1 -- 172.21.15.16:0/2620803753 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8648011830 con 0x7f864c0b2200 2023-05-20T01:22:24.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:24.230+0000 7f8650b75700 1 -- 172.21.15.16:0/2620803753 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8648011990 con 0x7f864c0b2200 2023-05-20T01:22:24.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:24.230+0000 7f8650b75700 1 -- 172.21.15.16:0/2620803753 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8648011d20 con 0x7f864c0b2200 2023-05-20T01:22:24.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:24.230+0000 7f8653ddb700 1 -- 172.21.15.16:0/2620803753 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f864c0b2200 msgr2=0x7f864c06dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:24.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:24.230+0000 7f8653ddb700 1 --2- 172.21.15.16:0/2620803753 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f864c0b2200 0x7f864c06dfc0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f864c066c20 tx=0x7f8648009b20 comp rx=0 tx=0).stop 2023-05-20T01:22:24.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:24.230+0000 7f8653ddb700 1 -- 172.21.15.16:0/2620803753 shutdown_connections 2023-05-20T01:22:24.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:24.230+0000 7f8653ddb700 1 --2- 172.21.15.16:0/2620803753 >> v2:172.21.15.132:3300/0 conn(0x7f864c0b2680 0x7f864c06e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:24.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:24.230+0000 7f8653ddb700 1 --2- 172.21.15.16:0/2620803753 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f864c0b2200 0x7f864c06dfc0 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:24.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:24.230+0000 7f8653ddb700 1 -- 172.21.15.16:0/2620803753 >> 172.21.15.16:0/2620803753 conn(0x7f864c0d0c70 msgr2=0x7f864c0b1af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:24.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:24.230+0000 7f8653ddb700 1 -- 172.21.15.16:0/2620803753 shutdown_connections 2023-05-20T01:22:24.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:24.230+0000 7f8653ddb700 1 -- 172.21.15.16:0/2620803753 wait complete. 2023-05-20T01:22:24.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:24.232+0000 7f8653ddb700 1 Processor -- start 2023-05-20T01:22:24.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:24.232+0000 7f8653ddb700 1 -- start start 2023-05-20T01:22:24.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:24.232+0000 7f8653ddb700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f864c0b2200 0x7f864c0c91c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:24.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:24.232+0000 7f8653ddb700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f864c0b2680 0x7f864c0c4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:24.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:24.232+0000 7f8653ddb700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f864c15e110 0x7f864c0c5d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:24.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:24.232+0000 7f8653ddb700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f864c0c3b10 con 0x7f864c0b2200 2023-05-20T01:22:24.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:24.232+0000 7f8653ddb700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f864c0c3df0 con 0x7f864c0b2680 2023-05-20T01:22:24.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:24.232+0000 7f8653ddb700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f864c0c40d0 con 0x7f864c15e110 2023-05-20T01:22:24.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:24.233+0000 7f8651b77700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f864c0b2200 0x7f864c0c91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:24.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:24.233+0000 7f8651b77700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f864c0b2200 0x7f864c0c91c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35294/0 (socket says 172.21.15.16:35294) 2023-05-20T01:22:24.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:24.233+0000 7f8651b77700 1 -- 172.21.15.16:0/4093705194 learned_addr learned my addr 172.21.15.16:0/4093705194 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:22:24.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:24.233+0000 7f8651376700 1 --2- 172.21.15.16:0/4093705194 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f864c0b2680 0x7f864c0c4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:24.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:24.233+0000 7f8652378700 1 --2- 172.21.15.16:0/4093705194 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f864c15e110 0x7f864c0c5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:24.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:24.233+0000 7f8651b77700 1 -- 172.21.15.16:0/4093705194 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f864c15e110 msgr2=0x7f864c0c5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:24.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:24.233+0000 7f8651b77700 1 --2- 172.21.15.16:0/4093705194 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f864c15e110 0x7f864c0c5d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:24.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:24.233+0000 7f8651b77700 1 -- 172.21.15.16:0/4093705194 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f864c0b2680 msgr2=0x7f864c0c4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:24.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:24.233+0000 7f8651b77700 1 --2- 172.21.15.16:0/4093705194 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f864c0b2680 0x7f864c0c4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:24.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:24.233+0000 7f8651b77700 1 -- 172.21.15.16:0/4093705194 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86480094a0 con 0x7f864c0b2200 2023-05-20T01:22:24.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:24.233+0000 7f8651b77700 1 --2- 172.21.15.16:0/4093705194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f864c0b2200 0x7f864c0c91c0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f863c00c550 tx=0x7f863c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:24.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:24.233+0000 7f86427fc700 1 -- 172.21.15.16:0/4093705194 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f863c004c80 con 0x7f864c0b2200 2023-05-20T01:22:24.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:24.233+0000 7f86427fc700 1 -- 172.21.15.16:0/4093705194 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f863c004e00 con 0x7f864c0b2200 2023-05-20T01:22:24.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:24.233+0000 7f8653ddb700 1 -- 172.21.15.16:0/4093705194 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f864c15f370 con 0x7f864c0b2200 2023-05-20T01:22:24.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:24.233+0000 7f86427fc700 1 -- 172.21.15.16:0/4093705194 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f863c007310 con 0x7f864c0b2200 2023-05-20T01:22:24.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:24.233+0000 7f8653ddb700 1 -- 172.21.15.16:0/4093705194 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f864c15f090 con 0x7f864c0b2200 2023-05-20T01:22:24.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:24.234+0000 7f8653ddb700 1 -- 172.21.15.16:0/4093705194 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f864c0cc7a0 con 0x7f864c0b2200 2023-05-20T01:22:24.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:24.235+0000 7f86427fc700 1 -- 172.21.15.16:0/4093705194 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f863c007470 con 0x7f864c0b2200 2023-05-20T01:22:24.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:24.235+0000 7f86427fc700 1 --2- 172.21.15.16:0/4093705194 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f863006ace0 0x7f863006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:24.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:24.235+0000 7f86427fc700 1 -- 172.21.15.16:0/4093705194 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7189+0+0 (secure 0 0 0) 0x7f863c089600 con 0x7f864c0b2200 2023-05-20T01:22:24.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:24.235+0000 7f8651376700 1 --2- 172.21.15.16:0/4093705194 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f863006ace0 0x7f863006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:24.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:24.236+0000 7f8651376700 1 --2- 172.21.15.16:0/4093705194 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f863006ace0 0x7f863006d190 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f86480090b0 tx=0x7f8648008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:24.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:24.240+0000 7f86427fc700 1 -- 172.21.15.16:0/4093705194 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f863c054240 con 0x7f864c0b2200 2023-05-20T01:22:24.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:24.483+0000 7f8653ddb700 1 -- 172.21.15.16:0/4093705194 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f864c0747b0 con 0x7f864c0b2200 2023-05-20T01:22:24.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:24.484+0000 7f86427fc700 1 -- 172.21.15.16:0/4093705194 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+4679 (secure 0 0 0) 0x7f863c057860 con 0x7f864c0b2200 2023-05-20T01:22:24.488 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:22:24.489 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":49,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5873,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3769502841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3769502841},{"type":"v1","addr":"172.21.15.16:6835","nonce":3769502841}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45},{"gid":5879,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/688624373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":688624373},{"type":"v1","addr":"172.21.15.132:6834","nonce":688624373}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45},{"gid":6307,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/1655712081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":1655712081},{"type":"v1","addr":"172.21.15.132:6835","nonce":1655712081}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-20T01:22:14.127671+0000","modified":"2023-05-20T01:22:24.390190+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":72,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6768},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6768":{"gid":6768,"name":"c","rank":0,"incarnation":46,"state":"up:active","state_seq":4,"addr":"172.21.15.16:6837/2351975329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2351975329},{"type":"v1","addr":"172.21.15.16:6837","nonce":2351975329}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10,11],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-05-20T01:22:24.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:24.486+0000 7f8653ddb700 1 -- 172.21.15.16:0/4093705194 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f863006ace0 msgr2=0x7f863006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:24.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:24.486+0000 7f8653ddb700 1 --2- 172.21.15.16:0/4093705194 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f863006ace0 0x7f863006d190 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f86480090b0 tx=0x7f8648008030 comp rx=0 tx=0).stop 2023-05-20T01:22:24.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:24.486+0000 7f8653ddb700 1 -- 172.21.15.16:0/4093705194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f864c0b2200 msgr2=0x7f864c0c91c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:24.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:24.486+0000 7f8653ddb700 1 --2- 172.21.15.16:0/4093705194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f864c0b2200 0x7f864c0c91c0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f863c00c550 tx=0x7f863c00c910 comp rx=0 tx=0).stop 2023-05-20T01:22:24.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:24.486+0000 7f8653ddb700 1 -- 172.21.15.16:0/4093705194 shutdown_connections 2023-05-20T01:22:24.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:24.486+0000 7f8653ddb700 1 --2- 172.21.15.16:0/4093705194 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f863006ace0 0x7f863006d190 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:24.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:24.486+0000 7f8653ddb700 1 --2- 172.21.15.16:0/4093705194 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f864c15e110 0x7f864c0c5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:24.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:24.486+0000 7f8653ddb700 1 --2- 172.21.15.16:0/4093705194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f864c0b2200 0x7f864c0c91c0 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:24.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:24.486+0000 7f8653ddb700 1 --2- 172.21.15.16:0/4093705194 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f864c0b2680 0x7f864c0c4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:24.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:24.486+0000 7f8653ddb700 1 -- 172.21.15.16:0/4093705194 >> 172.21.15.16:0/4093705194 conn(0x7f864c0d0c70 msgr2=0x7f864c071140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:24.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:24.486+0000 7f8653ddb700 1 -- 172.21.15.16:0/4093705194 shutdown_connections 2023-05-20T01:22:24.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:24.486+0000 7f8653ddb700 1 -- 172.21.15.16:0/4093705194 wait complete. 2023-05-20T01:22:24.492 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 49 2023-05-20T01:22:24.904 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 49, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-05-20T01:22:14.127671+0000', 'modified': '2023-05-20T01:22:24.390190+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 72, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6768}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6768': {'gid': 6768, 'name': 'c', 'rank': 0, 'incarnation': 46, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.16:6837/2351975329', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6836', 'nonce': 2351975329}, {'type': 'v1', 'addr': '172.21.15.16:6837', 'nonce': 2351975329}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [10, 11], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-05-20T01:22:24.905 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-05-20T01:22:24.905 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-05-20T01:22:25.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.036+0000 7f4e86715700 1 Processor -- start 2023-05-20T01:22:25.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.036+0000 7f4e86715700 1 -- start start 2023-05-20T01:22:25.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.037+0000 7f4e86715700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4e800af2d0 0x7f4e800cec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:25.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.037+0000 7f4e86715700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4e800cf160 0x7f4e800b0980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:25.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.037+0000 7f4e86715700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e800b2000 con 0x7f4e800d16c0 2023-05-20T01:22:25.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.037+0000 7f4e86715700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e800cf5e0 con 0x7f4e800af2d0 2023-05-20T01:22:25.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.037+0000 7f4e86715700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e800b22e0 con 0x7f4e800cf160 2023-05-20T01:22:25.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.037+0000 7f4e7f7fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4e800af2d0 0x7f4e800cec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:25.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.037+0000 7f4e7f7fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4e800af2d0 0x7f4e800cec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35304/0 (socket says 172.21.15.16:35304) 2023-05-20T01:22:25.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.037+0000 7f4e7f7fe700 1 -- 172.21.15.16:0/1884879814 learned_addr learned my addr 172.21.15.16:0/1884879814 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:22:25.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.037+0000 7f4e84cb2700 1 --2- 172.21.15.16:0/1884879814 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4e800cf160 0x7f4e800b0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:25.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.038+0000 7f4e7f7fe700 1 -- 172.21.15.16:0/1884879814 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4e800cf160 msgr2=0x7f4e800b0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:25.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.038+0000 7f4e7f7fe700 1 --2- 172.21.15.16:0/1884879814 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4e800cf160 0x7f4e800b0980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:25.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.038+0000 7f4e7f7fe700 1 -- 172.21.15.16:0/1884879814 >> v1:172.21.15.132:6789/0 conn(0x7f4e800d16c0 legacy=0x7f4e80144330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:25.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.038+0000 7f4e7f7fe700 1 -- 172.21.15.16:0/1884879814 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e800b25c0 con 0x7f4e800af2d0 2023-05-20T01:22:25.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.038+0000 7f4e7f7fe700 1 --2- 172.21.15.16:0/1884879814 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e800af2d0 0x7f4e800cec20 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f4e70009920 tx=0x7f4e70009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da93371583c15d68 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:25.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.038+0000 7f4e7effd700 1 -- 172.21.15.16:0/1884879814 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e70004230 con 0x7f4e800af2d0 2023-05-20T01:22:25.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.038+0000 7f4e7effd700 1 -- 172.21.15.16:0/1884879814 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e70004390 con 0x7f4e800af2d0 2023-05-20T01:22:25.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.038+0000 7f4e7effd700 1 -- 172.21.15.16:0/1884879814 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e70009e70 con 0x7f4e800af2d0 2023-05-20T01:22:25.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.038+0000 7f4e86715700 1 -- 172.21.15.16:0/1884879814 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e800af2d0 msgr2=0x7f4e800cec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:25.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.038+0000 7f4e86715700 1 --2- 172.21.15.16:0/1884879814 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e800af2d0 0x7f4e800cec20 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f4e70009920 tx=0x7f4e70009c30 comp rx=0 tx=0).stop 2023-05-20T01:22:25.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.039+0000 7f4e86715700 1 -- 172.21.15.16:0/1884879814 shutdown_connections 2023-05-20T01:22:25.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.039+0000 7f4e86715700 1 --2- 172.21.15.16:0/1884879814 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4e800cf160 0x7f4e800b0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:25.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.039+0000 7f4e86715700 1 --2- 172.21.15.16:0/1884879814 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e800af2d0 0x7f4e800cec20 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:25.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.039+0000 7f4e86715700 1 -- 172.21.15.16:0/1884879814 >> 172.21.15.16:0/1884879814 conn(0x7f4e8006ae30 msgr2=0x7f4e8009a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:25.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.039+0000 7f4e86715700 1 -- 172.21.15.16:0/1884879814 shutdown_connections 2023-05-20T01:22:25.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.039+0000 7f4e86715700 1 -- 172.21.15.16:0/1884879814 wait complete. 2023-05-20T01:22:25.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.040+0000 7f4e86715700 1 Processor -- start 2023-05-20T01:22:25.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.040+0000 7f4e86715700 1 -- start start 2023-05-20T01:22:25.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.041+0000 7f4e86715700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4e800af2d0 0x7f4e801f2320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:25.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.041+0000 7f4e86715700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e800cf160 0x7f4e800c4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:25.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.041+0000 7f4e86715700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4e800d16c0 0x7f4e800cd690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:25.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.041+0000 7f4e86715700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e800b5b40 con 0x7f4e800cf160 2023-05-20T01:22:25.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.041+0000 7f4e86715700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e800b4f70 con 0x7f4e800af2d0 2023-05-20T01:22:25.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.041+0000 7f4e86715700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e800b5210 con 0x7f4e800d16c0 2023-05-20T01:22:25.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.041+0000 7f4e7f7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e800cf160 0x7f4e800c4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:25.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.041+0000 7f4e7f7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e800cf160 0x7f4e800c4780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35310/0 (socket says 172.21.15.16:35310) 2023-05-20T01:22:25.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.041+0000 7f4e7f7fe700 1 -- 172.21.15.16:0/1289201177 learned_addr learned my addr 172.21.15.16:0/1289201177 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:22:25.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.041+0000 7f4e84cb2700 1 --2- 172.21.15.16:0/1289201177 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4e800d16c0 0x7f4e800cd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:25.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.041+0000 7f4e7ffff700 1 --2- 172.21.15.16:0/1289201177 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4e800af2d0 0x7f4e801f2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:25.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.041+0000 7f4e7f7fe700 1 -- 172.21.15.16:0/1289201177 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4e800d16c0 msgr2=0x7f4e800cd690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:25.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.041+0000 7f4e7f7fe700 1 --2- 172.21.15.16:0/1289201177 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4e800d16c0 0x7f4e800cd690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:25.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.041+0000 7f4e7f7fe700 1 -- 172.21.15.16:0/1289201177 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4e800af2d0 msgr2=0x7f4e801f2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:25.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.041+0000 7f4e7f7fe700 1 --2- 172.21.15.16:0/1289201177 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4e800af2d0 0x7f4e801f2320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:25.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.041+0000 7f4e7f7fe700 1 -- 172.21.15.16:0/1289201177 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e70009550 con 0x7f4e800cf160 2023-05-20T01:22:25.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.041+0000 7f4e7f7fe700 1 --2- 172.21.15.16:0/1289201177 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e800cf160 0x7f4e800c4780 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f4e700098f0 tx=0x7f4e70004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:25.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.042+0000 7f4e7cff9700 1 -- 172.21.15.16:0/1289201177 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e7000bc60 con 0x7f4e800cf160 2023-05-20T01:22:25.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.042+0000 7f4e7cff9700 1 -- 172.21.15.16:0/1289201177 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e7001f070 con 0x7f4e800cf160 2023-05-20T01:22:25.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.042+0000 7f4e86715700 1 -- 172.21.15.16:0/1289201177 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e800b28a0 con 0x7f4e800cf160 2023-05-20T01:22:25.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.042+0000 7f4e7cff9700 1 -- 172.21.15.16:0/1289201177 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e700036a0 con 0x7f4e800cf160 2023-05-20T01:22:25.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.042+0000 7f4e86715700 1 -- 172.21.15.16:0/1289201177 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e800adda0 con 0x7f4e800cf160 2023-05-20T01:22:25.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.043+0000 7f4e7cff9700 1 -- 172.21.15.16:0/1289201177 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4e70021030 con 0x7f4e800cf160 2023-05-20T01:22:25.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.043+0000 7f4e7cff9700 1 --2- 172.21.15.16:0/1289201177 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f4e6006ace0 0x7f4e6006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:25.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.044+0000 7f4e7cff9700 1 -- 172.21.15.16:0/1289201177 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7189+0+0 (secure 0 0 0) 0x7f4e7008ae80 con 0x7f4e800cf160 2023-05-20T01:22:25.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.044+0000 7f4e7ffff700 1 --2- 172.21.15.16:0/1289201177 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f4e6006ace0 0x7f4e6006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:25.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.044+0000 7f4e7ffff700 1 --2- 172.21.15.16:0/1289201177 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f4e6006ace0 0x7f4e6006d190 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f4e680016f0 tx=0x7f4e68009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:25.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.045+0000 7f4e928a1b80 1 Processor -- start 2023-05-20T01:22:25.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.046+0000 7f4e928a1b80 1 -- start start 2023-05-20T01:22:25.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.046+0000 7f4e928a1b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e58a53f400 0x55e58a53f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:25.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.046+0000 7f4e928a1b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e58a53b700 0x55e58a514460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:25.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.046+0000 7f4e928a1b80 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e58a512870 0x55e58a5149a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:25.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.046+0000 7f4e928a1b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e58a41a010 con 0x55e58a53b700 2023-05-20T01:22:25.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.046+0000 7f4e928a1b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e58a46c850 con 0x55e58a53f400 2023-05-20T01:22:25.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.046+0000 7f4e928a1b80 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e58a465ce0 con 0x55e58a512870 2023-05-20T01:22:25.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.046+0000 7f4e7f7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e58a53b700 0x55e58a514460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:25.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.046+0000 7f4e7f7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e58a53b700 0x55e58a514460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35318/0 (socket says 172.21.15.16:35318) 2023-05-20T01:22:25.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.046+0000 7f4e7f7fe700 1 -- 172.21.15.16:0/1155183825 learned_addr learned my addr 172.21.15.16:0/1155183825 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:22:25.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.047+0000 7f4e7ffff700 1 --2- 172.21.15.16:0/1155183825 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e58a53f400 0x55e58a53f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:25.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.047+0000 7f4e84cb2700 1 --2- 172.21.15.16:0/1155183825 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e58a512870 0x55e58a5149a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:25.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.047+0000 7f4e7f7fe700 1 -- 172.21.15.16:0/1155183825 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e58a512870 msgr2=0x55e58a5149a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:25.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.047+0000 7f4e7f7fe700 1 --2- 172.21.15.16:0/1155183825 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e58a512870 0x55e58a5149a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:25.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.047+0000 7f4e7f7fe700 1 -- 172.21.15.16:0/1155183825 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e58a53f400 msgr2=0x55e58a53f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:25.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.047+0000 7f4e7f7fe700 1 --2- 172.21.15.16:0/1155183825 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e58a53f400 0x55e58a53f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:25.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.047+0000 7f4e7f7fe700 1 -- 172.21.15.16:0/1155183825 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e58a2e75e0 con 0x55e58a53b700 2023-05-20T01:22:25.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.047+0000 7f4e7ffff700 1 --2- 172.21.15.16:0/1155183825 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e58a53f400 0x55e58a53f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:22:25.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.047+0000 7f4e7f7fe700 1 --2- 172.21.15.16:0/1155183825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e58a53b700 0x55e58a514460 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f4e700597a0 tx=0x7f4e70059480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15a357c257203d92 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:25.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.047+0000 7f4e75ffb700 1 -- 172.21.15.16:0/1155183825 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e70057bb0 con 0x55e58a53b700 2023-05-20T01:22:25.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.047+0000 7f4e75ffb700 1 -- 172.21.15.16:0/1155183825 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e70055f20 con 0x55e58a53b700 2023-05-20T01:22:25.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.047+0000 7f4e75ffb700 1 -- 172.21.15.16:0/1155183825 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e70058f20 con 0x55e58a53b700 2023-05-20T01:22:25.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.047+0000 7f4e928a1b80 1 -- 172.21.15.16:0/1155183825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e58a53b700 msgr2=0x55e58a514460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:25.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.047+0000 7f4e928a1b80 1 --2- 172.21.15.16:0/1155183825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e58a53b700 0x55e58a514460 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f4e700597a0 tx=0x7f4e70059480 comp rx=0 tx=0).stop 2023-05-20T01:22:25.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.048+0000 7f4e928a1b80 1 -- 172.21.15.16:0/1155183825 shutdown_connections 2023-05-20T01:22:25.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.048+0000 7f4e928a1b80 1 --2- 172.21.15.16:0/1155183825 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e58a512870 0x55e58a5149a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:25.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.048+0000 7f4e928a1b80 1 --2- 172.21.15.16:0/1155183825 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e58a53f400 0x55e58a53f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:25.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.048+0000 7f4e928a1b80 1 --2- 172.21.15.16:0/1155183825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e58a53b700 0x55e58a514460 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:25.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.048+0000 7f4e928a1b80 1 -- 172.21.15.16:0/1155183825 >> 172.21.15.16:0/1155183825 conn(0x55e58a509f30 msgr2=0x55e58a55a210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:25.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.048+0000 7f4e928a1b80 1 -- 172.21.15.16:0/1155183825 shutdown_connections 2023-05-20T01:22:25.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.048+0000 7f4e928a1b80 1 -- 172.21.15.16:0/1155183825 wait complete. 2023-05-20T01:22:25.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.048+0000 7f4e928a1b80 1 -- start start 2023-05-20T01:22:25.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.048+0000 7f4e928a1b80 1 Processor -- start 2023-05-20T01:22:25.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.049+0000 7f4e928a1b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e58a512870 0x55e58a538120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:25.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.049+0000 7f4e928a1b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e58a53b700 0x55e58a5149f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:25.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.049+0000 7f4e928a1b80 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e58a53f400 0x55e58a542990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:25.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.049+0000 7f4e928a1b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e58a407890 con 0x55e58a512870 2023-05-20T01:22:25.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.049+0000 7f4e928a1b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e58a407530 con 0x55e58a53b700 2023-05-20T01:22:25.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.049+0000 7f4e928a1b80 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e58a2e98d0 con 0x55e58a53f400 2023-05-20T01:22:25.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.049+0000 7f4e928a1b80 10 client.0 _ll_register_callbacks cb 0x55e58a425730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-05-20T01:22:25.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.049+0000 7f4e7ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e58a512870 0x55e58a538120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:25.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.049+0000 7f4e7ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e58a512870 0x55e58a538120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35322/0 (socket says 172.21.15.16:35322) 2023-05-20T01:22:25.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.049+0000 7f4e7ffff700 1 -- 172.21.15.16:0/4191607444 learned_addr learned my addr 172.21.15.16:0/4191607444 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:22:25.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.049+0000 7f4e7f7fe700 1 --2- 172.21.15.16:0/4191607444 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e58a53b700 0x55e58a5149f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:25.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.049+0000 7f4e84cb2700 1 --2- 172.21.15.16:0/4191607444 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e58a53f400 0x55e58a542990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:25.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.050+0000 7f4e7ffff700 1 -- 172.21.15.16:0/4191607444 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e58a53f400 msgr2=0x55e58a542990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:25.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.050+0000 7f4e7ffff700 1 --2- 172.21.15.16:0/4191607444 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e58a53f400 0x55e58a542990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:25.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.050+0000 7f4e7ffff700 1 -- 172.21.15.16:0/4191607444 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e58a53b700 msgr2=0x55e58a5149f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:25.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.050+0000 7f4e7ffff700 1 --2- 172.21.15.16:0/4191607444 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e58a53b700 0x55e58a5149f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:25.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.050+0000 7f4e7ffff700 1 -- 172.21.15.16:0/4191607444 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e800adda0 con 0x55e58a512870 2023-05-20T01:22:25.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.050+0000 7f4e7ffff700 1 --2- 172.21.15.16:0/4191607444 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e58a512870 0x55e58a538120 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f4e68003a00 tx=0x7f4e68011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ab6fcd65c483d27 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:25.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.050+0000 7f4e5effd700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-05-20T01:22:25.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.050+0000 7f4e5effd700 1 -- 172.21.15.16:0/4191607444 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e680155f0 con 0x55e58a512870 2023-05-20T01:22:25.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.050+0000 7f4e5effd700 1 -- 172.21.15.16:0/4191607444 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(75..75 src has 1..75) v4 ==== 7189+0+0 (secure 0 0 0) 0x7f4e68015750 con 0x55e58a512870 2023-05-20T01:22:25.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.050+0000 7f4e928a1b80 1 -- 172.21.15.16:0/4191607444 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e58a5076f0 con 0x55e58a512870 2023-05-20T01:22:25.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.050+0000 7f4e5effd700 1 -- 172.21.15.16:0/4191607444 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4e68015910 con 0x55e58a512870 2023-05-20T01:22:25.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.050+0000 7f4e928a1b80 10 client.6807 fetch_fsmap learned FSMap version 49 2023-05-20T01:22:25.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.050+0000 7f4e928a1b80 1 -- 172.21.15.16:0/4191607444 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=2+}) v3 -- 0x55e58a42c140 con 0x55e58a512870 2023-05-20T01:22:25.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.050+0000 7f4e5effd700 1 -- 172.21.15.16:0/4191607444 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e58a5076f0 con 0x55e58a512870 2023-05-20T01:22:25.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.050+0000 7f4e5effd700 1 -- 172.21.15.16:0/4191607444 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 49) v1 ==== 3122+0+0 (secure 0 0 0) 0x7f4e68015af0 con 0x55e58a512870 2023-05-20T01:22:25.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.050+0000 7f4e928a1b80 10 client.6807 fetch_fsmap finished waiting for FSMap version 49 2023-05-20T01:22:25.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.050+0000 7f4e928a1b80 10 client.6807 resolve_mds: resolved name 'c' to daemon mds.c 2023-05-20T01:22:25.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.050+0000 7f4e928a1b80 20 client.6807 populate_metadata read hostname 'smithi016' 2023-05-20T01:22:25.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.050+0000 7f4e928a1b80 1 --2- 172.21.15.16:0/4191607444 >> [v2:172.21.15.16:6836/2351975329,v1:172.21.15.16:6837/2351975329] conn(0x55e58a55bd00 0x55e58a55c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:25.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.050+0000 7f4e928a1b80 4 client.6807 mds_command: new command op to 6768 tid=0[{"prefix": "get_command_descriptions"}] 2023-05-20T01:22:25.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.050+0000 7f4e928a1b80 1 -- 172.21.15.16:0/4191607444 --> [v2:172.21.15.16:6836/2351975329,v1:172.21.15.16:6837/2351975329] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e58a428050 con 0x55e58a55bd00 2023-05-20T01:22:25.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.051+0000 7f4e7f7fe700 1 --2- 172.21.15.16:0/4191607444 >> [v2:172.21.15.16:6836/2351975329,v1:172.21.15.16:6837/2351975329] conn(0x55e58a55bd00 0x55e58a55c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:25.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.051+0000 7f4e7f7fe700 1 --2- 172.21.15.16:0/4191607444 >> [v2:172.21.15.16:6836/2351975329,v1:172.21.15.16:6837/2351975329] conn(0x55e58a55bd00 0x55e58a55c0d0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7a78229ae0995a8 server_cookie=c7e2e2d361ab2929 in_seq=0 out_seq=0 2023-05-20T01:22:25.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.051+0000 7f4e5effd700 10 client.6807 ms_handle_connect on v2:172.21.15.16:6836/2351975329 2023-05-20T01:22:25.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.052+0000 7f4e5effd700 1 -- 172.21.15.16:0/4191607444 <== mds.0 v2:172.21.15.16:6836/2351975329 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55e58a428050 con 0x55e58a55bd00 2023-05-20T01:22:25.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.052+0000 7f4e5effd700 10 client.6807 handle_command_reply: tid=0 2023-05-20T01:22:25.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.052+0000 7f4e928a1b80 1 client.6807 shutdown 2023-05-20T01:22:25.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.052+0000 7f4e928a1b80 1 -- 172.21.15.16:0/4191607444 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e58a512870 msgr2=0x55e58a538120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:25.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.052+0000 7f4e928a1b80 1 --2- 172.21.15.16:0/4191607444 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e58a512870 0x55e58a538120 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f4e68003a00 tx=0x7f4e68011000 comp rx=0 tx=0).stop 2023-05-20T01:22:25.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.052+0000 7f4e928a1b80 1 -- 172.21.15.16:0/4191607444 shutdown_connections 2023-05-20T01:22:25.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.052+0000 7f4e928a1b80 1 --2- 172.21.15.16:0/4191607444 >> [v2:172.21.15.16:6836/2351975329,v1:172.21.15.16:6837/2351975329] conn(0x55e58a55bd00 0x55e58a55c0d0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:25.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.052+0000 7f4e928a1b80 1 --2- 172.21.15.16:0/4191607444 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e58a53f400 0x55e58a542990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:25.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.052+0000 7f4e928a1b80 1 --2- 172.21.15.16:0/4191607444 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e58a512870 0x55e58a538120 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:25.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.052+0000 7f4e928a1b80 1 --2- 172.21.15.16:0/4191607444 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e58a53b700 0x55e58a5149f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:25.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.052+0000 7f4e5effd700 0 client.6807 ms_handle_reset on v2:172.21.15.16:6836/2351975329 2023-05-20T01:22:25.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.052+0000 7f4e928a1b80 1 -- 172.21.15.16:0/4191607444 >> 172.21.15.16:0/4191607444 conn(0x55e58a509f30 msgr2=0x55e58a5106a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:25.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.052+0000 7f4e928a1b80 1 -- 172.21.15.16:0/4191607444 shutdown_connections 2023-05-20T01:22:25.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.052+0000 7f4e928a1b80 1 -- 172.21.15.16:0/4191607444 wait complete. 2023-05-20T01:22:25.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.052+0000 7f4e928a1b80 20 client.6807 trim_cache size 0 max 16384 2023-05-20T01:22:25.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.070+0000 7f4e928a1b80 1 Processor -- start 2023-05-20T01:22:25.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.070+0000 7f4e928a1b80 1 -- start start 2023-05-20T01:22:25.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.071+0000 7f4e928a1b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e58a512870 0x55e58a50e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:25.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.071+0000 7f4e928a1b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e58a53b700 0x55e58a50e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:25.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.071+0000 7f4e928a1b80 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e58a53f400 0x55e58a5370e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:25.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.071+0000 7f4e928a1b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e58a50f9b0 con 0x55e58a53b700 2023-05-20T01:22:25.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.071+0000 7f4e928a1b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e58a55aeb0 con 0x55e58a512870 2023-05-20T01:22:25.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.071+0000 7f4e928a1b80 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e58a512120 con 0x55e58a53f400 2023-05-20T01:22:25.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.071+0000 7f4e7f7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e58a53b700 0x55e58a50e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:25.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.071+0000 7f4e7f7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e58a53b700 0x55e58a50e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35332/0 (socket says 172.21.15.16:35332) 2023-05-20T01:22:25.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.071+0000 7f4e7f7fe700 1 -- 172.21.15.16:0/268639162 learned_addr learned my addr 172.21.15.16:0/268639162 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:22:25.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.071+0000 7f4e7ffff700 1 --2- 172.21.15.16:0/268639162 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e58a512870 0x55e58a50e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:25.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.071+0000 7f4e84cb2700 1 --2- 172.21.15.16:0/268639162 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e58a53f400 0x55e58a5370e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:25.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.071+0000 7f4e7f7fe700 1 -- 172.21.15.16:0/268639162 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e58a53f400 msgr2=0x55e58a5370e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:25.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.071+0000 7f4e7f7fe700 1 --2- 172.21.15.16:0/268639162 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e58a53f400 0x55e58a5370e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:25.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.071+0000 7f4e7f7fe700 1 -- 172.21.15.16:0/268639162 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e58a512870 msgr2=0x55e58a50e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:25.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.071+0000 7f4e7f7fe700 1 --2- 172.21.15.16:0/268639162 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e58a512870 0x55e58a50e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:25.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.071+0000 7f4e7f7fe700 1 -- 172.21.15.16:0/268639162 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e58a42be90 con 0x55e58a53b700 2023-05-20T01:22:25.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.072+0000 7f4e7f7fe700 1 --2- 172.21.15.16:0/268639162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e58a53b700 0x55e58a50e660 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f4e70008000 tx=0x7f4e7005c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=294acce0b676c80c server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:25.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.072+0000 7f4e767fc700 1 -- 172.21.15.16:0/268639162 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e70057bb0 con 0x55e58a53b700 2023-05-20T01:22:25.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.072+0000 7f4e767fc700 1 -- 172.21.15.16:0/268639162 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e70058d00 con 0x55e58a53b700 2023-05-20T01:22:25.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.072+0000 7f4e767fc700 1 -- 172.21.15.16:0/268639162 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e700616d0 con 0x55e58a53b700 2023-05-20T01:22:25.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.072+0000 7f4e928a1b80 1 -- 172.21.15.16:0/268639162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e58a53b700 msgr2=0x55e58a50e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:25.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.072+0000 7f4e928a1b80 1 --2- 172.21.15.16:0/268639162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e58a53b700 0x55e58a50e660 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f4e70008000 tx=0x7f4e7005c000 comp rx=0 tx=0).stop 2023-05-20T01:22:25.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.072+0000 7f4e928a1b80 1 -- 172.21.15.16:0/268639162 shutdown_connections 2023-05-20T01:22:25.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.072+0000 7f4e928a1b80 1 --2- 172.21.15.16:0/268639162 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e58a53f400 0x55e58a5370e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:25.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.072+0000 7f4e928a1b80 1 --2- 172.21.15.16:0/268639162 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e58a512870 0x55e58a50e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:25.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.072+0000 7f4e928a1b80 1 --2- 172.21.15.16:0/268639162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e58a53b700 0x55e58a50e660 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:25.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.072+0000 7f4e928a1b80 1 -- 172.21.15.16:0/268639162 >> 172.21.15.16:0/268639162 conn(0x55e58a509f30 msgr2=0x55e58a50b400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:25.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.072+0000 7f4e928a1b80 1 -- 172.21.15.16:0/268639162 shutdown_connections 2023-05-20T01:22:25.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.072+0000 7f4e928a1b80 1 -- 172.21.15.16:0/268639162 wait complete. 2023-05-20T01:22:25.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.073+0000 7f4e928a1b80 1 -- start start 2023-05-20T01:22:25.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.073+0000 7f4e928a1b80 1 Processor -- start 2023-05-20T01:22:25.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.074+0000 7f4e928a1b80 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e58a512870 0x55e58a5bfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:25.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.074+0000 7f4e928a1b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e58a53b700 0x55e58a56bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:25.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.074+0000 7f4e928a1b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e58a53f400 0x55e58a56c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:25.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.074+0000 7f4e928a1b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e58a514170 con 0x55e58a53b700 2023-05-20T01:22:25.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.074+0000 7f4e928a1b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e58a53c8b0 con 0x55e58a53f400 2023-05-20T01:22:25.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.074+0000 7f4e928a1b80 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e58a5c0460 con 0x55e58a512870 2023-05-20T01:22:25.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.074+0000 7f4e928a1b80 10 client.0 _ll_register_callbacks cb 0x7f4e680050d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-05-20T01:22:25.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.074+0000 7f4e7f7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e58a53b700 0x55e58a56bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:25.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.074+0000 7f4e7f7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e58a53b700 0x55e58a56bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35338/0 (socket says 172.21.15.16:35338) 2023-05-20T01:22:25.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.074+0000 7f4e7f7fe700 1 -- 172.21.15.16:0/2101826471 learned_addr learned my addr 172.21.15.16:0/2101826471 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:22:25.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.074+0000 7f4e84cb2700 1 --2- 172.21.15.16:0/2101826471 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e58a53f400 0x55e58a56c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:25.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.074+0000 7f4e7ffff700 1 --2- 172.21.15.16:0/2101826471 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e58a512870 0x55e58a5bfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:25.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.074+0000 7f4e7f7fe700 1 -- 172.21.15.16:0/2101826471 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e58a512870 msgr2=0x55e58a5bfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:25.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.074+0000 7f4e7f7fe700 1 --2- 172.21.15.16:0/2101826471 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e58a512870 0x55e58a5bfd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:25.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.074+0000 7f4e7f7fe700 1 -- 172.21.15.16:0/2101826471 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e58a53f400 msgr2=0x55e58a56c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:25.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.074+0000 7f4e7f7fe700 1 --2- 172.21.15.16:0/2101826471 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e58a53f400 0x55e58a56c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:25.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.074+0000 7f4e7f7fe700 1 -- 172.21.15.16:0/2101826471 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e58a2e75e0 con 0x55e58a53b700 2023-05-20T01:22:25.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.074+0000 7f4e84cb2700 1 --2- 172.21.15.16:0/2101826471 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e58a53f400 0x55e58a56c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:22:25.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.074+0000 7f4e7ffff700 1 --2- 172.21.15.16:0/2101826471 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e58a512870 0x55e58a5bfd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:22:25.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.074+0000 7f4e7f7fe700 1 --2- 172.21.15.16:0/2101826471 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e58a53b700 0x55e58a56bba0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f4e70000c00 tx=0x7f4e7005f070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc2ad9ead1e90d22 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:25.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.074+0000 7f4e5effd700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-05-20T01:22:25.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.075+0000 7f4e5effd700 1 -- 172.21.15.16:0/2101826471 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e70057a10 con 0x55e58a53b700 2023-05-20T01:22:25.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.075+0000 7f4e5effd700 1 -- 172.21.15.16:0/2101826471 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(75..75 src has 1..75) v4 ==== 7189+0+0 (secure 0 0 0) 0x7f4e70058e70 con 0x55e58a53b700 2023-05-20T01:22:25.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.075+0000 7f4e928a1b80 1 -- 172.21.15.16:0/2101826471 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4e70055f20 con 0x55e58a53b700 2023-05-20T01:22:25.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.075+0000 7f4e5effd700 1 -- 172.21.15.16:0/2101826471 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4e700616d0 con 0x55e58a53b700 2023-05-20T01:22:25.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.075+0000 7f4e928a1b80 10 client.6813 fetch_fsmap learned FSMap version 49 2023-05-20T01:22:25.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.075+0000 7f4e928a1b80 1 -- 172.21.15.16:0/2101826471 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=2+}) v3 -- 0x55e58a27cc10 con 0x55e58a53b700 2023-05-20T01:22:25.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.075+0000 7f4e5effd700 1 -- 172.21.15.16:0/2101826471 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e70055f20 con 0x55e58a53b700 2023-05-20T01:22:25.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.075+0000 7f4e5effd700 1 -- 172.21.15.16:0/2101826471 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 49) v1 ==== 3122+0+0 (secure 0 0 0) 0x7f4e70055600 con 0x55e58a53b700 2023-05-20T01:22:25.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.075+0000 7f4e928a1b80 10 client.6813 fetch_fsmap finished waiting for FSMap version 49 2023-05-20T01:22:25.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.075+0000 7f4e928a1b80 10 client.6813 resolve_mds: resolved name 'c' to daemon mds.c 2023-05-20T01:22:25.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.075+0000 7f4e928a1b80 20 client.6813 populate_metadata read hostname 'smithi016' 2023-05-20T01:22:25.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.075+0000 7f4e928a1b80 1 --2- 172.21.15.16:0/2101826471 >> [v2:172.21.15.16:6836/2351975329,v1:172.21.15.16:6837/2351975329] conn(0x55e58a543480 0x55e58a53bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:25.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.075+0000 7f4e928a1b80 4 client.6813 mds_command: new command op to 6768 tid=0[{"prefix": "status"}] 2023-05-20T01:22:25.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.075+0000 7f4e928a1b80 1 -- 172.21.15.16:0/2101826471 --> [v2:172.21.15.16:6836/2351975329,v1:172.21.15.16:6837/2351975329] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55e58a434610 con 0x55e58a543480 2023-05-20T01:22:25.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.075+0000 7f4e7ffff700 1 --2- 172.21.15.16:0/2101826471 >> [v2:172.21.15.16:6836/2351975329,v1:172.21.15.16:6837/2351975329] conn(0x55e58a543480 0x55e58a53bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:25.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.075+0000 7f4e5effd700 10 client.6813 ms_handle_connect on v2:172.21.15.16:6836/2351975329 2023-05-20T01:22:25.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.076+0000 7f4e7ffff700 1 --2- 172.21.15.16:0/2101826471 >> [v2:172.21.15.16:6836/2351975329,v1:172.21.15.16:6837/2351975329] conn(0x55e58a543480 0x55e58a53bad0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4cb34439e7e9df server_cookie=4f883df1b515d47c in_seq=0 out_seq=0 2023-05-20T01:22:25.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.076+0000 7f4e5effd700 1 -- 172.21.15.16:0/2101826471 <== mds.0 v2:172.21.15.16:6836/2351975329 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55e58a434610 con 0x55e58a543480 2023-05-20T01:22:25.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.076+0000 7f4e5effd700 10 client.6813 handle_command_reply: tid=0 2023-05-20T01:22:25.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.076+0000 7f4e928a1b80 1 client.6813 shutdown 2023-05-20T01:22:25.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.076+0000 7f4e928a1b80 1 -- 172.21.15.16:0/2101826471 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e58a53b700 msgr2=0x55e58a56bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:25.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.076+0000 7f4e928a1b80 1 --2- 172.21.15.16:0/2101826471 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e58a53b700 0x55e58a56bba0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f4e70000c00 tx=0x7f4e7005f070 comp rx=0 tx=0).stop 2023-05-20T01:22:25.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.076+0000 7f4e928a1b80 1 -- 172.21.15.16:0/2101826471 shutdown_connections 2023-05-20T01:22:25.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.076+0000 7f4e928a1b80 1 --2- 172.21.15.16:0/2101826471 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e58a53f400 0x55e58a56c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:25.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.076+0000 7f4e928a1b80 1 --2- 172.21.15.16:0/2101826471 >> [v2:172.21.15.16:6836/2351975329,v1:172.21.15.16:6837/2351975329] conn(0x55e58a543480 0x55e58a53bad0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:25.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.076+0000 7f4e928a1b80 1 --2- 172.21.15.16:0/2101826471 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e58a512870 0x55e58a5bfd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:25.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.076+0000 7f4e928a1b80 1 --2- 172.21.15.16:0/2101826471 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e58a53b700 0x55e58a56bba0 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:25.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.076+0000 7f4e5effd700 0 client.6813 ms_handle_reset on v2:172.21.15.16:6836/2351975329 2023-05-20T01:22:25.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.076+0000 7f4e928a1b80 1 -- 172.21.15.16:0/2101826471 >> 172.21.15.16:0/2101826471 conn(0x55e58a509f30 msgr2=0x55e58a5370e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:25.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.076+0000 7f4e928a1b80 1 -- 172.21.15.16:0/2101826471 shutdown_connections 2023-05-20T01:22:25.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.076+0000 7f4e928a1b80 1 -- 172.21.15.16:0/2101826471 wait complete. 2023-05-20T01:22:25.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.076+0000 7f4e928a1b80 20 client.6813 trim_cache size 0 max 16384 2023-05-20T01:22:25.092 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-05-20T01:22:25.092 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "c2245cc5-ecab-4087-a59b-bc81db901676", 2023-05-20T01:22:25.092 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2023-05-20T01:22:25.092 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 6768, 2023-05-20T01:22:25.092 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2023-05-20T01:22:25.093 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2023-05-20T01:22:25.093 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2023-05-20T01:22:25.093 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 3.6934975739999998, 2023-05-20T01:22:25.093 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 49, 2023-05-20T01:22:25.093 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 75, 2023-05-20T01:22:25.093 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 75, 2023-05-20T01:22:25.093 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 4.5322986470000002 2023-05-20T01:22:25.094 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-05-20T01:22:25.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.077+0000 7f4e86715700 1 -- 172.21.15.16:0/1289201177 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f4e6006ace0 msgr2=0x7f4e6006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:25.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.077+0000 7f4e86715700 1 --2- 172.21.15.16:0/1289201177 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f4e6006ace0 0x7f4e6006d190 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f4e680016f0 tx=0x7f4e68009040 comp rx=0 tx=0).stop 2023-05-20T01:22:25.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.077+0000 7f4e86715700 1 -- 172.21.15.16:0/1289201177 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e800cf160 msgr2=0x7f4e800c4780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:25.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.077+0000 7f4e86715700 1 --2- 172.21.15.16:0/1289201177 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e800cf160 0x7f4e800c4780 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f4e700098f0 tx=0x7f4e70004750 comp rx=0 tx=0).stop 2023-05-20T01:22:25.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.077+0000 7f4e86715700 1 -- 172.21.15.16:0/1289201177 shutdown_connections 2023-05-20T01:22:25.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.077+0000 7f4e86715700 1 --2- 172.21.15.16:0/1289201177 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f4e6006ace0 0x7f4e6006d190 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:25.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.077+0000 7f4e86715700 1 --2- 172.21.15.16:0/1289201177 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4e800d16c0 0x7f4e800cd690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:25.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.077+0000 7f4e86715700 1 --2- 172.21.15.16:0/1289201177 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4e800af2d0 0x7f4e801f2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:25.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.077+0000 7f4e86715700 1 --2- 172.21.15.16:0/1289201177 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e800cf160 0x7f4e800c4780 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:25.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.077+0000 7f4e86715700 1 -- 172.21.15.16:0/1289201177 >> 172.21.15.16:0/1289201177 conn(0x7f4e8006ae30 msgr2=0x7f4e8006f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:25.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.077+0000 7f4e86715700 1 -- 172.21.15.16:0/1289201177 shutdown_connections 2023-05-20T01:22:25.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.077+0000 7f4e86715700 1 -- 172.21.15.16:0/1289201177 wait complete. 2023-05-20T01:22:25.097 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-20T01:22:25.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.219+0000 7f9a2d461700 1 Processor -- start 2023-05-20T01:22:25.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.219+0000 7f9a2d461700 1 -- start start 2023-05-20T01:22:25.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.220+0000 7f9a2d461700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9a2806d4b0 0x7f9a280cf890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:25.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.220+0000 7f9a2d461700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f9a2806a510 0x7f9a2806a950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:25.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.220+0000 7f9a2d461700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a280cc510 con 0x7f9a2806f450 2023-05-20T01:22:25.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.220+0000 7f9a2d461700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a2806ae90 con 0x7f9a2806d4b0 2023-05-20T01:22:25.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.220+0000 7f9a2d461700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a28071ed0 con 0x7f9a2806a510 2023-05-20T01:22:25.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.220+0000 7f9a267fc700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9a2806d4b0 0x7f9a280cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:25.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.220+0000 7f9a267fc700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9a2806d4b0 0x7f9a280cf890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35346/0 (socket says 172.21.15.16:35346) 2023-05-20T01:22:25.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.220+0000 7f9a267fc700 1 -- 172.21.15.16:0/1967124756 learned_addr learned my addr 172.21.15.16:0/1967124756 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:22:25.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.220+0000 7f9a277fe700 1 --2- 172.21.15.16:0/1967124756 >> v2:172.21.15.132:3300/0 conn(0x7f9a2806a510 0x7f9a2806a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:25.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.221+0000 7f9a267fc700 1 -- 172.21.15.16:0/1967124756 >> v2:172.21.15.132:3300/0 conn(0x7f9a2806a510 msgr2=0x7f9a2806a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:25.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.221+0000 7f9a267fc700 1 --2- 172.21.15.16:0/1967124756 >> v2:172.21.15.132:3300/0 conn(0x7f9a2806a510 0x7f9a2806a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:25.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.221+0000 7f9a267fc700 1 -- 172.21.15.16:0/1967124756 >> v1:172.21.15.132:6789/0 conn(0x7f9a2806f450 legacy=0x7f9a28161080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:25.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.221+0000 7f9a267fc700 1 -- 172.21.15.16:0/1967124756 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a280721b0 con 0x7f9a2806d4b0 2023-05-20T01:22:25.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.221+0000 7f9a267fc700 1 --2- 172.21.15.16:0/1967124756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a2806d4b0 0x7f9a280cf890 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f9a2806f9f0 tx=0x7f9a18009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1f7c4121af69688 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:25.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.221+0000 7f9a25ffb700 1 -- 172.21.15.16:0/1967124756 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a18004230 con 0x7f9a2806d4b0 2023-05-20T01:22:25.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.221+0000 7f9a25ffb700 1 -- 172.21.15.16:0/1967124756 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a18004390 con 0x7f9a2806d4b0 2023-05-20T01:22:25.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.221+0000 7f9a25ffb700 1 -- 172.21.15.16:0/1967124756 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a180036a0 con 0x7f9a2806d4b0 2023-05-20T01:22:25.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.221+0000 7f9a2d461700 1 -- 172.21.15.16:0/1967124756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a2806d4b0 msgr2=0x7f9a280cf890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:25.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.221+0000 7f9a2d461700 1 --2- 172.21.15.16:0/1967124756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a2806d4b0 0x7f9a280cf890 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f9a2806f9f0 tx=0x7f9a18009bd0 comp rx=0 tx=0).stop 2023-05-20T01:22:25.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.221+0000 7f9a2d461700 1 -- 172.21.15.16:0/1967124756 shutdown_connections 2023-05-20T01:22:25.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.221+0000 7f9a2d461700 1 --2- 172.21.15.16:0/1967124756 >> v2:172.21.15.132:3300/0 conn(0x7f9a2806a510 0x7f9a2806a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:25.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.221+0000 7f9a2d461700 1 --2- 172.21.15.16:0/1967124756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a2806d4b0 0x7f9a280cf890 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:25.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.221+0000 7f9a2d461700 1 -- 172.21.15.16:0/1967124756 >> 172.21.15.16:0/1967124756 conn(0x7f9a280cffb0 msgr2=0x7f9a280d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:25.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.222+0000 7f9a2d461700 1 -- 172.21.15.16:0/1967124756 shutdown_connections 2023-05-20T01:22:25.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.222+0000 7f9a2d461700 1 -- 172.21.15.16:0/1967124756 wait complete. 2023-05-20T01:22:25.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.223+0000 7f9a2d461700 1 Processor -- start 2023-05-20T01:22:25.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.223+0000 7f9a2d461700 1 -- start start 2023-05-20T01:22:25.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.224+0000 7f9a2d461700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a2806a510 0x7f9a28154c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:25.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.224+0000 7f9a2d461700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9a2806d4b0 0x7f9a2815afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:25.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.224+0000 7f9a2d461700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9a2806f450 0x7f9a2815b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:25.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.224+0000 7f9a2d461700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a28159140 con 0x7f9a2806a510 2023-05-20T01:22:25.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.224+0000 7f9a2d461700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a28159420 con 0x7f9a2806f450 2023-05-20T01:22:25.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.224+0000 7f9a2d461700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a28159700 con 0x7f9a2806d4b0 2023-05-20T01:22:25.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.224+0000 7f9a26ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a2806a510 0x7f9a28154c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:25.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.224+0000 7f9a26ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a2806a510 0x7f9a28154c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35350/0 (socket says 172.21.15.16:35350) 2023-05-20T01:22:25.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.224+0000 7f9a26ffd700 1 -- 172.21.15.16:0/3840538627 learned_addr learned my addr 172.21.15.16:0/3840538627 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:22:25.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.224+0000 7f9a277fe700 1 --2- 172.21.15.16:0/3840538627 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9a2806f450 0x7f9a2815b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:25.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.224+0000 7f9a267fc700 1 --2- 172.21.15.16:0/3840538627 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9a2806d4b0 0x7f9a2815afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:25.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.224+0000 7f9a26ffd700 1 -- 172.21.15.16:0/3840538627 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9a2806d4b0 msgr2=0x7f9a2815afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:25.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.224+0000 7f9a26ffd700 1 --2- 172.21.15.16:0/3840538627 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9a2806d4b0 0x7f9a2815afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:25.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.224+0000 7f9a26ffd700 1 -- 172.21.15.16:0/3840538627 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9a2806f450 msgr2=0x7f9a2815b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:25.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.224+0000 7f9a26ffd700 1 --2- 172.21.15.16:0/3840538627 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9a2806f450 0x7f9a2815b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:25.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.224+0000 7f9a26ffd700 1 -- 172.21.15.16:0/3840538627 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a18009550 con 0x7f9a2806a510 2023-05-20T01:22:25.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.224+0000 7f9a26ffd700 1 --2- 172.21.15.16:0/3840538627 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a2806a510 0x7f9a28154c60 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c00b4a0 tx=0x7f9a1c00b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:25.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.224+0000 7f9a0f7fe700 1 -- 172.21.15.16:0/3840538627 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a1c011830 con 0x7f9a2806a510 2023-05-20T01:22:25.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.224+0000 7f9a0f7fe700 1 -- 172.21.15.16:0/3840538627 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a1c011990 con 0x7f9a2806a510 2023-05-20T01:22:25.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.224+0000 7f9a2d461700 1 -- 172.21.15.16:0/3840538627 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a280cbe50 con 0x7f9a2806a510 2023-05-20T01:22:25.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.224+0000 7f9a0f7fe700 1 -- 172.21.15.16:0/3840538627 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a1c011d40 con 0x7f9a2806a510 2023-05-20T01:22:25.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.225+0000 7f9a2d461700 1 -- 172.21.15.16:0/3840538627 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a280cc1a0 con 0x7f9a2806a510 2023-05-20T01:22:25.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.226+0000 7f9a2d461700 1 -- 172.21.15.16:0/3840538627 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a280cc9a0 con 0x7f9a2806a510 2023-05-20T01:22:25.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.226+0000 7f9a0f7fe700 1 -- 172.21.15.16:0/3840538627 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9a1c01a400 con 0x7f9a2806a510 2023-05-20T01:22:25.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.226+0000 7f9a0f7fe700 1 --2- 172.21.15.16:0/3840538627 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f9a1406ace0 0x7f9a1406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:25.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.227+0000 7f9a0f7fe700 1 -- 172.21.15.16:0/3840538627 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7189+0+0 (secure 0 0 0) 0x7f9a1c0895a0 con 0x7f9a2806a510 2023-05-20T01:22:25.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.227+0000 7f9a267fc700 1 --2- 172.21.15.16:0/3840538627 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f9a1406ace0 0x7f9a1406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:25.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.227+0000 7f9a267fc700 1 --2- 172.21.15.16:0/3840538627 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f9a1406ace0 0x7f9a1406d190 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f9a18009130 tx=0x7f9a18008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:25.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.231+0000 7f9a0f7fe700 1 -- 172.21.15.16:0/3840538627 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f9a1c0541f0 con 0x7f9a2806a510 2023-05-20T01:22:25.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.473+0000 7f9a2d461700 1 -- 172.21.15.16:0/3840538627 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9a280747b0 con 0x7f9a2806a510 2023-05-20T01:22:25.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.474+0000 7f9a0f7fe700 1 -- 172.21.15.16:0/3840538627 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+4679 (secure 0 0 0) 0x7f9a1c057810 con 0x7f9a2806a510 2023-05-20T01:22:25.478 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:22:25.479 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":49,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5873,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3769502841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3769502841},{"type":"v1","addr":"172.21.15.16:6835","nonce":3769502841}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45},{"gid":5879,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/688624373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":688624373},{"type":"v1","addr":"172.21.15.132:6834","nonce":688624373}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45},{"gid":6307,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/1655712081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":1655712081},{"type":"v1","addr":"172.21.15.132:6835","nonce":1655712081}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-20T01:22:14.127671+0000","modified":"2023-05-20T01:22:24.390190+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":72,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6768},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6768":{"gid":6768,"name":"c","rank":0,"incarnation":46,"state":"up:active","state_seq":4,"addr":"172.21.15.16:6837/2351975329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2351975329},{"type":"v1","addr":"172.21.15.16:6837","nonce":2351975329}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10,11],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-05-20T01:22:25.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.476+0000 7f9a2d461700 1 -- 172.21.15.16:0/3840538627 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f9a1406ace0 msgr2=0x7f9a1406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:25.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.476+0000 7f9a2d461700 1 --2- 172.21.15.16:0/3840538627 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f9a1406ace0 0x7f9a1406d190 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f9a18009130 tx=0x7f9a18008030 comp rx=0 tx=0).stop 2023-05-20T01:22:25.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.476+0000 7f9a2d461700 1 -- 172.21.15.16:0/3840538627 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a2806a510 msgr2=0x7f9a28154c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:25.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.476+0000 7f9a2d461700 1 --2- 172.21.15.16:0/3840538627 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a2806a510 0x7f9a28154c60 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c00b4a0 tx=0x7f9a1c00b7b0 comp rx=0 tx=0).stop 2023-05-20T01:22:25.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.476+0000 7f9a2d461700 1 -- 172.21.15.16:0/3840538627 shutdown_connections 2023-05-20T01:22:25.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.476+0000 7f9a2d461700 1 --2- 172.21.15.16:0/3840538627 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9a2806f450 0x7f9a2815b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:25.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.476+0000 7f9a2d461700 1 --2- 172.21.15.16:0/3840538627 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a2806a510 0x7f9a28154c60 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:25.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.476+0000 7f9a2d461700 1 --2- 172.21.15.16:0/3840538627 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f9a1406ace0 0x7f9a1406d190 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:25.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.476+0000 7f9a2d461700 1 --2- 172.21.15.16:0/3840538627 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9a2806d4b0 0x7f9a2815afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:25.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.476+0000 7f9a2d461700 1 -- 172.21.15.16:0/3840538627 >> 172.21.15.16:0/3840538627 conn(0x7f9a280cffb0 msgr2=0x7f9a2815c530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:25.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.476+0000 7f9a2d461700 1 -- 172.21.15.16:0/3840538627 shutdown_connections 2023-05-20T01:22:25.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.476+0000 7f9a2d461700 1 -- 172.21.15.16:0/3840538627 wait complete. 2023-05-20T01:22:25.482 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 49 2023-05-20T01:22:25.493 INFO:tasks.cephfs.test_data_scan:{ "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 3, "epoch": 49, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 3, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-05-20T01:22:14.127671+0000", "damaged": [], "data_pools": [ 10, 11 ], "enabled": true, "epoch": 49, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_6768": { "addr": "172.21.15.16:6837/2351975329", "addrs": { "addrvec": [ { "addr": "172.21.15.16:6836", "nonce": 2351975329, "type": "v2" }, { "addr": "172.21.15.16:6837", "nonce": 2351975329, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6768, "incarnation": 46, "join_fscid": -1, "name": "c", "rank": 0, "state": "up:active", "state_seq": 4 } }, "last_failure": 0, "last_failure_osd_epoch": 72, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 9, "modified": "2023-05-20T01:22:24.390190+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 60, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 6768 } } } ], "standbys": [ { "addr": "172.21.15.16:6835/3769502841", "addrs": { "addrvec": [ { "addr": "172.21.15.16:6834", "nonce": 3769502841, "type": "v2" }, { "addr": "172.21.15.16:6835", "nonce": 3769502841, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 45, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5873, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.132:6834/688624373", "addrs": { "addrvec": [ { "addr": "172.21.15.132:6832", "nonce": 688624373, "type": "v2" }, { "addr": "172.21.15.132:6834", "nonce": 688624373, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 45, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5879, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.132:6835/1655712081", "addrs": { "addrvec": [ { "addr": "172.21.15.132:6833", "nonce": 1655712081, "type": "v2" }, { "addr": "172.21.15.132:6835", "nonce": 1655712081, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 45, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6307, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-05-20T01:22:25.494 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-05-20T01:22:25.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.616+0000 7fe8c1a00700 1 Processor -- start 2023-05-20T01:22:25.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.616+0000 7fe8c1a00700 1 -- start start 2023-05-20T01:22:25.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.617+0000 7fe8c1a00700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe8bc0d2c10 0x7fe8bc0ce080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:25.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.617+0000 7fe8c1a00700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8bc0b3b50 con 0x7fe8bc0b4750 2023-05-20T01:22:25.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.617+0000 7fe8c1a00700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8bc0d3070 con 0x7fe8bc0cfc50 2023-05-20T01:22:25.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.617+0000 7fe8c1a00700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8bc0cfad0 con 0x7fe8bc0d2c10 2023-05-20T01:22:25.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.617+0000 7fe8ba7fc700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fe8bc0b4750 0x7fe8bc0cd970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:46886/0 (socket says 172.21.15.16:46886) 2023-05-20T01:22:25.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.617+0000 7fe8ba7fc700 1 -- 172.21.15.16:0/1090330951 learned_addr learned my addr 172.21.15.16:0/1090330951 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:22:25.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.617+0000 7fe8bb7fe700 1 --2- 172.21.15.16:0/1090330951 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe8bc0d2c10 0x7fe8bc0ce080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:25.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.617+0000 7fe8b9ffb700 1 -- 172.21.15.16:0/1090330951 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2485732640 0 0) 0x7fe8bc0b3b50 con 0x7fe8bc0b4750 2023-05-20T01:22:25.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.617+0000 7fe8b9ffb700 1 -- 172.21.15.16:0/1090330951 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8a40035b0 con 0x7fe8bc0b4750 2023-05-20T01:22:25.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.618+0000 7fe8b9ffb700 1 -- 172.21.15.16:0/1090330951 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1852974254 0 0) 0x7fe8bc0d3070 con 0x7fe8bc0cfc50 2023-05-20T01:22:25.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.618+0000 7fe8b9ffb700 1 -- 172.21.15.16:0/1090330951 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8bc0b3b50 con 0x7fe8bc0cfc50 2023-05-20T01:22:25.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.618+0000 7fe8b9ffb700 1 -- 172.21.15.16:0/1090330951 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4102144626 0 0) 0x7fe8a40035b0 con 0x7fe8bc0b4750 2023-05-20T01:22:25.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.618+0000 7fe8b9ffb700 1 -- 172.21.15.16:0/1090330951 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe8bc0d2c10 msgr2=0x7fe8bc0ce080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:25.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.618+0000 7fe8b9ffb700 1 --2- 172.21.15.16:0/1090330951 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe8bc0d2c10 0x7fe8bc0ce080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:25.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.618+0000 7fe8b9ffb700 1 -- 172.21.15.16:0/1090330951 >> v1:172.21.15.132:6789/0 conn(0x7fe8bc0cfc50 legacy=0x7fe8bc0d0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:25.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.618+0000 7fe8b9ffb700 1 -- 172.21.15.16:0/1090330951 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8bc0d0b90 con 0x7fe8bc0b4750 2023-05-20T01:22:25.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.618+0000 7fe8b9ffb700 1 -- 172.21.15.16:0/1090330951 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7fe8ac001670 con 0x7fe8bc0b4750 2023-05-20T01:22:25.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.618+0000 7fe8b9ffb700 1 -- 172.21.15.16:0/1090330951 >> v1:172.21.15.16:6789/0 conn(0x7fe8bc0b4750 legacy=0x7fe8bc0cd970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:25.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.618+0000 7fe8b9ffb700 1 --2- 172.21.15.16:0/1090330951 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8a4003c30 0x7fe8a4004020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:25.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.618+0000 7fe8b9ffb700 1 --2- 172.21.15.16:0/1090330951 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe8a4004850 0x7fe8a4004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:25.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.618+0000 7fe8b9ffb700 1 --2- 172.21.15.16:0/1090330951 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe8a4005200 0x7fe8a400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:25.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.618+0000 7fe8b9ffb700 1 -- 172.21.15.16:0/1090330951 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8bc0cfad0 con 0x7fe8a4003c30 2023-05-20T01:22:25.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.618+0000 7fe8b9ffb700 1 -- 172.21.15.16:0/1090330951 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8a400c0e0 con 0x7fe8a4005200 2023-05-20T01:22:25.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.618+0000 7fe8b9ffb700 1 -- 172.21.15.16:0/1090330951 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8a400c2a0 con 0x7fe8a4004850 2023-05-20T01:22:25.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.618+0000 7fe8bb7fe700 1 --2- 172.21.15.16:0/1090330951 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe8bc0d2c10 0x7fe8bc0ce080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-05-20T01:22:25.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.618+0000 7fe8baffd700 1 --2- 172.21.15.16:0/1090330951 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8a4003c30 0x7fe8a4004020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:25.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.618+0000 7fe8ba7fc700 1 --2- 172.21.15.16:0/1090330951 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe8a4004850 0x7fe8a4004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:25.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.618+0000 7fe8bb7fe700 1 --2- 172.21.15.16:0/1090330951 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe8a4005200 0x7fe8a400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:25.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.618+0000 7fe8baffd700 1 -- 172.21.15.16:0/1090330951 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe8a4004850 msgr2=0x7fe8a4004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:25.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.618+0000 7fe8baffd700 1 --2- 172.21.15.16:0/1090330951 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe8a4004850 0x7fe8a4004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:25.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.618+0000 7fe8baffd700 1 -- 172.21.15.16:0/1090330951 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe8a4005200 msgr2=0x7fe8a400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:25.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.618+0000 7fe8baffd700 1 --2- 172.21.15.16:0/1090330951 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe8a4005200 0x7fe8a400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:25.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.618+0000 7fe8baffd700 1 -- 172.21.15.16:0/1090330951 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8a40035b0 con 0x7fe8a4003c30 2023-05-20T01:22:25.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.618+0000 7fe8ba7fc700 1 --2- 172.21.15.16:0/1090330951 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe8a4004850 0x7fe8a4004cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:22:25.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.618+0000 7fe8baffd700 1 --2- 172.21.15.16:0/1090330951 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8a4003c30 0x7fe8a4004020 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fe8b0001970 tx=0x7fe8a4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c50b69accc632bf8 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:25.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.619+0000 7fe8b9ffb700 1 -- 172.21.15.16:0/1090330951 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b00052d0 con 0x7fe8a4003c30 2023-05-20T01:22:25.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.619+0000 7fe8b9ffb700 1 -- 172.21.15.16:0/1090330951 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8b0005430 con 0x7fe8a4003c30 2023-05-20T01:22:25.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.619+0000 7fe8b9ffb700 1 -- 172.21.15.16:0/1090330951 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b00038c0 con 0x7fe8a4003c30 2023-05-20T01:22:25.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.619+0000 7fe8c1a00700 1 -- 172.21.15.16:0/1090330951 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8a4003c30 msgr2=0x7fe8a4004020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:25.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.619+0000 7fe8c1a00700 1 --2- 172.21.15.16:0/1090330951 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8a4003c30 0x7fe8a4004020 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fe8b0001970 tx=0x7fe8a4005880 comp rx=0 tx=0).stop 2023-05-20T01:22:25.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.619+0000 7fe8bb7fe700 1 -- 172.21.15.16:0/1090330951 reap_dead start 2023-05-20T01:22:25.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.619+0000 7fe8c1a00700 1 -- 172.21.15.16:0/1090330951 shutdown_connections 2023-05-20T01:22:25.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.619+0000 7fe8c1a00700 1 -- 172.21.15.16:0/1090330951 >> 172.21.15.16:0/1090330951 conn(0x7fe8bc09c590 msgr2=0x7fe8bc0d3240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:25.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.619+0000 7fe8c1a00700 1 -- 172.21.15.16:0/1090330951 shutdown_connections 2023-05-20T01:22:25.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.619+0000 7fe8c1a00700 1 -- 172.21.15.16:0/1090330951 wait complete. 2023-05-20T01:22:25.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.621+0000 7fe8c1a00700 1 Processor -- start 2023-05-20T01:22:25.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.621+0000 7fe8c1a00700 1 -- start start 2023-05-20T01:22:25.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.622+0000 7fe8c1a00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8a4003c30 0x7fe8bc1f3130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:25.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.622+0000 7fe8c1a00700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe8bc0b8310 0x7fe8bc0ccac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:25.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.622+0000 7fe8c1a00700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe8bc1efc00 0x7fe8bc15f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:25.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.622+0000 7fe8c1a00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8bc0b7550 con 0x7fe8a4003c30 2023-05-20T01:22:25.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.622+0000 7fe8c1a00700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8bc0b7830 con 0x7fe8bc1efc00 2023-05-20T01:22:25.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.622+0000 7fe8c1a00700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8bc0b7b10 con 0x7fe8bc0b8310 2023-05-20T01:22:25.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.622+0000 7fe8baffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8a4003c30 0x7fe8bc1f3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:25.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.622+0000 7fe8baffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8a4003c30 0x7fe8bc1f3130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35370/0 (socket says 172.21.15.16:35370) 2023-05-20T01:22:25.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.622+0000 7fe8baffd700 1 -- 172.21.15.16:0/1369546283 learned_addr learned my addr 172.21.15.16:0/1369546283 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:22:25.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.622+0000 7fe8ba7fc700 1 --2- 172.21.15.16:0/1369546283 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe8bc0b8310 0x7fe8bc0ccac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:25.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.622+0000 7fe8bb7fe700 1 --2- 172.21.15.16:0/1369546283 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe8bc1efc00 0x7fe8bc15f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:25.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.623+0000 7fe8baffd700 1 -- 172.21.15.16:0/1369546283 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe8bc0b8310 msgr2=0x7fe8bc0ccac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:25.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.623+0000 7fe8baffd700 1 --2- 172.21.15.16:0/1369546283 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe8bc0b8310 0x7fe8bc0ccac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:25.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.623+0000 7fe8baffd700 1 -- 172.21.15.16:0/1369546283 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe8bc1efc00 msgr2=0x7fe8bc15f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:25.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.623+0000 7fe8baffd700 1 --2- 172.21.15.16:0/1369546283 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe8bc1efc00 0x7fe8bc15f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:25.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.623+0000 7fe8baffd700 1 -- 172.21.15.16:0/1369546283 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8bc0b4380 con 0x7fe8a4003c30 2023-05-20T01:22:25.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.623+0000 7fe8baffd700 1 --2- 172.21.15.16:0/1369546283 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8a4003c30 0x7fe8bc1f3130 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fe8b000c3c0 tx=0x7fe8b0001310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:25.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.623+0000 7fe8a3fff700 1 -- 172.21.15.16:0/1369546283 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b0007240 con 0x7fe8a4003c30 2023-05-20T01:22:25.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.623+0000 7fe8a3fff700 1 -- 172.21.15.16:0/1369546283 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8b00073a0 con 0x7fe8a4003c30 2023-05-20T01:22:25.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.623+0000 7fe8c1a00700 1 -- 172.21.15.16:0/1369546283 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8bc15d0b0 con 0x7fe8a4003c30 2023-05-20T01:22:25.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.623+0000 7fe8a3fff700 1 -- 172.21.15.16:0/1369546283 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b0007750 con 0x7fe8a4003c30 2023-05-20T01:22:25.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.623+0000 7fe8c1a00700 1 -- 172.21.15.16:0/1369546283 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8bc0d4000 con 0x7fe8a4003c30 2023-05-20T01:22:25.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.624+0000 7fe8c1a00700 1 -- 172.21.15.16:0/1369546283 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8bc04e8c0 con 0x7fe8a4003c30 2023-05-20T01:22:25.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.625+0000 7fe8a3fff700 1 -- 172.21.15.16:0/1369546283 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe8b0003680 con 0x7fe8a4003c30 2023-05-20T01:22:25.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.625+0000 7fe8a3fff700 1 --2- 172.21.15.16:0/1369546283 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe8a806ad10 0x7fe8a806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:25.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.625+0000 7fe8a3fff700 1 -- 172.21.15.16:0/1369546283 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7189+0+0 (secure 0 0 0) 0x7fe8b008ab60 con 0x7fe8a4003c30 2023-05-20T01:22:25.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.625+0000 7fe8ba7fc700 1 --2- 172.21.15.16:0/1369546283 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe8a806ad10 0x7fe8a806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:25.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.626+0000 7fe8ba7fc700 1 --2- 172.21.15.16:0/1369546283 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe8a806ad10 0x7fe8a806d1c0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac001eb0 tx=0x7fe8ac007040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:25.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.629+0000 7fe8a3fff700 1 -- 172.21.15.16:0/1369546283 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fe8b00559c0 con 0x7fe8a4003c30 2023-05-20T01:22:25.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.875+0000 7fe8c1a00700 1 -- 172.21.15.16:0/1369546283 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fe8bc15f210 con 0x7fe8a4003c30 2023-05-20T01:22:25.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.875+0000 7fe8a3fff700 1 -- 172.21.15.16:0/1369546283 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v49) v1 ==== 53+0+91 (secure 0 0 0) 0x7fe8b0058fe0 con 0x7fe8a4003c30 2023-05-20T01:22:25.879 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_extra ] 2023-05-20T01:22:25.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.877+0000 7fe8c1a00700 1 -- 172.21.15.16:0/1369546283 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe8a806ad10 msgr2=0x7fe8a806d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:25.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.877+0000 7fe8c1a00700 1 --2- 172.21.15.16:0/1369546283 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe8a806ad10 0x7fe8a806d1c0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac001eb0 tx=0x7fe8ac007040 comp rx=0 tx=0).stop 2023-05-20T01:22:25.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.877+0000 7fe8c1a00700 1 -- 172.21.15.16:0/1369546283 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8a4003c30 msgr2=0x7fe8bc1f3130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:25.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.877+0000 7fe8c1a00700 1 --2- 172.21.15.16:0/1369546283 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8a4003c30 0x7fe8bc1f3130 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fe8b000c3c0 tx=0x7fe8b0001310 comp rx=0 tx=0).stop 2023-05-20T01:22:25.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.877+0000 7fe8c1a00700 1 -- 172.21.15.16:0/1369546283 shutdown_connections 2023-05-20T01:22:25.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.877+0000 7fe8c1a00700 1 --2- 172.21.15.16:0/1369546283 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe8bc1efc00 0x7fe8bc15f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:25.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.877+0000 7fe8c1a00700 1 --2- 172.21.15.16:0/1369546283 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8a4003c30 0x7fe8bc1f3130 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:25.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.877+0000 7fe8c1a00700 1 --2- 172.21.15.16:0/1369546283 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe8a806ad10 0x7fe8a806d1c0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:25.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.877+0000 7fe8c1a00700 1 --2- 172.21.15.16:0/1369546283 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe8bc0b8310 0x7fe8bc0ccac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:25.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.877+0000 7fe8c1a00700 1 -- 172.21.15.16:0/1369546283 >> 172.21.15.16:0/1369546283 conn(0x7fe8bc09c590 msgr2=0x7fe8bc0b2010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:25.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.877+0000 7fe8c1a00700 1 -- 172.21.15.16:0/1369546283 shutdown_connections 2023-05-20T01:22:25.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:25.877+0000 7fe8c1a00700 1 -- 172.21.15.16:0/1369546283 wait complete. 2023-05-20T01:22:25.894 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-05-20T01:22:25.896 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-05-20T01:22:25.896 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2023-05-20T01:22:25.896 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-05-20T01:22:25.897 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-05-20T01:22:25.897 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-05-20T01:22:25.897 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-05-20T01:22:25.897 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-05-20T01:22:25.898 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-05-20T01:22:25.898 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:22:25.899 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2023-05-20T01:22:25.917 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-05-20T01:22:25.917 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-05-20T01:22:25.917 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2023-05-20T01:22:25.917 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-05-20T01:22:25.918 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2023-05-20T01:22:25.919 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-05-20T01:22:25.919 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-05-20T01:22:25.919 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2023-05-20T01:22:25.919 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-05-20T01:22:25.919 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2023-05-20T01:22:25.919 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-05-20T01:22:25.920 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2023-05-20T01:22:25.920 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-05-20T01:22:25.920 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 42088sec preferred_lft 42088sec 2023-05-20T01:22:25.920 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link 2023-05-20T01:22:25.921 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-05-20T01:22:25.921 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-05-20T01:22:25.921 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2023-05-20T01:22:25.921 INFO:teuthology.orchestra.run.smithi016.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-05-20T01:22:25.921 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0a:2e:0c:6b:5d:42 brd ff:ff:ff:ff:ff:ff 2023-05-20T01:22:25.921 INFO:teuthology.orchestra.run.smithi016.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-05-20T01:22:25.921 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-05-20T01:22:25.922 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::7c00:b2ff:fef4:946/64 scope link 2023-05-20T01:22:25.922 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-05-20T01:22:25.922 INFO:teuthology.orchestra.run.smithi016.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-05-20T01:22:25.922 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether d2:e4:70:a7:b9:4b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-05-20T01:22:25.922 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::d0e4:70ff:fea7:b94b/64 scope link 2023-05-20T01:22:25.922 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-05-20T01:22:25.922 INFO:teuthology.orchestra.run.smithi016.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-05-20T01:22:25.923 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0a:2e:0c:6b:5d:42 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-05-20T01:22:25.923 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::82e:cff:fe6b:5d42/64 scope link 2023-05-20T01:22:25.923 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-05-20T01:22:25.923 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:22:25.924 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-05-20T01:22:25.983 INFO:teuthology.orchestra.run.smithi016.stdout:1 2023-05-20T01:22:25.985 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:22:25.986 DEBUG:teuthology.orchestra.run.smithi016:> route 2023-05-20T01:22:26.045 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2023-05-20T01:22:26.045 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-05-20T01:22:26.045 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-05-20T01:22:26.045 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-05-20T01:22:26.046 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-05-20T01:22:26.046 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:22:26.046 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2023-05-20T01:22:26.047 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2023-05-20T01:22:26.047 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-05-20T01:22:26.047 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-05-20T01:22:26.048 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-05-20T01:22:26.048 DEBUG:teuthology.orchestra.run.smithi016:> ') 2023-05-20T01:22:26.151 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:22:26.151 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2023-05-20T01:22:26.168 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-05-20T01:22:26.168 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-05-20T01:22:26.169 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-05-20T01:22:26.169 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:22:26.169 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.0 up 2023-05-20T01:22:26.236 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-05-20T01:22:26.237 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-05-20T01:22:26.293 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-05-20T01:22:26.294 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-05-20T01:22:26.295 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-05-20T01:22:26.350 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-05-20T01:22:26.351 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:22:26.351 DEBUG:teuthology.orchestra.run.smithi016:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-05-20T01:22:26.393 INFO:tasks.ceph.mds.c.smithi016.stderr:2023-05-20T01:22:26.389+0000 7fd0f461c700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-05-20T01:22:26.436 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-05-20T01:22:26.437 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-05-20T01:22:26.466 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-05-20T01:22:26.466 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-05-20T01:22:26.466 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.132:3300,172.21.15.132:3301" 2023-05-20T01:22:26.467 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: trying mount with new device syntax: 0@c2245cc5-ecab-4087-a59b-bc81db901676.cephfs=/ 2023-05-20T01:22:26.467 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.16:3300/172.21.15.132:3300/172.21.15.132:3301" will pass to kernel 2023-05-20T01:22:26.467 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: v2 mount support check returned 2 2023-05-20T01:22:26.467 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: kernel does not support v2 syntax 2023-05-20T01:22:26.467 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: switching to using v1 address with old syntax 2023-05-20T01:22:26.499 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-05-20T01:22:26.500 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-05-20T01:22:26.500 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.132:3300,172.21.15.132:3301" 2023-05-20T01:22:26.500 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: trying mount with new device syntax: 0@c2245cc5-ecab-4087-a59b-bc81db901676.cephfs=/ 2023-05-20T01:22:26.500 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.16:3300/172.21.15.132:3300/172.21.15.132:3301" will pass to kernel 2023-05-20T01:22:26.500 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: v2 mount support check returned 2 2023-05-20T01:22:26.500 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: kernel does not support v2 syntax 2023-05-20T01:22:26.501 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: switching to using v1 address with old syntax 2023-05-20T01:22:26.501 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:6789,172.21.15.132:6789,172.21.15.132:6790" 2023-05-20T01:22:26.501 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.16:6789,172.21.15.132:6789,172.21.15.132:6790:/ 2023-05-20T01:22:26.501 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=c2245cc5-ecab-4087-a59b-bc81db901676" will pass to kernel 2023-05-20T01:22:26.506 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-05-20T01:22:26.506 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:22:26.507 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-05-20T01:22:26.573 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:22:26.574 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-05-20T01:22:26.657 INFO:teuthology.orchestra.run.smithi016.stdout:client6828 2023-05-20T01:22:26.657 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-05-20T01:22:26.658 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2023-05-20T01:22:26.714 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start / repair,recursive 2023-05-20T01:22:26.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.878+0000 7f71c3acb700 1 Processor -- start 2023-05-20T01:22:26.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.878+0000 7f71c3acb700 1 -- start start 2023-05-20T01:22:26.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.879+0000 7f71c3acb700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f71bc0cbbd0 0x7f71bc06cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:26.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.879+0000 7f71c3acb700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f71bc06d190 0x7f71bc0cd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:26.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.879+0000 7f71c3acb700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71bc159c60 con 0x7f71bc0b2090 2023-05-20T01:22:26.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.879+0000 7f71c3acb700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71bc06d610 con 0x7f71bc06d190 2023-05-20T01:22:26.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.879+0000 7f71c3acb700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71bc0cc140 con 0x7f71bc0cbbd0 2023-05-20T01:22:26.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.879+0000 7f71c2068700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f71bc06d190 0x7f71bc0cd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:26.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.879+0000 7f71c1867700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f71bc0b2090 0x7f71bc071bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:46910/0 (socket says 172.21.15.16:46910) 2023-05-20T01:22:26.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.879+0000 7f71c1867700 1 -- 172.21.15.16:0/1847008210 learned_addr learned my addr 172.21.15.16:0/1847008210 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:22:26.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.879+0000 7f71c1066700 1 --2- 172.21.15.16:0/1847008210 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f71bc0cbbd0 0x7f71bc06cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:26.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.879+0000 7f71c2068700 1 -- 172.21.15.16:0/1847008210 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f71bc0cbbd0 msgr2=0x7f71bc06cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:26.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.879+0000 7f71c2068700 1 --2- 172.21.15.16:0/1847008210 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f71bc0cbbd0 0x7f71bc06cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:26.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.879+0000 7f71c2068700 1 -- 172.21.15.16:0/1847008210 >> v1:172.21.15.16:6789/0 conn(0x7f71bc0b2090 legacy=0x7f71bc071bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:26.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.879+0000 7f71c0865700 1 -- 172.21.15.16:0/1847008210 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2187583186 0 0) 0x7f71bc159c60 con 0x7f71bc0b2090 2023-05-20T01:22:26.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.879+0000 7f71c2068700 1 -- 172.21.15.16:0/1847008210 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71bc0cc420 con 0x7f71bc06d190 2023-05-20T01:22:26.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.879+0000 7f71c2068700 1 --2- 172.21.15.16:0/1847008210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71bc06d190 0x7f71bc0cd3c0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f71b4009d90 tx=0x7f71b400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9d6fda09bf12d4c server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:26.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.880+0000 7f71c0865700 1 -- 172.21.15.16:0/1847008210 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b400e070 con 0x7f71bc06d190 2023-05-20T01:22:26.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.880+0000 7f71c0865700 1 -- 172.21.15.16:0/1847008210 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71b400bbd0 con 0x7f71bc06d190 2023-05-20T01:22:26.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.880+0000 7f71c0865700 1 -- 172.21.15.16:0/1847008210 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b4012430 con 0x7f71bc06d190 2023-05-20T01:22:26.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.880+0000 7f71c3acb700 1 -- 172.21.15.16:0/1847008210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71bc06d190 msgr2=0x7f71bc0cd3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:26.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.880+0000 7f71c3acb700 1 --2- 172.21.15.16:0/1847008210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71bc06d190 0x7f71bc0cd3c0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f71b4009d90 tx=0x7f71b400b330 comp rx=0 tx=0).stop 2023-05-20T01:22:26.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.880+0000 7f71c3acb700 1 -- 172.21.15.16:0/1847008210 shutdown_connections 2023-05-20T01:22:26.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.880+0000 7f71c3acb700 1 --2- 172.21.15.16:0/1847008210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71bc06d190 0x7f71bc0cd3c0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:26.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.880+0000 7f71c3acb700 1 --2- 172.21.15.16:0/1847008210 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f71bc0cbbd0 0x7f71bc06cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:26.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.880+0000 7f71c3acb700 1 -- 172.21.15.16:0/1847008210 >> 172.21.15.16:0/1847008210 conn(0x7f71bc0661b0 msgr2=0x7f71bc0665b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:26.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.880+0000 7f71c3acb700 1 -- 172.21.15.16:0/1847008210 shutdown_connections 2023-05-20T01:22:26.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.880+0000 7f71c3acb700 1 -- 172.21.15.16:0/1847008210 wait complete. 2023-05-20T01:22:26.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.881+0000 7f71c3acb700 1 Processor -- start 2023-05-20T01:22:26.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.881+0000 7f71c3acb700 1 -- start start 2023-05-20T01:22:26.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.882+0000 7f71c3acb700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71bc06d190 0x7f71bc149ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:26.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.882+0000 7f71c3acb700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f71bc0b2090 0x7f71bc0c74e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:26.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.882+0000 7f71c3acb700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f71bc0cbbd0 0x7f71bc0cddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:26.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.882+0000 7f71c3acb700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71bc0c82f0 con 0x7f71bc06d190 2023-05-20T01:22:26.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.882+0000 7f71c3acb700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71bc15b3a0 con 0x7f71bc0b2090 2023-05-20T01:22:26.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.882+0000 7f71c3acb700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71bc0c7d30 con 0x7f71bc0cbbd0 2023-05-20T01:22:26.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.882+0000 7f71c1867700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71bc06d190 0x7f71bc149ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:26.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.882+0000 7f71c1867700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71bc06d190 0x7f71bc149ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35388/0 (socket says 172.21.15.16:35388) 2023-05-20T01:22:26.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.882+0000 7f71c1867700 1 -- 172.21.15.16:0/3121928475 learned_addr learned my addr 172.21.15.16:0/3121928475 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:22:26.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.883+0000 7f71c2068700 1 --2- 172.21.15.16:0/3121928475 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f71bc0cbbd0 0x7f71bc0cddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:26.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.883+0000 7f71c1066700 1 --2- 172.21.15.16:0/3121928475 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f71bc0b2090 0x7f71bc0c74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:26.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.883+0000 7f71c1867700 1 -- 172.21.15.16:0/3121928475 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f71bc0cbbd0 msgr2=0x7f71bc0cddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:26.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.883+0000 7f71c1867700 1 --2- 172.21.15.16:0/3121928475 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f71bc0cbbd0 0x7f71bc0cddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:26.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.883+0000 7f71c1867700 1 -- 172.21.15.16:0/3121928475 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f71bc0b2090 msgr2=0x7f71bc0c74e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:26.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.883+0000 7f71c1867700 1 --2- 172.21.15.16:0/3121928475 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f71bc0b2090 0x7f71bc0c74e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:26.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.883+0000 7f71c1867700 1 -- 172.21.15.16:0/3121928475 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71b4009a70 con 0x7f71bc06d190 2023-05-20T01:22:26.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.883+0000 7f71c1867700 1 --2- 172.21.15.16:0/3121928475 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71bc06d190 0x7f71bc149ab0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f71ac0010e0 tx=0x7f71ac00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:26.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.883+0000 7f71b27fc700 1 -- 172.21.15.16:0/3121928475 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71ac0074e0 con 0x7f71bc06d190 2023-05-20T01:22:26.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.883+0000 7f71b27fc700 1 -- 172.21.15.16:0/3121928475 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71ac007660 con 0x7f71bc06d190 2023-05-20T01:22:26.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.883+0000 7f71c3acb700 1 -- 172.21.15.16:0/3121928475 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71bc0cc9e0 con 0x7f71bc06d190 2023-05-20T01:22:26.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.883+0000 7f71b27fc700 1 -- 172.21.15.16:0/3121928475 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71ac005a70 con 0x7f71bc06d190 2023-05-20T01:22:26.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.883+0000 7f71c3acb700 1 -- 172.21.15.16:0/3121928475 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71bc0cc700 con 0x7f71bc06d190 2023-05-20T01:22:26.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.885+0000 7f71b27fc700 1 -- 172.21.15.16:0/3121928475 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f71ac005bd0 con 0x7f71bc06d190 2023-05-20T01:22:26.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.885+0000 7f71b27fc700 1 --2- 172.21.15.16:0/3121928475 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f71a006ace0 0x7f71a006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:26.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.885+0000 7f71b27fc700 1 -- 172.21.15.16:0/3121928475 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7189+0+0 (secure 0 0 0) 0x7f71ac088e90 con 0x7f71bc06d190 2023-05-20T01:22:26.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.885+0000 7f71c1066700 1 --2- 172.21.15.16:0/3121928475 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f71a006ace0 0x7f71a006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:26.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.886+0000 7f71c1066700 1 --2- 172.21.15.16:0/3121928475 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f71a006ace0 0x7f71a006d190 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f71bc156670 tx=0x7f71b8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:26.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.887+0000 7f71cfc57b80 1 Processor -- start 2023-05-20T01:22:26.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.887+0000 7f71cfc57b80 1 -- start start 2023-05-20T01:22:26.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.888+0000 7f71cfc57b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f6f7b234d0 0x55f6f7b238a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:26.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.888+0000 7f71cfc57b80 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55f6f7b1f7d0 0x55f6f7af8530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:26.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.888+0000 7f71cfc57b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f6f7af6940 0x55f6f7af8a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:26.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.888+0000 7f71cfc57b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6f79fe0e0 con 0x55f6f7af6940 2023-05-20T01:22:26.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.888+0000 7f71cfc57b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6f7a50920 con 0x55f6f7b234d0 2023-05-20T01:22:26.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.888+0000 7f71cfc57b80 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f6f7a49db0 con 0x55f6f7b1f7d0 2023-05-20T01:22:26.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.888+0000 7f71c2068700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f6f7af6940 0x55f6f7af8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:26.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.888+0000 7f71c2068700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f6f7af6940 0x55f6f7af8a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35400/0 (socket says 172.21.15.16:35400) 2023-05-20T01:22:26.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.888+0000 7f71c2068700 1 -- 172.21.15.16:0/372154828 learned_addr learned my addr 172.21.15.16:0/372154828 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:22:26.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.888+0000 7f71c1066700 1 --2- 172.21.15.16:0/372154828 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55f6f7b1f7d0 0x55f6f7af8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:26.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.888+0000 7f71c2068700 1 -- 172.21.15.16:0/372154828 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55f6f7b1f7d0 msgr2=0x55f6f7af8530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:26.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.888+0000 7f71c2068700 1 --2- 172.21.15.16:0/372154828 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55f6f7b1f7d0 0x55f6f7af8530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:26.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.888+0000 7f71c2068700 1 -- 172.21.15.16:0/372154828 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f6f7b234d0 msgr2=0x55f6f7b238a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:26.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.888+0000 7f71c1867700 1 --2- 172.21.15.16:0/372154828 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f6f7b234d0 0x55f6f7b238a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:26.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.888+0000 7f71c2068700 1 --2- 172.21.15.16:0/372154828 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f6f7b234d0 0x55f6f7b238a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:26.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.888+0000 7f71c2068700 1 -- 172.21.15.16:0/372154828 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f6f78cb6b0 con 0x55f6f7af6940 2023-05-20T01:22:26.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.889+0000 7f71c2068700 1 --2- 172.21.15.16:0/372154828 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f6f7af6940 0x55f6f7af8a70 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f71b4009f80 tx=0x7f71b4010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acad4c0117ff399a server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:26.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.889+0000 7f719f7fe700 1 -- 172.21.15.16:0/372154828 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b4003c20 con 0x55f6f7af6940 2023-05-20T01:22:26.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.889+0000 7f719f7fe700 1 -- 172.21.15.16:0/372154828 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71b4021070 con 0x55f6f7af6940 2023-05-20T01:22:26.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.889+0000 7f719f7fe700 1 -- 172.21.15.16:0/372154828 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b40125e0 con 0x55f6f7af6940 2023-05-20T01:22:26.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.889+0000 7f71cfc57b80 1 -- 172.21.15.16:0/372154828 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f6f7af6940 msgr2=0x55f6f7af8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:26.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.889+0000 7f71cfc57b80 1 --2- 172.21.15.16:0/372154828 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f6f7af6940 0x55f6f7af8a70 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f71b4009f80 tx=0x7f71b4010000 comp rx=0 tx=0).stop 2023-05-20T01:22:26.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.889+0000 7f71cfc57b80 1 -- 172.21.15.16:0/372154828 shutdown_connections 2023-05-20T01:22:26.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.889+0000 7f71cfc57b80 1 --2- 172.21.15.16:0/372154828 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f6f7af6940 0x55f6f7af8a70 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:26.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.889+0000 7f71cfc57b80 1 --2- 172.21.15.16:0/372154828 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f6f7b234d0 0x55f6f7b238a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:26.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.889+0000 7f71cfc57b80 1 --2- 172.21.15.16:0/372154828 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55f6f7b1f7d0 0x55f6f7af8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:26.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.889+0000 7f71cfc57b80 1 -- 172.21.15.16:0/372154828 >> 172.21.15.16:0/372154828 conn(0x55f6f7aee000 msgr2=0x55f6f7b3e2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:26.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.890+0000 7f71cfc57b80 1 -- 172.21.15.16:0/372154828 shutdown_connections 2023-05-20T01:22:26.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.890+0000 7f71cfc57b80 1 -- 172.21.15.16:0/372154828 wait complete. 2023-05-20T01:22:26.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.890+0000 7f71cfc57b80 1 -- start start 2023-05-20T01:22:26.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.890+0000 7f71cfc57b80 1 Processor -- start 2023-05-20T01:22:26.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.891+0000 7f71cfc57b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f6f7af6940 0x55f6f7b1ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:26.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.891+0000 7f71cfc57b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f6f7b1f7d0 0x55f6f7af9150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:26.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.891+0000 7f71cfc57b80 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55f6f7b234d0 0x55f6f7af9690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:26.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.891+0000 7f71cfc57b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6f79eb960 con 0x55f6f7b1f7d0 2023-05-20T01:22:26.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.891+0000 7f71cfc57b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6f79eb600 con 0x55f6f7af6940 2023-05-20T01:22:26.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.891+0000 7f71cfc57b80 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f6f78cd9a0 con 0x55f6f7b234d0 2023-05-20T01:22:26.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.891+0000 7f71cfc57b80 10 client.0 _ll_register_callbacks cb 0x55f6f7a09800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-05-20T01:22:26.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.891+0000 7f71c1066700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f6f7b1f7d0 0x55f6f7af9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:26.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.891+0000 7f71c1066700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f6f7b1f7d0 0x55f6f7af9150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35404/0 (socket says 172.21.15.16:35404) 2023-05-20T01:22:26.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.891+0000 7f71c1066700 1 -- 172.21.15.16:0/346440557 learned_addr learned my addr 172.21.15.16:0/346440557 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:22:26.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.891+0000 7f71c1867700 1 --2- 172.21.15.16:0/346440557 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f6f7af6940 0x55f6f7b1ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:26.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.891+0000 7f71c2068700 1 --2- 172.21.15.16:0/346440557 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55f6f7b234d0 0x55f6f7af9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:26.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.891+0000 7f71c1066700 1 -- 172.21.15.16:0/346440557 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55f6f7b234d0 msgr2=0x55f6f7af9690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:26.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.891+0000 7f71c1066700 1 --2- 172.21.15.16:0/346440557 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55f6f7b234d0 0x55f6f7af9690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:26.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.891+0000 7f71c1066700 1 -- 172.21.15.16:0/346440557 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f6f7af6940 msgr2=0x55f6f7b1ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:26.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.892+0000 7f71c1066700 1 --2- 172.21.15.16:0/346440557 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f6f7af6940 0x55f6f7b1ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:26.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.892+0000 7f71c1066700 1 -- 172.21.15.16:0/346440557 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71bc0cc420 con 0x55f6f7b1f7d0 2023-05-20T01:22:26.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.892+0000 7f71c2068700 1 --2- 172.21.15.16:0/346440557 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55f6f7b234d0 0x55f6f7af9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:22:26.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.892+0000 7f71c1066700 1 --2- 172.21.15.16:0/346440557 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f6f7b1f7d0 0x55f6f7af9150 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f71bc0b16e0 tx=0x7f71b80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=423ddcc54ddbea7d server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:26.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.892+0000 7f719cff9700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-05-20T01:22:26.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.892+0000 7f719cff9700 1 -- 172.21.15.16:0/346440557 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b8014e50 con 0x55f6f7b1f7d0 2023-05-20T01:22:26.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.892+0000 7f719cff9700 1 -- 172.21.15.16:0/346440557 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(75..75 src has 1..75) v4 ==== 7189+0+0 (secure 0 0 0) 0x7f71b8021410 con 0x55f6f7b1f7d0 2023-05-20T01:22:26.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.892+0000 7f71cfc57b80 1 -- 172.21.15.16:0/346440557 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f6f7aeb7c0 con 0x55f6f7b1f7d0 2023-05-20T01:22:26.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.892+0000 7f719cff9700 1 -- 172.21.15.16:0/346440557 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f71b80215d0 con 0x55f6f7b1f7d0 2023-05-20T01:22:26.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.892+0000 7f71cfc57b80 10 client.6843 fetch_fsmap learned FSMap version 49 2023-05-20T01:22:26.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.892+0000 7f71cfc57b80 1 -- 172.21.15.16:0/346440557 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=2+}) v3 -- 0x55f6f7a10210 con 0x55f6f7b1f7d0 2023-05-20T01:22:26.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.892+0000 7f719cff9700 1 -- 172.21.15.16:0/346440557 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f6f7aeb7c0 con 0x55f6f7b1f7d0 2023-05-20T01:22:26.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.892+0000 7f719cff9700 1 -- 172.21.15.16:0/346440557 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 49) v1 ==== 3122+0+0 (secure 0 0 0) 0x7f71b8021770 con 0x55f6f7b1f7d0 2023-05-20T01:22:26.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.892+0000 7f71cfc57b80 10 client.6843 fetch_fsmap finished waiting for FSMap version 49 2023-05-20T01:22:26.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.892+0000 7f71cfc57b80 10 client.6843 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2023-05-20T01:22:26.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.892+0000 7f71cfc57b80 20 client.6843 populate_metadata read hostname 'smithi016' 2023-05-20T01:22:26.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.892+0000 7f71cfc57b80 1 --2- 172.21.15.16:0/346440557 >> [v2:172.21.15.16:6836/2351975329,v1:172.21.15.16:6837/2351975329] conn(0x55f6f7b15140 0x55f6f7b3c380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:26.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.892+0000 7f71cfc57b80 4 client.6843 mds_command: new command op to 6768 tid=0[{"prefix": "get_command_descriptions"}] 2023-05-20T01:22:26.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.892+0000 7f71cfc57b80 1 -- 172.21.15.16:0/346440557 --> [v2:172.21.15.16:6836/2351975329,v1:172.21.15.16:6837/2351975329] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f6f7a0c120 con 0x55f6f7b15140 2023-05-20T01:22:26.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.892+0000 7f71c1867700 1 --2- 172.21.15.16:0/346440557 >> [v2:172.21.15.16:6836/2351975329,v1:172.21.15.16:6837/2351975329] conn(0x55f6f7b15140 0x55f6f7b3c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:26.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.893+0000 7f719cff9700 10 client.6843 ms_handle_connect on v2:172.21.15.16:6836/2351975329 2023-05-20T01:22:26.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.893+0000 7f71c1867700 1 --2- 172.21.15.16:0/346440557 >> [v2:172.21.15.16:6836/2351975329,v1:172.21.15.16:6837/2351975329] conn(0x55f6f7b15140 0x55f6f7b3c380 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ecadb1154bd6b3fa server_cookie=e491ede1beece446 in_seq=0 out_seq=0 2023-05-20T01:22:26.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.893+0000 7f719cff9700 1 -- 172.21.15.16:0/346440557 <== mds.0 v2:172.21.15.16:6836/2351975329 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x7f71ac080e90 con 0x55f6f7b15140 2023-05-20T01:22:26.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.893+0000 7f719cff9700 10 client.6843 handle_command_reply: tid=0 2023-05-20T01:22:26.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.893+0000 7f71cfc57b80 1 client.6843 shutdown 2023-05-20T01:22:26.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.893+0000 7f71cfc57b80 1 -- 172.21.15.16:0/346440557 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f6f7b1f7d0 msgr2=0x55f6f7af9150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:26.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.893+0000 7f71cfc57b80 1 --2- 172.21.15.16:0/346440557 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f6f7b1f7d0 0x55f6f7af9150 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f71bc0b16e0 tx=0x7f71b80047c0 comp rx=0 tx=0).stop 2023-05-20T01:22:26.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.893+0000 7f71cfc57b80 1 -- 172.21.15.16:0/346440557 shutdown_connections 2023-05-20T01:22:26.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.893+0000 7f71cfc57b80 1 --2- 172.21.15.16:0/346440557 >> [v2:172.21.15.16:6836/2351975329,v1:172.21.15.16:6837/2351975329] conn(0x55f6f7b15140 0x55f6f7b3c380 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:26.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.894+0000 7f71cfc57b80 1 --2- 172.21.15.16:0/346440557 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55f6f7b234d0 0x55f6f7af9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:26.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.894+0000 7f71cfc57b80 1 --2- 172.21.15.16:0/346440557 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f6f7af6940 0x55f6f7b1ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:26.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.894+0000 7f719cff9700 0 client.6843 ms_handle_reset on v2:172.21.15.16:6836/2351975329 2023-05-20T01:22:26.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.894+0000 7f71cfc57b80 1 --2- 172.21.15.16:0/346440557 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f6f7b1f7d0 0x55f6f7af9150 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:26.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.894+0000 7f71cfc57b80 1 -- 172.21.15.16:0/346440557 >> 172.21.15.16:0/346440557 conn(0x55f6f7aee000 msgr2=0x55f6f7b1fbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:26.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.894+0000 7f71cfc57b80 1 -- 172.21.15.16:0/346440557 shutdown_connections 2023-05-20T01:22:26.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.894+0000 7f71cfc57b80 1 -- 172.21.15.16:0/346440557 wait complete. 2023-05-20T01:22:26.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.894+0000 7f71cfc57b80 20 client.6843 trim_cache size 0 max 16384 2023-05-20T01:22:26.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.911+0000 7f71cfc57b80 1 Processor -- start 2023-05-20T01:22:26.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.911+0000 7f71cfc57b80 1 -- start start 2023-05-20T01:22:26.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.912+0000 7f71cfc57b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f6f7af6940 0x55f6f7b1ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:26.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.912+0000 7f71cfc57b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f6f7b1f7d0 0x55f6f7b3e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:26.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.912+0000 7f71cfc57b80 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55f6f7b234d0 0x55f6f7b3e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:26.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.912+0000 7f71cfc57b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6f7b3dee0 con 0x55f6f7af6940 2023-05-20T01:22:26.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.912+0000 7f71cfc57b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6f7af3820 con 0x55f6f7b1f7d0 2023-05-20T01:22:26.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.912+0000 7f71cfc57b80 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f6f7b11ac0 con 0x55f6f7b234d0 2023-05-20T01:22:26.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.912+0000 7f71c1867700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f6f7af6940 0x55f6f7b1ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:26.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.912+0000 7f71c1867700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f6f7af6940 0x55f6f7b1ffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35410/0 (socket says 172.21.15.16:35410) 2023-05-20T01:22:26.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.912+0000 7f71c1867700 1 -- 172.21.15.16:0/2442458893 learned_addr learned my addr 172.21.15.16:0/2442458893 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:22:26.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.912+0000 7f71c2068700 1 --2- 172.21.15.16:0/2442458893 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55f6f7b234d0 0x55f6f7b3e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:26.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.912+0000 7f71c1066700 1 --2- 172.21.15.16:0/2442458893 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f6f7b1f7d0 0x55f6f7b3e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:26.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.912+0000 7f71c1867700 1 -- 172.21.15.16:0/2442458893 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55f6f7b234d0 msgr2=0x55f6f7b3e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:26.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.912+0000 7f71c1867700 1 --2- 172.21.15.16:0/2442458893 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55f6f7b234d0 0x55f6f7b3e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:26.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.912+0000 7f71c1867700 1 -- 172.21.15.16:0/2442458893 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f6f7b1f7d0 msgr2=0x55f6f7b3e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:26.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.912+0000 7f71c1867700 1 --2- 172.21.15.16:0/2442458893 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f6f7b1f7d0 0x55f6f7b3e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:26.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.912+0000 7f71c1867700 1 -- 172.21.15.16:0/2442458893 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f6f7b23b70 con 0x55f6f7af6940 2023-05-20T01:22:26.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.912+0000 7f71c1066700 1 --2- 172.21.15.16:0/2442458893 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f6f7b1f7d0 0x55f6f7b3e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:22:26.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.912+0000 7f71c1867700 1 --2- 172.21.15.16:0/2442458893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f6f7af6940 0x55f6f7b1ffb0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f71ac00cc60 tx=0x7f71ac005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3f05afbe20cbb24 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:26.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.912+0000 7f719ffff700 1 -- 172.21.15.16:0/2442458893 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71ac055a90 con 0x55f6f7af6940 2023-05-20T01:22:26.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.912+0000 7f719ffff700 1 -- 172.21.15.16:0/2442458893 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71ac080e90 con 0x55f6f7af6940 2023-05-20T01:22:26.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.912+0000 7f719ffff700 1 -- 172.21.15.16:0/2442458893 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71ac05f7b0 con 0x55f6f7af6940 2023-05-20T01:22:26.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.913+0000 7f71cfc57b80 1 -- 172.21.15.16:0/2442458893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f6f7af6940 msgr2=0x55f6f7b1ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:26.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.913+0000 7f71cfc57b80 1 --2- 172.21.15.16:0/2442458893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f6f7af6940 0x55f6f7b1ffb0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f71ac00cc60 tx=0x7f71ac005e80 comp rx=0 tx=0).stop 2023-05-20T01:22:26.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.913+0000 7f71cfc57b80 1 -- 172.21.15.16:0/2442458893 shutdown_connections 2023-05-20T01:22:26.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.913+0000 7f71cfc57b80 1 --2- 172.21.15.16:0/2442458893 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55f6f7b234d0 0x55f6f7b3e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:26.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.913+0000 7f71cfc57b80 1 --2- 172.21.15.16:0/2442458893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f6f7af6940 0x55f6f7b1ffb0 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:26.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.913+0000 7f71cfc57b80 1 --2- 172.21.15.16:0/2442458893 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f6f7b1f7d0 0x55f6f7b3e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:26.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.913+0000 7f71cfc57b80 1 -- 172.21.15.16:0/2442458893 >> 172.21.15.16:0/2442458893 conn(0x55f6f7aee000 msgr2=0x55f6f7aef4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:26.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.913+0000 7f71cfc57b80 1 -- 172.21.15.16:0/2442458893 shutdown_connections 2023-05-20T01:22:26.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.913+0000 7f71cfc57b80 1 -- 172.21.15.16:0/2442458893 wait complete. 2023-05-20T01:22:26.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.913+0000 7f71cfc57b80 1 -- start start 2023-05-20T01:22:26.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.914+0000 7f71cfc57b80 1 Processor -- start 2023-05-20T01:22:26.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.914+0000 7f71cfc57b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f6f7af6940 0x55f6f7b3e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:26.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.915+0000 7f71cfc57b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f6f7b1f7d0 0x55f6f7af21f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:26.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.915+0000 7f71cfc57b80 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55f6f7b234d0 0x55f6f7af2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:26.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.915+0000 7f71cfc57b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6f7b4cef0 con 0x55f6f7b1f7d0 2023-05-20T01:22:26.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.915+0000 7f71cfc57b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6f7b155c0 con 0x55f6f7af6940 2023-05-20T01:22:26.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.915+0000 7f71cfc57b80 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f6f7b15810 con 0x55f6f7b234d0 2023-05-20T01:22:26.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.915+0000 7f71cfc57b80 10 client.0 _ll_register_callbacks cb 0x7f71b800fb10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-05-20T01:22:26.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.915+0000 7f71c1066700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f6f7b1f7d0 0x55f6f7af21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:26.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.915+0000 7f71c1066700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f6f7b1f7d0 0x55f6f7af21f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35418/0 (socket says 172.21.15.16:35418) 2023-05-20T01:22:26.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.915+0000 7f71c1066700 1 -- 172.21.15.16:0/2753457461 learned_addr learned my addr 172.21.15.16:0/2753457461 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:22:26.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.915+0000 7f71c1867700 1 --2- 172.21.15.16:0/2753457461 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f6f7af6940 0x55f6f7b3e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:26.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.915+0000 7f71c2068700 1 --2- 172.21.15.16:0/2753457461 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55f6f7b234d0 0x55f6f7af2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:26.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.915+0000 7f71c1066700 1 -- 172.21.15.16:0/2753457461 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55f6f7b234d0 msgr2=0x55f6f7af2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:26.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.915+0000 7f71c1066700 1 --2- 172.21.15.16:0/2753457461 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55f6f7b234d0 0x55f6f7af2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:26.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.915+0000 7f71c1066700 1 -- 172.21.15.16:0/2753457461 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f6f7af6940 msgr2=0x55f6f7b3e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:26.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.915+0000 7f71c1066700 1 --2- 172.21.15.16:0/2753457461 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f6f7af6940 0x55f6f7b3e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:26.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.915+0000 7f71c1066700 1 -- 172.21.15.16:0/2753457461 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71bc0cc700 con 0x55f6f7b1f7d0 2023-05-20T01:22:26.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.915+0000 7f71c1066700 1 --2- 172.21.15.16:0/2753457461 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f6f7b1f7d0 0x55f6f7af21f0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x55f6f7b3efc0 tx=0x7f71b8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad8105ddad4ec9b0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:26.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.915+0000 7f719cff9700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-05-20T01:22:26.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.915+0000 7f719cff9700 1 -- 172.21.15.16:0/2753457461 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b8014d50 con 0x55f6f7b1f7d0 2023-05-20T01:22:26.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.915+0000 7f719cff9700 1 -- 172.21.15.16:0/2753457461 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(75..75 src has 1..75) v4 ==== 7189+0+0 (secure 0 0 0) 0x7f71b8021410 con 0x55f6f7b1f7d0 2023-05-20T01:22:26.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.915+0000 7f71cfc57b80 1 -- 172.21.15.16:0/2753457461 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f71b4021070 con 0x55f6f7b1f7d0 2023-05-20T01:22:26.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.915+0000 7f719cff9700 1 -- 172.21.15.16:0/2753457461 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f71b8009690 con 0x55f6f7b1f7d0 2023-05-20T01:22:26.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.915+0000 7f71cfc57b80 10 client.6849 fetch_fsmap learned FSMap version 49 2023-05-20T01:22:26.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.916+0000 7f71cfc57b80 1 -- 172.21.15.16:0/2753457461 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=2+}) v3 -- 0x55f6f7877e60 con 0x55f6f7b1f7d0 2023-05-20T01:22:26.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.916+0000 7f719cff9700 1 -- 172.21.15.16:0/2753457461 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71b4021070 con 0x55f6f7b1f7d0 2023-05-20T01:22:26.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.916+0000 7f719cff9700 1 -- 172.21.15.16:0/2753457461 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 49) v1 ==== 3122+0+0 (secure 0 0 0) 0x7f71b8021610 con 0x55f6f7b1f7d0 2023-05-20T01:22:26.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.916+0000 7f71cfc57b80 10 client.6849 fetch_fsmap finished waiting for FSMap version 49 2023-05-20T01:22:26.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.916+0000 7f71cfc57b80 10 client.6849 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2023-05-20T01:22:26.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.916+0000 7f71cfc57b80 20 client.6849 populate_metadata read hostname 'smithi016' 2023-05-20T01:22:26.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.916+0000 7f71cfc57b80 1 --2- 172.21.15.16:0/2753457461 >> [v2:172.21.15.16:6836/2351975329,v1:172.21.15.16:6837/2351975329] conn(0x55f6f7b3c380 0x55f6f7b26870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:26.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.916+0000 7f71cfc57b80 4 client.6849 mds_command: new command op to 6768 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2023-05-20T01:22:26.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.916+0000 7f71cfc57b80 1 -- 172.21.15.16:0/2753457461 --> [v2:172.21.15.16:6836/2351975329,v1:172.21.15.16:6837/2351975329] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) v1 -- 0x55f6f7a186e0 con 0x55f6f7b3c380 2023-05-20T01:22:26.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.916+0000 7f71c1867700 1 --2- 172.21.15.16:0/2753457461 >> [v2:172.21.15.16:6836/2351975329,v1:172.21.15.16:6837/2351975329] conn(0x55f6f7b3c380 0x55f6f7b26870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:26.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.916+0000 7f719cff9700 10 client.6849 ms_handle_connect on v2:172.21.15.16:6836/2351975329 2023-05-20T01:22:26.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.916+0000 7f71c1867700 1 --2- 172.21.15.16:0/2753457461 >> [v2:172.21.15.16:6836/2351975329,v1:172.21.15.16:6837/2351975329] conn(0x55f6f7b3c380 0x55f6f7b26870 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a968eb81a8da12e5 server_cookie=e8604c7ff2ff5d7f in_seq=0 out_seq=0 2023-05-20T01:22:26.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.917+0000 7f719cff9700 1 -- 172.21.15.16:0/2753457461 <== mds.0 v2:172.21.15.16:6836/2351975329 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55f6f7a186e0 con 0x55f6f7b3c380 2023-05-20T01:22:26.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.917+0000 7f719cff9700 10 client.6849 handle_command_reply: tid=0 2023-05-20T01:22:26.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.917+0000 7f71cfc57b80 1 client.6849 shutdown 2023-05-20T01:22:26.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.917+0000 7f71cfc57b80 1 -- 172.21.15.16:0/2753457461 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f6f7b1f7d0 msgr2=0x55f6f7af21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:26.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.917+0000 7f71cfc57b80 1 --2- 172.21.15.16:0/2753457461 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f6f7b1f7d0 0x55f6f7af21f0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x55f6f7b3efc0 tx=0x7f71b8014000 comp rx=0 tx=0).stop 2023-05-20T01:22:26.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.917+0000 7f71cfc57b80 1 -- 172.21.15.16:0/2753457461 shutdown_connections 2023-05-20T01:22:26.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.917+0000 7f71cfc57b80 1 --2- 172.21.15.16:0/2753457461 >> [v2:172.21.15.16:6836/2351975329,v1:172.21.15.16:6837/2351975329] conn(0x55f6f7b3c380 0x55f6f7b26870 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:26.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.917+0000 7f71cfc57b80 1 --2- 172.21.15.16:0/2753457461 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55f6f7b234d0 0x55f6f7af2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:26.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.917+0000 7f71cfc57b80 1 --2- 172.21.15.16:0/2753457461 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f6f7af6940 0x55f6f7b3e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:26.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.917+0000 7f719cff9700 0 client.6849 ms_handle_reset on v2:172.21.15.16:6836/2351975329 2023-05-20T01:22:26.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.917+0000 7f71cfc57b80 1 --2- 172.21.15.16:0/2753457461 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f6f7b1f7d0 0x55f6f7af21f0 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:26.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.917+0000 7f71cfc57b80 1 -- 172.21.15.16:0/2753457461 >> 172.21.15.16:0/2753457461 conn(0x55f6f7aee000 msgr2=0x55f6f7b1ed20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:26.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.917+0000 7f71cfc57b80 1 -- 172.21.15.16:0/2753457461 shutdown_connections 2023-05-20T01:22:26.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.917+0000 7f71cfc57b80 1 -- 172.21.15.16:0/2753457461 wait complete. 2023-05-20T01:22:26.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.917+0000 7f71cfc57b80 20 client.6849 trim_cache size 0 max 16384 2023-05-20T01:22:26.931 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-05-20T01:22:26.931 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2023-05-20T01:22:26.931 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "0a81b0f4-34fe-417d-bdbf-a6436d7519fe", 2023-05-20T01:22:26.932 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2023-05-20T01:22:26.932 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-05-20T01:22:26.932 INFO:tasks.ceph.mds.c.smithi016.stderr:2023-05-20T01:22:26.918+0000 7fd0f0e15700 -1 log_channel(cluster) log [ERR] : bad backtrace on directory inode 0x10000000000 2023-05-20T01:22:26.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.918+0000 7f71c3acb700 1 -- 172.21.15.16:0/3121928475 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f71a006ace0 msgr2=0x7f71a006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:26.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.918+0000 7f71c3acb700 1 --2- 172.21.15.16:0/3121928475 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f71a006ace0 0x7f71a006d190 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f71bc156670 tx=0x7f71b8008040 comp rx=0 tx=0).stop 2023-05-20T01:22:26.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.918+0000 7f71c3acb700 1 -- 172.21.15.16:0/3121928475 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71bc06d190 msgr2=0x7f71bc149ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:26.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.918+0000 7f71c3acb700 1 --2- 172.21.15.16:0/3121928475 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71bc06d190 0x7f71bc149ab0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f71ac0010e0 tx=0x7f71ac00c860 comp rx=0 tx=0).stop 2023-05-20T01:22:26.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.918+0000 7f71c3acb700 1 -- 172.21.15.16:0/3121928475 shutdown_connections 2023-05-20T01:22:26.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.918+0000 7f71c3acb700 1 --2- 172.21.15.16:0/3121928475 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f71a006ace0 0x7f71a006d190 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:26.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.918+0000 7f71c3acb700 1 --2- 172.21.15.16:0/3121928475 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f71bc0cbbd0 0x7f71bc0cddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:26.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.918+0000 7f71c3acb700 1 --2- 172.21.15.16:0/3121928475 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71bc06d190 0x7f71bc149ab0 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:26.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.918+0000 7f71c3acb700 1 --2- 172.21.15.16:0/3121928475 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f71bc0b2090 0x7f71bc0c74e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:26.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.918+0000 7f71c3acb700 1 -- 172.21.15.16:0/3121928475 >> 172.21.15.16:0/3121928475 conn(0x7f71bc0661b0 msgr2=0x7f71bc0cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:26.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.918+0000 7f71c3acb700 1 -- 172.21.15.16:0/3121928475 shutdown_connections 2023-05-20T01:22:26.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:26.918+0000 7f71c3acb700 1 -- 172.21.15.16:0/3121928475 wait complete. 2023-05-20T01:22:26.936 INFO:tasks.ceph.mds.c.smithi016.stderr:2023-05-20T01:22:26.919+0000 7fd0f0e15700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /subdir/ auth v1 pv4 ap=3 DIRTYPARENT f(v0 1=1+0) n() (inest lock dirty) (ifile lock->sync w=1 flushing) (iversion lock) | dirtyscattered=2 lock=1 dirfrag=1 dirtyparent=1 waiter=1 authpin=1 scrubqueue=0 0x56135f757b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(9)0x10000000000:[<0x1/subdir v1>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 1=1+0)","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 1=1+0)","memory_value.rstat":"n()","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-61} 2023-05-20T01:22:26.936 INFO:tasks.ceph.mds.c.smithi016.stderr:2023-05-20T01:22:26.920+0000 7fd0f0e15700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000020a [2,head] /subdir/21 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x56135f827700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(11)0x1000000020a:[<0x10000000000/21 v212>,<0x1/subdir v238>]//[10]","memoryvalue":"(11)0x1000000020a:[<0x10000000000/21 v1>,<0x1/subdir v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-05-20T01:22:26.937 INFO:tasks.ceph.mds.c.smithi016.stderr:2023-05-20T01:22:26.920+0000 7fd0f0e15700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000208 [2,head] /subdir/19 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x56135f82cb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(11)0x10000000208:[<0x10000000000/19 v208>,<0x1/subdir v238>]//[10]","memoryvalue":"(11)0x10000000208:[<0x10000000000/19 v1>,<0x1/subdir v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-05-20T01:22:26.937 INFO:tasks.ceph.mds.c.smithi016.stderr:2023-05-20T01:22:26.920+0000 7fd0f0e15700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001fd [2,head] /subdir/8 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x56135f824000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(11)0x100000001fd:[<0x10000000000/8 v96>,<0x1/subdir v238>]//[10]","memoryvalue":"(11)0x100000001fd:[<0x10000000000/8 v1>,<0x1/subdir v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-05-20T01:22:26.937 INFO:tasks.ceph.mds.c.smithi016.stderr:2023-05-20T01:22:26.920+0000 7fd0f0e15700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001f8 [2,head] /subdir/3 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x56135f825b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(11)0x100000001f8:[<0x10000000000/3 v50>,<0x1/subdir v238>]//[10]","memoryvalue":"(11)0x100000001f8:[<0x10000000000/3 v1>,<0x1/subdir v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-05-20T01:22:26.937 INFO:tasks.ceph.mds.c.smithi016.stderr:2023-05-20T01:22:26.921+0000 7fd0f0e15700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001fe [2,head] /subdir/9 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x56135f77d700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(11)0x100000001fe:[<0x10000000000/9 v94>,<0x1/subdir v238>]//[10]","memoryvalue":"(11)0x100000001fe:[<0x10000000000/9 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-05-20T01:22:26.937 INFO:tasks.ceph.mds.c.smithi016.stderr:2023-05-20T01:22:26.922+0000 7fd0f0e15700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000204 [2,head] /subdir/15 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x56135f82e100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(11)0x10000000204:[<0x10000000000/15 v138>,<0x1/subdir v238>]//[10]","memoryvalue":"(11)0x10000000204:[<0x10000000000/15 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-05-20T01:22:26.938 INFO:tasks.ceph.mds.c.smithi016.stderr:2023-05-20T01:22:26.922+0000 7fd0f0e15700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000201 [2,head] /subdir/12 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x56135f82f180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(11)0x10000000201:[<0x10000000000/12 v136>,<0x1/subdir v238>]//[10]","memoryvalue":"(11)0x10000000201:[<0x10000000000/12 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-05-20T01:22:26.938 INFO:tasks.ceph.mds.c.smithi016.stderr:2023-05-20T01:22:26.922+0000 7fd0f0e15700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000209 [2,head] /subdir/20 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x56135f82c000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(11)0x10000000209:[<0x10000000000/20 v206>,<0x1/subdir v238>]//[10]","memoryvalue":"(11)0x10000000209:[<0x10000000000/20 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-05-20T01:22:26.938 INFO:tasks.ceph.mds.c.smithi016.stderr:2023-05-20T01:22:26.922+0000 7fd0f0e15700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001ff [2,head] /subdir/10 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x56135f834000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(11)0x100000001ff:[<0x10000000000/10 v220>,<0x1/subdir v238>]//[10]","memoryvalue":"(11)0x100000001ff:[<0x10000000000/10 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-05-20T01:22:26.939 INFO:tasks.ceph.mds.c.smithi016.stderr:2023-05-20T01:22:26.922+0000 7fd0f0e15700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000200 [2,head] /subdir/11 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x56135f82f700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(11)0x10000000200:[<0x10000000000/11 v132>,<0x1/subdir v238>]//[10]","memoryvalue":"(11)0x10000000200:[<0x10000000000/11 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-05-20T01:22:26.939 INFO:tasks.ceph.mds.c.smithi016.stderr:2023-05-20T01:22:26.922+0000 7fd0f0e15700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001f7 [2,head] /subdir/2 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x56135f826100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(11)0x100000001f7:[<0x10000000000/2 v48>,<0x1/subdir v238>]//[10]","memoryvalue":"(11)0x100000001f7:[<0x10000000000/2 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-05-20T01:22:26.939 INFO:tasks.ceph.mds.c.smithi016.stderr:2023-05-20T01:22:26.922+0000 7fd0f0e15700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001f6 [2,head] /subdir/1 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x56135f82c580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(11)0x100000001f6:[<0x10000000000/1 v44>,<0x1/subdir v238>]//[10]","memoryvalue":"(11)0x100000001f6:[<0x10000000000/1 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-05-20T01:22:26.939 INFO:tasks.ceph.mds.c.smithi016.stderr:2023-05-20T01:22:26.923+0000 7fd0f0e15700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001f9 [2,head] /subdir/4 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x56135f825600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(11)0x100000001f9:[<0x10000000000/4 v52>,<0x1/subdir v238>]//[10]","memoryvalue":"(11)0x100000001f9:[<0x10000000000/4 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-05-20T01:22:26.939 INFO:tasks.ceph.mds.c.smithi016.stderr:2023-05-20T01:22:26.923+0000 7fd0f0e15700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000020d [2,head] /subdir/24 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x56135f826680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(11)0x1000000020d:[<0x10000000000/24 v216>,<0x1/subdir v238>]//[10]","memoryvalue":"(11)0x1000000020d:[<0x10000000000/24 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-05-20T01:22:26.939 INFO:tasks.ceph.mds.c.smithi016.stderr:2023-05-20T01:22:26.923+0000 7fd0f0e15700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000205 [2,head] /subdir/16 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x56135f82db80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(11)0x10000000205:[<0x10000000000/16 v218>,<0x1/subdir v238>]//[10]","memoryvalue":"(11)0x10000000205:[<0x10000000000/16 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-05-20T01:22:26.940 INFO:tasks.ceph.mds.c.smithi016.stderr:2023-05-20T01:22:26.923+0000 7fd0f0e15700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /subdir/0 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x56135f834580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(11)0x10000000001:[<0x10000000000/0 v224>,<0x1/subdir v238>]//[10]","memoryvalue":"(11)0x10000000001:[<0x10000000000/0 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-05-20T01:22:26.940 INFO:tasks.ceph.mds.c.smithi016.stderr:2023-05-20T01:22:26.923+0000 7fd0f0e15700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000020b [2,head] /subdir/22 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x56135f827180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(11)0x1000000020b:[<0x10000000000/22 v210>,<0x1/subdir v238>]//[10]","memoryvalue":"(11)0x1000000020b:[<0x10000000000/22 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-05-20T01:22:26.940 INFO:tasks.ceph.mds.c.smithi016.stderr:2023-05-20T01:22:26.923+0000 7fd0f0e15700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000203 [2,head] /subdir/14 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x56135f82e680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(11)0x10000000203:[<0x10000000000/14 v134>,<0x1/subdir v238>]//[10]","memoryvalue":"(11)0x10000000203:[<0x10000000000/14 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-05-20T01:22:26.940 INFO:tasks.ceph.mds.c.smithi016.stderr:2023-05-20T01:22:26.923+0000 7fd0f0e15700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000207 [2,head] /subdir/18 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x56135f82d080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(11)0x10000000207:[<0x10000000000/18 v192>,<0x1/subdir v238>]//[10]","memoryvalue":"(11)0x10000000207:[<0x10000000000/18 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-05-20T01:22:26.940 INFO:tasks.ceph.mds.c.smithi016.stderr:2023-05-20T01:22:26.923+0000 7fd0f0e15700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001fa [2,head] /subdir/5 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x56135f825080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(11)0x100000001fa:[<0x10000000000/5 v222>,<0x1/subdir v238>]//[10]","memoryvalue":"(11)0x100000001fa:[<0x10000000000/5 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-05-20T01:22:26.940 INFO:tasks.ceph.mds.c.smithi016.stderr:2023-05-20T01:22:26.924+0000 7fd0f0e15700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000202 [2,head] /subdir/13 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x56135f82ec00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(11)0x10000000202:[<0x10000000000/13 v142>,<0x1/subdir v238>]//[10]","memoryvalue":"(11)0x10000000202:[<0x10000000000/13 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-05-20T01:22:26.940 INFO:tasks.ceph.mds.c.smithi016.stderr:2023-05-20T01:22:26.924+0000 7fd0f0e15700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1 [...2,head] / auth v5 ap=5 snaprealm=0x56135f751200 DIRTYPARENT f(v1 1=0+1) n(v0 rc2023-05-20T01:22:26.568635+0000 1=0+1)/n() (idft sync r=1) (inest sync r=1) (ifile sync->mix r=1) (iversion lock) caps={6828=pLsXs/-@45} | dirtyscattered=0 request=0 lock=3 dirfrag=1 caps=1 openingsnapparents=0 dirtyparent=1 dirty=1 waiter=1 authpin=1 scrubqueue=0 0x56135f757600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(9)0x1:[]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 1=0+1)","ondisk_value.rstat":"n(v0 2=0+2)","memory_value.dirstat":"f(v1 1=0+1)","memory_value.rstat":"n(v0 rc2023-05-20T01:22:26.568635+0000 1=0+1)","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-61} 2023-05-20T01:22:26.941 INFO:tasks.ceph.mds.c.smithi016.stderr:2023-05-20T01:22:26.925+0000 7fd0f0e15700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000020c [2,head] /subdir/23 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x56135f826c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(11)0x1000000020c:[<0x10000000000/23 v214>,<0x1/subdir v238>]//[10]","memoryvalue":"(11)0x1000000020c:[<0x10000000000/23 v1>,<0x1/subdir v7>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-05-20T01:22:26.941 INFO:tasks.ceph.mds.c.smithi016.stderr:2023-05-20T01:22:26.925+0000 7fd0f0e15700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001fc [2,head] /subdir/7 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x56135f824580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(11)0x100000001fc:[<0x10000000000/7 v86>,<0x1/subdir v238>]//[10]","memoryvalue":"(11)0x100000001fc:[<0x10000000000/7 v1>,<0x1/subdir v7>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-05-20T01:22:26.941 INFO:tasks.ceph.mds.c.smithi016.stderr:2023-05-20T01:22:26.925+0000 7fd0f0e15700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000206 [2,head] /subdir/17 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x56135f82d600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(11)0x10000000206:[<0x10000000000/17 v184>,<0x1/subdir v238>]//[10]","memoryvalue":"(11)0x10000000206:[<0x10000000000/17 v1>,<0x1/subdir v7>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-05-20T01:22:26.941 INFO:tasks.ceph.mds.c.smithi016.stderr:2023-05-20T01:22:26.925+0000 7fd0f0e15700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001fb [2,head] /subdir/6 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x56135f824b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(11)0x100000001fb:[<0x10000000000/6 v92>,<0x1/subdir v238>]//[10]","memoryvalue":"(11)0x100000001fb:[<0x10000000000/6 v1>,<0x1/subdir v7>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-05-20T01:22:26.942 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub status 2023-05-20T01:22:27.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.066+0000 7fd0bee52700 1 Processor -- start 2023-05-20T01:22:27.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.066+0000 7fd0bee52700 1 -- start start 2023-05-20T01:22:27.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.066+0000 7fd0bee52700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fd0b80cd070 0x7fd0b806e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:27.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.066+0000 7fd0bee52700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0b8072430 con 0x7fd0b815cf40 2023-05-20T01:22:27.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.067+0000 7fd0bee52700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0b8071820 con 0x7fd0b80710d0 2023-05-20T01:22:27.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.067+0000 7fd0bee52700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0b8071b00 con 0x7fd0b80cd070 2023-05-20T01:22:27.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.067+0000 7fd0bcbee700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fd0b815cf40 0x7fd0b80cc960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:46954/0 (socket says 172.21.15.16:46954) 2023-05-20T01:22:27.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.067+0000 7fd0bcbee700 1 -- 172.21.15.16:0/3559367357 learned_addr learned my addr 172.21.15.16:0/3559367357 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:22:27.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.067+0000 7fd0b7fff700 1 --2- 172.21.15.16:0/3559367357 >> v2:172.21.15.132:3300/0 conn(0x7fd0b80cd070 0x7fd0b806e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:27.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.067+0000 7fd0b77fe700 1 -- 172.21.15.16:0/3559367357 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 492913342 0 0) 0x7fd0b8072430 con 0x7fd0b815cf40 2023-05-20T01:22:27.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.067+0000 7fd0b77fe700 1 -- 172.21.15.16:0/3559367357 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd0a80035b0 con 0x7fd0b815cf40 2023-05-20T01:22:27.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.067+0000 7fd0b77fe700 1 -- 172.21.15.16:0/3559367357 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 312530185 0 0) 0x7fd0a80035b0 con 0x7fd0b815cf40 2023-05-20T01:22:27.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.067+0000 7fd0b7fff700 1 -- 172.21.15.16:0/3559367357 >> v1:172.21.15.132:6789/0 conn(0x7fd0b80710d0 legacy=0x7fd0b80cabb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:27.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.067+0000 7fd0b7fff700 1 -- 172.21.15.16:0/3559367357 >> v1:172.21.15.16:6789/0 conn(0x7fd0b815cf40 legacy=0x7fd0b80cc960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:27.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.067+0000 7fd0b7fff700 1 -- 172.21.15.16:0/3559367357 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0b8071de0 con 0x7fd0b80cd070 2023-05-20T01:22:27.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.067+0000 7fd0b7fff700 1 --2- 172.21.15.16:0/3559367357 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd0b80cd070 0x7fd0b806e020 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fd0a4009840 tx=0x7fd0a4009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8d08c3024082249 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:27.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.068+0000 7fd0b77fe700 1 -- 172.21.15.16:0/3559367357 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0ac001670 con 0x7fd0b80cd070 2023-05-20T01:22:27.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.068+0000 7fd0b77fe700 1 -- 172.21.15.16:0/3559367357 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0a4004030 con 0x7fd0b80cd070 2023-05-20T01:22:27.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.068+0000 7fd0b77fe700 1 -- 172.21.15.16:0/3559367357 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0a40043c0 con 0x7fd0b80cd070 2023-05-20T01:22:27.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.068+0000 7fd0bee52700 1 -- 172.21.15.16:0/3559367357 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd0b80cd070 msgr2=0x7fd0b806e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:27.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.068+0000 7fd0bee52700 1 --2- 172.21.15.16:0/3559367357 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd0b80cd070 0x7fd0b806e020 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fd0a4009840 tx=0x7fd0a4009b50 comp rx=0 tx=0).stop 2023-05-20T01:22:27.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.068+0000 7fd0bee52700 1 -- 172.21.15.16:0/3559367357 shutdown_connections 2023-05-20T01:22:27.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.068+0000 7fd0bee52700 1 --2- 172.21.15.16:0/3559367357 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd0b80cd070 0x7fd0b806e020 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:27.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.068+0000 7fd0bee52700 1 -- 172.21.15.16:0/3559367357 >> 172.21.15.16:0/3559367357 conn(0x7fd0b806c260 msgr2=0x7fd0b80cffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:27.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.068+0000 7fd0bee52700 1 -- 172.21.15.16:0/3559367357 shutdown_connections 2023-05-20T01:22:27.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.068+0000 7fd0bee52700 1 -- 172.21.15.16:0/3559367357 wait complete. 2023-05-20T01:22:27.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.069+0000 7fd0bee52700 1 Processor -- start 2023-05-20T01:22:27.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.070+0000 7fd0bee52700 1 -- start start 2023-05-20T01:22:27.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.070+0000 7fd0bee52700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0b80710d0 0x7fd0b80b3100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:27.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.070+0000 7fd0bee52700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd0b80cd070 0x7fd0b814fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:27.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.070+0000 7fd0bee52700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd0b815cf40 0x7fd0b8158550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:27.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.070+0000 7fd0bee52700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0b80b3640 con 0x7fd0b80710d0 2023-05-20T01:22:27.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.070+0000 7fd0bee52700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0b81500c0 con 0x7fd0b80cd070 2023-05-20T01:22:27.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.070+0000 7fd0bee52700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0b80c3b00 con 0x7fd0b815cf40 2023-05-20T01:22:27.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.071+0000 7fd0bcbee700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0b80710d0 0x7fd0b80b3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:27.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.071+0000 7fd0bcbee700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0b80710d0 0x7fd0b80b3100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35434/0 (socket says 172.21.15.16:35434) 2023-05-20T01:22:27.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.071+0000 7fd0bcbee700 1 -- 172.21.15.16:0/3235793373 learned_addr learned my addr 172.21.15.16:0/3235793373 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:22:27.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.071+0000 7fd0bd3ef700 1 --2- 172.21.15.16:0/3235793373 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd0b815cf40 0x7fd0b8158550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:27.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.071+0000 7fd0b7fff700 1 --2- 172.21.15.16:0/3235793373 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd0b80cd070 0x7fd0b814fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:27.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.071+0000 7fd0bcbee700 1 -- 172.21.15.16:0/3235793373 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd0b815cf40 msgr2=0x7fd0b8158550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:27.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.071+0000 7fd0bcbee700 1 --2- 172.21.15.16:0/3235793373 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd0b815cf40 0x7fd0b8158550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:27.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.071+0000 7fd0bcbee700 1 -- 172.21.15.16:0/3235793373 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd0b80cd070 msgr2=0x7fd0b814fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:27.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.071+0000 7fd0bcbee700 1 --2- 172.21.15.16:0/3235793373 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd0b80cd070 0x7fd0b814fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:27.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.071+0000 7fd0bcbee700 1 -- 172.21.15.16:0/3235793373 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0a40094a0 con 0x7fd0b80710d0 2023-05-20T01:22:27.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.071+0000 7fd0bcbee700 1 --2- 172.21.15.16:0/3235793373 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0b80710d0 0x7fd0b80b3100 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fd0ac00c520 tx=0x7fd0ac00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:27.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.071+0000 7fd0b57fa700 1 -- 172.21.15.16:0/3235793373 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0ac004c80 con 0x7fd0b80710d0 2023-05-20T01:22:27.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.071+0000 7fd0b57fa700 1 -- 172.21.15.16:0/3235793373 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0ac004e20 con 0x7fd0b80710d0 2023-05-20T01:22:27.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.071+0000 7fd0bee52700 1 -- 172.21.15.16:0/3235793373 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0b8071510 con 0x7fd0b80710d0 2023-05-20T01:22:27.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.071+0000 7fd0b57fa700 1 -- 172.21.15.16:0/3235793373 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0ac007310 con 0x7fd0b80710d0 2023-05-20T01:22:27.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.071+0000 7fd0bee52700 1 -- 172.21.15.16:0/3235793373 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0b80720c0 con 0x7fd0b80710d0 2023-05-20T01:22:27.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.073+0000 7fd0b57fa700 1 -- 172.21.15.16:0/3235793373 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd0ac007470 con 0x7fd0b80710d0 2023-05-20T01:22:27.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.073+0000 7fd0b57fa700 1 --2- 172.21.15.16:0/3235793373 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd09c06ace0 0x7fd09c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:27.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.073+0000 7fd0b57fa700 1 -- 172.21.15.16:0/3235793373 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7189+0+0 (secure 0 0 0) 0x7fd0ac0896b0 con 0x7fd0b80710d0 2023-05-20T01:22:27.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.073+0000 7fd0b7fff700 1 --2- 172.21.15.16:0/3235793373 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd09c06ace0 0x7fd09c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:27.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.074+0000 7fd0b7fff700 1 --2- 172.21.15.16:0/3235793373 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd09c06ace0 0x7fd09c06d190 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fd0a4009840 tx=0x7fd0a4008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:27.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.075+0000 7fd0cafdeb80 1 Processor -- start 2023-05-20T01:22:27.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.075+0000 7fd0cafdeb80 1 -- start start 2023-05-20T01:22:27.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.076+0000 7fd0cafdeb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ab992f0eb0 0x55ab993106e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:27.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.076+0000 7fd0cafdeb80 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ab992c78c0 0x55ab992f1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:27.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.076+0000 7fd0cafdeb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab992c94b0 0x55ab992c9920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:27.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.076+0000 7fd0cafdeb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab991cf060 con 0x55ab992c94b0 2023-05-20T01:22:27.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.076+0000 7fd0cafdeb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab992218a0 con 0x55ab992f0eb0 2023-05-20T01:22:27.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.076+0000 7fd0cafdeb80 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ab9921ad30 con 0x55ab992c78c0 2023-05-20T01:22:27.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.076+0000 7fd0bd3ef700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab992c94b0 0x55ab992c9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:27.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.076+0000 7fd0bd3ef700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab992c94b0 0x55ab992c9920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35444/0 (socket says 172.21.15.16:35444) 2023-05-20T01:22:27.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.076+0000 7fd0bd3ef700 1 -- 172.21.15.16:0/2198069364 learned_addr learned my addr 172.21.15.16:0/2198069364 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:22:27.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.076+0000 7fd0b7fff700 1 --2- 172.21.15.16:0/2198069364 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ab992c78c0 0x55ab992f1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:27.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.076+0000 7fd0bcbee700 1 --2- 172.21.15.16:0/2198069364 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ab992f0eb0 0x55ab993106e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:27.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.076+0000 7fd0bd3ef700 1 -- 172.21.15.16:0/2198069364 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ab992c78c0 msgr2=0x55ab992f1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:27.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.076+0000 7fd0bd3ef700 1 --2- 172.21.15.16:0/2198069364 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ab992c78c0 0x55ab992f1280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:27.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.076+0000 7fd0bd3ef700 1 -- 172.21.15.16:0/2198069364 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ab992f0eb0 msgr2=0x55ab993106e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:27.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.076+0000 7fd0bd3ef700 1 --2- 172.21.15.16:0/2198069364 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ab992f0eb0 0x55ab993106e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:27.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.076+0000 7fd0bd3ef700 1 -- 172.21.15.16:0/2198069364 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ab9909c630 con 0x55ab992c94b0 2023-05-20T01:22:27.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.076+0000 7fd0b7fff700 1 --2- 172.21.15.16:0/2198069364 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ab992c78c0 0x55ab992f1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:22:27.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.076+0000 7fd0bd3ef700 1 --2- 172.21.15.16:0/2198069364 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab992c94b0 0x55ab992c9920 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fd0b0002140 tx=0x7fd0b000e440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0a9b38467f18ea server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:27.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.077+0000 7fd0a27fc700 1 -- 172.21.15.16:0/2198069364 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0b000ecc0 con 0x55ab992c94b0 2023-05-20T01:22:27.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.077+0000 7fd0a27fc700 1 -- 172.21.15.16:0/2198069364 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0b000ee20 con 0x55ab992c94b0 2023-05-20T01:22:27.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.077+0000 7fd0a27fc700 1 -- 172.21.15.16:0/2198069364 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0b0003c20 con 0x55ab992c94b0 2023-05-20T01:22:27.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.077+0000 7fd0cafdeb80 1 -- 172.21.15.16:0/2198069364 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab992c94b0 msgr2=0x55ab992c9920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:27.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.077+0000 7fd0cafdeb80 1 --2- 172.21.15.16:0/2198069364 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab992c94b0 0x55ab992c9920 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fd0b0002140 tx=0x7fd0b000e440 comp rx=0 tx=0).stop 2023-05-20T01:22:27.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.077+0000 7fd0cafdeb80 1 -- 172.21.15.16:0/2198069364 shutdown_connections 2023-05-20T01:22:27.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.077+0000 7fd0cafdeb80 1 --2- 172.21.15.16:0/2198069364 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab992c94b0 0x55ab992c9920 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:27.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.077+0000 7fd0cafdeb80 1 --2- 172.21.15.16:0/2198069364 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ab992f0eb0 0x55ab993106e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:27.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.077+0000 7fd0cafdeb80 1 --2- 172.21.15.16:0/2198069364 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ab992c78c0 0x55ab992f1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:27.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.077+0000 7fd0cafdeb80 1 -- 172.21.15.16:0/2198069364 >> 172.21.15.16:0/2198069364 conn(0x55ab992bef80 msgr2=0x55ab9930f260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:27.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.077+0000 7fd0cafdeb80 1 -- 172.21.15.16:0/2198069364 shutdown_connections 2023-05-20T01:22:27.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.077+0000 7fd0cafdeb80 1 -- 172.21.15.16:0/2198069364 wait complete. 2023-05-20T01:22:27.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.078+0000 7fd0cafdeb80 1 -- start start 2023-05-20T01:22:27.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.078+0000 7fd0cafdeb80 1 Processor -- start 2023-05-20T01:22:27.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.079+0000 7fd0cafdeb80 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ab992c78c0 0x55ab99320140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:27.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.079+0000 7fd0cafdeb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab992c94b0 0x55ab992f6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:27.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.079+0000 7fd0cafdeb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ab992f0eb0 0x55ab99326250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:27.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.079+0000 7fd0cafdeb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab991bc8e0 con 0x55ab992c94b0 2023-05-20T01:22:27.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.079+0000 7fd0cafdeb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab991bc580 con 0x55ab992f0eb0 2023-05-20T01:22:27.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.079+0000 7fd0cafdeb80 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ab9909e920 con 0x55ab992c78c0 2023-05-20T01:22:27.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.079+0000 7fd0cafdeb80 10 client.0 _ll_register_callbacks cb 0x55ab991da780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-05-20T01:22:27.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.079+0000 7fd0b7fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab992c94b0 0x55ab992f6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:27.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.079+0000 7fd0b7fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab992c94b0 0x55ab992f6450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35448/0 (socket says 172.21.15.16:35448) 2023-05-20T01:22:27.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.079+0000 7fd0b7fff700 1 -- 172.21.15.16:0/503839028 learned_addr learned my addr 172.21.15.16:0/503839028 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:22:27.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.079+0000 7fd0bd3ef700 1 --2- 172.21.15.16:0/503839028 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ab992f0eb0 0x55ab99326250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:27.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.079+0000 7fd0bcbee700 1 --2- 172.21.15.16:0/503839028 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ab992c78c0 0x55ab99320140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:27.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.079+0000 7fd0b7fff700 1 -- 172.21.15.16:0/503839028 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ab992c78c0 msgr2=0x55ab99320140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:27.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.079+0000 7fd0b7fff700 1 --2- 172.21.15.16:0/503839028 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ab992c78c0 0x55ab99320140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:27.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.079+0000 7fd0b7fff700 1 -- 172.21.15.16:0/503839028 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ab992f0eb0 msgr2=0x55ab99326250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:27.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.079+0000 7fd0b7fff700 1 --2- 172.21.15.16:0/503839028 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ab992f0eb0 0x55ab99326250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:27.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.079+0000 7fd0b7fff700 1 -- 172.21.15.16:0/503839028 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0b000f040 con 0x55ab992c94b0 2023-05-20T01:22:27.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.079+0000 7fd0b7fff700 1 --2- 172.21.15.16:0/503839028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab992c94b0 0x55ab992f6450 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fd0a4004000 tx=0x7fd0a400e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a04c25714d46724 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:27.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.079+0000 7fd0977fe700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-05-20T01:22:27.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.080+0000 7fd0977fe700 1 -- 172.21.15.16:0/503839028 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0a402a040 con 0x55ab992c94b0 2023-05-20T01:22:27.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.080+0000 7fd0977fe700 1 -- 172.21.15.16:0/503839028 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(75..75 src has 1..75) v4 ==== 7189+0+0 (secure 0 0 0) 0x7fd0a4011630 con 0x55ab992c94b0 2023-05-20T01:22:27.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.080+0000 7fd0cafdeb80 1 -- 172.21.15.16:0/503839028 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ab992bc740 con 0x55ab992c94b0 2023-05-20T01:22:27.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.080+0000 7fd0977fe700 1 -- 172.21.15.16:0/503839028 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd0a401f070 con 0x55ab992c94b0 2023-05-20T01:22:27.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.080+0000 7fd0cafdeb80 10 client.6861 fetch_fsmap learned FSMap version 49 2023-05-20T01:22:27.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.080+0000 7fd0cafdeb80 1 -- 172.21.15.16:0/503839028 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=2+}) v3 -- 0x55ab991e1190 con 0x55ab992c94b0 2023-05-20T01:22:27.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.080+0000 7fd0977fe700 1 -- 172.21.15.16:0/503839028 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ab992bc740 con 0x55ab992c94b0 2023-05-20T01:22:27.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.080+0000 7fd0977fe700 1 -- 172.21.15.16:0/503839028 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 49) v1 ==== 3122+0+0 (secure 0 0 0) 0x7fd0a402d030 con 0x55ab992c94b0 2023-05-20T01:22:27.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.080+0000 7fd0cafdeb80 10 client.6861 fetch_fsmap finished waiting for FSMap version 49 2023-05-20T01:22:27.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.080+0000 7fd0cafdeb80 10 client.6861 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2023-05-20T01:22:27.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.080+0000 7fd0cafdeb80 20 client.6861 populate_metadata read hostname 'smithi016' 2023-05-20T01:22:27.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.080+0000 7fd0cafdeb80 1 --2- 172.21.15.16:0/503839028 >> [v2:172.21.15.16:6836/2351975329,v1:172.21.15.16:6837/2351975329] conn(0x55ab992f2340 0x55ab992f2740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:27.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.080+0000 7fd0cafdeb80 4 client.6861 mds_command: new command op to 6768 tid=0[{"prefix": "get_command_descriptions"}] 2023-05-20T01:22:27.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.080+0000 7fd0cafdeb80 1 -- 172.21.15.16:0/503839028 --> [v2:172.21.15.16:6836/2351975329,v1:172.21.15.16:6837/2351975329] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ab991dd0a0 con 0x55ab992f2340 2023-05-20T01:22:27.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.080+0000 7fd0bcbee700 1 --2- 172.21.15.16:0/503839028 >> [v2:172.21.15.16:6836/2351975329,v1:172.21.15.16:6837/2351975329] conn(0x55ab992f2340 0x55ab992f2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:27.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.081+0000 7fd0977fe700 10 client.6861 ms_handle_connect on v2:172.21.15.16:6836/2351975329 2023-05-20T01:22:27.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.081+0000 7fd0bcbee700 1 --2- 172.21.15.16:0/503839028 >> [v2:172.21.15.16:6836/2351975329,v1:172.21.15.16:6837/2351975329] conn(0x55ab992f2340 0x55ab992f2740 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb1877537a25fbec server_cookie=200ebc06ce7a518f in_seq=0 out_seq=0 2023-05-20T01:22:27.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.081+0000 7fd0977fe700 1 -- 172.21.15.16:0/503839028 <== mds.0 v2:172.21.15.16:6836/2351975329 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ab991dd0a0 con 0x55ab992f2340 2023-05-20T01:22:27.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.081+0000 7fd0977fe700 10 client.6861 handle_command_reply: tid=0 2023-05-20T01:22:27.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.081+0000 7fd0cafdeb80 1 client.6861 shutdown 2023-05-20T01:22:27.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.081+0000 7fd0cafdeb80 1 -- 172.21.15.16:0/503839028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab992c94b0 msgr2=0x55ab992f6450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:27.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.081+0000 7fd0cafdeb80 1 --2- 172.21.15.16:0/503839028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab992c94b0 0x55ab992f6450 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fd0a4004000 tx=0x7fd0a400e000 comp rx=0 tx=0).stop 2023-05-20T01:22:27.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.081+0000 7fd0cafdeb80 1 -- 172.21.15.16:0/503839028 shutdown_connections 2023-05-20T01:22:27.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.081+0000 7fd0cafdeb80 1 --2- 172.21.15.16:0/503839028 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ab992f0eb0 0x55ab99326250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:27.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.081+0000 7fd0cafdeb80 1 --2- 172.21.15.16:0/503839028 >> [v2:172.21.15.16:6836/2351975329,v1:172.21.15.16:6837/2351975329] conn(0x55ab992f2340 0x55ab992f2740 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:27.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.081+0000 7fd0cafdeb80 1 --2- 172.21.15.16:0/503839028 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ab992c78c0 0x55ab99320140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:27.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.081+0000 7fd0cafdeb80 1 --2- 172.21.15.16:0/503839028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab992c94b0 0x55ab992f6450 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:27.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.081+0000 7fd0977fe700 0 client.6861 ms_handle_reset on v2:172.21.15.16:6836/2351975329 2023-05-20T01:22:27.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.081+0000 7fd0cafdeb80 1 -- 172.21.15.16:0/503839028 >> 172.21.15.16:0/503839028 conn(0x55ab992bef80 msgr2=0x55ab992c4360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:27.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.081+0000 7fd0cafdeb80 1 -- 172.21.15.16:0/503839028 shutdown_connections 2023-05-20T01:22:27.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.081+0000 7fd0cafdeb80 1 -- 172.21.15.16:0/503839028 wait complete. 2023-05-20T01:22:27.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.082+0000 7fd0cafdeb80 20 client.6861 trim_cache size 0 max 16384 2023-05-20T01:22:27.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.099+0000 7fd0cafdeb80 1 Processor -- start 2023-05-20T01:22:27.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.099+0000 7fd0cafdeb80 1 -- start start 2023-05-20T01:22:27.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.099+0000 7fd0cafdeb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab992c78c0 0x55ab992f2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:27.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.099+0000 7fd0cafdeb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ab992c94b0 0x55ab992f2c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:27.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.099+0000 7fd0cafdeb80 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ab992f0eb0 0x55ab99322f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:27.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.099+0000 7fd0cafdeb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab992c5e40 con 0x55ab992c78c0 2023-05-20T01:22:27.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.099+0000 7fd0cafdeb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab992c8940 con 0x55ab992c94b0 2023-05-20T01:22:27.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.099+0000 7fd0cafdeb80 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ab992f01f0 con 0x55ab992f0eb0 2023-05-20T01:22:27.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.099+0000 7fd0bcbee700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab992c78c0 0x55ab992f2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:27.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.099+0000 7fd0bcbee700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab992c78c0 0x55ab992f2710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35454/0 (socket says 172.21.15.16:35454) 2023-05-20T01:22:27.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.099+0000 7fd0bcbee700 1 -- 172.21.15.16:0/2857800470 learned_addr learned my addr 172.21.15.16:0/2857800470 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:22:27.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.100+0000 7fd0bd3ef700 1 --2- 172.21.15.16:0/2857800470 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ab992f0eb0 0x55ab99322f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:27.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.100+0000 7fd0bcbee700 1 -- 172.21.15.16:0/2857800470 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ab992f0eb0 msgr2=0x55ab99322f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:27.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.100+0000 7fd0bcbee700 1 --2- 172.21.15.16:0/2857800470 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ab992f0eb0 0x55ab99322f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:27.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.100+0000 7fd0bcbee700 1 -- 172.21.15.16:0/2857800470 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ab992c94b0 msgr2=0x55ab992f2c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:27.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.100+0000 7fd0b7fff700 1 --2- 172.21.15.16:0/2857800470 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ab992c94b0 0x55ab992f2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:27.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.100+0000 7fd0bcbee700 1 --2- 172.21.15.16:0/2857800470 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ab992c94b0 0x55ab992f2c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:27.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.100+0000 7fd0bcbee700 1 -- 172.21.15.16:0/2857800470 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ab991e0ee0 con 0x55ab992c78c0 2023-05-20T01:22:27.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.100+0000 7fd0bcbee700 1 --2- 172.21.15.16:0/2857800470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab992c78c0 0x55ab992f2710 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fd0ac0544b0 tx=0x7fd0ac004f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c4ef89ad80cf58f server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:27.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.100+0000 7fd0a2ffd700 1 -- 172.21.15.16:0/2857800470 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0ac0560f0 con 0x55ab992c78c0 2023-05-20T01:22:27.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.100+0000 7fd0a2ffd700 1 -- 172.21.15.16:0/2857800470 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0ac056350 con 0x55ab992c78c0 2023-05-20T01:22:27.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.100+0000 7fd0a2ffd700 1 -- 172.21.15.16:0/2857800470 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0ac0574a0 con 0x55ab992c78c0 2023-05-20T01:22:27.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.101+0000 7fd0cafdeb80 1 -- 172.21.15.16:0/2857800470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab992c78c0 msgr2=0x55ab992f2710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:27.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.101+0000 7fd0cafdeb80 1 --2- 172.21.15.16:0/2857800470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab992c78c0 0x55ab992f2710 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fd0ac0544b0 tx=0x7fd0ac004f90 comp rx=0 tx=0).stop 2023-05-20T01:22:27.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.101+0000 7fd0cafdeb80 1 -- 172.21.15.16:0/2857800470 shutdown_connections 2023-05-20T01:22:27.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.101+0000 7fd0cafdeb80 1 --2- 172.21.15.16:0/2857800470 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ab992f0eb0 0x55ab99322f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:27.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.101+0000 7fd0cafdeb80 1 --2- 172.21.15.16:0/2857800470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab992c78c0 0x55ab992f2710 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:27.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.101+0000 7fd0cafdeb80 1 --2- 172.21.15.16:0/2857800470 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ab992c94b0 0x55ab992f2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:27.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.101+0000 7fd0cafdeb80 1 -- 172.21.15.16:0/2857800470 >> 172.21.15.16:0/2857800470 conn(0x55ab992bef80 msgr2=0x55ab992c0450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:27.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.101+0000 7fd0cafdeb80 1 -- 172.21.15.16:0/2857800470 shutdown_connections 2023-05-20T01:22:27.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.101+0000 7fd0cafdeb80 1 -- 172.21.15.16:0/2857800470 wait complete. 2023-05-20T01:22:27.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.101+0000 7fd0cafdeb80 1 -- start start 2023-05-20T01:22:27.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.102+0000 7fd0cafdeb80 1 Processor -- start 2023-05-20T01:22:27.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.102+0000 7fd0cafdeb80 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ab992c78c0 0x55ab99363300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:27.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.102+0000 7fd0cafdeb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab992c94b0 0x55ab992f2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:27.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.102+0000 7fd0cafdeb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ab992f0eb0 0x55ab9937d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:27.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.102+0000 7fd0cafdeb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab9937d9f0 con 0x55ab992c94b0 2023-05-20T01:22:27.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.102+0000 7fd0cafdeb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab9937de50 con 0x55ab992f0eb0 2023-05-20T01:22:27.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.102+0000 7fd0cafdeb80 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ab9931f000 con 0x55ab992c78c0 2023-05-20T01:22:27.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.102+0000 7fd0cafdeb80 10 client.0 _ll_register_callbacks cb 0x7fd0a4003770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-05-20T01:22:27.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.102+0000 7fd0b7fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab992c94b0 0x55ab992f2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:27.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.102+0000 7fd0b7fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab992c94b0 0x55ab992f2cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35462/0 (socket says 172.21.15.16:35462) 2023-05-20T01:22:27.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.102+0000 7fd0b7fff700 1 -- 172.21.15.16:0/2989848594 learned_addr learned my addr 172.21.15.16:0/2989848594 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:22:27.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.102+0000 7fd0bcbee700 1 --2- 172.21.15.16:0/2989848594 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ab992c78c0 0x55ab99363300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:27.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.103+0000 7fd0bd3ef700 1 --2- 172.21.15.16:0/2989848594 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ab992f0eb0 0x55ab9937d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:27.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.103+0000 7fd0b7fff700 1 -- 172.21.15.16:0/2989848594 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ab992c78c0 msgr2=0x55ab99363300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:27.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.103+0000 7fd0b7fff700 1 --2- 172.21.15.16:0/2989848594 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ab992c78c0 0x55ab99363300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:27.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.103+0000 7fd0b7fff700 1 -- 172.21.15.16:0/2989848594 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ab992f0eb0 msgr2=0x55ab9937d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:27.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.103+0000 7fd0b7fff700 1 --2- 172.21.15.16:0/2989848594 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ab992f0eb0 0x55ab9937d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:27.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.103+0000 7fd0b7fff700 1 -- 172.21.15.16:0/2989848594 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0b80720c0 con 0x55ab992c94b0 2023-05-20T01:22:27.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.103+0000 7fd0b7fff700 1 --2- 172.21.15.16:0/2989848594 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab992c94b0 0x55ab992f2cd0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fd0a4009b90 tx=0x7fd0a40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f566213cb28ba07d server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:27.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.103+0000 7fd0977fe700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-05-20T01:22:27.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.103+0000 7fd0977fe700 1 -- 172.21.15.16:0/2989848594 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0a401ac70 con 0x55ab992c94b0 2023-05-20T01:22:27.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.103+0000 7fd0977fe700 1 -- 172.21.15.16:0/2989848594 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(75..75 src has 1..75) v4 ==== 7189+0+0 (secure 0 0 0) 0x7fd0a4011630 con 0x55ab992c94b0 2023-05-20T01:22:27.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.103+0000 7fd0cafdeb80 1 -- 172.21.15.16:0/2989848594 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd0b000ee20 con 0x55ab992c94b0 2023-05-20T01:22:27.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.103+0000 7fd0977fe700 1 -- 172.21.15.16:0/2989848594 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd0a4011840 con 0x55ab992c94b0 2023-05-20T01:22:27.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.103+0000 7fd0cafdeb80 10 client.6867 fetch_fsmap learned FSMap version 49 2023-05-20T01:22:27.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.103+0000 7fd0cafdeb80 1 -- 172.21.15.16:0/2989848594 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=2+}) v3 -- 0x55ab99031c60 con 0x55ab992c94b0 2023-05-20T01:22:27.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.103+0000 7fd0977fe700 1 -- 172.21.15.16:0/2989848594 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0b000ee20 con 0x55ab992c94b0 2023-05-20T01:22:27.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.103+0000 7fd0977fe700 1 -- 172.21.15.16:0/2989848594 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 49) v1 ==== 3122+0+0 (secure 0 0 0) 0x7fd0a4018070 con 0x55ab992c94b0 2023-05-20T01:22:27.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.104+0000 7fd0cafdeb80 10 client.6867 fetch_fsmap finished waiting for FSMap version 49 2023-05-20T01:22:27.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.104+0000 7fd0cafdeb80 10 client.6867 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2023-05-20T01:22:27.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.104+0000 7fd0cafdeb80 20 client.6867 populate_metadata read hostname 'smithi016' 2023-05-20T01:22:27.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.104+0000 7fd0cafdeb80 1 --2- 172.21.15.16:0/2989848594 >> [v2:172.21.15.16:6836/2351975329,v1:172.21.15.16:6837/2351975329] conn(0x55ab9931eb50 0x55ab99310e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:27.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.104+0000 7fd0cafdeb80 4 client.6867 mds_command: new command op to 6768 tid=0[{"prefix": "scrub status"}] 2023-05-20T01:22:27.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.104+0000 7fd0cafdeb80 1 -- 172.21.15.16:0/2989848594 --> [v2:172.21.15.16:6836/2351975329,v1:172.21.15.16:6837/2351975329] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ab991e9660 con 0x55ab9931eb50 2023-05-20T01:22:27.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.104+0000 7fd0bcbee700 1 --2- 172.21.15.16:0/2989848594 >> [v2:172.21.15.16:6836/2351975329,v1:172.21.15.16:6837/2351975329] conn(0x55ab9931eb50 0x55ab99310e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:27.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.104+0000 7fd0977fe700 10 client.6867 ms_handle_connect on v2:172.21.15.16:6836/2351975329 2023-05-20T01:22:27.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.104+0000 7fd0bcbee700 1 --2- 172.21.15.16:0/2989848594 >> [v2:172.21.15.16:6836/2351975329,v1:172.21.15.16:6837/2351975329] conn(0x55ab9931eb50 0x55ab99310e50 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=651489a6ba6544b server_cookie=e467f9b53ad989be in_seq=0 out_seq=0 2023-05-20T01:22:27.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.104+0000 7fd0977fe700 1 -- 172.21.15.16:0/2989848594 <== mds.0 v2:172.21.15.16:6836/2351975329 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+260 (crc 0 0 0) 0x55ab991e9660 con 0x55ab9931eb50 2023-05-20T01:22:27.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.104+0000 7fd0977fe700 10 client.6867 handle_command_reply: tid=0 2023-05-20T01:22:27.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.104+0000 7fd0cafdeb80 1 client.6867 shutdown 2023-05-20T01:22:27.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.104+0000 7fd0cafdeb80 1 -- 172.21.15.16:0/2989848594 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab992c94b0 msgr2=0x55ab992f2cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:27.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.104+0000 7fd0cafdeb80 1 --2- 172.21.15.16:0/2989848594 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab992c94b0 0x55ab992f2cd0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fd0a4009b90 tx=0x7fd0a40047c0 comp rx=0 tx=0).stop 2023-05-20T01:22:27.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.104+0000 7fd0cafdeb80 1 -- 172.21.15.16:0/2989848594 shutdown_connections 2023-05-20T01:22:27.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.104+0000 7fd0cafdeb80 1 --2- 172.21.15.16:0/2989848594 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ab992f0eb0 0x55ab9937d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:27.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.104+0000 7fd0cafdeb80 1 --2- 172.21.15.16:0/2989848594 >> [v2:172.21.15.16:6836/2351975329,v1:172.21.15.16:6837/2351975329] conn(0x55ab9931eb50 0x55ab99310e50 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:27.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.104+0000 7fd0cafdeb80 1 --2- 172.21.15.16:0/2989848594 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ab992c78c0 0x55ab99363300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:27.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.104+0000 7fd0cafdeb80 1 --2- 172.21.15.16:0/2989848594 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab992c94b0 0x55ab992f2cd0 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:27.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.104+0000 7fd0977fe700 0 client.6867 ms_handle_reset on v2:172.21.15.16:6836/2351975329 2023-05-20T01:22:27.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.104+0000 7fd0cafdeb80 1 -- 172.21.15.16:0/2989848594 >> 172.21.15.16:0/2989848594 conn(0x55ab992bef80 msgr2=0x55ab992c7df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:27.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.105+0000 7fd0cafdeb80 1 -- 172.21.15.16:0/2989848594 shutdown_connections 2023-05-20T01:22:27.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.105+0000 7fd0cafdeb80 1 -- 172.21.15.16:0/2989848594 wait complete. 2023-05-20T01:22:27.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.105+0000 7fd0cafdeb80 20 client.6867 trim_cache size 0 max 16384 2023-05-20T01:22:27.121 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-05-20T01:22:27.121 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2023-05-20T01:22:27.121 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2023-05-20T01:22:27.121 INFO:teuthology.orchestra.run.smithi016.stdout: "0a81b0f4-34fe-417d-bdbf-a6436d7519fe": { 2023-05-20T01:22:27.122 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2023-05-20T01:22:27.122 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "0a81b0f4-34fe-417d-bdbf-a6436d7519fe", 2023-05-20T01:22:27.122 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,repair" 2023-05-20T01:22:27.122 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:22:27.122 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:22:27.122 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-05-20T01:22:27.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.105+0000 7fd0bee52700 1 -- 172.21.15.16:0/3235793373 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd09c06ace0 msgr2=0x7fd09c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:27.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.105+0000 7fd0bee52700 1 --2- 172.21.15.16:0/3235793373 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd09c06ace0 0x7fd09c06d190 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fd0a4009840 tx=0x7fd0a4008030 comp rx=0 tx=0).stop 2023-05-20T01:22:27.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.105+0000 7fd0bee52700 1 -- 172.21.15.16:0/3235793373 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0b80710d0 msgr2=0x7fd0b80b3100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:27.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.105+0000 7fd0bee52700 1 --2- 172.21.15.16:0/3235793373 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0b80710d0 0x7fd0b80b3100 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fd0ac00c520 tx=0x7fd0ac00c8e0 comp rx=0 tx=0).stop 2023-05-20T01:22:27.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.105+0000 7fd0bee52700 1 -- 172.21.15.16:0/3235793373 shutdown_connections 2023-05-20T01:22:27.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.105+0000 7fd0bee52700 1 --2- 172.21.15.16:0/3235793373 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd09c06ace0 0x7fd09c06d190 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:27.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.105+0000 7fd0bee52700 1 --2- 172.21.15.16:0/3235793373 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd0b815cf40 0x7fd0b8158550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:27.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.105+0000 7fd0bee52700 1 --2- 172.21.15.16:0/3235793373 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0b80710d0 0x7fd0b80b3100 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:27.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.105+0000 7fd0bee52700 1 --2- 172.21.15.16:0/3235793373 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd0b80cd070 0x7fd0b814fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:27.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.105+0000 7fd0bee52700 1 -- 172.21.15.16:0/3235793373 >> 172.21.15.16:0/3235793373 conn(0x7fd0b806c260 msgr2=0x7fd0b806a510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:27.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.105+0000 7fd0bee52700 1 -- 172.21.15.16:0/3235793373 shutdown_connections 2023-05-20T01:22:27.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:27.105+0000 7fd0bee52700 1 -- 172.21.15.16:0/3235793373 wait complete. 2023-05-20T01:22:27.126 INFO:tasks.cephfs.filesystem:scrub status for tag:0a81b0f4-34fe-417d-bdbf-a6436d7519fe - {'path': '/', 'tag': '0a81b0f4-34fe-417d-bdbf-a6436d7519fe', 'options': 'recursive,repair'} 2023-05-20T01:22:36.404 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:22:36.408 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:22:57.127 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub status 2023-05-20T01:22:57.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.265+0000 7fd8f9a3b700 1 Processor -- start 2023-05-20T01:22:57.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.265+0000 7fd8f9a3b700 1 -- start start 2023-05-20T01:22:57.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.265+0000 7fd8f9a3b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd8f415cd40 0x7fd8f40d3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:57.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.265+0000 7fd8f9a3b700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fd8f40b4850 0x7fd8f40d0e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:57.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.266+0000 7fd8f9a3b700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8f40b60a0 con 0x7fd8f40d1360 2023-05-20T01:22:57.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.266+0000 7fd8f9a3b700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8f40b08d0 con 0x7fd8f415cd40 2023-05-20T01:22:57.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.266+0000 7fd8f9a3b700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8f40d1780 con 0x7fd8f40b4850 2023-05-20T01:22:57.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.266+0000 7fd8f2ffd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd8f415cd40 0x7fd8f40d3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:57.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.266+0000 7fd8f37fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fd8f40d1360 0x7fd8f40b2af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:46998/0 (socket says 172.21.15.16:46998) 2023-05-20T01:22:57.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.266+0000 7fd8f37fe700 1 -- 172.21.15.16:0/2474496804 learned_addr learned my addr 172.21.15.16:0/2474496804 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:22:57.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.266+0000 7fd8f2ffd700 1 --2- 172.21.15.16:0/2474496804 >> v2:172.21.15.16:3300/0 conn(0x7fd8f415cd40 0x7fd8f40d3450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35468/0 (socket says 172.21.15.16:35468) 2023-05-20T01:22:57.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.266+0000 7fd8f2ffd700 1 -- 172.21.15.16:0/2474496804 >> v2:172.21.15.132:3300/0 conn(0x7fd8f40b4850 msgr2=0x7fd8f40d0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:57.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.266+0000 7fd8f27fc700 1 --2- 172.21.15.16:0/2474496804 >> v2:172.21.15.132:3300/0 conn(0x7fd8f40b4850 0x7fd8f40d0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:57.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.266+0000 7fd8f2ffd700 1 --2- 172.21.15.16:0/2474496804 >> v2:172.21.15.132:3300/0 conn(0x7fd8f40b4850 0x7fd8f40d0e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:57.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.266+0000 7fd8f1ffb700 1 -- 172.21.15.16:0/2474496804 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 829077492 0 0) 0x7fd8f40b60a0 con 0x7fd8f40d1360 2023-05-20T01:22:57.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.266+0000 7fd8f2ffd700 1 -- 172.21.15.16:0/2474496804 >> v1:172.21.15.16:6789/0 conn(0x7fd8f40d1360 legacy=0x7fd8f40b2af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:57.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.266+0000 7fd8f2ffd700 1 -- 172.21.15.16:0/2474496804 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8f40affa0 con 0x7fd8f415cd40 2023-05-20T01:22:57.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.266+0000 7fd8f27fc700 1 --2- 172.21.15.16:0/2474496804 >> v2:172.21.15.132:3300/0 conn(0x7fd8f40b4850 0x7fd8f40d0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:22:57.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.266+0000 7fd8f2ffd700 1 --2- 172.21.15.16:0/2474496804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8f415cd40 0x7fd8f40d3450 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fd8f40b2320 tx=0x7fd8e8009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb24c8c98b64879c server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:57.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.266+0000 7fd8f1ffb700 1 -- 172.21.15.16:0/2474496804 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e8011830 con 0x7fd8f415cd40 2023-05-20T01:22:57.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.266+0000 7fd8f1ffb700 1 -- 172.21.15.16:0/2474496804 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8e8011990 con 0x7fd8f415cd40 2023-05-20T01:22:57.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.266+0000 7fd8f1ffb700 1 -- 172.21.15.16:0/2474496804 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e8011d20 con 0x7fd8f415cd40 2023-05-20T01:22:57.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.267+0000 7fd8f9a3b700 1 -- 172.21.15.16:0/2474496804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8f415cd40 msgr2=0x7fd8f40d3450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:57.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.267+0000 7fd8f9a3b700 1 --2- 172.21.15.16:0/2474496804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8f415cd40 0x7fd8f40d3450 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fd8f40b2320 tx=0x7fd8e8009b20 comp rx=0 tx=0).stop 2023-05-20T01:22:57.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.267+0000 7fd8f9a3b700 1 -- 172.21.15.16:0/2474496804 shutdown_connections 2023-05-20T01:22:57.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.267+0000 7fd8f9a3b700 1 --2- 172.21.15.16:0/2474496804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8f415cd40 0x7fd8f40d3450 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:57.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.267+0000 7fd8f9a3b700 1 --2- 172.21.15.16:0/2474496804 >> v2:172.21.15.132:3300/0 conn(0x7fd8f40b4850 0x7fd8f40d0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:57.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.267+0000 7fd8f9a3b700 1 -- 172.21.15.16:0/2474496804 >> 172.21.15.16:0/2474496804 conn(0x7fd8f4072560 msgr2=0x7fd8f409c5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:57.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.267+0000 7fd8f9a3b700 1 -- 172.21.15.16:0/2474496804 shutdown_connections 2023-05-20T01:22:57.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.267+0000 7fd8f9a3b700 1 -- 172.21.15.16:0/2474496804 wait complete. 2023-05-20T01:22:57.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.268+0000 7fd8f9a3b700 1 Processor -- start 2023-05-20T01:22:57.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.269+0000 7fd8f9a3b700 1 -- start start 2023-05-20T01:22:57.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.269+0000 7fd8f9a3b700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd8f40b4850 0x7fd8f414f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:57.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.269+0000 7fd8f9a3b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd8f40d1360 0x7fd8f40ca9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:57.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.269+0000 7fd8f9a3b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8f40cc350 0x7fd8f40b3430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:57.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.269+0000 7fd8f9a3b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f415d620 con 0x7fd8f40cc350 2023-05-20T01:22:57.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.269+0000 7fd8f9a3b700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f40caf20 con 0x7fd8f40d1360 2023-05-20T01:22:57.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.269+0000 7fd8f9a3b700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f40cd220 con 0x7fd8f40b4850 2023-05-20T01:22:57.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.269+0000 7fd8f37fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8f40cc350 0x7fd8f40b3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:57.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.269+0000 7fd8f37fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8f40cc350 0x7fd8f40b3430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35476/0 (socket says 172.21.15.16:35476) 2023-05-20T01:22:57.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.269+0000 7fd8f37fe700 1 -- 172.21.15.16:0/560092539 learned_addr learned my addr 172.21.15.16:0/560092539 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:22:57.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.270+0000 7fd8f2ffd700 1 --2- 172.21.15.16:0/560092539 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd8f40b4850 0x7fd8f414f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:57.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.270+0000 7fd8f27fc700 1 --2- 172.21.15.16:0/560092539 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd8f40d1360 0x7fd8f40ca9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:57.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.270+0000 7fd8f37fe700 1 -- 172.21.15.16:0/560092539 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd8f40b4850 msgr2=0x7fd8f414f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:57.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.270+0000 7fd8f37fe700 1 --2- 172.21.15.16:0/560092539 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd8f40b4850 0x7fd8f414f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:57.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.270+0000 7fd8f37fe700 1 -- 172.21.15.16:0/560092539 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd8f40d1360 msgr2=0x7fd8f40ca9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:57.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.270+0000 7fd8f37fe700 1 --2- 172.21.15.16:0/560092539 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd8f40d1360 0x7fd8f40ca9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:57.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.270+0000 7fd8f37fe700 1 -- 172.21.15.16:0/560092539 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8e80094a0 con 0x7fd8f40cc350 2023-05-20T01:22:57.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.270+0000 7fd8f27fc700 1 --2- 172.21.15.16:0/560092539 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd8f40d1360 0x7fd8f40ca9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:22:57.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.270+0000 7fd8f37fe700 1 --2- 172.21.15.16:0/560092539 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8f40cc350 0x7fd8f40b3430 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fd8ec002230 tx=0x7fd8ec00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:57.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.270+0000 7fd8dbfff700 1 -- 172.21.15.16:0/560092539 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8ec00ac20 con 0x7fd8f40cc350 2023-05-20T01:22:57.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.270+0000 7fd8dbfff700 1 -- 172.21.15.16:0/560092539 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8ec00ada0 con 0x7fd8f40cc350 2023-05-20T01:22:57.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.270+0000 7fd8f9a3b700 1 -- 172.21.15.16:0/560092539 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8f40b0560 con 0x7fd8f40cc350 2023-05-20T01:22:57.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.270+0000 7fd8dbfff700 1 -- 172.21.15.16:0/560092539 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8ec007310 con 0x7fd8f40cc350 2023-05-20T01:22:57.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.270+0000 7fd8f9a3b700 1 -- 172.21.15.16:0/560092539 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8f40b0280 con 0x7fd8f40cc350 2023-05-20T01:22:57.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.272+0000 7fd8dbfff700 1 -- 172.21.15.16:0/560092539 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd8ec007470 con 0x7fd8f40cc350 2023-05-20T01:22:57.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.272+0000 7fd8dbfff700 1 --2- 172.21.15.16:0/560092539 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd8e006ace0 0x7fd8e006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:57.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.272+0000 7fd8dbfff700 1 -- 172.21.15.16:0/560092539 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7189+0+0 (secure 0 0 0) 0x7fd8ec0897e0 con 0x7fd8f40cc350 2023-05-20T01:22:57.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.272+0000 7fd8f2ffd700 1 --2- 172.21.15.16:0/560092539 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd8e006ace0 0x7fd8e006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:57.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.273+0000 7fd8f2ffd700 1 --2- 172.21.15.16:0/560092539 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd8e006ace0 0x7fd8e006d190 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fd8f408d9b0 tx=0x7fd8e8008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:57.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.274+0000 7fd905bc7b80 1 Processor -- start 2023-05-20T01:22:57.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.274+0000 7fd905bc7b80 1 -- start start 2023-05-20T01:22:57.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.275+0000 7fd905bc7b80 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556c3cf14450 0x556c3cf14820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:57.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.275+0000 7fd905bc7b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c3cf10750 0x556c3cee94b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:57.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.275+0000 7fd905bc7b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556c3cee78c0 0x556c3cee99f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:57.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.275+0000 7fd905bc7b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c3cdef060 con 0x556c3cf10750 2023-05-20T01:22:57.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.275+0000 7fd905bc7b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c3ce418a0 con 0x556c3cee78c0 2023-05-20T01:22:57.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.275+0000 7fd905bc7b80 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x556c3ce3ad30 con 0x556c3cf14450 2023-05-20T01:22:57.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.275+0000 7fd8f27fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c3cf10750 0x556c3cee94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:57.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.275+0000 7fd8f27fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c3cf10750 0x556c3cee94b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35484/0 (socket says 172.21.15.16:35484) 2023-05-20T01:22:57.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.275+0000 7fd8f27fc700 1 -- 172.21.15.16:0/1524012178 learned_addr learned my addr 172.21.15.16:0/1524012178 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:22:57.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.275+0000 7fd8f2ffd700 1 --2- 172.21.15.16:0/1524012178 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556c3cf14450 0x556c3cf14820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:57.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.275+0000 7fd8f37fe700 1 --2- 172.21.15.16:0/1524012178 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556c3cee78c0 0x556c3cee99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:57.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.276+0000 7fd8f27fc700 1 -- 172.21.15.16:0/1524012178 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556c3cf14450 msgr2=0x556c3cf14820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:57.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.276+0000 7fd8f27fc700 1 --2- 172.21.15.16:0/1524012178 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556c3cf14450 0x556c3cf14820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:57.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.276+0000 7fd8f27fc700 1 -- 172.21.15.16:0/1524012178 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556c3cee78c0 msgr2=0x556c3cee99f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:57.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.276+0000 7fd8f27fc700 1 --2- 172.21.15.16:0/1524012178 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556c3cee78c0 0x556c3cee99f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:57.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.276+0000 7fd8f27fc700 1 -- 172.21.15.16:0/1524012178 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556c3ccbc630 con 0x556c3cf10750 2023-05-20T01:22:57.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.276+0000 7fd8f27fc700 1 --2- 172.21.15.16:0/1524012178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c3cf10750 0x556c3cee94b0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fd8e400abf0 tx=0x7fd8e400af00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3606d88a9eca3240 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:57.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.276+0000 7fd8d97fa700 1 -- 172.21.15.16:0/1524012178 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e4004030 con 0x556c3cf10750 2023-05-20T01:22:57.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.276+0000 7fd8d97fa700 1 -- 172.21.15.16:0/1524012178 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8e400c050 con 0x556c3cf10750 2023-05-20T01:22:57.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.276+0000 7fd8d97fa700 1 -- 172.21.15.16:0/1524012178 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e40043f0 con 0x556c3cf10750 2023-05-20T01:22:57.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.277+0000 7fd905bc7b80 1 -- 172.21.15.16:0/1524012178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c3cf10750 msgr2=0x556c3cee94b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:57.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.277+0000 7fd905bc7b80 1 --2- 172.21.15.16:0/1524012178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c3cf10750 0x556c3cee94b0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fd8e400abf0 tx=0x7fd8e400af00 comp rx=0 tx=0).stop 2023-05-20T01:22:57.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.277+0000 7fd905bc7b80 1 -- 172.21.15.16:0/1524012178 shutdown_connections 2023-05-20T01:22:57.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.277+0000 7fd905bc7b80 1 --2- 172.21.15.16:0/1524012178 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556c3cee78c0 0x556c3cee99f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:57.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.277+0000 7fd905bc7b80 1 --2- 172.21.15.16:0/1524012178 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556c3cf14450 0x556c3cf14820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:57.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.277+0000 7fd905bc7b80 1 --2- 172.21.15.16:0/1524012178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c3cf10750 0x556c3cee94b0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:57.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.277+0000 7fd905bc7b80 1 -- 172.21.15.16:0/1524012178 >> 172.21.15.16:0/1524012178 conn(0x556c3cedef80 msgr2=0x556c3cf2f260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:57.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.277+0000 7fd905bc7b80 1 -- 172.21.15.16:0/1524012178 shutdown_connections 2023-05-20T01:22:57.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.277+0000 7fd905bc7b80 1 -- 172.21.15.16:0/1524012178 wait complete. 2023-05-20T01:22:57.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.277+0000 7fd905bc7b80 1 -- start start 2023-05-20T01:22:57.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.277+0000 7fd905bc7b80 1 Processor -- start 2023-05-20T01:22:57.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.279+0000 7fd905bc7b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c3cee78c0 0x556c3cf28210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:57.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.279+0000 7fd905bc7b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556c3cf10750 0x556c3cf306e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:57.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.279+0000 7fd905bc7b80 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556c3cf14450 0x556c3cf30c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:57.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.279+0000 7fd905bc7b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c3cddc8e0 con 0x556c3cee78c0 2023-05-20T01:22:57.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.279+0000 7fd905bc7b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c3cddc580 con 0x556c3cf10750 2023-05-20T01:22:57.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.279+0000 7fd905bc7b80 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x556c3ccbe920 con 0x556c3cf14450 2023-05-20T01:22:57.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.279+0000 7fd905bc7b80 10 client.0 _ll_register_callbacks cb 0x556c3cdfa780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-05-20T01:22:57.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.279+0000 7fd8f2ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c3cee78c0 0x556c3cf28210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:57.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.279+0000 7fd8f2ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c3cee78c0 0x556c3cf28210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35488/0 (socket says 172.21.15.16:35488) 2023-05-20T01:22:57.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.279+0000 7fd8f2ffd700 1 -- 172.21.15.16:0/2214568906 learned_addr learned my addr 172.21.15.16:0/2214568906 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:22:57.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.279+0000 7fd8f27fc700 1 --2- 172.21.15.16:0/2214568906 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556c3cf10750 0x556c3cf306e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:57.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.279+0000 7fd8f37fe700 1 --2- 172.21.15.16:0/2214568906 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556c3cf14450 0x556c3cf30c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:57.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.279+0000 7fd8f2ffd700 1 -- 172.21.15.16:0/2214568906 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556c3cf14450 msgr2=0x556c3cf30c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:57.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.279+0000 7fd8f2ffd700 1 --2- 172.21.15.16:0/2214568906 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556c3cf14450 0x556c3cf30c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:57.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.279+0000 7fd8f2ffd700 1 -- 172.21.15.16:0/2214568906 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556c3cf10750 msgr2=0x556c3cf306e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:57.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.279+0000 7fd8f2ffd700 1 --2- 172.21.15.16:0/2214568906 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556c3cf10750 0x556c3cf306e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:57.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.279+0000 7fd8f2ffd700 1 -- 172.21.15.16:0/2214568906 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8e400a7c0 con 0x556c3cee78c0 2023-05-20T01:22:57.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.279+0000 7fd8f37fe700 1 --2- 172.21.15.16:0/2214568906 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556c3cf14450 0x556c3cf30c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:22:57.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.279+0000 7fd8f27fc700 1 --2- 172.21.15.16:0/2214568906 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556c3cf10750 0x556c3cf306e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:22:57.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.280+0000 7fd8f2ffd700 1 --2- 172.21.15.16:0/2214568906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c3cee78c0 0x556c3cf28210 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fd8e8011800 tx=0x7fd8e800e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b080a148d7b9bc57 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:57.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.280+0000 7fd8d27fc700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-05-20T01:22:57.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.280+0000 7fd8d27fc700 1 -- 172.21.15.16:0/2214568906 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e801f070 con 0x556c3cee78c0 2023-05-20T01:22:57.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.280+0000 7fd8d27fc700 1 -- 172.21.15.16:0/2214568906 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(75..75 src has 1..75) v4 ==== 7189+0+0 (secure 0 0 0) 0x7fd8e8003c90 con 0x556c3cee78c0 2023-05-20T01:22:57.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.280+0000 7fd905bc7b80 1 -- 172.21.15.16:0/2214568906 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556c3cedc740 con 0x556c3cee78c0 2023-05-20T01:22:57.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.280+0000 7fd8d27fc700 1 -- 172.21.15.16:0/2214568906 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd8e8003e90 con 0x556c3cee78c0 2023-05-20T01:22:57.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.280+0000 7fd905bc7b80 10 client.6882 fetch_fsmap learned FSMap version 49 2023-05-20T01:22:57.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.280+0000 7fd905bc7b80 1 -- 172.21.15.16:0/2214568906 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=2+}) v3 -- 0x556c3ce01190 con 0x556c3cee78c0 2023-05-20T01:22:57.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.280+0000 7fd8d27fc700 1 -- 172.21.15.16:0/2214568906 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556c3cedc740 con 0x556c3cee78c0 2023-05-20T01:22:57.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.280+0000 7fd8d27fc700 1 -- 172.21.15.16:0/2214568906 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 49) v1 ==== 3122+0+0 (secure 0 0 0) 0x7fd8e802f070 con 0x556c3cee78c0 2023-05-20T01:22:57.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.280+0000 7fd905bc7b80 10 client.6882 fetch_fsmap finished waiting for FSMap version 49 2023-05-20T01:22:57.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.280+0000 7fd905bc7b80 10 client.6882 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2023-05-20T01:22:57.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.280+0000 7fd905bc7b80 20 client.6882 populate_metadata read hostname 'smithi016' 2023-05-20T01:22:57.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.280+0000 7fd905bc7b80 1 --2- 172.21.15.16:0/2214568906 >> [v2:172.21.15.16:6836/2351975329,v1:172.21.15.16:6837/2351975329] conn(0x556c3cf33440 0x556c3ceea2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:57.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.280+0000 7fd905bc7b80 4 client.6882 mds_command: new command op to 6768 tid=0[{"prefix": "get_command_descriptions"}] 2023-05-20T01:22:57.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.280+0000 7fd905bc7b80 1 -- 172.21.15.16:0/2214568906 --> [v2:172.21.15.16:6836/2351975329,v1:172.21.15.16:6837/2351975329] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556c3cdfd0a0 con 0x556c3cf33440 2023-05-20T01:22:57.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.280+0000 7fd8f27fc700 1 --2- 172.21.15.16:0/2214568906 >> [v2:172.21.15.16:6836/2351975329,v1:172.21.15.16:6837/2351975329] conn(0x556c3cf33440 0x556c3ceea2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:57.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.281+0000 7fd8d27fc700 10 client.6882 ms_handle_connect on v2:172.21.15.16:6836/2351975329 2023-05-20T01:22:57.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.281+0000 7fd8f27fc700 1 --2- 172.21.15.16:0/2214568906 >> [v2:172.21.15.16:6836/2351975329,v1:172.21.15.16:6837/2351975329] conn(0x556c3cf33440 0x556c3ceea2b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9d6715fd4046619 server_cookie=98d33974c377e320 in_seq=0 out_seq=0 2023-05-20T01:22:57.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.281+0000 7fd8d27fc700 1 -- 172.21.15.16:0/2214568906 <== mds.0 v2:172.21.15.16:6836/2351975329 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x556c3cdfd0a0 con 0x556c3cf33440 2023-05-20T01:22:57.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.281+0000 7fd8d27fc700 10 client.6882 handle_command_reply: tid=0 2023-05-20T01:22:57.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.281+0000 7fd905bc7b80 1 client.6882 shutdown 2023-05-20T01:22:57.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.281+0000 7fd905bc7b80 1 -- 172.21.15.16:0/2214568906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c3cee78c0 msgr2=0x556c3cf28210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:57.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.281+0000 7fd905bc7b80 1 --2- 172.21.15.16:0/2214568906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c3cee78c0 0x556c3cf28210 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fd8e8011800 tx=0x7fd8e800e000 comp rx=0 tx=0).stop 2023-05-20T01:22:57.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.281+0000 7fd905bc7b80 1 -- 172.21.15.16:0/2214568906 shutdown_connections 2023-05-20T01:22:57.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.281+0000 7fd905bc7b80 1 --2- 172.21.15.16:0/2214568906 >> [v2:172.21.15.16:6836/2351975329,v1:172.21.15.16:6837/2351975329] conn(0x556c3cf33440 0x556c3ceea2b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:57.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.281+0000 7fd905bc7b80 1 --2- 172.21.15.16:0/2214568906 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556c3cf14450 0x556c3cf30c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:57.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.281+0000 7fd905bc7b80 1 --2- 172.21.15.16:0/2214568906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c3cee78c0 0x556c3cf28210 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:57.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.281+0000 7fd8d27fc700 0 client.6882 ms_handle_reset on v2:172.21.15.16:6836/2351975329 2023-05-20T01:22:57.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.281+0000 7fd905bc7b80 1 --2- 172.21.15.16:0/2214568906 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556c3cf10750 0x556c3cf306e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:57.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.281+0000 7fd905bc7b80 1 -- 172.21.15.16:0/2214568906 >> 172.21.15.16:0/2214568906 conn(0x556c3cedef80 msgr2=0x556c3cee4430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:57.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.281+0000 7fd905bc7b80 1 -- 172.21.15.16:0/2214568906 shutdown_connections 2023-05-20T01:22:57.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.282+0000 7fd905bc7b80 1 -- 172.21.15.16:0/2214568906 wait complete. 2023-05-20T01:22:57.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.282+0000 7fd905bc7b80 20 client.6882 trim_cache size 0 max 16384 2023-05-20T01:22:57.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.299+0000 7fd905bc7b80 1 Processor -- start 2023-05-20T01:22:57.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.299+0000 7fd905bc7b80 1 -- start start 2023-05-20T01:22:57.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.300+0000 7fd905bc7b80 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556c3cee78c0 0x556c3cf0e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:57.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.300+0000 7fd905bc7b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556c3cf10750 0x556c3cf0e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:57.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.300+0000 7fd905bc7b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c3cf14450 0x556c3cf318b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:57.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.300+0000 7fd905bc7b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c3cf14930 con 0x556c3cf14450 2023-05-20T01:22:57.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.300+0000 7fd905bc7b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c3cf27490 con 0x556c3cf10750 2023-05-20T01:22:57.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.300+0000 7fd905bc7b80 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x556c3cedeb50 con 0x556c3cee78c0 2023-05-20T01:22:57.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.300+0000 7fd8f37fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c3cf14450 0x556c3cf318b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:57.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.300+0000 7fd8f37fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c3cf14450 0x556c3cf318b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35498/0 (socket says 172.21.15.16:35498) 2023-05-20T01:22:57.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.300+0000 7fd8f37fe700 1 -- 172.21.15.16:0/2641451296 learned_addr learned my addr 172.21.15.16:0/2641451296 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:22:57.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.300+0000 7fd8f27fc700 1 --2- 172.21.15.16:0/2641451296 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556c3cf10750 0x556c3cf0e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:57.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.300+0000 7fd8f2ffd700 1 --2- 172.21.15.16:0/2641451296 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556c3cee78c0 0x556c3cf0e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:57.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.300+0000 7fd8f37fe700 1 -- 172.21.15.16:0/2641451296 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556c3cee78c0 msgr2=0x556c3cf0e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:57.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.300+0000 7fd8f37fe700 1 --2- 172.21.15.16:0/2641451296 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556c3cee78c0 0x556c3cf0e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:57.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.300+0000 7fd8f37fe700 1 -- 172.21.15.16:0/2641451296 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556c3cf10750 msgr2=0x556c3cf0e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:57.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.300+0000 7fd8f37fe700 1 --2- 172.21.15.16:0/2641451296 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556c3cf10750 0x556c3cf0e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:57.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.300+0000 7fd8f37fe700 1 -- 172.21.15.16:0/2641451296 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556c3ce00ee0 con 0x556c3cf14450 2023-05-20T01:22:57.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.300+0000 7fd8f37fe700 1 --2- 172.21.15.16:0/2641451296 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c3cf14450 0x556c3cf318b0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fd8ec054670 tx=0x7fd8ec009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d2f02cc5e06ca62 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:57.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.300+0000 7fd8d9ffb700 1 -- 172.21.15.16:0/2641451296 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8ec060030 con 0x556c3cf14450 2023-05-20T01:22:57.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.300+0000 7fd8d9ffb700 1 -- 172.21.15.16:0/2641451296 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8ec060190 con 0x556c3cf14450 2023-05-20T01:22:57.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.300+0000 7fd8d9ffb700 1 -- 172.21.15.16:0/2641451296 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8ec060300 con 0x556c3cf14450 2023-05-20T01:22:57.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.301+0000 7fd905bc7b80 1 -- 172.21.15.16:0/2641451296 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c3cf14450 msgr2=0x556c3cf318b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:57.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.301+0000 7fd905bc7b80 1 --2- 172.21.15.16:0/2641451296 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c3cf14450 0x556c3cf318b0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fd8ec054670 tx=0x7fd8ec009040 comp rx=0 tx=0).stop 2023-05-20T01:22:57.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.301+0000 7fd905bc7b80 1 -- 172.21.15.16:0/2641451296 shutdown_connections 2023-05-20T01:22:57.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.301+0000 7fd905bc7b80 1 --2- 172.21.15.16:0/2641451296 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c3cf14450 0x556c3cf318b0 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:57.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.301+0000 7fd905bc7b80 1 --2- 172.21.15.16:0/2641451296 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556c3cee78c0 0x556c3cf0e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:57.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.301+0000 7fd905bc7b80 1 --2- 172.21.15.16:0/2641451296 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556c3cf10750 0x556c3cf0e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:57.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.301+0000 7fd905bc7b80 1 -- 172.21.15.16:0/2641451296 >> 172.21.15.16:0/2641451296 conn(0x556c3cedef80 msgr2=0x556c3cf510b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:57.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.301+0000 7fd905bc7b80 1 -- 172.21.15.16:0/2641451296 shutdown_connections 2023-05-20T01:22:57.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.301+0000 7fd905bc7b80 1 -- 172.21.15.16:0/2641451296 wait complete. 2023-05-20T01:22:57.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.302+0000 7fd905bc7b80 1 -- start start 2023-05-20T01:22:57.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.302+0000 7fd905bc7b80 1 Processor -- start 2023-05-20T01:22:57.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.303+0000 7fd905bc7b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c3cee78c0 0x556c3cf8c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:57.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.303+0000 7fd905bc7b80 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556c3cf10750 0x556c3cf3dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:57.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.303+0000 7fd905bc7b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556c3cf14450 0x556c3cf3e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:57.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.303+0000 7fd905bc7b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c3cf28410 con 0x556c3cee78c0 2023-05-20T01:22:57.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.303+0000 7fd905bc7b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c3cf8cbb0 con 0x556c3cf14450 2023-05-20T01:22:57.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.303+0000 7fd905bc7b80 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x556c3cf0f7b0 con 0x556c3cf10750 2023-05-20T01:22:57.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.303+0000 7fd905bc7b80 10 client.0 _ll_register_callbacks cb 0x7fd8e8011c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-05-20T01:22:57.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.303+0000 7fd8f2ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c3cee78c0 0x556c3cf8c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:57.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.303+0000 7fd8f2ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c3cee78c0 0x556c3cf8c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35500/0 (socket says 172.21.15.16:35500) 2023-05-20T01:22:57.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.303+0000 7fd8f2ffd700 1 -- 172.21.15.16:0/2764521907 learned_addr learned my addr 172.21.15.16:0/2764521907 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:22:57.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.303+0000 7fd8f27fc700 1 --2- 172.21.15.16:0/2764521907 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556c3cf10750 0x556c3cf3dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:57.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.303+0000 7fd8f37fe700 1 --2- 172.21.15.16:0/2764521907 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556c3cf14450 0x556c3cf3e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:57.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.303+0000 7fd8f2ffd700 1 -- 172.21.15.16:0/2764521907 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556c3cf10750 msgr2=0x556c3cf3dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:57.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.303+0000 7fd8f2ffd700 1 --2- 172.21.15.16:0/2764521907 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556c3cf10750 0x556c3cf3dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:57.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.303+0000 7fd8f2ffd700 1 -- 172.21.15.16:0/2764521907 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556c3cf14450 msgr2=0x556c3cf3e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:22:57.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.303+0000 7fd8f2ffd700 1 --2- 172.21.15.16:0/2764521907 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556c3cf14450 0x556c3cf3e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:57.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.303+0000 7fd8f2ffd700 1 -- 172.21.15.16:0/2764521907 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8f40b0280 con 0x556c3cee78c0 2023-05-20T01:22:57.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.303+0000 7fd8f27fc700 1 --2- 172.21.15.16:0/2764521907 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556c3cf10750 0x556c3cf3dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:22:57.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.303+0000 7fd8f37fe700 1 --2- 172.21.15.16:0/2764521907 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556c3cf14450 0x556c3cf3e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:22:57.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.303+0000 7fd8f2ffd700 1 --2- 172.21.15.16:0/2764521907 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c3cee78c0 0x556c3cf8c670 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fd8e8011800 tx=0x7fd8e80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84d2d26e46a4c71 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:22:57.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.303+0000 7fd8d27fc700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-05-20T01:22:57.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.303+0000 7fd8d27fc700 1 -- 172.21.15.16:0/2764521907 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e8011b00 con 0x556c3cee78c0 2023-05-20T01:22:57.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.303+0000 7fd8d27fc700 1 -- 172.21.15.16:0/2764521907 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(75..75 src has 1..75) v4 ==== 7189+0+0 (secure 0 0 0) 0x7fd8e8006cc0 con 0x556c3cee78c0 2023-05-20T01:22:57.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.303+0000 7fd905bc7b80 1 -- 172.21.15.16:0/2764521907 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd8e400c050 con 0x556c3cee78c0 2023-05-20T01:22:57.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.303+0000 7fd8d27fc700 1 -- 172.21.15.16:0/2764521907 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd8e801ac40 con 0x556c3cee78c0 2023-05-20T01:22:57.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.303+0000 7fd905bc7b80 10 client.6888 fetch_fsmap learned FSMap version 49 2023-05-20T01:22:57.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.303+0000 7fd905bc7b80 1 -- 172.21.15.16:0/2764521907 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=2+}) v3 -- 0x556c3cc51c60 con 0x556c3cee78c0 2023-05-20T01:22:57.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.304+0000 7fd8d27fc700 1 -- 172.21.15.16:0/2764521907 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8e400c050 con 0x556c3cee78c0 2023-05-20T01:22:57.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.304+0000 7fd8d27fc700 1 -- 172.21.15.16:0/2764521907 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 49) v1 ==== 3122+0+0 (secure 0 0 0) 0x7fd8e80041e0 con 0x556c3cee78c0 2023-05-20T01:22:57.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.304+0000 7fd905bc7b80 10 client.6888 fetch_fsmap finished waiting for FSMap version 49 2023-05-20T01:22:57.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.304+0000 7fd905bc7b80 10 client.6888 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2023-05-20T01:22:57.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.304+0000 7fd905bc7b80 20 client.6888 populate_metadata read hostname 'smithi016' 2023-05-20T01:22:57.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.304+0000 7fd905bc7b80 1 --2- 172.21.15.16:0/2764521907 >> [v2:172.21.15.16:6836/2351975329,v1:172.21.15.16:6837/2351975329] conn(0x556c3cf161c0 0x556c3cf95cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:22:57.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.304+0000 7fd905bc7b80 4 client.6888 mds_command: new command op to 6768 tid=0[{"prefix": "scrub status"}] 2023-05-20T01:22:57.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.304+0000 7fd905bc7b80 1 -- 172.21.15.16:0/2764521907 --> [v2:172.21.15.16:6836/2351975329,v1:172.21.15.16:6837/2351975329] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x556c3ce09660 con 0x556c3cf161c0 2023-05-20T01:22:57.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.304+0000 7fd8f27fc700 1 --2- 172.21.15.16:0/2764521907 >> [v2:172.21.15.16:6836/2351975329,v1:172.21.15.16:6837/2351975329] conn(0x556c3cf161c0 0x556c3cf95cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:22:57.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.304+0000 7fd8f27fc700 1 --2- 172.21.15.16:0/2764521907 >> [v2:172.21.15.16:6836/2351975329,v1:172.21.15.16:6837/2351975329] conn(0x556c3cf161c0 0x556c3cf95cb0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ead95bfd9ceb12cf server_cookie=ae45e495a42f53aa in_seq=0 out_seq=0 2023-05-20T01:22:57.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.304+0000 7fd8d27fc700 10 client.6888 ms_handle_connect on v2:172.21.15.16:6836/2351975329 2023-05-20T01:22:57.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.304+0000 7fd8d27fc700 1 -- 172.21.15.16:0/2764521907 <== mds.0 v2:172.21.15.16:6836/2351975329 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x556c3ce09660 con 0x556c3cf161c0 2023-05-20T01:22:57.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.304+0000 7fd8d27fc700 10 client.6888 handle_command_reply: tid=0 2023-05-20T01:22:57.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.304+0000 7fd905bc7b80 1 client.6888 shutdown 2023-05-20T01:22:57.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.304+0000 7fd905bc7b80 1 -- 172.21.15.16:0/2764521907 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c3cee78c0 msgr2=0x556c3cf8c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:57.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.304+0000 7fd905bc7b80 1 --2- 172.21.15.16:0/2764521907 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c3cee78c0 0x556c3cf8c670 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fd8e8011800 tx=0x7fd8e80047c0 comp rx=0 tx=0).stop 2023-05-20T01:22:57.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.305+0000 7fd905bc7b80 1 -- 172.21.15.16:0/2764521907 shutdown_connections 2023-05-20T01:22:57.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.305+0000 7fd905bc7b80 1 --2- 172.21.15.16:0/2764521907 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556c3cf14450 0x556c3cf3e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:57.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.305+0000 7fd905bc7b80 1 --2- 172.21.15.16:0/2764521907 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c3cee78c0 0x556c3cf8c670 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:57.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.305+0000 7fd905bc7b80 1 --2- 172.21.15.16:0/2764521907 >> [v2:172.21.15.16:6836/2351975329,v1:172.21.15.16:6837/2351975329] conn(0x556c3cf161c0 0x556c3cf95cb0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:57.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.305+0000 7fd905bc7b80 1 --2- 172.21.15.16:0/2764521907 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556c3cf10750 0x556c3cf3dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:57.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.305+0000 7fd8d27fc700 0 client.6888 ms_handle_reset on v2:172.21.15.16:6836/2351975329 2023-05-20T01:22:57.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.305+0000 7fd905bc7b80 1 -- 172.21.15.16:0/2764521907 >> 172.21.15.16:0/2764521907 conn(0x556c3cedef80 msgr2=0x556c3cf15400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:57.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.305+0000 7fd905bc7b80 1 -- 172.21.15.16:0/2764521907 shutdown_connections 2023-05-20T01:22:57.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.305+0000 7fd905bc7b80 1 -- 172.21.15.16:0/2764521907 wait complete. 2023-05-20T01:22:57.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.305+0000 7fd905bc7b80 20 client.6888 trim_cache size 0 max 16384 2023-05-20T01:22:57.325 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-05-20T01:22:57.325 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2023-05-20T01:22:57.325 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2023-05-20T01:22:57.326 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-05-20T01:22:57.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.305+0000 7fd8f9a3b700 1 -- 172.21.15.16:0/560092539 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd8e006ace0 msgr2=0x7fd8e006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:57.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.305+0000 7fd8f9a3b700 1 --2- 172.21.15.16:0/560092539 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd8e006ace0 0x7fd8e006d190 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fd8f408d9b0 tx=0x7fd8e8008030 comp rx=0 tx=0).stop 2023-05-20T01:22:57.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.305+0000 7fd8f9a3b700 1 -- 172.21.15.16:0/560092539 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8f40cc350 msgr2=0x7fd8f40b3430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:22:57.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.305+0000 7fd8f9a3b700 1 --2- 172.21.15.16:0/560092539 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8f40cc350 0x7fd8f40b3430 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fd8ec002230 tx=0x7fd8ec00a060 comp rx=0 tx=0).stop 2023-05-20T01:22:57.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.305+0000 7fd8f9a3b700 1 -- 172.21.15.16:0/560092539 shutdown_connections 2023-05-20T01:22:57.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.305+0000 7fd8f9a3b700 1 --2- 172.21.15.16:0/560092539 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8f40cc350 0x7fd8f40b3430 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:57.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.305+0000 7fd8f9a3b700 1 --2- 172.21.15.16:0/560092539 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd8e006ace0 0x7fd8e006d190 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:57.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.305+0000 7fd8f9a3b700 1 --2- 172.21.15.16:0/560092539 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd8f40b4850 0x7fd8f414f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:57.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.305+0000 7fd8f9a3b700 1 --2- 172.21.15.16:0/560092539 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd8f40d1360 0x7fd8f40ca9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:22:57.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.305+0000 7fd8f9a3b700 1 -- 172.21.15.16:0/560092539 >> 172.21.15.16:0/560092539 conn(0x7fd8f4072560 msgr2=0x7fd8f40b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:22:57.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.305+0000 7fd8f9a3b700 1 -- 172.21.15.16:0/560092539 shutdown_connections 2023-05-20T01:22:57.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:22:57.305+0000 7fd8f9a3b700 1 -- 172.21.15.16:0/560092539 wait complete. 2023-05-20T01:22:57.329 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-05-20T01:22:57.329 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/0 2023-05-20T01:22:57.330 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-05-20T01:22:57.330 DEBUG:teuthology.orchestra.run.smithi016:> import zlib 2023-05-20T01:22:57.330 DEBUG:teuthology.orchestra.run.smithi016:> path = "/home/ubuntu/cephtest/mnt.0/subdir/0" 2023-05-20T01:22:57.331 DEBUG:teuthology.orchestra.run.smithi016:> with open(path, '"'"'r'"'"') as f: 2023-05-20T01:22:57.331 DEBUG:teuthology.orchestra.run.smithi016:> bytes = f.read() 2023-05-20T01:22:57.331 DEBUG:teuthology.orchestra.run.smithi016:> if len(bytes) != 6291456: 2023-05-20T01:22:57.331 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-05-20T01:22:57.332 DEBUG:teuthology.orchestra.run.smithi016:> len(bytes), 6291456 2023-05-20T01:22:57.332 DEBUG:teuthology.orchestra.run.smithi016:> )) 2023-05-20T01:22:57.332 DEBUG:teuthology.orchestra.run.smithi016:> for i, b in enumerate(bytes): 2023-05-20T01:22:57.333 DEBUG:teuthology.orchestra.run.smithi016:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-05-20T01:22:57.333 DEBUG:teuthology.orchestra.run.smithi016:> if b != chr(val): 2023-05-20T01:22:57.333 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-05-20T01:22:57.334 DEBUG:teuthology.orchestra.run.smithi016:> ' 2023-05-20T01:23:01.805 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/1 2023-05-20T01:23:01.805 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-05-20T01:23:01.806 DEBUG:teuthology.orchestra.run.smithi016:> import zlib 2023-05-20T01:23:01.806 DEBUG:teuthology.orchestra.run.smithi016:> path = "/home/ubuntu/cephtest/mnt.0/subdir/1" 2023-05-20T01:23:01.806 DEBUG:teuthology.orchestra.run.smithi016:> with open(path, '"'"'r'"'"') as f: 2023-05-20T01:23:01.807 DEBUG:teuthology.orchestra.run.smithi016:> bytes = f.read() 2023-05-20T01:23:01.807 DEBUG:teuthology.orchestra.run.smithi016:> if len(bytes) != 6291456: 2023-05-20T01:23:01.807 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-05-20T01:23:01.808 DEBUG:teuthology.orchestra.run.smithi016:> len(bytes), 6291456 2023-05-20T01:23:01.808 DEBUG:teuthology.orchestra.run.smithi016:> )) 2023-05-20T01:23:01.808 DEBUG:teuthology.orchestra.run.smithi016:> for i, b in enumerate(bytes): 2023-05-20T01:23:01.809 DEBUG:teuthology.orchestra.run.smithi016:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-05-20T01:23:01.809 DEBUG:teuthology.orchestra.run.smithi016:> if b != chr(val): 2023-05-20T01:23:01.809 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-05-20T01:23:01.809 DEBUG:teuthology.orchestra.run.smithi016:> ' 2023-05-20T01:23:06.317 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/2 2023-05-20T01:23:06.317 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-05-20T01:23:06.318 DEBUG:teuthology.orchestra.run.smithi016:> import zlib 2023-05-20T01:23:06.318 DEBUG:teuthology.orchestra.run.smithi016:> path = "/home/ubuntu/cephtest/mnt.0/subdir/2" 2023-05-20T01:23:06.318 DEBUG:teuthology.orchestra.run.smithi016:> with open(path, '"'"'r'"'"') as f: 2023-05-20T01:23:06.319 DEBUG:teuthology.orchestra.run.smithi016:> bytes = f.read() 2023-05-20T01:23:06.319 DEBUG:teuthology.orchestra.run.smithi016:> if len(bytes) != 6291456: 2023-05-20T01:23:06.319 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-05-20T01:23:06.320 DEBUG:teuthology.orchestra.run.smithi016:> len(bytes), 6291456 2023-05-20T01:23:06.320 DEBUG:teuthology.orchestra.run.smithi016:> )) 2023-05-20T01:23:06.320 DEBUG:teuthology.orchestra.run.smithi016:> for i, b in enumerate(bytes): 2023-05-20T01:23:06.320 DEBUG:teuthology.orchestra.run.smithi016:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-05-20T01:23:06.321 DEBUG:teuthology.orchestra.run.smithi016:> if b != chr(val): 2023-05-20T01:23:06.321 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-05-20T01:23:06.321 DEBUG:teuthology.orchestra.run.smithi016:> ' 2023-05-20T01:23:06.467 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:23:06.469 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:23:10.829 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/3 2023-05-20T01:23:10.830 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-05-20T01:23:10.830 DEBUG:teuthology.orchestra.run.smithi016:> import zlib 2023-05-20T01:23:10.831 DEBUG:teuthology.orchestra.run.smithi016:> path = "/home/ubuntu/cephtest/mnt.0/subdir/3" 2023-05-20T01:23:10.832 DEBUG:teuthology.orchestra.run.smithi016:> with open(path, '"'"'r'"'"') as f: 2023-05-20T01:23:10.833 DEBUG:teuthology.orchestra.run.smithi016:> bytes = f.read() 2023-05-20T01:23:10.833 DEBUG:teuthology.orchestra.run.smithi016:> if len(bytes) != 6291456: 2023-05-20T01:23:10.834 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-05-20T01:23:10.835 DEBUG:teuthology.orchestra.run.smithi016:> len(bytes), 6291456 2023-05-20T01:23:10.835 DEBUG:teuthology.orchestra.run.smithi016:> )) 2023-05-20T01:23:10.836 DEBUG:teuthology.orchestra.run.smithi016:> for i, b in enumerate(bytes): 2023-05-20T01:23:10.837 DEBUG:teuthology.orchestra.run.smithi016:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-05-20T01:23:10.837 DEBUG:teuthology.orchestra.run.smithi016:> if b != chr(val): 2023-05-20T01:23:10.838 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-05-20T01:23:10.838 DEBUG:teuthology.orchestra.run.smithi016:> ' 2023-05-20T01:23:15.310 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/4 2023-05-20T01:23:15.311 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-05-20T01:23:15.311 DEBUG:teuthology.orchestra.run.smithi016:> import zlib 2023-05-20T01:23:15.311 DEBUG:teuthology.orchestra.run.smithi016:> path = "/home/ubuntu/cephtest/mnt.0/subdir/4" 2023-05-20T01:23:15.312 DEBUG:teuthology.orchestra.run.smithi016:> with open(path, '"'"'r'"'"') as f: 2023-05-20T01:23:15.312 DEBUG:teuthology.orchestra.run.smithi016:> bytes = f.read() 2023-05-20T01:23:15.312 DEBUG:teuthology.orchestra.run.smithi016:> if len(bytes) != 6291456: 2023-05-20T01:23:15.313 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-05-20T01:23:15.313 DEBUG:teuthology.orchestra.run.smithi016:> len(bytes), 6291456 2023-05-20T01:23:15.313 DEBUG:teuthology.orchestra.run.smithi016:> )) 2023-05-20T01:23:15.313 DEBUG:teuthology.orchestra.run.smithi016:> for i, b in enumerate(bytes): 2023-05-20T01:23:15.314 DEBUG:teuthology.orchestra.run.smithi016:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-05-20T01:23:15.314 DEBUG:teuthology.orchestra.run.smithi016:> if b != chr(val): 2023-05-20T01:23:15.314 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-05-20T01:23:15.315 DEBUG:teuthology.orchestra.run.smithi016:> ' 2023-05-20T01:23:19.824 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/5 2023-05-20T01:23:19.825 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-05-20T01:23:19.825 DEBUG:teuthology.orchestra.run.smithi016:> import zlib 2023-05-20T01:23:19.826 DEBUG:teuthology.orchestra.run.smithi016:> path = "/home/ubuntu/cephtest/mnt.0/subdir/5" 2023-05-20T01:23:19.826 DEBUG:teuthology.orchestra.run.smithi016:> with open(path, '"'"'r'"'"') as f: 2023-05-20T01:23:19.827 DEBUG:teuthology.orchestra.run.smithi016:> bytes = f.read() 2023-05-20T01:23:19.827 DEBUG:teuthology.orchestra.run.smithi016:> if len(bytes) != 6291456: 2023-05-20T01:23:19.828 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-05-20T01:23:19.829 DEBUG:teuthology.orchestra.run.smithi016:> len(bytes), 6291456 2023-05-20T01:23:19.829 DEBUG:teuthology.orchestra.run.smithi016:> )) 2023-05-20T01:23:19.830 DEBUG:teuthology.orchestra.run.smithi016:> for i, b in enumerate(bytes): 2023-05-20T01:23:19.830 DEBUG:teuthology.orchestra.run.smithi016:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-05-20T01:23:19.831 DEBUG:teuthology.orchestra.run.smithi016:> if b != chr(val): 2023-05-20T01:23:19.831 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-05-20T01:23:19.832 DEBUG:teuthology.orchestra.run.smithi016:> ' 2023-05-20T01:23:24.335 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/6 2023-05-20T01:23:24.336 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-05-20T01:23:24.336 DEBUG:teuthology.orchestra.run.smithi016:> import zlib 2023-05-20T01:23:24.337 DEBUG:teuthology.orchestra.run.smithi016:> path = "/home/ubuntu/cephtest/mnt.0/subdir/6" 2023-05-20T01:23:24.337 DEBUG:teuthology.orchestra.run.smithi016:> with open(path, '"'"'r'"'"') as f: 2023-05-20T01:23:24.337 DEBUG:teuthology.orchestra.run.smithi016:> bytes = f.read() 2023-05-20T01:23:24.338 DEBUG:teuthology.orchestra.run.smithi016:> if len(bytes) != 6291456: 2023-05-20T01:23:24.338 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-05-20T01:23:24.338 DEBUG:teuthology.orchestra.run.smithi016:> len(bytes), 6291456 2023-05-20T01:23:24.338 DEBUG:teuthology.orchestra.run.smithi016:> )) 2023-05-20T01:23:24.339 DEBUG:teuthology.orchestra.run.smithi016:> for i, b in enumerate(bytes): 2023-05-20T01:23:24.339 DEBUG:teuthology.orchestra.run.smithi016:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-05-20T01:23:24.339 DEBUG:teuthology.orchestra.run.smithi016:> if b != chr(val): 2023-05-20T01:23:24.339 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-05-20T01:23:24.340 DEBUG:teuthology.orchestra.run.smithi016:> ' 2023-05-20T01:23:29.048 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/7 2023-05-20T01:23:29.049 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-05-20T01:23:29.049 DEBUG:teuthology.orchestra.run.smithi016:> import zlib 2023-05-20T01:23:29.049 DEBUG:teuthology.orchestra.run.smithi016:> path = "/home/ubuntu/cephtest/mnt.0/subdir/7" 2023-05-20T01:23:29.050 DEBUG:teuthology.orchestra.run.smithi016:> with open(path, '"'"'r'"'"') as f: 2023-05-20T01:23:29.050 DEBUG:teuthology.orchestra.run.smithi016:> bytes = f.read() 2023-05-20T01:23:29.050 DEBUG:teuthology.orchestra.run.smithi016:> if len(bytes) != 6291456: 2023-05-20T01:23:29.050 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-05-20T01:23:29.051 DEBUG:teuthology.orchestra.run.smithi016:> len(bytes), 6291456 2023-05-20T01:23:29.051 DEBUG:teuthology.orchestra.run.smithi016:> )) 2023-05-20T01:23:29.051 DEBUG:teuthology.orchestra.run.smithi016:> for i, b in enumerate(bytes): 2023-05-20T01:23:29.051 DEBUG:teuthology.orchestra.run.smithi016:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-05-20T01:23:29.052 DEBUG:teuthology.orchestra.run.smithi016:> if b != chr(val): 2023-05-20T01:23:29.052 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-05-20T01:23:29.052 DEBUG:teuthology.orchestra.run.smithi016:> ' 2023-05-20T01:23:33.560 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/8 2023-05-20T01:23:33.561 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-05-20T01:23:33.561 DEBUG:teuthology.orchestra.run.smithi016:> import zlib 2023-05-20T01:23:33.562 DEBUG:teuthology.orchestra.run.smithi016:> path = "/home/ubuntu/cephtest/mnt.0/subdir/8" 2023-05-20T01:23:33.562 DEBUG:teuthology.orchestra.run.smithi016:> with open(path, '"'"'r'"'"') as f: 2023-05-20T01:23:33.562 DEBUG:teuthology.orchestra.run.smithi016:> bytes = f.read() 2023-05-20T01:23:33.563 DEBUG:teuthology.orchestra.run.smithi016:> if len(bytes) != 6291456: 2023-05-20T01:23:33.563 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-05-20T01:23:33.563 DEBUG:teuthology.orchestra.run.smithi016:> len(bytes), 6291456 2023-05-20T01:23:33.563 DEBUG:teuthology.orchestra.run.smithi016:> )) 2023-05-20T01:23:33.564 DEBUG:teuthology.orchestra.run.smithi016:> for i, b in enumerate(bytes): 2023-05-20T01:23:33.564 DEBUG:teuthology.orchestra.run.smithi016:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-05-20T01:23:33.564 DEBUG:teuthology.orchestra.run.smithi016:> if b != chr(val): 2023-05-20T01:23:33.565 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-05-20T01:23:33.565 DEBUG:teuthology.orchestra.run.smithi016:> ' 2023-05-20T01:23:36.526 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:23:36.529 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:23:38.273 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/9 2023-05-20T01:23:38.274 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-05-20T01:23:38.274 DEBUG:teuthology.orchestra.run.smithi016:> import zlib 2023-05-20T01:23:38.275 DEBUG:teuthology.orchestra.run.smithi016:> path = "/home/ubuntu/cephtest/mnt.0/subdir/9" 2023-05-20T01:23:38.275 DEBUG:teuthology.orchestra.run.smithi016:> with open(path, '"'"'r'"'"') as f: 2023-05-20T01:23:38.275 DEBUG:teuthology.orchestra.run.smithi016:> bytes = f.read() 2023-05-20T01:23:38.276 DEBUG:teuthology.orchestra.run.smithi016:> if len(bytes) != 6291456: 2023-05-20T01:23:38.276 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-05-20T01:23:38.276 DEBUG:teuthology.orchestra.run.smithi016:> len(bytes), 6291456 2023-05-20T01:23:38.276 DEBUG:teuthology.orchestra.run.smithi016:> )) 2023-05-20T01:23:38.277 DEBUG:teuthology.orchestra.run.smithi016:> for i, b in enumerate(bytes): 2023-05-20T01:23:38.277 DEBUG:teuthology.orchestra.run.smithi016:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-05-20T01:23:38.277 DEBUG:teuthology.orchestra.run.smithi016:> if b != chr(val): 2023-05-20T01:23:38.277 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-05-20T01:23:38.278 DEBUG:teuthology.orchestra.run.smithi016:> ' 2023-05-20T01:23:42.751 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/10 2023-05-20T01:23:42.752 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-05-20T01:23:42.753 DEBUG:teuthology.orchestra.run.smithi016:> import zlib 2023-05-20T01:23:42.753 DEBUG:teuthology.orchestra.run.smithi016:> path = "/home/ubuntu/cephtest/mnt.0/subdir/10" 2023-05-20T01:23:42.753 DEBUG:teuthology.orchestra.run.smithi016:> with open(path, '"'"'r'"'"') as f: 2023-05-20T01:23:42.753 DEBUG:teuthology.orchestra.run.smithi016:> bytes = f.read() 2023-05-20T01:23:42.754 DEBUG:teuthology.orchestra.run.smithi016:> if len(bytes) != 6291456: 2023-05-20T01:23:42.754 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-05-20T01:23:42.754 DEBUG:teuthology.orchestra.run.smithi016:> len(bytes), 6291456 2023-05-20T01:23:42.755 DEBUG:teuthology.orchestra.run.smithi016:> )) 2023-05-20T01:23:42.755 DEBUG:teuthology.orchestra.run.smithi016:> for i, b in enumerate(bytes): 2023-05-20T01:23:42.755 DEBUG:teuthology.orchestra.run.smithi016:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-05-20T01:23:42.755 DEBUG:teuthology.orchestra.run.smithi016:> if b != chr(val): 2023-05-20T01:23:42.756 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-05-20T01:23:42.756 DEBUG:teuthology.orchestra.run.smithi016:> ' 2023-05-20T01:23:47.266 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/11 2023-05-20T01:23:47.267 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-05-20T01:23:47.267 DEBUG:teuthology.orchestra.run.smithi016:> import zlib 2023-05-20T01:23:47.268 DEBUG:teuthology.orchestra.run.smithi016:> path = "/home/ubuntu/cephtest/mnt.0/subdir/11" 2023-05-20T01:23:47.268 DEBUG:teuthology.orchestra.run.smithi016:> with open(path, '"'"'r'"'"') as f: 2023-05-20T01:23:47.268 DEBUG:teuthology.orchestra.run.smithi016:> bytes = f.read() 2023-05-20T01:23:47.269 DEBUG:teuthology.orchestra.run.smithi016:> if len(bytes) != 6291456: 2023-05-20T01:23:47.269 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-05-20T01:23:47.269 DEBUG:teuthology.orchestra.run.smithi016:> len(bytes), 6291456 2023-05-20T01:23:47.269 DEBUG:teuthology.orchestra.run.smithi016:> )) 2023-05-20T01:23:47.270 DEBUG:teuthology.orchestra.run.smithi016:> for i, b in enumerate(bytes): 2023-05-20T01:23:47.270 DEBUG:teuthology.orchestra.run.smithi016:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-05-20T01:23:47.270 DEBUG:teuthology.orchestra.run.smithi016:> if b != chr(val): 2023-05-20T01:23:47.270 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-05-20T01:23:47.271 DEBUG:teuthology.orchestra.run.smithi016:> ' 2023-05-20T01:23:51.778 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/12 2023-05-20T01:23:51.779 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-05-20T01:23:51.779 DEBUG:teuthology.orchestra.run.smithi016:> import zlib 2023-05-20T01:23:51.780 DEBUG:teuthology.orchestra.run.smithi016:> path = "/home/ubuntu/cephtest/mnt.0/subdir/12" 2023-05-20T01:23:51.780 DEBUG:teuthology.orchestra.run.smithi016:> with open(path, '"'"'r'"'"') as f: 2023-05-20T01:23:51.780 DEBUG:teuthology.orchestra.run.smithi016:> bytes = f.read() 2023-05-20T01:23:51.781 DEBUG:teuthology.orchestra.run.smithi016:> if len(bytes) != 6291456: 2023-05-20T01:23:51.781 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-05-20T01:23:51.781 DEBUG:teuthology.orchestra.run.smithi016:> len(bytes), 6291456 2023-05-20T01:23:51.781 DEBUG:teuthology.orchestra.run.smithi016:> )) 2023-05-20T01:23:51.782 DEBUG:teuthology.orchestra.run.smithi016:> for i, b in enumerate(bytes): 2023-05-20T01:23:51.782 DEBUG:teuthology.orchestra.run.smithi016:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-05-20T01:23:51.782 DEBUG:teuthology.orchestra.run.smithi016:> if b != chr(val): 2023-05-20T01:23:51.783 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-05-20T01:23:51.783 DEBUG:teuthology.orchestra.run.smithi016:> ' 2023-05-20T01:23:56.491 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/13 2023-05-20T01:23:56.491 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-05-20T01:23:56.492 DEBUG:teuthology.orchestra.run.smithi016:> import zlib 2023-05-20T01:23:56.492 DEBUG:teuthology.orchestra.run.smithi016:> path = "/home/ubuntu/cephtest/mnt.0/subdir/13" 2023-05-20T01:23:56.492 DEBUG:teuthology.orchestra.run.smithi016:> with open(path, '"'"'r'"'"') as f: 2023-05-20T01:23:56.493 DEBUG:teuthology.orchestra.run.smithi016:> bytes = f.read() 2023-05-20T01:23:56.493 DEBUG:teuthology.orchestra.run.smithi016:> if len(bytes) != 6291456: 2023-05-20T01:23:56.493 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-05-20T01:23:56.493 DEBUG:teuthology.orchestra.run.smithi016:> len(bytes), 6291456 2023-05-20T01:23:56.494 DEBUG:teuthology.orchestra.run.smithi016:> )) 2023-05-20T01:23:56.494 DEBUG:teuthology.orchestra.run.smithi016:> for i, b in enumerate(bytes): 2023-05-20T01:23:56.495 DEBUG:teuthology.orchestra.run.smithi016:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-05-20T01:23:56.495 DEBUG:teuthology.orchestra.run.smithi016:> if b != chr(val): 2023-05-20T01:23:56.495 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-05-20T01:23:56.496 DEBUG:teuthology.orchestra.run.smithi016:> ' 2023-05-20T01:24:01.004 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/14 2023-05-20T01:24:01.004 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-05-20T01:24:01.005 DEBUG:teuthology.orchestra.run.smithi016:> import zlib 2023-05-20T01:24:01.005 DEBUG:teuthology.orchestra.run.smithi016:> path = "/home/ubuntu/cephtest/mnt.0/subdir/14" 2023-05-20T01:24:01.005 DEBUG:teuthology.orchestra.run.smithi016:> with open(path, '"'"'r'"'"') as f: 2023-05-20T01:24:01.006 DEBUG:teuthology.orchestra.run.smithi016:> bytes = f.read() 2023-05-20T01:24:01.006 DEBUG:teuthology.orchestra.run.smithi016:> if len(bytes) != 6291456: 2023-05-20T01:24:01.006 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-05-20T01:24:01.007 DEBUG:teuthology.orchestra.run.smithi016:> len(bytes), 6291456 2023-05-20T01:24:01.007 DEBUG:teuthology.orchestra.run.smithi016:> )) 2023-05-20T01:24:01.007 DEBUG:teuthology.orchestra.run.smithi016:> for i, b in enumerate(bytes): 2023-05-20T01:24:01.007 DEBUG:teuthology.orchestra.run.smithi016:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-05-20T01:24:01.008 DEBUG:teuthology.orchestra.run.smithi016:> if b != chr(val): 2023-05-20T01:24:01.008 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-05-20T01:24:01.008 DEBUG:teuthology.orchestra.run.smithi016:> ' 2023-05-20T01:24:05.517 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/15 2023-05-20T01:24:05.517 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-05-20T01:24:05.518 DEBUG:teuthology.orchestra.run.smithi016:> import zlib 2023-05-20T01:24:05.518 DEBUG:teuthology.orchestra.run.smithi016:> path = "/home/ubuntu/cephtest/mnt.0/subdir/15" 2023-05-20T01:24:05.518 DEBUG:teuthology.orchestra.run.smithi016:> with open(path, '"'"'r'"'"') as f: 2023-05-20T01:24:05.518 DEBUG:teuthology.orchestra.run.smithi016:> bytes = f.read() 2023-05-20T01:24:05.519 DEBUG:teuthology.orchestra.run.smithi016:> if len(bytes) != 6291456: 2023-05-20T01:24:05.519 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-05-20T01:24:05.519 DEBUG:teuthology.orchestra.run.smithi016:> len(bytes), 6291456 2023-05-20T01:24:05.520 DEBUG:teuthology.orchestra.run.smithi016:> )) 2023-05-20T01:24:05.520 DEBUG:teuthology.orchestra.run.smithi016:> for i, b in enumerate(bytes): 2023-05-20T01:24:05.520 DEBUG:teuthology.orchestra.run.smithi016:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-05-20T01:24:05.520 DEBUG:teuthology.orchestra.run.smithi016:> if b != chr(val): 2023-05-20T01:24:05.521 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-05-20T01:24:05.521 DEBUG:teuthology.orchestra.run.smithi016:> ' 2023-05-20T01:24:06.587 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:24:06.589 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:24:10.431 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/16 2023-05-20T01:24:10.431 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-05-20T01:24:10.432 DEBUG:teuthology.orchestra.run.smithi016:> import zlib 2023-05-20T01:24:10.432 DEBUG:teuthology.orchestra.run.smithi016:> path = "/home/ubuntu/cephtest/mnt.0/subdir/16" 2023-05-20T01:24:10.432 DEBUG:teuthology.orchestra.run.smithi016:> with open(path, '"'"'r'"'"') as f: 2023-05-20T01:24:10.433 DEBUG:teuthology.orchestra.run.smithi016:> bytes = f.read() 2023-05-20T01:24:10.433 DEBUG:teuthology.orchestra.run.smithi016:> if len(bytes) != 6291456: 2023-05-20T01:24:10.433 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-05-20T01:24:10.433 DEBUG:teuthology.orchestra.run.smithi016:> len(bytes), 6291456 2023-05-20T01:24:10.434 DEBUG:teuthology.orchestra.run.smithi016:> )) 2023-05-20T01:24:10.434 DEBUG:teuthology.orchestra.run.smithi016:> for i, b in enumerate(bytes): 2023-05-20T01:24:10.434 DEBUG:teuthology.orchestra.run.smithi016:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-05-20T01:24:10.435 DEBUG:teuthology.orchestra.run.smithi016:> if b != chr(val): 2023-05-20T01:24:10.435 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-05-20T01:24:10.435 DEBUG:teuthology.orchestra.run.smithi016:> ' 2023-05-20T01:24:14.907 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/17 2023-05-20T01:24:14.908 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-05-20T01:24:14.908 DEBUG:teuthology.orchestra.run.smithi016:> import zlib 2023-05-20T01:24:14.909 DEBUG:teuthology.orchestra.run.smithi016:> path = "/home/ubuntu/cephtest/mnt.0/subdir/17" 2023-05-20T01:24:14.909 DEBUG:teuthology.orchestra.run.smithi016:> with open(path, '"'"'r'"'"') as f: 2023-05-20T01:24:14.910 DEBUG:teuthology.orchestra.run.smithi016:> bytes = f.read() 2023-05-20T01:24:14.910 DEBUG:teuthology.orchestra.run.smithi016:> if len(bytes) != 6291456: 2023-05-20T01:24:14.910 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-05-20T01:24:14.911 DEBUG:teuthology.orchestra.run.smithi016:> len(bytes), 6291456 2023-05-20T01:24:14.911 DEBUG:teuthology.orchestra.run.smithi016:> )) 2023-05-20T01:24:14.912 DEBUG:teuthology.orchestra.run.smithi016:> for i, b in enumerate(bytes): 2023-05-20T01:24:14.912 DEBUG:teuthology.orchestra.run.smithi016:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-05-20T01:24:14.912 DEBUG:teuthology.orchestra.run.smithi016:> if b != chr(val): 2023-05-20T01:24:14.913 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-05-20T01:24:14.913 DEBUG:teuthology.orchestra.run.smithi016:> ' 2023-05-20T01:24:19.418 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/18 2023-05-20T01:24:19.419 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-05-20T01:24:19.419 DEBUG:teuthology.orchestra.run.smithi016:> import zlib 2023-05-20T01:24:19.420 DEBUG:teuthology.orchestra.run.smithi016:> path = "/home/ubuntu/cephtest/mnt.0/subdir/18" 2023-05-20T01:24:19.420 DEBUG:teuthology.orchestra.run.smithi016:> with open(path, '"'"'r'"'"') as f: 2023-05-20T01:24:19.421 DEBUG:teuthology.orchestra.run.smithi016:> bytes = f.read() 2023-05-20T01:24:19.421 DEBUG:teuthology.orchestra.run.smithi016:> if len(bytes) != 6291456: 2023-05-20T01:24:19.422 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-05-20T01:24:19.422 DEBUG:teuthology.orchestra.run.smithi016:> len(bytes), 6291456 2023-05-20T01:24:19.423 DEBUG:teuthology.orchestra.run.smithi016:> )) 2023-05-20T01:24:19.423 DEBUG:teuthology.orchestra.run.smithi016:> for i, b in enumerate(bytes): 2023-05-20T01:24:19.424 DEBUG:teuthology.orchestra.run.smithi016:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-05-20T01:24:19.425 DEBUG:teuthology.orchestra.run.smithi016:> if b != chr(val): 2023-05-20T01:24:19.425 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-05-20T01:24:19.426 DEBUG:teuthology.orchestra.run.smithi016:> ' 2023-05-20T01:24:23.931 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/19 2023-05-20T01:24:23.931 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-05-20T01:24:23.932 DEBUG:teuthology.orchestra.run.smithi016:> import zlib 2023-05-20T01:24:23.932 DEBUG:teuthology.orchestra.run.smithi016:> path = "/home/ubuntu/cephtest/mnt.0/subdir/19" 2023-05-20T01:24:23.932 DEBUG:teuthology.orchestra.run.smithi016:> with open(path, '"'"'r'"'"') as f: 2023-05-20T01:24:23.933 DEBUG:teuthology.orchestra.run.smithi016:> bytes = f.read() 2023-05-20T01:24:23.933 DEBUG:teuthology.orchestra.run.smithi016:> if len(bytes) != 6291456: 2023-05-20T01:24:23.933 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-05-20T01:24:23.934 DEBUG:teuthology.orchestra.run.smithi016:> len(bytes), 6291456 2023-05-20T01:24:23.934 DEBUG:teuthology.orchestra.run.smithi016:> )) 2023-05-20T01:24:23.934 DEBUG:teuthology.orchestra.run.smithi016:> for i, b in enumerate(bytes): 2023-05-20T01:24:23.934 DEBUG:teuthology.orchestra.run.smithi016:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-05-20T01:24:23.935 DEBUG:teuthology.orchestra.run.smithi016:> if b != chr(val): 2023-05-20T01:24:23.935 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-05-20T01:24:23.935 DEBUG:teuthology.orchestra.run.smithi016:> ' 2023-05-20T01:24:28.442 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/20 2023-05-20T01:24:28.442 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-05-20T01:24:28.443 DEBUG:teuthology.orchestra.run.smithi016:> import zlib 2023-05-20T01:24:28.443 DEBUG:teuthology.orchestra.run.smithi016:> path = "/home/ubuntu/cephtest/mnt.0/subdir/20" 2023-05-20T01:24:28.444 DEBUG:teuthology.orchestra.run.smithi016:> with open(path, '"'"'r'"'"') as f: 2023-05-20T01:24:28.444 DEBUG:teuthology.orchestra.run.smithi016:> bytes = f.read() 2023-05-20T01:24:28.444 DEBUG:teuthology.orchestra.run.smithi016:> if len(bytes) != 6291456: 2023-05-20T01:24:28.445 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-05-20T01:24:28.445 DEBUG:teuthology.orchestra.run.smithi016:> len(bytes), 6291456 2023-05-20T01:24:28.445 DEBUG:teuthology.orchestra.run.smithi016:> )) 2023-05-20T01:24:28.445 DEBUG:teuthology.orchestra.run.smithi016:> for i, b in enumerate(bytes): 2023-05-20T01:24:28.446 DEBUG:teuthology.orchestra.run.smithi016:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-05-20T01:24:28.446 DEBUG:teuthology.orchestra.run.smithi016:> if b != chr(val): 2023-05-20T01:24:28.446 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-05-20T01:24:28.447 DEBUG:teuthology.orchestra.run.smithi016:> ' 2023-05-20T01:24:32.955 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/21 2023-05-20T01:24:32.956 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-05-20T01:24:32.957 DEBUG:teuthology.orchestra.run.smithi016:> import zlib 2023-05-20T01:24:32.957 DEBUG:teuthology.orchestra.run.smithi016:> path = "/home/ubuntu/cephtest/mnt.0/subdir/21" 2023-05-20T01:24:32.958 DEBUG:teuthology.orchestra.run.smithi016:> with open(path, '"'"'r'"'"') as f: 2023-05-20T01:24:32.958 DEBUG:teuthology.orchestra.run.smithi016:> bytes = f.read() 2023-05-20T01:24:32.958 DEBUG:teuthology.orchestra.run.smithi016:> if len(bytes) != 6291456: 2023-05-20T01:24:32.958 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-05-20T01:24:32.959 DEBUG:teuthology.orchestra.run.smithi016:> len(bytes), 6291456 2023-05-20T01:24:32.959 DEBUG:teuthology.orchestra.run.smithi016:> )) 2023-05-20T01:24:32.959 DEBUG:teuthology.orchestra.run.smithi016:> for i, b in enumerate(bytes): 2023-05-20T01:24:32.960 DEBUG:teuthology.orchestra.run.smithi016:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-05-20T01:24:32.960 DEBUG:teuthology.orchestra.run.smithi016:> if b != chr(val): 2023-05-20T01:24:32.960 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-05-20T01:24:32.961 DEBUG:teuthology.orchestra.run.smithi016:> ' 2023-05-20T01:24:36.675 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:24:36.677 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:24:37.467 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/22 2023-05-20T01:24:37.467 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-05-20T01:24:37.468 DEBUG:teuthology.orchestra.run.smithi016:> import zlib 2023-05-20T01:24:37.468 DEBUG:teuthology.orchestra.run.smithi016:> path = "/home/ubuntu/cephtest/mnt.0/subdir/22" 2023-05-20T01:24:37.468 DEBUG:teuthology.orchestra.run.smithi016:> with open(path, '"'"'r'"'"') as f: 2023-05-20T01:24:37.469 DEBUG:teuthology.orchestra.run.smithi016:> bytes = f.read() 2023-05-20T01:24:37.469 DEBUG:teuthology.orchestra.run.smithi016:> if len(bytes) != 6291456: 2023-05-20T01:24:37.469 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-05-20T01:24:37.470 DEBUG:teuthology.orchestra.run.smithi016:> len(bytes), 6291456 2023-05-20T01:24:37.470 DEBUG:teuthology.orchestra.run.smithi016:> )) 2023-05-20T01:24:37.470 DEBUG:teuthology.orchestra.run.smithi016:> for i, b in enumerate(bytes): 2023-05-20T01:24:37.470 DEBUG:teuthology.orchestra.run.smithi016:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-05-20T01:24:37.471 DEBUG:teuthology.orchestra.run.smithi016:> if b != chr(val): 2023-05-20T01:24:37.471 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-05-20T01:24:37.471 DEBUG:teuthology.orchestra.run.smithi016:> ' 2023-05-20T01:24:41.944 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/23 2023-05-20T01:24:41.945 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-05-20T01:24:41.945 DEBUG:teuthology.orchestra.run.smithi016:> import zlib 2023-05-20T01:24:41.946 DEBUG:teuthology.orchestra.run.smithi016:> path = "/home/ubuntu/cephtest/mnt.0/subdir/23" 2023-05-20T01:24:41.946 DEBUG:teuthology.orchestra.run.smithi016:> with open(path, '"'"'r'"'"') as f: 2023-05-20T01:24:41.947 DEBUG:teuthology.orchestra.run.smithi016:> bytes = f.read() 2023-05-20T01:24:41.947 DEBUG:teuthology.orchestra.run.smithi016:> if len(bytes) != 6291456: 2023-05-20T01:24:41.948 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-05-20T01:24:41.948 DEBUG:teuthology.orchestra.run.smithi016:> len(bytes), 6291456 2023-05-20T01:24:41.949 DEBUG:teuthology.orchestra.run.smithi016:> )) 2023-05-20T01:24:41.949 DEBUG:teuthology.orchestra.run.smithi016:> for i, b in enumerate(bytes): 2023-05-20T01:24:41.950 DEBUG:teuthology.orchestra.run.smithi016:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-05-20T01:24:41.950 DEBUG:teuthology.orchestra.run.smithi016:> if b != chr(val): 2023-05-20T01:24:41.951 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-05-20T01:24:41.951 DEBUG:teuthology.orchestra.run.smithi016:> ' 2023-05-20T01:24:46.457 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/24 2023-05-20T01:24:46.457 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-05-20T01:24:46.458 DEBUG:teuthology.orchestra.run.smithi016:> import zlib 2023-05-20T01:24:46.458 DEBUG:teuthology.orchestra.run.smithi016:> path = "/home/ubuntu/cephtest/mnt.0/subdir/24" 2023-05-20T01:24:46.458 DEBUG:teuthology.orchestra.run.smithi016:> with open(path, '"'"'r'"'"') as f: 2023-05-20T01:24:46.459 DEBUG:teuthology.orchestra.run.smithi016:> bytes = f.read() 2023-05-20T01:24:46.459 DEBUG:teuthology.orchestra.run.smithi016:> if len(bytes) != 6291456: 2023-05-20T01:24:46.459 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-05-20T01:24:46.459 DEBUG:teuthology.orchestra.run.smithi016:> len(bytes), 6291456 2023-05-20T01:24:46.460 DEBUG:teuthology.orchestra.run.smithi016:> )) 2023-05-20T01:24:46.460 DEBUG:teuthology.orchestra.run.smithi016:> for i, b in enumerate(bytes): 2023-05-20T01:24:46.460 DEBUG:teuthology.orchestra.run.smithi016:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-05-20T01:24:46.461 DEBUG:teuthology.orchestra.run.smithi016:> if b != chr(val): 2023-05-20T01:24:46.461 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-05-20T01:24:46.461 DEBUG:teuthology.orchestra.run.smithi016:> ' 2023-05-20T01:24:50.968 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-05-20T01:24:50.969 DEBUG:teuthology.orchestra.run.smithi016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-05-20T01:24:51.048 DEBUG:teuthology.orchestra.run.smithi132:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-05-20T01:24:51.095 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-05-20T01:24:51.095 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-05-20T01:24:51.096 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2023-05-20T01:24:51.116 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-05-20T01:24:51.117 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2023-05-20T01:24:51.175 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-20T01:24:51.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.337+0000 7f26f4054700 1 Processor -- start 2023-05-20T01:24:51.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.338+0000 7f26f4054700 1 -- start start 2023-05-20T01:24:51.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.338+0000 7f26f4054700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f26ec0b4750 0x7f26ec0d2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:51.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.338+0000 7f26f4054700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26ec0b3b50 con 0x7f26ec0cfc50 2023-05-20T01:24:51.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.338+0000 7f26f4054700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26ec0d0b90 con 0x7f26ec0cd970 2023-05-20T01:24:51.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.338+0000 7f26f4054700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26ec0cfad0 con 0x7f26ec0b4750 2023-05-20T01:24:51.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.339+0000 7f26f1df0700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f26ec0cfc50 0x7f26ec0d0020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:47052/0 (socket says 172.21.15.16:47052) 2023-05-20T01:24:51.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.339+0000 7f26f1df0700 1 -- 172.21.15.16:0/249622757 learned_addr learned my addr 172.21.15.16:0/249622757 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:24:51.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.339+0000 7f26f15ef700 1 --2- 172.21.15.16:0/249622757 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f26ec0b4750 0x7f26ec0d2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:51.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.339+0000 7f26f0dee700 1 -- 172.21.15.16:0/249622757 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3876714727 0 0) 0x7f26ec0b3b50 con 0x7f26ec0cfc50 2023-05-20T01:24:51.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.339+0000 7f26f0dee700 1 -- 172.21.15.16:0/249622757 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f26d80035b0 con 0x7f26ec0cfc50 2023-05-20T01:24:51.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.339+0000 7f26f0dee700 1 -- 172.21.15.16:0/249622757 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 910809259 0 0) 0x7f26ec0d0b90 con 0x7f26ec0cd970 2023-05-20T01:24:51.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.339+0000 7f26f0dee700 1 -- 172.21.15.16:0/249622757 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f26ec0b3b50 con 0x7f26ec0cd970 2023-05-20T01:24:51.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.339+0000 7f26f0dee700 1 -- 172.21.15.16:0/249622757 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 708863045 0 0) 0x7f26d80035b0 con 0x7f26ec0cfc50 2023-05-20T01:24:51.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.339+0000 7f26f0dee700 1 -- 172.21.15.16:0/249622757 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f26ec0b4750 msgr2=0x7f26ec0d2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:24:51.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.339+0000 7f26f0dee700 1 --2- 172.21.15.16:0/249622757 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f26ec0b4750 0x7f26ec0d2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:51.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.339+0000 7f26f0dee700 1 -- 172.21.15.16:0/249622757 >> v1:172.21.15.132:6789/0 conn(0x7f26ec0cd970 legacy=0x7f26ec0cddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:51.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.339+0000 7f26f0dee700 1 -- 172.21.15.16:0/249622757 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26ec15d0b0 con 0x7f26ec0cfc50 2023-05-20T01:24:51.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.339+0000 7f26f0dee700 1 -- 172.21.15.16:0/249622757 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7f26e8001670 con 0x7f26ec0cfc50 2023-05-20T01:24:51.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.339+0000 7f26f0dee700 1 -- 172.21.15.16:0/249622757 >> v1:172.21.15.16:6789/0 conn(0x7f26ec0cfc50 legacy=0x7f26ec0d0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:51.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.339+0000 7f26f0dee700 1 --2- 172.21.15.16:0/249622757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26d8003c30 0x7f26d8004020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:51.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.339+0000 7f26f0dee700 1 --2- 172.21.15.16:0/249622757 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f26ec0b4750 0x7f26d8004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:51.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.339+0000 7f26f0dee700 1 --2- 172.21.15.16:0/249622757 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f26d8004d90 0x7f26d8005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:51.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.339+0000 7f26f0dee700 1 -- 172.21.15.16:0/249622757 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26ec0cfad0 con 0x7f26d8003c30 2023-05-20T01:24:51.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.339+0000 7f26f0dee700 1 -- 172.21.15.16:0/249622757 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26d800bbd0 con 0x7f26d8004d90 2023-05-20T01:24:51.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.339+0000 7f26f0dee700 1 -- 172.21.15.16:0/249622757 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26d800bd90 con 0x7f26ec0b4750 2023-05-20T01:24:51.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.339+0000 7f26f1df0700 1 --2- 172.21.15.16:0/249622757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26d8003c30 0x7f26d8004020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:51.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.339+0000 7f26f15ef700 1 --2- 172.21.15.16:0/249622757 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f26ec0b4750 0x7f26d8004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:51.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.339+0000 7f26f25f1700 1 --2- 172.21.15.16:0/249622757 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f26d8004d90 0x7f26d8005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:51.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.340+0000 7f26f1df0700 1 -- 172.21.15.16:0/249622757 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f26ec0b4750 msgr2=0x7f26d8004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:24:51.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.340+0000 7f26f1df0700 1 --2- 172.21.15.16:0/249622757 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f26ec0b4750 0x7f26d8004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:51.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.340+0000 7f26f1df0700 1 -- 172.21.15.16:0/249622757 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f26d8004d90 msgr2=0x7f26d8005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:24:51.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.340+0000 7f26f1df0700 1 --2- 172.21.15.16:0/249622757 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f26d8004d90 0x7f26d8005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:51.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.340+0000 7f26f1df0700 1 -- 172.21.15.16:0/249622757 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26d80035b0 con 0x7f26d8003c30 2023-05-20T01:24:51.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.340+0000 7f26f1df0700 1 --2- 172.21.15.16:0/249622757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26d8003c30 0x7f26d8004020 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f26e8001eb0 tx=0x7f26d8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5f1904f74afd16c server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:24:51.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.340+0000 7f26f0dee700 1 -- 172.21.15.16:0/249622757 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26e800cc70 con 0x7f26d8003c30 2023-05-20T01:24:51.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.340+0000 7f26f0dee700 1 -- 172.21.15.16:0/249622757 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26e800cdf0 con 0x7f26d8003c30 2023-05-20T01:24:51.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.340+0000 7f26f0dee700 1 -- 172.21.15.16:0/249622757 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26e8005ba0 con 0x7f26d8003c30 2023-05-20T01:24:51.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.340+0000 7f26f4054700 1 -- 172.21.15.16:0/249622757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26d8003c30 msgr2=0x7f26d8004020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:51.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.340+0000 7f26f4054700 1 --2- 172.21.15.16:0/249622757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26d8003c30 0x7f26d8004020 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f26e8001eb0 tx=0x7f26d8005880 comp rx=0 tx=0).stop 2023-05-20T01:24:51.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.340+0000 7f26f25f1700 1 -- 172.21.15.16:0/249622757 reap_dead start 2023-05-20T01:24:51.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.341+0000 7f26f4054700 1 -- 172.21.15.16:0/249622757 shutdown_connections 2023-05-20T01:24:51.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.341+0000 7f26f4054700 1 -- 172.21.15.16:0/249622757 >> 172.21.15.16:0/249622757 conn(0x7f26ec09c590 msgr2=0x7f26ec0d3240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:24:51.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.341+0000 7f26f4054700 1 -- 172.21.15.16:0/249622757 shutdown_connections 2023-05-20T01:24:51.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.341+0000 7f26f4054700 1 -- 172.21.15.16:0/249622757 wait complete. 2023-05-20T01:24:51.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.342+0000 7f26f4054700 1 Processor -- start 2023-05-20T01:24:51.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.342+0000 7f26f4054700 1 -- start start 2023-05-20T01:24:51.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.343+0000 7f26f4054700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f26d8003c30 0x7f26ec1592f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:51.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.343+0000 7f26f4054700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26ec15a4b0 0x7f26ec0cbad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:51.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.343+0000 7f26f4054700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f26ec0cc010 0x7f26ec0b0f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:51.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.343+0000 7f26f4054700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26ec0c6d50 con 0x7f26ec15a4b0 2023-05-20T01:24:51.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.343+0000 7f26f4054700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26ec15a940 con 0x7f26d8003c30 2023-05-20T01:24:51.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.343+0000 7f26f4054700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26ec15ac20 con 0x7f26ec0cc010 2023-05-20T01:24:51.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.343+0000 7f26f15ef700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26ec15a4b0 0x7f26ec0cbad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:51.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.343+0000 7f26f15ef700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26ec15a4b0 0x7f26ec0cbad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35538/0 (socket says 172.21.15.16:35538) 2023-05-20T01:24:51.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.343+0000 7f26f15ef700 1 -- 172.21.15.16:0/2569604963 learned_addr learned my addr 172.21.15.16:0/2569604963 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:24:51.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.343+0000 7f26f1df0700 1 --2- 172.21.15.16:0/2569604963 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f26d8003c30 0x7f26ec1592f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:51.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.343+0000 7f26f25f1700 1 --2- 172.21.15.16:0/2569604963 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f26ec0cc010 0x7f26ec0b0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:51.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.343+0000 7f26f15ef700 1 -- 172.21.15.16:0/2569604963 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f26ec0cc010 msgr2=0x7f26ec0b0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:51.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.343+0000 7f26f15ef700 1 --2- 172.21.15.16:0/2569604963 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f26ec0cc010 0x7f26ec0b0f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:51.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.343+0000 7f26f15ef700 1 -- 172.21.15.16:0/2569604963 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f26d8003c30 msgr2=0x7f26ec1592f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:51.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.343+0000 7f26f15ef700 1 --2- 172.21.15.16:0/2569604963 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f26d8003c30 0x7f26ec1592f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:51.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.343+0000 7f26f15ef700 1 -- 172.21.15.16:0/2569604963 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26ec0d4000 con 0x7f26ec15a4b0 2023-05-20T01:24:51.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.343+0000 7f26f15ef700 1 --2- 172.21.15.16:0/2569604963 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26ec15a4b0 0x7f26ec0cbad0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f26e0011aa0 tx=0x7f26e0011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:24:51.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.343+0000 7f26de7fc700 1 -- 172.21.15.16:0/2569604963 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26e000f980 con 0x7f26ec15a4b0 2023-05-20T01:24:51.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.343+0000 7f26de7fc700 1 -- 172.21.15.16:0/2569604963 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26e000fae0 con 0x7f26ec15a4b0 2023-05-20T01:24:51.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.343+0000 7f26f4054700 1 -- 172.21.15.16:0/2569604963 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26ec0b4380 con 0x7f26ec15a4b0 2023-05-20T01:24:51.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.343+0000 7f26de7fc700 1 -- 172.21.15.16:0/2569604963 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26e000fe90 con 0x7f26ec15a4b0 2023-05-20T01:24:51.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.343+0000 7f26f4054700 1 -- 172.21.15.16:0/2569604963 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26ec0b3da0 con 0x7f26ec15a4b0 2023-05-20T01:24:51.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.344+0000 7f26f4054700 1 -- 172.21.15.16:0/2569604963 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26ec0747b0 con 0x7f26ec15a4b0 2023-05-20T01:24:51.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.345+0000 7f26de7fc700 1 -- 172.21.15.16:0/2569604963 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f26e001d400 con 0x7f26ec15a4b0 2023-05-20T01:24:51.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.345+0000 7f26de7fc700 1 --2- 172.21.15.16:0/2569604963 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f26d006ace0 0x7f26d006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:51.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.346+0000 7f26f1df0700 1 --2- 172.21.15.16:0/2569604963 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f26d006ace0 0x7f26d006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:51.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.346+0000 7f26de7fc700 1 -- 172.21.15.16:0/2569604963 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7189+0+0 (secure 0 0 0) 0x7f26e008a040 con 0x7f26ec15a4b0 2023-05-20T01:24:51.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.346+0000 7f26f1df0700 1 --2- 172.21.15.16:0/2569604963 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f26d006ace0 0x7f26d006d190 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f26e8000c00 tx=0x7f26e8007000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:24:51.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.350+0000 7f26de7fc700 1 -- 172.21.15.16:0/2569604963 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f26e0058860 con 0x7f26ec15a4b0 2023-05-20T01:24:51.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.551+0000 7f26f4054700 1 -- 172.21.15.16:0/2569604963 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f26ec0d03a0 con 0x7f26ec15a4b0 2023-05-20T01:24:51.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.553+0000 7f26de7fc700 1 -- 172.21.15.16:0/2569604963 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) v1 ==== 74+0+20822 (secure 0 0 0) 0x7f26e00583f0 con 0x7f26ec15a4b0 2023-05-20T01:24:51.558 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:24:51.558 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":75,"fsid":"c2245cc5-ecab-4087-a59b-bc81db901676","created":"2023-05-20T01:16:09.549230+0000","modified":"2023-05-20T01:22:20.570959+0000","last_up_change":"2023-05-20T01:16:19.788624+0000","last_in_change":"2023-05-20T01:16:12.092160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-05-20T01:16:20.934366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-05-20T01:16:21.372294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-05-20T01:19:49.878849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-05-20T01:19:50.887306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data_extra","create_time":"2023-05-20T01:20:04.581773+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"317a83cd-19c8-4958-a301-6087ef342289","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6825","nonce":3923634179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6827","nonce":3923634179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6831","nonce":3923634179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6829","nonce":3923634179}]},"public_addr":"172.21.15.16:6825/3923634179","cluster_addr":"172.21.15.16:6827/3923634179","heartbeat_back_addr":"172.21.15.16:6831/3923634179","heartbeat_front_addr":"172.21.15.16:6829/3923634179","state":["exists","up"]},{"osd":1,"uuid":"5581046f-8219-4089-8d7d-b7846399505e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6809","nonce":3860264220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6811","nonce":3860264220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6815","nonce":3860264220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6813","nonce":3860264220}]},"public_addr":"172.21.15.16:6809/3860264220","cluster_addr":"172.21.15.16:6811/3860264220","heartbeat_back_addr":"172.21.15.16:6815/3860264220","heartbeat_front_addr":"172.21.15.16:6813/3860264220","state":["exists","up"]},{"osd":2,"uuid":"072e9e66-4133-478e-8d4a-64d46c2963f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6817","nonce":293187030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6819","nonce":293187030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6823","nonce":293187030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6821","nonce":293187030}]},"public_addr":"172.21.15.16:6817/293187030","cluster_addr":"172.21.15.16:6819/293187030","heartbeat_back_addr":"172.21.15.16:6823/293187030","heartbeat_front_addr":"172.21.15.16:6821/293187030","state":["exists","up"]},{"osd":3,"uuid":"11e1bba6-00f8-48dd-9028-d8d716b7f145","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6801","nonce":3669781890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6803","nonce":3669781890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6807","nonce":3669781890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6805","nonce":3669781890}]},"public_addr":"172.21.15.16:6801/3669781890","cluster_addr":"172.21.15.16:6803/3669781890","heartbeat_back_addr":"172.21.15.16:6807/3669781890","heartbeat_front_addr":"172.21.15.16:6805/3669781890","state":["exists","up"]},{"osd":4,"uuid":"b8909c3a-cf70-46cb-8d0b-009c707cf9d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6819","nonce":506317223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6823","nonce":506317223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6829","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6831","nonce":506317223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6827","nonce":506317223}]},"public_addr":"172.21.15.132:6819/506317223","cluster_addr":"172.21.15.132:6823/506317223","heartbeat_back_addr":"172.21.15.132:6831/506317223","heartbeat_front_addr":"172.21.15.132:6827/506317223","state":["exists","up"]},{"osd":5,"uuid":"b1818304-4583-47bf-a6ec-0bfbb5166233","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6818","nonce":1844689995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6822","nonce":1844689995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6830","nonce":1844689995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6826","nonce":1844689995}]},"public_addr":"172.21.15.132:6818/1844689995","cluster_addr":"172.21.15.132:6822/1844689995","heartbeat_back_addr":"172.21.15.132:6830/1844689995","heartbeat_front_addr":"172.21.15.132:6826/1844689995","state":["exists","up"]},{"osd":6,"uuid":"d37f4691-eed1-4a93-8818-18354f03f4a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6801","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6802","nonce":727581123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6805","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6807","nonce":727581123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6813","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6815","nonce":727581123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6811","nonce":727581123}]},"public_addr":"172.21.15.132:6802/727581123","cluster_addr":"172.21.15.132:6807/727581123","heartbeat_back_addr":"172.21.15.132:6815/727581123","heartbeat_front_addr":"172.21.15.132:6811/727581123","state":["exists","up"]},{"osd":7,"uuid":"4f712f62-5d1d-4816-8042-03209144f8b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6803","nonce":466001149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6806","nonce":466001149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6814","nonce":466001149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6810","nonce":466001149}]},"public_addr":"172.21.15.132:6803/466001149","cluster_addr":"172.21.15.132:6806/466001149","heartbeat_back_addr":"172.21.15.132:6814/466001149","heartbeat_front_addr":"172.21.15.132:6810/466001149","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.338863+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.278680+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305694+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.246835+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.276237+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.304441+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305421+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.267050+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.d","mappings":[{"from":7,"to":2}]},{"pgid":"10.10","mappings":[{"from":4,"to":3}]},{"pgid":"10.2e","mappings":[{"from":4,"to":3}]},{"pgid":"10.34","mappings":[{"from":7,"to":3}]},{"pgid":"10.37","mappings":[{"from":4,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6837/3291457857":"2023-05-21T01:22:20.558472+0000","172.21.15.16:6835/3291457857":"2023-05-21T01:22:20.558472+0000","172.21.15.16:6837/1427887654":"2023-05-21T01:22:14.372197+0000","172.21.15.16:6835/1427887654":"2023-05-21T01:22:14.372197+0000","172.21.15.16:6837/1427908556":"2023-05-21T01:22:12.737196+0000","172.21.15.16:6835/1427908556":"2023-05-21T01:22:12.737196+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-20T01:24:51.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.555+0000 7f26f4054700 1 -- 172.21.15.16:0/2569604963 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f26d006ace0 msgr2=0x7f26d006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:51.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.555+0000 7f26f4054700 1 --2- 172.21.15.16:0/2569604963 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f26d006ace0 0x7f26d006d190 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f26e8000c00 tx=0x7f26e8007000 comp rx=0 tx=0).stop 2023-05-20T01:24:51.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.555+0000 7f26f4054700 1 -- 172.21.15.16:0/2569604963 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26ec15a4b0 msgr2=0x7f26ec0cbad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:51.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.555+0000 7f26f4054700 1 --2- 172.21.15.16:0/2569604963 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26ec15a4b0 0x7f26ec0cbad0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f26e0011aa0 tx=0x7f26e0011e60 comp rx=0 tx=0).stop 2023-05-20T01:24:51.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.555+0000 7f26f4054700 1 -- 172.21.15.16:0/2569604963 shutdown_connections 2023-05-20T01:24:51.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.555+0000 7f26f4054700 1 --2- 172.21.15.16:0/2569604963 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f26d006ace0 0x7f26d006d190 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:51.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.555+0000 7f26f4054700 1 --2- 172.21.15.16:0/2569604963 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f26ec0cc010 0x7f26ec0b0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:51.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.555+0000 7f26f4054700 1 --2- 172.21.15.16:0/2569604963 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f26d8003c30 0x7f26ec1592f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:51.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.555+0000 7f26f4054700 1 --2- 172.21.15.16:0/2569604963 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26ec15a4b0 0x7f26ec0cbad0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:51.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.555+0000 7f26f4054700 1 -- 172.21.15.16:0/2569604963 >> 172.21.15.16:0/2569604963 conn(0x7f26ec09c590 msgr2=0x7f26ec0cebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:24:51.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.555+0000 7f26f4054700 1 -- 172.21.15.16:0/2569604963 shutdown_connections 2023-05-20T01:24:51.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.555+0000 7f26f4054700 1 -- 172.21.15.16:0/2569604963 wait complete. 2023-05-20T01:24:51.572 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-05-20T01:16:20.934366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-05-20T01:16:21.372294+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2023-05-20T01:19:49.878849+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '63', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2023-05-20T01:19:50.887306+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '64', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data_extra', 'create_time': '2023-05-20T01:20:04.581773+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '67', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-05-20T01:24:51.573 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-05-20T01:24:51.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.697+0000 7f66f55ab700 1 Processor -- start 2023-05-20T01:24:51.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.697+0000 7f66f55ab700 1 -- start start 2023-05-20T01:24:51.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.698+0000 7f66f55ab700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f66f015ca90 0x7f66f00cd890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:51.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.698+0000 7f66f55ab700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f66f00d2550 0x7f66f00b6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:51.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.698+0000 7f66f55ab700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f66f00d29a0 0x7f66f00b65d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:51.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.698+0000 7f66f55ab700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66f00d0c50 con 0x7f66f015ca90 2023-05-20T01:24:51.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.698+0000 7f66f55ab700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66f00b6b10 con 0x7f66f00d29a0 2023-05-20T01:24:51.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.698+0000 7f66f55ab700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66f00d1820 con 0x7f66f00d2550 2023-05-20T01:24:51.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.698+0000 7f66eeffd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f66f015ca90 0x7f66f00cd890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:51.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.698+0000 7f66eeffd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f66f015ca90 0x7f66f00cd890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35544/0 (socket says 172.21.15.16:35544) 2023-05-20T01:24:51.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.698+0000 7f66eeffd700 1 -- 172.21.15.16:0/2903439282 learned_addr learned my addr 172.21.15.16:0/2903439282 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:24:51.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.698+0000 7f66eeffd700 1 -- 172.21.15.16:0/2903439282 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f66f00d2550 msgr2=0x7f66f00b6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:24:51.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.698+0000 7f66ee7fc700 1 --2- 172.21.15.16:0/2903439282 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f66f00d2550 0x7f66f00b6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:51.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.698+0000 7f66ef7fe700 1 --2- 172.21.15.16:0/2903439282 >> v2:172.21.15.132:3300/0 conn(0x7f66f00d29a0 0x7f66f00b65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:51.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.698+0000 7f66eeffd700 1 --2- 172.21.15.16:0/2903439282 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f66f00d2550 0x7f66f00b6090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:51.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.698+0000 7f66eeffd700 1 -- 172.21.15.16:0/2903439282 >> v2:172.21.15.132:3300/0 conn(0x7f66f00d29a0 msgr2=0x7f66f00b65d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:24:51.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.698+0000 7f66eeffd700 1 --2- 172.21.15.16:0/2903439282 >> v2:172.21.15.132:3300/0 conn(0x7f66f00d29a0 0x7f66f00b65d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:51.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.698+0000 7f66eeffd700 1 -- 172.21.15.16:0/2903439282 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66f00cdf00 con 0x7f66f015ca90 2023-05-20T01:24:51.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.699+0000 7f66eeffd700 1 --2- 172.21.15.16:0/2903439282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66f015ca90 0x7f66f00cd890 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f66f00ca2a0 tx=0x7f66e4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3dc32fc2bc07686 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:24:51.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.699+0000 7f66edffb700 1 -- 172.21.15.16:0/2903439282 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e400b870 con 0x7f66f015ca90 2023-05-20T01:24:51.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.699+0000 7f66edffb700 1 -- 172.21.15.16:0/2903439282 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66e400b9d0 con 0x7f66f015ca90 2023-05-20T01:24:51.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.699+0000 7f66edffb700 1 -- 172.21.15.16:0/2903439282 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e400bd60 con 0x7f66f015ca90 2023-05-20T01:24:51.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.699+0000 7f66f55ab700 1 -- 172.21.15.16:0/2903439282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66f015ca90 msgr2=0x7f66f00cd890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:51.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.699+0000 7f66f55ab700 1 --2- 172.21.15.16:0/2903439282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66f015ca90 0x7f66f00cd890 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f66f00ca2a0 tx=0x7f66e4009bd0 comp rx=0 tx=0).stop 2023-05-20T01:24:51.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.699+0000 7f66f55ab700 1 -- 172.21.15.16:0/2903439282 shutdown_connections 2023-05-20T01:24:51.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.699+0000 7f66f55ab700 1 --2- 172.21.15.16:0/2903439282 >> v2:172.21.15.132:3300/0 conn(0x7f66f00d29a0 0x7f66f00b65d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:51.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.699+0000 7f66f55ab700 1 --2- 172.21.15.16:0/2903439282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66f015ca90 0x7f66f00cd890 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:51.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.699+0000 7f66f55ab700 1 --2- 172.21.15.16:0/2903439282 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f66f00d2550 0x7f66f00b6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:51.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.699+0000 7f66f55ab700 1 -- 172.21.15.16:0/2903439282 >> 172.21.15.16:0/2903439282 conn(0x7f66f006b000 msgr2=0x7f66f006b400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:24:51.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.699+0000 7f66f55ab700 1 -- 172.21.15.16:0/2903439282 shutdown_connections 2023-05-20T01:24:51.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.699+0000 7f66f55ab700 1 -- 172.21.15.16:0/2903439282 wait complete. 2023-05-20T01:24:51.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.701+0000 7f66f55ab700 1 Processor -- start 2023-05-20T01:24:51.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.701+0000 7f66f55ab700 1 -- start start 2023-05-20T01:24:51.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.702+0000 7f66f55ab700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f66f00d2550 0x7f66f01f1c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:51.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.702+0000 7f66f55ab700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f66f00d29a0 0x7f66f00b3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:51.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.702+0000 7f66f55ab700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66f015ca90 0x7f66f00c8900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:51.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.702+0000 7f66f55ab700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66f00b43c0 con 0x7f66f015ca90 2023-05-20T01:24:51.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.702+0000 7f66f55ab700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66f015cf70 con 0x7f66f00d29a0 2023-05-20T01:24:51.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.702+0000 7f66f55ab700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66f00b40e0 con 0x7f66f00d2550 2023-05-20T01:24:51.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.702+0000 7f66ef7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66f015ca90 0x7f66f00c8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:51.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.702+0000 7f66ef7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66f015ca90 0x7f66f00c8900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35554/0 (socket says 172.21.15.16:35554) 2023-05-20T01:24:51.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.702+0000 7f66ef7fe700 1 -- 172.21.15.16:0/1750727679 learned_addr learned my addr 172.21.15.16:0/1750727679 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:24:51.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.702+0000 7f66eeffd700 1 --2- 172.21.15.16:0/1750727679 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f66f00d2550 0x7f66f01f1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:51.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.702+0000 7f66ee7fc700 1 --2- 172.21.15.16:0/1750727679 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f66f00d29a0 0x7f66f00b3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:51.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.703+0000 7f66ef7fe700 1 -- 172.21.15.16:0/1750727679 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f66f00d2550 msgr2=0x7f66f01f1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:51.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.703+0000 7f66ef7fe700 1 --2- 172.21.15.16:0/1750727679 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f66f00d2550 0x7f66f01f1c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:51.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.703+0000 7f66ef7fe700 1 -- 172.21.15.16:0/1750727679 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f66f00d29a0 msgr2=0x7f66f00b3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:51.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.703+0000 7f66ef7fe700 1 --2- 172.21.15.16:0/1750727679 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f66f00d29a0 0x7f66f00b3690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:51.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.703+0000 7f66ef7fe700 1 -- 172.21.15.16:0/1750727679 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66e4009550 con 0x7f66f015ca90 2023-05-20T01:24:51.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.703+0000 7f66ef7fe700 1 --2- 172.21.15.16:0/1750727679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66f015ca90 0x7f66f00c8900 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f66e800cd40 tx=0x7f66e800a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:24:51.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.703+0000 7f66d77fe700 1 -- 172.21.15.16:0/1750727679 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e800ce90 con 0x7f66f015ca90 2023-05-20T01:24:51.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.703+0000 7f66d77fe700 1 -- 172.21.15.16:0/1750727679 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66e8003a20 con 0x7f66f015ca90 2023-05-20T01:24:51.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.703+0000 7f66d77fe700 1 -- 172.21.15.16:0/1750727679 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e8003db0 con 0x7f66f015ca90 2023-05-20T01:24:51.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.703+0000 7f66f55ab700 1 -- 172.21.15.16:0/1750727679 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66f00d1d50 con 0x7f66f015ca90 2023-05-20T01:24:51.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.703+0000 7f66f55ab700 1 -- 172.21.15.16:0/1750727679 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66f00d1a70 con 0x7f66f015ca90 2023-05-20T01:24:51.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.704+0000 7f66f55ab700 1 -- 172.21.15.16:0/1750727679 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66f00d2030 con 0x7f66f015ca90 2023-05-20T01:24:51.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.704+0000 7f66d77fe700 1 -- 172.21.15.16:0/1750727679 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f66e8003f10 con 0x7f66f015ca90 2023-05-20T01:24:51.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.705+0000 7f66d77fe700 1 --2- 172.21.15.16:0/1750727679 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f66dc06ace0 0x7f66dc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:51.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.705+0000 7f66d77fe700 1 -- 172.21.15.16:0/1750727679 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7189+0+0 (secure 0 0 0) 0x7f66e8012070 con 0x7f66f015ca90 2023-05-20T01:24:51.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.705+0000 7f66eeffd700 1 --2- 172.21.15.16:0/1750727679 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f66dc06ace0 0x7f66dc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:51.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.705+0000 7f66eeffd700 1 --2- 172.21.15.16:0/1750727679 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f66dc06ace0 0x7f66dc06d190 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f66e4014040 tx=0x7f66e4008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:24:51.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.709+0000 7f66d77fe700 1 -- 172.21.15.16:0/1750727679 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f66e8054bc0 con 0x7f66f015ca90 2023-05-20T01:24:51.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.904+0000 7f66f55ab700 1 -- 172.21.15.16:0/1750727679 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f66f00cb4d0 con 0x7f66f015ca90 2023-05-20T01:24:51.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.904+0000 7f66d77fe700 1 -- 172.21.15.16:0/1750727679 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v75) v1 ==== 92+0+10 (secure 0 0 0) 0x7f66f00cb4d0 con 0x7f66f015ca90 2023-05-20T01:24:51.909 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 8 2023-05-20T01:24:51.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.906+0000 7f66f55ab700 1 -- 172.21.15.16:0/1750727679 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f66dc06ace0 msgr2=0x7f66dc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:51.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.906+0000 7f66f55ab700 1 --2- 172.21.15.16:0/1750727679 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f66dc06ace0 0x7f66dc06d190 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f66e4014040 tx=0x7f66e4008030 comp rx=0 tx=0).stop 2023-05-20T01:24:51.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.906+0000 7f66f55ab700 1 -- 172.21.15.16:0/1750727679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66f015ca90 msgr2=0x7f66f00c8900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:51.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.906+0000 7f66f55ab700 1 --2- 172.21.15.16:0/1750727679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66f015ca90 0x7f66f00c8900 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f66e800cd40 tx=0x7f66e800a570 comp rx=0 tx=0).stop 2023-05-20T01:24:51.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.906+0000 7f66f55ab700 1 -- 172.21.15.16:0/1750727679 shutdown_connections 2023-05-20T01:24:51.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.906+0000 7f66f55ab700 1 --2- 172.21.15.16:0/1750727679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66f015ca90 0x7f66f00c8900 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:51.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.906+0000 7f66f55ab700 1 --2- 172.21.15.16:0/1750727679 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f66dc06ace0 0x7f66dc06d190 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:51.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.906+0000 7f66f55ab700 1 --2- 172.21.15.16:0/1750727679 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f66f00d2550 0x7f66f01f1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:51.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.906+0000 7f66f55ab700 1 --2- 172.21.15.16:0/1750727679 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f66f00d29a0 0x7f66f00b3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:51.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.906+0000 7f66f55ab700 1 -- 172.21.15.16:0/1750727679 >> 172.21.15.16:0/1750727679 conn(0x7f66f006b000 msgr2=0x7f66f00cf430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:24:51.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.906+0000 7f66f55ab700 1 -- 172.21.15.16:0/1750727679 shutdown_connections 2023-05-20T01:24:51.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:51.906+0000 7f66f55ab700 1 -- 172.21.15.16:0/1750727679 wait complete. 2023-05-20T01:24:51.922 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-05-20T01:24:52.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.046+0000 7f8d43a6d700 1 Processor -- start 2023-05-20T01:24:52.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.046+0000 7f8d43a6d700 1 -- start start 2023-05-20T01:24:52.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.047+0000 7f8d43a6d700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f8d3c159b50 0x7f8d3c06e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:52.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.047+0000 7f8d43a6d700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8d3c0717c0 0x7f8d3c0ca870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:52.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.047+0000 7f8d43a6d700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d3c0ccd30 con 0x7f8d3c0cd1c0 2023-05-20T01:24:52.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.047+0000 7f8d43a6d700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d3c0ce990 con 0x7f8d3c159b50 2023-05-20T01:24:52.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.047+0000 7f8d43a6d700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d3c0cc6e0 con 0x7f8d3c0717c0 2023-05-20T01:24:52.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.047+0000 7f8d41008700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f8d3c0cd1c0 0x7f8d3c0710b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:47086/0 (socket says 172.21.15.16:47086) 2023-05-20T01:24:52.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.047+0000 7f8d41008700 1 -- 172.21.15.16:0/3555055123 learned_addr learned my addr 172.21.15.16:0/3555055123 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:24:52.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.047+0000 7f8d41809700 1 --2- 172.21.15.16:0/3555055123 >> v2:172.21.15.132:3300/0 conn(0x7f8d3c159b50 0x7f8d3c06e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:52.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.047+0000 7f8d4200a700 1 --2- 172.21.15.16:0/3555055123 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8d3c0717c0 0x7f8d3c0ca870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:52.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.047+0000 7f8d40807700 1 -- 172.21.15.16:0/3555055123 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3445774750 0 0) 0x7f8d3c0ccd30 con 0x7f8d3c0cd1c0 2023-05-20T01:24:52.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.047+0000 7f8d40807700 1 -- 172.21.15.16:0/3555055123 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d280035b0 con 0x7f8d3c0cd1c0 2023-05-20T01:24:52.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.048+0000 7f8d40807700 1 -- 172.21.15.16:0/3555055123 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3635720852 0 0) 0x7f8d280035b0 con 0x7f8d3c0cd1c0 2023-05-20T01:24:52.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.048+0000 7f8d40807700 1 -- 172.21.15.16:0/3555055123 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8d3c0717c0 msgr2=0x7f8d3c0ca870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:24:52.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.048+0000 7f8d40807700 1 --2- 172.21.15.16:0/3555055123 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8d3c0717c0 0x7f8d3c0ca870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:52.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.048+0000 7f8d40807700 1 -- 172.21.15.16:0/3555055123 >> v2:172.21.15.132:3300/0 conn(0x7f8d3c159b50 msgr2=0x7f8d3c06e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:24:52.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.048+0000 7f8d40807700 1 --2- 172.21.15.16:0/3555055123 >> v2:172.21.15.132:3300/0 conn(0x7f8d3c159b50 0x7f8d3c06e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:52.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.048+0000 7f8d40807700 1 -- 172.21.15.16:0/3555055123 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d3c0cc9c0 con 0x7f8d3c0cd1c0 2023-05-20T01:24:52.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.048+0000 7f8d40807700 1 -- 172.21.15.16:0/3555055123 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7f8d38001670 con 0x7f8d3c0cd1c0 2023-05-20T01:24:52.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.048+0000 7f8d41809700 1 --2- 172.21.15.16:0/3555055123 >> v2:172.21.15.132:3300/0 conn(0x7f8d3c159b50 0x7f8d3c06e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-05-20T01:24:52.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.048+0000 7f8d40807700 1 -- 172.21.15.16:0/3555055123 >> v1:172.21.15.16:6789/0 conn(0x7f8d3c0cd1c0 legacy=0x7f8d3c0710b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:52.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.048+0000 7f8d40807700 1 --2- 172.21.15.16:0/3555055123 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8d3c0717c0 0x7f8d28003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:52.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.048+0000 7f8d40807700 1 --2- 172.21.15.16:0/3555055123 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8d28003f80 0x7f8d28004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:52.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.048+0000 7f8d40807700 1 --2- 172.21.15.16:0/3555055123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d28004d90 0x7f8d28005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:52.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.048+0000 7f8d40807700 1 -- 172.21.15.16:0/3555055123 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d3c0ce990 con 0x7f8d28004d90 2023-05-20T01:24:52.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.048+0000 7f8d40807700 1 -- 172.21.15.16:0/3555055123 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d3c0cc6e0 con 0x7f8d28003f80 2023-05-20T01:24:52.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.048+0000 7f8d40807700 1 -- 172.21.15.16:0/3555055123 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2800dc20 con 0x7f8d3c0717c0 2023-05-20T01:24:52.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.048+0000 7f8d4200a700 1 --2- 172.21.15.16:0/3555055123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d28004d90 0x7f8d28005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:52.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.048+0000 7f8d41809700 1 --2- 172.21.15.16:0/3555055123 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8d3c0717c0 0x7f8d28003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:52.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.048+0000 7f8d41008700 1 --2- 172.21.15.16:0/3555055123 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8d28003f80 0x7f8d28004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:52.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.048+0000 7f8d4200a700 1 -- 172.21.15.16:0/3555055123 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8d3c0717c0 msgr2=0x7f8d28003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:24:52.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.048+0000 7f8d4200a700 1 --2- 172.21.15.16:0/3555055123 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8d3c0717c0 0x7f8d28003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:52.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.048+0000 7f8d4200a700 1 -- 172.21.15.16:0/3555055123 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8d28003f80 msgr2=0x7f8d28004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:24:52.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.048+0000 7f8d4200a700 1 --2- 172.21.15.16:0/3555055123 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8d28003f80 0x7f8d28004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:52.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.048+0000 7f8d4200a700 1 -- 172.21.15.16:0/3555055123 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d280035b0 con 0x7f8d28004d90 2023-05-20T01:24:52.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.048+0000 7f8d4200a700 1 --2- 172.21.15.16:0/3555055123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d28004d90 0x7f8d28005200 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f8d34009f90 tx=0x7f8d28005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5d9713c15cf57f server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:24:52.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.049+0000 7f8d40807700 1 -- 172.21.15.16:0/3555055123 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d3400ec30 con 0x7f8d28004d90 2023-05-20T01:24:52.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.049+0000 7f8d40807700 1 -- 172.21.15.16:0/3555055123 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d3400f040 con 0x7f8d28004d90 2023-05-20T01:24:52.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.049+0000 7f8d40807700 1 -- 172.21.15.16:0/3555055123 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d34014480 con 0x7f8d28004d90 2023-05-20T01:24:52.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.049+0000 7f8d43a6d700 1 -- 172.21.15.16:0/3555055123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d28004d90 msgr2=0x7f8d28005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:52.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.049+0000 7f8d43a6d700 1 --2- 172.21.15.16:0/3555055123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d28004d90 0x7f8d28005200 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f8d34009f90 tx=0x7f8d28005880 comp rx=0 tx=0).stop 2023-05-20T01:24:52.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.049+0000 7f8d4200a700 1 -- 172.21.15.16:0/3555055123 reap_dead start 2023-05-20T01:24:52.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.049+0000 7f8d43a6d700 1 -- 172.21.15.16:0/3555055123 shutdown_connections 2023-05-20T01:24:52.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.049+0000 7f8d43a6d700 1 -- 172.21.15.16:0/3555055123 >> 172.21.15.16:0/3555055123 conn(0x7f8d3c0cffd0 msgr2=0x7f8d3c0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:24:52.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.049+0000 7f8d43a6d700 1 -- 172.21.15.16:0/3555055123 shutdown_connections 2023-05-20T01:24:52.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.049+0000 7f8d43a6d700 1 -- 172.21.15.16:0/3555055123 wait complete. 2023-05-20T01:24:52.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.050+0000 7f8d43a6d700 1 Processor -- start 2023-05-20T01:24:52.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.050+0000 7f8d43a6d700 1 -- start start 2023-05-20T01:24:52.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.051+0000 7f8d43a6d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8d3c151800 0x7f8d3c0c51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:52.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.051+0000 7f8d43a6d700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8d3c0c56f0 0x7f8d3c0c8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:52.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.051+0000 7f8d43a6d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d3c0c91e0 0x7f8d3c0c3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:52.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.051+0000 7f8d43a6d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d3c0b3610 con 0x7f8d3c0c91e0 2023-05-20T01:24:52.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.051+0000 7f8d43a6d700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d3c0b38f0 con 0x7f8d3c151800 2023-05-20T01:24:52.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.051+0000 7f8d43a6d700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d3c0b3bd0 con 0x7f8d3c0c56f0 2023-05-20T01:24:52.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.051+0000 7f8d4200a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d3c0c91e0 0x7f8d3c0c3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:52.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.051+0000 7f8d4200a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d3c0c91e0 0x7f8d3c0c3fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35572/0 (socket says 172.21.15.16:35572) 2023-05-20T01:24:52.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.051+0000 7f8d4200a700 1 -- 172.21.15.16:0/1864552339 learned_addr learned my addr 172.21.15.16:0/1864552339 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:24:52.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.051+0000 7f8d41008700 1 --2- 172.21.15.16:0/1864552339 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8d3c0c56f0 0x7f8d3c0c8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:52.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.051+0000 7f8d41809700 1 --2- 172.21.15.16:0/1864552339 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8d3c151800 0x7f8d3c0c51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:52.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.052+0000 7f8d4200a700 1 -- 172.21.15.16:0/1864552339 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8d3c0c56f0 msgr2=0x7f8d3c0c8ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:52.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.052+0000 7f8d4200a700 1 --2- 172.21.15.16:0/1864552339 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8d3c0c56f0 0x7f8d3c0c8ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:52.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.052+0000 7f8d4200a700 1 -- 172.21.15.16:0/1864552339 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8d3c151800 msgr2=0x7f8d3c0c51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:52.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.052+0000 7f8d4200a700 1 --2- 172.21.15.16:0/1864552339 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8d3c151800 0x7f8d3c0c51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:52.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.052+0000 7f8d4200a700 1 -- 172.21.15.16:0/1864552339 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d3c071e80 con 0x7f8d3c0c91e0 2023-05-20T01:24:52.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.052+0000 7f8d4200a700 1 --2- 172.21.15.16:0/1864552339 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d3c0c91e0 0x7f8d3c0c3fe0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f8d280046d0 tx=0x7f8d34009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:24:52.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.052+0000 7f8d327fc700 1 -- 172.21.15.16:0/1864552339 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d34007000 con 0x7f8d3c0c91e0 2023-05-20T01:24:52.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.052+0000 7f8d327fc700 1 -- 172.21.15.16:0/1864552339 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d34003e30 con 0x7f8d3c0c91e0 2023-05-20T01:24:52.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.052+0000 7f8d43a6d700 1 -- 172.21.15.16:0/1864552339 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d3c06e5e0 con 0x7f8d3c0c91e0 2023-05-20T01:24:52.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.052+0000 7f8d327fc700 1 -- 172.21.15.16:0/1864552339 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d34015a00 con 0x7f8d3c0c91e0 2023-05-20T01:24:52.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.052+0000 7f8d43a6d700 1 -- 172.21.15.16:0/1864552339 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d3c0cec70 con 0x7f8d3c0c91e0 2023-05-20T01:24:52.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.053+0000 7f8d43a6d700 1 -- 172.21.15.16:0/1864552339 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d3c0b4500 con 0x7f8d3c0c91e0 2023-05-20T01:24:52.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.054+0000 7f8d327fc700 1 -- 172.21.15.16:0/1864552339 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8d34024070 con 0x7f8d3c0c91e0 2023-05-20T01:24:52.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.054+0000 7f8d327fc700 1 --2- 172.21.15.16:0/1864552339 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f8d2006ace0 0x7f8d2006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:52.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.054+0000 7f8d327fc700 1 -- 172.21.15.16:0/1864552339 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7189+0+0 (secure 0 0 0) 0x7f8d3408a8d0 con 0x7f8d3c0c91e0 2023-05-20T01:24:52.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.054+0000 7f8d41809700 1 --2- 172.21.15.16:0/1864552339 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f8d2006ace0 0x7f8d2006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:52.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.055+0000 7f8d41809700 1 --2- 172.21.15.16:0/1864552339 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f8d2006ace0 0x7f8d2006d190 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c0afa40 tx=0x7f8d2c0091c0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:24:52.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.059+0000 7f8d327fc700 1 -- 172.21.15.16:0/1864552339 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f8d340555a0 con 0x7f8d3c0c91e0 2023-05-20T01:24:52.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.253+0000 7f8d43a6d700 1 -- 172.21.15.16:0/1864552339 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8d3c0c5ac0 con 0x7f8d3c0c91e0 2023-05-20T01:24:52.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.254+0000 7f8d327fc700 1 -- 172.21.15.16:0/1864552339 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v75) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8d34058bc0 con 0x7f8d3c0c91e0 2023-05-20T01:24:52.258 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-05-20T01:24:52.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.255+0000 7f8d43a6d700 1 -- 172.21.15.16:0/1864552339 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f8d2006ace0 msgr2=0x7f8d2006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:52.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.255+0000 7f8d43a6d700 1 --2- 172.21.15.16:0/1864552339 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f8d2006ace0 0x7f8d2006d190 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c0afa40 tx=0x7f8d2c0091c0 comp rx=0 tx=0).stop 2023-05-20T01:24:52.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.255+0000 7f8d43a6d700 1 -- 172.21.15.16:0/1864552339 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d3c0c91e0 msgr2=0x7f8d3c0c3fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:52.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.255+0000 7f8d43a6d700 1 --2- 172.21.15.16:0/1864552339 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d3c0c91e0 0x7f8d3c0c3fe0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f8d280046d0 tx=0x7f8d34009750 comp rx=0 tx=0).stop 2023-05-20T01:24:52.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.256+0000 7f8d43a6d700 1 -- 172.21.15.16:0/1864552339 shutdown_connections 2023-05-20T01:24:52.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.256+0000 7f8d43a6d700 1 --2- 172.21.15.16:0/1864552339 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d3c0c91e0 0x7f8d3c0c3fe0 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:52.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.256+0000 7f8d43a6d700 1 --2- 172.21.15.16:0/1864552339 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8d3c151800 0x7f8d3c0c51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:52.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.256+0000 7f8d43a6d700 1 --2- 172.21.15.16:0/1864552339 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f8d2006ace0 0x7f8d2006d190 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:52.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.256+0000 7f8d43a6d700 1 --2- 172.21.15.16:0/1864552339 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8d3c0c56f0 0x7f8d3c0c8ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:52.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.256+0000 7f8d43a6d700 1 -- 172.21.15.16:0/1864552339 >> 172.21.15.16:0/1864552339 conn(0x7f8d3c0cffd0 msgr2=0x7f8d3c06dde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:24:52.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.256+0000 7f8d43a6d700 1 -- 172.21.15.16:0/1864552339 shutdown_connections 2023-05-20T01:24:52.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.256+0000 7f8d43a6d700 1 -- 172.21.15.16:0/1864552339 wait complete. 2023-05-20T01:24:52.272 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-05-20T01:24:52.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.395+0000 7f13c801c700 1 Processor -- start 2023-05-20T01:24:52.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.396+0000 7f13c801c700 1 -- start start 2023-05-20T01:24:52.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.396+0000 7f13c801c700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f13c00cd1c0 0x7f13c00710b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:52.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.396+0000 7f13c801c700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13c00ccd30 con 0x7f13c00cd630 2023-05-20T01:24:52.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.396+0000 7f13c801c700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13c00ce990 con 0x7f13c0159b50 2023-05-20T01:24:52.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.396+0000 7f13c801c700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13c00cc6e0 con 0x7f13c00cd1c0 2023-05-20T01:24:52.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.397+0000 7f13c65b9700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f13c00cd630 0x7f13c00ca870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:47108/0 (socket says 172.21.15.16:47108) 2023-05-20T01:24:52.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.397+0000 7f13c65b9700 1 -- 172.21.15.16:0/741263876 learned_addr learned my addr 172.21.15.16:0/741263876 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:24:52.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.397+0000 7f13c4db6700 1 -- 172.21.15.16:0/741263876 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 484273935 0 0) 0x7f13c00ccd30 con 0x7f13c00cd630 2023-05-20T01:24:52.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.397+0000 7f13c55b7700 1 --2- 172.21.15.16:0/741263876 >> v2:172.21.15.132:3300/0 conn(0x7f13c00cd1c0 0x7f13c00710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:52.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.397+0000 7f13c4db6700 1 -- 172.21.15.16:0/741263876 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13ac0057c0 con 0x7f13c00cd630 2023-05-20T01:24:52.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.397+0000 7f13c4db6700 1 -- 172.21.15.16:0/741263876 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1351906937 0 0) 0x7f13ac0057c0 con 0x7f13c00cd630 2023-05-20T01:24:52.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.397+0000 7f13c4db6700 1 -- 172.21.15.16:0/741263876 >> v2:172.21.15.132:3300/0 conn(0x7f13c00cd1c0 msgr2=0x7f13c00710b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:24:52.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.397+0000 7f13c4db6700 1 --2- 172.21.15.16:0/741263876 >> v2:172.21.15.132:3300/0 conn(0x7f13c00cd1c0 0x7f13c00710b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:52.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.397+0000 7f13c4db6700 1 -- 172.21.15.16:0/741263876 >> v1:172.21.15.132:6789/0 conn(0x7f13c0159b50 legacy=0x7f13c006e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:52.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.397+0000 7f13c4db6700 1 -- 172.21.15.16:0/741263876 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13c00cc9c0 con 0x7f13c00cd630 2023-05-20T01:24:52.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.397+0000 7f13c4db6700 1 -- 172.21.15.16:0/741263876 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7f13b80019f0 con 0x7f13c00cd630 2023-05-20T01:24:52.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.397+0000 7f13c4db6700 1 -- 172.21.15.16:0/741263876 >> v1:172.21.15.16:6789/0 conn(0x7f13c00cd630 legacy=0x7f13c00ca870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:52.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.397+0000 7f13c4db6700 1 --2- 172.21.15.16:0/741263876 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f13ac005c10 0x7f13ac006020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:52.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.397+0000 7f13c4db6700 1 --2- 172.21.15.16:0/741263876 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13ac006a60 0x7f13ac006ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:52.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.397+0000 7f13c4db6700 1 --2- 172.21.15.16:0/741263876 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f13ac007410 0x7f13ac00dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:52.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.397+0000 7f13c4db6700 1 -- 172.21.15.16:0/741263876 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13c00cc6e0 con 0x7f13ac006a60 2023-05-20T01:24:52.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.397+0000 7f13c4db6700 1 -- 172.21.15.16:0/741263876 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13ac00e290 con 0x7f13ac005c10 2023-05-20T01:24:52.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.397+0000 7f13c4db6700 1 -- 172.21.15.16:0/741263876 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13ac00e450 con 0x7f13ac007410 2023-05-20T01:24:52.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.397+0000 7f13c55b7700 1 --2- 172.21.15.16:0/741263876 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13ac006a60 0x7f13ac006ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:52.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.397+0000 7f13c5db8700 1 --2- 172.21.15.16:0/741263876 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f13ac005c10 0x7f13ac006020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:52.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.397+0000 7f13c65b9700 1 --2- 172.21.15.16:0/741263876 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f13ac007410 0x7f13ac00dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:52.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.398+0000 7f13c55b7700 1 -- 172.21.15.16:0/741263876 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f13ac007410 msgr2=0x7f13ac00dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:24:52.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.398+0000 7f13c55b7700 1 --2- 172.21.15.16:0/741263876 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f13ac007410 0x7f13ac00dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:52.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.398+0000 7f13c55b7700 1 -- 172.21.15.16:0/741263876 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f13ac005c10 msgr2=0x7f13ac006020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:24:52.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.398+0000 7f13c55b7700 1 --2- 172.21.15.16:0/741263876 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f13ac005c10 0x7f13ac006020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:52.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.398+0000 7f13c55b7700 1 -- 172.21.15.16:0/741263876 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13ac0057c0 con 0x7f13ac006a60 2023-05-20T01:24:52.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.398+0000 7f13c65b9700 1 -- 172.21.15.16:0/741263876 reap_dead start 2023-05-20T01:24:52.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.398+0000 7f13c5db8700 1 --2- 172.21.15.16:0/741263876 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f13ac005c10 0x7f13ac006020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-05-20T01:24:52.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.398+0000 7f13c55b7700 1 --2- 172.21.15.16:0/741263876 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13ac006a60 0x7f13ac006ed0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f13b0009b60 tx=0x7f13ac007a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1df3dd1872cc64a server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:24:52.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.398+0000 7f13c4db6700 1 -- 172.21.15.16:0/741263876 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13b000e6c0 con 0x7f13ac006a60 2023-05-20T01:24:52.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.398+0000 7f13c4db6700 1 -- 172.21.15.16:0/741263876 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13b000e820 con 0x7f13ac006a60 2023-05-20T01:24:52.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.398+0000 7f13c4db6700 1 -- 172.21.15.16:0/741263876 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13b000eb90 con 0x7f13ac006a60 2023-05-20T01:24:52.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.399+0000 7f13c801c700 1 -- 172.21.15.16:0/741263876 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13ac006a60 msgr2=0x7f13ac006ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:52.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.399+0000 7f13c801c700 1 --2- 172.21.15.16:0/741263876 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13ac006a60 0x7f13ac006ed0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f13b0009b60 tx=0x7f13ac007a90 comp rx=0 tx=0).stop 2023-05-20T01:24:52.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.399+0000 7f13c801c700 1 -- 172.21.15.16:0/741263876 shutdown_connections 2023-05-20T01:24:52.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.399+0000 7f13c801c700 1 --2- 172.21.15.16:0/741263876 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13ac006a60 0x7f13ac006ed0 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:52.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.399+0000 7f13c801c700 1 -- 172.21.15.16:0/741263876 >> 172.21.15.16:0/741263876 conn(0x7f13c00cffd0 msgr2=0x7f13c00d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:24:52.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.399+0000 7f13c801c700 1 -- 172.21.15.16:0/741263876 shutdown_connections 2023-05-20T01:24:52.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.399+0000 7f13c801c700 1 -- 172.21.15.16:0/741263876 wait complete. 2023-05-20T01:24:52.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.400+0000 7f13c801c700 1 Processor -- start 2023-05-20T01:24:52.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.400+0000 7f13c801c700 1 -- start start 2023-05-20T01:24:52.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.401+0000 7f13c801c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f13ac006a60 0x7f13c00b2e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:52.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.401+0000 7f13c801c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13c00aecb0 0x7f13c0151f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:52.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.401+0000 7f13c801c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f13c00ca830 0x7f13c00caca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:52.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.401+0000 7f13c801c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13c00cb1e0 con 0x7f13c00aecb0 2023-05-20T01:24:52.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.401+0000 7f13c801c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13c00b51d0 con 0x7f13ac006a60 2023-05-20T01:24:52.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.401+0000 7f13c801c700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13c00b5cc0 con 0x7f13c00ca830 2023-05-20T01:24:52.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.401+0000 7f13c55b7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13c00aecb0 0x7f13c0151f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:52.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.401+0000 7f13c55b7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13c00aecb0 0x7f13c0151f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35590/0 (socket says 172.21.15.16:35590) 2023-05-20T01:24:52.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.401+0000 7f13c55b7700 1 -- 172.21.15.16:0/168561462 learned_addr learned my addr 172.21.15.16:0/168561462 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:24:52.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.401+0000 7f13c5db8700 1 --2- 172.21.15.16:0/168561462 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f13ac006a60 0x7f13c00b2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:52.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.401+0000 7f13c65b9700 1 --2- 172.21.15.16:0/168561462 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f13c00ca830 0x7f13c00caca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:52.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.401+0000 7f13c55b7700 1 -- 172.21.15.16:0/168561462 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f13c00ca830 msgr2=0x7f13c00caca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:52.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.401+0000 7f13c55b7700 1 --2- 172.21.15.16:0/168561462 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f13c00ca830 0x7f13c00caca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:52.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.401+0000 7f13c55b7700 1 -- 172.21.15.16:0/168561462 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f13ac006a60 msgr2=0x7f13c00b2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:52.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.401+0000 7f13c55b7700 1 --2- 172.21.15.16:0/168561462 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f13ac006a60 0x7f13c00b2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:52.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.401+0000 7f13c55b7700 1 -- 172.21.15.16:0/168561462 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13c00cec70 con 0x7f13c00aecb0 2023-05-20T01:24:52.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.401+0000 7f13c55b7700 1 --2- 172.21.15.16:0/168561462 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13c00aecb0 0x7f13c0151f90 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f13b00090b0 tx=0x7f13b00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:24:52.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.401+0000 7f13b67fc700 1 -- 172.21.15.16:0/168561462 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13b000e6c0 con 0x7f13c00aecb0 2023-05-20T01:24:52.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.401+0000 7f13b67fc700 1 -- 172.21.15.16:0/168561462 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13b0003ad0 con 0x7f13c00aecb0 2023-05-20T01:24:52.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.401+0000 7f13c801c700 1 -- 172.21.15.16:0/168561462 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13c0071e80 con 0x7f13c00aecb0 2023-05-20T01:24:52.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.401+0000 7f13b67fc700 1 -- 172.21.15.16:0/168561462 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13b0003e80 con 0x7f13c00aecb0 2023-05-20T01:24:52.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.401+0000 7f13c801c700 1 -- 172.21.15.16:0/168561462 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13c00ce5b0 con 0x7f13c00aecb0 2023-05-20T01:24:52.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.402+0000 7f13c801c700 1 -- 172.21.15.16:0/168561462 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13c00aca60 con 0x7f13c00aecb0 2023-05-20T01:24:52.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.403+0000 7f13b67fc700 1 -- 172.21.15.16:0/168561462 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f13b0013470 con 0x7f13c00aecb0 2023-05-20T01:24:52.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.403+0000 7f13b67fc700 1 --2- 172.21.15.16:0/168561462 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f13a406ace0 0x7f13a406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:52.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.403+0000 7f13c5db8700 1 --2- 172.21.15.16:0/168561462 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f13a406ace0 0x7f13a406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:52.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.403+0000 7f13b67fc700 1 -- 172.21.15.16:0/168561462 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7189+0+0 (secure 0 0 0) 0x7f13b008b240 con 0x7f13c00aecb0 2023-05-20T01:24:52.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.404+0000 7f13c5db8700 1 --2- 172.21.15.16:0/168561462 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f13a406ace0 0x7f13a406d190 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f13bc000c00 tx=0x7f13bc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:24:52.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.408+0000 7f13b67fc700 1 -- 172.21.15.16:0/168561462 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f13b0055f90 con 0x7f13c00aecb0 2023-05-20T01:24:52.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.606+0000 7f13c801c700 1 -- 172.21.15.16:0/168561462 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f13c00747b0 con 0x7f13c00aecb0 2023-05-20T01:24:52.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.606+0000 7f13b67fc700 1 -- 172.21.15.16:0/168561462 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v75) v1 ==== 104+0+11 (secure 0 0 0) 0x7f13b00595b0 con 0x7f13c00aecb0 2023-05-20T01:24:52.611 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-05-20T01:24:52.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.608+0000 7f13c801c700 1 -- 172.21.15.16:0/168561462 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f13a406ace0 msgr2=0x7f13a406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:52.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.608+0000 7f13c801c700 1 --2- 172.21.15.16:0/168561462 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f13a406ace0 0x7f13a406d190 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f13bc000c00 tx=0x7f13bc009040 comp rx=0 tx=0).stop 2023-05-20T01:24:52.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.608+0000 7f13c801c700 1 -- 172.21.15.16:0/168561462 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13c00aecb0 msgr2=0x7f13c0151f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:52.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.608+0000 7f13c801c700 1 --2- 172.21.15.16:0/168561462 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13c00aecb0 0x7f13c0151f90 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f13b00090b0 tx=0x7f13b00047c0 comp rx=0 tx=0).stop 2023-05-20T01:24:52.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.608+0000 7f13c801c700 1 -- 172.21.15.16:0/168561462 shutdown_connections 2023-05-20T01:24:52.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.608+0000 7f13c801c700 1 --2- 172.21.15.16:0/168561462 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f13a406ace0 0x7f13a406d190 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:52.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.608+0000 7f13c801c700 1 --2- 172.21.15.16:0/168561462 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f13c00ca830 0x7f13c00caca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:52.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.608+0000 7f13c801c700 1 --2- 172.21.15.16:0/168561462 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f13ac006a60 0x7f13c00b2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:52.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.608+0000 7f13c801c700 1 --2- 172.21.15.16:0/168561462 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13c00aecb0 0x7f13c0151f90 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:52.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.608+0000 7f13c801c700 1 -- 172.21.15.16:0/168561462 >> 172.21.15.16:0/168561462 conn(0x7f13c00cffd0 msgr2=0x7f13c01546a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:24:52.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.608+0000 7f13c801c700 1 -- 172.21.15.16:0/168561462 shutdown_connections 2023-05-20T01:24:52.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.608+0000 7f13c801c700 1 -- 172.21.15.16:0/168561462 wait complete. 2023-05-20T01:24:52.625 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-05-20T01:24:52.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.748+0000 7facd459d700 1 Processor -- start 2023-05-20T01:24:52.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.748+0000 7facd459d700 1 -- start start 2023-05-20T01:24:52.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.749+0000 7facd459d700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7faccc146710 0x7faccc06b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:52.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.749+0000 7facd459d700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faccc0cb6c0 con 0x7faccc0ce1e0 2023-05-20T01:24:52.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.749+0000 7facd459d700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faccc0ce840 con 0x7faccc06bb60 2023-05-20T01:24:52.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.749+0000 7facd459d700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faccc0ce640 con 0x7faccc146710 2023-05-20T01:24:52.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.749+0000 7facd2b3a700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7faccc0ce1e0 0x7faccc0cc3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:47124/0 (socket says 172.21.15.16:47124) 2023-05-20T01:24:52.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.749+0000 7facd2b3a700 1 -- 172.21.15.16:0/2340235955 learned_addr learned my addr 172.21.15.16:0/2340235955 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:24:52.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.750+0000 7facd2339700 1 --2- 172.21.15.16:0/2340235955 >> v2:172.21.15.132:3300/0 conn(0x7faccc146710 0x7faccc06b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:52.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.750+0000 7facd1337700 1 -- 172.21.15.16:0/2340235955 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1744132396 0 0) 0x7faccc0cb6c0 con 0x7faccc0ce1e0 2023-05-20T01:24:52.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.750+0000 7facd1337700 1 -- 172.21.15.16:0/2340235955 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7facb80035b0 con 0x7faccc0ce1e0 2023-05-20T01:24:52.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.750+0000 7facd1337700 1 -- 172.21.15.16:0/2340235955 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3104012798 0 0) 0x7facb80035b0 con 0x7faccc0ce1e0 2023-05-20T01:24:52.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.750+0000 7facd1337700 1 -- 172.21.15.16:0/2340235955 >> v2:172.21.15.132:3300/0 conn(0x7faccc146710 msgr2=0x7faccc06b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:24:52.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.750+0000 7facd1337700 1 --2- 172.21.15.16:0/2340235955 >> v2:172.21.15.132:3300/0 conn(0x7faccc146710 0x7faccc06b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:52.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.750+0000 7facd1337700 1 -- 172.21.15.16:0/2340235955 >> v1:172.21.15.132:6789/0 conn(0x7faccc06bb60 legacy=0x7faccc0cdad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:52.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.750+0000 7facd1337700 1 -- 172.21.15.16:0/2340235955 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faccc0cb910 con 0x7faccc0ce1e0 2023-05-20T01:24:52.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.750+0000 7facd1337700 1 -- 172.21.15.16:0/2340235955 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7facc40019f0 con 0x7faccc0ce1e0 2023-05-20T01:24:52.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.750+0000 7facd1337700 1 -- 172.21.15.16:0/2340235955 >> v1:172.21.15.16:6789/0 conn(0x7faccc0ce1e0 legacy=0x7faccc0cc3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:52.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.750+0000 7facd1337700 1 --2- 172.21.15.16:0/2340235955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facb8003a00 0x7facb8003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:52.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.750+0000 7facd1337700 1 --2- 172.21.15.16:0/2340235955 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7facb8004850 0x7facb8004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:52.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.750+0000 7facd1337700 1 --2- 172.21.15.16:0/2340235955 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7facb8005200 0x7facb800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:52.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.750+0000 7facd1337700 1 -- 172.21.15.16:0/2340235955 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faccc0ce640 con 0x7facb8003a00 2023-05-20T01:24:52.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.750+0000 7facd1337700 1 -- 172.21.15.16:0/2340235955 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facb800c080 con 0x7facb8005200 2023-05-20T01:24:52.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.750+0000 7facd1337700 1 -- 172.21.15.16:0/2340235955 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facb800c240 con 0x7facb8004850 2023-05-20T01:24:52.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.750+0000 7facd2339700 1 --2- 172.21.15.16:0/2340235955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facb8003a00 0x7facb8003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:52.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.750+0000 7facd2b3a700 1 --2- 172.21.15.16:0/2340235955 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7facb8005200 0x7facb800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:52.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.750+0000 7facd1b38700 1 --2- 172.21.15.16:0/2340235955 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7facb8004850 0x7facb8004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:52.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.750+0000 7facd2339700 1 -- 172.21.15.16:0/2340235955 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7facb8004850 msgr2=0x7facb8004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:24:52.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.750+0000 7facd2339700 1 --2- 172.21.15.16:0/2340235955 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7facb8004850 0x7facb8004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:52.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.750+0000 7facd2339700 1 -- 172.21.15.16:0/2340235955 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7facb8005200 msgr2=0x7facb800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:24:52.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.750+0000 7facd2339700 1 --2- 172.21.15.16:0/2340235955 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7facb8005200 0x7facb800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:52.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.750+0000 7facd2339700 1 -- 172.21.15.16:0/2340235955 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facb80035b0 con 0x7facb8003a00 2023-05-20T01:24:52.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.750+0000 7facd2b3a700 1 -- 172.21.15.16:0/2340235955 reap_dead start 2023-05-20T01:24:52.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.751+0000 7facd2339700 1 --2- 172.21.15.16:0/2340235955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facb8003a00 0x7facb8003e10 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7facbc009b60 tx=0x7facb8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dfef993656d2da server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:24:52.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.751+0000 7facd1337700 1 -- 172.21.15.16:0/2340235955 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facbc00e6c0 con 0x7facb8003a00 2023-05-20T01:24:52.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.751+0000 7facd1337700 1 -- 172.21.15.16:0/2340235955 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facbc00e820 con 0x7facb8003a00 2023-05-20T01:24:52.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.751+0000 7facd1337700 1 -- 172.21.15.16:0/2340235955 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facbc00eb90 con 0x7facb8003a00 2023-05-20T01:24:52.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.751+0000 7facd459d700 1 -- 172.21.15.16:0/2340235955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facb8003a00 msgr2=0x7facb8003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:52.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.751+0000 7facd459d700 1 --2- 172.21.15.16:0/2340235955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facb8003a00 0x7facb8003e10 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7facbc009b60 tx=0x7facb8005880 comp rx=0 tx=0).stop 2023-05-20T01:24:52.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.751+0000 7facd459d700 1 -- 172.21.15.16:0/2340235955 shutdown_connections 2023-05-20T01:24:52.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.751+0000 7facd459d700 1 --2- 172.21.15.16:0/2340235955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facb8003a00 0x7facb8003e10 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:52.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.751+0000 7facd459d700 1 -- 172.21.15.16:0/2340235955 >> 172.21.15.16:0/2340235955 conn(0x7faccc06ae30 msgr2=0x7faccc09a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:24:52.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.751+0000 7facd459d700 1 -- 172.21.15.16:0/2340235955 shutdown_connections 2023-05-20T01:24:52.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.751+0000 7facd459d700 1 -- 172.21.15.16:0/2340235955 wait complete. 2023-05-20T01:24:52.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.753+0000 7facd459d700 1 Processor -- start 2023-05-20T01:24:52.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.753+0000 7facd459d700 1 -- start start 2023-05-20T01:24:52.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.753+0000 7facd459d700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7facb8003a00 0x7faccc151200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:52.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.753+0000 7facd459d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faccc0c8db0 0x7faccc0c4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:52.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.753+0000 7facd459d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faccc0c50e0 0x7faccc0ca350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:52.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.753+0000 7facd459d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faccc0c5550 con 0x7faccc0c50e0 2023-05-20T01:24:52.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.753+0000 7facd459d700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faccc0c9ec0 con 0x7faccc0c8db0 2023-05-20T01:24:52.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.753+0000 7facd459d700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faccc0c92b0 con 0x7facb8003a00 2023-05-20T01:24:52.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.754+0000 7facd2b3a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faccc0c50e0 0x7faccc0ca350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:52.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.754+0000 7facd2b3a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faccc0c50e0 0x7faccc0ca350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35610/0 (socket says 172.21.15.16:35610) 2023-05-20T01:24:52.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.754+0000 7facd2b3a700 1 -- 172.21.15.16:0/1600591953 learned_addr learned my addr 172.21.15.16:0/1600591953 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:24:52.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.754+0000 7facd1b38700 1 --2- 172.21.15.16:0/1600591953 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faccc0c8db0 0x7faccc0c4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:52.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.754+0000 7facd2b3a700 1 -- 172.21.15.16:0/1600591953 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7facb8003a00 msgr2=0x7faccc151200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:52.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.754+0000 7facd2b3a700 1 --2- 172.21.15.16:0/1600591953 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7facb8003a00 0x7faccc151200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:52.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.754+0000 7facd2b3a700 1 -- 172.21.15.16:0/1600591953 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faccc0c8db0 msgr2=0x7faccc0c4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:52.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.754+0000 7facd2b3a700 1 --2- 172.21.15.16:0/1600591953 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faccc0c8db0 0x7faccc0c4ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:52.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.754+0000 7facd2b3a700 1 -- 172.21.15.16:0/1600591953 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faccc0cbed0 con 0x7faccc0c50e0 2023-05-20T01:24:52.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.754+0000 7facd2b3a700 1 --2- 172.21.15.16:0/1600591953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faccc0c50e0 0x7faccc0ca350 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7faccc066bc0 tx=0x7facc400a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:24:52.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.754+0000 7facc2ffd700 1 -- 172.21.15.16:0/1600591953 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facc400ad60 con 0x7faccc0c50e0 2023-05-20T01:24:52.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.754+0000 7facc2ffd700 1 -- 172.21.15.16:0/1600591953 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facc40133f0 con 0x7faccc0c50e0 2023-05-20T01:24:52.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.754+0000 7facd459d700 1 -- 172.21.15.16:0/1600591953 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faccc0cbbf0 con 0x7faccc0c50e0 2023-05-20T01:24:52.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.754+0000 7facc2ffd700 1 -- 172.21.15.16:0/1600591953 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facc4013760 con 0x7faccc0c50e0 2023-05-20T01:24:52.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.754+0000 7facd459d700 1 -- 172.21.15.16:0/1600591953 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faccc06b620 con 0x7faccc0c50e0 2023-05-20T01:24:52.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.755+0000 7facd459d700 1 -- 172.21.15.16:0/1600591953 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faccc0747b0 con 0x7faccc0c50e0 2023-05-20T01:24:52.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.756+0000 7facc2ffd700 1 -- 172.21.15.16:0/1600591953 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7facc40138c0 con 0x7faccc0c50e0 2023-05-20T01:24:52.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.756+0000 7facc2ffd700 1 --2- 172.21.15.16:0/1600591953 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7facb006ace0 0x7facb006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:52.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.756+0000 7facd2339700 1 --2- 172.21.15.16:0/1600591953 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7facb006ace0 0x7facb006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:52.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.756+0000 7facc2ffd700 1 -- 172.21.15.16:0/1600591953 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7189+0+0 (secure 0 0 0) 0x7facc408a140 con 0x7faccc0c50e0 2023-05-20T01:24:52.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.757+0000 7facd2339700 1 --2- 172.21.15.16:0/1600591953 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7facb006ace0 0x7facb006d190 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7facbc0090b0 tx=0x7facbc013000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:24:52.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.761+0000 7facc2ffd700 1 -- 172.21.15.16:0/1600591953 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7facc4054ec0 con 0x7faccc0c50e0 2023-05-20T01:24:52.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.957+0000 7facd459d700 1 -- 172.21.15.16:0/1600591953 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7faccc0cdc70 con 0x7faccc0c50e0 2023-05-20T01:24:52.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.958+0000 7facc2ffd700 1 -- 172.21.15.16:0/1600591953 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v75) v1 ==== 100+0+11 (secure 0 0 0) 0x7facc40584e0 con 0x7faccc0c50e0 2023-05-20T01:24:52.962 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2023-05-20T01:24:52.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.960+0000 7facd459d700 1 -- 172.21.15.16:0/1600591953 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7facb006ace0 msgr2=0x7facb006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:52.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.960+0000 7facd459d700 1 --2- 172.21.15.16:0/1600591953 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7facb006ace0 0x7facb006d190 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7facbc0090b0 tx=0x7facbc013000 comp rx=0 tx=0).stop 2023-05-20T01:24:52.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.960+0000 7facd459d700 1 -- 172.21.15.16:0/1600591953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faccc0c50e0 msgr2=0x7faccc0ca350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:52.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.960+0000 7facd459d700 1 --2- 172.21.15.16:0/1600591953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faccc0c50e0 0x7faccc0ca350 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7faccc066bc0 tx=0x7facc400a5f0 comp rx=0 tx=0).stop 2023-05-20T01:24:52.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.960+0000 7facd459d700 1 -- 172.21.15.16:0/1600591953 shutdown_connections 2023-05-20T01:24:52.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.960+0000 7facd459d700 1 --2- 172.21.15.16:0/1600591953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faccc0c50e0 0x7faccc0ca350 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:52.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.960+0000 7facd459d700 1 --2- 172.21.15.16:0/1600591953 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7facb006ace0 0x7facb006d190 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:52.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.960+0000 7facd459d700 1 --2- 172.21.15.16:0/1600591953 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7facb8003a00 0x7faccc151200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:52.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.960+0000 7facd459d700 1 --2- 172.21.15.16:0/1600591953 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faccc0c8db0 0x7faccc0c4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:52.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.960+0000 7facd459d700 1 -- 172.21.15.16:0/1600591953 >> 172.21.15.16:0/1600591953 conn(0x7faccc06ae30 msgr2=0x7faccc15f5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:24:52.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.960+0000 7facd459d700 1 -- 172.21.15.16:0/1600591953 shutdown_connections 2023-05-20T01:24:52.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:52.960+0000 7facd459d700 1 -- 172.21.15.16:0/1600591953 wait complete. 2023-05-20T01:24:52.977 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_extra pg_num 2023-05-20T01:24:53.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.101+0000 7f9f6f0c4700 1 Processor -- start 2023-05-20T01:24:53.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.101+0000 7f9f6f0c4700 1 -- start start 2023-05-20T01:24:53.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.102+0000 7f9f6f0c4700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9f680cd070 0x7f9f6806e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:53.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.102+0000 7f9f6f0c4700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9f680710d0 0x7f9f680cabb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:53.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.102+0000 7f9f6f0c4700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f68072430 con 0x7f9f6815cf40 2023-05-20T01:24:53.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.102+0000 7f9f6f0c4700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f68071550 con 0x7f9f680cd070 2023-05-20T01:24:53.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.102+0000 7f9f6f0c4700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f68071820 con 0x7f9f680710d0 2023-05-20T01:24:53.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.102+0000 7f9f67fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9f680cd070 0x7f9f6806e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:53.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.102+0000 7f9f67fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9f680cd070 0x7f9f6806e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35616/0 (socket says 172.21.15.16:35616) 2023-05-20T01:24:53.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.102+0000 7f9f67fff700 1 -- 172.21.15.16:0/387975269 learned_addr learned my addr 172.21.15.16:0/387975269 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:24:53.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.102+0000 7f9f6d661700 1 --2- 172.21.15.16:0/387975269 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9f680710d0 0x7f9f680cabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:53.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.102+0000 7f9f67fff700 1 -- 172.21.15.16:0/387975269 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9f680710d0 msgr2=0x7f9f680cabb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:24:53.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.102+0000 7f9f67fff700 1 --2- 172.21.15.16:0/387975269 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9f680710d0 0x7f9f680cabb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:53.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.102+0000 7f9f67fff700 1 -- 172.21.15.16:0/387975269 >> v1:172.21.15.132:6789/0 conn(0x7f9f6815cf40 legacy=0x7f9f680cc960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:53.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.102+0000 7f9f67fff700 1 -- 172.21.15.16:0/387975269 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f68071b00 con 0x7f9f680cd070 2023-05-20T01:24:53.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.102+0000 7f9f67fff700 1 --2- 172.21.15.16:0/387975269 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f680cd070 0x7f9f6806e020 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f9f5400bb90 tx=0x7f9f5400bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c53d25134daef2e server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:24:53.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.103+0000 7f9f677fe700 1 -- 172.21.15.16:0/387975269 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f5400dd50 con 0x7f9f680cd070 2023-05-20T01:24:53.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.103+0000 7f9f677fe700 1 -- 172.21.15.16:0/387975269 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f54010470 con 0x7f9f680cd070 2023-05-20T01:24:53.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.103+0000 7f9f677fe700 1 -- 172.21.15.16:0/387975269 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f54010800 con 0x7f9f680cd070 2023-05-20T01:24:53.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.103+0000 7f9f6f0c4700 1 -- 172.21.15.16:0/387975269 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f680cd070 msgr2=0x7f9f6806e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:53.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.103+0000 7f9f6f0c4700 1 --2- 172.21.15.16:0/387975269 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f680cd070 0x7f9f6806e020 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f9f5400bb90 tx=0x7f9f5400bea0 comp rx=0 tx=0).stop 2023-05-20T01:24:53.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.103+0000 7f9f6f0c4700 1 -- 172.21.15.16:0/387975269 shutdown_connections 2023-05-20T01:24:53.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.103+0000 7f9f6f0c4700 1 --2- 172.21.15.16:0/387975269 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9f680710d0 0x7f9f680cabb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:53.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.103+0000 7f9f6f0c4700 1 --2- 172.21.15.16:0/387975269 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f680cd070 0x7f9f6806e020 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:53.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.103+0000 7f9f6f0c4700 1 -- 172.21.15.16:0/387975269 >> 172.21.15.16:0/387975269 conn(0x7f9f6806c260 msgr2=0x7f9f680cffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:24:53.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.103+0000 7f9f6f0c4700 1 -- 172.21.15.16:0/387975269 shutdown_connections 2023-05-20T01:24:53.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.103+0000 7f9f6f0c4700 1 -- 172.21.15.16:0/387975269 wait complete. 2023-05-20T01:24:53.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.104+0000 7f9f6f0c4700 1 Processor -- start 2023-05-20T01:24:53.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.104+0000 7f9f6f0c4700 1 -- start start 2023-05-20T01:24:53.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.105+0000 7f9f6f0c4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f680710d0 0x7f9f680c8900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:53.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.105+0000 7f9f6f0c4700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9f680cd070 0x7f9f680c5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:53.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.105+0000 7f9f6f0c4700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9f6815cf40 0x7f9f68143df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:53.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.105+0000 7f9f6f0c4700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f680c65a0 con 0x7f9f680710d0 2023-05-20T01:24:53.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.105+0000 7f9f6f0c4700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f680c6880 con 0x7f9f6815cf40 2023-05-20T01:24:53.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.105+0000 7f9f6f0c4700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f680b2ee0 con 0x7f9f680cd070 2023-05-20T01:24:53.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.105+0000 7f9f6ce60700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f680710d0 0x7f9f680c8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:53.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.105+0000 7f9f6ce60700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f680710d0 0x7f9f680c8900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35620/0 (socket says 172.21.15.16:35620) 2023-05-20T01:24:53.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.105+0000 7f9f6ce60700 1 -- 172.21.15.16:0/1723076151 learned_addr learned my addr 172.21.15.16:0/1723076151 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:24:53.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.105+0000 7f9f6d661700 1 --2- 172.21.15.16:0/1723076151 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9f6815cf40 0x7f9f68143df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:53.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.106+0000 7f9f67fff700 1 --2- 172.21.15.16:0/1723076151 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9f680cd070 0x7f9f680c5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:53.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.106+0000 7f9f6ce60700 1 -- 172.21.15.16:0/1723076151 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9f680cd070 msgr2=0x7f9f680c5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:53.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.106+0000 7f9f6ce60700 1 --2- 172.21.15.16:0/1723076151 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9f680cd070 0x7f9f680c5ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:53.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.106+0000 7f9f6ce60700 1 -- 172.21.15.16:0/1723076151 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9f6815cf40 msgr2=0x7f9f68143df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:53.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.106+0000 7f9f6ce60700 1 --2- 172.21.15.16:0/1723076151 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9f6815cf40 0x7f9f68143df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:53.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.106+0000 7f9f6ce60700 1 -- 172.21.15.16:0/1723076151 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f5400b7f0 con 0x7f9f680710d0 2023-05-20T01:24:53.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.106+0000 7f9f6ce60700 1 --2- 172.21.15.16:0/1723076151 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f680710d0 0x7f9f680c8900 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c00c450 tx=0x7f9f5c00c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:24:53.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.106+0000 7f9f657fa700 1 -- 172.21.15.16:0/1723076151 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f5c0052d0 con 0x7f9f680710d0 2023-05-20T01:24:53.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.106+0000 7f9f657fa700 1 -- 172.21.15.16:0/1723076151 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f5c005430 con 0x7f9f680710d0 2023-05-20T01:24:53.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.106+0000 7f9f6f0c4700 1 -- 172.21.15.16:0/1723076151 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f680720c0 con 0x7f9f680710d0 2023-05-20T01:24:53.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.106+0000 7f9f657fa700 1 -- 172.21.15.16:0/1723076151 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f5c0038c0 con 0x7f9f680710d0 2023-05-20T01:24:53.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.106+0000 7f9f6f0c4700 1 -- 172.21.15.16:0/1723076151 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f68071de0 con 0x7f9f680710d0 2023-05-20T01:24:53.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.107+0000 7f9f6f0c4700 1 -- 172.21.15.16:0/1723076151 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f680cb0f0 con 0x7f9f680710d0 2023-05-20T01:24:53.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.107+0000 7f9f657fa700 1 -- 172.21.15.16:0/1723076151 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9f5c003a20 con 0x7f9f680710d0 2023-05-20T01:24:53.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.108+0000 7f9f657fa700 1 --2- 172.21.15.16:0/1723076151 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f9f4c06ace0 0x7f9f4c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:53.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.108+0000 7f9f67fff700 1 --2- 172.21.15.16:0/1723076151 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f9f4c06ace0 0x7f9f4c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:53.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.108+0000 7f9f657fa700 1 -- 172.21.15.16:0/1723076151 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7189+0+0 (secure 0 0 0) 0x7f9f5c088b80 con 0x7f9f680710d0 2023-05-20T01:24:53.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.108+0000 7f9f67fff700 1 --2- 172.21.15.16:0/1723076151 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f9f4c06ace0 0x7f9f4c06d190 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f9f5400bb90 tx=0x7f9f54013000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:24:53.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.112+0000 7f9f657fa700 1 -- 172.21.15.16:0/1723076151 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f9f5c053850 con 0x7f9f680710d0 2023-05-20T01:24:53.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.310+0000 7f9f6f0c4700 1 -- 172.21.15.16:0/1723076151 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"} v 0) v1 -- 0x7f9f6806dc30 con 0x7f9f680710d0 2023-05-20T01:24:53.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.310+0000 7f9f657fa700 1 -- 172.21.15.16:0/1723076151 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"}]=0 v75) v1 ==== 106+0+11 (secure 0 0 0) 0x7f9f5c00ce60 con 0x7f9f680710d0 2023-05-20T01:24:53.315 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-05-20T01:24:53.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.312+0000 7f9f6f0c4700 1 -- 172.21.15.16:0/1723076151 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f9f4c06ace0 msgr2=0x7f9f4c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:53.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.312+0000 7f9f6f0c4700 1 --2- 172.21.15.16:0/1723076151 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f9f4c06ace0 0x7f9f4c06d190 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f9f5400bb90 tx=0x7f9f54013000 comp rx=0 tx=0).stop 2023-05-20T01:24:53.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.312+0000 7f9f6f0c4700 1 -- 172.21.15.16:0/1723076151 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f680710d0 msgr2=0x7f9f680c8900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:53.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.312+0000 7f9f6f0c4700 1 --2- 172.21.15.16:0/1723076151 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f680710d0 0x7f9f680c8900 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c00c450 tx=0x7f9f5c00c760 comp rx=0 tx=0).stop 2023-05-20T01:24:53.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.312+0000 7f9f6f0c4700 1 -- 172.21.15.16:0/1723076151 shutdown_connections 2023-05-20T01:24:53.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.312+0000 7f9f6f0c4700 1 --2- 172.21.15.16:0/1723076151 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9f6815cf40 0x7f9f68143df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:53.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.312+0000 7f9f6f0c4700 1 --2- 172.21.15.16:0/1723076151 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f680710d0 0x7f9f680c8900 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:53.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.312+0000 7f9f6f0c4700 1 --2- 172.21.15.16:0/1723076151 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f9f4c06ace0 0x7f9f4c06d190 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:53.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.312+0000 7f9f6f0c4700 1 --2- 172.21.15.16:0/1723076151 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9f680cd070 0x7f9f680c5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:53.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.312+0000 7f9f6f0c4700 1 -- 172.21.15.16:0/1723076151 >> 172.21.15.16:0/1723076151 conn(0x7f9f6806c260 msgr2=0x7f9f681599b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:24:53.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.312+0000 7f9f6f0c4700 1 -- 172.21.15.16:0/1723076151 shutdown_connections 2023-05-20T01:24:53.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.312+0000 7f9f6f0c4700 1 -- 172.21.15.16:0/1723076151 wait complete. 2023-05-20T01:24:53.329 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-05-20T01:24:53.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.450+0000 7f31a47a1700 1 Processor -- start 2023-05-20T01:24:53.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.450+0000 7f31a47a1700 1 -- start start 2023-05-20T01:24:53.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.451+0000 7f31a47a1700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f319c071ad0 0x7f319c1645b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:53.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.451+0000 7f31a47a1700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f319c0cb590 con 0x7f319c0ccec0 2023-05-20T01:24:53.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.451+0000 7f31a47a1700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f319c0ca9a0 con 0x7f319c157190 2023-05-20T01:24:53.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.451+0000 7f31a47a1700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f319c071f50 con 0x7f319c071ad0 2023-05-20T01:24:53.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.451+0000 7f31a1d3c700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f319c0ccec0 0x7f319c0713c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:47156/0 (socket says 172.21.15.16:47156) 2023-05-20T01:24:53.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.451+0000 7f31a1d3c700 1 -- 172.21.15.16:0/2386543226 learned_addr learned my addr 172.21.15.16:0/2386543226 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:24:53.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.451+0000 7f31a2d3e700 1 --2- 172.21.15.16:0/2386543226 >> v2:172.21.15.132:3300/0 conn(0x7f319c071ad0 0x7f319c1645b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:53.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.451+0000 7f31a153b700 1 -- 172.21.15.16:0/2386543226 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2009351820 0 0) 0x7f319c0cb590 con 0x7f319c0ccec0 2023-05-20T01:24:53.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.451+0000 7f31a153b700 1 -- 172.21.15.16:0/2386543226 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f31880035b0 con 0x7f319c0ccec0 2023-05-20T01:24:53.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.452+0000 7f31a153b700 1 -- 172.21.15.16:0/2386543226 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1453319935 0 0) 0x7f31880035b0 con 0x7f319c0ccec0 2023-05-20T01:24:53.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.452+0000 7f31a153b700 1 -- 172.21.15.16:0/2386543226 >> v2:172.21.15.132:3300/0 conn(0x7f319c071ad0 msgr2=0x7f319c1645b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:24:53.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.452+0000 7f31a153b700 1 --2- 172.21.15.16:0/2386543226 >> v2:172.21.15.132:3300/0 conn(0x7f319c071ad0 0x7f319c1645b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:53.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.452+0000 7f31a153b700 1 -- 172.21.15.16:0/2386543226 >> v1:172.21.15.132:6789/0 conn(0x7f319c157190 legacy=0x7f319c0cc7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:53.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.452+0000 7f31a153b700 1 -- 172.21.15.16:0/2386543226 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f319c0cac60 con 0x7f319c0ccec0 2023-05-20T01:24:53.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.452+0000 7f31a153b700 1 -- 172.21.15.16:0/2386543226 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7f3198001670 con 0x7f319c0ccec0 2023-05-20T01:24:53.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.452+0000 7f31a153b700 1 -- 172.21.15.16:0/2386543226 >> v1:172.21.15.16:6789/0 conn(0x7f319c0ccec0 legacy=0x7f319c0713c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:53.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.452+0000 7f31a153b700 1 --2- 172.21.15.16:0/2386543226 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3188003a70 0x7f3188003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:53.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.452+0000 7f31a153b700 1 --2- 172.21.15.16:0/2386543226 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3188004850 0x7f3188004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:53.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.452+0000 7f31a153b700 1 --2- 172.21.15.16:0/2386543226 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3188005250 0x7f318800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:53.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.452+0000 7f31a153b700 1 -- 172.21.15.16:0/2386543226 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f319c071f50 con 0x7f3188005250 2023-05-20T01:24:53.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.452+0000 7f31a153b700 1 -- 172.21.15.16:0/2386543226 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f318800c190 con 0x7f3188003a70 2023-05-20T01:24:53.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.452+0000 7f31a153b700 1 -- 172.21.15.16:0/2386543226 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f318800c350 con 0x7f3188004850 2023-05-20T01:24:53.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.452+0000 7f31a2d3e700 1 --2- 172.21.15.16:0/2386543226 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3188005250 0x7f318800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:53.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.452+0000 7f31a253d700 1 --2- 172.21.15.16:0/2386543226 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3188003a70 0x7f3188003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:53.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.452+0000 7f31a1d3c700 1 --2- 172.21.15.16:0/2386543226 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3188004850 0x7f3188004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:53.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.452+0000 7f31a2d3e700 1 -- 172.21.15.16:0/2386543226 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3188004850 msgr2=0x7f3188004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:24:53.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.452+0000 7f31a2d3e700 1 --2- 172.21.15.16:0/2386543226 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3188004850 0x7f3188004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:53.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.452+0000 7f31a2d3e700 1 -- 172.21.15.16:0/2386543226 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3188003a70 msgr2=0x7f3188003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:24:53.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.452+0000 7f31a2d3e700 1 --2- 172.21.15.16:0/2386543226 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3188003a70 0x7f3188003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:53.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.452+0000 7f31a2d3e700 1 -- 172.21.15.16:0/2386543226 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31880035b0 con 0x7f3188005250 2023-05-20T01:24:53.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.452+0000 7f31a2d3e700 1 -- 172.21.15.16:0/2386543226 reap_dead start 2023-05-20T01:24:53.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.452+0000 7f31a2d3e700 1 --2- 172.21.15.16:0/2386543226 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3188005250 0x7f318800ba10 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f3194009f90 tx=0x7f3188005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d3e48a0532b7f8b server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:24:53.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.452+0000 7f31a153b700 1 -- 172.21.15.16:0/2386543226 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3194003a20 con 0x7f3188005250 2023-05-20T01:24:53.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.453+0000 7f31a153b700 1 -- 172.21.15.16:0/2386543226 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f319400f040 con 0x7f3188005250 2023-05-20T01:24:53.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.453+0000 7f31a153b700 1 -- 172.21.15.16:0/2386543226 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3194003dc0 con 0x7f3188005250 2023-05-20T01:24:53.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.453+0000 7f31a47a1700 1 -- 172.21.15.16:0/2386543226 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3188005250 msgr2=0x7f318800ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:53.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.453+0000 7f31a47a1700 1 --2- 172.21.15.16:0/2386543226 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3188005250 0x7f318800ba10 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f3194009f90 tx=0x7f3188005880 comp rx=0 tx=0).stop 2023-05-20T01:24:53.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.453+0000 7f31a47a1700 1 -- 172.21.15.16:0/2386543226 shutdown_connections 2023-05-20T01:24:53.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.453+0000 7f31a47a1700 1 --2- 172.21.15.16:0/2386543226 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3188005250 0x7f318800ba10 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:53.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.453+0000 7f31a47a1700 1 -- 172.21.15.16:0/2386543226 >> 172.21.15.16:0/2386543226 conn(0x7f319c0cffb0 msgr2=0x7f319c0d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:24:53.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.453+0000 7f31a47a1700 1 -- 172.21.15.16:0/2386543226 shutdown_connections 2023-05-20T01:24:53.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.453+0000 7f31a47a1700 1 -- 172.21.15.16:0/2386543226 wait complete. 2023-05-20T01:24:53.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.454+0000 7f31a47a1700 1 Processor -- start 2023-05-20T01:24:53.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.454+0000 7f31a47a1700 1 -- start start 2023-05-20T01:24:53.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.455+0000 7f31a47a1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3188005250 0x7f319c152ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:53.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.455+0000 7f31a47a1700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f319c154db0 0x7f319c155220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:53.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.455+0000 7f31a47a1700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f319c0ae220 0x7f319c0ae690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:53.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.455+0000 7f31a47a1700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f319c0ad770 con 0x7f3188005250 2023-05-20T01:24:53.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.455+0000 7f31a47a1700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f319c155760 con 0x7f319c154db0 2023-05-20T01:24:53.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.455+0000 7f31a47a1700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f319c0ac8a0 con 0x7f319c0ae220 2023-05-20T01:24:53.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.455+0000 7f31a253d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3188005250 0x7f319c152ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:53.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.455+0000 7f31a253d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3188005250 0x7f319c152ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35640/0 (socket says 172.21.15.16:35640) 2023-05-20T01:24:53.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.455+0000 7f31a253d700 1 -- 172.21.15.16:0/4133619858 learned_addr learned my addr 172.21.15.16:0/4133619858 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:24:53.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.455+0000 7f31a2d3e700 1 --2- 172.21.15.16:0/4133619858 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f319c0ae220 0x7f319c0ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:53.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.455+0000 7f31a1d3c700 1 --2- 172.21.15.16:0/4133619858 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f319c154db0 0x7f319c155220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:53.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.455+0000 7f31a253d700 1 -- 172.21.15.16:0/4133619858 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f319c0ae220 msgr2=0x7f319c0ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:53.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.456+0000 7f31a253d700 1 --2- 172.21.15.16:0/4133619858 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f319c0ae220 0x7f319c0ae690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:53.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.456+0000 7f31a253d700 1 -- 172.21.15.16:0/4133619858 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f319c154db0 msgr2=0x7f319c155220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:53.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.456+0000 7f31a253d700 1 --2- 172.21.15.16:0/4133619858 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f319c154db0 0x7f319c155220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:53.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.456+0000 7f31a253d700 1 -- 172.21.15.16:0/4133619858 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f319c0cb220 con 0x7f3188005250 2023-05-20T01:24:53.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.456+0000 7f31a253d700 1 --2- 172.21.15.16:0/4133619858 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3188005250 0x7f319c152ac0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f318c00f880 tx=0x7f318c00fc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:24:53.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.456+0000 7f3192ffd700 1 -- 172.21.15.16:0/4133619858 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f318c0050d0 con 0x7f3188005250 2023-05-20T01:24:53.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.456+0000 7f3192ffd700 1 -- 172.21.15.16:0/4133619858 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f318c005230 con 0x7f3188005250 2023-05-20T01:24:53.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.456+0000 7f31a47a1700 1 -- 172.21.15.16:0/4133619858 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f319c0caf40 con 0x7f3188005250 2023-05-20T01:24:53.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.456+0000 7f3192ffd700 1 -- 172.21.15.16:0/4133619858 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f318c003680 con 0x7f3188005250 2023-05-20T01:24:53.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.456+0000 7f31a47a1700 1 -- 172.21.15.16:0/4133619858 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f319c157560 con 0x7f3188005250 2023-05-20T01:24:53.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.457+0000 7f31a47a1700 1 -- 172.21.15.16:0/4133619858 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f319c0747b0 con 0x7f3188005250 2023-05-20T01:24:53.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.457+0000 7f3192ffd700 1 -- 172.21.15.16:0/4133619858 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f318c0037e0 con 0x7f3188005250 2023-05-20T01:24:53.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.458+0000 7f3192ffd700 1 --2- 172.21.15.16:0/4133619858 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f318006ace0 0x7f318006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:53.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.458+0000 7f3192ffd700 1 -- 172.21.15.16:0/4133619858 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7189+0+0 (secure 0 0 0) 0x7f318c0895d0 con 0x7f3188005250 2023-05-20T01:24:53.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.458+0000 7f31a1d3c700 1 --2- 172.21.15.16:0/4133619858 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f318006ace0 0x7f318006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:53.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.458+0000 7f31a1d3c700 1 --2- 172.21.15.16:0/4133619858 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f318006ace0 0x7f318006d190 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f3198001050 tx=0x7f3198009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:24:53.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.462+0000 7f3192ffd700 1 -- 172.21.15.16:0/4133619858 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f318c054210 con 0x7f3188005250 2023-05-20T01:24:53.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.678+0000 7f31a47a1700 1 -- 172.21.15.16:0/4133619858 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f319c15c880 con 0x7f3188005250 2023-05-20T01:24:53.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.678+0000 7f3192ffd700 1 -- 172.21.15.16:0/4133619858 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 6 entries v75) v1 ==== 80+0+366 (secure 0 0 0) 0x7f318c057830 con 0x7f3188005250 2023-05-20T01:24:53.683 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6837/3291457857 2023-05-21T01:22:20.558472+0000 2023-05-20T01:24:53.683 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6835/3291457857 2023-05-21T01:22:20.558472+0000 2023-05-20T01:24:53.684 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6837/1427887654 2023-05-21T01:22:14.372197+0000 2023-05-20T01:24:53.684 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6835/1427887654 2023-05-21T01:22:14.372197+0000 2023-05-20T01:24:53.684 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6837/1427908556 2023-05-21T01:22:12.737196+0000 2023-05-20T01:24:53.685 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6835/1427908556 2023-05-21T01:22:12.737196+0000 2023-05-20T01:24:53.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.680+0000 7f31a47a1700 1 -- 172.21.15.16:0/4133619858 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f318006ace0 msgr2=0x7f318006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:53.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.680+0000 7f31a47a1700 1 --2- 172.21.15.16:0/4133619858 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f318006ace0 0x7f318006d190 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f3198001050 tx=0x7f3198009040 comp rx=0 tx=0).stop 2023-05-20T01:24:53.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.680+0000 7f31a47a1700 1 -- 172.21.15.16:0/4133619858 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3188005250 msgr2=0x7f319c152ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:53.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.680+0000 7f31a47a1700 1 --2- 172.21.15.16:0/4133619858 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3188005250 0x7f319c152ac0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f318c00f880 tx=0x7f318c00fc40 comp rx=0 tx=0).stop 2023-05-20T01:24:53.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.680+0000 7f31a47a1700 1 -- 172.21.15.16:0/4133619858 shutdown_connections 2023-05-20T01:24:53.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.680+0000 7f31a47a1700 1 --2- 172.21.15.16:0/4133619858 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f318006ace0 0x7f318006d190 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:53.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.680+0000 7f31a47a1700 1 --2- 172.21.15.16:0/4133619858 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f319c0ae220 0x7f319c0ae690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:53.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.680+0000 7f31a47a1700 1 --2- 172.21.15.16:0/4133619858 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3188005250 0x7f319c152ac0 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:53.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.680+0000 7f31a47a1700 1 --2- 172.21.15.16:0/4133619858 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f319c154db0 0x7f319c155220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:53.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.680+0000 7f31a47a1700 1 -- 172.21.15.16:0/4133619858 >> 172.21.15.16:0/4133619858 conn(0x7f319c0cffb0 msgr2=0x7f319c159860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:24:53.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.680+0000 7f31a47a1700 1 -- 172.21.15.16:0/4133619858 shutdown_connections 2023-05-20T01:24:53.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.680+0000 7f31a47a1700 1 -- 172.21.15.16:0/4133619858 wait complete. 2023-05-20T01:24:53.690 INFO:teuthology.orchestra.run.smithi016.stderr:listed 6 entries 2023-05-20T01:24:53.696 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-05-20T01:24:53.697 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:24:53.697 DEBUG:teuthology.orchestra.run.smithi016:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-05-20T01:24:53.745 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2023-05-20T01:24:53.745 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:24:53.745 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-05-20T01:24:53.764 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2023-05-20T01:24:53.764 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/proc/self/mounts of=/dev/stdout 2023-05-20T01:24:53.785 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284284k,nr_inodes=4071071,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=45,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16175 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260556k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-05-20T01:24:53.785 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-05-20T01:24:53.785 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2023-05-20T01:24:53.821 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16287364k,nr_inodes=4071841,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=38,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16696 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3261172k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-05-20T01:24:53.821 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2023-05-20T01:24:53.821 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/proc/self/mounts of=/dev/stdout 2023-05-20T01:24:53.843 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284284k,nr_inodes=4071071,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=45,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16175 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260556k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-05-20T01:24:53.844 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-20T01:24:53.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.982+0000 7f366ccf6700 1 Processor -- start 2023-05-20T01:24:53.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.982+0000 7f366ccf6700 1 -- start start 2023-05-20T01:24:53.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.983+0000 7f366ccf6700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f366806ba90 0x7f36680cbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:53.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.983+0000 7f366ccf6700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36680ce3e0 con 0x7f366806b620 2023-05-20T01:24:53.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.983+0000 7f366ccf6700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36680cdab0 con 0x7f36680d1000 2023-05-20T01:24:53.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.983+0000 7f366ccf6700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f366806bef0 con 0x7f366806ba90 2023-05-20T01:24:53.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.983+0000 7f3666d9d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f366806ba90 0x7f36680cbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:53.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.983+0000 7f3665d9b700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f366806b620 0x7f36680cb6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:47176/0 (socket says 172.21.15.16:47176) 2023-05-20T01:24:53.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.983+0000 7f3665d9b700 1 -- 172.21.15.16:0/950037241 learned_addr learned my addr 172.21.15.16:0/950037241 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:24:53.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.984+0000 7f3666d9d700 1 -- 172.21.15.16:0/950037241 >> v1:172.21.15.132:6789/0 conn(0x7f36680d1000 legacy=0x7f36680d13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:53.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.984+0000 7f3666d9d700 1 -- 172.21.15.16:0/950037241 >> v1:172.21.15.16:6789/0 conn(0x7f366806b620 legacy=0x7f36680cb6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:53.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.984+0000 7f3666d9d700 1 -- 172.21.15.16:0/950037241 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36680cdd90 con 0x7f366806ba90 2023-05-20T01:24:53.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.984+0000 7f366559a700 1 -- 172.21.15.16:0/950037241 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2799349537 0 0) 0x7f36680ce3e0 con 0x7f366806b620 2023-05-20T01:24:53.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.984+0000 7f3666d9d700 1 --2- 172.21.15.16:0/950037241 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f366806ba90 0x7f36680cbdf0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f3660009d90 tx=0x7f366000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf0a69e02d339f9c server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:24:53.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.984+0000 7f366559a700 1 -- 172.21.15.16:0/950037241 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f366000e070 con 0x7f366806ba90 2023-05-20T01:24:53.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.984+0000 7f366559a700 1 -- 172.21.15.16:0/950037241 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f366000bbd0 con 0x7f366806ba90 2023-05-20T01:24:53.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.984+0000 7f366559a700 1 -- 172.21.15.16:0/950037241 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3660012430 con 0x7f366806ba90 2023-05-20T01:24:53.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.985+0000 7f366ccf6700 1 -- 172.21.15.16:0/950037241 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f366806ba90 msgr2=0x7f36680cbdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:53.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.985+0000 7f366ccf6700 1 --2- 172.21.15.16:0/950037241 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f366806ba90 0x7f36680cbdf0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f3660009d90 tx=0x7f366000b330 comp rx=0 tx=0).stop 2023-05-20T01:24:53.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.985+0000 7f366ccf6700 1 -- 172.21.15.16:0/950037241 shutdown_connections 2023-05-20T01:24:53.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.985+0000 7f366ccf6700 1 --2- 172.21.15.16:0/950037241 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f366806ba90 0x7f36680cbdf0 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:53.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.985+0000 7f366ccf6700 1 -- 172.21.15.16:0/950037241 >> 172.21.15.16:0/950037241 conn(0x7f366806ae30 msgr2=0x7f366809a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:24:53.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.985+0000 7f366ccf6700 1 -- 172.21.15.16:0/950037241 shutdown_connections 2023-05-20T01:24:53.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.985+0000 7f366ccf6700 1 -- 172.21.15.16:0/950037241 wait complete. 2023-05-20T01:24:53.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.986+0000 7f366ccf6700 1 Processor -- start 2023-05-20T01:24:53.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.987+0000 7f366ccf6700 1 -- start start 2023-05-20T01:24:53.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.987+0000 7f366ccf6700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f366806b620 0x7f36680c6f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:53.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.987+0000 7f366ccf6700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f366806ba90 0x7f366815b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:53.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.988+0000 7f366ccf6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36680d1000 0x7f366809afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:53.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.988+0000 7f366ccf6700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36680b5a50 con 0x7f36680d1000 2023-05-20T01:24:53.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.988+0000 7f366ccf6700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36680b5d30 con 0x7f366806ba90 2023-05-20T01:24:53.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.988+0000 7f366ccf6700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36680b6010 con 0x7f366806b620 2023-05-20T01:24:53.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.988+0000 7f3666d9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36680d1000 0x7f366809afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:53.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.988+0000 7f3666d9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36680d1000 0x7f366809afe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35658/0 (socket says 172.21.15.16:35658) 2023-05-20T01:24:53.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.988+0000 7f3666d9d700 1 -- 172.21.15.16:0/154349942 learned_addr learned my addr 172.21.15.16:0/154349942 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:24:53.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.988+0000 7f366659c700 1 --2- 172.21.15.16:0/154349942 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f366806b620 0x7f36680c6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:53.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.988+0000 7f3665d9b700 1 --2- 172.21.15.16:0/154349942 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f366806ba90 0x7f366815b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:53.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.988+0000 7f3666d9d700 1 -- 172.21.15.16:0/154349942 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f366806b620 msgr2=0x7f36680c6f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:53.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.988+0000 7f3666d9d700 1 --2- 172.21.15.16:0/154349942 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f366806b620 0x7f36680c6f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:53.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.988+0000 7f3666d9d700 1 -- 172.21.15.16:0/154349942 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f366806ba90 msgr2=0x7f366815b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:53.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.988+0000 7f3666d9d700 1 --2- 172.21.15.16:0/154349942 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f366806ba90 0x7f366815b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:53.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.988+0000 7f3666d9d700 1 -- 172.21.15.16:0/154349942 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3660009a70 con 0x7f36680d1000 2023-05-20T01:24:53.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.988+0000 7f3666d9d700 1 --2- 172.21.15.16:0/154349942 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36680d1000 0x7f366809afe0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f3660014040 tx=0x7f3660004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:24:53.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.988+0000 7f365effd700 1 -- 172.21.15.16:0/154349942 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3660012430 con 0x7f36680d1000 2023-05-20T01:24:53.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.988+0000 7f365effd700 1 -- 172.21.15.16:0/154349942 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3660020070 con 0x7f36680d1000 2023-05-20T01:24:53.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.988+0000 7f366ccf6700 1 -- 172.21.15.16:0/154349942 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36680b28b0 con 0x7f36680d1000 2023-05-20T01:24:53.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.988+0000 7f365effd700 1 -- 172.21.15.16:0/154349942 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3660004280 con 0x7f36680d1000 2023-05-20T01:24:53.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.988+0000 7f366ccf6700 1 -- 172.21.15.16:0/154349942 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36680ce070 con 0x7f36680d1000 2023-05-20T01:24:53.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.989+0000 7f366ccf6700 1 -- 172.21.15.16:0/154349942 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36680ce870 con 0x7f36680d1000 2023-05-20T01:24:53.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.990+0000 7f365effd700 1 -- 172.21.15.16:0/154349942 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f36600043e0 con 0x7f36680d1000 2023-05-20T01:24:53.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.990+0000 7f365effd700 1 --2- 172.21.15.16:0/154349942 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f364806ace0 0x7f364806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:53.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.990+0000 7f365effd700 1 -- 172.21.15.16:0/154349942 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7189+0+0 (secure 0 0 0) 0x7f3660022070 con 0x7f36680d1000 2023-05-20T01:24:53.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.990+0000 7f366659c700 1 --2- 172.21.15.16:0/154349942 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f364806ace0 0x7f364806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:53.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.991+0000 7f366659c700 1 --2- 172.21.15.16:0/154349942 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f364806ace0 0x7f364806d190 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f3650007fd0 tx=0x7f3650009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:24:54.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:53.995+0000 7f365effd700 1 -- 172.21.15.16:0/154349942 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f36600566b0 con 0x7f36680d1000 2023-05-20T01:24:54.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.236+0000 7f366ccf6700 1 -- 172.21.15.16:0/154349942 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f366806c4c0 con 0x7f36680d1000 2023-05-20T01:24:54.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.237+0000 7f365effd700 1 -- 172.21.15.16:0/154349942 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+4679 (secure 0 0 0) 0x7f3660059cd0 con 0x7f36680d1000 2023-05-20T01:24:54.241 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:24:54.241 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":49,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5873,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3769502841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3769502841},{"type":"v1","addr":"172.21.15.16:6835","nonce":3769502841}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45},{"gid":5879,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/688624373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":688624373},{"type":"v1","addr":"172.21.15.132:6834","nonce":688624373}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45},{"gid":6307,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/1655712081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":1655712081},{"type":"v1","addr":"172.21.15.132:6835","nonce":1655712081}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-20T01:22:14.127671+0000","modified":"2023-05-20T01:22:24.390190+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":72,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6768},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6768":{"gid":6768,"name":"c","rank":0,"incarnation":46,"state":"up:active","state_seq":4,"addr":"172.21.15.16:6837/2351975329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2351975329},{"type":"v1","addr":"172.21.15.16:6837","nonce":2351975329}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10,11],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-05-20T01:24:54.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.239+0000 7f366ccf6700 1 -- 172.21.15.16:0/154349942 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f364806ace0 msgr2=0x7f364806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:54.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.239+0000 7f366ccf6700 1 --2- 172.21.15.16:0/154349942 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f364806ace0 0x7f364806d190 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f3650007fd0 tx=0x7f3650009040 comp rx=0 tx=0).stop 2023-05-20T01:24:54.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.239+0000 7f366ccf6700 1 -- 172.21.15.16:0/154349942 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36680d1000 msgr2=0x7f366809afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:54.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.239+0000 7f366ccf6700 1 --2- 172.21.15.16:0/154349942 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36680d1000 0x7f366809afe0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f3660014040 tx=0x7f3660004ad0 comp rx=0 tx=0).stop 2023-05-20T01:24:54.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.239+0000 7f366ccf6700 1 -- 172.21.15.16:0/154349942 shutdown_connections 2023-05-20T01:24:54.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.239+0000 7f366ccf6700 1 --2- 172.21.15.16:0/154349942 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36680d1000 0x7f366809afe0 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:54.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.239+0000 7f366ccf6700 1 --2- 172.21.15.16:0/154349942 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f364806ace0 0x7f364806d190 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:54.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.239+0000 7f366ccf6700 1 --2- 172.21.15.16:0/154349942 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f366806b620 0x7f36680c6f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:54.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.239+0000 7f366ccf6700 1 --2- 172.21.15.16:0/154349942 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f366806ba90 0x7f366815b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:54.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.239+0000 7f366ccf6700 1 -- 172.21.15.16:0/154349942 >> 172.21.15.16:0/154349942 conn(0x7f366806ae30 msgr2=0x7f36680d13d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:24:54.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.239+0000 7f366ccf6700 1 -- 172.21.15.16:0/154349942 shutdown_connections 2023-05-20T01:24:54.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.239+0000 7f366ccf6700 1 -- 172.21.15.16:0/154349942 wait complete. 2023-05-20T01:24:54.245 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 49 2023-05-20T01:24:54.255 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-20T01:24:54.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.375+0000 7fedf87b8700 1 Processor -- start 2023-05-20T01:24:54.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.375+0000 7fedf87b8700 1 -- start start 2023-05-20T01:24:54.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.376+0000 7fedf87b8700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fedf015a9d0 0x7fedf015efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:54.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.376+0000 7fedf87b8700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fedf00b2250 0x7fedf006a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:54.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.376+0000 7fedf87b8700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedf006d940 con 0x7fedf006f850 2023-05-20T01:24:54.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.376+0000 7fedf87b8700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedf00cc8d0 con 0x7fedf00b2250 2023-05-20T01:24:54.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.376+0000 7fedf87b8700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedf015f500 con 0x7fedf015a9d0 2023-05-20T01:24:54.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.376+0000 7fedf5d53700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fedf00b2250 0x7fedf006a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:54.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.377+0000 7fedf5d53700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fedf00b2250 0x7fedf006a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35664/0 (socket says 172.21.15.16:35664) 2023-05-20T01:24:54.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.377+0000 7fedf5d53700 1 -- 172.21.15.16:0/3442713108 learned_addr learned my addr 172.21.15.16:0/3442713108 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:24:54.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.377+0000 7fedf6554700 1 --2- 172.21.15.16:0/3442713108 >> v2:172.21.15.132:3300/0 conn(0x7fedf015a9d0 0x7fedf015efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:54.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.377+0000 7fedf5d53700 1 -- 172.21.15.16:0/3442713108 >> v2:172.21.15.132:3300/0 conn(0x7fedf015a9d0 msgr2=0x7fedf015efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:24:54.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.377+0000 7fedf5d53700 1 --2- 172.21.15.16:0/3442713108 >> v2:172.21.15.132:3300/0 conn(0x7fedf015a9d0 0x7fedf015efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:54.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.377+0000 7fedf5d53700 1 -- 172.21.15.16:0/3442713108 >> v1:172.21.15.132:6789/0 conn(0x7fedf006f850 legacy=0x7fedf00710d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:24:54.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.377+0000 7fedf5d53700 1 -- 172.21.15.16:0/3442713108 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedf00cc030 con 0x7fedf00b2250 2023-05-20T01:24:54.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.377+0000 7fedf5d53700 1 --2- 172.21.15.16:0/3442713108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedf00b2250 0x7fedf006a510 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fedf0070fd0 tx=0x7fedec009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9367a93ea7944a4f server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:24:54.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.377+0000 7fedf5552700 1 -- 172.21.15.16:0/3442713108 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedec004230 con 0x7fedf00b2250 2023-05-20T01:24:54.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.377+0000 7fedf5552700 1 -- 172.21.15.16:0/3442713108 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedec004390 con 0x7fedf00b2250 2023-05-20T01:24:54.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.377+0000 7fedf5552700 1 -- 172.21.15.16:0/3442713108 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedec0036a0 con 0x7fedf00b2250 2023-05-20T01:24:54.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.378+0000 7fedf87b8700 1 -- 172.21.15.16:0/3442713108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedf00b2250 msgr2=0x7fedf006a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:54.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.378+0000 7fedf87b8700 1 --2- 172.21.15.16:0/3442713108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedf00b2250 0x7fedf006a510 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fedf0070fd0 tx=0x7fedec009bd0 comp rx=0 tx=0).stop 2023-05-20T01:24:54.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.378+0000 7fedf87b8700 1 -- 172.21.15.16:0/3442713108 shutdown_connections 2023-05-20T01:24:54.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.378+0000 7fedf87b8700 1 --2- 172.21.15.16:0/3442713108 >> v2:172.21.15.132:3300/0 conn(0x7fedf015a9d0 0x7fedf015efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:54.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.378+0000 7fedf87b8700 1 --2- 172.21.15.16:0/3442713108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedf00b2250 0x7fedf006a510 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:54.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.378+0000 7fedf87b8700 1 -- 172.21.15.16:0/3442713108 >> 172.21.15.16:0/3442713108 conn(0x7fedf00d0c30 msgr2=0x7fedf00b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:24:54.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.378+0000 7fedf87b8700 1 -- 172.21.15.16:0/3442713108 shutdown_connections 2023-05-20T01:24:54.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.378+0000 7fedf87b8700 1 -- 172.21.15.16:0/3442713108 wait complete. 2023-05-20T01:24:54.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.379+0000 7fedf87b8700 1 Processor -- start 2023-05-20T01:24:54.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.379+0000 7fedf87b8700 1 -- start start 2023-05-20T01:24:54.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.380+0000 7fedf87b8700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fedf006f850 0x7fedf0148c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:54.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.380+0000 7fedf87b8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedf00b2250 0x7fedf0145860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:54.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.380+0000 7fedf87b8700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fedf015a9d0 0x7fedf0158480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:54.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.380+0000 7fedf87b8700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedf00c80d0 con 0x7fedf00b2250 2023-05-20T01:24:54.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.380+0000 7fedf87b8700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedf0149180 con 0x7fedf006f850 2023-05-20T01:24:54.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.380+0000 7fedf87b8700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedf00c8320 con 0x7fedf015a9d0 2023-05-20T01:24:54.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.380+0000 7fedf5d53700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedf00b2250 0x7fedf0145860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:54.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.380+0000 7fedf5d53700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedf00b2250 0x7fedf0145860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35670/0 (socket says 172.21.15.16:35670) 2023-05-20T01:24:54.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.380+0000 7fedf5d53700 1 -- 172.21.15.16:0/2374513628 learned_addr learned my addr 172.21.15.16:0/2374513628 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:24:54.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.380+0000 7fedf6554700 1 --2- 172.21.15.16:0/2374513628 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fedf006f850 0x7fedf0148c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:54.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.380+0000 7fedf6d55700 1 --2- 172.21.15.16:0/2374513628 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fedf015a9d0 0x7fedf0158480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:54.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.380+0000 7fedf5d53700 1 -- 172.21.15.16:0/2374513628 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fedf015a9d0 msgr2=0x7fedf0158480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:54.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.380+0000 7fedf5d53700 1 --2- 172.21.15.16:0/2374513628 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fedf015a9d0 0x7fedf0158480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:54.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.380+0000 7fedf5d53700 1 -- 172.21.15.16:0/2374513628 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fedf006f850 msgr2=0x7fedf0148c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:54.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.380+0000 7fedf5d53700 1 --2- 172.21.15.16:0/2374513628 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fedf006f850 0x7fedf0148c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:54.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.380+0000 7fedf5d53700 1 -- 172.21.15.16:0/2374513628 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedec009550 con 0x7fedf00b2250 2023-05-20T01:24:54.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.380+0000 7fedf5d53700 1 --2- 172.21.15.16:0/2374513628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedf00b2250 0x7fedf0145860 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fedf0070d50 tx=0x7fedec004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:24:54.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.381+0000 7fede6ffd700 1 -- 172.21.15.16:0/2374513628 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedec003c10 con 0x7fedf00b2250 2023-05-20T01:24:54.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.381+0000 7fede6ffd700 1 -- 172.21.15.16:0/2374513628 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedec01f070 con 0x7fedf00b2250 2023-05-20T01:24:54.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.381+0000 7fedf87b8700 1 -- 172.21.15.16:0/2374513628 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedf00cc5f0 con 0x7fedf00b2250 2023-05-20T01:24:54.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.381+0000 7fede6ffd700 1 -- 172.21.15.16:0/2374513628 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedec011630 con 0x7fedf00b2250 2023-05-20T01:24:54.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.381+0000 7fedf87b8700 1 -- 172.21.15.16:0/2374513628 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedf00cc310 con 0x7fedf00b2250 2023-05-20T01:24:54.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.382+0000 7fedf87b8700 1 -- 172.21.15.16:0/2374513628 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedf015e7c0 con 0x7fedf00b2250 2023-05-20T01:24:54.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.382+0000 7fede6ffd700 1 -- 172.21.15.16:0/2374513628 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fedec021030 con 0x7fedf00b2250 2023-05-20T01:24:54.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.382+0000 7fede6ffd700 1 --2- 172.21.15.16:0/2374513628 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fedd406ace0 0x7fedd406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:54.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.383+0000 7fede6ffd700 1 -- 172.21.15.16:0/2374513628 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7189+0+0 (secure 0 0 0) 0x7fedec08ae40 con 0x7fedf00b2250 2023-05-20T01:24:54.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.383+0000 7fedf6554700 1 --2- 172.21.15.16:0/2374513628 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fedd406ace0 0x7fedd406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:54.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.383+0000 7fedf6554700 1 --2- 172.21.15.16:0/2374513628 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fedd406ace0 0x7fedd406d190 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fede0006b20 tx=0x7fede0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:24:54.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.387+0000 7fede6ffd700 1 -- 172.21.15.16:0/2374513628 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fedec055bb0 con 0x7fedf00b2250 2023-05-20T01:24:54.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.591+0000 7fedf87b8700 1 -- 172.21.15.16:0/2374513628 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fedf00747b0 con 0x7fedf00b2250 2023-05-20T01:24:54.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.593+0000 7fede6ffd700 1 -- 172.21.15.16:0/2374513628 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) v1 ==== 74+0+20822 (secure 0 0 0) 0x7fedec0591d0 con 0x7fedf00b2250 2023-05-20T01:24:54.597 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:24:54.598 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":75,"fsid":"c2245cc5-ecab-4087-a59b-bc81db901676","created":"2023-05-20T01:16:09.549230+0000","modified":"2023-05-20T01:22:20.570959+0000","last_up_change":"2023-05-20T01:16:19.788624+0000","last_in_change":"2023-05-20T01:16:12.092160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-05-20T01:16:20.934366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-05-20T01:16:21.372294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-05-20T01:19:49.878849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-05-20T01:19:50.887306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data_extra","create_time":"2023-05-20T01:20:04.581773+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"317a83cd-19c8-4958-a301-6087ef342289","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6825","nonce":3923634179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6827","nonce":3923634179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6831","nonce":3923634179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6829","nonce":3923634179}]},"public_addr":"172.21.15.16:6825/3923634179","cluster_addr":"172.21.15.16:6827/3923634179","heartbeat_back_addr":"172.21.15.16:6831/3923634179","heartbeat_front_addr":"172.21.15.16:6829/3923634179","state":["exists","up"]},{"osd":1,"uuid":"5581046f-8219-4089-8d7d-b7846399505e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6809","nonce":3860264220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6811","nonce":3860264220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6815","nonce":3860264220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6813","nonce":3860264220}]},"public_addr":"172.21.15.16:6809/3860264220","cluster_addr":"172.21.15.16:6811/3860264220","heartbeat_back_addr":"172.21.15.16:6815/3860264220","heartbeat_front_addr":"172.21.15.16:6813/3860264220","state":["exists","up"]},{"osd":2,"uuid":"072e9e66-4133-478e-8d4a-64d46c2963f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6817","nonce":293187030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6819","nonce":293187030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6823","nonce":293187030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6821","nonce":293187030}]},"public_addr":"172.21.15.16:6817/293187030","cluster_addr":"172.21.15.16:6819/293187030","heartbeat_back_addr":"172.21.15.16:6823/293187030","heartbeat_front_addr":"172.21.15.16:6821/293187030","state":["exists","up"]},{"osd":3,"uuid":"11e1bba6-00f8-48dd-9028-d8d716b7f145","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6801","nonce":3669781890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6803","nonce":3669781890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6807","nonce":3669781890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6805","nonce":3669781890}]},"public_addr":"172.21.15.16:6801/3669781890","cluster_addr":"172.21.15.16:6803/3669781890","heartbeat_back_addr":"172.21.15.16:6807/3669781890","heartbeat_front_addr":"172.21.15.16:6805/3669781890","state":["exists","up"]},{"osd":4,"uuid":"b8909c3a-cf70-46cb-8d0b-009c707cf9d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6819","nonce":506317223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6823","nonce":506317223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6829","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6831","nonce":506317223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6827","nonce":506317223}]},"public_addr":"172.21.15.132:6819/506317223","cluster_addr":"172.21.15.132:6823/506317223","heartbeat_back_addr":"172.21.15.132:6831/506317223","heartbeat_front_addr":"172.21.15.132:6827/506317223","state":["exists","up"]},{"osd":5,"uuid":"b1818304-4583-47bf-a6ec-0bfbb5166233","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6818","nonce":1844689995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6822","nonce":1844689995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6830","nonce":1844689995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6826","nonce":1844689995}]},"public_addr":"172.21.15.132:6818/1844689995","cluster_addr":"172.21.15.132:6822/1844689995","heartbeat_back_addr":"172.21.15.132:6830/1844689995","heartbeat_front_addr":"172.21.15.132:6826/1844689995","state":["exists","up"]},{"osd":6,"uuid":"d37f4691-eed1-4a93-8818-18354f03f4a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6801","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6802","nonce":727581123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6805","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6807","nonce":727581123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6813","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6815","nonce":727581123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6811","nonce":727581123}]},"public_addr":"172.21.15.132:6802/727581123","cluster_addr":"172.21.15.132:6807/727581123","heartbeat_back_addr":"172.21.15.132:6815/727581123","heartbeat_front_addr":"172.21.15.132:6811/727581123","state":["exists","up"]},{"osd":7,"uuid":"4f712f62-5d1d-4816-8042-03209144f8b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6803","nonce":466001149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6806","nonce":466001149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6814","nonce":466001149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6810","nonce":466001149}]},"public_addr":"172.21.15.132:6803/466001149","cluster_addr":"172.21.15.132:6806/466001149","heartbeat_back_addr":"172.21.15.132:6814/466001149","heartbeat_front_addr":"172.21.15.132:6810/466001149","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.338863+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.278680+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305694+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.246835+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.276237+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.304441+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305421+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.267050+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.d","mappings":[{"from":7,"to":2}]},{"pgid":"10.10","mappings":[{"from":4,"to":3}]},{"pgid":"10.2e","mappings":[{"from":4,"to":3}]},{"pgid":"10.34","mappings":[{"from":7,"to":3}]},{"pgid":"10.37","mappings":[{"from":4,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6837/3291457857":"2023-05-21T01:22:20.558472+0000","172.21.15.16:6835/3291457857":"2023-05-21T01:22:20.558472+0000","172.21.15.16:6837/1427887654":"2023-05-21T01:22:14.372197+0000","172.21.15.16:6835/1427887654":"2023-05-21T01:22:14.372197+0000","172.21.15.16:6837/1427908556":"2023-05-21T01:22:12.737196+0000","172.21.15.16:6835/1427908556":"2023-05-21T01:22:12.737196+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-20T01:24:54.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.595+0000 7fedf87b8700 1 -- 172.21.15.16:0/2374513628 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fedd406ace0 msgr2=0x7fedd406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:54.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.595+0000 7fedf87b8700 1 --2- 172.21.15.16:0/2374513628 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fedd406ace0 0x7fedd406d190 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fede0006b20 tx=0x7fede0008040 comp rx=0 tx=0).stop 2023-05-20T01:24:54.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.595+0000 7fedf87b8700 1 -- 172.21.15.16:0/2374513628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedf00b2250 msgr2=0x7fedf0145860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:54.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.595+0000 7fedf87b8700 1 --2- 172.21.15.16:0/2374513628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedf00b2250 0x7fedf0145860 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fedf0070d50 tx=0x7fedec004750 comp rx=0 tx=0).stop 2023-05-20T01:24:54.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.595+0000 7fedf87b8700 1 -- 172.21.15.16:0/2374513628 shutdown_connections 2023-05-20T01:24:54.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.595+0000 7fedf87b8700 1 --2- 172.21.15.16:0/2374513628 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fedd406ace0 0x7fedd406d190 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:54.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.595+0000 7fedf87b8700 1 --2- 172.21.15.16:0/2374513628 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fedf015a9d0 0x7fedf0158480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:54.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.595+0000 7fedf87b8700 1 --2- 172.21.15.16:0/2374513628 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fedf006f850 0x7fedf0148c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:54.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.595+0000 7fedf87b8700 1 --2- 172.21.15.16:0/2374513628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedf00b2250 0x7fedf0145860 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:54.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.595+0000 7fedf87b8700 1 -- 172.21.15.16:0/2374513628 >> 172.21.15.16:0/2374513628 conn(0x7fedf00d0c30 msgr2=0x7fedf00710d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:24:54.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.595+0000 7fedf87b8700 1 -- 172.21.15.16:0/2374513628 shutdown_connections 2023-05-20T01:24:54.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.595+0000 7fedf87b8700 1 -- 172.21.15.16:0/2374513628 wait complete. 2023-05-20T01:24:54.612 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-05-20T01:16:20.934366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-05-20T01:16:21.372294+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2023-05-20T01:19:49.878849+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '63', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2023-05-20T01:19:50.887306+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '64', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data_extra', 'create_time': '2023-05-20T01:20:04.581773+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '67', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-05-20T01:24:54.613 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-05-20T01:24:54.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.737+0000 7f20c5d32700 1 Processor -- start 2023-05-20T01:24:54.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.737+0000 7f20c5d32700 1 -- start start 2023-05-20T01:24:54.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.738+0000 7f20c5d32700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f20c0158f90 0x7f20c00cb7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:54.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.738+0000 7f20c5d32700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20c0159360 con 0x7f20c00cc760 2023-05-20T01:24:54.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.738+0000 7f20c5d32700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20c00ccba0 con 0x7f20c00ce0a0 2023-05-20T01:24:54.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.738+0000 7f20c5d32700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20c00cce10 con 0x7f20c0158f90 2023-05-20T01:24:54.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.738+0000 7f20bffff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f20c00cc760 0x7f20c0070930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:47208/0 (socket says 172.21.15.16:47208) 2023-05-20T01:24:54.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.738+0000 7f20bffff700 1 -- 172.21.15.16:0/3581294122 learned_addr learned my addr 172.21.15.16:0/3581294122 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:24:54.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.738+0000 7f20bf7fe700 1 --2- 172.21.15.16:0/3581294122 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f20c0158f90 0x7f20c00cb7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:54.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.738+0000 7f20be7fc700 1 -- 172.21.15.16:0/3581294122 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2902415755 0 0) 0x7f20b8001490 con 0x7f20c00cc760 2023-05-20T01:24:54.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.738+0000 7f20be7fc700 1 -- 172.21.15.16:0/3581294122 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20ac0035b0 con 0x7f20c00cc760 2023-05-20T01:24:54.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.739+0000 7f20be7fc700 1 -- 172.21.15.16:0/3581294122 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 101322758 0 0) 0x7f20ac0035b0 con 0x7f20c00cc760 2023-05-20T01:24:54.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.739+0000 7f20be7fc700 1 -- 172.21.15.16:0/3581294122 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f20c0158f90 msgr2=0x7f20c00cb7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:24:54.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.739+0000 7f20be7fc700 1 --2- 172.21.15.16:0/3581294122 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f20c0158f90 0x7f20c00cb7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:54.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.739+0000 7f20be7fc700 1 -- 172.21.15.16:0/3581294122 >> v1:172.21.15.132:6789/0 conn(0x7f20c00ce0a0 legacy=0x7f20c00cc050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:54.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.739+0000 7f20be7fc700 1 -- 172.21.15.16:0/3581294122 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20c00ca750 con 0x7f20c00cc760 2023-05-20T01:24:54.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.739+0000 7f20be7fc700 1 -- 172.21.15.16:0/3581294122 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7f20b8001b80 con 0x7f20c00cc760 2023-05-20T01:24:54.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.739+0000 7f20be7fc700 1 -- 172.21.15.16:0/3581294122 >> v1:172.21.15.16:6789/0 conn(0x7f20c00cc760 legacy=0x7f20c0070930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:54.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.739+0000 7f20be7fc700 1 --2- 172.21.15.16:0/3581294122 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f20c0158f90 0x7f20ac003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:54.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.739+0000 7f20be7fc700 1 --2- 172.21.15.16:0/3581294122 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f20ac003fb0 0x7f20ac004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:54.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.739+0000 7f20be7fc700 1 --2- 172.21.15.16:0/3581294122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20ac004d90 0x7f20ac005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:54.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.739+0000 7f20be7fc700 1 -- 172.21.15.16:0/3581294122 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20ac00bac0 con 0x7f20ac004d90 2023-05-20T01:24:54.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.739+0000 7f20be7fc700 1 -- 172.21.15.16:0/3581294122 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20ac00bc30 con 0x7f20ac003fb0 2023-05-20T01:24:54.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.739+0000 7f20be7fc700 1 -- 172.21.15.16:0/3581294122 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20ac00cf00 con 0x7f20c0158f90 2023-05-20T01:24:54.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.739+0000 7f20bffff700 1 --2- 172.21.15.16:0/3581294122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20ac004d90 0x7f20ac005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:54.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.739+0000 7f20bf7fe700 1 --2- 172.21.15.16:0/3581294122 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f20c0158f90 0x7f20ac003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:54.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.739+0000 7f20beffd700 1 --2- 172.21.15.16:0/3581294122 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f20ac003fb0 0x7f20ac004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:54.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.739+0000 7f20bffff700 1 -- 172.21.15.16:0/3581294122 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f20c0158f90 msgr2=0x7f20ac003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:24:54.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.739+0000 7f20bffff700 1 --2- 172.21.15.16:0/3581294122 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f20c0158f90 0x7f20ac003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:54.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.739+0000 7f20bffff700 1 -- 172.21.15.16:0/3581294122 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f20ac003fb0 msgr2=0x7f20ac004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:24:54.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.739+0000 7f20bffff700 1 --2- 172.21.15.16:0/3581294122 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f20ac003fb0 0x7f20ac004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:54.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.739+0000 7f20bffff700 1 -- 172.21.15.16:0/3581294122 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20ac0035b0 con 0x7f20ac004d90 2023-05-20T01:24:54.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.739+0000 7f20bffff700 1 --2- 172.21.15.16:0/3581294122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20ac004d90 0x7f20ac005200 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f20b800a7d0 tx=0x7f20ac005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba1648b731ac068 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:24:54.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.739+0000 7f20be7fc700 1 -- 172.21.15.16:0/3581294122 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20b800a460 con 0x7f20ac004d90 2023-05-20T01:24:54.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.739+0000 7f20be7fc700 1 -- 172.21.15.16:0/3581294122 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20b8012600 con 0x7f20ac004d90 2023-05-20T01:24:54.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.739+0000 7f20be7fc700 1 -- 172.21.15.16:0/3581294122 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20b80129b0 con 0x7f20ac004d90 2023-05-20T01:24:54.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.740+0000 7f20c5d32700 1 -- 172.21.15.16:0/3581294122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20ac004d90 msgr2=0x7f20ac005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:54.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.740+0000 7f20c5d32700 1 --2- 172.21.15.16:0/3581294122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20ac004d90 0x7f20ac005200 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f20b800a7d0 tx=0x7f20ac005880 comp rx=0 tx=0).stop 2023-05-20T01:24:54.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.740+0000 7f20bffff700 1 -- 172.21.15.16:0/3581294122 reap_dead start 2023-05-20T01:24:54.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.740+0000 7f20c5d32700 1 -- 172.21.15.16:0/3581294122 shutdown_connections 2023-05-20T01:24:54.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.740+0000 7f20c5d32700 1 -- 172.21.15.16:0/3581294122 >> 172.21.15.16:0/3581294122 conn(0x7f20c006cbb0 msgr2=0x7f20c006cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:24:54.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.740+0000 7f20c5d32700 1 -- 172.21.15.16:0/3581294122 shutdown_connections 2023-05-20T01:24:54.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.740+0000 7f20c5d32700 1 -- 172.21.15.16:0/3581294122 wait complete. 2023-05-20T01:24:54.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.742+0000 7f20c5d32700 1 Processor -- start 2023-05-20T01:24:54.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.742+0000 7f20c5d32700 1 -- start start 2023-05-20T01:24:54.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.743+0000 7f20c5d32700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f20c00c9090 0x7f20c01515f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:54.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.743+0000 7f20c5d32700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f20c015beb0 0x7f20c006cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:54.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.743+0000 7f20c5d32700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20c006d4c0 0x7f20c00b30c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:54.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.743+0000 7f20c5d32700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20c0152490 con 0x7f20c006d4c0 2023-05-20T01:24:54.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.743+0000 7f20c5d32700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20c006d930 con 0x7f20c00c9090 2023-05-20T01:24:54.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.743+0000 7f20c5d32700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20c00b2c30 con 0x7f20c015beb0 2023-05-20T01:24:54.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.743+0000 7f20bffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20c006d4c0 0x7f20c00b30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:54.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.743+0000 7f20bffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20c006d4c0 0x7f20c00b30c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35690/0 (socket says 172.21.15.16:35690) 2023-05-20T01:24:54.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.743+0000 7f20bffff700 1 -- 172.21.15.16:0/3425040580 learned_addr learned my addr 172.21.15.16:0/3425040580 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:24:54.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.743+0000 7f20bf7fe700 1 --2- 172.21.15.16:0/3425040580 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f20c00c9090 0x7f20c01515f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:54.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.743+0000 7f20beffd700 1 --2- 172.21.15.16:0/3425040580 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f20c015beb0 0x7f20c006cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:54.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.743+0000 7f20bffff700 1 -- 172.21.15.16:0/3425040580 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f20c015beb0 msgr2=0x7f20c006cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:54.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.743+0000 7f20bffff700 1 --2- 172.21.15.16:0/3425040580 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f20c015beb0 0x7f20c006cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:54.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.743+0000 7f20bffff700 1 -- 172.21.15.16:0/3425040580 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f20c00c9090 msgr2=0x7f20c01515f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:54.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.743+0000 7f20bffff700 1 --2- 172.21.15.16:0/3425040580 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f20c00c9090 0x7f20c01515f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:54.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.743+0000 7f20bffff700 1 -- 172.21.15.16:0/3425040580 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20c00cacd0 con 0x7f20c006d4c0 2023-05-20T01:24:54.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.743+0000 7f20bffff700 1 --2- 172.21.15.16:0/3425040580 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20c006d4c0 0x7f20c00b30c0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f20b8012000 tx=0x7f20c00ca8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:24:54.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.744+0000 7f20b7fff700 1 -- 172.21.15.16:0/3425040580 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20b800a460 con 0x7f20c006d4c0 2023-05-20T01:24:54.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.744+0000 7f20b7fff700 1 -- 172.21.15.16:0/3425040580 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20b8001490 con 0x7f20c006d4c0 2023-05-20T01:24:54.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.744+0000 7f20c5d32700 1 -- 172.21.15.16:0/3425040580 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20c00ca9f0 con 0x7f20c006d4c0 2023-05-20T01:24:54.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.744+0000 7f20b7fff700 1 -- 172.21.15.16:0/3425040580 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20b80011c0 con 0x7f20c006d4c0 2023-05-20T01:24:54.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.744+0000 7f20c5d32700 1 -- 172.21.15.16:0/3425040580 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20c00cafb0 con 0x7f20c006d4c0 2023-05-20T01:24:54.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.745+0000 7f20c5d32700 1 -- 172.21.15.16:0/3425040580 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20c00747b0 con 0x7f20c006d4c0 2023-05-20T01:24:54.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.745+0000 7f20b7fff700 1 -- 172.21.15.16:0/3425040580 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f20b8019070 con 0x7f20c006d4c0 2023-05-20T01:24:54.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.746+0000 7f20b7fff700 1 --2- 172.21.15.16:0/3425040580 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f20a006ace0 0x7f20a006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:54.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.746+0000 7f20b7fff700 1 -- 172.21.15.16:0/3425040580 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7189+0+0 (secure 0 0 0) 0x7f20b808a720 con 0x7f20c006d4c0 2023-05-20T01:24:54.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.746+0000 7f20bf7fe700 1 --2- 172.21.15.16:0/3425040580 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f20a006ace0 0x7f20a006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:54.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.746+0000 7f20bf7fe700 1 --2- 172.21.15.16:0/3425040580 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f20a006ace0 0x7f20a006d190 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f20b0009ba0 tx=0x7f20b0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:24:54.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.750+0000 7f20b7fff700 1 -- 172.21.15.16:0/3425040580 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f20b8055490 con 0x7f20c006d4c0 2023-05-20T01:24:54.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.947+0000 7f20c5d32700 1 -- 172.21.15.16:0/3425040580 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f20c0159360 con 0x7f20c006d4c0 2023-05-20T01:24:54.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.948+0000 7f20b7fff700 1 -- 172.21.15.16:0/3425040580 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v75) v1 ==== 92+0+10 (secure 0 0 0) 0x7f20b8017020 con 0x7f20c006d4c0 2023-05-20T01:24:54.952 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 8 2023-05-20T01:24:54.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.949+0000 7f20c5d32700 1 -- 172.21.15.16:0/3425040580 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f20a006ace0 msgr2=0x7f20a006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:54.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.949+0000 7f20c5d32700 1 --2- 172.21.15.16:0/3425040580 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f20a006ace0 0x7f20a006d190 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f20b0009ba0 tx=0x7f20b0008040 comp rx=0 tx=0).stop 2023-05-20T01:24:54.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.949+0000 7f20c5d32700 1 -- 172.21.15.16:0/3425040580 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20c006d4c0 msgr2=0x7f20c00b30c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:54.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.949+0000 7f20c5d32700 1 --2- 172.21.15.16:0/3425040580 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20c006d4c0 0x7f20c00b30c0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f20b8012000 tx=0x7f20c00ca8e0 comp rx=0 tx=0).stop 2023-05-20T01:24:54.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.949+0000 7f20c5d32700 1 -- 172.21.15.16:0/3425040580 shutdown_connections 2023-05-20T01:24:54.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.949+0000 7f20c5d32700 1 --2- 172.21.15.16:0/3425040580 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20c006d4c0 0x7f20c00b30c0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:54.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.949+0000 7f20c5d32700 1 --2- 172.21.15.16:0/3425040580 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f20c00c9090 0x7f20c01515f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:54.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.949+0000 7f20c5d32700 1 --2- 172.21.15.16:0/3425040580 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f20a006ace0 0x7f20a006d190 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:54.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.949+0000 7f20c5d32700 1 --2- 172.21.15.16:0/3425040580 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f20c015beb0 0x7f20c006cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:54.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.949+0000 7f20c5d32700 1 -- 172.21.15.16:0/3425040580 >> 172.21.15.16:0/3425040580 conn(0x7f20c006cbb0 msgr2=0x7f20c00c4720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:24:54.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.950+0000 7f20c5d32700 1 -- 172.21.15.16:0/3425040580 shutdown_connections 2023-05-20T01:24:54.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:54.950+0000 7f20c5d32700 1 -- 172.21.15.16:0/3425040580 wait complete. 2023-05-20T01:24:54.966 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-05-20T01:24:55.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.091+0000 7f9605031700 1 Processor -- start 2023-05-20T01:24:55.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.091+0000 7f9605031700 1 -- start start 2023-05-20T01:24:55.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.091+0000 7f9605031700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f960015cec0 0x7f9600071bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:55.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.091+0000 7f9605031700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f960015d290 0x7f9600072130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:55.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.091+0000 7f9605031700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f96000cf2c0 0x7f96000cf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:55.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.091+0000 7f9605031700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96000cd630 con 0x7f960015cec0 2023-05-20T01:24:55.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.091+0000 7f9605031700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96000cca60 con 0x7f96000cf2c0 2023-05-20T01:24:55.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.091+0000 7f9605031700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96000ccd00 con 0x7f960015d290 2023-05-20T01:24:55.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.092+0000 7f95fed9d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f960015cec0 0x7f9600071bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:55.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.092+0000 7f95fed9d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f960015cec0 0x7f9600071bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35696/0 (socket says 172.21.15.16:35696) 2023-05-20T01:24:55.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.092+0000 7f95fed9d700 1 -- 172.21.15.16:0/2305970544 learned_addr learned my addr 172.21.15.16:0/2305970544 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:24:55.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.092+0000 7f95fe59c700 1 --2- 172.21.15.16:0/2305970544 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f960015d290 0x7f9600072130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:55.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.092+0000 7f95ff59e700 1 --2- 172.21.15.16:0/2305970544 >> v2:172.21.15.132:3300/0 conn(0x7f96000cf2c0 0x7f96000cf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:55.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.092+0000 7f95fed9d700 1 -- 172.21.15.16:0/2305970544 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f960015d290 msgr2=0x7f9600072130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:24:55.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.092+0000 7f95fed9d700 1 --2- 172.21.15.16:0/2305970544 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f960015d290 0x7f9600072130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:55.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.092+0000 7f95fed9d700 1 -- 172.21.15.16:0/2305970544 >> v2:172.21.15.132:3300/0 conn(0x7f96000cf2c0 msgr2=0x7f96000cf720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:24:55.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.092+0000 7f95fed9d700 1 --2- 172.21.15.16:0/2305970544 >> v2:172.21.15.132:3300/0 conn(0x7f96000cf2c0 0x7f96000cf720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:55.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.092+0000 7f95fed9d700 1 -- 172.21.15.16:0/2305970544 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96000ccfe0 con 0x7f960015cec0 2023-05-20T01:24:55.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.092+0000 7f95ff59e700 1 --2- 172.21.15.16:0/2305970544 >> v2:172.21.15.132:3300/0 conn(0x7f96000cf2c0 0x7f96000cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:24:55.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.092+0000 7f95fed9d700 1 --2- 172.21.15.16:0/2305970544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f960015cec0 0x7f9600071bf0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f9600071140 tx=0x7f95f0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d348382507c3805 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:24:55.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.092+0000 7f95fdd9b700 1 -- 172.21.15.16:0/2305970544 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f0005e30 con 0x7f960015cec0 2023-05-20T01:24:55.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.092+0000 7f95fdd9b700 1 -- 172.21.15.16:0/2305970544 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95f0004030 con 0x7f960015cec0 2023-05-20T01:24:55.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.092+0000 7f95fdd9b700 1 -- 172.21.15.16:0/2305970544 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f00043e0 con 0x7f960015cec0 2023-05-20T01:24:55.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.093+0000 7f9605031700 1 -- 172.21.15.16:0/2305970544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f960015cec0 msgr2=0x7f9600071bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:55.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.093+0000 7f9605031700 1 --2- 172.21.15.16:0/2305970544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f960015cec0 0x7f9600071bf0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f9600071140 tx=0x7f95f0009bd0 comp rx=0 tx=0).stop 2023-05-20T01:24:55.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.093+0000 7f9605031700 1 -- 172.21.15.16:0/2305970544 shutdown_connections 2023-05-20T01:24:55.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.093+0000 7f9605031700 1 --2- 172.21.15.16:0/2305970544 >> v2:172.21.15.132:3300/0 conn(0x7f96000cf2c0 0x7f96000cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:55.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.093+0000 7f9605031700 1 --2- 172.21.15.16:0/2305970544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f960015cec0 0x7f9600071bf0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:55.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.093+0000 7f9605031700 1 --2- 172.21.15.16:0/2305970544 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f960015d290 0x7f9600072130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:55.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.093+0000 7f9605031700 1 -- 172.21.15.16:0/2305970544 >> 172.21.15.16:0/2305970544 conn(0x7f96000cffd0 msgr2=0x7f96000d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:24:55.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.093+0000 7f9605031700 1 -- 172.21.15.16:0/2305970544 shutdown_connections 2023-05-20T01:24:55.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.093+0000 7f9605031700 1 -- 172.21.15.16:0/2305970544 wait complete. 2023-05-20T01:24:55.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.095+0000 7f9605031700 1 Processor -- start 2023-05-20T01:24:55.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.095+0000 7f9605031700 1 -- start start 2023-05-20T01:24:55.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.095+0000 7f9605031700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f96000cf2c0 0x7f96000c9570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:55.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.095+0000 7f9605031700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f960015cec0 0x7f96001565c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:55.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.095+0000 7f9605031700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f960015d290 0x7f9600156b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:55.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.095+0000 7f9605031700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96000c9ab0 con 0x7f960015d290 2023-05-20T01:24:55.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.095+0000 7f9605031700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96000c90e0 con 0x7f96000cf2c0 2023-05-20T01:24:55.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.095+0000 7f9605031700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96000c84d0 con 0x7f960015cec0 2023-05-20T01:24:55.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.095+0000 7f95ff59e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f960015d290 0x7f9600156b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:55.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.095+0000 7f95ff59e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f960015d290 0x7f9600156b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35706/0 (socket says 172.21.15.16:35706) 2023-05-20T01:24:55.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.095+0000 7f95ff59e700 1 -- 172.21.15.16:0/1298282895 learned_addr learned my addr 172.21.15.16:0/1298282895 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:24:55.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.096+0000 7f95fed9d700 1 --2- 172.21.15.16:0/1298282895 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f96000cf2c0 0x7f96000c9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:55.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.096+0000 7f95fe59c700 1 --2- 172.21.15.16:0/1298282895 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f960015cec0 0x7f96001565c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:55.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.096+0000 7f95ff59e700 1 -- 172.21.15.16:0/1298282895 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f960015cec0 msgr2=0x7f96001565c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:55.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.096+0000 7f95ff59e700 1 --2- 172.21.15.16:0/1298282895 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f960015cec0 0x7f96001565c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:55.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.096+0000 7f95ff59e700 1 -- 172.21.15.16:0/1298282895 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f96000cf2c0 msgr2=0x7f96000c9570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:55.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.096+0000 7f95ff59e700 1 --2- 172.21.15.16:0/1298282895 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f96000cf2c0 0x7f96000c9570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:55.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.096+0000 7f95ff59e700 1 -- 172.21.15.16:0/1298282895 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95f0009500 con 0x7f960015d290 2023-05-20T01:24:55.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.096+0000 7f95fed9d700 1 --2- 172.21.15.16:0/1298282895 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f96000cf2c0 0x7f96000c9570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:24:55.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.096+0000 7f95ff59e700 1 --2- 172.21.15.16:0/1298282895 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f960015d290 0x7f9600156b00 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f95f800cd40 tx=0x7f95f800a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:24:55.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.096+0000 7f95f77fe700 1 -- 172.21.15.16:0/1298282895 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f800f480 con 0x7f960015d290 2023-05-20T01:24:55.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.096+0000 7f95f77fe700 1 -- 172.21.15.16:0/1298282895 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95f800f600 con 0x7f960015d290 2023-05-20T01:24:55.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.096+0000 7f9605031700 1 -- 172.21.15.16:0/1298282895 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96000cfc60 con 0x7f960015d290 2023-05-20T01:24:55.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.096+0000 7f95f77fe700 1 -- 172.21.15.16:0/1298282895 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f800f9b0 con 0x7f960015d290 2023-05-20T01:24:55.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.096+0000 7f9605031700 1 -- 172.21.15.16:0/1298282895 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96000cd2c0 con 0x7f960015d290 2023-05-20T01:24:55.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.097+0000 7f9605031700 1 -- 172.21.15.16:0/1298282895 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96000cfdf0 con 0x7f960015d290 2023-05-20T01:24:55.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.098+0000 7f95f77fe700 1 -- 172.21.15.16:0/1298282895 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f95f800b040 con 0x7f960015d290 2023-05-20T01:24:55.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.098+0000 7f95f77fe700 1 --2- 172.21.15.16:0/1298282895 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f95e006ace0 0x7f95e006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:55.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.098+0000 7f95f77fe700 1 -- 172.21.15.16:0/1298282895 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7189+0+0 (secure 0 0 0) 0x7f95f808a030 con 0x7f960015d290 2023-05-20T01:24:55.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.098+0000 7f95fed9d700 1 --2- 172.21.15.16:0/1298282895 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f95e006ace0 0x7f95e006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:55.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.099+0000 7f95fed9d700 1 --2- 172.21.15.16:0/1298282895 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f95e006ace0 0x7f95e006d190 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f9600071140 tx=0x7f95f0008070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:24:55.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.103+0000 7f95f77fe700 1 -- 172.21.15.16:0/1298282895 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f95f8054d10 con 0x7f960015d290 2023-05-20T01:24:55.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.305+0000 7f9605031700 1 -- 172.21.15.16:0/1298282895 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9600071270 con 0x7f960015d290 2023-05-20T01:24:55.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.305+0000 7f95f77fe700 1 -- 172.21.15.16:0/1298282895 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v75) v1 ==== 93+0+10 (secure 0 0 0) 0x7f95f801d020 con 0x7f960015d290 2023-05-20T01:24:55.310 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-05-20T01:24:55.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.307+0000 7f9605031700 1 -- 172.21.15.16:0/1298282895 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f95e006ace0 msgr2=0x7f95e006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:55.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.307+0000 7f9605031700 1 --2- 172.21.15.16:0/1298282895 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f95e006ace0 0x7f95e006d190 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f9600071140 tx=0x7f95f0008070 comp rx=0 tx=0).stop 2023-05-20T01:24:55.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.307+0000 7f9605031700 1 -- 172.21.15.16:0/1298282895 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f960015d290 msgr2=0x7f9600156b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:55.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.307+0000 7f9605031700 1 --2- 172.21.15.16:0/1298282895 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f960015d290 0x7f9600156b00 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f95f800cd40 tx=0x7f95f800a560 comp rx=0 tx=0).stop 2023-05-20T01:24:55.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.307+0000 7f9605031700 1 -- 172.21.15.16:0/1298282895 shutdown_connections 2023-05-20T01:24:55.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.307+0000 7f9605031700 1 --2- 172.21.15.16:0/1298282895 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f960015d290 0x7f9600156b00 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:55.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.307+0000 7f9605031700 1 --2- 172.21.15.16:0/1298282895 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f96000cf2c0 0x7f96000c9570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:55.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.307+0000 7f9605031700 1 --2- 172.21.15.16:0/1298282895 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f95e006ace0 0x7f95e006d190 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:55.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.307+0000 7f9605031700 1 --2- 172.21.15.16:0/1298282895 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f960015cec0 0x7f96001565c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:55.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.307+0000 7f9605031700 1 -- 172.21.15.16:0/1298282895 >> 172.21.15.16:0/1298282895 conn(0x7f96000cffd0 msgr2=0x7f96000cbd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:24:55.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.307+0000 7f9605031700 1 -- 172.21.15.16:0/1298282895 shutdown_connections 2023-05-20T01:24:55.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.307+0000 7f9605031700 1 -- 172.21.15.16:0/1298282895 wait complete. 2023-05-20T01:24:55.324 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-05-20T01:24:55.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.450+0000 7f1da1cf2700 1 Processor -- start 2023-05-20T01:24:55.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.450+0000 7f1da1cf2700 1 -- start start 2023-05-20T01:24:55.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.451+0000 7f1da1cf2700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f1d9c15ca90 0x7f1d9c0cd890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:55.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.451+0000 7f1da1cf2700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1d9c0d29a0 0x7f1d9c0cf570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:55.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.451+0000 7f1da1cf2700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d9c0d1820 con 0x7f1d9c0d2550 2023-05-20T01:24:55.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.451+0000 7f1da1cf2700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d9c0d0c50 con 0x7f1d9c15ca90 2023-05-20T01:24:55.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.451+0000 7f1da1cf2700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d9c0cdf00 con 0x7f1d9c0d29a0 2023-05-20T01:24:55.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.451+0000 7f1d9affd700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f1d9c0d2550 0x7f1d9c0b6090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:47238/0 (socket says 172.21.15.16:47238) 2023-05-20T01:24:55.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.451+0000 7f1d9affd700 1 -- 172.21.15.16:0/2771188442 learned_addr learned my addr 172.21.15.16:0/2771188442 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:24:55.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.451+0000 7f1d9b7fe700 1 --2- 172.21.15.16:0/2771188442 >> v2:172.21.15.132:3300/0 conn(0x7f1d9c15ca90 0x7f1d9c0cd890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:55.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.451+0000 7f1d9bfff700 1 --2- 172.21.15.16:0/2771188442 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1d9c0d29a0 0x7f1d9c0cf570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:55.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.452+0000 7f1d9a7fc700 1 -- 172.21.15.16:0/2771188442 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2199744680 0 0) 0x7f1d9c0d1820 con 0x7f1d9c0d2550 2023-05-20T01:24:55.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.452+0000 7f1d9a7fc700 1 -- 172.21.15.16:0/2771188442 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d840035b0 con 0x7f1d9c0d2550 2023-05-20T01:24:55.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.452+0000 7f1d9b7fe700 1 -- 172.21.15.16:0/2771188442 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1d9c0d29a0 msgr2=0x7f1d9c0cf570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:24:55.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.452+0000 7f1d9b7fe700 1 --2- 172.21.15.16:0/2771188442 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1d9c0d29a0 0x7f1d9c0cf570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:55.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.452+0000 7f1d9b7fe700 1 -- 172.21.15.16:0/2771188442 >> v1:172.21.15.16:6789/0 conn(0x7f1d9c0d2550 legacy=0x7f1d9c0b6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:55.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.452+0000 7f1d9b7fe700 1 -- 172.21.15.16:0/2771188442 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d9c0d1a70 con 0x7f1d9c15ca90 2023-05-20T01:24:55.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.452+0000 7f1d9b7fe700 1 --2- 172.21.15.16:0/2771188442 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d9c15ca90 0x7f1d9c0cd890 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f1d8c009840 tx=0x7f1d8c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=140ee4d1eb65f0ee server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:24:55.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.452+0000 7f1d9a7fc700 1 -- 172.21.15.16:0/2771188442 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d8c011830 con 0x7f1d9c15ca90 2023-05-20T01:24:55.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.452+0000 7f1d9a7fc700 1 -- 172.21.15.16:0/2771188442 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d8c011990 con 0x7f1d9c15ca90 2023-05-20T01:24:55.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.452+0000 7f1d9a7fc700 1 -- 172.21.15.16:0/2771188442 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d8c011d00 con 0x7f1d9c15ca90 2023-05-20T01:24:55.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.453+0000 7f1da1cf2700 1 -- 172.21.15.16:0/2771188442 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d9c15ca90 msgr2=0x7f1d9c0cd890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:55.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.453+0000 7f1da1cf2700 1 --2- 172.21.15.16:0/2771188442 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d9c15ca90 0x7f1d9c0cd890 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f1d8c009840 tx=0x7f1d8c009b50 comp rx=0 tx=0).stop 2023-05-20T01:24:55.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.453+0000 7f1da1cf2700 1 -- 172.21.15.16:0/2771188442 shutdown_connections 2023-05-20T01:24:55.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.453+0000 7f1da1cf2700 1 --2- 172.21.15.16:0/2771188442 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1d9c0d29a0 0x7f1d9c0cf570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:55.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.453+0000 7f1da1cf2700 1 --2- 172.21.15.16:0/2771188442 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d9c15ca90 0x7f1d9c0cd890 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:55.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.453+0000 7f1da1cf2700 1 -- 172.21.15.16:0/2771188442 >> 172.21.15.16:0/2771188442 conn(0x7f1d9c06b000 msgr2=0x7f1d9c06b400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:24:55.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.453+0000 7f1da1cf2700 1 -- 172.21.15.16:0/2771188442 shutdown_connections 2023-05-20T01:24:55.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.453+0000 7f1da1cf2700 1 -- 172.21.15.16:0/2771188442 wait complete. 2023-05-20T01:24:55.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.454+0000 7f1da1cf2700 1 Processor -- start 2023-05-20T01:24:55.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.454+0000 7f1da1cf2700 1 -- start start 2023-05-20T01:24:55.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.455+0000 7f1da1cf2700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d9c0d2550 0x7f1d9c15f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:55.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.455+0000 7f1da1cf2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d9c0d29a0 0x7f1d9c1f61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:55.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.455+0000 7f1da1cf2700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1d9c15ca90 0x7f1d9c1555d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:55.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.455+0000 7f1da1cf2700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d9c1f6710 con 0x7f1d9c0d29a0 2023-05-20T01:24:55.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.455+0000 7f1da1cf2700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d9c155b10 con 0x7f1d9c0d2550 2023-05-20T01:24:55.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.455+0000 7f1da1cf2700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d9c15fdb0 con 0x7f1d9c15ca90 2023-05-20T01:24:55.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.455+0000 7f1d9affd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d9c0d29a0 0x7f1d9c1f61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:55.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.455+0000 7f1d9affd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d9c0d29a0 0x7f1d9c1f61d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35718/0 (socket says 172.21.15.16:35718) 2023-05-20T01:24:55.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.455+0000 7f1d9affd700 1 -- 172.21.15.16:0/1054150326 learned_addr learned my addr 172.21.15.16:0/1054150326 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:24:55.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.455+0000 7f1d9bfff700 1 --2- 172.21.15.16:0/1054150326 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1d9c15ca90 0x7f1d9c1555d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:55.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.455+0000 7f1d9b7fe700 1 --2- 172.21.15.16:0/1054150326 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d9c0d2550 0x7f1d9c15f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:55.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.455+0000 7f1d9affd700 1 -- 172.21.15.16:0/1054150326 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1d9c15ca90 msgr2=0x7f1d9c1555d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:55.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.455+0000 7f1d9affd700 1 --2- 172.21.15.16:0/1054150326 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1d9c15ca90 0x7f1d9c1555d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:55.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.455+0000 7f1d9affd700 1 -- 172.21.15.16:0/1054150326 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d9c0d2550 msgr2=0x7f1d9c15f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:55.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.455+0000 7f1d9affd700 1 --2- 172.21.15.16:0/1054150326 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d9c0d2550 0x7f1d9c15f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:55.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.455+0000 7f1d9affd700 1 -- 172.21.15.16:0/1054150326 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d8c0094a0 con 0x7f1d9c0d29a0 2023-05-20T01:24:55.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.456+0000 7f1d9affd700 1 --2- 172.21.15.16:0/1054150326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d9c0d29a0 0x7f1d9c1f61d0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f1d9000c4d0 tx=0x7f1d9000c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:24:55.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.456+0000 7f1d83fff700 1 -- 172.21.15.16:0/1054150326 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d90005d70 con 0x7f1d9c0d29a0 2023-05-20T01:24:55.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.456+0000 7f1d83fff700 1 -- 172.21.15.16:0/1054150326 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d900070d0 con 0x7f1d9c0d29a0 2023-05-20T01:24:55.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.456+0000 7f1d83fff700 1 -- 172.21.15.16:0/1054150326 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d90007440 con 0x7f1d9c0d29a0 2023-05-20T01:24:55.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.456+0000 7f1da1cf2700 1 -- 172.21.15.16:0/1054150326 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d9c0d2030 con 0x7f1d9c0d29a0 2023-05-20T01:24:55.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.456+0000 7f1da1cf2700 1 -- 172.21.15.16:0/1054150326 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d9c0d1d50 con 0x7f1d9c0d29a0 2023-05-20T01:24:55.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.457+0000 7f1da1cf2700 1 -- 172.21.15.16:0/1054150326 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d9c15ce60 con 0x7f1d9c0d29a0 2023-05-20T01:24:55.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.457+0000 7f1d83fff700 1 -- 172.21.15.16:0/1054150326 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1d900075a0 con 0x7f1d9c0d29a0 2023-05-20T01:24:55.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.458+0000 7f1d83fff700 1 --2- 172.21.15.16:0/1054150326 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f1d8806ace0 0x7f1d8806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:55.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.458+0000 7f1d83fff700 1 -- 172.21.15.16:0/1054150326 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7189+0+0 (secure 0 0 0) 0x7f1d90088c20 con 0x7f1d9c0d29a0 2023-05-20T01:24:55.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.458+0000 7f1d9b7fe700 1 --2- 172.21.15.16:0/1054150326 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f1d8806ace0 0x7f1d8806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:55.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.458+0000 7f1d9b7fe700 1 --2- 172.21.15.16:0/1054150326 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f1d8806ace0 0x7f1d8806d190 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f1d8c000c00 tx=0x7f1d8c01c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:24:55.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.462+0000 7f1d83fff700 1 -- 172.21.15.16:0/1054150326 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f1d90053890 con 0x7f1d9c0d29a0 2023-05-20T01:24:55.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.661+0000 7f1da1cf2700 1 -- 172.21.15.16:0/1054150326 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1d9c04e8c0 con 0x7f1d9c0d29a0 2023-05-20T01:24:55.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.661+0000 7f1d83fff700 1 -- 172.21.15.16:0/1054150326 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v75) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1d90056eb0 con 0x7f1d9c0d29a0 2023-05-20T01:24:55.665 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-05-20T01:24:55.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.663+0000 7f1da1cf2700 1 -- 172.21.15.16:0/1054150326 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f1d8806ace0 msgr2=0x7f1d8806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:55.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.663+0000 7f1da1cf2700 1 --2- 172.21.15.16:0/1054150326 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f1d8806ace0 0x7f1d8806d190 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f1d8c000c00 tx=0x7f1d8c01c040 comp rx=0 tx=0).stop 2023-05-20T01:24:55.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.663+0000 7f1da1cf2700 1 -- 172.21.15.16:0/1054150326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d9c0d29a0 msgr2=0x7f1d9c1f61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:55.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.663+0000 7f1da1cf2700 1 --2- 172.21.15.16:0/1054150326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d9c0d29a0 0x7f1d9c1f61d0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f1d9000c4d0 tx=0x7f1d9000c7e0 comp rx=0 tx=0).stop 2023-05-20T01:24:55.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.663+0000 7f1da1cf2700 1 -- 172.21.15.16:0/1054150326 shutdown_connections 2023-05-20T01:24:55.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.663+0000 7f1da1cf2700 1 --2- 172.21.15.16:0/1054150326 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f1d8806ace0 0x7f1d8806d190 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:55.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.663+0000 7f1da1cf2700 1 --2- 172.21.15.16:0/1054150326 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1d9c15ca90 0x7f1d9c1555d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:55.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.663+0000 7f1da1cf2700 1 --2- 172.21.15.16:0/1054150326 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d9c0d2550 0x7f1d9c15f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:55.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.663+0000 7f1da1cf2700 1 --2- 172.21.15.16:0/1054150326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d9c0d29a0 0x7f1d9c1f61d0 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:55.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.663+0000 7f1da1cf2700 1 -- 172.21.15.16:0/1054150326 >> 172.21.15.16:0/1054150326 conn(0x7f1d9c06b000 msgr2=0x7f1d9c0b72d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:24:55.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.663+0000 7f1da1cf2700 1 -- 172.21.15.16:0/1054150326 shutdown_connections 2023-05-20T01:24:55.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.663+0000 7f1da1cf2700 1 -- 172.21.15.16:0/1054150326 wait complete. 2023-05-20T01:24:55.679 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-05-20T01:24:55.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.803+0000 7f4c5ff86700 1 Processor -- start 2023-05-20T01:24:55.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.803+0000 7f4c5ff86700 1 -- start start 2023-05-20T01:24:55.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.804+0000 7f4c5ff86700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4c58072300 0x7f4c580cf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:55.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.804+0000 7f4c5ff86700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4c5815d290 0x7f4c580cf800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:55.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.804+0000 7f4c5ff86700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c580cd630 con 0x7f4c5815cec0 2023-05-20T01:24:55.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.804+0000 7f4c5ff86700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c580cca60 con 0x7f4c5815d290 2023-05-20T01:24:55.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.804+0000 7f4c5ff86700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c580ccd00 con 0x7f4c58072300 2023-05-20T01:24:55.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.804+0000 7f4c5e523700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4c5815d290 0x7f4c580cf800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:55.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.804+0000 7f4c5dd22700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f4c5815cec0 0x7f4c58071bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:47252/0 (socket says 172.21.15.16:47252) 2023-05-20T01:24:55.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.804+0000 7f4c5dd22700 1 -- 172.21.15.16:0/4009683673 learned_addr learned my addr 172.21.15.16:0/4009683673 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:24:55.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.805+0000 7f4c5d521700 1 --2- 172.21.15.16:0/4009683673 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4c58072300 0x7f4c580cf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:55.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.805+0000 7f4c5e523700 1 -- 172.21.15.16:0/4009683673 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4c58072300 msgr2=0x7f4c580cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:24:55.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.805+0000 7f4c5e523700 1 --2- 172.21.15.16:0/4009683673 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4c58072300 0x7f4c580cf2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:55.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.805+0000 7f4c5e523700 1 -- 172.21.15.16:0/4009683673 >> v1:172.21.15.16:6789/0 conn(0x7f4c5815cec0 legacy=0x7f4c58071bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:55.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.805+0000 7f4c5cd20700 1 -- 172.21.15.16:0/4009683673 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2323879021 0 0) 0x7f4c580cd630 con 0x7f4c5815cec0 2023-05-20T01:24:55.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.805+0000 7f4c5e523700 1 -- 172.21.15.16:0/4009683673 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c580ccfe0 con 0x7f4c5815d290 2023-05-20T01:24:55.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.805+0000 7f4c5e523700 1 --2- 172.21.15.16:0/4009683673 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c5815d290 0x7f4c580cf800 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f4c50009d90 tx=0x7f4c5000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c86b9eeb0176b00 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:24:55.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.805+0000 7f4c5cd20700 1 -- 172.21.15.16:0/4009683673 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c5000e070 con 0x7f4c5815d290 2023-05-20T01:24:55.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.805+0000 7f4c5cd20700 1 -- 172.21.15.16:0/4009683673 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c5000bbd0 con 0x7f4c5815d290 2023-05-20T01:24:55.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.805+0000 7f4c5cd20700 1 -- 172.21.15.16:0/4009683673 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c50012430 con 0x7f4c5815d290 2023-05-20T01:24:55.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.805+0000 7f4c5ff86700 1 -- 172.21.15.16:0/4009683673 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c5815d290 msgr2=0x7f4c580cf800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:55.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.805+0000 7f4c5ff86700 1 --2- 172.21.15.16:0/4009683673 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c5815d290 0x7f4c580cf800 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f4c50009d90 tx=0x7f4c5000b330 comp rx=0 tx=0).stop 2023-05-20T01:24:55.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.806+0000 7f4c5ff86700 1 -- 172.21.15.16:0/4009683673 shutdown_connections 2023-05-20T01:24:55.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.806+0000 7f4c5ff86700 1 --2- 172.21.15.16:0/4009683673 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c5815d290 0x7f4c580cf800 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:55.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.806+0000 7f4c5ff86700 1 --2- 172.21.15.16:0/4009683673 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4c58072300 0x7f4c580cf2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:55.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.806+0000 7f4c5ff86700 1 -- 172.21.15.16:0/4009683673 >> 172.21.15.16:0/4009683673 conn(0x7f4c580cffd0 msgr2=0x7f4c580d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:24:55.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.806+0000 7f4c5ff86700 1 -- 172.21.15.16:0/4009683673 shutdown_connections 2023-05-20T01:24:55.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.806+0000 7f4c5ff86700 1 -- 172.21.15.16:0/4009683673 wait complete. 2023-05-20T01:24:55.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.807+0000 7f4c5ff86700 1 Processor -- start 2023-05-20T01:24:55.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.807+0000 7f4c5ff86700 1 -- start start 2023-05-20T01:24:55.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.808+0000 7f4c5ff86700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c58072300 0x7f4c581f21e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:55.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.808+0000 7f4c5ff86700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4c5815cec0 0x7f4c580ac8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:55.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.808+0000 7f4c5ff86700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4c5815d290 0x7f4c580b1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:55.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.808+0000 7f4c5ff86700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c580c70a0 con 0x7f4c58072300 2023-05-20T01:24:55.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.808+0000 7f4c5ff86700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c580af810 con 0x7f4c5815d290 2023-05-20T01:24:55.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.808+0000 7f4c5ff86700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c580c72f0 con 0x7f4c5815cec0 2023-05-20T01:24:55.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.808+0000 7f4c5dd22700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c58072300 0x7f4c581f21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:55.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.808+0000 7f4c5dd22700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c58072300 0x7f4c581f21e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35730/0 (socket says 172.21.15.16:35730) 2023-05-20T01:24:55.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.808+0000 7f4c5dd22700 1 -- 172.21.15.16:0/270837651 learned_addr learned my addr 172.21.15.16:0/270837651 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:24:55.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.808+0000 7f4c5d521700 1 --2- 172.21.15.16:0/270837651 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4c5815cec0 0x7f4c580ac8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:55.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.808+0000 7f4c5e523700 1 --2- 172.21.15.16:0/270837651 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4c5815d290 0x7f4c580b1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:55.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.808+0000 7f4c5dd22700 1 -- 172.21.15.16:0/270837651 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4c5815cec0 msgr2=0x7f4c580ac8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:55.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.808+0000 7f4c5dd22700 1 --2- 172.21.15.16:0/270837651 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4c5815cec0 0x7f4c580ac8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:55.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.808+0000 7f4c5dd22700 1 -- 172.21.15.16:0/270837651 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4c5815d290 msgr2=0x7f4c580b1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:55.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.808+0000 7f4c5dd22700 1 --2- 172.21.15.16:0/270837651 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4c5815d290 0x7f4c580b1ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:55.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.808+0000 7f4c5dd22700 1 -- 172.21.15.16:0/270837651 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c50009a70 con 0x7f4c58072300 2023-05-20T01:24:55.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.808+0000 7f4c5e523700 1 --2- 172.21.15.16:0/270837651 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4c5815d290 0x7f4c580b1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:24:55.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.808+0000 7f4c5dd22700 1 --2- 172.21.15.16:0/270837651 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c58072300 0x7f4c581f21e0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f4c480010e0 tx=0x7f4c4800cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:24:55.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.808+0000 7f4c4e7fc700 1 -- 172.21.15.16:0/270837651 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c480074e0 con 0x7f4c58072300 2023-05-20T01:24:55.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.808+0000 7f4c4e7fc700 1 -- 172.21.15.16:0/270837651 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c48007640 con 0x7f4c58072300 2023-05-20T01:24:55.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.808+0000 7f4c5ff86700 1 -- 172.21.15.16:0/270837651 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c580cfd40 con 0x7f4c58072300 2023-05-20T01:24:55.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.808+0000 7f4c4e7fc700 1 -- 172.21.15.16:0/270837651 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c48005a70 con 0x7f4c58072300 2023-05-20T01:24:55.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.809+0000 7f4c5ff86700 1 -- 172.21.15.16:0/270837651 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c580cd2c0 con 0x7f4c58072300 2023-05-20T01:24:55.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.809+0000 7f4c5ff86700 1 -- 172.21.15.16:0/270837651 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c580c7f00 con 0x7f4c58072300 2023-05-20T01:24:55.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.810+0000 7f4c4e7fc700 1 -- 172.21.15.16:0/270837651 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4c48005bd0 con 0x7f4c58072300 2023-05-20T01:24:55.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.810+0000 7f4c4e7fc700 1 --2- 172.21.15.16:0/270837651 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f4c3c06ace0 0x7f4c3c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:55.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.810+0000 7f4c4e7fc700 1 -- 172.21.15.16:0/270837651 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7189+0+0 (secure 0 0 0) 0x7f4c48088db0 con 0x7f4c58072300 2023-05-20T01:24:55.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.810+0000 7f4c5d521700 1 --2- 172.21.15.16:0/270837651 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f4c3c06ace0 0x7f4c3c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:55.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.811+0000 7f4c5d521700 1 --2- 172.21.15.16:0/270837651 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f4c3c06ace0 0x7f4c3c06d190 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f4c581597f0 tx=0x7f4c54008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:24:55.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:55.815+0000 7f4c4e7fc700 1 -- 172.21.15.16:0/270837651 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f4c48053a80 con 0x7f4c58072300 2023-05-20T01:24:56.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.016+0000 7f4c5ff86700 1 -- 172.21.15.16:0/270837651 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4c58160870 con 0x7f4c58072300 2023-05-20T01:24:56.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.016+0000 7f4c4e7fc700 1 -- 172.21.15.16:0/270837651 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v75) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4c48017020 con 0x7f4c58072300 2023-05-20T01:24:56.021 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2023-05-20T01:24:56.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.018+0000 7f4c5ff86700 1 -- 172.21.15.16:0/270837651 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f4c3c06ace0 msgr2=0x7f4c3c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:56.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.018+0000 7f4c5ff86700 1 --2- 172.21.15.16:0/270837651 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f4c3c06ace0 0x7f4c3c06d190 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f4c581597f0 tx=0x7f4c54008040 comp rx=0 tx=0).stop 2023-05-20T01:24:56.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.018+0000 7f4c5ff86700 1 -- 172.21.15.16:0/270837651 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c58072300 msgr2=0x7f4c581f21e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:56.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.018+0000 7f4c5ff86700 1 --2- 172.21.15.16:0/270837651 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c58072300 0x7f4c581f21e0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f4c480010e0 tx=0x7f4c4800cae0 comp rx=0 tx=0).stop 2023-05-20T01:24:56.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.018+0000 7f4c5ff86700 1 -- 172.21.15.16:0/270837651 shutdown_connections 2023-05-20T01:24:56.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.018+0000 7f4c5ff86700 1 --2- 172.21.15.16:0/270837651 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4c5815d290 0x7f4c580b1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:56.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.018+0000 7f4c5ff86700 1 --2- 172.21.15.16:0/270837651 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c58072300 0x7f4c581f21e0 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:56.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.018+0000 7f4c5ff86700 1 --2- 172.21.15.16:0/270837651 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f4c3c06ace0 0x7f4c3c06d190 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:56.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.018+0000 7f4c5ff86700 1 --2- 172.21.15.16:0/270837651 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4c5815cec0 0x7f4c580ac8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:56.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.018+0000 7f4c5ff86700 1 -- 172.21.15.16:0/270837651 >> 172.21.15.16:0/270837651 conn(0x7f4c580cffd0 msgr2=0x7f4c580cf560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:24:56.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.018+0000 7f4c5ff86700 1 -- 172.21.15.16:0/270837651 shutdown_connections 2023-05-20T01:24:56.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.018+0000 7f4c5ff86700 1 -- 172.21.15.16:0/270837651 wait complete. 2023-05-20T01:24:56.035 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_extra pg_num 2023-05-20T01:24:56.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.159+0000 7f5033c88700 1 Processor -- start 2023-05-20T01:24:56.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.159+0000 7f5033c88700 1 -- start start 2023-05-20T01:24:56.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.160+0000 7f5033c88700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f502c15ab80 0x7f502c0cda80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:56.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.160+0000 7f5033c88700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f502c0cdfc0 0x7f502c0cec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:56.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.160+0000 7f5033c88700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f502c0cf1c0 0x7f502c15e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:56.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.160+0000 7f5033c88700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f502c0cb700 con 0x7f502c15ab80 2023-05-20T01:24:56.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.160+0000 7f5033c88700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f502c0cf640 con 0x7f502c0cf1c0 2023-05-20T01:24:56.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.160+0000 7f5033c88700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f502c0ce4d0 con 0x7f502c0cdfc0 2023-05-20T01:24:56.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.160+0000 7f5031a24700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f502c15ab80 0x7f502c0cda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:56.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.160+0000 7f5031a24700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f502c15ab80 0x7f502c0cda80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35738/0 (socket says 172.21.15.16:35738) 2023-05-20T01:24:56.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.160+0000 7f5031a24700 1 -- 172.21.15.16:0/1372917904 learned_addr learned my addr 172.21.15.16:0/1372917904 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:24:56.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.160+0000 7f5031223700 1 --2- 172.21.15.16:0/1372917904 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f502c0cdfc0 0x7f502c0cec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:56.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.160+0000 7f5032225700 1 --2- 172.21.15.16:0/1372917904 >> v2:172.21.15.132:3300/0 conn(0x7f502c0cf1c0 0x7f502c15e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:56.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.160+0000 7f5031a24700 1 -- 172.21.15.16:0/1372917904 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f502c0cdfc0 msgr2=0x7f502c0cec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:24:56.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.160+0000 7f5031a24700 1 --2- 172.21.15.16:0/1372917904 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f502c0cdfc0 0x7f502c0cec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:56.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.160+0000 7f5031a24700 1 -- 172.21.15.16:0/1372917904 >> v2:172.21.15.132:3300/0 conn(0x7f502c0cf1c0 msgr2=0x7f502c15e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:24:56.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.160+0000 7f5031a24700 1 --2- 172.21.15.16:0/1372917904 >> v2:172.21.15.132:3300/0 conn(0x7f502c0cf1c0 0x7f502c15e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:56.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.160+0000 7f5031a24700 1 -- 172.21.15.16:0/1372917904 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f502c0ce7b0 con 0x7f502c15ab80 2023-05-20T01:24:56.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.160+0000 7f5031a24700 1 --2- 172.21.15.16:0/1372917904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f502c15ab80 0x7f502c0cda80 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f502c0d2020 tx=0x7f5028009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa644feb15e716e6 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:24:56.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.161+0000 7f5030a22700 1 -- 172.21.15.16:0/1372917904 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f502800b870 con 0x7f502c15ab80 2023-05-20T01:24:56.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.161+0000 7f5030a22700 1 -- 172.21.15.16:0/1372917904 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f502800b9d0 con 0x7f502c15ab80 2023-05-20T01:24:56.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.161+0000 7f5030a22700 1 -- 172.21.15.16:0/1372917904 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f502800bd60 con 0x7f502c15ab80 2023-05-20T01:24:56.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.161+0000 7f5033c88700 1 -- 172.21.15.16:0/1372917904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f502c15ab80 msgr2=0x7f502c0cda80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:56.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.161+0000 7f5033c88700 1 --2- 172.21.15.16:0/1372917904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f502c15ab80 0x7f502c0cda80 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f502c0d2020 tx=0x7f5028009bd0 comp rx=0 tx=0).stop 2023-05-20T01:24:56.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.161+0000 7f5033c88700 1 -- 172.21.15.16:0/1372917904 shutdown_connections 2023-05-20T01:24:56.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.161+0000 7f5033c88700 1 --2- 172.21.15.16:0/1372917904 >> v2:172.21.15.132:3300/0 conn(0x7f502c0cf1c0 0x7f502c15e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:56.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.161+0000 7f5033c88700 1 --2- 172.21.15.16:0/1372917904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f502c15ab80 0x7f502c0cda80 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:56.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.161+0000 7f5033c88700 1 --2- 172.21.15.16:0/1372917904 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f502c0cdfc0 0x7f502c0cec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:56.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.161+0000 7f5033c88700 1 -- 172.21.15.16:0/1372917904 >> 172.21.15.16:0/1372917904 conn(0x7f502c06ae30 msgr2=0x7f502c09a400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:24:56.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.161+0000 7f5033c88700 1 -- 172.21.15.16:0/1372917904 shutdown_connections 2023-05-20T01:24:56.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.161+0000 7f5033c88700 1 -- 172.21.15.16:0/1372917904 wait complete. 2023-05-20T01:24:56.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.163+0000 7f5033c88700 1 Processor -- start 2023-05-20T01:24:56.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.163+0000 7f5033c88700 1 -- start start 2023-05-20T01:24:56.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.163+0000 7f5033c88700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f502c0cdfc0 0x7f502c0b1f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:56.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.163+0000 7f5033c88700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f502c0cf1c0 0x7f502c1552d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:56.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.163+0000 7f5033c88700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f502c15ab80 0x7f502c0c78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:56.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.163+0000 7f5033c88700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f502c0c7e30 con 0x7f502c0cf1c0 2023-05-20T01:24:56.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.163+0000 7f5033c88700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f502c0b3280 con 0x7f502c0cdfc0 2023-05-20T01:24:56.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.163+0000 7f5033c88700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f502c0b2950 con 0x7f502c15ab80 2023-05-20T01:24:56.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.163+0000 7f5031223700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f502c0cf1c0 0x7f502c1552d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:56.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.163+0000 7f5031223700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f502c0cf1c0 0x7f502c1552d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35744/0 (socket says 172.21.15.16:35744) 2023-05-20T01:24:56.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.163+0000 7f5031223700 1 -- 172.21.15.16:0/3625036277 learned_addr learned my addr 172.21.15.16:0/3625036277 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:24:56.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.163+0000 7f5032225700 1 --2- 172.21.15.16:0/3625036277 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f502c15ab80 0x7f502c0c78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:56.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.163+0000 7f5031a24700 1 --2- 172.21.15.16:0/3625036277 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f502c0cdfc0 0x7f502c0b1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:56.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.164+0000 7f5031223700 1 -- 172.21.15.16:0/3625036277 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f502c15ab80 msgr2=0x7f502c0c78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:56.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.164+0000 7f5031223700 1 --2- 172.21.15.16:0/3625036277 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f502c15ab80 0x7f502c0c78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:56.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.164+0000 7f5031223700 1 -- 172.21.15.16:0/3625036277 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f502c0cdfc0 msgr2=0x7f502c0b1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:56.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.164+0000 7f5031223700 1 --2- 172.21.15.16:0/3625036277 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f502c0cdfc0 0x7f502c0b1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:56.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.164+0000 7f5031223700 1 -- 172.21.15.16:0/3625036277 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5028009550 con 0x7f502c0cf1c0 2023-05-20T01:24:56.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.164+0000 7f5031223700 1 --2- 172.21.15.16:0/3625036277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f502c0cf1c0 0x7f502c1552d0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f501c00d800 tx=0x7f501c00db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:24:56.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.164+0000 7f50227fc700 1 -- 172.21.15.16:0/3625036277 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f501c0125f0 con 0x7f502c0cf1c0 2023-05-20T01:24:56.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.164+0000 7f50227fc700 1 -- 172.21.15.16:0/3625036277 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f501c012750 con 0x7f502c0cf1c0 2023-05-20T01:24:56.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.164+0000 7f5033c88700 1 -- 172.21.15.16:0/3625036277 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f502c0cbbf0 con 0x7f502c0cf1c0 2023-05-20T01:24:56.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.164+0000 7f50227fc700 1 -- 172.21.15.16:0/3625036277 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f501c012ae0 con 0x7f502c0cf1c0 2023-05-20T01:24:56.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.164+0000 7f5033c88700 1 -- 172.21.15.16:0/3625036277 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f502c0cb910 con 0x7f502c0cf1c0 2023-05-20T01:24:56.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.165+0000 7f5033c88700 1 -- 172.21.15.16:0/3625036277 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f502c0d0e30 con 0x7f502c0cf1c0 2023-05-20T01:24:56.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.166+0000 7f50227fc700 1 -- 172.21.15.16:0/3625036277 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f501c012c40 con 0x7f502c0cf1c0 2023-05-20T01:24:56.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.166+0000 7f50227fc700 1 --2- 172.21.15.16:0/3625036277 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f501006ace0 0x7f501006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:56.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.166+0000 7f50227fc700 1 -- 172.21.15.16:0/3625036277 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7189+0+0 (secure 0 0 0) 0x7f501c089870 con 0x7f502c0cf1c0 2023-05-20T01:24:56.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.166+0000 7f5031a24700 1 --2- 172.21.15.16:0/3625036277 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f501006ace0 0x7f501006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:56.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.166+0000 7f5031a24700 1 --2- 172.21.15.16:0/3625036277 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f501006ace0 0x7f501006d190 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f5028000c00 tx=0x7f5028008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:24:56.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.170+0000 7f50227fc700 1 -- 172.21.15.16:0/3625036277 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f501c0544b0 con 0x7f502c0cf1c0 2023-05-20T01:24:56.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.367+0000 7f5033c88700 1 -- 172.21.15.16:0/3625036277 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"} v 0) v1 -- 0x7f502c0747b0 con 0x7f502c0cf1c0 2023-05-20T01:24:56.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.368+0000 7f50227fc700 1 -- 172.21.15.16:0/3625036277 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"}]=0 v75) v1 ==== 106+0+11 (secure 0 0 0) 0x7f501c057ad0 con 0x7f502c0cf1c0 2023-05-20T01:24:56.376 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-05-20T01:24:56.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.370+0000 7f5033c88700 1 -- 172.21.15.16:0/3625036277 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f501006ace0 msgr2=0x7f501006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:56.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.370+0000 7f5033c88700 1 --2- 172.21.15.16:0/3625036277 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f501006ace0 0x7f501006d190 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f5028000c00 tx=0x7f5028008030 comp rx=0 tx=0).stop 2023-05-20T01:24:56.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.370+0000 7f5033c88700 1 -- 172.21.15.16:0/3625036277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f502c0cf1c0 msgr2=0x7f502c1552d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:56.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.370+0000 7f5033c88700 1 --2- 172.21.15.16:0/3625036277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f502c0cf1c0 0x7f502c1552d0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f501c00d800 tx=0x7f501c00db10 comp rx=0 tx=0).stop 2023-05-20T01:24:56.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.370+0000 7f5033c88700 1 -- 172.21.15.16:0/3625036277 shutdown_connections 2023-05-20T01:24:56.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.370+0000 7f5033c88700 1 --2- 172.21.15.16:0/3625036277 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f501006ace0 0x7f501006d190 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:56.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.370+0000 7f5033c88700 1 --2- 172.21.15.16:0/3625036277 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f502c15ab80 0x7f502c0c78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:56.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.370+0000 7f5033c88700 1 --2- 172.21.15.16:0/3625036277 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f502c0cdfc0 0x7f502c0b1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:56.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.370+0000 7f5033c88700 1 --2- 172.21.15.16:0/3625036277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f502c0cf1c0 0x7f502c1552d0 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:56.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.370+0000 7f5033c88700 1 -- 172.21.15.16:0/3625036277 >> 172.21.15.16:0/3625036277 conn(0x7f502c06ae30 msgr2=0x7f502c06fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:24:56.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.370+0000 7f5033c88700 1 -- 172.21.15.16:0/3625036277 shutdown_connections 2023-05-20T01:24:56.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.370+0000 7f5033c88700 1 -- 172.21.15.16:0/3625036277 wait complete. 2023-05-20T01:24:56.386 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-20T01:24:56.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.506+0000 7f1480e06700 1 Processor -- start 2023-05-20T01:24:56.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.506+0000 7f1480e06700 1 -- start start 2023-05-20T01:24:56.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.507+0000 7f1480e06700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f147c0caaa0 0x7f147c0cea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:56.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.507+0000 7f1480e06700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f147c159d90 con 0x7f147c1599c0 2023-05-20T01:24:56.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.507+0000 7f1480e06700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f147c0cdb80 con 0x7f147c0cb670 2023-05-20T01:24:56.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.507+0000 7f1480e06700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f147c0cdf20 con 0x7f147c0caaa0 2023-05-20T01:24:56.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.507+0000 7f147a59c700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f147c1599c0 0x7f147c0caf60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:47280/0 (socket says 172.21.15.16:47280) 2023-05-20T01:24:56.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.507+0000 7f147a59c700 1 -- 172.21.15.16:0/4251080102 learned_addr learned my addr 172.21.15.16:0/4251080102 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:24:56.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.508+0000 7f147ad9d700 1 --2- 172.21.15.16:0/4251080102 >> v2:172.21.15.132:3300/0 conn(0x7f147c0caaa0 0x7f147c0cea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:56.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.508+0000 7f147959a700 1 -- 172.21.15.16:0/4251080102 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 624672170 0 0) 0x7f1470001110 con 0x7f147c1599c0 2023-05-20T01:24:56.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.508+0000 7f147959a700 1 -- 172.21.15.16:0/4251080102 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f14640035b0 con 0x7f147c1599c0 2023-05-20T01:24:56.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.508+0000 7f147959a700 1 -- 172.21.15.16:0/4251080102 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2003799178 0 0) 0x7f14640035b0 con 0x7f147c1599c0 2023-05-20T01:24:56.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.508+0000 7f147959a700 1 -- 172.21.15.16:0/4251080102 >> v2:172.21.15.132:3300/0 conn(0x7f147c0caaa0 msgr2=0x7f147c0cea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:24:56.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.508+0000 7f147959a700 1 --2- 172.21.15.16:0/4251080102 >> v2:172.21.15.132:3300/0 conn(0x7f147c0caaa0 0x7f147c0cea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:56.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.508+0000 7f147959a700 1 -- 172.21.15.16:0/4251080102 >> v1:172.21.15.132:6789/0 conn(0x7f147c0cb670 legacy=0x7f147c0ce340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:56.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.508+0000 7f147959a700 1 -- 172.21.15.16:0/4251080102 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f147c06eaf0 con 0x7f147c1599c0 2023-05-20T01:24:56.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.508+0000 7f147959a700 1 -- 172.21.15.16:0/4251080102 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7f1470001800 con 0x7f147c1599c0 2023-05-20T01:24:56.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.508+0000 7f147959a700 1 -- 172.21.15.16:0/4251080102 >> v1:172.21.15.16:6789/0 conn(0x7f147c1599c0 legacy=0x7f147c0caf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:56.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.508+0000 7f147959a700 1 --2- 172.21.15.16:0/4251080102 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1464003a70 0x7f1464003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:56.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.508+0000 7f147959a700 1 --2- 172.21.15.16:0/4251080102 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1464004850 0x7f1464004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:56.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.508+0000 7f147959a700 1 --2- 172.21.15.16:0/4251080102 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1464005200 0x7f146400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:56.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.508+0000 7f147959a700 1 -- 172.21.15.16:0/4251080102 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f147c0cdf20 con 0x7f1464003a70 2023-05-20T01:24:56.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.508+0000 7f147959a700 1 -- 172.21.15.16:0/4251080102 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f146400c0e0 con 0x7f1464004850 2023-05-20T01:24:56.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.508+0000 7f147959a700 1 -- 172.21.15.16:0/4251080102 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f146400c2a0 con 0x7f1464005200 2023-05-20T01:24:56.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.508+0000 7f147a59c700 1 --2- 172.21.15.16:0/4251080102 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1464003a70 0x7f1464003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:56.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.508+0000 7f1479d9b700 1 --2- 172.21.15.16:0/4251080102 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1464004850 0x7f1464004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:56.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.508+0000 7f147ad9d700 1 --2- 172.21.15.16:0/4251080102 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1464005200 0x7f146400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:56.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.508+0000 7f147a59c700 1 -- 172.21.15.16:0/4251080102 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1464005200 msgr2=0x7f146400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:24:56.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.508+0000 7f147a59c700 1 --2- 172.21.15.16:0/4251080102 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1464005200 0x7f146400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:56.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.508+0000 7f147a59c700 1 -- 172.21.15.16:0/4251080102 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1464004850 msgr2=0x7f1464004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:24:56.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.508+0000 7f147a59c700 1 --2- 172.21.15.16:0/4251080102 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1464004850 0x7f1464004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:56.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.508+0000 7f147a59c700 1 -- 172.21.15.16:0/4251080102 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14640035b0 con 0x7f1464003a70 2023-05-20T01:24:56.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.508+0000 7f147ad9d700 1 --2- 172.21.15.16:0/4251080102 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1464005200 0x7f146400ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:24:56.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.508+0000 7f147ad9d700 1 -- 172.21.15.16:0/4251080102 reap_dead start 2023-05-20T01:24:56.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.508+0000 7f147a59c700 1 --2- 172.21.15.16:0/4251080102 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1464003a70 0x7f1464003e80 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f147000c210 tx=0x7f1464005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a650878d48abab4 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:24:56.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.509+0000 7f147959a700 1 -- 172.21.15.16:0/4251080102 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f147000cdb0 con 0x7f1464003a70 2023-05-20T01:24:56.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.509+0000 7f147959a700 1 -- 172.21.15.16:0/4251080102 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1470005960 con 0x7f1464003a70 2023-05-20T01:24:56.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.509+0000 7f147959a700 1 -- 172.21.15.16:0/4251080102 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1470005d10 con 0x7f1464003a70 2023-05-20T01:24:56.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.509+0000 7f1480e06700 1 -- 172.21.15.16:0/4251080102 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1464003a70 msgr2=0x7f1464003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:56.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.509+0000 7f1480e06700 1 --2- 172.21.15.16:0/4251080102 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1464003a70 0x7f1464003e80 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f147000c210 tx=0x7f1464005880 comp rx=0 tx=0).stop 2023-05-20T01:24:56.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.509+0000 7f1480e06700 1 -- 172.21.15.16:0/4251080102 shutdown_connections 2023-05-20T01:24:56.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.509+0000 7f1480e06700 1 --2- 172.21.15.16:0/4251080102 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1464003a70 0x7f1464003e80 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:56.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.509+0000 7f1480e06700 1 -- 172.21.15.16:0/4251080102 >> 172.21.15.16:0/4251080102 conn(0x7f147c0d0c30 msgr2=0x7f147c0b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:24:56.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.509+0000 7f1480e06700 1 -- 172.21.15.16:0/4251080102 shutdown_connections 2023-05-20T01:24:56.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.509+0000 7f1480e06700 1 -- 172.21.15.16:0/4251080102 wait complete. 2023-05-20T01:24:56.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.511+0000 7f1480e06700 1 Processor -- start 2023-05-20T01:24:56.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.511+0000 7f1480e06700 1 -- start start 2023-05-20T01:24:56.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.511+0000 7f1480e06700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1464003a70 0x7f147c15a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:56.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.511+0000 7f1480e06700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f147c15a7e0 0x7f147c0710d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:56.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.511+0000 7f1480e06700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f147c071610 0x7f147c14ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:56.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.511+0000 7f1480e06700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f147c14de80 con 0x7f1464003a70 2023-05-20T01:24:56.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.511+0000 7f1480e06700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f147c071a90 con 0x7f147c071610 2023-05-20T01:24:56.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.511+0000 7f1480e06700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f147c14dba0 con 0x7f147c15a7e0 2023-05-20T01:24:56.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.512+0000 7f147a59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1464003a70 0x7f147c15a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:56.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.512+0000 7f147a59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1464003a70 0x7f147c15a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35764/0 (socket says 172.21.15.16:35764) 2023-05-20T01:24:56.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.512+0000 7f147a59c700 1 -- 172.21.15.16:0/2370311155 learned_addr learned my addr 172.21.15.16:0/2370311155 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:24:56.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.512+0000 7f1479d9b700 1 --2- 172.21.15.16:0/2370311155 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f147c15a7e0 0x7f147c0710d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:56.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.512+0000 7f147ad9d700 1 --2- 172.21.15.16:0/2370311155 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f147c071610 0x7f147c14ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:56.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.512+0000 7f147a59c700 1 -- 172.21.15.16:0/2370311155 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f147c15a7e0 msgr2=0x7f147c0710d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:56.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.512+0000 7f147a59c700 1 --2- 172.21.15.16:0/2370311155 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f147c15a7e0 0x7f147c0710d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:56.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.512+0000 7f147a59c700 1 -- 172.21.15.16:0/2370311155 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f147c071610 msgr2=0x7f147c14ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:56.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.512+0000 7f147a59c700 1 --2- 172.21.15.16:0/2370311155 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f147c071610 0x7f147c14ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:56.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.512+0000 7f147a59c700 1 -- 172.21.15.16:0/2370311155 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f147c06f0b0 con 0x7f1464003a70 2023-05-20T01:24:56.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.512+0000 7f147a59c700 1 --2- 172.21.15.16:0/2370311155 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1464003a70 0x7f147c15a2a0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f1464005830 tx=0x7f1470000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:24:56.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.512+0000 7f146affd700 1 -- 172.21.15.16:0/2370311155 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1470007d80 con 0x7f1464003a70 2023-05-20T01:24:56.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.512+0000 7f146affd700 1 -- 172.21.15.16:0/2370311155 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1470004870 con 0x7f1464003a70 2023-05-20T01:24:56.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.512+0000 7f1480e06700 1 -- 172.21.15.16:0/2370311155 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f147c06edd0 con 0x7f1464003a70 2023-05-20T01:24:56.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.512+0000 7f146affd700 1 -- 172.21.15.16:0/2370311155 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1470004be0 con 0x7f1464003a70 2023-05-20T01:24:56.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.512+0000 7f1480e06700 1 -- 172.21.15.16:0/2370311155 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f147c06e750 con 0x7f1464003a70 2023-05-20T01:24:56.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.513+0000 7f1480e06700 1 -- 172.21.15.16:0/2370311155 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f147c1e20d0 con 0x7f1464003a70 2023-05-20T01:24:56.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.514+0000 7f146affd700 1 -- 172.21.15.16:0/2370311155 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1470004d40 con 0x7f1464003a70 2023-05-20T01:24:56.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.514+0000 7f146affd700 1 --2- 172.21.15.16:0/2370311155 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f145c06ace0 0x7f145c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:56.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.514+0000 7f146affd700 1 -- 172.21.15.16:0/2370311155 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7189+0+0 (secure 0 0 0) 0x7f147008a5c0 con 0x7f1464003a70 2023-05-20T01:24:56.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.514+0000 7f1479d9b700 1 --2- 172.21.15.16:0/2370311155 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f145c06ace0 0x7f145c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:56.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.514+0000 7f1479d9b700 1 --2- 172.21.15.16:0/2370311155 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f145c06ace0 0x7f145c06d190 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f147c14e060 tx=0x7f146c012040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:24:56.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.518+0000 7f146affd700 1 -- 172.21.15.16:0/2370311155 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f147c1e20d0 con 0x7f1464003a70 2023-05-20T01:24:56.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.760+0000 7f1480e06700 1 -- 172.21.15.16:0/2370311155 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f147c159d90 con 0x7f1464003a70 2023-05-20T01:24:56.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.760+0000 7f146affd700 1 -- 172.21.15.16:0/2370311155 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+4679 (secure 0 0 0) 0x7f147c159d90 con 0x7f1464003a70 2023-05-20T01:24:56.765 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:24:56.765 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":49,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5873,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3769502841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3769502841},{"type":"v1","addr":"172.21.15.16:6835","nonce":3769502841}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45},{"gid":5879,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/688624373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":688624373},{"type":"v1","addr":"172.21.15.132:6834","nonce":688624373}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45},{"gid":6307,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/1655712081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":1655712081},{"type":"v1","addr":"172.21.15.132:6835","nonce":1655712081}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-20T01:22:14.127671+0000","modified":"2023-05-20T01:22:24.390190+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":72,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6768},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6768":{"gid":6768,"name":"c","rank":0,"incarnation":46,"state":"up:active","state_seq":4,"addr":"172.21.15.16:6837/2351975329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2351975329},{"type":"v1","addr":"172.21.15.16:6837","nonce":2351975329}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10,11],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-05-20T01:24:56.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.762+0000 7f1480e06700 1 -- 172.21.15.16:0/2370311155 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f145c06ace0 msgr2=0x7f145c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:56.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.762+0000 7f1480e06700 1 --2- 172.21.15.16:0/2370311155 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f145c06ace0 0x7f145c06d190 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f147c14e060 tx=0x7f146c012040 comp rx=0 tx=0).stop 2023-05-20T01:24:56.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.762+0000 7f1480e06700 1 -- 172.21.15.16:0/2370311155 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1464003a70 msgr2=0x7f147c15a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:56.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.762+0000 7f1480e06700 1 --2- 172.21.15.16:0/2370311155 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1464003a70 0x7f147c15a2a0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f1464005830 tx=0x7f1470000fe0 comp rx=0 tx=0).stop 2023-05-20T01:24:56.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.762+0000 7f1480e06700 1 -- 172.21.15.16:0/2370311155 shutdown_connections 2023-05-20T01:24:56.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.762+0000 7f1480e06700 1 --2- 172.21.15.16:0/2370311155 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f147c071610 0x7f147c14ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:56.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.762+0000 7f1480e06700 1 --2- 172.21.15.16:0/2370311155 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1464003a70 0x7f147c15a2a0 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:56.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.762+0000 7f1480e06700 1 --2- 172.21.15.16:0/2370311155 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f145c06ace0 0x7f145c06d190 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:56.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.762+0000 7f1480e06700 1 --2- 172.21.15.16:0/2370311155 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f147c15a7e0 0x7f147c0710d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:56.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.762+0000 7f1480e06700 1 -- 172.21.15.16:0/2370311155 >> 172.21.15.16:0/2370311155 conn(0x7f147c0d0c30 msgr2=0x7f147c0afcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:24:56.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.762+0000 7f1480e06700 1 -- 172.21.15.16:0/2370311155 shutdown_connections 2023-05-20T01:24:56.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.762+0000 7f1480e06700 1 -- 172.21.15.16:0/2370311155 wait complete. 2023-05-20T01:24:56.768 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 49 2023-05-20T01:24:56.779 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-20T01:24:56.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.899+0000 7f144b805700 1 Processor -- start 2023-05-20T01:24:56.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.900+0000 7f144b805700 1 -- start start 2023-05-20T01:24:56.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.900+0000 7f144b805700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f14440cfc50 0x7f14440d2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:56.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.900+0000 7f144b805700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f14440cd970 0x7f14440cdd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:56.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.900+0000 7f144b805700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14440b3b50 con 0x7f14440b4750 2023-05-20T01:24:56.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.900+0000 7f144b805700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14440cfad0 con 0x7f14440cd970 2023-05-20T01:24:56.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.900+0000 7f144b805700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14440d3df0 con 0x7f14440cfc50 2023-05-20T01:24:56.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.901+0000 7f1449da2700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f14440cd970 0x7f14440cdd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:56.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.901+0000 7f1449da2700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f14440cd970 0x7f14440cdd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35776/0 (socket says 172.21.15.16:35776) 2023-05-20T01:24:56.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.901+0000 7f1449da2700 1 -- 172.21.15.16:0/998663785 learned_addr learned my addr 172.21.15.16:0/998663785 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:24:56.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.901+0000 7f14495a1700 1 --2- 172.21.15.16:0/998663785 >> v2:172.21.15.132:3300/0 conn(0x7f14440cfc50 0x7f14440d2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:56.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.901+0000 7f1449da2700 1 -- 172.21.15.16:0/998663785 >> v2:172.21.15.132:3300/0 conn(0x7f14440cfc50 msgr2=0x7f14440d2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:24:56.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.901+0000 7f1449da2700 1 --2- 172.21.15.16:0/998663785 >> v2:172.21.15.132:3300/0 conn(0x7f14440cfc50 0x7f14440d2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:56.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.901+0000 7f1449da2700 1 -- 172.21.15.16:0/998663785 >> v1:172.21.15.132:6789/0 conn(0x7f14440b4750 legacy=0x7f14440d0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:24:56.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.901+0000 7f1449da2700 1 -- 172.21.15.16:0/998663785 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14440d0b90 con 0x7f14440cd970 2023-05-20T01:24:56.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.901+0000 7f1449da2700 1 --2- 172.21.15.16:0/998663785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14440cd970 0x7f14440cdd90 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f14440b5de0 tx=0x7f143c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42d03fdae0e08fca server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:24:56.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.901+0000 7f1437fff700 1 -- 172.21.15.16:0/998663785 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f143c00e070 con 0x7f14440cd970 2023-05-20T01:24:56.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.901+0000 7f1437fff700 1 -- 172.21.15.16:0/998663785 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f143c003a20 con 0x7f14440cd970 2023-05-20T01:24:56.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.901+0000 7f1437fff700 1 -- 172.21.15.16:0/998663785 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f143c003dd0 con 0x7f14440cd970 2023-05-20T01:24:56.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.902+0000 7f144b805700 1 -- 172.21.15.16:0/998663785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14440cd970 msgr2=0x7f14440cdd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:56.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.902+0000 7f144b805700 1 --2- 172.21.15.16:0/998663785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14440cd970 0x7f14440cdd90 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f14440b5de0 tx=0x7f143c00b330 comp rx=0 tx=0).stop 2023-05-20T01:24:56.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.902+0000 7f144b805700 1 -- 172.21.15.16:0/998663785 shutdown_connections 2023-05-20T01:24:56.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.902+0000 7f144b805700 1 --2- 172.21.15.16:0/998663785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14440cd970 0x7f14440cdd90 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:56.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.902+0000 7f144b805700 1 --2- 172.21.15.16:0/998663785 >> v2:172.21.15.132:3300/0 conn(0x7f14440cfc50 0x7f14440d2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:56.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.902+0000 7f144b805700 1 -- 172.21.15.16:0/998663785 >> 172.21.15.16:0/998663785 conn(0x7f144409c590 msgr2=0x7f14440d3240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:24:56.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.902+0000 7f144b805700 1 -- 172.21.15.16:0/998663785 shutdown_connections 2023-05-20T01:24:56.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.902+0000 7f144b805700 1 -- 172.21.15.16:0/998663785 wait complete. 2023-05-20T01:24:56.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.903+0000 7f144b805700 1 Processor -- start 2023-05-20T01:24:56.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.903+0000 7f144b805700 1 -- start start 2023-05-20T01:24:56.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.904+0000 7f144b805700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f14440b4750 0x7f14441e70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:56.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.904+0000 7f144b805700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f14440cd970 0x7f14440ccb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:56.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.904+0000 7f144b805700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14440cfc50 0x7f14440b7c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:56.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.904+0000 7f144b805700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14441e5040 con 0x7f14440cfc50 2023-05-20T01:24:56.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.904+0000 7f144b805700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14441e7600 con 0x7f14440cd970 2023-05-20T01:24:56.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.904+0000 7f144b805700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14440cb290 con 0x7f14440b4750 2023-05-20T01:24:56.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.904+0000 7f1449da2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14440cfc50 0x7f14440b7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:56.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.904+0000 7f1449da2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14440cfc50 0x7f14440b7c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35782/0 (socket says 172.21.15.16:35782) 2023-05-20T01:24:56.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.904+0000 7f1449da2700 1 -- 172.21.15.16:0/968433278 learned_addr learned my addr 172.21.15.16:0/968433278 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:24:56.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.904+0000 7f14495a1700 1 --2- 172.21.15.16:0/968433278 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f14440b4750 0x7f14441e70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:56.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.904+0000 7f1448da0700 1 --2- 172.21.15.16:0/968433278 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f14440cd970 0x7f14440ccb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:56.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.904+0000 7f1449da2700 1 -- 172.21.15.16:0/968433278 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f14440b4750 msgr2=0x7f14441e70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:56.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.904+0000 7f1449da2700 1 --2- 172.21.15.16:0/968433278 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f14440b4750 0x7f14441e70c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:56.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.904+0000 7f1449da2700 1 -- 172.21.15.16:0/968433278 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f14440cd970 msgr2=0x7f14440ccb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:56.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.904+0000 7f1449da2700 1 --2- 172.21.15.16:0/968433278 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f14440cd970 0x7f14440ccb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:56.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.904+0000 7f1449da2700 1 -- 172.21.15.16:0/968433278 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f143c009b30 con 0x7f14440cfc50 2023-05-20T01:24:56.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.904+0000 7f14495a1700 1 --2- 172.21.15.16:0/968433278 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f14440b4750 0x7f14441e70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:24:56.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.904+0000 7f1449da2700 1 --2- 172.21.15.16:0/968433278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14440cfc50 0x7f14440b7c30 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f143c00b370 tx=0x7f143c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:24:56.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.904+0000 7f1435ffb700 1 -- 172.21.15.16:0/968433278 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f143c004440 con 0x7f14440cfc50 2023-05-20T01:24:56.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.904+0000 7f1435ffb700 1 -- 172.21.15.16:0/968433278 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f143c0045a0 con 0x7f14440cfc50 2023-05-20T01:24:56.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.904+0000 7f144b805700 1 -- 172.21.15.16:0/968433278 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14440b4380 con 0x7f14440cfc50 2023-05-20T01:24:56.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.905+0000 7f1435ffb700 1 -- 172.21.15.16:0/968433278 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f143c004950 con 0x7f14440cfc50 2023-05-20T01:24:56.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.905+0000 7f144b805700 1 -- 172.21.15.16:0/968433278 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f144415d0b0 con 0x7f14440cfc50 2023-05-20T01:24:56.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.906+0000 7f144b805700 1 -- 172.21.15.16:0/968433278 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14440d4000 con 0x7f14440cfc50 2023-05-20T01:24:56.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.906+0000 7f1435ffb700 1 -- 172.21.15.16:0/968433278 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f143c012430 con 0x7f14440cfc50 2023-05-20T01:24:56.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.906+0000 7f1435ffb700 1 --2- 172.21.15.16:0/968433278 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f142806ace0 0x7f142806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:56.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.907+0000 7f1435ffb700 1 -- 172.21.15.16:0/968433278 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7189+0+0 (secure 0 0 0) 0x7f143c08acb0 con 0x7f14440cfc50 2023-05-20T01:24:56.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.907+0000 7f14495a1700 1 --2- 172.21.15.16:0/968433278 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f142806ace0 0x7f142806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:56.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.907+0000 7f14495a1700 1 --2- 172.21.15.16:0/968433278 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f142806ace0 0x7f142806d190 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f1440007a50 tx=0x7f1440008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:24:56.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:56.911+0000 7f1435ffb700 1 -- 172.21.15.16:0/968433278 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f143c055980 con 0x7f14440cfc50 2023-05-20T01:24:57.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.112+0000 7f144b805700 1 -- 172.21.15.16:0/968433278 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f144404e8c0 con 0x7f14440cfc50 2023-05-20T01:24:57.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.113+0000 7f1435ffb700 1 -- 172.21.15.16:0/968433278 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) v1 ==== 74+0+20822 (secure 0 0 0) 0x7f143c058fa0 con 0x7f14440cfc50 2023-05-20T01:24:57.118 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:24:57.119 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":75,"fsid":"c2245cc5-ecab-4087-a59b-bc81db901676","created":"2023-05-20T01:16:09.549230+0000","modified":"2023-05-20T01:22:20.570959+0000","last_up_change":"2023-05-20T01:16:19.788624+0000","last_in_change":"2023-05-20T01:16:12.092160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-05-20T01:16:20.934366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-05-20T01:16:21.372294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-05-20T01:19:49.878849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-05-20T01:19:50.887306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data_extra","create_time":"2023-05-20T01:20:04.581773+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"317a83cd-19c8-4958-a301-6087ef342289","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6825","nonce":3923634179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6827","nonce":3923634179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6831","nonce":3923634179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6829","nonce":3923634179}]},"public_addr":"172.21.15.16:6825/3923634179","cluster_addr":"172.21.15.16:6827/3923634179","heartbeat_back_addr":"172.21.15.16:6831/3923634179","heartbeat_front_addr":"172.21.15.16:6829/3923634179","state":["exists","up"]},{"osd":1,"uuid":"5581046f-8219-4089-8d7d-b7846399505e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6809","nonce":3860264220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6811","nonce":3860264220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6815","nonce":3860264220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6813","nonce":3860264220}]},"public_addr":"172.21.15.16:6809/3860264220","cluster_addr":"172.21.15.16:6811/3860264220","heartbeat_back_addr":"172.21.15.16:6815/3860264220","heartbeat_front_addr":"172.21.15.16:6813/3860264220","state":["exists","up"]},{"osd":2,"uuid":"072e9e66-4133-478e-8d4a-64d46c2963f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6817","nonce":293187030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6819","nonce":293187030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6823","nonce":293187030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6821","nonce":293187030}]},"public_addr":"172.21.15.16:6817/293187030","cluster_addr":"172.21.15.16:6819/293187030","heartbeat_back_addr":"172.21.15.16:6823/293187030","heartbeat_front_addr":"172.21.15.16:6821/293187030","state":["exists","up"]},{"osd":3,"uuid":"11e1bba6-00f8-48dd-9028-d8d716b7f145","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6801","nonce":3669781890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6803","nonce":3669781890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6807","nonce":3669781890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6805","nonce":3669781890}]},"public_addr":"172.21.15.16:6801/3669781890","cluster_addr":"172.21.15.16:6803/3669781890","heartbeat_back_addr":"172.21.15.16:6807/3669781890","heartbeat_front_addr":"172.21.15.16:6805/3669781890","state":["exists","up"]},{"osd":4,"uuid":"b8909c3a-cf70-46cb-8d0b-009c707cf9d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6819","nonce":506317223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6823","nonce":506317223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6829","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6831","nonce":506317223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6827","nonce":506317223}]},"public_addr":"172.21.15.132:6819/506317223","cluster_addr":"172.21.15.132:6823/506317223","heartbeat_back_addr":"172.21.15.132:6831/506317223","heartbeat_front_addr":"172.21.15.132:6827/506317223","state":["exists","up"]},{"osd":5,"uuid":"b1818304-4583-47bf-a6ec-0bfbb5166233","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6818","nonce":1844689995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6822","nonce":1844689995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6830","nonce":1844689995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6826","nonce":1844689995}]},"public_addr":"172.21.15.132:6818/1844689995","cluster_addr":"172.21.15.132:6822/1844689995","heartbeat_back_addr":"172.21.15.132:6830/1844689995","heartbeat_front_addr":"172.21.15.132:6826/1844689995","state":["exists","up"]},{"osd":6,"uuid":"d37f4691-eed1-4a93-8818-18354f03f4a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6801","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6802","nonce":727581123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6805","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6807","nonce":727581123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6813","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6815","nonce":727581123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6811","nonce":727581123}]},"public_addr":"172.21.15.132:6802/727581123","cluster_addr":"172.21.15.132:6807/727581123","heartbeat_back_addr":"172.21.15.132:6815/727581123","heartbeat_front_addr":"172.21.15.132:6811/727581123","state":["exists","up"]},{"osd":7,"uuid":"4f712f62-5d1d-4816-8042-03209144f8b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6803","nonce":466001149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6806","nonce":466001149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6814","nonce":466001149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6810","nonce":466001149}]},"public_addr":"172.21.15.132:6803/466001149","cluster_addr":"172.21.15.132:6806/466001149","heartbeat_back_addr":"172.21.15.132:6814/466001149","heartbeat_front_addr":"172.21.15.132:6810/466001149","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.338863+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.278680+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305694+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.246835+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.276237+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.304441+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305421+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.267050+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.d","mappings":[{"from":7,"to":2}]},{"pgid":"10.10","mappings":[{"from":4,"to":3}]},{"pgid":"10.2e","mappings":[{"from":4,"to":3}]},{"pgid":"10.34","mappings":[{"from":7,"to":3}]},{"pgid":"10.37","mappings":[{"from":4,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6837/3291457857":"2023-05-21T01:22:20.558472+0000","172.21.15.16:6835/3291457857":"2023-05-21T01:22:20.558472+0000","172.21.15.16:6837/1427887654":"2023-05-21T01:22:14.372197+0000","172.21.15.16:6835/1427887654":"2023-05-21T01:22:14.372197+0000","172.21.15.16:6837/1427908556":"2023-05-21T01:22:12.737196+0000","172.21.15.16:6835/1427908556":"2023-05-21T01:22:12.737196+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-20T01:24:57.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.116+0000 7f144b805700 1 -- 172.21.15.16:0/968433278 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f142806ace0 msgr2=0x7f142806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:57.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.116+0000 7f144b805700 1 --2- 172.21.15.16:0/968433278 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f142806ace0 0x7f142806d190 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f1440007a50 tx=0x7f1440008040 comp rx=0 tx=0).stop 2023-05-20T01:24:57.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.116+0000 7f144b805700 1 -- 172.21.15.16:0/968433278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14440cfc50 msgr2=0x7f14440b7c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:57.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.116+0000 7f144b805700 1 --2- 172.21.15.16:0/968433278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14440cfc50 0x7f14440b7c30 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f143c00b370 tx=0x7f143c004ad0 comp rx=0 tx=0).stop 2023-05-20T01:24:57.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.116+0000 7f144b805700 1 -- 172.21.15.16:0/968433278 shutdown_connections 2023-05-20T01:24:57.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.116+0000 7f144b805700 1 --2- 172.21.15.16:0/968433278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14440cfc50 0x7f14440b7c30 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:57.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.116+0000 7f144b805700 1 --2- 172.21.15.16:0/968433278 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f142806ace0 0x7f142806d190 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:57.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.116+0000 7f144b805700 1 --2- 172.21.15.16:0/968433278 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f14440b4750 0x7f14441e70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:57.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.116+0000 7f144b805700 1 --2- 172.21.15.16:0/968433278 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f14440cd970 0x7f14440ccb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:57.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.116+0000 7f144b805700 1 -- 172.21.15.16:0/968433278 >> 172.21.15.16:0/968433278 conn(0x7f144409c590 msgr2=0x7f14440d0020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:24:57.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.116+0000 7f144b805700 1 -- 172.21.15.16:0/968433278 shutdown_connections 2023-05-20T01:24:57.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.116+0000 7f144b805700 1 -- 172.21.15.16:0/968433278 wait complete. 2023-05-20T01:24:57.132 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-05-20T01:24:57.133 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-20T01:24:57.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.253+0000 7f29821fe700 1 Processor -- start 2023-05-20T01:24:57.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.253+0000 7f29821fe700 1 -- start start 2023-05-20T01:24:57.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.254+0000 7f29821fe700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f297c06ac20 0x7f297c15e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:57.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.254+0000 7f29821fe700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f297c0ccf40 con 0x7f297c1585d0 2023-05-20T01:24:57.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.254+0000 7f29821fe700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f297c0cd220 con 0x7f297c0cc9c0 2023-05-20T01:24:57.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.254+0000 7f29821fe700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f297c15e540 con 0x7f297c06ac20 2023-05-20T01:24:57.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.254+0000 7f297b7fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f297c1585d0 0x7f297c15ae20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:47314/0 (socket says 172.21.15.16:47314) 2023-05-20T01:24:57.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.254+0000 7f297b7fe700 1 -- 172.21.15.16:0/1242516104 learned_addr learned my addr 172.21.15.16:0/1242516104 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:24:57.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.254+0000 7f297bfff700 1 --2- 172.21.15.16:0/1242516104 >> v2:172.21.15.132:3300/0 conn(0x7f297c06ac20 0x7f297c15e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:57.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.254+0000 7f297a7fc700 1 -- 172.21.15.16:0/1242516104 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1876059325 0 0) 0x7f297c0ccf40 con 0x7f297c1585d0 2023-05-20T01:24:57.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.254+0000 7f297a7fc700 1 -- 172.21.15.16:0/1242516104 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f29680035b0 con 0x7f297c1585d0 2023-05-20T01:24:57.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.255+0000 7f297a7fc700 1 -- 172.21.15.16:0/1242516104 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2062753533 0 0) 0x7f29680035b0 con 0x7f297c1585d0 2023-05-20T01:24:57.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.255+0000 7f297a7fc700 1 -- 172.21.15.16:0/1242516104 >> v2:172.21.15.132:3300/0 conn(0x7f297c06ac20 msgr2=0x7f297c15e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:24:57.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.255+0000 7f297a7fc700 1 --2- 172.21.15.16:0/1242516104 >> v2:172.21.15.132:3300/0 conn(0x7f297c06ac20 0x7f297c15e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:57.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.255+0000 7f297a7fc700 1 -- 172.21.15.16:0/1242516104 >> v1:172.21.15.132:6789/0 conn(0x7f297c0cc9c0 legacy=0x7f297c06a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:57.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.255+0000 7f297a7fc700 1 -- 172.21.15.16:0/1242516104 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f297c0cd500 con 0x7f297c1585d0 2023-05-20T01:24:57.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.255+0000 7f297a7fc700 1 -- 172.21.15.16:0/1242516104 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7f296c001670 con 0x7f297c1585d0 2023-05-20T01:24:57.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.255+0000 7f297a7fc700 1 -- 172.21.15.16:0/1242516104 >> v1:172.21.15.16:6789/0 conn(0x7f297c1585d0 legacy=0x7f297c15ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:57.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.255+0000 7f297a7fc700 1 --2- 172.21.15.16:0/1242516104 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2968003a70 0x7f2968003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:57.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.255+0000 7f297a7fc700 1 --2- 172.21.15.16:0/1242516104 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2968004850 0x7f2968004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:57.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.255+0000 7f297a7fc700 1 --2- 172.21.15.16:0/1242516104 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2968005200 0x7f296800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:57.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.255+0000 7f297a7fc700 1 -- 172.21.15.16:0/1242516104 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f297c15e540 con 0x7f2968004850 2023-05-20T01:24:57.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.255+0000 7f297a7fc700 1 -- 172.21.15.16:0/1242516104 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f296800c0e0 con 0x7f2968005200 2023-05-20T01:24:57.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.255+0000 7f297a7fc700 1 -- 172.21.15.16:0/1242516104 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f296800c2a0 con 0x7f2968003a70 2023-05-20T01:24:57.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.255+0000 7f297affd700 1 --2- 172.21.15.16:0/1242516104 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2968004850 0x7f2968004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:57.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.255+0000 7f297b7fe700 1 --2- 172.21.15.16:0/1242516104 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2968003a70 0x7f2968003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:57.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.255+0000 7f297bfff700 1 --2- 172.21.15.16:0/1242516104 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2968005200 0x7f296800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:57.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.255+0000 7f297affd700 1 -- 172.21.15.16:0/1242516104 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2968003a70 msgr2=0x7f2968003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:24:57.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.255+0000 7f297affd700 1 --2- 172.21.15.16:0/1242516104 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2968003a70 0x7f2968003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:57.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.255+0000 7f297affd700 1 -- 172.21.15.16:0/1242516104 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2968005200 msgr2=0x7f296800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:24:57.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.255+0000 7f297affd700 1 --2- 172.21.15.16:0/1242516104 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2968005200 0x7f296800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:57.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.255+0000 7f297affd700 1 -- 172.21.15.16:0/1242516104 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29680035b0 con 0x7f2968004850 2023-05-20T01:24:57.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.255+0000 7f297bfff700 1 --2- 172.21.15.16:0/1242516104 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2968005200 0x7f296800ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:24:57.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.255+0000 7f297bfff700 1 -- 172.21.15.16:0/1242516104 reap_dead start 2023-05-20T01:24:57.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.255+0000 7f297affd700 1 --2- 172.21.15.16:0/1242516104 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2968004850 0x7f2968004cc0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f29640017c0 tx=0x7f2968005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a09d1410ced2956 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:24:57.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.255+0000 7f297a7fc700 1 -- 172.21.15.16:0/1242516104 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29640052d0 con 0x7f2968004850 2023-05-20T01:24:57.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.255+0000 7f297a7fc700 1 -- 172.21.15.16:0/1242516104 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2964005430 con 0x7f2968004850 2023-05-20T01:24:57.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.255+0000 7f297a7fc700 1 -- 172.21.15.16:0/1242516104 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29640038c0 con 0x7f2968004850 2023-05-20T01:24:57.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.256+0000 7f29821fe700 1 -- 172.21.15.16:0/1242516104 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2968004850 msgr2=0x7f2968004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:57.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.256+0000 7f29821fe700 1 --2- 172.21.15.16:0/1242516104 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2968004850 0x7f2968004cc0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f29640017c0 tx=0x7f2968005880 comp rx=0 tx=0).stop 2023-05-20T01:24:57.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.256+0000 7f29821fe700 1 -- 172.21.15.16:0/1242516104 shutdown_connections 2023-05-20T01:24:57.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.256+0000 7f29821fe700 1 --2- 172.21.15.16:0/1242516104 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2968004850 0x7f2968004cc0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:57.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.256+0000 7f29821fe700 1 -- 172.21.15.16:0/1242516104 >> 172.21.15.16:0/1242516104 conn(0x7f297c0cffb0 msgr2=0x7f297c0d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:24:57.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.256+0000 7f29821fe700 1 -- 172.21.15.16:0/1242516104 shutdown_connections 2023-05-20T01:24:57.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.256+0000 7f29821fe700 1 -- 172.21.15.16:0/1242516104 wait complete. 2023-05-20T01:24:57.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.258+0000 7f29821fe700 1 Processor -- start 2023-05-20T01:24:57.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.258+0000 7f29821fe700 1 -- start start 2023-05-20T01:24:57.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.259+0000 7f29821fe700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2968004850 0x7f297c0b0380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:57.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.259+0000 7f29821fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f297c149e00 0x7f297c149270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:57.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.259+0000 7f29821fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f297c1497b0 0x7f297c1f0250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:57.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.259+0000 7f29821fe700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f297c149be0 con 0x7f297c149e00 2023-05-20T01:24:57.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.259+0000 7f29821fe700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f297c14a270 con 0x7f297c1497b0 2023-05-20T01:24:57.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.259+0000 7f29821fe700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f297c0b0a30 con 0x7f2968004850 2023-05-20T01:24:57.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.259+0000 7f297affd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f297c149e00 0x7f297c149270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:57.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.259+0000 7f297affd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f297c149e00 0x7f297c149270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35800/0 (socket says 172.21.15.16:35800) 2023-05-20T01:24:57.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.259+0000 7f297affd700 1 -- 172.21.15.16:0/3257653035 learned_addr learned my addr 172.21.15.16:0/3257653035 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:24:57.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.259+0000 7f297b7fe700 1 --2- 172.21.15.16:0/3257653035 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2968004850 0x7f297c0b0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:57.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.259+0000 7f297bfff700 1 --2- 172.21.15.16:0/3257653035 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f297c1497b0 0x7f297c1f0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:57.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.259+0000 7f297affd700 1 -- 172.21.15.16:0/3257653035 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2968004850 msgr2=0x7f297c0b0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:57.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.259+0000 7f297affd700 1 --2- 172.21.15.16:0/3257653035 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2968004850 0x7f297c0b0380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:57.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.259+0000 7f297affd700 1 -- 172.21.15.16:0/3257653035 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f297c1497b0 msgr2=0x7f297c1f0250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:57.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.259+0000 7f297affd700 1 --2- 172.21.15.16:0/3257653035 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f297c1497b0 0x7f297c1f0250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:57.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.259+0000 7f297affd700 1 -- 172.21.15.16:0/3257653035 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f297c0719a0 con 0x7f297c149e00 2023-05-20T01:24:57.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.259+0000 7f297b7fe700 1 --2- 172.21.15.16:0/3257653035 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2968004850 0x7f297c0b0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:24:57.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.259+0000 7f297affd700 1 --2- 172.21.15.16:0/3257653035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f297c149e00 0x7f297c149270 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f2964005040 tx=0x7f2964001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:24:57.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.260+0000 7f2973fff700 1 -- 172.21.15.16:0/3257653035 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2964001290 con 0x7f297c149e00 2023-05-20T01:24:57.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.260+0000 7f2973fff700 1 -- 172.21.15.16:0/3257653035 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f296401d480 con 0x7f297c149e00 2023-05-20T01:24:57.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.260+0000 7f29821fe700 1 -- 172.21.15.16:0/3257653035 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f297c0cd7e0 con 0x7f297c149e00 2023-05-20T01:24:57.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.260+0000 7f2973fff700 1 -- 172.21.15.16:0/3257653035 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f296401d810 con 0x7f297c149e00 2023-05-20T01:24:57.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.260+0000 7f29821fe700 1 -- 172.21.15.16:0/3257653035 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f297c072170 con 0x7f297c149e00 2023-05-20T01:24:57.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.261+0000 7f29821fe700 1 -- 172.21.15.16:0/3257653035 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f297c149b80 con 0x7f297c149e00 2023-05-20T01:24:57.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.261+0000 7f2973fff700 1 -- 172.21.15.16:0/3257653035 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2964019040 con 0x7f297c149e00 2023-05-20T01:24:57.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.261+0000 7f2973fff700 1 --2- 172.21.15.16:0/3257653035 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f295c06ace0 0x7f295c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:57.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.262+0000 7f2973fff700 1 -- 172.21.15.16:0/3257653035 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7189+0+0 (secure 0 0 0) 0x7f296408a440 con 0x7f297c149e00 2023-05-20T01:24:57.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.262+0000 7f297b7fe700 1 --2- 172.21.15.16:0/3257653035 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f295c06ace0 0x7f295c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:57.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.262+0000 7f297b7fe700 1 --2- 172.21.15.16:0/3257653035 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f295c06ace0 0x7f295c06d190 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f296c000c00 tx=0x7f296c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:24:57.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.266+0000 7f2973fff700 1 -- 172.21.15.16:0/3257653035 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f297c149b80 con 0x7f297c149e00 2023-05-20T01:24:57.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.509+0000 7f29821fe700 1 -- 172.21.15.16:0/3257653035 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f297c0747e0 con 0x7f297c149e00 2023-05-20T01:24:57.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.509+0000 7f2973fff700 1 -- 172.21.15.16:0/3257653035 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+4679 (secure 0 0 0) 0x7f2964055230 con 0x7f297c149e00 2023-05-20T01:24:57.514 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:24:57.514 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":49,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5873,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3769502841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3769502841},{"type":"v1","addr":"172.21.15.16:6835","nonce":3769502841}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45},{"gid":5879,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/688624373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":688624373},{"type":"v1","addr":"172.21.15.132:6834","nonce":688624373}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45},{"gid":6307,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/1655712081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":1655712081},{"type":"v1","addr":"172.21.15.132:6835","nonce":1655712081}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-20T01:22:14.127671+0000","modified":"2023-05-20T01:22:24.390190+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":72,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6768},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6768":{"gid":6768,"name":"c","rank":0,"incarnation":46,"state":"up:active","state_seq":4,"addr":"172.21.15.16:6837/2351975329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2351975329},{"type":"v1","addr":"172.21.15.16:6837","nonce":2351975329}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10,11],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-05-20T01:24:57.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.511+0000 7f29821fe700 1 -- 172.21.15.16:0/3257653035 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f295c06ace0 msgr2=0x7f295c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:57.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.511+0000 7f29821fe700 1 --2- 172.21.15.16:0/3257653035 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f295c06ace0 0x7f295c06d190 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f296c000c00 tx=0x7f296c009040 comp rx=0 tx=0).stop 2023-05-20T01:24:57.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.511+0000 7f29821fe700 1 -- 172.21.15.16:0/3257653035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f297c149e00 msgr2=0x7f297c149270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:57.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.511+0000 7f29821fe700 1 --2- 172.21.15.16:0/3257653035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f297c149e00 0x7f297c149270 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f2964005040 tx=0x7f2964001120 comp rx=0 tx=0).stop 2023-05-20T01:24:57.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.511+0000 7f29821fe700 1 -- 172.21.15.16:0/3257653035 shutdown_connections 2023-05-20T01:24:57.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.511+0000 7f29821fe700 1 --2- 172.21.15.16:0/3257653035 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f297c1497b0 0x7f297c1f0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:57.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.511+0000 7f29821fe700 1 --2- 172.21.15.16:0/3257653035 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f295c06ace0 0x7f295c06d190 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:57.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.511+0000 7f29821fe700 1 --2- 172.21.15.16:0/3257653035 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2968004850 0x7f297c0b0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:57.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.511+0000 7f29821fe700 1 --2- 172.21.15.16:0/3257653035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f297c149e00 0x7f297c149270 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:57.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.511+0000 7f29821fe700 1 -- 172.21.15.16:0/3257653035 >> 172.21.15.16:0/3257653035 conn(0x7f297c0cffb0 msgr2=0x7f297c157890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:24:57.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.512+0000 7f29821fe700 1 -- 172.21.15.16:0/3257653035 shutdown_connections 2023-05-20T01:24:57.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.512+0000 7f29821fe700 1 -- 172.21.15.16:0/3257653035 wait complete. 2023-05-20T01:24:57.518 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 49 2023-05-20T01:24:57.528 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-20T01:24:57.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.649+0000 7fc300ca6700 1 Processor -- start 2023-05-20T01:24:57.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.649+0000 7fc300ca6700 1 -- start start 2023-05-20T01:24:57.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.650+0000 7fc300ca6700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc2fc0b2760 0x7fc2fc0d1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:57.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.650+0000 7fc300ca6700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc2fc144020 0x7fc2fc0b0980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:57.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.650+0000 7fc300ca6700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fc2fc0cec60 0x7fc2fc0cf0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:57.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.650+0000 7fc300ca6700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2fc0cb6e0 con 0x7fc2fc144020 2023-05-20T01:24:57.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.650+0000 7fc300ca6700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2fc0cf620 con 0x7fc2fc0cec60 2023-05-20T01:24:57.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.650+0000 7fc300ca6700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2fc1448a0 con 0x7fc2fc0b2760 2023-05-20T01:24:57.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.650+0000 7fc2f9d9b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc2fc144020 0x7fc2fc0b0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:57.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.650+0000 7fc2f9d9b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc2fc144020 0x7fc2fc0b0980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35808/0 (socket says 172.21.15.16:35808) 2023-05-20T01:24:57.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.650+0000 7fc2f9d9b700 1 -- 172.21.15.16:0/2201699383 learned_addr learned my addr 172.21.15.16:0/2201699383 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:24:57.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.650+0000 7fc2fa59c700 1 --2- 172.21.15.16:0/2201699383 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc2fc0b2760 0x7fc2fc0d1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:57.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.650+0000 7fc2fad9d700 1 --2- 172.21.15.16:0/2201699383 >> v2:172.21.15.132:3300/0 conn(0x7fc2fc0cec60 0x7fc2fc0cf0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:57.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.650+0000 7fc2f9d9b700 1 -- 172.21.15.16:0/2201699383 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc2fc0b2760 msgr2=0x7fc2fc0d1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:24:57.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.650+0000 7fc2f9d9b700 1 --2- 172.21.15.16:0/2201699383 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc2fc0b2760 0x7fc2fc0d1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:57.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.650+0000 7fc2f9d9b700 1 -- 172.21.15.16:0/2201699383 >> v2:172.21.15.132:3300/0 conn(0x7fc2fc0cec60 msgr2=0x7fc2fc0cf0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:24:57.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.650+0000 7fc2f9d9b700 1 --2- 172.21.15.16:0/2201699383 >> v2:172.21.15.132:3300/0 conn(0x7fc2fc0cec60 0x7fc2fc0cf0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:57.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.650+0000 7fc2f9d9b700 1 -- 172.21.15.16:0/2201699383 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2fc0cc500 con 0x7fc2fc144020 2023-05-20T01:24:57.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.650+0000 7fc2fad9d700 1 --2- 172.21.15.16:0/2201699383 >> v2:172.21.15.132:3300/0 conn(0x7fc2fc0cec60 0x7fc2fc0cf0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:24:57.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.650+0000 7fc2fa59c700 1 --2- 172.21.15.16:0/2201699383 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc2fc0b2760 0x7fc2fc0d1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:24:57.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.650+0000 7fc2f9d9b700 1 --2- 172.21.15.16:0/2201699383 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2fc144020 0x7fc2fc0b0980 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc071900 tx=0x7fc2f0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d6f64f762df2c12 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:24:57.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.651+0000 7fc2f959a700 1 -- 172.21.15.16:0/2201699383 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2f000b870 con 0x7fc2fc144020 2023-05-20T01:24:57.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.651+0000 7fc2f959a700 1 -- 172.21.15.16:0/2201699383 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2f000b9d0 con 0x7fc2fc144020 2023-05-20T01:24:57.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.651+0000 7fc2f959a700 1 -- 172.21.15.16:0/2201699383 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2f000bd60 con 0x7fc2fc144020 2023-05-20T01:24:57.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.651+0000 7fc300ca6700 1 -- 172.21.15.16:0/2201699383 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2fc144020 msgr2=0x7fc2fc0b0980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:57.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.651+0000 7fc300ca6700 1 --2- 172.21.15.16:0/2201699383 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2fc144020 0x7fc2fc0b0980 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc071900 tx=0x7fc2f0009bd0 comp rx=0 tx=0).stop 2023-05-20T01:24:57.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.651+0000 7fc300ca6700 1 -- 172.21.15.16:0/2201699383 shutdown_connections 2023-05-20T01:24:57.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.651+0000 7fc300ca6700 1 --2- 172.21.15.16:0/2201699383 >> v2:172.21.15.132:3300/0 conn(0x7fc2fc0cec60 0x7fc2fc0cf0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:57.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.651+0000 7fc300ca6700 1 --2- 172.21.15.16:0/2201699383 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc2fc0b2760 0x7fc2fc0d1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:57.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.651+0000 7fc300ca6700 1 --2- 172.21.15.16:0/2201699383 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2fc144020 0x7fc2fc0b0980 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:57.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.651+0000 7fc300ca6700 1 -- 172.21.15.16:0/2201699383 >> 172.21.15.16:0/2201699383 conn(0x7fc2fc06ae30 msgr2=0x7fc2fc09a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:24:57.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.651+0000 7fc300ca6700 1 -- 172.21.15.16:0/2201699383 shutdown_connections 2023-05-20T01:24:57.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.651+0000 7fc300ca6700 1 -- 172.21.15.16:0/2201699383 wait complete. 2023-05-20T01:24:57.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.653+0000 7fc300ca6700 1 Processor -- start 2023-05-20T01:24:57.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.653+0000 7fc300ca6700 1 -- start start 2023-05-20T01:24:57.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.653+0000 7fc300ca6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2fc0b2760 0x7fc2fc0b4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:57.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.654+0000 7fc300ca6700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc2fc0cec60 0x7fc2fc1f36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:57.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.654+0000 7fc300ca6700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc2fc144020 0x7fc2fc1f3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:57.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.654+0000 7fc300ca6700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2fc0cadb0 con 0x7fc2fc0b2760 2023-05-20T01:24:57.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.654+0000 7fc300ca6700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2fc0ca130 con 0x7fc2fc144020 2023-05-20T01:24:57.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.654+0000 7fc300ca6700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2fc0ca410 con 0x7fc2fc0cec60 2023-05-20T01:24:57.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.654+0000 7fc2fa59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2fc0b2760 0x7fc2fc0b4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:57.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.654+0000 7fc2fa59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2fc0b2760 0x7fc2fc0b4050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35812/0 (socket says 172.21.15.16:35812) 2023-05-20T01:24:57.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.654+0000 7fc2fa59c700 1 -- 172.21.15.16:0/2217379690 learned_addr learned my addr 172.21.15.16:0/2217379690 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:24:57.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.654+0000 7fc2f9d9b700 1 --2- 172.21.15.16:0/2217379690 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc2fc0cec60 0x7fc2fc1f36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:57.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.654+0000 7fc2fad9d700 1 --2- 172.21.15.16:0/2217379690 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc2fc144020 0x7fc2fc1f3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:57.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.654+0000 7fc2fa59c700 1 -- 172.21.15.16:0/2217379690 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc2fc0cec60 msgr2=0x7fc2fc1f36b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:57.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.654+0000 7fc2fa59c700 1 --2- 172.21.15.16:0/2217379690 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc2fc0cec60 0x7fc2fc1f36b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:57.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.654+0000 7fc2fa59c700 1 -- 172.21.15.16:0/2217379690 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc2fc144020 msgr2=0x7fc2fc1f3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:57.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.654+0000 7fc2fa59c700 1 --2- 172.21.15.16:0/2217379690 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc2fc144020 0x7fc2fc1f3bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:57.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.654+0000 7fc2fa59c700 1 -- 172.21.15.16:0/2217379690 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2f0009550 con 0x7fc2fc0b2760 2023-05-20T01:24:57.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.654+0000 7fc2fad9d700 1 --2- 172.21.15.16:0/2217379690 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc2fc144020 0x7fc2fc1f3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:24:57.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.654+0000 7fc2fa59c700 1 --2- 172.21.15.16:0/2217379690 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2fc0b2760 0x7fc2fc0b4050 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec00c7f0 tx=0x7fc2ec00cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:24:57.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.654+0000 7fc2eaffd700 1 -- 172.21.15.16:0/2217379690 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2ec011800 con 0x7fc2fc0b2760 2023-05-20T01:24:57.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.654+0000 7fc2eaffd700 1 -- 172.21.15.16:0/2217379690 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2ec011960 con 0x7fc2fc0b2760 2023-05-20T01:24:57.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.654+0000 7fc300ca6700 1 -- 172.21.15.16:0/2217379690 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2fc0cb8f0 con 0x7fc2fc0b2760 2023-05-20T01:24:57.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.654+0000 7fc2eaffd700 1 -- 172.21.15.16:0/2217379690 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2ec011d10 con 0x7fc2fc0b2760 2023-05-20T01:24:57.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.654+0000 7fc300ca6700 1 -- 172.21.15.16:0/2217379690 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2fc0b0ec0 con 0x7fc2fc0b2760 2023-05-20T01:24:57.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.655+0000 7fc300ca6700 1 -- 172.21.15.16:0/2217379690 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2fc0cbbd0 con 0x7fc2fc0b2760 2023-05-20T01:24:57.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.656+0000 7fc2eaffd700 1 -- 172.21.15.16:0/2217379690 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc2ec018360 con 0x7fc2fc0b2760 2023-05-20T01:24:57.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.656+0000 7fc2eaffd700 1 --2- 172.21.15.16:0/2217379690 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc2dc06ace0 0x7fc2dc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:57.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.656+0000 7fc2eaffd700 1 -- 172.21.15.16:0/2217379690 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7189+0+0 (secure 0 0 0) 0x7fc2ec089850 con 0x7fc2fc0b2760 2023-05-20T01:24:57.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.656+0000 7fc2f9d9b700 1 --2- 172.21.15.16:0/2217379690 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc2dc06ace0 0x7fc2dc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:57.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.657+0000 7fc2f9d9b700 1 --2- 172.21.15.16:0/2217379690 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc2dc06ace0 0x7fc2dc06d190 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fc2f0000c00 tx=0x7fc2f0008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:24:57.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.661+0000 7fc2eaffd700 1 -- 172.21.15.16:0/2217379690 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fc2ec054410 con 0x7fc2fc0b2760 2023-05-20T01:24:57.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.903+0000 7fc300ca6700 1 -- 172.21.15.16:0/2217379690 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc2fc0747b0 con 0x7fc2fc0b2760 2023-05-20T01:24:57.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.904+0000 7fc2eaffd700 1 -- 172.21.15.16:0/2217379690 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+4679 (secure 0 0 0) 0x7fc2ec057a30 con 0x7fc2fc0b2760 2023-05-20T01:24:57.908 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:24:57.908 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":49,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5873,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3769502841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3769502841},{"type":"v1","addr":"172.21.15.16:6835","nonce":3769502841}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45},{"gid":5879,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/688624373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":688624373},{"type":"v1","addr":"172.21.15.132:6834","nonce":688624373}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45},{"gid":6307,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/1655712081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":1655712081},{"type":"v1","addr":"172.21.15.132:6835","nonce":1655712081}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-20T01:22:14.127671+0000","modified":"2023-05-20T01:22:24.390190+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":72,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6768},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6768":{"gid":6768,"name":"c","rank":0,"incarnation":46,"state":"up:active","state_seq":4,"addr":"172.21.15.16:6837/2351975329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2351975329},{"type":"v1","addr":"172.21.15.16:6837","nonce":2351975329}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10,11],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-05-20T01:24:57.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.906+0000 7fc300ca6700 1 -- 172.21.15.16:0/2217379690 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc2dc06ace0 msgr2=0x7fc2dc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:57.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.906+0000 7fc300ca6700 1 --2- 172.21.15.16:0/2217379690 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc2dc06ace0 0x7fc2dc06d190 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fc2f0000c00 tx=0x7fc2f0008030 comp rx=0 tx=0).stop 2023-05-20T01:24:57.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.906+0000 7fc300ca6700 1 -- 172.21.15.16:0/2217379690 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2fc0b2760 msgr2=0x7fc2fc0b4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:57.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.906+0000 7fc300ca6700 1 --2- 172.21.15.16:0/2217379690 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2fc0b2760 0x7fc2fc0b4050 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec00c7f0 tx=0x7fc2ec00cb00 comp rx=0 tx=0).stop 2023-05-20T01:24:57.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.906+0000 7fc300ca6700 1 -- 172.21.15.16:0/2217379690 shutdown_connections 2023-05-20T01:24:57.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.906+0000 7fc300ca6700 1 --2- 172.21.15.16:0/2217379690 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc2fc144020 0x7fc2fc1f3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:57.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.906+0000 7fc300ca6700 1 --2- 172.21.15.16:0/2217379690 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2fc0b2760 0x7fc2fc0b4050 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:57.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.906+0000 7fc300ca6700 1 --2- 172.21.15.16:0/2217379690 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc2dc06ace0 0x7fc2dc06d190 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:57.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.906+0000 7fc300ca6700 1 --2- 172.21.15.16:0/2217379690 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc2fc0cec60 0x7fc2fc1f36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:57.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.906+0000 7fc300ca6700 1 -- 172.21.15.16:0/2217379690 >> 172.21.15.16:0/2217379690 conn(0x7fc2fc06ae30 msgr2=0x7fc2fc06e390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:24:57.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.906+0000 7fc300ca6700 1 -- 172.21.15.16:0/2217379690 shutdown_connections 2023-05-20T01:24:57.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:57.906+0000 7fc300ca6700 1 -- 172.21.15.16:0/2217379690 wait complete. 2023-05-20T01:24:57.915 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 49 2023-05-20T01:24:57.922 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-20T01:24:58.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.042+0000 7ffa486b9700 1 Processor -- start 2023-05-20T01:24:58.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.042+0000 7ffa486b9700 1 -- start start 2023-05-20T01:24:58.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.043+0000 7ffa486b9700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7ffa400ccec0 0x7ffa400713c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:58.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.043+0000 7ffa486b9700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa400cb590 con 0x7ffa40071900 2023-05-20T01:24:58.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.043+0000 7ffa486b9700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa400ca9a0 con 0x7ffa40157190 2023-05-20T01:24:58.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.043+0000 7ffa486b9700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa400cac60 con 0x7ffa400ccec0 2023-05-20T01:24:58.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.044+0000 7ffa46c56700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ffa40071900 0x7ffa400ce740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:47352/0 (socket says 172.21.15.16:47352) 2023-05-20T01:24:58.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.044+0000 7ffa46c56700 1 -- 172.21.15.16:0/741882643 learned_addr learned my addr 172.21.15.16:0/741882643 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:24:58.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.044+0000 7ffa45c54700 1 --2- 172.21.15.16:0/741882643 >> v2:172.21.15.132:3300/0 conn(0x7ffa400ccec0 0x7ffa400713c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:58.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.044+0000 7ffa45453700 1 -- 172.21.15.16:0/741882643 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 631246177 0 0) 0x7ffa400cb590 con 0x7ffa40071900 2023-05-20T01:24:58.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.044+0000 7ffa45453700 1 -- 172.21.15.16:0/741882643 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa2c0035b0 con 0x7ffa40071900 2023-05-20T01:24:58.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.044+0000 7ffa45453700 1 -- 172.21.15.16:0/741882643 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3428969649 0 0) 0x7ffa2c0035b0 con 0x7ffa40071900 2023-05-20T01:24:58.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.044+0000 7ffa45453700 1 -- 172.21.15.16:0/741882643 >> v2:172.21.15.132:3300/0 conn(0x7ffa400ccec0 msgr2=0x7ffa400713c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:24:58.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.044+0000 7ffa45453700 1 --2- 172.21.15.16:0/741882643 >> v2:172.21.15.132:3300/0 conn(0x7ffa400ccec0 0x7ffa400713c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:58.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.044+0000 7ffa45453700 1 -- 172.21.15.16:0/741882643 >> v1:172.21.15.132:6789/0 conn(0x7ffa40157190 legacy=0x7ffa400cc7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:58.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.044+0000 7ffa45453700 1 -- 172.21.15.16:0/741882643 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa400caf40 con 0x7ffa40071900 2023-05-20T01:24:58.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.044+0000 7ffa45453700 1 -- 172.21.15.16:0/741882643 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7ffa380019f0 con 0x7ffa40071900 2023-05-20T01:24:58.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.044+0000 7ffa45453700 1 -- 172.21.15.16:0/741882643 >> v1:172.21.15.16:6789/0 conn(0x7ffa40071900 legacy=0x7ffa400ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:58.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.044+0000 7ffa45453700 1 --2- 172.21.15.16:0/741882643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa2c003a70 0x7ffa2c003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:58.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.044+0000 7ffa45453700 1 --2- 172.21.15.16:0/741882643 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffa2c004850 0x7ffa2c004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:58.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.044+0000 7ffa45453700 1 --2- 172.21.15.16:0/741882643 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffa2c005250 0x7ffa2c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:58.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.044+0000 7ffa45453700 1 -- 172.21.15.16:0/741882643 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa400cac60 con 0x7ffa2c003a70 2023-05-20T01:24:58.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.044+0000 7ffa45453700 1 -- 172.21.15.16:0/741882643 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa2c00c190 con 0x7ffa2c005250 2023-05-20T01:24:58.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.044+0000 7ffa45453700 1 -- 172.21.15.16:0/741882643 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa2c00c350 con 0x7ffa2c004850 2023-05-20T01:24:58.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.044+0000 7ffa46455700 1 --2- 172.21.15.16:0/741882643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa2c003a70 0x7ffa2c003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:58.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.044+0000 7ffa45c54700 1 --2- 172.21.15.16:0/741882643 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffa2c004850 0x7ffa2c004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:58.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.044+0000 7ffa46c56700 1 --2- 172.21.15.16:0/741882643 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffa2c005250 0x7ffa2c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:58.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.045+0000 7ffa46455700 1 -- 172.21.15.16:0/741882643 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffa2c004850 msgr2=0x7ffa2c004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:24:58.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.045+0000 7ffa46455700 1 --2- 172.21.15.16:0/741882643 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffa2c004850 0x7ffa2c004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:58.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.045+0000 7ffa46455700 1 -- 172.21.15.16:0/741882643 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffa2c005250 msgr2=0x7ffa2c00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:24:58.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.045+0000 7ffa46455700 1 --2- 172.21.15.16:0/741882643 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffa2c005250 0x7ffa2c00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:58.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.045+0000 7ffa46455700 1 -- 172.21.15.16:0/741882643 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa2c0035b0 con 0x7ffa2c003a70 2023-05-20T01:24:58.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.045+0000 7ffa46c56700 1 --2- 172.21.15.16:0/741882643 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffa2c005250 0x7ffa2c00ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:24:58.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.045+0000 7ffa46c56700 1 -- 172.21.15.16:0/741882643 reap_dead start 2023-05-20T01:24:58.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.045+0000 7ffa46455700 1 --2- 172.21.15.16:0/741882643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa2c003a70 0x7ffa2c003e80 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c007fd0 tx=0x7ffa2c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d8d8260f4a0b999 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:24:58.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.045+0000 7ffa45453700 1 -- 172.21.15.16:0/741882643 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa3c0052d0 con 0x7ffa2c003a70 2023-05-20T01:24:58.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.045+0000 7ffa45453700 1 -- 172.21.15.16:0/741882643 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa3c005430 con 0x7ffa2c003a70 2023-05-20T01:24:58.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.045+0000 7ffa45453700 1 -- 172.21.15.16:0/741882643 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa3c0038c0 con 0x7ffa2c003a70 2023-05-20T01:24:58.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.045+0000 7ffa486b9700 1 -- 172.21.15.16:0/741882643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa2c003a70 msgr2=0x7ffa2c003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:58.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.045+0000 7ffa486b9700 1 --2- 172.21.15.16:0/741882643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa2c003a70 0x7ffa2c003e80 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c007fd0 tx=0x7ffa2c005880 comp rx=0 tx=0).stop 2023-05-20T01:24:58.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.045+0000 7ffa486b9700 1 -- 172.21.15.16:0/741882643 shutdown_connections 2023-05-20T01:24:58.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.045+0000 7ffa486b9700 1 --2- 172.21.15.16:0/741882643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa2c003a70 0x7ffa2c003e80 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:58.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.045+0000 7ffa486b9700 1 -- 172.21.15.16:0/741882643 >> 172.21.15.16:0/741882643 conn(0x7ffa400cffb0 msgr2=0x7ffa400d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:24:58.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.045+0000 7ffa486b9700 1 -- 172.21.15.16:0/741882643 shutdown_connections 2023-05-20T01:24:58.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.046+0000 7ffa486b9700 1 -- 172.21.15.16:0/741882643 wait complete. 2023-05-20T01:24:58.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.047+0000 7ffa486b9700 1 Processor -- start 2023-05-20T01:24:58.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.047+0000 7ffa486b9700 1 -- start start 2023-05-20T01:24:58.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.047+0000 7ffa486b9700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffa2c003a70 0x7ffa400c76b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:58.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.047+0000 7ffa486b9700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa40143330 0x7ffa401437a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:58.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.047+0000 7ffa486b9700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffa400c9c70 0x7ffa400ca0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:58.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.047+0000 7ffa486b9700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa401f0360 con 0x7ffa40143330 2023-05-20T01:24:58.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.047+0000 7ffa486b9700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa400c97e0 con 0x7ffa2c003a70 2023-05-20T01:24:58.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.047+0000 7ffa486b9700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa40145f10 con 0x7ffa400c9c70 2023-05-20T01:24:58.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.048+0000 7ffa45c54700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa40143330 0x7ffa401437a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:58.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.048+0000 7ffa45c54700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa40143330 0x7ffa401437a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35834/0 (socket says 172.21.15.16:35834) 2023-05-20T01:24:58.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.048+0000 7ffa45c54700 1 -- 172.21.15.16:0/3288468328 learned_addr learned my addr 172.21.15.16:0/3288468328 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:24:58.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.048+0000 7ffa46455700 1 --2- 172.21.15.16:0/3288468328 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffa2c003a70 0x7ffa400c76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:58.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.048+0000 7ffa46c56700 1 --2- 172.21.15.16:0/3288468328 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffa400c9c70 0x7ffa400ca0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:58.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.048+0000 7ffa45c54700 1 -- 172.21.15.16:0/3288468328 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffa400c9c70 msgr2=0x7ffa400ca0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:58.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.048+0000 7ffa45c54700 1 --2- 172.21.15.16:0/3288468328 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffa400c9c70 0x7ffa400ca0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:58.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.048+0000 7ffa45c54700 1 -- 172.21.15.16:0/3288468328 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffa2c003a70 msgr2=0x7ffa400c76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:58.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.048+0000 7ffa45c54700 1 --2- 172.21.15.16:0/3288468328 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffa2c003a70 0x7ffa400c76b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:58.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.048+0000 7ffa45c54700 1 -- 172.21.15.16:0/3288468328 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa40157560 con 0x7ffa40143330 2023-05-20T01:24:58.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.048+0000 7ffa46c56700 1 --2- 172.21.15.16:0/3288468328 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffa400c9c70 0x7ffa400ca0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:24:58.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.048+0000 7ffa46455700 1 --2- 172.21.15.16:0/3288468328 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffa2c003a70 0x7ffa400c76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:24:58.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.048+0000 7ffa45c54700 1 --2- 172.21.15.16:0/3288468328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa40143330 0x7ffa401437a0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7ffa34011ad0 tx=0x7ffa34011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:24:58.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.048+0000 7ffa32ffd700 1 -- 172.21.15.16:0/3288468328 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa3400fa00 con 0x7ffa40143330 2023-05-20T01:24:58.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.048+0000 7ffa32ffd700 1 -- 172.21.15.16:0/3288468328 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa3400fb60 con 0x7ffa40143330 2023-05-20T01:24:58.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.048+0000 7ffa486b9700 1 -- 172.21.15.16:0/3288468328 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa400cb220 con 0x7ffa40143330 2023-05-20T01:24:58.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.048+0000 7ffa32ffd700 1 -- 172.21.15.16:0/3288468328 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa34014440 con 0x7ffa40143330 2023-05-20T01:24:58.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.048+0000 7ffa486b9700 1 -- 172.21.15.16:0/3288468328 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa400cba20 con 0x7ffa40143330 2023-05-20T01:24:58.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.049+0000 7ffa486b9700 1 -- 172.21.15.16:0/3288468328 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa4006f3c0 con 0x7ffa40143330 2023-05-20T01:24:58.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.050+0000 7ffa32ffd700 1 -- 172.21.15.16:0/3288468328 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ffa340145a0 con 0x7ffa40143330 2023-05-20T01:24:58.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.050+0000 7ffa32ffd700 1 --2- 172.21.15.16:0/3288468328 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ffa2406ace0 0x7ffa2406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:58.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.050+0000 7ffa46455700 1 --2- 172.21.15.16:0/3288468328 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ffa2406ace0 0x7ffa2406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:58.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.050+0000 7ffa32ffd700 1 -- 172.21.15.16:0/3288468328 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7189+0+0 (secure 0 0 0) 0x7ffa34089fb0 con 0x7ffa40143330 2023-05-20T01:24:58.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.050+0000 7ffa46455700 1 --2- 172.21.15.16:0/3288468328 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ffa2406ace0 0x7ffa2406d190 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c015040 tx=0x7ffa3c009030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:24:58.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.055+0000 7ffa32ffd700 1 -- 172.21.15.16:0/3288468328 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7ffa340bc2d0 con 0x7ffa40143330 2023-05-20T01:24:58.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.253+0000 7ffa486b9700 1 -- 172.21.15.16:0/3288468328 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffa400747b0 con 0x7ffa40143330 2023-05-20T01:24:58.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.254+0000 7ffa32ffd700 1 -- 172.21.15.16:0/3288468328 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) v1 ==== 74+0+20822 (secure 0 0 0) 0x7ffa34054bf0 con 0x7ffa40143330 2023-05-20T01:24:58.259 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:24:58.259 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":75,"fsid":"c2245cc5-ecab-4087-a59b-bc81db901676","created":"2023-05-20T01:16:09.549230+0000","modified":"2023-05-20T01:22:20.570959+0000","last_up_change":"2023-05-20T01:16:19.788624+0000","last_in_change":"2023-05-20T01:16:12.092160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-05-20T01:16:20.934366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-05-20T01:16:21.372294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-05-20T01:19:49.878849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-05-20T01:19:50.887306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data_extra","create_time":"2023-05-20T01:20:04.581773+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"317a83cd-19c8-4958-a301-6087ef342289","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6825","nonce":3923634179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6827","nonce":3923634179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6831","nonce":3923634179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6829","nonce":3923634179}]},"public_addr":"172.21.15.16:6825/3923634179","cluster_addr":"172.21.15.16:6827/3923634179","heartbeat_back_addr":"172.21.15.16:6831/3923634179","heartbeat_front_addr":"172.21.15.16:6829/3923634179","state":["exists","up"]},{"osd":1,"uuid":"5581046f-8219-4089-8d7d-b7846399505e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6809","nonce":3860264220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6811","nonce":3860264220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6815","nonce":3860264220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6813","nonce":3860264220}]},"public_addr":"172.21.15.16:6809/3860264220","cluster_addr":"172.21.15.16:6811/3860264220","heartbeat_back_addr":"172.21.15.16:6815/3860264220","heartbeat_front_addr":"172.21.15.16:6813/3860264220","state":["exists","up"]},{"osd":2,"uuid":"072e9e66-4133-478e-8d4a-64d46c2963f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6817","nonce":293187030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6819","nonce":293187030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6823","nonce":293187030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6821","nonce":293187030}]},"public_addr":"172.21.15.16:6817/293187030","cluster_addr":"172.21.15.16:6819/293187030","heartbeat_back_addr":"172.21.15.16:6823/293187030","heartbeat_front_addr":"172.21.15.16:6821/293187030","state":["exists","up"]},{"osd":3,"uuid":"11e1bba6-00f8-48dd-9028-d8d716b7f145","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6801","nonce":3669781890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6803","nonce":3669781890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6807","nonce":3669781890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6805","nonce":3669781890}]},"public_addr":"172.21.15.16:6801/3669781890","cluster_addr":"172.21.15.16:6803/3669781890","heartbeat_back_addr":"172.21.15.16:6807/3669781890","heartbeat_front_addr":"172.21.15.16:6805/3669781890","state":["exists","up"]},{"osd":4,"uuid":"b8909c3a-cf70-46cb-8d0b-009c707cf9d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6819","nonce":506317223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6823","nonce":506317223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6829","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6831","nonce":506317223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6827","nonce":506317223}]},"public_addr":"172.21.15.132:6819/506317223","cluster_addr":"172.21.15.132:6823/506317223","heartbeat_back_addr":"172.21.15.132:6831/506317223","heartbeat_front_addr":"172.21.15.132:6827/506317223","state":["exists","up"]},{"osd":5,"uuid":"b1818304-4583-47bf-a6ec-0bfbb5166233","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6818","nonce":1844689995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6822","nonce":1844689995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6830","nonce":1844689995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6826","nonce":1844689995}]},"public_addr":"172.21.15.132:6818/1844689995","cluster_addr":"172.21.15.132:6822/1844689995","heartbeat_back_addr":"172.21.15.132:6830/1844689995","heartbeat_front_addr":"172.21.15.132:6826/1844689995","state":["exists","up"]},{"osd":6,"uuid":"d37f4691-eed1-4a93-8818-18354f03f4a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6801","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6802","nonce":727581123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6805","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6807","nonce":727581123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6813","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6815","nonce":727581123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6811","nonce":727581123}]},"public_addr":"172.21.15.132:6802/727581123","cluster_addr":"172.21.15.132:6807/727581123","heartbeat_back_addr":"172.21.15.132:6815/727581123","heartbeat_front_addr":"172.21.15.132:6811/727581123","state":["exists","up"]},{"osd":7,"uuid":"4f712f62-5d1d-4816-8042-03209144f8b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6803","nonce":466001149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6806","nonce":466001149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6814","nonce":466001149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6810","nonce":466001149}]},"public_addr":"172.21.15.132:6803/466001149","cluster_addr":"172.21.15.132:6806/466001149","heartbeat_back_addr":"172.21.15.132:6814/466001149","heartbeat_front_addr":"172.21.15.132:6810/466001149","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.338863+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.278680+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305694+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.246835+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.276237+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.304441+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305421+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.267050+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.d","mappings":[{"from":7,"to":2}]},{"pgid":"10.10","mappings":[{"from":4,"to":3}]},{"pgid":"10.2e","mappings":[{"from":4,"to":3}]},{"pgid":"10.34","mappings":[{"from":7,"to":3}]},{"pgid":"10.37","mappings":[{"from":4,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6837/3291457857":"2023-05-21T01:22:20.558472+0000","172.21.15.16:6835/3291457857":"2023-05-21T01:22:20.558472+0000","172.21.15.16:6837/1427887654":"2023-05-21T01:22:14.372197+0000","172.21.15.16:6835/1427887654":"2023-05-21T01:22:14.372197+0000","172.21.15.16:6837/1427908556":"2023-05-21T01:22:12.737196+0000","172.21.15.16:6835/1427908556":"2023-05-21T01:22:12.737196+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-20T01:24:58.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.257+0000 7ffa486b9700 1 -- 172.21.15.16:0/3288468328 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ffa2406ace0 msgr2=0x7ffa2406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:58.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.257+0000 7ffa486b9700 1 --2- 172.21.15.16:0/3288468328 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ffa2406ace0 0x7ffa2406d190 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c015040 tx=0x7ffa3c009030 comp rx=0 tx=0).stop 2023-05-20T01:24:58.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.257+0000 7ffa486b9700 1 -- 172.21.15.16:0/3288468328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa40143330 msgr2=0x7ffa401437a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:58.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.257+0000 7ffa486b9700 1 --2- 172.21.15.16:0/3288468328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa40143330 0x7ffa401437a0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7ffa34011ad0 tx=0x7ffa34011e90 comp rx=0 tx=0).stop 2023-05-20T01:24:58.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.257+0000 7ffa486b9700 1 -- 172.21.15.16:0/3288468328 shutdown_connections 2023-05-20T01:24:58.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.257+0000 7ffa486b9700 1 --2- 172.21.15.16:0/3288468328 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ffa2406ace0 0x7ffa2406d190 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:58.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.257+0000 7ffa486b9700 1 --2- 172.21.15.16:0/3288468328 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffa400c9c70 0x7ffa400ca0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:58.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.257+0000 7ffa486b9700 1 --2- 172.21.15.16:0/3288468328 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffa2c003a70 0x7ffa400c76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:58.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.257+0000 7ffa486b9700 1 --2- 172.21.15.16:0/3288468328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa40143330 0x7ffa401437a0 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:58.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.257+0000 7ffa486b9700 1 -- 172.21.15.16:0/3288468328 >> 172.21.15.16:0/3288468328 conn(0x7ffa400cffb0 msgr2=0x7ffa400c5af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:24:58.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.257+0000 7ffa486b9700 1 -- 172.21.15.16:0/3288468328 shutdown_connections 2023-05-20T01:24:58.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.257+0000 7ffa486b9700 1 -- 172.21.15.16:0/3288468328 wait complete. 2023-05-20T01:24:58.273 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-05-20T01:24:58.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.394+0000 7f60c8a59700 1 Processor -- start 2023-05-20T01:24:58.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.394+0000 7f60c8a59700 1 -- start start 2023-05-20T01:24:58.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.395+0000 7f60c8a59700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f60c4157190 0x7f60c40cc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:58.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.395+0000 7f60c8a59700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f60c4071ad0 0x7f60c40ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:58.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.395+0000 7f60c8a59700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60c4071f20 con 0x7f60c40cccf0 2023-05-20T01:24:58.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.395+0000 7f60c8a59700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60c40cb590 con 0x7f60c4071ad0 2023-05-20T01:24:58.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.395+0000 7f60c8a59700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60c40ca9a0 con 0x7f60c4157190 2023-05-20T01:24:58.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.395+0000 7f60c2d9d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f60c4071ad0 0x7f60c40ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:58.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.396+0000 7f60c2d9d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f60c4071ad0 0x7f60c40ce740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35844/0 (socket says 172.21.15.16:35844) 2023-05-20T01:24:58.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.396+0000 7f60c2d9d700 1 -- 172.21.15.16:0/1920390514 learned_addr learned my addr 172.21.15.16:0/1920390514 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:24:58.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.396+0000 7f60c1d9b700 1 --1- 172.21.15.16:0/1920390514 >> v1:172.21.15.16:6789/0 conn(0x7f60c40cccf0 0x7f60c40713c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:47370/0 (socket says 172.21.15.16:47370) 2023-05-20T01:24:58.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.396+0000 7f60c259c700 1 --2- 172.21.15.16:0/1920390514 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f60c4157190 0x7f60c40cc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:58.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.396+0000 7f60c2d9d700 1 -- 172.21.15.16:0/1920390514 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f60c4157190 msgr2=0x7f60c40cc7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:24:58.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.396+0000 7f60c2d9d700 1 --2- 172.21.15.16:0/1920390514 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f60c4157190 0x7f60c40cc7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:58.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.396+0000 7f60c2d9d700 1 -- 172.21.15.16:0/1920390514 >> v1:172.21.15.16:6789/0 conn(0x7f60c40cccf0 legacy=0x7f60c40713c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:58.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.396+0000 7f60c2d9d700 1 -- 172.21.15.16:0/1920390514 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60c40cac60 con 0x7f60c4071ad0 2023-05-20T01:24:58.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.396+0000 7f60c2d9d700 1 --2- 172.21.15.16:0/1920390514 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60c4071ad0 0x7f60c40ce740 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f60bc009e50 tx=0x7f60bc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cd0c9ba2b972bcf server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:24:58.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.396+0000 7f60c159a700 1 -- 172.21.15.16:0/1920390514 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60bc00e070 con 0x7f60c4071ad0 2023-05-20T01:24:58.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.396+0000 7f60c159a700 1 -- 172.21.15.16:0/1920390514 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60bc00b9c0 con 0x7f60c4071ad0 2023-05-20T01:24:58.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.396+0000 7f60c159a700 1 -- 172.21.15.16:0/1920390514 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60bc00bd70 con 0x7f60c4071ad0 2023-05-20T01:24:58.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.397+0000 7f60c8a59700 1 -- 172.21.15.16:0/1920390514 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60c4071ad0 msgr2=0x7f60c40ce740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:58.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.397+0000 7f60c8a59700 1 --2- 172.21.15.16:0/1920390514 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60c4071ad0 0x7f60c40ce740 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f60bc009e50 tx=0x7f60bc00b330 comp rx=0 tx=0).stop 2023-05-20T01:24:58.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.397+0000 7f60c8a59700 1 -- 172.21.15.16:0/1920390514 shutdown_connections 2023-05-20T01:24:58.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.397+0000 7f60c8a59700 1 --2- 172.21.15.16:0/1920390514 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60c4071ad0 0x7f60c40ce740 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:58.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.397+0000 7f60c8a59700 1 --2- 172.21.15.16:0/1920390514 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f60c4157190 0x7f60c40cc7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:58.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.397+0000 7f60c8a59700 1 -- 172.21.15.16:0/1920390514 >> 172.21.15.16:0/1920390514 conn(0x7f60c40cffb0 msgr2=0x7f60c40d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:24:58.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.397+0000 7f60c8a59700 1 -- 172.21.15.16:0/1920390514 shutdown_connections 2023-05-20T01:24:58.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.397+0000 7f60c8a59700 1 -- 172.21.15.16:0/1920390514 wait complete. 2023-05-20T01:24:58.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.398+0000 7f60c8a59700 1 Processor -- start 2023-05-20T01:24:58.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.398+0000 7f60c8a59700 1 -- start start 2023-05-20T01:24:58.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.399+0000 7f60c8a59700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f60c4071ad0 0x7f60c4156280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:58.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.399+0000 7f60c8a59700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f60c40cccf0 0x7f60c41e6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:58.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.399+0000 7f60c8a59700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60c4157190 0x7f60c41e6780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:58.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.399+0000 7f60c8a59700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60c41f2f00 con 0x7f60c4157190 2023-05-20T01:24:58.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.399+0000 7f60c8a59700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60c41f31e0 con 0x7f60c4071ad0 2023-05-20T01:24:58.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.399+0000 7f60c8a59700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60c41f34c0 con 0x7f60c40cccf0 2023-05-20T01:24:58.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.399+0000 7f60c2d9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60c4157190 0x7f60c41e6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:58.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.399+0000 7f60c2d9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60c4157190 0x7f60c41e6780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35850/0 (socket says 172.21.15.16:35850) 2023-05-20T01:24:58.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.399+0000 7f60c2d9d700 1 -- 172.21.15.16:0/1943058496 learned_addr learned my addr 172.21.15.16:0/1943058496 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:24:58.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.399+0000 7f60c1d9b700 1 --2- 172.21.15.16:0/1943058496 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f60c40cccf0 0x7f60c41e6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:58.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.399+0000 7f60c259c700 1 --2- 172.21.15.16:0/1943058496 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f60c4071ad0 0x7f60c4156280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:58.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.399+0000 7f60c2d9d700 1 -- 172.21.15.16:0/1943058496 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f60c40cccf0 msgr2=0x7f60c41e6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:58.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.399+0000 7f60c2d9d700 1 --2- 172.21.15.16:0/1943058496 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f60c40cccf0 0x7f60c41e6240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:58.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.399+0000 7f60c2d9d700 1 -- 172.21.15.16:0/1943058496 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f60c4071ad0 msgr2=0x7f60c4156280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:58.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.399+0000 7f60c2d9d700 1 --2- 172.21.15.16:0/1943058496 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f60c4071ad0 0x7f60c4156280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:58.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.399+0000 7f60c2d9d700 1 -- 172.21.15.16:0/1943058496 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60bc009b30 con 0x7f60c4157190 2023-05-20T01:24:58.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.399+0000 7f60c259c700 1 --2- 172.21.15.16:0/1943058496 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f60c4071ad0 0x7f60c4156280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:24:58.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.400+0000 7f60c2d9d700 1 --2- 172.21.15.16:0/1943058496 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60c4157190 0x7f60c41e6780 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f60c40d0f50 tx=0x7f60bc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:24:58.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.400+0000 7f60baffd700 1 -- 172.21.15.16:0/1943058496 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60bc004960 con 0x7f60c4157190 2023-05-20T01:24:58.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.400+0000 7f60baffd700 1 -- 172.21.15.16:0/1943058496 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60bc0099b0 con 0x7f60c4157190 2023-05-20T01:24:58.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.400+0000 7f60c8a59700 1 -- 172.21.15.16:0/1943058496 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60c40cb220 con 0x7f60c4157190 2023-05-20T01:24:58.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.400+0000 7f60baffd700 1 -- 172.21.15.16:0/1943058496 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60bc01c680 con 0x7f60c4157190 2023-05-20T01:24:58.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.400+0000 7f60c8a59700 1 -- 172.21.15.16:0/1943058496 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60c40caf40 con 0x7f60c4157190 2023-05-20T01:24:58.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.401+0000 7f60c8a59700 1 -- 172.21.15.16:0/1943058496 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60c4157560 con 0x7f60c4157190 2023-05-20T01:24:58.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.401+0000 7f60baffd700 1 -- 172.21.15.16:0/1943058496 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f60bc01c7e0 con 0x7f60c4157190 2023-05-20T01:24:58.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.402+0000 7f60baffd700 1 --2- 172.21.15.16:0/1943058496 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f60a406ace0 0x7f60a406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:58.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.402+0000 7f60baffd700 1 -- 172.21.15.16:0/1943058496 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7189+0+0 (secure 0 0 0) 0x7f60bc08b6c0 con 0x7f60c4157190 2023-05-20T01:24:58.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.402+0000 7f60c259c700 1 --2- 172.21.15.16:0/1943058496 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f60a406ace0 0x7f60a406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:58.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.402+0000 7f60c259c700 1 --2- 172.21.15.16:0/1943058496 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f60a406ace0 0x7f60a406d190 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f60b4009820 tx=0x7f60b4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:24:58.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.406+0000 7f60baffd700 1 -- 172.21.15.16:0/1943058496 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f60bc0562f0 con 0x7f60c4157190 2023-05-20T01:24:58.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:58.649+0000 7f60c8a59700 1 -- 172.21.15.16:0/1943058496 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f60c41f3980 con 0x7f60c4157190 2023-05-20T01:24:59.506 INFO:tasks.ceph.mon.a.smithi016.stderr:2023-05-20T01:24:59.502+0000 7f49eb8ae700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-05-20T01:24:59.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:59.512+0000 7f60baffd700 1 -- 172.21.15.16:0/1943058496 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v50) v1 ==== 161+0+0 (secure 0 0 0) 0x7f60bc059910 con 0x7f60c4157190 2023-05-20T01:24:59.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:59.515+0000 7f60c8a59700 1 -- 172.21.15.16:0/1943058496 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f60a406ace0 msgr2=0x7f60a406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:59.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:59.515+0000 7f60c8a59700 1 --2- 172.21.15.16:0/1943058496 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f60a406ace0 0x7f60a406d190 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f60b4009820 tx=0x7f60b4008040 comp rx=0 tx=0).stop 2023-05-20T01:24:59.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:59.515+0000 7f60c8a59700 1 -- 172.21.15.16:0/1943058496 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60c4157190 msgr2=0x7f60c41e6780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:59.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:59.515+0000 7f60c8a59700 1 --2- 172.21.15.16:0/1943058496 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60c4157190 0x7f60c41e6780 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f60c40d0f50 tx=0x7f60bc004ad0 comp rx=0 tx=0).stop 2023-05-20T01:24:59.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:59.515+0000 7f60c8a59700 1 -- 172.21.15.16:0/1943058496 shutdown_connections 2023-05-20T01:24:59.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:59.516+0000 7f60c8a59700 1 --2- 172.21.15.16:0/1943058496 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60c4157190 0x7f60c41e6780 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:59.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:59.516+0000 7f60c8a59700 1 --2- 172.21.15.16:0/1943058496 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f60c4071ad0 0x7f60c4156280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:59.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:59.516+0000 7f60c8a59700 1 --2- 172.21.15.16:0/1943058496 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f60a406ace0 0x7f60a406d190 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:59.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:59.516+0000 7f60c8a59700 1 --2- 172.21.15.16:0/1943058496 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f60c40cccf0 0x7f60c41e6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:59.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:59.516+0000 7f60c8a59700 1 -- 172.21.15.16:0/1943058496 >> 172.21.15.16:0/1943058496 conn(0x7f60c40cffb0 msgr2=0x7f60c40b4e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:24:59.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:59.516+0000 7f60c8a59700 1 -- 172.21.15.16:0/1943058496 shutdown_connections 2023-05-20T01:24:59.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:59.516+0000 7f60c8a59700 1 -- 172.21.15.16:0/1943058496 wait complete. 2023-05-20T01:24:59.522 INFO:teuthology.orchestra.run.smithi016.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-05-20T01:24:59.535 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-05-20T01:24:59.558 INFO:tasks.ceph.mds.c.smithi016.stdout:starting mds.c at 2023-05-20T01:24:59.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:59.667+0000 7f0adba02700 1 Processor -- start 2023-05-20T01:24:59.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:59.667+0000 7f0adba02700 1 -- start start 2023-05-20T01:24:59.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:59.668+0000 7f0adba02700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0ad40cd890 0x7f0ad40b6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:59.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:59.668+0000 7f0adba02700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ad40d1820 con 0x7f0ad40b65d0 2023-05-20T01:24:59.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:59.668+0000 7f0adba02700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ad40d2c60 con 0x7f0ad415ca90 2023-05-20T01:24:59.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:59.668+0000 7f0adba02700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ad40d0c50 con 0x7f0ad40cd890 2023-05-20T01:24:59.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:59.668+0000 7f0ad8f9d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0ad40cd890 0x7f0ad40b6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:59.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:59.668+0000 7f0ad9f9f700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f0ad40b65d0 0x7f0ad40cf570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:47398/0 (socket says 172.21.15.16:47398) 2023-05-20T01:24:59.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:59.668+0000 7f0ad9f9f700 1 -- 172.21.15.16:0/2160286849 learned_addr learned my addr 172.21.15.16:0/2160286849 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:24:59.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:59.668+0000 7f0ad8f9d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0ad40cd890 0x7f0ad40b6090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35868/0 (socket says 172.21.15.16:35868) 2023-05-20T01:24:59.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:59.668+0000 7f0ad8f9d700 1 -- 172.21.15.16:0/2160286849 >> v1:172.21.15.132:6789/0 conn(0x7f0ad415ca90 legacy=0x7f0ad40d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:24:59.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:59.668+0000 7f0ad8f9d700 1 -- 172.21.15.16:0/2160286849 >> v1:172.21.15.16:6789/0 conn(0x7f0ad40b65d0 legacy=0x7f0ad40cf570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:59.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:59.668+0000 7f0ad8f9d700 1 -- 172.21.15.16:0/2160286849 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ad40cdf00 con 0x7f0ad40cd890 2023-05-20T01:24:59.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:59.668+0000 7f0acbfff700 1 -- 172.21.15.16:0/2160286849 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 397554200 0 0) 0x7f0ad40d1820 con 0x7f0ad40b65d0 2023-05-20T01:24:59.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:59.668+0000 7f0ad8f9d700 1 --2- 172.21.15.16:0/2160286849 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ad40cd890 0x7f0ad40b6090 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f0ac4009840 tx=0x7f0ac4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b27808ec83465e1e server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:24:59.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:59.668+0000 7f0acbfff700 1 -- 172.21.15.16:0/2160286849 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ac4004230 con 0x7f0ad40cd890 2023-05-20T01:24:59.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:59.669+0000 7f0acbfff700 1 -- 172.21.15.16:0/2160286849 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ac4004390 con 0x7f0ad40cd890 2023-05-20T01:24:59.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:59.669+0000 7f0acbfff700 1 -- 172.21.15.16:0/2160286849 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ac40036a0 con 0x7f0ad40cd890 2023-05-20T01:24:59.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:59.669+0000 7f0adba02700 1 -- 172.21.15.16:0/2160286849 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ad40cd890 msgr2=0x7f0ad40b6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:59.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:59.669+0000 7f0adba02700 1 --2- 172.21.15.16:0/2160286849 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ad40cd890 0x7f0ad40b6090 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f0ac4009840 tx=0x7f0ac4009b50 comp rx=0 tx=0).stop 2023-05-20T01:24:59.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:59.669+0000 7f0adba02700 1 -- 172.21.15.16:0/2160286849 shutdown_connections 2023-05-20T01:24:59.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:59.669+0000 7f0adba02700 1 --2- 172.21.15.16:0/2160286849 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ad40cd890 0x7f0ad40b6090 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:59.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:59.669+0000 7f0adba02700 1 -- 172.21.15.16:0/2160286849 >> 172.21.15.16:0/2160286849 conn(0x7f0ad406b000 msgr2=0x7f0ad406b400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:24:59.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:59.669+0000 7f0adba02700 1 -- 172.21.15.16:0/2160286849 shutdown_connections 2023-05-20T01:24:59.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:59.669+0000 7f0adba02700 1 -- 172.21.15.16:0/2160286849 wait complete. 2023-05-20T01:24:59.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:59.670+0000 7f0adba02700 1 Processor -- start 2023-05-20T01:24:59.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:59.670+0000 7f0adba02700 1 -- start start 2023-05-20T01:24:59.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:59.671+0000 7f0adba02700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ad40b65d0 0x7f0ad40b5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:59.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:59.671+0000 7f0adba02700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0ad40cd890 0x7f0ad415dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:59.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:59.671+0000 7f0adba02700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0ad415ca90 0x7f0ad415ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:59.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:59.671+0000 7f0adba02700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ad40b41d0 con 0x7f0ad40b65d0 2023-05-20T01:24:59.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:59.671+0000 7f0adba02700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ad40b44b0 con 0x7f0ad40cd890 2023-05-20T01:24:59.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:59.671+0000 7f0adba02700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ad40b4790 con 0x7f0ad415ca90 2023-05-20T01:24:59.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:59.671+0000 7f0ad979e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ad40b65d0 0x7f0ad40b5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:59.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:59.671+0000 7f0ad979e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ad40b65d0 0x7f0ad40b5270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35872/0 (socket says 172.21.15.16:35872) 2023-05-20T01:24:59.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:59.671+0000 7f0ad979e700 1 -- 172.21.15.16:0/2803587422 learned_addr learned my addr 172.21.15.16:0/2803587422 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:24:59.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:59.671+0000 7f0ad8f9d700 1 --2- 172.21.15.16:0/2803587422 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0ad40cd890 0x7f0ad415dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:59.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:59.671+0000 7f0ad9f9f700 1 --2- 172.21.15.16:0/2803587422 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0ad415ca90 0x7f0ad415ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:59.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:59.672+0000 7f0ad979e700 1 -- 172.21.15.16:0/2803587422 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0ad415ca90 msgr2=0x7f0ad415ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:59.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:59.672+0000 7f0ad979e700 1 --2- 172.21.15.16:0/2803587422 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0ad415ca90 0x7f0ad415ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:59.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:59.672+0000 7f0ad979e700 1 -- 172.21.15.16:0/2803587422 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0ad40cd890 msgr2=0x7f0ad415dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:24:59.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:59.672+0000 7f0ad979e700 1 --2- 172.21.15.16:0/2803587422 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0ad40cd890 0x7f0ad415dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:24:59.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:59.672+0000 7f0ad979e700 1 -- 172.21.15.16:0/2803587422 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ac40094a0 con 0x7f0ad40b65d0 2023-05-20T01:24:59.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:59.672+0000 7f0ad979e700 1 --2- 172.21.15.16:0/2803587422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ad40b65d0 0x7f0ad40b5270 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f0ad000c4d0 tx=0x7f0ad000c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:24:59.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:59.672+0000 7f0ac9ffb700 1 -- 172.21.15.16:0/2803587422 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ad0011790 con 0x7f0ad40b65d0 2023-05-20T01:24:59.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:59.672+0000 7f0ac9ffb700 1 -- 172.21.15.16:0/2803587422 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ad00118f0 con 0x7f0ad40b65d0 2023-05-20T01:24:59.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:59.672+0000 7f0adba02700 1 -- 172.21.15.16:0/2803587422 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ad40d1d50 con 0x7f0ad40b65d0 2023-05-20T01:24:59.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:59.672+0000 7f0ac9ffb700 1 -- 172.21.15.16:0/2803587422 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ad0011ca0 con 0x7f0ad40b65d0 2023-05-20T01:24:59.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:59.672+0000 7f0adba02700 1 -- 172.21.15.16:0/2803587422 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ad40d1a70 con 0x7f0ad40b65d0 2023-05-20T01:24:59.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:59.673+0000 7f0adba02700 1 -- 172.21.15.16:0/2803587422 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ad40d2030 con 0x7f0ad40b65d0 2023-05-20T01:24:59.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:59.673+0000 7f0ac9ffb700 1 -- 172.21.15.16:0/2803587422 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0ad001c460 con 0x7f0ad40b65d0 2023-05-20T01:24:59.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:59.674+0000 7f0ac9ffb700 1 --2- 172.21.15.16:0/2803587422 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f0ab806ace0 0x7f0ab806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:24:59.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:59.674+0000 7f0ac9ffb700 1 -- 172.21.15.16:0/2803587422 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7275+0+0 (secure 0 0 0) 0x7f0ad0089660 con 0x7f0ad40b65d0 2023-05-20T01:24:59.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:59.674+0000 7f0ad8f9d700 1 --2- 172.21.15.16:0/2803587422 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f0ab806ace0 0x7f0ab806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:24:59.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:59.674+0000 7f0ad8f9d700 1 --2- 172.21.15.16:0/2803587422 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f0ab806ace0 0x7f0ab806d190 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f0ac40090b0 tx=0x7f0ac401c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:24:59.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:59.679+0000 7f0ac9ffb700 1 -- 172.21.15.16:0/2803587422 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f0ad0054250 con 0x7f0ad40b65d0 2023-05-20T01:24:59.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:24:59.921+0000 7f0adba02700 1 -- 172.21.15.16:0/2803587422 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f0ad404e8c0 con 0x7f0ad40b65d0 2023-05-20T01:25:00.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:00.515+0000 7f0ac9ffb700 1 -- 172.21.15.16:0/2803587422 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v51) v1 ==== 104+0+0 (secure 0 0 0) 0x7f0ad0057870 con 0x7f0ad40b65d0 2023-05-20T01:25:00.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:00.518+0000 7f0adba02700 1 -- 172.21.15.16:0/2803587422 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f0ab806ace0 msgr2=0x7f0ab806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:00.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:00.518+0000 7f0adba02700 1 --2- 172.21.15.16:0/2803587422 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f0ab806ace0 0x7f0ab806d190 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f0ac40090b0 tx=0x7f0ac401c040 comp rx=0 tx=0).stop 2023-05-20T01:25:00.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:00.518+0000 7f0adba02700 1 -- 172.21.15.16:0/2803587422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ad40b65d0 msgr2=0x7f0ad40b5270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:00.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:00.518+0000 7f0adba02700 1 --2- 172.21.15.16:0/2803587422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ad40b65d0 0x7f0ad40b5270 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f0ad000c4d0 tx=0x7f0ad000c7e0 comp rx=0 tx=0).stop 2023-05-20T01:25:00.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:00.518+0000 7f0adba02700 1 -- 172.21.15.16:0/2803587422 shutdown_connections 2023-05-20T01:25:00.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:00.518+0000 7f0adba02700 1 --2- 172.21.15.16:0/2803587422 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f0ab806ace0 0x7f0ab806d190 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:00.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:00.518+0000 7f0adba02700 1 --2- 172.21.15.16:0/2803587422 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0ad415ca90 0x7f0ad415ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:00.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:00.518+0000 7f0adba02700 1 --2- 172.21.15.16:0/2803587422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ad40b65d0 0x7f0ad40b5270 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:00.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:00.518+0000 7f0adba02700 1 --2- 172.21.15.16:0/2803587422 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0ad40cd890 0x7f0ad415dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:00.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:00.518+0000 7f0adba02700 1 -- 172.21.15.16:0/2803587422 >> 172.21.15.16:0/2803587422 conn(0x7f0ad406b000 msgr2=0x7f0ad40d2550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:00.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:00.518+0000 7f0adba02700 1 -- 172.21.15.16:0/2803587422 shutdown_connections 2023-05-20T01:25:00.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:00.518+0000 7f0adba02700 1 -- 172.21.15.16:0/2803587422 wait complete. 2023-05-20T01:25:00.536 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-05-20T01:25:00.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:00.710+0000 7f73f72ef700 1 Processor -- start 2023-05-20T01:25:00.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:00.710+0000 7f73f72ef700 1 -- start start 2023-05-20T01:25:00.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:00.711+0000 7f73f72ef700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f73f0159790 0x7f73f00caf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:00.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:00.711+0000 7f73f72ef700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f73f00cdf90 0x7f73f00712a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:00.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:00.711+0000 7f73f72ef700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73f00cc320 con 0x7f73f00717e0 2023-05-20T01:25:00.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:00.711+0000 7f73f72ef700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73f0071c20 con 0x7f73f0159790 2023-05-20T01:25:00.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:00.711+0000 7f73f72ef700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73f00cf410 con 0x7f73f00cdf90 2023-05-20T01:25:00.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:00.711+0000 7f73f508b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f73f0159790 0x7f73f00caf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:00.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:00.711+0000 7f73f508b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f73f0159790 0x7f73f00caf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35882/0 (socket says 172.21.15.16:35882) 2023-05-20T01:25:00.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:00.711+0000 7f73f508b700 1 -- 172.21.15.16:0/2178902455 learned_addr learned my addr 172.21.15.16:0/2178902455 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:00.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:00.712+0000 7f73f488a700 1 --2- 172.21.15.16:0/2178902455 >> v2:172.21.15.132:3300/0 conn(0x7f73f00cdf90 0x7f73f00712a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:00.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:00.712+0000 7f73f508b700 1 -- 172.21.15.16:0/2178902455 >> v2:172.21.15.132:3300/0 conn(0x7f73f00cdf90 msgr2=0x7f73f00712a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:00.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:00.712+0000 7f73f508b700 1 --2- 172.21.15.16:0/2178902455 >> v2:172.21.15.132:3300/0 conn(0x7f73f00cdf90 0x7f73f00712a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:00.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:00.712+0000 7f73f508b700 1 -- 172.21.15.16:0/2178902455 >> v1:172.21.15.132:6789/0 conn(0x7f73f00717e0 legacy=0x7f73f015d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:00.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:00.712+0000 7f73f508b700 1 -- 172.21.15.16:0/2178902455 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73f00cb450 con 0x7f73f0159790 2023-05-20T01:25:00.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:00.712+0000 7f73f508b700 1 --2- 172.21.15.16:0/2178902455 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73f0159790 0x7f73f00caf10 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f73f00afbd0 tx=0x7f73e0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64deb0779cadd8c1 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:00.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:00.712+0000 7f73e7fff700 1 -- 172.21.15.16:0/2178902455 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73e0004230 con 0x7f73f0159790 2023-05-20T01:25:00.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:00.712+0000 7f73e7fff700 1 -- 172.21.15.16:0/2178902455 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73e0004390 con 0x7f73f0159790 2023-05-20T01:25:00.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:00.712+0000 7f73e7fff700 1 -- 172.21.15.16:0/2178902455 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73e00036a0 con 0x7f73f0159790 2023-05-20T01:25:00.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:00.713+0000 7f73f72ef700 1 -- 172.21.15.16:0/2178902455 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73f0159790 msgr2=0x7f73f00caf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:00.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:00.713+0000 7f73f72ef700 1 --2- 172.21.15.16:0/2178902455 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73f0159790 0x7f73f00caf10 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f73f00afbd0 tx=0x7f73e0009bd0 comp rx=0 tx=0).stop 2023-05-20T01:25:00.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:00.713+0000 7f73f72ef700 1 -- 172.21.15.16:0/2178902455 shutdown_connections 2023-05-20T01:25:00.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:00.713+0000 7f73f72ef700 1 --2- 172.21.15.16:0/2178902455 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73f0159790 0x7f73f00caf10 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:00.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:00.713+0000 7f73f72ef700 1 --2- 172.21.15.16:0/2178902455 >> v2:172.21.15.132:3300/0 conn(0x7f73f00cdf90 0x7f73f00712a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:00.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:00.713+0000 7f73f72ef700 1 -- 172.21.15.16:0/2178902455 >> 172.21.15.16:0/2178902455 conn(0x7f73f00b2920 msgr2=0x7f73f00d0020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:00.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:00.713+0000 7f73f72ef700 1 -- 172.21.15.16:0/2178902455 shutdown_connections 2023-05-20T01:25:00.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:00.713+0000 7f73f72ef700 1 -- 172.21.15.16:0/2178902455 wait complete. 2023-05-20T01:25:00.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:00.714+0000 7f73f72ef700 1 Processor -- start 2023-05-20T01:25:00.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:00.714+0000 7f73f72ef700 1 -- start start 2023-05-20T01:25:00.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:00.715+0000 7f73f72ef700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f73f00717e0 0x7f73f014cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:00.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:00.715+0000 7f73f72ef700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f73f00cdf90 0x7f73f00c8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:00.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:00.715+0000 7f73f72ef700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73f0159790 0x7f73f006a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:00.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:00.715+0000 7f73f72ef700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73f014d400 con 0x7f73f0159790 2023-05-20T01:25:00.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:00.715+0000 7f73f72ef700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73f015a240 con 0x7f73f00cdf90 2023-05-20T01:25:00.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:00.715+0000 7f73f72ef700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73f015a520 con 0x7f73f00717e0 2023-05-20T01:25:00.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:00.715+0000 7f73f588c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73f0159790 0x7f73f006a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:00.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:00.715+0000 7f73f588c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73f0159790 0x7f73f006a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35890/0 (socket says 172.21.15.16:35890) 2023-05-20T01:25:00.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:00.715+0000 7f73f588c700 1 -- 172.21.15.16:0/1179902252 learned_addr learned my addr 172.21.15.16:0/1179902252 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:00.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:00.715+0000 7f73f488a700 1 --2- 172.21.15.16:0/1179902252 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f73f00cdf90 0x7f73f00c8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:00.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:00.715+0000 7f73f508b700 1 --2- 172.21.15.16:0/1179902252 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f73f00717e0 0x7f73f014cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:00.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:00.715+0000 7f73f588c700 1 -- 172.21.15.16:0/1179902252 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f73f00717e0 msgr2=0x7f73f014cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:00.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:00.715+0000 7f73f588c700 1 --2- 172.21.15.16:0/1179902252 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f73f00717e0 0x7f73f014cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:00.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:00.715+0000 7f73f588c700 1 -- 172.21.15.16:0/1179902252 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f73f00cdf90 msgr2=0x7f73f00c8bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:00.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:00.715+0000 7f73f588c700 1 --2- 172.21.15.16:0/1179902252 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f73f00cdf90 0x7f73f00c8bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:00.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:00.715+0000 7f73f588c700 1 -- 172.21.15.16:0/1179902252 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73e0009550 con 0x7f73f0159790 2023-05-20T01:25:00.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:00.715+0000 7f73f508b700 1 --2- 172.21.15.16:0/1179902252 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f73f00717e0 0x7f73f014cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:25:00.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:00.715+0000 7f73f588c700 1 --2- 172.21.15.16:0/1179902252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73f0159790 0x7f73f006a510 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f73e800c7a0 tx=0x7f73e800cb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:00.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:00.715+0000 7f73e5ffb700 1 -- 172.21.15.16:0/1179902252 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73e8003e10 con 0x7f73f0159790 2023-05-20T01:25:00.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:00.715+0000 7f73e5ffb700 1 -- 172.21.15.16:0/1179902252 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73e8005070 con 0x7f73f0159790 2023-05-20T01:25:00.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:00.715+0000 7f73f72ef700 1 -- 172.21.15.16:0/1179902252 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73f0159b60 con 0x7f73f0159790 2023-05-20T01:25:00.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:00.715+0000 7f73e5ffb700 1 -- 172.21.15.16:0/1179902252 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73e8005420 con 0x7f73f0159790 2023-05-20T01:25:00.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:00.716+0000 7f73f72ef700 1 -- 172.21.15.16:0/1179902252 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73f00c5f10 con 0x7f73f0159790 2023-05-20T01:25:00.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:00.717+0000 7f73f72ef700 1 -- 172.21.15.16:0/1179902252 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73f00cb710 con 0x7f73f0159790 2023-05-20T01:25:00.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:00.717+0000 7f73e5ffb700 1 -- 172.21.15.16:0/1179902252 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f73e8005580 con 0x7f73f0159790 2023-05-20T01:25:00.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:00.717+0000 7f73e5ffb700 1 --2- 172.21.15.16:0/1179902252 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f73d406ace0 0x7f73d406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:00.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:00.717+0000 7f73e5ffb700 1 -- 172.21.15.16:0/1179902252 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7275+0+0 (secure 0 0 0) 0x7f73e8089170 con 0x7f73f0159790 2023-05-20T01:25:00.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:00.718+0000 7f73f508b700 1 --2- 172.21.15.16:0/1179902252 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f73d406ace0 0x7f73d406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:00.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:00.718+0000 7f73f508b700 1 --2- 172.21.15.16:0/1179902252 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f73d406ace0 0x7f73d406d190 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f73e0009c10 tx=0x7f73e0008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:00.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:00.722+0000 7f73e5ffb700 1 -- 172.21.15.16:0/1179902252 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f73e8053cc0 con 0x7f73f0159790 2023-05-20T01:25:00.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:00.920+0000 7f73f72ef700 1 -- 172.21.15.16:0/1179902252 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f73f00747b0 con 0x7f73f0159790 2023-05-20T01:25:01.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:01.521+0000 7f73e5ffb700 1 -- 172.21.15.16:0/1179902252 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v77) v1 ==== 181+0+0 (secure 0 0 0) 0x7f73e80572e0 con 0x7f73f0159790 2023-05-20T01:25:01.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:01.526+0000 7f73f72ef700 1 -- 172.21.15.16:0/1179902252 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f73d406ace0 msgr2=0x7f73d406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:01.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:01.526+0000 7f73f72ef700 1 --2- 172.21.15.16:0/1179902252 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f73d406ace0 0x7f73d406d190 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f73e0009c10 tx=0x7f73e0008030 comp rx=0 tx=0).stop 2023-05-20T01:25:01.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:01.526+0000 7f73f72ef700 1 -- 172.21.15.16:0/1179902252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73f0159790 msgr2=0x7f73f006a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:01.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:01.526+0000 7f73f72ef700 1 --2- 172.21.15.16:0/1179902252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73f0159790 0x7f73f006a510 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f73e800c7a0 tx=0x7f73e800cb60 comp rx=0 tx=0).stop 2023-05-20T01:25:01.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:01.526+0000 7f73f72ef700 1 -- 172.21.15.16:0/1179902252 shutdown_connections 2023-05-20T01:25:01.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:01.526+0000 7f73f72ef700 1 --2- 172.21.15.16:0/1179902252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73f0159790 0x7f73f006a510 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:01.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:01.526+0000 7f73f72ef700 1 --2- 172.21.15.16:0/1179902252 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f73d406ace0 0x7f73d406d190 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:01.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:01.526+0000 7f73f72ef700 1 --2- 172.21.15.16:0/1179902252 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f73f00717e0 0x7f73f014cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:01.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:01.526+0000 7f73f72ef700 1 --2- 172.21.15.16:0/1179902252 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f73f00cdf90 0x7f73f00c8bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:01.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:01.526+0000 7f73f72ef700 1 -- 172.21.15.16:0/1179902252 >> 172.21.15.16:0/1179902252 conn(0x7f73f00b2920 msgr2=0x7f73f00cd400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:01.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:01.526+0000 7f73f72ef700 1 -- 172.21.15.16:0/1179902252 shutdown_connections 2023-05-20T01:25:01.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:01.526+0000 7f73f72ef700 1 -- 172.21.15.16:0/1179902252 wait complete. 2023-05-20T01:25:01.531 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' removed 2023-05-20T01:25:01.546 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-05-20T01:25:01.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:01.682+0000 7f158d411700 1 Processor -- start 2023-05-20T01:25:01.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:01.682+0000 7f158d411700 1 -- start start 2023-05-20T01:25:01.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:01.682+0000 7f158d411700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f158815df90 0x7f158806d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:01.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:01.682+0000 7f158d411700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f158806cb60 0x7f15880b09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:01.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:01.683+0000 7f158d411700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f158806c6d0 con 0x7f1588072170 2023-05-20T01:25:01.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:01.683+0000 7f158d411700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f158806dc00 con 0x7f158815df90 2023-05-20T01:25:01.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:01.683+0000 7f158d411700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f158806bda0 con 0x7f158806cb60 2023-05-20T01:25:01.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:01.683+0000 7f1586ffd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f158815df90 0x7f158806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:01.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:01.683+0000 7f15877fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f1588072170 0x7f158815e480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:47424/0 (socket says 172.21.15.16:47424) 2023-05-20T01:25:01.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:01.683+0000 7f15877fe700 1 -- 172.21.15.16:0/1247463347 learned_addr learned my addr 172.21.15.16:0/1247463347 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:25:01.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:01.683+0000 7f1586ffd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f158815df90 0x7f158806d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35894/0 (socket says 172.21.15.16:35894) 2023-05-20T01:25:01.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:01.683+0000 7f15867fc700 1 --2- 172.21.15.16:0/1247463347 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f158806cb60 0x7f15880b09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:01.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:01.683+0000 7f1586ffd700 1 -- 172.21.15.16:0/1247463347 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f158806cb60 msgr2=0x7f15880b09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:01.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:01.683+0000 7f1586ffd700 1 --2- 172.21.15.16:0/1247463347 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f158806cb60 0x7f15880b09f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:01.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:01.683+0000 7f1586ffd700 1 -- 172.21.15.16:0/1247463347 >> v1:172.21.15.16:6789/0 conn(0x7f1588072170 legacy=0x7f158815e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:01.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:01.683+0000 7f1586ffd700 1 -- 172.21.15.16:0/1247463347 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f158806c080 con 0x7f158815df90 2023-05-20T01:25:01.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:01.683+0000 7f1586ffd700 1 --2- 172.21.15.16:0/1247463347 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f158815df90 0x7f158806d360 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f157c009840 tx=0x7f157c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e35b9cb199cf318 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:01.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:01.683+0000 7f1585ffb700 1 -- 172.21.15.16:0/1247463347 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f157c011830 con 0x7f158815df90 2023-05-20T01:25:01.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:01.683+0000 7f1585ffb700 1 -- 172.21.15.16:0/1247463347 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f157c011990 con 0x7f158815df90 2023-05-20T01:25:01.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:01.683+0000 7f1585ffb700 1 -- 172.21.15.16:0/1247463347 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f157c011d20 con 0x7f158815df90 2023-05-20T01:25:01.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:01.684+0000 7f158d411700 1 -- 172.21.15.16:0/1247463347 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f158815df90 msgr2=0x7f158806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:01.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:01.684+0000 7f158d411700 1 --2- 172.21.15.16:0/1247463347 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f158815df90 0x7f158806d360 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f157c009840 tx=0x7f157c009b50 comp rx=0 tx=0).stop 2023-05-20T01:25:01.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:01.684+0000 7f158d411700 1 -- 172.21.15.16:0/1247463347 shutdown_connections 2023-05-20T01:25:01.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:01.684+0000 7f158d411700 1 --2- 172.21.15.16:0/1247463347 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f158815df90 0x7f158806d360 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:01.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:01.684+0000 7f158d411700 1 --2- 172.21.15.16:0/1247463347 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f158806cb60 0x7f15880b09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:01.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:01.684+0000 7f158d411700 1 -- 172.21.15.16:0/1247463347 >> 172.21.15.16:0/1247463347 conn(0x7f15880cb9f0 msgr2=0x7f15880cbdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:01.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:01.684+0000 7f158d411700 1 -- 172.21.15.16:0/1247463347 shutdown_connections 2023-05-20T01:25:01.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:01.684+0000 7f158d411700 1 -- 172.21.15.16:0/1247463347 wait complete. 2023-05-20T01:25:01.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:01.685+0000 7f158d411700 1 Processor -- start 2023-05-20T01:25:01.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:01.685+0000 7f158d411700 1 -- start start 2023-05-20T01:25:01.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:01.686+0000 7f158d411700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f158806cb60 0x7f15880cde00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:01.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:01.686+0000 7f158d411700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1588072170 0x7f15880c1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:01.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:01.686+0000 7f158d411700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f158815df90 0x7f15880c3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:01.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:01.686+0000 7f158d411700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1588153560 con 0x7f158815df90 2023-05-20T01:25:01.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:01.686+0000 7f158d411700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15880c7090 con 0x7f1588072170 2023-05-20T01:25:01.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:01.686+0000 7f158d411700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15880ce490 con 0x7f158806cb60 2023-05-20T01:25:01.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:01.686+0000 7f15877fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f158815df90 0x7f15880c3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:01.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:01.686+0000 7f15877fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f158815df90 0x7f15880c3ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35902/0 (socket says 172.21.15.16:35902) 2023-05-20T01:25:01.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:01.686+0000 7f15877fe700 1 -- 172.21.15.16:0/2082229295 learned_addr learned my addr 172.21.15.16:0/2082229295 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:01.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:01.686+0000 7f1586ffd700 1 --2- 172.21.15.16:0/2082229295 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f158806cb60 0x7f15880cde00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:01.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:01.686+0000 7f15867fc700 1 --2- 172.21.15.16:0/2082229295 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1588072170 0x7f15880c1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:01.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:01.687+0000 7f15877fe700 1 -- 172.21.15.16:0/2082229295 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f158806cb60 msgr2=0x7f15880cde00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:01.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:01.687+0000 7f15877fe700 1 --2- 172.21.15.16:0/2082229295 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f158806cb60 0x7f15880cde00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:01.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:01.687+0000 7f15877fe700 1 -- 172.21.15.16:0/2082229295 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1588072170 msgr2=0x7f15880c1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:01.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:01.687+0000 7f15877fe700 1 --2- 172.21.15.16:0/2082229295 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1588072170 0x7f15880c1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:01.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:01.687+0000 7f15877fe700 1 -- 172.21.15.16:0/2082229295 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f157c0094a0 con 0x7f158815df90 2023-05-20T01:25:01.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:01.687+0000 7f1586ffd700 1 --2- 172.21.15.16:0/2082229295 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f158806cb60 0x7f15880cde00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:25:01.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:01.687+0000 7f15877fe700 1 --2- 172.21.15.16:0/2082229295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f158815df90 0x7f15880c3ac0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f158000cbc0 tx=0x7f158000cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:01.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:01.687+0000 7f156f7fe700 1 -- 172.21.15.16:0/2082229295 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f158000ab50 con 0x7f158815df90 2023-05-20T01:25:01.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:01.687+0000 7f156f7fe700 1 -- 172.21.15.16:0/2082229295 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f158000acb0 con 0x7f158815df90 2023-05-20T01:25:01.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:01.687+0000 7f158d411700 1 -- 172.21.15.16:0/2082229295 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f158806d8a0 con 0x7f158815df90 2023-05-20T01:25:01.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:01.687+0000 7f156f7fe700 1 -- 172.21.15.16:0/2082229295 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1580012600 con 0x7f158815df90 2023-05-20T01:25:01.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:01.687+0000 7f158d411700 1 -- 172.21.15.16:0/2082229295 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f158806c360 con 0x7f158815df90 2023-05-20T01:25:01.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:01.688+0000 7f158d411700 1 -- 172.21.15.16:0/2082229295 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15880b0ef0 con 0x7f158815df90 2023-05-20T01:25:01.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:01.688+0000 7f156f7fe700 1 -- 172.21.15.16:0/2082229295 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f158001b3b0 con 0x7f158815df90 2023-05-20T01:25:01.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:01.689+0000 7f156f7fe700 1 --2- 172.21.15.16:0/2082229295 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f157406ace0 0x7f157406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:01.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:01.689+0000 7f1586ffd700 1 --2- 172.21.15.16:0/2082229295 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f157406ace0 0x7f157406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:01.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:01.689+0000 7f156f7fe700 1 -- 172.21.15.16:0/2082229295 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6808+0+0 (secure 0 0 0) 0x7f1580089300 con 0x7f158815df90 2023-05-20T01:25:01.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:01.689+0000 7f1586ffd700 1 --2- 172.21.15.16:0/2082229295 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f157406ace0 0x7f157406d190 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f157c0090b0 tx=0x7f157c01c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:01.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:01.693+0000 7f156f7fe700 1 -- 172.21.15.16:0/2082229295 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f1580054200 con 0x7f158815df90 2023-05-20T01:25:01.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:01.892+0000 7f158d411700 1 -- 172.21.15.16:0/2082229295 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f158804e8c0 con 0x7f158815df90 2023-05-20T01:25:02.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:02.525+0000 7f156f7fe700 1 -- 172.21.15.16:0/2082229295 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v78) v1 ==== 169+0+0 (secure 0 0 0) 0x7f1580057820 con 0x7f158815df90 2023-05-20T01:25:02.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:02.530+0000 7f158d411700 1 -- 172.21.15.16:0/2082229295 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f157406ace0 msgr2=0x7f157406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:02.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:02.530+0000 7f158d411700 1 --2- 172.21.15.16:0/2082229295 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f157406ace0 0x7f157406d190 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f157c0090b0 tx=0x7f157c01c040 comp rx=0 tx=0).stop 2023-05-20T01:25:02.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:02.531+0000 7f158d411700 1 -- 172.21.15.16:0/2082229295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f158815df90 msgr2=0x7f15880c3ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:02.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:02.531+0000 7f158d411700 1 --2- 172.21.15.16:0/2082229295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f158815df90 0x7f15880c3ac0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f158000cbc0 tx=0x7f158000cf80 comp rx=0 tx=0).stop 2023-05-20T01:25:02.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:02.531+0000 7f158d411700 1 -- 172.21.15.16:0/2082229295 shutdown_connections 2023-05-20T01:25:02.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:02.531+0000 7f158d411700 1 --2- 172.21.15.16:0/2082229295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f158815df90 0x7f15880c3ac0 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:02.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:02.531+0000 7f158d411700 1 --2- 172.21.15.16:0/2082229295 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f157406ace0 0x7f157406d190 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:02.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:02.531+0000 7f158d411700 1 --2- 172.21.15.16:0/2082229295 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f158806cb60 0x7f15880cde00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:02.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:02.531+0000 7f158d411700 1 --2- 172.21.15.16:0/2082229295 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1588072170 0x7f15880c1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:02.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:02.531+0000 7f158d411700 1 -- 172.21.15.16:0/2082229295 >> 172.21.15.16:0/2082229295 conn(0x7f15880cb9f0 msgr2=0x7f15880cbdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:02.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:02.531+0000 7f158d411700 1 -- 172.21.15.16:0/2082229295 shutdown_connections 2023-05-20T01:25:02.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:02.531+0000 7f158d411700 1 -- 172.21.15.16:0/2082229295 wait complete. 2023-05-20T01:25:02.536 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' removed 2023-05-20T01:25:02.550 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_extra cephfs_data_extra --yes-i-really-really-mean-it 2023-05-20T01:25:02.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:02.687+0000 7f0aa41f5700 1 Processor -- start 2023-05-20T01:25:02.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:02.687+0000 7f0aa41f5700 1 -- start start 2023-05-20T01:25:02.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:02.688+0000 7f0aa41f5700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f0a9c0b2670 0x7f0a9c0b60d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:02.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:02.688+0000 7f0aa41f5700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0a9c0d3a50 0x7f0a9c0cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:02.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:02.688+0000 7f0aa41f5700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a9c0d2eb0 con 0x7f0a9c0d1860 2023-05-20T01:25:02.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:02.688+0000 7f0aa41f5700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a9c0cd970 con 0x7f0a9c0d3a50 2023-05-20T01:25:02.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:02.688+0000 7f0aa41f5700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a9c0cdc50 con 0x7f0a9c0b2670 2023-05-20T01:25:02.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:02.688+0000 7f0aa2792700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0a9c0d3a50 0x7f0a9c0cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:02.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:02.688+0000 7f0aa2792700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0a9c0d3a50 0x7f0a9c0cf7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35908/0 (socket says 172.21.15.16:35908) 2023-05-20T01:25:02.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:02.688+0000 7f0aa2792700 1 -- 172.21.15.16:0/2361828607 learned_addr learned my addr 172.21.15.16:0/2361828607 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:02.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:02.689+0000 7f0aa1f91700 1 --2- 172.21.15.16:0/2361828607 >> v2:172.21.15.132:3300/0 conn(0x7f0a9c0b2670 0x7f0a9c0b60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:02.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:02.689+0000 7f0aa2792700 1 -- 172.21.15.16:0/2361828607 >> v2:172.21.15.132:3300/0 conn(0x7f0a9c0b2670 msgr2=0x7f0a9c0b60d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:02.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:02.689+0000 7f0aa2792700 1 --2- 172.21.15.16:0/2361828607 >> v2:172.21.15.132:3300/0 conn(0x7f0a9c0b2670 0x7f0a9c0b60d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:02.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:02.689+0000 7f0aa2792700 1 -- 172.21.15.16:0/2361828607 >> v1:172.21.15.132:6789/0 conn(0x7f0a9c0d1860 legacy=0x7f0a9c0d3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:02.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:02.689+0000 7f0aa2792700 1 -- 172.21.15.16:0/2361828607 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a9c0d2b40 con 0x7f0a9c0d3a50 2023-05-20T01:25:02.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:02.689+0000 7f0aa1f91700 1 --2- 172.21.15.16:0/2361828607 >> v2:172.21.15.132:3300/0 conn(0x7f0a9c0b2670 0x7f0a9c0b60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:25:02.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:02.689+0000 7f0aa2792700 1 --2- 172.21.15.16:0/2361828607 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a9c0d3a50 0x7f0a9c0cf7d0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c0d42f0 tx=0x7f0a9400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea020f4bb26fd07e server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:02.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:02.689+0000 7f0aa0f8f700 1 -- 172.21.15.16:0/2361828607 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a9400e070 con 0x7f0a9c0d3a50 2023-05-20T01:25:02.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:02.689+0000 7f0aa0f8f700 1 -- 172.21.15.16:0/2361828607 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a9400b9c0 con 0x7f0a9c0d3a50 2023-05-20T01:25:02.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:02.689+0000 7f0aa0f8f700 1 -- 172.21.15.16:0/2361828607 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a9400bd70 con 0x7f0a9c0d3a50 2023-05-20T01:25:02.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:02.690+0000 7f0aa41f5700 1 -- 172.21.15.16:0/2361828607 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a9c0d3a50 msgr2=0x7f0a9c0cf7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:02.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:02.690+0000 7f0aa41f5700 1 --2- 172.21.15.16:0/2361828607 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a9c0d3a50 0x7f0a9c0cf7d0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c0d42f0 tx=0x7f0a9400b330 comp rx=0 tx=0).stop 2023-05-20T01:25:02.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:02.690+0000 7f0aa41f5700 1 -- 172.21.15.16:0/2361828607 shutdown_connections 2023-05-20T01:25:02.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:02.690+0000 7f0aa41f5700 1 --2- 172.21.15.16:0/2361828607 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a9c0d3a50 0x7f0a9c0cf7d0 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:02.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:02.690+0000 7f0aa41f5700 1 --2- 172.21.15.16:0/2361828607 >> v2:172.21.15.132:3300/0 conn(0x7f0a9c0b2670 0x7f0a9c0b60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:02.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:02.690+0000 7f0aa41f5700 1 -- 172.21.15.16:0/2361828607 >> 172.21.15.16:0/2361828607 conn(0x7f0a9c071c40 msgr2=0x7f0a9c072010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:02.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:02.690+0000 7f0aa41f5700 1 -- 172.21.15.16:0/2361828607 shutdown_connections 2023-05-20T01:25:02.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:02.690+0000 7f0aa41f5700 1 -- 172.21.15.16:0/2361828607 wait complete. 2023-05-20T01:25:02.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:02.691+0000 7f0aa41f5700 1 Processor -- start 2023-05-20T01:25:02.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:02.691+0000 7f0aa41f5700 1 -- start start 2023-05-20T01:25:02.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:02.692+0000 7f0aa41f5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a9c0b2670 0x7f0a9c0cd2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:02.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:02.692+0000 7f0aa41f5700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0a9c0d1860 0x7f0a9c15d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:02.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:02.692+0000 7f0aa41f5700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0a9c0d3a50 0x7f0a9c1525e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:02.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:02.692+0000 7f0aa41f5700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9c15e4c0 con 0x7f0a9c0b2670 2023-05-20T01:25:02.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:02.692+0000 7f0aa41f5700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9c15d8b0 con 0x7f0a9c0d1860 2023-05-20T01:25:02.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:02.692+0000 7f0aa41f5700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9c15db90 con 0x7f0a9c0d3a50 2023-05-20T01:25:02.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:02.693+0000 7f0aa1f91700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a9c0b2670 0x7f0a9c0cd2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:02.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:02.693+0000 7f0aa1f91700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a9c0b2670 0x7f0a9c0cd2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35910/0 (socket says 172.21.15.16:35910) 2023-05-20T01:25:02.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:02.693+0000 7f0aa1f91700 1 -- 172.21.15.16:0/2636254539 learned_addr learned my addr 172.21.15.16:0/2636254539 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:02.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:02.693+0000 7f0aa2792700 1 --2- 172.21.15.16:0/2636254539 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0a9c0d3a50 0x7f0a9c1525e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:02.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:02.693+0000 7f0aa1790700 1 --2- 172.21.15.16:0/2636254539 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0a9c0d1860 0x7f0a9c15d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:02.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:02.693+0000 7f0aa1f91700 1 -- 172.21.15.16:0/2636254539 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0a9c0d3a50 msgr2=0x7f0a9c1525e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:02.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:02.693+0000 7f0aa1f91700 1 --2- 172.21.15.16:0/2636254539 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0a9c0d3a50 0x7f0a9c1525e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:02.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:02.693+0000 7f0aa1f91700 1 -- 172.21.15.16:0/2636254539 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0a9c0d1860 msgr2=0x7f0a9c15d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:02.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:02.693+0000 7f0aa1f91700 1 --2- 172.21.15.16:0/2636254539 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0a9c0d1860 0x7f0a9c15d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:02.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:02.693+0000 7f0aa1f91700 1 -- 172.21.15.16:0/2636254539 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a94009b30 con 0x7f0a9c0b2670 2023-05-20T01:25:02.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:02.693+0000 7f0aa1f91700 1 --2- 172.21.15.16:0/2636254539 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a9c0b2670 0x7f0a9c0cd2b0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c00e810 tx=0x7f0a8c00ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:02.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:02.693+0000 7f0a927fc700 1 -- 172.21.15.16:0/2636254539 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a8c013610 con 0x7f0a9c0b2670 2023-05-20T01:25:02.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:02.693+0000 7f0a927fc700 1 -- 172.21.15.16:0/2636254539 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a8c013770 con 0x7f0a9c0b2670 2023-05-20T01:25:02.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:02.693+0000 7f0aa41f5700 1 -- 172.21.15.16:0/2636254539 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a9c0b2a40 con 0x7f0a9c0b2670 2023-05-20T01:25:02.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:02.693+0000 7f0a927fc700 1 -- 172.21.15.16:0/2636254539 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a8c013b20 con 0x7f0a9c0b2670 2023-05-20T01:25:02.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:02.693+0000 7f0aa41f5700 1 -- 172.21.15.16:0/2636254539 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a9c0cdf30 con 0x7f0a9c0b2670 2023-05-20T01:25:02.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:02.694+0000 7f0aa41f5700 1 -- 172.21.15.16:0/2636254539 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a9c0d1cb0 con 0x7f0a9c0b2670 2023-05-20T01:25:02.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:02.695+0000 7f0a927fc700 1 -- 172.21.15.16:0/2636254539 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0a8c013c80 con 0x7f0a9c0b2670 2023-05-20T01:25:02.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:02.695+0000 7f0a927fc700 1 --2- 172.21.15.16:0/2636254539 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f0a8006ace0 0x7f0a8006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:02.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:02.695+0000 7f0a927fc700 1 -- 172.21.15.16:0/2636254539 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6278+0+0 (secure 0 0 0) 0x7f0a8c0892b0 con 0x7f0a9c0b2670 2023-05-20T01:25:02.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:02.695+0000 7f0aa1790700 1 --2- 172.21.15.16:0/2636254539 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f0a8006ace0 0x7f0a8006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:02.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:02.696+0000 7f0aa1790700 1 --2- 172.21.15.16:0/2636254539 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f0a8006ace0 0x7f0a8006d190 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c1532a0 tx=0x7f0a98009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:02.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:02.699+0000 7f0a927fc700 1 -- 172.21.15.16:0/2636254539 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f0a8c054280 con 0x7f0a9c0b2670 2023-05-20T01:25:02.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:02.896+0000 7f0aa41f5700 1 -- 172.21.15.16:0/2636254539 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_extra", "pool2": "cephfs_data_extra", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f0a9c15e7a0 con 0x7f0a9c0b2670 2023-05-20T01:25:03.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:03.530+0000 7f0a927fc700 1 -- 172.21.15.16:0/2636254539 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_extra", "pool2": "cephfs_data_extra", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_extra' removed v79) v1 ==== 187+0+0 (secure 0 0 0) 0x7f0a8c0578a0 con 0x7f0a9c0b2670 2023-05-20T01:25:03.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:03.536+0000 7f0aa41f5700 1 -- 172.21.15.16:0/2636254539 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f0a8006ace0 msgr2=0x7f0a8006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:03.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:03.536+0000 7f0aa41f5700 1 --2- 172.21.15.16:0/2636254539 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f0a8006ace0 0x7f0a8006d190 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c1532a0 tx=0x7f0a98009040 comp rx=0 tx=0).stop 2023-05-20T01:25:03.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:03.536+0000 7f0aa41f5700 1 -- 172.21.15.16:0/2636254539 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a9c0b2670 msgr2=0x7f0a9c0cd2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:03.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:03.536+0000 7f0aa41f5700 1 --2- 172.21.15.16:0/2636254539 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a9c0b2670 0x7f0a9c0cd2b0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c00e810 tx=0x7f0a8c00ebd0 comp rx=0 tx=0).stop 2023-05-20T01:25:03.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:03.536+0000 7f0aa41f5700 1 -- 172.21.15.16:0/2636254539 shutdown_connections 2023-05-20T01:25:03.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:03.536+0000 7f0aa41f5700 1 --2- 172.21.15.16:0/2636254539 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f0a8006ace0 0x7f0a8006d190 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:03.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:03.536+0000 7f0aa41f5700 1 --2- 172.21.15.16:0/2636254539 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0a9c0d3a50 0x7f0a9c1525e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:03.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:03.536+0000 7f0aa41f5700 1 --2- 172.21.15.16:0/2636254539 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a9c0b2670 0x7f0a9c0cd2b0 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:03.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:03.536+0000 7f0aa41f5700 1 --2- 172.21.15.16:0/2636254539 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0a9c0d1860 0x7f0a9c15d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:03.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:03.536+0000 7f0aa41f5700 1 -- 172.21.15.16:0/2636254539 >> 172.21.15.16:0/2636254539 conn(0x7f0a9c071c40 msgr2=0x7f0a9c15bb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:03.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:03.536+0000 7f0aa41f5700 1 -- 172.21.15.16:0/2636254539 shutdown_connections 2023-05-20T01:25:03.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:03.537+0000 7f0aa41f5700 1 -- 172.21.15.16:0/2636254539 wait complete. 2023-05-20T01:25:03.543 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data_extra' removed 2023-05-20T01:25:03.561 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-05-20T01:25:03.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:03.690+0000 7f828a9dd700 1 Processor -- start 2023-05-20T01:25:03.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:03.690+0000 7f828a9dd700 1 -- start start 2023-05-20T01:25:03.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:03.691+0000 7f828a9dd700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f828415aad0 0x7f82840b27d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:03.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:03.691+0000 7f828a9dd700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8284073dc0 0x7f82840cc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:03.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:03.691+0000 7f828a9dd700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f828415aea0 con 0x7f82840b1e20 2023-05-20T01:25:03.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:03.691+0000 7f828a9dd700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8284070930 con 0x7f828415aad0 2023-05-20T01:25:03.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:03.691+0000 7f828a9dd700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82840b1b00 con 0x7f8284073dc0 2023-05-20T01:25:03.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:03.691+0000 7f8288f7a700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f82840b1e20 0x7f82840716d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:47450/0 (socket says 172.21.15.16:47450) 2023-05-20T01:25:03.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:03.691+0000 7f8288f7a700 1 -- 172.21.15.16:0/724885238 learned_addr learned my addr 172.21.15.16:0/724885238 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:25:03.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:03.691+0000 7f82837fe700 1 --2- 172.21.15.16:0/724885238 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8284073dc0 0x7f82840cc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:03.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:03.691+0000 7f8283fff700 1 --2- 172.21.15.16:0/724885238 >> v2:172.21.15.132:3300/0 conn(0x7f828415aad0 0x7f82840b27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:03.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:03.691+0000 7f8282ffd700 1 -- 172.21.15.16:0/724885238 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 99402357 0 0) 0x7f828415aea0 con 0x7f82840b1e20 2023-05-20T01:25:03.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:03.691+0000 7f8282ffd700 1 -- 172.21.15.16:0/724885238 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f82700035b0 con 0x7f82840b1e20 2023-05-20T01:25:03.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:03.692+0000 7f8282ffd700 1 -- 172.21.15.16:0/724885238 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2130525640 0 0) 0x7f82700035b0 con 0x7f82840b1e20 2023-05-20T01:25:03.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:03.692+0000 7f8282ffd700 1 -- 172.21.15.16:0/724885238 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8284073dc0 msgr2=0x7f82840cc0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:03.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:03.692+0000 7f8282ffd700 1 --2- 172.21.15.16:0/724885238 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8284073dc0 0x7f82840cc0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:03.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:03.692+0000 7f8282ffd700 1 -- 172.21.15.16:0/724885238 >> v2:172.21.15.132:3300/0 conn(0x7f828415aad0 msgr2=0x7f82840b27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:03.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:03.692+0000 7f8282ffd700 1 --2- 172.21.15.16:0/724885238 >> v2:172.21.15.132:3300/0 conn(0x7f828415aad0 0x7f82840b27d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:03.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:03.692+0000 7f8282ffd700 1 -- 172.21.15.16:0/724885238 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8284070bf0 con 0x7f82840b1e20 2023-05-20T01:25:03.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:03.692+0000 7f8282ffd700 1 -- 172.21.15.16:0/724885238 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7f827c0019f0 con 0x7f82840b1e20 2023-05-20T01:25:03.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:03.692+0000 7f8282ffd700 1 -- 172.21.15.16:0/724885238 >> v1:172.21.15.16:6789/0 conn(0x7f82840b1e20 legacy=0x7f82840716d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:03.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:03.692+0000 7f8282ffd700 1 --2- 172.21.15.16:0/724885238 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8284073dc0 0x7f8270003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:03.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:03.692+0000 7f8282ffd700 1 --2- 172.21.15.16:0/724885238 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8270003f80 0x7f8270004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:03.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:03.692+0000 7f8282ffd700 1 --2- 172.21.15.16:0/724885238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8270004d90 0x7f8270005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:03.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:03.692+0000 7f8282ffd700 1 -- 172.21.15.16:0/724885238 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8284070930 con 0x7f8270004d90 2023-05-20T01:25:03.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:03.692+0000 7f8282ffd700 1 -- 172.21.15.16:0/724885238 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82840b1b00 con 0x7f8270003f80 2023-05-20T01:25:03.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:03.692+0000 7f8282ffd700 1 -- 172.21.15.16:0/724885238 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f827000ba40 con 0x7f8284073dc0 2023-05-20T01:25:03.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:03.692+0000 7f8288f7a700 1 --2- 172.21.15.16:0/724885238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8270004d90 0x7f8270005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:03.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:03.692+0000 7f8283fff700 1 --2- 172.21.15.16:0/724885238 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8284073dc0 0x7f8270003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:03.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:03.692+0000 7f82837fe700 1 --2- 172.21.15.16:0/724885238 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8270003f80 0x7f8270004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:03.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:03.692+0000 7f8288f7a700 1 -- 172.21.15.16:0/724885238 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8284073dc0 msgr2=0x7f8270003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:03.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:03.692+0000 7f8288f7a700 1 --2- 172.21.15.16:0/724885238 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8284073dc0 0x7f8270003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:03.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:03.692+0000 7f8288f7a700 1 -- 172.21.15.16:0/724885238 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8270003f80 msgr2=0x7f8270004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:03.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:03.692+0000 7f8288f7a700 1 --2- 172.21.15.16:0/724885238 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8270003f80 0x7f8270004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:03.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:03.692+0000 7f8288f7a700 1 -- 172.21.15.16:0/724885238 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82700035b0 con 0x7f8270004d90 2023-05-20T01:25:03.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:03.692+0000 7f8288f7a700 1 --2- 172.21.15.16:0/724885238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8270004d90 0x7f8270005200 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f827c00a860 tx=0x7f8270005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb88e0d2409f81e7 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:03.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:03.692+0000 7f8282ffd700 1 -- 172.21.15.16:0/724885238 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f827c012600 con 0x7f8270004d90 2023-05-20T01:25:03.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:03.692+0000 7f8282ffd700 1 -- 172.21.15.16:0/724885238 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f827c012780 con 0x7f8270004d90 2023-05-20T01:25:03.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:03.692+0000 7f8282ffd700 1 -- 172.21.15.16:0/724885238 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f827c012b30 con 0x7f8270004d90 2023-05-20T01:25:03.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:03.693+0000 7f828a9dd700 1 -- 172.21.15.16:0/724885238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8270004d90 msgr2=0x7f8270005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:03.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:03.693+0000 7f828a9dd700 1 --2- 172.21.15.16:0/724885238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8270004d90 0x7f8270005200 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f827c00a860 tx=0x7f8270005880 comp rx=0 tx=0).stop 2023-05-20T01:25:03.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:03.693+0000 7f8288f7a700 1 -- 172.21.15.16:0/724885238 reap_dead start 2023-05-20T01:25:03.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:03.693+0000 7f828a9dd700 1 -- 172.21.15.16:0/724885238 shutdown_connections 2023-05-20T01:25:03.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:03.693+0000 7f828a9dd700 1 -- 172.21.15.16:0/724885238 >> 172.21.15.16:0/724885238 conn(0x7f828406cbb0 msgr2=0x7f828406cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:03.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:03.693+0000 7f828a9dd700 1 -- 172.21.15.16:0/724885238 shutdown_connections 2023-05-20T01:25:03.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:03.693+0000 7f828a9dd700 1 -- 172.21.15.16:0/724885238 wait complete. 2023-05-20T01:25:03.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:03.694+0000 7f828a9dd700 1 Processor -- start 2023-05-20T01:25:03.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:03.695+0000 7f828a9dd700 1 -- start start 2023-05-20T01:25:03.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:03.695+0000 7f828a9dd700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f828414d4d0 0x7f82840c7400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:03.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:03.695+0000 7f828a9dd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f828414d8a0 0x7f82840c82c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:03.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:03.695+0000 7f828a9dd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f82840c8800 0x7f82840b0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:03.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:03.695+0000 7f828a9dd700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82840c8d60 con 0x7f828414d8a0 2023-05-20T01:25:03.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:03.695+0000 7f828a9dd700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82840c9040 con 0x7f82840c8800 2023-05-20T01:25:03.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:03.695+0000 7f828a9dd700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82840c6920 con 0x7f828414d4d0 2023-05-20T01:25:03.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:03.695+0000 7f82837fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f828414d8a0 0x7f82840c82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:03.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:03.695+0000 7f82837fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f828414d8a0 0x7f82840c82c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35932/0 (socket says 172.21.15.16:35932) 2023-05-20T01:25:03.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:03.695+0000 7f82837fe700 1 -- 172.21.15.16:0/124141793 learned_addr learned my addr 172.21.15.16:0/124141793 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:03.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:03.695+0000 7f8288f7a700 1 --2- 172.21.15.16:0/124141793 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f82840c8800 0x7f82840b0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:03.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:03.695+0000 7f8283fff700 1 --2- 172.21.15.16:0/124141793 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f828414d4d0 0x7f82840c7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:03.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:03.696+0000 7f82837fe700 1 -- 172.21.15.16:0/124141793 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f828414d4d0 msgr2=0x7f82840c7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:03.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:03.696+0000 7f82837fe700 1 --2- 172.21.15.16:0/124141793 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f828414d4d0 0x7f82840c7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:03.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:03.696+0000 7f82837fe700 1 -- 172.21.15.16:0/124141793 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f82840c8800 msgr2=0x7f82840b0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:03.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:03.696+0000 7f82837fe700 1 --2- 172.21.15.16:0/124141793 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f82840c8800 0x7f82840b0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:03.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:03.696+0000 7f82837fe700 1 -- 172.21.15.16:0/124141793 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82840711b0 con 0x7f828414d8a0 2023-05-20T01:25:03.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:03.696+0000 7f8283fff700 1 --2- 172.21.15.16:0/124141793 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f828414d4d0 0x7f82840c7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:25:03.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:03.696+0000 7f82837fe700 1 --2- 172.21.15.16:0/124141793 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f828414d8a0 0x7f82840c82c0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f82740119a0 tx=0x7f8274011cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:03.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:03.696+0000 7f8280ff9700 1 -- 172.21.15.16:0/124141793 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f827400f940 con 0x7f828414d8a0 2023-05-20T01:25:03.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:03.696+0000 7f8280ff9700 1 -- 172.21.15.16:0/124141793 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f827400faa0 con 0x7f828414d8a0 2023-05-20T01:25:03.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:03.696+0000 7f828a9dd700 1 -- 172.21.15.16:0/124141793 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8284070ed0 con 0x7f828414d8a0 2023-05-20T01:25:03.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:03.696+0000 7f8280ff9700 1 -- 172.21.15.16:0/124141793 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f827400fe30 con 0x7f828414d8a0 2023-05-20T01:25:03.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:03.696+0000 7f828a9dd700 1 -- 172.21.15.16:0/124141793 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82840b1720 con 0x7f828414d8a0 2023-05-20T01:25:03.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:03.697+0000 7f828a9dd700 1 -- 172.21.15.16:0/124141793 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f828406b4e0 con 0x7f828414d8a0 2023-05-20T01:25:03.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:03.697+0000 7f8280ff9700 1 -- 172.21.15.16:0/124141793 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8274014440 con 0x7f828414d8a0 2023-05-20T01:25:03.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:03.698+0000 7f8280ff9700 1 --2- 172.21.15.16:0/124141793 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f826406ace0 0x7f826406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:03.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:03.698+0000 7f8280ff9700 1 -- 172.21.15.16:0/124141793 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5874+0+0 (secure 0 0 0) 0x7f8274089b00 con 0x7f828414d8a0 2023-05-20T01:25:03.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:03.698+0000 7f8283fff700 1 --2- 172.21.15.16:0/124141793 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f826406ace0 0x7f826406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:03.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:03.698+0000 7f8283fff700 1 --2- 172.21.15.16:0/124141793 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f826406ace0 0x7f826406d190 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f82840c7f20 tx=0x7f826c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:03.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:03.702+0000 7f8280ff9700 1 -- 172.21.15.16:0/124141793 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f8274054c60 con 0x7f828414d8a0 2023-05-20T01:25:03.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:03.912+0000 7f828a9dd700 1 -- 172.21.15.16:0/124141793 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f8284159de0 con 0x7f828414d8a0 2023-05-20T01:25:03.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:03.916+0000 7f8280ff9700 1 -- 172.21.15.16:0/124141793 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7f8274058280 con 0x7f828414d8a0 2023-05-20T01:25:03.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:03.918+0000 7f828a9dd700 1 -- 172.21.15.16:0/124141793 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f826406ace0 msgr2=0x7f826406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:03.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:03.918+0000 7f828a9dd700 1 --2- 172.21.15.16:0/124141793 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f826406ace0 0x7f826406d190 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f82840c7f20 tx=0x7f826c008040 comp rx=0 tx=0).stop 2023-05-20T01:25:03.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:03.918+0000 7f828a9dd700 1 -- 172.21.15.16:0/124141793 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f828414d8a0 msgr2=0x7f82840c82c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:03.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:03.918+0000 7f828a9dd700 1 --2- 172.21.15.16:0/124141793 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f828414d8a0 0x7f82840c82c0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f82740119a0 tx=0x7f8274011cb0 comp rx=0 tx=0).stop 2023-05-20T01:25:03.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:03.918+0000 7f828a9dd700 1 -- 172.21.15.16:0/124141793 shutdown_connections 2023-05-20T01:25:03.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:03.918+0000 7f828a9dd700 1 --2- 172.21.15.16:0/124141793 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f82840c8800 0x7f82840b0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:03.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:03.918+0000 7f828a9dd700 1 --2- 172.21.15.16:0/124141793 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f826406ace0 0x7f826406d190 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:03.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:03.918+0000 7f828a9dd700 1 --2- 172.21.15.16:0/124141793 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f828414d4d0 0x7f82840c7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:03.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:03.918+0000 7f828a9dd700 1 --2- 172.21.15.16:0/124141793 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f828414d8a0 0x7f82840c82c0 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:03.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:03.918+0000 7f828a9dd700 1 -- 172.21.15.16:0/124141793 >> 172.21.15.16:0/124141793 conn(0x7f828406cbb0 msgr2=0x7f8284157800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:03.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:03.918+0000 7f828a9dd700 1 -- 172.21.15.16:0/124141793 shutdown_connections 2023-05-20T01:25:03.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:03.918+0000 7f828a9dd700 1 -- 172.21.15.16:0/124141793 wait complete. 2023-05-20T01:25:03.934 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_data_scan.TestDataScan.test_extra_data_pool_rebuild_many_files_many_workers' 2023-05-20T01:25:04.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.056+0000 7ff1d4583700 1 Processor -- start 2023-05-20T01:25:04.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.056+0000 7ff1d4583700 1 -- start start 2023-05-20T01:25:04.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.057+0000 7ff1d4583700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7ff1cc0cfc50 0x7ff1cc0d2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:04.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.057+0000 7ff1d4583700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff1cc0b4750 0x7ff1cc0d0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:04.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.057+0000 7ff1d4583700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff1cc0b2b40 0x7ff1cc0b2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:04.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.057+0000 7ff1d4583700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1cc0cfad0 con 0x7ff1cc0b4750 2023-05-20T01:25:04.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.057+0000 7ff1d4583700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1cc0d3df0 con 0x7ff1cc0cfc50 2023-05-20T01:25:04.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.057+0000 7ff1d4583700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1cc0d0b90 con 0x7ff1cc0b2b40 2023-05-20T01:25:04.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.057+0000 7ff1d1b1e700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff1cc0b4750 0x7ff1cc0d0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:04.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.057+0000 7ff1d1b1e700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff1cc0b4750 0x7ff1cc0d0020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35938/0 (socket says 172.21.15.16:35938) 2023-05-20T01:25:04.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.058+0000 7ff1d1b1e700 1 -- 172.21.15.16:0/1338055479 learned_addr learned my addr 172.21.15.16:0/1338055479 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:04.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.058+0000 7ff1d2b20700 1 --2- 172.21.15.16:0/1338055479 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff1cc0b2b40 0x7ff1cc0b2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:04.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.058+0000 7ff1d231f700 1 --2- 172.21.15.16:0/1338055479 >> v2:172.21.15.132:3300/0 conn(0x7ff1cc0cfc50 0x7ff1cc0d2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:04.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.058+0000 7ff1d1b1e700 1 -- 172.21.15.16:0/1338055479 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff1cc0b2b40 msgr2=0x7ff1cc0b2fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:04.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.058+0000 7ff1d1b1e700 1 --2- 172.21.15.16:0/1338055479 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff1cc0b2b40 0x7ff1cc0b2fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:04.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.058+0000 7ff1d1b1e700 1 -- 172.21.15.16:0/1338055479 >> v2:172.21.15.132:3300/0 conn(0x7ff1cc0cfc50 msgr2=0x7ff1cc0d2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:04.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.058+0000 7ff1d1b1e700 1 --2- 172.21.15.16:0/1338055479 >> v2:172.21.15.132:3300/0 conn(0x7ff1cc0cfc50 0x7ff1cc0d2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:04.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.058+0000 7ff1d1b1e700 1 -- 172.21.15.16:0/1338055479 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1cc0d0560 con 0x7ff1cc0b4750 2023-05-20T01:25:04.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.058+0000 7ff1d1b1e700 1 --2- 172.21.15.16:0/1338055479 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1cc0b4750 0x7ff1cc0d0020 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc0b5de0 tx=0x7ff1c8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0630c420bea02c9 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:04.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.058+0000 7ff1d131d700 1 -- 172.21.15.16:0/1338055479 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1c8011830 con 0x7ff1cc0b4750 2023-05-20T01:25:04.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.058+0000 7ff1d131d700 1 -- 172.21.15.16:0/1338055479 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1c8011990 con 0x7ff1cc0b4750 2023-05-20T01:25:04.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.058+0000 7ff1d131d700 1 -- 172.21.15.16:0/1338055479 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1c8011d20 con 0x7ff1cc0b4750 2023-05-20T01:25:04.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.059+0000 7ff1d4583700 1 -- 172.21.15.16:0/1338055479 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1cc0b4750 msgr2=0x7ff1cc0d0020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:04.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.059+0000 7ff1d4583700 1 --2- 172.21.15.16:0/1338055479 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1cc0b4750 0x7ff1cc0d0020 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc0b5de0 tx=0x7ff1c8009bd0 comp rx=0 tx=0).stop 2023-05-20T01:25:04.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.059+0000 7ff1d4583700 1 -- 172.21.15.16:0/1338055479 shutdown_connections 2023-05-20T01:25:04.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.059+0000 7ff1d4583700 1 --2- 172.21.15.16:0/1338055479 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff1cc0b2b40 0x7ff1cc0b2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:04.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.059+0000 7ff1d4583700 1 --2- 172.21.15.16:0/1338055479 >> v2:172.21.15.132:3300/0 conn(0x7ff1cc0cfc50 0x7ff1cc0d2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:04.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.059+0000 7ff1d4583700 1 --2- 172.21.15.16:0/1338055479 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1cc0b4750 0x7ff1cc0d0020 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:04.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.059+0000 7ff1d4583700 1 -- 172.21.15.16:0/1338055479 >> 172.21.15.16:0/1338055479 conn(0x7ff1cc09c590 msgr2=0x7ff1cc0d3240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:04.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.059+0000 7ff1d4583700 1 -- 172.21.15.16:0/1338055479 shutdown_connections 2023-05-20T01:25:04.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.059+0000 7ff1d4583700 1 -- 172.21.15.16:0/1338055479 wait complete. 2023-05-20T01:25:04.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.060+0000 7ff1d4583700 1 Processor -- start 2023-05-20T01:25:04.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.060+0000 7ff1d4583700 1 -- start start 2023-05-20T01:25:04.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.061+0000 7ff1d4583700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff1cc0b2b40 0x7ff1cc15d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:04.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.061+0000 7ff1d4583700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff1cc0b4750 0x7ff1cc15f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:04.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.061+0000 7ff1d4583700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1cc0cfc50 0x7ff1cc0b11b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:04.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.061+0000 7ff1d4583700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1cc1604e0 con 0x7ff1cc0cfc50 2023-05-20T01:25:04.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.061+0000 7ff1d4583700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1cc15dc40 con 0x7ff1cc0b2b40 2023-05-20T01:25:04.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.061+0000 7ff1d4583700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1cc15f5a0 con 0x7ff1cc0b4750 2023-05-20T01:25:04.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.061+0000 7ff1d2b20700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1cc0cfc50 0x7ff1cc0b11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:04.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.061+0000 7ff1d2b20700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1cc0cfc50 0x7ff1cc0b11b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35946/0 (socket says 172.21.15.16:35946) 2023-05-20T01:25:04.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.061+0000 7ff1d2b20700 1 -- 172.21.15.16:0/1125660726 learned_addr learned my addr 172.21.15.16:0/1125660726 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:04.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.061+0000 7ff1d1b1e700 1 --2- 172.21.15.16:0/1125660726 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff1cc0b4750 0x7ff1cc15f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:04.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.061+0000 7ff1d231f700 1 --2- 172.21.15.16:0/1125660726 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff1cc0b2b40 0x7ff1cc15d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:04.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.061+0000 7ff1d2b20700 1 -- 172.21.15.16:0/1125660726 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff1cc0b4750 msgr2=0x7ff1cc15f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:04.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.061+0000 7ff1d2b20700 1 --2- 172.21.15.16:0/1125660726 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff1cc0b4750 0x7ff1cc15f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:04.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.061+0000 7ff1d2b20700 1 -- 172.21.15.16:0/1125660726 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff1cc0b2b40 msgr2=0x7ff1cc15d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:04.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.061+0000 7ff1d2b20700 1 --2- 172.21.15.16:0/1125660726 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff1cc0b2b40 0x7ff1cc15d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:04.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.061+0000 7ff1d2b20700 1 -- 172.21.15.16:0/1125660726 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1c8009550 con 0x7ff1cc0cfc50 2023-05-20T01:25:04.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.061+0000 7ff1d231f700 1 --2- 172.21.15.16:0/1125660726 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff1cc0b2b40 0x7ff1cc15d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:25:04.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.061+0000 7ff1d1b1e700 1 --2- 172.21.15.16:0/1125660726 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff1cc0b4750 0x7ff1cc15f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:25:04.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.061+0000 7ff1d2b20700 1 --2- 172.21.15.16:0/1125660726 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1cc0cfc50 0x7ff1cc0b11b0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7ff1c400eed0 tx=0x7ff1c400b0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:04.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.061+0000 7ff1c2ffd700 1 -- 172.21.15.16:0/1125660726 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1c4010040 con 0x7ff1cc0cfc50 2023-05-20T01:25:04.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.062+0000 7ff1c2ffd700 1 -- 172.21.15.16:0/1125660726 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1c400bb10 con 0x7ff1cc0cfc50 2023-05-20T01:25:04.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.062+0000 7ff1d4583700 1 -- 172.21.15.16:0/1125660726 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1cc0b4380 con 0x7ff1cc0cfc50 2023-05-20T01:25:04.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.062+0000 7ff1c2ffd700 1 -- 172.21.15.16:0/1125660726 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1c4014400 con 0x7ff1cc0cfc50 2023-05-20T01:25:04.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.062+0000 7ff1d4583700 1 -- 172.21.15.16:0/1125660726 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1cc15d0b0 con 0x7ff1cc0cfc50 2023-05-20T01:25:04.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.062+0000 7ff1d4583700 1 -- 172.21.15.16:0/1125660726 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1cc0d4000 con 0x7ff1cc0cfc50 2023-05-20T01:25:04.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.063+0000 7ff1c2ffd700 1 -- 172.21.15.16:0/1125660726 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff1c4014560 con 0x7ff1cc0cfc50 2023-05-20T01:25:04.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.063+0000 7ff1c2ffd700 1 --2- 172.21.15.16:0/1125660726 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff1b006ace0 0x7ff1b006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:04.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.063+0000 7ff1c2ffd700 1 -- 172.21.15.16:0/1125660726 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5874+0+0 (secure 0 0 0) 0x7ff1c4089aa0 con 0x7ff1cc0cfc50 2023-05-20T01:25:04.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.063+0000 7ff1d231f700 1 --2- 172.21.15.16:0/1125660726 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff1b006ace0 0x7ff1b006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:04.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.064+0000 7ff1d231f700 1 --2- 172.21.15.16:0/1125660726 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff1b006ace0 0x7ff1b006d190 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc0cdac0 tx=0x7ff1bc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:04.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.068+0000 7ff1c2ffd700 1 -- 172.21.15.16:0/1125660726 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7ff1c4054c90 con 0x7ff1cc0cfc50 2023-05-20T01:25:04.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.346+0000 7ff1d4583700 1 -- 172.21.15.16:0/1125660726 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_data_scan.TestDataScan.test_extra_data_pool_rebuild_many_files_many_workers"]} v 0) v1 -- 0x7ff1cc0747b0 con 0x7ff1cc0cfc50 2023-05-20T01:25:04.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.530+0000 7ff1c2ffd700 1 -- 172.21.15.16:0/1125660726 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_data_scan.TestDataScan.test_extra_data_pool_rebuild_many_files_many_workers"]}]=0 v357) v1 ==== 172+0+0 (secure 0 0 0) 0x7ff1c40582b0 con 0x7ff1cc0cfc50 2023-05-20T01:25:04.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.532+0000 7ff1d4583700 1 -- 172.21.15.16:0/1125660726 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff1b006ace0 msgr2=0x7ff1b006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:04.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.532+0000 7ff1d4583700 1 --2- 172.21.15.16:0/1125660726 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff1b006ace0 0x7ff1b006d190 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc0cdac0 tx=0x7ff1bc008040 comp rx=0 tx=0).stop 2023-05-20T01:25:04.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.532+0000 7ff1d4583700 1 -- 172.21.15.16:0/1125660726 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1cc0cfc50 msgr2=0x7ff1cc0b11b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:04.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.532+0000 7ff1d4583700 1 --2- 172.21.15.16:0/1125660726 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1cc0cfc50 0x7ff1cc0b11b0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7ff1c400eed0 tx=0x7ff1c400b0b0 comp rx=0 tx=0).stop 2023-05-20T01:25:04.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.532+0000 7ff1d4583700 1 -- 172.21.15.16:0/1125660726 shutdown_connections 2023-05-20T01:25:04.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.532+0000 7ff1d4583700 1 --2- 172.21.15.16:0/1125660726 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1cc0cfc50 0x7ff1cc0b11b0 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:04.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.533+0000 7ff1d4583700 1 --2- 172.21.15.16:0/1125660726 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff1cc0b2b40 0x7ff1cc15d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:04.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.533+0000 7ff1d4583700 1 --2- 172.21.15.16:0/1125660726 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff1b006ace0 0x7ff1b006d190 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:04.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.533+0000 7ff1d4583700 1 --2- 172.21.15.16:0/1125660726 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff1cc0b4750 0x7ff1cc15f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:04.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.533+0000 7ff1d4583700 1 -- 172.21.15.16:0/1125660726 >> 172.21.15.16:0/1125660726 conn(0x7ff1cc09c590 msgr2=0x7ff1cc0ce730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:04.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.533+0000 7ff1d4583700 1 -- 172.21.15.16:0/1125660726 shutdown_connections 2023-05-20T01:25:04.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.533+0000 7ff1d4583700 1 -- 172.21.15.16:0/1125660726 wait complete. 2023-05-20T01:25:04.549 INFO:tasks.cephfs_test_runner:test_extra_data_pool_rebuild_many_files_many_workers (tasks.cephfs.test_data_scan.TestDataScan) ... ok 2023-05-20T01:25:04.550 INFO:tasks.cephfs_test_runner:Starting test: test_extra_data_pool_rebuild_simple (tasks.cephfs.test_data_scan.TestDataScan) 2023-05-20T01:25:04.550 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_data_scan.TestDataScan.test_extra_data_pool_rebuild_simple' 2023-05-20T01:25:04.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.670+0000 7f129931d700 1 Processor -- start 2023-05-20T01:25:04.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.671+0000 7f129931d700 1 -- start start 2023-05-20T01:25:04.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.671+0000 7f129931d700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f129415cc50 0x7f12940b4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:04.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.671+0000 7f129931d700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f129415d020 con 0x7f129406b730 2023-05-20T01:25:04.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.671+0000 7f129931d700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12940d1bf0 con 0x7f12940cf500 2023-05-20T01:25:04.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.671+0000 7f129931d700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12940cd720 con 0x7f129415cc50 2023-05-20T01:25:04.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.671+0000 7f12927fc700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f129406b730 0x7f12940b5200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:47480/0 (socket says 172.21.15.16:47480) 2023-05-20T01:25:04.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.671+0000 7f12927fc700 1 -- 172.21.15.16:0/943474204 learned_addr learned my addr 172.21.15.16:0/943474204 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:25:04.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.672+0000 7f1292ffd700 1 --2- 172.21.15.16:0/943474204 >> v2:172.21.15.132:3300/0 conn(0x7f129415cc50 0x7f12940b4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:04.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.672+0000 7f1291ffb700 1 -- 172.21.15.16:0/943474204 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 404424389 0 0) 0x7f1284001110 con 0x7f129406b730 2023-05-20T01:25:04.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.672+0000 7f1291ffb700 1 -- 172.21.15.16:0/943474204 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f12800035b0 con 0x7f129406b730 2023-05-20T01:25:04.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.672+0000 7f1291ffb700 1 -- 172.21.15.16:0/943474204 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 681749276 0 0) 0x7f12800035b0 con 0x7f129406b730 2023-05-20T01:25:04.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.672+0000 7f1292ffd700 1 -- 172.21.15.16:0/943474204 >> v1:172.21.15.132:6789/0 conn(0x7f12940cf500 legacy=0x7f12940cf920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:04.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.672+0000 7f1292ffd700 1 -- 172.21.15.16:0/943474204 >> v1:172.21.15.16:6789/0 conn(0x7f129406b730 legacy=0x7f12940b5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:04.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.672+0000 7f1292ffd700 1 -- 172.21.15.16:0/943474204 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12940d18c0 con 0x7f129415cc50 2023-05-20T01:25:04.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.672+0000 7f1292ffd700 1 --2- 172.21.15.16:0/943474204 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f129415cc50 0x7f12940b4cc0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f127c009840 tx=0x7f127c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee7b72969b55e9c3 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:04.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.672+0000 7f1291ffb700 1 -- 172.21.15.16:0/943474204 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1284001800 con 0x7f129415cc50 2023-05-20T01:25:04.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.672+0000 7f1291ffb700 1 -- 172.21.15.16:0/943474204 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f127c004030 con 0x7f129415cc50 2023-05-20T01:25:04.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.672+0000 7f1291ffb700 1 -- 172.21.15.16:0/943474204 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f127c0043c0 con 0x7f129415cc50 2023-05-20T01:25:04.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.673+0000 7f129931d700 1 -- 172.21.15.16:0/943474204 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f129415cc50 msgr2=0x7f12940b4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:04.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.673+0000 7f129931d700 1 --2- 172.21.15.16:0/943474204 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f129415cc50 0x7f12940b4cc0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f127c009840 tx=0x7f127c009b50 comp rx=0 tx=0).stop 2023-05-20T01:25:04.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.673+0000 7f129931d700 1 -- 172.21.15.16:0/943474204 shutdown_connections 2023-05-20T01:25:04.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.673+0000 7f129931d700 1 --2- 172.21.15.16:0/943474204 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f129415cc50 0x7f12940b4cc0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:04.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.673+0000 7f129931d700 1 -- 172.21.15.16:0/943474204 >> 172.21.15.16:0/943474204 conn(0x7f1294072860 msgr2=0x7f129406b020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:04.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.673+0000 7f129931d700 1 -- 172.21.15.16:0/943474204 shutdown_connections 2023-05-20T01:25:04.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.673+0000 7f129931d700 1 -- 172.21.15.16:0/943474204 wait complete. 2023-05-20T01:25:04.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.674+0000 7f129931d700 1 Processor -- start 2023-05-20T01:25:04.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.674+0000 7f129931d700 1 -- start start 2023-05-20T01:25:04.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.675+0000 7f129931d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f129406b730 0x7f12940b88a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:04.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.675+0000 7f129931d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12940cf500 0x7f129414f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:04.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.675+0000 7f129931d700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f129415cc50 0x7f12940b6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:04.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.675+0000 7f129931d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12940b8de0 con 0x7f12940cf500 2023-05-20T01:25:04.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.675+0000 7f129931d700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f129414fd80 con 0x7f129406b730 2023-05-20T01:25:04.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.675+0000 7f129931d700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f129414f820 con 0x7f129415cc50 2023-05-20T01:25:04.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.675+0000 7f12927fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12940cf500 0x7f129414f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:04.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.675+0000 7f12927fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12940cf500 0x7f129414f2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35958/0 (socket says 172.21.15.16:35958) 2023-05-20T01:25:04.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.675+0000 7f12927fc700 1 -- 172.21.15.16:0/682143886 learned_addr learned my addr 172.21.15.16:0/682143886 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:04.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.675+0000 7f12937fe700 1 --2- 172.21.15.16:0/682143886 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f129415cc50 0x7f12940b6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:04.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.675+0000 7f1292ffd700 1 --2- 172.21.15.16:0/682143886 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f129406b730 0x7f12940b88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:04.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.675+0000 7f12927fc700 1 -- 172.21.15.16:0/682143886 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f129415cc50 msgr2=0x7f12940b6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:04.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.676+0000 7f12927fc700 1 --2- 172.21.15.16:0/682143886 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f129415cc50 0x7f12940b6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:04.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.676+0000 7f12927fc700 1 -- 172.21.15.16:0/682143886 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f129406b730 msgr2=0x7f12940b88a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:04.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.676+0000 7f12927fc700 1 --2- 172.21.15.16:0/682143886 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f129406b730 0x7f12940b88a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:04.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.676+0000 7f12927fc700 1 -- 172.21.15.16:0/682143886 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f127c0094a0 con 0x7f12940cf500 2023-05-20T01:25:04.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.676+0000 7f12937fe700 1 --2- 172.21.15.16:0/682143886 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f129415cc50 0x7f12940b6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:25:04.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.676+0000 7f12927fc700 1 --2- 172.21.15.16:0/682143886 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12940cf500 0x7f129414f2e0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f128400c780 tx=0x7f128400cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:04.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.676+0000 7f128b7fe700 1 -- 172.21.15.16:0/682143886 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1284007560 con 0x7f12940cf500 2023-05-20T01:25:04.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.676+0000 7f128b7fe700 1 -- 172.21.15.16:0/682143886 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12840076e0 con 0x7f12940cf500 2023-05-20T01:25:04.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.676+0000 7f128b7fe700 1 -- 172.21.15.16:0/682143886 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1284004ab0 con 0x7f12940cf500 2023-05-20T01:25:04.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.676+0000 7f129931d700 1 -- 172.21.15.16:0/682143886 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12940cdca0 con 0x7f12940cf500 2023-05-20T01:25:04.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.676+0000 7f129931d700 1 -- 172.21.15.16:0/682143886 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12940cd9c0 con 0x7f12940cf500 2023-05-20T01:25:04.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.677+0000 7f129931d700 1 -- 172.21.15.16:0/682143886 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12940cdf80 con 0x7f12940cf500 2023-05-20T01:25:04.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.677+0000 7f128b7fe700 1 -- 172.21.15.16:0/682143886 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1284004c10 con 0x7f12940cf500 2023-05-20T01:25:04.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.678+0000 7f128b7fe700 1 --2- 172.21.15.16:0/682143886 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f127406ace0 0x7f127406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:04.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.678+0000 7f128b7fe700 1 -- 172.21.15.16:0/682143886 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5874+0+0 (secure 0 0 0) 0x7f1284088350 con 0x7f12940cf500 2023-05-20T01:25:04.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.678+0000 7f1292ffd700 1 --2- 172.21.15.16:0/682143886 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f127406ace0 0x7f127406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:04.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.678+0000 7f1292ffd700 1 --2- 172.21.15.16:0/682143886 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f127406ace0 0x7f127406d190 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f127c000c00 tx=0x7f127c013040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:04.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.682+0000 7f128b7fe700 1 -- 172.21.15.16:0/682143886 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f1284053540 con 0x7f12940cf500 2023-05-20T01:25:04.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:04.960+0000 7f129931d700 1 -- 172.21.15.16:0/682143886 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_data_scan.TestDataScan.test_extra_data_pool_rebuild_simple"]} v 0) v1 -- 0x7f12940cc9f0 con 0x7f12940cf500 2023-05-20T01:25:05.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:05.535+0000 7f128b7fe700 1 -- 172.21.15.16:0/682143886 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_data_scan.TestDataScan.test_extra_data_pool_rebuild_simple"]}]=0 v358) v1 ==== 158+0+0 (secure 0 0 0) 0x7f1284056b60 con 0x7f12940cf500 2023-05-20T01:25:05.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:05.538+0000 7f129931d700 1 -- 172.21.15.16:0/682143886 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f127406ace0 msgr2=0x7f127406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:05.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:05.538+0000 7f129931d700 1 --2- 172.21.15.16:0/682143886 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f127406ace0 0x7f127406d190 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f127c000c00 tx=0x7f127c013040 comp rx=0 tx=0).stop 2023-05-20T01:25:05.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:05.538+0000 7f129931d700 1 -- 172.21.15.16:0/682143886 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12940cf500 msgr2=0x7f129414f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:05.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:05.538+0000 7f129931d700 1 --2- 172.21.15.16:0/682143886 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12940cf500 0x7f129414f2e0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f128400c780 tx=0x7f128400cb40 comp rx=0 tx=0).stop 2023-05-20T01:25:05.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:05.538+0000 7f129931d700 1 -- 172.21.15.16:0/682143886 shutdown_connections 2023-05-20T01:25:05.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:05.538+0000 7f129931d700 1 --2- 172.21.15.16:0/682143886 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f127406ace0 0x7f127406d190 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:05.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:05.538+0000 7f129931d700 1 --2- 172.21.15.16:0/682143886 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f129415cc50 0x7f12940b6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:05.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:05.538+0000 7f129931d700 1 --2- 172.21.15.16:0/682143886 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f129406b730 0x7f12940b88a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:05.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:05.538+0000 7f129931d700 1 --2- 172.21.15.16:0/682143886 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12940cf500 0x7f129414f2e0 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:05.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:05.538+0000 7f129931d700 1 -- 172.21.15.16:0/682143886 >> 172.21.15.16:0/682143886 conn(0x7f1294072860 msgr2=0x7f1294156da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:05.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:05.538+0000 7f129931d700 1 -- 172.21.15.16:0/682143886 shutdown_connections 2023-05-20T01:25:05.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:05.538+0000 7f129931d700 1 -- 172.21.15.16:0/682143886 wait complete. 2023-05-20T01:25:05.557 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-05-20T01:25:05.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:05.689+0000 7fbb14d66700 1 Processor -- start 2023-05-20T01:25:05.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:05.689+0000 7fbb14d66700 1 -- start start 2023-05-20T01:25:05.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:05.690+0000 7fbb14d66700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbb100cec80 0x7fbb100cf0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:05.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:05.690+0000 7fbb14d66700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbb100cc110 0x7fbb100cc590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:05.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:05.690+0000 7fbb14d66700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb100ce4d0 con 0x7fbb1015ab80 2023-05-20T01:25:05.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:05.690+0000 7fbb14d66700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb100cb700 con 0x7fbb100cec80 2023-05-20T01:25:05.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:05.690+0000 7fbb14d66700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb100cf630 con 0x7fbb100cc110 2023-05-20T01:25:05.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:05.690+0000 7fbb0dd9b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbb100cec80 0x7fbb100cf0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:05.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:05.690+0000 7fbb0dd9b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbb100cec80 0x7fbb100cf0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35966/0 (socket says 172.21.15.16:35966) 2023-05-20T01:25:05.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:05.690+0000 7fbb0e59c700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fbb1015ab80 0x7fbb100cda80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:47492/0 (socket says 172.21.15.16:47492) 2023-05-20T01:25:05.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:05.690+0000 7fbb0dd9b700 1 -- 172.21.15.16:0/841003699 learned_addr learned my addr 172.21.15.16:0/841003699 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:05.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:05.690+0000 7fbb0ed9d700 1 --2- 172.21.15.16:0/841003699 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbb100cc110 0x7fbb100cc590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:05.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:05.690+0000 7fbb0dd9b700 1 -- 172.21.15.16:0/841003699 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbb100cc110 msgr2=0x7fbb100cc590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:05.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:05.690+0000 7fbb0dd9b700 1 --2- 172.21.15.16:0/841003699 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbb100cc110 0x7fbb100cc590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:05.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:05.690+0000 7fbb0dd9b700 1 -- 172.21.15.16:0/841003699 >> v1:172.21.15.16:6789/0 conn(0x7fbb1015ab80 legacy=0x7fbb100cda80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:05.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:05.690+0000 7fbb0dd9b700 1 -- 172.21.15.16:0/841003699 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb100ce7b0 con 0x7fbb100cec80 2023-05-20T01:25:05.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:05.690+0000 7fbb0d59a700 1 -- 172.21.15.16:0/841003699 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 623823737 0 0) 0x7fbb100ce4d0 con 0x7fbb1015ab80 2023-05-20T01:25:05.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:05.690+0000 7fbb0dd9b700 1 --2- 172.21.15.16:0/841003699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb100cec80 0x7fbb100cf0f0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fbb000098f0 tx=0x7fbb00009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e176b972ed60e00 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:05.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:05.691+0000 7fbb0d59a700 1 -- 172.21.15.16:0/841003699 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb0000b8a0 con 0x7fbb100cec80 2023-05-20T01:25:05.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:05.691+0000 7fbb0d59a700 1 -- 172.21.15.16:0/841003699 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb0000ba00 con 0x7fbb100cec80 2023-05-20T01:25:05.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:05.691+0000 7fbb0d59a700 1 -- 172.21.15.16:0/841003699 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb0000bd90 con 0x7fbb100cec80 2023-05-20T01:25:05.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:05.691+0000 7fbb14d66700 1 -- 172.21.15.16:0/841003699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb100cec80 msgr2=0x7fbb100cf0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:05.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:05.691+0000 7fbb14d66700 1 --2- 172.21.15.16:0/841003699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb100cec80 0x7fbb100cf0f0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fbb000098f0 tx=0x7fbb00009c00 comp rx=0 tx=0).stop 2023-05-20T01:25:05.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:05.691+0000 7fbb14d66700 1 -- 172.21.15.16:0/841003699 shutdown_connections 2023-05-20T01:25:05.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:05.691+0000 7fbb14d66700 1 --2- 172.21.15.16:0/841003699 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbb100cc110 0x7fbb100cc590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:05.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:05.691+0000 7fbb14d66700 1 --2- 172.21.15.16:0/841003699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb100cec80 0x7fbb100cf0f0 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:05.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:05.691+0000 7fbb14d66700 1 -- 172.21.15.16:0/841003699 >> 172.21.15.16:0/841003699 conn(0x7fbb1006ae30 msgr2=0x7fbb1009a400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:05.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:05.691+0000 7fbb14d66700 1 -- 172.21.15.16:0/841003699 shutdown_connections 2023-05-20T01:25:05.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:05.691+0000 7fbb14d66700 1 -- 172.21.15.16:0/841003699 wait complete. 2023-05-20T01:25:05.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:05.693+0000 7fbb14d66700 1 Processor -- start 2023-05-20T01:25:05.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:05.693+0000 7fbb14d66700 1 -- start start 2023-05-20T01:25:05.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:05.694+0000 7fbb14d66700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbb100cc110 0x7fbb100c9150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:05.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:05.694+0000 7fbb14d66700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbb100cec80 0x7fbb1006cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:05.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:05.694+0000 7fbb14d66700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb1015ab80 0x7fbb101596f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:05.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:05.694+0000 7fbb14d66700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb100b0a90 con 0x7fbb1015ab80 2023-05-20T01:25:05.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:05.694+0000 7fbb14d66700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb100b0d70 con 0x7fbb100cec80 2023-05-20T01:25:05.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:05.694+0000 7fbb14d66700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb100b1050 con 0x7fbb100cc110 2023-05-20T01:25:05.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:05.694+0000 7fbb0ed9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb1015ab80 0x7fbb101596f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:05.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:05.694+0000 7fbb0ed9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb1015ab80 0x7fbb101596f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35972/0 (socket says 172.21.15.16:35972) 2023-05-20T01:25:05.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:05.694+0000 7fbb0ed9d700 1 -- 172.21.15.16:0/3518547160 learned_addr learned my addr 172.21.15.16:0/3518547160 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:05.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:05.694+0000 7fbb0e59c700 1 --2- 172.21.15.16:0/3518547160 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbb100cc110 0x7fbb100c9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:05.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:05.694+0000 7fbb0dd9b700 1 --2- 172.21.15.16:0/3518547160 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbb100cec80 0x7fbb1006cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:05.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:05.694+0000 7fbb0ed9d700 1 -- 172.21.15.16:0/3518547160 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbb100cc110 msgr2=0x7fbb100c9150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:05.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:05.694+0000 7fbb0ed9d700 1 --2- 172.21.15.16:0/3518547160 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbb100cc110 0x7fbb100c9150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:05.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:05.694+0000 7fbb0ed9d700 1 -- 172.21.15.16:0/3518547160 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbb100cec80 msgr2=0x7fbb1006cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:05.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:05.694+0000 7fbb0ed9d700 1 --2- 172.21.15.16:0/3518547160 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbb100cec80 0x7fbb1006cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:05.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:05.694+0000 7fbb0ed9d700 1 -- 172.21.15.16:0/3518547160 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb00009550 con 0x7fbb1015ab80 2023-05-20T01:25:05.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:05.694+0000 7fbb0e59c700 1 --2- 172.21.15.16:0/3518547160 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbb100cc110 0x7fbb100c9150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:25:05.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:05.695+0000 7fbb0ed9d700 1 --2- 172.21.15.16:0/3518547160 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb1015ab80 0x7fbb101596f0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fbb08007e40 tx=0x7fbb0800e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:05.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:05.695+0000 7fbafeffd700 1 -- 172.21.15.16:0/3518547160 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb0800eea0 con 0x7fbb1015ab80 2023-05-20T01:25:05.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:05.695+0000 7fbafeffd700 1 -- 172.21.15.16:0/3518547160 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb08014420 con 0x7fbb1015ab80 2023-05-20T01:25:05.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:05.695+0000 7fbb14d66700 1 -- 172.21.15.16:0/3518547160 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb100cbbf0 con 0x7fbb1015ab80 2023-05-20T01:25:05.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:05.695+0000 7fbafeffd700 1 -- 172.21.15.16:0/3518547160 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb080147b0 con 0x7fbb1015ab80 2023-05-20T01:25:05.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:05.695+0000 7fbb14d66700 1 -- 172.21.15.16:0/3518547160 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb100cb910 con 0x7fbb1015ab80 2023-05-20T01:25:05.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:05.696+0000 7fbb14d66700 1 -- 172.21.15.16:0/3518547160 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb100ce190 con 0x7fbb1015ab80 2023-05-20T01:25:05.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:05.696+0000 7fbafeffd700 1 -- 172.21.15.16:0/3518547160 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fbb08014910 con 0x7fbb1015ab80 2023-05-20T01:25:05.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:05.696+0000 7fbafeffd700 1 --2- 172.21.15.16:0/3518547160 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fbaf006ace0 0x7fbaf006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:05.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:05.697+0000 7fbafeffd700 1 -- 172.21.15.16:0/3518547160 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5874+0+0 (secure 0 0 0) 0x7fbb08089ba0 con 0x7fbb1015ab80 2023-05-20T01:25:05.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:05.697+0000 7fbb0e59c700 1 --2- 172.21.15.16:0/3518547160 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fbaf006ace0 0x7fbaf006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:05.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:05.697+0000 7fbb0e59c700 1 --2- 172.21.15.16:0/3518547160 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fbaf006ace0 0x7fbaf006d190 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fbb04000f60 tx=0x7fbb04007d30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:05.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:05.701+0000 7fbafeffd700 1 -- 172.21.15.16:0/3518547160 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fbb08054d00 con 0x7fbb1015ab80 2023-05-20T01:25:05.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:05.902+0000 7fbb14d66700 1 -- 172.21.15.16:0/3518547160 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fbb100747b0 con 0x7fbb1015ab80 2023-05-20T01:25:05.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:05.905+0000 7fbafeffd700 1 -- 172.21.15.16:0/3518547160 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7fbb08058320 con 0x7fbb1015ab80 2023-05-20T01:25:05.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:05.907+0000 7fbb14d66700 1 -- 172.21.15.16:0/3518547160 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fbaf006ace0 msgr2=0x7fbaf006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:05.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:05.907+0000 7fbb14d66700 1 --2- 172.21.15.16:0/3518547160 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fbaf006ace0 0x7fbaf006d190 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fbb04000f60 tx=0x7fbb04007d30 comp rx=0 tx=0).stop 2023-05-20T01:25:05.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:05.907+0000 7fbb14d66700 1 -- 172.21.15.16:0/3518547160 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb1015ab80 msgr2=0x7fbb101596f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:05.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:05.907+0000 7fbb14d66700 1 --2- 172.21.15.16:0/3518547160 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb1015ab80 0x7fbb101596f0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fbb08007e40 tx=0x7fbb0800e3c0 comp rx=0 tx=0).stop 2023-05-20T01:25:05.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:05.908+0000 7fbb14d66700 1 -- 172.21.15.16:0/3518547160 shutdown_connections 2023-05-20T01:25:05.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:05.908+0000 7fbb14d66700 1 --2- 172.21.15.16:0/3518547160 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb1015ab80 0x7fbb101596f0 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:05.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:05.908+0000 7fbb14d66700 1 --2- 172.21.15.16:0/3518547160 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fbaf006ace0 0x7fbaf006d190 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:05.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:05.908+0000 7fbb14d66700 1 --2- 172.21.15.16:0/3518547160 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbb100cc110 0x7fbb100c9150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:05.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:05.908+0000 7fbb14d66700 1 --2- 172.21.15.16:0/3518547160 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbb100cec80 0x7fbb1006cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:05.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:05.908+0000 7fbb14d66700 1 -- 172.21.15.16:0/3518547160 >> 172.21.15.16:0/3518547160 conn(0x7fbb1006ae30 msgr2=0x7fbb1006e690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:05.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:05.908+0000 7fbb14d66700 1 -- 172.21.15.16:0/3518547160 shutdown_connections 2023-05-20T01:25:05.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:05.908+0000 7fbb14d66700 1 -- 172.21.15.16:0/3518547160 wait complete. 2023-05-20T01:25:05.924 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-05-20T01:25:05.924 DEBUG:teuthology.orchestra.run.smithi016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-05-20T01:25:05.970 DEBUG:teuthology.orchestra.run.smithi132:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-05-20T01:25:06.023 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-05-20T01:25:06.024 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-05-20T01:25:06.024 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2023-05-20T01:25:06.042 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16287364k,nr_inodes=4071841,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=38,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16696 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3261172k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-05-20T01:25:06.043 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2023-05-20T01:25:06.043 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/proc/self/mounts of=/dev/stdout 2023-05-20T01:25:06.063 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284284k,nr_inodes=4071071,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=45,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16175 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260556k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-05-20T01:25:06.064 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-05-20T01:25:06.065 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2023-05-20T01:25:06.098 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16287364k,nr_inodes=4071841,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=38,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16696 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3261172k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-05-20T01:25:06.099 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2023-05-20T01:25:06.099 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/proc/self/mounts of=/dev/stdout 2023-05-20T01:25:06.122 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284284k,nr_inodes=4071071,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=45,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16175 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260556k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-05-20T01:25:06.122 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-05-20T01:25:06.122 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-20T01:25:06.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.259+0000 7f176ccca700 1 Processor -- start 2023-05-20T01:25:06.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.259+0000 7f176ccca700 1 -- start start 2023-05-20T01:25:06.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.260+0000 7f176ccca700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f176806a510 0x7f176815e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:06.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.260+0000 7f176ccca700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f1768071340 0x7f1768071780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:06.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.260+0000 7f176ccca700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f176815e060 con 0x7f176806a8e0 2023-05-20T01:25:06.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.260+0000 7f176ccca700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f176815d450 con 0x7f176806a510 2023-05-20T01:25:06.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.260+0000 7f176ccca700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f176815d730 con 0x7f1768071340 2023-05-20T01:25:06.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.260+0000 7f176659c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f176806a510 0x7f176815e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:06.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.260+0000 7f1765d9b700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f176806a8e0 0x7f17680cf5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:47506/0 (socket says 172.21.15.16:47506) 2023-05-20T01:25:06.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.260+0000 7f1765d9b700 1 -- 172.21.15.16:0/679192523 learned_addr learned my addr 172.21.15.16:0/679192523 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:25:06.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.260+0000 7f176659c700 1 -- 172.21.15.16:0/679192523 >> v2:172.21.15.132:3300/0 conn(0x7f1768071340 msgr2=0x7f1768071780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:06.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.260+0000 7f1766d9d700 1 --2- 172.21.15.16:0/679192523 >> v2:172.21.15.132:3300/0 conn(0x7f1768071340 0x7f1768071780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:06.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.260+0000 7f176659c700 1 --2- 172.21.15.16:0/679192523 >> v2:172.21.15.132:3300/0 conn(0x7f1768071340 0x7f1768071780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:06.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.260+0000 7f176659c700 1 -- 172.21.15.16:0/679192523 >> v1:172.21.15.16:6789/0 conn(0x7f176806a8e0 legacy=0x7f17680cf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:06.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.260+0000 7f176559a700 1 -- 172.21.15.16:0/679192523 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2165868801 0 0) 0x7f176815e060 con 0x7f176806a8e0 2023-05-20T01:25:06.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.260+0000 7f176659c700 1 -- 172.21.15.16:0/679192523 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f176815da10 con 0x7f176806a510 2023-05-20T01:25:06.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.261+0000 7f176659c700 1 --2- 172.21.15.16:0/679192523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f176806a510 0x7f176815e2a0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f17680c6c50 tx=0x7f175c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a08debc708d311f server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:06.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.261+0000 7f176559a700 1 -- 172.21.15.16:0/679192523 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f175c011830 con 0x7f176806a510 2023-05-20T01:25:06.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.261+0000 7f176559a700 1 -- 172.21.15.16:0/679192523 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f175c011990 con 0x7f176806a510 2023-05-20T01:25:06.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.261+0000 7f176559a700 1 -- 172.21.15.16:0/679192523 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f175c011d20 con 0x7f176806a510 2023-05-20T01:25:06.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.261+0000 7f176ccca700 1 -- 172.21.15.16:0/679192523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f176806a510 msgr2=0x7f176815e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:06.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.261+0000 7f176ccca700 1 --2- 172.21.15.16:0/679192523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f176806a510 0x7f176815e2a0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f17680c6c50 tx=0x7f175c009b20 comp rx=0 tx=0).stop 2023-05-20T01:25:06.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.261+0000 7f176ccca700 1 -- 172.21.15.16:0/679192523 shutdown_connections 2023-05-20T01:25:06.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.261+0000 7f176ccca700 1 --2- 172.21.15.16:0/679192523 >> v2:172.21.15.132:3300/0 conn(0x7f1768071340 0x7f1768071780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:06.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.261+0000 7f176ccca700 1 --2- 172.21.15.16:0/679192523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f176806a510 0x7f176815e2a0 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:06.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.261+0000 7f176ccca700 1 -- 172.21.15.16:0/679192523 >> 172.21.15.16:0/679192523 conn(0x7f17680d0c30 msgr2=0x7f17680b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:06.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.261+0000 7f176ccca700 1 -- 172.21.15.16:0/679192523 shutdown_connections 2023-05-20T01:25:06.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.261+0000 7f176ccca700 1 -- 172.21.15.16:0/679192523 wait complete. 2023-05-20T01:25:06.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.263+0000 7f176ccca700 1 Processor -- start 2023-05-20T01:25:06.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.263+0000 7f176ccca700 1 -- start start 2023-05-20T01:25:06.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.263+0000 7f176ccca700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f176806a510 0x7f176815b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:06.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.263+0000 7f176ccca700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f176806a8e0 0x7f17681e5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:06.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.263+0000 7f176ccca700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1768071340 0x7f17681e57a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:06.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.263+0000 7f176ccca700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17680c5dd0 con 0x7f176806a510 2023-05-20T01:25:06.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.263+0000 7f176ccca700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f176815bcd0 con 0x7f176806a8e0 2023-05-20T01:25:06.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.263+0000 7f176ccca700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17680c51c0 con 0x7f1768071340 2023-05-20T01:25:06.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.264+0000 7f176659c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f176806a510 0x7f176815b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:06.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.264+0000 7f176659c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f176806a510 0x7f176815b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35984/0 (socket says 172.21.15.16:35984) 2023-05-20T01:25:06.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.264+0000 7f176659c700 1 -- 172.21.15.16:0/1423055543 learned_addr learned my addr 172.21.15.16:0/1423055543 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:06.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.264+0000 7f1765d9b700 1 --2- 172.21.15.16:0/1423055543 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f176806a8e0 0x7f17681e5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:06.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.264+0000 7f1766d9d700 1 --2- 172.21.15.16:0/1423055543 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1768071340 0x7f17681e57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:06.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.264+0000 7f176659c700 1 -- 172.21.15.16:0/1423055543 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1768071340 msgr2=0x7f17681e57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:06.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.264+0000 7f176659c700 1 --2- 172.21.15.16:0/1423055543 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1768071340 0x7f17681e57a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:06.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.264+0000 7f176659c700 1 -- 172.21.15.16:0/1423055543 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f176806a8e0 msgr2=0x7f17681e5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:06.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.264+0000 7f176659c700 1 --2- 172.21.15.16:0/1423055543 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f176806a8e0 0x7f17681e5260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:06.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.264+0000 7f176659c700 1 -- 172.21.15.16:0/1423055543 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f175c0094a0 con 0x7f176806a510 2023-05-20T01:25:06.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.264+0000 7f1766d9d700 1 --2- 172.21.15.16:0/1423055543 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1768071340 0x7f17681e57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:25:06.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.264+0000 7f1765d9b700 1 --2- 172.21.15.16:0/1423055543 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f176806a8e0 0x7f17681e5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:25:06.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.264+0000 7f176659c700 1 --2- 172.21.15.16:0/1423055543 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f176806a510 0x7f176815b790 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f175c014040 tx=0x7f175c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:06.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.264+0000 7f1756ffd700 1 -- 172.21.15.16:0/1423055543 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f175c0038a0 con 0x7f176806a510 2023-05-20T01:25:06.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.264+0000 7f1756ffd700 1 -- 172.21.15.16:0/1423055543 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f175c01e070 con 0x7f176806a510 2023-05-20T01:25:06.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.264+0000 7f176ccca700 1 -- 172.21.15.16:0/1423055543 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17680cfd00 con 0x7f176806a510 2023-05-20T01:25:06.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.264+0000 7f1756ffd700 1 -- 172.21.15.16:0/1423055543 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f175c003c40 con 0x7f176806a510 2023-05-20T01:25:06.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.264+0000 7f176ccca700 1 -- 172.21.15.16:0/1423055543 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f176815dcf0 con 0x7f176806a510 2023-05-20T01:25:06.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.265+0000 7f176ccca700 1 -- 172.21.15.16:0/1423055543 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f176806ad50 con 0x7f176806a510 2023-05-20T01:25:06.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.266+0000 7f1756ffd700 1 -- 172.21.15.16:0/1423055543 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f175c02d450 con 0x7f176806a510 2023-05-20T01:25:06.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.266+0000 7f1756ffd700 1 --2- 172.21.15.16:0/1423055543 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f174806ace0 0x7f174806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:06.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.266+0000 7f1756ffd700 1 -- 172.21.15.16:0/1423055543 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5874+0+0 (secure 0 0 0) 0x7f175c08a600 con 0x7f176806a510 2023-05-20T01:25:06.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.266+0000 7f1765d9b700 1 --2- 172.21.15.16:0/1423055543 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f174806ace0 0x7f174806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:06.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.267+0000 7f1765d9b700 1 --2- 172.21.15.16:0/1423055543 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f174806ace0 0x7f174806d190 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f17681e8e90 tx=0x7f1758007cc0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:06.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.271+0000 7f1756ffd700 1 -- 172.21.15.16:0/1423055543 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f175c055760 con 0x7f176806a510 2023-05-20T01:25:06.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.514+0000 7f176ccca700 1 -- 172.21.15.16:0/1423055543 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f17680747b0 con 0x7f176806a510 2023-05-20T01:25:06.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.515+0000 7f1756ffd700 1 -- 172.21.15.16:0/1423055543 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+3532 (secure 0 0 0) 0x7f175c020de0 con 0x7f176806a510 2023-05-20T01:25:06.520 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:25:06.520 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":51,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5873,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3769502841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3769502841},{"type":"v1","addr":"172.21.15.16:6835","nonce":3769502841}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45},{"gid":5879,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/688624373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":688624373},{"type":"v1","addr":"172.21.15.132:6834","nonce":688624373}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45},{"gid":6307,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/1655712081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":1655712081},{"type":"v1","addr":"172.21.15.132:6835","nonce":1655712081}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45},{"gid":7026,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/3834803477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":3834803477},{"type":"v1","addr":"172.21.15.16:6837","nonce":3834803477}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51}],"filesystems":[]} 2023-05-20T01:25:06.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.517+0000 7f176ccca700 1 -- 172.21.15.16:0/1423055543 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f174806ace0 msgr2=0x7f174806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:06.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.517+0000 7f176ccca700 1 --2- 172.21.15.16:0/1423055543 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f174806ace0 0x7f174806d190 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f17681e8e90 tx=0x7f1758007cc0 comp rx=0 tx=0).stop 2023-05-20T01:25:06.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.517+0000 7f176ccca700 1 -- 172.21.15.16:0/1423055543 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f176806a510 msgr2=0x7f176815b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:06.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.517+0000 7f176ccca700 1 --2- 172.21.15.16:0/1423055543 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f176806a510 0x7f176815b790 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f175c014040 tx=0x7f175c004750 comp rx=0 tx=0).stop 2023-05-20T01:25:06.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.517+0000 7f176ccca700 1 -- 172.21.15.16:0/1423055543 shutdown_connections 2023-05-20T01:25:06.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.517+0000 7f176ccca700 1 --2- 172.21.15.16:0/1423055543 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f174806ace0 0x7f174806d190 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:06.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.517+0000 7f176ccca700 1 --2- 172.21.15.16:0/1423055543 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1768071340 0x7f17681e57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:06.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.517+0000 7f176ccca700 1 --2- 172.21.15.16:0/1423055543 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f176806a510 0x7f176815b790 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:06.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.517+0000 7f176ccca700 1 --2- 172.21.15.16:0/1423055543 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f176806a8e0 0x7f17681e5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:06.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.517+0000 7f176ccca700 1 -- 172.21.15.16:0/1423055543 >> 172.21.15.16:0/1423055543 conn(0x7f17680d0c30 msgr2=0x7f1768071780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:06.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.517+0000 7f176ccca700 1 -- 172.21.15.16:0/1423055543 shutdown_connections 2023-05-20T01:25:06.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.517+0000 7f176ccca700 1 -- 172.21.15.16:0/1423055543 wait complete. 2023-05-20T01:25:06.525 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 51 2023-05-20T01:25:06.534 INFO:tasks.ceph.mds.a:Restarting daemon 2023-05-20T01:25:06.534 INFO:tasks.ceph.mds.a:Stopping old one... 2023-05-20T01:25:06.535 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-05-20T01:25:06.535 INFO:teuthology.orchestra.run:waiting for 300 2023-05-20T01:25:06.535 INFO:tasks.ceph.mds.c:Restarting daemon 2023-05-20T01:25:06.535 INFO:tasks.ceph.mds.c:Stopping old one... 2023-05-20T01:25:06.536 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-05-20T01:25:06.536 INFO:teuthology.orchestra.run:waiting for 300 2023-05-20T01:25:06.536 INFO:tasks.ceph.mds.b:Restarting daemon 2023-05-20T01:25:06.536 INFO:tasks.ceph.mds.b:Stopping old one... 2023-05-20T01:25:06.537 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-05-20T01:25:06.537 INFO:teuthology.orchestra.run:waiting for 300 2023-05-20T01:25:06.537 INFO:tasks.ceph.mds.d:Restarting daemon 2023-05-20T01:25:06.537 INFO:tasks.ceph.mds.d:Stopping old one... 2023-05-20T01:25:06.537 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-05-20T01:25:06.538 INFO:teuthology.orchestra.run:waiting for 300 2023-05-20T01:25:06.546 INFO:tasks.ceph.mds.a:Stopped 2023-05-20T01:25:06.546 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-05-20T01:25:06.548 INFO:tasks.ceph.mds.c:Stopped 2023-05-20T01:25:06.548 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-05-20T01:25:06.550 INFO:tasks.ceph.mds.b:Stopped 2023-05-20T01:25:06.551 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-05-20T01:25:06.552 INFO:tasks.ceph.mds.d:Stopped 2023-05-20T01:25:06.552 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-05-20T01:25:06.589 INFO:tasks.ceph.mds.a:Started 2023-05-20T01:25:06.589 DEBUG:teuthology.parallel:result is None 2023-05-20T01:25:06.590 INFO:tasks.ceph.mds.c:Started 2023-05-20T01:25:06.590 DEBUG:teuthology.parallel:result is None 2023-05-20T01:25:06.592 INFO:tasks.ceph.mds.b:Started 2023-05-20T01:25:06.593 DEBUG:teuthology.parallel:result is None 2023-05-20T01:25:06.593 INFO:tasks.ceph.mds.d:Started 2023-05-20T01:25:06.594 DEBUG:teuthology.parallel:result is None 2023-05-20T01:25:06.594 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-05-20T01:25:06.672 INFO:tasks.ceph.mds.c.smithi016.stdout:starting mds.c at 2023-05-20T01:25:06.672 INFO:tasks.ceph.mds.a.smithi016.stdout:starting mds.a at 2023-05-20T01:25:06.678 INFO:tasks.ceph.mds.d.smithi132.stdout:starting mds.d at 2023-05-20T01:25:06.678 INFO:tasks.ceph.mds.b.smithi132.stdout:starting mds.b at 2023-05-20T01:25:06.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.730+0000 7f7ac1e32700 1 Processor -- start 2023-05-20T01:25:06.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.730+0000 7f7ac1e32700 1 -- start start 2023-05-20T01:25:06.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.731+0000 7f7ac1e32700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7abc0d2c10 0x7f7abc0ce080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:06.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.731+0000 7f7ac1e32700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7abc0b3b50 con 0x7f7abc0b4750 2023-05-20T01:25:06.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.731+0000 7f7ac1e32700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7abc0d3070 con 0x7f7abc0cfc50 2023-05-20T01:25:06.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.731+0000 7f7ac1e32700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7abc0cfad0 con 0x7f7abc0d2c10 2023-05-20T01:25:06.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.731+0000 7f7abaffd700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f7abc0b4750 0x7f7abc0cd970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:47544/0 (socket says 172.21.15.16:47544) 2023-05-20T01:25:06.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.731+0000 7f7abaffd700 1 -- 172.21.15.16:0/3429951925 learned_addr learned my addr 172.21.15.16:0/3429951925 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:25:06.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.731+0000 7f7abbfff700 1 --2- 172.21.15.16:0/3429951925 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7abc0d2c10 0x7f7abc0ce080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:06.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.731+0000 7f7aba7fc700 1 -- 172.21.15.16:0/3429951925 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3529615772 0 0) 0x7f7abc0b3b50 con 0x7f7abc0b4750 2023-05-20T01:25:06.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.731+0000 7f7aba7fc700 1 -- 172.21.15.16:0/3429951925 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7aa40035b0 con 0x7f7abc0b4750 2023-05-20T01:25:06.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.731+0000 7f7aba7fc700 1 -- 172.21.15.16:0/3429951925 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2495926415 0 0) 0x7f7abc0d3070 con 0x7f7abc0cfc50 2023-05-20T01:25:06.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.731+0000 7f7aba7fc700 1 -- 172.21.15.16:0/3429951925 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7abc0b3b50 con 0x7f7abc0cfc50 2023-05-20T01:25:06.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.731+0000 7f7abbfff700 1 -- 172.21.15.16:0/3429951925 >> v1:172.21.15.132:6789/0 conn(0x7f7abc0cfc50 legacy=0x7f7abc0d0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:06.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.731+0000 7f7aba7fc700 1 -- 172.21.15.16:0/3429951925 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2245884752 0 0) 0x7f7aa40035b0 con 0x7f7abc0b4750 2023-05-20T01:25:06.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.731+0000 7f7abbfff700 1 -- 172.21.15.16:0/3429951925 >> v1:172.21.15.16:6789/0 conn(0x7f7abc0b4750 legacy=0x7f7abc0cd970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:06.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.731+0000 7f7abbfff700 1 -- 172.21.15.16:0/3429951925 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7abc0d0b90 con 0x7f7abc0d2c10 2023-05-20T01:25:06.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.731+0000 7f7abbfff700 1 --2- 172.21.15.16:0/3429951925 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7abc0d2c10 0x7f7abc0ce080 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f7ab4009dc0 tx=0x7f7ab400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c4ab20f1eb567487 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:06.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.732+0000 7f7aba7fc700 1 -- 172.21.15.16:0/3429951925 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ab400e070 con 0x7f7abc0d2c10 2023-05-20T01:25:06.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.732+0000 7f7aba7fc700 1 -- 172.21.15.16:0/3429951925 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ab4003a20 con 0x7f7abc0d2c10 2023-05-20T01:25:06.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.732+0000 7f7aba7fc700 1 -- 172.21.15.16:0/3429951925 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aac001670 con 0x7f7abc0d2c10 2023-05-20T01:25:06.739 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:25:06.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.732+0000 7f7ac1e32700 1 -- 172.21.15.16:0/3429951925 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7abc0d2c10 msgr2=0x7f7abc0ce080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:06.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.732+0000 7f7ac1e32700 1 --2- 172.21.15.16:0/3429951925 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7abc0d2c10 0x7f7abc0ce080 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f7ab4009dc0 tx=0x7f7ab400b330 comp rx=0 tx=0).stop 2023-05-20T01:25:06.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.732+0000 7f7ac1e32700 1 -- 172.21.15.16:0/3429951925 shutdown_connections 2023-05-20T01:25:06.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.732+0000 7f7ac1e32700 1 --2- 172.21.15.16:0/3429951925 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7abc0d2c10 0x7f7abc0ce080 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:06.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.732+0000 7f7ac1e32700 1 -- 172.21.15.16:0/3429951925 >> 172.21.15.16:0/3429951925 conn(0x7f7abc09c590 msgr2=0x7f7abc0d3240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:06.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.732+0000 7f7ac1e32700 1 -- 172.21.15.16:0/3429951925 shutdown_connections 2023-05-20T01:25:06.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.732+0000 7f7ac1e32700 1 -- 172.21.15.16:0/3429951925 wait complete. 2023-05-20T01:25:06.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.733+0000 7f7ac1e32700 1 Processor -- start 2023-05-20T01:25:06.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.734+0000 7f7ac1e32700 1 -- start start 2023-05-20T01:25:06.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.734+0000 7f7ac1e32700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7abc0b4750 0x7f7abc1f2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:06.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.734+0000 7f7ac1e32700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7abc0cfc50 0x7f7abc1492a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:06.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.734+0000 7f7ac1e32700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7abc0d2c10 0x7f7abc1497e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:06.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.734+0000 7f7ac1e32700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7abc0c7000 con 0x7f7abc0d2c10 2023-05-20T01:25:06.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.734+0000 7f7ac1e32700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7abc15a140 con 0x7f7abc0cfc50 2023-05-20T01:25:06.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.734+0000 7f7ac1e32700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7abc15a420 con 0x7f7abc0b4750 2023-05-20T01:25:06.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.734+0000 7f7abbfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7abc0d2c10 0x7f7abc1497e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:06.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.734+0000 7f7abaffd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7abc0cfc50 0x7f7abc1492a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:06.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.734+0000 7f7abbfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7abc0d2c10 0x7f7abc1497e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36024/0 (socket says 172.21.15.16:36024) 2023-05-20T01:25:06.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.734+0000 7f7abbfff700 1 -- 172.21.15.16:0/1456771788 learned_addr learned my addr 172.21.15.16:0/1456771788 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:06.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.734+0000 7f7abb7fe700 1 --2- 172.21.15.16:0/1456771788 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7abc0b4750 0x7f7abc1f2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:06.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.735+0000 7f7abbfff700 1 -- 172.21.15.16:0/1456771788 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7abc0b4750 msgr2=0x7f7abc1f2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:06.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.735+0000 7f7abbfff700 1 --2- 172.21.15.16:0/1456771788 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7abc0b4750 0x7f7abc1f2160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:06.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.735+0000 7f7abbfff700 1 -- 172.21.15.16:0/1456771788 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7abc0cfc50 msgr2=0x7f7abc1492a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:06.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.735+0000 7f7abbfff700 1 --2- 172.21.15.16:0/1456771788 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7abc0cfc50 0x7f7abc1492a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:06.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.735+0000 7f7abbfff700 1 -- 172.21.15.16:0/1456771788 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ab4009a70 con 0x7f7abc0d2c10 2023-05-20T01:25:06.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.735+0000 7f7abbfff700 1 --2- 172.21.15.16:0/1456771788 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7abc0d2c10 0x7f7abc1497e0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f7ab4009d90 tx=0x7f7ab4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:06.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.735+0000 7f7aa3fff700 1 -- 172.21.15.16:0/1456771788 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ab40042c0 con 0x7f7abc0d2c10 2023-05-20T01:25:06.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.735+0000 7f7aa3fff700 1 -- 172.21.15.16:0/1456771788 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ab4004440 con 0x7f7abc0d2c10 2023-05-20T01:25:06.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.735+0000 7f7ac1e32700 1 -- 172.21.15.16:0/1456771788 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7abc0b4380 con 0x7f7abc0d2c10 2023-05-20T01:25:06.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.735+0000 7f7aa3fff700 1 -- 172.21.15.16:0/1456771788 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ab40047f0 con 0x7f7abc0d2c10 2023-05-20T01:25:06.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.735+0000 7f7ac1e32700 1 -- 172.21.15.16:0/1456771788 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7abc15d0b0 con 0x7f7abc0d2c10 2023-05-20T01:25:06.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.736+0000 7f7ac1e32700 1 -- 172.21.15.16:0/1456771788 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7abc0d4000 con 0x7f7abc0d2c10 2023-05-20T01:25:06.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.737+0000 7f7aa3fff700 1 -- 172.21.15.16:0/1456771788 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7ab401c460 con 0x7f7abc0d2c10 2023-05-20T01:25:06.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.737+0000 7f7aa3fff700 1 --2- 172.21.15.16:0/1456771788 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f7aa806ace0 0x7f7aa806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:06.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.737+0000 7f7aa3fff700 1 -- 172.21.15.16:0/1456771788 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5874+0+0 (secure 0 0 0) 0x7f7ab408a9f0 con 0x7f7abc0d2c10 2023-05-20T01:25:06.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.737+0000 7f7abb7fe700 1 --2- 172.21.15.16:0/1456771788 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f7aa806ace0 0x7f7aa806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:06.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.737+0000 7f7abb7fe700 1 --2- 172.21.15.16:0/1456771788 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f7aa806ace0 0x7f7aa806d190 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f7ab0001220 tx=0x7f7ab0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:06.752 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:25:06.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.741+0000 7f7aa3fff700 1 -- 172.21.15.16:0/1456771788 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f7ab4055b60 con 0x7f7abc0d2c10 2023-05-20T01:25:06.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:06.960+0000 7f7ac1e32700 1 -- 172.21.15.16:0/1456771788 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f7abc0d0550 con 0x7f7abc0d2c10 2023-05-20T01:25:07.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:07.711+0000 7f7aa3fff700 1 -- 172.21.15.16:0/1456771788 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v84) v1 ==== 97+0+0 (secure 0 0 0) 0x7f7ab4059180 con 0x7f7abc0d2c10 2023-05-20T01:25:07.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:07.714+0000 7f7ac1e32700 1 -- 172.21.15.16:0/1456771788 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f7aa806ace0 msgr2=0x7f7aa806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:07.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:07.714+0000 7f7ac1e32700 1 --2- 172.21.15.16:0/1456771788 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f7aa806ace0 0x7f7aa806d190 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f7ab0001220 tx=0x7f7ab0009040 comp rx=0 tx=0).stop 2023-05-20T01:25:07.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:07.714+0000 7f7ac1e32700 1 -- 172.21.15.16:0/1456771788 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7abc0d2c10 msgr2=0x7f7abc1497e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:07.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:07.714+0000 7f7ac1e32700 1 --2- 172.21.15.16:0/1456771788 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7abc0d2c10 0x7f7abc1497e0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f7ab4009d90 tx=0x7f7ab4004ad0 comp rx=0 tx=0).stop 2023-05-20T01:25:07.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:07.714+0000 7f7ac1e32700 1 -- 172.21.15.16:0/1456771788 shutdown_connections 2023-05-20T01:25:07.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:07.714+0000 7f7ac1e32700 1 --2- 172.21.15.16:0/1456771788 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7abc0d2c10 0x7f7abc1497e0 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:07.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:07.714+0000 7f7ac1e32700 1 --2- 172.21.15.16:0/1456771788 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f7aa806ace0 0x7f7aa806d190 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:07.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:07.714+0000 7f7ac1e32700 1 --2- 172.21.15.16:0/1456771788 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7abc0b4750 0x7f7abc1f2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:07.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:07.714+0000 7f7ac1e32700 1 --2- 172.21.15.16:0/1456771788 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7abc0cfc50 0x7f7abc1492a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:07.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:07.714+0000 7f7ac1e32700 1 -- 172.21.15.16:0/1456771788 >> 172.21.15.16:0/1456771788 conn(0x7f7abc09c590 msgr2=0x7f7abc0ce5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:07.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:07.714+0000 7f7ac1e32700 1 -- 172.21.15.16:0/1456771788 shutdown_connections 2023-05-20T01:25:07.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:07.714+0000 7f7ac1e32700 1 -- 172.21.15.16:0/1456771788 wait complete. 2023-05-20T01:25:07.720 INFO:teuthology.orchestra.run.smithi016.stderr: removed all blocklist entries 2023-05-20T01:25:07.733 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-05-20T01:25:07.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:07.861+0000 7fb7acf31700 1 Processor -- start 2023-05-20T01:25:07.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:07.861+0000 7fb7acf31700 1 -- start start 2023-05-20T01:25:07.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:07.862+0000 7fb7acf31700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb7a806a510 0x7fb7a815e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:07.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:07.862+0000 7fb7acf31700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fb7a8071340 0x7fb7a8071780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:07.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:07.862+0000 7fb7acf31700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7a815e060 con 0x7fb7a806a8e0 2023-05-20T01:25:07.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:07.862+0000 7fb7acf31700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7a815d450 con 0x7fb7a806a510 2023-05-20T01:25:07.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:07.862+0000 7fb7acf31700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7a815d730 con 0x7fb7a8071340 2023-05-20T01:25:07.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:07.862+0000 7fb7a659c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb7a806a510 0x7fb7a815e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:07.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:07.862+0000 7fb7a659c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb7a806a510 0x7fb7a815e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36032/0 (socket says 172.21.15.16:36032) 2023-05-20T01:25:07.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:07.862+0000 7fb7a659c700 1 -- 172.21.15.16:0/1884933529 learned_addr learned my addr 172.21.15.16:0/1884933529 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:07.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:07.862+0000 7fb7a659c700 1 -- 172.21.15.16:0/1884933529 >> v2:172.21.15.132:3300/0 conn(0x7fb7a8071340 msgr2=0x7fb7a8071780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:07.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:07.862+0000 7fb7a6d9d700 1 --2- 172.21.15.16:0/1884933529 >> v2:172.21.15.132:3300/0 conn(0x7fb7a8071340 0x7fb7a8071780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:07.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:07.862+0000 7fb7a659c700 1 --2- 172.21.15.16:0/1884933529 >> v2:172.21.15.132:3300/0 conn(0x7fb7a8071340 0x7fb7a8071780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:07.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:07.862+0000 7fb7a659c700 1 -- 172.21.15.16:0/1884933529 >> v1:172.21.15.132:6789/0 conn(0x7fb7a806a8e0 legacy=0x7fb7a80cf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:07.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:07.862+0000 7fb7a659c700 1 -- 172.21.15.16:0/1884933529 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7a815da10 con 0x7fb7a806a510 2023-05-20T01:25:07.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:07.862+0000 7fb7a659c700 1 --2- 172.21.15.16:0/1884933529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7a806a510 0x7fb7a815e2a0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fb7a80c6c50 tx=0x7fb798009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8d3ece3bf7d9ea0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:07.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:07.863+0000 7fb7a559a700 1 -- 172.21.15.16:0/1884933529 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb798004230 con 0x7fb7a806a510 2023-05-20T01:25:07.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:07.863+0000 7fb7a559a700 1 -- 172.21.15.16:0/1884933529 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb798004390 con 0x7fb7a806a510 2023-05-20T01:25:07.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:07.863+0000 7fb7a559a700 1 -- 172.21.15.16:0/1884933529 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7980036a0 con 0x7fb7a806a510 2023-05-20T01:25:07.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:07.863+0000 7fb7acf31700 1 -- 172.21.15.16:0/1884933529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7a806a510 msgr2=0x7fb7a815e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:07.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:07.863+0000 7fb7acf31700 1 --2- 172.21.15.16:0/1884933529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7a806a510 0x7fb7a815e2a0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fb7a80c6c50 tx=0x7fb798009bd0 comp rx=0 tx=0).stop 2023-05-20T01:25:07.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:07.863+0000 7fb7acf31700 1 -- 172.21.15.16:0/1884933529 shutdown_connections 2023-05-20T01:25:07.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:07.863+0000 7fb7acf31700 1 --2- 172.21.15.16:0/1884933529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7a806a510 0x7fb7a815e2a0 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:07.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:07.863+0000 7fb7acf31700 1 --2- 172.21.15.16:0/1884933529 >> v2:172.21.15.132:3300/0 conn(0x7fb7a8071340 0x7fb7a8071780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:07.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:07.863+0000 7fb7acf31700 1 -- 172.21.15.16:0/1884933529 >> 172.21.15.16:0/1884933529 conn(0x7fb7a80d0c30 msgr2=0x7fb7a80b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:07.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:07.863+0000 7fb7acf31700 1 -- 172.21.15.16:0/1884933529 shutdown_connections 2023-05-20T01:25:07.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:07.863+0000 7fb7acf31700 1 -- 172.21.15.16:0/1884933529 wait complete. 2023-05-20T01:25:07.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:07.865+0000 7fb7acf31700 1 Processor -- start 2023-05-20T01:25:07.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:07.865+0000 7fb7acf31700 1 -- start start 2023-05-20T01:25:07.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:07.865+0000 7fb7acf31700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb7a806a510 0x7fb7a8150aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:07.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:07.865+0000 7fb7acf31700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb7a806a8e0 0x7fb7a8151050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:07.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:07.865+0000 7fb7acf31700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7a8071340 0x7fb7a80c83f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:07.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:07.865+0000 7fb7acf31700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a80c4170 con 0x7fb7a8071340 2023-05-20T01:25:07.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:07.865+0000 7fb7acf31700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a80b4790 con 0x7fb7a806a8e0 2023-05-20T01:25:07.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:07.865+0000 7fb7acf31700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a80b3ef0 con 0x7fb7a806a510 2023-05-20T01:25:07.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:07.865+0000 7fb7a6d9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7a8071340 0x7fb7a80c83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:07.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:07.865+0000 7fb7a6d9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7a8071340 0x7fb7a80c83f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36040/0 (socket says 172.21.15.16:36040) 2023-05-20T01:25:07.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:07.865+0000 7fb7a6d9d700 1 -- 172.21.15.16:0/2341747917 learned_addr learned my addr 172.21.15.16:0/2341747917 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:07.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:07.866+0000 7fb7a5d9b700 1 --2- 172.21.15.16:0/2341747917 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb7a806a8e0 0x7fb7a8151050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:07.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:07.866+0000 7fb7a659c700 1 --2- 172.21.15.16:0/2341747917 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb7a806a510 0x7fb7a8150aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:07.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:07.866+0000 7fb7a6d9d700 1 -- 172.21.15.16:0/2341747917 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb7a806a510 msgr2=0x7fb7a8150aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:07.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:07.866+0000 7fb7a6d9d700 1 --2- 172.21.15.16:0/2341747917 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb7a806a510 0x7fb7a8150aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:07.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:07.866+0000 7fb7a6d9d700 1 -- 172.21.15.16:0/2341747917 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb7a806a8e0 msgr2=0x7fb7a8151050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:07.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:07.866+0000 7fb7a6d9d700 1 --2- 172.21.15.16:0/2341747917 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb7a806a8e0 0x7fb7a8151050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:07.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:07.866+0000 7fb7a6d9d700 1 -- 172.21.15.16:0/2341747917 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb798009550 con 0x7fb7a8071340 2023-05-20T01:25:07.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:07.866+0000 7fb7a6d9d700 1 --2- 172.21.15.16:0/2341747917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7a8071340 0x7fb7a80c83f0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fb7a000cd70 tx=0x7fb7a000a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:07.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:07.866+0000 7fb796ffd700 1 -- 172.21.15.16:0/2341747917 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7a0003a20 con 0x7fb7a8071340 2023-05-20T01:25:07.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:07.866+0000 7fb796ffd700 1 -- 172.21.15.16:0/2341747917 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7a0003b80 con 0x7fb7a8071340 2023-05-20T01:25:07.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:07.866+0000 7fb7acf31700 1 -- 172.21.15.16:0/2341747917 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7a80cfd00 con 0x7fb7a8071340 2023-05-20T01:25:07.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:07.866+0000 7fb796ffd700 1 -- 172.21.15.16:0/2341747917 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7a0004020 con 0x7fb7a8071340 2023-05-20T01:25:07.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:07.866+0000 7fb7acf31700 1 -- 172.21.15.16:0/2341747917 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7a815dcf0 con 0x7fb7a8071340 2023-05-20T01:25:07.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:07.867+0000 7fb7acf31700 1 -- 172.21.15.16:0/2341747917 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7a806ad50 con 0x7fb7a8071340 2023-05-20T01:25:07.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:07.868+0000 7fb796ffd700 1 -- 172.21.15.16:0/2341747917 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb7a0004180 con 0x7fb7a8071340 2023-05-20T01:25:07.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:07.868+0000 7fb796ffd700 1 --2- 172.21.15.16:0/2341747917 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb78806ace0 0x7fb78806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:07.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:07.868+0000 7fb7a659c700 1 --2- 172.21.15.16:0/2341747917 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb78806ace0 0x7fb78806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:07.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:07.868+0000 7fb796ffd700 1 -- 172.21.15.16:0/2341747917 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5530+0+0 (secure 0 0 0) 0x7fb7a0012070 con 0x7fb7a8071340 2023-05-20T01:25:07.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:07.869+0000 7fb7a659c700 1 --2- 172.21.15.16:0/2341747917 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb78806ace0 0x7fb78806d190 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fb798009130 tx=0x7fb798008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:07.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:07.873+0000 7fb796ffd700 1 -- 172.21.15.16:0/2341747917 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fb7a0054bf0 con 0x7fb7a8071340 2023-05-20T01:25:08.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.150+0000 7fb7acf31700 1 -- 172.21.15.16:0/2341747917 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fb7a80c42b0 con 0x7fb7a8071340 2023-05-20T01:25:08.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.151+0000 7fb796ffd700 1 -- 172.21.15.16:0/2341747917 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v11) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fb7a0058210 con 0x7fb7a8071340 2023-05-20T01:25:08.155 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:25:08.155 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-05-20T01:25:08.156 INFO:teuthology.orchestra.run.smithi016.stdout: "auth_dump": [ 2023-05-20T01:25:08.156 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-05-20T01:25:08.156 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.a", 2023-05-20T01:25:08.156 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQA5H2hkq5FEJBAA1qUUPSLmwo43hTQS7GSENA==", 2023-05-20T01:25:08.156 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-05-20T01:25:08.156 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-05-20T01:25:08.157 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2023-05-20T01:25:08.157 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2023-05-20T01:25:08.157 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-05-20T01:25:08.157 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:25:08.157 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-05-20T01:25:08.157 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-05-20T01:25:08.157 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.b", 2023-05-20T01:25:08.158 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQA5H2hk/2XSMBAAnGCIm7TJZsO9HyXwOWxO7w==", 2023-05-20T01:25:08.158 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-05-20T01:25:08.158 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-05-20T01:25:08.158 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2023-05-20T01:25:08.158 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2023-05-20T01:25:08.158 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-05-20T01:25:08.158 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:25:08.158 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-05-20T01:25:08.159 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-05-20T01:25:08.159 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.c", 2023-05-20T01:25:08.159 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQA5H2hkBVK8KxAAgxy9+gbttxZeygwrAX+PHQ==", 2023-05-20T01:25:08.159 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-05-20T01:25:08.159 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-05-20T01:25:08.159 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2023-05-20T01:25:08.159 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2023-05-20T01:25:08.159 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-05-20T01:25:08.160 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:25:08.160 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-05-20T01:25:08.160 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-05-20T01:25:08.160 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.d", 2023-05-20T01:25:08.160 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQA5H2hkFxEMOBAAMPvue7F+/qqOFUD/C//U+g==", 2023-05-20T01:25:08.160 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-05-20T01:25:08.160 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-05-20T01:25:08.161 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2023-05-20T01:25:08.161 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2023-05-20T01:25:08.161 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-05-20T01:25:08.161 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:25:08.161 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-05-20T01:25:08.161 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-05-20T01:25:08.161 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.0", 2023-05-20T01:25:08.162 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQA7H2hkC4WwIRAA5KMQFqldcekzEuH92rZYHA==", 2023-05-20T01:25:08.162 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-05-20T01:25:08.162 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-05-20T01:25:08.162 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-05-20T01:25:08.162 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-05-20T01:25:08.162 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:25:08.162 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-05-20T01:25:08.162 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-05-20T01:25:08.163 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.1", 2023-05-20T01:25:08.163 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQA+H2hkvMH1IBAAhOQOjCiv3r2Pl3HcSS6eAA==", 2023-05-20T01:25:08.163 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-05-20T01:25:08.163 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-05-20T01:25:08.163 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-05-20T01:25:08.163 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-05-20T01:25:08.163 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:25:08.164 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-05-20T01:25:08.164 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-05-20T01:25:08.164 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.2", 2023-05-20T01:25:08.164 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQBBH2hkSJU5IBAAdFd4IhXxISVybFyT24gb0g==", 2023-05-20T01:25:08.164 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-05-20T01:25:08.164 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-05-20T01:25:08.165 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-05-20T01:25:08.165 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-05-20T01:25:08.165 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:25:08.165 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-05-20T01:25:08.165 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-05-20T01:25:08.165 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.3", 2023-05-20T01:25:08.165 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQBEH2hkrcH8HxAANlLWDyIYycLl1TrYCBN1hQ==", 2023-05-20T01:25:08.166 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-05-20T01:25:08.166 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-05-20T01:25:08.166 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-05-20T01:25:08.166 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-05-20T01:25:08.166 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:25:08.166 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-05-20T01:25:08.166 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-05-20T01:25:08.167 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.4", 2023-05-20T01:25:08.167 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQBKH2hkZr+4JBAAzbzKd4ulGI8kVGqXpjCDog==", 2023-05-20T01:25:08.167 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-05-20T01:25:08.167 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-05-20T01:25:08.167 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-05-20T01:25:08.167 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-05-20T01:25:08.167 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:25:08.167 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-05-20T01:25:08.168 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-05-20T01:25:08.168 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.5", 2023-05-20T01:25:08.168 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQBNH2hkNV02JBAALrC7eOTnD1dQM9njk+BgLQ==", 2023-05-20T01:25:08.168 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-05-20T01:25:08.168 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-05-20T01:25:08.168 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-05-20T01:25:08.168 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-05-20T01:25:08.168 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:25:08.169 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-05-20T01:25:08.169 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-05-20T01:25:08.169 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.6", 2023-05-20T01:25:08.169 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQBQH2hkGOfBIxAABCTVPLBi5+drKpqOC8xqdw==", 2023-05-20T01:25:08.169 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-05-20T01:25:08.169 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-05-20T01:25:08.169 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-05-20T01:25:08.170 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-05-20T01:25:08.170 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:25:08.170 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-05-20T01:25:08.170 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-05-20T01:25:08.170 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.7", 2023-05-20T01:25:08.170 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQBTH2hkpwEdIxAAaI3X5ZAVJ9MfVG0MjFJvSQ==", 2023-05-20T01:25:08.170 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-05-20T01:25:08.170 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-05-20T01:25:08.171 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-05-20T01:25:08.171 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-05-20T01:25:08.171 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:25:08.171 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-05-20T01:25:08.171 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-05-20T01:25:08.171 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.0", 2023-05-20T01:25:08.171 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQA6H2hkph3EABAA0277JHCqplw1LklDfidPWQ==", 2023-05-20T01:25:08.171 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-05-20T01:25:08.172 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-05-20T01:25:08.172 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow r", 2023-05-20T01:25:08.172 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-05-20T01:25:08.172 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:25:08.172 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-05-20T01:25:08.172 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-05-20T01:25:08.172 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.1", 2023-05-20T01:25:08.173 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQA6H2hk43WQBxAApSfBSu1qEaKSGWaYAvz/Jg==", 2023-05-20T01:25:08.173 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-05-20T01:25:08.173 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-05-20T01:25:08.173 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2023-05-20T01:25:08.173 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2023-05-20T01:25:08.173 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2023-05-20T01:25:08.173 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:25:08.173 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-05-20T01:25:08.174 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-05-20T01:25:08.174 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.2", 2023-05-20T01:25:08.174 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQA6H2hk+TgtBBAAc4+W2eOVgJfNd3xXjGvz4Q==", 2023-05-20T01:25:08.174 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-05-20T01:25:08.174 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-05-20T01:25:08.174 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2023-05-20T01:25:08.175 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2023-05-20T01:25:08.175 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2023-05-20T01:25:08.175 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:25:08.175 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-05-20T01:25:08.175 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-05-20T01:25:08.175 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.3", 2023-05-20T01:25:08.175 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQA6H2hkifVADRAAQP6vtdVNYyeNPV0d+agOCw==", 2023-05-20T01:25:08.175 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-05-20T01:25:08.176 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-05-20T01:25:08.176 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2023-05-20T01:25:08.176 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2023-05-20T01:25:08.176 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2023-05-20T01:25:08.176 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:25:08.176 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-05-20T01:25:08.176 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-05-20T01:25:08.177 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.admin", 2023-05-20T01:25:08.177 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQA5H2hkb4BMBhAAw7zpfDAb7x5szkLY+zTL7Q==", 2023-05-20T01:25:08.177 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-05-20T01:25:08.177 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2023-05-20T01:25:08.177 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2023-05-20T01:25:08.177 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2023-05-20T01:25:08.178 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-05-20T01:25:08.178 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:25:08.178 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-05-20T01:25:08.178 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-05-20T01:25:08.178 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mgr.x", 2023-05-20T01:25:08.178 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQA5H2hksQbMIBAAQFm/pY5stJHNudg6zcPBdQ==", 2023-05-20T01:25:08.178 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-05-20T01:25:08.178 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2023-05-20T01:25:08.179 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile mgr", 2023-05-20T01:25:08.179 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-05-20T01:25:08.179 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:25:08.179 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-05-20T01:25:08.179 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-05-20T01:25:08.179 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mgr.y", 2023-05-20T01:25:08.179 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQA5H2hkBQg7HBAAbOJZ+55rRACVBMd2eyjqtg==", 2023-05-20T01:25:08.179 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-05-20T01:25:08.180 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2023-05-20T01:25:08.180 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile mgr", 2023-05-20T01:25:08.180 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-05-20T01:25:08.180 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:25:08.180 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:25:08.180 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2023-05-20T01:25:08.180 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-05-20T01:25:08.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.153+0000 7fb7acf31700 1 -- 172.21.15.16:0/2341747917 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb78806ace0 msgr2=0x7fb78806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:08.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.153+0000 7fb7acf31700 1 --2- 172.21.15.16:0/2341747917 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb78806ace0 0x7fb78806d190 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fb798009130 tx=0x7fb798008030 comp rx=0 tx=0).stop 2023-05-20T01:25:08.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.153+0000 7fb7acf31700 1 -- 172.21.15.16:0/2341747917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7a8071340 msgr2=0x7fb7a80c83f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:08.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.153+0000 7fb7acf31700 1 --2- 172.21.15.16:0/2341747917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7a8071340 0x7fb7a80c83f0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fb7a000cd70 tx=0x7fb7a000a570 comp rx=0 tx=0).stop 2023-05-20T01:25:08.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.153+0000 7fb7acf31700 1 -- 172.21.15.16:0/2341747917 shutdown_connections 2023-05-20T01:25:08.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.153+0000 7fb7acf31700 1 --2- 172.21.15.16:0/2341747917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7a8071340 0x7fb7a80c83f0 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:08.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.153+0000 7fb7acf31700 1 --2- 172.21.15.16:0/2341747917 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb78806ace0 0x7fb78806d190 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:08.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.153+0000 7fb7acf31700 1 --2- 172.21.15.16:0/2341747917 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb7a806a510 0x7fb7a8150aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:08.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.153+0000 7fb7acf31700 1 --2- 172.21.15.16:0/2341747917 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb7a806a8e0 0x7fb7a8151050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:08.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.153+0000 7fb7acf31700 1 -- 172.21.15.16:0/2341747917 >> 172.21.15.16:0/2341747917 conn(0x7fb7a80d0c30 msgr2=0x7fb7a80cbeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:08.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.153+0000 7fb7acf31700 1 -- 172.21.15.16:0/2341747917 shutdown_connections 2023-05-20T01:25:08.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.153+0000 7fb7acf31700 1 -- 172.21.15.16:0/2341747917 wait complete. 2023-05-20T01:25:08.184 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-20T01:25:08.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.304+0000 7f91c342c700 1 Processor -- start 2023-05-20T01:25:08.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.305+0000 7f91c342c700 1 -- start start 2023-05-20T01:25:08.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.305+0000 7f91c342c700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f91bc06b620 0x7f91bc06ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:08.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.305+0000 7f91c342c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f91bc0cb6e0 0x7f91bc0cbb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:08.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.305+0000 7f91c342c700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91bc0ce3e0 con 0x7f91bc0d1000 2023-05-20T01:25:08.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.305+0000 7f91c342c700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91bc0cdab0 con 0x7f91bc06b620 2023-05-20T01:25:08.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.305+0000 7f91c342c700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91bc0cdd90 con 0x7f91bc0cb6e0 2023-05-20T01:25:08.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.306+0000 7f91c19c9700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f91bc0cb6e0 0x7f91bc0cbb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:08.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.306+0000 7f91c09c7700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f91bc06b620 0x7f91bc06ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:08.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.306+0000 7f91c11c8700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f91bc0d1000 0x7f91bc0d13d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.16:57286/0 (socket says 172.21.15.16:57286) 2023-05-20T01:25:08.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.306+0000 7f91c11c8700 1 -- 172.21.15.16:0/4255935864 learned_addr learned my addr 172.21.15.16:0/4255935864 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:25:08.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.306+0000 7f91c19c9700 1 -- 172.21.15.16:0/4255935864 >> v2:172.21.15.132:3300/0 conn(0x7f91bc06b620 msgr2=0x7f91bc06ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:08.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.306+0000 7f91c19c9700 1 --2- 172.21.15.16:0/4255935864 >> v2:172.21.15.132:3300/0 conn(0x7f91bc06b620 0x7f91bc06ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:08.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.306+0000 7f91b3fff700 1 -- 172.21.15.16:0/4255935864 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 634538652 0 0) 0x7f91bc0ce3e0 con 0x7f91bc0d1000 2023-05-20T01:25:08.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.306+0000 7f91c19c9700 1 -- 172.21.15.16:0/4255935864 >> v1:172.21.15.132:6789/0 conn(0x7f91bc0d1000 legacy=0x7f91bc0d13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:08.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.306+0000 7f91c19c9700 1 -- 172.21.15.16:0/4255935864 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91bc0ce070 con 0x7f91bc0cb6e0 2023-05-20T01:25:08.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.306+0000 7f91c09c7700 1 --2- 172.21.15.16:0/4255935864 >> v2:172.21.15.132:3300/0 conn(0x7f91bc06b620 0x7f91bc06ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-05-20T01:25:08.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.306+0000 7f91c19c9700 1 --2- 172.21.15.16:0/4255935864 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f91bc0cb6e0 0x7f91bc0cbb60 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f91b4009d90 tx=0x7f91b400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8cb04c6e6214ff4c server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:08.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.306+0000 7f91b3fff700 1 -- 172.21.15.16:0/4255935864 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91b400e070 con 0x7f91bc0cb6e0 2023-05-20T01:25:08.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.307+0000 7f91b3fff700 1 -- 172.21.15.16:0/4255935864 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91b400bb20 con 0x7f91bc0cb6e0 2023-05-20T01:25:08.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.307+0000 7f91b3fff700 1 -- 172.21.15.16:0/4255935864 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91b4012430 con 0x7f91bc0cb6e0 2023-05-20T01:25:08.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.307+0000 7f91c342c700 1 -- 172.21.15.16:0/4255935864 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f91bc0cb6e0 msgr2=0x7f91bc0cbb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:08.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.307+0000 7f91c342c700 1 --2- 172.21.15.16:0/4255935864 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f91bc0cb6e0 0x7f91bc0cbb60 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f91b4009d90 tx=0x7f91b400b330 comp rx=0 tx=0).stop 2023-05-20T01:25:08.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.307+0000 7f91c342c700 1 -- 172.21.15.16:0/4255935864 shutdown_connections 2023-05-20T01:25:08.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.307+0000 7f91c342c700 1 --2- 172.21.15.16:0/4255935864 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f91bc0cb6e0 0x7f91bc0cbb60 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:08.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.307+0000 7f91c342c700 1 --2- 172.21.15.16:0/4255935864 >> v2:172.21.15.132:3300/0 conn(0x7f91bc06b620 0x7f91bc06ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:08.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.307+0000 7f91c342c700 1 -- 172.21.15.16:0/4255935864 >> 172.21.15.16:0/4255935864 conn(0x7f91bc06ae30 msgr2=0x7f91bc09a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:08.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.307+0000 7f91c342c700 1 -- 172.21.15.16:0/4255935864 shutdown_connections 2023-05-20T01:25:08.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.307+0000 7f91c342c700 1 -- 172.21.15.16:0/4255935864 wait complete. 2023-05-20T01:25:08.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.308+0000 7f91c342c700 1 Processor -- start 2023-05-20T01:25:08.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.308+0000 7f91c342c700 1 -- start start 2023-05-20T01:25:08.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.309+0000 7f91c342c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91bc06b620 0x7f91bc148af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:08.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.309+0000 7f91c342c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f91bc0cb6e0 0x7f91bc14bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:08.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.309+0000 7f91c342c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f91bc0d1000 0x7f91bc0c7270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:08.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.309+0000 7f91c342c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91bc0c5fa0 con 0x7f91bc06b620 2023-05-20T01:25:08.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.309+0000 7f91c342c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91bc149030 con 0x7f91bc0cb6e0 2023-05-20T01:25:08.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.309+0000 7f91c342c700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91bc14c270 con 0x7f91bc0d1000 2023-05-20T01:25:08.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.309+0000 7f91c11c8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91bc06b620 0x7f91bc148af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:08.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.309+0000 7f91c11c8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91bc06b620 0x7f91bc148af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36050/0 (socket says 172.21.15.16:36050) 2023-05-20T01:25:08.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.309+0000 7f91c11c8700 1 -- 172.21.15.16:0/3204692322 learned_addr learned my addr 172.21.15.16:0/3204692322 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:08.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.309+0000 7f91c09c7700 1 --2- 172.21.15.16:0/3204692322 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f91bc0cb6e0 0x7f91bc14bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:08.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.309+0000 7f91c19c9700 1 --2- 172.21.15.16:0/3204692322 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f91bc0d1000 0x7f91bc0c7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:08.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.310+0000 7f91c11c8700 1 -- 172.21.15.16:0/3204692322 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f91bc0d1000 msgr2=0x7f91bc0c7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:08.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.310+0000 7f91c11c8700 1 --2- 172.21.15.16:0/3204692322 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f91bc0d1000 0x7f91bc0c7270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:08.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.310+0000 7f91c11c8700 1 -- 172.21.15.16:0/3204692322 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f91bc0cb6e0 msgr2=0x7f91bc14bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:08.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.310+0000 7f91c11c8700 1 --2- 172.21.15.16:0/3204692322 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f91bc0cb6e0 0x7f91bc14bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:08.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.310+0000 7f91c11c8700 1 -- 172.21.15.16:0/3204692322 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91ac0094a0 con 0x7f91bc06b620 2023-05-20T01:25:08.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.310+0000 7f91c11c8700 1 --2- 172.21.15.16:0/3204692322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91bc06b620 0x7f91bc148af0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f91b80011e0 tx=0x7f91b800cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:08.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.310+0000 7f91b1ffb700 1 -- 172.21.15.16:0/3204692322 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91b8004c80 con 0x7f91bc06b620 2023-05-20T01:25:08.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.310+0000 7f91b1ffb700 1 -- 172.21.15.16:0/3204692322 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91b8004e00 con 0x7f91bc06b620 2023-05-20T01:25:08.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.310+0000 7f91c342c700 1 -- 172.21.15.16:0/3204692322 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91b4009a70 con 0x7f91bc06b620 2023-05-20T01:25:08.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.310+0000 7f91b1ffb700 1 -- 172.21.15.16:0/3204692322 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91b8007310 con 0x7f91bc06b620 2023-05-20T01:25:08.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.310+0000 7f91c342c700 1 -- 172.21.15.16:0/3204692322 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91bc0ce870 con 0x7f91bc06b620 2023-05-20T01:25:08.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.311+0000 7f91c342c700 1 -- 172.21.15.16:0/3204692322 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91bc0cd390 con 0x7f91bc06b620 2023-05-20T01:25:08.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.311+0000 7f91b1ffb700 1 -- 172.21.15.16:0/3204692322 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f91b8007470 con 0x7f91bc06b620 2023-05-20T01:25:08.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.312+0000 7f91b1ffb700 1 --2- 172.21.15.16:0/3204692322 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f91a006ace0 0x7f91a006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:08.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.312+0000 7f91b1ffb700 1 -- 172.21.15.16:0/3204692322 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f91b80890a0 con 0x7f91bc06b620 2023-05-20T01:25:08.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.312+0000 7f91c09c7700 1 --2- 172.21.15.16:0/3204692322 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f91a006ace0 0x7f91a006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:08.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.312+0000 7f91c09c7700 1 --2- 172.21.15.16:0/3204692322 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f91a006ace0 0x7f91a006d190 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f91bc0af2a0 tx=0x7f91ac008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:08.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.316+0000 7f91b1ffb700 1 -- 172.21.15.16:0/3204692322 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f91b80542d0 con 0x7f91bc06b620 2023-05-20T01:25:08.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.518+0000 7f91c342c700 1 -- 172.21.15.16:0/3204692322 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f91bc0747b0 con 0x7f91bc06b620 2023-05-20T01:25:08.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.519+0000 7f91b1ffb700 1 -- 172.21.15.16:0/3204692322 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+14751 (secure 0 0 0) 0x7f91b80578f0 con 0x7f91bc06b620 2023-05-20T01:25:08.523 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:25:08.523 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":84,"fsid":"c2245cc5-ecab-4087-a59b-bc81db901676","created":"2023-05-20T01:16:09.549230+0000","modified":"2023-05-20T01:25:07.701476+0000","last_up_change":"2023-05-20T01:16:19.788624+0000","last_in_change":"2023-05-20T01:16:12.092160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-05-20T01:16:20.934366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-05-20T01:16:21.372294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"317a83cd-19c8-4958-a301-6087ef342289","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6825","nonce":3923634179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6827","nonce":3923634179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6831","nonce":3923634179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6829","nonce":3923634179}]},"public_addr":"172.21.15.16:6825/3923634179","cluster_addr":"172.21.15.16:6827/3923634179","heartbeat_back_addr":"172.21.15.16:6831/3923634179","heartbeat_front_addr":"172.21.15.16:6829/3923634179","state":["exists","up"]},{"osd":1,"uuid":"5581046f-8219-4089-8d7d-b7846399505e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6809","nonce":3860264220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6811","nonce":3860264220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6815","nonce":3860264220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6813","nonce":3860264220}]},"public_addr":"172.21.15.16:6809/3860264220","cluster_addr":"172.21.15.16:6811/3860264220","heartbeat_back_addr":"172.21.15.16:6815/3860264220","heartbeat_front_addr":"172.21.15.16:6813/3860264220","state":["exists","up"]},{"osd":2,"uuid":"072e9e66-4133-478e-8d4a-64d46c2963f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6817","nonce":293187030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6819","nonce":293187030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6823","nonce":293187030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6821","nonce":293187030}]},"public_addr":"172.21.15.16:6817/293187030","cluster_addr":"172.21.15.16:6819/293187030","heartbeat_back_addr":"172.21.15.16:6823/293187030","heartbeat_front_addr":"172.21.15.16:6821/293187030","state":["exists","up"]},{"osd":3,"uuid":"11e1bba6-00f8-48dd-9028-d8d716b7f145","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6801","nonce":3669781890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6803","nonce":3669781890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6807","nonce":3669781890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6805","nonce":3669781890}]},"public_addr":"172.21.15.16:6801/3669781890","cluster_addr":"172.21.15.16:6803/3669781890","heartbeat_back_addr":"172.21.15.16:6807/3669781890","heartbeat_front_addr":"172.21.15.16:6805/3669781890","state":["exists","up"]},{"osd":4,"uuid":"b8909c3a-cf70-46cb-8d0b-009c707cf9d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6819","nonce":506317223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6823","nonce":506317223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6829","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6831","nonce":506317223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6827","nonce":506317223}]},"public_addr":"172.21.15.132:6819/506317223","cluster_addr":"172.21.15.132:6823/506317223","heartbeat_back_addr":"172.21.15.132:6831/506317223","heartbeat_front_addr":"172.21.15.132:6827/506317223","state":["exists","up"]},{"osd":5,"uuid":"b1818304-4583-47bf-a6ec-0bfbb5166233","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6818","nonce":1844689995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6822","nonce":1844689995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6830","nonce":1844689995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6826","nonce":1844689995}]},"public_addr":"172.21.15.132:6818/1844689995","cluster_addr":"172.21.15.132:6822/1844689995","heartbeat_back_addr":"172.21.15.132:6830/1844689995","heartbeat_front_addr":"172.21.15.132:6826/1844689995","state":["exists","up"]},{"osd":6,"uuid":"d37f4691-eed1-4a93-8818-18354f03f4a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6801","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6802","nonce":727581123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6805","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6807","nonce":727581123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6813","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6815","nonce":727581123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6811","nonce":727581123}]},"public_addr":"172.21.15.132:6802/727581123","cluster_addr":"172.21.15.132:6807/727581123","heartbeat_back_addr":"172.21.15.132:6815/727581123","heartbeat_front_addr":"172.21.15.132:6811/727581123","state":["exists","up"]},{"osd":7,"uuid":"4f712f62-5d1d-4816-8042-03209144f8b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6803","nonce":466001149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6806","nonce":466001149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6814","nonce":466001149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6810","nonce":466001149}]},"public_addr":"172.21.15.132:6803/466001149","cluster_addr":"172.21.15.132:6806/466001149","heartbeat_back_addr":"172.21.15.132:6814/466001149","heartbeat_front_addr":"172.21.15.132:6810/466001149","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.338863+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.278680+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305694+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.246835+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.276237+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.304441+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305421+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.267050+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-20T01:25:08.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.521+0000 7f91c342c700 1 -- 172.21.15.16:0/3204692322 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f91a006ace0 msgr2=0x7f91a006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:08.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.521+0000 7f91c342c700 1 --2- 172.21.15.16:0/3204692322 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f91a006ace0 0x7f91a006d190 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f91bc0af2a0 tx=0x7f91ac008040 comp rx=0 tx=0).stop 2023-05-20T01:25:08.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.521+0000 7f91c342c700 1 -- 172.21.15.16:0/3204692322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91bc06b620 msgr2=0x7f91bc148af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:08.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.521+0000 7f91c342c700 1 --2- 172.21.15.16:0/3204692322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91bc06b620 0x7f91bc148af0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f91b80011e0 tx=0x7f91b800cae0 comp rx=0 tx=0).stop 2023-05-20T01:25:08.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.521+0000 7f91c342c700 1 -- 172.21.15.16:0/3204692322 shutdown_connections 2023-05-20T01:25:08.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.521+0000 7f91c342c700 1 --2- 172.21.15.16:0/3204692322 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f91a006ace0 0x7f91a006d190 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:08.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.521+0000 7f91c342c700 1 --2- 172.21.15.16:0/3204692322 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f91bc0d1000 0x7f91bc0c7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:08.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.521+0000 7f91c342c700 1 --2- 172.21.15.16:0/3204692322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91bc06b620 0x7f91bc148af0 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:08.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.521+0000 7f91c342c700 1 --2- 172.21.15.16:0/3204692322 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f91bc0cb6e0 0x7f91bc14bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:08.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.521+0000 7f91c342c700 1 -- 172.21.15.16:0/3204692322 >> 172.21.15.16:0/3204692322 conn(0x7f91bc06ae30 msgr2=0x7f91bc0d13d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:08.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.521+0000 7f91c342c700 1 -- 172.21.15.16:0/3204692322 shutdown_connections 2023-05-20T01:25:08.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.521+0000 7f91c342c700 1 -- 172.21.15.16:0/3204692322 wait complete. 2023-05-20T01:25:08.538 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-05-20T01:16:20.934366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-05-20T01:16:21.372294+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-05-20T01:25:08.538 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-05-20T01:25:08.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.662+0000 7f6b5a5d3700 1 Processor -- start 2023-05-20T01:25:08.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.662+0000 7f6b5a5d3700 1 -- start start 2023-05-20T01:25:08.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.663+0000 7f6b5a5d3700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6b540d0910 0x7f6b5415bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:08.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.663+0000 7f6b5a5d3700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f6b540cdc20 0x7f6b540ce0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:08.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.663+0000 7f6b5a5d3700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b5406a6b0 con 0x7f6b540cf090 2023-05-20T01:25:08.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.663+0000 7f6b5a5d3700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b5406b470 con 0x7f6b540cdc20 2023-05-20T01:25:08.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.663+0000 7f6b5a5d3700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b5406a990 con 0x7f6b540d0910 2023-05-20T01:25:08.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.664+0000 7f6b58b70700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f6b540cdc20 0x7f6b540ce0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:08.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.664+0000 7f6b537fe700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6b540d0910 0x7f6b5415bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:08.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.664+0000 7f6b53fff700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f6b540cf090 0x7f6b540cf460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.16:57300/0 (socket says 172.21.15.16:57300) 2023-05-20T01:25:08.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.664+0000 7f6b53fff700 1 -- 172.21.15.16:0/12077745 learned_addr learned my addr 172.21.15.16:0/12077745 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:25:08.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.664+0000 7f6b58b70700 1 -- 172.21.15.16:0/12077745 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6b540d0910 msgr2=0x7f6b5415bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:08.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.664+0000 7f6b58b70700 1 --2- 172.21.15.16:0/12077745 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6b540d0910 0x7f6b5415bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:08.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.664+0000 7f6b58b70700 1 -- 172.21.15.16:0/12077745 >> v1:172.21.15.132:6789/0 conn(0x7f6b540cf090 legacy=0x7f6b540cf460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:08.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.664+0000 7f6b58b70700 1 -- 172.21.15.16:0/12077745 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b5406ac70 con 0x7f6b540cdc20 2023-05-20T01:25:08.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.664+0000 7f6b537fe700 1 --2- 172.21.15.16:0/12077745 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6b540d0910 0x7f6b5415bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-05-20T01:25:08.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.664+0000 7f6b58b70700 1 --2- 172.21.15.16:0/12077745 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6b540cdc20 0x7f6b540ce0a0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c009d90 tx=0x7f6b4c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f802ca701e69a4e8 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:08.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.664+0000 7f6b52ffd700 1 -- 172.21.15.16:0/12077745 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b4c00e070 con 0x7f6b540cdc20 2023-05-20T01:25:08.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.664+0000 7f6b52ffd700 1 -- 172.21.15.16:0/12077745 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b4c00bb20 con 0x7f6b540cdc20 2023-05-20T01:25:08.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.664+0000 7f6b52ffd700 1 -- 172.21.15.16:0/12077745 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b4c012430 con 0x7f6b540cdc20 2023-05-20T01:25:08.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.665+0000 7f6b5a5d3700 1 -- 172.21.15.16:0/12077745 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6b540cdc20 msgr2=0x7f6b540ce0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:08.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.665+0000 7f6b5a5d3700 1 --2- 172.21.15.16:0/12077745 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6b540cdc20 0x7f6b540ce0a0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c009d90 tx=0x7f6b4c00b330 comp rx=0 tx=0).stop 2023-05-20T01:25:08.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.665+0000 7f6b5a5d3700 1 -- 172.21.15.16:0/12077745 shutdown_connections 2023-05-20T01:25:08.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.665+0000 7f6b5a5d3700 1 --2- 172.21.15.16:0/12077745 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6b540cdc20 0x7f6b540ce0a0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:08.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.665+0000 7f6b5a5d3700 1 --2- 172.21.15.16:0/12077745 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6b540d0910 0x7f6b5415bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:08.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.665+0000 7f6b5a5d3700 1 -- 172.21.15.16:0/12077745 >> 172.21.15.16:0/12077745 conn(0x7f6b540cd760 msgr2=0x7f6b540d0050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:08.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.665+0000 7f6b5a5d3700 1 -- 172.21.15.16:0/12077745 shutdown_connections 2023-05-20T01:25:08.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.665+0000 7f6b5a5d3700 1 -- 172.21.15.16:0/12077745 wait complete. 2023-05-20T01:25:08.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.666+0000 7f6b5a5d3700 1 Processor -- start 2023-05-20T01:25:08.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.667+0000 7f6b5a5d3700 1 -- start start 2023-05-20T01:25:08.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.668+0000 7f6b5a5d3700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6b540cdc20 0x7f6b5415d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:08.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.668+0000 7f6b5a5d3700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6b540cf090 0x7f6b54153180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:08.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.668+0000 7f6b5a5d3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b540d0910 0x7f6b541536c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:08.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.668+0000 7f6b5a5d3700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b540c7120 con 0x7f6b540d0910 2023-05-20T01:25:08.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.668+0000 7f6b5a5d3700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b540b5690 con 0x7f6b540cdc20 2023-05-20T01:25:08.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.668+0000 7f6b5a5d3700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b540b5970 con 0x7f6b540cf090 2023-05-20T01:25:08.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.668+0000 7f6b58b70700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b540d0910 0x7f6b541536c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:08.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.668+0000 7f6b58b70700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b540d0910 0x7f6b541536c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36068/0 (socket says 172.21.15.16:36068) 2023-05-20T01:25:08.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.668+0000 7f6b58b70700 1 -- 172.21.15.16:0/2521108130 learned_addr learned my addr 172.21.15.16:0/2521108130 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:08.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.668+0000 7f6b53fff700 1 --2- 172.21.15.16:0/2521108130 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6b540cdc20 0x7f6b5415d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:08.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.668+0000 7f6b537fe700 1 --2- 172.21.15.16:0/2521108130 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6b540cf090 0x7f6b54153180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:08.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.668+0000 7f6b58b70700 1 -- 172.21.15.16:0/2521108130 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6b540cf090 msgr2=0x7f6b54153180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:08.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.668+0000 7f6b58b70700 1 --2- 172.21.15.16:0/2521108130 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6b540cf090 0x7f6b54153180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:08.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.668+0000 7f6b58b70700 1 -- 172.21.15.16:0/2521108130 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6b540cdc20 msgr2=0x7f6b5415d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:08.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.668+0000 7f6b58b70700 1 --2- 172.21.15.16:0/2521108130 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6b540cdc20 0x7f6b5415d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:08.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.668+0000 7f6b58b70700 1 -- 172.21.15.16:0/2521108130 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b480094a0 con 0x7f6b540d0910 2023-05-20T01:25:08.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.668+0000 7f6b53fff700 1 --2- 172.21.15.16:0/2521108130 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6b540cdc20 0x7f6b5415d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:25:08.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.668+0000 7f6b537fe700 1 --2- 172.21.15.16:0/2521108130 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6b540cf090 0x7f6b54153180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:25:08.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.668+0000 7f6b58b70700 1 --2- 172.21.15.16:0/2521108130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b540d0910 0x7f6b541536c0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c014040 tx=0x7f6b4c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:08.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.668+0000 7f6b50ff9700 1 -- 172.21.15.16:0/2521108130 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b4c003ea0 con 0x7f6b540d0910 2023-05-20T01:25:08.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.668+0000 7f6b50ff9700 1 -- 172.21.15.16:0/2521108130 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b4c020070 con 0x7f6b540d0910 2023-05-20T01:25:08.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.668+0000 7f6b50ff9700 1 -- 172.21.15.16:0/2521108130 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b4c0276c0 con 0x7f6b540d0910 2023-05-20T01:25:08.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.668+0000 7f6b5a5d3700 1 -- 172.21.15.16:0/2521108130 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b4c009a70 con 0x7f6b540d0910 2023-05-20T01:25:08.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.669+0000 7f6b5a5d3700 1 -- 172.21.15.16:0/2521108130 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b540cb7e0 con 0x7f6b540d0910 2023-05-20T01:25:08.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.670+0000 7f6b5a5d3700 1 -- 172.21.15.16:0/2521108130 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b5406af50 con 0x7f6b540d0910 2023-05-20T01:25:08.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.670+0000 7f6b50ff9700 1 -- 172.21.15.16:0/2521108130 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6b4c012430 con 0x7f6b540d0910 2023-05-20T01:25:08.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.670+0000 7f6b50ff9700 1 --2- 172.21.15.16:0/2521108130 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f6b4006ace0 0x7f6b4006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:08.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.670+0000 7f6b50ff9700 1 -- 172.21.15.16:0/2521108130 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f6b4c022070 con 0x7f6b540d0910 2023-05-20T01:25:08.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.670+0000 7f6b53fff700 1 --2- 172.21.15.16:0/2521108130 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f6b4006ace0 0x7f6b4006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:08.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.671+0000 7f6b53fff700 1 --2- 172.21.15.16:0/2521108130 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f6b4006ace0 0x7f6b4006d190 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f6b44007fd0 tx=0x7f6b44009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:08.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.675+0000 7f6b50ff9700 1 -- 172.21.15.16:0/2521108130 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f6b4c0562f0 con 0x7f6b540d0910 2023-05-20T01:25:08.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.872+0000 7f6b5a5d3700 1 -- 172.21.15.16:0/2521108130 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f6b5415ae70 con 0x7f6b540d0910 2023-05-20T01:25:08.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.872+0000 7f6b50ff9700 1 -- 172.21.15.16:0/2521108130 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v84) v1 ==== 92+0+10 (secure 0 0 0) 0x7f6b4c02c020 con 0x7f6b540d0910 2023-05-20T01:25:08.877 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 8 2023-05-20T01:25:08.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.874+0000 7f6b5a5d3700 1 -- 172.21.15.16:0/2521108130 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f6b4006ace0 msgr2=0x7f6b4006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:08.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.874+0000 7f6b5a5d3700 1 --2- 172.21.15.16:0/2521108130 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f6b4006ace0 0x7f6b4006d190 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f6b44007fd0 tx=0x7f6b44009040 comp rx=0 tx=0).stop 2023-05-20T01:25:08.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.874+0000 7f6b5a5d3700 1 -- 172.21.15.16:0/2521108130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b540d0910 msgr2=0x7f6b541536c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:08.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.874+0000 7f6b5a5d3700 1 --2- 172.21.15.16:0/2521108130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b540d0910 0x7f6b541536c0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c014040 tx=0x7f6b4c004ad0 comp rx=0 tx=0).stop 2023-05-20T01:25:08.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.874+0000 7f6b5a5d3700 1 -- 172.21.15.16:0/2521108130 shutdown_connections 2023-05-20T01:25:08.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.874+0000 7f6b5a5d3700 1 --2- 172.21.15.16:0/2521108130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b540d0910 0x7f6b541536c0 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:08.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.874+0000 7f6b5a5d3700 1 --2- 172.21.15.16:0/2521108130 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6b540cdc20 0x7f6b5415d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:08.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.874+0000 7f6b5a5d3700 1 --2- 172.21.15.16:0/2521108130 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f6b4006ace0 0x7f6b4006d190 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:08.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.874+0000 7f6b5a5d3700 1 --2- 172.21.15.16:0/2521108130 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6b540cf090 0x7f6b54153180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:08.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.874+0000 7f6b5a5d3700 1 -- 172.21.15.16:0/2521108130 >> 172.21.15.16:0/2521108130 conn(0x7f6b540cd760 msgr2=0x7f6b5415cab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:08.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.874+0000 7f6b5a5d3700 1 -- 172.21.15.16:0/2521108130 shutdown_connections 2023-05-20T01:25:08.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:08.875+0000 7f6b5a5d3700 1 -- 172.21.15.16:0/2521108130 wait complete. 2023-05-20T01:25:08.891 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-05-20T01:25:09.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.015+0000 7f8d8afac700 1 Processor -- start 2023-05-20T01:25:09.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.015+0000 7f8d8afac700 1 -- start start 2023-05-20T01:25:09.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.015+0000 7f8d8afac700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8d840cd1c0 0x7f8d840710b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:09.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.015+0000 7f8d8afac700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8d840cd630 0x7f8d840715f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:09.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.015+0000 7f8d8afac700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d840ccd30 con 0x7f8d84159b50 2023-05-20T01:25:09.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.015+0000 7f8d8afac700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d840ce990 con 0x7f8d840cd630 2023-05-20T01:25:09.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.015+0000 7f8d8afac700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d840cc6e0 con 0x7f8d840cd1c0 2023-05-20T01:25:09.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.016+0000 7f8d89549700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8d840cd630 0x7f8d840715f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:09.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.016+0000 7f8d88d48700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f8d84159b50 0x7f8d8406e0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:47600/0 (socket says 172.21.15.16:47600) 2023-05-20T01:25:09.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.016+0000 7f8d88d48700 1 -- 172.21.15.16:0/167053562 learned_addr learned my addr 172.21.15.16:0/167053562 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:25:09.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.016+0000 7f8d83fff700 1 --2- 172.21.15.16:0/167053562 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8d840cd1c0 0x7f8d840710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:09.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.016+0000 7f8d89549700 1 -- 172.21.15.16:0/167053562 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8d840cd1c0 msgr2=0x7f8d840710b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:09.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.016+0000 7f8d89549700 1 --2- 172.21.15.16:0/167053562 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8d840cd1c0 0x7f8d840710b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:09.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.016+0000 7f8d89549700 1 -- 172.21.15.16:0/167053562 >> v1:172.21.15.16:6789/0 conn(0x7f8d84159b50 legacy=0x7f8d8406e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:09.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.016+0000 7f8d837fe700 1 -- 172.21.15.16:0/167053562 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3352934891 0 0) 0x7f8d840ccd30 con 0x7f8d84159b50 2023-05-20T01:25:09.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.016+0000 7f8d89549700 1 -- 172.21.15.16:0/167053562 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d840cc9c0 con 0x7f8d840cd630 2023-05-20T01:25:09.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.016+0000 7f8d89549700 1 --2- 172.21.15.16:0/167053562 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d840cd630 0x7f8d840715f0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c009d90 tx=0x7f8d7c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7bc5be87f7d545d server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:09.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.016+0000 7f8d837fe700 1 -- 172.21.15.16:0/167053562 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d7c00e070 con 0x7f8d840cd630 2023-05-20T01:25:09.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.016+0000 7f8d837fe700 1 -- 172.21.15.16:0/167053562 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d7c00bbd0 con 0x7f8d840cd630 2023-05-20T01:25:09.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.016+0000 7f8d837fe700 1 -- 172.21.15.16:0/167053562 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d7c012430 con 0x7f8d840cd630 2023-05-20T01:25:09.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.017+0000 7f8d8afac700 1 -- 172.21.15.16:0/167053562 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d840cd630 msgr2=0x7f8d840715f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:09.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.017+0000 7f8d8afac700 1 --2- 172.21.15.16:0/167053562 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d840cd630 0x7f8d840715f0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c009d90 tx=0x7f8d7c00b330 comp rx=0 tx=0).stop 2023-05-20T01:25:09.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.017+0000 7f8d8afac700 1 -- 172.21.15.16:0/167053562 shutdown_connections 2023-05-20T01:25:09.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.017+0000 7f8d8afac700 1 --2- 172.21.15.16:0/167053562 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d840cd630 0x7f8d840715f0 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:09.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.017+0000 7f8d8afac700 1 --2- 172.21.15.16:0/167053562 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8d840cd1c0 0x7f8d840710b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:09.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.017+0000 7f8d8afac700 1 -- 172.21.15.16:0/167053562 >> 172.21.15.16:0/167053562 conn(0x7f8d840cffd0 msgr2=0x7f8d840d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:09.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.017+0000 7f8d8afac700 1 -- 172.21.15.16:0/167053562 shutdown_connections 2023-05-20T01:25:09.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.017+0000 7f8d8afac700 1 -- 172.21.15.16:0/167053562 wait complete. 2023-05-20T01:25:09.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.018+0000 7f8d8afac700 1 Processor -- start 2023-05-20T01:25:09.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.018+0000 7f8d8afac700 1 -- start start 2023-05-20T01:25:09.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.019+0000 7f8d8afac700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8d840cd1c0 0x7f8d84158a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:09.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.019+0000 7f8d8afac700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d840cd630 0x7f8d840c6400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:09.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.019+0000 7f8d8afac700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8d84159b50 0x7f8d841e5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:09.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.019+0000 7f8d8afac700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d840c4940 con 0x7f8d840cd630 2023-05-20T01:25:09.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.019+0000 7f8d8afac700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d840c6940 con 0x7f8d840cd1c0 2023-05-20T01:25:09.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.019+0000 7f8d8afac700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d840c3ae0 con 0x7f8d84159b50 2023-05-20T01:25:09.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.019+0000 7f8d83fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d840cd630 0x7f8d840c6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:09.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.019+0000 7f8d83fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d840cd630 0x7f8d840c6400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36078/0 (socket says 172.21.15.16:36078) 2023-05-20T01:25:09.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.019+0000 7f8d83fff700 1 -- 172.21.15.16:0/11179870 learned_addr learned my addr 172.21.15.16:0/11179870 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:09.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.019+0000 7f8d83fff700 1 -- 172.21.15.16:0/11179870 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8d84159b50 msgr2=0x7f8d841e5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:09.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.019+0000 7f8d89549700 1 --2- 172.21.15.16:0/11179870 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8d84159b50 0x7f8d841e5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:09.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.019+0000 7f8d88d48700 1 --2- 172.21.15.16:0/11179870 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8d840cd1c0 0x7f8d84158a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:09.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.019+0000 7f8d83fff700 1 --2- 172.21.15.16:0/11179870 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8d84159b50 0x7f8d841e5470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:09.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.019+0000 7f8d83fff700 1 -- 172.21.15.16:0/11179870 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8d840cd1c0 msgr2=0x7f8d84158a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:09.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.019+0000 7f8d83fff700 1 --2- 172.21.15.16:0/11179870 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8d840cd1c0 0x7f8d84158a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:09.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.019+0000 7f8d83fff700 1 -- 172.21.15.16:0/11179870 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d7c009a70 con 0x7f8d840cd630 2023-05-20T01:25:09.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.020+0000 7f8d83fff700 1 --2- 172.21.15.16:0/11179870 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d840cd630 0x7f8d840c6400 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f8d7000d800 tx=0x7f8d7000dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:09.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.020+0000 7f8d817fa700 1 -- 172.21.15.16:0/11179870 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d700125f0 con 0x7f8d840cd630 2023-05-20T01:25:09.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.020+0000 7f8d817fa700 1 -- 172.21.15.16:0/11179870 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d70012750 con 0x7f8d840cd630 2023-05-20T01:25:09.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.020+0000 7f8d817fa700 1 -- 172.21.15.16:0/11179870 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d70012ae0 con 0x7f8d840cd630 2023-05-20T01:25:09.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.020+0000 7f8d8afac700 1 -- 172.21.15.16:0/11179870 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d840cec70 con 0x7f8d840cd630 2023-05-20T01:25:09.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.020+0000 7f8d8afac700 1 -- 172.21.15.16:0/11179870 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d84071e80 con 0x7f8d840cd630 2023-05-20T01:25:09.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.021+0000 7f8d8afac700 1 -- 172.21.15.16:0/11179870 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d8415a500 con 0x7f8d840cd630 2023-05-20T01:25:09.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.021+0000 7f8d817fa700 1 -- 172.21.15.16:0/11179870 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8d70012c40 con 0x7f8d840cd630 2023-05-20T01:25:09.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.022+0000 7f8d817fa700 1 --2- 172.21.15.16:0/11179870 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f8d6806ace0 0x7f8d6806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:09.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.022+0000 7f8d817fa700 1 -- 172.21.15.16:0/11179870 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f8d70089610 con 0x7f8d840cd630 2023-05-20T01:25:09.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.022+0000 7f8d88d48700 1 --2- 172.21.15.16:0/11179870 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f8d6806ace0 0x7f8d6806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:09.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.022+0000 7f8d88d48700 1 --2- 172.21.15.16:0/11179870 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f8d6806ace0 0x7f8d6806d190 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f8d841f25b0 tx=0x7f8d78009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:09.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.026+0000 7f8d817fa700 1 -- 172.21.15.16:0/11179870 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f8d700548d0 con 0x7f8d840cd630 2023-05-20T01:25:09.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.225+0000 7f8d8afac700 1 -- 172.21.15.16:0/11179870 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8d840747b0 con 0x7f8d840cd630 2023-05-20T01:25:09.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.226+0000 7f8d817fa700 1 -- 172.21.15.16:0/11179870 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8d70057ef0 con 0x7f8d840cd630 2023-05-20T01:25:09.230 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-05-20T01:25:09.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.228+0000 7f8d8afac700 1 -- 172.21.15.16:0/11179870 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f8d6806ace0 msgr2=0x7f8d6806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:09.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.228+0000 7f8d8afac700 1 --2- 172.21.15.16:0/11179870 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f8d6806ace0 0x7f8d6806d190 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f8d841f25b0 tx=0x7f8d78009040 comp rx=0 tx=0).stop 2023-05-20T01:25:09.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.228+0000 7f8d8afac700 1 -- 172.21.15.16:0/11179870 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d840cd630 msgr2=0x7f8d840c6400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:09.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.228+0000 7f8d8afac700 1 --2- 172.21.15.16:0/11179870 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d840cd630 0x7f8d840c6400 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f8d7000d800 tx=0x7f8d7000dbc0 comp rx=0 tx=0).stop 2023-05-20T01:25:09.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.228+0000 7f8d8afac700 1 -- 172.21.15.16:0/11179870 shutdown_connections 2023-05-20T01:25:09.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.228+0000 7f8d8afac700 1 --2- 172.21.15.16:0/11179870 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f8d6806ace0 0x7f8d6806d190 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:09.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.228+0000 7f8d8afac700 1 --2- 172.21.15.16:0/11179870 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8d84159b50 0x7f8d841e5470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:09.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.228+0000 7f8d8afac700 1 --2- 172.21.15.16:0/11179870 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8d840cd1c0 0x7f8d84158a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:09.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.228+0000 7f8d8afac700 1 --2- 172.21.15.16:0/11179870 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d840cd630 0x7f8d840c6400 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:09.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.228+0000 7f8d8afac700 1 -- 172.21.15.16:0/11179870 >> 172.21.15.16:0/11179870 conn(0x7f8d840cffd0 msgr2=0x7f8d840b1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:09.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.228+0000 7f8d8afac700 1 -- 172.21.15.16:0/11179870 shutdown_connections 2023-05-20T01:25:09.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.228+0000 7f8d8afac700 1 -- 172.21.15.16:0/11179870 wait complete. 2023-05-20T01:25:09.245 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-05-20T01:25:09.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.366+0000 7f6c4cd1e700 1 Processor -- start 2023-05-20T01:25:09.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.366+0000 7f6c4cd1e700 1 -- start start 2023-05-20T01:25:09.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.367+0000 7f6c4cd1e700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6c480d16c0 0x7f6c48144330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:09.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.367+0000 7f6c4cd1e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6c480cec20 0x7f6c480cf090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:09.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.367+0000 7f6c4cd1e700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c48144870 con 0x7f6c480ae2c0 2023-05-20T01:25:09.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.367+0000 7f6c4cd1e700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c480cf5d0 con 0x7f6c480d16c0 2023-05-20T01:25:09.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.367+0000 7f6c4cd1e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c480b2000 con 0x7f6c480cec20 2023-05-20T01:25:09.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.367+0000 7f6c4659c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6c480d16c0 0x7f6c48144330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:09.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.367+0000 7f6c46d9d700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f6c480ae2c0 0x7f6c480ae740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:47612/0 (socket says 172.21.15.16:47612) 2023-05-20T01:25:09.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.367+0000 7f6c46d9d700 1 -- 172.21.15.16:0/59511099 learned_addr learned my addr 172.21.15.16:0/59511099 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:25:09.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.368+0000 7f6c45d9b700 1 --2- 172.21.15.16:0/59511099 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6c480cec20 0x7f6c480cf090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:09.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.368+0000 7f6c4659c700 1 -- 172.21.15.16:0/59511099 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6c480cec20 msgr2=0x7f6c480cf090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:09.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.368+0000 7f6c4659c700 1 --2- 172.21.15.16:0/59511099 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6c480cec20 0x7f6c480cf090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:09.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.368+0000 7f6c4659c700 1 -- 172.21.15.16:0/59511099 >> v1:172.21.15.16:6789/0 conn(0x7f6c480ae2c0 legacy=0x7f6c480ae740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:09.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.368+0000 7f6c4559a700 1 -- 172.21.15.16:0/59511099 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1757406973 0 0) 0x7f6c48144870 con 0x7f6c480ae2c0 2023-05-20T01:25:09.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.368+0000 7f6c4659c700 1 -- 172.21.15.16:0/59511099 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c480b22e0 con 0x7f6c480d16c0 2023-05-20T01:25:09.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.368+0000 7f6c45d9b700 1 --2- 172.21.15.16:0/59511099 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6c480cec20 0x7f6c480cf090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:25:09.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.368+0000 7f6c4659c700 1 --2- 172.21.15.16:0/59511099 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c480d16c0 0x7f6c48144330 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c009840 tx=0x7f6c3c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8027f12d7c2f2f7 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:09.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.368+0000 7f6c4559a700 1 -- 172.21.15.16:0/59511099 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c3c011830 con 0x7f6c480d16c0 2023-05-20T01:25:09.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.368+0000 7f6c4559a700 1 -- 172.21.15.16:0/59511099 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c3c011990 con 0x7f6c480d16c0 2023-05-20T01:25:09.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.368+0000 7f6c4559a700 1 -- 172.21.15.16:0/59511099 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c3c011d20 con 0x7f6c480d16c0 2023-05-20T01:25:09.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.369+0000 7f6c4cd1e700 1 -- 172.21.15.16:0/59511099 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c480d16c0 msgr2=0x7f6c48144330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:09.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.369+0000 7f6c4cd1e700 1 --2- 172.21.15.16:0/59511099 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c480d16c0 0x7f6c48144330 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c009840 tx=0x7f6c3c009b50 comp rx=0 tx=0).stop 2023-05-20T01:25:09.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.369+0000 7f6c4cd1e700 1 -- 172.21.15.16:0/59511099 shutdown_connections 2023-05-20T01:25:09.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.369+0000 7f6c4cd1e700 1 --2- 172.21.15.16:0/59511099 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c480d16c0 0x7f6c48144330 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:09.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.369+0000 7f6c4cd1e700 1 --2- 172.21.15.16:0/59511099 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6c480cec20 0x7f6c480cf090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:09.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.369+0000 7f6c4cd1e700 1 -- 172.21.15.16:0/59511099 >> 172.21.15.16:0/59511099 conn(0x7f6c4806ae30 msgr2=0x7f6c4809a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:09.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.369+0000 7f6c4cd1e700 1 -- 172.21.15.16:0/59511099 shutdown_connections 2023-05-20T01:25:09.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.369+0000 7f6c4cd1e700 1 -- 172.21.15.16:0/59511099 wait complete. 2023-05-20T01:25:09.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.370+0000 7f6c4cd1e700 1 Processor -- start 2023-05-20T01:25:09.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.370+0000 7f6c4cd1e700 1 -- start start 2023-05-20T01:25:09.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.371+0000 7f6c4cd1e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6c480ae2c0 0x7f6c481f2350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:09.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.371+0000 7f6c4cd1e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6c480cec20 0x7f6c481e7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:09.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.371+0000 7f6c4cd1e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c480d16c0 0x7f6c480cdb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:09.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.371+0000 7f6c4cd1e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c481f3350 con 0x7f6c480d16c0 2023-05-20T01:25:09.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.371+0000 7f6c4cd1e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c481f2890 con 0x7f6c480ae2c0 2023-05-20T01:25:09.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.371+0000 7f6c4cd1e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c481e8100 con 0x7f6c480cec20 2023-05-20T01:25:09.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.371+0000 7f6c46d9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c480d16c0 0x7f6c480cdb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:09.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.371+0000 7f6c46d9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c480d16c0 0x7f6c480cdb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36092/0 (socket says 172.21.15.16:36092) 2023-05-20T01:25:09.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.371+0000 7f6c46d9d700 1 -- 172.21.15.16:0/1289442798 learned_addr learned my addr 172.21.15.16:0/1289442798 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:09.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.371+0000 7f6c45d9b700 1 --2- 172.21.15.16:0/1289442798 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6c480cec20 0x7f6c481e7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:09.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.371+0000 7f6c4659c700 1 --2- 172.21.15.16:0/1289442798 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6c480ae2c0 0x7f6c481f2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:09.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.371+0000 7f6c46d9d700 1 -- 172.21.15.16:0/1289442798 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6c480cec20 msgr2=0x7f6c481e7bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:09.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.371+0000 7f6c46d9d700 1 --2- 172.21.15.16:0/1289442798 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6c480cec20 0x7f6c481e7bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:09.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.371+0000 7f6c46d9d700 1 -- 172.21.15.16:0/1289442798 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6c480ae2c0 msgr2=0x7f6c481f2350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:09.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.371+0000 7f6c46d9d700 1 --2- 172.21.15.16:0/1289442798 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6c480ae2c0 0x7f6c481f2350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:09.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.371+0000 7f6c46d9d700 1 -- 172.21.15.16:0/1289442798 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c3c0094a0 con 0x7f6c480d16c0 2023-05-20T01:25:09.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.372+0000 7f6c46d9d700 1 --2- 172.21.15.16:0/1289442798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c480d16c0 0x7f6c480cdb00 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f6c4000cbd0 tx=0x7f6c4000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:09.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.372+0000 7f6c36ffd700 1 -- 172.21.15.16:0/1289442798 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c4000ab70 con 0x7f6c480d16c0 2023-05-20T01:25:09.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.372+0000 7f6c36ffd700 1 -- 172.21.15.16:0/1289442798 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c4000acf0 con 0x7f6c480d16c0 2023-05-20T01:25:09.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.372+0000 7f6c4cd1e700 1 -- 172.21.15.16:0/1289442798 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c480adda0 con 0x7f6c480d16c0 2023-05-20T01:25:09.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.372+0000 7f6c36ffd700 1 -- 172.21.15.16:0/1289442798 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c40007310 con 0x7f6c480d16c0 2023-05-20T01:25:09.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.372+0000 7f6c4cd1e700 1 -- 172.21.15.16:0/1289442798 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c480b25c0 con 0x7f6c480d16c0 2023-05-20T01:25:09.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.373+0000 7f6c4cd1e700 1 -- 172.21.15.16:0/1289442798 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c480b28a0 con 0x7f6c480d16c0 2023-05-20T01:25:09.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.373+0000 7f6c36ffd700 1 -- 172.21.15.16:0/1289442798 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6c40007470 con 0x7f6c480d16c0 2023-05-20T01:25:09.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.374+0000 7f6c36ffd700 1 --2- 172.21.15.16:0/1289442798 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f6c2806ace0 0x7f6c2806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:09.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.374+0000 7f6c36ffd700 1 -- 172.21.15.16:0/1289442798 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f6c40089170 con 0x7f6c480d16c0 2023-05-20T01:25:09.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.374+0000 7f6c4659c700 1 --2- 172.21.15.16:0/1289442798 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f6c2806ace0 0x7f6c2806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:09.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.374+0000 7f6c4659c700 1 --2- 172.21.15.16:0/1289442798 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f6c2806ace0 0x7f6c2806d190 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c000c00 tx=0x7f6c3c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:09.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.378+0000 7f6c36ffd700 1 -- 172.21.15.16:0/1289442798 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f6c40054390 con 0x7f6c480d16c0 2023-05-20T01:25:09.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.579+0000 7f6c4cd1e700 1 -- 172.21.15.16:0/1289442798 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f6c480cd440 con 0x7f6c480d16c0 2023-05-20T01:25:09.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.580+0000 7f6c36ffd700 1 -- 172.21.15.16:0/1289442798 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v84) v1 ==== 84+0+126 (secure 0 0 0) 0x7f6c480cd440 con 0x7f6c480d16c0 2023-05-20T01:25:09.584 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:25:09.585 INFO:teuthology.orchestra.run.smithi016.stdout:[ 2023-05-20T01:25:09.585 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-05-20T01:25:09.585 INFO:teuthology.orchestra.run.smithi016.stdout: "poolnum": 1, 2023-05-20T01:25:09.585 INFO:teuthology.orchestra.run.smithi016.stdout: "poolname": "rbd" 2023-05-20T01:25:09.585 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-05-20T01:25:09.585 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-05-20T01:25:09.586 INFO:teuthology.orchestra.run.smithi016.stdout: "poolnum": 2, 2023-05-20T01:25:09.586 INFO:teuthology.orchestra.run.smithi016.stdout: "poolname": ".mgr" 2023-05-20T01:25:09.586 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:25:09.586 INFO:teuthology.orchestra.run.smithi016.stdout:] 2023-05-20T01:25:09.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.582+0000 7f6c4cd1e700 1 -- 172.21.15.16:0/1289442798 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f6c2806ace0 msgr2=0x7f6c2806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:09.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.582+0000 7f6c4cd1e700 1 --2- 172.21.15.16:0/1289442798 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f6c2806ace0 0x7f6c2806d190 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c000c00 tx=0x7f6c3c008030 comp rx=0 tx=0).stop 2023-05-20T01:25:09.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.582+0000 7f6c4cd1e700 1 -- 172.21.15.16:0/1289442798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c480d16c0 msgr2=0x7f6c480cdb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:09.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.582+0000 7f6c4cd1e700 1 --2- 172.21.15.16:0/1289442798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c480d16c0 0x7f6c480cdb00 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f6c4000cbd0 tx=0x7f6c4000cf90 comp rx=0 tx=0).stop 2023-05-20T01:25:09.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.582+0000 7f6c4cd1e700 1 -- 172.21.15.16:0/1289442798 shutdown_connections 2023-05-20T01:25:09.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.582+0000 7f6c4cd1e700 1 --2- 172.21.15.16:0/1289442798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c480d16c0 0x7f6c480cdb00 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:09.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.582+0000 7f6c4cd1e700 1 --2- 172.21.15.16:0/1289442798 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6c480ae2c0 0x7f6c481f2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:09.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.582+0000 7f6c4cd1e700 1 --2- 172.21.15.16:0/1289442798 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f6c2806ace0 0x7f6c2806d190 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:09.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.582+0000 7f6c4cd1e700 1 --2- 172.21.15.16:0/1289442798 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6c480cec20 0x7f6c481e7bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:09.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.582+0000 7f6c4cd1e700 1 -- 172.21.15.16:0/1289442798 >> 172.21.15.16:0/1289442798 conn(0x7f6c4806ae30 msgr2=0x7f6c480ae690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:09.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.582+0000 7f6c4cd1e700 1 -- 172.21.15.16:0/1289442798 shutdown_connections 2023-05-20T01:25:09.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.582+0000 7f6c4cd1e700 1 -- 172.21.15.16:0/1289442798 wait complete. 2023-05-20T01:25:09.598 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-05-20T01:25:09.599 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-05-20T01:25:09.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.723+0000 7f7c0a0b7700 1 Processor -- start 2023-05-20T01:25:09.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.724+0000 7f7c0a0b7700 1 -- start start 2023-05-20T01:25:09.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.724+0000 7f7c0a0b7700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7c0415c830 0x7f7c04161070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:09.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.724+0000 7f7c0a0b7700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7c040c6b50 0x7f7c0406a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:09.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.724+0000 7f7c0a0b7700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c040cf750 con 0x7f7c0415cc00 2023-05-20T01:25:09.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.724+0000 7f7c0a0b7700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c040d0730 con 0x7f7c0415c830 2023-05-20T01:25:09.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.724+0000 7f7c0a0b7700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c041615b0 con 0x7f7c040c6b50 2023-05-20T01:25:09.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.725+0000 7f7c037fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7c0415c830 0x7f7c04161070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:09.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.725+0000 7f7c037fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7c0415c830 0x7f7c04161070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36096/0 (socket says 172.21.15.16:36096) 2023-05-20T01:25:09.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.725+0000 7f7c037fe700 1 -- 172.21.15.16:0/1173410189 learned_addr learned my addr 172.21.15.16:0/1173410189 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:09.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.725+0000 7f7c02ffd700 1 --2- 172.21.15.16:0/1173410189 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7c040c6b50 0x7f7c0406a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:09.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.725+0000 7f7c037fe700 1 -- 172.21.15.16:0/1173410189 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7c040c6b50 msgr2=0x7f7c0406a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:09.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.725+0000 7f7c037fe700 1 --2- 172.21.15.16:0/1173410189 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7c040c6b50 0x7f7c0406a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:09.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.725+0000 7f7c037fe700 1 -- 172.21.15.16:0/1173410189 >> v1:172.21.15.132:6789/0 conn(0x7f7c0415cc00 legacy=0x7f7c040712a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:09.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.725+0000 7f7c037fe700 1 -- 172.21.15.16:0/1173410189 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c040cf980 con 0x7f7c0415c830 2023-05-20T01:25:09.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.725+0000 7f7c037fe700 1 --2- 172.21.15.16:0/1173410189 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c0415c830 0x7f7c04161070 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f7bf40098f0 tx=0x7f7bf4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cee3b39bd2be23cd server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:09.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.725+0000 7f7c027fc700 1 -- 172.21.15.16:0/1173410189 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf400b8a0 con 0x7f7c0415c830 2023-05-20T01:25:09.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.725+0000 7f7c027fc700 1 -- 172.21.15.16:0/1173410189 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bf400ba00 con 0x7f7c0415c830 2023-05-20T01:25:09.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.725+0000 7f7c027fc700 1 -- 172.21.15.16:0/1173410189 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf400bd90 con 0x7f7c0415c830 2023-05-20T01:25:09.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.726+0000 7f7c0a0b7700 1 -- 172.21.15.16:0/1173410189 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c0415c830 msgr2=0x7f7c04161070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:09.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.726+0000 7f7c0a0b7700 1 --2- 172.21.15.16:0/1173410189 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c0415c830 0x7f7c04161070 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f7bf40098f0 tx=0x7f7bf4009c00 comp rx=0 tx=0).stop 2023-05-20T01:25:09.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.726+0000 7f7c0a0b7700 1 -- 172.21.15.16:0/1173410189 shutdown_connections 2023-05-20T01:25:09.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.726+0000 7f7c0a0b7700 1 --2- 172.21.15.16:0/1173410189 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c0415c830 0x7f7c04161070 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:09.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.726+0000 7f7c0a0b7700 1 --2- 172.21.15.16:0/1173410189 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7c040c6b50 0x7f7c0406a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:09.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.726+0000 7f7c0a0b7700 1 -- 172.21.15.16:0/1173410189 >> 172.21.15.16:0/1173410189 conn(0x7f7c040b2920 msgr2=0x7f7c040d0020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:09.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.726+0000 7f7c0a0b7700 1 -- 172.21.15.16:0/1173410189 shutdown_connections 2023-05-20T01:25:09.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.726+0000 7f7c0a0b7700 1 -- 172.21.15.16:0/1173410189 wait complete. 2023-05-20T01:25:09.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.727+0000 7f7c0a0b7700 1 Processor -- start 2023-05-20T01:25:09.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.727+0000 7f7c0a0b7700 1 -- start start 2023-05-20T01:25:09.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.728+0000 7f7c0a0b7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c040c6b50 0x7f7c04151bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:09.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.728+0000 7f7c0a0b7700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7c0415c830 0x7f7c041f2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:09.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.728+0000 7f7c0a0b7700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7c0415cc00 0x7f7c040ae800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:09.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.728+0000 7f7c0a0b7700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c040b3b70 con 0x7f7c040c6b50 2023-05-20T01:25:09.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.728+0000 7f7c0a0b7700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c041520f0 con 0x7f7c0415c830 2023-05-20T01:25:09.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.728+0000 7f7c0a0b7700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c040b3d80 con 0x7f7c0415cc00 2023-05-20T01:25:09.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.728+0000 7f7c037fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c040c6b50 0x7f7c04151bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:09.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.728+0000 7f7c037fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c040c6b50 0x7f7c04151bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36100/0 (socket says 172.21.15.16:36100) 2023-05-20T01:25:09.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.728+0000 7f7c037fe700 1 -- 172.21.15.16:0/2321397976 learned_addr learned my addr 172.21.15.16:0/2321397976 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:09.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.728+0000 7f7c02ffd700 1 --2- 172.21.15.16:0/2321397976 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7c0415c830 0x7f7c041f2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:09.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.728+0000 7f7c03fff700 1 --2- 172.21.15.16:0/2321397976 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7c0415cc00 0x7f7c040ae800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:09.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.729+0000 7f7c037fe700 1 -- 172.21.15.16:0/2321397976 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7c0415cc00 msgr2=0x7f7c040ae800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:09.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.729+0000 7f7c037fe700 1 --2- 172.21.15.16:0/2321397976 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7c0415cc00 0x7f7c040ae800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:09.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.729+0000 7f7c037fe700 1 -- 172.21.15.16:0/2321397976 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7c0415c830 msgr2=0x7f7c041f2aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:09.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.729+0000 7f7c037fe700 1 --2- 172.21.15.16:0/2321397976 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7c0415c830 0x7f7c041f2aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:09.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.729+0000 7f7c037fe700 1 -- 172.21.15.16:0/2321397976 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bf4009550 con 0x7f7c040c6b50 2023-05-20T01:25:09.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.729+0000 7f7c037fe700 1 --2- 172.21.15.16:0/2321397976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c040c6b50 0x7f7c04151bb0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f7bf4009c40 tx=0x7f7bf4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:09.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.729+0000 7f7bfbfff700 1 -- 172.21.15.16:0/2321397976 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf4004510 con 0x7f7c040c6b50 2023-05-20T01:25:09.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.729+0000 7f7bfbfff700 1 -- 172.21.15.16:0/2321397976 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bf401e070 con 0x7f7c040c6b50 2023-05-20T01:25:09.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.729+0000 7f7c0a0b7700 1 -- 172.21.15.16:0/2321397976 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c040d0980 con 0x7f7c040c6b50 2023-05-20T01:25:09.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.729+0000 7f7bfbfff700 1 -- 172.21.15.16:0/2321397976 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf40038e0 con 0x7f7c040c6b50 2023-05-20T01:25:09.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.729+0000 7f7c0a0b7700 1 -- 172.21.15.16:0/2321397976 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c040cfc60 con 0x7f7c040c6b50 2023-05-20T01:25:09.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.730+0000 7f7c0a0b7700 1 -- 172.21.15.16:0/2321397976 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c040d0c60 con 0x7f7c040c6b50 2023-05-20T01:25:09.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.730+0000 7f7bfbfff700 1 -- 172.21.15.16:0/2321397976 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7bf4003a40 con 0x7f7c040c6b50 2023-05-20T01:25:09.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.731+0000 7f7bfbfff700 1 --2- 172.21.15.16:0/2321397976 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f7be406ace0 0x7f7be406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:09.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.731+0000 7f7bfbfff700 1 -- 172.21.15.16:0/2321397976 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f7bf4020070 con 0x7f7c040c6b50 2023-05-20T01:25:09.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.731+0000 7f7c02ffd700 1 --2- 172.21.15.16:0/2321397976 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f7be406ace0 0x7f7be406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:09.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.731+0000 7f7c02ffd700 1 --2- 172.21.15.16:0/2321397976 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f7be406ace0 0x7f7be406d190 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f7c04066720 tx=0x7f7bec008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:09.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.735+0000 7f7bfbfff700 1 -- 172.21.15.16:0/2321397976 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f7bf4055910 con 0x7f7c040c6b50 2023-05-20T01:25:09.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:09.935+0000 7f7c0a0b7700 1 -- 172.21.15.16:0/2321397976 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f7c040747b0 con 0x7f7c040c6b50 2023-05-20T01:25:10.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:10.553+0000 7f7bfbfff700 1 -- 172.21.15.16:0/2321397976 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v85) v1 ==== 138+0+0 (secure 0 0 0) 0x7f7bf4058f30 con 0x7f7c040c6b50 2023-05-20T01:25:10.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:10.557+0000 7f7c0a0b7700 1 -- 172.21.15.16:0/2321397976 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f7be406ace0 msgr2=0x7f7be406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:10.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:10.557+0000 7f7c0a0b7700 1 --2- 172.21.15.16:0/2321397976 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f7be406ace0 0x7f7be406d190 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f7c04066720 tx=0x7f7bec008040 comp rx=0 tx=0).stop 2023-05-20T01:25:10.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:10.557+0000 7f7c0a0b7700 1 -- 172.21.15.16:0/2321397976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c040c6b50 msgr2=0x7f7c04151bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:10.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:10.557+0000 7f7c0a0b7700 1 --2- 172.21.15.16:0/2321397976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c040c6b50 0x7f7c04151bb0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f7bf4009c40 tx=0x7f7bf4004750 comp rx=0 tx=0).stop 2023-05-20T01:25:10.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:10.558+0000 7f7c0a0b7700 1 -- 172.21.15.16:0/2321397976 shutdown_connections 2023-05-20T01:25:10.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:10.558+0000 7f7c0a0b7700 1 --2- 172.21.15.16:0/2321397976 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f7be406ace0 0x7f7be406d190 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:10.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:10.558+0000 7f7c0a0b7700 1 --2- 172.21.15.16:0/2321397976 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7c0415cc00 0x7f7c040ae800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:10.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:10.558+0000 7f7c0a0b7700 1 --2- 172.21.15.16:0/2321397976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c040c6b50 0x7f7c04151bb0 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:10.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:10.558+0000 7f7c0a0b7700 1 --2- 172.21.15.16:0/2321397976 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7c0415c830 0x7f7c041f2aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:10.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:10.558+0000 7f7c0a0b7700 1 -- 172.21.15.16:0/2321397976 >> 172.21.15.16:0/2321397976 conn(0x7f7c040b2920 msgr2=0x7f7c040b0750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:10.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:10.558+0000 7f7c0a0b7700 1 -- 172.21.15.16:0/2321397976 shutdown_connections 2023-05-20T01:25:10.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:10.558+0000 7f7c0a0b7700 1 -- 172.21.15.16:0/2321397976 wait complete. 2023-05-20T01:25:10.565 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' created 2023-05-20T01:25:10.580 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-05-20T01:25:10.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:10.715+0000 7f9564370700 1 Processor -- start 2023-05-20T01:25:10.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:10.715+0000 7f9564370700 1 -- start start 2023-05-20T01:25:10.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:10.716+0000 7f9564370700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f955c0cc8d0 0x7f955c167960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:10.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:10.716+0000 7f9564370700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f955c0cc000 con 0x7f955c0cba20 2023-05-20T01:25:10.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:10.716+0000 7f9564370700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f955c0ca0e0 con 0x7f955c15ca80 2023-05-20T01:25:10.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:10.716+0000 7f9564370700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f955c0c9e40 con 0x7f955c0cc8d0 2023-05-20T01:25:10.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:10.716+0000 7f956190b700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f955c0cba20 0x7f955c0cc1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:47638/0 (socket says 172.21.15.16:47638) 2023-05-20T01:25:10.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:10.716+0000 7f956190b700 1 -- 172.21.15.16:0/3400547297 learned_addr learned my addr 172.21.15.16:0/3400547297 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:25:10.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:10.716+0000 7f956290d700 1 --2- 172.21.15.16:0/3400547297 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f955c0cc8d0 0x7f955c167960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:10.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:10.717+0000 7f956110a700 1 -- 172.21.15.16:0/3400547297 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2469927714 0 0) 0x7f9558001110 con 0x7f955c0cba20 2023-05-20T01:25:10.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:10.717+0000 7f956110a700 1 -- 172.21.15.16:0/3400547297 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95480035b0 con 0x7f955c0cba20 2023-05-20T01:25:10.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:10.717+0000 7f956110a700 1 -- 172.21.15.16:0/3400547297 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3834087505 0 0) 0x7f955c0ca0e0 con 0x7f955c15ca80 2023-05-20T01:25:10.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:10.717+0000 7f956110a700 1 -- 172.21.15.16:0/3400547297 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9558001110 con 0x7f955c15ca80 2023-05-20T01:25:10.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:10.717+0000 7f956110a700 1 -- 172.21.15.16:0/3400547297 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1372302225 0 0) 0x7f95480035b0 con 0x7f955c0cba20 2023-05-20T01:25:10.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:10.717+0000 7f956110a700 1 -- 172.21.15.16:0/3400547297 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f955c0cc8d0 msgr2=0x7f955c167960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:10.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:10.717+0000 7f956110a700 1 --2- 172.21.15.16:0/3400547297 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f955c0cc8d0 0x7f955c167960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:10.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:10.717+0000 7f956110a700 1 -- 172.21.15.16:0/3400547297 >> v1:172.21.15.132:6789/0 conn(0x7f955c15ca80 legacy=0x7f955c160fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:10.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:10.717+0000 7f956110a700 1 -- 172.21.15.16:0/3400547297 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f955c0cdae0 con 0x7f955c0cba20 2023-05-20T01:25:10.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:10.717+0000 7f956110a700 1 -- 172.21.15.16:0/3400547297 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7f9558001800 con 0x7f955c0cba20 2023-05-20T01:25:10.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:10.717+0000 7f956110a700 1 -- 172.21.15.16:0/3400547297 >> v1:172.21.15.16:6789/0 conn(0x7f955c0cba20 legacy=0x7f955c0cc1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:10.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:10.717+0000 7f956110a700 1 --2- 172.21.15.16:0/3400547297 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9548003bc0 0x7f9548003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:10.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:10.717+0000 7f956110a700 1 --2- 172.21.15.16:0/3400547297 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9548004850 0x7f9548004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:10.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:10.717+0000 7f956110a700 1 --2- 172.21.15.16:0/3400547297 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9548005200 0x7f954800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:10.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:10.717+0000 7f956110a700 1 -- 172.21.15.16:0/3400547297 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f955c0c9e40 con 0x7f9548005200 2023-05-20T01:25:10.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:10.717+0000 7f956110a700 1 -- 172.21.15.16:0/3400547297 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f954800bf50 con 0x7f9548003bc0 2023-05-20T01:25:10.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:10.717+0000 7f956110a700 1 -- 172.21.15.16:0/3400547297 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f954800c0e0 con 0x7f9548004850 2023-05-20T01:25:10.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:10.717+0000 7f956290d700 1 --2- 172.21.15.16:0/3400547297 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f955c0cc8d0 0x7f955c167960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-05-20T01:25:10.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:10.717+0000 7f956210c700 1 --2- 172.21.15.16:0/3400547297 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9548003bc0 0x7f9548003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:10.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:10.717+0000 7f956290d700 1 --2- 172.21.15.16:0/3400547297 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9548005200 0x7f954800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:10.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:10.717+0000 7f956190b700 1 --2- 172.21.15.16:0/3400547297 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9548004850 0x7f9548004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:10.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:10.717+0000 7f956290d700 1 -- 172.21.15.16:0/3400547297 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9548004850 msgr2=0x7f9548004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:10.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:10.717+0000 7f956290d700 1 --2- 172.21.15.16:0/3400547297 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9548004850 0x7f9548004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:10.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:10.717+0000 7f956290d700 1 -- 172.21.15.16:0/3400547297 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9548003bc0 msgr2=0x7f9548003fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:10.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:10.717+0000 7f956290d700 1 --2- 172.21.15.16:0/3400547297 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9548003bc0 0x7f9548003fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:10.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:10.717+0000 7f956290d700 1 -- 172.21.15.16:0/3400547297 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95480035b0 con 0x7f9548005200 2023-05-20T01:25:10.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:10.718+0000 7f956290d700 1 --2- 172.21.15.16:0/3400547297 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9548005200 0x7f954800ba10 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f955c06dd00 tx=0x7f9548005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=179aaaf2f8301d1b server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:10.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:10.718+0000 7f956110a700 1 -- 172.21.15.16:0/3400547297 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9554003c20 con 0x7f9548005200 2023-05-20T01:25:10.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:10.718+0000 7f956110a700 1 -- 172.21.15.16:0/3400547297 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f955400f040 con 0x7f9548005200 2023-05-20T01:25:10.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:10.718+0000 7f956110a700 1 -- 172.21.15.16:0/3400547297 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f955400ea30 con 0x7f9548005200 2023-05-20T01:25:10.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:10.718+0000 7f9564370700 1 -- 172.21.15.16:0/3400547297 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9548005200 msgr2=0x7f954800ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:10.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:10.718+0000 7f9564370700 1 --2- 172.21.15.16:0/3400547297 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9548005200 0x7f954800ba10 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f955c06dd00 tx=0x7f9548005880 comp rx=0 tx=0).stop 2023-05-20T01:25:10.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:10.718+0000 7f956290d700 1 -- 172.21.15.16:0/3400547297 reap_dead start 2023-05-20T01:25:10.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:10.718+0000 7f9564370700 1 -- 172.21.15.16:0/3400547297 shutdown_connections 2023-05-20T01:25:10.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:10.718+0000 7f9564370700 1 -- 172.21.15.16:0/3400547297 >> 172.21.15.16:0/3400547297 conn(0x7f955c06cbb0 msgr2=0x7f955c06cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:10.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:10.718+0000 7f9564370700 1 -- 172.21.15.16:0/3400547297 shutdown_connections 2023-05-20T01:25:10.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:10.718+0000 7f9564370700 1 -- 172.21.15.16:0/3400547297 wait complete. 2023-05-20T01:25:10.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:10.720+0000 7f9564370700 1 Processor -- start 2023-05-20T01:25:10.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:10.720+0000 7f9564370700 1 -- start start 2023-05-20T01:25:10.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:10.720+0000 7f9564370700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9548005200 0x7f955c1e5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:10.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:10.720+0000 7f9564370700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f955c15b630 0x7f955c14bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:10.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:10.720+0000 7f9564370700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f955c14b290 0x7f955c1e3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:10.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:10.720+0000 7f9564370700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f955c14bff0 con 0x7f9548005200 2023-05-20T01:25:10.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:10.720+0000 7f9564370700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f955c1e5d60 con 0x7f955c14b290 2023-05-20T01:25:10.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:10.720+0000 7f9564370700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f955c1e6040 con 0x7f955c15b630 2023-05-20T01:25:10.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:10.720+0000 7f956210c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9548005200 0x7f955c1e5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:10.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:10.721+0000 7f956210c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9548005200 0x7f955c1e5760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36120/0 (socket says 172.21.15.16:36120) 2023-05-20T01:25:10.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:10.721+0000 7f956210c700 1 -- 172.21.15.16:0/1931552446 learned_addr learned my addr 172.21.15.16:0/1931552446 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:10.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:10.721+0000 7f956190b700 1 --2- 172.21.15.16:0/1931552446 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f955c15b630 0x7f955c14bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:10.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:10.721+0000 7f956290d700 1 --2- 172.21.15.16:0/1931552446 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f955c14b290 0x7f955c1e3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:10.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:10.721+0000 7f956210c700 1 -- 172.21.15.16:0/1931552446 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f955c15b630 msgr2=0x7f955c14bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:10.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:10.721+0000 7f956210c700 1 --2- 172.21.15.16:0/1931552446 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f955c15b630 0x7f955c14bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:10.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:10.721+0000 7f956210c700 1 -- 172.21.15.16:0/1931552446 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f955c14b290 msgr2=0x7f955c1e3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:10.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:10.721+0000 7f956210c700 1 --2- 172.21.15.16:0/1931552446 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f955c14b290 0x7f955c1e3770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:10.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:10.721+0000 7f956210c700 1 -- 172.21.15.16:0/1931552446 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f955c0cd520 con 0x7f9548005200 2023-05-20T01:25:10.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:10.721+0000 7f956210c700 1 --2- 172.21.15.16:0/1931552446 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9548005200 0x7f955c1e5760 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f954c00f7b0 tx=0x7f954c00fac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:10.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:10.721+0000 7f9552ffd700 1 -- 172.21.15.16:0/1931552446 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f954c0052e0 con 0x7f9548005200 2023-05-20T01:25:10.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:10.721+0000 7f9552ffd700 1 -- 172.21.15.16:0/1931552446 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f954c005460 con 0x7f9548005200 2023-05-20T01:25:10.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:10.721+0000 7f9552ffd700 1 -- 172.21.15.16:0/1931552446 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f954c0038c0 con 0x7f9548005200 2023-05-20T01:25:10.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:10.721+0000 7f9564370700 1 -- 172.21.15.16:0/1931552446 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f955c15ce50 con 0x7f9548005200 2023-05-20T01:25:10.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:10.721+0000 7f9564370700 1 -- 172.21.15.16:0/1931552446 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f955c0cd800 con 0x7f9548005200 2023-05-20T01:25:10.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:10.722+0000 7f9564370700 1 -- 172.21.15.16:0/1931552446 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f955c0747b0 con 0x7f9548005200 2023-05-20T01:25:10.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:10.723+0000 7f9552ffd700 1 -- 172.21.15.16:0/1931552446 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f954c003a20 con 0x7f9548005200 2023-05-20T01:25:10.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:10.723+0000 7f9552ffd700 1 --2- 172.21.15.16:0/1931552446 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f954006ace0 0x7f954006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:10.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:10.723+0000 7f9552ffd700 1 -- 172.21.15.16:0/1931552446 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5900+0+0 (secure 0 0 0) 0x7f954c0891a0 con 0x7f9548005200 2023-05-20T01:25:10.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:10.723+0000 7f956190b700 1 --2- 172.21.15.16:0/1931552446 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f954006ace0 0x7f954006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:10.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:10.724+0000 7f956190b700 1 --2- 172.21.15.16:0/1931552446 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f954006ace0 0x7f954006d190 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f9558007fd0 tx=0x7f9558007a30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:10.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:10.728+0000 7f9552ffd700 1 -- 172.21.15.16:0/1931552446 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f954c0543b0 con 0x7f9548005200 2023-05-20T01:25:10.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:10.924+0000 7f9564370700 1 -- 172.21.15.16:0/1931552446 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f955c073f00 con 0x7f9548005200 2023-05-20T01:25:11.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:11.563+0000 7f9552ffd700 1 -- 172.21.15.16:0/1931552446 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v86) v1 ==== 170+0+0 (secure 0 0 0) 0x7f954c0579d0 con 0x7f9548005200 2023-05-20T01:25:11.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:11.569+0000 7f9564370700 1 -- 172.21.15.16:0/1931552446 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f954006ace0 msgr2=0x7f954006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:11.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:11.569+0000 7f9564370700 1 --2- 172.21.15.16:0/1931552446 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f954006ace0 0x7f954006d190 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f9558007fd0 tx=0x7f9558007a30 comp rx=0 tx=0).stop 2023-05-20T01:25:11.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:11.570+0000 7f9564370700 1 -- 172.21.15.16:0/1931552446 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9548005200 msgr2=0x7f955c1e5760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:11.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:11.570+0000 7f9564370700 1 --2- 172.21.15.16:0/1931552446 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9548005200 0x7f955c1e5760 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f954c00f7b0 tx=0x7f954c00fac0 comp rx=0 tx=0).stop 2023-05-20T01:25:11.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:11.570+0000 7f9564370700 1 -- 172.21.15.16:0/1931552446 shutdown_connections 2023-05-20T01:25:11.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:11.570+0000 7f9564370700 1 --2- 172.21.15.16:0/1931552446 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f955c14b290 0x7f955c1e3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:11.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:11.570+0000 7f9564370700 1 --2- 172.21.15.16:0/1931552446 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9548005200 0x7f955c1e5760 secure :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f954c00f7b0 tx=0x7f954c00fac0 comp rx=0 tx=0).stop 2023-05-20T01:25:11.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:11.570+0000 7f9564370700 1 --2- 172.21.15.16:0/1931552446 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f954006ace0 0x7f954006d190 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:11.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:11.570+0000 7f9564370700 1 --2- 172.21.15.16:0/1931552446 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f955c15b630 0x7f955c14bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:11.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:11.570+0000 7f9564370700 1 -- 172.21.15.16:0/1931552446 >> 172.21.15.16:0/1931552446 conn(0x7f955c06cbb0 msgr2=0x7f955c071be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:11.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:11.573+0000 7f9564370700 1 -- 172.21.15.16:0/1931552446 shutdown_connections 2023-05-20T01:25:11.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:11.574+0000 7f9564370700 1 -- 172.21.15.16:0/1931552446 wait complete. 2023-05-20T01:25:11.581 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' created 2023-05-20T01:25:11.600 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-05-20T01:25:11.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:11.730+0000 7f1cfa46b700 1 Processor -- start 2023-05-20T01:25:11.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:11.731+0000 7f1cfa46b700 1 -- start start 2023-05-20T01:25:11.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:11.731+0000 7f1cfa46b700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f1cf40cf090 0x7f1cf415bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:11.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:11.731+0000 7f1cfa46b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1cf40d0910 0x7f1cf40cf460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:11.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:11.731+0000 7f1cfa46b700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cf40cec30 con 0x7f1cf415e650 2023-05-20T01:25:11.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:11.731+0000 7f1cfa46b700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cf406b470 con 0x7f1cf40d0910 2023-05-20T01:25:11.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:11.731+0000 7f1cfa46b700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cf40cf9a0 con 0x7f1cf40cf090 2023-05-20T01:25:11.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:11.731+0000 7f1cf37fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1cf40d0910 0x7f1cf40cf460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:11.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:11.732+0000 7f1cf8a08700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f1cf415e650 0x7f1cf415ea70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:47660/0 (socket says 172.21.15.16:47660) 2023-05-20T01:25:11.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:11.732+0000 7f1cf37fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1cf40d0910 0x7f1cf40cf460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36130/0 (socket says 172.21.15.16:36130) 2023-05-20T01:25:11.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:11.732+0000 7f1cf8a08700 1 -- 172.21.15.16:0/3048745943 learned_addr learned my addr 172.21.15.16:0/3048745943 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:25:11.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:11.732+0000 7f1cf3fff700 1 --2- 172.21.15.16:0/3048745943 >> v2:172.21.15.132:3300/0 conn(0x7f1cf40cf090 0x7f1cf415bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:11.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:11.732+0000 7f1cf37fe700 1 -- 172.21.15.16:0/3048745943 >> v2:172.21.15.132:3300/0 conn(0x7f1cf40cf090 msgr2=0x7f1cf415bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:11.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:11.732+0000 7f1cf37fe700 1 --2- 172.21.15.16:0/3048745943 >> v2:172.21.15.132:3300/0 conn(0x7f1cf40cf090 0x7f1cf415bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:11.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:11.732+0000 7f1cf37fe700 1 -- 172.21.15.16:0/3048745943 >> v1:172.21.15.16:6789/0 conn(0x7f1cf415e650 legacy=0x7f1cf415ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:11.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:11.732+0000 7f1cf2ffd700 1 -- 172.21.15.16:0/3048745943 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2015674905 0 0) 0x7f1cf40cec30 con 0x7f1cf415e650 2023-05-20T01:25:11.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:11.732+0000 7f1cf37fe700 1 -- 172.21.15.16:0/3048745943 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cf406a6b0 con 0x7f1cf40d0910 2023-05-20T01:25:11.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:11.732+0000 7f1cf37fe700 1 --2- 172.21.15.16:0/3048745943 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1cf40d0910 0x7f1cf40cf460 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f1cf4074360 tx=0x7f1cdc009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=deda3db618adaf56 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:11.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:11.732+0000 7f1cf2ffd700 1 -- 172.21.15.16:0/3048745943 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cdc011830 con 0x7f1cf40d0910 2023-05-20T01:25:11.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:11.732+0000 7f1cf2ffd700 1 -- 172.21.15.16:0/3048745943 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cdc011990 con 0x7f1cf40d0910 2023-05-20T01:25:11.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:11.732+0000 7f1cf2ffd700 1 -- 172.21.15.16:0/3048745943 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cdc011d20 con 0x7f1cf40d0910 2023-05-20T01:25:11.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:11.733+0000 7f1cfa46b700 1 -- 172.21.15.16:0/3048745943 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1cf40d0910 msgr2=0x7f1cf40cf460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:11.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:11.733+0000 7f1cfa46b700 1 --2- 172.21.15.16:0/3048745943 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1cf40d0910 0x7f1cf40cf460 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f1cf4074360 tx=0x7f1cdc009b20 comp rx=0 tx=0).stop 2023-05-20T01:25:11.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:11.733+0000 7f1cfa46b700 1 -- 172.21.15.16:0/3048745943 shutdown_connections 2023-05-20T01:25:11.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:11.733+0000 7f1cfa46b700 1 --2- 172.21.15.16:0/3048745943 >> v2:172.21.15.132:3300/0 conn(0x7f1cf40cf090 0x7f1cf415bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:11.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:11.733+0000 7f1cfa46b700 1 --2- 172.21.15.16:0/3048745943 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1cf40d0910 0x7f1cf40cf460 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:11.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:11.733+0000 7f1cfa46b700 1 -- 172.21.15.16:0/3048745943 >> 172.21.15.16:0/3048745943 conn(0x7f1cf40cd760 msgr2=0x7f1cf40d0050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:11.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:11.733+0000 7f1cfa46b700 1 -- 172.21.15.16:0/3048745943 shutdown_connections 2023-05-20T01:25:11.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:11.733+0000 7f1cfa46b700 1 -- 172.21.15.16:0/3048745943 wait complete. 2023-05-20T01:25:11.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:11.734+0000 7f1cfa46b700 1 Processor -- start 2023-05-20T01:25:11.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:11.734+0000 7f1cfa46b700 1 -- start start 2023-05-20T01:25:11.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:11.735+0000 7f1cfa46b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1cf40cf090 0x7f1cf40c9ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:11.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:11.735+0000 7f1cfa46b700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1cf40d0910 0x7f1cf40b3090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:11.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:11.735+0000 7f1cfa46b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1cf415e650 0x7f1cf40b5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:11.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:11.735+0000 7f1cfa46b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf4152da0 con 0x7f1cf40cf090 2023-05-20T01:25:11.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:11.735+0000 7f1cfa46b700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf41f46c0 con 0x7f1cf415e650 2023-05-20T01:25:11.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:11.735+0000 7f1cfa46b700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf41f49a0 con 0x7f1cf40d0910 2023-05-20T01:25:11.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:11.735+0000 7f1cf3fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1cf40cf090 0x7f1cf40c9ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:11.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:11.735+0000 7f1cf3fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1cf40cf090 0x7f1cf40c9ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36134/0 (socket says 172.21.15.16:36134) 2023-05-20T01:25:11.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:11.735+0000 7f1cf3fff700 1 -- 172.21.15.16:0/1062133137 learned_addr learned my addr 172.21.15.16:0/1062133137 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:11.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:11.735+0000 7f1cf8a08700 1 --2- 172.21.15.16:0/1062133137 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1cf415e650 0x7f1cf40b5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:11.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:11.735+0000 7f1cf3fff700 1 -- 172.21.15.16:0/1062133137 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1cf40d0910 msgr2=0x7f1cf40b3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:11.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:11.735+0000 7f1cf3fff700 1 --2- 172.21.15.16:0/1062133137 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1cf40d0910 0x7f1cf40b3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:11.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:11.735+0000 7f1cf3fff700 1 -- 172.21.15.16:0/1062133137 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1cf415e650 msgr2=0x7f1cf40b5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:11.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:11.735+0000 7f1cf3fff700 1 --2- 172.21.15.16:0/1062133137 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1cf415e650 0x7f1cf40b5810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:11.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:11.735+0000 7f1cf3fff700 1 -- 172.21.15.16:0/1062133137 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cdc0094a0 con 0x7f1cf40cf090 2023-05-20T01:25:11.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:11.735+0000 7f1cf3fff700 1 --2- 172.21.15.16:0/1062133137 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1cf40cf090 0x7f1cf40c9ac0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f1ce400c7f0 tx=0x7f1ce400cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:11.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:11.736+0000 7f1cf0ff9700 1 -- 172.21.15.16:0/1062133137 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ce4011800 con 0x7f1cf40cf090 2023-05-20T01:25:11.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:11.736+0000 7f1cf0ff9700 1 -- 172.21.15.16:0/1062133137 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ce4011960 con 0x7f1cf40cf090 2023-05-20T01:25:11.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:11.736+0000 7f1cfa46b700 1 -- 172.21.15.16:0/1062133137 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cf406ac70 con 0x7f1cf40cf090 2023-05-20T01:25:11.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:11.736+0000 7f1cf0ff9700 1 -- 172.21.15.16:0/1062133137 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ce4011cf0 con 0x7f1cf40cf090 2023-05-20T01:25:11.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:11.736+0000 7f1cfa46b700 1 -- 172.21.15.16:0/1062133137 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cf406a990 con 0x7f1cf40cf090 2023-05-20T01:25:11.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:11.737+0000 7f1cfa46b700 1 -- 172.21.15.16:0/1062133137 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cf406af50 con 0x7f1cf40cf090 2023-05-20T01:25:11.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:11.738+0000 7f1cf0ff9700 1 -- 172.21.15.16:0/1062133137 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1ce4024480 con 0x7f1cf40cf090 2023-05-20T01:25:11.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:11.738+0000 7f1cf0ff9700 1 --2- 172.21.15.16:0/1062133137 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f1cd406ace0 0x7f1cd406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:11.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:11.738+0000 7f1cf0ff9700 1 -- 172.21.15.16:0/1062133137 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6282+0+0 (secure 0 0 0) 0x7f1ce40899d0 con 0x7f1cf40cf090 2023-05-20T01:25:11.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:11.738+0000 7f1cf37fe700 1 --2- 172.21.15.16:0/1062133137 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f1cd406ace0 0x7f1cd406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:11.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:11.739+0000 7f1cf37fe700 1 --2- 172.21.15.16:0/1062133137 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f1cd406ace0 0x7f1cd406d190 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f1cf4153c80 tx=0x7f1cdc010000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:11.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:11.742+0000 7f1cf0ff9700 1 -- 172.21.15.16:0/1062133137 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f1ce40584c0 con 0x7f1cf40cf090 2023-05-20T01:25:11.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:11.986+0000 7f1cfa46b700 1 -- 172.21.15.16:0/1062133137 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f1cf40c6b20 con 0x7f1cf40cf090 2023-05-20T01:25:11.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:11.992+0000 7f1cf0ff9700 1 -- 172.21.15.16:0/1062133137 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 12 and data pool 13 v53) v1 ==== 174+0+0 (secure 0 0 0) 0x7f1ce4058050 con 0x7f1cf40cf090 2023-05-20T01:25:12.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.004+0000 7f1cfa46b700 1 -- 172.21.15.16:0/1062133137 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f1cd406ace0 msgr2=0x7f1cd406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:12.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.004+0000 7f1cfa46b700 1 --2- 172.21.15.16:0/1062133137 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f1cd406ace0 0x7f1cd406d190 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f1cf4153c80 tx=0x7f1cdc010000 comp rx=0 tx=0).stop 2023-05-20T01:25:12.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.004+0000 7f1cfa46b700 1 -- 172.21.15.16:0/1062133137 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1cf40cf090 msgr2=0x7f1cf40c9ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:12.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.004+0000 7f1cfa46b700 1 --2- 172.21.15.16:0/1062133137 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1cf40cf090 0x7f1cf40c9ac0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f1ce400c7f0 tx=0x7f1ce400cbb0 comp rx=0 tx=0).stop 2023-05-20T01:25:12.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.004+0000 7f1cfa46b700 1 -- 172.21.15.16:0/1062133137 shutdown_connections 2023-05-20T01:25:12.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.004+0000 7f1cfa46b700 1 --2- 172.21.15.16:0/1062133137 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1cf415e650 0x7f1cf40b5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:12.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.004+0000 7f1cfa46b700 1 --2- 172.21.15.16:0/1062133137 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1cf40cf090 0x7f1cf40c9ac0 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:12.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.004+0000 7f1cfa46b700 1 --2- 172.21.15.16:0/1062133137 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f1cd406ace0 0x7f1cd406d190 secure :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f1cf4153c80 tx=0x7f1cdc010000 comp rx=0 tx=0).stop 2023-05-20T01:25:12.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.004+0000 7f1cfa46b700 1 --2- 172.21.15.16:0/1062133137 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1cf40d0910 0x7f1cf40b3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:12.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.004+0000 7f1cfa46b700 1 -- 172.21.15.16:0/1062133137 >> 172.21.15.16:0/1062133137 conn(0x7f1cf40cd760 msgr2=0x7f1cf4158a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:12.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.005+0000 7f1cfa46b700 1 -- 172.21.15.16:0/1062133137 shutdown_connections 2023-05-20T01:25:12.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.005+0000 7f1cfa46b700 1 -- 172.21.15.16:0/1062133137 wait complete. 2023-05-20T01:25:12.012 INFO:teuthology.orchestra.run.smithi016.stderr:new fs with metadata pool 12 and data pool 13 2023-05-20T01:25:12.032 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-20T01:25:12.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.153+0000 7f58e0539700 1 Processor -- start 2023-05-20T01:25:12.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.153+0000 7f58e0539700 1 -- start start 2023-05-20T01:25:12.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.154+0000 7f58e0539700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f58d8161190 0x7f58d806de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:12.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.154+0000 7f58e0539700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f58d806ac20 0x7f58d8071140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:12.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.154+0000 7f58e0539700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58d815d020 con 0x7f58d806e370 2023-05-20T01:25:12.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.154+0000 7f58e0539700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58d80cebf0 con 0x7f58d8161190 2023-05-20T01:25:12.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.154+0000 7f58e0539700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58d806e9a0 con 0x7f58d806ac20 2023-05-20T01:25:12.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.154+0000 7f58de2d5700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f58d8161190 0x7f58d806de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:12.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.154+0000 7f58de2d5700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f58d8161190 0x7f58d806de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36156/0 (socket says 172.21.15.16:36156) 2023-05-20T01:25:12.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.154+0000 7f58de2d5700 1 -- 172.21.15.16:0/1102620020 learned_addr learned my addr 172.21.15.16:0/1102620020 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:12.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.154+0000 7f58dead6700 1 --2- 172.21.15.16:0/1102620020 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f58d806ac20 0x7f58d8071140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:12.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.154+0000 7f58de2d5700 1 -- 172.21.15.16:0/1102620020 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f58d806ac20 msgr2=0x7f58d8071140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:12.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.154+0000 7f58de2d5700 1 --2- 172.21.15.16:0/1102620020 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f58d806ac20 0x7f58d8071140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:12.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.154+0000 7f58de2d5700 1 -- 172.21.15.16:0/1102620020 >> v1:172.21.15.132:6789/0 conn(0x7f58d806e370 legacy=0x7f58d806a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:12.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.154+0000 7f58de2d5700 1 -- 172.21.15.16:0/1102620020 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58d80cee90 con 0x7f58d8161190 2023-05-20T01:25:12.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.154+0000 7f58de2d5700 1 --2- 172.21.15.16:0/1102620020 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58d8161190 0x7f58d806de30 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f58d40098f0 tx=0x7f58d4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e8409c6632c4b3 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:12.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.155+0000 7f58dd2d3700 1 -- 172.21.15.16:0/1102620020 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d400b8a0 con 0x7f58d8161190 2023-05-20T01:25:12.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.155+0000 7f58dd2d3700 1 -- 172.21.15.16:0/1102620020 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58d400ba00 con 0x7f58d8161190 2023-05-20T01:25:12.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.155+0000 7f58dd2d3700 1 -- 172.21.15.16:0/1102620020 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d400bd90 con 0x7f58d8161190 2023-05-20T01:25:12.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.155+0000 7f58e0539700 1 -- 172.21.15.16:0/1102620020 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58d8161190 msgr2=0x7f58d806de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:12.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.155+0000 7f58e0539700 1 --2- 172.21.15.16:0/1102620020 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58d8161190 0x7f58d806de30 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f58d40098f0 tx=0x7f58d4009c00 comp rx=0 tx=0).stop 2023-05-20T01:25:12.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.155+0000 7f58e0539700 1 -- 172.21.15.16:0/1102620020 shutdown_connections 2023-05-20T01:25:12.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.155+0000 7f58e0539700 1 --2- 172.21.15.16:0/1102620020 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f58d806ac20 0x7f58d8071140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:12.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.155+0000 7f58e0539700 1 --2- 172.21.15.16:0/1102620020 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58d8161190 0x7f58d806de30 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:12.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.155+0000 7f58e0539700 1 -- 172.21.15.16:0/1102620020 >> 172.21.15.16:0/1102620020 conn(0x7f58d80d0c70 msgr2=0x7f58d80b1af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:12.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.155+0000 7f58e0539700 1 -- 172.21.15.16:0/1102620020 shutdown_connections 2023-05-20T01:25:12.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.155+0000 7f58e0539700 1 -- 172.21.15.16:0/1102620020 wait complete. 2023-05-20T01:25:12.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.156+0000 7f58e0539700 1 Processor -- start 2023-05-20T01:25:12.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.157+0000 7f58e0539700 1 -- start start 2023-05-20T01:25:12.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.157+0000 7f58e0539700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f58d806ac20 0x7f58d80c62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:12.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.157+0000 7f58e0539700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58d806e370 0x7f58d80b2640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:12.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.157+0000 7f58e0539700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58d8161190 0x7f58d815ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:12.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.157+0000 7f58e0539700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58d80aef30 con 0x7f58d8161190 2023-05-20T01:25:12.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.157+0000 7f58e0539700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58d80c68a0 con 0x7f58d806e370 2023-05-20T01:25:12.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.157+0000 7f58e0539700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58d81f5b30 con 0x7f58d806ac20 2023-05-20T01:25:12.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.157+0000 7f58dead6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58d8161190 0x7f58d815ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:12.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.157+0000 7f58dead6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58d8161190 0x7f58d815ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36164/0 (socket says 172.21.15.16:36164) 2023-05-20T01:25:12.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.157+0000 7f58dead6700 1 -- 172.21.15.16:0/4208173177 learned_addr learned my addr 172.21.15.16:0/4208173177 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:12.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.157+0000 7f58de2d5700 1 --2- 172.21.15.16:0/4208173177 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f58d806ac20 0x7f58d80c62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:12.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.157+0000 7f58ddad4700 1 --2- 172.21.15.16:0/4208173177 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58d806e370 0x7f58d80b2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:12.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.158+0000 7f58dead6700 1 -- 172.21.15.16:0/4208173177 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f58d806ac20 msgr2=0x7f58d80c62c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:12.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.158+0000 7f58dead6700 1 --2- 172.21.15.16:0/4208173177 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f58d806ac20 0x7f58d80c62c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:12.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.158+0000 7f58dead6700 1 -- 172.21.15.16:0/4208173177 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58d806e370 msgr2=0x7f58d80b2640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:12.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.158+0000 7f58dead6700 1 --2- 172.21.15.16:0/4208173177 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58d806e370 0x7f58d80b2640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:12.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.158+0000 7f58dead6700 1 -- 172.21.15.16:0/4208173177 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58d4009550 con 0x7f58d8161190 2023-05-20T01:25:12.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.158+0000 7f58ddad4700 1 --2- 172.21.15.16:0/4208173177 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58d806e370 0x7f58d80b2640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:25:12.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.158+0000 7f58dead6700 1 --2- 172.21.15.16:0/4208173177 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58d8161190 0x7f58d815ae30 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f58d0007e30 tx=0x7f58d000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:12.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.158+0000 7f58ceffd700 1 -- 172.21.15.16:0/4208173177 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d000ede0 con 0x7f58d8161190 2023-05-20T01:25:12.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.158+0000 7f58ceffd700 1 -- 172.21.15.16:0/4208173177 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58d0014420 con 0x7f58d8161190 2023-05-20T01:25:12.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.158+0000 7f58e0539700 1 -- 172.21.15.16:0/4208173177 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58d80cf450 con 0x7f58d8161190 2023-05-20T01:25:12.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.158+0000 7f58ceffd700 1 -- 172.21.15.16:0/4208173177 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d00147d0 con 0x7f58d8161190 2023-05-20T01:25:12.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.158+0000 7f58e0539700 1 -- 172.21.15.16:0/4208173177 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58d80cf170 con 0x7f58d8161190 2023-05-20T01:25:12.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.159+0000 7f58e0539700 1 -- 172.21.15.16:0/4208173177 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58d80cf730 con 0x7f58d8161190 2023-05-20T01:25:12.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.160+0000 7f58ceffd700 1 -- 172.21.15.16:0/4208173177 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f58d0014930 con 0x7f58d8161190 2023-05-20T01:25:12.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.160+0000 7f58ceffd700 1 --2- 172.21.15.16:0/4208173177 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f58bc06ace0 0x7f58bc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:12.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.160+0000 7f58ceffd700 1 -- 172.21.15.16:0/4208173177 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f58d0089d60 con 0x7f58d8161190 2023-05-20T01:25:12.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.160+0000 7f58de2d5700 1 --2- 172.21.15.16:0/4208173177 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f58bc06ace0 0x7f58bc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:12.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.160+0000 7f58de2d5700 1 --2- 172.21.15.16:0/4208173177 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f58bc06ace0 0x7f58bc06d190 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f58d4014040 tx=0x7f58d4008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:12.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.164+0000 7f58ceffd700 1 -- 172.21.15.16:0/4208173177 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f58d0054cd0 con 0x7f58d8161190 2023-05-20T01:25:12.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.361+0000 7f58e0539700 1 -- 172.21.15.16:0/4208173177 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f58d8161560 con 0x7f58d8161190 2023-05-20T01:25:12.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.363+0000 7f58ceffd700 1 -- 172.21.15.16:0/4208173177 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+18466 (secure 0 0 0) 0x7f58d00582f0 con 0x7f58d8161190 2023-05-20T01:25:12.367 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:25:12.368 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":87,"fsid":"c2245cc5-ecab-4087-a59b-bc81db901676","created":"2023-05-20T01:16:09.549230+0000","modified":"2023-05-20T01:25:11.987666+0000","last_up_change":"2023-05-20T01:16:19.788624+0000","last_in_change":"2023-05-20T01:16:12.092160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-05-20T01:16:20.934366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-05-20T01:16:21.372294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-05-20T01:25:09.967348+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-05-20T01:25:10.956221+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"317a83cd-19c8-4958-a301-6087ef342289","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6825","nonce":3923634179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6827","nonce":3923634179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6831","nonce":3923634179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6829","nonce":3923634179}]},"public_addr":"172.21.15.16:6825/3923634179","cluster_addr":"172.21.15.16:6827/3923634179","heartbeat_back_addr":"172.21.15.16:6831/3923634179","heartbeat_front_addr":"172.21.15.16:6829/3923634179","state":["exists","up"]},{"osd":1,"uuid":"5581046f-8219-4089-8d7d-b7846399505e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6809","nonce":3860264220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6811","nonce":3860264220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6815","nonce":3860264220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6813","nonce":3860264220}]},"public_addr":"172.21.15.16:6809/3860264220","cluster_addr":"172.21.15.16:6811/3860264220","heartbeat_back_addr":"172.21.15.16:6815/3860264220","heartbeat_front_addr":"172.21.15.16:6813/3860264220","state":["exists","up"]},{"osd":2,"uuid":"072e9e66-4133-478e-8d4a-64d46c2963f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6817","nonce":293187030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6819","nonce":293187030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6823","nonce":293187030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6821","nonce":293187030}]},"public_addr":"172.21.15.16:6817/293187030","cluster_addr":"172.21.15.16:6819/293187030","heartbeat_back_addr":"172.21.15.16:6823/293187030","heartbeat_front_addr":"172.21.15.16:6821/293187030","state":["exists","up"]},{"osd":3,"uuid":"11e1bba6-00f8-48dd-9028-d8d716b7f145","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6801","nonce":3669781890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6803","nonce":3669781890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6807","nonce":3669781890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6805","nonce":3669781890}]},"public_addr":"172.21.15.16:6801/3669781890","cluster_addr":"172.21.15.16:6803/3669781890","heartbeat_back_addr":"172.21.15.16:6807/3669781890","heartbeat_front_addr":"172.21.15.16:6805/3669781890","state":["exists","up"]},{"osd":4,"uuid":"b8909c3a-cf70-46cb-8d0b-009c707cf9d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6819","nonce":506317223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6823","nonce":506317223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6829","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6831","nonce":506317223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6827","nonce":506317223}]},"public_addr":"172.21.15.132:6819/506317223","cluster_addr":"172.21.15.132:6823/506317223","heartbeat_back_addr":"172.21.15.132:6831/506317223","heartbeat_front_addr":"172.21.15.132:6827/506317223","state":["exists","up"]},{"osd":5,"uuid":"b1818304-4583-47bf-a6ec-0bfbb5166233","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6818","nonce":1844689995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6822","nonce":1844689995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6830","nonce":1844689995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6826","nonce":1844689995}]},"public_addr":"172.21.15.132:6818/1844689995","cluster_addr":"172.21.15.132:6822/1844689995","heartbeat_back_addr":"172.21.15.132:6830/1844689995","heartbeat_front_addr":"172.21.15.132:6826/1844689995","state":["exists","up"]},{"osd":6,"uuid":"d37f4691-eed1-4a93-8818-18354f03f4a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6801","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6802","nonce":727581123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6805","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6807","nonce":727581123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6813","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6815","nonce":727581123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6811","nonce":727581123}]},"public_addr":"172.21.15.132:6802/727581123","cluster_addr":"172.21.15.132:6807/727581123","heartbeat_back_addr":"172.21.15.132:6815/727581123","heartbeat_front_addr":"172.21.15.132:6811/727581123","state":["exists","up"]},{"osd":7,"uuid":"4f712f62-5d1d-4816-8042-03209144f8b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6803","nonce":466001149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6806","nonce":466001149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6814","nonce":466001149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6810","nonce":466001149}]},"public_addr":"172.21.15.132:6803/466001149","cluster_addr":"172.21.15.132:6806/466001149","heartbeat_back_addr":"172.21.15.132:6814/466001149","heartbeat_front_addr":"172.21.15.132:6810/466001149","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.338863+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.278680+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305694+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.246835+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.276237+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.304441+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305421+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.267050+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-20T01:25:12.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.365+0000 7f58e0539700 1 -- 172.21.15.16:0/4208173177 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f58bc06ace0 msgr2=0x7f58bc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:12.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.365+0000 7f58e0539700 1 --2- 172.21.15.16:0/4208173177 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f58bc06ace0 0x7f58bc06d190 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f58d4014040 tx=0x7f58d4008030 comp rx=0 tx=0).stop 2023-05-20T01:25:12.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.365+0000 7f58e0539700 1 -- 172.21.15.16:0/4208173177 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58d8161190 msgr2=0x7f58d815ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:12.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.365+0000 7f58e0539700 1 --2- 172.21.15.16:0/4208173177 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58d8161190 0x7f58d815ae30 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f58d0007e30 tx=0x7f58d000e3c0 comp rx=0 tx=0).stop 2023-05-20T01:25:12.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.365+0000 7f58e0539700 1 -- 172.21.15.16:0/4208173177 shutdown_connections 2023-05-20T01:25:12.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.365+0000 7f58e0539700 1 --2- 172.21.15.16:0/4208173177 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58d8161190 0x7f58d815ae30 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:12.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.365+0000 7f58e0539700 1 --2- 172.21.15.16:0/4208173177 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f58bc06ace0 0x7f58bc06d190 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:12.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.365+0000 7f58e0539700 1 --2- 172.21.15.16:0/4208173177 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f58d806ac20 0x7f58d80c62c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:12.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.365+0000 7f58e0539700 1 --2- 172.21.15.16:0/4208173177 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58d806e370 0x7f58d80b2640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:12.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.365+0000 7f58e0539700 1 -- 172.21.15.16:0/4208173177 >> 172.21.15.16:0/4208173177 conn(0x7f58d80d0c70 msgr2=0x7f58d80ae030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:12.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.365+0000 7f58e0539700 1 -- 172.21.15.16:0/4208173177 shutdown_connections 2023-05-20T01:25:12.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.365+0000 7f58e0539700 1 -- 172.21.15.16:0/4208173177 wait complete. 2023-05-20T01:25:12.381 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-20T01:25:12.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.503+0000 7f83f696e700 1 Processor -- start 2023-05-20T01:25:12.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.503+0000 7f83f696e700 1 -- start start 2023-05-20T01:25:12.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.504+0000 7f83f696e700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f83f01585d0 0x7f83f015ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:12.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.504+0000 7f83f696e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f83f015b360 0x7f83f006a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:12.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.504+0000 7f83f696e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f83f00cc9c0 0x7f83f006aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:12.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.504+0000 7f83f696e700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83f015e540 con 0x7f83f01585d0 2023-05-20T01:25:12.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.504+0000 7f83f696e700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83f00ccf40 con 0x7f83f015b360 2023-05-20T01:25:12.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.504+0000 7f83f696e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83f00cd220 con 0x7f83f00cc9c0 2023-05-20T01:25:12.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.504+0000 7f83effff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f83f01585d0 0x7f83f015ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:12.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.504+0000 7f83effff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f83f01585d0 0x7f83f015ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36168/0 (socket says 172.21.15.16:36168) 2023-05-20T01:25:12.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.504+0000 7f83effff700 1 -- 172.21.15.16:0/389385663 learned_addr learned my addr 172.21.15.16:0/389385663 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:12.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.504+0000 7f83ef7fe700 1 --2- 172.21.15.16:0/389385663 >> v2:172.21.15.132:3300/0 conn(0x7f83f015b360 0x7f83f006a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:12.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.504+0000 7f83f4f0b700 1 --2- 172.21.15.16:0/389385663 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f83f00cc9c0 0x7f83f006aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:12.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.504+0000 7f83effff700 1 -- 172.21.15.16:0/389385663 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f83f00cc9c0 msgr2=0x7f83f006aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:12.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.505+0000 7f83effff700 1 --2- 172.21.15.16:0/389385663 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f83f00cc9c0 0x7f83f006aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:12.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.505+0000 7f83effff700 1 -- 172.21.15.16:0/389385663 >> v2:172.21.15.132:3300/0 conn(0x7f83f015b360 msgr2=0x7f83f006a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:12.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.505+0000 7f83effff700 1 --2- 172.21.15.16:0/389385663 >> v2:172.21.15.132:3300/0 conn(0x7f83f015b360 0x7f83f006a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:12.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.505+0000 7f83effff700 1 -- 172.21.15.16:0/389385663 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83f00cd500 con 0x7f83f01585d0 2023-05-20T01:25:12.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.505+0000 7f83ef7fe700 1 --2- 172.21.15.16:0/389385663 >> v2:172.21.15.132:3300/0 conn(0x7f83f015b360 0x7f83f006a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:25:12.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.505+0000 7f83effff700 1 --2- 172.21.15.16:0/389385663 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83f01585d0 0x7f83f015ae20 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f83f0070c40 tx=0x7f83e0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa70ff8a37719090 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:12.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.505+0000 7f83eeffd700 1 -- 172.21.15.16:0/389385663 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e000b870 con 0x7f83f01585d0 2023-05-20T01:25:12.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.505+0000 7f83eeffd700 1 -- 172.21.15.16:0/389385663 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83e000b9d0 con 0x7f83f01585d0 2023-05-20T01:25:12.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.505+0000 7f83eeffd700 1 -- 172.21.15.16:0/389385663 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e000bd60 con 0x7f83f01585d0 2023-05-20T01:25:12.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.505+0000 7f83f696e700 1 -- 172.21.15.16:0/389385663 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83f01585d0 msgr2=0x7f83f015ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:12.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.505+0000 7f83f696e700 1 --2- 172.21.15.16:0/389385663 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83f01585d0 0x7f83f015ae20 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f83f0070c40 tx=0x7f83e0009bd0 comp rx=0 tx=0).stop 2023-05-20T01:25:12.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.505+0000 7f83f696e700 1 -- 172.21.15.16:0/389385663 shutdown_connections 2023-05-20T01:25:12.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.505+0000 7f83f696e700 1 --2- 172.21.15.16:0/389385663 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f83f00cc9c0 0x7f83f006aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:12.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.506+0000 7f83f696e700 1 --2- 172.21.15.16:0/389385663 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83f01585d0 0x7f83f015ae20 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:12.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.506+0000 7f83f696e700 1 --2- 172.21.15.16:0/389385663 >> v2:172.21.15.132:3300/0 conn(0x7f83f015b360 0x7f83f006a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:12.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.506+0000 7f83f696e700 1 -- 172.21.15.16:0/389385663 >> 172.21.15.16:0/389385663 conn(0x7f83f00cffb0 msgr2=0x7f83f00d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:12.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.506+0000 7f83f696e700 1 -- 172.21.15.16:0/389385663 shutdown_connections 2023-05-20T01:25:12.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.506+0000 7f83f696e700 1 -- 172.21.15.16:0/389385663 wait complete. 2023-05-20T01:25:12.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.507+0000 7f83f696e700 1 Processor -- start 2023-05-20T01:25:12.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.507+0000 7f83f696e700 1 -- start start 2023-05-20T01:25:12.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.508+0000 7f83f696e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83f00cc9c0 0x7f83f00b3a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:12.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.508+0000 7f83f696e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f83f01585d0 0x7f83f015bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:12.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.508+0000 7f83f696e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f83f015b360 0x7f83f00c4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:12.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.508+0000 7f83f696e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83f014a960 con 0x7f83f00cc9c0 2023-05-20T01:25:12.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.508+0000 7f83f696e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83f01470b0 con 0x7f83f01585d0 2023-05-20T01:25:12.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.508+0000 7f83f696e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83f015c300 con 0x7f83f015b360 2023-05-20T01:25:12.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.508+0000 7f83effff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83f00cc9c0 0x7f83f00b3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:12.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.508+0000 7f83effff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83f00cc9c0 0x7f83f00b3a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36174/0 (socket says 172.21.15.16:36174) 2023-05-20T01:25:12.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.508+0000 7f83effff700 1 -- 172.21.15.16:0/3277859857 learned_addr learned my addr 172.21.15.16:0/3277859857 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:12.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.508+0000 7f83ef7fe700 1 --2- 172.21.15.16:0/3277859857 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f83f01585d0 0x7f83f015bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:12.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.508+0000 7f83f4f0b700 1 --2- 172.21.15.16:0/3277859857 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f83f015b360 0x7f83f00c4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:12.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.508+0000 7f83effff700 1 -- 172.21.15.16:0/3277859857 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f83f015b360 msgr2=0x7f83f00c4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:12.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.508+0000 7f83effff700 1 --2- 172.21.15.16:0/3277859857 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f83f015b360 0x7f83f00c4430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:12.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.508+0000 7f83effff700 1 -- 172.21.15.16:0/3277859857 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f83f01585d0 msgr2=0x7f83f015bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:12.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.508+0000 7f83effff700 1 --2- 172.21.15.16:0/3277859857 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f83f01585d0 0x7f83f015bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:12.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.508+0000 7f83effff700 1 -- 172.21.15.16:0/3277859857 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83e0009550 con 0x7f83f00cc9c0 2023-05-20T01:25:12.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.508+0000 7f83f4f0b700 1 --2- 172.21.15.16:0/3277859857 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f83f015b360 0x7f83f00c4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:25:12.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.508+0000 7f83ef7fe700 1 --2- 172.21.15.16:0/3277859857 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f83f01585d0 0x7f83f015bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:25:12.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.508+0000 7f83effff700 1 --2- 172.21.15.16:0/3277859857 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83f00cc9c0 0x7f83f00b3a30 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f83f00d0cc0 tx=0x7f83e0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:12.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.508+0000 7f83ecff9700 1 -- 172.21.15.16:0/3277859857 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e0004310 con 0x7f83f00cc9c0 2023-05-20T01:25:12.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.508+0000 7f83ecff9700 1 -- 172.21.15.16:0/3277859857 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83e001f070 con 0x7f83f00cc9c0 2023-05-20T01:25:12.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.509+0000 7f83f696e700 1 -- 172.21.15.16:0/3277859857 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83f00719a0 con 0x7f83f00cc9c0 2023-05-20T01:25:12.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.509+0000 7f83ecff9700 1 -- 172.21.15.16:0/3277859857 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e00036a0 con 0x7f83f00cc9c0 2023-05-20T01:25:12.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.509+0000 7f83f696e700 1 -- 172.21.15.16:0/3277859857 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83f00cd7e0 con 0x7f83f00cc9c0 2023-05-20T01:25:12.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.509+0000 7f83f696e700 1 -- 172.21.15.16:0/3277859857 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83f00c4970 con 0x7f83f00cc9c0 2023-05-20T01:25:12.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.510+0000 7f83ecff9700 1 -- 172.21.15.16:0/3277859857 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f83e0021030 con 0x7f83f00cc9c0 2023-05-20T01:25:12.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.510+0000 7f83ecff9700 1 --2- 172.21.15.16:0/3277859857 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f83dc06ace0 0x7f83dc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:12.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.510+0000 7f83ecff9700 1 -- 172.21.15.16:0/3277859857 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f83e008abb0 con 0x7f83f00cc9c0 2023-05-20T01:25:12.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.510+0000 7f83ef7fe700 1 --2- 172.21.15.16:0/3277859857 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f83dc06ace0 0x7f83dc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:12.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.511+0000 7f83ef7fe700 1 --2- 172.21.15.16:0/3277859857 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f83dc06ace0 0x7f83dc06d190 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f83f01573a0 tx=0x7f83e4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:12.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.515+0000 7f83ecff9700 1 -- 172.21.15.16:0/3277859857 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f83e0055bc0 con 0x7f83f00cc9c0 2023-05-20T01:25:12.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.714+0000 7f83f696e700 1 -- 172.21.15.16:0/3277859857 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f83f00747b0 con 0x7f83f00cc9c0 2023-05-20T01:25:12.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.715+0000 7f83ecff9700 1 -- 172.21.15.16:0/3277859857 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+18466 (secure 0 0 0) 0x7f83e00591e0 con 0x7f83f00cc9c0 2023-05-20T01:25:12.720 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:25:12.720 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":87,"fsid":"c2245cc5-ecab-4087-a59b-bc81db901676","created":"2023-05-20T01:16:09.549230+0000","modified":"2023-05-20T01:25:11.987666+0000","last_up_change":"2023-05-20T01:16:19.788624+0000","last_in_change":"2023-05-20T01:16:12.092160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-05-20T01:16:20.934366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-05-20T01:16:21.372294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-05-20T01:25:09.967348+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-05-20T01:25:10.956221+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"317a83cd-19c8-4958-a301-6087ef342289","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6825","nonce":3923634179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6827","nonce":3923634179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6831","nonce":3923634179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6829","nonce":3923634179}]},"public_addr":"172.21.15.16:6825/3923634179","cluster_addr":"172.21.15.16:6827/3923634179","heartbeat_back_addr":"172.21.15.16:6831/3923634179","heartbeat_front_addr":"172.21.15.16:6829/3923634179","state":["exists","up"]},{"osd":1,"uuid":"5581046f-8219-4089-8d7d-b7846399505e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6809","nonce":3860264220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6811","nonce":3860264220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6815","nonce":3860264220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6813","nonce":3860264220}]},"public_addr":"172.21.15.16:6809/3860264220","cluster_addr":"172.21.15.16:6811/3860264220","heartbeat_back_addr":"172.21.15.16:6815/3860264220","heartbeat_front_addr":"172.21.15.16:6813/3860264220","state":["exists","up"]},{"osd":2,"uuid":"072e9e66-4133-478e-8d4a-64d46c2963f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6817","nonce":293187030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6819","nonce":293187030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6823","nonce":293187030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6821","nonce":293187030}]},"public_addr":"172.21.15.16:6817/293187030","cluster_addr":"172.21.15.16:6819/293187030","heartbeat_back_addr":"172.21.15.16:6823/293187030","heartbeat_front_addr":"172.21.15.16:6821/293187030","state":["exists","up"]},{"osd":3,"uuid":"11e1bba6-00f8-48dd-9028-d8d716b7f145","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6801","nonce":3669781890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6803","nonce":3669781890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6807","nonce":3669781890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6805","nonce":3669781890}]},"public_addr":"172.21.15.16:6801/3669781890","cluster_addr":"172.21.15.16:6803/3669781890","heartbeat_back_addr":"172.21.15.16:6807/3669781890","heartbeat_front_addr":"172.21.15.16:6805/3669781890","state":["exists","up"]},{"osd":4,"uuid":"b8909c3a-cf70-46cb-8d0b-009c707cf9d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6819","nonce":506317223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6823","nonce":506317223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6829","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6831","nonce":506317223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6827","nonce":506317223}]},"public_addr":"172.21.15.132:6819/506317223","cluster_addr":"172.21.15.132:6823/506317223","heartbeat_back_addr":"172.21.15.132:6831/506317223","heartbeat_front_addr":"172.21.15.132:6827/506317223","state":["exists","up"]},{"osd":5,"uuid":"b1818304-4583-47bf-a6ec-0bfbb5166233","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6818","nonce":1844689995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6822","nonce":1844689995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6830","nonce":1844689995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6826","nonce":1844689995}]},"public_addr":"172.21.15.132:6818/1844689995","cluster_addr":"172.21.15.132:6822/1844689995","heartbeat_back_addr":"172.21.15.132:6830/1844689995","heartbeat_front_addr":"172.21.15.132:6826/1844689995","state":["exists","up"]},{"osd":6,"uuid":"d37f4691-eed1-4a93-8818-18354f03f4a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6801","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6802","nonce":727581123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6805","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6807","nonce":727581123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6813","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6815","nonce":727581123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6811","nonce":727581123}]},"public_addr":"172.21.15.132:6802/727581123","cluster_addr":"172.21.15.132:6807/727581123","heartbeat_back_addr":"172.21.15.132:6815/727581123","heartbeat_front_addr":"172.21.15.132:6811/727581123","state":["exists","up"]},{"osd":7,"uuid":"4f712f62-5d1d-4816-8042-03209144f8b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6803","nonce":466001149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6806","nonce":466001149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6814","nonce":466001149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6810","nonce":466001149}]},"public_addr":"172.21.15.132:6803/466001149","cluster_addr":"172.21.15.132:6806/466001149","heartbeat_back_addr":"172.21.15.132:6814/466001149","heartbeat_front_addr":"172.21.15.132:6810/466001149","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.338863+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.278680+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305694+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.246835+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.276237+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.304441+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305421+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.267050+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-20T01:25:12.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.717+0000 7f83f696e700 1 -- 172.21.15.16:0/3277859857 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f83dc06ace0 msgr2=0x7f83dc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:12.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.717+0000 7f83f696e700 1 --2- 172.21.15.16:0/3277859857 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f83dc06ace0 0x7f83dc06d190 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f83f01573a0 tx=0x7f83e4008040 comp rx=0 tx=0).stop 2023-05-20T01:25:12.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.717+0000 7f83f696e700 1 -- 172.21.15.16:0/3277859857 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83f00cc9c0 msgr2=0x7f83f00b3a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:12.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.717+0000 7f83f696e700 1 --2- 172.21.15.16:0/3277859857 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83f00cc9c0 0x7f83f00b3a30 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f83f00d0cc0 tx=0x7f83e0004750 comp rx=0 tx=0).stop 2023-05-20T01:25:12.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.718+0000 7f83f696e700 1 -- 172.21.15.16:0/3277859857 shutdown_connections 2023-05-20T01:25:12.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.718+0000 7f83f696e700 1 --2- 172.21.15.16:0/3277859857 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f83dc06ace0 0x7f83dc06d190 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:12.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.718+0000 7f83f696e700 1 --2- 172.21.15.16:0/3277859857 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f83f015b360 0x7f83f00c4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:12.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.718+0000 7f83f696e700 1 --2- 172.21.15.16:0/3277859857 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83f00cc9c0 0x7f83f00b3a30 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:12.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.718+0000 7f83f696e700 1 --2- 172.21.15.16:0/3277859857 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f83f01585d0 0x7f83f015bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:12.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.718+0000 7f83f696e700 1 -- 172.21.15.16:0/3277859857 >> 172.21.15.16:0/3277859857 conn(0x7f83f00cffb0 msgr2=0x7f83f01578f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:12.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.718+0000 7f83f696e700 1 -- 172.21.15.16:0/3277859857 shutdown_connections 2023-05-20T01:25:12.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.718+0000 7f83f696e700 1 -- 172.21.15.16:0/3277859857 wait complete. 2023-05-20T01:25:12.734 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-05-20T01:25:12.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.858+0000 7fb7ba9d6700 1 Processor -- start 2023-05-20T01:25:12.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.858+0000 7fb7ba9d6700 1 -- start start 2023-05-20T01:25:12.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.859+0000 7fb7ba9d6700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb7b415ab80 0x7fb7b40cda80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:12.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.859+0000 7fb7ba9d6700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fb7b40cdfc0 0x7fb7b40cec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:12.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.859+0000 7fb7ba9d6700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb7b40cf1c0 0x7fb7b415e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:12.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.859+0000 7fb7ba9d6700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7b40cb700 con 0x7fb7b40cf1c0 2023-05-20T01:25:12.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.859+0000 7fb7ba9d6700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7b40cf640 con 0x7fb7b40cdfc0 2023-05-20T01:25:12.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.859+0000 7fb7ba9d6700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7b40ce4d0 con 0x7fb7b415ab80 2023-05-20T01:25:12.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.859+0000 7fb7b8f73700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb7b40cf1c0 0x7fb7b415e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:12.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.859+0000 7fb7b8f73700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb7b40cf1c0 0x7fb7b415e4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36182/0 (socket says 172.21.15.16:36182) 2023-05-20T01:25:12.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.859+0000 7fb7b8f73700 1 -- 172.21.15.16:0/2537506294 learned_addr learned my addr 172.21.15.16:0/2537506294 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:12.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.860+0000 7fb7b3fff700 1 --2- 172.21.15.16:0/2537506294 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb7b415ab80 0x7fb7b40cda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:12.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.860+0000 7fb7b37fe700 1 --2- 172.21.15.16:0/2537506294 >> v2:172.21.15.132:3300/0 conn(0x7fb7b40cdfc0 0x7fb7b40cec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:12.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.860+0000 7fb7b8f73700 1 -- 172.21.15.16:0/2537506294 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb7b415ab80 msgr2=0x7fb7b40cda80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:12.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.860+0000 7fb7b8f73700 1 --2- 172.21.15.16:0/2537506294 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb7b415ab80 0x7fb7b40cda80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:12.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.860+0000 7fb7b8f73700 1 -- 172.21.15.16:0/2537506294 >> v2:172.21.15.132:3300/0 conn(0x7fb7b40cdfc0 msgr2=0x7fb7b40cec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:12.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.860+0000 7fb7b8f73700 1 --2- 172.21.15.16:0/2537506294 >> v2:172.21.15.132:3300/0 conn(0x7fb7b40cdfc0 0x7fb7b40cec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:12.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.860+0000 7fb7b8f73700 1 -- 172.21.15.16:0/2537506294 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7b40ce7b0 con 0x7fb7b40cf1c0 2023-05-20T01:25:12.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.860+0000 7fb7b8f73700 1 --2- 172.21.15.16:0/2537506294 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7b40cf1c0 0x7fb7b415e4e0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fb7b4072000 tx=0x7fb7ac00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abd13d83bcf8cff1 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:12.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.860+0000 7fb7b2ffd700 1 -- 172.21.15.16:0/2537506294 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7ac00e070 con 0x7fb7b40cf1c0 2023-05-20T01:25:12.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.860+0000 7fb7b2ffd700 1 -- 172.21.15.16:0/2537506294 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7ac00b9c0 con 0x7fb7b40cf1c0 2023-05-20T01:25:12.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.860+0000 7fb7b2ffd700 1 -- 172.21.15.16:0/2537506294 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7ac00bd70 con 0x7fb7b40cf1c0 2023-05-20T01:25:12.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.861+0000 7fb7ba9d6700 1 -- 172.21.15.16:0/2537506294 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7b40cf1c0 msgr2=0x7fb7b415e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:12.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.861+0000 7fb7ba9d6700 1 --2- 172.21.15.16:0/2537506294 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7b40cf1c0 0x7fb7b415e4e0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fb7b4072000 tx=0x7fb7ac00b330 comp rx=0 tx=0).stop 2023-05-20T01:25:12.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.861+0000 7fb7ba9d6700 1 -- 172.21.15.16:0/2537506294 shutdown_connections 2023-05-20T01:25:12.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.861+0000 7fb7ba9d6700 1 --2- 172.21.15.16:0/2537506294 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7b40cf1c0 0x7fb7b415e4e0 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:12.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.861+0000 7fb7ba9d6700 1 --2- 172.21.15.16:0/2537506294 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb7b415ab80 0x7fb7b40cda80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:12.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.861+0000 7fb7ba9d6700 1 --2- 172.21.15.16:0/2537506294 >> v2:172.21.15.132:3300/0 conn(0x7fb7b40cdfc0 0x7fb7b40cec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:12.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.861+0000 7fb7ba9d6700 1 -- 172.21.15.16:0/2537506294 >> 172.21.15.16:0/2537506294 conn(0x7fb7b406ae30 msgr2=0x7fb7b409a400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:12.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.861+0000 7fb7ba9d6700 1 -- 172.21.15.16:0/2537506294 shutdown_connections 2023-05-20T01:25:12.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.861+0000 7fb7ba9d6700 1 -- 172.21.15.16:0/2537506294 wait complete. 2023-05-20T01:25:12.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.862+0000 7fb7ba9d6700 1 Processor -- start 2023-05-20T01:25:12.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.863+0000 7fb7ba9d6700 1 -- start start 2023-05-20T01:25:12.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.863+0000 7fb7ba9d6700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb7b40cdfc0 0x7fb7b415c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:12.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.863+0000 7fb7ba9d6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7b40cf1c0 0x7fb7b4155460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:12.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.863+0000 7fb7ba9d6700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb7b415ab80 0x7fb7b41f35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:12.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.863+0000 7fb7ba9d6700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7b415bde0 con 0x7fb7b40cf1c0 2023-05-20T01:25:12.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.863+0000 7fb7ba9d6700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7b41f4040 con 0x7fb7b40cdfc0 2023-05-20T01:25:12.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.863+0000 7fb7ba9d6700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7b41f4320 con 0x7fb7b415ab80 2023-05-20T01:25:12.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.863+0000 7fb7b37fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7b40cf1c0 0x7fb7b4155460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:12.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.864+0000 7fb7b37fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7b40cf1c0 0x7fb7b4155460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36190/0 (socket says 172.21.15.16:36190) 2023-05-20T01:25:12.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.864+0000 7fb7b37fe700 1 -- 172.21.15.16:0/1680472202 learned_addr learned my addr 172.21.15.16:0/1680472202 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:12.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.864+0000 7fb7b8f73700 1 --2- 172.21.15.16:0/1680472202 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb7b415ab80 0x7fb7b41f35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:12.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.864+0000 7fb7b3fff700 1 --2- 172.21.15.16:0/1680472202 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb7b40cdfc0 0x7fb7b415c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:12.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.864+0000 7fb7b37fe700 1 -- 172.21.15.16:0/1680472202 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb7b415ab80 msgr2=0x7fb7b41f35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:12.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.864+0000 7fb7b37fe700 1 --2- 172.21.15.16:0/1680472202 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb7b415ab80 0x7fb7b41f35f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:12.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.864+0000 7fb7b37fe700 1 -- 172.21.15.16:0/1680472202 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb7b40cdfc0 msgr2=0x7fb7b415c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:12.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.864+0000 7fb7b37fe700 1 --2- 172.21.15.16:0/1680472202 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb7b40cdfc0 0x7fb7b415c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:12.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.864+0000 7fb7b37fe700 1 -- 172.21.15.16:0/1680472202 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7ac009b30 con 0x7fb7b40cf1c0 2023-05-20T01:25:12.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.864+0000 7fb7b37fe700 1 --2- 172.21.15.16:0/1680472202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7b40cf1c0 0x7fb7b4155460 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fb79c00d800 tx=0x7fb79c00db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:12.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.864+0000 7fb7b0ff9700 1 -- 172.21.15.16:0/1680472202 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb79c0125f0 con 0x7fb7b40cf1c0 2023-05-20T01:25:12.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.864+0000 7fb7b0ff9700 1 -- 172.21.15.16:0/1680472202 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb79c012750 con 0x7fb7b40cf1c0 2023-05-20T01:25:12.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.864+0000 7fb7b0ff9700 1 -- 172.21.15.16:0/1680472202 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb79c012ae0 con 0x7fb7b40cf1c0 2023-05-20T01:25:12.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.864+0000 7fb7ba9d6700 1 -- 172.21.15.16:0/1680472202 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7b40cbbf0 con 0x7fb7b40cf1c0 2023-05-20T01:25:12.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.864+0000 7fb7ba9d6700 1 -- 172.21.15.16:0/1680472202 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7b40cb910 con 0x7fb7b40cf1c0 2023-05-20T01:25:12.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.865+0000 7fb7ba9d6700 1 -- 172.21.15.16:0/1680472202 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7b40d0e30 con 0x7fb7b40cf1c0 2023-05-20T01:25:12.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.866+0000 7fb7b0ff9700 1 -- 172.21.15.16:0/1680472202 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb79c012c40 con 0x7fb7b40cf1c0 2023-05-20T01:25:12.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.866+0000 7fb7b0ff9700 1 --2- 172.21.15.16:0/1680472202 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb79406ace0 0x7fb79406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:12.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.866+0000 7fb7b0ff9700 1 -- 172.21.15.16:0/1680472202 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fb79c089560 con 0x7fb7b40cf1c0 2023-05-20T01:25:12.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.866+0000 7fb7b3fff700 1 --2- 172.21.15.16:0/1680472202 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb79406ace0 0x7fb79406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:12.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.866+0000 7fb7b3fff700 1 --2- 172.21.15.16:0/1680472202 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb79406ace0 0x7fb79406d190 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fb7b4159fa0 tx=0x7fb7a4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:12.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:12.870+0000 7fb7b0ff9700 1 -- 172.21.15.16:0/1680472202 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fb79c054580 con 0x7fb7b40cf1c0 2023-05-20T01:25:13.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:13.118+0000 7fb7ba9d6700 1 -- 172.21.15.16:0/1680472202 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fb7b415b8b0 con 0x7fb7b40cf1c0 2023-05-20T01:25:14.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:13.999+0000 7fb7b0ff9700 1 -- 172.21.15.16:0/1680472202 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v56) v1 ==== 118+0+0 (secure 0 0 0) 0x7fb79c057ba0 con 0x7fb7b40cf1c0 2023-05-20T01:25:14.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:14.002+0000 7fb7ba9d6700 1 -- 172.21.15.16:0/1680472202 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb79406ace0 msgr2=0x7fb79406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:14.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:14.002+0000 7fb7ba9d6700 1 --2- 172.21.15.16:0/1680472202 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb79406ace0 0x7fb79406d190 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fb7b4159fa0 tx=0x7fb7a4008040 comp rx=0 tx=0).stop 2023-05-20T01:25:14.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:14.002+0000 7fb7ba9d6700 1 -- 172.21.15.16:0/1680472202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7b40cf1c0 msgr2=0x7fb7b4155460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:14.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:14.002+0000 7fb7ba9d6700 1 --2- 172.21.15.16:0/1680472202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7b40cf1c0 0x7fb7b4155460 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fb79c00d800 tx=0x7fb79c00db10 comp rx=0 tx=0).stop 2023-05-20T01:25:14.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:14.003+0000 7fb7ba9d6700 1 -- 172.21.15.16:0/1680472202 shutdown_connections 2023-05-20T01:25:14.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:14.003+0000 7fb7ba9d6700 1 --2- 172.21.15.16:0/1680472202 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb79406ace0 0x7fb79406d190 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:14.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:14.003+0000 7fb7ba9d6700 1 --2- 172.21.15.16:0/1680472202 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb7b415ab80 0x7fb7b41f35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:14.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:14.003+0000 7fb7ba9d6700 1 --2- 172.21.15.16:0/1680472202 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb7b40cdfc0 0x7fb7b415c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:14.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:14.003+0000 7fb7ba9d6700 1 --2- 172.21.15.16:0/1680472202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7b40cf1c0 0x7fb7b4155460 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:14.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:14.003+0000 7fb7ba9d6700 1 -- 172.21.15.16:0/1680472202 >> 172.21.15.16:0/1680472202 conn(0x7fb7b406ae30 msgr2=0x7fb7b40b5ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:14.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:14.003+0000 7fb7ba9d6700 1 -- 172.21.15.16:0/1680472202 shutdown_connections 2023-05-20T01:25:14.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:14.003+0000 7fb7ba9d6700 1 -- 172.21.15.16:0/1680472202 wait complete. 2023-05-20T01:25:14.022 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-05-20T01:25:14.023 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-05-20T01:25:14.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:14.154+0000 7f2eabbfb700 1 Processor -- start 2023-05-20T01:25:14.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:14.154+0000 7f2eabbfb700 1 -- start start 2023-05-20T01:25:14.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:14.155+0000 7f2eabbfb700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f2ea40717c0 0x7f2ea40ca870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:14.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:14.155+0000 7f2eabbfb700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ea40ccd30 con 0x7f2ea4159b50 2023-05-20T01:25:14.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:14.155+0000 7f2eabbfb700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ea40ce990 con 0x7f2ea40cd1c0 2023-05-20T01:25:14.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:14.155+0000 7f2eabbfb700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ea40cc6e0 con 0x7f2ea40717c0 2023-05-20T01:25:14.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:14.155+0000 7f2ea9997700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f2ea4159b50 0x7f2ea406e0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:47724/0 (socket says 172.21.15.16:47724) 2023-05-20T01:25:14.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:14.155+0000 7f2ea9997700 1 -- 172.21.15.16:0/981341429 learned_addr learned my addr 172.21.15.16:0/981341429 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:25:14.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:14.155+0000 7f2ea8995700 1 -- 172.21.15.16:0/981341429 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3432137329 0 0) 0x7f2ea40ccd30 con 0x7f2ea4159b50 2023-05-20T01:25:14.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:14.155+0000 7f2eaa198700 1 --2- 172.21.15.16:0/981341429 >> v2:172.21.15.132:3300/0 conn(0x7f2ea40717c0 0x7f2ea40ca870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:14.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:14.155+0000 7f2ea8995700 1 -- 172.21.15.16:0/981341429 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e900035b0 con 0x7f2ea4159b50 2023-05-20T01:25:14.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:14.155+0000 7f2ea8995700 1 -- 172.21.15.16:0/981341429 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1628524387 0 0) 0x7f2e900035b0 con 0x7f2ea4159b50 2023-05-20T01:25:14.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:14.155+0000 7f2ea8995700 1 -- 172.21.15.16:0/981341429 >> v2:172.21.15.132:3300/0 conn(0x7f2ea40717c0 msgr2=0x7f2ea40ca870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:14.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:14.155+0000 7f2ea8995700 1 --2- 172.21.15.16:0/981341429 >> v2:172.21.15.132:3300/0 conn(0x7f2ea40717c0 0x7f2ea40ca870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:14.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:14.155+0000 7f2ea8995700 1 -- 172.21.15.16:0/981341429 >> v1:172.21.15.132:6789/0 conn(0x7f2ea40cd1c0 legacy=0x7f2ea40710b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:14.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:14.155+0000 7f2ea8995700 1 -- 172.21.15.16:0/981341429 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ea40cc9c0 con 0x7f2ea4159b50 2023-05-20T01:25:14.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:14.155+0000 7f2ea8995700 1 -- 172.21.15.16:0/981341429 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7f2ea00017f0 con 0x7f2ea4159b50 2023-05-20T01:25:14.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:14.156+0000 7f2ea8995700 1 -- 172.21.15.16:0/981341429 >> v1:172.21.15.16:6789/0 conn(0x7f2ea4159b50 legacy=0x7f2ea406e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:14.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:14.156+0000 7f2ea8995700 1 --2- 172.21.15.16:0/981341429 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2e90003a00 0x7f2e90003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:14.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:14.156+0000 7f2ea8995700 1 --2- 172.21.15.16:0/981341429 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2e90004850 0x7f2e90004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:14.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:14.156+0000 7f2ea8995700 1 --2- 172.21.15.16:0/981341429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e90005200 0x7f2e9000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:14.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:14.156+0000 7f2ea8995700 1 -- 172.21.15.16:0/981341429 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea40cc6e0 con 0x7f2e90005200 2023-05-20T01:25:14.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:14.156+0000 7f2ea8995700 1 -- 172.21.15.16:0/981341429 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e9000c080 con 0x7f2e90003a00 2023-05-20T01:25:14.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:14.156+0000 7f2ea8995700 1 -- 172.21.15.16:0/981341429 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e9000c240 con 0x7f2e90004850 2023-05-20T01:25:14.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:14.156+0000 7f2eaa198700 1 --2- 172.21.15.16:0/981341429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e90005200 0x7f2e9000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:14.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:14.156+0000 7f2ea9997700 1 --2- 172.21.15.16:0/981341429 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2e90003a00 0x7f2e90003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:14.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:14.156+0000 7f2ea9196700 1 --2- 172.21.15.16:0/981341429 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2e90004850 0x7f2e90004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:14.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:14.156+0000 7f2eaa198700 1 -- 172.21.15.16:0/981341429 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2e90004850 msgr2=0x7f2e90004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:14.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:14.156+0000 7f2eaa198700 1 --2- 172.21.15.16:0/981341429 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2e90004850 0x7f2e90004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:14.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:14.156+0000 7f2eaa198700 1 -- 172.21.15.16:0/981341429 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2e90003a00 msgr2=0x7f2e90003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:14.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:14.156+0000 7f2eaa198700 1 --2- 172.21.15.16:0/981341429 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2e90003a00 0x7f2e90003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:14.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:14.156+0000 7f2eaa198700 1 -- 172.21.15.16:0/981341429 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e900035b0 con 0x7f2e90005200 2023-05-20T01:25:14.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:14.156+0000 7f2ea9196700 1 --2- 172.21.15.16:0/981341429 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2e90004850 0x7f2e90004cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:25:14.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:14.156+0000 7f2eaa198700 1 -- 172.21.15.16:0/981341429 reap_dead start 2023-05-20T01:25:14.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:14.156+0000 7f2eaa198700 1 --2- 172.21.15.16:0/981341429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e90005200 0x7f2e9000ba10 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f2ea415e580 tx=0x7f2e90005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2125e4d9f9452df server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:14.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:14.156+0000 7f2ea8995700 1 -- 172.21.15.16:0/981341429 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e9c018070 con 0x7f2e90005200 2023-05-20T01:25:14.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:14.156+0000 7f2ea8995700 1 -- 172.21.15.16:0/981341429 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e9c00f040 con 0x7f2e90005200 2023-05-20T01:25:14.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:14.156+0000 7f2ea8995700 1 -- 172.21.15.16:0/981341429 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e9c00ec10 con 0x7f2e90005200 2023-05-20T01:25:14.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:14.157+0000 7f2eabbfb700 1 -- 172.21.15.16:0/981341429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e90005200 msgr2=0x7f2e9000ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:14.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:14.157+0000 7f2eabbfb700 1 --2- 172.21.15.16:0/981341429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e90005200 0x7f2e9000ba10 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f2ea415e580 tx=0x7f2e90005880 comp rx=0 tx=0).stop 2023-05-20T01:25:14.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:14.157+0000 7f2eabbfb700 1 -- 172.21.15.16:0/981341429 shutdown_connections 2023-05-20T01:25:14.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:14.157+0000 7f2eabbfb700 1 --2- 172.21.15.16:0/981341429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e90005200 0x7f2e9000ba10 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:14.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:14.157+0000 7f2eabbfb700 1 -- 172.21.15.16:0/981341429 >> 172.21.15.16:0/981341429 conn(0x7f2ea40cffd0 msgr2=0x7f2ea40d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:14.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:14.157+0000 7f2eabbfb700 1 -- 172.21.15.16:0/981341429 shutdown_connections 2023-05-20T01:25:14.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:14.157+0000 7f2eabbfb700 1 -- 172.21.15.16:0/981341429 wait complete. 2023-05-20T01:25:14.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:14.158+0000 7f2eabbfb700 1 Processor -- start 2023-05-20T01:25:14.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:14.158+0000 7f2eabbfb700 1 -- start start 2023-05-20T01:25:14.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:14.159+0000 7f2eabbfb700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2e90005200 0x7f2ea40af140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:14.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:14.159+0000 7f2eabbfb700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2ea40adb50 0x7f2ea4152620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:14.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:14.159+0000 7f2eabbfb700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ea40c7ab0 0x7f2ea40b4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:14.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:14.159+0000 7f2eabbfb700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea4152b60 con 0x7f2ea40c7ab0 2023-05-20T01:25:14.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:14.159+0000 7f2eabbfb700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea40ae0a0 con 0x7f2ea40adb50 2023-05-20T01:25:14.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:14.159+0000 7f2eabbfb700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea40ae380 con 0x7f2e90005200 2023-05-20T01:25:14.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:14.159+0000 7f2eaa198700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ea40c7ab0 0x7f2ea40b4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:14.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:14.159+0000 7f2eaa198700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ea40c7ab0 0x7f2ea40b4430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36212/0 (socket says 172.21.15.16:36212) 2023-05-20T01:25:14.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:14.159+0000 7f2eaa198700 1 -- 172.21.15.16:0/186457649 learned_addr learned my addr 172.21.15.16:0/186457649 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:14.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:14.159+0000 7f2ea9997700 1 --2- 172.21.15.16:0/186457649 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2e90005200 0x7f2ea40af140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:14.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:14.159+0000 7f2ea9196700 1 --2- 172.21.15.16:0/186457649 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2ea40adb50 0x7f2ea4152620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:14.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:14.159+0000 7f2eaa198700 1 -- 172.21.15.16:0/186457649 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2e90005200 msgr2=0x7f2ea40af140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:14.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:14.159+0000 7f2eaa198700 1 --2- 172.21.15.16:0/186457649 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2e90005200 0x7f2ea40af140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:14.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:14.159+0000 7f2eaa198700 1 -- 172.21.15.16:0/186457649 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2ea40adb50 msgr2=0x7f2ea4152620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:14.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:14.159+0000 7f2eaa198700 1 --2- 172.21.15.16:0/186457649 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2ea40adb50 0x7f2ea4152620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:14.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:14.159+0000 7f2eaa198700 1 -- 172.21.15.16:0/186457649 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ea40cec70 con 0x7f2ea40c7ab0 2023-05-20T01:25:14.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:14.159+0000 7f2ea9196700 1 --2- 172.21.15.16:0/186457649 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2ea40adb50 0x7f2ea4152620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:25:14.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:14.159+0000 7f2ea9997700 1 --2- 172.21.15.16:0/186457649 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2e90005200 0x7f2ea40af140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:25:14.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:14.160+0000 7f2eaa198700 1 --2- 172.21.15.16:0/186457649 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ea40c7ab0 0x7f2ea40b4430 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f2ea40b6120 tx=0x7f2e9c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:14.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:14.160+0000 7f2e9a7fc700 1 -- 172.21.15.16:0/186457649 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e9c018040 con 0x7f2ea40c7ab0 2023-05-20T01:25:14.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:14.160+0000 7f2e9a7fc700 1 -- 172.21.15.16:0/186457649 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e9c014e20 con 0x7f2ea40c7ab0 2023-05-20T01:25:14.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:14.160+0000 7f2eabbfb700 1 -- 172.21.15.16:0/186457649 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ea4071e80 con 0x7f2ea40c7ab0 2023-05-20T01:25:14.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:14.160+0000 7f2e9a7fc700 1 -- 172.21.15.16:0/186457649 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e9c007000 con 0x7f2ea40c7ab0 2023-05-20T01:25:14.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:14.160+0000 7f2eabbfb700 1 -- 172.21.15.16:0/186457649 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ea40ce5b0 con 0x7f2ea40c7ab0 2023-05-20T01:25:14.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:14.161+0000 7f2eabbfb700 1 -- 172.21.15.16:0/186457649 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ea40aef90 con 0x7f2ea40c7ab0 2023-05-20T01:25:14.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:14.161+0000 7f2e9a7fc700 1 -- 172.21.15.16:0/186457649 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2e9c0206c0 con 0x7f2ea40c7ab0 2023-05-20T01:25:14.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:14.161+0000 7f2e9a7fc700 1 --2- 172.21.15.16:0/186457649 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f2e8806ace0 0x7f2e8806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:14.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:14.162+0000 7f2e9a7fc700 1 -- 172.21.15.16:0/186457649 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f2e9c08aec0 con 0x7f2ea40c7ab0 2023-05-20T01:25:14.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:14.162+0000 7f2ea9997700 1 --2- 172.21.15.16:0/186457649 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f2e8806ace0 0x7f2e8806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:14.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:14.162+0000 7f2ea9997700 1 --2- 172.21.15.16:0/186457649 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f2e8806ace0 0x7f2e8806d190 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f2ea0000ef0 tx=0x7f2ea0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:14.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:14.166+0000 7f2e9a7fc700 1 -- 172.21.15.16:0/186457649 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f2e9c055ec0 con 0x7f2ea40c7ab0 2023-05-20T01:25:14.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:14.413+0000 7f2eabbfb700 1 -- 172.21.15.16:0/186457649 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f2ea41593f0 con 0x7f2ea40c7ab0 2023-05-20T01:25:15.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.398+0000 7f2e9a7fc700 1 -- 172.21.15.16:0/186457649 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v57) v1 ==== 122+0+0 (secure 0 0 0) 0x7f2e9c0594e0 con 0x7f2ea40c7ab0 2023-05-20T01:25:15.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.401+0000 7f2eabbfb700 1 -- 172.21.15.16:0/186457649 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f2e8806ace0 msgr2=0x7f2e8806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:15.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.401+0000 7f2eabbfb700 1 --2- 172.21.15.16:0/186457649 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f2e8806ace0 0x7f2e8806d190 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f2ea0000ef0 tx=0x7f2ea0009040 comp rx=0 tx=0).stop 2023-05-20T01:25:15.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.401+0000 7f2eabbfb700 1 -- 172.21.15.16:0/186457649 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ea40c7ab0 msgr2=0x7f2ea40b4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:15.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.401+0000 7f2eabbfb700 1 --2- 172.21.15.16:0/186457649 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ea40c7ab0 0x7f2ea40b4430 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f2ea40b6120 tx=0x7f2e9c009750 comp rx=0 tx=0).stop 2023-05-20T01:25:15.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.401+0000 7f2eabbfb700 1 -- 172.21.15.16:0/186457649 shutdown_connections 2023-05-20T01:25:15.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.401+0000 7f2eabbfb700 1 --2- 172.21.15.16:0/186457649 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ea40c7ab0 0x7f2ea40b4430 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:15.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.401+0000 7f2eabbfb700 1 --2- 172.21.15.16:0/186457649 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f2e8806ace0 0x7f2e8806d190 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:15.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.401+0000 7f2eabbfb700 1 --2- 172.21.15.16:0/186457649 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2e90005200 0x7f2ea40af140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:15.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.401+0000 7f2eabbfb700 1 --2- 172.21.15.16:0/186457649 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2ea40adb50 0x7f2ea4152620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:15.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.401+0000 7f2eabbfb700 1 -- 172.21.15.16:0/186457649 >> 172.21.15.16:0/186457649 conn(0x7f2ea40cffd0 msgr2=0x7f2ea40b2270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:15.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.401+0000 7f2eabbfb700 1 -- 172.21.15.16:0/186457649 shutdown_connections 2023-05-20T01:25:15.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.401+0000 7f2eabbfb700 1 -- 172.21.15.16:0/186457649 wait complete. 2023-05-20T01:25:15.421 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-05-20T01:25:15.421 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-20T01:25:15.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.549+0000 7f877b955700 1 Processor -- start 2023-05-20T01:25:15.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.549+0000 7f877b955700 1 -- start start 2023-05-20T01:25:15.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.550+0000 7f877b955700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f877406a510 0x7f877415e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:15.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.550+0000 7f877b955700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f877406a8e0 0x7f87740cf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:15.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.550+0000 7f877b955700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f87740cfb30 0x7f8774071340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:15.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.550+0000 7f877b955700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f877415e060 con 0x7f877406a510 2023-05-20T01:25:15.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.550+0000 7f877b955700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f877415d450 con 0x7f87740cfb30 2023-05-20T01:25:15.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.550+0000 7f877b955700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f877415d730 con 0x7f877406a8e0 2023-05-20T01:25:15.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.550+0000 7f87796f1700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f877406a510 0x7f877415e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:15.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.550+0000 7f87796f1700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f877406a510 0x7f877415e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36216/0 (socket says 172.21.15.16:36216) 2023-05-20T01:25:15.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.550+0000 7f87796f1700 1 -- 172.21.15.16:0/2773052437 learned_addr learned my addr 172.21.15.16:0/2773052437 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:15.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.550+0000 7f8779ef2700 1 --2- 172.21.15.16:0/2773052437 >> v2:172.21.15.132:3300/0 conn(0x7f87740cfb30 0x7f8774071340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:15.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.550+0000 7f8778ef0700 1 --2- 172.21.15.16:0/2773052437 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f877406a8e0 0x7f87740cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:15.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.550+0000 7f87796f1700 1 -- 172.21.15.16:0/2773052437 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f877406a8e0 msgr2=0x7f87740cf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:15.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.550+0000 7f87796f1700 1 --2- 172.21.15.16:0/2773052437 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f877406a8e0 0x7f87740cf5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:15.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.550+0000 7f87796f1700 1 -- 172.21.15.16:0/2773052437 >> v2:172.21.15.132:3300/0 conn(0x7f87740cfb30 msgr2=0x7f8774071340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:15.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.550+0000 7f87796f1700 1 --2- 172.21.15.16:0/2773052437 >> v2:172.21.15.132:3300/0 conn(0x7f87740cfb30 0x7f8774071340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:15.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.550+0000 7f87796f1700 1 -- 172.21.15.16:0/2773052437 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f877415da10 con 0x7f877406a510 2023-05-20T01:25:15.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.550+0000 7f87796f1700 1 --2- 172.21.15.16:0/2773052437 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f877406a510 0x7f877415e2a0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f87740b24b0 tx=0x7f8770009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48cdc81258de5c7a server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:15.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.551+0000 7f876bfff700 1 -- 172.21.15.16:0/2773052437 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f877000b870 con 0x7f877406a510 2023-05-20T01:25:15.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.551+0000 7f876bfff700 1 -- 172.21.15.16:0/2773052437 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f877000b9d0 con 0x7f877406a510 2023-05-20T01:25:15.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.551+0000 7f876bfff700 1 -- 172.21.15.16:0/2773052437 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f877000bd60 con 0x7f877406a510 2023-05-20T01:25:15.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.551+0000 7f877b955700 1 -- 172.21.15.16:0/2773052437 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f877406a510 msgr2=0x7f877415e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:15.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.551+0000 7f877b955700 1 --2- 172.21.15.16:0/2773052437 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f877406a510 0x7f877415e2a0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f87740b24b0 tx=0x7f8770009bd0 comp rx=0 tx=0).stop 2023-05-20T01:25:15.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.551+0000 7f877b955700 1 -- 172.21.15.16:0/2773052437 shutdown_connections 2023-05-20T01:25:15.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.551+0000 7f877b955700 1 --2- 172.21.15.16:0/2773052437 >> v2:172.21.15.132:3300/0 conn(0x7f87740cfb30 0x7f8774071340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:15.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.551+0000 7f877b955700 1 --2- 172.21.15.16:0/2773052437 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f877406a510 0x7f877415e2a0 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:15.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.551+0000 7f877b955700 1 --2- 172.21.15.16:0/2773052437 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f877406a8e0 0x7f87740cf5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:15.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.551+0000 7f877b955700 1 -- 172.21.15.16:0/2773052437 >> 172.21.15.16:0/2773052437 conn(0x7f87740d0c30 msgr2=0x7f87740b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:15.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.551+0000 7f877b955700 1 -- 172.21.15.16:0/2773052437 shutdown_connections 2023-05-20T01:25:15.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.551+0000 7f877b955700 1 -- 172.21.15.16:0/2773052437 wait complete. 2023-05-20T01:25:15.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.552+0000 7f877b955700 1 Processor -- start 2023-05-20T01:25:15.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.553+0000 7f877b955700 1 -- start start 2023-05-20T01:25:15.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.553+0000 7f877b955700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f877406a510 0x7f87740c5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:15.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.553+0000 7f877b955700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f877406a8e0 0x7f87741f6c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:15.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.553+0000 7f877b955700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87740cfb30 0x7f877414e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:15.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.553+0000 7f877b955700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87741f71d0 con 0x7f87740cfb30 2023-05-20T01:25:15.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.553+0000 7f877b955700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f877414e9e0 con 0x7f877406a8e0 2023-05-20T01:25:15.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.553+0000 7f877b955700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87740c40a0 con 0x7f877406a510 2023-05-20T01:25:15.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.553+0000 7f8779ef2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87740cfb30 0x7f877414e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:15.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.553+0000 7f8779ef2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87740cfb30 0x7f877414e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36226/0 (socket says 172.21.15.16:36226) 2023-05-20T01:25:15.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.553+0000 7f8779ef2700 1 -- 172.21.15.16:0/3841906772 learned_addr learned my addr 172.21.15.16:0/3841906772 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:15.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.553+0000 7f87796f1700 1 --2- 172.21.15.16:0/3841906772 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f877406a510 0x7f87740c5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:15.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.553+0000 7f8778ef0700 1 --2- 172.21.15.16:0/3841906772 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f877406a8e0 0x7f87741f6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:15.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.554+0000 7f8779ef2700 1 -- 172.21.15.16:0/3841906772 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f877406a510 msgr2=0x7f87740c5140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:15.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.554+0000 7f8779ef2700 1 --2- 172.21.15.16:0/3841906772 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f877406a510 0x7f87740c5140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:15.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.554+0000 7f8779ef2700 1 -- 172.21.15.16:0/3841906772 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f877406a8e0 msgr2=0x7f87741f6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:15.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.554+0000 7f8779ef2700 1 --2- 172.21.15.16:0/3841906772 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f877406a8e0 0x7f87741f6c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:15.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.554+0000 7f8779ef2700 1 -- 172.21.15.16:0/3841906772 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8770009550 con 0x7f87740cfb30 2023-05-20T01:25:15.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.554+0000 7f8779ef2700 1 --2- 172.21.15.16:0/3841906772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87740cfb30 0x7f877414e4a0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f876c007e40 tx=0x7f876c00e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:15.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.554+0000 7f8769ffb700 1 -- 172.21.15.16:0/3841906772 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f876c00ed30 con 0x7f87740cfb30 2023-05-20T01:25:15.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.554+0000 7f8769ffb700 1 -- 172.21.15.16:0/3841906772 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f876c014420 con 0x7f87740cfb30 2023-05-20T01:25:15.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.554+0000 7f877b955700 1 -- 172.21.15.16:0/3841906772 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f877406ad50 con 0x7f87740cfb30 2023-05-20T01:25:15.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.554+0000 7f8769ffb700 1 -- 172.21.15.16:0/3841906772 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f876c0147d0 con 0x7f87740cfb30 2023-05-20T01:25:15.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.554+0000 7f877b955700 1 -- 172.21.15.16:0/3841906772 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f877415dcf0 con 0x7f87740cfb30 2023-05-20T01:25:15.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.555+0000 7f877b955700 1 -- 172.21.15.16:0/3841906772 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f877415d080 con 0x7f87740cfb30 2023-05-20T01:25:15.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.556+0000 7f8769ffb700 1 -- 172.21.15.16:0/3841906772 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f876c014930 con 0x7f87740cfb30 2023-05-20T01:25:15.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.556+0000 7f8769ffb700 1 --2- 172.21.15.16:0/3841906772 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f875806ace0 0x7f875806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:15.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.556+0000 7f8769ffb700 1 -- 172.21.15.16:0/3841906772 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f876c089cb0 con 0x7f87740cfb30 2023-05-20T01:25:15.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.556+0000 7f87796f1700 1 --2- 172.21.15.16:0/3841906772 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f875806ace0 0x7f875806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:15.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.557+0000 7f87796f1700 1 --2- 172.21.15.16:0/3841906772 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f875806ace0 0x7f875806d190 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f8770014040 tx=0x7f8770008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:15.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.561+0000 7f8769ffb700 1 -- 172.21.15.16:0/3841906772 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f876c054cd0 con 0x7f87740cfb30 2023-05-20T01:25:15.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.804+0000 7f877b955700 1 -- 172.21.15.16:0/3841906772 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f87740747b0 con 0x7f87740cfb30 2023-05-20T01:25:15.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.804+0000 7f8769ffb700 1 -- 172.21.15.16:0/3841906772 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+4673 (secure 0 0 0) 0x7f876c0582f0 con 0x7f87740cfb30 2023-05-20T01:25:15.809 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:25:15.809 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":57,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6077,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/1241894038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":1241894038},{"type":"v1","addr":"172.21.15.132:6835","nonce":1241894038}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":6562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/3771738903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3771738903},{"type":"v1","addr":"172.21.15.132:6834","nonce":3771738903}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":7101,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/3876038711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":3876038711},{"type":"v1","addr":"172.21.15.16:6836","nonce":3876038711}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-20T01:25:11.987993+0000","modified":"2023-05-20T01:25:15.395667+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7104},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7104":{"gid":7104,"name":"c","rank":0,"incarnation":53,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6837/28561908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":28561908},{"type":"v1","addr":"172.21.15.16:6837","nonce":28561908}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-05-20T01:25:15.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.806+0000 7f877b955700 1 -- 172.21.15.16:0/3841906772 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f875806ace0 msgr2=0x7f875806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:15.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.806+0000 7f877b955700 1 --2- 172.21.15.16:0/3841906772 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f875806ace0 0x7f875806d190 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f8770014040 tx=0x7f8770008030 comp rx=0 tx=0).stop 2023-05-20T01:25:15.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.806+0000 7f877b955700 1 -- 172.21.15.16:0/3841906772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87740cfb30 msgr2=0x7f877414e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:15.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.806+0000 7f877b955700 1 --2- 172.21.15.16:0/3841906772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87740cfb30 0x7f877414e4a0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f876c007e40 tx=0x7f876c00e3c0 comp rx=0 tx=0).stop 2023-05-20T01:25:15.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.806+0000 7f877b955700 1 -- 172.21.15.16:0/3841906772 shutdown_connections 2023-05-20T01:25:15.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.806+0000 7f877b955700 1 --2- 172.21.15.16:0/3841906772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87740cfb30 0x7f877414e4a0 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:15.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.806+0000 7f877b955700 1 --2- 172.21.15.16:0/3841906772 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f875806ace0 0x7f875806d190 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:15.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.806+0000 7f877b955700 1 --2- 172.21.15.16:0/3841906772 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f877406a510 0x7f87740c5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:15.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.806+0000 7f877b955700 1 --2- 172.21.15.16:0/3841906772 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f877406a8e0 0x7f87741f6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:15.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.806+0000 7f877b955700 1 -- 172.21.15.16:0/3841906772 >> 172.21.15.16:0/3841906772 conn(0x7f87740d0c30 msgr2=0x7f87740c9090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:15.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.806+0000 7f877b955700 1 -- 172.21.15.16:0/3841906772 shutdown_connections 2023-05-20T01:25:15.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.806+0000 7f877b955700 1 -- 172.21.15.16:0/3841906772 wait complete. 2023-05-20T01:25:15.814 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 57 2023-05-20T01:25:15.823 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-20T01:25:15.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.944+0000 7f5ad0ac5700 1 Processor -- start 2023-05-20T01:25:15.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.944+0000 7f5ad0ac5700 1 -- start start 2023-05-20T01:25:15.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.945+0000 7f5ad0ac5700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5acc0caaa0 0x7f5acc0cea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:15.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.945+0000 7f5ad0ac5700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5acc159d90 con 0x7f5acc0cb670 2023-05-20T01:25:15.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.945+0000 7f5ad0ac5700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5acc0cdb80 con 0x7f5acc1599c0 2023-05-20T01:25:15.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.945+0000 7f5ad0ac5700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5acc0cdf20 con 0x7f5acc0caaa0 2023-05-20T01:25:15.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.945+0000 7f5ac9d9b700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f5acc0cb670 0x7f5acc0ce340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:47758/0 (socket says 172.21.15.16:47758) 2023-05-20T01:25:15.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.945+0000 7f5ac9d9b700 1 -- 172.21.15.16:0/983911119 learned_addr learned my addr 172.21.15.16:0/983911119 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:25:15.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.945+0000 7f5ac959a700 1 -- 172.21.15.16:0/983911119 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3871945183 0 0) 0x7f5abc001110 con 0x7f5acc0cb670 2023-05-20T01:25:15.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.945+0000 7f5acad9d700 1 --2- 172.21.15.16:0/983911119 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5acc0caaa0 0x7f5acc0cea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:15.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.945+0000 7f5ac959a700 1 -- 172.21.15.16:0/983911119 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ab40035b0 con 0x7f5acc0cb670 2023-05-20T01:25:15.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.946+0000 7f5ac959a700 1 -- 172.21.15.16:0/983911119 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3493647369 0 0) 0x7f5ab40035b0 con 0x7f5acc0cb670 2023-05-20T01:25:15.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.946+0000 7f5ac959a700 1 -- 172.21.15.16:0/983911119 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5acc0caaa0 msgr2=0x7f5acc0cea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:15.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.946+0000 7f5ac959a700 1 --2- 172.21.15.16:0/983911119 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5acc0caaa0 0x7f5acc0cea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:15.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.946+0000 7f5ac959a700 1 -- 172.21.15.16:0/983911119 >> v1:172.21.15.132:6789/0 conn(0x7f5acc1599c0 legacy=0x7f5acc0caf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:15.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.946+0000 7f5ac959a700 1 -- 172.21.15.16:0/983911119 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5acc06eaf0 con 0x7f5acc0cb670 2023-05-20T01:25:15.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.946+0000 7f5ac959a700 1 -- 172.21.15.16:0/983911119 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7f5abc001800 con 0x7f5acc0cb670 2023-05-20T01:25:15.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.946+0000 7f5ac959a700 1 -- 172.21.15.16:0/983911119 >> v1:172.21.15.16:6789/0 conn(0x7f5acc0cb670 legacy=0x7f5acc0ce340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:15.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.946+0000 7f5ac959a700 1 --2- 172.21.15.16:0/983911119 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5acc0caaa0 0x7f5ab4003aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:15.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.946+0000 7f5ac959a700 1 --2- 172.21.15.16:0/983911119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ab4004020 0x7f5ab4004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:15.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.946+0000 7f5ac959a700 1 --2- 172.21.15.16:0/983911119 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5ab4004d90 0x7f5ab4005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:15.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.946+0000 7f5ac959a700 1 -- 172.21.15.16:0/983911119 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5acc0cdf20 con 0x7f5ab4004020 2023-05-20T01:25:15.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.946+0000 7f5ac959a700 1 -- 172.21.15.16:0/983911119 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab400bb50 con 0x7f5ab4004d90 2023-05-20T01:25:15.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.946+0000 7f5ac959a700 1 -- 172.21.15.16:0/983911119 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab400bd10 con 0x7f5acc0caaa0 2023-05-20T01:25:15.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.946+0000 7f5ac9d9b700 1 --2- 172.21.15.16:0/983911119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ab4004020 0x7f5ab4004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:15.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.946+0000 7f5acad9d700 1 --2- 172.21.15.16:0/983911119 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5ab4004d90 0x7f5ab4005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:15.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.946+0000 7f5aca59c700 1 --2- 172.21.15.16:0/983911119 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5acc0caaa0 0x7f5ab4003aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:15.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.946+0000 7f5ac9d9b700 1 -- 172.21.15.16:0/983911119 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5acc0caaa0 msgr2=0x7f5ab4003aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:15.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.946+0000 7f5ac9d9b700 1 --2- 172.21.15.16:0/983911119 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5acc0caaa0 0x7f5ab4003aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:15.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.946+0000 7f5ac9d9b700 1 -- 172.21.15.16:0/983911119 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5ab4004d90 msgr2=0x7f5ab4005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:15.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.946+0000 7f5ac9d9b700 1 --2- 172.21.15.16:0/983911119 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5ab4004d90 0x7f5ab4005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:15.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.946+0000 7f5ac9d9b700 1 -- 172.21.15.16:0/983911119 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ab40035b0 con 0x7f5ab4004020 2023-05-20T01:25:15.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.946+0000 7f5ac9d9b700 1 --2- 172.21.15.16:0/983911119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ab4004020 0x7f5ab4004850 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f5abc00c240 tx=0x7f5ab4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8befac3f986ea3f0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:15.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.946+0000 7f5ac959a700 1 -- 172.21.15.16:0/983911119 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5abc007db0 con 0x7f5ab4004020 2023-05-20T01:25:15.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.946+0000 7f5ac959a700 1 -- 172.21.15.16:0/983911119 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5abc00ce00 con 0x7f5ab4004020 2023-05-20T01:25:15.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.947+0000 7f5ac959a700 1 -- 172.21.15.16:0/983911119 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5abc005ba0 con 0x7f5ab4004020 2023-05-20T01:25:15.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.947+0000 7f5ad0ac5700 1 -- 172.21.15.16:0/983911119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ab4004020 msgr2=0x7f5ab4004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:15.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.947+0000 7f5ad0ac5700 1 --2- 172.21.15.16:0/983911119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ab4004020 0x7f5ab4004850 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f5abc00c240 tx=0x7f5ab4005880 comp rx=0 tx=0).stop 2023-05-20T01:25:15.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.947+0000 7f5acad9d700 1 -- 172.21.15.16:0/983911119 reap_dead start 2023-05-20T01:25:15.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.947+0000 7f5ad0ac5700 1 -- 172.21.15.16:0/983911119 shutdown_connections 2023-05-20T01:25:15.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.947+0000 7f5ad0ac5700 1 -- 172.21.15.16:0/983911119 >> 172.21.15.16:0/983911119 conn(0x7f5acc0d0c30 msgr2=0x7f5acc0b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:15.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.947+0000 7f5ad0ac5700 1 -- 172.21.15.16:0/983911119 shutdown_connections 2023-05-20T01:25:15.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.947+0000 7f5ad0ac5700 1 -- 172.21.15.16:0/983911119 wait complete. 2023-05-20T01:25:15.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.949+0000 7f5ad0ac5700 1 Processor -- start 2023-05-20T01:25:15.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.949+0000 7f5ad0ac5700 1 -- start start 2023-05-20T01:25:15.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.949+0000 7f5ad0ac5700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5ab4004020 0x7f5acc0c4190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:15.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.949+0000 7f5ad0ac5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5acc150890 0x7f5acc150220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:15.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.949+0000 7f5ad0ac5700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5acc0c5c80 0x7f5acc0c60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:15.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.949+0000 7f5ad0ac5700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5acc0c3b40 con 0x7f5acc150890 2023-05-20T01:25:15.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.949+0000 7f5ad0ac5700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5acc0c4850 con 0x7f5acc0c5c80 2023-05-20T01:25:15.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.949+0000 7f5ad0ac5700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5acc1572e0 con 0x7f5ab4004020 2023-05-20T01:25:15.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.949+0000 7f5ac9d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5acc150890 0x7f5acc150220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:15.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.949+0000 7f5ac9d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5acc150890 0x7f5acc150220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36242/0 (socket says 172.21.15.16:36242) 2023-05-20T01:25:15.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.949+0000 7f5ac9d9b700 1 -- 172.21.15.16:0/1228489570 learned_addr learned my addr 172.21.15.16:0/1228489570 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:15.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.950+0000 7f5aca59c700 1 --2- 172.21.15.16:0/1228489570 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5ab4004020 0x7f5acc0c4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:15.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.950+0000 7f5acad9d700 1 --2- 172.21.15.16:0/1228489570 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5acc0c5c80 0x7f5acc0c60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:15.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.950+0000 7f5ac9d9b700 1 -- 172.21.15.16:0/1228489570 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5ab4004020 msgr2=0x7f5acc0c4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:15.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.950+0000 7f5ac9d9b700 1 --2- 172.21.15.16:0/1228489570 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5ab4004020 0x7f5acc0c4190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:15.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.950+0000 7f5ac9d9b700 1 -- 172.21.15.16:0/1228489570 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5acc0c5c80 msgr2=0x7f5acc0c60f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:15.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.950+0000 7f5ac9d9b700 1 --2- 172.21.15.16:0/1228489570 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5acc0c5c80 0x7f5acc0c60f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:15.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.950+0000 7f5ac9d9b700 1 -- 172.21.15.16:0/1228489570 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5acc06f0b0 con 0x7f5acc150890 2023-05-20T01:25:15.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.950+0000 7f5acad9d700 1 --2- 172.21.15.16:0/1228489570 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5acc0c5c80 0x7f5acc0c60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:25:15.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.950+0000 7f5aca59c700 1 --2- 172.21.15.16:0/1228489570 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5ab4004020 0x7f5acc0c4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:25:15.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.950+0000 7f5ac9d9b700 1 --2- 172.21.15.16:0/1228489570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5acc150890 0x7f5acc150220 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f5abc000c00 tx=0x7f5abc001b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:15.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.950+0000 7f5abaffd700 1 -- 172.21.15.16:0/1228489570 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5abc021070 con 0x7f5acc150890 2023-05-20T01:25:15.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.950+0000 7f5abaffd700 1 -- 172.21.15.16:0/1228489570 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5abc001c00 con 0x7f5acc150890 2023-05-20T01:25:15.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.950+0000 7f5ad0ac5700 1 -- 172.21.15.16:0/1228489570 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5acc06edd0 con 0x7f5acc150890 2023-05-20T01:25:15.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.950+0000 7f5abaffd700 1 -- 172.21.15.16:0/1228489570 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5abc004ca0 con 0x7f5acc150890 2023-05-20T01:25:15.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.950+0000 7f5ad0ac5700 1 -- 172.21.15.16:0/1228489570 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5acc06e750 con 0x7f5acc150890 2023-05-20T01:25:15.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.951+0000 7f5ad0ac5700 1 -- 172.21.15.16:0/1228489570 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5acc0c7f60 con 0x7f5acc150890 2023-05-20T01:25:15.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.952+0000 7f5abaffd700 1 -- 172.21.15.16:0/1228489570 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5abc0070d0 con 0x7f5acc150890 2023-05-20T01:25:15.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.952+0000 7f5abaffd700 1 --2- 172.21.15.16:0/1228489570 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f5aac06ace0 0x7f5aac06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:15.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.952+0000 7f5abaffd700 1 -- 172.21.15.16:0/1228489570 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f5abc08bcf0 con 0x7f5acc150890 2023-05-20T01:25:15.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.952+0000 7f5aca59c700 1 --2- 172.21.15.16:0/1228489570 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f5aac06ace0 0x7f5aac06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:15.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.953+0000 7f5aca59c700 1 --2- 172.21.15.16:0/1228489570 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f5aac06ace0 0x7f5aac06d190 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f5ac0000c00 tx=0x7f5ac0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:15.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:15.957+0000 7f5abaffd700 1 -- 172.21.15.16:0/1228489570 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f5abc05a700 con 0x7f5acc150890 2023-05-20T01:25:16.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.158+0000 7f5ad0ac5700 1 -- 172.21.15.16:0/1228489570 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5acc159d90 con 0x7f5acc150890 2023-05-20T01:25:16.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.160+0000 7f5abaffd700 1 -- 172.21.15.16:0/1228489570 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+18453 (secure 0 0 0) 0x7f5acc159d90 con 0x7f5acc150890 2023-05-20T01:25:16.164 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:25:16.164 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":88,"fsid":"c2245cc5-ecab-4087-a59b-bc81db901676","created":"2023-05-20T01:16:09.549230+0000","modified":"2023-05-20T01:25:12.989737+0000","last_up_change":"2023-05-20T01:16:19.788624+0000","last_in_change":"2023-05-20T01:16:12.092160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-05-20T01:16:20.934366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-05-20T01:16:21.372294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-05-20T01:25:09.967348+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-05-20T01:25:10.956221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"317a83cd-19c8-4958-a301-6087ef342289","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6825","nonce":3923634179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6827","nonce":3923634179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6831","nonce":3923634179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6829","nonce":3923634179}]},"public_addr":"172.21.15.16:6825/3923634179","cluster_addr":"172.21.15.16:6827/3923634179","heartbeat_back_addr":"172.21.15.16:6831/3923634179","heartbeat_front_addr":"172.21.15.16:6829/3923634179","state":["exists","up"]},{"osd":1,"uuid":"5581046f-8219-4089-8d7d-b7846399505e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6809","nonce":3860264220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6811","nonce":3860264220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6815","nonce":3860264220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6813","nonce":3860264220}]},"public_addr":"172.21.15.16:6809/3860264220","cluster_addr":"172.21.15.16:6811/3860264220","heartbeat_back_addr":"172.21.15.16:6815/3860264220","heartbeat_front_addr":"172.21.15.16:6813/3860264220","state":["exists","up"]},{"osd":2,"uuid":"072e9e66-4133-478e-8d4a-64d46c2963f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6817","nonce":293187030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6819","nonce":293187030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6823","nonce":293187030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6821","nonce":293187030}]},"public_addr":"172.21.15.16:6817/293187030","cluster_addr":"172.21.15.16:6819/293187030","heartbeat_back_addr":"172.21.15.16:6823/293187030","heartbeat_front_addr":"172.21.15.16:6821/293187030","state":["exists","up"]},{"osd":3,"uuid":"11e1bba6-00f8-48dd-9028-d8d716b7f145","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6801","nonce":3669781890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6803","nonce":3669781890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6807","nonce":3669781890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6805","nonce":3669781890}]},"public_addr":"172.21.15.16:6801/3669781890","cluster_addr":"172.21.15.16:6803/3669781890","heartbeat_back_addr":"172.21.15.16:6807/3669781890","heartbeat_front_addr":"172.21.15.16:6805/3669781890","state":["exists","up"]},{"osd":4,"uuid":"b8909c3a-cf70-46cb-8d0b-009c707cf9d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6819","nonce":506317223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6823","nonce":506317223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6829","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6831","nonce":506317223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6827","nonce":506317223}]},"public_addr":"172.21.15.132:6819/506317223","cluster_addr":"172.21.15.132:6823/506317223","heartbeat_back_addr":"172.21.15.132:6831/506317223","heartbeat_front_addr":"172.21.15.132:6827/506317223","state":["exists","up"]},{"osd":5,"uuid":"b1818304-4583-47bf-a6ec-0bfbb5166233","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6818","nonce":1844689995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6822","nonce":1844689995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6830","nonce":1844689995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6826","nonce":1844689995}]},"public_addr":"172.21.15.132:6818/1844689995","cluster_addr":"172.21.15.132:6822/1844689995","heartbeat_back_addr":"172.21.15.132:6830/1844689995","heartbeat_front_addr":"172.21.15.132:6826/1844689995","state":["exists","up"]},{"osd":6,"uuid":"d37f4691-eed1-4a93-8818-18354f03f4a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6801","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6802","nonce":727581123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6805","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6807","nonce":727581123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6813","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6815","nonce":727581123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6811","nonce":727581123}]},"public_addr":"172.21.15.132:6802/727581123","cluster_addr":"172.21.15.132:6807/727581123","heartbeat_back_addr":"172.21.15.132:6815/727581123","heartbeat_front_addr":"172.21.15.132:6811/727581123","state":["exists","up"]},{"osd":7,"uuid":"4f712f62-5d1d-4816-8042-03209144f8b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6803","nonce":466001149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6806","nonce":466001149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6814","nonce":466001149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6810","nonce":466001149}]},"public_addr":"172.21.15.132:6803/466001149","cluster_addr":"172.21.15.132:6806/466001149","heartbeat_back_addr":"172.21.15.132:6814/466001149","heartbeat_front_addr":"172.21.15.132:6810/466001149","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.338863+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.278680+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305694+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.246835+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.276237+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.304441+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305421+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.267050+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-20T01:25:16.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.161+0000 7f5ad0ac5700 1 -- 172.21.15.16:0/1228489570 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f5aac06ace0 msgr2=0x7f5aac06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:16.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.161+0000 7f5ad0ac5700 1 --2- 172.21.15.16:0/1228489570 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f5aac06ace0 0x7f5aac06d190 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f5ac0000c00 tx=0x7f5ac0009040 comp rx=0 tx=0).stop 2023-05-20T01:25:16.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.161+0000 7f5ad0ac5700 1 -- 172.21.15.16:0/1228489570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5acc150890 msgr2=0x7f5acc150220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:16.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.161+0000 7f5ad0ac5700 1 --2- 172.21.15.16:0/1228489570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5acc150890 0x7f5acc150220 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f5abc000c00 tx=0x7f5abc001b90 comp rx=0 tx=0).stop 2023-05-20T01:25:16.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.162+0000 7f5ad0ac5700 1 -- 172.21.15.16:0/1228489570 shutdown_connections 2023-05-20T01:25:16.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.162+0000 7f5ad0ac5700 1 --2- 172.21.15.16:0/1228489570 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5acc0c5c80 0x7f5acc0c60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:16.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.162+0000 7f5ad0ac5700 1 --2- 172.21.15.16:0/1228489570 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f5aac06ace0 0x7f5aac06d190 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:16.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.162+0000 7f5ad0ac5700 1 --2- 172.21.15.16:0/1228489570 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5ab4004020 0x7f5acc0c4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:16.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.162+0000 7f5ad0ac5700 1 --2- 172.21.15.16:0/1228489570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5acc150890 0x7f5acc150220 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:16.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.162+0000 7f5ad0ac5700 1 -- 172.21.15.16:0/1228489570 >> 172.21.15.16:0/1228489570 conn(0x7f5acc0d0c30 msgr2=0x7f5acc0b24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:16.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.162+0000 7f5ad0ac5700 1 -- 172.21.15.16:0/1228489570 shutdown_connections 2023-05-20T01:25:16.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.162+0000 7f5ad0ac5700 1 -- 172.21.15.16:0/1228489570 wait complete. 2023-05-20T01:25:16.178 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-05-20T01:25:16.179 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-05-20T01:25:16.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.299+0000 7f2192087700 1 Processor -- start 2023-05-20T01:25:16.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.299+0000 7f2192087700 1 -- start start 2023-05-20T01:25:16.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.300+0000 7f2192087700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f218c0cb670 0x7f218c0ce340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:16.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.300+0000 7f2192087700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f218c159d90 con 0x7f218c0caaa0 2023-05-20T01:25:16.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.300+0000 7f2192087700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f218c0cdb80 con 0x7f218c1599c0 2023-05-20T01:25:16.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.300+0000 7f2192087700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f218c0cdf20 con 0x7f218c0cb670 2023-05-20T01:25:16.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.300+0000 7f218bfff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f218c0caaa0 0x7f218c0ce880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:47780/0 (socket says 172.21.15.16:47780) 2023-05-20T01:25:16.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.300+0000 7f218bfff700 1 -- 172.21.15.16:0/3035038229 learned_addr learned my addr 172.21.15.16:0/3035038229 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:25:16.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.300+0000 7f218affd700 1 --2- 172.21.15.16:0/3035038229 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f218c0cb670 0x7f218c0ce340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:16.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.300+0000 7f218a7fc700 1 -- 172.21.15.16:0/3035038229 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2628843733 0 0) 0x7f2184001490 con 0x7f218c0caaa0 2023-05-20T01:25:16.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.300+0000 7f218a7fc700 1 -- 172.21.15.16:0/3035038229 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21780035b0 con 0x7f218c0caaa0 2023-05-20T01:25:16.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.300+0000 7f218a7fc700 1 -- 172.21.15.16:0/3035038229 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1464555775 0 0) 0x7f21780035b0 con 0x7f218c0caaa0 2023-05-20T01:25:16.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.300+0000 7f218a7fc700 1 -- 172.21.15.16:0/3035038229 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f218c0cb670 msgr2=0x7f218c0ce340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:16.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.300+0000 7f218a7fc700 1 --2- 172.21.15.16:0/3035038229 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f218c0cb670 0x7f218c0ce340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:16.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.300+0000 7f218a7fc700 1 -- 172.21.15.16:0/3035038229 >> v1:172.21.15.132:6789/0 conn(0x7f218c1599c0 legacy=0x7f218c0caf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:16.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.300+0000 7f218a7fc700 1 -- 172.21.15.16:0/3035038229 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f218c06eaf0 con 0x7f218c0caaa0 2023-05-20T01:25:16.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.300+0000 7f218a7fc700 1 -- 172.21.15.16:0/3035038229 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7f2184001b80 con 0x7f218c0caaa0 2023-05-20T01:25:16.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.300+0000 7f218a7fc700 1 -- 172.21.15.16:0/3035038229 >> v1:172.21.15.16:6789/0 conn(0x7f218c0caaa0 legacy=0x7f218c0ce880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:16.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.300+0000 7f218a7fc700 1 --2- 172.21.15.16:0/3035038229 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2178003aa0 0x7f2178003eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:16.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.300+0000 7f218a7fc700 1 --2- 172.21.15.16:0/3035038229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2178004850 0x7f2178004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:16.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.300+0000 7f218a7fc700 1 --2- 172.21.15.16:0/3035038229 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f218c0cb670 0x7f2178005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:16.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.300+0000 7f218a7fc700 1 -- 172.21.15.16:0/3035038229 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f218c0cdf20 con 0x7f2178004850 2023-05-20T01:25:16.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.301+0000 7f218a7fc700 1 -- 172.21.15.16:0/3035038229 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f217800baf0 con 0x7f2178003aa0 2023-05-20T01:25:16.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.301+0000 7f218a7fc700 1 -- 172.21.15.16:0/3035038229 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f217800bcb0 con 0x7f218c0cb670 2023-05-20T01:25:16.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.301+0000 7f218affd700 1 --2- 172.21.15.16:0/3035038229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2178004850 0x7f2178004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:16.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.301+0000 7f218b7fe700 1 --2- 172.21.15.16:0/3035038229 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2178003aa0 0x7f2178003eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:16.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.301+0000 7f218bfff700 1 --2- 172.21.15.16:0/3035038229 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f218c0cb670 0x7f2178005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:16.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.301+0000 7f218affd700 1 -- 172.21.15.16:0/3035038229 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f218c0cb670 msgr2=0x7f2178005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:16.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.301+0000 7f218affd700 1 --2- 172.21.15.16:0/3035038229 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f218c0cb670 0x7f2178005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:16.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.301+0000 7f218affd700 1 -- 172.21.15.16:0/3035038229 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2178003aa0 msgr2=0x7f2178003eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:16.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.301+0000 7f218affd700 1 --2- 172.21.15.16:0/3035038229 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2178003aa0 0x7f2178003eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:16.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.301+0000 7f218affd700 1 -- 172.21.15.16:0/3035038229 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21780035b0 con 0x7f2178004850 2023-05-20T01:25:16.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.301+0000 7f218bfff700 1 --2- 172.21.15.16:0/3035038229 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f218c0cb670 0x7f2178005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:25:16.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.301+0000 7f218affd700 1 --2- 172.21.15.16:0/3035038229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2178004850 0x7f2178004cc0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f217c009b30 tx=0x7f2178005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cce37e7da3a4d73 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:16.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.301+0000 7f218a7fc700 1 -- 172.21.15.16:0/3035038229 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f217c00e6c0 con 0x7f2178004850 2023-05-20T01:25:16.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.301+0000 7f218a7fc700 1 -- 172.21.15.16:0/3035038229 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f217c00e820 con 0x7f2178004850 2023-05-20T01:25:16.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.301+0000 7f218a7fc700 1 -- 172.21.15.16:0/3035038229 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f217c00eb90 con 0x7f2178004850 2023-05-20T01:25:16.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.302+0000 7f2192087700 1 -- 172.21.15.16:0/3035038229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2178004850 msgr2=0x7f2178004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:16.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.302+0000 7f2192087700 1 --2- 172.21.15.16:0/3035038229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2178004850 0x7f2178004cc0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f217c009b30 tx=0x7f2178005880 comp rx=0 tx=0).stop 2023-05-20T01:25:16.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.302+0000 7f218bfff700 1 -- 172.21.15.16:0/3035038229 reap_dead start 2023-05-20T01:25:16.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.302+0000 7f2192087700 1 -- 172.21.15.16:0/3035038229 shutdown_connections 2023-05-20T01:25:16.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.302+0000 7f2192087700 1 -- 172.21.15.16:0/3035038229 >> 172.21.15.16:0/3035038229 conn(0x7f218c0d0c30 msgr2=0x7f218c0b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:16.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.302+0000 7f2192087700 1 -- 172.21.15.16:0/3035038229 shutdown_connections 2023-05-20T01:25:16.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.302+0000 7f2192087700 1 -- 172.21.15.16:0/3035038229 wait complete. 2023-05-20T01:25:16.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.303+0000 7f2192087700 1 Processor -- start 2023-05-20T01:25:16.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.303+0000 7f2192087700 1 -- start start 2023-05-20T01:25:16.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.304+0000 7f2192087700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2178004850 0x7f218c1579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:16.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.304+0000 7f2192087700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f218c0c9960 0x7f218c1f04d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:16.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.304+0000 7f2192087700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f218c0c9dd0 0x7f218c15bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:16.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.304+0000 7f2192087700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f218c0c71b0 con 0x7f218c0c9dd0 2023-05-20T01:25:16.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.304+0000 7f2192087700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f218c0c7490 con 0x7f218c0c9960 2023-05-20T01:25:16.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.304+0000 7f2192087700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f218c0c7770 con 0x7f2178004850 2023-05-20T01:25:16.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.304+0000 7f218bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f218c0c9dd0 0x7f218c15bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:16.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.304+0000 7f218bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f218c0c9dd0 0x7f218c15bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36264/0 (socket says 172.21.15.16:36264) 2023-05-20T01:25:16.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.304+0000 7f218bfff700 1 -- 172.21.15.16:0/1831686432 learned_addr learned my addr 172.21.15.16:0/1831686432 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:16.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.304+0000 7f218bfff700 1 -- 172.21.15.16:0/1831686432 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2178004850 msgr2=0x7f218c1579a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:16.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.304+0000 7f218b7fe700 1 --2- 172.21.15.16:0/1831686432 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2178004850 0x7f218c1579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:16.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.304+0000 7f218affd700 1 --2- 172.21.15.16:0/1831686432 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f218c0c9960 0x7f218c1f04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:16.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.304+0000 7f218bfff700 1 --2- 172.21.15.16:0/1831686432 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2178004850 0x7f218c1579a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:16.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.304+0000 7f218bfff700 1 -- 172.21.15.16:0/1831686432 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f218c0c9960 msgr2=0x7f218c1f04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:16.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.304+0000 7f218bfff700 1 --2- 172.21.15.16:0/1831686432 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f218c0c9960 0x7f218c1f04d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:16.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.304+0000 7f218bfff700 1 -- 172.21.15.16:0/1831686432 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f218c06f0b0 con 0x7f218c0c9dd0 2023-05-20T01:25:16.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.305+0000 7f218bfff700 1 --2- 172.21.15.16:0/1831686432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f218c0c9dd0 0x7f218c15bda0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f218c0ca860 tx=0x7f218400e440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:16.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.305+0000 7f2183fff700 1 -- 172.21.15.16:0/1831686432 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f218400ec70 con 0x7f218c0c9dd0 2023-05-20T01:25:16.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.305+0000 7f2183fff700 1 -- 172.21.15.16:0/1831686432 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f218400edd0 con 0x7f218c0c9dd0 2023-05-20T01:25:16.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.305+0000 7f2192087700 1 -- 172.21.15.16:0/1831686432 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f218c06edd0 con 0x7f218c0c9dd0 2023-05-20T01:25:16.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.305+0000 7f2183fff700 1 -- 172.21.15.16:0/1831686432 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2184013660 con 0x7f218c0c9dd0 2023-05-20T01:25:16.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.305+0000 7f2192087700 1 -- 172.21.15.16:0/1831686432 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f218c06e750 con 0x7f218c0c9dd0 2023-05-20T01:25:16.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.306+0000 7f2192087700 1 -- 172.21.15.16:0/1831686432 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f218c0747b0 con 0x7f218c0c9dd0 2023-05-20T01:25:16.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.306+0000 7f2183fff700 1 -- 172.21.15.16:0/1831686432 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f21840137c0 con 0x7f218c0c9dd0 2023-05-20T01:25:16.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.306+0000 7f2183fff700 1 --2- 172.21.15.16:0/1831686432 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f216c06ace0 0x7f216c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:16.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.307+0000 7f2183fff700 1 -- 172.21.15.16:0/1831686432 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f2184089960 con 0x7f218c0c9dd0 2023-05-20T01:25:16.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.307+0000 7f218b7fe700 1 --2- 172.21.15.16:0/1831686432 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f216c06ace0 0x7f216c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:16.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.307+0000 7f218b7fe700 1 --2- 172.21.15.16:0/1831686432 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f216c06ace0 0x7f216c06d190 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f218c0b45c0 tx=0x7f2174009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:16.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.311+0000 7f2183fff700 1 -- 172.21.15.16:0/1831686432 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f2184054a20 con 0x7f218c0c9dd0 2023-05-20T01:25:16.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.505+0000 7f2192087700 1 -- 172.21.15.16:0/1831686432 --> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f218c0b3bb0 con 0x7f216c06ace0 2023-05-20T01:25:16.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.514+0000 7f2183fff700 1 -- 172.21.15.16:0/1831686432 <== mgr.4104 v2:172.21.15.16:6832/3817237998 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+296973 (secure 0 0 0) 0x7f218c0b3bb0 con 0x7f216c06ace0 2023-05-20T01:25:16.521 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:25:16.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.518+0000 7f2192087700 1 -- 172.21.15.16:0/1831686432 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f216c06ace0 msgr2=0x7f216c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:16.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.518+0000 7f2192087700 1 --2- 172.21.15.16:0/1831686432 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f216c06ace0 0x7f216c06d190 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f218c0b45c0 tx=0x7f2174009040 comp rx=0 tx=0).stop 2023-05-20T01:25:16.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.518+0000 7f2192087700 1 -- 172.21.15.16:0/1831686432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f218c0c9dd0 msgr2=0x7f218c15bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:16.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.518+0000 7f2192087700 1 --2- 172.21.15.16:0/1831686432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f218c0c9dd0 0x7f218c15bda0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f218c0ca860 tx=0x7f218400e440 comp rx=0 tx=0).stop 2023-05-20T01:25:16.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.518+0000 7f2192087700 1 -- 172.21.15.16:0/1831686432 shutdown_connections 2023-05-20T01:25:16.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.518+0000 7f2192087700 1 --2- 172.21.15.16:0/1831686432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f218c0c9dd0 0x7f218c15bda0 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:16.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.518+0000 7f2192087700 1 --2- 172.21.15.16:0/1831686432 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f216c06ace0 0x7f216c06d190 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:16.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.518+0000 7f2192087700 1 --2- 172.21.15.16:0/1831686432 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2178004850 0x7f218c1579a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:16.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.518+0000 7f2192087700 1 --2- 172.21.15.16:0/1831686432 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f218c0c9960 0x7f218c1f04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:16.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.518+0000 7f2192087700 1 -- 172.21.15.16:0/1831686432 >> 172.21.15.16:0/1831686432 conn(0x7f218c0d0c30 msgr2=0x7f218c0ccbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:16.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.518+0000 7f2192087700 1 -- 172.21.15.16:0/1831686432 shutdown_connections 2023-05-20T01:25:16.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.518+0000 7f2192087700 1 -- 172.21.15.16:0/1831686432 wait complete. 2023-05-20T01:25:16.524 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2023-05-20T01:25:16.534 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":349,"stamp":"2023-05-20T01:25:15.434324+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":94944,"kb_used_data":9184,"kb_used_omap":0,"kb_used_meta":85696,"kb_avail":754879776,"statfs":{"total":773094113280,"available":772996890624,"internally_reserved":0,"allocated":9404416,"data_stored":5307966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":87752704},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.881661"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:10.545755+0000","last_change":"2023-05-20T01:25:10.545755+0000","last_active":"2023-05-20T01:25:10.545755+0000","last_peered":"2023-05-20T01:25:10.545755+0000","last_clean":"2023-05-20T01:25:10.545755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:10.545755+0000","last_undegraded":"2023-05-20T01:25:10.545755+0000","last_fullsized":"2023-05-20T01:25:10.545755+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:10.545755+0000","last_change":"2023-05-20T01:25:10.545755+0000","last_active":"2023-05-20T01:25:10.545755+0000","last_peered":"2023-05-20T01:25:10.545755+0000","last_clean":"2023-05-20T01:25:10.545755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:10.545755+0000","last_undegraded":"2023-05-20T01:25:10.545755+0000","last_fullsized":"2023-05-20T01:25:10.545755+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:10.545755+0000","last_change":"2023-05-20T01:25:10.545755+0000","last_active":"2023-05-20T01:25:10.545755+0000","last_peered":"2023-05-20T01:25:10.545755+0000","last_clean":"2023-05-20T01:25:10.545755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:10.545755+0000","last_undegraded":"2023-05-20T01:25:10.545755+0000","last_fullsized":"2023-05-20T01:25:10.545755+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:10.545755+0000","last_change":"2023-05-20T01:25:10.545755+0000","last_active":"2023-05-20T01:25:10.545755+0000","last_peered":"2023-05-20T01:25:10.545755+0000","last_clean":"2023-05-20T01:25:10.545755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:10.545755+0000","last_undegraded":"2023-05-20T01:25:10.545755+0000","last_fullsized":"2023-05-20T01:25:10.545755+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:10.545755+0000","last_change":"2023-05-20T01:25:10.545755+0000","last_active":"2023-05-20T01:25:10.545755+0000","last_peered":"2023-05-20T01:25:10.545755+0000","last_clean":"2023-05-20T01:25:10.545755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:10.545755+0000","last_undegraded":"2023-05-20T01:25:10.545755+0000","last_fullsized":"2023-05-20T01:25:10.545755+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:10.545755+0000","last_change":"2023-05-20T01:25:10.545755+0000","last_active":"2023-05-20T01:25:10.545755+0000","last_peered":"2023-05-20T01:25:10.545755+0000","last_clean":"2023-05-20T01:25:10.545755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:10.545755+0000","last_undegraded":"2023-05-20T01:25:10.545755+0000","last_fullsized":"2023-05-20T01:25:10.545755+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:10.545755+0000","last_change":"2023-05-20T01:25:10.545755+0000","last_active":"2023-05-20T01:25:10.545755+0000","last_peered":"2023-05-20T01:25:10.545755+0000","last_clean":"2023-05-20T01:25:10.545755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:10.545755+0000","last_undegraded":"2023-05-20T01:25:10.545755+0000","last_fullsized":"2023-05-20T01:25:10.545755+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:10.545755+0000","last_change":"2023-05-20T01:25:10.545755+0000","last_active":"2023-05-20T01:25:10.545755+0000","last_peered":"2023-05-20T01:25:10.545755+0000","last_clean":"2023-05-20T01:25:10.545755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:10.545755+0000","last_undegraded":"2023-05-20T01:25:10.545755+0000","last_fullsized":"2023-05-20T01:25:10.545755+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:10.545755+0000","last_change":"2023-05-20T01:25:10.545755+0000","last_active":"2023-05-20T01:25:10.545755+0000","last_peered":"2023-05-20T01:25:10.545755+0000","last_clean":"2023-05-20T01:25:10.545755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:10.545755+0000","last_undegraded":"2023-05-20T01:25:10.545755+0000","last_fullsized":"2023-05-20T01:25:10.545755+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:10.545755+0000","last_change":"2023-05-20T01:25:10.545755+0000","last_active":"2023-05-20T01:25:10.545755+0000","last_peered":"2023-05-20T01:25:10.545755+0000","last_clean":"2023-05-20T01:25:10.545755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:10.545755+0000","last_undegraded":"2023-05-20T01:25:10.545755+0000","last_fullsized":"2023-05-20T01:25:10.545755+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:10.545755+0000","last_change":"2023-05-20T01:25:10.545755+0000","last_active":"2023-05-20T01:25:10.545755+0000","last_peered":"2023-05-20T01:25:10.545755+0000","last_clean":"2023-05-20T01:25:10.545755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:10.545755+0000","last_undegraded":"2023-05-20T01:25:10.545755+0000","last_fullsized":"2023-05-20T01:25:10.545755+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:10.545755+0000","last_change":"2023-05-20T01:25:10.545755+0000","last_active":"2023-05-20T01:25:10.545755+0000","last_peered":"2023-05-20T01:25:10.545755+0000","last_clean":"2023-05-20T01:25:10.545755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:10.545755+0000","last_undegraded":"2023-05-20T01:25:10.545755+0000","last_fullsized":"2023-05-20T01:25:10.545755+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:10.545755+0000","last_change":"2023-05-20T01:25:10.545755+0000","last_active":"2023-05-20T01:25:10.545755+0000","last_peered":"2023-05-20T01:25:10.545755+0000","last_clean":"2023-05-20T01:25:10.545755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:10.545755+0000","last_undegraded":"2023-05-20T01:25:10.545755+0000","last_fullsized":"2023-05-20T01:25:10.545755+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:10.545755+0000","last_change":"2023-05-20T01:25:10.545755+0000","last_active":"2023-05-20T01:25:10.545755+0000","last_peered":"2023-05-20T01:25:10.545755+0000","last_clean":"2023-05-20T01:25:10.545755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:10.545755+0000","last_undegraded":"2023-05-20T01:25:10.545755+0000","last_fullsized":"2023-05-20T01:25:10.545755+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:10.545755+0000","last_change":"2023-05-20T01:25:10.545755+0000","last_active":"2023-05-20T01:25:10.545755+0000","last_peered":"2023-05-20T01:25:10.545755+0000","last_clean":"2023-05-20T01:25:10.545755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:10.545755+0000","last_undegraded":"2023-05-20T01:25:10.545755+0000","last_fullsized":"2023-05-20T01:25:10.545755+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:10.545755+0000","last_change":"2023-05-20T01:25:10.545755+0000","last_active":"2023-05-20T01:25:10.545755+0000","last_peered":"2023-05-20T01:25:10.545755+0000","last_clean":"2023-05-20T01:25:10.545755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:10.545755+0000","last_undegraded":"2023-05-20T01:25:10.545755+0000","last_fullsized":"2023-05-20T01:25:10.545755+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:10.545755+0000","last_change":"2023-05-20T01:25:10.545755+0000","last_active":"2023-05-20T01:25:10.545755+0000","last_peered":"2023-05-20T01:25:10.545755+0000","last_clean":"2023-05-20T01:25:10.545755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:10.545755+0000","last_undegraded":"2023-05-20T01:25:10.545755+0000","last_fullsized":"2023-05-20T01:25:10.545755+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:10.545755+0000","last_change":"2023-05-20T01:25:10.545755+0000","last_active":"2023-05-20T01:25:10.545755+0000","last_peered":"2023-05-20T01:25:10.545755+0000","last_clean":"2023-05-20T01:25:10.545755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:10.545755+0000","last_undegraded":"2023-05-20T01:25:10.545755+0000","last_fullsized":"2023-05-20T01:25:10.545755+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:10.545755+0000","last_change":"2023-05-20T01:25:10.545755+0000","last_active":"2023-05-20T01:25:10.545755+0000","last_peered":"2023-05-20T01:25:10.545755+0000","last_clean":"2023-05-20T01:25:10.545755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:10.545755+0000","last_undegraded":"2023-05-20T01:25:10.545755+0000","last_fullsized":"2023-05-20T01:25:10.545755+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:10.545755+0000","last_change":"2023-05-20T01:25:10.545755+0000","last_active":"2023-05-20T01:25:10.545755+0000","last_peered":"2023-05-20T01:25:10.545755+0000","last_clean":"2023-05-20T01:25:10.545755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:10.545755+0000","last_undegraded":"2023-05-20T01:25:10.545755+0000","last_fullsized":"2023-05-20T01:25:10.545755+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:10.545755+0000","last_change":"2023-05-20T01:25:10.545755+0000","last_active":"2023-05-20T01:25:10.545755+0000","last_peered":"2023-05-20T01:25:10.545755+0000","last_clean":"2023-05-20T01:25:10.545755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:10.545755+0000","last_undegraded":"2023-05-20T01:25:10.545755+0000","last_fullsized":"2023-05-20T01:25:10.545755+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:10.545755+0000","last_change":"2023-05-20T01:25:10.545755+0000","last_active":"2023-05-20T01:25:10.545755+0000","last_peered":"2023-05-20T01:25:10.545755+0000","last_clean":"2023-05-20T01:25:10.545755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:10.545755+0000","last_undegraded":"2023-05-20T01:25:10.545755+0000","last_fullsized":"2023-05-20T01:25:10.545755+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:10.545755+0000","last_change":"2023-05-20T01:25:10.545755+0000","last_active":"2023-05-20T01:25:10.545755+0000","last_peered":"2023-05-20T01:25:10.545755+0000","last_clean":"2023-05-20T01:25:10.545755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:10.545755+0000","last_undegraded":"2023-05-20T01:25:10.545755+0000","last_fullsized":"2023-05-20T01:25:10.545755+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:10.545755+0000","last_change":"2023-05-20T01:25:10.545755+0000","last_active":"2023-05-20T01:25:10.545755+0000","last_peered":"2023-05-20T01:25:10.545755+0000","last_clean":"2023-05-20T01:25:10.545755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:10.545755+0000","last_undegraded":"2023-05-20T01:25:10.545755+0000","last_fullsized":"2023-05-20T01:25:10.545755+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:10.545755+0000","last_change":"2023-05-20T01:25:10.545755+0000","last_active":"2023-05-20T01:25:10.545755+0000","last_peered":"2023-05-20T01:25:10.545755+0000","last_clean":"2023-05-20T01:25:10.545755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:10.545755+0000","last_undegraded":"2023-05-20T01:25:10.545755+0000","last_fullsized":"2023-05-20T01:25:10.545755+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":143,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-05-20T01:25:07.873873+0000","last_change":"2023-05-20T01:16:23.289068+0000","last_active":"2023-05-20T01:25:07.873873+0000","last_peered":"2023-05-20T01:25:07.873873+0000","last_clean":"2023-05-20T01:25:07.873873+0000","last_became_active":"2023-05-20T01:16:23.288781+0000","last_became_peered":"2023-05-20T01:16:23.288781+0000","last_unstale":"2023-05-20T01:25:07.873873+0000","last_undegraded":"2023-05-20T01:25:07.873873+0000","last_fullsized":"2023-05-20T01:25:07.873873+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:48:00.157178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:10.545755+0000","last_change":"2023-05-20T01:25:10.545755+0000","last_active":"2023-05-20T01:25:10.545755+0000","last_peered":"2023-05-20T01:25:10.545755+0000","last_clean":"2023-05-20T01:25:10.545755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:10.545755+0000","last_undegraded":"2023-05-20T01:25:10.545755+0000","last_fullsized":"2023-05-20T01:25:10.545755+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":149,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-05-20T01:25:07.711852+0000","last_change":"2023-05-20T01:16:22.829324+0000","last_active":"2023-05-20T01:25:07.711852+0000","last_peered":"2023-05-20T01:25:07.711852+0000","last_clean":"2023-05-20T01:25:07.711852+0000","last_became_active":"2023-05-20T01:16:22.828964+0000","last_became_peered":"2023-05-20T01:16:22.828964+0000","last_unstale":"2023-05-20T01:25:07.711852+0000","last_undegraded":"2023-05-20T01:25:07.711852+0000","last_fullsized":"2023-05-20T01:25:07.711852+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:53:20.474800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:10.545755+0000","last_change":"2023-05-20T01:25:10.545755+0000","last_active":"2023-05-20T01:25:10.545755+0000","last_peered":"2023-05-20T01:25:10.545755+0000","last_clean":"2023-05-20T01:25:10.545755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:10.545755+0000","last_undegraded":"2023-05-20T01:25:10.545755+0000","last_fullsized":"2023-05-20T01:25:10.545755+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":135,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-05-20T01:25:07.873896+0000","last_change":"2023-05-20T01:16:22.834754+0000","last_active":"2023-05-20T01:25:07.873896+0000","last_peered":"2023-05-20T01:25:07.873896+0000","last_clean":"2023-05-20T01:25:07.873896+0000","last_became_active":"2023-05-20T01:16:22.834437+0000","last_became_peered":"2023-05-20T01:16:22.834437+0000","last_unstale":"2023-05-20T01:25:07.873896+0000","last_undegraded":"2023-05-20T01:25:07.873896+0000","last_fullsized":"2023-05-20T01:25:07.873896+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:39:16.284912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":271,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-05-20T01:25:07.874026+0000","last_change":"2023-05-20T01:16:22.834920+0000","last_active":"2023-05-20T01:25:07.874026+0000","last_peered":"2023-05-20T01:25:07.874026+0000","last_clean":"2023-05-20T01:25:07.874026+0000","last_became_active":"2023-05-20T01:16:22.833924+0000","last_became_peered":"2023-05-20T01:16:22.833924+0000","last_unstale":"2023-05-20T01:25:07.874026+0000","last_undegraded":"2023-05-20T01:25:07.874026+0000","last_fullsized":"2023-05-20T01:25:07.874026+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:25:38.846936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:10.545755+0000","last_change":"2023-05-20T01:25:10.545755+0000","last_active":"2023-05-20T01:25:10.545755+0000","last_peered":"2023-05-20T01:25:10.545755+0000","last_clean":"2023-05-20T01:25:10.545755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:10.545755+0000","last_undegraded":"2023-05-20T01:25:10.545755+0000","last_fullsized":"2023-05-20T01:25:10.545755+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":121,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-05-20T01:25:07.873782+0000","last_change":"2023-05-20T01:16:23.290259+0000","last_active":"2023-05-20T01:25:07.873782+0000","last_peered":"2023-05-20T01:25:07.873782+0000","last_clean":"2023-05-20T01:25:07.873782+0000","last_became_active":"2023-05-20T01:16:23.289911+0000","last_became_peered":"2023-05-20T01:16:23.289911+0000","last_unstale":"2023-05-20T01:25:07.873782+0000","last_undegraded":"2023-05-20T01:25:07.873782+0000","last_fullsized":"2023-05-20T01:25:07.873782+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:55:46.978560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:10.545755+0000","last_change":"2023-05-20T01:25:10.545755+0000","last_active":"2023-05-20T01:25:10.545755+0000","last_peered":"2023-05-20T01:25:10.545755+0000","last_clean":"2023-05-20T01:25:10.545755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:10.545755+0000","last_undegraded":"2023-05-20T01:25:10.545755+0000","last_fullsized":"2023-05-20T01:25:10.545755+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":121,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-05-20T01:25:07.874033+0000","last_change":"2023-05-20T01:16:23.290207+0000","last_active":"2023-05-20T01:25:07.874033+0000","last_peered":"2023-05-20T01:25:07.874033+0000","last_clean":"2023-05-20T01:25:07.874033+0000","last_became_active":"2023-05-20T01:16:23.289928+0000","last_became_peered":"2023-05-20T01:16:23.289928+0000","last_unstale":"2023-05-20T01:25:07.874033+0000","last_undegraded":"2023-05-20T01:25:07.874033+0000","last_fullsized":"2023-05-20T01:25:07.874033+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:19:51.946682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:10.545755+0000","last_change":"2023-05-20T01:25:10.545755+0000","last_active":"2023-05-20T01:25:10.545755+0000","last_peered":"2023-05-20T01:25:10.545755+0000","last_clean":"2023-05-20T01:25:10.545755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:10.545755+0000","last_undegraded":"2023-05-20T01:25:10.545755+0000","last_fullsized":"2023-05-20T01:25:10.545755+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":121,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-05-20T01:25:07.873924+0000","last_change":"2023-05-20T01:16:23.290538+0000","last_active":"2023-05-20T01:25:07.873924+0000","last_peered":"2023-05-20T01:25:07.873924+0000","last_clean":"2023-05-20T01:25:07.873924+0000","last_became_active":"2023-05-20T01:16:23.290230+0000","last_became_peered":"2023-05-20T01:16:23.290230+0000","last_unstale":"2023-05-20T01:25:07.873924+0000","last_undegraded":"2023-05-20T01:25:07.873924+0000","last_fullsized":"2023-05-20T01:25:07.873924+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:15:27.984665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:10.545755+0000","last_change":"2023-05-20T01:25:10.545755+0000","last_active":"2023-05-20T01:25:10.545755+0000","last_peered":"2023-05-20T01:25:10.545755+0000","last_clean":"2023-05-20T01:25:10.545755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:10.545755+0000","last_undegraded":"2023-05-20T01:25:10.545755+0000","last_fullsized":"2023-05-20T01:25:10.545755+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":127,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-05-20T01:25:07.714758+0000","last_change":"2023-05-20T01:16:23.290341+0000","last_active":"2023-05-20T01:25:07.714758+0000","last_peered":"2023-05-20T01:25:07.714758+0000","last_clean":"2023-05-20T01:25:07.714758+0000","last_became_active":"2023-05-20T01:16:23.290053+0000","last_became_peered":"2023-05-20T01:16:23.290053+0000","last_unstale":"2023-05-20T01:25:07.714758+0000","last_undegraded":"2023-05-20T01:25:07.714758+0000","last_fullsized":"2023-05-20T01:25:07.714758+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:37:03.379326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:10.545755+0000","last_change":"2023-05-20T01:25:10.545755+0000","last_active":"2023-05-20T01:25:10.545755+0000","last_peered":"2023-05-20T01:25:10.545755+0000","last_clean":"2023-05-20T01:25:10.545755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:10.545755+0000","last_undegraded":"2023-05-20T01:25:10.545755+0000","last_fullsized":"2023-05-20T01:25:10.545755+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":127,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-05-20T01:25:07.714773+0000","last_change":"2023-05-20T01:16:22.834330+0000","last_active":"2023-05-20T01:25:07.714773+0000","last_peered":"2023-05-20T01:25:07.714773+0000","last_clean":"2023-05-20T01:25:07.714773+0000","last_became_active":"2023-05-20T01:16:22.833946+0000","last_became_peered":"2023-05-20T01:16:22.833946+0000","last_unstale":"2023-05-20T01:25:07.714773+0000","last_undegraded":"2023-05-20T01:25:07.714773+0000","last_fullsized":"2023-05-20T01:25:07.714773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:28:56.043585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574958,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":11576,"kb_used_data":1584,"kb_used_omap":0,"kb_used_meta":9984,"kb_avail":94360264,"statfs":{"total":96636764160,"available":96624910336,"internally_reserved":0,"allocated":1622016,"data_stored":1106512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":10223616},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:24:34 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.47699999999999998,"15min":0.51400000000000001},"min":{"1min":0.26900000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.68799999999999994,"5min":0.73199999999999998,"15min":0.88200000000000001},"last":0.623},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.503,"15min":0.54700000000000004},"min":{"1min":0.33100000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.63600000000000001,"5min":0.752,"15min":0.90000000000000002},"last":0.501}]},{"osd":1,"last update":"Sat May 20 01:24:34 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.49299999999999999,"15min":0.75},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.628,"5min":0.70299999999999996,"15min":8.8879999999999999},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.52200000000000002,"15min":0.78900000000000003},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.74199999999999999,"5min":0.75,"15min":8.875},"last":0.74099999999999999}]},{"osd":2,"last update":"Sat May 20 01:24:34 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.503,"15min":0.73599999999999999},"min":{"1min":0.34000000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.65900000000000003,"5min":0.77100000000000002,"15min":8.9120000000000008},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.504,"15min":0.73899999999999999},"min":{"1min":0.35999999999999999,"5min":0.35499999999999998,"15min":0.35099999999999998},"max":{"1min":0.58999999999999997,"5min":0.72399999999999998,"15min":8.8520000000000003},"last":0.60199999999999998}]},{"osd":3,"last update":"Sat May 20 01:24:34 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.49399999999999999,"15min":0.79700000000000004},"min":{"1min":0.36399999999999999,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.60599999999999998,"5min":0.73799999999999999,"15min":10.884},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.55100000000000005,"15min":0.85099999999999998},"min":{"1min":0.40600000000000003,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":10.952},"last":0.77300000000000002}]},{"osd":4,"last update":"Sat May 20 01:24:34 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51100000000000001,"15min":0.51000000000000001},"min":{"1min":0.39200000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.66700000000000004,"5min":0.74299999999999999,"15min":0.77000000000000002},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.312},"max":{"1min":0.71599999999999997,"5min":0.78500000000000003,"15min":0.875},"last":0.58499999999999996}]},{"osd":5,"last update":"Sat May 20 01:24:34 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.56299999999999994,"15min":0.54200000000000004},"min":{"1min":0.35999999999999999,"5min":0.35799999999999998,"15min":0.32300000000000001},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.57099999999999995,"15min":0.57599999999999996},"min":{"1min":0.35999999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.745,"5min":0.79600000000000004,"15min":0.80200000000000005},"last":0.68799999999999994}]},{"osd":6,"last update":"Sat May 20 01:24:34 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.59599999999999997,"15min":0.59499999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.378},"max":{"1min":0.72699999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.57799999999999996,"15min":0.58399999999999996},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.73399999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.752}]}]},{"osd":6,"up_from":13,"seq":55834574957,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10164,"kb_used_data":1004,"kb_used_omap":0,"kb_used_meta":9152,"kb_avail":94361676,"statfs":{"total":96636764160,"available":96626356224,"internally_reserved":0,"allocated":1028096,"data_stored":516144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9371648},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:24:36 2023","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.45400000000000001,"15min":0.47799999999999998},"min":{"1min":0.29099999999999998,"5min":0.28999999999999998,"15min":0.26900000000000002},"max":{"1min":0.55500000000000005,"5min":0.72699999999999998,"15min":0.77200000000000002},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.438,"5min":0.47199999999999998,"15min":0.51800000000000002},"min":{"1min":0.309,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.60299999999999998,"5min":0.70099999999999996,"15min":0.73999999999999999},"last":0.48499999999999999}]},{"osd":1,"last update":"Sat May 20 01:24:36 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.495,"15min":0.52100000000000002},"min":{"1min":0.312,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.70499999999999996,"5min":0.71699999999999997,"15min":0.83799999999999997},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.50800000000000001,"15min":0.52400000000000002},"min":{"1min":0.33000000000000002,"5min":0.32600000000000001,"15min":0.318},"max":{"1min":0.68600000000000005,"5min":0.749,"15min":0.79200000000000004},"last":0.57999999999999996}]},{"osd":2,"last update":"Sat May 20 01:24:36 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.48199999999999998,"15min":0.504},"min":{"1min":0.36799999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.52700000000000002,"5min":0.72199999999999998,"15min":0.76600000000000001},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.51800000000000002,"15min":0.55000000000000004},"min":{"1min":0.35899999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.629,"5min":0.76200000000000001,"15min":0.76800000000000002},"last":0.64300000000000002}]},{"osd":3,"last update":"Sat May 20 01:24:36 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.51500000000000001,"15min":0.53500000000000003},"min":{"1min":0.32400000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.80800000000000005},"last":0.624},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.52100000000000002,"15min":0.54000000000000004},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.58099999999999996,"5min":0.76400000000000001,"15min":0.79500000000000004},"last":0.50600000000000001}]},{"osd":4,"last update":"Sat May 20 01:24:36 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.53800000000000003,"15min":0.55000000000000004},"min":{"1min":0.42599999999999999,"5min":0.33300000000000002,"15min":0.32600000000000001},"max":{"1min":0.754,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.54700000000000004,"15min":0.56899999999999995},"min":{"1min":0.40300000000000002,"5min":0.378,"15min":0.32700000000000001},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.80100000000000005},"last":0.44}]},{"osd":5,"last update":"Sat May 20 01:24:36 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.54100000000000004,"15min":0.54400000000000004},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.65000000000000002,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.38},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.57599999999999996,"15min":0.60399999999999998},"min":{"1min":0.46100000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.78600000000000003,"5min":0.81599999999999995,"15min":0.89400000000000002},"last":0.51500000000000001}]},{"osd":7,"last update":"Sat May 20 01:24:36 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57699999999999996,"15min":0.59799999999999998},"min":{"1min":0.44,"5min":0.371,"15min":0.371},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.92900000000000005},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.58099999999999996,"15min":0.60999999999999999},"min":{"1min":0.48099999999999998,"5min":0.379,"15min":0.379},"max":{"1min":0.68100000000000005,"5min":0.82799999999999996,"15min":0.85199999999999998},"last":0.59499999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574957,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":16376,"kb_used_data":1584,"kb_used_omap":0,"kb_used_meta":14784,"kb_avail":94355464,"statfs":{"total":96636764160,"available":96619995136,"internally_reserved":0,"allocated":1622016,"data_stored":1106512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":15138816},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:24:33 2023","interfaces":[{"interface":"back","average":{"1min":0.254,"5min":0.28199999999999997,"15min":0.313},"min":{"1min":0.186,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":0.37,"5min":0.57099999999999995,"15min":1.194},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.28299999999999997,"5min":0.314,"15min":0.316},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.41399999999999998,"5min":0.61499999999999999,"15min":0.75800000000000001},"last":0.73199999999999998}]},{"osd":2,"last update":"Sat May 20 01:24:33 2023","interfaces":[{"interface":"back","average":{"1min":0.27200000000000002,"5min":0.30199999999999999,"15min":0.317},"min":{"1min":0.20100000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.33800000000000002,"5min":0.52400000000000002,"15min":0.66400000000000003},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.33700000000000002,"5min":0.36799999999999999,"15min":0.40600000000000003},"min":{"1min":0.27800000000000002,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.41299999999999998,"5min":0.60299999999999998,"15min":1.3580000000000001},"last":0.497}]},{"osd":3,"last update":"Sat May 20 01:24:33 2023","interfaces":[{"interface":"back","average":{"1min":0.315,"5min":0.34999999999999998,"15min":0.36899999999999999},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.40500000000000003,"5min":0.57399999999999995,"15min":1.619},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.32100000000000001,"5min":0.34699999999999998,"15min":0.379},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.41299999999999998,"5min":0.60899999999999999,"15min":1.8400000000000001},"last":0.55300000000000005}]},{"osd":4,"last update":"Sat May 20 01:24:33 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.55900000000000005,"15min":0.58099999999999996},"min":{"1min":0.379,"5min":0.34899999999999998,"15min":0.32200000000000001},"max":{"1min":0.72799999999999998,"5min":0.85299999999999998,"15min":1.2769999999999999},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.56999999999999995,"15min":0.58299999999999996},"min":{"1min":0.41699999999999998,"5min":0.312,"15min":0.312},"max":{"1min":0.65600000000000003,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.56899999999999995}]},{"osd":5,"last update":"Sat May 20 01:24:33 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.58399999999999996,"15min":0.58899999999999997},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.754,"5min":0.81899999999999995,"15min":1.4510000000000001},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.59299999999999997,"15min":0.60599999999999998},"min":{"1min":0.436,"5min":0.39100000000000001,"15min":0.378},"max":{"1min":0.77200000000000002,"5min":0.79900000000000004,"15min":1.409},"last":0.58699999999999997}]},{"osd":6,"last update":"Sat May 20 01:24:33 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.59499999999999997,"15min":0.60399999999999998},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.73599999999999999,"5min":0.876,"15min":1.575},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.58499999999999996,"15min":0.59399999999999997},"min":{"1min":0.41299999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.66400000000000003,"5min":0.83099999999999996,"15min":1.0600000000000001},"last":0.60799999999999998}]},{"osd":7,"last update":"Sat May 20 01:24:33 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.59499999999999997,"15min":0.60499999999999998},"min":{"1min":0.40500000000000003,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.75600000000000001,"5min":0.88700000000000001,"15min":1.496},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.623,"15min":0.63800000000000001},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.77300000000000002,"5min":2.1709999999999998,"15min":2.1709999999999998},"last":1.5009999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574957,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10224,"kb_used_data":1000,"kb_used_omap":0,"kb_used_meta":9216,"kb_avail":94361616,"statfs":{"total":96636764160,"available":96626294784,"internally_reserved":0,"allocated":1024000,"data_stored":515183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9437184},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat May 20 01:24:32 2023","interfaces":[{"interface":"back","average":{"1min":0.26200000000000001,"5min":0.29199999999999998,"15min":0.318},"min":{"1min":0.192,"5min":0.17999999999999999,"15min":0.17999999999999999},"max":{"1min":0.371,"5min":0.54600000000000004,"15min":0.56599999999999995},"last":0.247},{"interface":"front","average":{"1min":0.26400000000000001,"5min":0.29699999999999999,"15min":0.34200000000000003},"min":{"1min":0.21099999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.33300000000000002,"5min":0.52000000000000002,"15min":0.98799999999999999},"last":0.36899999999999999}]},{"osd":2,"last update":"Sat May 20 01:24:32 2023","interfaces":[{"interface":"back","average":{"1min":0.28100000000000003,"5min":0.29999999999999999,"15min":0.32800000000000001},"min":{"1min":0.20100000000000001,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":0.35999999999999999,"5min":0.52300000000000002,"15min":0.55700000000000005},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.33100000000000002,"5min":0.34899999999999998,"15min":0.38200000000000001},"min":{"1min":0.26200000000000001,"5min":0.216,"15min":0.216},"max":{"1min":0.41699999999999998,"5min":0.60299999999999998,"15min":0.86199999999999999},"last":0.41799999999999998}]},{"osd":3,"last update":"Sat May 20 01:24:32 2023","interfaces":[{"interface":"back","average":{"1min":0.29599999999999999,"5min":0.33200000000000002,"15min":0.35999999999999999},"min":{"1min":0.24299999999999999,"5min":0.22,"15min":0.22},"max":{"1min":0.374,"5min":0.623,"15min":0.68300000000000005},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.35399999999999998,"15min":0.36599999999999999},"min":{"1min":0.28000000000000003,"5min":0.245,"15min":0.245},"max":{"1min":0.44700000000000001,"5min":0.60899999999999999,"15min":0.60899999999999999},"last":0.44900000000000001}]},{"osd":4,"last update":"Sat May 20 01:24:32 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.57099999999999995,"15min":0.55100000000000005},"min":{"1min":0.433,"5min":0.372,"15min":0.34399999999999997},"max":{"1min":0.78800000000000003,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.503},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.57999999999999996,"15min":0.56899999999999995},"min":{"1min":0.40799999999999997,"5min":0.39200000000000002,"15min":0.36299999999999999},"max":{"1min":0.70899999999999996,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.42799999999999999}]},{"osd":5,"last update":"Sat May 20 01:24:32 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.57799999999999996,"15min":0.57599999999999996},"min":{"1min":0.379,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.74299999999999999,"5min":0.80500000000000005,"15min":0.88100000000000001},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.63400000000000001,"15min":0.58899999999999997},"min":{"1min":0.40899999999999997,"5min":0.374,"15min":0.374},"max":{"1min":0.76300000000000001,"5min":4.4720000000000004,"15min":4.4720000000000004},"last":0.623}]},{"osd":6,"last update":"Sat May 20 01:24:32 2023","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":0.72799999999999998,"15min":0.625},"min":{"1min":0.45800000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":10.042999999999999,"5min":10.042999999999999,"15min":10.042999999999999},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.60499999999999998,"15min":0.61599999999999999},"min":{"1min":0.45200000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.73399999999999999,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.40200000000000002}]},{"osd":7,"last update":"Sat May 20 01:24:32 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.59499999999999997,"15min":0.59299999999999997},"min":{"1min":0.433,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.74199999999999999,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.57499999999999996},{"interface":"front","average":{"1min":1.1040000000000001,"5min":0.72499999999999998,"15min":0.65700000000000003},"min":{"1min":0.44800000000000001,"5min":0.438,"15min":0.42099999999999999},"max":{"1min":10.085000000000001,"5min":10.085000000000001,"15min":10.085000000000001},"last":0.53600000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574957,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10292,"kb_used_data":1004,"kb_used_omap":0,"kb_used_meta":9280,"kb_avail":94361548,"statfs":{"total":96636764160,"available":96626225152,"internally_reserved":0,"allocated":1028096,"data_stored":516144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9502720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:24:44 2023","interfaces":[{"interface":"back","average":{"1min":0.255,"5min":0.27500000000000002,"15min":0.32400000000000001},"min":{"1min":0.19500000000000001,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":0.36799999999999999,"5min":0.53400000000000003,"15min":1.038},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.26700000000000002,"5min":0.29699999999999999,"15min":0.34200000000000003},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.36799999999999999,"5min":0.46500000000000002,"15min":1.079},"last":0.23899999999999999}]},{"osd":1,"last update":"Sat May 20 01:24:44 2023","interfaces":[{"interface":"back","average":{"1min":0.28999999999999998,"5min":0.314,"15min":0.38100000000000001},"min":{"1min":0.23400000000000001,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.34899999999999998,"5min":0.61399999999999999,"15min":1.4950000000000001},"last":0.434},{"interface":"front","average":{"1min":0.318,"5min":0.35499999999999998,"15min":0.41799999999999998},"min":{"1min":0.27400000000000002,"5min":0.249,"15min":0.249},"max":{"1min":0.36099999999999999,"5min":0.629,"15min":1.5189999999999999},"last":0.313}]},{"osd":3,"last update":"Sat May 20 01:24:44 2023","interfaces":[{"interface":"back","average":{"1min":0.27800000000000002,"5min":0.32400000000000001,"15min":0.38200000000000001},"min":{"1min":0.218,"5min":0.216,"15min":0.216},"max":{"1min":0.42499999999999999,"5min":0.60599999999999998,"15min":1.2090000000000001},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.34000000000000002,"5min":0.36699999999999999,"15min":0.432},"min":{"1min":0.30199999999999999,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.38800000000000001,"5min":0.68100000000000005,"15min":1.5620000000000001},"last":0.38900000000000001}]},{"osd":4,"last update":"Sat May 20 01:24:44 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.55500000000000005,"15min":0.57699999999999996},"min":{"1min":0.38800000000000001,"5min":0.35899999999999999,"15min":0.34899999999999998},"max":{"1min":0.69199999999999995,"5min":0.85199999999999998,"15min":1.319},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.58499999999999996,"15min":0.61199999999999999},"min":{"1min":0.432,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.69499999999999995,"5min":0.89000000000000001,"15min":1.4319999999999999},"last":0.51400000000000001}]},{"osd":5,"last update":"Sat May 20 01:24:44 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.55800000000000005,"15min":0.62},"min":{"1min":0.41999999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.73599999999999999,"5min":0.95599999999999996,"15min":1.758},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.58299999999999996,"15min":0.63800000000000001},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.72999999999999998,"5min":0.85099999999999998,"15min":1.7110000000000001},"last":0.66900000000000004}]},{"osd":6,"last update":"Sat May 20 01:24:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.58599999999999997,"15min":0.63200000000000001},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.39700000000000002},"max":{"1min":0.746,"5min":0.88900000000000001,"15min":1.665},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.58299999999999996,"15min":0.63200000000000001},"min":{"1min":0.41699999999999998,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.74299999999999999,"5min":0.93500000000000005,"15min":1.819},"last":0.48999999999999999}]},{"osd":7,"last update":"Sat May 20 01:24:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.59499999999999997,"15min":0.65700000000000003},"min":{"1min":0.44400000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.76900000000000002,"5min":0.85099999999999998,"15min":1.7310000000000001},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.59899999999999998,"15min":0.65300000000000002},"min":{"1min":0.46999999999999997,"5min":0.42399999999999999,"15min":0.42199999999999999},"max":{"1min":0.72499999999999998,"5min":0.90600000000000003,"15min":1.8460000000000001},"last":0.52800000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574957,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10420,"kb_used_data":1004,"kb_used_omap":0,"kb_used_meta":9408,"kb_avail":94361420,"statfs":{"total":96636764160,"available":96626094080,"internally_reserved":0,"allocated":1028096,"data_stored":516144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9633792},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:24:32 2023","interfaces":[{"interface":"back","average":{"1min":0.27300000000000002,"5min":0.30599999999999999,"15min":0.317},"min":{"1min":0.21199999999999999,"5min":0.188,"15min":0.188},"max":{"1min":0.375,"5min":1.528,"15min":1.528},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.28000000000000003,"5min":0.34200000000000003,"15min":0.42299999999999999},"min":{"1min":0.214,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.40699999999999997,"5min":2.4209999999999998,"15min":3.1890000000000001},"last":0.35799999999999998}]},{"osd":1,"last update":"Sat May 20 01:24:32 2023","interfaces":[{"interface":"back","average":{"1min":0.27300000000000002,"5min":0.33600000000000002,"15min":0.41499999999999998},"min":{"1min":0.19900000000000001,"5min":0.189,"15min":0.189},"max":{"1min":0.377,"5min":3.2999999999999998,"15min":3.4260000000000002},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.33200000000000002,"5min":0.34200000000000003,"15min":0.35599999999999998},"min":{"1min":0.23799999999999999,"5min":0.22900000000000001,"15min":0.221},"max":{"1min":0.45500000000000002,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.32300000000000001}]},{"osd":2,"last update":"Sat May 20 01:24:32 2023","interfaces":[{"interface":"back","average":{"1min":0.30599999999999999,"5min":0.34300000000000003,"15min":0.38700000000000001},"min":{"1min":0.26400000000000001,"5min":0.245,"15min":0.224},"max":{"1min":0.39200000000000002,"5min":0.625,"15min":1.1619999999999999},"last":0.503},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.371,"15min":0.41299999999999998},"min":{"1min":0.28499999999999998,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.41499999999999998,"5min":0.72399999999999998,"15min":1.413},"last":0.35099999999999998}]},{"osd":4,"last update":"Sat May 20 01:24:32 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.55100000000000005,"15min":0.58699999999999997},"min":{"1min":0.40300000000000002,"5min":0.38600000000000001,"15min":0.38200000000000001},"max":{"1min":0.72899999999999998,"5min":0.95399999999999996,"15min":1.0700000000000001},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.61199999999999999,"15min":0.68000000000000005},"min":{"1min":0.41099999999999998,"5min":0.40200000000000002,"15min":0.35899999999999999},"max":{"1min":0.72899999999999998,"5min":3.1789999999999998,"15min":3.4660000000000002},"last":0.58399999999999996}]},{"osd":5,"last update":"Sat May 20 01:24:32 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.59899999999999998,"15min":0.68500000000000005},"min":{"1min":0.38400000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.751,"5min":3.2170000000000001,"15min":3.5379999999999998},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.59999999999999998,"15min":0.63200000000000001},"min":{"1min":0.41999999999999998,"5min":0.40799999999999997,"15min":0.374},"max":{"1min":0.74199999999999999,"5min":1.026,"15min":1.385},"last":0.46700000000000003}]},{"osd":6,"last update":"Sat May 20 01:24:32 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.59599999999999997,"15min":0.63400000000000001},"min":{"1min":0.439,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.70199999999999996,"5min":0.99199999999999999,"15min":1.347},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.58499999999999996,"15min":0.63},"min":{"1min":0.39500000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.752,"5min":0.94099999999999995,"15min":1.107},"last":0.54000000000000004}]},{"osd":7,"last update":"Sat May 20 01:24:32 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.59699999999999998,"15min":0.63700000000000001},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.64000000000000001,"5min":0.94199999999999995,"15min":1.1359999999999999},"last":0.439},{"interface":"front","average":{"1min":0.625,"5min":0.65100000000000002,"15min":0.72499999999999998},"min":{"1min":0.47599999999999998,"5min":0.436,"15min":0.436},"max":{"1min":0.73799999999999999,"5min":3.2559999999999998,"15min":3.5009999999999999},"last":0.60099999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574957,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":15732,"kb_used_data":1004,"kb_used_omap":0,"kb_used_meta":14720,"kb_avail":94356108,"statfs":{"total":96636764160,"available":96620654592,"internally_reserved":0,"allocated":1028096,"data_stored":516144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":15073280},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:24:33 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.47299999999999998,"15min":0.48099999999999998},"min":{"1min":0.32200000000000001,"5min":0.247,"15min":0.247},"max":{"1min":0.59199999999999997,"5min":0.73699999999999999,"15min":0.75600000000000001},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.50700000000000001,"15min":0.498},"min":{"1min":0.30599999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.64800000000000002,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":0.44900000000000001}]},{"osd":1,"last update":"Sat May 20 01:24:33 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.49199999999999999,"15min":0.51900000000000002},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.65300000000000002,"5min":0.746,"15min":0.82799999999999996},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.51900000000000002,"15min":0.53000000000000003},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.63600000000000001,"5min":1.7190000000000001,"15min":2.0739999999999998},"last":0.42399999999999999}]},{"osd":2,"last update":"Sat May 20 01:24:33 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.51900000000000002,"15min":0.50800000000000001},"min":{"1min":0.36599999999999999,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.64900000000000002,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.54200000000000004,"15min":0.55000000000000004},"min":{"1min":0.378,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.61399999999999999,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.51100000000000001}]},{"osd":3,"last update":"Sat May 20 01:24:33 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.52600000000000002,"15min":0.52400000000000002},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34399999999999997},"max":{"1min":0.626,"5min":0.81399999999999995,"15min":1.827},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.57099999999999995,"15min":0.65500000000000003},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.65000000000000002,"5min":0.89300000000000002,"15min":3.1749999999999998},"last":0.57599999999999996}]},{"osd":5,"last update":"Sat May 20 01:24:33 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.53400000000000003,"15min":0.48699999999999999},"min":{"1min":0.36899999999999999,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.76700000000000002,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.55400000000000005,"15min":0.51300000000000001},"min":{"1min":0.35099999999999998,"5min":0.33400000000000002,"15min":0.29899999999999999},"max":{"1min":0.78400000000000003,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.58899999999999997}]},{"osd":6,"last update":"Sat May 20 01:24:33 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.58799999999999997,"15min":0.54700000000000004},"min":{"1min":0.432,"5min":0.40400000000000003,"15min":0.32400000000000001},"max":{"1min":0.80300000000000005,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.61599999999999999,"15min":0.59099999999999997},"min":{"1min":0.51100000000000001,"5min":0.42399999999999999,"15min":0.39400000000000002},"max":{"1min":0.72299999999999998,"5min":0.84699999999999998,"15min":0.85899999999999999},"last":0.60799999999999998}]},{"osd":7,"last update":"Sat May 20 01:24:33 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.57799999999999996,"15min":0.55700000000000005},"min":{"1min":0.41999999999999998,"5min":0.36099999999999999,"15min":0.34200000000000003},"max":{"1min":0.71799999999999997,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.61399999999999999,"15min":0.59799999999999998},"min":{"1min":0.499,"5min":0.36399999999999999,"15min":0.34000000000000002},"max":{"1min":0.73699999999999999,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.65400000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574958,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10160,"kb_used_data":1000,"kb_used_omap":0,"kb_used_meta":9152,"kb_avail":94361680,"statfs":{"total":96636764160,"available":96626360320,"internally_reserved":0,"allocated":1024000,"data_stored":515183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9371648},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:24:38 2023","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.45400000000000001,"15min":0.65300000000000002},"min":{"1min":0.29999999999999999,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.68899999999999995,"5min":0.72599999999999998,"15min":6.3929999999999998},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.46000000000000002,"15min":0.68000000000000005},"min":{"1min":0.34999999999999998,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.61099999999999999,"5min":0.67500000000000004,"15min":6.3339999999999996},"last":0.499}]},{"osd":1,"last update":"Sat May 20 01:24:38 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.46000000000000002,"15min":0.58799999999999997},"min":{"1min":0.313,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":3.2450000000000001},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.498,"5min":0.499,"15min":0.53400000000000003},"min":{"1min":0.38900000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.72599999999999998,"5min":0.752,"15min":1.1819999999999999},"last":0.48199999999999998}]},{"osd":2,"last update":"Sat May 20 01:24:38 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48299999999999998,"15min":0.54100000000000004},"min":{"1min":0.33200000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":1.1639999999999999},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.48399999999999999,"15min":0.54400000000000004},"min":{"1min":0.308,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.628,"5min":0.71799999999999997,"15min":1.258},"last":0.58799999999999997}]},{"osd":3,"last update":"Sat May 20 01:24:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.51500000000000001,"15min":0.72499999999999998},"min":{"1min":0.35099999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.69499999999999995,"5min":0.77800000000000002,"15min":6.351},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.53200000000000003,"15min":0.75800000000000001},"min":{"1min":0.40699999999999997,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.67500000000000004,"5min":0.76500000000000001,"15min":6.3730000000000002},"last":0.55400000000000005}]},{"osd":4,"last update":"Sat May 20 01:24:38 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.503,"15min":0.49299999999999999},"min":{"1min":0.38900000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.64500000000000002,"5min":0.68400000000000005,"15min":0.70199999999999996},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.54000000000000004,"15min":0.55100000000000005},"min":{"1min":0.41599999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.749,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.41399999999999998}]},{"osd":6,"last update":"Sat May 20 01:24:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.54600000000000004,"15min":0.55200000000000005},"min":{"1min":0.48699999999999999,"5min":0.38100000000000001,"15min":0.29999999999999999},"max":{"1min":0.72899999999999998,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.54500000000000004,"15min":0.55800000000000005},"min":{"1min":0.437,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.72399999999999998,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.51200000000000001}]},{"osd":7,"last update":"Sat May 20 01:24:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.56799999999999995,"15min":0.58199999999999996},"min":{"1min":0.46500000000000002,"5min":0.33400000000000002,"15min":0.33000000000000002},"max":{"1min":0.77300000000000002,"5min":0.82299999999999995,"15min":0.91000000000000003},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.57699999999999996,"15min":0.60799999999999998},"min":{"1min":0.44700000000000001,"5min":0.40999999999999998,"15min":0.39500000000000002},"max":{"1min":0.76200000000000001,"5min":0.81399999999999995,"15min":0.94499999999999995},"last":0.61299999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-05-20T01:25:16.542 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-05-20T01:25:16.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.696+0000 7f704eaa9700 1 Processor -- start 2023-05-20T01:25:16.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.696+0000 7f704eaa9700 1 -- start start 2023-05-20T01:25:16.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.697+0000 7f704eaa9700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f704806f450 0x7f70480cf890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:16.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.697+0000 7f704eaa9700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70480cc510 con 0x7f704806d4b0 2023-05-20T01:25:16.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.697+0000 7f704eaa9700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7048071ed0 con 0x7f704806a510 2023-05-20T01:25:16.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.697+0000 7f704eaa9700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70480721b0 con 0x7f704806f450 2023-05-20T01:25:16.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.697+0000 7f7047fff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f704806d4b0 0x7f7048161080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:47798/0 (socket says 172.21.15.16:47798) 2023-05-20T01:25:16.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.697+0000 7f7047fff700 1 -- 172.21.15.16:0/460042169 learned_addr learned my addr 172.21.15.16:0/460042169 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:25:16.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.697+0000 7f704c845700 1 --2- 172.21.15.16:0/460042169 >> v2:172.21.15.132:3300/0 conn(0x7f704806f450 0x7f70480cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:16.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.697+0000 7f70477fe700 1 -- 172.21.15.16:0/460042169 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 215898814 0 0) 0x7f70480cc510 con 0x7f704806d4b0 2023-05-20T01:25:16.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.697+0000 7f70477fe700 1 -- 172.21.15.16:0/460042169 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f70380035b0 con 0x7f704806d4b0 2023-05-20T01:25:16.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.697+0000 7f70477fe700 1 -- 172.21.15.16:0/460042169 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 705459584 0 0) 0x7f70380035b0 con 0x7f704806d4b0 2023-05-20T01:25:16.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.697+0000 7f70477fe700 1 -- 172.21.15.16:0/460042169 >> v2:172.21.15.132:3300/0 conn(0x7f704806f450 msgr2=0x7f70480cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:16.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.697+0000 7f70477fe700 1 --2- 172.21.15.16:0/460042169 >> v2:172.21.15.132:3300/0 conn(0x7f704806f450 0x7f70480cf890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:16.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.697+0000 7f70477fe700 1 -- 172.21.15.16:0/460042169 >> v1:172.21.15.132:6789/0 conn(0x7f704806a510 legacy=0x7f70480cc9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:16.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.697+0000 7f70477fe700 1 -- 172.21.15.16:0/460042169 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70480cc1a0 con 0x7f704806d4b0 2023-05-20T01:25:16.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.697+0000 7f70477fe700 1 -- 172.21.15.16:0/460042169 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7f7034001670 con 0x7f704806d4b0 2023-05-20T01:25:16.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.697+0000 7f70477fe700 1 -- 172.21.15.16:0/460042169 >> v1:172.21.15.16:6789/0 conn(0x7f704806d4b0 legacy=0x7f7048161080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:16.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.697+0000 7f70477fe700 1 --2- 172.21.15.16:0/460042169 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7038003a70 0x7f7038003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:16.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.697+0000 7f70477fe700 1 --2- 172.21.15.16:0/460042169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7038004850 0x7f7038004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:16.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.697+0000 7f70477fe700 1 --2- 172.21.15.16:0/460042169 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7038005250 0x7f703800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:16.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.697+0000 7f70477fe700 1 -- 172.21.15.16:0/460042169 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70480721b0 con 0x7f7038004850 2023-05-20T01:25:16.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.697+0000 7f70477fe700 1 -- 172.21.15.16:0/460042169 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f703800c190 con 0x7f7038005250 2023-05-20T01:25:16.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.697+0000 7f70477fe700 1 -- 172.21.15.16:0/460042169 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f703800c350 con 0x7f7038003a70 2023-05-20T01:25:16.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.697+0000 7f704c845700 1 --2- 172.21.15.16:0/460042169 >> v2:172.21.15.132:3300/0 conn(0x7f704806f450 0x7f70480cf890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-05-20T01:25:16.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.698+0000 7f7047fff700 1 --2- 172.21.15.16:0/460042169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7038004850 0x7f7038004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:16.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.698+0000 7f704d046700 1 --2- 172.21.15.16:0/460042169 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7038005250 0x7f703800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:16.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.698+0000 7f704c845700 1 --2- 172.21.15.16:0/460042169 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7038003a70 0x7f7038003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:16.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.698+0000 7f7047fff700 1 -- 172.21.15.16:0/460042169 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7038003a70 msgr2=0x7f7038003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:16.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.698+0000 7f7047fff700 1 --2- 172.21.15.16:0/460042169 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7038003a70 0x7f7038003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:16.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.698+0000 7f7047fff700 1 -- 172.21.15.16:0/460042169 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7038005250 msgr2=0x7f703800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:16.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.698+0000 7f7047fff700 1 --2- 172.21.15.16:0/460042169 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7038005250 0x7f703800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:16.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.698+0000 7f704d046700 1 -- 172.21.15.16:0/460042169 reap_dead start 2023-05-20T01:25:16.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.698+0000 7f7047fff700 1 -- 172.21.15.16:0/460042169 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70380035b0 con 0x7f7038004850 2023-05-20T01:25:16.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.698+0000 7f704c845700 1 --2- 172.21.15.16:0/460042169 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7038003a70 0x7f7038003e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:25:16.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.698+0000 7f7047fff700 1 --2- 172.21.15.16:0/460042169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7038004850 0x7f7038004cc0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f703400c210 tx=0x7f7038005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=451aa82b1dc3eae9 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:16.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.698+0000 7f70477fe700 1 -- 172.21.15.16:0/460042169 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7034007c10 con 0x7f7038004850 2023-05-20T01:25:16.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.698+0000 7f70477fe700 1 -- 172.21.15.16:0/460042169 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f703400cdd0 con 0x7f7038004850 2023-05-20T01:25:16.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.698+0000 7f70477fe700 1 -- 172.21.15.16:0/460042169 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7034005ba0 con 0x7f7038004850 2023-05-20T01:25:16.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.699+0000 7f704eaa9700 1 -- 172.21.15.16:0/460042169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7038004850 msgr2=0x7f7038004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:16.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.699+0000 7f704eaa9700 1 --2- 172.21.15.16:0/460042169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7038004850 0x7f7038004cc0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f703400c210 tx=0x7f7038005880 comp rx=0 tx=0).stop 2023-05-20T01:25:16.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.699+0000 7f704eaa9700 1 -- 172.21.15.16:0/460042169 shutdown_connections 2023-05-20T01:25:16.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.699+0000 7f704eaa9700 1 --2- 172.21.15.16:0/460042169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7038004850 0x7f7038004cc0 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:16.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.699+0000 7f704eaa9700 1 -- 172.21.15.16:0/460042169 >> 172.21.15.16:0/460042169 conn(0x7f70480cffb0 msgr2=0x7f70480d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:16.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.699+0000 7f704eaa9700 1 -- 172.21.15.16:0/460042169 shutdown_connections 2023-05-20T01:25:16.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.699+0000 7f704eaa9700 1 -- 172.21.15.16:0/460042169 wait complete. 2023-05-20T01:25:16.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.700+0000 7f704eaa9700 1 Processor -- start 2023-05-20T01:25:16.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.700+0000 7f704eaa9700 1 -- start start 2023-05-20T01:25:16.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.701+0000 7f704eaa9700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7038004850 0x7f70480c4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:16.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.701+0000 7f704eaa9700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f70480b3f70 0x7f70480b43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:16.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.701+0000 7f704eaa9700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7048146f90 0x7f7048147400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:16.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.701+0000 7f704eaa9700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7048147940 con 0x7f7038004850 2023-05-20T01:25:16.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.701+0000 7f704eaa9700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70481f3520 con 0x7f70480b3f70 2023-05-20T01:25:16.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.701+0000 7f704eaa9700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70481f2910 con 0x7f7048146f90 2023-05-20T01:25:16.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.701+0000 7f704c845700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7038004850 0x7f70480c4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:16.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.701+0000 7f704c845700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7038004850 0x7f70480c4f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36280/0 (socket says 172.21.15.16:36280) 2023-05-20T01:25:16.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.701+0000 7f704c845700 1 -- 172.21.15.16:0/3762546873 learned_addr learned my addr 172.21.15.16:0/3762546873 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:16.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.701+0000 7f704c845700 1 -- 172.21.15.16:0/3762546873 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7048146f90 msgr2=0x7f7048147400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:16.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.701+0000 7f704c845700 1 --2- 172.21.15.16:0/3762546873 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7048146f90 0x7f7048147400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:16.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.701+0000 7f704c845700 1 -- 172.21.15.16:0/3762546873 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f70480b3f70 msgr2=0x7f70480b43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:16.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.701+0000 7f704c845700 1 --2- 172.21.15.16:0/3762546873 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f70480b3f70 0x7f70480b43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:16.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.701+0000 7f704c845700 1 -- 172.21.15.16:0/3762546873 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70480cf890 con 0x7f7038004850 2023-05-20T01:25:16.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.701+0000 7f704c845700 1 --2- 172.21.15.16:0/3762546873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7038004850 0x7f70480c4f00 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f703c011830 tx=0x7f703c011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:16.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.701+0000 7f70457fa700 1 -- 172.21.15.16:0/3762546873 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f703c015870 con 0x7f7038004850 2023-05-20T01:25:16.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.702+0000 7f70457fa700 1 -- 172.21.15.16:0/3762546873 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f703c0159d0 con 0x7f7038004850 2023-05-20T01:25:16.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.702+0000 7f704eaa9700 1 -- 172.21.15.16:0/3762546873 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70480cbe50 con 0x7f7038004850 2023-05-20T01:25:16.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.702+0000 7f70457fa700 1 -- 172.21.15.16:0/3762546873 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f703c015d40 con 0x7f7038004850 2023-05-20T01:25:16.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.702+0000 7f704eaa9700 1 -- 172.21.15.16:0/3762546873 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70480cfa20 con 0x7f7038004850 2023-05-20T01:25:16.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.703+0000 7f704eaa9700 1 -- 172.21.15.16:0/3762546873 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70480747b0 con 0x7f7038004850 2023-05-20T01:25:16.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.703+0000 7f70457fa700 1 -- 172.21.15.16:0/3762546873 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f703c01d400 con 0x7f7038004850 2023-05-20T01:25:16.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.704+0000 7f70457fa700 1 --2- 172.21.15.16:0/3762546873 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f702c06ace0 0x7f702c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:16.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.704+0000 7f70457fa700 1 -- 172.21.15.16:0/3762546873 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f703c089e10 con 0x7f7038004850 2023-05-20T01:25:16.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.704+0000 7f7047fff700 1 --2- 172.21.15.16:0/3762546873 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f702c06ace0 0x7f702c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:16.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.704+0000 7f7047fff700 1 --2- 172.21.15.16:0/3762546873 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f702c06ace0 0x7f702c06d190 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f704815c900 tx=0x7f703400a000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:16.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.708+0000 7f70457fa700 1 -- 172.21.15.16:0/3762546873 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f703c054d80 con 0x7f7038004850 2023-05-20T01:25:16.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.900+0000 7f704eaa9700 1 -- 172.21.15.16:0/3762546873 --> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7048071c10 con 0x7f702c06ace0 2023-05-20T01:25:16.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.908+0000 7f70457fa700 1 -- 172.21.15.16:0/3762546873 <== mgr.4104 v2:172.21.15.16:6832/3817237998 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+296973 (secure 0 0 0) 0x7f7048071c10 con 0x7f702c06ace0 2023-05-20T01:25:16.916 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:25:16.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.912+0000 7f704eaa9700 1 -- 172.21.15.16:0/3762546873 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f702c06ace0 msgr2=0x7f702c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:16.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.912+0000 7f704eaa9700 1 --2- 172.21.15.16:0/3762546873 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f702c06ace0 0x7f702c06d190 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f704815c900 tx=0x7f703400a000 comp rx=0 tx=0).stop 2023-05-20T01:25:16.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.912+0000 7f704eaa9700 1 -- 172.21.15.16:0/3762546873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7038004850 msgr2=0x7f70480c4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:16.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.912+0000 7f704eaa9700 1 --2- 172.21.15.16:0/3762546873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7038004850 0x7f70480c4f00 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f703c011830 tx=0x7f703c011bf0 comp rx=0 tx=0).stop 2023-05-20T01:25:16.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.912+0000 7f704eaa9700 1 -- 172.21.15.16:0/3762546873 shutdown_connections 2023-05-20T01:25:16.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.912+0000 7f704eaa9700 1 --2- 172.21.15.16:0/3762546873 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f702c06ace0 0x7f702c06d190 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:16.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.912+0000 7f704eaa9700 1 --2- 172.21.15.16:0/3762546873 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7048146f90 0x7f7048147400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:16.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.912+0000 7f704eaa9700 1 --2- 172.21.15.16:0/3762546873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7038004850 0x7f70480c4f00 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:16.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.912+0000 7f704eaa9700 1 --2- 172.21.15.16:0/3762546873 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f70480b3f70 0x7f70480b43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:16.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.912+0000 7f704eaa9700 1 -- 172.21.15.16:0/3762546873 >> 172.21.15.16:0/3762546873 conn(0x7f70480cffb0 msgr2=0x7f70480cc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:16.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.912+0000 7f704eaa9700 1 -- 172.21.15.16:0/3762546873 shutdown_connections 2023-05-20T01:25:16.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:16.912+0000 7f704eaa9700 1 -- 172.21.15.16:0/3762546873 wait complete. 2023-05-20T01:25:16.919 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2023-05-20T01:25:16.928 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":349,"stamp":"2023-05-20T01:25:15.434324+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":94944,"kb_used_data":9184,"kb_used_omap":0,"kb_used_meta":85696,"kb_avail":754879776,"statfs":{"total":773094113280,"available":772996890624,"internally_reserved":0,"allocated":9404416,"data_stored":5307966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":87752704},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.881661"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:10.545755+0000","last_change":"2023-05-20T01:25:10.545755+0000","last_active":"2023-05-20T01:25:10.545755+0000","last_peered":"2023-05-20T01:25:10.545755+0000","last_clean":"2023-05-20T01:25:10.545755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:10.545755+0000","last_undegraded":"2023-05-20T01:25:10.545755+0000","last_fullsized":"2023-05-20T01:25:10.545755+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:10.545755+0000","last_change":"2023-05-20T01:25:10.545755+0000","last_active":"2023-05-20T01:25:10.545755+0000","last_peered":"2023-05-20T01:25:10.545755+0000","last_clean":"2023-05-20T01:25:10.545755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:10.545755+0000","last_undegraded":"2023-05-20T01:25:10.545755+0000","last_fullsized":"2023-05-20T01:25:10.545755+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:10.545755+0000","last_change":"2023-05-20T01:25:10.545755+0000","last_active":"2023-05-20T01:25:10.545755+0000","last_peered":"2023-05-20T01:25:10.545755+0000","last_clean":"2023-05-20T01:25:10.545755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:10.545755+0000","last_undegraded":"2023-05-20T01:25:10.545755+0000","last_fullsized":"2023-05-20T01:25:10.545755+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:10.545755+0000","last_change":"2023-05-20T01:25:10.545755+0000","last_active":"2023-05-20T01:25:10.545755+0000","last_peered":"2023-05-20T01:25:10.545755+0000","last_clean":"2023-05-20T01:25:10.545755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:10.545755+0000","last_undegraded":"2023-05-20T01:25:10.545755+0000","last_fullsized":"2023-05-20T01:25:10.545755+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:10.545755+0000","last_change":"2023-05-20T01:25:10.545755+0000","last_active":"2023-05-20T01:25:10.545755+0000","last_peered":"2023-05-20T01:25:10.545755+0000","last_clean":"2023-05-20T01:25:10.545755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:10.545755+0000","last_undegraded":"2023-05-20T01:25:10.545755+0000","last_fullsized":"2023-05-20T01:25:10.545755+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:10.545755+0000","last_change":"2023-05-20T01:25:10.545755+0000","last_active":"2023-05-20T01:25:10.545755+0000","last_peered":"2023-05-20T01:25:10.545755+0000","last_clean":"2023-05-20T01:25:10.545755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:10.545755+0000","last_undegraded":"2023-05-20T01:25:10.545755+0000","last_fullsized":"2023-05-20T01:25:10.545755+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:10.545755+0000","last_change":"2023-05-20T01:25:10.545755+0000","last_active":"2023-05-20T01:25:10.545755+0000","last_peered":"2023-05-20T01:25:10.545755+0000","last_clean":"2023-05-20T01:25:10.545755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:10.545755+0000","last_undegraded":"2023-05-20T01:25:10.545755+0000","last_fullsized":"2023-05-20T01:25:10.545755+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:10.545755+0000","last_change":"2023-05-20T01:25:10.545755+0000","last_active":"2023-05-20T01:25:10.545755+0000","last_peered":"2023-05-20T01:25:10.545755+0000","last_clean":"2023-05-20T01:25:10.545755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:10.545755+0000","last_undegraded":"2023-05-20T01:25:10.545755+0000","last_fullsized":"2023-05-20T01:25:10.545755+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:10.545755+0000","last_change":"2023-05-20T01:25:10.545755+0000","last_active":"2023-05-20T01:25:10.545755+0000","last_peered":"2023-05-20T01:25:10.545755+0000","last_clean":"2023-05-20T01:25:10.545755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:10.545755+0000","last_undegraded":"2023-05-20T01:25:10.545755+0000","last_fullsized":"2023-05-20T01:25:10.545755+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:10.545755+0000","last_change":"2023-05-20T01:25:10.545755+0000","last_active":"2023-05-20T01:25:10.545755+0000","last_peered":"2023-05-20T01:25:10.545755+0000","last_clean":"2023-05-20T01:25:10.545755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:10.545755+0000","last_undegraded":"2023-05-20T01:25:10.545755+0000","last_fullsized":"2023-05-20T01:25:10.545755+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:10.545755+0000","last_change":"2023-05-20T01:25:10.545755+0000","last_active":"2023-05-20T01:25:10.545755+0000","last_peered":"2023-05-20T01:25:10.545755+0000","last_clean":"2023-05-20T01:25:10.545755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:10.545755+0000","last_undegraded":"2023-05-20T01:25:10.545755+0000","last_fullsized":"2023-05-20T01:25:10.545755+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:10.545755+0000","last_change":"2023-05-20T01:25:10.545755+0000","last_active":"2023-05-20T01:25:10.545755+0000","last_peered":"2023-05-20T01:25:10.545755+0000","last_clean":"2023-05-20T01:25:10.545755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:10.545755+0000","last_undegraded":"2023-05-20T01:25:10.545755+0000","last_fullsized":"2023-05-20T01:25:10.545755+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:10.545755+0000","last_change":"2023-05-20T01:25:10.545755+0000","last_active":"2023-05-20T01:25:10.545755+0000","last_peered":"2023-05-20T01:25:10.545755+0000","last_clean":"2023-05-20T01:25:10.545755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:10.545755+0000","last_undegraded":"2023-05-20T01:25:10.545755+0000","last_fullsized":"2023-05-20T01:25:10.545755+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:10.545755+0000","last_change":"2023-05-20T01:25:10.545755+0000","last_active":"2023-05-20T01:25:10.545755+0000","last_peered":"2023-05-20T01:25:10.545755+0000","last_clean":"2023-05-20T01:25:10.545755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:10.545755+0000","last_undegraded":"2023-05-20T01:25:10.545755+0000","last_fullsized":"2023-05-20T01:25:10.545755+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:10.545755+0000","last_change":"2023-05-20T01:25:10.545755+0000","last_active":"2023-05-20T01:25:10.545755+0000","last_peered":"2023-05-20T01:25:10.545755+0000","last_clean":"2023-05-20T01:25:10.545755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:10.545755+0000","last_undegraded":"2023-05-20T01:25:10.545755+0000","last_fullsized":"2023-05-20T01:25:10.545755+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:10.545755+0000","last_change":"2023-05-20T01:25:10.545755+0000","last_active":"2023-05-20T01:25:10.545755+0000","last_peered":"2023-05-20T01:25:10.545755+0000","last_clean":"2023-05-20T01:25:10.545755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:10.545755+0000","last_undegraded":"2023-05-20T01:25:10.545755+0000","last_fullsized":"2023-05-20T01:25:10.545755+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:10.545755+0000","last_change":"2023-05-20T01:25:10.545755+0000","last_active":"2023-05-20T01:25:10.545755+0000","last_peered":"2023-05-20T01:25:10.545755+0000","last_clean":"2023-05-20T01:25:10.545755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:10.545755+0000","last_undegraded":"2023-05-20T01:25:10.545755+0000","last_fullsized":"2023-05-20T01:25:10.545755+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:10.545755+0000","last_change":"2023-05-20T01:25:10.545755+0000","last_active":"2023-05-20T01:25:10.545755+0000","last_peered":"2023-05-20T01:25:10.545755+0000","last_clean":"2023-05-20T01:25:10.545755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:10.545755+0000","last_undegraded":"2023-05-20T01:25:10.545755+0000","last_fullsized":"2023-05-20T01:25:10.545755+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:10.545755+0000","last_change":"2023-05-20T01:25:10.545755+0000","last_active":"2023-05-20T01:25:10.545755+0000","last_peered":"2023-05-20T01:25:10.545755+0000","last_clean":"2023-05-20T01:25:10.545755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:10.545755+0000","last_undegraded":"2023-05-20T01:25:10.545755+0000","last_fullsized":"2023-05-20T01:25:10.545755+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:10.545755+0000","last_change":"2023-05-20T01:25:10.545755+0000","last_active":"2023-05-20T01:25:10.545755+0000","last_peered":"2023-05-20T01:25:10.545755+0000","last_clean":"2023-05-20T01:25:10.545755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:10.545755+0000","last_undegraded":"2023-05-20T01:25:10.545755+0000","last_fullsized":"2023-05-20T01:25:10.545755+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:10.545755+0000","last_change":"2023-05-20T01:25:10.545755+0000","last_active":"2023-05-20T01:25:10.545755+0000","last_peered":"2023-05-20T01:25:10.545755+0000","last_clean":"2023-05-20T01:25:10.545755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:10.545755+0000","last_undegraded":"2023-05-20T01:25:10.545755+0000","last_fullsized":"2023-05-20T01:25:10.545755+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:10.545755+0000","last_change":"2023-05-20T01:25:10.545755+0000","last_active":"2023-05-20T01:25:10.545755+0000","last_peered":"2023-05-20T01:25:10.545755+0000","last_clean":"2023-05-20T01:25:10.545755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:10.545755+0000","last_undegraded":"2023-05-20T01:25:10.545755+0000","last_fullsized":"2023-05-20T01:25:10.545755+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:10.545755+0000","last_change":"2023-05-20T01:25:10.545755+0000","last_active":"2023-05-20T01:25:10.545755+0000","last_peered":"2023-05-20T01:25:10.545755+0000","last_clean":"2023-05-20T01:25:10.545755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:10.545755+0000","last_undegraded":"2023-05-20T01:25:10.545755+0000","last_fullsized":"2023-05-20T01:25:10.545755+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:10.545755+0000","last_change":"2023-05-20T01:25:10.545755+0000","last_active":"2023-05-20T01:25:10.545755+0000","last_peered":"2023-05-20T01:25:10.545755+0000","last_clean":"2023-05-20T01:25:10.545755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:10.545755+0000","last_undegraded":"2023-05-20T01:25:10.545755+0000","last_fullsized":"2023-05-20T01:25:10.545755+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:10.545755+0000","last_change":"2023-05-20T01:25:10.545755+0000","last_active":"2023-05-20T01:25:10.545755+0000","last_peered":"2023-05-20T01:25:10.545755+0000","last_clean":"2023-05-20T01:25:10.545755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:10.545755+0000","last_undegraded":"2023-05-20T01:25:10.545755+0000","last_fullsized":"2023-05-20T01:25:10.545755+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":143,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-05-20T01:25:07.873873+0000","last_change":"2023-05-20T01:16:23.289068+0000","last_active":"2023-05-20T01:25:07.873873+0000","last_peered":"2023-05-20T01:25:07.873873+0000","last_clean":"2023-05-20T01:25:07.873873+0000","last_became_active":"2023-05-20T01:16:23.288781+0000","last_became_peered":"2023-05-20T01:16:23.288781+0000","last_unstale":"2023-05-20T01:25:07.873873+0000","last_undegraded":"2023-05-20T01:25:07.873873+0000","last_fullsized":"2023-05-20T01:25:07.873873+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:48:00.157178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:10.545755+0000","last_change":"2023-05-20T01:25:10.545755+0000","last_active":"2023-05-20T01:25:10.545755+0000","last_peered":"2023-05-20T01:25:10.545755+0000","last_clean":"2023-05-20T01:25:10.545755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:10.545755+0000","last_undegraded":"2023-05-20T01:25:10.545755+0000","last_fullsized":"2023-05-20T01:25:10.545755+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":149,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-05-20T01:25:07.711852+0000","last_change":"2023-05-20T01:16:22.829324+0000","last_active":"2023-05-20T01:25:07.711852+0000","last_peered":"2023-05-20T01:25:07.711852+0000","last_clean":"2023-05-20T01:25:07.711852+0000","last_became_active":"2023-05-20T01:16:22.828964+0000","last_became_peered":"2023-05-20T01:16:22.828964+0000","last_unstale":"2023-05-20T01:25:07.711852+0000","last_undegraded":"2023-05-20T01:25:07.711852+0000","last_fullsized":"2023-05-20T01:25:07.711852+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:53:20.474800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:10.545755+0000","last_change":"2023-05-20T01:25:10.545755+0000","last_active":"2023-05-20T01:25:10.545755+0000","last_peered":"2023-05-20T01:25:10.545755+0000","last_clean":"2023-05-20T01:25:10.545755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:10.545755+0000","last_undegraded":"2023-05-20T01:25:10.545755+0000","last_fullsized":"2023-05-20T01:25:10.545755+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":135,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-05-20T01:25:07.873896+0000","last_change":"2023-05-20T01:16:22.834754+0000","last_active":"2023-05-20T01:25:07.873896+0000","last_peered":"2023-05-20T01:25:07.873896+0000","last_clean":"2023-05-20T01:25:07.873896+0000","last_became_active":"2023-05-20T01:16:22.834437+0000","last_became_peered":"2023-05-20T01:16:22.834437+0000","last_unstale":"2023-05-20T01:25:07.873896+0000","last_undegraded":"2023-05-20T01:25:07.873896+0000","last_fullsized":"2023-05-20T01:25:07.873896+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:39:16.284912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":271,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-05-20T01:25:07.874026+0000","last_change":"2023-05-20T01:16:22.834920+0000","last_active":"2023-05-20T01:25:07.874026+0000","last_peered":"2023-05-20T01:25:07.874026+0000","last_clean":"2023-05-20T01:25:07.874026+0000","last_became_active":"2023-05-20T01:16:22.833924+0000","last_became_peered":"2023-05-20T01:16:22.833924+0000","last_unstale":"2023-05-20T01:25:07.874026+0000","last_undegraded":"2023-05-20T01:25:07.874026+0000","last_fullsized":"2023-05-20T01:25:07.874026+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:25:38.846936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:10.545755+0000","last_change":"2023-05-20T01:25:10.545755+0000","last_active":"2023-05-20T01:25:10.545755+0000","last_peered":"2023-05-20T01:25:10.545755+0000","last_clean":"2023-05-20T01:25:10.545755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:10.545755+0000","last_undegraded":"2023-05-20T01:25:10.545755+0000","last_fullsized":"2023-05-20T01:25:10.545755+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":121,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-05-20T01:25:07.873782+0000","last_change":"2023-05-20T01:16:23.290259+0000","last_active":"2023-05-20T01:25:07.873782+0000","last_peered":"2023-05-20T01:25:07.873782+0000","last_clean":"2023-05-20T01:25:07.873782+0000","last_became_active":"2023-05-20T01:16:23.289911+0000","last_became_peered":"2023-05-20T01:16:23.289911+0000","last_unstale":"2023-05-20T01:25:07.873782+0000","last_undegraded":"2023-05-20T01:25:07.873782+0000","last_fullsized":"2023-05-20T01:25:07.873782+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:55:46.978560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:10.545755+0000","last_change":"2023-05-20T01:25:10.545755+0000","last_active":"2023-05-20T01:25:10.545755+0000","last_peered":"2023-05-20T01:25:10.545755+0000","last_clean":"2023-05-20T01:25:10.545755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:10.545755+0000","last_undegraded":"2023-05-20T01:25:10.545755+0000","last_fullsized":"2023-05-20T01:25:10.545755+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":121,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-05-20T01:25:07.874033+0000","last_change":"2023-05-20T01:16:23.290207+0000","last_active":"2023-05-20T01:25:07.874033+0000","last_peered":"2023-05-20T01:25:07.874033+0000","last_clean":"2023-05-20T01:25:07.874033+0000","last_became_active":"2023-05-20T01:16:23.289928+0000","last_became_peered":"2023-05-20T01:16:23.289928+0000","last_unstale":"2023-05-20T01:25:07.874033+0000","last_undegraded":"2023-05-20T01:25:07.874033+0000","last_fullsized":"2023-05-20T01:25:07.874033+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:19:51.946682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:10.545755+0000","last_change":"2023-05-20T01:25:10.545755+0000","last_active":"2023-05-20T01:25:10.545755+0000","last_peered":"2023-05-20T01:25:10.545755+0000","last_clean":"2023-05-20T01:25:10.545755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:10.545755+0000","last_undegraded":"2023-05-20T01:25:10.545755+0000","last_fullsized":"2023-05-20T01:25:10.545755+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":121,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-05-20T01:25:07.873924+0000","last_change":"2023-05-20T01:16:23.290538+0000","last_active":"2023-05-20T01:25:07.873924+0000","last_peered":"2023-05-20T01:25:07.873924+0000","last_clean":"2023-05-20T01:25:07.873924+0000","last_became_active":"2023-05-20T01:16:23.290230+0000","last_became_peered":"2023-05-20T01:16:23.290230+0000","last_unstale":"2023-05-20T01:25:07.873924+0000","last_undegraded":"2023-05-20T01:25:07.873924+0000","last_fullsized":"2023-05-20T01:25:07.873924+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:15:27.984665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:10.545755+0000","last_change":"2023-05-20T01:25:10.545755+0000","last_active":"2023-05-20T01:25:10.545755+0000","last_peered":"2023-05-20T01:25:10.545755+0000","last_clean":"2023-05-20T01:25:10.545755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:10.545755+0000","last_undegraded":"2023-05-20T01:25:10.545755+0000","last_fullsized":"2023-05-20T01:25:10.545755+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":127,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-05-20T01:25:07.714758+0000","last_change":"2023-05-20T01:16:23.290341+0000","last_active":"2023-05-20T01:25:07.714758+0000","last_peered":"2023-05-20T01:25:07.714758+0000","last_clean":"2023-05-20T01:25:07.714758+0000","last_became_active":"2023-05-20T01:16:23.290053+0000","last_became_peered":"2023-05-20T01:16:23.290053+0000","last_unstale":"2023-05-20T01:25:07.714758+0000","last_undegraded":"2023-05-20T01:25:07.714758+0000","last_fullsized":"2023-05-20T01:25:07.714758+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:37:03.379326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:10.545755+0000","last_change":"2023-05-20T01:25:10.545755+0000","last_active":"2023-05-20T01:25:10.545755+0000","last_peered":"2023-05-20T01:25:10.545755+0000","last_clean":"2023-05-20T01:25:10.545755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:10.545755+0000","last_undegraded":"2023-05-20T01:25:10.545755+0000","last_fullsized":"2023-05-20T01:25:10.545755+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":127,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-05-20T01:25:07.714773+0000","last_change":"2023-05-20T01:16:22.834330+0000","last_active":"2023-05-20T01:25:07.714773+0000","last_peered":"2023-05-20T01:25:07.714773+0000","last_clean":"2023-05-20T01:25:07.714773+0000","last_became_active":"2023-05-20T01:16:22.833946+0000","last_became_peered":"2023-05-20T01:16:22.833946+0000","last_unstale":"2023-05-20T01:25:07.714773+0000","last_undegraded":"2023-05-20T01:25:07.714773+0000","last_fullsized":"2023-05-20T01:25:07.714773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:28:56.043585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574958,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":11576,"kb_used_data":1584,"kb_used_omap":0,"kb_used_meta":9984,"kb_avail":94360264,"statfs":{"total":96636764160,"available":96624910336,"internally_reserved":0,"allocated":1622016,"data_stored":1106512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":10223616},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:24:34 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.47699999999999998,"15min":0.51400000000000001},"min":{"1min":0.26900000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.68799999999999994,"5min":0.73199999999999998,"15min":0.88200000000000001},"last":0.623},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.503,"15min":0.54700000000000004},"min":{"1min":0.33100000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.63600000000000001,"5min":0.752,"15min":0.90000000000000002},"last":0.501}]},{"osd":1,"last update":"Sat May 20 01:24:34 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.49299999999999999,"15min":0.75},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.628,"5min":0.70299999999999996,"15min":8.8879999999999999},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.52200000000000002,"15min":0.78900000000000003},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.74199999999999999,"5min":0.75,"15min":8.875},"last":0.74099999999999999}]},{"osd":2,"last update":"Sat May 20 01:24:34 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.503,"15min":0.73599999999999999},"min":{"1min":0.34000000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.65900000000000003,"5min":0.77100000000000002,"15min":8.9120000000000008},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.504,"15min":0.73899999999999999},"min":{"1min":0.35999999999999999,"5min":0.35499999999999998,"15min":0.35099999999999998},"max":{"1min":0.58999999999999997,"5min":0.72399999999999998,"15min":8.8520000000000003},"last":0.60199999999999998}]},{"osd":3,"last update":"Sat May 20 01:24:34 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.49399999999999999,"15min":0.79700000000000004},"min":{"1min":0.36399999999999999,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.60599999999999998,"5min":0.73799999999999999,"15min":10.884},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.55100000000000005,"15min":0.85099999999999998},"min":{"1min":0.40600000000000003,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":10.952},"last":0.77300000000000002}]},{"osd":4,"last update":"Sat May 20 01:24:34 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51100000000000001,"15min":0.51000000000000001},"min":{"1min":0.39200000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.66700000000000004,"5min":0.74299999999999999,"15min":0.77000000000000002},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.312},"max":{"1min":0.71599999999999997,"5min":0.78500000000000003,"15min":0.875},"last":0.58499999999999996}]},{"osd":5,"last update":"Sat May 20 01:24:34 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.56299999999999994,"15min":0.54200000000000004},"min":{"1min":0.35999999999999999,"5min":0.35799999999999998,"15min":0.32300000000000001},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.57099999999999995,"15min":0.57599999999999996},"min":{"1min":0.35999999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.745,"5min":0.79600000000000004,"15min":0.80200000000000005},"last":0.68799999999999994}]},{"osd":6,"last update":"Sat May 20 01:24:34 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.59599999999999997,"15min":0.59499999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.378},"max":{"1min":0.72699999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.57799999999999996,"15min":0.58399999999999996},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.73399999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.752}]}]},{"osd":6,"up_from":13,"seq":55834574957,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10164,"kb_used_data":1004,"kb_used_omap":0,"kb_used_meta":9152,"kb_avail":94361676,"statfs":{"total":96636764160,"available":96626356224,"internally_reserved":0,"allocated":1028096,"data_stored":516144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9371648},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:24:36 2023","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.45400000000000001,"15min":0.47799999999999998},"min":{"1min":0.29099999999999998,"5min":0.28999999999999998,"15min":0.26900000000000002},"max":{"1min":0.55500000000000005,"5min":0.72699999999999998,"15min":0.77200000000000002},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.438,"5min":0.47199999999999998,"15min":0.51800000000000002},"min":{"1min":0.309,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.60299999999999998,"5min":0.70099999999999996,"15min":0.73999999999999999},"last":0.48499999999999999}]},{"osd":1,"last update":"Sat May 20 01:24:36 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.495,"15min":0.52100000000000002},"min":{"1min":0.312,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.70499999999999996,"5min":0.71699999999999997,"15min":0.83799999999999997},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.50800000000000001,"15min":0.52400000000000002},"min":{"1min":0.33000000000000002,"5min":0.32600000000000001,"15min":0.318},"max":{"1min":0.68600000000000005,"5min":0.749,"15min":0.79200000000000004},"last":0.57999999999999996}]},{"osd":2,"last update":"Sat May 20 01:24:36 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.48199999999999998,"15min":0.504},"min":{"1min":0.36799999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.52700000000000002,"5min":0.72199999999999998,"15min":0.76600000000000001},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.51800000000000002,"15min":0.55000000000000004},"min":{"1min":0.35899999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.629,"5min":0.76200000000000001,"15min":0.76800000000000002},"last":0.64300000000000002}]},{"osd":3,"last update":"Sat May 20 01:24:36 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.51500000000000001,"15min":0.53500000000000003},"min":{"1min":0.32400000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.80800000000000005},"last":0.624},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.52100000000000002,"15min":0.54000000000000004},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.58099999999999996,"5min":0.76400000000000001,"15min":0.79500000000000004},"last":0.50600000000000001}]},{"osd":4,"last update":"Sat May 20 01:24:36 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.53800000000000003,"15min":0.55000000000000004},"min":{"1min":0.42599999999999999,"5min":0.33300000000000002,"15min":0.32600000000000001},"max":{"1min":0.754,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.54700000000000004,"15min":0.56899999999999995},"min":{"1min":0.40300000000000002,"5min":0.378,"15min":0.32700000000000001},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.80100000000000005},"last":0.44}]},{"osd":5,"last update":"Sat May 20 01:24:36 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.54100000000000004,"15min":0.54400000000000004},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.65000000000000002,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.38},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.57599999999999996,"15min":0.60399999999999998},"min":{"1min":0.46100000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.78600000000000003,"5min":0.81599999999999995,"15min":0.89400000000000002},"last":0.51500000000000001}]},{"osd":7,"last update":"Sat May 20 01:24:36 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57699999999999996,"15min":0.59799999999999998},"min":{"1min":0.44,"5min":0.371,"15min":0.371},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.92900000000000005},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.58099999999999996,"15min":0.60999999999999999},"min":{"1min":0.48099999999999998,"5min":0.379,"15min":0.379},"max":{"1min":0.68100000000000005,"5min":0.82799999999999996,"15min":0.85199999999999998},"last":0.59499999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574957,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":16376,"kb_used_data":1584,"kb_used_omap":0,"kb_used_meta":14784,"kb_avail":94355464,"statfs":{"total":96636764160,"available":96619995136,"internally_reserved":0,"allocated":1622016,"data_stored":1106512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":15138816},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:24:33 2023","interfaces":[{"interface":"back","average":{"1min":0.254,"5min":0.28199999999999997,"15min":0.313},"min":{"1min":0.186,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":0.37,"5min":0.57099999999999995,"15min":1.194},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.28299999999999997,"5min":0.314,"15min":0.316},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.41399999999999998,"5min":0.61499999999999999,"15min":0.75800000000000001},"last":0.73199999999999998}]},{"osd":2,"last update":"Sat May 20 01:24:33 2023","interfaces":[{"interface":"back","average":{"1min":0.27200000000000002,"5min":0.30199999999999999,"15min":0.317},"min":{"1min":0.20100000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.33800000000000002,"5min":0.52400000000000002,"15min":0.66400000000000003},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.33700000000000002,"5min":0.36799999999999999,"15min":0.40600000000000003},"min":{"1min":0.27800000000000002,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.41299999999999998,"5min":0.60299999999999998,"15min":1.3580000000000001},"last":0.497}]},{"osd":3,"last update":"Sat May 20 01:24:33 2023","interfaces":[{"interface":"back","average":{"1min":0.315,"5min":0.34999999999999998,"15min":0.36899999999999999},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.40500000000000003,"5min":0.57399999999999995,"15min":1.619},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.32100000000000001,"5min":0.34699999999999998,"15min":0.379},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.41299999999999998,"5min":0.60899999999999999,"15min":1.8400000000000001},"last":0.55300000000000005}]},{"osd":4,"last update":"Sat May 20 01:24:33 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.55900000000000005,"15min":0.58099999999999996},"min":{"1min":0.379,"5min":0.34899999999999998,"15min":0.32200000000000001},"max":{"1min":0.72799999999999998,"5min":0.85299999999999998,"15min":1.2769999999999999},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.56999999999999995,"15min":0.58299999999999996},"min":{"1min":0.41699999999999998,"5min":0.312,"15min":0.312},"max":{"1min":0.65600000000000003,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.56899999999999995}]},{"osd":5,"last update":"Sat May 20 01:24:33 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.58399999999999996,"15min":0.58899999999999997},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.754,"5min":0.81899999999999995,"15min":1.4510000000000001},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.59299999999999997,"15min":0.60599999999999998},"min":{"1min":0.436,"5min":0.39100000000000001,"15min":0.378},"max":{"1min":0.77200000000000002,"5min":0.79900000000000004,"15min":1.409},"last":0.58699999999999997}]},{"osd":6,"last update":"Sat May 20 01:24:33 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.59499999999999997,"15min":0.60399999999999998},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.73599999999999999,"5min":0.876,"15min":1.575},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.58499999999999996,"15min":0.59399999999999997},"min":{"1min":0.41299999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.66400000000000003,"5min":0.83099999999999996,"15min":1.0600000000000001},"last":0.60799999999999998}]},{"osd":7,"last update":"Sat May 20 01:24:33 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.59499999999999997,"15min":0.60499999999999998},"min":{"1min":0.40500000000000003,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.75600000000000001,"5min":0.88700000000000001,"15min":1.496},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.623,"15min":0.63800000000000001},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.77300000000000002,"5min":2.1709999999999998,"15min":2.1709999999999998},"last":1.5009999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574957,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10224,"kb_used_data":1000,"kb_used_omap":0,"kb_used_meta":9216,"kb_avail":94361616,"statfs":{"total":96636764160,"available":96626294784,"internally_reserved":0,"allocated":1024000,"data_stored":515183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9437184},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat May 20 01:24:32 2023","interfaces":[{"interface":"back","average":{"1min":0.26200000000000001,"5min":0.29199999999999998,"15min":0.318},"min":{"1min":0.192,"5min":0.17999999999999999,"15min":0.17999999999999999},"max":{"1min":0.371,"5min":0.54600000000000004,"15min":0.56599999999999995},"last":0.247},{"interface":"front","average":{"1min":0.26400000000000001,"5min":0.29699999999999999,"15min":0.34200000000000003},"min":{"1min":0.21099999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.33300000000000002,"5min":0.52000000000000002,"15min":0.98799999999999999},"last":0.36899999999999999}]},{"osd":2,"last update":"Sat May 20 01:24:32 2023","interfaces":[{"interface":"back","average":{"1min":0.28100000000000003,"5min":0.29999999999999999,"15min":0.32800000000000001},"min":{"1min":0.20100000000000001,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":0.35999999999999999,"5min":0.52300000000000002,"15min":0.55700000000000005},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.33100000000000002,"5min":0.34899999999999998,"15min":0.38200000000000001},"min":{"1min":0.26200000000000001,"5min":0.216,"15min":0.216},"max":{"1min":0.41699999999999998,"5min":0.60299999999999998,"15min":0.86199999999999999},"last":0.41799999999999998}]},{"osd":3,"last update":"Sat May 20 01:24:32 2023","interfaces":[{"interface":"back","average":{"1min":0.29599999999999999,"5min":0.33200000000000002,"15min":0.35999999999999999},"min":{"1min":0.24299999999999999,"5min":0.22,"15min":0.22},"max":{"1min":0.374,"5min":0.623,"15min":0.68300000000000005},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.35399999999999998,"15min":0.36599999999999999},"min":{"1min":0.28000000000000003,"5min":0.245,"15min":0.245},"max":{"1min":0.44700000000000001,"5min":0.60899999999999999,"15min":0.60899999999999999},"last":0.44900000000000001}]},{"osd":4,"last update":"Sat May 20 01:24:32 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.57099999999999995,"15min":0.55100000000000005},"min":{"1min":0.433,"5min":0.372,"15min":0.34399999999999997},"max":{"1min":0.78800000000000003,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.503},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.57999999999999996,"15min":0.56899999999999995},"min":{"1min":0.40799999999999997,"5min":0.39200000000000002,"15min":0.36299999999999999},"max":{"1min":0.70899999999999996,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.42799999999999999}]},{"osd":5,"last update":"Sat May 20 01:24:32 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.57799999999999996,"15min":0.57599999999999996},"min":{"1min":0.379,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.74299999999999999,"5min":0.80500000000000005,"15min":0.88100000000000001},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.63400000000000001,"15min":0.58899999999999997},"min":{"1min":0.40899999999999997,"5min":0.374,"15min":0.374},"max":{"1min":0.76300000000000001,"5min":4.4720000000000004,"15min":4.4720000000000004},"last":0.623}]},{"osd":6,"last update":"Sat May 20 01:24:32 2023","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":0.72799999999999998,"15min":0.625},"min":{"1min":0.45800000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":10.042999999999999,"5min":10.042999999999999,"15min":10.042999999999999},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.60499999999999998,"15min":0.61599999999999999},"min":{"1min":0.45200000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.73399999999999999,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.40200000000000002}]},{"osd":7,"last update":"Sat May 20 01:24:32 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.59499999999999997,"15min":0.59299999999999997},"min":{"1min":0.433,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.74199999999999999,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.57499999999999996},{"interface":"front","average":{"1min":1.1040000000000001,"5min":0.72499999999999998,"15min":0.65700000000000003},"min":{"1min":0.44800000000000001,"5min":0.438,"15min":0.42099999999999999},"max":{"1min":10.085000000000001,"5min":10.085000000000001,"15min":10.085000000000001},"last":0.53600000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574957,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10292,"kb_used_data":1004,"kb_used_omap":0,"kb_used_meta":9280,"kb_avail":94361548,"statfs":{"total":96636764160,"available":96626225152,"internally_reserved":0,"allocated":1028096,"data_stored":516144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9502720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:24:44 2023","interfaces":[{"interface":"back","average":{"1min":0.255,"5min":0.27500000000000002,"15min":0.32400000000000001},"min":{"1min":0.19500000000000001,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":0.36799999999999999,"5min":0.53400000000000003,"15min":1.038},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.26700000000000002,"5min":0.29699999999999999,"15min":0.34200000000000003},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.36799999999999999,"5min":0.46500000000000002,"15min":1.079},"last":0.23899999999999999}]},{"osd":1,"last update":"Sat May 20 01:24:44 2023","interfaces":[{"interface":"back","average":{"1min":0.28999999999999998,"5min":0.314,"15min":0.38100000000000001},"min":{"1min":0.23400000000000001,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.34899999999999998,"5min":0.61399999999999999,"15min":1.4950000000000001},"last":0.434},{"interface":"front","average":{"1min":0.318,"5min":0.35499999999999998,"15min":0.41799999999999998},"min":{"1min":0.27400000000000002,"5min":0.249,"15min":0.249},"max":{"1min":0.36099999999999999,"5min":0.629,"15min":1.5189999999999999},"last":0.313}]},{"osd":3,"last update":"Sat May 20 01:24:44 2023","interfaces":[{"interface":"back","average":{"1min":0.27800000000000002,"5min":0.32400000000000001,"15min":0.38200000000000001},"min":{"1min":0.218,"5min":0.216,"15min":0.216},"max":{"1min":0.42499999999999999,"5min":0.60599999999999998,"15min":1.2090000000000001},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.34000000000000002,"5min":0.36699999999999999,"15min":0.432},"min":{"1min":0.30199999999999999,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.38800000000000001,"5min":0.68100000000000005,"15min":1.5620000000000001},"last":0.38900000000000001}]},{"osd":4,"last update":"Sat May 20 01:24:44 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.55500000000000005,"15min":0.57699999999999996},"min":{"1min":0.38800000000000001,"5min":0.35899999999999999,"15min":0.34899999999999998},"max":{"1min":0.69199999999999995,"5min":0.85199999999999998,"15min":1.319},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.58499999999999996,"15min":0.61199999999999999},"min":{"1min":0.432,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.69499999999999995,"5min":0.89000000000000001,"15min":1.4319999999999999},"last":0.51400000000000001}]},{"osd":5,"last update":"Sat May 20 01:24:44 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.55800000000000005,"15min":0.62},"min":{"1min":0.41999999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.73599999999999999,"5min":0.95599999999999996,"15min":1.758},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.58299999999999996,"15min":0.63800000000000001},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.72999999999999998,"5min":0.85099999999999998,"15min":1.7110000000000001},"last":0.66900000000000004}]},{"osd":6,"last update":"Sat May 20 01:24:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.58599999999999997,"15min":0.63200000000000001},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.39700000000000002},"max":{"1min":0.746,"5min":0.88900000000000001,"15min":1.665},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.58299999999999996,"15min":0.63200000000000001},"min":{"1min":0.41699999999999998,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.74299999999999999,"5min":0.93500000000000005,"15min":1.819},"last":0.48999999999999999}]},{"osd":7,"last update":"Sat May 20 01:24:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.59499999999999997,"15min":0.65700000000000003},"min":{"1min":0.44400000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.76900000000000002,"5min":0.85099999999999998,"15min":1.7310000000000001},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.59899999999999998,"15min":0.65300000000000002},"min":{"1min":0.46999999999999997,"5min":0.42399999999999999,"15min":0.42199999999999999},"max":{"1min":0.72499999999999998,"5min":0.90600000000000003,"15min":1.8460000000000001},"last":0.52800000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574957,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10420,"kb_used_data":1004,"kb_used_omap":0,"kb_used_meta":9408,"kb_avail":94361420,"statfs":{"total":96636764160,"available":96626094080,"internally_reserved":0,"allocated":1028096,"data_stored":516144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9633792},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:24:32 2023","interfaces":[{"interface":"back","average":{"1min":0.27300000000000002,"5min":0.30599999999999999,"15min":0.317},"min":{"1min":0.21199999999999999,"5min":0.188,"15min":0.188},"max":{"1min":0.375,"5min":1.528,"15min":1.528},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.28000000000000003,"5min":0.34200000000000003,"15min":0.42299999999999999},"min":{"1min":0.214,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.40699999999999997,"5min":2.4209999999999998,"15min":3.1890000000000001},"last":0.35799999999999998}]},{"osd":1,"last update":"Sat May 20 01:24:32 2023","interfaces":[{"interface":"back","average":{"1min":0.27300000000000002,"5min":0.33600000000000002,"15min":0.41499999999999998},"min":{"1min":0.19900000000000001,"5min":0.189,"15min":0.189},"max":{"1min":0.377,"5min":3.2999999999999998,"15min":3.4260000000000002},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.33200000000000002,"5min":0.34200000000000003,"15min":0.35599999999999998},"min":{"1min":0.23799999999999999,"5min":0.22900000000000001,"15min":0.221},"max":{"1min":0.45500000000000002,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.32300000000000001}]},{"osd":2,"last update":"Sat May 20 01:24:32 2023","interfaces":[{"interface":"back","average":{"1min":0.30599999999999999,"5min":0.34300000000000003,"15min":0.38700000000000001},"min":{"1min":0.26400000000000001,"5min":0.245,"15min":0.224},"max":{"1min":0.39200000000000002,"5min":0.625,"15min":1.1619999999999999},"last":0.503},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.371,"15min":0.41299999999999998},"min":{"1min":0.28499999999999998,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.41499999999999998,"5min":0.72399999999999998,"15min":1.413},"last":0.35099999999999998}]},{"osd":4,"last update":"Sat May 20 01:24:32 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.55100000000000005,"15min":0.58699999999999997},"min":{"1min":0.40300000000000002,"5min":0.38600000000000001,"15min":0.38200000000000001},"max":{"1min":0.72899999999999998,"5min":0.95399999999999996,"15min":1.0700000000000001},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.61199999999999999,"15min":0.68000000000000005},"min":{"1min":0.41099999999999998,"5min":0.40200000000000002,"15min":0.35899999999999999},"max":{"1min":0.72899999999999998,"5min":3.1789999999999998,"15min":3.4660000000000002},"last":0.58399999999999996}]},{"osd":5,"last update":"Sat May 20 01:24:32 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.59899999999999998,"15min":0.68500000000000005},"min":{"1min":0.38400000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.751,"5min":3.2170000000000001,"15min":3.5379999999999998},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.59999999999999998,"15min":0.63200000000000001},"min":{"1min":0.41999999999999998,"5min":0.40799999999999997,"15min":0.374},"max":{"1min":0.74199999999999999,"5min":1.026,"15min":1.385},"last":0.46700000000000003}]},{"osd":6,"last update":"Sat May 20 01:24:32 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.59599999999999997,"15min":0.63400000000000001},"min":{"1min":0.439,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.70199999999999996,"5min":0.99199999999999999,"15min":1.347},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.58499999999999996,"15min":0.63},"min":{"1min":0.39500000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.752,"5min":0.94099999999999995,"15min":1.107},"last":0.54000000000000004}]},{"osd":7,"last update":"Sat May 20 01:24:32 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.59699999999999998,"15min":0.63700000000000001},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.64000000000000001,"5min":0.94199999999999995,"15min":1.1359999999999999},"last":0.439},{"interface":"front","average":{"1min":0.625,"5min":0.65100000000000002,"15min":0.72499999999999998},"min":{"1min":0.47599999999999998,"5min":0.436,"15min":0.436},"max":{"1min":0.73799999999999999,"5min":3.2559999999999998,"15min":3.5009999999999999},"last":0.60099999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574957,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":15732,"kb_used_data":1004,"kb_used_omap":0,"kb_used_meta":14720,"kb_avail":94356108,"statfs":{"total":96636764160,"available":96620654592,"internally_reserved":0,"allocated":1028096,"data_stored":516144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":15073280},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:24:33 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.47299999999999998,"15min":0.48099999999999998},"min":{"1min":0.32200000000000001,"5min":0.247,"15min":0.247},"max":{"1min":0.59199999999999997,"5min":0.73699999999999999,"15min":0.75600000000000001},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.50700000000000001,"15min":0.498},"min":{"1min":0.30599999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.64800000000000002,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":0.44900000000000001}]},{"osd":1,"last update":"Sat May 20 01:24:33 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.49199999999999999,"15min":0.51900000000000002},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.65300000000000002,"5min":0.746,"15min":0.82799999999999996},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.51900000000000002,"15min":0.53000000000000003},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.63600000000000001,"5min":1.7190000000000001,"15min":2.0739999999999998},"last":0.42399999999999999}]},{"osd":2,"last update":"Sat May 20 01:24:33 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.51900000000000002,"15min":0.50800000000000001},"min":{"1min":0.36599999999999999,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.64900000000000002,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.54200000000000004,"15min":0.55000000000000004},"min":{"1min":0.378,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.61399999999999999,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.51100000000000001}]},{"osd":3,"last update":"Sat May 20 01:24:33 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.52600000000000002,"15min":0.52400000000000002},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34399999999999997},"max":{"1min":0.626,"5min":0.81399999999999995,"15min":1.827},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.57099999999999995,"15min":0.65500000000000003},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.65000000000000002,"5min":0.89300000000000002,"15min":3.1749999999999998},"last":0.57599999999999996}]},{"osd":5,"last update":"Sat May 20 01:24:33 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.53400000000000003,"15min":0.48699999999999999},"min":{"1min":0.36899999999999999,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.76700000000000002,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.55400000000000005,"15min":0.51300000000000001},"min":{"1min":0.35099999999999998,"5min":0.33400000000000002,"15min":0.29899999999999999},"max":{"1min":0.78400000000000003,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.58899999999999997}]},{"osd":6,"last update":"Sat May 20 01:24:33 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.58799999999999997,"15min":0.54700000000000004},"min":{"1min":0.432,"5min":0.40400000000000003,"15min":0.32400000000000001},"max":{"1min":0.80300000000000005,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.61599999999999999,"15min":0.59099999999999997},"min":{"1min":0.51100000000000001,"5min":0.42399999999999999,"15min":0.39400000000000002},"max":{"1min":0.72299999999999998,"5min":0.84699999999999998,"15min":0.85899999999999999},"last":0.60799999999999998}]},{"osd":7,"last update":"Sat May 20 01:24:33 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.57799999999999996,"15min":0.55700000000000005},"min":{"1min":0.41999999999999998,"5min":0.36099999999999999,"15min":0.34200000000000003},"max":{"1min":0.71799999999999997,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.61399999999999999,"15min":0.59799999999999998},"min":{"1min":0.499,"5min":0.36399999999999999,"15min":0.34000000000000002},"max":{"1min":0.73699999999999999,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.65400000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574958,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10160,"kb_used_data":1000,"kb_used_omap":0,"kb_used_meta":9152,"kb_avail":94361680,"statfs":{"total":96636764160,"available":96626360320,"internally_reserved":0,"allocated":1024000,"data_stored":515183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9371648},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:24:38 2023","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.45400000000000001,"15min":0.65300000000000002},"min":{"1min":0.29999999999999999,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.68899999999999995,"5min":0.72599999999999998,"15min":6.3929999999999998},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.46000000000000002,"15min":0.68000000000000005},"min":{"1min":0.34999999999999998,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.61099999999999999,"5min":0.67500000000000004,"15min":6.3339999999999996},"last":0.499}]},{"osd":1,"last update":"Sat May 20 01:24:38 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.46000000000000002,"15min":0.58799999999999997},"min":{"1min":0.313,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":3.2450000000000001},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.498,"5min":0.499,"15min":0.53400000000000003},"min":{"1min":0.38900000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.72599999999999998,"5min":0.752,"15min":1.1819999999999999},"last":0.48199999999999998}]},{"osd":2,"last update":"Sat May 20 01:24:38 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48299999999999998,"15min":0.54100000000000004},"min":{"1min":0.33200000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":1.1639999999999999},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.48399999999999999,"15min":0.54400000000000004},"min":{"1min":0.308,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.628,"5min":0.71799999999999997,"15min":1.258},"last":0.58799999999999997}]},{"osd":3,"last update":"Sat May 20 01:24:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.51500000000000001,"15min":0.72499999999999998},"min":{"1min":0.35099999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.69499999999999995,"5min":0.77800000000000002,"15min":6.351},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.53200000000000003,"15min":0.75800000000000001},"min":{"1min":0.40699999999999997,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.67500000000000004,"5min":0.76500000000000001,"15min":6.3730000000000002},"last":0.55400000000000005}]},{"osd":4,"last update":"Sat May 20 01:24:38 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.503,"15min":0.49299999999999999},"min":{"1min":0.38900000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.64500000000000002,"5min":0.68400000000000005,"15min":0.70199999999999996},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.54000000000000004,"15min":0.55100000000000005},"min":{"1min":0.41599999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.749,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.41399999999999998}]},{"osd":6,"last update":"Sat May 20 01:24:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.54600000000000004,"15min":0.55200000000000005},"min":{"1min":0.48699999999999999,"5min":0.38100000000000001,"15min":0.29999999999999999},"max":{"1min":0.72899999999999998,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.54500000000000004,"15min":0.55800000000000005},"min":{"1min":0.437,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.72399999999999998,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.51200000000000001}]},{"osd":7,"last update":"Sat May 20 01:24:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.56799999999999995,"15min":0.58199999999999996},"min":{"1min":0.46500000000000002,"5min":0.33400000000000002,"15min":0.33000000000000002},"max":{"1min":0.77300000000000002,"5min":0.82299999999999995,"15min":0.91000000000000003},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.57699999999999996,"15min":0.60799999999999998},"min":{"1min":0.44700000000000001,"5min":0.40999999999999998,"15min":0.39500000000000002},"max":{"1min":0.76200000000000001,"5min":0.81399999999999995,"15min":0.94499999999999995},"last":0.61299999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-05-20T01:25:16.935 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-05-20T01:25:17.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.089+0000 7fa7f57d4700 1 Processor -- start 2023-05-20T01:25:17.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.089+0000 7fa7f57d4700 1 -- start start 2023-05-20T01:25:17.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.090+0000 7fa7f57d4700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa7f006a510 0x7fa7f015e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:17.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.090+0000 7fa7f57d4700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fa7f006a8e0 0x7fa7f00cf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:17.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.090+0000 7fa7f57d4700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7f015e060 con 0x7fa7f00cfb30 2023-05-20T01:25:17.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.090+0000 7fa7f57d4700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7f015d450 con 0x7fa7f006a8e0 2023-05-20T01:25:17.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.090+0000 7fa7f57d4700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f015d730 con 0x7fa7f006a510 2023-05-20T01:25:17.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.090+0000 7fa7ef7fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fa7f00cfb30 0x7fa7f0071340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:47820/0 (socket says 172.21.15.16:47820) 2023-05-20T01:25:17.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.090+0000 7fa7ef7fe700 1 -- 172.21.15.16:0/144219126 learned_addr learned my addr 172.21.15.16:0/144219126 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:25:17.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.090+0000 7fa7eeffd700 1 --2- 172.21.15.16:0/144219126 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa7f006a510 0x7fa7f015e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:17.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.090+0000 7fa7ee7fc700 1 --2- 172.21.15.16:0/144219126 >> v2:172.21.15.132:3300/0 conn(0x7fa7f006a8e0 0x7fa7f00cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:17.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.091+0000 7fa7edffb700 1 -- 172.21.15.16:0/144219126 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2319679424 0 0) 0x7fa7f015e060 con 0x7fa7f00cfb30 2023-05-20T01:25:17.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.091+0000 7fa7edffb700 1 -- 172.21.15.16:0/144219126 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa7d80035b0 con 0x7fa7f00cfb30 2023-05-20T01:25:17.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.091+0000 7fa7edffb700 1 -- 172.21.15.16:0/144219126 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1249994430 0 0) 0x7fa7d80035b0 con 0x7fa7f00cfb30 2023-05-20T01:25:17.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.091+0000 7fa7eeffd700 1 -- 172.21.15.16:0/144219126 >> v2:172.21.15.132:3300/0 conn(0x7fa7f006a8e0 msgr2=0x7fa7f00cf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:17.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.091+0000 7fa7eeffd700 1 --2- 172.21.15.16:0/144219126 >> v2:172.21.15.132:3300/0 conn(0x7fa7f006a8e0 0x7fa7f00cf5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:17.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.091+0000 7fa7eeffd700 1 -- 172.21.15.16:0/144219126 >> v1:172.21.15.16:6789/0 conn(0x7fa7f00cfb30 legacy=0x7fa7f0071340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:17.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.091+0000 7fa7eeffd700 1 -- 172.21.15.16:0/144219126 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7f015da10 con 0x7fa7f006a510 2023-05-20T01:25:17.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.091+0000 7fa7ee7fc700 1 --2- 172.21.15.16:0/144219126 >> v2:172.21.15.132:3300/0 conn(0x7fa7f006a8e0 0x7fa7f00cf5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-05-20T01:25:17.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.091+0000 7fa7eeffd700 1 --2- 172.21.15.16:0/144219126 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa7f006a510 0x7fa7f015e2a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa7e4009840 tx=0x7fa7e4009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69d4dc02041e5361 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:17.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.091+0000 7fa7edffb700 1 -- 172.21.15.16:0/144219126 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7e80019f0 con 0x7fa7f006a510 2023-05-20T01:25:17.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.091+0000 7fa7edffb700 1 -- 172.21.15.16:0/144219126 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7e4004030 con 0x7fa7f006a510 2023-05-20T01:25:17.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.091+0000 7fa7edffb700 1 -- 172.21.15.16:0/144219126 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7e40043c0 con 0x7fa7f006a510 2023-05-20T01:25:17.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.092+0000 7fa7f57d4700 1 -- 172.21.15.16:0/144219126 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa7f006a510 msgr2=0x7fa7f015e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:17.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.092+0000 7fa7f57d4700 1 --2- 172.21.15.16:0/144219126 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa7f006a510 0x7fa7f015e2a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa7e4009840 tx=0x7fa7e4009b50 comp rx=0 tx=0).stop 2023-05-20T01:25:17.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.092+0000 7fa7f57d4700 1 -- 172.21.15.16:0/144219126 shutdown_connections 2023-05-20T01:25:17.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.092+0000 7fa7f57d4700 1 --2- 172.21.15.16:0/144219126 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa7f006a510 0x7fa7f015e2a0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:17.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.092+0000 7fa7f57d4700 1 --2- 172.21.15.16:0/144219126 >> v2:172.21.15.132:3300/0 conn(0x7fa7f006a8e0 0x7fa7f00cf5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:17.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.092+0000 7fa7f57d4700 1 -- 172.21.15.16:0/144219126 >> 172.21.15.16:0/144219126 conn(0x7fa7f00d0c30 msgr2=0x7fa7f00b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:17.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.092+0000 7fa7f57d4700 1 -- 172.21.15.16:0/144219126 shutdown_connections 2023-05-20T01:25:17.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.092+0000 7fa7f57d4700 1 -- 172.21.15.16:0/144219126 wait complete. 2023-05-20T01:25:17.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.093+0000 7fa7f57d4700 1 Processor -- start 2023-05-20T01:25:17.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.093+0000 7fa7f57d4700 1 -- start start 2023-05-20T01:25:17.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.094+0000 7fa7f57d4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7f006a510 0x7fa7f01f6150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:17.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.094+0000 7fa7f57d4700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa7f006a8e0 0x7fa7f00c8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:17.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.094+0000 7fa7f57d4700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa7f00cfb30 0x7fa7f00c60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:17.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.094+0000 7fa7f57d4700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f00c9350 con 0x7fa7f006a510 2023-05-20T01:25:17.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.094+0000 7fa7f57d4700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f00c9630 con 0x7fa7f006a8e0 2023-05-20T01:25:17.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.094+0000 7fa7f57d4700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f00c9910 con 0x7fa7f00cfb30 2023-05-20T01:25:17.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.094+0000 7fa7eeffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7f006a510 0x7fa7f01f6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:17.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.094+0000 7fa7eeffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7f006a510 0x7fa7f01f6150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36294/0 (socket says 172.21.15.16:36294) 2023-05-20T01:25:17.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.094+0000 7fa7eeffd700 1 -- 172.21.15.16:0/591595398 learned_addr learned my addr 172.21.15.16:0/591595398 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:17.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.094+0000 7fa7ee7fc700 1 --2- 172.21.15.16:0/591595398 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa7f006a8e0 0x7fa7f00c8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:17.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.094+0000 7fa7ef7fe700 1 --2- 172.21.15.16:0/591595398 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa7f00cfb30 0x7fa7f00c60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:17.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.094+0000 7fa7eeffd700 1 -- 172.21.15.16:0/591595398 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa7f00cfb30 msgr2=0x7fa7f00c60e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:17.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.094+0000 7fa7eeffd700 1 --2- 172.21.15.16:0/591595398 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa7f00cfb30 0x7fa7f00c60e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:17.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.094+0000 7fa7eeffd700 1 -- 172.21.15.16:0/591595398 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa7f006a8e0 msgr2=0x7fa7f00c8ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:17.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.094+0000 7fa7eeffd700 1 --2- 172.21.15.16:0/591595398 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa7f006a8e0 0x7fa7f00c8ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:17.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.094+0000 7fa7eeffd700 1 -- 172.21.15.16:0/591595398 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7e00094a0 con 0x7fa7f006a510 2023-05-20T01:25:17.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.094+0000 7fa7ee7fc700 1 --2- 172.21.15.16:0/591595398 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa7f006a8e0 0x7fa7f00c8ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:25:17.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.095+0000 7fa7eeffd700 1 --2- 172.21.15.16:0/591595398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7f006a510 0x7fa7f01f6150 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fa7e400a010 tx=0x7fa7e4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:17.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.095+0000 7fa7d77fe700 1 -- 172.21.15.16:0/591595398 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7e4003c40 con 0x7fa7f006a510 2023-05-20T01:25:17.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.095+0000 7fa7d77fe700 1 -- 172.21.15.16:0/591595398 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7e401f070 con 0x7fa7f006a510 2023-05-20T01:25:17.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.095+0000 7fa7f57d4700 1 -- 172.21.15.16:0/591595398 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7e40094a0 con 0x7fa7f006a510 2023-05-20T01:25:17.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.095+0000 7fa7d77fe700 1 -- 172.21.15.16:0/591595398 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7e4011630 con 0x7fa7f006a510 2023-05-20T01:25:17.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.095+0000 7fa7f57d4700 1 -- 172.21.15.16:0/591595398 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7f006ad50 con 0x7fa7f006a510 2023-05-20T01:25:17.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.096+0000 7fa7f57d4700 1 -- 172.21.15.16:0/591595398 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7f015dcf0 con 0x7fa7f006a510 2023-05-20T01:25:17.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.096+0000 7fa7d77fe700 1 -- 172.21.15.16:0/591595398 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa7e4021030 con 0x7fa7f006a510 2023-05-20T01:25:17.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.097+0000 7fa7d77fe700 1 --2- 172.21.15.16:0/591595398 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa7dc06ace0 0x7fa7dc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:17.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.097+0000 7fa7d77fe700 1 -- 172.21.15.16:0/591595398 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fa7e408a950 con 0x7fa7f006a510 2023-05-20T01:25:17.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.097+0000 7fa7ee7fc700 1 --2- 172.21.15.16:0/591595398 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa7dc06ace0 0x7fa7dc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:17.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.097+0000 7fa7ee7fc700 1 --2- 172.21.15.16:0/591595398 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa7dc06ace0 0x7fa7dc06d190 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fa7f015f840 tx=0x7fa7e0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:17.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.101+0000 7fa7d77fe700 1 -- 172.21.15.16:0/591595398 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fa7e40558d0 con 0x7fa7f006a510 2023-05-20T01:25:17.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.297+0000 7fa7f57d4700 1 -- 172.21.15.16:0/591595398 --> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa7f015d080 con 0x7fa7dc06ace0 2023-05-20T01:25:17.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.305+0000 7fa7d77fe700 1 -- 172.21.15.16:0/591595398 <== mgr.4104 v2:172.21.15.16:6832/3817237998 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+296973 (secure 0 0 0) 0x7fa7f015d080 con 0x7fa7dc06ace0 2023-05-20T01:25:17.315 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:25:17.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.309+0000 7fa7f57d4700 1 -- 172.21.15.16:0/591595398 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa7dc06ace0 msgr2=0x7fa7dc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:17.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.309+0000 7fa7f57d4700 1 --2- 172.21.15.16:0/591595398 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa7dc06ace0 0x7fa7dc06d190 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fa7f015f840 tx=0x7fa7e0008040 comp rx=0 tx=0).stop 2023-05-20T01:25:17.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.309+0000 7fa7f57d4700 1 -- 172.21.15.16:0/591595398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7f006a510 msgr2=0x7fa7f01f6150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:17.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.309+0000 7fa7f57d4700 1 --2- 172.21.15.16:0/591595398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7f006a510 0x7fa7f01f6150 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fa7e400a010 tx=0x7fa7e4004750 comp rx=0 tx=0).stop 2023-05-20T01:25:17.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.309+0000 7fa7f57d4700 1 -- 172.21.15.16:0/591595398 shutdown_connections 2023-05-20T01:25:17.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.309+0000 7fa7f57d4700 1 --2- 172.21.15.16:0/591595398 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa7dc06ace0 0x7fa7dc06d190 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:17.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.309+0000 7fa7f57d4700 1 --2- 172.21.15.16:0/591595398 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa7f00cfb30 0x7fa7f00c60e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:17.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.309+0000 7fa7f57d4700 1 --2- 172.21.15.16:0/591595398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7f006a510 0x7fa7f01f6150 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:17.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.309+0000 7fa7f57d4700 1 --2- 172.21.15.16:0/591595398 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa7f006a8e0 0x7fa7f00c8ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:17.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.309+0000 7fa7f57d4700 1 -- 172.21.15.16:0/591595398 >> 172.21.15.16:0/591595398 conn(0x7fa7f00d0c30 msgr2=0x7fa7f015e210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:17.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.309+0000 7fa7f57d4700 1 -- 172.21.15.16:0/591595398 shutdown_connections 2023-05-20T01:25:17.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.309+0000 7fa7f57d4700 1 -- 172.21.15.16:0/591595398 wait complete. 2023-05-20T01:25:17.318 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2023-05-20T01:25:17.325 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":349,"stamp":"2023-05-20T01:25:15.434324+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":94944,"kb_used_data":9184,"kb_used_omap":0,"kb_used_meta":85696,"kb_avail":754879776,"statfs":{"total":773094113280,"available":772996890624,"internally_reserved":0,"allocated":9404416,"data_stored":5307966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":87752704},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.881661"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:10.545755+0000","last_change":"2023-05-20T01:25:10.545755+0000","last_active":"2023-05-20T01:25:10.545755+0000","last_peered":"2023-05-20T01:25:10.545755+0000","last_clean":"2023-05-20T01:25:10.545755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:10.545755+0000","last_undegraded":"2023-05-20T01:25:10.545755+0000","last_fullsized":"2023-05-20T01:25:10.545755+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:10.545755+0000","last_change":"2023-05-20T01:25:10.545755+0000","last_active":"2023-05-20T01:25:10.545755+0000","last_peered":"2023-05-20T01:25:10.545755+0000","last_clean":"2023-05-20T01:25:10.545755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:10.545755+0000","last_undegraded":"2023-05-20T01:25:10.545755+0000","last_fullsized":"2023-05-20T01:25:10.545755+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:10.545755+0000","last_change":"2023-05-20T01:25:10.545755+0000","last_active":"2023-05-20T01:25:10.545755+0000","last_peered":"2023-05-20T01:25:10.545755+0000","last_clean":"2023-05-20T01:25:10.545755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:10.545755+0000","last_undegraded":"2023-05-20T01:25:10.545755+0000","last_fullsized":"2023-05-20T01:25:10.545755+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:10.545755+0000","last_change":"2023-05-20T01:25:10.545755+0000","last_active":"2023-05-20T01:25:10.545755+0000","last_peered":"2023-05-20T01:25:10.545755+0000","last_clean":"2023-05-20T01:25:10.545755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:10.545755+0000","last_undegraded":"2023-05-20T01:25:10.545755+0000","last_fullsized":"2023-05-20T01:25:10.545755+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:10.545755+0000","last_change":"2023-05-20T01:25:10.545755+0000","last_active":"2023-05-20T01:25:10.545755+0000","last_peered":"2023-05-20T01:25:10.545755+0000","last_clean":"2023-05-20T01:25:10.545755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:10.545755+0000","last_undegraded":"2023-05-20T01:25:10.545755+0000","last_fullsized":"2023-05-20T01:25:10.545755+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:10.545755+0000","last_change":"2023-05-20T01:25:10.545755+0000","last_active":"2023-05-20T01:25:10.545755+0000","last_peered":"2023-05-20T01:25:10.545755+0000","last_clean":"2023-05-20T01:25:10.545755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:10.545755+0000","last_undegraded":"2023-05-20T01:25:10.545755+0000","last_fullsized":"2023-05-20T01:25:10.545755+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:10.545755+0000","last_change":"2023-05-20T01:25:10.545755+0000","last_active":"2023-05-20T01:25:10.545755+0000","last_peered":"2023-05-20T01:25:10.545755+0000","last_clean":"2023-05-20T01:25:10.545755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:10.545755+0000","last_undegraded":"2023-05-20T01:25:10.545755+0000","last_fullsized":"2023-05-20T01:25:10.545755+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:10.545755+0000","last_change":"2023-05-20T01:25:10.545755+0000","last_active":"2023-05-20T01:25:10.545755+0000","last_peered":"2023-05-20T01:25:10.545755+0000","last_clean":"2023-05-20T01:25:10.545755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:10.545755+0000","last_undegraded":"2023-05-20T01:25:10.545755+0000","last_fullsized":"2023-05-20T01:25:10.545755+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:10.545755+0000","last_change":"2023-05-20T01:25:10.545755+0000","last_active":"2023-05-20T01:25:10.545755+0000","last_peered":"2023-05-20T01:25:10.545755+0000","last_clean":"2023-05-20T01:25:10.545755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:10.545755+0000","last_undegraded":"2023-05-20T01:25:10.545755+0000","last_fullsized":"2023-05-20T01:25:10.545755+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:10.545755+0000","last_change":"2023-05-20T01:25:10.545755+0000","last_active":"2023-05-20T01:25:10.545755+0000","last_peered":"2023-05-20T01:25:10.545755+0000","last_clean":"2023-05-20T01:25:10.545755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:10.545755+0000","last_undegraded":"2023-05-20T01:25:10.545755+0000","last_fullsized":"2023-05-20T01:25:10.545755+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:10.545755+0000","last_change":"2023-05-20T01:25:10.545755+0000","last_active":"2023-05-20T01:25:10.545755+0000","last_peered":"2023-05-20T01:25:10.545755+0000","last_clean":"2023-05-20T01:25:10.545755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:10.545755+0000","last_undegraded":"2023-05-20T01:25:10.545755+0000","last_fullsized":"2023-05-20T01:25:10.545755+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:10.545755+0000","last_change":"2023-05-20T01:25:10.545755+0000","last_active":"2023-05-20T01:25:10.545755+0000","last_peered":"2023-05-20T01:25:10.545755+0000","last_clean":"2023-05-20T01:25:10.545755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:10.545755+0000","last_undegraded":"2023-05-20T01:25:10.545755+0000","last_fullsized":"2023-05-20T01:25:10.545755+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:10.545755+0000","last_change":"2023-05-20T01:25:10.545755+0000","last_active":"2023-05-20T01:25:10.545755+0000","last_peered":"2023-05-20T01:25:10.545755+0000","last_clean":"2023-05-20T01:25:10.545755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:10.545755+0000","last_undegraded":"2023-05-20T01:25:10.545755+0000","last_fullsized":"2023-05-20T01:25:10.545755+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:10.545755+0000","last_change":"2023-05-20T01:25:10.545755+0000","last_active":"2023-05-20T01:25:10.545755+0000","last_peered":"2023-05-20T01:25:10.545755+0000","last_clean":"2023-05-20T01:25:10.545755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:10.545755+0000","last_undegraded":"2023-05-20T01:25:10.545755+0000","last_fullsized":"2023-05-20T01:25:10.545755+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:10.545755+0000","last_change":"2023-05-20T01:25:10.545755+0000","last_active":"2023-05-20T01:25:10.545755+0000","last_peered":"2023-05-20T01:25:10.545755+0000","last_clean":"2023-05-20T01:25:10.545755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:10.545755+0000","last_undegraded":"2023-05-20T01:25:10.545755+0000","last_fullsized":"2023-05-20T01:25:10.545755+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:10.545755+0000","last_change":"2023-05-20T01:25:10.545755+0000","last_active":"2023-05-20T01:25:10.545755+0000","last_peered":"2023-05-20T01:25:10.545755+0000","last_clean":"2023-05-20T01:25:10.545755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:10.545755+0000","last_undegraded":"2023-05-20T01:25:10.545755+0000","last_fullsized":"2023-05-20T01:25:10.545755+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:10.545755+0000","last_change":"2023-05-20T01:25:10.545755+0000","last_active":"2023-05-20T01:25:10.545755+0000","last_peered":"2023-05-20T01:25:10.545755+0000","last_clean":"2023-05-20T01:25:10.545755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:10.545755+0000","last_undegraded":"2023-05-20T01:25:10.545755+0000","last_fullsized":"2023-05-20T01:25:10.545755+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:10.545755+0000","last_change":"2023-05-20T01:25:10.545755+0000","last_active":"2023-05-20T01:25:10.545755+0000","last_peered":"2023-05-20T01:25:10.545755+0000","last_clean":"2023-05-20T01:25:10.545755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:10.545755+0000","last_undegraded":"2023-05-20T01:25:10.545755+0000","last_fullsized":"2023-05-20T01:25:10.545755+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:10.545755+0000","last_change":"2023-05-20T01:25:10.545755+0000","last_active":"2023-05-20T01:25:10.545755+0000","last_peered":"2023-05-20T01:25:10.545755+0000","last_clean":"2023-05-20T01:25:10.545755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:10.545755+0000","last_undegraded":"2023-05-20T01:25:10.545755+0000","last_fullsized":"2023-05-20T01:25:10.545755+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:10.545755+0000","last_change":"2023-05-20T01:25:10.545755+0000","last_active":"2023-05-20T01:25:10.545755+0000","last_peered":"2023-05-20T01:25:10.545755+0000","last_clean":"2023-05-20T01:25:10.545755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:10.545755+0000","last_undegraded":"2023-05-20T01:25:10.545755+0000","last_fullsized":"2023-05-20T01:25:10.545755+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:10.545755+0000","last_change":"2023-05-20T01:25:10.545755+0000","last_active":"2023-05-20T01:25:10.545755+0000","last_peered":"2023-05-20T01:25:10.545755+0000","last_clean":"2023-05-20T01:25:10.545755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:10.545755+0000","last_undegraded":"2023-05-20T01:25:10.545755+0000","last_fullsized":"2023-05-20T01:25:10.545755+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:10.545755+0000","last_change":"2023-05-20T01:25:10.545755+0000","last_active":"2023-05-20T01:25:10.545755+0000","last_peered":"2023-05-20T01:25:10.545755+0000","last_clean":"2023-05-20T01:25:10.545755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:10.545755+0000","last_undegraded":"2023-05-20T01:25:10.545755+0000","last_fullsized":"2023-05-20T01:25:10.545755+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:10.545755+0000","last_change":"2023-05-20T01:25:10.545755+0000","last_active":"2023-05-20T01:25:10.545755+0000","last_peered":"2023-05-20T01:25:10.545755+0000","last_clean":"2023-05-20T01:25:10.545755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:10.545755+0000","last_undegraded":"2023-05-20T01:25:10.545755+0000","last_fullsized":"2023-05-20T01:25:10.545755+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:10.545755+0000","last_change":"2023-05-20T01:25:10.545755+0000","last_active":"2023-05-20T01:25:10.545755+0000","last_peered":"2023-05-20T01:25:10.545755+0000","last_clean":"2023-05-20T01:25:10.545755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:10.545755+0000","last_undegraded":"2023-05-20T01:25:10.545755+0000","last_fullsized":"2023-05-20T01:25:10.545755+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:10.545755+0000","last_change":"2023-05-20T01:25:10.545755+0000","last_active":"2023-05-20T01:25:10.545755+0000","last_peered":"2023-05-20T01:25:10.545755+0000","last_clean":"2023-05-20T01:25:10.545755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:10.545755+0000","last_undegraded":"2023-05-20T01:25:10.545755+0000","last_fullsized":"2023-05-20T01:25:10.545755+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":143,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-05-20T01:25:07.873873+0000","last_change":"2023-05-20T01:16:23.289068+0000","last_active":"2023-05-20T01:25:07.873873+0000","last_peered":"2023-05-20T01:25:07.873873+0000","last_clean":"2023-05-20T01:25:07.873873+0000","last_became_active":"2023-05-20T01:16:23.288781+0000","last_became_peered":"2023-05-20T01:16:23.288781+0000","last_unstale":"2023-05-20T01:25:07.873873+0000","last_undegraded":"2023-05-20T01:25:07.873873+0000","last_fullsized":"2023-05-20T01:25:07.873873+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:48:00.157178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:10.545755+0000","last_change":"2023-05-20T01:25:10.545755+0000","last_active":"2023-05-20T01:25:10.545755+0000","last_peered":"2023-05-20T01:25:10.545755+0000","last_clean":"2023-05-20T01:25:10.545755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:10.545755+0000","last_undegraded":"2023-05-20T01:25:10.545755+0000","last_fullsized":"2023-05-20T01:25:10.545755+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":149,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-05-20T01:25:07.711852+0000","last_change":"2023-05-20T01:16:22.829324+0000","last_active":"2023-05-20T01:25:07.711852+0000","last_peered":"2023-05-20T01:25:07.711852+0000","last_clean":"2023-05-20T01:25:07.711852+0000","last_became_active":"2023-05-20T01:16:22.828964+0000","last_became_peered":"2023-05-20T01:16:22.828964+0000","last_unstale":"2023-05-20T01:25:07.711852+0000","last_undegraded":"2023-05-20T01:25:07.711852+0000","last_fullsized":"2023-05-20T01:25:07.711852+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:53:20.474800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:10.545755+0000","last_change":"2023-05-20T01:25:10.545755+0000","last_active":"2023-05-20T01:25:10.545755+0000","last_peered":"2023-05-20T01:25:10.545755+0000","last_clean":"2023-05-20T01:25:10.545755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:10.545755+0000","last_undegraded":"2023-05-20T01:25:10.545755+0000","last_fullsized":"2023-05-20T01:25:10.545755+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":135,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-05-20T01:25:07.873896+0000","last_change":"2023-05-20T01:16:22.834754+0000","last_active":"2023-05-20T01:25:07.873896+0000","last_peered":"2023-05-20T01:25:07.873896+0000","last_clean":"2023-05-20T01:25:07.873896+0000","last_became_active":"2023-05-20T01:16:22.834437+0000","last_became_peered":"2023-05-20T01:16:22.834437+0000","last_unstale":"2023-05-20T01:25:07.873896+0000","last_undegraded":"2023-05-20T01:25:07.873896+0000","last_fullsized":"2023-05-20T01:25:07.873896+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:39:16.284912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":271,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-05-20T01:25:07.874026+0000","last_change":"2023-05-20T01:16:22.834920+0000","last_active":"2023-05-20T01:25:07.874026+0000","last_peered":"2023-05-20T01:25:07.874026+0000","last_clean":"2023-05-20T01:25:07.874026+0000","last_became_active":"2023-05-20T01:16:22.833924+0000","last_became_peered":"2023-05-20T01:16:22.833924+0000","last_unstale":"2023-05-20T01:25:07.874026+0000","last_undegraded":"2023-05-20T01:25:07.874026+0000","last_fullsized":"2023-05-20T01:25:07.874026+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:25:38.846936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:10.545755+0000","last_change":"2023-05-20T01:25:10.545755+0000","last_active":"2023-05-20T01:25:10.545755+0000","last_peered":"2023-05-20T01:25:10.545755+0000","last_clean":"2023-05-20T01:25:10.545755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:10.545755+0000","last_undegraded":"2023-05-20T01:25:10.545755+0000","last_fullsized":"2023-05-20T01:25:10.545755+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":121,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-05-20T01:25:07.873782+0000","last_change":"2023-05-20T01:16:23.290259+0000","last_active":"2023-05-20T01:25:07.873782+0000","last_peered":"2023-05-20T01:25:07.873782+0000","last_clean":"2023-05-20T01:25:07.873782+0000","last_became_active":"2023-05-20T01:16:23.289911+0000","last_became_peered":"2023-05-20T01:16:23.289911+0000","last_unstale":"2023-05-20T01:25:07.873782+0000","last_undegraded":"2023-05-20T01:25:07.873782+0000","last_fullsized":"2023-05-20T01:25:07.873782+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:55:46.978560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:10.545755+0000","last_change":"2023-05-20T01:25:10.545755+0000","last_active":"2023-05-20T01:25:10.545755+0000","last_peered":"2023-05-20T01:25:10.545755+0000","last_clean":"2023-05-20T01:25:10.545755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:10.545755+0000","last_undegraded":"2023-05-20T01:25:10.545755+0000","last_fullsized":"2023-05-20T01:25:10.545755+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":121,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-05-20T01:25:07.874033+0000","last_change":"2023-05-20T01:16:23.290207+0000","last_active":"2023-05-20T01:25:07.874033+0000","last_peered":"2023-05-20T01:25:07.874033+0000","last_clean":"2023-05-20T01:25:07.874033+0000","last_became_active":"2023-05-20T01:16:23.289928+0000","last_became_peered":"2023-05-20T01:16:23.289928+0000","last_unstale":"2023-05-20T01:25:07.874033+0000","last_undegraded":"2023-05-20T01:25:07.874033+0000","last_fullsized":"2023-05-20T01:25:07.874033+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:19:51.946682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:10.545755+0000","last_change":"2023-05-20T01:25:10.545755+0000","last_active":"2023-05-20T01:25:10.545755+0000","last_peered":"2023-05-20T01:25:10.545755+0000","last_clean":"2023-05-20T01:25:10.545755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:10.545755+0000","last_undegraded":"2023-05-20T01:25:10.545755+0000","last_fullsized":"2023-05-20T01:25:10.545755+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":121,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-05-20T01:25:07.873924+0000","last_change":"2023-05-20T01:16:23.290538+0000","last_active":"2023-05-20T01:25:07.873924+0000","last_peered":"2023-05-20T01:25:07.873924+0000","last_clean":"2023-05-20T01:25:07.873924+0000","last_became_active":"2023-05-20T01:16:23.290230+0000","last_became_peered":"2023-05-20T01:16:23.290230+0000","last_unstale":"2023-05-20T01:25:07.873924+0000","last_undegraded":"2023-05-20T01:25:07.873924+0000","last_fullsized":"2023-05-20T01:25:07.873924+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:15:27.984665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:10.545755+0000","last_change":"2023-05-20T01:25:10.545755+0000","last_active":"2023-05-20T01:25:10.545755+0000","last_peered":"2023-05-20T01:25:10.545755+0000","last_clean":"2023-05-20T01:25:10.545755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:10.545755+0000","last_undegraded":"2023-05-20T01:25:10.545755+0000","last_fullsized":"2023-05-20T01:25:10.545755+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":127,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-05-20T01:25:07.714758+0000","last_change":"2023-05-20T01:16:23.290341+0000","last_active":"2023-05-20T01:25:07.714758+0000","last_peered":"2023-05-20T01:25:07.714758+0000","last_clean":"2023-05-20T01:25:07.714758+0000","last_became_active":"2023-05-20T01:16:23.290053+0000","last_became_peered":"2023-05-20T01:16:23.290053+0000","last_unstale":"2023-05-20T01:25:07.714758+0000","last_undegraded":"2023-05-20T01:25:07.714758+0000","last_fullsized":"2023-05-20T01:25:07.714758+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:37:03.379326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:11.552637+0000","last_change":"2023-05-20T01:25:11.552637+0000","last_active":"2023-05-20T01:25:11.552637+0000","last_peered":"2023-05-20T01:25:11.552637+0000","last_clean":"2023-05-20T01:25:11.552637+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:11.552637+0000","last_undegraded":"2023-05-20T01:25:11.552637+0000","last_fullsized":"2023-05-20T01:25:11.552637+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:25:10.545755+0000","last_change":"2023-05-20T01:25:10.545755+0000","last_active":"2023-05-20T01:25:10.545755+0000","last_peered":"2023-05-20T01:25:10.545755+0000","last_clean":"2023-05-20T01:25:10.545755+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:25:10.545755+0000","last_undegraded":"2023-05-20T01:25:10.545755+0000","last_fullsized":"2023-05-20T01:25:10.545755+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":127,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-05-20T01:25:07.714773+0000","last_change":"2023-05-20T01:16:22.834330+0000","last_active":"2023-05-20T01:25:07.714773+0000","last_peered":"2023-05-20T01:25:07.714773+0000","last_clean":"2023-05-20T01:25:07.714773+0000","last_became_active":"2023-05-20T01:16:22.833946+0000","last_became_peered":"2023-05-20T01:16:22.833946+0000","last_unstale":"2023-05-20T01:25:07.714773+0000","last_undegraded":"2023-05-20T01:25:07.714773+0000","last_fullsized":"2023-05-20T01:25:07.714773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:28:56.043585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574958,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":11576,"kb_used_data":1584,"kb_used_omap":0,"kb_used_meta":9984,"kb_avail":94360264,"statfs":{"total":96636764160,"available":96624910336,"internally_reserved":0,"allocated":1622016,"data_stored":1106512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":10223616},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:24:34 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.47699999999999998,"15min":0.51400000000000001},"min":{"1min":0.26900000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.68799999999999994,"5min":0.73199999999999998,"15min":0.88200000000000001},"last":0.623},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.503,"15min":0.54700000000000004},"min":{"1min":0.33100000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.63600000000000001,"5min":0.752,"15min":0.90000000000000002},"last":0.501}]},{"osd":1,"last update":"Sat May 20 01:24:34 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.49299999999999999,"15min":0.75},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.628,"5min":0.70299999999999996,"15min":8.8879999999999999},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.52200000000000002,"15min":0.78900000000000003},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.74199999999999999,"5min":0.75,"15min":8.875},"last":0.74099999999999999}]},{"osd":2,"last update":"Sat May 20 01:24:34 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.503,"15min":0.73599999999999999},"min":{"1min":0.34000000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.65900000000000003,"5min":0.77100000000000002,"15min":8.9120000000000008},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.504,"15min":0.73899999999999999},"min":{"1min":0.35999999999999999,"5min":0.35499999999999998,"15min":0.35099999999999998},"max":{"1min":0.58999999999999997,"5min":0.72399999999999998,"15min":8.8520000000000003},"last":0.60199999999999998}]},{"osd":3,"last update":"Sat May 20 01:24:34 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.49399999999999999,"15min":0.79700000000000004},"min":{"1min":0.36399999999999999,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.60599999999999998,"5min":0.73799999999999999,"15min":10.884},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.55100000000000005,"15min":0.85099999999999998},"min":{"1min":0.40600000000000003,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":10.952},"last":0.77300000000000002}]},{"osd":4,"last update":"Sat May 20 01:24:34 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51100000000000001,"15min":0.51000000000000001},"min":{"1min":0.39200000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.66700000000000004,"5min":0.74299999999999999,"15min":0.77000000000000002},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.312},"max":{"1min":0.71599999999999997,"5min":0.78500000000000003,"15min":0.875},"last":0.58499999999999996}]},{"osd":5,"last update":"Sat May 20 01:24:34 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.56299999999999994,"15min":0.54200000000000004},"min":{"1min":0.35999999999999999,"5min":0.35799999999999998,"15min":0.32300000000000001},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.57099999999999995,"15min":0.57599999999999996},"min":{"1min":0.35999999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.745,"5min":0.79600000000000004,"15min":0.80200000000000005},"last":0.68799999999999994}]},{"osd":6,"last update":"Sat May 20 01:24:34 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.59599999999999997,"15min":0.59499999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.378},"max":{"1min":0.72699999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.57799999999999996,"15min":0.58399999999999996},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.73399999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.752}]}]},{"osd":6,"up_from":13,"seq":55834574957,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10164,"kb_used_data":1004,"kb_used_omap":0,"kb_used_meta":9152,"kb_avail":94361676,"statfs":{"total":96636764160,"available":96626356224,"internally_reserved":0,"allocated":1028096,"data_stored":516144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9371648},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:24:36 2023","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.45400000000000001,"15min":0.47799999999999998},"min":{"1min":0.29099999999999998,"5min":0.28999999999999998,"15min":0.26900000000000002},"max":{"1min":0.55500000000000005,"5min":0.72699999999999998,"15min":0.77200000000000002},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.438,"5min":0.47199999999999998,"15min":0.51800000000000002},"min":{"1min":0.309,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.60299999999999998,"5min":0.70099999999999996,"15min":0.73999999999999999},"last":0.48499999999999999}]},{"osd":1,"last update":"Sat May 20 01:24:36 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.495,"15min":0.52100000000000002},"min":{"1min":0.312,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.70499999999999996,"5min":0.71699999999999997,"15min":0.83799999999999997},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.50800000000000001,"15min":0.52400000000000002},"min":{"1min":0.33000000000000002,"5min":0.32600000000000001,"15min":0.318},"max":{"1min":0.68600000000000005,"5min":0.749,"15min":0.79200000000000004},"last":0.57999999999999996}]},{"osd":2,"last update":"Sat May 20 01:24:36 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.48199999999999998,"15min":0.504},"min":{"1min":0.36799999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.52700000000000002,"5min":0.72199999999999998,"15min":0.76600000000000001},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.51800000000000002,"15min":0.55000000000000004},"min":{"1min":0.35899999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.629,"5min":0.76200000000000001,"15min":0.76800000000000002},"last":0.64300000000000002}]},{"osd":3,"last update":"Sat May 20 01:24:36 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.51500000000000001,"15min":0.53500000000000003},"min":{"1min":0.32400000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.80800000000000005},"last":0.624},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.52100000000000002,"15min":0.54000000000000004},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.58099999999999996,"5min":0.76400000000000001,"15min":0.79500000000000004},"last":0.50600000000000001}]},{"osd":4,"last update":"Sat May 20 01:24:36 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.53800000000000003,"15min":0.55000000000000004},"min":{"1min":0.42599999999999999,"5min":0.33300000000000002,"15min":0.32600000000000001},"max":{"1min":0.754,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.54700000000000004,"15min":0.56899999999999995},"min":{"1min":0.40300000000000002,"5min":0.378,"15min":0.32700000000000001},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.80100000000000005},"last":0.44}]},{"osd":5,"last update":"Sat May 20 01:24:36 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.54100000000000004,"15min":0.54400000000000004},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.65000000000000002,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.38},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.57599999999999996,"15min":0.60399999999999998},"min":{"1min":0.46100000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.78600000000000003,"5min":0.81599999999999995,"15min":0.89400000000000002},"last":0.51500000000000001}]},{"osd":7,"last update":"Sat May 20 01:24:36 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57699999999999996,"15min":0.59799999999999998},"min":{"1min":0.44,"5min":0.371,"15min":0.371},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.92900000000000005},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.58099999999999996,"15min":0.60999999999999999},"min":{"1min":0.48099999999999998,"5min":0.379,"15min":0.379},"max":{"1min":0.68100000000000005,"5min":0.82799999999999996,"15min":0.85199999999999998},"last":0.59499999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574957,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":16376,"kb_used_data":1584,"kb_used_omap":0,"kb_used_meta":14784,"kb_avail":94355464,"statfs":{"total":96636764160,"available":96619995136,"internally_reserved":0,"allocated":1622016,"data_stored":1106512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":15138816},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:24:33 2023","interfaces":[{"interface":"back","average":{"1min":0.254,"5min":0.28199999999999997,"15min":0.313},"min":{"1min":0.186,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":0.37,"5min":0.57099999999999995,"15min":1.194},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.28299999999999997,"5min":0.314,"15min":0.316},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.41399999999999998,"5min":0.61499999999999999,"15min":0.75800000000000001},"last":0.73199999999999998}]},{"osd":2,"last update":"Sat May 20 01:24:33 2023","interfaces":[{"interface":"back","average":{"1min":0.27200000000000002,"5min":0.30199999999999999,"15min":0.317},"min":{"1min":0.20100000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.33800000000000002,"5min":0.52400000000000002,"15min":0.66400000000000003},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.33700000000000002,"5min":0.36799999999999999,"15min":0.40600000000000003},"min":{"1min":0.27800000000000002,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.41299999999999998,"5min":0.60299999999999998,"15min":1.3580000000000001},"last":0.497}]},{"osd":3,"last update":"Sat May 20 01:24:33 2023","interfaces":[{"interface":"back","average":{"1min":0.315,"5min":0.34999999999999998,"15min":0.36899999999999999},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.40500000000000003,"5min":0.57399999999999995,"15min":1.619},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.32100000000000001,"5min":0.34699999999999998,"15min":0.379},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.41299999999999998,"5min":0.60899999999999999,"15min":1.8400000000000001},"last":0.55300000000000005}]},{"osd":4,"last update":"Sat May 20 01:24:33 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.55900000000000005,"15min":0.58099999999999996},"min":{"1min":0.379,"5min":0.34899999999999998,"15min":0.32200000000000001},"max":{"1min":0.72799999999999998,"5min":0.85299999999999998,"15min":1.2769999999999999},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.56999999999999995,"15min":0.58299999999999996},"min":{"1min":0.41699999999999998,"5min":0.312,"15min":0.312},"max":{"1min":0.65600000000000003,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.56899999999999995}]},{"osd":5,"last update":"Sat May 20 01:24:33 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.58399999999999996,"15min":0.58899999999999997},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.754,"5min":0.81899999999999995,"15min":1.4510000000000001},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.59299999999999997,"15min":0.60599999999999998},"min":{"1min":0.436,"5min":0.39100000000000001,"15min":0.378},"max":{"1min":0.77200000000000002,"5min":0.79900000000000004,"15min":1.409},"last":0.58699999999999997}]},{"osd":6,"last update":"Sat May 20 01:24:33 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.59499999999999997,"15min":0.60399999999999998},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.73599999999999999,"5min":0.876,"15min":1.575},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.58499999999999996,"15min":0.59399999999999997},"min":{"1min":0.41299999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.66400000000000003,"5min":0.83099999999999996,"15min":1.0600000000000001},"last":0.60799999999999998}]},{"osd":7,"last update":"Sat May 20 01:24:33 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.59499999999999997,"15min":0.60499999999999998},"min":{"1min":0.40500000000000003,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.75600000000000001,"5min":0.88700000000000001,"15min":1.496},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.623,"15min":0.63800000000000001},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.77300000000000002,"5min":2.1709999999999998,"15min":2.1709999999999998},"last":1.5009999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574957,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10224,"kb_used_data":1000,"kb_used_omap":0,"kb_used_meta":9216,"kb_avail":94361616,"statfs":{"total":96636764160,"available":96626294784,"internally_reserved":0,"allocated":1024000,"data_stored":515183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9437184},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat May 20 01:24:32 2023","interfaces":[{"interface":"back","average":{"1min":0.26200000000000001,"5min":0.29199999999999998,"15min":0.318},"min":{"1min":0.192,"5min":0.17999999999999999,"15min":0.17999999999999999},"max":{"1min":0.371,"5min":0.54600000000000004,"15min":0.56599999999999995},"last":0.247},{"interface":"front","average":{"1min":0.26400000000000001,"5min":0.29699999999999999,"15min":0.34200000000000003},"min":{"1min":0.21099999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.33300000000000002,"5min":0.52000000000000002,"15min":0.98799999999999999},"last":0.36899999999999999}]},{"osd":2,"last update":"Sat May 20 01:24:32 2023","interfaces":[{"interface":"back","average":{"1min":0.28100000000000003,"5min":0.29999999999999999,"15min":0.32800000000000001},"min":{"1min":0.20100000000000001,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":0.35999999999999999,"5min":0.52300000000000002,"15min":0.55700000000000005},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.33100000000000002,"5min":0.34899999999999998,"15min":0.38200000000000001},"min":{"1min":0.26200000000000001,"5min":0.216,"15min":0.216},"max":{"1min":0.41699999999999998,"5min":0.60299999999999998,"15min":0.86199999999999999},"last":0.41799999999999998}]},{"osd":3,"last update":"Sat May 20 01:24:32 2023","interfaces":[{"interface":"back","average":{"1min":0.29599999999999999,"5min":0.33200000000000002,"15min":0.35999999999999999},"min":{"1min":0.24299999999999999,"5min":0.22,"15min":0.22},"max":{"1min":0.374,"5min":0.623,"15min":0.68300000000000005},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.35399999999999998,"15min":0.36599999999999999},"min":{"1min":0.28000000000000003,"5min":0.245,"15min":0.245},"max":{"1min":0.44700000000000001,"5min":0.60899999999999999,"15min":0.60899999999999999},"last":0.44900000000000001}]},{"osd":4,"last update":"Sat May 20 01:24:32 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.57099999999999995,"15min":0.55100000000000005},"min":{"1min":0.433,"5min":0.372,"15min":0.34399999999999997},"max":{"1min":0.78800000000000003,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.503},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.57999999999999996,"15min":0.56899999999999995},"min":{"1min":0.40799999999999997,"5min":0.39200000000000002,"15min":0.36299999999999999},"max":{"1min":0.70899999999999996,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.42799999999999999}]},{"osd":5,"last update":"Sat May 20 01:24:32 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.57799999999999996,"15min":0.57599999999999996},"min":{"1min":0.379,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.74299999999999999,"5min":0.80500000000000005,"15min":0.88100000000000001},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.63400000000000001,"15min":0.58899999999999997},"min":{"1min":0.40899999999999997,"5min":0.374,"15min":0.374},"max":{"1min":0.76300000000000001,"5min":4.4720000000000004,"15min":4.4720000000000004},"last":0.623}]},{"osd":6,"last update":"Sat May 20 01:24:32 2023","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":0.72799999999999998,"15min":0.625},"min":{"1min":0.45800000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":10.042999999999999,"5min":10.042999999999999,"15min":10.042999999999999},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.60499999999999998,"15min":0.61599999999999999},"min":{"1min":0.45200000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.73399999999999999,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.40200000000000002}]},{"osd":7,"last update":"Sat May 20 01:24:32 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.59499999999999997,"15min":0.59299999999999997},"min":{"1min":0.433,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.74199999999999999,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.57499999999999996},{"interface":"front","average":{"1min":1.1040000000000001,"5min":0.72499999999999998,"15min":0.65700000000000003},"min":{"1min":0.44800000000000001,"5min":0.438,"15min":0.42099999999999999},"max":{"1min":10.085000000000001,"5min":10.085000000000001,"15min":10.085000000000001},"last":0.53600000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574957,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10292,"kb_used_data":1004,"kb_used_omap":0,"kb_used_meta":9280,"kb_avail":94361548,"statfs":{"total":96636764160,"available":96626225152,"internally_reserved":0,"allocated":1028096,"data_stored":516144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9502720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:24:44 2023","interfaces":[{"interface":"back","average":{"1min":0.255,"5min":0.27500000000000002,"15min":0.32400000000000001},"min":{"1min":0.19500000000000001,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":0.36799999999999999,"5min":0.53400000000000003,"15min":1.038},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.26700000000000002,"5min":0.29699999999999999,"15min":0.34200000000000003},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.36799999999999999,"5min":0.46500000000000002,"15min":1.079},"last":0.23899999999999999}]},{"osd":1,"last update":"Sat May 20 01:24:44 2023","interfaces":[{"interface":"back","average":{"1min":0.28999999999999998,"5min":0.314,"15min":0.38100000000000001},"min":{"1min":0.23400000000000001,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.34899999999999998,"5min":0.61399999999999999,"15min":1.4950000000000001},"last":0.434},{"interface":"front","average":{"1min":0.318,"5min":0.35499999999999998,"15min":0.41799999999999998},"min":{"1min":0.27400000000000002,"5min":0.249,"15min":0.249},"max":{"1min":0.36099999999999999,"5min":0.629,"15min":1.5189999999999999},"last":0.313}]},{"osd":3,"last update":"Sat May 20 01:24:44 2023","interfaces":[{"interface":"back","average":{"1min":0.27800000000000002,"5min":0.32400000000000001,"15min":0.38200000000000001},"min":{"1min":0.218,"5min":0.216,"15min":0.216},"max":{"1min":0.42499999999999999,"5min":0.60599999999999998,"15min":1.2090000000000001},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.34000000000000002,"5min":0.36699999999999999,"15min":0.432},"min":{"1min":0.30199999999999999,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.38800000000000001,"5min":0.68100000000000005,"15min":1.5620000000000001},"last":0.38900000000000001}]},{"osd":4,"last update":"Sat May 20 01:24:44 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.55500000000000005,"15min":0.57699999999999996},"min":{"1min":0.38800000000000001,"5min":0.35899999999999999,"15min":0.34899999999999998},"max":{"1min":0.69199999999999995,"5min":0.85199999999999998,"15min":1.319},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.58499999999999996,"15min":0.61199999999999999},"min":{"1min":0.432,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.69499999999999995,"5min":0.89000000000000001,"15min":1.4319999999999999},"last":0.51400000000000001}]},{"osd":5,"last update":"Sat May 20 01:24:44 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.55800000000000005,"15min":0.62},"min":{"1min":0.41999999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.73599999999999999,"5min":0.95599999999999996,"15min":1.758},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.58299999999999996,"15min":0.63800000000000001},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.72999999999999998,"5min":0.85099999999999998,"15min":1.7110000000000001},"last":0.66900000000000004}]},{"osd":6,"last update":"Sat May 20 01:24:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.58599999999999997,"15min":0.63200000000000001},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.39700000000000002},"max":{"1min":0.746,"5min":0.88900000000000001,"15min":1.665},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.58299999999999996,"15min":0.63200000000000001},"min":{"1min":0.41699999999999998,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.74299999999999999,"5min":0.93500000000000005,"15min":1.819},"last":0.48999999999999999}]},{"osd":7,"last update":"Sat May 20 01:24:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.59499999999999997,"15min":0.65700000000000003},"min":{"1min":0.44400000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.76900000000000002,"5min":0.85099999999999998,"15min":1.7310000000000001},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.59899999999999998,"15min":0.65300000000000002},"min":{"1min":0.46999999999999997,"5min":0.42399999999999999,"15min":0.42199999999999999},"max":{"1min":0.72499999999999998,"5min":0.90600000000000003,"15min":1.8460000000000001},"last":0.52800000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574957,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10420,"kb_used_data":1004,"kb_used_omap":0,"kb_used_meta":9408,"kb_avail":94361420,"statfs":{"total":96636764160,"available":96626094080,"internally_reserved":0,"allocated":1028096,"data_stored":516144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9633792},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:24:32 2023","interfaces":[{"interface":"back","average":{"1min":0.27300000000000002,"5min":0.30599999999999999,"15min":0.317},"min":{"1min":0.21199999999999999,"5min":0.188,"15min":0.188},"max":{"1min":0.375,"5min":1.528,"15min":1.528},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.28000000000000003,"5min":0.34200000000000003,"15min":0.42299999999999999},"min":{"1min":0.214,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.40699999999999997,"5min":2.4209999999999998,"15min":3.1890000000000001},"last":0.35799999999999998}]},{"osd":1,"last update":"Sat May 20 01:24:32 2023","interfaces":[{"interface":"back","average":{"1min":0.27300000000000002,"5min":0.33600000000000002,"15min":0.41499999999999998},"min":{"1min":0.19900000000000001,"5min":0.189,"15min":0.189},"max":{"1min":0.377,"5min":3.2999999999999998,"15min":3.4260000000000002},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.33200000000000002,"5min":0.34200000000000003,"15min":0.35599999999999998},"min":{"1min":0.23799999999999999,"5min":0.22900000000000001,"15min":0.221},"max":{"1min":0.45500000000000002,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.32300000000000001}]},{"osd":2,"last update":"Sat May 20 01:24:32 2023","interfaces":[{"interface":"back","average":{"1min":0.30599999999999999,"5min":0.34300000000000003,"15min":0.38700000000000001},"min":{"1min":0.26400000000000001,"5min":0.245,"15min":0.224},"max":{"1min":0.39200000000000002,"5min":0.625,"15min":1.1619999999999999},"last":0.503},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.371,"15min":0.41299999999999998},"min":{"1min":0.28499999999999998,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.41499999999999998,"5min":0.72399999999999998,"15min":1.413},"last":0.35099999999999998}]},{"osd":4,"last update":"Sat May 20 01:24:32 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.55100000000000005,"15min":0.58699999999999997},"min":{"1min":0.40300000000000002,"5min":0.38600000000000001,"15min":0.38200000000000001},"max":{"1min":0.72899999999999998,"5min":0.95399999999999996,"15min":1.0700000000000001},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.61199999999999999,"15min":0.68000000000000005},"min":{"1min":0.41099999999999998,"5min":0.40200000000000002,"15min":0.35899999999999999},"max":{"1min":0.72899999999999998,"5min":3.1789999999999998,"15min":3.4660000000000002},"last":0.58399999999999996}]},{"osd":5,"last update":"Sat May 20 01:24:32 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.59899999999999998,"15min":0.68500000000000005},"min":{"1min":0.38400000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.751,"5min":3.2170000000000001,"15min":3.5379999999999998},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.59999999999999998,"15min":0.63200000000000001},"min":{"1min":0.41999999999999998,"5min":0.40799999999999997,"15min":0.374},"max":{"1min":0.74199999999999999,"5min":1.026,"15min":1.385},"last":0.46700000000000003}]},{"osd":6,"last update":"Sat May 20 01:24:32 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.59599999999999997,"15min":0.63400000000000001},"min":{"1min":0.439,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.70199999999999996,"5min":0.99199999999999999,"15min":1.347},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.58499999999999996,"15min":0.63},"min":{"1min":0.39500000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.752,"5min":0.94099999999999995,"15min":1.107},"last":0.54000000000000004}]},{"osd":7,"last update":"Sat May 20 01:24:32 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.59699999999999998,"15min":0.63700000000000001},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.64000000000000001,"5min":0.94199999999999995,"15min":1.1359999999999999},"last":0.439},{"interface":"front","average":{"1min":0.625,"5min":0.65100000000000002,"15min":0.72499999999999998},"min":{"1min":0.47599999999999998,"5min":0.436,"15min":0.436},"max":{"1min":0.73799999999999999,"5min":3.2559999999999998,"15min":3.5009999999999999},"last":0.60099999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574957,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":15732,"kb_used_data":1004,"kb_used_omap":0,"kb_used_meta":14720,"kb_avail":94356108,"statfs":{"total":96636764160,"available":96620654592,"internally_reserved":0,"allocated":1028096,"data_stored":516144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":15073280},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:24:33 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.47299999999999998,"15min":0.48099999999999998},"min":{"1min":0.32200000000000001,"5min":0.247,"15min":0.247},"max":{"1min":0.59199999999999997,"5min":0.73699999999999999,"15min":0.75600000000000001},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.50700000000000001,"15min":0.498},"min":{"1min":0.30599999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.64800000000000002,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":0.44900000000000001}]},{"osd":1,"last update":"Sat May 20 01:24:33 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.49199999999999999,"15min":0.51900000000000002},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.65300000000000002,"5min":0.746,"15min":0.82799999999999996},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.51900000000000002,"15min":0.53000000000000003},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.63600000000000001,"5min":1.7190000000000001,"15min":2.0739999999999998},"last":0.42399999999999999}]},{"osd":2,"last update":"Sat May 20 01:24:33 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.51900000000000002,"15min":0.50800000000000001},"min":{"1min":0.36599999999999999,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.64900000000000002,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.54200000000000004,"15min":0.55000000000000004},"min":{"1min":0.378,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.61399999999999999,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.51100000000000001}]},{"osd":3,"last update":"Sat May 20 01:24:33 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.52600000000000002,"15min":0.52400000000000002},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34399999999999997},"max":{"1min":0.626,"5min":0.81399999999999995,"15min":1.827},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.57099999999999995,"15min":0.65500000000000003},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.65000000000000002,"5min":0.89300000000000002,"15min":3.1749999999999998},"last":0.57599999999999996}]},{"osd":5,"last update":"Sat May 20 01:24:33 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.53400000000000003,"15min":0.48699999999999999},"min":{"1min":0.36899999999999999,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.76700000000000002,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.55400000000000005,"15min":0.51300000000000001},"min":{"1min":0.35099999999999998,"5min":0.33400000000000002,"15min":0.29899999999999999},"max":{"1min":0.78400000000000003,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.58899999999999997}]},{"osd":6,"last update":"Sat May 20 01:24:33 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.58799999999999997,"15min":0.54700000000000004},"min":{"1min":0.432,"5min":0.40400000000000003,"15min":0.32400000000000001},"max":{"1min":0.80300000000000005,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.61599999999999999,"15min":0.59099999999999997},"min":{"1min":0.51100000000000001,"5min":0.42399999999999999,"15min":0.39400000000000002},"max":{"1min":0.72299999999999998,"5min":0.84699999999999998,"15min":0.85899999999999999},"last":0.60799999999999998}]},{"osd":7,"last update":"Sat May 20 01:24:33 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.57799999999999996,"15min":0.55700000000000005},"min":{"1min":0.41999999999999998,"5min":0.36099999999999999,"15min":0.34200000000000003},"max":{"1min":0.71799999999999997,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.61399999999999999,"15min":0.59799999999999998},"min":{"1min":0.499,"5min":0.36399999999999999,"15min":0.34000000000000002},"max":{"1min":0.73699999999999999,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.65400000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574958,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10160,"kb_used_data":1000,"kb_used_omap":0,"kb_used_meta":9152,"kb_avail":94361680,"statfs":{"total":96636764160,"available":96626360320,"internally_reserved":0,"allocated":1024000,"data_stored":515183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9371648},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:24:38 2023","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.45400000000000001,"15min":0.65300000000000002},"min":{"1min":0.29999999999999999,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.68899999999999995,"5min":0.72599999999999998,"15min":6.3929999999999998},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.46000000000000002,"15min":0.68000000000000005},"min":{"1min":0.34999999999999998,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.61099999999999999,"5min":0.67500000000000004,"15min":6.3339999999999996},"last":0.499}]},{"osd":1,"last update":"Sat May 20 01:24:38 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.46000000000000002,"15min":0.58799999999999997},"min":{"1min":0.313,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":3.2450000000000001},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.498,"5min":0.499,"15min":0.53400000000000003},"min":{"1min":0.38900000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.72599999999999998,"5min":0.752,"15min":1.1819999999999999},"last":0.48199999999999998}]},{"osd":2,"last update":"Sat May 20 01:24:38 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48299999999999998,"15min":0.54100000000000004},"min":{"1min":0.33200000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":1.1639999999999999},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.48399999999999999,"15min":0.54400000000000004},"min":{"1min":0.308,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.628,"5min":0.71799999999999997,"15min":1.258},"last":0.58799999999999997}]},{"osd":3,"last update":"Sat May 20 01:24:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.51500000000000001,"15min":0.72499999999999998},"min":{"1min":0.35099999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.69499999999999995,"5min":0.77800000000000002,"15min":6.351},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.53200000000000003,"15min":0.75800000000000001},"min":{"1min":0.40699999999999997,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.67500000000000004,"5min":0.76500000000000001,"15min":6.3730000000000002},"last":0.55400000000000005}]},{"osd":4,"last update":"Sat May 20 01:24:38 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.503,"15min":0.49299999999999999},"min":{"1min":0.38900000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.64500000000000002,"5min":0.68400000000000005,"15min":0.70199999999999996},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.54000000000000004,"15min":0.55100000000000005},"min":{"1min":0.41599999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.749,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.41399999999999998}]},{"osd":6,"last update":"Sat May 20 01:24:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.54600000000000004,"15min":0.55200000000000005},"min":{"1min":0.48699999999999999,"5min":0.38100000000000001,"15min":0.29999999999999999},"max":{"1min":0.72899999999999998,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.54500000000000004,"15min":0.55800000000000005},"min":{"1min":0.437,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.72399999999999998,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.51200000000000001}]},{"osd":7,"last update":"Sat May 20 01:24:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.56799999999999995,"15min":0.58199999999999996},"min":{"1min":0.46500000000000002,"5min":0.33400000000000002,"15min":0.33000000000000002},"max":{"1min":0.77300000000000002,"5min":0.82299999999999995,"15min":0.91000000000000003},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.57699999999999996,"15min":0.60799999999999998},"min":{"1min":0.44700000000000001,"5min":0.40999999999999998,"15min":0.39500000000000002},"max":{"1min":0.76200000000000001,"5min":0.81399999999999995,"15min":0.94499999999999995},"last":0.61299999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-05-20T01:25:17.335 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.3f is not active+clean 2023-05-20T01:25:17.335 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:25:11.552637+0000', 'last_change': '2023-05-20T01:25:11.552637+0000', 'last_active': '2023-05-20T01:25:11.552637+0000', 'last_peered': '2023-05-20T01:25:11.552637+0000', 'last_clean': '2023-05-20T01:25:11.552637+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:25:11.552637+0000', 'last_undegraded': '2023-05-20T01:25:11.552637+0000', 'last_fullsized': '2023-05-20T01:25:11.552637+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_clean_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:25:17.336 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.3e is not active+clean 2023-05-20T01:25:17.336 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:25:11.552637+0000', 'last_change': '2023-05-20T01:25:11.552637+0000', 'last_active': '2023-05-20T01:25:11.552637+0000', 'last_peered': '2023-05-20T01:25:11.552637+0000', 'last_clean': '2023-05-20T01:25:11.552637+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:25:11.552637+0000', 'last_undegraded': '2023-05-20T01:25:11.552637+0000', 'last_fullsized': '2023-05-20T01:25:11.552637+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_clean_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:25:17.336 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.3d is not active+clean 2023-05-20T01:25:17.337 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:25:11.552637+0000', 'last_change': '2023-05-20T01:25:11.552637+0000', 'last_active': '2023-05-20T01:25:11.552637+0000', 'last_peered': '2023-05-20T01:25:11.552637+0000', 'last_clean': '2023-05-20T01:25:11.552637+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:25:11.552637+0000', 'last_undegraded': '2023-05-20T01:25:11.552637+0000', 'last_fullsized': '2023-05-20T01:25:11.552637+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_clean_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:25:17.337 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.3c is not active+clean 2023-05-20T01:25:17.337 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:25:11.552637+0000', 'last_change': '2023-05-20T01:25:11.552637+0000', 'last_active': '2023-05-20T01:25:11.552637+0000', 'last_peered': '2023-05-20T01:25:11.552637+0000', 'last_clean': '2023-05-20T01:25:11.552637+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:25:11.552637+0000', 'last_undegraded': '2023-05-20T01:25:11.552637+0000', 'last_fullsized': '2023-05-20T01:25:11.552637+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_clean_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:25:17.337 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.3b is not active+clean 2023-05-20T01:25:17.338 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:25:11.552637+0000', 'last_change': '2023-05-20T01:25:11.552637+0000', 'last_active': '2023-05-20T01:25:11.552637+0000', 'last_peered': '2023-05-20T01:25:11.552637+0000', 'last_clean': '2023-05-20T01:25:11.552637+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:25:11.552637+0000', 'last_undegraded': '2023-05-20T01:25:11.552637+0000', 'last_fullsized': '2023-05-20T01:25:11.552637+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_clean_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:25:17.338 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.3a is not active+clean 2023-05-20T01:25:17.338 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:25:11.552637+0000', 'last_change': '2023-05-20T01:25:11.552637+0000', 'last_active': '2023-05-20T01:25:11.552637+0000', 'last_peered': '2023-05-20T01:25:11.552637+0000', 'last_clean': '2023-05-20T01:25:11.552637+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:25:11.552637+0000', 'last_undegraded': '2023-05-20T01:25:11.552637+0000', 'last_fullsized': '2023-05-20T01:25:11.552637+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_clean_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:25:17.339 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.39 is not active+clean 2023-05-20T01:25:17.339 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:25:11.552637+0000', 'last_change': '2023-05-20T01:25:11.552637+0000', 'last_active': '2023-05-20T01:25:11.552637+0000', 'last_peered': '2023-05-20T01:25:11.552637+0000', 'last_clean': '2023-05-20T01:25:11.552637+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:25:11.552637+0000', 'last_undegraded': '2023-05-20T01:25:11.552637+0000', 'last_fullsized': '2023-05-20T01:25:11.552637+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_clean_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:25:17.339 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.38 is not active+clean 2023-05-20T01:25:17.340 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:25:11.552637+0000', 'last_change': '2023-05-20T01:25:11.552637+0000', 'last_active': '2023-05-20T01:25:11.552637+0000', 'last_peered': '2023-05-20T01:25:11.552637+0000', 'last_clean': '2023-05-20T01:25:11.552637+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:25:11.552637+0000', 'last_undegraded': '2023-05-20T01:25:11.552637+0000', 'last_fullsized': '2023-05-20T01:25:11.552637+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_clean_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:25:17.340 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.37 is not active+clean 2023-05-20T01:25:17.340 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:25:11.552637+0000', 'last_change': '2023-05-20T01:25:11.552637+0000', 'last_active': '2023-05-20T01:25:11.552637+0000', 'last_peered': '2023-05-20T01:25:11.552637+0000', 'last_clean': '2023-05-20T01:25:11.552637+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:25:11.552637+0000', 'last_undegraded': '2023-05-20T01:25:11.552637+0000', 'last_fullsized': '2023-05-20T01:25:11.552637+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_clean_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:25:17.341 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.36 is not active+clean 2023-05-20T01:25:17.341 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:25:11.552637+0000', 'last_change': '2023-05-20T01:25:11.552637+0000', 'last_active': '2023-05-20T01:25:11.552637+0000', 'last_peered': '2023-05-20T01:25:11.552637+0000', 'last_clean': '2023-05-20T01:25:11.552637+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:25:11.552637+0000', 'last_undegraded': '2023-05-20T01:25:11.552637+0000', 'last_fullsized': '2023-05-20T01:25:11.552637+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_clean_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:25:17.341 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.35 is not active+clean 2023-05-20T01:25:17.342 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:25:11.552637+0000', 'last_change': '2023-05-20T01:25:11.552637+0000', 'last_active': '2023-05-20T01:25:11.552637+0000', 'last_peered': '2023-05-20T01:25:11.552637+0000', 'last_clean': '2023-05-20T01:25:11.552637+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:25:11.552637+0000', 'last_undegraded': '2023-05-20T01:25:11.552637+0000', 'last_fullsized': '2023-05-20T01:25:11.552637+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_clean_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:25:17.342 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.34 is not active+clean 2023-05-20T01:25:17.342 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:25:11.552637+0000', 'last_change': '2023-05-20T01:25:11.552637+0000', 'last_active': '2023-05-20T01:25:11.552637+0000', 'last_peered': '2023-05-20T01:25:11.552637+0000', 'last_clean': '2023-05-20T01:25:11.552637+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:25:11.552637+0000', 'last_undegraded': '2023-05-20T01:25:11.552637+0000', 'last_fullsized': '2023-05-20T01:25:11.552637+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_clean_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:25:17.343 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.33 is not active+clean 2023-05-20T01:25:17.343 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:25:11.552637+0000', 'last_change': '2023-05-20T01:25:11.552637+0000', 'last_active': '2023-05-20T01:25:11.552637+0000', 'last_peered': '2023-05-20T01:25:11.552637+0000', 'last_clean': '2023-05-20T01:25:11.552637+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:25:11.552637+0000', 'last_undegraded': '2023-05-20T01:25:11.552637+0000', 'last_fullsized': '2023-05-20T01:25:11.552637+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_clean_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:25:17.343 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.32 is not active+clean 2023-05-20T01:25:17.343 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:25:11.552637+0000', 'last_change': '2023-05-20T01:25:11.552637+0000', 'last_active': '2023-05-20T01:25:11.552637+0000', 'last_peered': '2023-05-20T01:25:11.552637+0000', 'last_clean': '2023-05-20T01:25:11.552637+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:25:11.552637+0000', 'last_undegraded': '2023-05-20T01:25:11.552637+0000', 'last_fullsized': '2023-05-20T01:25:11.552637+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_clean_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:25:17.344 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.31 is not active+clean 2023-05-20T01:25:17.345 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:25:11.552637+0000', 'last_change': '2023-05-20T01:25:11.552637+0000', 'last_active': '2023-05-20T01:25:11.552637+0000', 'last_peered': '2023-05-20T01:25:11.552637+0000', 'last_clean': '2023-05-20T01:25:11.552637+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:25:11.552637+0000', 'last_undegraded': '2023-05-20T01:25:11.552637+0000', 'last_fullsized': '2023-05-20T01:25:11.552637+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_clean_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:25:17.345 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.30 is not active+clean 2023-05-20T01:25:17.346 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:25:11.552637+0000', 'last_change': '2023-05-20T01:25:11.552637+0000', 'last_active': '2023-05-20T01:25:11.552637+0000', 'last_peered': '2023-05-20T01:25:11.552637+0000', 'last_clean': '2023-05-20T01:25:11.552637+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:25:11.552637+0000', 'last_undegraded': '2023-05-20T01:25:11.552637+0000', 'last_fullsized': '2023-05-20T01:25:11.552637+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_clean_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:25:17.347 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.2f is not active+clean 2023-05-20T01:25:17.348 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:25:11.552637+0000', 'last_change': '2023-05-20T01:25:11.552637+0000', 'last_active': '2023-05-20T01:25:11.552637+0000', 'last_peered': '2023-05-20T01:25:11.552637+0000', 'last_clean': '2023-05-20T01:25:11.552637+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:25:11.552637+0000', 'last_undegraded': '2023-05-20T01:25:11.552637+0000', 'last_fullsized': '2023-05-20T01:25:11.552637+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_clean_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:25:17.348 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.2e is not active+clean 2023-05-20T01:25:17.349 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:25:11.552637+0000', 'last_change': '2023-05-20T01:25:11.552637+0000', 'last_active': '2023-05-20T01:25:11.552637+0000', 'last_peered': '2023-05-20T01:25:11.552637+0000', 'last_clean': '2023-05-20T01:25:11.552637+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:25:11.552637+0000', 'last_undegraded': '2023-05-20T01:25:11.552637+0000', 'last_fullsized': '2023-05-20T01:25:11.552637+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_clean_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:25:17.350 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.2d is not active+clean 2023-05-20T01:25:17.351 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:25:11.552637+0000', 'last_change': '2023-05-20T01:25:11.552637+0000', 'last_active': '2023-05-20T01:25:11.552637+0000', 'last_peered': '2023-05-20T01:25:11.552637+0000', 'last_clean': '2023-05-20T01:25:11.552637+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:25:11.552637+0000', 'last_undegraded': '2023-05-20T01:25:11.552637+0000', 'last_fullsized': '2023-05-20T01:25:11.552637+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_clean_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:25:17.351 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.2c is not active+clean 2023-05-20T01:25:17.352 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:25:11.552637+0000', 'last_change': '2023-05-20T01:25:11.552637+0000', 'last_active': '2023-05-20T01:25:11.552637+0000', 'last_peered': '2023-05-20T01:25:11.552637+0000', 'last_clean': '2023-05-20T01:25:11.552637+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:25:11.552637+0000', 'last_undegraded': '2023-05-20T01:25:11.552637+0000', 'last_fullsized': '2023-05-20T01:25:11.552637+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_clean_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:25:17.353 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.2b is not active+clean 2023-05-20T01:25:17.353 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:25:11.552637+0000', 'last_change': '2023-05-20T01:25:11.552637+0000', 'last_active': '2023-05-20T01:25:11.552637+0000', 'last_peered': '2023-05-20T01:25:11.552637+0000', 'last_clean': '2023-05-20T01:25:11.552637+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:25:11.552637+0000', 'last_undegraded': '2023-05-20T01:25:11.552637+0000', 'last_fullsized': '2023-05-20T01:25:11.552637+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_clean_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:25:17.354 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.2a is not active+clean 2023-05-20T01:25:17.354 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:25:11.552637+0000', 'last_change': '2023-05-20T01:25:11.552637+0000', 'last_active': '2023-05-20T01:25:11.552637+0000', 'last_peered': '2023-05-20T01:25:11.552637+0000', 'last_clean': '2023-05-20T01:25:11.552637+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:25:11.552637+0000', 'last_undegraded': '2023-05-20T01:25:11.552637+0000', 'last_fullsized': '2023-05-20T01:25:11.552637+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_clean_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:25:17.355 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.29 is not active+clean 2023-05-20T01:25:17.355 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:25:11.552637+0000', 'last_change': '2023-05-20T01:25:11.552637+0000', 'last_active': '2023-05-20T01:25:11.552637+0000', 'last_peered': '2023-05-20T01:25:11.552637+0000', 'last_clean': '2023-05-20T01:25:11.552637+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:25:11.552637+0000', 'last_undegraded': '2023-05-20T01:25:11.552637+0000', 'last_fullsized': '2023-05-20T01:25:11.552637+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_clean_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:25:17.356 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.28 is not active+clean 2023-05-20T01:25:17.356 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:25:11.552637+0000', 'last_change': '2023-05-20T01:25:11.552637+0000', 'last_active': '2023-05-20T01:25:11.552637+0000', 'last_peered': '2023-05-20T01:25:11.552637+0000', 'last_clean': '2023-05-20T01:25:11.552637+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:25:11.552637+0000', 'last_undegraded': '2023-05-20T01:25:11.552637+0000', 'last_fullsized': '2023-05-20T01:25:11.552637+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_clean_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:25:17.356 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.27 is not active+clean 2023-05-20T01:25:17.357 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:25:11.552637+0000', 'last_change': '2023-05-20T01:25:11.552637+0000', 'last_active': '2023-05-20T01:25:11.552637+0000', 'last_peered': '2023-05-20T01:25:11.552637+0000', 'last_clean': '2023-05-20T01:25:11.552637+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:25:11.552637+0000', 'last_undegraded': '2023-05-20T01:25:11.552637+0000', 'last_fullsized': '2023-05-20T01:25:11.552637+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_clean_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:25:17.357 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.26 is not active+clean 2023-05-20T01:25:17.357 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:25:11.552637+0000', 'last_change': '2023-05-20T01:25:11.552637+0000', 'last_active': '2023-05-20T01:25:11.552637+0000', 'last_peered': '2023-05-20T01:25:11.552637+0000', 'last_clean': '2023-05-20T01:25:11.552637+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:25:11.552637+0000', 'last_undegraded': '2023-05-20T01:25:11.552637+0000', 'last_fullsized': '2023-05-20T01:25:11.552637+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_clean_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:25:17.358 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.25 is not active+clean 2023-05-20T01:25:17.358 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:25:11.552637+0000', 'last_change': '2023-05-20T01:25:11.552637+0000', 'last_active': '2023-05-20T01:25:11.552637+0000', 'last_peered': '2023-05-20T01:25:11.552637+0000', 'last_clean': '2023-05-20T01:25:11.552637+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:25:11.552637+0000', 'last_undegraded': '2023-05-20T01:25:11.552637+0000', 'last_fullsized': '2023-05-20T01:25:11.552637+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_clean_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:25:17.358 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.24 is not active+clean 2023-05-20T01:25:17.358 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:25:11.552637+0000', 'last_change': '2023-05-20T01:25:11.552637+0000', 'last_active': '2023-05-20T01:25:11.552637+0000', 'last_peered': '2023-05-20T01:25:11.552637+0000', 'last_clean': '2023-05-20T01:25:11.552637+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:25:11.552637+0000', 'last_undegraded': '2023-05-20T01:25:11.552637+0000', 'last_fullsized': '2023-05-20T01:25:11.552637+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_clean_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:25:17.359 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.23 is not active+clean 2023-05-20T01:25:17.359 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:25:11.552637+0000', 'last_change': '2023-05-20T01:25:11.552637+0000', 'last_active': '2023-05-20T01:25:11.552637+0000', 'last_peered': '2023-05-20T01:25:11.552637+0000', 'last_clean': '2023-05-20T01:25:11.552637+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:25:11.552637+0000', 'last_undegraded': '2023-05-20T01:25:11.552637+0000', 'last_fullsized': '2023-05-20T01:25:11.552637+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_clean_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:25:17.359 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.22 is not active+clean 2023-05-20T01:25:17.360 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:25:11.552637+0000', 'last_change': '2023-05-20T01:25:11.552637+0000', 'last_active': '2023-05-20T01:25:11.552637+0000', 'last_peered': '2023-05-20T01:25:11.552637+0000', 'last_clean': '2023-05-20T01:25:11.552637+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:25:11.552637+0000', 'last_undegraded': '2023-05-20T01:25:11.552637+0000', 'last_fullsized': '2023-05-20T01:25:11.552637+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_clean_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:25:17.360 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.21 is not active+clean 2023-05-20T01:25:17.360 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:25:11.552637+0000', 'last_change': '2023-05-20T01:25:11.552637+0000', 'last_active': '2023-05-20T01:25:11.552637+0000', 'last_peered': '2023-05-20T01:25:11.552637+0000', 'last_clean': '2023-05-20T01:25:11.552637+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:25:11.552637+0000', 'last_undegraded': '2023-05-20T01:25:11.552637+0000', 'last_fullsized': '2023-05-20T01:25:11.552637+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_clean_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:25:17.361 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.20 is not active+clean 2023-05-20T01:25:17.361 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:25:11.552637+0000', 'last_change': '2023-05-20T01:25:11.552637+0000', 'last_active': '2023-05-20T01:25:11.552637+0000', 'last_peered': '2023-05-20T01:25:11.552637+0000', 'last_clean': '2023-05-20T01:25:11.552637+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:25:11.552637+0000', 'last_undegraded': '2023-05-20T01:25:11.552637+0000', 'last_fullsized': '2023-05-20T01:25:11.552637+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_clean_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:25:17.361 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.1e is not active+clean 2023-05-20T01:25:17.362 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:25:11.552637+0000', 'last_change': '2023-05-20T01:25:11.552637+0000', 'last_active': '2023-05-20T01:25:11.552637+0000', 'last_peered': '2023-05-20T01:25:11.552637+0000', 'last_clean': '2023-05-20T01:25:11.552637+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:25:11.552637+0000', 'last_undegraded': '2023-05-20T01:25:11.552637+0000', 'last_fullsized': '2023-05-20T01:25:11.552637+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_clean_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:25:17.362 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.1f is not active+clean 2023-05-20T01:25:17.362 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:25:10.545755+0000', 'last_change': '2023-05-20T01:25:10.545755+0000', 'last_active': '2023-05-20T01:25:10.545755+0000', 'last_peered': '2023-05-20T01:25:10.545755+0000', 'last_clean': '2023-05-20T01:25:10.545755+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:25:10.545755+0000', 'last_undegraded': '2023-05-20T01:25:10.545755+0000', 'last_fullsized': '2023-05-20T01:25:10.545755+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:25:10.545755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:25:10.545755+0000', 'last_clean_scrub_stamp': '2023-05-20T01:25:10.545755+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:25:17.363 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.1f is not active+clean 2023-05-20T01:25:17.363 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:25:11.552637+0000', 'last_change': '2023-05-20T01:25:11.552637+0000', 'last_active': '2023-05-20T01:25:11.552637+0000', 'last_peered': '2023-05-20T01:25:11.552637+0000', 'last_clean': '2023-05-20T01:25:11.552637+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:25:11.552637+0000', 'last_undegraded': '2023-05-20T01:25:11.552637+0000', 'last_fullsized': '2023-05-20T01:25:11.552637+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_clean_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:25:17.363 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.1e is not active+clean 2023-05-20T01:25:17.364 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:25:10.545755+0000', 'last_change': '2023-05-20T01:25:10.545755+0000', 'last_active': '2023-05-20T01:25:10.545755+0000', 'last_peered': '2023-05-20T01:25:10.545755+0000', 'last_clean': '2023-05-20T01:25:10.545755+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:25:10.545755+0000', 'last_undegraded': '2023-05-20T01:25:10.545755+0000', 'last_fullsized': '2023-05-20T01:25:10.545755+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:25:10.545755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:25:10.545755+0000', 'last_clean_scrub_stamp': '2023-05-20T01:25:10.545755+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:25:17.364 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.1c is not active+clean 2023-05-20T01:25:17.364 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:25:11.552637+0000', 'last_change': '2023-05-20T01:25:11.552637+0000', 'last_active': '2023-05-20T01:25:11.552637+0000', 'last_peered': '2023-05-20T01:25:11.552637+0000', 'last_clean': '2023-05-20T01:25:11.552637+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:25:11.552637+0000', 'last_undegraded': '2023-05-20T01:25:11.552637+0000', 'last_fullsized': '2023-05-20T01:25:11.552637+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_clean_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:25:17.365 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.1d is not active+clean 2023-05-20T01:25:17.365 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:25:10.545755+0000', 'last_change': '2023-05-20T01:25:10.545755+0000', 'last_active': '2023-05-20T01:25:10.545755+0000', 'last_peered': '2023-05-20T01:25:10.545755+0000', 'last_clean': '2023-05-20T01:25:10.545755+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:25:10.545755+0000', 'last_undegraded': '2023-05-20T01:25:10.545755+0000', 'last_fullsized': '2023-05-20T01:25:10.545755+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:25:10.545755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:25:10.545755+0000', 'last_clean_scrub_stamp': '2023-05-20T01:25:10.545755+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:25:17.365 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.1d is not active+clean 2023-05-20T01:25:17.365 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:25:11.552637+0000', 'last_change': '2023-05-20T01:25:11.552637+0000', 'last_active': '2023-05-20T01:25:11.552637+0000', 'last_peered': '2023-05-20T01:25:11.552637+0000', 'last_clean': '2023-05-20T01:25:11.552637+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:25:11.552637+0000', 'last_undegraded': '2023-05-20T01:25:11.552637+0000', 'last_fullsized': '2023-05-20T01:25:11.552637+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_clean_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:25:17.366 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.1c is not active+clean 2023-05-20T01:25:17.366 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:25:10.545755+0000', 'last_change': '2023-05-20T01:25:10.545755+0000', 'last_active': '2023-05-20T01:25:10.545755+0000', 'last_peered': '2023-05-20T01:25:10.545755+0000', 'last_clean': '2023-05-20T01:25:10.545755+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:25:10.545755+0000', 'last_undegraded': '2023-05-20T01:25:10.545755+0000', 'last_fullsized': '2023-05-20T01:25:10.545755+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:25:10.545755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:25:10.545755+0000', 'last_clean_scrub_stamp': '2023-05-20T01:25:10.545755+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:25:17.366 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.1a is not active+clean 2023-05-20T01:25:17.367 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:25:11.552637+0000', 'last_change': '2023-05-20T01:25:11.552637+0000', 'last_active': '2023-05-20T01:25:11.552637+0000', 'last_peered': '2023-05-20T01:25:11.552637+0000', 'last_clean': '2023-05-20T01:25:11.552637+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:25:11.552637+0000', 'last_undegraded': '2023-05-20T01:25:11.552637+0000', 'last_fullsized': '2023-05-20T01:25:11.552637+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_clean_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:25:17.367 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.1b is not active+clean 2023-05-20T01:25:17.367 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:25:10.545755+0000', 'last_change': '2023-05-20T01:25:10.545755+0000', 'last_active': '2023-05-20T01:25:10.545755+0000', 'last_peered': '2023-05-20T01:25:10.545755+0000', 'last_clean': '2023-05-20T01:25:10.545755+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:25:10.545755+0000', 'last_undegraded': '2023-05-20T01:25:10.545755+0000', 'last_fullsized': '2023-05-20T01:25:10.545755+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:25:10.545755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:25:10.545755+0000', 'last_clean_scrub_stamp': '2023-05-20T01:25:10.545755+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:25:17.368 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.1b is not active+clean 2023-05-20T01:25:17.368 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:25:11.552637+0000', 'last_change': '2023-05-20T01:25:11.552637+0000', 'last_active': '2023-05-20T01:25:11.552637+0000', 'last_peered': '2023-05-20T01:25:11.552637+0000', 'last_clean': '2023-05-20T01:25:11.552637+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:25:11.552637+0000', 'last_undegraded': '2023-05-20T01:25:11.552637+0000', 'last_fullsized': '2023-05-20T01:25:11.552637+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_clean_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:25:17.369 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.1a is not active+clean 2023-05-20T01:25:17.369 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:25:10.545755+0000', 'last_change': '2023-05-20T01:25:10.545755+0000', 'last_active': '2023-05-20T01:25:10.545755+0000', 'last_peered': '2023-05-20T01:25:10.545755+0000', 'last_clean': '2023-05-20T01:25:10.545755+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:25:10.545755+0000', 'last_undegraded': '2023-05-20T01:25:10.545755+0000', 'last_fullsized': '2023-05-20T01:25:10.545755+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:25:10.545755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:25:10.545755+0000', 'last_clean_scrub_stamp': '2023-05-20T01:25:10.545755+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:25:17.369 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.18 is not active+clean 2023-05-20T01:25:17.370 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:25:11.552637+0000', 'last_change': '2023-05-20T01:25:11.552637+0000', 'last_active': '2023-05-20T01:25:11.552637+0000', 'last_peered': '2023-05-20T01:25:11.552637+0000', 'last_clean': '2023-05-20T01:25:11.552637+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:25:11.552637+0000', 'last_undegraded': '2023-05-20T01:25:11.552637+0000', 'last_fullsized': '2023-05-20T01:25:11.552637+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_clean_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:25:17.370 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.19 is not active+clean 2023-05-20T01:25:17.370 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:25:10.545755+0000', 'last_change': '2023-05-20T01:25:10.545755+0000', 'last_active': '2023-05-20T01:25:10.545755+0000', 'last_peered': '2023-05-20T01:25:10.545755+0000', 'last_clean': '2023-05-20T01:25:10.545755+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:25:10.545755+0000', 'last_undegraded': '2023-05-20T01:25:10.545755+0000', 'last_fullsized': '2023-05-20T01:25:10.545755+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:25:10.545755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:25:10.545755+0000', 'last_clean_scrub_stamp': '2023-05-20T01:25:10.545755+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:25:17.371 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.19 is not active+clean 2023-05-20T01:25:17.371 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:25:11.552637+0000', 'last_change': '2023-05-20T01:25:11.552637+0000', 'last_active': '2023-05-20T01:25:11.552637+0000', 'last_peered': '2023-05-20T01:25:11.552637+0000', 'last_clean': '2023-05-20T01:25:11.552637+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:25:11.552637+0000', 'last_undegraded': '2023-05-20T01:25:11.552637+0000', 'last_fullsized': '2023-05-20T01:25:11.552637+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_clean_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:25:17.371 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.18 is not active+clean 2023-05-20T01:25:17.372 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:25:10.545755+0000', 'last_change': '2023-05-20T01:25:10.545755+0000', 'last_active': '2023-05-20T01:25:10.545755+0000', 'last_peered': '2023-05-20T01:25:10.545755+0000', 'last_clean': '2023-05-20T01:25:10.545755+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:25:10.545755+0000', 'last_undegraded': '2023-05-20T01:25:10.545755+0000', 'last_fullsized': '2023-05-20T01:25:10.545755+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:25:10.545755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:25:10.545755+0000', 'last_clean_scrub_stamp': '2023-05-20T01:25:10.545755+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:25:17.372 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.16 is not active+clean 2023-05-20T01:25:17.372 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:25:11.552637+0000', 'last_change': '2023-05-20T01:25:11.552637+0000', 'last_active': '2023-05-20T01:25:11.552637+0000', 'last_peered': '2023-05-20T01:25:11.552637+0000', 'last_clean': '2023-05-20T01:25:11.552637+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:25:11.552637+0000', 'last_undegraded': '2023-05-20T01:25:11.552637+0000', 'last_fullsized': '2023-05-20T01:25:11.552637+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_clean_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:25:17.373 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.17 is not active+clean 2023-05-20T01:25:17.373 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:25:10.545755+0000', 'last_change': '2023-05-20T01:25:10.545755+0000', 'last_active': '2023-05-20T01:25:10.545755+0000', 'last_peered': '2023-05-20T01:25:10.545755+0000', 'last_clean': '2023-05-20T01:25:10.545755+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:25:10.545755+0000', 'last_undegraded': '2023-05-20T01:25:10.545755+0000', 'last_fullsized': '2023-05-20T01:25:10.545755+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:25:10.545755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:25:10.545755+0000', 'last_clean_scrub_stamp': '2023-05-20T01:25:10.545755+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:25:17.373 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.17 is not active+clean 2023-05-20T01:25:17.374 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:25:11.552637+0000', 'last_change': '2023-05-20T01:25:11.552637+0000', 'last_active': '2023-05-20T01:25:11.552637+0000', 'last_peered': '2023-05-20T01:25:11.552637+0000', 'last_clean': '2023-05-20T01:25:11.552637+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:25:11.552637+0000', 'last_undegraded': '2023-05-20T01:25:11.552637+0000', 'last_fullsized': '2023-05-20T01:25:11.552637+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_clean_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:25:17.375 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.16 is not active+clean 2023-05-20T01:25:17.375 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:25:10.545755+0000', 'last_change': '2023-05-20T01:25:10.545755+0000', 'last_active': '2023-05-20T01:25:10.545755+0000', 'last_peered': '2023-05-20T01:25:10.545755+0000', 'last_clean': '2023-05-20T01:25:10.545755+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:25:10.545755+0000', 'last_undegraded': '2023-05-20T01:25:10.545755+0000', 'last_fullsized': '2023-05-20T01:25:10.545755+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:25:10.545755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:25:10.545755+0000', 'last_clean_scrub_stamp': '2023-05-20T01:25:10.545755+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:25:17.375 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.14 is not active+clean 2023-05-20T01:25:17.376 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:25:11.552637+0000', 'last_change': '2023-05-20T01:25:11.552637+0000', 'last_active': '2023-05-20T01:25:11.552637+0000', 'last_peered': '2023-05-20T01:25:11.552637+0000', 'last_clean': '2023-05-20T01:25:11.552637+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:25:11.552637+0000', 'last_undegraded': '2023-05-20T01:25:11.552637+0000', 'last_fullsized': '2023-05-20T01:25:11.552637+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_clean_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:25:17.376 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.15 is not active+clean 2023-05-20T01:25:17.376 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:25:10.545755+0000', 'last_change': '2023-05-20T01:25:10.545755+0000', 'last_active': '2023-05-20T01:25:10.545755+0000', 'last_peered': '2023-05-20T01:25:10.545755+0000', 'last_clean': '2023-05-20T01:25:10.545755+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:25:10.545755+0000', 'last_undegraded': '2023-05-20T01:25:10.545755+0000', 'last_fullsized': '2023-05-20T01:25:10.545755+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:25:10.545755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:25:10.545755+0000', 'last_clean_scrub_stamp': '2023-05-20T01:25:10.545755+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:25:17.377 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.15 is not active+clean 2023-05-20T01:25:17.377 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:25:11.552637+0000', 'last_change': '2023-05-20T01:25:11.552637+0000', 'last_active': '2023-05-20T01:25:11.552637+0000', 'last_peered': '2023-05-20T01:25:11.552637+0000', 'last_clean': '2023-05-20T01:25:11.552637+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:25:11.552637+0000', 'last_undegraded': '2023-05-20T01:25:11.552637+0000', 'last_fullsized': '2023-05-20T01:25:11.552637+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_clean_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:25:17.378 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.14 is not active+clean 2023-05-20T01:25:17.378 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:25:10.545755+0000', 'last_change': '2023-05-20T01:25:10.545755+0000', 'last_active': '2023-05-20T01:25:10.545755+0000', 'last_peered': '2023-05-20T01:25:10.545755+0000', 'last_clean': '2023-05-20T01:25:10.545755+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:25:10.545755+0000', 'last_undegraded': '2023-05-20T01:25:10.545755+0000', 'last_fullsized': '2023-05-20T01:25:10.545755+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:25:10.545755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:25:10.545755+0000', 'last_clean_scrub_stamp': '2023-05-20T01:25:10.545755+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:25:17.378 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.12 is not active+clean 2023-05-20T01:25:17.379 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:25:11.552637+0000', 'last_change': '2023-05-20T01:25:11.552637+0000', 'last_active': '2023-05-20T01:25:11.552637+0000', 'last_peered': '2023-05-20T01:25:11.552637+0000', 'last_clean': '2023-05-20T01:25:11.552637+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:25:11.552637+0000', 'last_undegraded': '2023-05-20T01:25:11.552637+0000', 'last_fullsized': '2023-05-20T01:25:11.552637+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_clean_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:25:17.380 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.13 is not active+clean 2023-05-20T01:25:17.380 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:25:10.545755+0000', 'last_change': '2023-05-20T01:25:10.545755+0000', 'last_active': '2023-05-20T01:25:10.545755+0000', 'last_peered': '2023-05-20T01:25:10.545755+0000', 'last_clean': '2023-05-20T01:25:10.545755+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:25:10.545755+0000', 'last_undegraded': '2023-05-20T01:25:10.545755+0000', 'last_fullsized': '2023-05-20T01:25:10.545755+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:25:10.545755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:25:10.545755+0000', 'last_clean_scrub_stamp': '2023-05-20T01:25:10.545755+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:25:17.381 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.13 is not active+clean 2023-05-20T01:25:17.382 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:25:11.552637+0000', 'last_change': '2023-05-20T01:25:11.552637+0000', 'last_active': '2023-05-20T01:25:11.552637+0000', 'last_peered': '2023-05-20T01:25:11.552637+0000', 'last_clean': '2023-05-20T01:25:11.552637+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:25:11.552637+0000', 'last_undegraded': '2023-05-20T01:25:11.552637+0000', 'last_fullsized': '2023-05-20T01:25:11.552637+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_clean_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:25:17.383 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.12 is not active+clean 2023-05-20T01:25:17.384 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:25:10.545755+0000', 'last_change': '2023-05-20T01:25:10.545755+0000', 'last_active': '2023-05-20T01:25:10.545755+0000', 'last_peered': '2023-05-20T01:25:10.545755+0000', 'last_clean': '2023-05-20T01:25:10.545755+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:25:10.545755+0000', 'last_undegraded': '2023-05-20T01:25:10.545755+0000', 'last_fullsized': '2023-05-20T01:25:10.545755+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:25:10.545755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:25:10.545755+0000', 'last_clean_scrub_stamp': '2023-05-20T01:25:10.545755+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:25:17.384 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.10 is not active+clean 2023-05-20T01:25:17.385 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:25:11.552637+0000', 'last_change': '2023-05-20T01:25:11.552637+0000', 'last_active': '2023-05-20T01:25:11.552637+0000', 'last_peered': '2023-05-20T01:25:11.552637+0000', 'last_clean': '2023-05-20T01:25:11.552637+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:25:11.552637+0000', 'last_undegraded': '2023-05-20T01:25:11.552637+0000', 'last_fullsized': '2023-05-20T01:25:11.552637+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_clean_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:25:17.385 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.11 is not active+clean 2023-05-20T01:25:17.386 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:25:10.545755+0000', 'last_change': '2023-05-20T01:25:10.545755+0000', 'last_active': '2023-05-20T01:25:10.545755+0000', 'last_peered': '2023-05-20T01:25:10.545755+0000', 'last_clean': '2023-05-20T01:25:10.545755+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:25:10.545755+0000', 'last_undegraded': '2023-05-20T01:25:10.545755+0000', 'last_fullsized': '2023-05-20T01:25:10.545755+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:25:10.545755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:25:10.545755+0000', 'last_clean_scrub_stamp': '2023-05-20T01:25:10.545755+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:25:17.386 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.11 is not active+clean 2023-05-20T01:25:17.386 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:25:11.552637+0000', 'last_change': '2023-05-20T01:25:11.552637+0000', 'last_active': '2023-05-20T01:25:11.552637+0000', 'last_peered': '2023-05-20T01:25:11.552637+0000', 'last_clean': '2023-05-20T01:25:11.552637+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:25:11.552637+0000', 'last_undegraded': '2023-05-20T01:25:11.552637+0000', 'last_fullsized': '2023-05-20T01:25:11.552637+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_clean_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:25:17.387 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.10 is not active+clean 2023-05-20T01:25:17.387 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:25:10.545755+0000', 'last_change': '2023-05-20T01:25:10.545755+0000', 'last_active': '2023-05-20T01:25:10.545755+0000', 'last_peered': '2023-05-20T01:25:10.545755+0000', 'last_clean': '2023-05-20T01:25:10.545755+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:25:10.545755+0000', 'last_undegraded': '2023-05-20T01:25:10.545755+0000', 'last_fullsized': '2023-05-20T01:25:10.545755+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:25:10.545755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:25:10.545755+0000', 'last_clean_scrub_stamp': '2023-05-20T01:25:10.545755+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:25:17.387 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.6 is not active+clean 2023-05-20T01:25:17.387 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:25:11.552637+0000', 'last_change': '2023-05-20T01:25:11.552637+0000', 'last_active': '2023-05-20T01:25:11.552637+0000', 'last_peered': '2023-05-20T01:25:11.552637+0000', 'last_clean': '2023-05-20T01:25:11.552637+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:25:11.552637+0000', 'last_undegraded': '2023-05-20T01:25:11.552637+0000', 'last_fullsized': '2023-05-20T01:25:11.552637+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_clean_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:25:17.388 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.7 is not active+clean 2023-05-20T01:25:17.388 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:25:10.545755+0000', 'last_change': '2023-05-20T01:25:10.545755+0000', 'last_active': '2023-05-20T01:25:10.545755+0000', 'last_peered': '2023-05-20T01:25:10.545755+0000', 'last_clean': '2023-05-20T01:25:10.545755+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:25:10.545755+0000', 'last_undegraded': '2023-05-20T01:25:10.545755+0000', 'last_fullsized': '2023-05-20T01:25:10.545755+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:25:10.545755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:25:10.545755+0000', 'last_clean_scrub_stamp': '2023-05-20T01:25:10.545755+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:25:17.388 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.7 is not active+clean 2023-05-20T01:25:17.389 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:25:11.552637+0000', 'last_change': '2023-05-20T01:25:11.552637+0000', 'last_active': '2023-05-20T01:25:11.552637+0000', 'last_peered': '2023-05-20T01:25:11.552637+0000', 'last_clean': '2023-05-20T01:25:11.552637+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:25:11.552637+0000', 'last_undegraded': '2023-05-20T01:25:11.552637+0000', 'last_fullsized': '2023-05-20T01:25:11.552637+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_clean_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:25:17.389 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.6 is not active+clean 2023-05-20T01:25:17.389 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:25:10.545755+0000', 'last_change': '2023-05-20T01:25:10.545755+0000', 'last_active': '2023-05-20T01:25:10.545755+0000', 'last_peered': '2023-05-20T01:25:10.545755+0000', 'last_clean': '2023-05-20T01:25:10.545755+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:25:10.545755+0000', 'last_undegraded': '2023-05-20T01:25:10.545755+0000', 'last_fullsized': '2023-05-20T01:25:10.545755+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:25:10.545755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:25:10.545755+0000', 'last_clean_scrub_stamp': '2023-05-20T01:25:10.545755+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:25:17.390 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.4 is not active+clean 2023-05-20T01:25:17.390 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:25:11.552637+0000', 'last_change': '2023-05-20T01:25:11.552637+0000', 'last_active': '2023-05-20T01:25:11.552637+0000', 'last_peered': '2023-05-20T01:25:11.552637+0000', 'last_clean': '2023-05-20T01:25:11.552637+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:25:11.552637+0000', 'last_undegraded': '2023-05-20T01:25:11.552637+0000', 'last_fullsized': '2023-05-20T01:25:11.552637+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_clean_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:25:17.390 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.5 is not active+clean 2023-05-20T01:25:17.391 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:25:10.545755+0000', 'last_change': '2023-05-20T01:25:10.545755+0000', 'last_active': '2023-05-20T01:25:10.545755+0000', 'last_peered': '2023-05-20T01:25:10.545755+0000', 'last_clean': '2023-05-20T01:25:10.545755+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:25:10.545755+0000', 'last_undegraded': '2023-05-20T01:25:10.545755+0000', 'last_fullsized': '2023-05-20T01:25:10.545755+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:25:10.545755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:25:10.545755+0000', 'last_clean_scrub_stamp': '2023-05-20T01:25:10.545755+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:25:17.391 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.5 is not active+clean 2023-05-20T01:25:17.391 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:25:11.552637+0000', 'last_change': '2023-05-20T01:25:11.552637+0000', 'last_active': '2023-05-20T01:25:11.552637+0000', 'last_peered': '2023-05-20T01:25:11.552637+0000', 'last_clean': '2023-05-20T01:25:11.552637+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:25:11.552637+0000', 'last_undegraded': '2023-05-20T01:25:11.552637+0000', 'last_fullsized': '2023-05-20T01:25:11.552637+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_clean_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:25:17.392 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.4 is not active+clean 2023-05-20T01:25:17.392 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:25:10.545755+0000', 'last_change': '2023-05-20T01:25:10.545755+0000', 'last_active': '2023-05-20T01:25:10.545755+0000', 'last_peered': '2023-05-20T01:25:10.545755+0000', 'last_clean': '2023-05-20T01:25:10.545755+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:25:10.545755+0000', 'last_undegraded': '2023-05-20T01:25:10.545755+0000', 'last_fullsized': '2023-05-20T01:25:10.545755+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:25:10.545755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:25:10.545755+0000', 'last_clean_scrub_stamp': '2023-05-20T01:25:10.545755+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:25:17.392 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.2 is not active+clean 2023-05-20T01:25:17.392 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:25:11.552637+0000', 'last_change': '2023-05-20T01:25:11.552637+0000', 'last_active': '2023-05-20T01:25:11.552637+0000', 'last_peered': '2023-05-20T01:25:11.552637+0000', 'last_clean': '2023-05-20T01:25:11.552637+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:25:11.552637+0000', 'last_undegraded': '2023-05-20T01:25:11.552637+0000', 'last_fullsized': '2023-05-20T01:25:11.552637+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_clean_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:25:17.393 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.3 is not active+clean 2023-05-20T01:25:17.393 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:25:10.545755+0000', 'last_change': '2023-05-20T01:25:10.545755+0000', 'last_active': '2023-05-20T01:25:10.545755+0000', 'last_peered': '2023-05-20T01:25:10.545755+0000', 'last_clean': '2023-05-20T01:25:10.545755+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:25:10.545755+0000', 'last_undegraded': '2023-05-20T01:25:10.545755+0000', 'last_fullsized': '2023-05-20T01:25:10.545755+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:25:10.545755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:25:10.545755+0000', 'last_clean_scrub_stamp': '2023-05-20T01:25:10.545755+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:25:17.393 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.3 is not active+clean 2023-05-20T01:25:17.394 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:25:11.552637+0000', 'last_change': '2023-05-20T01:25:11.552637+0000', 'last_active': '2023-05-20T01:25:11.552637+0000', 'last_peered': '2023-05-20T01:25:11.552637+0000', 'last_clean': '2023-05-20T01:25:11.552637+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:25:11.552637+0000', 'last_undegraded': '2023-05-20T01:25:11.552637+0000', 'last_fullsized': '2023-05-20T01:25:11.552637+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_clean_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:25:17.394 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.2 is not active+clean 2023-05-20T01:25:17.395 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:25:10.545755+0000', 'last_change': '2023-05-20T01:25:10.545755+0000', 'last_active': '2023-05-20T01:25:10.545755+0000', 'last_peered': '2023-05-20T01:25:10.545755+0000', 'last_clean': '2023-05-20T01:25:10.545755+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:25:10.545755+0000', 'last_undegraded': '2023-05-20T01:25:10.545755+0000', 'last_fullsized': '2023-05-20T01:25:10.545755+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:25:10.545755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:25:10.545755+0000', 'last_clean_scrub_stamp': '2023-05-20T01:25:10.545755+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:25:17.395 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.0 is not active+clean 2023-05-20T01:25:17.395 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:25:11.552637+0000', 'last_change': '2023-05-20T01:25:11.552637+0000', 'last_active': '2023-05-20T01:25:11.552637+0000', 'last_peered': '2023-05-20T01:25:11.552637+0000', 'last_clean': '2023-05-20T01:25:11.552637+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:25:11.552637+0000', 'last_undegraded': '2023-05-20T01:25:11.552637+0000', 'last_fullsized': '2023-05-20T01:25:11.552637+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_clean_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:25:17.396 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.1 is not active+clean 2023-05-20T01:25:17.396 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:25:10.545755+0000', 'last_change': '2023-05-20T01:25:10.545755+0000', 'last_active': '2023-05-20T01:25:10.545755+0000', 'last_peered': '2023-05-20T01:25:10.545755+0000', 'last_clean': '2023-05-20T01:25:10.545755+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:25:10.545755+0000', 'last_undegraded': '2023-05-20T01:25:10.545755+0000', 'last_fullsized': '2023-05-20T01:25:10.545755+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:25:10.545755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:25:10.545755+0000', 'last_clean_scrub_stamp': '2023-05-20T01:25:10.545755+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:25:17.396 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.1 is not active+clean 2023-05-20T01:25:17.396 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:25:11.552637+0000', 'last_change': '2023-05-20T01:25:11.552637+0000', 'last_active': '2023-05-20T01:25:11.552637+0000', 'last_peered': '2023-05-20T01:25:11.552637+0000', 'last_clean': '2023-05-20T01:25:11.552637+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:25:11.552637+0000', 'last_undegraded': '2023-05-20T01:25:11.552637+0000', 'last_fullsized': '2023-05-20T01:25:11.552637+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_clean_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:25:17.397 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.0 is not active+clean 2023-05-20T01:25:17.397 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:25:10.545755+0000', 'last_change': '2023-05-20T01:25:10.545755+0000', 'last_active': '2023-05-20T01:25:10.545755+0000', 'last_peered': '2023-05-20T01:25:10.545755+0000', 'last_clean': '2023-05-20T01:25:10.545755+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:25:10.545755+0000', 'last_undegraded': '2023-05-20T01:25:10.545755+0000', 'last_fullsized': '2023-05-20T01:25:10.545755+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:25:10.545755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:25:10.545755+0000', 'last_clean_scrub_stamp': '2023-05-20T01:25:10.545755+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:25:17.397 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.9 is not active+clean 2023-05-20T01:25:17.398 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:25:11.552637+0000', 'last_change': '2023-05-20T01:25:11.552637+0000', 'last_active': '2023-05-20T01:25:11.552637+0000', 'last_peered': '2023-05-20T01:25:11.552637+0000', 'last_clean': '2023-05-20T01:25:11.552637+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:25:11.552637+0000', 'last_undegraded': '2023-05-20T01:25:11.552637+0000', 'last_fullsized': '2023-05-20T01:25:11.552637+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_clean_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:25:17.398 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.8 is not active+clean 2023-05-20T01:25:17.398 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:25:10.545755+0000', 'last_change': '2023-05-20T01:25:10.545755+0000', 'last_active': '2023-05-20T01:25:10.545755+0000', 'last_peered': '2023-05-20T01:25:10.545755+0000', 'last_clean': '2023-05-20T01:25:10.545755+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:25:10.545755+0000', 'last_undegraded': '2023-05-20T01:25:10.545755+0000', 'last_fullsized': '2023-05-20T01:25:10.545755+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:25:10.545755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:25:10.545755+0000', 'last_clean_scrub_stamp': '2023-05-20T01:25:10.545755+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:25:17.399 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.8 is not active+clean 2023-05-20T01:25:17.399 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:25:11.552637+0000', 'last_change': '2023-05-20T01:25:11.552637+0000', 'last_active': '2023-05-20T01:25:11.552637+0000', 'last_peered': '2023-05-20T01:25:11.552637+0000', 'last_clean': '2023-05-20T01:25:11.552637+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:25:11.552637+0000', 'last_undegraded': '2023-05-20T01:25:11.552637+0000', 'last_fullsized': '2023-05-20T01:25:11.552637+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_clean_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:25:17.399 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.9 is not active+clean 2023-05-20T01:25:17.400 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:25:10.545755+0000', 'last_change': '2023-05-20T01:25:10.545755+0000', 'last_active': '2023-05-20T01:25:10.545755+0000', 'last_peered': '2023-05-20T01:25:10.545755+0000', 'last_clean': '2023-05-20T01:25:10.545755+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:25:10.545755+0000', 'last_undegraded': '2023-05-20T01:25:10.545755+0000', 'last_fullsized': '2023-05-20T01:25:10.545755+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:25:10.545755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:25:10.545755+0000', 'last_clean_scrub_stamp': '2023-05-20T01:25:10.545755+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:25:17.400 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.f is not active+clean 2023-05-20T01:25:17.400 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:25:11.552637+0000', 'last_change': '2023-05-20T01:25:11.552637+0000', 'last_active': '2023-05-20T01:25:11.552637+0000', 'last_peered': '2023-05-20T01:25:11.552637+0000', 'last_clean': '2023-05-20T01:25:11.552637+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:25:11.552637+0000', 'last_undegraded': '2023-05-20T01:25:11.552637+0000', 'last_fullsized': '2023-05-20T01:25:11.552637+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_clean_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:25:17.401 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.e is not active+clean 2023-05-20T01:25:17.401 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:25:10.545755+0000', 'last_change': '2023-05-20T01:25:10.545755+0000', 'last_active': '2023-05-20T01:25:10.545755+0000', 'last_peered': '2023-05-20T01:25:10.545755+0000', 'last_clean': '2023-05-20T01:25:10.545755+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:25:10.545755+0000', 'last_undegraded': '2023-05-20T01:25:10.545755+0000', 'last_fullsized': '2023-05-20T01:25:10.545755+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:25:10.545755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:25:10.545755+0000', 'last_clean_scrub_stamp': '2023-05-20T01:25:10.545755+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:25:17.401 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.e is not active+clean 2023-05-20T01:25:17.401 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:25:11.552637+0000', 'last_change': '2023-05-20T01:25:11.552637+0000', 'last_active': '2023-05-20T01:25:11.552637+0000', 'last_peered': '2023-05-20T01:25:11.552637+0000', 'last_clean': '2023-05-20T01:25:11.552637+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:25:11.552637+0000', 'last_undegraded': '2023-05-20T01:25:11.552637+0000', 'last_fullsized': '2023-05-20T01:25:11.552637+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_clean_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:25:17.402 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.f is not active+clean 2023-05-20T01:25:17.402 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:25:10.545755+0000', 'last_change': '2023-05-20T01:25:10.545755+0000', 'last_active': '2023-05-20T01:25:10.545755+0000', 'last_peered': '2023-05-20T01:25:10.545755+0000', 'last_clean': '2023-05-20T01:25:10.545755+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:25:10.545755+0000', 'last_undegraded': '2023-05-20T01:25:10.545755+0000', 'last_fullsized': '2023-05-20T01:25:10.545755+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:25:10.545755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:25:10.545755+0000', 'last_clean_scrub_stamp': '2023-05-20T01:25:10.545755+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:25:17.402 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.c is not active+clean 2023-05-20T01:25:17.403 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:25:11.552637+0000', 'last_change': '2023-05-20T01:25:11.552637+0000', 'last_active': '2023-05-20T01:25:11.552637+0000', 'last_peered': '2023-05-20T01:25:11.552637+0000', 'last_clean': '2023-05-20T01:25:11.552637+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:25:11.552637+0000', 'last_undegraded': '2023-05-20T01:25:11.552637+0000', 'last_fullsized': '2023-05-20T01:25:11.552637+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_clean_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:25:17.403 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.d is not active+clean 2023-05-20T01:25:17.403 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:25:10.545755+0000', 'last_change': '2023-05-20T01:25:10.545755+0000', 'last_active': '2023-05-20T01:25:10.545755+0000', 'last_peered': '2023-05-20T01:25:10.545755+0000', 'last_clean': '2023-05-20T01:25:10.545755+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:25:10.545755+0000', 'last_undegraded': '2023-05-20T01:25:10.545755+0000', 'last_fullsized': '2023-05-20T01:25:10.545755+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:25:10.545755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:25:10.545755+0000', 'last_clean_scrub_stamp': '2023-05-20T01:25:10.545755+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:25:17.404 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.d is not active+clean 2023-05-20T01:25:17.404 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:25:11.552637+0000', 'last_change': '2023-05-20T01:25:11.552637+0000', 'last_active': '2023-05-20T01:25:11.552637+0000', 'last_peered': '2023-05-20T01:25:11.552637+0000', 'last_clean': '2023-05-20T01:25:11.552637+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:25:11.552637+0000', 'last_undegraded': '2023-05-20T01:25:11.552637+0000', 'last_fullsized': '2023-05-20T01:25:11.552637+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_clean_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:25:17.404 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.c is not active+clean 2023-05-20T01:25:17.405 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:25:10.545755+0000', 'last_change': '2023-05-20T01:25:10.545755+0000', 'last_active': '2023-05-20T01:25:10.545755+0000', 'last_peered': '2023-05-20T01:25:10.545755+0000', 'last_clean': '2023-05-20T01:25:10.545755+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:25:10.545755+0000', 'last_undegraded': '2023-05-20T01:25:10.545755+0000', 'last_fullsized': '2023-05-20T01:25:10.545755+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:25:10.545755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:25:10.545755+0000', 'last_clean_scrub_stamp': '2023-05-20T01:25:10.545755+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:25:17.405 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.a is not active+clean 2023-05-20T01:25:17.405 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:25:11.552637+0000', 'last_change': '2023-05-20T01:25:11.552637+0000', 'last_active': '2023-05-20T01:25:11.552637+0000', 'last_peered': '2023-05-20T01:25:11.552637+0000', 'last_clean': '2023-05-20T01:25:11.552637+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:25:11.552637+0000', 'last_undegraded': '2023-05-20T01:25:11.552637+0000', 'last_fullsized': '2023-05-20T01:25:11.552637+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_clean_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:25:17.406 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.b is not active+clean 2023-05-20T01:25:17.406 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:25:10.545755+0000', 'last_change': '2023-05-20T01:25:10.545755+0000', 'last_active': '2023-05-20T01:25:10.545755+0000', 'last_peered': '2023-05-20T01:25:10.545755+0000', 'last_clean': '2023-05-20T01:25:10.545755+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:25:10.545755+0000', 'last_undegraded': '2023-05-20T01:25:10.545755+0000', 'last_fullsized': '2023-05-20T01:25:10.545755+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:25:10.545755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:25:10.545755+0000', 'last_clean_scrub_stamp': '2023-05-20T01:25:10.545755+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:25:17.406 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.b is not active+clean 2023-05-20T01:25:17.407 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:25:11.552637+0000', 'last_change': '2023-05-20T01:25:11.552637+0000', 'last_active': '2023-05-20T01:25:11.552637+0000', 'last_peered': '2023-05-20T01:25:11.552637+0000', 'last_clean': '2023-05-20T01:25:11.552637+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:25:11.552637+0000', 'last_undegraded': '2023-05-20T01:25:11.552637+0000', 'last_fullsized': '2023-05-20T01:25:11.552637+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'last_clean_scrub_stamp': '2023-05-20T01:25:11.552637+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:25:17.407 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.a is not active+clean 2023-05-20T01:25:17.407 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:25:10.545755+0000', 'last_change': '2023-05-20T01:25:10.545755+0000', 'last_active': '2023-05-20T01:25:10.545755+0000', 'last_peered': '2023-05-20T01:25:10.545755+0000', 'last_clean': '2023-05-20T01:25:10.545755+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:25:10.545755+0000', 'last_undegraded': '2023-05-20T01:25:10.545755+0000', 'last_fullsized': '2023-05-20T01:25:10.545755+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:25:10.545755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:25:10.545755+0000', 'last_clean_scrub_stamp': '2023-05-20T01:25:10.545755+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:25:17.408 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-05-20T01:25:17.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.529+0000 7fc0316a0700 1 Processor -- start 2023-05-20T01:25:17.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.529+0000 7fc0316a0700 1 -- start start 2023-05-20T01:25:17.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.529+0000 7fc0316a0700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc02c0cd650 0x7fc02c15cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:17.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.529+0000 7fc0316a0700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc02c0b13b0 con 0x7fc02c0cdad0 2023-05-20T01:25:17.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.529+0000 7fc0316a0700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc02c0b1690 con 0x7fc02c156d30 2023-05-20T01:25:17.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.529+0000 7fc0316a0700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc02c0b1970 con 0x7fc02c0cd650 2023-05-20T01:25:17.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.530+0000 7fc02a7fc700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc02c0cd650 0x7fc02c15cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:17.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.530+0000 7fc02b7fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fc02c0cdad0 0x7fc02c06ba70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:47834/0 (socket says 172.21.15.16:47834) 2023-05-20T01:25:17.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.530+0000 7fc02b7fe700 1 -- 172.21.15.16:0/3856155654 learned_addr learned my addr 172.21.15.16:0/3856155654 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:25:17.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.530+0000 7fc02a7fc700 1 -- 172.21.15.16:0/3856155654 >> v1:172.21.15.132:6789/0 conn(0x7fc02c156d30 legacy=0x7fc02c0ce550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:17.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.530+0000 7fc02a7fc700 1 -- 172.21.15.16:0/3856155654 >> v1:172.21.15.16:6789/0 conn(0x7fc02c0cdad0 legacy=0x7fc02c06ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:17.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.530+0000 7fc02a7fc700 1 -- 172.21.15.16:0/3856155654 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc02c0cce50 con 0x7fc02c0cd650 2023-05-20T01:25:17.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.530+0000 7fc029ffb700 1 -- 172.21.15.16:0/3856155654 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3412713418 0 0) 0x7fc02c0b13b0 con 0x7fc02c0cdad0 2023-05-20T01:25:17.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.530+0000 7fc02a7fc700 1 --2- 172.21.15.16:0/3856155654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc02c0cd650 0x7fc02c15cb60 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fc020009840 tx=0x7fc020009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dd9a5b260b5938d server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:17.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.530+0000 7fc029ffb700 1 -- 172.21.15.16:0/3856155654 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc020011830 con 0x7fc02c0cd650 2023-05-20T01:25:17.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.530+0000 7fc029ffb700 1 -- 172.21.15.16:0/3856155654 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc020011990 con 0x7fc02c0cd650 2023-05-20T01:25:17.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.530+0000 7fc029ffb700 1 -- 172.21.15.16:0/3856155654 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc020011d20 con 0x7fc02c0cd650 2023-05-20T01:25:17.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.531+0000 7fc0316a0700 1 -- 172.21.15.16:0/3856155654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc02c0cd650 msgr2=0x7fc02c15cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:17.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.531+0000 7fc0316a0700 1 --2- 172.21.15.16:0/3856155654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc02c0cd650 0x7fc02c15cb60 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fc020009840 tx=0x7fc020009b50 comp rx=0 tx=0).stop 2023-05-20T01:25:17.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.531+0000 7fc0316a0700 1 -- 172.21.15.16:0/3856155654 shutdown_connections 2023-05-20T01:25:17.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.531+0000 7fc0316a0700 1 --2- 172.21.15.16:0/3856155654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc02c0cd650 0x7fc02c15cb60 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:17.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.531+0000 7fc0316a0700 1 -- 172.21.15.16:0/3856155654 >> 172.21.15.16:0/3856155654 conn(0x7fc02c0cb910 msgr2=0x7fc02c0cbd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:17.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.531+0000 7fc0316a0700 1 -- 172.21.15.16:0/3856155654 shutdown_connections 2023-05-20T01:25:17.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.531+0000 7fc0316a0700 1 -- 172.21.15.16:0/3856155654 wait complete. 2023-05-20T01:25:17.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.532+0000 7fc0316a0700 1 Processor -- start 2023-05-20T01:25:17.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.532+0000 7fc0316a0700 1 -- start start 2023-05-20T01:25:17.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.533+0000 7fc0316a0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc02c0cd650 0x7fc02c0710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:17.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.533+0000 7fc0316a0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc02c0cdad0 0x7fc02c0aea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:17.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.533+0000 7fc0316a0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc02c156d30 0x7fc02c148fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:17.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.533+0000 7fc0316a0700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc02c0cec60 con 0x7fc02c156d30 2023-05-20T01:25:17.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.533+0000 7fc0316a0700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc02c070060 con 0x7fc02c0cd650 2023-05-20T01:25:17.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.533+0000 7fc0316a0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc02c070300 con 0x7fc02c0cdad0 2023-05-20T01:25:17.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.533+0000 7fc02b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc02c156d30 0x7fc02c148fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:17.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.533+0000 7fc02b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc02c156d30 0x7fc02c148fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36312/0 (socket says 172.21.15.16:36312) 2023-05-20T01:25:17.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.533+0000 7fc02b7fe700 1 -- 172.21.15.16:0/2283227228 learned_addr learned my addr 172.21.15.16:0/2283227228 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:17.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.533+0000 7fc02affd700 1 --2- 172.21.15.16:0/2283227228 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc02c0cd650 0x7fc02c0710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:17.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.533+0000 7fc02b7fe700 1 -- 172.21.15.16:0/2283227228 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc02c0cdad0 msgr2=0x7fc02c0aea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:17.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.533+0000 7fc02a7fc700 1 --2- 172.21.15.16:0/2283227228 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc02c0cdad0 0x7fc02c0aea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:17.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.533+0000 7fc02b7fe700 1 --2- 172.21.15.16:0/2283227228 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc02c0cdad0 0x7fc02c0aea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:17.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.533+0000 7fc02b7fe700 1 -- 172.21.15.16:0/2283227228 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc02c0cd650 msgr2=0x7fc02c0710c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:17.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.533+0000 7fc02b7fe700 1 --2- 172.21.15.16:0/2283227228 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc02c0cd650 0x7fc02c0710c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:17.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.533+0000 7fc02b7fe700 1 -- 172.21.15.16:0/2283227228 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0200094a0 con 0x7fc02c156d30 2023-05-20T01:25:17.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.533+0000 7fc02affd700 1 --2- 172.21.15.16:0/2283227228 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc02c0cd650 0x7fc02c0710c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:25:17.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.533+0000 7fc02a7fc700 1 --2- 172.21.15.16:0/2283227228 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc02c0cdad0 0x7fc02c0aea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:25:17.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.533+0000 7fc02b7fe700 1 --2- 172.21.15.16:0/2283227228 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc02c156d30 0x7fc02c148fa0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fc02400cc70 tx=0x7fc02400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:17.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.534+0000 7fc0137fe700 1 -- 172.21.15.16:0/2283227228 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc02400ad80 con 0x7fc02c156d30 2023-05-20T01:25:17.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.534+0000 7fc0137fe700 1 -- 172.21.15.16:0/2283227228 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc02400ce70 con 0x7fc02c156d30 2023-05-20T01:25:17.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.534+0000 7fc0316a0700 1 -- 172.21.15.16:0/2283227228 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc02c0cb4a0 con 0x7fc02c156d30 2023-05-20T01:25:17.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.534+0000 7fc0137fe700 1 -- 172.21.15.16:0/2283227228 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0240072f0 con 0x7fc02c156d30 2023-05-20T01:25:17.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.534+0000 7fc0316a0700 1 -- 172.21.15.16:0/2283227228 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc02c0cd130 con 0x7fc02c156d30 2023-05-20T01:25:17.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.535+0000 7fc0316a0700 1 -- 172.21.15.16:0/2283227228 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc02c0cb630 con 0x7fc02c156d30 2023-05-20T01:25:17.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.535+0000 7fc0137fe700 1 -- 172.21.15.16:0/2283227228 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc024007450 con 0x7fc02c156d30 2023-05-20T01:25:17.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.535+0000 7fc0137fe700 1 --2- 172.21.15.16:0/2283227228 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc01806ace0 0x7fc01806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:17.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.536+0000 7fc0137fe700 1 -- 172.21.15.16:0/2283227228 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fc0240894c0 con 0x7fc02c156d30 2023-05-20T01:25:17.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.536+0000 7fc02affd700 1 --2- 172.21.15.16:0/2283227228 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc01806ace0 0x7fc01806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:17.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.536+0000 7fc02affd700 1 --2- 172.21.15.16:0/2283227228 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc01806ace0 0x7fc01806d190 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fc01c000f60 tx=0x7fc01c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:17.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.540+0000 7fc0137fe700 1 -- 172.21.15.16:0/2283227228 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fc024054430 con 0x7fc02c156d30 2023-05-20T01:25:17.553 INFO:tasks.ceph.mds.c.smithi016.stderr:2023-05-20T01:25:17.549+0000 7f147f091700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-05-20T01:25:17.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.818+0000 7fc0316a0700 1 -- 172.21.15.16:0/2283227228 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fc02c070de0 con 0x7fc02c156d30 2023-05-20T01:25:17.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.819+0000 7fc0137fe700 1 -- 172.21.15.16:0/2283227228 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1023 (secure 0 0 0) 0x7fc02c070de0 con 0x7fc02c156d30 2023-05-20T01:25:17.824 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:25:17.824 INFO:teuthology.orchestra.run.smithi016.stdout:{"fsid":"c2245cc5-ecab-4087-a59b-bc81db901676","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":548,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":88,"num_osds":8,"num_up_osds":8,"osd_up_since":1684545379,"num_in_osds":8,"osd_in_since":1684545372,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":96},{"state_name":"active+clean","count":9}],"num_pgs":105,"num_pools":4,"num_objects":2,"data_bytes":590368,"bytes_used":97222656,"bytes_avail":772996890624,"bytes_total":773094113280,"unknown_pgs_ratio":0.91428571939468384},"fsmap":{"epoch":57,"id":4,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":4,"rank":0,"name":"c","status":"up:active","gid":7104}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":16,"modified":"2023-05-20T01:23:41.412868+0000","services":{}},"progress_events":{}} 2023-05-20T01:25:17.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.821+0000 7fc0316a0700 1 -- 172.21.15.16:0/2283227228 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc01806ace0 msgr2=0x7fc01806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:17.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.821+0000 7fc0316a0700 1 --2- 172.21.15.16:0/2283227228 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc01806ace0 0x7fc01806d190 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fc01c000f60 tx=0x7fc01c009040 comp rx=0 tx=0).stop 2023-05-20T01:25:17.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.821+0000 7fc0316a0700 1 -- 172.21.15.16:0/2283227228 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc02c156d30 msgr2=0x7fc02c148fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:17.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.821+0000 7fc0316a0700 1 --2- 172.21.15.16:0/2283227228 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc02c156d30 0x7fc02c148fa0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fc02400cc70 tx=0x7fc02400a340 comp rx=0 tx=0).stop 2023-05-20T01:25:17.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.821+0000 7fc0316a0700 1 -- 172.21.15.16:0/2283227228 shutdown_connections 2023-05-20T01:25:17.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.821+0000 7fc0316a0700 1 --2- 172.21.15.16:0/2283227228 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc02c156d30 0x7fc02c148fa0 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:17.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.821+0000 7fc0316a0700 1 --2- 172.21.15.16:0/2283227228 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc02c0cd650 0x7fc02c0710c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:17.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.821+0000 7fc0316a0700 1 --2- 172.21.15.16:0/2283227228 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc01806ace0 0x7fc01806d190 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:17.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.821+0000 7fc0316a0700 1 --2- 172.21.15.16:0/2283227228 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc02c0cdad0 0x7fc02c0aea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:17.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.821+0000 7fc0316a0700 1 -- 172.21.15.16:0/2283227228 >> 172.21.15.16:0/2283227228 conn(0x7fc02c0cb910 msgr2=0x7fc02c157190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:17.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.821+0000 7fc0316a0700 1 -- 172.21.15.16:0/2283227228 shutdown_connections 2023-05-20T01:25:17.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.821+0000 7fc0316a0700 1 -- 172.21.15.16:0/2283227228 wait complete. 2023-05-20T01:25:17.837 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-05-20T01:25:17.837 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-05-20T01:25:17.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.958+0000 7fd93ff25700 1 Processor -- start 2023-05-20T01:25:17.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.958+0000 7fd93ff25700 1 -- start start 2023-05-20T01:25:17.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.959+0000 7fd93ff25700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd93806ba90 0x7fd9380cbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:17.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.959+0000 7fd93ff25700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9380ce3e0 con 0x7fd9380d1000 2023-05-20T01:25:17.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.959+0000 7fd93ff25700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9380cdab0 con 0x7fd93806b620 2023-05-20T01:25:17.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.959+0000 7fd93ff25700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9380cdd90 con 0x7fd93806ba90 2023-05-20T01:25:17.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.959+0000 7fd93dcc1700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fd9380d1000 0x7fd9380d13d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:47844/0 (socket says 172.21.15.16:47844) 2023-05-20T01:25:17.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.959+0000 7fd93dcc1700 1 -- 172.21.15.16:0/3214934062 learned_addr learned my addr 172.21.15.16:0/3214934062 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:25:17.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.959+0000 7fd93e4c2700 1 --2- 172.21.15.16:0/3214934062 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd93806ba90 0x7fd9380cbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:17.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.959+0000 7fd93ccbf700 1 -- 172.21.15.16:0/3214934062 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2857434953 0 0) 0x7fd9380ce3e0 con 0x7fd9380d1000 2023-05-20T01:25:17.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.959+0000 7fd93ccbf700 1 -- 172.21.15.16:0/3214934062 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9240035b0 con 0x7fd9380d1000 2023-05-20T01:25:17.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.960+0000 7fd93ccbf700 1 -- 172.21.15.16:0/3214934062 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1410514988 0 0) 0x7fd9380cdab0 con 0x7fd93806b620 2023-05-20T01:25:17.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.960+0000 7fd93ccbf700 1 -- 172.21.15.16:0/3214934062 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9380ce3e0 con 0x7fd93806b620 2023-05-20T01:25:17.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.960+0000 7fd93ccbf700 1 -- 172.21.15.16:0/3214934062 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2787448161 0 0) 0x7fd9240035b0 con 0x7fd9380d1000 2023-05-20T01:25:17.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.960+0000 7fd93e4c2700 1 -- 172.21.15.16:0/3214934062 >> v1:172.21.15.132:6789/0 conn(0x7fd93806b620 legacy=0x7fd9380cb6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:17.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.960+0000 7fd93e4c2700 1 -- 172.21.15.16:0/3214934062 >> v1:172.21.15.16:6789/0 conn(0x7fd9380d1000 legacy=0x7fd9380d13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:17.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.960+0000 7fd93e4c2700 1 -- 172.21.15.16:0/3214934062 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9380ce070 con 0x7fd93806ba90 2023-05-20T01:25:17.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.960+0000 7fd93e4c2700 1 --2- 172.21.15.16:0/3214934062 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd93806ba90 0x7fd9380cbdf0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd930009dc0 tx=0x7fd93000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d3a5f1bb18be643 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:17.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.960+0000 7fd93ccbf700 1 -- 172.21.15.16:0/3214934062 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd93000e070 con 0x7fd93806ba90 2023-05-20T01:25:17.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.960+0000 7fd93ccbf700 1 -- 172.21.15.16:0/3214934062 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd930003a20 con 0x7fd93806ba90 2023-05-20T01:25:17.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.960+0000 7fd93ccbf700 1 -- 172.21.15.16:0/3214934062 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd928001670 con 0x7fd93806ba90 2023-05-20T01:25:17.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.961+0000 7fd93ff25700 1 -- 172.21.15.16:0/3214934062 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd93806ba90 msgr2=0x7fd9380cbdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:17.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.961+0000 7fd93ff25700 1 --2- 172.21.15.16:0/3214934062 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd93806ba90 0x7fd9380cbdf0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd930009dc0 tx=0x7fd93000b330 comp rx=0 tx=0).stop 2023-05-20T01:25:17.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.961+0000 7fd93ff25700 1 -- 172.21.15.16:0/3214934062 shutdown_connections 2023-05-20T01:25:17.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.961+0000 7fd93ff25700 1 --2- 172.21.15.16:0/3214934062 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd93806ba90 0x7fd9380cbdf0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:17.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.961+0000 7fd93ff25700 1 -- 172.21.15.16:0/3214934062 >> 172.21.15.16:0/3214934062 conn(0x7fd93806ae30 msgr2=0x7fd93809a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:17.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.961+0000 7fd93ff25700 1 -- 172.21.15.16:0/3214934062 shutdown_connections 2023-05-20T01:25:17.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.961+0000 7fd93ff25700 1 -- 172.21.15.16:0/3214934062 wait complete. 2023-05-20T01:25:17.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.962+0000 7fd93ff25700 1 Processor -- start 2023-05-20T01:25:17.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.962+0000 7fd93ff25700 1 -- start start 2023-05-20T01:25:17.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.963+0000 7fd93ff25700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd93806b620 0x7fd93815a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:17.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.963+0000 7fd93ff25700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd93806ba90 0x7fd9381e6e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:17.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.963+0000 7fd93ff25700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd9380d1000 0x7fd9381e4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:17.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.963+0000 7fd93ff25700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9380b4ab0 con 0x7fd93806ba90 2023-05-20T01:25:17.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.963+0000 7fd93ff25700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd93815adb0 con 0x7fd93806b620 2023-05-20T01:25:17.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.963+0000 7fd93ff25700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9380b3ec0 con 0x7fd9380d1000 2023-05-20T01:25:17.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.963+0000 7fd93d4c0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd93806ba90 0x7fd9381e6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:17.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.963+0000 7fd93d4c0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd93806ba90 0x7fd9381e6e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36324/0 (socket says 172.21.15.16:36324) 2023-05-20T01:25:17.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.963+0000 7fd93d4c0700 1 -- 172.21.15.16:0/162611433 learned_addr learned my addr 172.21.15.16:0/162611433 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:17.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.963+0000 7fd93e4c2700 1 --2- 172.21.15.16:0/162611433 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd9380d1000 0x7fd9381e4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:17.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.963+0000 7fd93dcc1700 1 --2- 172.21.15.16:0/162611433 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd93806b620 0x7fd93815a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:17.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.963+0000 7fd93d4c0700 1 -- 172.21.15.16:0/162611433 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd9380d1000 msgr2=0x7fd9381e4df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:17.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.963+0000 7fd93d4c0700 1 --2- 172.21.15.16:0/162611433 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd9380d1000 0x7fd9381e4df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:17.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.963+0000 7fd93d4c0700 1 -- 172.21.15.16:0/162611433 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd93806b620 msgr2=0x7fd93815a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:17.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.963+0000 7fd93d4c0700 1 --2- 172.21.15.16:0/162611433 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd93806b620 0x7fd93815a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:17.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.963+0000 7fd93d4c0700 1 -- 172.21.15.16:0/162611433 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd930009a70 con 0x7fd93806ba90 2023-05-20T01:25:17.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.963+0000 7fd93d4c0700 1 --2- 172.21.15.16:0/162611433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd93806ba90 0x7fd9381e6e90 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fd93400c500 tx=0x7fd93400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:17.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.963+0000 7fd92e7fc700 1 -- 172.21.15.16:0/162611433 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9340052d0 con 0x7fd93806ba90 2023-05-20T01:25:17.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.963+0000 7fd92e7fc700 1 -- 172.21.15.16:0/162611433 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd934005450 con 0x7fd93806ba90 2023-05-20T01:25:17.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.964+0000 7fd92e7fc700 1 -- 172.21.15.16:0/162611433 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9340038c0 con 0x7fd93806ba90 2023-05-20T01:25:17.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.964+0000 7fd93ff25700 1 -- 172.21.15.16:0/162611433 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9380ce870 con 0x7fd93806ba90 2023-05-20T01:25:17.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.964+0000 7fd93ff25700 1 -- 172.21.15.16:0/162611433 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9380b28b0 con 0x7fd93806ba90 2023-05-20T01:25:17.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.965+0000 7fd93ff25700 1 -- 172.21.15.16:0/162611433 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd93815a960 con 0x7fd93806ba90 2023-05-20T01:25:17.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.965+0000 7fd92e7fc700 1 -- 172.21.15.16:0/162611433 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd934003a20 con 0x7fd93806ba90 2023-05-20T01:25:17.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.965+0000 7fd92e7fc700 1 --2- 172.21.15.16:0/162611433 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd91c06ace0 0x7fd91c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:17.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.965+0000 7fd92e7fc700 1 -- 172.21.15.16:0/162611433 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fd934088a70 con 0x7fd93806ba90 2023-05-20T01:25:17.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.965+0000 7fd93dcc1700 1 --2- 172.21.15.16:0/162611433 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd91c06ace0 0x7fd91c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:17.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.966+0000 7fd93dcc1700 1 --2- 172.21.15.16:0/162611433 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd91c06ace0 0x7fd91c06d190 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fd928005990 tx=0x7fd928001b80 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:17.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:17.970+0000 7fd92e7fc700 1 -- 172.21.15.16:0/162611433 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fd934053a70 con 0x7fd93806ba90 2023-05-20T01:25:18.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:18.164+0000 7fd93ff25700 1 -- 172.21.15.16:0/162611433 --> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd9380cd390 con 0x7fd91c06ace0 2023-05-20T01:25:18.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:18.173+0000 7fd92e7fc700 1 -- 172.21.15.16:0/162611433 <== mgr.4104 v2:172.21.15.16:6832/3817237998 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311779 (secure 0 0 0) 0x7fd9380cd390 con 0x7fd91c06ace0 2023-05-20T01:25:18.181 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:25:18.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:18.176+0000 7fd93ff25700 1 -- 172.21.15.16:0/162611433 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd91c06ace0 msgr2=0x7fd91c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:18.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:18.176+0000 7fd93ff25700 1 --2- 172.21.15.16:0/162611433 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd91c06ace0 0x7fd91c06d190 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fd928005990 tx=0x7fd928001b80 comp rx=0 tx=0).stop 2023-05-20T01:25:18.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:18.176+0000 7fd93ff25700 1 -- 172.21.15.16:0/162611433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd93806ba90 msgr2=0x7fd9381e6e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:18.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:18.176+0000 7fd93ff25700 1 --2- 172.21.15.16:0/162611433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd93806ba90 0x7fd9381e6e90 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fd93400c500 tx=0x7fd93400c8c0 comp rx=0 tx=0).stop 2023-05-20T01:25:18.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:18.176+0000 7fd93ff25700 1 -- 172.21.15.16:0/162611433 shutdown_connections 2023-05-20T01:25:18.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:18.176+0000 7fd93ff25700 1 --2- 172.21.15.16:0/162611433 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd91c06ace0 0x7fd91c06d190 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:18.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:18.176+0000 7fd93ff25700 1 --2- 172.21.15.16:0/162611433 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd9380d1000 0x7fd9381e4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:18.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:18.176+0000 7fd93ff25700 1 --2- 172.21.15.16:0/162611433 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd93806b620 0x7fd93815a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:18.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:18.176+0000 7fd93ff25700 1 --2- 172.21.15.16:0/162611433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd93806ba90 0x7fd9381e6e90 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:18.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:18.176+0000 7fd93ff25700 1 -- 172.21.15.16:0/162611433 >> 172.21.15.16:0/162611433 conn(0x7fd93806ae30 msgr2=0x7fd93815c670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:18.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:18.176+0000 7fd93ff25700 1 -- 172.21.15.16:0/162611433 shutdown_connections 2023-05-20T01:25:18.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:18.176+0000 7fd93ff25700 1 -- 172.21.15.16:0/162611433 wait complete. 2023-05-20T01:25:18.186 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2023-05-20T01:25:18.193 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":350,"stamp":"2023-05-20T01:25:17.434657+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":160,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":96672,"kb_used_data":9632,"kb_used_omap":0,"kb_used_meta":86976,"kb_avail":754878048,"statfs":{"total":773094113280,"available":772995121152,"internally_reserved":0,"allocated":9863168,"data_stored":5535562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":89063424},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001143"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:13.059738+0000","last_change":"2023-05-20T01:25:12.010861+0000","last_active":"2023-05-20T01:25:13.059738+0000","last_peered":"2023-05-20T01:25:13.059738+0000","last_clean":"2023-05-20T01:25:13.059738+0000","last_became_active":"2023-05-20T01:25:12.010462+0000","last_became_peered":"2023-05-20T01:25:12.010462+0000","last_unstale":"2023-05-20T01:25:13.059738+0000","last_undegraded":"2023-05-20T01:25:13.059738+0000","last_fullsized":"2023-05-20T01:25:13.059738+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:47:16.902590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:14.300358+0000","last_change":"2023-05-20T01:25:12.015916+0000","last_active":"2023-05-20T01:25:14.300358+0000","last_peered":"2023-05-20T01:25:14.300358+0000","last_clean":"2023-05-20T01:25:14.300358+0000","last_became_active":"2023-05-20T01:25:12.012714+0000","last_became_peered":"2023-05-20T01:25:12.012714+0000","last_unstale":"2023-05-20T01:25:14.300358+0000","last_undegraded":"2023-05-20T01:25:14.300358+0000","last_fullsized":"2023-05-20T01:25:14.300358+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:21:39.218741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:13.572745+0000","last_change":"2023-05-20T01:25:12.012942+0000","last_active":"2023-05-20T01:25:13.572745+0000","last_peered":"2023-05-20T01:25:13.572745+0000","last_clean":"2023-05-20T01:25:13.572745+0000","last_became_active":"2023-05-20T01:25:12.012644+0000","last_became_peered":"2023-05-20T01:25:12.012644+0000","last_unstale":"2023-05-20T01:25:13.572745+0000","last_undegraded":"2023-05-20T01:25:13.572745+0000","last_fullsized":"2023-05-20T01:25:13.572745+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:34:19.045913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:12.997404+0000","last_change":"2023-05-20T01:25:12.022797+0000","last_active":"2023-05-20T01:25:12.997404+0000","last_peered":"2023-05-20T01:25:12.997404+0000","last_clean":"2023-05-20T01:25:12.997404+0000","last_became_active":"2023-05-20T01:25:12.022426+0000","last_became_peered":"2023-05-20T01:25:12.022426+0000","last_unstale":"2023-05-20T01:25:12.997404+0000","last_undegraded":"2023-05-20T01:25:12.997404+0000","last_fullsized":"2023-05-20T01:25:12.997404+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:48:31.993109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:12.997603+0000","last_change":"2023-05-20T01:25:12.015355+0000","last_active":"2023-05-20T01:25:12.997603+0000","last_peered":"2023-05-20T01:25:12.997603+0000","last_clean":"2023-05-20T01:25:12.997603+0000","last_became_active":"2023-05-20T01:25:12.015075+0000","last_became_peered":"2023-05-20T01:25:12.015075+0000","last_unstale":"2023-05-20T01:25:12.997603+0000","last_undegraded":"2023-05-20T01:25:12.997603+0000","last_fullsized":"2023-05-20T01:25:12.997603+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:51:13.117657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:13.804021+0000","last_change":"2023-05-20T01:25:12.022136+0000","last_active":"2023-05-20T01:25:13.804021+0000","last_peered":"2023-05-20T01:25:13.804021+0000","last_clean":"2023-05-20T01:25:13.804021+0000","last_became_active":"2023-05-20T01:25:12.021786+0000","last_became_peered":"2023-05-20T01:25:12.021786+0000","last_unstale":"2023-05-20T01:25:13.804021+0000","last_undegraded":"2023-05-20T01:25:13.804021+0000","last_fullsized":"2023-05-20T01:25:13.804021+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:32:22.089588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:12.996987+0000","last_change":"2023-05-20T01:25:12.007895+0000","last_active":"2023-05-20T01:25:12.996987+0000","last_peered":"2023-05-20T01:25:12.996987+0000","last_clean":"2023-05-20T01:25:12.996987+0000","last_became_active":"2023-05-20T01:25:12.007519+0000","last_became_peered":"2023-05-20T01:25:12.007519+0000","last_unstale":"2023-05-20T01:25:12.996987+0000","last_undegraded":"2023-05-20T01:25:12.996987+0000","last_fullsized":"2023-05-20T01:25:12.996987+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:24:52.476098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:13.803950+0000","last_change":"2023-05-20T01:25:12.016456+0000","last_active":"2023-05-20T01:25:13.803950+0000","last_peered":"2023-05-20T01:25:13.803950+0000","last_clean":"2023-05-20T01:25:13.803950+0000","last_became_active":"2023-05-20T01:25:12.016080+0000","last_became_peered":"2023-05-20T01:25:12.016080+0000","last_unstale":"2023-05-20T01:25:13.803950+0000","last_undegraded":"2023-05-20T01:25:13.803950+0000","last_fullsized":"2023-05-20T01:25:13.803950+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T01:54:04.144226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:13.804161+0000","last_change":"2023-05-20T01:25:12.016224+0000","last_active":"2023-05-20T01:25:13.804161+0000","last_peered":"2023-05-20T01:25:13.804161+0000","last_clean":"2023-05-20T01:25:13.804161+0000","last_became_active":"2023-05-20T01:25:12.015944+0000","last_became_peered":"2023-05-20T01:25:12.015944+0000","last_unstale":"2023-05-20T01:25:13.804161+0000","last_undegraded":"2023-05-20T01:25:13.804161+0000","last_fullsized":"2023-05-20T01:25:13.804161+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:32:29.054378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:13.059787+0000","last_change":"2023-05-20T01:25:12.006466+0000","last_active":"2023-05-20T01:25:13.059787+0000","last_peered":"2023-05-20T01:25:13.059787+0000","last_clean":"2023-05-20T01:25:13.059787+0000","last_became_active":"2023-05-20T01:25:12.006231+0000","last_became_peered":"2023-05-20T01:25:12.006231+0000","last_unstale":"2023-05-20T01:25:13.059787+0000","last_undegraded":"2023-05-20T01:25:13.059787+0000","last_fullsized":"2023-05-20T01:25:13.059787+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:46:57.605709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:14.300592+0000","last_change":"2023-05-20T01:25:12.013977+0000","last_active":"2023-05-20T01:25:14.300592+0000","last_peered":"2023-05-20T01:25:14.300592+0000","last_clean":"2023-05-20T01:25:14.300592+0000","last_became_active":"2023-05-20T01:25:12.011695+0000","last_became_peered":"2023-05-20T01:25:12.011695+0000","last_unstale":"2023-05-20T01:25:14.300592+0000","last_undegraded":"2023-05-20T01:25:14.300592+0000","last_fullsized":"2023-05-20T01:25:14.300592+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:50:31.388498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:13.060161+0000","last_change":"2023-05-20T01:25:12.007183+0000","last_active":"2023-05-20T01:25:13.060161+0000","last_peered":"2023-05-20T01:25:13.060161+0000","last_clean":"2023-05-20T01:25:13.060161+0000","last_became_active":"2023-05-20T01:25:12.006930+0000","last_became_peered":"2023-05-20T01:25:12.006930+0000","last_unstale":"2023-05-20T01:25:13.060161+0000","last_undegraded":"2023-05-20T01:25:13.060161+0000","last_fullsized":"2023-05-20T01:25:13.060161+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:46:12.148885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:13.572651+0000","last_change":"2023-05-20T01:25:12.002684+0000","last_active":"2023-05-20T01:25:13.572651+0000","last_peered":"2023-05-20T01:25:13.572651+0000","last_clean":"2023-05-20T01:25:13.572651+0000","last_became_active":"2023-05-20T01:25:12.002422+0000","last_became_peered":"2023-05-20T01:25:12.002422+0000","last_unstale":"2023-05-20T01:25:13.572651+0000","last_undegraded":"2023-05-20T01:25:13.572651+0000","last_fullsized":"2023-05-20T01:25:13.572651+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:58:00.068009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:12.997242+0000","last_change":"2023-05-20T01:25:12.017371+0000","last_active":"2023-05-20T01:25:12.997242+0000","last_peered":"2023-05-20T01:25:12.997242+0000","last_clean":"2023-05-20T01:25:12.997242+0000","last_became_active":"2023-05-20T01:25:12.017036+0000","last_became_peered":"2023-05-20T01:25:12.017036+0000","last_unstale":"2023-05-20T01:25:12.997242+0000","last_undegraded":"2023-05-20T01:25:12.997242+0000","last_fullsized":"2023-05-20T01:25:12.997242+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:31:18.466752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:12.995661+0000","last_change":"2023-05-20T01:25:11.996397+0000","last_active":"2023-05-20T01:25:12.995661+0000","last_peered":"2023-05-20T01:25:12.995661+0000","last_clean":"2023-05-20T01:25:12.995661+0000","last_became_active":"2023-05-20T01:25:11.996130+0000","last_became_peered":"2023-05-20T01:25:11.996130+0000","last_unstale":"2023-05-20T01:25:12.995661+0000","last_undegraded":"2023-05-20T01:25:12.995661+0000","last_fullsized":"2023-05-20T01:25:12.995661+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:36:05.566447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:12.997081+0000","last_change":"2023-05-20T01:25:12.007726+0000","last_active":"2023-05-20T01:25:12.997081+0000","last_peered":"2023-05-20T01:25:12.997081+0000","last_clean":"2023-05-20T01:25:12.997081+0000","last_became_active":"2023-05-20T01:25:12.007310+0000","last_became_peered":"2023-05-20T01:25:12.007310+0000","last_unstale":"2023-05-20T01:25:12.997081+0000","last_undegraded":"2023-05-20T01:25:12.997081+0000","last_fullsized":"2023-05-20T01:25:12.997081+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:06:16.779810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:12.997668+0000","last_change":"2023-05-20T01:25:12.008442+0000","last_active":"2023-05-20T01:25:12.997668+0000","last_peered":"2023-05-20T01:25:12.997668+0000","last_clean":"2023-05-20T01:25:12.997668+0000","last_became_active":"2023-05-20T01:25:12.008160+0000","last_became_peered":"2023-05-20T01:25:12.008160+0000","last_unstale":"2023-05-20T01:25:12.997668+0000","last_undegraded":"2023-05-20T01:25:12.997668+0000","last_fullsized":"2023-05-20T01:25:12.997668+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:33:35.277012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:12.997210+0000","last_change":"2023-05-20T01:25:12.014908+0000","last_active":"2023-05-20T01:25:12.997210+0000","last_peered":"2023-05-20T01:25:12.997210+0000","last_clean":"2023-05-20T01:25:12.997210+0000","last_became_active":"2023-05-20T01:25:12.014637+0000","last_became_peered":"2023-05-20T01:25:12.014637+0000","last_unstale":"2023-05-20T01:25:12.997210+0000","last_undegraded":"2023-05-20T01:25:12.997210+0000","last_fullsized":"2023-05-20T01:25:12.997210+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:06:03.483921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:13.174222+0000","last_change":"2023-05-20T01:25:12.012914+0000","last_active":"2023-05-20T01:25:13.174222+0000","last_peered":"2023-05-20T01:25:13.174222+0000","last_clean":"2023-05-20T01:25:13.174222+0000","last_became_active":"2023-05-20T01:25:12.012607+0000","last_became_peered":"2023-05-20T01:25:12.012607+0000","last_unstale":"2023-05-20T01:25:13.174222+0000","last_undegraded":"2023-05-20T01:25:13.174222+0000","last_fullsized":"2023-05-20T01:25:13.174222+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:48:48.891631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:12.997616+0000","last_change":"2023-05-20T01:25:12.016923+0000","last_active":"2023-05-20T01:25:12.997616+0000","last_peered":"2023-05-20T01:25:12.997616+0000","last_clean":"2023-05-20T01:25:12.997616+0000","last_became_active":"2023-05-20T01:25:12.016657+0000","last_became_peered":"2023-05-20T01:25:12.016657+0000","last_unstale":"2023-05-20T01:25:12.997616+0000","last_undegraded":"2023-05-20T01:25:12.997616+0000","last_fullsized":"2023-05-20T01:25:12.997616+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T13:17:09.147871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:12.997303+0000","last_change":"2023-05-20T01:25:12.016842+0000","last_active":"2023-05-20T01:25:12.997303+0000","last_peered":"2023-05-20T01:25:12.997303+0000","last_clean":"2023-05-20T01:25:12.997303+0000","last_became_active":"2023-05-20T01:25:12.016589+0000","last_became_peered":"2023-05-20T01:25:12.016589+0000","last_unstale":"2023-05-20T01:25:12.997303+0000","last_undegraded":"2023-05-20T01:25:12.997303+0000","last_fullsized":"2023-05-20T01:25:12.997303+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:05:16.230863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:14.300444+0000","last_change":"2023-05-20T01:25:12.021437+0000","last_active":"2023-05-20T01:25:14.300444+0000","last_peered":"2023-05-20T01:25:14.300444+0000","last_clean":"2023-05-20T01:25:14.300444+0000","last_became_active":"2023-05-20T01:25:12.021161+0000","last_became_peered":"2023-05-20T01:25:12.021161+0000","last_unstale":"2023-05-20T01:25:14.300444+0000","last_undegraded":"2023-05-20T01:25:14.300444+0000","last_fullsized":"2023-05-20T01:25:14.300444+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:51:41.067097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:13.059427+0000","last_change":"2023-05-20T01:25:12.014864+0000","last_active":"2023-05-20T01:25:13.059427+0000","last_peered":"2023-05-20T01:25:13.059427+0000","last_clean":"2023-05-20T01:25:13.059427+0000","last_became_active":"2023-05-20T01:25:12.014584+0000","last_became_peered":"2023-05-20T01:25:12.014584+0000","last_unstale":"2023-05-20T01:25:13.059427+0000","last_undegraded":"2023-05-20T01:25:13.059427+0000","last_fullsized":"2023-05-20T01:25:13.059427+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:56:41.729759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:12.996931+0000","last_change":"2023-05-20T01:25:12.019279+0000","last_active":"2023-05-20T01:25:12.996931+0000","last_peered":"2023-05-20T01:25:12.996931+0000","last_clean":"2023-05-20T01:25:12.996931+0000","last_became_active":"2023-05-20T01:25:12.018988+0000","last_became_peered":"2023-05-20T01:25:12.018988+0000","last_unstale":"2023-05-20T01:25:12.996931+0000","last_undegraded":"2023-05-20T01:25:12.996931+0000","last_fullsized":"2023-05-20T01:25:12.996931+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:04:33.650813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:14.300566+0000","last_change":"2023-05-20T01:25:12.015996+0000","last_active":"2023-05-20T01:25:14.300566+0000","last_peered":"2023-05-20T01:25:14.300566+0000","last_clean":"2023-05-20T01:25:14.300566+0000","last_became_active":"2023-05-20T01:25:12.015783+0000","last_became_peered":"2023-05-20T01:25:12.015783+0000","last_unstale":"2023-05-20T01:25:14.300566+0000","last_undegraded":"2023-05-20T01:25:14.300566+0000","last_fullsized":"2023-05-20T01:25:14.300566+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T01:28:37.267333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:13.573024+0000","last_change":"2023-05-20T01:25:12.016956+0000","last_active":"2023-05-20T01:25:13.573024+0000","last_peered":"2023-05-20T01:25:13.573024+0000","last_clean":"2023-05-20T01:25:13.573024+0000","last_became_active":"2023-05-20T01:25:12.016686+0000","last_became_peered":"2023-05-20T01:25:12.016686+0000","last_unstale":"2023-05-20T01:25:13.573024+0000","last_undegraded":"2023-05-20T01:25:13.573024+0000","last_fullsized":"2023-05-20T01:25:13.573024+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:23:02.561469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:12.997309+0000","last_change":"2023-05-20T01:25:12.002001+0000","last_active":"2023-05-20T01:25:12.997309+0000","last_peered":"2023-05-20T01:25:12.997309+0000","last_clean":"2023-05-20T01:25:12.997309+0000","last_became_active":"2023-05-20T01:25:12.001712+0000","last_became_peered":"2023-05-20T01:25:12.001712+0000","last_unstale":"2023-05-20T01:25:12.997309+0000","last_undegraded":"2023-05-20T01:25:12.997309+0000","last_fullsized":"2023-05-20T01:25:12.997309+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:04:26.356222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:13.059504+0000","last_change":"2023-05-20T01:25:12.010877+0000","last_active":"2023-05-20T01:25:13.059504+0000","last_peered":"2023-05-20T01:25:13.059504+0000","last_clean":"2023-05-20T01:25:13.059504+0000","last_became_active":"2023-05-20T01:25:12.010604+0000","last_became_peered":"2023-05-20T01:25:12.010604+0000","last_unstale":"2023-05-20T01:25:13.059504+0000","last_undegraded":"2023-05-20T01:25:13.059504+0000","last_fullsized":"2023-05-20T01:25:13.059504+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:02:40.263778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:14.298066+0000","last_change":"2023-05-20T01:25:12.012184+0000","last_active":"2023-05-20T01:25:14.298066+0000","last_peered":"2023-05-20T01:25:14.298066+0000","last_clean":"2023-05-20T01:25:14.298066+0000","last_became_active":"2023-05-20T01:25:12.011853+0000","last_became_peered":"2023-05-20T01:25:12.011853+0000","last_unstale":"2023-05-20T01:25:14.298066+0000","last_undegraded":"2023-05-20T01:25:14.298066+0000","last_fullsized":"2023-05-20T01:25:14.298066+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:15:58.796775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:14.300492+0000","last_change":"2023-05-20T01:25:12.012794+0000","last_active":"2023-05-20T01:25:14.300492+0000","last_peered":"2023-05-20T01:25:14.300492+0000","last_clean":"2023-05-20T01:25:14.300492+0000","last_became_active":"2023-05-20T01:25:12.011794+0000","last_became_peered":"2023-05-20T01:25:12.011794+0000","last_unstale":"2023-05-20T01:25:14.300492+0000","last_undegraded":"2023-05-20T01:25:14.300492+0000","last_fullsized":"2023-05-20T01:25:14.300492+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:57:36.203630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:12.997104+0000","last_change":"2023-05-20T01:25:12.013474+0000","last_active":"2023-05-20T01:25:12.997104+0000","last_peered":"2023-05-20T01:25:12.997104+0000","last_clean":"2023-05-20T01:25:12.997104+0000","last_became_active":"2023-05-20T01:25:12.013220+0000","last_became_peered":"2023-05-20T01:25:12.013220+0000","last_unstale":"2023-05-20T01:25:12.997104+0000","last_undegraded":"2023-05-20T01:25:12.997104+0000","last_fullsized":"2023-05-20T01:25:12.997104+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:23:43.680789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:13.059396+0000","last_change":"2023-05-20T01:25:12.016253+0000","last_active":"2023-05-20T01:25:13.059396+0000","last_peered":"2023-05-20T01:25:13.059396+0000","last_clean":"2023-05-20T01:25:13.059396+0000","last_became_active":"2023-05-20T01:25:12.015995+0000","last_became_peered":"2023-05-20T01:25:12.015995+0000","last_unstale":"2023-05-20T01:25:13.059396+0000","last_undegraded":"2023-05-20T01:25:13.059396+0000","last_fullsized":"2023-05-20T01:25:13.059396+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:44:53.263956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:12.997462+0000","last_change":"2023-05-20T01:25:12.007395+0000","last_active":"2023-05-20T01:25:12.997462+0000","last_peered":"2023-05-20T01:25:12.997462+0000","last_clean":"2023-05-20T01:25:12.997462+0000","last_became_active":"2023-05-20T01:25:12.007124+0000","last_became_peered":"2023-05-20T01:25:12.007124+0000","last_unstale":"2023-05-20T01:25:12.997462+0000","last_undegraded":"2023-05-20T01:25:12.997462+0000","last_fullsized":"2023-05-20T01:25:12.997462+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:16:00.645250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"87'4","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:13.060070+0000","last_change":"2023-05-20T01:25:11.580921+0000","last_active":"2023-05-20T01:25:13.060070+0000","last_peered":"2023-05-20T01:25:13.060070+0000","last_clean":"2023-05-20T01:25:13.060070+0000","last_became_active":"2023-05-20T01:25:11.580640+0000","last_became_peered":"2023-05-20T01:25:11.580640+0000","last_unstale":"2023-05-20T01:25:13.060070+0000","last_undegraded":"2023-05-20T01:25:13.060070+0000","last_fullsized":"2023-05-20T01:25:13.060070+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:48:47.884748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:13.060202+0000","last_change":"2023-05-20T01:25:12.012998+0000","last_active":"2023-05-20T01:25:13.060202+0000","last_peered":"2023-05-20T01:25:13.060202+0000","last_clean":"2023-05-20T01:25:13.060202+0000","last_became_active":"2023-05-20T01:25:12.012623+0000","last_became_peered":"2023-05-20T01:25:12.012623+0000","last_unstale":"2023-05-20T01:25:13.060202+0000","last_undegraded":"2023-05-20T01:25:13.060202+0000","last_fullsized":"2023-05-20T01:25:13.060202+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:57:45.442959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"87'1","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:12.997642+0000","last_change":"2023-05-20T01:25:11.569727+0000","last_active":"2023-05-20T01:25:12.997642+0000","last_peered":"2023-05-20T01:25:12.997642+0000","last_clean":"2023-05-20T01:25:12.997642+0000","last_became_active":"2023-05-20T01:25:11.569385+0000","last_became_peered":"2023-05-20T01:25:11.569385+0000","last_unstale":"2023-05-20T01:25:12.997642+0000","last_undegraded":"2023-05-20T01:25:12.997642+0000","last_fullsized":"2023-05-20T01:25:12.997642+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:45:07.423138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:13.059616+0000","last_change":"2023-05-20T01:25:12.021990+0000","last_active":"2023-05-20T01:25:13.059616+0000","last_peered":"2023-05-20T01:25:13.059616+0000","last_clean":"2023-05-20T01:25:13.059616+0000","last_became_active":"2023-05-20T01:25:12.021702+0000","last_became_peered":"2023-05-20T01:25:12.021702+0000","last_unstale":"2023-05-20T01:25:13.059616+0000","last_undegraded":"2023-05-20T01:25:13.059616+0000","last_fullsized":"2023-05-20T01:25:13.059616+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:13:45.855545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"87'2","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:13.174463+0000","last_change":"2023-05-20T01:25:11.580723+0000","last_active":"2023-05-20T01:25:13.174463+0000","last_peered":"2023-05-20T01:25:13.174463+0000","last_clean":"2023-05-20T01:25:13.174463+0000","last_became_active":"2023-05-20T01:25:11.580402+0000","last_became_peered":"2023-05-20T01:25:11.580402+0000","last_unstale":"2023-05-20T01:25:13.174463+0000","last_undegraded":"2023-05-20T01:25:13.174463+0000","last_fullsized":"2023-05-20T01:25:13.174463+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:46:11.142002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:13.572834+0000","last_change":"2023-05-20T01:25:12.008759+0000","last_active":"2023-05-20T01:25:13.572834+0000","last_peered":"2023-05-20T01:25:13.572834+0000","last_clean":"2023-05-20T01:25:13.572834+0000","last_became_active":"2023-05-20T01:25:12.008430+0000","last_became_peered":"2023-05-20T01:25:12.008430+0000","last_unstale":"2023-05-20T01:25:13.572834+0000","last_undegraded":"2023-05-20T01:25:13.572834+0000","last_fullsized":"2023-05-20T01:25:13.572834+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:34:03.861279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"87'2","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:13.174303+0000","last_change":"2023-05-20T01:25:11.588731+0000","last_active":"2023-05-20T01:25:13.174303+0000","last_peered":"2023-05-20T01:25:13.174303+0000","last_clean":"2023-05-20T01:25:13.174303+0000","last_became_active":"2023-05-20T01:25:11.586813+0000","last_became_peered":"2023-05-20T01:25:11.586813+0000","last_unstale":"2023-05-20T01:25:13.174303+0000","last_undegraded":"2023-05-20T01:25:13.174303+0000","last_fullsized":"2023-05-20T01:25:13.174303+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:30:17.004521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:14.298060+0000","last_change":"2023-05-20T01:25:12.012381+0000","last_active":"2023-05-20T01:25:14.298060+0000","last_peered":"2023-05-20T01:25:14.298060+0000","last_clean":"2023-05-20T01:25:14.298060+0000","last_became_active":"2023-05-20T01:25:12.012131+0000","last_became_peered":"2023-05-20T01:25:12.012131+0000","last_unstale":"2023-05-20T01:25:14.298060+0000","last_undegraded":"2023-05-20T01:25:14.298060+0000","last_fullsized":"2023-05-20T01:25:14.298060+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:33:06.332534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"87'1","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:12.997690+0000","last_change":"2023-05-20T01:25:11.584464+0000","last_active":"2023-05-20T01:25:12.997690+0000","last_peered":"2023-05-20T01:25:12.997690+0000","last_clean":"2023-05-20T01:25:12.997690+0000","last_became_active":"2023-05-20T01:25:11.584196+0000","last_became_peered":"2023-05-20T01:25:11.584196+0000","last_unstale":"2023-05-20T01:25:12.997690+0000","last_undegraded":"2023-05-20T01:25:12.997690+0000","last_fullsized":"2023-05-20T01:25:12.997690+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:54:24.464550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:12.997535+0000","last_change":"2023-05-20T01:25:12.008538+0000","last_active":"2023-05-20T01:25:12.997535+0000","last_peered":"2023-05-20T01:25:12.997535+0000","last_clean":"2023-05-20T01:25:12.997535+0000","last_became_active":"2023-05-20T01:25:12.008253+0000","last_became_peered":"2023-05-20T01:25:12.008253+0000","last_unstale":"2023-05-20T01:25:12.997535+0000","last_undegraded":"2023-05-20T01:25:12.997535+0000","last_fullsized":"2023-05-20T01:25:12.997535+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:29:22.170776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"87'1","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:12.997125+0000","last_change":"2023-05-20T01:25:11.576719+0000","last_active":"2023-05-20T01:25:12.997125+0000","last_peered":"2023-05-20T01:25:12.997125+0000","last_clean":"2023-05-20T01:25:12.997125+0000","last_became_active":"2023-05-20T01:25:11.576440+0000","last_became_peered":"2023-05-20T01:25:11.576440+0000","last_unstale":"2023-05-20T01:25:12.997125+0000","last_undegraded":"2023-05-20T01:25:12.997125+0000","last_fullsized":"2023-05-20T01:25:12.997125+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:45:01.382666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:13.572664+0000","last_change":"2023-05-20T01:25:12.018685+0000","last_active":"2023-05-20T01:25:13.572664+0000","last_peered":"2023-05-20T01:25:13.572664+0000","last_clean":"2023-05-20T01:25:13.572664+0000","last_became_active":"2023-05-20T01:25:12.018438+0000","last_became_peered":"2023-05-20T01:25:12.018438+0000","last_unstale":"2023-05-20T01:25:13.572664+0000","last_undegraded":"2023-05-20T01:25:13.572664+0000","last_fullsized":"2023-05-20T01:25:13.572664+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:33:35.277012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:13.173987+0000","last_change":"2023-05-20T01:25:11.581486+0000","last_active":"2023-05-20T01:25:13.173987+0000","last_peered":"2023-05-20T01:25:13.173987+0000","last_clean":"2023-05-20T01:25:13.173987+0000","last_became_active":"2023-05-20T01:25:11.581156+0000","last_became_peered":"2023-05-20T01:25:11.581156+0000","last_unstale":"2023-05-20T01:25:13.173987+0000","last_undegraded":"2023-05-20T01:25:13.173987+0000","last_fullsized":"2023-05-20T01:25:13.173987+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:35:33.948338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:12.995799+0000","last_change":"2023-05-20T01:25:11.996411+0000","last_active":"2023-05-20T01:25:12.995799+0000","last_peered":"2023-05-20T01:25:12.995799+0000","last_clean":"2023-05-20T01:25:12.995799+0000","last_became_active":"2023-05-20T01:25:11.996143+0000","last_became_peered":"2023-05-20T01:25:11.996143+0000","last_unstale":"2023-05-20T01:25:12.995799+0000","last_undegraded":"2023-05-20T01:25:12.995799+0000","last_fullsized":"2023-05-20T01:25:12.995799+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:00:32.352548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:14.297960+0000","last_change":"2023-05-20T01:25:11.581997+0000","last_active":"2023-05-20T01:25:14.297960+0000","last_peered":"2023-05-20T01:25:14.297960+0000","last_clean":"2023-05-20T01:25:14.297960+0000","last_became_active":"2023-05-20T01:25:11.581698+0000","last_became_peered":"2023-05-20T01:25:11.581698+0000","last_unstale":"2023-05-20T01:25:14.297960+0000","last_undegraded":"2023-05-20T01:25:14.297960+0000","last_fullsized":"2023-05-20T01:25:14.297960+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:05:19.570386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:12.996105+0000","last_change":"2023-05-20T01:25:12.013481+0000","last_active":"2023-05-20T01:25:12.996105+0000","last_peered":"2023-05-20T01:25:12.996105+0000","last_clean":"2023-05-20T01:25:12.996105+0000","last_became_active":"2023-05-20T01:25:12.013165+0000","last_became_peered":"2023-05-20T01:25:12.013165+0000","last_unstale":"2023-05-20T01:25:12.996105+0000","last_undegraded":"2023-05-20T01:25:12.996105+0000","last_fullsized":"2023-05-20T01:25:12.996105+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:03:38.637833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:13.059918+0000","last_change":"2023-05-20T01:25:11.577858+0000","last_active":"2023-05-20T01:25:13.059918+0000","last_peered":"2023-05-20T01:25:13.059918+0000","last_clean":"2023-05-20T01:25:13.059918+0000","last_became_active":"2023-05-20T01:25:11.577471+0000","last_became_peered":"2023-05-20T01:25:11.577471+0000","last_unstale":"2023-05-20T01:25:13.059918+0000","last_undegraded":"2023-05-20T01:25:13.059918+0000","last_fullsized":"2023-05-20T01:25:13.059918+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:53:16.752429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:13.804138+0000","last_change":"2023-05-20T01:25:12.022103+0000","last_active":"2023-05-20T01:25:13.804138+0000","last_peered":"2023-05-20T01:25:13.804138+0000","last_clean":"2023-05-20T01:25:13.804138+0000","last_became_active":"2023-05-20T01:25:12.021766+0000","last_became_peered":"2023-05-20T01:25:12.021766+0000","last_unstale":"2023-05-20T01:25:13.804138+0000","last_undegraded":"2023-05-20T01:25:13.804138+0000","last_fullsized":"2023-05-20T01:25:13.804138+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:12:14.338149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:13.059980+0000","last_change":"2023-05-20T01:25:11.580330+0000","last_active":"2023-05-20T01:25:13.059980+0000","last_peered":"2023-05-20T01:25:13.059980+0000","last_clean":"2023-05-20T01:25:13.059980+0000","last_became_active":"2023-05-20T01:25:11.580062+0000","last_became_peered":"2023-05-20T01:25:11.580062+0000","last_unstale":"2023-05-20T01:25:13.059980+0000","last_undegraded":"2023-05-20T01:25:13.059980+0000","last_fullsized":"2023-05-20T01:25:13.059980+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:23:31.146768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:13.174216+0000","last_change":"2023-05-20T01:25:12.014329+0000","last_active":"2023-05-20T01:25:13.174216+0000","last_peered":"2023-05-20T01:25:13.174216+0000","last_clean":"2023-05-20T01:25:13.174216+0000","last_became_active":"2023-05-20T01:25:12.014014+0000","last_became_peered":"2023-05-20T01:25:12.014014+0000","last_unstale":"2023-05-20T01:25:13.174216+0000","last_undegraded":"2023-05-20T01:25:13.174216+0000","last_fullsized":"2023-05-20T01:25:13.174216+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:23:32.153651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"87'3","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:13.059910+0000","last_change":"2023-05-20T01:25:11.584259+0000","last_active":"2023-05-20T01:25:13.059910+0000","last_peered":"2023-05-20T01:25:13.059910+0000","last_clean":"2023-05-20T01:25:13.059910+0000","last_became_active":"2023-05-20T01:25:11.583981+0000","last_became_peered":"2023-05-20T01:25:11.583981+0000","last_unstale":"2023-05-20T01:25:13.059910+0000","last_undegraded":"2023-05-20T01:25:13.059910+0000","last_fullsized":"2023-05-20T01:25:13.059910+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T13:09:54.839617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:12.997368+0000","last_change":"2023-05-20T01:25:12.013250+0000","last_active":"2023-05-20T01:25:12.997368+0000","last_peered":"2023-05-20T01:25:12.997368+0000","last_clean":"2023-05-20T01:25:12.997368+0000","last_became_active":"2023-05-20T01:25:12.012905+0000","last_became_peered":"2023-05-20T01:25:12.012905+0000","last_unstale":"2023-05-20T01:25:12.997368+0000","last_undegraded":"2023-05-20T01:25:12.997368+0000","last_fullsized":"2023-05-20T01:25:12.997368+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:30:46.395594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"87'6","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:13.804199+0000","last_change":"2023-05-20T01:25:11.579260+0000","last_active":"2023-05-20T01:25:13.804199+0000","last_peered":"2023-05-20T01:25:13.804199+0000","last_clean":"2023-05-20T01:25:13.804199+0000","last_became_active":"2023-05-20T01:25:11.578988+0000","last_became_peered":"2023-05-20T01:25:11.578988+0000","last_unstale":"2023-05-20T01:25:13.804199+0000","last_undegraded":"2023-05-20T01:25:13.804199+0000","last_fullsized":"2023-05-20T01:25:13.804199+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:11:19.510387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:13.174017+0000","last_change":"2023-05-20T01:25:12.021021+0000","last_active":"2023-05-20T01:25:13.174017+0000","last_peered":"2023-05-20T01:25:13.174017+0000","last_clean":"2023-05-20T01:25:13.174017+0000","last_became_active":"2023-05-20T01:25:12.020666+0000","last_became_peered":"2023-05-20T01:25:12.020666+0000","last_unstale":"2023-05-20T01:25:13.174017+0000","last_undegraded":"2023-05-20T01:25:13.174017+0000","last_fullsized":"2023-05-20T01:25:13.174017+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T13:09:55.846499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"87'2","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:14.298078+0000","last_change":"2023-05-20T01:25:11.585058+0000","last_active":"2023-05-20T01:25:14.298078+0000","last_peered":"2023-05-20T01:25:14.298078+0000","last_clean":"2023-05-20T01:25:14.298078+0000","last_became_active":"2023-05-20T01:25:11.584753+0000","last_became_peered":"2023-05-20T01:25:11.584753+0000","last_unstale":"2023-05-20T01:25:14.298078+0000","last_undegraded":"2023-05-20T01:25:14.298078+0000","last_fullsized":"2023-05-20T01:25:14.298078+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:58:27.539898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:13.572692+0000","last_change":"2023-05-20T01:25:12.016142+0000","last_active":"2023-05-20T01:25:13.572692+0000","last_peered":"2023-05-20T01:25:13.572692+0000","last_clean":"2023-05-20T01:25:13.572692+0000","last_became_active":"2023-05-20T01:25:12.015860+0000","last_became_peered":"2023-05-20T01:25:12.015860+0000","last_unstale":"2023-05-20T01:25:13.572692+0000","last_undegraded":"2023-05-20T01:25:13.572692+0000","last_fullsized":"2023-05-20T01:25:13.572692+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:31:51.281533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:12.995870+0000","last_change":"2023-05-20T01:25:11.585536+0000","last_active":"2023-05-20T01:25:12.995870+0000","last_peered":"2023-05-20T01:25:12.995870+0000","last_clean":"2023-05-20T01:25:12.995870+0000","last_became_active":"2023-05-20T01:25:11.585257+0000","last_became_peered":"2023-05-20T01:25:11.585257+0000","last_unstale":"2023-05-20T01:25:12.995870+0000","last_undegraded":"2023-05-20T01:25:12.995870+0000","last_fullsized":"2023-05-20T01:25:12.995870+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:11:13.264714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:14.297964+0000","last_change":"2023-05-20T01:25:12.012341+0000","last_active":"2023-05-20T01:25:14.297964+0000","last_peered":"2023-05-20T01:25:14.297964+0000","last_clean":"2023-05-20T01:25:14.297964+0000","last_became_active":"2023-05-20T01:25:12.012069+0000","last_became_peered":"2023-05-20T01:25:12.012069+0000","last_unstale":"2023-05-20T01:25:14.297964+0000","last_undegraded":"2023-05-20T01:25:14.297964+0000","last_fullsized":"2023-05-20T01:25:14.297964+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:28:31.900808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:13.803900+0000","last_change":"2023-05-20T01:25:11.580046+0000","last_active":"2023-05-20T01:25:13.803900+0000","last_peered":"2023-05-20T01:25:13.803900+0000","last_clean":"2023-05-20T01:25:13.803900+0000","last_became_active":"2023-05-20T01:25:11.579830+0000","last_became_peered":"2023-05-20T01:25:11.579830+0000","last_unstale":"2023-05-20T01:25:13.803900+0000","last_undegraded":"2023-05-20T01:25:13.803900+0000","last_fullsized":"2023-05-20T01:25:13.803900+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:26:47.257707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:14.297873+0000","last_change":"2023-05-20T01:25:12.008915+0000","last_active":"2023-05-20T01:25:14.297873+0000","last_peered":"2023-05-20T01:25:14.297873+0000","last_clean":"2023-05-20T01:25:14.297873+0000","last_became_active":"2023-05-20T01:25:12.008720+0000","last_became_peered":"2023-05-20T01:25:12.008720+0000","last_unstale":"2023-05-20T01:25:14.297873+0000","last_undegraded":"2023-05-20T01:25:14.297873+0000","last_fullsized":"2023-05-20T01:25:14.297873+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:10:37.482767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"87'8","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:13.173906+0000","last_change":"2023-05-20T01:25:11.581483+0000","last_active":"2023-05-20T01:25:13.173906+0000","last_peered":"2023-05-20T01:25:13.173906+0000","last_clean":"2023-05-20T01:25:13.173906+0000","last_became_active":"2023-05-20T01:25:11.581189+0000","last_became_peered":"2023-05-20T01:25:11.581189+0000","last_unstale":"2023-05-20T01:25:13.173906+0000","last_undegraded":"2023-05-20T01:25:13.173906+0000","last_fullsized":"2023-05-20T01:25:13.173906+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:46:56.598827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:12.997562+0000","last_change":"2023-05-20T01:25:12.009760+0000","last_active":"2023-05-20T01:25:12.997562+0000","last_peered":"2023-05-20T01:25:12.997562+0000","last_clean":"2023-05-20T01:25:12.997562+0000","last_became_active":"2023-05-20T01:25:12.009426+0000","last_became_peered":"2023-05-20T01:25:12.009426+0000","last_unstale":"2023-05-20T01:25:12.997562+0000","last_undegraded":"2023-05-20T01:25:12.997562+0000","last_fullsized":"2023-05-20T01:25:12.997562+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:20:12.600985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"87'2","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:14.300608+0000","last_change":"2023-05-20T01:25:11.585068+0000","last_active":"2023-05-20T01:25:14.300608+0000","last_peered":"2023-05-20T01:25:14.300608+0000","last_clean":"2023-05-20T01:25:14.300608+0000","last_became_active":"2023-05-20T01:25:11.584758+0000","last_became_peered":"2023-05-20T01:25:11.584758+0000","last_unstale":"2023-05-20T01:25:14.300608+0000","last_undegraded":"2023-05-20T01:25:14.300608+0000","last_fullsized":"2023-05-20T01:25:14.300608+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:21:03.608718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:13.573163+0000","last_change":"2023-05-20T01:25:12.011123+0000","last_active":"2023-05-20T01:25:13.573163+0000","last_peered":"2023-05-20T01:25:13.573163+0000","last_clean":"2023-05-20T01:25:13.573163+0000","last_became_active":"2023-05-20T01:25:12.010912+0000","last_became_peered":"2023-05-20T01:25:12.010912+0000","last_unstale":"2023-05-20T01:25:13.573163+0000","last_undegraded":"2023-05-20T01:25:13.573163+0000","last_fullsized":"2023-05-20T01:25:13.573163+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:51:01.728862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:14.300508+0000","last_change":"2023-05-20T01:25:11.586764+0000","last_active":"2023-05-20T01:25:14.300508+0000","last_peered":"2023-05-20T01:25:14.300508+0000","last_clean":"2023-05-20T01:25:14.300508+0000","last_became_active":"2023-05-20T01:25:11.586458+0000","last_became_peered":"2023-05-20T01:25:11.586458+0000","last_unstale":"2023-05-20T01:25:14.300508+0000","last_undegraded":"2023-05-20T01:25:14.300508+0000","last_fullsized":"2023-05-20T01:25:14.300508+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:33:40.069024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:13.059684+0000","last_change":"2023-05-20T01:25:12.009977+0000","last_active":"2023-05-20T01:25:13.059684+0000","last_peered":"2023-05-20T01:25:13.059684+0000","last_clean":"2023-05-20T01:25:13.059684+0000","last_became_active":"2023-05-20T01:25:12.009706+0000","last_became_peered":"2023-05-20T01:25:12.009706+0000","last_unstale":"2023-05-20T01:25:13.059684+0000","last_undegraded":"2023-05-20T01:25:13.059684+0000","last_fullsized":"2023-05-20T01:25:13.059684+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:09:50.800709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:13.059704+0000","last_change":"2023-05-20T01:25:11.580023+0000","last_active":"2023-05-20T01:25:13.059704+0000","last_peered":"2023-05-20T01:25:13.059704+0000","last_clean":"2023-05-20T01:25:13.059704+0000","last_became_active":"2023-05-20T01:25:11.579731+0000","last_became_peered":"2023-05-20T01:25:11.579731+0000","last_unstale":"2023-05-20T01:25:13.059704+0000","last_undegraded":"2023-05-20T01:25:13.059704+0000","last_fullsized":"2023-05-20T01:25:13.059704+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:50:07.905717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:12.997625+0000","last_change":"2023-05-20T01:25:12.013914+0000","last_active":"2023-05-20T01:25:12.997625+0000","last_peered":"2023-05-20T01:25:12.997625+0000","last_clean":"2023-05-20T01:25:12.997625+0000","last_became_active":"2023-05-20T01:25:12.013647+0000","last_became_peered":"2023-05-20T01:25:12.013647+0000","last_unstale":"2023-05-20T01:25:12.997625+0000","last_undegraded":"2023-05-20T01:25:12.997625+0000","last_fullsized":"2023-05-20T01:25:12.997625+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:30:05.135875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:13.060142+0000","last_change":"2023-05-20T01:25:11.579261+0000","last_active":"2023-05-20T01:25:13.060142+0000","last_peered":"2023-05-20T01:25:13.060142+0000","last_clean":"2023-05-20T01:25:13.060142+0000","last_became_active":"2023-05-20T01:25:11.578954+0000","last_became_peered":"2023-05-20T01:25:11.578954+0000","last_unstale":"2023-05-20T01:25:13.060142+0000","last_undegraded":"2023-05-20T01:25:13.060142+0000","last_fullsized":"2023-05-20T01:25:13.060142+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T01:55:32.923901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:13.803987+0000","last_change":"2023-05-20T01:25:12.007820+0000","last_active":"2023-05-20T01:25:13.803987+0000","last_peered":"2023-05-20T01:25:13.803987+0000","last_clean":"2023-05-20T01:25:13.803987+0000","last_became_active":"2023-05-20T01:25:12.007537+0000","last_became_peered":"2023-05-20T01:25:12.007537+0000","last_unstale":"2023-05-20T01:25:13.803987+0000","last_undegraded":"2023-05-20T01:25:13.803987+0000","last_fullsized":"2023-05-20T01:25:13.803987+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:11:21.233757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:13.174216+0000","last_change":"2023-05-20T01:25:11.579108+0000","last_active":"2023-05-20T01:25:13.174216+0000","last_peered":"2023-05-20T01:25:13.174216+0000","last_clean":"2023-05-20T01:25:13.174216+0000","last_became_active":"2023-05-20T01:25:11.578740+0000","last_became_peered":"2023-05-20T01:25:11.578740+0000","last_unstale":"2023-05-20T01:25:13.174216+0000","last_undegraded":"2023-05-20T01:25:13.174216+0000","last_fullsized":"2023-05-20T01:25:13.174216+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T13:19:29.600229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:13.174195+0000","last_change":"2023-05-20T01:25:12.022288+0000","last_active":"2023-05-20T01:25:13.174195+0000","last_peered":"2023-05-20T01:25:13.174195+0000","last_clean":"2023-05-20T01:25:13.174195+0000","last_became_active":"2023-05-20T01:25:12.020042+0000","last_became_peered":"2023-05-20T01:25:12.020042+0000","last_unstale":"2023-05-20T01:25:13.174195+0000","last_undegraded":"2023-05-20T01:25:13.174195+0000","last_fullsized":"2023-05-20T01:25:13.174195+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:53:17.759312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:12.997395+0000","last_change":"2023-05-20T01:25:11.574489+0000","last_active":"2023-05-20T01:25:12.997395+0000","last_peered":"2023-05-20T01:25:12.997395+0000","last_clean":"2023-05-20T01:25:12.997395+0000","last_became_active":"2023-05-20T01:25:11.574182+0000","last_became_peered":"2023-05-20T01:25:11.574182+0000","last_unstale":"2023-05-20T01:25:12.997395+0000","last_undegraded":"2023-05-20T01:25:12.997395+0000","last_fullsized":"2023-05-20T01:25:12.997395+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:53:41.459514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:13.572588+0000","last_change":"2023-05-20T01:25:12.012592+0000","last_active":"2023-05-20T01:25:13.572588+0000","last_peered":"2023-05-20T01:25:13.572588+0000","last_clean":"2023-05-20T01:25:13.572588+0000","last_became_active":"2023-05-20T01:25:12.012306+0000","last_became_peered":"2023-05-20T01:25:12.012306+0000","last_unstale":"2023-05-20T01:25:13.572588+0000","last_undegraded":"2023-05-20T01:25:13.572588+0000","last_fullsized":"2023-05-20T01:25:13.572588+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:54:25.471432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:13.059703+0000","last_change":"2023-05-20T01:25:11.578666+0000","last_active":"2023-05-20T01:25:13.059703+0000","last_peered":"2023-05-20T01:25:13.059703+0000","last_clean":"2023-05-20T01:25:13.059703+0000","last_became_active":"2023-05-20T01:25:11.577385+0000","last_became_peered":"2023-05-20T01:25:11.577385+0000","last_unstale":"2023-05-20T01:25:13.059703+0000","last_undegraded":"2023-05-20T01:25:13.059703+0000","last_fullsized":"2023-05-20T01:25:13.059703+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:00:47.634174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:13.059693+0000","last_change":"2023-05-20T01:25:12.009771+0000","last_active":"2023-05-20T01:25:13.059693+0000","last_peered":"2023-05-20T01:25:13.059693+0000","last_clean":"2023-05-20T01:25:13.059693+0000","last_became_active":"2023-05-20T01:25:12.009463+0000","last_became_peered":"2023-05-20T01:25:12.009463+0000","last_unstale":"2023-05-20T01:25:13.059693+0000","last_undegraded":"2023-05-20T01:25:13.059693+0000","last_fullsized":"2023-05-20T01:25:13.059693+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:42:51.410878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:12.997143+0000","last_change":"2023-05-20T01:25:11.578192+0000","last_active":"2023-05-20T01:25:12.997143+0000","last_peered":"2023-05-20T01:25:12.997143+0000","last_clean":"2023-05-20T01:25:12.997143+0000","last_became_active":"2023-05-20T01:25:11.577843+0000","last_became_peered":"2023-05-20T01:25:11.577843+0000","last_unstale":"2023-05-20T01:25:12.997143+0000","last_undegraded":"2023-05-20T01:25:12.997143+0000","last_fullsized":"2023-05-20T01:25:12.997143+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T01:37:44.036317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:13.803987+0000","last_change":"2023-05-20T01:25:12.016442+0000","last_active":"2023-05-20T01:25:13.803987+0000","last_peered":"2023-05-20T01:25:13.803987+0000","last_clean":"2023-05-20T01:25:13.803987+0000","last_became_active":"2023-05-20T01:25:12.016182+0000","last_became_peered":"2023-05-20T01:25:12.016182+0000","last_unstale":"2023-05-20T01:25:13.803987+0000","last_undegraded":"2023-05-20T01:25:13.803987+0000","last_fullsized":"2023-05-20T01:25:13.803987+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:31:37.871606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:12.995985+0000","last_change":"2023-05-20T01:25:11.581315+0000","last_active":"2023-05-20T01:25:12.995985+0000","last_peered":"2023-05-20T01:25:12.995985+0000","last_clean":"2023-05-20T01:25:12.995985+0000","last_became_active":"2023-05-20T01:25:11.581043+0000","last_became_peered":"2023-05-20T01:25:11.581043+0000","last_unstale":"2023-05-20T01:25:12.995985+0000","last_undegraded":"2023-05-20T01:25:12.995985+0000","last_fullsized":"2023-05-20T01:25:12.995985+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:55:54.205242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":151,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:12.997619+0000","last_change":"2023-05-20T01:16:23.289068+0000","last_active":"2023-05-20T01:25:12.997619+0000","last_peered":"2023-05-20T01:25:12.997619+0000","last_clean":"2023-05-20T01:25:12.997619+0000","last_became_active":"2023-05-20T01:16:23.288781+0000","last_became_peered":"2023-05-20T01:16:23.288781+0000","last_unstale":"2023-05-20T01:25:12.997619+0000","last_undegraded":"2023-05-20T01:25:12.997619+0000","last_fullsized":"2023-05-20T01:25:12.997619+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:21:48.543560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:12.996991+0000","last_change":"2023-05-20T01:25:12.015626+0000","last_active":"2023-05-20T01:25:12.996991+0000","last_peered":"2023-05-20T01:25:12.996991+0000","last_clean":"2023-05-20T01:25:12.996991+0000","last_became_active":"2023-05-20T01:25:12.015388+0000","last_became_peered":"2023-05-20T01:25:12.015388+0000","last_unstale":"2023-05-20T01:25:12.996991+0000","last_undegraded":"2023-05-20T01:25:12.996991+0000","last_fullsized":"2023-05-20T01:25:12.996991+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T01:46:01.851831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:12.997062+0000","last_change":"2023-05-20T01:25:11.582487+0000","last_active":"2023-05-20T01:25:12.997062+0000","last_peered":"2023-05-20T01:25:12.997062+0000","last_clean":"2023-05-20T01:25:12.997062+0000","last_became_active":"2023-05-20T01:25:11.582061+0000","last_became_peered":"2023-05-20T01:25:11.582061+0000","last_unstale":"2023-05-20T01:25:12.997062+0000","last_undegraded":"2023-05-20T01:25:12.997062+0000","last_fullsized":"2023-05-20T01:25:12.997062+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T13:22:10.696056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":157,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:12.995934+0000","last_change":"2023-05-20T01:16:22.829324+0000","last_active":"2023-05-20T01:25:12.995934+0000","last_peered":"2023-05-20T01:25:12.995934+0000","last_clean":"2023-05-20T01:25:12.995934+0000","last_became_active":"2023-05-20T01:16:22.828964+0000","last_became_peered":"2023-05-20T01:16:22.828964+0000","last_unstale":"2023-05-20T01:25:12.995934+0000","last_undegraded":"2023-05-20T01:25:12.995934+0000","last_fullsized":"2023-05-20T01:25:12.995934+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T01:22:38.911204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:13.060139+0000","last_change":"2023-05-20T01:25:12.010469+0000","last_active":"2023-05-20T01:25:13.060139+0000","last_peered":"2023-05-20T01:25:13.060139+0000","last_clean":"2023-05-20T01:25:13.060139+0000","last_became_active":"2023-05-20T01:25:12.010160+0000","last_became_peered":"2023-05-20T01:25:12.010160+0000","last_unstale":"2023-05-20T01:25:13.060139+0000","last_undegraded":"2023-05-20T01:25:13.060139+0000","last_fullsized":"2023-05-20T01:25:13.060139+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:29:00.306831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"87'2","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:13.174273+0000","last_change":"2023-05-20T01:25:11.581614+0000","last_active":"2023-05-20T01:25:13.174273+0000","last_peered":"2023-05-20T01:25:13.174273+0000","last_clean":"2023-05-20T01:25:13.174273+0000","last_became_active":"2023-05-20T01:25:11.581114+0000","last_became_peered":"2023-05-20T01:25:11.581114+0000","last_unstale":"2023-05-20T01:25:13.174273+0000","last_undegraded":"2023-05-20T01:25:13.174273+0000","last_fullsized":"2023-05-20T01:25:13.174273+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T01:55:32.923901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":143,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:14.300404+0000","last_change":"2023-05-20T01:16:22.834754+0000","last_active":"2023-05-20T01:25:14.300404+0000","last_peered":"2023-05-20T01:25:14.300404+0000","last_clean":"2023-05-20T01:25:14.300404+0000","last_became_active":"2023-05-20T01:16:22.834437+0000","last_became_peered":"2023-05-20T01:16:22.834437+0000","last_unstale":"2023-05-20T01:25:14.300404+0000","last_undegraded":"2023-05-20T01:25:14.300404+0000","last_fullsized":"2023-05-20T01:25:14.300404+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:45:10.965559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":279,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:13.572681+0000","last_change":"2023-05-20T01:16:22.834920+0000","last_active":"2023-05-20T01:25:13.572681+0000","last_peered":"2023-05-20T01:25:13.572681+0000","last_clean":"2023-05-20T01:25:13.572681+0000","last_became_active":"2023-05-20T01:16:22.833924+0000","last_became_peered":"2023-05-20T01:16:22.833924+0000","last_unstale":"2023-05-20T01:25:13.572681+0000","last_undegraded":"2023-05-20T01:25:13.572681+0000","last_fullsized":"2023-05-20T01:25:13.572681+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:43:02.732240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:13.573031+0000","last_change":"2023-05-20T01:25:12.006623+0000","last_active":"2023-05-20T01:25:13.573031+0000","last_peered":"2023-05-20T01:25:13.573031+0000","last_clean":"2023-05-20T01:25:13.573031+0000","last_became_active":"2023-05-20T01:25:12.006338+0000","last_became_peered":"2023-05-20T01:25:12.006338+0000","last_unstale":"2023-05-20T01:25:13.573031+0000","last_undegraded":"2023-05-20T01:25:13.573031+0000","last_fullsized":"2023-05-20T01:25:13.573031+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:34:13.585780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"87'1","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:13.059954+0000","last_change":"2023-05-20T01:25:11.578913+0000","last_active":"2023-05-20T01:25:13.059954+0000","last_peered":"2023-05-20T01:25:13.059954+0000","last_clean":"2023-05-20T01:25:13.059954+0000","last_became_active":"2023-05-20T01:25:11.578494+0000","last_became_peered":"2023-05-20T01:25:11.578494+0000","last_unstale":"2023-05-20T01:25:13.059954+0000","last_undegraded":"2023-05-20T01:25:13.059954+0000","last_fullsized":"2023-05-20T01:25:13.059954+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T13:19:29.600229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":129,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:13.572495+0000","last_change":"2023-05-20T01:16:23.290259+0000","last_active":"2023-05-20T01:25:13.572495+0000","last_peered":"2023-05-20T01:25:13.572495+0000","last_clean":"2023-05-20T01:25:13.572495+0000","last_became_active":"2023-05-20T01:16:23.289911+0000","last_became_peered":"2023-05-20T01:16:23.289911+0000","last_unstale":"2023-05-20T01:25:13.572495+0000","last_undegraded":"2023-05-20T01:25:13.572495+0000","last_fullsized":"2023-05-20T01:25:13.572495+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:21:48.543560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:14.300559+0000","last_change":"2023-05-20T01:25:12.021584+0000","last_active":"2023-05-20T01:25:14.300559+0000","last_peered":"2023-05-20T01:25:14.300559+0000","last_clean":"2023-05-20T01:25:14.300559+0000","last_became_active":"2023-05-20T01:25:12.021308+0000","last_became_peered":"2023-05-20T01:25:12.021308+0000","last_unstale":"2023-05-20T01:25:14.300559+0000","last_undegraded":"2023-05-20T01:25:14.300559+0000","last_fullsized":"2023-05-20T01:25:14.300559+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:09:36.643641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"87'1","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:13.572964+0000","last_change":"2023-05-20T01:25:11.586246+0000","last_active":"2023-05-20T01:25:13.572964+0000","last_peered":"2023-05-20T01:25:13.572964+0000","last_clean":"2023-05-20T01:25:13.572964+0000","last_became_active":"2023-05-20T01:25:11.585952+0000","last_became_peered":"2023-05-20T01:25:11.585952+0000","last_unstale":"2023-05-20T01:25:13.572964+0000","last_undegraded":"2023-05-20T01:25:13.572964+0000","last_fullsized":"2023-05-20T01:25:13.572964+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T13:16:44.672035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":129,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:13.572763+0000","last_change":"2023-05-20T01:16:23.290207+0000","last_active":"2023-05-20T01:25:13.572763+0000","last_peered":"2023-05-20T01:25:13.572763+0000","last_clean":"2023-05-20T01:25:13.572763+0000","last_became_active":"2023-05-20T01:16:23.289928+0000","last_became_peered":"2023-05-20T01:16:23.289928+0000","last_unstale":"2023-05-20T01:25:13.572763+0000","last_undegraded":"2023-05-20T01:25:13.572763+0000","last_fullsized":"2023-05-20T01:25:13.572763+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:48:37.292605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:12.997485+0000","last_change":"2023-05-20T01:25:12.014668+0000","last_active":"2023-05-20T01:25:12.997485+0000","last_peered":"2023-05-20T01:25:12.997485+0000","last_clean":"2023-05-20T01:25:12.997485+0000","last_became_active":"2023-05-20T01:25:12.014376+0000","last_became_peered":"2023-05-20T01:25:12.014376+0000","last_unstale":"2023-05-20T01:25:12.997485+0000","last_undegraded":"2023-05-20T01:25:12.997485+0000","last_fullsized":"2023-05-20T01:25:12.997485+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:16:14.399334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:13.174300+0000","last_change":"2023-05-20T01:25:11.586258+0000","last_active":"2023-05-20T01:25:13.174300+0000","last_peered":"2023-05-20T01:25:13.174300+0000","last_clean":"2023-05-20T01:25:13.174300+0000","last_became_active":"2023-05-20T01:25:11.585977+0000","last_became_peered":"2023-05-20T01:25:11.585977+0000","last_unstale":"2023-05-20T01:25:13.174300+0000","last_undegraded":"2023-05-20T01:25:13.174300+0000","last_fullsized":"2023-05-20T01:25:13.174300+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:28:59.299949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":129,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:13.804116+0000","last_change":"2023-05-20T01:16:23.290538+0000","last_active":"2023-05-20T01:25:13.804116+0000","last_peered":"2023-05-20T01:25:13.804116+0000","last_clean":"2023-05-20T01:25:13.804116+0000","last_became_active":"2023-05-20T01:16:23.290230+0000","last_became_peered":"2023-05-20T01:16:23.290230+0000","last_unstale":"2023-05-20T01:25:13.804116+0000","last_undegraded":"2023-05-20T01:25:13.804116+0000","last_fullsized":"2023-05-20T01:25:13.804116+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:53:47.907406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:12.997415+0000","last_change":"2023-05-20T01:25:12.017330+0000","last_active":"2023-05-20T01:25:12.997415+0000","last_peered":"2023-05-20T01:25:12.997415+0000","last_clean":"2023-05-20T01:25:12.997415+0000","last_became_active":"2023-05-20T01:25:12.016956+0000","last_became_peered":"2023-05-20T01:25:12.016956+0000","last_unstale":"2023-05-20T01:25:12.997415+0000","last_undegraded":"2023-05-20T01:25:12.997415+0000","last_fullsized":"2023-05-20T01:25:12.997415+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:35:47.348775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"87'2","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:12.995961+0000","last_change":"2023-05-20T01:25:11.579978+0000","last_active":"2023-05-20T01:25:12.995961+0000","last_peered":"2023-05-20T01:25:12.995961+0000","last_clean":"2023-05-20T01:25:12.995961+0000","last_became_active":"2023-05-20T01:25:11.579628+0000","last_became_peered":"2023-05-20T01:25:11.579628+0000","last_unstale":"2023-05-20T01:25:12.995961+0000","last_undegraded":"2023-05-20T01:25:12.995961+0000","last_fullsized":"2023-05-20T01:25:12.995961+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:12:09.573361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":135,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:13.060078+0000","last_change":"2023-05-20T01:16:23.290341+0000","last_active":"2023-05-20T01:25:13.060078+0000","last_peered":"2023-05-20T01:25:13.060078+0000","last_clean":"2023-05-20T01:25:13.060078+0000","last_became_active":"2023-05-20T01:16:23.290053+0000","last_became_peered":"2023-05-20T01:16:23.290053+0000","last_unstale":"2023-05-20T01:25:13.060078+0000","last_undegraded":"2023-05-20T01:25:13.060078+0000","last_fullsized":"2023-05-20T01:25:13.060078+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:21:28.264291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:13.804235+0000","last_change":"2023-05-20T01:25:12.006518+0000","last_active":"2023-05-20T01:25:13.804235+0000","last_peered":"2023-05-20T01:25:13.804235+0000","last_clean":"2023-05-20T01:25:13.804235+0000","last_became_active":"2023-05-20T01:25:12.006233+0000","last_became_peered":"2023-05-20T01:25:12.006233+0000","last_unstale":"2023-05-20T01:25:13.804235+0000","last_undegraded":"2023-05-20T01:25:13.804235+0000","last_fullsized":"2023-05-20T01:25:13.804235+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:10:37.595259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:12.997472+0000","last_change":"2023-05-20T01:25:11.587195+0000","last_active":"2023-05-20T01:25:12.997472+0000","last_peered":"2023-05-20T01:25:12.997472+0000","last_clean":"2023-05-20T01:25:12.997472+0000","last_became_active":"2023-05-20T01:25:11.586759+0000","last_became_peered":"2023-05-20T01:25:11.586759+0000","last_unstale":"2023-05-20T01:25:12.997472+0000","last_undegraded":"2023-05-20T01:25:12.997472+0000","last_fullsized":"2023-05-20T01:25:12.997472+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T01:39:36.762742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":135,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:13.059738+0000","last_change":"2023-05-20T01:16:22.834330+0000","last_active":"2023-05-20T01:25:13.059738+0000","last_peered":"2023-05-20T01:25:13.059738+0000","last_clean":"2023-05-20T01:25:13.059738+0000","last_became_active":"2023-05-20T01:16:22.833946+0000","last_became_peered":"2023-05-20T01:16:22.833946+0000","last_unstale":"2023-05-20T01:25:13.059738+0000","last_undegraded":"2023-05-20T01:25:13.059738+0000","last_fullsized":"2023-05-20T01:25:13.059738+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:26:45.208108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574959,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":11756,"kb_used_data":1636,"kb_used_omap":0,"kb_used_meta":10112,"kb_avail":94360084,"statfs":{"total":96636764160,"available":96624726016,"internally_reserved":0,"allocated":1675264,"data_stored":1134441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":10354688},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:24:34 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.47699999999999998,"15min":0.51400000000000001},"min":{"1min":0.26900000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.68799999999999994,"5min":0.73199999999999998,"15min":0.88200000000000001},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.503,"15min":0.54700000000000004},"min":{"1min":0.33100000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.63600000000000001,"5min":0.752,"15min":0.90000000000000002},"last":0.77100000000000002}]},{"osd":1,"last update":"Sat May 20 01:24:34 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.49299999999999999,"15min":0.75},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.628,"5min":0.70299999999999996,"15min":8.8879999999999999},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.52200000000000002,"15min":0.78900000000000003},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.74199999999999999,"5min":0.75,"15min":8.875},"last":0.5}]},{"osd":2,"last update":"Sat May 20 01:24:34 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.503,"15min":0.73599999999999999},"min":{"1min":0.34000000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.65900000000000003,"5min":0.77100000000000002,"15min":8.9120000000000008},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.504,"15min":0.73899999999999999},"min":{"1min":0.35999999999999999,"5min":0.35499999999999998,"15min":0.35099999999999998},"max":{"1min":0.58999999999999997,"5min":0.72399999999999998,"15min":8.8520000000000003},"last":0.64000000000000001}]},{"osd":3,"last update":"Sat May 20 01:24:34 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.49399999999999999,"15min":0.79700000000000004},"min":{"1min":0.36399999999999999,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.60599999999999998,"5min":0.73799999999999999,"15min":10.884},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.55100000000000005,"15min":0.85099999999999998},"min":{"1min":0.40600000000000003,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":10.952},"last":0.622}]},{"osd":4,"last update":"Sat May 20 01:24:34 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51100000000000001,"15min":0.51000000000000001},"min":{"1min":0.39200000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.66700000000000004,"5min":0.74299999999999999,"15min":0.77000000000000002},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.312},"max":{"1min":0.71599999999999997,"5min":0.78500000000000003,"15min":0.875},"last":0.59899999999999998}]},{"osd":5,"last update":"Sat May 20 01:24:34 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.56299999999999994,"15min":0.54200000000000004},"min":{"1min":0.35999999999999999,"5min":0.35799999999999998,"15min":0.32300000000000001},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.57099999999999995,"15min":0.57599999999999996},"min":{"1min":0.35999999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.745,"5min":0.79600000000000004,"15min":0.80200000000000005},"last":0.68799999999999994}]},{"osd":6,"last update":"Sat May 20 01:24:34 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.59599999999999997,"15min":0.59499999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.378},"max":{"1min":0.72699999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.57799999999999996,"15min":0.58399999999999996},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.73399999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.754}]}]},{"osd":6,"up_from":13,"seq":55834574958,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10416,"kb_used_data":1064,"kb_used_omap":0,"kb_used_meta":9344,"kb_avail":94361424,"statfs":{"total":96636764160,"available":96626098176,"internally_reserved":0,"allocated":1089536,"data_stored":544697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9568256},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:24:36 2023","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.45400000000000001,"15min":0.47799999999999998},"min":{"1min":0.29099999999999998,"5min":0.28999999999999998,"15min":0.26900000000000002},"max":{"1min":0.55500000000000005,"5min":0.72699999999999998,"15min":0.77200000000000002},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.438,"5min":0.47199999999999998,"15min":0.51800000000000002},"min":{"1min":0.309,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.60299999999999998,"5min":0.70099999999999996,"15min":0.73999999999999999},"last":0.497}]},{"osd":1,"last update":"Sat May 20 01:24:36 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.495,"15min":0.52100000000000002},"min":{"1min":0.312,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.70499999999999996,"5min":0.71699999999999997,"15min":0.83799999999999997},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.50800000000000001,"15min":0.52400000000000002},"min":{"1min":0.33000000000000002,"5min":0.32600000000000001,"15min":0.318},"max":{"1min":0.68600000000000005,"5min":0.749,"15min":0.79200000000000004},"last":0.66500000000000004}]},{"osd":2,"last update":"Sat May 20 01:24:36 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.48199999999999998,"15min":0.504},"min":{"1min":0.36799999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.52700000000000002,"5min":0.72199999999999998,"15min":0.76600000000000001},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.51800000000000002,"15min":0.55000000000000004},"min":{"1min":0.35899999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.629,"5min":0.76200000000000001,"15min":0.76800000000000002},"last":0.57599999999999996}]},{"osd":3,"last update":"Sat May 20 01:24:36 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.51500000000000001,"15min":0.53500000000000003},"min":{"1min":0.32400000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.80800000000000005},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.52100000000000002,"15min":0.54000000000000004},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.58099999999999996,"5min":0.76400000000000001,"15min":0.79500000000000004},"last":0.52500000000000002}]},{"osd":4,"last update":"Sat May 20 01:24:36 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.53800000000000003,"15min":0.55000000000000004},"min":{"1min":0.42599999999999999,"5min":0.33300000000000002,"15min":0.32600000000000001},"max":{"1min":0.754,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.54700000000000004,"15min":0.56899999999999995},"min":{"1min":0.40300000000000002,"5min":0.378,"15min":0.32700000000000001},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.80100000000000005},"last":0.71799999999999997}]},{"osd":5,"last update":"Sat May 20 01:24:36 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.54100000000000004,"15min":0.54400000000000004},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.65000000000000002,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.57599999999999996,"15min":0.60399999999999998},"min":{"1min":0.46100000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.78600000000000003,"5min":0.81599999999999995,"15min":0.89400000000000002},"last":0.624}]},{"osd":7,"last update":"Sat May 20 01:24:36 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57699999999999996,"15min":0.59799999999999998},"min":{"1min":0.44,"5min":0.371,"15min":0.371},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.92900000000000005},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.58099999999999996,"15min":0.60999999999999999},"min":{"1min":0.48099999999999998,"5min":0.379,"15min":0.379},"max":{"1min":0.68100000000000005,"5min":0.82799999999999996,"15min":0.85199999999999998},"last":0.60999999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574958,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":16628,"kb_used_data":1644,"kb_used_omap":0,"kb_used_meta":14976,"kb_avail":94355212,"statfs":{"total":96636764160,"available":96619737088,"internally_reserved":0,"allocated":1683456,"data_stored":1134565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":15335424},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:24:33 2023","interfaces":[{"interface":"back","average":{"1min":0.254,"5min":0.28199999999999997,"15min":0.313},"min":{"1min":0.186,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":0.37,"5min":0.57099999999999995,"15min":1.194},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.28299999999999997,"5min":0.314,"15min":0.316},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.41399999999999998,"5min":0.61499999999999999,"15min":0.75800000000000001},"last":0.51200000000000001}]},{"osd":2,"last update":"Sat May 20 01:24:33 2023","interfaces":[{"interface":"back","average":{"1min":0.27200000000000002,"5min":0.30199999999999999,"15min":0.317},"min":{"1min":0.20100000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.33800000000000002,"5min":0.52400000000000002,"15min":0.66400000000000003},"last":0.437},{"interface":"front","average":{"1min":0.33700000000000002,"5min":0.36799999999999999,"15min":0.40600000000000003},"min":{"1min":0.27800000000000002,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.41299999999999998,"5min":0.60299999999999998,"15min":1.3580000000000001},"last":0.63200000000000001}]},{"osd":3,"last update":"Sat May 20 01:24:33 2023","interfaces":[{"interface":"back","average":{"1min":0.315,"5min":0.34999999999999998,"15min":0.36899999999999999},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.40500000000000003,"5min":0.57399999999999995,"15min":1.619},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.32100000000000001,"5min":0.34699999999999998,"15min":0.379},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.41299999999999998,"5min":0.60899999999999999,"15min":1.8400000000000001},"last":0.57699999999999996}]},{"osd":4,"last update":"Sat May 20 01:24:33 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.55900000000000005,"15min":0.58099999999999996},"min":{"1min":0.379,"5min":0.34899999999999998,"15min":0.32200000000000001},"max":{"1min":0.72799999999999998,"5min":0.85299999999999998,"15min":1.2769999999999999},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.56999999999999995,"15min":0.58299999999999996},"min":{"1min":0.41699999999999998,"5min":0.312,"15min":0.312},"max":{"1min":0.65600000000000003,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.78000000000000003}]},{"osd":5,"last update":"Sat May 20 01:24:33 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.58399999999999996,"15min":0.58899999999999997},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.754,"5min":0.81899999999999995,"15min":1.4510000000000001},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.59299999999999997,"15min":0.60599999999999998},"min":{"1min":0.436,"5min":0.39100000000000001,"15min":0.378},"max":{"1min":0.77200000000000002,"5min":0.79900000000000004,"15min":1.409},"last":0.71699999999999997}]},{"osd":6,"last update":"Sat May 20 01:24:33 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.59499999999999997,"15min":0.60399999999999998},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.73599999999999999,"5min":0.876,"15min":1.575},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.58499999999999996,"15min":0.59399999999999997},"min":{"1min":0.41299999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.66400000000000003,"5min":0.83099999999999996,"15min":1.0600000000000001},"last":0.69499999999999995}]},{"osd":7,"last update":"Sat May 20 01:24:33 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.59499999999999997,"15min":0.60499999999999998},"min":{"1min":0.40500000000000003,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.75600000000000001,"5min":0.88700000000000001,"15min":1.496},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.623,"15min":0.63800000000000001},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.77300000000000002,"5min":2.1709999999999998,"15min":2.1709999999999998},"last":0.79500000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574958,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10468,"kb_used_data":1052,"kb_used_omap":0,"kb_used_meta":9408,"kb_avail":94361372,"statfs":{"total":96636764160,"available":96626044928,"internally_reserved":0,"allocated":1077248,"data_stored":543076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9633792},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat May 20 01:24:32 2023","interfaces":[{"interface":"back","average":{"1min":0.26200000000000001,"5min":0.29199999999999998,"15min":0.318},"min":{"1min":0.192,"5min":0.17999999999999999,"15min":0.17999999999999999},"max":{"1min":0.371,"5min":0.54600000000000004,"15min":0.56599999999999995},"last":0.224},{"interface":"front","average":{"1min":0.26400000000000001,"5min":0.29699999999999999,"15min":0.34200000000000003},"min":{"1min":0.21099999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.33300000000000002,"5min":0.52000000000000002,"15min":0.98799999999999999},"last":0.249}]},{"osd":2,"last update":"Sat May 20 01:24:32 2023","interfaces":[{"interface":"back","average":{"1min":0.28100000000000003,"5min":0.29999999999999999,"15min":0.32800000000000001},"min":{"1min":0.20100000000000001,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":0.35999999999999999,"5min":0.52300000000000002,"15min":0.55700000000000005},"last":0.26700000000000002},{"interface":"front","average":{"1min":0.33100000000000002,"5min":0.34899999999999998,"15min":0.38200000000000001},"min":{"1min":0.26200000000000001,"5min":0.216,"15min":0.216},"max":{"1min":0.41699999999999998,"5min":0.60299999999999998,"15min":0.86199999999999999},"last":0.32800000000000001}]},{"osd":3,"last update":"Sat May 20 01:24:32 2023","interfaces":[{"interface":"back","average":{"1min":0.29599999999999999,"5min":0.33200000000000002,"15min":0.35999999999999999},"min":{"1min":0.24299999999999999,"5min":0.22,"15min":0.22},"max":{"1min":0.374,"5min":0.623,"15min":0.68300000000000005},"last":0.315},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.35399999999999998,"15min":0.36599999999999999},"min":{"1min":0.28000000000000003,"5min":0.245,"15min":0.245},"max":{"1min":0.44700000000000001,"5min":0.60899999999999999,"15min":0.60899999999999999},"last":0.34000000000000002}]},{"osd":4,"last update":"Sat May 20 01:24:32 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.57099999999999995,"15min":0.55100000000000005},"min":{"1min":0.433,"5min":0.372,"15min":0.34399999999999997},"max":{"1min":0.78800000000000003,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.57999999999999996,"15min":0.56899999999999995},"min":{"1min":0.40799999999999997,"5min":0.39200000000000002,"15min":0.36299999999999999},"max":{"1min":0.70899999999999996,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.54200000000000004}]},{"osd":5,"last update":"Sat May 20 01:24:32 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.57799999999999996,"15min":0.57599999999999996},"min":{"1min":0.379,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.74299999999999999,"5min":0.80500000000000005,"15min":0.88100000000000001},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.63400000000000001,"15min":0.58899999999999997},"min":{"1min":0.40899999999999997,"5min":0.374,"15min":0.374},"max":{"1min":0.76300000000000001,"5min":4.4720000000000004,"15min":4.4720000000000004},"last":0.47199999999999998}]},{"osd":6,"last update":"Sat May 20 01:24:32 2023","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":0.72799999999999998,"15min":0.625},"min":{"1min":0.45800000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":10.042999999999999,"5min":10.042999999999999,"15min":10.042999999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.60499999999999998,"15min":0.61599999999999999},"min":{"1min":0.45200000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.73399999999999999,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.64300000000000002}]},{"osd":7,"last update":"Sat May 20 01:24:32 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.59499999999999997,"15min":0.59299999999999997},"min":{"1min":0.433,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.74199999999999999,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.501},{"interface":"front","average":{"1min":1.1040000000000001,"5min":0.72499999999999998,"15min":0.65700000000000003},"min":{"1min":0.44800000000000001,"5min":0.438,"15min":0.42099999999999999},"max":{"1min":10.085000000000001,"5min":10.085000000000001,"15min":10.085000000000001},"last":0.60199999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574958,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10536,"kb_used_data":1056,"kb_used_omap":0,"kb_used_meta":9472,"kb_avail":94361304,"statfs":{"total":96636764160,"available":96625975296,"internally_reserved":0,"allocated":1081344,"data_stored":544585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9699328},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:24:44 2023","interfaces":[{"interface":"back","average":{"1min":0.255,"5min":0.27500000000000002,"15min":0.32400000000000001},"min":{"1min":0.19500000000000001,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":0.36799999999999999,"5min":0.53400000000000003,"15min":1.038},"last":0.24199999999999999},{"interface":"front","average":{"1min":0.26700000000000002,"5min":0.29699999999999999,"15min":0.34200000000000003},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.36799999999999999,"5min":0.46500000000000002,"15min":1.079},"last":0.28399999999999997}]},{"osd":1,"last update":"Sat May 20 01:24:44 2023","interfaces":[{"interface":"back","average":{"1min":0.28999999999999998,"5min":0.314,"15min":0.38100000000000001},"min":{"1min":0.23400000000000001,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.34899999999999998,"5min":0.61399999999999999,"15min":1.4950000000000001},"last":0.30399999999999999},{"interface":"front","average":{"1min":0.318,"5min":0.35499999999999998,"15min":0.41799999999999998},"min":{"1min":0.27400000000000002,"5min":0.249,"15min":0.249},"max":{"1min":0.36099999999999999,"5min":0.629,"15min":1.5189999999999999},"last":0.33300000000000002}]},{"osd":3,"last update":"Sat May 20 01:24:44 2023","interfaces":[{"interface":"back","average":{"1min":0.27800000000000002,"5min":0.32400000000000001,"15min":0.38200000000000001},"min":{"1min":0.218,"5min":0.216,"15min":0.216},"max":{"1min":0.42499999999999999,"5min":0.60599999999999998,"15min":1.2090000000000001},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.34000000000000002,"5min":0.36699999999999999,"15min":0.432},"min":{"1min":0.30199999999999999,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.38800000000000001,"5min":0.68100000000000005,"15min":1.5620000000000001},"last":0.35499999999999998}]},{"osd":4,"last update":"Sat May 20 01:24:44 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.55500000000000005,"15min":0.57699999999999996},"min":{"1min":0.38800000000000001,"5min":0.35899999999999999,"15min":0.34899999999999998},"max":{"1min":0.69199999999999995,"5min":0.85199999999999998,"15min":1.319},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.58499999999999996,"15min":0.61199999999999999},"min":{"1min":0.432,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.69499999999999995,"5min":0.89000000000000001,"15min":1.4319999999999999},"last":0.40999999999999998}]},{"osd":5,"last update":"Sat May 20 01:24:44 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.55800000000000005,"15min":0.62},"min":{"1min":0.41999999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.73599999999999999,"5min":0.95599999999999996,"15min":1.758},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.58299999999999996,"15min":0.63800000000000001},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.72999999999999998,"5min":0.85099999999999998,"15min":1.7110000000000001},"last":0.38300000000000001}]},{"osd":6,"last update":"Sat May 20 01:24:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.58599999999999997,"15min":0.63200000000000001},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.39700000000000002},"max":{"1min":0.746,"5min":0.88900000000000001,"15min":1.665},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.58299999999999996,"15min":0.63200000000000001},"min":{"1min":0.41699999999999998,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.74299999999999999,"5min":0.93500000000000005,"15min":1.819},"last":0.39900000000000002}]},{"osd":7,"last update":"Sat May 20 01:24:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.59499999999999997,"15min":0.65700000000000003},"min":{"1min":0.44400000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.76900000000000002,"5min":0.85099999999999998,"15min":1.7310000000000001},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.59899999999999998,"15min":0.65300000000000002},"min":{"1min":0.46999999999999997,"5min":0.42399999999999999,"15min":0.42199999999999999},"max":{"1min":0.72499999999999998,"5min":0.90600000000000003,"15min":1.8460000000000001},"last":0.47799999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574958,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10608,"kb_used_data":1064,"kb_used_omap":0,"kb_used_meta":9536,"kb_avail":94361232,"statfs":{"total":96636764160,"available":96625901568,"internally_reserved":0,"allocated":1089536,"data_stored":545567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9764864},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:24:32 2023","interfaces":[{"interface":"back","average":{"1min":0.27300000000000002,"5min":0.30599999999999999,"15min":0.317},"min":{"1min":0.21199999999999999,"5min":0.188,"15min":0.188},"max":{"1min":0.375,"5min":1.528,"15min":1.528},"last":0.22700000000000001},{"interface":"front","average":{"1min":0.28000000000000003,"5min":0.34200000000000003,"15min":0.42299999999999999},"min":{"1min":0.214,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.40699999999999997,"5min":2.4209999999999998,"15min":3.1890000000000001},"last":0.314}]},{"osd":1,"last update":"Sat May 20 01:24:32 2023","interfaces":[{"interface":"back","average":{"1min":0.27300000000000002,"5min":0.33600000000000002,"15min":0.41499999999999998},"min":{"1min":0.19900000000000001,"5min":0.189,"15min":0.189},"max":{"1min":0.377,"5min":3.2999999999999998,"15min":3.4260000000000002},"last":0.24199999999999999},{"interface":"front","average":{"1min":0.33200000000000002,"5min":0.34200000000000003,"15min":0.35599999999999998},"min":{"1min":0.23799999999999999,"5min":0.22900000000000001,"15min":0.221},"max":{"1min":0.45500000000000002,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.33200000000000002}]},{"osd":2,"last update":"Sat May 20 01:24:32 2023","interfaces":[{"interface":"back","average":{"1min":0.30599999999999999,"5min":0.34300000000000003,"15min":0.38700000000000001},"min":{"1min":0.26400000000000001,"5min":0.245,"15min":0.224},"max":{"1min":0.39200000000000002,"5min":0.625,"15min":1.1619999999999999},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.371,"15min":0.41299999999999998},"min":{"1min":0.28499999999999998,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.41499999999999998,"5min":0.72399999999999998,"15min":1.413},"last":0.29499999999999998}]},{"osd":4,"last update":"Sat May 20 01:24:32 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.55100000000000005,"15min":0.58699999999999997},"min":{"1min":0.40300000000000002,"5min":0.38600000000000001,"15min":0.38200000000000001},"max":{"1min":0.72899999999999998,"5min":0.95399999999999996,"15min":1.0700000000000001},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.61199999999999999,"15min":0.68000000000000005},"min":{"1min":0.41099999999999998,"5min":0.40200000000000002,"15min":0.35899999999999999},"max":{"1min":0.72899999999999998,"5min":3.1789999999999998,"15min":3.4660000000000002},"last":0.48999999999999999}]},{"osd":5,"last update":"Sat May 20 01:24:32 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.59899999999999998,"15min":0.68500000000000005},"min":{"1min":0.38400000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.751,"5min":3.2170000000000001,"15min":3.5379999999999998},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.59999999999999998,"15min":0.63200000000000001},"min":{"1min":0.41999999999999998,"5min":0.40799999999999997,"15min":0.374},"max":{"1min":0.74199999999999999,"5min":1.026,"15min":1.385},"last":0.52100000000000002}]},{"osd":6,"last update":"Sat May 20 01:24:32 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.59599999999999997,"15min":0.63400000000000001},"min":{"1min":0.439,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.70199999999999996,"5min":0.99199999999999999,"15min":1.347},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.58499999999999996,"15min":0.63},"min":{"1min":0.39500000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.752,"5min":0.94099999999999995,"15min":1.107},"last":0.433}]},{"osd":7,"last update":"Sat May 20 01:24:32 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.59699999999999998,"15min":0.63700000000000001},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.64000000000000001,"5min":0.94199999999999995,"15min":1.1359999999999999},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.625,"5min":0.65100000000000002,"15min":0.72499999999999998},"min":{"1min":0.47599999999999998,"5min":0.436,"15min":0.436},"max":{"1min":0.73799999999999999,"5min":3.2559999999999998,"15min":3.5009999999999999},"last":0.54800000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574958,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":15908,"kb_used_data":1052,"kb_used_omap":0,"kb_used_meta":14848,"kb_avail":94355932,"statfs":{"total":96636764160,"available":96620474368,"internally_reserved":0,"allocated":1077248,"data_stored":544003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":15204352},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:24:33 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.47299999999999998,"15min":0.48099999999999998},"min":{"1min":0.32200000000000001,"5min":0.247,"15min":0.247},"max":{"1min":0.59199999999999997,"5min":0.73699999999999999,"15min":0.75600000000000001},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.50700000000000001,"15min":0.498},"min":{"1min":0.30599999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.64800000000000002,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":0.67400000000000004}]},{"osd":1,"last update":"Sat May 20 01:24:33 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.49199999999999999,"15min":0.51900000000000002},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.65300000000000002,"5min":0.746,"15min":0.82799999999999996},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.51900000000000002,"15min":0.53000000000000003},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.63600000000000001,"5min":1.7190000000000001,"15min":2.0739999999999998},"last":0.58499999999999996}]},{"osd":2,"last update":"Sat May 20 01:24:33 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.51900000000000002,"15min":0.50800000000000001},"min":{"1min":0.36599999999999999,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.64900000000000002,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.54200000000000004,"15min":0.55000000000000004},"min":{"1min":0.378,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.61399999999999999,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.626}]},{"osd":3,"last update":"Sat May 20 01:24:33 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.52600000000000002,"15min":0.52400000000000002},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34399999999999997},"max":{"1min":0.626,"5min":0.81399999999999995,"15min":1.827},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.57099999999999995,"15min":0.65500000000000003},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.65000000000000002,"5min":0.89300000000000002,"15min":3.1749999999999998},"last":0.64700000000000002}]},{"osd":5,"last update":"Sat May 20 01:24:33 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.53400000000000003,"15min":0.48699999999999999},"min":{"1min":0.36899999999999999,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.76700000000000002,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.55400000000000005,"15min":0.51300000000000001},"min":{"1min":0.35099999999999998,"5min":0.33400000000000002,"15min":0.29899999999999999},"max":{"1min":0.78400000000000003,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.52000000000000002}]},{"osd":6,"last update":"Sat May 20 01:24:33 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.58799999999999997,"15min":0.54700000000000004},"min":{"1min":0.432,"5min":0.40400000000000003,"15min":0.32400000000000001},"max":{"1min":0.80300000000000005,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.443},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.61599999999999999,"15min":0.59099999999999997},"min":{"1min":0.51100000000000001,"5min":0.42399999999999999,"15min":0.39400000000000002},"max":{"1min":0.72299999999999998,"5min":0.84699999999999998,"15min":0.85899999999999999},"last":0.66000000000000003}]},{"osd":7,"last update":"Sat May 20 01:24:33 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.57799999999999996,"15min":0.55700000000000005},"min":{"1min":0.41999999999999998,"5min":0.36099999999999999,"15min":0.34200000000000003},"max":{"1min":0.71799999999999997,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.61399999999999999,"15min":0.59799999999999998},"min":{"1min":0.499,"5min":0.36399999999999999,"15min":0.34000000000000002},"max":{"1min":0.73699999999999999,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.56399999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574959,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10352,"kb_used_data":1064,"kb_used_omap":0,"kb_used_meta":9280,"kb_avail":94361488,"statfs":{"total":96636764160,"available":96626163712,"internally_reserved":0,"allocated":1089536,"data_stored":544628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9502720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:24:38 2023","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.45400000000000001,"15min":0.65300000000000002},"min":{"1min":0.29999999999999999,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.68899999999999995,"5min":0.72599999999999998,"15min":6.3929999999999998},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.46000000000000002,"15min":0.68000000000000005},"min":{"1min":0.34999999999999998,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.61099999999999999,"5min":0.67500000000000004,"15min":6.3339999999999996},"last":0.83699999999999997}]},{"osd":1,"last update":"Sat May 20 01:24:38 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.46000000000000002,"15min":0.58799999999999997},"min":{"1min":0.313,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":3.2450000000000001},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.498,"5min":0.499,"15min":0.53400000000000003},"min":{"1min":0.38900000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.72599999999999998,"5min":0.752,"15min":1.1819999999999999},"last":0.77700000000000002}]},{"osd":2,"last update":"Sat May 20 01:24:38 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48299999999999998,"15min":0.54100000000000004},"min":{"1min":0.33200000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":1.1639999999999999},"last":0.751},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.48399999999999999,"15min":0.54400000000000004},"min":{"1min":0.308,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.628,"5min":0.71799999999999997,"15min":1.258},"last":0.76400000000000001}]},{"osd":3,"last update":"Sat May 20 01:24:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.51500000000000001,"15min":0.72499999999999998},"min":{"1min":0.35099999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.69499999999999995,"5min":0.77800000000000002,"15min":6.351},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.53200000000000003,"15min":0.75800000000000001},"min":{"1min":0.40699999999999997,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.67500000000000004,"5min":0.76500000000000001,"15min":6.3730000000000002},"last":0.79000000000000004}]},{"osd":4,"last update":"Sat May 20 01:24:38 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.503,"15min":0.49299999999999999},"min":{"1min":0.38900000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.64500000000000002,"5min":0.68400000000000005,"15min":0.70199999999999996},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.54000000000000004,"15min":0.55100000000000005},"min":{"1min":0.41599999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.749,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.48999999999999999}]},{"osd":6,"last update":"Sat May 20 01:24:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.54600000000000004,"15min":0.55200000000000005},"min":{"1min":0.48699999999999999,"5min":0.38100000000000001,"15min":0.29999999999999999},"max":{"1min":0.72899999999999998,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.54500000000000004,"15min":0.55800000000000005},"min":{"1min":0.437,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.72399999999999998,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.51300000000000001}]},{"osd":7,"last update":"Sat May 20 01:24:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.56799999999999995,"15min":0.58199999999999996},"min":{"1min":0.46500000000000002,"5min":0.33400000000000002,"15min":0.33000000000000002},"max":{"1min":0.77300000000000002,"5min":0.82299999999999995,"15min":0.91000000000000003},"last":0.629},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.57699999999999996,"15min":0.60799999999999998},"min":{"1min":0.44700000000000001,"5min":0.40999999999999998,"15min":0.39500000000000002},"max":{"1min":0.76200000000000001,"5min":0.81399999999999995,"15min":0.94499999999999995},"last":0.53900000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-05-20T01:25:21.200 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-05-20T01:25:21.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.335+0000 7f11d27ee700 1 Processor -- start 2023-05-20T01:25:21.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.335+0000 7f11d27ee700 1 -- start start 2023-05-20T01:25:21.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.336+0000 7f11d27ee700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f11cc157190 0x7f11cc0cc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:21.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.336+0000 7f11d27ee700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f11cc071ad0 0x7f11cc0ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:21.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.336+0000 7f11d27ee700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11cc0cb590 con 0x7f11cc0cccf0 2023-05-20T01:25:21.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.336+0000 7f11d27ee700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11cc071f50 con 0x7f11cc071ad0 2023-05-20T01:25:21.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.336+0000 7f11d27ee700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11cc0ca9a0 con 0x7f11cc157190 2023-05-20T01:25:21.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.336+0000 7f11d0d8b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f11cc071ad0 0x7f11cc0ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:21.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.336+0000 7f11d0d8b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f11cc071ad0 0x7f11cc0ce740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36334/0 (socket says 172.21.15.16:36334) 2023-05-20T01:25:21.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.336+0000 7f11d0d8b700 1 -- 172.21.15.16:0/3893351077 learned_addr learned my addr 172.21.15.16:0/3893351077 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:21.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.336+0000 7f11cbfff700 1 --2- 172.21.15.16:0/3893351077 >> v2:172.21.15.132:3300/0 conn(0x7f11cc157190 0x7f11cc0cc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:21.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.336+0000 7f11d0d8b700 1 -- 172.21.15.16:0/3893351077 >> v2:172.21.15.132:3300/0 conn(0x7f11cc157190 msgr2=0x7f11cc0cc7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:21.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.336+0000 7f11d0d8b700 1 --2- 172.21.15.16:0/3893351077 >> v2:172.21.15.132:3300/0 conn(0x7f11cc157190 0x7f11cc0cc7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:21.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.336+0000 7f11d0d8b700 1 -- 172.21.15.16:0/3893351077 >> v1:172.21.15.132:6789/0 conn(0x7f11cc0cccf0 legacy=0x7f11cc0713c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:21.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.336+0000 7f11d0d8b700 1 -- 172.21.15.16:0/3893351077 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11cc0cac60 con 0x7f11cc071ad0 2023-05-20T01:25:21.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.336+0000 7f11d0d8b700 1 --2- 172.21.15.16:0/3893351077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11cc071ad0 0x7f11cc0ce740 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f11cc070c40 tx=0x7f11c400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c84c519bf3d5c90a server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:21.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.336+0000 7f11caffd700 1 -- 172.21.15.16:0/3893351077 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11c400e070 con 0x7f11cc071ad0 2023-05-20T01:25:21.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.337+0000 7f11caffd700 1 -- 172.21.15.16:0/3893351077 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11c400b9c0 con 0x7f11cc071ad0 2023-05-20T01:25:21.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.337+0000 7f11caffd700 1 -- 172.21.15.16:0/3893351077 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11c400bd70 con 0x7f11cc071ad0 2023-05-20T01:25:21.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.337+0000 7f11d27ee700 1 -- 172.21.15.16:0/3893351077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11cc071ad0 msgr2=0x7f11cc0ce740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:21.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.337+0000 7f11d27ee700 1 --2- 172.21.15.16:0/3893351077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11cc071ad0 0x7f11cc0ce740 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f11cc070c40 tx=0x7f11c400b330 comp rx=0 tx=0).stop 2023-05-20T01:25:21.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.337+0000 7f11d27ee700 1 -- 172.21.15.16:0/3893351077 shutdown_connections 2023-05-20T01:25:21.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.337+0000 7f11d27ee700 1 --2- 172.21.15.16:0/3893351077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11cc071ad0 0x7f11cc0ce740 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:21.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.337+0000 7f11d27ee700 1 --2- 172.21.15.16:0/3893351077 >> v2:172.21.15.132:3300/0 conn(0x7f11cc157190 0x7f11cc0cc7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:21.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.337+0000 7f11d27ee700 1 -- 172.21.15.16:0/3893351077 >> 172.21.15.16:0/3893351077 conn(0x7f11cc0cffb0 msgr2=0x7f11cc0d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:21.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.337+0000 7f11d27ee700 1 -- 172.21.15.16:0/3893351077 shutdown_connections 2023-05-20T01:25:21.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.337+0000 7f11d27ee700 1 -- 172.21.15.16:0/3893351077 wait complete. 2023-05-20T01:25:21.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.338+0000 7f11d27ee700 1 Processor -- start 2023-05-20T01:25:21.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.339+0000 7f11d27ee700 1 -- start start 2023-05-20T01:25:21.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.339+0000 7f11d27ee700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f11cc071ad0 0x7f11cc0c6520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:21.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.339+0000 7f11d27ee700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f11cc0cccf0 0x7f11cc1484d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:21.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.339+0000 7f11d27ee700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11cc157190 0x7f11cc15bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:21.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.339+0000 7f11d27ee700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11cc1495d0 con 0x7f11cc157190 2023-05-20T01:25:21.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.339+0000 7f11d27ee700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11cc0c8b90 con 0x7f11cc071ad0 2023-05-20T01:25:21.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.339+0000 7f11d27ee700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11cc0c8e70 con 0x7f11cc0cccf0 2023-05-20T01:25:21.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.339+0000 7f11d0d8b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11cc157190 0x7f11cc15bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:21.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.339+0000 7f11d0d8b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11cc157190 0x7f11cc15bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36340/0 (socket says 172.21.15.16:36340) 2023-05-20T01:25:21.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.339+0000 7f11d0d8b700 1 -- 172.21.15.16:0/2002621555 learned_addr learned my addr 172.21.15.16:0/2002621555 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:21.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.340+0000 7f11cbfff700 1 --2- 172.21.15.16:0/2002621555 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f11cc071ad0 0x7f11cc0c6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:21.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.340+0000 7f11cb7fe700 1 --2- 172.21.15.16:0/2002621555 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f11cc0cccf0 0x7f11cc1484d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:21.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.340+0000 7f11d0d8b700 1 -- 172.21.15.16:0/2002621555 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f11cc0cccf0 msgr2=0x7f11cc1484d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:21.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.340+0000 7f11d0d8b700 1 --2- 172.21.15.16:0/2002621555 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f11cc0cccf0 0x7f11cc1484d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:21.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.340+0000 7f11d0d8b700 1 -- 172.21.15.16:0/2002621555 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f11cc071ad0 msgr2=0x7f11cc0c6520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:21.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.340+0000 7f11d0d8b700 1 --2- 172.21.15.16:0/2002621555 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f11cc071ad0 0x7f11cc0c6520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:21.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.340+0000 7f11d0d8b700 1 -- 172.21.15.16:0/2002621555 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11c4009b30 con 0x7f11cc157190 2023-05-20T01:25:21.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.340+0000 7f11d0d8b700 1 --2- 172.21.15.16:0/2002621555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11cc157190 0x7f11cc15bd50 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f11cc0d0f50 tx=0x7f11c4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:21.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.340+0000 7f11c8ff9700 1 -- 172.21.15.16:0/2002621555 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11c4004880 con 0x7f11cc157190 2023-05-20T01:25:21.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.340+0000 7f11c8ff9700 1 -- 172.21.15.16:0/2002621555 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11c40099b0 con 0x7f11cc157190 2023-05-20T01:25:21.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.340+0000 7f11d27ee700 1 -- 172.21.15.16:0/2002621555 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11cc0cb220 con 0x7f11cc157190 2023-05-20T01:25:21.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.340+0000 7f11c8ff9700 1 -- 172.21.15.16:0/2002621555 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11c401c680 con 0x7f11cc157190 2023-05-20T01:25:21.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.340+0000 7f11d27ee700 1 -- 172.21.15.16:0/2002621555 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11cc0caf40 con 0x7f11cc157190 2023-05-20T01:25:21.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.341+0000 7f11d27ee700 1 -- 172.21.15.16:0/2002621555 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11cc157560 con 0x7f11cc157190 2023-05-20T01:25:21.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.342+0000 7f11c8ff9700 1 -- 172.21.15.16:0/2002621555 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f11c401c7e0 con 0x7f11cc157190 2023-05-20T01:25:21.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.342+0000 7f11c8ff9700 1 --2- 172.21.15.16:0/2002621555 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f11b806ace0 0x7f11b806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:21.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.342+0000 7f11cbfff700 1 --2- 172.21.15.16:0/2002621555 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f11b806ace0 0x7f11b806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:21.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.342+0000 7f11c8ff9700 1 -- 172.21.15.16:0/2002621555 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f11c408ae30 con 0x7f11cc157190 2023-05-20T01:25:21.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.343+0000 7f11cbfff700 1 --2- 172.21.15.16:0/2002621555 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f11b806ace0 0x7f11b806d190 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f11bc007a30 tx=0x7f11bc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:21.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.347+0000 7f11c8ff9700 1 -- 172.21.15.16:0/2002621555 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f11c4055e30 con 0x7f11cc157190 2023-05-20T01:25:21.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.541+0000 7f11d27ee700 1 -- 172.21.15.16:0/2002621555 --> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f11cc0747b0 con 0x7f11b806ace0 2023-05-20T01:25:21.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.550+0000 7f11c8ff9700 1 -- 172.21.15.16:0/2002621555 <== mgr.4104 v2:172.21.15.16:6832/3817237998 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311684 (secure 0 0 0) 0x7f11cc0747b0 con 0x7f11b806ace0 2023-05-20T01:25:21.557 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:25:21.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.554+0000 7f11d27ee700 1 -- 172.21.15.16:0/2002621555 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f11b806ace0 msgr2=0x7f11b806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:21.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.554+0000 7f11d27ee700 1 --2- 172.21.15.16:0/2002621555 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f11b806ace0 0x7f11b806d190 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f11bc007a30 tx=0x7f11bc008040 comp rx=0 tx=0).stop 2023-05-20T01:25:21.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.554+0000 7f11d27ee700 1 -- 172.21.15.16:0/2002621555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11cc157190 msgr2=0x7f11cc15bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:21.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.554+0000 7f11d27ee700 1 --2- 172.21.15.16:0/2002621555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11cc157190 0x7f11cc15bd50 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f11cc0d0f50 tx=0x7f11c4004ad0 comp rx=0 tx=0).stop 2023-05-20T01:25:21.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.554+0000 7f11d27ee700 1 -- 172.21.15.16:0/2002621555 shutdown_connections 2023-05-20T01:25:21.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.554+0000 7f11d27ee700 1 --2- 172.21.15.16:0/2002621555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11cc157190 0x7f11cc15bd50 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:21.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.554+0000 7f11d27ee700 1 --2- 172.21.15.16:0/2002621555 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f11cc071ad0 0x7f11cc0c6520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:21.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.554+0000 7f11d27ee700 1 --2- 172.21.15.16:0/2002621555 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f11b806ace0 0x7f11b806d190 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:21.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.554+0000 7f11d27ee700 1 --2- 172.21.15.16:0/2002621555 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f11cc0cccf0 0x7f11cc1484d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:21.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.554+0000 7f11d27ee700 1 -- 172.21.15.16:0/2002621555 >> 172.21.15.16:0/2002621555 conn(0x7f11cc0cffb0 msgr2=0x7f11cc0d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:21.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.554+0000 7f11d27ee700 1 -- 172.21.15.16:0/2002621555 shutdown_connections 2023-05-20T01:25:21.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.554+0000 7f11d27ee700 1 -- 172.21.15.16:0/2002621555 wait complete. 2023-05-20T01:25:21.561 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2023-05-20T01:25:21.570 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":352,"stamp":"2023-05-20T01:25:21.435555+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":96672,"kb_used_data":9632,"kb_used_omap":0,"kb_used_meta":86976,"kb_avail":754878048,"statfs":{"total":773094113280,"available":772995121152,"internally_reserved":0,"allocated":9863168,"data_stored":5535562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":89063424},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.445577"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:13.059738+0000","last_change":"2023-05-20T01:25:12.010861+0000","last_active":"2023-05-20T01:25:13.059738+0000","last_peered":"2023-05-20T01:25:13.059738+0000","last_clean":"2023-05-20T01:25:13.059738+0000","last_became_active":"2023-05-20T01:25:12.010462+0000","last_became_peered":"2023-05-20T01:25:12.010462+0000","last_unstale":"2023-05-20T01:25:13.059738+0000","last_undegraded":"2023-05-20T01:25:13.059738+0000","last_fullsized":"2023-05-20T01:25:13.059738+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:47:16.902590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:14.300358+0000","last_change":"2023-05-20T01:25:12.015916+0000","last_active":"2023-05-20T01:25:14.300358+0000","last_peered":"2023-05-20T01:25:14.300358+0000","last_clean":"2023-05-20T01:25:14.300358+0000","last_became_active":"2023-05-20T01:25:12.012714+0000","last_became_peered":"2023-05-20T01:25:12.012714+0000","last_unstale":"2023-05-20T01:25:14.300358+0000","last_undegraded":"2023-05-20T01:25:14.300358+0000","last_fullsized":"2023-05-20T01:25:14.300358+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:21:39.218741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:13.572745+0000","last_change":"2023-05-20T01:25:12.012942+0000","last_active":"2023-05-20T01:25:13.572745+0000","last_peered":"2023-05-20T01:25:13.572745+0000","last_clean":"2023-05-20T01:25:13.572745+0000","last_became_active":"2023-05-20T01:25:12.012644+0000","last_became_peered":"2023-05-20T01:25:12.012644+0000","last_unstale":"2023-05-20T01:25:13.572745+0000","last_undegraded":"2023-05-20T01:25:13.572745+0000","last_fullsized":"2023-05-20T01:25:13.572745+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:34:19.045913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:12.997404+0000","last_change":"2023-05-20T01:25:12.022797+0000","last_active":"2023-05-20T01:25:12.997404+0000","last_peered":"2023-05-20T01:25:12.997404+0000","last_clean":"2023-05-20T01:25:12.997404+0000","last_became_active":"2023-05-20T01:25:12.022426+0000","last_became_peered":"2023-05-20T01:25:12.022426+0000","last_unstale":"2023-05-20T01:25:12.997404+0000","last_undegraded":"2023-05-20T01:25:12.997404+0000","last_fullsized":"2023-05-20T01:25:12.997404+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:48:31.993109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:12.997603+0000","last_change":"2023-05-20T01:25:12.015355+0000","last_active":"2023-05-20T01:25:12.997603+0000","last_peered":"2023-05-20T01:25:12.997603+0000","last_clean":"2023-05-20T01:25:12.997603+0000","last_became_active":"2023-05-20T01:25:12.015075+0000","last_became_peered":"2023-05-20T01:25:12.015075+0000","last_unstale":"2023-05-20T01:25:12.997603+0000","last_undegraded":"2023-05-20T01:25:12.997603+0000","last_fullsized":"2023-05-20T01:25:12.997603+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:51:13.117657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:13.804021+0000","last_change":"2023-05-20T01:25:12.022136+0000","last_active":"2023-05-20T01:25:13.804021+0000","last_peered":"2023-05-20T01:25:13.804021+0000","last_clean":"2023-05-20T01:25:13.804021+0000","last_became_active":"2023-05-20T01:25:12.021786+0000","last_became_peered":"2023-05-20T01:25:12.021786+0000","last_unstale":"2023-05-20T01:25:13.804021+0000","last_undegraded":"2023-05-20T01:25:13.804021+0000","last_fullsized":"2023-05-20T01:25:13.804021+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:32:22.089588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:12.996987+0000","last_change":"2023-05-20T01:25:12.007895+0000","last_active":"2023-05-20T01:25:12.996987+0000","last_peered":"2023-05-20T01:25:12.996987+0000","last_clean":"2023-05-20T01:25:12.996987+0000","last_became_active":"2023-05-20T01:25:12.007519+0000","last_became_peered":"2023-05-20T01:25:12.007519+0000","last_unstale":"2023-05-20T01:25:12.996987+0000","last_undegraded":"2023-05-20T01:25:12.996987+0000","last_fullsized":"2023-05-20T01:25:12.996987+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:24:52.476098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:13.803950+0000","last_change":"2023-05-20T01:25:12.016456+0000","last_active":"2023-05-20T01:25:13.803950+0000","last_peered":"2023-05-20T01:25:13.803950+0000","last_clean":"2023-05-20T01:25:13.803950+0000","last_became_active":"2023-05-20T01:25:12.016080+0000","last_became_peered":"2023-05-20T01:25:12.016080+0000","last_unstale":"2023-05-20T01:25:13.803950+0000","last_undegraded":"2023-05-20T01:25:13.803950+0000","last_fullsized":"2023-05-20T01:25:13.803950+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T01:54:04.144226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:13.804161+0000","last_change":"2023-05-20T01:25:12.016224+0000","last_active":"2023-05-20T01:25:13.804161+0000","last_peered":"2023-05-20T01:25:13.804161+0000","last_clean":"2023-05-20T01:25:13.804161+0000","last_became_active":"2023-05-20T01:25:12.015944+0000","last_became_peered":"2023-05-20T01:25:12.015944+0000","last_unstale":"2023-05-20T01:25:13.804161+0000","last_undegraded":"2023-05-20T01:25:13.804161+0000","last_fullsized":"2023-05-20T01:25:13.804161+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:32:29.054378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:13.059787+0000","last_change":"2023-05-20T01:25:12.006466+0000","last_active":"2023-05-20T01:25:13.059787+0000","last_peered":"2023-05-20T01:25:13.059787+0000","last_clean":"2023-05-20T01:25:13.059787+0000","last_became_active":"2023-05-20T01:25:12.006231+0000","last_became_peered":"2023-05-20T01:25:12.006231+0000","last_unstale":"2023-05-20T01:25:13.059787+0000","last_undegraded":"2023-05-20T01:25:13.059787+0000","last_fullsized":"2023-05-20T01:25:13.059787+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:46:57.605709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:14.300592+0000","last_change":"2023-05-20T01:25:12.013977+0000","last_active":"2023-05-20T01:25:14.300592+0000","last_peered":"2023-05-20T01:25:14.300592+0000","last_clean":"2023-05-20T01:25:14.300592+0000","last_became_active":"2023-05-20T01:25:12.011695+0000","last_became_peered":"2023-05-20T01:25:12.011695+0000","last_unstale":"2023-05-20T01:25:14.300592+0000","last_undegraded":"2023-05-20T01:25:14.300592+0000","last_fullsized":"2023-05-20T01:25:14.300592+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:50:31.388498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:13.060161+0000","last_change":"2023-05-20T01:25:12.007183+0000","last_active":"2023-05-20T01:25:13.060161+0000","last_peered":"2023-05-20T01:25:13.060161+0000","last_clean":"2023-05-20T01:25:13.060161+0000","last_became_active":"2023-05-20T01:25:12.006930+0000","last_became_peered":"2023-05-20T01:25:12.006930+0000","last_unstale":"2023-05-20T01:25:13.060161+0000","last_undegraded":"2023-05-20T01:25:13.060161+0000","last_fullsized":"2023-05-20T01:25:13.060161+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:46:12.148885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:13.572651+0000","last_change":"2023-05-20T01:25:12.002684+0000","last_active":"2023-05-20T01:25:13.572651+0000","last_peered":"2023-05-20T01:25:13.572651+0000","last_clean":"2023-05-20T01:25:13.572651+0000","last_became_active":"2023-05-20T01:25:12.002422+0000","last_became_peered":"2023-05-20T01:25:12.002422+0000","last_unstale":"2023-05-20T01:25:13.572651+0000","last_undegraded":"2023-05-20T01:25:13.572651+0000","last_fullsized":"2023-05-20T01:25:13.572651+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:58:00.068009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:12.997242+0000","last_change":"2023-05-20T01:25:12.017371+0000","last_active":"2023-05-20T01:25:12.997242+0000","last_peered":"2023-05-20T01:25:12.997242+0000","last_clean":"2023-05-20T01:25:12.997242+0000","last_became_active":"2023-05-20T01:25:12.017036+0000","last_became_peered":"2023-05-20T01:25:12.017036+0000","last_unstale":"2023-05-20T01:25:12.997242+0000","last_undegraded":"2023-05-20T01:25:12.997242+0000","last_fullsized":"2023-05-20T01:25:12.997242+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:31:18.466752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:12.995661+0000","last_change":"2023-05-20T01:25:11.996397+0000","last_active":"2023-05-20T01:25:12.995661+0000","last_peered":"2023-05-20T01:25:12.995661+0000","last_clean":"2023-05-20T01:25:12.995661+0000","last_became_active":"2023-05-20T01:25:11.996130+0000","last_became_peered":"2023-05-20T01:25:11.996130+0000","last_unstale":"2023-05-20T01:25:12.995661+0000","last_undegraded":"2023-05-20T01:25:12.995661+0000","last_fullsized":"2023-05-20T01:25:12.995661+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:36:05.566447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:12.997081+0000","last_change":"2023-05-20T01:25:12.007726+0000","last_active":"2023-05-20T01:25:12.997081+0000","last_peered":"2023-05-20T01:25:12.997081+0000","last_clean":"2023-05-20T01:25:12.997081+0000","last_became_active":"2023-05-20T01:25:12.007310+0000","last_became_peered":"2023-05-20T01:25:12.007310+0000","last_unstale":"2023-05-20T01:25:12.997081+0000","last_undegraded":"2023-05-20T01:25:12.997081+0000","last_fullsized":"2023-05-20T01:25:12.997081+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:06:16.779810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:12.997668+0000","last_change":"2023-05-20T01:25:12.008442+0000","last_active":"2023-05-20T01:25:12.997668+0000","last_peered":"2023-05-20T01:25:12.997668+0000","last_clean":"2023-05-20T01:25:12.997668+0000","last_became_active":"2023-05-20T01:25:12.008160+0000","last_became_peered":"2023-05-20T01:25:12.008160+0000","last_unstale":"2023-05-20T01:25:12.997668+0000","last_undegraded":"2023-05-20T01:25:12.997668+0000","last_fullsized":"2023-05-20T01:25:12.997668+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:33:35.277012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:12.997210+0000","last_change":"2023-05-20T01:25:12.014908+0000","last_active":"2023-05-20T01:25:12.997210+0000","last_peered":"2023-05-20T01:25:12.997210+0000","last_clean":"2023-05-20T01:25:12.997210+0000","last_became_active":"2023-05-20T01:25:12.014637+0000","last_became_peered":"2023-05-20T01:25:12.014637+0000","last_unstale":"2023-05-20T01:25:12.997210+0000","last_undegraded":"2023-05-20T01:25:12.997210+0000","last_fullsized":"2023-05-20T01:25:12.997210+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:06:03.483921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:13.174222+0000","last_change":"2023-05-20T01:25:12.012914+0000","last_active":"2023-05-20T01:25:13.174222+0000","last_peered":"2023-05-20T01:25:13.174222+0000","last_clean":"2023-05-20T01:25:13.174222+0000","last_became_active":"2023-05-20T01:25:12.012607+0000","last_became_peered":"2023-05-20T01:25:12.012607+0000","last_unstale":"2023-05-20T01:25:13.174222+0000","last_undegraded":"2023-05-20T01:25:13.174222+0000","last_fullsized":"2023-05-20T01:25:13.174222+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:48:48.891631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:12.997616+0000","last_change":"2023-05-20T01:25:12.016923+0000","last_active":"2023-05-20T01:25:12.997616+0000","last_peered":"2023-05-20T01:25:12.997616+0000","last_clean":"2023-05-20T01:25:12.997616+0000","last_became_active":"2023-05-20T01:25:12.016657+0000","last_became_peered":"2023-05-20T01:25:12.016657+0000","last_unstale":"2023-05-20T01:25:12.997616+0000","last_undegraded":"2023-05-20T01:25:12.997616+0000","last_fullsized":"2023-05-20T01:25:12.997616+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T13:17:09.147871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:12.997303+0000","last_change":"2023-05-20T01:25:12.016842+0000","last_active":"2023-05-20T01:25:12.997303+0000","last_peered":"2023-05-20T01:25:12.997303+0000","last_clean":"2023-05-20T01:25:12.997303+0000","last_became_active":"2023-05-20T01:25:12.016589+0000","last_became_peered":"2023-05-20T01:25:12.016589+0000","last_unstale":"2023-05-20T01:25:12.997303+0000","last_undegraded":"2023-05-20T01:25:12.997303+0000","last_fullsized":"2023-05-20T01:25:12.997303+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:05:16.230863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:14.300444+0000","last_change":"2023-05-20T01:25:12.021437+0000","last_active":"2023-05-20T01:25:14.300444+0000","last_peered":"2023-05-20T01:25:14.300444+0000","last_clean":"2023-05-20T01:25:14.300444+0000","last_became_active":"2023-05-20T01:25:12.021161+0000","last_became_peered":"2023-05-20T01:25:12.021161+0000","last_unstale":"2023-05-20T01:25:14.300444+0000","last_undegraded":"2023-05-20T01:25:14.300444+0000","last_fullsized":"2023-05-20T01:25:14.300444+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:51:41.067097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:13.059427+0000","last_change":"2023-05-20T01:25:12.014864+0000","last_active":"2023-05-20T01:25:13.059427+0000","last_peered":"2023-05-20T01:25:13.059427+0000","last_clean":"2023-05-20T01:25:13.059427+0000","last_became_active":"2023-05-20T01:25:12.014584+0000","last_became_peered":"2023-05-20T01:25:12.014584+0000","last_unstale":"2023-05-20T01:25:13.059427+0000","last_undegraded":"2023-05-20T01:25:13.059427+0000","last_fullsized":"2023-05-20T01:25:13.059427+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:56:41.729759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:12.996931+0000","last_change":"2023-05-20T01:25:12.019279+0000","last_active":"2023-05-20T01:25:12.996931+0000","last_peered":"2023-05-20T01:25:12.996931+0000","last_clean":"2023-05-20T01:25:12.996931+0000","last_became_active":"2023-05-20T01:25:12.018988+0000","last_became_peered":"2023-05-20T01:25:12.018988+0000","last_unstale":"2023-05-20T01:25:12.996931+0000","last_undegraded":"2023-05-20T01:25:12.996931+0000","last_fullsized":"2023-05-20T01:25:12.996931+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:04:33.650813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:14.300566+0000","last_change":"2023-05-20T01:25:12.015996+0000","last_active":"2023-05-20T01:25:14.300566+0000","last_peered":"2023-05-20T01:25:14.300566+0000","last_clean":"2023-05-20T01:25:14.300566+0000","last_became_active":"2023-05-20T01:25:12.015783+0000","last_became_peered":"2023-05-20T01:25:12.015783+0000","last_unstale":"2023-05-20T01:25:14.300566+0000","last_undegraded":"2023-05-20T01:25:14.300566+0000","last_fullsized":"2023-05-20T01:25:14.300566+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T01:28:37.267333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:13.573024+0000","last_change":"2023-05-20T01:25:12.016956+0000","last_active":"2023-05-20T01:25:13.573024+0000","last_peered":"2023-05-20T01:25:13.573024+0000","last_clean":"2023-05-20T01:25:13.573024+0000","last_became_active":"2023-05-20T01:25:12.016686+0000","last_became_peered":"2023-05-20T01:25:12.016686+0000","last_unstale":"2023-05-20T01:25:13.573024+0000","last_undegraded":"2023-05-20T01:25:13.573024+0000","last_fullsized":"2023-05-20T01:25:13.573024+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:23:02.561469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:12.997309+0000","last_change":"2023-05-20T01:25:12.002001+0000","last_active":"2023-05-20T01:25:12.997309+0000","last_peered":"2023-05-20T01:25:12.997309+0000","last_clean":"2023-05-20T01:25:12.997309+0000","last_became_active":"2023-05-20T01:25:12.001712+0000","last_became_peered":"2023-05-20T01:25:12.001712+0000","last_unstale":"2023-05-20T01:25:12.997309+0000","last_undegraded":"2023-05-20T01:25:12.997309+0000","last_fullsized":"2023-05-20T01:25:12.997309+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:04:26.356222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:13.059504+0000","last_change":"2023-05-20T01:25:12.010877+0000","last_active":"2023-05-20T01:25:13.059504+0000","last_peered":"2023-05-20T01:25:13.059504+0000","last_clean":"2023-05-20T01:25:13.059504+0000","last_became_active":"2023-05-20T01:25:12.010604+0000","last_became_peered":"2023-05-20T01:25:12.010604+0000","last_unstale":"2023-05-20T01:25:13.059504+0000","last_undegraded":"2023-05-20T01:25:13.059504+0000","last_fullsized":"2023-05-20T01:25:13.059504+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:02:40.263778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:14.298066+0000","last_change":"2023-05-20T01:25:12.012184+0000","last_active":"2023-05-20T01:25:14.298066+0000","last_peered":"2023-05-20T01:25:14.298066+0000","last_clean":"2023-05-20T01:25:14.298066+0000","last_became_active":"2023-05-20T01:25:12.011853+0000","last_became_peered":"2023-05-20T01:25:12.011853+0000","last_unstale":"2023-05-20T01:25:14.298066+0000","last_undegraded":"2023-05-20T01:25:14.298066+0000","last_fullsized":"2023-05-20T01:25:14.298066+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:15:58.796775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:14.300492+0000","last_change":"2023-05-20T01:25:12.012794+0000","last_active":"2023-05-20T01:25:14.300492+0000","last_peered":"2023-05-20T01:25:14.300492+0000","last_clean":"2023-05-20T01:25:14.300492+0000","last_became_active":"2023-05-20T01:25:12.011794+0000","last_became_peered":"2023-05-20T01:25:12.011794+0000","last_unstale":"2023-05-20T01:25:14.300492+0000","last_undegraded":"2023-05-20T01:25:14.300492+0000","last_fullsized":"2023-05-20T01:25:14.300492+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:57:36.203630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:12.997104+0000","last_change":"2023-05-20T01:25:12.013474+0000","last_active":"2023-05-20T01:25:12.997104+0000","last_peered":"2023-05-20T01:25:12.997104+0000","last_clean":"2023-05-20T01:25:12.997104+0000","last_became_active":"2023-05-20T01:25:12.013220+0000","last_became_peered":"2023-05-20T01:25:12.013220+0000","last_unstale":"2023-05-20T01:25:12.997104+0000","last_undegraded":"2023-05-20T01:25:12.997104+0000","last_fullsized":"2023-05-20T01:25:12.997104+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:23:43.680789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:13.059396+0000","last_change":"2023-05-20T01:25:12.016253+0000","last_active":"2023-05-20T01:25:13.059396+0000","last_peered":"2023-05-20T01:25:13.059396+0000","last_clean":"2023-05-20T01:25:13.059396+0000","last_became_active":"2023-05-20T01:25:12.015995+0000","last_became_peered":"2023-05-20T01:25:12.015995+0000","last_unstale":"2023-05-20T01:25:13.059396+0000","last_undegraded":"2023-05-20T01:25:13.059396+0000","last_fullsized":"2023-05-20T01:25:13.059396+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:44:53.263956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:12.997462+0000","last_change":"2023-05-20T01:25:12.007395+0000","last_active":"2023-05-20T01:25:12.997462+0000","last_peered":"2023-05-20T01:25:12.997462+0000","last_clean":"2023-05-20T01:25:12.997462+0000","last_became_active":"2023-05-20T01:25:12.007124+0000","last_became_peered":"2023-05-20T01:25:12.007124+0000","last_unstale":"2023-05-20T01:25:12.997462+0000","last_undegraded":"2023-05-20T01:25:12.997462+0000","last_fullsized":"2023-05-20T01:25:12.997462+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:16:00.645250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"87'4","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:13.060070+0000","last_change":"2023-05-20T01:25:11.580921+0000","last_active":"2023-05-20T01:25:13.060070+0000","last_peered":"2023-05-20T01:25:13.060070+0000","last_clean":"2023-05-20T01:25:13.060070+0000","last_became_active":"2023-05-20T01:25:11.580640+0000","last_became_peered":"2023-05-20T01:25:11.580640+0000","last_unstale":"2023-05-20T01:25:13.060070+0000","last_undegraded":"2023-05-20T01:25:13.060070+0000","last_fullsized":"2023-05-20T01:25:13.060070+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:48:47.884748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:13.060202+0000","last_change":"2023-05-20T01:25:12.012998+0000","last_active":"2023-05-20T01:25:13.060202+0000","last_peered":"2023-05-20T01:25:13.060202+0000","last_clean":"2023-05-20T01:25:13.060202+0000","last_became_active":"2023-05-20T01:25:12.012623+0000","last_became_peered":"2023-05-20T01:25:12.012623+0000","last_unstale":"2023-05-20T01:25:13.060202+0000","last_undegraded":"2023-05-20T01:25:13.060202+0000","last_fullsized":"2023-05-20T01:25:13.060202+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:57:45.442959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"87'1","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:12.997642+0000","last_change":"2023-05-20T01:25:11.569727+0000","last_active":"2023-05-20T01:25:12.997642+0000","last_peered":"2023-05-20T01:25:12.997642+0000","last_clean":"2023-05-20T01:25:12.997642+0000","last_became_active":"2023-05-20T01:25:11.569385+0000","last_became_peered":"2023-05-20T01:25:11.569385+0000","last_unstale":"2023-05-20T01:25:12.997642+0000","last_undegraded":"2023-05-20T01:25:12.997642+0000","last_fullsized":"2023-05-20T01:25:12.997642+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:45:07.423138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:13.059616+0000","last_change":"2023-05-20T01:25:12.021990+0000","last_active":"2023-05-20T01:25:13.059616+0000","last_peered":"2023-05-20T01:25:13.059616+0000","last_clean":"2023-05-20T01:25:13.059616+0000","last_became_active":"2023-05-20T01:25:12.021702+0000","last_became_peered":"2023-05-20T01:25:12.021702+0000","last_unstale":"2023-05-20T01:25:13.059616+0000","last_undegraded":"2023-05-20T01:25:13.059616+0000","last_fullsized":"2023-05-20T01:25:13.059616+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:13:45.855545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"87'2","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:13.174463+0000","last_change":"2023-05-20T01:25:11.580723+0000","last_active":"2023-05-20T01:25:13.174463+0000","last_peered":"2023-05-20T01:25:13.174463+0000","last_clean":"2023-05-20T01:25:13.174463+0000","last_became_active":"2023-05-20T01:25:11.580402+0000","last_became_peered":"2023-05-20T01:25:11.580402+0000","last_unstale":"2023-05-20T01:25:13.174463+0000","last_undegraded":"2023-05-20T01:25:13.174463+0000","last_fullsized":"2023-05-20T01:25:13.174463+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:46:11.142002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:13.572834+0000","last_change":"2023-05-20T01:25:12.008759+0000","last_active":"2023-05-20T01:25:13.572834+0000","last_peered":"2023-05-20T01:25:13.572834+0000","last_clean":"2023-05-20T01:25:13.572834+0000","last_became_active":"2023-05-20T01:25:12.008430+0000","last_became_peered":"2023-05-20T01:25:12.008430+0000","last_unstale":"2023-05-20T01:25:13.572834+0000","last_undegraded":"2023-05-20T01:25:13.572834+0000","last_fullsized":"2023-05-20T01:25:13.572834+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:34:03.861279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"87'2","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:13.174303+0000","last_change":"2023-05-20T01:25:11.588731+0000","last_active":"2023-05-20T01:25:13.174303+0000","last_peered":"2023-05-20T01:25:13.174303+0000","last_clean":"2023-05-20T01:25:13.174303+0000","last_became_active":"2023-05-20T01:25:11.586813+0000","last_became_peered":"2023-05-20T01:25:11.586813+0000","last_unstale":"2023-05-20T01:25:13.174303+0000","last_undegraded":"2023-05-20T01:25:13.174303+0000","last_fullsized":"2023-05-20T01:25:13.174303+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:30:17.004521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:14.298060+0000","last_change":"2023-05-20T01:25:12.012381+0000","last_active":"2023-05-20T01:25:14.298060+0000","last_peered":"2023-05-20T01:25:14.298060+0000","last_clean":"2023-05-20T01:25:14.298060+0000","last_became_active":"2023-05-20T01:25:12.012131+0000","last_became_peered":"2023-05-20T01:25:12.012131+0000","last_unstale":"2023-05-20T01:25:14.298060+0000","last_undegraded":"2023-05-20T01:25:14.298060+0000","last_fullsized":"2023-05-20T01:25:14.298060+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:33:06.332534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"87'1","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:12.997690+0000","last_change":"2023-05-20T01:25:11.584464+0000","last_active":"2023-05-20T01:25:12.997690+0000","last_peered":"2023-05-20T01:25:12.997690+0000","last_clean":"2023-05-20T01:25:12.997690+0000","last_became_active":"2023-05-20T01:25:11.584196+0000","last_became_peered":"2023-05-20T01:25:11.584196+0000","last_unstale":"2023-05-20T01:25:12.997690+0000","last_undegraded":"2023-05-20T01:25:12.997690+0000","last_fullsized":"2023-05-20T01:25:12.997690+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:54:24.464550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:12.997535+0000","last_change":"2023-05-20T01:25:12.008538+0000","last_active":"2023-05-20T01:25:12.997535+0000","last_peered":"2023-05-20T01:25:12.997535+0000","last_clean":"2023-05-20T01:25:12.997535+0000","last_became_active":"2023-05-20T01:25:12.008253+0000","last_became_peered":"2023-05-20T01:25:12.008253+0000","last_unstale":"2023-05-20T01:25:12.997535+0000","last_undegraded":"2023-05-20T01:25:12.997535+0000","last_fullsized":"2023-05-20T01:25:12.997535+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:29:22.170776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"87'1","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:12.997125+0000","last_change":"2023-05-20T01:25:11.576719+0000","last_active":"2023-05-20T01:25:12.997125+0000","last_peered":"2023-05-20T01:25:12.997125+0000","last_clean":"2023-05-20T01:25:12.997125+0000","last_became_active":"2023-05-20T01:25:11.576440+0000","last_became_peered":"2023-05-20T01:25:11.576440+0000","last_unstale":"2023-05-20T01:25:12.997125+0000","last_undegraded":"2023-05-20T01:25:12.997125+0000","last_fullsized":"2023-05-20T01:25:12.997125+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:45:01.382666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:13.572664+0000","last_change":"2023-05-20T01:25:12.018685+0000","last_active":"2023-05-20T01:25:13.572664+0000","last_peered":"2023-05-20T01:25:13.572664+0000","last_clean":"2023-05-20T01:25:13.572664+0000","last_became_active":"2023-05-20T01:25:12.018438+0000","last_became_peered":"2023-05-20T01:25:12.018438+0000","last_unstale":"2023-05-20T01:25:13.572664+0000","last_undegraded":"2023-05-20T01:25:13.572664+0000","last_fullsized":"2023-05-20T01:25:13.572664+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:33:35.277012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:13.173987+0000","last_change":"2023-05-20T01:25:11.581486+0000","last_active":"2023-05-20T01:25:13.173987+0000","last_peered":"2023-05-20T01:25:13.173987+0000","last_clean":"2023-05-20T01:25:13.173987+0000","last_became_active":"2023-05-20T01:25:11.581156+0000","last_became_peered":"2023-05-20T01:25:11.581156+0000","last_unstale":"2023-05-20T01:25:13.173987+0000","last_undegraded":"2023-05-20T01:25:13.173987+0000","last_fullsized":"2023-05-20T01:25:13.173987+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:35:33.948338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:12.995799+0000","last_change":"2023-05-20T01:25:11.996411+0000","last_active":"2023-05-20T01:25:12.995799+0000","last_peered":"2023-05-20T01:25:12.995799+0000","last_clean":"2023-05-20T01:25:12.995799+0000","last_became_active":"2023-05-20T01:25:11.996143+0000","last_became_peered":"2023-05-20T01:25:11.996143+0000","last_unstale":"2023-05-20T01:25:12.995799+0000","last_undegraded":"2023-05-20T01:25:12.995799+0000","last_fullsized":"2023-05-20T01:25:12.995799+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:00:32.352548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:14.297960+0000","last_change":"2023-05-20T01:25:11.581997+0000","last_active":"2023-05-20T01:25:14.297960+0000","last_peered":"2023-05-20T01:25:14.297960+0000","last_clean":"2023-05-20T01:25:14.297960+0000","last_became_active":"2023-05-20T01:25:11.581698+0000","last_became_peered":"2023-05-20T01:25:11.581698+0000","last_unstale":"2023-05-20T01:25:14.297960+0000","last_undegraded":"2023-05-20T01:25:14.297960+0000","last_fullsized":"2023-05-20T01:25:14.297960+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:05:19.570386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:12.996105+0000","last_change":"2023-05-20T01:25:12.013481+0000","last_active":"2023-05-20T01:25:12.996105+0000","last_peered":"2023-05-20T01:25:12.996105+0000","last_clean":"2023-05-20T01:25:12.996105+0000","last_became_active":"2023-05-20T01:25:12.013165+0000","last_became_peered":"2023-05-20T01:25:12.013165+0000","last_unstale":"2023-05-20T01:25:12.996105+0000","last_undegraded":"2023-05-20T01:25:12.996105+0000","last_fullsized":"2023-05-20T01:25:12.996105+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:03:38.637833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:13.059918+0000","last_change":"2023-05-20T01:25:11.577858+0000","last_active":"2023-05-20T01:25:13.059918+0000","last_peered":"2023-05-20T01:25:13.059918+0000","last_clean":"2023-05-20T01:25:13.059918+0000","last_became_active":"2023-05-20T01:25:11.577471+0000","last_became_peered":"2023-05-20T01:25:11.577471+0000","last_unstale":"2023-05-20T01:25:13.059918+0000","last_undegraded":"2023-05-20T01:25:13.059918+0000","last_fullsized":"2023-05-20T01:25:13.059918+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:53:16.752429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:13.804138+0000","last_change":"2023-05-20T01:25:12.022103+0000","last_active":"2023-05-20T01:25:13.804138+0000","last_peered":"2023-05-20T01:25:13.804138+0000","last_clean":"2023-05-20T01:25:13.804138+0000","last_became_active":"2023-05-20T01:25:12.021766+0000","last_became_peered":"2023-05-20T01:25:12.021766+0000","last_unstale":"2023-05-20T01:25:13.804138+0000","last_undegraded":"2023-05-20T01:25:13.804138+0000","last_fullsized":"2023-05-20T01:25:13.804138+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:12:14.338149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:13.059980+0000","last_change":"2023-05-20T01:25:11.580330+0000","last_active":"2023-05-20T01:25:13.059980+0000","last_peered":"2023-05-20T01:25:13.059980+0000","last_clean":"2023-05-20T01:25:13.059980+0000","last_became_active":"2023-05-20T01:25:11.580062+0000","last_became_peered":"2023-05-20T01:25:11.580062+0000","last_unstale":"2023-05-20T01:25:13.059980+0000","last_undegraded":"2023-05-20T01:25:13.059980+0000","last_fullsized":"2023-05-20T01:25:13.059980+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:23:31.146768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:13.174216+0000","last_change":"2023-05-20T01:25:12.014329+0000","last_active":"2023-05-20T01:25:13.174216+0000","last_peered":"2023-05-20T01:25:13.174216+0000","last_clean":"2023-05-20T01:25:13.174216+0000","last_became_active":"2023-05-20T01:25:12.014014+0000","last_became_peered":"2023-05-20T01:25:12.014014+0000","last_unstale":"2023-05-20T01:25:13.174216+0000","last_undegraded":"2023-05-20T01:25:13.174216+0000","last_fullsized":"2023-05-20T01:25:13.174216+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:23:32.153651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"87'3","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:13.059910+0000","last_change":"2023-05-20T01:25:11.584259+0000","last_active":"2023-05-20T01:25:13.059910+0000","last_peered":"2023-05-20T01:25:13.059910+0000","last_clean":"2023-05-20T01:25:13.059910+0000","last_became_active":"2023-05-20T01:25:11.583981+0000","last_became_peered":"2023-05-20T01:25:11.583981+0000","last_unstale":"2023-05-20T01:25:13.059910+0000","last_undegraded":"2023-05-20T01:25:13.059910+0000","last_fullsized":"2023-05-20T01:25:13.059910+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T13:09:54.839617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:12.997368+0000","last_change":"2023-05-20T01:25:12.013250+0000","last_active":"2023-05-20T01:25:12.997368+0000","last_peered":"2023-05-20T01:25:12.997368+0000","last_clean":"2023-05-20T01:25:12.997368+0000","last_became_active":"2023-05-20T01:25:12.012905+0000","last_became_peered":"2023-05-20T01:25:12.012905+0000","last_unstale":"2023-05-20T01:25:12.997368+0000","last_undegraded":"2023-05-20T01:25:12.997368+0000","last_fullsized":"2023-05-20T01:25:12.997368+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:30:46.395594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"87'6","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:13.804199+0000","last_change":"2023-05-20T01:25:11.579260+0000","last_active":"2023-05-20T01:25:13.804199+0000","last_peered":"2023-05-20T01:25:13.804199+0000","last_clean":"2023-05-20T01:25:13.804199+0000","last_became_active":"2023-05-20T01:25:11.578988+0000","last_became_peered":"2023-05-20T01:25:11.578988+0000","last_unstale":"2023-05-20T01:25:13.804199+0000","last_undegraded":"2023-05-20T01:25:13.804199+0000","last_fullsized":"2023-05-20T01:25:13.804199+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:11:19.510387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:13.174017+0000","last_change":"2023-05-20T01:25:12.021021+0000","last_active":"2023-05-20T01:25:13.174017+0000","last_peered":"2023-05-20T01:25:13.174017+0000","last_clean":"2023-05-20T01:25:13.174017+0000","last_became_active":"2023-05-20T01:25:12.020666+0000","last_became_peered":"2023-05-20T01:25:12.020666+0000","last_unstale":"2023-05-20T01:25:13.174017+0000","last_undegraded":"2023-05-20T01:25:13.174017+0000","last_fullsized":"2023-05-20T01:25:13.174017+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T13:09:55.846499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"87'2","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:14.298078+0000","last_change":"2023-05-20T01:25:11.585058+0000","last_active":"2023-05-20T01:25:14.298078+0000","last_peered":"2023-05-20T01:25:14.298078+0000","last_clean":"2023-05-20T01:25:14.298078+0000","last_became_active":"2023-05-20T01:25:11.584753+0000","last_became_peered":"2023-05-20T01:25:11.584753+0000","last_unstale":"2023-05-20T01:25:14.298078+0000","last_undegraded":"2023-05-20T01:25:14.298078+0000","last_fullsized":"2023-05-20T01:25:14.298078+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:58:27.539898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:13.572692+0000","last_change":"2023-05-20T01:25:12.016142+0000","last_active":"2023-05-20T01:25:13.572692+0000","last_peered":"2023-05-20T01:25:13.572692+0000","last_clean":"2023-05-20T01:25:13.572692+0000","last_became_active":"2023-05-20T01:25:12.015860+0000","last_became_peered":"2023-05-20T01:25:12.015860+0000","last_unstale":"2023-05-20T01:25:13.572692+0000","last_undegraded":"2023-05-20T01:25:13.572692+0000","last_fullsized":"2023-05-20T01:25:13.572692+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:31:51.281533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:12.995870+0000","last_change":"2023-05-20T01:25:11.585536+0000","last_active":"2023-05-20T01:25:12.995870+0000","last_peered":"2023-05-20T01:25:12.995870+0000","last_clean":"2023-05-20T01:25:12.995870+0000","last_became_active":"2023-05-20T01:25:11.585257+0000","last_became_peered":"2023-05-20T01:25:11.585257+0000","last_unstale":"2023-05-20T01:25:12.995870+0000","last_undegraded":"2023-05-20T01:25:12.995870+0000","last_fullsized":"2023-05-20T01:25:12.995870+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:11:13.264714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:14.297964+0000","last_change":"2023-05-20T01:25:12.012341+0000","last_active":"2023-05-20T01:25:14.297964+0000","last_peered":"2023-05-20T01:25:14.297964+0000","last_clean":"2023-05-20T01:25:14.297964+0000","last_became_active":"2023-05-20T01:25:12.012069+0000","last_became_peered":"2023-05-20T01:25:12.012069+0000","last_unstale":"2023-05-20T01:25:14.297964+0000","last_undegraded":"2023-05-20T01:25:14.297964+0000","last_fullsized":"2023-05-20T01:25:14.297964+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:28:31.900808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:13.803900+0000","last_change":"2023-05-20T01:25:11.580046+0000","last_active":"2023-05-20T01:25:13.803900+0000","last_peered":"2023-05-20T01:25:13.803900+0000","last_clean":"2023-05-20T01:25:13.803900+0000","last_became_active":"2023-05-20T01:25:11.579830+0000","last_became_peered":"2023-05-20T01:25:11.579830+0000","last_unstale":"2023-05-20T01:25:13.803900+0000","last_undegraded":"2023-05-20T01:25:13.803900+0000","last_fullsized":"2023-05-20T01:25:13.803900+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:26:47.257707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:14.297873+0000","last_change":"2023-05-20T01:25:12.008915+0000","last_active":"2023-05-20T01:25:14.297873+0000","last_peered":"2023-05-20T01:25:14.297873+0000","last_clean":"2023-05-20T01:25:14.297873+0000","last_became_active":"2023-05-20T01:25:12.008720+0000","last_became_peered":"2023-05-20T01:25:12.008720+0000","last_unstale":"2023-05-20T01:25:14.297873+0000","last_undegraded":"2023-05-20T01:25:14.297873+0000","last_fullsized":"2023-05-20T01:25:14.297873+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:10:37.482767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"87'8","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:13.173906+0000","last_change":"2023-05-20T01:25:11.581483+0000","last_active":"2023-05-20T01:25:13.173906+0000","last_peered":"2023-05-20T01:25:13.173906+0000","last_clean":"2023-05-20T01:25:13.173906+0000","last_became_active":"2023-05-20T01:25:11.581189+0000","last_became_peered":"2023-05-20T01:25:11.581189+0000","last_unstale":"2023-05-20T01:25:13.173906+0000","last_undegraded":"2023-05-20T01:25:13.173906+0000","last_fullsized":"2023-05-20T01:25:13.173906+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:46:56.598827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:12.997562+0000","last_change":"2023-05-20T01:25:12.009760+0000","last_active":"2023-05-20T01:25:12.997562+0000","last_peered":"2023-05-20T01:25:12.997562+0000","last_clean":"2023-05-20T01:25:12.997562+0000","last_became_active":"2023-05-20T01:25:12.009426+0000","last_became_peered":"2023-05-20T01:25:12.009426+0000","last_unstale":"2023-05-20T01:25:12.997562+0000","last_undegraded":"2023-05-20T01:25:12.997562+0000","last_fullsized":"2023-05-20T01:25:12.997562+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:20:12.600985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"87'2","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:14.300608+0000","last_change":"2023-05-20T01:25:11.585068+0000","last_active":"2023-05-20T01:25:14.300608+0000","last_peered":"2023-05-20T01:25:14.300608+0000","last_clean":"2023-05-20T01:25:14.300608+0000","last_became_active":"2023-05-20T01:25:11.584758+0000","last_became_peered":"2023-05-20T01:25:11.584758+0000","last_unstale":"2023-05-20T01:25:14.300608+0000","last_undegraded":"2023-05-20T01:25:14.300608+0000","last_fullsized":"2023-05-20T01:25:14.300608+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:21:03.608718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:13.573163+0000","last_change":"2023-05-20T01:25:12.011123+0000","last_active":"2023-05-20T01:25:13.573163+0000","last_peered":"2023-05-20T01:25:13.573163+0000","last_clean":"2023-05-20T01:25:13.573163+0000","last_became_active":"2023-05-20T01:25:12.010912+0000","last_became_peered":"2023-05-20T01:25:12.010912+0000","last_unstale":"2023-05-20T01:25:13.573163+0000","last_undegraded":"2023-05-20T01:25:13.573163+0000","last_fullsized":"2023-05-20T01:25:13.573163+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:51:01.728862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"88'1","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:15.435525+0000","last_change":"2023-05-20T01:25:11.586764+0000","last_active":"2023-05-20T01:25:15.435525+0000","last_peered":"2023-05-20T01:25:15.435525+0000","last_clean":"2023-05-20T01:25:15.435525+0000","last_became_active":"2023-05-20T01:25:11.586458+0000","last_became_peered":"2023-05-20T01:25:11.586458+0000","last_unstale":"2023-05-20T01:25:15.435525+0000","last_undegraded":"2023-05-20T01:25:15.435525+0000","last_fullsized":"2023-05-20T01:25:15.435525+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:33:40.069024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:13.059684+0000","last_change":"2023-05-20T01:25:12.009977+0000","last_active":"2023-05-20T01:25:13.059684+0000","last_peered":"2023-05-20T01:25:13.059684+0000","last_clean":"2023-05-20T01:25:13.059684+0000","last_became_active":"2023-05-20T01:25:12.009706+0000","last_became_peered":"2023-05-20T01:25:12.009706+0000","last_unstale":"2023-05-20T01:25:13.059684+0000","last_undegraded":"2023-05-20T01:25:13.059684+0000","last_fullsized":"2023-05-20T01:25:13.059684+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:09:50.800709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:13.059704+0000","last_change":"2023-05-20T01:25:11.580023+0000","last_active":"2023-05-20T01:25:13.059704+0000","last_peered":"2023-05-20T01:25:13.059704+0000","last_clean":"2023-05-20T01:25:13.059704+0000","last_became_active":"2023-05-20T01:25:11.579731+0000","last_became_peered":"2023-05-20T01:25:11.579731+0000","last_unstale":"2023-05-20T01:25:13.059704+0000","last_undegraded":"2023-05-20T01:25:13.059704+0000","last_fullsized":"2023-05-20T01:25:13.059704+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:50:07.905717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:12.997625+0000","last_change":"2023-05-20T01:25:12.013914+0000","last_active":"2023-05-20T01:25:12.997625+0000","last_peered":"2023-05-20T01:25:12.997625+0000","last_clean":"2023-05-20T01:25:12.997625+0000","last_became_active":"2023-05-20T01:25:12.013647+0000","last_became_peered":"2023-05-20T01:25:12.013647+0000","last_unstale":"2023-05-20T01:25:12.997625+0000","last_undegraded":"2023-05-20T01:25:12.997625+0000","last_fullsized":"2023-05-20T01:25:12.997625+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:30:05.135875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:13.060142+0000","last_change":"2023-05-20T01:25:11.579261+0000","last_active":"2023-05-20T01:25:13.060142+0000","last_peered":"2023-05-20T01:25:13.060142+0000","last_clean":"2023-05-20T01:25:13.060142+0000","last_became_active":"2023-05-20T01:25:11.578954+0000","last_became_peered":"2023-05-20T01:25:11.578954+0000","last_unstale":"2023-05-20T01:25:13.060142+0000","last_undegraded":"2023-05-20T01:25:13.060142+0000","last_fullsized":"2023-05-20T01:25:13.060142+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T01:55:32.923901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:13.803987+0000","last_change":"2023-05-20T01:25:12.007820+0000","last_active":"2023-05-20T01:25:13.803987+0000","last_peered":"2023-05-20T01:25:13.803987+0000","last_clean":"2023-05-20T01:25:13.803987+0000","last_became_active":"2023-05-20T01:25:12.007537+0000","last_became_peered":"2023-05-20T01:25:12.007537+0000","last_unstale":"2023-05-20T01:25:13.803987+0000","last_undegraded":"2023-05-20T01:25:13.803987+0000","last_fullsized":"2023-05-20T01:25:13.803987+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:11:21.233757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:13.174216+0000","last_change":"2023-05-20T01:25:11.579108+0000","last_active":"2023-05-20T01:25:13.174216+0000","last_peered":"2023-05-20T01:25:13.174216+0000","last_clean":"2023-05-20T01:25:13.174216+0000","last_became_active":"2023-05-20T01:25:11.578740+0000","last_became_peered":"2023-05-20T01:25:11.578740+0000","last_unstale":"2023-05-20T01:25:13.174216+0000","last_undegraded":"2023-05-20T01:25:13.174216+0000","last_fullsized":"2023-05-20T01:25:13.174216+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T13:19:29.600229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:13.174195+0000","last_change":"2023-05-20T01:25:12.022288+0000","last_active":"2023-05-20T01:25:13.174195+0000","last_peered":"2023-05-20T01:25:13.174195+0000","last_clean":"2023-05-20T01:25:13.174195+0000","last_became_active":"2023-05-20T01:25:12.020042+0000","last_became_peered":"2023-05-20T01:25:12.020042+0000","last_unstale":"2023-05-20T01:25:13.174195+0000","last_undegraded":"2023-05-20T01:25:13.174195+0000","last_fullsized":"2023-05-20T01:25:13.174195+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:53:17.759312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:12.997395+0000","last_change":"2023-05-20T01:25:11.574489+0000","last_active":"2023-05-20T01:25:12.997395+0000","last_peered":"2023-05-20T01:25:12.997395+0000","last_clean":"2023-05-20T01:25:12.997395+0000","last_became_active":"2023-05-20T01:25:11.574182+0000","last_became_peered":"2023-05-20T01:25:11.574182+0000","last_unstale":"2023-05-20T01:25:12.997395+0000","last_undegraded":"2023-05-20T01:25:12.997395+0000","last_fullsized":"2023-05-20T01:25:12.997395+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:53:41.459514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:13.572588+0000","last_change":"2023-05-20T01:25:12.012592+0000","last_active":"2023-05-20T01:25:13.572588+0000","last_peered":"2023-05-20T01:25:13.572588+0000","last_clean":"2023-05-20T01:25:13.572588+0000","last_became_active":"2023-05-20T01:25:12.012306+0000","last_became_peered":"2023-05-20T01:25:12.012306+0000","last_unstale":"2023-05-20T01:25:13.572588+0000","last_undegraded":"2023-05-20T01:25:13.572588+0000","last_fullsized":"2023-05-20T01:25:13.572588+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:54:25.471432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:13.059703+0000","last_change":"2023-05-20T01:25:11.578666+0000","last_active":"2023-05-20T01:25:13.059703+0000","last_peered":"2023-05-20T01:25:13.059703+0000","last_clean":"2023-05-20T01:25:13.059703+0000","last_became_active":"2023-05-20T01:25:11.577385+0000","last_became_peered":"2023-05-20T01:25:11.577385+0000","last_unstale":"2023-05-20T01:25:13.059703+0000","last_undegraded":"2023-05-20T01:25:13.059703+0000","last_fullsized":"2023-05-20T01:25:13.059703+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:00:47.634174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:13.059693+0000","last_change":"2023-05-20T01:25:12.009771+0000","last_active":"2023-05-20T01:25:13.059693+0000","last_peered":"2023-05-20T01:25:13.059693+0000","last_clean":"2023-05-20T01:25:13.059693+0000","last_became_active":"2023-05-20T01:25:12.009463+0000","last_became_peered":"2023-05-20T01:25:12.009463+0000","last_unstale":"2023-05-20T01:25:13.059693+0000","last_undegraded":"2023-05-20T01:25:13.059693+0000","last_fullsized":"2023-05-20T01:25:13.059693+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:42:51.410878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:12.997143+0000","last_change":"2023-05-20T01:25:11.578192+0000","last_active":"2023-05-20T01:25:12.997143+0000","last_peered":"2023-05-20T01:25:12.997143+0000","last_clean":"2023-05-20T01:25:12.997143+0000","last_became_active":"2023-05-20T01:25:11.577843+0000","last_became_peered":"2023-05-20T01:25:11.577843+0000","last_unstale":"2023-05-20T01:25:12.997143+0000","last_undegraded":"2023-05-20T01:25:12.997143+0000","last_fullsized":"2023-05-20T01:25:12.997143+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T01:37:44.036317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:13.803987+0000","last_change":"2023-05-20T01:25:12.016442+0000","last_active":"2023-05-20T01:25:13.803987+0000","last_peered":"2023-05-20T01:25:13.803987+0000","last_clean":"2023-05-20T01:25:13.803987+0000","last_became_active":"2023-05-20T01:25:12.016182+0000","last_became_peered":"2023-05-20T01:25:12.016182+0000","last_unstale":"2023-05-20T01:25:13.803987+0000","last_undegraded":"2023-05-20T01:25:13.803987+0000","last_fullsized":"2023-05-20T01:25:13.803987+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:31:37.871606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:12.995985+0000","last_change":"2023-05-20T01:25:11.581315+0000","last_active":"2023-05-20T01:25:12.995985+0000","last_peered":"2023-05-20T01:25:12.995985+0000","last_clean":"2023-05-20T01:25:12.995985+0000","last_became_active":"2023-05-20T01:25:11.581043+0000","last_became_peered":"2023-05-20T01:25:11.581043+0000","last_unstale":"2023-05-20T01:25:12.995985+0000","last_undegraded":"2023-05-20T01:25:12.995985+0000","last_fullsized":"2023-05-20T01:25:12.995985+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:55:54.205242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":151,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:12.997619+0000","last_change":"2023-05-20T01:16:23.289068+0000","last_active":"2023-05-20T01:25:12.997619+0000","last_peered":"2023-05-20T01:25:12.997619+0000","last_clean":"2023-05-20T01:25:12.997619+0000","last_became_active":"2023-05-20T01:16:23.288781+0000","last_became_peered":"2023-05-20T01:16:23.288781+0000","last_unstale":"2023-05-20T01:25:12.997619+0000","last_undegraded":"2023-05-20T01:25:12.997619+0000","last_fullsized":"2023-05-20T01:25:12.997619+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:21:48.543560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:12.996991+0000","last_change":"2023-05-20T01:25:12.015626+0000","last_active":"2023-05-20T01:25:12.996991+0000","last_peered":"2023-05-20T01:25:12.996991+0000","last_clean":"2023-05-20T01:25:12.996991+0000","last_became_active":"2023-05-20T01:25:12.015388+0000","last_became_peered":"2023-05-20T01:25:12.015388+0000","last_unstale":"2023-05-20T01:25:12.996991+0000","last_undegraded":"2023-05-20T01:25:12.996991+0000","last_fullsized":"2023-05-20T01:25:12.996991+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T01:46:01.851831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:12.997062+0000","last_change":"2023-05-20T01:25:11.582487+0000","last_active":"2023-05-20T01:25:12.997062+0000","last_peered":"2023-05-20T01:25:12.997062+0000","last_clean":"2023-05-20T01:25:12.997062+0000","last_became_active":"2023-05-20T01:25:11.582061+0000","last_became_peered":"2023-05-20T01:25:11.582061+0000","last_unstale":"2023-05-20T01:25:12.997062+0000","last_undegraded":"2023-05-20T01:25:12.997062+0000","last_fullsized":"2023-05-20T01:25:12.997062+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T13:22:10.696056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":157,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:12.995934+0000","last_change":"2023-05-20T01:16:22.829324+0000","last_active":"2023-05-20T01:25:12.995934+0000","last_peered":"2023-05-20T01:25:12.995934+0000","last_clean":"2023-05-20T01:25:12.995934+0000","last_became_active":"2023-05-20T01:16:22.828964+0000","last_became_peered":"2023-05-20T01:16:22.828964+0000","last_unstale":"2023-05-20T01:25:12.995934+0000","last_undegraded":"2023-05-20T01:25:12.995934+0000","last_fullsized":"2023-05-20T01:25:12.995934+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T01:22:38.911204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:13.060139+0000","last_change":"2023-05-20T01:25:12.010469+0000","last_active":"2023-05-20T01:25:13.060139+0000","last_peered":"2023-05-20T01:25:13.060139+0000","last_clean":"2023-05-20T01:25:13.060139+0000","last_became_active":"2023-05-20T01:25:12.010160+0000","last_became_peered":"2023-05-20T01:25:12.010160+0000","last_unstale":"2023-05-20T01:25:13.060139+0000","last_undegraded":"2023-05-20T01:25:13.060139+0000","last_fullsized":"2023-05-20T01:25:13.060139+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:29:00.306831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"87'2","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:13.174273+0000","last_change":"2023-05-20T01:25:11.581614+0000","last_active":"2023-05-20T01:25:13.174273+0000","last_peered":"2023-05-20T01:25:13.174273+0000","last_clean":"2023-05-20T01:25:13.174273+0000","last_became_active":"2023-05-20T01:25:11.581114+0000","last_became_peered":"2023-05-20T01:25:11.581114+0000","last_unstale":"2023-05-20T01:25:13.174273+0000","last_undegraded":"2023-05-20T01:25:13.174273+0000","last_fullsized":"2023-05-20T01:25:13.174273+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T01:55:32.923901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":143,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:14.300404+0000","last_change":"2023-05-20T01:16:22.834754+0000","last_active":"2023-05-20T01:25:14.300404+0000","last_peered":"2023-05-20T01:25:14.300404+0000","last_clean":"2023-05-20T01:25:14.300404+0000","last_became_active":"2023-05-20T01:16:22.834437+0000","last_became_peered":"2023-05-20T01:16:22.834437+0000","last_unstale":"2023-05-20T01:25:14.300404+0000","last_undegraded":"2023-05-20T01:25:14.300404+0000","last_fullsized":"2023-05-20T01:25:14.300404+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:45:10.965559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":279,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:13.572681+0000","last_change":"2023-05-20T01:16:22.834920+0000","last_active":"2023-05-20T01:25:13.572681+0000","last_peered":"2023-05-20T01:25:13.572681+0000","last_clean":"2023-05-20T01:25:13.572681+0000","last_became_active":"2023-05-20T01:16:22.833924+0000","last_became_peered":"2023-05-20T01:16:22.833924+0000","last_unstale":"2023-05-20T01:25:13.572681+0000","last_undegraded":"2023-05-20T01:25:13.572681+0000","last_fullsized":"2023-05-20T01:25:13.572681+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:43:02.732240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:13.573031+0000","last_change":"2023-05-20T01:25:12.006623+0000","last_active":"2023-05-20T01:25:13.573031+0000","last_peered":"2023-05-20T01:25:13.573031+0000","last_clean":"2023-05-20T01:25:13.573031+0000","last_became_active":"2023-05-20T01:25:12.006338+0000","last_became_peered":"2023-05-20T01:25:12.006338+0000","last_unstale":"2023-05-20T01:25:13.573031+0000","last_undegraded":"2023-05-20T01:25:13.573031+0000","last_fullsized":"2023-05-20T01:25:13.573031+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:34:13.585780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"87'1","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:13.059954+0000","last_change":"2023-05-20T01:25:11.578913+0000","last_active":"2023-05-20T01:25:13.059954+0000","last_peered":"2023-05-20T01:25:13.059954+0000","last_clean":"2023-05-20T01:25:13.059954+0000","last_became_active":"2023-05-20T01:25:11.578494+0000","last_became_peered":"2023-05-20T01:25:11.578494+0000","last_unstale":"2023-05-20T01:25:13.059954+0000","last_undegraded":"2023-05-20T01:25:13.059954+0000","last_fullsized":"2023-05-20T01:25:13.059954+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T13:19:29.600229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":129,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:13.572495+0000","last_change":"2023-05-20T01:16:23.290259+0000","last_active":"2023-05-20T01:25:13.572495+0000","last_peered":"2023-05-20T01:25:13.572495+0000","last_clean":"2023-05-20T01:25:13.572495+0000","last_became_active":"2023-05-20T01:16:23.289911+0000","last_became_peered":"2023-05-20T01:16:23.289911+0000","last_unstale":"2023-05-20T01:25:13.572495+0000","last_undegraded":"2023-05-20T01:25:13.572495+0000","last_fullsized":"2023-05-20T01:25:13.572495+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:21:48.543560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:14.300559+0000","last_change":"2023-05-20T01:25:12.021584+0000","last_active":"2023-05-20T01:25:14.300559+0000","last_peered":"2023-05-20T01:25:14.300559+0000","last_clean":"2023-05-20T01:25:14.300559+0000","last_became_active":"2023-05-20T01:25:12.021308+0000","last_became_peered":"2023-05-20T01:25:12.021308+0000","last_unstale":"2023-05-20T01:25:14.300559+0000","last_undegraded":"2023-05-20T01:25:14.300559+0000","last_fullsized":"2023-05-20T01:25:14.300559+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:09:36.643641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"87'1","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:13.572964+0000","last_change":"2023-05-20T01:25:11.586246+0000","last_active":"2023-05-20T01:25:13.572964+0000","last_peered":"2023-05-20T01:25:13.572964+0000","last_clean":"2023-05-20T01:25:13.572964+0000","last_became_active":"2023-05-20T01:25:11.585952+0000","last_became_peered":"2023-05-20T01:25:11.585952+0000","last_unstale":"2023-05-20T01:25:13.572964+0000","last_undegraded":"2023-05-20T01:25:13.572964+0000","last_fullsized":"2023-05-20T01:25:13.572964+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T13:16:44.672035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":129,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:13.572763+0000","last_change":"2023-05-20T01:16:23.290207+0000","last_active":"2023-05-20T01:25:13.572763+0000","last_peered":"2023-05-20T01:25:13.572763+0000","last_clean":"2023-05-20T01:25:13.572763+0000","last_became_active":"2023-05-20T01:16:23.289928+0000","last_became_peered":"2023-05-20T01:16:23.289928+0000","last_unstale":"2023-05-20T01:25:13.572763+0000","last_undegraded":"2023-05-20T01:25:13.572763+0000","last_fullsized":"2023-05-20T01:25:13.572763+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:48:37.292605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:12.997485+0000","last_change":"2023-05-20T01:25:12.014668+0000","last_active":"2023-05-20T01:25:12.997485+0000","last_peered":"2023-05-20T01:25:12.997485+0000","last_clean":"2023-05-20T01:25:12.997485+0000","last_became_active":"2023-05-20T01:25:12.014376+0000","last_became_peered":"2023-05-20T01:25:12.014376+0000","last_unstale":"2023-05-20T01:25:12.997485+0000","last_undegraded":"2023-05-20T01:25:12.997485+0000","last_fullsized":"2023-05-20T01:25:12.997485+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:16:14.399334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:13.174300+0000","last_change":"2023-05-20T01:25:11.586258+0000","last_active":"2023-05-20T01:25:13.174300+0000","last_peered":"2023-05-20T01:25:13.174300+0000","last_clean":"2023-05-20T01:25:13.174300+0000","last_became_active":"2023-05-20T01:25:11.585977+0000","last_became_peered":"2023-05-20T01:25:11.585977+0000","last_unstale":"2023-05-20T01:25:13.174300+0000","last_undegraded":"2023-05-20T01:25:13.174300+0000","last_fullsized":"2023-05-20T01:25:13.174300+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:28:59.299949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":129,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:13.804116+0000","last_change":"2023-05-20T01:16:23.290538+0000","last_active":"2023-05-20T01:25:13.804116+0000","last_peered":"2023-05-20T01:25:13.804116+0000","last_clean":"2023-05-20T01:25:13.804116+0000","last_became_active":"2023-05-20T01:16:23.290230+0000","last_became_peered":"2023-05-20T01:16:23.290230+0000","last_unstale":"2023-05-20T01:25:13.804116+0000","last_undegraded":"2023-05-20T01:25:13.804116+0000","last_fullsized":"2023-05-20T01:25:13.804116+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:53:47.907406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:12.997415+0000","last_change":"2023-05-20T01:25:12.017330+0000","last_active":"2023-05-20T01:25:12.997415+0000","last_peered":"2023-05-20T01:25:12.997415+0000","last_clean":"2023-05-20T01:25:12.997415+0000","last_became_active":"2023-05-20T01:25:12.016956+0000","last_became_peered":"2023-05-20T01:25:12.016956+0000","last_unstale":"2023-05-20T01:25:12.997415+0000","last_undegraded":"2023-05-20T01:25:12.997415+0000","last_fullsized":"2023-05-20T01:25:12.997415+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:35:47.348775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"87'2","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:12.995961+0000","last_change":"2023-05-20T01:25:11.579978+0000","last_active":"2023-05-20T01:25:12.995961+0000","last_peered":"2023-05-20T01:25:12.995961+0000","last_clean":"2023-05-20T01:25:12.995961+0000","last_became_active":"2023-05-20T01:25:11.579628+0000","last_became_peered":"2023-05-20T01:25:11.579628+0000","last_unstale":"2023-05-20T01:25:12.995961+0000","last_undegraded":"2023-05-20T01:25:12.995961+0000","last_fullsized":"2023-05-20T01:25:12.995961+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:12:09.573361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":135,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:13.060078+0000","last_change":"2023-05-20T01:16:23.290341+0000","last_active":"2023-05-20T01:25:13.060078+0000","last_peered":"2023-05-20T01:25:13.060078+0000","last_clean":"2023-05-20T01:25:13.060078+0000","last_became_active":"2023-05-20T01:16:23.290053+0000","last_became_peered":"2023-05-20T01:16:23.290053+0000","last_unstale":"2023-05-20T01:25:13.060078+0000","last_undegraded":"2023-05-20T01:25:13.060078+0000","last_fullsized":"2023-05-20T01:25:13.060078+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:21:28.264291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:13.804235+0000","last_change":"2023-05-20T01:25:12.006518+0000","last_active":"2023-05-20T01:25:13.804235+0000","last_peered":"2023-05-20T01:25:13.804235+0000","last_clean":"2023-05-20T01:25:13.804235+0000","last_became_active":"2023-05-20T01:25:12.006233+0000","last_became_peered":"2023-05-20T01:25:12.006233+0000","last_unstale":"2023-05-20T01:25:13.804235+0000","last_undegraded":"2023-05-20T01:25:13.804235+0000","last_fullsized":"2023-05-20T01:25:13.804235+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:11.552637+0000","last_clean_scrub_stamp":"2023-05-20T01:25:11.552637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:10:37.595259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:12.997472+0000","last_change":"2023-05-20T01:25:11.587195+0000","last_active":"2023-05-20T01:25:12.997472+0000","last_peered":"2023-05-20T01:25:12.997472+0000","last_clean":"2023-05-20T01:25:12.997472+0000","last_became_active":"2023-05-20T01:25:11.586759+0000","last_became_peered":"2023-05-20T01:25:11.586759+0000","last_unstale":"2023-05-20T01:25:12.997472+0000","last_undegraded":"2023-05-20T01:25:12.997472+0000","last_fullsized":"2023-05-20T01:25:12.997472+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:25:10.545755+0000","last_clean_scrub_stamp":"2023-05-20T01:25:10.545755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T01:39:36.762742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":135,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-05-20T01:25:13.059738+0000","last_change":"2023-05-20T01:16:22.834330+0000","last_active":"2023-05-20T01:25:13.059738+0000","last_peered":"2023-05-20T01:25:13.059738+0000","last_clean":"2023-05-20T01:25:13.059738+0000","last_became_active":"2023-05-20T01:16:22.833946+0000","last_became_peered":"2023-05-20T01:16:22.833946+0000","last_unstale":"2023-05-20T01:25:13.059738+0000","last_undegraded":"2023-05-20T01:25:13.059738+0000","last_fullsized":"2023-05-20T01:25:13.059738+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:26:45.208108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574960,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":11756,"kb_used_data":1636,"kb_used_omap":0,"kb_used_meta":10112,"kb_avail":94360084,"statfs":{"total":96636764160,"available":96624726016,"internally_reserved":0,"allocated":1675264,"data_stored":1134441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":10354688},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:24:34 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.47699999999999998,"15min":0.51400000000000001},"min":{"1min":0.26900000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.68799999999999994,"5min":0.73199999999999998,"15min":0.88200000000000001},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.503,"15min":0.54700000000000004},"min":{"1min":0.33100000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.63600000000000001,"5min":0.752,"15min":0.90000000000000002},"last":0.64600000000000002}]},{"osd":1,"last update":"Sat May 20 01:24:34 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.49299999999999999,"15min":0.75},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.628,"5min":0.70299999999999996,"15min":8.8879999999999999},"last":0.623},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.52200000000000002,"15min":0.78900000000000003},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.74199999999999999,"5min":0.75,"15min":8.875},"last":0.63}]},{"osd":2,"last update":"Sat May 20 01:24:34 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.503,"15min":0.73599999999999999},"min":{"1min":0.34000000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.65900000000000003,"5min":0.77100000000000002,"15min":8.9120000000000008},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.504,"15min":0.73899999999999999},"min":{"1min":0.35999999999999999,"5min":0.35499999999999998,"15min":0.35099999999999998},"max":{"1min":0.58999999999999997,"5min":0.72399999999999998,"15min":8.8520000000000003},"last":0.61199999999999999}]},{"osd":3,"last update":"Sat May 20 01:24:34 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.49399999999999999,"15min":0.79700000000000004},"min":{"1min":0.36399999999999999,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.60599999999999998,"5min":0.73799999999999999,"15min":10.884},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.55100000000000005,"15min":0.85099999999999998},"min":{"1min":0.40600000000000003,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":10.952},"last":0.52600000000000002}]},{"osd":4,"last update":"Sat May 20 01:24:34 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51100000000000001,"15min":0.51000000000000001},"min":{"1min":0.39200000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.66700000000000004,"5min":0.74299999999999999,"15min":0.77000000000000002},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.312},"max":{"1min":0.71599999999999997,"5min":0.78500000000000003,"15min":0.875},"last":0.501}]},{"osd":5,"last update":"Sat May 20 01:24:34 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.56299999999999994,"15min":0.54200000000000004},"min":{"1min":0.35999999999999999,"5min":0.35799999999999998,"15min":0.32300000000000001},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.57099999999999995,"15min":0.57599999999999996},"min":{"1min":0.35999999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.745,"5min":0.79600000000000004,"15min":0.80200000000000005},"last":0.42299999999999999}]},{"osd":6,"last update":"Sat May 20 01:24:34 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.59599999999999997,"15min":0.59499999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.378},"max":{"1min":0.72699999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.57799999999999996,"15min":0.58399999999999996},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.73399999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.439}]}]},{"osd":6,"up_from":13,"seq":55834574959,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10416,"kb_used_data":1064,"kb_used_omap":0,"kb_used_meta":9344,"kb_avail":94361424,"statfs":{"total":96636764160,"available":96626098176,"internally_reserved":0,"allocated":1089536,"data_stored":544697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9568256},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:24:36 2023","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.45400000000000001,"15min":0.47799999999999998},"min":{"1min":0.29099999999999998,"5min":0.28999999999999998,"15min":0.26900000000000002},"max":{"1min":0.55500000000000005,"5min":0.72699999999999998,"15min":0.77200000000000002},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.438,"5min":0.47199999999999998,"15min":0.51800000000000002},"min":{"1min":0.309,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.60299999999999998,"5min":0.70099999999999996,"15min":0.73999999999999999},"last":0.51400000000000001}]},{"osd":1,"last update":"Sat May 20 01:24:36 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.495,"15min":0.52100000000000002},"min":{"1min":0.312,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.70499999999999996,"5min":0.71699999999999997,"15min":0.83799999999999997},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.50800000000000001,"15min":0.52400000000000002},"min":{"1min":0.33000000000000002,"5min":0.32600000000000001,"15min":0.318},"max":{"1min":0.68600000000000005,"5min":0.749,"15min":0.79200000000000004},"last":0.55500000000000005}]},{"osd":2,"last update":"Sat May 20 01:24:36 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.48199999999999998,"15min":0.504},"min":{"1min":0.36799999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.52700000000000002,"5min":0.72199999999999998,"15min":0.76600000000000001},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.51800000000000002,"15min":0.55000000000000004},"min":{"1min":0.35899999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.629,"5min":0.76200000000000001,"15min":0.76800000000000002},"last":0.64500000000000002}]},{"osd":3,"last update":"Sat May 20 01:24:36 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.51500000000000001,"15min":0.53500000000000003},"min":{"1min":0.32400000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.80800000000000005},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.52100000000000002,"15min":0.54000000000000004},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.58099999999999996,"5min":0.76400000000000001,"15min":0.79500000000000004},"last":0.57899999999999996}]},{"osd":4,"last update":"Sat May 20 01:24:36 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.53800000000000003,"15min":0.55000000000000004},"min":{"1min":0.42599999999999999,"5min":0.33300000000000002,"15min":0.32600000000000001},"max":{"1min":0.754,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.54700000000000004,"15min":0.56899999999999995},"min":{"1min":0.40300000000000002,"5min":0.378,"15min":0.32700000000000001},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.80100000000000005},"last":0.439}]},{"osd":5,"last update":"Sat May 20 01:24:36 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.54100000000000004,"15min":0.54400000000000004},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.65000000000000002,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.499},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.57599999999999996,"15min":0.60399999999999998},"min":{"1min":0.46100000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.78600000000000003,"5min":0.81599999999999995,"15min":0.89400000000000002},"last":0.61699999999999999}]},{"osd":7,"last update":"Sat May 20 01:24:36 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57699999999999996,"15min":0.59799999999999998},"min":{"1min":0.44,"5min":0.371,"15min":0.371},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.92900000000000005},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.58099999999999996,"15min":0.60999999999999999},"min":{"1min":0.48099999999999998,"5min":0.379,"15min":0.379},"max":{"1min":0.68100000000000005,"5min":0.82799999999999996,"15min":0.85199999999999998},"last":0.60799999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574959,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":16628,"kb_used_data":1644,"kb_used_omap":0,"kb_used_meta":14976,"kb_avail":94355212,"statfs":{"total":96636764160,"available":96619737088,"internally_reserved":0,"allocated":1683456,"data_stored":1134565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":15335424},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:24:33 2023","interfaces":[{"interface":"back","average":{"1min":0.254,"5min":0.28199999999999997,"15min":0.313},"min":{"1min":0.186,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":0.37,"5min":0.57099999999999995,"15min":1.194},"last":0.27400000000000002},{"interface":"front","average":{"1min":0.28299999999999997,"5min":0.314,"15min":0.316},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.41399999999999998,"5min":0.61499999999999999,"15min":0.75800000000000001},"last":0.29499999999999998}]},{"osd":2,"last update":"Sat May 20 01:24:33 2023","interfaces":[{"interface":"back","average":{"1min":0.27200000000000002,"5min":0.30199999999999999,"15min":0.317},"min":{"1min":0.20100000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.33800000000000002,"5min":0.52400000000000002,"15min":0.66400000000000003},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.33700000000000002,"5min":0.36799999999999999,"15min":0.40600000000000003},"min":{"1min":0.27800000000000002,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.41299999999999998,"5min":0.60299999999999998,"15min":1.3580000000000001},"last":0.39500000000000002}]},{"osd":3,"last update":"Sat May 20 01:24:33 2023","interfaces":[{"interface":"back","average":{"1min":0.315,"5min":0.34999999999999998,"15min":0.36899999999999999},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.40500000000000003,"5min":0.57399999999999995,"15min":1.619},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.32100000000000001,"5min":0.34699999999999998,"15min":0.379},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.41299999999999998,"5min":0.60899999999999999,"15min":1.8400000000000001},"last":0.33400000000000002}]},{"osd":4,"last update":"Sat May 20 01:24:33 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.55900000000000005,"15min":0.58099999999999996},"min":{"1min":0.379,"5min":0.34899999999999998,"15min":0.32200000000000001},"max":{"1min":0.72799999999999998,"5min":0.85299999999999998,"15min":1.2769999999999999},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.56999999999999995,"15min":0.58299999999999996},"min":{"1min":0.41699999999999998,"5min":0.312,"15min":0.312},"max":{"1min":0.65600000000000003,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.501}]},{"osd":5,"last update":"Sat May 20 01:24:33 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.58399999999999996,"15min":0.58899999999999997},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.754,"5min":0.81899999999999995,"15min":1.4510000000000001},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.59299999999999997,"15min":0.60599999999999998},"min":{"1min":0.436,"5min":0.39100000000000001,"15min":0.378},"max":{"1min":0.77200000000000002,"5min":0.79900000000000004,"15min":1.409},"last":0.55500000000000005}]},{"osd":6,"last update":"Sat May 20 01:24:33 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.59499999999999997,"15min":0.60399999999999998},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.73599999999999999,"5min":0.876,"15min":1.575},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.58499999999999996,"15min":0.59399999999999997},"min":{"1min":0.41299999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.66400000000000003,"5min":0.83099999999999996,"15min":1.0600000000000001},"last":0.54300000000000004}]},{"osd":7,"last update":"Sat May 20 01:24:33 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.59499999999999997,"15min":0.60499999999999998},"min":{"1min":0.40500000000000003,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.75600000000000001,"5min":0.88700000000000001,"15min":1.496},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.623,"15min":0.63800000000000001},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.77300000000000002,"5min":2.1709999999999998,"15min":2.1709999999999998},"last":0.60599999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574959,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10468,"kb_used_data":1052,"kb_used_omap":0,"kb_used_meta":9408,"kb_avail":94361372,"statfs":{"total":96636764160,"available":96626044928,"internally_reserved":0,"allocated":1077248,"data_stored":543076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9633792},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat May 20 01:24:32 2023","interfaces":[{"interface":"back","average":{"1min":0.26200000000000001,"5min":0.29199999999999998,"15min":0.318},"min":{"1min":0.192,"5min":0.17999999999999999,"15min":0.17999999999999999},"max":{"1min":0.371,"5min":0.54600000000000004,"15min":0.56599999999999995},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.26400000000000001,"5min":0.29699999999999999,"15min":0.34200000000000003},"min":{"1min":0.21099999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.33300000000000002,"5min":0.52000000000000002,"15min":0.98799999999999999},"last":0.38700000000000001}]},{"osd":2,"last update":"Sat May 20 01:24:32 2023","interfaces":[{"interface":"back","average":{"1min":0.28100000000000003,"5min":0.29999999999999999,"15min":0.32800000000000001},"min":{"1min":0.20100000000000001,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":0.35999999999999999,"5min":0.52300000000000002,"15min":0.55700000000000005},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.33100000000000002,"5min":0.34899999999999998,"15min":0.38200000000000001},"min":{"1min":0.26200000000000001,"5min":0.216,"15min":0.216},"max":{"1min":0.41699999999999998,"5min":0.60299999999999998,"15min":0.86199999999999999},"last":0.35199999999999998}]},{"osd":3,"last update":"Sat May 20 01:24:32 2023","interfaces":[{"interface":"back","average":{"1min":0.29599999999999999,"5min":0.33200000000000002,"15min":0.35999999999999999},"min":{"1min":0.24299999999999999,"5min":0.22,"15min":0.22},"max":{"1min":0.374,"5min":0.623,"15min":0.68300000000000005},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.35399999999999998,"15min":0.36599999999999999},"min":{"1min":0.28000000000000003,"5min":0.245,"15min":0.245},"max":{"1min":0.44700000000000001,"5min":0.60899999999999999,"15min":0.60899999999999999},"last":0.44500000000000001}]},{"osd":4,"last update":"Sat May 20 01:24:32 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.57099999999999995,"15min":0.55100000000000005},"min":{"1min":0.433,"5min":0.372,"15min":0.34399999999999997},"max":{"1min":0.78800000000000003,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.57999999999999996,"15min":0.56899999999999995},"min":{"1min":0.40799999999999997,"5min":0.39200000000000002,"15min":0.36299999999999999},"max":{"1min":0.70899999999999996,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.76200000000000001}]},{"osd":5,"last update":"Sat May 20 01:24:32 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.57799999999999996,"15min":0.57599999999999996},"min":{"1min":0.379,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.74299999999999999,"5min":0.80500000000000005,"15min":0.88100000000000001},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.63400000000000001,"15min":0.58899999999999997},"min":{"1min":0.40899999999999997,"5min":0.374,"15min":0.374},"max":{"1min":0.76300000000000001,"5min":4.4720000000000004,"15min":4.4720000000000004},"last":0.71899999999999997}]},{"osd":6,"last update":"Sat May 20 01:24:32 2023","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":0.72799999999999998,"15min":0.625},"min":{"1min":0.45800000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":10.042999999999999,"5min":10.042999999999999,"15min":10.042999999999999},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.60499999999999998,"15min":0.61599999999999999},"min":{"1min":0.45200000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.73399999999999999,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.73999999999999999}]},{"osd":7,"last update":"Sat May 20 01:24:32 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.59499999999999997,"15min":0.59299999999999997},"min":{"1min":0.433,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.74199999999999999,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.66700000000000004},{"interface":"front","average":{"1min":1.1040000000000001,"5min":0.72499999999999998,"15min":0.65700000000000003},"min":{"1min":0.44800000000000001,"5min":0.438,"15min":0.42099999999999999},"max":{"1min":10.085000000000001,"5min":10.085000000000001,"15min":10.085000000000001},"last":0.78100000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574959,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10536,"kb_used_data":1056,"kb_used_omap":0,"kb_used_meta":9472,"kb_avail":94361304,"statfs":{"total":96636764160,"available":96625975296,"internally_reserved":0,"allocated":1081344,"data_stored":544585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9699328},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:24:44 2023","interfaces":[{"interface":"back","average":{"1min":0.255,"5min":0.27500000000000002,"15min":0.32400000000000001},"min":{"1min":0.19500000000000001,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":0.36799999999999999,"5min":0.53400000000000003,"15min":1.038},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.26700000000000002,"5min":0.29699999999999999,"15min":0.34200000000000003},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.36799999999999999,"5min":0.46500000000000002,"15min":1.079},"last":0.47999999999999998}]},{"osd":1,"last update":"Sat May 20 01:24:44 2023","interfaces":[{"interface":"back","average":{"1min":0.28999999999999998,"5min":0.314,"15min":0.38100000000000001},"min":{"1min":0.23400000000000001,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.34899999999999998,"5min":0.61399999999999999,"15min":1.4950000000000001},"last":0.308},{"interface":"front","average":{"1min":0.318,"5min":0.35499999999999998,"15min":0.41799999999999998},"min":{"1min":0.27400000000000002,"5min":0.249,"15min":0.249},"max":{"1min":0.36099999999999999,"5min":0.629,"15min":1.5189999999999999},"last":0.41899999999999998}]},{"osd":3,"last update":"Sat May 20 01:24:44 2023","interfaces":[{"interface":"back","average":{"1min":0.27800000000000002,"5min":0.32400000000000001,"15min":0.38200000000000001},"min":{"1min":0.218,"5min":0.216,"15min":0.216},"max":{"1min":0.42499999999999999,"5min":0.60599999999999998,"15min":1.2090000000000001},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.34000000000000002,"5min":0.36699999999999999,"15min":0.432},"min":{"1min":0.30199999999999999,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.38800000000000001,"5min":0.68100000000000005,"15min":1.5620000000000001},"last":0.497}]},{"osd":4,"last update":"Sat May 20 01:24:44 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.55500000000000005,"15min":0.57699999999999996},"min":{"1min":0.38800000000000001,"5min":0.35899999999999999,"15min":0.34899999999999998},"max":{"1min":0.69199999999999995,"5min":0.85199999999999998,"15min":1.319},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.58499999999999996,"15min":0.61199999999999999},"min":{"1min":0.432,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.69499999999999995,"5min":0.89000000000000001,"15min":1.4319999999999999},"last":0.62}]},{"osd":5,"last update":"Sat May 20 01:24:44 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.55800000000000005,"15min":0.62},"min":{"1min":0.41999999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.73599999999999999,"5min":0.95599999999999996,"15min":1.758},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.58299999999999996,"15min":0.63800000000000001},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.72999999999999998,"5min":0.85099999999999998,"15min":1.7110000000000001},"last":0.74399999999999999}]},{"osd":6,"last update":"Sat May 20 01:24:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.58599999999999997,"15min":0.63200000000000001},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.39700000000000002},"max":{"1min":0.746,"5min":0.88900000000000001,"15min":1.665},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.58299999999999996,"15min":0.63200000000000001},"min":{"1min":0.41699999999999998,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.74299999999999999,"5min":0.93500000000000005,"15min":1.819},"last":0.59199999999999997}]},{"osd":7,"last update":"Sat May 20 01:24:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.59499999999999997,"15min":0.65700000000000003},"min":{"1min":0.44400000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.76900000000000002,"5min":0.85099999999999998,"15min":1.7310000000000001},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.59899999999999998,"15min":0.65300000000000002},"min":{"1min":0.46999999999999997,"5min":0.42399999999999999,"15min":0.42199999999999999},"max":{"1min":0.72499999999999998,"5min":0.90600000000000003,"15min":1.8460000000000001},"last":0.63600000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574959,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10608,"kb_used_data":1064,"kb_used_omap":0,"kb_used_meta":9536,"kb_avail":94361232,"statfs":{"total":96636764160,"available":96625901568,"internally_reserved":0,"allocated":1089536,"data_stored":545567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9764864},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:24:32 2023","interfaces":[{"interface":"back","average":{"1min":0.27300000000000002,"5min":0.30599999999999999,"15min":0.317},"min":{"1min":0.21199999999999999,"5min":0.188,"15min":0.188},"max":{"1min":0.375,"5min":1.528,"15min":1.528},"last":0.23400000000000001},{"interface":"front","average":{"1min":0.28000000000000003,"5min":0.34200000000000003,"15min":0.42299999999999999},"min":{"1min":0.214,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.40699999999999997,"5min":2.4209999999999998,"15min":3.1890000000000001},"last":0.36899999999999999}]},{"osd":1,"last update":"Sat May 20 01:24:32 2023","interfaces":[{"interface":"back","average":{"1min":0.27300000000000002,"5min":0.33600000000000002,"15min":0.41499999999999998},"min":{"1min":0.19900000000000001,"5min":0.189,"15min":0.189},"max":{"1min":0.377,"5min":3.2999999999999998,"15min":3.4260000000000002},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.33200000000000002,"5min":0.34200000000000003,"15min":0.35599999999999998},"min":{"1min":0.23799999999999999,"5min":0.22900000000000001,"15min":0.221},"max":{"1min":0.45500000000000002,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.27200000000000002}]},{"osd":2,"last update":"Sat May 20 01:24:32 2023","interfaces":[{"interface":"back","average":{"1min":0.30599999999999999,"5min":0.34300000000000003,"15min":0.38700000000000001},"min":{"1min":0.26400000000000001,"5min":0.245,"15min":0.224},"max":{"1min":0.39200000000000002,"5min":0.625,"15min":1.1619999999999999},"last":0.309},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.371,"15min":0.41299999999999998},"min":{"1min":0.28499999999999998,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.41499999999999998,"5min":0.72399999999999998,"15min":1.413},"last":0.33700000000000002}]},{"osd":4,"last update":"Sat May 20 01:24:32 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.55100000000000005,"15min":0.58699999999999997},"min":{"1min":0.40300000000000002,"5min":0.38600000000000001,"15min":0.38200000000000001},"max":{"1min":0.72899999999999998,"5min":0.95399999999999996,"15min":1.0700000000000001},"last":0.442},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.61199999999999999,"15min":0.68000000000000005},"min":{"1min":0.41099999999999998,"5min":0.40200000000000002,"15min":0.35899999999999999},"max":{"1min":0.72899999999999998,"5min":3.1789999999999998,"15min":3.4660000000000002},"last":0.504}]},{"osd":5,"last update":"Sat May 20 01:24:32 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.59899999999999998,"15min":0.68500000000000005},"min":{"1min":0.38400000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.751,"5min":3.2170000000000001,"15min":3.5379999999999998},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.59999999999999998,"15min":0.63200000000000001},"min":{"1min":0.41999999999999998,"5min":0.40799999999999997,"15min":0.374},"max":{"1min":0.74199999999999999,"5min":1.026,"15min":1.385},"last":0.47799999999999998}]},{"osd":6,"last update":"Sat May 20 01:24:32 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.59599999999999997,"15min":0.63400000000000001},"min":{"1min":0.439,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.70199999999999996,"5min":0.99199999999999999,"15min":1.347},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.58499999999999996,"15min":0.63},"min":{"1min":0.39500000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.752,"5min":0.94099999999999995,"15min":1.107},"last":0.55700000000000005}]},{"osd":7,"last update":"Sat May 20 01:24:32 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.59699999999999998,"15min":0.63700000000000001},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.64000000000000001,"5min":0.94199999999999995,"15min":1.1359999999999999},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.625,"5min":0.65100000000000002,"15min":0.72499999999999998},"min":{"1min":0.47599999999999998,"5min":0.436,"15min":0.436},"max":{"1min":0.73799999999999999,"5min":3.2559999999999998,"15min":3.5009999999999999},"last":0.46500000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574959,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":15908,"kb_used_data":1052,"kb_used_omap":0,"kb_used_meta":14848,"kb_avail":94355932,"statfs":{"total":96636764160,"available":96620474368,"internally_reserved":0,"allocated":1077248,"data_stored":544003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":15204352},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:24:33 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.47299999999999998,"15min":0.48099999999999998},"min":{"1min":0.32200000000000001,"5min":0.247,"15min":0.247},"max":{"1min":0.59199999999999997,"5min":0.73699999999999999,"15min":0.75600000000000001},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.50700000000000001,"15min":0.498},"min":{"1min":0.30599999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.64800000000000002,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":0.39400000000000002}]},{"osd":1,"last update":"Sat May 20 01:24:33 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.49199999999999999,"15min":0.51900000000000002},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.65300000000000002,"5min":0.746,"15min":0.82799999999999996},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.51900000000000002,"15min":0.53000000000000003},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.63600000000000001,"5min":1.7190000000000001,"15min":2.0739999999999998},"last":0.46999999999999997}]},{"osd":2,"last update":"Sat May 20 01:24:33 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.51900000000000002,"15min":0.50800000000000001},"min":{"1min":0.36599999999999999,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.64900000000000002,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.505},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.54200000000000004,"15min":0.55000000000000004},"min":{"1min":0.378,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.61399999999999999,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.58099999999999996}]},{"osd":3,"last update":"Sat May 20 01:24:33 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.52600000000000002,"15min":0.52400000000000002},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34399999999999997},"max":{"1min":0.626,"5min":0.81399999999999995,"15min":1.827},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.57099999999999995,"15min":0.65500000000000003},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.65000000000000002,"5min":0.89300000000000002,"15min":3.1749999999999998},"last":0.56799999999999995}]},{"osd":5,"last update":"Sat May 20 01:24:33 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.53400000000000003,"15min":0.48699999999999999},"min":{"1min":0.36899999999999999,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.76700000000000002,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.55400000000000005,"15min":0.51300000000000001},"min":{"1min":0.35099999999999998,"5min":0.33400000000000002,"15min":0.29899999999999999},"max":{"1min":0.78400000000000003,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.70099999999999996}]},{"osd":6,"last update":"Sat May 20 01:24:33 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.58799999999999997,"15min":0.54700000000000004},"min":{"1min":0.432,"5min":0.40400000000000003,"15min":0.32400000000000001},"max":{"1min":0.80300000000000005,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.61599999999999999,"15min":0.59099999999999997},"min":{"1min":0.51100000000000001,"5min":0.42399999999999999,"15min":0.39400000000000002},"max":{"1min":0.72299999999999998,"5min":0.84699999999999998,"15min":0.85899999999999999},"last":0.66900000000000004}]},{"osd":7,"last update":"Sat May 20 01:24:33 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.57799999999999996,"15min":0.55700000000000005},"min":{"1min":0.41999999999999998,"5min":0.36099999999999999,"15min":0.34200000000000003},"max":{"1min":0.71799999999999997,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.61399999999999999,"15min":0.59799999999999998},"min":{"1min":0.499,"5min":0.36399999999999999,"15min":0.34000000000000002},"max":{"1min":0.73699999999999999,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.68500000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574960,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10352,"kb_used_data":1064,"kb_used_omap":0,"kb_used_meta":9280,"kb_avail":94361488,"statfs":{"total":96636764160,"available":96626163712,"internally_reserved":0,"allocated":1089536,"data_stored":544628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9502720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:24:38 2023","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.45400000000000001,"15min":0.65300000000000002},"min":{"1min":0.29999999999999999,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.68899999999999995,"5min":0.72599999999999998,"15min":6.3929999999999998},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.46000000000000002,"15min":0.68000000000000005},"min":{"1min":0.34999999999999998,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.61099999999999999,"5min":0.67500000000000004,"15min":6.3339999999999996},"last":0.42499999999999999}]},{"osd":1,"last update":"Sat May 20 01:24:38 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.46000000000000002,"15min":0.58799999999999997},"min":{"1min":0.313,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":3.2450000000000001},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.498,"5min":0.499,"15min":0.53400000000000003},"min":{"1min":0.38900000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.72599999999999998,"5min":0.752,"15min":1.1819999999999999},"last":0.57199999999999995}]},{"osd":2,"last update":"Sat May 20 01:24:38 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48299999999999998,"15min":0.54100000000000004},"min":{"1min":0.33200000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":1.1639999999999999},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.48399999999999999,"15min":0.54400000000000004},"min":{"1min":0.308,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.628,"5min":0.71799999999999997,"15min":1.258},"last":0.53600000000000003}]},{"osd":3,"last update":"Sat May 20 01:24:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.51500000000000001,"15min":0.72499999999999998},"min":{"1min":0.35099999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.69499999999999995,"5min":0.77800000000000002,"15min":6.351},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.53200000000000003,"15min":0.75800000000000001},"min":{"1min":0.40699999999999997,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.67500000000000004,"5min":0.76500000000000001,"15min":6.3730000000000002},"last":0.68999999999999995}]},{"osd":4,"last update":"Sat May 20 01:24:38 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.503,"15min":0.49299999999999999},"min":{"1min":0.38900000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.64500000000000002,"5min":0.68400000000000005,"15min":0.70199999999999996},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.54000000000000004,"15min":0.55100000000000005},"min":{"1min":0.41599999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.749,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.65600000000000003}]},{"osd":6,"last update":"Sat May 20 01:24:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.54600000000000004,"15min":0.55200000000000005},"min":{"1min":0.48699999999999999,"5min":0.38100000000000001,"15min":0.29999999999999999},"max":{"1min":0.72899999999999998,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.54500000000000004,"15min":0.55800000000000005},"min":{"1min":0.437,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.72399999999999998,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.67000000000000004}]},{"osd":7,"last update":"Sat May 20 01:24:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.56799999999999995,"15min":0.58199999999999996},"min":{"1min":0.46500000000000002,"5min":0.33400000000000002,"15min":0.33000000000000002},"max":{"1min":0.77300000000000002,"5min":0.82299999999999995,"15min":0.91000000000000003},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.57699999999999996,"15min":0.60799999999999998},"min":{"1min":0.44700000000000001,"5min":0.40999999999999998,"15min":0.39500000000000002},"max":{"1min":0.76200000000000001,"5min":0.81399999999999995,"15min":0.94499999999999995},"last":0.748}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-05-20T01:25:21.577 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-05-20T01:25:21.578 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-05-20T01:25:21.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.737+0000 7f37a057b700 1 Processor -- start 2023-05-20T01:25:21.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.737+0000 7f37a057b700 1 -- start start 2023-05-20T01:25:21.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.738+0000 7f37a057b700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f3798157450 0x7f3798159e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:21.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.738+0000 7f37a057b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3798071e10 0x7f37980caff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:21.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.738+0000 7f37a057b700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f379815d5b0 con 0x7f37980cb530 2023-05-20T01:25:21.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.738+0000 7f37a057b700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37980cb990 con 0x7f3798071e10 2023-05-20T01:25:21.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.738+0000 7f37a057b700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3798157820 con 0x7f3798157450 2023-05-20T01:25:21.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.738+0000 7f379db16700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3798071e10 0x7f37980caff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:21.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.738+0000 7f379db16700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3798071e10 0x7f37980caff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36344/0 (socket says 172.21.15.16:36344) 2023-05-20T01:25:21.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.739+0000 7f379db16700 1 -- 172.21.15.16:0/1262477581 learned_addr learned my addr 172.21.15.16:0/1262477581 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:21.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.739+0000 7f379e317700 1 --2- 172.21.15.16:0/1262477581 >> v2:172.21.15.132:3300/0 conn(0x7f3798157450 0x7f3798159e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:21.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.739+0000 7f379db16700 1 -- 172.21.15.16:0/1262477581 >> v2:172.21.15.132:3300/0 conn(0x7f3798157450 msgr2=0x7f3798159e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:21.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.739+0000 7f379db16700 1 --2- 172.21.15.16:0/1262477581 >> v2:172.21.15.132:3300/0 conn(0x7f3798157450 0x7f3798159e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:21.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.739+0000 7f379db16700 1 -- 172.21.15.16:0/1262477581 >> v1:172.21.15.132:6789/0 conn(0x7f37980cb530 legacy=0x7f379815da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:21.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.739+0000 7f379db16700 1 -- 172.21.15.16:0/1262477581 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f379815a4c0 con 0x7f3798071e10 2023-05-20T01:25:21.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.739+0000 7f379db16700 1 --2- 172.21.15.16:0/1262477581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3798071e10 0x7f37980caff0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f37940098f0 tx=0x7f3794009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb9c5bf7de45df08 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:21.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.739+0000 7f379d315700 1 -- 172.21.15.16:0/1262477581 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f379400b860 con 0x7f3798071e10 2023-05-20T01:25:21.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.739+0000 7f379d315700 1 -- 172.21.15.16:0/1262477581 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f379400b9c0 con 0x7f3798071e10 2023-05-20T01:25:21.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.739+0000 7f379d315700 1 -- 172.21.15.16:0/1262477581 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f379400bd50 con 0x7f3798071e10 2023-05-20T01:25:21.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.740+0000 7f37a057b700 1 -- 172.21.15.16:0/1262477581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3798071e10 msgr2=0x7f37980caff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:21.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.740+0000 7f37a057b700 1 --2- 172.21.15.16:0/1262477581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3798071e10 0x7f37980caff0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f37940098f0 tx=0x7f3794009c00 comp rx=0 tx=0).stop 2023-05-20T01:25:21.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.740+0000 7f37a057b700 1 -- 172.21.15.16:0/1262477581 shutdown_connections 2023-05-20T01:25:21.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.740+0000 7f37a057b700 1 --2- 172.21.15.16:0/1262477581 >> v2:172.21.15.132:3300/0 conn(0x7f3798157450 0x7f3798159e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:21.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.740+0000 7f37a057b700 1 --2- 172.21.15.16:0/1262477581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3798071e10 0x7f37980caff0 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:21.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.740+0000 7f37a057b700 1 -- 172.21.15.16:0/1262477581 >> 172.21.15.16:0/1262477581 conn(0x7f379806cbb0 msgr2=0x7f379806cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:21.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.740+0000 7f37a057b700 1 -- 172.21.15.16:0/1262477581 shutdown_connections 2023-05-20T01:25:21.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.740+0000 7f37a057b700 1 -- 172.21.15.16:0/1262477581 wait complete. 2023-05-20T01:25:21.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.741+0000 7f37a057b700 1 Processor -- start 2023-05-20T01:25:21.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.741+0000 7f37a057b700 1 -- start start 2023-05-20T01:25:21.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.742+0000 7f37a057b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3798071e10 0x7f37980c9150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:21.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.742+0000 7f37a057b700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f37980cb530 0x7f37980c7940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:21.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.742+0000 7f37a057b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3798157450 0x7f37980cc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:21.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.742+0000 7f37a057b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37980c7e80 con 0x7f3798071e10 2023-05-20T01:25:21.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.742+0000 7f37a057b700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37980c80b0 con 0x7f3798157450 2023-05-20T01:25:21.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.742+0000 7f37a057b700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37980c8390 con 0x7f37980cb530 2023-05-20T01:25:21.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.742+0000 7f379e317700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3798071e10 0x7f37980c9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:21.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.743+0000 7f379e317700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3798071e10 0x7f37980c9150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36348/0 (socket says 172.21.15.16:36348) 2023-05-20T01:25:21.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.743+0000 7f379e317700 1 -- 172.21.15.16:0/3864306501 learned_addr learned my addr 172.21.15.16:0/3864306501 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:21.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.743+0000 7f379db16700 1 --2- 172.21.15.16:0/3864306501 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f37980cb530 0x7f37980c7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:21.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.743+0000 7f379eb18700 1 --2- 172.21.15.16:0/3864306501 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3798157450 0x7f37980cc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:21.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.743+0000 7f379e317700 1 -- 172.21.15.16:0/3864306501 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f37980cb530 msgr2=0x7f37980c7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:21.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.743+0000 7f379e317700 1 --2- 172.21.15.16:0/3864306501 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f37980cb530 0x7f37980c7940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:21.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.743+0000 7f379e317700 1 -- 172.21.15.16:0/3864306501 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3798157450 msgr2=0x7f37980cc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:21.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.743+0000 7f379e317700 1 --2- 172.21.15.16:0/3864306501 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3798157450 0x7f37980cc4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:21.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.743+0000 7f379e317700 1 -- 172.21.15.16:0/3864306501 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3794009550 con 0x7f3798071e10 2023-05-20T01:25:21.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.743+0000 7f379db16700 1 --2- 172.21.15.16:0/3864306501 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f37980cb530 0x7f37980c7940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:25:21.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.743+0000 7f379e317700 1 --2- 172.21.15.16:0/3864306501 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3798071e10 0x7f37980c9150 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f378800d800 tx=0x7f378800dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:21.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.743+0000 7f378effd700 1 -- 172.21.15.16:0/3864306501 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37880125f0 con 0x7f3798071e10 2023-05-20T01:25:21.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.743+0000 7f378effd700 1 -- 172.21.15.16:0/3864306501 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3788012750 con 0x7f3798071e10 2023-05-20T01:25:21.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.743+0000 7f37a057b700 1 -- 172.21.15.16:0/3864306501 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f379815aa80 con 0x7f3798071e10 2023-05-20T01:25:21.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.743+0000 7f378effd700 1 -- 172.21.15.16:0/3864306501 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3788012b00 con 0x7f3798071e10 2023-05-20T01:25:21.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.743+0000 7f37a057b700 1 -- 172.21.15.16:0/3864306501 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f379815a7a0 con 0x7f3798071e10 2023-05-20T01:25:21.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.744+0000 7f37a057b700 1 -- 172.21.15.16:0/3864306501 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f379815ad60 con 0x7f3798071e10 2023-05-20T01:25:21.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.745+0000 7f378effd700 1 -- 172.21.15.16:0/3864306501 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3788012c60 con 0x7f3798071e10 2023-05-20T01:25:21.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.745+0000 7f378effd700 1 --2- 172.21.15.16:0/3864306501 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f377c06ace0 0x7f377c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:21.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.745+0000 7f378effd700 1 -- 172.21.15.16:0/3864306501 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f3788089a80 con 0x7f3798071e10 2023-05-20T01:25:21.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.745+0000 7f379db16700 1 --2- 172.21.15.16:0/3864306501 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f377c06ace0 0x7f377c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:21.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.745+0000 7f379db16700 1 --2- 172.21.15.16:0/3864306501 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f377c06ace0 0x7f377c06d190 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f3794014040 tx=0x7f3794008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:21.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:21.750+0000 7f378effd700 1 -- 172.21.15.16:0/3864306501 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f3788054970 con 0x7f3798071e10 2023-05-20T01:25:22.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.027+0000 7f37a057b700 1 -- 172.21.15.16:0/3864306501 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f37980747b0 con 0x7f3798071e10 2023-05-20T01:25:22.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.031+0000 7f378effd700 1 -- 172.21.15.16:0/3864306501 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v12) v1 ==== 190+0+0 (secure 0 0 0) 0x7f3788057f90 con 0x7f3798071e10 2023-05-20T01:25:22.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.033+0000 7f37a057b700 1 -- 172.21.15.16:0/3864306501 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f377c06ace0 msgr2=0x7f377c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:22.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.033+0000 7f37a057b700 1 --2- 172.21.15.16:0/3864306501 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f377c06ace0 0x7f377c06d190 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f3794014040 tx=0x7f3794008030 comp rx=0 tx=0).stop 2023-05-20T01:25:22.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.033+0000 7f37a057b700 1 -- 172.21.15.16:0/3864306501 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3798071e10 msgr2=0x7f37980c9150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:22.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.033+0000 7f37a057b700 1 --2- 172.21.15.16:0/3864306501 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3798071e10 0x7f37980c9150 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f378800d800 tx=0x7f378800dbc0 comp rx=0 tx=0).stop 2023-05-20T01:25:22.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.033+0000 7f37a057b700 1 -- 172.21.15.16:0/3864306501 shutdown_connections 2023-05-20T01:25:22.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.033+0000 7f37a057b700 1 --2- 172.21.15.16:0/3864306501 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3798157450 0x7f37980cc4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:22.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.033+0000 7f37a057b700 1 --2- 172.21.15.16:0/3864306501 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3798071e10 0x7f37980c9150 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:22.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.033+0000 7f37a057b700 1 --2- 172.21.15.16:0/3864306501 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f377c06ace0 0x7f377c06d190 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:22.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.033+0000 7f37a057b700 1 --2- 172.21.15.16:0/3864306501 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f37980cb530 0x7f37980c7940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:22.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.033+0000 7f37a057b700 1 -- 172.21.15.16:0/3864306501 >> 172.21.15.16:0/3864306501 conn(0x7f379806cbb0 msgr2=0x7f37980c5850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:22.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.033+0000 7f37a057b700 1 -- 172.21.15.16:0/3864306501 shutdown_connections 2023-05-20T01:25:22.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.033+0000 7f37a057b700 1 -- 172.21.15.16:0/3864306501 wait complete. 2023-05-20T01:25:22.039 INFO:teuthology.orchestra.run.smithi016.stderr:updated caps for client.0 2023-05-20T01:25:22.050 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-20T01:25:22.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.170+0000 7fe110b4a700 1 Processor -- start 2023-05-20T01:25:22.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.170+0000 7fe110b4a700 1 -- start start 2023-05-20T01:25:22.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.171+0000 7fe110b4a700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe10c0d16c0 0x7fe10c144330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:22.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.171+0000 7fe110b4a700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe10c144870 con 0x7fe10c0b1990 2023-05-20T01:25:22.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.171+0000 7fe110b4a700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe10c0b2000 con 0x7fe10c0cec20 2023-05-20T01:25:22.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.171+0000 7fe110b4a700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe10c0b1e10 con 0x7fe10c0d16c0 2023-05-20T01:25:22.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.171+0000 7fe10ad9d700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fe10c0b1990 0x7fe10c0ae9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:47888/0 (socket says 172.21.15.16:47888) 2023-05-20T01:25:22.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.171+0000 7fe10ad9d700 1 -- 172.21.15.16:0/1850828460 learned_addr learned my addr 172.21.15.16:0/1850828460 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:25:22.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.172+0000 7fe10a59c700 1 --2- 172.21.15.16:0/1850828460 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe10c0d16c0 0x7fe10c144330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:22.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.172+0000 7fe10959a700 1 -- 172.21.15.16:0/1850828460 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4022535878 0 0) 0x7fe10c144870 con 0x7fe10c0b1990 2023-05-20T01:25:22.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.172+0000 7fe10959a700 1 -- 172.21.15.16:0/1850828460 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe0f40035b0 con 0x7fe10c0b1990 2023-05-20T01:25:22.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.172+0000 7fe10959a700 1 -- 172.21.15.16:0/1850828460 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3111388853 0 0) 0x7fe10c0b2000 con 0x7fe10c0cec20 2023-05-20T01:25:22.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.172+0000 7fe10959a700 1 -- 172.21.15.16:0/1850828460 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe10c144870 con 0x7fe10c0cec20 2023-05-20T01:25:22.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.172+0000 7fe10a59c700 1 -- 172.21.15.16:0/1850828460 >> v1:172.21.15.132:6789/0 conn(0x7fe10c0cec20 legacy=0x7fe10c0ae2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:22.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.172+0000 7fe10a59c700 1 -- 172.21.15.16:0/1850828460 >> v1:172.21.15.16:6789/0 conn(0x7fe10c0b1990 legacy=0x7fe10c0ae9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:22.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.172+0000 7fe10959a700 1 -- 172.21.15.16:0/1850828460 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 413502531 0 0) 0x7fe0f40035b0 con 0x7fe10c0b1990 2023-05-20T01:25:22.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.172+0000 7fe10a59c700 1 -- 172.21.15.16:0/1850828460 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe10c0b22e0 con 0x7fe10c0d16c0 2023-05-20T01:25:22.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.172+0000 7fe10a59c700 1 --2- 172.21.15.16:0/1850828460 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe10c0d16c0 0x7fe10c144330 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe100009870 tx=0x7fe100009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e017a016ddd6ff5 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:22.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.172+0000 7fe10959a700 1 -- 172.21.15.16:0/1850828460 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1040019f0 con 0x7fe10c0d16c0 2023-05-20T01:25:22.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.172+0000 7fe10959a700 1 -- 172.21.15.16:0/1850828460 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe100004030 con 0x7fe10c0d16c0 2023-05-20T01:25:22.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.172+0000 7fe10959a700 1 -- 172.21.15.16:0/1850828460 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1000043c0 con 0x7fe10c0d16c0 2023-05-20T01:25:22.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.173+0000 7fe110b4a700 1 -- 172.21.15.16:0/1850828460 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe10c0d16c0 msgr2=0x7fe10c144330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:22.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.173+0000 7fe110b4a700 1 --2- 172.21.15.16:0/1850828460 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe10c0d16c0 0x7fe10c144330 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe100009870 tx=0x7fe100009b80 comp rx=0 tx=0).stop 2023-05-20T01:25:22.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.173+0000 7fe110b4a700 1 -- 172.21.15.16:0/1850828460 shutdown_connections 2023-05-20T01:25:22.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.173+0000 7fe110b4a700 1 --2- 172.21.15.16:0/1850828460 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe10c0d16c0 0x7fe10c144330 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:22.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.173+0000 7fe110b4a700 1 -- 172.21.15.16:0/1850828460 >> 172.21.15.16:0/1850828460 conn(0x7fe10c06ae30 msgr2=0x7fe10c09a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:22.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.173+0000 7fe110b4a700 1 -- 172.21.15.16:0/1850828460 shutdown_connections 2023-05-20T01:25:22.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.173+0000 7fe110b4a700 1 -- 172.21.15.16:0/1850828460 wait complete. 2023-05-20T01:25:22.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.174+0000 7fe110b4a700 1 Processor -- start 2023-05-20T01:25:22.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.174+0000 7fe110b4a700 1 -- start start 2023-05-20T01:25:22.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.175+0000 7fe110b4a700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe10c0b1990 0x7fe10c0b58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:22.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.175+0000 7fe110b4a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe10c0cec20 0x7fe10c06f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:22.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.175+0000 7fe110b4a700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe10c0d16c0 0x7fe10c0cc720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:22.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.175+0000 7fe110b4a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe10c0cb680 con 0x7fe10c0cec20 2023-05-20T01:25:22.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.175+0000 7fe110b4a700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe10c0cb960 con 0x7fe10c0d16c0 2023-05-20T01:25:22.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.175+0000 7fe110b4a700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe10c0cbc40 con 0x7fe10c0b1990 2023-05-20T01:25:22.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.175+0000 7fe109d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe10c0cec20 0x7fe10c06f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:22.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.175+0000 7fe109d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe10c0cec20 0x7fe10c06f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36364/0 (socket says 172.21.15.16:36364) 2023-05-20T01:25:22.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.175+0000 7fe109d9b700 1 -- 172.21.15.16:0/1099420015 learned_addr learned my addr 172.21.15.16:0/1099420015 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:22.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.175+0000 7fe10a59c700 1 --2- 172.21.15.16:0/1099420015 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe10c0b1990 0x7fe10c0b58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:22.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.175+0000 7fe10ad9d700 1 --2- 172.21.15.16:0/1099420015 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe10c0d16c0 0x7fe10c0cc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:22.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.176+0000 7fe109d9b700 1 -- 172.21.15.16:0/1099420015 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe10c0b1990 msgr2=0x7fe10c0b58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:22.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.176+0000 7fe109d9b700 1 --2- 172.21.15.16:0/1099420015 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe10c0b1990 0x7fe10c0b58e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:22.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.176+0000 7fe109d9b700 1 -- 172.21.15.16:0/1099420015 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe10c0d16c0 msgr2=0x7fe10c0cc720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:22.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.176+0000 7fe109d9b700 1 --2- 172.21.15.16:0/1099420015 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe10c0d16c0 0x7fe10c0cc720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:22.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.176+0000 7fe109d9b700 1 -- 172.21.15.16:0/1099420015 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1000094a0 con 0x7fe10c0cec20 2023-05-20T01:25:22.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.176+0000 7fe10ad9d700 1 --2- 172.21.15.16:0/1099420015 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe10c0d16c0 0x7fe10c0cc720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:25:22.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.176+0000 7fe109d9b700 1 --2- 172.21.15.16:0/1099420015 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe10c0cec20 0x7fe10c06f780 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc00c520 tx=0x7fe0fc00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:22.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.176+0000 7fe0faffd700 1 -- 172.21.15.16:0/1099420015 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0fc0052d0 con 0x7fe10c0cec20 2023-05-20T01:25:22.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.176+0000 7fe0faffd700 1 -- 172.21.15.16:0/1099420015 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0fc005430 con 0x7fe10c0cec20 2023-05-20T01:25:22.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.176+0000 7fe110b4a700 1 -- 172.21.15.16:0/1099420015 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe10c0adda0 con 0x7fe10c0cec20 2023-05-20T01:25:22.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.176+0000 7fe0faffd700 1 -- 172.21.15.16:0/1099420015 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0fc0038c0 con 0x7fe10c0cec20 2023-05-20T01:25:22.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.176+0000 7fe110b4a700 1 -- 172.21.15.16:0/1099420015 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe10c0b25c0 con 0x7fe10c0cec20 2023-05-20T01:25:22.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.177+0000 7fe110b4a700 1 -- 172.21.15.16:0/1099420015 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe10c0b28a0 con 0x7fe10c0cec20 2023-05-20T01:25:22.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.177+0000 7fe0faffd700 1 -- 172.21.15.16:0/1099420015 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe0fc003a20 con 0x7fe10c0cec20 2023-05-20T01:25:22.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.178+0000 7fe0faffd700 1 --2- 172.21.15.16:0/1099420015 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe0ec06ad10 0x7fe0ec06d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:22.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.178+0000 7fe0faffd700 1 -- 172.21.15.16:0/1099420015 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fe0fc088b00 con 0x7fe10c0cec20 2023-05-20T01:25:22.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.178+0000 7fe10a59c700 1 --2- 172.21.15.16:0/1099420015 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe0ec06ad10 0x7fe0ec06d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:22.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.178+0000 7fe10a59c700 1 --2- 172.21.15.16:0/1099420015 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe0ec06ad10 0x7fe0ec06d1c0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fe1000097c0 tx=0x7fe100013040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:22.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.182+0000 7fe0faffd700 1 -- 172.21.15.16:0/1099420015 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fe0fc053a70 con 0x7fe10c0cec20 2023-05-20T01:25:22.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.427+0000 7fe110b4a700 1 -- 172.21.15.16:0/1099420015 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe10c0747b0 con 0x7fe10c0cec20 2023-05-20T01:25:22.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.428+0000 7fe0faffd700 1 -- 172.21.15.16:0/1099420015 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+4673 (secure 0 0 0) 0x7fe0fc057090 con 0x7fe10c0cec20 2023-05-20T01:25:22.432 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:25:22.433 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":57,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6077,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/1241894038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":1241894038},{"type":"v1","addr":"172.21.15.132:6835","nonce":1241894038}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":6562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/3771738903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3771738903},{"type":"v1","addr":"172.21.15.132:6834","nonce":3771738903}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":7101,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/3876038711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":3876038711},{"type":"v1","addr":"172.21.15.16:6836","nonce":3876038711}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-20T01:25:11.987993+0000","modified":"2023-05-20T01:25:15.395667+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7104},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7104":{"gid":7104,"name":"c","rank":0,"incarnation":53,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6837/28561908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":28561908},{"type":"v1","addr":"172.21.15.16:6837","nonce":28561908}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-05-20T01:25:22.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.430+0000 7fe110b4a700 1 -- 172.21.15.16:0/1099420015 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe0ec06ad10 msgr2=0x7fe0ec06d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:22.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.430+0000 7fe110b4a700 1 --2- 172.21.15.16:0/1099420015 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe0ec06ad10 0x7fe0ec06d1c0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fe1000097c0 tx=0x7fe100013040 comp rx=0 tx=0).stop 2023-05-20T01:25:22.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.430+0000 7fe110b4a700 1 -- 172.21.15.16:0/1099420015 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe10c0cec20 msgr2=0x7fe10c06f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:22.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.430+0000 7fe110b4a700 1 --2- 172.21.15.16:0/1099420015 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe10c0cec20 0x7fe10c06f780 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc00c520 tx=0x7fe0fc00c8e0 comp rx=0 tx=0).stop 2023-05-20T01:25:22.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.430+0000 7fe110b4a700 1 -- 172.21.15.16:0/1099420015 shutdown_connections 2023-05-20T01:25:22.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.430+0000 7fe110b4a700 1 --2- 172.21.15.16:0/1099420015 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe10c0d16c0 0x7fe10c0cc720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:22.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.430+0000 7fe110b4a700 1 --2- 172.21.15.16:0/1099420015 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe0ec06ad10 0x7fe0ec06d1c0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:22.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.430+0000 7fe110b4a700 1 --2- 172.21.15.16:0/1099420015 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe10c0b1990 0x7fe10c0b58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:22.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.430+0000 7fe110b4a700 1 --2- 172.21.15.16:0/1099420015 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe10c0cec20 0x7fe10c06f780 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:22.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.430+0000 7fe110b4a700 1 -- 172.21.15.16:0/1099420015 >> 172.21.15.16:0/1099420015 conn(0x7fe10c06ae30 msgr2=0x7fe10c15d0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:22.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.430+0000 7fe110b4a700 1 -- 172.21.15.16:0/1099420015 shutdown_connections 2023-05-20T01:25:22.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.430+0000 7fe110b4a700 1 -- 172.21.15.16:0/1099420015 wait complete. 2023-05-20T01:25:22.437 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 57 2023-05-20T01:25:22.848 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 57, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-05-20T01:25:11.987993+0000', 'modified': '2023-05-20T01:25:15.395667+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7104}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7104': {'gid': 7104, 'name': 'c', 'rank': 0, 'incarnation': 53, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.16:6837/28561908', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6834', 'nonce': 28561908}, {'type': 'v1', 'addr': '172.21.15.16:6837', 'nonce': 28561908}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-05-20T01:25:22.849 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-05-20T01:25:22.849 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-05-20T01:25:22.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.978+0000 7fd0f28f5700 1 Processor -- start 2023-05-20T01:25:22.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.978+0000 7fd0f28f5700 1 -- start start 2023-05-20T01:25:22.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.978+0000 7fd0f28f5700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fd0ec06f450 0x7fd0ec0cf890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:22.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.978+0000 7fd0f28f5700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd0ec06d4b0 0x7fd0ec161080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:22.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.978+0000 7fd0f28f5700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0ec0cc510 con 0x7fd0ec06a510 2023-05-20T01:25:22.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.978+0000 7fd0f28f5700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0ec1615c0 con 0x7fd0ec06d4b0 2023-05-20T01:25:22.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.978+0000 7fd0f28f5700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0ec071ed0 con 0x7fd0ec06f450 2023-05-20T01:25:22.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.979+0000 7fd0eb7fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd0ec06d4b0 0x7fd0ec161080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:22.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.979+0000 7fd0eb7fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd0ec06d4b0 0x7fd0ec161080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36372/0 (socket says 172.21.15.16:36372) 2023-05-20T01:25:22.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.979+0000 7fd0eb7fe700 1 -- 172.21.15.16:0/3188107713 learned_addr learned my addr 172.21.15.16:0/3188107713 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:22.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.979+0000 7fd0ebfff700 1 --2- 172.21.15.16:0/3188107713 >> v2:172.21.15.132:3300/0 conn(0x7fd0ec06f450 0x7fd0ec0cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:22.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.979+0000 7fd0eb7fe700 1 -- 172.21.15.16:0/3188107713 >> v2:172.21.15.132:3300/0 conn(0x7fd0ec06f450 msgr2=0x7fd0ec0cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:22.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.979+0000 7fd0eb7fe700 1 --2- 172.21.15.16:0/3188107713 >> v2:172.21.15.132:3300/0 conn(0x7fd0ec06f450 0x7fd0ec0cf890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:22.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.979+0000 7fd0eb7fe700 1 -- 172.21.15.16:0/3188107713 >> v1:172.21.15.132:6789/0 conn(0x7fd0ec06a510 legacy=0x7fd0ec0cdb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:22.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.979+0000 7fd0eb7fe700 1 -- 172.21.15.16:0/3188107713 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0ec0721b0 con 0x7fd0ec06d4b0 2023-05-20T01:25:22.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.979+0000 7fd0eb7fe700 1 --2- 172.21.15.16:0/3188107713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0ec06d4b0 0x7fd0ec161080 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec0cdb00 tx=0x7fd0d4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8beeb18417d8373c server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:22.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.979+0000 7fd0eaffd700 1 -- 172.21.15.16:0/3188107713 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0d4004230 con 0x7fd0ec06d4b0 2023-05-20T01:25:22.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.979+0000 7fd0eaffd700 1 -- 172.21.15.16:0/3188107713 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0d4004390 con 0x7fd0ec06d4b0 2023-05-20T01:25:22.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.979+0000 7fd0eaffd700 1 -- 172.21.15.16:0/3188107713 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0d40036a0 con 0x7fd0ec06d4b0 2023-05-20T01:25:22.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.980+0000 7fd0f28f5700 1 -- 172.21.15.16:0/3188107713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0ec06d4b0 msgr2=0x7fd0ec161080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:22.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.980+0000 7fd0f28f5700 1 --2- 172.21.15.16:0/3188107713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0ec06d4b0 0x7fd0ec161080 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec0cdb00 tx=0x7fd0d4009bd0 comp rx=0 tx=0).stop 2023-05-20T01:25:22.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.980+0000 7fd0f28f5700 1 -- 172.21.15.16:0/3188107713 shutdown_connections 2023-05-20T01:25:22.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.980+0000 7fd0f28f5700 1 --2- 172.21.15.16:0/3188107713 >> v2:172.21.15.132:3300/0 conn(0x7fd0ec06f450 0x7fd0ec0cf890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:22.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.980+0000 7fd0f28f5700 1 --2- 172.21.15.16:0/3188107713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0ec06d4b0 0x7fd0ec161080 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:22.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.980+0000 7fd0f28f5700 1 -- 172.21.15.16:0/3188107713 >> 172.21.15.16:0/3188107713 conn(0x7fd0ec0cffb0 msgr2=0x7fd0ec0d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:22.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.980+0000 7fd0f28f5700 1 -- 172.21.15.16:0/3188107713 shutdown_connections 2023-05-20T01:25:22.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.980+0000 7fd0f28f5700 1 -- 172.21.15.16:0/3188107713 wait complete. 2023-05-20T01:25:22.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.981+0000 7fd0f28f5700 1 Processor -- start 2023-05-20T01:25:22.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.981+0000 7fd0f28f5700 1 -- start start 2023-05-20T01:25:22.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.982+0000 7fd0f28f5700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd0ec06a510 0x7fd0ec1f3320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:22.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.982+0000 7fd0f28f5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0ec06d4b0 0x7fd0ec1e8840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:22.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.982+0000 7fd0f28f5700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd0ec06f450 0x7fd0ec1e8d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:22.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.982+0000 7fd0f28f5700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0ec1f2e90 con 0x7fd0ec06d4b0 2023-05-20T01:25:22.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.982+0000 7fd0f28f5700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0ec1495e0 con 0x7fd0ec06f450 2023-05-20T01:25:22.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.982+0000 7fd0f28f5700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0ec1498c0 con 0x7fd0ec06a510 2023-05-20T01:25:22.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.982+0000 7fd0eb7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0ec06d4b0 0x7fd0ec1e8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:22.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.982+0000 7fd0eb7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0ec06d4b0 0x7fd0ec1e8840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36376/0 (socket says 172.21.15.16:36376) 2023-05-20T01:25:22.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.982+0000 7fd0eb7fe700 1 -- 172.21.15.16:0/3498235723 learned_addr learned my addr 172.21.15.16:0/3498235723 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:22.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.982+0000 7fd0f0e92700 1 --2- 172.21.15.16:0/3498235723 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd0ec06f450 0x7fd0ec1e8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:22.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.982+0000 7fd0ebfff700 1 --2- 172.21.15.16:0/3498235723 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd0ec06a510 0x7fd0ec1f3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:22.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.982+0000 7fd0eb7fe700 1 -- 172.21.15.16:0/3498235723 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd0ec06a510 msgr2=0x7fd0ec1f3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:22.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.982+0000 7fd0eb7fe700 1 --2- 172.21.15.16:0/3498235723 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd0ec06a510 0x7fd0ec1f3320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:22.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.982+0000 7fd0eb7fe700 1 -- 172.21.15.16:0/3498235723 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd0ec06f450 msgr2=0x7fd0ec1e8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:22.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.982+0000 7fd0eb7fe700 1 --2- 172.21.15.16:0/3498235723 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd0ec06f450 0x7fd0ec1e8d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:22.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.982+0000 7fd0eb7fe700 1 -- 172.21.15.16:0/3498235723 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0d4009550 con 0x7fd0ec06d4b0 2023-05-20T01:25:22.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.983+0000 7fd0eb7fe700 1 --2- 172.21.15.16:0/3498235723 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0ec06d4b0 0x7fd0ec1e8840 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fd0d4009130 tx=0x7fd0d4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:22.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.983+0000 7fd0e8ff9700 1 -- 172.21.15.16:0/3498235723 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0d4003d40 con 0x7fd0ec06d4b0 2023-05-20T01:25:22.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.983+0000 7fd0e8ff9700 1 -- 172.21.15.16:0/3498235723 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0d401f070 con 0x7fd0ec06d4b0 2023-05-20T01:25:22.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.983+0000 7fd0f28f5700 1 -- 172.21.15.16:0/3498235723 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0ec0cbe50 con 0x7fd0ec06d4b0 2023-05-20T01:25:22.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.983+0000 7fd0e8ff9700 1 -- 172.21.15.16:0/3498235723 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0d4011870 con 0x7fd0ec06d4b0 2023-05-20T01:25:22.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.983+0000 7fd0f28f5700 1 -- 172.21.15.16:0/3498235723 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0ec0cc1a0 con 0x7fd0ec06d4b0 2023-05-20T01:25:22.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.984+0000 7fd0e8ff9700 1 -- 172.21.15.16:0/3498235723 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd0d4021030 con 0x7fd0ec06d4b0 2023-05-20T01:25:22.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.984+0000 7fd0e8ff9700 1 --2- 172.21.15.16:0/3498235723 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd0cc06ad10 0x7fd0cc06d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:22.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.985+0000 7fd0e8ff9700 1 -- 172.21.15.16:0/3498235723 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fd0d408ab60 con 0x7fd0ec06d4b0 2023-05-20T01:25:22.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.985+0000 7fd0ebfff700 1 --2- 172.21.15.16:0/3498235723 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd0cc06ad10 0x7fd0cc06d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:22.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.985+0000 7fd0ebfff700 1 --2- 172.21.15.16:0/3498235723 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd0cc06ad10 0x7fd0cc06d1c0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc00b9b0 tx=0x7fd0dc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:22.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.987+0000 7fd0fea81b80 1 Processor -- start 2023-05-20T01:25:22.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.987+0000 7fd0fea81b80 1 -- start start 2023-05-20T01:25:22.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.987+0000 7fd0fea81b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5568c47da400 0x5568c47da7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:22.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.987+0000 7fd0fea81b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568c47d6700 0x5568c47af460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:22.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.988+0000 7fd0fea81b80 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5568c47ad870 0x5568c47af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:22.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.988+0000 7fd0fea81b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568c46b5010 con 0x5568c47d6700 2023-05-20T01:25:22.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.988+0000 7fd0fea81b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568c4707850 con 0x5568c47da400 2023-05-20T01:25:22.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.988+0000 7fd0fea81b80 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x5568c4700ce0 con 0x5568c47ad870 2023-05-20T01:25:22.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.988+0000 7fd0eb7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568c47d6700 0x5568c47af460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:22.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.988+0000 7fd0eb7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568c47d6700 0x5568c47af460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36386/0 (socket says 172.21.15.16:36386) 2023-05-20T01:25:22.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.988+0000 7fd0eb7fe700 1 -- 172.21.15.16:0/1941205586 learned_addr learned my addr 172.21.15.16:0/1941205586 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:22.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.988+0000 7fd0ebfff700 1 --2- 172.21.15.16:0/1941205586 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5568c47da400 0x5568c47da7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:22.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.988+0000 7fd0f0e92700 1 --2- 172.21.15.16:0/1941205586 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5568c47ad870 0x5568c47af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:22.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.988+0000 7fd0eb7fe700 1 -- 172.21.15.16:0/1941205586 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5568c47ad870 msgr2=0x5568c47af9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:22.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.988+0000 7fd0eb7fe700 1 --2- 172.21.15.16:0/1941205586 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5568c47ad870 0x5568c47af9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:22.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.988+0000 7fd0eb7fe700 1 -- 172.21.15.16:0/1941205586 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5568c47da400 msgr2=0x5568c47da7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:22.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.988+0000 7fd0eb7fe700 1 --2- 172.21.15.16:0/1941205586 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5568c47da400 0x5568c47da7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:22.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.988+0000 7fd0eb7fe700 1 -- 172.21.15.16:0/1941205586 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5568c45825e0 con 0x5568c47d6700 2023-05-20T01:25:22.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.988+0000 7fd0eb7fe700 1 --2- 172.21.15.16:0/1941205586 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568c47d6700 0x5568c47af460 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fd0d40037d0 tx=0x7fd0d4059160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35f0f040e297706b server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:22.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.988+0000 7fd0e1ffb700 1 -- 172.21.15.16:0/1941205586 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0d4006ea0 con 0x5568c47d6700 2023-05-20T01:25:22.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.988+0000 7fd0e1ffb700 1 -- 172.21.15.16:0/1941205586 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0d405d740 con 0x5568c47d6700 2023-05-20T01:25:22.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.988+0000 7fd0e1ffb700 1 -- 172.21.15.16:0/1941205586 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0d4061ad0 con 0x5568c47d6700 2023-05-20T01:25:22.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.989+0000 7fd0fea81b80 1 -- 172.21.15.16:0/1941205586 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568c47d6700 msgr2=0x5568c47af460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:22.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.989+0000 7fd0fea81b80 1 --2- 172.21.15.16:0/1941205586 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568c47d6700 0x5568c47af460 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fd0d40037d0 tx=0x7fd0d4059160 comp rx=0 tx=0).stop 2023-05-20T01:25:22.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.989+0000 7fd0fea81b80 1 -- 172.21.15.16:0/1941205586 shutdown_connections 2023-05-20T01:25:22.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.989+0000 7fd0fea81b80 1 --2- 172.21.15.16:0/1941205586 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5568c47ad870 0x5568c47af9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:22.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.989+0000 7fd0fea81b80 1 --2- 172.21.15.16:0/1941205586 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5568c47da400 0x5568c47da7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:22.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.989+0000 7fd0fea81b80 1 --2- 172.21.15.16:0/1941205586 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568c47d6700 0x5568c47af460 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:22.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.989+0000 7fd0fea81b80 1 -- 172.21.15.16:0/1941205586 >> 172.21.15.16:0/1941205586 conn(0x5568c47a4f30 msgr2=0x5568c47f5210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:22.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.989+0000 7fd0fea81b80 1 -- 172.21.15.16:0/1941205586 shutdown_connections 2023-05-20T01:25:22.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.989+0000 7fd0fea81b80 1 -- 172.21.15.16:0/1941205586 wait complete. 2023-05-20T01:25:22.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.990+0000 7fd0fea81b80 1 -- start start 2023-05-20T01:25:22.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.990+0000 7fd0fea81b80 1 Processor -- start 2023-05-20T01:25:23.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.991+0000 7fd0fea81b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5568c47ad870 0x5568c47d3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:23.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.991+0000 7fd0fea81b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568c47d6700 0x5568c47f8e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:23.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.991+0000 7fd0fea81b80 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5568c47da400 0x5568c47af710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:23.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.991+0000 7fd0fea81b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568c46a2890 con 0x5568c47d6700 2023-05-20T01:25:23.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.991+0000 7fd0fea81b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568c46a2530 con 0x5568c47ad870 2023-05-20T01:25:23.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.991+0000 7fd0fea81b80 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x5568c45848d0 con 0x5568c47da400 2023-05-20T01:25:23.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.991+0000 7fd0fea81b80 10 client.0 _ll_register_callbacks cb 0x5568c46c0730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-05-20T01:25:23.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.991+0000 7fd0eb7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568c47d6700 0x5568c47f8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:23.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.991+0000 7fd0eb7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568c47d6700 0x5568c47f8e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36392/0 (socket says 172.21.15.16:36392) 2023-05-20T01:25:23.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.991+0000 7fd0eb7fe700 1 -- 172.21.15.16:0/1503829243 learned_addr learned my addr 172.21.15.16:0/1503829243 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:23.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.991+0000 7fd0ebfff700 1 --2- 172.21.15.16:0/1503829243 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5568c47ad870 0x5568c47d3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:23.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.991+0000 7fd0f0e92700 1 --2- 172.21.15.16:0/1503829243 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5568c47da400 0x5568c47af710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:23.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.991+0000 7fd0eb7fe700 1 -- 172.21.15.16:0/1503829243 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5568c47da400 msgr2=0x5568c47af710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:23.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.991+0000 7fd0eb7fe700 1 --2- 172.21.15.16:0/1503829243 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5568c47da400 0x5568c47af710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:23.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.991+0000 7fd0eb7fe700 1 -- 172.21.15.16:0/1503829243 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5568c47ad870 msgr2=0x5568c47d3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:23.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.991+0000 7fd0eb7fe700 1 --2- 172.21.15.16:0/1503829243 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5568c47ad870 0x5568c47d3240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:23.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.991+0000 7fd0eb7fe700 1 -- 172.21.15.16:0/1503829243 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0ec0cc1a0 con 0x5568c47d6700 2023-05-20T01:25:23.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.991+0000 7fd0f0e92700 1 --2- 172.21.15.16:0/1503829243 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5568c47da400 0x5568c47af710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:25:23.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.991+0000 7fd0eb7fe700 1 --2- 172.21.15.16:0/1503829243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568c47d6700 0x5568c47f8e20 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fd0d4059710 tx=0x7fd0d405c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=362f7f9af807eeaf server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:23.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.991+0000 7fd0caffd700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-05-20T01:25:23.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.991+0000 7fd0caffd700 1 -- 172.21.15.16:0/1503829243 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0d4006ea0 con 0x5568c47d6700 2023-05-20T01:25:23.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.991+0000 7fd0caffd700 1 -- 172.21.15.16:0/1503829243 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fd0d4058c70 con 0x5568c47d6700 2023-05-20T01:25:23.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.991+0000 7fd0fea81b80 1 -- 172.21.15.16:0/1503829243 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5568c47a26f0 con 0x5568c47d6700 2023-05-20T01:25:23.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.991+0000 7fd0caffd700 1 -- 172.21.15.16:0/1503829243 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd0d4011ea0 con 0x5568c47d6700 2023-05-20T01:25:23.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.991+0000 7fd0fea81b80 10 client.7272 fetch_fsmap learned FSMap version 57 2023-05-20T01:25:23.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.991+0000 7fd0fea81b80 1 -- 172.21.15.16:0/1503829243 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=2+}) v3 -- 0x5568c46c7140 con 0x5568c47d6700 2023-05-20T01:25:23.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.992+0000 7fd0caffd700 1 -- 172.21.15.16:0/1503829243 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5568c47a26f0 con 0x5568c47d6700 2023-05-20T01:25:23.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.992+0000 7fd0caffd700 1 -- 172.21.15.16:0/1503829243 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 57) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fd0d4061d20 con 0x5568c47d6700 2023-05-20T01:25:23.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.992+0000 7fd0fea81b80 10 client.7272 fetch_fsmap finished waiting for FSMap version 57 2023-05-20T01:25:23.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.992+0000 7fd0fea81b80 10 client.7272 resolve_mds: resolved name 'c' to daemon mds.c 2023-05-20T01:25:23.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.992+0000 7fd0fea81b80 20 client.7272 populate_metadata read hostname 'smithi016' 2023-05-20T01:25:23.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.992+0000 7fd0fea81b80 1 --2- 172.21.15.16:0/1503829243 >> [v2:172.21.15.16:6834/28561908,v1:172.21.15.16:6837/28561908] conn(0x5568c47ed680 0x5568c47eda50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:23.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.992+0000 7fd0fea81b80 4 client.7272 mds_command: new command op to 7104 tid=0[{"prefix": "get_command_descriptions"}] 2023-05-20T01:25:23.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.992+0000 7fd0fea81b80 1 -- 172.21.15.16:0/1503829243 --> [v2:172.21.15.16:6834/28561908,v1:172.21.15.16:6837/28561908] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5568c46c3050 con 0x5568c47ed680 2023-05-20T01:25:23.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.992+0000 7fd0ebfff700 1 --2- 172.21.15.16:0/1503829243 >> [v2:172.21.15.16:6834/28561908,v1:172.21.15.16:6837/28561908] conn(0x5568c47ed680 0x5568c47eda50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:23.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.992+0000 7fd0caffd700 10 client.7272 ms_handle_connect on v2:172.21.15.16:6834/28561908 2023-05-20T01:25:23.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.992+0000 7fd0ebfff700 1 --2- 172.21.15.16:0/1503829243 >> [v2:172.21.15.16:6834/28561908,v1:172.21.15.16:6837/28561908] conn(0x5568c47ed680 0x5568c47eda50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fdcce24a35095f82 server_cookie=cda7cdc550d0ef08 in_seq=0 out_seq=0 2023-05-20T01:25:23.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.993+0000 7fd0caffd700 1 -- 172.21.15.16:0/1503829243 <== mds.0 v2:172.21.15.16:6834/28561908 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5568c46c3050 con 0x5568c47ed680 2023-05-20T01:25:23.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.993+0000 7fd0caffd700 10 client.7272 handle_command_reply: tid=0 2023-05-20T01:25:23.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.993+0000 7fd0fea81b80 1 client.7272 shutdown 2023-05-20T01:25:23.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.993+0000 7fd0fea81b80 1 -- 172.21.15.16:0/1503829243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568c47d6700 msgr2=0x5568c47f8e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:23.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.993+0000 7fd0fea81b80 1 --2- 172.21.15.16:0/1503829243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568c47d6700 0x5568c47f8e20 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fd0d4059710 tx=0x7fd0d405c040 comp rx=0 tx=0).stop 2023-05-20T01:25:23.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.993+0000 7fd0fea81b80 1 -- 172.21.15.16:0/1503829243 shutdown_connections 2023-05-20T01:25:23.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.993+0000 7fd0fea81b80 1 --2- 172.21.15.16:0/1503829243 >> [v2:172.21.15.16:6834/28561908,v1:172.21.15.16:6837/28561908] conn(0x5568c47ed680 0x5568c47eda50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:23.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.993+0000 7fd0fea81b80 1 --2- 172.21.15.16:0/1503829243 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5568c47da400 0x5568c47af710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:23.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.993+0000 7fd0fea81b80 1 --2- 172.21.15.16:0/1503829243 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5568c47ad870 0x5568c47d3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:23.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.993+0000 7fd0caffd700 0 client.7272 ms_handle_reset on v2:172.21.15.16:6834/28561908 2023-05-20T01:25:23.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.993+0000 7fd0fea81b80 1 --2- 172.21.15.16:0/1503829243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568c47d6700 0x5568c47f8e20 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:23.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.993+0000 7fd0fea81b80 1 -- 172.21.15.16:0/1503829243 >> 172.21.15.16:0/1503829243 conn(0x5568c47a4f30 msgr2=0x5568c47d6b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:23.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.993+0000 7fd0fea81b80 1 -- 172.21.15.16:0/1503829243 shutdown_connections 2023-05-20T01:25:23.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.993+0000 7fd0fea81b80 1 -- 172.21.15.16:0/1503829243 wait complete. 2023-05-20T01:25:23.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:22.993+0000 7fd0fea81b80 20 client.7272 trim_cache size 0 max 16384 2023-05-20T01:25:23.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.011+0000 7fd0fea81b80 1 Processor -- start 2023-05-20T01:25:23.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.011+0000 7fd0fea81b80 1 -- start start 2023-05-20T01:25:23.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.012+0000 7fd0fea81b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5568c47ad870 0x5568c47d6130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:23.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.012+0000 7fd0fea81b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568c47d6700 0x5568c47f4a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:23.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.012+0000 7fd0fea81b80 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5568c47da400 0x5568c47ee4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:23.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.012+0000 7fd0fea81b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568c47eda50 con 0x5568c47d6700 2023-05-20T01:25:23.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.012+0000 7fd0fea81b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568c4804040 con 0x5568c47ad870 2023-05-20T01:25:23.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.012+0000 7fd0fea81b80 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x5568c47a4d40 con 0x5568c47da400 2023-05-20T01:25:23.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.012+0000 7fd0eb7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568c47d6700 0x5568c47f4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:23.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.012+0000 7fd0eb7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568c47d6700 0x5568c47f4a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36398/0 (socket says 172.21.15.16:36398) 2023-05-20T01:25:23.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.012+0000 7fd0eb7fe700 1 -- 172.21.15.16:0/3724685847 learned_addr learned my addr 172.21.15.16:0/3724685847 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:23.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.012+0000 7fd0eb7fe700 1 -- 172.21.15.16:0/3724685847 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5568c47da400 msgr2=0x5568c47ee4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:23.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.012+0000 7fd0eb7fe700 1 --2- 172.21.15.16:0/3724685847 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5568c47da400 0x5568c47ee4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:23.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.012+0000 7fd0eb7fe700 1 -- 172.21.15.16:0/3724685847 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5568c47ad870 msgr2=0x5568c47d6130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:23.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.012+0000 7fd0eb7fe700 1 --2- 172.21.15.16:0/3724685847 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5568c47ad870 0x5568c47d6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:23.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.012+0000 7fd0eb7fe700 1 -- 172.21.15.16:0/3724685847 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5568c46c6e90 con 0x5568c47d6700 2023-05-20T01:25:23.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.013+0000 7fd0eb7fe700 1 --2- 172.21.15.16:0/3724685847 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568c47d6700 0x5568c47f4a80 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fd0d405abc0 tx=0x7fd0d4083000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40dd693a8cdaa38b server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:23.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.013+0000 7fd0e27fc700 1 -- 172.21.15.16:0/3724685847 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0d4006ea0 con 0x5568c47d6700 2023-05-20T01:25:23.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.013+0000 7fd0e27fc700 1 -- 172.21.15.16:0/3724685847 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0d405ae60 con 0x5568c47d6700 2023-05-20T01:25:23.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.013+0000 7fd0e27fc700 1 -- 172.21.15.16:0/3724685847 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0d4061880 con 0x5568c47d6700 2023-05-20T01:25:23.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.013+0000 7fd0fea81b80 1 -- 172.21.15.16:0/3724685847 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568c47d6700 msgr2=0x5568c47f4a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:23.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.013+0000 7fd0fea81b80 1 --2- 172.21.15.16:0/3724685847 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568c47d6700 0x5568c47f4a80 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fd0d405abc0 tx=0x7fd0d4083000 comp rx=0 tx=0).stop 2023-05-20T01:25:23.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.013+0000 7fd0fea81b80 1 -- 172.21.15.16:0/3724685847 shutdown_connections 2023-05-20T01:25:23.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.013+0000 7fd0fea81b80 1 --2- 172.21.15.16:0/3724685847 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5568c47da400 0x5568c47ee4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:23.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.013+0000 7fd0fea81b80 1 --2- 172.21.15.16:0/3724685847 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5568c47ad870 0x5568c47d6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:23.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.013+0000 7fd0fea81b80 1 --2- 172.21.15.16:0/3724685847 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568c47d6700 0x5568c47f4a80 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:23.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.013+0000 7fd0fea81b80 1 -- 172.21.15.16:0/3724685847 >> 172.21.15.16:0/3724685847 conn(0x5568c47a4f30 msgr2=0x5568c47ddee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:23.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.013+0000 7fd0fea81b80 1 -- 172.21.15.16:0/3724685847 shutdown_connections 2023-05-20T01:25:23.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.013+0000 7fd0fea81b80 1 -- 172.21.15.16:0/3724685847 wait complete. 2023-05-20T01:25:23.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.014+0000 7fd0fea81b80 1 -- start start 2023-05-20T01:25:23.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.014+0000 7fd0fea81b80 1 Processor -- start 2023-05-20T01:25:23.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.015+0000 7fd0fea81b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5568c47ad870 0x5568c47cf790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:23.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.015+0000 7fd0fea81b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568c47d6700 0x5568c485b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:23.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.015+0000 7fd0fea81b80 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5568c47da400 0x5568c485bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:23.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.015+0000 7fd0fea81b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568c485b0a0 con 0x5568c47d6700 2023-05-20T01:25:23.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.015+0000 7fd0fea81b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568c47d0020 con 0x5568c47ad870 2023-05-20T01:25:23.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.015+0000 7fd0fea81b80 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x5568c47eebf0 con 0x5568c47da400 2023-05-20T01:25:23.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.015+0000 7fd0fea81b80 10 client.0 _ll_register_callbacks cb 0x7fd0d4085070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-05-20T01:25:23.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.015+0000 7fd0eb7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568c47d6700 0x5568c485b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:23.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.015+0000 7fd0eb7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568c47d6700 0x5568c485b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36406/0 (socket says 172.21.15.16:36406) 2023-05-20T01:25:23.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.015+0000 7fd0eb7fe700 1 -- 172.21.15.16:0/495852417 learned_addr learned my addr 172.21.15.16:0/495852417 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:23.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.015+0000 7fd0ebfff700 1 --2- 172.21.15.16:0/495852417 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5568c47ad870 0x5568c47cf790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:23.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.015+0000 7fd0eb7fe700 1 -- 172.21.15.16:0/495852417 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5568c47da400 msgr2=0x5568c485bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:23.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.015+0000 7fd0f0e92700 1 --2- 172.21.15.16:0/495852417 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5568c47da400 0x5568c485bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:23.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.015+0000 7fd0eb7fe700 1 --2- 172.21.15.16:0/495852417 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5568c47da400 0x5568c485bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:23.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.015+0000 7fd0eb7fe700 1 -- 172.21.15.16:0/495852417 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5568c47ad870 msgr2=0x5568c47cf790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:23.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.015+0000 7fd0eb7fe700 1 --2- 172.21.15.16:0/495852417 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5568c47ad870 0x5568c47cf790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:23.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.015+0000 7fd0eb7fe700 1 -- 172.21.15.16:0/495852417 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0d4074070 con 0x5568c47d6700 2023-05-20T01:25:23.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.015+0000 7fd0ebfff700 1 --2- 172.21.15.16:0/495852417 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5568c47ad870 0x5568c47cf790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:25:23.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.016+0000 7fd0eb7fe700 1 --2- 172.21.15.16:0/495852417 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568c47d6700 0x5568c485b7b0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fd0d4055c80 tx=0x7fd0d4055ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c36a4859dfd85ec server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:23.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.016+0000 7fd0caffd700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-05-20T01:25:23.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.016+0000 7fd0caffd700 1 -- 172.21.15.16:0/495852417 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0d4061840 con 0x5568c47d6700 2023-05-20T01:25:23.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.016+0000 7fd0caffd700 1 -- 172.21.15.16:0/495852417 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fd0d4058c70 con 0x5568c47d6700 2023-05-20T01:25:23.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.016+0000 7fd0fea81b80 1 -- 172.21.15.16:0/495852417 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd0d405d740 con 0x5568c47d6700 2023-05-20T01:25:23.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.016+0000 7fd0caffd700 1 -- 172.21.15.16:0/495852417 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd0d40119d0 con 0x5568c47d6700 2023-05-20T01:25:23.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.016+0000 7fd0fea81b80 10 client.7278 fetch_fsmap learned FSMap version 57 2023-05-20T01:25:23.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.016+0000 7fd0fea81b80 1 -- 172.21.15.16:0/495852417 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=2+}) v3 -- 0x5568c4517c10 con 0x5568c47d6700 2023-05-20T01:25:23.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.016+0000 7fd0caffd700 1 -- 172.21.15.16:0/495852417 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0d405d740 con 0x5568c47d6700 2023-05-20T01:25:23.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.016+0000 7fd0caffd700 1 -- 172.21.15.16:0/495852417 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 57) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fd0d4061d20 con 0x5568c47d6700 2023-05-20T01:25:23.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.016+0000 7fd0fea81b80 10 client.7278 fetch_fsmap finished waiting for FSMap version 57 2023-05-20T01:25:23.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.016+0000 7fd0fea81b80 10 client.7278 resolve_mds: resolved name 'c' to daemon mds.c 2023-05-20T01:25:23.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.016+0000 7fd0fea81b80 20 client.7278 populate_metadata read hostname 'smithi016' 2023-05-20T01:25:23.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.016+0000 7fd0fea81b80 1 --2- 172.21.15.16:0/495852417 >> [v2:172.21.15.16:6834/28561908,v1:172.21.15.16:6837/28561908] conn(0x5568c47ddf60 0x5568c47d59f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:23.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.016+0000 7fd0fea81b80 4 client.7278 mds_command: new command op to 7104 tid=0[{"prefix": "status"}] 2023-05-20T01:25:23.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.016+0000 7fd0fea81b80 1 -- 172.21.15.16:0/495852417 --> [v2:172.21.15.16:6834/28561908,v1:172.21.15.16:6837/28561908] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5568c46cf610 con 0x5568c47ddf60 2023-05-20T01:25:23.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.016+0000 7fd0ebfff700 1 --2- 172.21.15.16:0/495852417 >> [v2:172.21.15.16:6834/28561908,v1:172.21.15.16:6837/28561908] conn(0x5568c47ddf60 0x5568c47d59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:23.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.016+0000 7fd0caffd700 10 client.7278 ms_handle_connect on v2:172.21.15.16:6834/28561908 2023-05-20T01:25:23.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.016+0000 7fd0ebfff700 1 --2- 172.21.15.16:0/495852417 >> [v2:172.21.15.16:6834/28561908,v1:172.21.15.16:6837/28561908] conn(0x5568c47ddf60 0x5568c47d59f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1af712be3a3a24b server_cookie=f8a7f49652b98833 in_seq=0 out_seq=0 2023-05-20T01:25:23.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.017+0000 7fd0caffd700 1 -- 172.21.15.16:0/495852417 <== mds.0 v2:172.21.15.16:6834/28561908 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+319 (crc 0 0 0) 0x5568c46cf610 con 0x5568c47ddf60 2023-05-20T01:25:23.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.017+0000 7fd0caffd700 10 client.7278 handle_command_reply: tid=0 2023-05-20T01:25:23.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.017+0000 7fd0fea81b80 1 client.7278 shutdown 2023-05-20T01:25:23.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.017+0000 7fd0fea81b80 1 -- 172.21.15.16:0/495852417 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568c47d6700 msgr2=0x5568c485b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:23.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.017+0000 7fd0fea81b80 1 --2- 172.21.15.16:0/495852417 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568c47d6700 0x5568c485b7b0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fd0d4055c80 tx=0x7fd0d4055ac0 comp rx=0 tx=0).stop 2023-05-20T01:25:23.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.017+0000 7fd0fea81b80 1 -- 172.21.15.16:0/495852417 shutdown_connections 2023-05-20T01:25:23.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.017+0000 7fd0fea81b80 1 --2- 172.21.15.16:0/495852417 >> [v2:172.21.15.16:6834/28561908,v1:172.21.15.16:6837/28561908] conn(0x5568c47ddf60 0x5568c47d59f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:23.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.017+0000 7fd0fea81b80 1 --2- 172.21.15.16:0/495852417 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5568c47da400 0x5568c485bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:23.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.017+0000 7fd0fea81b80 1 --2- 172.21.15.16:0/495852417 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5568c47ad870 0x5568c47cf790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:23.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.017+0000 7fd0caffd700 0 client.7278 ms_handle_reset on v2:172.21.15.16:6834/28561908 2023-05-20T01:25:23.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.017+0000 7fd0fea81b80 1 --2- 172.21.15.16:0/495852417 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568c47d6700 0x5568c485b7b0 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:23.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.017+0000 7fd0fea81b80 1 -- 172.21.15.16:0/495852417 >> 172.21.15.16:0/495852417 conn(0x5568c47a4f30 msgr2=0x5568c47d3050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:23.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.017+0000 7fd0fea81b80 1 -- 172.21.15.16:0/495852417 shutdown_connections 2023-05-20T01:25:23.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.017+0000 7fd0fea81b80 1 -- 172.21.15.16:0/495852417 wait complete. 2023-05-20T01:25:23.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.017+0000 7fd0fea81b80 20 client.7278 trim_cache size 0 max 16384 2023-05-20T01:25:23.032 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-05-20T01:25:23.032 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "c2245cc5-ecab-4087-a59b-bc81db901676", 2023-05-20T01:25:23.032 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2023-05-20T01:25:23.032 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 7104, 2023-05-20T01:25:23.032 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2023-05-20T01:25:23.032 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2023-05-20T01:25:23.033 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2023-05-20T01:25:23.033 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 11.028214195, 2023-05-20T01:25:23.033 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 57, 2023-05-20T01:25:23.033 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 88, 2023-05-20T01:25:23.033 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 88, 2023-05-20T01:25:23.033 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 16.347150917 2023-05-20T01:25:23.033 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-05-20T01:25:23.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.018+0000 7fd0f28f5700 1 -- 172.21.15.16:0/3498235723 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd0cc06ad10 msgr2=0x7fd0cc06d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:23.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.018+0000 7fd0f28f5700 1 --2- 172.21.15.16:0/3498235723 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd0cc06ad10 0x7fd0cc06d1c0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc00b9b0 tx=0x7fd0dc008040 comp rx=0 tx=0).stop 2023-05-20T01:25:23.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.018+0000 7fd0f28f5700 1 -- 172.21.15.16:0/3498235723 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0ec06d4b0 msgr2=0x7fd0ec1e8840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:23.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.018+0000 7fd0f28f5700 1 --2- 172.21.15.16:0/3498235723 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0ec06d4b0 0x7fd0ec1e8840 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fd0d4009130 tx=0x7fd0d4004750 comp rx=0 tx=0).stop 2023-05-20T01:25:23.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.018+0000 7fd0f28f5700 1 -- 172.21.15.16:0/3498235723 shutdown_connections 2023-05-20T01:25:23.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.018+0000 7fd0f28f5700 1 --2- 172.21.15.16:0/3498235723 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd0ec06f450 0x7fd0ec1e8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:23.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.018+0000 7fd0f28f5700 1 --2- 172.21.15.16:0/3498235723 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd0cc06ad10 0x7fd0cc06d1c0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:23.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.018+0000 7fd0f28f5700 1 --2- 172.21.15.16:0/3498235723 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd0ec06a510 0x7fd0ec1f3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:23.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.018+0000 7fd0f28f5700 1 --2- 172.21.15.16:0/3498235723 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0ec06d4b0 0x7fd0ec1e8840 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:23.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.018+0000 7fd0f28f5700 1 -- 172.21.15.16:0/3498235723 >> 172.21.15.16:0/3498235723 conn(0x7fd0ec0cffb0 msgr2=0x7fd0ec0c7120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:23.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.018+0000 7fd0f28f5700 1 -- 172.21.15.16:0/3498235723 shutdown_connections 2023-05-20T01:25:23.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.018+0000 7fd0f28f5700 1 -- 172.21.15.16:0/3498235723 wait complete. 2023-05-20T01:25:23.037 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-05-20T01:25:23.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.157+0000 7f6796073700 1 Processor -- start 2023-05-20T01:25:23.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.157+0000 7f6796073700 1 -- start start 2023-05-20T01:25:23.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.158+0000 7f6796073700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f679006d4b0 0x7f67900cf890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:23.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.158+0000 7f6796073700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67900cc510 con 0x7f679006f450 2023-05-20T01:25:23.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.158+0000 7f6796073700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6790071ed0 con 0x7f679006a510 2023-05-20T01:25:23.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.158+0000 7f6796073700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67900721b0 con 0x7f679006d4b0 2023-05-20T01:25:23.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.158+0000 7f678effd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f679006d4b0 0x7f67900cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:23.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.158+0000 7f678f7fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f679006f450 0x7f6790161080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:47940/0 (socket says 172.21.15.16:47940) 2023-05-20T01:25:23.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.158+0000 7f678effd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f679006d4b0 0x7f67900cf890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36414/0 (socket says 172.21.15.16:36414) 2023-05-20T01:25:23.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.158+0000 7f678f7fe700 1 -- 172.21.15.16:0/1360615993 learned_addr learned my addr 172.21.15.16:0/1360615993 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:25:23.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.158+0000 7f678effd700 1 -- 172.21.15.16:0/1360615993 >> v1:172.21.15.132:6789/0 conn(0x7f679006a510 legacy=0x7f67900cc9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:23.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.158+0000 7f678effd700 1 -- 172.21.15.16:0/1360615993 >> v1:172.21.15.16:6789/0 conn(0x7f679006f450 legacy=0x7f6790161080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:23.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.159+0000 7f678effd700 1 -- 172.21.15.16:0/1360615993 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67900cc1a0 con 0x7f679006d4b0 2023-05-20T01:25:23.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.159+0000 7f678effd700 1 --2- 172.21.15.16:0/1360615993 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f679006d4b0 0x7f67900cf890 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f6784009840 tx=0x7f6784009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3d7dc1ad209e469 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:23.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.159+0000 7f678e7fc700 1 -- 172.21.15.16:0/1360615993 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6784004230 con 0x7f679006d4b0 2023-05-20T01:25:23.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.159+0000 7f678e7fc700 1 -- 172.21.15.16:0/1360615993 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6784004390 con 0x7f679006d4b0 2023-05-20T01:25:23.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.159+0000 7f678e7fc700 1 -- 172.21.15.16:0/1360615993 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67840036a0 con 0x7f679006d4b0 2023-05-20T01:25:23.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.159+0000 7f6796073700 1 -- 172.21.15.16:0/1360615993 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f679006d4b0 msgr2=0x7f67900cf890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:23.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.159+0000 7f6796073700 1 --2- 172.21.15.16:0/1360615993 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f679006d4b0 0x7f67900cf890 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f6784009840 tx=0x7f6784009b50 comp rx=0 tx=0).stop 2023-05-20T01:25:23.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.159+0000 7f6796073700 1 -- 172.21.15.16:0/1360615993 shutdown_connections 2023-05-20T01:25:23.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.159+0000 7f6796073700 1 --2- 172.21.15.16:0/1360615993 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f679006d4b0 0x7f67900cf890 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:23.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.159+0000 7f6796073700 1 -- 172.21.15.16:0/1360615993 >> 172.21.15.16:0/1360615993 conn(0x7f67900cffb0 msgr2=0x7f67900d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:23.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.159+0000 7f6796073700 1 -- 172.21.15.16:0/1360615993 shutdown_connections 2023-05-20T01:25:23.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.159+0000 7f6796073700 1 -- 172.21.15.16:0/1360615993 wait complete. 2023-05-20T01:25:23.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.161+0000 7f6796073700 1 Processor -- start 2023-05-20T01:25:23.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.161+0000 7f6796073700 1 -- start start 2023-05-20T01:25:23.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.162+0000 7f6796073700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f679006a510 0x7f679014fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:23.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.162+0000 7f6796073700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f679006d4b0 0x7f6790150fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:23.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.162+0000 7f6796073700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f679006f450 0x7f6790151500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:23.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.162+0000 7f6796073700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67901f2750 con 0x7f679006a510 2023-05-20T01:25:23.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.162+0000 7f6796073700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6790157290 con 0x7f679006d4b0 2023-05-20T01:25:23.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.162+0000 7f6796073700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6790157570 con 0x7f679006f450 2023-05-20T01:25:23.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.162+0000 7f678f7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f679006a510 0x7f679014fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:23.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.162+0000 7f678f7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f679006a510 0x7f679014fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36418/0 (socket says 172.21.15.16:36418) 2023-05-20T01:25:23.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.162+0000 7f678f7fe700 1 -- 172.21.15.16:0/420918756 learned_addr learned my addr 172.21.15.16:0/420918756 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:23.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.162+0000 7f678effd700 1 --2- 172.21.15.16:0/420918756 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f679006d4b0 0x7f6790150fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:23.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.162+0000 7f678ffff700 1 --2- 172.21.15.16:0/420918756 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f679006f450 0x7f6790151500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:23.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.162+0000 7f678f7fe700 1 -- 172.21.15.16:0/420918756 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f679006f450 msgr2=0x7f6790151500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:23.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.162+0000 7f678f7fe700 1 --2- 172.21.15.16:0/420918756 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f679006f450 0x7f6790151500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:23.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.162+0000 7f678f7fe700 1 -- 172.21.15.16:0/420918756 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f679006d4b0 msgr2=0x7f6790150fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:23.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.162+0000 7f678f7fe700 1 --2- 172.21.15.16:0/420918756 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f679006d4b0 0x7f6790150fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:23.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.162+0000 7f678f7fe700 1 -- 172.21.15.16:0/420918756 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67840094a0 con 0x7f679006a510 2023-05-20T01:25:23.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.162+0000 7f678effd700 1 --2- 172.21.15.16:0/420918756 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f679006d4b0 0x7f6790150fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:25:23.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.162+0000 7f678f7fe700 1 --2- 172.21.15.16:0/420918756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f679006a510 0x7f679014fdd0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f678000c430 tx=0x7f678000c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:23.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.162+0000 7f6777fff700 1 -- 172.21.15.16:0/420918756 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6780005d70 con 0x7f679006a510 2023-05-20T01:25:23.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.162+0000 7f6777fff700 1 -- 172.21.15.16:0/420918756 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67800070d0 con 0x7f679006a510 2023-05-20T01:25:23.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.162+0000 7f6796073700 1 -- 172.21.15.16:0/420918756 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f679006a950 con 0x7f679006a510 2023-05-20T01:25:23.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.162+0000 7f6777fff700 1 -- 172.21.15.16:0/420918756 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6780007460 con 0x7f679006a510 2023-05-20T01:25:23.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.162+0000 7f6796073700 1 -- 172.21.15.16:0/420918756 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67900cbe50 con 0x7f679006a510 2023-05-20T01:25:23.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.163+0000 7f6796073700 1 -- 172.21.15.16:0/420918756 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f679006aae0 con 0x7f679006a510 2023-05-20T01:25:23.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.164+0000 7f6777fff700 1 -- 172.21.15.16:0/420918756 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f67800075c0 con 0x7f679006a510 2023-05-20T01:25:23.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.164+0000 7f6777fff700 1 --2- 172.21.15.16:0/420918756 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f677c06ace0 0x7f677c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:23.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.164+0000 7f6777fff700 1 -- 172.21.15.16:0/420918756 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f6780088a20 con 0x7f679006a510 2023-05-20T01:25:23.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.164+0000 7f678effd700 1 --2- 172.21.15.16:0/420918756 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f677c06ace0 0x7f677c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:23.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.165+0000 7f678effd700 1 --2- 172.21.15.16:0/420918756 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f677c06ace0 0x7f677c06d190 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f67900b5940 tx=0x7f6784008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:23.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.169+0000 7f6777fff700 1 -- 172.21.15.16:0/420918756 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f6780053990 con 0x7f679006a510 2023-05-20T01:25:23.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.411+0000 7f6796073700 1 -- 172.21.15.16:0/420918756 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f67900747b0 con 0x7f679006a510 2023-05-20T01:25:23.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.412+0000 7f6777fff700 1 -- 172.21.15.16:0/420918756 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v57) v1 ==== 53+0+73 (secure 0 0 0) 0x7f6780017070 con 0x7f679006a510 2023-05-20T01:25:23.416 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-05-20T01:25:23.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.414+0000 7f6796073700 1 -- 172.21.15.16:0/420918756 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f677c06ace0 msgr2=0x7f677c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:23.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.414+0000 7f6796073700 1 --2- 172.21.15.16:0/420918756 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f677c06ace0 0x7f677c06d190 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f67900b5940 tx=0x7f6784008030 comp rx=0 tx=0).stop 2023-05-20T01:25:23.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.414+0000 7f6796073700 1 -- 172.21.15.16:0/420918756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f679006a510 msgr2=0x7f679014fdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:23.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.414+0000 7f6796073700 1 --2- 172.21.15.16:0/420918756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f679006a510 0x7f679014fdd0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f678000c430 tx=0x7f678000c7f0 comp rx=0 tx=0).stop 2023-05-20T01:25:23.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.414+0000 7f6796073700 1 -- 172.21.15.16:0/420918756 shutdown_connections 2023-05-20T01:25:23.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.414+0000 7f6796073700 1 --2- 172.21.15.16:0/420918756 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f677c06ace0 0x7f677c06d190 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:23.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.414+0000 7f6796073700 1 --2- 172.21.15.16:0/420918756 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f679006f450 0x7f6790151500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:23.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.414+0000 7f6796073700 1 --2- 172.21.15.16:0/420918756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f679006a510 0x7f679014fdd0 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:23.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.414+0000 7f6796073700 1 --2- 172.21.15.16:0/420918756 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f679006d4b0 0x7f6790150fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:23.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.414+0000 7f6796073700 1 -- 172.21.15.16:0/420918756 >> 172.21.15.16:0/420918756 conn(0x7f67900cffb0 msgr2=0x7f679006e850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:23.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.414+0000 7f6796073700 1 -- 172.21.15.16:0/420918756 shutdown_connections 2023-05-20T01:25:23.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:23.414+0000 7f6796073700 1 -- 172.21.15.16:0/420918756 wait complete. 2023-05-20T01:25:23.430 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-05-20T01:25:23.430 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-05-20T01:25:23.431 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2023-05-20T01:25:23.431 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-05-20T01:25:23.431 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-05-20T01:25:23.432 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-05-20T01:25:23.432 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-05-20T01:25:23.432 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-05-20T01:25:23.433 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-05-20T01:25:23.433 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:25:23.433 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2023-05-20T01:25:23.452 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-05-20T01:25:23.452 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-05-20T01:25:23.452 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2023-05-20T01:25:23.453 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-05-20T01:25:23.453 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2023-05-20T01:25:23.453 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-05-20T01:25:23.454 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-05-20T01:25:23.454 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2023-05-20T01:25:23.454 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-05-20T01:25:23.454 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2023-05-20T01:25:23.455 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-05-20T01:25:23.455 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2023-05-20T01:25:23.455 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-05-20T01:25:23.455 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 41910sec preferred_lft 41910sec 2023-05-20T01:25:23.456 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link 2023-05-20T01:25:23.456 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-05-20T01:25:23.456 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-05-20T01:25:23.456 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2023-05-20T01:25:23.457 INFO:teuthology.orchestra.run.smithi016.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-05-20T01:25:23.457 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0a:2e:0c:6b:5d:42 brd ff:ff:ff:ff:ff:ff 2023-05-20T01:25:23.457 INFO:teuthology.orchestra.run.smithi016.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-05-20T01:25:23.457 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-05-20T01:25:23.457 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::7c00:b2ff:fef4:946/64 scope link 2023-05-20T01:25:23.457 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-05-20T01:25:23.458 INFO:teuthology.orchestra.run.smithi016.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-05-20T01:25:23.458 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether d2:e4:70:a7:b9:4b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-05-20T01:25:23.458 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::d0e4:70ff:fea7:b94b/64 scope link 2023-05-20T01:25:23.458 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-05-20T01:25:23.458 INFO:teuthology.orchestra.run.smithi016.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-05-20T01:25:23.458 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0a:2e:0c:6b:5d:42 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-05-20T01:25:23.458 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::82e:cff:fe6b:5d42/64 scope link 2023-05-20T01:25:23.459 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-05-20T01:25:23.459 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:25:23.459 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-05-20T01:25:23.518 INFO:teuthology.orchestra.run.smithi016.stdout:1 2023-05-20T01:25:23.519 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:25:23.520 DEBUG:teuthology.orchestra.run.smithi016:> route 2023-05-20T01:25:23.579 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2023-05-20T01:25:23.579 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-05-20T01:25:23.579 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-05-20T01:25:23.579 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-05-20T01:25:23.579 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-05-20T01:25:23.580 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:25:23.580 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2023-05-20T01:25:23.581 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2023-05-20T01:25:23.581 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-05-20T01:25:23.581 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-05-20T01:25:23.581 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-05-20T01:25:23.582 DEBUG:teuthology.orchestra.run.smithi016:> ') 2023-05-20T01:25:23.685 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:25:23.686 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2023-05-20T01:25:23.702 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-05-20T01:25:23.703 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-05-20T01:25:23.703 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-05-20T01:25:23.703 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:25:23.704 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.0 up 2023-05-20T01:25:23.770 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-05-20T01:25:23.770 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-05-20T01:25:23.827 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-05-20T01:25:23.828 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-05-20T01:25:23.829 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-05-20T01:25:23.884 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-05-20T01:25:23.885 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:25:23.886 DEBUG:teuthology.orchestra.run.smithi016:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-05-20T01:25:23.971 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-05-20T01:25:23.971 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-05-20T01:25:24.003 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-05-20T01:25:24.003 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-05-20T01:25:24.003 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.132:3300,172.21.15.132:3301" 2023-05-20T01:25:24.003 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: trying mount with new device syntax: 0@c2245cc5-ecab-4087-a59b-bc81db901676.cephfs=/ 2023-05-20T01:25:24.004 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.16:3300/172.21.15.132:3300/172.21.15.132:3301" will pass to kernel 2023-05-20T01:25:24.004 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: v2 mount support check returned 2 2023-05-20T01:25:24.004 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: kernel does not support v2 syntax 2023-05-20T01:25:24.004 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: switching to using v1 address with old syntax 2023-05-20T01:25:24.039 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-05-20T01:25:24.039 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-05-20T01:25:24.039 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.132:3300,172.21.15.132:3301" 2023-05-20T01:25:24.040 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: trying mount with new device syntax: 0@c2245cc5-ecab-4087-a59b-bc81db901676.cephfs=/ 2023-05-20T01:25:24.040 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.16:3300/172.21.15.132:3300/172.21.15.132:3301" will pass to kernel 2023-05-20T01:25:24.040 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: v2 mount support check returned 2 2023-05-20T01:25:24.040 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: kernel does not support v2 syntax 2023-05-20T01:25:24.040 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: switching to using v1 address with old syntax 2023-05-20T01:25:24.040 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:6789,172.21.15.132:6789,172.21.15.132:6790" 2023-05-20T01:25:24.040 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.16:6789,172.21.15.132:6789,172.21.15.132:6790:/ 2023-05-20T01:25:24.041 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=c2245cc5-ecab-4087-a59b-bc81db901676" will pass to kernel 2023-05-20T01:25:24.045 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-05-20T01:25:24.045 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:25:24.046 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-05-20T01:25:24.082 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:25:24.083 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-05-20T01:25:24.162 INFO:teuthology.orchestra.run.smithi016.stdout:client7290 2023-05-20T01:25:24.162 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-05-20T01:25:24.163 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2023-05-20T01:25:24.221 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_extra --pg_num_min 64 2023-05-20T01:25:24.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:24.385+0000 7f52096cd700 1 Processor -- start 2023-05-20T01:25:24.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:24.385+0000 7f52096cd700 1 -- start start 2023-05-20T01:25:24.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:24.386+0000 7f52096cd700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f52040b2670 0x7f52040b60d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:24.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:24.386+0000 7f52096cd700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f52040d1860 0x7f52040d3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:24.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:24.386+0000 7f52096cd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f52040d3880 0x7f52040d3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:24.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:24.386+0000 7f52096cd700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52040d2eb0 con 0x7f52040d3880 2023-05-20T01:25:24.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:24.386+0000 7f52096cd700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52040cd970 con 0x7f52040d1860 2023-05-20T01:25:24.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:24.386+0000 7f52096cd700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52040cdc50 con 0x7f52040b2670 2023-05-20T01:25:24.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:24.386+0000 7f52037fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f52040d3880 0x7f52040d3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:24.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:24.386+0000 7f52037fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f52040d3880 0x7f52040d3d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36434/0 (socket says 172.21.15.16:36434) 2023-05-20T01:25:24.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:24.386+0000 7f52037fe700 1 -- 172.21.15.16:0/4049651839 learned_addr learned my addr 172.21.15.16:0/4049651839 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:24.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:24.386+0000 7f5202ffd700 1 --2- 172.21.15.16:0/4049651839 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f52040b2670 0x7f52040b60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:24.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:24.386+0000 7f52027fc700 1 --2- 172.21.15.16:0/4049651839 >> v2:172.21.15.132:3300/0 conn(0x7f52040d1860 0x7f52040d3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:24.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:24.387+0000 7f52037fe700 1 -- 172.21.15.16:0/4049651839 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f52040b2670 msgr2=0x7f52040b60d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:24.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:24.387+0000 7f52037fe700 1 --2- 172.21.15.16:0/4049651839 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f52040b2670 0x7f52040b60d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:24.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:24.387+0000 7f52037fe700 1 -- 172.21.15.16:0/4049651839 >> v2:172.21.15.132:3300/0 conn(0x7f52040d1860 msgr2=0x7f52040d3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:24.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:24.387+0000 7f52037fe700 1 --2- 172.21.15.16:0/4049651839 >> v2:172.21.15.132:3300/0 conn(0x7f52040d1860 0x7f52040d3340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:24.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:24.387+0000 7f52037fe700 1 -- 172.21.15.16:0/4049651839 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52040d2b40 con 0x7f52040d3880 2023-05-20T01:25:24.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:24.387+0000 7f52027fc700 1 --2- 172.21.15.16:0/4049651839 >> v2:172.21.15.132:3300/0 conn(0x7f52040d1860 0x7f52040d3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:25:24.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:24.387+0000 7f5202ffd700 1 --2- 172.21.15.16:0/4049651839 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f52040b2670 0x7f52040b60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:25:24.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:24.387+0000 7f52037fe700 1 --2- 172.21.15.16:0/4049651839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52040d3880 0x7f52040d3d00 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f520406c360 tx=0x7f51fc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46700e053391eb4b server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:24.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:24.387+0000 7f5201ffb700 1 -- 172.21.15.16:0/4049651839 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51fc00e070 con 0x7f52040d3880 2023-05-20T01:25:24.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:24.387+0000 7f5201ffb700 1 -- 172.21.15.16:0/4049651839 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51fc00b9c0 con 0x7f52040d3880 2023-05-20T01:25:24.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:24.387+0000 7f5201ffb700 1 -- 172.21.15.16:0/4049651839 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51fc00bd70 con 0x7f52040d3880 2023-05-20T01:25:24.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:24.387+0000 7f52096cd700 1 -- 172.21.15.16:0/4049651839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52040d3880 msgr2=0x7f52040d3d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:24.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:24.387+0000 7f52096cd700 1 --2- 172.21.15.16:0/4049651839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52040d3880 0x7f52040d3d00 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f520406c360 tx=0x7f51fc00b330 comp rx=0 tx=0).stop 2023-05-20T01:25:24.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:24.387+0000 7f52096cd700 1 -- 172.21.15.16:0/4049651839 shutdown_connections 2023-05-20T01:25:24.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:24.387+0000 7f52096cd700 1 --2- 172.21.15.16:0/4049651839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52040d3880 0x7f52040d3d00 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:24.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:24.387+0000 7f52096cd700 1 --2- 172.21.15.16:0/4049651839 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f52040b2670 0x7f52040b60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:24.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:24.387+0000 7f52096cd700 1 --2- 172.21.15.16:0/4049651839 >> v2:172.21.15.132:3300/0 conn(0x7f52040d1860 0x7f52040d3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:24.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:24.387+0000 7f52096cd700 1 -- 172.21.15.16:0/4049651839 >> 172.21.15.16:0/4049651839 conn(0x7f5204071c40 msgr2=0x7f5204072010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:24.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:24.388+0000 7f52096cd700 1 -- 172.21.15.16:0/4049651839 shutdown_connections 2023-05-20T01:25:24.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:24.388+0000 7f52096cd700 1 -- 172.21.15.16:0/4049651839 wait complete. 2023-05-20T01:25:24.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:24.389+0000 7f52096cd700 1 Processor -- start 2023-05-20T01:25:24.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:24.389+0000 7f52096cd700 1 -- start start 2023-05-20T01:25:24.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:24.390+0000 7f52096cd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f52040b2670 0x7f52041f17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:24.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:24.390+0000 7f52096cd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52040d1860 0x7f5204155320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:24.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:24.390+0000 7f52096cd700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f52040d3880 0x7f5204153410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:24.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:24.390+0000 7f52096cd700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f520415e140 con 0x7f52040d1860 2023-05-20T01:25:24.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:24.390+0000 7f52096cd700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5204155d00 con 0x7f52040b2670 2023-05-20T01:25:24.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:24.390+0000 7f52096cd700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5204155fe0 con 0x7f52040d3880 2023-05-20T01:25:24.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:24.390+0000 7f52027fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52040d1860 0x7f5204155320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:24.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:24.390+0000 7f52027fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52040d1860 0x7f5204155320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36438/0 (socket says 172.21.15.16:36438) 2023-05-20T01:25:24.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:24.390+0000 7f52027fc700 1 -- 172.21.15.16:0/3259965997 learned_addr learned my addr 172.21.15.16:0/3259965997 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:24.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:24.390+0000 7f52027fc700 1 -- 172.21.15.16:0/3259965997 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f52040d3880 msgr2=0x7f5204153410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:24.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:24.390+0000 7f52027fc700 1 --2- 172.21.15.16:0/3259965997 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f52040d3880 0x7f5204153410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:24.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:24.390+0000 7f52027fc700 1 -- 172.21.15.16:0/3259965997 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f52040b2670 msgr2=0x7f52041f17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:24.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:24.390+0000 7f52027fc700 1 --2- 172.21.15.16:0/3259965997 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f52040b2670 0x7f52041f17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:24.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:24.390+0000 7f52027fc700 1 -- 172.21.15.16:0/3259965997 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51fc009b30 con 0x7f52040d1860 2023-05-20T01:25:24.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:24.390+0000 7f52027fc700 1 --2- 172.21.15.16:0/3259965997 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52040d1860 0x7f5204155320 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f51f400c750 tx=0x7f51f400cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:24.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:24.390+0000 7f51eb7fe700 1 -- 172.21.15.16:0/3259965997 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f4011800 con 0x7f52040d1860 2023-05-20T01:25:24.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:24.390+0000 7f51eb7fe700 1 -- 172.21.15.16:0/3259965997 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51f4011960 con 0x7f52040d1860 2023-05-20T01:25:24.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:24.390+0000 7f52096cd700 1 -- 172.21.15.16:0/3259965997 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52040b2a40 con 0x7f52040d1860 2023-05-20T01:25:24.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:24.390+0000 7f51eb7fe700 1 -- 172.21.15.16:0/3259965997 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f4011cd0 con 0x7f52040d1860 2023-05-20T01:25:24.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:24.390+0000 7f52096cd700 1 -- 172.21.15.16:0/3259965997 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52040cdf30 con 0x7f52040d1860 2023-05-20T01:25:24.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:24.391+0000 7f52096cd700 1 -- 172.21.15.16:0/3259965997 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52040d1cb0 con 0x7f52040d1860 2023-05-20T01:25:24.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:24.392+0000 7f51eb7fe700 1 -- 172.21.15.16:0/3259965997 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f51f4021350 con 0x7f52040d1860 2023-05-20T01:25:24.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:24.392+0000 7f51eb7fe700 1 --2- 172.21.15.16:0/3259965997 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f51f006ace0 0x7f51f006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:24.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:24.392+0000 7f51eb7fe700 1 -- 172.21.15.16:0/3259965997 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f51f4089490 con 0x7f52040d1860 2023-05-20T01:25:24.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:24.392+0000 7f5202ffd700 1 --2- 172.21.15.16:0/3259965997 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f51f006ace0 0x7f51f006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:24.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:24.393+0000 7f5202ffd700 1 --2- 172.21.15.16:0/3259965997 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f51f006ace0 0x7f51f006d190 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f51f800a8b0 tx=0x7f51f8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:24.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:24.397+0000 7f51eb7fe700 1 -- 172.21.15.16:0/3259965997 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f51f4054490 con 0x7f52040d1860 2023-05-20T01:25:24.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:24.598+0000 7f52096cd700 1 -- 172.21.15.16:0/3259965997 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_extra", "pg_num_min": 64} v 0) v1 -- 0x7f520404e8c0 con 0x7f52040d1860 2023-05-20T01:25:25.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:25.509+0000 7f51eb7fe700 1 -- 172.21.15.16:0/3259965997 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_extra", "pg_num_min": 64}]=0 pool 'cephfs_data_extra' created v89) v1 ==== 142+0+0 (secure 0 0 0) 0x7f51f4057ab0 con 0x7f52040d1860 2023-05-20T01:25:25.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:25.514+0000 7f52096cd700 1 -- 172.21.15.16:0/3259965997 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f51f006ace0 msgr2=0x7f51f006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:25.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:25.514+0000 7f52096cd700 1 --2- 172.21.15.16:0/3259965997 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f51f006ace0 0x7f51f006d190 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f51f800a8b0 tx=0x7f51f8008040 comp rx=0 tx=0).stop 2023-05-20T01:25:25.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:25.514+0000 7f52096cd700 1 -- 172.21.15.16:0/3259965997 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52040d1860 msgr2=0x7f5204155320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:25.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:25.514+0000 7f52096cd700 1 --2- 172.21.15.16:0/3259965997 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52040d1860 0x7f5204155320 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f51f400c750 tx=0x7f51f400cb10 comp rx=0 tx=0).stop 2023-05-20T01:25:25.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:25.515+0000 7f52096cd700 1 -- 172.21.15.16:0/3259965997 shutdown_connections 2023-05-20T01:25:25.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:25.515+0000 7f52096cd700 1 --2- 172.21.15.16:0/3259965997 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f51f006ace0 0x7f51f006d190 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:25.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:25.515+0000 7f52096cd700 1 --2- 172.21.15.16:0/3259965997 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f52040d3880 0x7f5204153410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:25.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:25.515+0000 7f52096cd700 1 --2- 172.21.15.16:0/3259965997 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f52040b2670 0x7f52041f17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:25.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:25.515+0000 7f52096cd700 1 --2- 172.21.15.16:0/3259965997 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52040d1860 0x7f5204155320 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:25.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:25.515+0000 7f52096cd700 1 -- 172.21.15.16:0/3259965997 >> 172.21.15.16:0/3259965997 conn(0x7f5204071c40 msgr2=0x7f52040b6be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:25.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:25.516+0000 7f52096cd700 1 -- 172.21.15.16:0/3259965997 shutdown_connections 2023-05-20T01:25:25.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:25.516+0000 7f52096cd700 1 -- 172.21.15.16:0/3259965997 wait complete. 2023-05-20T01:25:25.524 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data_extra' created 2023-05-20T01:25:25.537 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_extra 2023-05-20T01:25:25.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:25.667+0000 7f941b390700 1 Processor -- start 2023-05-20T01:25:25.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:25.667+0000 7f941b390700 1 -- start start 2023-05-20T01:25:25.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:25.668+0000 7f941b390700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f9414159b50 0x7f941406e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:25.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:25.668+0000 7f941b390700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f94140cd1c0 0x7f94140710b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:25.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:25.668+0000 7f941b390700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94140ccd30 con 0x7f94140cd630 2023-05-20T01:25:25.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:25.668+0000 7f941b390700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94140ce990 con 0x7f9414159b50 2023-05-20T01:25:25.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:25.668+0000 7f941b390700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94140cc6e0 con 0x7f94140cd1c0 2023-05-20T01:25:25.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:25.668+0000 7f941992d700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f94140cd630 0x7f94140ca870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:47976/0 (socket says 172.21.15.16:47976) 2023-05-20T01:25:25.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:25.668+0000 7f941992d700 1 -- 172.21.15.16:0/2754966890 learned_addr learned my addr 172.21.15.16:0/2754966890 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:25:25.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:25.668+0000 7f940bfff700 1 -- 172.21.15.16:0/2754966890 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3403467530 0 0) 0x7f94140ccd30 con 0x7f94140cd630 2023-05-20T01:25:25.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:25.668+0000 7f941892b700 1 --2- 172.21.15.16:0/2754966890 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f94140cd1c0 0x7f94140710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:25.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:25.668+0000 7f940bfff700 1 -- 172.21.15.16:0/2754966890 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94000035b0 con 0x7f94140cd630 2023-05-20T01:25:25.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:25.668+0000 7f941912c700 1 --2- 172.21.15.16:0/2754966890 >> v2:172.21.15.132:3300/0 conn(0x7f9414159b50 0x7f941406e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:25.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:25.669+0000 7f940bfff700 1 -- 172.21.15.16:0/2754966890 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 829372903 0 0) 0x7f94000035b0 con 0x7f94140cd630 2023-05-20T01:25:25.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:25.669+0000 7f940bfff700 1 -- 172.21.15.16:0/2754966890 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f94140cd1c0 msgr2=0x7f94140710b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:25.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:25.669+0000 7f940bfff700 1 --2- 172.21.15.16:0/2754966890 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f94140cd1c0 0x7f94140710b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:25.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:25.669+0000 7f940bfff700 1 -- 172.21.15.16:0/2754966890 >> v2:172.21.15.132:3300/0 conn(0x7f9414159b50 msgr2=0x7f941406e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:25.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:25.669+0000 7f940bfff700 1 --2- 172.21.15.16:0/2754966890 >> v2:172.21.15.132:3300/0 conn(0x7f9414159b50 0x7f941406e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:25.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:25.669+0000 7f940bfff700 1 -- 172.21.15.16:0/2754966890 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94140cc9c0 con 0x7f94140cd630 2023-05-20T01:25:25.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:25.669+0000 7f940bfff700 1 -- 172.21.15.16:0/2754966890 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7f940c0019f0 con 0x7f94140cd630 2023-05-20T01:25:25.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:25.669+0000 7f940bfff700 1 -- 172.21.15.16:0/2754966890 >> v1:172.21.15.16:6789/0 conn(0x7f94140cd630 legacy=0x7f94140ca870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:25.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:25.669+0000 7f940bfff700 1 --2- 172.21.15.16:0/2754966890 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f94140cd1c0 0x7f9400003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:25.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:25.669+0000 7f940bfff700 1 --2- 172.21.15.16:0/2754966890 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9400003fd0 0x7f9400004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:25.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:25.669+0000 7f940bfff700 1 --2- 172.21.15.16:0/2754966890 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9400004d90 0x7f9400005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:25.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:25.669+0000 7f940bfff700 1 -- 172.21.15.16:0/2754966890 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94140ce990 con 0x7f9400004d90 2023-05-20T01:25:25.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:25.669+0000 7f940bfff700 1 -- 172.21.15.16:0/2754966890 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94140cc6e0 con 0x7f9400003fd0 2023-05-20T01:25:25.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:25.669+0000 7f940bfff700 1 -- 172.21.15.16:0/2754966890 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f940000ba10 con 0x7f94140cd1c0 2023-05-20T01:25:25.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:25.669+0000 7f941992d700 1 --2- 172.21.15.16:0/2754966890 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9400004d90 0x7f9400005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:25.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:25.669+0000 7f941892b700 1 --2- 172.21.15.16:0/2754966890 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9400003fd0 0x7f9400004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:25.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:25.669+0000 7f941912c700 1 --2- 172.21.15.16:0/2754966890 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f94140cd1c0 0x7f9400003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:25.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:25.669+0000 7f941992d700 1 -- 172.21.15.16:0/2754966890 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f94140cd1c0 msgr2=0x7f9400003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:25.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:25.669+0000 7f941992d700 1 --2- 172.21.15.16:0/2754966890 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f94140cd1c0 0x7f9400003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:25.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:25.669+0000 7f941992d700 1 -- 172.21.15.16:0/2754966890 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9400003fd0 msgr2=0x7f9400004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:25.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:25.669+0000 7f941992d700 1 --2- 172.21.15.16:0/2754966890 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9400003fd0 0x7f9400004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:25.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:25.669+0000 7f941992d700 1 -- 172.21.15.16:0/2754966890 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94000035b0 con 0x7f9400004d90 2023-05-20T01:25:25.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:25.669+0000 7f941992d700 1 --2- 172.21.15.16:0/2754966890 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9400004d90 0x7f9400005200 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f940c00a780 tx=0x7f9400005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6bb40b864428b95 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:25.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:25.669+0000 7f940bfff700 1 -- 172.21.15.16:0/2754966890 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f940c0072d0 con 0x7f9400004d90 2023-05-20T01:25:25.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:25.669+0000 7f940bfff700 1 -- 172.21.15.16:0/2754966890 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f940c007430 con 0x7f9400004d90 2023-05-20T01:25:25.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:25.669+0000 7f940bfff700 1 -- 172.21.15.16:0/2754966890 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f940c0077e0 con 0x7f9400004d90 2023-05-20T01:25:25.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:25.670+0000 7f941b390700 1 -- 172.21.15.16:0/2754966890 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9400004d90 msgr2=0x7f9400005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:25.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:25.670+0000 7f941b390700 1 --2- 172.21.15.16:0/2754966890 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9400004d90 0x7f9400005200 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f940c00a780 tx=0x7f9400005880 comp rx=0 tx=0).stop 2023-05-20T01:25:25.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:25.670+0000 7f941992d700 1 -- 172.21.15.16:0/2754966890 reap_dead start 2023-05-20T01:25:25.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:25.670+0000 7f941b390700 1 -- 172.21.15.16:0/2754966890 shutdown_connections 2023-05-20T01:25:25.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:25.670+0000 7f941b390700 1 -- 172.21.15.16:0/2754966890 >> 172.21.15.16:0/2754966890 conn(0x7f94140cffd0 msgr2=0x7f94140d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:25.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:25.670+0000 7f941b390700 1 -- 172.21.15.16:0/2754966890 shutdown_connections 2023-05-20T01:25:25.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:25.670+0000 7f941b390700 1 -- 172.21.15.16:0/2754966890 wait complete. 2023-05-20T01:25:25.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:25.671+0000 7f941b390700 1 Processor -- start 2023-05-20T01:25:25.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:25.671+0000 7f941b390700 1 -- start start 2023-05-20T01:25:25.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:25.672+0000 7f941b390700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9400004d90 0x7f9414153510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:25.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:25.672+0000 7f941b390700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f94140b41c0 0x7f9414155ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:25.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:25.672+0000 7f941b390700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9414158720 0x7f9414158b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:25.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:25.672+0000 7f941b390700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9414153a50 con 0x7f9400004d90 2023-05-20T01:25:25.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:25.672+0000 7f941b390700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94140b2e40 con 0x7f94140b41c0 2023-05-20T01:25:25.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:25.672+0000 7f941b390700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94140b3120 con 0x7f9414158720 2023-05-20T01:25:25.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:25.672+0000 7f941912c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9400004d90 0x7f9414153510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:25.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:25.672+0000 7f941912c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9400004d90 0x7f9414153510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36456/0 (socket says 172.21.15.16:36456) 2023-05-20T01:25:25.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:25.672+0000 7f941912c700 1 -- 172.21.15.16:0/2766566027 learned_addr learned my addr 172.21.15.16:0/2766566027 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:25.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:25.672+0000 7f941892b700 1 --2- 172.21.15.16:0/2766566027 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f94140b41c0 0x7f9414155ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:25.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:25.672+0000 7f941992d700 1 --2- 172.21.15.16:0/2766566027 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9414158720 0x7f9414158b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:25.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:25.673+0000 7f941912c700 1 -- 172.21.15.16:0/2766566027 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9414158720 msgr2=0x7f9414158b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:25.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:25.673+0000 7f941912c700 1 --2- 172.21.15.16:0/2766566027 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9414158720 0x7f9414158b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:25.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:25.673+0000 7f941912c700 1 -- 172.21.15.16:0/2766566027 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f94140b41c0 msgr2=0x7f9414155ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:25.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:25.673+0000 7f941912c700 1 --2- 172.21.15.16:0/2766566027 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f94140b41c0 0x7f9414155ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:25.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:25.673+0000 7f941912c700 1 -- 172.21.15.16:0/2766566027 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9414071e80 con 0x7f9400004d90 2023-05-20T01:25:25.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:25.673+0000 7f941912c700 1 --2- 172.21.15.16:0/2766566027 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9400004d90 0x7f9414153510 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f9404011a70 tx=0x7f9404011d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:25.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:25.673+0000 7f9409ffb700 1 -- 172.21.15.16:0/2766566027 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9404005a20 con 0x7f9400004d90 2023-05-20T01:25:25.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:25.673+0000 7f9409ffb700 1 -- 172.21.15.16:0/2766566027 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9404005b80 con 0x7f9400004d90 2023-05-20T01:25:25.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:25.673+0000 7f941b390700 1 -- 172.21.15.16:0/2766566027 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f941406e5e0 con 0x7f9400004d90 2023-05-20T01:25:25.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:25.673+0000 7f9409ffb700 1 -- 172.21.15.16:0/2766566027 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f940400f940 con 0x7f9400004d90 2023-05-20T01:25:25.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:25.673+0000 7f941b390700 1 -- 172.21.15.16:0/2766566027 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94140cec70 con 0x7f9400004d90 2023-05-20T01:25:25.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:25.674+0000 7f941b390700 1 -- 172.21.15.16:0/2766566027 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94140b4010 con 0x7f9400004d90 2023-05-20T01:25:25.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:25.675+0000 7f9409ffb700 1 -- 172.21.15.16:0/2766566027 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f940400faa0 con 0x7f9400004d90 2023-05-20T01:25:25.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:25.675+0000 7f9409ffb700 1 --2- 172.21.15.16:0/2766566027 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f93f806ace0 0x7f93f806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:25.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:25.675+0000 7f9409ffb700 1 -- 172.21.15.16:0/2766566027 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6754+0+0 (secure 0 0 0) 0x7f9404089bc0 con 0x7f9400004d90 2023-05-20T01:25:25.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:25.675+0000 7f941892b700 1 --2- 172.21.15.16:0/2766566027 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f93f806ace0 0x7f93f806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:25.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:25.676+0000 7f941892b700 1 --2- 172.21.15.16:0/2766566027 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f93f806ace0 0x7f93f806d190 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f94140b2600 tx=0x7f9410009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:25.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:25.680+0000 7f9409ffb700 1 -- 172.21.15.16:0/2766566027 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f9404054930 con 0x7f9400004d90 2023-05-20T01:25:25.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:25.926+0000 7f941b390700 1 -- 172.21.15.16:0/2766566027 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_extra"} v 0) v1 -- 0x7f94140747b0 con 0x7f9400004d90 2023-05-20T01:25:25.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:25.931+0000 7f9409ffb700 1 -- 172.21.15.16:0/2766566027 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_extra"}]=0 added data pool 14 to fsmap v58) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9404011ea0 con 0x7f9400004d90 2023-05-20T01:25:25.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:25.934+0000 7f941b390700 1 -- 172.21.15.16:0/2766566027 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f93f806ace0 msgr2=0x7f93f806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:25.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:25.934+0000 7f941b390700 1 --2- 172.21.15.16:0/2766566027 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f93f806ace0 0x7f93f806d190 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f94140b2600 tx=0x7f9410009210 comp rx=0 tx=0).stop 2023-05-20T01:25:25.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:25.934+0000 7f941b390700 1 -- 172.21.15.16:0/2766566027 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9400004d90 msgr2=0x7f9414153510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:25.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:25.934+0000 7f941b390700 1 --2- 172.21.15.16:0/2766566027 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9400004d90 0x7f9414153510 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f9404011a70 tx=0x7f9404011d80 comp rx=0 tx=0).stop 2023-05-20T01:25:25.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:25.934+0000 7f941b390700 1 -- 172.21.15.16:0/2766566027 shutdown_connections 2023-05-20T01:25:25.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:25.934+0000 7f941b390700 1 --2- 172.21.15.16:0/2766566027 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f93f806ace0 0x7f93f806d190 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:25.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:25.934+0000 7f941b390700 1 --2- 172.21.15.16:0/2766566027 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9414158720 0x7f9414158b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:25.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:25.934+0000 7f941b390700 1 --2- 172.21.15.16:0/2766566027 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9400004d90 0x7f9414153510 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:25.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:25.934+0000 7f941b390700 1 --2- 172.21.15.16:0/2766566027 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f94140b41c0 0x7f9414155ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:25.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:25.934+0000 7f941b390700 1 -- 172.21.15.16:0/2766566027 >> 172.21.15.16:0/2766566027 conn(0x7f94140cffd0 msgr2=0x7f941406a510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:25.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:25.935+0000 7f941b390700 1 -- 172.21.15.16:0/2766566027 shutdown_connections 2023-05-20T01:25:25.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:25.935+0000 7f941b390700 1 -- 172.21.15.16:0/2766566027 wait complete. 2023-05-20T01:25:25.941 INFO:teuthology.orchestra.run.smithi016.stderr:added data pool 14 to fsmap 2023-05-20T01:25:25.959 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-20T01:25:26.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.079+0000 7f36abacf700 1 Processor -- start 2023-05-20T01:25:26.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.079+0000 7f36abacf700 1 -- start start 2023-05-20T01:25:26.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.080+0000 7f36abacf700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f36a4071340 0x7f36a4071760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:26.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.080+0000 7f36abacf700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36a415e060 con 0x7f36a415e2a0 2023-05-20T01:25:26.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.080+0000 7f36abacf700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36a415d450 con 0x7f36a406a510 2023-05-20T01:25:26.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.080+0000 7f36abacf700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36a415d730 con 0x7f36a4071340 2023-05-20T01:25:26.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.080+0000 7f36a906a700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f36a415e2a0 0x7f36a40cf5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:47992/0 (socket says 172.21.15.16:47992) 2023-05-20T01:25:26.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.080+0000 7f36a906a700 1 -- 172.21.15.16:0/1818436716 learned_addr learned my addr 172.21.15.16:0/1818436716 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:25:26.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.080+0000 7f36aa06c700 1 --2- 172.21.15.16:0/1818436716 >> v2:172.21.15.16:3300/0 conn(0x7f36a4071340 0x7f36a4071760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:26.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.080+0000 7f36a8869700 1 -- 172.21.15.16:0/1818436716 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1459279261 0 0) 0x7f36a415e060 con 0x7f36a415e2a0 2023-05-20T01:25:26.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.080+0000 7f36a8869700 1 -- 172.21.15.16:0/1818436716 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f36900035b0 con 0x7f36a415e2a0 2023-05-20T01:25:26.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.080+0000 7f36aa06c700 1 -- 172.21.15.16:0/1818436716 >> v1:172.21.15.132:6789/0 conn(0x7f36a406a510 legacy=0x7f36a406a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:26.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.080+0000 7f36aa06c700 1 -- 172.21.15.16:0/1818436716 >> v1:172.21.15.16:6789/0 conn(0x7f36a415e2a0 legacy=0x7f36a40cf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:26.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.080+0000 7f36aa06c700 1 -- 172.21.15.16:0/1818436716 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36a415da10 con 0x7f36a4071340 2023-05-20T01:25:26.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.080+0000 7f36aa06c700 1 --2- 172.21.15.16:0/1818436716 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36a4071340 0x7f36a4071760 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f369c009d90 tx=0x7f369c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f15234ebd2dbd9e4 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:26.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.081+0000 7f36a8869700 1 -- 172.21.15.16:0/1818436716 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f369c00e070 con 0x7f36a4071340 2023-05-20T01:25:26.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.081+0000 7f36a8869700 1 -- 172.21.15.16:0/1818436716 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f369c00bb20 con 0x7f36a4071340 2023-05-20T01:25:26.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.081+0000 7f36a8869700 1 -- 172.21.15.16:0/1818436716 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f369c012430 con 0x7f36a4071340 2023-05-20T01:25:26.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.081+0000 7f36abacf700 1 -- 172.21.15.16:0/1818436716 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36a4071340 msgr2=0x7f36a4071760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:26.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.081+0000 7f36abacf700 1 --2- 172.21.15.16:0/1818436716 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36a4071340 0x7f36a4071760 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f369c009d90 tx=0x7f369c00b330 comp rx=0 tx=0).stop 2023-05-20T01:25:26.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.081+0000 7f36abacf700 1 -- 172.21.15.16:0/1818436716 shutdown_connections 2023-05-20T01:25:26.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.081+0000 7f36abacf700 1 --2- 172.21.15.16:0/1818436716 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36a4071340 0x7f36a4071760 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:26.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.081+0000 7f36abacf700 1 -- 172.21.15.16:0/1818436716 >> 172.21.15.16:0/1818436716 conn(0x7f36a40d0c30 msgr2=0x7f36a40b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:26.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.081+0000 7f36abacf700 1 -- 172.21.15.16:0/1818436716 shutdown_connections 2023-05-20T01:25:26.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.081+0000 7f36abacf700 1 -- 172.21.15.16:0/1818436716 wait complete. 2023-05-20T01:25:26.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.082+0000 7f36abacf700 1 Processor -- start 2023-05-20T01:25:26.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.082+0000 7f36abacf700 1 -- start start 2023-05-20T01:25:26.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.083+0000 7f36abacf700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f36a406a510 0x7f36a40c9450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:26.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.083+0000 7f36abacf700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36a4071340 0x7f36a415ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:26.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.083+0000 7f36abacf700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f36a415e2a0 0x7f36a415b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:26.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.083+0000 7f36abacf700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36a40c8e80 con 0x7f36a4071340 2023-05-20T01:25:26.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.083+0000 7f36abacf700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36a40c7de0 con 0x7f36a415e2a0 2023-05-20T01:25:26.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.083+0000 7f36abacf700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36a40c80c0 con 0x7f36a406a510 2023-05-20T01:25:26.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.083+0000 7f36a906a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36a4071340 0x7f36a415ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:26.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.083+0000 7f36a906a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36a4071340 0x7f36a415ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36472/0 (socket says 172.21.15.16:36472) 2023-05-20T01:25:26.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.083+0000 7f36a906a700 1 -- 172.21.15.16:0/323896799 learned_addr learned my addr 172.21.15.16:0/323896799 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:26.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.083+0000 7f36aa06c700 1 --2- 172.21.15.16:0/323896799 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f36a415e2a0 0x7f36a415b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:26.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.083+0000 7f36a986b700 1 --2- 172.21.15.16:0/323896799 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f36a406a510 0x7f36a40c9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:26.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.083+0000 7f36a906a700 1 -- 172.21.15.16:0/323896799 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f36a406a510 msgr2=0x7f36a40c9450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:26.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.084+0000 7f36a906a700 1 --2- 172.21.15.16:0/323896799 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f36a406a510 0x7f36a40c9450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:26.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.084+0000 7f36a906a700 1 -- 172.21.15.16:0/323896799 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f36a415e2a0 msgr2=0x7f36a415b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:26.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.084+0000 7f36a906a700 1 --2- 172.21.15.16:0/323896799 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f36a415e2a0 0x7f36a415b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:26.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.084+0000 7f36a906a700 1 -- 172.21.15.16:0/323896799 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f369c009a70 con 0x7f36a4071340 2023-05-20T01:25:26.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.084+0000 7f36aa06c700 1 --2- 172.21.15.16:0/323896799 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f36a415e2a0 0x7f36a415b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:25:26.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.084+0000 7f36a986b700 1 --2- 172.21.15.16:0/323896799 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f36a406a510 0x7f36a40c9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:25:26.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.084+0000 7f36a906a700 1 --2- 172.21.15.16:0/323896799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36a4071340 0x7f36a415ae90 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f36a0001eb0 tx=0x7f36a000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:26.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.084+0000 7f369a7fc700 1 -- 172.21.15.16:0/323896799 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a0005d70 con 0x7f36a4071340 2023-05-20T01:25:26.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.084+0000 7f369a7fc700 1 -- 172.21.15.16:0/323896799 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36a00070d0 con 0x7f36a4071340 2023-05-20T01:25:26.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.084+0000 7f36abacf700 1 -- 172.21.15.16:0/323896799 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36a40cfd00 con 0x7f36a4071340 2023-05-20T01:25:26.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.084+0000 7f369a7fc700 1 -- 172.21.15.16:0/323896799 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a0007460 con 0x7f36a4071340 2023-05-20T01:25:26.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.084+0000 7f36abacf700 1 -- 172.21.15.16:0/323896799 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36a415dcf0 con 0x7f36a4071340 2023-05-20T01:25:26.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.085+0000 7f36abacf700 1 -- 172.21.15.16:0/323896799 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36a415d080 con 0x7f36a4071340 2023-05-20T01:25:26.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.085+0000 7f369a7fc700 1 -- 172.21.15.16:0/323896799 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f36a00075c0 con 0x7f36a4071340 2023-05-20T01:25:26.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.086+0000 7f369a7fc700 1 --2- 172.21.15.16:0/323896799 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f368806ace0 0x7f368806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:26.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.086+0000 7f369a7fc700 1 -- 172.21.15.16:0/323896799 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6786+0+0 (secure 0 0 0) 0x7f36a0088d20 con 0x7f36a4071340 2023-05-20T01:25:26.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.086+0000 7f36a986b700 1 --2- 172.21.15.16:0/323896799 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f368806ace0 0x7f368806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:26.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.086+0000 7f36a986b700 1 --2- 172.21.15.16:0/323896799 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f368806ace0 0x7f368806d190 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f3694000f60 tx=0x7f3694009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:26.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.090+0000 7f369a7fc700 1 -- 172.21.15.16:0/323896799 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f36a0053af0 con 0x7f36a4071340 2023-05-20T01:25:26.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.332+0000 7f36abacf700 1 -- 172.21.15.16:0/323896799 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f36a40ca7f0 con 0x7f36a4071340 2023-05-20T01:25:26.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.333+0000 7f369a7fc700 1 -- 172.21.15.16:0/323896799 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+4676 (secure 0 0 0) 0x7f36a0057110 con 0x7f36a4071340 2023-05-20T01:25:26.338 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:25:26.338 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":58,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6077,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/1241894038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":1241894038},{"type":"v1","addr":"172.21.15.132:6835","nonce":1241894038}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":6562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/3771738903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3771738903},{"type":"v1","addr":"172.21.15.132:6834","nonce":3771738903}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":7101,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/3876038711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":3876038711},{"type":"v1","addr":"172.21.15.16:6836","nonce":3876038711}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-20T01:25:11.987993+0000","modified":"2023-05-20T01:25:25.928170+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7104},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7104":{"gid":7104,"name":"c","rank":0,"incarnation":53,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6837/28561908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":28561908},{"type":"v1","addr":"172.21.15.16:6837","nonce":28561908}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13,14],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-05-20T01:25:26.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.335+0000 7f36abacf700 1 -- 172.21.15.16:0/323896799 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f368806ace0 msgr2=0x7f368806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:26.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.335+0000 7f36abacf700 1 --2- 172.21.15.16:0/323896799 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f368806ace0 0x7f368806d190 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f3694000f60 tx=0x7f3694009040 comp rx=0 tx=0).stop 2023-05-20T01:25:26.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.335+0000 7f36abacf700 1 -- 172.21.15.16:0/323896799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36a4071340 msgr2=0x7f36a415ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:26.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.335+0000 7f36abacf700 1 --2- 172.21.15.16:0/323896799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36a4071340 0x7f36a415ae90 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f36a0001eb0 tx=0x7f36a000c860 comp rx=0 tx=0).stop 2023-05-20T01:25:26.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.335+0000 7f36abacf700 1 -- 172.21.15.16:0/323896799 shutdown_connections 2023-05-20T01:25:26.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.335+0000 7f36abacf700 1 --2- 172.21.15.16:0/323896799 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f36a415e2a0 0x7f36a415b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:26.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.335+0000 7f36abacf700 1 --2- 172.21.15.16:0/323896799 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f368806ace0 0x7f368806d190 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:26.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.335+0000 7f36abacf700 1 --2- 172.21.15.16:0/323896799 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f36a406a510 0x7f36a40c9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:26.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.335+0000 7f36abacf700 1 --2- 172.21.15.16:0/323896799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36a4071340 0x7f36a415ae90 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:26.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.335+0000 7f36abacf700 1 -- 172.21.15.16:0/323896799 >> 172.21.15.16:0/323896799 conn(0x7f36a40d0c30 msgr2=0x7f36a406a8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:26.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.335+0000 7f36abacf700 1 -- 172.21.15.16:0/323896799 shutdown_connections 2023-05-20T01:25:26.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.335+0000 7f36abacf700 1 -- 172.21.15.16:0/323896799 wait complete. 2023-05-20T01:25:26.341 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 58 2023-05-20T01:25:26.351 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-20T01:25:26.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.472+0000 7fd989c97700 1 Processor -- start 2023-05-20T01:25:26.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.472+0000 7fd989c97700 1 -- start start 2023-05-20T01:25:26.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.473+0000 7fd989c97700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd98406a510 0x7fd98415e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:26.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.473+0000 7fd989c97700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fd98406a8e0 0x7fd9840cf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:26.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.473+0000 7fd989c97700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd98415e060 con 0x7fd9840cfb30 2023-05-20T01:25:26.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.473+0000 7fd989c97700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd98415d450 con 0x7fd98406a510 2023-05-20T01:25:26.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.473+0000 7fd989c97700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd98415d730 con 0x7fd98406a8e0 2023-05-20T01:25:26.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.473+0000 7fd9837fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd98406a510 0x7fd98415e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:26.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.473+0000 7fd983fff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fd9840cfb30 0x7fd984071340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:48010/0 (socket says 172.21.15.16:48010) 2023-05-20T01:25:26.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.473+0000 7fd983fff700 1 -- 172.21.15.16:0/2112245263 learned_addr learned my addr 172.21.15.16:0/2112245263 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:25:26.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.473+0000 7fd982ffd700 1 --2- 172.21.15.16:0/2112245263 >> v2:172.21.15.132:3300/0 conn(0x7fd98406a8e0 0x7fd9840cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:26.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.473+0000 7fd9837fe700 1 -- 172.21.15.16:0/2112245263 >> v2:172.21.15.132:3300/0 conn(0x7fd98406a8e0 msgr2=0x7fd9840cf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:26.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.473+0000 7fd9837fe700 1 --2- 172.21.15.16:0/2112245263 >> v2:172.21.15.132:3300/0 conn(0x7fd98406a8e0 0x7fd9840cf5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:26.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.473+0000 7fd9837fe700 1 -- 172.21.15.16:0/2112245263 >> v1:172.21.15.16:6789/0 conn(0x7fd9840cfb30 legacy=0x7fd984071340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:26.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.473+0000 7fd9827fc700 1 -- 172.21.15.16:0/2112245263 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 433697220 0 0) 0x7fd98415e060 con 0x7fd9840cfb30 2023-05-20T01:25:26.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.473+0000 7fd9837fe700 1 -- 172.21.15.16:0/2112245263 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd98415da10 con 0x7fd98406a510 2023-05-20T01:25:26.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.473+0000 7fd982ffd700 1 --2- 172.21.15.16:0/2112245263 >> v2:172.21.15.132:3300/0 conn(0x7fd98406a8e0 0x7fd9840cf5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:25:26.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.473+0000 7fd9837fe700 1 --2- 172.21.15.16:0/2112245263 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd98406a510 0x7fd98415e2a0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fd984070fd0 tx=0x7fd974009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1640a148813710ef server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:26.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.473+0000 7fd9827fc700 1 -- 172.21.15.16:0/2112245263 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd974011830 con 0x7fd98406a510 2023-05-20T01:25:26.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.473+0000 7fd9827fc700 1 -- 172.21.15.16:0/2112245263 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd974011990 con 0x7fd98406a510 2023-05-20T01:25:26.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.473+0000 7fd9827fc700 1 -- 172.21.15.16:0/2112245263 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd974011d20 con 0x7fd98406a510 2023-05-20T01:25:26.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.474+0000 7fd989c97700 1 -- 172.21.15.16:0/2112245263 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd98406a510 msgr2=0x7fd98415e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:26.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.474+0000 7fd989c97700 1 --2- 172.21.15.16:0/2112245263 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd98406a510 0x7fd98415e2a0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fd984070fd0 tx=0x7fd974009b20 comp rx=0 tx=0).stop 2023-05-20T01:25:26.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.474+0000 7fd989c97700 1 -- 172.21.15.16:0/2112245263 shutdown_connections 2023-05-20T01:25:26.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.474+0000 7fd989c97700 1 --2- 172.21.15.16:0/2112245263 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd98406a510 0x7fd98415e2a0 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:26.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.474+0000 7fd989c97700 1 --2- 172.21.15.16:0/2112245263 >> v2:172.21.15.132:3300/0 conn(0x7fd98406a8e0 0x7fd9840cf5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:26.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.474+0000 7fd989c97700 1 -- 172.21.15.16:0/2112245263 >> 172.21.15.16:0/2112245263 conn(0x7fd9840d0c30 msgr2=0x7fd9840b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:26.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.474+0000 7fd989c97700 1 -- 172.21.15.16:0/2112245263 shutdown_connections 2023-05-20T01:25:26.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.474+0000 7fd989c97700 1 -- 172.21.15.16:0/2112245263 wait complete. 2023-05-20T01:25:26.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.476+0000 7fd989c97700 1 Processor -- start 2023-05-20T01:25:26.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.476+0000 7fd989c97700 1 -- start start 2023-05-20T01:25:26.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.477+0000 7fd989c97700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd98406a510 0x7fd984156620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:26.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.477+0000 7fd989c97700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd98406a8e0 0x7fd9840c7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:26.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.477+0000 7fd989c97700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9840cfb30 0x7fd9840c7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:26.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.477+0000 7fd989c97700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9841f23a0 con 0x7fd9840cfb30 2023-05-20T01:25:26.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.477+0000 7fd989c97700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd98406e610 con 0x7fd98406a510 2023-05-20T01:25:26.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.477+0000 7fd989c97700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd984156b60 con 0x7fd98406a8e0 2023-05-20T01:25:26.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.477+0000 7fd983fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9840cfb30 0x7fd9840c7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:26.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.477+0000 7fd983fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9840cfb30 0x7fd9840c7ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36488/0 (socket says 172.21.15.16:36488) 2023-05-20T01:25:26.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.477+0000 7fd983fff700 1 -- 172.21.15.16:0/2132101344 learned_addr learned my addr 172.21.15.16:0/2132101344 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:26.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.477+0000 7fd9837fe700 1 --2- 172.21.15.16:0/2132101344 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd98406a510 0x7fd984156620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:26.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.477+0000 7fd982ffd700 1 --2- 172.21.15.16:0/2132101344 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd98406a8e0 0x7fd9840c7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:26.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.477+0000 7fd983fff700 1 -- 172.21.15.16:0/2132101344 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd98406a8e0 msgr2=0x7fd9840c7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:26.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.477+0000 7fd983fff700 1 --2- 172.21.15.16:0/2132101344 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd98406a8e0 0x7fd9840c7080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:26.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.477+0000 7fd983fff700 1 -- 172.21.15.16:0/2132101344 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd98406a510 msgr2=0x7fd984156620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:26.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.477+0000 7fd983fff700 1 --2- 172.21.15.16:0/2132101344 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd98406a510 0x7fd984156620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:26.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.477+0000 7fd983fff700 1 -- 172.21.15.16:0/2132101344 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9740094a0 con 0x7fd9840cfb30 2023-05-20T01:25:26.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.477+0000 7fd983fff700 1 --2- 172.21.15.16:0/2132101344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9840cfb30 0x7fd9840c7ab0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fd97c00cbd0 tx=0x7fd97c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:26.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.477+0000 7fd96bfff700 1 -- 172.21.15.16:0/2132101344 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd97c00ab70 con 0x7fd9840cfb30 2023-05-20T01:25:26.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.478+0000 7fd96bfff700 1 -- 172.21.15.16:0/2132101344 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd97c00acf0 con 0x7fd9840cfb30 2023-05-20T01:25:26.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.478+0000 7fd989c97700 1 -- 172.21.15.16:0/2132101344 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd98406ad50 con 0x7fd9840cfb30 2023-05-20T01:25:26.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.478+0000 7fd96bfff700 1 -- 172.21.15.16:0/2132101344 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd97c007310 con 0x7fd9840cfb30 2023-05-20T01:25:26.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.478+0000 7fd989c97700 1 -- 172.21.15.16:0/2132101344 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd98415dcf0 con 0x7fd9840cfb30 2023-05-20T01:25:26.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.478+0000 7fd989c97700 1 -- 172.21.15.16:0/2132101344 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd98415d080 con 0x7fd9840cfb30 2023-05-20T01:25:26.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.479+0000 7fd96bfff700 1 -- 172.21.15.16:0/2132101344 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd97c007470 con 0x7fd9840cfb30 2023-05-20T01:25:26.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.479+0000 7fd96bfff700 1 --2- 172.21.15.16:0/2132101344 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd97006ace0 0x7fd97006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:26.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.479+0000 7fd96bfff700 1 -- 172.21.15.16:0/2132101344 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6786+0+0 (secure 0 0 0) 0x7fd97c0895c0 con 0x7fd9840cfb30 2023-05-20T01:25:26.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.480+0000 7fd9837fe700 1 --2- 172.21.15.16:0/2132101344 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd97006ace0 0x7fd97006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:26.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.480+0000 7fd9837fe700 1 --2- 172.21.15.16:0/2132101344 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd97006ace0 0x7fd97006d190 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fd9740090b0 tx=0x7fd974008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:26.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.484+0000 7fd96bfff700 1 -- 172.21.15.16:0/2132101344 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fd97c054390 con 0x7fd9840cfb30 2023-05-20T01:25:26.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.684+0000 7fd989c97700 1 -- 172.21.15.16:0/2132101344 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd98406e780 con 0x7fd9840cfb30 2023-05-20T01:25:26.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.685+0000 7fd96bfff700 1 -- 172.21.15.16:0/2132101344 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+20251 (secure 0 0 0) 0x7fd97c00ae60 con 0x7fd9840cfb30 2023-05-20T01:25:26.690 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:25:26.690 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":90,"fsid":"c2245cc5-ecab-4087-a59b-bc81db901676","created":"2023-05-20T01:16:09.549230+0000","modified":"2023-05-20T01:25:25.927886+0000","last_up_change":"2023-05-20T01:16:19.788624+0000","last_in_change":"2023-05-20T01:16:12.092160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-05-20T01:16:20.934366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-05-20T01:16:21.372294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-05-20T01:25:09.967348+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-05-20T01:25:10.956221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data_extra","create_time":"2023-05-20T01:25:24.629648+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"317a83cd-19c8-4958-a301-6087ef342289","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6825","nonce":3923634179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6827","nonce":3923634179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6831","nonce":3923634179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6829","nonce":3923634179}]},"public_addr":"172.21.15.16:6825/3923634179","cluster_addr":"172.21.15.16:6827/3923634179","heartbeat_back_addr":"172.21.15.16:6831/3923634179","heartbeat_front_addr":"172.21.15.16:6829/3923634179","state":["exists","up"]},{"osd":1,"uuid":"5581046f-8219-4089-8d7d-b7846399505e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6809","nonce":3860264220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6811","nonce":3860264220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6815","nonce":3860264220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6813","nonce":3860264220}]},"public_addr":"172.21.15.16:6809/3860264220","cluster_addr":"172.21.15.16:6811/3860264220","heartbeat_back_addr":"172.21.15.16:6815/3860264220","heartbeat_front_addr":"172.21.15.16:6813/3860264220","state":["exists","up"]},{"osd":2,"uuid":"072e9e66-4133-478e-8d4a-64d46c2963f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6817","nonce":293187030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6819","nonce":293187030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6823","nonce":293187030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6821","nonce":293187030}]},"public_addr":"172.21.15.16:6817/293187030","cluster_addr":"172.21.15.16:6819/293187030","heartbeat_back_addr":"172.21.15.16:6823/293187030","heartbeat_front_addr":"172.21.15.16:6821/293187030","state":["exists","up"]},{"osd":3,"uuid":"11e1bba6-00f8-48dd-9028-d8d716b7f145","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6801","nonce":3669781890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6803","nonce":3669781890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6807","nonce":3669781890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6805","nonce":3669781890}]},"public_addr":"172.21.15.16:6801/3669781890","cluster_addr":"172.21.15.16:6803/3669781890","heartbeat_back_addr":"172.21.15.16:6807/3669781890","heartbeat_front_addr":"172.21.15.16:6805/3669781890","state":["exists","up"]},{"osd":4,"uuid":"b8909c3a-cf70-46cb-8d0b-009c707cf9d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6819","nonce":506317223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6823","nonce":506317223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6829","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6831","nonce":506317223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6827","nonce":506317223}]},"public_addr":"172.21.15.132:6819/506317223","cluster_addr":"172.21.15.132:6823/506317223","heartbeat_back_addr":"172.21.15.132:6831/506317223","heartbeat_front_addr":"172.21.15.132:6827/506317223","state":["exists","up"]},{"osd":5,"uuid":"b1818304-4583-47bf-a6ec-0bfbb5166233","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6818","nonce":1844689995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6822","nonce":1844689995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6830","nonce":1844689995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6826","nonce":1844689995}]},"public_addr":"172.21.15.132:6818/1844689995","cluster_addr":"172.21.15.132:6822/1844689995","heartbeat_back_addr":"172.21.15.132:6830/1844689995","heartbeat_front_addr":"172.21.15.132:6826/1844689995","state":["exists","up"]},{"osd":6,"uuid":"d37f4691-eed1-4a93-8818-18354f03f4a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6801","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6802","nonce":727581123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6805","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6807","nonce":727581123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6813","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6815","nonce":727581123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6811","nonce":727581123}]},"public_addr":"172.21.15.132:6802/727581123","cluster_addr":"172.21.15.132:6807/727581123","heartbeat_back_addr":"172.21.15.132:6815/727581123","heartbeat_front_addr":"172.21.15.132:6811/727581123","state":["exists","up"]},{"osd":7,"uuid":"4f712f62-5d1d-4816-8042-03209144f8b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6803","nonce":466001149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6806","nonce":466001149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6814","nonce":466001149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6810","nonce":466001149}]},"public_addr":"172.21.15.132:6803/466001149","cluster_addr":"172.21.15.132:6806/466001149","heartbeat_back_addr":"172.21.15.132:6814/466001149","heartbeat_front_addr":"172.21.15.132:6810/466001149","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.338863+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.278680+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305694+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.246835+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.276237+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.304441+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305421+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.267050+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-20T01:25:26.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.687+0000 7fd989c97700 1 -- 172.21.15.16:0/2132101344 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd97006ace0 msgr2=0x7fd97006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:26.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.688+0000 7fd989c97700 1 --2- 172.21.15.16:0/2132101344 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd97006ace0 0x7fd97006d190 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fd9740090b0 tx=0x7fd974008030 comp rx=0 tx=0).stop 2023-05-20T01:25:26.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.688+0000 7fd989c97700 1 -- 172.21.15.16:0/2132101344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9840cfb30 msgr2=0x7fd9840c7ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:26.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.688+0000 7fd989c97700 1 --2- 172.21.15.16:0/2132101344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9840cfb30 0x7fd9840c7ab0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fd97c00cbd0 tx=0x7fd97c00cf90 comp rx=0 tx=0).stop 2023-05-20T01:25:26.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.688+0000 7fd989c97700 1 -- 172.21.15.16:0/2132101344 shutdown_connections 2023-05-20T01:25:26.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.688+0000 7fd989c97700 1 --2- 172.21.15.16:0/2132101344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9840cfb30 0x7fd9840c7ab0 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:26.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.688+0000 7fd989c97700 1 --2- 172.21.15.16:0/2132101344 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd98406a510 0x7fd984156620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:26.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.688+0000 7fd989c97700 1 --2- 172.21.15.16:0/2132101344 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd97006ace0 0x7fd97006d190 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:26.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.688+0000 7fd989c97700 1 --2- 172.21.15.16:0/2132101344 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd98406a8e0 0x7fd9840c7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:26.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.688+0000 7fd989c97700 1 -- 172.21.15.16:0/2132101344 >> 172.21.15.16:0/2132101344 conn(0x7fd9840d0c30 msgr2=0x7fd98415c510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:26.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.688+0000 7fd989c97700 1 -- 172.21.15.16:0/2132101344 shutdown_connections 2023-05-20T01:25:26.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:26.688+0000 7fd989c97700 1 -- 172.21.15.16:0/2132101344 wait complete. 2023-05-20T01:25:26.704 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:25:26.705 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.layout.pool -v cephfs_data_extra .) 2023-05-20T01:25:26.725 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:25:26.725 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir subdir) 2023-05-20T01:25:26.785 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-05-20T01:25:26.786 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2023-05-20T01:25:26.843 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:25:26.843 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=subdir/sixmegs bs=1M conv=fdatasync count=6 seek=0) 2023-05-20T01:25:26.952 INFO:teuthology.orchestra.run.smithi016.stderr:6+0 records in 2023-05-20T01:25:26.953 INFO:teuthology.orchestra.run.smithi016.stderr:6+0 records out 2023-05-20T01:25:26.953 INFO:teuthology.orchestra.run.smithi016.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.0517092 s, 122 MB/s 2023-05-20T01:25:26.954 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits daemon-helper kill python3 -c ' 2023-05-20T01:25:26.954 DEBUG:teuthology.orchestra.run.smithi016:> import os 2023-05-20T01:25:26.954 DEBUG:teuthology.orchestra.run.smithi016:> import stat 2023-05-20T01:25:26.955 DEBUG:teuthology.orchestra.run.smithi016:> import json 2023-05-20T01:25:26.955 DEBUG:teuthology.orchestra.run.smithi016:> import sys 2023-05-20T01:25:26.956 DEBUG:teuthology.orchestra.run.smithi016:> 2023-05-20T01:25:26.957 DEBUG:teuthology.orchestra.run.smithi016:> try: 2023-05-20T01:25:26.957 DEBUG:teuthology.orchestra.run.smithi016:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/subdir/sixmegs'"'"') 2023-05-20T01:25:26.958 DEBUG:teuthology.orchestra.run.smithi016:> except OSError as e: 2023-05-20T01:25:26.959 DEBUG:teuthology.orchestra.run.smithi016:> sys.exit(e.errno) 2023-05-20T01:25:26.959 DEBUG:teuthology.orchestra.run.smithi016:> 2023-05-20T01:25:26.960 DEBUG:teuthology.orchestra.run.smithi016:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2023-05-20T01:25:26.960 DEBUG:teuthology.orchestra.run.smithi016:> print(json.dumps( 2023-05-20T01:25:26.961 DEBUG:teuthology.orchestra.run.smithi016:> dict([(a, getattr(s, a)) for a in attrs]), 2023-05-20T01:25:26.961 DEBUG:teuthology.orchestra.run.smithi016:> indent=2)) 2023-05-20T01:25:26.962 DEBUG:teuthology.orchestra.run.smithi016:> ' 2023-05-20T01:25:27.035 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-05-20T01:25:27.035 INFO:teuthology.orchestra.run.smithi016.stdout: "st_mode": 33204, 2023-05-20T01:25:27.035 INFO:teuthology.orchestra.run.smithi016.stdout: "st_ino": 1099511627777, 2023-05-20T01:25:27.035 INFO:teuthology.orchestra.run.smithi016.stdout: "st_dev": 47, 2023-05-20T01:25:27.036 INFO:teuthology.orchestra.run.smithi016.stdout: "st_nlink": 1, 2023-05-20T01:25:27.036 INFO:teuthology.orchestra.run.smithi016.stdout: "st_uid": 1000, 2023-05-20T01:25:27.036 INFO:teuthology.orchestra.run.smithi016.stdout: "st_gid": 1265, 2023-05-20T01:25:27.036 INFO:teuthology.orchestra.run.smithi016.stdout: "st_size": 6291456, 2023-05-20T01:25:27.036 INFO:teuthology.orchestra.run.smithi016.stdout: "st_atime": 1684545926.8966012, 2023-05-20T01:25:27.036 INFO:teuthology.orchestra.run.smithi016.stdout: "st_mtime": 1684545926.9266007, 2023-05-20T01:25:27.036 INFO:teuthology.orchestra.run.smithi016.stdout: "st_ctime": 1684545926.9266007 2023-05-20T01:25:27.036 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-05-20T01:25:27.220 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-05-20T01:25:27.220 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2023-05-20T01:25:27.279 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-05-20T01:25:27.280 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2023-05-20T01:25:27.337 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-20T01:25:27.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.499+0000 7f8779dc9700 1 Processor -- start 2023-05-20T01:25:27.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.499+0000 7f8779dc9700 1 -- start start 2023-05-20T01:25:27.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.500+0000 7f8779dc9700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f87740af2d0 0x7f87740cec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:27.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.500+0000 7f8779dc9700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87740cf5c0 con 0x7f87740d16c0 2023-05-20T01:25:27.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.500+0000 7f8779dc9700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87740b2000 con 0x7f87740cf160 2023-05-20T01:25:27.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.500+0000 7f8779dc9700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87740b22e0 con 0x7f87740af2d0 2023-05-20T01:25:27.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.500+0000 7f8772ffd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f87740af2d0 0x7f87740cec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:27.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.500+0000 7f8772ffd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f87740af2d0 0x7f87740cec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36498/0 (socket says 172.21.15.16:36498) 2023-05-20T01:25:27.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.500+0000 7f87737fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f87740d16c0 0x7f8774144330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:48024/0 (socket says 172.21.15.16:48024) 2023-05-20T01:25:27.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.500+0000 7f8772ffd700 1 -- 172.21.15.16:0/1439185598 learned_addr learned my addr 172.21.15.16:0/1439185598 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:27.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.501+0000 7f8772ffd700 1 -- 172.21.15.16:0/1439185598 >> v1:172.21.15.132:6789/0 conn(0x7f87740cf160 legacy=0x7f87740b0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:27.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.501+0000 7f8772ffd700 1 -- 172.21.15.16:0/1439185598 >> v1:172.21.15.16:6789/0 conn(0x7f87740d16c0 legacy=0x7f8774144330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:27.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.501+0000 7f8772ffd700 1 -- 172.21.15.16:0/1439185598 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87740b25c0 con 0x7f87740af2d0 2023-05-20T01:25:27.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.501+0000 7f8772ffd700 1 --2- 172.21.15.16:0/1439185598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87740af2d0 0x7f87740cec20 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f8768009920 tx=0x7f8768009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85039b814ba04a45 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:27.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.501+0000 7f87727fc700 1 -- 172.21.15.16:0/1439185598 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f876800ba20 con 0x7f87740af2d0 2023-05-20T01:25:27.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.501+0000 7f87727fc700 1 -- 172.21.15.16:0/1439185598 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f876800bb80 con 0x7f87740af2d0 2023-05-20T01:25:27.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.501+0000 7f87727fc700 1 -- 172.21.15.16:0/1439185598 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8768009e70 con 0x7f87740af2d0 2023-05-20T01:25:27.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.502+0000 7f8779dc9700 1 -- 172.21.15.16:0/1439185598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87740af2d0 msgr2=0x7f87740cec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:27.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.502+0000 7f8779dc9700 1 --2- 172.21.15.16:0/1439185598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87740af2d0 0x7f87740cec20 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f8768009920 tx=0x7f8768009c30 comp rx=0 tx=0).stop 2023-05-20T01:25:27.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.502+0000 7f8779dc9700 1 -- 172.21.15.16:0/1439185598 shutdown_connections 2023-05-20T01:25:27.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.502+0000 7f8779dc9700 1 --2- 172.21.15.16:0/1439185598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87740af2d0 0x7f87740cec20 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:27.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.502+0000 7f8779dc9700 1 -- 172.21.15.16:0/1439185598 >> 172.21.15.16:0/1439185598 conn(0x7f877406ae30 msgr2=0x7f877409a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:27.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.502+0000 7f8779dc9700 1 -- 172.21.15.16:0/1439185598 shutdown_connections 2023-05-20T01:25:27.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.502+0000 7f8779dc9700 1 -- 172.21.15.16:0/1439185598 wait complete. 2023-05-20T01:25:27.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.503+0000 7f8779dc9700 1 Processor -- start 2023-05-20T01:25:27.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.503+0000 7f8779dc9700 1 -- start start 2023-05-20T01:25:27.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.504+0000 7f8779dc9700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87740af2d0 0x7f87740ca520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:27.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.504+0000 7f8779dc9700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87740cf160 0x7f87740c71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:27.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.504+0000 7f8779dc9700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f87740d16c0 0x7f87741f39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:27.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.504+0000 7f8779dc9700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87740b3e80 con 0x7f87740af2d0 2023-05-20T01:25:27.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.504+0000 7f8779dc9700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87740b4690 con 0x7f87740d16c0 2023-05-20T01:25:27.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.504+0000 7f8779dc9700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87740b40d0 con 0x7f87740cf160 2023-05-20T01:25:27.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.504+0000 7f87737fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87740af2d0 0x7f87740ca520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:27.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.504+0000 7f87737fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87740af2d0 0x7f87740ca520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36504/0 (socket says 172.21.15.16:36504) 2023-05-20T01:25:27.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.504+0000 7f87737fe700 1 -- 172.21.15.16:0/351291877 learned_addr learned my addr 172.21.15.16:0/351291877 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:27.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.504+0000 7f8773fff700 1 --2- 172.21.15.16:0/351291877 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f87740d16c0 0x7f87741f39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:27.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.504+0000 7f8772ffd700 1 --2- 172.21.15.16:0/351291877 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87740cf160 0x7f87740c71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:27.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.504+0000 7f87737fe700 1 -- 172.21.15.16:0/351291877 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87740cf160 msgr2=0x7f87740c71b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:27.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.504+0000 7f87737fe700 1 --2- 172.21.15.16:0/351291877 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87740cf160 0x7f87740c71b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:27.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.504+0000 7f87737fe700 1 -- 172.21.15.16:0/351291877 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f87740d16c0 msgr2=0x7f87741f39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:27.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.504+0000 7f87737fe700 1 --2- 172.21.15.16:0/351291877 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f87740d16c0 0x7f87741f39b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:27.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.504+0000 7f87737fe700 1 -- 172.21.15.16:0/351291877 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8768009550 con 0x7f87740af2d0 2023-05-20T01:25:27.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.504+0000 7f87737fe700 1 --2- 172.21.15.16:0/351291877 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87740af2d0 0x7f87740ca520 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f876400c530 tx=0x7f876400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:27.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.505+0000 7f875bfff700 1 -- 172.21.15.16:0/351291877 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8764005cb0 con 0x7f87740af2d0 2023-05-20T01:25:27.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.505+0000 7f875bfff700 1 -- 172.21.15.16:0/351291877 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8764005e10 con 0x7f87740af2d0 2023-05-20T01:25:27.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.505+0000 7f8779dc9700 1 -- 172.21.15.16:0/351291877 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87740b28a0 con 0x7f87740af2d0 2023-05-20T01:25:27.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.505+0000 7f875bfff700 1 -- 172.21.15.16:0/351291877 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87640072d0 con 0x7f87740af2d0 2023-05-20T01:25:27.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.505+0000 7f8779dc9700 1 -- 172.21.15.16:0/351291877 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87740adda0 con 0x7f87740af2d0 2023-05-20T01:25:27.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.506+0000 7f8779dc9700 1 -- 172.21.15.16:0/351291877 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87740b1090 con 0x7f87740af2d0 2023-05-20T01:25:27.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.506+0000 7f875bfff700 1 -- 172.21.15.16:0/351291877 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8764007430 con 0x7f87740af2d0 2023-05-20T01:25:27.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.506+0000 7f875bfff700 1 --2- 172.21.15.16:0/351291877 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f876006ace0 0x7f876006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:27.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.507+0000 7f875bfff700 1 -- 172.21.15.16:0/351291877 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6786+0+0 (secure 0 0 0) 0x7f8764088a30 con 0x7f87740af2d0 2023-05-20T01:25:27.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.507+0000 7f8772ffd700 1 --2- 172.21.15.16:0/351291877 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f876006ace0 0x7f876006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:27.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.507+0000 7f8772ffd700 1 --2- 172.21.15.16:0/351291877 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f876006ace0 0x7f876006d190 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f8768009c70 tx=0x7f8768008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:27.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.511+0000 7f875bfff700 1 -- 172.21.15.16:0/351291877 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f87640538b0 con 0x7f87740af2d0 2023-05-20T01:25:27.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.712+0000 7f8779dc9700 1 -- 172.21.15.16:0/351291877 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f87740c9b80 con 0x7f87740af2d0 2023-05-20T01:25:27.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.713+0000 7f875bfff700 1 -- 172.21.15.16:0/351291877 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+20238 (secure 0 0 0) 0x7f87740c9b80 con 0x7f87740af2d0 2023-05-20T01:25:27.717 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:25:27.718 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":91,"fsid":"c2245cc5-ecab-4087-a59b-bc81db901676","created":"2023-05-20T01:16:09.549230+0000","modified":"2023-05-20T01:25:26.929807+0000","last_up_change":"2023-05-20T01:16:19.788624+0000","last_in_change":"2023-05-20T01:16:12.092160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-05-20T01:16:20.934366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-05-20T01:16:21.372294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-05-20T01:25:09.967348+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-05-20T01:25:10.956221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data_extra","create_time":"2023-05-20T01:25:24.629648+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"317a83cd-19c8-4958-a301-6087ef342289","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6825","nonce":3923634179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6827","nonce":3923634179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6831","nonce":3923634179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6829","nonce":3923634179}]},"public_addr":"172.21.15.16:6825/3923634179","cluster_addr":"172.21.15.16:6827/3923634179","heartbeat_back_addr":"172.21.15.16:6831/3923634179","heartbeat_front_addr":"172.21.15.16:6829/3923634179","state":["exists","up"]},{"osd":1,"uuid":"5581046f-8219-4089-8d7d-b7846399505e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6809","nonce":3860264220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6811","nonce":3860264220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6815","nonce":3860264220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6813","nonce":3860264220}]},"public_addr":"172.21.15.16:6809/3860264220","cluster_addr":"172.21.15.16:6811/3860264220","heartbeat_back_addr":"172.21.15.16:6815/3860264220","heartbeat_front_addr":"172.21.15.16:6813/3860264220","state":["exists","up"]},{"osd":2,"uuid":"072e9e66-4133-478e-8d4a-64d46c2963f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6817","nonce":293187030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6819","nonce":293187030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6823","nonce":293187030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6821","nonce":293187030}]},"public_addr":"172.21.15.16:6817/293187030","cluster_addr":"172.21.15.16:6819/293187030","heartbeat_back_addr":"172.21.15.16:6823/293187030","heartbeat_front_addr":"172.21.15.16:6821/293187030","state":["exists","up"]},{"osd":3,"uuid":"11e1bba6-00f8-48dd-9028-d8d716b7f145","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6801","nonce":3669781890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6803","nonce":3669781890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6807","nonce":3669781890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6805","nonce":3669781890}]},"public_addr":"172.21.15.16:6801/3669781890","cluster_addr":"172.21.15.16:6803/3669781890","heartbeat_back_addr":"172.21.15.16:6807/3669781890","heartbeat_front_addr":"172.21.15.16:6805/3669781890","state":["exists","up"]},{"osd":4,"uuid":"b8909c3a-cf70-46cb-8d0b-009c707cf9d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6819","nonce":506317223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6823","nonce":506317223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6829","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6831","nonce":506317223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6827","nonce":506317223}]},"public_addr":"172.21.15.132:6819/506317223","cluster_addr":"172.21.15.132:6823/506317223","heartbeat_back_addr":"172.21.15.132:6831/506317223","heartbeat_front_addr":"172.21.15.132:6827/506317223","state":["exists","up"]},{"osd":5,"uuid":"b1818304-4583-47bf-a6ec-0bfbb5166233","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6818","nonce":1844689995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6822","nonce":1844689995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6830","nonce":1844689995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6826","nonce":1844689995}]},"public_addr":"172.21.15.132:6818/1844689995","cluster_addr":"172.21.15.132:6822/1844689995","heartbeat_back_addr":"172.21.15.132:6830/1844689995","heartbeat_front_addr":"172.21.15.132:6826/1844689995","state":["exists","up"]},{"osd":6,"uuid":"d37f4691-eed1-4a93-8818-18354f03f4a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6801","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6802","nonce":727581123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6805","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6807","nonce":727581123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6813","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6815","nonce":727581123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6811","nonce":727581123}]},"public_addr":"172.21.15.132:6802/727581123","cluster_addr":"172.21.15.132:6807/727581123","heartbeat_back_addr":"172.21.15.132:6815/727581123","heartbeat_front_addr":"172.21.15.132:6811/727581123","state":["exists","up"]},{"osd":7,"uuid":"4f712f62-5d1d-4816-8042-03209144f8b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6803","nonce":466001149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6806","nonce":466001149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6814","nonce":466001149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6810","nonce":466001149}]},"public_addr":"172.21.15.132:6803/466001149","cluster_addr":"172.21.15.132:6806/466001149","heartbeat_back_addr":"172.21.15.132:6814/466001149","heartbeat_front_addr":"172.21.15.132:6810/466001149","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.338863+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.278680+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305694+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.246835+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.276237+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.304441+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305421+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.267050+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-20T01:25:27.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.715+0000 7f8779dc9700 1 -- 172.21.15.16:0/351291877 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f876006ace0 msgr2=0x7f876006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:27.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.715+0000 7f8779dc9700 1 --2- 172.21.15.16:0/351291877 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f876006ace0 0x7f876006d190 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f8768009c70 tx=0x7f8768008030 comp rx=0 tx=0).stop 2023-05-20T01:25:27.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.715+0000 7f8779dc9700 1 -- 172.21.15.16:0/351291877 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87740af2d0 msgr2=0x7f87740ca520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:27.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.715+0000 7f8779dc9700 1 --2- 172.21.15.16:0/351291877 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87740af2d0 0x7f87740ca520 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f876400c530 tx=0x7f876400c8f0 comp rx=0 tx=0).stop 2023-05-20T01:25:27.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.715+0000 7f8779dc9700 1 -- 172.21.15.16:0/351291877 shutdown_connections 2023-05-20T01:25:27.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.715+0000 7f8779dc9700 1 --2- 172.21.15.16:0/351291877 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f87740d16c0 0x7f87741f39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:27.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.715+0000 7f8779dc9700 1 --2- 172.21.15.16:0/351291877 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87740af2d0 0x7f87740ca520 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:27.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.715+0000 7f8779dc9700 1 --2- 172.21.15.16:0/351291877 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f876006ace0 0x7f876006d190 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:27.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.715+0000 7f8779dc9700 1 --2- 172.21.15.16:0/351291877 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87740cf160 0x7f87740c71b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:27.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.715+0000 7f8779dc9700 1 -- 172.21.15.16:0/351291877 >> 172.21.15.16:0/351291877 conn(0x7f877406ae30 msgr2=0x7f87740ae110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:27.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.715+0000 7f8779dc9700 1 -- 172.21.15.16:0/351291877 shutdown_connections 2023-05-20T01:25:27.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.715+0000 7f8779dc9700 1 -- 172.21.15.16:0/351291877 wait complete. 2023-05-20T01:25:27.732 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-05-20T01:16:20.934366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-05-20T01:16:21.372294+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2023-05-20T01:25:09.967348+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2023-05-20T01:25:10.956221+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data_extra', 'create_time': '2023-05-20T01:25:24.629648+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-05-20T01:25:27.733 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-05-20T01:25:27.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.857+0000 7f7cb295f700 1 Processor -- start 2023-05-20T01:25:27.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.857+0000 7f7cb295f700 1 -- start start 2023-05-20T01:25:27.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.857+0000 7f7cb295f700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7cac0cd4f0 0x7f7cac0cf1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:27.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.857+0000 7f7cb295f700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f7cac06a510 0x7f7cac06a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:27.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.857+0000 7f7cb295f700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cac06dd00 con 0x7f7cac15cec0 2023-05-20T01:25:27.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.857+0000 7f7cb295f700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cac0cd940 con 0x7f7cac0cd4f0 2023-05-20T01:25:27.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.858+0000 7f7cb295f700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cac06aed0 con 0x7f7cac06a510 2023-05-20T01:25:27.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.858+0000 7f7cab7fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7cac0cd4f0 0x7f7cac0cf1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:27.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.858+0000 7f7cab7fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7cac0cd4f0 0x7f7cac0cf1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36512/0 (socket says 172.21.15.16:36512) 2023-05-20T01:25:27.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.858+0000 7f7cab7fe700 1 -- 172.21.15.16:0/2720111150 learned_addr learned my addr 172.21.15.16:0/2720111150 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:27.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.858+0000 7f7cb0efc700 1 --2- 172.21.15.16:0/2720111150 >> v2:172.21.15.132:3300/0 conn(0x7f7cac06a510 0x7f7cac06a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:27.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.858+0000 7f7cab7fe700 1 -- 172.21.15.16:0/2720111150 >> v2:172.21.15.132:3300/0 conn(0x7f7cac06a510 msgr2=0x7f7cac06a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:27.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.858+0000 7f7cab7fe700 1 --2- 172.21.15.16:0/2720111150 >> v2:172.21.15.132:3300/0 conn(0x7f7cac06a510 0x7f7cac06a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:27.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.858+0000 7f7cab7fe700 1 -- 172.21.15.16:0/2720111150 >> v1:172.21.15.132:6789/0 conn(0x7f7cac15cec0 legacy=0x7f7cac06e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:27.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.858+0000 7f7cab7fe700 1 -- 172.21.15.16:0/2720111150 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cac0cfa60 con 0x7f7cac0cd4f0 2023-05-20T01:25:27.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.858+0000 7f7cb0efc700 1 --2- 172.21.15.16:0/2720111150 >> v2:172.21.15.132:3300/0 conn(0x7f7cac06a510 0x7f7cac06a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:25:27.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.858+0000 7f7cab7fe700 1 --2- 172.21.15.16:0/2720111150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7cac0cd4f0 0x7f7cac0cf1f0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f7cac0c6c70 tx=0x7f7c9c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c040e45b479fe8ff server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:27.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.858+0000 7f7caaffd700 1 -- 172.21.15.16:0/2720111150 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c9c004230 con 0x7f7cac0cd4f0 2023-05-20T01:25:27.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.858+0000 7f7caaffd700 1 -- 172.21.15.16:0/2720111150 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c9c004390 con 0x7f7cac0cd4f0 2023-05-20T01:25:27.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.858+0000 7f7caaffd700 1 -- 172.21.15.16:0/2720111150 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c9c0036a0 con 0x7f7cac0cd4f0 2023-05-20T01:25:27.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.859+0000 7f7cb295f700 1 -- 172.21.15.16:0/2720111150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7cac0cd4f0 msgr2=0x7f7cac0cf1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:27.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.859+0000 7f7cb295f700 1 --2- 172.21.15.16:0/2720111150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7cac0cd4f0 0x7f7cac0cf1f0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f7cac0c6c70 tx=0x7f7c9c009bd0 comp rx=0 tx=0).stop 2023-05-20T01:25:27.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.859+0000 7f7cb295f700 1 -- 172.21.15.16:0/2720111150 shutdown_connections 2023-05-20T01:25:27.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.859+0000 7f7cb295f700 1 --2- 172.21.15.16:0/2720111150 >> v2:172.21.15.132:3300/0 conn(0x7f7cac06a510 0x7f7cac06a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:27.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.859+0000 7f7cb295f700 1 --2- 172.21.15.16:0/2720111150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7cac0cd4f0 0x7f7cac0cf1f0 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:27.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.859+0000 7f7cb295f700 1 -- 172.21.15.16:0/2720111150 >> 172.21.15.16:0/2720111150 conn(0x7f7cac06c260 msgr2=0x7f7cac0cffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:27.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.859+0000 7f7cb295f700 1 -- 172.21.15.16:0/2720111150 shutdown_connections 2023-05-20T01:25:27.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.859+0000 7f7cb295f700 1 -- 172.21.15.16:0/2720111150 wait complete. 2023-05-20T01:25:27.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.860+0000 7f7cb295f700 1 Processor -- start 2023-05-20T01:25:27.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.860+0000 7f7cb295f700 1 -- start start 2023-05-20T01:25:27.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.861+0000 7f7cb295f700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7cac06a510 0x7f7cac153750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:27.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.861+0000 7f7cb295f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7cac0cd4f0 0x7f7cac0b0370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:27.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.861+0000 7f7cb295f700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7cac15cec0 0x7f7cac0c59b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:27.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.861+0000 7f7cb295f700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cac0b1c60 con 0x7f7cac0cd4f0 2023-05-20T01:25:27.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.861+0000 7f7cb295f700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cac0b0bc0 con 0x7f7cac15cec0 2023-05-20T01:25:27.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.861+0000 7f7cb295f700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cac0b0ea0 con 0x7f7cac06a510 2023-05-20T01:25:27.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.861+0000 7f7cab7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7cac0cd4f0 0x7f7cac0b0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:27.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.861+0000 7f7cab7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7cac0cd4f0 0x7f7cac0b0370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36518/0 (socket says 172.21.15.16:36518) 2023-05-20T01:25:27.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.861+0000 7f7cab7fe700 1 -- 172.21.15.16:0/1267861059 learned_addr learned my addr 172.21.15.16:0/1267861059 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:27.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.861+0000 7f7cb0efc700 1 --2- 172.21.15.16:0/1267861059 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7cac15cec0 0x7f7cac0c59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:27.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.861+0000 7f7cab7fe700 1 -- 172.21.15.16:0/1267861059 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7cac06a510 msgr2=0x7f7cac153750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:27.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.861+0000 7f7cabfff700 1 --2- 172.21.15.16:0/1267861059 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7cac06a510 0x7f7cac153750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:27.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.861+0000 7f7cab7fe700 1 --2- 172.21.15.16:0/1267861059 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7cac06a510 0x7f7cac153750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:27.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.861+0000 7f7cab7fe700 1 -- 172.21.15.16:0/1267861059 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7cac15cec0 msgr2=0x7f7cac0c59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:27.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.861+0000 7f7cab7fe700 1 --2- 172.21.15.16:0/1267861059 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7cac15cec0 0x7f7cac0c59b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:27.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.861+0000 7f7cab7fe700 1 -- 172.21.15.16:0/1267861059 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c9c009550 con 0x7f7cac0cd4f0 2023-05-20T01:25:27.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.862+0000 7f7cb0efc700 1 --2- 172.21.15.16:0/1267861059 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7cac15cec0 0x7f7cac0c59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:25:27.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.862+0000 7f7cabfff700 1 --2- 172.21.15.16:0/1267861059 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7cac06a510 0x7f7cac153750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:25:27.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.862+0000 7f7cab7fe700 1 --2- 172.21.15.16:0/1267861059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7cac0cd4f0 0x7f7cac0b0370 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c009840 tx=0x7f7c9c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:27.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.862+0000 7f7ca8ff9700 1 -- 172.21.15.16:0/1267861059 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c9c003dc0 con 0x7f7cac0cd4f0 2023-05-20T01:25:27.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.862+0000 7f7ca8ff9700 1 -- 172.21.15.16:0/1267861059 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c9c01f070 con 0x7f7cac0cd4f0 2023-05-20T01:25:27.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.862+0000 7f7cb295f700 1 -- 172.21.15.16:0/1267861059 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cac06d990 con 0x7f7cac0cd4f0 2023-05-20T01:25:27.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.862+0000 7f7ca8ff9700 1 -- 172.21.15.16:0/1267861059 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c9c011870 con 0x7f7cac0cd4f0 2023-05-20T01:25:27.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.862+0000 7f7cb295f700 1 -- 172.21.15.16:0/1267861059 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cac06d6b0 con 0x7f7cac0cd4f0 2023-05-20T01:25:27.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.863+0000 7f7cb295f700 1 -- 172.21.15.16:0/1267861059 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cac15d290 con 0x7f7cac0cd4f0 2023-05-20T01:25:27.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.863+0000 7f7ca8ff9700 1 -- 172.21.15.16:0/1267861059 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7c9c021030 con 0x7f7cac0cd4f0 2023-05-20T01:25:27.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.864+0000 7f7ca8ff9700 1 --2- 172.21.15.16:0/1267861059 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f7c8c06ace0 0x7f7c8c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:27.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.864+0000 7f7ca8ff9700 1 -- 172.21.15.16:0/1267861059 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6786+0+0 (secure 0 0 0) 0x7f7c9c08ad10 con 0x7f7cac0cd4f0 2023-05-20T01:25:27.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.864+0000 7f7cabfff700 1 --2- 172.21.15.16:0/1267861059 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f7c8c06ace0 0x7f7c8c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:27.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.864+0000 7f7cabfff700 1 --2- 172.21.15.16:0/1267861059 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f7c8c06ace0 0x7f7c8c06d190 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f7c94003f40 tx=0x7f7c94009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:27.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:27.868+0000 7f7ca8ff9700 1 -- 172.21.15.16:0/1267861059 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f7c9c055c90 con 0x7f7cac0cd4f0 2023-05-20T01:25:28.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.066+0000 7f7cb295f700 1 -- 172.21.15.16:0/1267861059 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f7cac0747b0 con 0x7f7cac0cd4f0 2023-05-20T01:25:28.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.066+0000 7f7ca8ff9700 1 -- 172.21.15.16:0/1267861059 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v91) v1 ==== 92+0+10 (secure 0 0 0) 0x7f7c9c029020 con 0x7f7cac0cd4f0 2023-05-20T01:25:28.071 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 8 2023-05-20T01:25:28.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.069+0000 7f7cb295f700 1 -- 172.21.15.16:0/1267861059 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f7c8c06ace0 msgr2=0x7f7c8c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:28.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.069+0000 7f7cb295f700 1 --2- 172.21.15.16:0/1267861059 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f7c8c06ace0 0x7f7c8c06d190 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f7c94003f40 tx=0x7f7c94009040 comp rx=0 tx=0).stop 2023-05-20T01:25:28.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.069+0000 7f7cb295f700 1 -- 172.21.15.16:0/1267861059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7cac0cd4f0 msgr2=0x7f7cac0b0370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:28.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.069+0000 7f7cb295f700 1 --2- 172.21.15.16:0/1267861059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7cac0cd4f0 0x7f7cac0b0370 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c009840 tx=0x7f7c9c004750 comp rx=0 tx=0).stop 2023-05-20T01:25:28.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.069+0000 7f7cb295f700 1 -- 172.21.15.16:0/1267861059 shutdown_connections 2023-05-20T01:25:28.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.069+0000 7f7cb295f700 1 --2- 172.21.15.16:0/1267861059 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7cac15cec0 0x7f7cac0c59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:28.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.069+0000 7f7cb295f700 1 --2- 172.21.15.16:0/1267861059 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f7c8c06ace0 0x7f7c8c06d190 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:28.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.069+0000 7f7cb295f700 1 --2- 172.21.15.16:0/1267861059 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7cac06a510 0x7f7cac153750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:28.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.069+0000 7f7cb295f700 1 --2- 172.21.15.16:0/1267861059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7cac0cd4f0 0x7f7cac0b0370 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:28.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.069+0000 7f7cb295f700 1 -- 172.21.15.16:0/1267861059 >> 172.21.15.16:0/1267861059 conn(0x7f7cac06c260 msgr2=0x7f7cac0b3100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:28.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.069+0000 7f7cb295f700 1 -- 172.21.15.16:0/1267861059 shutdown_connections 2023-05-20T01:25:28.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.069+0000 7f7cb295f700 1 -- 172.21.15.16:0/1267861059 wait complete. 2023-05-20T01:25:28.085 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-05-20T01:25:28.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.208+0000 7fc40015b700 1 Processor -- start 2023-05-20T01:25:28.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.209+0000 7fc40015b700 1 -- start start 2023-05-20T01:25:28.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.209+0000 7fc40015b700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fc3f8159b50 0x7fc3f806e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:28.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.209+0000 7fc40015b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc3f80cd1c0 0x7fc3f80710b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:28.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.209+0000 7fc40015b700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc3f80cd630 0x7fc3f80715f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:28.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.209+0000 7fc40015b700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3f80ccd30 con 0x7fc3f80cd1c0 2023-05-20T01:25:28.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.209+0000 7fc40015b700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3f80ce990 con 0x7fc3f8159b50 2023-05-20T01:25:28.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.209+0000 7fc40015b700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f80cc6e0 con 0x7fc3f80cd630 2023-05-20T01:25:28.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.209+0000 7fc3fd6f6700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc3f80cd1c0 0x7fc3f80710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:28.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.209+0000 7fc3fd6f6700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc3f80cd1c0 0x7fc3f80710b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36524/0 (socket says 172.21.15.16:36524) 2023-05-20T01:25:28.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.210+0000 7fc3fd6f6700 1 -- 172.21.15.16:0/2038977496 learned_addr learned my addr 172.21.15.16:0/2038977496 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:28.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.210+0000 7fc3fd6f6700 1 -- 172.21.15.16:0/2038977496 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc3f80cd630 msgr2=0x7fc3f80715f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:28.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.210+0000 7fc3fd6f6700 1 --2- 172.21.15.16:0/2038977496 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc3f80cd630 0x7fc3f80715f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:28.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.210+0000 7fc3fdef7700 1 --2- 172.21.15.16:0/2038977496 >> v2:172.21.15.132:3300/0 conn(0x7fc3f8159b50 0x7fc3f806e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:28.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.210+0000 7fc3fd6f6700 1 -- 172.21.15.16:0/2038977496 >> v2:172.21.15.132:3300/0 conn(0x7fc3f8159b50 msgr2=0x7fc3f806e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:28.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.210+0000 7fc3fd6f6700 1 --2- 172.21.15.16:0/2038977496 >> v2:172.21.15.132:3300/0 conn(0x7fc3f8159b50 0x7fc3f806e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:28.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.210+0000 7fc3fd6f6700 1 -- 172.21.15.16:0/2038977496 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3f80cc9c0 con 0x7fc3f80cd1c0 2023-05-20T01:25:28.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.210+0000 7fc3fdef7700 1 --2- 172.21.15.16:0/2038977496 >> v2:172.21.15.132:3300/0 conn(0x7fc3f8159b50 0x7fc3f806e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:25:28.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.210+0000 7fc3fd6f6700 1 --2- 172.21.15.16:0/2038977496 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3f80cd1c0 0x7fc3f80710b0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fc3f8079820 tx=0x7fc3ec009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48f43f3b12f6c97c server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:28.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.210+0000 7fc3fcef5700 1 -- 172.21.15.16:0/2038977496 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3ec00b870 con 0x7fc3f80cd1c0 2023-05-20T01:25:28.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.210+0000 7fc3fcef5700 1 -- 172.21.15.16:0/2038977496 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3ec00b9d0 con 0x7fc3f80cd1c0 2023-05-20T01:25:28.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.210+0000 7fc3fcef5700 1 -- 172.21.15.16:0/2038977496 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3ec00bd60 con 0x7fc3f80cd1c0 2023-05-20T01:25:28.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.211+0000 7fc40015b700 1 -- 172.21.15.16:0/2038977496 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3f80cd1c0 msgr2=0x7fc3f80710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:28.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.211+0000 7fc40015b700 1 --2- 172.21.15.16:0/2038977496 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3f80cd1c0 0x7fc3f80710b0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fc3f8079820 tx=0x7fc3ec009bd0 comp rx=0 tx=0).stop 2023-05-20T01:25:28.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.211+0000 7fc40015b700 1 -- 172.21.15.16:0/2038977496 shutdown_connections 2023-05-20T01:25:28.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.211+0000 7fc40015b700 1 --2- 172.21.15.16:0/2038977496 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc3f80cd630 0x7fc3f80715f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:28.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.211+0000 7fc40015b700 1 --2- 172.21.15.16:0/2038977496 >> v2:172.21.15.132:3300/0 conn(0x7fc3f8159b50 0x7fc3f806e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:28.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.211+0000 7fc40015b700 1 --2- 172.21.15.16:0/2038977496 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3f80cd1c0 0x7fc3f80710b0 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:28.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.211+0000 7fc40015b700 1 -- 172.21.15.16:0/2038977496 >> 172.21.15.16:0/2038977496 conn(0x7fc3f80cffd0 msgr2=0x7fc3f80d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:28.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.211+0000 7fc40015b700 1 -- 172.21.15.16:0/2038977496 shutdown_connections 2023-05-20T01:25:28.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.211+0000 7fc40015b700 1 -- 172.21.15.16:0/2038977496 wait complete. 2023-05-20T01:25:28.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.212+0000 7fc40015b700 1 Processor -- start 2023-05-20T01:25:28.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.212+0000 7fc40015b700 1 -- start start 2023-05-20T01:25:28.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.213+0000 7fc40015b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc3f80cd1c0 0x7fc3f8155070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:28.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.213+0000 7fc40015b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3f80cd630 0x7fc3f8157910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:28.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.213+0000 7fc40015b700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc3f8159b50 0x7fc3f80b3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:28.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.213+0000 7fc40015b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f81f2f10 con 0x7fc3f80cd630 2023-05-20T01:25:28.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.213+0000 7fc40015b700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f81f2370 con 0x7fc3f80cd1c0 2023-05-20T01:25:28.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.213+0000 7fc40015b700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f81f3330 con 0x7fc3f8159b50 2023-05-20T01:25:28.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.213+0000 7fc3fd6f6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3f80cd630 0x7fc3f8157910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:28.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.213+0000 7fc3fd6f6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3f80cd630 0x7fc3f8157910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36532/0 (socket says 172.21.15.16:36532) 2023-05-20T01:25:28.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.213+0000 7fc3fd6f6700 1 -- 172.21.15.16:0/2924507547 learned_addr learned my addr 172.21.15.16:0/2924507547 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:28.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.213+0000 7fc3fd6f6700 1 -- 172.21.15.16:0/2924507547 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc3f8159b50 msgr2=0x7fc3f80b3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:28.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.213+0000 7fc3fe6f8700 1 --2- 172.21.15.16:0/2924507547 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc3f8159b50 0x7fc3f80b3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:28.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.213+0000 7fc3fdef7700 1 --2- 172.21.15.16:0/2924507547 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc3f80cd1c0 0x7fc3f8155070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:28.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.213+0000 7fc3fd6f6700 1 --2- 172.21.15.16:0/2924507547 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc3f8159b50 0x7fc3f80b3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:28.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.213+0000 7fc3fd6f6700 1 -- 172.21.15.16:0/2924507547 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc3f80cd1c0 msgr2=0x7fc3f8155070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:28.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.213+0000 7fc3fd6f6700 1 --2- 172.21.15.16:0/2924507547 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc3f80cd1c0 0x7fc3f8155070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:28.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.213+0000 7fc3fd6f6700 1 -- 172.21.15.16:0/2924507547 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3ec009550 con 0x7fc3f80cd630 2023-05-20T01:25:28.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.213+0000 7fc3fdef7700 1 --2- 172.21.15.16:0/2924507547 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc3f80cd1c0 0x7fc3f8155070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:25:28.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.213+0000 7fc3fe6f8700 1 --2- 172.21.15.16:0/2924507547 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc3f8159b50 0x7fc3f80b3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:25:28.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.213+0000 7fc3fd6f6700 1 --2- 172.21.15.16:0/2924507547 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3f80cd630 0x7fc3f8157910 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec014040 tx=0x7fc3ec004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:28.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.213+0000 7fc3ea7fc700 1 -- 172.21.15.16:0/2924507547 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3ec004500 con 0x7fc3f80cd630 2023-05-20T01:25:28.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.213+0000 7fc3ea7fc700 1 -- 172.21.15.16:0/2924507547 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3ec0036a0 con 0x7fc3f80cd630 2023-05-20T01:25:28.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.213+0000 7fc40015b700 1 -- 172.21.15.16:0/2924507547 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3f8071e80 con 0x7fc3f80cd630 2023-05-20T01:25:28.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.214+0000 7fc3ea7fc700 1 -- 172.21.15.16:0/2924507547 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3ec003a30 con 0x7fc3f80cd630 2023-05-20T01:25:28.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.214+0000 7fc40015b700 1 -- 172.21.15.16:0/2924507547 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3f806e5e0 con 0x7fc3f80cd630 2023-05-20T01:25:28.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.215+0000 7fc40015b700 1 -- 172.21.15.16:0/2924507547 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3f80cec70 con 0x7fc3f80cd630 2023-05-20T01:25:28.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.215+0000 7fc3ea7fc700 1 -- 172.21.15.16:0/2924507547 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc3ec003b90 con 0x7fc3f80cd630 2023-05-20T01:25:28.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.215+0000 7fc3ea7fc700 1 --2- 172.21.15.16:0/2924507547 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc3dc06ace0 0x7fc3dc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:28.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.216+0000 7fc3ea7fc700 1 -- 172.21.15.16:0/2924507547 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6786+0+0 (secure 0 0 0) 0x7fc3ec08ad10 con 0x7fc3f80cd630 2023-05-20T01:25:28.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.216+0000 7fc3fdef7700 1 --2- 172.21.15.16:0/2924507547 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc3dc06ace0 0x7fc3dc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:28.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.216+0000 7fc3fdef7700 1 --2- 172.21.15.16:0/2924507547 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc3dc06ace0 0x7fc3dc06d190 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fc3f4006fd0 tx=0x7fc3f4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:28.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.220+0000 7fc3ea7fc700 1 -- 172.21.15.16:0/2924507547 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fc3ec055b70 con 0x7fc3f80cd630 2023-05-20T01:25:28.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.418+0000 7fc40015b700 1 -- 172.21.15.16:0/2924507547 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc3f81f3050 con 0x7fc3f80cd630 2023-05-20T01:25:28.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.418+0000 7fc3ea7fc700 1 -- 172.21.15.16:0/2924507547 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc3f81f3050 con 0x7fc3f80cd630 2023-05-20T01:25:28.423 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-05-20T01:25:28.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.420+0000 7fc40015b700 1 -- 172.21.15.16:0/2924507547 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc3dc06ace0 msgr2=0x7fc3dc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:28.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.420+0000 7fc40015b700 1 --2- 172.21.15.16:0/2924507547 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc3dc06ace0 0x7fc3dc06d190 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fc3f4006fd0 tx=0x7fc3f4008040 comp rx=0 tx=0).stop 2023-05-20T01:25:28.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.420+0000 7fc40015b700 1 -- 172.21.15.16:0/2924507547 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3f80cd630 msgr2=0x7fc3f8157910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:28.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.420+0000 7fc40015b700 1 --2- 172.21.15.16:0/2924507547 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3f80cd630 0x7fc3f8157910 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec014040 tx=0x7fc3ec004750 comp rx=0 tx=0).stop 2023-05-20T01:25:28.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.420+0000 7fc40015b700 1 -- 172.21.15.16:0/2924507547 shutdown_connections 2023-05-20T01:25:28.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.420+0000 7fc40015b700 1 --2- 172.21.15.16:0/2924507547 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc3dc06ace0 0x7fc3dc06d190 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:28.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.420+0000 7fc40015b700 1 --2- 172.21.15.16:0/2924507547 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc3f8159b50 0x7fc3f80b3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:28.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.420+0000 7fc40015b700 1 --2- 172.21.15.16:0/2924507547 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc3f80cd1c0 0x7fc3f8155070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:28.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.420+0000 7fc40015b700 1 --2- 172.21.15.16:0/2924507547 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3f80cd630 0x7fc3f8157910 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:28.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.420+0000 7fc40015b700 1 -- 172.21.15.16:0/2924507547 >> 172.21.15.16:0/2924507547 conn(0x7fc3f80cffd0 msgr2=0x7fc3f80cbfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:28.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.421+0000 7fc40015b700 1 -- 172.21.15.16:0/2924507547 shutdown_connections 2023-05-20T01:25:28.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.421+0000 7fc40015b700 1 -- 172.21.15.16:0/2924507547 wait complete. 2023-05-20T01:25:28.437 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-05-20T01:25:28.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.561+0000 7f1207df7700 1 Processor -- start 2023-05-20T01:25:28.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.561+0000 7f1207df7700 1 -- start start 2023-05-20T01:25:28.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.562+0000 7f1207df7700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f12000cec80 0x7f12000cf0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:28.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.562+0000 7f1207df7700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f12000cc110 0x7f12000cc590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:28.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.562+0000 7f1207df7700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12000ce4d0 con 0x7f120015ab80 2023-05-20T01:25:28.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.562+0000 7f1207df7700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12000cb700 con 0x7f12000cec80 2023-05-20T01:25:28.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.562+0000 7f1207df7700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12000cf630 con 0x7f12000cc110 2023-05-20T01:25:28.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.562+0000 7f1205392700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f12000cec80 0x7f12000cf0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:28.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.562+0000 7f1205b93700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f120015ab80 0x7f12000cda80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:48066/0 (socket says 172.21.15.16:48066) 2023-05-20T01:25:28.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.562+0000 7f1205b93700 1 -- 172.21.15.16:0/3606673245 learned_addr learned my addr 172.21.15.16:0/3606673245 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:25:28.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.562+0000 7f1206394700 1 --2- 172.21.15.16:0/3606673245 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f12000cc110 0x7f12000cc590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:28.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.562+0000 7f1205392700 1 -- 172.21.15.16:0/3606673245 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f12000cc110 msgr2=0x7f12000cc590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:28.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.562+0000 7f1205392700 1 --2- 172.21.15.16:0/3606673245 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f12000cc110 0x7f12000cc590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:28.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.562+0000 7f1205392700 1 -- 172.21.15.16:0/3606673245 >> v1:172.21.15.16:6789/0 conn(0x7f120015ab80 legacy=0x7f12000cda80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:28.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.562+0000 7f1205392700 1 -- 172.21.15.16:0/3606673245 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12000ce7b0 con 0x7f12000cec80 2023-05-20T01:25:28.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.562+0000 7f1204b91700 1 -- 172.21.15.16:0/3606673245 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 638514354 0 0) 0x7f12000ce4d0 con 0x7f120015ab80 2023-05-20T01:25:28.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.562+0000 7f1206394700 1 --2- 172.21.15.16:0/3606673245 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f12000cc110 0x7f12000cc590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:25:28.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.562+0000 7f1205392700 1 --2- 172.21.15.16:0/3606673245 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12000cec80 0x7f12000cf0f0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f11f0009840 tx=0x7f11f0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cafa17e22686ee05 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:28.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.563+0000 7f1204b91700 1 -- 172.21.15.16:0/3606673245 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f0011830 con 0x7f12000cec80 2023-05-20T01:25:28.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.563+0000 7f1204b91700 1 -- 172.21.15.16:0/3606673245 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11f0011990 con 0x7f12000cec80 2023-05-20T01:25:28.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.563+0000 7f1204b91700 1 -- 172.21.15.16:0/3606673245 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f0011d20 con 0x7f12000cec80 2023-05-20T01:25:28.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.563+0000 7f1207df7700 1 -- 172.21.15.16:0/3606673245 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12000cec80 msgr2=0x7f12000cf0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:28.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.563+0000 7f1207df7700 1 --2- 172.21.15.16:0/3606673245 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12000cec80 0x7f12000cf0f0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f11f0009840 tx=0x7f11f0009b50 comp rx=0 tx=0).stop 2023-05-20T01:25:28.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.563+0000 7f1207df7700 1 -- 172.21.15.16:0/3606673245 shutdown_connections 2023-05-20T01:25:28.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.563+0000 7f1207df7700 1 --2- 172.21.15.16:0/3606673245 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f12000cc110 0x7f12000cc590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:28.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.563+0000 7f1207df7700 1 --2- 172.21.15.16:0/3606673245 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12000cec80 0x7f12000cf0f0 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:28.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.563+0000 7f1207df7700 1 -- 172.21.15.16:0/3606673245 >> 172.21.15.16:0/3606673245 conn(0x7f120006ae30 msgr2=0x7f120009a400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:28.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.563+0000 7f1207df7700 1 -- 172.21.15.16:0/3606673245 shutdown_connections 2023-05-20T01:25:28.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.563+0000 7f1207df7700 1 -- 172.21.15.16:0/3606673245 wait complete. 2023-05-20T01:25:28.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.564+0000 7f1207df7700 1 Processor -- start 2023-05-20T01:25:28.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.564+0000 7f1207df7700 1 -- start start 2023-05-20T01:25:28.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.565+0000 7f1207df7700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f12000cc110 0x7f12001599d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:28.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.565+0000 7f1207df7700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f12000cec80 0x7f120006ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:28.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.565+0000 7f1207df7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f120015ab80 0x7f12000b6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:28.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.565+0000 7f1207df7700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f120006b620 con 0x7f120015ab80 2023-05-20T01:25:28.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.565+0000 7f1207df7700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f120006b900 con 0x7f12000cec80 2023-05-20T01:25:28.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.565+0000 7f1207df7700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f120006bbe0 con 0x7f12000cc110 2023-05-20T01:25:28.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.565+0000 7f1206394700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f120015ab80 0x7f12000b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:28.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.565+0000 7f1206394700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f120015ab80 0x7f12000b6130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36548/0 (socket says 172.21.15.16:36548) 2023-05-20T01:25:28.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.565+0000 7f1206394700 1 -- 172.21.15.16:0/538472634 learned_addr learned my addr 172.21.15.16:0/538472634 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:28.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.565+0000 7f1205392700 1 --2- 172.21.15.16:0/538472634 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f12000cec80 0x7f120006ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:28.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.565+0000 7f1205b93700 1 --2- 172.21.15.16:0/538472634 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f12000cc110 0x7f12001599d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:28.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.565+0000 7f1206394700 1 -- 172.21.15.16:0/538472634 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f12000cc110 msgr2=0x7f12001599d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:28.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.565+0000 7f1206394700 1 --2- 172.21.15.16:0/538472634 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f12000cc110 0x7f12001599d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:28.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.566+0000 7f1206394700 1 -- 172.21.15.16:0/538472634 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f12000cec80 msgr2=0x7f120006ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:28.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.566+0000 7f1206394700 1 --2- 172.21.15.16:0/538472634 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f12000cec80 0x7f120006ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:28.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.566+0000 7f1206394700 1 -- 172.21.15.16:0/538472634 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11f00094a0 con 0x7f120015ab80 2023-05-20T01:25:28.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.566+0000 7f1205b93700 1 --2- 172.21.15.16:0/538472634 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f12000cc110 0x7f12001599d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:25:28.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.566+0000 7f1206394700 1 --2- 172.21.15.16:0/538472634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f120015ab80 0x7f12000b6130 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f11f800cd50 tx=0x7f11f800a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:28.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.566+0000 7f11f67fc700 1 -- 172.21.15.16:0/538472634 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f800f480 con 0x7f120015ab80 2023-05-20T01:25:28.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.566+0000 7f11f67fc700 1 -- 172.21.15.16:0/538472634 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11f800f600 con 0x7f120015ab80 2023-05-20T01:25:28.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.566+0000 7f1207df7700 1 -- 172.21.15.16:0/538472634 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12000cbbf0 con 0x7f120015ab80 2023-05-20T01:25:28.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.566+0000 7f11f67fc700 1 -- 172.21.15.16:0/538472634 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f800f9b0 con 0x7f120015ab80 2023-05-20T01:25:28.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.566+0000 7f1207df7700 1 -- 172.21.15.16:0/538472634 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12000cb910 con 0x7f120015ab80 2023-05-20T01:25:28.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.567+0000 7f1207df7700 1 -- 172.21.15.16:0/538472634 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12000ce190 con 0x7f120015ab80 2023-05-20T01:25:28.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.567+0000 7f11f67fc700 1 -- 172.21.15.16:0/538472634 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f11f800b040 con 0x7f120015ab80 2023-05-20T01:25:28.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.568+0000 7f11f67fc700 1 --2- 172.21.15.16:0/538472634 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f11e406ace0 0x7f11e406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:28.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.568+0000 7f11f67fc700 1 -- 172.21.15.16:0/538472634 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6786+0+0 (secure 0 0 0) 0x7f11f808a090 con 0x7f120015ab80 2023-05-20T01:25:28.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.568+0000 7f1205b93700 1 --2- 172.21.15.16:0/538472634 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f11e406ace0 0x7f11e406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:28.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.568+0000 7f1205b93700 1 --2- 172.21.15.16:0/538472634 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f11e406ace0 0x7f11e406d190 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f11fc000f60 tx=0x7f11fc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:28.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.572+0000 7f11f67fc700 1 -- 172.21.15.16:0/538472634 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f11f8054dc0 con 0x7f120015ab80 2023-05-20T01:25:28.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.767+0000 7f1207df7700 1 -- 172.21.15.16:0/538472634 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f12000747b0 con 0x7f120015ab80 2023-05-20T01:25:28.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.767+0000 7f11f67fc700 1 -- 172.21.15.16:0/538472634 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) v1 ==== 104+0+11 (secure 0 0 0) 0x7f11f80583e0 con 0x7f120015ab80 2023-05-20T01:25:28.772 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-05-20T01:25:28.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.769+0000 7f1207df7700 1 -- 172.21.15.16:0/538472634 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f11e406ace0 msgr2=0x7f11e406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:28.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.769+0000 7f1207df7700 1 --2- 172.21.15.16:0/538472634 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f11e406ace0 0x7f11e406d190 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f11fc000f60 tx=0x7f11fc009040 comp rx=0 tx=0).stop 2023-05-20T01:25:28.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.769+0000 7f1207df7700 1 -- 172.21.15.16:0/538472634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f120015ab80 msgr2=0x7f12000b6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:28.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.769+0000 7f1207df7700 1 --2- 172.21.15.16:0/538472634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f120015ab80 0x7f12000b6130 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f11f800cd50 tx=0x7f11f800a560 comp rx=0 tx=0).stop 2023-05-20T01:25:28.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.769+0000 7f1207df7700 1 -- 172.21.15.16:0/538472634 shutdown_connections 2023-05-20T01:25:28.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.769+0000 7f1207df7700 1 --2- 172.21.15.16:0/538472634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f120015ab80 0x7f12000b6130 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:28.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.769+0000 7f1207df7700 1 --2- 172.21.15.16:0/538472634 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f11e406ace0 0x7f11e406d190 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:28.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.769+0000 7f1207df7700 1 --2- 172.21.15.16:0/538472634 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f12000cc110 0x7f12001599d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:28.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.769+0000 7f1207df7700 1 --2- 172.21.15.16:0/538472634 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f12000cec80 0x7f120006ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:28.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.769+0000 7f1207df7700 1 -- 172.21.15.16:0/538472634 >> 172.21.15.16:0/538472634 conn(0x7f120006ae30 msgr2=0x7f12000b09f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:28.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.769+0000 7f1207df7700 1 -- 172.21.15.16:0/538472634 shutdown_connections 2023-05-20T01:25:28.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.769+0000 7f1207df7700 1 -- 172.21.15.16:0/538472634 wait complete. 2023-05-20T01:25:28.785 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-05-20T01:25:28.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.908+0000 7f066c98d700 1 Processor -- start 2023-05-20T01:25:28.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.908+0000 7f066c98d700 1 -- start start 2023-05-20T01:25:28.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.909+0000 7f066c98d700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f066815ca90 0x7f06680cd890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:28.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.909+0000 7f066c98d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f06680d2550 0x7f06680b6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:28.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.909+0000 7f066c98d700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06680d1820 con 0x7f06680d29a0 2023-05-20T01:25:28.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.909+0000 7f066c98d700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06680d0c50 con 0x7f06680d2550 2023-05-20T01:25:28.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.909+0000 7f066c98d700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06680cdf00 con 0x7f066815ca90 2023-05-20T01:25:28.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.909+0000 7f0665d9b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f06680d2550 0x7f06680b6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:28.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.909+0000 7f0665d9b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f06680d2550 0x7f06680b6090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36552/0 (socket says 172.21.15.16:36552) 2023-05-20T01:25:28.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.909+0000 7f0665d9b700 1 -- 172.21.15.16:0/347281670 learned_addr learned my addr 172.21.15.16:0/347281670 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:28.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.909+0000 7f066659c700 1 --2- 172.21.15.16:0/347281670 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f066815ca90 0x7f06680cd890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:28.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.910+0000 7f0665d9b700 1 -- 172.21.15.16:0/347281670 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f066815ca90 msgr2=0x7f06680cd890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:28.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.910+0000 7f0665d9b700 1 --2- 172.21.15.16:0/347281670 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f066815ca90 0x7f06680cd890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:28.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.910+0000 7f0665d9b700 1 -- 172.21.15.16:0/347281670 >> v1:172.21.15.132:6789/0 conn(0x7f06680d29a0 legacy=0x7f06680b4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:28.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.910+0000 7f0665d9b700 1 -- 172.21.15.16:0/347281670 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06680d1a70 con 0x7f06680d2550 2023-05-20T01:25:28.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.910+0000 7f0665d9b700 1 --2- 172.21.15.16:0/347281670 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06680d2550 0x7f06680b6090 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f06580098f0 tx=0x7f0658009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f47e6ce35e6800f server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:28.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.910+0000 7f066559a700 1 -- 172.21.15.16:0/347281670 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0658011830 con 0x7f06680d2550 2023-05-20T01:25:28.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.910+0000 7f066559a700 1 -- 172.21.15.16:0/347281670 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0658011990 con 0x7f06680d2550 2023-05-20T01:25:28.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.910+0000 7f066559a700 1 -- 172.21.15.16:0/347281670 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0658011d20 con 0x7f06680d2550 2023-05-20T01:25:28.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.910+0000 7f066c98d700 1 -- 172.21.15.16:0/347281670 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06680d2550 msgr2=0x7f06680b6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:28.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.910+0000 7f066c98d700 1 --2- 172.21.15.16:0/347281670 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06680d2550 0x7f06680b6090 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f06580098f0 tx=0x7f0658009c00 comp rx=0 tx=0).stop 2023-05-20T01:25:28.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.911+0000 7f066c98d700 1 -- 172.21.15.16:0/347281670 shutdown_connections 2023-05-20T01:25:28.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.911+0000 7f066c98d700 1 --2- 172.21.15.16:0/347281670 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f066815ca90 0x7f06680cd890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:28.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.911+0000 7f066c98d700 1 --2- 172.21.15.16:0/347281670 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06680d2550 0x7f06680b6090 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:28.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.911+0000 7f066c98d700 1 -- 172.21.15.16:0/347281670 >> 172.21.15.16:0/347281670 conn(0x7f066806b000 msgr2=0x7f066806b400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:28.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.911+0000 7f066c98d700 1 -- 172.21.15.16:0/347281670 shutdown_connections 2023-05-20T01:25:28.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.911+0000 7f066c98d700 1 -- 172.21.15.16:0/347281670 wait complete. 2023-05-20T01:25:28.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.912+0000 7f066c98d700 1 Processor -- start 2023-05-20T01:25:28.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.912+0000 7f066c98d700 1 -- start start 2023-05-20T01:25:28.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.913+0000 7f066c98d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06680d2550 0x7f06680ca310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:28.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.913+0000 7f066c98d700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f06680d29a0 0x7f06680cf6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:28.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.913+0000 7f066c98d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f066815ca90 0x7f06680b2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:28.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.913+0000 7f066c98d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06681e4320 con 0x7f06680d2550 2023-05-20T01:25:28.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.913+0000 7f066c98d700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06680b3610 con 0x7f066815ca90 2023-05-20T01:25:28.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.913+0000 7f066c98d700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06680b38f0 con 0x7f06680d29a0 2023-05-20T01:25:28.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.913+0000 7f066659c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06680d2550 0x7f06680ca310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:28.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.913+0000 7f066659c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06680d2550 0x7f06680ca310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36558/0 (socket says 172.21.15.16:36558) 2023-05-20T01:25:28.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.913+0000 7f066659c700 1 -- 172.21.15.16:0/708476371 learned_addr learned my addr 172.21.15.16:0/708476371 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:28.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.913+0000 7f0665d9b700 1 --2- 172.21.15.16:0/708476371 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f06680d29a0 0x7f06680cf6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:28.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.913+0000 7f0666d9d700 1 --2- 172.21.15.16:0/708476371 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f066815ca90 0x7f06680b2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:28.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.913+0000 7f066659c700 1 -- 172.21.15.16:0/708476371 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f06680d29a0 msgr2=0x7f06680cf6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:28.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.913+0000 7f066659c700 1 --2- 172.21.15.16:0/708476371 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f06680d29a0 0x7f06680cf6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:28.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.913+0000 7f066659c700 1 -- 172.21.15.16:0/708476371 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f066815ca90 msgr2=0x7f06680b2ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:28.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.913+0000 7f066659c700 1 --2- 172.21.15.16:0/708476371 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f066815ca90 0x7f06680b2ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:28.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.913+0000 7f066659c700 1 -- 172.21.15.16:0/708476371 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0658009550 con 0x7f06680d2550 2023-05-20T01:25:28.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.913+0000 7f0665d9b700 1 --2- 172.21.15.16:0/708476371 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f06680d29a0 0x7f06680cf6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:25:28.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.913+0000 7f0666d9d700 1 --2- 172.21.15.16:0/708476371 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f066815ca90 0x7f06680b2ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:25:28.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.913+0000 7f066659c700 1 --2- 172.21.15.16:0/708476371 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06680d2550 0x7f06680ca310 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f065c00e870 tx=0x7f065c00ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:28.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.913+0000 7f0656ffd700 1 -- 172.21.15.16:0/708476371 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f065c009590 con 0x7f06680d2550 2023-05-20T01:25:28.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.914+0000 7f0656ffd700 1 -- 172.21.15.16:0/708476371 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f065c00ee70 con 0x7f06680d2550 2023-05-20T01:25:28.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.914+0000 7f066c98d700 1 -- 172.21.15.16:0/708476371 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06680d2030 con 0x7f06680d2550 2023-05-20T01:25:28.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.914+0000 7f0656ffd700 1 -- 172.21.15.16:0/708476371 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f065c00ca50 con 0x7f06680d2550 2023-05-20T01:25:28.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.914+0000 7f066c98d700 1 -- 172.21.15.16:0/708476371 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06680d1d50 con 0x7f06680d2550 2023-05-20T01:25:28.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.914+0000 7f066c98d700 1 -- 172.21.15.16:0/708476371 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f066815ce60 con 0x7f06680d2550 2023-05-20T01:25:28.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.915+0000 7f0656ffd700 1 -- 172.21.15.16:0/708476371 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f065c00cbb0 con 0x7f06680d2550 2023-05-20T01:25:28.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.915+0000 7f0656ffd700 1 --2- 172.21.15.16:0/708476371 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f064806ace0 0x7f064806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:28.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.915+0000 7f0656ffd700 1 -- 172.21.15.16:0/708476371 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6786+0+0 (secure 0 0 0) 0x7f065c0897d0 con 0x7f06680d2550 2023-05-20T01:25:28.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.915+0000 7f0665d9b700 1 --2- 172.21.15.16:0/708476371 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f064806ace0 0x7f064806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:28.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.916+0000 7f0665d9b700 1 --2- 172.21.15.16:0/708476371 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f064806ace0 0x7f064806d190 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f06680b4760 tx=0x7f0658008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:28.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:28.920+0000 7f0656ffd700 1 -- 172.21.15.16:0/708476371 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f065c057f90 con 0x7f06680d2550 2023-05-20T01:25:29.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.120+0000 7f066c98d700 1 -- 172.21.15.16:0/708476371 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f06680b3ab0 con 0x7f06680d2550 2023-05-20T01:25:29.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.121+0000 7f0656ffd700 1 -- 172.21.15.16:0/708476371 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) v1 ==== 100+0+11 (secure 0 0 0) 0x7f065c00ce60 con 0x7f06680d2550 2023-05-20T01:25:29.126 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2023-05-20T01:25:29.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.123+0000 7f066c98d700 1 -- 172.21.15.16:0/708476371 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f064806ace0 msgr2=0x7f064806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:29.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.123+0000 7f066c98d700 1 --2- 172.21.15.16:0/708476371 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f064806ace0 0x7f064806d190 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f06680b4760 tx=0x7f0658008030 comp rx=0 tx=0).stop 2023-05-20T01:25:29.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.123+0000 7f066c98d700 1 -- 172.21.15.16:0/708476371 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06680d2550 msgr2=0x7f06680ca310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:29.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.123+0000 7f066c98d700 1 --2- 172.21.15.16:0/708476371 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06680d2550 0x7f06680ca310 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f065c00e870 tx=0x7f065c00ec30 comp rx=0 tx=0).stop 2023-05-20T01:25:29.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.123+0000 7f066c98d700 1 -- 172.21.15.16:0/708476371 shutdown_connections 2023-05-20T01:25:29.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.123+0000 7f066c98d700 1 --2- 172.21.15.16:0/708476371 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f066815ca90 0x7f06680b2ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:29.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.123+0000 7f066c98d700 1 --2- 172.21.15.16:0/708476371 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06680d2550 0x7f06680ca310 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:29.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.123+0000 7f066c98d700 1 --2- 172.21.15.16:0/708476371 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f064806ace0 0x7f064806d190 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:29.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.123+0000 7f066c98d700 1 --2- 172.21.15.16:0/708476371 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f06680d29a0 0x7f06680cf6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:29.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.123+0000 7f066c98d700 1 -- 172.21.15.16:0/708476371 >> 172.21.15.16:0/708476371 conn(0x7f066806b000 msgr2=0x7f066815bc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:29.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.123+0000 7f066c98d700 1 -- 172.21.15.16:0/708476371 shutdown_connections 2023-05-20T01:25:29.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.123+0000 7f066c98d700 1 -- 172.21.15.16:0/708476371 wait complete. 2023-05-20T01:25:29.139 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_extra pg_num 2023-05-20T01:25:29.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.264+0000 7fc5aacdb700 1 Processor -- start 2023-05-20T01:25:29.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.264+0000 7fc5aacdb700 1 -- start start 2023-05-20T01:25:29.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.265+0000 7fc5aacdb700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc5a4159b50 0x7fc5a406e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:29.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.265+0000 7fc5aacdb700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc5a40cd1c0 0x7fc5a40710b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:29.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.265+0000 7fc5aacdb700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5a40ccd30 con 0x7fc5a40cd630 2023-05-20T01:25:29.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.265+0000 7fc5aacdb700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5a40ce990 con 0x7fc5a4159b50 2023-05-20T01:25:29.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.265+0000 7fc5aacdb700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a40cc6e0 con 0x7fc5a40cd1c0 2023-05-20T01:25:29.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.265+0000 7fc5a8a77700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc5a4159b50 0x7fc5a406e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:29.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.265+0000 7fc5a9278700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fc5a40cd630 0x7fc5a40ca870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:48096/0 (socket says 172.21.15.16:48096) 2023-05-20T01:25:29.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.265+0000 7fc5a9278700 1 -- 172.21.15.16:0/3652706825 learned_addr learned my addr 172.21.15.16:0/3652706825 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:25:29.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.265+0000 7fc5a3fff700 1 --2- 172.21.15.16:0/3652706825 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc5a40cd1c0 0x7fc5a40710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:29.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.265+0000 7fc5a8a77700 1 -- 172.21.15.16:0/3652706825 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc5a40cd1c0 msgr2=0x7fc5a40710b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:29.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.265+0000 7fc5a8a77700 1 --2- 172.21.15.16:0/3652706825 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc5a40cd1c0 0x7fc5a40710b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:29.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.265+0000 7fc5a8a77700 1 -- 172.21.15.16:0/3652706825 >> v1:172.21.15.16:6789/0 conn(0x7fc5a40cd630 legacy=0x7fc5a40ca870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:29.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.265+0000 7fc5a8a77700 1 -- 172.21.15.16:0/3652706825 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5a40cc9c0 con 0x7fc5a4159b50 2023-05-20T01:25:29.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.265+0000 7fc5a37fe700 1 -- 172.21.15.16:0/3652706825 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3546788801 0 0) 0x7fc5a40ccd30 con 0x7fc5a40cd630 2023-05-20T01:25:29.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.265+0000 7fc5a3fff700 1 --2- 172.21.15.16:0/3652706825 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc5a40cd1c0 0x7fc5a40710b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:25:29.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.265+0000 7fc5a8a77700 1 --2- 172.21.15.16:0/3652706825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5a4159b50 0x7fc5a406e0a0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fc598009820 tx=0x7fc598009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70d8d65f9462921a server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:29.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.265+0000 7fc5a37fe700 1 -- 172.21.15.16:0/3652706825 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc598005e30 con 0x7fc5a4159b50 2023-05-20T01:25:29.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.265+0000 7fc5a37fe700 1 -- 172.21.15.16:0/3652706825 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc598006030 con 0x7fc5a4159b50 2023-05-20T01:25:29.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.265+0000 7fc5a37fe700 1 -- 172.21.15.16:0/3652706825 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5980063e0 con 0x7fc5a4159b50 2023-05-20T01:25:29.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.266+0000 7fc5aacdb700 1 -- 172.21.15.16:0/3652706825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5a4159b50 msgr2=0x7fc5a406e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:29.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.266+0000 7fc5aacdb700 1 --2- 172.21.15.16:0/3652706825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5a4159b50 0x7fc5a406e0a0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fc598009820 tx=0x7fc598009b30 comp rx=0 tx=0).stop 2023-05-20T01:25:29.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.266+0000 7fc5aacdb700 1 -- 172.21.15.16:0/3652706825 shutdown_connections 2023-05-20T01:25:29.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.266+0000 7fc5aacdb700 1 --2- 172.21.15.16:0/3652706825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5a4159b50 0x7fc5a406e0a0 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:29.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.266+0000 7fc5aacdb700 1 --2- 172.21.15.16:0/3652706825 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc5a40cd1c0 0x7fc5a40710b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:29.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.266+0000 7fc5aacdb700 1 -- 172.21.15.16:0/3652706825 >> 172.21.15.16:0/3652706825 conn(0x7fc5a40cffd0 msgr2=0x7fc5a40d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:29.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.266+0000 7fc5aacdb700 1 -- 172.21.15.16:0/3652706825 shutdown_connections 2023-05-20T01:25:29.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.266+0000 7fc5aacdb700 1 -- 172.21.15.16:0/3652706825 wait complete. 2023-05-20T01:25:29.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.267+0000 7fc5aacdb700 1 Processor -- start 2023-05-20T01:25:29.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.267+0000 7fc5aacdb700 1 -- start start 2023-05-20T01:25:29.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.268+0000 7fc5aacdb700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5a40cd1c0 0x7fc5a41f25a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:29.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.268+0000 7fc5aacdb700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc5a40cd630 0x7fc5a40b3100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:29.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.268+0000 7fc5aacdb700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc5a4159b50 0x7fc5a40c88f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:29.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.268+0000 7fc5aacdb700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a40c84d0 con 0x7fc5a40cd1c0 2023-05-20T01:25:29.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.268+0000 7fc5aacdb700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a41f2ef0 con 0x7fc5a40cd630 2023-05-20T01:25:29.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.268+0000 7fc5aacdb700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a41f31d0 con 0x7fc5a4159b50 2023-05-20T01:25:29.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.268+0000 7fc5a8a77700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5a40cd1c0 0x7fc5a41f25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:29.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.268+0000 7fc5a8a77700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5a40cd1c0 0x7fc5a41f25a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36570/0 (socket says 172.21.15.16:36570) 2023-05-20T01:25:29.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.268+0000 7fc5a8a77700 1 -- 172.21.15.16:0/260502407 learned_addr learned my addr 172.21.15.16:0/260502407 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:29.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.268+0000 7fc5a8a77700 1 -- 172.21.15.16:0/260502407 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc5a4159b50 msgr2=0x7fc5a40c88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:29.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.268+0000 7fc5a8a77700 1 --2- 172.21.15.16:0/260502407 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc5a4159b50 0x7fc5a40c88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:29.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.269+0000 7fc5a8a77700 1 -- 172.21.15.16:0/260502407 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc5a40cd630 msgr2=0x7fc5a40b3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:29.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.269+0000 7fc5a3fff700 1 --2- 172.21.15.16:0/260502407 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc5a40cd630 0x7fc5a40b3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:29.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.269+0000 7fc5a8a77700 1 --2- 172.21.15.16:0/260502407 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc5a40cd630 0x7fc5a40b3100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:29.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.269+0000 7fc5a8a77700 1 -- 172.21.15.16:0/260502407 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc598009430 con 0x7fc5a40cd1c0 2023-05-20T01:25:29.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.269+0000 7fc5a3fff700 1 --2- 172.21.15.16:0/260502407 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc5a40cd630 0x7fc5a40b3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:25:29.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.269+0000 7fc5a8a77700 1 --2- 172.21.15.16:0/260502407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5a40cd1c0 0x7fc5a41f25a0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fc598009b70 tx=0x7fc598004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:29.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.269+0000 7fc5a17fa700 1 -- 172.21.15.16:0/260502407 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc598004040 con 0x7fc5a40cd1c0 2023-05-20T01:25:29.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.269+0000 7fc5a17fa700 1 -- 172.21.15.16:0/260502407 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc598020070 con 0x7fc5a40cd1c0 2023-05-20T01:25:29.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.269+0000 7fc5aacdb700 1 -- 172.21.15.16:0/260502407 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5a4071e80 con 0x7fc5a40cd1c0 2023-05-20T01:25:29.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.269+0000 7fc5a17fa700 1 -- 172.21.15.16:0/260502407 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc598004400 con 0x7fc5a40cd1c0 2023-05-20T01:25:29.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.269+0000 7fc5aacdb700 1 -- 172.21.15.16:0/260502407 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5a406e5e0 con 0x7fc5a40cd1c0 2023-05-20T01:25:29.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.270+0000 7fc5aacdb700 1 -- 172.21.15.16:0/260502407 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5a40cec70 con 0x7fc5a40cd1c0 2023-05-20T01:25:29.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.270+0000 7fc5a17fa700 1 -- 172.21.15.16:0/260502407 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc598022030 con 0x7fc5a40cd1c0 2023-05-20T01:25:29.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.271+0000 7fc5a17fa700 1 --2- 172.21.15.16:0/260502407 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc58806ace0 0x7fc58806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:29.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.271+0000 7fc5a17fa700 1 -- 172.21.15.16:0/260502407 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6786+0+0 (secure 0 0 0) 0x7fc59808ac20 con 0x7fc5a40cd1c0 2023-05-20T01:25:29.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.271+0000 7fc5a3fff700 1 --2- 172.21.15.16:0/260502407 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc58806ace0 0x7fc58806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:29.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.271+0000 7fc5a3fff700 1 --2- 172.21.15.16:0/260502407 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc58806ace0 0x7fc58806d190 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fc5a4070c90 tx=0x7fc590008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:29.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.275+0000 7fc5a17fa700 1 -- 172.21.15.16:0/260502407 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fc598055a80 con 0x7fc5a40cd1c0 2023-05-20T01:25:29.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.478+0000 7fc5aacdb700 1 -- 172.21.15.16:0/260502407 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"} v 0) v1 -- 0x7fc5a40747b0 con 0x7fc5a40cd1c0 2023-05-20T01:25:29.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.478+0000 7fc5a17fa700 1 -- 172.21.15.16:0/260502407 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"}]=0 v91) v1 ==== 106+0+11 (secure 0 0 0) 0x7fc5980590a0 con 0x7fc5a40cd1c0 2023-05-20T01:25:29.482 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-05-20T01:25:29.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.480+0000 7fc5aacdb700 1 -- 172.21.15.16:0/260502407 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc58806ace0 msgr2=0x7fc58806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:29.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.480+0000 7fc5aacdb700 1 --2- 172.21.15.16:0/260502407 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc58806ace0 0x7fc58806d190 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fc5a4070c90 tx=0x7fc590008040 comp rx=0 tx=0).stop 2023-05-20T01:25:29.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.480+0000 7fc5aacdb700 1 -- 172.21.15.16:0/260502407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5a40cd1c0 msgr2=0x7fc5a41f25a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:29.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.480+0000 7fc5aacdb700 1 --2- 172.21.15.16:0/260502407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5a40cd1c0 0x7fc5a41f25a0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fc598009b70 tx=0x7fc598004750 comp rx=0 tx=0).stop 2023-05-20T01:25:29.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.480+0000 7fc5aacdb700 1 -- 172.21.15.16:0/260502407 shutdown_connections 2023-05-20T01:25:29.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.480+0000 7fc5aacdb700 1 --2- 172.21.15.16:0/260502407 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc58806ace0 0x7fc58806d190 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:29.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.480+0000 7fc5aacdb700 1 --2- 172.21.15.16:0/260502407 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc5a4159b50 0x7fc5a40c88f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:29.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.480+0000 7fc5aacdb700 1 --2- 172.21.15.16:0/260502407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5a40cd1c0 0x7fc5a41f25a0 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:29.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.480+0000 7fc5aacdb700 1 --2- 172.21.15.16:0/260502407 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc5a40cd630 0x7fc5a40b3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:29.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.480+0000 7fc5aacdb700 1 -- 172.21.15.16:0/260502407 >> 172.21.15.16:0/260502407 conn(0x7fc5a40cffd0 msgr2=0x7fc5a40cb940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:29.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.480+0000 7fc5aacdb700 1 -- 172.21.15.16:0/260502407 shutdown_connections 2023-05-20T01:25:29.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.480+0000 7fc5aacdb700 1 -- 172.21.15.16:0/260502407 wait complete. 2023-05-20T01:25:29.496 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-05-20T01:25:29.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.619+0000 7f2bd5397700 1 Processor -- start 2023-05-20T01:25:29.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.619+0000 7f2bd5397700 1 -- start start 2023-05-20T01:25:29.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.620+0000 7f2bd5397700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2bd006b730 0x7f2bd00b53d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:29.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.620+0000 7f2bd5397700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bd015d020 con 0x7f2bd015cc50 2023-05-20T01:25:29.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.620+0000 7f2bd5397700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bd00d1bf0 con 0x7f2bd00cf500 2023-05-20T01:25:29.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.620+0000 7f2bd5397700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd00cd720 con 0x7f2bd006b730 2023-05-20T01:25:29.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.620+0000 7f2bceffd700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f2bd015cc50 0x7f2bd00b4cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:48106/0 (socket says 172.21.15.16:48106) 2023-05-20T01:25:29.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.620+0000 7f2bceffd700 1 -- 172.21.15.16:0/34122159 learned_addr learned my addr 172.21.15.16:0/34122159 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:25:29.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.620+0000 7f2bcdffb700 1 -- 172.21.15.16:0/34122159 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1298396050 0 0) 0x7f2bc0001110 con 0x7f2bd015cc50 2023-05-20T01:25:29.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.620+0000 7f2bce7fc700 1 --2- 172.21.15.16:0/34122159 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2bd006b730 0x7f2bd00b53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:29.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.620+0000 7f2bcdffb700 1 -- 172.21.15.16:0/34122159 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2bb80035b0 con 0x7f2bd015cc50 2023-05-20T01:25:29.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.620+0000 7f2bcdffb700 1 -- 172.21.15.16:0/34122159 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3458877424 0 0) 0x7f2bb80035b0 con 0x7f2bd015cc50 2023-05-20T01:25:29.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.620+0000 7f2bcdffb700 1 -- 172.21.15.16:0/34122159 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2bd006b730 msgr2=0x7f2bd00b53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:29.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.620+0000 7f2bcdffb700 1 --2- 172.21.15.16:0/34122159 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2bd006b730 0x7f2bd00b53d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:29.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.620+0000 7f2bcdffb700 1 -- 172.21.15.16:0/34122159 >> v1:172.21.15.132:6789/0 conn(0x7f2bd00cf500 legacy=0x7f2bd00cf960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:29.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.620+0000 7f2bcdffb700 1 -- 172.21.15.16:0/34122159 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bd00d18c0 con 0x7f2bd015cc50 2023-05-20T01:25:29.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.620+0000 7f2bcdffb700 1 -- 172.21.15.16:0/34122159 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7f2bc0001800 con 0x7f2bd015cc50 2023-05-20T01:25:29.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.620+0000 7f2bcdffb700 1 -- 172.21.15.16:0/34122159 >> v1:172.21.15.16:6789/0 conn(0x7f2bd015cc50 legacy=0x7f2bd00b4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:29.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.620+0000 7f2bcdffb700 1 --2- 172.21.15.16:0/34122159 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bb8003aa0 0x7f2bb8003eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:29.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.621+0000 7f2bcdffb700 1 --2- 172.21.15.16:0/34122159 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2bb8004850 0x7f2bb8004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:29.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.621+0000 7f2bcdffb700 1 --2- 172.21.15.16:0/34122159 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2bd006b730 0x7f2bb800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:29.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.621+0000 7f2bcdffb700 1 -- 172.21.15.16:0/34122159 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb8005500 con 0x7f2bb8003aa0 2023-05-20T01:25:29.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.621+0000 7f2bcdffb700 1 -- 172.21.15.16:0/34122159 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb8005670 con 0x7f2bb8004850 2023-05-20T01:25:29.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.621+0000 7f2bcdffb700 1 -- 172.21.15.16:0/34122159 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb800bf50 con 0x7f2bd006b730 2023-05-20T01:25:29.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.621+0000 7f2bceffd700 1 --2- 172.21.15.16:0/34122159 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bb8003aa0 0x7f2bb8003eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:29.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.621+0000 7f2bce7fc700 1 --2- 172.21.15.16:0/34122159 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2bb8004850 0x7f2bb8004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:29.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.621+0000 7f2bcf7fe700 1 --2- 172.21.15.16:0/34122159 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2bd006b730 0x7f2bb800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:29.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.621+0000 7f2bceffd700 1 -- 172.21.15.16:0/34122159 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2bd006b730 msgr2=0x7f2bb800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:29.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.621+0000 7f2bceffd700 1 --2- 172.21.15.16:0/34122159 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2bd006b730 0x7f2bb800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:29.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.621+0000 7f2bceffd700 1 -- 172.21.15.16:0/34122159 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2bb8004850 msgr2=0x7f2bb8004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:29.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.621+0000 7f2bceffd700 1 --2- 172.21.15.16:0/34122159 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2bb8004850 0x7f2bb8004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:29.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.621+0000 7f2bceffd700 1 -- 172.21.15.16:0/34122159 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bb80035b0 con 0x7f2bb8003aa0 2023-05-20T01:25:29.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.621+0000 7f2bceffd700 1 --2- 172.21.15.16:0/34122159 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bb8003aa0 0x7f2bb8003eb0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f2bc000c240 tx=0x7f2bb8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0094f3f6641ccbd server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:29.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.621+0000 7f2bcdffb700 1 -- 172.21.15.16:0/34122159 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc0007db0 con 0x7f2bb8003aa0 2023-05-20T01:25:29.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.621+0000 7f2bcdffb700 1 -- 172.21.15.16:0/34122159 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bc000cde0 con 0x7f2bb8003aa0 2023-05-20T01:25:29.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.621+0000 7f2bcdffb700 1 -- 172.21.15.16:0/34122159 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc0005b80 con 0x7f2bb8003aa0 2023-05-20T01:25:29.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.622+0000 7f2bd5397700 1 -- 172.21.15.16:0/34122159 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bb8003aa0 msgr2=0x7f2bb8003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:29.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.622+0000 7f2bd5397700 1 --2- 172.21.15.16:0/34122159 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bb8003aa0 0x7f2bb8003eb0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f2bc000c240 tx=0x7f2bb8005880 comp rx=0 tx=0).stop 2023-05-20T01:25:29.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.622+0000 7f2bcf7fe700 1 -- 172.21.15.16:0/34122159 reap_dead start 2023-05-20T01:25:29.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.622+0000 7f2bd5397700 1 -- 172.21.15.16:0/34122159 shutdown_connections 2023-05-20T01:25:29.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.622+0000 7f2bd5397700 1 -- 172.21.15.16:0/34122159 >> 172.21.15.16:0/34122159 conn(0x7f2bd0072860 msgr2=0x7f2bd006b020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:29.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.622+0000 7f2bd5397700 1 -- 172.21.15.16:0/34122159 shutdown_connections 2023-05-20T01:25:29.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.622+0000 7f2bd5397700 1 -- 172.21.15.16:0/34122159 wait complete. 2023-05-20T01:25:29.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.623+0000 7f2bd5397700 1 Processor -- start 2023-05-20T01:25:29.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.623+0000 7f2bd5397700 1 -- start start 2023-05-20T01:25:29.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.624+0000 7f2bd5397700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2bb8003aa0 0x7f2bd0142cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:29.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.624+0000 7f2bd5397700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2bd00b8080 0x7f2bd0145970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:29.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.624+0000 7f2bd5397700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bd0145eb0 0x7f2bd00b6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:29.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.624+0000 7f2bd5397700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd00b9100 con 0x7f2bd0145eb0 2023-05-20T01:25:29.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.624+0000 7f2bd5397700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd0143210 con 0x7f2bb8003aa0 2023-05-20T01:25:29.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.624+0000 7f2bd5397700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd00cc8f0 con 0x7f2bd00b8080 2023-05-20T01:25:29.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.624+0000 7f2bcf7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bd0145eb0 0x7f2bd00b6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:29.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.624+0000 7f2bcf7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bd0145eb0 0x7f2bd00b6090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36594/0 (socket says 172.21.15.16:36594) 2023-05-20T01:25:29.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.624+0000 7f2bcf7fe700 1 -- 172.21.15.16:0/985683614 learned_addr learned my addr 172.21.15.16:0/985683614 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:29.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.624+0000 7f2bce7fc700 1 --2- 172.21.15.16:0/985683614 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2bd00b8080 0x7f2bd0145970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:29.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.624+0000 7f2bceffd700 1 --2- 172.21.15.16:0/985683614 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2bb8003aa0 0x7f2bd0142cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:29.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.624+0000 7f2bcf7fe700 1 -- 172.21.15.16:0/985683614 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2bd00b8080 msgr2=0x7f2bd0145970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:29.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.624+0000 7f2bcf7fe700 1 --2- 172.21.15.16:0/985683614 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2bd00b8080 0x7f2bd0145970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:29.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.624+0000 7f2bcf7fe700 1 -- 172.21.15.16:0/985683614 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2bb8003aa0 msgr2=0x7f2bd0142cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:29.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.624+0000 7f2bcf7fe700 1 --2- 172.21.15.16:0/985683614 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2bb8003aa0 0x7f2bd0142cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:29.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.624+0000 7f2bcf7fe700 1 -- 172.21.15.16:0/985683614 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bd00cdca0 con 0x7f2bd0145eb0 2023-05-20T01:25:29.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.624+0000 7f2bceffd700 1 --2- 172.21.15.16:0/985683614 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2bb8003aa0 0x7f2bd0142cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:25:29.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.625+0000 7f2bcf7fe700 1 --2- 172.21.15.16:0/985683614 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bd0145eb0 0x7f2bd00b6090 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f2bd008c1c0 tx=0x7f2bc800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:29.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.625+0000 7f2bb77fe700 1 -- 172.21.15.16:0/985683614 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc8003c10 con 0x7f2bd0145eb0 2023-05-20T01:25:29.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.625+0000 7f2bb77fe700 1 -- 172.21.15.16:0/985683614 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bc8003d90 con 0x7f2bd0145eb0 2023-05-20T01:25:29.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.625+0000 7f2bd5397700 1 -- 172.21.15.16:0/985683614 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bd00cd9c0 con 0x7f2bd0145eb0 2023-05-20T01:25:29.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.625+0000 7f2bb77fe700 1 -- 172.21.15.16:0/985683614 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc8005310 con 0x7f2bd0145eb0 2023-05-20T01:25:29.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.625+0000 7f2bd5397700 1 -- 172.21.15.16:0/985683614 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bd00cdf80 con 0x7f2bd0145eb0 2023-05-20T01:25:29.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.626+0000 7f2bd5397700 1 -- 172.21.15.16:0/985683614 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bd004e8c0 con 0x7f2bd0145eb0 2023-05-20T01:25:29.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.626+0000 7f2bb77fe700 1 -- 172.21.15.16:0/985683614 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2bc8005470 con 0x7f2bd0145eb0 2023-05-20T01:25:29.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.627+0000 7f2bb77fe700 1 --2- 172.21.15.16:0/985683614 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f2bbc06ace0 0x7f2bbc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:29.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.627+0000 7f2bb77fe700 1 -- 172.21.15.16:0/985683614 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6786+0+0 (secure 0 0 0) 0x7f2bc8089780 con 0x7f2bd0145eb0 2023-05-20T01:25:29.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.627+0000 7f2bceffd700 1 --2- 172.21.15.16:0/985683614 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f2bbc06ace0 0x7f2bbc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:29.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.627+0000 7f2bceffd700 1 --2- 172.21.15.16:0/985683614 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f2bbc06ace0 0x7f2bbc06d190 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f2bc0001ac0 tx=0x7f2bc0007000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:29.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.631+0000 7f2bb77fe700 1 -- 172.21.15.16:0/985683614 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f2bc80545e0 con 0x7f2bd0145eb0 2023-05-20T01:25:29.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.847+0000 7f2bd5397700 1 -- 172.21.15.16:0/985683614 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f2bd00b84f0 con 0x7f2bd0145eb0 2023-05-20T01:25:29.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.848+0000 7f2bb77fe700 1 -- 172.21.15.16:0/985683614 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v91) v1 ==== 80+0+0 (secure 0 0 0) 0x7f2bc8057c00 con 0x7f2bd0145eb0 2023-05-20T01:25:29.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.850+0000 7f2bd5397700 1 -- 172.21.15.16:0/985683614 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f2bbc06ace0 msgr2=0x7f2bbc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:29.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.850+0000 7f2bd5397700 1 --2- 172.21.15.16:0/985683614 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f2bbc06ace0 0x7f2bbc06d190 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f2bc0001ac0 tx=0x7f2bc0007000 comp rx=0 tx=0).stop 2023-05-20T01:25:29.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.850+0000 7f2bd5397700 1 -- 172.21.15.16:0/985683614 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bd0145eb0 msgr2=0x7f2bd00b6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:29.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.850+0000 7f2bd5397700 1 --2- 172.21.15.16:0/985683614 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bd0145eb0 0x7f2bd00b6090 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f2bd008c1c0 tx=0x7f2bc800ee80 comp rx=0 tx=0).stop 2023-05-20T01:25:29.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.850+0000 7f2bd5397700 1 -- 172.21.15.16:0/985683614 shutdown_connections 2023-05-20T01:25:29.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.850+0000 7f2bd5397700 1 --2- 172.21.15.16:0/985683614 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bd0145eb0 0x7f2bd00b6090 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:29.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.850+0000 7f2bd5397700 1 --2- 172.21.15.16:0/985683614 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2bb8003aa0 0x7f2bd0142cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:29.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.850+0000 7f2bd5397700 1 --2- 172.21.15.16:0/985683614 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f2bbc06ace0 0x7f2bbc06d190 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:29.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.850+0000 7f2bd5397700 1 --2- 172.21.15.16:0/985683614 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2bd00b8080 0x7f2bd0145970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:29.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.850+0000 7f2bd5397700 1 -- 172.21.15.16:0/985683614 >> 172.21.15.16:0/985683614 conn(0x7f2bd0072860 msgr2=0x7f2bd015bed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:29.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.850+0000 7f2bd5397700 1 -- 172.21.15.16:0/985683614 shutdown_connections 2023-05-20T01:25:29.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:29.850+0000 7f2bd5397700 1 -- 172.21.15.16:0/985683614 wait complete. 2023-05-20T01:25:29.856 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2023-05-20T01:25:29.867 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-05-20T01:25:29.868 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:25:29.868 DEBUG:teuthology.orchestra.run.smithi016:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-05-20T01:25:29.938 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2023-05-20T01:25:29.938 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:25:29.939 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-05-20T01:25:29.971 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-20T01:25:30.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.132+0000 7f34216a7700 1 Processor -- start 2023-05-20T01:25:30.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.132+0000 7f34216a7700 1 -- start start 2023-05-20T01:25:30.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.133+0000 7f34216a7700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f341c06a510 0x7f341c15e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:30.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.133+0000 7f34216a7700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f341c15e060 con 0x7f341c071340 2023-05-20T01:25:30.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.133+0000 7f34216a7700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f341c15d450 con 0x7f341c06a8e0 2023-05-20T01:25:30.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.133+0000 7f34216a7700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f341c15d730 con 0x7f341c06a510 2023-05-20T01:25:30.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.133+0000 7f341b7fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f341c071340 0x7f341c071780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:48132/0 (socket says 172.21.15.16:48132) 2023-05-20T01:25:30.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.133+0000 7f341b7fe700 1 -- 172.21.15.16:0/2191246297 learned_addr learned my addr 172.21.15.16:0/2191246297 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:25:30.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.134+0000 7f3419ffb700 1 -- 172.21.15.16:0/2191246297 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3260048361 0 0) 0x7f341c15e060 con 0x7f341c071340 2023-05-20T01:25:30.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.134+0000 7f341affd700 1 --2- 172.21.15.16:0/2191246297 >> v2:172.21.15.132:3300/0 conn(0x7f341c06a510 0x7f341c15e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:30.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.134+0000 7f3419ffb700 1 -- 172.21.15.16:0/2191246297 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f34080035b0 con 0x7f341c071340 2023-05-20T01:25:30.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.134+0000 7f3419ffb700 1 -- 172.21.15.16:0/2191246297 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2583516754 0 0) 0x7f34080035b0 con 0x7f341c071340 2023-05-20T01:25:30.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.134+0000 7f3419ffb700 1 -- 172.21.15.16:0/2191246297 >> v2:172.21.15.132:3300/0 conn(0x7f341c06a510 msgr2=0x7f341c15e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:30.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.134+0000 7f3419ffb700 1 --2- 172.21.15.16:0/2191246297 >> v2:172.21.15.132:3300/0 conn(0x7f341c06a510 0x7f341c15e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:30.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.134+0000 7f3419ffb700 1 -- 172.21.15.16:0/2191246297 >> v1:172.21.15.132:6789/0 conn(0x7f341c06a8e0 legacy=0x7f341c0cf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:30.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.134+0000 7f3419ffb700 1 -- 172.21.15.16:0/2191246297 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f341c15da10 con 0x7f341c071340 2023-05-20T01:25:30.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.134+0000 7f3419ffb700 1 -- 172.21.15.16:0/2191246297 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7f34140019f0 con 0x7f341c071340 2023-05-20T01:25:30.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.134+0000 7f3419ffb700 1 -- 172.21.15.16:0/2191246297 >> v1:172.21.15.16:6789/0 conn(0x7f341c071340 legacy=0x7f341c071780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:30.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.134+0000 7f3419ffb700 1 --2- 172.21.15.16:0/2191246297 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3408003a70 0x7f3408003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:30.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.134+0000 7f3419ffb700 1 --2- 172.21.15.16:0/2191246297 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3408004850 0x7f3408004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:30.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.134+0000 7f3419ffb700 1 --2- 172.21.15.16:0/2191246297 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3408005200 0x7f340800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:30.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.134+0000 7f3419ffb700 1 -- 172.21.15.16:0/2191246297 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f341c15d730 con 0x7f3408004850 2023-05-20T01:25:30.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.134+0000 7f3419ffb700 1 -- 172.21.15.16:0/2191246297 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f340800c0e0 con 0x7f3408003a70 2023-05-20T01:25:30.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.134+0000 7f3419ffb700 1 -- 172.21.15.16:0/2191246297 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f340800c2a0 con 0x7f3408005200 2023-05-20T01:25:30.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.134+0000 7f341a7fc700 1 --2- 172.21.15.16:0/2191246297 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3408004850 0x7f3408004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:30.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.134+0000 7f341affd700 1 --2- 172.21.15.16:0/2191246297 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3408003a70 0x7f3408003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:30.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.134+0000 7f341b7fe700 1 --2- 172.21.15.16:0/2191246297 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3408005200 0x7f340800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:30.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.135+0000 7f341a7fc700 1 -- 172.21.15.16:0/2191246297 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3408005200 msgr2=0x7f340800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:30.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.135+0000 7f341a7fc700 1 --2- 172.21.15.16:0/2191246297 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3408005200 0x7f340800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:30.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.135+0000 7f341a7fc700 1 -- 172.21.15.16:0/2191246297 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3408003a70 msgr2=0x7f3408003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:30.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.135+0000 7f341a7fc700 1 --2- 172.21.15.16:0/2191246297 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3408003a70 0x7f3408003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:30.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.135+0000 7f341a7fc700 1 -- 172.21.15.16:0/2191246297 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34080035b0 con 0x7f3408004850 2023-05-20T01:25:30.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.135+0000 7f341b7fe700 1 --2- 172.21.15.16:0/2191246297 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3408005200 0x7f340800ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:25:30.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.135+0000 7f341b7fe700 1 -- 172.21.15.16:0/2191246297 reap_dead start 2023-05-20T01:25:30.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.135+0000 7f341a7fc700 1 --2- 172.21.15.16:0/2191246297 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3408004850 0x7f3408004cc0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f34040017c0 tx=0x7f3408005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8096754829ae5e4 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:30.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.135+0000 7f3419ffb700 1 -- 172.21.15.16:0/2191246297 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34040052d0 con 0x7f3408004850 2023-05-20T01:25:30.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.135+0000 7f3419ffb700 1 -- 172.21.15.16:0/2191246297 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3404005430 con 0x7f3408004850 2023-05-20T01:25:30.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.135+0000 7f3419ffb700 1 -- 172.21.15.16:0/2191246297 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34040038c0 con 0x7f3408004850 2023-05-20T01:25:30.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.135+0000 7f34216a7700 1 -- 172.21.15.16:0/2191246297 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3408004850 msgr2=0x7f3408004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:30.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.135+0000 7f34216a7700 1 --2- 172.21.15.16:0/2191246297 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3408004850 0x7f3408004cc0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f34040017c0 tx=0x7f3408005880 comp rx=0 tx=0).stop 2023-05-20T01:25:30.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.135+0000 7f34216a7700 1 -- 172.21.15.16:0/2191246297 shutdown_connections 2023-05-20T01:25:30.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.135+0000 7f34216a7700 1 --2- 172.21.15.16:0/2191246297 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3408004850 0x7f3408004cc0 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:30.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.135+0000 7f34216a7700 1 -- 172.21.15.16:0/2191246297 >> 172.21.15.16:0/2191246297 conn(0x7f341c0d0c30 msgr2=0x7f341c0b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:30.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.135+0000 7f34216a7700 1 -- 172.21.15.16:0/2191246297 shutdown_connections 2023-05-20T01:25:30.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.135+0000 7f34216a7700 1 -- 172.21.15.16:0/2191246297 wait complete. 2023-05-20T01:25:30.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.137+0000 7f34216a7700 1 Processor -- start 2023-05-20T01:25:30.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.137+0000 7f34216a7700 1 -- start start 2023-05-20T01:25:30.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.138+0000 7f34216a7700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3408004850 0x7f341c0b3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:30.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.138+0000 7f34216a7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f341c15a630 0x7f341c153410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:30.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.138+0000 7f34216a7700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f341c157600 0x7f341c0c9090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:30.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.138+0000 7f34216a7700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f341c0c3b60 con 0x7f341c15a630 2023-05-20T01:25:30.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.138+0000 7f34216a7700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f341c0c3e40 con 0x7f3408004850 2023-05-20T01:25:30.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.138+0000 7f34216a7700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f341c0c4120 con 0x7f341c157600 2023-05-20T01:25:30.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.138+0000 7f341a7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f341c15a630 0x7f341c153410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:30.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.138+0000 7f341a7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f341c15a630 0x7f341c153410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36614/0 (socket says 172.21.15.16:36614) 2023-05-20T01:25:30.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.138+0000 7f341a7fc700 1 -- 172.21.15.16:0/3741233983 learned_addr learned my addr 172.21.15.16:0/3741233983 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:30.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.138+0000 7f341affd700 1 --2- 172.21.15.16:0/3741233983 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3408004850 0x7f341c0b3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:30.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.138+0000 7f341b7fe700 1 --2- 172.21.15.16:0/3741233983 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f341c157600 0x7f341c0c9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:30.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.138+0000 7f341a7fc700 1 -- 172.21.15.16:0/3741233983 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f341c157600 msgr2=0x7f341c0c9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:30.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.138+0000 7f341a7fc700 1 --2- 172.21.15.16:0/3741233983 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f341c157600 0x7f341c0c9090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:30.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.138+0000 7f341a7fc700 1 -- 172.21.15.16:0/3741233983 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3408004850 msgr2=0x7f341c0b3850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:30.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.138+0000 7f341a7fc700 1 --2- 172.21.15.16:0/3741233983 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3408004850 0x7f341c0b3850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:30.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.138+0000 7f341a7fc700 1 -- 172.21.15.16:0/3741233983 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f341c06ad50 con 0x7f341c15a630 2023-05-20T01:25:30.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.138+0000 7f341a7fc700 1 --2- 172.21.15.16:0/3741233983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f341c15a630 0x7f341c153410 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f3404015040 tx=0x7f3404001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:30.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.138+0000 7f34137fe700 1 -- 172.21.15.16:0/3741233983 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3404001290 con 0x7f341c15a630 2023-05-20T01:25:30.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.138+0000 7f34137fe700 1 -- 172.21.15.16:0/3741233983 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f340401d480 con 0x7f341c15a630 2023-05-20T01:25:30.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.138+0000 7f34216a7700 1 -- 172.21.15.16:0/3741233983 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f341c15dcf0 con 0x7f341c15a630 2023-05-20T01:25:30.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.138+0000 7f34137fe700 1 -- 172.21.15.16:0/3741233983 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f340401d830 con 0x7f341c15a630 2023-05-20T01:25:30.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.138+0000 7f34216a7700 1 -- 172.21.15.16:0/3741233983 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f341c15d080 con 0x7f341c15a630 2023-05-20T01:25:30.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.139+0000 7f34216a7700 1 -- 172.21.15.16:0/3741233983 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f341c0c4a50 con 0x7f341c15a630 2023-05-20T01:25:30.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.140+0000 7f34137fe700 1 -- 172.21.15.16:0/3741233983 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3404019040 con 0x7f341c15a630 2023-05-20T01:25:30.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.140+0000 7f34137fe700 1 --2- 172.21.15.16:0/3741233983 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f33fc06ace0 0x7f33fc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:30.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.140+0000 7f34137fe700 1 -- 172.21.15.16:0/3741233983 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6786+0+0 (secure 0 0 0) 0x7f340408a330 con 0x7f341c15a630 2023-05-20T01:25:30.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.140+0000 7f341affd700 1 --2- 172.21.15.16:0/3741233983 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f33fc06ace0 0x7f33fc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:30.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.141+0000 7f341affd700 1 --2- 172.21.15.16:0/3741233983 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f33fc06ace0 0x7f33fc06d190 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f341c0c9c10 tx=0x7f340c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:30.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.145+0000 7f34137fe700 1 -- 172.21.15.16:0/3741233983 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f3404055190 con 0x7f341c15a630 2023-05-20T01:25:30.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.387+0000 7f34216a7700 1 -- 172.21.15.16:0/3741233983 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f341c0747b0 con 0x7f341c15a630 2023-05-20T01:25:30.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.387+0000 7f34137fe700 1 -- 172.21.15.16:0/3741233983 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+4676 (secure 0 0 0) 0x7f3404017090 con 0x7f341c15a630 2023-05-20T01:25:30.392 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:25:30.392 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":58,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6077,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/1241894038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":1241894038},{"type":"v1","addr":"172.21.15.132:6835","nonce":1241894038}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":6562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/3771738903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3771738903},{"type":"v1","addr":"172.21.15.132:6834","nonce":3771738903}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":7101,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/3876038711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":3876038711},{"type":"v1","addr":"172.21.15.16:6836","nonce":3876038711}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-20T01:25:11.987993+0000","modified":"2023-05-20T01:25:25.928170+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7104},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7104":{"gid":7104,"name":"c","rank":0,"incarnation":53,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6837/28561908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":28561908},{"type":"v1","addr":"172.21.15.16:6837","nonce":28561908}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13,14],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-05-20T01:25:30.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.389+0000 7f34216a7700 1 -- 172.21.15.16:0/3741233983 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f33fc06ace0 msgr2=0x7f33fc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:30.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.389+0000 7f34216a7700 1 --2- 172.21.15.16:0/3741233983 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f33fc06ace0 0x7f33fc06d190 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f341c0c9c10 tx=0x7f340c008040 comp rx=0 tx=0).stop 2023-05-20T01:25:30.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.389+0000 7f34216a7700 1 -- 172.21.15.16:0/3741233983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f341c15a630 msgr2=0x7f341c153410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:30.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.389+0000 7f34216a7700 1 --2- 172.21.15.16:0/3741233983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f341c15a630 0x7f341c153410 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f3404015040 tx=0x7f3404001120 comp rx=0 tx=0).stop 2023-05-20T01:25:30.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.389+0000 7f34216a7700 1 -- 172.21.15.16:0/3741233983 shutdown_connections 2023-05-20T01:25:30.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.389+0000 7f34216a7700 1 --2- 172.21.15.16:0/3741233983 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f33fc06ace0 0x7f33fc06d190 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:30.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.389+0000 7f34216a7700 1 --2- 172.21.15.16:0/3741233983 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f341c157600 0x7f341c0c9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:30.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.389+0000 7f34216a7700 1 --2- 172.21.15.16:0/3741233983 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3408004850 0x7f341c0b3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:30.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.389+0000 7f34216a7700 1 --2- 172.21.15.16:0/3741233983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f341c15a630 0x7f341c153410 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:30.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.390+0000 7f34216a7700 1 -- 172.21.15.16:0/3741233983 >> 172.21.15.16:0/3741233983 conn(0x7f341c0d0c30 msgr2=0x7f341c0cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:30.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.390+0000 7f34216a7700 1 -- 172.21.15.16:0/3741233983 shutdown_connections 2023-05-20T01:25:30.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.390+0000 7f34216a7700 1 -- 172.21.15.16:0/3741233983 wait complete. 2023-05-20T01:25:30.396 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 58 2023-05-20T01:25:30.406 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2023-05-20T01:25:30.557 INFO:teuthology.orchestra.run.smithi016.stdout:{"message":"","return_code":0} 2023-05-20T01:25:30.558 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2023-05-20T01:25:30.558 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-05-20T01:25:30.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.680+0000 7f3115477700 1 Processor -- start 2023-05-20T01:25:30.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.680+0000 7f3115477700 1 -- start start 2023-05-20T01:25:30.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.681+0000 7f3115477700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f31100cfc50 0x7f31100d2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:30.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.681+0000 7f3115477700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f31100b4750 0x7f31100d0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:30.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.681+0000 7f3115477700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f31100b2b40 0x7f31100b2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:30.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.681+0000 7f3115477700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31100cfad0 con 0x7f31100b2b40 2023-05-20T01:25:30.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.681+0000 7f3115477700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31100d3df0 con 0x7f31100cfc50 2023-05-20T01:25:30.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.681+0000 7f3115477700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31100d0b90 con 0x7f31100b4750 2023-05-20T01:25:30.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.681+0000 7f310f7fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f31100b2b40 0x7f31100b2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:30.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.681+0000 7f310f7fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f31100b2b40 0x7f31100b2fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36624/0 (socket says 172.21.15.16:36624) 2023-05-20T01:25:30.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.681+0000 7f310f7fe700 1 -- 172.21.15.16:0/2346828757 learned_addr learned my addr 172.21.15.16:0/2346828757 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:30.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.681+0000 7f310effd700 1 --2- 172.21.15.16:0/2346828757 >> v2:172.21.15.132:3300/0 conn(0x7f31100cfc50 0x7f31100d2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:30.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.681+0000 7f310e7fc700 1 --2- 172.21.15.16:0/2346828757 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f31100b4750 0x7f31100d0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:30.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.681+0000 7f310f7fe700 1 -- 172.21.15.16:0/2346828757 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f31100b4750 msgr2=0x7f31100d0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:30.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.681+0000 7f310f7fe700 1 --2- 172.21.15.16:0/2346828757 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f31100b4750 0x7f31100d0020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:30.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.681+0000 7f310f7fe700 1 -- 172.21.15.16:0/2346828757 >> v2:172.21.15.132:3300/0 conn(0x7f31100cfc50 msgr2=0x7f31100d2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:30.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.681+0000 7f310f7fe700 1 --2- 172.21.15.16:0/2346828757 >> v2:172.21.15.132:3300/0 conn(0x7f31100cfc50 0x7f31100d2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:30.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.681+0000 7f310f7fe700 1 -- 172.21.15.16:0/2346828757 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31100d0560 con 0x7f31100b2b40 2023-05-20T01:25:30.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.681+0000 7f310f7fe700 1 --2- 172.21.15.16:0/2346828757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31100b2b40 0x7f31100b2fc0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f31100b5de0 tx=0x7f310800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3839a9b9c2bfef5 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:30.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.682+0000 7f310dffb700 1 -- 172.21.15.16:0/2346828757 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f310800e070 con 0x7f31100b2b40 2023-05-20T01:25:30.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.682+0000 7f310dffb700 1 -- 172.21.15.16:0/2346828757 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f310800b910 con 0x7f31100b2b40 2023-05-20T01:25:30.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.682+0000 7f310dffb700 1 -- 172.21.15.16:0/2346828757 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f310800bcc0 con 0x7f31100b2b40 2023-05-20T01:25:30.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.682+0000 7f3115477700 1 -- 172.21.15.16:0/2346828757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31100b2b40 msgr2=0x7f31100b2fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:30.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.682+0000 7f3115477700 1 --2- 172.21.15.16:0/2346828757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31100b2b40 0x7f31100b2fc0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f31100b5de0 tx=0x7f310800b330 comp rx=0 tx=0).stop 2023-05-20T01:25:30.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.682+0000 7f3115477700 1 -- 172.21.15.16:0/2346828757 shutdown_connections 2023-05-20T01:25:30.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.682+0000 7f3115477700 1 --2- 172.21.15.16:0/2346828757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31100b2b40 0x7f31100b2fc0 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:30.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.682+0000 7f3115477700 1 --2- 172.21.15.16:0/2346828757 >> v2:172.21.15.132:3300/0 conn(0x7f31100cfc50 0x7f31100d2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:30.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.682+0000 7f3115477700 1 --2- 172.21.15.16:0/2346828757 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f31100b4750 0x7f31100d0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:30.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.682+0000 7f3115477700 1 -- 172.21.15.16:0/2346828757 >> 172.21.15.16:0/2346828757 conn(0x7f311009c590 msgr2=0x7f31100d3240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:30.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.682+0000 7f3115477700 1 -- 172.21.15.16:0/2346828757 shutdown_connections 2023-05-20T01:25:30.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.682+0000 7f3115477700 1 -- 172.21.15.16:0/2346828757 wait complete. 2023-05-20T01:25:30.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.684+0000 7f3115477700 1 Processor -- start 2023-05-20T01:25:30.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.684+0000 7f3115477700 1 -- start start 2023-05-20T01:25:30.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.685+0000 7f3115477700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31100b2b40 0x7f31100b6a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:30.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.685+0000 7f3115477700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f31100b4750 0x7f311015b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:30.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.685+0000 7f3115477700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f31100cfc50 0x7f311009d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:30.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.685+0000 7f3115477700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31100b7ee0 con 0x7f31100b2b40 2023-05-20T01:25:30.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.685+0000 7f3115477700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31100b72d0 con 0x7f31100cfc50 2023-05-20T01:25:30.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.685+0000 7f3115477700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31100b75b0 con 0x7f31100b4750 2023-05-20T01:25:30.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.685+0000 7f310effd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31100b2b40 0x7f31100b6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:30.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.685+0000 7f310effd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31100b2b40 0x7f31100b6a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36626/0 (socket says 172.21.15.16:36626) 2023-05-20T01:25:30.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.685+0000 7f310effd700 1 -- 172.21.15.16:0/3250218904 learned_addr learned my addr 172.21.15.16:0/3250218904 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:30.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.685+0000 7f310e7fc700 1 --2- 172.21.15.16:0/3250218904 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f31100b4750 0x7f311015b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:30.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.685+0000 7f310f7fe700 1 --2- 172.21.15.16:0/3250218904 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f31100cfc50 0x7f311009d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:30.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.685+0000 7f310effd700 1 -- 172.21.15.16:0/3250218904 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f31100b4750 msgr2=0x7f311015b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:30.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.685+0000 7f310effd700 1 --2- 172.21.15.16:0/3250218904 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f31100b4750 0x7f311015b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:30.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.685+0000 7f310effd700 1 -- 172.21.15.16:0/3250218904 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f31100cfc50 msgr2=0x7f311009d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:30.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.685+0000 7f310effd700 1 --2- 172.21.15.16:0/3250218904 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f31100cfc50 0x7f311009d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:30.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.685+0000 7f310effd700 1 -- 172.21.15.16:0/3250218904 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3108009b30 con 0x7f31100b2b40 2023-05-20T01:25:30.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.686+0000 7f310effd700 1 --2- 172.21.15.16:0/3250218904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31100b2b40 0x7f31100b6a80 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f310400e870 tx=0x7f310400ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:30.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.686+0000 7f30f77fe700 1 -- 172.21.15.16:0/3250218904 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3104009590 con 0x7f31100b2b40 2023-05-20T01:25:30.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.686+0000 7f30f77fe700 1 -- 172.21.15.16:0/3250218904 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f310400ee70 con 0x7f31100b2b40 2023-05-20T01:25:30.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.686+0000 7f3115477700 1 -- 172.21.15.16:0/3250218904 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31100b4380 con 0x7f31100b2b40 2023-05-20T01:25:30.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.686+0000 7f30f77fe700 1 -- 172.21.15.16:0/3250218904 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f310400ca50 con 0x7f31100b2b40 2023-05-20T01:25:30.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.686+0000 7f3115477700 1 -- 172.21.15.16:0/3250218904 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f311015d0b0 con 0x7f31100b2b40 2023-05-20T01:25:30.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.687+0000 7f3115477700 1 -- 172.21.15.16:0/3250218904 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31100d4000 con 0x7f31100b2b40 2023-05-20T01:25:30.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.687+0000 7f30f77fe700 1 -- 172.21.15.16:0/3250218904 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f310400cbb0 con 0x7f31100b2b40 2023-05-20T01:25:30.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.688+0000 7f30f77fe700 1 --2- 172.21.15.16:0/3250218904 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f30fc06ace0 0x7f30fc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:30.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.688+0000 7f30f77fe700 1 -- 172.21.15.16:0/3250218904 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6786+0+0 (secure 0 0 0) 0x7f31040896d0 con 0x7f31100b2b40 2023-05-20T01:25:30.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.688+0000 7f310e7fc700 1 --2- 172.21.15.16:0/3250218904 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f30fc06ace0 0x7f30fc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:30.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.688+0000 7f310e7fc700 1 --2- 172.21.15.16:0/3250218904 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f30fc06ace0 0x7f30fc06d190 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f31101f5890 tx=0x7f3100008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:30.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.692+0000 7f30f77fe700 1 -- 172.21.15.16:0/3250218904 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f3104057f20 con 0x7f31100b2b40 2023-05-20T01:25:30.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:30.938+0000 7f3115477700 1 -- 172.21.15.16:0/3250218904 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f311004e8c0 con 0x7f31100b2b40 2023-05-20T01:25:31.520 INFO:tasks.ceph.mon.a.smithi016.stderr:2023-05-20T01:25:31.516+0000 7f49eb8ae700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-05-20T01:25:31.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.523+0000 7f30f77fe700 1 -- 172.21.15.16:0/3250218904 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v59) v1 ==== 161+0+0 (secure 0 0 0) 0x7f310400ce60 con 0x7f31100b2b40 2023-05-20T01:25:31.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.526+0000 7f3115477700 1 -- 172.21.15.16:0/3250218904 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f30fc06ace0 msgr2=0x7f30fc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:31.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.526+0000 7f3115477700 1 --2- 172.21.15.16:0/3250218904 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f30fc06ace0 0x7f30fc06d190 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f31101f5890 tx=0x7f3100008040 comp rx=0 tx=0).stop 2023-05-20T01:25:31.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.526+0000 7f3115477700 1 -- 172.21.15.16:0/3250218904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31100b2b40 msgr2=0x7f31100b6a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:31.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.526+0000 7f3115477700 1 --2- 172.21.15.16:0/3250218904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31100b2b40 0x7f31100b6a80 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f310400e870 tx=0x7f310400ec30 comp rx=0 tx=0).stop 2023-05-20T01:25:31.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.527+0000 7f3115477700 1 -- 172.21.15.16:0/3250218904 shutdown_connections 2023-05-20T01:25:31.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.527+0000 7f3115477700 1 --2- 172.21.15.16:0/3250218904 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f31100cfc50 0x7f311009d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:31.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.527+0000 7f3115477700 1 --2- 172.21.15.16:0/3250218904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31100b2b40 0x7f31100b6a80 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:31.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.527+0000 7f3115477700 1 --2- 172.21.15.16:0/3250218904 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f30fc06ace0 0x7f30fc06d190 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:31.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.527+0000 7f3115477700 1 --2- 172.21.15.16:0/3250218904 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f31100b4750 0x7f311015b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:31.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.527+0000 7f3115477700 1 -- 172.21.15.16:0/3250218904 >> 172.21.15.16:0/3250218904 conn(0x7f311009c590 msgr2=0x7f31100b4cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:31.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.527+0000 7f3115477700 1 -- 172.21.15.16:0/3250218904 shutdown_connections 2023-05-20T01:25:31.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.527+0000 7f3115477700 1 -- 172.21.15.16:0/3250218904 wait complete. 2023-05-20T01:25:31.533 INFO:teuthology.orchestra.run.smithi016.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-05-20T01:25:31.533 INFO:tasks.ceph.mds.c.smithi016.stderr: -5426> 2023-05-20T01:25:17.549+0000 7f147f091700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-05-20T01:25:31.547 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-05-20T01:25:31.550 DEBUG:teuthology.orchestra.run.smithi132:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-05-20T01:25:31.567 INFO:tasks.ceph.mds.c.smithi016.stdout:starting mds.c at 2023-05-20T01:25:31.601 INFO:teuthology.orchestra.run.smithi016.stdout:[global] 2023-05-20T01:25:31.601 INFO:teuthology.orchestra.run.smithi016.stdout: chdir = "" 2023-05-20T01:25:31.601 INFO:teuthology.orchestra.run.smithi016.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-05-20T01:25:31.601 INFO:teuthology.orchestra.run.smithi016.stdout: auth supported = cephx 2023-05-20T01:25:31.602 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:25:31.602 INFO:teuthology.orchestra.run.smithi016.stdout: filestore xattr use omap = true 2023-05-20T01:25:31.602 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:25:31.602 INFO:teuthology.orchestra.run.smithi016.stdout: mon clock drift allowed = 1.000 2023-05-20T01:25:31.602 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:25:31.602 INFO:teuthology.orchestra.run.smithi016.stdout: osd crush chooseleaf type = 0 2023-05-20T01:25:31.602 INFO:teuthology.orchestra.run.smithi016.stdout: auth debug = true 2023-05-20T01:25:31.603 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:25:31.603 INFO:teuthology.orchestra.run.smithi016.stdout: ms die on old message = true 2023-05-20T01:25:31.603 INFO:teuthology.orchestra.run.smithi016.stdout: ms die on bug = true 2023-05-20T01:25:31.603 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:25:31.603 INFO:teuthology.orchestra.run.smithi016.stdout: mon max pg per osd = 10000 # >= luminous 2023-05-20T01:25:31.603 INFO:teuthology.orchestra.run.smithi016.stdout: mon pg warn max object skew = 0 2023-05-20T01:25:31.603 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:25:31.603 INFO:teuthology.orchestra.run.smithi016.stdout: # disable pg_autoscaler by default for new pools 2023-05-20T01:25:31.604 INFO:teuthology.orchestra.run.smithi016.stdout: osd_pool_default_pg_autoscale_mode = off 2023-05-20T01:25:31.604 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:25:31.604 INFO:teuthology.orchestra.run.smithi016.stdout: osd pool default size = 2 2023-05-20T01:25:31.604 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:25:31.604 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd allow primary affinity = true 2023-05-20T01:25:31.604 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd allow pg remap = true 2023-05-20T01:25:31.604 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on legacy crush tunables = false 2023-05-20T01:25:31.605 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on crush straw calc version zero = false 2023-05-20T01:25:31.605 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on no sortbitwise = false 2023-05-20T01:25:31.605 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on osd down out interval zero = false 2023-05-20T01:25:31.605 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on too few osds = false 2023-05-20T01:25:31.605 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-05-20T01:25:31.605 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_pool_no_redundancy = false 2023-05-20T01:25:31.605 INFO:teuthology.orchestra.run.smithi016.stdout: mon_allow_pool_size_one = true 2023-05-20T01:25:31.606 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:25:31.606 INFO:teuthology.orchestra.run.smithi016.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-05-20T01:25:31.606 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:25:31.606 INFO:teuthology.orchestra.run.smithi016.stdout: osd default data pool replay window = 5 2023-05-20T01:25:31.606 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:25:31.606 INFO:teuthology.orchestra.run.smithi016.stdout: mon allow pool delete = true 2023-05-20T01:25:31.606 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:25:31.606 INFO:teuthology.orchestra.run.smithi016.stdout: mon cluster log file level = debug 2023-05-20T01:25:31.607 INFO:teuthology.orchestra.run.smithi016.stdout: debug asserts on shutdown = true 2023-05-20T01:25:31.607 INFO:teuthology.orchestra.run.smithi016.stdout: mon health detail to clog = false 2023-05-20T01:25:31.607 INFO:teuthology.orchestra.run.smithi016.stdout: mon host = "172.21.15.16,172.21.15.132,[v2:172.21.15.132:3301,v1:172.21.15.132:6790]" 2023-05-20T01:25:31.607 INFO:teuthology.orchestra.run.smithi016.stdout: ms die on skipped message = False 2023-05-20T01:25:31.607 INFO:teuthology.orchestra.run.smithi016.stdout: fsid = c2245cc5-ecab-4087-a59b-bc81db901676 2023-05-20T01:25:31.607 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:25:31.607 INFO:teuthology.orchestra.run.smithi016.stdout:[osd] 2023-05-20T01:25:31.608 INFO:teuthology.orchestra.run.smithi016.stdout: osd journal size = 100 2023-05-20T01:25:31.608 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:25:31.608 INFO:teuthology.orchestra.run.smithi016.stdout: osd scrub load threshold = 5.0 2023-05-20T01:25:31.608 INFO:teuthology.orchestra.run.smithi016.stdout: osd scrub max interval = 600 2023-05-20T01:25:31.608 INFO:teuthology.orchestra.run.smithi016.stdout: osd mclock profile = high_recovery_ops 2023-05-20T01:25:31.608 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:25:31.608 INFO:teuthology.orchestra.run.smithi016.stdout: osd recover clone overlap = true 2023-05-20T01:25:31.609 INFO:teuthology.orchestra.run.smithi016.stdout: osd recovery max chunk = 1048576 2023-05-20T01:25:31.609 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:25:31.609 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug shutdown = true 2023-05-20T01:25:31.609 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug op order = true 2023-05-20T01:25:31.609 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug verify stray on activate = true 2023-05-20T01:25:31.609 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:25:31.609 INFO:teuthology.orchestra.run.smithi016.stdout: osd open classes on start = true 2023-05-20T01:25:31.610 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug pg log writeout = true 2023-05-20T01:25:31.610 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:25:31.610 INFO:teuthology.orchestra.run.smithi016.stdout: osd deep scrub update digest min age = 30 2023-05-20T01:25:31.610 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:25:31.610 INFO:teuthology.orchestra.run.smithi016.stdout: osd map max advance = 10 2023-05-20T01:25:31.610 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:25:31.610 INFO:teuthology.orchestra.run.smithi016.stdout: journal zero on create = true 2023-05-20T01:25:31.610 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:25:31.611 INFO:teuthology.orchestra.run.smithi016.stdout: filestore ondisk finisher threads = 3 2023-05-20T01:25:31.611 INFO:teuthology.orchestra.run.smithi016.stdout: filestore apply finisher threads = 3 2023-05-20T01:25:31.611 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:25:31.611 INFO:teuthology.orchestra.run.smithi016.stdout: bdev debug aio = true 2023-05-20T01:25:31.611 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug misdirected ops = true 2023-05-20T01:25:31.611 INFO:teuthology.orchestra.run.smithi016.stdout: bluestore block size = 96636764160 2023-05-20T01:25:31.611 INFO:teuthology.orchestra.run.smithi016.stdout: bluestore fsck on mount = True 2023-05-20T01:25:31.612 INFO:teuthology.orchestra.run.smithi016.stdout: debug bluefs = 20 2023-05-20T01:25:31.612 INFO:teuthology.orchestra.run.smithi016.stdout: debug bluestore = 20 2023-05-20T01:25:31.612 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2023-05-20T01:25:31.612 INFO:teuthology.orchestra.run.smithi016.stdout: debug osd = 20 2023-05-20T01:25:31.612 INFO:teuthology.orchestra.run.smithi016.stdout: debug rocksdb = 10 2023-05-20T01:25:31.612 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd backfillfull_ratio = 0.85 2023-05-20T01:25:31.612 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd full ratio = 0.9 2023-05-20T01:25:31.612 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd nearfull ratio = 0.8 2023-05-20T01:25:31.613 INFO:teuthology.orchestra.run.smithi016.stdout: osd failsafe full ratio = 0.95 2023-05-20T01:25:31.613 INFO:teuthology.orchestra.run.smithi016.stdout: osd objectstore = bluestore 2023-05-20T01:25:31.613 INFO:teuthology.orchestra.run.smithi016.stdout: osd op complaint time = 180 2023-05-20T01:25:31.613 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:25:31.613 INFO:teuthology.orchestra.run.smithi016.stdout:[mgr] 2023-05-20T01:25:31.613 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2023-05-20T01:25:31.613 INFO:teuthology.orchestra.run.smithi016.stdout: debug mgr = 20 2023-05-20T01:25:31.613 INFO:teuthology.orchestra.run.smithi016.stdout: debug mon = 20 2023-05-20T01:25:31.614 INFO:teuthology.orchestra.run.smithi016.stdout: debug auth = 20 2023-05-20T01:25:31.614 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min pgs per osd = 4 2023-05-20T01:25:31.614 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min bytes per osd = 10 2023-05-20T01:25:31.614 INFO:teuthology.orchestra.run.smithi016.stdout: mgr/telemetry/nag = false 2023-05-20T01:25:31.614 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:25:31.614 INFO:teuthology.orchestra.run.smithi016.stdout:[mon] 2023-05-20T01:25:31.614 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2023-05-20T01:25:31.614 INFO:teuthology.orchestra.run.smithi016.stdout: debug mon = 20 2023-05-20T01:25:31.615 INFO:teuthology.orchestra.run.smithi016.stdout: debug paxos = 20 2023-05-20T01:25:31.615 INFO:teuthology.orchestra.run.smithi016.stdout: debug auth = 20 2023-05-20T01:25:31.615 INFO:teuthology.orchestra.run.smithi016.stdout: mon data avail warn = 5 2023-05-20T01:25:31.615 INFO:teuthology.orchestra.run.smithi016.stdout: mon mgr mkfs grace = 240 2023-05-20T01:25:31.615 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min pgs per osd = 4 2023-05-20T01:25:31.615 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd reporter subtree level = osd 2023-05-20T01:25:31.615 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd prime pg temp = true 2023-05-20T01:25:31.616 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min bytes per osd = 10 2023-05-20T01:25:31.616 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:25:31.616 INFO:teuthology.orchestra.run.smithi016.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-05-20T01:25:31.616 INFO:teuthology.orchestra.run.smithi016.stdout: auth mon ticket ttl = 660 # 11m 2023-05-20T01:25:31.616 INFO:teuthology.orchestra.run.smithi016.stdout: auth service ticket ttl = 240 # 4m 2023-05-20T01:25:31.616 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:25:31.616 INFO:teuthology.orchestra.run.smithi016.stdout: # don't complain about insecure global_id in the test suite 2023-05-20T01:25:31.616 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-05-20T01:25:31.617 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-05-20T01:25:31.617 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:25:31.617 INFO:teuthology.orchestra.run.smithi016.stdout: # 1m isn't quite enough 2023-05-20T01:25:31.617 INFO:teuthology.orchestra.run.smithi016.stdout: mon_down_mkfs_grace = 2m 2023-05-20T01:25:31.617 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:25:31.617 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_filestore_osds = false 2023-05-20T01:25:31.617 INFO:teuthology.orchestra.run.smithi016.stdout: mon op complaint time = 120 2023-05-20T01:25:31.617 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:25:31.618 INFO:teuthology.orchestra.run.smithi016.stdout:[client] 2023-05-20T01:25:31.618 INFO:teuthology.orchestra.run.smithi016.stdout: rgw cache enabled = true 2023-05-20T01:25:31.618 INFO:teuthology.orchestra.run.smithi016.stdout: rgw enable ops log = true 2023-05-20T01:25:31.618 INFO:teuthology.orchestra.run.smithi016.stdout: rgw enable usage log = true 2023-05-20T01:25:31.618 INFO:teuthology.orchestra.run.smithi016.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-05-20T01:25:31.618 INFO:teuthology.orchestra.run.smithi016.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-05-20T01:25:31.618 INFO:teuthology.orchestra.run.smithi016.stdout: client mount timeout = 600 2023-05-20T01:25:31.619 INFO:teuthology.orchestra.run.smithi016.stdout: debug client = 20 2023-05-20T01:25:31.619 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2023-05-20T01:25:31.619 INFO:teuthology.orchestra.run.smithi016.stdout: pid file = "" 2023-05-20T01:25:31.619 INFO:teuthology.orchestra.run.smithi016.stdout: rados mon op timeout = 900 2023-05-20T01:25:31.619 INFO:teuthology.orchestra.run.smithi016.stdout: rados osd op timeout = 900 2023-05-20T01:25:31.619 INFO:teuthology.orchestra.run.smithi016.stdout:[mon.a] 2023-05-20T01:25:31.619 INFO:teuthology.orchestra.run.smithi016.stdout:[mon.b] 2023-05-20T01:25:31.619 INFO:teuthology.orchestra.run.smithi016.stdout:[mon.c] 2023-05-20T01:25:31.620 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.a] 2023-05-20T01:25:31.620 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.c] 2023-05-20T01:25:31.620 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.b] 2023-05-20T01:25:31.620 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.d] 2023-05-20T01:25:31.620 INFO:teuthology.orchestra.run.smithi016.stdout:[mds] 2023-05-20T01:25:31.620 INFO:teuthology.orchestra.run.smithi016.stdout: debug mds = 20 2023-05-20T01:25:31.620 INFO:teuthology.orchestra.run.smithi016.stdout: debug mds balancer = 20 2023-05-20T01:25:31.620 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2023-05-20T01:25:31.621 INFO:teuthology.orchestra.run.smithi016.stdout: mds debug frag = True 2023-05-20T01:25:31.621 INFO:teuthology.orchestra.run.smithi016.stdout: mds debug scatterstat = False 2023-05-20T01:25:31.621 INFO:teuthology.orchestra.run.smithi016.stdout: mds op complaint time = 180 2023-05-20T01:25:31.621 INFO:teuthology.orchestra.run.smithi016.stdout: mds verify scatter = False 2023-05-20T01:25:31.621 INFO:teuthology.orchestra.run.smithi016.stdout: osd op complaint time = 180 2023-05-20T01:25:31.621 INFO:teuthology.orchestra.run.smithi016.stdout: rados mon op timeout = 900 2023-05-20T01:25:31.621 INFO:teuthology.orchestra.run.smithi016.stdout: rados osd op timeout = 900 2023-05-20T01:25:31.623 INFO:teuthology.orchestra.run.smithi132.stdout:[global] 2023-05-20T01:25:31.623 INFO:teuthology.orchestra.run.smithi132.stdout: chdir = "" 2023-05-20T01:25:31.623 INFO:teuthology.orchestra.run.smithi132.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-05-20T01:25:31.623 INFO:teuthology.orchestra.run.smithi132.stdout: auth supported = cephx 2023-05-20T01:25:31.623 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:25:31.623 INFO:teuthology.orchestra.run.smithi132.stdout: filestore xattr use omap = true 2023-05-20T01:25:31.624 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:25:31.624 INFO:teuthology.orchestra.run.smithi132.stdout: mon clock drift allowed = 1.000 2023-05-20T01:25:31.624 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:25:31.624 INFO:teuthology.orchestra.run.smithi132.stdout: osd crush chooseleaf type = 0 2023-05-20T01:25:31.624 INFO:teuthology.orchestra.run.smithi132.stdout: auth debug = true 2023-05-20T01:25:31.624 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:25:31.624 INFO:teuthology.orchestra.run.smithi132.stdout: ms die on old message = true 2023-05-20T01:25:31.625 INFO:teuthology.orchestra.run.smithi132.stdout: ms die on bug = true 2023-05-20T01:25:31.625 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:25:31.625 INFO:teuthology.orchestra.run.smithi132.stdout: mon max pg per osd = 10000 # >= luminous 2023-05-20T01:25:31.625 INFO:teuthology.orchestra.run.smithi132.stdout: mon pg warn max object skew = 0 2023-05-20T01:25:31.625 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:25:31.625 INFO:teuthology.orchestra.run.smithi132.stdout: # disable pg_autoscaler by default for new pools 2023-05-20T01:25:31.625 INFO:teuthology.orchestra.run.smithi132.stdout: osd_pool_default_pg_autoscale_mode = off 2023-05-20T01:25:31.625 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:25:31.626 INFO:teuthology.orchestra.run.smithi132.stdout: osd pool default size = 2 2023-05-20T01:25:31.626 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:25:31.626 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd allow primary affinity = true 2023-05-20T01:25:31.626 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd allow pg remap = true 2023-05-20T01:25:31.626 INFO:teuthology.orchestra.run.smithi132.stdout: mon warn on legacy crush tunables = false 2023-05-20T01:25:31.626 INFO:teuthology.orchestra.run.smithi132.stdout: mon warn on crush straw calc version zero = false 2023-05-20T01:25:31.626 INFO:teuthology.orchestra.run.smithi132.stdout: mon warn on no sortbitwise = false 2023-05-20T01:25:31.626 INFO:teuthology.orchestra.run.smithi132.stdout: mon warn on osd down out interval zero = false 2023-05-20T01:25:31.627 INFO:teuthology.orchestra.run.smithi132.stdout: mon warn on too few osds = false 2023-05-20T01:25:31.627 INFO:teuthology.orchestra.run.smithi132.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-05-20T01:25:31.627 INFO:teuthology.orchestra.run.smithi132.stdout: mon_warn_on_pool_no_redundancy = false 2023-05-20T01:25:31.627 INFO:teuthology.orchestra.run.smithi132.stdout: mon_allow_pool_size_one = true 2023-05-20T01:25:31.627 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:25:31.627 INFO:teuthology.orchestra.run.smithi132.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-05-20T01:25:31.627 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:25:31.628 INFO:teuthology.orchestra.run.smithi132.stdout: osd default data pool replay window = 5 2023-05-20T01:25:31.628 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:25:31.628 INFO:teuthology.orchestra.run.smithi132.stdout: mon allow pool delete = true 2023-05-20T01:25:31.628 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:25:31.628 INFO:teuthology.orchestra.run.smithi132.stdout: mon cluster log file level = debug 2023-05-20T01:25:31.628 INFO:teuthology.orchestra.run.smithi132.stdout: debug asserts on shutdown = true 2023-05-20T01:25:31.628 INFO:teuthology.orchestra.run.smithi132.stdout: mon health detail to clog = false 2023-05-20T01:25:31.628 INFO:teuthology.orchestra.run.smithi132.stdout: mon host = "172.21.15.16,172.21.15.132,[v2:172.21.15.132:3301,v1:172.21.15.132:6790]" 2023-05-20T01:25:31.629 INFO:teuthology.orchestra.run.smithi132.stdout: ms die on skipped message = False 2023-05-20T01:25:31.629 INFO:teuthology.orchestra.run.smithi132.stdout: fsid = c2245cc5-ecab-4087-a59b-bc81db901676 2023-05-20T01:25:31.629 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:25:31.629 INFO:teuthology.orchestra.run.smithi132.stdout:[osd] 2023-05-20T01:25:31.629 INFO:teuthology.orchestra.run.smithi132.stdout: osd journal size = 100 2023-05-20T01:25:31.629 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:25:31.629 INFO:teuthology.orchestra.run.smithi132.stdout: osd scrub load threshold = 5.0 2023-05-20T01:25:31.629 INFO:teuthology.orchestra.run.smithi132.stdout: osd scrub max interval = 600 2023-05-20T01:25:31.630 INFO:teuthology.orchestra.run.smithi132.stdout: osd mclock profile = high_recovery_ops 2023-05-20T01:25:31.630 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:25:31.630 INFO:teuthology.orchestra.run.smithi132.stdout: osd recover clone overlap = true 2023-05-20T01:25:31.630 INFO:teuthology.orchestra.run.smithi132.stdout: osd recovery max chunk = 1048576 2023-05-20T01:25:31.630 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:25:31.630 INFO:teuthology.orchestra.run.smithi132.stdout: osd debug shutdown = true 2023-05-20T01:25:31.630 INFO:teuthology.orchestra.run.smithi132.stdout: osd debug op order = true 2023-05-20T01:25:31.631 INFO:teuthology.orchestra.run.smithi132.stdout: osd debug verify stray on activate = true 2023-05-20T01:25:31.631 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:25:31.631 INFO:teuthology.orchestra.run.smithi132.stdout: osd open classes on start = true 2023-05-20T01:25:31.631 INFO:teuthology.orchestra.run.smithi132.stdout: osd debug pg log writeout = true 2023-05-20T01:25:31.631 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:25:31.631 INFO:teuthology.orchestra.run.smithi132.stdout: osd deep scrub update digest min age = 30 2023-05-20T01:25:31.631 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:25:31.631 INFO:teuthology.orchestra.run.smithi132.stdout: osd map max advance = 10 2023-05-20T01:25:31.632 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:25:31.632 INFO:teuthology.orchestra.run.smithi132.stdout: journal zero on create = true 2023-05-20T01:25:31.632 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:25:31.632 INFO:teuthology.orchestra.run.smithi132.stdout: filestore ondisk finisher threads = 3 2023-05-20T01:25:31.632 INFO:teuthology.orchestra.run.smithi132.stdout: filestore apply finisher threads = 3 2023-05-20T01:25:31.632 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:25:31.632 INFO:teuthology.orchestra.run.smithi132.stdout: bdev debug aio = true 2023-05-20T01:25:31.632 INFO:teuthology.orchestra.run.smithi132.stdout: osd debug misdirected ops = true 2023-05-20T01:25:31.633 INFO:teuthology.orchestra.run.smithi132.stdout: bluestore block size = 96636764160 2023-05-20T01:25:31.633 INFO:teuthology.orchestra.run.smithi132.stdout: bluestore fsck on mount = True 2023-05-20T01:25:31.633 INFO:teuthology.orchestra.run.smithi132.stdout: debug bluefs = 20 2023-05-20T01:25:31.633 INFO:teuthology.orchestra.run.smithi132.stdout: debug bluestore = 20 2023-05-20T01:25:31.633 INFO:teuthology.orchestra.run.smithi132.stdout: debug ms = 1 2023-05-20T01:25:31.633 INFO:teuthology.orchestra.run.smithi132.stdout: debug osd = 20 2023-05-20T01:25:31.633 INFO:teuthology.orchestra.run.smithi132.stdout: debug rocksdb = 10 2023-05-20T01:25:31.634 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd backfillfull_ratio = 0.85 2023-05-20T01:25:31.634 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd full ratio = 0.9 2023-05-20T01:25:31.634 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd nearfull ratio = 0.8 2023-05-20T01:25:31.634 INFO:teuthology.orchestra.run.smithi132.stdout: osd failsafe full ratio = 0.95 2023-05-20T01:25:31.634 INFO:teuthology.orchestra.run.smithi132.stdout: osd objectstore = bluestore 2023-05-20T01:25:31.634 INFO:teuthology.orchestra.run.smithi132.stdout: osd op complaint time = 180 2023-05-20T01:25:31.634 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:25:31.634 INFO:teuthology.orchestra.run.smithi132.stdout:[mgr] 2023-05-20T01:25:31.635 INFO:teuthology.orchestra.run.smithi132.stdout: debug ms = 1 2023-05-20T01:25:31.635 INFO:teuthology.orchestra.run.smithi132.stdout: debug mgr = 20 2023-05-20T01:25:31.635 INFO:teuthology.orchestra.run.smithi132.stdout: debug mon = 20 2023-05-20T01:25:31.635 INFO:teuthology.orchestra.run.smithi132.stdout: debug auth = 20 2023-05-20T01:25:31.635 INFO:teuthology.orchestra.run.smithi132.stdout: mon reweight min pgs per osd = 4 2023-05-20T01:25:31.635 INFO:teuthology.orchestra.run.smithi132.stdout: mon reweight min bytes per osd = 10 2023-05-20T01:25:31.635 INFO:teuthology.orchestra.run.smithi132.stdout: mgr/telemetry/nag = false 2023-05-20T01:25:31.636 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:25:31.636 INFO:teuthology.orchestra.run.smithi132.stdout:[mon] 2023-05-20T01:25:31.636 INFO:teuthology.orchestra.run.smithi132.stdout: debug ms = 1 2023-05-20T01:25:31.636 INFO:teuthology.orchestra.run.smithi132.stdout: debug mon = 20 2023-05-20T01:25:31.636 INFO:teuthology.orchestra.run.smithi132.stdout: debug paxos = 20 2023-05-20T01:25:31.636 INFO:teuthology.orchestra.run.smithi132.stdout: debug auth = 20 2023-05-20T01:25:31.636 INFO:teuthology.orchestra.run.smithi132.stdout: mon data avail warn = 5 2023-05-20T01:25:31.636 INFO:teuthology.orchestra.run.smithi132.stdout: mon mgr mkfs grace = 240 2023-05-20T01:25:31.637 INFO:teuthology.orchestra.run.smithi132.stdout: mon reweight min pgs per osd = 4 2023-05-20T01:25:31.637 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd reporter subtree level = osd 2023-05-20T01:25:31.637 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd prime pg temp = true 2023-05-20T01:25:31.637 INFO:teuthology.orchestra.run.smithi132.stdout: mon reweight min bytes per osd = 10 2023-05-20T01:25:31.637 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:25:31.637 INFO:teuthology.orchestra.run.smithi132.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-05-20T01:25:31.637 INFO:teuthology.orchestra.run.smithi132.stdout: auth mon ticket ttl = 660 # 11m 2023-05-20T01:25:31.638 INFO:teuthology.orchestra.run.smithi132.stdout: auth service ticket ttl = 240 # 4m 2023-05-20T01:25:31.638 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:25:31.638 INFO:teuthology.orchestra.run.smithi132.stdout: # don't complain about insecure global_id in the test suite 2023-05-20T01:25:31.638 INFO:teuthology.orchestra.run.smithi132.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-05-20T01:25:31.638 INFO:teuthology.orchestra.run.smithi132.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-05-20T01:25:31.638 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:25:31.638 INFO:teuthology.orchestra.run.smithi132.stdout: # 1m isn't quite enough 2023-05-20T01:25:31.638 INFO:teuthology.orchestra.run.smithi132.stdout: mon_down_mkfs_grace = 2m 2023-05-20T01:25:31.639 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:25:31.639 INFO:teuthology.orchestra.run.smithi132.stdout: mon_warn_on_filestore_osds = false 2023-05-20T01:25:31.639 INFO:teuthology.orchestra.run.smithi132.stdout: mon op complaint time = 120 2023-05-20T01:25:31.639 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:25:31.639 INFO:teuthology.orchestra.run.smithi132.stdout:[client] 2023-05-20T01:25:31.639 INFO:teuthology.orchestra.run.smithi132.stdout: rgw cache enabled = true 2023-05-20T01:25:31.639 INFO:teuthology.orchestra.run.smithi132.stdout: rgw enable ops log = true 2023-05-20T01:25:31.639 INFO:teuthology.orchestra.run.smithi132.stdout: rgw enable usage log = true 2023-05-20T01:25:31.640 INFO:teuthology.orchestra.run.smithi132.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-05-20T01:25:31.640 INFO:teuthology.orchestra.run.smithi132.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-05-20T01:25:31.640 INFO:teuthology.orchestra.run.smithi132.stdout: client mount timeout = 600 2023-05-20T01:25:31.640 INFO:teuthology.orchestra.run.smithi132.stdout: debug client = 20 2023-05-20T01:25:31.640 INFO:teuthology.orchestra.run.smithi132.stdout: debug ms = 1 2023-05-20T01:25:31.640 INFO:teuthology.orchestra.run.smithi132.stdout: pid file = "" 2023-05-20T01:25:31.640 INFO:teuthology.orchestra.run.smithi132.stdout: rados mon op timeout = 900 2023-05-20T01:25:31.640 INFO:teuthology.orchestra.run.smithi132.stdout: rados osd op timeout = 900 2023-05-20T01:25:31.641 INFO:teuthology.orchestra.run.smithi132.stdout:[mon.a] 2023-05-20T01:25:31.641 INFO:teuthology.orchestra.run.smithi132.stdout:[mon.b] 2023-05-20T01:25:31.641 INFO:teuthology.orchestra.run.smithi132.stdout:[mon.c] 2023-05-20T01:25:31.641 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.a] 2023-05-20T01:25:31.641 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.c] 2023-05-20T01:25:31.641 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.b] 2023-05-20T01:25:31.641 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.d] 2023-05-20T01:25:31.641 INFO:teuthology.orchestra.run.smithi132.stdout:[mds] 2023-05-20T01:25:31.642 INFO:teuthology.orchestra.run.smithi132.stdout: debug mds = 20 2023-05-20T01:25:31.642 INFO:teuthology.orchestra.run.smithi132.stdout: debug mds balancer = 20 2023-05-20T01:25:31.642 INFO:teuthology.orchestra.run.smithi132.stdout: debug ms = 1 2023-05-20T01:25:31.642 INFO:teuthology.orchestra.run.smithi132.stdout: mds debug frag = True 2023-05-20T01:25:31.642 INFO:teuthology.orchestra.run.smithi132.stdout: mds debug scatterstat = False 2023-05-20T01:25:31.642 INFO:teuthology.orchestra.run.smithi132.stdout: mds op complaint time = 180 2023-05-20T01:25:31.642 INFO:teuthology.orchestra.run.smithi132.stdout: mds verify scatter = False 2023-05-20T01:25:31.643 INFO:teuthology.orchestra.run.smithi132.stdout: osd op complaint time = 180 2023-05-20T01:25:31.643 INFO:teuthology.orchestra.run.smithi132.stdout: rados mon op timeout = 900 2023-05-20T01:25:31.643 INFO:teuthology.orchestra.run.smithi132.stdout: rados osd op timeout = 900 2023-05-20T01:25:31.644 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-05-20T01:25:31.665 DEBUG:teuthology.orchestra.run.smithi132:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-05-20T01:25:31.711 INFO:teuthology.orchestra.run.smithi016.stdout:[global] 2023-05-20T01:25:31.711 INFO:teuthology.orchestra.run.smithi016.stdout: chdir = "" 2023-05-20T01:25:31.711 INFO:teuthology.orchestra.run.smithi016.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-05-20T01:25:31.711 INFO:teuthology.orchestra.run.smithi016.stdout: auth supported = cephx 2023-05-20T01:25:31.711 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:25:31.712 INFO:teuthology.orchestra.run.smithi016.stdout: filestore xattr use omap = true 2023-05-20T01:25:31.712 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:25:31.712 INFO:teuthology.orchestra.run.smithi016.stdout: mon clock drift allowed = 1.000 2023-05-20T01:25:31.712 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:25:31.712 INFO:teuthology.orchestra.run.smithi016.stdout: osd crush chooseleaf type = 0 2023-05-20T01:25:31.712 INFO:teuthology.orchestra.run.smithi016.stdout: auth debug = true 2023-05-20T01:25:31.712 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:25:31.713 INFO:teuthology.orchestra.run.smithi016.stdout: ms die on old message = true 2023-05-20T01:25:31.713 INFO:teuthology.orchestra.run.smithi016.stdout: ms die on bug = true 2023-05-20T01:25:31.713 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:25:31.713 INFO:teuthology.orchestra.run.smithi016.stdout: mon max pg per osd = 10000 # >= luminous 2023-05-20T01:25:31.713 INFO:teuthology.orchestra.run.smithi016.stdout: mon pg warn max object skew = 0 2023-05-20T01:25:31.714 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:25:31.714 INFO:teuthology.orchestra.run.smithi016.stdout: # disable pg_autoscaler by default for new pools 2023-05-20T01:25:31.714 INFO:teuthology.orchestra.run.smithi016.stdout: osd_pool_default_pg_autoscale_mode = off 2023-05-20T01:25:31.714 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:25:31.714 INFO:teuthology.orchestra.run.smithi016.stdout: osd pool default size = 2 2023-05-20T01:25:31.714 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:25:31.715 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd allow primary affinity = true 2023-05-20T01:25:31.715 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd allow pg remap = true 2023-05-20T01:25:31.715 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on legacy crush tunables = false 2023-05-20T01:25:31.715 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on crush straw calc version zero = false 2023-05-20T01:25:31.715 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on no sortbitwise = false 2023-05-20T01:25:31.715 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on osd down out interval zero = false 2023-05-20T01:25:31.715 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on too few osds = false 2023-05-20T01:25:31.716 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-05-20T01:25:31.716 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_pool_no_redundancy = false 2023-05-20T01:25:31.716 INFO:teuthology.orchestra.run.smithi016.stdout: mon_allow_pool_size_one = true 2023-05-20T01:25:31.716 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:25:31.716 INFO:teuthology.orchestra.run.smithi016.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-05-20T01:25:31.716 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:25:31.716 INFO:teuthology.orchestra.run.smithi016.stdout: osd default data pool replay window = 5 2023-05-20T01:25:31.717 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:25:31.717 INFO:teuthology.orchestra.run.smithi016.stdout: mon allow pool delete = true 2023-05-20T01:25:31.717 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:25:31.717 INFO:teuthology.orchestra.run.smithi016.stdout: mon cluster log file level = debug 2023-05-20T01:25:31.717 INFO:teuthology.orchestra.run.smithi016.stdout: debug asserts on shutdown = true 2023-05-20T01:25:31.717 INFO:teuthology.orchestra.run.smithi016.stdout: mon health detail to clog = false 2023-05-20T01:25:31.717 INFO:teuthology.orchestra.run.smithi016.stdout: mon host = "172.21.15.16,172.21.15.132,[v2:172.21.15.132:3301,v1:172.21.15.132:6790]" 2023-05-20T01:25:31.717 INFO:teuthology.orchestra.run.smithi016.stdout: ms die on skipped message = False 2023-05-20T01:25:31.718 INFO:teuthology.orchestra.run.smithi016.stdout: fsid = c2245cc5-ecab-4087-a59b-bc81db901676 2023-05-20T01:25:31.718 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:25:31.718 INFO:teuthology.orchestra.run.smithi016.stdout:[osd] 2023-05-20T01:25:31.718 INFO:teuthology.orchestra.run.smithi016.stdout: osd journal size = 100 2023-05-20T01:25:31.718 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:25:31.718 INFO:teuthology.orchestra.run.smithi016.stdout: osd scrub load threshold = 5.0 2023-05-20T01:25:31.718 INFO:teuthology.orchestra.run.smithi016.stdout: osd scrub max interval = 600 2023-05-20T01:25:31.719 INFO:teuthology.orchestra.run.smithi016.stdout: osd mclock profile = high_recovery_ops 2023-05-20T01:25:31.719 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:25:31.719 INFO:teuthology.orchestra.run.smithi016.stdout: osd recover clone overlap = true 2023-05-20T01:25:31.719 INFO:teuthology.orchestra.run.smithi016.stdout: osd recovery max chunk = 1048576 2023-05-20T01:25:31.719 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:25:31.719 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug shutdown = true 2023-05-20T01:25:31.719 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug op order = true 2023-05-20T01:25:31.719 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug verify stray on activate = true 2023-05-20T01:25:31.720 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:25:31.720 INFO:teuthology.orchestra.run.smithi016.stdout: osd open classes on start = true 2023-05-20T01:25:31.720 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug pg log writeout = true 2023-05-20T01:25:31.720 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:25:31.720 INFO:teuthology.orchestra.run.smithi016.stdout: osd deep scrub update digest min age = 30 2023-05-20T01:25:31.721 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:25:31.721 INFO:teuthology.orchestra.run.smithi016.stdout: osd map max advance = 10 2023-05-20T01:25:31.721 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:25:31.721 INFO:teuthology.orchestra.run.smithi016.stdout: journal zero on create = true 2023-05-20T01:25:31.721 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:25:31.721 INFO:teuthology.orchestra.run.smithi016.stdout: filestore ondisk finisher threads = 3 2023-05-20T01:25:31.721 INFO:teuthology.orchestra.run.smithi016.stdout: filestore apply finisher threads = 3 2023-05-20T01:25:31.721 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:25:31.722 INFO:teuthology.orchestra.run.smithi016.stdout: bdev debug aio = true 2023-05-20T01:25:31.722 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug misdirected ops = true 2023-05-20T01:25:31.722 INFO:teuthology.orchestra.run.smithi016.stdout: bluestore block size = 96636764160 2023-05-20T01:25:31.722 INFO:teuthology.orchestra.run.smithi016.stdout: bluestore fsck on mount = True 2023-05-20T01:25:31.722 INFO:teuthology.orchestra.run.smithi016.stdout: debug bluefs = 20 2023-05-20T01:25:31.722 INFO:teuthology.orchestra.run.smithi016.stdout: debug bluestore = 20 2023-05-20T01:25:31.722 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2023-05-20T01:25:31.722 INFO:teuthology.orchestra.run.smithi016.stdout: debug osd = 20 2023-05-20T01:25:31.723 INFO:teuthology.orchestra.run.smithi016.stdout: debug rocksdb = 10 2023-05-20T01:25:31.723 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd backfillfull_ratio = 0.85 2023-05-20T01:25:31.723 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd full ratio = 0.9 2023-05-20T01:25:31.723 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd nearfull ratio = 0.8 2023-05-20T01:25:31.723 INFO:teuthology.orchestra.run.smithi016.stdout: osd failsafe full ratio = 0.95 2023-05-20T01:25:31.723 INFO:teuthology.orchestra.run.smithi016.stdout: osd objectstore = bluestore 2023-05-20T01:25:31.723 INFO:teuthology.orchestra.run.smithi016.stdout: osd op complaint time = 180 2023-05-20T01:25:31.723 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:25:31.724 INFO:teuthology.orchestra.run.smithi016.stdout:[mgr] 2023-05-20T01:25:31.724 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2023-05-20T01:25:31.724 INFO:teuthology.orchestra.run.smithi016.stdout: debug mgr = 20 2023-05-20T01:25:31.724 INFO:teuthology.orchestra.run.smithi016.stdout: debug mon = 20 2023-05-20T01:25:31.724 INFO:teuthology.orchestra.run.smithi016.stdout: debug auth = 20 2023-05-20T01:25:31.724 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min pgs per osd = 4 2023-05-20T01:25:31.724 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min bytes per osd = 10 2023-05-20T01:25:31.725 INFO:teuthology.orchestra.run.smithi016.stdout: mgr/telemetry/nag = false 2023-05-20T01:25:31.725 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:25:31.725 INFO:teuthology.orchestra.run.smithi016.stdout:[mon] 2023-05-20T01:25:31.725 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2023-05-20T01:25:31.725 INFO:teuthology.orchestra.run.smithi016.stdout: debug mon = 20 2023-05-20T01:25:31.725 INFO:teuthology.orchestra.run.smithi016.stdout: debug paxos = 20 2023-05-20T01:25:31.725 INFO:teuthology.orchestra.run.smithi016.stdout: debug auth = 20 2023-05-20T01:25:31.725 INFO:teuthology.orchestra.run.smithi016.stdout: mon data avail warn = 5 2023-05-20T01:25:31.726 INFO:teuthology.orchestra.run.smithi016.stdout: mon mgr mkfs grace = 240 2023-05-20T01:25:31.726 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min pgs per osd = 4 2023-05-20T01:25:31.726 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd reporter subtree level = osd 2023-05-20T01:25:31.726 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd prime pg temp = true 2023-05-20T01:25:31.726 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min bytes per osd = 10 2023-05-20T01:25:31.726 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:25:31.726 INFO:teuthology.orchestra.run.smithi016.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-05-20T01:25:31.727 INFO:teuthology.orchestra.run.smithi016.stdout: auth mon ticket ttl = 660 # 11m 2023-05-20T01:25:31.727 INFO:teuthology.orchestra.run.smithi016.stdout: auth service ticket ttl = 240 # 4m 2023-05-20T01:25:31.727 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:25:31.727 INFO:teuthology.orchestra.run.smithi016.stdout: # don't complain about insecure global_id in the test suite 2023-05-20T01:25:31.727 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-05-20T01:25:31.727 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-05-20T01:25:31.727 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:25:31.727 INFO:teuthology.orchestra.run.smithi016.stdout: # 1m isn't quite enough 2023-05-20T01:25:31.728 INFO:teuthology.orchestra.run.smithi016.stdout: mon_down_mkfs_grace = 2m 2023-05-20T01:25:31.728 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:25:31.728 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_filestore_osds = false 2023-05-20T01:25:31.728 INFO:teuthology.orchestra.run.smithi016.stdout: mon op complaint time = 120 2023-05-20T01:25:31.728 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:25:31.728 INFO:teuthology.orchestra.run.smithi016.stdout:[client] 2023-05-20T01:25:31.729 INFO:teuthology.orchestra.run.smithi016.stdout: rgw cache enabled = true 2023-05-20T01:25:31.729 INFO:teuthology.orchestra.run.smithi016.stdout: rgw enable ops log = true 2023-05-20T01:25:31.729 INFO:teuthology.orchestra.run.smithi016.stdout: rgw enable usage log = true 2023-05-20T01:25:31.729 INFO:teuthology.orchestra.run.smithi016.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-05-20T01:25:31.729 INFO:teuthology.orchestra.run.smithi016.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-05-20T01:25:31.729 INFO:teuthology.orchestra.run.smithi016.stdout: client mount timeout = 600 2023-05-20T01:25:31.729 INFO:teuthology.orchestra.run.smithi016.stdout: debug client = 20 2023-05-20T01:25:31.729 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2023-05-20T01:25:31.730 INFO:teuthology.orchestra.run.smithi016.stdout: pid file = "" 2023-05-20T01:25:31.730 INFO:teuthology.orchestra.run.smithi016.stdout: rados mon op timeout = 900 2023-05-20T01:25:31.730 INFO:teuthology.orchestra.run.smithi016.stdout: rados osd op timeout = 900 2023-05-20T01:25:31.730 INFO:teuthology.orchestra.run.smithi016.stdout:[mon.a] 2023-05-20T01:25:31.730 INFO:teuthology.orchestra.run.smithi016.stdout:[mon.b] 2023-05-20T01:25:31.730 INFO:teuthology.orchestra.run.smithi016.stdout:[mon.c] 2023-05-20T01:25:31.731 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.a] 2023-05-20T01:25:31.731 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.c] 2023-05-20T01:25:31.731 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.b] 2023-05-20T01:25:31.731 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.d] 2023-05-20T01:25:31.731 INFO:teuthology.orchestra.run.smithi016.stdout:[mds] 2023-05-20T01:25:31.731 INFO:teuthology.orchestra.run.smithi016.stdout: debug mds = 20 2023-05-20T01:25:31.731 INFO:teuthology.orchestra.run.smithi016.stdout: debug mds balancer = 20 2023-05-20T01:25:31.732 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2023-05-20T01:25:31.732 INFO:teuthology.orchestra.run.smithi016.stdout: mds debug frag = True 2023-05-20T01:25:31.732 INFO:teuthology.orchestra.run.smithi016.stdout: mds debug scatterstat = False 2023-05-20T01:25:31.732 INFO:teuthology.orchestra.run.smithi016.stdout: mds op complaint time = 180 2023-05-20T01:25:31.732 INFO:teuthology.orchestra.run.smithi016.stdout: mds verify scatter = False 2023-05-20T01:25:31.732 INFO:teuthology.orchestra.run.smithi016.stdout: osd op complaint time = 180 2023-05-20T01:25:31.732 INFO:teuthology.orchestra.run.smithi016.stdout: rados mon op timeout = 900 2023-05-20T01:25:31.732 INFO:teuthology.orchestra.run.smithi016.stdout: rados osd op timeout = 900 2023-05-20T01:25:31.734 INFO:teuthology.orchestra.run.smithi132.stdout:[global] 2023-05-20T01:25:31.734 INFO:teuthology.orchestra.run.smithi132.stdout: chdir = "" 2023-05-20T01:25:31.734 INFO:teuthology.orchestra.run.smithi132.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-05-20T01:25:31.734 INFO:teuthology.orchestra.run.smithi132.stdout: auth supported = cephx 2023-05-20T01:25:31.735 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:25:31.735 INFO:teuthology.orchestra.run.smithi132.stdout: filestore xattr use omap = true 2023-05-20T01:25:31.735 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:25:31.735 INFO:teuthology.orchestra.run.smithi132.stdout: mon clock drift allowed = 1.000 2023-05-20T01:25:31.735 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:25:31.735 INFO:teuthology.orchestra.run.smithi132.stdout: osd crush chooseleaf type = 0 2023-05-20T01:25:31.735 INFO:teuthology.orchestra.run.smithi132.stdout: auth debug = true 2023-05-20T01:25:31.736 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:25:31.736 INFO:teuthology.orchestra.run.smithi132.stdout: ms die on old message = true 2023-05-20T01:25:31.736 INFO:teuthology.orchestra.run.smithi132.stdout: ms die on bug = true 2023-05-20T01:25:31.736 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:25:31.736 INFO:teuthology.orchestra.run.smithi132.stdout: mon max pg per osd = 10000 # >= luminous 2023-05-20T01:25:31.737 INFO:teuthology.orchestra.run.smithi132.stdout: mon pg warn max object skew = 0 2023-05-20T01:25:31.737 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:25:31.737 INFO:teuthology.orchestra.run.smithi132.stdout: # disable pg_autoscaler by default for new pools 2023-05-20T01:25:31.737 INFO:teuthology.orchestra.run.smithi132.stdout: osd_pool_default_pg_autoscale_mode = off 2023-05-20T01:25:31.737 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:25:31.737 INFO:teuthology.orchestra.run.smithi132.stdout: osd pool default size = 2 2023-05-20T01:25:31.738 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:25:31.738 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd allow primary affinity = true 2023-05-20T01:25:31.738 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd allow pg remap = true 2023-05-20T01:25:31.738 INFO:teuthology.orchestra.run.smithi132.stdout: mon warn on legacy crush tunables = false 2023-05-20T01:25:31.738 INFO:teuthology.orchestra.run.smithi132.stdout: mon warn on crush straw calc version zero = false 2023-05-20T01:25:31.738 INFO:teuthology.orchestra.run.smithi132.stdout: mon warn on no sortbitwise = false 2023-05-20T01:25:31.738 INFO:teuthology.orchestra.run.smithi132.stdout: mon warn on osd down out interval zero = false 2023-05-20T01:25:31.739 INFO:teuthology.orchestra.run.smithi132.stdout: mon warn on too few osds = false 2023-05-20T01:25:31.739 INFO:teuthology.orchestra.run.smithi132.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-05-20T01:25:31.739 INFO:teuthology.orchestra.run.smithi132.stdout: mon_warn_on_pool_no_redundancy = false 2023-05-20T01:25:31.739 INFO:teuthology.orchestra.run.smithi132.stdout: mon_allow_pool_size_one = true 2023-05-20T01:25:31.739 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:25:31.739 INFO:teuthology.orchestra.run.smithi132.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-05-20T01:25:31.740 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:25:31.740 INFO:teuthology.orchestra.run.smithi132.stdout: osd default data pool replay window = 5 2023-05-20T01:25:31.740 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:25:31.740 INFO:teuthology.orchestra.run.smithi132.stdout: mon allow pool delete = true 2023-05-20T01:25:31.740 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:25:31.740 INFO:teuthology.orchestra.run.smithi132.stdout: mon cluster log file level = debug 2023-05-20T01:25:31.740 INFO:teuthology.orchestra.run.smithi132.stdout: debug asserts on shutdown = true 2023-05-20T01:25:31.741 INFO:teuthology.orchestra.run.smithi132.stdout: mon health detail to clog = false 2023-05-20T01:25:31.741 INFO:teuthology.orchestra.run.smithi132.stdout: mon host = "172.21.15.16,172.21.15.132,[v2:172.21.15.132:3301,v1:172.21.15.132:6790]" 2023-05-20T01:25:31.741 INFO:teuthology.orchestra.run.smithi132.stdout: ms die on skipped message = False 2023-05-20T01:25:31.741 INFO:teuthology.orchestra.run.smithi132.stdout: fsid = c2245cc5-ecab-4087-a59b-bc81db901676 2023-05-20T01:25:31.741 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:25:31.741 INFO:teuthology.orchestra.run.smithi132.stdout:[osd] 2023-05-20T01:25:31.742 INFO:teuthology.orchestra.run.smithi132.stdout: osd journal size = 100 2023-05-20T01:25:31.742 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:25:31.742 INFO:teuthology.orchestra.run.smithi132.stdout: osd scrub load threshold = 5.0 2023-05-20T01:25:31.742 INFO:teuthology.orchestra.run.smithi132.stdout: osd scrub max interval = 600 2023-05-20T01:25:31.742 INFO:teuthology.orchestra.run.smithi132.stdout: osd mclock profile = high_recovery_ops 2023-05-20T01:25:31.742 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:25:31.742 INFO:teuthology.orchestra.run.smithi132.stdout: osd recover clone overlap = true 2023-05-20T01:25:31.743 INFO:teuthology.orchestra.run.smithi132.stdout: osd recovery max chunk = 1048576 2023-05-20T01:25:31.743 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:25:31.743 INFO:teuthology.orchestra.run.smithi132.stdout: osd debug shutdown = true 2023-05-20T01:25:31.743 INFO:teuthology.orchestra.run.smithi132.stdout: osd debug op order = true 2023-05-20T01:25:31.743 INFO:teuthology.orchestra.run.smithi132.stdout: osd debug verify stray on activate = true 2023-05-20T01:25:31.743 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:25:31.743 INFO:teuthology.orchestra.run.smithi132.stdout: osd open classes on start = true 2023-05-20T01:25:31.743 INFO:teuthology.orchestra.run.smithi132.stdout: osd debug pg log writeout = true 2023-05-20T01:25:31.744 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:25:31.744 INFO:teuthology.orchestra.run.smithi132.stdout: osd deep scrub update digest min age = 30 2023-05-20T01:25:31.744 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:25:31.744 INFO:teuthology.orchestra.run.smithi132.stdout: osd map max advance = 10 2023-05-20T01:25:31.744 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:25:31.745 INFO:teuthology.orchestra.run.smithi132.stdout: journal zero on create = true 2023-05-20T01:25:31.745 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:25:31.745 INFO:teuthology.orchestra.run.smithi132.stdout: filestore ondisk finisher threads = 3 2023-05-20T01:25:31.745 INFO:teuthology.orchestra.run.smithi132.stdout: filestore apply finisher threads = 3 2023-05-20T01:25:31.745 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:25:31.745 INFO:teuthology.orchestra.run.smithi132.stdout: bdev debug aio = true 2023-05-20T01:25:31.745 INFO:teuthology.orchestra.run.smithi132.stdout: osd debug misdirected ops = true 2023-05-20T01:25:31.745 INFO:teuthology.orchestra.run.smithi132.stdout: bluestore block size = 96636764160 2023-05-20T01:25:31.746 INFO:teuthology.orchestra.run.smithi132.stdout: bluestore fsck on mount = True 2023-05-20T01:25:31.746 INFO:teuthology.orchestra.run.smithi132.stdout: debug bluefs = 20 2023-05-20T01:25:31.746 INFO:teuthology.orchestra.run.smithi132.stdout: debug bluestore = 20 2023-05-20T01:25:31.746 INFO:teuthology.orchestra.run.smithi132.stdout: debug ms = 1 2023-05-20T01:25:31.746 INFO:teuthology.orchestra.run.smithi132.stdout: debug osd = 20 2023-05-20T01:25:31.746 INFO:teuthology.orchestra.run.smithi132.stdout: debug rocksdb = 10 2023-05-20T01:25:31.746 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd backfillfull_ratio = 0.85 2023-05-20T01:25:31.746 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd full ratio = 0.9 2023-05-20T01:25:31.747 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd nearfull ratio = 0.8 2023-05-20T01:25:31.747 INFO:teuthology.orchestra.run.smithi132.stdout: osd failsafe full ratio = 0.95 2023-05-20T01:25:31.747 INFO:teuthology.orchestra.run.smithi132.stdout: osd objectstore = bluestore 2023-05-20T01:25:31.747 INFO:teuthology.orchestra.run.smithi132.stdout: osd op complaint time = 180 2023-05-20T01:25:31.747 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:25:31.747 INFO:teuthology.orchestra.run.smithi132.stdout:[mgr] 2023-05-20T01:25:31.747 INFO:teuthology.orchestra.run.smithi132.stdout: debug ms = 1 2023-05-20T01:25:31.747 INFO:teuthology.orchestra.run.smithi132.stdout: debug mgr = 20 2023-05-20T01:25:31.748 INFO:teuthology.orchestra.run.smithi132.stdout: debug mon = 20 2023-05-20T01:25:31.748 INFO:teuthology.orchestra.run.smithi132.stdout: debug auth = 20 2023-05-20T01:25:31.748 INFO:teuthology.orchestra.run.smithi132.stdout: mon reweight min pgs per osd = 4 2023-05-20T01:25:31.748 INFO:teuthology.orchestra.run.smithi132.stdout: mon reweight min bytes per osd = 10 2023-05-20T01:25:31.748 INFO:teuthology.orchestra.run.smithi132.stdout: mgr/telemetry/nag = false 2023-05-20T01:25:31.748 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:25:31.749 INFO:teuthology.orchestra.run.smithi132.stdout:[mon] 2023-05-20T01:25:31.749 INFO:teuthology.orchestra.run.smithi132.stdout: debug ms = 1 2023-05-20T01:25:31.749 INFO:teuthology.orchestra.run.smithi132.stdout: debug mon = 20 2023-05-20T01:25:31.749 INFO:teuthology.orchestra.run.smithi132.stdout: debug paxos = 20 2023-05-20T01:25:31.749 INFO:teuthology.orchestra.run.smithi132.stdout: debug auth = 20 2023-05-20T01:25:31.749 INFO:teuthology.orchestra.run.smithi132.stdout: mon data avail warn = 5 2023-05-20T01:25:31.749 INFO:teuthology.orchestra.run.smithi132.stdout: mon mgr mkfs grace = 240 2023-05-20T01:25:31.749 INFO:teuthology.orchestra.run.smithi132.stdout: mon reweight min pgs per osd = 4 2023-05-20T01:25:31.750 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd reporter subtree level = osd 2023-05-20T01:25:31.750 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd prime pg temp = true 2023-05-20T01:25:31.750 INFO:teuthology.orchestra.run.smithi132.stdout: mon reweight min bytes per osd = 10 2023-05-20T01:25:31.750 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:25:31.750 INFO:teuthology.orchestra.run.smithi132.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-05-20T01:25:31.750 INFO:teuthology.orchestra.run.smithi132.stdout: auth mon ticket ttl = 660 # 11m 2023-05-20T01:25:31.750 INFO:teuthology.orchestra.run.smithi132.stdout: auth service ticket ttl = 240 # 4m 2023-05-20T01:25:31.751 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:25:31.751 INFO:teuthology.orchestra.run.smithi132.stdout: # don't complain about insecure global_id in the test suite 2023-05-20T01:25:31.751 INFO:teuthology.orchestra.run.smithi132.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-05-20T01:25:31.751 INFO:teuthology.orchestra.run.smithi132.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-05-20T01:25:31.751 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:25:31.751 INFO:teuthology.orchestra.run.smithi132.stdout: # 1m isn't quite enough 2023-05-20T01:25:31.751 INFO:teuthology.orchestra.run.smithi132.stdout: mon_down_mkfs_grace = 2m 2023-05-20T01:25:31.752 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:25:31.752 INFO:teuthology.orchestra.run.smithi132.stdout: mon_warn_on_filestore_osds = false 2023-05-20T01:25:31.752 INFO:teuthology.orchestra.run.smithi132.stdout: mon op complaint time = 120 2023-05-20T01:25:31.752 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:25:31.752 INFO:teuthology.orchestra.run.smithi132.stdout:[client] 2023-05-20T01:25:31.753 INFO:teuthology.orchestra.run.smithi132.stdout: rgw cache enabled = true 2023-05-20T01:25:31.753 INFO:teuthology.orchestra.run.smithi132.stdout: rgw enable ops log = true 2023-05-20T01:25:31.753 INFO:teuthology.orchestra.run.smithi132.stdout: rgw enable usage log = true 2023-05-20T01:25:31.753 INFO:teuthology.orchestra.run.smithi132.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-05-20T01:25:31.753 INFO:teuthology.orchestra.run.smithi132.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-05-20T01:25:31.753 INFO:teuthology.orchestra.run.smithi132.stdout: client mount timeout = 600 2023-05-20T01:25:31.753 INFO:teuthology.orchestra.run.smithi132.stdout: debug client = 20 2023-05-20T01:25:31.754 INFO:teuthology.orchestra.run.smithi132.stdout: debug ms = 1 2023-05-20T01:25:31.754 INFO:teuthology.orchestra.run.smithi132.stdout: pid file = "" 2023-05-20T01:25:31.754 INFO:teuthology.orchestra.run.smithi132.stdout: rados mon op timeout = 900 2023-05-20T01:25:31.754 INFO:teuthology.orchestra.run.smithi132.stdout: rados osd op timeout = 900 2023-05-20T01:25:31.754 INFO:teuthology.orchestra.run.smithi132.stdout:[mon.a] 2023-05-20T01:25:31.754 INFO:teuthology.orchestra.run.smithi132.stdout:[mon.b] 2023-05-20T01:25:31.754 INFO:teuthology.orchestra.run.smithi132.stdout:[mon.c] 2023-05-20T01:25:31.755 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.a] 2023-05-20T01:25:31.755 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.c] 2023-05-20T01:25:31.755 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.b] 2023-05-20T01:25:31.755 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.d] 2023-05-20T01:25:31.755 INFO:teuthology.orchestra.run.smithi132.stdout:[mds] 2023-05-20T01:25:31.755 INFO:teuthology.orchestra.run.smithi132.stdout: debug mds = 20 2023-05-20T01:25:31.755 INFO:teuthology.orchestra.run.smithi132.stdout: debug mds balancer = 20 2023-05-20T01:25:31.755 INFO:teuthology.orchestra.run.smithi132.stdout: debug ms = 1 2023-05-20T01:25:31.756 INFO:teuthology.orchestra.run.smithi132.stdout: mds debug frag = True 2023-05-20T01:25:31.756 INFO:teuthology.orchestra.run.smithi132.stdout: mds debug scatterstat = False 2023-05-20T01:25:31.756 INFO:teuthology.orchestra.run.smithi132.stdout: mds op complaint time = 180 2023-05-20T01:25:31.756 INFO:teuthology.orchestra.run.smithi132.stdout: mds verify scatter = False 2023-05-20T01:25:31.756 INFO:teuthology.orchestra.run.smithi132.stdout: osd op complaint time = 180 2023-05-20T01:25:31.756 INFO:teuthology.orchestra.run.smithi132.stdout: rados mon op timeout = 900 2023-05-20T01:25:31.756 INFO:teuthology.orchestra.run.smithi132.stdout: rados osd op timeout = 900 2023-05-20T01:25:31.757 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph purge cephfs_metadata --yes-i-really-really-mean-it 2023-05-20T01:25:31.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.812+0000 7f0d9b3fb080 1 Processor -- start 2023-05-20T01:25:31.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.812+0000 7f0d9b3fb080 1 -- start start 2023-05-20T01:25:31.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.813+0000 7f0d9b3fb080 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5571bc41f590 0x5571bc44f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:31.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.813+0000 7f0d9b3fb080 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5571bc4204f0 0x5571bc44db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:31.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.813+0000 7f0d9b3fb080 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571bc4f4e70 0x5571bc57e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:31.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.813+0000 7f0d9b3fb080 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571bc382b20 con 0x5571bc4f4e70 2023-05-20T01:25:31.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.813+0000 7f0d9b3fb080 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571bc358fe0 con 0x5571bc41f590 2023-05-20T01:25:31.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.813+0000 7f0d9b3fb080 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x5571bc465e30 con 0x5571bc4204f0 2023-05-20T01:25:31.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.813+0000 7f0d91cfc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571bc4f4e70 0x5571bc57e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:31.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.813+0000 7f0d91cfc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571bc4f4e70 0x5571bc57e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36654/0 (socket says 172.21.15.16:36654) 2023-05-20T01:25:31.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.813+0000 7f0d91cfc700 1 -- 172.21.15.16:0/83435040 learned_addr learned my addr 172.21.15.16:0/83435040 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:31.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.813+0000 7f0d90cfa700 1 --2- 172.21.15.16:0/83435040 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5571bc4204f0 0x5571bc44db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:31.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.813+0000 7f0d914fb700 1 --2- 172.21.15.16:0/83435040 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5571bc41f590 0x5571bc44f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:31.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.813+0000 7f0d91cfc700 1 -- 172.21.15.16:0/83435040 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5571bc4204f0 msgr2=0x5571bc44db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:31.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.813+0000 7f0d91cfc700 1 --2- 172.21.15.16:0/83435040 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5571bc4204f0 0x5571bc44db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:31.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.813+0000 7f0d91cfc700 1 -- 172.21.15.16:0/83435040 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5571bc41f590 msgr2=0x5571bc44f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:31.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.813+0000 7f0d91cfc700 1 --2- 172.21.15.16:0/83435040 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5571bc41f590 0x5571bc44f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:31.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.813+0000 7f0d91cfc700 1 -- 172.21.15.16:0/83435040 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d7c005de0 con 0x5571bc4f4e70 2023-05-20T01:25:31.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.813+0000 7f0d91cfc700 1 --2- 172.21.15.16:0/83435040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571bc4f4e70 0x5571bc57e990 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c00b5e0 tx=0x7f0d8c00b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22f4409117a58411 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:31.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.814+0000 7f0d8b7fe700 1 -- 172.21.15.16:0/83435040 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d8c014070 con 0x5571bc4f4e70 2023-05-20T01:25:31.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.814+0000 7f0d8b7fe700 1 -- 172.21.15.16:0/83435040 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d8c00b050 con 0x5571bc4f4e70 2023-05-20T01:25:31.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.814+0000 7f0d8b7fe700 1 -- 172.21.15.16:0/83435040 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d8c00f7c0 con 0x5571bc4f4e70 2023-05-20T01:25:31.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.814+0000 7f0d9b3fb080 1 -- 172.21.15.16:0/83435040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571bc4f4e70 msgr2=0x5571bc57e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:31.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.814+0000 7f0d9b3fb080 1 --2- 172.21.15.16:0/83435040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571bc4f4e70 0x5571bc57e990 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c00b5e0 tx=0x7f0d8c00b8f0 comp rx=0 tx=0).stop 2023-05-20T01:25:31.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.814+0000 7f0d9b3fb080 1 -- 172.21.15.16:0/83435040 shutdown_connections 2023-05-20T01:25:31.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.814+0000 7f0d9b3fb080 1 --2- 172.21.15.16:0/83435040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571bc4f4e70 0x5571bc57e990 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:31.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.814+0000 7f0d9b3fb080 1 --2- 172.21.15.16:0/83435040 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5571bc41f590 0x5571bc44f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:31.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.814+0000 7f0d9b3fb080 1 --2- 172.21.15.16:0/83435040 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5571bc4204f0 0x5571bc44db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:31.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.814+0000 7f0d9b3fb080 1 -- 172.21.15.16:0/83435040 >> 172.21.15.16:0/83435040 conn(0x5571bc4670a0 msgr2=0x5571bc4695c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:31.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.814+0000 7f0d9b3fb080 1 -- 172.21.15.16:0/83435040 shutdown_connections 2023-05-20T01:25:31.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.814+0000 7f0d9b3fb080 1 -- 172.21.15.16:0/83435040 wait complete. 2023-05-20T01:25:31.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.815+0000 7f0d9b3fb080 1 Processor -- start 2023-05-20T01:25:31.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.815+0000 7f0d9b3fb080 1 -- start start 2023-05-20T01:25:31.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.815+0000 7f0d9b3fb080 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571bc41f590 0x5571bc4ea240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:31.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.815+0000 7f0d9b3fb080 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5571bc4204f0 0x5571bc44e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:31.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.815+0000 7f0d9b3fb080 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5571bc4f4e70 0x5571bc44ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:31.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.815+0000 7f0d9b3fb080 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571bc4eb930 con 0x5571bc41f590 2023-05-20T01:25:31.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.815+0000 7f0d9b3fb080 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571bc44f100 con 0x5571bc4f4e70 2023-05-20T01:25:31.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.815+0000 7f0d9b3fb080 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x5571bc4ead00 con 0x5571bc4204f0 2023-05-20T01:25:31.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.816+0000 7f0d914fb700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571bc41f590 0x5571bc4ea240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:31.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.816+0000 7f0d914fb700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571bc41f590 0x5571bc4ea240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36658/0 (socket says 172.21.15.16:36658) 2023-05-20T01:25:31.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.816+0000 7f0d914fb700 1 -- 172.21.15.16:0/3801786603 learned_addr learned my addr 172.21.15.16:0/3801786603 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:31.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.816+0000 7f0d90cfa700 1 --2- 172.21.15.16:0/3801786603 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5571bc4204f0 0x5571bc44e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:31.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.816+0000 7f0d91cfc700 1 --2- 172.21.15.16:0/3801786603 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5571bc4f4e70 0x5571bc44ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:31.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.816+0000 7f0d914fb700 1 -- 172.21.15.16:0/3801786603 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5571bc4204f0 msgr2=0x5571bc44e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:31.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.816+0000 7f0d914fb700 1 --2- 172.21.15.16:0/3801786603 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5571bc4204f0 0x5571bc44e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:31.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.816+0000 7f0d914fb700 1 -- 172.21.15.16:0/3801786603 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5571bc4f4e70 msgr2=0x5571bc44ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:31.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.816+0000 7f0d914fb700 1 --2- 172.21.15.16:0/3801786603 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5571bc4f4e70 0x5571bc44ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:31.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.816+0000 7f0d914fb700 1 -- 172.21.15.16:0/3801786603 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d8c00b210 con 0x5571bc41f590 2023-05-20T01:25:31.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.816+0000 7f0d914fb700 1 --2- 172.21.15.16:0/3801786603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571bc41f590 0x5571bc4ea240 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c012000 tx=0x7f0d7c002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:31.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.816+0000 7f0d8a7fc700 1 -- 172.21.15.16:0/3801786603 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d7c012820 con 0x5571bc41f590 2023-05-20T01:25:31.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.816+0000 7f0d8a7fc700 1 -- 172.21.15.16:0/3801786603 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d7c0129a0 con 0x5571bc41f590 2023-05-20T01:25:31.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.816+0000 7f0d8a7fc700 1 -- 172.21.15.16:0/3801786603 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d7c012d50 con 0x5571bc41f590 2023-05-20T01:25:31.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.816+0000 7f0d9b3fb080 1 -- 172.21.15.16:0/3801786603 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5571bc38b240 con 0x5571bc41f590 2023-05-20T01:25:31.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.816+0000 7f0d9b3fb080 1 -- 172.21.15.16:0/3801786603 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5571bc3644a0 con 0x5571bc41f590 2023-05-20T01:25:31.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.818+0000 7f0d8a7fc700 1 -- 172.21.15.16:0/3801786603 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0d7c01d3f0 con 0x5571bc41f590 2023-05-20T01:25:31.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.818+0000 7f0d8a7fc700 1 --2- 172.21.15.16:0/3801786603 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f0d74069600 0x7f0d7406bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:31.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.818+0000 7f0d8a7fc700 1 -- 172.21.15.16:0/3801786603 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f0d7c089aa0 con 0x5571bc41f590 2023-05-20T01:25:31.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.818+0000 7f0d90cfa700 1 --2- 172.21.15.16:0/3801786603 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f0d74069600 0x7f0d7406bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:31.838 INFO:teuthology.orchestra.run.smithi016.stdout:Warning: using slow linear search 2023-05-20T01:25:31.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.818+0000 7f0d9b3fb080 1 --2- 172.21.15.16:0/3801786603 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x5571bc597a50 0x5571bc599e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:31.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.818+0000 7f0d9b3fb080 1 -- 172.21.15.16:0/3801786603 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:1 12.0 12:00000000:%01:::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) v8 -- 0x5571bc44e0c0 con 0x5571bc597a50 2023-05-20T01:25:31.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.818+0000 7f0d91cfc700 1 --2- 172.21.15.16:0/3801786603 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x5571bc597a50 0x5571bc599e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:31.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.818+0000 7f0d90cfa700 1 --2- 172.21.15.16:0/3801786603 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f0d74069600 0x7f0d7406bab0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x5571bc468020 tx=0x7f0d8400b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:31.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.819+0000 7f0d91cfc700 1 --2- 172.21.15.16:0/3801786603 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x5571bc597a50 0x5571bc599e30 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:31.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.819+0000 7f0d91cfc700 1 -- 172.21.15.16:0/3801786603 <== osd.4 v2:172.21.15.132:6817/506317223 1 ==== osd_op_reply(1 [pgnls start_epoch 92 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0d8c016070 con 0x5571bc597a50 2023-05-20T01:25:31.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.819+0000 7f0d91cfc700 1 --2- 172.21.15.16:0/3801786603 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x7f0d8c01b940 0x7f0d8c010410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:31.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.819+0000 7f0d91cfc700 1 -- 172.21.15.16:0/3801786603 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:2 12.10 12:08000000:%01:::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) v8 -- 0x5571bc44e0c0 con 0x7f0d8c01b940 2023-05-20T01:25:31.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.819+0000 7f0d914fb700 1 --2- 172.21.15.16:0/3801786603 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x7f0d8c01b940 0x7f0d8c010410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:31.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.820+0000 7f0d914fb700 1 --2- 172.21.15.16:0/3801786603 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x7f0d8c01b940 0x7f0d8c010410 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:31.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.820+0000 7f0d914fb700 1 -- 172.21.15.16:0/3801786603 <== osd.0 v2:172.21.15.16:6824/3923634179 1 ==== osd_op_reply(2 [pgnls start_epoch 92 out=177b] v91'10 uv10 ondisk = 1) v8 ==== 144+0+177 (crc 0 0 0) 0x7f0d7c019040 con 0x7f0d8c01b940 2023-05-20T01:25:31.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.820+0000 7f0d914fb700 1 --2- 172.21.15.16:0/3801786603 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x7f0d7c02f060 0x7f0d7c0574d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:31.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.820+0000 7f0d914fb700 1 -- 172.21.15.16:0/3801786603 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:3 12.8 12:10000000:%01:::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) v8 -- 0x5571bc44e0c0 con 0x7f0d7c02f060 2023-05-20T01:25:31.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.820+0000 7f0d90cfa700 1 --2- 172.21.15.16:0/3801786603 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x7f0d7c02f060 0x7f0d7c0574d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:31.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.820+0000 7f0d90cfa700 1 --2- 172.21.15.16:0/3801786603 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x7f0d7c02f060 0x7f0d7c0574d0 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:31.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.821+0000 7f0d90cfa700 1 -- 172.21.15.16:0/3801786603 <== osd.3 v2:172.21.15.16:6800/3669781890 1 ==== osd_op_reply(3 [pgnls start_epoch 92 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0d8401f040 con 0x7f0d7c02f060 2023-05-20T01:25:31.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.821+0000 7f0d90cfa700 1 --2- 172.21.15.16:0/3801786603 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x7f0d84015920 0x7f0d84006400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:31.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.821+0000 7f0d90cfa700 1 -- 172.21.15.16:0/3801786603 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:4 12.18 12:18000000:%01:::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) v8 -- 0x5571bc44e0c0 con 0x7f0d84015920 2023-05-20T01:25:31.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.821+0000 7f0d91cfc700 1 --2- 172.21.15.16:0/3801786603 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x7f0d84015920 0x7f0d84006400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:31.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.821+0000 7f0d91cfc700 1 --2- 172.21.15.16:0/3801786603 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x7f0d84015920 0x7f0d84006400 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:31.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.821+0000 7f0d91cfc700 1 -- 172.21.15.16:0/3801786603 <== osd.1 v2:172.21.15.16:6808/3860264220 1 ==== osd_op_reply(4 [pgnls start_epoch 92 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0d8c016070 con 0x7f0d84015920 2023-05-20T01:25:31.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.822+0000 7f0d91cfc700 1 --2- 172.21.15.16:0/3801786603 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x7f0d8c0198c0 0x7f0d8c0183a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:31.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.822+0000 7f0d91cfc700 1 -- 172.21.15.16:0/3801786603 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:5 12.4 12:20000000:%01:::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) v8 -- 0x5571bc44e0c0 con 0x7f0d8c0198c0 2023-05-20T01:25:31.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.822+0000 7f0d914fb700 1 --2- 172.21.15.16:0/3801786603 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x7f0d8c0198c0 0x7f0d8c0183a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:31.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.822+0000 7f0d914fb700 1 --2- 172.21.15.16:0/3801786603 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x7f0d8c0198c0 0x7f0d8c0183a0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:31.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.823+0000 7f0d914fb700 1 -- 172.21.15.16:0/3801786603 <== osd.6 v2:172.21.15.132:6801/727581123 1 ==== osd_op_reply(5 [pgnls start_epoch 92 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0d7c019040 con 0x7f0d8c0198c0 2023-05-20T01:25:31.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.823+0000 7f0d914fb700 1 --2- 172.21.15.16:0/3801786603 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x7f0d7c009040 0x7f0d7c058570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:31.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.823+0000 7f0d914fb700 1 -- 172.21.15.16:0/3801786603 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:6 12.14 12:28000000:%01:::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) v8 -- 0x5571bc44e0c0 con 0x7f0d7c009040 2023-05-20T01:25:31.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.823+0000 7f0d90cfa700 1 --2- 172.21.15.16:0/3801786603 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x7f0d7c009040 0x7f0d7c058570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:31.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.823+0000 7f0d90cfa700 1 --2- 172.21.15.16:0/3801786603 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x7f0d7c009040 0x7f0d7c058570 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:31.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.823+0000 7f0d90cfa700 1 -- 172.21.15.16:0/3801786603 <== osd.5 v2:172.21.15.132:6816/1844689995 1 ==== osd_op_reply(6 [pgnls start_epoch 92 out=149b] v91'21 uv21 ondisk = 1) v8 ==== 144+0+149 (crc 0 0 0) 0x7f0d8401f040 con 0x7f0d7c009040 2023-05-20T01:25:31.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.824+0000 7f0d90cfa700 1 -- 172.21.15.16:0/3801786603 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:7 12.c 12:30000000:%01:::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) v8 -- 0x5571bc44e0c0 con 0x7f0d8c01b940 2023-05-20T01:25:31.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.824+0000 7f0d914fb700 1 -- 172.21.15.16:0/3801786603 <== osd.0 v2:172.21.15.16:6824/3923634179 2 ==== osd_op_reply(7 [pgnls start_epoch 92 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0d7c019040 con 0x7f0d8c01b940 2023-05-20T01:25:31.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.824+0000 7f0d914fb700 1 -- 172.21.15.16:0/3801786603 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:8 12.1c 12:38000000:%01:::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) v8 -- 0x5571bc44e0c0 con 0x7f0d8c01b940 2023-05-20T01:25:31.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.824+0000 7f0d914fb700 1 -- 172.21.15.16:0/3801786603 <== osd.0 v2:172.21.15.16:6824/3923634179 3 ==== osd_op_reply(8 [pgnls start_epoch 92 out=73b] v87'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x7f0d7c019040 con 0x7f0d8c01b940 2023-05-20T01:25:31.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.824+0000 7f0d914fb700 1 -- 172.21.15.16:0/3801786603 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:9 12.2 12:40000000:%01:::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) v8 -- 0x5571bc44e0c0 con 0x5571bc597a50 2023-05-20T01:25:31.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.824+0000 7f0d91cfc700 1 -- 172.21.15.16:0/3801786603 <== osd.4 v2:172.21.15.132:6817/506317223 2 ==== osd_op_reply(9 [pgnls start_epoch 92 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0d8c016070 con 0x5571bc597a50 2023-05-20T01:25:31.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.824+0000 7f0d91cfc700 1 -- 172.21.15.16:0/3801786603 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:10 12.12 12:48000000:%01:::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) v8 -- 0x5571bc44e0c0 con 0x7f0d7c02f060 2023-05-20T01:25:31.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.825+0000 7f0d90cfa700 1 -- 172.21.15.16:0/3801786603 <== osd.3 v2:172.21.15.16:6800/3669781890 2 ==== osd_op_reply(10 [pgnls start_epoch 92 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0d8401f040 con 0x7f0d7c02f060 2023-05-20T01:25:31.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.825+0000 7f0d90cfa700 1 --2- 172.21.15.16:0/3801786603 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x7f0d84024900 0x7f0d8402f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:31.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.825+0000 7f0d90cfa700 1 -- 172.21.15.16:0/3801786603 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:11 12.a 12:50000000:%01:::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) v8 -- 0x5571bc44e0c0 con 0x7f0d84024900 2023-05-20T01:25:31.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.825+0000 7f0d91cfc700 1 --2- 172.21.15.16:0/3801786603 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x7f0d84024900 0x7f0d8402f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:31.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.825+0000 7f0d91cfc700 1 --2- 172.21.15.16:0/3801786603 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x7f0d84024900 0x7f0d8402f3e0 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:31.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.825+0000 7f0d91cfc700 1 -- 172.21.15.16:0/3801786603 <== osd.2 v2:172.21.15.16:6816/293187030 1 ==== osd_op_reply(11 [pgnls start_epoch 92 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0d8c016070 con 0x7f0d84024900 2023-05-20T01:25:31.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.825+0000 7f0d91cfc700 1 -- 172.21.15.16:0/3801786603 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:12 12.1a 12:58000000:%01:::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) v8 -- 0x5571bc44e0c0 con 0x5571bc597a50 2023-05-20T01:25:31.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.826+0000 7f0d91cfc700 1 -- 172.21.15.16:0/3801786603 <== osd.4 v2:172.21.15.132:6817/506317223 3 ==== osd_op_reply(12 [pgnls start_epoch 92 out=49b] v87'1 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0d8c016070 con 0x5571bc597a50 2023-05-20T01:25:31.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.826+0000 7f0d91cfc700 1 -- 172.21.15.16:0/3801786603 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:13 12.6 12:60000000:%01:::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) v8 -- 0x5571bc44e0c0 con 0x7f0d84015920 2023-05-20T01:25:31.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.826+0000 7f0d91cfc700 1 -- 172.21.15.16:0/3801786603 <== osd.1 v2:172.21.15.16:6808/3860264220 2 ==== osd_op_reply(13 [pgnls start_epoch 92 out=77b] v91'2 uv2 ondisk = 1) v8 ==== 144+0+77 (crc 0 0 0) 0x7f0d8c016070 con 0x7f0d84015920 2023-05-20T01:25:31.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.826+0000 7f0d91cfc700 1 -- 172.21.15.16:0/3801786603 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:14 12.16 12:68000000:%01:::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) v8 -- 0x5571bc44e0c0 con 0x7f0d8c0198c0 2023-05-20T01:25:31.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.826+0000 7f0d914fb700 1 -- 172.21.15.16:0/3801786603 <== osd.6 v2:172.21.15.132:6801/727581123 2 ==== osd_op_reply(14 [pgnls start_epoch 92 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0d7c075040 con 0x7f0d8c0198c0 2023-05-20T01:25:31.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.826+0000 7f0d914fb700 1 -- 172.21.15.16:0/3801786603 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:15 12.e 12:70000000:%01:::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) v8 -- 0x5571bc44e0c0 con 0x7f0d8c01b940 2023-05-20T01:25:31.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.826+0000 7f0d914fb700 1 -- 172.21.15.16:0/3801786603 <== osd.0 v2:172.21.15.16:6824/3923634179 4 ==== osd_op_reply(15 [pgnls start_epoch 92 out=73b] v87'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x7f0d7c075040 con 0x7f0d8c01b940 2023-05-20T01:25:31.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.826+0000 7f0d914fb700 1 -- 172.21.15.16:0/3801786603 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:16 12.1e 12:78000000:%01:::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) v8 -- 0x5571bc44e0c0 con 0x5571bc597a50 2023-05-20T01:25:31.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.827+0000 7f0d91cfc700 1 -- 172.21.15.16:0/3801786603 <== osd.4 v2:172.21.15.132:6817/506317223 4 ==== osd_op_reply(16 [pgnls start_epoch 92 out=49b] v87'1 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0d8c016070 con 0x5571bc597a50 2023-05-20T01:25:31.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.827+0000 7f0d91cfc700 1 -- 172.21.15.16:0/3801786603 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:17 12.1 12:80000000:%01:::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) v8 -- 0x5571bc44e0c0 con 0x7f0d8c0198c0 2023-05-20T01:25:31.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.827+0000 7f0d914fb700 1 -- 172.21.15.16:0/3801786603 <== osd.6 v2:172.21.15.132:6801/727581123 3 ==== osd_op_reply(17 [pgnls start_epoch 92 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0d7c075040 con 0x7f0d8c0198c0 2023-05-20T01:25:31.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.827+0000 7f0d914fb700 1 -- 172.21.15.16:0/3801786603 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:18 12.11 12:88000000:%01:::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) v8 -- 0x5571bc44e0c0 con 0x7f0d7c009040 2023-05-20T01:25:31.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.827+0000 7f0d90cfa700 1 -- 172.21.15.16:0/3801786603 <== osd.5 v2:172.21.15.132:6816/1844689995 2 ==== osd_op_reply(18 [pgnls start_epoch 92 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0d8401f040 con 0x7f0d7c009040 2023-05-20T01:25:31.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.827+0000 7f0d90cfa700 1 -- 172.21.15.16:0/3801786603 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:19 12.9 12:90000000:%01:::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) v8 -- 0x5571bc44e0c0 con 0x7f0d84024900 2023-05-20T01:25:31.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.828+0000 7f0d91cfc700 1 -- 172.21.15.16:0/3801786603 <== osd.2 v2:172.21.15.16:6816/293187030 2 ==== osd_op_reply(19 [pgnls start_epoch 92 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0d8c016070 con 0x7f0d84024900 2023-05-20T01:25:31.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.828+0000 7f0d91cfc700 1 -- 172.21.15.16:0/3801786603 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:20 12.19 12:98000000:%01:::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) v8 -- 0x5571bc44e0c0 con 0x7f0d8c01b940 2023-05-20T01:25:31.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.828+0000 7f0d914fb700 1 -- 172.21.15.16:0/3801786603 <== osd.0 v2:172.21.15.16:6824/3923634179 5 ==== osd_op_reply(20 [pgnls start_epoch 92 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0d7c075040 con 0x7f0d8c01b940 2023-05-20T01:25:31.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.828+0000 7f0d914fb700 1 -- 172.21.15.16:0/3801786603 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:21 12.5 12:a0000000:%01:::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) v8 -- 0x5571bc44e0c0 con 0x7f0d8c0198c0 2023-05-20T01:25:31.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.828+0000 7f0d914fb700 1 -- 172.21.15.16:0/3801786603 <== osd.6 v2:172.21.15.132:6801/727581123 4 ==== osd_op_reply(21 [pgnls start_epoch 92 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0d7c075040 con 0x7f0d8c0198c0 2023-05-20T01:25:31.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.828+0000 7f0d914fb700 1 -- 172.21.15.16:0/3801786603 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:22 12.15 12:a8000000:%01:::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) v8 -- 0x5571bc44e0c0 con 0x7f0d8c0198c0 2023-05-20T01:25:31.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.829+0000 7f0d914fb700 1 -- 172.21.15.16:0/3801786603 <== osd.6 v2:172.21.15.132:6801/727581123 5 ==== osd_op_reply(22 [pgnls start_epoch 92 out=97b] v87'3 uv3 ondisk = 1) v8 ==== 144+0+97 (crc 0 0 0) 0x7f0d7c075040 con 0x7f0d8c0198c0 2023-05-20T01:25:31.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.829+0000 7f0d914fb700 1 --2- 172.21.15.16:0/3801786603 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x7f0d7c060430 0x7f0d7c05c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:31.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.829+0000 7f0d914fb700 1 -- 172.21.15.16:0/3801786603 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:23 12.d 12:b0000000:%01:::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) v8 -- 0x5571bc44e0c0 con 0x7f0d7c060430 2023-05-20T01:25:31.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.829+0000 7f0d914fb700 1 --2- 172.21.15.16:0/3801786603 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x7f0d7c060430 0x7f0d7c05c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:31.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.829+0000 7f0d914fb700 1 --2- 172.21.15.16:0/3801786603 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x7f0d7c060430 0x7f0d7c05c790 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:31.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.829+0000 7f0d914fb700 1 -- 172.21.15.16:0/3801786603 <== osd.7 v2:172.21.15.132:6800/466001149 1 ==== osd_op_reply(23 [pgnls start_epoch 92 out=74b] v87'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x7f0d7c075040 con 0x7f0d7c060430 2023-05-20T01:25:31.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.829+0000 7f0d914fb700 1 -- 172.21.15.16:0/3801786603 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:24 12.1d 12:b8000000:%01:::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) v8 -- 0x5571bc44e0c0 con 0x7f0d8c01b940 2023-05-20T01:25:31.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.830+0000 7f0d914fb700 1 -- 172.21.15.16:0/3801786603 <== osd.0 v2:172.21.15.16:6824/3923634179 6 ==== osd_op_reply(24 [pgnls start_epoch 92 out=73b] v87'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x7f0d7c075040 con 0x7f0d8c01b940 2023-05-20T01:25:31.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.830+0000 7f0d914fb700 1 -- 172.21.15.16:0/3801786603 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:25 12.3 12:c0000000:%01:::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) v8 -- 0x5571bc44e0c0 con 0x7f0d8c01b940 2023-05-20T01:25:31.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.830+0000 7f0d914fb700 1 -- 172.21.15.16:0/3801786603 <== osd.0 v2:172.21.15.16:6824/3923634179 7 ==== osd_op_reply(25 [pgnls start_epoch 92 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0d7c075040 con 0x7f0d8c01b940 2023-05-20T01:25:31.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.830+0000 7f0d914fb700 1 -- 172.21.15.16:0/3801786603 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:26 12.13 12:c8000000:%01:::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) v8 -- 0x5571bc44e0c0 con 0x7f0d84015920 2023-05-20T01:25:31.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.830+0000 7f0d91cfc700 1 -- 172.21.15.16:0/3801786603 <== osd.1 v2:172.21.15.16:6808/3860264220 3 ==== osd_op_reply(26 [pgnls start_epoch 92 out=98b] v91'3 uv3 ondisk = 1) v8 ==== 144+0+98 (crc 0 0 0) 0x7f0d8c03f040 con 0x7f0d84015920 2023-05-20T01:25:31.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.830+0000 7f0d91cfc700 1 -- 172.21.15.16:0/3801786603 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:27 12.b 12:d0000000:%01:::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) v8 -- 0x5571bc44e0c0 con 0x7f0d7c02f060 2023-05-20T01:25:31.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.831+0000 7f0d90cfa700 1 -- 172.21.15.16:0/3801786603 <== osd.3 v2:172.21.15.16:6800/3669781890 3 ==== osd_op_reply(27 [pgnls start_epoch 92 out=76b] v91'3 uv3 ondisk = 1) v8 ==== 144+0+76 (crc 0 0 0) 0x7f0d8401f040 con 0x7f0d7c02f060 2023-05-20T01:25:31.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.831+0000 7f0d90cfa700 1 -- 172.21.15.16:0/3801786603 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:28 12.1b 12:d8000000:%01:::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) v8 -- 0x5571bc44e0c0 con 0x5571bc597a50 2023-05-20T01:25:31.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.831+0000 7f0d91cfc700 1 -- 172.21.15.16:0/3801786603 <== osd.4 v2:172.21.15.132:6817/506317223 5 ==== osd_op_reply(28 [pgnls start_epoch 92 out=49b] v87'1 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0d8c03f040 con 0x5571bc597a50 2023-05-20T01:25:31.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.831+0000 7f0d91cfc700 1 -- 172.21.15.16:0/3801786603 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:29 12.7 12:e0000000:%01:::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) v8 -- 0x5571bc44e0c0 con 0x7f0d84015920 2023-05-20T01:25:31.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.831+0000 7f0d91cfc700 1 -- 172.21.15.16:0/3801786603 <== osd.1 v2:172.21.15.16:6808/3860264220 4 ==== osd_op_reply(29 [pgnls start_epoch 92 out=73b] v87'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x7f0d8c03f040 con 0x7f0d84015920 2023-05-20T01:25:31.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.831+0000 7f0d91cfc700 1 -- 172.21.15.16:0/3801786603 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:30 12.17 12:e8000000:%01:::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) v8 -- 0x5571bc44e0c0 con 0x7f0d8c0198c0 2023-05-20T01:25:31.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.831+0000 7f0d914fb700 1 -- 172.21.15.16:0/3801786603 <== osd.6 v2:172.21.15.132:6801/727581123 6 ==== osd_op_reply(30 [pgnls start_epoch 92 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0d7c075040 con 0x7f0d8c0198c0 2023-05-20T01:25:31.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.832+0000 7f0d914fb700 1 -- 172.21.15.16:0/3801786603 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:31 12.f 12:f0000000:%01:::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) v8 -- 0x5571bc44e0c0 con 0x7f0d8c0198c0 2023-05-20T01:25:31.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.832+0000 7f0d914fb700 1 -- 172.21.15.16:0/3801786603 <== osd.6 v2:172.21.15.132:6801/727581123 7 ==== osd_op_reply(31 [pgnls start_epoch 92 out=73b] v87'1 uv1 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x7f0d7c075040 con 0x7f0d8c0198c0 2023-05-20T01:25:31.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.832+0000 7f0d914fb700 1 -- 172.21.15.16:0/3801786603 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:32 12.1f 12:f8000000:%01:::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) v8 -- 0x5571bc44e0c0 con 0x7f0d8c0198c0 2023-05-20T01:25:31.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.832+0000 7f0d914fb700 1 -- 172.21.15.16:0/3801786603 <== osd.6 v2:172.21.15.132:6801/727581123 8 ==== osd_op_reply(32 [pgnls start_epoch 92 out=101b] v91'6 uv6 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x7f0d7c075040 con 0x7f0d8c0198c0 2023-05-20T01:25:31.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.832+0000 7f0d9b3fb080 1 -- 172.21.15.16:0/3801786603 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:33 12.10 12:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e92) v8 -- 0x5571bc4f5540 con 0x7f0d8c01b940 2023-05-20T01:25:31.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.832+0000 7f0d9b3fb080 1 -- 172.21.15.16:0/3801786603 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:34 12.10 12:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e92) v8 -- 0x5571bc4f5d70 con 0x7f0d8c01b940 2023-05-20T01:25:31.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.832+0000 7f0d9b3fb080 1 -- 172.21.15.16:0/3801786603 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:35 12.10 12:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e92) v8 -- 0x5571bc59a810 con 0x7f0d8c01b940 2023-05-20T01:25:31.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.832+0000 7f0d9b3fb080 1 -- 172.21.15.16:0/3801786603 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:36 12.10 12:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e92) v8 -- 0x5571bc59b010 con 0x7f0d8c01b940 2023-05-20T01:25:31.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.832+0000 7f0d9b3fb080 1 -- 172.21.15.16:0/3801786603 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:37 12.10 12:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e92) v8 -- 0x5571bc59b810 con 0x7f0d8c01b940 2023-05-20T01:25:31.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.832+0000 7f0d9b3fb080 1 -- 172.21.15.16:0/3801786603 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:38 12.14 12:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e92) v8 -- 0x5571bc59c010 con 0x7f0d7c009040 2023-05-20T01:25:31.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.832+0000 7f0d9b3fb080 1 -- 172.21.15.16:0/3801786603 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:39 12.14 12:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e92) v8 -- 0x5571bc59c810 con 0x7f0d7c009040 2023-05-20T01:25:31.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.832+0000 7f0d9b3fb080 1 -- 172.21.15.16:0/3801786603 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:40 12.14 12:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e92) v8 -- 0x5571bc59d010 con 0x7f0d7c009040 2023-05-20T01:25:31.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.832+0000 7f0d9b3fb080 1 -- 172.21.15.16:0/3801786603 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:41 12.14 12:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e92) v8 -- 0x5571bc59d810 con 0x7f0d7c009040 2023-05-20T01:25:31.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.832+0000 7f0d9b3fb080 1 -- 172.21.15.16:0/3801786603 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:42 12.1c 12:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e92) v8 -- 0x5571bc59e010 con 0x7f0d8c01b940 2023-05-20T01:25:31.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.832+0000 7f0d9b3fb080 1 -- 172.21.15.16:0/3801786603 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:43 12.6 12:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e92) v8 -- 0x5571bc59e810 con 0x7f0d84015920 2023-05-20T01:25:31.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.832+0000 7f0d9b3fb080 1 -- 172.21.15.16:0/3801786603 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:44 12.e 12:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e92) v8 -- 0x5571bc59f270 con 0x7f0d8c01b940 2023-05-20T01:25:31.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.832+0000 7f0d9b3fb080 1 -- 172.21.15.16:0/3801786603 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:45 12.15 12:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e92) v8 -- 0x5571bc59fd60 con 0x7f0d8c0198c0 2023-05-20T01:25:31.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.832+0000 7f0d9b3fb080 1 -- 172.21.15.16:0/3801786603 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:46 12.15 12:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e92) v8 -- 0x5571bc5a07e0 con 0x7f0d8c0198c0 2023-05-20T01:25:31.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.832+0000 7f0d9b3fb080 1 -- 172.21.15.16:0/3801786603 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:47 12.d 12:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e92) v8 -- 0x5571bc5a0fe0 con 0x7f0d7c060430 2023-05-20T01:25:31.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.832+0000 7f0d9b3fb080 1 -- 172.21.15.16:0/3801786603 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:48 12.1d 12:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e92) v8 -- 0x5571bc5a07e0 con 0x7f0d8c01b940 2023-05-20T01:25:31.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.834+0000 7f0d90cfa700 1 -- 172.21.15.16:0/3801786603 <== osd.5 v2:172.21.15.132:6816/1844689995 3 ==== osd_op_reply(38 1.00000000.inode [delete] v92'22 uv22 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f0d8401f040 con 0x7f0d7c009040 2023-05-20T01:25:31.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.834+0000 7f0d9b3fb080 1 -- 172.21.15.16:0/3801786603 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:49 12.13 12:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e92) v8 -- 0x5571bc5a0fe0 con 0x7f0d84015920 2023-05-20T01:25:31.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.834+0000 7f0d914fb700 1 -- 172.21.15.16:0/3801786603 <== osd.0 v2:172.21.15.16:6824/3923634179 8 ==== osd_op_reply(44 608.00000000 [delete] v92'3 uv3 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f0d7c075040 con 0x7f0d8c01b940 2023-05-20T01:25:31.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.834+0000 7f0d914fb700 1 -- 172.21.15.16:0/3801786603 <== osd.6 v2:172.21.15.132:6801/727581123 9 ==== osd_op_reply(45 604.00000000 [delete] v92'4 uv4 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f0d7c075040 con 0x7f0d8c0198c0 2023-05-20T01:25:31.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.835+0000 7f0d914fb700 1 -- 172.21.15.16:0/3801786603 <== osd.0 v2:172.21.15.16:6824/3923634179 9 ==== osd_op_reply(42 607.00000000 [delete] v92'3 uv3 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f0d7c075040 con 0x7f0d8c01b940 2023-05-20T01:25:31.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.835+0000 7f0d9b3fb080 1 -- 172.21.15.16:0/3801786603 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:50 12.13 12:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e92) v8 -- 0x5571bc5a07e0 con 0x7f0d84015920 2023-05-20T01:25:31.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.835+0000 7f0d9b3fb080 1 -- 172.21.15.16:0/3801786603 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:51 12.b 12:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e92) v8 -- 0x5571bc59f880 con 0x7f0d7c02f060 2023-05-20T01:25:31.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.835+0000 7f0d9b3fb080 1 -- 172.21.15.16:0/3801786603 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:52 12.7 12:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e92) v8 -- 0x5571bc5a1d00 con 0x7f0d84015920 2023-05-20T01:25:31.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.835+0000 7f0d914fb700 1 -- 172.21.15.16:0/3801786603 <== osd.0 v2:172.21.15.16:6824/3923634179 10 ==== osd_op_reply(48 605.00000000 [delete] v92'3 uv3 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f0d7c075040 con 0x7f0d8c01b940 2023-05-20T01:25:31.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.835+0000 7f0d9b3fb080 1 -- 172.21.15.16:0/3801786603 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:53 12.f 12:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e92) v8 -- 0x5571bc5a20f0 con 0x7f0d8c0198c0 2023-05-20T01:25:31.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.835+0000 7f0d914fb700 1 -- 172.21.15.16:0/3801786603 <== osd.0 v2:172.21.15.16:6824/3923634179 11 ==== osd_op_reply(33 601.00000000 [delete] v92'11 uv11 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f0d7c075040 con 0x7f0d8c01b940 2023-05-20T01:25:31.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.835+0000 7f0d914fb700 1 -- 172.21.15.16:0/3801786603 <== osd.0 v2:172.21.15.16:6824/3923634179 12 ==== osd_op_reply(34 602.00000000 [delete] v92'12 uv12 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f0d7c075040 con 0x7f0d8c01b940 2023-05-20T01:25:31.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.835+0000 7f0d914fb700 1 -- 172.21.15.16:0/3801786603 <== osd.0 v2:172.21.15.16:6824/3923634179 13 ==== osd_op_reply(35 10000000000.00000000 [delete] v92'13 uv13 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f0d7c075040 con 0x7f0d8c01b940 2023-05-20T01:25:31.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.835+0000 7f0d90cfa700 1 -- 172.21.15.16:0/3801786603 <== osd.5 v2:172.21.15.132:6816/1844689995 4 ==== osd_op_reply(39 200.00000000 [delete] v92'23 uv23 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f0d8401f040 con 0x7f0d7c009040 2023-05-20T01:25:31.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.835+0000 7f0d90cfa700 1 -- 172.21.15.16:0/3801786603 <== osd.5 v2:172.21.15.132:6816/1844689995 5 ==== osd_op_reply(40 200.00000001 [delete] v92'24 uv24 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f0d8401f040 con 0x7f0d7c009040 2023-05-20T01:25:31.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.835+0000 7f0d914fb700 1 -- 172.21.15.16:0/3801786603 <== osd.6 v2:172.21.15.132:6801/727581123 10 ==== osd_op_reply(46 500.00000000 [delete] v92'5 uv5 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f0d7c075040 con 0x7f0d8c0198c0 2023-05-20T01:25:31.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.835+0000 7f0d90cfa700 1 -- 172.21.15.16:0/3801786603 <== osd.5 v2:172.21.15.132:6816/1844689995 6 ==== osd_op_reply(41 606.00000000 [delete] v92'25 uv25 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f0d8401f040 con 0x7f0d7c009040 2023-05-20T01:25:31.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.835+0000 7f0d9b3fb080 1 -- 172.21.15.16:0/3801786603 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:54 12.1f 12:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e92) v8 -- 0x5571bc5a20f0 con 0x7f0d8c0198c0 2023-05-20T01:25:31.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.836+0000 7f0d9b3fb080 1 -- 172.21.15.16:0/3801786603 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:55 12.1f 12:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e92) v8 -- 0x5571bc59a370 con 0x7f0d8c0198c0 2023-05-20T01:25:31.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.836+0000 7f0d91cfc700 1 -- 172.21.15.16:0/3801786603 <== osd.1 v2:172.21.15.16:6808/3860264220 5 ==== osd_op_reply(43 mds0_openfiles.0 [delete] v92'3 uv3 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f0d8c03f040 con 0x7f0d84015920 2023-05-20T01:25:31.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.836+0000 7f0d914fb700 1 -- 172.21.15.16:0/3801786603 <== osd.7 v2:172.21.15.132:6800/466001149 2 ==== osd_op_reply(47 mds_snaptable [delete] v92'2 uv2 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f0d7c075040 con 0x7f0d7c060430 2023-05-20T01:25:31.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.837+0000 7f0d91cfc700 1 -- 172.21.15.16:0/3801786603 <== osd.1 v2:172.21.15.16:6808/3860264220 6 ==== osd_op_reply(49 mds0_inotable [delete] v92'4 uv4 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f0d8c03f040 con 0x7f0d84015920 2023-05-20T01:25:31.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.837+0000 7f0d90cfa700 1 -- 172.21.15.16:0/3801786603 <== osd.3 v2:172.21.15.16:6800/3669781890 4 ==== osd_op_reply(51 mds0_sessionmap [delete] v92'4 uv4 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f0d8401f040 con 0x7f0d7c02f060 2023-05-20T01:25:31.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.837+0000 7f0d91cfc700 1 -- 172.21.15.16:0/3801786603 <== osd.1 v2:172.21.15.16:6808/3860264220 7 ==== osd_op_reply(50 100.00000000 [delete] v92'5 uv5 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f0d8c03f040 con 0x7f0d84015920 2023-05-20T01:25:31.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.837+0000 7f0d914fb700 1 -- 172.21.15.16:0/3801786603 <== osd.6 v2:172.21.15.132:6801/727581123 11 ==== osd_op_reply(53 400.00000000 [delete] v92'2 uv2 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f0d7c075040 con 0x7f0d8c0198c0 2023-05-20T01:25:31.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.837+0000 7f0d914fb700 1 -- 172.21.15.16:0/3801786603 <== osd.0 v2:172.21.15.16:6824/3923634179 14 ==== osd_op_reply(36 600.00000000 [delete] v92'14 uv14 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f0d7c075040 con 0x7f0d8c01b940 2023-05-20T01:25:31.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.837+0000 7f0d914fb700 1 -- 172.21.15.16:0/3801786603 <== osd.6 v2:172.21.15.132:6801/727581123 12 ==== osd_op_reply(54 100.00000000.inode [delete] v92'7 uv7 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f0d7c075040 con 0x7f0d8c0198c0 2023-05-20T01:25:31.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.837+0000 7f0d914fb700 1 -- 172.21.15.16:0/3801786603 <== osd.6 v2:172.21.15.132:6801/727581123 13 ==== osd_op_reply(55 1.00000000 [delete] v92'8 uv8 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f0d7c075040 con 0x7f0d8c0198c0 2023-05-20T01:25:31.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.838+0000 7f0d914fb700 1 -- 172.21.15.16:0/3801786603 <== osd.0 v2:172.21.15.16:6824/3923634179 15 ==== osd_op_reply(37 603.00000000 [delete] v92'15 uv15 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f0d7c075040 con 0x7f0d8c01b940 2023-05-20T01:25:31.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.838+0000 7f0d91cfc700 1 -- 172.21.15.16:0/3801786603 <== osd.1 v2:172.21.15.16:6808/3860264220 8 ==== osd_op_reply(52 609.00000000 [delete] v92'3 uv3 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f0d8c03f040 con 0x7f0d84015920 2023-05-20T01:25:31.871 INFO:teuthology.orchestra.run.smithi016.stdout:Removed 23 objects 2023-05-20T01:25:31.872 INFO:teuthology.orchestra.run.smithi016.stdout:successfully purged pool cephfs_metadata 2023-05-20T01:25:31.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.838+0000 7f0d9b3fb080 1 -- 172.21.15.16:0/3801786603 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x7f0d8c01b940 msgr2=0x7f0d8c010410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:31.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.838+0000 7f0d9b3fb080 1 --2- 172.21.15.16:0/3801786603 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x7f0d8c01b940 0x7f0d8c010410 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:31.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.838+0000 7f0d9b3fb080 1 -- 172.21.15.16:0/3801786603 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x7f0d84015920 msgr2=0x7f0d84006400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:31.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.838+0000 7f0d9b3fb080 1 --2- 172.21.15.16:0/3801786603 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x7f0d84015920 0x7f0d84006400 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:31.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.838+0000 7f0d9b3fb080 1 -- 172.21.15.16:0/3801786603 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x7f0d84024900 msgr2=0x7f0d8402f3e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:31.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.838+0000 7f0d9b3fb080 1 --2- 172.21.15.16:0/3801786603 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x7f0d84024900 0x7f0d8402f3e0 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:31.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.838+0000 7f0d9b3fb080 1 -- 172.21.15.16:0/3801786603 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x7f0d7c02f060 msgr2=0x7f0d7c0574d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:31.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.838+0000 7f0d9b3fb080 1 --2- 172.21.15.16:0/3801786603 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x7f0d7c02f060 0x7f0d7c0574d0 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:31.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.838+0000 7f0d9b3fb080 1 -- 172.21.15.16:0/3801786603 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x5571bc597a50 msgr2=0x5571bc599e30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:31.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.838+0000 7f0d9b3fb080 1 --2- 172.21.15.16:0/3801786603 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x5571bc597a50 0x5571bc599e30 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:31.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.838+0000 7f0d91cfc700 1 -- 172.21.15.16:0/3801786603 reap_dead start 2023-05-20T01:25:31.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.838+0000 7f0d9b3fb080 1 -- 172.21.15.16:0/3801786603 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x7f0d7c009040 msgr2=0x7f0d7c058570 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:31.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.838+0000 7f0d9b3fb080 1 --2- 172.21.15.16:0/3801786603 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x7f0d7c009040 0x7f0d7c058570 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:31.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.838+0000 7f0d9b3fb080 1 -- 172.21.15.16:0/3801786603 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x7f0d8c0198c0 msgr2=0x7f0d8c0183a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:31.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.838+0000 7f0d9b3fb080 1 --2- 172.21.15.16:0/3801786603 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x7f0d8c0198c0 0x7f0d8c0183a0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:31.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.838+0000 7f0d9b3fb080 1 -- 172.21.15.16:0/3801786603 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x7f0d7c060430 msgr2=0x7f0d7c05c790 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:31.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.838+0000 7f0d91cfc700 1 -- 172.21.15.16:0/3801786603 reap_dead start 2023-05-20T01:25:31.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.838+0000 7f0d9b3fb080 1 --2- 172.21.15.16:0/3801786603 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x7f0d7c060430 0x7f0d7c05c790 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:31.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.838+0000 7f0d91cfc700 1 -- 172.21.15.16:0/3801786603 reap_dead start 2023-05-20T01:25:31.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.838+0000 7f0d9b3fb080 1 -- 172.21.15.16:0/3801786603 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f0d74069600 msgr2=0x7f0d7406bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:31.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.838+0000 7f0d9b3fb080 1 --2- 172.21.15.16:0/3801786603 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f0d74069600 0x7f0d7406bab0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x5571bc468020 tx=0x7f0d8400b040 comp rx=0 tx=0).stop 2023-05-20T01:25:31.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.838+0000 7f0d9b3fb080 1 -- 172.21.15.16:0/3801786603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571bc41f590 msgr2=0x5571bc4ea240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:31.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.838+0000 7f0d9b3fb080 1 --2- 172.21.15.16:0/3801786603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571bc41f590 0x5571bc4ea240 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c012000 tx=0x7f0d7c002fb0 comp rx=0 tx=0).stop 2023-05-20T01:25:31.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.838+0000 7f0d9b3fb080 1 -- 172.21.15.16:0/3801786603 shutdown_connections 2023-05-20T01:25:31.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.838+0000 7f0d9b3fb080 1 --2- 172.21.15.16:0/3801786603 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f0d74069600 0x7f0d7406bab0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:31.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.838+0000 7f0d9b3fb080 1 --2- 172.21.15.16:0/3801786603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571bc41f590 0x5571bc4ea240 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:31.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.838+0000 7f0d9b3fb080 1 -- 172.21.15.16:0/3801786603 >> 172.21.15.16:0/3801786603 conn(0x5571bc4670a0 msgr2=0x5571bc467570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:31.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.838+0000 7f0d9b3fb080 1 -- 172.21.15.16:0/3801786603 shutdown_connections 2023-05-20T01:25:31.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:31.838+0000 7f0d9b3fb080 1 -- 172.21.15.16:0/3801786603 wait complete. 2023-05-20T01:25:31.878 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs reset cephfs --yes-i-really-mean-it 2023-05-20T01:25:32.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.037+0000 7f5019172700 1 Processor -- start 2023-05-20T01:25:32.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.037+0000 7f5019172700 1 -- start start 2023-05-20T01:25:32.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.037+0000 7f5019172700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f50140cd400 0x7f50140ce1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:32.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.038+0000 7f5019172700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50140cb6c0 con 0x7f5014146710 2023-05-20T01:25:32.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.038+0000 7f5019172700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50140ce840 con 0x7f501406bd30 2023-05-20T01:25:32.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.038+0000 7f5019172700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50140cb910 con 0x7f50140cd400 2023-05-20T01:25:32.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.038+0000 7f5012d9d700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f5014146710 0x7f501406b620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:48210/0 (socket says 172.21.15.16:48210) 2023-05-20T01:25:32.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.038+0000 7f5012d9d700 1 -- 172.21.15.16:0/298202963 learned_addr learned my addr 172.21.15.16:0/298202963 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:25:32.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.038+0000 7f501359e700 1 --2- 172.21.15.16:0/298202963 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f50140cd400 0x7f50140ce1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:32.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.038+0000 7f5011d9b700 1 -- 172.21.15.16:0/298202963 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 559850478 0 0) 0x7f50140cb6c0 con 0x7f5014146710 2023-05-20T01:25:32.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.038+0000 7f5011d9b700 1 -- 172.21.15.16:0/298202963 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ffc0035b0 con 0x7f5014146710 2023-05-20T01:25:32.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.038+0000 7f5011d9b700 1 -- 172.21.15.16:0/298202963 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2214642245 0 0) 0x7f4ffc0035b0 con 0x7f5014146710 2023-05-20T01:25:32.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.038+0000 7f5011d9b700 1 -- 172.21.15.16:0/298202963 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f50140cd400 msgr2=0x7f50140ce1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:32.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.038+0000 7f5011d9b700 1 --2- 172.21.15.16:0/298202963 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f50140cd400 0x7f50140ce1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:32.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.038+0000 7f5011d9b700 1 -- 172.21.15.16:0/298202963 >> v1:172.21.15.132:6789/0 conn(0x7f501406bd30 legacy=0x7f50140cdad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:32.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.038+0000 7f5011d9b700 1 -- 172.21.15.16:0/298202963 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50140cbbf0 con 0x7f5014146710 2023-05-20T01:25:32.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.038+0000 7f5011d9b700 1 -- 172.21.15.16:0/298202963 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7f5004001670 con 0x7f5014146710 2023-05-20T01:25:32.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.038+0000 7f5011d9b700 1 -- 172.21.15.16:0/298202963 >> v1:172.21.15.16:6789/0 conn(0x7f5014146710 legacy=0x7f501406b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:32.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.038+0000 7f5011d9b700 1 --2- 172.21.15.16:0/298202963 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ffc003a30 0x7f4ffc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:32.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.038+0000 7f5011d9b700 1 --2- 172.21.15.16:0/298202963 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4ffc0043d0 0x7f4ffc004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:32.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.038+0000 7f5011d9b700 1 --2- 172.21.15.16:0/298202963 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4ffc004fb0 0x7f4ffc00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:32.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.038+0000 7f5011d9b700 1 -- 172.21.15.16:0/298202963 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50140cb910 con 0x7f4ffc003a30 2023-05-20T01:25:32.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.038+0000 7f5011d9b700 1 -- 172.21.15.16:0/298202963 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ffc0056c0 con 0x7f4ffc004fb0 2023-05-20T01:25:32.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.038+0000 7f5011d9b700 1 -- 172.21.15.16:0/298202963 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ffc00bf80 con 0x7f4ffc0043d0 2023-05-20T01:25:32.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.039+0000 7f5012d9d700 1 --2- 172.21.15.16:0/298202963 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ffc003a30 0x7f4ffc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:32.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.039+0000 7f501259c700 1 --2- 172.21.15.16:0/298202963 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4ffc0043d0 0x7f4ffc004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:32.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.039+0000 7f501359e700 1 --2- 172.21.15.16:0/298202963 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4ffc004fb0 0x7f4ffc00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:32.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.039+0000 7f5012d9d700 1 -- 172.21.15.16:0/298202963 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4ffc0043d0 msgr2=0x7f4ffc004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:32.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.039+0000 7f5012d9d700 1 --2- 172.21.15.16:0/298202963 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4ffc0043d0 0x7f4ffc004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:32.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.039+0000 7f5012d9d700 1 -- 172.21.15.16:0/298202963 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4ffc004fb0 msgr2=0x7f4ffc00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:32.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.039+0000 7f5012d9d700 1 --2- 172.21.15.16:0/298202963 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4ffc004fb0 0x7f4ffc00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:32.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.039+0000 7f5012d9d700 1 -- 172.21.15.16:0/298202963 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ffc0035b0 con 0x7f4ffc003a30 2023-05-20T01:25:32.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.039+0000 7f5012d9d700 1 --2- 172.21.15.16:0/298202963 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ffc003a30 0x7f4ffc003e00 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f500400c240 tx=0x7f4ffc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef2cac68ff2758d2 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:32.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.039+0000 7f5011d9b700 1 -- 172.21.15.16:0/298202963 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f500400cd20 con 0x7f4ffc003a30 2023-05-20T01:25:32.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.039+0000 7f5011d9b700 1 -- 172.21.15.16:0/298202963 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f500400ce80 con 0x7f4ffc003a30 2023-05-20T01:25:32.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.039+0000 7f5011d9b700 1 -- 172.21.15.16:0/298202963 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5004005ba0 con 0x7f4ffc003a30 2023-05-20T01:25:32.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.040+0000 7f5019172700 1 -- 172.21.15.16:0/298202963 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ffc003a30 msgr2=0x7f4ffc003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:32.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.040+0000 7f5019172700 1 --2- 172.21.15.16:0/298202963 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ffc003a30 0x7f4ffc003e00 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f500400c240 tx=0x7f4ffc005880 comp rx=0 tx=0).stop 2023-05-20T01:25:32.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.040+0000 7f501359e700 1 -- 172.21.15.16:0/298202963 reap_dead start 2023-05-20T01:25:32.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.040+0000 7f5019172700 1 -- 172.21.15.16:0/298202963 shutdown_connections 2023-05-20T01:25:32.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.040+0000 7f5019172700 1 -- 172.21.15.16:0/298202963 >> 172.21.15.16:0/298202963 conn(0x7f501406ae30 msgr2=0x7f501409a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:32.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.040+0000 7f5019172700 1 -- 172.21.15.16:0/298202963 shutdown_connections 2023-05-20T01:25:32.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.040+0000 7f5019172700 1 -- 172.21.15.16:0/298202963 wait complete. 2023-05-20T01:25:32.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.042+0000 7f5019172700 1 Processor -- start 2023-05-20T01:25:32.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.042+0000 7f5019172700 1 -- start start 2023-05-20T01:25:32.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.043+0000 7f5019172700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4ffc003a30 0x7f50141548c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:32.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.043+0000 7f5019172700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50140c55b0 0x7f50140c78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:32.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.043+0000 7f5019172700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f50140aed70 0x7f50140af210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:32.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.043+0000 7f5019172700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50140c5fe0 con 0x7f50140c55b0 2023-05-20T01:25:32.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.043+0000 7f5019172700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50140af750 con 0x7f4ffc003a30 2023-05-20T01:25:32.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.043+0000 7f5019172700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50140c7e30 con 0x7f50140aed70 2023-05-20T01:25:32.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.043+0000 7f501259c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50140c55b0 0x7f50140c78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:32.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.043+0000 7f501259c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50140c55b0 0x7f50140c78f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36696/0 (socket says 172.21.15.16:36696) 2023-05-20T01:25:32.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.043+0000 7f501259c700 1 -- 172.21.15.16:0/245370163 learned_addr learned my addr 172.21.15.16:0/245370163 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:32.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.043+0000 7f5012d9d700 1 --2- 172.21.15.16:0/245370163 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4ffc003a30 0x7f50141548c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:32.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.043+0000 7f501359e700 1 --2- 172.21.15.16:0/245370163 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f50140aed70 0x7f50140af210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:32.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.043+0000 7f501259c700 1 -- 172.21.15.16:0/245370163 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f50140aed70 msgr2=0x7f50140af210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:32.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.043+0000 7f501259c700 1 --2- 172.21.15.16:0/245370163 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f50140aed70 0x7f50140af210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:32.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.043+0000 7f501259c700 1 -- 172.21.15.16:0/245370163 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4ffc003a30 msgr2=0x7f50141548c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:32.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.043+0000 7f501259c700 1 --2- 172.21.15.16:0/245370163 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4ffc003a30 0x7f50141548c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:32.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.043+0000 7f501259c700 1 -- 172.21.15.16:0/245370163 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50140ce1e0 con 0x7f50140c55b0 2023-05-20T01:25:32.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.043+0000 7f501359e700 1 --2- 172.21.15.16:0/245370163 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f50140aed70 0x7f50140af210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:25:32.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.043+0000 7f5012d9d700 1 --2- 172.21.15.16:0/245370163 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4ffc003a30 0x7f50141548c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:25:32.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.043+0000 7f501259c700 1 --2- 172.21.15.16:0/245370163 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50140c55b0 0x7f50140c78f0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f500800f830 tx=0x7f500800fbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:32.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.043+0000 7f4ffb7fe700 1 -- 172.21.15.16:0/245370163 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50080052e0 con 0x7f50140c55b0 2023-05-20T01:25:32.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.043+0000 7f4ffb7fe700 1 -- 172.21.15.16:0/245370163 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5008005460 con 0x7f50140c55b0 2023-05-20T01:25:32.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.043+0000 7f5019172700 1 -- 172.21.15.16:0/245370163 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50140cbed0 con 0x7f50140c55b0 2023-05-20T01:25:32.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.044+0000 7f4ffb7fe700 1 -- 172.21.15.16:0/245370163 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50080038c0 con 0x7f50140c55b0 2023-05-20T01:25:32.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.044+0000 7f5019172700 1 -- 172.21.15.16:0/245370163 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50140ce370 con 0x7f50140c55b0 2023-05-20T01:25:32.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.045+0000 7f5019172700 1 -- 172.21.15.16:0/245370163 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50140747b0 con 0x7f50140c55b0 2023-05-20T01:25:32.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.045+0000 7f4ffb7fe700 1 -- 172.21.15.16:0/245370163 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5008003a20 con 0x7f50140c55b0 2023-05-20T01:25:32.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.045+0000 7f4ffb7fe700 1 --2- 172.21.15.16:0/245370163 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f500006ace0 0x7f500006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:32.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.045+0000 7f4ffb7fe700 1 -- 172.21.15.16:0/245370163 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f50080896e0 con 0x7f50140c55b0 2023-05-20T01:25:32.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.045+0000 7f5012d9d700 1 --2- 172.21.15.16:0/245370163 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f500006ace0 0x7f500006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:32.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.046+0000 7f5012d9d700 1 --2- 172.21.15.16:0/245370163 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f500006ace0 0x7f500006d190 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f50141f5810 tx=0x7f500400a000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:32.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.050+0000 7f4ffb7fe700 1 -- 172.21.15.16:0/245370163 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f5008054460 con 0x7f50140c55b0 2023-05-20T01:25:32.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.293+0000 7f5019172700 1 -- 172.21.15.16:0/245370163 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs reset", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f501406f7b0 con 0x7f50140c55b0 2023-05-20T01:25:32.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.526+0000 7f4ffb7fe700 1 -- 172.21.15.16:0/245370163 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs reset", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v60) v1 ==== 107+0+0 (secure 0 0 0) 0x7f5008057a80 con 0x7f50140c55b0 2023-05-20T01:25:32.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.529+0000 7f5019172700 1 -- 172.21.15.16:0/245370163 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f500006ace0 msgr2=0x7f500006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:32.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.529+0000 7f5019172700 1 --2- 172.21.15.16:0/245370163 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f500006ace0 0x7f500006d190 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f50141f5810 tx=0x7f500400a000 comp rx=0 tx=0).stop 2023-05-20T01:25:32.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.529+0000 7f5019172700 1 -- 172.21.15.16:0/245370163 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50140c55b0 msgr2=0x7f50140c78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:32.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.529+0000 7f5019172700 1 --2- 172.21.15.16:0/245370163 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50140c55b0 0x7f50140c78f0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f500800f830 tx=0x7f500800fbf0 comp rx=0 tx=0).stop 2023-05-20T01:25:32.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.529+0000 7f5019172700 1 -- 172.21.15.16:0/245370163 shutdown_connections 2023-05-20T01:25:32.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.529+0000 7f5019172700 1 --2- 172.21.15.16:0/245370163 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f500006ace0 0x7f500006d190 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:32.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.529+0000 7f5019172700 1 --2- 172.21.15.16:0/245370163 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f50140aed70 0x7f50140af210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:32.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.529+0000 7f5019172700 1 --2- 172.21.15.16:0/245370163 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4ffc003a30 0x7f50141548c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:32.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.529+0000 7f5019172700 1 --2- 172.21.15.16:0/245370163 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50140c55b0 0x7f50140c78f0 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:32.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.529+0000 7f5019172700 1 -- 172.21.15.16:0/245370163 >> 172.21.15.16:0/245370163 conn(0x7f501406ae30 msgr2=0x7f50140b3ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:32.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.529+0000 7f5019172700 1 -- 172.21.15.16:0/245370163 shutdown_connections 2023-05-20T01:25:32.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.529+0000 7f5019172700 1 -- 172.21.15.16:0/245370163 wait complete. 2023-05-20T01:25:32.536 INFO:tasks.ceph.mds.c.smithi016.stderr:2023-05-20T01:25:32.533+0000 7f7c5a0e4700 -1 mds.0.sessionmap _load_finish got (2) No such file or directory 2023-05-20T01:25:32.537 INFO:tasks.ceph.mds.c.smithi016.stderr:2023-05-20T01:25:32.533+0000 7f7c5a0e4700 -1 log_channel(cluster) log [ERR] : error reading sessionmap 'mds0_sessionmap' -2 ((2) No such file or directory) 2023-05-20T01:25:32.551 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-05-20T01:25:32.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.675+0000 7fda4bc01700 1 Processor -- start 2023-05-20T01:25:32.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.675+0000 7fda4bc01700 1 -- start start 2023-05-20T01:25:32.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.676+0000 7fda4bc01700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fda440cd1c0 0x7fda440710b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:32.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.676+0000 7fda4bc01700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fda440cd630 0x7fda440715f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:32.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.676+0000 7fda4bc01700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda440ccd30 con 0x7fda44159b50 2023-05-20T01:25:32.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.676+0000 7fda4bc01700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda440ce990 con 0x7fda440cd1c0 2023-05-20T01:25:32.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.676+0000 7fda4bc01700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda440cc6e0 con 0x7fda440cd630 2023-05-20T01:25:32.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.676+0000 7fda4919c700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fda440cd1c0 0x7fda440710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:32.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.676+0000 7fda4a19e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fda440cd630 0x7fda440715f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:32.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.676+0000 7fda4999d700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7fda44159b50 0x7fda4406e0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.16:57954/0 (socket says 172.21.15.16:57954) 2023-05-20T01:25:32.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.676+0000 7fda4999d700 1 -- 172.21.15.16:0/249539487 learned_addr learned my addr 172.21.15.16:0/249539487 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:25:32.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.677+0000 7fda4a19e700 1 -- 172.21.15.16:0/249539487 >> v2:172.21.15.132:3300/0 conn(0x7fda440cd1c0 msgr2=0x7fda440710b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:32.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.677+0000 7fda4a19e700 1 --2- 172.21.15.16:0/249539487 >> v2:172.21.15.132:3300/0 conn(0x7fda440cd1c0 0x7fda440710b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:32.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.677+0000 7fda4a19e700 1 -- 172.21.15.16:0/249539487 >> v1:172.21.15.132:6789/0 conn(0x7fda44159b50 legacy=0x7fda4406e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:32.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.677+0000 7fda4a19e700 1 -- 172.21.15.16:0/249539487 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda440cc9c0 con 0x7fda440cd630 2023-05-20T01:25:32.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.677+0000 7fda4919c700 1 --2- 172.21.15.16:0/249539487 >> v2:172.21.15.132:3300/0 conn(0x7fda440cd1c0 0x7fda440710b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-05-20T01:25:32.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.677+0000 7fda4a19e700 1 --2- 172.21.15.16:0/249539487 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fda440cd630 0x7fda440715f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fda3c00bfa0 tx=0x7fda3c00d330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc4e104f60ed3a6b server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:32.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.677+0000 7fda4899b700 1 -- 172.21.15.16:0/249539487 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda3c010070 con 0x7fda440cd630 2023-05-20T01:25:32.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.677+0000 7fda4899b700 1 -- 172.21.15.16:0/249539487 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda3c00b2e0 con 0x7fda440cd630 2023-05-20T01:25:32.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.677+0000 7fda4899b700 1 -- 172.21.15.16:0/249539487 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda3c00dc40 con 0x7fda440cd630 2023-05-20T01:25:32.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.678+0000 7fda4bc01700 1 -- 172.21.15.16:0/249539487 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fda440cd630 msgr2=0x7fda440715f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:32.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.678+0000 7fda4bc01700 1 --2- 172.21.15.16:0/249539487 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fda440cd630 0x7fda440715f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fda3c00bfa0 tx=0x7fda3c00d330 comp rx=0 tx=0).stop 2023-05-20T01:25:32.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.678+0000 7fda4bc01700 1 -- 172.21.15.16:0/249539487 shutdown_connections 2023-05-20T01:25:32.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.678+0000 7fda4bc01700 1 --2- 172.21.15.16:0/249539487 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fda440cd630 0x7fda440715f0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:32.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.678+0000 7fda4bc01700 1 --2- 172.21.15.16:0/249539487 >> v2:172.21.15.132:3300/0 conn(0x7fda440cd1c0 0x7fda440710b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:32.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.678+0000 7fda4bc01700 1 -- 172.21.15.16:0/249539487 >> 172.21.15.16:0/249539487 conn(0x7fda440cffd0 msgr2=0x7fda440d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:32.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.678+0000 7fda4bc01700 1 -- 172.21.15.16:0/249539487 shutdown_connections 2023-05-20T01:25:32.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.678+0000 7fda4bc01700 1 -- 172.21.15.16:0/249539487 wait complete. 2023-05-20T01:25:32.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.679+0000 7fda4bc01700 1 Processor -- start 2023-05-20T01:25:32.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.679+0000 7fda4bc01700 1 -- start start 2023-05-20T01:25:32.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.680+0000 7fda4bc01700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fda440cd1c0 0x7fda44151800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:32.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.680+0000 7fda4bc01700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda440cd630 0x7fda44150ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:32.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.680+0000 7fda4bc01700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fda44159b50 0x7fda440cb000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:32.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.680+0000 7fda4bc01700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda441e3e60 con 0x7fda440cd630 2023-05-20T01:25:32.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.680+0000 7fda4bc01700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda440c7410 con 0x7fda44159b50 2023-05-20T01:25:32.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.680+0000 7fda4bc01700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda440c76f0 con 0x7fda440cd1c0 2023-05-20T01:25:32.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.680+0000 7fda4919c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda440cd630 0x7fda44150ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:32.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.680+0000 7fda4919c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda440cd630 0x7fda44150ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36720/0 (socket says 172.21.15.16:36720) 2023-05-20T01:25:32.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.680+0000 7fda4919c700 1 -- 172.21.15.16:0/4072796801 learned_addr learned my addr 172.21.15.16:0/4072796801 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:32.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.680+0000 7fda4999d700 1 --2- 172.21.15.16:0/4072796801 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fda440cd1c0 0x7fda44151800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:32.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.680+0000 7fda4a19e700 1 --2- 172.21.15.16:0/4072796801 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fda44159b50 0x7fda440cb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:32.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.680+0000 7fda4919c700 1 -- 172.21.15.16:0/4072796801 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fda440cd1c0 msgr2=0x7fda44151800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:32.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.680+0000 7fda4919c700 1 --2- 172.21.15.16:0/4072796801 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fda440cd1c0 0x7fda44151800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:32.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.680+0000 7fda4919c700 1 -- 172.21.15.16:0/4072796801 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fda44159b50 msgr2=0x7fda440cb000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:32.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.680+0000 7fda4919c700 1 --2- 172.21.15.16:0/4072796801 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fda44159b50 0x7fda440cb000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:32.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.680+0000 7fda4919c700 1 -- 172.21.15.16:0/4072796801 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda400094a0 con 0x7fda440cd630 2023-05-20T01:25:32.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.681+0000 7fda4919c700 1 --2- 172.21.15.16:0/4072796801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda440cd630 0x7fda44150ad0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fda4000ebe0 tx=0x7fda4000eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:32.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.681+0000 7fda3a7fc700 1 -- 172.21.15.16:0/4072796801 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda4000c850 con 0x7fda440cd630 2023-05-20T01:25:32.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.681+0000 7fda3a7fc700 1 -- 172.21.15.16:0/4072796801 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda4000c9b0 con 0x7fda440cd630 2023-05-20T01:25:32.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.681+0000 7fda4bc01700 1 -- 172.21.15.16:0/4072796801 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda3c00bc80 con 0x7fda440cd630 2023-05-20T01:25:32.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.681+0000 7fda3a7fc700 1 -- 172.21.15.16:0/4072796801 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda4000cd40 con 0x7fda440cd630 2023-05-20T01:25:32.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.681+0000 7fda4bc01700 1 -- 172.21.15.16:0/4072796801 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda440cec70 con 0x7fda440cd630 2023-05-20T01:25:32.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.682+0000 7fda4bc01700 1 -- 172.21.15.16:0/4072796801 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda44071e80 con 0x7fda440cd630 2023-05-20T01:25:32.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.682+0000 7fda3a7fc700 1 -- 172.21.15.16:0/4072796801 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fda40013470 con 0x7fda440cd630 2023-05-20T01:25:32.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.683+0000 7fda3a7fc700 1 --2- 172.21.15.16:0/4072796801 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fda2806ace0 0x7fda2806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:32.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.683+0000 7fda3a7fc700 1 -- 172.21.15.16:0/4072796801 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6958+0+0 (secure 0 0 0) 0x7fda40089900 con 0x7fda440cd630 2023-05-20T01:25:32.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.683+0000 7fda4999d700 1 --2- 172.21.15.16:0/4072796801 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fda2806ace0 0x7fda2806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:32.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.683+0000 7fda4999d700 1 --2- 172.21.15.16:0/4072796801 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fda2806ace0 0x7fda2806d190 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fda34005960 tx=0x7fda34009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:32.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.687+0000 7fda3a7fc700 1 -- 172.21.15.16:0/4072796801 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fda40054640 con 0x7fda440cd630 2023-05-20T01:25:32.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:32.931+0000 7fda4bc01700 1 -- 172.21.15.16:0/4072796801 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7fda440cf550 con 0x7fda440cd630 2023-05-20T01:25:33.533 INFO:tasks.ceph.mon.a.smithi016.stderr:2023-05-20T01:25:33.529+0000 7f49eb8ae700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-05-20T01:25:33.533 INFO:tasks.ceph.mon.a.smithi016.stderr:2023-05-20T01:25:33.529+0000 7f49eb8ae700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2023-05-20T01:25:33.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:33.532+0000 7fda3a7fc700 1 -- 172.21.15.16:0/4072796801 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v62) v1 ==== 162+0+0 (secure 0 0 0) 0x7fda4000cea0 con 0x7fda440cd630 2023-05-20T01:25:33.537 INFO:tasks.ceph.mds.c.smithi016.stderr: -32> 2023-05-20T01:25:32.533+0000 7f7c5a0e4700 -1 mds.0.sessionmap _load_finish got (2) No such file or directory 2023-05-20T01:25:33.537 INFO:tasks.ceph.mds.c.smithi016.stderr: -31> 2023-05-20T01:25:32.533+0000 7f7c5a0e4700 -1 log_channel(cluster) log [ERR] : error reading sessionmap 'mds0_sessionmap' -2 ((2) No such file or directory) 2023-05-20T01:25:33.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:33.534+0000 7fda4bc01700 1 -- 172.21.15.16:0/4072796801 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fda2806ace0 msgr2=0x7fda2806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:33.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:33.534+0000 7fda4bc01700 1 --2- 172.21.15.16:0/4072796801 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fda2806ace0 0x7fda2806d190 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fda34005960 tx=0x7fda34009040 comp rx=0 tx=0).stop 2023-05-20T01:25:33.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:33.535+0000 7fda4bc01700 1 -- 172.21.15.16:0/4072796801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda440cd630 msgr2=0x7fda44150ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:33.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:33.535+0000 7fda4bc01700 1 --2- 172.21.15.16:0/4072796801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda440cd630 0x7fda44150ad0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fda4000ebe0 tx=0x7fda4000eef0 comp rx=0 tx=0).stop 2023-05-20T01:25:33.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:33.535+0000 7fda4bc01700 1 -- 172.21.15.16:0/4072796801 shutdown_connections 2023-05-20T01:25:33.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:33.535+0000 7fda4bc01700 1 --2- 172.21.15.16:0/4072796801 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fda44159b50 0x7fda440cb000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:33.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:33.535+0000 7fda4bc01700 1 --2- 172.21.15.16:0/4072796801 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fda2806ace0 0x7fda2806d190 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:33.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:33.535+0000 7fda4bc01700 1 --2- 172.21.15.16:0/4072796801 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fda440cd1c0 0x7fda44151800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:33.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:33.535+0000 7fda4bc01700 1 --2- 172.21.15.16:0/4072796801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda440cd630 0x7fda44150ad0 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:33.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:33.535+0000 7fda4bc01700 1 -- 172.21.15.16:0/4072796801 >> 172.21.15.16:0/4072796801 conn(0x7fda440cffd0 msgr2=0x7fda440b5210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:33.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:33.535+0000 7fda4bc01700 1 -- 172.21.15.16:0/4072796801 shutdown_connections 2023-05-20T01:25:33.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:33.535+0000 7fda4bc01700 1 -- 172.21.15.16:0/4072796801 wait complete. 2023-05-20T01:25:33.541 INFO:teuthology.orchestra.run.smithi016.stderr:cephfs marked joinable; MDS may join as newly active. 2023-05-20T01:25:33.556 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-20T01:25:33.563 INFO:tasks.ceph.mds.c.smithi016.stdout:starting mds.c at 2023-05-20T01:25:33.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:33.683+0000 7f6503770700 1 Processor -- start 2023-05-20T01:25:33.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:33.684+0000 7f6503770700 1 -- start start 2023-05-20T01:25:33.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:33.684+0000 7f6503770700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f64fc06ba90 0x7f64fc0cbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:33.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:33.684+0000 7f6503770700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64fc0ce3e0 con 0x7f64fc0d1000 2023-05-20T01:25:33.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:33.684+0000 7f6503770700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64fc0cdab0 con 0x7f64fc06b620 2023-05-20T01:25:33.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:33.684+0000 7f6503770700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64fc06bef0 con 0x7f64fc06ba90 2023-05-20T01:25:33.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:33.685+0000 7f650150c700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f64fc0d1000 0x7f64fc0d13d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:48266/0 (socket says 172.21.15.16:48266) 2023-05-20T01:25:33.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:33.685+0000 7f650150c700 1 -- 172.21.15.16:0/4189311995 learned_addr learned my addr 172.21.15.16:0/4189311995 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:25:33.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:33.685+0000 7f6501d0d700 1 --2- 172.21.15.16:0/4189311995 >> v2:172.21.15.16:3300/0 conn(0x7f64fc06ba90 0x7f64fc0cbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:33.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:33.685+0000 7f64f3fff700 1 -- 172.21.15.16:0/4189311995 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1330997375 0 0) 0x7f64fc0ce3e0 con 0x7f64fc0d1000 2023-05-20T01:25:33.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:33.685+0000 7f64f3fff700 1 -- 172.21.15.16:0/4189311995 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64e80035b0 con 0x7f64fc0d1000 2023-05-20T01:25:33.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:33.685+0000 7f6501d0d700 1 -- 172.21.15.16:0/4189311995 >> v1:172.21.15.132:6789/0 conn(0x7f64fc06b620 legacy=0x7f64fc0cb6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:33.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:33.685+0000 7f6501d0d700 1 -- 172.21.15.16:0/4189311995 >> v1:172.21.15.16:6789/0 conn(0x7f64fc0d1000 legacy=0x7f64fc0d13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:33.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:33.685+0000 7f6501d0d700 1 -- 172.21.15.16:0/4189311995 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64fc0cdd90 con 0x7f64fc06ba90 2023-05-20T01:25:33.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:33.685+0000 7f6501d0d700 1 --2- 172.21.15.16:0/4189311995 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64fc06ba90 0x7f64fc0cbdf0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f64f4009d90 tx=0x7f64f400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e870dda4f212137 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:33.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:33.685+0000 7f64f3fff700 1 -- 172.21.15.16:0/4189311995 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64f400e070 con 0x7f64fc06ba90 2023-05-20T01:25:33.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:33.685+0000 7f64f3fff700 1 -- 172.21.15.16:0/4189311995 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64f400bb20 con 0x7f64fc06ba90 2023-05-20T01:25:33.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:33.685+0000 7f64f3fff700 1 -- 172.21.15.16:0/4189311995 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64f4012430 con 0x7f64fc06ba90 2023-05-20T01:25:33.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:33.686+0000 7f6503770700 1 -- 172.21.15.16:0/4189311995 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64fc06ba90 msgr2=0x7f64fc0cbdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:33.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:33.686+0000 7f6503770700 1 --2- 172.21.15.16:0/4189311995 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64fc06ba90 0x7f64fc0cbdf0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f64f4009d90 tx=0x7f64f400b330 comp rx=0 tx=0).stop 2023-05-20T01:25:33.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:33.686+0000 7f6503770700 1 -- 172.21.15.16:0/4189311995 shutdown_connections 2023-05-20T01:25:33.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:33.686+0000 7f6503770700 1 --2- 172.21.15.16:0/4189311995 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64fc06ba90 0x7f64fc0cbdf0 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:33.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:33.686+0000 7f6503770700 1 -- 172.21.15.16:0/4189311995 >> 172.21.15.16:0/4189311995 conn(0x7f64fc06ae30 msgr2=0x7f64fc09a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:33.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:33.686+0000 7f6503770700 1 -- 172.21.15.16:0/4189311995 shutdown_connections 2023-05-20T01:25:33.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:33.686+0000 7f6503770700 1 -- 172.21.15.16:0/4189311995 wait complete. 2023-05-20T01:25:33.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:33.687+0000 7f6503770700 1 Processor -- start 2023-05-20T01:25:33.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:33.687+0000 7f6503770700 1 -- start start 2023-05-20T01:25:33.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:33.688+0000 7f6503770700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f64fc06b620 0x7f64fc15aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:33.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:33.688+0000 7f6503770700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f64fc06ba90 0x7f64fc09b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:33.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:33.688+0000 7f6503770700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64fc0d1000 0x7f64fc0b4f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:33.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:33.688+0000 7f6503770700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64fc15bb70 con 0x7f64fc0d1000 2023-05-20T01:25:33.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:33.688+0000 7f6503770700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64fc09bd10 con 0x7f64fc06ba90 2023-05-20T01:25:33.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:33.688+0000 7f6503770700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64fc09ebe0 con 0x7f64fc06b620 2023-05-20T01:25:33.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:33.688+0000 7f6501d0d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64fc0d1000 0x7f64fc0b4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:33.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:33.688+0000 7f6501d0d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64fc0d1000 0x7f64fc0b4f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36746/0 (socket says 172.21.15.16:36746) 2023-05-20T01:25:33.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:33.688+0000 7f6501d0d700 1 -- 172.21.15.16:0/220641772 learned_addr learned my addr 172.21.15.16:0/220641772 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:33.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:33.688+0000 7f6500d0b700 1 --2- 172.21.15.16:0/220641772 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f64fc06ba90 0x7f64fc09b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:33.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:33.688+0000 7f6501d0d700 1 -- 172.21.15.16:0/220641772 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f64fc06b620 msgr2=0x7f64fc15aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:33.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:33.688+0000 7f6501d0d700 1 --2- 172.21.15.16:0/220641772 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f64fc06b620 0x7f64fc15aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:33.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:33.688+0000 7f6501d0d700 1 -- 172.21.15.16:0/220641772 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f64fc06ba90 msgr2=0x7f64fc09b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:33.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:33.688+0000 7f6501d0d700 1 --2- 172.21.15.16:0/220641772 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f64fc06ba90 0x7f64fc09b7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:33.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:33.688+0000 7f6501d0d700 1 -- 172.21.15.16:0/220641772 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64f4009a70 con 0x7f64fc0d1000 2023-05-20T01:25:33.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:33.689+0000 7f6501d0d700 1 --2- 172.21.15.16:0/220641772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64fc0d1000 0x7f64fc0b4f10 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f64fc0cec10 tx=0x7f64f4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:33.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:33.689+0000 7f64f1ffb700 1 -- 172.21.15.16:0/220641772 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64f4012430 con 0x7f64fc0d1000 2023-05-20T01:25:33.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:33.689+0000 7f64f1ffb700 1 -- 172.21.15.16:0/220641772 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64f4020070 con 0x7f64fc0d1000 2023-05-20T01:25:33.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:33.689+0000 7f6503770700 1 -- 172.21.15.16:0/220641772 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64fc0b28b0 con 0x7f64fc0d1000 2023-05-20T01:25:33.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:33.689+0000 7f64f1ffb700 1 -- 172.21.15.16:0/220641772 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64f4004280 con 0x7f64fc0d1000 2023-05-20T01:25:33.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:33.689+0000 7f6503770700 1 -- 172.21.15.16:0/220641772 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64fc0ce070 con 0x7f64fc0d1000 2023-05-20T01:25:33.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:33.690+0000 7f6503770700 1 -- 172.21.15.16:0/220641772 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64fc0ce870 con 0x7f64fc0d1000 2023-05-20T01:25:33.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:33.691+0000 7f64f1ffb700 1 -- 172.21.15.16:0/220641772 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f64f40043e0 con 0x7f64fc0d1000 2023-05-20T01:25:33.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:33.691+0000 7f64f1ffb700 1 --2- 172.21.15.16:0/220641772 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f64e006ace0 0x7f64e006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:33.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:33.691+0000 7f64f1ffb700 1 -- 172.21.15.16:0/220641772 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6958+0+0 (secure 0 0 0) 0x7f64f4022070 con 0x7f64fc0d1000 2023-05-20T01:25:33.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:33.691+0000 7f650150c700 1 --2- 172.21.15.16:0/220641772 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f64e006ace0 0x7f64e006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:33.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:33.692+0000 7f650150c700 1 --2- 172.21.15.16:0/220641772 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f64e006ace0 0x7f64e006d190 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f64ec0016b0 tx=0x7f64ec0079c0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:33.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:33.695+0000 7f64f1ffb700 1 -- 172.21.15.16:0/220641772 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f64f40568c0 con 0x7f64fc0d1000 2023-05-20T01:25:33.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:33.939+0000 7f6503770700 1 -- 172.21.15.16:0/220641772 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f64fc0747b0 con 0x7f64fc0d1000 2023-05-20T01:25:33.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:33.940+0000 7f64f1ffb700 1 -- 172.21.15.16:0/220641772 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+3917 (secure 0 0 0) 0x7f64f4059ee0 con 0x7f64fc0d1000 2023-05-20T01:25:33.944 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:25:33.945 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":62,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6077,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/1241894038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":1241894038},{"type":"v1","addr":"172.21.15.132:6835","nonce":1241894038}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":6562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/3771738903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3771738903},{"type":"v1","addr":"172.21.15.132:6834","nonce":3771738903}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":7101,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/3876038711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":3876038711},{"type":"v1","addr":"172.21.15.16:6836","nonce":3876038711}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-20T01:25:32.295399+0000","modified":"2023-05-20T01:25:33.530206+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":93,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13,14],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-05-20T01:25:33.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:33.942+0000 7f6503770700 1 -- 172.21.15.16:0/220641772 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f64e006ace0 msgr2=0x7f64e006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:33.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:33.942+0000 7f6503770700 1 --2- 172.21.15.16:0/220641772 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f64e006ace0 0x7f64e006d190 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f64ec0016b0 tx=0x7f64ec0079c0 comp rx=0 tx=0).stop 2023-05-20T01:25:33.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:33.942+0000 7f6503770700 1 -- 172.21.15.16:0/220641772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64fc0d1000 msgr2=0x7f64fc0b4f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:33.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:33.942+0000 7f6503770700 1 --2- 172.21.15.16:0/220641772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64fc0d1000 0x7f64fc0b4f10 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f64fc0cec10 tx=0x7f64f4004ad0 comp rx=0 tx=0).stop 2023-05-20T01:25:33.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:33.942+0000 7f6503770700 1 -- 172.21.15.16:0/220641772 shutdown_connections 2023-05-20T01:25:33.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:33.942+0000 7f6503770700 1 --2- 172.21.15.16:0/220641772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64fc0d1000 0x7f64fc0b4f10 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:33.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:33.942+0000 7f6503770700 1 --2- 172.21.15.16:0/220641772 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f64e006ace0 0x7f64e006d190 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:33.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:33.942+0000 7f6503770700 1 --2- 172.21.15.16:0/220641772 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f64fc06b620 0x7f64fc15aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:33.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:33.942+0000 7f6503770700 1 --2- 172.21.15.16:0/220641772 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f64fc06ba90 0x7f64fc09b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:33.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:33.942+0000 7f6503770700 1 -- 172.21.15.16:0/220641772 >> 172.21.15.16:0/220641772 conn(0x7f64fc06ae30 msgr2=0x7f64fc0b0920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:33.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:33.942+0000 7f6503770700 1 -- 172.21.15.16:0/220641772 shutdown_connections 2023-05-20T01:25:33.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:33.942+0000 7f6503770700 1 -- 172.21.15.16:0/220641772 wait complete. 2023-05-20T01:25:33.948 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 62 2023-05-20T01:25:33.960 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-05-20T01:25:33.961 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-20T01:25:34.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.121+0000 7f395b368700 1 Processor -- start 2023-05-20T01:25:34.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.121+0000 7f395b368700 1 -- start start 2023-05-20T01:25:34.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.122+0000 7f395b368700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f395406d4b0 0x7f39540cf890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:34.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.122+0000 7f395b368700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39540cc510 con 0x7f395406a510 2023-05-20T01:25:34.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.122+0000 7f395b368700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3954071ed0 con 0x7f395406f450 2023-05-20T01:25:34.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.122+0000 7f395b368700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39540721b0 con 0x7f395406d4b0 2023-05-20T01:25:34.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.122+0000 7f3958903700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f395406d4b0 0x7f39540cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:34.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.122+0000 7f3959905700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f395406a510 0x7f39540cc9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:48282/0 (socket says 172.21.15.16:48282) 2023-05-20T01:25:34.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.122+0000 7f3959905700 1 -- 172.21.15.16:0/2990784865 learned_addr learned my addr 172.21.15.16:0/2990784865 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:25:34.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.122+0000 7f3958903700 1 -- 172.21.15.16:0/2990784865 >> v1:172.21.15.132:6789/0 conn(0x7f395406f450 legacy=0x7f3954161080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:34.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.122+0000 7f3958903700 1 -- 172.21.15.16:0/2990784865 >> v1:172.21.15.16:6789/0 conn(0x7f395406a510 legacy=0x7f39540cc9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:34.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.122+0000 7f3958903700 1 -- 172.21.15.16:0/2990784865 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39540cc1a0 con 0x7f395406d4b0 2023-05-20T01:25:34.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.122+0000 7f394bfff700 1 -- 172.21.15.16:0/2990784865 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 570084542 0 0) 0x7f39540cc510 con 0x7f395406a510 2023-05-20T01:25:34.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.122+0000 7f3958903700 1 --2- 172.21.15.16:0/2990784865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f395406d4b0 0x7f39540cf890 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f3944009840 tx=0x7f3944009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ac7c3d3f6f57aff server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:34.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.122+0000 7f394bfff700 1 -- 172.21.15.16:0/2990784865 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3944004230 con 0x7f395406d4b0 2023-05-20T01:25:34.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.123+0000 7f394bfff700 1 -- 172.21.15.16:0/2990784865 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3944004390 con 0x7f395406d4b0 2023-05-20T01:25:34.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.123+0000 7f394bfff700 1 -- 172.21.15.16:0/2990784865 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39440036a0 con 0x7f395406d4b0 2023-05-20T01:25:34.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.123+0000 7f395b368700 1 -- 172.21.15.16:0/2990784865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f395406d4b0 msgr2=0x7f39540cf890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:34.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.123+0000 7f395b368700 1 --2- 172.21.15.16:0/2990784865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f395406d4b0 0x7f39540cf890 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f3944009840 tx=0x7f3944009b50 comp rx=0 tx=0).stop 2023-05-20T01:25:34.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.123+0000 7f395b368700 1 -- 172.21.15.16:0/2990784865 shutdown_connections 2023-05-20T01:25:34.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.123+0000 7f395b368700 1 --2- 172.21.15.16:0/2990784865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f395406d4b0 0x7f39540cf890 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:34.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.123+0000 7f395b368700 1 -- 172.21.15.16:0/2990784865 >> 172.21.15.16:0/2990784865 conn(0x7f39540cffb0 msgr2=0x7f39540d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:34.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.123+0000 7f395b368700 1 -- 172.21.15.16:0/2990784865 shutdown_connections 2023-05-20T01:25:34.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.123+0000 7f395b368700 1 -- 172.21.15.16:0/2990784865 wait complete. 2023-05-20T01:25:34.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.125+0000 7f395b368700 1 Processor -- start 2023-05-20T01:25:34.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.125+0000 7f395b368700 1 -- start start 2023-05-20T01:25:34.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.126+0000 7f395b368700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f395406a510 0x7f3954148360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:34.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.126+0000 7f395b368700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f395406d4b0 0x7f39540b4130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:34.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.126+0000 7f395b368700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f395406f450 0x7f3954148e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:34.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.126+0000 7f395b368700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f395414bd80 con 0x7f395406d4b0 2023-05-20T01:25:34.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.126+0000 7f395b368700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39540b2e40 con 0x7f395406a510 2023-05-20T01:25:34.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.126+0000 7f395b368700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39540b3ca0 con 0x7f395406f450 2023-05-20T01:25:34.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.126+0000 7f3958903700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f395406d4b0 0x7f39540b4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:34.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.126+0000 7f3958903700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f395406d4b0 0x7f39540b4130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36758/0 (socket says 172.21.15.16:36758) 2023-05-20T01:25:34.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.126+0000 7f3958903700 1 -- 172.21.15.16:0/2149034177 learned_addr learned my addr 172.21.15.16:0/2149034177 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:34.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.126+0000 7f3959104700 1 --2- 172.21.15.16:0/2149034177 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f395406a510 0x7f3954148360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:34.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.126+0000 7f3959905700 1 --2- 172.21.15.16:0/2149034177 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f395406f450 0x7f3954148e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:34.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.126+0000 7f3958903700 1 -- 172.21.15.16:0/2149034177 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f395406f450 msgr2=0x7f3954148e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:34.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.126+0000 7f3958903700 1 --2- 172.21.15.16:0/2149034177 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f395406f450 0x7f3954148e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:34.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.126+0000 7f3958903700 1 -- 172.21.15.16:0/2149034177 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f395406a510 msgr2=0x7f3954148360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:34.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.126+0000 7f3958903700 1 --2- 172.21.15.16:0/2149034177 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f395406a510 0x7f3954148360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:34.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.126+0000 7f3958903700 1 -- 172.21.15.16:0/2149034177 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39440094a0 con 0x7f395406d4b0 2023-05-20T01:25:34.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.126+0000 7f3958903700 1 --2- 172.21.15.16:0/2149034177 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f395406d4b0 0x7f39540b4130 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f3944009b90 tx=0x7f3944004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:34.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.127+0000 7f3949ffb700 1 -- 172.21.15.16:0/2149034177 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3944011630 con 0x7f395406d4b0 2023-05-20T01:25:34.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.127+0000 7f3949ffb700 1 -- 172.21.15.16:0/2149034177 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f394401e070 con 0x7f395406d4b0 2023-05-20T01:25:34.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.127+0000 7f395b368700 1 -- 172.21.15.16:0/2149034177 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f395406a950 con 0x7f395406d4b0 2023-05-20T01:25:34.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.127+0000 7f3949ffb700 1 -- 172.21.15.16:0/2149034177 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39440119b0 con 0x7f395406d4b0 2023-05-20T01:25:34.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.127+0000 7f395b368700 1 -- 172.21.15.16:0/2149034177 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39540cbe50 con 0x7f395406d4b0 2023-05-20T01:25:34.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.128+0000 7f395b368700 1 -- 172.21.15.16:0/2149034177 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f395406aae0 con 0x7f395406d4b0 2023-05-20T01:25:34.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.128+0000 7f3949ffb700 1 -- 172.21.15.16:0/2149034177 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3944011b10 con 0x7f395406d4b0 2023-05-20T01:25:34.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.129+0000 7f3949ffb700 1 --2- 172.21.15.16:0/2149034177 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f393806ace0 0x7f393806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:34.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.129+0000 7f3949ffb700 1 -- 172.21.15.16:0/2149034177 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6958+0+0 (secure 0 0 0) 0x7f394408ac10 con 0x7f395406d4b0 2023-05-20T01:25:34.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.129+0000 7f3959104700 1 --2- 172.21.15.16:0/2149034177 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f393806ace0 0x7f393806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:34.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.129+0000 7f3959104700 1 --2- 172.21.15.16:0/2149034177 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f393806ace0 0x7f393806d190 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f3950009870 tx=0x7f3950008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:34.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.133+0000 7f3949ffb700 1 -- 172.21.15.16:0/2149034177 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f39440559d0 con 0x7f395406d4b0 2023-05-20T01:25:34.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.373+0000 7f395b368700 1 -- 172.21.15.16:0/2149034177 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f39540b3e30 con 0x7f395406d4b0 2023-05-20T01:25:34.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.374+0000 7f3949ffb700 1 -- 172.21.15.16:0/2149034177 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+3917 (secure 0 0 0) 0x7f3944020090 con 0x7f395406d4b0 2023-05-20T01:25:34.379 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:25:34.379 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":62,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6077,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/1241894038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":1241894038},{"type":"v1","addr":"172.21.15.132:6835","nonce":1241894038}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":6562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/3771738903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3771738903},{"type":"v1","addr":"172.21.15.132:6834","nonce":3771738903}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":7101,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/3876038711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":3876038711},{"type":"v1","addr":"172.21.15.16:6836","nonce":3876038711}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-20T01:25:32.295399+0000","modified":"2023-05-20T01:25:33.530206+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":93,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13,14],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-05-20T01:25:34.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.376+0000 7f395b368700 1 -- 172.21.15.16:0/2149034177 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f393806ace0 msgr2=0x7f393806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:34.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.376+0000 7f395b368700 1 --2- 172.21.15.16:0/2149034177 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f393806ace0 0x7f393806d190 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f3950009870 tx=0x7f3950008040 comp rx=0 tx=0).stop 2023-05-20T01:25:34.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.376+0000 7f395b368700 1 -- 172.21.15.16:0/2149034177 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f395406d4b0 msgr2=0x7f39540b4130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:34.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.376+0000 7f395b368700 1 --2- 172.21.15.16:0/2149034177 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f395406d4b0 0x7f39540b4130 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f3944009b90 tx=0x7f3944004750 comp rx=0 tx=0).stop 2023-05-20T01:25:34.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.376+0000 7f395b368700 1 -- 172.21.15.16:0/2149034177 shutdown_connections 2023-05-20T01:25:34.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.376+0000 7f395b368700 1 --2- 172.21.15.16:0/2149034177 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f393806ace0 0x7f393806d190 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:34.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.376+0000 7f395b368700 1 --2- 172.21.15.16:0/2149034177 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f395406f450 0x7f3954148e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:34.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.376+0000 7f395b368700 1 --2- 172.21.15.16:0/2149034177 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f395406a510 0x7f3954148360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:34.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.376+0000 7f395b368700 1 --2- 172.21.15.16:0/2149034177 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f395406d4b0 0x7f39540b4130 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:34.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.376+0000 7f395b368700 1 -- 172.21.15.16:0/2149034177 >> 172.21.15.16:0/2149034177 conn(0x7f39540cffb0 msgr2=0x7f3954159500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:34.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.376+0000 7f395b368700 1 -- 172.21.15.16:0/2149034177 shutdown_connections 2023-05-20T01:25:34.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.376+0000 7f395b368700 1 -- 172.21.15.16:0/2149034177 wait complete. 2023-05-20T01:25:34.382 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 62 2023-05-20T01:25:34.393 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-20T01:25:34.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.514+0000 7fd11335c700 1 Processor -- start 2023-05-20T01:25:34.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.514+0000 7fd11335c700 1 -- start start 2023-05-20T01:25:34.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.515+0000 7fd11335c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd10c06ac20 0x7fd10c0cf890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:34.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.515+0000 7fd11335c700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd10c0cc510 con 0x7fd10c06f450 2023-05-20T01:25:34.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.515+0000 7fd11335c700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd10c071ed0 con 0x7fd10c06d4b0 2023-05-20T01:25:34.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.515+0000 7fd11335c700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd10c0721b0 con 0x7fd10c06ac20 2023-05-20T01:25:34.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.515+0000 7fd1110f8700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fd10c06f450 0x7fd10c161080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:48292/0 (socket says 172.21.15.16:48292) 2023-05-20T01:25:34.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.515+0000 7fd1110f8700 1 -- 172.21.15.16:0/1860625502 learned_addr learned my addr 172.21.15.16:0/1860625502 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:25:34.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.515+0000 7fd1118f9700 1 --2- 172.21.15.16:0/1860625502 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd10c06ac20 0x7fd10c0cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:34.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.515+0000 7fd0fffff700 1 -- 172.21.15.16:0/1860625502 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 323347614 0 0) 0x7fd10c0cc510 con 0x7fd10c06f450 2023-05-20T01:25:34.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.515+0000 7fd0fffff700 1 -- 172.21.15.16:0/1860625502 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd0f80035b0 con 0x7fd10c06f450 2023-05-20T01:25:34.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.515+0000 7fd0fffff700 1 -- 172.21.15.16:0/1860625502 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1134071088 0 0) 0x7fd10c071ed0 con 0x7fd10c06d4b0 2023-05-20T01:25:34.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.515+0000 7fd0fffff700 1 -- 172.21.15.16:0/1860625502 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd10c0cc510 con 0x7fd10c06d4b0 2023-05-20T01:25:34.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.516+0000 7fd0fffff700 1 -- 172.21.15.16:0/1860625502 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3312470006 0 0) 0x7fd0f80035b0 con 0x7fd10c06f450 2023-05-20T01:25:34.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.516+0000 7fd1118f9700 1 -- 172.21.15.16:0/1860625502 >> v1:172.21.15.132:6789/0 conn(0x7fd10c06d4b0 legacy=0x7fd10c06a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:34.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.516+0000 7fd1118f9700 1 -- 172.21.15.16:0/1860625502 >> v1:172.21.15.16:6789/0 conn(0x7fd10c06f450 legacy=0x7fd10c161080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:34.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.516+0000 7fd1118f9700 1 -- 172.21.15.16:0/1860625502 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd10c0cc1a0 con 0x7fd10c06ac20 2023-05-20T01:25:34.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.516+0000 7fd1118f9700 1 --2- 172.21.15.16:0/1860625502 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd10c06ac20 0x7fd10c0cf890 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd104009dc0 tx=0x7fd10400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df4f2bce90510c63 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:34.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.516+0000 7fd0fffff700 1 -- 172.21.15.16:0/1860625502 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd10400e070 con 0x7fd10c06ac20 2023-05-20T01:25:34.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.516+0000 7fd0fffff700 1 -- 172.21.15.16:0/1860625502 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd104003a20 con 0x7fd10c06ac20 2023-05-20T01:25:34.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.516+0000 7fd0fffff700 1 -- 172.21.15.16:0/1860625502 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd100001670 con 0x7fd10c06ac20 2023-05-20T01:25:34.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.516+0000 7fd11335c700 1 -- 172.21.15.16:0/1860625502 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd10c06ac20 msgr2=0x7fd10c0cf890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:34.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.516+0000 7fd11335c700 1 --2- 172.21.15.16:0/1860625502 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd10c06ac20 0x7fd10c0cf890 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd104009dc0 tx=0x7fd10400b330 comp rx=0 tx=0).stop 2023-05-20T01:25:34.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.517+0000 7fd11335c700 1 -- 172.21.15.16:0/1860625502 shutdown_connections 2023-05-20T01:25:34.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.517+0000 7fd11335c700 1 --2- 172.21.15.16:0/1860625502 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd10c06ac20 0x7fd10c0cf890 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:34.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.517+0000 7fd11335c700 1 -- 172.21.15.16:0/1860625502 >> 172.21.15.16:0/1860625502 conn(0x7fd10c0cffb0 msgr2=0x7fd10c0d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:34.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.517+0000 7fd11335c700 1 -- 172.21.15.16:0/1860625502 shutdown_connections 2023-05-20T01:25:34.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.517+0000 7fd11335c700 1 -- 172.21.15.16:0/1860625502 wait complete. 2023-05-20T01:25:34.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.518+0000 7fd11335c700 1 Processor -- start 2023-05-20T01:25:34.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.518+0000 7fd11335c700 1 -- start start 2023-05-20T01:25:34.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.518+0000 7fd11335c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd10c06ac20 0x7fd10c1502d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:34.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.519+0000 7fd11335c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd10c06d4b0 0x7fd10c14fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:34.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.519+0000 7fd11335c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd10c06f450 0x7fd10c0c5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:34.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.519+0000 7fd11335c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd10c0c4f10 con 0x7fd10c06d4b0 2023-05-20T01:25:34.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.519+0000 7fd11335c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd10c1510b0 con 0x7fd10c06ac20 2023-05-20T01:25:34.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.519+0000 7fd11335c700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd10c150940 con 0x7fd10c06f450 2023-05-20T01:25:34.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.519+0000 7fd1108f7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd10c06d4b0 0x7fd10c14fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:34.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.519+0000 7fd1108f7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd10c06d4b0 0x7fd10c14fb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36772/0 (socket says 172.21.15.16:36772) 2023-05-20T01:25:34.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.519+0000 7fd1108f7700 1 -- 172.21.15.16:0/2935507550 learned_addr learned my addr 172.21.15.16:0/2935507550 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:34.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.519+0000 7fd1108f7700 1 -- 172.21.15.16:0/2935507550 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd10c06f450 msgr2=0x7fd10c0c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:34.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.519+0000 7fd1108f7700 1 --2- 172.21.15.16:0/2935507550 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd10c06f450 0x7fd10c0c5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:34.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.519+0000 7fd1108f7700 1 -- 172.21.15.16:0/2935507550 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd10c06ac20 msgr2=0x7fd10c1502d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:34.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.519+0000 7fd1108f7700 1 --2- 172.21.15.16:0/2935507550 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd10c06ac20 0x7fd10c1502d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:34.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.519+0000 7fd1108f7700 1 -- 172.21.15.16:0/2935507550 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd104009a70 con 0x7fd10c06d4b0 2023-05-20T01:25:34.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.519+0000 7fd1108f7700 1 --2- 172.21.15.16:0/2935507550 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd10c06d4b0 0x7fd10c14fb00 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fd10800c550 tx=0x7fd10800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:34.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.519+0000 7fd0fdffb700 1 -- 172.21.15.16:0/2935507550 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1080052d0 con 0x7fd10c06d4b0 2023-05-20T01:25:34.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.519+0000 7fd11335c700 1 -- 172.21.15.16:0/2935507550 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd10c167a00 con 0x7fd10c06d4b0 2023-05-20T01:25:34.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.519+0000 7fd0fdffb700 1 -- 172.21.15.16:0/2935507550 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd108005430 con 0x7fd10c06d4b0 2023-05-20T01:25:34.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.519+0000 7fd0fdffb700 1 -- 172.21.15.16:0/2935507550 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1080038c0 con 0x7fd10c06d4b0 2023-05-20T01:25:34.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.519+0000 7fd11335c700 1 -- 172.21.15.16:0/2935507550 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd10c0cbe50 con 0x7fd10c06d4b0 2023-05-20T01:25:34.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.520+0000 7fd11335c700 1 -- 172.21.15.16:0/2935507550 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd10c167b90 con 0x7fd10c06d4b0 2023-05-20T01:25:34.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.521+0000 7fd0fdffb700 1 -- 172.21.15.16:0/2935507550 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd108003a20 con 0x7fd10c06d4b0 2023-05-20T01:25:34.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.521+0000 7fd0fdffb700 1 --2- 172.21.15.16:0/2935507550 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd0f006ace0 0x7fd0f006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:34.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.521+0000 7fd0fdffb700 1 -- 172.21.15.16:0/2935507550 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6958+0+0 (secure 0 0 0) 0x7fd108088d70 con 0x7fd10c06d4b0 2023-05-20T01:25:34.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.521+0000 7fd1110f8700 1 --2- 172.21.15.16:0/2935507550 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd0f006ace0 0x7fd0f006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:34.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.522+0000 7fd1110f8700 1 --2- 172.21.15.16:0/2935507550 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd0f006ace0 0x7fd0f006d190 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fd1000017a0 tx=0x7fd100001270 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:34.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.526+0000 7fd0fdffb700 1 -- 172.21.15.16:0/2935507550 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fd108053ab0 con 0x7fd10c06d4b0 2023-05-20T01:25:34.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.770+0000 7fd11335c700 1 -- 172.21.15.16:0/2935507550 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd10c0c5050 con 0x7fd10c06d4b0 2023-05-20T01:25:34.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.770+0000 7fd0fdffb700 1 -- 172.21.15.16:0/2935507550 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+4674 (secure 0 0 0) 0x7fd10c0c5050 con 0x7fd10c06d4b0 2023-05-20T01:25:34.775 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:25:34.775 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":63,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6077,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/1241894038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":1241894038},{"type":"v1","addr":"172.21.15.132:6835","nonce":1241894038}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":6562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/3771738903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3771738903},{"type":"v1","addr":"172.21.15.132:6834","nonce":3771738903}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":7101,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/3876038711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":3876038711},{"type":"v1","addr":"172.21.15.16:6836","nonce":3876038711}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":7410,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2740585996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2740585996},{"type":"v1","addr":"172.21.15.16:6837","nonce":2740585996}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-20T01:25:32.295399+0000","modified":"2023-05-20T01:25:33.530206+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":93,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13,14],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-05-20T01:25:34.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.772+0000 7fd11335c700 1 -- 172.21.15.16:0/2935507550 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd0f006ace0 msgr2=0x7fd0f006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:34.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.772+0000 7fd11335c700 1 --2- 172.21.15.16:0/2935507550 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd0f006ace0 0x7fd0f006d190 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fd1000017a0 tx=0x7fd100001270 comp rx=0 tx=0).stop 2023-05-20T01:25:34.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.772+0000 7fd11335c700 1 -- 172.21.15.16:0/2935507550 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd10c06d4b0 msgr2=0x7fd10c14fb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:34.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.772+0000 7fd11335c700 1 --2- 172.21.15.16:0/2935507550 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd10c06d4b0 0x7fd10c14fb00 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fd10800c550 tx=0x7fd10800c910 comp rx=0 tx=0).stop 2023-05-20T01:25:34.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.772+0000 7fd11335c700 1 -- 172.21.15.16:0/2935507550 shutdown_connections 2023-05-20T01:25:34.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.772+0000 7fd11335c700 1 --2- 172.21.15.16:0/2935507550 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd0f006ace0 0x7fd0f006d190 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:34.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.772+0000 7fd11335c700 1 --2- 172.21.15.16:0/2935507550 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd10c06f450 0x7fd10c0c5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:34.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.772+0000 7fd11335c700 1 --2- 172.21.15.16:0/2935507550 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd10c06ac20 0x7fd10c1502d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:34.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.772+0000 7fd11335c700 1 --2- 172.21.15.16:0/2935507550 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd10c06d4b0 0x7fd10c14fb00 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:34.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.772+0000 7fd11335c700 1 -- 172.21.15.16:0/2935507550 >> 172.21.15.16:0/2935507550 conn(0x7fd10c0cffb0 msgr2=0x7fd10c0d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:34.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.772+0000 7fd11335c700 1 -- 172.21.15.16:0/2935507550 shutdown_connections 2023-05-20T01:25:34.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.772+0000 7fd11335c700 1 -- 172.21.15.16:0/2935507550 wait complete. 2023-05-20T01:25:34.779 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 63 2023-05-20T01:25:34.789 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-20T01:25:34.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.910+0000 7fc9af456700 1 Processor -- start 2023-05-20T01:25:34.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.911+0000 7fc9af456700 1 -- start start 2023-05-20T01:25:34.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.911+0000 7fc9af456700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fc9a80cfc50 0x7fc9a80d2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:34.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.911+0000 7fc9af456700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc9a80b4750 0x7fc9a80d0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:34.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.911+0000 7fc9af456700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9a80d0b90 con 0x7fc9a80b2b40 2023-05-20T01:25:34.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.911+0000 7fc9af456700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9a80cfad0 con 0x7fc9a80b4750 2023-05-20T01:25:34.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.911+0000 7fc9af456700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9a80d3df0 con 0x7fc9a80cfc50 2023-05-20T01:25:34.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.911+0000 7fc9ac9f1700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc9a80b4750 0x7fc9a80d0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:34.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.911+0000 7fc9ad9f3700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fc9a80b2b40 0x7fc9a80b2f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:48312/0 (socket says 172.21.15.16:48312) 2023-05-20T01:25:34.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.911+0000 7fc9ad9f3700 1 -- 172.21.15.16:0/621674366 learned_addr learned my addr 172.21.15.16:0/621674366 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:25:34.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.912+0000 7fc9ad1f2700 1 --2- 172.21.15.16:0/621674366 >> v2:172.21.15.132:3300/0 conn(0x7fc9a80cfc50 0x7fc9a80d2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:34.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.912+0000 7fc9ac9f1700 1 -- 172.21.15.16:0/621674366 >> v2:172.21.15.132:3300/0 conn(0x7fc9a80cfc50 msgr2=0x7fc9a80d2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:34.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.912+0000 7fc9ac9f1700 1 --2- 172.21.15.16:0/621674366 >> v2:172.21.15.132:3300/0 conn(0x7fc9a80cfc50 0x7fc9a80d2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:34.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.912+0000 7fc9ac9f1700 1 -- 172.21.15.16:0/621674366 >> v1:172.21.15.16:6789/0 conn(0x7fc9a80b2b40 legacy=0x7fc9a80b2f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:34.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.912+0000 7fc9ac9f1700 1 -- 172.21.15.16:0/621674366 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9a80d0560 con 0x7fc9a80b4750 2023-05-20T01:25:34.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.912+0000 7fc99ffff700 1 -- 172.21.15.16:0/621674366 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2175292285 0 0) 0x7fc9a80d0b90 con 0x7fc9a80b2b40 2023-05-20T01:25:34.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.912+0000 7fc9ad1f2700 1 --2- 172.21.15.16:0/621674366 >> v2:172.21.15.132:3300/0 conn(0x7fc9a80cfc50 0x7fc9a80d2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:25:34.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.912+0000 7fc9ac9f1700 1 --2- 172.21.15.16:0/621674366 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9a80b4750 0x7fc9a80d0020 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7fc9a80b5de0 tx=0x7fc9a4009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9db662d06b4baf16 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:34.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.912+0000 7fc99ffff700 1 -- 172.21.15.16:0/621674366 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9a4011830 con 0x7fc9a80b4750 2023-05-20T01:25:34.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.912+0000 7fc99ffff700 1 -- 172.21.15.16:0/621674366 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9a4011990 con 0x7fc9a80b4750 2023-05-20T01:25:34.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.912+0000 7fc99ffff700 1 -- 172.21.15.16:0/621674366 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9a4011d20 con 0x7fc9a80b4750 2023-05-20T01:25:34.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.913+0000 7fc9af456700 1 -- 172.21.15.16:0/621674366 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9a80b4750 msgr2=0x7fc9a80d0020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:34.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.913+0000 7fc9af456700 1 --2- 172.21.15.16:0/621674366 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9a80b4750 0x7fc9a80d0020 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7fc9a80b5de0 tx=0x7fc9a4009b20 comp rx=0 tx=0).stop 2023-05-20T01:25:34.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.913+0000 7fc9af456700 1 -- 172.21.15.16:0/621674366 shutdown_connections 2023-05-20T01:25:34.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.913+0000 7fc9af456700 1 --2- 172.21.15.16:0/621674366 >> v2:172.21.15.132:3300/0 conn(0x7fc9a80cfc50 0x7fc9a80d2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:34.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.913+0000 7fc9af456700 1 --2- 172.21.15.16:0/621674366 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9a80b4750 0x7fc9a80d0020 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:34.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.913+0000 7fc9af456700 1 -- 172.21.15.16:0/621674366 >> 172.21.15.16:0/621674366 conn(0x7fc9a809c590 msgr2=0x7fc9a80d3240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:34.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.913+0000 7fc9af456700 1 -- 172.21.15.16:0/621674366 shutdown_connections 2023-05-20T01:25:34.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.913+0000 7fc9af456700 1 -- 172.21.15.16:0/621674366 wait complete. 2023-05-20T01:25:34.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.914+0000 7fc9af456700 1 Processor -- start 2023-05-20T01:25:34.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.914+0000 7fc9af456700 1 -- start start 2023-05-20T01:25:34.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.915+0000 7fc9af456700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc9a80b2b40 0x7fc9a80cc590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:34.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.915+0000 7fc9af456700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9a80b4750 0x7fc9a814f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:34.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.915+0000 7fc9af456700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc9a80cfc50 0x7fc9a80c92e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:34.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.915+0000 7fc9af456700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9a80c8ec0 con 0x7fc9a80b4750 2023-05-20T01:25:34.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.915+0000 7fc9af456700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9a80b7c40 con 0x7fc9a80cfc50 2023-05-20T01:25:34.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.915+0000 7fc9af456700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9a80c8520 con 0x7fc9a80b2b40 2023-05-20T01:25:34.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.915+0000 7fc9ac9f1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9a80b4750 0x7fc9a814f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:34.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.915+0000 7fc9ac9f1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9a80b4750 0x7fc9a814f0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36788/0 (socket says 172.21.15.16:36788) 2023-05-20T01:25:34.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.915+0000 7fc9ac9f1700 1 -- 172.21.15.16:0/1763889618 learned_addr learned my addr 172.21.15.16:0/1763889618 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:34.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.915+0000 7fc9ad1f2700 1 --2- 172.21.15.16:0/1763889618 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc9a80b2b40 0x7fc9a80cc590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:34.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.915+0000 7fc9ad9f3700 1 --2- 172.21.15.16:0/1763889618 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc9a80cfc50 0x7fc9a80c92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:34.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.915+0000 7fc9ac9f1700 1 -- 172.21.15.16:0/1763889618 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc9a80b2b40 msgr2=0x7fc9a80cc590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:34.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.915+0000 7fc9ac9f1700 1 --2- 172.21.15.16:0/1763889618 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc9a80b2b40 0x7fc9a80cc590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:34.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.915+0000 7fc9ac9f1700 1 -- 172.21.15.16:0/1763889618 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc9a80cfc50 msgr2=0x7fc9a80c92e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:34.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.915+0000 7fc9ac9f1700 1 --2- 172.21.15.16:0/1763889618 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc9a80cfc50 0x7fc9a80c92e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:34.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.915+0000 7fc9ac9f1700 1 -- 172.21.15.16:0/1763889618 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9a40094a0 con 0x7fc9a80b4750 2023-05-20T01:25:34.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.915+0000 7fc9ad1f2700 1 --2- 172.21.15.16:0/1763889618 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc9a80b2b40 0x7fc9a80cc590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:25:34.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.915+0000 7fc9ad9f3700 1 --2- 172.21.15.16:0/1763889618 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc9a80cfc50 0x7fc9a80c92e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:25:34.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.915+0000 7fc9ac9f1700 1 --2- 172.21.15.16:0/1763889618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9a80b4750 0x7fc9a814f0e0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7fc9a4014040 tx=0x7fc9a4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:34.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.915+0000 7fc99dffb700 1 -- 172.21.15.16:0/1763889618 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9a40038a0 con 0x7fc9a80b4750 2023-05-20T01:25:34.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.916+0000 7fc99dffb700 1 -- 172.21.15.16:0/1763889618 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9a401e070 con 0x7fc9a80b4750 2023-05-20T01:25:34.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.916+0000 7fc9af456700 1 -- 172.21.15.16:0/1763889618 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9a80b4380 con 0x7fc9a80b4750 2023-05-20T01:25:34.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.916+0000 7fc99dffb700 1 -- 172.21.15.16:0/1763889618 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9a4003c20 con 0x7fc9a80b4750 2023-05-20T01:25:34.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.916+0000 7fc9af456700 1 -- 172.21.15.16:0/1763889618 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9a815d0b0 con 0x7fc9a80b4750 2023-05-20T01:25:34.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.917+0000 7fc9af456700 1 -- 172.21.15.16:0/1763889618 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9a80d4000 con 0x7fc9a80b4750 2023-05-20T01:25:34.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.917+0000 7fc99dffb700 1 -- 172.21.15.16:0/1763889618 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc9a402d450 con 0x7fc9a80b4750 2023-05-20T01:25:34.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.918+0000 7fc99dffb700 1 --2- 172.21.15.16:0/1763889618 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc98c06ace0 0x7fc98c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:34.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.918+0000 7fc99dffb700 1 -- 172.21.15.16:0/1763889618 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6958+0+0 (secure 0 0 0) 0x7fc9a408abf0 con 0x7fc9a80b4750 2023-05-20T01:25:34.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.918+0000 7fc9ad1f2700 1 --2- 172.21.15.16:0/1763889618 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc98c06ace0 0x7fc98c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:34.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.918+0000 7fc9ad1f2700 1 --2- 172.21.15.16:0/1763889618 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc98c06ace0 0x7fc98c06d190 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fc998009960 tx=0x7fc998008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:34.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:34.922+0000 7fc99dffb700 1 -- 172.21.15.16:0/1763889618 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fc9a4055a90 con 0x7fc9a80b4750 2023-05-20T01:25:35.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.164+0000 7fc9af456700 1 -- 172.21.15.16:0/1763889618 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc9a80c9000 con 0x7fc9a80b4750 2023-05-20T01:25:35.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.164+0000 7fc99dffb700 1 -- 172.21.15.16:0/1763889618 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+4674 (secure 0 0 0) 0x7fc9a80c9000 con 0x7fc9a80b4750 2023-05-20T01:25:35.169 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:25:35.169 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":63,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6077,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/1241894038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":1241894038},{"type":"v1","addr":"172.21.15.132:6835","nonce":1241894038}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":6562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/3771738903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3771738903},{"type":"v1","addr":"172.21.15.132:6834","nonce":3771738903}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":7101,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/3876038711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":3876038711},{"type":"v1","addr":"172.21.15.16:6836","nonce":3876038711}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":7410,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2740585996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2740585996},{"type":"v1","addr":"172.21.15.16:6837","nonce":2740585996}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-20T01:25:32.295399+0000","modified":"2023-05-20T01:25:33.530206+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":93,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13,14],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-05-20T01:25:35.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.166+0000 7fc9af456700 1 -- 172.21.15.16:0/1763889618 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc98c06ace0 msgr2=0x7fc98c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:35.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.166+0000 7fc9af456700 1 --2- 172.21.15.16:0/1763889618 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc98c06ace0 0x7fc98c06d190 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fc998009960 tx=0x7fc998008040 comp rx=0 tx=0).stop 2023-05-20T01:25:35.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.166+0000 7fc9af456700 1 -- 172.21.15.16:0/1763889618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9a80b4750 msgr2=0x7fc9a814f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:35.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.166+0000 7fc9af456700 1 --2- 172.21.15.16:0/1763889618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9a80b4750 0x7fc9a814f0e0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7fc9a4014040 tx=0x7fc9a4004750 comp rx=0 tx=0).stop 2023-05-20T01:25:35.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.166+0000 7fc9af456700 1 -- 172.21.15.16:0/1763889618 shutdown_connections 2023-05-20T01:25:35.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.166+0000 7fc9af456700 1 --2- 172.21.15.16:0/1763889618 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc9a80cfc50 0x7fc9a80c92e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:35.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.166+0000 7fc9af456700 1 --2- 172.21.15.16:0/1763889618 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc98c06ace0 0x7fc98c06d190 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:35.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.166+0000 7fc9af456700 1 --2- 172.21.15.16:0/1763889618 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc9a80b2b40 0x7fc9a80cc590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:35.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.166+0000 7fc9af456700 1 --2- 172.21.15.16:0/1763889618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9a80b4750 0x7fc9a814f0e0 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:35.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.166+0000 7fc9af456700 1 -- 172.21.15.16:0/1763889618 >> 172.21.15.16:0/1763889618 conn(0x7fc9a809c590 msgr2=0x7fc9a806b090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:35.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.166+0000 7fc9af456700 1 -- 172.21.15.16:0/1763889618 shutdown_connections 2023-05-20T01:25:35.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.166+0000 7fc9af456700 1 -- 172.21.15.16:0/1763889618 wait complete. 2023-05-20T01:25:35.173 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 63 2023-05-20T01:25:35.183 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-20T01:25:35.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.303+0000 7f77a329f700 1 Processor -- start 2023-05-20T01:25:35.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.303+0000 7f77a329f700 1 -- start start 2023-05-20T01:25:35.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.304+0000 7f77a329f700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f779c156d30 0x7f779c0ce550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:35.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.304+0000 7f77a329f700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f779c0cd650 0x7f779c15cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:35.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.304+0000 7f77a329f700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f779c0cdad0 0x7f779c15d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:35.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.304+0000 7f77a329f700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f779c0b1970 con 0x7f779c156d30 2023-05-20T01:25:35.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.304+0000 7f77a329f700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f779c0b13b0 con 0x7f779c0cd650 2023-05-20T01:25:35.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.304+0000 7f77a329f700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f779c0b1690 con 0x7f779c0cdad0 2023-05-20T01:25:35.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.304+0000 7f77a103b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f779c156d30 0x7f779c0ce550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:35.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.304+0000 7f77a103b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f779c156d30 0x7f779c0ce550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36794/0 (socket says 172.21.15.16:36794) 2023-05-20T01:25:35.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.304+0000 7f77a103b700 1 -- 172.21.15.16:0/2802532424 learned_addr learned my addr 172.21.15.16:0/2802532424 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:35.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.304+0000 7f77a183c700 1 --2- 172.21.15.16:0/2802532424 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f779c0cdad0 0x7f779c15d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:35.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.304+0000 7f77a083a700 1 --2- 172.21.15.16:0/2802532424 >> v2:172.21.15.132:3300/0 conn(0x7f779c0cd650 0x7f779c15cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:35.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.304+0000 7f77a103b700 1 -- 172.21.15.16:0/2802532424 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f779c0cdad0 msgr2=0x7f779c15d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:35.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.304+0000 7f77a103b700 1 --2- 172.21.15.16:0/2802532424 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f779c0cdad0 0x7f779c15d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:35.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.304+0000 7f77a103b700 1 -- 172.21.15.16:0/2802532424 >> v2:172.21.15.132:3300/0 conn(0x7f779c0cd650 msgr2=0x7f779c15cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:35.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.304+0000 7f77a103b700 1 --2- 172.21.15.16:0/2802532424 >> v2:172.21.15.132:3300/0 conn(0x7f779c0cd650 0x7f779c15cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:35.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.304+0000 7f77a103b700 1 -- 172.21.15.16:0/2802532424 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f779c0cce50 con 0x7f779c156d30 2023-05-20T01:25:35.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.305+0000 7f77a103b700 1 --2- 172.21.15.16:0/2802532424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f779c156d30 0x7f779c0ce550 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f779c06fa00 tx=0x7f778c00ad70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5abaf752669c1903 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:35.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.305+0000 7f7793fff700 1 -- 172.21.15.16:0/2802532424 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f778c00cb70 con 0x7f779c156d30 2023-05-20T01:25:35.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.305+0000 7f7793fff700 1 -- 172.21.15.16:0/2802532424 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f778c00ccd0 con 0x7f779c156d30 2023-05-20T01:25:35.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.305+0000 7f7793fff700 1 -- 172.21.15.16:0/2802532424 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f778c00f690 con 0x7f779c156d30 2023-05-20T01:25:35.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.305+0000 7f77a329f700 1 -- 172.21.15.16:0/2802532424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f779c156d30 msgr2=0x7f779c0ce550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:35.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.305+0000 7f77a329f700 1 --2- 172.21.15.16:0/2802532424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f779c156d30 0x7f779c0ce550 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f779c06fa00 tx=0x7f778c00ad70 comp rx=0 tx=0).stop 2023-05-20T01:25:35.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.305+0000 7f77a329f700 1 -- 172.21.15.16:0/2802532424 shutdown_connections 2023-05-20T01:25:35.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.305+0000 7f77a329f700 1 --2- 172.21.15.16:0/2802532424 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f779c0cdad0 0x7f779c15d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:35.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.305+0000 7f77a329f700 1 --2- 172.21.15.16:0/2802532424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f779c156d30 0x7f779c0ce550 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:35.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.305+0000 7f77a329f700 1 --2- 172.21.15.16:0/2802532424 >> v2:172.21.15.132:3300/0 conn(0x7f779c0cd650 0x7f779c15cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:35.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.305+0000 7f77a329f700 1 -- 172.21.15.16:0/2802532424 >> 172.21.15.16:0/2802532424 conn(0x7f779c0cb910 msgr2=0x7f779c0cbd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:35.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.306+0000 7f77a329f700 1 -- 172.21.15.16:0/2802532424 shutdown_connections 2023-05-20T01:25:35.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.306+0000 7f77a329f700 1 -- 172.21.15.16:0/2802532424 wait complete. 2023-05-20T01:25:35.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.307+0000 7f77a329f700 1 Processor -- start 2023-05-20T01:25:35.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.307+0000 7f77a329f700 1 -- start start 2023-05-20T01:25:35.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.307+0000 7f77a329f700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f779c0cd650 0x7f779c154050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:35.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.308+0000 7f77a329f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f779c0cdad0 0x7f779c0c6e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:35.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.308+0000 7f77a329f700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f779c156d30 0x7f779c0b3f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:35.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.308+0000 7f77a329f700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f779c0c36c0 con 0x7f779c0cdad0 2023-05-20T01:25:35.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.308+0000 7f77a329f700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f779c0c2a40 con 0x7f779c0cd650 2023-05-20T01:25:35.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.308+0000 7f77a329f700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f779c0c2d20 con 0x7f779c156d30 2023-05-20T01:25:35.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.308+0000 7f77a083a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f779c0cdad0 0x7f779c0c6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:35.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.308+0000 7f77a083a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f779c0cdad0 0x7f779c0c6e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36802/0 (socket says 172.21.15.16:36802) 2023-05-20T01:25:35.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.308+0000 7f77a083a700 1 -- 172.21.15.16:0/1490801156 learned_addr learned my addr 172.21.15.16:0/1490801156 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:35.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.308+0000 7f77a103b700 1 --2- 172.21.15.16:0/1490801156 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f779c0cd650 0x7f779c154050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:35.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.308+0000 7f77a183c700 1 --2- 172.21.15.16:0/1490801156 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f779c156d30 0x7f779c0b3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:35.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.308+0000 7f77a083a700 1 -- 172.21.15.16:0/1490801156 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f779c156d30 msgr2=0x7f779c0b3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:35.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.308+0000 7f77a083a700 1 --2- 172.21.15.16:0/1490801156 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f779c156d30 0x7f779c0b3f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:35.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.308+0000 7f77a083a700 1 -- 172.21.15.16:0/1490801156 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f779c0cd650 msgr2=0x7f779c154050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:35.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.308+0000 7f77a083a700 1 --2- 172.21.15.16:0/1490801156 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f779c0cd650 0x7f779c154050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:35.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.308+0000 7f77a083a700 1 -- 172.21.15.16:0/1490801156 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f778c00a6f0 con 0x7f779c0cdad0 2023-05-20T01:25:35.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.308+0000 7f77a103b700 1 --2- 172.21.15.16:0/1490801156 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f779c0cd650 0x7f779c154050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:25:35.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.308+0000 7f77a083a700 1 --2- 172.21.15.16:0/1490801156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f779c0cdad0 0x7f779c0c6e70 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f779800e810 tx=0x7f779800ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:35.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.308+0000 7f7791ffb700 1 -- 172.21.15.16:0/1490801156 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7798013610 con 0x7f779c0cdad0 2023-05-20T01:25:35.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.308+0000 7f7791ffb700 1 -- 172.21.15.16:0/1490801156 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7798013770 con 0x7f779c0cdad0 2023-05-20T01:25:35.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.308+0000 7f7791ffb700 1 -- 172.21.15.16:0/1490801156 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7798013b00 con 0x7f779c0cdad0 2023-05-20T01:25:35.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.308+0000 7f77a329f700 1 -- 172.21.15.16:0/1490801156 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f779c0cea90 con 0x7f779c0cdad0 2023-05-20T01:25:35.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.309+0000 7f77a329f700 1 -- 172.21.15.16:0/1490801156 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f779c0cd130 con 0x7f779c0cdad0 2023-05-20T01:25:35.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.309+0000 7f77a329f700 1 -- 172.21.15.16:0/1490801156 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f779c0cb4a0 con 0x7f779c0cdad0 2023-05-20T01:25:35.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.310+0000 7f7791ffb700 1 -- 172.21.15.16:0/1490801156 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7798013c60 con 0x7f779c0cdad0 2023-05-20T01:25:35.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.310+0000 7f7791ffb700 1 --2- 172.21.15.16:0/1490801156 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f778006ace0 0x7f778006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:35.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.310+0000 7f7791ffb700 1 -- 172.21.15.16:0/1490801156 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6958+0+0 (secure 0 0 0) 0x7f7798089420 con 0x7f779c0cdad0 2023-05-20T01:25:35.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.310+0000 7f77a103b700 1 --2- 172.21.15.16:0/1490801156 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f778006ace0 0x7f778006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:35.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.311+0000 7f77a103b700 1 --2- 172.21.15.16:0/1490801156 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f778006ace0 0x7f778006d190 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f778c008010 tx=0x7f778c012000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:35.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.315+0000 7f7791ffb700 1 -- 172.21.15.16:0/1490801156 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f77980541e0 con 0x7f779c0cdad0 2023-05-20T01:25:35.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.556+0000 7f77a329f700 1 -- 172.21.15.16:0/1490801156 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f779c04e8c0 con 0x7f779c0cdad0 2023-05-20T01:25:35.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.556+0000 7f7791ffb700 1 -- 172.21.15.16:0/1490801156 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+4674 (secure 0 0 0) 0x7f7798057800 con 0x7f779c0cdad0 2023-05-20T01:25:35.562 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:25:35.562 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":63,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6077,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/1241894038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":1241894038},{"type":"v1","addr":"172.21.15.132:6835","nonce":1241894038}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":6562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/3771738903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3771738903},{"type":"v1","addr":"172.21.15.132:6834","nonce":3771738903}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":7101,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/3876038711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":3876038711},{"type":"v1","addr":"172.21.15.16:6836","nonce":3876038711}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":7410,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2740585996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2740585996},{"type":"v1","addr":"172.21.15.16:6837","nonce":2740585996}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-20T01:25:32.295399+0000","modified":"2023-05-20T01:25:33.530206+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":93,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13,14],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-05-20T01:25:35.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.559+0000 7f77a329f700 1 -- 172.21.15.16:0/1490801156 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f778006ace0 msgr2=0x7f778006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:35.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.559+0000 7f77a329f700 1 --2- 172.21.15.16:0/1490801156 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f778006ace0 0x7f778006d190 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f778c008010 tx=0x7f778c012000 comp rx=0 tx=0).stop 2023-05-20T01:25:35.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.559+0000 7f77a329f700 1 -- 172.21.15.16:0/1490801156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f779c0cdad0 msgr2=0x7f779c0c6e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:35.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.559+0000 7f77a329f700 1 --2- 172.21.15.16:0/1490801156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f779c0cdad0 0x7f779c0c6e70 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f779800e810 tx=0x7f779800ebd0 comp rx=0 tx=0).stop 2023-05-20T01:25:35.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.559+0000 7f77a329f700 1 -- 172.21.15.16:0/1490801156 shutdown_connections 2023-05-20T01:25:35.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.559+0000 7f77a329f700 1 --2- 172.21.15.16:0/1490801156 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f778006ace0 0x7f778006d190 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:35.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.559+0000 7f77a329f700 1 --2- 172.21.15.16:0/1490801156 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f779c156d30 0x7f779c0b3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:35.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.559+0000 7f77a329f700 1 --2- 172.21.15.16:0/1490801156 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f779c0cd650 0x7f779c154050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:35.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.559+0000 7f77a329f700 1 --2- 172.21.15.16:0/1490801156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f779c0cdad0 0x7f779c0c6e70 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:35.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.559+0000 7f77a329f700 1 -- 172.21.15.16:0/1490801156 >> 172.21.15.16:0/1490801156 conn(0x7f779c0cb910 msgr2=0x7f779c0cdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:35.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.559+0000 7f77a329f700 1 -- 172.21.15.16:0/1490801156 shutdown_connections 2023-05-20T01:25:35.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.559+0000 7f77a329f700 1 -- 172.21.15.16:0/1490801156 wait complete. 2023-05-20T01:25:35.566 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 63 2023-05-20T01:25:35.575 DEBUG:teuthology.orchestra.run.smithi016:> cephfs-table-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 cephfs:0 reset session 2023-05-20T01:25:35.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.614+0000 7fdd533ca2c0 1 -- start start 2023-05-20T01:25:35.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.614+0000 7fdd533ca2c0 1 Processor -- start 2023-05-20T01:25:35.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.615+0000 7fdd533ca2c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555870b2f000 0x555870bc8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:35.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.615+0000 7fdd533ca2c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x555870b2ec00 0x555870bc8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:35.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.615+0000 7fdd533ca2c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555870b2e800 0x555870bc9080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:35.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.615+0000 7fdd533ca2c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x555870b6bcc0 con 0x555870b2e800 2023-05-20T01:25:35.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.615+0000 7fdd533ca2c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x555870b6b400 con 0x555870b2f000 2023-05-20T01:25:35.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.615+0000 7fdd533ca2c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x555870b6be00 con 0x555870b2ec00 2023-05-20T01:25:35.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.615+0000 7fdd51b77700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555870b2e800 0x555870bc9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:35.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.615+0000 7fdd51b77700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555870b2e800 0x555870bc9080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36818/0 (socket says 172.21.15.16:36818) 2023-05-20T01:25:35.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.615+0000 7fdd51b77700 1 -- 172.21.15.16:0/3172169839 learned_addr learned my addr 172.21.15.16:0/3172169839 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:35.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.615+0000 7fdd51376700 1 --2- 172.21.15.16:0/3172169839 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555870b2f000 0x555870bc8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:35.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.615+0000 7fdd50b75700 1 --2- 172.21.15.16:0/3172169839 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x555870b2ec00 0x555870bc8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:35.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.615+0000 7fdd51b77700 1 -- 172.21.15.16:0/3172169839 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x555870b2ec00 msgr2=0x555870bc8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:35.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.615+0000 7fdd51b77700 1 --2- 172.21.15.16:0/3172169839 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x555870b2ec00 0x555870bc8b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:35.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.615+0000 7fdd51b77700 1 -- 172.21.15.16:0/3172169839 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555870b2f000 msgr2=0x555870bc8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:35.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.615+0000 7fdd51b77700 1 --2- 172.21.15.16:0/3172169839 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555870b2f000 0x555870bc8580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:35.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.615+0000 7fdd51b77700 1 -- 172.21.15.16:0/3172169839 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555870b52d00 con 0x555870b2e800 2023-05-20T01:25:35.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.615+0000 7fdd50b75700 1 --2- 172.21.15.16:0/3172169839 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x555870b2ec00 0x555870bc8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:25:35.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.616+0000 7fdd51b77700 1 --2- 172.21.15.16:0/3172169839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555870b2e800 0x555870bc9080 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x555871990720 tx=0x555871998540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b152a2270adf975a server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:35.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.616+0000 7fdd4eb71700 1 -- 172.21.15.16:0/3172169839 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x555870b63600 con 0x555870b2e800 2023-05-20T01:25:35.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.616+0000 7fdd4eb71700 1 -- 172.21.15.16:0/3172169839 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555870b80600 con 0x555870b2e800 2023-05-20T01:25:35.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.616+0000 7fdd4eb71700 1 -- 172.21.15.16:0/3172169839 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x555870b631e0 con 0x555870b2e800 2023-05-20T01:25:35.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.616+0000 7fdd533ca2c0 1 -- 172.21.15.16:0/3172169839 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555870b536c0 con 0x555870b2e800 2023-05-20T01:25:35.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.616+0000 7fdd533ca2c0 1 -- 172.21.15.16:0/3172169839 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x555870b53520 con 0x555870b2e800 2023-05-20T01:25:35.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.616+0000 7fdd533ca2c0 4 waiting for MDS map... 2023-05-20T01:25:35.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.616+0000 7fdd4eb71700 1 -- 172.21.15.16:0/3172169839 <== mon.0 v2:172.21.15.16:3300/0 4 ==== osd_map(93..93 src has 1..93) v4 ==== 6958+0+0 (secure 0 0 0) 0x555870b70c40 con 0x555870b2e800 2023-05-20T01:25:35.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.616+0000 7fdd4eb71700 1 -- 172.21.15.16:0/3172169839 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 63) v1 ==== 3126+0+0 (secure 0 0 0) 0x55587199a300 con 0x555870b2e800 2023-05-20T01:25:35.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.616+0000 7fdd533ca2c0 4 Got MDS map 63 2023-05-20T01:25:35.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.616+0000 7fdd533ca2c0 10 main: main 2023-05-20T01:25:35.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.616+0000 7fdd533ca2c0 4 main: connecting to RADOS... 2023-05-20T01:25:35.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.617+0000 7fdd533ca2c0 1 Processor -- start 2023-05-20T01:25:35.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.617+0000 7fdd533ca2c0 1 -- start start 2023-05-20T01:25:35.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.617+0000 7fdd533ca2c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555870b2fc00 0x555870bc9600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:35.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.617+0000 7fdd533ca2c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555870e3a000 0x555870bc9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:35.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.617+0000 7fdd533ca2c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x555870e3a400 0x555870bca100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:35.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.617+0000 7fdd533ca2c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x555870b6aa00 con 0x555870e3a000 2023-05-20T01:25:35.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.618+0000 7fdd533ca2c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x555870b6b400 con 0x555870b2fc00 2023-05-20T01:25:35.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.618+0000 7fdd533ca2c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x555870b6ab40 con 0x555870e3a400 2023-05-20T01:25:35.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.618+0000 7fdd51376700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555870e3a000 0x555870bc9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:35.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.618+0000 7fdd51376700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555870e3a000 0x555870bc9b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36822/0 (socket says 172.21.15.16:36822) 2023-05-20T01:25:35.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.618+0000 7fdd51376700 1 -- 172.21.15.16:0/1977578707 learned_addr learned my addr 172.21.15.16:0/1977578707 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:35.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.618+0000 7fdd50b75700 1 --2- 172.21.15.16:0/1977578707 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555870b2fc00 0x555870bc9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:35.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.618+0000 7fdd50b75700 1 --2- 172.21.15.16:0/1977578707 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x555870e3a400 0x555870bca100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:35.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.618+0000 7fdd51376700 1 -- 172.21.15.16:0/1977578707 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x555870e3a400 msgr2=0x555870bca100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:35.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.618+0000 7fdd51376700 1 --2- 172.21.15.16:0/1977578707 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x555870e3a400 0x555870bca100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:35.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.618+0000 7fdd51376700 1 -- 172.21.15.16:0/1977578707 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555870b2fc00 msgr2=0x555870bc9600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:35.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.618+0000 7fdd51376700 1 --2- 172.21.15.16:0/1977578707 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555870b2fc00 0x555870bc9600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:35.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.618+0000 7fdd51376700 1 -- 172.21.15.16:0/1977578707 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555870b53380 con 0x555870e3a000 2023-05-20T01:25:35.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.618+0000 7fdd50b75700 1 --2- 172.21.15.16:0/1977578707 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555870b2fc00 0x555870bc9600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:25:35.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.618+0000 7fdd51376700 1 --2- 172.21.15.16:0/1977578707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555870e3a000 0x555870bc9b80 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x5558719915f0 tx=0x5558719bad20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8aee78ae30dd9cf server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:35.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.618+0000 7fdd4cb6d700 1 -- 172.21.15.16:0/1977578707 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x555870b63080 con 0x555870e3a000 2023-05-20T01:25:35.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.618+0000 7fdd4cb6d700 1 -- 172.21.15.16:0/1977578707 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555870b81c80 con 0x555870e3a000 2023-05-20T01:25:35.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.618+0000 7fdd4cb6d700 1 -- 172.21.15.16:0/1977578707 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x555870b63600 con 0x555870e3a000 2023-05-20T01:25:35.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.619+0000 7fdd533ca2c0 1 -- 172.21.15.16:0/1977578707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555870e3a000 msgr2=0x555870bc9b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:35.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.619+0000 7fdd533ca2c0 1 --2- 172.21.15.16:0/1977578707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555870e3a000 0x555870bc9b80 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x5558719915f0 tx=0x5558719bad20 comp rx=0 tx=0).stop 2023-05-20T01:25:35.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.619+0000 7fdd533ca2c0 1 -- 172.21.15.16:0/1977578707 shutdown_connections 2023-05-20T01:25:35.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.619+0000 7fdd533ca2c0 1 --2- 172.21.15.16:0/1977578707 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x555870e3a400 0x555870bca100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:35.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.619+0000 7fdd533ca2c0 1 --2- 172.21.15.16:0/1977578707 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555870b2fc00 0x555870bc9600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:35.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.619+0000 7fdd533ca2c0 1 --2- 172.21.15.16:0/1977578707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555870e3a000 0x555870bc9b80 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:35.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.619+0000 7fdd533ca2c0 1 -- 172.21.15.16:0/1977578707 >> 172.21.15.16:0/1977578707 conn(0x555870b2e400 msgr2=0x555870c0b800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:35.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.619+0000 7fdd533ca2c0 1 -- 172.21.15.16:0/1977578707 shutdown_connections 2023-05-20T01:25:35.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.619+0000 7fdd533ca2c0 1 -- 172.21.15.16:0/1977578707 wait complete. 2023-05-20T01:25:35.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.619+0000 7fdd533ca2c0 1 Processor -- start 2023-05-20T01:25:35.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.619+0000 7fdd533ca2c0 1 -- start start 2023-05-20T01:25:35.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.620+0000 7fdd533ca2c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x555870b2e400 0x555870bcb180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:35.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.620+0000 7fdd533ca2c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555870e50000 0x555870bcac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:35.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.620+0000 7fdd533ca2c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555870e50400 0x555870bca680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:35.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.620+0000 7fdd533ca2c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x555870b6ac80 con 0x555870e50400 2023-05-20T01:25:35.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.620+0000 7fdd533ca2c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x555870b6a000 con 0x555870e50000 2023-05-20T01:25:35.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.620+0000 7fdd533ca2c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x555870b6b180 con 0x555870b2e400 2023-05-20T01:25:35.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.620+0000 7fdd50b75700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555870e50400 0x555870bca680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:35.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.620+0000 7fdd50b75700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555870e50400 0x555870bca680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36830/0 (socket says 172.21.15.16:36830) 2023-05-20T01:25:35.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.620+0000 7fdd50b75700 1 -- 172.21.15.16:0/1197354353 learned_addr learned my addr 172.21.15.16:0/1197354353 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:35.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.620+0000 7fdd51376700 1 --2- 172.21.15.16:0/1197354353 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555870e50000 0x555870bcac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:35.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.620+0000 7fdd50b75700 1 --2- 172.21.15.16:0/1197354353 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x555870b2e400 0x555870bcb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:35.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.620+0000 7fdd50b75700 1 -- 172.21.15.16:0/1197354353 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x555870b2e400 msgr2=0x555870bcb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:35.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.620+0000 7fdd50b75700 1 --2- 172.21.15.16:0/1197354353 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x555870b2e400 0x555870bcb180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:35.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.620+0000 7fdd50b75700 1 -- 172.21.15.16:0/1197354353 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555870e50000 msgr2=0x555870bcac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:35.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.620+0000 7fdd50b75700 1 --2- 172.21.15.16:0/1197354353 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555870e50000 0x555870bcac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:35.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.620+0000 7fdd50b75700 1 -- 172.21.15.16:0/1197354353 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555870b531e0 con 0x555870e50400 2023-05-20T01:25:35.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.620+0000 7fdd50b75700 1 --2- 172.21.15.16:0/1197354353 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555870e50400 0x555870bca680 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x5558719a86f0 tx=0x5558719981e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:35.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.620+0000 7fdd4b36a700 1 -- 172.21.15.16:0/1197354353 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x555870b63080 con 0x555870e50400 2023-05-20T01:25:35.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.620+0000 7fdd4b36a700 1 -- 172.21.15.16:0/1197354353 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555870b80480 con 0x555870e50400 2023-05-20T01:25:35.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.621+0000 7fdd4b36a700 1 -- 172.21.15.16:0/1197354353 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x555870b63600 con 0x555870e50400 2023-05-20T01:25:35.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.621+0000 7fdd533ca2c0 1 -- 172.21.15.16:0/1197354353 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x555870b53d40 con 0x555870e50400 2023-05-20T01:25:35.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.621+0000 7fdd533ca2c0 1 -- 172.21.15.16:0/1197354353 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555870b53380 con 0x555870e50400 2023-05-20T01:25:35.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.621+0000 7fdd533ca2c0 4 main: resolving pool 12 2023-05-20T01:25:35.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.622+0000 7fdd4b36a700 1 -- 172.21.15.16:0/1197354353 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x555870b90840 con 0x555870e50400 2023-05-20T01:25:35.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.622+0000 7fdd4b36a700 1 --2- 172.21.15.16:0/1197354353 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x555870e60400 0x555870bcb700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:35.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.622+0000 7fdd4b36a700 1 -- 172.21.15.16:0/1197354353 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6958+0+0 (secure 0 0 0) 0x555870b70a80 con 0x555870e50400 2023-05-20T01:25:35.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.622+0000 7fdd51376700 1 --2- 172.21.15.16:0/1197354353 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x555870e60400 0x555870bcb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:35.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.622+0000 7fdd533ca2c0 4 main: creating IoCtx.. 2023-05-20T01:25:35.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.622+0000 7fdd533ca2c0 1 --2- 172.21.15.16:0/1197354353 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x555870e50800 0x555870bca100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:35.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.622+0000 7fdd533ca2c0 1 -- 172.21.15.16:0/1197354353 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:1 12.b 12:d0630e4c:::mds0_sessionmap:head [omap-clear,omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x5558719b0700 con 0x555870e50800 2023-05-20T01:25:35.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.622+0000 7fdd50b75700 1 --2- 172.21.15.16:0/1197354353 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x555870e50800 0x555870bca100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:35.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.623+0000 7fdd51376700 1 --2- 172.21.15.16:0/1197354353 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x555870e60400 0x555870bcb700 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x5558719915f0 tx=0x5558719bac00 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:35.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.623+0000 7fdd50b75700 1 --2- 172.21.15.16:0/1197354353 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x555870e50800 0x555870bca100 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:35.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.624+0000 7fdd50b75700 1 -- 172.21.15.16:0/1197354353 <== osd.3 v2:172.21.15.16:6800/3669781890 1 ==== osd_op_reply(1 mds0_sessionmap [omap-clear,omap-set-header] v93'5 uv5 ondisk = 0) v8 ==== 201+0+0 (crc 0 0 0) 0x55587198cfc0 con 0x555870e50800 2023-05-20T01:25:35.645 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-05-20T01:25:35.645 INFO:teuthology.orchestra.run.smithi016.stdout: "0": { 2023-05-20T01:25:35.645 INFO:teuthology.orchestra.run.smithi016.stdout: "data": {}, 2023-05-20T01:25:35.645 INFO:teuthology.orchestra.run.smithi016.stdout: "result": 0 2023-05-20T01:25:35.645 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:25:35.646 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-05-20T01:25:35.646 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:25:35.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.624+0000 7fdd533ca2c0 1 -- 172.21.15.16:0/1197354353 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x555870e50800 msgr2=0x555870bca100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:35.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.624+0000 7fdd533ca2c0 1 --2- 172.21.15.16:0/1197354353 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x555870e50800 0x555870bca100 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:35.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.624+0000 7fdd533ca2c0 1 -- 172.21.15.16:0/1197354353 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x555870e60400 msgr2=0x555870bcb700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:35.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.624+0000 7fdd533ca2c0 1 --2- 172.21.15.16:0/1197354353 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x555870e60400 0x555870bcb700 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x5558719915f0 tx=0x5558719bac00 comp rx=0 tx=0).stop 2023-05-20T01:25:35.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.624+0000 7fdd533ca2c0 1 -- 172.21.15.16:0/1197354353 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555870e50400 msgr2=0x555870bca680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:35.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.624+0000 7fdd533ca2c0 1 --2- 172.21.15.16:0/1197354353 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555870e50400 0x555870bca680 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x5558719a86f0 tx=0x5558719981e0 comp rx=0 tx=0).stop 2023-05-20T01:25:35.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.624+0000 7fdd51376700 1 -- 172.21.15.16:0/1197354353 reap_dead start 2023-05-20T01:25:35.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.624+0000 7fdd533ca2c0 1 -- 172.21.15.16:0/1197354353 shutdown_connections 2023-05-20T01:25:35.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.624+0000 7fdd533ca2c0 1 -- 172.21.15.16:0/1197354353 >> 172.21.15.16:0/1197354353 conn(0x555870b2fc00 msgr2=0x555870c0b800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:35.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.624+0000 7fdd533ca2c0 1 -- 172.21.15.16:0/1197354353 shutdown_connections 2023-05-20T01:25:35.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.624+0000 7fdd533ca2c0 1 -- 172.21.15.16:0/1197354353 wait complete. 2023-05-20T01:25:35.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.625+0000 7fdd533ca2c0 1 -- 172.21.15.16:0/3172169839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555870b2e800 msgr2=0x555870bc9080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:35.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.625+0000 7fdd533ca2c0 1 --2- 172.21.15.16:0/3172169839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555870b2e800 0x555870bc9080 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x555871990720 tx=0x555871998540 comp rx=0 tx=0).stop 2023-05-20T01:25:35.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.625+0000 7fdd533ca2c0 1 -- 172.21.15.16:0/3172169839 shutdown_connections 2023-05-20T01:25:35.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.625+0000 7fdd533ca2c0 1 --2- 172.21.15.16:0/3172169839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555870b2e800 0x555870bc9080 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:35.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.625+0000 7fdd533ca2c0 1 --2- 172.21.15.16:0/3172169839 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555870b2f000 0x555870bc8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:35.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.625+0000 7fdd533ca2c0 1 --2- 172.21.15.16:0/3172169839 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x555870b2ec00 0x555870bc8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:35.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.625+0000 7fdd533ca2c0 1 -- 172.21.15.16:0/3172169839 >> 172.21.15.16:0/3172169839 conn(0x555870b2f400 msgr2=0x555870c0b000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:35.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.625+0000 7fdd533ca2c0 1 -- 172.21.15.16:0/3172169839 shutdown_connections 2023-05-20T01:25:35.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.625+0000 7fdd533ca2c0 1 -- 172.21.15.16:0/3172169839 wait complete. 2023-05-20T01:25:35.651 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-table-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'cephfs:0', 'reset', 'session'] in time 0:00:00.075418, result: { "0": { "data": {}, "result": 0 } } 2023-05-20T01:25:35.651 DEBUG:teuthology.orchestra.run.smithi016:> cephfs-table-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 cephfs:0 reset snap 2023-05-20T01:25:35.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.725+0000 7fec7e5af2c0 1 -- start start 2023-05-20T01:25:35.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.725+0000 7fec7e5af2c0 1 Processor -- start 2023-05-20T01:25:35.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.725+0000 7fec7e5af2c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5563b56db000 0x5563b5775b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:35.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.725+0000 7fec7e5af2c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5563b56dac00 0x5563b5775600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:35.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.725+0000 7fec7e5af2c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563b56da800 0x5563b5775080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:35.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.725+0000 7fec7e5af2c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563b5717cc0 con 0x5563b56da800 2023-05-20T01:25:35.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.725+0000 7fec7e5af2c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563b5717400 con 0x5563b56dac00 2023-05-20T01:25:35.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.725+0000 7fec7e5af2c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x5563b5716a00 con 0x5563b56db000 2023-05-20T01:25:35.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.725+0000 7fec7cd5c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563b56da800 0x5563b5775080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:35.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.725+0000 7fec7cd5c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563b56da800 0x5563b5775080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36846/0 (socket says 172.21.15.16:36846) 2023-05-20T01:25:35.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.725+0000 7fec7cd5c700 1 -- 172.21.15.16:0/2001254912 learned_addr learned my addr 172.21.15.16:0/2001254912 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:35.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.726+0000 7fec7c55b700 1 --2- 172.21.15.16:0/2001254912 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5563b56db000 0x5563b5775b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:35.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.726+0000 7fec7bd5a700 1 --2- 172.21.15.16:0/2001254912 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5563b56dac00 0x5563b5775600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:35.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.726+0000 7fec7cd5c700 1 -- 172.21.15.16:0/2001254912 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5563b56db000 msgr2=0x5563b5775b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:35.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.726+0000 7fec7cd5c700 1 --2- 172.21.15.16:0/2001254912 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5563b56db000 0x5563b5775b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:35.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.726+0000 7fec7cd5c700 1 -- 172.21.15.16:0/2001254912 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5563b56dac00 msgr2=0x5563b5775600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:35.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.726+0000 7fec7cd5c700 1 --2- 172.21.15.16:0/2001254912 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5563b56dac00 0x5563b5775600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:35.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.726+0000 7fec7cd5c700 1 -- 172.21.15.16:0/2001254912 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5563b56fed00 con 0x5563b56da800 2023-05-20T01:25:35.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.726+0000 7fec7cd5c700 1 --2- 172.21.15.16:0/2001254912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563b56da800 0x5563b5775080 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x5563b6542630 tx=0x5563b64733e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24fa858ae2c122c3 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:35.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.726+0000 7fec79d56700 1 -- 172.21.15.16:0/2001254912 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5563b570f080 con 0x5563b56da800 2023-05-20T01:25:35.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.726+0000 7fec79d56700 1 -- 172.21.15.16:0/2001254912 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5563b572d800 con 0x5563b56da800 2023-05-20T01:25:35.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.726+0000 7fec79d56700 1 -- 172.21.15.16:0/2001254912 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5563b570f600 con 0x5563b56da800 2023-05-20T01:25:35.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.726+0000 7fec7e5af2c0 1 -- 172.21.15.16:0/2001254912 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5563b56ff6c0 con 0x5563b56da800 2023-05-20T01:25:35.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.726+0000 7fec7e5af2c0 1 -- 172.21.15.16:0/2001254912 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x5563b56ff520 con 0x5563b56da800 2023-05-20T01:25:35.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.726+0000 7fec7e5af2c0 4 waiting for MDS map... 2023-05-20T01:25:35.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.726+0000 7fec79d56700 1 -- 172.21.15.16:0/2001254912 <== mon.0 v2:172.21.15.16:3300/0 4 ==== osd_map(93..93 src has 1..93) v4 ==== 6958+0+0 (secure 0 0 0) 0x5563b571cc40 con 0x5563b56da800 2023-05-20T01:25:35.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.726+0000 7fec79d56700 1 -- 172.21.15.16:0/2001254912 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 63) v1 ==== 3126+0+0 (secure 0 0 0) 0x5563b5720f00 con 0x5563b56da800 2023-05-20T01:25:35.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.726+0000 7fec7e5af2c0 4 Got MDS map 63 2023-05-20T01:25:35.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.726+0000 7fec7e5af2c0 10 main: main 2023-05-20T01:25:35.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.727+0000 7fec7e5af2c0 4 main: connecting to RADOS... 2023-05-20T01:25:35.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.727+0000 7fec7e5af2c0 1 Processor -- start 2023-05-20T01:25:35.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.727+0000 7fec7e5af2c0 1 -- start start 2023-05-20T01:25:35.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.728+0000 7fec7e5af2c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5563b56dbc00 0x5563b5774b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:35.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.728+0000 7fec7e5af2c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5563b59e6000 0x5563b5774580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:35.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.728+0000 7fec7e5af2c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563b59e6400 0x5563b5776100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:35.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.728+0000 7fec7e5af2c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563b5716b40 con 0x5563b59e6400 2023-05-20T01:25:35.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.728+0000 7fec7e5af2c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563b5716a00 con 0x5563b59e6000 2023-05-20T01:25:35.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.728+0000 7fec7e5af2c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x5563b5717400 con 0x5563b56dbc00 2023-05-20T01:25:35.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.728+0000 7fec7bd5a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563b59e6400 0x5563b5776100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:35.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.728+0000 7fec7bd5a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563b59e6400 0x5563b5776100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36852/0 (socket says 172.21.15.16:36852) 2023-05-20T01:25:35.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.728+0000 7fec7c55b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5563b59e6000 0x5563b5774580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:35.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.728+0000 7fec7bd5a700 1 -- 172.21.15.16:0/2715480884 learned_addr learned my addr 172.21.15.16:0/2715480884 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:35.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.728+0000 7fec7bd5a700 1 --2- 172.21.15.16:0/2715480884 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5563b56dbc00 0x5563b5774b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:35.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.728+0000 7fec7bd5a700 1 -- 172.21.15.16:0/2715480884 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5563b56dbc00 msgr2=0x5563b5774b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:35.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.728+0000 7fec7bd5a700 1 --2- 172.21.15.16:0/2715480884 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5563b56dbc00 0x5563b5774b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:35.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.728+0000 7fec7bd5a700 1 -- 172.21.15.16:0/2715480884 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5563b59e6000 msgr2=0x5563b5774580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:35.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.728+0000 7fec7bd5a700 1 --2- 172.21.15.16:0/2715480884 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5563b59e6000 0x5563b5774580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:35.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.728+0000 7fec7bd5a700 1 -- 172.21.15.16:0/2715480884 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5563b56ff380 con 0x5563b59e6400 2023-05-20T01:25:35.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.728+0000 7fec7bd5a700 1 --2- 172.21.15.16:0/2715480884 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563b59e6400 0x5563b5776100 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x5563b6543b90 tx=0x5563b6562cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17fdcbaece97c7d9 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:35.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.728+0000 7fec77d52700 1 -- 172.21.15.16:0/2715480884 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5563b570f080 con 0x5563b59e6400 2023-05-20T01:25:35.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.728+0000 7fec77d52700 1 -- 172.21.15.16:0/2715480884 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5563b572c480 con 0x5563b59e6400 2023-05-20T01:25:35.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.728+0000 7fec77d52700 1 -- 172.21.15.16:0/2715480884 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5563b570f600 con 0x5563b59e6400 2023-05-20T01:25:35.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.729+0000 7fec7e5af2c0 1 -- 172.21.15.16:0/2715480884 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563b59e6400 msgr2=0x5563b5776100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:35.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.729+0000 7fec7e5af2c0 1 --2- 172.21.15.16:0/2715480884 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563b59e6400 0x5563b5776100 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x5563b6543b90 tx=0x5563b6562cc0 comp rx=0 tx=0).stop 2023-05-20T01:25:35.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.729+0000 7fec7e5af2c0 1 -- 172.21.15.16:0/2715480884 shutdown_connections 2023-05-20T01:25:35.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.729+0000 7fec7e5af2c0 1 --2- 172.21.15.16:0/2715480884 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563b59e6400 0x5563b5776100 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:35.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.729+0000 7fec7e5af2c0 1 --2- 172.21.15.16:0/2715480884 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5563b56dbc00 0x5563b5774b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:35.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.729+0000 7fec7e5af2c0 1 --2- 172.21.15.16:0/2715480884 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5563b59e6000 0x5563b5774580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:35.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.729+0000 7fec7e5af2c0 1 -- 172.21.15.16:0/2715480884 >> 172.21.15.16:0/2715480884 conn(0x5563b56da400 msgr2=0x5563b57b7800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:35.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.729+0000 7fec7e5af2c0 1 -- 172.21.15.16:0/2715480884 shutdown_connections 2023-05-20T01:25:35.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.729+0000 7fec7e5af2c0 1 -- 172.21.15.16:0/2715480884 wait complete. 2023-05-20T01:25:35.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.729+0000 7fec7e5af2c0 1 Processor -- start 2023-05-20T01:25:35.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.729+0000 7fec7e5af2c0 1 -- start start 2023-05-20T01:25:35.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.730+0000 7fec7e5af2c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5563b56da400 0x5563b5774b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:35.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.730+0000 7fec7e5af2c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5563b5a02000 0x5563b5774580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:35.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.730+0000 7fec7e5af2c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563b5a02400 0x5563b5776100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:35.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.730+0000 7fec7e5af2c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563b5716c80 con 0x5563b5a02400 2023-05-20T01:25:35.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.730+0000 7fec7e5af2c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563b5716000 con 0x5563b5a02000 2023-05-20T01:25:35.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.730+0000 7fec7e5af2c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x5563b5717180 con 0x5563b56da400 2023-05-20T01:25:35.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.730+0000 7fec7bd5a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563b5a02400 0x5563b5776100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:35.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.730+0000 7fec7bd5a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563b5a02400 0x5563b5776100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36858/0 (socket says 172.21.15.16:36858) 2023-05-20T01:25:35.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.730+0000 7fec7bd5a700 1 -- 172.21.15.16:0/3755306105 learned_addr learned my addr 172.21.15.16:0/3755306105 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:35.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.730+0000 7fec7bd5a700 1 --2- 172.21.15.16:0/3755306105 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5563b56da400 0x5563b5774b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:35.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.730+0000 7fec7c55b700 1 --2- 172.21.15.16:0/3755306105 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5563b5a02000 0x5563b5774580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:35.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.730+0000 7fec7bd5a700 1 -- 172.21.15.16:0/3755306105 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5563b56da400 msgr2=0x5563b5774b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:35.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.730+0000 7fec7bd5a700 1 --2- 172.21.15.16:0/3755306105 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5563b56da400 0x5563b5774b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:35.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.730+0000 7fec7bd5a700 1 -- 172.21.15.16:0/3755306105 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5563b5a02000 msgr2=0x5563b5774580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:35.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.730+0000 7fec7bd5a700 1 --2- 172.21.15.16:0/3755306105 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5563b5a02000 0x5563b5774580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:35.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.730+0000 7fec7bd5a700 1 -- 172.21.15.16:0/3755306105 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5563b56ff1e0 con 0x5563b5a02400 2023-05-20T01:25:35.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.731+0000 7fec7bd5a700 1 --2- 172.21.15.16:0/3755306105 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563b5a02400 0x5563b5776100 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x5563b5a00660 tx=0x5563b6562c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:35.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.731+0000 7fec7654f700 1 -- 172.21.15.16:0/3755306105 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5563b570edc0 con 0x5563b5a02400 2023-05-20T01:25:35.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.731+0000 7fec7654f700 1 -- 172.21.15.16:0/3755306105 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5563b572d980 con 0x5563b5a02400 2023-05-20T01:25:35.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.731+0000 7fec7e5af2c0 1 -- 172.21.15.16:0/3755306105 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5563b56ffd40 con 0x5563b5a02400 2023-05-20T01:25:35.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.731+0000 7fec7654f700 1 -- 172.21.15.16:0/3755306105 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5563b570f080 con 0x5563b5a02400 2023-05-20T01:25:35.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.731+0000 7fec7e5af2c0 1 -- 172.21.15.16:0/3755306105 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5563b56fed00 con 0x5563b5a02400 2023-05-20T01:25:35.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.731+0000 7fec7e5af2c0 4 main: resolving pool 12 2023-05-20T01:25:35.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.732+0000 7fec7654f700 1 -- 172.21.15.16:0/3755306105 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x5563b573c840 con 0x5563b5a02400 2023-05-20T01:25:35.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.732+0000 7fec7654f700 1 --2- 172.21.15.16:0/3755306105 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x5563b5a08400 0x5563b5af4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:35.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.732+0000 7fec7654f700 1 -- 172.21.15.16:0/3755306105 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6958+0+0 (secure 0 0 0) 0x5563b571ca80 con 0x5563b5a02400 2023-05-20T01:25:35.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.732+0000 7fec7e5af2c0 4 main: creating IoCtx.. 2023-05-20T01:25:35.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.732+0000 7fec7c55b700 1 --2- 172.21.15.16:0/3755306105 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x5563b5a08400 0x5563b5af4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:35.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.732+0000 7fec7e5af2c0 1 --2- 172.21.15.16:0/3755306105 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x5563b5a02800 0x5563b5776680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:35.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.733+0000 7fec7e5af2c0 1 -- 172.21.15.16:0/3755306105 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:1 12.d 12:b50e409b:::mds_snaptable:head [writefull 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x5563b655a700 con 0x5563b5a02800 2023-05-20T01:25:35.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.733+0000 7fec7c55b700 1 --2- 172.21.15.16:0/3755306105 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x5563b5a08400 0x5563b5af4000 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x5563b6455c80 tx=0x5563b6563ec0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:35.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.733+0000 7fec7bd5a700 1 --2- 172.21.15.16:0/3755306105 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x5563b5a02800 0x5563b5776680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:35.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.733+0000 7fec7bd5a700 1 --2- 172.21.15.16:0/3755306105 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x5563b5a02800 0x5563b5776680 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:35.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.735+0000 7fec7bd5a700 1 -- 172.21.15.16:0/3755306105 <== osd.7 v2:172.21.15.132:6800/466001149 1 ==== osd_op_reply(1 mds_snaptable [writefull 0~70] v93'3 uv3 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x5563b653efc0 con 0x5563b5a02800 2023-05-20T01:25:35.754 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-05-20T01:25:35.754 INFO:teuthology.orchestra.run.smithi016.stdout: "result": 0 2023-05-20T01:25:35.755 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-05-20T01:25:35.755 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:25:35.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.735+0000 7fec7e5af2c0 1 -- 172.21.15.16:0/3755306105 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x5563b5a02800 msgr2=0x5563b5776680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:35.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.735+0000 7fec7e5af2c0 1 --2- 172.21.15.16:0/3755306105 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x5563b5a02800 0x5563b5776680 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:35.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.735+0000 7fec7e5af2c0 1 -- 172.21.15.16:0/3755306105 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x5563b5a08400 msgr2=0x5563b5af4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:35.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.735+0000 7fec7e5af2c0 1 --2- 172.21.15.16:0/3755306105 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x5563b5a08400 0x5563b5af4000 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x5563b6455c80 tx=0x5563b6563ec0 comp rx=0 tx=0).stop 2023-05-20T01:25:35.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.735+0000 7fec7e5af2c0 1 -- 172.21.15.16:0/3755306105 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563b5a02400 msgr2=0x5563b5776100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:35.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.735+0000 7fec7e5af2c0 1 --2- 172.21.15.16:0/3755306105 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563b5a02400 0x5563b5776100 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x5563b5a00660 tx=0x5563b6562c00 comp rx=0 tx=0).stop 2023-05-20T01:25:35.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.735+0000 7fec7c55b700 1 -- 172.21.15.16:0/3755306105 reap_dead start 2023-05-20T01:25:35.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.735+0000 7fec7e5af2c0 1 -- 172.21.15.16:0/3755306105 shutdown_connections 2023-05-20T01:25:35.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.735+0000 7fec7e5af2c0 1 -- 172.21.15.16:0/3755306105 >> 172.21.15.16:0/3755306105 conn(0x5563b56dbc00 msgr2=0x5563b57b7800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:35.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.735+0000 7fec7e5af2c0 1 -- 172.21.15.16:0/3755306105 shutdown_connections 2023-05-20T01:25:35.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.735+0000 7fec7e5af2c0 1 -- 172.21.15.16:0/3755306105 wait complete. 2023-05-20T01:25:35.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.735+0000 7fec7e5af2c0 1 -- 172.21.15.16:0/2001254912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563b56da800 msgr2=0x5563b5775080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:35.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.735+0000 7fec7e5af2c0 1 --2- 172.21.15.16:0/2001254912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563b56da800 0x5563b5775080 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x5563b6542630 tx=0x5563b64733e0 comp rx=0 tx=0).stop 2023-05-20T01:25:35.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.736+0000 7fec7e5af2c0 1 -- 172.21.15.16:0/2001254912 shutdown_connections 2023-05-20T01:25:35.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.736+0000 7fec7e5af2c0 1 --2- 172.21.15.16:0/2001254912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563b56da800 0x5563b5775080 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:35.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.736+0000 7fec7e5af2c0 1 --2- 172.21.15.16:0/2001254912 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5563b56db000 0x5563b5775b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:35.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.736+0000 7fec7e5af2c0 1 --2- 172.21.15.16:0/2001254912 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5563b56dac00 0x5563b5775600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:35.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.736+0000 7fec7e5af2c0 1 -- 172.21.15.16:0/2001254912 >> 172.21.15.16:0/2001254912 conn(0x5563b56db400 msgr2=0x5563b57b7000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:35.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.736+0000 7fec7e5af2c0 1 -- 172.21.15.16:0/2001254912 shutdown_connections 2023-05-20T01:25:35.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.736+0000 7fec7e5af2c0 1 -- 172.21.15.16:0/2001254912 wait complete. 2023-05-20T01:25:35.765 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-table-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'cephfs:0', 'reset', 'snap'] in time 0:00:00.114297, result: { "result": 0 } 2023-05-20T01:25:35.766 DEBUG:teuthology.orchestra.run.smithi016:> cephfs-table-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 cephfs:0 reset inode 2023-05-20T01:25:35.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.836+0000 7ff3259ac2c0 1 -- start start 2023-05-20T01:25:35.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.836+0000 7ff3259ac2c0 1 Processor -- start 2023-05-20T01:25:35.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.837+0000 7ff3259ac2c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5611706cb000 0x561170764580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:35.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.837+0000 7ff3259ac2c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5611706cac00 0x561170764b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:35.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.837+0000 7ff3259ac2c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5611706ca800 0x561170765080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:35.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.837+0000 7ff3259ac2c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x561170707cc0 con 0x5611706cac00 2023-05-20T01:25:35.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.837+0000 7ff3259ac2c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x561170707400 con 0x5611706ca800 2023-05-20T01:25:35.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.837+0000 7ff3259ac2c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x561170707e00 con 0x5611706cb000 2023-05-20T01:25:35.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.837+0000 7ff323157700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5611706cac00 0x561170764b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:35.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.837+0000 7ff323157700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5611706cac00 0x561170764b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36872/0 (socket says 172.21.15.16:36872) 2023-05-20T01:25:35.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.837+0000 7ff323157700 1 -- 172.21.15.16:0/4097255097 learned_addr learned my addr 172.21.15.16:0/4097255097 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:35.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.837+0000 7ff324159700 1 --2- 172.21.15.16:0/4097255097 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5611706ca800 0x561170765080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:35.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.837+0000 7ff323958700 1 --2- 172.21.15.16:0/4097255097 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5611706cb000 0x561170764580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:35.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.837+0000 7ff323157700 1 -- 172.21.15.16:0/4097255097 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5611706cb000 msgr2=0x561170764580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:35.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.837+0000 7ff323157700 1 --2- 172.21.15.16:0/4097255097 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5611706cb000 0x561170764580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:35.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.837+0000 7ff323157700 1 -- 172.21.15.16:0/4097255097 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5611706ca800 msgr2=0x561170765080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:35.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.837+0000 7ff323157700 1 --2- 172.21.15.16:0/4097255097 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5611706ca800 0x561170765080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:35.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.837+0000 7ff323157700 1 -- 172.21.15.16:0/4097255097 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5611706ee4e0 con 0x5611706cac00 2023-05-20T01:25:35.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.837+0000 7ff323958700 1 --2- 172.21.15.16:0/4097255097 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5611706cb000 0x561170764580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:25:35.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.837+0000 7ff324159700 1 --2- 172.21.15.16:0/4097255097 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5611706ca800 0x561170765080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:25:35.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.837+0000 7ff323157700 1 --2- 172.21.15.16:0/4097255097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5611706cac00 0x561170764b00 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x5611715466c0 tx=0x5611714a6de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79ebffa8820ce583 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:35.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.837+0000 7ff321153700 1 -- 172.21.15.16:0/4097255097 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5611706ff600 con 0x5611706cac00 2023-05-20T01:25:35.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.837+0000 7ff321153700 1 -- 172.21.15.16:0/4097255097 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56117071c480 con 0x5611706cac00 2023-05-20T01:25:35.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.837+0000 7ff321153700 1 -- 172.21.15.16:0/4097255097 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5611706ff1e0 con 0x5611706cac00 2023-05-20T01:25:35.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.837+0000 7ff3259ac2c0 1 -- 172.21.15.16:0/4097255097 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5611706ef520 con 0x5611706cac00 2023-05-20T01:25:35.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.838+0000 7ff3259ac2c0 1 -- 172.21.15.16:0/4097255097 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x5611706ef380 con 0x5611706cac00 2023-05-20T01:25:35.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.838+0000 7ff3259ac2c0 4 waiting for MDS map... 2023-05-20T01:25:35.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.838+0000 7ff321153700 1 -- 172.21.15.16:0/4097255097 <== mon.0 v2:172.21.15.16:3300/0 4 ==== osd_map(93..93 src has 1..93) v4 ==== 6958+0+0 (secure 0 0 0) 0x56117070cc40 con 0x5611706cac00 2023-05-20T01:25:35.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.838+0000 7ff321153700 1 -- 172.21.15.16:0/4097255097 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 63) v1 ==== 3126+0+0 (secure 0 0 0) 0x561171551200 con 0x5611706cac00 2023-05-20T01:25:35.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.838+0000 7ff3259ac2c0 4 Got MDS map 63 2023-05-20T01:25:35.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.838+0000 7ff3259ac2c0 10 main: main 2023-05-20T01:25:35.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.838+0000 7ff3259ac2c0 4 main: connecting to RADOS... 2023-05-20T01:25:35.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.838+0000 7ff3259ac2c0 1 Processor -- start 2023-05-20T01:25:35.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.838+0000 7ff3259ac2c0 1 -- start start 2023-05-20T01:25:35.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.839+0000 7ff3259ac2c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5611706cbc00 0x561170765600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:35.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.839+0000 7ff3259ac2c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5611709e8000 0x561170765b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:35.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.839+0000 7ff3259ac2c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5611709e8400 0x561170766100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:35.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.839+0000 7ff3259ac2c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x561170706a00 con 0x5611709e8400 2023-05-20T01:25:35.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.839+0000 7ff3259ac2c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x561170707400 con 0x5611709e8000 2023-05-20T01:25:35.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.839+0000 7ff3259ac2c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x561170706b40 con 0x5611706cbc00 2023-05-20T01:25:35.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.839+0000 7ff323157700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5611709e8400 0x561170766100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:35.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.839+0000 7ff323157700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5611709e8400 0x561170766100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36880/0 (socket says 172.21.15.16:36880) 2023-05-20T01:25:35.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.839+0000 7ff323157700 1 -- 172.21.15.16:0/799440678 learned_addr learned my addr 172.21.15.16:0/799440678 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:35.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.839+0000 7ff324159700 1 --2- 172.21.15.16:0/799440678 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5611706cbc00 0x561170765600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:35.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.839+0000 7ff323958700 1 --2- 172.21.15.16:0/799440678 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5611709e8000 0x561170765b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:35.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.839+0000 7ff323157700 1 -- 172.21.15.16:0/799440678 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5611706cbc00 msgr2=0x561170765600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:35.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.839+0000 7ff323157700 1 --2- 172.21.15.16:0/799440678 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5611706cbc00 0x561170765600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:35.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.839+0000 7ff323157700 1 -- 172.21.15.16:0/799440678 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5611709e8000 msgr2=0x561170765b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:35.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.839+0000 7ff323157700 1 --2- 172.21.15.16:0/799440678 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5611709e8000 0x561170765b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:35.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.839+0000 7ff323157700 1 -- 172.21.15.16:0/799440678 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5611706efd40 con 0x5611709e8400 2023-05-20T01:25:35.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.839+0000 7ff323958700 1 --2- 172.21.15.16:0/799440678 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5611709e8000 0x561170765b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:25:35.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.839+0000 7ff323157700 1 --2- 172.21.15.16:0/799440678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5611709e8400 0x561170766100 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x561171547590 tx=0x5611714a6000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ce06829ee2d627a server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:35.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.840+0000 7ff31f14f700 1 -- 172.21.15.16:0/799440678 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5611706fef20 con 0x5611709e8400 2023-05-20T01:25:35.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.840+0000 7ff31f14f700 1 -- 172.21.15.16:0/799440678 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561171562c00 con 0x5611709e8400 2023-05-20T01:25:35.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.840+0000 7ff31f14f700 1 -- 172.21.15.16:0/799440678 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5611706ff600 con 0x5611709e8400 2023-05-20T01:25:35.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.840+0000 7ff3259ac2c0 1 -- 172.21.15.16:0/799440678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5611709e8400 msgr2=0x561170766100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:35.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.840+0000 7ff3259ac2c0 1 --2- 172.21.15.16:0/799440678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5611709e8400 0x561170766100 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x561171547590 tx=0x5611714a6000 comp rx=0 tx=0).stop 2023-05-20T01:25:35.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.840+0000 7ff3259ac2c0 1 -- 172.21.15.16:0/799440678 shutdown_connections 2023-05-20T01:25:35.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.840+0000 7ff3259ac2c0 1 --2- 172.21.15.16:0/799440678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5611709e8400 0x561170766100 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:35.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.840+0000 7ff3259ac2c0 1 --2- 172.21.15.16:0/799440678 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5611706cbc00 0x561170765600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:35.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.840+0000 7ff3259ac2c0 1 --2- 172.21.15.16:0/799440678 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5611709e8000 0x561170765b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:35.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.840+0000 7ff3259ac2c0 1 -- 172.21.15.16:0/799440678 >> 172.21.15.16:0/799440678 conn(0x5611706ca400 msgr2=0x5611707a7800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:35.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.840+0000 7ff3259ac2c0 1 -- 172.21.15.16:0/799440678 shutdown_connections 2023-05-20T01:25:35.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.840+0000 7ff3259ac2c0 1 -- 172.21.15.16:0/799440678 wait complete. 2023-05-20T01:25:35.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.841+0000 7ff3259ac2c0 1 Processor -- start 2023-05-20T01:25:35.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.841+0000 7ff3259ac2c0 1 -- start start 2023-05-20T01:25:35.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.841+0000 7ff3259ac2c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5611706ca400 0x561170767180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:35.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.841+0000 7ff3259ac2c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5611709fa000 0x561170766c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:35.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.841+0000 7ff3259ac2c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5611709fa400 0x561170766680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:35.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.841+0000 7ff3259ac2c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x561170706c80 con 0x5611709fa000 2023-05-20T01:25:35.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.841+0000 7ff3259ac2c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x561170706000 con 0x5611706ca400 2023-05-20T01:25:35.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.841+0000 7ff3259ac2c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x561170707180 con 0x5611709fa400 2023-05-20T01:25:35.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.841+0000 7ff323958700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5611709fa000 0x561170766c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:35.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.841+0000 7ff323958700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5611709fa000 0x561170766c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36884/0 (socket says 172.21.15.16:36884) 2023-05-20T01:25:35.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.841+0000 7ff323958700 1 -- 172.21.15.16:0/1921288811 learned_addr learned my addr 172.21.15.16:0/1921288811 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:35.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.841+0000 7ff324159700 1 --2- 172.21.15.16:0/1921288811 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5611706ca400 0x561170767180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:35.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.842+0000 7ff323157700 1 --2- 172.21.15.16:0/1921288811 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5611709fa400 0x561170766680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:35.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.842+0000 7ff323958700 1 -- 172.21.15.16:0/1921288811 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5611709fa400 msgr2=0x561170766680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:35.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.842+0000 7ff323958700 1 --2- 172.21.15.16:0/1921288811 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5611709fa400 0x561170766680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:35.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.842+0000 7ff323958700 1 -- 172.21.15.16:0/1921288811 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5611706ca400 msgr2=0x561170767180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:35.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.842+0000 7ff323958700 1 --2- 172.21.15.16:0/1921288811 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5611706ca400 0x561170767180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:35.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.842+0000 7ff323958700 1 -- 172.21.15.16:0/1921288811 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5611709f61a0 con 0x5611709fa000 2023-05-20T01:25:35.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.842+0000 7ff323958700 1 --2- 172.21.15.16:0/1921288811 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5611709fa000 0x561170766c00 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x5611709f8c30 tx=0x5611714633e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:35.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.842+0000 7ff31d94c700 1 -- 172.21.15.16:0/1921288811 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5611706ff080 con 0x5611709fa000 2023-05-20T01:25:35.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.842+0000 7ff31d94c700 1 -- 172.21.15.16:0/1921288811 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56117071d980 con 0x5611709fa000 2023-05-20T01:25:35.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.842+0000 7ff31d94c700 1 -- 172.21.15.16:0/1921288811 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5611706fef20 con 0x5611709fa000 2023-05-20T01:25:35.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.842+0000 7ff3259ac2c0 1 -- 172.21.15.16:0/1921288811 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5611709f6000 con 0x5611709fa000 2023-05-20T01:25:35.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.842+0000 7ff3259ac2c0 1 -- 172.21.15.16:0/1921288811 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5611709f6340 con 0x5611709fa000 2023-05-20T01:25:35.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.842+0000 7ff3259ac2c0 4 main: resolving pool 12 2023-05-20T01:25:35.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.843+0000 7ff31d94c700 1 -- 172.21.15.16:0/1921288811 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x56117072c840 con 0x5611709fa000 2023-05-20T01:25:35.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.844+0000 7ff31d94c700 1 --2- 172.21.15.16:0/1921288811 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x561170a0a400 0x561170767700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:35.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.844+0000 7ff31d94c700 1 -- 172.21.15.16:0/1921288811 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6958+0+0 (secure 0 0 0) 0x56117070ca80 con 0x5611709fa000 2023-05-20T01:25:35.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.844+0000 7ff3259ac2c0 4 main: creating IoCtx.. 2023-05-20T01:25:35.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.844+0000 7ff324159700 1 --2- 172.21.15.16:0/1921288811 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x561170a0a400 0x561170767700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:35.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.844+0000 7ff3259ac2c0 1 --2- 172.21.15.16:0/1921288811 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x5611709fa800 0x561170766100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:35.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.844+0000 7ff3259ac2c0 1 -- 172.21.15.16:0/1921288811 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:1 12.13 12:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x56117155c700 con 0x5611709fa800 2023-05-20T01:25:35.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.844+0000 7ff323157700 1 --2- 172.21.15.16:0/1921288811 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x5611709fa800 0x561170766100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:35.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.844+0000 7ff324159700 1 --2- 172.21.15.16:0/1921288811 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x561170a0a400 0x561170767700 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x561171445d10 tx=0x561170afc000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:35.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.844+0000 7ff323157700 1 --2- 172.21.15.16:0/1921288811 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x5611709fa800 0x561170766100 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:35.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.846+0000 7ff323157700 1 -- 172.21.15.16:0/1921288811 <== osd.1 v2:172.21.15.16:6808/3860264220 1 ==== osd_op_reply(1 mds0_inotable [writefull 0~34] v93'6 uv6 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x561171542fc0 con 0x5611709fa800 2023-05-20T01:25:35.879 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-05-20T01:25:35.879 INFO:teuthology.orchestra.run.smithi016.stdout: "0": { 2023-05-20T01:25:35.880 INFO:teuthology.orchestra.run.smithi016.stdout: "data": {}, 2023-05-20T01:25:35.880 INFO:teuthology.orchestra.run.smithi016.stdout: "result": 0 2023-05-20T01:25:35.880 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:25:35.880 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-05-20T01:25:35.880 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:25:35.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.846+0000 7ff3259ac2c0 1 -- 172.21.15.16:0/1921288811 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x5611709fa800 msgr2=0x561170766100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:35.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.846+0000 7ff3259ac2c0 1 --2- 172.21.15.16:0/1921288811 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x5611709fa800 0x561170766100 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:35.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.846+0000 7ff3259ac2c0 1 -- 172.21.15.16:0/1921288811 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x561170a0a400 msgr2=0x561170767700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:35.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.846+0000 7ff3259ac2c0 1 --2- 172.21.15.16:0/1921288811 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x561170a0a400 0x561170767700 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x561171445d10 tx=0x561170afc000 comp rx=0 tx=0).stop 2023-05-20T01:25:35.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.846+0000 7ff3259ac2c0 1 -- 172.21.15.16:0/1921288811 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5611709fa000 msgr2=0x561170766c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:35.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.846+0000 7ff3259ac2c0 1 --2- 172.21.15.16:0/1921288811 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5611709fa000 0x561170766c00 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x5611709f8c30 tx=0x5611714633e0 comp rx=0 tx=0).stop 2023-05-20T01:25:35.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.846+0000 7ff323958700 1 -- 172.21.15.16:0/1921288811 reap_dead start 2023-05-20T01:25:35.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.846+0000 7ff3259ac2c0 1 -- 172.21.15.16:0/1921288811 shutdown_connections 2023-05-20T01:25:35.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.846+0000 7ff3259ac2c0 1 -- 172.21.15.16:0/1921288811 >> 172.21.15.16:0/1921288811 conn(0x5611706cbc00 msgr2=0x5611707a7800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:35.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.846+0000 7ff3259ac2c0 1 -- 172.21.15.16:0/1921288811 shutdown_connections 2023-05-20T01:25:35.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.846+0000 7ff3259ac2c0 1 -- 172.21.15.16:0/1921288811 wait complete. 2023-05-20T01:25:35.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.846+0000 7ff3259ac2c0 1 -- 172.21.15.16:0/4097255097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5611706cac00 msgr2=0x561170764b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:35.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.846+0000 7ff3259ac2c0 1 --2- 172.21.15.16:0/4097255097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5611706cac00 0x561170764b00 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x5611715466c0 tx=0x5611714a6de0 comp rx=0 tx=0).stop 2023-05-20T01:25:35.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.846+0000 7ff3259ac2c0 1 -- 172.21.15.16:0/4097255097 shutdown_connections 2023-05-20T01:25:35.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.846+0000 7ff3259ac2c0 1 --2- 172.21.15.16:0/4097255097 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5611706ca800 0x561170765080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:35.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.846+0000 7ff3259ac2c0 1 --2- 172.21.15.16:0/4097255097 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5611706cb000 0x561170764580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:35.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.846+0000 7ff3259ac2c0 1 --2- 172.21.15.16:0/4097255097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5611706cac00 0x561170764b00 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:35.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.846+0000 7ff3259ac2c0 1 -- 172.21.15.16:0/4097255097 >> 172.21.15.16:0/4097255097 conn(0x5611706cb400 msgr2=0x5611707a7000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:35.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.846+0000 7ff3259ac2c0 1 -- 172.21.15.16:0/4097255097 shutdown_connections 2023-05-20T01:25:35.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.846+0000 7ff3259ac2c0 1 -- 172.21.15.16:0/4097255097 wait complete. 2023-05-20T01:25:35.885 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-table-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'cephfs:0', 'reset', 'inode'] in time 0:00:00.118861, result: { "0": { "data": {}, "result": 0 } } 2023-05-20T01:25:35.886 DEBUG:teuthology.orchestra.run.smithi016:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 journal reset --force 2023-05-20T01:25:35.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.962+0000 7f8b600ba2c0 1 -- start start 2023-05-20T01:25:35.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.962+0000 7f8b600ba2c0 1 Processor -- start 2023-05-20T01:25:35.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.963+0000 7f8b600ba2c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55be6d823000 0x55be6d8bc580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:35.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.963+0000 7f8b600ba2c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55be6d822c00 0x55be6d8bcb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:35.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.963+0000 7f8b600ba2c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be6d822800 0x55be6d8bd080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:35.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.963+0000 7f8b600ba2c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be6d85fcc0 con 0x55be6d822800 2023-05-20T01:25:35.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.963+0000 7f8b600ba2c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be6d85f400 con 0x55be6d823000 2023-05-20T01:25:35.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.963+0000 7f8b600ba2c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55be6d85fe00 con 0x55be6d822c00 2023-05-20T01:25:35.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.963+0000 7f8b5e867700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be6d822800 0x55be6d8bd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:35.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.963+0000 7f8b5e867700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be6d822800 0x55be6d8bd080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36900/0 (socket says 172.21.15.16:36900) 2023-05-20T01:25:35.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.963+0000 7f8b5e867700 1 -- 172.21.15.16:0/2527693782 learned_addr learned my addr 172.21.15.16:0/2527693782 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:35.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.963+0000 7f8b5e867700 1 -- 172.21.15.16:0/2527693782 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55be6d822c00 msgr2=0x55be6d8bcb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:35.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.963+0000 7f8b5e066700 1 --2- 172.21.15.16:0/2527693782 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55be6d823000 0x55be6d8bc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:35.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.963+0000 7f8b5e867700 1 --2- 172.21.15.16:0/2527693782 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55be6d822c00 0x55be6d8bcb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:35.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.963+0000 7f8b5e867700 1 -- 172.21.15.16:0/2527693782 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55be6d823000 msgr2=0x55be6d8bc580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:35.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.963+0000 7f8b5e867700 1 --2- 172.21.15.16:0/2527693782 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55be6d823000 0x55be6d8bc580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:35.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.963+0000 7f8b5e867700 1 -- 172.21.15.16:0/2527693782 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be6d8464e0 con 0x55be6d822800 2023-05-20T01:25:35.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.963+0000 7f8b5e867700 1 --2- 172.21.15.16:0/2527693782 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be6d822800 0x55be6d8bd080 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x55be6e6846c0 tx=0x55be6e69c0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a1a0b183f35cf8b server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:35.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.964+0000 7f8b5b861700 1 -- 172.21.15.16:0/2527693782 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55be6d857600 con 0x55be6d822800 2023-05-20T01:25:35.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.964+0000 7f8b5b861700 1 -- 172.21.15.16:0/2527693782 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55be6d874480 con 0x55be6d822800 2023-05-20T01:25:35.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.964+0000 7f8b5b861700 1 -- 172.21.15.16:0/2527693782 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55be6d8571e0 con 0x55be6d822800 2023-05-20T01:25:35.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.964+0000 7f8b600ba2c0 1 -- 172.21.15.16:0/2527693782 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55be6d847520 con 0x55be6d822800 2023-05-20T01:25:35.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.964+0000 7f8b600ba2c0 1 -- 172.21.15.16:0/2527693782 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55be6d847380 con 0x55be6d822800 2023-05-20T01:25:35.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.964+0000 7f8b600ba2c0 4 waiting for MDS map... 2023-05-20T01:25:35.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.964+0000 7f8b5b861700 1 -- 172.21.15.16:0/2527693782 <== mon.0 v2:172.21.15.16:3300/0 4 ==== osd_map(93..93 src has 1..93) v4 ==== 6958+0+0 (secure 0 0 0) 0x55be6d864c40 con 0x55be6d822800 2023-05-20T01:25:35.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.964+0000 7f8b5b861700 1 -- 172.21.15.16:0/2527693782 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 63) v1 ==== 3126+0+0 (secure 0 0 0) 0x55be6e69f200 con 0x55be6d822800 2023-05-20T01:25:35.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.964+0000 7f8b600ba2c0 4 Got MDS map 63 2023-05-20T01:25:35.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.964+0000 7f8b600ba2c0 10 main: JournalTool::main 2023-05-20T01:25:35.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.964+0000 7f8b600ba2c0 4 main: JournalTool: connecting to RADOS... 2023-05-20T01:25:35.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.964+0000 7f8b600ba2c0 1 Processor -- start 2023-05-20T01:25:35.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.964+0000 7f8b600ba2c0 1 -- start start 2023-05-20T01:25:35.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.965+0000 7f8b600ba2c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55be6d823c00 0x55be6d8bd600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:35.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.965+0000 7f8b600ba2c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55be6db32000 0x55be6d8bdb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:35.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.965+0000 7f8b600ba2c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be6db32400 0x55be6d8be100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:35.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.965+0000 7f8b600ba2c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be6d85ea00 con 0x55be6db32400 2023-05-20T01:25:35.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.965+0000 7f8b600ba2c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be6d85f400 con 0x55be6db32000 2023-05-20T01:25:35.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.965+0000 7f8b600ba2c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55be6d85eb40 con 0x55be6d823c00 2023-05-20T01:25:35.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.965+0000 7f8b5d865700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be6db32400 0x55be6d8be100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:35.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.965+0000 7f8b5d865700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be6db32400 0x55be6d8be100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36908/0 (socket says 172.21.15.16:36908) 2023-05-20T01:25:35.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.965+0000 7f8b5d865700 1 -- 172.21.15.16:0/5237417 learned_addr learned my addr 172.21.15.16:0/5237417 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:35.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.965+0000 7f8b5e066700 1 --2- 172.21.15.16:0/5237417 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55be6db32000 0x55be6d8bdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:35.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.965+0000 7f8b5d865700 1 --2- 172.21.15.16:0/5237417 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55be6d823c00 0x55be6d8bd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:35.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.965+0000 7f8b5d865700 1 -- 172.21.15.16:0/5237417 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55be6d823c00 msgr2=0x55be6d8bd600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:35.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.965+0000 7f8b5d865700 1 --2- 172.21.15.16:0/5237417 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55be6d823c00 0x55be6d8bd600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:35.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.965+0000 7f8b5d865700 1 -- 172.21.15.16:0/5237417 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55be6db32000 msgr2=0x55be6d8bdb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:35.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.965+0000 7f8b5d865700 1 --2- 172.21.15.16:0/5237417 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55be6db32000 0x55be6d8bdb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:35.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.965+0000 7f8b5d865700 1 -- 172.21.15.16:0/5237417 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be6d847d40 con 0x55be6db32400 2023-05-20T01:25:35.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.966+0000 7f8b5d865700 1 --2- 172.21.15.16:0/5237417 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be6db32400 0x55be6d8be100 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x55be6e685cb0 tx=0x55be6e6b2d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57206a37c5babc28 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:35.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.966+0000 7f8b5985d700 1 -- 172.21.15.16:0/5237417 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55be6d857600 con 0x55be6db32400 2023-05-20T01:25:35.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.966+0000 7f8b5985d700 1 -- 172.21.15.16:0/5237417 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55be6d875e00 con 0x55be6db32400 2023-05-20T01:25:35.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.966+0000 7f8b5985d700 1 -- 172.21.15.16:0/5237417 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55be6d8571e0 con 0x55be6db32400 2023-05-20T01:25:35.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.966+0000 7f8b600ba2c0 1 -- 172.21.15.16:0/5237417 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be6db32400 msgr2=0x55be6d8be100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:35.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.966+0000 7f8b600ba2c0 1 --2- 172.21.15.16:0/5237417 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be6db32400 0x55be6d8be100 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x55be6e685cb0 tx=0x55be6e6b2d80 comp rx=0 tx=0).stop 2023-05-20T01:25:35.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.966+0000 7f8b600ba2c0 1 -- 172.21.15.16:0/5237417 shutdown_connections 2023-05-20T01:25:35.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.966+0000 7f8b600ba2c0 1 --2- 172.21.15.16:0/5237417 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be6db32400 0x55be6d8be100 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:35.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.966+0000 7f8b600ba2c0 1 --2- 172.21.15.16:0/5237417 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55be6d823c00 0x55be6d8bd600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:35.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.966+0000 7f8b600ba2c0 1 --2- 172.21.15.16:0/5237417 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55be6db32000 0x55be6d8bdb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:35.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.966+0000 7f8b600ba2c0 1 -- 172.21.15.16:0/5237417 >> 172.21.15.16:0/5237417 conn(0x55be6d822400 msgr2=0x55be6d8ff800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:35.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.966+0000 7f8b600ba2c0 1 -- 172.21.15.16:0/5237417 shutdown_connections 2023-05-20T01:25:35.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.966+0000 7f8b600ba2c0 1 -- 172.21.15.16:0/5237417 wait complete. 2023-05-20T01:25:35.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.967+0000 7f8b600ba2c0 1 Processor -- start 2023-05-20T01:25:35.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.967+0000 7f8b600ba2c0 1 -- start start 2023-05-20T01:25:35.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.967+0000 7f8b600ba2c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55be6d822400 0x55be6d8bf180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:35.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.967+0000 7f8b600ba2c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55be6db56000 0x55be6d8bec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:35.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.967+0000 7f8b600ba2c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be6db56400 0x55be6d8be680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:35.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.967+0000 7f8b600ba2c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be6d85ec80 con 0x55be6db56400 2023-05-20T01:25:35.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.967+0000 7f8b600ba2c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be6d85e000 con 0x55be6d822400 2023-05-20T01:25:35.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.967+0000 7f8b600ba2c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55be6d85f180 con 0x55be6db56000 2023-05-20T01:25:35.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.968+0000 7f8b5d865700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be6db56400 0x55be6d8be680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:35.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.968+0000 7f8b5d865700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be6db56400 0x55be6d8be680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36914/0 (socket says 172.21.15.16:36914) 2023-05-20T01:25:35.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.968+0000 7f8b5d865700 1 -- 172.21.15.16:0/3443922796 learned_addr learned my addr 172.21.15.16:0/3443922796 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:35.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.968+0000 7f8b5e066700 1 --2- 172.21.15.16:0/3443922796 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55be6db56000 0x55be6d8bec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:35.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.968+0000 7f8b5d865700 1 --2- 172.21.15.16:0/3443922796 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55be6d822400 0x55be6d8bf180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:35.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.968+0000 7f8b5d865700 1 -- 172.21.15.16:0/3443922796 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55be6db56000 msgr2=0x55be6d8bec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:35.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.968+0000 7f8b5d865700 1 --2- 172.21.15.16:0/3443922796 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55be6db56000 0x55be6d8bec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:35.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.968+0000 7f8b5d865700 1 -- 172.21.15.16:0/3443922796 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55be6d822400 msgr2=0x55be6d8bf180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:35.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.968+0000 7f8b5d865700 1 --2- 172.21.15.16:0/3443922796 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55be6d822400 0x55be6d8bf180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:35.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.968+0000 7f8b5d865700 1 -- 172.21.15.16:0/3443922796 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be6db4a1a0 con 0x55be6db56400 2023-05-20T01:25:35.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.968+0000 7f8b5d865700 1 --2- 172.21.15.16:0/3443922796 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be6db56400 0x55be6d8be680 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x55be6e685f20 tx=0x55be6e5bb140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:35.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.968+0000 7f8b5805a700 1 -- 172.21.15.16:0/3443922796 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55be6d856dc0 con 0x55be6db56400 2023-05-20T01:25:35.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.968+0000 7f8b5805a700 1 -- 172.21.15.16:0/3443922796 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55be6e6acc00 con 0x55be6db56400 2023-05-20T01:25:35.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.968+0000 7f8b600ba2c0 1 -- 172.21.15.16:0/3443922796 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55be6db4a000 con 0x55be6db56400 2023-05-20T01:25:35.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.968+0000 7f8b5805a700 1 -- 172.21.15.16:0/3443922796 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55be6d857600 con 0x55be6db56400 2023-05-20T01:25:35.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.968+0000 7f8b600ba2c0 1 -- 172.21.15.16:0/3443922796 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55be6db4a340 con 0x55be6db56400 2023-05-20T01:25:35.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.968+0000 7f8b600ba2c0 4 main: JournalTool: resolving pool 12 2023-05-20T01:25:35.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.969+0000 7f8b5805a700 1 -- 172.21.15.16:0/3443922796 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x55be6d884840 con 0x55be6db56400 2023-05-20T01:25:35.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.970+0000 7f8b5805a700 1 --2- 172.21.15.16:0/3443922796 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x55be6db60400 0x55be6d8bf700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:35.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.970+0000 7f8b5805a700 1 -- 172.21.15.16:0/3443922796 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6958+0+0 (secure 0 0 0) 0x55be6d864a80 con 0x55be6db56400 2023-05-20T01:25:35.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.970+0000 7f8b5e066700 1 --2- 172.21.15.16:0/3443922796 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x55be6db60400 0x55be6d8bf700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:35.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.970+0000 7f8b600ba2c0 4 main: JournalTool: creating IoCtx.. 2023-05-20T01:25:35.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.970+0000 7f8b600ba2c0 4 main: Executing for rank 0 2023-05-20T01:25:35.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.970+0000 7f8b5e066700 1 --2- 172.21.15.16:0/3443922796 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x55be6db60400 0x55be6d8bf700 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x55be6db01020 tx=0x55be6e6b30e0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:35.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.970+0000 7f8b600ba2c0 1 -- start start 2023-05-20T01:25:35.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.970+0000 7f8b600ba2c0 1 Processor -- start 2023-05-20T01:25:35.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.971+0000 7f8b600ba2c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55be6db56c00 0x55be6d8be100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:35.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.971+0000 7f8b600ba2c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be6db57000 0x55be6d8bdb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:35.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.971+0000 7f8b600ba2c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55be6db57400 0x55be6d8bd600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:35.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.971+0000 7f8b600ba2c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be6d85f180 con 0x55be6db57000 2023-05-20T01:25:35.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.971+0000 7f8b600ba2c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be6d85e000 con 0x55be6db57400 2023-05-20T01:25:35.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.971+0000 7f8b600ba2c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55be6d85eb40 con 0x55be6db56c00 2023-05-20T01:25:35.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.971+0000 7f8b5e066700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be6db57000 0x55be6d8bdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:35.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.971+0000 7f8b5e066700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be6db57000 0x55be6d8bdb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36920/0 (socket says 172.21.15.16:36920) 2023-05-20T01:25:35.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.971+0000 7f8b5e066700 1 -- 172.21.15.16:0/805712339 learned_addr learned my addr 172.21.15.16:0/805712339 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:35.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.971+0000 7f8b5e066700 1 -- 172.21.15.16:0/805712339 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55be6db56c00 msgr2=0x55be6d8be100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:35.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.971+0000 7f8b5e066700 1 --2- 172.21.15.16:0/805712339 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55be6db56c00 0x55be6d8be100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:35.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.971+0000 7f8b5e066700 1 -- 172.21.15.16:0/805712339 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55be6db57400 msgr2=0x55be6d8bd600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:35.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.971+0000 7f8b5e066700 1 --2- 172.21.15.16:0/805712339 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55be6db57400 0x55be6d8bd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:35.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.971+0000 7f8b5e066700 1 -- 172.21.15.16:0/805712339 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be6db4a680 con 0x55be6db57000 2023-05-20T01:25:35.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.972+0000 7f8b5e066700 1 --2- 172.21.15.16:0/805712339 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be6db57000 0x55be6d8bdb80 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x55be6dc2cff0 tx=0x55be6e6b3020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=323f6a43f9b63a7f server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:35.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.972+0000 7f8b55054700 1 -- 172.21.15.16:0/805712339 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55be6d857080 con 0x55be6db57000 2023-05-20T01:25:35.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.972+0000 7f8b55054700 1 -- 172.21.15.16:0/805712339 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55be6d875980 con 0x55be6db57000 2023-05-20T01:25:35.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.972+0000 7f8b55054700 1 -- 172.21.15.16:0/805712339 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55be6d8566e0 con 0x55be6db57000 2023-05-20T01:25:35.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.972+0000 7f8b600ba2c0 1 -- 172.21.15.16:0/805712339 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55be6db4a820 con 0x55be6db57000 2023-05-20T01:25:35.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.972+0000 7f8b600ba2c0 1 -- 172.21.15.16:0/805712339 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55be6db4a9c0 con 0x55be6db57000 2023-05-20T01:25:35.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.972+0000 7f8b600ba2c0 4 waiting for MDS map... 2023-05-20T01:25:35.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.972+0000 7f8b55054700 1 -- 172.21.15.16:0/805712339 <== mon.0 v2:172.21.15.16:3300/0 4 ==== osd_map(93..93 src has 1..93) v4 ==== 6958+0+0 (secure 0 0 0) 0x55be6d8648c0 con 0x55be6db57000 2023-05-20T01:25:35.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.972+0000 7f8b55054700 1 -- 172.21.15.16:0/805712339 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 63) v1 ==== 3126+0+0 (secure 0 0 0) 0x55be6dc4db00 con 0x55be6db57000 2023-05-20T01:25:35.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.972+0000 7f8b600ba2c0 4 Got MDS map 63 2023-05-20T01:25:35.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.972+0000 7f8b600ba2c0 1 --2- 172.21.15.16:0/805712339 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x55be6db57800 0x55be6dcb8000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:35.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.972+0000 7f8b600ba2c0 1 -- 172.21.15.16:0/805712339 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:1 12.f 12:f1f69726:::400.00000000:head [writefull 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55be6e6a8e00 con 0x55be6db57800 2023-05-20T01:25:35.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.972+0000 7f8b5e867700 1 --2- 172.21.15.16:0/805712339 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x55be6db57800 0x55be6dcb8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:35.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.973+0000 7f8b5e867700 1 --2- 172.21.15.16:0/805712339 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x55be6db57800 0x55be6dcb8000 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=b71cbea4a3e053b7 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:35.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.974+0000 7f8b5e867700 1 -- 172.21.15.16:0/805712339 <== osd.6 v2:172.21.15.132:6801/727581123 1 ==== osd_op_reply(1 400.00000000 [writefull 0~22] v93'4 uv4 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x55be6e683200 con 0x55be6db57800 2023-05-20T01:25:35.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.975+0000 7f8b600ba2c0 1 --2- 172.21.15.16:0/805712339 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x55be6db57c00 0x55be6dcb8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:35.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.975+0000 7f8b600ba2c0 1 -- 172.21.15.16:0/805712339 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:2 12.14 12:292cf221:::200.00000000:head [writefull 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55be6e6a9180 con 0x55be6db57c00 2023-05-20T01:25:35.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.975+0000 7f8b5d865700 1 --2- 172.21.15.16:0/805712339 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x55be6db57c00 0x55be6dcb8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:35.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.975+0000 7f8b5d865700 1 --2- 172.21.15.16:0/805712339 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x55be6db57c00 0x55be6dcb8580 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=4d39c23fe11a3c50 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:35.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.977+0000 7f8b5d865700 1 -- 172.21.15.16:0/805712339 <== osd.5 v2:172.21.15.132:6816/1844689995 1 ==== osd_op_reply(2 200.00000000 [writefull 0~90] v93'26 uv26 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x55be6e683440 con 0x55be6db57c00 2023-05-20T01:25:35.991 INFO:teuthology.orchestra.run.smithi016.stdout:writing EResetJournal entry 2023-05-20T01:25:35.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.977+0000 7f8b600ba2c0 1 -- 172.21.15.16:0/805712339 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:3 12.14 12:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55be6e6a9500 con 0x55be6db57c00 2023-05-20T01:25:35.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.977+0000 7f8b600ba2c0 1 --2- 172.21.15.16:0/805712339 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x55be6db60c00 0x55be6dcb8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:35.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.977+0000 7f8b600ba2c0 1 -- 172.21.15.16:0/805712339 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:4 12.1b 12:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55be6e6a9180 con 0x55be6db60c00 2023-05-20T01:25:35.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.977+0000 7f8b600ba2c0 1 --2- 172.21.15.16:0/805712339 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x55be6db61000 0x55be6dcb9080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:35.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.977+0000 7f8b600ba2c0 1 -- 172.21.15.16:0/805712339 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:5 12.b 12:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55be6e6a9500 con 0x55be6db61000 2023-05-20T01:25:35.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.977+0000 7f8b600ba2c0 1 -- 172.21.15.16:0/805712339 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:6 12.1f 12:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55be6e6a9880 con 0x55be6db57800 2023-05-20T01:25:35.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.977+0000 7f8b600ba2c0 1 -- 172.21.15.16:0/805712339 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:7 12.1a 12:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55be6e6a9c00 con 0x55be6db60c00 2023-05-20T01:25:35.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.977+0000 7f8b600ba2c0 1 -- 172.21.15.16:0/805712339 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:8 12.1e 12:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55be6dcd8000 con 0x55be6db60c00 2023-05-20T01:25:35.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.977+0000 7f8b5e066700 1 --2- 172.21.15.16:0/805712339 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x55be6db61000 0x55be6dcb9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:35.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.977+0000 7f8b5e867700 1 --2- 172.21.15.16:0/805712339 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x55be6db60c00 0x55be6dcb8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:35.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.977+0000 7f8b5e867700 1 --2- 172.21.15.16:0/805712339 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x55be6db60c00 0x55be6dcb8b00 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=fca27a96abb689ed server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:35.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.977+0000 7f8b5e066700 1 --2- 172.21.15.16:0/805712339 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x55be6db61000 0x55be6dcb9080 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=70721ab1e4a6890a server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:35.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.978+0000 7f8b5d865700 1 -- 172.21.15.16:0/805712339 <== osd.5 v2:172.21.15.132:6816/1844689995 2 ==== osd_op_reply(3 200.00000001 [delete] v93'27 uv26 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x55be6e683440 con 0x55be6db57c00 2023-05-20T01:25:35.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.978+0000 7f8b5e066700 1 -- 172.21.15.16:0/805712339 <== osd.3 v2:172.21.15.16:6800/3669781890 1 ==== osd_op_reply(5 200.00000003 [delete] v93'6 uv5 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x55be6e683b00 con 0x55be6db61000 2023-05-20T01:25:35.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.979+0000 7f8b5e867700 1 -- 172.21.15.16:0/805712339 <== osd.6 v2:172.21.15.132:6801/727581123 2 ==== osd_op_reply(6 200.00000004 [delete] v93'9 uv8 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x55be6e683200 con 0x55be6db57800 2023-05-20T01:25:35.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.979+0000 7f8b5e867700 1 -- 172.21.15.16:0/805712339 <== osd.4 v2:172.21.15.132:6817/506317223 1 ==== osd_op_reply(4 200.00000002 [delete] v93'2 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x55be6e683200 con 0x55be6db60c00 2023-05-20T01:25:35.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.979+0000 7f8b5e867700 1 -- 172.21.15.16:0/805712339 <== osd.4 v2:172.21.15.132:6817/506317223 2 ==== osd_op_reply(7 200.00000005 [delete] v93'2 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x55be6e683200 con 0x55be6db60c00 2023-05-20T01:25:35.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.979+0000 7f8b53851700 1 -- 172.21.15.16:0/805712339 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:9 12.14 12:2e2fa760:::200.00000001:head [write 0~48 in=48b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55be6e6a8e00 con 0x55be6db57c00 2023-05-20T01:25:35.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.980+0000 7f8b5d865700 1 -- 172.21.15.16:0/805712339 <== osd.5 v2:172.21.15.132:6816/1844689995 3 ==== osd_op_reply(9 200.00000001 [write 0~48] v93'28 uv28 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x55be6e683440 con 0x55be6db57c00 2023-05-20T01:25:35.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.980+0000 7f8b5e867700 1 -- 172.21.15.16:0/805712339 <== osd.4 v2:172.21.15.132:6817/506317223 3 ==== osd_op_reply(8 200.00000006 [delete] v93'2 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x55be6e683200 con 0x55be6db60c00 2023-05-20T01:25:35.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.980+0000 7f8b600ba2c0 4 Successfully wrote new journal pointer and header for rank 4:0 2023-05-20T01:25:35.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.980+0000 7f8b600ba2c0 1 -- 172.21.15.16:0/805712339 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x55be6db61000 msgr2=0x55be6dcb9080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:35.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.980+0000 7f8b600ba2c0 1 --2- 172.21.15.16:0/805712339 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x55be6db61000 0x55be6dcb9080 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:35.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.980+0000 7f8b600ba2c0 1 -- 172.21.15.16:0/805712339 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x55be6db60c00 msgr2=0x55be6dcb8b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:35.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.980+0000 7f8b600ba2c0 1 --2- 172.21.15.16:0/805712339 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x55be6db60c00 0x55be6dcb8b00 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:35.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.980+0000 7f8b600ba2c0 1 -- 172.21.15.16:0/805712339 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x55be6db57c00 msgr2=0x55be6dcb8580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:35.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.980+0000 7f8b600ba2c0 1 --2- 172.21.15.16:0/805712339 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x55be6db57c00 0x55be6dcb8580 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:35.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.980+0000 7f8b600ba2c0 1 -- 172.21.15.16:0/805712339 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x55be6db57800 msgr2=0x55be6dcb8000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:35.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.980+0000 7f8b600ba2c0 1 --2- 172.21.15.16:0/805712339 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x55be6db57800 0x55be6dcb8000 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:35.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.980+0000 7f8b5d865700 1 -- 172.21.15.16:0/805712339 reap_dead start 2023-05-20T01:25:35.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.981+0000 7f8b5d865700 1 -- 172.21.15.16:0/805712339 reap_dead start 2023-05-20T01:25:35.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.981+0000 7f8b600ba2c0 1 -- 172.21.15.16:0/805712339 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be6db57000 msgr2=0x55be6d8bdb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:35.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.981+0000 7f8b600ba2c0 1 --2- 172.21.15.16:0/805712339 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be6db57000 0x55be6d8bdb80 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x55be6dc2cff0 tx=0x55be6e6b3020 comp rx=0 tx=0).stop 2023-05-20T01:25:35.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.981+0000 7f8b600ba2c0 1 -- 172.21.15.16:0/805712339 shutdown_connections 2023-05-20T01:25:35.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.981+0000 7f8b600ba2c0 1 --2- 172.21.15.16:0/805712339 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be6db57000 0x55be6d8bdb80 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:35.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.981+0000 7f8b600ba2c0 1 -- 172.21.15.16:0/805712339 >> 172.21.15.16:0/805712339 conn(0x55be6db56800 msgr2=0x55be6d900000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:35.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.981+0000 7f8b600ba2c0 1 -- 172.21.15.16:0/805712339 shutdown_connections 2023-05-20T01:25:35.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.981+0000 7f8b600ba2c0 1 -- 172.21.15.16:0/805712339 wait complete. 2023-05-20T01:25:35.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.981+0000 7f8b600ba2c0 1 -- 172.21.15.16:0/3443922796 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x55be6db60400 msgr2=0x55be6d8bf700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:36.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.981+0000 7f8b600ba2c0 1 --2- 172.21.15.16:0/3443922796 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x55be6db60400 0x55be6d8bf700 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x55be6db01020 tx=0x55be6e6b30e0 comp rx=0 tx=0).stop 2023-05-20T01:25:36.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.981+0000 7f8b600ba2c0 1 -- 172.21.15.16:0/3443922796 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be6db56400 msgr2=0x55be6d8be680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:36.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.981+0000 7f8b600ba2c0 1 --2- 172.21.15.16:0/3443922796 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be6db56400 0x55be6d8be680 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x55be6e685f20 tx=0x55be6e5bb140 comp rx=0 tx=0).stop 2023-05-20T01:25:36.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.981+0000 7f8b600ba2c0 1 -- 172.21.15.16:0/3443922796 shutdown_connections 2023-05-20T01:25:36.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.981+0000 7f8b600ba2c0 1 --2- 172.21.15.16:0/3443922796 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be6db56400 0x55be6d8be680 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:36.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.981+0000 7f8b600ba2c0 1 --2- 172.21.15.16:0/3443922796 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55be6d822400 0x55be6d8bf180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:36.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.981+0000 7f8b600ba2c0 1 --2- 172.21.15.16:0/3443922796 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x55be6db60400 0x55be6d8bf700 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:36.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.981+0000 7f8b600ba2c0 1 --2- 172.21.15.16:0/3443922796 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55be6db56000 0x55be6d8bec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:36.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.981+0000 7f8b600ba2c0 1 -- 172.21.15.16:0/3443922796 >> 172.21.15.16:0/3443922796 conn(0x55be6d823c00 msgr2=0x55be6d8ff800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:36.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.981+0000 7f8b600ba2c0 1 -- 172.21.15.16:0/3443922796 shutdown_connections 2023-05-20T01:25:36.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.981+0000 7f8b600ba2c0 1 -- 172.21.15.16:0/3443922796 wait complete. 2023-05-20T01:25:36.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.981+0000 7f8b600ba2c0 1 -- 172.21.15.16:0/2527693782 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be6d822800 msgr2=0x55be6d8bd080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:36.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.981+0000 7f8b600ba2c0 1 --2- 172.21.15.16:0/2527693782 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be6d822800 0x55be6d8bd080 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x55be6e6846c0 tx=0x55be6e69c0c0 comp rx=0 tx=0).stop 2023-05-20T01:25:36.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.981+0000 7f8b600ba2c0 1 -- 172.21.15.16:0/2527693782 shutdown_connections 2023-05-20T01:25:36.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.981+0000 7f8b600ba2c0 1 --2- 172.21.15.16:0/2527693782 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be6d822800 0x55be6d8bd080 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:36.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.981+0000 7f8b600ba2c0 1 --2- 172.21.15.16:0/2527693782 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55be6d823000 0x55be6d8bc580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:36.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.981+0000 7f8b600ba2c0 1 --2- 172.21.15.16:0/2527693782 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55be6d822c00 0x55be6d8bcb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:36.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.981+0000 7f8b600ba2c0 1 -- 172.21.15.16:0/2527693782 >> 172.21.15.16:0/2527693782 conn(0x55be6d823400 msgr2=0x55be6d8ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:36.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.982+0000 7f8b600ba2c0 1 -- 172.21.15.16:0/2527693782 shutdown_connections 2023-05-20T01:25:36.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:35.982+0000 7f8b600ba2c0 1 -- 172.21.15.16:0/2527693782 wait complete. 2023-05-20T01:25:36.002 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'journal', 'reset', '--force'] in time 0:00:00.116849, result: writing EResetJournal entry 2023-05-20T01:25:36.003 DEBUG:teuthology.orchestra.run.smithi016:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 init 2023-05-20T01:25:36.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.040+0000 7f4b211857c0 1 -- start start 2023-05-20T01:25:36.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.040+0000 7f4b211857c0 1 Processor -- start 2023-05-20T01:25:36.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.040+0000 7f4b211857c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bb233ed000 0x55bb23486580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:36.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.040+0000 7f4b211857c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55bb233ecc00 0x55bb23486b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:36.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.040+0000 7f4b211857c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb233ec800 0x55bb23487080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:36.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.040+0000 7f4b211857c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb23429cc0 con 0x55bb233ec800 2023-05-20T01:25:36.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.040+0000 7f4b211857c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb23429400 con 0x55bb233ed000 2023-05-20T01:25:36.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.040+0000 7f4b211857c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bb23428a00 con 0x55bb233ecc00 2023-05-20T01:25:36.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.040+0000 7f4b1f934700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb233ec800 0x55bb23487080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:36.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.041+0000 7f4b1f934700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb233ec800 0x55bb23487080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36942/0 (socket says 172.21.15.16:36942) 2023-05-20T01:25:36.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.041+0000 7f4b1f934700 1 -- 172.21.15.16:0/3976968757 learned_addr learned my addr 172.21.15.16:0/3976968757 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:36.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.041+0000 7f4b1f133700 1 --2- 172.21.15.16:0/3976968757 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bb233ed000 0x55bb23486580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:36.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.041+0000 7f4b1e932700 1 --2- 172.21.15.16:0/3976968757 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55bb233ecc00 0x55bb23486b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:36.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.041+0000 7f4b1f934700 1 -- 172.21.15.16:0/3976968757 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55bb233ecc00 msgr2=0x55bb23486b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:36.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.041+0000 7f4b1f934700 1 --2- 172.21.15.16:0/3976968757 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55bb233ecc00 0x55bb23486b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:36.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.041+0000 7f4b1f934700 1 -- 172.21.15.16:0/3976968757 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bb233ed000 msgr2=0x55bb23486580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:36.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.041+0000 7f4b1f934700 1 --2- 172.21.15.16:0/3976968757 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bb233ed000 0x55bb23486580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:36.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.041+0000 7f4b1f934700 1 -- 172.21.15.16:0/3976968757 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bb234104e0 con 0x55bb233ec800 2023-05-20T01:25:36.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.041+0000 7f4b1f934700 1 --2- 172.21.15.16:0/3976968757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb233ec800 0x55bb23487080 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x55bb241b8cc0 tx=0x55bb24262240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7c39f8e3935daf9 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:36.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.041+0000 7f4b1c92e700 1 -- 172.21.15.16:0/3976968757 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55bb23421600 con 0x55bb233ec800 2023-05-20T01:25:36.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.041+0000 7f4b1c92e700 1 -- 172.21.15.16:0/3976968757 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55bb2343e600 con 0x55bb233ec800 2023-05-20T01:25:36.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.041+0000 7f4b1c92e700 1 -- 172.21.15.16:0/3976968757 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55bb234211e0 con 0x55bb233ec800 2023-05-20T01:25:36.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.041+0000 7f4b211857c0 1 -- 172.21.15.16:0/3976968757 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bb23411520 con 0x55bb233ec800 2023-05-20T01:25:36.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.041+0000 7f4b211857c0 1 -- 172.21.15.16:0/3976968757 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55bb23411380 con 0x55bb233ec800 2023-05-20T01:25:36.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.041+0000 7f4b211857c0 4 waiting for MDS map... 2023-05-20T01:25:36.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.041+0000 7f4b1c92e700 1 -- 172.21.15.16:0/3976968757 <== mon.0 v2:172.21.15.16:3300/0 4 ==== osd_map(93..93 src has 1..93) v4 ==== 6958+0+0 (secure 0 0 0) 0x55bb2342ec40 con 0x55bb233ec800 2023-05-20T01:25:36.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.041+0000 7f4b1c92e700 1 -- 172.21.15.16:0/3976968757 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 63) v1 ==== 3126+0+0 (secure 0 0 0) 0x55bb2426b200 con 0x55bb233ec800 2023-05-20T01:25:36.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.042+0000 7f4b211857c0 4 Got MDS map 63 2023-05-20T01:25:36.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.042+0000 7f4b211857c0 4 datascan.main: Using metadata pool output 2023-05-20T01:25:36.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.042+0000 7f4b211857c0 4 datascan.main: connecting to RADOS... 2023-05-20T01:25:36.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.042+0000 7f4b211857c0 1 Processor -- start 2023-05-20T01:25:36.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.042+0000 7f4b211857c0 1 -- start start 2023-05-20T01:25:36.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.043+0000 7f4b211857c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb233edc00 0x55bb23487600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:36.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.043+0000 7f4b211857c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55bb23708000 0x55bb23487b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:36.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.043+0000 7f4b211857c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bb23708400 0x55bb23488100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:36.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.043+0000 7f4b211857c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb23428b40 con 0x55bb233edc00 2023-05-20T01:25:36.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.043+0000 7f4b211857c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb23428c80 con 0x55bb23708400 2023-05-20T01:25:36.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.043+0000 7f4b211857c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bb23429400 con 0x55bb23708000 2023-05-20T01:25:36.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.043+0000 7f4b1e932700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb233edc00 0x55bb23487600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:36.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.043+0000 7f4b1e932700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb233edc00 0x55bb23487600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36944/0 (socket says 172.21.15.16:36944) 2023-05-20T01:25:36.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.043+0000 7f4b1e932700 1 -- 172.21.15.16:0/4230853139 learned_addr learned my addr 172.21.15.16:0/4230853139 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:36.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.043+0000 7f4b1f133700 1 --2- 172.21.15.16:0/4230853139 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55bb23708000 0x55bb23487b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:36.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.043+0000 7f4b1e932700 1 --2- 172.21.15.16:0/4230853139 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bb23708400 0x55bb23488100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:36.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.043+0000 7f4b1e932700 1 -- 172.21.15.16:0/4230853139 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55bb23708000 msgr2=0x55bb23487b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:36.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.043+0000 7f4b1e932700 1 --2- 172.21.15.16:0/4230853139 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55bb23708000 0x55bb23487b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:36.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.043+0000 7f4b1e932700 1 -- 172.21.15.16:0/4230853139 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bb23708400 msgr2=0x55bb23488100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:36.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.043+0000 7f4b1e932700 1 --2- 172.21.15.16:0/4230853139 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bb23708400 0x55bb23488100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:36.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.043+0000 7f4b1e932700 1 -- 172.21.15.16:0/4230853139 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bb23411d40 con 0x55bb233edc00 2023-05-20T01:25:36.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.043+0000 7f4b1e932700 1 --2- 172.21.15.16:0/4230853139 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb233edc00 0x55bb23487600 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x55bb23712120 tx=0x55bb24284c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d35df1982bce97c server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:36.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.043+0000 7f4b1a92a700 1 -- 172.21.15.16:0/4230853139 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55bb23421600 con 0x55bb233edc00 2023-05-20T01:25:36.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.043+0000 7f4b1a92a700 1 -- 172.21.15.16:0/4230853139 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55bb2343fe00 con 0x55bb233edc00 2023-05-20T01:25:36.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.043+0000 7f4b1a92a700 1 -- 172.21.15.16:0/4230853139 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55bb234211e0 con 0x55bb233edc00 2023-05-20T01:25:36.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.044+0000 7f4b211857c0 1 -- 172.21.15.16:0/4230853139 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb233edc00 msgr2=0x55bb23487600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:36.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.044+0000 7f4b211857c0 1 --2- 172.21.15.16:0/4230853139 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb233edc00 0x55bb23487600 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x55bb23712120 tx=0x55bb24284c60 comp rx=0 tx=0).stop 2023-05-20T01:25:36.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.044+0000 7f4b211857c0 1 -- 172.21.15.16:0/4230853139 shutdown_connections 2023-05-20T01:25:36.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.044+0000 7f4b211857c0 1 --2- 172.21.15.16:0/4230853139 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bb23708400 0x55bb23488100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:36.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.044+0000 7f4b211857c0 1 --2- 172.21.15.16:0/4230853139 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb233edc00 0x55bb23487600 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:36.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.044+0000 7f4b211857c0 1 --2- 172.21.15.16:0/4230853139 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55bb23708000 0x55bb23487b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:36.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.044+0000 7f4b211857c0 1 -- 172.21.15.16:0/4230853139 >> 172.21.15.16:0/4230853139 conn(0x55bb233ec400 msgr2=0x55bb234c9800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:36.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.044+0000 7f4b211857c0 1 -- 172.21.15.16:0/4230853139 shutdown_connections 2023-05-20T01:25:36.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.044+0000 7f4b211857c0 1 -- 172.21.15.16:0/4230853139 wait complete. 2023-05-20T01:25:36.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.044+0000 7f4b211857c0 1 Processor -- start 2023-05-20T01:25:36.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.045+0000 7f4b211857c0 1 -- start start 2023-05-20T01:25:36.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.045+0000 7f4b211857c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bb233ec400 0x55bb23489180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:36.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.045+0000 7f4b211857c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55bb23708000 0x55bb23488c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:36.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.045+0000 7f4b211857c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb23708400 0x55bb23488680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:36.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.045+0000 7f4b211857c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb23428000 con 0x55bb23708400 2023-05-20T01:25:36.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.045+0000 7f4b211857c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb23429180 con 0x55bb233ec400 2023-05-20T01:25:36.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.045+0000 7f4b211857c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bb23429040 con 0x55bb23708000 2023-05-20T01:25:36.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.045+0000 7f4b1e932700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb23708400 0x55bb23488680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:36.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.045+0000 7f4b1e932700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb23708400 0x55bb23488680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36954/0 (socket says 172.21.15.16:36954) 2023-05-20T01:25:36.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.045+0000 7f4b1e932700 1 -- 172.21.15.16:0/2596565691 learned_addr learned my addr 172.21.15.16:0/2596565691 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:36.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.045+0000 7f4b1f133700 1 --2- 172.21.15.16:0/2596565691 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55bb23708000 0x55bb23488c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:36.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.045+0000 7f4b1e932700 1 --2- 172.21.15.16:0/2596565691 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bb233ec400 0x55bb23489180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:36.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.045+0000 7f4b1e932700 1 -- 172.21.15.16:0/2596565691 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55bb23708000 msgr2=0x55bb23488c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:36.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.045+0000 7f4b1e932700 1 --2- 172.21.15.16:0/2596565691 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55bb23708000 0x55bb23488c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:36.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.045+0000 7f4b1e932700 1 -- 172.21.15.16:0/2596565691 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bb233ec400 msgr2=0x55bb23489180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:36.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.046+0000 7f4b1e932700 1 --2- 172.21.15.16:0/2596565691 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bb233ec400 0x55bb23489180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:36.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.046+0000 7f4b1e932700 1 -- 172.21.15.16:0/2596565691 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bb242684e0 con 0x55bb23708400 2023-05-20T01:25:36.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.046+0000 7f4b1f133700 1 --2- 172.21.15.16:0/2596565691 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55bb23708000 0x55bb23488c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:25:36.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.046+0000 7f4b1e932700 1 --2- 172.21.15.16:0/2596565691 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb23708400 0x55bb23488680 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x55bb237133e0 tx=0x55bb241853e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:36.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.046+0000 7f4b19127700 1 -- 172.21.15.16:0/2596565691 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55bb23420dc0 con 0x55bb23708400 2023-05-20T01:25:36.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.046+0000 7f4b19127700 1 -- 172.21.15.16:0/2596565691 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55bb2343fc80 con 0x55bb23708400 2023-05-20T01:25:36.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.046+0000 7f4b211857c0 1 -- 172.21.15.16:0/2596565691 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55bb24268340 con 0x55bb23708400 2023-05-20T01:25:36.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.046+0000 7f4b19127700 1 -- 172.21.15.16:0/2596565691 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55bb23421600 con 0x55bb23708400 2023-05-20T01:25:36.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.046+0000 7f4b211857c0 1 -- 172.21.15.16:0/2596565691 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bb24268680 con 0x55bb23708400 2023-05-20T01:25:36.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.046+0000 7f4b211857c0 4 datascan.init: resolving metadata pool 12 2023-05-20T01:25:36.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.047+0000 7f4b19127700 1 -- 172.21.15.16:0/2596565691 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x55bb2344e840 con 0x55bb23708400 2023-05-20T01:25:36.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.047+0000 7f4b19127700 1 --2- 172.21.15.16:0/2596565691 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x55bb23732400 0x55bb23489700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:36.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.047+0000 7f4b19127700 1 -- 172.21.15.16:0/2596565691 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6958+0+0 (secure 0 0 0) 0x55bb2342ea80 con 0x55bb23708400 2023-05-20T01:25:36.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.047+0000 7f4b1f133700 1 --2- 172.21.15.16:0/2596565691 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x55bb23732400 0x55bb23489700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:36.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.047+0000 7f4b211857c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-05-20T01:25:36.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.048+0000 7f4b211857c0 1 --2- 172.21.15.16:0/2596565691 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x55bb23708800 0x55bb23488100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:36.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.048+0000 7f4b211857c0 1 -- 172.21.15.16:0/2596565691 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:1 12.14 12:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x55bb2427c700 con 0x55bb23708800 2023-05-20T01:25:36.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.048+0000 7f4b1f133700 1 --2- 172.21.15.16:0/2596565691 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x55bb23732400 0x55bb23489700 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x55bb23447620 tx=0x55bb24284fc0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:36.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.048+0000 7f4b1e932700 1 --2- 172.21.15.16:0/2596565691 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x55bb23708800 0x55bb23488100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:36.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.048+0000 7f4b1e932700 1 --2- 172.21.15.16:0/2596565691 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x55bb23708800 0x55bb23488100 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:36.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.049+0000 7f4b1e932700 1 -- 172.21.15.16:0/2596565691 <== osd.5 v2:172.21.15.132:6816/1844689995 1 ==== osd_op_reply(1 1.00000000.inode [stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 160+0+0 (crc 0 0 0) 0x55bb23703440 con 0x55bb23708800 2023-05-20T01:25:36.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.049+0000 7f4b211857c0 1 -- 172.21.15.16:0/2596565691 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:2 12.14 12:287034c4:::1.00000000.inode:head [writefull 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55bb2427ca80 con 0x55bb23708800 2023-05-20T01:25:36.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.050+0000 7f4b1e932700 1 -- 172.21.15.16:0/2596565691 <== osd.5 v2:172.21.15.132:6816/1844689995 2 ==== osd_op_reply(2 1.00000000.inode [writefull 0~582] v93'29 uv29 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x55bb23703440 con 0x55bb23708800 2023-05-20T01:25:36.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.050+0000 7f4b211857c0 1 --2- 172.21.15.16:0/2596565691 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x55bb23708c00 0x55bb23487b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:36.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.050+0000 7f4b211857c0 1 -- 172.21.15.16:0/2596565691 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:3 12.1f 12:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x55bb2427ce00 con 0x55bb23708c00 2023-05-20T01:25:36.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.050+0000 7f4b1f934700 1 --2- 172.21.15.16:0/2596565691 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x55bb23708c00 0x55bb23487b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:36.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.051+0000 7f4b1f934700 1 --2- 172.21.15.16:0/2596565691 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x55bb23708c00 0x55bb23487b80 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:36.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.051+0000 7f4b1f934700 1 -- 172.21.15.16:0/2596565691 <== osd.6 v2:172.21.15.132:6801/727581123 1 ==== osd_op_reply(3 100.00000000.inode [stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 162+0+0 (crc 0 0 0) 0x55bb23703b00 con 0x55bb23708c00 2023-05-20T01:25:36.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.051+0000 7f4b211857c0 1 -- 172.21.15.16:0/2596565691 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:4 12.1f 12:fe07bba1:::100.00000000.inode:head [writefull 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55bb2427c700 con 0x55bb23708c00 2023-05-20T01:25:36.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.052+0000 7f4b1f934700 1 -- 172.21.15.16:0/2596565691 <== osd.6 v2:172.21.15.132:6801/727581123 2 ==== osd_op_reply(4 100.00000000.inode [writefull 0~582] v93'10 uv10 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x55bb23703b00 con 0x55bb23708c00 2023-05-20T01:25:36.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.053+0000 7f4b211857c0 1 --2- 172.21.15.16:0/2596565691 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x55bb23709000 0x55bb23487600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:36.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.053+0000 7f4b211857c0 1 -- 172.21.15.16:0/2596565691 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:5 12.13 12:cd5a64a3:::100.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x55bb2427ca80 con 0x55bb23709000 2023-05-20T01:25:36.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.053+0000 7f4b1f133700 1 --2- 172.21.15.16:0/2596565691 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x55bb23709000 0x55bb23487600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:36.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.053+0000 7f4b1f133700 1 --2- 172.21.15.16:0/2596565691 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x55bb23709000 0x55bb23487600 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:36.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.053+0000 7f4b1f133700 1 -- 172.21.15.16:0/2596565691 <== osd.1 v2:172.21.15.16:6808/3860264220 1 ==== osd_op_reply(5 100.00000000 [omap-get-header] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x55bb23842000 con 0x55bb23709000 2023-05-20T01:25:36.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.053+0000 7f4b211857c0 10 datascan.find_or_create_dirfrag: read_version = 0 2023-05-20T01:25:36.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.053+0000 7f4b211857c0 1 -- 172.21.15.16:0/2596565691 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:6 12.13 12:cd5a64a3:::100.00000000:head [create,omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55bb2427d180 con 0x55bb23709000 2023-05-20T01:25:36.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.054+0000 7f4b1f133700 1 -- 172.21.15.16:0/2596565691 <== osd.1 v2:172.21.15.16:6808/3860264220 2 ==== osd_op_reply(6 100.00000000 [create,omap-set-header] v93'7 uv7 ondisk = 0) v8 ==== 198+0+0 (crc 0 0 0) 0x55bb23842000 con 0x55bb23709000 2023-05-20T01:25:36.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.054+0000 7f4b211857c0 10 datascan.find_or_create_dirfrag: Created dirfrag: 0x0x100 2023-05-20T01:25:36.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.054+0000 7f4b211857c0 1 -- 172.21.15.16:0/2596565691 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x55bb23709000 msgr2=0x55bb23487600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:36.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.054+0000 7f4b211857c0 1 --2- 172.21.15.16:0/2596565691 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x55bb23709000 0x55bb23487600 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:36.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.054+0000 7f4b211857c0 1 -- 172.21.15.16:0/2596565691 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x55bb23708800 msgr2=0x55bb23488100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:36.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.054+0000 7f4b211857c0 1 --2- 172.21.15.16:0/2596565691 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x55bb23708800 0x55bb23488100 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:36.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.054+0000 7f4b211857c0 1 -- 172.21.15.16:0/2596565691 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x55bb23708c00 msgr2=0x55bb23487b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:36.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.054+0000 7f4b211857c0 1 --2- 172.21.15.16:0/2596565691 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x55bb23708c00 0x55bb23487b80 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:36.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.055+0000 7f4b1f133700 1 -- 172.21.15.16:0/2596565691 reap_dead start 2023-05-20T01:25:36.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.055+0000 7f4b211857c0 1 -- 172.21.15.16:0/2596565691 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x55bb23732400 msgr2=0x55bb23489700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:36.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.055+0000 7f4b211857c0 1 --2- 172.21.15.16:0/2596565691 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x55bb23732400 0x55bb23489700 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x55bb23447620 tx=0x55bb24284fc0 comp rx=0 tx=0).stop 2023-05-20T01:25:36.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.055+0000 7f4b211857c0 1 -- 172.21.15.16:0/2596565691 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb23708400 msgr2=0x55bb23488680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:36.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.055+0000 7f4b211857c0 1 --2- 172.21.15.16:0/2596565691 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb23708400 0x55bb23488680 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x55bb237133e0 tx=0x55bb241853e0 comp rx=0 tx=0).stop 2023-05-20T01:25:36.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.055+0000 7f4b211857c0 1 -- 172.21.15.16:0/2596565691 shutdown_connections 2023-05-20T01:25:36.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.055+0000 7f4b211857c0 1 --2- 172.21.15.16:0/2596565691 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x55bb23732400 0x55bb23489700 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:36.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.055+0000 7f4b211857c0 1 --2- 172.21.15.16:0/2596565691 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb23708400 0x55bb23488680 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:36.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.055+0000 7f4b211857c0 1 -- 172.21.15.16:0/2596565691 >> 172.21.15.16:0/2596565691 conn(0x55bb233edc00 msgr2=0x55bb234c9800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:36.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.055+0000 7f4b211857c0 1 -- 172.21.15.16:0/2596565691 shutdown_connections 2023-05-20T01:25:36.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.055+0000 7f4b211857c0 1 -- 172.21.15.16:0/2596565691 wait complete. 2023-05-20T01:25:36.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.055+0000 7f4b211857c0 1 -- 172.21.15.16:0/3976968757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb233ec800 msgr2=0x55bb23487080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:36.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.055+0000 7f4b211857c0 1 --2- 172.21.15.16:0/3976968757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb233ec800 0x55bb23487080 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x55bb241b8cc0 tx=0x55bb24262240 comp rx=0 tx=0).stop 2023-05-20T01:25:36.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.055+0000 7f4b211857c0 1 -- 172.21.15.16:0/3976968757 shutdown_connections 2023-05-20T01:25:36.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.055+0000 7f4b211857c0 1 --2- 172.21.15.16:0/3976968757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb233ec800 0x55bb23487080 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:36.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.055+0000 7f4b211857c0 1 --2- 172.21.15.16:0/3976968757 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bb233ed000 0x55bb23486580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:36.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.055+0000 7f4b211857c0 1 --2- 172.21.15.16:0/3976968757 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55bb233ecc00 0x55bb23486b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:36.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.055+0000 7f4b211857c0 1 -- 172.21.15.16:0/3976968757 >> 172.21.15.16:0/3976968757 conn(0x55bb233ed400 msgr2=0x55bb234c9000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:36.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.055+0000 7f4b211857c0 1 -- 172.21.15.16:0/3976968757 shutdown_connections 2023-05-20T01:25:36.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.055+0000 7f4b211857c0 1 -- 172.21.15.16:0/3976968757 wait complete. 2023-05-20T01:25:36.080 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'init'] in time 0:00:00.077286, result: 2023-05-20T01:25:36.080 DEBUG:teuthology.orchestra.run.smithi016:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_extents 2023-05-20T01:25:36.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.150+0000 7f73d1ff67c0 1 -- start start 2023-05-20T01:25:36.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.150+0000 7f73d1ff67c0 1 Processor -- start 2023-05-20T01:25:36.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.150+0000 7f73d1ff67c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f679f83000 0x55f67a01c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:36.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.150+0000 7f73d1ff67c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f679f82c00 0x55f67a01cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:36.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.150+0000 7f73d1ff67c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55f679f82800 0x55f67a01d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:36.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.150+0000 7f73d1ff67c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f679fbfcc0 con 0x55f679f83000 2023-05-20T01:25:36.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.150+0000 7f73d1ff67c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f679fbf400 con 0x55f679f82c00 2023-05-20T01:25:36.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.150+0000 7f73d1ff67c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f679fbfe00 con 0x55f679f82800 2023-05-20T01:25:36.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.151+0000 7f73cffa4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f679f83000 0x55f67a01c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:36.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.151+0000 7f73cffa4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f679f83000 0x55f67a01c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36970/0 (socket says 172.21.15.16:36970) 2023-05-20T01:25:36.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.151+0000 7f73cffa4700 1 -- 172.21.15.16:0/3808048638 learned_addr learned my addr 172.21.15.16:0/3808048638 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:36.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.151+0000 7f73d07a5700 1 --2- 172.21.15.16:0/3808048638 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55f679f82800 0x55f67a01d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:36.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.151+0000 7f73cf7a3700 1 --2- 172.21.15.16:0/3808048638 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f679f82c00 0x55f67a01cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:36.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.151+0000 7f73cffa4700 1 -- 172.21.15.16:0/3808048638 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55f679f82800 msgr2=0x55f67a01d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:36.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.151+0000 7f73cffa4700 1 --2- 172.21.15.16:0/3808048638 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55f679f82800 0x55f67a01d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:36.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.151+0000 7f73cffa4700 1 -- 172.21.15.16:0/3808048638 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f679f82c00 msgr2=0x55f67a01cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:36.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.151+0000 7f73cffa4700 1 --2- 172.21.15.16:0/3808048638 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f679f82c00 0x55f67a01cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:36.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.151+0000 7f73cffa4700 1 -- 172.21.15.16:0/3808048638 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f679fa64e0 con 0x55f679f83000 2023-05-20T01:25:36.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.151+0000 7f73cffa4700 1 --2- 172.21.15.16:0/3808048638 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f679f83000 0x55f67a01c580 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x55f67a0dee40 tx=0x55f67a0ecea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c77b8e65639876f3 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:36.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.151+0000 7f73cd79f700 1 -- 172.21.15.16:0/3808048638 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55f679fb7600 con 0x55f679f83000 2023-05-20T01:25:36.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.151+0000 7f73cd79f700 1 -- 172.21.15.16:0/3808048638 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f679fd4c00 con 0x55f679f83000 2023-05-20T01:25:36.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.151+0000 7f73cd79f700 1 -- 172.21.15.16:0/3808048638 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55f679fb71e0 con 0x55f679f83000 2023-05-20T01:25:36.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.151+0000 7f73d1ff67c0 1 -- 172.21.15.16:0/3808048638 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f679fa7520 con 0x55f679f83000 2023-05-20T01:25:36.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.151+0000 7f73d1ff67c0 1 -- 172.21.15.16:0/3808048638 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55f679fa7380 con 0x55f679f83000 2023-05-20T01:25:36.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.151+0000 7f73d1ff67c0 4 waiting for MDS map... 2023-05-20T01:25:36.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.151+0000 7f73cd79f700 1 -- 172.21.15.16:0/3808048638 <== mon.0 v2:172.21.15.16:3300/0 4 ==== osd_map(93..93 src has 1..93) v4 ==== 6958+0+0 (secure 0 0 0) 0x55f679fc4c40 con 0x55f679f83000 2023-05-20T01:25:36.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.152+0000 7f73cd79f700 1 -- 172.21.15.16:0/3808048638 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 63) v1 ==== 3126+0+0 (secure 0 0 0) 0x55f67adef200 con 0x55f679f83000 2023-05-20T01:25:36.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.152+0000 7f73d1ff67c0 4 Got MDS map 63 2023-05-20T01:25:36.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.152+0000 7f73d1ff67c0 4 datascan.main: Using metadata pool output 2023-05-20T01:25:36.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.152+0000 7f73d1ff67c0 4 datascan.main: connecting to RADOS... 2023-05-20T01:25:36.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.152+0000 7f73d1ff67c0 1 Processor -- start 2023-05-20T01:25:36.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.152+0000 7f73d1ff67c0 1 -- start start 2023-05-20T01:25:36.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.153+0000 7f73d1ff67c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55f679f83c00 0x55f67a01d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:36.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.153+0000 7f73d1ff67c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f67a2a0000 0x55f67a01db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:36.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.153+0000 7f73d1ff67c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f67a2a0400 0x55f67a01e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:36.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.153+0000 7f73d1ff67c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f679fbea00 con 0x55f67a2a0400 2023-05-20T01:25:36.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.153+0000 7f73d1ff67c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f679fbeb40 con 0x55f67a2a0000 2023-05-20T01:25:36.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.153+0000 7f73d1ff67c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f679fbf400 con 0x55f679f83c00 2023-05-20T01:25:36.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.153+0000 7f73cf7a3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f67a2a0400 0x55f67a01e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:36.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.153+0000 7f73cf7a3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f67a2a0400 0x55f67a01e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36980/0 (socket says 172.21.15.16:36980) 2023-05-20T01:25:36.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.153+0000 7f73cf7a3700 1 -- 172.21.15.16:0/2063006578 learned_addr learned my addr 172.21.15.16:0/2063006578 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:36.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.153+0000 7f73d07a5700 1 --2- 172.21.15.16:0/2063006578 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55f679f83c00 0x55f67a01d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:36.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.153+0000 7f73cffa4700 1 --2- 172.21.15.16:0/2063006578 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f67a2a0000 0x55f67a01db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:36.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.153+0000 7f73cf7a3700 1 -- 172.21.15.16:0/2063006578 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55f679f83c00 msgr2=0x55f67a01d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:36.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.153+0000 7f73cf7a3700 1 --2- 172.21.15.16:0/2063006578 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55f679f83c00 0x55f67a01d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:36.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.153+0000 7f73cf7a3700 1 -- 172.21.15.16:0/2063006578 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f67a2a0000 msgr2=0x55f67a01db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:36.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.153+0000 7f73cf7a3700 1 --2- 172.21.15.16:0/2063006578 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f67a2a0000 0x55f67a01db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:36.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.153+0000 7f73cf7a3700 1 -- 172.21.15.16:0/2063006578 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f679fa7d40 con 0x55f67a2a0400 2023-05-20T01:25:36.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.153+0000 7f73d07a5700 1 --2- 172.21.15.16:0/2063006578 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55f679f83c00 0x55f67a01d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:25:36.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.153+0000 7f73cffa4700 1 --2- 172.21.15.16:0/2063006578 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f67a2a0000 0x55f67a01db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:25:36.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.153+0000 7f73cf7a3700 1 --2- 172.21.15.16:0/2063006578 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f67a2a0400 0x55f67a01e100 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x55f67a2a60c0 tx=0x55f67ae12c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=546af34e216c9bbc server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:36.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.153+0000 7f73cb79b700 1 -- 172.21.15.16:0/2063006578 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55f679fb7600 con 0x55f67a2a0400 2023-05-20T01:25:36.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.154+0000 7f73cb79b700 1 -- 172.21.15.16:0/2063006578 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f679fd5e00 con 0x55f67a2a0400 2023-05-20T01:25:36.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.154+0000 7f73cb79b700 1 -- 172.21.15.16:0/2063006578 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55f679fb71e0 con 0x55f67a2a0400 2023-05-20T01:25:36.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.154+0000 7f73d1ff67c0 1 -- 172.21.15.16:0/2063006578 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f67a2a0400 msgr2=0x55f67a01e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:36.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.154+0000 7f73d1ff67c0 1 --2- 172.21.15.16:0/2063006578 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f67a2a0400 0x55f67a01e100 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x55f67a2a60c0 tx=0x55f67ae12c00 comp rx=0 tx=0).stop 2023-05-20T01:25:36.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.154+0000 7f73d1ff67c0 1 -- 172.21.15.16:0/2063006578 shutdown_connections 2023-05-20T01:25:36.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.154+0000 7f73d1ff67c0 1 --2- 172.21.15.16:0/2063006578 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f67a2a0400 0x55f67a01e100 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:36.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.154+0000 7f73d1ff67c0 1 --2- 172.21.15.16:0/2063006578 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55f679f83c00 0x55f67a01d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:36.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.154+0000 7f73d1ff67c0 1 --2- 172.21.15.16:0/2063006578 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f67a2a0000 0x55f67a01db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:36.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.154+0000 7f73d1ff67c0 1 -- 172.21.15.16:0/2063006578 >> 172.21.15.16:0/2063006578 conn(0x55f679f82400 msgr2=0x55f67a05f800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:36.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.154+0000 7f73d1ff67c0 1 -- 172.21.15.16:0/2063006578 shutdown_connections 2023-05-20T01:25:36.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.154+0000 7f73d1ff67c0 1 -- 172.21.15.16:0/2063006578 wait complete. 2023-05-20T01:25:36.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.154+0000 7f73d1ff67c0 1 Processor -- start 2023-05-20T01:25:36.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.155+0000 7f73d1ff67c0 1 -- start start 2023-05-20T01:25:36.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.155+0000 7f73d1ff67c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55f679f82400 0x55f67a01f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:36.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.155+0000 7f73d1ff67c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f67a2bc000 0x55f67a01ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:36.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.155+0000 7f73d1ff67c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f67a2bc400 0x55f67a01e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:36.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.155+0000 7f73d1ff67c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f679fbec80 con 0x55f67a2bc400 2023-05-20T01:25:36.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.155+0000 7f73d1ff67c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f679fbe000 con 0x55f67a2bc000 2023-05-20T01:25:36.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.155+0000 7f73d1ff67c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f679fbf180 con 0x55f679f82400 2023-05-20T01:25:36.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.155+0000 7f73cf7a3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f67a2bc400 0x55f67a01e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:36.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.155+0000 7f73cf7a3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f67a2bc400 0x55f67a01e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36986/0 (socket says 172.21.15.16:36986) 2023-05-20T01:25:36.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.155+0000 7f73cffa4700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f67a2bc000 0x55f67a01ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:36.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.155+0000 7f73cf7a3700 1 -- 172.21.15.16:0/3803033970 learned_addr learned my addr 172.21.15.16:0/3803033970 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:36.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.155+0000 7f73d07a5700 1 --2- 172.21.15.16:0/3803033970 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55f679f82400 0x55f67a01f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:36.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.156+0000 7f73cf7a3700 1 -- 172.21.15.16:0/3803033970 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55f679f82400 msgr2=0x55f67a01f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:36.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.156+0000 7f73cf7a3700 1 --2- 172.21.15.16:0/3803033970 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55f679f82400 0x55f67a01f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:36.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.156+0000 7f73cf7a3700 1 -- 172.21.15.16:0/3803033970 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f67a2bc000 msgr2=0x55f67a01ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:36.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.156+0000 7f73cf7a3700 1 --2- 172.21.15.16:0/3803033970 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f67a2bc000 0x55f67a01ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:36.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.156+0000 7f73cf7a3700 1 -- 172.21.15.16:0/3803033970 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f67a2ae000 con 0x55f67a2bc400 2023-05-20T01:25:36.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.156+0000 7f73cf7a3700 1 --2- 172.21.15.16:0/3803033970 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f67a2bc400 0x55f67a01e680 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x55f67a2a7380 tx=0x55f67ad1b3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:36.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.156+0000 7f73c9f98700 1 -- 172.21.15.16:0/3803033970 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55f679fb6dc0 con 0x55f67a2bc400 2023-05-20T01:25:36.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.156+0000 7f73c9f98700 1 -- 172.21.15.16:0/3803033970 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f67ae0ca80 con 0x55f67a2bc400 2023-05-20T01:25:36.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.156+0000 7f73d1ff67c0 1 -- 172.21.15.16:0/3803033970 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55f679fa6d00 con 0x55f67a2bc400 2023-05-20T01:25:36.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.156+0000 7f73c9f98700 1 -- 172.21.15.16:0/3803033970 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55f679fb7600 con 0x55f67a2bc400 2023-05-20T01:25:36.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.156+0000 7f73d1ff67c0 1 -- 172.21.15.16:0/3803033970 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f67a2ae1a0 con 0x55f67a2bc400 2023-05-20T01:25:36.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.156+0000 7f73d1ff67c0 4 datascan.init: resolving metadata pool 12 2023-05-20T01:25:36.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.157+0000 7f73c9f98700 1 -- 172.21.15.16:0/3803033970 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x55f679fe4840 con 0x55f67a2bc400 2023-05-20T01:25:36.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.157+0000 7f73c9f98700 1 --2- 172.21.15.16:0/3803033970 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x55f67a2c6400 0x55f67a01f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:36.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.157+0000 7f73c9f98700 1 -- 172.21.15.16:0/3803033970 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6958+0+0 (secure 0 0 0) 0x55f679fc4a80 con 0x55f67a2bc400 2023-05-20T01:25:36.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.158+0000 7f73d07a5700 1 --2- 172.21.15.16:0/3803033970 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x55f67a2c6400 0x55f67a01f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:36.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.158+0000 7f73d1ff67c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-05-20T01:25:36.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.158+0000 7f73d1ff67c0 4 datascan.main: data pool 'cephfs_data' has ID 13 2023-05-20T01:25:36.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.158+0000 7f73d1ff67c0 4 datascan.main: opening data pool 'cephfs_data' 2023-05-20T01:25:36.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.158+0000 7f73d1ff67c0 4 datascan.main: data pool 'cephfs_data_extra' has ID 14 2023-05-20T01:25:36.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.158+0000 7f73d1ff67c0 4 datascan.main: opening data pool 'cephfs_data_extra' 2023-05-20T01:25:36.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.158+0000 7f73d1ff67c0 1 --2- 172.21.15.16:0/3803033970 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x55f67a2bc800 0x55f67a01e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:36.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.158+0000 7f73d1ff67c0 1 -- 172.21.15.16:0/3803033970 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:1 13.0 13:00000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55f67ae08700 con 0x55f67a2bc800 2023-05-20T01:25:36.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.158+0000 7f73cffa4700 1 --2- 172.21.15.16:0/3803033970 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x55f67a2bc800 0x55f67a01e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:36.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.158+0000 7f73d07a5700 1 --2- 172.21.15.16:0/3803033970 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x55f67a2c6400 0x55f67a01f700 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x55f679fdd590 tx=0x55f67ad1b2c0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:36.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.158+0000 7f73cffa4700 1 --2- 172.21.15.16:0/3803033970 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x55f67a2bc800 0x55f67a01e100 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:36.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.159+0000 7f73cffa4700 1 -- 172.21.15.16:0/3803033970 <== osd.7 v2:172.21.15.132:6800/466001149 1 ==== osd_op_reply(1 [pgnls start_epoch 93 out=49b] v91'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f67a291440 con 0x55f67a2bc800 2023-05-20T01:25:36.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.159+0000 7f73caf9a700 1 --2- 172.21.15.16:0/3803033970 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x55f67a2c6800 0x55f67a3d0000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:36.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.159+0000 7f73caf9a700 1 -- 172.21.15.16:0/3803033970 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:2 13.20 13:04000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55f67ae09180 con 0x55f67a2c6800 2023-05-20T01:25:36.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.159+0000 7f73d07a5700 1 --2- 172.21.15.16:0/3803033970 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x55f67a2c6800 0x55f67a3d0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:36.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.159+0000 7f73d07a5700 1 --2- 172.21.15.16:0/3803033970 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x55f67a2c6800 0x55f67a3d0000 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:36.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.160+0000 7f73d07a5700 1 -- 172.21.15.16:0/3803033970 <== osd.6 v2:172.21.15.132:6801/727581123 1 ==== osd_op_reply(2 [pgnls start_epoch 93 out=49b] v91'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f67a291680 con 0x55f67a2c6800 2023-05-20T01:25:36.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.160+0000 7f73cb79b700 1 --2- 172.21.15.16:0/3803033970 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x55f67a2c6c00 0x55f67a3d0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:36.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.160+0000 7f73cb79b700 1 -- 172.21.15.16:0/3803033970 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:3 13.10 13:08000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55f67ae09500 con 0x55f67a2c6c00 2023-05-20T01:25:36.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.160+0000 7f73cffa4700 1 --2- 172.21.15.16:0/3803033970 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x55f67a2c6c00 0x55f67a3d0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:36.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.160+0000 7f73cffa4700 1 --2- 172.21.15.16:0/3803033970 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x55f67a2c6c00 0x55f67a3d0580 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:36.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.160+0000 7f73cffa4700 1 -- 172.21.15.16:0/3803033970 <== osd.1 v2:172.21.15.16:6808/3860264220 1 ==== osd_op_reply(3 [pgnls start_epoch 93 out=49b] v91'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f67a291440 con 0x55f67a2c6c00 2023-05-20T01:25:36.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.160+0000 7f73caf9a700 1 --2- 172.21.15.16:0/3803033970 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x55f67a2c7000 0x55f67a3d0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:36.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.160+0000 7f73caf9a700 1 -- 172.21.15.16:0/3803033970 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:4 13.30 13:0c000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55f67ae09880 con 0x55f67a2c7000 2023-05-20T01:25:36.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.161+0000 7f73cf7a3700 1 --2- 172.21.15.16:0/3803033970 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x55f67a2c7000 0x55f67a3d0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:36.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.161+0000 7f73cf7a3700 1 --2- 172.21.15.16:0/3803033970 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x55f67a2c7000 0x55f67a3d0b00 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:36.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.161+0000 7f73cf7a3700 1 -- 172.21.15.16:0/3803033970 <== osd.4 v2:172.21.15.132:6817/506317223 1 ==== osd_op_reply(4 [pgnls start_epoch 93 out=49b] v91'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f67a2918c0 con 0x55f67a2c7000 2023-05-20T01:25:36.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.161+0000 7f73cb79b700 1 --2- 172.21.15.16:0/3803033970 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x55f67a2c7800 0x55f67a3d1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:36.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.161+0000 7f73cb79b700 1 -- 172.21.15.16:0/3803033970 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:5 13.8 13:10000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55f67a3e6000 con 0x55f67a2c7800 2023-05-20T01:25:36.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.161+0000 7f73d07a5700 1 --2- 172.21.15.16:0/3803033970 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x55f67a2c7800 0x55f67a3d1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:36.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.162+0000 7f73d07a5700 1 --2- 172.21.15.16:0/3803033970 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x55f67a2c7800 0x55f67a3d1600 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:36.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.162+0000 7f73d07a5700 1 -- 172.21.15.16:0/3803033970 <== osd.2 v2:172.21.15.16:6816/293187030 1 ==== osd_op_reply(5 [pgnls start_epoch 93 out=49b] v91'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f67a291680 con 0x55f67a2c7800 2023-05-20T01:25:36.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.162+0000 7f73caf9a700 1 -- 172.21.15.16:0/3803033970 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:6 13.28 13:14000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55f67ae09c00 con 0x55f67a2c7800 2023-05-20T01:25:36.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.162+0000 7f73d07a5700 1 -- 172.21.15.16:0/3803033970 <== osd.2 v2:172.21.15.16:6816/293187030 2 ==== osd_op_reply(6 [pgnls start_epoch 93 out=49b] v91'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f67a291680 con 0x55f67a2c7800 2023-05-20T01:25:36.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.162+0000 7f73cb79b700 1 -- 172.21.15.16:0/3803033970 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:7 13.18 13:18000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55f67a3e6380 con 0x55f67a2bc800 2023-05-20T01:25:36.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.162+0000 7f73cffa4700 1 -- 172.21.15.16:0/3803033970 <== osd.7 v2:172.21.15.132:6800/466001149 2 ==== osd_op_reply(7 [pgnls start_epoch 93 out=49b] v91'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f67a291440 con 0x55f67a2bc800 2023-05-20T01:25:36.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.162+0000 7f73caf9a700 1 --2- 172.21.15.16:0/3803033970 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x55f67a2c7400 0x55f67a3d1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:36.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.162+0000 7f73caf9a700 1 -- 172.21.15.16:0/3803033970 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:8 13.38 13:1c000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55f67ae08700 con 0x55f67a2c7400 2023-05-20T01:25:36.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.163+0000 7f73cffa4700 1 --2- 172.21.15.16:0/3803033970 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x55f67a2c7400 0x55f67a3d1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:36.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.163+0000 7f73cffa4700 1 --2- 172.21.15.16:0/3803033970 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x55f67a2c7400 0x55f67a3d1080 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:36.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.163+0000 7f73cffa4700 1 -- 172.21.15.16:0/3803033970 <== osd.5 v2:172.21.15.132:6816/1844689995 1 ==== osd_op_reply(8 [pgnls start_epoch 93 out=49b] v91'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f67a291440 con 0x55f67a2c7400 2023-05-20T01:25:36.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.163+0000 7f73cb79b700 1 -- 172.21.15.16:0/3803033970 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:9 13.4 13:20000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55f67a3ea000 con 0x55f67a2c6800 2023-05-20T01:25:36.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.164+0000 7f73d07a5700 1 -- 172.21.15.16:0/3803033970 <== osd.6 v2:172.21.15.132:6801/727581123 2 ==== osd_op_reply(9 [pgnls start_epoch 93 out=49b] v91'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f67a291680 con 0x55f67a2c6800 2023-05-20T01:25:36.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.164+0000 7f73caf9a700 1 -- 172.21.15.16:0/3803033970 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:10 13.24 13:24000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55f67ae09500 con 0x55f67a2c6800 2023-05-20T01:25:36.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.164+0000 7f73d07a5700 1 -- 172.21.15.16:0/3803033970 <== osd.6 v2:172.21.15.132:6801/727581123 3 ==== osd_op_reply(10 [pgnls start_epoch 93 out=49b] v91'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f67a291680 con 0x55f67a2c6800 2023-05-20T01:25:36.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.164+0000 7f73cb79b700 1 --2- 172.21.15.16:0/3803033970 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x55f67a2c7c00 0x55f67a3d1b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:36.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.164+0000 7f73cb79b700 1 -- 172.21.15.16:0/3803033970 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:11 13.14 13:28000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55f67a3ea380 con 0x55f67a2c7c00 2023-05-20T01:25:36.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.164+0000 7f73cf7a3700 1 --2- 172.21.15.16:0/3803033970 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x55f67a2c7c00 0x55f67a3d1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:36.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.165+0000 7f73cf7a3700 1 --2- 172.21.15.16:0/3803033970 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x55f67a2c7c00 0x55f67a3d1b80 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:36.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.165+0000 7f73cf7a3700 1 -- 172.21.15.16:0/3803033970 <== osd.0 v2:172.21.15.16:6824/3923634179 1 ==== osd_op_reply(11 [pgnls start_epoch 93 out=49b] v91'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f67a2918c0 con 0x55f67a2c7c00 2023-05-20T01:25:36.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.165+0000 7f73caf9a700 1 -- 172.21.15.16:0/3803033970 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:12 13.34 13:2c000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55f67ae09180 con 0x55f67a2c6800 2023-05-20T01:25:36.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.165+0000 7f73d07a5700 1 -- 172.21.15.16:0/3803033970 <== osd.6 v2:172.21.15.132:6801/727581123 4 ==== osd_op_reply(12 [pgnls start_epoch 93 out=49b] v91'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f67a291680 con 0x55f67a2c6800 2023-05-20T01:25:36.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.165+0000 7f73cb79b700 1 -- 172.21.15.16:0/3803033970 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:13 13.c 13:30000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55f67a3ea700 con 0x55f67a2c6c00 2023-05-20T01:25:36.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.165+0000 7f73cffa4700 1 -- 172.21.15.16:0/3803033970 <== osd.1 v2:172.21.15.16:6808/3860264220 2 ==== osd_op_reply(13 [pgnls start_epoch 93 out=49b] v91'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f67a291440 con 0x55f67a2c6c00 2023-05-20T01:25:36.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.165+0000 7f73caf9a700 1 -- 172.21.15.16:0/3803033970 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:14 13.2c 13:34000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55f67a3ea380 con 0x55f67a2c7800 2023-05-20T01:25:36.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.166+0000 7f73d07a5700 1 -- 172.21.15.16:0/3803033970 <== osd.2 v2:172.21.15.16:6816/293187030 3 ==== osd_op_reply(14 [pgnls start_epoch 93 out=49b] v91'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f67a291680 con 0x55f67a2c7800 2023-05-20T01:25:36.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.166+0000 7f73cb79b700 1 -- 172.21.15.16:0/3803033970 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:15 13.1c 13:38000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55f67a3ea000 con 0x55f67a2c6800 2023-05-20T01:25:36.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.166+0000 7f73d07a5700 1 -- 172.21.15.16:0/3803033970 <== osd.6 v2:172.21.15.132:6801/727581123 5 ==== osd_op_reply(15 [pgnls start_epoch 93 out=49b] v91'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f67a291680 con 0x55f67a2c6800 2023-05-20T01:25:36.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.166+0000 7f73caf9a700 1 -- 172.21.15.16:0/3803033970 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:16 13.3c 13:3c000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55f67ae09880 con 0x55f67a2c7800 2023-05-20T01:25:36.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.166+0000 7f73d07a5700 1 -- 172.21.15.16:0/3803033970 <== osd.2 v2:172.21.15.16:6816/293187030 4 ==== osd_op_reply(16 [pgnls start_epoch 93 out=49b] v91'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f67a291680 con 0x55f67a2c7800 2023-05-20T01:25:36.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.166+0000 7f73cb79b700 1 -- 172.21.15.16:0/3803033970 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:17 13.2 13:40000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55f67a3eaa80 con 0x55f67a2c7400 2023-05-20T01:25:36.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.167+0000 7f73cffa4700 1 -- 172.21.15.16:0/3803033970 <== osd.5 v2:172.21.15.132:6816/1844689995 2 ==== osd_op_reply(17 [pgnls start_epoch 93 out=49b] v91'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f67a291440 con 0x55f67a2c7400 2023-05-20T01:25:36.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.167+0000 7f73caf9a700 1 -- 172.21.15.16:0/3803033970 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:18 13.22 13:44000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55f67ae09c00 con 0x55f67a2c6c00 2023-05-20T01:25:36.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.167+0000 7f73cffa4700 1 -- 172.21.15.16:0/3803033970 <== osd.1 v2:172.21.15.16:6808/3860264220 3 ==== osd_op_reply(18 [pgnls start_epoch 93 out=49b] v91'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f67a291440 con 0x55f67a2c6c00 2023-05-20T01:25:36.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.167+0000 7f73cb79b700 1 -- 172.21.15.16:0/3803033970 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:19 13.12 13:48000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55f67a3eae00 con 0x55f67a2c7c00 2023-05-20T01:25:36.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.168+0000 7f73cf7a3700 1 -- 172.21.15.16:0/3803033970 <== osd.0 v2:172.21.15.16:6824/3923634179 2 ==== osd_op_reply(19 [pgnls start_epoch 93 out=49b] v91'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f67a2918c0 con 0x55f67a2c7c00 2023-05-20T01:25:36.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.168+0000 7f73caf9a700 1 -- 172.21.15.16:0/3803033970 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:20 13.32 13:4c000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55f67ae09500 con 0x55f67a2c7800 2023-05-20T01:25:36.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.168+0000 7f73d07a5700 1 -- 172.21.15.16:0/3803033970 <== osd.2 v2:172.21.15.16:6816/293187030 5 ==== osd_op_reply(20 [pgnls start_epoch 93 out=49b] v91'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f67a291680 con 0x55f67a2c7800 2023-05-20T01:25:36.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.168+0000 7f73cb79b700 1 -- 172.21.15.16:0/3803033970 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:21 13.a 13:50000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55f67a3eb180 con 0x55f67a2c7800 2023-05-20T01:25:36.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.168+0000 7f73d07a5700 1 -- 172.21.15.16:0/3803033970 <== osd.2 v2:172.21.15.16:6816/293187030 6 ==== osd_op_reply(21 [pgnls start_epoch 93 out=49b] v91'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f67a291680 con 0x55f67a2c7800 2023-05-20T01:25:36.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.168+0000 7f73caf9a700 1 -- 172.21.15.16:0/3803033970 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:22 13.2a 13:54000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55f67a3ea700 con 0x55f67a2c6c00 2023-05-20T01:25:36.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.168+0000 7f73cffa4700 1 -- 172.21.15.16:0/3803033970 <== osd.1 v2:172.21.15.16:6808/3860264220 4 ==== osd_op_reply(22 [pgnls start_epoch 93 out=49b] v91'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f67a291440 con 0x55f67a2c6c00 2023-05-20T01:25:36.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.168+0000 7f73cb79b700 1 -- 172.21.15.16:0/3803033970 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:23 13.1a 13:58000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55f67a3eaa80 con 0x55f67a2c6c00 2023-05-20T01:25:36.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.169+0000 7f73cffa4700 1 -- 172.21.15.16:0/3803033970 <== osd.1 v2:172.21.15.16:6808/3860264220 5 ==== osd_op_reply(23 [pgnls start_epoch 93 out=49b] v91'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f67a291440 con 0x55f67a2c6c00 2023-05-20T01:25:36.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.169+0000 7f73caf9a700 1 -- 172.21.15.16:0/3803033970 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:24 13.3a 13:5c000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55f67ae08700 con 0x55f67a2c7400 2023-05-20T01:25:36.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.169+0000 7f73cffa4700 1 -- 172.21.15.16:0/3803033970 <== osd.5 v2:172.21.15.132:6816/1844689995 3 ==== osd_op_reply(24 [pgnls start_epoch 93 out=49b] v91'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f67a291440 con 0x55f67a2c7400 2023-05-20T01:25:36.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.169+0000 7f73cb79b700 1 -- 172.21.15.16:0/3803033970 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:25 13.6 13:60000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55f67a3ea380 con 0x55f67a2c7000 2023-05-20T01:25:36.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.169+0000 7f73cf7a3700 1 -- 172.21.15.16:0/3803033970 <== osd.4 v2:172.21.15.132:6817/506317223 2 ==== osd_op_reply(25 [pgnls start_epoch 93 out=49b] v91'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f67a2918c0 con 0x55f67a2c7000 2023-05-20T01:25:36.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.169+0000 7f73caf9a700 1 -- 172.21.15.16:0/3803033970 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:26 13.26 13:64000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55f67ae09c00 con 0x55f67a2bc800 2023-05-20T01:25:36.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.170+0000 7f73cffa4700 1 -- 172.21.15.16:0/3803033970 <== osd.7 v2:172.21.15.132:6800/466001149 3 ==== osd_op_reply(26 [pgnls start_epoch 93 out=49b] v91'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f67a291440 con 0x55f67a2bc800 2023-05-20T01:25:36.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.170+0000 7f73cb79b700 1 --2- 172.21.15.16:0/3803033970 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x55f67a3f4000 0x55f67a3d2100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:36.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.170+0000 7f73cb79b700 1 -- 172.21.15.16:0/3803033970 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:27 13.16 13:68000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55f67a3ea000 con 0x55f67a3f4000 2023-05-20T01:25:36.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.170+0000 7f73d07a5700 1 --2- 172.21.15.16:0/3803033970 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x55f67a3f4000 0x55f67a3d2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:36.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.170+0000 7f73d07a5700 1 --2- 172.21.15.16:0/3803033970 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x55f67a3f4000 0x55f67a3d2100 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:36.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.170+0000 7f73d07a5700 1 -- 172.21.15.16:0/3803033970 <== osd.3 v2:172.21.15.16:6800/3669781890 1 ==== osd_op_reply(27 [pgnls start_epoch 93 out=49b] v91'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f67a291680 con 0x55f67a3f4000 2023-05-20T01:25:36.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.170+0000 7f73caf9a700 1 -- 172.21.15.16:0/3803033970 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:28 13.36 13:6c000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55f67ae09180 con 0x55f67a2c6800 2023-05-20T01:25:36.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.171+0000 7f73d07a5700 1 -- 172.21.15.16:0/3803033970 <== osd.6 v2:172.21.15.132:6801/727581123 6 ==== osd_op_reply(28 [pgnls start_epoch 93 out=49b] v91'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f67a291680 con 0x55f67a2c6800 2023-05-20T01:25:36.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.171+0000 7f73cb79b700 1 -- 172.21.15.16:0/3803033970 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:29 13.e 13:70000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55f67a3eb500 con 0x55f67a2bc800 2023-05-20T01:25:36.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.171+0000 7f73cffa4700 1 -- 172.21.15.16:0/3803033970 <== osd.7 v2:172.21.15.132:6800/466001149 4 ==== osd_op_reply(29 [pgnls start_epoch 93 out=49b] v91'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f67a291440 con 0x55f67a2bc800 2023-05-20T01:25:36.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.171+0000 7f73caf9a700 1 -- 172.21.15.16:0/3803033970 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:30 13.2e 13:74000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55f67ae09880 con 0x55f67a2c7800 2023-05-20T01:25:36.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.171+0000 7f73d07a5700 1 -- 172.21.15.16:0/3803033970 <== osd.2 v2:172.21.15.16:6816/293187030 7 ==== osd_op_reply(30 [pgnls start_epoch 93 out=49b] v91'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f67a291680 con 0x55f67a2c7800 2023-05-20T01:25:36.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.171+0000 7f73cb79b700 1 -- 172.21.15.16:0/3803033970 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:31 13.1e 13:78000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55f67a3eb880 con 0x55f67a2c7000 2023-05-20T01:25:36.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.172+0000 7f73cf7a3700 1 -- 172.21.15.16:0/3803033970 <== osd.4 v2:172.21.15.132:6817/506317223 3 ==== osd_op_reply(31 [pgnls start_epoch 93 out=49b] v91'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f67a2918c0 con 0x55f67a2c7000 2023-05-20T01:25:36.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.172+0000 7f73caf9a700 1 -- 172.21.15.16:0/3803033970 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:32 13.3e 13:7c000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55f67ae09500 con 0x55f67a2c6c00 2023-05-20T01:25:36.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.172+0000 7f73cffa4700 1 -- 172.21.15.16:0/3803033970 <== osd.1 v2:172.21.15.16:6808/3860264220 6 ==== osd_op_reply(32 [pgnls start_epoch 93 out=49b] v91'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f67a291440 con 0x55f67a2c6c00 2023-05-20T01:25:36.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.172+0000 7f73cb79b700 1 -- 172.21.15.16:0/3803033970 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:33 13.1 13:80000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55f67a3ea000 con 0x55f67a2c6800 2023-05-20T01:25:36.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.172+0000 7f73d07a5700 1 -- 172.21.15.16:0/3803033970 <== osd.6 v2:172.21.15.132:6801/727581123 7 ==== osd_op_reply(33 [pgnls start_epoch 93 out=49b] v91'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f67a291680 con 0x55f67a2c6800 2023-05-20T01:25:36.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.172+0000 7f73caf9a700 1 -- 172.21.15.16:0/3803033970 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:34 13.21 13:84000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55f67ae09180 con 0x55f67a2c7000 2023-05-20T01:25:36.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.173+0000 7f73cf7a3700 1 -- 172.21.15.16:0/3803033970 <== osd.4 v2:172.21.15.132:6817/506317223 4 ==== osd_op_reply(34 [pgnls start_epoch 93 out=49b] v91'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f67a2918c0 con 0x55f67a2c7000 2023-05-20T01:25:36.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.173+0000 7f73cb79b700 1 -- 172.21.15.16:0/3803033970 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:35 13.11 13:88000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55f67a3ea380 con 0x55f67a2c6c00 2023-05-20T01:25:36.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.173+0000 7f73cffa4700 1 -- 172.21.15.16:0/3803033970 <== osd.1 v2:172.21.15.16:6808/3860264220 7 ==== osd_op_reply(35 [pgnls start_epoch 93 out=49b] v91'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f67a291440 con 0x55f67a2c6c00 2023-05-20T01:25:36.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.173+0000 7f73caf9a700 1 -- 172.21.15.16:0/3803033970 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:36 13.31 13:8c000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55f67ae09880 con 0x55f67a3f4000 2023-05-20T01:25:36.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.173+0000 7f73d07a5700 1 -- 172.21.15.16:0/3803033970 <== osd.3 v2:172.21.15.16:6800/3669781890 2 ==== osd_op_reply(36 [pgnls start_epoch 93 out=49b] v91'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f67a291680 con 0x55f67a3f4000 2023-05-20T01:25:36.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.173+0000 7f73cb79b700 1 -- 172.21.15.16:0/3803033970 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:37 13.9 13:90000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55f67a3ea700 con 0x55f67a2c7400 2023-05-20T01:25:36.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.174+0000 7f73cffa4700 1 -- 172.21.15.16:0/3803033970 <== osd.5 v2:172.21.15.132:6816/1844689995 4 ==== osd_op_reply(37 [pgnls start_epoch 93 out=49b] v91'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f67a291440 con 0x55f67a2c7400 2023-05-20T01:25:36.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.174+0000 7f73caf9a700 1 -- 172.21.15.16:0/3803033970 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:38 13.29 13:94000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55f67ae08700 con 0x55f67a2c6800 2023-05-20T01:25:36.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.174+0000 7f73d07a5700 1 -- 172.21.15.16:0/3803033970 <== osd.6 v2:172.21.15.132:6801/727581123 8 ==== osd_op_reply(38 [pgnls start_epoch 93 out=49b] v91'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f67a291680 con 0x55f67a2c6800 2023-05-20T01:25:36.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.174+0000 7f73cb79b700 1 -- 172.21.15.16:0/3803033970 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:39 13.19 13:98000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55f67a3eaa80 con 0x55f67a3f4000 2023-05-20T01:25:36.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.174+0000 7f73d07a5700 1 -- 172.21.15.16:0/3803033970 <== osd.3 v2:172.21.15.16:6800/3669781890 3 ==== osd_op_reply(39 [pgnls start_epoch 93 out=49b] v91'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f67a291680 con 0x55f67a3f4000 2023-05-20T01:25:36.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.174+0000 7f73caf9a700 1 -- 172.21.15.16:0/3803033970 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:40 13.39 13:9c000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55f67ae09c00 con 0x55f67a2c7800 2023-05-20T01:25:36.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.174+0000 7f73d07a5700 1 -- 172.21.15.16:0/3803033970 <== osd.2 v2:172.21.15.16:6816/293187030 8 ==== osd_op_reply(40 [pgnls start_epoch 93 out=49b] v91'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f67a291680 con 0x55f67a2c7800 2023-05-20T01:25:36.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.174+0000 7f73cb79b700 1 -- 172.21.15.16:0/3803033970 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:41 13.5 13:a0000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55f67a3eae00 con 0x55f67a2c7000 2023-05-20T01:25:36.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.175+0000 7f73cf7a3700 1 -- 172.21.15.16:0/3803033970 <== osd.4 v2:172.21.15.132:6817/506317223 5 ==== osd_op_reply(41 [pgnls start_epoch 93 out=49b] v91'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f67a2918c0 con 0x55f67a2c7000 2023-05-20T01:25:36.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.175+0000 7f73caf9a700 1 -- 172.21.15.16:0/3803033970 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:42 13.25 13:a4000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55f67a3eb500 con 0x55f67a2c7000 2023-05-20T01:25:36.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.175+0000 7f73cf7a3700 1 -- 172.21.15.16:0/3803033970 <== osd.4 v2:172.21.15.132:6817/506317223 6 ==== osd_op_reply(42 [pgnls start_epoch 93 out=49b] v91'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f67a2918c0 con 0x55f67a2c7000 2023-05-20T01:25:36.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.175+0000 7f73cb79b700 1 -- 172.21.15.16:0/3803033970 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:43 13.15 13:a8000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55f67a3eb180 con 0x55f67a2c7000 2023-05-20T01:25:36.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.175+0000 7f73cf7a3700 1 -- 172.21.15.16:0/3803033970 <== osd.4 v2:172.21.15.132:6817/506317223 7 ==== osd_op_reply(43 [pgnls start_epoch 93 out=49b] v91'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f67a2918c0 con 0x55f67a2c7000 2023-05-20T01:25:36.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.175+0000 7f73caf9a700 1 -- 172.21.15.16:0/3803033970 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:44 13.35 13:ac000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55f67a3eb880 con 0x55f67a2c6c00 2023-05-20T01:25:36.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.176+0000 7f73cffa4700 1 -- 172.21.15.16:0/3803033970 <== osd.1 v2:172.21.15.16:6808/3860264220 8 ==== osd_op_reply(44 [pgnls start_epoch 93 out=49b] v91'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f67a291440 con 0x55f67a2c6c00 2023-05-20T01:25:36.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.176+0000 7f73cb79b700 1 -- 172.21.15.16:0/3803033970 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:45 13.d 13:b0000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55f67a3eae00 con 0x55f67a2c7000 2023-05-20T01:25:36.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.176+0000 7f73cf7a3700 1 -- 172.21.15.16:0/3803033970 <== osd.4 v2:172.21.15.132:6817/506317223 8 ==== osd_op_reply(45 [pgnls start_epoch 93 out=49b] v91'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f67a2918c0 con 0x55f67a2c7000 2023-05-20T01:25:36.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.176+0000 7f73caf9a700 1 -- 172.21.15.16:0/3803033970 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:46 13.2d 13:b4000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55f67a3ebc00 con 0x55f67a2c7c00 2023-05-20T01:25:36.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.176+0000 7f73cf7a3700 1 -- 172.21.15.16:0/3803033970 <== osd.0 v2:172.21.15.16:6824/3923634179 3 ==== osd_op_reply(46 [pgnls start_epoch 93 out=49b] v91'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f67a2918c0 con 0x55f67a2c7c00 2023-05-20T01:25:36.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.176+0000 7f73cb79b700 1 -- 172.21.15.16:0/3803033970 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:47 13.1d 13:b8000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55f67a3eb500 con 0x55f67a2bc800 2023-05-20T01:25:36.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.176+0000 7f73cffa4700 1 -- 172.21.15.16:0/3803033970 <== osd.7 v2:172.21.15.132:6800/466001149 5 ==== osd_op_reply(47 [pgnls start_epoch 93 out=49b] v91'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f67a291440 con 0x55f67a2bc800 2023-05-20T01:25:36.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.177+0000 7f73caf9a700 1 -- 172.21.15.16:0/3803033970 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:48 13.3d 13:bc000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55f67a3fe000 con 0x55f67a2bc800 2023-05-20T01:25:36.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.177+0000 7f73cffa4700 1 -- 172.21.15.16:0/3803033970 <== osd.7 v2:172.21.15.132:6800/466001149 6 ==== osd_op_reply(48 [pgnls start_epoch 93 out=49b] v91'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f67a291440 con 0x55f67a2bc800 2023-05-20T01:25:36.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.177+0000 7f73cb79b700 1 -- 172.21.15.16:0/3803033970 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:49 13.3 13:c0000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55f67a3eb180 con 0x55f67a2c7c00 2023-05-20T01:25:36.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.177+0000 7f73cf7a3700 1 -- 172.21.15.16:0/3803033970 <== osd.0 v2:172.21.15.16:6824/3923634179 4 ==== osd_op_reply(49 [pgnls start_epoch 93 out=49b] v91'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f67a2918c0 con 0x55f67a2c7c00 2023-05-20T01:25:36.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.177+0000 7f73caf9a700 1 -- 172.21.15.16:0/3803033970 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:50 13.23 13:c4000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55f67a3fe380 con 0x55f67a2c6c00 2023-05-20T01:25:36.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.177+0000 7f73cffa4700 1 -- 172.21.15.16:0/3803033970 <== osd.1 v2:172.21.15.16:6808/3860264220 9 ==== osd_op_reply(50 [pgnls start_epoch 93 out=49b] v91'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f67a291440 con 0x55f67a2c6c00 2023-05-20T01:25:36.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.177+0000 7f73cb79b700 1 -- 172.21.15.16:0/3803033970 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:51 13.13 13:c8000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55f67ae09180 con 0x55f67a2bc800 2023-05-20T01:25:36.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.178+0000 7f73cffa4700 1 -- 172.21.15.16:0/3803033970 <== osd.7 v2:172.21.15.132:6800/466001149 7 ==== osd_op_reply(51 [pgnls start_epoch 93 out=49b] v91'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f67a291440 con 0x55f67a2bc800 2023-05-20T01:25:36.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.178+0000 7f73caf9a700 1 -- 172.21.15.16:0/3803033970 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:52 13.33 13:cc000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55f67a3fe700 con 0x55f67a2bc800 2023-05-20T01:25:36.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.178+0000 7f73cffa4700 1 -- 172.21.15.16:0/3803033970 <== osd.7 v2:172.21.15.132:6800/466001149 8 ==== osd_op_reply(52 [pgnls start_epoch 93 out=49b] v91'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f67a291440 con 0x55f67a2bc800 2023-05-20T01:25:36.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.178+0000 7f73cb79b700 1 -- 172.21.15.16:0/3803033970 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:53 13.b 13:d0000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55f67a3fe000 con 0x55f67a2c7400 2023-05-20T01:25:36.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.178+0000 7f73cffa4700 1 -- 172.21.15.16:0/3803033970 <== osd.5 v2:172.21.15.132:6816/1844689995 5 ==== osd_op_reply(53 [pgnls start_epoch 93 out=49b] v91'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f67a291440 con 0x55f67a2c7400 2023-05-20T01:25:36.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.178+0000 7f73caf9a700 1 -- 172.21.15.16:0/3803033970 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:54 13.2b 13:d4000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55f67a3fea80 con 0x55f67a2c7800 2023-05-20T01:25:36.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.179+0000 7f73d07a5700 1 -- 172.21.15.16:0/3803033970 <== osd.2 v2:172.21.15.16:6816/293187030 9 ==== osd_op_reply(54 [pgnls start_epoch 93 out=49b] v91'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f67a291680 con 0x55f67a2c7800 2023-05-20T01:25:36.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.179+0000 7f73cb79b700 1 -- 172.21.15.16:0/3803033970 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:55 13.1b 13:d8000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55f67a3ffc00 con 0x55f67a2c7000 2023-05-20T01:25:36.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.179+0000 7f73cf7a3700 1 -- 172.21.15.16:0/3803033970 <== osd.4 v2:172.21.15.132:6817/506317223 9 ==== osd_op_reply(55 [pgnls start_epoch 93 out=49b] v91'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f67a2918c0 con 0x55f67a2c7000 2023-05-20T01:25:36.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.179+0000 7f73caf9a700 1 -- 172.21.15.16:0/3803033970 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:56 13.3b 13:dc000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55f67a3fee00 con 0x55f67a2c7000 2023-05-20T01:25:36.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.179+0000 7f73cf7a3700 1 -- 172.21.15.16:0/3803033970 <== osd.4 v2:172.21.15.132:6817/506317223 10 ==== osd_op_reply(56 [pgnls start_epoch 93 out=49b] v91'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f67a2918c0 con 0x55f67a2c7000 2023-05-20T01:25:36.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.179+0000 7f73cb79b700 1 -- 172.21.15.16:0/3803033970 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:57 13.7 13:e0000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55f67a3ea380 con 0x55f67a2bc800 2023-05-20T01:25:36.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.180+0000 7f73cffa4700 1 -- 172.21.15.16:0/3803033970 <== osd.7 v2:172.21.15.132:6800/466001149 9 ==== osd_op_reply(57 [pgnls start_epoch 93 out=81b] v91'6 uv6 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55f67a291440 con 0x55f67a2bc800 2023-05-20T01:25:36.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.180+0000 7f73d1ff67c0 1 -- 172.21.15.16:0/3803033970 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:58 13.7 13:e2e07930:::10000000001.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x55f67ae08a80 con 0x55f67a2bc800 2023-05-20T01:25:36.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.180+0000 7f73cffa4700 1 -- 172.21.15.16:0/3803033970 <== osd.7 v2:172.21.15.132:6800/466001149 10 ==== osd_op_reply(58 10000000001.00000000 [stat out=16b] v0'0 uv6 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55f67a291440 con 0x55f67a2bc800 2023-05-20T01:25:36.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.180+0000 7f73d1ff67c0 10 datascan.operator(): handling object 1099511627777.0 2023-05-20T01:25:36.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.180+0000 7f73d1ff67c0 1 -- 172.21.15.16:0/3803033970 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:59 13.7 13:e2e07930:::10000000001.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55f67ae08e00 con 0x55f67a2bc800 2023-05-20T01:25:36.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.182+0000 7f73cffa4700 1 -- 172.21.15.16:0/3803033970 <== osd.7 v2:172.21.15.132:6800/466001149 11 ==== osd_op_reply(59 10000000001.00000000 [call] v93'7 uv7 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55f67a291440 con 0x55f67a2bc800 2023-05-20T01:25:36.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.182+0000 7f73d1ff67c0 1 -- 172.21.15.16:0/3803033970 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:60 13.27 13:e4000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55f67a3ea380 con 0x55f67a2c6c00 2023-05-20T01:25:36.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.182+0000 7f73cffa4700 1 -- 172.21.15.16:0/3803033970 <== osd.1 v2:172.21.15.16:6808/3860264220 10 ==== osd_op_reply(60 [pgnls start_epoch 93 out=49b] v91'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f67a291440 con 0x55f67a2c6c00 2023-05-20T01:25:36.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.182+0000 7f73cb79b700 1 -- 172.21.15.16:0/3803033970 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:61 13.17 13:e8000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55f67a3ea700 con 0x55f67a2c7400 2023-05-20T01:25:36.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.182+0000 7f73cffa4700 1 -- 172.21.15.16:0/3803033970 <== osd.5 v2:172.21.15.132:6816/1844689995 6 ==== osd_op_reply(61 [pgnls start_epoch 93 out=49b] v91'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f67a291440 con 0x55f67a2c7400 2023-05-20T01:25:36.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.182+0000 7f73caf9a700 1 -- 172.21.15.16:0/3803033970 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:62 13.37 13:ec000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55f67a3ff180 con 0x55f67a2c7400 2023-05-20T01:25:36.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.183+0000 7f73cffa4700 1 -- 172.21.15.16:0/3803033970 <== osd.5 v2:172.21.15.132:6816/1844689995 7 ==== osd_op_reply(62 [pgnls start_epoch 93 out=49b] v91'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f67a291440 con 0x55f67a2c7400 2023-05-20T01:25:36.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.183+0000 7f73cb79b700 1 -- 172.21.15.16:0/3803033970 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:63 13.f 13:f0000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55f67a3eb880 con 0x55f67a2c6800 2023-05-20T01:25:36.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.183+0000 7f73d07a5700 1 -- 172.21.15.16:0/3803033970 <== osd.6 v2:172.21.15.132:6801/727581123 9 ==== osd_op_reply(63 [pgnls start_epoch 93 out=49b] v91'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f67a291680 con 0x55f67a2c6800 2023-05-20T01:25:36.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.183+0000 7f73caf9a700 1 -- 172.21.15.16:0/3803033970 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:64 13.2f 13:f4000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55f67a3ff500 con 0x55f67a2c7000 2023-05-20T01:25:36.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.183+0000 7f73cf7a3700 1 -- 172.21.15.16:0/3803033970 <== osd.4 v2:172.21.15.132:6817/506317223 11 ==== osd_op_reply(64 [pgnls start_epoch 93 out=49b] v91'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f67a2918c0 con 0x55f67a2c7000 2023-05-20T01:25:36.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.183+0000 7f73cb79b700 1 -- 172.21.15.16:0/3803033970 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:65 13.1f 13:f8000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55f67a3eb500 con 0x55f67a2c6800 2023-05-20T01:25:36.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.184+0000 7f73d07a5700 1 -- 172.21.15.16:0/3803033970 <== osd.6 v2:172.21.15.132:6801/727581123 10 ==== osd_op_reply(65 [pgnls start_epoch 93 out=49b] v91'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f67a291680 con 0x55f67a2c6800 2023-05-20T01:25:36.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.184+0000 7f73caf9a700 1 -- 172.21.15.16:0/3803033970 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:66 13.3f 13:fc000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55f67a3ff880 con 0x55f67a2c6800 2023-05-20T01:25:36.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.184+0000 7f73d07a5700 1 -- 172.21.15.16:0/3803033970 <== osd.6 v2:172.21.15.132:6801/727581123 11 ==== osd_op_reply(66 [pgnls start_epoch 93 out=49b] v91'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f67a291680 con 0x55f67a2c6800 2023-05-20T01:25:36.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.184+0000 7f73d1ff67c0 1 -- 172.21.15.16:0/3803033970 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:67 14.0 14:00000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55f67a3eae00 con 0x55f67a2c7800 2023-05-20T01:25:36.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.184+0000 7f73d07a5700 1 -- 172.21.15.16:0/3803033970 <== osd.2 v2:172.21.15.16:6816/293187030 10 ==== osd_op_reply(67 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f67a291680 con 0x55f67a2c7800 2023-05-20T01:25:36.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.184+0000 7f73caf9a700 1 -- 172.21.15.16:0/3803033970 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:68 14.10 14:08000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55f67a3fe380 con 0x55f67a2c6c00 2023-05-20T01:25:36.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.184+0000 7f73cffa4700 1 -- 172.21.15.16:0/3803033970 <== osd.1 v2:172.21.15.16:6808/3860264220 11 ==== osd_op_reply(68 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f67a291440 con 0x55f67a2c6c00 2023-05-20T01:25:36.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.184+0000 7f73cb79b700 1 -- 172.21.15.16:0/3803033970 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:69 14.8 14:10000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55f67ae09500 con 0x55f67a2c7c00 2023-05-20T01:25:36.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.185+0000 7f73cf7a3700 1 -- 172.21.15.16:0/3803033970 <== osd.0 v2:172.21.15.16:6824/3923634179 5 ==== osd_op_reply(69 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f67a2918c0 con 0x55f67a2c7c00 2023-05-20T01:25:36.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.185+0000 7f73caf9a700 1 -- 172.21.15.16:0/3803033970 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:70 14.18 14:18000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55f67ae09180 con 0x55f67a2c6c00 2023-05-20T01:25:36.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.185+0000 7f73cffa4700 1 -- 172.21.15.16:0/3803033970 <== osd.1 v2:172.21.15.16:6808/3860264220 12 ==== osd_op_reply(70 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f67a291440 con 0x55f67a2c6c00 2023-05-20T01:25:36.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.185+0000 7f73cb79b700 1 -- 172.21.15.16:0/3803033970 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:71 14.4 14:20000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55f67a3fe700 con 0x55f67a2c6800 2023-05-20T01:25:36.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.185+0000 7f73d07a5700 1 -- 172.21.15.16:0/3803033970 <== osd.6 v2:172.21.15.132:6801/727581123 12 ==== osd_op_reply(71 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f67a291680 con 0x55f67a2c6800 2023-05-20T01:25:36.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.185+0000 7f73caf9a700 1 -- 172.21.15.16:0/3803033970 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:72 14.14 14:28000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55f67ae08a80 con 0x55f67a2c7000 2023-05-20T01:25:36.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.186+0000 7f73cf7a3700 1 -- 172.21.15.16:0/3803033970 <== osd.4 v2:172.21.15.132:6817/506317223 12 ==== osd_op_reply(72 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f67a2918c0 con 0x55f67a2c7000 2023-05-20T01:25:36.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.186+0000 7f73cb79b700 1 -- 172.21.15.16:0/3803033970 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:73 14.c 14:30000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55f67a3fe000 con 0x55f67a2c7c00 2023-05-20T01:25:36.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.186+0000 7f73cf7a3700 1 -- 172.21.15.16:0/3803033970 <== osd.0 v2:172.21.15.16:6824/3923634179 6 ==== osd_op_reply(73 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f67a2918c0 con 0x55f67a2c7c00 2023-05-20T01:25:36.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.186+0000 7f73caf9a700 1 -- 172.21.15.16:0/3803033970 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:74 14.1c 14:38000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55f67ae08e00 con 0x55f67a2c7c00 2023-05-20T01:25:36.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.186+0000 7f73cf7a3700 1 -- 172.21.15.16:0/3803033970 <== osd.0 v2:172.21.15.16:6824/3923634179 7 ==== osd_op_reply(74 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f67a2918c0 con 0x55f67a2c7c00 2023-05-20T01:25:36.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.186+0000 7f73cb79b700 1 -- 172.21.15.16:0/3803033970 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:75 14.2 14:40000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55f67a3ffc00 con 0x55f67a2c6800 2023-05-20T01:25:36.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.186+0000 7f73d07a5700 1 -- 172.21.15.16:0/3803033970 <== osd.6 v2:172.21.15.132:6801/727581123 13 ==== osd_op_reply(75 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f67a291680 con 0x55f67a2c6800 2023-05-20T01:25:36.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.186+0000 7f73caf9a700 1 -- 172.21.15.16:0/3803033970 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:76 14.12 14:48000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55f67ae09880 con 0x55f67a2c7800 2023-05-20T01:25:36.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.187+0000 7f73d07a5700 1 -- 172.21.15.16:0/3803033970 <== osd.2 v2:172.21.15.16:6816/293187030 11 ==== osd_op_reply(76 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f67a291680 con 0x55f67a2c7800 2023-05-20T01:25:36.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.187+0000 7f73cb79b700 1 -- 172.21.15.16:0/3803033970 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:77 14.a 14:50000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55f67a3fee00 con 0x55f67a2c7400 2023-05-20T01:25:36.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.187+0000 7f73cffa4700 1 -- 172.21.15.16:0/3803033970 <== osd.5 v2:172.21.15.132:6816/1844689995 8 ==== osd_op_reply(77 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f67a291440 con 0x55f67a2c7400 2023-05-20T01:25:36.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.187+0000 7f73caf9a700 1 -- 172.21.15.16:0/3803033970 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:78 14.1a 14:58000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55f67ae08700 con 0x55f67a2c7c00 2023-05-20T01:25:36.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.187+0000 7f73cf7a3700 1 -- 172.21.15.16:0/3803033970 <== osd.0 v2:172.21.15.16:6824/3923634179 8 ==== osd_op_reply(78 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f67a2918c0 con 0x55f67a2c7c00 2023-05-20T01:25:36.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.187+0000 7f73cb79b700 1 -- 172.21.15.16:0/3803033970 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:79 14.6 14:60000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55f67a3fea80 con 0x55f67a2c7c00 2023-05-20T01:25:36.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.187+0000 7f73cf7a3700 1 -- 172.21.15.16:0/3803033970 <== osd.0 v2:172.21.15.16:6824/3923634179 9 ==== osd_op_reply(79 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f67a2918c0 con 0x55f67a2c7c00 2023-05-20T01:25:36.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.187+0000 7f73caf9a700 1 -- 172.21.15.16:0/3803033970 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:80 14.16 14:68000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55f67ae09c00 con 0x55f67a3f4000 2023-05-20T01:25:36.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.188+0000 7f73d07a5700 1 -- 172.21.15.16:0/3803033970 <== osd.3 v2:172.21.15.16:6800/3669781890 4 ==== osd_op_reply(80 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f67a291680 con 0x55f67a3f4000 2023-05-20T01:25:36.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.188+0000 7f73cb79b700 1 -- 172.21.15.16:0/3803033970 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:81 14.e 14:70000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55f67ae09500 con 0x55f67a2c7000 2023-05-20T01:25:36.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.188+0000 7f73cf7a3700 1 -- 172.21.15.16:0/3803033970 <== osd.4 v2:172.21.15.132:6817/506317223 13 ==== osd_op_reply(81 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f67a2918c0 con 0x55f67a2c7000 2023-05-20T01:25:36.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.188+0000 7f73caf9a700 1 -- 172.21.15.16:0/3803033970 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:82 14.1e 14:78000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55f67a3ea000 con 0x55f67a2bc800 2023-05-20T01:25:36.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.188+0000 7f73cffa4700 1 -- 172.21.15.16:0/3803033970 <== osd.7 v2:172.21.15.132:6800/466001149 12 ==== osd_op_reply(82 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f67a291440 con 0x55f67a2bc800 2023-05-20T01:25:36.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.188+0000 7f73cb79b700 1 -- 172.21.15.16:0/3803033970 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:83 14.1 14:80000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55f67ae08a80 con 0x55f67a3f4000 2023-05-20T01:25:36.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.189+0000 7f73d07a5700 1 -- 172.21.15.16:0/3803033970 <== osd.3 v2:172.21.15.16:6800/3669781890 5 ==== osd_op_reply(83 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f67a291680 con 0x55f67a3f4000 2023-05-20T01:25:36.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.189+0000 7f73caf9a700 1 -- 172.21.15.16:0/3803033970 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:84 14.11 14:88000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55f67a3eaa80 con 0x55f67a2c7000 2023-05-20T01:25:36.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.189+0000 7f73cf7a3700 1 -- 172.21.15.16:0/3803033970 <== osd.4 v2:172.21.15.132:6817/506317223 14 ==== osd_op_reply(84 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f67a2918c0 con 0x55f67a2c7000 2023-05-20T01:25:36.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.189+0000 7f73cb79b700 1 -- 172.21.15.16:0/3803033970 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:85 14.9 14:90000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55f67a3ff500 con 0x55f67a2c7400 2023-05-20T01:25:36.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.190+0000 7f73cffa4700 1 -- 172.21.15.16:0/3803033970 <== osd.5 v2:172.21.15.132:6816/1844689995 9 ==== osd_op_reply(85 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f67a291440 con 0x55f67a2c7400 2023-05-20T01:25:36.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.190+0000 7f73caf9a700 1 -- 172.21.15.16:0/3803033970 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:86 14.19 14:98000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55f67ae09180 con 0x55f67a2c6800 2023-05-20T01:25:36.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.190+0000 7f73d07a5700 1 -- 172.21.15.16:0/3803033970 <== osd.6 v2:172.21.15.132:6801/727581123 14 ==== osd_op_reply(86 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f67a291680 con 0x55f67a2c6800 2023-05-20T01:25:36.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.190+0000 7f73cb79b700 1 -- 172.21.15.16:0/3803033970 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:87 14.5 14:a0000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55f67a3fe000 con 0x55f67a2c7c00 2023-05-20T01:25:36.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.190+0000 7f73cf7a3700 1 -- 172.21.15.16:0/3803033970 <== osd.0 v2:172.21.15.16:6824/3923634179 10 ==== osd_op_reply(87 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f67a2918c0 con 0x55f67a2c7c00 2023-05-20T01:25:36.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.190+0000 7f73caf9a700 1 -- 172.21.15.16:0/3803033970 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:88 14.15 14:a8000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55f67ae08e00 con 0x55f67a2bc800 2023-05-20T01:25:36.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.191+0000 7f73cffa4700 1 -- 172.21.15.16:0/3803033970 <== osd.7 v2:172.21.15.132:6800/466001149 13 ==== osd_op_reply(88 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f67a291440 con 0x55f67a2bc800 2023-05-20T01:25:36.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.191+0000 7f73cb79b700 1 -- 172.21.15.16:0/3803033970 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:89 14.d 14:b0000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55f67a3ff880 con 0x55f67a2c6800 2023-05-20T01:25:36.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.191+0000 7f73d07a5700 1 -- 172.21.15.16:0/3803033970 <== osd.6 v2:172.21.15.132:6801/727581123 15 ==== osd_op_reply(89 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f67a291680 con 0x55f67a2c6800 2023-05-20T01:25:36.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.191+0000 7f73caf9a700 1 -- 172.21.15.16:0/3803033970 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:90 14.1d 14:b8000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55f67ae08700 con 0x55f67a2c7c00 2023-05-20T01:25:36.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.191+0000 7f73cf7a3700 1 -- 172.21.15.16:0/3803033970 <== osd.0 v2:172.21.15.16:6824/3923634179 11 ==== osd_op_reply(90 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f67a2918c0 con 0x55f67a2c7c00 2023-05-20T01:25:36.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.191+0000 7f73cb79b700 1 -- 172.21.15.16:0/3803033970 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:91 14.3 14:c0000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55f67a3fe700 con 0x55f67a2c7c00 2023-05-20T01:25:36.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.191+0000 7f73cf7a3700 1 -- 172.21.15.16:0/3803033970 <== osd.0 v2:172.21.15.16:6824/3923634179 12 ==== osd_op_reply(91 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f67a2918c0 con 0x55f67a2c7c00 2023-05-20T01:25:36.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.191+0000 7f73caf9a700 1 -- 172.21.15.16:0/3803033970 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:92 14.13 14:c8000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55f67a3ff180 con 0x55f67a2c7800 2023-05-20T01:25:36.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.192+0000 7f73d07a5700 1 -- 172.21.15.16:0/3803033970 <== osd.2 v2:172.21.15.16:6816/293187030 12 ==== osd_op_reply(92 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f67a291680 con 0x55f67a2c7800 2023-05-20T01:25:36.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.192+0000 7f73cb79b700 1 -- 172.21.15.16:0/3803033970 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:93 14.b 14:d0000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55f67a3ffc00 con 0x55f67a2c7c00 2023-05-20T01:25:36.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.192+0000 7f73cf7a3700 1 -- 172.21.15.16:0/3803033970 <== osd.0 v2:172.21.15.16:6824/3923634179 13 ==== osd_op_reply(93 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f67a2918c0 con 0x55f67a2c7c00 2023-05-20T01:25:36.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.192+0000 7f73caf9a700 1 -- 172.21.15.16:0/3803033970 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:94 14.1b 14:d8000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55f67a3fe380 con 0x55f67a2c6c00 2023-05-20T01:25:36.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.192+0000 7f73cffa4700 1 -- 172.21.15.16:0/3803033970 <== osd.1 v2:172.21.15.16:6808/3860264220 13 ==== osd_op_reply(94 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f67a291440 con 0x55f67a2c6c00 2023-05-20T01:25:36.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.192+0000 7f73cb79b700 1 -- 172.21.15.16:0/3803033970 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:95 14.7 14:e0000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55f67a3eb880 con 0x55f67a2c7400 2023-05-20T01:25:36.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.193+0000 7f73cffa4700 1 -- 172.21.15.16:0/3803033970 <== osd.5 v2:172.21.15.132:6816/1844689995 10 ==== osd_op_reply(95 [pgnls start_epoch 93 out=81b] v91'3 uv3 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55f67a291440 con 0x55f67a2c7400 2023-05-20T01:25:36.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.193+0000 7f73d1ff67c0 1 -- 172.21.15.16:0/3803033970 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:96 14.7 14:e2e07930:::10000000001.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x55f67a3ebc00 con 0x55f67a2c7400 2023-05-20T01:25:36.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.193+0000 7f73cffa4700 1 -- 172.21.15.16:0/3803033970 <== osd.5 v2:172.21.15.132:6816/1844689995 11 ==== osd_op_reply(96 10000000001.00000000 [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55f67a291440 con 0x55f67a2c7400 2023-05-20T01:25:36.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.193+0000 7f73d1ff67c0 10 datascan.operator(): handling object 1099511627777.0 2023-05-20T01:25:36.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.193+0000 7f73d1ff67c0 1 -- 172.21.15.16:0/3803033970 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:97 13.7 13:e2e07930:::10000000001.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55f67a3eb180 con 0x55f67a2bc800 2023-05-20T01:25:36.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.194+0000 7f73cffa4700 1 -- 172.21.15.16:0/3803033970 <== osd.7 v2:172.21.15.132:6800/466001149 14 ==== osd_op_reply(97 10000000001.00000000 [call,setxattr (8)] v93'8 uv8 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x55f67a291440 con 0x55f67a2bc800 2023-05-20T01:25:36.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.194+0000 7f73d1ff67c0 1 -- 172.21.15.16:0/3803033970 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:98 14.17 14:e8000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55f67a3fea80 con 0x55f67a2c6c00 2023-05-20T01:25:36.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.194+0000 7f73cffa4700 1 -- 172.21.15.16:0/3803033970 <== osd.1 v2:172.21.15.16:6808/3860264220 14 ==== osd_op_reply(98 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f67a291440 con 0x55f67a2c6c00 2023-05-20T01:25:36.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.194+0000 7f73cb79b700 1 -- 172.21.15.16:0/3803033970 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:99 14.f 14:f0000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55f67a3eb500 con 0x55f67a2bc800 2023-05-20T01:25:36.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.195+0000 7f73cffa4700 1 -- 172.21.15.16:0/3803033970 <== osd.7 v2:172.21.15.132:6800/466001149 15 ==== osd_op_reply(99 [pgnls start_epoch 93 out=81b] v90'1 uv1 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55f67a291440 con 0x55f67a2bc800 2023-05-20T01:25:36.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.195+0000 7f73d1ff67c0 1 -- 172.21.15.16:0/3803033970 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:100 14.f 14:f526278d:::10000000001.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x55f67a3fe000 con 0x55f67a2bc800 2023-05-20T01:25:36.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.195+0000 7f73cffa4700 1 -- 172.21.15.16:0/3803033970 <== osd.7 v2:172.21.15.132:6800/466001149 16 ==== osd_op_reply(100 10000000001.00000001 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55f67a291440 con 0x55f67a2bc800 2023-05-20T01:25:36.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.195+0000 7f73d1ff67c0 10 datascan.operator(): handling object 1099511627777.1 2023-05-20T01:25:36.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.195+0000 7f73d1ff67c0 1 -- 172.21.15.16:0/3803033970 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:101 13.7 13:e2e07930:::10000000001.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55f67a3fe700 con 0x55f67a2bc800 2023-05-20T01:25:36.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.196+0000 7f73cffa4700 1 -- 172.21.15.16:0/3803033970 <== osd.7 v2:172.21.15.132:6800/466001149 17 ==== osd_op_reply(101 10000000001.00000000 [call,setxattr (8)] v93'9 uv9 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x55f67a291440 con 0x55f67a2bc800 2023-05-20T01:25:36.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.196+0000 7f73d1ff67c0 1 -- 172.21.15.16:0/3803033970 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:102 14.1f 14:f8000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55f67a3ffc00 con 0x55f67a2c7c00 2023-05-20T01:25:36.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.197+0000 7f73cf7a3700 1 -- 172.21.15.16:0/3803033970 <== osd.0 v2:172.21.15.16:6824/3923634179 14 ==== osd_op_reply(102 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f67a2918c0 con 0x55f67a2c7c00 2023-05-20T01:25:36.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.197+0000 7f73d1ff67c0 1 -- 172.21.15.16:0/3803033970 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x55f67a2c7c00 msgr2=0x55f67a3d1b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:36.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.197+0000 7f73d1ff67c0 1 --2- 172.21.15.16:0/3803033970 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x55f67a2c7c00 0x55f67a3d1b80 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:36.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.197+0000 7f73d1ff67c0 1 -- 172.21.15.16:0/3803033970 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x55f67a2c6c00 msgr2=0x55f67a3d0580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:36.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.197+0000 7f73d1ff67c0 1 --2- 172.21.15.16:0/3803033970 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x55f67a2c6c00 0x55f67a3d0580 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:36.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.197+0000 7f73d1ff67c0 1 -- 172.21.15.16:0/3803033970 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x55f67a2c7800 msgr2=0x55f67a3d1600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:36.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.197+0000 7f73d1ff67c0 1 --2- 172.21.15.16:0/3803033970 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x55f67a2c7800 0x55f67a3d1600 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:36.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.197+0000 7f73d1ff67c0 1 -- 172.21.15.16:0/3803033970 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x55f67a3f4000 msgr2=0x55f67a3d2100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:36.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.197+0000 7f73d1ff67c0 1 --2- 172.21.15.16:0/3803033970 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x55f67a3f4000 0x55f67a3d2100 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:36.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.197+0000 7f73cf7a3700 1 -- 172.21.15.16:0/3803033970 reap_dead start 2023-05-20T01:25:36.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.197+0000 7f73d1ff67c0 1 -- 172.21.15.16:0/3803033970 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x55f67a2c7000 msgr2=0x55f67a3d0b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:36.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.197+0000 7f73d1ff67c0 1 --2- 172.21.15.16:0/3803033970 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x55f67a2c7000 0x55f67a3d0b00 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:36.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.197+0000 7f73cf7a3700 1 -- 172.21.15.16:0/3803033970 reap_dead start 2023-05-20T01:25:36.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.197+0000 7f73d1ff67c0 1 -- 172.21.15.16:0/3803033970 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x55f67a2c7400 msgr2=0x55f67a3d1080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:36.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.197+0000 7f73d1ff67c0 1 --2- 172.21.15.16:0/3803033970 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x55f67a2c7400 0x55f67a3d1080 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:36.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.197+0000 7f73d1ff67c0 1 -- 172.21.15.16:0/3803033970 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x55f67a2c6800 msgr2=0x55f67a3d0000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:36.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.197+0000 7f73d1ff67c0 1 --2- 172.21.15.16:0/3803033970 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x55f67a2c6800 0x55f67a3d0000 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:36.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.197+0000 7f73d1ff67c0 1 -- 172.21.15.16:0/3803033970 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x55f67a2bc800 msgr2=0x55f67a01e100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:36.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.197+0000 7f73d1ff67c0 1 --2- 172.21.15.16:0/3803033970 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x55f67a2bc800 0x55f67a01e100 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:36.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.197+0000 7f73d1ff67c0 1 -- 172.21.15.16:0/3803033970 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x55f67a2c6400 msgr2=0x55f67a01f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:36.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.197+0000 7f73d1ff67c0 1 --2- 172.21.15.16:0/3803033970 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x55f67a2c6400 0x55f67a01f700 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x55f679fdd590 tx=0x55f67ad1b2c0 comp rx=0 tx=0).stop 2023-05-20T01:25:36.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.197+0000 7f73d1ff67c0 1 -- 172.21.15.16:0/3803033970 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f67a2bc400 msgr2=0x55f67a01e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:36.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.197+0000 7f73d1ff67c0 1 --2- 172.21.15.16:0/3803033970 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f67a2bc400 0x55f67a01e680 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x55f67a2a7380 tx=0x55f67ad1b3e0 comp rx=0 tx=0).stop 2023-05-20T01:25:36.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.197+0000 7f73cf7a3700 1 -- 172.21.15.16:0/3803033970 reap_dead start 2023-05-20T01:25:36.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.197+0000 7f73d1ff67c0 1 -- 172.21.15.16:0/3803033970 shutdown_connections 2023-05-20T01:25:36.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.197+0000 7f73d1ff67c0 1 -- 172.21.15.16:0/3803033970 >> 172.21.15.16:0/3803033970 conn(0x55f679f83c00 msgr2=0x55f67a05f800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:36.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.197+0000 7f73d1ff67c0 1 -- 172.21.15.16:0/3803033970 shutdown_connections 2023-05-20T01:25:36.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.197+0000 7f73d1ff67c0 1 -- 172.21.15.16:0/3803033970 wait complete. 2023-05-20T01:25:36.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.197+0000 7f73d1ff67c0 1 -- 172.21.15.16:0/3808048638 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f679f83000 msgr2=0x55f67a01c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:36.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.197+0000 7f73d1ff67c0 1 --2- 172.21.15.16:0/3808048638 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f679f83000 0x55f67a01c580 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x55f67a0dee40 tx=0x55f67a0ecea0 comp rx=0 tx=0).stop 2023-05-20T01:25:36.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.198+0000 7f73d1ff67c0 1 -- 172.21.15.16:0/3808048638 shutdown_connections 2023-05-20T01:25:36.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.198+0000 7f73d1ff67c0 1 --2- 172.21.15.16:0/3808048638 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55f679f82800 0x55f67a01d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:36.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.198+0000 7f73d1ff67c0 1 --2- 172.21.15.16:0/3808048638 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f679f83000 0x55f67a01c580 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:36.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.198+0000 7f73d1ff67c0 1 --2- 172.21.15.16:0/3808048638 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f679f82c00 0x55f67a01cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:36.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.198+0000 7f73d1ff67c0 1 -- 172.21.15.16:0/3808048638 >> 172.21.15.16:0/3808048638 conn(0x55f679f83400 msgr2=0x55f67a05f000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:36.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.198+0000 7f73d1ff67c0 1 -- 172.21.15.16:0/3808048638 shutdown_connections 2023-05-20T01:25:36.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.198+0000 7f73d1ff67c0 1 -- 172.21.15.16:0/3808048638 wait complete. 2023-05-20T01:25:36.227 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_extents'] in time 0:00:00.147167, result: 2023-05-20T01:25:36.228 DEBUG:teuthology.orchestra.run.smithi016:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_inodes 2023-05-20T01:25:36.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.280+0000 7fe14ac697c0 1 -- start start 2023-05-20T01:25:36.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.280+0000 7fe14ac697c0 1 Processor -- start 2023-05-20T01:25:36.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.281+0000 7fe14ac697c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f636a1000 0x559f6373bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:36.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.281+0000 7fe14ac697c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559f636a0c00 0x559f6373b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:36.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.281+0000 7fe14ac697c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559f636a0800 0x559f6373b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:36.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.281+0000 7fe14ac697c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f636ddcc0 con 0x559f636a1000 2023-05-20T01:25:36.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.281+0000 7fe14ac697c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f636dd400 con 0x559f636a0800 2023-05-20T01:25:36.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.281+0000 7fe14ac697c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x559f636dca00 con 0x559f636a0c00 2023-05-20T01:25:36.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.281+0000 7fe148c17700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f636a1000 0x559f6373bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:36.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.281+0000 7fe148c17700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f636a1000 0x559f6373bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37010/0 (socket says 172.21.15.16:37010) 2023-05-20T01:25:36.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.281+0000 7fe148c17700 1 -- 172.21.15.16:0/1036135947 learned_addr learned my addr 172.21.15.16:0/1036135947 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:36.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.281+0000 7fe149418700 1 --2- 172.21.15.16:0/1036135947 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559f636a0800 0x559f6373b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:36.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.281+0000 7fe148c17700 1 -- 172.21.15.16:0/1036135947 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559f636a0c00 msgr2=0x559f6373b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:36.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.281+0000 7fe148416700 1 --2- 172.21.15.16:0/1036135947 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559f636a0c00 0x559f6373b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:36.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.281+0000 7fe148c17700 1 --2- 172.21.15.16:0/1036135947 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559f636a0c00 0x559f6373b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:36.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.281+0000 7fe148c17700 1 -- 172.21.15.16:0/1036135947 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559f636a0800 msgr2=0x559f6373b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:36.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.281+0000 7fe148c17700 1 --2- 172.21.15.16:0/1036135947 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559f636a0800 0x559f6373b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:36.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.281+0000 7fe148c17700 1 -- 172.21.15.16:0/1036135947 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559f636c4d00 con 0x559f636a1000 2023-05-20T01:25:36.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.281+0000 7fe148c17700 1 --2- 172.21.15.16:0/1036135947 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f636a1000 0x559f6373bb80 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x559f64436d50 tx=0x559f64439320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=335c39aa8a2ba2c4 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:36.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.281+0000 7fe146412700 1 -- 172.21.15.16:0/1036135947 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x559f636d5080 con 0x559f636a1000 2023-05-20T01:25:36.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.281+0000 7fe146412700 1 -- 172.21.15.16:0/1036135947 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559f636f3800 con 0x559f636a1000 2023-05-20T01:25:36.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.281+0000 7fe146412700 1 -- 172.21.15.16:0/1036135947 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x559f636d5600 con 0x559f636a1000 2023-05-20T01:25:36.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.282+0000 7fe14ac697c0 1 -- 172.21.15.16:0/1036135947 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559f636c56c0 con 0x559f636a1000 2023-05-20T01:25:36.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.282+0000 7fe14ac697c0 1 -- 172.21.15.16:0/1036135947 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x559f636c5520 con 0x559f636a1000 2023-05-20T01:25:36.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.282+0000 7fe14ac697c0 4 waiting for MDS map... 2023-05-20T01:25:36.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.282+0000 7fe146412700 1 -- 172.21.15.16:0/1036135947 <== mon.0 v2:172.21.15.16:3300/0 4 ==== osd_map(93..93 src has 1..93) v4 ==== 6958+0+0 (secure 0 0 0) 0x559f636e2c40 con 0x559f636a1000 2023-05-20T01:25:36.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.282+0000 7fe146412700 1 -- 172.21.15.16:0/1036135947 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 63) v1 ==== 3126+0+0 (secure 0 0 0) 0x559f636e6f00 con 0x559f636a1000 2023-05-20T01:25:36.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.282+0000 7fe14ac697c0 4 Got MDS map 63 2023-05-20T01:25:36.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.282+0000 7fe14ac697c0 4 datascan.main: Using metadata pool output 2023-05-20T01:25:36.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.282+0000 7fe14ac697c0 4 datascan.main: connecting to RADOS... 2023-05-20T01:25:36.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.282+0000 7fe14ac697c0 1 Processor -- start 2023-05-20T01:25:36.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.283+0000 7fe14ac697c0 1 -- start start 2023-05-20T01:25:36.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.283+0000 7fe14ac697c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f636a1c00 0x559f6373ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:36.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.283+0000 7fe14ac697c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559f639ba000 0x559f6373a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:36.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.283+0000 7fe14ac697c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559f639ba400 0x559f6373c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:36.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.283+0000 7fe14ac697c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f636dcb40 con 0x559f636a1c00 2023-05-20T01:25:36.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.283+0000 7fe14ac697c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f636dca00 con 0x559f639ba000 2023-05-20T01:25:36.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.283+0000 7fe14ac697c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x559f636dd400 con 0x559f639ba400 2023-05-20T01:25:36.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.283+0000 7fe149418700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f636a1c00 0x559f6373ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:36.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.283+0000 7fe149418700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f636a1c00 0x559f6373ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37016/0 (socket says 172.21.15.16:37016) 2023-05-20T01:25:36.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.283+0000 7fe149418700 1 -- 172.21.15.16:0/3777707909 learned_addr learned my addr 172.21.15.16:0/3777707909 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:36.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.284+0000 7fe148c17700 1 --2- 172.21.15.16:0/3777707909 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559f639ba000 0x559f6373a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:36.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.284+0000 7fe149418700 1 -- 172.21.15.16:0/3777707909 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559f639ba400 msgr2=0x559f6373c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:36.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.284+0000 7fe148416700 1 --2- 172.21.15.16:0/3777707909 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559f639ba400 0x559f6373c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:36.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.284+0000 7fe149418700 1 --2- 172.21.15.16:0/3777707909 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559f639ba400 0x559f6373c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:36.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.284+0000 7fe149418700 1 -- 172.21.15.16:0/3777707909 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559f639ba000 msgr2=0x559f6373a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:36.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.284+0000 7fe149418700 1 --2- 172.21.15.16:0/3777707909 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559f639ba000 0x559f6373a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:36.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.284+0000 7fe149418700 1 -- 172.21.15.16:0/3777707909 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559f636c5380 con 0x559f636a1c00 2023-05-20T01:25:36.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.284+0000 7fe148c17700 1 --2- 172.21.15.16:0/3777707909 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559f639ba000 0x559f6373a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:25:36.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.284+0000 7fe148416700 1 --2- 172.21.15.16:0/3777707909 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559f639ba400 0x559f6373c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:25:36.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.284+0000 7fe149418700 1 --2- 172.21.15.16:0/3777707909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f636a1c00 0x559f6373ab00 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x559f639c00c0 tx=0x559f64474f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96fab578ef57c663 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:36.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.284+0000 7fe14440e700 1 -- 172.21.15.16:0/3777707909 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x559f636d5080 con 0x559f636a1c00 2023-05-20T01:25:36.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.284+0000 7fe14440e700 1 -- 172.21.15.16:0/3777707909 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559f636f2480 con 0x559f636a1c00 2023-05-20T01:25:36.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.284+0000 7fe14440e700 1 -- 172.21.15.16:0/3777707909 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x559f636d5600 con 0x559f636a1c00 2023-05-20T01:25:36.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.284+0000 7fe14ac697c0 1 -- 172.21.15.16:0/3777707909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f636a1c00 msgr2=0x559f6373ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:36.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.284+0000 7fe14ac697c0 1 --2- 172.21.15.16:0/3777707909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f636a1c00 0x559f6373ab00 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x559f639c00c0 tx=0x559f64474f00 comp rx=0 tx=0).stop 2023-05-20T01:25:36.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.285+0000 7fe14ac697c0 1 -- 172.21.15.16:0/3777707909 shutdown_connections 2023-05-20T01:25:36.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.285+0000 7fe14ac697c0 1 --2- 172.21.15.16:0/3777707909 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559f639ba400 0x559f6373c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:36.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.285+0000 7fe14ac697c0 1 --2- 172.21.15.16:0/3777707909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f636a1c00 0x559f6373ab00 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:36.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.285+0000 7fe14ac697c0 1 --2- 172.21.15.16:0/3777707909 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559f639ba000 0x559f6373a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:36.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.285+0000 7fe14ac697c0 1 -- 172.21.15.16:0/3777707909 >> 172.21.15.16:0/3777707909 conn(0x559f636a0400 msgr2=0x559f63771800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:36.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.285+0000 7fe14ac697c0 1 -- 172.21.15.16:0/3777707909 shutdown_connections 2023-05-20T01:25:36.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.285+0000 7fe14ac697c0 1 -- 172.21.15.16:0/3777707909 wait complete. 2023-05-20T01:25:36.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.285+0000 7fe14ac697c0 1 Processor -- start 2023-05-20T01:25:36.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.285+0000 7fe14ac697c0 1 -- start start 2023-05-20T01:25:36.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.286+0000 7fe14ac697c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559f636a0400 0x559f6373ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:36.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.286+0000 7fe14ac697c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559f639d6000 0x559f6373a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:36.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.286+0000 7fe14ac697c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f639d6400 0x559f6373c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:36.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.286+0000 7fe14ac697c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f636dcc80 con 0x559f639d6400 2023-05-20T01:25:36.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.286+0000 7fe14ac697c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f636dc000 con 0x559f636a0400 2023-05-20T01:25:36.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.286+0000 7fe14ac697c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x559f636dd180 con 0x559f639d6000 2023-05-20T01:25:36.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.286+0000 7fe148416700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f639d6400 0x559f6373c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:36.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.286+0000 7fe148416700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f639d6400 0x559f6373c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37026/0 (socket says 172.21.15.16:37026) 2023-05-20T01:25:36.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.286+0000 7fe148416700 1 -- 172.21.15.16:0/1013299390 learned_addr learned my addr 172.21.15.16:0/1013299390 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:36.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.286+0000 7fe149418700 1 --2- 172.21.15.16:0/1013299390 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559f636a0400 0x559f6373ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:36.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.286+0000 7fe148c17700 1 --2- 172.21.15.16:0/1013299390 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559f639d6000 0x559f6373a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:36.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.286+0000 7fe148416700 1 -- 172.21.15.16:0/1013299390 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559f639d6000 msgr2=0x559f6373a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:36.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.286+0000 7fe148416700 1 --2- 172.21.15.16:0/1013299390 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559f639d6000 0x559f6373a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:36.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.286+0000 7fe148416700 1 -- 172.21.15.16:0/1013299390 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559f636a0400 msgr2=0x559f6373ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:36.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.286+0000 7fe148416700 1 --2- 172.21.15.16:0/1013299390 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559f636a0400 0x559f6373ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:36.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.286+0000 7fe148416700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559f636c51e0 con 0x559f639d6400 2023-05-20T01:25:36.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.286+0000 7fe149418700 1 --2- 172.21.15.16:0/1013299390 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559f636a0400 0x559f6373ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:25:36.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.286+0000 7fe148c17700 1 --2- 172.21.15.16:0/1013299390 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559f639d6000 0x559f6373a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:25:36.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.286+0000 7fe148416700 1 --2- 172.21.15.16:0/1013299390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f639d6400 0x559f6373c100 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x559f639c1410 tx=0x559f644750e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:36.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.286+0000 7fe142c0b700 1 -- 172.21.15.16:0/1013299390 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x559f636d5080 con 0x559f639d6400 2023-05-20T01:25:36.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.286+0000 7fe142c0b700 1 -- 172.21.15.16:0/1013299390 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559f6452e780 con 0x559f639d6400 2023-05-20T01:25:36.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.286+0000 7fe14ac697c0 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x559f636c5d40 con 0x559f639d6400 2023-05-20T01:25:36.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.286+0000 7fe142c0b700 1 -- 172.21.15.16:0/1013299390 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x559f636d5600 con 0x559f639d6400 2023-05-20T01:25:36.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.287+0000 7fe14ac697c0 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559f636c4d00 con 0x559f639d6400 2023-05-20T01:25:36.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.287+0000 7fe14ac697c0 4 datascan.init: resolving metadata pool 12 2023-05-20T01:25:36.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.288+0000 7fe142c0b700 1 -- 172.21.15.16:0/1013299390 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x559f63702840 con 0x559f639d6400 2023-05-20T01:25:36.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.288+0000 7fe142c0b700 1 --2- 172.21.15.16:0/1013299390 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x559f639dc400 0x559f63ad0000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:36.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.288+0000 7fe142c0b700 1 -- 172.21.15.16:0/1013299390 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6958+0+0 (secure 0 0 0) 0x559f636e2a80 con 0x559f639d6400 2023-05-20T01:25:36.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.288+0000 7fe14ac697c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-05-20T01:25:36.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.288+0000 7fe14ac697c0 4 datascan.main: data pool 'cephfs_data' has ID 13 2023-05-20T01:25:36.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.288+0000 7fe14ac697c0 4 datascan.main: opening data pool 'cephfs_data' 2023-05-20T01:25:36.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.288+0000 7fe149418700 1 --2- 172.21.15.16:0/1013299390 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x559f639dc400 0x559f63ad0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:36.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.288+0000 7fe14ac697c0 1 --2- 172.21.15.16:0/1013299390 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x559f639d6800 0x559f6373c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:36.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.288+0000 7fe14ac697c0 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:1 12.14 12:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f64524700 con 0x559f639d6800 2023-05-20T01:25:36.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.288+0000 7fe148c17700 1 --2- 172.21.15.16:0/1013299390 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x559f639d6800 0x559f6373c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:36.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.288+0000 7fe149418700 1 --2- 172.21.15.16:0/1013299390 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x559f639dc400 0x559f63ad0000 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x559f639c00c0 tx=0x559f64474000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:36.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.289+0000 7fe148c17700 1 --2- 172.21.15.16:0/1013299390 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x559f639d6800 0x559f6373c680 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:36.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.289+0000 7fe148c17700 1 -- 172.21.15.16:0/1013299390 <== osd.5 v2:172.21.15.132:6816/1844689995 1 ==== osd_op_reply(1 1.00000000.inode [stat out=16b] v0'0 uv29 ondisk = 0) v8 ==== 160+0+16 (crc 0 0 0) 0x559f639b5440 con 0x559f639d6800 2023-05-20T01:25:36.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.289+0000 7fe14ac697c0 1 --2- 172.21.15.16:0/1013299390 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x559f639d6c00 0x559f6373cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:36.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.289+0000 7fe14ac697c0 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:2 12.1f 12:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f64524a80 con 0x559f639d6c00 2023-05-20T01:25:36.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.289+0000 7fe149418700 1 --2- 172.21.15.16:0/1013299390 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x559f639d6c00 0x559f6373cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:36.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.290+0000 7fe149418700 1 --2- 172.21.15.16:0/1013299390 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x559f639d6c00 0x559f6373cc00 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:36.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.290+0000 7fe149418700 1 -- 172.21.15.16:0/1013299390 <== osd.6 v2:172.21.15.132:6801/727581123 1 ==== osd_op_reply(2 100.00000000.inode [stat out=16b] v0'0 uv10 ondisk = 0) v8 ==== 162+0+16 (crc 0 0 0) 0x559f639b5680 con 0x559f639d6c00 2023-05-20T01:25:36.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.290+0000 7fe14ac697c0 1 --2- 172.21.15.16:0/1013299390 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x559f639d7000 0x559f6373d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:36.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.290+0000 7fe14ac697c0 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:3 13.0 13:00000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f64524e00 con 0x559f639d7000 2023-05-20T01:25:36.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.290+0000 7fe148c17700 1 --2- 172.21.15.16:0/1013299390 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x559f639d7000 0x559f6373d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:36.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.291+0000 7fe148c17700 1 --2- 172.21.15.16:0/1013299390 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x559f639d7000 0x559f6373d180 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:36.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.291+0000 7fe148c17700 1 -- 172.21.15.16:0/1013299390 <== osd.7 v2:172.21.15.132:6800/466001149 1 ==== osd_op_reply(3 [pgnls-filter start_epoch 93 out=49b] v91'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b5440 con 0x559f639d7000 2023-05-20T01:25:36.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.291+0000 7fe143c0d700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:4 13.20 13:04000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f64525180 con 0x559f639d6c00 2023-05-20T01:25:36.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.291+0000 7fe149418700 1 -- 172.21.15.16:0/1013299390 <== osd.6 v2:172.21.15.132:6801/727581123 2 ==== osd_op_reply(4 [pgnls-filter start_epoch 93 out=49b] v91'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b5680 con 0x559f639d6c00 2023-05-20T01:25:36.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.291+0000 7fe14440e700 1 --2- 172.21.15.16:0/1013299390 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x559f639dc800 0x559f63ad0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:36.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.291+0000 7fe14440e700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:5 13.10 13:08000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f64524a80 con 0x559f639dc800 2023-05-20T01:25:36.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.291+0000 7fe148416700 1 --2- 172.21.15.16:0/1013299390 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x559f639dc800 0x559f63ad0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:36.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.292+0000 7fe148416700 1 --2- 172.21.15.16:0/1013299390 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x559f639dc800 0x559f63ad0580 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:36.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.292+0000 7fe148416700 1 -- 172.21.15.16:0/1013299390 <== osd.1 v2:172.21.15.16:6808/3860264220 1 ==== osd_op_reply(5 [pgnls-filter start_epoch 93 out=49b] v91'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b58c0 con 0x559f639dc800 2023-05-20T01:25:36.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.292+0000 7fe143c0d700 1 --2- 172.21.15.16:0/1013299390 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x559f639dcc00 0x559f63ad0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:36.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.292+0000 7fe143c0d700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:6 13.30 13:0c000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f64525180 con 0x559f639dcc00 2023-05-20T01:25:36.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.292+0000 7fe149418700 1 --2- 172.21.15.16:0/1013299390 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x559f639dcc00 0x559f63ad0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:36.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.293+0000 7fe149418700 1 --2- 172.21.15.16:0/1013299390 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x559f639dcc00 0x559f63ad0b00 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:36.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.293+0000 7fe149418700 1 -- 172.21.15.16:0/1013299390 <== osd.4 v2:172.21.15.132:6817/506317223 1 ==== osd_op_reply(6 [pgnls-filter start_epoch 93 out=49b] v91'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b5680 con 0x559f639dcc00 2023-05-20T01:25:36.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.293+0000 7fe14440e700 1 --2- 172.21.15.16:0/1013299390 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x559f639dd000 0x559f63ad1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:36.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.293+0000 7fe14440e700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:7 13.8 13:10000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f64525880 con 0x559f639dd000 2023-05-20T01:25:36.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.293+0000 7fe148c17700 1 --2- 172.21.15.16:0/1013299390 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x559f639dd000 0x559f63ad1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:36.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.293+0000 7fe148c17700 1 --2- 172.21.15.16:0/1013299390 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x559f639dd000 0x559f63ad1080 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:36.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.294+0000 7fe148c17700 1 -- 172.21.15.16:0/1013299390 <== osd.2 v2:172.21.15.16:6816/293187030 1 ==== osd_op_reply(7 [pgnls-filter start_epoch 93 out=49b] v91'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b5440 con 0x559f639dd000 2023-05-20T01:25:36.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.294+0000 7fe143c0d700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:8 13.28 13:14000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f64525500 con 0x559f639dd000 2023-05-20T01:25:36.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.294+0000 7fe148c17700 1 -- 172.21.15.16:0/1013299390 <== osd.2 v2:172.21.15.16:6816/293187030 2 ==== osd_op_reply(8 [pgnls-filter start_epoch 93 out=49b] v91'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b5440 con 0x559f639dd000 2023-05-20T01:25:36.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.294+0000 7fe14440e700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:9 13.18 13:18000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f64525c00 con 0x559f639d7000 2023-05-20T01:25:36.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.294+0000 7fe148c17700 1 -- 172.21.15.16:0/1013299390 <== osd.7 v2:172.21.15.132:6800/466001149 2 ==== osd_op_reply(9 [pgnls-filter start_epoch 93 out=49b] v91'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b5440 con 0x559f639d7000 2023-05-20T01:25:36.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.294+0000 7fe143c0d700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:10 13.38 13:1c000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f64524700 con 0x559f639d6800 2023-05-20T01:25:36.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.294+0000 7fe148c17700 1 -- 172.21.15.16:0/1013299390 <== osd.5 v2:172.21.15.132:6816/1844689995 2 ==== osd_op_reply(10 [pgnls-filter start_epoch 93 out=49b] v91'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b5440 con 0x559f639d6800 2023-05-20T01:25:36.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.295+0000 7fe14440e700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:11 13.4 13:20000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f63b02000 con 0x559f639d6c00 2023-05-20T01:25:36.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.295+0000 7fe149418700 1 -- 172.21.15.16:0/1013299390 <== osd.6 v2:172.21.15.132:6801/727581123 3 ==== osd_op_reply(11 [pgnls-filter start_epoch 93 out=49b] v91'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b5680 con 0x559f639d6c00 2023-05-20T01:25:36.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.295+0000 7fe143c0d700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:12 13.24 13:24000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f64524e00 con 0x559f639d6c00 2023-05-20T01:25:36.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.295+0000 7fe149418700 1 -- 172.21.15.16:0/1013299390 <== osd.6 v2:172.21.15.132:6801/727581123 4 ==== osd_op_reply(12 [pgnls-filter start_epoch 93 out=49b] v91'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b5680 con 0x559f639d6c00 2023-05-20T01:25:36.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.295+0000 7fe14440e700 1 --2- 172.21.15.16:0/1013299390 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x559f639dd400 0x559f63ad1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:36.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.295+0000 7fe14440e700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:13 13.14 13:28000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f63b02380 con 0x559f639dd400 2023-05-20T01:25:36.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.295+0000 7fe148416700 1 --2- 172.21.15.16:0/1013299390 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x559f639dd400 0x559f63ad1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:36.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.296+0000 7fe148416700 1 --2- 172.21.15.16:0/1013299390 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x559f639dd400 0x559f63ad1600 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:36.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.296+0000 7fe148416700 1 -- 172.21.15.16:0/1013299390 <== osd.0 v2:172.21.15.16:6824/3923634179 1 ==== osd_op_reply(13 [pgnls-filter start_epoch 93 out=49b] v91'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b58c0 con 0x559f639dd400 2023-05-20T01:25:36.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.296+0000 7fe143c0d700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:14 13.34 13:2c000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f64525880 con 0x559f639d6c00 2023-05-20T01:25:36.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.296+0000 7fe149418700 1 -- 172.21.15.16:0/1013299390 <== osd.6 v2:172.21.15.132:6801/727581123 5 ==== osd_op_reply(14 [pgnls-filter start_epoch 93 out=49b] v91'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b5680 con 0x559f639d6c00 2023-05-20T01:25:36.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.296+0000 7fe14440e700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:15 13.c 13:30000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f63b02700 con 0x559f639dc800 2023-05-20T01:25:36.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.296+0000 7fe148416700 1 -- 172.21.15.16:0/1013299390 <== osd.1 v2:172.21.15.16:6808/3860264220 2 ==== osd_op_reply(15 [pgnls-filter start_epoch 93 out=49b] v91'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b58c0 con 0x559f639dc800 2023-05-20T01:25:36.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.296+0000 7fe143c0d700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:16 13.2c 13:34000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f63b02380 con 0x559f639dd000 2023-05-20T01:25:36.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.297+0000 7fe148c17700 1 -- 172.21.15.16:0/1013299390 <== osd.2 v2:172.21.15.16:6816/293187030 3 ==== osd_op_reply(16 [pgnls-filter start_epoch 93 out=49b] v91'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b5440 con 0x559f639dd000 2023-05-20T01:25:36.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.297+0000 7fe14440e700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:17 13.1c 13:38000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f63b02380 con 0x559f639d6c00 2023-05-20T01:25:36.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.297+0000 7fe149418700 1 -- 172.21.15.16:0/1013299390 <== osd.6 v2:172.21.15.132:6801/727581123 6 ==== osd_op_reply(17 [pgnls-filter start_epoch 93 out=49b] v91'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b5680 con 0x559f639d6c00 2023-05-20T01:25:36.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.297+0000 7fe143c0d700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:18 13.3c 13:3c000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f64524a80 con 0x559f639dd000 2023-05-20T01:25:36.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.297+0000 7fe148c17700 1 -- 172.21.15.16:0/1013299390 <== osd.2 v2:172.21.15.16:6816/293187030 4 ==== osd_op_reply(18 [pgnls-filter start_epoch 93 out=49b] v91'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b5440 con 0x559f639dd000 2023-05-20T01:25:36.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.297+0000 7fe14440e700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:19 13.2 13:40000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f64525500 con 0x559f639d6800 2023-05-20T01:25:36.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.298+0000 7fe148c17700 1 -- 172.21.15.16:0/1013299390 <== osd.5 v2:172.21.15.132:6816/1844689995 3 ==== osd_op_reply(19 [pgnls-filter start_epoch 93 out=49b] v91'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b5440 con 0x559f639d6800 2023-05-20T01:25:36.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.298+0000 7fe143c0d700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:20 13.22 13:44000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f64525180 con 0x559f639dc800 2023-05-20T01:25:36.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.298+0000 7fe148416700 1 -- 172.21.15.16:0/1013299390 <== osd.1 v2:172.21.15.16:6808/3860264220 3 ==== osd_op_reply(20 [pgnls-filter start_epoch 93 out=49b] v91'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b58c0 con 0x559f639dc800 2023-05-20T01:25:36.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.298+0000 7fe14440e700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:21 13.12 13:48000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f64525c00 con 0x559f639dd400 2023-05-20T01:25:36.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.298+0000 7fe148416700 1 -- 172.21.15.16:0/1013299390 <== osd.0 v2:172.21.15.16:6824/3923634179 2 ==== osd_op_reply(21 [pgnls-filter start_epoch 93 out=49b] v91'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b58c0 con 0x559f639dd400 2023-05-20T01:25:36.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.298+0000 7fe143c0d700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:22 13.32 13:4c000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f64524e00 con 0x559f639dd000 2023-05-20T01:25:36.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.299+0000 7fe148c17700 1 -- 172.21.15.16:0/1013299390 <== osd.2 v2:172.21.15.16:6816/293187030 5 ==== osd_op_reply(22 [pgnls-filter start_epoch 93 out=49b] v91'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b5440 con 0x559f639dd000 2023-05-20T01:25:36.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.299+0000 7fe14440e700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:23 13.a 13:50000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f64524700 con 0x559f639dd000 2023-05-20T01:25:36.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.299+0000 7fe148c17700 1 -- 172.21.15.16:0/1013299390 <== osd.2 v2:172.21.15.16:6816/293187030 6 ==== osd_op_reply(23 [pgnls-filter start_epoch 93 out=49b] v91'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b5440 con 0x559f639dd000 2023-05-20T01:25:36.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.299+0000 7fe143c0d700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:24 13.2a 13:54000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f63b02700 con 0x559f639dc800 2023-05-20T01:25:36.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.299+0000 7fe148416700 1 -- 172.21.15.16:0/1013299390 <== osd.1 v2:172.21.15.16:6808/3860264220 4 ==== osd_op_reply(24 [pgnls-filter start_epoch 93 out=49b] v91'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b58c0 con 0x559f639dc800 2023-05-20T01:25:36.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.299+0000 7fe14440e700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:25 13.1a 13:58000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f63b02e00 con 0x559f639dc800 2023-05-20T01:25:36.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.299+0000 7fe148416700 1 -- 172.21.15.16:0/1013299390 <== osd.1 v2:172.21.15.16:6808/3860264220 5 ==== osd_op_reply(25 [pgnls-filter start_epoch 93 out=49b] v91'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b58c0 con 0x559f639dc800 2023-05-20T01:25:36.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.299+0000 7fe143c0d700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:26 13.3a 13:5c000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f63b02000 con 0x559f639d6800 2023-05-20T01:25:36.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.300+0000 7fe148c17700 1 -- 172.21.15.16:0/1013299390 <== osd.5 v2:172.21.15.132:6816/1844689995 4 ==== osd_op_reply(26 [pgnls-filter start_epoch 93 out=49b] v91'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b5440 con 0x559f639d6800 2023-05-20T01:25:36.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.300+0000 7fe14440e700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:27 13.6 13:60000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f63b03180 con 0x559f639dcc00 2023-05-20T01:25:36.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.300+0000 7fe149418700 1 -- 172.21.15.16:0/1013299390 <== osd.4 v2:172.21.15.132:6817/506317223 2 ==== osd_op_reply(27 [pgnls-filter start_epoch 93 out=49b] v91'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b5680 con 0x559f639dcc00 2023-05-20T01:25:36.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.300+0000 7fe143c0d700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:28 13.26 13:64000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f63b02a80 con 0x559f639d7000 2023-05-20T01:25:36.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.300+0000 7fe148c17700 1 -- 172.21.15.16:0/1013299390 <== osd.7 v2:172.21.15.132:6800/466001149 3 ==== osd_op_reply(28 [pgnls-filter start_epoch 93 out=49b] v91'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b5440 con 0x559f639d7000 2023-05-20T01:25:36.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.300+0000 7fe14440e700 1 --2- 172.21.15.16:0/1013299390 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x559f639dd800 0x559f63ad1b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:36.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.300+0000 7fe14440e700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:29 13.16 13:68000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f63b03500 con 0x559f639dd800 2023-05-20T01:25:36.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.300+0000 7fe149418700 1 --2- 172.21.15.16:0/1013299390 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x559f639dd800 0x559f63ad1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:36.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.301+0000 7fe149418700 1 --2- 172.21.15.16:0/1013299390 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x559f639dd800 0x559f63ad1b80 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:36.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.301+0000 7fe149418700 1 -- 172.21.15.16:0/1013299390 <== osd.3 v2:172.21.15.16:6800/3669781890 1 ==== osd_op_reply(29 [pgnls-filter start_epoch 93 out=49b] v91'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b5680 con 0x559f639dd800 2023-05-20T01:25:36.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.301+0000 7fe143c0d700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:30 13.36 13:6c000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f64525180 con 0x559f639d6c00 2023-05-20T01:25:36.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.301+0000 7fe149418700 1 -- 172.21.15.16:0/1013299390 <== osd.6 v2:172.21.15.132:6801/727581123 7 ==== osd_op_reply(30 [pgnls-filter start_epoch 93 out=49b] v91'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b5680 con 0x559f639d6c00 2023-05-20T01:25:36.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.301+0000 7fe14440e700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:31 13.e 13:70000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f63b03880 con 0x559f639d7000 2023-05-20T01:25:36.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.302+0000 7fe148c17700 1 -- 172.21.15.16:0/1013299390 <== osd.7 v2:172.21.15.132:6800/466001149 4 ==== osd_op_reply(31 [pgnls-filter start_epoch 93 out=49b] v91'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b5440 con 0x559f639d7000 2023-05-20T01:25:36.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.302+0000 7fe143c0d700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:32 13.2e 13:74000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f64525c00 con 0x559f639dd000 2023-05-20T01:25:36.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.302+0000 7fe148c17700 1 -- 172.21.15.16:0/1013299390 <== osd.2 v2:172.21.15.16:6816/293187030 7 ==== osd_op_reply(32 [pgnls-filter start_epoch 93 out=49b] v91'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b5440 con 0x559f639dd000 2023-05-20T01:25:36.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.302+0000 7fe14440e700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:33 13.1e 13:78000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f63b03c00 con 0x559f639dcc00 2023-05-20T01:25:36.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.302+0000 7fe149418700 1 -- 172.21.15.16:0/1013299390 <== osd.4 v2:172.21.15.132:6817/506317223 3 ==== osd_op_reply(33 [pgnls-filter start_epoch 93 out=49b] v91'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b5680 con 0x559f639dcc00 2023-05-20T01:25:36.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.302+0000 7fe143c0d700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:34 13.3e 13:7c000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f63b02380 con 0x559f639dc800 2023-05-20T01:25:36.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.303+0000 7fe148416700 1 -- 172.21.15.16:0/1013299390 <== osd.1 v2:172.21.15.16:6808/3860264220 6 ==== osd_op_reply(34 [pgnls-filter start_epoch 93 out=49b] v91'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b58c0 con 0x559f639dc800 2023-05-20T01:25:36.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.303+0000 7fe14440e700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:35 13.1 13:80000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f63b03180 con 0x559f639d6c00 2023-05-20T01:25:36.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.303+0000 7fe149418700 1 -- 172.21.15.16:0/1013299390 <== osd.6 v2:172.21.15.132:6801/727581123 8 ==== osd_op_reply(35 [pgnls-filter start_epoch 93 out=49b] v91'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b5680 con 0x559f639d6c00 2023-05-20T01:25:36.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.303+0000 7fe143c0d700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:36 13.21 13:84000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f64525880 con 0x559f639dcc00 2023-05-20T01:25:36.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.303+0000 7fe149418700 1 -- 172.21.15.16:0/1013299390 <== osd.4 v2:172.21.15.132:6817/506317223 4 ==== osd_op_reply(36 [pgnls-filter start_epoch 93 out=49b] v91'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b5680 con 0x559f639dcc00 2023-05-20T01:25:36.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.303+0000 7fe14440e700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:37 13.11 13:88000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f63b03500 con 0x559f639dc800 2023-05-20T01:25:36.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.304+0000 7fe148416700 1 -- 172.21.15.16:0/1013299390 <== osd.1 v2:172.21.15.16:6808/3860264220 7 ==== osd_op_reply(37 [pgnls-filter start_epoch 93 out=49b] v91'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b58c0 con 0x559f639dc800 2023-05-20T01:25:36.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.304+0000 7fe143c0d700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:38 13.31 13:8c000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f64524a80 con 0x559f639dd800 2023-05-20T01:25:36.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.304+0000 7fe149418700 1 -- 172.21.15.16:0/1013299390 <== osd.3 v2:172.21.15.16:6800/3669781890 2 ==== osd_op_reply(38 [pgnls-filter start_epoch 93 out=49b] v91'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b5680 con 0x559f639dd800 2023-05-20T01:25:36.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.304+0000 7fe14440e700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:39 13.9 13:90000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f63b02000 con 0x559f639d6800 2023-05-20T01:25:36.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.304+0000 7fe148c17700 1 -- 172.21.15.16:0/1013299390 <== osd.5 v2:172.21.15.132:6816/1844689995 5 ==== osd_op_reply(39 [pgnls-filter start_epoch 93 out=49b] v91'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b5440 con 0x559f639d6800 2023-05-20T01:25:36.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.304+0000 7fe143c0d700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:40 13.29 13:94000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f64525500 con 0x559f639d6c00 2023-05-20T01:25:36.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.304+0000 7fe149418700 1 -- 172.21.15.16:0/1013299390 <== osd.6 v2:172.21.15.132:6801/727581123 9 ==== osd_op_reply(40 [pgnls-filter start_epoch 93 out=49b] v91'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b5680 con 0x559f639d6c00 2023-05-20T01:25:36.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.304+0000 7fe14440e700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:41 13.19 13:98000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f63b03c00 con 0x559f639dd800 2023-05-20T01:25:36.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.305+0000 7fe149418700 1 -- 172.21.15.16:0/1013299390 <== osd.3 v2:172.21.15.16:6800/3669781890 3 ==== osd_op_reply(41 [pgnls-filter start_epoch 93 out=49b] v91'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b5680 con 0x559f639dd800 2023-05-20T01:25:36.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.305+0000 7fe143c0d700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:42 13.39 13:9c000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f64524e00 con 0x559f639dd000 2023-05-20T01:25:36.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.305+0000 7fe148c17700 1 -- 172.21.15.16:0/1013299390 <== osd.2 v2:172.21.15.16:6816/293187030 8 ==== osd_op_reply(42 [pgnls-filter start_epoch 93 out=49b] v91'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b5440 con 0x559f639dd000 2023-05-20T01:25:36.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.305+0000 7fe14440e700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:43 13.5 13:a0000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f63b03180 con 0x559f639dcc00 2023-05-20T01:25:36.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.305+0000 7fe149418700 1 -- 172.21.15.16:0/1013299390 <== osd.4 v2:172.21.15.132:6817/506317223 5 ==== osd_op_reply(43 [pgnls-filter start_epoch 93 out=49b] v91'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b5680 con 0x559f639dcc00 2023-05-20T01:25:36.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.305+0000 7fe143c0d700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:44 13.25 13:a4000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f64524700 con 0x559f639dcc00 2023-05-20T01:25:36.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.306+0000 7fe149418700 1 -- 172.21.15.16:0/1013299390 <== osd.4 v2:172.21.15.132:6817/506317223 6 ==== osd_op_reply(44 [pgnls-filter start_epoch 93 out=49b] v91'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b5680 con 0x559f639dcc00 2023-05-20T01:25:36.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.306+0000 7fe14440e700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:45 13.15 13:a8000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f63b02700 con 0x559f639dcc00 2023-05-20T01:25:36.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.306+0000 7fe149418700 1 -- 172.21.15.16:0/1013299390 <== osd.4 v2:172.21.15.132:6817/506317223 7 ==== osd_op_reply(45 [pgnls-filter start_epoch 93 out=49b] v91'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b5680 con 0x559f639dcc00 2023-05-20T01:25:36.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.306+0000 7fe143c0d700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:46 13.35 13:ac000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f63b02380 con 0x559f639dc800 2023-05-20T01:25:36.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.306+0000 7fe148416700 1 -- 172.21.15.16:0/1013299390 <== osd.1 v2:172.21.15.16:6808/3860264220 8 ==== osd_op_reply(46 [pgnls-filter start_epoch 93 out=49b] v91'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b58c0 con 0x559f639dc800 2023-05-20T01:25:36.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.306+0000 7fe14440e700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:47 13.d 13:b0000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f63b02e00 con 0x559f639dcc00 2023-05-20T01:25:36.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.306+0000 7fe149418700 1 -- 172.21.15.16:0/1013299390 <== osd.4 v2:172.21.15.132:6817/506317223 8 ==== osd_op_reply(47 [pgnls-filter start_epoch 93 out=49b] v91'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b5680 con 0x559f639dcc00 2023-05-20T01:25:36.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.306+0000 7fe143c0d700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:48 13.2d 13:b4000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f63b03500 con 0x559f639dd400 2023-05-20T01:25:36.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.307+0000 7fe148416700 1 -- 172.21.15.16:0/1013299390 <== osd.0 v2:172.21.15.16:6824/3923634179 3 ==== osd_op_reply(48 [pgnls-filter start_epoch 93 out=49b] v91'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b58c0 con 0x559f639dd400 2023-05-20T01:25:36.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.307+0000 7fe14440e700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:49 13.1d 13:b8000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f64525180 con 0x559f639d7000 2023-05-20T01:25:36.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.307+0000 7fe148c17700 1 -- 172.21.15.16:0/1013299390 <== osd.7 v2:172.21.15.132:6800/466001149 5 ==== osd_op_reply(49 [pgnls-filter start_epoch 93 out=49b] v91'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b5440 con 0x559f639d7000 2023-05-20T01:25:36.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.307+0000 7fe143c0d700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:50 13.3d 13:bc000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f63b02a80 con 0x559f639d7000 2023-05-20T01:25:36.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.307+0000 7fe148c17700 1 -- 172.21.15.16:0/1013299390 <== osd.7 v2:172.21.15.132:6800/466001149 6 ==== osd_op_reply(50 [pgnls-filter start_epoch 93 out=49b] v91'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b5440 con 0x559f639d7000 2023-05-20T01:25:36.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.307+0000 7fe14440e700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:51 13.3 13:c0000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f64525880 con 0x559f639dd400 2023-05-20T01:25:36.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.308+0000 7fe148416700 1 -- 172.21.15.16:0/1013299390 <== osd.0 v2:172.21.15.16:6824/3923634179 4 ==== osd_op_reply(51 [pgnls-filter start_epoch 93 out=49b] v91'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b58c0 con 0x559f639dd400 2023-05-20T01:25:36.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.308+0000 7fe143c0d700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:52 13.23 13:c4000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f63b03880 con 0x559f639dc800 2023-05-20T01:25:36.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.308+0000 7fe148416700 1 -- 172.21.15.16:0/1013299390 <== osd.1 v2:172.21.15.16:6808/3860264220 9 ==== osd_op_reply(52 [pgnls-filter start_epoch 93 out=49b] v91'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b58c0 con 0x559f639dc800 2023-05-20T01:25:36.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.308+0000 7fe14440e700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:53 13.13 13:c8000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f64524a80 con 0x559f639d7000 2023-05-20T01:25:36.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.308+0000 7fe148c17700 1 -- 172.21.15.16:0/1013299390 <== osd.7 v2:172.21.15.132:6800/466001149 7 ==== osd_op_reply(53 [pgnls-filter start_epoch 93 out=49b] v91'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b5440 con 0x559f639d7000 2023-05-20T01:25:36.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.308+0000 7fe143c0d700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:54 13.33 13:cc000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f63b02000 con 0x559f639d7000 2023-05-20T01:25:36.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.309+0000 7fe148c17700 1 -- 172.21.15.16:0/1013299390 <== osd.7 v2:172.21.15.132:6800/466001149 8 ==== osd_op_reply(54 [pgnls-filter start_epoch 93 out=49b] v91'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b5440 con 0x559f639d7000 2023-05-20T01:25:36.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.309+0000 7fe14440e700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:55 13.b 13:d0000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f63b03c00 con 0x559f639d6800 2023-05-20T01:25:36.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.309+0000 7fe148c17700 1 -- 172.21.15.16:0/1013299390 <== osd.5 v2:172.21.15.132:6816/1844689995 6 ==== osd_op_reply(55 [pgnls-filter start_epoch 93 out=49b] v91'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b5440 con 0x559f639d6800 2023-05-20T01:25:36.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.309+0000 7fe143c0d700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:56 13.2b 13:d4000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f64525c00 con 0x559f639dd000 2023-05-20T01:25:36.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.309+0000 7fe148c17700 1 -- 172.21.15.16:0/1013299390 <== osd.2 v2:172.21.15.16:6816/293187030 9 ==== osd_op_reply(56 [pgnls-filter start_epoch 93 out=49b] v91'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b5440 con 0x559f639dd000 2023-05-20T01:25:36.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.309+0000 7fe14440e700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:57 13.1b 13:d8000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f63b03180 con 0x559f639dcc00 2023-05-20T01:25:36.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.309+0000 7fe149418700 1 -- 172.21.15.16:0/1013299390 <== osd.4 v2:172.21.15.132:6817/506317223 9 ==== osd_op_reply(57 [pgnls-filter start_epoch 93 out=49b] v91'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b5680 con 0x559f639dcc00 2023-05-20T01:25:36.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.310+0000 7fe143c0d700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:58 13.3b 13:dc000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f64524e00 con 0x559f639dcc00 2023-05-20T01:25:36.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.310+0000 7fe149418700 1 -- 172.21.15.16:0/1013299390 <== osd.4 v2:172.21.15.132:6817/506317223 10 ==== osd_op_reply(58 [pgnls-filter start_epoch 93 out=49b] v91'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b5680 con 0x559f639dcc00 2023-05-20T01:25:36.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.310+0000 7fe14440e700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:59 13.7 13:e0000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f63b02700 con 0x559f639d7000 2023-05-20T01:25:36.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.310+0000 7fe148c17700 1 -- 172.21.15.16:0/1013299390 <== osd.7 v2:172.21.15.132:6800/466001149 9 ==== osd_op_reply(59 [pgnls-filter start_epoch 93 out=81b] v93'9 uv9 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x559f639b5440 con 0x559f639d7000 2023-05-20T01:25:36.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.310+0000 7fe14ac697c0 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:60 13.0 13:00000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f63b16000 con 0x559f639d7000 2023-05-20T01:25:36.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.311+0000 7fe148c17700 1 -- 172.21.15.16:0/1013299390 <== osd.7 v2:172.21.15.132:6800/466001149 10 ==== osd_op_reply(60 [pgnls-filter start_epoch 93 out=49b] v91'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b5440 con 0x559f639d7000 2023-05-20T01:25:36.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.311+0000 7fe14440e700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:61 13.20 13:04000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f63b02e00 con 0x559f639d6c00 2023-05-20T01:25:36.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.311+0000 7fe149418700 1 -- 172.21.15.16:0/1013299390 <== osd.6 v2:172.21.15.132:6801/727581123 10 ==== osd_op_reply(61 [pgnls-filter start_epoch 93 out=49b] v91'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b5680 con 0x559f639d6c00 2023-05-20T01:25:36.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.311+0000 7fe143c0d700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:62 13.10 13:08000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f64525180 con 0x559f639dc800 2023-05-20T01:25:36.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.311+0000 7fe148416700 1 -- 172.21.15.16:0/1013299390 <== osd.1 v2:172.21.15.16:6808/3860264220 10 ==== osd_op_reply(62 [pgnls-filter start_epoch 93 out=49b] v91'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b58c0 con 0x559f639dc800 2023-05-20T01:25:36.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.311+0000 7fe14440e700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:63 13.30 13:0c000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f63b02380 con 0x559f639dcc00 2023-05-20T01:25:36.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.311+0000 7fe149418700 1 -- 172.21.15.16:0/1013299390 <== osd.4 v2:172.21.15.132:6817/506317223 11 ==== osd_op_reply(63 [pgnls-filter start_epoch 93 out=49b] v91'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b5680 con 0x559f639dcc00 2023-05-20T01:25:36.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.311+0000 7fe143c0d700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:64 13.8 13:10000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f64525500 con 0x559f639dd000 2023-05-20T01:25:36.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.312+0000 7fe148c17700 1 -- 172.21.15.16:0/1013299390 <== osd.2 v2:172.21.15.16:6816/293187030 10 ==== osd_op_reply(64 [pgnls-filter start_epoch 93 out=49b] v91'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b5440 con 0x559f639dd000 2023-05-20T01:25:36.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.312+0000 7fe14440e700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:65 13.28 13:14000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f63b03500 con 0x559f639dd000 2023-05-20T01:25:36.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.312+0000 7fe148c17700 1 -- 172.21.15.16:0/1013299390 <== osd.2 v2:172.21.15.16:6816/293187030 11 ==== osd_op_reply(65 [pgnls-filter start_epoch 93 out=49b] v91'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b5440 con 0x559f639dd000 2023-05-20T01:25:36.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.312+0000 7fe143c0d700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:66 13.18 13:18000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f64524700 con 0x559f639d7000 2023-05-20T01:25:36.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.312+0000 7fe148c17700 1 -- 172.21.15.16:0/1013299390 <== osd.7 v2:172.21.15.132:6800/466001149 11 ==== osd_op_reply(66 [pgnls-filter start_epoch 93 out=49b] v91'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b5440 con 0x559f639d7000 2023-05-20T01:25:36.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.312+0000 7fe14440e700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:67 13.38 13:1c000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f63b03880 con 0x559f639d6800 2023-05-20T01:25:36.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.313+0000 7fe148c17700 1 -- 172.21.15.16:0/1013299390 <== osd.5 v2:172.21.15.132:6816/1844689995 7 ==== osd_op_reply(67 [pgnls-filter start_epoch 93 out=49b] v91'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b5440 con 0x559f639d6800 2023-05-20T01:25:36.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.313+0000 7fe143c0d700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:68 13.4 13:20000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f63b16000 con 0x559f639d6c00 2023-05-20T01:25:36.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.313+0000 7fe149418700 1 -- 172.21.15.16:0/1013299390 <== osd.6 v2:172.21.15.132:6801/727581123 11 ==== osd_op_reply(68 [pgnls-filter start_epoch 93 out=49b] v91'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b5680 con 0x559f639d6c00 2023-05-20T01:25:36.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.313+0000 7fe14440e700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:69 13.24 13:24000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f63b02a80 con 0x559f639d6c00 2023-05-20T01:25:36.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.313+0000 7fe149418700 1 -- 172.21.15.16:0/1013299390 <== osd.6 v2:172.21.15.132:6801/727581123 12 ==== osd_op_reply(69 [pgnls-filter start_epoch 93 out=49b] v91'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b5680 con 0x559f639d6c00 2023-05-20T01:25:36.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.313+0000 7fe143c0d700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:70 13.14 13:28000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f63b16e00 con 0x559f639dd400 2023-05-20T01:25:36.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.313+0000 7fe148416700 1 -- 172.21.15.16:0/1013299390 <== osd.0 v2:172.21.15.16:6824/3923634179 5 ==== osd_op_reply(70 [pgnls-filter start_epoch 93 out=49b] v91'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b58c0 con 0x559f639dd400 2023-05-20T01:25:36.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.313+0000 7fe14440e700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:71 13.34 13:2c000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f63b02000 con 0x559f639d6c00 2023-05-20T01:25:36.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.314+0000 7fe149418700 1 -- 172.21.15.16:0/1013299390 <== osd.6 v2:172.21.15.132:6801/727581123 13 ==== osd_op_reply(71 [pgnls-filter start_epoch 93 out=49b] v91'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b5680 con 0x559f639d6c00 2023-05-20T01:25:36.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.314+0000 7fe143c0d700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:72 13.c 13:30000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f63b17180 con 0x559f639dc800 2023-05-20T01:25:36.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.314+0000 7fe148416700 1 -- 172.21.15.16:0/1013299390 <== osd.1 v2:172.21.15.16:6808/3860264220 11 ==== osd_op_reply(72 [pgnls-filter start_epoch 93 out=49b] v91'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b58c0 con 0x559f639dc800 2023-05-20T01:25:36.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.314+0000 7fe14440e700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:73 13.2c 13:34000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f64525880 con 0x559f639dd000 2023-05-20T01:25:36.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.314+0000 7fe148c17700 1 -- 172.21.15.16:0/1013299390 <== osd.2 v2:172.21.15.16:6816/293187030 12 ==== osd_op_reply(73 [pgnls-filter start_epoch 93 out=49b] v91'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b5440 con 0x559f639dd000 2023-05-20T01:25:36.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.314+0000 7fe143c0d700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:74 13.1c 13:38000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f63b17500 con 0x559f639d6c00 2023-05-20T01:25:36.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.314+0000 7fe149418700 1 -- 172.21.15.16:0/1013299390 <== osd.6 v2:172.21.15.132:6801/727581123 14 ==== osd_op_reply(74 [pgnls-filter start_epoch 93 out=49b] v91'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b5680 con 0x559f639d6c00 2023-05-20T01:25:36.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.314+0000 7fe14440e700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:75 13.3c 13:3c000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f64525180 con 0x559f639dd000 2023-05-20T01:25:36.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.315+0000 7fe148c17700 1 -- 172.21.15.16:0/1013299390 <== osd.2 v2:172.21.15.16:6816/293187030 13 ==== osd_op_reply(75 [pgnls-filter start_epoch 93 out=49b] v91'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b5440 con 0x559f639dd000 2023-05-20T01:25:36.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.315+0000 7fe143c0d700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:76 13.2 13:40000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f63b17880 con 0x559f639d6800 2023-05-20T01:25:36.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.315+0000 7fe148c17700 1 -- 172.21.15.16:0/1013299390 <== osd.5 v2:172.21.15.132:6816/1844689995 8 ==== osd_op_reply(76 [pgnls-filter start_epoch 93 out=49b] v91'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b5440 con 0x559f639d6800 2023-05-20T01:25:36.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.315+0000 7fe14440e700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:77 13.22 13:44000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f64524a80 con 0x559f639dc800 2023-05-20T01:25:36.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.315+0000 7fe148416700 1 -- 172.21.15.16:0/1013299390 <== osd.1 v2:172.21.15.16:6808/3860264220 12 ==== osd_op_reply(77 [pgnls-filter start_epoch 93 out=49b] v91'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b58c0 con 0x559f639dc800 2023-05-20T01:25:36.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.315+0000 7fe143c0d700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:78 13.12 13:48000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f63b17c00 con 0x559f639dd400 2023-05-20T01:25:36.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.316+0000 7fe148416700 1 -- 172.21.15.16:0/1013299390 <== osd.0 v2:172.21.15.16:6824/3923634179 6 ==== osd_op_reply(78 [pgnls-filter start_epoch 93 out=49b] v91'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b58c0 con 0x559f639dd400 2023-05-20T01:25:36.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.316+0000 7fe14440e700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:79 13.32 13:4c000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f64525c00 con 0x559f639dd000 2023-05-20T01:25:36.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.316+0000 7fe148c17700 1 -- 172.21.15.16:0/1013299390 <== osd.2 v2:172.21.15.16:6816/293187030 14 ==== osd_op_reply(79 [pgnls-filter start_epoch 93 out=49b] v91'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b5440 con 0x559f639dd000 2023-05-20T01:25:36.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.316+0000 7fe143c0d700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:80 13.a 13:50000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f63b03c00 con 0x559f639dd000 2023-05-20T01:25:36.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.316+0000 7fe148c17700 1 -- 172.21.15.16:0/1013299390 <== osd.2 v2:172.21.15.16:6816/293187030 15 ==== osd_op_reply(80 [pgnls-filter start_epoch 93 out=49b] v91'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b5440 con 0x559f639dd000 2023-05-20T01:25:36.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.316+0000 7fe14440e700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:81 13.2a 13:54000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f63b03500 con 0x559f639dc800 2023-05-20T01:25:36.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.316+0000 7fe148416700 1 -- 172.21.15.16:0/1013299390 <== osd.1 v2:172.21.15.16:6808/3860264220 13 ==== osd_op_reply(81 [pgnls-filter start_epoch 93 out=49b] v91'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b58c0 con 0x559f639dc800 2023-05-20T01:25:36.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.316+0000 7fe143c0d700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:82 13.1a 13:58000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f63b02700 con 0x559f639dc800 2023-05-20T01:25:36.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.317+0000 7fe148416700 1 -- 172.21.15.16:0/1013299390 <== osd.1 v2:172.21.15.16:6808/3860264220 14 ==== osd_op_reply(82 [pgnls-filter start_epoch 93 out=49b] v91'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b58c0 con 0x559f639dc800 2023-05-20T01:25:36.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.317+0000 7fe14440e700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:83 13.3a 13:5c000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f63b03880 con 0x559f639d6800 2023-05-20T01:25:36.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.317+0000 7fe148c17700 1 -- 172.21.15.16:0/1013299390 <== osd.5 v2:172.21.15.132:6816/1844689995 9 ==== osd_op_reply(83 [pgnls-filter start_epoch 93 out=49b] v91'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b5440 con 0x559f639d6800 2023-05-20T01:25:36.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.317+0000 7fe143c0d700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:84 13.6 13:60000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f63b03500 con 0x559f639dcc00 2023-05-20T01:25:36.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.317+0000 7fe149418700 1 -- 172.21.15.16:0/1013299390 <== osd.4 v2:172.21.15.132:6817/506317223 12 ==== osd_op_reply(84 [pgnls-filter start_epoch 93 out=49b] v91'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b5680 con 0x559f639dcc00 2023-05-20T01:25:36.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.317+0000 7fe14440e700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:85 13.26 13:64000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f63b17880 con 0x559f639d7000 2023-05-20T01:25:36.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.318+0000 7fe148c17700 1 -- 172.21.15.16:0/1013299390 <== osd.7 v2:172.21.15.132:6800/466001149 12 ==== osd_op_reply(85 [pgnls-filter start_epoch 93 out=49b] v91'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b5440 con 0x559f639d7000 2023-05-20T01:25:36.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.318+0000 7fe143c0d700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:86 13.16 13:68000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f63b02700 con 0x559f639dd800 2023-05-20T01:25:36.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.318+0000 7fe149418700 1 -- 172.21.15.16:0/1013299390 <== osd.3 v2:172.21.15.16:6800/3669781890 4 ==== osd_op_reply(86 [pgnls-filter start_epoch 93 out=49b] v91'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b5680 con 0x559f639dd800 2023-05-20T01:25:36.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.318+0000 7fe14440e700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:87 13.36 13:6c000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f63b16000 con 0x559f639d6c00 2023-05-20T01:25:36.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.318+0000 7fe149418700 1 -- 172.21.15.16:0/1013299390 <== osd.6 v2:172.21.15.132:6801/727581123 15 ==== osd_op_reply(87 [pgnls-filter start_epoch 93 out=49b] v91'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b5680 con 0x559f639d6c00 2023-05-20T01:25:36.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.318+0000 7fe143c0d700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:88 13.e 13:70000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f63b03180 con 0x559f639d7000 2023-05-20T01:25:36.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.319+0000 7fe148c17700 1 -- 172.21.15.16:0/1013299390 <== osd.7 v2:172.21.15.132:6800/466001149 13 ==== osd_op_reply(88 [pgnls-filter start_epoch 93 out=49b] v91'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b5440 con 0x559f639d7000 2023-05-20T01:25:36.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.319+0000 7fe14440e700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:89 13.2e 13:74000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f64525500 con 0x559f639dd000 2023-05-20T01:25:36.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.319+0000 7fe148c17700 1 -- 172.21.15.16:0/1013299390 <== osd.2 v2:172.21.15.16:6816/293187030 16 ==== osd_op_reply(89 [pgnls-filter start_epoch 93 out=49b] v91'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b5440 con 0x559f639dd000 2023-05-20T01:25:36.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.319+0000 7fe143c0d700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:90 13.1e 13:78000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f63b02e00 con 0x559f639dcc00 2023-05-20T01:25:36.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.319+0000 7fe149418700 1 -- 172.21.15.16:0/1013299390 <== osd.4 v2:172.21.15.132:6817/506317223 13 ==== osd_op_reply(90 [pgnls-filter start_epoch 93 out=49b] v91'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b5680 con 0x559f639dcc00 2023-05-20T01:25:36.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.319+0000 7fe14440e700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:91 13.3e 13:7c000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f64524700 con 0x559f639dc800 2023-05-20T01:25:36.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.319+0000 7fe148416700 1 -- 172.21.15.16:0/1013299390 <== osd.1 v2:172.21.15.16:6808/3860264220 15 ==== osd_op_reply(91 [pgnls-filter start_epoch 93 out=49b] v91'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b58c0 con 0x559f639dc800 2023-05-20T01:25:36.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.319+0000 7fe143c0d700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:92 13.1 13:80000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f63b02380 con 0x559f639d6c00 2023-05-20T01:25:36.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.320+0000 7fe149418700 1 -- 172.21.15.16:0/1013299390 <== osd.6 v2:172.21.15.132:6801/727581123 16 ==== osd_op_reply(92 [pgnls-filter start_epoch 93 out=49b] v91'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b5680 con 0x559f639d6c00 2023-05-20T01:25:36.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.320+0000 7fe14440e700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:93 13.21 13:84000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f64525880 con 0x559f639dcc00 2023-05-20T01:25:36.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.320+0000 7fe149418700 1 -- 172.21.15.16:0/1013299390 <== osd.4 v2:172.21.15.132:6817/506317223 14 ==== osd_op_reply(93 [pgnls-filter start_epoch 93 out=49b] v91'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b5680 con 0x559f639dcc00 2023-05-20T01:25:36.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.320+0000 7fe143c0d700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:94 13.11 13:88000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f63b02a80 con 0x559f639dc800 2023-05-20T01:25:36.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.320+0000 7fe148416700 1 -- 172.21.15.16:0/1013299390 <== osd.1 v2:172.21.15.16:6808/3860264220 16 ==== osd_op_reply(94 [pgnls-filter start_epoch 93 out=49b] v91'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b58c0 con 0x559f639dc800 2023-05-20T01:25:36.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.321+0000 7fe14440e700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:95 13.31 13:8c000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f64525180 con 0x559f639dd800 2023-05-20T01:25:36.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.321+0000 7fe149418700 1 -- 172.21.15.16:0/1013299390 <== osd.3 v2:172.21.15.16:6800/3669781890 5 ==== osd_op_reply(95 [pgnls-filter start_epoch 93 out=49b] v91'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b5680 con 0x559f639dd800 2023-05-20T01:25:36.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.321+0000 7fe143c0d700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:96 13.9 13:90000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f63b16e00 con 0x559f639d6800 2023-05-20T01:25:36.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.321+0000 7fe148c17700 1 -- 172.21.15.16:0/1013299390 <== osd.5 v2:172.21.15.132:6816/1844689995 10 ==== osd_op_reply(96 [pgnls-filter start_epoch 93 out=49b] v91'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b5440 con 0x559f639d6800 2023-05-20T01:25:36.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.321+0000 7fe14440e700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:97 13.29 13:94000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f64525c00 con 0x559f639d6c00 2023-05-20T01:25:36.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.321+0000 7fe149418700 1 -- 172.21.15.16:0/1013299390 <== osd.6 v2:172.21.15.132:6801/727581123 17 ==== osd_op_reply(97 [pgnls-filter start_epoch 93 out=49b] v91'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b5680 con 0x559f639d6c00 2023-05-20T01:25:36.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.321+0000 7fe143c0d700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:98 13.19 13:98000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f63b17180 con 0x559f639dd800 2023-05-20T01:25:36.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.322+0000 7fe149418700 1 -- 172.21.15.16:0/1013299390 <== osd.3 v2:172.21.15.16:6800/3669781890 6 ==== osd_op_reply(98 [pgnls-filter start_epoch 93 out=49b] v91'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b5680 con 0x559f639dd800 2023-05-20T01:25:36.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.322+0000 7fe14440e700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:99 13.39 13:9c000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f64524e00 con 0x559f639dd000 2023-05-20T01:25:36.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.322+0000 7fe148c17700 1 -- 172.21.15.16:0/1013299390 <== osd.2 v2:172.21.15.16:6816/293187030 17 ==== osd_op_reply(99 [pgnls-filter start_epoch 93 out=49b] v91'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b5440 con 0x559f639dd000 2023-05-20T01:25:36.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.322+0000 7fe143c0d700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:100 13.5 13:a0000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f63b17c00 con 0x559f639dcc00 2023-05-20T01:25:36.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.322+0000 7fe149418700 1 -- 172.21.15.16:0/1013299390 <== osd.4 v2:172.21.15.132:6817/506317223 15 ==== osd_op_reply(100 [pgnls-filter start_epoch 93 out=49b] v91'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b5680 con 0x559f639dcc00 2023-05-20T01:25:36.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.322+0000 7fe14440e700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:101 13.25 13:a4000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f63b03500 con 0x559f639dcc00 2023-05-20T01:25:36.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.323+0000 7fe149418700 1 -- 172.21.15.16:0/1013299390 <== osd.4 v2:172.21.15.132:6817/506317223 16 ==== osd_op_reply(101 [pgnls-filter start_epoch 93 out=49b] v91'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b5680 con 0x559f639dcc00 2023-05-20T01:25:36.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.323+0000 7fe143c0d700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:102 13.15 13:a8000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f64524a80 con 0x559f639dcc00 2023-05-20T01:25:36.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.323+0000 7fe149418700 1 -- 172.21.15.16:0/1013299390 <== osd.4 v2:172.21.15.132:6817/506317223 17 ==== osd_op_reply(102 [pgnls-filter start_epoch 93 out=49b] v91'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b5680 con 0x559f639dcc00 2023-05-20T01:25:36.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.323+0000 7fe14440e700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:103 13.35 13:ac000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f63b02700 con 0x559f639dc800 2023-05-20T01:25:36.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.323+0000 7fe148416700 1 -- 172.21.15.16:0/1013299390 <== osd.1 v2:172.21.15.16:6808/3860264220 17 ==== osd_op_reply(103 [pgnls-filter start_epoch 93 out=49b] v91'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b58c0 con 0x559f639dc800 2023-05-20T01:25:36.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.323+0000 7fe143c0d700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:104 13.d 13:b0000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f63b22000 con 0x559f639dcc00 2023-05-20T01:25:36.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.324+0000 7fe149418700 1 -- 172.21.15.16:0/1013299390 <== osd.4 v2:172.21.15.132:6817/506317223 18 ==== osd_op_reply(104 [pgnls-filter start_epoch 93 out=49b] v91'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b5680 con 0x559f639dcc00 2023-05-20T01:25:36.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.324+0000 7fe14440e700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:105 13.2d 13:b4000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f63b02e00 con 0x559f639dd400 2023-05-20T01:25:36.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.324+0000 7fe148416700 1 -- 172.21.15.16:0/1013299390 <== osd.0 v2:172.21.15.16:6824/3923634179 7 ==== osd_op_reply(105 [pgnls-filter start_epoch 93 out=49b] v91'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b58c0 con 0x559f639dd400 2023-05-20T01:25:36.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.324+0000 7fe143c0d700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:106 13.1d 13:b8000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f63b22380 con 0x559f639d7000 2023-05-20T01:25:36.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.324+0000 7fe148c17700 1 -- 172.21.15.16:0/1013299390 <== osd.7 v2:172.21.15.132:6800/466001149 14 ==== osd_op_reply(106 [pgnls-filter start_epoch 93 out=49b] v91'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b5440 con 0x559f639d7000 2023-05-20T01:25:36.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.324+0000 7fe14440e700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:107 13.3d 13:bc000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f63b02380 con 0x559f639d7000 2023-05-20T01:25:36.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.324+0000 7fe148c17700 1 -- 172.21.15.16:0/1013299390 <== osd.7 v2:172.21.15.132:6800/466001149 15 ==== osd_op_reply(107 [pgnls-filter start_epoch 93 out=49b] v91'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b5440 con 0x559f639d7000 2023-05-20T01:25:36.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.324+0000 7fe143c0d700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:108 13.3 13:c0000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f63b22700 con 0x559f639dd400 2023-05-20T01:25:36.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.325+0000 7fe148416700 1 -- 172.21.15.16:0/1013299390 <== osd.0 v2:172.21.15.16:6824/3923634179 8 ==== osd_op_reply(108 [pgnls-filter start_epoch 93 out=49b] v91'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b58c0 con 0x559f639dd400 2023-05-20T01:25:36.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.325+0000 7fe14440e700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:109 13.23 13:c4000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f63b02000 con 0x559f639dc800 2023-05-20T01:25:36.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.325+0000 7fe148416700 1 -- 172.21.15.16:0/1013299390 <== osd.1 v2:172.21.15.16:6808/3860264220 18 ==== osd_op_reply(109 [pgnls-filter start_epoch 93 out=49b] v91'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b58c0 con 0x559f639dc800 2023-05-20T01:25:36.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.325+0000 7fe143c0d700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:110 13.13 13:c8000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f63b22a80 con 0x559f639d7000 2023-05-20T01:25:36.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.325+0000 7fe148c17700 1 -- 172.21.15.16:0/1013299390 <== osd.7 v2:172.21.15.132:6800/466001149 16 ==== osd_op_reply(110 [pgnls-filter start_epoch 93 out=49b] v91'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b5440 con 0x559f639d7000 2023-05-20T01:25:36.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.325+0000 7fe14440e700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:111 13.33 13:cc000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f64525880 con 0x559f639d7000 2023-05-20T01:25:36.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.326+0000 7fe148c17700 1 -- 172.21.15.16:0/1013299390 <== osd.7 v2:172.21.15.132:6800/466001149 17 ==== osd_op_reply(111 [pgnls-filter start_epoch 93 out=49b] v91'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b5440 con 0x559f639d7000 2023-05-20T01:25:36.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.326+0000 7fe143c0d700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:112 13.b 13:d0000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f63b22e00 con 0x559f639d6800 2023-05-20T01:25:36.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.326+0000 7fe148c17700 1 -- 172.21.15.16:0/1013299390 <== osd.5 v2:172.21.15.132:6816/1844689995 11 ==== osd_op_reply(112 [pgnls-filter start_epoch 93 out=49b] v91'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b5440 con 0x559f639d6800 2023-05-20T01:25:36.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.326+0000 7fe14440e700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:113 13.2b 13:d4000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f64525180 con 0x559f639dd000 2023-05-20T01:25:36.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.326+0000 7fe148c17700 1 -- 172.21.15.16:0/1013299390 <== osd.2 v2:172.21.15.16:6816/293187030 18 ==== osd_op_reply(113 [pgnls-filter start_epoch 93 out=49b] v91'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b5440 con 0x559f639dd000 2023-05-20T01:25:36.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.326+0000 7fe143c0d700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:114 13.1b 13:d8000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f63b23180 con 0x559f639dcc00 2023-05-20T01:25:36.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.327+0000 7fe149418700 1 -- 172.21.15.16:0/1013299390 <== osd.4 v2:172.21.15.132:6817/506317223 19 ==== osd_op_reply(114 [pgnls-filter start_epoch 93 out=49b] v91'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b5680 con 0x559f639dcc00 2023-05-20T01:25:36.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.327+0000 7fe14440e700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:115 13.3b 13:dc000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f63b17500 con 0x559f639dcc00 2023-05-20T01:25:36.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.327+0000 7fe149418700 1 -- 172.21.15.16:0/1013299390 <== osd.4 v2:172.21.15.132:6817/506317223 20 ==== osd_op_reply(115 [pgnls-filter start_epoch 93 out=49b] v91'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b5680 con 0x559f639dcc00 2023-05-20T01:25:36.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.327+0000 7fe143c0d700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:116 13.7 13:e0000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f63b23500 con 0x559f639d7000 2023-05-20T01:25:36.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.327+0000 7fe148c17700 1 -- 172.21.15.16:0/1013299390 <== osd.7 v2:172.21.15.132:6800/466001149 18 ==== osd_op_reply(116 [pgnls-filter start_epoch 93 out=81b] v93'9 uv9 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x559f639b5440 con 0x559f639d7000 2023-05-20T01:25:36.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.327+0000 7fe14ac697c0 20 datascan.forall_objects: OSD matched oid 10000000001.00000000 2023-05-20T01:25:36.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.327+0000 7fe14ac697c0 10 datascan.operator(): handling object 10000000001.0 2023-05-20T01:25:36.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.327+0000 7fe14ac697c0 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:117 13.7 13:e2e07930:::10000000001.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f63b16380 con 0x559f639d7000 2023-05-20T01:25:36.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.328+0000 7fe148c17700 1 -- 172.21.15.16:0/1013299390 <== osd.7 v2:172.21.15.132:6800/466001149 19 ==== osd_op_reply(117 10000000001.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (103) out=103b,getxattr,getxattr] v0'0 uv9 ondisk = 0) v8 ==== 416+0+149 (crc 0 0 0) 0x559f639b5440 con 0x559f639d7000 2023-05-20T01:25:36.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.328+0000 7fe14ac697c0 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:118 14.7 14:e2e07930:::10000000001.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f63b16a80 con 0x559f639d6800 2023-05-20T01:25:36.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.328+0000 7fe148c17700 1 -- 172.21.15.16:0/1013299390 <== osd.5 v2:172.21.15.132:6816/1844689995 12 ==== osd_op_reply(118 10000000001.00000000 [getxattr (30) out=30b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x559f639b5440 con 0x559f639d6800 2023-05-20T01:25:36.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.328+0000 7fe14ac697c0 20 datascan.operator(): loaded layout from xattr: pi: 14 os: 4194304 sc: 1 su: 4194304 2023-05-20T01:25:36.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.328+0000 7fe14ac697c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000001 2023-05-20T01:25:36.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.328+0000 7fe14ac697c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/sixmegs 2023-05-20T01:25:36.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.328+0000 7fe14ac697c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=sixmegs 2023-05-20T01:25:36.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.328+0000 7fe14ac697c0 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:119 12.10 12:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f63b22380 con 0x559f639dd400 2023-05-20T01:25:36.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.328+0000 7fe148416700 1 -- 172.21.15.16:0/1013299390 <== osd.0 v2:172.21.15.16:6824/3923634179 9 ==== osd_op_reply(119 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 164+0+0 (crc 0 0 0) 0x559f639b58c0 con 0x559f639dd400 2023-05-20T01:25:36.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.328+0000 7fe14ac697c0 4 datascan.get_frag_of: Unexpected error on '10000000000.00000000': (2) No such file or directory 2023-05-20T01:25:36.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.328+0000 7fe14ac697c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/sixmegs, will insert to root 2023-05-20T01:25:36.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.328+0000 7fe14ac697c0 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:120 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f64525500 con 0x559f639dd400 2023-05-20T01:25:36.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.328+0000 7fe148416700 1 -- 172.21.15.16:0/1013299390 <== osd.0 v2:172.21.15.16:6824/3923634179 10 ==== osd_op_reply(120 10000000000.00000000 [omap-get-header] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 164+0+0 (crc 0 0 0) 0x559f639b58c0 con 0x559f639dd400 2023-05-20T01:25:36.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.328+0000 7fe14ac697c0 10 datascan.find_or_create_dirfrag: read_version = 0 2023-05-20T01:25:36.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.328+0000 7fe14ac697c0 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:121 12.10 12:0cf6ad0f:::10000000000.00000000:head [create,omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f64524e00 con 0x559f639dd400 2023-05-20T01:25:36.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.330+0000 7fe148416700 1 -- 172.21.15.16:0/1013299390 <== osd.0 v2:172.21.15.16:6824/3923634179 11 ==== osd_op_reply(121 10000000000.00000000 [create,omap-set-header] v93'16 uv16 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x559f639b58c0 con 0x559f639dd400 2023-05-20T01:25:36.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.330+0000 7fe14ac697c0 10 datascan.find_or_create_dirfrag: Created dirfrag: 0x0x10000000000 2023-05-20T01:25:36.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.330+0000 7fe14ac697c0 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:122 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f63b17880 con 0x559f639dd400 2023-05-20T01:25:36.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.330+0000 7fe148416700 1 -- 172.21.15.16:0/1013299390 <== osd.0 v2:172.21.15.16:6824/3923634179 12 ==== osd_op_reply(122 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv16 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x559f639b58c0 con 0x559f639dd400 2023-05-20T01:25:36.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.330+0000 7fe14ac697c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=sixmegs frag=*, r=0 2023-05-20T01:25:36.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.330+0000 7fe14ac697c0 20 datascan.read_dentry: sixmegs_head not found in result 2023-05-20T01:25:36.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.330+0000 7fe14ac697c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000001 at 0x0x10000000000/sixmegs with size=6291456 bytes 2023-05-20T01:25:36.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.330+0000 7fe14ac697c0 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:123 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=508b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f63b16e00 con 0x559f639dd400 2023-05-20T01:25:36.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.331+0000 7fe148416700 1 -- 172.21.15.16:0/1013299390 <== osd.0 v2:172.21.15.16:6824/3923634179 13 ==== osd_op_reply(123 10000000000.00000000 [omap-set-vals] v93'17 uv17 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x559f639b58c0 con 0x559f639dd400 2023-05-20T01:25:36.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.331+0000 7fe14ac697c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/sixmegs pointing to 0x0x10000000001 2023-05-20T01:25:36.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.331+0000 7fe14ac697c0 10 datascan.inject_with_backtrace: backptr: 0x0x1/subdir 2023-05-20T01:25:36.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.331+0000 7fe14ac697c0 20 datascan.get_frag_of: dirino=0x1 target_dname=subdir 2023-05-20T01:25:36.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.331+0000 7fe14ac697c0 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:124 12.1f 12:ff5b34d6:::1.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f63b02a80 con 0x559f639d6c00 2023-05-20T01:25:36.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.331+0000 7fe149418700 1 -- 172.21.15.16:0/1013299390 <== osd.6 v2:172.21.15.132:6801/727581123 18 ==== osd_op_reply(124 1.00000000 [getxattr] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x559f639b5680 con 0x559f639d6c00 2023-05-20T01:25:36.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.331+0000 7fe14ac697c0 4 datascan.get_frag_of: Unexpected error on '1.00000000': (2) No such file or directory 2023-05-20T01:25:36.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.331+0000 7fe14ac697c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x1/subdir, will insert to root 2023-05-20T01:25:36.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.331+0000 7fe14ac697c0 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:125 12.1f 12:ff5b34d6:::1.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f63b02700 con 0x559f639d6c00 2023-05-20T01:25:36.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.332+0000 7fe149418700 1 -- 172.21.15.16:0/1013299390 <== osd.6 v2:172.21.15.132:6801/727581123 19 ==== osd_op_reply(125 1.00000000 [omap-get-header] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x559f639b5680 con 0x559f639d6c00 2023-05-20T01:25:36.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.332+0000 7fe14ac697c0 10 datascan.find_or_create_dirfrag: read_version = 0 2023-05-20T01:25:36.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.332+0000 7fe14ac697c0 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:126 12.1f 12:ff5b34d6:::1.00000000:head [create,omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f63b02e00 con 0x559f639d6c00 2023-05-20T01:25:36.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.333+0000 7fe149418700 1 -- 172.21.15.16:0/1013299390 <== osd.6 v2:172.21.15.132:6801/727581123 20 ==== osd_op_reply(126 1.00000000 [create,omap-set-header] v93'11 uv11 ondisk = 0) v8 ==== 196+0+0 (crc 0 0 0) 0x559f639b5680 con 0x559f639d6c00 2023-05-20T01:25:36.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.333+0000 7fe14ac697c0 10 datascan.find_or_create_dirfrag: Created dirfrag: 0x0x1 2023-05-20T01:25:36.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.333+0000 7fe14ac697c0 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:127 12.1f 12:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f63b22700 con 0x559f639d6c00 2023-05-20T01:25:36.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.333+0000 7fe149418700 1 -- 172.21.15.16:0/1013299390 <== osd.6 v2:172.21.15.132:6801/727581123 21 ==== osd_op_reply(127 1.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv11 ondisk = 0) v8 ==== 154+0+4 (crc 0 0 0) 0x559f639b5680 con 0x559f639d6c00 2023-05-20T01:25:36.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.333+0000 7fe14ac697c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-05-20T01:25:36.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.333+0000 7fe14ac697c0 20 datascan.read_dentry: subdir_head not found in result 2023-05-20T01:25:36.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.333+0000 7fe14ac697c0 10 datascan.inject_with_backtrace: Linking ancestor directory of inode 0x0x10000000000 at 0x0x1:subdir 2023-05-20T01:25:36.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.333+0000 7fe14ac697c0 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:128 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-vals in=507b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f63b22380 con 0x559f639d6c00 2023-05-20T01:25:36.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.334+0000 7fe149418700 1 -- 172.21.15.16:0/1013299390 <== osd.6 v2:172.21.15.132:6801/727581123 22 ==== osd_op_reply(128 1.00000000 [omap-set-vals] v93'12 uv12 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x559f639b5680 con 0x559f639d6c00 2023-05-20T01:25:36.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.334+0000 7fe14ac697c0 20 datascan.inject_linkage: Injected dentry 0x0x1/subdir pointing to 0x0x10000000000 2023-05-20T01:25:36.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.334+0000 7fe14ac697c0 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:129 13.27 13:e4000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f64524700 con 0x559f639dc800 2023-05-20T01:25:36.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.334+0000 7fe148416700 1 -- 172.21.15.16:0/1013299390 <== osd.1 v2:172.21.15.16:6808/3860264220 19 ==== osd_op_reply(129 [pgnls-filter start_epoch 93 out=49b] v91'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b58c0 con 0x559f639dc800 2023-05-20T01:25:36.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.334+0000 7fe143c0d700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:130 13.17 13:e8000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f63b23880 con 0x559f639d6800 2023-05-20T01:25:36.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.334+0000 7fe148c17700 1 -- 172.21.15.16:0/1013299390 <== osd.5 v2:172.21.15.132:6816/1844689995 13 ==== osd_op_reply(130 [pgnls-filter start_epoch 93 out=49b] v91'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b5440 con 0x559f639d6800 2023-05-20T01:25:36.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.335+0000 7fe14440e700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:131 13.37 13:ec000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f63b16000 con 0x559f639d6800 2023-05-20T01:25:36.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.335+0000 7fe148c17700 1 -- 172.21.15.16:0/1013299390 <== osd.5 v2:172.21.15.132:6816/1844689995 14 ==== osd_op_reply(131 [pgnls-filter start_epoch 93 out=49b] v91'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b5440 con 0x559f639d6800 2023-05-20T01:25:36.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.335+0000 7fe143c0d700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:132 13.f 13:f0000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f63b23c00 con 0x559f639d6c00 2023-05-20T01:25:36.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.335+0000 7fe149418700 1 -- 172.21.15.16:0/1013299390 <== osd.6 v2:172.21.15.132:6801/727581123 23 ==== osd_op_reply(132 [pgnls-filter start_epoch 93 out=49b] v91'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b5680 con 0x559f639d6c00 2023-05-20T01:25:36.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.335+0000 7fe14440e700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:133 13.2f 13:f4000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f63b16700 con 0x559f639dcc00 2023-05-20T01:25:36.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.335+0000 7fe149418700 1 -- 172.21.15.16:0/1013299390 <== osd.4 v2:172.21.15.132:6817/506317223 21 ==== osd_op_reply(133 [pgnls-filter start_epoch 93 out=49b] v91'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b5680 con 0x559f639dcc00 2023-05-20T01:25:36.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.335+0000 7fe143c0d700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:134 13.1f 13:f8000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f63b03500 con 0x559f639d6c00 2023-05-20T01:25:36.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.336+0000 7fe149418700 1 -- 172.21.15.16:0/1013299390 <== osd.6 v2:172.21.15.132:6801/727581123 24 ==== osd_op_reply(134 [pgnls-filter start_epoch 93 out=49b] v91'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b5680 con 0x559f639d6c00 2023-05-20T01:25:36.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.336+0000 7fe14440e700 1 -- 172.21.15.16:0/1013299390 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:135 13.3f 13:fc000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x559f63b16380 con 0x559f639d6c00 2023-05-20T01:25:36.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.336+0000 7fe149418700 1 -- 172.21.15.16:0/1013299390 <== osd.6 v2:172.21.15.132:6801/727581123 25 ==== osd_op_reply(135 [pgnls-filter start_epoch 93 out=49b] v91'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f639b5680 con 0x559f639d6c00 2023-05-20T01:25:36.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.336+0000 7fe14ac697c0 1 -- 172.21.15.16:0/1013299390 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x559f639dd400 msgr2=0x559f63ad1600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:36.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.336+0000 7fe14ac697c0 1 --2- 172.21.15.16:0/1013299390 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x559f639dd400 0x559f63ad1600 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:36.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.336+0000 7fe14ac697c0 1 -- 172.21.15.16:0/1013299390 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x559f639dc800 msgr2=0x559f63ad0580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:36.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.336+0000 7fe14ac697c0 1 --2- 172.21.15.16:0/1013299390 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x559f639dc800 0x559f63ad0580 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:36.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.336+0000 7fe14ac697c0 1 -- 172.21.15.16:0/1013299390 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x559f639dd000 msgr2=0x559f63ad1080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:36.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.336+0000 7fe14ac697c0 1 --2- 172.21.15.16:0/1013299390 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x559f639dd000 0x559f63ad1080 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:36.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.336+0000 7fe14ac697c0 1 -- 172.21.15.16:0/1013299390 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x559f639dd800 msgr2=0x559f63ad1b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:36.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.336+0000 7fe14ac697c0 1 --2- 172.21.15.16:0/1013299390 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x559f639dd800 0x559f63ad1b80 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:36.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.336+0000 7fe14ac697c0 1 -- 172.21.15.16:0/1013299390 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x559f639dcc00 msgr2=0x559f63ad0b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:36.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.336+0000 7fe14ac697c0 1 --2- 172.21.15.16:0/1013299390 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x559f639dcc00 0x559f63ad0b00 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:36.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.336+0000 7fe14ac697c0 1 -- 172.21.15.16:0/1013299390 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x559f639d6800 msgr2=0x559f6373c680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:36.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.336+0000 7fe14ac697c0 1 --2- 172.21.15.16:0/1013299390 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x559f639d6800 0x559f6373c680 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:36.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.336+0000 7fe14ac697c0 1 -- 172.21.15.16:0/1013299390 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x559f639d6c00 msgr2=0x559f6373cc00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:36.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.336+0000 7fe14ac697c0 1 --2- 172.21.15.16:0/1013299390 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x559f639d6c00 0x559f6373cc00 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:36.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.336+0000 7fe14ac697c0 1 -- 172.21.15.16:0/1013299390 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x559f639d7000 msgr2=0x559f6373d180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:36.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.336+0000 7fe14ac697c0 1 --2- 172.21.15.16:0/1013299390 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x559f639d7000 0x559f6373d180 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:36.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.336+0000 7fe148416700 1 -- 172.21.15.16:0/1013299390 reap_dead start 2023-05-20T01:25:36.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.336+0000 7fe14ac697c0 1 -- 172.21.15.16:0/1013299390 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x559f639dc400 msgr2=0x559f63ad0000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:36.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.336+0000 7fe14ac697c0 1 --2- 172.21.15.16:0/1013299390 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x559f639dc400 0x559f63ad0000 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x559f639c00c0 tx=0x559f64474000 comp rx=0 tx=0).stop 2023-05-20T01:25:36.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.336+0000 7fe14ac697c0 1 -- 172.21.15.16:0/1013299390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f639d6400 msgr2=0x559f6373c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:36.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.336+0000 7fe14ac697c0 1 --2- 172.21.15.16:0/1013299390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f639d6400 0x559f6373c100 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x559f639c1410 tx=0x559f644750e0 comp rx=0 tx=0).stop 2023-05-20T01:25:36.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.336+0000 7fe14ac697c0 1 -- 172.21.15.16:0/1013299390 shutdown_connections 2023-05-20T01:25:36.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.336+0000 7fe14ac697c0 1 --2- 172.21.15.16:0/1013299390 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x559f639dc400 0x559f63ad0000 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:36.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.337+0000 7fe14ac697c0 1 --2- 172.21.15.16:0/1013299390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f639d6400 0x559f6373c100 secure :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x559f639c1410 tx=0x559f644750e0 comp rx=0 tx=0).stop 2023-05-20T01:25:36.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.337+0000 7fe14ac697c0 1 -- 172.21.15.16:0/1013299390 >> 172.21.15.16:0/1013299390 conn(0x559f636a1c00 msgr2=0x559f63771800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:36.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.337+0000 7fe14ac697c0 1 -- 172.21.15.16:0/1013299390 shutdown_connections 2023-05-20T01:25:36.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.337+0000 7fe14ac697c0 1 -- 172.21.15.16:0/1013299390 wait complete. 2023-05-20T01:25:36.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.337+0000 7fe14ac697c0 1 -- 172.21.15.16:0/1036135947 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f636a1000 msgr2=0x559f6373bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:36.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.337+0000 7fe14ac697c0 1 --2- 172.21.15.16:0/1036135947 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f636a1000 0x559f6373bb80 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x559f64436d50 tx=0x559f64439320 comp rx=0 tx=0).stop 2023-05-20T01:25:36.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.337+0000 7fe14ac697c0 1 -- 172.21.15.16:0/1036135947 shutdown_connections 2023-05-20T01:25:36.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.337+0000 7fe14ac697c0 1 --2- 172.21.15.16:0/1036135947 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559f636a0800 0x559f6373b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:36.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.337+0000 7fe14ac697c0 1 --2- 172.21.15.16:0/1036135947 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f636a1000 0x559f6373bb80 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:36.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.337+0000 7fe14ac697c0 1 --2- 172.21.15.16:0/1036135947 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559f636a0c00 0x559f6373b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:36.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.337+0000 7fe14ac697c0 1 -- 172.21.15.16:0/1036135947 >> 172.21.15.16:0/1036135947 conn(0x559f636a1400 msgr2=0x559f63771000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:36.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.337+0000 7fe14ac697c0 1 -- 172.21.15.16:0/1036135947 shutdown_connections 2023-05-20T01:25:36.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.337+0000 7fe14ac697c0 1 -- 172.21.15.16:0/1036135947 wait complete. 2023-05-20T01:25:36.374 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_inodes'] in time 0:00:00.146194, result: 2023-05-20T01:25:36.374 DEBUG:teuthology.orchestra.run.smithi016:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_links 2023-05-20T01:25:36.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.427+0000 7fe382e377c0 1 -- start start 2023-05-20T01:25:36.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.427+0000 7fe382e377c0 1 Processor -- start 2023-05-20T01:25:36.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.427+0000 7fe382e377c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563bb011d000 0x563bb01b6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:36.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.427+0000 7fe382e377c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563bb011cc00 0x563bb01b6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:36.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.427+0000 7fe382e377c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563bb011c800 0x563bb01b7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:36.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.427+0000 7fe382e377c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bb0159cc0 con 0x563bb011c800 2023-05-20T01:25:36.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.427+0000 7fe382e377c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bb0159e00 con 0x563bb011d000 2023-05-20T01:25:36.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.427+0000 7fe382e377c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x563bb0159400 con 0x563bb011cc00 2023-05-20T01:25:36.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.427+0000 7fe3815e6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563bb011c800 0x563bb01b7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:36.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.427+0000 7fe3815e6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563bb011c800 0x563bb01b7080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37056/0 (socket says 172.21.15.16:37056) 2023-05-20T01:25:36.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.427+0000 7fe3815e6700 1 -- 172.21.15.16:0/346080380 learned_addr learned my addr 172.21.15.16:0/346080380 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:36.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.427+0000 7fe380de5700 1 --2- 172.21.15.16:0/346080380 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563bb011d000 0x563bb01b6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:36.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.427+0000 7fe3805e4700 1 --2- 172.21.15.16:0/346080380 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563bb011cc00 0x563bb01b6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:36.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.428+0000 7fe3815e6700 1 -- 172.21.15.16:0/346080380 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563bb011cc00 msgr2=0x563bb01b6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:36.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.428+0000 7fe3815e6700 1 --2- 172.21.15.16:0/346080380 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563bb011cc00 0x563bb01b6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:36.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.428+0000 7fe3815e6700 1 -- 172.21.15.16:0/346080380 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563bb011d000 msgr2=0x563bb01b6580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:36.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.428+0000 7fe3815e6700 1 --2- 172.21.15.16:0/346080380 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563bb011d000 0x563bb01b6580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:36.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.428+0000 7fe3815e6700 1 -- 172.21.15.16:0/346080380 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563bb01404e0 con 0x563bb011c800 2023-05-20T01:25:36.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.428+0000 7fe3815e6700 1 --2- 172.21.15.16:0/346080380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563bb011c800 0x563bb01b7080 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x563bb027ce10 tx=0x563bb0286480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b7d5b6af80581f6 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:36.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.428+0000 7fe37e5e0700 1 -- 172.21.15.16:0/346080380 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x563bb0151600 con 0x563bb011c800 2023-05-20T01:25:36.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.428+0000 7fe37e5e0700 1 -- 172.21.15.16:0/346080380 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x563bb016e480 con 0x563bb011c800 2023-05-20T01:25:36.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.428+0000 7fe37e5e0700 1 -- 172.21.15.16:0/346080380 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x563bb01511e0 con 0x563bb011c800 2023-05-20T01:25:36.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.428+0000 7fe382e377c0 1 -- 172.21.15.16:0/346080380 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563bb0141520 con 0x563bb011c800 2023-05-20T01:25:36.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.428+0000 7fe382e377c0 1 -- 172.21.15.16:0/346080380 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x563bb0141380 con 0x563bb011c800 2023-05-20T01:25:36.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.428+0000 7fe382e377c0 4 waiting for MDS map... 2023-05-20T01:25:36.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.428+0000 7fe37e5e0700 1 -- 172.21.15.16:0/346080380 <== mon.0 v2:172.21.15.16:3300/0 4 ==== osd_map(93..93 src has 1..93) v4 ==== 6958+0+0 (secure 0 0 0) 0x563bb015ec40 con 0x563bb011c800 2023-05-20T01:25:36.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.428+0000 7fe37e5e0700 1 -- 172.21.15.16:0/346080380 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 63) v1 ==== 3126+0+0 (secure 0 0 0) 0x563bb0ee8300 con 0x563bb011c800 2023-05-20T01:25:36.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.428+0000 7fe382e377c0 4 Got MDS map 63 2023-05-20T01:25:36.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.429+0000 7fe382e377c0 4 datascan.main: Using metadata pool output 2023-05-20T01:25:36.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.429+0000 7fe382e377c0 4 datascan.main: connecting to RADOS... 2023-05-20T01:25:36.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.429+0000 7fe382e377c0 1 Processor -- start 2023-05-20T01:25:36.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.429+0000 7fe382e377c0 1 -- start start 2023-05-20T01:25:36.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.430+0000 7fe382e377c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563bb011dc00 0x563bb01b7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:36.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.430+0000 7fe382e377c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563bb0430000 0x563bb01b7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:36.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.430+0000 7fe382e377c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563bb0430400 0x563bb01b8100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:36.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.430+0000 7fe382e377c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bb0158a00 con 0x563bb0430000 2023-05-20T01:25:36.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.430+0000 7fe382e377c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bb0158b40 con 0x563bb0430400 2023-05-20T01:25:36.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.430+0000 7fe382e377c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x563bb0159e00 con 0x563bb011dc00 2023-05-20T01:25:36.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.430+0000 7fe380de5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563bb0430000 0x563bb01b7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:36.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.430+0000 7fe380de5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563bb0430000 0x563bb01b7b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37060/0 (socket says 172.21.15.16:37060) 2023-05-20T01:25:36.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.430+0000 7fe380de5700 1 -- 172.21.15.16:0/2699930093 learned_addr learned my addr 172.21.15.16:0/2699930093 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:36.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.430+0000 7fe3805e4700 1 --2- 172.21.15.16:0/2699930093 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563bb011dc00 0x563bb01b7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:36.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.430+0000 7fe3805e4700 1 --2- 172.21.15.16:0/2699930093 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563bb0430400 0x563bb01b8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:36.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.430+0000 7fe380de5700 1 -- 172.21.15.16:0/2699930093 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563bb011dc00 msgr2=0x563bb01b7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:36.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.430+0000 7fe380de5700 1 --2- 172.21.15.16:0/2699930093 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563bb011dc00 0x563bb01b7600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:36.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.430+0000 7fe380de5700 1 -- 172.21.15.16:0/2699930093 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563bb0430400 msgr2=0x563bb01b8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:36.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.430+0000 7fe380de5700 1 --2- 172.21.15.16:0/2699930093 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563bb0430400 0x563bb01b8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:36.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.430+0000 7fe380de5700 1 -- 172.21.15.16:0/2699930093 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563bb0141d40 con 0x563bb0430000 2023-05-20T01:25:36.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.430+0000 7fe3805e4700 1 --2- 172.21.15.16:0/2699930093 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563bb011dc00 0x563bb01b7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:25:36.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.430+0000 7fe380de5700 1 --2- 172.21.15.16:0/2699930093 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563bb0430000 0x563bb01b7b80 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x563bb043e0f0 tx=0x563bb0f9ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9319e79f4705f50a server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:36.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.430+0000 7fe37c5dc700 1 -- 172.21.15.16:0/2699930093 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x563bb0151600 con 0x563bb0430000 2023-05-20T01:25:36.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.431+0000 7fe37c5dc700 1 -- 172.21.15.16:0/2699930093 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x563bb016fe00 con 0x563bb0430000 2023-05-20T01:25:36.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.431+0000 7fe37c5dc700 1 -- 172.21.15.16:0/2699930093 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x563bb01511e0 con 0x563bb0430000 2023-05-20T01:25:36.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.431+0000 7fe382e377c0 1 -- 172.21.15.16:0/2699930093 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563bb0430000 msgr2=0x563bb01b7b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:36.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.431+0000 7fe382e377c0 1 --2- 172.21.15.16:0/2699930093 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563bb0430000 0x563bb01b7b80 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x563bb043e0f0 tx=0x563bb0f9ec00 comp rx=0 tx=0).stop 2023-05-20T01:25:36.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.431+0000 7fe382e377c0 1 -- 172.21.15.16:0/2699930093 shutdown_connections 2023-05-20T01:25:36.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.431+0000 7fe382e377c0 1 --2- 172.21.15.16:0/2699930093 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563bb0430400 0x563bb01b8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:36.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.431+0000 7fe382e377c0 1 --2- 172.21.15.16:0/2699930093 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563bb011dc00 0x563bb01b7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:36.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.431+0000 7fe382e377c0 1 --2- 172.21.15.16:0/2699930093 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563bb0430000 0x563bb01b7b80 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:36.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.431+0000 7fe382e377c0 1 -- 172.21.15.16:0/2699930093 >> 172.21.15.16:0/2699930093 conn(0x563bb011c400 msgr2=0x563bb01ed800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:36.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.431+0000 7fe382e377c0 1 -- 172.21.15.16:0/2699930093 shutdown_connections 2023-05-20T01:25:36.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.431+0000 7fe382e377c0 1 -- 172.21.15.16:0/2699930093 wait complete. 2023-05-20T01:25:36.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.431+0000 7fe382e377c0 1 Processor -- start 2023-05-20T01:25:36.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.432+0000 7fe382e377c0 1 -- start start 2023-05-20T01:25:36.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.432+0000 7fe382e377c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563bb011c400 0x563bb01b9180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:36.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.432+0000 7fe382e377c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563bb0430000 0x563bb01b8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:36.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.432+0000 7fe382e377c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563bb0430400 0x563bb01b8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:36.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.432+0000 7fe382e377c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bb0158c80 con 0x563bb0430400 2023-05-20T01:25:36.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.432+0000 7fe382e377c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bb0158000 con 0x563bb011c400 2023-05-20T01:25:36.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.432+0000 7fe382e377c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x563bb0159180 con 0x563bb0430000 2023-05-20T01:25:36.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.432+0000 7fe3805e4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563bb0430400 0x563bb01b8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:36.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.432+0000 7fe3805e4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563bb0430400 0x563bb01b8680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37068/0 (socket says 172.21.15.16:37068) 2023-05-20T01:25:36.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.432+0000 7fe3805e4700 1 -- 172.21.15.16:0/674568391 learned_addr learned my addr 172.21.15.16:0/674568391 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:36.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.432+0000 7fe3805e4700 1 --2- 172.21.15.16:0/674568391 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563bb011c400 0x563bb01b9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:36.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.432+0000 7fe380de5700 1 --2- 172.21.15.16:0/674568391 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563bb0430000 0x563bb01b8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:36.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.433+0000 7fe3805e4700 1 -- 172.21.15.16:0/674568391 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563bb0430000 msgr2=0x563bb01b8c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:36.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.433+0000 7fe3805e4700 1 --2- 172.21.15.16:0/674568391 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563bb0430000 0x563bb01b8c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:36.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.433+0000 7fe3805e4700 1 -- 172.21.15.16:0/674568391 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563bb011c400 msgr2=0x563bb01b9180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:36.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.433+0000 7fe3805e4700 1 --2- 172.21.15.16:0/674568391 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563bb011c400 0x563bb01b9180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:36.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.433+0000 7fe3805e4700 1 -- 172.21.15.16:0/674568391 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563bb04541a0 con 0x563bb0430400 2023-05-20T01:25:36.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.433+0000 7fe3805e4700 1 --2- 172.21.15.16:0/674568391 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563bb0430400 0x563bb01b8680 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x563bb043f3b0 tx=0x563bb0286300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:36.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.433+0000 7fe37add9700 1 -- 172.21.15.16:0/674568391 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x563bb0151080 con 0x563bb0430400 2023-05-20T01:25:36.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.433+0000 7fe37add9700 1 -- 172.21.15.16:0/674568391 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x563bb016f800 con 0x563bb0430400 2023-05-20T01:25:36.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.433+0000 7fe37add9700 1 -- 172.21.15.16:0/674568391 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x563bb0151600 con 0x563bb0430400 2023-05-20T01:25:36.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.433+0000 7fe382e377c0 1 -- 172.21.15.16:0/674568391 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x563bb0454000 con 0x563bb0430400 2023-05-20T01:25:36.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.433+0000 7fe382e377c0 1 -- 172.21.15.16:0/674568391 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563bb0454340 con 0x563bb0430400 2023-05-20T01:25:36.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.433+0000 7fe382e377c0 4 datascan.init: resolving metadata pool 12 2023-05-20T01:25:36.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.434+0000 7fe37add9700 1 -- 172.21.15.16:0/674568391 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x563bb017e840 con 0x563bb0430400 2023-05-20T01:25:36.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.434+0000 7fe37add9700 1 --2- 172.21.15.16:0/674568391 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x563bb0460400 0x563bb01b9700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:36.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.434+0000 7fe37add9700 1 -- 172.21.15.16:0/674568391 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6958+0+0 (secure 0 0 0) 0x563bb015ea80 con 0x563bb0430400 2023-05-20T01:25:36.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.434+0000 7fe382e377c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-05-20T01:25:36.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.434+0000 7fe380de5700 1 --2- 172.21.15.16:0/674568391 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x563bb0460400 0x563bb01b9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:36.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.434+0000 7fe382e377c0 4 datascan.main: resolving metadata pool 12 2023-05-20T01:25:36.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.434+0000 7fe382e377c0 1 --2- 172.21.15.16:0/674568391 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x563bb0430800 0x563bb01b8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:36.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.435+0000 7fe382e377c0 1 -- 172.21.15.16:0/674568391 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:1 12.0 12:00000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x563bb0f82700 con 0x563bb0430800 2023-05-20T01:25:36.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.435+0000 7fe3805e4700 1 --2- 172.21.15.16:0/674568391 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x563bb0430800 0x563bb01b8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:36.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.435+0000 7fe380de5700 1 --2- 172.21.15.16:0/674568391 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x563bb0460400 0x563bb01b9700 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x563bb043e0f0 tx=0x563bb0286420 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:36.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.435+0000 7fe3805e4700 1 --2- 172.21.15.16:0/674568391 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x563bb0430800 0x563bb01b8100 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:36.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.435+0000 7fe3805e4700 1 -- 172.21.15.16:0/674568391 <== osd.4 v2:172.21.15.132:6817/506317223 1 ==== osd_op_reply(1 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563bb0425440 con 0x563bb0430800 2023-05-20T01:25:36.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.435+0000 7fe3805e4700 1 --2- 172.21.15.16:0/674568391 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x563bb0460800 0x563bb0560000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:36.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.435+0000 7fe3805e4700 1 -- 172.21.15.16:0/674568391 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:2 12.10 12:08000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x563bb0f82700 con 0x563bb0460800 2023-05-20T01:25:36.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.436+0000 7fe3815e6700 1 --2- 172.21.15.16:0/674568391 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x563bb0460800 0x563bb0560000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:36.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.436+0000 7fe3815e6700 1 --2- 172.21.15.16:0/674568391 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x563bb0460800 0x563bb0560000 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:36.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.436+0000 7fe3815e6700 1 -- 172.21.15.16:0/674568391 <== osd.0 v2:172.21.15.16:6824/3923634179 1 ==== osd_op_reply(2 [pgnls start_epoch 93 out=81b] v93'17 uv17 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x563bb0425680 con 0x563bb0460800 2023-05-20T01:25:36.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.436+0000 7fe3815e6700 1 --2- 172.21.15.16:0/674568391 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x563bb0461000 0x563bb0560580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:36.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.436+0000 7fe3815e6700 1 -- 172.21.15.16:0/674568391 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:3 12.8 12:10000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x563bb0f82700 con 0x563bb0461000 2023-05-20T01:25:36.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.436+0000 7fe380de5700 1 --2- 172.21.15.16:0/674568391 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x563bb0461000 0x563bb0560580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:36.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.437+0000 7fe380de5700 1 --2- 172.21.15.16:0/674568391 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x563bb0461000 0x563bb0560580 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:36.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.437+0000 7fe380de5700 1 -- 172.21.15.16:0/674568391 <== osd.3 v2:172.21.15.16:6800/3669781890 1 ==== osd_op_reply(3 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563bb04258c0 con 0x563bb0461000 2023-05-20T01:25:36.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.437+0000 7fe380de5700 1 --2- 172.21.15.16:0/674568391 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x563bb0461800 0x563bb0560b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:36.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.437+0000 7fe380de5700 1 -- 172.21.15.16:0/674568391 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:4 12.18 12:18000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x563bb0f82700 con 0x563bb0461800 2023-05-20T01:25:36.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.437+0000 7fe3805e4700 1 --2- 172.21.15.16:0/674568391 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x563bb0461800 0x563bb0560b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:36.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.437+0000 7fe3805e4700 1 --2- 172.21.15.16:0/674568391 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x563bb0461800 0x563bb0560b00 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:36.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.438+0000 7fe3805e4700 1 -- 172.21.15.16:0/674568391 <== osd.1 v2:172.21.15.16:6808/3860264220 1 ==== osd_op_reply(4 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563bb0425440 con 0x563bb0461800 2023-05-20T01:25:36.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.438+0000 7fe3805e4700 1 --2- 172.21.15.16:0/674568391 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x563bb0460c00 0x563bb0561080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:36.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.438+0000 7fe3805e4700 1 -- 172.21.15.16:0/674568391 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:5 12.4 12:20000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x563bb0f82700 con 0x563bb0460c00 2023-05-20T01:25:36.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.438+0000 7fe3815e6700 1 --2- 172.21.15.16:0/674568391 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x563bb0460c00 0x563bb0561080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:36.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.438+0000 7fe3815e6700 1 --2- 172.21.15.16:0/674568391 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x563bb0460c00 0x563bb0561080 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:36.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.438+0000 7fe3815e6700 1 -- 172.21.15.16:0/674568391 <== osd.6 v2:172.21.15.132:6801/727581123 1 ==== osd_op_reply(5 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563bb0425680 con 0x563bb0460c00 2023-05-20T01:25:36.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.438+0000 7fe3815e6700 1 --2- 172.21.15.16:0/674568391 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x563bb0461400 0x563bb0561b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:36.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.438+0000 7fe3815e6700 1 -- 172.21.15.16:0/674568391 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:6 12.14 12:28000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x563bb0f82700 con 0x563bb0461400 2023-05-20T01:25:36.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.439+0000 7fe380de5700 1 --2- 172.21.15.16:0/674568391 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x563bb0461400 0x563bb0561b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:36.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.439+0000 7fe380de5700 1 --2- 172.21.15.16:0/674568391 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x563bb0461400 0x563bb0561b80 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:36.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.439+0000 7fe380de5700 1 -- 172.21.15.16:0/674568391 <== osd.5 v2:172.21.15.132:6816/1844689995 1 ==== osd_op_reply(6 [pgnls start_epoch 93 out=125b] v93'29 uv29 ondisk = 1) v8 ==== 144+0+125 (crc 0 0 0) 0x563bb04258c0 con 0x563bb0461400 2023-05-20T01:25:36.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.439+0000 7fe380de5700 1 -- 172.21.15.16:0/674568391 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:7 12.c 12:30000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x563bb0f82700 con 0x563bb0460800 2023-05-20T01:25:36.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.439+0000 7fe3815e6700 1 -- 172.21.15.16:0/674568391 <== osd.0 v2:172.21.15.16:6824/3923634179 2 ==== osd_op_reply(7 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563bb0425680 con 0x563bb0460800 2023-05-20T01:25:36.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.439+0000 7fe3815e6700 1 -- 172.21.15.16:0/674568391 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:8 12.1c 12:38000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x563bb0f82700 con 0x563bb0460800 2023-05-20T01:25:36.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.440+0000 7fe3815e6700 1 -- 172.21.15.16:0/674568391 <== osd.0 v2:172.21.15.16:6824/3923634179 3 ==== osd_op_reply(8 [pgnls start_epoch 93 out=49b] v92'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563bb0425680 con 0x563bb0460800 2023-05-20T01:25:36.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.440+0000 7fe3815e6700 1 -- 172.21.15.16:0/674568391 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:9 12.2 12:40000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x563bb0f82700 con 0x563bb0430800 2023-05-20T01:25:36.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.440+0000 7fe3805e4700 1 -- 172.21.15.16:0/674568391 <== osd.4 v2:172.21.15.132:6817/506317223 2 ==== osd_op_reply(9 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563bb0425440 con 0x563bb0430800 2023-05-20T01:25:36.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.440+0000 7fe3805e4700 1 -- 172.21.15.16:0/674568391 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:10 12.12 12:48000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x563bb0f82700 con 0x563bb0461000 2023-05-20T01:25:36.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.440+0000 7fe380de5700 1 -- 172.21.15.16:0/674568391 <== osd.3 v2:172.21.15.16:6800/3669781890 2 ==== osd_op_reply(10 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563bb04258c0 con 0x563bb0461000 2023-05-20T01:25:36.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.440+0000 7fe380de5700 1 --2- 172.21.15.16:0/674568391 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x563bb0461c00 0x563bb0562680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:36.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.440+0000 7fe380de5700 1 -- 172.21.15.16:0/674568391 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:11 12.a 12:50000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x563bb0f82700 con 0x563bb0461c00 2023-05-20T01:25:36.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.440+0000 7fe3805e4700 1 --2- 172.21.15.16:0/674568391 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x563bb0461c00 0x563bb0562680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:36.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.441+0000 7fe3805e4700 1 --2- 172.21.15.16:0/674568391 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x563bb0461c00 0x563bb0562680 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:36.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.441+0000 7fe3805e4700 1 -- 172.21.15.16:0/674568391 <== osd.2 v2:172.21.15.16:6816/293187030 1 ==== osd_op_reply(11 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563bb0425440 con 0x563bb0461c00 2023-05-20T01:25:36.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.441+0000 7fe3805e4700 1 -- 172.21.15.16:0/674568391 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:12 12.1a 12:58000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x563bb0f82700 con 0x563bb0430800 2023-05-20T01:25:36.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.441+0000 7fe3805e4700 1 -- 172.21.15.16:0/674568391 <== osd.4 v2:172.21.15.132:6817/506317223 3 ==== osd_op_reply(12 [pgnls start_epoch 93 out=49b] v93'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563bb0425440 con 0x563bb0430800 2023-05-20T01:25:36.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.441+0000 7fe3805e4700 1 -- 172.21.15.16:0/674568391 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:13 12.6 12:60000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x563bb0f82700 con 0x563bb0461800 2023-05-20T01:25:36.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.441+0000 7fe3805e4700 1 -- 172.21.15.16:0/674568391 <== osd.1 v2:172.21.15.16:6808/3860264220 2 ==== osd_op_reply(13 [pgnls start_epoch 93 out=49b] v92'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563bb0425440 con 0x563bb0461800 2023-05-20T01:25:36.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.441+0000 7fe3805e4700 1 -- 172.21.15.16:0/674568391 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:14 12.16 12:68000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x563bb0f82700 con 0x563bb0460c00 2023-05-20T01:25:36.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.442+0000 7fe3815e6700 1 -- 172.21.15.16:0/674568391 <== osd.6 v2:172.21.15.132:6801/727581123 2 ==== osd_op_reply(14 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563bb0425680 con 0x563bb0460c00 2023-05-20T01:25:36.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.442+0000 7fe3815e6700 1 -- 172.21.15.16:0/674568391 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:15 12.e 12:70000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x563bb0f82700 con 0x563bb0460800 2023-05-20T01:25:36.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.442+0000 7fe3815e6700 1 -- 172.21.15.16:0/674568391 <== osd.0 v2:172.21.15.16:6824/3923634179 4 ==== osd_op_reply(15 [pgnls start_epoch 93 out=49b] v92'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563bb0425680 con 0x563bb0460800 2023-05-20T01:25:36.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.442+0000 7fe3815e6700 1 -- 172.21.15.16:0/674568391 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:16 12.1e 12:78000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x563bb0f82700 con 0x563bb0430800 2023-05-20T01:25:36.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.442+0000 7fe3805e4700 1 -- 172.21.15.16:0/674568391 <== osd.4 v2:172.21.15.132:6817/506317223 4 ==== osd_op_reply(16 [pgnls start_epoch 93 out=49b] v93'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563bb0425440 con 0x563bb0430800 2023-05-20T01:25:36.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.442+0000 7fe3805e4700 1 -- 172.21.15.16:0/674568391 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:17 12.1 12:80000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x563bb0f82700 con 0x563bb0460c00 2023-05-20T01:25:36.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.443+0000 7fe3815e6700 1 -- 172.21.15.16:0/674568391 <== osd.6 v2:172.21.15.132:6801/727581123 3 ==== osd_op_reply(17 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563bb0425680 con 0x563bb0460c00 2023-05-20T01:25:36.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.443+0000 7fe3815e6700 1 -- 172.21.15.16:0/674568391 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:18 12.11 12:88000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x563bb0f82700 con 0x563bb0461400 2023-05-20T01:25:36.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.443+0000 7fe380de5700 1 -- 172.21.15.16:0/674568391 <== osd.5 v2:172.21.15.132:6816/1844689995 2 ==== osd_op_reply(18 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563bb04258c0 con 0x563bb0461400 2023-05-20T01:25:36.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.443+0000 7fe380de5700 1 -- 172.21.15.16:0/674568391 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:19 12.9 12:90000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x563bb0f82700 con 0x563bb0461c00 2023-05-20T01:25:36.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.443+0000 7fe3805e4700 1 -- 172.21.15.16:0/674568391 <== osd.2 v2:172.21.15.16:6816/293187030 2 ==== osd_op_reply(19 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563bb0425440 con 0x563bb0461c00 2023-05-20T01:25:36.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.443+0000 7fe3805e4700 1 -- 172.21.15.16:0/674568391 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:20 12.19 12:98000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x563bb0f82700 con 0x563bb0460800 2023-05-20T01:25:36.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.443+0000 7fe3815e6700 1 -- 172.21.15.16:0/674568391 <== osd.0 v2:172.21.15.16:6824/3923634179 5 ==== osd_op_reply(20 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563bb0425680 con 0x563bb0460800 2023-05-20T01:25:36.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.443+0000 7fe3815e6700 1 -- 172.21.15.16:0/674568391 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:21 12.5 12:a0000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x563bb0f82700 con 0x563bb0460c00 2023-05-20T01:25:36.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.444+0000 7fe3815e6700 1 -- 172.21.15.16:0/674568391 <== osd.6 v2:172.21.15.132:6801/727581123 4 ==== osd_op_reply(21 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563bb0425680 con 0x563bb0460c00 2023-05-20T01:25:36.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.444+0000 7fe3815e6700 1 -- 172.21.15.16:0/674568391 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:22 12.15 12:a8000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x563bb0f82700 con 0x563bb0460c00 2023-05-20T01:25:36.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.444+0000 7fe3815e6700 1 -- 172.21.15.16:0/674568391 <== osd.6 v2:172.21.15.132:6801/727581123 5 ==== osd_op_reply(22 [pgnls start_epoch 93 out=73b] v92'6 uv6 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x563bb0425680 con 0x563bb0460c00 2023-05-20T01:25:36.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.444+0000 7fe3815e6700 1 --2- 172.21.15.16:0/674568391 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x563bb0582000 0x563bb0562100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:36.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.444+0000 7fe3815e6700 1 -- 172.21.15.16:0/674568391 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:23 12.d 12:b0000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x563bb0f82700 con 0x563bb0582000 2023-05-20T01:25:36.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.444+0000 7fe3815e6700 1 --2- 172.21.15.16:0/674568391 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x563bb0582000 0x563bb0562100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:36.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.445+0000 7fe3815e6700 1 --2- 172.21.15.16:0/674568391 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x563bb0582000 0x563bb0562100 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:36.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.445+0000 7fe3815e6700 1 -- 172.21.15.16:0/674568391 <== osd.7 v2:172.21.15.132:6800/466001149 1 ==== osd_op_reply(23 [pgnls start_epoch 93 out=74b] v93'3 uv3 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x563bb0425680 con 0x563bb0582000 2023-05-20T01:25:36.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.445+0000 7fe3815e6700 1 -- 172.21.15.16:0/674568391 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:24 12.1d 12:b8000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x563bb0f82700 con 0x563bb0460800 2023-05-20T01:25:36.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.445+0000 7fe3815e6700 1 -- 172.21.15.16:0/674568391 <== osd.0 v2:172.21.15.16:6824/3923634179 6 ==== osd_op_reply(24 [pgnls start_epoch 93 out=49b] v92'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563bb0425680 con 0x563bb0460800 2023-05-20T01:25:36.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.445+0000 7fe3815e6700 1 -- 172.21.15.16:0/674568391 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:25 12.3 12:c0000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x563bb0f82700 con 0x563bb0460800 2023-05-20T01:25:36.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.445+0000 7fe3815e6700 1 -- 172.21.15.16:0/674568391 <== osd.0 v2:172.21.15.16:6824/3923634179 7 ==== osd_op_reply(25 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563bb0425680 con 0x563bb0460800 2023-05-20T01:25:36.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.445+0000 7fe3815e6700 1 -- 172.21.15.16:0/674568391 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:26 12.13 12:c8000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x563bb0f82700 con 0x563bb0461800 2023-05-20T01:25:36.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.446+0000 7fe3805e4700 1 -- 172.21.15.16:0/674568391 <== osd.1 v2:172.21.15.16:6808/3860264220 3 ==== osd_op_reply(26 [pgnls start_epoch 93 out=98b] v93'7 uv7 ondisk = 1) v8 ==== 144+0+98 (crc 0 0 0) 0x563bb0425440 con 0x563bb0461800 2023-05-20T01:25:36.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.446+0000 7fe3805e4700 1 -- 172.21.15.16:0/674568391 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:27 12.b 12:d0000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x563bb0f82700 con 0x563bb0461000 2023-05-20T01:25:36.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.446+0000 7fe380de5700 1 -- 172.21.15.16:0/674568391 <== osd.3 v2:172.21.15.16:6800/3669781890 3 ==== osd_op_reply(27 [pgnls start_epoch 93 out=76b] v93'6 uv5 ondisk = 1) v8 ==== 144+0+76 (crc 0 0 0) 0x563bb04258c0 con 0x563bb0461000 2023-05-20T01:25:36.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.446+0000 7fe380de5700 1 -- 172.21.15.16:0/674568391 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:28 12.1b 12:d8000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x563bb0f82700 con 0x563bb0430800 2023-05-20T01:25:36.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.446+0000 7fe3805e4700 1 -- 172.21.15.16:0/674568391 <== osd.4 v2:172.21.15.132:6817/506317223 5 ==== osd_op_reply(28 [pgnls start_epoch 93 out=49b] v93'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563bb0425440 con 0x563bb0430800 2023-05-20T01:25:36.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.446+0000 7fe3805e4700 1 -- 172.21.15.16:0/674568391 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:29 12.7 12:e0000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x563bb0f82700 con 0x563bb0461800 2023-05-20T01:25:36.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.447+0000 7fe3805e4700 1 -- 172.21.15.16:0/674568391 <== osd.1 v2:172.21.15.16:6808/3860264220 4 ==== osd_op_reply(29 [pgnls start_epoch 93 out=49b] v92'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563bb0425440 con 0x563bb0461800 2023-05-20T01:25:36.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.447+0000 7fe3805e4700 1 -- 172.21.15.16:0/674568391 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:30 12.17 12:e8000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x563bb0f82700 con 0x563bb0460c00 2023-05-20T01:25:36.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.447+0000 7fe3815e6700 1 -- 172.21.15.16:0/674568391 <== osd.6 v2:172.21.15.132:6801/727581123 6 ==== osd_op_reply(30 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563bb0425680 con 0x563bb0460c00 2023-05-20T01:25:36.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.447+0000 7fe3815e6700 1 -- 172.21.15.16:0/674568391 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:31 12.f 12:f0000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x563bb0f82700 con 0x563bb0460c00 2023-05-20T01:25:36.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.447+0000 7fe3815e6700 1 -- 172.21.15.16:0/674568391 <== osd.6 v2:172.21.15.132:6801/727581123 7 ==== osd_op_reply(31 [pgnls start_epoch 93 out=73b] v93'4 uv4 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x563bb0425680 con 0x563bb0460c00 2023-05-20T01:25:36.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.447+0000 7fe3815e6700 1 -- 172.21.15.16:0/674568391 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:32 12.1f 12:f8000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x563bb0f82700 con 0x563bb0460c00 2023-05-20T01:25:36.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.447+0000 7fe3815e6700 1 -- 172.21.15.16:0/674568391 <== osd.6 v2:172.21.15.132:6801/727581123 8 ==== osd_op_reply(32 [pgnls start_epoch 93 out=101b] v93'12 uv12 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x563bb0425680 con 0x563bb0460c00 2023-05-20T01:25:36.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.448+0000 7fe382e377c0 10 datascan.scan_links: step 1: handling object 10000000000.00000000 2023-05-20T01:25:36.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.448+0000 7fe382e377c0 1 -- 172.21.15.16:0/674568391 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:33 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x563bb0f82a80 con 0x563bb0460800 2023-05-20T01:25:36.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.448+0000 7fe3815e6700 1 -- 172.21.15.16:0/674568391 <== osd.0 v2:172.21.15.16:6824/3923634179 8 ==== osd_op_reply(33 10000000000.00000000 [omap-get-vals out=509b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+509 (crc 0 0 0) 0x563bb0425680 con 0x563bb0460800 2023-05-20T01:25:36.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.448+0000 7fe382e377c0 20 datascan.scan_links: injected ino detected 2023-05-20T01:25:36.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.448+0000 7fe382e377c0 10 datascan.scan_links: step 1: handling object 1.00000000.inode 2023-05-20T01:25:36.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.448+0000 7fe382e377c0 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2023-05-20T01:25:36.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.448+0000 7fe382e377c0 10 datascan.scan_links: step 1: handling object 200.00000000 2023-05-20T01:25:36.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.448+0000 7fe382e377c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2023-05-20T01:25:36.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.448+0000 7fe382e377c0 10 datascan.scan_links: step 1: handling object 200.00000001 2023-05-20T01:25:36.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.448+0000 7fe382e377c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2023-05-20T01:25:36.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.448+0000 7fe382e377c0 10 datascan.scan_links: step 1: handling object 500.00000000 2023-05-20T01:25:36.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.448+0000 7fe382e377c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2023-05-20T01:25:36.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.448+0000 7fe382e377c0 10 datascan.scan_links: step 1: handling object mds_snaptable 2023-05-20T01:25:36.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.448+0000 7fe382e377c0 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2023-05-20T01:25:36.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.448+0000 7fe382e377c0 10 datascan.scan_links: step 1: handling object mds0_inotable 2023-05-20T01:25:36.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.448+0000 7fe382e377c0 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2023-05-20T01:25:36.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.448+0000 7fe382e377c0 10 datascan.scan_links: step 1: handling object 100.00000000 2023-05-20T01:25:36.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.448+0000 7fe382e377c0 1 -- 172.21.15.16:0/674568391 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:34 12.13 12:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x563bb0f82e00 con 0x563bb0461800 2023-05-20T01:25:36.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.448+0000 7fe3805e4700 1 -- 172.21.15.16:0/674568391 <== osd.1 v2:172.21.15.16:6808/3860264220 5 ==== osd_op_reply(34 100.00000000 [omap-get-vals out=5b] v0'0 uv7 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x563bb0425440 con 0x563bb0461800 2023-05-20T01:25:36.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.448+0000 7fe382e377c0 10 datascan.scan_links: step 1: handling object mds0_sessionmap 2023-05-20T01:25:36.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.448+0000 7fe382e377c0 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2023-05-20T01:25:36.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.448+0000 7fe382e377c0 10 datascan.scan_links: step 1: handling object 400.00000000 2023-05-20T01:25:36.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.448+0000 7fe382e377c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2023-05-20T01:25:36.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.448+0000 7fe382e377c0 10 datascan.scan_links: step 1: handling object 100.00000000.inode 2023-05-20T01:25:36.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.448+0000 7fe382e377c0 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2023-05-20T01:25:36.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.448+0000 7fe382e377c0 10 datascan.scan_links: step 1: handling object 1.00000000 2023-05-20T01:25:36.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.448+0000 7fe382e377c0 1 -- 172.21.15.16:0/674568391 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:35 12.1f 12:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x563bb0f83180 con 0x563bb0460c00 2023-05-20T01:25:36.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.449+0000 7fe3815e6700 1 -- 172.21.15.16:0/674568391 <== osd.6 v2:172.21.15.132:6801/727581123 9 ==== osd_op_reply(35 1.00000000 [omap-get-vals out=508b] v0'0 uv12 ondisk = 0) v8 ==== 154+0+508 (crc 0 0 0) 0x563bb0425680 con 0x563bb0460c00 2023-05-20T01:25:36.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.449+0000 7fe382e377c0 20 datascan.scan_links: injected ino detected 2023-05-20T01:25:36.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.449+0000 7fe382e377c0 1 -- 172.21.15.16:0/674568391 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:36 12.0 12:00000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x563bb0f83500 con 0x563bb0430800 2023-05-20T01:25:36.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.449+0000 7fe3805e4700 1 -- 172.21.15.16:0/674568391 <== osd.4 v2:172.21.15.132:6817/506317223 6 ==== osd_op_reply(36 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563bb0425440 con 0x563bb0430800 2023-05-20T01:25:36.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.449+0000 7fe3805e4700 1 -- 172.21.15.16:0/674568391 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:37 12.10 12:08000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x563bb0f82e00 con 0x563bb0460800 2023-05-20T01:25:36.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.449+0000 7fe3815e6700 1 -- 172.21.15.16:0/674568391 <== osd.0 v2:172.21.15.16:6824/3923634179 9 ==== osd_op_reply(37 [pgnls start_epoch 93 out=81b] v93'17 uv17 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x563bb0425680 con 0x563bb0460800 2023-05-20T01:25:36.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.449+0000 7fe3815e6700 1 -- 172.21.15.16:0/674568391 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:38 12.8 12:10000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x563bb0f82e00 con 0x563bb0461000 2023-05-20T01:25:36.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.450+0000 7fe380de5700 1 -- 172.21.15.16:0/674568391 <== osd.3 v2:172.21.15.16:6800/3669781890 4 ==== osd_op_reply(38 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563bb04258c0 con 0x563bb0461000 2023-05-20T01:25:36.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.450+0000 7fe380de5700 1 -- 172.21.15.16:0/674568391 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:39 12.18 12:18000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x563bb0f82e00 con 0x563bb0461800 2023-05-20T01:25:36.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.450+0000 7fe3805e4700 1 -- 172.21.15.16:0/674568391 <== osd.1 v2:172.21.15.16:6808/3860264220 6 ==== osd_op_reply(39 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563bb0425440 con 0x563bb0461800 2023-05-20T01:25:36.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.450+0000 7fe3805e4700 1 -- 172.21.15.16:0/674568391 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:40 12.4 12:20000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x563bb0f82e00 con 0x563bb0460c00 2023-05-20T01:25:36.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.450+0000 7fe3815e6700 1 -- 172.21.15.16:0/674568391 <== osd.6 v2:172.21.15.132:6801/727581123 10 ==== osd_op_reply(40 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563bb0425680 con 0x563bb0460c00 2023-05-20T01:25:36.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.450+0000 7fe3815e6700 1 -- 172.21.15.16:0/674568391 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:41 12.14 12:28000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x563bb0f82e00 con 0x563bb0461400 2023-05-20T01:25:36.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.450+0000 7fe380de5700 1 -- 172.21.15.16:0/674568391 <== osd.5 v2:172.21.15.132:6816/1844689995 3 ==== osd_op_reply(41 [pgnls start_epoch 93 out=125b] v93'29 uv29 ondisk = 1) v8 ==== 144+0+125 (crc 0 0 0) 0x563bb04258c0 con 0x563bb0461400 2023-05-20T01:25:36.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.450+0000 7fe380de5700 1 -- 172.21.15.16:0/674568391 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:42 12.c 12:30000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x563bb0f82e00 con 0x563bb0460800 2023-05-20T01:25:36.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.451+0000 7fe3815e6700 1 -- 172.21.15.16:0/674568391 <== osd.0 v2:172.21.15.16:6824/3923634179 10 ==== osd_op_reply(42 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563bb0425680 con 0x563bb0460800 2023-05-20T01:25:36.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.451+0000 7fe3815e6700 1 -- 172.21.15.16:0/674568391 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:43 12.1c 12:38000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x563bb0f82e00 con 0x563bb0460800 2023-05-20T01:25:36.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.451+0000 7fe3815e6700 1 -- 172.21.15.16:0/674568391 <== osd.0 v2:172.21.15.16:6824/3923634179 11 ==== osd_op_reply(43 [pgnls start_epoch 93 out=49b] v92'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563bb0425680 con 0x563bb0460800 2023-05-20T01:25:36.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.451+0000 7fe3815e6700 1 -- 172.21.15.16:0/674568391 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:44 12.2 12:40000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x563bb0f82e00 con 0x563bb0430800 2023-05-20T01:25:36.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.451+0000 7fe3805e4700 1 -- 172.21.15.16:0/674568391 <== osd.4 v2:172.21.15.132:6817/506317223 7 ==== osd_op_reply(44 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563bb0425440 con 0x563bb0430800 2023-05-20T01:25:36.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.451+0000 7fe3805e4700 1 -- 172.21.15.16:0/674568391 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:45 12.12 12:48000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x563bb0f82e00 con 0x563bb0461000 2023-05-20T01:25:36.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.451+0000 7fe380de5700 1 -- 172.21.15.16:0/674568391 <== osd.3 v2:172.21.15.16:6800/3669781890 5 ==== osd_op_reply(45 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563bb04258c0 con 0x563bb0461000 2023-05-20T01:25:36.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.451+0000 7fe380de5700 1 -- 172.21.15.16:0/674568391 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:46 12.a 12:50000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x563bb0f82e00 con 0x563bb0461c00 2023-05-20T01:25:36.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.452+0000 7fe3805e4700 1 -- 172.21.15.16:0/674568391 <== osd.2 v2:172.21.15.16:6816/293187030 3 ==== osd_op_reply(46 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563bb0425440 con 0x563bb0461c00 2023-05-20T01:25:36.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.452+0000 7fe3805e4700 1 -- 172.21.15.16:0/674568391 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:47 12.1a 12:58000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x563bb0f82e00 con 0x563bb0430800 2023-05-20T01:25:36.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.452+0000 7fe3805e4700 1 -- 172.21.15.16:0/674568391 <== osd.4 v2:172.21.15.132:6817/506317223 8 ==== osd_op_reply(47 [pgnls start_epoch 93 out=49b] v93'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563bb0425440 con 0x563bb0430800 2023-05-20T01:25:36.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.452+0000 7fe3805e4700 1 -- 172.21.15.16:0/674568391 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:48 12.6 12:60000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x563bb0f82e00 con 0x563bb0461800 2023-05-20T01:25:36.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.452+0000 7fe3805e4700 1 -- 172.21.15.16:0/674568391 <== osd.1 v2:172.21.15.16:6808/3860264220 7 ==== osd_op_reply(48 [pgnls start_epoch 93 out=49b] v92'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563bb0425440 con 0x563bb0461800 2023-05-20T01:25:36.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.452+0000 7fe3805e4700 1 -- 172.21.15.16:0/674568391 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:49 12.16 12:68000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x563bb0f82e00 con 0x563bb0460c00 2023-05-20T01:25:36.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.452+0000 7fe3815e6700 1 -- 172.21.15.16:0/674568391 <== osd.6 v2:172.21.15.132:6801/727581123 11 ==== osd_op_reply(49 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563bb0425680 con 0x563bb0460c00 2023-05-20T01:25:36.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.453+0000 7fe3815e6700 1 -- 172.21.15.16:0/674568391 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:50 12.e 12:70000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x563bb0f82e00 con 0x563bb0460800 2023-05-20T01:25:36.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.453+0000 7fe3815e6700 1 -- 172.21.15.16:0/674568391 <== osd.0 v2:172.21.15.16:6824/3923634179 12 ==== osd_op_reply(50 [pgnls start_epoch 93 out=49b] v92'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563bb0425680 con 0x563bb0460800 2023-05-20T01:25:36.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.453+0000 7fe3815e6700 1 -- 172.21.15.16:0/674568391 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:51 12.1e 12:78000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x563bb0f82e00 con 0x563bb0430800 2023-05-20T01:25:36.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.453+0000 7fe3805e4700 1 -- 172.21.15.16:0/674568391 <== osd.4 v2:172.21.15.132:6817/506317223 9 ==== osd_op_reply(51 [pgnls start_epoch 93 out=49b] v93'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563bb0425440 con 0x563bb0430800 2023-05-20T01:25:36.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.453+0000 7fe3805e4700 1 -- 172.21.15.16:0/674568391 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:52 12.1 12:80000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x563bb0f82e00 con 0x563bb0460c00 2023-05-20T01:25:36.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.453+0000 7fe3815e6700 1 -- 172.21.15.16:0/674568391 <== osd.6 v2:172.21.15.132:6801/727581123 12 ==== osd_op_reply(52 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563bb0425680 con 0x563bb0460c00 2023-05-20T01:25:36.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.453+0000 7fe3815e6700 1 -- 172.21.15.16:0/674568391 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:53 12.11 12:88000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x563bb0f82e00 con 0x563bb0461400 2023-05-20T01:25:36.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.454+0000 7fe380de5700 1 -- 172.21.15.16:0/674568391 <== osd.5 v2:172.21.15.132:6816/1844689995 4 ==== osd_op_reply(53 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563bb04258c0 con 0x563bb0461400 2023-05-20T01:25:36.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.454+0000 7fe380de5700 1 -- 172.21.15.16:0/674568391 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:54 12.9 12:90000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x563bb0f82e00 con 0x563bb0461c00 2023-05-20T01:25:36.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.454+0000 7fe3805e4700 1 -- 172.21.15.16:0/674568391 <== osd.2 v2:172.21.15.16:6816/293187030 4 ==== osd_op_reply(54 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563bb0425440 con 0x563bb0461c00 2023-05-20T01:25:36.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.454+0000 7fe3805e4700 1 -- 172.21.15.16:0/674568391 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:55 12.19 12:98000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x563bb0f82e00 con 0x563bb0460800 2023-05-20T01:25:36.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.454+0000 7fe3815e6700 1 -- 172.21.15.16:0/674568391 <== osd.0 v2:172.21.15.16:6824/3923634179 13 ==== osd_op_reply(55 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563bb0425680 con 0x563bb0460800 2023-05-20T01:25:36.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.454+0000 7fe3815e6700 1 -- 172.21.15.16:0/674568391 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:56 12.5 12:a0000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x563bb0f82e00 con 0x563bb0460c00 2023-05-20T01:25:36.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.454+0000 7fe3815e6700 1 -- 172.21.15.16:0/674568391 <== osd.6 v2:172.21.15.132:6801/727581123 13 ==== osd_op_reply(56 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563bb0425680 con 0x563bb0460c00 2023-05-20T01:25:36.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.455+0000 7fe3815e6700 1 -- 172.21.15.16:0/674568391 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:57 12.15 12:a8000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x563bb0f82e00 con 0x563bb0460c00 2023-05-20T01:25:36.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.455+0000 7fe3815e6700 1 -- 172.21.15.16:0/674568391 <== osd.6 v2:172.21.15.132:6801/727581123 14 ==== osd_op_reply(57 [pgnls start_epoch 93 out=73b] v92'6 uv6 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x563bb0425680 con 0x563bb0460c00 2023-05-20T01:25:36.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.455+0000 7fe3815e6700 1 -- 172.21.15.16:0/674568391 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:58 12.d 12:b0000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x563bb0f82e00 con 0x563bb0582000 2023-05-20T01:25:36.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.455+0000 7fe3815e6700 1 -- 172.21.15.16:0/674568391 <== osd.7 v2:172.21.15.132:6800/466001149 2 ==== osd_op_reply(58 [pgnls start_epoch 93 out=74b] v93'3 uv3 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x563bb0425680 con 0x563bb0582000 2023-05-20T01:25:36.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.455+0000 7fe3815e6700 1 -- 172.21.15.16:0/674568391 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:59 12.1d 12:b8000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x563bb0f82e00 con 0x563bb0460800 2023-05-20T01:25:36.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.455+0000 7fe3815e6700 1 -- 172.21.15.16:0/674568391 <== osd.0 v2:172.21.15.16:6824/3923634179 14 ==== osd_op_reply(59 [pgnls start_epoch 93 out=49b] v92'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563bb0425680 con 0x563bb0460800 2023-05-20T01:25:36.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.455+0000 7fe3815e6700 1 -- 172.21.15.16:0/674568391 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:60 12.3 12:c0000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x563bb0f82e00 con 0x563bb0460800 2023-05-20T01:25:36.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.455+0000 7fe3815e6700 1 -- 172.21.15.16:0/674568391 <== osd.0 v2:172.21.15.16:6824/3923634179 15 ==== osd_op_reply(60 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563bb0425680 con 0x563bb0460800 2023-05-20T01:25:36.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.455+0000 7fe3815e6700 1 -- 172.21.15.16:0/674568391 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:61 12.13 12:c8000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x563bb0f82e00 con 0x563bb0461800 2023-05-20T01:25:36.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.456+0000 7fe3805e4700 1 -- 172.21.15.16:0/674568391 <== osd.1 v2:172.21.15.16:6808/3860264220 8 ==== osd_op_reply(61 [pgnls start_epoch 93 out=98b] v93'7 uv7 ondisk = 1) v8 ==== 144+0+98 (crc 0 0 0) 0x563bb0425440 con 0x563bb0461800 2023-05-20T01:25:36.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.456+0000 7fe3805e4700 1 -- 172.21.15.16:0/674568391 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:62 12.b 12:d0000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x563bb0f82e00 con 0x563bb0461000 2023-05-20T01:25:36.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.456+0000 7fe380de5700 1 -- 172.21.15.16:0/674568391 <== osd.3 v2:172.21.15.16:6800/3669781890 6 ==== osd_op_reply(62 [pgnls start_epoch 93 out=76b] v93'6 uv5 ondisk = 1) v8 ==== 144+0+76 (crc 0 0 0) 0x563bb04258c0 con 0x563bb0461000 2023-05-20T01:25:36.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.456+0000 7fe380de5700 1 -- 172.21.15.16:0/674568391 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:63 12.1b 12:d8000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x563bb0f82e00 con 0x563bb0430800 2023-05-20T01:25:36.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.456+0000 7fe3805e4700 1 -- 172.21.15.16:0/674568391 <== osd.4 v2:172.21.15.132:6817/506317223 10 ==== osd_op_reply(63 [pgnls start_epoch 93 out=49b] v93'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563bb0425440 con 0x563bb0430800 2023-05-20T01:25:36.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.456+0000 7fe3805e4700 1 -- 172.21.15.16:0/674568391 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:64 12.7 12:e0000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x563bb0f82e00 con 0x563bb0461800 2023-05-20T01:25:36.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.457+0000 7fe3805e4700 1 -- 172.21.15.16:0/674568391 <== osd.1 v2:172.21.15.16:6808/3860264220 9 ==== osd_op_reply(64 [pgnls start_epoch 93 out=49b] v92'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563bb0425440 con 0x563bb0461800 2023-05-20T01:25:36.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.457+0000 7fe3805e4700 1 -- 172.21.15.16:0/674568391 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:65 12.17 12:e8000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x563bb0f82e00 con 0x563bb0460c00 2023-05-20T01:25:36.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.457+0000 7fe3815e6700 1 -- 172.21.15.16:0/674568391 <== osd.6 v2:172.21.15.132:6801/727581123 15 ==== osd_op_reply(65 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563bb0425680 con 0x563bb0460c00 2023-05-20T01:25:36.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.457+0000 7fe3815e6700 1 -- 172.21.15.16:0/674568391 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:66 12.f 12:f0000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x563bb0f82e00 con 0x563bb0460c00 2023-05-20T01:25:36.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.457+0000 7fe3815e6700 1 -- 172.21.15.16:0/674568391 <== osd.6 v2:172.21.15.132:6801/727581123 16 ==== osd_op_reply(66 [pgnls start_epoch 93 out=73b] v93'4 uv4 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x563bb0425680 con 0x563bb0460c00 2023-05-20T01:25:36.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.457+0000 7fe3815e6700 1 -- 172.21.15.16:0/674568391 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:67 12.1f 12:f8000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x563bb0f82e00 con 0x563bb0460c00 2023-05-20T01:25:36.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.458+0000 7fe3815e6700 1 -- 172.21.15.16:0/674568391 <== osd.6 v2:172.21.15.132:6801/727581123 17 ==== osd_op_reply(67 [pgnls start_epoch 93 out=101b] v93'12 uv12 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x563bb0425680 con 0x563bb0460c00 2023-05-20T01:25:36.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.458+0000 7fe382e377c0 10 datascan.scan_links: step 2: handling object 10000000000.00000000 2023-05-20T01:25:36.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.458+0000 7fe382e377c0 1 -- 172.21.15.16:0/674568391 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:68 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x563bb0f83880 con 0x563bb0460800 2023-05-20T01:25:36.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.458+0000 7fe3815e6700 1 -- 172.21.15.16:0/674568391 <== osd.0 v2:172.21.15.16:6824/3923634179 16 ==== osd_op_reply(68 10000000000.00000000 [omap-get-vals out=509b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+509 (crc 0 0 0) 0x563bb0425680 con 0x563bb0460800 2023-05-20T01:25:36.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.458+0000 7fe382e377c0 20 datascan.scan_links: injected ino detected 2023-05-20T01:25:36.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.458+0000 7fe382e377c0 20 datascan.scan_links: adding 0x10000000001 for future processing to fix dnfirst 2023-05-20T01:25:36.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.458+0000 7fe382e377c0 10 datascan.scan_links: step 2: handling object 1.00000000.inode 2023-05-20T01:25:36.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.458+0000 7fe382e377c0 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2023-05-20T01:25:36.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.458+0000 7fe382e377c0 10 datascan.scan_links: step 2: handling object 200.00000000 2023-05-20T01:25:36.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.458+0000 7fe382e377c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2023-05-20T01:25:36.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.458+0000 7fe382e377c0 10 datascan.scan_links: step 2: handling object 200.00000001 2023-05-20T01:25:36.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.458+0000 7fe382e377c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2023-05-20T01:25:36.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.458+0000 7fe382e377c0 10 datascan.scan_links: step 2: handling object 500.00000000 2023-05-20T01:25:36.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.458+0000 7fe382e377c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2023-05-20T01:25:36.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.458+0000 7fe382e377c0 10 datascan.scan_links: step 2: handling object mds_snaptable 2023-05-20T01:25:36.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.458+0000 7fe382e377c0 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2023-05-20T01:25:36.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.458+0000 7fe382e377c0 10 datascan.scan_links: step 2: handling object mds0_inotable 2023-05-20T01:25:36.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.458+0000 7fe382e377c0 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2023-05-20T01:25:36.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.458+0000 7fe382e377c0 10 datascan.scan_links: step 2: handling object 100.00000000 2023-05-20T01:25:36.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.458+0000 7fe382e377c0 1 -- 172.21.15.16:0/674568391 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:69 12.13 12:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x563bb0f83c00 con 0x563bb0461800 2023-05-20T01:25:36.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.458+0000 7fe3805e4700 1 -- 172.21.15.16:0/674568391 <== osd.1 v2:172.21.15.16:6808/3860264220 10 ==== osd_op_reply(69 100.00000000 [omap-get-vals out=5b] v0'0 uv7 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x563bb0425440 con 0x563bb0461800 2023-05-20T01:25:36.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.458+0000 7fe382e377c0 10 datascan.scan_links: step 2: handling object mds0_sessionmap 2023-05-20T01:25:36.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.458+0000 7fe382e377c0 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2023-05-20T01:25:36.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.458+0000 7fe382e377c0 10 datascan.scan_links: step 2: handling object 400.00000000 2023-05-20T01:25:36.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.458+0000 7fe382e377c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2023-05-20T01:25:36.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.458+0000 7fe382e377c0 10 datascan.scan_links: step 2: handling object 100.00000000.inode 2023-05-20T01:25:36.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.458+0000 7fe382e377c0 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2023-05-20T01:25:36.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.458+0000 7fe382e377c0 10 datascan.scan_links: step 2: handling object 1.00000000 2023-05-20T01:25:36.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.458+0000 7fe382e377c0 1 -- 172.21.15.16:0/674568391 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:70 12.1f 12:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x563bb0f82700 con 0x563bb0460c00 2023-05-20T01:25:36.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.459+0000 7fe3815e6700 1 -- 172.21.15.16:0/674568391 <== osd.6 v2:172.21.15.132:6801/727581123 18 ==== osd_op_reply(70 1.00000000 [omap-get-vals out=508b] v0'0 uv12 ondisk = 0) v8 ==== 154+0+508 (crc 0 0 0) 0x563bb0425680 con 0x563bb0460c00 2023-05-20T01:25:36.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.459+0000 7fe382e377c0 20 datascan.scan_links: injected ino detected 2023-05-20T01:25:36.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.459+0000 7fe382e377c0 20 datascan.scan_links: adding 0x10000000000 for future processing to fix dnfirst 2023-05-20T01:25:36.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.459+0000 7fe382e377c0 10 datascan.scan_links: processing 0 dup_primaries, 0 remote_links 2023-05-20T01:25:36.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.459+0000 7fe382e377c0 10 datascan.scan_links: removing dup dentries from 0 objects 2023-05-20T01:25:36.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.459+0000 7fe382e377c0 10 datascan.scan_links: processing 0 bad_nlink_inos 2023-05-20T01:25:36.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.459+0000 7fe382e377c0 10 datascan.scan_links: processing 2 injected_inos 2023-05-20T01:25:36.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.459+0000 7fe382e377c0 10 datascan.scan_links: handling injected_ino 0x10000000000 2023-05-20T01:25:36.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.459+0000 7fe382e377c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x1:subdir 2023-05-20T01:25:36.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.459+0000 7fe382e377c0 1 -- 172.21.15.16:0/674568391 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:71 12.1f 12:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x563bb0f82a80 con 0x563bb0460c00 2023-05-20T01:25:36.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.459+0000 7fe3815e6700 1 -- 172.21.15.16:0/674568391 <== osd.6 v2:172.21.15.132:6801/727581123 19 ==== osd_op_reply(71 1.00000000 [omap-get-vals-by-keys out=507b] v0'0 uv12 ondisk = 0) v8 ==== 154+0+507 (crc 0 0 0) 0x563bb0425680 con 0x563bb0460c00 2023-05-20T01:25:36.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.459+0000 7fe382e377c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-05-20T01:25:36.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.459+0000 7fe382e377c0 20 datascan.scan_links: first is now 2 2023-05-20T01:25:36.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.459+0000 7fe382e377c0 1 -- 172.21.15.16:0/674568391 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:72 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-vals in=507b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x563bb0f83180 con 0x563bb0460c00 2023-05-20T01:25:36.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.460+0000 7fe3815e6700 1 -- 172.21.15.16:0/674568391 <== osd.6 v2:172.21.15.132:6801/727581123 20 ==== osd_op_reply(72 1.00000000 [omap-set-vals] v93'13 uv13 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x563bb0425680 con 0x563bb0460c00 2023-05-20T01:25:36.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.460+0000 7fe382e377c0 20 datascan.inject_linkage: Injected dentry 0x0x1/subdir pointing to 0x0x10000000000 2023-05-20T01:25:36.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.460+0000 7fe382e377c0 10 datascan.scan_links: handling injected_ino 0x10000000001 2023-05-20T01:25:36.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.460+0000 7fe382e377c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:sixmegs 2023-05-20T01:25:36.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.460+0000 7fe382e377c0 1 -- 172.21.15.16:0/674568391 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:73 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x563bb0f82e00 con 0x563bb0460800 2023-05-20T01:25:36.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.461+0000 7fe3815e6700 1 -- 172.21.15.16:0/674568391 <== osd.0 v2:172.21.15.16:6824/3923634179 17 ==== osd_op_reply(73 10000000000.00000000 [omap-get-vals-by-keys out=508b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+508 (crc 0 0 0) 0x563bb0425680 con 0x563bb0460800 2023-05-20T01:25:36.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.461+0000 7fe382e377c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=sixmegs frag=*, r=0 2023-05-20T01:25:36.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.461+0000 7fe382e377c0 20 datascan.scan_links: first is now 2 2023-05-20T01:25:36.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.461+0000 7fe382e377c0 1 -- 172.21.15.16:0/674568391 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:74 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=508b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x563bb0f83880 con 0x563bb0460800 2023-05-20T01:25:36.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.462+0000 7fe3815e6700 1 -- 172.21.15.16:0/674568391 <== osd.0 v2:172.21.15.16:6824/3923634179 18 ==== osd_op_reply(74 10000000000.00000000 [omap-set-vals] v93'18 uv18 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563bb0425680 con 0x563bb0460800 2023-05-20T01:25:36.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.462+0000 7fe382e377c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/sixmegs pointing to 0x0x10000000001 2023-05-20T01:25:36.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.462+0000 7fe382e377c0 10 datascan.scan_links: updating inotable 2023-05-20T01:25:36.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.462+0000 7fe382e377c0 1 -- 172.21.15.16:0/674568391 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:75 12.13 12:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x563bb0f82700 con 0x563bb0461800 2023-05-20T01:25:36.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.462+0000 7fe3805e4700 1 -- 172.21.15.16:0/674568391 <== osd.1 v2:172.21.15.16:6808/3860264220 11 ==== osd_op_reply(75 mds0_inotable [read 0~34 out=34b] v0'0 uv6 ondisk = 0) v8 ==== 157+0+34 (crc 0 0 0) 0x563bb0425440 con 0x563bb0461800 2023-05-20T01:25:36.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.462+0000 7fe382e377c0 10 mds.0.inotable: skip_inos was [0x10000000000~0x10000000000] 2023-05-20T01:25:36.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.462+0000 7fe382e377c0 10 mds.0.inotable: skip_inos now [0x10000000002~0xfffffffffe] 2023-05-20T01:25:36.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.462+0000 7fe382e377c0 1 -- 172.21.15.16:0/674568391 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:76 12.13 12:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x563bb0f82a80 con 0x563bb0461800 2023-05-20T01:25:36.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.463+0000 7fe3805e4700 1 -- 172.21.15.16:0/674568391 <== osd.1 v2:172.21.15.16:6808/3860264220 12 ==== osd_op_reply(76 mds0_inotable [writefull 0~34] v93'8 uv8 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x563bb0425440 con 0x563bb0461800 2023-05-20T01:25:36.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.463+0000 7fe382e377c0 10 datascan.scan_links: updating snaptable 2023-05-20T01:25:36.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.463+0000 7fe382e377c0 1 -- 172.21.15.16:0/674568391 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:77 12.d 12:b50e409b:::mds_snaptable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x563bb0f83500 con 0x563bb0582000 2023-05-20T01:25:36.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.464+0000 7fe3815e6700 1 -- 172.21.15.16:0/674568391 <== osd.7 v2:172.21.15.132:6800/466001149 3 ==== osd_op_reply(77 mds_snaptable [read 0~70 out=70b] v0'0 uv3 ondisk = 0) v8 ==== 157+0+70 (crc 0 0 0) 0x563bb0425680 con 0x563bb0582000 2023-05-20T01:25:36.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.464+0000 7fe382e377c0 1 -- 172.21.15.16:0/674568391 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x563bb0460800 msgr2=0x563bb0560000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:36.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.464+0000 7fe382e377c0 1 --2- 172.21.15.16:0/674568391 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x563bb0460800 0x563bb0560000 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:36.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.464+0000 7fe382e377c0 1 -- 172.21.15.16:0/674568391 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x563bb0461800 msgr2=0x563bb0560b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:36.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.464+0000 7fe382e377c0 1 --2- 172.21.15.16:0/674568391 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x563bb0461800 0x563bb0560b00 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:36.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.464+0000 7fe382e377c0 1 -- 172.21.15.16:0/674568391 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x563bb0461c00 msgr2=0x563bb0562680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:36.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.464+0000 7fe382e377c0 1 --2- 172.21.15.16:0/674568391 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x563bb0461c00 0x563bb0562680 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:36.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.464+0000 7fe382e377c0 1 -- 172.21.15.16:0/674568391 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x563bb0461000 msgr2=0x563bb0560580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:36.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.464+0000 7fe382e377c0 1 --2- 172.21.15.16:0/674568391 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x563bb0461000 0x563bb0560580 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:36.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.464+0000 7fe382e377c0 1 -- 172.21.15.16:0/674568391 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x563bb0430800 msgr2=0x563bb01b8100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:36.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.464+0000 7fe382e377c0 1 --2- 172.21.15.16:0/674568391 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x563bb0430800 0x563bb01b8100 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:36.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.464+0000 7fe380de5700 1 -- 172.21.15.16:0/674568391 reap_dead start 2023-05-20T01:25:36.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.464+0000 7fe382e377c0 1 -- 172.21.15.16:0/674568391 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x563bb0461400 msgr2=0x563bb0561b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:36.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.464+0000 7fe382e377c0 1 --2- 172.21.15.16:0/674568391 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x563bb0461400 0x563bb0561b80 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:36.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.464+0000 7fe380de5700 1 -- 172.21.15.16:0/674568391 reap_dead start 2023-05-20T01:25:36.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.464+0000 7fe382e377c0 1 -- 172.21.15.16:0/674568391 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x563bb0460c00 msgr2=0x563bb0561080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:36.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.464+0000 7fe382e377c0 1 --2- 172.21.15.16:0/674568391 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x563bb0460c00 0x563bb0561080 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:36.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.464+0000 7fe382e377c0 1 -- 172.21.15.16:0/674568391 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x563bb0582000 msgr2=0x563bb0562100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:36.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.464+0000 7fe382e377c0 1 --2- 172.21.15.16:0/674568391 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x563bb0582000 0x563bb0562100 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:36.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.464+0000 7fe380de5700 1 -- 172.21.15.16:0/674568391 reap_dead start 2023-05-20T01:25:36.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.464+0000 7fe382e377c0 1 -- 172.21.15.16:0/674568391 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x563bb0460400 msgr2=0x563bb01b9700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:36.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.464+0000 7fe382e377c0 1 --2- 172.21.15.16:0/674568391 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x563bb0460400 0x563bb01b9700 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x563bb043e0f0 tx=0x563bb0286420 comp rx=0 tx=0).stop 2023-05-20T01:25:36.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.464+0000 7fe382e377c0 1 -- 172.21.15.16:0/674568391 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563bb0430400 msgr2=0x563bb01b8680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:36.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.464+0000 7fe382e377c0 1 --2- 172.21.15.16:0/674568391 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563bb0430400 0x563bb01b8680 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x563bb043f3b0 tx=0x563bb0286300 comp rx=0 tx=0).stop 2023-05-20T01:25:36.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.464+0000 7fe382e377c0 1 -- 172.21.15.16:0/674568391 shutdown_connections 2023-05-20T01:25:36.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.464+0000 7fe382e377c0 1 --2- 172.21.15.16:0/674568391 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x563bb0460400 0x563bb01b9700 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:36.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.464+0000 7fe382e377c0 1 --2- 172.21.15.16:0/674568391 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563bb0430400 0x563bb01b8680 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:36.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.464+0000 7fe382e377c0 1 -- 172.21.15.16:0/674568391 >> 172.21.15.16:0/674568391 conn(0x563bb011dc00 msgr2=0x563bb01ed800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:36.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.464+0000 7fe382e377c0 1 -- 172.21.15.16:0/674568391 shutdown_connections 2023-05-20T01:25:36.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.464+0000 7fe382e377c0 1 -- 172.21.15.16:0/674568391 wait complete. 2023-05-20T01:25:36.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.465+0000 7fe382e377c0 1 -- 172.21.15.16:0/346080380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563bb011c800 msgr2=0x563bb01b7080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:36.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.465+0000 7fe382e377c0 1 --2- 172.21.15.16:0/346080380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563bb011c800 0x563bb01b7080 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x563bb027ce10 tx=0x563bb0286480 comp rx=0 tx=0).stop 2023-05-20T01:25:36.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.465+0000 7fe382e377c0 1 -- 172.21.15.16:0/346080380 shutdown_connections 2023-05-20T01:25:36.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.465+0000 7fe382e377c0 1 --2- 172.21.15.16:0/346080380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563bb011c800 0x563bb01b7080 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:36.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.465+0000 7fe382e377c0 1 --2- 172.21.15.16:0/346080380 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563bb011d000 0x563bb01b6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:36.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.465+0000 7fe382e377c0 1 --2- 172.21.15.16:0/346080380 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563bb011cc00 0x563bb01b6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:36.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.465+0000 7fe382e377c0 1 -- 172.21.15.16:0/346080380 >> 172.21.15.16:0/346080380 conn(0x563bb011d400 msgr2=0x563bb01ed000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:36.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.465+0000 7fe382e377c0 1 -- 172.21.15.16:0/346080380 shutdown_connections 2023-05-20T01:25:36.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.465+0000 7fe382e377c0 1 -- 172.21.15.16:0/346080380 wait complete. 2023-05-20T01:25:36.502 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_links'] in time 0:00:00.127576, result: 2023-05-20T01:25:36.502 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2023-05-20T01:25:36.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.658+0000 7f0995d15700 1 Processor -- start 2023-05-20T01:25:36.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.658+0000 7f0995d15700 1 -- start start 2023-05-20T01:25:36.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.658+0000 7f0995d15700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f09900cc9c0 0x7f099006a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:36.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.659+0000 7f0995d15700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0990072170 0x7f099006aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:36.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.659+0000 7f0995d15700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09900ccf40 con 0x7f09901585d0 2023-05-20T01:25:36.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.659+0000 7f0995d15700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f099015e540 con 0x7f09900cc9c0 2023-05-20T01:25:36.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.659+0000 7f0995d15700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09900cd220 con 0x7f0990072170 2023-05-20T01:25:36.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.659+0000 7f098effd700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f09900cc9c0 0x7f099006a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:36.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.659+0000 7f098f7fe700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f09901585d0 0x7f099015ae20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.16:58330/0 (socket says 172.21.15.16:58330) 2023-05-20T01:25:36.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.659+0000 7f098f7fe700 1 -- 172.21.15.16:0/3504528222 learned_addr learned my addr 172.21.15.16:0/3504528222 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:25:36.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.659+0000 7f098ffff700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0990072170 0x7f099006aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:36.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.659+0000 7f098e7fc700 1 -- 172.21.15.16:0/3504528222 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1082985825 0 0) 0x7f09900ccf40 con 0x7f09901585d0 2023-05-20T01:25:36.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.659+0000 7f098e7fc700 1 -- 172.21.15.16:0/3504528222 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09780035b0 con 0x7f09901585d0 2023-05-20T01:25:36.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.659+0000 7f098ffff700 1 -- 172.21.15.16:0/3504528222 >> v2:172.21.15.132:3300/0 conn(0x7f09900cc9c0 msgr2=0x7f099006a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:36.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.659+0000 7f098ffff700 1 --2- 172.21.15.16:0/3504528222 >> v2:172.21.15.132:3300/0 conn(0x7f09900cc9c0 0x7f099006a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:36.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.659+0000 7f098ffff700 1 -- 172.21.15.16:0/3504528222 >> v1:172.21.15.132:6789/0 conn(0x7f09901585d0 legacy=0x7f099015ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:36.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.659+0000 7f098ffff700 1 -- 172.21.15.16:0/3504528222 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09900cd500 con 0x7f0990072170 2023-05-20T01:25:36.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.659+0000 7f098effd700 1 --2- 172.21.15.16:0/3504528222 >> v2:172.21.15.132:3300/0 conn(0x7f09900cc9c0 0x7f099006a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-05-20T01:25:36.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.659+0000 7f098ffff700 1 --2- 172.21.15.16:0/3504528222 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0990072170 0x7f099006aa50 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f0988009d90 tx=0x7f098800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0dc377bb00eee64 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:36.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.660+0000 7f098e7fc700 1 -- 172.21.15.16:0/3504528222 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f098800e070 con 0x7f0990072170 2023-05-20T01:25:36.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.660+0000 7f098e7fc700 1 -- 172.21.15.16:0/3504528222 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f098800bb20 con 0x7f0990072170 2023-05-20T01:25:36.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.660+0000 7f098e7fc700 1 -- 172.21.15.16:0/3504528222 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0988012430 con 0x7f0990072170 2023-05-20T01:25:36.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.660+0000 7f0995d15700 1 -- 172.21.15.16:0/3504528222 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0990072170 msgr2=0x7f099006aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:36.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.660+0000 7f0995d15700 1 --2- 172.21.15.16:0/3504528222 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0990072170 0x7f099006aa50 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f0988009d90 tx=0x7f098800b330 comp rx=0 tx=0).stop 2023-05-20T01:25:36.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.660+0000 7f0995d15700 1 -- 172.21.15.16:0/3504528222 shutdown_connections 2023-05-20T01:25:36.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.660+0000 7f0995d15700 1 --2- 172.21.15.16:0/3504528222 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0990072170 0x7f099006aa50 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:36.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.660+0000 7f0995d15700 1 --2- 172.21.15.16:0/3504528222 >> v2:172.21.15.132:3300/0 conn(0x7f09900cc9c0 0x7f099006a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:36.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.660+0000 7f0995d15700 1 -- 172.21.15.16:0/3504528222 >> 172.21.15.16:0/3504528222 conn(0x7f09900cffb0 msgr2=0x7f09900d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:36.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.660+0000 7f0995d15700 1 -- 172.21.15.16:0/3504528222 shutdown_connections 2023-05-20T01:25:36.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.660+0000 7f0995d15700 1 -- 172.21.15.16:0/3504528222 wait complete. 2023-05-20T01:25:36.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.661+0000 7f0995d15700 1 Processor -- start 2023-05-20T01:25:36.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.662+0000 7f0995d15700 1 -- start start 2023-05-20T01:25:36.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.662+0000 7f0995d15700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0990072170 0x7f0990155370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:36.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.662+0000 7f0995d15700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09900cc9c0 0x7f09900ad280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:36.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.662+0000 7f0995d15700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f09901585d0 0x7f09900b5010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:36.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.662+0000 7f0995d15700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09900adbe0 con 0x7f09900cc9c0 2023-05-20T01:25:36.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.662+0000 7f0995d15700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09900ad7c0 con 0x7f0990072170 2023-05-20T01:25:36.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.662+0000 7f0995d15700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09900ade80 con 0x7f09901585d0 2023-05-20T01:25:36.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.662+0000 7f098effd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09900cc9c0 0x7f09900ad280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:36.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.662+0000 7f098f7fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0990072170 0x7f0990155370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:36.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.662+0000 7f098ffff700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f09901585d0 0x7f09900b5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:36.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.662+0000 7f098effd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09900cc9c0 0x7f09900ad280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37098/0 (socket says 172.21.15.16:37098) 2023-05-20T01:25:36.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.662+0000 7f098f7fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0990072170 0x7f0990155370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.16:34116/0 (socket says 172.21.15.16:34116) 2023-05-20T01:25:36.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.663+0000 7f098effd700 1 -- 172.21.15.16:0/603559642 learned_addr learned my addr 172.21.15.16:0/603559642 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:36.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.663+0000 7f098ffff700 1 -- 172.21.15.16:0/603559642 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0990072170 msgr2=0x7f0990155370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:36.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.663+0000 7f098ffff700 1 --2- 172.21.15.16:0/603559642 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0990072170 0x7f0990155370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:36.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.663+0000 7f098ffff700 1 -- 172.21.15.16:0/603559642 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09900cc9c0 msgr2=0x7f09900ad280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:36.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.663+0000 7f098ffff700 1 --2- 172.21.15.16:0/603559642 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09900cc9c0 0x7f09900ad280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:36.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.663+0000 7f098ffff700 1 -- 172.21.15.16:0/603559642 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09840094a0 con 0x7f09901585d0 2023-05-20T01:25:36.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.663+0000 7f098ffff700 1 --2- 172.21.15.16:0/603559642 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f09901585d0 0x7f09900b5010 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f0988016000 tx=0x7f0988004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:36.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.663+0000 7f0977fff700 1 -- 172.21.15.16:0/603559642 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0988012430 con 0x7f09901585d0 2023-05-20T01:25:36.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.663+0000 7f0977fff700 1 -- 172.21.15.16:0/603559642 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0988020070 con 0x7f09901585d0 2023-05-20T01:25:36.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.663+0000 7f0995d15700 1 -- 172.21.15.16:0/603559642 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0988009a70 con 0x7f09901585d0 2023-05-20T01:25:36.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.663+0000 7f0977fff700 1 -- 172.21.15.16:0/603559642 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09880098f0 con 0x7f09901585d0 2023-05-20T01:25:36.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.663+0000 7f0995d15700 1 -- 172.21.15.16:0/603559642 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09900719a0 con 0x7f09901585d0 2023-05-20T01:25:36.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.664+0000 7f0995d15700 1 -- 172.21.15.16:0/603559642 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09900cd7e0 con 0x7f09901585d0 2023-05-20T01:25:36.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.665+0000 7f0977fff700 1 -- 172.21.15.16:0/603559642 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0988004040 con 0x7f09901585d0 2023-05-20T01:25:36.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.665+0000 7f0977fff700 1 --2- 172.21.15.16:0/603559642 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f097c06ace0 0x7f097c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:36.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.665+0000 7f0977fff700 1 -- 172.21.15.16:0/603559642 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6958+0+0 (secure 0 0 0) 0x7f0988022070 con 0x7f09901585d0 2023-05-20T01:25:36.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.665+0000 7f098f7fe700 1 --2- 172.21.15.16:0/603559642 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f097c06ace0 0x7f097c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:36.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.666+0000 7f098f7fe700 1 --2- 172.21.15.16:0/603559642 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f097c06ace0 0x7f097c06d190 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f0980007fd0 tx=0x7f0980007dc0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:36.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.670+0000 7f0977fff700 1 -- 172.21.15.16:0/603559642 <== mon.2 v2:172.21.15.132:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f0988056450 con 0x7f09901585d0 2023-05-20T01:25:36.816 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:25:36.819 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:25:36.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:36.958+0000 7f0995d15700 1 -- 172.21.15.16:0/603559642 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) v1 -- 0x7f099015b530 con 0x7f09901585d0 2023-05-20T01:25:37.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:37.551+0000 7f0977fff700 1 -- 172.21.15.16:0/603559642 <== mon.2 v2:172.21.15.132:3301/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 4:0 v64) v1 ==== 101+0+0 (secure 0 0 0) 0x7f0988059a70 con 0x7f09901585d0 2023-05-20T01:25:37.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:37.554+0000 7f0995d15700 1 -- 172.21.15.16:0/603559642 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f097c06ace0 msgr2=0x7f097c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:37.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:37.554+0000 7f0995d15700 1 --2- 172.21.15.16:0/603559642 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f097c06ace0 0x7f097c06d190 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f0980007fd0 tx=0x7f0980007dc0 comp rx=0 tx=0).stop 2023-05-20T01:25:37.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:37.554+0000 7f0995d15700 1 -- 172.21.15.16:0/603559642 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f09901585d0 msgr2=0x7f09900b5010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:37.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:37.554+0000 7f0995d15700 1 --2- 172.21.15.16:0/603559642 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f09901585d0 0x7f09900b5010 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f0988016000 tx=0x7f0988004ad0 comp rx=0 tx=0).stop 2023-05-20T01:25:37.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:37.554+0000 7f0995d15700 1 -- 172.21.15.16:0/603559642 shutdown_connections 2023-05-20T01:25:37.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:37.554+0000 7f0995d15700 1 --2- 172.21.15.16:0/603559642 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f097c06ace0 0x7f097c06d190 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:37.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:37.554+0000 7f0995d15700 1 --2- 172.21.15.16:0/603559642 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f09901585d0 0x7f09900b5010 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:37.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:37.554+0000 7f0995d15700 1 --2- 172.21.15.16:0/603559642 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0990072170 0x7f0990155370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:37.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:37.554+0000 7f0995d15700 1 --2- 172.21.15.16:0/603559642 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09900cc9c0 0x7f09900ad280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:37.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:37.554+0000 7f0995d15700 1 -- 172.21.15.16:0/603559642 >> 172.21.15.16:0/603559642 conn(0x7f09900cffb0 msgr2=0x7f0990156180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:37.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:37.554+0000 7f0995d15700 1 -- 172.21.15.16:0/603559642 shutdown_connections 2023-05-20T01:25:37.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:37.554+0000 7f0995d15700 1 -- 172.21.15.16:0/603559642 wait complete. 2023-05-20T01:25:37.562 INFO:teuthology.orchestra.run.smithi016.stderr:repaired: restoring rank 4:0 2023-05-20T01:25:37.563 INFO:tasks.ceph.mds.c.smithi016.stderr:2023-05-20T01:25:37.559+0000 7ff569d14700 -1 mds.0.openfiles _load_finish got (2) No such file or directory 2023-05-20T01:25:37.575 INFO:tasks.ceph.mds.a:Restarting daemon 2023-05-20T01:25:37.576 INFO:tasks.ceph.mds.a:Stopping old one... 2023-05-20T01:25:37.576 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-05-20T01:25:37.576 INFO:teuthology.orchestra.run:waiting for 300 2023-05-20T01:25:37.577 INFO:tasks.ceph.mds.c:Restarting daemon 2023-05-20T01:25:37.577 INFO:tasks.ceph.mds.c:Stopping old one... 2023-05-20T01:25:37.577 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-05-20T01:25:37.577 INFO:teuthology.orchestra.run:waiting for 300 2023-05-20T01:25:37.578 INFO:tasks.ceph.mds.b:Restarting daemon 2023-05-20T01:25:37.578 INFO:tasks.ceph.mds.b:Stopping old one... 2023-05-20T01:25:37.578 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-05-20T01:25:37.578 INFO:teuthology.orchestra.run:waiting for 300 2023-05-20T01:25:37.578 INFO:tasks.ceph.mds.d:Restarting daemon 2023-05-20T01:25:37.579 INFO:tasks.ceph.mds.d:Stopping old one... 2023-05-20T01:25:37.579 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-05-20T01:25:37.579 INFO:teuthology.orchestra.run:waiting for 300 2023-05-20T01:25:37.586 INFO:tasks.ceph.mds.a:Stopped 2023-05-20T01:25:37.586 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-05-20T01:25:37.587 INFO:tasks.ceph.mds.c:Stopped 2023-05-20T01:25:37.588 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-05-20T01:25:37.589 INFO:tasks.ceph.mds.b:Stopped 2023-05-20T01:25:37.589 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-05-20T01:25:37.590 INFO:tasks.ceph.mds.d:Stopped 2023-05-20T01:25:37.590 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-05-20T01:25:37.630 INFO:tasks.ceph.mds.a:Started 2023-05-20T01:25:37.630 DEBUG:teuthology.parallel:result is None 2023-05-20T01:25:37.631 INFO:tasks.ceph.mds.c:Started 2023-05-20T01:25:37.632 DEBUG:teuthology.parallel:result is None 2023-05-20T01:25:37.632 INFO:tasks.ceph.mds.b:Started 2023-05-20T01:25:37.633 DEBUG:teuthology.parallel:result is None 2023-05-20T01:25:37.633 INFO:tasks.ceph.mds.d:Started 2023-05-20T01:25:37.634 DEBUG:teuthology.parallel:result is None 2023-05-20T01:25:37.634 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-20T01:25:37.739 INFO:tasks.ceph.mds.a.smithi016.stdout:starting mds.a at 2023-05-20T01:25:37.739 INFO:tasks.ceph.mds.c.smithi016.stdout:starting mds.c at 2023-05-20T01:25:37.740 INFO:tasks.ceph.mds.d.smithi132.stdout:starting mds.d at 2023-05-20T01:25:37.740 INFO:tasks.ceph.mds.b.smithi132.stdout:starting mds.b at 2023-05-20T01:25:37.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:37.788+0000 7f5c70754700 1 Processor -- start 2023-05-20T01:25:37.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:37.788+0000 7f5c70754700 1 -- start start 2023-05-20T01:25:37.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:37.789+0000 7f5c70754700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5c6806f340 0x7f5c6806f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:37.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:37.789+0000 7f5c70754700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5c6806a510 0x7f5c6806a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:37.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:37.789+0000 7f5c70754700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c680cf560 con 0x7f5c68161230 2023-05-20T01:25:37.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:37.789+0000 7f5c70754700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c6806aed0 con 0x7f5c6806a510 2023-05-20T01:25:37.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:37.789+0000 7f5c70754700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c68161600 con 0x7f5c6806f340 2023-05-20T01:25:37.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:37.789+0000 7f5c6ecf1700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5c6806a510 0x7f5c6806a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:37.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:37.789+0000 7f5c6ecf1700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5c6806a510 0x7f5c6806a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37140/0 (socket says 172.21.15.16:37140) 2023-05-20T01:25:37.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:37.789+0000 7f5c6ecf1700 1 -- 172.21.15.16:0/962164237 learned_addr learned my addr 172.21.15.16:0/962164237 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:37.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:37.789+0000 7f5c6dcef700 1 --2- 172.21.15.16:0/962164237 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5c6806f340 0x7f5c6806f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:37.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:37.790+0000 7f5c6ecf1700 1 -- 172.21.15.16:0/962164237 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5c6806f340 msgr2=0x7f5c6806f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:37.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:37.790+0000 7f5c6ecf1700 1 --2- 172.21.15.16:0/962164237 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5c6806f340 0x7f5c6806f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:37.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:37.790+0000 7f5c6ecf1700 1 -- 172.21.15.16:0/962164237 >> v1:172.21.15.132:6789/0 conn(0x7f5c68161230 legacy=0x7f5c6806d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:37.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:37.790+0000 7f5c6ecf1700 1 -- 172.21.15.16:0/962164237 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c680cfd90 con 0x7f5c6806a510 2023-05-20T01:25:37.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:37.790+0000 7f5c6ecf1700 1 --2- 172.21.15.16:0/962164237 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c6806a510 0x7f5c6806a990 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f5c640098f0 tx=0x7f5c64009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7577413efaddd9f2 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:37.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:37.790+0000 7f5c6d4ee700 1 -- 172.21.15.16:0/962164237 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c64011830 con 0x7f5c6806a510 2023-05-20T01:25:37.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:37.790+0000 7f5c6d4ee700 1 -- 172.21.15.16:0/962164237 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c64011990 con 0x7f5c6806a510 2023-05-20T01:25:37.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:37.790+0000 7f5c6d4ee700 1 -- 172.21.15.16:0/962164237 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c64011d20 con 0x7f5c6806a510 2023-05-20T01:25:37.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:37.791+0000 7f5c70754700 1 -- 172.21.15.16:0/962164237 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c6806a510 msgr2=0x7f5c6806a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:37.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:37.791+0000 7f5c70754700 1 --2- 172.21.15.16:0/962164237 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c6806a510 0x7f5c6806a990 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f5c640098f0 tx=0x7f5c64009c00 comp rx=0 tx=0).stop 2023-05-20T01:25:37.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:37.792+0000 7f5c70754700 1 -- 172.21.15.16:0/962164237 shutdown_connections 2023-05-20T01:25:37.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:37.792+0000 7f5c70754700 1 --2- 172.21.15.16:0/962164237 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c6806a510 0x7f5c6806a990 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:37.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:37.792+0000 7f5c70754700 1 --2- 172.21.15.16:0/962164237 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5c6806f340 0x7f5c6806f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:37.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:37.792+0000 7f5c70754700 1 -- 172.21.15.16:0/962164237 >> 172.21.15.16:0/962164237 conn(0x7f5c680d0c30 msgr2=0x7f5c680b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:37.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:37.792+0000 7f5c70754700 1 -- 172.21.15.16:0/962164237 shutdown_connections 2023-05-20T01:25:37.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:37.792+0000 7f5c70754700 1 -- 172.21.15.16:0/962164237 wait complete. 2023-05-20T01:25:37.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:37.793+0000 7f5c70754700 1 Processor -- start 2023-05-20T01:25:37.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:37.793+0000 7f5c70754700 1 -- start start 2023-05-20T01:25:37.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:37.794+0000 7f5c70754700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c6806a510 0x7f5c680b5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:37.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:37.794+0000 7f5c70754700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5c6806f340 0x7f5c680c7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:37.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:37.794+0000 7f5c70754700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5c68161230 0x7f5c68155370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:37.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:37.794+0000 7f5c70754700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c680c75c0 con 0x7f5c6806a510 2023-05-20T01:25:37.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:37.794+0000 7f5c70754700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c681558b0 con 0x7f5c6806f340 2023-05-20T01:25:37.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:37.794+0000 7f5c70754700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c680c7870 con 0x7f5c68161230 2023-05-20T01:25:37.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:37.794+0000 7f5c6e4f0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c6806a510 0x7f5c680b5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:37.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:37.794+0000 7f5c6e4f0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c6806a510 0x7f5c680b5210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37144/0 (socket says 172.21.15.16:37144) 2023-05-20T01:25:37.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:37.794+0000 7f5c6e4f0700 1 -- 172.21.15.16:0/1603891308 learned_addr learned my addr 172.21.15.16:0/1603891308 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:37.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:37.794+0000 7f5c6ecf1700 1 --2- 172.21.15.16:0/1603891308 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5c68161230 0x7f5c68155370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:37.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:37.794+0000 7f5c6dcef700 1 --2- 172.21.15.16:0/1603891308 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5c6806f340 0x7f5c680c7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:37.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:37.794+0000 7f5c6e4f0700 1 -- 172.21.15.16:0/1603891308 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5c68161230 msgr2=0x7f5c68155370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:37.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:37.794+0000 7f5c6e4f0700 1 --2- 172.21.15.16:0/1603891308 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5c68161230 0x7f5c68155370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:37.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:37.794+0000 7f5c6e4f0700 1 -- 172.21.15.16:0/1603891308 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5c6806f340 msgr2=0x7f5c680c7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:37.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:37.794+0000 7f5c6e4f0700 1 --2- 172.21.15.16:0/1603891308 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5c6806f340 0x7f5c680c7080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:37.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:37.794+0000 7f5c6e4f0700 1 -- 172.21.15.16:0/1603891308 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c64009550 con 0x7f5c6806a510 2023-05-20T01:25:37.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:37.794+0000 7f5c6e4f0700 1 --2- 172.21.15.16:0/1603891308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c6806a510 0x7f5c680b5210 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f5c60001ed0 tx=0x7f5c600021e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:37.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:37.794+0000 7f5c5effd700 1 -- 172.21.15.16:0/1603891308 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c60003c00 con 0x7f5c6806a510 2023-05-20T01:25:37.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:37.795+0000 7f5c5effd700 1 -- 172.21.15.16:0/1603891308 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c60003d60 con 0x7f5c6806a510 2023-05-20T01:25:37.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:37.795+0000 7f5c70754700 1 -- 172.21.15.16:0/1603891308 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c680cfab0 con 0x7f5c6806a510 2023-05-20T01:25:37.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:37.795+0000 7f5c5effd700 1 -- 172.21.15.16:0/1603891308 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c6000ab40 con 0x7f5c6806a510 2023-05-20T01:25:37.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:37.795+0000 7f5c70754700 1 -- 172.21.15.16:0/1603891308 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c680cf7d0 con 0x7f5c6806a510 2023-05-20T01:25:37.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:37.796+0000 7f5c70754700 1 -- 172.21.15.16:0/1603891308 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c680cdc20 con 0x7f5c6806a510 2023-05-20T01:25:37.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:37.796+0000 7f5c5effd700 1 -- 172.21.15.16:0/1603891308 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5c6000aca0 con 0x7f5c6806a510 2023-05-20T01:25:37.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:37.796+0000 7f5c5effd700 1 --2- 172.21.15.16:0/1603891308 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f5c4c06ace0 0x7f5c4c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:37.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:37.796+0000 7f5c5effd700 1 -- 172.21.15.16:0/1603891308 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 7044+0+0 (secure 0 0 0) 0x7f5c600898d0 con 0x7f5c6806a510 2023-05-20T01:25:37.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:37.797+0000 7f5c6dcef700 1 --2- 172.21.15.16:0/1603891308 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f5c4c06ace0 0x7f5c4c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:37.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:37.797+0000 7f5c6dcef700 1 --2- 172.21.15.16:0/1603891308 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f5c4c06ace0 0x7f5c4c06d190 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f5c6806ead0 tx=0x7f5c58009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:37.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:37.801+0000 7f5c5effd700 1 -- 172.21.15.16:0/1603891308 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f5c600545a0 con 0x7f5c6806a510 2023-05-20T01:25:38.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:38.043+0000 7f5c70754700 1 -- 172.21.15.16:0/1603891308 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5c680c3f80 con 0x7f5c6806a510 2023-05-20T01:25:38.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:38.043+0000 7f5c5effd700 1 -- 172.21.15.16:0/1603891308 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+4683 (secure 0 0 0) 0x7f5c60057bc0 con 0x7f5c6806a510 2023-05-20T01:25:38.048 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:25:38.048 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":65,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6077,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/1241894038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":1241894038},{"type":"v1","addr":"172.21.15.132:6835","nonce":1241894038}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":6562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/3771738903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3771738903},{"type":"v1","addr":"172.21.15.132:6834","nonce":3771738903}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":7101,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/3876038711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":3876038711},{"type":"v1","addr":"172.21.15.16:6836","nonce":3876038711}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-20T01:25:32.295399+0000","modified":"2023-05-20T01:25:37.552270+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":93,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7410},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7410":{"gid":7410,"name":"c","rank":0,"incarnation":65,"state":"up:replay","state_seq":1,"addr":"172.21.15.16:6837/2740585996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2740585996},{"type":"v1","addr":"172.21.15.16:6837","nonce":2740585996}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13,14],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-05-20T01:25:38.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:38.046+0000 7f5c70754700 1 -- 172.21.15.16:0/1603891308 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f5c4c06ace0 msgr2=0x7f5c4c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:38.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:38.046+0000 7f5c70754700 1 --2- 172.21.15.16:0/1603891308 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f5c4c06ace0 0x7f5c4c06d190 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f5c6806ead0 tx=0x7f5c58009210 comp rx=0 tx=0).stop 2023-05-20T01:25:38.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:38.046+0000 7f5c70754700 1 -- 172.21.15.16:0/1603891308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c6806a510 msgr2=0x7f5c680b5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:38.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:38.046+0000 7f5c70754700 1 --2- 172.21.15.16:0/1603891308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c6806a510 0x7f5c680b5210 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f5c60001ed0 tx=0x7f5c600021e0 comp rx=0 tx=0).stop 2023-05-20T01:25:38.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:38.046+0000 7f5c70754700 1 -- 172.21.15.16:0/1603891308 shutdown_connections 2023-05-20T01:25:38.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:38.046+0000 7f5c70754700 1 --2- 172.21.15.16:0/1603891308 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f5c4c06ace0 0x7f5c4c06d190 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:38.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:38.046+0000 7f5c70754700 1 --2- 172.21.15.16:0/1603891308 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5c68161230 0x7f5c68155370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:38.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:38.046+0000 7f5c70754700 1 --2- 172.21.15.16:0/1603891308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c6806a510 0x7f5c680b5210 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:38.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:38.046+0000 7f5c70754700 1 --2- 172.21.15.16:0/1603891308 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5c6806f340 0x7f5c680c7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:38.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:38.046+0000 7f5c70754700 1 -- 172.21.15.16:0/1603891308 >> 172.21.15.16:0/1603891308 conn(0x7f5c680d0c30 msgr2=0x7f5c680c30a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:38.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:38.046+0000 7f5c70754700 1 -- 172.21.15.16:0/1603891308 shutdown_connections 2023-05-20T01:25:38.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:38.046+0000 7f5c70754700 1 -- 172.21.15.16:0/1603891308 wait complete. 2023-05-20T01:25:38.052 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 65 2023-05-20T01:25:38.465 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 65, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-05-20T01:25:32.295399+0000', 'modified': '2023-05-20T01:25:37.552270+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 93, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7410}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7410': {'gid': 7410, 'name': 'c', 'rank': 0, 'incarnation': 65, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.16:6837/2740585996', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6834', 'nonce': 2740585996}, {'type': 'v1', 'addr': '172.21.15.16:6837', 'nonce': 2740585996}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [13, 14], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-05-20T01:25:38.465 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7410:up:replay 2023-05-20T01:25:38.559 INFO:tasks.ceph.mon.a.smithi016.stderr:2023-05-20T01:25:38.555+0000 7f49eb8ae700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-05-20T01:25:38.572 INFO:tasks.ceph.mds.c.smithi016.stderr:2023-05-20T01:25:38.568+0000 7fd0cbb5b700 -1 mds.0.openfiles _load_finish got (2) No such file or directory 2023-05-20T01:25:39.466 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-20T01:25:39.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:39.603+0000 7f51a339f700 1 Processor -- start 2023-05-20T01:25:39.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:39.603+0000 7f51a339f700 1 -- start start 2023-05-20T01:25:39.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:39.604+0000 7f51a339f700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f519c15cdb0 0x7f519c0cedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:39.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:39.604+0000 7f51a339f700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f519c0cd460 0x7f519c0cf310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:39.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:39.604+0000 7f51a339f700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f519c06b620 con 0x7f519c0ccdc0 2023-05-20T01:25:39.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:39.604+0000 7f51a339f700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f519c0b2800 con 0x7f519c15cdb0 2023-05-20T01:25:39.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:39.604+0000 7f51a339f700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f519c15d180 con 0x7f519c0cd460 2023-05-20T01:25:39.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:39.604+0000 7f51a2b9e700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f519c0ccdc0 0x7f519c0cb520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:48700/0 (socket says 172.21.15.16:48700) 2023-05-20T01:25:39.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:39.604+0000 7f51a239d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f519c15cdb0 0x7f519c0cedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:39.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:39.604+0000 7f51a2b9e700 1 -- 172.21.15.16:0/2466160318 learned_addr learned my addr 172.21.15.16:0/2466160318 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:25:39.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:39.605+0000 7f51a1b9c700 1 --2- 172.21.15.16:0/2466160318 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f519c0cd460 0x7f519c0cf310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:39.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:39.605+0000 7f51a239d700 1 -- 172.21.15.16:0/2466160318 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f519c0cd460 msgr2=0x7f519c0cf310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:39.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:39.605+0000 7f51a239d700 1 --2- 172.21.15.16:0/2466160318 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f519c0cd460 0x7f519c0cf310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:39.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:39.605+0000 7f51a239d700 1 -- 172.21.15.16:0/2466160318 >> v1:172.21.15.16:6789/0 conn(0x7f519c0ccdc0 legacy=0x7f519c0cb520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:39.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:39.605+0000 7f51a239d700 1 -- 172.21.15.16:0/2466160318 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f519c0cca30 con 0x7f519c15cdb0 2023-05-20T01:25:39.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:39.605+0000 7f51a139b700 1 -- 172.21.15.16:0/2466160318 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2818191784 0 0) 0x7f519c06b620 con 0x7f519c0ccdc0 2023-05-20T01:25:39.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:39.605+0000 7f51a1b9c700 1 --2- 172.21.15.16:0/2466160318 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f519c0cd460 0x7f519c0cf310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:25:39.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:39.605+0000 7f51a239d700 1 --2- 172.21.15.16:0/2466160318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f519c15cdb0 0x7f519c0cedd0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f518c009840 tx=0x7f518c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf34be7962baa0a7 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:39.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:39.605+0000 7f51a139b700 1 -- 172.21.15.16:0/2466160318 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f518c011830 con 0x7f519c15cdb0 2023-05-20T01:25:39.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:39.605+0000 7f51a139b700 1 -- 172.21.15.16:0/2466160318 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f518c011990 con 0x7f519c15cdb0 2023-05-20T01:25:39.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:39.605+0000 7f51a139b700 1 -- 172.21.15.16:0/2466160318 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f518c011d20 con 0x7f519c15cdb0 2023-05-20T01:25:39.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:39.606+0000 7f51a339f700 1 -- 172.21.15.16:0/2466160318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f519c15cdb0 msgr2=0x7f519c0cedd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:39.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:39.606+0000 7f51a339f700 1 --2- 172.21.15.16:0/2466160318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f519c15cdb0 0x7f519c0cedd0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f518c009840 tx=0x7f518c009b50 comp rx=0 tx=0).stop 2023-05-20T01:25:39.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:39.606+0000 7f51a339f700 1 -- 172.21.15.16:0/2466160318 shutdown_connections 2023-05-20T01:25:39.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:39.606+0000 7f51a339f700 1 --2- 172.21.15.16:0/2466160318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f519c15cdb0 0x7f519c0cedd0 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:39.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:39.606+0000 7f51a339f700 1 --2- 172.21.15.16:0/2466160318 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f519c0cd460 0x7f519c0cf310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:39.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:39.606+0000 7f51a339f700 1 -- 172.21.15.16:0/2466160318 >> 172.21.15.16:0/2466160318 conn(0x7f519c09a200 msgr2=0x7f519c0d0ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:39.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:39.606+0000 7f51a339f700 1 -- 172.21.15.16:0/2466160318 shutdown_connections 2023-05-20T01:25:39.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:39.606+0000 7f51a339f700 1 -- 172.21.15.16:0/2466160318 wait complete. 2023-05-20T01:25:39.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:39.608+0000 7f51a339f700 1 Processor -- start 2023-05-20T01:25:39.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:39.608+0000 7f51a339f700 1 -- start start 2023-05-20T01:25:39.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:39.609+0000 7f51a339f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f519c0ccdc0 0x7f519c0c4b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:39.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:39.609+0000 7f51a339f700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f519c15cdb0 0x7f519c0c7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:39.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:39.609+0000 7f51a339f700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f519c15ba90 0x7f519c14ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:39.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:39.609+0000 7f51a339f700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f519c0c7d90 con 0x7f519c0ccdc0 2023-05-20T01:25:39.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:39.609+0000 7f51a339f700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f519c1f5f50 con 0x7f519c15ba90 2023-05-20T01:25:39.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:39.609+0000 7f51a339f700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f519c0c7b40 con 0x7f519c15cdb0 2023-05-20T01:25:39.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:39.609+0000 7f51a239d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f519c0ccdc0 0x7f519c0c4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:39.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:39.609+0000 7f51a239d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f519c0ccdc0 0x7f519c0c4b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37174/0 (socket says 172.21.15.16:37174) 2023-05-20T01:25:39.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:39.609+0000 7f51a239d700 1 -- 172.21.15.16:0/763080531 learned_addr learned my addr 172.21.15.16:0/763080531 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:39.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:39.609+0000 7f51a1b9c700 1 --2- 172.21.15.16:0/763080531 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f519c15cdb0 0x7f519c0c7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:39.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:39.609+0000 7f51a2b9e700 1 --2- 172.21.15.16:0/763080531 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f519c15ba90 0x7f519c14ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:39.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:39.609+0000 7f51a239d700 1 -- 172.21.15.16:0/763080531 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f519c15cdb0 msgr2=0x7f519c0c7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:39.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:39.609+0000 7f51a239d700 1 --2- 172.21.15.16:0/763080531 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f519c15cdb0 0x7f519c0c7600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:39.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:39.609+0000 7f51a239d700 1 -- 172.21.15.16:0/763080531 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f519c15ba90 msgr2=0x7f519c14ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:39.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:39.609+0000 7f51a239d700 1 --2- 172.21.15.16:0/763080531 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f519c15ba90 0x7f519c14ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:39.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:39.609+0000 7f51a239d700 1 -- 172.21.15.16:0/763080531 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f518c0094a0 con 0x7f519c0ccdc0 2023-05-20T01:25:39.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:39.609+0000 7f51a2b9e700 1 --2- 172.21.15.16:0/763080531 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f519c15ba90 0x7f519c14ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:25:39.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:39.609+0000 7f51a239d700 1 --2- 172.21.15.16:0/763080531 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f519c0ccdc0 0x7f519c0c4b20 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f518c0090b0 tx=0x7f518c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:39.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:39.609+0000 7f5192ffd700 1 -- 172.21.15.16:0/763080531 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f518c0045d0 con 0x7f519c0ccdc0 2023-05-20T01:25:39.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:39.610+0000 7f5192ffd700 1 -- 172.21.15.16:0/763080531 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f518c01f070 con 0x7f519c0ccdc0 2023-05-20T01:25:39.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:39.610+0000 7f51a339f700 1 -- 172.21.15.16:0/763080531 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f519c06bb50 con 0x7f519c0ccdc0 2023-05-20T01:25:39.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:39.610+0000 7f5192ffd700 1 -- 172.21.15.16:0/763080531 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f518c0038c0 con 0x7f519c0ccdc0 2023-05-20T01:25:39.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:39.610+0000 7f51a339f700 1 -- 172.21.15.16:0/763080531 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f519c06b870 con 0x7f519c0ccdc0 2023-05-20T01:25:39.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:39.611+0000 7f51a339f700 1 -- 172.21.15.16:0/763080531 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f519c06be30 con 0x7f519c0ccdc0 2023-05-20T01:25:39.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:39.611+0000 7f5192ffd700 1 -- 172.21.15.16:0/763080531 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f518c003a20 con 0x7f519c0ccdc0 2023-05-20T01:25:39.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:39.612+0000 7f5192ffd700 1 --2- 172.21.15.16:0/763080531 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f518006ace0 0x7f518006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:39.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:39.612+0000 7f5192ffd700 1 -- 172.21.15.16:0/763080531 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 7044+0+0 (secure 0 0 0) 0x7f518c021070 con 0x7f519c0ccdc0 2023-05-20T01:25:39.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:39.612+0000 7f51a1b9c700 1 --2- 172.21.15.16:0/763080531 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f518006ace0 0x7f518006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:39.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:39.612+0000 7f51a1b9c700 1 --2- 172.21.15.16:0/763080531 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f518006ace0 0x7f518006d190 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f519c070050 tx=0x7f5198008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:39.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:39.616+0000 7f5192ffd700 1 -- 172.21.15.16:0/763080531 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f518c055e50 con 0x7f519c0ccdc0 2023-05-20T01:25:39.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:39.861+0000 7f51a339f700 1 -- 172.21.15.16:0/763080531 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f519c062120 con 0x7f519c0ccdc0 2023-05-20T01:25:39.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:39.861+0000 7f5192ffd700 1 -- 172.21.15.16:0/763080531 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+4674 (secure 0 0 0) 0x7f518c059470 con 0x7f519c0ccdc0 2023-05-20T01:25:39.866 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:25:39.866 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":68,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6323,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/59032067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":59032067},{"type":"v1","addr":"172.21.15.132:6834","nonce":59032067}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66},{"gid":6329,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/100162236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":100162236},{"type":"v1","addr":"172.21.15.132:6835","nonce":100162236}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66},{"gid":7572,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/3753560731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3753560731},{"type":"v1","addr":"172.21.15.16:6836","nonce":3753560731}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-20T01:25:32.295399+0000","modified":"2023-05-20T01:25:39.563588+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":95,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7575},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7575":{"gid":7575,"name":"c","rank":0,"incarnation":67,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.16:6837/307899603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":307899603},{"type":"v1","addr":"172.21.15.16:6837","nonce":307899603}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13,14],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-05-20T01:25:39.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:39.863+0000 7f51a339f700 1 -- 172.21.15.16:0/763080531 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f518006ace0 msgr2=0x7f518006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:39.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:39.863+0000 7f51a339f700 1 --2- 172.21.15.16:0/763080531 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f518006ace0 0x7f518006d190 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f519c070050 tx=0x7f5198008040 comp rx=0 tx=0).stop 2023-05-20T01:25:39.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:39.863+0000 7f51a339f700 1 -- 172.21.15.16:0/763080531 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f519c0ccdc0 msgr2=0x7f519c0c4b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:39.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:39.863+0000 7f51a339f700 1 --2- 172.21.15.16:0/763080531 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f519c0ccdc0 0x7f519c0c4b20 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f518c0090b0 tx=0x7f518c004750 comp rx=0 tx=0).stop 2023-05-20T01:25:39.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:39.863+0000 7f51a339f700 1 -- 172.21.15.16:0/763080531 shutdown_connections 2023-05-20T01:25:39.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:39.863+0000 7f51a339f700 1 --2- 172.21.15.16:0/763080531 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f519c15ba90 0x7f519c14ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:39.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:39.863+0000 7f51a339f700 1 --2- 172.21.15.16:0/763080531 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f519c0ccdc0 0x7f519c0c4b20 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:39.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:39.863+0000 7f51a339f700 1 --2- 172.21.15.16:0/763080531 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f518006ace0 0x7f518006d190 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:39.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:39.863+0000 7f51a339f700 1 --2- 172.21.15.16:0/763080531 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f519c15cdb0 0x7f519c0c7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:39.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:39.863+0000 7f51a339f700 1 -- 172.21.15.16:0/763080531 >> 172.21.15.16:0/763080531 conn(0x7f519c09a200 msgr2=0x7f519c1e5530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:39.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:39.863+0000 7f51a339f700 1 -- 172.21.15.16:0/763080531 shutdown_connections 2023-05-20T01:25:39.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:39.863+0000 7f51a339f700 1 -- 172.21.15.16:0/763080531 wait complete. 2023-05-20T01:25:39.870 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 68 2023-05-20T01:25:40.282 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 68, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-05-20T01:25:32.295399+0000', 'modified': '2023-05-20T01:25:39.563588+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 95, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7575}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7575': {'gid': 7575, 'name': 'c', 'rank': 0, 'incarnation': 67, 'state': 'up:reconnect', 'state_seq': 2, 'addr': '172.21.15.16:6837/307899603', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6835', 'nonce': 307899603}, {'type': 'v1', 'addr': '172.21.15.16:6837', 'nonce': 307899603}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [13, 14], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-05-20T01:25:40.283 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7575:up:reconnect 2023-05-20T01:25:41.284 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-20T01:25:41.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:41.419+0000 7ff6691a5700 1 Processor -- start 2023-05-20T01:25:41.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:41.419+0000 7ff6691a5700 1 -- start start 2023-05-20T01:25:41.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:41.420+0000 7ff6691a5700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff6640b2760 0x7ff6640d1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:41.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:41.420+0000 7ff6691a5700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7ff664144020 0x7ff6640b0980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:41.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:41.420+0000 7ff6691a5700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6640cc500 con 0x7ff6640cec60 2023-05-20T01:25:41.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:41.420+0000 7ff6691a5700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6640cb6e0 con 0x7ff664144020 2023-05-20T01:25:41.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:41.420+0000 7ff6691a5700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6641448a0 con 0x7ff6640b2760 2023-05-20T01:25:41.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:41.420+0000 7ff66359e700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ff6640cec60 0x7ff6640cc990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:48712/0 (socket says 172.21.15.16:48712) 2023-05-20T01:25:41.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:41.420+0000 7ff66359e700 1 -- 172.21.15.16:0/179916121 learned_addr learned my addr 172.21.15.16:0/179916121 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:25:41.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:41.420+0000 7ff66259c700 1 --2- 172.21.15.16:0/179916121 >> v2:172.21.15.132:3300/0 conn(0x7ff664144020 0x7ff6640b0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:41.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:41.421+0000 7ff662d9d700 1 --2- 172.21.15.16:0/179916121 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff6640b2760 0x7ff6640d1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:41.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:41.421+0000 7ff661d9b700 1 -- 172.21.15.16:0/179916121 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3918559187 0 0) 0x7ff6640cc500 con 0x7ff6640cec60 2023-05-20T01:25:41.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:41.421+0000 7ff661d9b700 1 -- 172.21.15.16:0/179916121 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff6500035b0 con 0x7ff6640cec60 2023-05-20T01:25:41.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:41.421+0000 7ff661d9b700 1 -- 172.21.15.16:0/179916121 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 192827236 0 0) 0x7ff6500035b0 con 0x7ff6640cec60 2023-05-20T01:25:41.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:41.421+0000 7ff661d9b700 1 -- 172.21.15.16:0/179916121 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff6640b2760 msgr2=0x7ff6640d1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:41.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:41.421+0000 7ff661d9b700 1 --2- 172.21.15.16:0/179916121 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff6640b2760 0x7ff6640d1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:41.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:41.421+0000 7ff661d9b700 1 -- 172.21.15.16:0/179916121 >> v2:172.21.15.132:3300/0 conn(0x7ff664144020 msgr2=0x7ff6640b0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:41.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:41.421+0000 7ff661d9b700 1 --2- 172.21.15.16:0/179916121 >> v2:172.21.15.132:3300/0 conn(0x7ff664144020 0x7ff6640b0980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:41.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:41.421+0000 7ff661d9b700 1 -- 172.21.15.16:0/179916121 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6640b0ec0 con 0x7ff6640cec60 2023-05-20T01:25:41.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:41.421+0000 7ff661d9b700 1 -- 172.21.15.16:0/179916121 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7ff65c0019f0 con 0x7ff6640cec60 2023-05-20T01:25:41.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:41.421+0000 7ff661d9b700 1 -- 172.21.15.16:0/179916121 >> v1:172.21.15.16:6789/0 conn(0x7ff6640cec60 legacy=0x7ff6640cc990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:41.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:41.421+0000 7ff661d9b700 1 --2- 172.21.15.16:0/179916121 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff650003a70 0x7ff650003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:41.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:41.421+0000 7ff661d9b700 1 --2- 172.21.15.16:0/179916121 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff650004850 0x7ff650004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:41.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:41.421+0000 7ff661d9b700 1 --2- 172.21.15.16:0/179916121 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff6640b2760 0x7ff650005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:41.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:41.421+0000 7ff661d9b700 1 -- 172.21.15.16:0/179916121 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6640cb6e0 con 0x7ff650003a70 2023-05-20T01:25:41.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:41.421+0000 7ff661d9b700 1 -- 172.21.15.16:0/179916121 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6641448a0 con 0x7ff650004850 2023-05-20T01:25:41.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:41.421+0000 7ff661d9b700 1 -- 172.21.15.16:0/179916121 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff65000ba10 con 0x7ff6640b2760 2023-05-20T01:25:41.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:41.421+0000 7ff662d9d700 1 --2- 172.21.15.16:0/179916121 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff650003a70 0x7ff650003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:41.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:41.421+0000 7ff66259c700 1 --2- 172.21.15.16:0/179916121 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff650004850 0x7ff650004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:41.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:41.421+0000 7ff66359e700 1 --2- 172.21.15.16:0/179916121 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff6640b2760 0x7ff650005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:41.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:41.421+0000 7ff662d9d700 1 -- 172.21.15.16:0/179916121 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff6640b2760 msgr2=0x7ff650005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:41.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:41.421+0000 7ff662d9d700 1 --2- 172.21.15.16:0/179916121 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff6640b2760 0x7ff650005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:41.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:41.421+0000 7ff662d9d700 1 -- 172.21.15.16:0/179916121 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff650004850 msgr2=0x7ff650004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:41.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:41.421+0000 7ff662d9d700 1 --2- 172.21.15.16:0/179916121 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff650004850 0x7ff650004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:41.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:41.421+0000 7ff662d9d700 1 -- 172.21.15.16:0/179916121 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6500035b0 con 0x7ff650003a70 2023-05-20T01:25:41.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:41.421+0000 7ff66359e700 1 --2- 172.21.15.16:0/179916121 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff6640b2760 0x7ff650005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:25:41.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:41.422+0000 7ff662d9d700 1 --2- 172.21.15.16:0/179916121 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff650003a70 0x7ff650003e80 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7ff64c009b60 tx=0x7ff650005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cf900454b1e03f6 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:41.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:41.422+0000 7ff661d9b700 1 -- 172.21.15.16:0/179916121 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff64c00e6c0 con 0x7ff650003a70 2023-05-20T01:25:41.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:41.422+0000 7ff661d9b700 1 -- 172.21.15.16:0/179916121 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff64c00e820 con 0x7ff650003a70 2023-05-20T01:25:41.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:41.422+0000 7ff661d9b700 1 -- 172.21.15.16:0/179916121 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff64c00eb90 con 0x7ff650003a70 2023-05-20T01:25:41.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:41.422+0000 7ff6691a5700 1 -- 172.21.15.16:0/179916121 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff650003a70 msgr2=0x7ff650003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:41.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:41.422+0000 7ff6691a5700 1 --2- 172.21.15.16:0/179916121 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff650003a70 0x7ff650003e80 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7ff64c009b60 tx=0x7ff650005880 comp rx=0 tx=0).stop 2023-05-20T01:25:41.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:41.422+0000 7ff66359e700 1 -- 172.21.15.16:0/179916121 reap_dead start 2023-05-20T01:25:41.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:41.422+0000 7ff6691a5700 1 -- 172.21.15.16:0/179916121 shutdown_connections 2023-05-20T01:25:41.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:41.422+0000 7ff6691a5700 1 -- 172.21.15.16:0/179916121 >> 172.21.15.16:0/179916121 conn(0x7ff66406ae30 msgr2=0x7ff66409a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:41.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:41.422+0000 7ff6691a5700 1 -- 172.21.15.16:0/179916121 shutdown_connections 2023-05-20T01:25:41.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:41.422+0000 7ff6691a5700 1 -- 172.21.15.16:0/179916121 wait complete. 2023-05-20T01:25:41.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:41.424+0000 7ff6691a5700 1 Processor -- start 2023-05-20T01:25:41.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:41.424+0000 7ff6691a5700 1 -- start start 2023-05-20T01:25:41.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:41.425+0000 7ff6691a5700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff650003a70 0x7ff664159d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:41.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:41.425+0000 7ff6691a5700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff66406ec70 0x7ff66406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:41.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:41.425+0000 7ff6691a5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff66415cef0 0x7ff66415d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:41.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:41.425+0000 7ff6691a5700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff66406f620 con 0x7ff66415cef0 2023-05-20T01:25:41.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:41.425+0000 7ff6691a5700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff66415a260 con 0x7ff66406ec70 2023-05-20T01:25:41.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:41.425+0000 7ff6691a5700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff66406b6c0 con 0x7ff650003a70 2023-05-20T01:25:41.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:41.425+0000 7ff66359e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff66415cef0 0x7ff66415d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:41.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:41.425+0000 7ff66359e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff66415cef0 0x7ff66415d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37198/0 (socket says 172.21.15.16:37198) 2023-05-20T01:25:41.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:41.425+0000 7ff66359e700 1 -- 172.21.15.16:0/2687928065 learned_addr learned my addr 172.21.15.16:0/2687928065 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:41.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:41.425+0000 7ff662d9d700 1 --2- 172.21.15.16:0/2687928065 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff650003a70 0x7ff664159d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:41.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:41.425+0000 7ff66259c700 1 --2- 172.21.15.16:0/2687928065 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff66406ec70 0x7ff66406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:41.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:41.425+0000 7ff66359e700 1 -- 172.21.15.16:0/2687928065 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff650003a70 msgr2=0x7ff664159d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:41.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:41.425+0000 7ff66359e700 1 --2- 172.21.15.16:0/2687928065 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff650003a70 0x7ff664159d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:41.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:41.425+0000 7ff66359e700 1 -- 172.21.15.16:0/2687928065 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff66406ec70 msgr2=0x7ff66406f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:41.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:41.425+0000 7ff66359e700 1 --2- 172.21.15.16:0/2687928065 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff66406ec70 0x7ff66406f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:41.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:41.425+0000 7ff66359e700 1 -- 172.21.15.16:0/2687928065 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6640cbbd0 con 0x7ff66415cef0 2023-05-20T01:25:41.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:41.425+0000 7ff66359e700 1 --2- 172.21.15.16:0/2687928065 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff66415cef0 0x7ff66415d370 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7ff66415e530 tx=0x7ff65c00e4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:41.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:41.426+0000 7ff65b7fe700 1 -- 172.21.15.16:0/2687928065 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff65c00ed20 con 0x7ff66415cef0 2023-05-20T01:25:41.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:41.426+0000 7ff65b7fe700 1 -- 172.21.15.16:0/2687928065 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff65c00ee80 con 0x7ff66415cef0 2023-05-20T01:25:41.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:41.426+0000 7ff6691a5700 1 -- 172.21.15.16:0/2687928065 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6640cb8f0 con 0x7ff66415cef0 2023-05-20T01:25:41.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:41.426+0000 7ff65b7fe700 1 -- 172.21.15.16:0/2687928065 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff65c013660 con 0x7ff66415cef0 2023-05-20T01:25:41.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:41.426+0000 7ff6691a5700 1 -- 172.21.15.16:0/2687928065 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6640cbeb0 con 0x7ff66415cef0 2023-05-20T01:25:41.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:41.427+0000 7ff6691a5700 1 -- 172.21.15.16:0/2687928065 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6640747b0 con 0x7ff66415cef0 2023-05-20T01:25:41.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:41.427+0000 7ff65b7fe700 1 -- 172.21.15.16:0/2687928065 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff65c0137c0 con 0x7ff66415cef0 2023-05-20T01:25:41.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:41.428+0000 7ff65b7fe700 1 --2- 172.21.15.16:0/2687928065 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff64406ace0 0x7ff64406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:41.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:41.428+0000 7ff65b7fe700 1 -- 172.21.15.16:0/2687928065 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 7044+0+0 (secure 0 0 0) 0x7ff65c089ee0 con 0x7ff66415cef0 2023-05-20T01:25:41.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:41.428+0000 7ff662d9d700 1 --2- 172.21.15.16:0/2687928065 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff64406ace0 0x7ff64406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:41.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:41.428+0000 7ff662d9d700 1 --2- 172.21.15.16:0/2687928065 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff64406ace0 0x7ff64406d190 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7ff64c0090b0 tx=0x7ff64c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:41.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:41.432+0000 7ff65b7fe700 1 -- 172.21.15.16:0/2687928065 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7ff65c054bb0 con 0x7ff66415cef0 2023-05-20T01:25:41.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:41.679+0000 7ff6691a5700 1 -- 172.21.15.16:0/2687928065 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff66415ae40 con 0x7ff66415cef0 2023-05-20T01:25:41.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:41.680+0000 7ff65b7fe700 1 -- 172.21.15.16:0/2687928065 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) v1 ==== 94+0+4671 (secure 0 0 0) 0x7ff65c0581d0 con 0x7ff66415cef0 2023-05-20T01:25:41.685 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:25:41.685 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":70,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6323,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/59032067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":59032067},{"type":"v1","addr":"172.21.15.132:6834","nonce":59032067}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66},{"gid":6329,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/100162236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":100162236},{"type":"v1","addr":"172.21.15.132:6835","nonce":100162236}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66},{"gid":7572,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/3753560731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3753560731},{"type":"v1","addr":"172.21.15.16:6836","nonce":3753560731}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":70,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-20T01:25:32.295399+0000","modified":"2023-05-20T01:25:41.570561+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":95,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7575},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7575":{"gid":7575,"name":"c","rank":0,"incarnation":67,"state":"up:active","state_seq":4,"addr":"172.21.15.16:6837/307899603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":307899603},{"type":"v1","addr":"172.21.15.16:6837","nonce":307899603}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13,14],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-05-20T01:25:41.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:41.682+0000 7ff6691a5700 1 -- 172.21.15.16:0/2687928065 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff64406ace0 msgr2=0x7ff64406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:41.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:41.682+0000 7ff6691a5700 1 --2- 172.21.15.16:0/2687928065 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff64406ace0 0x7ff64406d190 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7ff64c0090b0 tx=0x7ff64c008030 comp rx=0 tx=0).stop 2023-05-20T01:25:41.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:41.682+0000 7ff6691a5700 1 -- 172.21.15.16:0/2687928065 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff66415cef0 msgr2=0x7ff66415d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:41.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:41.682+0000 7ff6691a5700 1 --2- 172.21.15.16:0/2687928065 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff66415cef0 0x7ff66415d370 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7ff66415e530 tx=0x7ff65c00e4f0 comp rx=0 tx=0).stop 2023-05-20T01:25:41.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:41.682+0000 7ff6691a5700 1 -- 172.21.15.16:0/2687928065 shutdown_connections 2023-05-20T01:25:41.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:41.682+0000 7ff6691a5700 1 --2- 172.21.15.16:0/2687928065 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff66415cef0 0x7ff66415d370 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:41.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:41.682+0000 7ff6691a5700 1 --2- 172.21.15.16:0/2687928065 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff64406ace0 0x7ff64406d190 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:41.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:41.682+0000 7ff6691a5700 1 --2- 172.21.15.16:0/2687928065 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff650003a70 0x7ff664159d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:41.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:41.682+0000 7ff6691a5700 1 --2- 172.21.15.16:0/2687928065 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff66406ec70 0x7ff66406f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:41.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:41.682+0000 7ff6691a5700 1 -- 172.21.15.16:0/2687928065 >> 172.21.15.16:0/2687928065 conn(0x7ff66406ae30 msgr2=0x7ff6640cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:41.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:41.682+0000 7ff6691a5700 1 -- 172.21.15.16:0/2687928065 shutdown_connections 2023-05-20T01:25:41.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:41.682+0000 7ff6691a5700 1 -- 172.21.15.16:0/2687928065 wait complete. 2023-05-20T01:25:41.689 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 70 2023-05-20T01:25:42.101 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 70, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-05-20T01:25:32.295399+0000', 'modified': '2023-05-20T01:25:41.570561+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 95, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7575}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7575': {'gid': 7575, 'name': 'c', 'rank': 0, 'incarnation': 67, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.16:6837/307899603', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6835', 'nonce': 307899603}, {'type': 'v1', 'addr': '172.21.15.16:6837', 'nonce': 307899603}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [13, 14], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-05-20T01:25:42.101 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-05-20T01:25:42.101 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-05-20T01:25:42.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.229+0000 7f4446554700 1 Processor -- start 2023-05-20T01:25:42.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.229+0000 7f4446554700 1 -- start start 2023-05-20T01:25:42.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.229+0000 7f4446554700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f444006f450 0x7f44400cf890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:42.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.230+0000 7f4446554700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f444006d4b0 0x7f4440161080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:42.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.230+0000 7f4446554700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f444006a510 0x7f444006a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:42.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.230+0000 7f4446554700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f444006aef0 con 0x7f444006a510 2023-05-20T01:25:42.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.230+0000 7f4446554700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44400cc510 con 0x7f444006d4b0 2023-05-20T01:25:42.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.230+0000 7f4446554700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44401615c0 con 0x7f444006f450 2023-05-20T01:25:42.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.230+0000 7f4444af1700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f444006a510 0x7f444006a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:42.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.230+0000 7f4444af1700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f444006a510 0x7f444006a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37206/0 (socket says 172.21.15.16:37206) 2023-05-20T01:25:42.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.230+0000 7f4444af1700 1 -- 172.21.15.16:0/499814730 learned_addr learned my addr 172.21.15.16:0/499814730 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:42.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.230+0000 7f443f7fe700 1 --2- 172.21.15.16:0/499814730 >> v2:172.21.15.132:3300/0 conn(0x7f444006d4b0 0x7f4440161080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:42.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.230+0000 7f443ffff700 1 --2- 172.21.15.16:0/499814730 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f444006f450 0x7f44400cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:42.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.230+0000 7f4444af1700 1 -- 172.21.15.16:0/499814730 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f444006f450 msgr2=0x7f44400cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:42.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.230+0000 7f4444af1700 1 --2- 172.21.15.16:0/499814730 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f444006f450 0x7f44400cf890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:42.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.230+0000 7f4444af1700 1 -- 172.21.15.16:0/499814730 >> v2:172.21.15.132:3300/0 conn(0x7f444006d4b0 msgr2=0x7f4440161080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:42.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.230+0000 7f4444af1700 1 --2- 172.21.15.16:0/499814730 >> v2:172.21.15.132:3300/0 conn(0x7f444006d4b0 0x7f4440161080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:42.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.230+0000 7f4444af1700 1 -- 172.21.15.16:0/499814730 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4440071ed0 con 0x7f444006a510 2023-05-20T01:25:42.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.230+0000 7f4444af1700 1 --2- 172.21.15.16:0/499814730 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f444006a510 0x7f444006a9b0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f44400d1020 tx=0x7f443800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f84fd39b205f8e4b server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:42.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.230+0000 7f443effd700 1 -- 172.21.15.16:0/499814730 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f443800e070 con 0x7f444006a510 2023-05-20T01:25:42.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.230+0000 7f443effd700 1 -- 172.21.15.16:0/499814730 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f443800b9c0 con 0x7f444006a510 2023-05-20T01:25:42.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.230+0000 7f443effd700 1 -- 172.21.15.16:0/499814730 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f443800bd70 con 0x7f444006a510 2023-05-20T01:25:42.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.231+0000 7f4446554700 1 -- 172.21.15.16:0/499814730 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f444006a510 msgr2=0x7f444006a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:42.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.231+0000 7f4446554700 1 --2- 172.21.15.16:0/499814730 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f444006a510 0x7f444006a9b0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f44400d1020 tx=0x7f443800b330 comp rx=0 tx=0).stop 2023-05-20T01:25:42.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.231+0000 7f4446554700 1 -- 172.21.15.16:0/499814730 shutdown_connections 2023-05-20T01:25:42.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.231+0000 7f4446554700 1 --2- 172.21.15.16:0/499814730 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f444006a510 0x7f444006a9b0 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:42.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.231+0000 7f4446554700 1 --2- 172.21.15.16:0/499814730 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f444006f450 0x7f44400cf890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:42.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.231+0000 7f4446554700 1 --2- 172.21.15.16:0/499814730 >> v2:172.21.15.132:3300/0 conn(0x7f444006d4b0 0x7f4440161080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:42.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.231+0000 7f4446554700 1 -- 172.21.15.16:0/499814730 >> 172.21.15.16:0/499814730 conn(0x7f44400cffb0 msgr2=0x7f44400d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:42.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.231+0000 7f4446554700 1 -- 172.21.15.16:0/499814730 shutdown_connections 2023-05-20T01:25:42.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.231+0000 7f4446554700 1 -- 172.21.15.16:0/499814730 wait complete. 2023-05-20T01:25:42.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.232+0000 7f4446554700 1 Processor -- start 2023-05-20T01:25:42.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.232+0000 7f4446554700 1 -- start start 2023-05-20T01:25:42.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.233+0000 7f4446554700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f444006a510 0x7f44401f27b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:42.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.233+0000 7f4446554700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f444006d4b0 0x7f4440160880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:42.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.233+0000 7f4446554700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f444006f450 0x7f44401ef800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:42.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.233+0000 7f4446554700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44400c4370 con 0x7f444006d4b0 2023-05-20T01:25:42.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.233+0000 7f4446554700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44401f3090 con 0x7f444006a510 2023-05-20T01:25:42.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.233+0000 7f4446554700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44401f3370 con 0x7f444006f450 2023-05-20T01:25:42.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.233+0000 7f443f7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f444006d4b0 0x7f4440160880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:42.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.233+0000 7f443f7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f444006d4b0 0x7f4440160880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37210/0 (socket says 172.21.15.16:37210) 2023-05-20T01:25:42.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.233+0000 7f443f7fe700 1 -- 172.21.15.16:0/690861708 learned_addr learned my addr 172.21.15.16:0/690861708 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:42.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.233+0000 7f443ffff700 1 --2- 172.21.15.16:0/690861708 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f444006a510 0x7f44401f27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:42.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.233+0000 7f4444af1700 1 --2- 172.21.15.16:0/690861708 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f444006f450 0x7f44401ef800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:42.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.234+0000 7f443f7fe700 1 -- 172.21.15.16:0/690861708 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f444006f450 msgr2=0x7f44401ef800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:42.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.234+0000 7f443f7fe700 1 --2- 172.21.15.16:0/690861708 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f444006f450 0x7f44401ef800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:42.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.234+0000 7f443f7fe700 1 -- 172.21.15.16:0/690861708 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f444006a510 msgr2=0x7f44401f27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:42.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.234+0000 7f443f7fe700 1 --2- 172.21.15.16:0/690861708 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f444006a510 0x7f44401f27b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:42.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.234+0000 7f443f7fe700 1 -- 172.21.15.16:0/690861708 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4438009b30 con 0x7f444006d4b0 2023-05-20T01:25:42.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.234+0000 7f443f7fe700 1 --2- 172.21.15.16:0/690861708 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f444006d4b0 0x7f4440160880 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f443400e810 tx=0x7f443400eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:42.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.234+0000 7f443cff9700 1 -- 172.21.15.16:0/690861708 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f443400c070 con 0x7f444006d4b0 2023-05-20T01:25:42.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.234+0000 7f443cff9700 1 -- 172.21.15.16:0/690861708 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4434013610 con 0x7f444006d4b0 2023-05-20T01:25:42.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.234+0000 7f4446554700 1 -- 172.21.15.16:0/690861708 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44400cc1a0 con 0x7f444006d4b0 2023-05-20T01:25:42.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.234+0000 7f443cff9700 1 -- 172.21.15.16:0/690861708 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44340139a0 con 0x7f444006d4b0 2023-05-20T01:25:42.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.234+0000 7f4446554700 1 -- 172.21.15.16:0/690861708 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44400721b0 con 0x7f444006d4b0 2023-05-20T01:25:42.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.236+0000 7f443cff9700 1 -- 172.21.15.16:0/690861708 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4434013b00 con 0x7f444006d4b0 2023-05-20T01:25:42.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.236+0000 7f443cff9700 1 --2- 172.21.15.16:0/690861708 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f442c06ace0 0x7f442c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:42.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.236+0000 7f443cff9700 1 -- 172.21.15.16:0/690861708 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 7044+0+0 (secure 0 0 0) 0x7f44340894a0 con 0x7f444006d4b0 2023-05-20T01:25:42.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.236+0000 7f443ffff700 1 --2- 172.21.15.16:0/690861708 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f442c06ace0 0x7f442c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:42.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.236+0000 7f443ffff700 1 --2- 172.21.15.16:0/690861708 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f442c06ace0 0x7f442c06d190 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f4430006b20 tx=0x7f4430008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:42.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.238+0000 7f44526e0b80 1 Processor -- start 2023-05-20T01:25:42.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.238+0000 7f44526e0b80 1 -- start start 2023-05-20T01:25:42.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.239+0000 7f44526e0b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eef7699400 0x55eef76997d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:42.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.239+0000 7f44526e0b80 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55eef7695700 0x55eef766e460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:42.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.239+0000 7f44526e0b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55eef766c870 0x55eef766e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:42.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.239+0000 7f44526e0b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eef7574010 con 0x55eef7699400 2023-05-20T01:25:42.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.239+0000 7f44526e0b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eef75c6850 con 0x55eef766c870 2023-05-20T01:25:42.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.239+0000 7f44526e0b80 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55eef75bfce0 con 0x55eef7695700 2023-05-20T01:25:42.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.239+0000 7f443ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eef7699400 0x55eef76997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:42.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.239+0000 7f443ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eef7699400 0x55eef76997d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37216/0 (socket says 172.21.15.16:37216) 2023-05-20T01:25:42.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.239+0000 7f443ffff700 1 -- 172.21.15.16:0/2661127910 learned_addr learned my addr 172.21.15.16:0/2661127910 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:42.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.240+0000 7f4444af1700 1 --2- 172.21.15.16:0/2661127910 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55eef766c870 0x55eef766e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:42.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.240+0000 7f443f7fe700 1 --2- 172.21.15.16:0/2661127910 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55eef7695700 0x55eef766e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:42.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.240+0000 7f443ffff700 1 -- 172.21.15.16:0/2661127910 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55eef7695700 msgr2=0x55eef766e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:42.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.240+0000 7f443ffff700 1 --2- 172.21.15.16:0/2661127910 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55eef7695700 0x55eef766e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:42.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.240+0000 7f443ffff700 1 -- 172.21.15.16:0/2661127910 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55eef766c870 msgr2=0x55eef766e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:42.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.240+0000 7f443ffff700 1 --2- 172.21.15.16:0/2661127910 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55eef766c870 0x55eef766e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:42.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.240+0000 7f443ffff700 1 -- 172.21.15.16:0/2661127910 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eef74415e0 con 0x55eef7699400 2023-05-20T01:25:42.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.240+0000 7f4444af1700 1 --2- 172.21.15.16:0/2661127910 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55eef766c870 0x55eef766e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:25:42.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.240+0000 7f443ffff700 1 --2- 172.21.15.16:0/2661127910 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eef7699400 0x55eef76997d0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f44300109d0 tx=0x7f4430013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfa910b1aa6d5f5d server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:42.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.240+0000 7f4425ffb700 1 -- 172.21.15.16:0/2661127910 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4430010e50 con 0x55eef7699400 2023-05-20T01:25:42.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.240+0000 7f4425ffb700 1 -- 172.21.15.16:0/2661127910 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f443001a070 con 0x55eef7699400 2023-05-20T01:25:42.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.240+0000 7f4425ffb700 1 -- 172.21.15.16:0/2661127910 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4430016d20 con 0x55eef7699400 2023-05-20T01:25:42.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.241+0000 7f44526e0b80 1 -- 172.21.15.16:0/2661127910 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eef7699400 msgr2=0x55eef76997d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:42.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.241+0000 7f44526e0b80 1 --2- 172.21.15.16:0/2661127910 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eef7699400 0x55eef76997d0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f44300109d0 tx=0x7f4430013000 comp rx=0 tx=0).stop 2023-05-20T01:25:42.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.241+0000 7f44526e0b80 1 -- 172.21.15.16:0/2661127910 shutdown_connections 2023-05-20T01:25:42.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.241+0000 7f44526e0b80 1 --2- 172.21.15.16:0/2661127910 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55eef766c870 0x55eef766e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:42.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.241+0000 7f44526e0b80 1 --2- 172.21.15.16:0/2661127910 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eef7699400 0x55eef76997d0 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:42.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.241+0000 7f44526e0b80 1 --2- 172.21.15.16:0/2661127910 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55eef7695700 0x55eef766e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:42.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.241+0000 7f44526e0b80 1 -- 172.21.15.16:0/2661127910 >> 172.21.15.16:0/2661127910 conn(0x55eef7663f30 msgr2=0x55eef76b4210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:42.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.241+0000 7f44526e0b80 1 -- 172.21.15.16:0/2661127910 shutdown_connections 2023-05-20T01:25:42.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.241+0000 7f44526e0b80 1 -- 172.21.15.16:0/2661127910 wait complete. 2023-05-20T01:25:42.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.241+0000 7f44526e0b80 1 -- start start 2023-05-20T01:25:42.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.242+0000 7f44526e0b80 1 Processor -- start 2023-05-20T01:25:42.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.243+0000 7f44526e0b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eef766c870 0x55eef768cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:42.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.243+0000 7f44526e0b80 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55eef7695700 0x55eef7690880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:42.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.243+0000 7f44526e0b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55eef7699400 0x55eef7690dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:42.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.243+0000 7f44526e0b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eef7561890 con 0x55eef766c870 2023-05-20T01:25:42.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.243+0000 7f44526e0b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eef7561530 con 0x55eef7699400 2023-05-20T01:25:42.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.243+0000 7f44526e0b80 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55eef74438d0 con 0x55eef7695700 2023-05-20T01:25:42.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.243+0000 7f44526e0b80 10 client.0 _ll_register_callbacks cb 0x55eef757f730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-05-20T01:25:42.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.243+0000 7f443ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eef766c870 0x55eef768cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:42.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.243+0000 7f443ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eef766c870 0x55eef768cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37222/0 (socket says 172.21.15.16:37222) 2023-05-20T01:25:42.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.243+0000 7f443ffff700 1 -- 172.21.15.16:0/3764885479 learned_addr learned my addr 172.21.15.16:0/3764885479 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:42.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.243+0000 7f443f7fe700 1 --2- 172.21.15.16:0/3764885479 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55eef7695700 0x55eef7690880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:42.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.243+0000 7f4444af1700 1 --2- 172.21.15.16:0/3764885479 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55eef7699400 0x55eef7690dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:42.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.243+0000 7f443ffff700 1 -- 172.21.15.16:0/3764885479 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55eef7695700 msgr2=0x55eef7690880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:42.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.243+0000 7f443ffff700 1 --2- 172.21.15.16:0/3764885479 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55eef7695700 0x55eef7690880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:42.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.243+0000 7f443ffff700 1 -- 172.21.15.16:0/3764885479 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55eef7699400 msgr2=0x55eef7690dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:42.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.243+0000 7f443ffff700 1 --2- 172.21.15.16:0/3764885479 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55eef7699400 0x55eef7690dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:42.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.243+0000 7f443ffff700 1 -- 172.21.15.16:0/3764885479 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f443000e040 con 0x55eef766c870 2023-05-20T01:25:42.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.244+0000 7f443ffff700 1 --2- 172.21.15.16:0/3764885479 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eef766c870 0x55eef768cb10 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f4430006f10 tx=0x7f44300047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0ddd1b68271b474 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:42.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.244+0000 7f441effd700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-05-20T01:25:42.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.244+0000 7f441effd700 1 -- 172.21.15.16:0/3764885479 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4430010e50 con 0x55eef766c870 2023-05-20T01:25:42.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.244+0000 7f441effd700 1 -- 172.21.15.16:0/3764885479 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 7044+0+0 (secure 0 0 0) 0x7f4430005c30 con 0x55eef766c870 2023-05-20T01:25:42.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.244+0000 7f44526e0b80 1 -- 172.21.15.16:0/3764885479 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55eef76616f0 con 0x55eef766c870 2023-05-20T01:25:42.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.244+0000 7f441effd700 1 -- 172.21.15.16:0/3764885479 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4430005df0 con 0x55eef766c870 2023-05-20T01:25:42.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.244+0000 7f44526e0b80 10 client.7611 fetch_fsmap learned FSMap version 70 2023-05-20T01:25:42.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.244+0000 7f44526e0b80 1 -- 172.21.15.16:0/3764885479 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=2+}) v3 -- 0x55eef7586140 con 0x55eef766c870 2023-05-20T01:25:42.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.244+0000 7f441effd700 1 -- 172.21.15.16:0/3764885479 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55eef76616f0 con 0x55eef766c870 2023-05-20T01:25:42.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.244+0000 7f441effd700 1 -- 172.21.15.16:0/3764885479 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 70) v1 ==== 3122+0+0 (secure 0 0 0) 0x7f44300219d0 con 0x55eef766c870 2023-05-20T01:25:42.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.244+0000 7f44526e0b80 10 client.7611 fetch_fsmap finished waiting for FSMap version 70 2023-05-20T01:25:42.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.244+0000 7f44526e0b80 10 client.7611 resolve_mds: resolved name 'c' to daemon mds.c 2023-05-20T01:25:42.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.244+0000 7f44526e0b80 20 client.7611 populate_metadata read hostname 'smithi016' 2023-05-20T01:25:42.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.244+0000 7f44526e0b80 1 --2- 172.21.15.16:0/3764885479 >> [v2:172.21.15.16:6835/307899603,v1:172.21.15.16:6837/307899603] conn(0x55eef766d1d0 0x55eef768d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:42.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.244+0000 7f44526e0b80 4 client.7611 mds_command: new command op to 7575 tid=0[{"prefix": "get_command_descriptions"}] 2023-05-20T01:25:42.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.244+0000 7f44526e0b80 1 -- 172.21.15.16:0/3764885479 --> [v2:172.21.15.16:6835/307899603,v1:172.21.15.16:6837/307899603] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55eef7582050 con 0x55eef766d1d0 2023-05-20T01:25:42.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.244+0000 7f443f7fe700 1 --2- 172.21.15.16:0/3764885479 >> [v2:172.21.15.16:6835/307899603,v1:172.21.15.16:6837/307899603] conn(0x55eef766d1d0 0x55eef768d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:42.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.245+0000 7f443f7fe700 1 --2- 172.21.15.16:0/3764885479 >> [v2:172.21.15.16:6835/307899603,v1:172.21.15.16:6837/307899603] conn(0x55eef766d1d0 0x55eef768d6b0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61b8f86266939ecf server_cookie=a73ac52a69e47683 in_seq=0 out_seq=0 2023-05-20T01:25:42.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.245+0000 7f441effd700 10 client.7611 ms_handle_connect on v2:172.21.15.16:6835/307899603 2023-05-20T01:25:42.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.245+0000 7f441effd700 1 -- 172.21.15.16:0/3764885479 <== mds.0 v2:172.21.15.16:6835/307899603 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55eef7582050 con 0x55eef766d1d0 2023-05-20T01:25:42.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.245+0000 7f441effd700 10 client.7611 handle_command_reply: tid=0 2023-05-20T01:25:42.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.245+0000 7f44526e0b80 1 client.7611 shutdown 2023-05-20T01:25:42.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.245+0000 7f44526e0b80 1 -- 172.21.15.16:0/3764885479 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eef766c870 msgr2=0x55eef768cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:42.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.245+0000 7f44526e0b80 1 --2- 172.21.15.16:0/3764885479 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eef766c870 0x55eef768cb10 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f4430006f10 tx=0x7f44300047c0 comp rx=0 tx=0).stop 2023-05-20T01:25:42.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.245+0000 7f44526e0b80 1 -- 172.21.15.16:0/3764885479 shutdown_connections 2023-05-20T01:25:42.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.245+0000 7f44526e0b80 1 --2- 172.21.15.16:0/3764885479 >> [v2:172.21.15.16:6835/307899603,v1:172.21.15.16:6837/307899603] conn(0x55eef766d1d0 0x55eef768d6b0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:42.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.245+0000 7f44526e0b80 1 --2- 172.21.15.16:0/3764885479 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55eef7699400 0x55eef7690dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:42.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.245+0000 7f44526e0b80 1 --2- 172.21.15.16:0/3764885479 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eef766c870 0x55eef768cb10 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:42.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.245+0000 7f44526e0b80 1 --2- 172.21.15.16:0/3764885479 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55eef7695700 0x55eef7690880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:42.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.245+0000 7f441effd700 0 client.7611 ms_handle_reset on v2:172.21.15.16:6835/307899603 2023-05-20T01:25:42.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.245+0000 7f44526e0b80 1 -- 172.21.15.16:0/3764885479 >> 172.21.15.16:0/3764885479 conn(0x55eef7663f30 msgr2=0x55eef76693e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:42.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.246+0000 7f44526e0b80 1 -- 172.21.15.16:0/3764885479 shutdown_connections 2023-05-20T01:25:42.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.246+0000 7f44526e0b80 1 -- 172.21.15.16:0/3764885479 wait complete. 2023-05-20T01:25:42.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.246+0000 7f44526e0b80 20 client.7611 trim_cache size 0 max 16384 2023-05-20T01:25:42.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.264+0000 7f44526e0b80 1 Processor -- start 2023-05-20T01:25:42.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.264+0000 7f44526e0b80 1 -- start start 2023-05-20T01:25:42.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.264+0000 7f44526e0b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eef766c870 0x55eef766e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:42.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.264+0000 7f44526e0b80 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55eef7695700 0x55eef7696ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:42.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.264+0000 7f44526e0b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55eef7699400 0x55eef76c6220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:42.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.264+0000 7f44526e0b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eef76c5ec0 con 0x55eef766c870 2023-05-20T01:25:42.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.264+0000 7f44526e0b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eef7697000 con 0x55eef7699400 2023-05-20T01:25:42.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.264+0000 7f44526e0b80 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55eef76c5c70 con 0x55eef7695700 2023-05-20T01:25:42.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.264+0000 7f443ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eef766c870 0x55eef766e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:42.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.265+0000 7f443ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eef766c870 0x55eef766e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37230/0 (socket says 172.21.15.16:37230) 2023-05-20T01:25:42.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.265+0000 7f443ffff700 1 -- 172.21.15.16:0/687051189 learned_addr learned my addr 172.21.15.16:0/687051189 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:42.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.265+0000 7f4444af1700 1 --2- 172.21.15.16:0/687051189 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55eef7699400 0x55eef76c6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:42.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.265+0000 7f443f7fe700 1 --2- 172.21.15.16:0/687051189 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55eef7695700 0x55eef7696ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:42.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.265+0000 7f443ffff700 1 -- 172.21.15.16:0/687051189 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55eef7695700 msgr2=0x55eef7696ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:42.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.265+0000 7f443ffff700 1 --2- 172.21.15.16:0/687051189 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55eef7695700 0x55eef7696ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:42.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.265+0000 7f443ffff700 1 -- 172.21.15.16:0/687051189 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55eef7699400 msgr2=0x55eef76c6220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:42.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.265+0000 7f443ffff700 1 --2- 172.21.15.16:0/687051189 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55eef7699400 0x55eef76c6220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:42.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.265+0000 7f443ffff700 1 -- 172.21.15.16:0/687051189 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eef7585e90 con 0x55eef766c870 2023-05-20T01:25:42.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.265+0000 7f4444af1700 1 --2- 172.21.15.16:0/687051189 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55eef7699400 0x55eef76c6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:25:42.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.265+0000 7f443f7fe700 1 --2- 172.21.15.16:0/687051189 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55eef7695700 0x55eef7696ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:25:42.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.265+0000 7f443ffff700 1 --2- 172.21.15.16:0/687051189 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eef766c870 0x55eef766e100 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f4430007840 tx=0x7f443002d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57ccdef73755d03 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:42.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.265+0000 7f44267fc700 1 -- 172.21.15.16:0/687051189 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4430010870 con 0x55eef766c870 2023-05-20T01:25:42.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.265+0000 7f44267fc700 1 -- 172.21.15.16:0/687051189 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4430016760 con 0x55eef766c870 2023-05-20T01:25:42.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.265+0000 7f44267fc700 1 -- 172.21.15.16:0/687051189 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4430021a70 con 0x55eef766c870 2023-05-20T01:25:42.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.266+0000 7f44526e0b80 1 -- 172.21.15.16:0/687051189 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eef766c870 msgr2=0x55eef766e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:42.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.266+0000 7f44526e0b80 1 --2- 172.21.15.16:0/687051189 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eef766c870 0x55eef766e100 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f4430007840 tx=0x7f443002d000 comp rx=0 tx=0).stop 2023-05-20T01:25:42.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.266+0000 7f44526e0b80 1 -- 172.21.15.16:0/687051189 shutdown_connections 2023-05-20T01:25:42.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.266+0000 7f44526e0b80 1 --2- 172.21.15.16:0/687051189 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55eef7699400 0x55eef76c6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:42.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.266+0000 7f44526e0b80 1 --2- 172.21.15.16:0/687051189 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eef766c870 0x55eef766e100 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:42.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.266+0000 7f44526e0b80 1 --2- 172.21.15.16:0/687051189 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55eef7695700 0x55eef7696ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:42.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.266+0000 7f44526e0b80 1 -- 172.21.15.16:0/687051189 >> 172.21.15.16:0/687051189 conn(0x55eef7663f30 msgr2=0x55eef76d4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:42.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.266+0000 7f44526e0b80 1 -- 172.21.15.16:0/687051189 shutdown_connections 2023-05-20T01:25:42.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.266+0000 7f44526e0b80 1 -- 172.21.15.16:0/687051189 wait complete. 2023-05-20T01:25:42.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.266+0000 7f44526e0b80 1 -- start start 2023-05-20T01:25:42.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.266+0000 7f44526e0b80 1 Processor -- start 2023-05-20T01:25:42.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.267+0000 7f44526e0b80 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55eef766c870 0x55eef7699950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:42.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.267+0000 7f44526e0b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eef7695700 0x55eef7700a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:42.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.267+0000 7f44526e0b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55eef7699400 0x55eef76b4210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:42.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.267+0000 7f44526e0b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eef768b590 con 0x55eef7695700 2023-05-20T01:25:42.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.267+0000 7f44526e0b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eef7702300 con 0x55eef7699400 2023-05-20T01:25:42.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.267+0000 7f44526e0b80 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55eef76c6500 con 0x55eef766c870 2023-05-20T01:25:42.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.267+0000 7f44526e0b80 10 client.0 _ll_register_callbacks cb 0x7f4430016e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-05-20T01:25:42.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.267+0000 7f443f7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eef7695700 0x55eef7700a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:42.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.267+0000 7f443f7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eef7695700 0x55eef7700a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37238/0 (socket says 172.21.15.16:37238) 2023-05-20T01:25:42.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.267+0000 7f443f7fe700 1 -- 172.21.15.16:0/1015271111 learned_addr learned my addr 172.21.15.16:0/1015271111 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:42.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.267+0000 7f443ffff700 1 --2- 172.21.15.16:0/1015271111 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55eef766c870 0x55eef7699950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:42.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.267+0000 7f4444af1700 1 --2- 172.21.15.16:0/1015271111 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55eef7699400 0x55eef76b4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:42.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.268+0000 7f443f7fe700 1 -- 172.21.15.16:0/1015271111 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55eef766c870 msgr2=0x55eef7699950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:42.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.268+0000 7f443f7fe700 1 --2- 172.21.15.16:0/1015271111 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55eef766c870 0x55eef7699950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:42.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.268+0000 7f443f7fe700 1 -- 172.21.15.16:0/1015271111 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55eef7699400 msgr2=0x55eef76b4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:42.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.268+0000 7f443f7fe700 1 --2- 172.21.15.16:0/1015271111 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55eef7699400 0x55eef76b4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:42.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.268+0000 7f443f7fe700 1 -- 172.21.15.16:0/1015271111 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55eef7586140 con 0x55eef7695700 2023-05-20T01:25:42.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.268+0000 7f4444af1700 1 --2- 172.21.15.16:0/1015271111 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55eef7699400 0x55eef76b4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:25:42.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.268+0000 7f443f7fe700 1 --2- 172.21.15.16:0/1015271111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eef7695700 0x55eef7700a30 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f44340543c0 tx=0x7f4434089040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c8558d07b0253d server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:42.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.268+0000 7f441effd700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-05-20T01:25:42.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.268+0000 7f441effd700 1 -- 172.21.15.16:0/1015271111 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4434073070 con 0x55eef7695700 2023-05-20T01:25:42.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.268+0000 7f441effd700 1 -- 172.21.15.16:0/1015271111 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 7044+0+0 (secure 0 0 0) 0x7f44340890b0 con 0x55eef7695700 2023-05-20T01:25:42.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.268+0000 7f44526e0b80 1 -- 172.21.15.16:0/1015271111 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f443001a070 con 0x55eef7695700 2023-05-20T01:25:42.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.268+0000 7f441effd700 1 -- 172.21.15.16:0/1015271111 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4434089270 con 0x55eef7695700 2023-05-20T01:25:42.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.268+0000 7f44526e0b80 10 client.7617 fetch_fsmap learned FSMap version 70 2023-05-20T01:25:42.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.268+0000 7f44526e0b80 1 -- 172.21.15.16:0/1015271111 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=2+}) v3 -- 0x55eef73d6c10 con 0x55eef7695700 2023-05-20T01:25:42.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.268+0000 7f441effd700 1 -- 172.21.15.16:0/1015271111 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f443001a070 con 0x55eef7695700 2023-05-20T01:25:42.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.268+0000 7f441effd700 1 -- 172.21.15.16:0/1015271111 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 70) v1 ==== 3122+0+0 (secure 0 0 0) 0x7f4434076030 con 0x55eef7695700 2023-05-20T01:25:42.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.268+0000 7f44526e0b80 10 client.7617 fetch_fsmap finished waiting for FSMap version 70 2023-05-20T01:25:42.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.268+0000 7f44526e0b80 10 client.7617 resolve_mds: resolved name 'c' to daemon mds.c 2023-05-20T01:25:42.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.268+0000 7f44526e0b80 20 client.7617 populate_metadata read hostname 'smithi016' 2023-05-20T01:25:42.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.268+0000 7f44526e0b80 1 --2- 172.21.15.16:0/1015271111 >> [v2:172.21.15.16:6835/307899603,v1:172.21.15.16:6837/307899603] conn(0x55eef77013f0 0x55eef77017c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:42.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.268+0000 7f44526e0b80 4 client.7617 mds_command: new command op to 7575 tid=0[{"prefix": "status"}] 2023-05-20T01:25:42.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.268+0000 7f44526e0b80 1 -- 172.21.15.16:0/1015271111 --> [v2:172.21.15.16:6835/307899603,v1:172.21.15.16:6837/307899603] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55eef758e610 con 0x55eef77013f0 2023-05-20T01:25:42.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.269+0000 7f443ffff700 1 --2- 172.21.15.16:0/1015271111 >> [v2:172.21.15.16:6835/307899603,v1:172.21.15.16:6837/307899603] conn(0x55eef77013f0 0x55eef77017c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:42.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.269+0000 7f441effd700 10 client.7617 ms_handle_connect on v2:172.21.15.16:6835/307899603 2023-05-20T01:25:42.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.269+0000 7f443ffff700 1 --2- 172.21.15.16:0/1015271111 >> [v2:172.21.15.16:6835/307899603,v1:172.21.15.16:6837/307899603] conn(0x55eef77013f0 0x55eef77017c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5b2dbbfec2a8340 server_cookie=5fe5ee23e154ab5f in_seq=0 out_seq=0 2023-05-20T01:25:42.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.269+0000 7f441effd700 1 -- 172.21.15.16:0/1015271111 <== mds.0 v2:172.21.15.16:6835/307899603 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x55eef758e610 con 0x55eef77013f0 2023-05-20T01:25:42.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.269+0000 7f441effd700 10 client.7617 handle_command_reply: tid=0 2023-05-20T01:25:42.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.269+0000 7f44526e0b80 1 client.7617 shutdown 2023-05-20T01:25:42.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.269+0000 7f44526e0b80 1 -- 172.21.15.16:0/1015271111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eef7695700 msgr2=0x55eef7700a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:42.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.269+0000 7f44526e0b80 1 --2- 172.21.15.16:0/1015271111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eef7695700 0x55eef7700a30 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f44340543c0 tx=0x7f4434089040 comp rx=0 tx=0).stop 2023-05-20T01:25:42.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.269+0000 7f44526e0b80 1 -- 172.21.15.16:0/1015271111 shutdown_connections 2023-05-20T01:25:42.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.269+0000 7f44526e0b80 1 --2- 172.21.15.16:0/1015271111 >> [v2:172.21.15.16:6835/307899603,v1:172.21.15.16:6837/307899603] conn(0x55eef77013f0 0x55eef77017c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:42.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.269+0000 7f44526e0b80 1 --2- 172.21.15.16:0/1015271111 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55eef7699400 0x55eef76b4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:42.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.269+0000 7f441effd700 0 client.7617 ms_handle_reset on v2:172.21.15.16:6835/307899603 2023-05-20T01:25:42.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.269+0000 7f44526e0b80 1 --2- 172.21.15.16:0/1015271111 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55eef766c870 0x55eef7699950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:42.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.269+0000 7f44526e0b80 1 --2- 172.21.15.16:0/1015271111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eef7695700 0x55eef7700a30 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:42.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.269+0000 7f44526e0b80 1 -- 172.21.15.16:0/1015271111 >> 172.21.15.16:0/1015271111 conn(0x55eef7663f30 msgr2=0x55eef766b520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:42.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.269+0000 7f44526e0b80 1 -- 172.21.15.16:0/1015271111 shutdown_connections 2023-05-20T01:25:42.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.269+0000 7f44526e0b80 1 -- 172.21.15.16:0/1015271111 wait complete. 2023-05-20T01:25:42.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.269+0000 7f44526e0b80 20 client.7617 trim_cache size 0 max 16384 2023-05-20T01:25:42.312 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-05-20T01:25:42.312 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "c2245cc5-ecab-4087-a59b-bc81db901676", 2023-05-20T01:25:42.313 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2023-05-20T01:25:42.313 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 7575, 2023-05-20T01:25:42.313 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2023-05-20T01:25:42.313 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2023-05-20T01:25:42.313 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2023-05-20T01:25:42.313 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 3.7036264710000002, 2023-05-20T01:25:42.313 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 70, 2023-05-20T01:25:42.314 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 97, 2023-05-20T01:25:42.314 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 97, 2023-05-20T01:25:42.314 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 4.531850522 2023-05-20T01:25:42.314 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-05-20T01:25:42.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.270+0000 7f4446554700 1 -- 172.21.15.16:0/690861708 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f442c06ace0 msgr2=0x7f442c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:42.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.270+0000 7f4446554700 1 --2- 172.21.15.16:0/690861708 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f442c06ace0 0x7f442c06d190 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f4430006b20 tx=0x7f4430008040 comp rx=0 tx=0).stop 2023-05-20T01:25:42.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.270+0000 7f4446554700 1 -- 172.21.15.16:0/690861708 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f444006d4b0 msgr2=0x7f4440160880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:42.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.270+0000 7f4446554700 1 --2- 172.21.15.16:0/690861708 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f444006d4b0 0x7f4440160880 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f443400e810 tx=0x7f443400eb20 comp rx=0 tx=0).stop 2023-05-20T01:25:42.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.270+0000 7f4446554700 1 -- 172.21.15.16:0/690861708 shutdown_connections 2023-05-20T01:25:42.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.270+0000 7f4446554700 1 --2- 172.21.15.16:0/690861708 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f442c06ace0 0x7f442c06d190 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:42.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.270+0000 7f4446554700 1 --2- 172.21.15.16:0/690861708 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f444006f450 0x7f44401ef800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:42.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.270+0000 7f4446554700 1 --2- 172.21.15.16:0/690861708 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f444006a510 0x7f44401f27b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:42.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.270+0000 7f4446554700 1 --2- 172.21.15.16:0/690861708 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f444006d4b0 0x7f4440160880 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:42.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.270+0000 7f4446554700 1 -- 172.21.15.16:0/690861708 >> 172.21.15.16:0/690861708 conn(0x7f44400cffb0 msgr2=0x7f44400d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:42.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.270+0000 7f4446554700 1 -- 172.21.15.16:0/690861708 shutdown_connections 2023-05-20T01:25:42.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.270+0000 7f4446554700 1 -- 172.21.15.16:0/690861708 wait complete. 2023-05-20T01:25:42.318 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-20T01:25:42.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.439+0000 7f72cbfc0700 1 Processor -- start 2023-05-20T01:25:42.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.439+0000 7f72cbfc0700 1 -- start start 2023-05-20T01:25:42.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.440+0000 7f72cbfc0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f72c4071ad0 0x7f72c41645b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:42.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.440+0000 7f72cbfc0700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72c40cb590 con 0x7f72c40ccec0 2023-05-20T01:25:42.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.440+0000 7f72cbfc0700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72c40ca9a0 con 0x7f72c4157190 2023-05-20T01:25:42.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.440+0000 7f72cbfc0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72c4071f50 con 0x7f72c4071ad0 2023-05-20T01:25:42.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.440+0000 7f72c955b700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f72c40ccec0 0x7f72c40713c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:48772/0 (socket says 172.21.15.16:48772) 2023-05-20T01:25:42.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.440+0000 7f72c955b700 1 -- 172.21.15.16:0/3099503763 learned_addr learned my addr 172.21.15.16:0/3099503763 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:25:42.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.440+0000 7f72ca55d700 1 --2- 172.21.15.16:0/3099503763 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f72c4071ad0 0x7f72c41645b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:42.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.440+0000 7f72c8d5a700 1 -- 172.21.15.16:0/3099503763 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1058328218 0 0) 0x7f72c40cb590 con 0x7f72c40ccec0 2023-05-20T01:25:42.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.440+0000 7f72c8d5a700 1 -- 172.21.15.16:0/3099503763 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f72b00057c0 con 0x7f72c40ccec0 2023-05-20T01:25:42.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.441+0000 7f72c8d5a700 1 -- 172.21.15.16:0/3099503763 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1844587644 0 0) 0x7f72b00057c0 con 0x7f72c40ccec0 2023-05-20T01:25:42.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.441+0000 7f72c8d5a700 1 -- 172.21.15.16:0/3099503763 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f72c4071ad0 msgr2=0x7f72c41645b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:42.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.441+0000 7f72c8d5a700 1 --2- 172.21.15.16:0/3099503763 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f72c4071ad0 0x7f72c41645b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:42.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.441+0000 7f72c8d5a700 1 -- 172.21.15.16:0/3099503763 >> v1:172.21.15.132:6789/0 conn(0x7f72c4157190 legacy=0x7f72c40cc7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:42.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.441+0000 7f72c8d5a700 1 -- 172.21.15.16:0/3099503763 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72c40cac60 con 0x7f72c40ccec0 2023-05-20T01:25:42.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.441+0000 7f72c8d5a700 1 -- 172.21.15.16:0/3099503763 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7f72b4001670 con 0x7f72c40ccec0 2023-05-20T01:25:42.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.441+0000 7f72c8d5a700 1 -- 172.21.15.16:0/3099503763 >> v1:172.21.15.16:6789/0 conn(0x7f72c40ccec0 legacy=0x7f72c40713c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:42.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.441+0000 7f72c8d5a700 1 --2- 172.21.15.16:0/3099503763 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72b0005cb0 0x7f72b00060c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:42.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.441+0000 7f72c8d5a700 1 --2- 172.21.15.16:0/3099503763 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f72c4071ad0 0x7f72b0006a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:42.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.441+0000 7f72c8d5a700 1 --2- 172.21.15.16:0/3099503763 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f72b0006fa0 0x7f72b0007410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:42.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.441+0000 7f72c8d5a700 1 -- 172.21.15.16:0/3099503763 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72c4071f50 con 0x7f72b0005cb0 2023-05-20T01:25:42.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.441+0000 7f72c8d5a700 1 -- 172.21.15.16:0/3099503763 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72b000dde0 con 0x7f72b0006fa0 2023-05-20T01:25:42.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.441+0000 7f72c8d5a700 1 -- 172.21.15.16:0/3099503763 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72b000dfa0 con 0x7f72c4071ad0 2023-05-20T01:25:42.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.441+0000 7f72c9d5c700 1 --2- 172.21.15.16:0/3099503763 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72b0005cb0 0x7f72b00060c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:42.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.441+0000 7f72c955b700 1 --2- 172.21.15.16:0/3099503763 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f72c4071ad0 0x7f72b0006a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:42.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.441+0000 7f72ca55d700 1 --2- 172.21.15.16:0/3099503763 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f72b0006fa0 0x7f72b0007410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:42.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.441+0000 7f72c9d5c700 1 -- 172.21.15.16:0/3099503763 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f72c4071ad0 msgr2=0x7f72b0006a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:42.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.441+0000 7f72c9d5c700 1 --2- 172.21.15.16:0/3099503763 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f72c4071ad0 0x7f72b0006a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:42.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.441+0000 7f72c9d5c700 1 -- 172.21.15.16:0/3099503763 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f72b0006fa0 msgr2=0x7f72b0007410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:42.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.441+0000 7f72c9d5c700 1 --2- 172.21.15.16:0/3099503763 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f72b0006fa0 0x7f72b0007410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:42.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.441+0000 7f72c9d5c700 1 -- 172.21.15.16:0/3099503763 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72b00057c0 con 0x7f72b0005cb0 2023-05-20T01:25:42.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.441+0000 7f72c9d5c700 1 --2- 172.21.15.16:0/3099503763 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72b0005cb0 0x7f72b00060c0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f72c0001800 tx=0x7f72b0007a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac53fa1249403d28 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:42.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.441+0000 7f72c8d5a700 1 -- 172.21.15.16:0/3099503763 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c00012f0 con 0x7f72b0005cb0 2023-05-20T01:25:42.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.441+0000 7f72c8d5a700 1 -- 172.21.15.16:0/3099503763 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72c00052f0 con 0x7f72b0005cb0 2023-05-20T01:25:42.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.441+0000 7f72c8d5a700 1 -- 172.21.15.16:0/3099503763 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c0006860 con 0x7f72b0005cb0 2023-05-20T01:25:42.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.442+0000 7f72cbfc0700 1 -- 172.21.15.16:0/3099503763 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72b0005cb0 msgr2=0x7f72b00060c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:42.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.442+0000 7f72cbfc0700 1 --2- 172.21.15.16:0/3099503763 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72b0005cb0 0x7f72b00060c0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f72c0001800 tx=0x7f72b0007a90 comp rx=0 tx=0).stop 2023-05-20T01:25:42.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.442+0000 7f72ca55d700 1 -- 172.21.15.16:0/3099503763 reap_dead start 2023-05-20T01:25:42.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.442+0000 7f72cbfc0700 1 -- 172.21.15.16:0/3099503763 shutdown_connections 2023-05-20T01:25:42.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.442+0000 7f72cbfc0700 1 -- 172.21.15.16:0/3099503763 >> 172.21.15.16:0/3099503763 conn(0x7f72c40cffb0 msgr2=0x7f72c40d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:42.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.442+0000 7f72cbfc0700 1 -- 172.21.15.16:0/3099503763 shutdown_connections 2023-05-20T01:25:42.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.442+0000 7f72cbfc0700 1 -- 172.21.15.16:0/3099503763 wait complete. 2023-05-20T01:25:42.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.443+0000 7f72cbfc0700 1 Processor -- start 2023-05-20T01:25:42.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.443+0000 7f72cbfc0700 1 -- start start 2023-05-20T01:25:42.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.444+0000 7f72cbfc0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72b0005cb0 0x7f72c41547c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:42.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.444+0000 7f72cbfc0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f72c40c44b0 0x7f72c414a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:42.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.444+0000 7f72cbfc0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f72c40c5f90 0x7f72c40c6400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:42.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.444+0000 7f72cbfc0700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72c40c4920 con 0x7f72b0005cb0 2023-05-20T01:25:42.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.444+0000 7f72cbfc0700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72c40c6940 con 0x7f72c40c44b0 2023-05-20T01:25:42.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.444+0000 7f72cbfc0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72c40c53e0 con 0x7f72c40c5f90 2023-05-20T01:25:42.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.444+0000 7f72c9d5c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72b0005cb0 0x7f72c41547c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:42.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.444+0000 7f72c9d5c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72b0005cb0 0x7f72c41547c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37256/0 (socket says 172.21.15.16:37256) 2023-05-20T01:25:42.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.444+0000 7f72c9d5c700 1 -- 172.21.15.16:0/1210818230 learned_addr learned my addr 172.21.15.16:0/1210818230 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:42.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.444+0000 7f72c955b700 1 --2- 172.21.15.16:0/1210818230 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f72c40c44b0 0x7f72c414a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:42.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.444+0000 7f72ca55d700 1 --2- 172.21.15.16:0/1210818230 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f72c40c5f90 0x7f72c40c6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:42.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.445+0000 7f72c9d5c700 1 -- 172.21.15.16:0/1210818230 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f72c40c5f90 msgr2=0x7f72c40c6400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:42.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.445+0000 7f72c9d5c700 1 --2- 172.21.15.16:0/1210818230 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f72c40c5f90 0x7f72c40c6400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:42.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.445+0000 7f72c9d5c700 1 -- 172.21.15.16:0/1210818230 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f72c40c44b0 msgr2=0x7f72c414a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:42.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.445+0000 7f72c9d5c700 1 --2- 172.21.15.16:0/1210818230 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f72c40c44b0 0x7f72c414a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:42.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.445+0000 7f72c9d5c700 1 -- 172.21.15.16:0/1210818230 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72c40cb220 con 0x7f72b0005cb0 2023-05-20T01:25:42.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.445+0000 7f72c9d5c700 1 --2- 172.21.15.16:0/1210818230 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72b0005cb0 0x7f72c41547c0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f72c0000ff0 tx=0x7f72c0001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:42.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.445+0000 7f72ba7fc700 1 -- 172.21.15.16:0/1210818230 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c0003880 con 0x7f72b0005cb0 2023-05-20T01:25:42.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.445+0000 7f72ba7fc700 1 -- 172.21.15.16:0/1210818230 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72c00039e0 con 0x7f72b0005cb0 2023-05-20T01:25:42.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.445+0000 7f72cbfc0700 1 -- 172.21.15.16:0/1210818230 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72c40caf40 con 0x7f72b0005cb0 2023-05-20T01:25:42.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.445+0000 7f72ba7fc700 1 -- 172.21.15.16:0/1210818230 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c0003d50 con 0x7f72b0005cb0 2023-05-20T01:25:42.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.445+0000 7f72cbfc0700 1 -- 172.21.15.16:0/1210818230 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72c4157560 con 0x7f72b0005cb0 2023-05-20T01:25:42.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.446+0000 7f72cbfc0700 1 -- 172.21.15.16:0/1210818230 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72c40747b0 con 0x7f72b0005cb0 2023-05-20T01:25:42.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.446+0000 7f72ba7fc700 1 -- 172.21.15.16:0/1210818230 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f72c001e450 con 0x7f72b0005cb0 2023-05-20T01:25:42.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.447+0000 7f72ba7fc700 1 --2- 172.21.15.16:0/1210818230 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f72a806ad10 0x7f72a806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:42.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.447+0000 7f72ba7fc700 1 -- 172.21.15.16:0/1210818230 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 7044+0+0 (secure 0 0 0) 0x7f72c008ab90 con 0x7f72b0005cb0 2023-05-20T01:25:42.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.447+0000 7f72c955b700 1 --2- 172.21.15.16:0/1210818230 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f72a806ad10 0x7f72a806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:42.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.447+0000 7f72c955b700 1 --2- 172.21.15.16:0/1210818230 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f72a806ad10 0x7f72a806d1c0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f72b4000fb0 tx=0x7f72b4007980 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:42.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.451+0000 7f72ba7fc700 1 -- 172.21.15.16:0/1210818230 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f72c0055970 con 0x7f72b0005cb0 2023-05-20T01:25:42.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.697+0000 7f72cbfc0700 1 -- 172.21.15.16:0/1210818230 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f72c40ce120 con 0x7f72b0005cb0 2023-05-20T01:25:42.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.698+0000 7f72ba7fc700 1 -- 172.21.15.16:0/1210818230 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) v1 ==== 94+0+4671 (secure 0 0 0) 0x7f72c0058f90 con 0x7f72b0005cb0 2023-05-20T01:25:42.702 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:25:42.703 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":70,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6323,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/59032067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":59032067},{"type":"v1","addr":"172.21.15.132:6834","nonce":59032067}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66},{"gid":6329,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/100162236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":100162236},{"type":"v1","addr":"172.21.15.132:6835","nonce":100162236}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66},{"gid":7572,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/3753560731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3753560731},{"type":"v1","addr":"172.21.15.16:6836","nonce":3753560731}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":70,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-20T01:25:32.295399+0000","modified":"2023-05-20T01:25:41.570561+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":95,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7575},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7575":{"gid":7575,"name":"c","rank":0,"incarnation":67,"state":"up:active","state_seq":4,"addr":"172.21.15.16:6837/307899603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":307899603},{"type":"v1","addr":"172.21.15.16:6837","nonce":307899603}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13,14],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-05-20T01:25:42.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.700+0000 7f72cbfc0700 1 -- 172.21.15.16:0/1210818230 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f72a806ad10 msgr2=0x7f72a806d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:42.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.700+0000 7f72cbfc0700 1 --2- 172.21.15.16:0/1210818230 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f72a806ad10 0x7f72a806d1c0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f72b4000fb0 tx=0x7f72b4007980 comp rx=0 tx=0).stop 2023-05-20T01:25:42.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.700+0000 7f72cbfc0700 1 -- 172.21.15.16:0/1210818230 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72b0005cb0 msgr2=0x7f72c41547c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:42.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.700+0000 7f72cbfc0700 1 --2- 172.21.15.16:0/1210818230 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72b0005cb0 0x7f72c41547c0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f72c0000ff0 tx=0x7f72c0001120 comp rx=0 tx=0).stop 2023-05-20T01:25:42.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.700+0000 7f72cbfc0700 1 -- 172.21.15.16:0/1210818230 shutdown_connections 2023-05-20T01:25:42.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.700+0000 7f72cbfc0700 1 --2- 172.21.15.16:0/1210818230 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f72a806ad10 0x7f72a806d1c0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:42.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.700+0000 7f72cbfc0700 1 --2- 172.21.15.16:0/1210818230 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f72c40c5f90 0x7f72c40c6400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:42.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.700+0000 7f72cbfc0700 1 --2- 172.21.15.16:0/1210818230 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72b0005cb0 0x7f72c41547c0 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:42.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.700+0000 7f72cbfc0700 1 --2- 172.21.15.16:0/1210818230 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f72c40c44b0 0x7f72c414a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:42.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.700+0000 7f72cbfc0700 1 -- 172.21.15.16:0/1210818230 >> 172.21.15.16:0/1210818230 conn(0x7f72c40cffb0 msgr2=0x7f72c4155f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:42.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.700+0000 7f72cbfc0700 1 -- 172.21.15.16:0/1210818230 shutdown_connections 2023-05-20T01:25:42.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.700+0000 7f72cbfc0700 1 -- 172.21.15.16:0/1210818230 wait complete. 2023-05-20T01:25:42.706 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 70 2023-05-20T01:25:42.717 INFO:tasks.cephfs.test_data_scan:{ "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 4, "epoch": 70, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 4, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-05-20T01:25:32.295399+0000", "damaged": [], "data_pools": [ 13, 14 ], "enabled": true, "epoch": 70, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_7575": { "addr": "172.21.15.16:6837/307899603", "addrs": { "addrvec": [ { "addr": "172.21.15.16:6835", "nonce": 307899603, "type": "v2" }, { "addr": "172.21.15.16:6837", "nonce": 307899603, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 7575, "incarnation": 67, "join_fscid": -1, "name": "c", "rank": 0, "state": "up:active", "state_seq": 4 } }, "last_failure": 0, "last_failure_osd_epoch": 95, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 12, "modified": "2023-05-20T01:25:41.570561+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 60, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 7575 } } } ], "standbys": [ { "addr": "172.21.15.132:6834/59032067", "addrs": { "addrvec": [ { "addr": "172.21.15.132:6832", "nonce": 59032067, "type": "v2" }, { "addr": "172.21.15.132:6834", "nonce": 59032067, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 66, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6323, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.132:6835/100162236", "addrs": { "addrvec": [ { "addr": "172.21.15.132:6833", "nonce": 100162236, "type": "v2" }, { "addr": "172.21.15.132:6835", "nonce": 100162236, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 66, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6329, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.16:6836/3753560731", "addrs": { "addrvec": [ { "addr": "172.21.15.16:6834", "nonce": 3753560731, "type": "v2" }, { "addr": "172.21.15.16:6836", "nonce": 3753560731, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 66, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 7572, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-05-20T01:25:42.718 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-05-20T01:25:42.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.839+0000 7f4ff1dcf700 1 Processor -- start 2023-05-20T01:25:42.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.839+0000 7f4ff1dcf700 1 -- start start 2023-05-20T01:25:42.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.840+0000 7f4ff1dcf700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4fec06d4b0 0x7f4fec0cf890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:42.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.840+0000 7f4ff1dcf700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4fec06a510 0x7f4fec06a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:42.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.840+0000 7f4ff1dcf700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fec0cc510 con 0x7f4fec06f450 2023-05-20T01:25:42.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.840+0000 7f4ff1dcf700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fec06aef0 con 0x7f4fec06a510 2023-05-20T01:25:42.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.840+0000 7f4ff1dcf700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fec071ed0 con 0x7f4fec06d4b0 2023-05-20T01:25:42.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.840+0000 7f4febfff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4fec06a510 0x7f4fec06a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:42.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.840+0000 7f4feb7fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f4fec06f450 0x7f4fec161080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:48792/0 (socket says 172.21.15.16:48792) 2023-05-20T01:25:42.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.840+0000 7f4feb7fe700 1 -- 172.21.15.16:0/2397161513 learned_addr learned my addr 172.21.15.16:0/2397161513 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:25:42.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.840+0000 7f4feaffd700 1 --2- 172.21.15.16:0/2397161513 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4fec06d4b0 0x7f4fec0cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:42.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.840+0000 7f4febfff700 1 -- 172.21.15.16:0/2397161513 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4fec06d4b0 msgr2=0x7f4fec0cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:42.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.840+0000 7f4febfff700 1 --2- 172.21.15.16:0/2397161513 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4fec06d4b0 0x7f4fec0cf890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:42.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.840+0000 7f4febfff700 1 -- 172.21.15.16:0/2397161513 >> v1:172.21.15.16:6789/0 conn(0x7f4fec06f450 legacy=0x7f4fec161080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:42.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.840+0000 7f4febfff700 1 -- 172.21.15.16:0/2397161513 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fec0721b0 con 0x7f4fec06a510 2023-05-20T01:25:42.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.840+0000 7f4fea7fc700 1 -- 172.21.15.16:0/2397161513 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 173717286 0 0) 0x7f4fec0cc510 con 0x7f4fec06f450 2023-05-20T01:25:42.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.840+0000 7f4feaffd700 1 --2- 172.21.15.16:0/2397161513 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4fec06d4b0 0x7f4fec0cf890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:25:42.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.840+0000 7f4febfff700 1 --2- 172.21.15.16:0/2397161513 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4fec06a510 0x7f4fec06a9b0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f4fe4009d90 tx=0x7f4fe400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc67b3b906f46cec server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:42.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.840+0000 7f4fea7fc700 1 -- 172.21.15.16:0/2397161513 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe400e070 con 0x7f4fec06a510 2023-05-20T01:25:42.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.840+0000 7f4fea7fc700 1 -- 172.21.15.16:0/2397161513 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fe400bbd0 con 0x7f4fec06a510 2023-05-20T01:25:42.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.841+0000 7f4fea7fc700 1 -- 172.21.15.16:0/2397161513 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe4012430 con 0x7f4fec06a510 2023-05-20T01:25:42.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.841+0000 7f4ff1dcf700 1 -- 172.21.15.16:0/2397161513 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4fec06a510 msgr2=0x7f4fec06a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:42.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.841+0000 7f4ff1dcf700 1 --2- 172.21.15.16:0/2397161513 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4fec06a510 0x7f4fec06a9b0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f4fe4009d90 tx=0x7f4fe400b330 comp rx=0 tx=0).stop 2023-05-20T01:25:42.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.841+0000 7f4ff1dcf700 1 -- 172.21.15.16:0/2397161513 shutdown_connections 2023-05-20T01:25:42.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.841+0000 7f4ff1dcf700 1 --2- 172.21.15.16:0/2397161513 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4fec06a510 0x7f4fec06a9b0 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:42.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.841+0000 7f4ff1dcf700 1 --2- 172.21.15.16:0/2397161513 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4fec06d4b0 0x7f4fec0cf890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:42.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.841+0000 7f4ff1dcf700 1 -- 172.21.15.16:0/2397161513 >> 172.21.15.16:0/2397161513 conn(0x7f4fec0cffb0 msgr2=0x7f4fec0d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:42.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.841+0000 7f4ff1dcf700 1 -- 172.21.15.16:0/2397161513 shutdown_connections 2023-05-20T01:25:42.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.841+0000 7f4ff1dcf700 1 -- 172.21.15.16:0/2397161513 wait complete. 2023-05-20T01:25:42.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.842+0000 7f4ff1dcf700 1 Processor -- start 2023-05-20T01:25:42.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.842+0000 7f4ff1dcf700 1 -- start start 2023-05-20T01:25:42.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.843+0000 7f4ff1dcf700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4fec06a510 0x7f4fec1e2910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:42.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.843+0000 7f4ff1dcf700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4fec06d4b0 0x7f4fec0b1240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:42.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.843+0000 7f4ff1dcf700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4fec06f450 0x7f4fec14e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:42.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.843+0000 7f4ff1dcf700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fec0b2690 con 0x7f4fec06d4b0 2023-05-20T01:25:42.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.843+0000 7f4ff1dcf700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fec1e2e50 con 0x7f4fec06a510 2023-05-20T01:25:42.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.843+0000 7f4ff1dcf700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fec0b1780 con 0x7f4fec06f450 2023-05-20T01:25:42.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.843+0000 7f4feaffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4fec06d4b0 0x7f4fec0b1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:42.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.843+0000 7f4feaffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4fec06d4b0 0x7f4fec0b1240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37272/0 (socket says 172.21.15.16:37272) 2023-05-20T01:25:42.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.843+0000 7f4feaffd700 1 -- 172.21.15.16:0/3877118139 learned_addr learned my addr 172.21.15.16:0/3877118139 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:42.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.843+0000 7f4febfff700 1 --2- 172.21.15.16:0/3877118139 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4fec06f450 0x7f4fec14e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:42.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.843+0000 7f4feb7fe700 1 --2- 172.21.15.16:0/3877118139 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4fec06a510 0x7f4fec1e2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:42.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.844+0000 7f4feaffd700 1 -- 172.21.15.16:0/3877118139 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4fec06f450 msgr2=0x7f4fec14e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:42.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.844+0000 7f4feaffd700 1 --2- 172.21.15.16:0/3877118139 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4fec06f450 0x7f4fec14e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:42.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.844+0000 7f4feaffd700 1 -- 172.21.15.16:0/3877118139 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4fec06a510 msgr2=0x7f4fec1e2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:42.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.844+0000 7f4feaffd700 1 --2- 172.21.15.16:0/3877118139 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4fec06a510 0x7f4fec1e2910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:42.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.844+0000 7f4feaffd700 1 -- 172.21.15.16:0/3877118139 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fe4009a70 con 0x7f4fec06d4b0 2023-05-20T01:25:42.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.844+0000 7f4febfff700 1 --2- 172.21.15.16:0/3877118139 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4fec06f450 0x7f4fec14e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:25:42.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.844+0000 7f4feb7fe700 1 --2- 172.21.15.16:0/3877118139 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4fec06a510 0x7f4fec1e2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:25:42.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.844+0000 7f4feaffd700 1 --2- 172.21.15.16:0/3877118139 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4fec06d4b0 0x7f4fec0b1240 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f4fe000c750 tx=0x7f4fe000cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:42.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.844+0000 7f4fd3fff700 1 -- 172.21.15.16:0/3877118139 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe0011800 con 0x7f4fec06d4b0 2023-05-20T01:25:42.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.844+0000 7f4fd3fff700 1 -- 172.21.15.16:0/3877118139 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fe0011960 con 0x7f4fec06d4b0 2023-05-20T01:25:42.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.844+0000 7f4ff1dcf700 1 -- 172.21.15.16:0/3877118139 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fec0cbe50 con 0x7f4fec06d4b0 2023-05-20T01:25:42.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.844+0000 7f4fd3fff700 1 -- 172.21.15.16:0/3877118139 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe0011cd0 con 0x7f4fec06d4b0 2023-05-20T01:25:42.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.844+0000 7f4ff1dcf700 1 -- 172.21.15.16:0/3877118139 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fec0cc1a0 con 0x7f4fec06d4b0 2023-05-20T01:25:42.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.845+0000 7f4ff1dcf700 1 -- 172.21.15.16:0/3877118139 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fec0cc980 con 0x7f4fec06d4b0 2023-05-20T01:25:42.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.845+0000 7f4fd3fff700 1 -- 172.21.15.16:0/3877118139 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4fe0021350 con 0x7f4fec06d4b0 2023-05-20T01:25:42.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.846+0000 7f4fd3fff700 1 --2- 172.21.15.16:0/3877118139 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f4fd806ace0 0x7f4fd806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:42.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.846+0000 7f4feb7fe700 1 --2- 172.21.15.16:0/3877118139 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f4fd806ace0 0x7f4fd806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:42.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.846+0000 7f4fd3fff700 1 -- 172.21.15.16:0/3877118139 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 7044+0+0 (secure 0 0 0) 0x7f4fe00897b0 con 0x7f4fec06d4b0 2023-05-20T01:25:42.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.846+0000 7f4feb7fe700 1 --2- 172.21.15.16:0/3877118139 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f4fd806ace0 0x7f4fd806d190 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc000f60 tx=0x7f4fdc007d30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:42.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:42.851+0000 7f4fd3fff700 1 -- 172.21.15.16:0/3877118139 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f4fe0057fa0 con 0x7f4fec06d4b0 2023-05-20T01:25:43.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:43.095+0000 7f4ff1dcf700 1 -- 172.21.15.16:0/3877118139 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f4fec0b27d0 con 0x7f4fec06d4b0 2023-05-20T01:25:43.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:43.095+0000 7f4fd3fff700 1 -- 172.21.15.16:0/3877118139 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v70) v1 ==== 53+0+91 (secure 0 0 0) 0x7f4fec0b27d0 con 0x7f4fec06d4b0 2023-05-20T01:25:43.100 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_extra ] 2023-05-20T01:25:43.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:43.097+0000 7f4ff1dcf700 1 -- 172.21.15.16:0/3877118139 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f4fd806ace0 msgr2=0x7f4fd806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:43.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:43.097+0000 7f4ff1dcf700 1 --2- 172.21.15.16:0/3877118139 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f4fd806ace0 0x7f4fd806d190 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc000f60 tx=0x7f4fdc007d30 comp rx=0 tx=0).stop 2023-05-20T01:25:43.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:43.097+0000 7f4ff1dcf700 1 -- 172.21.15.16:0/3877118139 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4fec06d4b0 msgr2=0x7f4fec0b1240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:43.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:43.097+0000 7f4ff1dcf700 1 --2- 172.21.15.16:0/3877118139 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4fec06d4b0 0x7f4fec0b1240 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f4fe000c750 tx=0x7f4fe000cb10 comp rx=0 tx=0).stop 2023-05-20T01:25:43.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:43.097+0000 7f4ff1dcf700 1 -- 172.21.15.16:0/3877118139 shutdown_connections 2023-05-20T01:25:43.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:43.097+0000 7f4ff1dcf700 1 --2- 172.21.15.16:0/3877118139 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f4fd806ace0 0x7f4fd806d190 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:43.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:43.097+0000 7f4ff1dcf700 1 --2- 172.21.15.16:0/3877118139 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4fec06f450 0x7f4fec14e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:43.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:43.097+0000 7f4ff1dcf700 1 --2- 172.21.15.16:0/3877118139 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4fec06a510 0x7f4fec1e2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:43.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:43.097+0000 7f4ff1dcf700 1 --2- 172.21.15.16:0/3877118139 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4fec06d4b0 0x7f4fec0b1240 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:43.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:43.097+0000 7f4ff1dcf700 1 -- 172.21.15.16:0/3877118139 >> 172.21.15.16:0/3877118139 conn(0x7f4fec0cffb0 msgr2=0x7f4fec0c98e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:43.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:43.097+0000 7f4ff1dcf700 1 -- 172.21.15.16:0/3877118139 shutdown_connections 2023-05-20T01:25:43.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:43.097+0000 7f4ff1dcf700 1 -- 172.21.15.16:0/3877118139 wait complete. 2023-05-20T01:25:43.114 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-05-20T01:25:43.114 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-05-20T01:25:43.115 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2023-05-20T01:25:43.115 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-05-20T01:25:43.116 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-05-20T01:25:43.116 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-05-20T01:25:43.117 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-05-20T01:25:43.117 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-05-20T01:25:43.118 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-05-20T01:25:43.119 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:25:43.119 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2023-05-20T01:25:43.138 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-05-20T01:25:43.139 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-05-20T01:25:43.139 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2023-05-20T01:25:43.140 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-05-20T01:25:43.140 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2023-05-20T01:25:43.140 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-05-20T01:25:43.141 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-05-20T01:25:43.141 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2023-05-20T01:25:43.141 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-05-20T01:25:43.142 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2023-05-20T01:25:43.142 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-05-20T01:25:43.142 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2023-05-20T01:25:43.142 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-05-20T01:25:43.142 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 41891sec preferred_lft 41891sec 2023-05-20T01:25:43.142 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link 2023-05-20T01:25:43.143 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-05-20T01:25:43.143 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-05-20T01:25:43.143 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2023-05-20T01:25:43.143 INFO:teuthology.orchestra.run.smithi016.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-05-20T01:25:43.143 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0a:2e:0c:6b:5d:42 brd ff:ff:ff:ff:ff:ff 2023-05-20T01:25:43.143 INFO:teuthology.orchestra.run.smithi016.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-05-20T01:25:43.143 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-05-20T01:25:43.144 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::7c00:b2ff:fef4:946/64 scope link 2023-05-20T01:25:43.144 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-05-20T01:25:43.144 INFO:teuthology.orchestra.run.smithi016.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-05-20T01:25:43.144 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether d2:e4:70:a7:b9:4b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-05-20T01:25:43.144 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::d0e4:70ff:fea7:b94b/64 scope link 2023-05-20T01:25:43.144 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-05-20T01:25:43.144 INFO:teuthology.orchestra.run.smithi016.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-05-20T01:25:43.145 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0a:2e:0c:6b:5d:42 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-05-20T01:25:43.145 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::82e:cff:fe6b:5d42/64 scope link 2023-05-20T01:25:43.145 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-05-20T01:25:43.146 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:25:43.147 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-05-20T01:25:43.205 INFO:teuthology.orchestra.run.smithi016.stdout:1 2023-05-20T01:25:43.206 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:25:43.207 DEBUG:teuthology.orchestra.run.smithi016:> route 2023-05-20T01:25:43.266 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2023-05-20T01:25:43.266 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-05-20T01:25:43.266 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-05-20T01:25:43.266 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-05-20T01:25:43.267 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-05-20T01:25:43.267 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:25:43.268 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2023-05-20T01:25:43.269 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2023-05-20T01:25:43.269 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-05-20T01:25:43.269 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-05-20T01:25:43.270 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-05-20T01:25:43.270 DEBUG:teuthology.orchestra.run.smithi016:> ') 2023-05-20T01:25:43.373 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:25:43.374 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2023-05-20T01:25:43.392 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-05-20T01:25:43.393 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-05-20T01:25:43.393 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-05-20T01:25:43.393 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:25:43.394 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.0 up 2023-05-20T01:25:43.461 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-05-20T01:25:43.461 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-05-20T01:25:43.518 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-05-20T01:25:43.518 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-05-20T01:25:43.518 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-05-20T01:25:43.571 INFO:tasks.ceph.mds.c.smithi016.stderr:2023-05-20T01:25:43.567+0000 7fd0cf362700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-05-20T01:25:43.575 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-05-20T01:25:43.575 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:25:43.576 DEBUG:teuthology.orchestra.run.smithi016:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-05-20T01:25:43.662 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-05-20T01:25:43.662 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-05-20T01:25:43.693 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-05-20T01:25:43.693 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-05-20T01:25:43.694 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.132:3300,172.21.15.132:3301" 2023-05-20T01:25:43.694 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: trying mount with new device syntax: 0@c2245cc5-ecab-4087-a59b-bc81db901676.cephfs=/ 2023-05-20T01:25:43.694 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.16:3300/172.21.15.132:3300/172.21.15.132:3301" will pass to kernel 2023-05-20T01:25:43.694 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: v2 mount support check returned 2 2023-05-20T01:25:43.694 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: kernel does not support v2 syntax 2023-05-20T01:25:43.694 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: switching to using v1 address with old syntax 2023-05-20T01:25:43.768 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-05-20T01:25:43.769 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-05-20T01:25:43.769 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.132:3300,172.21.15.132:3301" 2023-05-20T01:25:43.769 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: trying mount with new device syntax: 0@c2245cc5-ecab-4087-a59b-bc81db901676.cephfs=/ 2023-05-20T01:25:43.769 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.16:3300/172.21.15.132:3300/172.21.15.132:3301" will pass to kernel 2023-05-20T01:25:43.769 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: v2 mount support check returned 2 2023-05-20T01:25:43.769 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: kernel does not support v2 syntax 2023-05-20T01:25:43.769 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: switching to using v1 address with old syntax 2023-05-20T01:25:43.770 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:6789,172.21.15.132:6789,172.21.15.132:6790" 2023-05-20T01:25:43.770 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.16:6789,172.21.15.132:6789,172.21.15.132:6790:/ 2023-05-20T01:25:43.770 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=c2245cc5-ecab-4087-a59b-bc81db901676" will pass to kernel 2023-05-20T01:25:43.771 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-05-20T01:25:43.772 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:25:43.772 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-05-20T01:25:43.800 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:25:43.801 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-05-20T01:25:43.882 INFO:teuthology.orchestra.run.smithi016.stdout:client6377 2023-05-20T01:25:43.882 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-05-20T01:25:43.883 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2023-05-20T01:25:43.940 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:0 scrub start / repair,recursive 2023-05-20T01:25:44.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.106+0000 7f362afc3700 1 Processor -- start 2023-05-20T01:25:44.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.106+0000 7f362afc3700 1 -- start start 2023-05-20T01:25:44.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.107+0000 7f362afc3700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f362415ced0 0x7f36240b1f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:44.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.107+0000 7f362afc3700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f36240d09b0 0x7f362406b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:44.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.107+0000 7f362afc3700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f362406bb60 0x7f36240ced70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:44.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.107+0000 7f362afc3700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f362406bfe0 con 0x7f362406bb60 2023-05-20T01:25:44.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.107+0000 7f362afc3700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36240b4770 con 0x7f36240d09b0 2023-05-20T01:25:44.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.107+0000 7f362afc3700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36240b2820 con 0x7f362415ced0 2023-05-20T01:25:44.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.107+0000 7f3629560700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f362406bb60 0x7f36240ced70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:44.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.107+0000 7f3629560700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f362406bb60 0x7f36240ced70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37286/0 (socket says 172.21.15.16:37286) 2023-05-20T01:25:44.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.107+0000 7f3629560700 1 -- 172.21.15.16:0/984953615 learned_addr learned my addr 172.21.15.16:0/984953615 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:44.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.107+0000 7f3628d5f700 1 --2- 172.21.15.16:0/984953615 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f362415ced0 0x7f36240b1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:44.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.107+0000 7f3623fff700 1 --2- 172.21.15.16:0/984953615 >> v2:172.21.15.132:3300/0 conn(0x7f36240d09b0 0x7f362406b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:44.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.107+0000 7f3629560700 1 -- 172.21.15.16:0/984953615 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f362415ced0 msgr2=0x7f36240b1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:44.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.107+0000 7f3629560700 1 --2- 172.21.15.16:0/984953615 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f362415ced0 0x7f36240b1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:44.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.107+0000 7f3629560700 1 -- 172.21.15.16:0/984953615 >> v2:172.21.15.132:3300/0 conn(0x7f36240d09b0 msgr2=0x7f362406b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:44.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.107+0000 7f3629560700 1 --2- 172.21.15.16:0/984953615 >> v2:172.21.15.132:3300/0 conn(0x7f36240d09b0 0x7f362406b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:44.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.107+0000 7f3629560700 1 -- 172.21.15.16:0/984953615 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36240b2540 con 0x7f362406bb60 2023-05-20T01:25:44.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.107+0000 7f3629560700 1 --2- 172.21.15.16:0/984953615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f362406bb60 0x7f36240ced70 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f3624071aa0 tx=0x7f361c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61be42a09d48c4 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:44.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.107+0000 7f36237fe700 1 -- 172.21.15.16:0/984953615 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f361c00e070 con 0x7f362406bb60 2023-05-20T01:25:44.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.108+0000 7f36237fe700 1 -- 172.21.15.16:0/984953615 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f361c00b9c0 con 0x7f362406bb60 2023-05-20T01:25:44.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.108+0000 7f36237fe700 1 -- 172.21.15.16:0/984953615 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f361c00bd70 con 0x7f362406bb60 2023-05-20T01:25:44.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.108+0000 7f362afc3700 1 -- 172.21.15.16:0/984953615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f362406bb60 msgr2=0x7f36240ced70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:44.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.108+0000 7f362afc3700 1 --2- 172.21.15.16:0/984953615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f362406bb60 0x7f36240ced70 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f3624071aa0 tx=0x7f361c00b330 comp rx=0 tx=0).stop 2023-05-20T01:25:44.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.108+0000 7f362afc3700 1 -- 172.21.15.16:0/984953615 shutdown_connections 2023-05-20T01:25:44.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.108+0000 7f362afc3700 1 --2- 172.21.15.16:0/984953615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f362406bb60 0x7f36240ced70 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:44.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.108+0000 7f362afc3700 1 --2- 172.21.15.16:0/984953615 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f362415ced0 0x7f36240b1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:44.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.108+0000 7f362afc3700 1 --2- 172.21.15.16:0/984953615 >> v2:172.21.15.132:3300/0 conn(0x7f36240d09b0 0x7f362406b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:44.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.108+0000 7f362afc3700 1 -- 172.21.15.16:0/984953615 >> 172.21.15.16:0/984953615 conn(0x7f362409a470 msgr2=0x7f36240d11a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:44.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.108+0000 7f362afc3700 1 -- 172.21.15.16:0/984953615 shutdown_connections 2023-05-20T01:25:44.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.108+0000 7f362afc3700 1 -- 172.21.15.16:0/984953615 wait complete. 2023-05-20T01:25:44.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.109+0000 7f362afc3700 1 Processor -- start 2023-05-20T01:25:44.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.109+0000 7f362afc3700 1 -- start start 2023-05-20T01:25:44.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.110+0000 7f362afc3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f362406bb60 0x7f362415ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:44.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.110+0000 7f362afc3700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f36240d09b0 0x7f362415f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:44.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.110+0000 7f362afc3700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f362415ced0 0x7f36241f5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:44.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.110+0000 7f362afc3700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36241f69b0 con 0x7f362406bb60 2023-05-20T01:25:44.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.110+0000 7f362afc3700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36240c8310 con 0x7f36240d09b0 2023-05-20T01:25:44.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.110+0000 7f362afc3700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36240c85f0 con 0x7f362415ced0 2023-05-20T01:25:44.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.110+0000 7f3628d5f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f362406bb60 0x7f362415ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:44.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.110+0000 7f3628d5f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f362406bb60 0x7f362415ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37288/0 (socket says 172.21.15.16:37288) 2023-05-20T01:25:44.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.110+0000 7f3628d5f700 1 -- 172.21.15.16:0/66063473 learned_addr learned my addr 172.21.15.16:0/66063473 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:44.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.110+0000 7f3623fff700 1 --2- 172.21.15.16:0/66063473 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f36240d09b0 0x7f362415f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:44.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.110+0000 7f3629560700 1 --2- 172.21.15.16:0/66063473 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f362415ced0 0x7f36241f5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:44.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.111+0000 7f3628d5f700 1 -- 172.21.15.16:0/66063473 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f362415ced0 msgr2=0x7f36241f5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:44.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.111+0000 7f3628d5f700 1 --2- 172.21.15.16:0/66063473 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f362415ced0 0x7f36241f5ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:44.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.111+0000 7f3628d5f700 1 -- 172.21.15.16:0/66063473 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f36240d09b0 msgr2=0x7f362415f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:44.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.111+0000 7f3628d5f700 1 --2- 172.21.15.16:0/66063473 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f36240d09b0 0x7f362415f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:44.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.111+0000 7f3628d5f700 1 -- 172.21.15.16:0/66063473 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f361c009b30 con 0x7f362406bb60 2023-05-20T01:25:44.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.111+0000 7f3628d5f700 1 --2- 172.21.15.16:0/66063473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f362406bb60 0x7f362415ee70 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f361800e810 tx=0x7f361800eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:44.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.111+0000 7f36217fa700 1 -- 172.21.15.16:0/66063473 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f361800c070 con 0x7f362406bb60 2023-05-20T01:25:44.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.111+0000 7f36217fa700 1 -- 172.21.15.16:0/66063473 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3618013610 con 0x7f362406bb60 2023-05-20T01:25:44.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.111+0000 7f362afc3700 1 -- 172.21.15.16:0/66063473 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36240b4400 con 0x7f362406bb60 2023-05-20T01:25:44.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.111+0000 7f36217fa700 1 -- 172.21.15.16:0/66063473 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36180139c0 con 0x7f362406bb60 2023-05-20T01:25:44.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.111+0000 7f362afc3700 1 -- 172.21.15.16:0/66063473 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36240b4120 con 0x7f362406bb60 2023-05-20T01:25:44.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.112+0000 7f36217fa700 1 -- 172.21.15.16:0/66063473 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3618013b20 con 0x7f362406bb60 2023-05-20T01:25:44.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.113+0000 7f36217fa700 1 --2- 172.21.15.16:0/66063473 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f360806ad10 0x7f360806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:44.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.113+0000 7f3623fff700 1 --2- 172.21.15.16:0/66063473 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f360806ad10 0x7f360806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:44.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.113+0000 7f36217fa700 1 -- 172.21.15.16:0/66063473 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 7044+0+0 (secure 0 0 0) 0x7f3618089580 con 0x7f362406bb60 2023-05-20T01:25:44.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.114+0000 7f3623fff700 1 --2- 172.21.15.16:0/66063473 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f360806ad10 0x7f360806d1c0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f362415b2e0 tx=0x7f3610008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:44.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.115+0000 7f363714fb80 1 Processor -- start 2023-05-20T01:25:44.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.115+0000 7f363714fb80 1 -- start start 2023-05-20T01:25:44.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.116+0000 7f363714fb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562038bab4d0 0x562038bab8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:44.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.116+0000 7f363714fb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562038ba77d0 0x562038b80530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:44.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.116+0000 7f363714fb80 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x562038b7e940 0x562038b80a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:44.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.116+0000 7f363714fb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x562038a860e0 con 0x562038ba77d0 2023-05-20T01:25:44.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.116+0000 7f363714fb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x562038ad8920 con 0x562038bab4d0 2023-05-20T01:25:44.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.116+0000 7f363714fb80 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x562038ad1db0 con 0x562038b7e940 2023-05-20T01:25:44.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.116+0000 7f3623fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562038ba77d0 0x562038b80530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:44.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.116+0000 7f3623fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562038ba77d0 0x562038b80530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37298/0 (socket says 172.21.15.16:37298) 2023-05-20T01:25:44.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.116+0000 7f3623fff700 1 -- 172.21.15.16:0/2787164251 learned_addr learned my addr 172.21.15.16:0/2787164251 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:44.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.116+0000 7f3628d5f700 1 --2- 172.21.15.16:0/2787164251 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562038bab4d0 0x562038bab8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:44.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.116+0000 7f3623fff700 1 -- 172.21.15.16:0/2787164251 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x562038b7e940 msgr2=0x562038b80a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:44.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.116+0000 7f3623fff700 1 --2- 172.21.15.16:0/2787164251 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x562038b7e940 0x562038b80a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:44.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.116+0000 7f3623fff700 1 -- 172.21.15.16:0/2787164251 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562038bab4d0 msgr2=0x562038bab8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:44.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.116+0000 7f3623fff700 1 --2- 172.21.15.16:0/2787164251 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562038bab4d0 0x562038bab8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:44.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.116+0000 7f3623fff700 1 -- 172.21.15.16:0/2787164251 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5620389536b0 con 0x562038ba77d0 2023-05-20T01:25:44.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.116+0000 7f3623fff700 1 --2- 172.21.15.16:0/2787164251 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562038ba77d0 0x562038b80530 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f361000b7c0 tx=0x7f36100046a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a78bc63674d7e84e server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:44.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.117+0000 7f360e7fc700 1 -- 172.21.15.16:0/2787164251 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36100167d0 con 0x562038ba77d0 2023-05-20T01:25:44.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.117+0000 7f360e7fc700 1 -- 172.21.15.16:0/2787164251 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f361001a070 con 0x562038ba77d0 2023-05-20T01:25:44.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.117+0000 7f360e7fc700 1 -- 172.21.15.16:0/2787164251 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3610016b30 con 0x562038ba77d0 2023-05-20T01:25:44.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.117+0000 7f363714fb80 1 -- 172.21.15.16:0/2787164251 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562038ba77d0 msgr2=0x562038b80530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:44.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.117+0000 7f363714fb80 1 --2- 172.21.15.16:0/2787164251 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562038ba77d0 0x562038b80530 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f361000b7c0 tx=0x7f36100046a0 comp rx=0 tx=0).stop 2023-05-20T01:25:44.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.117+0000 7f363714fb80 1 -- 172.21.15.16:0/2787164251 shutdown_connections 2023-05-20T01:25:44.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.117+0000 7f363714fb80 1 --2- 172.21.15.16:0/2787164251 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x562038b7e940 0x562038b80a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:44.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.117+0000 7f363714fb80 1 --2- 172.21.15.16:0/2787164251 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562038bab4d0 0x562038bab8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:44.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.117+0000 7f363714fb80 1 --2- 172.21.15.16:0/2787164251 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562038ba77d0 0x562038b80530 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:44.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.117+0000 7f363714fb80 1 -- 172.21.15.16:0/2787164251 >> 172.21.15.16:0/2787164251 conn(0x562038b76000 msgr2=0x562038bc62e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:44.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.117+0000 7f363714fb80 1 -- 172.21.15.16:0/2787164251 shutdown_connections 2023-05-20T01:25:44.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.117+0000 7f363714fb80 1 -- 172.21.15.16:0/2787164251 wait complete. 2023-05-20T01:25:44.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.118+0000 7f363714fb80 1 -- start start 2023-05-20T01:25:44.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.118+0000 7f363714fb80 1 Processor -- start 2023-05-20T01:25:44.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.119+0000 7f363714fb80 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x562038b7e940 0x562038ba45c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:44.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.119+0000 7f363714fb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562038ba77d0 0x562038bc9f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:44.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.119+0000 7f363714fb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562038bab4d0 0x562038ba5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:44.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.119+0000 7f363714fb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x562038a73960 con 0x562038ba77d0 2023-05-20T01:25:44.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.119+0000 7f363714fb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x562038a73600 con 0x562038bab4d0 2023-05-20T01:25:44.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.119+0000 7f363714fb80 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x5620389559a0 con 0x562038b7e940 2023-05-20T01:25:44.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.119+0000 7f363714fb80 10 client.0 _ll_register_callbacks cb 0x562038a91800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-05-20T01:25:44.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.119+0000 7f3623fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562038ba77d0 0x562038bc9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:44.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.119+0000 7f3623fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562038ba77d0 0x562038bc9f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37304/0 (socket says 172.21.15.16:37304) 2023-05-20T01:25:44.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.119+0000 7f3623fff700 1 -- 172.21.15.16:0/2196069147 learned_addr learned my addr 172.21.15.16:0/2196069147 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:44.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.119+0000 7f3628d5f700 1 --2- 172.21.15.16:0/2196069147 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x562038b7e940 0x562038ba45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:44.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.119+0000 7f3629560700 1 --2- 172.21.15.16:0/2196069147 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562038bab4d0 0x562038ba5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:44.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.119+0000 7f3623fff700 1 -- 172.21.15.16:0/2196069147 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x562038b7e940 msgr2=0x562038ba45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:44.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.119+0000 7f3623fff700 1 --2- 172.21.15.16:0/2196069147 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x562038b7e940 0x562038ba45c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:44.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.119+0000 7f3623fff700 1 -- 172.21.15.16:0/2196069147 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562038bab4d0 msgr2=0x562038ba5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:44.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.119+0000 7f3623fff700 1 --2- 172.21.15.16:0/2196069147 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562038bab4d0 0x562038ba5d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:44.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.119+0000 7f3623fff700 1 -- 172.21.15.16:0/2196069147 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f361000e040 con 0x562038ba77d0 2023-05-20T01:25:44.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.119+0000 7f3628d5f700 1 --2- 172.21.15.16:0/2196069147 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x562038b7e940 0x562038ba45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:25:44.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.119+0000 7f3623fff700 1 --2- 172.21.15.16:0/2196069147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562038ba77d0 0x562038bc9f40 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f361000b730 tx=0x7f36100047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c0f3afb6797243a server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:44.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.119+0000 7f36037fe700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-05-20T01:25:44.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.120+0000 7f36037fe700 1 -- 172.21.15.16:0/2196069147 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f361000a030 con 0x562038ba77d0 2023-05-20T01:25:44.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.120+0000 7f36037fe700 1 -- 172.21.15.16:0/2196069147 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 7044+0+0 (secure 0 0 0) 0x7f3610021650 con 0x562038ba77d0 2023-05-20T01:25:44.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.120+0000 7f363714fb80 1 -- 172.21.15.16:0/2196069147 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562038b737c0 con 0x562038ba77d0 2023-05-20T01:25:44.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.120+0000 7f36037fe700 1 -- 172.21.15.16:0/2196069147 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3610021810 con 0x562038ba77d0 2023-05-20T01:25:44.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.120+0000 7f363714fb80 10 client.7644 fetch_fsmap learned FSMap version 70 2023-05-20T01:25:44.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.120+0000 7f363714fb80 1 -- 172.21.15.16:0/2196069147 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=2+}) v3 -- 0x562038a98210 con 0x562038ba77d0 2023-05-20T01:25:44.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.120+0000 7f36037fe700 1 -- 172.21.15.16:0/2196069147 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562038b737c0 con 0x562038ba77d0 2023-05-20T01:25:44.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.120+0000 7f36037fe700 1 -- 172.21.15.16:0/2196069147 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 70) v1 ==== 3122+0+0 (secure 0 0 0) 0x7f3610031020 con 0x562038ba77d0 2023-05-20T01:25:44.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.120+0000 7f363714fb80 10 client.7644 fetch_fsmap finished waiting for FSMap version 70 2023-05-20T01:25:44.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.120+0000 7f363714fb80 10 client.7644 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2023-05-20T01:25:44.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.120+0000 7f363714fb80 20 client.7644 populate_metadata read hostname 'smithi016' 2023-05-20T01:25:44.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.120+0000 7f363714fb80 1 --2- 172.21.15.16:0/2196069147 >> [v2:172.21.15.16:6835/307899603,v1:172.21.15.16:6837/307899603] conn(0x562038bc9070 0x562038ba6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:44.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.120+0000 7f363714fb80 4 client.7644 mds_command: new command op to 7575 tid=0[{"prefix": "get_command_descriptions"}] 2023-05-20T01:25:44.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.120+0000 7f363714fb80 1 -- 172.21.15.16:0/2196069147 --> [v2:172.21.15.16:6835/307899603,v1:172.21.15.16:6837/307899603] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562038a94120 con 0x562038bc9070 2023-05-20T01:25:44.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.120+0000 7f3628d5f700 1 --2- 172.21.15.16:0/2196069147 >> [v2:172.21.15.16:6835/307899603,v1:172.21.15.16:6837/307899603] conn(0x562038bc9070 0x562038ba6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:44.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.121+0000 7f3628d5f700 1 --2- 172.21.15.16:0/2196069147 >> [v2:172.21.15.16:6835/307899603,v1:172.21.15.16:6837/307899603] conn(0x562038bc9070 0x562038ba6240 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb969186c63d38 server_cookie=c703c473f4a18229 in_seq=0 out_seq=0 2023-05-20T01:25:44.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.121+0000 7f36037fe700 10 client.7644 ms_handle_connect on v2:172.21.15.16:6835/307899603 2023-05-20T01:25:44.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.121+0000 7f36037fe700 1 -- 172.21.15.16:0/2196069147 <== mds.0 v2:172.21.15.16:6835/307899603 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x562038a94120 con 0x562038bc9070 2023-05-20T01:25:44.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.121+0000 7f36037fe700 10 client.7644 handle_command_reply: tid=0 2023-05-20T01:25:44.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.121+0000 7f363714fb80 1 client.7644 shutdown 2023-05-20T01:25:44.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.121+0000 7f363714fb80 1 -- 172.21.15.16:0/2196069147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562038ba77d0 msgr2=0x562038bc9f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:44.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.121+0000 7f363714fb80 1 --2- 172.21.15.16:0/2196069147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562038ba77d0 0x562038bc9f40 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f361000b730 tx=0x7f36100047c0 comp rx=0 tx=0).stop 2023-05-20T01:25:44.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.121+0000 7f363714fb80 1 -- 172.21.15.16:0/2196069147 shutdown_connections 2023-05-20T01:25:44.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.121+0000 7f363714fb80 1 --2- 172.21.15.16:0/2196069147 >> [v2:172.21.15.16:6835/307899603,v1:172.21.15.16:6837/307899603] conn(0x562038bc9070 0x562038ba6240 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:44.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.121+0000 7f363714fb80 1 --2- 172.21.15.16:0/2196069147 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562038bab4d0 0x562038ba5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:44.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.121+0000 7f363714fb80 1 --2- 172.21.15.16:0/2196069147 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x562038b7e940 0x562038ba45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:44.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.121+0000 7f36037fe700 0 client.7644 ms_handle_reset on v2:172.21.15.16:6835/307899603 2023-05-20T01:25:44.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.121+0000 7f363714fb80 1 --2- 172.21.15.16:0/2196069147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562038ba77d0 0x562038bc9f40 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:44.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.121+0000 7f363714fb80 1 -- 172.21.15.16:0/2196069147 >> 172.21.15.16:0/2196069147 conn(0x562038b76000 msgr2=0x562038b7c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:44.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.122+0000 7f363714fb80 1 -- 172.21.15.16:0/2196069147 shutdown_connections 2023-05-20T01:25:44.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.122+0000 7f363714fb80 1 -- 172.21.15.16:0/2196069147 wait complete. 2023-05-20T01:25:44.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.122+0000 7f363714fb80 20 client.7644 trim_cache size 0 max 16384 2023-05-20T01:25:44.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.139+0000 7f363714fb80 1 Processor -- start 2023-05-20T01:25:44.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.139+0000 7f363714fb80 1 -- start start 2023-05-20T01:25:44.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.139+0000 7f363714fb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562038b7e940 0x562038bab9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:44.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.139+0000 7f363714fb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562038ba77d0 0x562038ba43d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:44.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.139+0000 7f363714fb80 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x562038bab4d0 0x562038ba5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:44.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.139+0000 7f363714fb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x562038bae8e0 con 0x562038ba77d0 2023-05-20T01:25:44.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.139+0000 7f363714fb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x562038bc6b60 con 0x562038b7e940 2023-05-20T01:25:44.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.139+0000 7f363714fb80 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x562038b7c280 con 0x562038bab4d0 2023-05-20T01:25:44.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.139+0000 7f3623fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562038ba77d0 0x562038ba43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:44.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.140+0000 7f3623fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562038ba77d0 0x562038ba43d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37312/0 (socket says 172.21.15.16:37312) 2023-05-20T01:25:44.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.140+0000 7f3623fff700 1 -- 172.21.15.16:0/2249332257 learned_addr learned my addr 172.21.15.16:0/2249332257 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:44.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.140+0000 7f3628d5f700 1 --2- 172.21.15.16:0/2249332257 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562038b7e940 0x562038bab9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:44.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.140+0000 7f3629560700 1 --2- 172.21.15.16:0/2249332257 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x562038bab4d0 0x562038ba5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:44.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.140+0000 7f3623fff700 1 -- 172.21.15.16:0/2249332257 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x562038bab4d0 msgr2=0x562038ba5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:44.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.140+0000 7f3623fff700 1 --2- 172.21.15.16:0/2249332257 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x562038bab4d0 0x562038ba5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:44.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.140+0000 7f3623fff700 1 -- 172.21.15.16:0/2249332257 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562038b7e940 msgr2=0x562038bab9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:44.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.140+0000 7f3623fff700 1 --2- 172.21.15.16:0/2249332257 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562038b7e940 0x562038bab9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:44.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.140+0000 7f3623fff700 1 -- 172.21.15.16:0/2249332257 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562038a97f60 con 0x562038ba77d0 2023-05-20T01:25:44.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.140+0000 7f3629560700 1 --2- 172.21.15.16:0/2249332257 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x562038bab4d0 0x562038ba5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:25:44.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.140+0000 7f3623fff700 1 --2- 172.21.15.16:0/2249332257 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562038ba77d0 0x562038ba43d0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f3610003e60 tx=0x7f361000b050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2472d3b24b85f023 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:44.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.140+0000 7f360effd700 1 -- 172.21.15.16:0/2249332257 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f361000a040 con 0x562038ba77d0 2023-05-20T01:25:44.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.140+0000 7f360effd700 1 -- 172.21.15.16:0/2249332257 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3610016e00 con 0x562038ba77d0 2023-05-20T01:25:44.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.140+0000 7f360effd700 1 -- 172.21.15.16:0/2249332257 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3610021d70 con 0x562038ba77d0 2023-05-20T01:25:44.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.141+0000 7f363714fb80 1 -- 172.21.15.16:0/2249332257 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562038ba77d0 msgr2=0x562038ba43d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:44.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.141+0000 7f363714fb80 1 --2- 172.21.15.16:0/2249332257 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562038ba77d0 0x562038ba43d0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f3610003e60 tx=0x7f361000b050 comp rx=0 tx=0).stop 2023-05-20T01:25:44.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.141+0000 7f363714fb80 1 -- 172.21.15.16:0/2249332257 shutdown_connections 2023-05-20T01:25:44.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.141+0000 7f363714fb80 1 --2- 172.21.15.16:0/2249332257 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x562038bab4d0 0x562038ba5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:44.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.141+0000 7f363714fb80 1 --2- 172.21.15.16:0/2249332257 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562038b7e940 0x562038bab9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:44.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.141+0000 7f363714fb80 1 --2- 172.21.15.16:0/2249332257 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562038ba77d0 0x562038ba43d0 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:44.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.141+0000 7f363714fb80 1 -- 172.21.15.16:0/2249332257 >> 172.21.15.16:0/2249332257 conn(0x562038b76000 msgr2=0x562038bbd1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:44.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.141+0000 7f363714fb80 1 -- 172.21.15.16:0/2249332257 shutdown_connections 2023-05-20T01:25:44.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.141+0000 7f363714fb80 1 -- 172.21.15.16:0/2249332257 wait complete. 2023-05-20T01:25:44.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.141+0000 7f363714fb80 1 -- start start 2023-05-20T01:25:44.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.142+0000 7f363714fb80 1 Processor -- start 2023-05-20T01:25:44.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.142+0000 7f363714fb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562038b7e940 0x562038c2e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:44.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.142+0000 7f363714fb80 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x562038ba77d0 0x562038b81050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:44.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.142+0000 7f363714fb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562038bab4d0 0x562038bc9490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:44.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.142+0000 7f363714fb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x562038c2d960 con 0x562038b7e940 2023-05-20T01:25:44.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.142+0000 7f363714fb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x562038c31b60 con 0x562038bab4d0 2023-05-20T01:25:44.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.142+0000 7f363714fb80 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x562038bd7b10 con 0x562038ba77d0 2023-05-20T01:25:44.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.142+0000 7f363714fb80 10 client.0 _ll_register_callbacks cb 0x7f361000bf90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-05-20T01:25:44.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.142+0000 7f3628d5f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562038b7e940 0x562038c2e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:44.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.142+0000 7f3628d5f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562038b7e940 0x562038c2e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37316/0 (socket says 172.21.15.16:37316) 2023-05-20T01:25:44.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.142+0000 7f3628d5f700 1 -- 172.21.15.16:0/1450280845 learned_addr learned my addr 172.21.15.16:0/1450280845 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:44.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.143+0000 7f3623fff700 1 --2- 172.21.15.16:0/1450280845 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x562038ba77d0 0x562038b81050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:44.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.143+0000 7f3629560700 1 --2- 172.21.15.16:0/1450280845 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562038bab4d0 0x562038bc9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:44.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.143+0000 7f3628d5f700 1 -- 172.21.15.16:0/1450280845 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x562038ba77d0 msgr2=0x562038b81050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:44.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.143+0000 7f3628d5f700 1 --2- 172.21.15.16:0/1450280845 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x562038ba77d0 0x562038b81050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:44.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.143+0000 7f3628d5f700 1 -- 172.21.15.16:0/1450280845 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562038bab4d0 msgr2=0x562038bc9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:44.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.143+0000 7f3628d5f700 1 --2- 172.21.15.16:0/1450280845 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562038bab4d0 0x562038bc9490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:44.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.143+0000 7f3628d5f700 1 -- 172.21.15.16:0/1450280845 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562038a98210 con 0x562038b7e940 2023-05-20T01:25:44.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.143+0000 7f3629560700 1 --2- 172.21.15.16:0/1450280845 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562038bab4d0 0x562038bc9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:25:44.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.143+0000 7f3628d5f700 1 --2- 172.21.15.16:0/1450280845 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562038b7e940 0x562038c2e340 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f361800e7e0 tx=0x7f3618004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bbe1d596dfaff22 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:44.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.143+0000 7f36037fe700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-05-20T01:25:44.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.143+0000 7f36037fe700 1 -- 172.21.15.16:0/1450280845 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f361805fde0 con 0x562038b7e940 2023-05-20T01:25:44.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.143+0000 7f36037fe700 1 -- 172.21.15.16:0/1450280845 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 7044+0+0 (secure 0 0 0) 0x7f361805ff40 con 0x562038b7e940 2023-05-20T01:25:44.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.143+0000 7f363714fb80 1 -- 172.21.15.16:0/1450280845 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f361001a070 con 0x562038b7e940 2023-05-20T01:25:44.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.143+0000 7f36037fe700 1 -- 172.21.15.16:0/1450280845 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7f36180871a0 con 0x562038b7e940 2023-05-20T01:25:44.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.143+0000 7f363714fb80 10 client.7650 fetch_fsmap learned FSMap version 70 2023-05-20T01:25:44.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.143+0000 7f363714fb80 1 -- 172.21.15.16:0/1450280845 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=2+}) v3 -- 0x5620388e8ce0 con 0x562038b7e940 2023-05-20T01:25:44.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.143+0000 7f36037fe700 1 -- 172.21.15.16:0/1450280845 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f361001a070 con 0x562038b7e940 2023-05-20T01:25:44.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.143+0000 7f36037fe700 1 -- 172.21.15.16:0/1450280845 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 70) v1 ==== 3122+0+0 (secure 0 0 0) 0x7f36180890b0 con 0x562038b7e940 2023-05-20T01:25:44.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.143+0000 7f363714fb80 10 client.7650 fetch_fsmap finished waiting for FSMap version 70 2023-05-20T01:25:44.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.143+0000 7f363714fb80 10 client.7650 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2023-05-20T01:25:44.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.143+0000 7f363714fb80 20 client.7650 populate_metadata read hostname 'smithi016' 2023-05-20T01:25:44.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.143+0000 7f363714fb80 1 --2- 172.21.15.16:0/1450280845 >> [v2:172.21.15.16:6835/307899603,v1:172.21.15.16:6837/307899603] conn(0x562038c16db0 0x562038bc99d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:44.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.143+0000 7f363714fb80 4 client.7650 mds_command: new command op to 7575 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2023-05-20T01:25:44.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.143+0000 7f363714fb80 1 -- 172.21.15.16:0/1450280845 --> [v2:172.21.15.16:6835/307899603,v1:172.21.15.16:6837/307899603] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) v1 -- 0x562038aa06e0 con 0x562038c16db0 2023-05-20T01:25:44.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.144+0000 7f3623fff700 1 --2- 172.21.15.16:0/1450280845 >> [v2:172.21.15.16:6835/307899603,v1:172.21.15.16:6837/307899603] conn(0x562038c16db0 0x562038bc99d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:44.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.144+0000 7f36037fe700 10 client.7650 ms_handle_connect on v2:172.21.15.16:6835/307899603 2023-05-20T01:25:44.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.144+0000 7f3623fff700 1 --2- 172.21.15.16:0/1450280845 >> [v2:172.21.15.16:6835/307899603,v1:172.21.15.16:6837/307899603] conn(0x562038c16db0 0x562038bc99d0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c1122e361cedeb4 server_cookie=a3123e08f9073e10 in_seq=0 out_seq=0 2023-05-20T01:25:44.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.144+0000 7f36037fe700 1 -- 172.21.15.16:0/1450280845 <== mds.0 v2:172.21.15.16:6835/307899603 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x562038aa06e0 con 0x562038c16db0 2023-05-20T01:25:44.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.144+0000 7f36037fe700 10 client.7650 handle_command_reply: tid=0 2023-05-20T01:25:44.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.144+0000 7f363714fb80 1 client.7650 shutdown 2023-05-20T01:25:44.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.145+0000 7f363714fb80 1 -- 172.21.15.16:0/1450280845 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562038b7e940 msgr2=0x562038c2e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:44.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.145+0000 7f363714fb80 1 --2- 172.21.15.16:0/1450280845 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562038b7e940 0x562038c2e340 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f361800e7e0 tx=0x7f3618004750 comp rx=0 tx=0).stop 2023-05-20T01:25:44.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.145+0000 7f363714fb80 1 -- 172.21.15.16:0/1450280845 shutdown_connections 2023-05-20T01:25:44.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.145+0000 7f363714fb80 1 --2- 172.21.15.16:0/1450280845 >> [v2:172.21.15.16:6835/307899603,v1:172.21.15.16:6837/307899603] conn(0x562038c16db0 0x562038bc99d0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:44.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.145+0000 7f363714fb80 1 --2- 172.21.15.16:0/1450280845 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562038bab4d0 0x562038bc9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:44.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.145+0000 7f363714fb80 1 --2- 172.21.15.16:0/1450280845 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562038b7e940 0x562038c2e340 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:44.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.145+0000 7f36037fe700 0 client.7650 ms_handle_reset on v2:172.21.15.16:6835/307899603 2023-05-20T01:25:44.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.145+0000 7f363714fb80 1 --2- 172.21.15.16:0/1450280845 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x562038ba77d0 0x562038b81050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:44.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.145+0000 7f363714fb80 1 -- 172.21.15.16:0/1450280845 >> 172.21.15.16:0/1450280845 conn(0x562038b76000 msgr2=0x562038ba6d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:44.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.145+0000 7f363714fb80 1 -- 172.21.15.16:0/1450280845 shutdown_connections 2023-05-20T01:25:44.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.145+0000 7f363714fb80 1 -- 172.21.15.16:0/1450280845 wait complete. 2023-05-20T01:25:44.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.145+0000 7f363714fb80 20 client.7650 trim_cache size 0 max 16384 2023-05-20T01:25:44.160 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-05-20T01:25:44.160 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2023-05-20T01:25:44.160 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "633f2d59-6072-47a0-8866-7a1dffeed959", 2023-05-20T01:25:44.160 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2023-05-20T01:25:44.160 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-05-20T01:25:44.161 INFO:tasks.ceph.mds.c.smithi016.stderr:2023-05-20T01:25:44.145+0000 7fd0cbb5b700 -1 log_channel(cluster) log [ERR] : bad backtrace on directory inode 0x10000000000 2023-05-20T01:25:44.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.146+0000 7f362afc3700 1 -- 172.21.15.16:0/66063473 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f360806ad10 msgr2=0x7f360806d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:44.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.146+0000 7f362afc3700 1 --2- 172.21.15.16:0/66063473 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f360806ad10 0x7f360806d1c0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f362415b2e0 tx=0x7f3610008040 comp rx=0 tx=0).stop 2023-05-20T01:25:44.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.146+0000 7f362afc3700 1 -- 172.21.15.16:0/66063473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f362406bb60 msgr2=0x7f362415ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:44.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.146+0000 7f362afc3700 1 --2- 172.21.15.16:0/66063473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f362406bb60 0x7f362415ee70 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f361800e810 tx=0x7f361800eb20 comp rx=0 tx=0).stop 2023-05-20T01:25:44.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.146+0000 7f362afc3700 1 -- 172.21.15.16:0/66063473 shutdown_connections 2023-05-20T01:25:44.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.146+0000 7f362afc3700 1 --2- 172.21.15.16:0/66063473 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f360806ad10 0x7f360806d1c0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:44.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.146+0000 7f362afc3700 1 --2- 172.21.15.16:0/66063473 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f362415ced0 0x7f36241f5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:44.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.146+0000 7f362afc3700 1 --2- 172.21.15.16:0/66063473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f362406bb60 0x7f362415ee70 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:44.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.146+0000 7f362afc3700 1 --2- 172.21.15.16:0/66063473 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f36240d09b0 0x7f362415f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:44.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.146+0000 7f362afc3700 1 -- 172.21.15.16:0/66063473 >> 172.21.15.16:0/66063473 conn(0x7f362409a470 msgr2=0x7f362415a940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:44.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.146+0000 7f362afc3700 1 -- 172.21.15.16:0/66063473 shutdown_connections 2023-05-20T01:25:44.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.146+0000 7f362afc3700 1 -- 172.21.15.16:0/66063473 wait complete. 2023-05-20T01:25:44.164 INFO:tasks.ceph.mds.c.smithi016.stderr:2023-05-20T01:25:44.146+0000 7fd0cbb5b700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /subdir/ auth v1 pv4 ap=3 DIRTYPARENT f(v0 1=1+0) n() (inest lock dirty) (ifile lock->sync w=1 flushing) (iversion lock) | dirtyscattered=2 lock=1 dirfrag=1 dirtyparent=1 waiter=1 authpin=1 scrubqueue=0 0x55cf68973b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(12)0x10000000000:[<0x1/subdir v1>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 1=1+0)","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 1=1+0)","memory_value.rstat":"n()","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-61} 2023-05-20T01:25:44.164 INFO:tasks.ceph.mds.c.smithi016.stderr:2023-05-20T01:25:44.147+0000 7fd0cbb5b700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /subdir/sixmegs auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55cf689bb700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(14)0x10000000001:[<0x10000000000/sixmegs v10>,<0x1/subdir v12>]//[13]","memoryvalue":"(14)0x10000000001:[<0x10000000000/sixmegs v1>,<0x1/subdir v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-05-20T01:25:44.164 INFO:tasks.ceph.mds.c.smithi016.stderr:2023-05-20T01:25:44.154+0000 7fd0cbb5b700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1 [...2,head] / auth v5 ap=5 snaprealm=0x55cf6896ad80 DIRTYPARENT f(v1 1=0+1) n(v0 rc2023-05-20T01:25:43.794316+0000 1=0+1)/n() (idft sync r=1) (inest sync r=1) (ifile sync->mix r=1) (iversion lock) caps={6377=pLsXs/-@45} | dirtyscattered=0 request=0 lock=3 dirfrag=1 caps=1 openingsnapparents=0 dirtyparent=1 dirty=1 waiter=1 authpin=1 scrubqueue=0 0x55cf68973080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(12)0x1:[]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 1=0+1)","ondisk_value.rstat":"n(v0 2=0+2)","memory_value.dirstat":"f(v1 1=0+1)","memory_value.rstat":"n(v0 rc2023-05-20T01:25:43.794316+0000 1=0+1)","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-61} 2023-05-20T01:25:44.165 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:0 scrub status 2023-05-20T01:25:44.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.287+0000 7eff8803a700 1 Processor -- start 2023-05-20T01:25:44.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.287+0000 7eff8803a700 1 -- start start 2023-05-20T01:25:44.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.288+0000 7eff8803a700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7eff800cd070 0x7eff8006e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:44.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.288+0000 7eff8803a700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7eff800710d0 0x7eff800cabb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:44.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.288+0000 7eff8803a700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff80072430 con 0x7eff8015cf40 2023-05-20T01:25:44.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.288+0000 7eff8803a700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff80071820 con 0x7eff800cd070 2023-05-20T01:25:44.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.288+0000 7eff8803a700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff80071b00 con 0x7eff800710d0 2023-05-20T01:25:44.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.288+0000 7eff855d5700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7eff800cd070 0x7eff8006e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:44.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.288+0000 7eff85dd6700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7eff8015cf40 0x7eff800cc960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:48854/0 (socket says 172.21.15.16:48854) 2023-05-20T01:25:44.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.288+0000 7eff85dd6700 1 -- 172.21.15.16:0/4056318562 learned_addr learned my addr 172.21.15.16:0/4056318562 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:25:44.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.289+0000 7eff865d7700 1 --2- 172.21.15.16:0/4056318562 >> v2:172.21.15.132:3300/0 conn(0x7eff800710d0 0x7eff800cabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:44.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.289+0000 7eff855d5700 1 -- 172.21.15.16:0/4056318562 >> v2:172.21.15.132:3300/0 conn(0x7eff800710d0 msgr2=0x7eff800cabb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:44.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.289+0000 7eff855d5700 1 --2- 172.21.15.16:0/4056318562 >> v2:172.21.15.132:3300/0 conn(0x7eff800710d0 0x7eff800cabb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:44.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.289+0000 7eff855d5700 1 -- 172.21.15.16:0/4056318562 >> v1:172.21.15.16:6789/0 conn(0x7eff8015cf40 legacy=0x7eff800cc960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:44.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.289+0000 7eff855d5700 1 -- 172.21.15.16:0/4056318562 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff80071de0 con 0x7eff800cd070 2023-05-20T01:25:44.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.289+0000 7eff84dd4700 1 -- 172.21.15.16:0/4056318562 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3237797071 0 0) 0x7eff80072430 con 0x7eff8015cf40 2023-05-20T01:25:44.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.289+0000 7eff865d7700 1 --2- 172.21.15.16:0/4056318562 >> v2:172.21.15.132:3300/0 conn(0x7eff800710d0 0x7eff800cabb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:25:44.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.289+0000 7eff855d5700 1 --2- 172.21.15.16:0/4056318562 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff800cd070 0x7eff8006e020 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7eff8006f060 tx=0x7eff7c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac76bd479d310b43 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:44.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.289+0000 7eff84dd4700 1 -- 172.21.15.16:0/4056318562 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff7c011830 con 0x7eff800cd070 2023-05-20T01:25:44.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.289+0000 7eff84dd4700 1 -- 172.21.15.16:0/4056318562 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff7c011990 con 0x7eff800cd070 2023-05-20T01:25:44.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.289+0000 7eff84dd4700 1 -- 172.21.15.16:0/4056318562 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff7c011d20 con 0x7eff800cd070 2023-05-20T01:25:44.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.290+0000 7eff8803a700 1 -- 172.21.15.16:0/4056318562 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff800cd070 msgr2=0x7eff8006e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:44.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.290+0000 7eff8803a700 1 --2- 172.21.15.16:0/4056318562 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff800cd070 0x7eff8006e020 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7eff8006f060 tx=0x7eff7c009b20 comp rx=0 tx=0).stop 2023-05-20T01:25:44.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.290+0000 7eff8803a700 1 -- 172.21.15.16:0/4056318562 shutdown_connections 2023-05-20T01:25:44.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.290+0000 7eff8803a700 1 --2- 172.21.15.16:0/4056318562 >> v2:172.21.15.132:3300/0 conn(0x7eff800710d0 0x7eff800cabb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:44.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.290+0000 7eff8803a700 1 --2- 172.21.15.16:0/4056318562 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff800cd070 0x7eff8006e020 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:44.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.290+0000 7eff8803a700 1 -- 172.21.15.16:0/4056318562 >> 172.21.15.16:0/4056318562 conn(0x7eff8006c260 msgr2=0x7eff800cffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:44.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.290+0000 7eff8803a700 1 -- 172.21.15.16:0/4056318562 shutdown_connections 2023-05-20T01:25:44.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.290+0000 7eff8803a700 1 -- 172.21.15.16:0/4056318562 wait complete. 2023-05-20T01:25:44.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.291+0000 7eff8803a700 1 Processor -- start 2023-05-20T01:25:44.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.291+0000 7eff8803a700 1 -- start start 2023-05-20T01:25:44.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.292+0000 7eff8803a700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7eff800710d0 0x7eff80150160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:44.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.292+0000 7eff8803a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff800cd070 0x7eff801e3470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:44.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.292+0000 7eff8803a700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7eff8015cf40 0x7eff800c41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:44.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.292+0000 7eff8803a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff8015b430 con 0x7eff800cd070 2023-05-20T01:25:44.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.292+0000 7eff8803a700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff800c92b0 con 0x7eff800710d0 2023-05-20T01:25:44.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.292+0000 7eff8803a700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff800c9590 con 0x7eff8015cf40 2023-05-20T01:25:44.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.292+0000 7eff855d5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff800cd070 0x7eff801e3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:44.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.292+0000 7eff855d5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff800cd070 0x7eff801e3470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37334/0 (socket says 172.21.15.16:37334) 2023-05-20T01:25:44.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.292+0000 7eff855d5700 1 -- 172.21.15.16:0/1452700985 learned_addr learned my addr 172.21.15.16:0/1452700985 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:44.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.293+0000 7eff85dd6700 1 --2- 172.21.15.16:0/1452700985 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7eff800710d0 0x7eff80150160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:44.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.293+0000 7eff865d7700 1 --2- 172.21.15.16:0/1452700985 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7eff8015cf40 0x7eff800c41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:44.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.293+0000 7eff855d5700 1 -- 172.21.15.16:0/1452700985 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7eff8015cf40 msgr2=0x7eff800c41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:44.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.293+0000 7eff855d5700 1 --2- 172.21.15.16:0/1452700985 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7eff8015cf40 0x7eff800c41c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:44.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.293+0000 7eff855d5700 1 -- 172.21.15.16:0/1452700985 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7eff800710d0 msgr2=0x7eff80150160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:44.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.293+0000 7eff855d5700 1 --2- 172.21.15.16:0/1452700985 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7eff800710d0 0x7eff80150160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:44.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.293+0000 7eff855d5700 1 -- 172.21.15.16:0/1452700985 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff7c0094a0 con 0x7eff800cd070 2023-05-20T01:25:44.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.293+0000 7eff855d5700 1 --2- 172.21.15.16:0/1452700985 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff800cd070 0x7eff801e3470 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7eff7c000c00 tx=0x7eff7c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:44.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.293+0000 7eff767fc700 1 -- 172.21.15.16:0/1452700985 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff7c0036a0 con 0x7eff800cd070 2023-05-20T01:25:44.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.293+0000 7eff767fc700 1 -- 172.21.15.16:0/1452700985 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff7c01f070 con 0x7eff800cd070 2023-05-20T01:25:44.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.293+0000 7eff767fc700 1 -- 172.21.15.16:0/1452700985 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff7c003a20 con 0x7eff800cd070 2023-05-20T01:25:44.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.293+0000 7eff8803a700 1 -- 172.21.15.16:0/1452700985 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff80071510 con 0x7eff800cd070 2023-05-20T01:25:44.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.293+0000 7eff8803a700 1 -- 172.21.15.16:0/1452700985 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff800720c0 con 0x7eff800cd070 2023-05-20T01:25:44.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.295+0000 7eff767fc700 1 -- 172.21.15.16:0/1452700985 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7eff7c003b80 con 0x7eff800cd070 2023-05-20T01:25:44.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.295+0000 7eff767fc700 1 --2- 172.21.15.16:0/1452700985 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7eff6406ad10 0x7eff6406d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:44.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.295+0000 7eff767fc700 1 -- 172.21.15.16:0/1452700985 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 7044+0+0 (secure 0 0 0) 0x7eff7c021070 con 0x7eff800cd070 2023-05-20T01:25:44.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.295+0000 7eff85dd6700 1 --2- 172.21.15.16:0/1452700985 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7eff6406ad10 0x7eff6406d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:44.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.296+0000 7eff85dd6700 1 --2- 172.21.15.16:0/1452700985 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7eff6406ad10 0x7eff6406d1c0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7eff70000f60 tx=0x7eff70007e70 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:44.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.297+0000 7eff941c6b80 1 Processor -- start 2023-05-20T01:25:44.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.297+0000 7eff941c6b80 1 -- start start 2023-05-20T01:25:44.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.298+0000 7eff941c6b80 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ebbb677eb0 0x55ebbb6976e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:44.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.298+0000 7eff941c6b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ebbb64e8c0 0x55ebbb678280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:44.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.298+0000 7eff941c6b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ebbb6504b0 0x55ebbb650920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:44.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.298+0000 7eff941c6b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebbb556060 con 0x55ebbb64e8c0 2023-05-20T01:25:44.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.298+0000 7eff941c6b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebbb5a88a0 con 0x55ebbb6504b0 2023-05-20T01:25:44.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.298+0000 7eff941c6b80 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ebbb5a1d30 con 0x55ebbb677eb0 2023-05-20T01:25:44.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.298+0000 7eff855d5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ebbb64e8c0 0x55ebbb678280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:44.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.298+0000 7eff855d5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ebbb64e8c0 0x55ebbb678280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37342/0 (socket says 172.21.15.16:37342) 2023-05-20T01:25:44.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.298+0000 7eff855d5700 1 -- 172.21.15.16:0/442792548 learned_addr learned my addr 172.21.15.16:0/442792548 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:44.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.298+0000 7eff85dd6700 1 --2- 172.21.15.16:0/442792548 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ebbb677eb0 0x55ebbb6976e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:44.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.298+0000 7eff865d7700 1 --2- 172.21.15.16:0/442792548 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ebbb6504b0 0x55ebbb650920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:44.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.299+0000 7eff855d5700 1 -- 172.21.15.16:0/442792548 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ebbb677eb0 msgr2=0x55ebbb6976e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:44.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.299+0000 7eff855d5700 1 --2- 172.21.15.16:0/442792548 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ebbb677eb0 0x55ebbb6976e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:44.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.299+0000 7eff855d5700 1 -- 172.21.15.16:0/442792548 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ebbb6504b0 msgr2=0x55ebbb650920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:44.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.299+0000 7eff855d5700 1 --2- 172.21.15.16:0/442792548 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ebbb6504b0 0x55ebbb650920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:44.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.299+0000 7eff855d5700 1 -- 172.21.15.16:0/442792548 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ebbb423630 con 0x55ebbb64e8c0 2023-05-20T01:25:44.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.299+0000 7eff865d7700 1 --2- 172.21.15.16:0/442792548 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ebbb6504b0 0x55ebbb650920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:25:44.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.299+0000 7eff855d5700 1 --2- 172.21.15.16:0/442792548 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ebbb64e8c0 0x55ebbb678280 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7eff7c088a80 tx=0x7eff7c058e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0a657842ee1a123 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:44.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.299+0000 7eff637fe700 1 -- 172.21.15.16:0/442792548 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff7c05d470 con 0x55ebbb64e8c0 2023-05-20T01:25:44.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.299+0000 7eff637fe700 1 -- 172.21.15.16:0/442792548 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff7c061800 con 0x55ebbb64e8c0 2023-05-20T01:25:44.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.299+0000 7eff637fe700 1 -- 172.21.15.16:0/442792548 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff7c05d170 con 0x55ebbb64e8c0 2023-05-20T01:25:44.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.300+0000 7eff941c6b80 1 -- 172.21.15.16:0/442792548 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ebbb64e8c0 msgr2=0x55ebbb678280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:44.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.300+0000 7eff941c6b80 1 --2- 172.21.15.16:0/442792548 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ebbb64e8c0 0x55ebbb678280 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7eff7c088a80 tx=0x7eff7c058e90 comp rx=0 tx=0).stop 2023-05-20T01:25:44.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.300+0000 7eff941c6b80 1 -- 172.21.15.16:0/442792548 shutdown_connections 2023-05-20T01:25:44.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.300+0000 7eff941c6b80 1 --2- 172.21.15.16:0/442792548 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ebbb6504b0 0x55ebbb650920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:44.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.300+0000 7eff941c6b80 1 --2- 172.21.15.16:0/442792548 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ebbb677eb0 0x55ebbb6976e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:44.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.300+0000 7eff941c6b80 1 --2- 172.21.15.16:0/442792548 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ebbb64e8c0 0x55ebbb678280 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:44.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.300+0000 7eff941c6b80 1 -- 172.21.15.16:0/442792548 >> 172.21.15.16:0/442792548 conn(0x55ebbb645f80 msgr2=0x55ebbb696260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:44.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.300+0000 7eff941c6b80 1 -- 172.21.15.16:0/442792548 shutdown_connections 2023-05-20T01:25:44.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.300+0000 7eff941c6b80 1 -- 172.21.15.16:0/442792548 wait complete. 2023-05-20T01:25:44.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.300+0000 7eff941c6b80 1 -- start start 2023-05-20T01:25:44.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.300+0000 7eff941c6b80 1 Processor -- start 2023-05-20T01:25:44.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.301+0000 7eff941c6b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ebbb64e8c0 0x55ebbb674420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:44.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.301+0000 7eff941c6b80 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ebbb6504b0 0x55ebbb67b920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:44.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.301+0000 7eff941c6b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ebbb677eb0 0x55ebbb67d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:44.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.301+0000 7eff941c6b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebbb5438e0 con 0x55ebbb64e8c0 2023-05-20T01:25:44.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.301+0000 7eff941c6b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebbb543580 con 0x55ebbb677eb0 2023-05-20T01:25:44.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.301+0000 7eff941c6b80 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ebbb425920 con 0x55ebbb6504b0 2023-05-20T01:25:44.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.301+0000 7eff941c6b80 10 client.0 _ll_register_callbacks cb 0x55ebbb561780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-05-20T01:25:44.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.301+0000 7eff85dd6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ebbb64e8c0 0x55ebbb674420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:44.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.301+0000 7eff85dd6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ebbb64e8c0 0x55ebbb674420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37346/0 (socket says 172.21.15.16:37346) 2023-05-20T01:25:44.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.301+0000 7eff85dd6700 1 -- 172.21.15.16:0/2331982584 learned_addr learned my addr 172.21.15.16:0/2331982584 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:44.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.301+0000 7eff865d7700 1 --2- 172.21.15.16:0/2331982584 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ebbb677eb0 0x55ebbb67d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:44.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.301+0000 7eff855d5700 1 --2- 172.21.15.16:0/2331982584 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ebbb6504b0 0x55ebbb67b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:44.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.302+0000 7eff85dd6700 1 -- 172.21.15.16:0/2331982584 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ebbb6504b0 msgr2=0x55ebbb67b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:44.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.302+0000 7eff85dd6700 1 --2- 172.21.15.16:0/2331982584 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ebbb6504b0 0x55ebbb67b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:44.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.302+0000 7eff85dd6700 1 -- 172.21.15.16:0/2331982584 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ebbb677eb0 msgr2=0x55ebbb67d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:44.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.302+0000 7eff85dd6700 1 --2- 172.21.15.16:0/2331982584 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ebbb677eb0 0x55ebbb67d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:44.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.302+0000 7eff85dd6700 1 -- 172.21.15.16:0/2331982584 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff800720c0 con 0x55ebbb64e8c0 2023-05-20T01:25:44.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.302+0000 7eff85dd6700 1 --2- 172.21.15.16:0/2331982584 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ebbb64e8c0 0x55ebbb674420 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7eff70001310 tx=0x7eff70000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de7b216251b082ac server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:44.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.302+0000 7eff60ff9700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-05-20T01:25:44.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.302+0000 7eff60ff9700 1 -- 172.21.15.16:0/2331982584 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff700134e0 con 0x55ebbb64e8c0 2023-05-20T01:25:44.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.302+0000 7eff60ff9700 1 -- 172.21.15.16:0/2331982584 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 7044+0+0 (secure 0 0 0) 0x7eff7001b070 con 0x55ebbb64e8c0 2023-05-20T01:25:44.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.302+0000 7eff941c6b80 1 -- 172.21.15.16:0/2331982584 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ebbb643740 con 0x55ebbb64e8c0 2023-05-20T01:25:44.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.302+0000 7eff60ff9700 1 -- 172.21.15.16:0/2331982584 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7eff70001960 con 0x55ebbb64e8c0 2023-05-20T01:25:44.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.302+0000 7eff941c6b80 10 client.7665 fetch_fsmap learned FSMap version 70 2023-05-20T01:25:44.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.302+0000 7eff941c6b80 1 -- 172.21.15.16:0/2331982584 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=2+}) v3 -- 0x55ebbb568190 con 0x55ebbb64e8c0 2023-05-20T01:25:44.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.302+0000 7eff60ff9700 1 -- 172.21.15.16:0/2331982584 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ebbb643740 con 0x55ebbb64e8c0 2023-05-20T01:25:44.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.302+0000 7eff60ff9700 1 -- 172.21.15.16:0/2331982584 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 70) v1 ==== 3122+0+0 (secure 0 0 0) 0x7eff70001ac0 con 0x55ebbb64e8c0 2023-05-20T01:25:44.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.302+0000 7eff941c6b80 10 client.7665 fetch_fsmap finished waiting for FSMap version 70 2023-05-20T01:25:44.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.302+0000 7eff941c6b80 10 client.7665 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2023-05-20T01:25:44.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.302+0000 7eff941c6b80 20 client.7665 populate_metadata read hostname 'smithi016' 2023-05-20T01:25:44.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.302+0000 7eff941c6b80 1 --2- 172.21.15.16:0/2331982584 >> [v2:172.21.15.16:6835/307899603,v1:172.21.15.16:6837/307899603] conn(0x55ebbb6aa4b0 0x55ebbb67a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:44.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.302+0000 7eff941c6b80 4 client.7665 mds_command: new command op to 7575 tid=0[{"prefix": "get_command_descriptions"}] 2023-05-20T01:25:44.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.302+0000 7eff941c6b80 1 -- 172.21.15.16:0/2331982584 --> [v2:172.21.15.16:6835/307899603,v1:172.21.15.16:6837/307899603] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ebbb5640a0 con 0x55ebbb6aa4b0 2023-05-20T01:25:44.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.303+0000 7eff855d5700 1 --2- 172.21.15.16:0/2331982584 >> [v2:172.21.15.16:6835/307899603,v1:172.21.15.16:6837/307899603] conn(0x55ebbb6aa4b0 0x55ebbb67a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:44.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.303+0000 7eff855d5700 1 --2- 172.21.15.16:0/2331982584 >> [v2:172.21.15.16:6835/307899603,v1:172.21.15.16:6837/307899603] conn(0x55ebbb6aa4b0 0x55ebbb67a200 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b469c0ac442cbd9d server_cookie=f2aa03d2251d5bc0 in_seq=0 out_seq=0 2023-05-20T01:25:44.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.303+0000 7eff60ff9700 10 client.7665 ms_handle_connect on v2:172.21.15.16:6835/307899603 2023-05-20T01:25:44.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.303+0000 7eff60ff9700 1 -- 172.21.15.16:0/2331982584 <== mds.0 v2:172.21.15.16:6835/307899603 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ebbb5640a0 con 0x55ebbb6aa4b0 2023-05-20T01:25:44.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.303+0000 7eff60ff9700 10 client.7665 handle_command_reply: tid=0 2023-05-20T01:25:44.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.303+0000 7eff941c6b80 1 client.7665 shutdown 2023-05-20T01:25:44.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.304+0000 7eff941c6b80 1 -- 172.21.15.16:0/2331982584 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ebbb64e8c0 msgr2=0x55ebbb674420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:44.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.304+0000 7eff941c6b80 1 --2- 172.21.15.16:0/2331982584 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ebbb64e8c0 0x55ebbb674420 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7eff70001310 tx=0x7eff70000ed0 comp rx=0 tx=0).stop 2023-05-20T01:25:44.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.304+0000 7eff941c6b80 1 -- 172.21.15.16:0/2331982584 shutdown_connections 2023-05-20T01:25:44.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.304+0000 7eff941c6b80 1 --2- 172.21.15.16:0/2331982584 >> [v2:172.21.15.16:6835/307899603,v1:172.21.15.16:6837/307899603] conn(0x55ebbb6aa4b0 0x55ebbb67a200 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:44.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.304+0000 7eff941c6b80 1 --2- 172.21.15.16:0/2331982584 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ebbb677eb0 0x55ebbb67d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:44.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.304+0000 7eff941c6b80 1 --2- 172.21.15.16:0/2331982584 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ebbb64e8c0 0x55ebbb674420 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:44.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.304+0000 7eff941c6b80 1 --2- 172.21.15.16:0/2331982584 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ebbb6504b0 0x55ebbb67b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:44.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.304+0000 7eff60ff9700 0 client.7665 ms_handle_reset on v2:172.21.15.16:6835/307899603 2023-05-20T01:25:44.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.304+0000 7eff941c6b80 1 -- 172.21.15.16:0/2331982584 >> 172.21.15.16:0/2331982584 conn(0x55ebbb645f80 msgr2=0x55ebbb6997e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:44.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.304+0000 7eff941c6b80 1 -- 172.21.15.16:0/2331982584 shutdown_connections 2023-05-20T01:25:44.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.304+0000 7eff941c6b80 1 -- 172.21.15.16:0/2331982584 wait complete. 2023-05-20T01:25:44.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.304+0000 7eff941c6b80 20 client.7665 trim_cache size 0 max 16384 2023-05-20T01:25:44.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.321+0000 7eff941c6b80 1 Processor -- start 2023-05-20T01:25:44.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.321+0000 7eff941c6b80 1 -- start start 2023-05-20T01:25:44.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.322+0000 7eff941c6b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ebbb64e8c0 0x55ebbb64ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:44.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.322+0000 7eff941c6b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ebbb6504b0 0x55ebbb6a9810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:44.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.322+0000 7eff941c6b80 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ebbb677eb0 0x55ebbb6a9d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:44.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.322+0000 7eff941c6b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebbb690520 con 0x55ebbb6504b0 2023-05-20T01:25:44.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.322+0000 7eff941c6b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebbb690dd0 con 0x55ebbb64e8c0 2023-05-20T01:25:44.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.322+0000 7eff941c6b80 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ebbb64e550 con 0x55ebbb677eb0 2023-05-20T01:25:44.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.322+0000 7eff855d5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ebbb6504b0 0x55ebbb6a9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:44.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.322+0000 7eff855d5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ebbb6504b0 0x55ebbb6a9810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37356/0 (socket says 172.21.15.16:37356) 2023-05-20T01:25:44.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.322+0000 7eff855d5700 1 -- 172.21.15.16:0/483861347 learned_addr learned my addr 172.21.15.16:0/483861347 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:44.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.322+0000 7eff85dd6700 1 --2- 172.21.15.16:0/483861347 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ebbb64e8c0 0x55ebbb64ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:44.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.322+0000 7eff865d7700 1 --2- 172.21.15.16:0/483861347 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ebbb677eb0 0x55ebbb6a9d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:44.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.322+0000 7eff855d5700 1 -- 172.21.15.16:0/483861347 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ebbb677eb0 msgr2=0x55ebbb6a9d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:44.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.322+0000 7eff855d5700 1 --2- 172.21.15.16:0/483861347 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ebbb677eb0 0x55ebbb6a9d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:44.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.322+0000 7eff855d5700 1 -- 172.21.15.16:0/483861347 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ebbb64e8c0 msgr2=0x55ebbb64ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:44.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.322+0000 7eff855d5700 1 --2- 172.21.15.16:0/483861347 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ebbb64e8c0 0x55ebbb64ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:44.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.322+0000 7eff855d5700 1 -- 172.21.15.16:0/483861347 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff800c9120 con 0x55ebbb6504b0 2023-05-20T01:25:44.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.322+0000 7eff855d5700 1 --2- 172.21.15.16:0/483861347 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ebbb6504b0 0x55ebbb6a9810 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7eff7c028040 tx=0x7eff7c02b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba58a25838b5036b server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:44.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.322+0000 7eff63fff700 1 -- 172.21.15.16:0/483861347 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff7c088040 con 0x55ebbb6504b0 2023-05-20T01:25:44.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.322+0000 7eff63fff700 1 -- 172.21.15.16:0/483861347 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff7c059280 con 0x55ebbb6504b0 2023-05-20T01:25:44.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.322+0000 7eff63fff700 1 -- 172.21.15.16:0/483861347 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff7c057850 con 0x55ebbb6504b0 2023-05-20T01:25:44.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.323+0000 7eff941c6b80 1 -- 172.21.15.16:0/483861347 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ebbb6504b0 msgr2=0x55ebbb6a9810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:44.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.323+0000 7eff941c6b80 1 --2- 172.21.15.16:0/483861347 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ebbb6504b0 0x55ebbb6a9810 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7eff7c028040 tx=0x7eff7c02b000 comp rx=0 tx=0).stop 2023-05-20T01:25:44.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.323+0000 7eff941c6b80 1 -- 172.21.15.16:0/483861347 shutdown_connections 2023-05-20T01:25:44.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.323+0000 7eff941c6b80 1 --2- 172.21.15.16:0/483861347 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ebbb677eb0 0x55ebbb6a9d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:44.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.323+0000 7eff941c6b80 1 --2- 172.21.15.16:0/483861347 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ebbb64e8c0 0x55ebbb64ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:44.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.323+0000 7eff941c6b80 1 --2- 172.21.15.16:0/483861347 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ebbb6504b0 0x55ebbb6a9810 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:44.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.323+0000 7eff941c6b80 1 -- 172.21.15.16:0/483861347 >> 172.21.15.16:0/483861347 conn(0x55ebbb645f80 msgr2=0x55ebbb674230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:44.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.323+0000 7eff941c6b80 1 -- 172.21.15.16:0/483861347 shutdown_connections 2023-05-20T01:25:44.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.323+0000 7eff941c6b80 1 -- 172.21.15.16:0/483861347 wait complete. 2023-05-20T01:25:44.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.324+0000 7eff941c6b80 1 -- start start 2023-05-20T01:25:44.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.324+0000 7eff941c6b80 1 Processor -- start 2023-05-20T01:25:44.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.325+0000 7eff941c6b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ebbb64e8c0 0x55ebbb6e9350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:44.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.325+0000 7eff941c6b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ebbb6504b0 0x55ebbb698020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:44.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.325+0000 7eff941c6b80 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ebbb677eb0 0x55ebbb67dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:44.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.325+0000 7eff941c6b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebbb67ab70 con 0x55ebbb64e8c0 2023-05-20T01:25:44.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.325+0000 7eff941c6b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebbb698560 con 0x55ebbb6504b0 2023-05-20T01:25:44.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.325+0000 7eff941c6b80 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ebbb698990 con 0x55ebbb677eb0 2023-05-20T01:25:44.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.325+0000 7eff941c6b80 10 client.0 _ll_register_callbacks cb 0x7eff700139e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-05-20T01:25:44.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.325+0000 7eff85dd6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ebbb64e8c0 0x55ebbb6e9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:44.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.325+0000 7eff85dd6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ebbb64e8c0 0x55ebbb6e9350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37360/0 (socket says 172.21.15.16:37360) 2023-05-20T01:25:44.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.325+0000 7eff85dd6700 1 -- 172.21.15.16:0/582572341 learned_addr learned my addr 172.21.15.16:0/582572341 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:25:44.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.325+0000 7eff855d5700 1 --2- 172.21.15.16:0/582572341 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ebbb6504b0 0x55ebbb698020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:44.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.325+0000 7eff865d7700 1 --2- 172.21.15.16:0/582572341 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ebbb677eb0 0x55ebbb67dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:44.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.325+0000 7eff85dd6700 1 -- 172.21.15.16:0/582572341 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ebbb677eb0 msgr2=0x55ebbb67dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:44.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.325+0000 7eff85dd6700 1 --2- 172.21.15.16:0/582572341 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ebbb677eb0 0x55ebbb67dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:44.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.325+0000 7eff85dd6700 1 -- 172.21.15.16:0/582572341 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ebbb6504b0 msgr2=0x55ebbb698020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:25:44.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.325+0000 7eff85dd6700 1 --2- 172.21.15.16:0/582572341 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ebbb6504b0 0x55ebbb698020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:44.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.325+0000 7eff85dd6700 1 -- 172.21.15.16:0/582572341 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ebbb423630 con 0x55ebbb64e8c0 2023-05-20T01:25:44.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.325+0000 7eff865d7700 1 --2- 172.21.15.16:0/582572341 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ebbb677eb0 0x55ebbb67dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:25:44.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.325+0000 7eff85dd6700 1 --2- 172.21.15.16:0/582572341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ebbb64e8c0 0x55ebbb6e9350 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7eff70001d30 tx=0x7eff7000a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75c815326eb0b928 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:25:44.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.325+0000 7eff60ff9700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-05-20T01:25:44.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.325+0000 7eff60ff9700 1 -- 172.21.15.16:0/582572341 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff70004ab0 con 0x55ebbb64e8c0 2023-05-20T01:25:44.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.325+0000 7eff941c6b80 1 -- 172.21.15.16:0/582572341 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7eff7c061800 con 0x55ebbb64e8c0 2023-05-20T01:25:44.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.325+0000 7eff60ff9700 1 -- 172.21.15.16:0/582572341 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 7044+0+0 (secure 0 0 0) 0x7eff7002c4f0 con 0x55ebbb64e8c0 2023-05-20T01:25:44.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.325+0000 7eff60ff9700 1 -- 172.21.15.16:0/582572341 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7eff7002c940 con 0x55ebbb64e8c0 2023-05-20T01:25:44.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.325+0000 7eff941c6b80 10 client.7671 fetch_fsmap learned FSMap version 70 2023-05-20T01:25:44.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.325+0000 7eff941c6b80 1 -- 172.21.15.16:0/582572341 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=2+}) v3 -- 0x55ebbb3cfde0 con 0x55ebbb64e8c0 2023-05-20T01:25:44.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.326+0000 7eff60ff9700 1 -- 172.21.15.16:0/582572341 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff7c061800 con 0x55ebbb64e8c0 2023-05-20T01:25:44.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.326+0000 7eff60ff9700 1 -- 172.21.15.16:0/582572341 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 70) v1 ==== 3122+0+0 (secure 0 0 0) 0x7eff7002cae0 con 0x55ebbb64e8c0 2023-05-20T01:25:44.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.326+0000 7eff941c6b80 10 client.7671 fetch_fsmap finished waiting for FSMap version 70 2023-05-20T01:25:44.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.326+0000 7eff941c6b80 10 client.7671 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2023-05-20T01:25:44.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.326+0000 7eff941c6b80 20 client.7671 populate_metadata read hostname 'smithi016' 2023-05-20T01:25:44.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.326+0000 7eff941c6b80 1 --2- 172.21.15.16:0/582572341 >> [v2:172.21.15.16:6835/307899603,v1:172.21.15.16:6837/307899603] conn(0x55ebbb67af90 0x55ebbb67b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:25:44.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.326+0000 7eff941c6b80 4 client.7671 mds_command: new command op to 7575 tid=0[{"prefix": "scrub status"}] 2023-05-20T01:25:44.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.326+0000 7eff941c6b80 1 -- 172.21.15.16:0/582572341 --> [v2:172.21.15.16:6835/307899603,v1:172.21.15.16:6837/307899603] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ebbb570660 con 0x55ebbb67af90 2023-05-20T01:25:44.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.326+0000 7eff855d5700 1 --2- 172.21.15.16:0/582572341 >> [v2:172.21.15.16:6835/307899603,v1:172.21.15.16:6837/307899603] conn(0x55ebbb67af90 0x55ebbb67b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:25:44.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.326+0000 7eff60ff9700 10 client.7671 ms_handle_connect on v2:172.21.15.16:6835/307899603 2023-05-20T01:25:44.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.326+0000 7eff855d5700 1 --2- 172.21.15.16:0/582572341 >> [v2:172.21.15.16:6835/307899603,v1:172.21.15.16:6837/307899603] conn(0x55ebbb67af90 0x55ebbb67b360 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2479fb70f6ee2a61 server_cookie=331a454e370febe9 in_seq=0 out_seq=0 2023-05-20T01:25:44.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.326+0000 7eff60ff9700 1 -- 172.21.15.16:0/582572341 <== mds.0 v2:172.21.15.16:6835/307899603 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+260 (crc 0 0 0) 0x55ebbb570660 con 0x55ebbb67af90 2023-05-20T01:25:44.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.326+0000 7eff60ff9700 10 client.7671 handle_command_reply: tid=0 2023-05-20T01:25:44.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.326+0000 7eff941c6b80 1 client.7671 shutdown 2023-05-20T01:25:44.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.326+0000 7eff941c6b80 1 -- 172.21.15.16:0/582572341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ebbb64e8c0 msgr2=0x55ebbb6e9350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:44.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.326+0000 7eff941c6b80 1 --2- 172.21.15.16:0/582572341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ebbb64e8c0 0x55ebbb6e9350 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7eff70001d30 tx=0x7eff7000a000 comp rx=0 tx=0).stop 2023-05-20T01:25:44.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.326+0000 7eff941c6b80 1 -- 172.21.15.16:0/582572341 shutdown_connections 2023-05-20T01:25:44.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.326+0000 7eff941c6b80 1 --2- 172.21.15.16:0/582572341 >> [v2:172.21.15.16:6835/307899603,v1:172.21.15.16:6837/307899603] conn(0x55ebbb67af90 0x55ebbb67b360 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:44.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.326+0000 7eff941c6b80 1 --2- 172.21.15.16:0/582572341 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ebbb677eb0 0x55ebbb67dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:44.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.326+0000 7eff941c6b80 1 --2- 172.21.15.16:0/582572341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ebbb64e8c0 0x55ebbb6e9350 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:44.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.326+0000 7eff60ff9700 0 client.7671 ms_handle_reset on v2:172.21.15.16:6835/307899603 2023-05-20T01:25:44.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.326+0000 7eff941c6b80 1 --2- 172.21.15.16:0/582572341 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ebbb6504b0 0x55ebbb698020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:44.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.326+0000 7eff941c6b80 1 -- 172.21.15.16:0/582572341 >> 172.21.15.16:0/582572341 conn(0x55ebbb645f80 msgr2=0x55ebbb64a870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:44.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.327+0000 7eff941c6b80 1 -- 172.21.15.16:0/582572341 shutdown_connections 2023-05-20T01:25:44.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.327+0000 7eff941c6b80 1 -- 172.21.15.16:0/582572341 wait complete. 2023-05-20T01:25:44.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.327+0000 7eff941c6b80 20 client.7671 trim_cache size 0 max 16384 2023-05-20T01:25:44.345 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-05-20T01:25:44.345 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2023-05-20T01:25:44.346 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2023-05-20T01:25:44.346 INFO:teuthology.orchestra.run.smithi016.stdout: "633f2d59-6072-47a0-8866-7a1dffeed959": { 2023-05-20T01:25:44.346 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2023-05-20T01:25:44.347 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "633f2d59-6072-47a0-8866-7a1dffeed959", 2023-05-20T01:25:44.347 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,repair" 2023-05-20T01:25:44.347 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:25:44.348 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:25:44.348 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-05-20T01:25:44.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.327+0000 7eff8803a700 1 -- 172.21.15.16:0/1452700985 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7eff6406ad10 msgr2=0x7eff6406d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:44.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.327+0000 7eff8803a700 1 --2- 172.21.15.16:0/1452700985 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7eff6406ad10 0x7eff6406d1c0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7eff70000f60 tx=0x7eff70007e70 comp rx=0 tx=0).stop 2023-05-20T01:25:44.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.327+0000 7eff8803a700 1 -- 172.21.15.16:0/1452700985 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff800cd070 msgr2=0x7eff801e3470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:25:44.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.327+0000 7eff8803a700 1 --2- 172.21.15.16:0/1452700985 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff800cd070 0x7eff801e3470 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7eff7c000c00 tx=0x7eff7c004750 comp rx=0 tx=0).stop 2023-05-20T01:25:44.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.327+0000 7eff8803a700 1 -- 172.21.15.16:0/1452700985 shutdown_connections 2023-05-20T01:25:44.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.327+0000 7eff8803a700 1 --2- 172.21.15.16:0/1452700985 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7eff6406ad10 0x7eff6406d1c0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:44.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.327+0000 7eff8803a700 1 --2- 172.21.15.16:0/1452700985 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7eff8015cf40 0x7eff800c41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:44.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.327+0000 7eff8803a700 1 --2- 172.21.15.16:0/1452700985 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7eff800710d0 0x7eff80150160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:44.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.327+0000 7eff8803a700 1 --2- 172.21.15.16:0/1452700985 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff800cd070 0x7eff801e3470 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:25:44.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.327+0000 7eff8803a700 1 -- 172.21.15.16:0/1452700985 >> 172.21.15.16:0/1452700985 conn(0x7eff8006c260 msgr2=0x7eff8006dde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:25:44.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.327+0000 7eff8803a700 1 -- 172.21.15.16:0/1452700985 shutdown_connections 2023-05-20T01:25:44.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:25:44.327+0000 7eff8803a700 1 -- 172.21.15.16:0/1452700985 wait complete. 2023-05-20T01:25:44.354 INFO:tasks.cephfs.filesystem:scrub status for tag:633f2d59-6072-47a0-8866-7a1dffeed959 - {'path': '/', 'tag': '633f2d59-6072-47a0-8866-7a1dffeed959', 'options': 'recursive,repair'} 2023-05-20T01:26:06.874 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:26:06.877 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:26:14.355 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:0 scrub status 2023-05-20T01:26:14.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.492+0000 7f7f6e30a700 1 Processor -- start 2023-05-20T01:26:14.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.492+0000 7f7f6e30a700 1 -- start start 2023-05-20T01:26:14.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.493+0000 7f7f6e30a700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7f68158f90 0x7f7f680cb7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:14.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.493+0000 7f7f6e30a700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7f680ce0a0 0x7f7f680ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:14.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.493+0000 7f7f6e30a700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f68159360 con 0x7f7f680cc050 2023-05-20T01:26:14.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.493+0000 7f7f6e30a700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f680cce10 con 0x7f7f680ce0a0 2023-05-20T01:26:14.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.493+0000 7f7f6e30a700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f680cea50 con 0x7f7f68158f90 2023-05-20T01:26:14.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.493+0000 7f7f677fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7f680ce0a0 0x7f7f680ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:14.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.493+0000 7f7f6c8a7700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f7f680cc050 0x7f7f680cc490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:48898/0 (socket says 172.21.15.16:48898) 2023-05-20T01:26:14.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.493+0000 7f7f6c8a7700 1 -- 172.21.15.16:0/2037090948 learned_addr learned my addr 172.21.15.16:0/2037090948 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:26:14.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.494+0000 7f7f67fff700 1 --2- 172.21.15.16:0/2037090948 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7f68158f90 0x7f7f680cb7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:14.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.494+0000 7f7f677fe700 1 -- 172.21.15.16:0/2037090948 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7f68158f90 msgr2=0x7f7f680cb7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:14.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.494+0000 7f7f677fe700 1 --2- 172.21.15.16:0/2037090948 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7f68158f90 0x7f7f680cb7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:14.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.494+0000 7f7f677fe700 1 -- 172.21.15.16:0/2037090948 >> v1:172.21.15.16:6789/0 conn(0x7f7f680cc050 legacy=0x7f7f680cc490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:14.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.494+0000 7f7f66ffd700 1 -- 172.21.15.16:0/2037090948 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4176626085 0 0) 0x7f7f60001490 con 0x7f7f680cc050 2023-05-20T01:26:14.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.494+0000 7f7f677fe700 1 -- 172.21.15.16:0/2037090948 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f680ccba0 con 0x7f7f680ce0a0 2023-05-20T01:26:14.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.494+0000 7f7f67fff700 1 --2- 172.21.15.16:0/2037090948 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7f68158f90 0x7f7f680cb7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:26:14.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.494+0000 7f7f677fe700 1 --2- 172.21.15.16:0/2037090948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f680ce0a0 0x7f7f680ce510 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f7f58009870 tx=0x7f7f58009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b886b62e462d3f69 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:14.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.494+0000 7f7f66ffd700 1 -- 172.21.15.16:0/2037090948 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f58011830 con 0x7f7f680ce0a0 2023-05-20T01:26:14.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.494+0000 7f7f66ffd700 1 -- 172.21.15.16:0/2037090948 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f58011990 con 0x7f7f680ce0a0 2023-05-20T01:26:14.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.494+0000 7f7f66ffd700 1 -- 172.21.15.16:0/2037090948 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f58011d20 con 0x7f7f680ce0a0 2023-05-20T01:26:14.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.495+0000 7f7f6e30a700 1 -- 172.21.15.16:0/2037090948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f680ce0a0 msgr2=0x7f7f680ce510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:14.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.495+0000 7f7f6e30a700 1 --2- 172.21.15.16:0/2037090948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f680ce0a0 0x7f7f680ce510 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f7f58009870 tx=0x7f7f58009b80 comp rx=0 tx=0).stop 2023-05-20T01:26:14.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.495+0000 7f7f6e30a700 1 -- 172.21.15.16:0/2037090948 shutdown_connections 2023-05-20T01:26:14.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.495+0000 7f7f6e30a700 1 --2- 172.21.15.16:0/2037090948 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7f68158f90 0x7f7f680cb7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:14.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.495+0000 7f7f6e30a700 1 --2- 172.21.15.16:0/2037090948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f680ce0a0 0x7f7f680ce510 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:14.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.495+0000 7f7f6e30a700 1 -- 172.21.15.16:0/2037090948 >> 172.21.15.16:0/2037090948 conn(0x7f7f6806cbb0 msgr2=0x7f7f6806cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:14.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.495+0000 7f7f6e30a700 1 -- 172.21.15.16:0/2037090948 shutdown_connections 2023-05-20T01:26:14.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.495+0000 7f7f6e30a700 1 -- 172.21.15.16:0/2037090948 wait complete. 2023-05-20T01:26:14.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.496+0000 7f7f6e30a700 1 Processor -- start 2023-05-20T01:26:14.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.496+0000 7f7f6e30a700 1 -- start start 2023-05-20T01:26:14.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.497+0000 7f7f6e30a700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7f680cc050 0x7f7f680b0d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:14.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.497+0000 7f7f6e30a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f680ce0a0 0x7f7f680c4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:14.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.497+0000 7f7f6e30a700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f68158f90 0x7f7f680c4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:14.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.497+0000 7f7f6e30a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f68152b10 con 0x7f7f680ce0a0 2023-05-20T01:26:14.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.497+0000 7f7f6e30a700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f68152df0 con 0x7f7f68158f90 2023-05-20T01:26:14.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.497+0000 7f7f6e30a700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f681530d0 con 0x7f7f680cc050 2023-05-20T01:26:14.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.497+0000 7f7f677fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f680ce0a0 0x7f7f680c4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:14.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.497+0000 7f7f677fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f680ce0a0 0x7f7f680c4770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37374/0 (socket says 172.21.15.16:37374) 2023-05-20T01:26:14.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.497+0000 7f7f677fe700 1 -- 172.21.15.16:0/3849293873 learned_addr learned my addr 172.21.15.16:0/3849293873 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:26:14.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.497+0000 7f7f677fe700 1 -- 172.21.15.16:0/3849293873 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7f680cc050 msgr2=0x7f7f680b0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:14.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.497+0000 7f7f6c8a7700 1 --2- 172.21.15.16:0/3849293873 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f68158f90 0x7f7f680c4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:14.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.497+0000 7f7f67fff700 1 --2- 172.21.15.16:0/3849293873 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7f680cc050 0x7f7f680b0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:14.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.497+0000 7f7f677fe700 1 --2- 172.21.15.16:0/3849293873 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7f680cc050 0x7f7f680b0d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:14.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.497+0000 7f7f677fe700 1 -- 172.21.15.16:0/3849293873 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f68158f90 msgr2=0x7f7f680c4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:14.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.497+0000 7f7f677fe700 1 --2- 172.21.15.16:0/3849293873 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f68158f90 0x7f7f680c4cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:14.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.497+0000 7f7f677fe700 1 -- 172.21.15.16:0/3849293873 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f580094a0 con 0x7f7f680ce0a0 2023-05-20T01:26:14.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.497+0000 7f7f67fff700 1 --2- 172.21.15.16:0/3849293873 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7f680cc050 0x7f7f680b0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:26:14.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.497+0000 7f7f6c8a7700 1 --2- 172.21.15.16:0/3849293873 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f68158f90 0x7f7f680c4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:26:14.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.498+0000 7f7f677fe700 1 --2- 172.21.15.16:0/3849293873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f680ce0a0 0x7f7f680c4770 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f7f580090b0 tx=0x7f7f58004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:14.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.498+0000 7f7f64ff9700 1 -- 172.21.15.16:0/3849293873 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f580036a0 con 0x7f7f680ce0a0 2023-05-20T01:26:14.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.498+0000 7f7f64ff9700 1 -- 172.21.15.16:0/3849293873 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f5801f070 con 0x7f7f680ce0a0 2023-05-20T01:26:14.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.498+0000 7f7f6e30a700 1 -- 172.21.15.16:0/3849293873 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f680ca9f0 con 0x7f7f680ce0a0 2023-05-20T01:26:14.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.498+0000 7f7f64ff9700 1 -- 172.21.15.16:0/3849293873 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f58003a40 con 0x7f7f680ce0a0 2023-05-20T01:26:14.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.498+0000 7f7f6e30a700 1 -- 172.21.15.16:0/3849293873 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f680ca750 con 0x7f7f680ce0a0 2023-05-20T01:26:14.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.499+0000 7f7f64ff9700 1 -- 172.21.15.16:0/3849293873 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7f58003ba0 con 0x7f7f680ce0a0 2023-05-20T01:26:14.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.500+0000 7f7f64ff9700 1 --2- 172.21.15.16:0/3849293873 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f7f4806ace0 0x7f7f4806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:14.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.500+0000 7f7f64ff9700 1 -- 172.21.15.16:0/3849293873 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 7044+0+0 (secure 0 0 0) 0x7f7f58021070 con 0x7f7f680ce0a0 2023-05-20T01:26:14.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.500+0000 7f7f67fff700 1 --2- 172.21.15.16:0/3849293873 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f7f4806ace0 0x7f7f4806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:14.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.500+0000 7f7f67fff700 1 --2- 172.21.15.16:0/3849293873 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f7f4806ace0 0x7f7f4806d190 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f7f500089b0 tx=0x7f7f50008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:14.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.502+0000 7f7f7a496b80 1 Processor -- start 2023-05-20T01:26:14.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.502+0000 7f7f7a496b80 1 -- start start 2023-05-20T01:26:14.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.503+0000 7f7f7a496b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cebb28f450 0x55cebb28f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:14.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.503+0000 7f7f7a496b80 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55cebb28b750 0x55cebb2644b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:14.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.503+0000 7f7f7a496b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cebb2628c0 0x55cebb2649f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:14.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.503+0000 7f7f7a496b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cebb16a060 con 0x55cebb2628c0 2023-05-20T01:26:14.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.503+0000 7f7f7a496b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cebb1bc8a0 con 0x55cebb28f450 2023-05-20T01:26:14.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.503+0000 7f7f7a496b80 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cebb1b5d30 con 0x55cebb28b750 2023-05-20T01:26:14.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.503+0000 7f7f6c8a7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cebb2628c0 0x55cebb2649f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:14.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.503+0000 7f7f6c8a7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cebb2628c0 0x55cebb2649f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37384/0 (socket says 172.21.15.16:37384) 2023-05-20T01:26:14.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.503+0000 7f7f6c8a7700 1 -- 172.21.15.16:0/2687747650 learned_addr learned my addr 172.21.15.16:0/2687747650 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:26:14.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.503+0000 7f7f67fff700 1 --2- 172.21.15.16:0/2687747650 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cebb28f450 0x55cebb28f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:14.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.503+0000 7f7f677fe700 1 --2- 172.21.15.16:0/2687747650 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55cebb28b750 0x55cebb2644b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:14.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.503+0000 7f7f6c8a7700 1 -- 172.21.15.16:0/2687747650 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55cebb28b750 msgr2=0x55cebb2644b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:14.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.503+0000 7f7f6c8a7700 1 --2- 172.21.15.16:0/2687747650 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55cebb28b750 0x55cebb2644b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:14.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.503+0000 7f7f6c8a7700 1 -- 172.21.15.16:0/2687747650 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cebb28f450 msgr2=0x55cebb28f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:14.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.503+0000 7f7f6c8a7700 1 --2- 172.21.15.16:0/2687747650 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cebb28f450 0x55cebb28f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:14.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.503+0000 7f7f6c8a7700 1 -- 172.21.15.16:0/2687747650 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cebb037630 con 0x55cebb2628c0 2023-05-20T01:26:14.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.504+0000 7f7f6c8a7700 1 --2- 172.21.15.16:0/2687747650 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cebb2628c0 0x55cebb2649f0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f7f6000d180 tx=0x7f7f6000d490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2f057a356551690 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:14.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.504+0000 7f7f5dffb700 1 -- 172.21.15.16:0/2687747650 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f600070d0 con 0x55cebb2628c0 2023-05-20T01:26:14.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.504+0000 7f7f5dffb700 1 -- 172.21.15.16:0/2687747650 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f60007230 con 0x55cebb2628c0 2023-05-20T01:26:14.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.504+0000 7f7f5dffb700 1 -- 172.21.15.16:0/2687747650 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f600075a0 con 0x55cebb2628c0 2023-05-20T01:26:14.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.504+0000 7f7f7a496b80 1 -- 172.21.15.16:0/2687747650 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cebb2628c0 msgr2=0x55cebb2649f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:14.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.504+0000 7f7f7a496b80 1 --2- 172.21.15.16:0/2687747650 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cebb2628c0 0x55cebb2649f0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f7f6000d180 tx=0x7f7f6000d490 comp rx=0 tx=0).stop 2023-05-20T01:26:14.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.504+0000 7f7f7a496b80 1 -- 172.21.15.16:0/2687747650 shutdown_connections 2023-05-20T01:26:14.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.504+0000 7f7f7a496b80 1 --2- 172.21.15.16:0/2687747650 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cebb2628c0 0x55cebb2649f0 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:14.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.504+0000 7f7f7a496b80 1 --2- 172.21.15.16:0/2687747650 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cebb28f450 0x55cebb28f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:14.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.504+0000 7f7f7a496b80 1 --2- 172.21.15.16:0/2687747650 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55cebb28b750 0x55cebb2644b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:14.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.504+0000 7f7f7a496b80 1 -- 172.21.15.16:0/2687747650 >> 172.21.15.16:0/2687747650 conn(0x55cebb259f80 msgr2=0x55cebb2aa260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:14.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.504+0000 7f7f7a496b80 1 -- 172.21.15.16:0/2687747650 shutdown_connections 2023-05-20T01:26:14.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.504+0000 7f7f7a496b80 1 -- 172.21.15.16:0/2687747650 wait complete. 2023-05-20T01:26:14.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.505+0000 7f7f7a496b80 1 -- start start 2023-05-20T01:26:14.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.505+0000 7f7f7a496b80 1 Processor -- start 2023-05-20T01:26:14.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.507+0000 7f7f7a496b80 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55cebb2628c0 0x55cebb2ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:14.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.507+0000 7f7f7a496b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cebb28b750 0x55cebb290400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:14.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.507+0000 7f7f7a496b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cebb28f450 0x55cebb2917c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:14.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.507+0000 7f7f7a496b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cebb1578e0 con 0x55cebb28b750 2023-05-20T01:26:14.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.507+0000 7f7f7a496b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cebb157580 con 0x55cebb28f450 2023-05-20T01:26:14.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.507+0000 7f7f7a496b80 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cebb039920 con 0x55cebb2628c0 2023-05-20T01:26:14.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.507+0000 7f7f7a496b80 10 client.0 _ll_register_callbacks cb 0x55cebb175780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-05-20T01:26:14.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.507+0000 7f7f677fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cebb28b750 0x55cebb290400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:14.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.507+0000 7f7f677fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cebb28b750 0x55cebb290400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37388/0 (socket says 172.21.15.16:37388) 2023-05-20T01:26:14.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.507+0000 7f7f677fe700 1 -- 172.21.15.16:0/3844806262 learned_addr learned my addr 172.21.15.16:0/3844806262 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:26:14.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.507+0000 7f7f67fff700 1 --2- 172.21.15.16:0/3844806262 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55cebb2628c0 0x55cebb2ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:14.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.507+0000 7f7f6c8a7700 1 --2- 172.21.15.16:0/3844806262 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cebb28f450 0x55cebb2917c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:14.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.507+0000 7f7f677fe700 1 -- 172.21.15.16:0/3844806262 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55cebb2628c0 msgr2=0x55cebb2ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:14.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.507+0000 7f7f677fe700 1 --2- 172.21.15.16:0/3844806262 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55cebb2628c0 0x55cebb2ad720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:14.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.507+0000 7f7f677fe700 1 -- 172.21.15.16:0/3844806262 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cebb28f450 msgr2=0x55cebb2917c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:14.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.507+0000 7f7f677fe700 1 --2- 172.21.15.16:0/3844806262 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cebb28f450 0x55cebb2917c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:14.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.507+0000 7f7f677fe700 1 -- 172.21.15.16:0/3844806262 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f60001e50 con 0x55cebb28b750 2023-05-20T01:26:14.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.507+0000 7f7f677fe700 1 --2- 172.21.15.16:0/3844806262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cebb28b750 0x55cebb290400 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f7f58009bc0 tx=0x7f7f5805dc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cfd8b013f6c62bc server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:14.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.507+0000 7f7f46ffd700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-05-20T01:26:14.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.508+0000 7f7f46ffd700 1 -- 172.21.15.16:0/3844806262 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f580619e0 con 0x55cebb28b750 2023-05-20T01:26:14.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.508+0000 7f7f46ffd700 1 -- 172.21.15.16:0/3844806262 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 7044+0+0 (secure 0 0 0) 0x7f7f58061b40 con 0x55cebb28b750 2023-05-20T01:26:14.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.508+0000 7f7f7a496b80 1 -- 172.21.15.16:0/3844806262 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cebb257740 con 0x55cebb28b750 2023-05-20T01:26:14.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.508+0000 7f7f46ffd700 1 -- 172.21.15.16:0/3844806262 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7f58075030 con 0x55cebb28b750 2023-05-20T01:26:14.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.508+0000 7f7f7a496b80 10 client.7686 fetch_fsmap learned FSMap version 70 2023-05-20T01:26:14.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.508+0000 7f7f7a496b80 1 -- 172.21.15.16:0/3844806262 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=2+}) v3 -- 0x55cebb17c190 con 0x55cebb28b750 2023-05-20T01:26:14.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.508+0000 7f7f46ffd700 1 -- 172.21.15.16:0/3844806262 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55cebb257740 con 0x55cebb28b750 2023-05-20T01:26:14.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.508+0000 7f7f46ffd700 1 -- 172.21.15.16:0/3844806262 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 70) v1 ==== 3122+0+0 (secure 0 0 0) 0x7f7f58078300 con 0x55cebb28b750 2023-05-20T01:26:14.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.508+0000 7f7f7a496b80 10 client.7686 fetch_fsmap finished waiting for FSMap version 70 2023-05-20T01:26:14.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.508+0000 7f7f7a496b80 10 client.7686 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2023-05-20T01:26:14.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.508+0000 7f7f7a496b80 20 client.7686 populate_metadata read hostname 'smithi016' 2023-05-20T01:26:14.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.508+0000 7f7f7a496b80 1 --2- 172.21.15.16:0/3844806262 >> [v2:172.21.15.16:6835/307899603,v1:172.21.15.16:6837/307899603] conn(0x55cebb290940 0x55cebb2a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:14.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.509+0000 7f7f7a496b80 4 client.7686 mds_command: new command op to 7575 tid=0[{"prefix": "get_command_descriptions"}] 2023-05-20T01:26:14.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.509+0000 7f7f7a496b80 1 -- 172.21.15.16:0/3844806262 --> [v2:172.21.15.16:6835/307899603,v1:172.21.15.16:6837/307899603] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cebb1780a0 con 0x55cebb290940 2023-05-20T01:26:14.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.509+0000 7f7f67fff700 1 --2- 172.21.15.16:0/3844806262 >> [v2:172.21.15.16:6835/307899603,v1:172.21.15.16:6837/307899603] conn(0x55cebb290940 0x55cebb2a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:14.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.509+0000 7f7f46ffd700 10 client.7686 ms_handle_connect on v2:172.21.15.16:6835/307899603 2023-05-20T01:26:14.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.509+0000 7f7f67fff700 1 --2- 172.21.15.16:0/3844806262 >> [v2:172.21.15.16:6835/307899603,v1:172.21.15.16:6837/307899603] conn(0x55cebb290940 0x55cebb2a37a0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c51604953ccd70a server_cookie=ea91875ac6583267 in_seq=0 out_seq=0 2023-05-20T01:26:14.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.510+0000 7f7f46ffd700 1 -- 172.21.15.16:0/3844806262 <== mds.0 v2:172.21.15.16:6835/307899603 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55cebb1780a0 con 0x55cebb290940 2023-05-20T01:26:14.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.510+0000 7f7f46ffd700 10 client.7686 handle_command_reply: tid=0 2023-05-20T01:26:14.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.510+0000 7f7f7a496b80 1 client.7686 shutdown 2023-05-20T01:26:14.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.510+0000 7f7f7a496b80 1 -- 172.21.15.16:0/3844806262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cebb28b750 msgr2=0x55cebb290400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:14.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.510+0000 7f7f7a496b80 1 --2- 172.21.15.16:0/3844806262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cebb28b750 0x55cebb290400 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f7f58009bc0 tx=0x7f7f5805dc20 comp rx=0 tx=0).stop 2023-05-20T01:26:14.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.510+0000 7f7f7a496b80 1 -- 172.21.15.16:0/3844806262 shutdown_connections 2023-05-20T01:26:14.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.510+0000 7f7f7a496b80 1 --2- 172.21.15.16:0/3844806262 >> [v2:172.21.15.16:6835/307899603,v1:172.21.15.16:6837/307899603] conn(0x55cebb290940 0x55cebb2a37a0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:14.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.510+0000 7f7f7a496b80 1 --2- 172.21.15.16:0/3844806262 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cebb28f450 0x55cebb2917c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:14.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.510+0000 7f7f7a496b80 1 --2- 172.21.15.16:0/3844806262 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55cebb2628c0 0x55cebb2ad720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:14.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.510+0000 7f7f7a496b80 1 --2- 172.21.15.16:0/3844806262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cebb28b750 0x55cebb290400 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:14.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.510+0000 7f7f46ffd700 0 client.7686 ms_handle_reset on v2:172.21.15.16:6835/307899603 2023-05-20T01:26:14.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.510+0000 7f7f7a496b80 1 -- 172.21.15.16:0/3844806262 >> 172.21.15.16:0/3844806262 conn(0x55cebb259f80 msgr2=0x55cebb25f430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:14.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.510+0000 7f7f7a496b80 1 -- 172.21.15.16:0/3844806262 shutdown_connections 2023-05-20T01:26:14.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.510+0000 7f7f7a496b80 1 -- 172.21.15.16:0/3844806262 wait complete. 2023-05-20T01:26:14.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.510+0000 7f7f7a496b80 20 client.7686 trim_cache size 0 max 16384 2023-05-20T01:26:14.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.527+0000 7f7f7a496b80 1 Processor -- start 2023-05-20T01:26:14.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.527+0000 7f7f7a496b80 1 -- start start 2023-05-20T01:26:14.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.528+0000 7f7f7a496b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cebb2628c0 0x55cebb2aa450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:14.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.528+0000 7f7f7a496b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cebb28b750 0x55cebb2a1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:14.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.528+0000 7f7f7a496b80 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55cebb28f450 0x55cebb2a2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:14.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.528+0000 7f7f7a496b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cebb2660c0 con 0x55cebb2628c0 2023-05-20T01:26:14.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.528+0000 7f7f7a496b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cebb28be20 con 0x55cebb28b750 2023-05-20T01:26:14.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.528+0000 7f7f7a496b80 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cebb292a10 con 0x55cebb28f450 2023-05-20T01:26:14.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.528+0000 7f7f67fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cebb2628c0 0x55cebb2aa450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:14.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.528+0000 7f7f67fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cebb2628c0 0x55cebb2aa450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37394/0 (socket says 172.21.15.16:37394) 2023-05-20T01:26:14.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.528+0000 7f7f67fff700 1 -- 172.21.15.16:0/1808953056 learned_addr learned my addr 172.21.15.16:0/1808953056 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:26:14.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.528+0000 7f7f677fe700 1 --2- 172.21.15.16:0/1808953056 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cebb28b750 0x55cebb2a1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:14.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.528+0000 7f7f6c8a7700 1 --2- 172.21.15.16:0/1808953056 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55cebb28f450 0x55cebb2a2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:14.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.528+0000 7f7f67fff700 1 -- 172.21.15.16:0/1808953056 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55cebb28f450 msgr2=0x55cebb2a2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:14.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.528+0000 7f7f67fff700 1 --2- 172.21.15.16:0/1808953056 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55cebb28f450 0x55cebb2a2140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:14.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.528+0000 7f7f67fff700 1 -- 172.21.15.16:0/1808953056 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cebb28b750 msgr2=0x55cebb2a1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:14.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.528+0000 7f7f67fff700 1 --2- 172.21.15.16:0/1808953056 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cebb28b750 0x55cebb2a1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:14.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.528+0000 7f7f67fff700 1 -- 172.21.15.16:0/1808953056 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cebb17bee0 con 0x55cebb2628c0 2023-05-20T01:26:14.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.528+0000 7f7f677fe700 1 --2- 172.21.15.16:0/1808953056 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cebb28b750 0x55cebb2a1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:26:14.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.529+0000 7f7f67fff700 1 --2- 172.21.15.16:0/1808953056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cebb2628c0 0x55cebb2aa450 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f7f5000d4a0 tx=0x7f7f50016040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4558fdfe95f0383 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:14.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.529+0000 7f7f5e7fc700 1 -- 172.21.15.16:0/1808953056 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f50026040 con 0x55cebb2628c0 2023-05-20T01:26:14.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.529+0000 7f7f5e7fc700 1 -- 172.21.15.16:0/1808953056 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f5000c040 con 0x55cebb2628c0 2023-05-20T01:26:14.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.529+0000 7f7f5e7fc700 1 -- 172.21.15.16:0/1808953056 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f5000b050 con 0x55cebb2628c0 2023-05-20T01:26:14.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.529+0000 7f7f7a496b80 1 -- 172.21.15.16:0/1808953056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cebb2628c0 msgr2=0x55cebb2aa450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:14.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.529+0000 7f7f7a496b80 1 --2- 172.21.15.16:0/1808953056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cebb2628c0 0x55cebb2aa450 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f7f5000d4a0 tx=0x7f7f50016040 comp rx=0 tx=0).stop 2023-05-20T01:26:14.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.529+0000 7f7f7a496b80 1 -- 172.21.15.16:0/1808953056 shutdown_connections 2023-05-20T01:26:14.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.529+0000 7f7f7a496b80 1 --2- 172.21.15.16:0/1808953056 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55cebb28f450 0x55cebb2a2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:14.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.529+0000 7f7f7a496b80 1 --2- 172.21.15.16:0/1808953056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cebb2628c0 0x55cebb2aa450 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:14.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.529+0000 7f7f7a496b80 1 --2- 172.21.15.16:0/1808953056 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cebb28b750 0x55cebb2a1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:14.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.529+0000 7f7f7a496b80 1 -- 172.21.15.16:0/1808953056 >> 172.21.15.16:0/1808953056 conn(0x55cebb259f80 msgr2=0x55cebb291570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:14.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.529+0000 7f7f7a496b80 1 -- 172.21.15.16:0/1808953056 shutdown_connections 2023-05-20T01:26:14.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.529+0000 7f7f7a496b80 1 -- 172.21.15.16:0/1808953056 wait complete. 2023-05-20T01:26:14.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.530+0000 7f7f7a496b80 1 -- start start 2023-05-20T01:26:14.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.530+0000 7f7f7a496b80 1 Processor -- start 2023-05-20T01:26:14.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.531+0000 7f7f7a496b80 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55cebb2628c0 0x55cebb310580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:14.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.531+0000 7f7f7a496b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cebb28b750 0x55cebb2a1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:14.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.531+0000 7f7f7a496b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cebb28f450 0x55cebb2a2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:14.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.531+0000 7f7f7a496b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cebb290780 con 0x55cebb28f450 2023-05-20T01:26:14.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.531+0000 7f7f7a496b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cebb310e80 con 0x55cebb28b750 2023-05-20T01:26:14.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.531+0000 7f7f7a496b80 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cebb28c110 con 0x55cebb2628c0 2023-05-20T01:26:14.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.531+0000 7f7f7a496b80 10 client.0 _ll_register_callbacks cb 0x7f7f58059620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-05-20T01:26:14.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.531+0000 7f7f6c8a7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cebb28f450 0x55cebb2a2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:14.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.531+0000 7f7f6c8a7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cebb28f450 0x55cebb2a2140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37404/0 (socket says 172.21.15.16:37404) 2023-05-20T01:26:14.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.531+0000 7f7f6c8a7700 1 -- 172.21.15.16:0/3012729382 learned_addr learned my addr 172.21.15.16:0/3012729382 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:26:14.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.531+0000 7f7f67fff700 1 --2- 172.21.15.16:0/3012729382 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55cebb2628c0 0x55cebb310580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:14.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.531+0000 7f7f677fe700 1 --2- 172.21.15.16:0/3012729382 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cebb28b750 0x55cebb2a1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:14.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.531+0000 7f7f6c8a7700 1 -- 172.21.15.16:0/3012729382 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55cebb2628c0 msgr2=0x55cebb310580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:14.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.531+0000 7f7f6c8a7700 1 --2- 172.21.15.16:0/3012729382 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55cebb2628c0 0x55cebb310580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:14.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.531+0000 7f7f6c8a7700 1 -- 172.21.15.16:0/3012729382 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cebb28b750 msgr2=0x55cebb2a1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:14.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.531+0000 7f7f6c8a7700 1 --2- 172.21.15.16:0/3012729382 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cebb28b750 0x55cebb2a1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:14.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.531+0000 7f7f6c8a7700 1 -- 172.21.15.16:0/3012729382 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f5000d7f0 con 0x55cebb28f450 2023-05-20T01:26:14.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.532+0000 7f7f6c8a7700 1 --2- 172.21.15.16:0/3012729382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cebb28f450 0x55cebb2a2140 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f7f6000d150 tx=0x7f7f60001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6df236ffba2a5d60 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:14.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.532+0000 7f7f46ffd700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-05-20T01:26:14.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.532+0000 7f7f46ffd700 1 -- 172.21.15.16:0/3012729382 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f600018e0 con 0x55cebb28f450 2023-05-20T01:26:14.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.532+0000 7f7f46ffd700 1 -- 172.21.15.16:0/3012729382 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 7044+0+0 (secure 0 0 0) 0x7f7f60001b20 con 0x55cebb28f450 2023-05-20T01:26:14.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.532+0000 7f7f7a496b80 1 -- 172.21.15.16:0/3012729382 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cebb265860 con 0x55cebb28f450 2023-05-20T01:26:14.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.532+0000 7f7f46ffd700 1 -- 172.21.15.16:0/3012729382 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7f6002a470 con 0x55cebb28f450 2023-05-20T01:26:14.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.532+0000 7f7f7a496b80 10 client.7692 fetch_fsmap learned FSMap version 70 2023-05-20T01:26:14.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.532+0000 7f7f7a496b80 1 -- 172.21.15.16:0/3012729382 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=2+}) v3 -- 0x55cebafccc60 con 0x55cebb28f450 2023-05-20T01:26:14.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.532+0000 7f7f46ffd700 1 -- 172.21.15.16:0/3012729382 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55cebb265860 con 0x55cebb28f450 2023-05-20T01:26:14.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.532+0000 7f7f46ffd700 1 -- 172.21.15.16:0/3012729382 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 70) v1 ==== 3122+0+0 (secure 0 0 0) 0x7f7f6002a610 con 0x55cebb28f450 2023-05-20T01:26:14.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.532+0000 7f7f7a496b80 10 client.7692 fetch_fsmap finished waiting for FSMap version 70 2023-05-20T01:26:14.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.532+0000 7f7f7a496b80 10 client.7692 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2023-05-20T01:26:14.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.532+0000 7f7f7a496b80 20 client.7692 populate_metadata read hostname 'smithi016' 2023-05-20T01:26:14.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.532+0000 7f7f7a496b80 1 --2- 172.21.15.16:0/3012729382 >> [v2:172.21.15.16:6835/307899603,v1:172.21.15.16:6837/307899603] conn(0x55cebb293520 0x55cebb283b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:14.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.532+0000 7f7f7a496b80 4 client.7692 mds_command: new command op to 7575 tid=0[{"prefix": "scrub status"}] 2023-05-20T01:26:14.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.532+0000 7f7f7a496b80 1 -- 172.21.15.16:0/3012729382 --> [v2:172.21.15.16:6835/307899603,v1:172.21.15.16:6837/307899603] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x7f7f60007230 con 0x55cebb293520 2023-05-20T01:26:14.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.532+0000 7f7f67fff700 1 --2- 172.21.15.16:0/3012729382 >> [v2:172.21.15.16:6835/307899603,v1:172.21.15.16:6837/307899603] conn(0x55cebb293520 0x55cebb283b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:14.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.533+0000 7f7f46ffd700 10 client.7692 ms_handle_connect on v2:172.21.15.16:6835/307899603 2023-05-20T01:26:14.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.533+0000 7f7f67fff700 1 --2- 172.21.15.16:0/3012729382 >> [v2:172.21.15.16:6835/307899603,v1:172.21.15.16:6837/307899603] conn(0x55cebb293520 0x55cebb283b70 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e42a7677aef0d2b server_cookie=532dfeb794a4e376 in_seq=0 out_seq=0 2023-05-20T01:26:14.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.533+0000 7f7f46ffd700 1 -- 172.21.15.16:0/3012729382 <== mds.0 v2:172.21.15.16:6835/307899603 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x7f7f60007230 con 0x55cebb293520 2023-05-20T01:26:14.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.533+0000 7f7f46ffd700 10 client.7692 handle_command_reply: tid=0 2023-05-20T01:26:14.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.533+0000 7f7f7a496b80 1 client.7692 shutdown 2023-05-20T01:26:14.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.533+0000 7f7f7a496b80 1 -- 172.21.15.16:0/3012729382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cebb28f450 msgr2=0x55cebb2a2140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:14.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.533+0000 7f7f7a496b80 1 --2- 172.21.15.16:0/3012729382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cebb28f450 0x55cebb2a2140 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f7f6000d150 tx=0x7f7f60001250 comp rx=0 tx=0).stop 2023-05-20T01:26:14.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.533+0000 7f7f7a496b80 1 -- 172.21.15.16:0/3012729382 shutdown_connections 2023-05-20T01:26:14.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.533+0000 7f7f7a496b80 1 --2- 172.21.15.16:0/3012729382 >> [v2:172.21.15.16:6835/307899603,v1:172.21.15.16:6837/307899603] conn(0x55cebb293520 0x55cebb283b70 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:14.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.533+0000 7f7f7a496b80 1 --2- 172.21.15.16:0/3012729382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cebb28f450 0x55cebb2a2140 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:14.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.533+0000 7f7f7a496b80 1 --2- 172.21.15.16:0/3012729382 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55cebb2628c0 0x55cebb310580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:14.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.533+0000 7f7f46ffd700 0 client.7692 ms_handle_reset on v2:172.21.15.16:6835/307899603 2023-05-20T01:26:14.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.533+0000 7f7f7a496b80 1 --2- 172.21.15.16:0/3012729382 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cebb28b750 0x55cebb2a1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:14.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.533+0000 7f7f7a496b80 1 -- 172.21.15.16:0/3012729382 >> 172.21.15.16:0/3012729382 conn(0x55cebb259f80 msgr2=0x55cebb25e170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:14.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.533+0000 7f7f7a496b80 1 -- 172.21.15.16:0/3012729382 shutdown_connections 2023-05-20T01:26:14.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.533+0000 7f7f7a496b80 1 -- 172.21.15.16:0/3012729382 wait complete. 2023-05-20T01:26:14.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.533+0000 7f7f7a496b80 20 client.7692 trim_cache size 0 max 16384 2023-05-20T01:26:14.548 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-05-20T01:26:14.548 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2023-05-20T01:26:14.549 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2023-05-20T01:26:14.549 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-05-20T01:26:14.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.534+0000 7f7f6e30a700 1 -- 172.21.15.16:0/3849293873 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f7f4806ace0 msgr2=0x7f7f4806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:14.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.534+0000 7f7f6e30a700 1 --2- 172.21.15.16:0/3849293873 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f7f4806ace0 0x7f7f4806d190 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f7f500089b0 tx=0x7f7f50008040 comp rx=0 tx=0).stop 2023-05-20T01:26:14.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.534+0000 7f7f6e30a700 1 -- 172.21.15.16:0/3849293873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f680ce0a0 msgr2=0x7f7f680c4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:14.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.534+0000 7f7f6e30a700 1 --2- 172.21.15.16:0/3849293873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f680ce0a0 0x7f7f680c4770 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f7f580090b0 tx=0x7f7f58004750 comp rx=0 tx=0).stop 2023-05-20T01:26:14.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.534+0000 7f7f6e30a700 1 -- 172.21.15.16:0/3849293873 shutdown_connections 2023-05-20T01:26:14.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.534+0000 7f7f6e30a700 1 --2- 172.21.15.16:0/3849293873 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f68158f90 0x7f7f680c4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:14.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.534+0000 7f7f6e30a700 1 --2- 172.21.15.16:0/3849293873 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f7f4806ace0 0x7f7f4806d190 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:14.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.534+0000 7f7f6e30a700 1 --2- 172.21.15.16:0/3849293873 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7f680cc050 0x7f7f680b0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:14.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.534+0000 7f7f6e30a700 1 --2- 172.21.15.16:0/3849293873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f680ce0a0 0x7f7f680c4770 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:14.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.534+0000 7f7f6e30a700 1 -- 172.21.15.16:0/3849293873 >> 172.21.15.16:0/3849293873 conn(0x7f7f6806cbb0 msgr2=0x7f7f68157af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:14.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.534+0000 7f7f6e30a700 1 -- 172.21.15.16:0/3849293873 shutdown_connections 2023-05-20T01:26:14.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:14.534+0000 7f7f6e30a700 1 -- 172.21.15.16:0/3849293873 wait complete. 2023-05-20T01:26:14.552 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-05-20T01:26:14.552 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:26:14.553 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo ls subdir) 2023-05-20T01:26:14.600 INFO:teuthology.orchestra.run.smithi016.stdout:sixmegs 2023-05-20T01:26:14.604 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-05-20T01:26:14.604 DEBUG:teuthology.orchestra.run.smithi016:> import os 2023-05-20T01:26:14.605 DEBUG:teuthology.orchestra.run.smithi016:> import stat 2023-05-20T01:26:14.605 DEBUG:teuthology.orchestra.run.smithi016:> import json 2023-05-20T01:26:14.605 DEBUG:teuthology.orchestra.run.smithi016:> import sys 2023-05-20T01:26:14.605 DEBUG:teuthology.orchestra.run.smithi016:> 2023-05-20T01:26:14.606 DEBUG:teuthology.orchestra.run.smithi016:> try: 2023-05-20T01:26:14.606 DEBUG:teuthology.orchestra.run.smithi016:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/subdir/sixmegs'"'"') 2023-05-20T01:26:14.606 DEBUG:teuthology.orchestra.run.smithi016:> except OSError as e: 2023-05-20T01:26:14.607 DEBUG:teuthology.orchestra.run.smithi016:> sys.exit(e.errno) 2023-05-20T01:26:14.607 DEBUG:teuthology.orchestra.run.smithi016:> 2023-05-20T01:26:14.607 DEBUG:teuthology.orchestra.run.smithi016:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2023-05-20T01:26:14.607 DEBUG:teuthology.orchestra.run.smithi016:> print(json.dumps( 2023-05-20T01:26:14.608 DEBUG:teuthology.orchestra.run.smithi016:> dict([(a, getattr(s, a)) for a in attrs]), 2023-05-20T01:26:14.608 DEBUG:teuthology.orchestra.run.smithi016:> indent=2)) 2023-05-20T01:26:14.608 DEBUG:teuthology.orchestra.run.smithi016:> ' 2023-05-20T01:26:14.710 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-05-20T01:26:14.710 INFO:teuthology.orchestra.run.smithi016.stdout: "st_mode": 33088, 2023-05-20T01:26:14.710 INFO:teuthology.orchestra.run.smithi016.stdout: "st_ino": 1099511627777, 2023-05-20T01:26:14.710 INFO:teuthology.orchestra.run.smithi016.stdout: "st_dev": 47, 2023-05-20T01:26:14.710 INFO:teuthology.orchestra.run.smithi016.stdout: "st_nlink": 1, 2023-05-20T01:26:14.711 INFO:teuthology.orchestra.run.smithi016.stdout: "st_uid": 0, 2023-05-20T01:26:14.711 INFO:teuthology.orchestra.run.smithi016.stdout: "st_gid": 0, 2023-05-20T01:26:14.711 INFO:teuthology.orchestra.run.smithi016.stdout: "st_size": 6291456, 2023-05-20T01:26:14.711 INFO:teuthology.orchestra.run.smithi016.stdout: "st_atime": 1684545930.0, 2023-05-20T01:26:14.711 INFO:teuthology.orchestra.run.smithi016.stdout: "st_mtime": 1684545930.0, 2023-05-20T01:26:14.711 INFO:teuthology.orchestra.run.smithi016.stdout: "st_ctime": 1684545930.0 2023-05-20T01:26:14.711 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-05-20T01:26:14.877 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-05-20T01:26:14.877 DEBUG:teuthology.orchestra.run.smithi016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-05-20T01:26:14.965 DEBUG:teuthology.orchestra.run.smithi132:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-05-20T01:26:15.012 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-05-20T01:26:15.013 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-05-20T01:26:15.013 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2023-05-20T01:26:15.031 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-05-20T01:26:15.032 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2023-05-20T01:26:15.089 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-20T01:26:15.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.249+0000 7f58ac46d700 1 Processor -- start 2023-05-20T01:26:15.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.249+0000 7f58ac46d700 1 -- start start 2023-05-20T01:26:15.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.250+0000 7f58ac46d700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f58a40cb670 0x7f58a40ce340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:15.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.250+0000 7f58ac46d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f58a40caaa0 0x7f58a40ce880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:15.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.250+0000 7f58ac46d700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58a4159d90 con 0x7f58a41599c0 2023-05-20T01:26:15.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.250+0000 7f58ac46d700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58a40cdf20 con 0x7f58a40caaa0 2023-05-20T01:26:15.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.250+0000 7f58ac46d700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58a40cdb80 con 0x7f58a40cb670 2023-05-20T01:26:15.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.250+0000 7f58aaa0a700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f58a40caaa0 0x7f58a40ce880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:15.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.250+0000 7f58aaa0a700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f58a40caaa0 0x7f58a40ce880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37412/0 (socket says 172.21.15.16:37412) 2023-05-20T01:26:15.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.250+0000 7f58aaa0a700 1 -- 172.21.15.16:0/890040471 learned_addr learned my addr 172.21.15.16:0/890040471 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:26:15.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.250+0000 7f58a9a08700 1 --2- 172.21.15.16:0/890040471 >> v2:172.21.15.132:3300/0 conn(0x7f58a40cb670 0x7f58a40ce340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:15.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.250+0000 7f58aaa0a700 1 -- 172.21.15.16:0/890040471 >> v2:172.21.15.132:3300/0 conn(0x7f58a40cb670 msgr2=0x7f58a40ce340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:15.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.250+0000 7f58aaa0a700 1 --2- 172.21.15.16:0/890040471 >> v2:172.21.15.132:3300/0 conn(0x7f58a40cb670 0x7f58a40ce340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:15.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.250+0000 7f58aaa0a700 1 -- 172.21.15.16:0/890040471 >> v1:172.21.15.132:6789/0 conn(0x7f58a41599c0 legacy=0x7f58a40caf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:15.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.250+0000 7f58aaa0a700 1 -- 172.21.15.16:0/890040471 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58a406eaf0 con 0x7f58a40caaa0 2023-05-20T01:26:15.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.251+0000 7f58aaa0a700 1 --2- 172.21.15.16:0/890040471 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58a40caaa0 0x7f58a40ce880 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f58a40cc990 tx=0x7f589c00d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bba848ab5be31bf server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:15.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.251+0000 7f58a9207700 1 -- 172.21.15.16:0/890040471 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f589c010070 con 0x7f58a40caaa0 2023-05-20T01:26:15.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.251+0000 7f58a9207700 1 -- 172.21.15.16:0/890040471 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f589c00dd50 con 0x7f58a40caaa0 2023-05-20T01:26:15.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.251+0000 7f58a9207700 1 -- 172.21.15.16:0/890040471 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f589c00b310 con 0x7f58a40caaa0 2023-05-20T01:26:15.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.251+0000 7f58ac46d700 1 -- 172.21.15.16:0/890040471 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58a40caaa0 msgr2=0x7f58a40ce880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:15.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.251+0000 7f58ac46d700 1 --2- 172.21.15.16:0/890040471 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58a40caaa0 0x7f58a40ce880 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f58a40cc990 tx=0x7f589c00d430 comp rx=0 tx=0).stop 2023-05-20T01:26:15.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.251+0000 7f58ac46d700 1 -- 172.21.15.16:0/890040471 shutdown_connections 2023-05-20T01:26:15.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.251+0000 7f58ac46d700 1 --2- 172.21.15.16:0/890040471 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58a40caaa0 0x7f58a40ce880 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:15.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.251+0000 7f58ac46d700 1 --2- 172.21.15.16:0/890040471 >> v2:172.21.15.132:3300/0 conn(0x7f58a40cb670 0x7f58a40ce340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:15.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.251+0000 7f58ac46d700 1 -- 172.21.15.16:0/890040471 >> 172.21.15.16:0/890040471 conn(0x7f58a40d0c30 msgr2=0x7f58a40b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:15.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.251+0000 7f58ac46d700 1 -- 172.21.15.16:0/890040471 shutdown_connections 2023-05-20T01:26:15.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.251+0000 7f58ac46d700 1 -- 172.21.15.16:0/890040471 wait complete. 2023-05-20T01:26:15.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.253+0000 7f58ac46d700 1 Processor -- start 2023-05-20T01:26:15.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.253+0000 7f58ac46d700 1 -- start start 2023-05-20T01:26:15.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.253+0000 7f58ac46d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58a40caaa0 0x7f58a41f2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:15.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.253+0000 7f58ac46d700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f58a40cb670 0x7f58a40b4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:15.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.253+0000 7f58ac46d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58a41599c0 0x7f58a40b5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:15.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.253+0000 7f58ac46d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58a40ae820 con 0x7f58a40caaa0 2023-05-20T01:26:15.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.253+0000 7f58ac46d700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58a41f33e0 con 0x7f58a41599c0 2023-05-20T01:26:15.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.253+0000 7f58ac46d700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58a41f59f0 con 0x7f58a40cb670 2023-05-20T01:26:15.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.253+0000 7f58aa209700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58a40caaa0 0x7f58a41f2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:15.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.253+0000 7f58aa209700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58a40caaa0 0x7f58a41f2ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37414/0 (socket says 172.21.15.16:37414) 2023-05-20T01:26:15.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.253+0000 7f58aa209700 1 -- 172.21.15.16:0/216828863 learned_addr learned my addr 172.21.15.16:0/216828863 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:26:15.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.254+0000 7f58a9a08700 1 --2- 172.21.15.16:0/216828863 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f58a40cb670 0x7f58a40b4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:15.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.254+0000 7f58aaa0a700 1 --2- 172.21.15.16:0/216828863 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58a41599c0 0x7f58a40b5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:15.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.254+0000 7f58aa209700 1 -- 172.21.15.16:0/216828863 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f58a40cb670 msgr2=0x7f58a40b4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:15.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.254+0000 7f58aa209700 1 --2- 172.21.15.16:0/216828863 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f58a40cb670 0x7f58a40b4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:15.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.254+0000 7f58aa209700 1 -- 172.21.15.16:0/216828863 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58a41599c0 msgr2=0x7f58a40b5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:15.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.254+0000 7f58aa209700 1 --2- 172.21.15.16:0/216828863 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58a41599c0 0x7f58a40b5170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:15.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.254+0000 7f58aa209700 1 -- 172.21.15.16:0/216828863 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f589c00bd40 con 0x7f58a40caaa0 2023-05-20T01:26:15.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.254+0000 7f58aa209700 1 --2- 172.21.15.16:0/216828863 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58a40caaa0 0x7f58a41f2ea0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f589400b4a0 tx=0x7f589400b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:15.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.254+0000 7f589affd700 1 -- 172.21.15.16:0/216828863 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5894011830 con 0x7f58a40caaa0 2023-05-20T01:26:15.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.254+0000 7f589affd700 1 -- 172.21.15.16:0/216828863 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5894011990 con 0x7f58a40caaa0 2023-05-20T01:26:15.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.254+0000 7f58ac46d700 1 -- 172.21.15.16:0/216828863 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58a406f0b0 con 0x7f58a40caaa0 2023-05-20T01:26:15.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.254+0000 7f589affd700 1 -- 172.21.15.16:0/216828863 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5894011d40 con 0x7f58a40caaa0 2023-05-20T01:26:15.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.254+0000 7f58ac46d700 1 -- 172.21.15.16:0/216828863 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58a406edd0 con 0x7f58a40caaa0 2023-05-20T01:26:15.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.255+0000 7f58ac46d700 1 -- 172.21.15.16:0/216828863 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58a406e750 con 0x7f58a40caaa0 2023-05-20T01:26:15.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.256+0000 7f589affd700 1 -- 172.21.15.16:0/216828863 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f589401a400 con 0x7f58a40caaa0 2023-05-20T01:26:15.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.256+0000 7f589affd700 1 --2- 172.21.15.16:0/216828863 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f588806ace0 0x7f588806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:15.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.256+0000 7f589affd700 1 -- 172.21.15.16:0/216828863 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 7044+0+0 (secure 0 0 0) 0x7f58940893c0 con 0x7f58a40caaa0 2023-05-20T01:26:15.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.256+0000 7f58a9a08700 1 --2- 172.21.15.16:0/216828863 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f588806ace0 0x7f588806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:15.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.257+0000 7f58a9a08700 1 --2- 172.21.15.16:0/216828863 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f588806ace0 0x7f588806d190 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f58a40b4af0 tx=0x7f58a0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:15.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.261+0000 7f589affd700 1 -- 172.21.15.16:0/216828863 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f5894054120 con 0x7f58a40caaa0 2023-05-20T01:26:15.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.464+0000 7f58ac46d700 1 -- 172.21.15.16:0/216828863 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f58a40747b0 con 0x7f58a40caaa0 2023-05-20T01:26:15.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.466+0000 7f589affd700 1 -- 172.21.15.16:0/216828863 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) v1 ==== 74+0+20623 (secure 0 0 0) 0x7f5894057740 con 0x7f58a40caaa0 2023-05-20T01:26:15.470 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:26:15.470 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":97,"fsid":"c2245cc5-ecab-4087-a59b-bc81db901676","created":"2023-05-20T01:16:09.549230+0000","modified":"2023-05-20T01:25:37.774232+0000","last_up_change":"2023-05-20T01:16:19.788624+0000","last_in_change":"2023-05-20T01:16:12.092160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-05-20T01:16:20.934366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-05-20T01:16:21.372294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-05-20T01:25:09.967348+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-05-20T01:25:10.956221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data_extra","create_time":"2023-05-20T01:25:24.629648+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"317a83cd-19c8-4958-a301-6087ef342289","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6825","nonce":3923634179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6827","nonce":3923634179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6831","nonce":3923634179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6829","nonce":3923634179}]},"public_addr":"172.21.15.16:6825/3923634179","cluster_addr":"172.21.15.16:6827/3923634179","heartbeat_back_addr":"172.21.15.16:6831/3923634179","heartbeat_front_addr":"172.21.15.16:6829/3923634179","state":["exists","up"]},{"osd":1,"uuid":"5581046f-8219-4089-8d7d-b7846399505e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6809","nonce":3860264220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6811","nonce":3860264220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6815","nonce":3860264220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6813","nonce":3860264220}]},"public_addr":"172.21.15.16:6809/3860264220","cluster_addr":"172.21.15.16:6811/3860264220","heartbeat_back_addr":"172.21.15.16:6815/3860264220","heartbeat_front_addr":"172.21.15.16:6813/3860264220","state":["exists","up"]},{"osd":2,"uuid":"072e9e66-4133-478e-8d4a-64d46c2963f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6817","nonce":293187030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6819","nonce":293187030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6823","nonce":293187030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6821","nonce":293187030}]},"public_addr":"172.21.15.16:6817/293187030","cluster_addr":"172.21.15.16:6819/293187030","heartbeat_back_addr":"172.21.15.16:6823/293187030","heartbeat_front_addr":"172.21.15.16:6821/293187030","state":["exists","up"]},{"osd":3,"uuid":"11e1bba6-00f8-48dd-9028-d8d716b7f145","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6801","nonce":3669781890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6803","nonce":3669781890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6807","nonce":3669781890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6805","nonce":3669781890}]},"public_addr":"172.21.15.16:6801/3669781890","cluster_addr":"172.21.15.16:6803/3669781890","heartbeat_back_addr":"172.21.15.16:6807/3669781890","heartbeat_front_addr":"172.21.15.16:6805/3669781890","state":["exists","up"]},{"osd":4,"uuid":"b8909c3a-cf70-46cb-8d0b-009c707cf9d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6819","nonce":506317223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6823","nonce":506317223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6829","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6831","nonce":506317223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6827","nonce":506317223}]},"public_addr":"172.21.15.132:6819/506317223","cluster_addr":"172.21.15.132:6823/506317223","heartbeat_back_addr":"172.21.15.132:6831/506317223","heartbeat_front_addr":"172.21.15.132:6827/506317223","state":["exists","up"]},{"osd":5,"uuid":"b1818304-4583-47bf-a6ec-0bfbb5166233","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6818","nonce":1844689995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6822","nonce":1844689995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6830","nonce":1844689995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6826","nonce":1844689995}]},"public_addr":"172.21.15.132:6818/1844689995","cluster_addr":"172.21.15.132:6822/1844689995","heartbeat_back_addr":"172.21.15.132:6830/1844689995","heartbeat_front_addr":"172.21.15.132:6826/1844689995","state":["exists","up"]},{"osd":6,"uuid":"d37f4691-eed1-4a93-8818-18354f03f4a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6801","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6802","nonce":727581123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6805","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6807","nonce":727581123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6813","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6815","nonce":727581123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6811","nonce":727581123}]},"public_addr":"172.21.15.132:6802/727581123","cluster_addr":"172.21.15.132:6807/727581123","heartbeat_back_addr":"172.21.15.132:6815/727581123","heartbeat_front_addr":"172.21.15.132:6811/727581123","state":["exists","up"]},{"osd":7,"uuid":"4f712f62-5d1d-4816-8042-03209144f8b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6803","nonce":466001149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6806","nonce":466001149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6814","nonce":466001149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6810","nonce":466001149}]},"public_addr":"172.21.15.132:6803/466001149","cluster_addr":"172.21.15.132:6806/466001149","heartbeat_back_addr":"172.21.15.132:6814/466001149","heartbeat_front_addr":"172.21.15.132:6810/466001149","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.338863+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.278680+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305694+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.246835+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.276237+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.304441+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305421+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.267050+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6834/2740585996":"2023-05-21T01:25:37.758863+0000","172.21.15.16:6837/2740585996":"2023-05-21T01:25:37.758863+0000","172.21.15.16:6837/1327905747":"2023-05-21T01:25:32.534353+0000","172.21.15.16:6834/1327905747":"2023-05-21T01:25:32.534353+0000","172.21.15.16:6837/28561908":"2023-05-21T01:25:30.939570+0000","172.21.15.16:6834/28561908":"2023-05-21T01:25:30.939570+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-20T01:26:15.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.468+0000 7f58ac46d700 1 -- 172.21.15.16:0/216828863 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f588806ace0 msgr2=0x7f588806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:15.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.468+0000 7f58ac46d700 1 --2- 172.21.15.16:0/216828863 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f588806ace0 0x7f588806d190 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f58a40b4af0 tx=0x7f58a0008040 comp rx=0 tx=0).stop 2023-05-20T01:26:15.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.468+0000 7f58ac46d700 1 -- 172.21.15.16:0/216828863 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58a40caaa0 msgr2=0x7f58a41f2ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:15.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.468+0000 7f58ac46d700 1 --2- 172.21.15.16:0/216828863 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58a40caaa0 0x7f58a41f2ea0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f589400b4a0 tx=0x7f589400b7b0 comp rx=0 tx=0).stop 2023-05-20T01:26:15.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.468+0000 7f58ac46d700 1 -- 172.21.15.16:0/216828863 shutdown_connections 2023-05-20T01:26:15.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.468+0000 7f58ac46d700 1 --2- 172.21.15.16:0/216828863 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58a41599c0 0x7f58a40b5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:15.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.468+0000 7f58ac46d700 1 --2- 172.21.15.16:0/216828863 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58a40caaa0 0x7f58a41f2ea0 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:15.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.468+0000 7f58ac46d700 1 --2- 172.21.15.16:0/216828863 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f588806ace0 0x7f588806d190 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:15.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.468+0000 7f58ac46d700 1 --2- 172.21.15.16:0/216828863 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f58a40cb670 0x7f58a40b4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:15.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.468+0000 7f58ac46d700 1 -- 172.21.15.16:0/216828863 >> 172.21.15.16:0/216828863 conn(0x7f58a40d0c30 msgr2=0x7f58a4158910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:15.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.468+0000 7f58ac46d700 1 -- 172.21.15.16:0/216828863 shutdown_connections 2023-05-20T01:26:15.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.468+0000 7f58ac46d700 1 -- 172.21.15.16:0/216828863 wait complete. 2023-05-20T01:26:15.485 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-05-20T01:16:20.934366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-05-20T01:16:21.372294+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2023-05-20T01:25:09.967348+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2023-05-20T01:25:10.956221+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data_extra', 'create_time': '2023-05-20T01:25:24.629648+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-05-20T01:26:15.487 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-05-20T01:26:15.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.612+0000 7f135244d700 1 Processor -- start 2023-05-20T01:26:15.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.612+0000 7f135244d700 1 -- start start 2023-05-20T01:26:15.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.613+0000 7f135244d700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f134c15d290 0x7f134c0cf9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:15.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.613+0000 7f135244d700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f134c0cd630 con 0x7f134c15cec0 2023-05-20T01:26:15.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.613+0000 7f135244d700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f134c0cca60 con 0x7f134c072300 2023-05-20T01:26:15.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.613+0000 7f135244d700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f134c0ccd00 con 0x7f134c15d290 2023-05-20T01:26:15.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.613+0000 7f134bfff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f134c15cec0 0x7f134c071bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:48950/0 (socket says 172.21.15.16:48950) 2023-05-20T01:26:15.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.613+0000 7f134bfff700 1 -- 172.21.15.16:0/2580918349 learned_addr learned my addr 172.21.15.16:0/2580918349 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:26:15.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.613+0000 7f13509ea700 1 --2- 172.21.15.16:0/2580918349 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f134c15d290 0x7f134c0cf9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:15.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.614+0000 7f134affd700 1 -- 172.21.15.16:0/2580918349 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2521041429 0 0) 0x7f134c0cd630 con 0x7f134c15cec0 2023-05-20T01:26:15.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.614+0000 7f134affd700 1 -- 172.21.15.16:0/2580918349 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13340035b0 con 0x7f134c15cec0 2023-05-20T01:26:15.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.614+0000 7f134affd700 1 -- 172.21.15.16:0/2580918349 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4104184112 0 0) 0x7f134c0cca60 con 0x7f134c072300 2023-05-20T01:26:15.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.614+0000 7f134affd700 1 -- 172.21.15.16:0/2580918349 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f134c0cd630 con 0x7f134c072300 2023-05-20T01:26:15.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.614+0000 7f134affd700 1 -- 172.21.15.16:0/2580918349 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1156452216 0 0) 0x7f13340035b0 con 0x7f134c15cec0 2023-05-20T01:26:15.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.614+0000 7f134affd700 1 -- 172.21.15.16:0/2580918349 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f134c15d290 msgr2=0x7f134c0cf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:15.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.614+0000 7f134affd700 1 --2- 172.21.15.16:0/2580918349 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f134c15d290 0x7f134c0cf9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:15.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.614+0000 7f134affd700 1 -- 172.21.15.16:0/2580918349 >> v1:172.21.15.132:6789/0 conn(0x7f134c072300 legacy=0x7f134c0cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:15.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.614+0000 7f134affd700 1 -- 172.21.15.16:0/2580918349 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f134c0ccfe0 con 0x7f134c15cec0 2023-05-20T01:26:15.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.614+0000 7f134affd700 1 -- 172.21.15.16:0/2580918349 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7f13400017f0 con 0x7f134c15cec0 2023-05-20T01:26:15.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.614+0000 7f134affd700 1 -- 172.21.15.16:0/2580918349 >> v1:172.21.15.16:6789/0 conn(0x7f134c15cec0 legacy=0x7f134c071bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:15.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.614+0000 7f134affd700 1 --2- 172.21.15.16:0/2580918349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1334003bc0 0x7f1334003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:15.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.614+0000 7f134affd700 1 --2- 172.21.15.16:0/2580918349 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f134c15d290 0x7f1334004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:15.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.614+0000 7f134affd700 1 --2- 172.21.15.16:0/2580918349 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1334004d90 0x7f1334005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:15.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.614+0000 7f134affd700 1 -- 172.21.15.16:0/2580918349 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f134c0ccd00 con 0x7f1334003bc0 2023-05-20T01:26:15.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.614+0000 7f134affd700 1 -- 172.21.15.16:0/2580918349 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f133400bba0 con 0x7f1334004d90 2023-05-20T01:26:15.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.614+0000 7f134affd700 1 -- 172.21.15.16:0/2580918349 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f133400bd10 con 0x7f134c15d290 2023-05-20T01:26:15.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.614+0000 7f134bfff700 1 --2- 172.21.15.16:0/2580918349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1334003bc0 0x7f1334003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:15.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.614+0000 7f134b7fe700 1 --2- 172.21.15.16:0/2580918349 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f134c15d290 0x7f1334004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:15.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.614+0000 7f13509ea700 1 --2- 172.21.15.16:0/2580918349 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1334004d90 0x7f1334005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:15.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.614+0000 7f134bfff700 1 -- 172.21.15.16:0/2580918349 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f134c15d290 msgr2=0x7f1334004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:15.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.614+0000 7f134bfff700 1 --2- 172.21.15.16:0/2580918349 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f134c15d290 0x7f1334004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:15.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.614+0000 7f134bfff700 1 -- 172.21.15.16:0/2580918349 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1334004d90 msgr2=0x7f1334005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:15.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.614+0000 7f134bfff700 1 --2- 172.21.15.16:0/2580918349 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1334004d90 0x7f1334005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:15.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.614+0000 7f134bfff700 1 -- 172.21.15.16:0/2580918349 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13340035b0 con 0x7f1334003bc0 2023-05-20T01:26:15.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.615+0000 7f134bfff700 1 --2- 172.21.15.16:0/2580918349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1334003bc0 0x7f1334003fb0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f134000c0e0 tx=0x7f1334005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9202621f6d136e9 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:15.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.615+0000 7f134affd700 1 -- 172.21.15.16:0/2580918349 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f134000cc80 con 0x7f1334003bc0 2023-05-20T01:26:15.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.615+0000 7f134affd700 1 -- 172.21.15.16:0/2580918349 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f134000cde0 con 0x7f1334003bc0 2023-05-20T01:26:15.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.615+0000 7f134affd700 1 -- 172.21.15.16:0/2580918349 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1340007310 con 0x7f1334003bc0 2023-05-20T01:26:15.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.615+0000 7f135244d700 1 -- 172.21.15.16:0/2580918349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1334003bc0 msgr2=0x7f1334003fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:15.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.615+0000 7f135244d700 1 --2- 172.21.15.16:0/2580918349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1334003bc0 0x7f1334003fb0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f134000c0e0 tx=0x7f1334005880 comp rx=0 tx=0).stop 2023-05-20T01:26:15.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.615+0000 7f13509ea700 1 -- 172.21.15.16:0/2580918349 reap_dead start 2023-05-20T01:26:15.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.615+0000 7f135244d700 1 -- 172.21.15.16:0/2580918349 shutdown_connections 2023-05-20T01:26:15.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.615+0000 7f135244d700 1 -- 172.21.15.16:0/2580918349 >> 172.21.15.16:0/2580918349 conn(0x7f134c0cffd0 msgr2=0x7f134c0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:15.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.615+0000 7f135244d700 1 -- 172.21.15.16:0/2580918349 shutdown_connections 2023-05-20T01:26:15.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.615+0000 7f135244d700 1 -- 172.21.15.16:0/2580918349 wait complete. 2023-05-20T01:26:15.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.617+0000 7f135244d700 1 Processor -- start 2023-05-20T01:26:15.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.617+0000 7f135244d700 1 -- start start 2023-05-20T01:26:15.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.618+0000 7f135244d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1334003bc0 0x7f134c1f1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:15.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.618+0000 7f135244d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f134c0ae1d0 0x7f134c0aefe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:15.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.618+0000 7f135244d700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f134c15dd20 0x7f134c15e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:15.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.618+0000 7f135244d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f134c15e6e0 con 0x7f134c0ae1d0 2023-05-20T01:26:15.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.618+0000 7f135244d700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f134c0b2e40 con 0x7f1334003bc0 2023-05-20T01:26:15.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.618+0000 7f135244d700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f134c0b3090 con 0x7f134c15dd20 2023-05-20T01:26:15.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.618+0000 7f134b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f134c0ae1d0 0x7f134c0aefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:15.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.618+0000 7f134b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f134c0ae1d0 0x7f134c0aefe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37436/0 (socket says 172.21.15.16:37436) 2023-05-20T01:26:15.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.618+0000 7f134b7fe700 1 -- 172.21.15.16:0/1262622349 learned_addr learned my addr 172.21.15.16:0/1262622349 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:26:15.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.618+0000 7f13509ea700 1 --2- 172.21.15.16:0/1262622349 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f134c15dd20 0x7f134c15e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:15.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.618+0000 7f134b7fe700 1 -- 172.21.15.16:0/1262622349 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f134c15dd20 msgr2=0x7f134c15e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:15.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.618+0000 7f134bfff700 1 --2- 172.21.15.16:0/1262622349 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1334003bc0 0x7f134c1f1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:15.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.618+0000 7f134b7fe700 1 --2- 172.21.15.16:0/1262622349 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f134c15dd20 0x7f134c15e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:15.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.618+0000 7f134b7fe700 1 -- 172.21.15.16:0/1262622349 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1334003bc0 msgr2=0x7f134c1f1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:15.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.618+0000 7f134b7fe700 1 --2- 172.21.15.16:0/1262622349 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1334003bc0 0x7f134c1f1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:15.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.618+0000 7f134b7fe700 1 -- 172.21.15.16:0/1262622349 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f134c0cf9d0 con 0x7f134c0ae1d0 2023-05-20T01:26:15.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.619+0000 7f13509ea700 1 --2- 172.21.15.16:0/1262622349 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f134c15dd20 0x7f134c15e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:26:15.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.619+0000 7f134bfff700 1 --2- 172.21.15.16:0/1262622349 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1334003bc0 0x7f134c1f1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:26:15.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.619+0000 7f134b7fe700 1 --2- 172.21.15.16:0/1262622349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f134c0ae1d0 0x7f134c0aefe0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f133c00f7f0 tx=0x7f133c00fbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:15.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.619+0000 7f1348ff9700 1 -- 172.21.15.16:0/1262622349 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f133c005320 con 0x7f134c0ae1d0 2023-05-20T01:26:15.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.619+0000 7f1348ff9700 1 -- 172.21.15.16:0/1262622349 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f133c005480 con 0x7f134c0ae1d0 2023-05-20T01:26:15.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.619+0000 7f135244d700 1 -- 172.21.15.16:0/1262622349 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f134c0cd2c0 con 0x7f134c0ae1d0 2023-05-20T01:26:15.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.619+0000 7f1348ff9700 1 -- 172.21.15.16:0/1262622349 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f133c006aa0 con 0x7f134c0ae1d0 2023-05-20T01:26:15.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.619+0000 7f135244d700 1 -- 172.21.15.16:0/1262622349 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f134c0cfb60 con 0x7f134c0ae1d0 2023-05-20T01:26:15.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.620+0000 7f135244d700 1 -- 172.21.15.16:0/1262622349 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f134c0747b0 con 0x7f134c0ae1d0 2023-05-20T01:26:15.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.620+0000 7f1348ff9700 1 -- 172.21.15.16:0/1262622349 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f133c006c00 con 0x7f134c0ae1d0 2023-05-20T01:26:15.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.621+0000 7f1348ff9700 1 --2- 172.21.15.16:0/1262622349 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f133806ace0 0x7f133806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:15.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.621+0000 7f1348ff9700 1 -- 172.21.15.16:0/1262622349 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 7044+0+0 (secure 0 0 0) 0x7f133c089170 con 0x7f134c0ae1d0 2023-05-20T01:26:15.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.621+0000 7f134bfff700 1 --2- 172.21.15.16:0/1262622349 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f133806ace0 0x7f133806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:15.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.621+0000 7f134bfff700 1 --2- 172.21.15.16:0/1262622349 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f133806ace0 0x7f133806d190 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f1340000c00 tx=0x7f1340007000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:15.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.625+0000 7f1348ff9700 1 -- 172.21.15.16:0/1262622349 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f133c053ed0 con 0x7f134c0ae1d0 2023-05-20T01:26:15.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.822+0000 7f135244d700 1 -- 172.21.15.16:0/1262622349 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f134c0d0d00 con 0x7f134c0ae1d0 2023-05-20T01:26:15.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.823+0000 7f1348ff9700 1 -- 172.21.15.16:0/1262622349 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v97) v1 ==== 92+0+10 (secure 0 0 0) 0x7f133c0574f0 con 0x7f134c0ae1d0 2023-05-20T01:26:15.827 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 8 2023-05-20T01:26:15.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.825+0000 7f135244d700 1 -- 172.21.15.16:0/1262622349 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f133806ace0 msgr2=0x7f133806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:15.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.825+0000 7f135244d700 1 --2- 172.21.15.16:0/1262622349 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f133806ace0 0x7f133806d190 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f1340000c00 tx=0x7f1340007000 comp rx=0 tx=0).stop 2023-05-20T01:26:15.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.825+0000 7f135244d700 1 -- 172.21.15.16:0/1262622349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f134c0ae1d0 msgr2=0x7f134c0aefe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:15.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.825+0000 7f135244d700 1 --2- 172.21.15.16:0/1262622349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f134c0ae1d0 0x7f134c0aefe0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f133c00f7f0 tx=0x7f133c00fbb0 comp rx=0 tx=0).stop 2023-05-20T01:26:15.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.825+0000 7f135244d700 1 -- 172.21.15.16:0/1262622349 shutdown_connections 2023-05-20T01:26:15.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.825+0000 7f135244d700 1 --2- 172.21.15.16:0/1262622349 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f133806ace0 0x7f133806d190 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:15.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.825+0000 7f135244d700 1 --2- 172.21.15.16:0/1262622349 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f134c15dd20 0x7f134c15e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:15.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.825+0000 7f135244d700 1 --2- 172.21.15.16:0/1262622349 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1334003bc0 0x7f134c1f1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:15.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.825+0000 7f135244d700 1 --2- 172.21.15.16:0/1262622349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f134c0ae1d0 0x7f134c0aefe0 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:15.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.825+0000 7f135244d700 1 -- 172.21.15.16:0/1262622349 >> 172.21.15.16:0/1262622349 conn(0x7f134c0cffd0 msgr2=0x7f134c0cacc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:15.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.825+0000 7f135244d700 1 -- 172.21.15.16:0/1262622349 shutdown_connections 2023-05-20T01:26:15.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.825+0000 7f135244d700 1 -- 172.21.15.16:0/1262622349 wait complete. 2023-05-20T01:26:15.841 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-05-20T01:26:15.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.964+0000 7fa9b560f700 1 Processor -- start 2023-05-20T01:26:15.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.964+0000 7fa9b560f700 1 -- start start 2023-05-20T01:26:15.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.965+0000 7fa9b560f700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa9b00cd890 0x7fa9b00b6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:15.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.965+0000 7fa9b560f700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fa9b00b65d0 0x7fa9b00cf570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:15.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.965+0000 7fa9b560f700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9b00d1820 con 0x7fa9b015ca90 2023-05-20T01:26:15.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.965+0000 7fa9b560f700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9b00d0c50 con 0x7fa9b00b65d0 2023-05-20T01:26:15.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.965+0000 7fa9b560f700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9b00cdce0 con 0x7fa9b00cd890 2023-05-20T01:26:15.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.965+0000 7fa9ae7fc700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa9b00cd890 0x7fa9b00b6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:15.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.965+0000 7fa9af7fe700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fa9b00b65d0 0x7fa9b00cf570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:15.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.965+0000 7fa9af7fe700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fa9b00b65d0 0x7fa9b00cf570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.16:34466/0 (socket says 172.21.15.16:34466) 2023-05-20T01:26:15.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.965+0000 7fa9af7fe700 1 -- 172.21.15.16:0/3253459425 learned_addr learned my addr 172.21.15.16:0/3253459425 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:26:15.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.965+0000 7fa9aeffd700 1 --1- 172.21.15.16:0/3253459425 >> v1:172.21.15.132:6789/0 conn(0x7fa9b015ca90 0x7fa9b00d2550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.16:58684/0 (socket says 172.21.15.16:58684) 2023-05-20T01:26:15.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.966+0000 7fa9adffb700 1 -- 172.21.15.16:0/3253459425 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2629886385 0 0) 0x7fa9b00d1820 con 0x7fa9b015ca90 2023-05-20T01:26:15.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.966+0000 7fa9af7fe700 1 -- 172.21.15.16:0/3253459425 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa9b00cd890 msgr2=0x7fa9b00b6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:15.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.966+0000 7fa9af7fe700 1 --2- 172.21.15.16:0/3253459425 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa9b00cd890 0x7fa9b00b6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:15.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.966+0000 7fa9af7fe700 1 -- 172.21.15.16:0/3253459425 >> v1:172.21.15.132:6789/0 conn(0x7fa9b015ca90 legacy=0x7fa9b00d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:15.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.966+0000 7fa9af7fe700 1 -- 172.21.15.16:0/3253459425 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9b00d2c60 con 0x7fa9b00b65d0 2023-05-20T01:26:15.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.966+0000 7fa9af7fe700 1 --2- 172.21.15.16:0/3253459425 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9b00b65d0 0x7fa9b00cf570 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fa9a8009e50 tx=0x7fa9a800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5ff62d5cb69e254 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:15.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.966+0000 7fa9adffb700 1 -- 172.21.15.16:0/3253459425 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9a800e070 con 0x7fa9b00b65d0 2023-05-20T01:26:15.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.966+0000 7fa9adffb700 1 -- 172.21.15.16:0/3253459425 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9a800b910 con 0x7fa9b00b65d0 2023-05-20T01:26:15.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.966+0000 7fa9adffb700 1 -- 172.21.15.16:0/3253459425 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9a800bcc0 con 0x7fa9b00b65d0 2023-05-20T01:26:15.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.967+0000 7fa9b560f700 1 -- 172.21.15.16:0/3253459425 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9b00b65d0 msgr2=0x7fa9b00cf570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:15.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.967+0000 7fa9b560f700 1 --2- 172.21.15.16:0/3253459425 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9b00b65d0 0x7fa9b00cf570 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fa9a8009e50 tx=0x7fa9a800b330 comp rx=0 tx=0).stop 2023-05-20T01:26:15.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.967+0000 7fa9b560f700 1 -- 172.21.15.16:0/3253459425 shutdown_connections 2023-05-20T01:26:15.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.967+0000 7fa9b560f700 1 --2- 172.21.15.16:0/3253459425 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9b00b65d0 0x7fa9b00cf570 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:15.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.967+0000 7fa9b560f700 1 --2- 172.21.15.16:0/3253459425 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa9b00cd890 0x7fa9b00b6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:15.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.967+0000 7fa9b560f700 1 -- 172.21.15.16:0/3253459425 >> 172.21.15.16:0/3253459425 conn(0x7fa9b006b000 msgr2=0x7fa9b006b400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:15.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.967+0000 7fa9b560f700 1 -- 172.21.15.16:0/3253459425 shutdown_connections 2023-05-20T01:26:15.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.967+0000 7fa9b560f700 1 -- 172.21.15.16:0/3253459425 wait complete. 2023-05-20T01:26:15.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.968+0000 7fa9b560f700 1 Processor -- start 2023-05-20T01:26:15.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.968+0000 7fa9b560f700 1 -- start start 2023-05-20T01:26:15.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.969+0000 7fa9b560f700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa9b00b65d0 0x7fa9b01f34b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:15.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.969+0000 7fa9b560f700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9b00cd890 0x7fa9b014d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:15.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.969+0000 7fa9b560f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9b015ca90 0x7fa9b00b44f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:15.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.969+0000 7fa9b560f700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9b00cab60 con 0x7fa9b015ca90 2023-05-20T01:26:15.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.969+0000 7fa9b560f700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9b01f39f0 con 0x7fa9b00cd890 2023-05-20T01:26:15.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.969+0000 7fa9b560f700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9b01f27e0 con 0x7fa9b00b65d0 2023-05-20T01:26:15.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.969+0000 7fa9af7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9b015ca90 0x7fa9b00b44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:15.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.969+0000 7fa9af7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9b015ca90 0x7fa9b00b44f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37452/0 (socket says 172.21.15.16:37452) 2023-05-20T01:26:15.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.969+0000 7fa9af7fe700 1 -- 172.21.15.16:0/3776920402 learned_addr learned my addr 172.21.15.16:0/3776920402 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:26:15.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.969+0000 7fa9aeffd700 1 --2- 172.21.15.16:0/3776920402 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa9b00b65d0 0x7fa9b01f34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:15.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.969+0000 7fa9ae7fc700 1 --2- 172.21.15.16:0/3776920402 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9b00cd890 0x7fa9b014d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:15.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.969+0000 7fa9af7fe700 1 -- 172.21.15.16:0/3776920402 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa9b00b65d0 msgr2=0x7fa9b01f34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:15.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.969+0000 7fa9af7fe700 1 --2- 172.21.15.16:0/3776920402 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa9b00b65d0 0x7fa9b01f34b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:15.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.969+0000 7fa9af7fe700 1 -- 172.21.15.16:0/3776920402 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9b00cd890 msgr2=0x7fa9b014d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:15.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.969+0000 7fa9af7fe700 1 --2- 172.21.15.16:0/3776920402 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9b00cd890 0x7fa9b014d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:15.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.969+0000 7fa9af7fe700 1 -- 172.21.15.16:0/3776920402 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9a8009b30 con 0x7fa9b015ca90 2023-05-20T01:26:15.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.969+0000 7fa9ae7fc700 1 --2- 172.21.15.16:0/3776920402 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9b00cd890 0x7fa9b014d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:26:15.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.970+0000 7fa9af7fe700 1 --2- 172.21.15.16:0/3776920402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9b015ca90 0x7fa9b00b44f0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7fa9b008c000 tx=0x7fa9a8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:15.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.970+0000 7fa9a77fe700 1 -- 172.21.15.16:0/3776920402 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9a8003ea0 con 0x7fa9b015ca90 2023-05-20T01:26:15.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.970+0000 7fa9a77fe700 1 -- 172.21.15.16:0/3776920402 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9a80099b0 con 0x7fa9b015ca90 2023-05-20T01:26:15.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.970+0000 7fa9a77fe700 1 -- 172.21.15.16:0/3776920402 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9a801c680 con 0x7fa9b015ca90 2023-05-20T01:26:15.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.970+0000 7fa9b560f700 1 -- 172.21.15.16:0/3776920402 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9b00cdf00 con 0x7fa9b015ca90 2023-05-20T01:26:15.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.970+0000 7fa9b560f700 1 -- 172.21.15.16:0/3776920402 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9b00b6a50 con 0x7fa9b015ca90 2023-05-20T01:26:15.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.971+0000 7fa9b560f700 1 -- 172.21.15.16:0/3776920402 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9b00d1a70 con 0x7fa9b015ca90 2023-05-20T01:26:15.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.971+0000 7fa9a77fe700 1 -- 172.21.15.16:0/3776920402 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa9a801c7e0 con 0x7fa9b015ca90 2023-05-20T01:26:15.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.972+0000 7fa9a77fe700 1 --2- 172.21.15.16:0/3776920402 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa99006ace0 0x7fa99006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:15.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.972+0000 7fa9a77fe700 1 -- 172.21.15.16:0/3776920402 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 7044+0+0 (secure 0 0 0) 0x7fa9a808b1d0 con 0x7fa9b015ca90 2023-05-20T01:26:15.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.972+0000 7fa9aeffd700 1 --2- 172.21.15.16:0/3776920402 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa99006ace0 0x7fa99006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:15.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.972+0000 7fa9aeffd700 1 --2- 172.21.15.16:0/3776920402 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa99006ace0 0x7fa99006d190 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fa9a0000f60 tx=0x7fa9a0007be0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:15.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:15.976+0000 7fa9a77fe700 1 -- 172.21.15.16:0/3776920402 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fa9a8055ea0 con 0x7fa9b015ca90 2023-05-20T01:26:16.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.175+0000 7fa9b560f700 1 -- 172.21.15.16:0/3776920402 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa9b004e8c0 con 0x7fa9b015ca90 2023-05-20T01:26:16.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.175+0000 7fa9a77fe700 1 -- 172.21.15.16:0/3776920402 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v97) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa9a80594c0 con 0x7fa9b015ca90 2023-05-20T01:26:16.180 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-05-20T01:26:16.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.177+0000 7fa9b560f700 1 -- 172.21.15.16:0/3776920402 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa99006ace0 msgr2=0x7fa99006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:16.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.177+0000 7fa9b560f700 1 --2- 172.21.15.16:0/3776920402 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa99006ace0 0x7fa99006d190 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fa9a0000f60 tx=0x7fa9a0007be0 comp rx=0 tx=0).stop 2023-05-20T01:26:16.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.177+0000 7fa9b560f700 1 -- 172.21.15.16:0/3776920402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9b015ca90 msgr2=0x7fa9b00b44f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:16.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.177+0000 7fa9b560f700 1 --2- 172.21.15.16:0/3776920402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9b015ca90 0x7fa9b00b44f0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7fa9b008c000 tx=0x7fa9a8004ad0 comp rx=0 tx=0).stop 2023-05-20T01:26:16.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.177+0000 7fa9b560f700 1 -- 172.21.15.16:0/3776920402 shutdown_connections 2023-05-20T01:26:16.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.177+0000 7fa9b560f700 1 --2- 172.21.15.16:0/3776920402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9b015ca90 0x7fa9b00b44f0 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:16.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.177+0000 7fa9b560f700 1 --2- 172.21.15.16:0/3776920402 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa99006ace0 0x7fa99006d190 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:16.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.177+0000 7fa9b560f700 1 --2- 172.21.15.16:0/3776920402 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa9b00b65d0 0x7fa9b01f34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:16.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.177+0000 7fa9b560f700 1 --2- 172.21.15.16:0/3776920402 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9b00cd890 0x7fa9b014d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:16.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.177+0000 7fa9b560f700 1 -- 172.21.15.16:0/3776920402 >> 172.21.15.16:0/3776920402 conn(0x7fa9b006b000 msgr2=0x7fa9b00d0fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:16.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.177+0000 7fa9b560f700 1 -- 172.21.15.16:0/3776920402 shutdown_connections 2023-05-20T01:26:16.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.177+0000 7fa9b560f700 1 -- 172.21.15.16:0/3776920402 wait complete. 2023-05-20T01:26:16.194 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-05-20T01:26:16.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.317+0000 7f25b05f7700 1 Processor -- start 2023-05-20T01:26:16.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.317+0000 7f25b05f7700 1 -- start start 2023-05-20T01:26:16.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.318+0000 7f25b05f7700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f25a80af2f0 0x7f25a80cec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:16.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.318+0000 7f25b05f7700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f25a80cf180 0x7f25a80b09a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:16.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.318+0000 7f25b05f7700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25a80b2020 con 0x7f25a80d16e0 2023-05-20T01:26:16.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.318+0000 7f25b05f7700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25a80cf600 con 0x7f25a80cf180 2023-05-20T01:26:16.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.318+0000 7f25b05f7700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25a80b2300 con 0x7f25a80af2f0 2023-05-20T01:26:16.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.318+0000 7f25ae393700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f25a80d16e0 0x7f25a8144350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:48984/0 (socket says 172.21.15.16:48984) 2023-05-20T01:26:16.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.318+0000 7f25ae393700 1 -- 172.21.15.16:0/2211386460 learned_addr learned my addr 172.21.15.16:0/2211386460 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:26:16.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.318+0000 7f25aeb94700 1 --2- 172.21.15.16:0/2211386460 >> v2:172.21.15.132:3300/0 conn(0x7f25a80cf180 0x7f25a80b09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:16.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.318+0000 7f25adb92700 1 --2- 172.21.15.16:0/2211386460 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f25a80af2f0 0x7f25a80cec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:16.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.319+0000 7f25ad391700 1 -- 172.21.15.16:0/2211386460 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1958338464 0 0) 0x7f25a80b2020 con 0x7f25a80d16e0 2023-05-20T01:26:16.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.319+0000 7f25ad391700 1 -- 172.21.15.16:0/2211386460 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f25940035b0 con 0x7f25a80d16e0 2023-05-20T01:26:16.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.319+0000 7f25ad391700 1 -- 172.21.15.16:0/2211386460 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3406625740 0 0) 0x7f25940035b0 con 0x7f25a80d16e0 2023-05-20T01:26:16.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.319+0000 7f25ad391700 1 -- 172.21.15.16:0/2211386460 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f25a80af2f0 msgr2=0x7f25a80cec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:16.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.319+0000 7f25ad391700 1 --2- 172.21.15.16:0/2211386460 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f25a80af2f0 0x7f25a80cec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:16.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.319+0000 7f25ad391700 1 -- 172.21.15.16:0/2211386460 >> v2:172.21.15.132:3300/0 conn(0x7f25a80cf180 msgr2=0x7f25a80b09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:16.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.319+0000 7f25ad391700 1 --2- 172.21.15.16:0/2211386460 >> v2:172.21.15.132:3300/0 conn(0x7f25a80cf180 0x7f25a80b09a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:16.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.319+0000 7f25ad391700 1 -- 172.21.15.16:0/2211386460 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25a80b25e0 con 0x7f25a80d16e0 2023-05-20T01:26:16.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.319+0000 7f25ad391700 1 -- 172.21.15.16:0/2211386460 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7f2598001670 con 0x7f25a80d16e0 2023-05-20T01:26:16.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.319+0000 7f25aeb94700 1 --2- 172.21.15.16:0/2211386460 >> v2:172.21.15.132:3300/0 conn(0x7f25a80cf180 0x7f25a80b09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-05-20T01:26:16.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.319+0000 7f25adb92700 1 --2- 172.21.15.16:0/2211386460 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f25a80af2f0 0x7f25a80cec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-05-20T01:26:16.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.319+0000 7f25ad391700 1 -- 172.21.15.16:0/2211386460 >> v1:172.21.15.16:6789/0 conn(0x7f25a80d16e0 legacy=0x7f25a8144350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:16.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.319+0000 7f25ad391700 1 --2- 172.21.15.16:0/2211386460 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2594003a00 0x7f2594003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:16.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.319+0000 7f25ad391700 1 --2- 172.21.15.16:0/2211386460 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f25a80af2f0 0x7f2594004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:16.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.319+0000 7f25ad391700 1 --2- 172.21.15.16:0/2211386460 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2594004d90 0x7f2594005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:16.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.319+0000 7f25ad391700 1 -- 172.21.15.16:0/2211386460 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25a80cf600 con 0x7f2594003a00 2023-05-20T01:26:16.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.319+0000 7f25ad391700 1 -- 172.21.15.16:0/2211386460 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25a80b2300 con 0x7f2594004d90 2023-05-20T01:26:16.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.319+0000 7f25ad391700 1 -- 172.21.15.16:0/2211386460 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f259400ba10 con 0x7f25a80af2f0 2023-05-20T01:26:16.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.319+0000 7f25ae393700 1 --2- 172.21.15.16:0/2211386460 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2594003a00 0x7f2594003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:16.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.319+0000 7f25adb92700 1 --2- 172.21.15.16:0/2211386460 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f25a80af2f0 0x7f2594004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:16.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.319+0000 7f25aeb94700 1 --2- 172.21.15.16:0/2211386460 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2594004d90 0x7f2594005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:16.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.319+0000 7f25ae393700 1 -- 172.21.15.16:0/2211386460 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f25a80af2f0 msgr2=0x7f2594004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:16.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.319+0000 7f25ae393700 1 --2- 172.21.15.16:0/2211386460 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f25a80af2f0 0x7f2594004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:16.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.319+0000 7f25ae393700 1 -- 172.21.15.16:0/2211386460 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2594004d90 msgr2=0x7f2594005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:16.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.319+0000 7f25ae393700 1 --2- 172.21.15.16:0/2211386460 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2594004d90 0x7f2594005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:16.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.319+0000 7f25ae393700 1 -- 172.21.15.16:0/2211386460 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25a40094a0 con 0x7f2594003a00 2023-05-20T01:26:16.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.320+0000 7f25ae393700 1 --2- 172.21.15.16:0/2211386460 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2594003a00 0x7f2594003e10 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f259800c320 tx=0x7f2594005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48f93dfa22c429b7 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:16.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.320+0000 7f25ad391700 1 -- 172.21.15.16:0/2211386460 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f259800ce00 con 0x7f2594003a00 2023-05-20T01:26:16.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.320+0000 7f25ad391700 1 -- 172.21.15.16:0/2211386460 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2598005960 con 0x7f2594003a00 2023-05-20T01:26:16.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.320+0000 7f25ad391700 1 -- 172.21.15.16:0/2211386460 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2598005d10 con 0x7f2594003a00 2023-05-20T01:26:16.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.320+0000 7f25b05f7700 1 -- 172.21.15.16:0/2211386460 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2594003a00 msgr2=0x7f2594003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:16.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.320+0000 7f25b05f7700 1 --2- 172.21.15.16:0/2211386460 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2594003a00 0x7f2594003e10 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f259800c320 tx=0x7f2594005880 comp rx=0 tx=0).stop 2023-05-20T01:26:16.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.320+0000 7f25aeb94700 1 -- 172.21.15.16:0/2211386460 reap_dead start 2023-05-20T01:26:16.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.320+0000 7f25b05f7700 1 -- 172.21.15.16:0/2211386460 shutdown_connections 2023-05-20T01:26:16.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.320+0000 7f25b05f7700 1 -- 172.21.15.16:0/2211386460 >> 172.21.15.16:0/2211386460 conn(0x7f25a806ae30 msgr2=0x7f25a809a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:16.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.320+0000 7f25b05f7700 1 -- 172.21.15.16:0/2211386460 shutdown_connections 2023-05-20T01:26:16.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.320+0000 7f25b05f7700 1 -- 172.21.15.16:0/2211386460 wait complete. 2023-05-20T01:26:16.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.322+0000 7f25b05f7700 1 Processor -- start 2023-05-20T01:26:16.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.322+0000 7f25b05f7700 1 -- start start 2023-05-20T01:26:16.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.322+0000 7f25b05f7700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2594003a00 0x7f25a8159ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:16.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.322+0000 7f25b05f7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25a81f5940 0x7f25a81f5db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:16.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.322+0000 7f25b05f7700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f25a809d710 0x7f25a809db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:16.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.322+0000 7f25b05f7700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25a806bf50 con 0x7f25a81f5940 2023-05-20T01:26:16.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.322+0000 7f25b05f7700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25a81f6390 con 0x7f2594003a00 2023-05-20T01:26:16.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.322+0000 7f25b05f7700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25a81f6670 con 0x7f25a809d710 2023-05-20T01:26:16.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.322+0000 7f25adb92700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25a81f5940 0x7f25a81f5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:16.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.322+0000 7f25adb92700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25a81f5940 0x7f25a81f5db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37470/0 (socket says 172.21.15.16:37470) 2023-05-20T01:26:16.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.322+0000 7f25adb92700 1 -- 172.21.15.16:0/2827799397 learned_addr learned my addr 172.21.15.16:0/2827799397 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:26:16.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.323+0000 7f25aeb94700 1 --2- 172.21.15.16:0/2827799397 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f25a809d710 0x7f25a809db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:16.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.323+0000 7f25ae393700 1 --2- 172.21.15.16:0/2827799397 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2594003a00 0x7f25a8159ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:16.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.323+0000 7f25adb92700 1 -- 172.21.15.16:0/2827799397 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f25a809d710 msgr2=0x7f25a809db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:16.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.323+0000 7f25adb92700 1 --2- 172.21.15.16:0/2827799397 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f25a809d710 0x7f25a809db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:16.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.323+0000 7f25adb92700 1 -- 172.21.15.16:0/2827799397 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2594003a00 msgr2=0x7f25a8159ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:16.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.323+0000 7f25adb92700 1 --2- 172.21.15.16:0/2827799397 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2594003a00 0x7f25a8159ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:16.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.323+0000 7f25adb92700 1 -- 172.21.15.16:0/2827799397 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25a80b28c0 con 0x7f25a81f5940 2023-05-20T01:26:16.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.323+0000 7f25adb92700 1 --2- 172.21.15.16:0/2827799397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25a81f5940 0x7f25a81f5db0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f25a4011ca0 tx=0x7f25a4009e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:16.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.323+0000 7f259effd700 1 -- 172.21.15.16:0/2827799397 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a400fa00 con 0x7f25a81f5940 2023-05-20T01:26:16.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.323+0000 7f259effd700 1 -- 172.21.15.16:0/2827799397 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25a400fb60 con 0x7f25a81f5940 2023-05-20T01:26:16.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.323+0000 7f25b05f7700 1 -- 172.21.15.16:0/2827799397 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25a80addc0 con 0x7f25a81f5940 2023-05-20T01:26:16.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.323+0000 7f259effd700 1 -- 172.21.15.16:0/2827799397 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a4004020 con 0x7f25a81f5940 2023-05-20T01:26:16.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.323+0000 7f25b05f7700 1 -- 172.21.15.16:0/2827799397 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25a80cec40 con 0x7f25a81f5940 2023-05-20T01:26:16.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.324+0000 7f25b05f7700 1 -- 172.21.15.16:0/2827799397 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25a80747b0 con 0x7f25a81f5940 2023-05-20T01:26:16.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.325+0000 7f259effd700 1 -- 172.21.15.16:0/2827799397 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f25a4004180 con 0x7f25a81f5940 2023-05-20T01:26:16.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.325+0000 7f259effd700 1 --2- 172.21.15.16:0/2827799397 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f258c06ace0 0x7f258c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:16.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.325+0000 7f259effd700 1 -- 172.21.15.16:0/2827799397 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 7044+0+0 (secure 0 0 0) 0x7f25a4089c50 con 0x7f25a81f5940 2023-05-20T01:26:16.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.325+0000 7f25ae393700 1 --2- 172.21.15.16:0/2827799397 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f258c06ace0 0x7f258c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:16.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.325+0000 7f25ae393700 1 --2- 172.21.15.16:0/2827799397 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f258c06ace0 0x7f258c06d190 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f2598008fd0 tx=0x7f259800a000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:16.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.329+0000 7f259effd700 1 -- 172.21.15.16:0/2827799397 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f25a408f020 con 0x7f25a81f5940 2023-05-20T01:26:16.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.529+0000 7f25b05f7700 1 -- 172.21.15.16:0/2827799397 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f25a815ae20 con 0x7f25a81f5940 2023-05-20T01:26:16.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.530+0000 7f259effd700 1 -- 172.21.15.16:0/2827799397 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v98) v1 ==== 104+0+11 (secure 0 0 0) 0x7f25a4022020 con 0x7f25a81f5940 2023-05-20T01:26:16.534 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-05-20T01:26:16.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.532+0000 7f25b05f7700 1 -- 172.21.15.16:0/2827799397 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f258c06ace0 msgr2=0x7f258c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:16.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.532+0000 7f25b05f7700 1 --2- 172.21.15.16:0/2827799397 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f258c06ace0 0x7f258c06d190 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f2598008fd0 tx=0x7f259800a000 comp rx=0 tx=0).stop 2023-05-20T01:26:16.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.532+0000 7f25b05f7700 1 -- 172.21.15.16:0/2827799397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25a81f5940 msgr2=0x7f25a81f5db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:16.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.532+0000 7f25b05f7700 1 --2- 172.21.15.16:0/2827799397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25a81f5940 0x7f25a81f5db0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f25a4011ca0 tx=0x7f25a4009e80 comp rx=0 tx=0).stop 2023-05-20T01:26:16.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.532+0000 7f25b05f7700 1 -- 172.21.15.16:0/2827799397 shutdown_connections 2023-05-20T01:26:16.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.532+0000 7f25b05f7700 1 --2- 172.21.15.16:0/2827799397 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f258c06ace0 0x7f258c06d190 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:16.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.532+0000 7f25b05f7700 1 --2- 172.21.15.16:0/2827799397 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f25a809d710 0x7f25a809db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:16.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.532+0000 7f25b05f7700 1 --2- 172.21.15.16:0/2827799397 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2594003a00 0x7f25a8159ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:16.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.532+0000 7f25b05f7700 1 --2- 172.21.15.16:0/2827799397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25a81f5940 0x7f25a81f5db0 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:16.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.532+0000 7f25b05f7700 1 -- 172.21.15.16:0/2827799397 >> 172.21.15.16:0/2827799397 conn(0x7f25a806ae30 msgr2=0x7f25a80b2a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:16.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.532+0000 7f25b05f7700 1 -- 172.21.15.16:0/2827799397 shutdown_connections 2023-05-20T01:26:16.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.532+0000 7f25b05f7700 1 -- 172.21.15.16:0/2827799397 wait complete. 2023-05-20T01:26:16.548 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-05-20T01:26:16.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.672+0000 7fe59b6c5700 1 Processor -- start 2023-05-20T01:26:16.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.672+0000 7fe59b6c5700 1 -- start start 2023-05-20T01:26:16.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.673+0000 7fe59b6c5700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe59415cec0 0x7fe594071bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:16.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.673+0000 7fe59b6c5700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fe59415d290 0x7fe594072130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:16.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.673+0000 7fe59b6c5700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5940cfe30 con 0x7fe5940cf2c0 2023-05-20T01:26:16.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.673+0000 7fe59b6c5700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5940cd630 con 0x7fe59415d290 2023-05-20T01:26:16.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.673+0000 7fe59b6c5700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5940cca60 con 0x7fe59415cec0 2023-05-20T01:26:16.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.673+0000 7fe599c62700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fe5940cf2c0 0x7fe5940cf720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49008/0 (socket says 172.21.15.16:49008) 2023-05-20T01:26:16.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.673+0000 7fe599c62700 1 -- 172.21.15.16:0/3169812633 learned_addr learned my addr 172.21.15.16:0/3169812633 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:26:16.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.673+0000 7fe58bfff700 1 -- 172.21.15.16:0/3169812633 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1600968198 0 0) 0x7fe58c001490 con 0x7fe5940cf2c0 2023-05-20T01:26:16.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.673+0000 7fe598c60700 1 --2- 172.21.15.16:0/3169812633 >> v2:172.21.15.132:3300/0 conn(0x7fe59415d290 0x7fe594072130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:16.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.673+0000 7fe599461700 1 --2- 172.21.15.16:0/3169812633 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe59415cec0 0x7fe594071bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:16.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.674+0000 7fe58bfff700 1 -- 172.21.15.16:0/3169812633 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5800035b0 con 0x7fe5940cf2c0 2023-05-20T01:26:16.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.674+0000 7fe58bfff700 1 -- 172.21.15.16:0/3169812633 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2481271228 0 0) 0x7fe5800035b0 con 0x7fe5940cf2c0 2023-05-20T01:26:16.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.674+0000 7fe58bfff700 1 -- 172.21.15.16:0/3169812633 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe59415cec0 msgr2=0x7fe594071bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:16.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.674+0000 7fe58bfff700 1 --2- 172.21.15.16:0/3169812633 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe59415cec0 0x7fe594071bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:16.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.674+0000 7fe58bfff700 1 -- 172.21.15.16:0/3169812633 >> v2:172.21.15.132:3300/0 conn(0x7fe59415d290 msgr2=0x7fe594072130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:16.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.674+0000 7fe58bfff700 1 --2- 172.21.15.16:0/3169812633 >> v2:172.21.15.132:3300/0 conn(0x7fe59415d290 0x7fe594072130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:16.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.674+0000 7fe58bfff700 1 -- 172.21.15.16:0/3169812633 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5940ccd00 con 0x7fe5940cf2c0 2023-05-20T01:26:16.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.674+0000 7fe58bfff700 1 -- 172.21.15.16:0/3169812633 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7fe58c001b80 con 0x7fe5940cf2c0 2023-05-20T01:26:16.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.674+0000 7fe58bfff700 1 -- 172.21.15.16:0/3169812633 >> v1:172.21.15.16:6789/0 conn(0x7fe5940cf2c0 legacy=0x7fe5940cf720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:16.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.674+0000 7fe58bfff700 1 --2- 172.21.15.16:0/3169812633 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe59415cec0 0x7fe580003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:16.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.674+0000 7fe58bfff700 1 --2- 172.21.15.16:0/3169812633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe580003f80 0x7fe580004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:16.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.674+0000 7fe58bfff700 1 --2- 172.21.15.16:0/3169812633 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe580004d90 0x7fe580005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:16.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.674+0000 7fe58bfff700 1 -- 172.21.15.16:0/3169812633 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5940cd630 con 0x7fe580003f80 2023-05-20T01:26:16.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.674+0000 7fe58bfff700 1 -- 172.21.15.16:0/3169812633 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5940cca60 con 0x7fe580004d90 2023-05-20T01:26:16.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.674+0000 7fe58bfff700 1 -- 172.21.15.16:0/3169812633 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe58000ba40 con 0x7fe59415cec0 2023-05-20T01:26:16.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.674+0000 7fe598c60700 1 --2- 172.21.15.16:0/3169812633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe580003f80 0x7fe580004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:16.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.674+0000 7fe599461700 1 --2- 172.21.15.16:0/3169812633 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe59415cec0 0x7fe580003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:16.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.674+0000 7fe599c62700 1 --2- 172.21.15.16:0/3169812633 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe580004d90 0x7fe580005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:16.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.674+0000 7fe598c60700 1 -- 172.21.15.16:0/3169812633 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe59415cec0 msgr2=0x7fe580003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:16.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.674+0000 7fe598c60700 1 --2- 172.21.15.16:0/3169812633 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe59415cec0 0x7fe580003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:16.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.674+0000 7fe598c60700 1 -- 172.21.15.16:0/3169812633 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe580004d90 msgr2=0x7fe580005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:16.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.674+0000 7fe598c60700 1 --2- 172.21.15.16:0/3169812633 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe580004d90 0x7fe580005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:16.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.674+0000 7fe598c60700 1 -- 172.21.15.16:0/3169812633 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5800035b0 con 0x7fe580003f80 2023-05-20T01:26:16.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.674+0000 7fe598c60700 1 --2- 172.21.15.16:0/3169812633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe580003f80 0x7fe580004850 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7fe590009b60 tx=0x7fe580005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=851ec54eb9de8848 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:16.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.675+0000 7fe58bfff700 1 -- 172.21.15.16:0/3169812633 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe59000e660 con 0x7fe580003f80 2023-05-20T01:26:16.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.675+0000 7fe58bfff700 1 -- 172.21.15.16:0/3169812633 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe59000e7c0 con 0x7fe580003f80 2023-05-20T01:26:16.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.675+0000 7fe58bfff700 1 -- 172.21.15.16:0/3169812633 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe59000eb30 con 0x7fe580003f80 2023-05-20T01:26:16.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.675+0000 7fe59b6c5700 1 -- 172.21.15.16:0/3169812633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe580003f80 msgr2=0x7fe580004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:16.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.675+0000 7fe59b6c5700 1 --2- 172.21.15.16:0/3169812633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe580003f80 0x7fe580004850 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7fe590009b60 tx=0x7fe580005880 comp rx=0 tx=0).stop 2023-05-20T01:26:16.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.675+0000 7fe599c62700 1 -- 172.21.15.16:0/3169812633 reap_dead start 2023-05-20T01:26:16.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.675+0000 7fe59b6c5700 1 -- 172.21.15.16:0/3169812633 shutdown_connections 2023-05-20T01:26:16.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.675+0000 7fe59b6c5700 1 -- 172.21.15.16:0/3169812633 >> 172.21.15.16:0/3169812633 conn(0x7fe5940cffd0 msgr2=0x7fe5940d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:16.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.675+0000 7fe59b6c5700 1 -- 172.21.15.16:0/3169812633 shutdown_connections 2023-05-20T01:26:16.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.675+0000 7fe59b6c5700 1 -- 172.21.15.16:0/3169812633 wait complete. 2023-05-20T01:26:16.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.677+0000 7fe59b6c5700 1 Processor -- start 2023-05-20T01:26:16.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.677+0000 7fe59b6c5700 1 -- start start 2023-05-20T01:26:16.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.677+0000 7fe59b6c5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe580003f80 0x7fe5940c6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:16.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.677+0000 7fe59b6c5700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe5941f5b20 0x7fe5940ac8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:16.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.677+0000 7fe59b6c5700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe5941550f0 0x7fe59415a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:16.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.677+0000 7fe59b6c5700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe594152d40 con 0x7fe580003f80 2023-05-20T01:26:16.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.677+0000 7fe59b6c5700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5941f6000 con 0x7fe5941f5b20 2023-05-20T01:26:16.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.677+0000 7fe59b6c5700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5941f62e0 con 0x7fe5941550f0 2023-05-20T01:26:16.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.677+0000 7fe599461700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe580003f80 0x7fe5940c6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:16.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.677+0000 7fe599461700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe580003f80 0x7fe5940c6ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37488/0 (socket says 172.21.15.16:37488) 2023-05-20T01:26:16.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.677+0000 7fe599461700 1 -- 172.21.15.16:0/2743146442 learned_addr learned my addr 172.21.15.16:0/2743146442 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:26:16.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.678+0000 7fe599c62700 1 --2- 172.21.15.16:0/2743146442 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe5941550f0 0x7fe59415a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:16.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.678+0000 7fe598c60700 1 --2- 172.21.15.16:0/2743146442 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe5941f5b20 0x7fe5940ac8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:16.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.678+0000 7fe599461700 1 -- 172.21.15.16:0/2743146442 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe5941550f0 msgr2=0x7fe59415a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:16.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.678+0000 7fe599461700 1 --2- 172.21.15.16:0/2743146442 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe5941550f0 0x7fe59415a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:16.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.678+0000 7fe599461700 1 -- 172.21.15.16:0/2743146442 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe5941f5b20 msgr2=0x7fe5940ac8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:16.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.678+0000 7fe599461700 1 --2- 172.21.15.16:0/2743146442 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe5941f5b20 0x7fe5940ac8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:16.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.678+0000 7fe599461700 1 -- 172.21.15.16:0/2743146442 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5940cd2c0 con 0x7fe580003f80 2023-05-20T01:26:16.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.678+0000 7fe599461700 1 --2- 172.21.15.16:0/2743146442 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe580003f80 0x7fe5940c6ab0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7fe5840119f0 tx=0x7fe584011d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:16.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.678+0000 7fe589ffb700 1 -- 172.21.15.16:0/2743146442 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe584011e90 con 0x7fe580003f80 2023-05-20T01:26:16.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.678+0000 7fe589ffb700 1 -- 172.21.15.16:0/2743146442 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe584005c30 con 0x7fe580003f80 2023-05-20T01:26:16.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.678+0000 7fe59b6c5700 1 -- 172.21.15.16:0/2743146442 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5940ccfe0 con 0x7fe580003f80 2023-05-20T01:26:16.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.678+0000 7fe589ffb700 1 -- 172.21.15.16:0/2743146442 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe58400f730 con 0x7fe580003f80 2023-05-20T01:26:16.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.678+0000 7fe59b6c5700 1 -- 172.21.15.16:0/2743146442 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe594071bf0 con 0x7fe580003f80 2023-05-20T01:26:16.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.679+0000 7fe59b6c5700 1 -- 172.21.15.16:0/2743146442 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5940747b0 con 0x7fe580003f80 2023-05-20T01:26:16.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.680+0000 7fe589ffb700 1 -- 172.21.15.16:0/2743146442 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe58400f890 con 0x7fe580003f80 2023-05-20T01:26:16.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.680+0000 7fe589ffb700 1 --2- 172.21.15.16:0/2743146442 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe57806ace0 0x7fe57806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:16.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.680+0000 7fe589ffb700 1 -- 172.21.15.16:0/2743146442 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 7102+0+0 (secure 0 0 0) 0x7fe584089d70 con 0x7fe580003f80 2023-05-20T01:26:16.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.680+0000 7fe598c60700 1 --2- 172.21.15.16:0/2743146442 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe57806ace0 0x7fe57806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:16.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.681+0000 7fe598c60700 1 --2- 172.21.15.16:0/2743146442 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe57806ace0 0x7fe57806d190 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fe5900090b0 tx=0x7fe59001f040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:16.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.685+0000 7fe589ffb700 1 -- 172.21.15.16:0/2743146442 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fe584054b60 con 0x7fe580003f80 2023-05-20T01:26:16.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.882+0000 7fe59b6c5700 1 -- 172.21.15.16:0/2743146442 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe5940cdba0 con 0x7fe580003f80 2023-05-20T01:26:16.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.882+0000 7fe589ffb700 1 -- 172.21.15.16:0/2743146442 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v98) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe5840210a0 con 0x7fe580003f80 2023-05-20T01:26:16.887 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2023-05-20T01:26:16.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.884+0000 7fe59b6c5700 1 -- 172.21.15.16:0/2743146442 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe57806ace0 msgr2=0x7fe57806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:16.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.884+0000 7fe59b6c5700 1 --2- 172.21.15.16:0/2743146442 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe57806ace0 0x7fe57806d190 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fe5900090b0 tx=0x7fe59001f040 comp rx=0 tx=0).stop 2023-05-20T01:26:16.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.884+0000 7fe59b6c5700 1 -- 172.21.15.16:0/2743146442 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe580003f80 msgr2=0x7fe5940c6ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:16.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.884+0000 7fe59b6c5700 1 --2- 172.21.15.16:0/2743146442 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe580003f80 0x7fe5940c6ab0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7fe5840119f0 tx=0x7fe584011d00 comp rx=0 tx=0).stop 2023-05-20T01:26:16.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.885+0000 7fe59b6c5700 1 -- 172.21.15.16:0/2743146442 shutdown_connections 2023-05-20T01:26:16.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.885+0000 7fe59b6c5700 1 --2- 172.21.15.16:0/2743146442 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe57806ace0 0x7fe57806d190 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:16.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.885+0000 7fe59b6c5700 1 --2- 172.21.15.16:0/2743146442 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe5941550f0 0x7fe59415a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:16.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.885+0000 7fe59b6c5700 1 --2- 172.21.15.16:0/2743146442 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe580003f80 0x7fe5940c6ab0 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:16.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.885+0000 7fe59b6c5700 1 --2- 172.21.15.16:0/2743146442 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe5941f5b20 0x7fe5940ac8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:16.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.885+0000 7fe59b6c5700 1 -- 172.21.15.16:0/2743146442 >> 172.21.15.16:0/2743146442 conn(0x7fe5940cffd0 msgr2=0x7fe59415b570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:16.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.885+0000 7fe59b6c5700 1 -- 172.21.15.16:0/2743146442 shutdown_connections 2023-05-20T01:26:16.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:16.885+0000 7fe59b6c5700 1 -- 172.21.15.16:0/2743146442 wait complete. 2023-05-20T01:26:16.901 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_extra pg_num 2023-05-20T01:26:17.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.023+0000 7fa316464700 1 Processor -- start 2023-05-20T01:26:17.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.024+0000 7fa316464700 1 -- start start 2023-05-20T01:26:17.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.024+0000 7fa316464700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa3100cd0f0 0x7fa3100ce620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:17.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.024+0000 7fa316464700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3100ca1b0 con 0x7fa3101599b0 2023-05-20T01:26:17.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.024+0000 7fa316464700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3100cd580 con 0x7fa3100b0910 2023-05-20T01:26:17.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.024+0000 7fa316464700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3100cd860 con 0x7fa3100cd0f0 2023-05-20T01:26:17.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.025+0000 7fa30ffff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fa3101599b0 0x7fa31015ddc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49024/0 (socket says 172.21.15.16:49024) 2023-05-20T01:26:17.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.025+0000 7fa30ffff700 1 -- 172.21.15.16:0/1851472869 learned_addr learned my addr 172.21.15.16:0/1851472869 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:26:17.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.025+0000 7fa30f7fe700 1 --2- 172.21.15.16:0/1851472869 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa3100cd0f0 0x7fa3100ce620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:17.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.025+0000 7fa30effd700 1 -- 172.21.15.16:0/1851472869 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1312254190 0 0) 0x7fa3100ca1b0 con 0x7fa3101599b0 2023-05-20T01:26:17.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.025+0000 7fa30effd700 1 -- 172.21.15.16:0/1851472869 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2fc0035b0 con 0x7fa3101599b0 2023-05-20T01:26:17.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.025+0000 7fa30effd700 1 -- 172.21.15.16:0/1851472869 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1507681879 0 0) 0x7fa3100cd580 con 0x7fa3100b0910 2023-05-20T01:26:17.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.025+0000 7fa30f7fe700 1 -- 172.21.15.16:0/1851472869 >> v1:172.21.15.132:6789/0 conn(0x7fa3100b0910 legacy=0x7fa310070310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:17.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.025+0000 7fa30f7fe700 1 -- 172.21.15.16:0/1851472869 >> v1:172.21.15.16:6789/0 conn(0x7fa3101599b0 legacy=0x7fa31015ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:17.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.025+0000 7fa30f7fe700 1 -- 172.21.15.16:0/1851472869 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3100cdb40 con 0x7fa3100cd0f0 2023-05-20T01:26:17.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.025+0000 7fa30f7fe700 1 --2- 172.21.15.16:0/1851472869 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa3100cd0f0 0x7fa3100ce620 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa300009870 tx=0x7fa300009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=150f11a64c63a4e2 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:17.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.025+0000 7fa30effd700 1 -- 172.21.15.16:0/1851472869 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa300011830 con 0x7fa3100cd0f0 2023-05-20T01:26:17.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.026+0000 7fa30effd700 1 -- 172.21.15.16:0/1851472869 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa300011990 con 0x7fa3100cd0f0 2023-05-20T01:26:17.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.026+0000 7fa30effd700 1 -- 172.21.15.16:0/1851472869 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa300011d00 con 0x7fa3100cd0f0 2023-05-20T01:26:17.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.026+0000 7fa316464700 1 -- 172.21.15.16:0/1851472869 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa3100cd0f0 msgr2=0x7fa3100ce620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:17.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.026+0000 7fa316464700 1 --2- 172.21.15.16:0/1851472869 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa3100cd0f0 0x7fa3100ce620 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa300009870 tx=0x7fa300009b80 comp rx=0 tx=0).stop 2023-05-20T01:26:17.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.026+0000 7fa316464700 1 -- 172.21.15.16:0/1851472869 shutdown_connections 2023-05-20T01:26:17.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.026+0000 7fa316464700 1 --2- 172.21.15.16:0/1851472869 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa3100cd0f0 0x7fa3100ce620 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:17.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.026+0000 7fa316464700 1 -- 172.21.15.16:0/1851472869 >> 172.21.15.16:0/1851472869 conn(0x7fa3100cb910 msgr2=0x7fa3100cbd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:17.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.026+0000 7fa316464700 1 -- 172.21.15.16:0/1851472869 shutdown_connections 2023-05-20T01:26:17.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.026+0000 7fa316464700 1 -- 172.21.15.16:0/1851472869 wait complete. 2023-05-20T01:26:17.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.027+0000 7fa316464700 1 Processor -- start 2023-05-20T01:26:17.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.028+0000 7fa316464700 1 -- start start 2023-05-20T01:26:17.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.028+0000 7fa316464700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3100b0910 0x7fa31006cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:17.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.028+0000 7fa316464700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa3100cd0f0 0x7fa31014f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:17.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.028+0000 7fa316464700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa3101599b0 0x7fa31014f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:17.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.028+0000 7fa316464700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3100b2010 con 0x7fa3100b0910 2023-05-20T01:26:17.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.028+0000 7fa316464700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa310151ee0 con 0x7fa3100cd0f0 2023-05-20T01:26:17.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.028+0000 7fa316464700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3101521c0 con 0x7fa3101599b0 2023-05-20T01:26:17.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.028+0000 7fa30ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3100b0910 0x7fa31006cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:17.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.028+0000 7fa30ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3100b0910 0x7fa31006cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37502/0 (socket says 172.21.15.16:37502) 2023-05-20T01:26:17.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.028+0000 7fa30ffff700 1 -- 172.21.15.16:0/820398277 learned_addr learned my addr 172.21.15.16:0/820398277 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:26:17.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.029+0000 7fa30f7fe700 1 --2- 172.21.15.16:0/820398277 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa3100cd0f0 0x7fa31014f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:17.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.029+0000 7fa314a01700 1 --2- 172.21.15.16:0/820398277 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa3101599b0 0x7fa31014f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:17.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.029+0000 7fa30ffff700 1 -- 172.21.15.16:0/820398277 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa3101599b0 msgr2=0x7fa31014f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:17.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.029+0000 7fa30ffff700 1 --2- 172.21.15.16:0/820398277 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa3101599b0 0x7fa31014f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:17.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.029+0000 7fa30ffff700 1 -- 172.21.15.16:0/820398277 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa3100cd0f0 msgr2=0x7fa31014f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:17.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.029+0000 7fa30ffff700 1 --2- 172.21.15.16:0/820398277 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa3100cd0f0 0x7fa31014f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:17.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.029+0000 7fa30ffff700 1 -- 172.21.15.16:0/820398277 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3000094a0 con 0x7fa3100b0910 2023-05-20T01:26:17.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.029+0000 7fa30ffff700 1 --2- 172.21.15.16:0/820398277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3100b0910 0x7fa31006cd90 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fa2f800c420 tx=0x7fa2f800c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:17.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.029+0000 7fa30cff9700 1 -- 172.21.15.16:0/820398277 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2f8005d70 con 0x7fa3100b0910 2023-05-20T01:26:17.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.029+0000 7fa30cff9700 1 -- 172.21.15.16:0/820398277 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2f80070d0 con 0x7fa3100b0910 2023-05-20T01:26:17.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.029+0000 7fa316464700 1 -- 172.21.15.16:0/820398277 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3100ce100 con 0x7fa3100b0910 2023-05-20T01:26:17.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.029+0000 7fa30cff9700 1 -- 172.21.15.16:0/820398277 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2f8007440 con 0x7fa3100b0910 2023-05-20T01:26:17.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.029+0000 7fa316464700 1 -- 172.21.15.16:0/820398277 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3100cde20 con 0x7fa3100b0910 2023-05-20T01:26:17.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.030+0000 7fa316464700 1 -- 172.21.15.16:0/820398277 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3100ceb60 con 0x7fa3100b0910 2023-05-20T01:26:17.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.031+0000 7fa30cff9700 1 -- 172.21.15.16:0/820398277 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa2f80075a0 con 0x7fa3100b0910 2023-05-20T01:26:17.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.031+0000 7fa30cff9700 1 --2- 172.21.15.16:0/820398277 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa2f006ace0 0x7fa2f006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:17.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.031+0000 7fa30cff9700 1 -- 172.21.15.16:0/820398277 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 7102+0+0 (secure 0 0 0) 0x7fa2f8088b80 con 0x7fa3100b0910 2023-05-20T01:26:17.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.031+0000 7fa30f7fe700 1 --2- 172.21.15.16:0/820398277 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa2f006ace0 0x7fa2f006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:17.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.031+0000 7fa30f7fe700 1 --2- 172.21.15.16:0/820398277 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa2f006ace0 0x7fa2f006d190 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fa3000090b0 tx=0x7fa300008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:17.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.035+0000 7fa30cff9700 1 -- 172.21.15.16:0/820398277 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fa2f8053960 con 0x7fa3100b0910 2023-05-20T01:26:17.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.231+0000 7fa316464700 1 -- 172.21.15.16:0/820398277 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"} v 0) v1 -- 0x7fa3100b2150 con 0x7fa3100b0910 2023-05-20T01:26:17.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.232+0000 7fa30cff9700 1 -- 172.21.15.16:0/820398277 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"}]=0 v98) v1 ==== 106+0+11 (secure 0 0 0) 0x7fa2f8017020 con 0x7fa3100b0910 2023-05-20T01:26:17.241 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-05-20T01:26:17.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.234+0000 7fa316464700 1 -- 172.21.15.16:0/820398277 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa2f006ace0 msgr2=0x7fa2f006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:17.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.234+0000 7fa316464700 1 --2- 172.21.15.16:0/820398277 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa2f006ace0 0x7fa2f006d190 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fa3000090b0 tx=0x7fa300008030 comp rx=0 tx=0).stop 2023-05-20T01:26:17.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.234+0000 7fa316464700 1 -- 172.21.15.16:0/820398277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3100b0910 msgr2=0x7fa31006cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:17.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.234+0000 7fa316464700 1 --2- 172.21.15.16:0/820398277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3100b0910 0x7fa31006cd90 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fa2f800c420 tx=0x7fa2f800c7e0 comp rx=0 tx=0).stop 2023-05-20T01:26:17.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.234+0000 7fa316464700 1 -- 172.21.15.16:0/820398277 shutdown_connections 2023-05-20T01:26:17.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.234+0000 7fa316464700 1 --2- 172.21.15.16:0/820398277 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa2f006ace0 0x7fa2f006d190 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:17.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.234+0000 7fa316464700 1 --2- 172.21.15.16:0/820398277 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa3101599b0 0x7fa31014f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:17.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.234+0000 7fa316464700 1 --2- 172.21.15.16:0/820398277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3100b0910 0x7fa31006cd90 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:17.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.234+0000 7fa316464700 1 --2- 172.21.15.16:0/820398277 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa3100cd0f0 0x7fa31014f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:17.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.234+0000 7fa316464700 1 -- 172.21.15.16:0/820398277 >> 172.21.15.16:0/820398277 conn(0x7fa3100cb910 msgr2=0x7fa31006bae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:17.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.234+0000 7fa316464700 1 -- 172.21.15.16:0/820398277 shutdown_connections 2023-05-20T01:26:17.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.234+0000 7fa316464700 1 -- 172.21.15.16:0/820398277 wait complete. 2023-05-20T01:26:17.250 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-05-20T01:26:17.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.372+0000 7f18c3ea3700 1 Processor -- start 2023-05-20T01:26:17.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.372+0000 7f18c3ea3700 1 -- start start 2023-05-20T01:26:17.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.373+0000 7f18c3ea3700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f18bc0b2760 0x7f18bc0d1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:17.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.373+0000 7f18c3ea3700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18bc0cc500 con 0x7f18bc144020 2023-05-20T01:26:17.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.373+0000 7f18c3ea3700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18bc0cb8f0 con 0x7f18bc0cec60 2023-05-20T01:26:17.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.373+0000 7f18c3ea3700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18bc0cb6e0 con 0x7f18bc0b2760 2023-05-20T01:26:17.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.373+0000 7f18c143e700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f18bc144020 0x7f18bc0b0980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49040/0 (socket says 172.21.15.16:49040) 2023-05-20T01:26:17.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.373+0000 7f18c143e700 1 -- 172.21.15.16:0/3125705484 learned_addr learned my addr 172.21.15.16:0/3125705484 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:26:17.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.373+0000 7f18c1c3f700 1 --2- 172.21.15.16:0/3125705484 >> v2:172.21.15.132:3300/0 conn(0x7f18bc0b2760 0x7f18bc0d1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:17.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.373+0000 7f18c0c3d700 1 -- 172.21.15.16:0/3125705484 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3402178018 0 0) 0x7f18bc0cc500 con 0x7f18bc144020 2023-05-20T01:26:17.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.373+0000 7f18c0c3d700 1 -- 172.21.15.16:0/3125705484 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f18a80035b0 con 0x7f18bc144020 2023-05-20T01:26:17.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.374+0000 7f18c0c3d700 1 -- 172.21.15.16:0/3125705484 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2594761491 0 0) 0x7f18a80035b0 con 0x7f18bc144020 2023-05-20T01:26:17.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.374+0000 7f18c0c3d700 1 -- 172.21.15.16:0/3125705484 >> v2:172.21.15.132:3300/0 conn(0x7f18bc0b2760 msgr2=0x7f18bc0d1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:17.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.374+0000 7f18c0c3d700 1 --2- 172.21.15.16:0/3125705484 >> v2:172.21.15.132:3300/0 conn(0x7f18bc0b2760 0x7f18bc0d1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:17.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.374+0000 7f18c0c3d700 1 -- 172.21.15.16:0/3125705484 >> v1:172.21.15.132:6789/0 conn(0x7f18bc0cec60 legacy=0x7f18bc0cc990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:17.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.374+0000 7f18c0c3d700 1 -- 172.21.15.16:0/3125705484 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18bc0cbbd0 con 0x7f18bc144020 2023-05-20T01:26:17.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.374+0000 7f18c0c3d700 1 -- 172.21.15.16:0/3125705484 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7f18b8001670 con 0x7f18bc144020 2023-05-20T01:26:17.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.374+0000 7f18c1c3f700 1 --2- 172.21.15.16:0/3125705484 >> v2:172.21.15.132:3300/0 conn(0x7f18bc0b2760 0x7f18bc0d1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-05-20T01:26:17.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.374+0000 7f18c0c3d700 1 -- 172.21.15.16:0/3125705484 >> v1:172.21.15.16:6789/0 conn(0x7f18bc144020 legacy=0x7f18bc0b0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:17.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.374+0000 7f18c0c3d700 1 --2- 172.21.15.16:0/3125705484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18a8003a70 0x7f18a8003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:17.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.374+0000 7f18c0c3d700 1 --2- 172.21.15.16:0/3125705484 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f18a8004850 0x7f18a8004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:17.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.374+0000 7f18c0c3d700 1 --2- 172.21.15.16:0/3125705484 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f18a8005200 0x7f18a800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:17.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.374+0000 7f18c0c3d700 1 -- 172.21.15.16:0/3125705484 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18bc0cb6e0 con 0x7f18a8003a70 2023-05-20T01:26:17.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.374+0000 7f18c0c3d700 1 -- 172.21.15.16:0/3125705484 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18a800c0e0 con 0x7f18a8005200 2023-05-20T01:26:17.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.374+0000 7f18c0c3d700 1 -- 172.21.15.16:0/3125705484 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18a800c2a0 con 0x7f18a8004850 2023-05-20T01:26:17.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.374+0000 7f18c1c3f700 1 --2- 172.21.15.16:0/3125705484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18a8003a70 0x7f18a8003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:17.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.374+0000 7f18c143e700 1 --2- 172.21.15.16:0/3125705484 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f18a8004850 0x7f18a8004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:17.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.374+0000 7f18c2440700 1 --2- 172.21.15.16:0/3125705484 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f18a8005200 0x7f18a800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:17.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.374+0000 7f18c1c3f700 1 -- 172.21.15.16:0/3125705484 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f18a8004850 msgr2=0x7f18a8004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:17.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.374+0000 7f18c1c3f700 1 --2- 172.21.15.16:0/3125705484 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f18a8004850 0x7f18a8004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:17.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.374+0000 7f18c1c3f700 1 -- 172.21.15.16:0/3125705484 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f18a8005200 msgr2=0x7f18a800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:17.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.374+0000 7f18c1c3f700 1 --2- 172.21.15.16:0/3125705484 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f18a8005200 0x7f18a800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:17.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.374+0000 7f18c1c3f700 1 -- 172.21.15.16:0/3125705484 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18a80035b0 con 0x7f18a8003a70 2023-05-20T01:26:17.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.374+0000 7f18c2440700 1 --2- 172.21.15.16:0/3125705484 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f18a8005200 0x7f18a800ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:26:17.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.374+0000 7f18c2440700 1 -- 172.21.15.16:0/3125705484 reap_dead start 2023-05-20T01:26:17.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.374+0000 7f18c1c3f700 1 --2- 172.21.15.16:0/3125705484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18a8003a70 0x7f18a8003e80 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f18ac009d30 tx=0x7f18a8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c36f111053f2992 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:17.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.374+0000 7f18c0c3d700 1 -- 172.21.15.16:0/3125705484 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18ac009e80 con 0x7f18a8003a70 2023-05-20T01:26:17.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.374+0000 7f18c0c3d700 1 -- 172.21.15.16:0/3125705484 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18ac00e9a0 con 0x7f18a8003a70 2023-05-20T01:26:17.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.374+0000 7f18c0c3d700 1 -- 172.21.15.16:0/3125705484 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18ac00ed10 con 0x7f18a8003a70 2023-05-20T01:26:17.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.375+0000 7f18c3ea3700 1 -- 172.21.15.16:0/3125705484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18a8003a70 msgr2=0x7f18a8003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:17.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.375+0000 7f18c3ea3700 1 --2- 172.21.15.16:0/3125705484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18a8003a70 0x7f18a8003e80 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f18ac009d30 tx=0x7f18a8005880 comp rx=0 tx=0).stop 2023-05-20T01:26:17.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.375+0000 7f18c3ea3700 1 -- 172.21.15.16:0/3125705484 shutdown_connections 2023-05-20T01:26:17.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.375+0000 7f18c3ea3700 1 --2- 172.21.15.16:0/3125705484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18a8003a70 0x7f18a8003e80 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:17.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.375+0000 7f18c3ea3700 1 -- 172.21.15.16:0/3125705484 >> 172.21.15.16:0/3125705484 conn(0x7f18bc06ae30 msgr2=0x7f18bc09a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:17.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.375+0000 7f18c3ea3700 1 -- 172.21.15.16:0/3125705484 shutdown_connections 2023-05-20T01:26:17.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.375+0000 7f18c3ea3700 1 -- 172.21.15.16:0/3125705484 wait complete. 2023-05-20T01:26:17.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.376+0000 7f18c3ea3700 1 Processor -- start 2023-05-20T01:26:17.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.376+0000 7f18c3ea3700 1 -- start start 2023-05-20T01:26:17.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.377+0000 7f18c3ea3700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f18a8003a70 0x7f18bc1f0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:17.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.377+0000 7f18c3ea3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18bc159140 0x7f18bc158910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:17.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.377+0000 7f18c3ea3700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f18bc0b65b0 0x7f18bc0b4740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:17.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.377+0000 7f18c3ea3700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18bc1f0f60 con 0x7f18bc159140 2023-05-20T01:26:17.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.377+0000 7f18c3ea3700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18bc15c280 con 0x7f18bc0b65b0 2023-05-20T01:26:17.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.377+0000 7f18c3ea3700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18bc15c560 con 0x7f18a8003a70 2023-05-20T01:26:17.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.377+0000 7f18c143e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18bc159140 0x7f18bc158910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:17.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.377+0000 7f18c143e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18bc159140 0x7f18bc158910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37524/0 (socket says 172.21.15.16:37524) 2023-05-20T01:26:17.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.377+0000 7f18c143e700 1 -- 172.21.15.16:0/3174289818 learned_addr learned my addr 172.21.15.16:0/3174289818 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:26:17.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.377+0000 7f18c1c3f700 1 --2- 172.21.15.16:0/3174289818 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f18a8003a70 0x7f18bc1f0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:17.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.377+0000 7f18c2440700 1 --2- 172.21.15.16:0/3174289818 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f18bc0b65b0 0x7f18bc0b4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:17.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.377+0000 7f18c143e700 1 -- 172.21.15.16:0/3174289818 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f18a8003a70 msgr2=0x7f18bc1f0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:17.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.377+0000 7f18c143e700 1 --2- 172.21.15.16:0/3174289818 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f18a8003a70 0x7f18bc1f0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:17.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.377+0000 7f18c143e700 1 -- 172.21.15.16:0/3174289818 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f18bc0b65b0 msgr2=0x7f18bc0b4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:17.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.377+0000 7f18c143e700 1 --2- 172.21.15.16:0/3174289818 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f18bc0b65b0 0x7f18bc0b4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:17.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.377+0000 7f18c143e700 1 -- 172.21.15.16:0/3174289818 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18bc0cc190 con 0x7f18bc159140 2023-05-20T01:26:17.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.377+0000 7f18c2440700 1 --2- 172.21.15.16:0/3174289818 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f18bc0b65b0 0x7f18bc0b4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:26:17.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.377+0000 7f18c143e700 1 --2- 172.21.15.16:0/3174289818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18bc159140 0x7f18bc158910 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f18b80011e0 tx=0x7f18b8007620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:17.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.378+0000 7f18b27fc700 1 -- 172.21.15.16:0/3174289818 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18b800f6c0 con 0x7f18bc159140 2023-05-20T01:26:17.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.378+0000 7f18b27fc700 1 -- 172.21.15.16:0/3174289818 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18b800f840 con 0x7f18bc159140 2023-05-20T01:26:17.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.378+0000 7f18b27fc700 1 -- 172.21.15.16:0/3174289818 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18b800fbf0 con 0x7f18bc159140 2023-05-20T01:26:17.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.378+0000 7f18c3ea3700 1 -- 172.21.15.16:0/3174289818 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18bc0cbeb0 con 0x7f18bc159140 2023-05-20T01:26:17.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.378+0000 7f18c3ea3700 1 -- 172.21.15.16:0/3174289818 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18bc0d1680 con 0x7f18bc159140 2023-05-20T01:26:17.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.379+0000 7f18c3ea3700 1 -- 172.21.15.16:0/3174289818 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18bc0747b0 con 0x7f18bc159140 2023-05-20T01:26:17.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.379+0000 7f18b27fc700 1 -- 172.21.15.16:0/3174289818 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f18b800fd50 con 0x7f18bc159140 2023-05-20T01:26:17.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.380+0000 7f18b27fc700 1 --2- 172.21.15.16:0/3174289818 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f18a006ace0 0x7f18a006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:17.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.380+0000 7f18c1c3f700 1 --2- 172.21.15.16:0/3174289818 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f18a006ace0 0x7f18a006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:17.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.380+0000 7f18b27fc700 1 -- 172.21.15.16:0/3174289818 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 7102+0+0 (secure 0 0 0) 0x7f18b8089840 con 0x7f18bc159140 2023-05-20T01:26:17.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.380+0000 7f18c1c3f700 1 --2- 172.21.15.16:0/3174289818 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f18a006ace0 0x7f18a006d190 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f18ac0090b0 tx=0x7f18ac008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:17.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.384+0000 7f18b27fc700 1 -- 172.21.15.16:0/3174289818 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f18b8054570 con 0x7f18bc159140 2023-05-20T01:26:17.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.602+0000 7f18c3ea3700 1 -- 172.21.15.16:0/3174289818 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f18bc0665d0 con 0x7f18bc159140 2023-05-20T01:26:17.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.602+0000 7f18b27fc700 1 -- 172.21.15.16:0/3174289818 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 6 entries v99) v1 ==== 80+0+362 (secure 0 0 0) 0x7f18b8057b90 con 0x7f18bc159140 2023-05-20T01:26:17.607 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6834/2740585996 2023-05-21T01:25:37.758863+0000 2023-05-20T01:26:17.607 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6837/2740585996 2023-05-21T01:25:37.758863+0000 2023-05-20T01:26:17.607 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6837/1327905747 2023-05-21T01:25:32.534353+0000 2023-05-20T01:26:17.607 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6834/1327905747 2023-05-21T01:25:32.534353+0000 2023-05-20T01:26:17.607 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6837/28561908 2023-05-21T01:25:30.939570+0000 2023-05-20T01:26:17.607 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6834/28561908 2023-05-21T01:25:30.939570+0000 2023-05-20T01:26:17.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.604+0000 7f18c3ea3700 1 -- 172.21.15.16:0/3174289818 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f18a006ace0 msgr2=0x7f18a006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:17.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.604+0000 7f18c3ea3700 1 --2- 172.21.15.16:0/3174289818 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f18a006ace0 0x7f18a006d190 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f18ac0090b0 tx=0x7f18ac008030 comp rx=0 tx=0).stop 2023-05-20T01:26:17.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.604+0000 7f18c3ea3700 1 -- 172.21.15.16:0/3174289818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18bc159140 msgr2=0x7f18bc158910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:17.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.604+0000 7f18c3ea3700 1 --2- 172.21.15.16:0/3174289818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18bc159140 0x7f18bc158910 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f18b80011e0 tx=0x7f18b8007620 comp rx=0 tx=0).stop 2023-05-20T01:26:17.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.604+0000 7f18c3ea3700 1 -- 172.21.15.16:0/3174289818 shutdown_connections 2023-05-20T01:26:17.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.604+0000 7f18c3ea3700 1 --2- 172.21.15.16:0/3174289818 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f18bc0b65b0 0x7f18bc0b4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:17.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.604+0000 7f18c3ea3700 1 --2- 172.21.15.16:0/3174289818 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f18a006ace0 0x7f18a006d190 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:17.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.604+0000 7f18c3ea3700 1 --2- 172.21.15.16:0/3174289818 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f18a8003a70 0x7f18bc1f0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:17.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.604+0000 7f18c3ea3700 1 --2- 172.21.15.16:0/3174289818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18bc159140 0x7f18bc158910 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:17.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.604+0000 7f18c3ea3700 1 -- 172.21.15.16:0/3174289818 >> 172.21.15.16:0/3174289818 conn(0x7f18bc06ae30 msgr2=0x7f18bc0cd1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:17.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.604+0000 7f18c3ea3700 1 -- 172.21.15.16:0/3174289818 shutdown_connections 2023-05-20T01:26:17.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.604+0000 7f18c3ea3700 1 -- 172.21.15.16:0/3174289818 wait complete. 2023-05-20T01:26:17.610 INFO:teuthology.orchestra.run.smithi016.stderr:listed 6 entries 2023-05-20T01:26:17.621 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-05-20T01:26:17.621 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:26:17.621 DEBUG:teuthology.orchestra.run.smithi016:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-05-20T01:26:17.664 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2023-05-20T01:26:17.664 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:26:17.665 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-05-20T01:26:17.723 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2023-05-20T01:26:17.724 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/proc/self/mounts of=/dev/stdout 2023-05-20T01:26:17.746 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284284k,nr_inodes=4071071,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=45,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16175 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260556k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-05-20T01:26:17.746 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-05-20T01:26:17.747 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2023-05-20T01:26:17.783 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16287364k,nr_inodes=4071841,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=38,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16696 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3261172k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-05-20T01:26:17.784 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2023-05-20T01:26:17.785 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/proc/self/mounts of=/dev/stdout 2023-05-20T01:26:17.806 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284284k,nr_inodes=4071071,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=45,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16175 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260556k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-05-20T01:26:17.806 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-20T01:26:17.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.943+0000 7f5e51d09700 1 Processor -- start 2023-05-20T01:26:17.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.943+0000 7f5e51d09700 1 -- start start 2023-05-20T01:26:17.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.944+0000 7f5e51d09700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5e4c06f450 0x7f5e4c0cf890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:17.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.944+0000 7f5e51d09700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f5e4c06a510 0x7f5e4c06a950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:17.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.944+0000 7f5e51d09700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e4c0cc510 con 0x7f5e4c06d4b0 2023-05-20T01:26:17.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.944+0000 7f5e51d09700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e4c06ae90 con 0x7f5e4c06f450 2023-05-20T01:26:17.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.944+0000 7f5e51d09700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e4c071ed0 con 0x7f5e4c06a510 2023-05-20T01:26:17.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.944+0000 7f5e4b7fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5e4c06f450 0x7f5e4c0cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:17.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.944+0000 7f5e4b7fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5e4c06f450 0x7f5e4c0cf890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37530/0 (socket says 172.21.15.16:37530) 2023-05-20T01:26:17.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.944+0000 7f5e4b7fe700 1 -- 172.21.15.16:0/3890932059 learned_addr learned my addr 172.21.15.16:0/3890932059 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:26:17.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.944+0000 7f5e4b7fe700 1 -- 172.21.15.16:0/3890932059 >> v2:172.21.15.132:3300/0 conn(0x7f5e4c06a510 msgr2=0x7f5e4c06a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:17.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.944+0000 7f5e4b7fe700 1 --2- 172.21.15.16:0/3890932059 >> v2:172.21.15.132:3300/0 conn(0x7f5e4c06a510 0x7f5e4c06a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:17.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.944+0000 7f5e4b7fe700 1 -- 172.21.15.16:0/3890932059 >> v1:172.21.15.132:6789/0 conn(0x7f5e4c06d4b0 legacy=0x7f5e4c161080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:17.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.944+0000 7f5e4b7fe700 1 -- 172.21.15.16:0/3890932059 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e4c0721b0 con 0x7f5e4c06f450 2023-05-20T01:26:17.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.944+0000 7f5e4b7fe700 1 --2- 172.21.15.16:0/3890932059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e4c06f450 0x7f5e4c0cf890 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c06f9f0 tx=0x7f5e3c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f66b9f53cda7d92f server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:17.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.945+0000 7f5e4a7fc700 1 -- 172.21.15.16:0/3890932059 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e3c004230 con 0x7f5e4c06f450 2023-05-20T01:26:17.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.945+0000 7f5e4a7fc700 1 -- 172.21.15.16:0/3890932059 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e3c004390 con 0x7f5e4c06f450 2023-05-20T01:26:17.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.945+0000 7f5e4a7fc700 1 -- 172.21.15.16:0/3890932059 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e3c0036a0 con 0x7f5e4c06f450 2023-05-20T01:26:17.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.945+0000 7f5e51d09700 1 -- 172.21.15.16:0/3890932059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e4c06f450 msgr2=0x7f5e4c0cf890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:17.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.945+0000 7f5e51d09700 1 --2- 172.21.15.16:0/3890932059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e4c06f450 0x7f5e4c0cf890 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c06f9f0 tx=0x7f5e3c009bd0 comp rx=0 tx=0).stop 2023-05-20T01:26:17.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.945+0000 7f5e51d09700 1 -- 172.21.15.16:0/3890932059 shutdown_connections 2023-05-20T01:26:17.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.945+0000 7f5e51d09700 1 --2- 172.21.15.16:0/3890932059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e4c06f450 0x7f5e4c0cf890 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:17.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.945+0000 7f5e51d09700 1 --2- 172.21.15.16:0/3890932059 >> v2:172.21.15.132:3300/0 conn(0x7f5e4c06a510 0x7f5e4c06a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:17.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.945+0000 7f5e51d09700 1 -- 172.21.15.16:0/3890932059 >> 172.21.15.16:0/3890932059 conn(0x7f5e4c0cffb0 msgr2=0x7f5e4c0d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:17.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.945+0000 7f5e51d09700 1 -- 172.21.15.16:0/3890932059 shutdown_connections 2023-05-20T01:26:17.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.945+0000 7f5e51d09700 1 -- 172.21.15.16:0/3890932059 wait complete. 2023-05-20T01:26:17.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.946+0000 7f5e51d09700 1 Processor -- start 2023-05-20T01:26:17.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.947+0000 7f5e51d09700 1 -- start start 2023-05-20T01:26:17.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.947+0000 7f5e51d09700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5e4c06a510 0x7f5e4c14b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:17.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.947+0000 7f5e51d09700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e4c06d4b0 0x7f5e4c0ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:17.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.947+0000 7f5e51d09700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5e4c06f450 0x7f5e4c0c72b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:17.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.947+0000 7f5e51d09700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e4c14ba40 con 0x7f5e4c06d4b0 2023-05-20T01:26:17.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.947+0000 7f5e51d09700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e4c0c5bd0 con 0x7f5e4c06a510 2023-05-20T01:26:17.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.947+0000 7f5e51d09700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e4c0c4fc0 con 0x7f5e4c06f450 2023-05-20T01:26:17.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.948+0000 7f5e4affd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e4c06d4b0 0x7f5e4c0ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:17.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.948+0000 7f5e4affd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e4c06d4b0 0x7f5e4c0ae8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37536/0 (socket says 172.21.15.16:37536) 2023-05-20T01:26:17.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.948+0000 7f5e4affd700 1 -- 172.21.15.16:0/3893093803 learned_addr learned my addr 172.21.15.16:0/3893093803 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:26:17.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.948+0000 7f5e4b7fe700 1 --2- 172.21.15.16:0/3893093803 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5e4c06a510 0x7f5e4c14b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:17.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.948+0000 7f5e4bfff700 1 --2- 172.21.15.16:0/3893093803 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5e4c06f450 0x7f5e4c0c72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:17.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.948+0000 7f5e4affd700 1 -- 172.21.15.16:0/3893093803 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5e4c06f450 msgr2=0x7f5e4c0c72b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:17.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.948+0000 7f5e4affd700 1 --2- 172.21.15.16:0/3893093803 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5e4c06f450 0x7f5e4c0c72b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:17.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.948+0000 7f5e4affd700 1 -- 172.21.15.16:0/3893093803 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5e4c06a510 msgr2=0x7f5e4c14b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:17.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.948+0000 7f5e4affd700 1 --2- 172.21.15.16:0/3893093803 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5e4c06a510 0x7f5e4c14b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:17.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.948+0000 7f5e4affd700 1 -- 172.21.15.16:0/3893093803 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e3c009550 con 0x7f5e4c06d4b0 2023-05-20T01:26:17.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.948+0000 7f5e4affd700 1 --2- 172.21.15.16:0/3893093803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e4c06d4b0 0x7f5e4c0ae8e0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f5e3400c4a0 tx=0x7f5e3400c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:17.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.948+0000 7f5e43fff700 1 -- 172.21.15.16:0/3893093803 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e340054e0 con 0x7f5e4c06d4b0 2023-05-20T01:26:17.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.948+0000 7f5e43fff700 1 -- 172.21.15.16:0/3893093803 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e34003680 con 0x7f5e4c06d4b0 2023-05-20T01:26:17.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.948+0000 7f5e51d09700 1 -- 172.21.15.16:0/3893093803 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e4c0cbe50 con 0x7f5e4c06d4b0 2023-05-20T01:26:17.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.948+0000 7f5e43fff700 1 -- 172.21.15.16:0/3893093803 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e34003a30 con 0x7f5e4c06d4b0 2023-05-20T01:26:17.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.948+0000 7f5e51d09700 1 -- 172.21.15.16:0/3893093803 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e4c0cc1a0 con 0x7f5e4c06d4b0 2023-05-20T01:26:17.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.949+0000 7f5e51d09700 1 -- 172.21.15.16:0/3893093803 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e4c0cc9a0 con 0x7f5e4c06d4b0 2023-05-20T01:26:17.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.950+0000 7f5e43fff700 1 -- 172.21.15.16:0/3893093803 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5e34003b90 con 0x7f5e4c06d4b0 2023-05-20T01:26:17.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.950+0000 7f5e43fff700 1 --2- 172.21.15.16:0/3893093803 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f5e2c06ace0 0x7f5e2c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:17.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.950+0000 7f5e43fff700 1 -- 172.21.15.16:0/3893093803 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 7102+0+0 (secure 0 0 0) 0x7f5e34088ca0 con 0x7f5e4c06d4b0 2023-05-20T01:26:17.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.950+0000 7f5e4b7fe700 1 --2- 172.21.15.16:0/3893093803 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f5e2c06ace0 0x7f5e2c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:17.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.951+0000 7f5e4b7fe700 1 --2- 172.21.15.16:0/3893093803 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f5e2c06ace0 0x7f5e2c06d190 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c009130 tx=0x7f5e3c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:17.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:17.955+0000 7f5e43fff700 1 -- 172.21.15.16:0/3893093803 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f5e34053940 con 0x7f5e4c06d4b0 2023-05-20T01:26:18.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.197+0000 7f5e51d09700 1 -- 172.21.15.16:0/3893093803 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5e4c0cb8c0 con 0x7f5e4c06d4b0 2023-05-20T01:26:18.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.197+0000 7f5e43fff700 1 -- 172.21.15.16:0/3893093803 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) v1 ==== 94+0+4671 (secure 0 0 0) 0x7f5e34017090 con 0x7f5e4c06d4b0 2023-05-20T01:26:18.202 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:26:18.202 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":70,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6323,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/59032067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":59032067},{"type":"v1","addr":"172.21.15.132:6834","nonce":59032067}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66},{"gid":6329,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/100162236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":100162236},{"type":"v1","addr":"172.21.15.132:6835","nonce":100162236}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66},{"gid":7572,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/3753560731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3753560731},{"type":"v1","addr":"172.21.15.16:6836","nonce":3753560731}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":70,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-20T01:25:32.295399+0000","modified":"2023-05-20T01:25:41.570561+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":95,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7575},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7575":{"gid":7575,"name":"c","rank":0,"incarnation":67,"state":"up:active","state_seq":4,"addr":"172.21.15.16:6837/307899603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":307899603},{"type":"v1","addr":"172.21.15.16:6837","nonce":307899603}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13,14],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-05-20T01:26:18.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.199+0000 7f5e51d09700 1 -- 172.21.15.16:0/3893093803 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f5e2c06ace0 msgr2=0x7f5e2c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:18.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.199+0000 7f5e51d09700 1 --2- 172.21.15.16:0/3893093803 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f5e2c06ace0 0x7f5e2c06d190 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c009130 tx=0x7f5e3c008030 comp rx=0 tx=0).stop 2023-05-20T01:26:18.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.199+0000 7f5e51d09700 1 -- 172.21.15.16:0/3893093803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e4c06d4b0 msgr2=0x7f5e4c0ae8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:18.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.199+0000 7f5e51d09700 1 --2- 172.21.15.16:0/3893093803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e4c06d4b0 0x7f5e4c0ae8e0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f5e3400c4a0 tx=0x7f5e3400c7b0 comp rx=0 tx=0).stop 2023-05-20T01:26:18.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.199+0000 7f5e51d09700 1 -- 172.21.15.16:0/3893093803 shutdown_connections 2023-05-20T01:26:18.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.199+0000 7f5e51d09700 1 --2- 172.21.15.16:0/3893093803 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f5e2c06ace0 0x7f5e2c06d190 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:18.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.199+0000 7f5e51d09700 1 --2- 172.21.15.16:0/3893093803 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5e4c06f450 0x7f5e4c0c72b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:18.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.199+0000 7f5e51d09700 1 --2- 172.21.15.16:0/3893093803 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5e4c06a510 0x7f5e4c14b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:18.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.199+0000 7f5e51d09700 1 --2- 172.21.15.16:0/3893093803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e4c06d4b0 0x7f5e4c0ae8e0 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:18.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.199+0000 7f5e51d09700 1 -- 172.21.15.16:0/3893093803 >> 172.21.15.16:0/3893093803 conn(0x7f5e4c0cffb0 msgr2=0x7f5e4c0b4e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:18.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.199+0000 7f5e51d09700 1 -- 172.21.15.16:0/3893093803 shutdown_connections 2023-05-20T01:26:18.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.199+0000 7f5e51d09700 1 -- 172.21.15.16:0/3893093803 wait complete. 2023-05-20T01:26:18.206 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 70 2023-05-20T01:26:18.216 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-20T01:26:18.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.335+0000 7f6ad6fdb700 1 Processor -- start 2023-05-20T01:26:18.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.335+0000 7f6ad6fdb700 1 -- start start 2023-05-20T01:26:18.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.336+0000 7f6ad6fdb700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f6ad006cb90 0x7f6ad015ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:18.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.336+0000 7f6ad6fdb700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6ad00b54f0 0x7f6ad00b29e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:18.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.336+0000 7f6ad6fdb700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ad00b2f20 con 0x7f6ad006cf60 2023-05-20T01:26:18.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.336+0000 7f6ad6fdb700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ad00cc990 con 0x7f6ad00b54f0 2023-05-20T01:26:18.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.336+0000 7f6ad6fdb700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ad00b31a0 con 0x7f6ad006cb90 2023-05-20T01:26:18.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.336+0000 7f6acffff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6ad00b54f0 0x7f6ad00b29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:18.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.336+0000 7f6acffff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6ad00b54f0 0x7f6ad00b29e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37544/0 (socket says 172.21.15.16:37544) 2023-05-20T01:26:18.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.336+0000 7f6acffff700 1 -- 172.21.15.16:0/835803344 learned_addr learned my addr 172.21.15.16:0/835803344 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:26:18.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.336+0000 7f6ad4d77700 1 --2- 172.21.15.16:0/835803344 >> v2:172.21.15.132:3300/0 conn(0x7f6ad006cb90 0x7f6ad015ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:18.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.337+0000 7f6acffff700 1 -- 172.21.15.16:0/835803344 >> v2:172.21.15.132:3300/0 conn(0x7f6ad006cb90 msgr2=0x7f6ad015ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:18.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.337+0000 7f6acffff700 1 --2- 172.21.15.16:0/835803344 >> v2:172.21.15.132:3300/0 conn(0x7f6ad006cb90 0x7f6ad015ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:18.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.337+0000 7f6acffff700 1 -- 172.21.15.16:0/835803344 >> v1:172.21.15.132:6789/0 conn(0x7f6ad006cf60 legacy=0x7f6ad0070910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:18.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.337+0000 7f6acffff700 1 -- 172.21.15.16:0/835803344 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ad00b3480 con 0x7f6ad00b54f0 2023-05-20T01:26:18.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.337+0000 7f6ad4d77700 1 --2- 172.21.15.16:0/835803344 >> v2:172.21.15.132:3300/0 conn(0x7f6ad006cb90 0x7f6ad015ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:26:18.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.337+0000 7f6acffff700 1 --2- 172.21.15.16:0/835803344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ad00b54f0 0x7f6ad00b29e0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f6ad00ccf70 tx=0x7f6abc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bea2d2e46cfe72b server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:18.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.337+0000 7f6acf7fe700 1 -- 172.21.15.16:0/835803344 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6abc004230 con 0x7f6ad00b54f0 2023-05-20T01:26:18.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.337+0000 7f6acf7fe700 1 -- 172.21.15.16:0/835803344 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6abc004390 con 0x7f6ad00b54f0 2023-05-20T01:26:18.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.337+0000 7f6acf7fe700 1 -- 172.21.15.16:0/835803344 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6abc0036a0 con 0x7f6ad00b54f0 2023-05-20T01:26:18.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.337+0000 7f6ad6fdb700 1 -- 172.21.15.16:0/835803344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ad00b54f0 msgr2=0x7f6ad00b29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:18.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.337+0000 7f6ad6fdb700 1 --2- 172.21.15.16:0/835803344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ad00b54f0 0x7f6ad00b29e0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f6ad00ccf70 tx=0x7f6abc009bd0 comp rx=0 tx=0).stop 2023-05-20T01:26:18.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.337+0000 7f6ad6fdb700 1 -- 172.21.15.16:0/835803344 shutdown_connections 2023-05-20T01:26:18.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.337+0000 7f6ad6fdb700 1 --2- 172.21.15.16:0/835803344 >> v2:172.21.15.132:3300/0 conn(0x7f6ad006cb90 0x7f6ad015ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:18.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.337+0000 7f6ad6fdb700 1 --2- 172.21.15.16:0/835803344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ad00b54f0 0x7f6ad00b29e0 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:18.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.337+0000 7f6ad6fdb700 1 -- 172.21.15.16:0/835803344 >> 172.21.15.16:0/835803344 conn(0x7f6ad00661b0 msgr2=0x7f6ad0066580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:18.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.338+0000 7f6ad6fdb700 1 -- 172.21.15.16:0/835803344 shutdown_connections 2023-05-20T01:26:18.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.338+0000 7f6ad6fdb700 1 -- 172.21.15.16:0/835803344 wait complete. 2023-05-20T01:26:18.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.339+0000 7f6ad6fdb700 1 Processor -- start 2023-05-20T01:26:18.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.339+0000 7f6ad6fdb700 1 -- start start 2023-05-20T01:26:18.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.340+0000 7f6ad6fdb700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6ad006cb90 0x7f6ad01594c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:18.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.340+0000 7f6ad6fdb700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ad006cf60 0x7f6ad0157c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:18.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.340+0000 7f6ad6fdb700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ad00b54f0 0x7f6ad0141c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:18.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.340+0000 7f6ad6fdb700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad0156600 con 0x7f6ad00b54f0 2023-05-20T01:26:18.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.340+0000 7f6ad6fdb700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad01568e0 con 0x7f6ad006cf60 2023-05-20T01:26:18.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.340+0000 7f6ad6fdb700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad0156bc0 con 0x7f6ad006cb90 2023-05-20T01:26:18.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.340+0000 7f6ad5578700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ad00b54f0 0x7f6ad0141c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:18.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.340+0000 7f6ad5578700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ad00b54f0 0x7f6ad0141c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37554/0 (socket says 172.21.15.16:37554) 2023-05-20T01:26:18.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.340+0000 7f6ad5578700 1 -- 172.21.15.16:0/169260300 learned_addr learned my addr 172.21.15.16:0/169260300 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:26:18.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.341+0000 7f6acffff700 1 --2- 172.21.15.16:0/169260300 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ad006cf60 0x7f6ad0157c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:18.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.341+0000 7f6ad4d77700 1 --2- 172.21.15.16:0/169260300 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6ad006cb90 0x7f6ad01594c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:18.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.341+0000 7f6ad5578700 1 -- 172.21.15.16:0/169260300 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6ad006cb90 msgr2=0x7f6ad01594c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:18.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.341+0000 7f6ad5578700 1 --2- 172.21.15.16:0/169260300 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6ad006cb90 0x7f6ad01594c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:18.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.341+0000 7f6ad5578700 1 -- 172.21.15.16:0/169260300 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ad006cf60 msgr2=0x7f6ad0157c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:18.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.341+0000 7f6ad5578700 1 --2- 172.21.15.16:0/169260300 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ad006cf60 0x7f6ad0157c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:18.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.341+0000 7f6ad5578700 1 -- 172.21.15.16:0/169260300 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6abc009550 con 0x7f6ad00b54f0 2023-05-20T01:26:18.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.341+0000 7f6ad5578700 1 --2- 172.21.15.16:0/169260300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ad00b54f0 0x7f6ad0141c20 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f6ac800b970 tx=0x7f6ac800bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:18.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.341+0000 7f6acd7fa700 1 -- 172.21.15.16:0/169260300 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ac800d860 con 0x7f6ad00b54f0 2023-05-20T01:26:18.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.341+0000 7f6acd7fa700 1 -- 172.21.15.16:0/169260300 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ac800d9c0 con 0x7f6ad00b54f0 2023-05-20T01:26:18.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.341+0000 7f6ad6fdb700 1 -- 172.21.15.16:0/169260300 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ad015b240 con 0x7f6ad00b54f0 2023-05-20T01:26:18.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.341+0000 7f6acd7fa700 1 -- 172.21.15.16:0/169260300 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ac800dd70 con 0x7f6ad00b54f0 2023-05-20T01:26:18.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.341+0000 7f6ad6fdb700 1 -- 172.21.15.16:0/169260300 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ad015af60 con 0x7f6ad00b54f0 2023-05-20T01:26:18.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.342+0000 7f6ad6fdb700 1 -- 172.21.15.16:0/169260300 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ad015b520 con 0x7f6ad00b54f0 2023-05-20T01:26:18.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.343+0000 7f6acd7fa700 1 -- 172.21.15.16:0/169260300 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6ac800e430 con 0x7f6ad00b54f0 2023-05-20T01:26:18.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.343+0000 7f6acd7fa700 1 --2- 172.21.15.16:0/169260300 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f6ab406ace0 0x7f6ab406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:18.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.343+0000 7f6acd7fa700 1 -- 172.21.15.16:0/169260300 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 7102+0+0 (secure 0 0 0) 0x7f6ac80899c0 con 0x7f6ad00b54f0 2023-05-20T01:26:18.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.343+0000 7f6ad4d77700 1 --2- 172.21.15.16:0/169260300 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f6ab406ace0 0x7f6ab406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:18.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.343+0000 7f6ad4d77700 1 --2- 172.21.15.16:0/169260300 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f6ab406ace0 0x7f6ab406d190 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f6ac4006fd0 tx=0x7f6ac4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:18.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.347+0000 7f6acd7fa700 1 -- 172.21.15.16:0/169260300 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f6ac8054660 con 0x7f6ad00b54f0 2023-05-20T01:26:18.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.546+0000 7f6ad6fdb700 1 -- 172.21.15.16:0/169260300 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6ad00b45f0 con 0x7f6ad00b54f0 2023-05-20T01:26:18.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.547+0000 7f6acd7fa700 1 -- 172.21.15.16:0/169260300 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) v1 ==== 74+0+20720 (secure 0 0 0) 0x7f6ac8057c80 con 0x7f6ad00b54f0 2023-05-20T01:26:18.552 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:26:18.552 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":99,"fsid":"c2245cc5-ecab-4087-a59b-bc81db901676","created":"2023-05-20T01:16:09.549230+0000","modified":"2023-05-20T01:26:17.513405+0000","last_up_change":"2023-05-20T01:16:19.788624+0000","last_in_change":"2023-05-20T01:16:12.092160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-05-20T01:16:20.934366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-05-20T01:16:21.372294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-05-20T01:25:09.967348+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-05-20T01:25:10.956221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data_extra","create_time":"2023-05-20T01:25:24.629648+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"317a83cd-19c8-4958-a301-6087ef342289","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6825","nonce":3923634179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6827","nonce":3923634179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6831","nonce":3923634179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6829","nonce":3923634179}]},"public_addr":"172.21.15.16:6825/3923634179","cluster_addr":"172.21.15.16:6827/3923634179","heartbeat_back_addr":"172.21.15.16:6831/3923634179","heartbeat_front_addr":"172.21.15.16:6829/3923634179","state":["exists","up"]},{"osd":1,"uuid":"5581046f-8219-4089-8d7d-b7846399505e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6809","nonce":3860264220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6811","nonce":3860264220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6815","nonce":3860264220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6813","nonce":3860264220}]},"public_addr":"172.21.15.16:6809/3860264220","cluster_addr":"172.21.15.16:6811/3860264220","heartbeat_back_addr":"172.21.15.16:6815/3860264220","heartbeat_front_addr":"172.21.15.16:6813/3860264220","state":["exists","up"]},{"osd":2,"uuid":"072e9e66-4133-478e-8d4a-64d46c2963f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6817","nonce":293187030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6819","nonce":293187030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6823","nonce":293187030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6821","nonce":293187030}]},"public_addr":"172.21.15.16:6817/293187030","cluster_addr":"172.21.15.16:6819/293187030","heartbeat_back_addr":"172.21.15.16:6823/293187030","heartbeat_front_addr":"172.21.15.16:6821/293187030","state":["exists","up"]},{"osd":3,"uuid":"11e1bba6-00f8-48dd-9028-d8d716b7f145","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6801","nonce":3669781890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6803","nonce":3669781890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6807","nonce":3669781890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6805","nonce":3669781890}]},"public_addr":"172.21.15.16:6801/3669781890","cluster_addr":"172.21.15.16:6803/3669781890","heartbeat_back_addr":"172.21.15.16:6807/3669781890","heartbeat_front_addr":"172.21.15.16:6805/3669781890","state":["exists","up"]},{"osd":4,"uuid":"b8909c3a-cf70-46cb-8d0b-009c707cf9d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6819","nonce":506317223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6823","nonce":506317223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6829","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6831","nonce":506317223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6827","nonce":506317223}]},"public_addr":"172.21.15.132:6819/506317223","cluster_addr":"172.21.15.132:6823/506317223","heartbeat_back_addr":"172.21.15.132:6831/506317223","heartbeat_front_addr":"172.21.15.132:6827/506317223","state":["exists","up"]},{"osd":5,"uuid":"b1818304-4583-47bf-a6ec-0bfbb5166233","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6818","nonce":1844689995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6822","nonce":1844689995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6830","nonce":1844689995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6826","nonce":1844689995}]},"public_addr":"172.21.15.132:6818/1844689995","cluster_addr":"172.21.15.132:6822/1844689995","heartbeat_back_addr":"172.21.15.132:6830/1844689995","heartbeat_front_addr":"172.21.15.132:6826/1844689995","state":["exists","up"]},{"osd":6,"uuid":"d37f4691-eed1-4a93-8818-18354f03f4a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6801","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6802","nonce":727581123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6805","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6807","nonce":727581123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6813","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6815","nonce":727581123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6811","nonce":727581123}]},"public_addr":"172.21.15.132:6802/727581123","cluster_addr":"172.21.15.132:6807/727581123","heartbeat_back_addr":"172.21.15.132:6815/727581123","heartbeat_front_addr":"172.21.15.132:6811/727581123","state":["exists","up"]},{"osd":7,"uuid":"4f712f62-5d1d-4816-8042-03209144f8b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6803","nonce":466001149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6806","nonce":466001149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6814","nonce":466001149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6810","nonce":466001149}]},"public_addr":"172.21.15.132:6803/466001149","cluster_addr":"172.21.15.132:6806/466001149","heartbeat_back_addr":"172.21.15.132:6814/466001149","heartbeat_front_addr":"172.21.15.132:6810/466001149","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.338863+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.278680+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305694+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.246835+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.276237+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.304441+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305421+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.267050+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.f","mappings":[{"from":6,"to":3}]},{"pgid":"13.23","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6834/2740585996":"2023-05-21T01:25:37.758863+0000","172.21.15.16:6837/2740585996":"2023-05-21T01:25:37.758863+0000","172.21.15.16:6837/1327905747":"2023-05-21T01:25:32.534353+0000","172.21.15.16:6834/1327905747":"2023-05-21T01:25:32.534353+0000","172.21.15.16:6837/28561908":"2023-05-21T01:25:30.939570+0000","172.21.15.16:6834/28561908":"2023-05-21T01:25:30.939570+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-20T01:26:18.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.549+0000 7f6ad6fdb700 1 -- 172.21.15.16:0/169260300 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f6ab406ace0 msgr2=0x7f6ab406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:18.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.549+0000 7f6ad6fdb700 1 --2- 172.21.15.16:0/169260300 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f6ab406ace0 0x7f6ab406d190 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f6ac4006fd0 tx=0x7f6ac4008040 comp rx=0 tx=0).stop 2023-05-20T01:26:18.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.549+0000 7f6ad6fdb700 1 -- 172.21.15.16:0/169260300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ad00b54f0 msgr2=0x7f6ad0141c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:18.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.549+0000 7f6ad6fdb700 1 --2- 172.21.15.16:0/169260300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ad00b54f0 0x7f6ad0141c20 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f6ac800b970 tx=0x7f6ac800bc80 comp rx=0 tx=0).stop 2023-05-20T01:26:18.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.549+0000 7f6ad6fdb700 1 -- 172.21.15.16:0/169260300 shutdown_connections 2023-05-20T01:26:18.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.549+0000 7f6ad6fdb700 1 --2- 172.21.15.16:0/169260300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ad00b54f0 0x7f6ad0141c20 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:18.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.549+0000 7f6ad6fdb700 1 --2- 172.21.15.16:0/169260300 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f6ab406ace0 0x7f6ab406d190 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:18.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.549+0000 7f6ad6fdb700 1 --2- 172.21.15.16:0/169260300 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6ad006cb90 0x7f6ad01594c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:18.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.549+0000 7f6ad6fdb700 1 --2- 172.21.15.16:0/169260300 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ad006cf60 0x7f6ad0157c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:18.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.549+0000 7f6ad6fdb700 1 -- 172.21.15.16:0/169260300 >> 172.21.15.16:0/169260300 conn(0x7f6ad00661b0 msgr2=0x7f6ad015a370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:18.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.549+0000 7f6ad6fdb700 1 -- 172.21.15.16:0/169260300 shutdown_connections 2023-05-20T01:26:18.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.549+0000 7f6ad6fdb700 1 -- 172.21.15.16:0/169260300 wait complete. 2023-05-20T01:26:18.566 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-05-20T01:16:20.934366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-05-20T01:16:21.372294+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2023-05-20T01:25:09.967348+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2023-05-20T01:25:10.956221+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data_extra', 'create_time': '2023-05-20T01:25:24.629648+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-05-20T01:26:18.567 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-05-20T01:26:18.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.691+0000 7f8fbdfda700 1 Processor -- start 2023-05-20T01:26:18.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.691+0000 7f8fbdfda700 1 -- start start 2023-05-20T01:26:18.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.692+0000 7f8fbdfda700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f8fb8158f90 0x7f8fb80cb7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:18.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.692+0000 7f8fbdfda700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fb8159360 con 0x7f8fb80ce0a0 2023-05-20T01:26:18.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.692+0000 7f8fbdfda700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fb80ca750 con 0x7f8fb80cc760 2023-05-20T01:26:18.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.692+0000 7f8fbdfda700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fb80cce10 con 0x7f8fb8158f90 2023-05-20T01:26:18.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.692+0000 7f8fb6ffd700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f8fb80ce0a0 0x7f8fb80cc050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49088/0 (socket says 172.21.15.16:49088) 2023-05-20T01:26:18.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.692+0000 7f8fb6ffd700 1 -- 172.21.15.16:0/1212537983 learned_addr learned my addr 172.21.15.16:0/1212537983 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:26:18.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.692+0000 7f8fb77fe700 1 --2- 172.21.15.16:0/1212537983 >> v2:172.21.15.132:3300/0 conn(0x7f8fb8158f90 0x7f8fb80cb7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:18.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.693+0000 7f8fb67fc700 1 -- 172.21.15.16:0/1212537983 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1332573926 0 0) 0x7f8fa0001110 con 0x7f8fb80ce0a0 2023-05-20T01:26:18.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.693+0000 7f8fb67fc700 1 -- 172.21.15.16:0/1212537983 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8fa40035b0 con 0x7f8fb80ce0a0 2023-05-20T01:26:18.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.693+0000 7f8fb67fc700 1 -- 172.21.15.16:0/1212537983 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3428128905 0 0) 0x7f8fa40035b0 con 0x7f8fb80ce0a0 2023-05-20T01:26:18.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.693+0000 7f8fb67fc700 1 -- 172.21.15.16:0/1212537983 >> v2:172.21.15.132:3300/0 conn(0x7f8fb8158f90 msgr2=0x7f8fb80cb7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:18.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.693+0000 7f8fb67fc700 1 --2- 172.21.15.16:0/1212537983 >> v2:172.21.15.132:3300/0 conn(0x7f8fb8158f90 0x7f8fb80cb7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:18.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.693+0000 7f8fb67fc700 1 -- 172.21.15.16:0/1212537983 >> v1:172.21.15.132:6789/0 conn(0x7f8fb80cc760 legacy=0x7f8fb8070930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:18.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.693+0000 7f8fb67fc700 1 -- 172.21.15.16:0/1212537983 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fb80ccb60 con 0x7f8fb80ce0a0 2023-05-20T01:26:18.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.693+0000 7f8fb67fc700 1 -- 172.21.15.16:0/1212537983 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7f8fa0001800 con 0x7f8fb80ce0a0 2023-05-20T01:26:18.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.693+0000 7f8fb67fc700 1 -- 172.21.15.16:0/1212537983 >> v1:172.21.15.16:6789/0 conn(0x7f8fb80ce0a0 legacy=0x7f8fb80cc050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:18.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.693+0000 7f8fb67fc700 1 --2- 172.21.15.16:0/1212537983 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8fa4003a00 0x7f8fa4003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:18.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.693+0000 7f8fb67fc700 1 --2- 172.21.15.16:0/1212537983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8fa4004850 0x7f8fa4004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:18.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.693+0000 7f8fb67fc700 1 --2- 172.21.15.16:0/1212537983 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8fa4005200 0x7f8fa400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:18.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.693+0000 7f8fb67fc700 1 -- 172.21.15.16:0/1212537983 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fa400c050 con 0x7f8fa4004850 2023-05-20T01:26:18.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.693+0000 7f8fb67fc700 1 -- 172.21.15.16:0/1212537983 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fa400c1c0 con 0x7f8fa4005200 2023-05-20T01:26:18.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.693+0000 7f8fb67fc700 1 -- 172.21.15.16:0/1212537983 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fa400c380 con 0x7f8fa4003a00 2023-05-20T01:26:18.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.693+0000 7f8fb6ffd700 1 --2- 172.21.15.16:0/1212537983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8fa4004850 0x7f8fa4004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:18.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.693+0000 7f8fb77fe700 1 --2- 172.21.15.16:0/1212537983 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8fa4003a00 0x7f8fa4003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:18.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.693+0000 7f8fb7fff700 1 --2- 172.21.15.16:0/1212537983 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8fa4005200 0x7f8fa400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:18.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.693+0000 7f8fb6ffd700 1 -- 172.21.15.16:0/1212537983 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8fa4003a00 msgr2=0x7f8fa4003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:18.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.693+0000 7f8fb6ffd700 1 --2- 172.21.15.16:0/1212537983 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8fa4003a00 0x7f8fa4003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:18.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.693+0000 7f8fb6ffd700 1 -- 172.21.15.16:0/1212537983 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8fa4005200 msgr2=0x7f8fa400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:18.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.693+0000 7f8fb6ffd700 1 --2- 172.21.15.16:0/1212537983 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8fa4005200 0x7f8fa400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:18.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.693+0000 7f8fb6ffd700 1 -- 172.21.15.16:0/1212537983 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fa40035b0 con 0x7f8fa4004850 2023-05-20T01:26:18.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.693+0000 7f8fb7fff700 1 -- 172.21.15.16:0/1212537983 reap_dead start 2023-05-20T01:26:18.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.694+0000 7f8fb6ffd700 1 --2- 172.21.15.16:0/1212537983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8fa4004850 0x7f8fa4004cc0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f8fa000c210 tx=0x7f8fa4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10f3c709a554cc2 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:18.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.694+0000 7f8fb67fc700 1 -- 172.21.15.16:0/1212537983 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fa0007d90 con 0x7f8fa4004850 2023-05-20T01:26:18.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.694+0000 7f8fb67fc700 1 -- 172.21.15.16:0/1212537983 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fa000cdd0 con 0x7f8fa4004850 2023-05-20T01:26:18.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.694+0000 7f8fb67fc700 1 -- 172.21.15.16:0/1212537983 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fa0005ba0 con 0x7f8fa4004850 2023-05-20T01:26:18.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.694+0000 7f8fbdfda700 1 -- 172.21.15.16:0/1212537983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8fa4004850 msgr2=0x7f8fa4004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:18.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.694+0000 7f8fbdfda700 1 --2- 172.21.15.16:0/1212537983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8fa4004850 0x7f8fa4004cc0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f8fa000c210 tx=0x7f8fa4005880 comp rx=0 tx=0).stop 2023-05-20T01:26:18.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.694+0000 7f8fbdfda700 1 -- 172.21.15.16:0/1212537983 shutdown_connections 2023-05-20T01:26:18.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.694+0000 7f8fbdfda700 1 --2- 172.21.15.16:0/1212537983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8fa4004850 0x7f8fa4004cc0 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:18.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.694+0000 7f8fbdfda700 1 -- 172.21.15.16:0/1212537983 >> 172.21.15.16:0/1212537983 conn(0x7f8fb806cbb0 msgr2=0x7f8fb806cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:18.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.694+0000 7f8fbdfda700 1 -- 172.21.15.16:0/1212537983 shutdown_connections 2023-05-20T01:26:18.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.694+0000 7f8fbdfda700 1 -- 172.21.15.16:0/1212537983 wait complete. 2023-05-20T01:26:18.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.696+0000 7f8fbdfda700 1 Processor -- start 2023-05-20T01:26:18.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.696+0000 7f8fbdfda700 1 -- start start 2023-05-20T01:26:18.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.696+0000 7f8fbdfda700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8fa4004850 0x7f8fb81452c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:18.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.696+0000 7f8fbdfda700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8fb8140fe0 0x7f8fb8147210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:18.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.696+0000 7f8fbdfda700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8fb813be00 0x7f8fb813c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:18.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.696+0000 7f8fbdfda700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fb815c430 con 0x7f8fb8140fe0 2023-05-20T01:26:18.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.696+0000 7f8fbdfda700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fb815bfa0 con 0x7f8fa4004850 2023-05-20T01:26:18.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.696+0000 7f8fbdfda700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fb813c7b0 con 0x7f8fb813be00 2023-05-20T01:26:18.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.697+0000 7f8fb6ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8fb8140fe0 0x7f8fb8147210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:18.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.697+0000 7f8fb6ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8fb8140fe0 0x7f8fb8147210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37572/0 (socket says 172.21.15.16:37572) 2023-05-20T01:26:18.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.697+0000 7f8fb6ffd700 1 -- 172.21.15.16:0/3832826216 learned_addr learned my addr 172.21.15.16:0/3832826216 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:26:18.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.697+0000 7f8fb77fe700 1 --2- 172.21.15.16:0/3832826216 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8fa4004850 0x7f8fb81452c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:18.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.697+0000 7f8fb7fff700 1 --2- 172.21.15.16:0/3832826216 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8fb813be00 0x7f8fb813c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:18.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.697+0000 7f8fb6ffd700 1 -- 172.21.15.16:0/3832826216 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8fb813be00 msgr2=0x7f8fb813c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:18.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.697+0000 7f8fb6ffd700 1 --2- 172.21.15.16:0/3832826216 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8fb813be00 0x7f8fb813c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:18.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.697+0000 7f8fb6ffd700 1 -- 172.21.15.16:0/3832826216 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8fa4004850 msgr2=0x7f8fb81452c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:18.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.697+0000 7f8fb6ffd700 1 --2- 172.21.15.16:0/3832826216 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8fa4004850 0x7f8fb81452c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:18.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.697+0000 7f8fb6ffd700 1 -- 172.21.15.16:0/3832826216 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fb80cacd0 con 0x7f8fb8140fe0 2023-05-20T01:26:18.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.697+0000 7f8fb6ffd700 1 --2- 172.21.15.16:0/3832826216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8fb8140fe0 0x7f8fb8147210 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f8fa0000ed0 tx=0x7f8fb80ca8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:18.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.697+0000 7f8faffff700 1 -- 172.21.15.16:0/3832826216 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fa0005ea0 con 0x7f8fb8140fe0 2023-05-20T01:26:18.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.697+0000 7f8faffff700 1 -- 172.21.15.16:0/3832826216 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fa00017c0 con 0x7f8fb8140fe0 2023-05-20T01:26:18.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.697+0000 7f8fbdfda700 1 -- 172.21.15.16:0/3832826216 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fb80ca9f0 con 0x7f8fb8140fe0 2023-05-20T01:26:18.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.697+0000 7f8faffff700 1 -- 172.21.15.16:0/3832826216 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fa0007310 con 0x7f8fb8140fe0 2023-05-20T01:26:18.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.697+0000 7f8fbdfda700 1 -- 172.21.15.16:0/3832826216 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fb80cafb0 con 0x7f8fb8140fe0 2023-05-20T01:26:18.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.698+0000 7f8fbdfda700 1 -- 172.21.15.16:0/3832826216 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fb80747b0 con 0x7f8fb8140fe0 2023-05-20T01:26:18.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.699+0000 7f8faffff700 1 -- 172.21.15.16:0/3832826216 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8fa0007470 con 0x7f8fb8140fe0 2023-05-20T01:26:18.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.699+0000 7f8faffff700 1 --2- 172.21.15.16:0/3832826216 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f8f9806ace0 0x7f8f9806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:18.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.700+0000 7f8fb77fe700 1 --2- 172.21.15.16:0/3832826216 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f8f9806ace0 0x7f8f9806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:18.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.700+0000 7f8faffff700 1 -- 172.21.15.16:0/3832826216 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 7102+0+0 (secure 0 0 0) 0x7f8fa008aba0 con 0x7f8fb8140fe0 2023-05-20T01:26:18.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.700+0000 7f8fb77fe700 1 --2- 172.21.15.16:0/3832826216 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f8f9806ace0 0x7f8f9806d190 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f8fa8009ce0 tx=0x7f8fa8009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:18.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.704+0000 7f8faffff700 1 -- 172.21.15.16:0/3832826216 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f8fa00558d0 con 0x7f8fb8140fe0 2023-05-20T01:26:18.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.901+0000 7f8fbdfda700 1 -- 172.21.15.16:0/3832826216 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f8fb8159360 con 0x7f8fb8140fe0 2023-05-20T01:26:18.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.901+0000 7f8faffff700 1 -- 172.21.15.16:0/3832826216 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v99) v1 ==== 92+0+10 (secure 0 0 0) 0x7f8fb8159360 con 0x7f8fb8140fe0 2023-05-20T01:26:18.906 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 8 2023-05-20T01:26:18.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.903+0000 7f8fbdfda700 1 -- 172.21.15.16:0/3832826216 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f8f9806ace0 msgr2=0x7f8f9806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:18.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.903+0000 7f8fbdfda700 1 --2- 172.21.15.16:0/3832826216 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f8f9806ace0 0x7f8f9806d190 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f8fa8009ce0 tx=0x7f8fa8009210 comp rx=0 tx=0).stop 2023-05-20T01:26:18.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.903+0000 7f8fbdfda700 1 -- 172.21.15.16:0/3832826216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8fb8140fe0 msgr2=0x7f8fb8147210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:18.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.903+0000 7f8fbdfda700 1 --2- 172.21.15.16:0/3832826216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8fb8140fe0 0x7f8fb8147210 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f8fa0000ed0 tx=0x7f8fb80ca8e0 comp rx=0 tx=0).stop 2023-05-20T01:26:18.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.903+0000 7f8fbdfda700 1 -- 172.21.15.16:0/3832826216 shutdown_connections 2023-05-20T01:26:18.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.903+0000 7f8fbdfda700 1 --2- 172.21.15.16:0/3832826216 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f8f9806ace0 0x7f8f9806d190 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:18.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.903+0000 7f8fbdfda700 1 --2- 172.21.15.16:0/3832826216 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8fb813be00 0x7f8fb813c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:18.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.903+0000 7f8fbdfda700 1 --2- 172.21.15.16:0/3832826216 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8fa4004850 0x7f8fb81452c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:18.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.903+0000 7f8fbdfda700 1 --2- 172.21.15.16:0/3832826216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8fb8140fe0 0x7f8fb8147210 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:18.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.903+0000 7f8fbdfda700 1 -- 172.21.15.16:0/3832826216 >> 172.21.15.16:0/3832826216 conn(0x7f8fb806cbb0 msgr2=0x7f8fb80cd950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:18.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.903+0000 7f8fbdfda700 1 -- 172.21.15.16:0/3832826216 shutdown_connections 2023-05-20T01:26:18.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:18.903+0000 7f8fbdfda700 1 -- 172.21.15.16:0/3832826216 wait complete. 2023-05-20T01:26:18.919 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-05-20T01:26:19.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.043+0000 7f52badbc700 1 Processor -- start 2023-05-20T01:26:19.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.043+0000 7f52badbc700 1 -- start start 2023-05-20T01:26:19.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.044+0000 7f52badbc700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f52b4159b50 0x7f52b406e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:19.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.044+0000 7f52badbc700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f52b40cd1c0 0x7f52b40710b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:19.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.044+0000 7f52badbc700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52b40ccd30 con 0x7f52b40cd630 2023-05-20T01:26:19.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.044+0000 7f52badbc700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52b40ce990 con 0x7f52b40cd1c0 2023-05-20T01:26:19.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.044+0000 7f52badbc700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52b40cc6e0 con 0x7f52b4159b50 2023-05-20T01:26:19.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.044+0000 7f52b3fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f52b40cd1c0 0x7f52b40710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:19.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.044+0000 7f52b9359700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f52b40cd630 0x7f52b40ca870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49106/0 (socket says 172.21.15.16:49106) 2023-05-20T01:26:19.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.044+0000 7f52b9359700 1 -- 172.21.15.16:0/1965423233 learned_addr learned my addr 172.21.15.16:0/1965423233 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:26:19.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.044+0000 7f52b8b58700 1 --2- 172.21.15.16:0/1965423233 >> v2:172.21.15.132:3300/0 conn(0x7f52b4159b50 0x7f52b406e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:19.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.044+0000 7f52b3fff700 1 -- 172.21.15.16:0/1965423233 >> v2:172.21.15.132:3300/0 conn(0x7f52b4159b50 msgr2=0x7f52b406e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:19.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.044+0000 7f52b3fff700 1 --2- 172.21.15.16:0/1965423233 >> v2:172.21.15.132:3300/0 conn(0x7f52b4159b50 0x7f52b406e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:19.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.044+0000 7f52b3fff700 1 -- 172.21.15.16:0/1965423233 >> v1:172.21.15.16:6789/0 conn(0x7f52b40cd630 legacy=0x7f52b40ca870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:19.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.044+0000 7f52b3fff700 1 -- 172.21.15.16:0/1965423233 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52b40cc9c0 con 0x7f52b40cd1c0 2023-05-20T01:26:19.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.044+0000 7f52b37fe700 1 -- 172.21.15.16:0/1965423233 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2059485783 0 0) 0x7f52b40ccd30 con 0x7f52b40cd630 2023-05-20T01:26:19.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.045+0000 7f52b3fff700 1 --2- 172.21.15.16:0/1965423233 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52b40cd1c0 0x7f52b40710b0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f52b4079820 tx=0x7f52a0009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20ed7031050c263d server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:19.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.045+0000 7f52b37fe700 1 -- 172.21.15.16:0/1965423233 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a0011830 con 0x7f52b40cd1c0 2023-05-20T01:26:19.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.045+0000 7f52b37fe700 1 -- 172.21.15.16:0/1965423233 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52a0011990 con 0x7f52b40cd1c0 2023-05-20T01:26:19.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.045+0000 7f52b37fe700 1 -- 172.21.15.16:0/1965423233 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a0011d20 con 0x7f52b40cd1c0 2023-05-20T01:26:19.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.045+0000 7f52badbc700 1 -- 172.21.15.16:0/1965423233 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52b40cd1c0 msgr2=0x7f52b40710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:19.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.045+0000 7f52badbc700 1 --2- 172.21.15.16:0/1965423233 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52b40cd1c0 0x7f52b40710b0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f52b4079820 tx=0x7f52a0009b20 comp rx=0 tx=0).stop 2023-05-20T01:26:19.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.045+0000 7f52badbc700 1 -- 172.21.15.16:0/1965423233 shutdown_connections 2023-05-20T01:26:19.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.045+0000 7f52badbc700 1 --2- 172.21.15.16:0/1965423233 >> v2:172.21.15.132:3300/0 conn(0x7f52b4159b50 0x7f52b406e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:19.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.045+0000 7f52badbc700 1 --2- 172.21.15.16:0/1965423233 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52b40cd1c0 0x7f52b40710b0 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:19.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.045+0000 7f52badbc700 1 -- 172.21.15.16:0/1965423233 >> 172.21.15.16:0/1965423233 conn(0x7f52b40cffd0 msgr2=0x7f52b40d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:19.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.045+0000 7f52badbc700 1 -- 172.21.15.16:0/1965423233 shutdown_connections 2023-05-20T01:26:19.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.045+0000 7f52badbc700 1 -- 172.21.15.16:0/1965423233 wait complete. 2023-05-20T01:26:19.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.047+0000 7f52badbc700 1 Processor -- start 2023-05-20T01:26:19.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.047+0000 7f52badbc700 1 -- start start 2023-05-20T01:26:19.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.047+0000 7f52badbc700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f52b40cd1c0 0x7f52b4155c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:19.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.047+0000 7f52badbc700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f52b40cd630 0x7f52b4158da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:19.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.048+0000 7f52badbc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52b4159b50 0x7f52b414a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:19.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.048+0000 7f52badbc700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52b414a930 con 0x7f52b4159b50 2023-05-20T01:26:19.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.048+0000 7f52badbc700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52b4156250 con 0x7f52b40cd630 2023-05-20T01:26:19.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.048+0000 7f52badbc700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52b4156530 con 0x7f52b40cd1c0 2023-05-20T01:26:19.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.048+0000 7f52b9359700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52b4159b50 0x7f52b414a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:19.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.048+0000 7f52b9359700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52b4159b50 0x7f52b414a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37584/0 (socket says 172.21.15.16:37584) 2023-05-20T01:26:19.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.048+0000 7f52b9359700 1 -- 172.21.15.16:0/1378887121 learned_addr learned my addr 172.21.15.16:0/1378887121 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:26:19.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.048+0000 7f52b8b58700 1 --2- 172.21.15.16:0/1378887121 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f52b40cd1c0 0x7f52b4155c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:19.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.048+0000 7f52b3fff700 1 --2- 172.21.15.16:0/1378887121 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f52b40cd630 0x7f52b4158da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:19.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.048+0000 7f52b9359700 1 -- 172.21.15.16:0/1378887121 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f52b40cd1c0 msgr2=0x7f52b4155c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:19.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.048+0000 7f52b9359700 1 --2- 172.21.15.16:0/1378887121 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f52b40cd1c0 0x7f52b4155c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:19.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.048+0000 7f52b9359700 1 -- 172.21.15.16:0/1378887121 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f52b40cd630 msgr2=0x7f52b4158da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:19.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.048+0000 7f52b9359700 1 --2- 172.21.15.16:0/1378887121 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f52b40cd630 0x7f52b4158da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:19.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.048+0000 7f52b9359700 1 -- 172.21.15.16:0/1378887121 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52a00094a0 con 0x7f52b4159b50 2023-05-20T01:26:19.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.048+0000 7f52b9359700 1 --2- 172.21.15.16:0/1378887121 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52b4159b50 0x7f52b414a3f0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f52ac00cc70 tx=0x7f52ac00cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:19.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.048+0000 7f52b17fa700 1 -- 172.21.15.16:0/1378887121 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52ac00aac0 con 0x7f52b4159b50 2023-05-20T01:26:19.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.048+0000 7f52b17fa700 1 -- 172.21.15.16:0/1378887121 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52ac00ac40 con 0x7f52b4159b50 2023-05-20T01:26:19.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.048+0000 7f52badbc700 1 -- 172.21.15.16:0/1378887121 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52b4071e80 con 0x7f52b4159b50 2023-05-20T01:26:19.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.048+0000 7f52b17fa700 1 -- 172.21.15.16:0/1378887121 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52ac0070d0 con 0x7f52b4159b50 2023-05-20T01:26:19.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.048+0000 7f52badbc700 1 -- 172.21.15.16:0/1378887121 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52b406e5e0 con 0x7f52b4159b50 2023-05-20T01:26:19.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.049+0000 7f52badbc700 1 -- 172.21.15.16:0/1378887121 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52b40cec70 con 0x7f52b4159b50 2023-05-20T01:26:19.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.050+0000 7f52b17fa700 1 -- 172.21.15.16:0/1378887121 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f52ac007250 con 0x7f52b4159b50 2023-05-20T01:26:19.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.050+0000 7f52b17fa700 1 --2- 172.21.15.16:0/1378887121 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f529806ace0 0x7f529806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:19.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.050+0000 7f52b17fa700 1 -- 172.21.15.16:0/1378887121 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 7102+0+0 (secure 0 0 0) 0x7f52ac089670 con 0x7f52b4159b50 2023-05-20T01:26:19.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.050+0000 7f52b8b58700 1 --2- 172.21.15.16:0/1378887121 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f529806ace0 0x7f529806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:19.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.051+0000 7f52b8b58700 1 --2- 172.21.15.16:0/1378887121 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f529806ace0 0x7f529806d190 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f52a800ab40 tx=0x7f52a8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:19.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.055+0000 7f52b17fa700 1 -- 172.21.15.16:0/1378887121 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f52ac054310 con 0x7f52b4159b50 2023-05-20T01:26:19.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.253+0000 7f52badbc700 1 -- 172.21.15.16:0/1378887121 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f52b40747b0 con 0x7f52b4159b50 2023-05-20T01:26:19.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.253+0000 7f52b17fa700 1 -- 172.21.15.16:0/1378887121 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v99) v1 ==== 93+0+10 (secure 0 0 0) 0x7f52ac057930 con 0x7f52b4159b50 2023-05-20T01:26:19.258 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-05-20T01:26:19.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.255+0000 7f52badbc700 1 -- 172.21.15.16:0/1378887121 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f529806ace0 msgr2=0x7f529806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:19.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.255+0000 7f52badbc700 1 --2- 172.21.15.16:0/1378887121 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f529806ace0 0x7f529806d190 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f52a800ab40 tx=0x7f52a8008040 comp rx=0 tx=0).stop 2023-05-20T01:26:19.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.255+0000 7f52badbc700 1 -- 172.21.15.16:0/1378887121 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52b4159b50 msgr2=0x7f52b414a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:19.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.255+0000 7f52badbc700 1 --2- 172.21.15.16:0/1378887121 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52b4159b50 0x7f52b414a3f0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f52ac00cc70 tx=0x7f52ac00cf80 comp rx=0 tx=0).stop 2023-05-20T01:26:19.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.255+0000 7f52badbc700 1 -- 172.21.15.16:0/1378887121 shutdown_connections 2023-05-20T01:26:19.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.255+0000 7f52badbc700 1 --2- 172.21.15.16:0/1378887121 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52b4159b50 0x7f52b414a3f0 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:19.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.255+0000 7f52badbc700 1 --2- 172.21.15.16:0/1378887121 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f529806ace0 0x7f529806d190 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:19.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.255+0000 7f52badbc700 1 --2- 172.21.15.16:0/1378887121 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f52b40cd1c0 0x7f52b4155c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:19.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.255+0000 7f52badbc700 1 --2- 172.21.15.16:0/1378887121 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f52b40cd630 0x7f52b4158da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:19.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.255+0000 7f52badbc700 1 -- 172.21.15.16:0/1378887121 >> 172.21.15.16:0/1378887121 conn(0x7f52b40cffd0 msgr2=0x7f52b40d03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:19.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.255+0000 7f52badbc700 1 -- 172.21.15.16:0/1378887121 shutdown_connections 2023-05-20T01:26:19.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.255+0000 7f52badbc700 1 -- 172.21.15.16:0/1378887121 wait complete. 2023-05-20T01:26:19.271 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-05-20T01:26:19.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.396+0000 7f2567610700 1 Processor -- start 2023-05-20T01:26:19.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.396+0000 7f2567610700 1 -- start start 2023-05-20T01:26:19.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.397+0000 7f2567610700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2560073dc0 0x7f25600cc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:19.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.397+0000 7f2567610700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f256015aea0 con 0x7f256015aad0 2023-05-20T01:26:19.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.397+0000 7f2567610700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25600b2ee0 con 0x7f25600b1e20 2023-05-20T01:26:19.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.397+0000 7f2567610700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2560070930 con 0x7f2560073dc0 2023-05-20T01:26:19.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.397+0000 7f2564bab700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2560073dc0 0x7f25600cc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:19.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.397+0000 7f25653ac700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f256015aad0 0x7f25600b27d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49116/0 (socket says 172.21.15.16:49116) 2023-05-20T01:26:19.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.397+0000 7f25653ac700 1 -- 172.21.15.16:0/565302979 learned_addr learned my addr 172.21.15.16:0/565302979 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:26:19.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.397+0000 7f2564bab700 1 -- 172.21.15.16:0/565302979 >> v1:172.21.15.132:6789/0 conn(0x7f25600b1e20 legacy=0x7f25600716d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:19.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.397+0000 7f2564bab700 1 -- 172.21.15.16:0/565302979 >> v1:172.21.15.16:6789/0 conn(0x7f256015aad0 legacy=0x7f25600b27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:19.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.397+0000 7f2564bab700 1 -- 172.21.15.16:0/565302979 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25600b1b00 con 0x7f2560073dc0 2023-05-20T01:26:19.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.397+0000 7f2564bab700 1 --2- 172.21.15.16:0/565302979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2560073dc0 0x7f25600cc0c0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f2550009840 tx=0x7f2550009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2b659bc98a4be34 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:19.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.397+0000 7f2557fff700 1 -- 172.21.15.16:0/565302979 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2550004230 con 0x7f2560073dc0 2023-05-20T01:26:19.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.397+0000 7f2557fff700 1 -- 172.21.15.16:0/565302979 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2550004390 con 0x7f2560073dc0 2023-05-20T01:26:19.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.397+0000 7f2557fff700 1 -- 172.21.15.16:0/565302979 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25500036a0 con 0x7f2560073dc0 2023-05-20T01:26:19.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.398+0000 7f2567610700 1 -- 172.21.15.16:0/565302979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2560073dc0 msgr2=0x7f25600cc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:19.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.398+0000 7f2567610700 1 --2- 172.21.15.16:0/565302979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2560073dc0 0x7f25600cc0c0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f2550009840 tx=0x7f2550009b50 comp rx=0 tx=0).stop 2023-05-20T01:26:19.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.398+0000 7f2567610700 1 -- 172.21.15.16:0/565302979 shutdown_connections 2023-05-20T01:26:19.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.398+0000 7f2567610700 1 --2- 172.21.15.16:0/565302979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2560073dc0 0x7f25600cc0c0 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:19.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.398+0000 7f2567610700 1 -- 172.21.15.16:0/565302979 >> 172.21.15.16:0/565302979 conn(0x7f256006cbb0 msgr2=0x7f256006cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:19.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.398+0000 7f2567610700 1 -- 172.21.15.16:0/565302979 shutdown_connections 2023-05-20T01:26:19.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.398+0000 7f2567610700 1 -- 172.21.15.16:0/565302979 wait complete. 2023-05-20T01:26:19.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.399+0000 7f2567610700 1 Processor -- start 2023-05-20T01:26:19.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.399+0000 7f2567610700 1 -- start start 2023-05-20T01:26:19.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.400+0000 7f2567610700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2560073dc0 0x7f25600c6d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:19.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.400+0000 7f2567610700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f25600b1e20 0x7f256015b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:19.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.400+0000 7f2567610700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f256015aad0 0x7f25600ce350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:19.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.400+0000 7f2567610700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25600c83a0 con 0x7f256015aad0 2023-05-20T01:26:19.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.400+0000 7f2567610700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25600cd380 con 0x7f2560073dc0 2023-05-20T01:26:19.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.400+0000 7f2567610700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25600cd590 con 0x7f25600b1e20 2023-05-20T01:26:19.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.400+0000 7f2565bad700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f256015aad0 0x7f25600ce350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:19.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.400+0000 7f2565bad700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f256015aad0 0x7f25600ce350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37598/0 (socket says 172.21.15.16:37598) 2023-05-20T01:26:19.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.400+0000 7f2565bad700 1 -- 172.21.15.16:0/2831008670 learned_addr learned my addr 172.21.15.16:0/2831008670 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:26:19.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.400+0000 7f2564bab700 1 --2- 172.21.15.16:0/2831008670 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f25600b1e20 0x7f256015b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:19.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.400+0000 7f25653ac700 1 --2- 172.21.15.16:0/2831008670 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2560073dc0 0x7f25600c6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:19.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.400+0000 7f2565bad700 1 -- 172.21.15.16:0/2831008670 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f25600b1e20 msgr2=0x7f256015b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:19.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.400+0000 7f2565bad700 1 --2- 172.21.15.16:0/2831008670 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f25600b1e20 0x7f256015b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:19.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.400+0000 7f2565bad700 1 -- 172.21.15.16:0/2831008670 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2560073dc0 msgr2=0x7f25600c6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:19.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.400+0000 7f2565bad700 1 --2- 172.21.15.16:0/2831008670 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2560073dc0 0x7f25600c6d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:19.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.400+0000 7f2565bad700 1 -- 172.21.15.16:0/2831008670 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25500094a0 con 0x7f256015aad0 2023-05-20T01:26:19.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.400+0000 7f2564bab700 1 --2- 172.21.15.16:0/2831008670 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f25600b1e20 0x7f256015b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:26:19.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.400+0000 7f25653ac700 1 --2- 172.21.15.16:0/2831008670 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2560073dc0 0x7f25600c6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:26:19.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.401+0000 7f2565bad700 1 --2- 172.21.15.16:0/2831008670 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f256015aad0 0x7f25600ce350 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f255800c7a0 tx=0x7f255800cb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:19.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.401+0000 7f2555ffb700 1 -- 172.21.15.16:0/2831008670 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2558003e10 con 0x7f256015aad0 2023-05-20T01:26:19.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.401+0000 7f2555ffb700 1 -- 172.21.15.16:0/2831008670 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2558005070 con 0x7f256015aad0 2023-05-20T01:26:19.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.401+0000 7f2555ffb700 1 -- 172.21.15.16:0/2831008670 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2558005420 con 0x7f256015aad0 2023-05-20T01:26:19.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.401+0000 7f2567610700 1 -- 172.21.15.16:0/2831008670 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2560070ed0 con 0x7f256015aad0 2023-05-20T01:26:19.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.401+0000 7f2567610700 1 -- 172.21.15.16:0/2831008670 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2560070bf0 con 0x7f256015aad0 2023-05-20T01:26:19.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.402+0000 7f2567610700 1 -- 172.21.15.16:0/2831008670 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25600711b0 con 0x7f256015aad0 2023-05-20T01:26:19.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.402+0000 7f2555ffb700 1 -- 172.21.15.16:0/2831008670 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2558005580 con 0x7f256015aad0 2023-05-20T01:26:19.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.403+0000 7f2555ffb700 1 --2- 172.21.15.16:0/2831008670 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f254406ace0 0x7f254406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:19.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.403+0000 7f25653ac700 1 --2- 172.21.15.16:0/2831008670 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f254406ace0 0x7f254406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:19.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.403+0000 7f2555ffb700 1 -- 172.21.15.16:0/2831008670 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 7102+0+0 (secure 0 0 0) 0x7f2558088c60 con 0x7f256015aad0 2023-05-20T01:26:19.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.403+0000 7f25653ac700 1 --2- 172.21.15.16:0/2831008670 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f254406ace0 0x7f254406d190 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f255c000f60 tx=0x7f255c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:19.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.407+0000 7f2555ffb700 1 -- 172.21.15.16:0/2831008670 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f2558053900 con 0x7f256015aad0 2023-05-20T01:26:19.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.604+0000 7f2567610700 1 -- 172.21.15.16:0/2831008670 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f25600ce1a0 con 0x7f256015aad0 2023-05-20T01:26:19.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.604+0000 7f2555ffb700 1 -- 172.21.15.16:0/2831008670 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v99) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2558017020 con 0x7f256015aad0 2023-05-20T01:26:19.609 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-05-20T01:26:19.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.606+0000 7f2567610700 1 -- 172.21.15.16:0/2831008670 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f254406ace0 msgr2=0x7f254406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:19.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.606+0000 7f2567610700 1 --2- 172.21.15.16:0/2831008670 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f254406ace0 0x7f254406d190 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f255c000f60 tx=0x7f255c009040 comp rx=0 tx=0).stop 2023-05-20T01:26:19.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.606+0000 7f2567610700 1 -- 172.21.15.16:0/2831008670 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f256015aad0 msgr2=0x7f25600ce350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:19.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.606+0000 7f2567610700 1 --2- 172.21.15.16:0/2831008670 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f256015aad0 0x7f25600ce350 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f255800c7a0 tx=0x7f255800cb60 comp rx=0 tx=0).stop 2023-05-20T01:26:19.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.606+0000 7f2567610700 1 -- 172.21.15.16:0/2831008670 shutdown_connections 2023-05-20T01:26:19.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.606+0000 7f2567610700 1 --2- 172.21.15.16:0/2831008670 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f256015aad0 0x7f25600ce350 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:19.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.606+0000 7f2567610700 1 --2- 172.21.15.16:0/2831008670 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2560073dc0 0x7f25600c6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:19.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.606+0000 7f2567610700 1 --2- 172.21.15.16:0/2831008670 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f254406ace0 0x7f254406d190 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:19.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.606+0000 7f2567610700 1 --2- 172.21.15.16:0/2831008670 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f25600b1e20 0x7f256015b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:19.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.606+0000 7f2567610700 1 -- 172.21.15.16:0/2831008670 >> 172.21.15.16:0/2831008670 conn(0x7f256006cbb0 msgr2=0x7f25600c9de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:19.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.606+0000 7f2567610700 1 -- 172.21.15.16:0/2831008670 shutdown_connections 2023-05-20T01:26:19.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.606+0000 7f2567610700 1 -- 172.21.15.16:0/2831008670 wait complete. 2023-05-20T01:26:19.623 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-05-20T01:26:19.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.746+0000 7f8c2bfbd700 1 Processor -- start 2023-05-20T01:26:19.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.747+0000 7f8c2bfbd700 1 -- start start 2023-05-20T01:26:19.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.747+0000 7f8c2bfbd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8c240cb700 0x7f8c24159e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:19.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.747+0000 7f8c2bfbd700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8c24071e10 0x7f8c2415da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:19.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.747+0000 7f8c2bfbd700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c2415d5b0 con 0x7f8c24157450 2023-05-20T01:26:19.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.747+0000 7f8c2bfbd700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c24157820 con 0x7f8c240cb700 2023-05-20T01:26:19.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.747+0000 7f8c2bfbd700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c2415a4c0 con 0x7f8c24071e10 2023-05-20T01:26:19.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.747+0000 7f8c29558700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8c240cb700 0x7f8c24159e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:19.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.748+0000 7f8c29d59700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f8c24157450 0x7f8c240caff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49130/0 (socket says 172.21.15.16:49130) 2023-05-20T01:26:19.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.748+0000 7f8c29558700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8c240cb700 0x7f8c24159e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37604/0 (socket says 172.21.15.16:37604) 2023-05-20T01:26:19.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.748+0000 7f8c29d59700 1 -- 172.21.15.16:0/1969994963 learned_addr learned my addr 172.21.15.16:0/1969994963 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:26:19.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.748+0000 7f8c2a55a700 1 --2- 172.21.15.16:0/1969994963 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8c24071e10 0x7f8c2415da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:19.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.748+0000 7f8c29558700 1 -- 172.21.15.16:0/1969994963 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8c24071e10 msgr2=0x7f8c2415da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:19.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.748+0000 7f8c29558700 1 --2- 172.21.15.16:0/1969994963 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8c24071e10 0x7f8c2415da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:19.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.748+0000 7f8c29558700 1 -- 172.21.15.16:0/1969994963 >> v1:172.21.15.16:6789/0 conn(0x7f8c24157450 legacy=0x7f8c240caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:19.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.748+0000 7f8c29558700 1 -- 172.21.15.16:0/1969994963 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c2415a7a0 con 0x7f8c240cb700 2023-05-20T01:26:19.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.748+0000 7f8c28d57700 1 -- 172.21.15.16:0/1969994963 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4212461327 0 0) 0x7f8c2415d5b0 con 0x7f8c24157450 2023-05-20T01:26:19.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.748+0000 7f8c2a55a700 1 --2- 172.21.15.16:0/1969994963 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8c24071e10 0x7f8c2415da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:26:19.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.748+0000 7f8c29558700 1 --2- 172.21.15.16:0/1969994963 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c240cb700 0x7f8c24159e40 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f8c18009840 tx=0x7f8c18009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1de071ca7d4fb16 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:19.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.748+0000 7f8c28d57700 1 -- 172.21.15.16:0/1969994963 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c18011830 con 0x7f8c240cb700 2023-05-20T01:26:19.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.748+0000 7f8c28d57700 1 -- 172.21.15.16:0/1969994963 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c18011990 con 0x7f8c240cb700 2023-05-20T01:26:19.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.748+0000 7f8c28d57700 1 -- 172.21.15.16:0/1969994963 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c18011d20 con 0x7f8c240cb700 2023-05-20T01:26:19.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.749+0000 7f8c2bfbd700 1 -- 172.21.15.16:0/1969994963 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c240cb700 msgr2=0x7f8c24159e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:19.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.749+0000 7f8c2bfbd700 1 --2- 172.21.15.16:0/1969994963 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c240cb700 0x7f8c24159e40 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f8c18009840 tx=0x7f8c18009b50 comp rx=0 tx=0).stop 2023-05-20T01:26:19.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.749+0000 7f8c2bfbd700 1 -- 172.21.15.16:0/1969994963 shutdown_connections 2023-05-20T01:26:19.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.749+0000 7f8c2bfbd700 1 --2- 172.21.15.16:0/1969994963 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8c24071e10 0x7f8c2415da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:19.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.749+0000 7f8c2bfbd700 1 --2- 172.21.15.16:0/1969994963 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c240cb700 0x7f8c24159e40 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:19.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.749+0000 7f8c2bfbd700 1 -- 172.21.15.16:0/1969994963 >> 172.21.15.16:0/1969994963 conn(0x7f8c2406cbb0 msgr2=0x7f8c2406cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:19.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.749+0000 7f8c2bfbd700 1 -- 172.21.15.16:0/1969994963 shutdown_connections 2023-05-20T01:26:19.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.749+0000 7f8c2bfbd700 1 -- 172.21.15.16:0/1969994963 wait complete. 2023-05-20T01:26:19.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.750+0000 7f8c2bfbd700 1 Processor -- start 2023-05-20T01:26:19.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.751+0000 7f8c2bfbd700 1 -- start start 2023-05-20T01:26:19.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.751+0000 7f8c2bfbd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c24071e10 0x7f8c240c9370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:19.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.751+0000 7f8c2bfbd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c240cb700 0x7f8c24158fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:19.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.751+0000 7f8c2bfbd700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8c24157450 0x7f8c241655d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:19.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.751+0000 7f8c2bfbd700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c240c6880 con 0x7f8c240cb700 2023-05-20T01:26:19.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.751+0000 7f8c2bfbd700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c240c6b60 con 0x7f8c24071e10 2023-05-20T01:26:19.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.751+0000 7f8c2bfbd700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c240c6e40 con 0x7f8c24157450 2023-05-20T01:26:19.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.752+0000 7f8c29558700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c240cb700 0x7f8c24158fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:19.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.752+0000 7f8c29558700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c240cb700 0x7f8c24158fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37610/0 (socket says 172.21.15.16:37610) 2023-05-20T01:26:19.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.752+0000 7f8c29558700 1 -- 172.21.15.16:0/395468445 learned_addr learned my addr 172.21.15.16:0/395468445 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:26:19.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.752+0000 7f8c29d59700 1 --2- 172.21.15.16:0/395468445 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c24071e10 0x7f8c240c9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:19.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.752+0000 7f8c2a55a700 1 --2- 172.21.15.16:0/395468445 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8c24157450 0x7f8c241655d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:19.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.752+0000 7f8c29558700 1 -- 172.21.15.16:0/395468445 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8c24157450 msgr2=0x7f8c241655d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:19.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.752+0000 7f8c29558700 1 --2- 172.21.15.16:0/395468445 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8c24157450 0x7f8c241655d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:19.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.752+0000 7f8c29558700 1 -- 172.21.15.16:0/395468445 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c24071e10 msgr2=0x7f8c240c9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:19.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.752+0000 7f8c29558700 1 --2- 172.21.15.16:0/395468445 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c24071e10 0x7f8c240c9370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:19.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.752+0000 7f8c29558700 1 -- 172.21.15.16:0/395468445 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c180094a0 con 0x7f8c240cb700 2023-05-20T01:26:19.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.752+0000 7f8c2a55a700 1 --2- 172.21.15.16:0/395468445 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8c24157450 0x7f8c241655d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:26:19.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.752+0000 7f8c29d59700 1 --2- 172.21.15.16:0/395468445 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c24071e10 0x7f8c240c9370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:26:19.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.752+0000 7f8c29558700 1 --2- 172.21.15.16:0/395468445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c240cb700 0x7f8c24158fa0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f8c180090b0 tx=0x7f8c18004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:19.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.752+0000 7f8c167fc700 1 -- 172.21.15.16:0/395468445 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c180038a0 con 0x7f8c240cb700 2023-05-20T01:26:19.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.752+0000 7f8c167fc700 1 -- 172.21.15.16:0/395468445 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c1801f070 con 0x7f8c240cb700 2023-05-20T01:26:19.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.752+0000 7f8c2bfbd700 1 -- 172.21.15.16:0/395468445 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c2415ad60 con 0x7f8c240cb700 2023-05-20T01:26:19.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.752+0000 7f8c167fc700 1 -- 172.21.15.16:0/395468445 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c18003c20 con 0x7f8c240cb700 2023-05-20T01:26:19.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.752+0000 7f8c2bfbd700 1 -- 172.21.15.16:0/395468445 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c2415aa80 con 0x7f8c240cb700 2023-05-20T01:26:19.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.753+0000 7f8c2bfbd700 1 -- 172.21.15.16:0/395468445 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c24073b00 con 0x7f8c240cb700 2023-05-20T01:26:19.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.754+0000 7f8c167fc700 1 -- 172.21.15.16:0/395468445 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8c1801a430 con 0x7f8c240cb700 2023-05-20T01:26:19.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.754+0000 7f8c167fc700 1 --2- 172.21.15.16:0/395468445 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f8c0806ace0 0x7f8c0806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:19.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.754+0000 7f8c167fc700 1 -- 172.21.15.16:0/395468445 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 7102+0+0 (secure 0 0 0) 0x7f8c18021070 con 0x7f8c240cb700 2023-05-20T01:26:19.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.754+0000 7f8c29d59700 1 --2- 172.21.15.16:0/395468445 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f8c0806ace0 0x7f8c0806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:19.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.754+0000 7f8c29d59700 1 --2- 172.21.15.16:0/395468445 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f8c0806ace0 0x7f8c0806d190 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f8c20000f60 tx=0x7f8c20007cc0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:19.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.759+0000 7f8c167fc700 1 -- 172.21.15.16:0/395468445 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f8c18055fb0 con 0x7f8c240cb700 2023-05-20T01:26:19.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.957+0000 7f8c2bfbd700 1 -- 172.21.15.16:0/395468445 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8c240747b0 con 0x7f8c240cb700 2023-05-20T01:26:19.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.957+0000 7f8c167fc700 1 -- 172.21.15.16:0/395468445 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v99) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8c180595d0 con 0x7f8c240cb700 2023-05-20T01:26:19.962 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2023-05-20T01:26:19.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.959+0000 7f8c2bfbd700 1 -- 172.21.15.16:0/395468445 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f8c0806ace0 msgr2=0x7f8c0806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:19.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.959+0000 7f8c2bfbd700 1 --2- 172.21.15.16:0/395468445 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f8c0806ace0 0x7f8c0806d190 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f8c20000f60 tx=0x7f8c20007cc0 comp rx=0 tx=0).stop 2023-05-20T01:26:19.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.959+0000 7f8c2bfbd700 1 -- 172.21.15.16:0/395468445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c240cb700 msgr2=0x7f8c24158fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:19.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.959+0000 7f8c2bfbd700 1 --2- 172.21.15.16:0/395468445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c240cb700 0x7f8c24158fa0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f8c180090b0 tx=0x7f8c18004750 comp rx=0 tx=0).stop 2023-05-20T01:26:19.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.959+0000 7f8c2bfbd700 1 -- 172.21.15.16:0/395468445 shutdown_connections 2023-05-20T01:26:19.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.959+0000 7f8c2bfbd700 1 --2- 172.21.15.16:0/395468445 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f8c0806ace0 0x7f8c0806d190 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:19.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.959+0000 7f8c2bfbd700 1 --2- 172.21.15.16:0/395468445 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8c24157450 0x7f8c241655d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:19.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.959+0000 7f8c2bfbd700 1 --2- 172.21.15.16:0/395468445 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c24071e10 0x7f8c240c9370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:19.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.959+0000 7f8c2bfbd700 1 --2- 172.21.15.16:0/395468445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c240cb700 0x7f8c24158fa0 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:19.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.959+0000 7f8c2bfbd700 1 -- 172.21.15.16:0/395468445 >> 172.21.15.16:0/395468445 conn(0x7f8c2406cbb0 msgr2=0x7f8c240c5ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:19.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.959+0000 7f8c2bfbd700 1 -- 172.21.15.16:0/395468445 shutdown_connections 2023-05-20T01:26:19.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:19.959+0000 7f8c2bfbd700 1 -- 172.21.15.16:0/395468445 wait complete. 2023-05-20T01:26:19.975 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_extra pg_num 2023-05-20T01:26:20.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.098+0000 7ff7083b3700 1 Processor -- start 2023-05-20T01:26:20.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.098+0000 7ff7083b3700 1 -- start start 2023-05-20T01:26:20.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.099+0000 7ff7083b3700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7ff70015ab80 0x7ff7000cda80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:20.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.099+0000 7ff7083b3700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff7000cdfc0 0x7ff7000cec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:20.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.099+0000 7ff7083b3700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7000ce4d0 con 0x7ff7000cf1c0 2023-05-20T01:26:20.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.099+0000 7ff7083b3700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7000cb700 con 0x7ff70015ab80 2023-05-20T01:26:20.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.099+0000 7ff7083b3700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7000cf640 con 0x7ff7000cdfc0 2023-05-20T01:26:20.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.099+0000 7ff706950700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ff7000cf1c0 0x7ff70015e4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49146/0 (socket says 172.21.15.16:49146) 2023-05-20T01:26:20.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.099+0000 7ff706950700 1 -- 172.21.15.16:0/324062091 learned_addr learned my addr 172.21.15.16:0/324062091 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:26:20.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.099+0000 7ff70594e700 1 --2- 172.21.15.16:0/324062091 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff7000cdfc0 0x7ff7000cec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:20.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.100+0000 7ff70614f700 1 --2- 172.21.15.16:0/324062091 >> v2:172.21.15.132:3300/0 conn(0x7ff70015ab80 0x7ff7000cda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:20.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.100+0000 7ff70514d700 1 -- 172.21.15.16:0/324062091 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3569753664 0 0) 0x7ff7000ce4d0 con 0x7ff7000cf1c0 2023-05-20T01:26:20.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.100+0000 7ff70514d700 1 -- 172.21.15.16:0/324062091 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff6ec0035b0 con 0x7ff7000cf1c0 2023-05-20T01:26:20.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.100+0000 7ff70514d700 1 -- 172.21.15.16:0/324062091 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2670958601 0 0) 0x7ff6ec0035b0 con 0x7ff7000cf1c0 2023-05-20T01:26:20.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.100+0000 7ff70594e700 1 -- 172.21.15.16:0/324062091 >> v2:172.21.15.132:3300/0 conn(0x7ff70015ab80 msgr2=0x7ff7000cda80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:20.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.100+0000 7ff70594e700 1 --2- 172.21.15.16:0/324062091 >> v2:172.21.15.132:3300/0 conn(0x7ff70015ab80 0x7ff7000cda80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:20.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.100+0000 7ff70594e700 1 -- 172.21.15.16:0/324062091 >> v1:172.21.15.16:6789/0 conn(0x7ff7000cf1c0 legacy=0x7ff70015e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:20.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.100+0000 7ff70594e700 1 -- 172.21.15.16:0/324062091 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7000ce7b0 con 0x7ff7000cdfc0 2023-05-20T01:26:20.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.100+0000 7ff70614f700 1 --2- 172.21.15.16:0/324062091 >> v2:172.21.15.132:3300/0 conn(0x7ff70015ab80 0x7ff7000cda80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-05-20T01:26:20.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.100+0000 7ff70594e700 1 --2- 172.21.15.16:0/324062091 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff7000cdfc0 0x7ff7000cec80 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc009840 tx=0x7ff6fc009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4911d5713f61d736 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:20.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.100+0000 7ff70514d700 1 -- 172.21.15.16:0/324062091 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f80019f0 con 0x7ff7000cdfc0 2023-05-20T01:26:20.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.100+0000 7ff70514d700 1 -- 172.21.15.16:0/324062091 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6fc004030 con 0x7ff7000cdfc0 2023-05-20T01:26:20.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.100+0000 7ff70514d700 1 -- 172.21.15.16:0/324062091 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6fc0043c0 con 0x7ff7000cdfc0 2023-05-20T01:26:20.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.101+0000 7ff7083b3700 1 -- 172.21.15.16:0/324062091 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff7000cdfc0 msgr2=0x7ff7000cec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:20.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.101+0000 7ff7083b3700 1 --2- 172.21.15.16:0/324062091 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff7000cdfc0 0x7ff7000cec80 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc009840 tx=0x7ff6fc009b50 comp rx=0 tx=0).stop 2023-05-20T01:26:20.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.101+0000 7ff7083b3700 1 -- 172.21.15.16:0/324062091 shutdown_connections 2023-05-20T01:26:20.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.101+0000 7ff7083b3700 1 --2- 172.21.15.16:0/324062091 >> v2:172.21.15.132:3300/0 conn(0x7ff70015ab80 0x7ff7000cda80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:20.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.101+0000 7ff7083b3700 1 --2- 172.21.15.16:0/324062091 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff7000cdfc0 0x7ff7000cec80 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:20.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.101+0000 7ff7083b3700 1 -- 172.21.15.16:0/324062091 >> 172.21.15.16:0/324062091 conn(0x7ff70006ae30 msgr2=0x7ff70009a400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:20.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.101+0000 7ff7083b3700 1 -- 172.21.15.16:0/324062091 shutdown_connections 2023-05-20T01:26:20.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.101+0000 7ff7083b3700 1 -- 172.21.15.16:0/324062091 wait complete. 2023-05-20T01:26:20.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.102+0000 7ff7083b3700 1 Processor -- start 2023-05-20T01:26:20.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.102+0000 7ff7083b3700 1 -- start start 2023-05-20T01:26:20.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.103+0000 7ff7083b3700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff7000cdfc0 0x7ff7000c8180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:20.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.103+0000 7ff7083b3700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff7000cf1c0 0x7ff7000c73e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:20.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.103+0000 7ff7083b3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff70015ab80 0x7ff7000c9b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:20.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.103+0000 7ff7083b3700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7000c4be0 con 0x7ff70015ab80 2023-05-20T01:26:20.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.103+0000 7ff7083b3700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7001e47d0 con 0x7ff7000cf1c0 2023-05-20T01:26:20.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.103+0000 7ff7083b3700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff70006cc10 con 0x7ff7000cdfc0 2023-05-20T01:26:20.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.103+0000 7ff706950700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff70015ab80 0x7ff7000c9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:20.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.103+0000 7ff706950700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff70015ab80 0x7ff7000c9b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37624/0 (socket says 172.21.15.16:37624) 2023-05-20T01:26:20.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.103+0000 7ff706950700 1 -- 172.21.15.16:0/3990844934 learned_addr learned my addr 172.21.15.16:0/3990844934 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:26:20.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.103+0000 7ff70614f700 1 --2- 172.21.15.16:0/3990844934 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff7000cdfc0 0x7ff7000c8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:20.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.103+0000 7ff70594e700 1 --2- 172.21.15.16:0/3990844934 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff7000cf1c0 0x7ff7000c73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:20.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.103+0000 7ff706950700 1 -- 172.21.15.16:0/3990844934 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff7000cdfc0 msgr2=0x7ff7000c8180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:20.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.103+0000 7ff706950700 1 --2- 172.21.15.16:0/3990844934 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff7000cdfc0 0x7ff7000c8180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:20.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.103+0000 7ff706950700 1 -- 172.21.15.16:0/3990844934 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff7000cf1c0 msgr2=0x7ff7000c73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:20.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.103+0000 7ff706950700 1 --2- 172.21.15.16:0/3990844934 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff7000cf1c0 0x7ff7000c73e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:20.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.103+0000 7ff706950700 1 -- 172.21.15.16:0/3990844934 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6f00094a0 con 0x7ff70015ab80 2023-05-20T01:26:20.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.103+0000 7ff70594e700 1 --2- 172.21.15.16:0/3990844934 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff7000cf1c0 0x7ff7000c73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:26:20.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.104+0000 7ff706950700 1 --2- 172.21.15.16:0/3990844934 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff70015ab80 0x7ff7000c9b70 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7ff6f800cc90 tx=0x7ff6f800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:20.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.104+0000 7ff6f6ffd700 1 -- 172.21.15.16:0/3990844934 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f800ada0 con 0x7ff70015ab80 2023-05-20T01:26:20.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.104+0000 7ff6f6ffd700 1 -- 172.21.15.16:0/3990844934 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6f800ce90 con 0x7ff70015ab80 2023-05-20T01:26:20.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.104+0000 7ff7083b3700 1 -- 172.21.15.16:0/3990844934 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6fc0094a0 con 0x7ff70015ab80 2023-05-20T01:26:20.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.104+0000 7ff6f6ffd700 1 -- 172.21.15.16:0/3990844934 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f8007310 con 0x7ff70015ab80 2023-05-20T01:26:20.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.104+0000 7ff7083b3700 1 -- 172.21.15.16:0/3990844934 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7000cbbf0 con 0x7ff70015ab80 2023-05-20T01:26:20.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.105+0000 7ff7083b3700 1 -- 172.21.15.16:0/3990844934 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7000cb910 con 0x7ff70015ab80 2023-05-20T01:26:20.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.105+0000 7ff6f6ffd700 1 -- 172.21.15.16:0/3990844934 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff6f8007470 con 0x7ff70015ab80 2023-05-20T01:26:20.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.105+0000 7ff6f6ffd700 1 --2- 172.21.15.16:0/3990844934 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff6e406ace0 0x7ff6e406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:20.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.106+0000 7ff6f6ffd700 1 -- 172.21.15.16:0/3990844934 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 7102+0+0 (secure 0 0 0) 0x7ff6f80898a0 con 0x7ff70015ab80 2023-05-20T01:26:20.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.106+0000 7ff70614f700 1 --2- 172.21.15.16:0/3990844934 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff6e406ace0 0x7ff6e406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:20.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.106+0000 7ff70614f700 1 --2- 172.21.15.16:0/3990844934 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff6e406ace0 0x7ff6e406d190 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7ff6f0009d00 tx=0x7ff6f0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:20.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.110+0000 7ff6f6ffd700 1 -- 172.21.15.16:0/3990844934 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7ff6f8054540 con 0x7ff70015ab80 2023-05-20T01:26:20.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.306+0000 7ff7083b3700 1 -- 172.21.15.16:0/3990844934 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"} v 0) v1 -- 0x7ff7000d0e30 con 0x7ff70015ab80 2023-05-20T01:26:20.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.306+0000 7ff6f6ffd700 1 -- 172.21.15.16:0/3990844934 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"}]=0 v99) v1 ==== 106+0+11 (secure 0 0 0) 0x7ff6f8057b60 con 0x7ff70015ab80 2023-05-20T01:26:20.311 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-05-20T01:26:20.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.308+0000 7ff7083b3700 1 -- 172.21.15.16:0/3990844934 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff6e406ace0 msgr2=0x7ff6e406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:20.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.308+0000 7ff7083b3700 1 --2- 172.21.15.16:0/3990844934 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff6e406ace0 0x7ff6e406d190 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7ff6f0009d00 tx=0x7ff6f0008040 comp rx=0 tx=0).stop 2023-05-20T01:26:20.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.308+0000 7ff7083b3700 1 -- 172.21.15.16:0/3990844934 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff70015ab80 msgr2=0x7ff7000c9b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:20.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.308+0000 7ff7083b3700 1 --2- 172.21.15.16:0/3990844934 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff70015ab80 0x7ff7000c9b70 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7ff6f800cc90 tx=0x7ff6f800a340 comp rx=0 tx=0).stop 2023-05-20T01:26:20.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.308+0000 7ff7083b3700 1 -- 172.21.15.16:0/3990844934 shutdown_connections 2023-05-20T01:26:20.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.308+0000 7ff7083b3700 1 --2- 172.21.15.16:0/3990844934 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff70015ab80 0x7ff7000c9b70 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:20.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.308+0000 7ff7083b3700 1 --2- 172.21.15.16:0/3990844934 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff6e406ace0 0x7ff6e406d190 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:20.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.308+0000 7ff7083b3700 1 --2- 172.21.15.16:0/3990844934 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff7000cdfc0 0x7ff7000c8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:20.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.308+0000 7ff7083b3700 1 --2- 172.21.15.16:0/3990844934 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff7000cf1c0 0x7ff7000c73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:20.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.308+0000 7ff7083b3700 1 -- 172.21.15.16:0/3990844934 >> 172.21.15.16:0/3990844934 conn(0x7ff70006ae30 msgr2=0x7ff7000d1120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:20.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.308+0000 7ff7083b3700 1 -- 172.21.15.16:0/3990844934 shutdown_connections 2023-05-20T01:26:20.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.308+0000 7ff7083b3700 1 -- 172.21.15.16:0/3990844934 wait complete. 2023-05-20T01:26:20.325 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-20T01:26:20.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.446+0000 7f94a14db700 1 Processor -- start 2023-05-20T01:26:20.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.446+0000 7f94a14db700 1 -- start start 2023-05-20T01:26:20.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.447+0000 7f94a14db700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f949c157190 0x7f949c0cc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:20.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.447+0000 7f94a14db700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f949c0cccf0 0x7f949c0713c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:20.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.447+0000 7f94a14db700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f949c071900 0x7f949c15d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:20.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.447+0000 7f94a14db700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f949c0cb590 con 0x7f949c071900 2023-05-20T01:26:20.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.447+0000 7f94a14db700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f949c0ca9a0 con 0x7f949c0cccf0 2023-05-20T01:26:20.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.447+0000 7f94a14db700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f949c0cac60 con 0x7f949c157190 2023-05-20T01:26:20.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.447+0000 7f949b7fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f949c071900 0x7f949c15d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:20.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.447+0000 7f949b7fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f949c071900 0x7f949c15d220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37632/0 (socket says 172.21.15.16:37632) 2023-05-20T01:26:20.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.447+0000 7f949b7fe700 1 -- 172.21.15.16:0/2040749885 learned_addr learned my addr 172.21.15.16:0/2040749885 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:26:20.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.447+0000 7f949a7fc700 1 --2- 172.21.15.16:0/2040749885 >> v2:172.21.15.132:3300/0 conn(0x7f949c0cccf0 0x7f949c0713c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:20.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.447+0000 7f949affd700 1 --2- 172.21.15.16:0/2040749885 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f949c157190 0x7f949c0cc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:20.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.447+0000 7f949b7fe700 1 -- 172.21.15.16:0/2040749885 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f949c157190 msgr2=0x7f949c0cc7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:20.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.447+0000 7f949b7fe700 1 --2- 172.21.15.16:0/2040749885 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f949c157190 0x7f949c0cc7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:20.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.447+0000 7f949b7fe700 1 -- 172.21.15.16:0/2040749885 >> v2:172.21.15.132:3300/0 conn(0x7f949c0cccf0 msgr2=0x7f949c0713c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:20.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.447+0000 7f949b7fe700 1 --2- 172.21.15.16:0/2040749885 >> v2:172.21.15.132:3300/0 conn(0x7f949c0cccf0 0x7f949c0713c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:20.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.447+0000 7f949b7fe700 1 -- 172.21.15.16:0/2040749885 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f949c0caf40 con 0x7f949c071900 2023-05-20T01:26:20.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.447+0000 7f949affd700 1 --2- 172.21.15.16:0/2040749885 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f949c157190 0x7f949c0cc7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:26:20.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.448+0000 7f949b7fe700 1 --2- 172.21.15.16:0/2040749885 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f949c071900 0x7f949c15d220 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f949c0d0c00 tx=0x7f949400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=617ef04a23a8ffa4 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:20.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.448+0000 7f9499ffb700 1 -- 172.21.15.16:0/2040749885 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f949400e070 con 0x7f949c071900 2023-05-20T01:26:20.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.448+0000 7f9499ffb700 1 -- 172.21.15.16:0/2040749885 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f949400b9c0 con 0x7f949c071900 2023-05-20T01:26:20.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.448+0000 7f9499ffb700 1 -- 172.21.15.16:0/2040749885 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f949400bd70 con 0x7f949c071900 2023-05-20T01:26:20.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.448+0000 7f94a14db700 1 -- 172.21.15.16:0/2040749885 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f949c071900 msgr2=0x7f949c15d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:20.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.448+0000 7f94a14db700 1 --2- 172.21.15.16:0/2040749885 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f949c071900 0x7f949c15d220 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f949c0d0c00 tx=0x7f949400b330 comp rx=0 tx=0).stop 2023-05-20T01:26:20.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.448+0000 7f94a14db700 1 -- 172.21.15.16:0/2040749885 shutdown_connections 2023-05-20T01:26:20.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.448+0000 7f94a14db700 1 --2- 172.21.15.16:0/2040749885 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f949c071900 0x7f949c15d220 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:20.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.448+0000 7f94a14db700 1 --2- 172.21.15.16:0/2040749885 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f949c157190 0x7f949c0cc7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:20.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.448+0000 7f94a14db700 1 --2- 172.21.15.16:0/2040749885 >> v2:172.21.15.132:3300/0 conn(0x7f949c0cccf0 0x7f949c0713c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:20.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.448+0000 7f94a14db700 1 -- 172.21.15.16:0/2040749885 >> 172.21.15.16:0/2040749885 conn(0x7f949c0cffb0 msgr2=0x7f949c0d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:20.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.448+0000 7f94a14db700 1 -- 172.21.15.16:0/2040749885 shutdown_connections 2023-05-20T01:26:20.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.448+0000 7f94a14db700 1 -- 172.21.15.16:0/2040749885 wait complete. 2023-05-20T01:26:20.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.450+0000 7f94a14db700 1 Processor -- start 2023-05-20T01:26:20.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.450+0000 7f94a14db700 1 -- start start 2023-05-20T01:26:20.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.451+0000 7f94a14db700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f949c071900 0x7f949c153ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:20.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.451+0000 7f94a14db700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f949c0cccf0 0x7f949c0c4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:20.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.451+0000 7f94a14db700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f949c157190 0x7f949c155420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:20.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.451+0000 7f94a14db700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f949c0c6560 con 0x7f949c0cccf0 2023-05-20T01:26:20.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.451+0000 7f94a14db700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f949c06eab0 con 0x7f949c157190 2023-05-20T01:26:20.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.451+0000 7f94a14db700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f949c0c54c0 con 0x7f949c071900 2023-05-20T01:26:20.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.451+0000 7f949a7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f949c0cccf0 0x7f949c0c4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:20.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.451+0000 7f949a7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f949c0cccf0 0x7f949c0c4ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37636/0 (socket says 172.21.15.16:37636) 2023-05-20T01:26:20.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.451+0000 7f949a7fc700 1 -- 172.21.15.16:0/1710315029 learned_addr learned my addr 172.21.15.16:0/1710315029 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:26:20.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.451+0000 7f949affd700 1 --2- 172.21.15.16:0/1710315029 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f949c071900 0x7f949c153ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:20.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.451+0000 7f949b7fe700 1 --2- 172.21.15.16:0/1710315029 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f949c157190 0x7f949c155420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:20.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.451+0000 7f949a7fc700 1 -- 172.21.15.16:0/1710315029 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f949c071900 msgr2=0x7f949c153ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:20.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.451+0000 7f949a7fc700 1 --2- 172.21.15.16:0/1710315029 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f949c071900 0x7f949c153ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:20.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.451+0000 7f949a7fc700 1 -- 172.21.15.16:0/1710315029 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f949c157190 msgr2=0x7f949c155420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:20.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.451+0000 7f949a7fc700 1 --2- 172.21.15.16:0/1710315029 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f949c157190 0x7f949c155420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:20.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.451+0000 7f949a7fc700 1 -- 172.21.15.16:0/1710315029 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9494009b30 con 0x7f949c0cccf0 2023-05-20T01:26:20.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.451+0000 7f949a7fc700 1 --2- 172.21.15.16:0/1710315029 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f949c0cccf0 0x7f949c0c4ae0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f948c00e810 tx=0x7f948c00eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:20.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.452+0000 7f94837fe700 1 -- 172.21.15.16:0/1710315029 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f948c00c070 con 0x7f949c0cccf0 2023-05-20T01:26:20.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.452+0000 7f94837fe700 1 -- 172.21.15.16:0/1710315029 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f948c013610 con 0x7f949c0cccf0 2023-05-20T01:26:20.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.452+0000 7f94a14db700 1 -- 172.21.15.16:0/1710315029 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f949c157560 con 0x7f949c0cccf0 2023-05-20T01:26:20.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.452+0000 7f94837fe700 1 -- 172.21.15.16:0/1710315029 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f948c0139c0 con 0x7f949c0cccf0 2023-05-20T01:26:20.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.452+0000 7f94a14db700 1 -- 172.21.15.16:0/1710315029 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f949c0cb220 con 0x7f949c0cccf0 2023-05-20T01:26:20.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.453+0000 7f94a14db700 1 -- 172.21.15.16:0/1710315029 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f949c0cd120 con 0x7f949c0cccf0 2023-05-20T01:26:20.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.453+0000 7f94837fe700 1 -- 172.21.15.16:0/1710315029 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f948c013b20 con 0x7f949c0cccf0 2023-05-20T01:26:20.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.454+0000 7f94837fe700 1 --2- 172.21.15.16:0/1710315029 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f948806ace0 0x7f948806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:20.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.454+0000 7f949affd700 1 --2- 172.21.15.16:0/1710315029 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f948806ace0 0x7f948806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:20.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.454+0000 7f94837fe700 1 -- 172.21.15.16:0/1710315029 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 7102+0+0 (secure 0 0 0) 0x7f948c089680 con 0x7f949c0cccf0 2023-05-20T01:26:20.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.454+0000 7f949affd700 1 --2- 172.21.15.16:0/1710315029 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f948806ace0 0x7f948806d190 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f9490006fd0 tx=0x7f9490008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:20.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.458+0000 7f94837fe700 1 -- 172.21.15.16:0/1710315029 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f948c057dc0 con 0x7f949c0cccf0 2023-05-20T01:26:20.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.701+0000 7f94a14db700 1 -- 172.21.15.16:0/1710315029 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f949c0cba20 con 0x7f949c0cccf0 2023-05-20T01:26:20.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.701+0000 7f94837fe700 1 -- 172.21.15.16:0/1710315029 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) v1 ==== 94+0+4671 (secure 0 0 0) 0x7f948c057950 con 0x7f949c0cccf0 2023-05-20T01:26:20.706 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:26:20.706 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":70,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6323,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/59032067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":59032067},{"type":"v1","addr":"172.21.15.132:6834","nonce":59032067}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66},{"gid":6329,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/100162236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":100162236},{"type":"v1","addr":"172.21.15.132:6835","nonce":100162236}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66},{"gid":7572,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/3753560731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3753560731},{"type":"v1","addr":"172.21.15.16:6836","nonce":3753560731}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":70,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-20T01:25:32.295399+0000","modified":"2023-05-20T01:25:41.570561+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":95,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7575},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7575":{"gid":7575,"name":"c","rank":0,"incarnation":67,"state":"up:active","state_seq":4,"addr":"172.21.15.16:6837/307899603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":307899603},{"type":"v1","addr":"172.21.15.16:6837","nonce":307899603}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13,14],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-05-20T01:26:20.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.703+0000 7f94a14db700 1 -- 172.21.15.16:0/1710315029 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f948806ace0 msgr2=0x7f948806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:20.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.703+0000 7f94a14db700 1 --2- 172.21.15.16:0/1710315029 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f948806ace0 0x7f948806d190 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f9490006fd0 tx=0x7f9490008040 comp rx=0 tx=0).stop 2023-05-20T01:26:20.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.703+0000 7f94a14db700 1 -- 172.21.15.16:0/1710315029 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f949c0cccf0 msgr2=0x7f949c0c4ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:20.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.703+0000 7f94a14db700 1 --2- 172.21.15.16:0/1710315029 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f949c0cccf0 0x7f949c0c4ae0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f948c00e810 tx=0x7f948c00eb20 comp rx=0 tx=0).stop 2023-05-20T01:26:20.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.703+0000 7f94a14db700 1 -- 172.21.15.16:0/1710315029 shutdown_connections 2023-05-20T01:26:20.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.703+0000 7f94a14db700 1 --2- 172.21.15.16:0/1710315029 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f949c157190 0x7f949c155420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:20.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.703+0000 7f94a14db700 1 --2- 172.21.15.16:0/1710315029 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f948806ace0 0x7f948806d190 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:20.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.703+0000 7f94a14db700 1 --2- 172.21.15.16:0/1710315029 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f949c071900 0x7f949c153ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:20.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.703+0000 7f94a14db700 1 --2- 172.21.15.16:0/1710315029 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f949c0cccf0 0x7f949c0c4ae0 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:20.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.703+0000 7f94a14db700 1 -- 172.21.15.16:0/1710315029 >> 172.21.15.16:0/1710315029 conn(0x7f949c0cffb0 msgr2=0x7f949c06a510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:20.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.703+0000 7f94a14db700 1 -- 172.21.15.16:0/1710315029 shutdown_connections 2023-05-20T01:26:20.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.703+0000 7f94a14db700 1 -- 172.21.15.16:0/1710315029 wait complete. 2023-05-20T01:26:20.709 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 70 2023-05-20T01:26:20.720 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-20T01:26:20.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.840+0000 7fe670dd5700 1 Processor -- start 2023-05-20T01:26:20.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.840+0000 7fe670dd5700 1 -- start start 2023-05-20T01:26:20.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.841+0000 7fe670dd5700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe66c0ca980 0x7fe66c0aef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:20.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.841+0000 7fe670dd5700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe66c0af450 0x7fe66c15dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:20.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.841+0000 7fe670dd5700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe66c0ce920 con 0x7fe66c0cad50 2023-05-20T01:26:20.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.841+0000 7fe670dd5700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe66c0af8f0 con 0x7fe66c0af450 2023-05-20T01:26:20.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.841+0000 7fe670dd5700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe66c0725d0 con 0x7fe66c0ca980 2023-05-20T01:26:20.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.841+0000 7fe669d9b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe66c0af450 0x7fe66c15dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:20.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.841+0000 7fe66ad9d700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fe66c0cad50 0x7fe66c15e1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49174/0 (socket says 172.21.15.16:49174) 2023-05-20T01:26:20.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.841+0000 7fe66ad9d700 1 -- 172.21.15.16:0/947784170 learned_addr learned my addr 172.21.15.16:0/947784170 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:26:20.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.841+0000 7fe66a59c700 1 --2- 172.21.15.16:0/947784170 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe66c0ca980 0x7fe66c0aef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:20.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.841+0000 7fe669d9b700 1 -- 172.21.15.16:0/947784170 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe66c0ca980 msgr2=0x7fe66c0aef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:20.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.841+0000 7fe669d9b700 1 --2- 172.21.15.16:0/947784170 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe66c0ca980 0x7fe66c0aef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:20.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.841+0000 7fe669d9b700 1 -- 172.21.15.16:0/947784170 >> v1:172.21.15.16:6789/0 conn(0x7fe66c0cad50 legacy=0x7fe66c15e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:20.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.841+0000 7fe669d9b700 1 -- 172.21.15.16:0/947784170 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe66c0722f0 con 0x7fe66c0af450 2023-05-20T01:26:20.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.841+0000 7fe669d9b700 1 --2- 172.21.15.16:0/947784170 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe66c0af450 0x7fe66c15dc70 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7fe660009840 tx=0x7fe660009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66e6eb55e92ebaa8 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:20.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.842+0000 7fe66959a700 1 -- 172.21.15.16:0/947784170 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe660011830 con 0x7fe66c0af450 2023-05-20T01:26:20.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.842+0000 7fe66959a700 1 -- 172.21.15.16:0/947784170 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe660011990 con 0x7fe66c0af450 2023-05-20T01:26:20.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.842+0000 7fe66959a700 1 -- 172.21.15.16:0/947784170 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe660011d20 con 0x7fe66c0af450 2023-05-20T01:26:20.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.842+0000 7fe670dd5700 1 -- 172.21.15.16:0/947784170 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe66c0af450 msgr2=0x7fe66c15dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:20.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.842+0000 7fe670dd5700 1 --2- 172.21.15.16:0/947784170 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe66c0af450 0x7fe66c15dc70 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7fe660009840 tx=0x7fe660009b50 comp rx=0 tx=0).stop 2023-05-20T01:26:20.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.842+0000 7fe670dd5700 1 -- 172.21.15.16:0/947784170 shutdown_connections 2023-05-20T01:26:20.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.842+0000 7fe670dd5700 1 --2- 172.21.15.16:0/947784170 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe66c0ca980 0x7fe66c0aef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:20.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.842+0000 7fe670dd5700 1 --2- 172.21.15.16:0/947784170 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe66c0af450 0x7fe66c15dc70 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:20.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.842+0000 7fe670dd5700 1 -- 172.21.15.16:0/947784170 >> 172.21.15.16:0/947784170 conn(0x7fe66c0d0c70 msgr2=0x7fe66c0b1af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:20.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.842+0000 7fe670dd5700 1 -- 172.21.15.16:0/947784170 shutdown_connections 2023-05-20T01:26:20.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.842+0000 7fe670dd5700 1 -- 172.21.15.16:0/947784170 wait complete. 2023-05-20T01:26:20.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.844+0000 7fe670dd5700 1 Processor -- start 2023-05-20T01:26:20.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.844+0000 7fe670dd5700 1 -- start start 2023-05-20T01:26:20.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.844+0000 7fe670dd5700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe66c0af450 0x7fe66c0c4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:20.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.844+0000 7fe670dd5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe66c0ca980 0x7fe66c0c9530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:20.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.844+0000 7fe670dd5700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe66c0cad50 0x7fe66c1d5d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:20.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.844+0000 7fe670dd5700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe66c0c6380 con 0x7fe66c0ca980 2023-05-20T01:26:20.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.844+0000 7fe670dd5700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe66c0c5700 con 0x7fe66c0cad50 2023-05-20T01:26:20.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.844+0000 7fe670dd5700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe66c0c59e0 con 0x7fe66c0af450 2023-05-20T01:26:20.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.844+0000 7fe669d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe66c0ca980 0x7fe66c0c9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:20.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.845+0000 7fe669d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe66c0ca980 0x7fe66c0c9530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37650/0 (socket says 172.21.15.16:37650) 2023-05-20T01:26:20.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.845+0000 7fe669d9b700 1 -- 172.21.15.16:0/1263040201 learned_addr learned my addr 172.21.15.16:0/1263040201 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:26:20.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.845+0000 7fe66a59c700 1 --2- 172.21.15.16:0/1263040201 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe66c0af450 0x7fe66c0c4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:20.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.845+0000 7fe66ad9d700 1 --2- 172.21.15.16:0/1263040201 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe66c0cad50 0x7fe66c1d5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:20.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.845+0000 7fe669d9b700 1 -- 172.21.15.16:0/1263040201 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe66c0af450 msgr2=0x7fe66c0c4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:20.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.845+0000 7fe669d9b700 1 --2- 172.21.15.16:0/1263040201 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe66c0af450 0x7fe66c0c4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:20.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.845+0000 7fe669d9b700 1 -- 172.21.15.16:0/1263040201 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe66c0cad50 msgr2=0x7fe66c1d5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:20.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.845+0000 7fe669d9b700 1 --2- 172.21.15.16:0/1263040201 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe66c0cad50 0x7fe66c1d5d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:20.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.845+0000 7fe669d9b700 1 -- 172.21.15.16:0/1263040201 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6600094a0 con 0x7fe66c0ca980 2023-05-20T01:26:20.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.845+0000 7fe66ad9d700 1 --2- 172.21.15.16:0/1263040201 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe66c0cad50 0x7fe66c1d5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:26:20.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.845+0000 7fe669d9b700 1 --2- 172.21.15.16:0/1263040201 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe66c0ca980 0x7fe66c0c9530 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7fe660009b90 tx=0x7fe660004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:20.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.845+0000 7fe65affd700 1 -- 172.21.15.16:0/1263040201 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6600036a0 con 0x7fe66c0ca980 2023-05-20T01:26:20.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.845+0000 7fe65affd700 1 -- 172.21.15.16:0/1263040201 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe66001f070 con 0x7fe66c0ca980 2023-05-20T01:26:20.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.845+0000 7fe670dd5700 1 -- 172.21.15.16:0/1263040201 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe66c0cc1c0 con 0x7fe66c0ca980 2023-05-20T01:26:20.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.845+0000 7fe65affd700 1 -- 172.21.15.16:0/1263040201 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe660003a40 con 0x7fe66c0ca980 2023-05-20T01:26:20.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.845+0000 7fe670dd5700 1 -- 172.21.15.16:0/1263040201 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe66c0ce5b0 con 0x7fe66c0ca980 2023-05-20T01:26:20.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.846+0000 7fe670dd5700 1 -- 172.21.15.16:0/1263040201 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe66c0cc350 con 0x7fe66c0ca980 2023-05-20T01:26:20.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.847+0000 7fe65affd700 1 -- 172.21.15.16:0/1263040201 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe660003ba0 con 0x7fe66c0ca980 2023-05-20T01:26:20.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.847+0000 7fe65affd700 1 --2- 172.21.15.16:0/1263040201 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe64c06ace0 0x7fe64c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:20.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.847+0000 7fe65affd700 1 -- 172.21.15.16:0/1263040201 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 7102+0+0 (secure 0 0 0) 0x7fe660021070 con 0x7fe66c0ca980 2023-05-20T01:26:20.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.847+0000 7fe66a59c700 1 --2- 172.21.15.16:0/1263040201 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe64c06ace0 0x7fe64c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:20.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.847+0000 7fe66a59c700 1 --2- 172.21.15.16:0/1263040201 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe64c06ace0 0x7fe64c06d190 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fe65c009930 tx=0x7fe65c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:20.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:20.851+0000 7fe65affd700 1 -- 172.21.15.16:0/1263040201 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fe660055f70 con 0x7fe66c0ca980 2023-05-20T01:26:21.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.047+0000 7fe670dd5700 1 -- 172.21.15.16:0/1263040201 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe66c0c64c0 con 0x7fe66c0ca980 2023-05-20T01:26:21.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.049+0000 7fe65affd700 1 -- 172.21.15.16:0/1263040201 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) v1 ==== 74+0+20720 (secure 0 0 0) 0x7fe66c0c64c0 con 0x7fe66c0ca980 2023-05-20T01:26:21.054 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:26:21.054 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":99,"fsid":"c2245cc5-ecab-4087-a59b-bc81db901676","created":"2023-05-20T01:16:09.549230+0000","modified":"2023-05-20T01:26:17.513405+0000","last_up_change":"2023-05-20T01:16:19.788624+0000","last_in_change":"2023-05-20T01:16:12.092160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-05-20T01:16:20.934366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-05-20T01:16:21.372294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-05-20T01:25:09.967348+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-05-20T01:25:10.956221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data_extra","create_time":"2023-05-20T01:25:24.629648+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"317a83cd-19c8-4958-a301-6087ef342289","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6825","nonce":3923634179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6827","nonce":3923634179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6831","nonce":3923634179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6829","nonce":3923634179}]},"public_addr":"172.21.15.16:6825/3923634179","cluster_addr":"172.21.15.16:6827/3923634179","heartbeat_back_addr":"172.21.15.16:6831/3923634179","heartbeat_front_addr":"172.21.15.16:6829/3923634179","state":["exists","up"]},{"osd":1,"uuid":"5581046f-8219-4089-8d7d-b7846399505e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6809","nonce":3860264220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6811","nonce":3860264220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6815","nonce":3860264220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6813","nonce":3860264220}]},"public_addr":"172.21.15.16:6809/3860264220","cluster_addr":"172.21.15.16:6811/3860264220","heartbeat_back_addr":"172.21.15.16:6815/3860264220","heartbeat_front_addr":"172.21.15.16:6813/3860264220","state":["exists","up"]},{"osd":2,"uuid":"072e9e66-4133-478e-8d4a-64d46c2963f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6817","nonce":293187030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6819","nonce":293187030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6823","nonce":293187030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6821","nonce":293187030}]},"public_addr":"172.21.15.16:6817/293187030","cluster_addr":"172.21.15.16:6819/293187030","heartbeat_back_addr":"172.21.15.16:6823/293187030","heartbeat_front_addr":"172.21.15.16:6821/293187030","state":["exists","up"]},{"osd":3,"uuid":"11e1bba6-00f8-48dd-9028-d8d716b7f145","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6801","nonce":3669781890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6803","nonce":3669781890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6807","nonce":3669781890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6805","nonce":3669781890}]},"public_addr":"172.21.15.16:6801/3669781890","cluster_addr":"172.21.15.16:6803/3669781890","heartbeat_back_addr":"172.21.15.16:6807/3669781890","heartbeat_front_addr":"172.21.15.16:6805/3669781890","state":["exists","up"]},{"osd":4,"uuid":"b8909c3a-cf70-46cb-8d0b-009c707cf9d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6819","nonce":506317223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6823","nonce":506317223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6829","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6831","nonce":506317223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6827","nonce":506317223}]},"public_addr":"172.21.15.132:6819/506317223","cluster_addr":"172.21.15.132:6823/506317223","heartbeat_back_addr":"172.21.15.132:6831/506317223","heartbeat_front_addr":"172.21.15.132:6827/506317223","state":["exists","up"]},{"osd":5,"uuid":"b1818304-4583-47bf-a6ec-0bfbb5166233","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6818","nonce":1844689995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6822","nonce":1844689995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6830","nonce":1844689995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6826","nonce":1844689995}]},"public_addr":"172.21.15.132:6818/1844689995","cluster_addr":"172.21.15.132:6822/1844689995","heartbeat_back_addr":"172.21.15.132:6830/1844689995","heartbeat_front_addr":"172.21.15.132:6826/1844689995","state":["exists","up"]},{"osd":6,"uuid":"d37f4691-eed1-4a93-8818-18354f03f4a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6801","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6802","nonce":727581123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6805","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6807","nonce":727581123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6813","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6815","nonce":727581123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6811","nonce":727581123}]},"public_addr":"172.21.15.132:6802/727581123","cluster_addr":"172.21.15.132:6807/727581123","heartbeat_back_addr":"172.21.15.132:6815/727581123","heartbeat_front_addr":"172.21.15.132:6811/727581123","state":["exists","up"]},{"osd":7,"uuid":"4f712f62-5d1d-4816-8042-03209144f8b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6803","nonce":466001149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6806","nonce":466001149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6814","nonce":466001149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6810","nonce":466001149}]},"public_addr":"172.21.15.132:6803/466001149","cluster_addr":"172.21.15.132:6806/466001149","heartbeat_back_addr":"172.21.15.132:6814/466001149","heartbeat_front_addr":"172.21.15.132:6810/466001149","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.338863+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.278680+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305694+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.246835+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.276237+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.304441+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305421+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.267050+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.f","mappings":[{"from":6,"to":3}]},{"pgid":"13.23","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6834/2740585996":"2023-05-21T01:25:37.758863+0000","172.21.15.16:6837/2740585996":"2023-05-21T01:25:37.758863+0000","172.21.15.16:6837/1327905747":"2023-05-21T01:25:32.534353+0000","172.21.15.16:6834/1327905747":"2023-05-21T01:25:32.534353+0000","172.21.15.16:6837/28561908":"2023-05-21T01:25:30.939570+0000","172.21.15.16:6834/28561908":"2023-05-21T01:25:30.939570+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-20T01:26:21.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.051+0000 7fe670dd5700 1 -- 172.21.15.16:0/1263040201 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe64c06ace0 msgr2=0x7fe64c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:21.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.051+0000 7fe670dd5700 1 --2- 172.21.15.16:0/1263040201 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe64c06ace0 0x7fe64c06d190 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fe65c009930 tx=0x7fe65c008040 comp rx=0 tx=0).stop 2023-05-20T01:26:21.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.051+0000 7fe670dd5700 1 -- 172.21.15.16:0/1263040201 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe66c0ca980 msgr2=0x7fe66c0c9530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:21.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.051+0000 7fe670dd5700 1 --2- 172.21.15.16:0/1263040201 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe66c0ca980 0x7fe66c0c9530 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7fe660009b90 tx=0x7fe660004750 comp rx=0 tx=0).stop 2023-05-20T01:26:21.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.051+0000 7fe670dd5700 1 -- 172.21.15.16:0/1263040201 shutdown_connections 2023-05-20T01:26:21.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.051+0000 7fe670dd5700 1 --2- 172.21.15.16:0/1263040201 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe66c0cad50 0x7fe66c1d5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:21.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.051+0000 7fe670dd5700 1 --2- 172.21.15.16:0/1263040201 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe64c06ace0 0x7fe64c06d190 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:21.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.051+0000 7fe670dd5700 1 --2- 172.21.15.16:0/1263040201 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe66c0af450 0x7fe66c0c4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:21.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.051+0000 7fe670dd5700 1 --2- 172.21.15.16:0/1263040201 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe66c0ca980 0x7fe66c0c9530 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:21.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.051+0000 7fe670dd5700 1 -- 172.21.15.16:0/1263040201 >> 172.21.15.16:0/1263040201 conn(0x7fe66c0d0c70 msgr2=0x7fe66c06e3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:21.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.051+0000 7fe670dd5700 1 -- 172.21.15.16:0/1263040201 shutdown_connections 2023-05-20T01:26:21.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.051+0000 7fe670dd5700 1 -- 172.21.15.16:0/1263040201 wait complete. 2023-05-20T01:26:21.069 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-05-20T01:26:21.070 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-20T01:26:21.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.190+0000 7f400445e700 1 Processor -- start 2023-05-20T01:26:21.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.190+0000 7f400445e700 1 -- start start 2023-05-20T01:26:21.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.191+0000 7f400445e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3ffc0cfc50 0x7f3ffc0d2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:21.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.191+0000 7f400445e700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3ffc0cd970 0x7f3ffc0cddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:21.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.191+0000 7f400445e700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ffc0b3b50 con 0x7f3ffc0b4750 2023-05-20T01:26:21.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.191+0000 7f400445e700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ffc0cfad0 con 0x7f3ffc0cd970 2023-05-20T01:26:21.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.191+0000 7f400445e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ffc0d3df0 con 0x7f3ffc0cfc50 2023-05-20T01:26:21.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.191+0000 7f40029fb700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3ffc0cd970 0x7f3ffc0cddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:21.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.191+0000 7f40019f9700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f3ffc0b4750 0x7f3ffc0d0020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49184/0 (socket says 172.21.15.16:49184) 2023-05-20T01:26:21.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.191+0000 7f40019f9700 1 -- 172.21.15.16:0/373310548 learned_addr learned my addr 172.21.15.16:0/373310548 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:26:21.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.191+0000 7f40021fa700 1 --2- 172.21.15.16:0/373310548 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3ffc0cfc50 0x7f3ffc0d2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:21.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.191+0000 7f40029fb700 1 -- 172.21.15.16:0/373310548 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3ffc0cfc50 msgr2=0x7f3ffc0d2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:21.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.191+0000 7f40029fb700 1 --2- 172.21.15.16:0/373310548 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3ffc0cfc50 0x7f3ffc0d2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:21.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.191+0000 7f40029fb700 1 -- 172.21.15.16:0/373310548 >> v1:172.21.15.16:6789/0 conn(0x7f3ffc0b4750 legacy=0x7f3ffc0d0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:21.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.192+0000 7f40029fb700 1 -- 172.21.15.16:0/373310548 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ffc0d0b90 con 0x7f3ffc0cd970 2023-05-20T01:26:21.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.191+0000 7f40011f8700 1 -- 172.21.15.16:0/373310548 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2678306993 0 0) 0x7f3ffc0b3b50 con 0x7f3ffc0b4750 2023-05-20T01:26:21.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.192+0000 7f40021fa700 1 --2- 172.21.15.16:0/373310548 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3ffc0cfc50 0x7f3ffc0d2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:26:21.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.192+0000 7f40029fb700 1 --2- 172.21.15.16:0/373310548 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ffc0cd970 0x7f3ffc0cddf0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f3ff4009d90 tx=0x7f3ff400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f04c5a7e8864acb0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:21.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.192+0000 7f40011f8700 1 -- 172.21.15.16:0/373310548 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ff400e070 con 0x7f3ffc0cd970 2023-05-20T01:26:21.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.192+0000 7f40011f8700 1 -- 172.21.15.16:0/373310548 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ff400bbd0 con 0x7f3ffc0cd970 2023-05-20T01:26:21.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.192+0000 7f40011f8700 1 -- 172.21.15.16:0/373310548 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ff4012430 con 0x7f3ffc0cd970 2023-05-20T01:26:21.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.192+0000 7f400445e700 1 -- 172.21.15.16:0/373310548 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ffc0cd970 msgr2=0x7f3ffc0cddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:21.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.192+0000 7f400445e700 1 --2- 172.21.15.16:0/373310548 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ffc0cd970 0x7f3ffc0cddf0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f3ff4009d90 tx=0x7f3ff400b330 comp rx=0 tx=0).stop 2023-05-20T01:26:21.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.192+0000 7f400445e700 1 -- 172.21.15.16:0/373310548 shutdown_connections 2023-05-20T01:26:21.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.192+0000 7f400445e700 1 --2- 172.21.15.16:0/373310548 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ffc0cd970 0x7f3ffc0cddf0 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:21.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.192+0000 7f400445e700 1 --2- 172.21.15.16:0/373310548 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3ffc0cfc50 0x7f3ffc0d2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:21.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.192+0000 7f400445e700 1 -- 172.21.15.16:0/373310548 >> 172.21.15.16:0/373310548 conn(0x7f3ffc09c590 msgr2=0x7f3ffc0d3240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:21.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.193+0000 7f400445e700 1 -- 172.21.15.16:0/373310548 shutdown_connections 2023-05-20T01:26:21.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.193+0000 7f400445e700 1 -- 172.21.15.16:0/373310548 wait complete. 2023-05-20T01:26:21.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.194+0000 7f400445e700 1 Processor -- start 2023-05-20T01:26:21.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.194+0000 7f400445e700 1 -- start start 2023-05-20T01:26:21.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.195+0000 7f400445e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3ffc0b4750 0x7f3ffc0ca310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:21.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.195+0000 7f400445e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3ffc0cd970 0x7f3ffc0ca850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:21.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.195+0000 7f400445e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ffc0cfc50 0x7f3ffc0cad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:21.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.195+0000 7f400445e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ffc0b9010 con 0x7f3ffc0cfc50 2023-05-20T01:26:21.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.195+0000 7f400445e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ffc0b8b80 con 0x7f3ffc0cd970 2023-05-20T01:26:21.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.195+0000 7f400445e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ffc0b7f70 con 0x7f3ffc0b4750 2023-05-20T01:26:21.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.195+0000 7f40029fb700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ffc0cfc50 0x7f3ffc0cad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:21.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.195+0000 7f40029fb700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ffc0cfc50 0x7f3ffc0cad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37666/0 (socket says 172.21.15.16:37666) 2023-05-20T01:26:21.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.195+0000 7f40029fb700 1 -- 172.21.15.16:0/787509076 learned_addr learned my addr 172.21.15.16:0/787509076 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:26:21.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.195+0000 7f40021fa700 1 --2- 172.21.15.16:0/787509076 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3ffc0b4750 0x7f3ffc0ca310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:21.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.195+0000 7f40019f9700 1 --2- 172.21.15.16:0/787509076 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3ffc0cd970 0x7f3ffc0ca850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:21.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.195+0000 7f40029fb700 1 -- 172.21.15.16:0/787509076 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3ffc0b4750 msgr2=0x7f3ffc0ca310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:21.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.195+0000 7f40029fb700 1 --2- 172.21.15.16:0/787509076 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3ffc0b4750 0x7f3ffc0ca310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:21.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.195+0000 7f40029fb700 1 -- 172.21.15.16:0/787509076 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3ffc0cd970 msgr2=0x7f3ffc0ca850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:21.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.195+0000 7f40029fb700 1 --2- 172.21.15.16:0/787509076 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3ffc0cd970 0x7f3ffc0ca850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:21.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.195+0000 7f40029fb700 1 -- 172.21.15.16:0/787509076 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ff4009a70 con 0x7f3ffc0cfc50 2023-05-20T01:26:21.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.195+0000 7f40019f9700 1 --2- 172.21.15.16:0/787509076 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3ffc0cd970 0x7f3ffc0ca850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:26:21.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.195+0000 7f40029fb700 1 --2- 172.21.15.16:0/787509076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ffc0cfc50 0x7f3ffc0cad90 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f3ffc073aa0 tx=0x7f3ff4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:21.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.195+0000 7f3ff2ffd700 1 -- 172.21.15.16:0/787509076 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ff4012430 con 0x7f3ffc0cfc50 2023-05-20T01:26:21.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.195+0000 7f3ff2ffd700 1 -- 172.21.15.16:0/787509076 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ff4020070 con 0x7f3ffc0cfc50 2023-05-20T01:26:21.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.195+0000 7f400445e700 1 -- 172.21.15.16:0/787509076 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ffc0b4380 con 0x7f3ffc0cfc50 2023-05-20T01:26:21.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.195+0000 7f3ff2ffd700 1 -- 172.21.15.16:0/787509076 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ff4004280 con 0x7f3ffc0cfc50 2023-05-20T01:26:21.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.195+0000 7f400445e700 1 -- 172.21.15.16:0/787509076 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ffc15d0b0 con 0x7f3ffc0cfc50 2023-05-20T01:26:21.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.197+0000 7f400445e700 1 -- 172.21.15.16:0/787509076 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ffc0d4000 con 0x7f3ffc0cfc50 2023-05-20T01:26:21.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.197+0000 7f3ff2ffd700 1 -- 172.21.15.16:0/787509076 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3ff40043e0 con 0x7f3ffc0cfc50 2023-05-20T01:26:21.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.197+0000 7f3ff2ffd700 1 --2- 172.21.15.16:0/787509076 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f3fe006ace0 0x7f3fe006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:21.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.197+0000 7f3ff2ffd700 1 -- 172.21.15.16:0/787509076 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 7102+0+0 (secure 0 0 0) 0x7f3ff4022070 con 0x7f3ffc0cfc50 2023-05-20T01:26:21.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.197+0000 7f40021fa700 1 --2- 172.21.15.16:0/787509076 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f3fe006ace0 0x7f3fe006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:21.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.198+0000 7f40021fa700 1 --2- 172.21.15.16:0/787509076 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f3fe006ace0 0x7f3fe006d190 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f3fec00c940 tx=0x7f3fec008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:21.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.202+0000 7f3ff2ffd700 1 -- 172.21.15.16:0/787509076 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f3ff4056820 con 0x7f3ffc0cfc50 2023-05-20T01:26:21.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.442+0000 7f400445e700 1 -- 172.21.15.16:0/787509076 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3ffc0747b0 con 0x7f3ffc0cfc50 2023-05-20T01:26:21.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.443+0000 7f3ff2ffd700 1 -- 172.21.15.16:0/787509076 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) v1 ==== 94+0+4671 (secure 0 0 0) 0x7f3ff401a380 con 0x7f3ffc0cfc50 2023-05-20T01:26:21.447 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:26:21.448 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":70,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6323,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/59032067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":59032067},{"type":"v1","addr":"172.21.15.132:6834","nonce":59032067}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66},{"gid":6329,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/100162236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":100162236},{"type":"v1","addr":"172.21.15.132:6835","nonce":100162236}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66},{"gid":7572,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/3753560731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3753560731},{"type":"v1","addr":"172.21.15.16:6836","nonce":3753560731}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":70,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-20T01:25:32.295399+0000","modified":"2023-05-20T01:25:41.570561+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":95,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7575},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7575":{"gid":7575,"name":"c","rank":0,"incarnation":67,"state":"up:active","state_seq":4,"addr":"172.21.15.16:6837/307899603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":307899603},{"type":"v1","addr":"172.21.15.16:6837","nonce":307899603}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13,14],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-05-20T01:26:21.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.445+0000 7f400445e700 1 -- 172.21.15.16:0/787509076 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f3fe006ace0 msgr2=0x7f3fe006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:21.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.445+0000 7f400445e700 1 --2- 172.21.15.16:0/787509076 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f3fe006ace0 0x7f3fe006d190 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f3fec00c940 tx=0x7f3fec008040 comp rx=0 tx=0).stop 2023-05-20T01:26:21.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.445+0000 7f400445e700 1 -- 172.21.15.16:0/787509076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ffc0cfc50 msgr2=0x7f3ffc0cad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:21.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.445+0000 7f400445e700 1 --2- 172.21.15.16:0/787509076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ffc0cfc50 0x7f3ffc0cad90 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f3ffc073aa0 tx=0x7f3ff4004ad0 comp rx=0 tx=0).stop 2023-05-20T01:26:21.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.445+0000 7f400445e700 1 -- 172.21.15.16:0/787509076 shutdown_connections 2023-05-20T01:26:21.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.445+0000 7f400445e700 1 --2- 172.21.15.16:0/787509076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ffc0cfc50 0x7f3ffc0cad90 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:21.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.445+0000 7f400445e700 1 --2- 172.21.15.16:0/787509076 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f3fe006ace0 0x7f3fe006d190 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:21.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.445+0000 7f400445e700 1 --2- 172.21.15.16:0/787509076 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3ffc0b4750 0x7f3ffc0ca310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:21.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.445+0000 7f400445e700 1 --2- 172.21.15.16:0/787509076 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3ffc0cd970 0x7f3ffc0ca850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:21.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.445+0000 7f400445e700 1 -- 172.21.15.16:0/787509076 >> 172.21.15.16:0/787509076 conn(0x7f3ffc09c590 msgr2=0x7f3ffc06b120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:21.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.445+0000 7f400445e700 1 -- 172.21.15.16:0/787509076 shutdown_connections 2023-05-20T01:26:21.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.445+0000 7f400445e700 1 -- 172.21.15.16:0/787509076 wait complete. 2023-05-20T01:26:21.451 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 70 2023-05-20T01:26:21.461 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-20T01:26:21.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.584+0000 7f728ef23700 1 Processor -- start 2023-05-20T01:26:21.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.584+0000 7f728ef23700 1 -- start start 2023-05-20T01:26:21.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.585+0000 7f728ef23700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f728815a9d0 0x7f728815efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:21.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.585+0000 7f728ef23700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f72880b2250 0x7f728806a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:21.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.585+0000 7f728ef23700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f728806f850 0x7f728806aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:21.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.585+0000 7f728ef23700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72880cc8d0 con 0x7f728815a9d0 2023-05-20T01:26:21.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.585+0000 7f728ef23700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f728806d940 con 0x7f728806f850 2023-05-20T01:26:21.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.585+0000 7f728ef23700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f728815f500 con 0x7f72880b2250 2023-05-20T01:26:21.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.585+0000 7f728ccbf700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f728815a9d0 0x7f728815efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:21.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.585+0000 7f728ccbf700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f728815a9d0 0x7f728815efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37670/0 (socket says 172.21.15.16:37670) 2023-05-20T01:26:21.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.585+0000 7f728ccbf700 1 -- 172.21.15.16:0/261644249 learned_addr learned my addr 172.21.15.16:0/261644249 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:26:21.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.586+0000 7f7287fff700 1 --2- 172.21.15.16:0/261644249 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f72880b2250 0x7f728806a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:21.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.586+0000 7f728ccbf700 1 -- 172.21.15.16:0/261644249 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f72880b2250 msgr2=0x7f728806a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:21.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.586+0000 7f728d4c0700 1 --2- 172.21.15.16:0/261644249 >> v2:172.21.15.132:3300/0 conn(0x7f728806f850 0x7f728806aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:21.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.586+0000 7f728ccbf700 1 --2- 172.21.15.16:0/261644249 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f72880b2250 0x7f728806a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:21.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.586+0000 7f728ccbf700 1 -- 172.21.15.16:0/261644249 >> v2:172.21.15.132:3300/0 conn(0x7f728806f850 msgr2=0x7f728806aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:21.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.586+0000 7f728ccbf700 1 --2- 172.21.15.16:0/261644249 >> v2:172.21.15.132:3300/0 conn(0x7f728806f850 0x7f728806aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:21.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.586+0000 7f728ccbf700 1 -- 172.21.15.16:0/261644249 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72880cc030 con 0x7f728815a9d0 2023-05-20T01:26:21.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.586+0000 7f728ccbf700 1 --2- 172.21.15.16:0/261644249 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f728815a9d0 0x7f728815efc0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f7288072c40 tx=0x7f7278009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2eb5d4730c46f574 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:21.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.586+0000 7f72877fe700 1 -- 172.21.15.16:0/261644249 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f727800b870 con 0x7f728815a9d0 2023-05-20T01:26:21.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.586+0000 7f72877fe700 1 -- 172.21.15.16:0/261644249 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f727800b9d0 con 0x7f728815a9d0 2023-05-20T01:26:21.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.586+0000 7f72877fe700 1 -- 172.21.15.16:0/261644249 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f727800bd60 con 0x7f728815a9d0 2023-05-20T01:26:21.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.587+0000 7f728ef23700 1 -- 172.21.15.16:0/261644249 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f728815a9d0 msgr2=0x7f728815efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:21.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.587+0000 7f728ef23700 1 --2- 172.21.15.16:0/261644249 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f728815a9d0 0x7f728815efc0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f7288072c40 tx=0x7f7278009bd0 comp rx=0 tx=0).stop 2023-05-20T01:26:21.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.587+0000 7f728ef23700 1 -- 172.21.15.16:0/261644249 shutdown_connections 2023-05-20T01:26:21.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.587+0000 7f728ef23700 1 --2- 172.21.15.16:0/261644249 >> v2:172.21.15.132:3300/0 conn(0x7f728806f850 0x7f728806aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:21.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.587+0000 7f728ef23700 1 --2- 172.21.15.16:0/261644249 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f728815a9d0 0x7f728815efc0 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:21.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.587+0000 7f728ef23700 1 --2- 172.21.15.16:0/261644249 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f72880b2250 0x7f728806a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:21.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.587+0000 7f728ef23700 1 -- 172.21.15.16:0/261644249 >> 172.21.15.16:0/261644249 conn(0x7f72880d0c30 msgr2=0x7f72880b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:21.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.587+0000 7f728ef23700 1 -- 172.21.15.16:0/261644249 shutdown_connections 2023-05-20T01:26:21.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.587+0000 7f728ef23700 1 -- 172.21.15.16:0/261644249 wait complete. 2023-05-20T01:26:21.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.588+0000 7f728ef23700 1 Processor -- start 2023-05-20T01:26:21.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.588+0000 7f728ef23700 1 -- start start 2023-05-20T01:26:21.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.589+0000 7f728ef23700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f728806f850 0x7f7288156230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:21.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.589+0000 7f728ef23700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72880b2250 0x7f72880c8410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:21.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.589+0000 7f728ef23700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f728815a9d0 0x7f72881f3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:21.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.589+0000 7f728ef23700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7288157540 con 0x7f72880b2250 2023-05-20T01:26:21.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.589+0000 7f728ef23700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72880c8950 con 0x7f728815a9d0 2023-05-20T01:26:21.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.589+0000 7f728ef23700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7288156930 con 0x7f728806f850 2023-05-20T01:26:21.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.589+0000 7f7287fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72880b2250 0x7f72880c8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:21.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.589+0000 7f7287fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72880b2250 0x7f72880c8410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37676/0 (socket says 172.21.15.16:37676) 2023-05-20T01:26:21.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.589+0000 7f7287fff700 1 -- 172.21.15.16:0/1143514791 learned_addr learned my addr 172.21.15.16:0/1143514791 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:26:21.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.589+0000 7f728d4c0700 1 --2- 172.21.15.16:0/1143514791 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f728815a9d0 0x7f72881f3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:21.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.589+0000 7f728ccbf700 1 --2- 172.21.15.16:0/1143514791 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f728806f850 0x7f7288156230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:21.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.589+0000 7f7287fff700 1 -- 172.21.15.16:0/1143514791 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f728806f850 msgr2=0x7f7288156230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:21.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.589+0000 7f7287fff700 1 --2- 172.21.15.16:0/1143514791 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f728806f850 0x7f7288156230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:21.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.589+0000 7f7287fff700 1 -- 172.21.15.16:0/1143514791 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f728815a9d0 msgr2=0x7f72881f3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:21.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.589+0000 7f7287fff700 1 --2- 172.21.15.16:0/1143514791 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f728815a9d0 0x7f72881f3790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:21.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.589+0000 7f7287fff700 1 -- 172.21.15.16:0/1143514791 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7278009550 con 0x7f72880b2250 2023-05-20T01:26:21.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.589+0000 7f7287fff700 1 --2- 172.21.15.16:0/1143514791 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72880b2250 0x7f72880c8410 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f727000c7f0 tx=0x7f727000cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:21.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.589+0000 7f72857fa700 1 -- 172.21.15.16:0/1143514791 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7270011800 con 0x7f72880b2250 2023-05-20T01:26:21.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.590+0000 7f72857fa700 1 -- 172.21.15.16:0/1143514791 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7270011960 con 0x7f72880b2250 2023-05-20T01:26:21.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.590+0000 7f728ef23700 1 -- 172.21.15.16:0/1143514791 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72880cc5f0 con 0x7f72880b2250 2023-05-20T01:26:21.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.590+0000 7f72857fa700 1 -- 172.21.15.16:0/1143514791 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7270011cf0 con 0x7f72880b2250 2023-05-20T01:26:21.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.590+0000 7f728ef23700 1 -- 172.21.15.16:0/1143514791 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72880cc310 con 0x7f72880b2250 2023-05-20T01:26:21.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.590+0000 7f728ef23700 1 -- 172.21.15.16:0/1143514791 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f728815e7c0 con 0x7f72880b2250 2023-05-20T01:26:21.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.591+0000 7f72857fa700 1 -- 172.21.15.16:0/1143514791 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7270018360 con 0x7f72880b2250 2023-05-20T01:26:21.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.591+0000 7f72857fa700 1 --2- 172.21.15.16:0/1143514791 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f726806ace0 0x7f726806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:21.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.591+0000 7f72857fa700 1 -- 172.21.15.16:0/1143514791 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 7102+0+0 (secure 0 0 0) 0x7f7270089840 con 0x7f72880b2250 2023-05-20T01:26:21.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.591+0000 7f728ccbf700 1 --2- 172.21.15.16:0/1143514791 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f726806ace0 0x7f726806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:21.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.592+0000 7f728ccbf700 1 --2- 172.21.15.16:0/1143514791 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f726806ace0 0x7f726806d190 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f7278014040 tx=0x7f7278008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:21.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.596+0000 7f72857fa700 1 -- 172.21.15.16:0/1143514791 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f7270057f80 con 0x7f72880b2250 2023-05-20T01:26:21.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.840+0000 7f728ef23700 1 -- 172.21.15.16:0/1143514791 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f72880747b0 con 0x7f72880b2250 2023-05-20T01:26:21.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.841+0000 7f72857fa700 1 -- 172.21.15.16:0/1143514791 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) v1 ==== 94+0+4671 (secure 0 0 0) 0x7f7270057b10 con 0x7f72880b2250 2023-05-20T01:26:21.845 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:26:21.846 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":70,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6323,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/59032067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":59032067},{"type":"v1","addr":"172.21.15.132:6834","nonce":59032067}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66},{"gid":6329,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/100162236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":100162236},{"type":"v1","addr":"172.21.15.132:6835","nonce":100162236}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66},{"gid":7572,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/3753560731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3753560731},{"type":"v1","addr":"172.21.15.16:6836","nonce":3753560731}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":70,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-20T01:25:32.295399+0000","modified":"2023-05-20T01:25:41.570561+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":95,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7575},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7575":{"gid":7575,"name":"c","rank":0,"incarnation":67,"state":"up:active","state_seq":4,"addr":"172.21.15.16:6837/307899603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":307899603},{"type":"v1","addr":"172.21.15.16:6837","nonce":307899603}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13,14],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-05-20T01:26:21.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.842+0000 7f728ef23700 1 -- 172.21.15.16:0/1143514791 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f726806ace0 msgr2=0x7f726806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:21.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.842+0000 7f728ef23700 1 --2- 172.21.15.16:0/1143514791 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f726806ace0 0x7f726806d190 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f7278014040 tx=0x7f7278008030 comp rx=0 tx=0).stop 2023-05-20T01:26:21.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.842+0000 7f728ef23700 1 -- 172.21.15.16:0/1143514791 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72880b2250 msgr2=0x7f72880c8410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:21.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.842+0000 7f728ef23700 1 --2- 172.21.15.16:0/1143514791 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72880b2250 0x7f72880c8410 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f727000c7f0 tx=0x7f727000cbb0 comp rx=0 tx=0).stop 2023-05-20T01:26:21.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.843+0000 7f728ef23700 1 -- 172.21.15.16:0/1143514791 shutdown_connections 2023-05-20T01:26:21.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.843+0000 7f728ef23700 1 --2- 172.21.15.16:0/1143514791 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f728815a9d0 0x7f72881f3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:21.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.843+0000 7f728ef23700 1 --2- 172.21.15.16:0/1143514791 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f726806ace0 0x7f726806d190 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:21.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.843+0000 7f728ef23700 1 --2- 172.21.15.16:0/1143514791 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f728806f850 0x7f7288156230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:21.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.843+0000 7f728ef23700 1 --2- 172.21.15.16:0/1143514791 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72880b2250 0x7f72880c8410 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:21.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.843+0000 7f728ef23700 1 -- 172.21.15.16:0/1143514791 >> 172.21.15.16:0/1143514791 conn(0x7f72880d0c30 msgr2=0x7f72880b2e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:21.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.843+0000 7f728ef23700 1 -- 172.21.15.16:0/1143514791 shutdown_connections 2023-05-20T01:26:21.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.843+0000 7f728ef23700 1 -- 172.21.15.16:0/1143514791 wait complete. 2023-05-20T01:26:21.849 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 70 2023-05-20T01:26:21.859 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-20T01:26:21.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.980+0000 7ff2410f4700 1 Processor -- start 2023-05-20T01:26:21.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.980+0000 7ff2410f4700 1 -- start start 2023-05-20T01:26:21.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.981+0000 7ff2410f4700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff23c144020 0x7ff23c0d1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:21.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.981+0000 7ff2410f4700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff23c0cc500 con 0x7ff23c0cec60 2023-05-20T01:26:21.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.981+0000 7ff2410f4700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff23c0cb8f0 con 0x7ff23c0b2760 2023-05-20T01:26:21.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.981+0000 7ff2410f4700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff23c0cb6e0 con 0x7ff23c144020 2023-05-20T01:26:21.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.981+0000 7ff23a59c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff23c144020 0x7ff23c0d1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:21.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.981+0000 7ff23b59e700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ff23c0cec60 0x7ff23c0cc990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49214/0 (socket says 172.21.15.16:49214) 2023-05-20T01:26:21.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.981+0000 7ff23b59e700 1 -- 172.21.15.16:0/1450739123 learned_addr learned my addr 172.21.15.16:0/1450739123 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:26:21.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.981+0000 7ff23a59c700 1 -- 172.21.15.16:0/1450739123 >> v1:172.21.15.132:6789/0 conn(0x7ff23c0b2760 legacy=0x7ff23c0b0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:21.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.981+0000 7ff23a59c700 1 -- 172.21.15.16:0/1450739123 >> v1:172.21.15.16:6789/0 conn(0x7ff23c0cec60 legacy=0x7ff23c0cc990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:21.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.981+0000 7ff23a59c700 1 -- 172.21.15.16:0/1450739123 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff23c0cbbd0 con 0x7ff23c144020 2023-05-20T01:26:21.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.981+0000 7ff23a59c700 1 --2- 172.21.15.16:0/1450739123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff23c144020 0x7ff23c0d1680 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7ff230009840 tx=0x7ff230009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f58a21a6292979fb server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:21.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.981+0000 7ff239d9b700 1 -- 172.21.15.16:0/1450739123 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff230004230 con 0x7ff23c144020 2023-05-20T01:26:21.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.981+0000 7ff239d9b700 1 -- 172.21.15.16:0/1450739123 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff230004390 con 0x7ff23c144020 2023-05-20T01:26:21.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.981+0000 7ff239d9b700 1 -- 172.21.15.16:0/1450739123 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2300036a0 con 0x7ff23c144020 2023-05-20T01:26:21.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.982+0000 7ff2410f4700 1 -- 172.21.15.16:0/1450739123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff23c144020 msgr2=0x7ff23c0d1680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:21.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.982+0000 7ff2410f4700 1 --2- 172.21.15.16:0/1450739123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff23c144020 0x7ff23c0d1680 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7ff230009840 tx=0x7ff230009b50 comp rx=0 tx=0).stop 2023-05-20T01:26:21.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.982+0000 7ff2410f4700 1 -- 172.21.15.16:0/1450739123 shutdown_connections 2023-05-20T01:26:21.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.982+0000 7ff2410f4700 1 --2- 172.21.15.16:0/1450739123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff23c144020 0x7ff23c0d1680 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:21.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.982+0000 7ff2410f4700 1 -- 172.21.15.16:0/1450739123 >> 172.21.15.16:0/1450739123 conn(0x7ff23c06ae30 msgr2=0x7ff23c09a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:21.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.982+0000 7ff2410f4700 1 -- 172.21.15.16:0/1450739123 shutdown_connections 2023-05-20T01:26:21.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.982+0000 7ff2410f4700 1 -- 172.21.15.16:0/1450739123 wait complete. 2023-05-20T01:26:21.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.983+0000 7ff2410f4700 1 Processor -- start 2023-05-20T01:26:21.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.984+0000 7ff2410f4700 1 -- start start 2023-05-20T01:26:21.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.984+0000 7ff2410f4700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff23c0b2760 0x7ff23c156d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:21.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.984+0000 7ff2410f4700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff23c0cec60 0x7ff23c06f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:21.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.984+0000 7ff2410f4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff23c144020 0x7ff23c15b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:21.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.984+0000 7ff2410f4700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff23c0b4060 con 0x7ff23c144020 2023-05-20T01:26:21.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.984+0000 7ff2410f4700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff23c0b4340 con 0x7ff23c0b2760 2023-05-20T01:26:21.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.984+0000 7ff2410f4700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff23c0b4620 con 0x7ff23c0cec60 2023-05-20T01:26:21.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.984+0000 7ff23b59e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff23c144020 0x7ff23c15b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:21.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.984+0000 7ff23b59e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff23c144020 0x7ff23c15b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37692/0 (socket says 172.21.15.16:37692) 2023-05-20T01:26:21.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.984+0000 7ff23b59e700 1 -- 172.21.15.16:0/4175018091 learned_addr learned my addr 172.21.15.16:0/4175018091 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:26:21.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.985+0000 7ff23a59c700 1 --2- 172.21.15.16:0/4175018091 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff23c0cec60 0x7ff23c06f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:21.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.985+0000 7ff23ad9d700 1 --2- 172.21.15.16:0/4175018091 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff23c0b2760 0x7ff23c156d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:21.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.985+0000 7ff23b59e700 1 -- 172.21.15.16:0/4175018091 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff23c0cec60 msgr2=0x7ff23c06f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:21.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.985+0000 7ff23b59e700 1 --2- 172.21.15.16:0/4175018091 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff23c0cec60 0x7ff23c06f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:21.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.985+0000 7ff23b59e700 1 -- 172.21.15.16:0/4175018091 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff23c0b2760 msgr2=0x7ff23c156d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:22.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.985+0000 7ff23b59e700 1 --2- 172.21.15.16:0/4175018091 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff23c0b2760 0x7ff23c156d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:22.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.985+0000 7ff23b59e700 1 -- 172.21.15.16:0/4175018091 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2300094a0 con 0x7ff23c144020 2023-05-20T01:26:22.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.985+0000 7ff23ad9d700 1 --2- 172.21.15.16:0/4175018091 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff23c0b2760 0x7ff23c156d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:26:22.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.985+0000 7ff23b59e700 1 --2- 172.21.15.16:0/4175018091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff23c144020 0x7ff23c15b0d0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7ff23400cbc0 tx=0x7ff23400ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:22.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.985+0000 7ff2237fe700 1 -- 172.21.15.16:0/4175018091 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff23400aaa0 con 0x7ff23c144020 2023-05-20T01:26:22.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.985+0000 7ff2237fe700 1 -- 172.21.15.16:0/4175018091 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff23400ac00 con 0x7ff23c144020 2023-05-20T01:26:22.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.985+0000 7ff2237fe700 1 -- 172.21.15.16:0/4175018091 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff234012400 con 0x7ff23c144020 2023-05-20T01:26:22.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.985+0000 7ff2410f4700 1 -- 172.21.15.16:0/4175018091 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff23c0cc190 con 0x7ff23c144020 2023-05-20T01:26:22.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.985+0000 7ff2410f4700 1 -- 172.21.15.16:0/4175018091 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff23c0cbeb0 con 0x7ff23c144020 2023-05-20T01:26:22.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.986+0000 7ff2410f4700 1 -- 172.21.15.16:0/4175018091 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff23c0cf0c0 con 0x7ff23c144020 2023-05-20T01:26:22.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.987+0000 7ff2237fe700 1 -- 172.21.15.16:0/4175018091 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff23401b3b0 con 0x7ff23c144020 2023-05-20T01:26:22.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.987+0000 7ff2237fe700 1 --2- 172.21.15.16:0/4175018091 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff22806ace0 0x7ff22806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:22.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.987+0000 7ff23ad9d700 1 --2- 172.21.15.16:0/4175018091 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff22806ace0 0x7ff22806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:22.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.987+0000 7ff2237fe700 1 -- 172.21.15.16:0/4175018091 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 7102+0+0 (secure 0 0 0) 0x7ff234089430 con 0x7ff23c144020 2023-05-20T01:26:22.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.988+0000 7ff23ad9d700 1 --2- 172.21.15.16:0/4175018091 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff22806ace0 0x7ff22806d190 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7ff22c001850 tx=0x7ff22c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:22.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:21.992+0000 7ff2237fe700 1 -- 172.21.15.16:0/4175018091 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7ff234054180 con 0x7ff23c144020 2023-05-20T01:26:22.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:22.189+0000 7ff2410f4700 1 -- 172.21.15.16:0/4175018091 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff23c0b41a0 con 0x7ff23c144020 2023-05-20T01:26:22.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:22.191+0000 7ff2237fe700 1 -- 172.21.15.16:0/4175018091 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) v1 ==== 74+0+20720 (secure 0 0 0) 0x7ff23c0b41a0 con 0x7ff23c144020 2023-05-20T01:26:22.196 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:26:22.197 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":99,"fsid":"c2245cc5-ecab-4087-a59b-bc81db901676","created":"2023-05-20T01:16:09.549230+0000","modified":"2023-05-20T01:26:17.513405+0000","last_up_change":"2023-05-20T01:16:19.788624+0000","last_in_change":"2023-05-20T01:16:12.092160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-05-20T01:16:20.934366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-05-20T01:16:21.372294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-05-20T01:25:09.967348+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-05-20T01:25:10.956221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data_extra","create_time":"2023-05-20T01:25:24.629648+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"317a83cd-19c8-4958-a301-6087ef342289","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6825","nonce":3923634179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6827","nonce":3923634179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6831","nonce":3923634179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6829","nonce":3923634179}]},"public_addr":"172.21.15.16:6825/3923634179","cluster_addr":"172.21.15.16:6827/3923634179","heartbeat_back_addr":"172.21.15.16:6831/3923634179","heartbeat_front_addr":"172.21.15.16:6829/3923634179","state":["exists","up"]},{"osd":1,"uuid":"5581046f-8219-4089-8d7d-b7846399505e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6809","nonce":3860264220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6811","nonce":3860264220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6815","nonce":3860264220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6813","nonce":3860264220}]},"public_addr":"172.21.15.16:6809/3860264220","cluster_addr":"172.21.15.16:6811/3860264220","heartbeat_back_addr":"172.21.15.16:6815/3860264220","heartbeat_front_addr":"172.21.15.16:6813/3860264220","state":["exists","up"]},{"osd":2,"uuid":"072e9e66-4133-478e-8d4a-64d46c2963f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6817","nonce":293187030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6819","nonce":293187030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6823","nonce":293187030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6821","nonce":293187030}]},"public_addr":"172.21.15.16:6817/293187030","cluster_addr":"172.21.15.16:6819/293187030","heartbeat_back_addr":"172.21.15.16:6823/293187030","heartbeat_front_addr":"172.21.15.16:6821/293187030","state":["exists","up"]},{"osd":3,"uuid":"11e1bba6-00f8-48dd-9028-d8d716b7f145","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6801","nonce":3669781890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6803","nonce":3669781890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6807","nonce":3669781890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6805","nonce":3669781890}]},"public_addr":"172.21.15.16:6801/3669781890","cluster_addr":"172.21.15.16:6803/3669781890","heartbeat_back_addr":"172.21.15.16:6807/3669781890","heartbeat_front_addr":"172.21.15.16:6805/3669781890","state":["exists","up"]},{"osd":4,"uuid":"b8909c3a-cf70-46cb-8d0b-009c707cf9d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6819","nonce":506317223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6823","nonce":506317223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6829","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6831","nonce":506317223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6827","nonce":506317223}]},"public_addr":"172.21.15.132:6819/506317223","cluster_addr":"172.21.15.132:6823/506317223","heartbeat_back_addr":"172.21.15.132:6831/506317223","heartbeat_front_addr":"172.21.15.132:6827/506317223","state":["exists","up"]},{"osd":5,"uuid":"b1818304-4583-47bf-a6ec-0bfbb5166233","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6818","nonce":1844689995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6822","nonce":1844689995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6830","nonce":1844689995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6826","nonce":1844689995}]},"public_addr":"172.21.15.132:6818/1844689995","cluster_addr":"172.21.15.132:6822/1844689995","heartbeat_back_addr":"172.21.15.132:6830/1844689995","heartbeat_front_addr":"172.21.15.132:6826/1844689995","state":["exists","up"]},{"osd":6,"uuid":"d37f4691-eed1-4a93-8818-18354f03f4a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6801","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6802","nonce":727581123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6805","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6807","nonce":727581123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6813","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6815","nonce":727581123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6811","nonce":727581123}]},"public_addr":"172.21.15.132:6802/727581123","cluster_addr":"172.21.15.132:6807/727581123","heartbeat_back_addr":"172.21.15.132:6815/727581123","heartbeat_front_addr":"172.21.15.132:6811/727581123","state":["exists","up"]},{"osd":7,"uuid":"4f712f62-5d1d-4816-8042-03209144f8b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6803","nonce":466001149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6806","nonce":466001149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6814","nonce":466001149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6810","nonce":466001149}]},"public_addr":"172.21.15.132:6803/466001149","cluster_addr":"172.21.15.132:6806/466001149","heartbeat_back_addr":"172.21.15.132:6814/466001149","heartbeat_front_addr":"172.21.15.132:6810/466001149","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.338863+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.278680+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305694+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.246835+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.276237+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.304441+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305421+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.267050+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.f","mappings":[{"from":6,"to":3}]},{"pgid":"13.23","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6834/2740585996":"2023-05-21T01:25:37.758863+0000","172.21.15.16:6837/2740585996":"2023-05-21T01:25:37.758863+0000","172.21.15.16:6837/1327905747":"2023-05-21T01:25:32.534353+0000","172.21.15.16:6834/1327905747":"2023-05-21T01:25:32.534353+0000","172.21.15.16:6837/28561908":"2023-05-21T01:25:30.939570+0000","172.21.15.16:6834/28561908":"2023-05-21T01:25:30.939570+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-20T01:26:22.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:22.193+0000 7ff2410f4700 1 -- 172.21.15.16:0/4175018091 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff22806ace0 msgr2=0x7ff22806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:22.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:22.193+0000 7ff2410f4700 1 --2- 172.21.15.16:0/4175018091 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff22806ace0 0x7ff22806d190 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7ff22c001850 tx=0x7ff22c009040 comp rx=0 tx=0).stop 2023-05-20T01:26:22.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:22.193+0000 7ff2410f4700 1 -- 172.21.15.16:0/4175018091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff23c144020 msgr2=0x7ff23c15b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:22.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:22.193+0000 7ff2410f4700 1 --2- 172.21.15.16:0/4175018091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff23c144020 0x7ff23c15b0d0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7ff23400cbc0 tx=0x7ff23400ced0 comp rx=0 tx=0).stop 2023-05-20T01:26:22.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:22.193+0000 7ff2410f4700 1 -- 172.21.15.16:0/4175018091 shutdown_connections 2023-05-20T01:26:22.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:22.193+0000 7ff2410f4700 1 --2- 172.21.15.16:0/4175018091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff23c144020 0x7ff23c15b0d0 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:22.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:22.193+0000 7ff2410f4700 1 --2- 172.21.15.16:0/4175018091 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff23c0b2760 0x7ff23c156d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:22.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:22.193+0000 7ff2410f4700 1 --2- 172.21.15.16:0/4175018091 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff22806ace0 0x7ff22806d190 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:22.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:22.193+0000 7ff2410f4700 1 --2- 172.21.15.16:0/4175018091 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff23c0cec60 0x7ff23c06f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:22.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:22.193+0000 7ff2410f4700 1 -- 172.21.15.16:0/4175018091 >> 172.21.15.16:0/4175018091 conn(0x7ff23c06ae30 msgr2=0x7ff23c0af7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:22.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:22.193+0000 7ff2410f4700 1 -- 172.21.15.16:0/4175018091 shutdown_connections 2023-05-20T01:26:22.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:22.193+0000 7ff2410f4700 1 -- 172.21.15.16:0/4175018091 wait complete. 2023-05-20T01:26:22.210 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-05-20T01:26:22.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:22.330+0000 7fe7d982f700 1 Processor -- start 2023-05-20T01:26:22.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:22.330+0000 7fe7d982f700 1 -- start start 2023-05-20T01:26:22.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:22.331+0000 7fe7d982f700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fe7d415cc50 0x7fe7d40b4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:22.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:22.331+0000 7fe7d982f700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe7d406b730 0x7fe7d40b5200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:22.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:22.331+0000 7fe7d982f700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe7d40b5740 0x7fe7d40d2080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:22.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:22.331+0000 7fe7d982f700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7d40cd720 con 0x7fe7d40b5740 2023-05-20T01:26:22.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:22.331+0000 7fe7d982f700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7d415d020 con 0x7fe7d415cc50 2023-05-20T01:26:22.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:22.331+0000 7fe7d982f700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7d40d1bf0 con 0x7fe7d406b730 2023-05-20T01:26:22.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:22.331+0000 7fe7d37fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe7d40b5740 0x7fe7d40d2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:22.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:22.331+0000 7fe7d37fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe7d40b5740 0x7fe7d40d2080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37700/0 (socket says 172.21.15.16:37700) 2023-05-20T01:26:22.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:22.331+0000 7fe7d37fe700 1 -- 172.21.15.16:0/4192162525 learned_addr learned my addr 172.21.15.16:0/4192162525 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:26:22.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:22.331+0000 7fe7d27fc700 1 --2- 172.21.15.16:0/4192162525 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe7d406b730 0x7fe7d40b5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:22.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:22.331+0000 7fe7d2ffd700 1 --2- 172.21.15.16:0/4192162525 >> v2:172.21.15.132:3300/0 conn(0x7fe7d415cc50 0x7fe7d40b4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:22.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:22.331+0000 7fe7d37fe700 1 -- 172.21.15.16:0/4192162525 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe7d406b730 msgr2=0x7fe7d40b5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:22.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:22.331+0000 7fe7d37fe700 1 --2- 172.21.15.16:0/4192162525 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe7d406b730 0x7fe7d40b5200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:22.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:22.331+0000 7fe7d37fe700 1 -- 172.21.15.16:0/4192162525 >> v2:172.21.15.132:3300/0 conn(0x7fe7d415cc50 msgr2=0x7fe7d40b4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:22.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:22.331+0000 7fe7d37fe700 1 --2- 172.21.15.16:0/4192162525 >> v2:172.21.15.132:3300/0 conn(0x7fe7d415cc50 0x7fe7d40b4cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:22.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:22.331+0000 7fe7d37fe700 1 -- 172.21.15.16:0/4192162525 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7d40d18c0 con 0x7fe7d40b5740 2023-05-20T01:26:22.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:22.331+0000 7fe7d2ffd700 1 --2- 172.21.15.16:0/4192162525 >> v2:172.21.15.132:3300/0 conn(0x7fe7d415cc50 0x7fe7d40b4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:26:22.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:22.331+0000 7fe7d27fc700 1 --2- 172.21.15.16:0/4192162525 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe7d406b730 0x7fe7d40b5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:26:22.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:22.331+0000 7fe7d37fe700 1 --2- 172.21.15.16:0/4192162525 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7d40b5740 0x7fe7d40d2080 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fe7d408c510 tx=0x7fe7cc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1ab8a46635b7370 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:22.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:22.332+0000 7fe7d1ffb700 1 -- 172.21.15.16:0/4192162525 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7cc00e070 con 0x7fe7d40b5740 2023-05-20T01:26:22.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:22.332+0000 7fe7d1ffb700 1 -- 172.21.15.16:0/4192162525 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7cc00b9c0 con 0x7fe7d40b5740 2023-05-20T01:26:22.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:22.332+0000 7fe7d1ffb700 1 -- 172.21.15.16:0/4192162525 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7cc00bd70 con 0x7fe7d40b5740 2023-05-20T01:26:22.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:22.332+0000 7fe7d982f700 1 -- 172.21.15.16:0/4192162525 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7d40b5740 msgr2=0x7fe7d40d2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:22.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:22.332+0000 7fe7d982f700 1 --2- 172.21.15.16:0/4192162525 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7d40b5740 0x7fe7d40d2080 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fe7d408c510 tx=0x7fe7cc00b330 comp rx=0 tx=0).stop 2023-05-20T01:26:22.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:22.332+0000 7fe7d982f700 1 -- 172.21.15.16:0/4192162525 shutdown_connections 2023-05-20T01:26:22.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:22.332+0000 7fe7d982f700 1 --2- 172.21.15.16:0/4192162525 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7d40b5740 0x7fe7d40d2080 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:22.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:22.332+0000 7fe7d982f700 1 --2- 172.21.15.16:0/4192162525 >> v2:172.21.15.132:3300/0 conn(0x7fe7d415cc50 0x7fe7d40b4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:22.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:22.332+0000 7fe7d982f700 1 --2- 172.21.15.16:0/4192162525 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe7d406b730 0x7fe7d40b5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:22.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:22.332+0000 7fe7d982f700 1 -- 172.21.15.16:0/4192162525 >> 172.21.15.16:0/4192162525 conn(0x7fe7d4072860 msgr2=0x7fe7d406b020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:22.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:22.332+0000 7fe7d982f700 1 -- 172.21.15.16:0/4192162525 shutdown_connections 2023-05-20T01:26:22.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:22.332+0000 7fe7d982f700 1 -- 172.21.15.16:0/4192162525 wait complete. 2023-05-20T01:26:22.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:22.333+0000 7fe7d982f700 1 Processor -- start 2023-05-20T01:26:22.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:22.334+0000 7fe7d982f700 1 -- start start 2023-05-20T01:26:22.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:22.334+0000 7fe7d982f700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe7d406b730 0x7fe7d4153e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:22.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:22.334+0000 7fe7d982f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7d40b5740 0x7fe7d40b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:22.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:22.334+0000 7fe7d982f700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe7d415cc50 0x7fe7d40b3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:22.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:22.334+0000 7fe7d982f700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7d40c6d70 con 0x7fe7d40b5740 2023-05-20T01:26:22.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:22.334+0000 7fe7d982f700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7d40c7a90 con 0x7fe7d415cc50 2023-05-20T01:26:22.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:22.334+0000 7fe7d982f700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7d40c6fc0 con 0x7fe7d406b730 2023-05-20T01:26:22.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:22.334+0000 7fe7d27fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7d40b5740 0x7fe7d40b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:22.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:22.334+0000 7fe7d2ffd700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe7d406b730 0x7fe7d4153e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:22.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:22.334+0000 7fe7d27fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7d40b5740 0x7fe7d40b3590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37704/0 (socket says 172.21.15.16:37704) 2023-05-20T01:26:22.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:22.334+0000 7fe7d27fc700 1 -- 172.21.15.16:0/182343898 learned_addr learned my addr 172.21.15.16:0/182343898 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:26:22.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:22.334+0000 7fe7d37fe700 1 --2- 172.21.15.16:0/182343898 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe7d415cc50 0x7fe7d40b3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:22.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:22.335+0000 7fe7d27fc700 1 -- 172.21.15.16:0/182343898 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe7d406b730 msgr2=0x7fe7d4153e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:22.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:22.335+0000 7fe7d27fc700 1 --2- 172.21.15.16:0/182343898 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe7d406b730 0x7fe7d4153e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:22.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:22.335+0000 7fe7d27fc700 1 -- 172.21.15.16:0/182343898 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe7d415cc50 msgr2=0x7fe7d40b3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:22.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:22.335+0000 7fe7d27fc700 1 --2- 172.21.15.16:0/182343898 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe7d415cc50 0x7fe7d40b3ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:22.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:22.335+0000 7fe7d27fc700 1 -- 172.21.15.16:0/182343898 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7cc009b30 con 0x7fe7d40b5740 2023-05-20T01:26:22.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:22.335+0000 7fe7d27fc700 1 --2- 172.21.15.16:0/182343898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7d40b5740 0x7fe7d40b3590 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fe7c400c7f0 tx=0x7fe7c400cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:22.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:22.335+0000 7fe7bbfff700 1 -- 172.21.15.16:0/182343898 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7c4011800 con 0x7fe7d40b5740 2023-05-20T01:26:22.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:22.335+0000 7fe7bbfff700 1 -- 172.21.15.16:0/182343898 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7c4011960 con 0x7fe7d40b5740 2023-05-20T01:26:22.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:22.335+0000 7fe7d982f700 1 -- 172.21.15.16:0/182343898 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7d40cdca0 con 0x7fe7d40b5740 2023-05-20T01:26:22.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:22.335+0000 7fe7bbfff700 1 -- 172.21.15.16:0/182343898 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7c4011d10 con 0x7fe7d40b5740 2023-05-20T01:26:22.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:22.335+0000 7fe7d982f700 1 -- 172.21.15.16:0/182343898 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7d40cd9c0 con 0x7fe7d40b5740 2023-05-20T01:26:22.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:22.336+0000 7fe7d982f700 1 -- 172.21.15.16:0/182343898 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7d40cdf80 con 0x7fe7d40b5740 2023-05-20T01:26:22.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:22.337+0000 7fe7bbfff700 1 -- 172.21.15.16:0/182343898 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe7c4018360 con 0x7fe7d40b5740 2023-05-20T01:26:22.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:22.337+0000 7fe7bbfff700 1 --2- 172.21.15.16:0/182343898 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe7c006ace0 0x7fe7c006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:22.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:22.337+0000 7fe7bbfff700 1 -- 172.21.15.16:0/182343898 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 7102+0+0 (secure 0 0 0) 0x7fe7c4089630 con 0x7fe7d40b5740 2023-05-20T01:26:22.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:22.337+0000 7fe7d2ffd700 1 --2- 172.21.15.16:0/182343898 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe7c006ace0 0x7fe7c006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:22.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:22.337+0000 7fe7d2ffd700 1 --2- 172.21.15.16:0/182343898 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe7c006ace0 0x7fe7c006d190 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fe7c8006fd0 tx=0x7fe7c8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:22.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:22.341+0000 7fe7bbfff700 1 -- 172.21.15.16:0/182343898 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fe7c4054360 con 0x7fe7d40b5740 2023-05-20T01:26:22.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:22.585+0000 7fe7d982f700 1 -- 172.21.15.16:0/182343898 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fe7d404e8c0 con 0x7fe7d40b5740 2023-05-20T01:26:23.538 INFO:tasks.ceph.mon.a.smithi016.stderr:2023-05-20T01:26:23.534+0000 7f49eb8ae700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-05-20T01:26:23.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:23.544+0000 7fe7bbfff700 1 -- 172.21.15.16:0/182343898 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v71) v1 ==== 161+0+0 (secure 0 0 0) 0x7fe7c4057980 con 0x7fe7d40b5740 2023-05-20T01:26:23.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:23.546+0000 7fe7d982f700 1 -- 172.21.15.16:0/182343898 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe7c006ace0 msgr2=0x7fe7c006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:23.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:23.546+0000 7fe7d982f700 1 --2- 172.21.15.16:0/182343898 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe7c006ace0 0x7fe7c006d190 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fe7c8006fd0 tx=0x7fe7c8008040 comp rx=0 tx=0).stop 2023-05-20T01:26:23.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:23.546+0000 7fe7d982f700 1 -- 172.21.15.16:0/182343898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7d40b5740 msgr2=0x7fe7d40b3590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:23.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:23.546+0000 7fe7d982f700 1 --2- 172.21.15.16:0/182343898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7d40b5740 0x7fe7d40b3590 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fe7c400c7f0 tx=0x7fe7c400cb00 comp rx=0 tx=0).stop 2023-05-20T01:26:23.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:23.547+0000 7fe7d982f700 1 -- 172.21.15.16:0/182343898 shutdown_connections 2023-05-20T01:26:23.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:23.547+0000 7fe7d982f700 1 --2- 172.21.15.16:0/182343898 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe7d415cc50 0x7fe7d40b3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:23.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:23.547+0000 7fe7d982f700 1 --2- 172.21.15.16:0/182343898 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe7c006ace0 0x7fe7c006d190 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:23.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:23.547+0000 7fe7d982f700 1 --2- 172.21.15.16:0/182343898 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe7d406b730 0x7fe7d4153e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:23.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:23.547+0000 7fe7d982f700 1 --2- 172.21.15.16:0/182343898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7d40b5740 0x7fe7d40b3590 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:23.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:23.547+0000 7fe7d982f700 1 -- 172.21.15.16:0/182343898 >> 172.21.15.16:0/182343898 conn(0x7fe7d4072860 msgr2=0x7fe7d415b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:23.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:23.547+0000 7fe7d982f700 1 -- 172.21.15.16:0/182343898 shutdown_connections 2023-05-20T01:26:23.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:23.547+0000 7fe7d982f700 1 -- 172.21.15.16:0/182343898 wait complete. 2023-05-20T01:26:23.553 INFO:tasks.ceph.mds.c.smithi016.stderr: -6676> 2023-05-20T01:25:38.568+0000 7fd0cbb5b700 -1 mds.0.openfiles _load_finish got (2) No such file or directory 2023-05-20T01:26:23.553 INFO:teuthology.orchestra.run.smithi016.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-05-20T01:26:23.554 INFO:tasks.ceph.mds.c.smithi016.stderr: -5214> 2023-05-20T01:25:43.567+0000 7fd0cf362700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-05-20T01:26:23.557 INFO:tasks.ceph.mds.c.smithi016.stderr: -2664> 2023-05-20T01:25:44.145+0000 7fd0cbb5b700 -1 log_channel(cluster) log [ERR] : bad backtrace on directory inode 0x10000000000 2023-05-20T01:26:23.557 INFO:tasks.ceph.mds.c.smithi016.stderr: -2528> 2023-05-20T01:25:44.146+0000 7fd0cbb5b700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /subdir/ auth v1 pv4 ap=3 DIRTYPARENT f(v0 1=1+0) n() (inest lock dirty) (ifile lock->sync w=1 flushing) (iversion lock) | dirtyscattered=2 lock=1 dirfrag=1 dirtyparent=1 waiter=1 authpin=1 scrubqueue=0 0x55cf68973b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(12)0x10000000000:[<0x1/subdir v1>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 1=1+0)","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 1=1+0)","memory_value.rstat":"n()","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-61} 2023-05-20T01:26:23.557 INFO:tasks.ceph.mds.c.smithi016.stderr: -2474> 2023-05-20T01:25:44.147+0000 7fd0cbb5b700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /subdir/sixmegs auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55cf689bb700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(14)0x10000000001:[<0x10000000000/sixmegs v10>,<0x1/subdir v12>]//[13]","memoryvalue":"(14)0x10000000001:[<0x10000000000/sixmegs v1>,<0x1/subdir v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-05-20T01:26:23.558 INFO:tasks.ceph.mds.c.smithi016.stderr: -2208> 2023-05-20T01:25:44.154+0000 7fd0cbb5b700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1 [...2,head] / auth v5 ap=5 snaprealm=0x55cf6896ad80 DIRTYPARENT f(v1 1=0+1) n(v0 rc2023-05-20T01:25:43.794316+0000 1=0+1)/n() (idft sync r=1) (inest sync r=1) (ifile sync->mix r=1) (iversion lock) caps={6377=pLsXs/-@45} | dirtyscattered=0 request=0 lock=3 dirfrag=1 caps=1 openingsnapparents=0 dirtyparent=1 dirty=1 waiter=1 authpin=1 scrubqueue=0 0x55cf68973080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(12)0x1:[]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 1=0+1)","ondisk_value.rstat":"n(v0 2=0+2)","memory_value.dirstat":"f(v1 1=0+1)","memory_value.rstat":"n(v0 rc2023-05-20T01:25:43.794316+0000 1=0+1)","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-61} 2023-05-20T01:26:23.567 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-05-20T01:26:23.589 INFO:tasks.ceph.mds.c.smithi016.stdout:starting mds.c at 2023-05-20T01:26:23.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:23.700+0000 7fcc36d45700 1 Processor -- start 2023-05-20T01:26:23.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:23.700+0000 7fcc36d45700 1 -- start start 2023-05-20T01:26:23.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:23.701+0000 7fcc36d45700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fcc300cd4f0 0x7fcc300cf1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:23.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:23.701+0000 7fcc36d45700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc3006dd00 con 0x7fcc3006a510 2023-05-20T01:26:23.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:23.701+0000 7fcc36d45700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc300cfa60 con 0x7fcc3015cec0 2023-05-20T01:26:23.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:23.701+0000 7fcc36d45700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc300cd940 con 0x7fcc300cd4f0 2023-05-20T01:26:23.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:23.701+0000 7fcc352e2700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fcc3006a510 0x7fcc300710d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49252/0 (socket says 172.21.15.16:49252) 2023-05-20T01:26:23.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:23.701+0000 7fcc352e2700 1 -- 172.21.15.16:0/1791454007 learned_addr learned my addr 172.21.15.16:0/1791454007 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:26:23.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:23.701+0000 7fcc2ffff700 1 --2- 172.21.15.16:0/1791454007 >> v2:172.21.15.132:3300/0 conn(0x7fcc300cd4f0 0x7fcc300cf1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:23.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:23.701+0000 7fcc2f7fe700 1 -- 172.21.15.16:0/1791454007 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2760595085 0 0) 0x7fcc3006dd00 con 0x7fcc3006a510 2023-05-20T01:26:23.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:23.701+0000 7fcc2f7fe700 1 -- 172.21.15.16:0/1791454007 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc200035b0 con 0x7fcc3006a510 2023-05-20T01:26:23.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:23.702+0000 7fcc2f7fe700 1 -- 172.21.15.16:0/1791454007 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 187600737 0 0) 0x7fcc200035b0 con 0x7fcc3006a510 2023-05-20T01:26:23.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:23.702+0000 7fcc2ffff700 1 -- 172.21.15.16:0/1791454007 >> v1:172.21.15.132:6789/0 conn(0x7fcc3015cec0 legacy=0x7fcc3006e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:23.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:23.702+0000 7fcc2ffff700 1 -- 172.21.15.16:0/1791454007 >> v1:172.21.15.16:6789/0 conn(0x7fcc3006a510 legacy=0x7fcc300710d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:23.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:23.702+0000 7fcc2ffff700 1 -- 172.21.15.16:0/1791454007 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc3006d6b0 con 0x7fcc300cd4f0 2023-05-20T01:26:23.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:23.702+0000 7fcc2ffff700 1 --2- 172.21.15.16:0/1791454007 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcc300cd4f0 0x7fcc300cf1f0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c009840 tx=0x7fcc1c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f41d99abb6353c42 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:23.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:23.702+0000 7fcc2f7fe700 1 -- 172.21.15.16:0/1791454007 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc280019f0 con 0x7fcc300cd4f0 2023-05-20T01:26:23.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:23.702+0000 7fcc2f7fe700 1 -- 172.21.15.16:0/1791454007 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc1c004030 con 0x7fcc300cd4f0 2023-05-20T01:26:23.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:23.702+0000 7fcc2f7fe700 1 -- 172.21.15.16:0/1791454007 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc1c0043c0 con 0x7fcc300cd4f0 2023-05-20T01:26:23.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:23.703+0000 7fcc36d45700 1 -- 172.21.15.16:0/1791454007 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcc300cd4f0 msgr2=0x7fcc300cf1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:23.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:23.703+0000 7fcc36d45700 1 --2- 172.21.15.16:0/1791454007 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcc300cd4f0 0x7fcc300cf1f0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c009840 tx=0x7fcc1c009b50 comp rx=0 tx=0).stop 2023-05-20T01:26:23.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:23.703+0000 7fcc36d45700 1 -- 172.21.15.16:0/1791454007 shutdown_connections 2023-05-20T01:26:23.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:23.703+0000 7fcc36d45700 1 --2- 172.21.15.16:0/1791454007 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcc300cd4f0 0x7fcc300cf1f0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:23.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:23.703+0000 7fcc36d45700 1 -- 172.21.15.16:0/1791454007 >> 172.21.15.16:0/1791454007 conn(0x7fcc3006c260 msgr2=0x7fcc300cffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:23.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:23.703+0000 7fcc36d45700 1 -- 172.21.15.16:0/1791454007 shutdown_connections 2023-05-20T01:26:23.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:23.703+0000 7fcc36d45700 1 -- 172.21.15.16:0/1791454007 wait complete. 2023-05-20T01:26:23.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:23.704+0000 7fcc36d45700 1 Processor -- start 2023-05-20T01:26:23.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:23.704+0000 7fcc36d45700 1 -- start start 2023-05-20T01:26:23.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:23.705+0000 7fcc36d45700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc3006a510 0x7fcc3015dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:23.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:23.705+0000 7fcc36d45700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcc300cd4f0 0x7fcc300c5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:23.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:23.705+0000 7fcc36d45700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcc3015cec0 0x7fcc3014f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:23.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:23.705+0000 7fcc36d45700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc300c5fb0 con 0x7fcc3006a510 2023-05-20T01:26:23.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:23.705+0000 7fcc36d45700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc300c6290 con 0x7fcc3015cec0 2023-05-20T01:26:23.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:23.705+0000 7fcc36d45700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc300c6570 con 0x7fcc300cd4f0 2023-05-20T01:26:23.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:23.705+0000 7fcc34ae1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc3006a510 0x7fcc3015dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:23.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:23.705+0000 7fcc34ae1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc3006a510 0x7fcc3015dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37728/0 (socket says 172.21.15.16:37728) 2023-05-20T01:26:23.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:23.705+0000 7fcc34ae1700 1 -- 172.21.15.16:0/1911265486 learned_addr learned my addr 172.21.15.16:0/1911265486 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:26:23.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:23.705+0000 7fcc2ffff700 1 --2- 172.21.15.16:0/1911265486 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcc300cd4f0 0x7fcc300c5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:23.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:23.705+0000 7fcc34ae1700 1 -- 172.21.15.16:0/1911265486 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcc300cd4f0 msgr2=0x7fcc300c5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:23.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:23.705+0000 7fcc352e2700 1 --2- 172.21.15.16:0/1911265486 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcc3015cec0 0x7fcc3014f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:23.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:23.705+0000 7fcc34ae1700 1 --2- 172.21.15.16:0/1911265486 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcc300cd4f0 0x7fcc300c5530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:23.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:23.705+0000 7fcc34ae1700 1 -- 172.21.15.16:0/1911265486 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcc3015cec0 msgr2=0x7fcc3014f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:23.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:23.705+0000 7fcc34ae1700 1 --2- 172.21.15.16:0/1911265486 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcc3015cec0 0x7fcc3014f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:23.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:23.705+0000 7fcc34ae1700 1 -- 172.21.15.16:0/1911265486 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc1c0094a0 con 0x7fcc3006a510 2023-05-20T01:26:23.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:23.705+0000 7fcc34ae1700 1 --2- 172.21.15.16:0/1911265486 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc3006a510 0x7fcc3015dfa0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7fcc2400c4e0 tx=0x7fcc2400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:23.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:23.706+0000 7fcc2d7fa700 1 -- 172.21.15.16:0/1911265486 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc240052d0 con 0x7fcc3006a510 2023-05-20T01:26:23.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:23.706+0000 7fcc2d7fa700 1 -- 172.21.15.16:0/1911265486 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc24005430 con 0x7fcc3006a510 2023-05-20T01:26:23.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:23.706+0000 7fcc36d45700 1 -- 172.21.15.16:0/1911265486 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc3015d290 con 0x7fcc3006a510 2023-05-20T01:26:23.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:23.706+0000 7fcc2d7fa700 1 -- 172.21.15.16:0/1911265486 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc240038c0 con 0x7fcc3006a510 2023-05-20T01:26:23.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:23.706+0000 7fcc36d45700 1 -- 172.21.15.16:0/1911265486 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc3006d990 con 0x7fcc3006a510 2023-05-20T01:26:23.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:23.707+0000 7fcc36d45700 1 -- 172.21.15.16:0/1911265486 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc300cfd40 con 0x7fcc3006a510 2023-05-20T01:26:23.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:23.707+0000 7fcc2d7fa700 1 -- 172.21.15.16:0/1911265486 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fcc24003a20 con 0x7fcc3006a510 2023-05-20T01:26:23.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:23.708+0000 7fcc2d7fa700 1 --2- 172.21.15.16:0/1911265486 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fcc1406ace0 0x7fcc1406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:23.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:23.708+0000 7fcc2d7fa700 1 -- 172.21.15.16:0/1911265486 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 7188+0+0 (secure 0 0 0) 0x7fcc24088e10 con 0x7fcc3006a510 2023-05-20T01:26:23.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:23.708+0000 7fcc2ffff700 1 --2- 172.21.15.16:0/1911265486 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fcc1406ace0 0x7fcc1406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:23.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:23.708+0000 7fcc2ffff700 1 --2- 172.21.15.16:0/1911265486 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fcc1406ace0 0x7fcc1406d190 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fcc300cbbb0 tx=0x7fcc1c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:23.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:23.712+0000 7fcc2d7fa700 1 -- 172.21.15.16:0/1911265486 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fcc24053a50 con 0x7fcc3006a510 2023-05-20T01:26:23.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:23.959+0000 7fcc36d45700 1 -- 172.21.15.16:0/1911265486 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fcc300747b0 con 0x7fcc3006a510 2023-05-20T01:26:24.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:24.591+0000 7fcc2d7fa700 1 -- 172.21.15.16:0/1911265486 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v72) v1 ==== 104+0+0 (secure 0 0 0) 0x7fcc24057070 con 0x7fcc3006a510 2023-05-20T01:26:24.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:24.594+0000 7fcc36d45700 1 -- 172.21.15.16:0/1911265486 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fcc1406ace0 msgr2=0x7fcc1406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:24.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:24.594+0000 7fcc36d45700 1 --2- 172.21.15.16:0/1911265486 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fcc1406ace0 0x7fcc1406d190 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fcc300cbbb0 tx=0x7fcc1c008030 comp rx=0 tx=0).stop 2023-05-20T01:26:24.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:24.594+0000 7fcc36d45700 1 -- 172.21.15.16:0/1911265486 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc3006a510 msgr2=0x7fcc3015dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:24.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:24.594+0000 7fcc36d45700 1 --2- 172.21.15.16:0/1911265486 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc3006a510 0x7fcc3015dfa0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7fcc2400c4e0 tx=0x7fcc2400c8a0 comp rx=0 tx=0).stop 2023-05-20T01:26:24.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:24.594+0000 7fcc36d45700 1 -- 172.21.15.16:0/1911265486 shutdown_connections 2023-05-20T01:26:24.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:24.594+0000 7fcc36d45700 1 --2- 172.21.15.16:0/1911265486 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcc3015cec0 0x7fcc3014f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:24.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:24.594+0000 7fcc36d45700 1 --2- 172.21.15.16:0/1911265486 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc3006a510 0x7fcc3015dfa0 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:24.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:24.594+0000 7fcc36d45700 1 --2- 172.21.15.16:0/1911265486 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fcc1406ace0 0x7fcc1406d190 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:24.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:24.594+0000 7fcc36d45700 1 --2- 172.21.15.16:0/1911265486 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcc300cd4f0 0x7fcc300c5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:24.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:24.594+0000 7fcc36d45700 1 -- 172.21.15.16:0/1911265486 >> 172.21.15.16:0/1911265486 conn(0x7fcc3006c260 msgr2=0x7fcc3006aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:24.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:24.595+0000 7fcc36d45700 1 -- 172.21.15.16:0/1911265486 shutdown_connections 2023-05-20T01:26:24.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:24.595+0000 7fcc36d45700 1 -- 172.21.15.16:0/1911265486 wait complete. 2023-05-20T01:26:24.614 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-05-20T01:26:24.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:24.787+0000 7f80d8ce0700 1 Processor -- start 2023-05-20T01:26:24.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:24.787+0000 7f80d8ce0700 1 -- start start 2023-05-20T01:26:24.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:24.788+0000 7f80d8ce0700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f80d40c6b50 0x7f80d406a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:24.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:24.788+0000 7f80d8ce0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f80d415cc00 0x7f80d406aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:24.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:24.788+0000 7f80d8ce0700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80d40cf750 con 0x7f80d415c830 2023-05-20T01:26:24.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:24.788+0000 7f80d8ce0700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80d40d0730 con 0x7f80d40c6b50 2023-05-20T01:26:24.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:24.788+0000 7f80d8ce0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80d40cf980 con 0x7f80d415cc00 2023-05-20T01:26:24.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:24.789+0000 7f80d1d9b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f80d40c6b50 0x7f80d406a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:24.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:24.789+0000 7f80d259c700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f80d415c830 0x7f80d4161070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49266/0 (socket says 172.21.15.16:49266) 2023-05-20T01:26:24.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:24.789+0000 7f80d259c700 1 -- 172.21.15.16:0/2811026377 learned_addr learned my addr 172.21.15.16:0/2811026377 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:26:24.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:24.789+0000 7f80d2d9d700 1 --2- 172.21.15.16:0/2811026377 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f80d415cc00 0x7f80d406aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:24.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:24.789+0000 7f80d1d9b700 1 -- 172.21.15.16:0/2811026377 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f80d415cc00 msgr2=0x7f80d406aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:24.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:24.789+0000 7f80d1d9b700 1 --2- 172.21.15.16:0/2811026377 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f80d415cc00 0x7f80d406aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:24.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:24.789+0000 7f80d1d9b700 1 -- 172.21.15.16:0/2811026377 >> v1:172.21.15.16:6789/0 conn(0x7f80d415c830 legacy=0x7f80d4161070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:24.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:24.789+0000 7f80d159a700 1 -- 172.21.15.16:0/2811026377 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 221314669 0 0) 0x7f80d40cf750 con 0x7f80d415c830 2023-05-20T01:26:24.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:24.789+0000 7f80d1d9b700 1 -- 172.21.15.16:0/2811026377 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80d40cfc60 con 0x7f80d40c6b50 2023-05-20T01:26:24.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:24.789+0000 7f80d2d9d700 1 --2- 172.21.15.16:0/2811026377 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f80d415cc00 0x7f80d406aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:26:24.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:24.789+0000 7f80d1d9b700 1 --2- 172.21.15.16:0/2811026377 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80d40c6b50 0x7f80d406a510 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f80c4009840 tx=0x7f80c4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba7720a9dc293b99 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:24.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:24.789+0000 7f80d159a700 1 -- 172.21.15.16:0/2811026377 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80c4011830 con 0x7f80d40c6b50 2023-05-20T01:26:24.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:24.789+0000 7f80d159a700 1 -- 172.21.15.16:0/2811026377 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80c4011990 con 0x7f80d40c6b50 2023-05-20T01:26:24.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:24.789+0000 7f80d159a700 1 -- 172.21.15.16:0/2811026377 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80c4011d20 con 0x7f80d40c6b50 2023-05-20T01:26:24.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:24.790+0000 7f80d8ce0700 1 -- 172.21.15.16:0/2811026377 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80d40c6b50 msgr2=0x7f80d406a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:24.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:24.790+0000 7f80d8ce0700 1 --2- 172.21.15.16:0/2811026377 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80d40c6b50 0x7f80d406a510 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f80c4009840 tx=0x7f80c4009b50 comp rx=0 tx=0).stop 2023-05-20T01:26:24.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:24.790+0000 7f80d8ce0700 1 -- 172.21.15.16:0/2811026377 shutdown_connections 2023-05-20T01:26:24.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:24.790+0000 7f80d8ce0700 1 --2- 172.21.15.16:0/2811026377 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f80d415cc00 0x7f80d406aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:24.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:24.790+0000 7f80d8ce0700 1 --2- 172.21.15.16:0/2811026377 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80d40c6b50 0x7f80d406a510 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:24.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:24.790+0000 7f80d8ce0700 1 -- 172.21.15.16:0/2811026377 >> 172.21.15.16:0/2811026377 conn(0x7f80d40b2920 msgr2=0x7f80d40d0020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:24.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:24.790+0000 7f80d8ce0700 1 -- 172.21.15.16:0/2811026377 shutdown_connections 2023-05-20T01:26:24.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:24.790+0000 7f80d8ce0700 1 -- 172.21.15.16:0/2811026377 wait complete. 2023-05-20T01:26:24.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:24.792+0000 7f80d8ce0700 1 Processor -- start 2023-05-20T01:26:24.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:24.792+0000 7f80d8ce0700 1 -- start start 2023-05-20T01:26:24.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:24.793+0000 7f80d8ce0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f80d40c6b50 0x7f80d406f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:24.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:24.793+0000 7f80d8ce0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80d415c830 0x7f80d40ca9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:24.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:24.793+0000 7f80d8ce0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f80d415cc00 0x7f80d40c7300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:24.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:24.793+0000 7f80d8ce0700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80d406f030 con 0x7f80d415c830 2023-05-20T01:26:24.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:24.793+0000 7f80d8ce0700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80d406e700 con 0x7f80d415cc00 2023-05-20T01:26:24.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:24.793+0000 7f80d8ce0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80d406e9e0 con 0x7f80d40c6b50 2023-05-20T01:26:24.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:24.793+0000 7f80d1d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80d415c830 0x7f80d40ca9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:24.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:24.793+0000 7f80d1d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80d415c830 0x7f80d40ca9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37746/0 (socket says 172.21.15.16:37746) 2023-05-20T01:26:24.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:24.793+0000 7f80d1d9b700 1 -- 172.21.15.16:0/2726017151 learned_addr learned my addr 172.21.15.16:0/2726017151 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:26:24.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:24.793+0000 7f80d259c700 1 --2- 172.21.15.16:0/2726017151 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f80d40c6b50 0x7f80d406f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:24.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:24.793+0000 7f80d2d9d700 1 --2- 172.21.15.16:0/2726017151 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f80d415cc00 0x7f80d40c7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:24.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:24.793+0000 7f80d1d9b700 1 -- 172.21.15.16:0/2726017151 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f80d40c6b50 msgr2=0x7f80d406f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:24.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:24.793+0000 7f80d1d9b700 1 --2- 172.21.15.16:0/2726017151 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f80d40c6b50 0x7f80d406f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:24.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:24.793+0000 7f80d1d9b700 1 -- 172.21.15.16:0/2726017151 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f80d415cc00 msgr2=0x7f80d40c7300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:24.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:24.793+0000 7f80d1d9b700 1 --2- 172.21.15.16:0/2726017151 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f80d415cc00 0x7f80d40c7300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:24.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:24.793+0000 7f80d1d9b700 1 -- 172.21.15.16:0/2726017151 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80c40094a0 con 0x7f80d415c830 2023-05-20T01:26:24.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:24.793+0000 7f80d2d9d700 1 --2- 172.21.15.16:0/2726017151 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f80d415cc00 0x7f80d40c7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:26:24.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:24.793+0000 7f80d1d9b700 1 --2- 172.21.15.16:0/2726017151 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80d415c830 0x7f80d40ca9c0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f80c40090b0 tx=0x7f80c4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:24.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:24.794+0000 7f80c2ffd700 1 -- 172.21.15.16:0/2726017151 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80c40038a0 con 0x7f80d415c830 2023-05-20T01:26:24.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:24.794+0000 7f80c2ffd700 1 -- 172.21.15.16:0/2726017151 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80c401f070 con 0x7f80d415c830 2023-05-20T01:26:24.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:24.794+0000 7f80d8ce0700 1 -- 172.21.15.16:0/2726017151 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80d40d0c60 con 0x7f80d415c830 2023-05-20T01:26:24.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:24.794+0000 7f80c2ffd700 1 -- 172.21.15.16:0/2726017151 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80c4003c60 con 0x7f80d415c830 2023-05-20T01:26:24.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:24.794+0000 7f80d8ce0700 1 -- 172.21.15.16:0/2726017151 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80d40d0980 con 0x7f80d415c830 2023-05-20T01:26:24.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:24.795+0000 7f80d8ce0700 1 -- 172.21.15.16:0/2726017151 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80d40cded0 con 0x7f80d415c830 2023-05-20T01:26:24.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:24.795+0000 7f80c2ffd700 1 -- 172.21.15.16:0/2726017151 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f80c401a430 con 0x7f80d415c830 2023-05-20T01:26:24.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:24.795+0000 7f80c2ffd700 1 --2- 172.21.15.16:0/2726017151 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f80b406ace0 0x7f80b406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:24.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:24.796+0000 7f80c2ffd700 1 -- 172.21.15.16:0/2726017151 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 7188+0+0 (secure 0 0 0) 0x7f80c4021070 con 0x7f80d415c830 2023-05-20T01:26:24.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:24.796+0000 7f80d259c700 1 --2- 172.21.15.16:0/2726017151 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f80b406ace0 0x7f80b406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:24.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:24.796+0000 7f80d259c700 1 --2- 172.21.15.16:0/2726017151 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f80b406ace0 0x7f80b406d190 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f80c8000f60 tx=0x7f80c8007be0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:24.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:24.800+0000 7f80c2ffd700 1 -- 172.21.15.16:0/2726017151 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f80c4055fd0 con 0x7f80d415c830 2023-05-20T01:26:25.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:24.999+0000 7f80d8ce0700 1 -- 172.21.15.16:0/2726017151 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f80d406e840 con 0x7f80d415c830 2023-05-20T01:26:25.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:25.596+0000 7f80c2ffd700 1 -- 172.21.15.16:0/2726017151 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v101) v1 ==== 181+0+0 (secure 0 0 0) 0x7f80d406e840 con 0x7f80d415c830 2023-05-20T01:26:25.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:25.600+0000 7f80d8ce0700 1 -- 172.21.15.16:0/2726017151 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f80b406ace0 msgr2=0x7f80b406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:25.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:25.600+0000 7f80d8ce0700 1 --2- 172.21.15.16:0/2726017151 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f80b406ace0 0x7f80b406d190 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f80c8000f60 tx=0x7f80c8007be0 comp rx=0 tx=0).stop 2023-05-20T01:26:25.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:25.600+0000 7f80d8ce0700 1 -- 172.21.15.16:0/2726017151 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80d415c830 msgr2=0x7f80d40ca9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:25.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:25.600+0000 7f80d8ce0700 1 --2- 172.21.15.16:0/2726017151 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80d415c830 0x7f80d40ca9c0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f80c40090b0 tx=0x7f80c4004750 comp rx=0 tx=0).stop 2023-05-20T01:26:25.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:25.600+0000 7f80d8ce0700 1 -- 172.21.15.16:0/2726017151 shutdown_connections 2023-05-20T01:26:25.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:25.600+0000 7f80d8ce0700 1 --2- 172.21.15.16:0/2726017151 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f80d415cc00 0x7f80d40c7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:25.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:25.600+0000 7f80d8ce0700 1 --2- 172.21.15.16:0/2726017151 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f80b406ace0 0x7f80b406d190 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:25.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:25.600+0000 7f80d8ce0700 1 --2- 172.21.15.16:0/2726017151 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f80d40c6b50 0x7f80d406f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:25.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:25.600+0000 7f80d8ce0700 1 --2- 172.21.15.16:0/2726017151 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80d415c830 0x7f80d40ca9c0 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:25.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:25.600+0000 7f80d8ce0700 1 -- 172.21.15.16:0/2726017151 >> 172.21.15.16:0/2726017151 conn(0x7f80d40b2920 msgr2=0x7f80d4151f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:25.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:25.600+0000 7f80d8ce0700 1 -- 172.21.15.16:0/2726017151 shutdown_connections 2023-05-20T01:26:25.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:25.600+0000 7f80d8ce0700 1 -- 172.21.15.16:0/2726017151 wait complete. 2023-05-20T01:26:25.608 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' removed 2023-05-20T01:26:25.619 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-05-20T01:26:25.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:25.751+0000 7f74991c5700 1 Processor -- start 2023-05-20T01:26:25.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:25.751+0000 7f74991c5700 1 -- start start 2023-05-20T01:26:25.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:25.752+0000 7f74991c5700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f74940b2670 0x7f74940b60d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:25.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:25.752+0000 7f74991c5700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f74940d3a50 0x7f74940cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:25.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:25.752+0000 7f74991c5700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74940d2eb0 con 0x7f74940d1860 2023-05-20T01:26:25.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:25.752+0000 7f74991c5700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74940cd970 con 0x7f74940d3a50 2023-05-20T01:26:25.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:25.752+0000 7f74991c5700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74940cdc50 con 0x7f74940b2670 2023-05-20T01:26:25.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:25.752+0000 7f749359e700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f74940d3a50 0x7f74940cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:25.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:25.752+0000 7f749259c700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f74940d1860 0x7f74940d3340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49280/0 (socket says 172.21.15.16:49280) 2023-05-20T01:26:25.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:25.752+0000 7f749259c700 1 -- 172.21.15.16:0/2512232757 learned_addr learned my addr 172.21.15.16:0/2512232757 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:26:25.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:25.752+0000 7f7492d9d700 1 --2- 172.21.15.16:0/2512232757 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f74940b2670 0x7f74940b60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:25.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:25.752+0000 7f749359e700 1 -- 172.21.15.16:0/2512232757 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f74940b2670 msgr2=0x7f74940b60d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:25.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:25.752+0000 7f749359e700 1 --2- 172.21.15.16:0/2512232757 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f74940b2670 0x7f74940b60d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:25.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:25.752+0000 7f749359e700 1 -- 172.21.15.16:0/2512232757 >> v1:172.21.15.16:6789/0 conn(0x7f74940d1860 legacy=0x7f74940d3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:25.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:25.752+0000 7f7491d9b700 1 -- 172.21.15.16:0/2512232757 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 891682709 0 0) 0x7f74940d2eb0 con 0x7f74940d1860 2023-05-20T01:26:25.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:25.752+0000 7f749359e700 1 -- 172.21.15.16:0/2512232757 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74940d2b40 con 0x7f74940d3a50 2023-05-20T01:26:25.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:25.752+0000 7f7492d9d700 1 --2- 172.21.15.16:0/2512232757 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f74940b2670 0x7f74940b60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:26:25.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:25.752+0000 7f749359e700 1 --2- 172.21.15.16:0/2512232757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74940d3a50 0x7f74940cf7d0 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f748c009d90 tx=0x7f748c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e3fd1340680c7c5 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:25.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:25.753+0000 7f7491d9b700 1 -- 172.21.15.16:0/2512232757 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f748c00e070 con 0x7f74940d3a50 2023-05-20T01:26:25.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:25.753+0000 7f7491d9b700 1 -- 172.21.15.16:0/2512232757 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f748c00bbd0 con 0x7f74940d3a50 2023-05-20T01:26:25.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:25.753+0000 7f7491d9b700 1 -- 172.21.15.16:0/2512232757 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f748c012430 con 0x7f74940d3a50 2023-05-20T01:26:25.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:25.753+0000 7f74991c5700 1 -- 172.21.15.16:0/2512232757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74940d3a50 msgr2=0x7f74940cf7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:25.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:25.753+0000 7f74991c5700 1 --2- 172.21.15.16:0/2512232757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74940d3a50 0x7f74940cf7d0 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f748c009d90 tx=0x7f748c00b330 comp rx=0 tx=0).stop 2023-05-20T01:26:25.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:25.753+0000 7f74991c5700 1 -- 172.21.15.16:0/2512232757 shutdown_connections 2023-05-20T01:26:25.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:25.753+0000 7f74991c5700 1 --2- 172.21.15.16:0/2512232757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74940d3a50 0x7f74940cf7d0 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:25.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:25.753+0000 7f74991c5700 1 --2- 172.21.15.16:0/2512232757 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f74940b2670 0x7f74940b60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:25.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:25.753+0000 7f74991c5700 1 -- 172.21.15.16:0/2512232757 >> 172.21.15.16:0/2512232757 conn(0x7f7494071c40 msgr2=0x7f7494072010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:25.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:25.753+0000 7f74991c5700 1 -- 172.21.15.16:0/2512232757 shutdown_connections 2023-05-20T01:26:25.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:25.753+0000 7f74991c5700 1 -- 172.21.15.16:0/2512232757 wait complete. 2023-05-20T01:26:25.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:25.755+0000 7f74991c5700 1 Processor -- start 2023-05-20T01:26:25.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:25.755+0000 7f74991c5700 1 -- start start 2023-05-20T01:26:25.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:25.755+0000 7f74991c5700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f74940b2670 0x7f74941497a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:25.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:25.755+0000 7f74991c5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74940d1860 0x7f74940c9900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:25.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:25.755+0000 7f74991c5700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f74940d3a50 0x7f74940b35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:25.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:25.755+0000 7f74991c5700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74940ca7c0 con 0x7f74940d1860 2023-05-20T01:26:25.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:25.755+0000 7f74991c5700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74940caaa0 con 0x7f74940b2670 2023-05-20T01:26:25.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:25.755+0000 7f74991c5700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74940cad80 con 0x7f74940d3a50 2023-05-20T01:26:25.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:25.756+0000 7f749259c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74940d1860 0x7f74940c9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:25.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:25.756+0000 7f749259c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74940d1860 0x7f74940c9900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37758/0 (socket says 172.21.15.16:37758) 2023-05-20T01:26:25.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:25.756+0000 7f749259c700 1 -- 172.21.15.16:0/581304449 learned_addr learned my addr 172.21.15.16:0/581304449 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:26:25.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:25.756+0000 7f749359e700 1 --2- 172.21.15.16:0/581304449 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f74940d3a50 0x7f74940b35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:25.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:25.756+0000 7f7492d9d700 1 --2- 172.21.15.16:0/581304449 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f74940b2670 0x7f74941497a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:25.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:25.756+0000 7f749259c700 1 -- 172.21.15.16:0/581304449 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f74940d3a50 msgr2=0x7f74940b35e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:25.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:25.756+0000 7f749259c700 1 --2- 172.21.15.16:0/581304449 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f74940d3a50 0x7f74940b35e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:25.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:25.756+0000 7f749259c700 1 -- 172.21.15.16:0/581304449 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f74940b2670 msgr2=0x7f74941497a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:25.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:25.756+0000 7f749259c700 1 --2- 172.21.15.16:0/581304449 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f74940b2670 0x7f74941497a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:25.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:25.756+0000 7f749259c700 1 -- 172.21.15.16:0/581304449 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f748c009a70 con 0x7f74940d1860 2023-05-20T01:26:25.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:25.756+0000 7f749259c700 1 --2- 172.21.15.16:0/581304449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74940d1860 0x7f74940c9900 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f74840011e0 tx=0x7f748400ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:25.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:25.756+0000 7f747b7fe700 1 -- 172.21.15.16:0/581304449 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7484004c80 con 0x7f74940d1860 2023-05-20T01:26:25.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:25.756+0000 7f747b7fe700 1 -- 172.21.15.16:0/581304449 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7484004e00 con 0x7f74940d1860 2023-05-20T01:26:25.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:25.756+0000 7f74991c5700 1 -- 172.21.15.16:0/581304449 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74940b2a40 con 0x7f74940d1860 2023-05-20T01:26:25.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:25.756+0000 7f747b7fe700 1 -- 172.21.15.16:0/581304449 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7484007310 con 0x7f74940d1860 2023-05-20T01:26:25.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:25.756+0000 7f74991c5700 1 -- 172.21.15.16:0/581304449 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74940cdf30 con 0x7f74940d1860 2023-05-20T01:26:25.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:25.757+0000 7f74991c5700 1 -- 172.21.15.16:0/581304449 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74940d1cb0 con 0x7f74940d1860 2023-05-20T01:26:25.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:25.758+0000 7f747b7fe700 1 -- 172.21.15.16:0/581304449 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7484007470 con 0x7f74940d1860 2023-05-20T01:26:25.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:25.758+0000 7f747b7fe700 1 --2- 172.21.15.16:0/581304449 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f748006ace0 0x7f748006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:25.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:25.758+0000 7f747b7fe700 1 -- 172.21.15.16:0/581304449 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6750+0+0 (secure 0 0 0) 0x7f74840892f0 con 0x7f74940d1860 2023-05-20T01:26:25.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:25.758+0000 7f7492d9d700 1 --2- 172.21.15.16:0/581304449 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f748006ace0 0x7f748006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:25.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:25.759+0000 7f7492d9d700 1 --2- 172.21.15.16:0/581304449 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f748006ace0 0x7f748006d190 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f7488006fd0 tx=0x7f7488008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:25.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:25.763+0000 7f747b7fe700 1 -- 172.21.15.16:0/581304449 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f7484054180 con 0x7f74940d1860 2023-05-20T01:26:25.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:25.959+0000 7f74991c5700 1 -- 172.21.15.16:0/581304449 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f749404e8c0 con 0x7f74940d1860 2023-05-20T01:26:26.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:26.604+0000 7f747b7fe700 1 -- 172.21.15.16:0/581304449 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v102) v1 ==== 169+0+0 (secure 0 0 0) 0x7f74840577a0 con 0x7f74940d1860 2023-05-20T01:26:26.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:26.608+0000 7f74991c5700 1 -- 172.21.15.16:0/581304449 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f748006ace0 msgr2=0x7f748006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:26.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:26.608+0000 7f74991c5700 1 --2- 172.21.15.16:0/581304449 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f748006ace0 0x7f748006d190 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f7488006fd0 tx=0x7f7488008040 comp rx=0 tx=0).stop 2023-05-20T01:26:26.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:26.608+0000 7f74991c5700 1 -- 172.21.15.16:0/581304449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74940d1860 msgr2=0x7f74940c9900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:26.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:26.608+0000 7f74991c5700 1 --2- 172.21.15.16:0/581304449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74940d1860 0x7f74940c9900 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f74840011e0 tx=0x7f748400ca30 comp rx=0 tx=0).stop 2023-05-20T01:26:26.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:26.609+0000 7f74991c5700 1 -- 172.21.15.16:0/581304449 shutdown_connections 2023-05-20T01:26:26.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:26.609+0000 7f74991c5700 1 --2- 172.21.15.16:0/581304449 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f748006ace0 0x7f748006d190 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:26.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:26.609+0000 7f74991c5700 1 --2- 172.21.15.16:0/581304449 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f74940d3a50 0x7f74940b35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:26.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:26.609+0000 7f74991c5700 1 --2- 172.21.15.16:0/581304449 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f74940b2670 0x7f74941497a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:26.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:26.609+0000 7f74991c5700 1 --2- 172.21.15.16:0/581304449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74940d1860 0x7f74940c9900 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:26.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:26.609+0000 7f74991c5700 1 -- 172.21.15.16:0/581304449 >> 172.21.15.16:0/581304449 conn(0x7f7494071c40 msgr2=0x7f74940b2bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:26.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:26.610+0000 7f74991c5700 1 -- 172.21.15.16:0/581304449 shutdown_connections 2023-05-20T01:26:26.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:26.610+0000 7f74991c5700 1 -- 172.21.15.16:0/581304449 wait complete. 2023-05-20T01:26:26.616 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' removed 2023-05-20T01:26:26.632 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_extra cephfs_data_extra --yes-i-really-really-mean-it 2023-05-20T01:26:26.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:26.765+0000 7fb18393c700 1 Processor -- start 2023-05-20T01:26:26.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:26.765+0000 7fb18393c700 1 -- start start 2023-05-20T01:26:26.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:26.766+0000 7fb18393c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb17c06ce60 0x7fb17c15efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:26.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:26.766+0000 7fb18393c700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fb17c0ccca0 0x7fb17c0b1720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:26.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:26.766+0000 7fb18393c700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb17c0b1f70 con 0x7fb17c06d230 2023-05-20T01:26:26.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:26.766+0000 7fb18393c700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb17c15ebb0 con 0x7fb17c06ce60 2023-05-20T01:26:26.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:26.766+0000 7fb18393c700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb17c15e4f0 con 0x7fb17c0ccca0 2023-05-20T01:26:26.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:26.766+0000 7fb1816d8700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb17c06ce60 0x7fb17c15efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:26.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:26.766+0000 7fb1816d8700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb17c06ce60 0x7fb17c15efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37764/0 (socket says 172.21.15.16:37764) 2023-05-20T01:26:26.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:26.766+0000 7fb1816d8700 1 -- 172.21.15.16:0/110861197 learned_addr learned my addr 172.21.15.16:0/110861197 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:26:26.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:26.766+0000 7fb180ed7700 1 --2- 172.21.15.16:0/110861197 >> v2:172.21.15.132:3300/0 conn(0x7fb17c0ccca0 0x7fb17c0b1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:26.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:26.766+0000 7fb1816d8700 1 -- 172.21.15.16:0/110861197 >> v2:172.21.15.132:3300/0 conn(0x7fb17c0ccca0 msgr2=0x7fb17c0b1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:26.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:26.766+0000 7fb1816d8700 1 --2- 172.21.15.16:0/110861197 >> v2:172.21.15.132:3300/0 conn(0x7fb17c0ccca0 0x7fb17c0b1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:26.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:26.766+0000 7fb1816d8700 1 -- 172.21.15.16:0/110861197 >> v1:172.21.15.132:6789/0 conn(0x7fb17c06d230 legacy=0x7fb17c0cf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:26.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:26.766+0000 7fb1816d8700 1 -- 172.21.15.16:0/110861197 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb17c15e7d0 con 0x7fb17c06ce60 2023-05-20T01:26:26.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:26.766+0000 7fb180ed7700 1 --2- 172.21.15.16:0/110861197 >> v2:172.21.15.132:3300/0 conn(0x7fb17c0ccca0 0x7fb17c0b1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:26:26.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:26.766+0000 7fb1816d8700 1 --2- 172.21.15.16:0/110861197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb17c06ce60 0x7fb17c15efd0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7fb17c074210 tx=0x7fb178009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9eaf4d04931840c3 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:26.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:26.766+0000 7fb173fff700 1 -- 172.21.15.16:0/110861197 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb178005e30 con 0x7fb17c06ce60 2023-05-20T01:26:26.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:26.766+0000 7fb173fff700 1 -- 172.21.15.16:0/110861197 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb178006030 con 0x7fb17c06ce60 2023-05-20T01:26:26.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:26.766+0000 7fb173fff700 1 -- 172.21.15.16:0/110861197 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1780063e0 con 0x7fb17c06ce60 2023-05-20T01:26:26.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:26.767+0000 7fb18393c700 1 -- 172.21.15.16:0/110861197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb17c06ce60 msgr2=0x7fb17c15efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:26.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:26.767+0000 7fb18393c700 1 --2- 172.21.15.16:0/110861197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb17c06ce60 0x7fb17c15efd0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7fb17c074210 tx=0x7fb178009bd0 comp rx=0 tx=0).stop 2023-05-20T01:26:26.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:26.767+0000 7fb18393c700 1 -- 172.21.15.16:0/110861197 shutdown_connections 2023-05-20T01:26:26.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:26.767+0000 7fb18393c700 1 --2- 172.21.15.16:0/110861197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb17c06ce60 0x7fb17c15efd0 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:26.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:26.767+0000 7fb18393c700 1 --2- 172.21.15.16:0/110861197 >> v2:172.21.15.132:3300/0 conn(0x7fb17c0ccca0 0x7fb17c0b1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:26.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:26.767+0000 7fb18393c700 1 -- 172.21.15.16:0/110861197 >> 172.21.15.16:0/110861197 conn(0x7fb17c0661b0 msgr2=0x7fb17c0665b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:26.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:26.767+0000 7fb18393c700 1 -- 172.21.15.16:0/110861197 shutdown_connections 2023-05-20T01:26:26.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:26.767+0000 7fb18393c700 1 -- 172.21.15.16:0/110861197 wait complete. 2023-05-20T01:26:26.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:26.768+0000 7fb18393c700 1 Processor -- start 2023-05-20T01:26:26.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:26.768+0000 7fb18393c700 1 -- start start 2023-05-20T01:26:26.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:26.769+0000 7fb18393c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb17c06ce60 0x7fb17c13fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:26.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:26.769+0000 7fb18393c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb17c06d230 0x7fb17c15b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:26.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:26.769+0000 7fb18393c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb17c0ccca0 0x7fb17c154310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:26.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:26.769+0000 7fb18393c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb17c15cdf0 con 0x7fb17c0ccca0 2023-05-20T01:26:26.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:26.769+0000 7fb18393c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb17c15bd50 con 0x7fb17c06ce60 2023-05-20T01:26:26.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:26.769+0000 7fb18393c700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb17c15c030 con 0x7fb17c06d230 2023-05-20T01:26:26.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:26.769+0000 7fb181ed9700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb17c0ccca0 0x7fb17c154310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:26.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:26.769+0000 7fb181ed9700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb17c0ccca0 0x7fb17c154310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37774/0 (socket says 172.21.15.16:37774) 2023-05-20T01:26:26.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:26.769+0000 7fb181ed9700 1 -- 172.21.15.16:0/3881187819 learned_addr learned my addr 172.21.15.16:0/3881187819 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:26:26.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:26.769+0000 7fb180ed7700 1 --2- 172.21.15.16:0/3881187819 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb17c06d230 0x7fb17c15b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:26.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:26.769+0000 7fb1816d8700 1 --2- 172.21.15.16:0/3881187819 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb17c06ce60 0x7fb17c13fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:26.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:26.769+0000 7fb181ed9700 1 -- 172.21.15.16:0/3881187819 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb17c06d230 msgr2=0x7fb17c15b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:26.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:26.769+0000 7fb181ed9700 1 --2- 172.21.15.16:0/3881187819 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb17c06d230 0x7fb17c15b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:26.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:26.769+0000 7fb181ed9700 1 -- 172.21.15.16:0/3881187819 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb17c06ce60 msgr2=0x7fb17c13fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:26.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:26.769+0000 7fb181ed9700 1 --2- 172.21.15.16:0/3881187819 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb17c06ce60 0x7fb17c13fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:26.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:26.769+0000 7fb181ed9700 1 -- 172.21.15.16:0/3881187819 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb178009500 con 0x7fb17c0ccca0 2023-05-20T01:26:26.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:26.769+0000 7fb1816d8700 1 --2- 172.21.15.16:0/3881187819 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb17c06ce60 0x7fb17c13fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:26:26.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:26.770+0000 7fb181ed9700 1 --2- 172.21.15.16:0/3881187819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb17c0ccca0 0x7fb17c154310 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7fb17400c7a0 tx=0x7fb17400cb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:26.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:26.770+0000 7fb171ffb700 1 -- 172.21.15.16:0/3881187819 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb174003e10 con 0x7fb17c0ccca0 2023-05-20T01:26:26.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:26.770+0000 7fb171ffb700 1 -- 172.21.15.16:0/3881187819 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb174005070 con 0x7fb17c0ccca0 2023-05-20T01:26:26.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:26.770+0000 7fb18393c700 1 -- 172.21.15.16:0/3881187819 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb17c0b2250 con 0x7fb17c0ccca0 2023-05-20T01:26:26.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:26.770+0000 7fb171ffb700 1 -- 172.21.15.16:0/3881187819 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb174005420 con 0x7fb17c0ccca0 2023-05-20T01:26:26.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:26.770+0000 7fb18393c700 1 -- 172.21.15.16:0/3881187819 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb17c0b1c60 con 0x7fb17c0ccca0 2023-05-20T01:26:26.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:26.771+0000 7fb18393c700 1 -- 172.21.15.16:0/3881187819 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb17c0cfd90 con 0x7fb17c0ccca0 2023-05-20T01:26:26.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:26.771+0000 7fb171ffb700 1 -- 172.21.15.16:0/3881187819 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb174005580 con 0x7fb17c0ccca0 2023-05-20T01:26:26.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:26.772+0000 7fb171ffb700 1 --2- 172.21.15.16:0/3881187819 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb16006ace0 0x7fb16006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:26.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:26.772+0000 7fb171ffb700 1 -- 172.21.15.16:0/3881187819 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6278+0+0 (secure 0 0 0) 0x7fb174088c60 con 0x7fb17c0ccca0 2023-05-20T01:26:26.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:26.772+0000 7fb1816d8700 1 --2- 172.21.15.16:0/3881187819 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb16006ace0 0x7fb16006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:26.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:26.772+0000 7fb1816d8700 1 --2- 172.21.15.16:0/3881187819 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb16006ace0 0x7fb16006d190 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fb17c074210 tx=0x7fb178008070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:26.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:26.776+0000 7fb171ffb700 1 -- 172.21.15.16:0/3881187819 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fb174053cc0 con 0x7fb17c0ccca0 2023-05-20T01:26:26.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:26.974+0000 7fb18393c700 1 -- 172.21.15.16:0/3881187819 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_extra", "pool2": "cephfs_data_extra", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb17c0747b0 con 0x7fb17c0ccca0 2023-05-20T01:26:27.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:27.610+0000 7fb171ffb700 1 -- 172.21.15.16:0/3881187819 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_extra", "pool2": "cephfs_data_extra", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_extra' removed v103) v1 ==== 187+0+0 (secure 0 0 0) 0x7fb174017020 con 0x7fb17c0ccca0 2023-05-20T01:26:27.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:27.615+0000 7fb18393c700 1 -- 172.21.15.16:0/3881187819 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb16006ace0 msgr2=0x7fb16006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:27.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:27.615+0000 7fb18393c700 1 --2- 172.21.15.16:0/3881187819 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb16006ace0 0x7fb16006d190 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fb17c074210 tx=0x7fb178008070 comp rx=0 tx=0).stop 2023-05-20T01:26:27.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:27.615+0000 7fb18393c700 1 -- 172.21.15.16:0/3881187819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb17c0ccca0 msgr2=0x7fb17c154310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:27.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:27.615+0000 7fb18393c700 1 --2- 172.21.15.16:0/3881187819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb17c0ccca0 0x7fb17c154310 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7fb17400c7a0 tx=0x7fb17400cb60 comp rx=0 tx=0).stop 2023-05-20T01:26:27.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:27.616+0000 7fb18393c700 1 -- 172.21.15.16:0/3881187819 shutdown_connections 2023-05-20T01:26:27.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:27.616+0000 7fb18393c700 1 --2- 172.21.15.16:0/3881187819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb17c0ccca0 0x7fb17c154310 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:27.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:27.616+0000 7fb18393c700 1 --2- 172.21.15.16:0/3881187819 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb17c06ce60 0x7fb17c13fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:27.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:27.616+0000 7fb18393c700 1 --2- 172.21.15.16:0/3881187819 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb16006ace0 0x7fb16006d190 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:27.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:27.616+0000 7fb18393c700 1 --2- 172.21.15.16:0/3881187819 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb17c06d230 0x7fb17c15b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:27.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:27.616+0000 7fb18393c700 1 -- 172.21.15.16:0/3881187819 >> 172.21.15.16:0/3881187819 conn(0x7fb17c0661b0 msgr2=0x7fb17c0c5840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:27.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:27.616+0000 7fb18393c700 1 -- 172.21.15.16:0/3881187819 shutdown_connections 2023-05-20T01:26:27.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:27.616+0000 7fb18393c700 1 -- 172.21.15.16:0/3881187819 wait complete. 2023-05-20T01:26:27.622 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data_extra' removed 2023-05-20T01:26:27.635 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-05-20T01:26:27.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:27.765+0000 7f646c45f700 1 Processor -- start 2023-05-20T01:26:27.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:27.765+0000 7f646c45f700 1 -- start start 2023-05-20T01:26:27.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:27.766+0000 7f646c45f700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f64640cfcb0 0x7f64640d0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:27.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:27.766+0000 7f646c45f700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f64640cd920 0x7f64640cdd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:27.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:27.766+0000 7f646c45f700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64640ce2c0 con 0x7f646415cd50 2023-05-20T01:26:27.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:27.766+0000 7f646c45f700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64640d0660 con 0x7f64640cd920 2023-05-20T01:26:27.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:27.766+0000 7f646c45f700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64640d1c10 con 0x7f64640cfcb0 2023-05-20T01:26:27.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:27.766+0000 7f646a9fc700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f64640cd920 0x7f64640cdd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:27.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:27.766+0000 7f646a9fc700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f64640cd920 0x7f64640cdd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37782/0 (socket says 172.21.15.16:37782) 2023-05-20T01:26:27.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:27.766+0000 7f646a9fc700 1 -- 172.21.15.16:0/1207523569 learned_addr learned my addr 172.21.15.16:0/1207523569 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:26:27.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:27.767+0000 7f64699fa700 1 --2- 172.21.15.16:0/1207523569 >> v2:172.21.15.132:3300/0 conn(0x7f64640cfcb0 0x7f64640d0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:27.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:27.767+0000 7f646a9fc700 1 -- 172.21.15.16:0/1207523569 >> v2:172.21.15.132:3300/0 conn(0x7f64640cfcb0 msgr2=0x7f64640d0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:27.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:27.767+0000 7f646a9fc700 1 --2- 172.21.15.16:0/1207523569 >> v2:172.21.15.132:3300/0 conn(0x7f64640cfcb0 0x7f64640d0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:27.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:27.767+0000 7f646a9fc700 1 -- 172.21.15.16:0/1207523569 >> v1:172.21.15.132:6789/0 conn(0x7f646415cd50 legacy=0x7f64640d20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:27.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:27.767+0000 7f646a9fc700 1 -- 172.21.15.16:0/1207523569 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f646415d120 con 0x7f64640cd920 2023-05-20T01:26:27.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:27.767+0000 7f64699fa700 1 --2- 172.21.15.16:0/1207523569 >> v2:172.21.15.132:3300/0 conn(0x7f64640cfcb0 0x7f64640d0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:26:27.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:27.767+0000 7f646a9fc700 1 --2- 172.21.15.16:0/1207523569 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64640cd920 0x7f64640cdd80 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f6464067580 tx=0x7f645c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e5ac14f77295955 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:27.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:27.767+0000 7f64691f9700 1 -- 172.21.15.16:0/1207523569 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f645c00e070 con 0x7f64640cd920 2023-05-20T01:26:27.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:27.767+0000 7f64691f9700 1 -- 172.21.15.16:0/1207523569 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f645c003a20 con 0x7f64640cd920 2023-05-20T01:26:27.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:27.767+0000 7f64691f9700 1 -- 172.21.15.16:0/1207523569 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f645c003dd0 con 0x7f64640cd920 2023-05-20T01:26:27.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:27.768+0000 7f646c45f700 1 -- 172.21.15.16:0/1207523569 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64640cd920 msgr2=0x7f64640cdd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:27.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:27.768+0000 7f646c45f700 1 --2- 172.21.15.16:0/1207523569 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64640cd920 0x7f64640cdd80 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f6464067580 tx=0x7f645c00b330 comp rx=0 tx=0).stop 2023-05-20T01:26:27.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:27.768+0000 7f646c45f700 1 -- 172.21.15.16:0/1207523569 shutdown_connections 2023-05-20T01:26:27.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:27.768+0000 7f646c45f700 1 --2- 172.21.15.16:0/1207523569 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64640cd920 0x7f64640cdd80 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:27.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:27.768+0000 7f646c45f700 1 --2- 172.21.15.16:0/1207523569 >> v2:172.21.15.132:3300/0 conn(0x7f64640cfcb0 0x7f64640d0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:27.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:27.768+0000 7f646c45f700 1 -- 172.21.15.16:0/1207523569 >> 172.21.15.16:0/1207523569 conn(0x7f646406b070 msgr2=0x7f646406b470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:27.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:27.768+0000 7f646c45f700 1 -- 172.21.15.16:0/1207523569 shutdown_connections 2023-05-20T01:26:27.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:27.768+0000 7f646c45f700 1 -- 172.21.15.16:0/1207523569 wait complete. 2023-05-20T01:26:27.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:27.769+0000 7f646c45f700 1 Processor -- start 2023-05-20T01:26:27.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:27.769+0000 7f646c45f700 1 -- start start 2023-05-20T01:26:27.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:27.770+0000 7f646c45f700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f64640cd920 0x7f64640b56d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:27.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:27.770+0000 7f646c45f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64640cfcb0 0x7f6464158c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:27.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:27.770+0000 7f646c45f700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f646415cd50 0x7f646415c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:27.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:27.770+0000 7f646c45f700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f646415c620 con 0x7f64640cfcb0 2023-05-20T01:26:27.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:27.770+0000 7f646c45f700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64640cbd30 con 0x7f646415cd50 2023-05-20T01:26:27.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:27.770+0000 7f646c45f700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6464159320 con 0x7f64640cd920 2023-05-20T01:26:27.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:27.770+0000 7f64699fa700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64640cfcb0 0x7f6464158c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:27.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:27.770+0000 7f64699fa700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64640cfcb0 0x7f6464158c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37786/0 (socket says 172.21.15.16:37786) 2023-05-20T01:26:27.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:27.770+0000 7f64699fa700 1 -- 172.21.15.16:0/3518739003 learned_addr learned my addr 172.21.15.16:0/3518739003 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:26:27.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:27.770+0000 7f646a9fc700 1 --2- 172.21.15.16:0/3518739003 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f646415cd50 0x7f646415c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:27.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:27.770+0000 7f646a1fb700 1 --2- 172.21.15.16:0/3518739003 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f64640cd920 0x7f64640b56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:27.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:27.770+0000 7f64699fa700 1 -- 172.21.15.16:0/3518739003 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f64640cd920 msgr2=0x7f64640b56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:27.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:27.770+0000 7f64699fa700 1 --2- 172.21.15.16:0/3518739003 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f64640cd920 0x7f64640b56d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:27.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:27.770+0000 7f64699fa700 1 -- 172.21.15.16:0/3518739003 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f646415cd50 msgr2=0x7f646415c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:27.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:27.770+0000 7f64699fa700 1 --2- 172.21.15.16:0/3518739003 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f646415cd50 0x7f646415c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:27.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:27.770+0000 7f64699fa700 1 -- 172.21.15.16:0/3518739003 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f645c009b30 con 0x7f64640cfcb0 2023-05-20T01:26:27.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:27.770+0000 7f64699fa700 1 --2- 172.21.15.16:0/3518739003 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64640cfcb0 0x7f6464158c20 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f646000c750 tx=0x7f646000ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:27.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:27.770+0000 7f645affd700 1 -- 172.21.15.16:0/3518739003 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6460011800 con 0x7f64640cfcb0 2023-05-20T01:26:27.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:27.771+0000 7f645affd700 1 -- 172.21.15.16:0/3518739003 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6460011960 con 0x7f64640cfcb0 2023-05-20T01:26:27.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:27.771+0000 7f646c45f700 1 -- 172.21.15.16:0/3518739003 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64640d15c0 con 0x7f64640cfcb0 2023-05-20T01:26:27.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:27.771+0000 7f645affd700 1 -- 172.21.15.16:0/3518739003 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6460011cd0 con 0x7f64640cfcb0 2023-05-20T01:26:27.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:27.771+0000 7f646c45f700 1 -- 172.21.15.16:0/3518739003 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64640d12e0 con 0x7f64640cfcb0 2023-05-20T01:26:27.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:27.772+0000 7f646c45f700 1 -- 172.21.15.16:0/3518739003 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64640d18a0 con 0x7f64640cfcb0 2023-05-20T01:26:27.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:27.772+0000 7f645affd700 1 -- 172.21.15.16:0/3518739003 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6460023480 con 0x7f64640cfcb0 2023-05-20T01:26:27.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:27.773+0000 7f645affd700 1 --2- 172.21.15.16:0/3518739003 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f644806ace0 0x7f644806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:27.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:27.773+0000 7f645affd700 1 -- 172.21.15.16:0/3518739003 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5874+0+0 (secure 0 0 0) 0x7f6460089580 con 0x7f64640cfcb0 2023-05-20T01:26:27.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:27.773+0000 7f646a1fb700 1 --2- 172.21.15.16:0/3518739003 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f644806ace0 0x7f644806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:27.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:27.773+0000 7f646a1fb700 1 --2- 172.21.15.16:0/3518739003 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f644806ace0 0x7f644806d190 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f6454006fd0 tx=0x7f64540090e0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:27.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:27.777+0000 7f645affd700 1 -- 172.21.15.16:0/3518739003 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f64600582e0 con 0x7f64640cfcb0 2023-05-20T01:26:27.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:27.973+0000 7f646c45f700 1 -- 172.21.15.16:0/3518739003 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f64640d0aa0 con 0x7f64640cfcb0 2023-05-20T01:26:27.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:27.977+0000 7f645affd700 1 -- 172.21.15.16:0/3518739003 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) v1 ==== 104+0+0 (secure 0 0 0) 0x7f6460057e70 con 0x7f64640cfcb0 2023-05-20T01:26:27.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:27.979+0000 7f646c45f700 1 -- 172.21.15.16:0/3518739003 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f644806ace0 msgr2=0x7f644806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:27.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:27.979+0000 7f646c45f700 1 --2- 172.21.15.16:0/3518739003 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f644806ace0 0x7f644806d190 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f6454006fd0 tx=0x7f64540090e0 comp rx=0 tx=0).stop 2023-05-20T01:26:27.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:27.979+0000 7f646c45f700 1 -- 172.21.15.16:0/3518739003 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64640cfcb0 msgr2=0x7f6464158c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:27.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:27.979+0000 7f646c45f700 1 --2- 172.21.15.16:0/3518739003 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64640cfcb0 0x7f6464158c20 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f646000c750 tx=0x7f646000ca60 comp rx=0 tx=0).stop 2023-05-20T01:26:27.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:27.979+0000 7f646c45f700 1 -- 172.21.15.16:0/3518739003 shutdown_connections 2023-05-20T01:26:27.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:27.979+0000 7f646c45f700 1 --2- 172.21.15.16:0/3518739003 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f646415cd50 0x7f646415c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:27.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:27.979+0000 7f646c45f700 1 --2- 172.21.15.16:0/3518739003 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f644806ace0 0x7f644806d190 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:27.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:27.979+0000 7f646c45f700 1 --2- 172.21.15.16:0/3518739003 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f64640cd920 0x7f64640b56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:27.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:27.979+0000 7f646c45f700 1 --2- 172.21.15.16:0/3518739003 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64640cfcb0 0x7f6464158c20 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:27.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:27.979+0000 7f646c45f700 1 -- 172.21.15.16:0/3518739003 >> 172.21.15.16:0/3518739003 conn(0x7f646406b070 msgr2=0x7f64641581d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:27.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:27.979+0000 7f646c45f700 1 -- 172.21.15.16:0/3518739003 shutdown_connections 2023-05-20T01:26:27.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:27.979+0000 7f646c45f700 1 -- 172.21.15.16:0/3518739003 wait complete. 2023-05-20T01:26:27.996 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_data_scan.TestDataScan.test_extra_data_pool_rebuild_simple' 2023-05-20T01:26:28.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.117+0000 7f68b9f96700 1 Processor -- start 2023-05-20T01:26:28.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.117+0000 7f68b9f96700 1 -- start start 2023-05-20T01:26:28.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.118+0000 7f68b9f96700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f68b406f450 0x7f68b40cf890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:28.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.118+0000 7f68b9f96700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f68b406d4b0 0x7f68b4161080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:28.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.118+0000 7f68b9f96700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f68b406a510 0x7f68b406a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:28.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.118+0000 7f68b9f96700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68b406aef0 con 0x7f68b406d4b0 2023-05-20T01:26:28.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.118+0000 7f68b9f96700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68b40cc510 con 0x7f68b406a510 2023-05-20T01:26:28.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.118+0000 7f68b9f96700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68b41615c0 con 0x7f68b406f450 2023-05-20T01:26:28.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.118+0000 7f68b2ffd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f68b406d4b0 0x7f68b4161080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:28.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.118+0000 7f68b2ffd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f68b406d4b0 0x7f68b4161080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37794/0 (socket says 172.21.15.16:37794) 2023-05-20T01:26:28.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.118+0000 7f68b2ffd700 1 -- 172.21.15.16:0/3398873068 learned_addr learned my addr 172.21.15.16:0/3398873068 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:26:28.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.118+0000 7f68b37fe700 1 --2- 172.21.15.16:0/3398873068 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f68b406f450 0x7f68b40cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:28.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.118+0000 7f68b3fff700 1 --2- 172.21.15.16:0/3398873068 >> v2:172.21.15.132:3300/0 conn(0x7f68b406a510 0x7f68b406a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:28.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.118+0000 7f68b2ffd700 1 -- 172.21.15.16:0/3398873068 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f68b406f450 msgr2=0x7f68b40cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:28.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.118+0000 7f68b2ffd700 1 --2- 172.21.15.16:0/3398873068 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f68b406f450 0x7f68b40cf890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:28.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.118+0000 7f68b2ffd700 1 -- 172.21.15.16:0/3398873068 >> v2:172.21.15.132:3300/0 conn(0x7f68b406a510 msgr2=0x7f68b406a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:28.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.118+0000 7f68b2ffd700 1 --2- 172.21.15.16:0/3398873068 >> v2:172.21.15.132:3300/0 conn(0x7f68b406a510 0x7f68b406a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:28.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.118+0000 7f68b2ffd700 1 -- 172.21.15.16:0/3398873068 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68b4071ed0 con 0x7f68b406d4b0 2023-05-20T01:26:28.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.118+0000 7f68b2ffd700 1 --2- 172.21.15.16:0/3398873068 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68b406d4b0 0x7f68b4161080 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f68b40c7010 tx=0x7f689c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=250cabfcb1ac95ca server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:28.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.118+0000 7f68b27fc700 1 -- 172.21.15.16:0/3398873068 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f689c00b870 con 0x7f68b406d4b0 2023-05-20T01:26:28.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.119+0000 7f68b27fc700 1 -- 172.21.15.16:0/3398873068 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f689c00b9d0 con 0x7f68b406d4b0 2023-05-20T01:26:28.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.119+0000 7f68b27fc700 1 -- 172.21.15.16:0/3398873068 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f689c00bd60 con 0x7f68b406d4b0 2023-05-20T01:26:28.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.119+0000 7f68b9f96700 1 -- 172.21.15.16:0/3398873068 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68b406d4b0 msgr2=0x7f68b4161080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:28.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.119+0000 7f68b9f96700 1 --2- 172.21.15.16:0/3398873068 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68b406d4b0 0x7f68b4161080 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f68b40c7010 tx=0x7f689c009bd0 comp rx=0 tx=0).stop 2023-05-20T01:26:28.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.119+0000 7f68b9f96700 1 -- 172.21.15.16:0/3398873068 shutdown_connections 2023-05-20T01:26:28.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.119+0000 7f68b9f96700 1 --2- 172.21.15.16:0/3398873068 >> v2:172.21.15.132:3300/0 conn(0x7f68b406a510 0x7f68b406a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:28.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.119+0000 7f68b9f96700 1 --2- 172.21.15.16:0/3398873068 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f68b406f450 0x7f68b40cf890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:28.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.119+0000 7f68b9f96700 1 --2- 172.21.15.16:0/3398873068 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68b406d4b0 0x7f68b4161080 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:28.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.119+0000 7f68b9f96700 1 -- 172.21.15.16:0/3398873068 >> 172.21.15.16:0/3398873068 conn(0x7f68b40cffb0 msgr2=0x7f68b40d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:28.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.119+0000 7f68b9f96700 1 -- 172.21.15.16:0/3398873068 shutdown_connections 2023-05-20T01:26:28.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.119+0000 7f68b9f96700 1 -- 172.21.15.16:0/3398873068 wait complete. 2023-05-20T01:26:28.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.120+0000 7f68b9f96700 1 Processor -- start 2023-05-20T01:26:28.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.121+0000 7f68b9f96700 1 -- start start 2023-05-20T01:26:28.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.121+0000 7f68b9f96700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68b406a510 0x7f68b4153fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:28.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.121+0000 7f68b9f96700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f68b406d4b0 0x7f68b40c7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:28.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.121+0000 7f68b9f96700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f68b406f450 0x7f68b40c75c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:28.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.121+0000 7f68b9f96700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68b406e570 con 0x7f68b406a510 2023-05-20T01:26:28.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.121+0000 7f68b9f96700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68b4155cf0 con 0x7f68b406f450 2023-05-20T01:26:28.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.121+0000 7f68b9f96700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68b4155fd0 con 0x7f68b406d4b0 2023-05-20T01:26:28.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.121+0000 7f68b37fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68b406a510 0x7f68b4153fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:28.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.121+0000 7f68b37fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68b406a510 0x7f68b4153fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37798/0 (socket says 172.21.15.16:37798) 2023-05-20T01:26:28.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.121+0000 7f68b37fe700 1 -- 172.21.15.16:0/193505156 learned_addr learned my addr 172.21.15.16:0/193505156 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:26:28.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.122+0000 7f68b2ffd700 1 --2- 172.21.15.16:0/193505156 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f68b406d4b0 0x7f68b40c7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:28.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.122+0000 7f68b3fff700 1 --2- 172.21.15.16:0/193505156 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f68b406f450 0x7f68b40c75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:28.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.122+0000 7f68b37fe700 1 -- 172.21.15.16:0/193505156 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f68b406d4b0 msgr2=0x7f68b40c7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:28.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.122+0000 7f68b37fe700 1 --2- 172.21.15.16:0/193505156 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f68b406d4b0 0x7f68b40c7080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:28.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.122+0000 7f68b37fe700 1 -- 172.21.15.16:0/193505156 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f68b406f450 msgr2=0x7f68b40c75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:28.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.122+0000 7f68b37fe700 1 --2- 172.21.15.16:0/193505156 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f68b406f450 0x7f68b40c75c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:28.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.122+0000 7f68b37fe700 1 -- 172.21.15.16:0/193505156 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f689c009550 con 0x7f68b406a510 2023-05-20T01:26:28.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.122+0000 7f68b2ffd700 1 --2- 172.21.15.16:0/193505156 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f68b406d4b0 0x7f68b40c7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:26:28.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.122+0000 7f68b3fff700 1 --2- 172.21.15.16:0/193505156 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f68b406f450 0x7f68b40c75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:26:28.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.122+0000 7f68b37fe700 1 --2- 172.21.15.16:0/193505156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68b406a510 0x7f68b4153fb0 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f68a400c7f0 tx=0x7f68a400cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:28.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.122+0000 7f68abfff700 1 -- 172.21.15.16:0/193505156 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a4011800 con 0x7f68b406a510 2023-05-20T01:26:28.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.122+0000 7f68abfff700 1 -- 172.21.15.16:0/193505156 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68a4011960 con 0x7f68b406a510 2023-05-20T01:26:28.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.122+0000 7f68b9f96700 1 -- 172.21.15.16:0/193505156 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68b40cc1a0 con 0x7f68b406a510 2023-05-20T01:26:28.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.122+0000 7f68abfff700 1 -- 172.21.15.16:0/193505156 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a4011cf0 con 0x7f68b406a510 2023-05-20T01:26:28.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.122+0000 7f68b9f96700 1 -- 172.21.15.16:0/193505156 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68b40721b0 con 0x7f68b406a510 2023-05-20T01:26:28.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.123+0000 7f68b9f96700 1 -- 172.21.15.16:0/193505156 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68b40cbe50 con 0x7f68b406a510 2023-05-20T01:26:28.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.123+0000 7f68abfff700 1 -- 172.21.15.16:0/193505156 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f68a4018360 con 0x7f68b406a510 2023-05-20T01:26:28.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.124+0000 7f68abfff700 1 --2- 172.21.15.16:0/193505156 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f689406ace0 0x7f689406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:28.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.124+0000 7f68abfff700 1 -- 172.21.15.16:0/193505156 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5874+0+0 (secure 0 0 0) 0x7f68a40893b0 con 0x7f68b406a510 2023-05-20T01:26:28.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.124+0000 7f68b2ffd700 1 --2- 172.21.15.16:0/193505156 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f689406ace0 0x7f689406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:28.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.124+0000 7f68b2ffd700 1 --2- 172.21.15.16:0/193505156 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f689406ace0 0x7f689406d190 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f689c009130 tx=0x7f689c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:28.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.128+0000 7f68abfff700 1 -- 172.21.15.16:0/193505156 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f68a4054510 con 0x7f68b406a510 2023-05-20T01:26:28.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.413+0000 7f68b9f96700 1 -- 172.21.15.16:0/193505156 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_data_scan.TestDataScan.test_extra_data_pool_rebuild_simple"]} v 0) v1 -- 0x7f68b4156490 con 0x7f68b406a510 2023-05-20T01:26:28.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.610+0000 7f68abfff700 1 -- 172.21.15.16:0/193505156 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_data_scan.TestDataScan.test_extra_data_pool_rebuild_simple"]}]=0 v426) v1 ==== 155+0+0 (secure 0 0 0) 0x7f68a4057b30 con 0x7f68b406a510 2023-05-20T01:26:28.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.612+0000 7f68b9f96700 1 -- 172.21.15.16:0/193505156 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f689406ace0 msgr2=0x7f689406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:28.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.612+0000 7f68b9f96700 1 --2- 172.21.15.16:0/193505156 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f689406ace0 0x7f689406d190 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f689c009130 tx=0x7f689c008030 comp rx=0 tx=0).stop 2023-05-20T01:26:28.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.612+0000 7f68b9f96700 1 -- 172.21.15.16:0/193505156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68b406a510 msgr2=0x7f68b4153fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:28.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.612+0000 7f68b9f96700 1 --2- 172.21.15.16:0/193505156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68b406a510 0x7f68b4153fb0 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f68a400c7f0 tx=0x7f68a400cbb0 comp rx=0 tx=0).stop 2023-05-20T01:26:28.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.612+0000 7f68b9f96700 1 -- 172.21.15.16:0/193505156 shutdown_connections 2023-05-20T01:26:28.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.612+0000 7f68b9f96700 1 --2- 172.21.15.16:0/193505156 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f68b406f450 0x7f68b40c75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:28.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.612+0000 7f68b9f96700 1 --2- 172.21.15.16:0/193505156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68b406a510 0x7f68b4153fb0 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:28.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.612+0000 7f68b9f96700 1 --2- 172.21.15.16:0/193505156 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f689406ace0 0x7f689406d190 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:28.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.612+0000 7f68b9f96700 1 --2- 172.21.15.16:0/193505156 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f68b406d4b0 0x7f68b40c7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:28.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.612+0000 7f68b9f96700 1 -- 172.21.15.16:0/193505156 >> 172.21.15.16:0/193505156 conn(0x7f68b40cffb0 msgr2=0x7f68b415ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:28.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.612+0000 7f68b9f96700 1 -- 172.21.15.16:0/193505156 shutdown_connections 2023-05-20T01:26:28.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.612+0000 7f68b9f96700 1 -- 172.21.15.16:0/193505156 wait complete. 2023-05-20T01:26:28.630 INFO:tasks.cephfs_test_runner:test_extra_data_pool_rebuild_simple (tasks.cephfs.test_data_scan.TestDataScan) ... ok 2023-05-20T01:26:28.631 INFO:tasks.cephfs_test_runner:Starting test: test_extra_data_pool_stashed_layout (tasks.cephfs.test_data_scan.TestDataScan) 2023-05-20T01:26:28.631 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_data_scan.TestDataScan.test_extra_data_pool_stashed_layout' 2023-05-20T01:26:28.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.754+0000 7ff00914f700 1 Processor -- start 2023-05-20T01:26:28.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.754+0000 7ff00914f700 1 -- start start 2023-05-20T01:26:28.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.755+0000 7ff00914f700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff004157190 0x7ff0040cc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:28.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.755+0000 7ff00914f700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff004071f20 con 0x7ff004071ad0 2023-05-20T01:26:28.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.755+0000 7ff00914f700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0040cb590 con 0x7ff0040cccf0 2023-05-20T01:26:28.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.755+0000 7ff00914f700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0040ca9a0 con 0x7ff004157190 2023-05-20T01:26:28.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.755+0000 7ff00359e700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ff004071ad0 0x7ff0040ce740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49338/0 (socket says 172.21.15.16:49338) 2023-05-20T01:26:28.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.755+0000 7ff00359e700 1 -- 172.21.15.16:0/3263252532 learned_addr learned my addr 172.21.15.16:0/3263252532 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:26:28.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.755+0000 7ff002d9d700 1 --2- 172.21.15.16:0/3263252532 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff004157190 0x7ff0040cc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:28.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.755+0000 7ff001d9b700 1 -- 172.21.15.16:0/3263252532 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1384317778 0 0) 0x7feffc001490 con 0x7ff004071ad0 2023-05-20T01:26:28.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.755+0000 7ff001d9b700 1 -- 172.21.15.16:0/3263252532 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fefec0035b0 con 0x7ff004071ad0 2023-05-20T01:26:28.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.756+0000 7ff001d9b700 1 -- 172.21.15.16:0/3263252532 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 381442245 0 0) 0x7fefec0035b0 con 0x7ff004071ad0 2023-05-20T01:26:28.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.756+0000 7ff001d9b700 1 -- 172.21.15.16:0/3263252532 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff004157190 msgr2=0x7ff0040cc7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:28.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.756+0000 7ff001d9b700 1 --2- 172.21.15.16:0/3263252532 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff004157190 0x7ff0040cc7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:28.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.756+0000 7ff001d9b700 1 -- 172.21.15.16:0/3263252532 >> v1:172.21.15.132:6789/0 conn(0x7ff0040cccf0 legacy=0x7ff0040713c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:28.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.756+0000 7ff001d9b700 1 -- 172.21.15.16:0/3263252532 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0040cac60 con 0x7ff004071ad0 2023-05-20T01:26:28.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.756+0000 7ff001d9b700 1 -- 172.21.15.16:0/3263252532 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7feffc001b80 con 0x7ff004071ad0 2023-05-20T01:26:28.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.756+0000 7ff001d9b700 1 -- 172.21.15.16:0/3263252532 >> v1:172.21.15.16:6789/0 conn(0x7ff004071ad0 legacy=0x7ff0040ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:28.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.756+0000 7ff001d9b700 1 --2- 172.21.15.16:0/3263252532 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fefec003aa0 0x7fefec003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:28.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.756+0000 7ff001d9b700 1 --2- 172.21.15.16:0/3263252532 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff004157190 0x7fefec004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:28.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.756+0000 7ff001d9b700 1 --2- 172.21.15.16:0/3263252532 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefec004d90 0x7fefec005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:28.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.756+0000 7ff001d9b700 1 -- 172.21.15.16:0/3263252532 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0040ca9a0 con 0x7fefec004d90 2023-05-20T01:26:28.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.756+0000 7ff001d9b700 1 -- 172.21.15.16:0/3263252532 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefec00bac0 con 0x7fefec003aa0 2023-05-20T01:26:28.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.756+0000 7ff001d9b700 1 -- 172.21.15.16:0/3263252532 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefec00de40 con 0x7ff004157190 2023-05-20T01:26:28.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.756+0000 7ff00359e700 1 --2- 172.21.15.16:0/3263252532 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefec004d90 0x7fefec005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:28.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.756+0000 7ff002d9d700 1 --2- 172.21.15.16:0/3263252532 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fefec003aa0 0x7fefec003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:28.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.756+0000 7ff00259c700 1 --2- 172.21.15.16:0/3263252532 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff004157190 0x7fefec004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:28.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.756+0000 7ff00359e700 1 -- 172.21.15.16:0/3263252532 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff004157190 msgr2=0x7fefec004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:28.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.756+0000 7ff00359e700 1 --2- 172.21.15.16:0/3263252532 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff004157190 0x7fefec004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:28.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.756+0000 7ff00359e700 1 -- 172.21.15.16:0/3263252532 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fefec003aa0 msgr2=0x7fefec003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:28.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.756+0000 7ff00359e700 1 --2- 172.21.15.16:0/3263252532 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fefec003aa0 0x7fefec003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:28.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.756+0000 7ff00359e700 1 -- 172.21.15.16:0/3263252532 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefec0035b0 con 0x7fefec004d90 2023-05-20T01:26:28.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.756+0000 7ff00359e700 1 --2- 172.21.15.16:0/3263252532 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefec004d90 0x7fefec005200 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7feffc00a840 tx=0x7fefec005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6ceec20baa7ed3f server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:28.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.756+0000 7ff001d9b700 1 -- 172.21.15.16:0/3263252532 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feffc012600 con 0x7fefec004d90 2023-05-20T01:26:28.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.756+0000 7ff001d9b700 1 -- 172.21.15.16:0/3263252532 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feffc012760 con 0x7fefec004d90 2023-05-20T01:26:28.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.756+0000 7ff001d9b700 1 -- 172.21.15.16:0/3263252532 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feffc012af0 con 0x7fefec004d90 2023-05-20T01:26:28.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.757+0000 7ff00914f700 1 -- 172.21.15.16:0/3263252532 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefec004d90 msgr2=0x7fefec005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:28.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.757+0000 7ff00914f700 1 --2- 172.21.15.16:0/3263252532 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefec004d90 0x7fefec005200 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7feffc00a840 tx=0x7fefec005880 comp rx=0 tx=0).stop 2023-05-20T01:26:28.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.757+0000 7ff00359e700 1 -- 172.21.15.16:0/3263252532 reap_dead start 2023-05-20T01:26:28.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.757+0000 7ff00914f700 1 -- 172.21.15.16:0/3263252532 shutdown_connections 2023-05-20T01:26:28.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.757+0000 7ff00914f700 1 -- 172.21.15.16:0/3263252532 >> 172.21.15.16:0/3263252532 conn(0x7ff0040cffb0 msgr2=0x7ff0040d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:28.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.757+0000 7ff00914f700 1 -- 172.21.15.16:0/3263252532 shutdown_connections 2023-05-20T01:26:28.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.757+0000 7ff00914f700 1 -- 172.21.15.16:0/3263252532 wait complete. 2023-05-20T01:26:28.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.758+0000 7ff00914f700 1 Processor -- start 2023-05-20T01:26:28.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.759+0000 7ff00914f700 1 -- start start 2023-05-20T01:26:28.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.759+0000 7ff00914f700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff004152cc0 0x7ff0040b3470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:28.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.759+0000 7ff00914f700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff0040b2e40 0x7ff0040c5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:28.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.759+0000 7ff00914f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0040b1c20 0x7ff0040c6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:28.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.759+0000 7ff00914f700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0040c6650 con 0x7ff0040b1c20 2023-05-20T01:26:28.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.759+0000 7ff00914f700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0040c4b30 con 0x7ff004152cc0 2023-05-20T01:26:28.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.759+0000 7ff00914f700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0040c4e10 con 0x7ff0040b2e40 2023-05-20T01:26:28.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.759+0000 7ff00359e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0040b1c20 0x7ff0040c6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:28.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.759+0000 7ff00359e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0040b1c20 0x7ff0040c6110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37822/0 (socket says 172.21.15.16:37822) 2023-05-20T01:26:28.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.759+0000 7ff00359e700 1 -- 172.21.15.16:0/3342302162 learned_addr learned my addr 172.21.15.16:0/3342302162 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:26:28.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.759+0000 7ff002d9d700 1 --2- 172.21.15.16:0/3342302162 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff004152cc0 0x7ff0040b3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:28.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.759+0000 7ff00259c700 1 --2- 172.21.15.16:0/3342302162 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff0040b2e40 0x7ff0040c5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:28.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.760+0000 7ff00359e700 1 -- 172.21.15.16:0/3342302162 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff0040b2e40 msgr2=0x7ff0040c5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:28.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.760+0000 7ff00359e700 1 --2- 172.21.15.16:0/3342302162 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff0040b2e40 0x7ff0040c5bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:28.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.760+0000 7ff00359e700 1 -- 172.21.15.16:0/3342302162 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff004152cc0 msgr2=0x7ff0040b3470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:28.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.760+0000 7ff00359e700 1 --2- 172.21.15.16:0/3342302162 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff004152cc0 0x7ff0040b3470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:28.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.760+0000 7ff00359e700 1 -- 172.21.15.16:0/3342302162 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0040cb220 con 0x7ff0040b1c20 2023-05-20T01:26:28.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.760+0000 7ff002d9d700 1 --2- 172.21.15.16:0/3342302162 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff004152cc0 0x7ff0040b3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-05-20T01:26:28.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.760+0000 7ff00359e700 1 --2- 172.21.15.16:0/3342302162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0040b1c20 0x7ff0040c6110 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7feffc00a0b0 tx=0x7feffc001f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:28.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.760+0000 7fefeb7fe700 1 -- 172.21.15.16:0/3342302162 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feffc001490 con 0x7ff0040b1c20 2023-05-20T01:26:28.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.760+0000 7fefeb7fe700 1 -- 172.21.15.16:0/3342302162 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feffc001250 con 0x7ff0040b1c20 2023-05-20T01:26:28.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.760+0000 7fefeb7fe700 1 -- 172.21.15.16:0/3342302162 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feffc01d420 con 0x7ff0040b1c20 2023-05-20T01:26:28.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.760+0000 7ff00914f700 1 -- 172.21.15.16:0/3342302162 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0040caf40 con 0x7ff0040b1c20 2023-05-20T01:26:28.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.760+0000 7ff00914f700 1 -- 172.21.15.16:0/3342302162 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff004157560 con 0x7ff0040b1c20 2023-05-20T01:26:28.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.761+0000 7ff00914f700 1 -- 172.21.15.16:0/3342302162 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0040c34a0 con 0x7ff0040b1c20 2023-05-20T01:26:28.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.761+0000 7fefeb7fe700 1 -- 172.21.15.16:0/3342302162 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7feffc01d580 con 0x7ff0040b1c20 2023-05-20T01:26:28.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.762+0000 7fefeb7fe700 1 --2- 172.21.15.16:0/3342302162 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7feff006ace0 0x7feff006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:28.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.762+0000 7fefeb7fe700 1 -- 172.21.15.16:0/3342302162 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5874+0+0 (secure 0 0 0) 0x7feffc08ab20 con 0x7ff0040b1c20 2023-05-20T01:26:28.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.762+0000 7ff002d9d700 1 --2- 172.21.15.16:0/3342302162 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7feff006ace0 0x7feff006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:28.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.762+0000 7ff002d9d700 1 --2- 172.21.15.16:0/3342302162 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7feff006ace0 0x7feff006d190 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7feff4014400 tx=0x7feff4009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:28.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:28.766+0000 7fefeb7fe700 1 -- 172.21.15.16:0/3342302162 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7feffc055ef0 con 0x7ff0040b1c20 2023-05-20T01:26:29.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:29.047+0000 7ff00914f700 1 -- 172.21.15.16:0/3342302162 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_data_scan.TestDataScan.test_extra_data_pool_stashed_layout"]} v 0) v1 -- 0x7ff004071f20 con 0x7ff0040b1c20 2023-05-20T01:26:29.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:29.611+0000 7fefeb7fe700 1 -- 172.21.15.16:0/3342302162 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_data_scan.TestDataScan.test_extra_data_pool_stashed_layout"]}]=0 v427) v1 ==== 158+0+0 (secure 0 0 0) 0x7ff004071f20 con 0x7ff0040b1c20 2023-05-20T01:26:29.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:29.614+0000 7ff00914f700 1 -- 172.21.15.16:0/3342302162 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7feff006ace0 msgr2=0x7feff006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:29.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:29.614+0000 7ff00914f700 1 --2- 172.21.15.16:0/3342302162 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7feff006ace0 0x7feff006d190 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7feff4014400 tx=0x7feff4009210 comp rx=0 tx=0).stop 2023-05-20T01:26:29.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:29.614+0000 7ff00914f700 1 -- 172.21.15.16:0/3342302162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0040b1c20 msgr2=0x7ff0040c6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:29.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:29.614+0000 7ff00914f700 1 --2- 172.21.15.16:0/3342302162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0040b1c20 0x7ff0040c6110 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7feffc00a0b0 tx=0x7feffc001f10 comp rx=0 tx=0).stop 2023-05-20T01:26:29.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:29.614+0000 7ff00914f700 1 -- 172.21.15.16:0/3342302162 shutdown_connections 2023-05-20T01:26:29.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:29.614+0000 7ff00914f700 1 --2- 172.21.15.16:0/3342302162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0040b1c20 0x7ff0040c6110 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:29.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:29.614+0000 7ff00914f700 1 --2- 172.21.15.16:0/3342302162 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff004152cc0 0x7ff0040b3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:29.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:29.614+0000 7ff00914f700 1 --2- 172.21.15.16:0/3342302162 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7feff006ace0 0x7feff006d190 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:29.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:29.614+0000 7ff00914f700 1 --2- 172.21.15.16:0/3342302162 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff0040b2e40 0x7ff0040c5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:29.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:29.614+0000 7ff00914f700 1 -- 172.21.15.16:0/3342302162 >> 172.21.15.16:0/3342302162 conn(0x7ff0040cffb0 msgr2=0x7ff00415baf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:29.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:29.614+0000 7ff00914f700 1 -- 172.21.15.16:0/3342302162 shutdown_connections 2023-05-20T01:26:29.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:29.614+0000 7ff00914f700 1 -- 172.21.15.16:0/3342302162 wait complete. 2023-05-20T01:26:29.634 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-05-20T01:26:29.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:29.768+0000 7f0b04c4b700 1 Processor -- start 2023-05-20T01:26:29.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:29.769+0000 7f0b04c4b700 1 -- start start 2023-05-20T01:26:29.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:29.770+0000 7f0b04c4b700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0b000cb7b0 0x7f0b000cc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:29.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:29.770+0000 7f0b04c4b700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b000cbbd0 con 0x7f0b000ce7b0 2023-05-20T01:26:29.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:29.770+0000 7f0b04c4b700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b00159360 con 0x7f0b00158f90 2023-05-20T01:26:29.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:29.770+0000 7f0b04c4b700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b000cce10 con 0x7f0b000cb7b0 2023-05-20T01:26:29.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:29.770+0000 7f0afdd9b700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f0b000ce7b0 0x7f0b000cc050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49356/0 (socket says 172.21.15.16:49356) 2023-05-20T01:26:29.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:29.770+0000 7f0afdd9b700 1 -- 172.21.15.16:0/1583328327 learned_addr learned my addr 172.21.15.16:0/1583328327 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:26:29.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:29.770+0000 7f0afed9d700 1 --2- 172.21.15.16:0/1583328327 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0b000cb7b0 0x7f0b000cc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:29.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:29.770+0000 7f0afd59a700 1 -- 172.21.15.16:0/1583328327 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 851312625 0 0) 0x7f0b000cbbd0 con 0x7f0b000ce7b0 2023-05-20T01:26:29.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:29.770+0000 7f0afd59a700 1 -- 172.21.15.16:0/1583328327 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ae80035b0 con 0x7f0b000ce7b0 2023-05-20T01:26:29.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:29.771+0000 7f0afd59a700 1 -- 172.21.15.16:0/1583328327 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2421610931 0 0) 0x7f0ae80035b0 con 0x7f0b000ce7b0 2023-05-20T01:26:29.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:29.771+0000 7f0afd59a700 1 -- 172.21.15.16:0/1583328327 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0b000cb7b0 msgr2=0x7f0b000cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:29.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:29.771+0000 7f0afd59a700 1 --2- 172.21.15.16:0/1583328327 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0b000cb7b0 0x7f0b000cc760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:29.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:29.771+0000 7f0afd59a700 1 -- 172.21.15.16:0/1583328327 >> v1:172.21.15.132:6789/0 conn(0x7f0b00158f90 legacy=0x7f0b000ce0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:29.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:29.771+0000 7f0afd59a700 1 -- 172.21.15.16:0/1583328327 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b000ca750 con 0x7f0b000ce7b0 2023-05-20T01:26:29.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:29.771+0000 7f0afd59a700 1 -- 172.21.15.16:0/1583328327 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7f0af4001670 con 0x7f0b000ce7b0 2023-05-20T01:26:29.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:29.771+0000 7f0afd59a700 1 -- 172.21.15.16:0/1583328327 >> v1:172.21.15.16:6789/0 conn(0x7f0b000ce7b0 legacy=0x7f0b000cc050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:29.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:29.771+0000 7f0afd59a700 1 --2- 172.21.15.16:0/1583328327 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0b000cb7b0 0x7f0ae8003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:29.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:29.771+0000 7f0afd59a700 1 --2- 172.21.15.16:0/1583328327 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0ae8003f70 0x7f0ae8004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:29.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:29.771+0000 7f0afd59a700 1 --2- 172.21.15.16:0/1583328327 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ae8004d90 0x7f0ae8005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:29.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:29.771+0000 7f0afd59a700 1 -- 172.21.15.16:0/1583328327 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ae800ba10 con 0x7f0ae8004d90 2023-05-20T01:26:29.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:29.771+0000 7f0afd59a700 1 -- 172.21.15.16:0/1583328327 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ae800bb80 con 0x7f0ae8003f70 2023-05-20T01:26:29.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:29.771+0000 7f0afd59a700 1 -- 172.21.15.16:0/1583328327 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ae800df50 con 0x7f0b000cb7b0 2023-05-20T01:26:29.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:29.771+0000 7f0afed9d700 1 --2- 172.21.15.16:0/1583328327 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ae8004d90 0x7f0ae8005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:29.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:29.771+0000 7f0afe59c700 1 --2- 172.21.15.16:0/1583328327 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0b000cb7b0 0x7f0ae8003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:29.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:29.771+0000 7f0afdd9b700 1 --2- 172.21.15.16:0/1583328327 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0ae8003f70 0x7f0ae8004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:29.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:29.771+0000 7f0afed9d700 1 -- 172.21.15.16:0/1583328327 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0b000cb7b0 msgr2=0x7f0ae8003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:29.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:29.771+0000 7f0afed9d700 1 --2- 172.21.15.16:0/1583328327 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0b000cb7b0 0x7f0ae8003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:29.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:29.771+0000 7f0afed9d700 1 -- 172.21.15.16:0/1583328327 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0ae8003f70 msgr2=0x7f0ae8004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:29.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:29.771+0000 7f0afed9d700 1 --2- 172.21.15.16:0/1583328327 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0ae8003f70 0x7f0ae8004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:29.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:29.771+0000 7f0afed9d700 1 -- 172.21.15.16:0/1583328327 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ae80035b0 con 0x7f0ae8004d90 2023-05-20T01:26:29.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:29.771+0000 7f0afed9d700 1 --2- 172.21.15.16:0/1583328327 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ae8004d90 0x7f0ae8005200 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f0af800a250 tx=0x7f0ae8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8b63b7ba5217719 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:29.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:29.772+0000 7f0afd59a700 1 -- 172.21.15.16:0/1583328327 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0af8009e20 con 0x7f0ae8004d90 2023-05-20T01:26:29.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:29.772+0000 7f0afd59a700 1 -- 172.21.15.16:0/1583328327 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0af8010040 con 0x7f0ae8004d90 2023-05-20T01:26:29.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:29.772+0000 7f0afd59a700 1 -- 172.21.15.16:0/1583328327 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0af80145f0 con 0x7f0ae8004d90 2023-05-20T01:26:29.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:29.772+0000 7f0b04c4b700 1 -- 172.21.15.16:0/1583328327 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ae8004d90 msgr2=0x7f0ae8005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:29.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:29.772+0000 7f0b04c4b700 1 --2- 172.21.15.16:0/1583328327 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ae8004d90 0x7f0ae8005200 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f0af800a250 tx=0x7f0ae8005880 comp rx=0 tx=0).stop 2023-05-20T01:26:29.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:29.772+0000 7f0afed9d700 1 -- 172.21.15.16:0/1583328327 reap_dead start 2023-05-20T01:26:29.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:29.772+0000 7f0b04c4b700 1 -- 172.21.15.16:0/1583328327 shutdown_connections 2023-05-20T01:26:29.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:29.772+0000 7f0b04c4b700 1 -- 172.21.15.16:0/1583328327 >> 172.21.15.16:0/1583328327 conn(0x7f0b0006cbb0 msgr2=0x7f0b0006cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:29.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:29.772+0000 7f0b04c4b700 1 -- 172.21.15.16:0/1583328327 shutdown_connections 2023-05-20T01:26:29.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:29.772+0000 7f0b04c4b700 1 -- 172.21.15.16:0/1583328327 wait complete. 2023-05-20T01:26:29.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:29.774+0000 7f0b04c4b700 1 Processor -- start 2023-05-20T01:26:29.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:29.774+0000 7f0b04c4b700 1 -- start start 2023-05-20T01:26:29.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:29.775+0000 7f0b04c4b700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0ae8004d90 0x7f0b0015c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:29.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:29.775+0000 7f0b04c4b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b000b39b0 0x7f0b0015a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:29.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:29.775+0000 7f0b04c4b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0b00144f00 0x7f0b00070f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:29.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:29.775+0000 7f0b04c4b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b0015bc60 con 0x7f0b000b39b0 2023-05-20T01:26:29.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:29.775+0000 7f0b04c4b700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b0015ad40 con 0x7f0b00144f00 2023-05-20T01:26:29.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:29.775+0000 7f0b04c4b700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b0015afe0 con 0x7f0ae8004d90 2023-05-20T01:26:29.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:29.775+0000 7f0afdd9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b000b39b0 0x7f0b0015a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:29.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:29.775+0000 7f0afdd9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b000b39b0 0x7f0b0015a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37840/0 (socket says 172.21.15.16:37840) 2023-05-20T01:26:29.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:29.775+0000 7f0afdd9b700 1 -- 172.21.15.16:0/2426068113 learned_addr learned my addr 172.21.15.16:0/2426068113 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:26:29.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:29.775+0000 7f0afed9d700 1 --2- 172.21.15.16:0/2426068113 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0b00144f00 0x7f0b00070f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:29.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:29.775+0000 7f0afe59c700 1 --2- 172.21.15.16:0/2426068113 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0ae8004d90 0x7f0b0015c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:29.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:29.775+0000 7f0afdd9b700 1 -- 172.21.15.16:0/2426068113 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0ae8004d90 msgr2=0x7f0b0015c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:29.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:29.775+0000 7f0afdd9b700 1 --2- 172.21.15.16:0/2426068113 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0ae8004d90 0x7f0b0015c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:29.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:29.775+0000 7f0afdd9b700 1 -- 172.21.15.16:0/2426068113 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0b00144f00 msgr2=0x7f0b00070f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:29.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:29.775+0000 7f0afdd9b700 1 --2- 172.21.15.16:0/2426068113 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0b00144f00 0x7f0b00070f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:29.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:29.775+0000 7f0afdd9b700 1 -- 172.21.15.16:0/2426068113 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b000cacd0 con 0x7f0b000b39b0 2023-05-20T01:26:29.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:29.776+0000 7f0afdd9b700 1 --2- 172.21.15.16:0/2426068113 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b000b39b0 0x7f0b0015a800 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f0af40072c0 tx=0x7f0af40075d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:29.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:29.776+0000 7f0aeeffd700 1 -- 172.21.15.16:0/2426068113 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0af40077b0 con 0x7f0b000b39b0 2023-05-20T01:26:29.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:29.776+0000 7f0aeeffd700 1 -- 172.21.15.16:0/2426068113 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0af400f8d0 con 0x7f0b000b39b0 2023-05-20T01:26:29.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:29.776+0000 7f0b04c4b700 1 -- 172.21.15.16:0/2426068113 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b000ca9f0 con 0x7f0b000b39b0 2023-05-20T01:26:29.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:29.776+0000 7f0aeeffd700 1 -- 172.21.15.16:0/2426068113 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0af400fc80 con 0x7f0b000b39b0 2023-05-20T01:26:29.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:29.776+0000 7f0b04c4b700 1 -- 172.21.15.16:0/2426068113 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b000cafb0 con 0x7f0b000b39b0 2023-05-20T01:26:29.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:29.777+0000 7f0b04c4b700 1 -- 172.21.15.16:0/2426068113 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b000747b0 con 0x7f0b000b39b0 2023-05-20T01:26:29.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:29.777+0000 7f0aeeffd700 1 -- 172.21.15.16:0/2426068113 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0af401d460 con 0x7f0b000b39b0 2023-05-20T01:26:29.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:29.778+0000 7f0aeeffd700 1 --2- 172.21.15.16:0/2426068113 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f0ae006ace0 0x7f0ae006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:29.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:29.778+0000 7f0aeeffd700 1 -- 172.21.15.16:0/2426068113 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5874+0+0 (secure 0 0 0) 0x7f0af40894c0 con 0x7f0b000b39b0 2023-05-20T01:26:29.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:29.778+0000 7f0afe59c700 1 --2- 172.21.15.16:0/2426068113 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f0ae006ace0 0x7f0ae006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:29.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:29.778+0000 7f0afe59c700 1 --2- 172.21.15.16:0/2426068113 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f0ae006ace0 0x7f0ae006d190 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f0af0007fd0 tx=0x7f0af0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:29.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:29.782+0000 7f0aeeffd700 1 -- 172.21.15.16:0/2426068113 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f0af4054630 con 0x7f0b000b39b0 2023-05-20T01:26:29.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:29.981+0000 7f0b04c4b700 1 -- 172.21.15.16:0/2426068113 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f0b000c64e0 con 0x7f0b000b39b0 2023-05-20T01:26:29.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:29.985+0000 7f0aeeffd700 1 -- 172.21.15.16:0/2426068113 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7f0af4057c50 con 0x7f0b000b39b0 2023-05-20T01:26:29.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:29.987+0000 7f0b04c4b700 1 -- 172.21.15.16:0/2426068113 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f0ae006ace0 msgr2=0x7f0ae006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:29.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:29.987+0000 7f0b04c4b700 1 --2- 172.21.15.16:0/2426068113 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f0ae006ace0 0x7f0ae006d190 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f0af0007fd0 tx=0x7f0af0009040 comp rx=0 tx=0).stop 2023-05-20T01:26:29.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:29.987+0000 7f0b04c4b700 1 -- 172.21.15.16:0/2426068113 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b000b39b0 msgr2=0x7f0b0015a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:29.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:29.987+0000 7f0b04c4b700 1 --2- 172.21.15.16:0/2426068113 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b000b39b0 0x7f0b0015a800 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f0af40072c0 tx=0x7f0af40075d0 comp rx=0 tx=0).stop 2023-05-20T01:26:29.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:29.987+0000 7f0b04c4b700 1 -- 172.21.15.16:0/2426068113 shutdown_connections 2023-05-20T01:26:29.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:29.987+0000 7f0b04c4b700 1 --2- 172.21.15.16:0/2426068113 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0b00144f00 0x7f0b00070f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:29.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:29.987+0000 7f0b04c4b700 1 --2- 172.21.15.16:0/2426068113 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f0ae006ace0 0x7f0ae006d190 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:29.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:29.987+0000 7f0b04c4b700 1 --2- 172.21.15.16:0/2426068113 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0ae8004d90 0x7f0b0015c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:29.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:29.987+0000 7f0b04c4b700 1 --2- 172.21.15.16:0/2426068113 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b000b39b0 0x7f0b0015a800 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:29.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:29.987+0000 7f0b04c4b700 1 -- 172.21.15.16:0/2426068113 >> 172.21.15.16:0/2426068113 conn(0x7f0b0006cbb0 msgr2=0x7f0b0006e150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:29.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:29.987+0000 7f0b04c4b700 1 -- 172.21.15.16:0/2426068113 shutdown_connections 2023-05-20T01:26:29.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:29.987+0000 7f0b04c4b700 1 -- 172.21.15.16:0/2426068113 wait complete. 2023-05-20T01:26:30.004 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-05-20T01:26:30.004 DEBUG:teuthology.orchestra.run.smithi016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-05-20T01:26:30.057 DEBUG:teuthology.orchestra.run.smithi132:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-05-20T01:26:30.105 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-05-20T01:26:30.105 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-05-20T01:26:30.105 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2023-05-20T01:26:30.124 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16287364k,nr_inodes=4071841,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=38,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16696 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3261172k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-05-20T01:26:30.125 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2023-05-20T01:26:30.125 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/proc/self/mounts of=/dev/stdout 2023-05-20T01:26:30.144 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284284k,nr_inodes=4071071,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=45,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16175 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260556k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-05-20T01:26:30.144 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-05-20T01:26:30.145 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2023-05-20T01:26:30.182 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16287364k,nr_inodes=4071841,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=38,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16696 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3261172k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-05-20T01:26:30.183 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2023-05-20T01:26:30.183 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/proc/self/mounts of=/dev/stdout 2023-05-20T01:26:30.202 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284284k,nr_inodes=4071071,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=45,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16175 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260556k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-05-20T01:26:30.202 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-05-20T01:26:30.203 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-20T01:26:30.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.342+0000 7fd422acb700 1 Processor -- start 2023-05-20T01:26:30.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.342+0000 7fd422acb700 1 -- start start 2023-05-20T01:26:30.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.343+0000 7fd422acb700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fd41c06d4b0 0x7fd41c0cf890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:30.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.343+0000 7fd422acb700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd41c06a510 0x7fd41c06a950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:30.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.343+0000 7fd422acb700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd41c0cc510 con 0x7fd41c06f450 2023-05-20T01:26:30.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.343+0000 7fd422acb700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd41c06ae90 con 0x7fd41c06a510 2023-05-20T01:26:30.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.343+0000 7fd422acb700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd41c071ed0 con 0x7fd41c06d4b0 2023-05-20T01:26:30.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.343+0000 7fd421068700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd41c06a510 0x7fd41c06a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:30.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.343+0000 7fd421068700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd41c06a510 0x7fd41c06a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37850/0 (socket says 172.21.15.16:37850) 2023-05-20T01:26:30.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.343+0000 7fd421068700 1 -- 172.21.15.16:0/902013601 learned_addr learned my addr 172.21.15.16:0/902013601 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:26:30.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.343+0000 7fd421068700 1 -- 172.21.15.16:0/902013601 >> v2:172.21.15.132:3300/0 conn(0x7fd41c06d4b0 msgr2=0x7fd41c0cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:30.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.343+0000 7fd421068700 1 --2- 172.21.15.16:0/902013601 >> v2:172.21.15.132:3300/0 conn(0x7fd41c06d4b0 0x7fd41c0cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:30.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.343+0000 7fd421068700 1 -- 172.21.15.16:0/902013601 >> v1:172.21.15.132:6789/0 conn(0x7fd41c06f450 legacy=0x7fd41c161080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:30.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.343+0000 7fd421068700 1 -- 172.21.15.16:0/902013601 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd41c0721b0 con 0x7fd41c06a510 2023-05-20T01:26:30.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.343+0000 7fd421068700 1 --2- 172.21.15.16:0/902013601 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd41c06a510 0x7fd41c06a950 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7fd41c070c40 tx=0x7fd41400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2640b3afaaa769c4 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:30.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.344+0000 7fd41b7fe700 1 -- 172.21.15.16:0/902013601 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd41400e070 con 0x7fd41c06a510 2023-05-20T01:26:30.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.344+0000 7fd41b7fe700 1 -- 172.21.15.16:0/902013601 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd414003a20 con 0x7fd41c06a510 2023-05-20T01:26:30.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.344+0000 7fd41b7fe700 1 -- 172.21.15.16:0/902013601 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd414003dd0 con 0x7fd41c06a510 2023-05-20T01:26:30.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.344+0000 7fd422acb700 1 -- 172.21.15.16:0/902013601 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd41c06a510 msgr2=0x7fd41c06a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:30.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.344+0000 7fd422acb700 1 --2- 172.21.15.16:0/902013601 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd41c06a510 0x7fd41c06a950 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7fd41c070c40 tx=0x7fd41400b330 comp rx=0 tx=0).stop 2023-05-20T01:26:30.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.344+0000 7fd422acb700 1 -- 172.21.15.16:0/902013601 shutdown_connections 2023-05-20T01:26:30.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.344+0000 7fd422acb700 1 --2- 172.21.15.16:0/902013601 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd41c06a510 0x7fd41c06a950 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:30.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.344+0000 7fd422acb700 1 --2- 172.21.15.16:0/902013601 >> v2:172.21.15.132:3300/0 conn(0x7fd41c06d4b0 0x7fd41c0cf890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:30.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.344+0000 7fd422acb700 1 -- 172.21.15.16:0/902013601 >> 172.21.15.16:0/902013601 conn(0x7fd41c0cffb0 msgr2=0x7fd41c0d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:30.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.344+0000 7fd422acb700 1 -- 172.21.15.16:0/902013601 shutdown_connections 2023-05-20T01:26:30.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.344+0000 7fd422acb700 1 -- 172.21.15.16:0/902013601 wait complete. 2023-05-20T01:26:30.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.345+0000 7fd422acb700 1 Processor -- start 2023-05-20T01:26:30.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.345+0000 7fd422acb700 1 -- start start 2023-05-20T01:26:30.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.346+0000 7fd422acb700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd41c06a510 0x7fd41c0c3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:30.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.346+0000 7fd422acb700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd41c06d4b0 0x7fd41c1587f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:30.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.346+0000 7fd422acb700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd41c06f450 0x7fd41c06e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:30.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.346+0000 7fd422acb700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd41c0c6dc0 con 0x7fd41c06f450 2023-05-20T01:26:30.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.346+0000 7fd422acb700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd41c0c40c0 con 0x7fd41c06a510 2023-05-20T01:26:30.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.346+0000 7fd422acb700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd41c0c6490 con 0x7fd41c06d4b0 2023-05-20T01:26:30.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.346+0000 7fd421068700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd41c06f450 0x7fd41c06e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:30.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.347+0000 7fd421068700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd41c06f450 0x7fd41c06e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37856/0 (socket says 172.21.15.16:37856) 2023-05-20T01:26:30.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.347+0000 7fd421068700 1 -- 172.21.15.16:0/2210689756 learned_addr learned my addr 172.21.15.16:0/2210689756 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:26:30.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.347+0000 7fd420867700 1 --2- 172.21.15.16:0/2210689756 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd41c06a510 0x7fd41c0c3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:30.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.347+0000 7fd41bfff700 1 --2- 172.21.15.16:0/2210689756 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd41c06d4b0 0x7fd41c1587f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:30.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.347+0000 7fd421068700 1 -- 172.21.15.16:0/2210689756 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd41c06d4b0 msgr2=0x7fd41c1587f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:30.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.347+0000 7fd421068700 1 --2- 172.21.15.16:0/2210689756 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd41c06d4b0 0x7fd41c1587f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:30.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.347+0000 7fd421068700 1 -- 172.21.15.16:0/2210689756 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd41c06a510 msgr2=0x7fd41c0c3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:30.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.347+0000 7fd421068700 1 --2- 172.21.15.16:0/2210689756 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd41c06a510 0x7fd41c0c3ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:30.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.347+0000 7fd421068700 1 -- 172.21.15.16:0/2210689756 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd414009b30 con 0x7fd41c06f450 2023-05-20T01:26:30.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.347+0000 7fd421068700 1 --2- 172.21.15.16:0/2210689756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd41c06f450 0x7fd41c06e080 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7fd414015040 tx=0x7fd414004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:30.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.347+0000 7fd4197fa700 1 -- 172.21.15.16:0/2210689756 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4140092e0 con 0x7fd41c06f450 2023-05-20T01:26:30.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.347+0000 7fd4197fa700 1 -- 172.21.15.16:0/2210689756 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd414004240 con 0x7fd41c06f450 2023-05-20T01:26:30.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.347+0000 7fd422acb700 1 -- 172.21.15.16:0/2210689756 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd41c0cbe50 con 0x7fd41c06f450 2023-05-20T01:26:30.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.347+0000 7fd4197fa700 1 -- 172.21.15.16:0/2210689756 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4140045f0 con 0x7fd41c06f450 2023-05-20T01:26:30.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.347+0000 7fd422acb700 1 -- 172.21.15.16:0/2210689756 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd41c0cc1a0 con 0x7fd41c06f450 2023-05-20T01:26:30.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.348+0000 7fd422acb700 1 -- 172.21.15.16:0/2210689756 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd41c0cc980 con 0x7fd41c06f450 2023-05-20T01:26:30.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.349+0000 7fd4197fa700 1 -- 172.21.15.16:0/2210689756 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd414019070 con 0x7fd41c06f450 2023-05-20T01:26:30.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.349+0000 7fd4197fa700 1 --2- 172.21.15.16:0/2210689756 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd40006ace0 0x7fd40006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:30.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.349+0000 7fd420867700 1 --2- 172.21.15.16:0/2210689756 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd40006ace0 0x7fd40006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:30.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.349+0000 7fd4197fa700 1 -- 172.21.15.16:0/2210689756 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5874+0+0 (secure 0 0 0) 0x7fd41408a240 con 0x7fd41c06f450 2023-05-20T01:26:30.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.350+0000 7fd420867700 1 --2- 172.21.15.16:0/2210689756 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd40006ace0 0x7fd40006d190 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fd410009960 tx=0x7fd4100090e0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:30.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.354+0000 7fd4197fa700 1 -- 172.21.15.16:0/2210689756 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fd414058ec0 con 0x7fd41c06f450 2023-05-20T01:26:30.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.598+0000 7fd422acb700 1 -- 172.21.15.16:0/2210689756 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd41c0cfd80 con 0x7fd41c06f450 2023-05-20T01:26:30.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.599+0000 7fd4197fa700 1 -- 172.21.15.16:0/2210689756 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+3523 (secure 0 0 0) 0x7fd414020de0 con 0x7fd41c06f450 2023-05-20T01:26:30.603 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:26:30.603 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":72,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6323,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/59032067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":59032067},{"type":"v1","addr":"172.21.15.132:6834","nonce":59032067}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66},{"gid":6329,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/100162236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":100162236},{"type":"v1","addr":"172.21.15.132:6835","nonce":100162236}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66},{"gid":7572,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/3753560731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3753560731},{"type":"v1","addr":"172.21.15.16:6836","nonce":3753560731}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66},{"gid":7830,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/898596831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":898596831},{"type":"v1","addr":"172.21.15.16:6837","nonce":898596831}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":72}],"filesystems":[]} 2023-05-20T01:26:30.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.600+0000 7fd422acb700 1 -- 172.21.15.16:0/2210689756 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd40006ace0 msgr2=0x7fd40006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:30.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.600+0000 7fd422acb700 1 --2- 172.21.15.16:0/2210689756 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd40006ace0 0x7fd40006d190 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fd410009960 tx=0x7fd4100090e0 comp rx=0 tx=0).stop 2023-05-20T01:26:30.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.600+0000 7fd422acb700 1 -- 172.21.15.16:0/2210689756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd41c06f450 msgr2=0x7fd41c06e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:30.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.600+0000 7fd422acb700 1 --2- 172.21.15.16:0/2210689756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd41c06f450 0x7fd41c06e080 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7fd414015040 tx=0x7fd414004ad0 comp rx=0 tx=0).stop 2023-05-20T01:26:30.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.601+0000 7fd422acb700 1 -- 172.21.15.16:0/2210689756 shutdown_connections 2023-05-20T01:26:30.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.601+0000 7fd422acb700 1 --2- 172.21.15.16:0/2210689756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd41c06f450 0x7fd41c06e080 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:30.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.601+0000 7fd422acb700 1 --2- 172.21.15.16:0/2210689756 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd41c06a510 0x7fd41c0c3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:30.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.601+0000 7fd422acb700 1 --2- 172.21.15.16:0/2210689756 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd40006ace0 0x7fd40006d190 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:30.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.601+0000 7fd422acb700 1 --2- 172.21.15.16:0/2210689756 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd41c06d4b0 0x7fd41c1587f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:30.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.601+0000 7fd422acb700 1 -- 172.21.15.16:0/2210689756 >> 172.21.15.16:0/2210689756 conn(0x7fd41c0cffb0 msgr2=0x7fd41c0ccdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:30.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.601+0000 7fd422acb700 1 -- 172.21.15.16:0/2210689756 shutdown_connections 2023-05-20T01:26:30.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.601+0000 7fd422acb700 1 -- 172.21.15.16:0/2210689756 wait complete. 2023-05-20T01:26:30.607 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 72 2023-05-20T01:26:30.617 INFO:tasks.ceph.mds.a:Restarting daemon 2023-05-20T01:26:30.618 INFO:tasks.ceph.mds.a:Stopping old one... 2023-05-20T01:26:30.618 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-05-20T01:26:30.619 INFO:teuthology.orchestra.run:waiting for 300 2023-05-20T01:26:30.619 INFO:tasks.ceph.mds.c:Restarting daemon 2023-05-20T01:26:30.619 INFO:tasks.ceph.mds.c:Stopping old one... 2023-05-20T01:26:30.619 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-05-20T01:26:30.620 INFO:teuthology.orchestra.run:waiting for 300 2023-05-20T01:26:30.620 INFO:tasks.ceph.mds.b:Restarting daemon 2023-05-20T01:26:30.620 INFO:tasks.ceph.mds.b:Stopping old one... 2023-05-20T01:26:30.620 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-05-20T01:26:30.620 INFO:teuthology.orchestra.run:waiting for 300 2023-05-20T01:26:30.621 INFO:tasks.ceph.mds.d:Restarting daemon 2023-05-20T01:26:30.621 INFO:tasks.ceph.mds.d:Stopping old one... 2023-05-20T01:26:30.621 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-05-20T01:26:30.621 INFO:teuthology.orchestra.run:waiting for 300 2023-05-20T01:26:30.629 INFO:tasks.ceph.mds.a:Stopped 2023-05-20T01:26:30.630 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-05-20T01:26:30.630 INFO:tasks.ceph.mds.c:Stopped 2023-05-20T01:26:30.631 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-05-20T01:26:30.632 INFO:tasks.ceph.mds.b:Stopped 2023-05-20T01:26:30.632 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-05-20T01:26:30.633 INFO:tasks.ceph.mds.d:Stopped 2023-05-20T01:26:30.633 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-05-20T01:26:30.672 INFO:tasks.ceph.mds.a:Started 2023-05-20T01:26:30.672 DEBUG:teuthology.parallel:result is None 2023-05-20T01:26:30.673 INFO:tasks.ceph.mds.c:Started 2023-05-20T01:26:30.674 DEBUG:teuthology.parallel:result is None 2023-05-20T01:26:30.674 INFO:tasks.ceph.mds.b:Started 2023-05-20T01:26:30.675 DEBUG:teuthology.parallel:result is None 2023-05-20T01:26:30.675 INFO:tasks.ceph.mds.d:Started 2023-05-20T01:26:30.676 DEBUG:teuthology.parallel:result is None 2023-05-20T01:26:30.676 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-05-20T01:26:30.754 INFO:tasks.ceph.mds.c.smithi016.stdout:starting mds.c at 2023-05-20T01:26:30.755 INFO:tasks.ceph.mds.a.smithi016.stdout:starting mds.a at 2023-05-20T01:26:30.763 INFO:tasks.ceph.mds.b.smithi132.stdout:starting mds.b at 2023-05-20T01:26:30.763 INFO:tasks.ceph.mds.d.smithi132.stdout:starting mds.d at 2023-05-20T01:26:30.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.806+0000 7f2fe6fc3700 1 Processor -- start 2023-05-20T01:26:30.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.806+0000 7f2fe6fc3700 1 -- start start 2023-05-20T01:26:30.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.807+0000 7f2fe6fc3700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f2fe00b2760 0x7f2fe00d1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:30.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.807+0000 7f2fe6fc3700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2fe0144020 0x7f2fe00b0980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:30.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.807+0000 7f2fe6fc3700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fe00cc500 con 0x7f2fe00cec60 2023-05-20T01:26:30.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.807+0000 7f2fe6fc3700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fe00cb6e0 con 0x7f2fe0144020 2023-05-20T01:26:30.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.807+0000 7f2fe6fc3700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fe01448a0 con 0x7f2fe00b2760 2023-05-20T01:26:30.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.807+0000 7f2fdffff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2fe0144020 0x7f2fe00b0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:30.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.807+0000 7f2fe5560700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f2fe00cec60 0x7f2fe00cc990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49416/0 (socket says 172.21.15.16:49416) 2023-05-20T01:26:30.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.807+0000 7f2fdffff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2fe0144020 0x7f2fe00b0980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37884/0 (socket says 172.21.15.16:37884) 2023-05-20T01:26:30.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.807+0000 7f2fe5560700 1 -- 172.21.15.16:0/1955089192 learned_addr learned my addr 172.21.15.16:0/1955089192 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:26:30.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.807+0000 7f2fe4d5f700 1 --2- 172.21.15.16:0/1955089192 >> v2:172.21.15.132:3300/0 conn(0x7f2fe00b2760 0x7f2fe00d1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:30.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.807+0000 7f2fdffff700 1 -- 172.21.15.16:0/1955089192 >> v2:172.21.15.132:3300/0 conn(0x7f2fe00b2760 msgr2=0x7f2fe00d1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:30.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.807+0000 7f2fdffff700 1 --2- 172.21.15.16:0/1955089192 >> v2:172.21.15.132:3300/0 conn(0x7f2fe00b2760 0x7f2fe00d1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:30.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.807+0000 7f2fdffff700 1 -- 172.21.15.16:0/1955089192 >> v1:172.21.15.16:6789/0 conn(0x7f2fe00cec60 legacy=0x7f2fe00cc990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:30.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.807+0000 7f2fdffff700 1 -- 172.21.15.16:0/1955089192 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fe00b0ec0 con 0x7f2fe0144020 2023-05-20T01:26:30.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.807+0000 7f2fe4d5f700 1 --2- 172.21.15.16:0/1955089192 >> v2:172.21.15.132:3300/0 conn(0x7f2fe00b2760 0x7f2fe00d1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:26:30.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.807+0000 7f2fdffff700 1 --2- 172.21.15.16:0/1955089192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fe0144020 0x7f2fe00b0980 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f2fe0070d30 tx=0x7f2fcc009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8677f5f6418fa7 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:30.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.808+0000 7f2fdf7fe700 1 -- 172.21.15.16:0/1955089192 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fcc011830 con 0x7f2fe0144020 2023-05-20T01:26:30.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.808+0000 7f2fdf7fe700 1 -- 172.21.15.16:0/1955089192 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fcc011990 con 0x7f2fe0144020 2023-05-20T01:26:30.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.808+0000 7f2fdf7fe700 1 -- 172.21.15.16:0/1955089192 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fcc011d20 con 0x7f2fe0144020 2023-05-20T01:26:30.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.808+0000 7f2fe6fc3700 1 -- 172.21.15.16:0/1955089192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fe0144020 msgr2=0x7f2fe00b0980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:30.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.808+0000 7f2fe6fc3700 1 --2- 172.21.15.16:0/1955089192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fe0144020 0x7f2fe00b0980 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f2fe0070d30 tx=0x7f2fcc009b20 comp rx=0 tx=0).stop 2023-05-20T01:26:30.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.808+0000 7f2fe6fc3700 1 -- 172.21.15.16:0/1955089192 shutdown_connections 2023-05-20T01:26:30.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.808+0000 7f2fe6fc3700 1 --2- 172.21.15.16:0/1955089192 >> v2:172.21.15.132:3300/0 conn(0x7f2fe00b2760 0x7f2fe00d1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:30.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.808+0000 7f2fe6fc3700 1 --2- 172.21.15.16:0/1955089192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fe0144020 0x7f2fe00b0980 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:30.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.808+0000 7f2fe6fc3700 1 -- 172.21.15.16:0/1955089192 >> 172.21.15.16:0/1955089192 conn(0x7f2fe006ae30 msgr2=0x7f2fe009a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:30.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.808+0000 7f2fe6fc3700 1 -- 172.21.15.16:0/1955089192 shutdown_connections 2023-05-20T01:26:30.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.808+0000 7f2fe6fc3700 1 -- 172.21.15.16:0/1955089192 wait complete. 2023-05-20T01:26:30.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.809+0000 7f2fe6fc3700 1 Processor -- start 2023-05-20T01:26:30.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.809+0000 7f2fe6fc3700 1 -- start start 2023-05-20T01:26:30.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.810+0000 7f2fe6fc3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fe00b2760 0x7f2fe015a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:30.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.810+0000 7f2fe6fc3700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2fe00cec60 0x7f2fe0159540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:30.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.810+0000 7f2fe6fc3700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2fe0144020 0x7f2fe015bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:30.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.810+0000 7f2fe6fc3700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe015a680 con 0x7f2fe00b2760 2023-05-20T01:26:30.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.810+0000 7f2fe6fc3700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe015baf0 con 0x7f2fe0144020 2023-05-20T01:26:30.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.810+0000 7f2fe6fc3700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe015b1c0 con 0x7f2fe00cec60 2023-05-20T01:26:30.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.810+0000 7f2fe4d5f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fe00b2760 0x7f2fe015a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:30.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.810+0000 7f2fe4d5f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fe00b2760 0x7f2fe015a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37890/0 (socket says 172.21.15.16:37890) 2023-05-20T01:26:30.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.810+0000 7f2fe4d5f700 1 -- 172.21.15.16:0/2856102864 learned_addr learned my addr 172.21.15.16:0/2856102864 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:26:30.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.811+0000 7f2fe5560700 1 --2- 172.21.15.16:0/2856102864 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2fe0144020 0x7f2fe015bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:30.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.811+0000 7f2fdffff700 1 --2- 172.21.15.16:0/2856102864 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2fe00cec60 0x7f2fe0159540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:30.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.811+0000 7f2fe4d5f700 1 -- 172.21.15.16:0/2856102864 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2fe00cec60 msgr2=0x7f2fe0159540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:30.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.811+0000 7f2fe4d5f700 1 --2- 172.21.15.16:0/2856102864 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2fe00cec60 0x7f2fe0159540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:30.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.811+0000 7f2fe4d5f700 1 -- 172.21.15.16:0/2856102864 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2fe0144020 msgr2=0x7f2fe015bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:30.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.811+0000 7f2fe4d5f700 1 --2- 172.21.15.16:0/2856102864 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2fe0144020 0x7f2fe015bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:30.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.811+0000 7f2fe4d5f700 1 -- 172.21.15.16:0/2856102864 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fcc0094a0 con 0x7f2fe00b2760 2023-05-20T01:26:30.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.811+0000 7f2fdffff700 1 --2- 172.21.15.16:0/2856102864 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2fe00cec60 0x7f2fe0159540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:26:30.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.811+0000 7f2fe4d5f700 1 --2- 172.21.15.16:0/2856102864 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fe00b2760 0x7f2fe015a140 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f2fd400c7f0 tx=0x7f2fd400cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:30.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.811+0000 7f2fdd7fa700 1 -- 172.21.15.16:0/2856102864 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd4011800 con 0x7f2fe00b2760 2023-05-20T01:26:30.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.811+0000 7f2fdd7fa700 1 -- 172.21.15.16:0/2856102864 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fd4011960 con 0x7f2fe00b2760 2023-05-20T01:26:30.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.811+0000 7f2fdd7fa700 1 -- 172.21.15.16:0/2856102864 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd4011d10 con 0x7f2fe00b2760 2023-05-20T01:26:30.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.811+0000 7f2fe6fc3700 1 -- 172.21.15.16:0/2856102864 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fe00cbbd0 con 0x7f2fe00b2760 2023-05-20T01:26:30.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.811+0000 7f2fe6fc3700 1 -- 172.21.15.16:0/2856102864 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fe00cb8f0 con 0x7f2fe00b2760 2023-05-20T01:26:30.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.812+0000 7f2fe6fc3700 1 -- 172.21.15.16:0/2856102864 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fe00cbeb0 con 0x7f2fe00b2760 2023-05-20T01:26:30.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.812+0000 7f2fdd7fa700 1 -- 172.21.15.16:0/2856102864 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2fd4018360 con 0x7f2fe00b2760 2023-05-20T01:26:30.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.813+0000 7f2fdd7fa700 1 --2- 172.21.15.16:0/2856102864 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f2fc406ace0 0x7f2fc406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:30.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.813+0000 7f2fdd7fa700 1 -- 172.21.15.16:0/2856102864 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5874+0+0 (secure 0 0 0) 0x7f2fd4089360 con 0x7f2fe00b2760 2023-05-20T01:26:30.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.813+0000 7f2fdffff700 1 --2- 172.21.15.16:0/2856102864 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f2fc406ace0 0x7f2fc406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:30.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.813+0000 7f2fdffff700 1 --2- 172.21.15.16:0/2856102864 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f2fc406ace0 0x7f2fc406d190 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc0090b0 tx=0x7f2fcc008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:30.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:30.817+0000 7f2fdd7fa700 1 -- 172.21.15.16:0/2856102864 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f2fd4057fe0 con 0x7f2fe00b2760 2023-05-20T01:26:31.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:31.031+0000 7f2fe6fc3700 1 -- 172.21.15.16:0/2856102864 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f2fe00747b0 con 0x7f2fe00b2760 2023-05-20T01:26:31.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:31.808+0000 7f2fdd7fa700 1 -- 172.21.15.16:0/2856102864 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v108) v1 ==== 97+0+0 (secure 0 0 0) 0x7f2fd4057b70 con 0x7f2fe00b2760 2023-05-20T01:26:31.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:31.811+0000 7f2fe6fc3700 1 -- 172.21.15.16:0/2856102864 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f2fc406ace0 msgr2=0x7f2fc406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:31.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:31.811+0000 7f2fe6fc3700 1 --2- 172.21.15.16:0/2856102864 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f2fc406ace0 0x7f2fc406d190 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc0090b0 tx=0x7f2fcc008030 comp rx=0 tx=0).stop 2023-05-20T01:26:31.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:31.811+0000 7f2fe6fc3700 1 -- 172.21.15.16:0/2856102864 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fe00b2760 msgr2=0x7f2fe015a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:31.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:31.811+0000 7f2fe6fc3700 1 --2- 172.21.15.16:0/2856102864 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fe00b2760 0x7f2fe015a140 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f2fd400c7f0 tx=0x7f2fd400cbb0 comp rx=0 tx=0).stop 2023-05-20T01:26:31.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:31.811+0000 7f2fe6fc3700 1 -- 172.21.15.16:0/2856102864 shutdown_connections 2023-05-20T01:26:31.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:31.811+0000 7f2fe6fc3700 1 --2- 172.21.15.16:0/2856102864 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2fe0144020 0x7f2fe015bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:31.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:31.811+0000 7f2fe6fc3700 1 --2- 172.21.15.16:0/2856102864 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fe00b2760 0x7f2fe015a140 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:31.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:31.811+0000 7f2fe6fc3700 1 --2- 172.21.15.16:0/2856102864 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f2fc406ace0 0x7f2fc406d190 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:31.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:31.811+0000 7f2fe6fc3700 1 --2- 172.21.15.16:0/2856102864 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2fe00cec60 0x7f2fe0159540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:31.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:31.811+0000 7f2fe6fc3700 1 -- 172.21.15.16:0/2856102864 >> 172.21.15.16:0/2856102864 conn(0x7f2fe006ae30 msgr2=0x7f2fe006e390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:31.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:31.811+0000 7f2fe6fc3700 1 -- 172.21.15.16:0/2856102864 shutdown_connections 2023-05-20T01:26:31.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:31.811+0000 7f2fe6fc3700 1 -- 172.21.15.16:0/2856102864 wait complete. 2023-05-20T01:26:31.819 INFO:teuthology.orchestra.run.smithi016.stderr: removed all blocklist entries 2023-05-20T01:26:31.830 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-05-20T01:26:31.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:31.958+0000 7f646095a700 1 Processor -- start 2023-05-20T01:26:31.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:31.958+0000 7f646095a700 1 -- start start 2023-05-20T01:26:31.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:31.959+0000 7f646095a700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f645c1599c0 0x7f645c0caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:31.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:31.959+0000 7f646095a700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f645c0caaa0 0x7f645c0cb4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:31.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:31.959+0000 7f646095a700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f645c159d90 con 0x7f645c06f3f0 2023-05-20T01:26:31.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:31.959+0000 7f646095a700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f645c0cdf20 con 0x7f645c0caaa0 2023-05-20T01:26:31.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:31.959+0000 7f646095a700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f645c0cdb80 con 0x7f645c1599c0 2023-05-20T01:26:31.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:31.959+0000 7f645ad9d700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f645c06f3f0 0x7f645c0710d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49434/0 (socket says 172.21.15.16:49434) 2023-05-20T01:26:31.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:31.959+0000 7f645ad9d700 1 -- 172.21.15.16:0/871672109 learned_addr learned my addr 172.21.15.16:0/871672109 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:26:31.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:31.959+0000 7f645a59c700 1 --2- 172.21.15.16:0/871672109 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f645c1599c0 0x7f645c0caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:31.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:31.959+0000 7f6459d9b700 1 --2- 172.21.15.16:0/871672109 >> v2:172.21.15.132:3300/0 conn(0x7f645c0caaa0 0x7f645c0cb4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:31.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:31.959+0000 7f645959a700 1 -- 172.21.15.16:0/871672109 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 144170610 0 0) 0x7f6454001490 con 0x7f645c06f3f0 2023-05-20T01:26:31.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:31.959+0000 7f645959a700 1 -- 172.21.15.16:0/871672109 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64440035b0 con 0x7f645c06f3f0 2023-05-20T01:26:31.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:31.960+0000 7f645959a700 1 -- 172.21.15.16:0/871672109 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2110434354 0 0) 0x7f64440035b0 con 0x7f645c06f3f0 2023-05-20T01:26:31.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:31.960+0000 7f645959a700 1 -- 172.21.15.16:0/871672109 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f645c1599c0 msgr2=0x7f645c0caf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:31.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:31.960+0000 7f645959a700 1 --2- 172.21.15.16:0/871672109 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f645c1599c0 0x7f645c0caf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:31.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:31.960+0000 7f645959a700 1 -- 172.21.15.16:0/871672109 >> v2:172.21.15.132:3300/0 conn(0x7f645c0caaa0 msgr2=0x7f645c0cb4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:31.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:31.960+0000 7f645959a700 1 --2- 172.21.15.16:0/871672109 >> v2:172.21.15.132:3300/0 conn(0x7f645c0caaa0 0x7f645c0cb4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:31.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:31.960+0000 7f645959a700 1 -- 172.21.15.16:0/871672109 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f645c06eaf0 con 0x7f645c06f3f0 2023-05-20T01:26:31.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:31.960+0000 7f645959a700 1 -- 172.21.15.16:0/871672109 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7f6454001b80 con 0x7f645c06f3f0 2023-05-20T01:26:31.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:31.960+0000 7f645959a700 1 -- 172.21.15.16:0/871672109 >> v1:172.21.15.16:6789/0 conn(0x7f645c06f3f0 legacy=0x7f645c0710d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:31.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:31.960+0000 7f645959a700 1 --2- 172.21.15.16:0/871672109 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6444003a70 0x7f6444003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:31.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:31.960+0000 7f645959a700 1 --2- 172.21.15.16:0/871672109 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f645c1599c0 0x7f6444004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:31.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:31.960+0000 7f645959a700 1 --2- 172.21.15.16:0/871672109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6444004d90 0x7f6444005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:31.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:31.960+0000 7f645959a700 1 -- 172.21.15.16:0/871672109 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f645c0cdf20 con 0x7f6444004d90 2023-05-20T01:26:31.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:31.960+0000 7f645959a700 1 -- 172.21.15.16:0/871672109 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f645c0cdb80 con 0x7f6444003a70 2023-05-20T01:26:31.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:31.960+0000 7f645959a700 1 -- 172.21.15.16:0/871672109 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f644400ba10 con 0x7f645c1599c0 2023-05-20T01:26:31.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:31.960+0000 7f645ad9d700 1 --2- 172.21.15.16:0/871672109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6444004d90 0x7f6444005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:31.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:31.960+0000 7f645a59c700 1 --2- 172.21.15.16:0/871672109 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6444003a70 0x7f6444003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:31.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:31.960+0000 7f6459d9b700 1 --2- 172.21.15.16:0/871672109 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f645c1599c0 0x7f6444004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:31.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:31.960+0000 7f645ad9d700 1 -- 172.21.15.16:0/871672109 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f645c1599c0 msgr2=0x7f6444004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:31.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:31.960+0000 7f645ad9d700 1 --2- 172.21.15.16:0/871672109 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f645c1599c0 0x7f6444004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:31.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:31.960+0000 7f645ad9d700 1 -- 172.21.15.16:0/871672109 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6444003a70 msgr2=0x7f6444003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:31.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:31.960+0000 7f645ad9d700 1 --2- 172.21.15.16:0/871672109 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6444003a70 0x7f6444003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:31.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:31.960+0000 7f645ad9d700 1 -- 172.21.15.16:0/871672109 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64440035b0 con 0x7f6444004d90 2023-05-20T01:26:31.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:31.960+0000 7f645ad9d700 1 --2- 172.21.15.16:0/871672109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6444004d90 0x7f6444005200 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f645400a860 tx=0x7f6444005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b5f879c0c03cde3 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:31.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:31.961+0000 7f645959a700 1 -- 172.21.15.16:0/871672109 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f645400a4c0 con 0x7f6444004d90 2023-05-20T01:26:31.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:31.961+0000 7f645959a700 1 -- 172.21.15.16:0/871672109 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6454012600 con 0x7f6444004d90 2023-05-20T01:26:31.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:31.961+0000 7f645959a700 1 -- 172.21.15.16:0/871672109 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64540129b0 con 0x7f6444004d90 2023-05-20T01:26:31.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:31.961+0000 7f646095a700 1 -- 172.21.15.16:0/871672109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6444004d90 msgr2=0x7f6444005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:31.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:31.961+0000 7f646095a700 1 --2- 172.21.15.16:0/871672109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6444004d90 0x7f6444005200 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f645400a860 tx=0x7f6444005880 comp rx=0 tx=0).stop 2023-05-20T01:26:31.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:31.961+0000 7f645ad9d700 1 -- 172.21.15.16:0/871672109 reap_dead start 2023-05-20T01:26:31.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:31.961+0000 7f646095a700 1 -- 172.21.15.16:0/871672109 shutdown_connections 2023-05-20T01:26:31.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:31.961+0000 7f646095a700 1 -- 172.21.15.16:0/871672109 >> 172.21.15.16:0/871672109 conn(0x7f645c0d0c30 msgr2=0x7f645c0b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:31.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:31.961+0000 7f646095a700 1 -- 172.21.15.16:0/871672109 shutdown_connections 2023-05-20T01:26:31.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:31.961+0000 7f646095a700 1 -- 172.21.15.16:0/871672109 wait complete. 2023-05-20T01:26:31.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:31.963+0000 7f646095a700 1 Processor -- start 2023-05-20T01:26:31.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:31.963+0000 7f646095a700 1 -- start start 2023-05-20T01:26:31.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:31.963+0000 7f646095a700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6444004d90 0x7f645c152600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:31.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:31.964+0000 7f646095a700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f645c159f30 0x7f645c0aef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:31.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:31.964+0000 7f646095a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f645c0af460 0x7f645c0c9250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:31.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:31.964+0000 7f646095a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f645c152b40 con 0x7f645c0af460 2023-05-20T01:26:31.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:31.964+0000 7f646095a700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f645c0c4790 con 0x7f6444004d90 2023-05-20T01:26:31.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:31.964+0000 7f646095a700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f645c0c4a70 con 0x7f645c159f30 2023-05-20T01:26:31.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:31.964+0000 7f645ad9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f645c0af460 0x7f645c0c9250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:31.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:31.964+0000 7f645ad9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f645c0af460 0x7f645c0c9250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37918/0 (socket says 172.21.15.16:37918) 2023-05-20T01:26:31.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:31.964+0000 7f645ad9d700 1 -- 172.21.15.16:0/4276375486 learned_addr learned my addr 172.21.15.16:0/4276375486 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:26:31.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:31.964+0000 7f6459d9b700 1 --2- 172.21.15.16:0/4276375486 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f645c159f30 0x7f645c0aef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:31.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:31.964+0000 7f645a59c700 1 --2- 172.21.15.16:0/4276375486 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6444004d90 0x7f645c152600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:31.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:31.964+0000 7f645ad9d700 1 -- 172.21.15.16:0/4276375486 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f645c159f30 msgr2=0x7f645c0aef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:31.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:31.964+0000 7f645ad9d700 1 --2- 172.21.15.16:0/4276375486 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f645c159f30 0x7f645c0aef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:31.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:31.964+0000 7f645ad9d700 1 -- 172.21.15.16:0/4276375486 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6444004d90 msgr2=0x7f645c152600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:31.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:31.964+0000 7f645ad9d700 1 --2- 172.21.15.16:0/4276375486 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6444004d90 0x7f645c152600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:31.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:31.964+0000 7f645ad9d700 1 -- 172.21.15.16:0/4276375486 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f645c06f0b0 con 0x7f645c0af460 2023-05-20T01:26:31.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:31.964+0000 7f645a59c700 1 --2- 172.21.15.16:0/4276375486 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6444004d90 0x7f645c152600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:26:31.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:31.964+0000 7f645ad9d700 1 --2- 172.21.15.16:0/4276375486 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f645c0af460 0x7f645c0c9250 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f6454012000 tx=0x7f6454001360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:31.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:31.964+0000 7f644affd700 1 -- 172.21.15.16:0/4276375486 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6454001b50 con 0x7f645c0af460 2023-05-20T01:26:31.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:31.964+0000 7f644affd700 1 -- 172.21.15.16:0/4276375486 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64540011c0 con 0x7f645c0af460 2023-05-20T01:26:31.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:31.964+0000 7f646095a700 1 -- 172.21.15.16:0/4276375486 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f645c06edd0 con 0x7f645c0af460 2023-05-20T01:26:31.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:31.964+0000 7f644affd700 1 -- 172.21.15.16:0/4276375486 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f645401e670 con 0x7f645c0af460 2023-05-20T01:26:31.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:31.964+0000 7f646095a700 1 -- 172.21.15.16:0/4276375486 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f645c06e750 con 0x7f645c0af460 2023-05-20T01:26:31.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:31.965+0000 7f646095a700 1 -- 172.21.15.16:0/4276375486 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f645c0c5960 con 0x7f645c0af460 2023-05-20T01:26:31.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:31.966+0000 7f644affd700 1 -- 172.21.15.16:0/4276375486 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f645401e7d0 con 0x7f645c0af460 2023-05-20T01:26:31.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:31.966+0000 7f644affd700 1 --2- 172.21.15.16:0/4276375486 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f643c06ace0 0x7f643c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:31.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:31.966+0000 7f644affd700 1 -- 172.21.15.16:0/4276375486 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f645408aba0 con 0x7f645c0af460 2023-05-20T01:26:31.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:31.966+0000 7f645a59c700 1 --2- 172.21.15.16:0/4276375486 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f643c06ace0 0x7f643c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:31.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:31.967+0000 7f645a59c700 1 --2- 172.21.15.16:0/4276375486 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f643c06ace0 0x7f643c06d190 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f6450009b50 tx=0x7f6450008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:31.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:31.971+0000 7f644affd700 1 -- 172.21.15.16:0/4276375486 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f6454056010 con 0x7f645c0af460 2023-05-20T01:26:32.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.249+0000 7f646095a700 1 -- 172.21.15.16:0/4276375486 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f645c159d90 con 0x7f645c0af460 2023-05-20T01:26:32.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.249+0000 7f644affd700 1 -- 172.21.15.16:0/4276375486 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v12) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f645c159d90 con 0x7f645c0af460 2023-05-20T01:26:32.254 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:26:32.254 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-05-20T01:26:32.255 INFO:teuthology.orchestra.run.smithi016.stdout: "auth_dump": [ 2023-05-20T01:26:32.255 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-05-20T01:26:32.255 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.a", 2023-05-20T01:26:32.255 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQA5H2hkq5FEJBAA1qUUPSLmwo43hTQS7GSENA==", 2023-05-20T01:26:32.255 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-05-20T01:26:32.255 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-05-20T01:26:32.255 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2023-05-20T01:26:32.256 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2023-05-20T01:26:32.256 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-05-20T01:26:32.256 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:26:32.256 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-05-20T01:26:32.256 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-05-20T01:26:32.256 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.b", 2023-05-20T01:26:32.257 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQA5H2hk/2XSMBAAnGCIm7TJZsO9HyXwOWxO7w==", 2023-05-20T01:26:32.257 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-05-20T01:26:32.257 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-05-20T01:26:32.257 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2023-05-20T01:26:32.257 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2023-05-20T01:26:32.257 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-05-20T01:26:32.257 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:26:32.258 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-05-20T01:26:32.258 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-05-20T01:26:32.258 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.c", 2023-05-20T01:26:32.258 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQA5H2hkBVK8KxAAgxy9+gbttxZeygwrAX+PHQ==", 2023-05-20T01:26:32.258 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-05-20T01:26:32.258 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-05-20T01:26:32.258 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2023-05-20T01:26:32.258 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2023-05-20T01:26:32.259 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-05-20T01:26:32.259 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:26:32.259 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-05-20T01:26:32.259 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-05-20T01:26:32.259 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.d", 2023-05-20T01:26:32.260 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQA5H2hkFxEMOBAAMPvue7F+/qqOFUD/C//U+g==", 2023-05-20T01:26:32.260 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-05-20T01:26:32.260 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-05-20T01:26:32.260 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2023-05-20T01:26:32.260 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2023-05-20T01:26:32.260 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-05-20T01:26:32.260 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:26:32.261 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-05-20T01:26:32.261 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-05-20T01:26:32.261 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.0", 2023-05-20T01:26:32.261 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQA7H2hkC4WwIRAA5KMQFqldcekzEuH92rZYHA==", 2023-05-20T01:26:32.261 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-05-20T01:26:32.261 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-05-20T01:26:32.261 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-05-20T01:26:32.262 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-05-20T01:26:32.262 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:26:32.262 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-05-20T01:26:32.262 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-05-20T01:26:32.263 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.1", 2023-05-20T01:26:32.263 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQA+H2hkvMH1IBAAhOQOjCiv3r2Pl3HcSS6eAA==", 2023-05-20T01:26:32.263 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-05-20T01:26:32.263 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-05-20T01:26:32.263 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-05-20T01:26:32.264 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-05-20T01:26:32.264 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:26:32.264 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-05-20T01:26:32.264 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-05-20T01:26:32.264 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.2", 2023-05-20T01:26:32.264 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQBBH2hkSJU5IBAAdFd4IhXxISVybFyT24gb0g==", 2023-05-20T01:26:32.264 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-05-20T01:26:32.265 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-05-20T01:26:32.265 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-05-20T01:26:32.265 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-05-20T01:26:32.265 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:26:32.265 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-05-20T01:26:32.265 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-05-20T01:26:32.265 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.3", 2023-05-20T01:26:32.266 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQBEH2hkrcH8HxAANlLWDyIYycLl1TrYCBN1hQ==", 2023-05-20T01:26:32.266 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-05-20T01:26:32.266 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-05-20T01:26:32.266 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-05-20T01:26:32.266 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-05-20T01:26:32.266 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:26:32.266 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-05-20T01:26:32.267 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-05-20T01:26:32.267 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.4", 2023-05-20T01:26:32.267 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQBKH2hkZr+4JBAAzbzKd4ulGI8kVGqXpjCDog==", 2023-05-20T01:26:32.267 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-05-20T01:26:32.267 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-05-20T01:26:32.267 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-05-20T01:26:32.267 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-05-20T01:26:32.267 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:26:32.268 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-05-20T01:26:32.268 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-05-20T01:26:32.268 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.5", 2023-05-20T01:26:32.268 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQBNH2hkNV02JBAALrC7eOTnD1dQM9njk+BgLQ==", 2023-05-20T01:26:32.268 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-05-20T01:26:32.268 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-05-20T01:26:32.268 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-05-20T01:26:32.269 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-05-20T01:26:32.269 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:26:32.269 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-05-20T01:26:32.269 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-05-20T01:26:32.269 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.6", 2023-05-20T01:26:32.269 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQBQH2hkGOfBIxAABCTVPLBi5+drKpqOC8xqdw==", 2023-05-20T01:26:32.269 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-05-20T01:26:32.270 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-05-20T01:26:32.270 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-05-20T01:26:32.270 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-05-20T01:26:32.270 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:26:32.270 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-05-20T01:26:32.270 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-05-20T01:26:32.270 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.7", 2023-05-20T01:26:32.271 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQBTH2hkpwEdIxAAaI3X5ZAVJ9MfVG0MjFJvSQ==", 2023-05-20T01:26:32.271 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-05-20T01:26:32.271 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-05-20T01:26:32.271 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-05-20T01:26:32.271 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-05-20T01:26:32.271 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:26:32.271 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-05-20T01:26:32.272 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-05-20T01:26:32.272 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.0", 2023-05-20T01:26:32.272 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQA6H2hkph3EABAA0277JHCqplw1LklDfidPWQ==", 2023-05-20T01:26:32.272 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-05-20T01:26:32.272 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-05-20T01:26:32.273 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow r", 2023-05-20T01:26:32.273 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-05-20T01:26:32.273 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:26:32.273 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-05-20T01:26:32.274 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-05-20T01:26:32.274 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.1", 2023-05-20T01:26:32.274 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQA6H2hk43WQBxAApSfBSu1qEaKSGWaYAvz/Jg==", 2023-05-20T01:26:32.274 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-05-20T01:26:32.274 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-05-20T01:26:32.274 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2023-05-20T01:26:32.274 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2023-05-20T01:26:32.275 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2023-05-20T01:26:32.275 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:26:32.275 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-05-20T01:26:32.275 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-05-20T01:26:32.275 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.2", 2023-05-20T01:26:32.275 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQA6H2hk+TgtBBAAc4+W2eOVgJfNd3xXjGvz4Q==", 2023-05-20T01:26:32.275 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-05-20T01:26:32.276 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-05-20T01:26:32.276 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2023-05-20T01:26:32.276 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2023-05-20T01:26:32.280 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2023-05-20T01:26:32.280 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:26:32.280 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-05-20T01:26:32.280 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-05-20T01:26:32.281 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.3", 2023-05-20T01:26:32.281 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQA6H2hkifVADRAAQP6vtdVNYyeNPV0d+agOCw==", 2023-05-20T01:26:32.281 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-05-20T01:26:32.281 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-05-20T01:26:32.281 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2023-05-20T01:26:32.281 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2023-05-20T01:26:32.281 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2023-05-20T01:26:32.282 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:26:32.282 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-05-20T01:26:32.282 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-05-20T01:26:32.282 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.admin", 2023-05-20T01:26:32.282 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQA5H2hkb4BMBhAAw7zpfDAb7x5szkLY+zTL7Q==", 2023-05-20T01:26:32.282 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-05-20T01:26:32.282 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2023-05-20T01:26:32.283 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2023-05-20T01:26:32.283 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2023-05-20T01:26:32.283 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-05-20T01:26:32.283 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:26:32.283 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-05-20T01:26:32.283 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-05-20T01:26:32.283 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mgr.x", 2023-05-20T01:26:32.284 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQA5H2hksQbMIBAAQFm/pY5stJHNudg6zcPBdQ==", 2023-05-20T01:26:32.284 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-05-20T01:26:32.284 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2023-05-20T01:26:32.284 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile mgr", 2023-05-20T01:26:32.284 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-05-20T01:26:32.284 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:26:32.284 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-05-20T01:26:32.285 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-05-20T01:26:32.285 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mgr.y", 2023-05-20T01:26:32.285 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQA5H2hkBQg7HBAAbOJZ+55rRACVBMd2eyjqtg==", 2023-05-20T01:26:32.285 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-05-20T01:26:32.285 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2023-05-20T01:26:32.285 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile mgr", 2023-05-20T01:26:32.285 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-05-20T01:26:32.286 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:26:32.286 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:26:32.286 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2023-05-20T01:26:32.286 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-05-20T01:26:32.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.252+0000 7f646095a700 1 -- 172.21.15.16:0/4276375486 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f643c06ace0 msgr2=0x7f643c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:32.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.252+0000 7f646095a700 1 --2- 172.21.15.16:0/4276375486 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f643c06ace0 0x7f643c06d190 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f6450009b50 tx=0x7f6450008040 comp rx=0 tx=0).stop 2023-05-20T01:26:32.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.252+0000 7f646095a700 1 -- 172.21.15.16:0/4276375486 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f645c0af460 msgr2=0x7f645c0c9250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:32.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.252+0000 7f646095a700 1 --2- 172.21.15.16:0/4276375486 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f645c0af460 0x7f645c0c9250 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f6454012000 tx=0x7f6454001360 comp rx=0 tx=0).stop 2023-05-20T01:26:32.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.252+0000 7f646095a700 1 -- 172.21.15.16:0/4276375486 shutdown_connections 2023-05-20T01:26:32.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.252+0000 7f646095a700 1 --2- 172.21.15.16:0/4276375486 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f645c0af460 0x7f645c0c9250 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:32.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.252+0000 7f646095a700 1 --2- 172.21.15.16:0/4276375486 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6444004d90 0x7f645c152600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:32.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.252+0000 7f646095a700 1 --2- 172.21.15.16:0/4276375486 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f643c06ace0 0x7f643c06d190 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:32.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.252+0000 7f646095a700 1 --2- 172.21.15.16:0/4276375486 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f645c159f30 0x7f645c0aef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:32.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.252+0000 7f646095a700 1 -- 172.21.15.16:0/4276375486 >> 172.21.15.16:0/4276375486 conn(0x7f645c0d0c30 msgr2=0x7f645c0cbf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:32.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.252+0000 7f646095a700 1 -- 172.21.15.16:0/4276375486 shutdown_connections 2023-05-20T01:26:32.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.252+0000 7f646095a700 1 -- 172.21.15.16:0/4276375486 wait complete. 2023-05-20T01:26:32.290 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-20T01:26:32.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.411+0000 7fe6ed123700 1 Processor -- start 2023-05-20T01:26:32.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.411+0000 7fe6ed123700 1 -- start start 2023-05-20T01:26:32.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.412+0000 7fe6ed123700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fe6e80af2d0 0x7fe6e80cec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:32.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.412+0000 7fe6ed123700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe6e80cf160 0x7fe6e80b0980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:32.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.412+0000 7fe6ed123700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6e80b2000 con 0x7fe6e80d16c0 2023-05-20T01:26:32.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.412+0000 7fe6ed123700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6e80cf5e0 con 0x7fe6e80af2d0 2023-05-20T01:26:32.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.412+0000 7fe6ed123700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e80b22e0 con 0x7fe6e80cf160 2023-05-20T01:26:32.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.412+0000 7fe6e6d9d700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fe6e80d16c0 0x7fe6e8144330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49450/0 (socket says 172.21.15.16:49450) 2023-05-20T01:26:32.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.412+0000 7fe6e6d9d700 1 -- 172.21.15.16:0/4076920323 learned_addr learned my addr 172.21.15.16:0/4076920323 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:26:32.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.413+0000 7fe6e659c700 1 --2- 172.21.15.16:0/4076920323 >> v2:172.21.15.132:3300/0 conn(0x7fe6e80af2d0 0x7fe6e80cec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:32.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.413+0000 7fe6e759e700 1 --2- 172.21.15.16:0/4076920323 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe6e80cf160 0x7fe6e80b0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:32.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.413+0000 7fe6e5d9b700 1 -- 172.21.15.16:0/4076920323 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 864818603 0 0) 0x7fe6e80b2000 con 0x7fe6e80d16c0 2023-05-20T01:26:32.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.413+0000 7fe6e5d9b700 1 -- 172.21.15.16:0/4076920323 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6d00035b0 con 0x7fe6e80d16c0 2023-05-20T01:26:32.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.413+0000 7fe6e659c700 1 -- 172.21.15.16:0/4076920323 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe6e80cf160 msgr2=0x7fe6e80b0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:32.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.413+0000 7fe6e659c700 1 --2- 172.21.15.16:0/4076920323 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe6e80cf160 0x7fe6e80b0980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:32.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.413+0000 7fe6e659c700 1 -- 172.21.15.16:0/4076920323 >> v1:172.21.15.16:6789/0 conn(0x7fe6e80d16c0 legacy=0x7fe6e8144330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:32.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.413+0000 7fe6e659c700 1 -- 172.21.15.16:0/4076920323 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6e80b25c0 con 0x7fe6e80af2d0 2023-05-20T01:26:32.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.413+0000 7fe6e659c700 1 --2- 172.21.15.16:0/4076920323 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe6e80af2d0 0x7fe6e80cec20 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fe6d8009870 tx=0x7fe6d8009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=546feb6dadd4b51 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:32.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.413+0000 7fe6e5d9b700 1 -- 172.21.15.16:0/4076920323 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6d8011830 con 0x7fe6e80af2d0 2023-05-20T01:26:32.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.413+0000 7fe6e5d9b700 1 -- 172.21.15.16:0/4076920323 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6d8011990 con 0x7fe6e80af2d0 2023-05-20T01:26:32.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.413+0000 7fe6e5d9b700 1 -- 172.21.15.16:0/4076920323 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6d8011d00 con 0x7fe6e80af2d0 2023-05-20T01:26:32.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.414+0000 7fe6ed123700 1 -- 172.21.15.16:0/4076920323 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe6e80af2d0 msgr2=0x7fe6e80cec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:32.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.414+0000 7fe6ed123700 1 --2- 172.21.15.16:0/4076920323 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe6e80af2d0 0x7fe6e80cec20 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fe6d8009870 tx=0x7fe6d8009b80 comp rx=0 tx=0).stop 2023-05-20T01:26:32.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.414+0000 7fe6ed123700 1 -- 172.21.15.16:0/4076920323 shutdown_connections 2023-05-20T01:26:32.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.414+0000 7fe6ed123700 1 --2- 172.21.15.16:0/4076920323 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe6e80cf160 0x7fe6e80b0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:32.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.414+0000 7fe6ed123700 1 --2- 172.21.15.16:0/4076920323 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe6e80af2d0 0x7fe6e80cec20 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:32.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.414+0000 7fe6ed123700 1 -- 172.21.15.16:0/4076920323 >> 172.21.15.16:0/4076920323 conn(0x7fe6e806ae30 msgr2=0x7fe6e809a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:32.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.414+0000 7fe6ed123700 1 -- 172.21.15.16:0/4076920323 shutdown_connections 2023-05-20T01:26:32.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.414+0000 7fe6ed123700 1 -- 172.21.15.16:0/4076920323 wait complete. 2023-05-20T01:26:32.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.415+0000 7fe6ed123700 1 Processor -- start 2023-05-20T01:26:32.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.416+0000 7fe6ed123700 1 -- start start 2023-05-20T01:26:32.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.416+0000 7fe6ed123700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe6e80af2d0 0x7fe6e80cd980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:32.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.416+0000 7fe6ed123700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6e80cf160 0x7fe6e80cdec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:32.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.416+0000 7fe6ed123700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe6e80d16c0 0x7fe6e80c8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:32.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.416+0000 7fe6ed123700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e80c6320 con 0x7fe6e80cf160 2023-05-20T01:26:32.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.416+0000 7fe6ed123700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e81f1cb0 con 0x7fe6e80af2d0 2023-05-20T01:26:32.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.416+0000 7fe6ed123700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e81f1980 con 0x7fe6e80d16c0 2023-05-20T01:26:32.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.416+0000 7fe6e659c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6e80cf160 0x7fe6e80cdec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:32.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.416+0000 7fe6e659c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6e80cf160 0x7fe6e80cdec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37926/0 (socket says 172.21.15.16:37926) 2023-05-20T01:26:32.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.416+0000 7fe6e659c700 1 -- 172.21.15.16:0/2055309293 learned_addr learned my addr 172.21.15.16:0/2055309293 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:26:32.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.417+0000 7fe6e6d9d700 1 --2- 172.21.15.16:0/2055309293 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe6e80af2d0 0x7fe6e80cd980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:32.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.417+0000 7fe6e759e700 1 --2- 172.21.15.16:0/2055309293 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe6e80d16c0 0x7fe6e80c8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:32.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.417+0000 7fe6e659c700 1 -- 172.21.15.16:0/2055309293 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe6e80d16c0 msgr2=0x7fe6e80c8b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:32.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.417+0000 7fe6e659c700 1 --2- 172.21.15.16:0/2055309293 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe6e80d16c0 0x7fe6e80c8b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:32.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.417+0000 7fe6e659c700 1 -- 172.21.15.16:0/2055309293 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe6e80af2d0 msgr2=0x7fe6e80cd980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:32.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.417+0000 7fe6e659c700 1 --2- 172.21.15.16:0/2055309293 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe6e80af2d0 0x7fe6e80cd980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:32.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.417+0000 7fe6e659c700 1 -- 172.21.15.16:0/2055309293 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6d80094a0 con 0x7fe6e80cf160 2023-05-20T01:26:32.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.417+0000 7fe6e6d9d700 1 --2- 172.21.15.16:0/2055309293 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe6e80af2d0 0x7fe6e80cd980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:26:32.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.417+0000 7fe6e659c700 1 --2- 172.21.15.16:0/2055309293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6e80cf160 0x7fe6e80cdec0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7fe6d8000c00 tx=0x7fe6d8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:32.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.417+0000 7fe6cf7fe700 1 -- 172.21.15.16:0/2055309293 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6d80045c0 con 0x7fe6e80cf160 2023-05-20T01:26:32.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.417+0000 7fe6ed123700 1 -- 172.21.15.16:0/2055309293 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6e80b28a0 con 0x7fe6e80cf160 2023-05-20T01:26:32.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.417+0000 7fe6cf7fe700 1 -- 172.21.15.16:0/2055309293 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6d801f070 con 0x7fe6e80cf160 2023-05-20T01:26:32.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.417+0000 7fe6cf7fe700 1 -- 172.21.15.16:0/2055309293 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6d80038c0 con 0x7fe6e80cf160 2023-05-20T01:26:32.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.417+0000 7fe6ed123700 1 -- 172.21.15.16:0/2055309293 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6e80adda0 con 0x7fe6e80cf160 2023-05-20T01:26:32.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.418+0000 7fe6ed123700 1 -- 172.21.15.16:0/2055309293 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6e80b0ec0 con 0x7fe6e80cf160 2023-05-20T01:26:32.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.419+0000 7fe6cf7fe700 1 -- 172.21.15.16:0/2055309293 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe6d8003a20 con 0x7fe6e80cf160 2023-05-20T01:26:32.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.419+0000 7fe6cf7fe700 1 --2- 172.21.15.16:0/2055309293 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe6d406ad10 0x7fe6d406d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:32.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.419+0000 7fe6cf7fe700 1 -- 172.21.15.16:0/2055309293 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5530+0+0 (secure 0 0 0) 0x7fe6d8021070 con 0x7fe6e80cf160 2023-05-20T01:26:32.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.419+0000 7fe6e6d9d700 1 --2- 172.21.15.16:0/2055309293 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe6d406ad10 0x7fe6d406d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:32.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.420+0000 7fe6e6d9d700 1 --2- 172.21.15.16:0/2055309293 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe6d406ad10 0x7fe6d406d1c0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fe6dc005910 tx=0x7fe6dc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:32.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.424+0000 7fe6cf7fe700 1 -- 172.21.15.16:0/2055309293 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fe6d8055e20 con 0x7fe6e80cf160 2023-05-20T01:26:32.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.620+0000 7fe6ed123700 1 -- 172.21.15.16:0/2055309293 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe6e80cf8e0 con 0x7fe6e80cf160 2023-05-20T01:26:32.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.620+0000 7fe6cf7fe700 1 -- 172.21.15.16:0/2055309293 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v108) v1 ==== 74+0+14752 (secure 0 0 0) 0x7fe6d8015090 con 0x7fe6e80cf160 2023-05-20T01:26:32.625 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:26:32.625 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":108,"fsid":"c2245cc5-ecab-4087-a59b-bc81db901676","created":"2023-05-20T01:16:09.549230+0000","modified":"2023-05-20T01:26:31.798061+0000","last_up_change":"2023-05-20T01:16:19.788624+0000","last_in_change":"2023-05-20T01:16:12.092160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-05-20T01:16:20.934366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-05-20T01:16:21.372294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"317a83cd-19c8-4958-a301-6087ef342289","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6825","nonce":3923634179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6827","nonce":3923634179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6831","nonce":3923634179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6829","nonce":3923634179}]},"public_addr":"172.21.15.16:6825/3923634179","cluster_addr":"172.21.15.16:6827/3923634179","heartbeat_back_addr":"172.21.15.16:6831/3923634179","heartbeat_front_addr":"172.21.15.16:6829/3923634179","state":["exists","up"]},{"osd":1,"uuid":"5581046f-8219-4089-8d7d-b7846399505e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6809","nonce":3860264220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6811","nonce":3860264220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6815","nonce":3860264220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6813","nonce":3860264220}]},"public_addr":"172.21.15.16:6809/3860264220","cluster_addr":"172.21.15.16:6811/3860264220","heartbeat_back_addr":"172.21.15.16:6815/3860264220","heartbeat_front_addr":"172.21.15.16:6813/3860264220","state":["exists","up"]},{"osd":2,"uuid":"072e9e66-4133-478e-8d4a-64d46c2963f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6817","nonce":293187030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6819","nonce":293187030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6823","nonce":293187030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6821","nonce":293187030}]},"public_addr":"172.21.15.16:6817/293187030","cluster_addr":"172.21.15.16:6819/293187030","heartbeat_back_addr":"172.21.15.16:6823/293187030","heartbeat_front_addr":"172.21.15.16:6821/293187030","state":["exists","up"]},{"osd":3,"uuid":"11e1bba6-00f8-48dd-9028-d8d716b7f145","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6801","nonce":3669781890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6803","nonce":3669781890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6807","nonce":3669781890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6805","nonce":3669781890}]},"public_addr":"172.21.15.16:6801/3669781890","cluster_addr":"172.21.15.16:6803/3669781890","heartbeat_back_addr":"172.21.15.16:6807/3669781890","heartbeat_front_addr":"172.21.15.16:6805/3669781890","state":["exists","up"]},{"osd":4,"uuid":"b8909c3a-cf70-46cb-8d0b-009c707cf9d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6819","nonce":506317223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6823","nonce":506317223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6829","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6831","nonce":506317223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6827","nonce":506317223}]},"public_addr":"172.21.15.132:6819/506317223","cluster_addr":"172.21.15.132:6823/506317223","heartbeat_back_addr":"172.21.15.132:6831/506317223","heartbeat_front_addr":"172.21.15.132:6827/506317223","state":["exists","up"]},{"osd":5,"uuid":"b1818304-4583-47bf-a6ec-0bfbb5166233","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6818","nonce":1844689995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6822","nonce":1844689995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6830","nonce":1844689995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6826","nonce":1844689995}]},"public_addr":"172.21.15.132:6818/1844689995","cluster_addr":"172.21.15.132:6822/1844689995","heartbeat_back_addr":"172.21.15.132:6830/1844689995","heartbeat_front_addr":"172.21.15.132:6826/1844689995","state":["exists","up"]},{"osd":6,"uuid":"d37f4691-eed1-4a93-8818-18354f03f4a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6801","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6802","nonce":727581123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6805","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6807","nonce":727581123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6813","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6815","nonce":727581123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6811","nonce":727581123}]},"public_addr":"172.21.15.132:6802/727581123","cluster_addr":"172.21.15.132:6807/727581123","heartbeat_back_addr":"172.21.15.132:6815/727581123","heartbeat_front_addr":"172.21.15.132:6811/727581123","state":["exists","up"]},{"osd":7,"uuid":"4f712f62-5d1d-4816-8042-03209144f8b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6803","nonce":466001149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6806","nonce":466001149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6814","nonce":466001149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6810","nonce":466001149}]},"public_addr":"172.21.15.132:6803/466001149","cluster_addr":"172.21.15.132:6806/466001149","heartbeat_back_addr":"172.21.15.132:6814/466001149","heartbeat_front_addr":"172.21.15.132:6810/466001149","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.338863+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.278680+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305694+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.246835+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.276237+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.304441+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305421+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.267050+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-20T01:26:32.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.623+0000 7fe6ed123700 1 -- 172.21.15.16:0/2055309293 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe6d406ad10 msgr2=0x7fe6d406d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:32.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.623+0000 7fe6ed123700 1 --2- 172.21.15.16:0/2055309293 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe6d406ad10 0x7fe6d406d1c0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fe6dc005910 tx=0x7fe6dc009040 comp rx=0 tx=0).stop 2023-05-20T01:26:32.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.623+0000 7fe6ed123700 1 -- 172.21.15.16:0/2055309293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6e80cf160 msgr2=0x7fe6e80cdec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:32.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.623+0000 7fe6ed123700 1 --2- 172.21.15.16:0/2055309293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6e80cf160 0x7fe6e80cdec0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7fe6d8000c00 tx=0x7fe6d8004750 comp rx=0 tx=0).stop 2023-05-20T01:26:32.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.623+0000 7fe6ed123700 1 -- 172.21.15.16:0/2055309293 shutdown_connections 2023-05-20T01:26:32.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.623+0000 7fe6ed123700 1 --2- 172.21.15.16:0/2055309293 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe6d406ad10 0x7fe6d406d1c0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:32.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.623+0000 7fe6ed123700 1 --2- 172.21.15.16:0/2055309293 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe6e80d16c0 0x7fe6e80c8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:32.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.623+0000 7fe6ed123700 1 --2- 172.21.15.16:0/2055309293 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe6e80af2d0 0x7fe6e80cd980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:32.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.623+0000 7fe6ed123700 1 --2- 172.21.15.16:0/2055309293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6e80cf160 0x7fe6e80cdec0 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:32.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.623+0000 7fe6ed123700 1 -- 172.21.15.16:0/2055309293 >> 172.21.15.16:0/2055309293 conn(0x7fe6e806ae30 msgr2=0x7fe6e80c79c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:32.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.623+0000 7fe6ed123700 1 -- 172.21.15.16:0/2055309293 shutdown_connections 2023-05-20T01:26:32.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.623+0000 7fe6ed123700 1 -- 172.21.15.16:0/2055309293 wait complete. 2023-05-20T01:26:32.640 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-05-20T01:16:20.934366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-05-20T01:16:21.372294+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-05-20T01:26:32.641 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-05-20T01:26:32.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.766+0000 7f3521782700 1 Processor -- start 2023-05-20T01:26:32.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.766+0000 7f3521782700 1 -- start start 2023-05-20T01:26:32.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.767+0000 7f3521782700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f351c0cd9e0 0x7f351c0cde30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:32.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.767+0000 7f3521782700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f351c15ccf0 0x7f351c15d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:32.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.767+0000 7f3521782700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f351c06c480 con 0x7f351c15c920 2023-05-20T01:26:32.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.767+0000 7f3521782700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f351c0d0e80 con 0x7f351c0cd9e0 2023-05-20T01:26:32.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.767+0000 7f3521782700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f351c0ce370 con 0x7f351c15ccf0 2023-05-20T01:26:32.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.767+0000 7f351a7fc700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f351c0cd9e0 0x7f351c0cde30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:32.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.767+0000 7f351a7fc700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f351c0cd9e0 0x7f351c0cde30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37934/0 (socket says 172.21.15.16:37934) 2023-05-20T01:26:32.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.767+0000 7f351a7fc700 1 -- 172.21.15.16:0/1667191136 learned_addr learned my addr 172.21.15.16:0/1667191136 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:26:32.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.768+0000 7f351b7fe700 1 --2- 172.21.15.16:0/1667191136 >> v2:172.21.15.132:3300/0 conn(0x7f351c15ccf0 0x7f351c15d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:32.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.768+0000 7f351a7fc700 1 -- 172.21.15.16:0/1667191136 >> v2:172.21.15.132:3300/0 conn(0x7f351c15ccf0 msgr2=0x7f351c15d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:32.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.768+0000 7f351a7fc700 1 --2- 172.21.15.16:0/1667191136 >> v2:172.21.15.132:3300/0 conn(0x7f351c15ccf0 0x7f351c15d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:32.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.768+0000 7f351a7fc700 1 -- 172.21.15.16:0/1667191136 >> v1:172.21.15.132:6789/0 conn(0x7f351c15c920 legacy=0x7f351c06fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:32.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.768+0000 7f351a7fc700 1 -- 172.21.15.16:0/1667191136 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f351c06d610 con 0x7f351c0cd9e0 2023-05-20T01:26:32.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.768+0000 7f351b7fe700 1 --2- 172.21.15.16:0/1667191136 >> v2:172.21.15.132:3300/0 conn(0x7f351c15ccf0 0x7f351c15d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:26:32.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.768+0000 7f351a7fc700 1 --2- 172.21.15.16:0/1667191136 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f351c0cd9e0 0x7f351c0cde30 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f351c0cee30 tx=0x7f350c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b95864da4f2bc1f8 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:32.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.768+0000 7f3519ffb700 1 -- 172.21.15.16:0/1667191136 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f350c00b870 con 0x7f351c0cd9e0 2023-05-20T01:26:32.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.768+0000 7f3519ffb700 1 -- 172.21.15.16:0/1667191136 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f350c00b9d0 con 0x7f351c0cd9e0 2023-05-20T01:26:32.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.768+0000 7f3519ffb700 1 -- 172.21.15.16:0/1667191136 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f350c00bd60 con 0x7f351c0cd9e0 2023-05-20T01:26:32.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.769+0000 7f3521782700 1 -- 172.21.15.16:0/1667191136 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f351c0cd9e0 msgr2=0x7f351c0cde30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:32.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.769+0000 7f3521782700 1 --2- 172.21.15.16:0/1667191136 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f351c0cd9e0 0x7f351c0cde30 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f351c0cee30 tx=0x7f350c009bd0 comp rx=0 tx=0).stop 2023-05-20T01:26:32.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.769+0000 7f3521782700 1 -- 172.21.15.16:0/1667191136 shutdown_connections 2023-05-20T01:26:32.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.769+0000 7f3521782700 1 --2- 172.21.15.16:0/1667191136 >> v2:172.21.15.132:3300/0 conn(0x7f351c15ccf0 0x7f351c15d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:32.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.769+0000 7f3521782700 1 --2- 172.21.15.16:0/1667191136 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f351c0cd9e0 0x7f351c0cde30 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:32.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.769+0000 7f3521782700 1 -- 172.21.15.16:0/1667191136 >> 172.21.15.16:0/1667191136 conn(0x7f351c06e2c0 msgr2=0x7f351c06e6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:32.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.769+0000 7f3521782700 1 -- 172.21.15.16:0/1667191136 shutdown_connections 2023-05-20T01:26:32.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.769+0000 7f3521782700 1 -- 172.21.15.16:0/1667191136 wait complete. 2023-05-20T01:26:32.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.770+0000 7f3521782700 1 Processor -- start 2023-05-20T01:26:32.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.770+0000 7f3521782700 1 -- start start 2023-05-20T01:26:32.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.771+0000 7f3521782700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f351c0cd9e0 0x7f351c15bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:32.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.771+0000 7f3521782700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f351c15c920 0x7f351c14d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:32.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.771+0000 7f3521782700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f351c15ccf0 0x7f351c06fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:32.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.771+0000 7f3521782700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f351c15e1b0 con 0x7f351c0cd9e0 2023-05-20T01:26:32.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.771+0000 7f3521782700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f351c15e400 con 0x7f351c15c920 2023-05-20T01:26:32.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.771+0000 7f3521782700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f351c15e6e0 con 0x7f351c15ccf0 2023-05-20T01:26:32.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.771+0000 7f351affd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f351c0cd9e0 0x7f351c15bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:32.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.771+0000 7f351affd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f351c0cd9e0 0x7f351c15bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37938/0 (socket says 172.21.15.16:37938) 2023-05-20T01:26:32.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.771+0000 7f351affd700 1 -- 172.21.15.16:0/2804688346 learned_addr learned my addr 172.21.15.16:0/2804688346 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:26:32.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.771+0000 7f351a7fc700 1 --2- 172.21.15.16:0/2804688346 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f351c15c920 0x7f351c14d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:32.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.771+0000 7f351b7fe700 1 --2- 172.21.15.16:0/2804688346 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f351c15ccf0 0x7f351c06fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:32.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.771+0000 7f351affd700 1 -- 172.21.15.16:0/2804688346 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f351c15ccf0 msgr2=0x7f351c06fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:32.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.771+0000 7f351affd700 1 --2- 172.21.15.16:0/2804688346 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f351c15ccf0 0x7f351c06fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:32.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.771+0000 7f351affd700 1 -- 172.21.15.16:0/2804688346 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f351c15c920 msgr2=0x7f351c14d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:32.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.771+0000 7f351affd700 1 --2- 172.21.15.16:0/2804688346 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f351c15c920 0x7f351c14d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:32.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.771+0000 7f351affd700 1 -- 172.21.15.16:0/2804688346 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f350c009550 con 0x7f351c0cd9e0 2023-05-20T01:26:32.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.771+0000 7f351b7fe700 1 --2- 172.21.15.16:0/2804688346 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f351c15ccf0 0x7f351c06fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:26:32.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.771+0000 7f351affd700 1 --2- 172.21.15.16:0/2804688346 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f351c0cd9e0 0x7f351c15bc20 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f351000c4e0 tx=0x7f351000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:32.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.772+0000 7f35037fe700 1 -- 172.21.15.16:0/2804688346 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35100052d0 con 0x7f351c0cd9e0 2023-05-20T01:26:32.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.772+0000 7f35037fe700 1 -- 172.21.15.16:0/2804688346 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3510005430 con 0x7f351c0cd9e0 2023-05-20T01:26:32.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.772+0000 7f3521782700 1 -- 172.21.15.16:0/2804688346 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f351c06b900 con 0x7f351c0cd9e0 2023-05-20T01:26:32.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.772+0000 7f35037fe700 1 -- 172.21.15.16:0/2804688346 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35100038c0 con 0x7f351c0cd9e0 2023-05-20T01:26:32.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.772+0000 7f3521782700 1 -- 172.21.15.16:0/2804688346 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f351c06b620 con 0x7f351c0cd9e0 2023-05-20T01:26:32.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.773+0000 7f3521782700 1 -- 172.21.15.16:0/2804688346 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f351c06bbe0 con 0x7f351c0cd9e0 2023-05-20T01:26:32.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.773+0000 7f35037fe700 1 -- 172.21.15.16:0/2804688346 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3510003a20 con 0x7f351c0cd9e0 2023-05-20T01:26:32.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.774+0000 7f35037fe700 1 --2- 172.21.15.16:0/2804688346 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f350806ace0 0x7f350806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:32.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.774+0000 7f35037fe700 1 -- 172.21.15.16:0/2804688346 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f35100887a0 con 0x7f351c0cd9e0 2023-05-20T01:26:32.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.774+0000 7f351a7fc700 1 --2- 172.21.15.16:0/2804688346 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f350806ace0 0x7f350806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:32.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.774+0000 7f351a7fc700 1 --2- 172.21.15.16:0/2804688346 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f350806ace0 0x7f350806d190 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f351c157720 tx=0x7f350c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:32.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.778+0000 7f35037fe700 1 -- 172.21.15.16:0/2804688346 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f3510053a50 con 0x7f351c0cd9e0 2023-05-20T01:26:32.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.975+0000 7f3521782700 1 -- 172.21.15.16:0/2804688346 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f351c066ce0 con 0x7f351c0cd9e0 2023-05-20T01:26:32.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.975+0000 7f35037fe700 1 -- 172.21.15.16:0/2804688346 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v108) v1 ==== 92+0+10 (secure 0 0 0) 0x7f3510057070 con 0x7f351c0cd9e0 2023-05-20T01:26:32.980 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 8 2023-05-20T01:26:32.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.977+0000 7f3521782700 1 -- 172.21.15.16:0/2804688346 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f350806ace0 msgr2=0x7f350806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:32.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.977+0000 7f3521782700 1 --2- 172.21.15.16:0/2804688346 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f350806ace0 0x7f350806d190 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f351c157720 tx=0x7f350c008030 comp rx=0 tx=0).stop 2023-05-20T01:26:32.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.977+0000 7f3521782700 1 -- 172.21.15.16:0/2804688346 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f351c0cd9e0 msgr2=0x7f351c15bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:32.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.977+0000 7f3521782700 1 --2- 172.21.15.16:0/2804688346 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f351c0cd9e0 0x7f351c15bc20 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f351000c4e0 tx=0x7f351000c8a0 comp rx=0 tx=0).stop 2023-05-20T01:26:32.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.977+0000 7f3521782700 1 -- 172.21.15.16:0/2804688346 shutdown_connections 2023-05-20T01:26:32.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.977+0000 7f3521782700 1 --2- 172.21.15.16:0/2804688346 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f350806ace0 0x7f350806d190 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:32.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.977+0000 7f3521782700 1 --2- 172.21.15.16:0/2804688346 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f351c15ccf0 0x7f351c06fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:32.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.977+0000 7f3521782700 1 --2- 172.21.15.16:0/2804688346 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f351c0cd9e0 0x7f351c15bc20 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:32.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.977+0000 7f3521782700 1 --2- 172.21.15.16:0/2804688346 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f351c15c920 0x7f351c14d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:32.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.977+0000 7f3521782700 1 -- 172.21.15.16:0/2804688346 >> 172.21.15.16:0/2804688346 conn(0x7f351c06e2c0 msgr2=0x7f351c0b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:32.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.978+0000 7f3521782700 1 -- 172.21.15.16:0/2804688346 shutdown_connections 2023-05-20T01:26:32.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:32.978+0000 7f3521782700 1 -- 172.21.15.16:0/2804688346 wait complete. 2023-05-20T01:26:32.994 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-05-20T01:26:33.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.117+0000 7f41222d4700 1 Processor -- start 2023-05-20T01:26:33.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.117+0000 7f41222d4700 1 -- start start 2023-05-20T01:26:33.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.118+0000 7f41222d4700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f411c0cb700 0x7f411c159e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:33.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.118+0000 7f41222d4700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f411c071e10 0x7f411c15da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:33.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.118+0000 7f41222d4700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f411c15d5b0 con 0x7f411c157450 2023-05-20T01:26:33.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.118+0000 7f41222d4700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f411c157820 con 0x7f411c071e10 2023-05-20T01:26:33.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.118+0000 7f41222d4700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f411c15a4c0 con 0x7f411c0cb700 2023-05-20T01:26:33.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.118+0000 7f411bfff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f411c157450 0x7f411c0caff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49474/0 (socket says 172.21.15.16:49474) 2023-05-20T01:26:33.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.118+0000 7f411bfff700 1 -- 172.21.15.16:0/1182790698 learned_addr learned my addr 172.21.15.16:0/1182790698 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:26:33.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.118+0000 7f4120871700 1 --2- 172.21.15.16:0/1182790698 >> v2:172.21.15.132:3300/0 conn(0x7f411c071e10 0x7f411c15da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:33.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.118+0000 7f411b7fe700 1 --2- 172.21.15.16:0/1182790698 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f411c0cb700 0x7f411c159e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:33.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.118+0000 7f411affd700 1 -- 172.21.15.16:0/1182790698 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1795553455 0 0) 0x7f411c15d5b0 con 0x7f411c157450 2023-05-20T01:26:33.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.118+0000 7f411affd700 1 -- 172.21.15.16:0/1182790698 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41080035b0 con 0x7f411c157450 2023-05-20T01:26:33.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.118+0000 7f411affd700 1 -- 172.21.15.16:0/1182790698 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1789859194 0 0) 0x7f41080035b0 con 0x7f411c157450 2023-05-20T01:26:33.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.118+0000 7f411affd700 1 -- 172.21.15.16:0/1182790698 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f411c0cb700 msgr2=0x7f411c159e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:33.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.118+0000 7f411affd700 1 --2- 172.21.15.16:0/1182790698 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f411c0cb700 0x7f411c159e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:33.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.118+0000 7f411affd700 1 -- 172.21.15.16:0/1182790698 >> v2:172.21.15.132:3300/0 conn(0x7f411c071e10 msgr2=0x7f411c15da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:33.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.118+0000 7f411affd700 1 --2- 172.21.15.16:0/1182790698 >> v2:172.21.15.132:3300/0 conn(0x7f411c071e10 0x7f411c15da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:33.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.118+0000 7f411affd700 1 -- 172.21.15.16:0/1182790698 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f411c15a7a0 con 0x7f411c157450 2023-05-20T01:26:33.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.118+0000 7f411affd700 1 -- 172.21.15.16:0/1182790698 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7f4104001670 con 0x7f411c157450 2023-05-20T01:26:33.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.118+0000 7f411affd700 1 -- 172.21.15.16:0/1182790698 >> v1:172.21.15.16:6789/0 conn(0x7f411c157450 legacy=0x7f411c0caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:33.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.119+0000 7f411affd700 1 --2- 172.21.15.16:0/1182790698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4108003a00 0x7f4108003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:33.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.119+0000 7f411affd700 1 --2- 172.21.15.16:0/1182790698 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4108004430 0x7f41080048f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:33.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.119+0000 7f411affd700 1 --2- 172.21.15.16:0/1182790698 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f411c0cb700 0x7f4108004fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:33.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.119+0000 7f411affd700 1 -- 172.21.15.16:0/1182790698 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f411c157820 con 0x7f4108003a00 2023-05-20T01:26:33.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.119+0000 7f411affd700 1 -- 172.21.15.16:0/1182790698 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f411c15a4c0 con 0x7f4108004430 2023-05-20T01:26:33.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.119+0000 7f411affd700 1 -- 172.21.15.16:0/1182790698 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f410800ba10 con 0x7f411c0cb700 2023-05-20T01:26:33.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.119+0000 7f411bfff700 1 --2- 172.21.15.16:0/1182790698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4108003a00 0x7f4108003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:33.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.119+0000 7f411b7fe700 1 --2- 172.21.15.16:0/1182790698 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4108004430 0x7f41080048f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:33.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.119+0000 7f4120871700 1 --2- 172.21.15.16:0/1182790698 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f411c0cb700 0x7f4108004fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:33.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.119+0000 7f411bfff700 1 -- 172.21.15.16:0/1182790698 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f411c0cb700 msgr2=0x7f4108004fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:33.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.119+0000 7f411bfff700 1 --2- 172.21.15.16:0/1182790698 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f411c0cb700 0x7f4108004fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:33.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.119+0000 7f411bfff700 1 -- 172.21.15.16:0/1182790698 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4108004430 msgr2=0x7f41080048f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:33.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.119+0000 7f411bfff700 1 --2- 172.21.15.16:0/1182790698 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4108004430 0x7f41080048f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:33.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.119+0000 7f411bfff700 1 -- 172.21.15.16:0/1182790698 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41080035b0 con 0x7f4108003a00 2023-05-20T01:26:33.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.119+0000 7f411b7fe700 1 --2- 172.21.15.16:0/1182790698 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4108004430 0x7f41080048f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:26:33.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.119+0000 7f411bfff700 1 --2- 172.21.15.16:0/1182790698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4108003a00 0x7f4108003e10 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f410400c320 tx=0x7f4108005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e27dbd51d0639fd server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:33.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.119+0000 7f411affd700 1 -- 172.21.15.16:0/1182790698 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f410400ce60 con 0x7f4108003a00 2023-05-20T01:26:33.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.119+0000 7f411affd700 1 -- 172.21.15.16:0/1182790698 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4104005960 con 0x7f4108003a00 2023-05-20T01:26:33.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.119+0000 7f411affd700 1 -- 172.21.15.16:0/1182790698 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4104005cf0 con 0x7f4108003a00 2023-05-20T01:26:33.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.120+0000 7f41222d4700 1 -- 172.21.15.16:0/1182790698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4108003a00 msgr2=0x7f4108003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:33.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.120+0000 7f41222d4700 1 --2- 172.21.15.16:0/1182790698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4108003a00 0x7f4108003e10 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f410400c320 tx=0x7f4108005880 comp rx=0 tx=0).stop 2023-05-20T01:26:33.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.120+0000 7f4120871700 1 -- 172.21.15.16:0/1182790698 reap_dead start 2023-05-20T01:26:33.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.120+0000 7f41222d4700 1 -- 172.21.15.16:0/1182790698 shutdown_connections 2023-05-20T01:26:33.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.120+0000 7f41222d4700 1 -- 172.21.15.16:0/1182790698 >> 172.21.15.16:0/1182790698 conn(0x7f411c06cbb0 msgr2=0x7f411c06cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:33.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.120+0000 7f41222d4700 1 -- 172.21.15.16:0/1182790698 shutdown_connections 2023-05-20T01:26:33.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.120+0000 7f41222d4700 1 -- 172.21.15.16:0/1182790698 wait complete. 2023-05-20T01:26:33.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.121+0000 7f41222d4700 1 Processor -- start 2023-05-20T01:26:33.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.122+0000 7f41222d4700 1 -- start start 2023-05-20T01:26:33.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.123+0000 7f41222d4700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4108003a00 0x7f411c145f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:33.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.123+0000 7f41222d4700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f411c1f4420 0x7f411c06e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:33.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.123+0000 7f41222d4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f411c06e690 0x7f411c1488d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:33.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.123+0000 7f41222d4700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f411c1f4000 con 0x7f411c06e690 2023-05-20T01:26:33.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.123+0000 7f41222d4700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f411c06eb30 con 0x7f411c1f4420 2023-05-20T01:26:33.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.123+0000 7f41222d4700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f411c1498a0 con 0x7f4108003a00 2023-05-20T01:26:33.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.123+0000 7f4120871700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f411c06e690 0x7f411c1488d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:33.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.123+0000 7f4120871700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f411c06e690 0x7f411c1488d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37962/0 (socket says 172.21.15.16:37962) 2023-05-20T01:26:33.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.123+0000 7f4120871700 1 -- 172.21.15.16:0/1971274699 learned_addr learned my addr 172.21.15.16:0/1971274699 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:26:33.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.123+0000 7f411bfff700 1 --2- 172.21.15.16:0/1971274699 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4108003a00 0x7f411c145f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:33.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.123+0000 7f411b7fe700 1 --2- 172.21.15.16:0/1971274699 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f411c1f4420 0x7f411c06e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:33.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.123+0000 7f4120871700 1 -- 172.21.15.16:0/1971274699 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4108003a00 msgr2=0x7f411c145f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:33.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.123+0000 7f4120871700 1 --2- 172.21.15.16:0/1971274699 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4108003a00 0x7f411c145f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:33.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.123+0000 7f4120871700 1 -- 172.21.15.16:0/1971274699 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f411c1f4420 msgr2=0x7f411c06e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:33.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.123+0000 7f4120871700 1 --2- 172.21.15.16:0/1971274699 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f411c1f4420 0x7f411c06e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:33.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.123+0000 7f4120871700 1 -- 172.21.15.16:0/1971274699 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f411c15ad60 con 0x7f411c06e690 2023-05-20T01:26:33.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.123+0000 7f411bfff700 1 --2- 172.21.15.16:0/1971274699 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4108003a00 0x7f411c145f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:26:33.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.123+0000 7f4120871700 1 --2- 172.21.15.16:0/1971274699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f411c06e690 0x7f411c1488d0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f411c0c6810 tx=0x7f411400f490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:33.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.123+0000 7f4118ff9700 1 -- 172.21.15.16:0/1971274699 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f411400e040 con 0x7f411c06e690 2023-05-20T01:26:33.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.123+0000 7f4118ff9700 1 -- 172.21.15.16:0/1971274699 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f411400f040 con 0x7f411c06e690 2023-05-20T01:26:33.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.123+0000 7f41222d4700 1 -- 172.21.15.16:0/1971274699 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f411c15aa80 con 0x7f411c06e690 2023-05-20T01:26:33.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.123+0000 7f4118ff9700 1 -- 172.21.15.16:0/1971274699 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4114015430 con 0x7f411c06e690 2023-05-20T01:26:33.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.123+0000 7f41222d4700 1 -- 172.21.15.16:0/1971274699 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f411c073b00 con 0x7f411c06e690 2023-05-20T01:26:33.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.124+0000 7f41222d4700 1 -- 172.21.15.16:0/1971274699 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f411c0747b0 con 0x7f411c06e690 2023-05-20T01:26:33.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.125+0000 7f4118ff9700 1 -- 172.21.15.16:0/1971274699 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4114015590 con 0x7f411c06e690 2023-05-20T01:26:33.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.125+0000 7f4118ff9700 1 --2- 172.21.15.16:0/1971274699 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f40fc06ace0 0x7f40fc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:33.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.126+0000 7f411bfff700 1 --2- 172.21.15.16:0/1971274699 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f40fc06ace0 0x7f40fc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:33.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.126+0000 7f4118ff9700 1 -- 172.21.15.16:0/1971274699 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f4114089a80 con 0x7f411c06e690 2023-05-20T01:26:33.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.126+0000 7f411bfff700 1 --2- 172.21.15.16:0/1971274699 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f40fc06ace0 0x7f40fc06d190 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f4104015040 tx=0x7f410400a000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:33.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.130+0000 7f4118ff9700 1 -- 172.21.15.16:0/1971274699 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f4114054dd0 con 0x7f411c06e690 2023-05-20T01:26:33.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.325+0000 7f41222d4700 1 -- 172.21.15.16:0/1971274699 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f411c1f4140 con 0x7f411c06e690 2023-05-20T01:26:33.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.325+0000 7f4118ff9700 1 -- 172.21.15.16:0/1971274699 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v108) v1 ==== 93+0+10 (secure 0 0 0) 0x7f411c1f4140 con 0x7f411c06e690 2023-05-20T01:26:33.330 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-05-20T01:26:33.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.328+0000 7f41222d4700 1 -- 172.21.15.16:0/1971274699 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f40fc06ace0 msgr2=0x7f40fc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:33.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.328+0000 7f41222d4700 1 --2- 172.21.15.16:0/1971274699 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f40fc06ace0 0x7f40fc06d190 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f4104015040 tx=0x7f410400a000 comp rx=0 tx=0).stop 2023-05-20T01:26:33.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.328+0000 7f41222d4700 1 -- 172.21.15.16:0/1971274699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f411c06e690 msgr2=0x7f411c1488d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:33.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.328+0000 7f41222d4700 1 --2- 172.21.15.16:0/1971274699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f411c06e690 0x7f411c1488d0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f411c0c6810 tx=0x7f411400f490 comp rx=0 tx=0).stop 2023-05-20T01:26:33.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.328+0000 7f41222d4700 1 -- 172.21.15.16:0/1971274699 shutdown_connections 2023-05-20T01:26:33.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.328+0000 7f41222d4700 1 --2- 172.21.15.16:0/1971274699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f411c06e690 0x7f411c1488d0 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:33.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.328+0000 7f41222d4700 1 --2- 172.21.15.16:0/1971274699 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f40fc06ace0 0x7f40fc06d190 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:33.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.328+0000 7f41222d4700 1 --2- 172.21.15.16:0/1971274699 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4108003a00 0x7f411c145f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:33.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.328+0000 7f41222d4700 1 --2- 172.21.15.16:0/1971274699 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f411c1f4420 0x7f411c06e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:33.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.328+0000 7f41222d4700 1 -- 172.21.15.16:0/1971274699 >> 172.21.15.16:0/1971274699 conn(0x7f411c06cbb0 msgr2=0x7f411c1562c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:33.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.328+0000 7f41222d4700 1 -- 172.21.15.16:0/1971274699 shutdown_connections 2023-05-20T01:26:33.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.328+0000 7f41222d4700 1 -- 172.21.15.16:0/1971274699 wait complete. 2023-05-20T01:26:33.344 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-05-20T01:26:33.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.464+0000 7f7120338700 1 Processor -- start 2023-05-20T01:26:33.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.464+0000 7f7120338700 1 -- start start 2023-05-20T01:26:33.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.465+0000 7f7120338700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f711806cb90 0x7f711815ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:33.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.465+0000 7f7120338700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f71180b54f0 0x7f71180b29e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:33.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.465+0000 7f7120338700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71180b2f20 con 0x7f711806cf60 2023-05-20T01:26:33.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.465+0000 7f7120338700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71180cc990 con 0x7f71180b54f0 2023-05-20T01:26:33.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.465+0000 7f7120338700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71180b31a0 con 0x7f711806cb90 2023-05-20T01:26:33.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.466+0000 7f711e8d5700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f711806cf60 0x7f7118070910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49498/0 (socket says 172.21.15.16:49498) 2023-05-20T01:26:33.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.466+0000 7f711e8d5700 1 -- 172.21.15.16:0/517298587 learned_addr learned my addr 172.21.15.16:0/517298587 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:26:33.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.466+0000 7f711e0d4700 1 --2- 172.21.15.16:0/517298587 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f711806cb90 0x7f711815ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:33.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.466+0000 7f711d8d3700 1 --2- 172.21.15.16:0/517298587 >> v2:172.21.15.132:3300/0 conn(0x7f71180b54f0 0x7f71180b29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:33.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.466+0000 7f711d0d2700 1 -- 172.21.15.16:0/517298587 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1111401415 0 0) 0x7f71180b2f20 con 0x7f711806cf60 2023-05-20T01:26:33.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.466+0000 7f711d0d2700 1 -- 172.21.15.16:0/517298587 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f71040035b0 con 0x7f711806cf60 2023-05-20T01:26:33.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.466+0000 7f711d0d2700 1 -- 172.21.15.16:0/517298587 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4087234466 0 0) 0x7f71040035b0 con 0x7f711806cf60 2023-05-20T01:26:33.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.466+0000 7f711d0d2700 1 -- 172.21.15.16:0/517298587 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f711806cb90 msgr2=0x7f711815ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:33.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.466+0000 7f711d0d2700 1 --2- 172.21.15.16:0/517298587 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f711806cb90 0x7f711815ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:33.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.466+0000 7f711d0d2700 1 -- 172.21.15.16:0/517298587 >> v2:172.21.15.132:3300/0 conn(0x7f71180b54f0 msgr2=0x7f71180b29e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:33.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.466+0000 7f711d0d2700 1 --2- 172.21.15.16:0/517298587 >> v2:172.21.15.132:3300/0 conn(0x7f71180b54f0 0x7f71180b29e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:33.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.466+0000 7f711d0d2700 1 -- 172.21.15.16:0/517298587 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71180b3480 con 0x7f711806cf60 2023-05-20T01:26:33.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.466+0000 7f711d0d2700 1 -- 172.21.15.16:0/517298587 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7f71100019f0 con 0x7f711806cf60 2023-05-20T01:26:33.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.466+0000 7f711d0d2700 1 -- 172.21.15.16:0/517298587 >> v1:172.21.15.16:6789/0 conn(0x7f711806cf60 legacy=0x7f7118070910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:33.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.466+0000 7f711d0d2700 1 --2- 172.21.15.16:0/517298587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7104003a70 0x7f7104003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:33.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.466+0000 7f711d0d2700 1 --2- 172.21.15.16:0/517298587 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7104004850 0x7f7104004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:33.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.466+0000 7f711d0d2700 1 --2- 172.21.15.16:0/517298587 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f711806cb90 0x7f7104005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:33.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.466+0000 7f711d0d2700 1 -- 172.21.15.16:0/517298587 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71180cc990 con 0x7f7104003a70 2023-05-20T01:26:33.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.466+0000 7f711d0d2700 1 -- 172.21.15.16:0/517298587 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71180b31a0 con 0x7f7104004850 2023-05-20T01:26:33.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.466+0000 7f711d0d2700 1 -- 172.21.15.16:0/517298587 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f710400ba10 con 0x7f711806cb90 2023-05-20T01:26:33.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.466+0000 7f711e0d4700 1 --2- 172.21.15.16:0/517298587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7104003a70 0x7f7104003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:33.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.467+0000 7f711d8d3700 1 --2- 172.21.15.16:0/517298587 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7104004850 0x7f7104004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:33.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.467+0000 7f711e8d5700 1 --2- 172.21.15.16:0/517298587 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f711806cb90 0x7f7104005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:33.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.467+0000 7f711e0d4700 1 -- 172.21.15.16:0/517298587 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f711806cb90 msgr2=0x7f7104005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:33.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.467+0000 7f711e0d4700 1 --2- 172.21.15.16:0/517298587 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f711806cb90 0x7f7104005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:33.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.467+0000 7f711e0d4700 1 -- 172.21.15.16:0/517298587 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7104004850 msgr2=0x7f7104004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:33.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.467+0000 7f711e0d4700 1 --2- 172.21.15.16:0/517298587 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7104004850 0x7f7104004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:33.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.467+0000 7f711e0d4700 1 -- 172.21.15.16:0/517298587 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71040035b0 con 0x7f7104003a70 2023-05-20T01:26:33.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.467+0000 7f711e8d5700 1 --2- 172.21.15.16:0/517298587 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f711806cb90 0x7f7104005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:26:33.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.467+0000 7f711e0d4700 1 --2- 172.21.15.16:0/517298587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7104003a70 0x7f7104003e80 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f7114009b60 tx=0x7f7104005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=502a8becf1e895b server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:33.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.467+0000 7f711d0d2700 1 -- 172.21.15.16:0/517298587 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f711400e6c0 con 0x7f7104003a70 2023-05-20T01:26:33.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.467+0000 7f711d0d2700 1 -- 172.21.15.16:0/517298587 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f711400e820 con 0x7f7104003a70 2023-05-20T01:26:33.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.467+0000 7f711d0d2700 1 -- 172.21.15.16:0/517298587 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f711400eb90 con 0x7f7104003a70 2023-05-20T01:26:33.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.468+0000 7f7120338700 1 -- 172.21.15.16:0/517298587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7104003a70 msgr2=0x7f7104003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:33.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.468+0000 7f7120338700 1 --2- 172.21.15.16:0/517298587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7104003a70 0x7f7104003e80 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f7114009b60 tx=0x7f7104005880 comp rx=0 tx=0).stop 2023-05-20T01:26:33.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.468+0000 7f711e8d5700 1 -- 172.21.15.16:0/517298587 reap_dead start 2023-05-20T01:26:33.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.468+0000 7f7120338700 1 -- 172.21.15.16:0/517298587 shutdown_connections 2023-05-20T01:26:33.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.468+0000 7f7120338700 1 -- 172.21.15.16:0/517298587 >> 172.21.15.16:0/517298587 conn(0x7f71180661b0 msgr2=0x7f7118066580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:33.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.468+0000 7f7120338700 1 -- 172.21.15.16:0/517298587 shutdown_connections 2023-05-20T01:26:33.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.468+0000 7f7120338700 1 -- 172.21.15.16:0/517298587 wait complete. 2023-05-20T01:26:33.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.469+0000 7f7120338700 1 Processor -- start 2023-05-20T01:26:33.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.469+0000 7f7120338700 1 -- start start 2023-05-20T01:26:33.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.470+0000 7f7120338700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7104003a70 0x7f7118140250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:33.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.470+0000 7f7120338700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f711814cb10 0x7f711814f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:33.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.470+0000 7f7120338700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71181f0260 0x7f7118153990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:33.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.470+0000 7f7120338700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f711814f0d0 con 0x7f71181f0260 2023-05-20T01:26:33.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.470+0000 7f7120338700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7118140790 con 0x7f711814cb10 2023-05-20T01:26:33.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.470+0000 7f7120338700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71181f0c30 con 0x7f7104003a70 2023-05-20T01:26:33.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.471+0000 7f711e8d5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71181f0260 0x7f7118153990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:33.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.471+0000 7f711e8d5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71181f0260 0x7f7118153990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37982/0 (socket says 172.21.15.16:37982) 2023-05-20T01:26:33.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.471+0000 7f711e8d5700 1 -- 172.21.15.16:0/4273406098 learned_addr learned my addr 172.21.15.16:0/4273406098 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:26:33.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.471+0000 7f711e0d4700 1 --2- 172.21.15.16:0/4273406098 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7104003a70 0x7f7118140250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:33.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.471+0000 7f711d8d3700 1 --2- 172.21.15.16:0/4273406098 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f711814cb10 0x7f711814f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:33.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.471+0000 7f711e8d5700 1 -- 172.21.15.16:0/4273406098 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7104003a70 msgr2=0x7f7118140250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:33.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.471+0000 7f711e8d5700 1 --2- 172.21.15.16:0/4273406098 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7104003a70 0x7f7118140250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:33.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.471+0000 7f711e8d5700 1 -- 172.21.15.16:0/4273406098 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f711814cb10 msgr2=0x7f711814f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:33.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.471+0000 7f711e8d5700 1 --2- 172.21.15.16:0/4273406098 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f711814cb10 0x7f711814f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:33.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.471+0000 7f711e8d5700 1 -- 172.21.15.16:0/4273406098 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f711815b240 con 0x7f71181f0260 2023-05-20T01:26:33.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.471+0000 7f711e8d5700 1 --2- 172.21.15.16:0/4273406098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71181f0260 0x7f7118153990 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f71180cc5a0 tx=0x7f711000e440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:33.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.471+0000 7f710affd700 1 -- 172.21.15.16:0/4273406098 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7110017070 con 0x7f71181f0260 2023-05-20T01:26:33.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.471+0000 7f710affd700 1 -- 172.21.15.16:0/4273406098 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7110001aa0 con 0x7f71181f0260 2023-05-20T01:26:33.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.471+0000 7f7120338700 1 -- 172.21.15.16:0/4273406098 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f711815af60 con 0x7f71181f0260 2023-05-20T01:26:33.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.471+0000 7f710affd700 1 -- 172.21.15.16:0/4273406098 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f711000ebb0 con 0x7f71181f0260 2023-05-20T01:26:33.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.471+0000 7f7120338700 1 -- 172.21.15.16:0/4273406098 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f711815b520 con 0x7f71181f0260 2023-05-20T01:26:33.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.472+0000 7f7120338700 1 -- 172.21.15.16:0/4273406098 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f711814f210 con 0x7f71181f0260 2023-05-20T01:26:33.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.473+0000 7f710affd700 1 -- 172.21.15.16:0/4273406098 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f711000ed10 con 0x7f71181f0260 2023-05-20T01:26:33.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.473+0000 7f710affd700 1 --2- 172.21.15.16:0/4273406098 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f70fc06ace0 0x7f70fc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:33.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.473+0000 7f710affd700 1 -- 172.21.15.16:0/4273406098 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f7110088e60 con 0x7f71181f0260 2023-05-20T01:26:33.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.473+0000 7f711e0d4700 1 --2- 172.21.15.16:0/4273406098 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f70fc06ace0 0x7f70fc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:33.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.473+0000 7f711e0d4700 1 --2- 172.21.15.16:0/4273406098 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f70fc06ace0 0x7f70fc06d190 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f71140090b0 tx=0x7f7114008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:33.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.477+0000 7f710affd700 1 -- 172.21.15.16:0/4273406098 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f711814f210 con 0x7f71181f0260 2023-05-20T01:26:33.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.679+0000 7f7120338700 1 -- 172.21.15.16:0/4273406098 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f711814f210 con 0x7f71181f0260 2023-05-20T01:26:33.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.679+0000 7f710affd700 1 -- 172.21.15.16:0/4273406098 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v108) v1 ==== 84+0+126 (secure 0 0 0) 0x7f711814f210 con 0x7f71181f0260 2023-05-20T01:26:33.683 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:26:33.684 INFO:teuthology.orchestra.run.smithi016.stdout:[ 2023-05-20T01:26:33.684 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-05-20T01:26:33.684 INFO:teuthology.orchestra.run.smithi016.stdout: "poolnum": 1, 2023-05-20T01:26:33.684 INFO:teuthology.orchestra.run.smithi016.stdout: "poolname": "rbd" 2023-05-20T01:26:33.684 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-05-20T01:26:33.684 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-05-20T01:26:33.684 INFO:teuthology.orchestra.run.smithi016.stdout: "poolnum": 2, 2023-05-20T01:26:33.685 INFO:teuthology.orchestra.run.smithi016.stdout: "poolname": ".mgr" 2023-05-20T01:26:33.685 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:26:33.685 INFO:teuthology.orchestra.run.smithi016.stdout:] 2023-05-20T01:26:33.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.681+0000 7f7120338700 1 -- 172.21.15.16:0/4273406098 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f70fc06ace0 msgr2=0x7f70fc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:33.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.681+0000 7f7120338700 1 --2- 172.21.15.16:0/4273406098 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f70fc06ace0 0x7f70fc06d190 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f71140090b0 tx=0x7f7114008030 comp rx=0 tx=0).stop 2023-05-20T01:26:33.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.681+0000 7f7120338700 1 -- 172.21.15.16:0/4273406098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71181f0260 msgr2=0x7f7118153990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:33.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.681+0000 7f7120338700 1 --2- 172.21.15.16:0/4273406098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71181f0260 0x7f7118153990 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f71180cc5a0 tx=0x7f711000e440 comp rx=0 tx=0).stop 2023-05-20T01:26:33.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.681+0000 7f7120338700 1 -- 172.21.15.16:0/4273406098 shutdown_connections 2023-05-20T01:26:33.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.681+0000 7f7120338700 1 --2- 172.21.15.16:0/4273406098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71181f0260 0x7f7118153990 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:33.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.681+0000 7f7120338700 1 --2- 172.21.15.16:0/4273406098 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f70fc06ace0 0x7f70fc06d190 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:33.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.681+0000 7f7120338700 1 --2- 172.21.15.16:0/4273406098 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7104003a70 0x7f7118140250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:33.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.681+0000 7f7120338700 1 --2- 172.21.15.16:0/4273406098 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f711814cb10 0x7f711814f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:33.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.681+0000 7f7120338700 1 -- 172.21.15.16:0/4273406098 >> 172.21.15.16:0/4273406098 conn(0x7f71180661b0 msgr2=0x7f71181521d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:33.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.681+0000 7f7120338700 1 -- 172.21.15.16:0/4273406098 shutdown_connections 2023-05-20T01:26:33.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.681+0000 7f7120338700 1 -- 172.21.15.16:0/4273406098 wait complete. 2023-05-20T01:26:33.701 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-05-20T01:26:33.701 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-05-20T01:26:33.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.825+0000 7f285b4d4700 1 Processor -- start 2023-05-20T01:26:33.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.825+0000 7f285b4d4700 1 -- start start 2023-05-20T01:26:33.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.826+0000 7f285b4d4700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f28540d0cd0 0x7f28540b6930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:33.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.826+0000 7f285b4d4700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f28540b6e70 0x7f28540cb7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:33.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.826+0000 7f285b4d4700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28540b64a0 con 0x7f28540cefe0 2023-05-20T01:26:33.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.826+0000 7f285b4d4700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28540b59c0 con 0x7f28540b6e70 2023-05-20T01:26:33.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.826+0000 7f285b4d4700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28540b2540 con 0x7f28540d0cd0 2023-05-20T01:26:33.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.826+0000 7f2859a71700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f28540b6e70 0x7f28540cb7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:33.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.827+0000 7f2858a6f700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f28540d0cd0 0x7f28540b6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:33.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.827+0000 7f2859a71700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f28540b6e70 0x7f28540cb7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.16:35012/0 (socket says 172.21.15.16:35012) 2023-05-20T01:26:33.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.827+0000 7f2859270700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f28540cefe0 0x7f28540b1e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.16:59228/0 (socket says 172.21.15.16:59228) 2023-05-20T01:26:33.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.827+0000 7f2859a71700 1 -- 172.21.15.16:0/2423727920 learned_addr learned my addr 172.21.15.16:0/2423727920 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:26:33.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.827+0000 7f2858a6f700 1 -- 172.21.15.16:0/2423727920 >> v2:172.21.15.132:3300/0 conn(0x7f28540b6e70 msgr2=0x7f28540cb7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:33.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.827+0000 7f2858a6f700 1 --2- 172.21.15.16:0/2423727920 >> v2:172.21.15.132:3300/0 conn(0x7f28540b6e70 0x7f28540cb7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:33.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.827+0000 7f2858a6f700 1 -- 172.21.15.16:0/2423727920 >> v1:172.21.15.132:6789/0 conn(0x7f28540cefe0 legacy=0x7f28540b1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:33.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.827+0000 7f2858a6f700 1 -- 172.21.15.16:0/2423727920 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28540b5bb0 con 0x7f28540d0cd0 2023-05-20T01:26:33.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.827+0000 7f2859a71700 1 --2- 172.21.15.16:0/2423727920 >> v2:172.21.15.132:3300/0 conn(0x7f28540b6e70 0x7f28540cb7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-05-20T01:26:33.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.827+0000 7f2858a6f700 1 --2- 172.21.15.16:0/2423727920 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f28540d0cd0 0x7f28540b6930 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2850009840 tx=0x7f2850009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49b78ad43b19ef7e server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:33.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.827+0000 7f284bfff700 1 -- 172.21.15.16:0/2423727920 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2850011830 con 0x7f28540d0cd0 2023-05-20T01:26:33.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.827+0000 7f284bfff700 1 -- 172.21.15.16:0/2423727920 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2850011990 con 0x7f28540d0cd0 2023-05-20T01:26:33.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.827+0000 7f284bfff700 1 -- 172.21.15.16:0/2423727920 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2850011d20 con 0x7f28540d0cd0 2023-05-20T01:26:33.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.828+0000 7f285b4d4700 1 -- 172.21.15.16:0/2423727920 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f28540d0cd0 msgr2=0x7f28540b6930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:33.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.828+0000 7f285b4d4700 1 --2- 172.21.15.16:0/2423727920 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f28540d0cd0 0x7f28540b6930 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2850009840 tx=0x7f2850009b50 comp rx=0 tx=0).stop 2023-05-20T01:26:33.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.828+0000 7f285b4d4700 1 -- 172.21.15.16:0/2423727920 shutdown_connections 2023-05-20T01:26:33.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.828+0000 7f285b4d4700 1 --2- 172.21.15.16:0/2423727920 >> v2:172.21.15.132:3300/0 conn(0x7f28540b6e70 0x7f28540cb7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:33.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.828+0000 7f285b4d4700 1 --2- 172.21.15.16:0/2423727920 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f28540d0cd0 0x7f28540b6930 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:33.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.828+0000 7f285b4d4700 1 -- 172.21.15.16:0/2423727920 >> 172.21.15.16:0/2423727920 conn(0x7f285409a470 msgr2=0x7f28540d11a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:33.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.828+0000 7f285b4d4700 1 -- 172.21.15.16:0/2423727920 shutdown_connections 2023-05-20T01:26:33.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.828+0000 7f285b4d4700 1 -- 172.21.15.16:0/2423727920 wait complete. 2023-05-20T01:26:33.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.829+0000 7f285b4d4700 1 Processor -- start 2023-05-20T01:26:33.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.829+0000 7f285b4d4700 1 -- start start 2023-05-20T01:26:33.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.830+0000 7f285b4d4700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f28540b6e70 0x7f285414c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:33.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.830+0000 7f285b4d4700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f28540cefe0 0x7f28541533e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:33.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.830+0000 7f285b4d4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28540d0cd0 0x7f285414d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:33.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.830+0000 7f285b4d4700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2854153920 con 0x7f28540d0cd0 2023-05-20T01:26:33.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.830+0000 7f285b4d4700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28541511b0 con 0x7f28540cefe0 2023-05-20T01:26:33.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.830+0000 7f285b4d4700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28540c6db0 con 0x7f28540b6e70 2023-05-20T01:26:33.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.830+0000 7f2859a71700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28540d0cd0 0x7f285414d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:33.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.830+0000 7f2859a71700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28540d0cd0 0x7f285414d030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37996/0 (socket says 172.21.15.16:37996) 2023-05-20T01:26:33.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.830+0000 7f2859a71700 1 -- 172.21.15.16:0/3971708128 learned_addr learned my addr 172.21.15.16:0/3971708128 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:26:33.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.830+0000 7f2859270700 1 --2- 172.21.15.16:0/3971708128 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f28540b6e70 0x7f285414c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:33.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.830+0000 7f2858a6f700 1 --2- 172.21.15.16:0/3971708128 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f28540cefe0 0x7f28541533e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:33.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.830+0000 7f2859a71700 1 -- 172.21.15.16:0/3971708128 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f28540b6e70 msgr2=0x7f285414c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:33.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.830+0000 7f2859a71700 1 --2- 172.21.15.16:0/3971708128 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f28540b6e70 0x7f285414c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:33.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.830+0000 7f2859a71700 1 -- 172.21.15.16:0/3971708128 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f28540cefe0 msgr2=0x7f28541533e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:33.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.830+0000 7f2859a71700 1 --2- 172.21.15.16:0/3971708128 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f28540cefe0 0x7f28541533e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:33.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.830+0000 7f2859a71700 1 -- 172.21.15.16:0/3971708128 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f284c009b30 con 0x7f28540d0cd0 2023-05-20T01:26:33.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.830+0000 7f2858a6f700 1 --2- 172.21.15.16:0/3971708128 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f28540cefe0 0x7f28541533e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:26:33.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.831+0000 7f2859a71700 1 --2- 172.21.15.16:0/3971708128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28540d0cd0 0x7f285414d030 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f284c00efc0 tx=0x7f284c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:33.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.831+0000 7f2849ffb700 1 -- 172.21.15.16:0/3971708128 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f284c018070 con 0x7f28540d0cd0 2023-05-20T01:26:33.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.831+0000 7f2849ffb700 1 -- 172.21.15.16:0/3971708128 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f284c00f040 con 0x7f28540d0cd0 2023-05-20T01:26:33.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.831+0000 7f285b4d4700 1 -- 172.21.15.16:0/3971708128 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28500094a0 con 0x7f28540d0cd0 2023-05-20T01:26:33.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.831+0000 7f2849ffb700 1 -- 172.21.15.16:0/3971708128 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f284c003c40 con 0x7f28540d0cd0 2023-05-20T01:26:33.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.831+0000 7f285b4d4700 1 -- 172.21.15.16:0/3971708128 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28540b6130 con 0x7f28540d0cd0 2023-05-20T01:26:33.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.832+0000 7f285b4d4700 1 -- 172.21.15.16:0/3971708128 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28540b5e50 con 0x7f28540d0cd0 2023-05-20T01:26:33.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.832+0000 7f2849ffb700 1 -- 172.21.15.16:0/3971708128 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f284c00c9e0 con 0x7f28540d0cd0 2023-05-20T01:26:33.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.833+0000 7f2849ffb700 1 --2- 172.21.15.16:0/3971708128 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f283806ace0 0x7f283806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:33.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.833+0000 7f2849ffb700 1 -- 172.21.15.16:0/3971708128 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f284c089550 con 0x7f28540d0cd0 2023-05-20T01:26:33.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.833+0000 7f2859270700 1 --2- 172.21.15.16:0/3971708128 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f283806ace0 0x7f283806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:33.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.833+0000 7f2859270700 1 --2- 172.21.15.16:0/3971708128 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f283806ace0 0x7f283806d190 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f2844000f60 tx=0x7f2844007d30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:33.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:33.837+0000 7f2849ffb700 1 -- 172.21.15.16:0/3971708128 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f284c054800 con 0x7f28540d0cd0 2023-05-20T01:26:34.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:34.035+0000 7f285b4d4700 1 -- 172.21.15.16:0/3971708128 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f28540cf3b0 con 0x7f28540d0cd0 2023-05-20T01:26:34.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:34.636+0000 7f2849ffb700 1 -- 172.21.15.16:0/3971708128 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v109) v1 ==== 138+0+0 (secure 0 0 0) 0x7f284c057e20 con 0x7f28540d0cd0 2023-05-20T01:26:34.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:34.641+0000 7f285b4d4700 1 -- 172.21.15.16:0/3971708128 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f283806ace0 msgr2=0x7f283806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:34.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:34.641+0000 7f285b4d4700 1 --2- 172.21.15.16:0/3971708128 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f283806ace0 0x7f283806d190 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f2844000f60 tx=0x7f2844007d30 comp rx=0 tx=0).stop 2023-05-20T01:26:34.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:34.641+0000 7f285b4d4700 1 -- 172.21.15.16:0/3971708128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28540d0cd0 msgr2=0x7f285414d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:34.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:34.641+0000 7f285b4d4700 1 --2- 172.21.15.16:0/3971708128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28540d0cd0 0x7f285414d030 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f284c00efc0 tx=0x7f284c009f90 comp rx=0 tx=0).stop 2023-05-20T01:26:34.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:34.642+0000 7f285b4d4700 1 -- 172.21.15.16:0/3971708128 shutdown_connections 2023-05-20T01:26:34.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:34.642+0000 7f285b4d4700 1 --2- 172.21.15.16:0/3971708128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28540d0cd0 0x7f285414d030 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:34.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:34.642+0000 7f285b4d4700 1 --2- 172.21.15.16:0/3971708128 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f283806ace0 0x7f283806d190 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:34.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:34.642+0000 7f285b4d4700 1 --2- 172.21.15.16:0/3971708128 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f28540b6e70 0x7f285414c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:34.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:34.642+0000 7f285b4d4700 1 --2- 172.21.15.16:0/3971708128 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f28540cefe0 0x7f28541533e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:34.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:34.642+0000 7f285b4d4700 1 -- 172.21.15.16:0/3971708128 >> 172.21.15.16:0/3971708128 conn(0x7f285409a470 msgr2=0x7f285406fc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:34.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:34.642+0000 7f285b4d4700 1 -- 172.21.15.16:0/3971708128 shutdown_connections 2023-05-20T01:26:34.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:34.643+0000 7f285b4d4700 1 -- 172.21.15.16:0/3971708128 wait complete. 2023-05-20T01:26:34.648 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' created 2023-05-20T01:26:34.662 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-05-20T01:26:34.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:34.798+0000 7fe345e0a700 1 Processor -- start 2023-05-20T01:26:34.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:34.798+0000 7fe345e0a700 1 -- start start 2023-05-20T01:26:34.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:34.799+0000 7fe345e0a700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe340159b50 0x7fe34006e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:34.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:34.799+0000 7fe345e0a700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3400ccd30 con 0x7fe3400cd1c0 2023-05-20T01:26:34.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:34.799+0000 7fe345e0a700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3400ce990 con 0x7fe3400717c0 2023-05-20T01:26:34.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:34.799+0000 7fe345e0a700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3400cc6e0 con 0x7fe340159b50 2023-05-20T01:26:34.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:34.799+0000 7fe33f7fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe340159b50 0x7fe34006e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:34.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:34.799+0000 7fe33effd700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fe3400cd1c0 0x7fe3400710b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49528/0 (socket says 172.21.15.16:49528) 2023-05-20T01:26:34.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:34.799+0000 7fe33effd700 1 -- 172.21.15.16:0/1137138535 learned_addr learned my addr 172.21.15.16:0/1137138535 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:26:34.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:34.799+0000 7fe33f7fe700 1 -- 172.21.15.16:0/1137138535 >> v1:172.21.15.132:6789/0 conn(0x7fe3400717c0 legacy=0x7fe3400ca870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:34.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:34.799+0000 7fe33f7fe700 1 -- 172.21.15.16:0/1137138535 >> v1:172.21.15.16:6789/0 conn(0x7fe3400cd1c0 legacy=0x7fe3400710b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:34.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:34.799+0000 7fe33e7fc700 1 -- 172.21.15.16:0/1137138535 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3140480167 0 0) 0x7fe3400ccd30 con 0x7fe3400cd1c0 2023-05-20T01:26:34.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:34.799+0000 7fe33f7fe700 1 -- 172.21.15.16:0/1137138535 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3400cc9c0 con 0x7fe340159b50 2023-05-20T01:26:34.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:34.799+0000 7fe33f7fe700 1 --2- 172.21.15.16:0/1137138535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe340159b50 0x7fe34006e0a0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7fe330009820 tx=0x7fe330009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef7fd81e268f9a79 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:34.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:34.799+0000 7fe33e7fc700 1 -- 172.21.15.16:0/1137138535 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe330005e30 con 0x7fe340159b50 2023-05-20T01:26:34.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:34.800+0000 7fe33e7fc700 1 -- 172.21.15.16:0/1137138535 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe330006030 con 0x7fe340159b50 2023-05-20T01:26:34.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:34.800+0000 7fe33e7fc700 1 -- 172.21.15.16:0/1137138535 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3300063e0 con 0x7fe340159b50 2023-05-20T01:26:34.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:34.800+0000 7fe345e0a700 1 -- 172.21.15.16:0/1137138535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe340159b50 msgr2=0x7fe34006e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:34.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:34.800+0000 7fe345e0a700 1 --2- 172.21.15.16:0/1137138535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe340159b50 0x7fe34006e0a0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7fe330009820 tx=0x7fe330009b30 comp rx=0 tx=0).stop 2023-05-20T01:26:34.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:34.800+0000 7fe345e0a700 1 -- 172.21.15.16:0/1137138535 shutdown_connections 2023-05-20T01:26:34.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:34.800+0000 7fe345e0a700 1 --2- 172.21.15.16:0/1137138535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe340159b50 0x7fe34006e0a0 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:34.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:34.800+0000 7fe345e0a700 1 -- 172.21.15.16:0/1137138535 >> 172.21.15.16:0/1137138535 conn(0x7fe3400cffd0 msgr2=0x7fe3400d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:34.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:34.800+0000 7fe345e0a700 1 -- 172.21.15.16:0/1137138535 shutdown_connections 2023-05-20T01:26:34.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:34.800+0000 7fe345e0a700 1 -- 172.21.15.16:0/1137138535 wait complete. 2023-05-20T01:26:34.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:34.801+0000 7fe345e0a700 1 Processor -- start 2023-05-20T01:26:34.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:34.802+0000 7fe345e0a700 1 -- start start 2023-05-20T01:26:34.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:34.802+0000 7fe345e0a700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe3400717c0 0x7fe340154cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:34.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:34.802+0000 7fe345e0a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3400cd1c0 0x7fe34014e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:34.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:34.802+0000 7fe345e0a700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe340159b50 0x7fe3400c81e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:34.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:34.802+0000 7fe345e0a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3400c97b0 con 0x7fe3400cd1c0 2023-05-20T01:26:34.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:34.802+0000 7fe345e0a700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3400c8b30 con 0x7fe340159b50 2023-05-20T01:26:34.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:34.802+0000 7fe345e0a700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3400c8e10 con 0x7fe3400717c0 2023-05-20T01:26:34.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:34.802+0000 7fe33effd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3400cd1c0 0x7fe34014e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:34.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:34.802+0000 7fe33effd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3400cd1c0 0x7fe34014e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38008/0 (socket says 172.21.15.16:38008) 2023-05-20T01:26:34.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:34.802+0000 7fe33effd700 1 -- 172.21.15.16:0/2928003229 learned_addr learned my addr 172.21.15.16:0/2928003229 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:26:34.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:34.803+0000 7fe33ffff700 1 --2- 172.21.15.16:0/2928003229 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe340159b50 0x7fe3400c81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:34.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:34.803+0000 7fe33f7fe700 1 --2- 172.21.15.16:0/2928003229 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe3400717c0 0x7fe340154cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:34.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:34.803+0000 7fe33effd700 1 -- 172.21.15.16:0/2928003229 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe3400717c0 msgr2=0x7fe340154cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:34.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:34.803+0000 7fe33effd700 1 --2- 172.21.15.16:0/2928003229 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe3400717c0 0x7fe340154cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:34.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:34.803+0000 7fe33effd700 1 -- 172.21.15.16:0/2928003229 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe340159b50 msgr2=0x7fe3400c81e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:34.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:34.803+0000 7fe33effd700 1 --2- 172.21.15.16:0/2928003229 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe340159b50 0x7fe3400c81e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:34.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:34.803+0000 7fe33effd700 1 -- 172.21.15.16:0/2928003229 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe330009430 con 0x7fe3400cd1c0 2023-05-20T01:26:34.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:34.803+0000 7fe33effd700 1 --2- 172.21.15.16:0/2928003229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3400cd1c0 0x7fe34014e6e0 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7fe33400c520 tx=0x7fe33400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:34.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:34.803+0000 7fe327fff700 1 -- 172.21.15.16:0/2928003229 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe334004c80 con 0x7fe3400cd1c0 2023-05-20T01:26:34.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:34.803+0000 7fe327fff700 1 -- 172.21.15.16:0/2928003229 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe334004e00 con 0x7fe3400cd1c0 2023-05-20T01:26:34.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:34.803+0000 7fe345e0a700 1 -- 172.21.15.16:0/2928003229 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe340071e80 con 0x7fe3400cd1c0 2023-05-20T01:26:34.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:34.803+0000 7fe327fff700 1 -- 172.21.15.16:0/2928003229 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe334007310 con 0x7fe3400cd1c0 2023-05-20T01:26:34.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:34.803+0000 7fe345e0a700 1 -- 172.21.15.16:0/2928003229 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe34006e5e0 con 0x7fe3400cd1c0 2023-05-20T01:26:34.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:34.804+0000 7fe345e0a700 1 -- 172.21.15.16:0/2928003229 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3400cec70 con 0x7fe3400cd1c0 2023-05-20T01:26:34.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:34.805+0000 7fe327fff700 1 -- 172.21.15.16:0/2928003229 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe334007470 con 0x7fe3400cd1c0 2023-05-20T01:26:34.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:34.805+0000 7fe327fff700 1 --2- 172.21.15.16:0/2928003229 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe32c06ace0 0x7fe32c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:34.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:34.805+0000 7fe327fff700 1 -- 172.21.15.16:0/2928003229 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 5900+0+0 (secure 0 0 0) 0x7fe334089a30 con 0x7fe3400cd1c0 2023-05-20T01:26:34.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:34.805+0000 7fe33f7fe700 1 --2- 172.21.15.16:0/2928003229 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe32c06ace0 0x7fe32c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:34.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:34.805+0000 7fe33f7fe700 1 --2- 172.21.15.16:0/2928003229 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe32c06ace0 0x7fe32c06d190 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fe330000c00 tx=0x7fe330008070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:34.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:34.810+0000 7fe327fff700 1 -- 172.21.15.16:0/2928003229 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fe334054260 con 0x7fe3400cd1c0 2023-05-20T01:26:35.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:35.008+0000 7fe345e0a700 1 -- 172.21.15.16:0/2928003229 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fe3400747b0 con 0x7fe3400cd1c0 2023-05-20T01:26:35.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:35.642+0000 7fe327fff700 1 -- 172.21.15.16:0/2928003229 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v110) v1 ==== 170+0+0 (secure 0 0 0) 0x7fe334057880 con 0x7fe3400cd1c0 2023-05-20T01:26:35.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:35.646+0000 7fe345e0a700 1 -- 172.21.15.16:0/2928003229 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe32c06ace0 msgr2=0x7fe32c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:35.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:35.646+0000 7fe345e0a700 1 --2- 172.21.15.16:0/2928003229 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe32c06ace0 0x7fe32c06d190 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fe330000c00 tx=0x7fe330008070 comp rx=0 tx=0).stop 2023-05-20T01:26:35.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:35.646+0000 7fe345e0a700 1 -- 172.21.15.16:0/2928003229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3400cd1c0 msgr2=0x7fe34014e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:35.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:35.646+0000 7fe345e0a700 1 --2- 172.21.15.16:0/2928003229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3400cd1c0 0x7fe34014e6e0 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7fe33400c520 tx=0x7fe33400c8e0 comp rx=0 tx=0).stop 2023-05-20T01:26:35.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:35.647+0000 7fe345e0a700 1 -- 172.21.15.16:0/2928003229 shutdown_connections 2023-05-20T01:26:35.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:35.647+0000 7fe345e0a700 1 --2- 172.21.15.16:0/2928003229 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe340159b50 0x7fe3400c81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:35.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:35.647+0000 7fe345e0a700 1 --2- 172.21.15.16:0/2928003229 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe32c06ace0 0x7fe32c06d190 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:35.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:35.647+0000 7fe345e0a700 1 --2- 172.21.15.16:0/2928003229 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe3400717c0 0x7fe340154cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:35.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:35.647+0000 7fe345e0a700 1 --2- 172.21.15.16:0/2928003229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3400cd1c0 0x7fe34014e6e0 secure :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7fe33400c520 tx=0x7fe33400c8e0 comp rx=0 tx=0).stop 2023-05-20T01:26:35.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:35.647+0000 7fe345e0a700 1 -- 172.21.15.16:0/2928003229 >> 172.21.15.16:0/2928003229 conn(0x7fe3400cffd0 msgr2=0x7fe3400cbb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:35.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:35.650+0000 7fe345e0a700 1 -- 172.21.15.16:0/2928003229 shutdown_connections 2023-05-20T01:26:35.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:35.651+0000 7fe345e0a700 1 -- 172.21.15.16:0/2928003229 wait complete. 2023-05-20T01:26:35.666 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' created 2023-05-20T01:26:35.680 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-05-20T01:26:35.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:35.808+0000 7fad8ad72700 1 Processor -- start 2023-05-20T01:26:35.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:35.809+0000 7fad8ad72700 1 -- start start 2023-05-20T01:26:35.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:35.809+0000 7fad8ad72700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fad84146710 0x7fad8406b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:35.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:35.809+0000 7fad8ad72700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fad8406bb60 0x7fad840cdad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:35.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:35.809+0000 7fad8ad72700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fad840ce010 0x7fad84160730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:35.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:35.809+0000 7fad8ad72700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad8406bfd0 con 0x7fad840ce010 2023-05-20T01:26:35.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:35.809+0000 7fad8ad72700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad840cb6c0 con 0x7fad8406bb60 2023-05-20T01:26:35.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:35.809+0000 7fad8ad72700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad840ce840 con 0x7fad84146710 2023-05-20T01:26:35.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:35.810+0000 7fad8930f700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fad840ce010 0x7fad84160730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:35.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:35.810+0000 7fad8930f700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fad840ce010 0x7fad84160730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38018/0 (socket says 172.21.15.16:38018) 2023-05-20T01:26:35.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:35.810+0000 7fad8930f700 1 -- 172.21.15.16:0/412375700 learned_addr learned my addr 172.21.15.16:0/412375700 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:26:35.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:35.810+0000 7fad83fff700 1 --2- 172.21.15.16:0/412375700 >> v2:172.21.15.132:3300/0 conn(0x7fad8406bb60 0x7fad840cdad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:35.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:35.810+0000 7fad88b0e700 1 --2- 172.21.15.16:0/412375700 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fad84146710 0x7fad8406b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:35.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:35.810+0000 7fad8930f700 1 -- 172.21.15.16:0/412375700 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fad84146710 msgr2=0x7fad8406b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:35.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:35.810+0000 7fad8930f700 1 --2- 172.21.15.16:0/412375700 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fad84146710 0x7fad8406b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:35.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:35.810+0000 7fad8930f700 1 -- 172.21.15.16:0/412375700 >> v2:172.21.15.132:3300/0 conn(0x7fad8406bb60 msgr2=0x7fad840cdad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:35.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:35.810+0000 7fad8930f700 1 --2- 172.21.15.16:0/412375700 >> v2:172.21.15.132:3300/0 conn(0x7fad8406bb60 0x7fad840cdad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:35.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:35.810+0000 7fad8930f700 1 -- 172.21.15.16:0/412375700 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad840cb910 con 0x7fad840ce010 2023-05-20T01:26:35.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:35.810+0000 7fad88b0e700 1 --2- 172.21.15.16:0/412375700 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fad84146710 0x7fad8406b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:26:35.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:35.810+0000 7fad8930f700 1 --2- 172.21.15.16:0/412375700 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad840ce010 0x7fad84160730 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7fad840b3c60 tx=0x7fad7c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38f305895215b3f6 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:35.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:35.810+0000 7fad837fe700 1 -- 172.21.15.16:0/412375700 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad7c00e070 con 0x7fad840ce010 2023-05-20T01:26:35.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:35.810+0000 7fad837fe700 1 -- 172.21.15.16:0/412375700 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad7c00b910 con 0x7fad840ce010 2023-05-20T01:26:35.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:35.810+0000 7fad837fe700 1 -- 172.21.15.16:0/412375700 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad7c00bcc0 con 0x7fad840ce010 2023-05-20T01:26:35.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:35.811+0000 7fad8ad72700 1 -- 172.21.15.16:0/412375700 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad840ce010 msgr2=0x7fad84160730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:35.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:35.811+0000 7fad8ad72700 1 --2- 172.21.15.16:0/412375700 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad840ce010 0x7fad84160730 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7fad840b3c60 tx=0x7fad7c00b330 comp rx=0 tx=0).stop 2023-05-20T01:26:35.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:35.811+0000 7fad8ad72700 1 -- 172.21.15.16:0/412375700 shutdown_connections 2023-05-20T01:26:35.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:35.811+0000 7fad8ad72700 1 --2- 172.21.15.16:0/412375700 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad840ce010 0x7fad84160730 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:35.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:35.811+0000 7fad8ad72700 1 --2- 172.21.15.16:0/412375700 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fad84146710 0x7fad8406b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:35.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:35.811+0000 7fad8ad72700 1 --2- 172.21.15.16:0/412375700 >> v2:172.21.15.132:3300/0 conn(0x7fad8406bb60 0x7fad840cdad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:35.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:35.811+0000 7fad8ad72700 1 -- 172.21.15.16:0/412375700 >> 172.21.15.16:0/412375700 conn(0x7fad8406ae30 msgr2=0x7fad8409a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:35.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:35.811+0000 7fad8ad72700 1 -- 172.21.15.16:0/412375700 shutdown_connections 2023-05-20T01:26:35.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:35.811+0000 7fad8ad72700 1 -- 172.21.15.16:0/412375700 wait complete. 2023-05-20T01:26:35.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:35.812+0000 7fad8ad72700 1 Processor -- start 2023-05-20T01:26:35.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:35.812+0000 7fad8ad72700 1 -- start start 2023-05-20T01:26:35.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:35.813+0000 7fad8ad72700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad8406bb60 0x7fad841f6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:35.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:35.813+0000 7fad8ad72700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fad840ce010 0x7fad840b09d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:35.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:35.813+0000 7fad8ad72700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fad84146710 0x7fad840b0f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:35.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:35.813+0000 7fad8ad72700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad841f5ef0 con 0x7fad8406bb60 2023-05-20T01:26:35.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:35.813+0000 7fad8ad72700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad841f2b70 con 0x7fad84146710 2023-05-20T01:26:35.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:35.813+0000 7fad8ad72700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad841f2580 con 0x7fad840ce010 2023-05-20T01:26:35.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:35.813+0000 7fad88b0e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad8406bb60 0x7fad841f6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:35.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:35.813+0000 7fad88b0e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad8406bb60 0x7fad841f6260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38020/0 (socket says 172.21.15.16:38020) 2023-05-20T01:26:35.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:35.813+0000 7fad88b0e700 1 -- 172.21.15.16:0/327902971 learned_addr learned my addr 172.21.15.16:0/327902971 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:26:35.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:35.813+0000 7fad8930f700 1 --2- 172.21.15.16:0/327902971 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fad84146710 0x7fad840b0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:35.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:35.813+0000 7fad88b0e700 1 -- 172.21.15.16:0/327902971 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fad840ce010 msgr2=0x7fad840b09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:35.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:35.813+0000 7fad83fff700 1 --2- 172.21.15.16:0/327902971 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fad840ce010 0x7fad840b09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:35.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:35.813+0000 7fad88b0e700 1 --2- 172.21.15.16:0/327902971 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fad840ce010 0x7fad840b09d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:35.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:35.813+0000 7fad88b0e700 1 -- 172.21.15.16:0/327902971 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fad84146710 msgr2=0x7fad840b0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:35.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:35.813+0000 7fad88b0e700 1 --2- 172.21.15.16:0/327902971 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fad84146710 0x7fad840b0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:35.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:35.813+0000 7fad88b0e700 1 -- 172.21.15.16:0/327902971 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad7c009b30 con 0x7fad8406bb60 2023-05-20T01:26:35.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:35.813+0000 7fad8930f700 1 --2- 172.21.15.16:0/327902971 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fad84146710 0x7fad840b0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:26:35.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:35.814+0000 7fad88b0e700 1 --2- 172.21.15.16:0/327902971 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad8406bb60 0x7fad841f6260 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7fad7800e810 tx=0x7fad7800ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:35.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:35.814+0000 7fad817fa700 1 -- 172.21.15.16:0/327902971 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad7800c020 con 0x7fad8406bb60 2023-05-20T01:26:35.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:35.814+0000 7fad817fa700 1 -- 172.21.15.16:0/327902971 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad78004250 con 0x7fad8406bb60 2023-05-20T01:26:35.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:35.814+0000 7fad817fa700 1 -- 172.21.15.16:0/327902971 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad780036a0 con 0x7fad8406bb60 2023-05-20T01:26:35.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:35.814+0000 7fad8ad72700 1 -- 172.21.15.16:0/327902971 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad840cbed0 con 0x7fad8406bb60 2023-05-20T01:26:35.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:35.814+0000 7fad8ad72700 1 -- 172.21.15.16:0/327902971 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad840cbbf0 con 0x7fad8406bb60 2023-05-20T01:26:35.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:35.815+0000 7fad8ad72700 1 -- 172.21.15.16:0/327902971 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad840ce490 con 0x7fad8406bb60 2023-05-20T01:26:35.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:35.815+0000 7fad817fa700 1 -- 172.21.15.16:0/327902971 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fad78003800 con 0x7fad8406bb60 2023-05-20T01:26:35.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:35.816+0000 7fad817fa700 1 --2- 172.21.15.16:0/327902971 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fad6806ace0 0x7fad6806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:35.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:35.816+0000 7fad817fa700 1 -- 172.21.15.16:0/327902971 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6282+0+0 (secure 0 0 0) 0x7fad78088f70 con 0x7fad8406bb60 2023-05-20T01:26:35.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:35.816+0000 7fad83fff700 1 --2- 172.21.15.16:0/327902971 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fad6806ace0 0x7fad6806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:35.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:35.816+0000 7fad83fff700 1 --2- 172.21.15.16:0/327902971 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fad6806ace0 0x7fad6806d190 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fad70009990 tx=0x7fad70008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:35.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:35.820+0000 7fad817fa700 1 -- 172.21.15.16:0/327902971 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fad78053fd0 con 0x7fad8406bb60 2023-05-20T01:26:36.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.065+0000 7fad8ad72700 1 -- 172.21.15.16:0/327902971 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fad840747b0 con 0x7fad8406bb60 2023-05-20T01:26:36.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.071+0000 7fad817fa700 1 -- 172.21.15.16:0/327902971 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 15 and data pool 16 v74) v1 ==== 174+0+0 (secure 0 0 0) 0x7fad780575f0 con 0x7fad8406bb60 2023-05-20T01:26:36.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.076+0000 7fad8ad72700 1 -- 172.21.15.16:0/327902971 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fad6806ace0 msgr2=0x7fad6806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:36.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.076+0000 7fad8ad72700 1 --2- 172.21.15.16:0/327902971 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fad6806ace0 0x7fad6806d190 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fad70009990 tx=0x7fad70008040 comp rx=0 tx=0).stop 2023-05-20T01:26:36.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.076+0000 7fad8ad72700 1 -- 172.21.15.16:0/327902971 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad8406bb60 msgr2=0x7fad841f6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:36.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.076+0000 7fad8ad72700 1 --2- 172.21.15.16:0/327902971 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad8406bb60 0x7fad841f6260 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7fad7800e810 tx=0x7fad7800ebd0 comp rx=0 tx=0).stop 2023-05-20T01:26:36.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.076+0000 7fad8ad72700 1 -- 172.21.15.16:0/327902971 shutdown_connections 2023-05-20T01:26:36.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.076+0000 7fad8ad72700 1 --2- 172.21.15.16:0/327902971 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fad84146710 0x7fad840b0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:36.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.076+0000 7fad8ad72700 1 --2- 172.21.15.16:0/327902971 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad8406bb60 0x7fad841f6260 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:36.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.076+0000 7fad8ad72700 1 --2- 172.21.15.16:0/327902971 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fad6806ace0 0x7fad6806d190 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:36.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.076+0000 7fad8ad72700 1 --2- 172.21.15.16:0/327902971 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fad840ce010 0x7fad840b09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:36.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.076+0000 7fad8ad72700 1 -- 172.21.15.16:0/327902971 >> 172.21.15.16:0/327902971 conn(0x7fad8406ae30 msgr2=0x7fad8406f2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:36.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.078+0000 7fad8ad72700 1 -- 172.21.15.16:0/327902971 shutdown_connections 2023-05-20T01:26:36.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.080+0000 7fad8ad72700 1 -- 172.21.15.16:0/327902971 wait complete. 2023-05-20T01:26:36.093 INFO:teuthology.orchestra.run.smithi016.stderr:new fs with metadata pool 15 and data pool 16 2023-05-20T01:26:36.114 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-20T01:26:36.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.235+0000 7fbb314e1700 1 Processor -- start 2023-05-20T01:26:36.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.235+0000 7fbb314e1700 1 -- start start 2023-05-20T01:26:36.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.235+0000 7fbb314e1700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fbb2c0cfc50 0x7fbb2c0d2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:36.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.236+0000 7fbb314e1700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbb2c0cd970 0x7fbb2c0cddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:36.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.236+0000 7fbb314e1700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb2c0b3b50 con 0x7fbb2c0b4750 2023-05-20T01:26:36.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.236+0000 7fbb314e1700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb2c0cfad0 con 0x7fbb2c0cfc50 2023-05-20T01:26:36.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.236+0000 7fbb314e1700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb2c0d3df0 con 0x7fbb2c0cd970 2023-05-20T01:26:36.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.236+0000 7fbb2a7fc700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fbb2c0b4750 0x7fbb2c0d0020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49570/0 (socket says 172.21.15.16:49570) 2023-05-20T01:26:36.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.236+0000 7fbb2a7fc700 1 -- 172.21.15.16:0/4007924608 learned_addr learned my addr 172.21.15.16:0/4007924608 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:26:36.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.236+0000 7fbb29ffb700 1 -- 172.21.15.16:0/4007924608 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3748682314 0 0) 0x7fbb2c0b3b50 con 0x7fbb2c0b4750 2023-05-20T01:26:36.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.236+0000 7fbb29ffb700 1 -- 172.21.15.16:0/4007924608 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb140035b0 con 0x7fbb2c0b4750 2023-05-20T01:26:36.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.236+0000 7fbb2b7fe700 1 --2- 172.21.15.16:0/4007924608 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbb2c0cd970 0x7fbb2c0cddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:36.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.236+0000 7fbb2affd700 1 --2- 172.21.15.16:0/4007924608 >> v2:172.21.15.132:3300/0 conn(0x7fbb2c0cfc50 0x7fbb2c0d2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:36.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.236+0000 7fbb29ffb700 1 -- 172.21.15.16:0/4007924608 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2900916201 0 0) 0x7fbb140035b0 con 0x7fbb2c0b4750 2023-05-20T01:26:36.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.236+0000 7fbb29ffb700 1 -- 172.21.15.16:0/4007924608 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbb2c0cd970 msgr2=0x7fbb2c0cddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:36.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.236+0000 7fbb29ffb700 1 --2- 172.21.15.16:0/4007924608 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbb2c0cd970 0x7fbb2c0cddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:36.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.236+0000 7fbb29ffb700 1 -- 172.21.15.16:0/4007924608 >> v2:172.21.15.132:3300/0 conn(0x7fbb2c0cfc50 msgr2=0x7fbb2c0d2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:36.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.236+0000 7fbb29ffb700 1 --2- 172.21.15.16:0/4007924608 >> v2:172.21.15.132:3300/0 conn(0x7fbb2c0cfc50 0x7fbb2c0d2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:36.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.236+0000 7fbb29ffb700 1 -- 172.21.15.16:0/4007924608 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb2c0d0b90 con 0x7fbb2c0b4750 2023-05-20T01:26:36.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.236+0000 7fbb29ffb700 1 -- 172.21.15.16:0/4007924608 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7fbb20001670 con 0x7fbb2c0b4750 2023-05-20T01:26:36.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.236+0000 7fbb29ffb700 1 -- 172.21.15.16:0/4007924608 >> v1:172.21.15.16:6789/0 conn(0x7fbb2c0b4750 legacy=0x7fbb2c0d0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:36.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.236+0000 7fbb29ffb700 1 --2- 172.21.15.16:0/4007924608 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbb14003a70 0x7fbb14003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:36.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.236+0000 7fbb29ffb700 1 --2- 172.21.15.16:0/4007924608 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb14004850 0x7fbb14004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:36.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.236+0000 7fbb29ffb700 1 --2- 172.21.15.16:0/4007924608 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbb2c0cd970 0x7fbb14005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:36.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.236+0000 7fbb29ffb700 1 -- 172.21.15.16:0/4007924608 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb2c0cfad0 con 0x7fbb14004850 2023-05-20T01:26:36.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.236+0000 7fbb29ffb700 1 -- 172.21.15.16:0/4007924608 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb2c0d3df0 con 0x7fbb14003a70 2023-05-20T01:26:36.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.236+0000 7fbb29ffb700 1 -- 172.21.15.16:0/4007924608 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1400ba40 con 0x7fbb2c0cd970 2023-05-20T01:26:36.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.236+0000 7fbb2a7fc700 1 --2- 172.21.15.16:0/4007924608 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb14004850 0x7fbb14004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:36.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.237+0000 7fbb2affd700 1 --2- 172.21.15.16:0/4007924608 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbb14003a70 0x7fbb14003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:36.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.237+0000 7fbb2b7fe700 1 --2- 172.21.15.16:0/4007924608 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbb2c0cd970 0x7fbb14005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:36.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.237+0000 7fbb2a7fc700 1 -- 172.21.15.16:0/4007924608 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbb2c0cd970 msgr2=0x7fbb14005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:36.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.237+0000 7fbb2a7fc700 1 --2- 172.21.15.16:0/4007924608 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbb2c0cd970 0x7fbb14005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:36.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.237+0000 7fbb2a7fc700 1 -- 172.21.15.16:0/4007924608 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbb14003a70 msgr2=0x7fbb14003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:36.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.237+0000 7fbb2a7fc700 1 --2- 172.21.15.16:0/4007924608 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbb14003a70 0x7fbb14003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:36.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.237+0000 7fbb2a7fc700 1 -- 172.21.15.16:0/4007924608 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb140035b0 con 0x7fbb14004850 2023-05-20T01:26:36.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.237+0000 7fbb2a7fc700 1 --2- 172.21.15.16:0/4007924608 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb14004850 0x7fbb14004cc0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7fbb2000c370 tx=0x7fbb14005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3d53125b3c9beb4 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:36.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.237+0000 7fbb29ffb700 1 -- 172.21.15.16:0/4007924608 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb20007c60 con 0x7fbb14004850 2023-05-20T01:26:36.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.237+0000 7fbb29ffb700 1 -- 172.21.15.16:0/4007924608 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb20005960 con 0x7fbb14004850 2023-05-20T01:26:36.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.237+0000 7fbb29ffb700 1 -- 172.21.15.16:0/4007924608 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb20005d10 con 0x7fbb14004850 2023-05-20T01:26:36.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.238+0000 7fbb314e1700 1 -- 172.21.15.16:0/4007924608 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb14004850 msgr2=0x7fbb14004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:36.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.238+0000 7fbb314e1700 1 --2- 172.21.15.16:0/4007924608 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb14004850 0x7fbb14004cc0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7fbb2000c370 tx=0x7fbb14005880 comp rx=0 tx=0).stop 2023-05-20T01:26:36.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.238+0000 7fbb2b7fe700 1 -- 172.21.15.16:0/4007924608 reap_dead start 2023-05-20T01:26:36.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.238+0000 7fbb314e1700 1 -- 172.21.15.16:0/4007924608 shutdown_connections 2023-05-20T01:26:36.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.238+0000 7fbb314e1700 1 -- 172.21.15.16:0/4007924608 >> 172.21.15.16:0/4007924608 conn(0x7fbb2c09c590 msgr2=0x7fbb2c0d3240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:36.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.238+0000 7fbb314e1700 1 -- 172.21.15.16:0/4007924608 shutdown_connections 2023-05-20T01:26:36.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.238+0000 7fbb314e1700 1 -- 172.21.15.16:0/4007924608 wait complete. 2023-05-20T01:26:36.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.239+0000 7fbb314e1700 1 Processor -- start 2023-05-20T01:26:36.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.239+0000 7fbb314e1700 1 -- start start 2023-05-20T01:26:36.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.240+0000 7fbb314e1700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbb14004850 0x7fbb2c14c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:36.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.240+0000 7fbb314e1700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbb2c0b8970 0x7fbb2c1f1550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:36.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.240+0000 7fbb314e1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb2c0b8de0 0x7fbb2c1f1a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:36.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.240+0000 7fbb314e1700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb2c0ccdb0 con 0x7fbb2c0b8de0 2023-05-20T01:26:36.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.240+0000 7fbb314e1700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb2c0cc130 con 0x7fbb14004850 2023-05-20T01:26:36.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.240+0000 7fbb314e1700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb2c0cc410 con 0x7fbb2c0b8970 2023-05-20T01:26:36.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.240+0000 7fbb2b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb2c0b8de0 0x7fbb2c1f1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:36.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.240+0000 7fbb2b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb2c0b8de0 0x7fbb2c1f1a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38058/0 (socket says 172.21.15.16:38058) 2023-05-20T01:26:36.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.240+0000 7fbb2b7fe700 1 -- 172.21.15.16:0/3161106662 learned_addr learned my addr 172.21.15.16:0/3161106662 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:26:36.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.240+0000 7fbb2affd700 1 --2- 172.21.15.16:0/3161106662 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbb14004850 0x7fbb2c14c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:36.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.240+0000 7fbb2a7fc700 1 --2- 172.21.15.16:0/3161106662 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbb2c0b8970 0x7fbb2c1f1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:36.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.240+0000 7fbb2b7fe700 1 -- 172.21.15.16:0/3161106662 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbb2c0b8970 msgr2=0x7fbb2c1f1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:36.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.240+0000 7fbb2b7fe700 1 --2- 172.21.15.16:0/3161106662 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbb2c0b8970 0x7fbb2c1f1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:36.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.240+0000 7fbb2b7fe700 1 -- 172.21.15.16:0/3161106662 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbb14004850 msgr2=0x7fbb2c14c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:36.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.240+0000 7fbb2b7fe700 1 --2- 172.21.15.16:0/3161106662 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbb14004850 0x7fbb2c14c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:36.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.240+0000 7fbb2b7fe700 1 -- 172.21.15.16:0/3161106662 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb2c0b4380 con 0x7fbb2c0b8de0 2023-05-20T01:26:36.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.240+0000 7fbb2b7fe700 1 --2- 172.21.15.16:0/3161106662 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb2c0b8de0 0x7fbb2c1f1a90 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c0d26a0 tx=0x7fbb24011ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:36.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.241+0000 7fbb137fe700 1 -- 172.21.15.16:0/3161106662 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb2400e040 con 0x7fbb2c0b8de0 2023-05-20T01:26:36.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.241+0000 7fbb137fe700 1 -- 172.21.15.16:0/3161106662 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb2400f040 con 0x7fbb2c0b8de0 2023-05-20T01:26:36.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.241+0000 7fbb314e1700 1 -- 172.21.15.16:0/3161106662 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb2c15d0b0 con 0x7fbb2c0b8de0 2023-05-20T01:26:36.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.241+0000 7fbb137fe700 1 -- 172.21.15.16:0/3161106662 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb2400fc40 con 0x7fbb2c0b8de0 2023-05-20T01:26:36.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.241+0000 7fbb314e1700 1 -- 172.21.15.16:0/3161106662 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb2c0d4000 con 0x7fbb2c0b8de0 2023-05-20T01:26:36.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.242+0000 7fbb314e1700 1 -- 172.21.15.16:0/3161106662 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb2c04e8c0 con 0x7fbb2c0b8de0 2023-05-20T01:26:36.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.242+0000 7fbb137fe700 1 -- 172.21.15.16:0/3161106662 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fbb24015430 con 0x7fbb2c0b8de0 2023-05-20T01:26:36.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.243+0000 7fbb137fe700 1 --2- 172.21.15.16:0/3161106662 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fbb1806ace0 0x7fbb1806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:36.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.243+0000 7fbb137fe700 1 -- 172.21.15.16:0/3161106662 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fbb24089c50 con 0x7fbb2c0b8de0 2023-05-20T01:26:36.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.243+0000 7fbb2affd700 1 --2- 172.21.15.16:0/3161106662 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fbb1806ace0 0x7fbb1806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:36.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.243+0000 7fbb2affd700 1 --2- 172.21.15.16:0/3161106662 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fbb1806ace0 0x7fbb1806d190 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c009c60 tx=0x7fbb1c009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:36.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.247+0000 7fbb137fe700 1 -- 172.21.15.16:0/3161106662 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fbb240586f0 con 0x7fbb2c0b8de0 2023-05-20T01:26:36.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.448+0000 7fbb314e1700 1 -- 172.21.15.16:0/3161106662 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbb2c15e9c0 con 0x7fbb2c0b8de0 2023-05-20T01:26:36.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.449+0000 7fbb137fe700 1 -- 172.21.15.16:0/3161106662 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v111) v1 ==== 74+0+18417 (secure 0 0 0) 0x7fbb24058280 con 0x7fbb2c0b8de0 2023-05-20T01:26:36.453 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:26:36.454 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":111,"fsid":"c2245cc5-ecab-4087-a59b-bc81db901676","created":"2023-05-20T01:16:09.549230+0000","modified":"2023-05-20T01:26:36.067072+0000","last_up_change":"2023-05-20T01:16:19.788624+0000","last_in_change":"2023-05-20T01:16:12.092160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-05-20T01:16:20.934366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-05-20T01:16:21.372294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-05-20T01:26:34.067017+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-05-20T01:26:35.040602+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"317a83cd-19c8-4958-a301-6087ef342289","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6825","nonce":3923634179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6827","nonce":3923634179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6831","nonce":3923634179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6829","nonce":3923634179}]},"public_addr":"172.21.15.16:6825/3923634179","cluster_addr":"172.21.15.16:6827/3923634179","heartbeat_back_addr":"172.21.15.16:6831/3923634179","heartbeat_front_addr":"172.21.15.16:6829/3923634179","state":["exists","up"]},{"osd":1,"uuid":"5581046f-8219-4089-8d7d-b7846399505e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6809","nonce":3860264220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6811","nonce":3860264220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6815","nonce":3860264220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6813","nonce":3860264220}]},"public_addr":"172.21.15.16:6809/3860264220","cluster_addr":"172.21.15.16:6811/3860264220","heartbeat_back_addr":"172.21.15.16:6815/3860264220","heartbeat_front_addr":"172.21.15.16:6813/3860264220","state":["exists","up"]},{"osd":2,"uuid":"072e9e66-4133-478e-8d4a-64d46c2963f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6817","nonce":293187030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6819","nonce":293187030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6823","nonce":293187030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6821","nonce":293187030}]},"public_addr":"172.21.15.16:6817/293187030","cluster_addr":"172.21.15.16:6819/293187030","heartbeat_back_addr":"172.21.15.16:6823/293187030","heartbeat_front_addr":"172.21.15.16:6821/293187030","state":["exists","up"]},{"osd":3,"uuid":"11e1bba6-00f8-48dd-9028-d8d716b7f145","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6801","nonce":3669781890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6803","nonce":3669781890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6807","nonce":3669781890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6805","nonce":3669781890}]},"public_addr":"172.21.15.16:6801/3669781890","cluster_addr":"172.21.15.16:6803/3669781890","heartbeat_back_addr":"172.21.15.16:6807/3669781890","heartbeat_front_addr":"172.21.15.16:6805/3669781890","state":["exists","up"]},{"osd":4,"uuid":"b8909c3a-cf70-46cb-8d0b-009c707cf9d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6819","nonce":506317223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6823","nonce":506317223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6829","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6831","nonce":506317223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6827","nonce":506317223}]},"public_addr":"172.21.15.132:6819/506317223","cluster_addr":"172.21.15.132:6823/506317223","heartbeat_back_addr":"172.21.15.132:6831/506317223","heartbeat_front_addr":"172.21.15.132:6827/506317223","state":["exists","up"]},{"osd":5,"uuid":"b1818304-4583-47bf-a6ec-0bfbb5166233","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6818","nonce":1844689995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6822","nonce":1844689995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6830","nonce":1844689995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6826","nonce":1844689995}]},"public_addr":"172.21.15.132:6818/1844689995","cluster_addr":"172.21.15.132:6822/1844689995","heartbeat_back_addr":"172.21.15.132:6830/1844689995","heartbeat_front_addr":"172.21.15.132:6826/1844689995","state":["exists","up"]},{"osd":6,"uuid":"d37f4691-eed1-4a93-8818-18354f03f4a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6801","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6802","nonce":727581123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6805","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6807","nonce":727581123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6813","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6815","nonce":727581123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6811","nonce":727581123}]},"public_addr":"172.21.15.132:6802/727581123","cluster_addr":"172.21.15.132:6807/727581123","heartbeat_back_addr":"172.21.15.132:6815/727581123","heartbeat_front_addr":"172.21.15.132:6811/727581123","state":["exists","up"]},{"osd":7,"uuid":"4f712f62-5d1d-4816-8042-03209144f8b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6803","nonce":466001149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6806","nonce":466001149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6814","nonce":466001149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6810","nonce":466001149}]},"public_addr":"172.21.15.132:6803/466001149","cluster_addr":"172.21.15.132:6806/466001149","heartbeat_back_addr":"172.21.15.132:6814/466001149","heartbeat_front_addr":"172.21.15.132:6810/466001149","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.338863+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.278680+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305694+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.246835+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.276237+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.304441+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305421+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.267050+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-20T01:26:36.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.451+0000 7fbb314e1700 1 -- 172.21.15.16:0/3161106662 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fbb1806ace0 msgr2=0x7fbb1806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:36.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.451+0000 7fbb314e1700 1 --2- 172.21.15.16:0/3161106662 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fbb1806ace0 0x7fbb1806d190 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c009c60 tx=0x7fbb1c009210 comp rx=0 tx=0).stop 2023-05-20T01:26:36.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.451+0000 7fbb314e1700 1 -- 172.21.15.16:0/3161106662 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb2c0b8de0 msgr2=0x7fbb2c1f1a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:36.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.451+0000 7fbb314e1700 1 --2- 172.21.15.16:0/3161106662 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb2c0b8de0 0x7fbb2c1f1a90 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c0d26a0 tx=0x7fbb24011ed0 comp rx=0 tx=0).stop 2023-05-20T01:26:36.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.451+0000 7fbb314e1700 1 -- 172.21.15.16:0/3161106662 shutdown_connections 2023-05-20T01:26:36.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.451+0000 7fbb314e1700 1 --2- 172.21.15.16:0/3161106662 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb2c0b8de0 0x7fbb2c1f1a90 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:36.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.451+0000 7fbb314e1700 1 --2- 172.21.15.16:0/3161106662 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbb14004850 0x7fbb2c14c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:36.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.451+0000 7fbb314e1700 1 --2- 172.21.15.16:0/3161106662 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fbb1806ace0 0x7fbb1806d190 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:36.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.451+0000 7fbb314e1700 1 --2- 172.21.15.16:0/3161106662 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbb2c0b8970 0x7fbb2c1f1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:36.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.451+0000 7fbb314e1700 1 -- 172.21.15.16:0/3161106662 >> 172.21.15.16:0/3161106662 conn(0x7fbb2c09c590 msgr2=0x7fbb2c0ce900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:36.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.451+0000 7fbb314e1700 1 -- 172.21.15.16:0/3161106662 shutdown_connections 2023-05-20T01:26:36.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.451+0000 7fbb314e1700 1 -- 172.21.15.16:0/3161106662 wait complete. 2023-05-20T01:26:36.468 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-20T01:26:36.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.589+0000 7f44826c1700 1 Processor -- start 2023-05-20T01:26:36.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.589+0000 7f44826c1700 1 -- start start 2023-05-20T01:26:36.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.590+0000 7f44826c1700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f447c0d2da0 0x7f447c0d0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:36.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.590+0000 7f44826c1700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f447c0d1730 con 0x7f447c0b3880 2023-05-20T01:26:36.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.590+0000 7f44826c1700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f447c0b4520 con 0x7f447c0d1330 2023-05-20T01:26:36.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.590+0000 7f44826c1700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f447c0b7a60 con 0x7f447c0d2da0 2023-05-20T01:26:36.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.590+0000 7f447bfff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f447c0b3880 0x7f447c0b3c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49590/0 (socket says 172.21.15.16:49590) 2023-05-20T01:26:36.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.590+0000 7f447bfff700 1 -- 172.21.15.16:0/839506330 learned_addr learned my addr 172.21.15.16:0/839506330 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:26:36.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.590+0000 7f447b7fe700 1 --2- 172.21.15.16:0/839506330 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f447c0d2da0 0x7f447c0d0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:36.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.590+0000 7f447affd700 1 -- 172.21.15.16:0/839506330 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2582725132 0 0) 0x7f447c0d1730 con 0x7f447c0b3880 2023-05-20T01:26:36.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.590+0000 7f447affd700 1 -- 172.21.15.16:0/839506330 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44640035b0 con 0x7f447c0b3880 2023-05-20T01:26:36.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.590+0000 7f447affd700 1 -- 172.21.15.16:0/839506330 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1677599008 0 0) 0x7f447c0b4520 con 0x7f447c0d1330 2023-05-20T01:26:36.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.590+0000 7f447b7fe700 1 -- 172.21.15.16:0/839506330 >> v1:172.21.15.132:6789/0 conn(0x7f447c0d1330 legacy=0x7f447c15d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:36.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.590+0000 7f447b7fe700 1 -- 172.21.15.16:0/839506330 >> v1:172.21.15.16:6789/0 conn(0x7f447c0b3880 legacy=0x7f447c0b3c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:36.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.591+0000 7f447b7fe700 1 -- 172.21.15.16:0/839506330 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f447c0b7c70 con 0x7f447c0d2da0 2023-05-20T01:26:36.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.591+0000 7f447b7fe700 1 --2- 172.21.15.16:0/839506330 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f447c0d2da0 0x7f447c0d0df0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f446c009870 tx=0x7f446c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f265cbb828b11e4 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:36.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.591+0000 7f447affd700 1 -- 172.21.15.16:0/839506330 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44700015f0 con 0x7f447c0d2da0 2023-05-20T01:26:36.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.591+0000 7f447affd700 1 -- 172.21.15.16:0/839506330 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f446c004030 con 0x7f447c0d2da0 2023-05-20T01:26:36.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.591+0000 7f447affd700 1 -- 172.21.15.16:0/839506330 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f446c0043a0 con 0x7f447c0d2da0 2023-05-20T01:26:36.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.591+0000 7f44826c1700 1 -- 172.21.15.16:0/839506330 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f447c0d2da0 msgr2=0x7f447c0d0df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:36.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.591+0000 7f44826c1700 1 --2- 172.21.15.16:0/839506330 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f447c0d2da0 0x7f447c0d0df0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f446c009870 tx=0x7f446c009b80 comp rx=0 tx=0).stop 2023-05-20T01:26:36.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.591+0000 7f44826c1700 1 -- 172.21.15.16:0/839506330 shutdown_connections 2023-05-20T01:26:36.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.591+0000 7f44826c1700 1 --2- 172.21.15.16:0/839506330 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f447c0d2da0 0x7f447c0d0df0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:36.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.591+0000 7f44826c1700 1 -- 172.21.15.16:0/839506330 >> 172.21.15.16:0/839506330 conn(0x7f447c09c590 msgr2=0x7f447c0d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:36.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.591+0000 7f44826c1700 1 -- 172.21.15.16:0/839506330 shutdown_connections 2023-05-20T01:26:36.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.592+0000 7f44826c1700 1 -- 172.21.15.16:0/839506330 wait complete. 2023-05-20T01:26:36.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.593+0000 7f44826c1700 1 Processor -- start 2023-05-20T01:26:36.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.593+0000 7f44826c1700 1 -- start start 2023-05-20T01:26:36.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.594+0000 7f44826c1700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f447c0b3880 0x7f447c156760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:36.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.594+0000 7f44826c1700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f447c0d1330 0x7f447c158450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:36.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.594+0000 7f44826c1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f447c0d2da0 0x7f447c0c91f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:36.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.594+0000 7f44826c1700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f447c156ca0 con 0x7f447c0d2da0 2023-05-20T01:26:36.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.594+0000 7f44826c1700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f447c0c6a90 con 0x7f447c0d1330 2023-05-20T01:26:36.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.594+0000 7f44826c1700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f447c0c6d70 con 0x7f447c0b3880 2023-05-20T01:26:36.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.594+0000 7f4480c5e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f447c0d2da0 0x7f447c0c91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:36.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.594+0000 7f4480c5e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f447c0d2da0 0x7f447c0c91f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38072/0 (socket says 172.21.15.16:38072) 2023-05-20T01:26:36.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.594+0000 7f4480c5e700 1 -- 172.21.15.16:0/973259444 learned_addr learned my addr 172.21.15.16:0/973259444 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:26:36.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.594+0000 7f447bfff700 1 --2- 172.21.15.16:0/973259444 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f447c0b3880 0x7f447c156760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:36.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.594+0000 7f447b7fe700 1 --2- 172.21.15.16:0/973259444 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f447c0d1330 0x7f447c158450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:36.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.594+0000 7f4480c5e700 1 -- 172.21.15.16:0/973259444 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f447c0b3880 msgr2=0x7f447c156760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:36.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.594+0000 7f4480c5e700 1 --2- 172.21.15.16:0/973259444 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f447c0b3880 0x7f447c156760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:36.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.594+0000 7f4480c5e700 1 -- 172.21.15.16:0/973259444 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f447c0d1330 msgr2=0x7f447c158450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:36.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.594+0000 7f4480c5e700 1 --2- 172.21.15.16:0/973259444 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f447c0d1330 0x7f447c158450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:36.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.594+0000 7f4480c5e700 1 -- 172.21.15.16:0/973259444 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f446c0094a0 con 0x7f447c0d2da0 2023-05-20T01:26:36.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.594+0000 7f4480c5e700 1 --2- 172.21.15.16:0/973259444 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f447c0d2da0 0x7f447c0c91f0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f447400c790 tx=0x7f447400caa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:36.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.594+0000 7f4478ff9700 1 -- 172.21.15.16:0/973259444 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4474003e30 con 0x7f447c0d2da0 2023-05-20T01:26:36.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.594+0000 7f4478ff9700 1 -- 172.21.15.16:0/973259444 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4474005070 con 0x7f447c0d2da0 2023-05-20T01:26:36.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.595+0000 7f44826c1700 1 -- 172.21.15.16:0/973259444 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f447c0b8230 con 0x7f447c0d2da0 2023-05-20T01:26:36.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.595+0000 7f4478ff9700 1 -- 172.21.15.16:0/973259444 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44740053e0 con 0x7f447c0d2da0 2023-05-20T01:26:36.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.595+0000 7f44826c1700 1 -- 172.21.15.16:0/973259444 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f447c0b7f50 con 0x7f447c0d2da0 2023-05-20T01:26:36.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.595+0000 7f44826c1700 1 -- 172.21.15.16:0/973259444 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f447c09c990 con 0x7f447c0d2da0 2023-05-20T01:26:36.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.596+0000 7f4478ff9700 1 -- 172.21.15.16:0/973259444 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4474005540 con 0x7f447c0d2da0 2023-05-20T01:26:36.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.596+0000 7f4478ff9700 1 --2- 172.21.15.16:0/973259444 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f446806ace0 0x7f446806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:36.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.597+0000 7f4478ff9700 1 -- 172.21.15.16:0/973259444 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f4474089140 con 0x7f447c0d2da0 2023-05-20T01:26:36.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.597+0000 7f447bfff700 1 --2- 172.21.15.16:0/973259444 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f446806ace0 0x7f446806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:36.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.597+0000 7f447bfff700 1 --2- 172.21.15.16:0/973259444 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f446806ace0 0x7f446806d190 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f44700011e0 tx=0x7f4470001810 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:36.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.601+0000 7f4478ff9700 1 -- 172.21.15.16:0/973259444 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f44740540b0 con 0x7f447c0d2da0 2023-05-20T01:26:36.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.799+0000 7f44826c1700 1 -- 172.21.15.16:0/973259444 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f447c04e8c0 con 0x7f447c0d2da0 2023-05-20T01:26:36.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.801+0000 7f4478ff9700 1 -- 172.21.15.16:0/973259444 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v111) v1 ==== 74+0+18417 (secure 0 0 0) 0x7f44740057f0 con 0x7f447c0d2da0 2023-05-20T01:26:36.806 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:26:36.806 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":111,"fsid":"c2245cc5-ecab-4087-a59b-bc81db901676","created":"2023-05-20T01:16:09.549230+0000","modified":"2023-05-20T01:26:36.067072+0000","last_up_change":"2023-05-20T01:16:19.788624+0000","last_in_change":"2023-05-20T01:16:12.092160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-05-20T01:16:20.934366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-05-20T01:16:21.372294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-05-20T01:26:34.067017+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-05-20T01:26:35.040602+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"317a83cd-19c8-4958-a301-6087ef342289","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6825","nonce":3923634179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6827","nonce":3923634179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6831","nonce":3923634179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6829","nonce":3923634179}]},"public_addr":"172.21.15.16:6825/3923634179","cluster_addr":"172.21.15.16:6827/3923634179","heartbeat_back_addr":"172.21.15.16:6831/3923634179","heartbeat_front_addr":"172.21.15.16:6829/3923634179","state":["exists","up"]},{"osd":1,"uuid":"5581046f-8219-4089-8d7d-b7846399505e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6809","nonce":3860264220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6811","nonce":3860264220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6815","nonce":3860264220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6813","nonce":3860264220}]},"public_addr":"172.21.15.16:6809/3860264220","cluster_addr":"172.21.15.16:6811/3860264220","heartbeat_back_addr":"172.21.15.16:6815/3860264220","heartbeat_front_addr":"172.21.15.16:6813/3860264220","state":["exists","up"]},{"osd":2,"uuid":"072e9e66-4133-478e-8d4a-64d46c2963f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6817","nonce":293187030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6819","nonce":293187030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6823","nonce":293187030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6821","nonce":293187030}]},"public_addr":"172.21.15.16:6817/293187030","cluster_addr":"172.21.15.16:6819/293187030","heartbeat_back_addr":"172.21.15.16:6823/293187030","heartbeat_front_addr":"172.21.15.16:6821/293187030","state":["exists","up"]},{"osd":3,"uuid":"11e1bba6-00f8-48dd-9028-d8d716b7f145","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6801","nonce":3669781890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6803","nonce":3669781890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6807","nonce":3669781890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6805","nonce":3669781890}]},"public_addr":"172.21.15.16:6801/3669781890","cluster_addr":"172.21.15.16:6803/3669781890","heartbeat_back_addr":"172.21.15.16:6807/3669781890","heartbeat_front_addr":"172.21.15.16:6805/3669781890","state":["exists","up"]},{"osd":4,"uuid":"b8909c3a-cf70-46cb-8d0b-009c707cf9d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6819","nonce":506317223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6823","nonce":506317223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6829","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6831","nonce":506317223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6827","nonce":506317223}]},"public_addr":"172.21.15.132:6819/506317223","cluster_addr":"172.21.15.132:6823/506317223","heartbeat_back_addr":"172.21.15.132:6831/506317223","heartbeat_front_addr":"172.21.15.132:6827/506317223","state":["exists","up"]},{"osd":5,"uuid":"b1818304-4583-47bf-a6ec-0bfbb5166233","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6818","nonce":1844689995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6822","nonce":1844689995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6830","nonce":1844689995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6826","nonce":1844689995}]},"public_addr":"172.21.15.132:6818/1844689995","cluster_addr":"172.21.15.132:6822/1844689995","heartbeat_back_addr":"172.21.15.132:6830/1844689995","heartbeat_front_addr":"172.21.15.132:6826/1844689995","state":["exists","up"]},{"osd":6,"uuid":"d37f4691-eed1-4a93-8818-18354f03f4a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6801","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6802","nonce":727581123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6805","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6807","nonce":727581123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6813","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6815","nonce":727581123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6811","nonce":727581123}]},"public_addr":"172.21.15.132:6802/727581123","cluster_addr":"172.21.15.132:6807/727581123","heartbeat_back_addr":"172.21.15.132:6815/727581123","heartbeat_front_addr":"172.21.15.132:6811/727581123","state":["exists","up"]},{"osd":7,"uuid":"4f712f62-5d1d-4816-8042-03209144f8b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6803","nonce":466001149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6806","nonce":466001149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6814","nonce":466001149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6810","nonce":466001149}]},"public_addr":"172.21.15.132:6803/466001149","cluster_addr":"172.21.15.132:6806/466001149","heartbeat_back_addr":"172.21.15.132:6814/466001149","heartbeat_front_addr":"172.21.15.132:6810/466001149","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.338863+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.278680+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305694+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.246835+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.276237+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.304441+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305421+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.267050+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-20T01:26:36.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.803+0000 7f44826c1700 1 -- 172.21.15.16:0/973259444 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f446806ace0 msgr2=0x7f446806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:36.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.803+0000 7f44826c1700 1 --2- 172.21.15.16:0/973259444 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f446806ace0 0x7f446806d190 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f44700011e0 tx=0x7f4470001810 comp rx=0 tx=0).stop 2023-05-20T01:26:36.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.803+0000 7f44826c1700 1 -- 172.21.15.16:0/973259444 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f447c0d2da0 msgr2=0x7f447c0c91f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:36.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.803+0000 7f44826c1700 1 --2- 172.21.15.16:0/973259444 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f447c0d2da0 0x7f447c0c91f0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f447400c790 tx=0x7f447400caa0 comp rx=0 tx=0).stop 2023-05-20T01:26:36.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.803+0000 7f44826c1700 1 -- 172.21.15.16:0/973259444 shutdown_connections 2023-05-20T01:26:36.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.803+0000 7f44826c1700 1 --2- 172.21.15.16:0/973259444 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f447c0d2da0 0x7f447c0c91f0 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:36.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.803+0000 7f44826c1700 1 --2- 172.21.15.16:0/973259444 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f446806ace0 0x7f446806d190 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:36.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.803+0000 7f44826c1700 1 --2- 172.21.15.16:0/973259444 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f447c0b3880 0x7f447c156760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:36.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.803+0000 7f44826c1700 1 --2- 172.21.15.16:0/973259444 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f447c0d1330 0x7f447c158450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:36.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.803+0000 7f44826c1700 1 -- 172.21.15.16:0/973259444 >> 172.21.15.16:0/973259444 conn(0x7f447c09c590 msgr2=0x7f447c15f660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:36.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.803+0000 7f44826c1700 1 -- 172.21.15.16:0/973259444 shutdown_connections 2023-05-20T01:26:36.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.803+0000 7f44826c1700 1 -- 172.21.15.16:0/973259444 wait complete. 2023-05-20T01:26:36.820 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-05-20T01:26:36.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.945+0000 7f6184807700 1 Processor -- start 2023-05-20T01:26:36.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.945+0000 7f6184807700 1 -- start start 2023-05-20T01:26:36.949 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:26:36.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.945+0000 7f6184807700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f618015f000 0x7f61800cbc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:36.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.945+0000 7f6184807700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6180072340 0x7f61800cc180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:36.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.946+0000 7f6184807700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f618006a5b0 con 0x7f61800b57b0 2023-05-20T01:26:36.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.946+0000 7f6184807700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f618006a850 con 0x7f6180072340 2023-05-20T01:26:36.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.946+0000 7f6184807700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f618006ab30 con 0x7f618015f000 2023-05-20T01:26:36.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.946+0000 7f617ed9d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6180072340 0x7f61800cc180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:36.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.946+0000 7f617e59c700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f61800b57b0 0x7f6180071c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49604/0 (socket says 172.21.15.16:49604) 2023-05-20T01:26:36.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.946+0000 7f617ed9d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6180072340 0x7f61800cc180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38080/0 (socket says 172.21.15.16:38080) 2023-05-20T01:26:36.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.946+0000 7f617e59c700 1 -- 172.21.15.16:0/3025604215 learned_addr learned my addr 172.21.15.16:0/3025604215 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:26:36.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.946+0000 7f617dd9b700 1 --2- 172.21.15.16:0/3025604215 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f618015f000 0x7f61800cbc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:36.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.946+0000 7f617ed9d700 1 -- 172.21.15.16:0/3025604215 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f618015f000 msgr2=0x7f61800cbc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:36.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.946+0000 7f617ed9d700 1 --2- 172.21.15.16:0/3025604215 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f618015f000 0x7f61800cbc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:36.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.946+0000 7f617ed9d700 1 -- 172.21.15.16:0/3025604215 >> v1:172.21.15.16:6789/0 conn(0x7f61800b57b0 legacy=0x7f6180071c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:36.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.946+0000 7f617ed9d700 1 -- 172.21.15.16:0/3025604215 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f618006ae10 con 0x7f6180072340 2023-05-20T01:26:36.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.946+0000 7f617d59a700 1 -- 172.21.15.16:0/3025604215 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 759815333 0 0) 0x7f618006a5b0 con 0x7f61800b57b0 2023-05-20T01:26:36.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.946+0000 7f617ed9d700 1 --2- 172.21.15.16:0/3025604215 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6180072340 0x7f61800cc180 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f6178009d90 tx=0x7f617800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de35c31dbe663518 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:36.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.946+0000 7f617d59a700 1 -- 172.21.15.16:0/3025604215 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f617800e070 con 0x7f6180072340 2023-05-20T01:26:36.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.946+0000 7f617d59a700 1 -- 172.21.15.16:0/3025604215 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f617800bb20 con 0x7f6180072340 2023-05-20T01:26:36.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.946+0000 7f617d59a700 1 -- 172.21.15.16:0/3025604215 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6178012430 con 0x7f6180072340 2023-05-20T01:26:36.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.947+0000 7f6184807700 1 -- 172.21.15.16:0/3025604215 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6180072340 msgr2=0x7f61800cc180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:36.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.947+0000 7f6184807700 1 --2- 172.21.15.16:0/3025604215 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6180072340 0x7f61800cc180 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f6178009d90 tx=0x7f617800b330 comp rx=0 tx=0).stop 2023-05-20T01:26:36.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.947+0000 7f6184807700 1 -- 172.21.15.16:0/3025604215 shutdown_connections 2023-05-20T01:26:36.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.947+0000 7f6184807700 1 --2- 172.21.15.16:0/3025604215 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6180072340 0x7f61800cc180 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:36.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.947+0000 7f6184807700 1 --2- 172.21.15.16:0/3025604215 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f618015f000 0x7f61800cbc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:36.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.947+0000 7f6184807700 1 -- 172.21.15.16:0/3025604215 >> 172.21.15.16:0/3025604215 conn(0x7f618006c260 msgr2=0x7f61800cffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:36.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.947+0000 7f6184807700 1 -- 172.21.15.16:0/3025604215 shutdown_connections 2023-05-20T01:26:36.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.947+0000 7f6184807700 1 -- 172.21.15.16:0/3025604215 wait complete. 2023-05-20T01:26:36.956 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:26:36.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.948+0000 7f6184807700 1 Processor -- start 2023-05-20T01:26:36.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.948+0000 7f6184807700 1 -- start start 2023-05-20T01:26:36.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.949+0000 7f6184807700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6180072340 0x7f61800c8960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:36.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.949+0000 7f6184807700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61800b57b0 0x7f6180153430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:36.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.949+0000 7f6184807700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f618015f000 0x7f61800b16a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:36.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.949+0000 7f6184807700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f618015adc0 con 0x7f61800b57b0 2023-05-20T01:26:36.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.949+0000 7f6184807700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61800c45e0 con 0x7f6180072340 2023-05-20T01:26:36.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.949+0000 7f6184807700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61800c48c0 con 0x7f618015f000 2023-05-20T01:26:36.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.949+0000 7f617dd9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61800b57b0 0x7f6180153430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:36.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.949+0000 7f617dd9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61800b57b0 0x7f6180153430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38084/0 (socket says 172.21.15.16:38084) 2023-05-20T01:26:36.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.949+0000 7f617dd9b700 1 -- 172.21.15.16:0/951141174 learned_addr learned my addr 172.21.15.16:0/951141174 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:26:36.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.949+0000 7f617ed9d700 1 --2- 172.21.15.16:0/951141174 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f618015f000 0x7f61800b16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:36.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.949+0000 7f617e59c700 1 --2- 172.21.15.16:0/951141174 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6180072340 0x7f61800c8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:36.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.950+0000 7f617dd9b700 1 -- 172.21.15.16:0/951141174 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f618015f000 msgr2=0x7f61800b16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:36.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.950+0000 7f617dd9b700 1 --2- 172.21.15.16:0/951141174 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f618015f000 0x7f61800b16a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:36.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.950+0000 7f617dd9b700 1 -- 172.21.15.16:0/951141174 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6180072340 msgr2=0x7f61800c8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:36.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.950+0000 7f617dd9b700 1 --2- 172.21.15.16:0/951141174 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6180072340 0x7f61800c8960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:36.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.950+0000 7f617dd9b700 1 -- 172.21.15.16:0/951141174 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6178009a70 con 0x7f61800b57b0 2023-05-20T01:26:36.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.950+0000 7f617ed9d700 1 --2- 172.21.15.16:0/951141174 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f618015f000 0x7f61800b16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:26:36.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.950+0000 7f617e59c700 1 --2- 172.21.15.16:0/951141174 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6180072340 0x7f61800c8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:26:36.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.950+0000 7f617dd9b700 1 --2- 172.21.15.16:0/951141174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61800b57b0 0x7f6180153430 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f617400e840 tx=0x7f617400ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:36.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.950+0000 7f616effd700 1 -- 172.21.15.16:0/951141174 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6174009590 con 0x7f61800b57b0 2023-05-20T01:26:36.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.950+0000 7f616effd700 1 -- 172.21.15.16:0/951141174 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6174013680 con 0x7f61800b57b0 2023-05-20T01:26:36.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.950+0000 7f6184807700 1 -- 172.21.15.16:0/951141174 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61800af710 con 0x7f61800b57b0 2023-05-20T01:26:36.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.950+0000 7f616effd700 1 -- 172.21.15.16:0/951141174 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6174013a30 con 0x7f61800b57b0 2023-05-20T01:26:36.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.950+0000 7f6184807700 1 -- 172.21.15.16:0/951141174 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f618015f4e0 con 0x7f61800b57b0 2023-05-20T01:26:36.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.951+0000 7f6184807700 1 -- 172.21.15.16:0/951141174 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61800af8a0 con 0x7f61800b57b0 2023-05-20T01:26:36.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.952+0000 7f616effd700 1 -- 172.21.15.16:0/951141174 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6174013b90 con 0x7f61800b57b0 2023-05-20T01:26:36.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.952+0000 7f616effd700 1 --2- 172.21.15.16:0/951141174 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f616006ace0 0x7f616006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:36.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.952+0000 7f616effd700 1 -- 172.21.15.16:0/951141174 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f6174089660 con 0x7f61800b57b0 2023-05-20T01:26:36.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.952+0000 7f617e59c700 1 --2- 172.21.15.16:0/951141174 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f616006ace0 0x7f616006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:36.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.953+0000 7f617e59c700 1 --2- 172.21.15.16:0/951141174 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f616006ace0 0x7f616006d190 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f6170000f60 tx=0x7f6170007ca0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:36.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:36.957+0000 7f616effd700 1 -- 172.21.15.16:0/951141174 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f6174057f50 con 0x7f61800b57b0 2023-05-20T01:26:37.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:37.207+0000 7f6184807700 1 -- 172.21.15.16:0/951141174 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f61800c4a80 con 0x7f61800b57b0 2023-05-20T01:26:38.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:38.083+0000 7f616effd700 1 -- 172.21.15.16:0/951141174 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v77) v1 ==== 118+0+0 (secure 0 0 0) 0x7f6174057ae0 con 0x7f61800b57b0 2023-05-20T01:26:38.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:38.086+0000 7f6184807700 1 -- 172.21.15.16:0/951141174 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f616006ace0 msgr2=0x7f616006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:38.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:38.086+0000 7f6184807700 1 --2- 172.21.15.16:0/951141174 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f616006ace0 0x7f616006d190 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f6170000f60 tx=0x7f6170007ca0 comp rx=0 tx=0).stop 2023-05-20T01:26:38.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:38.086+0000 7f6184807700 1 -- 172.21.15.16:0/951141174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61800b57b0 msgr2=0x7f6180153430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:38.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:38.086+0000 7f6184807700 1 --2- 172.21.15.16:0/951141174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61800b57b0 0x7f6180153430 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f617400e840 tx=0x7f617400ec00 comp rx=0 tx=0).stop 2023-05-20T01:26:38.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:38.086+0000 7f6184807700 1 -- 172.21.15.16:0/951141174 shutdown_connections 2023-05-20T01:26:38.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:38.086+0000 7f6184807700 1 --2- 172.21.15.16:0/951141174 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f616006ace0 0x7f616006d190 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:38.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:38.086+0000 7f6184807700 1 --2- 172.21.15.16:0/951141174 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f618015f000 0x7f61800b16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:38.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:38.086+0000 7f6184807700 1 --2- 172.21.15.16:0/951141174 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6180072340 0x7f61800c8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:38.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:38.086+0000 7f6184807700 1 --2- 172.21.15.16:0/951141174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61800b57b0 0x7f6180153430 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:38.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:38.086+0000 7f6184807700 1 -- 172.21.15.16:0/951141174 >> 172.21.15.16:0/951141174 conn(0x7f618006c260 msgr2=0x7f6180071c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:38.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:38.087+0000 7f6184807700 1 -- 172.21.15.16:0/951141174 shutdown_connections 2023-05-20T01:26:38.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:38.087+0000 7f6184807700 1 -- 172.21.15.16:0/951141174 wait complete. 2023-05-20T01:26:38.106 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-05-20T01:26:38.106 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-05-20T01:26:38.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:38.237+0000 7fb58eb62700 1 Processor -- start 2023-05-20T01:26:38.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:38.237+0000 7fb58eb62700 1 -- start start 2023-05-20T01:26:38.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:38.238+0000 7fb58eb62700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb58815e150 0x7fb588159e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:38.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:38.238+0000 7fb58eb62700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb58815d5b0 con 0x7fb5880cb700 2023-05-20T01:26:38.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:38.238+0000 7fb58eb62700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb588157820 con 0x7fb588157450 2023-05-20T01:26:38.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:38.238+0000 7fb58eb62700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb58815a4c0 con 0x7fb58815e150 2023-05-20T01:26:38.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:38.238+0000 7fb587fff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fb5880cb700 0x7fb58815da40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49620/0 (socket says 172.21.15.16:49620) 2023-05-20T01:26:38.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:38.238+0000 7fb587fff700 1 -- 172.21.15.16:0/550493766 learned_addr learned my addr 172.21.15.16:0/550493766 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:26:38.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:38.238+0000 7fb58d0ff700 1 --2- 172.21.15.16:0/550493766 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb58815e150 0x7fb588159e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:38.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:38.238+0000 7fb5877fe700 1 -- 172.21.15.16:0/550493766 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 364655091 0 0) 0x7fb58815d5b0 con 0x7fb5880cb700 2023-05-20T01:26:38.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:38.238+0000 7fb5877fe700 1 -- 172.21.15.16:0/550493766 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb5780035b0 con 0x7fb5880cb700 2023-05-20T01:26:38.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:38.238+0000 7fb5877fe700 1 -- 172.21.15.16:0/550493766 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 281895509 0 0) 0x7fb5780035b0 con 0x7fb5880cb700 2023-05-20T01:26:38.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:38.238+0000 7fb5877fe700 1 -- 172.21.15.16:0/550493766 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb58815e150 msgr2=0x7fb588159e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:38.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:38.238+0000 7fb5877fe700 1 --2- 172.21.15.16:0/550493766 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb58815e150 0x7fb588159e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:38.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:38.238+0000 7fb5877fe700 1 -- 172.21.15.16:0/550493766 >> v1:172.21.15.132:6789/0 conn(0x7fb588157450 legacy=0x7fb5880caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:38.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:38.238+0000 7fb5877fe700 1 -- 172.21.15.16:0/550493766 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb58815a7a0 con 0x7fb5880cb700 2023-05-20T01:26:38.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:38.238+0000 7fb5877fe700 1 -- 172.21.15.16:0/550493766 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7fb574001670 con 0x7fb5880cb700 2023-05-20T01:26:38.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:38.238+0000 7fb5877fe700 1 -- 172.21.15.16:0/550493766 >> v1:172.21.15.16:6789/0 conn(0x7fb5880cb700 legacy=0x7fb58815da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:38.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:38.238+0000 7fb5877fe700 1 --2- 172.21.15.16:0/550493766 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb58815e150 0x7fb578003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:38.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:38.238+0000 7fb5877fe700 1 --2- 172.21.15.16:0/550493766 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb578003fb0 0x7fb578004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:38.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:38.238+0000 7fb5877fe700 1 --2- 172.21.15.16:0/550493766 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb578004d90 0x7fb578005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:38.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:38.238+0000 7fb5877fe700 1 -- 172.21.15.16:0/550493766 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb58815a4c0 con 0x7fb578003fb0 2023-05-20T01:26:38.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:38.238+0000 7fb5877fe700 1 -- 172.21.15.16:0/550493766 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb57800baf0 con 0x7fb578004d90 2023-05-20T01:26:38.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:38.238+0000 7fb5877fe700 1 -- 172.21.15.16:0/550493766 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb57800bcb0 con 0x7fb58815e150 2023-05-20T01:26:38.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:38.238+0000 7fb587fff700 1 --2- 172.21.15.16:0/550493766 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb578003fb0 0x7fb578004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:38.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:38.239+0000 7fb58c8fe700 1 --2- 172.21.15.16:0/550493766 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb58815e150 0x7fb578003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:38.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:38.239+0000 7fb58d0ff700 1 --2- 172.21.15.16:0/550493766 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb578004d90 0x7fb578005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:38.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:38.239+0000 7fb587fff700 1 -- 172.21.15.16:0/550493766 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb58815e150 msgr2=0x7fb578003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:38.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:38.239+0000 7fb587fff700 1 --2- 172.21.15.16:0/550493766 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb58815e150 0x7fb578003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:38.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:38.239+0000 7fb587fff700 1 -- 172.21.15.16:0/550493766 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb578004d90 msgr2=0x7fb578005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:38.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:38.239+0000 7fb587fff700 1 --2- 172.21.15.16:0/550493766 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb578004d90 0x7fb578005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:38.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:38.239+0000 7fb587fff700 1 -- 172.21.15.16:0/550493766 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5780035b0 con 0x7fb578003fb0 2023-05-20T01:26:38.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:38.239+0000 7fb58d0ff700 1 --2- 172.21.15.16:0/550493766 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb578004d90 0x7fb578005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:26:38.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:38.239+0000 7fb587fff700 1 --2- 172.21.15.16:0/550493766 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb578003fb0 0x7fb578004850 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7fb57400c210 tx=0x7fb578005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffe98d0e7cb50a54 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:38.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:38.239+0000 7fb5877fe700 1 -- 172.21.15.16:0/550493766 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb574007c10 con 0x7fb578003fb0 2023-05-20T01:26:38.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:38.239+0000 7fb5877fe700 1 -- 172.21.15.16:0/550493766 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb57400cdd0 con 0x7fb578003fb0 2023-05-20T01:26:38.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:38.239+0000 7fb5877fe700 1 -- 172.21.15.16:0/550493766 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb574005ba0 con 0x7fb578003fb0 2023-05-20T01:26:38.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:38.240+0000 7fb58eb62700 1 -- 172.21.15.16:0/550493766 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb578003fb0 msgr2=0x7fb578004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:38.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:38.240+0000 7fb58eb62700 1 --2- 172.21.15.16:0/550493766 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb578003fb0 0x7fb578004850 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7fb57400c210 tx=0x7fb578005880 comp rx=0 tx=0).stop 2023-05-20T01:26:38.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:38.240+0000 7fb58d0ff700 1 -- 172.21.15.16:0/550493766 reap_dead start 2023-05-20T01:26:38.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:38.240+0000 7fb58eb62700 1 -- 172.21.15.16:0/550493766 shutdown_connections 2023-05-20T01:26:38.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:38.240+0000 7fb58eb62700 1 -- 172.21.15.16:0/550493766 >> 172.21.15.16:0/550493766 conn(0x7fb58806cbb0 msgr2=0x7fb58806cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:38.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:38.240+0000 7fb58eb62700 1 -- 172.21.15.16:0/550493766 shutdown_connections 2023-05-20T01:26:38.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:38.240+0000 7fb58eb62700 1 -- 172.21.15.16:0/550493766 wait complete. 2023-05-20T01:26:38.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:38.241+0000 7fb58eb62700 1 Processor -- start 2023-05-20T01:26:38.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:38.241+0000 7fb58eb62700 1 -- start start 2023-05-20T01:26:38.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:38.242+0000 7fb58eb62700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb578003fb0 0x7fb5880c82e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:38.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:38.242+0000 7fb58eb62700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb58814e3d0 0x7fb5881586b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:38.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:38.242+0000 7fb58eb62700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb588158bf0 0x7fb588159060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:38.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:38.242+0000 7fb58eb62700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5881f2020 con 0x7fb588158bf0 2023-05-20T01:26:38.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:38.242+0000 7fb58eb62700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5880c8be0 con 0x7fb578003fb0 2023-05-20T01:26:38.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:38.242+0000 7fb58eb62700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5880c8ec0 con 0x7fb58814e3d0 2023-05-20T01:26:38.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:38.242+0000 7fb58d0ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb588158bf0 0x7fb588159060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:38.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:38.242+0000 7fb58d0ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb588158bf0 0x7fb588159060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38106/0 (socket says 172.21.15.16:38106) 2023-05-20T01:26:38.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:38.242+0000 7fb58d0ff700 1 -- 172.21.15.16:0/2342276304 learned_addr learned my addr 172.21.15.16:0/2342276304 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:26:38.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:38.242+0000 7fb587fff700 1 --2- 172.21.15.16:0/2342276304 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb58814e3d0 0x7fb5881586b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:38.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:38.242+0000 7fb58c8fe700 1 --2- 172.21.15.16:0/2342276304 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb578003fb0 0x7fb5880c82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:38.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:38.242+0000 7fb58d0ff700 1 -- 172.21.15.16:0/2342276304 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb58814e3d0 msgr2=0x7fb5881586b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:38.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:38.242+0000 7fb58d0ff700 1 --2- 172.21.15.16:0/2342276304 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb58814e3d0 0x7fb5881586b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:38.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:38.242+0000 7fb58d0ff700 1 -- 172.21.15.16:0/2342276304 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb578003fb0 msgr2=0x7fb5880c82e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:38.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:38.242+0000 7fb58d0ff700 1 --2- 172.21.15.16:0/2342276304 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb578003fb0 0x7fb5880c82e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:38.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:38.242+0000 7fb58d0ff700 1 -- 172.21.15.16:0/2342276304 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb58815ad60 con 0x7fb588158bf0 2023-05-20T01:26:38.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:38.242+0000 7fb58c8fe700 1 --2- 172.21.15.16:0/2342276304 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb578003fb0 0x7fb5880c82e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:26:38.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:38.242+0000 7fb58d0ff700 1 --2- 172.21.15.16:0/2342276304 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb588158bf0 0x7fb588159060 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7fb5880c6810 tx=0x7fb5800104a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:38.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:38.242+0000 7fb5857fa700 1 -- 172.21.15.16:0/2342276304 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb58001b070 con 0x7fb588158bf0 2023-05-20T01:26:38.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:38.242+0000 7fb5857fa700 1 -- 172.21.15.16:0/2342276304 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb58000f040 con 0x7fb588158bf0 2023-05-20T01:26:38.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:38.242+0000 7fb58eb62700 1 -- 172.21.15.16:0/2342276304 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb58815aa80 con 0x7fb588158bf0 2023-05-20T01:26:38.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:38.243+0000 7fb5857fa700 1 -- 172.21.15.16:0/2342276304 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb58000e050 con 0x7fb588158bf0 2023-05-20T01:26:38.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:38.243+0000 7fb58eb62700 1 -- 172.21.15.16:0/2342276304 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb588073b00 con 0x7fb588158bf0 2023-05-20T01:26:38.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:38.243+0000 7fb58eb62700 1 -- 172.21.15.16:0/2342276304 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5880cc920 con 0x7fb588158bf0 2023-05-20T01:26:38.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:38.244+0000 7fb5857fa700 1 -- 172.21.15.16:0/2342276304 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb580003a20 con 0x7fb588158bf0 2023-05-20T01:26:38.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:38.244+0000 7fb5857fa700 1 --2- 172.21.15.16:0/2342276304 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb56c06ace0 0x7fb56c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:38.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:38.244+0000 7fb5857fa700 1 -- 172.21.15.16:0/2342276304 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fb580089920 con 0x7fb588158bf0 2023-05-20T01:26:38.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:38.244+0000 7fb58c8fe700 1 --2- 172.21.15.16:0/2342276304 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb56c06ace0 0x7fb56c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:38.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:38.245+0000 7fb58c8fe700 1 --2- 172.21.15.16:0/2342276304 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb56c06ace0 0x7fb56c06d190 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fb57c007fd0 tx=0x7fb57c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:38.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:38.249+0000 7fb5857fa700 1 -- 172.21.15.16:0/2342276304 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fb580054920 con 0x7fb588158bf0 2023-05-20T01:26:38.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:38.492+0000 7fb58eb62700 1 -- 172.21.15.16:0/2342276304 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fb5880c47b0 con 0x7fb588158bf0 2023-05-20T01:26:39.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.085+0000 7fb5857fa700 1 -- 172.21.15.16:0/2342276304 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v78) v1 ==== 122+0+0 (secure 0 0 0) 0x7fb580057f40 con 0x7fb588158bf0 2023-05-20T01:26:39.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.088+0000 7fb58eb62700 1 -- 172.21.15.16:0/2342276304 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb56c06ace0 msgr2=0x7fb56c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:39.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.088+0000 7fb58eb62700 1 --2- 172.21.15.16:0/2342276304 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb56c06ace0 0x7fb56c06d190 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fb57c007fd0 tx=0x7fb57c009040 comp rx=0 tx=0).stop 2023-05-20T01:26:39.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.088+0000 7fb58eb62700 1 -- 172.21.15.16:0/2342276304 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb588158bf0 msgr2=0x7fb588159060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:39.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.088+0000 7fb58eb62700 1 --2- 172.21.15.16:0/2342276304 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb588158bf0 0x7fb588159060 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7fb5880c6810 tx=0x7fb5800104a0 comp rx=0 tx=0).stop 2023-05-20T01:26:39.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.088+0000 7fb58eb62700 1 -- 172.21.15.16:0/2342276304 shutdown_connections 2023-05-20T01:26:39.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.088+0000 7fb58eb62700 1 --2- 172.21.15.16:0/2342276304 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb588158bf0 0x7fb588159060 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:39.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.088+0000 7fb58eb62700 1 --2- 172.21.15.16:0/2342276304 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb578003fb0 0x7fb5880c82e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:39.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.088+0000 7fb58eb62700 1 --2- 172.21.15.16:0/2342276304 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb56c06ace0 0x7fb56c06d190 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:39.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.088+0000 7fb58eb62700 1 --2- 172.21.15.16:0/2342276304 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb58814e3d0 0x7fb5881586b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:39.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.088+0000 7fb58eb62700 1 -- 172.21.15.16:0/2342276304 >> 172.21.15.16:0/2342276304 conn(0x7fb58806cbb0 msgr2=0x7fb5880b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:39.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.088+0000 7fb58eb62700 1 -- 172.21.15.16:0/2342276304 shutdown_connections 2023-05-20T01:26:39.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.088+0000 7fb58eb62700 1 -- 172.21.15.16:0/2342276304 wait complete. 2023-05-20T01:26:39.107 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-05-20T01:26:39.108 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-20T01:26:39.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.235+0000 7f83d4bc2700 1 Processor -- start 2023-05-20T01:26:39.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.235+0000 7f83d4bc2700 1 -- start start 2023-05-20T01:26:39.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.236+0000 7f83d4bc2700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f83d006a510 0x7f83d006a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:39.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.236+0000 7f83d4bc2700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f83d0071140 0x7f83d0071580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:39.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.236+0000 7f83d4bc2700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83d015d020 con 0x7f83d0161190 2023-05-20T01:26:39.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.236+0000 7f83d4bc2700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83d006aed0 con 0x7f83d006a510 2023-05-20T01:26:39.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.236+0000 7f83d4bc2700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83d00cebf0 con 0x7f83d0071140 2023-05-20T01:26:39.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.236+0000 7f83cdd9b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f83d006a510 0x7f83d006a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:39.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.236+0000 7f83cdd9b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f83d006a510 0x7f83d006a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38112/0 (socket says 172.21.15.16:38112) 2023-05-20T01:26:39.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.236+0000 7f83cdd9b700 1 -- 172.21.15.16:0/2868580389 learned_addr learned my addr 172.21.15.16:0/2868580389 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:26:39.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.236+0000 7f83ced9d700 1 --2- 172.21.15.16:0/2868580389 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f83d0071140 0x7f83d0071580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:39.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.236+0000 7f83cdd9b700 1 -- 172.21.15.16:0/2868580389 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f83d0071140 msgr2=0x7f83d0071580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:39.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.236+0000 7f83cdd9b700 1 --2- 172.21.15.16:0/2868580389 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f83d0071140 0x7f83d0071580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:39.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.236+0000 7f83cdd9b700 1 -- 172.21.15.16:0/2868580389 >> v1:172.21.15.16:6789/0 conn(0x7f83d0161190 legacy=0x7f83d006de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:39.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.236+0000 7f83cdd9b700 1 -- 172.21.15.16:0/2868580389 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83d006e9a0 con 0x7f83d006a510 2023-05-20T01:26:39.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.236+0000 7f83ced9d700 1 --2- 172.21.15.16:0/2868580389 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f83d0071140 0x7f83d0071580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:26:39.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.236+0000 7f83cdd9b700 1 --2- 172.21.15.16:0/2868580389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83d006a510 0x7f83d006a990 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f83c40098f0 tx=0x7f83c4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=506105d5e2098ae8 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:39.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.236+0000 7f83cd59a700 1 -- 172.21.15.16:0/2868580389 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83c400b8a0 con 0x7f83d006a510 2023-05-20T01:26:39.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.237+0000 7f83cd59a700 1 -- 172.21.15.16:0/2868580389 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83c400ba00 con 0x7f83d006a510 2023-05-20T01:26:39.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.237+0000 7f83cd59a700 1 -- 172.21.15.16:0/2868580389 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83c400bd90 con 0x7f83d006a510 2023-05-20T01:26:39.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.237+0000 7f83d4bc2700 1 -- 172.21.15.16:0/2868580389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83d006a510 msgr2=0x7f83d006a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:39.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.237+0000 7f83d4bc2700 1 --2- 172.21.15.16:0/2868580389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83d006a510 0x7f83d006a990 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f83c40098f0 tx=0x7f83c4009c00 comp rx=0 tx=0).stop 2023-05-20T01:26:39.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.237+0000 7f83d4bc2700 1 -- 172.21.15.16:0/2868580389 shutdown_connections 2023-05-20T01:26:39.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.237+0000 7f83d4bc2700 1 --2- 172.21.15.16:0/2868580389 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f83d0071140 0x7f83d0071580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:39.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.237+0000 7f83d4bc2700 1 --2- 172.21.15.16:0/2868580389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83d006a510 0x7f83d006a990 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:39.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.237+0000 7f83d4bc2700 1 -- 172.21.15.16:0/2868580389 >> 172.21.15.16:0/2868580389 conn(0x7f83d00d0c70 msgr2=0x7f83d00b1af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:39.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.237+0000 7f83d4bc2700 1 -- 172.21.15.16:0/2868580389 shutdown_connections 2023-05-20T01:26:39.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.237+0000 7f83d4bc2700 1 -- 172.21.15.16:0/2868580389 wait complete. 2023-05-20T01:26:39.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.238+0000 7f83d4bc2700 1 Processor -- start 2023-05-20T01:26:39.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.239+0000 7f83d4bc2700 1 -- start start 2023-05-20T01:26:39.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.239+0000 7f83d4bc2700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f83d006a510 0x7f83d01ea110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:39.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.239+0000 7f83d4bc2700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f83d0071140 0x7f83d00b3f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:39.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.239+0000 7f83d4bc2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83d0161190 0x7f83d01e8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:39.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.239+0000 7f83d4bc2700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83d0145e10 con 0x7f83d0161190 2023-05-20T01:26:39.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.239+0000 7f83d4bc2700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83d00b47e0 con 0x7f83d006a510 2023-05-20T01:26:39.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.239+0000 7f83d4bc2700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83d00b4ac0 con 0x7f83d0071140 2023-05-20T01:26:39.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.239+0000 7f83ced9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83d0161190 0x7f83d01e8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:39.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.239+0000 7f83ced9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83d0161190 0x7f83d01e8dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38120/0 (socket says 172.21.15.16:38120) 2023-05-20T01:26:39.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.239+0000 7f83ced9d700 1 -- 172.21.15.16:0/3733655696 learned_addr learned my addr 172.21.15.16:0/3733655696 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:26:39.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.240+0000 7f83cdd9b700 1 --2- 172.21.15.16:0/3733655696 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f83d0071140 0x7f83d00b3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:39.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.240+0000 7f83ce59c700 1 --2- 172.21.15.16:0/3733655696 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f83d006a510 0x7f83d01ea110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:39.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.240+0000 7f83ced9d700 1 -- 172.21.15.16:0/3733655696 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f83d0071140 msgr2=0x7f83d00b3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:39.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.240+0000 7f83ced9d700 1 --2- 172.21.15.16:0/3733655696 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f83d0071140 0x7f83d00b3f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:39.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.240+0000 7f83ced9d700 1 -- 172.21.15.16:0/3733655696 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f83d006a510 msgr2=0x7f83d01ea110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:39.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.240+0000 7f83ced9d700 1 --2- 172.21.15.16:0/3733655696 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f83d006a510 0x7f83d01ea110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:39.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.240+0000 7f83ced9d700 1 -- 172.21.15.16:0/3733655696 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83c4009550 con 0x7f83d0161190 2023-05-20T01:26:39.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.240+0000 7f83ced9d700 1 --2- 172.21.15.16:0/3733655696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83d0161190 0x7f83d01e8dc0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f83c8007ef0 tx=0x7f83c800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:39.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.240+0000 7f83beffd700 1 -- 172.21.15.16:0/3733655696 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83c800dd80 con 0x7f83d0161190 2023-05-20T01:26:39.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.240+0000 7f83beffd700 1 -- 172.21.15.16:0/3733655696 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83c800f040 con 0x7f83d0161190 2023-05-20T01:26:39.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.240+0000 7f83d4bc2700 1 -- 172.21.15.16:0/3733655696 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83d00cf170 con 0x7f83d0161190 2023-05-20T01:26:39.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.240+0000 7f83beffd700 1 -- 172.21.15.16:0/3733655696 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83c8003c60 con 0x7f83d0161190 2023-05-20T01:26:39.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.240+0000 7f83d4bc2700 1 -- 172.21.15.16:0/3733655696 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83d00cee90 con 0x7f83d0161190 2023-05-20T01:26:39.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.241+0000 7f83d4bc2700 1 -- 172.21.15.16:0/3733655696 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83d00cf450 con 0x7f83d0161190 2023-05-20T01:26:39.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.242+0000 7f83beffd700 1 -- 172.21.15.16:0/3733655696 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f83c8004030 con 0x7f83d0161190 2023-05-20T01:26:39.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.242+0000 7f83beffd700 1 --2- 172.21.15.16:0/3733655696 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f83b006ace0 0x7f83b006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:39.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.242+0000 7f83beffd700 1 -- 172.21.15.16:0/3733655696 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f83c8089660 con 0x7f83d0161190 2023-05-20T01:26:39.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.242+0000 7f83ce59c700 1 --2- 172.21.15.16:0/3733655696 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f83b006ace0 0x7f83b006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:39.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.242+0000 7f83ce59c700 1 --2- 172.21.15.16:0/3733655696 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f83b006ace0 0x7f83b006d190 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f83c0001800 tx=0x7f83c0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:39.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.246+0000 7f83beffd700 1 -- 172.21.15.16:0/3733655696 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f83c80545d0 con 0x7f83d0161190 2023-05-20T01:26:39.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.488+0000 7f83d4bc2700 1 -- 172.21.15.16:0/3733655696 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f83d00747b0 con 0x7f83d0161190 2023-05-20T01:26:39.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.488+0000 7f83beffd700 1 -- 172.21.15.16:0/3733655696 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+4676 (secure 0 0 0) 0x7f83c8057bf0 con 0x7f83d0161190 2023-05-20T01:26:39.493 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:26:39.493 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":78,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7069,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/3619911175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":3619911175},{"type":"v1","addr":"172.21.15.16:6837","nonce":3619911175}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73},{"gid":7081,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/959638003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":959638003},{"type":"v1","addr":"172.21.15.132:6834","nonce":959638003}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73},{"gid":7084,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2596482253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":2596482253},{"type":"v1","addr":"172.21.15.132:6835","nonce":2596482253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-20T01:26:36.067393+0000","modified":"2023-05-20T01:26:39.083145+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7896},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7896":{"gid":7896,"name":"a","rank":0,"incarnation":74,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6836/3765848068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3765848068},{"type":"v1","addr":"172.21.15.16:6836","nonce":3765848068}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-05-20T01:26:39.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.490+0000 7f83d4bc2700 1 -- 172.21.15.16:0/3733655696 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f83b006ace0 msgr2=0x7f83b006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:39.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.490+0000 7f83d4bc2700 1 --2- 172.21.15.16:0/3733655696 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f83b006ace0 0x7f83b006d190 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f83c0001800 tx=0x7f83c0009040 comp rx=0 tx=0).stop 2023-05-20T01:26:39.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.490+0000 7f83d4bc2700 1 -- 172.21.15.16:0/3733655696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83d0161190 msgr2=0x7f83d01e8dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:39.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.490+0000 7f83d4bc2700 1 --2- 172.21.15.16:0/3733655696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83d0161190 0x7f83d01e8dc0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f83c8007ef0 tx=0x7f83c800d3b0 comp rx=0 tx=0).stop 2023-05-20T01:26:39.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.490+0000 7f83d4bc2700 1 -- 172.21.15.16:0/3733655696 shutdown_connections 2023-05-20T01:26:39.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.490+0000 7f83d4bc2700 1 --2- 172.21.15.16:0/3733655696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83d0161190 0x7f83d01e8dc0 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:39.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.491+0000 7f83d4bc2700 1 --2- 172.21.15.16:0/3733655696 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f83d006a510 0x7f83d01ea110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:39.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.491+0000 7f83d4bc2700 1 --2- 172.21.15.16:0/3733655696 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f83b006ace0 0x7f83b006d190 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:39.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.491+0000 7f83d4bc2700 1 --2- 172.21.15.16:0/3733655696 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f83d0071140 0x7f83d00b3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:39.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.491+0000 7f83d4bc2700 1 -- 172.21.15.16:0/3733655696 >> 172.21.15.16:0/3733655696 conn(0x7f83d00d0c70 msgr2=0x7f83d0071e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:39.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.491+0000 7f83d4bc2700 1 -- 172.21.15.16:0/3733655696 shutdown_connections 2023-05-20T01:26:39.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.491+0000 7f83d4bc2700 1 -- 172.21.15.16:0/3733655696 wait complete. 2023-05-20T01:26:39.497 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 78 2023-05-20T01:26:39.507 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-20T01:26:39.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.628+0000 7fa2d41e1700 1 Processor -- start 2023-05-20T01:26:39.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.628+0000 7fa2d41e1700 1 -- start start 2023-05-20T01:26:39.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.629+0000 7fa2d41e1700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa2cc0ccec0 0x7fa2cc0713c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:39.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.629+0000 7fa2d41e1700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2cc0cb590 con 0x7fa2cc157190 2023-05-20T01:26:39.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.629+0000 7fa2d41e1700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2cc0ca9a0 con 0x7fa2cc071900 2023-05-20T01:26:39.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.629+0000 7fa2d41e1700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2cc0cac60 con 0x7fa2cc0ccec0 2023-05-20T01:26:39.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.629+0000 7fa2d1f7d700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fa2cc157190 0x7fa2cc0cc7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49652/0 (socket says 172.21.15.16:49652) 2023-05-20T01:26:39.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.629+0000 7fa2d1f7d700 1 -- 172.21.15.16:0/124022010 learned_addr learned my addr 172.21.15.16:0/124022010 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:26:39.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.629+0000 7fa2d177c700 1 --2- 172.21.15.16:0/124022010 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa2cc0ccec0 0x7fa2cc0713c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:39.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.629+0000 7fa2d0f7b700 1 -- 172.21.15.16:0/124022010 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1826143089 0 0) 0x7fa2cc0cb590 con 0x7fa2cc157190 2023-05-20T01:26:39.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.629+0000 7fa2d0f7b700 1 -- 172.21.15.16:0/124022010 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2b80057c0 con 0x7fa2cc157190 2023-05-20T01:26:39.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.629+0000 7fa2d0f7b700 1 -- 172.21.15.16:0/124022010 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3164476555 0 0) 0x7fa2cc0ca9a0 con 0x7fa2cc071900 2023-05-20T01:26:39.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.629+0000 7fa2d0f7b700 1 -- 172.21.15.16:0/124022010 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2cc0cb590 con 0x7fa2cc071900 2023-05-20T01:26:39.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.630+0000 7fa2d0f7b700 1 -- 172.21.15.16:0/124022010 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2554093447 0 0) 0x7fa2b80057c0 con 0x7fa2cc157190 2023-05-20T01:26:39.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.630+0000 7fa2d0f7b700 1 -- 172.21.15.16:0/124022010 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa2cc0ccec0 msgr2=0x7fa2cc0713c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:39.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.630+0000 7fa2d0f7b700 1 --2- 172.21.15.16:0/124022010 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa2cc0ccec0 0x7fa2cc0713c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:39.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.630+0000 7fa2d0f7b700 1 -- 172.21.15.16:0/124022010 >> v1:172.21.15.132:6789/0 conn(0x7fa2cc071900 legacy=0x7fa2cc0ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:39.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.630+0000 7fa2d0f7b700 1 -- 172.21.15.16:0/124022010 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2cc0caf40 con 0x7fa2cc157190 2023-05-20T01:26:39.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.630+0000 7fa2d0f7b700 1 -- 172.21.15.16:0/124022010 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7fa2c8001670 con 0x7fa2cc157190 2023-05-20T01:26:39.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.630+0000 7fa2d177c700 1 --2- 172.21.15.16:0/124022010 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa2cc0ccec0 0x7fa2cc0713c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-05-20T01:26:39.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.630+0000 7fa2d0f7b700 1 -- 172.21.15.16:0/124022010 >> v1:172.21.15.16:6789/0 conn(0x7fa2cc157190 legacy=0x7fa2cc0cc7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:39.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.630+0000 7fa2d0f7b700 1 --2- 172.21.15.16:0/124022010 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa2b8005e40 0x7fa2b8006230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:39.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.630+0000 7fa2d0f7b700 1 --2- 172.21.15.16:0/124022010 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2b8006a60 0x7fa2b8006ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:39.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.630+0000 7fa2d0f7b700 1 --2- 172.21.15.16:0/124022010 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa2cc0ccec0 0x7fa2b8007480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:39.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.630+0000 7fa2d0f7b700 1 -- 172.21.15.16:0/124022010 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2cc0cac60 con 0x7fa2b8006a60 2023-05-20T01:26:39.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.630+0000 7fa2d0f7b700 1 -- 172.21.15.16:0/124022010 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b800dd30 con 0x7fa2b8005e40 2023-05-20T01:26:39.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.630+0000 7fa2d0f7b700 1 -- 172.21.15.16:0/124022010 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b800def0 con 0x7fa2cc0ccec0 2023-05-20T01:26:39.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.630+0000 7fa2d177c700 1 --2- 172.21.15.16:0/124022010 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2b8006a60 0x7fa2b8006ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:39.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.630+0000 7fa2d1f7d700 1 --2- 172.21.15.16:0/124022010 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa2b8005e40 0x7fa2b8006230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:39.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.630+0000 7fa2d277e700 1 --2- 172.21.15.16:0/124022010 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa2cc0ccec0 0x7fa2b8007480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:39.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.630+0000 7fa2d177c700 1 -- 172.21.15.16:0/124022010 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa2cc0ccec0 msgr2=0x7fa2b8007480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:39.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.630+0000 7fa2d177c700 1 --2- 172.21.15.16:0/124022010 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa2cc0ccec0 0x7fa2b8007480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:39.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.630+0000 7fa2d177c700 1 -- 172.21.15.16:0/124022010 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa2b8005e40 msgr2=0x7fa2b8006230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:39.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.630+0000 7fa2d177c700 1 --2- 172.21.15.16:0/124022010 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa2b8005e40 0x7fa2b8006230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:39.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.630+0000 7fa2d177c700 1 -- 172.21.15.16:0/124022010 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2bc0094a0 con 0x7fa2b8006a60 2023-05-20T01:26:39.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.630+0000 7fa2d177c700 1 --2- 172.21.15.16:0/124022010 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2b8006a60 0x7fa2b8006ed0 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc009d30 tx=0x7fa2b8007a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27eb4ab7f1406d6f server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:39.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.630+0000 7fa2d0f7b700 1 -- 172.21.15.16:0/124022010 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2bc009e80 con 0x7fa2b8006a60 2023-05-20T01:26:39.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.630+0000 7fa2d0f7b700 1 -- 172.21.15.16:0/124022010 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2bc00e9a0 con 0x7fa2b8006a60 2023-05-20T01:26:39.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.630+0000 7fa2d0f7b700 1 -- 172.21.15.16:0/124022010 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2bc00ed10 con 0x7fa2b8006a60 2023-05-20T01:26:39.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.631+0000 7fa2d41e1700 1 -- 172.21.15.16:0/124022010 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2b8006a60 msgr2=0x7fa2b8006ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:39.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.631+0000 7fa2d41e1700 1 --2- 172.21.15.16:0/124022010 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2b8006a60 0x7fa2b8006ed0 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc009d30 tx=0x7fa2b8007a90 comp rx=0 tx=0).stop 2023-05-20T01:26:39.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.631+0000 7fa2d277e700 1 -- 172.21.15.16:0/124022010 reap_dead start 2023-05-20T01:26:39.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.631+0000 7fa2d41e1700 1 -- 172.21.15.16:0/124022010 shutdown_connections 2023-05-20T01:26:39.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.631+0000 7fa2d41e1700 1 -- 172.21.15.16:0/124022010 >> 172.21.15.16:0/124022010 conn(0x7fa2cc0cffb0 msgr2=0x7fa2cc0d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:39.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.631+0000 7fa2d41e1700 1 -- 172.21.15.16:0/124022010 shutdown_connections 2023-05-20T01:26:39.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.631+0000 7fa2d41e1700 1 -- 172.21.15.16:0/124022010 wait complete. 2023-05-20T01:26:39.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.632+0000 7fa2d41e1700 1 Processor -- start 2023-05-20T01:26:39.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.632+0000 7fa2d41e1700 1 -- start start 2023-05-20T01:26:39.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.633+0000 7fa2d41e1700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa2b8006a60 0x7fa2cc0b5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:39.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.633+0000 7fa2d41e1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2cc0ac8a0 0x7fa2cc0afa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:39.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.633+0000 7fa2d41e1700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa2cc15b8c0 0x7fa2cc1540f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:39.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.633+0000 7fa2d41e1700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2cc0b3cf0 con 0x7fa2cc0ac8a0 2023-05-20T01:26:39.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.633+0000 7fa2d41e1700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2cc1e46d0 con 0x7fa2cc15b8c0 2023-05-20T01:26:39.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.633+0000 7fa2d41e1700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2cc1e4150 con 0x7fa2b8006a60 2023-05-20T01:26:39.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.633+0000 7fa2d177c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2cc0ac8a0 0x7fa2cc0afa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:39.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.633+0000 7fa2d177c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2cc0ac8a0 0x7fa2cc0afa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38138/0 (socket says 172.21.15.16:38138) 2023-05-20T01:26:39.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.633+0000 7fa2d177c700 1 -- 172.21.15.16:0/530108749 learned_addr learned my addr 172.21.15.16:0/530108749 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:26:39.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.633+0000 7fa2d277e700 1 --2- 172.21.15.16:0/530108749 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa2cc15b8c0 0x7fa2cc1540f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:39.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.633+0000 7fa2d1f7d700 1 --2- 172.21.15.16:0/530108749 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa2b8006a60 0x7fa2cc0b5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:39.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.633+0000 7fa2d177c700 1 -- 172.21.15.16:0/530108749 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa2b8006a60 msgr2=0x7fa2cc0b5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:39.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.633+0000 7fa2d177c700 1 --2- 172.21.15.16:0/530108749 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa2b8006a60 0x7fa2cc0b5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:39.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.633+0000 7fa2d177c700 1 -- 172.21.15.16:0/530108749 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa2cc15b8c0 msgr2=0x7fa2cc1540f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:39.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.633+0000 7fa2d177c700 1 --2- 172.21.15.16:0/530108749 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa2cc15b8c0 0x7fa2cc1540f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:39.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.633+0000 7fa2d177c700 1 -- 172.21.15.16:0/530108749 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2cc157560 con 0x7fa2cc0ac8a0 2023-05-20T01:26:39.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.634+0000 7fa2d177c700 1 --2- 172.21.15.16:0/530108749 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2cc0ac8a0 0x7fa2cc0afa90 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc0090b0 tx=0x7fa2bc0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:39.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.634+0000 7fa2c27fc700 1 -- 172.21.15.16:0/530108749 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2bc003ab0 con 0x7fa2cc0ac8a0 2023-05-20T01:26:39.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.634+0000 7fa2c27fc700 1 -- 172.21.15.16:0/530108749 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2bc003c10 con 0x7fa2cc0ac8a0 2023-05-20T01:26:39.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.634+0000 7fa2d41e1700 1 -- 172.21.15.16:0/530108749 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2cc0cb220 con 0x7fa2cc0ac8a0 2023-05-20T01:26:39.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.634+0000 7fa2c27fc700 1 -- 172.21.15.16:0/530108749 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2bc009e80 con 0x7fa2cc0ac8a0 2023-05-20T01:26:39.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.634+0000 7fa2d41e1700 1 -- 172.21.15.16:0/530108749 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2cc0cba20 con 0x7fa2cc0ac8a0 2023-05-20T01:26:39.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.635+0000 7fa2d41e1700 1 -- 172.21.15.16:0/530108749 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2cc0b3ea0 con 0x7fa2cc0ac8a0 2023-05-20T01:26:39.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.635+0000 7fa2c27fc700 1 -- 172.21.15.16:0/530108749 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa2bc013470 con 0x7fa2cc0ac8a0 2023-05-20T01:26:39.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.636+0000 7fa2c27fc700 1 --2- 172.21.15.16:0/530108749 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa2b006ad10 0x7fa2b006d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:39.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.636+0000 7fa2c27fc700 1 -- 172.21.15.16:0/530108749 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fa2bc08b260 con 0x7fa2cc0ac8a0 2023-05-20T01:26:39.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.636+0000 7fa2d1f7d700 1 --2- 172.21.15.16:0/530108749 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa2b006ad10 0x7fa2b006d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:39.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.636+0000 7fa2d1f7d700 1 --2- 172.21.15.16:0/530108749 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa2b006ad10 0x7fa2b006d1c0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fa2c8001eb0 tx=0x7fa2c8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:39.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.640+0000 7fa2c27fc700 1 -- 172.21.15.16:0/530108749 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fa2cc0b3ea0 con 0x7fa2cc0ac8a0 2023-05-20T01:26:39.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.838+0000 7fa2d41e1700 1 -- 172.21.15.16:0/530108749 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa2cc0747b0 con 0x7fa2cc0ac8a0 2023-05-20T01:26:39.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.840+0000 7fa2c27fc700 1 -- 172.21.15.16:0/530108749 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v112) v1 ==== 74+0+18404 (secure 0 0 0) 0x7fa2bc0561e0 con 0x7fa2cc0ac8a0 2023-05-20T01:26:39.844 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:26:39.845 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":112,"fsid":"c2245cc5-ecab-4087-a59b-bc81db901676","created":"2023-05-20T01:16:09.549230+0000","modified":"2023-05-20T01:26:37.069211+0000","last_up_change":"2023-05-20T01:16:19.788624+0000","last_in_change":"2023-05-20T01:16:12.092160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-05-20T01:16:20.934366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-05-20T01:16:21.372294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-05-20T01:26:34.067017+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-05-20T01:26:35.040602+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"317a83cd-19c8-4958-a301-6087ef342289","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6825","nonce":3923634179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6827","nonce":3923634179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6831","nonce":3923634179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6829","nonce":3923634179}]},"public_addr":"172.21.15.16:6825/3923634179","cluster_addr":"172.21.15.16:6827/3923634179","heartbeat_back_addr":"172.21.15.16:6831/3923634179","heartbeat_front_addr":"172.21.15.16:6829/3923634179","state":["exists","up"]},{"osd":1,"uuid":"5581046f-8219-4089-8d7d-b7846399505e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6809","nonce":3860264220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6811","nonce":3860264220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6815","nonce":3860264220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6813","nonce":3860264220}]},"public_addr":"172.21.15.16:6809/3860264220","cluster_addr":"172.21.15.16:6811/3860264220","heartbeat_back_addr":"172.21.15.16:6815/3860264220","heartbeat_front_addr":"172.21.15.16:6813/3860264220","state":["exists","up"]},{"osd":2,"uuid":"072e9e66-4133-478e-8d4a-64d46c2963f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6817","nonce":293187030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6819","nonce":293187030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6823","nonce":293187030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6821","nonce":293187030}]},"public_addr":"172.21.15.16:6817/293187030","cluster_addr":"172.21.15.16:6819/293187030","heartbeat_back_addr":"172.21.15.16:6823/293187030","heartbeat_front_addr":"172.21.15.16:6821/293187030","state":["exists","up"]},{"osd":3,"uuid":"11e1bba6-00f8-48dd-9028-d8d716b7f145","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6801","nonce":3669781890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6803","nonce":3669781890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6807","nonce":3669781890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6805","nonce":3669781890}]},"public_addr":"172.21.15.16:6801/3669781890","cluster_addr":"172.21.15.16:6803/3669781890","heartbeat_back_addr":"172.21.15.16:6807/3669781890","heartbeat_front_addr":"172.21.15.16:6805/3669781890","state":["exists","up"]},{"osd":4,"uuid":"b8909c3a-cf70-46cb-8d0b-009c707cf9d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6819","nonce":506317223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6823","nonce":506317223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6829","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6831","nonce":506317223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6827","nonce":506317223}]},"public_addr":"172.21.15.132:6819/506317223","cluster_addr":"172.21.15.132:6823/506317223","heartbeat_back_addr":"172.21.15.132:6831/506317223","heartbeat_front_addr":"172.21.15.132:6827/506317223","state":["exists","up"]},{"osd":5,"uuid":"b1818304-4583-47bf-a6ec-0bfbb5166233","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6818","nonce":1844689995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6822","nonce":1844689995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6830","nonce":1844689995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6826","nonce":1844689995}]},"public_addr":"172.21.15.132:6818/1844689995","cluster_addr":"172.21.15.132:6822/1844689995","heartbeat_back_addr":"172.21.15.132:6830/1844689995","heartbeat_front_addr":"172.21.15.132:6826/1844689995","state":["exists","up"]},{"osd":6,"uuid":"d37f4691-eed1-4a93-8818-18354f03f4a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6801","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6802","nonce":727581123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6805","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6807","nonce":727581123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6813","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6815","nonce":727581123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6811","nonce":727581123}]},"public_addr":"172.21.15.132:6802/727581123","cluster_addr":"172.21.15.132:6807/727581123","heartbeat_back_addr":"172.21.15.132:6815/727581123","heartbeat_front_addr":"172.21.15.132:6811/727581123","state":["exists","up"]},{"osd":7,"uuid":"4f712f62-5d1d-4816-8042-03209144f8b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6803","nonce":466001149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6806","nonce":466001149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6814","nonce":466001149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6810","nonce":466001149}]},"public_addr":"172.21.15.132:6803/466001149","cluster_addr":"172.21.15.132:6806/466001149","heartbeat_back_addr":"172.21.15.132:6814/466001149","heartbeat_front_addr":"172.21.15.132:6810/466001149","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.338863+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.278680+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305694+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.246835+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.276237+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.304441+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305421+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.267050+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-20T01:26:39.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.842+0000 7fa2d41e1700 1 -- 172.21.15.16:0/530108749 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa2b006ad10 msgr2=0x7fa2b006d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:39.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.842+0000 7fa2d41e1700 1 --2- 172.21.15.16:0/530108749 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa2b006ad10 0x7fa2b006d1c0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fa2c8001eb0 tx=0x7fa2c8009040 comp rx=0 tx=0).stop 2023-05-20T01:26:39.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.842+0000 7fa2d41e1700 1 -- 172.21.15.16:0/530108749 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2cc0ac8a0 msgr2=0x7fa2cc0afa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:39.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.842+0000 7fa2d41e1700 1 --2- 172.21.15.16:0/530108749 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2cc0ac8a0 0x7fa2cc0afa90 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc0090b0 tx=0x7fa2bc0047c0 comp rx=0 tx=0).stop 2023-05-20T01:26:39.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.842+0000 7fa2d41e1700 1 -- 172.21.15.16:0/530108749 shutdown_connections 2023-05-20T01:26:39.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.842+0000 7fa2d41e1700 1 --2- 172.21.15.16:0/530108749 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa2cc15b8c0 0x7fa2cc1540f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:39.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.842+0000 7fa2d41e1700 1 --2- 172.21.15.16:0/530108749 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa2b006ad10 0x7fa2b006d1c0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:39.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.842+0000 7fa2d41e1700 1 --2- 172.21.15.16:0/530108749 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa2b8006a60 0x7fa2cc0b5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:39.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.842+0000 7fa2d41e1700 1 --2- 172.21.15.16:0/530108749 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2cc0ac8a0 0x7fa2cc0afa90 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:39.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.842+0000 7fa2d41e1700 1 -- 172.21.15.16:0/530108749 >> 172.21.15.16:0/530108749 conn(0x7fa2cc0cffb0 msgr2=0x7fa2cc06aea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:39.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.842+0000 7fa2d41e1700 1 -- 172.21.15.16:0/530108749 shutdown_connections 2023-05-20T01:26:39.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.842+0000 7fa2d41e1700 1 -- 172.21.15.16:0/530108749 wait complete. 2023-05-20T01:26:39.859 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-05-20T01:26:39.859 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-05-20T01:26:39.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.979+0000 7f003aca9700 1 Processor -- start 2023-05-20T01:26:39.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.979+0000 7f003aca9700 1 -- start start 2023-05-20T01:26:39.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.980+0000 7f003aca9700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f00340ccec0 0x7f00340713c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:39.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.980+0000 7f003aca9700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0034071900 0x7f00340ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:39.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.980+0000 7f003aca9700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00340cb590 con 0x7f0034157190 2023-05-20T01:26:39.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.980+0000 7f003aca9700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00340ca9a0 con 0x7f0034071900 2023-05-20T01:26:39.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.980+0000 7f003aca9700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00340cac60 con 0x7f00340ccec0 2023-05-20T01:26:39.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.980+0000 7f0039246700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0034071900 0x7f00340ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:39.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.980+0000 7f0038a45700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f0034157190 0x7f00340cc7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49672/0 (socket says 172.21.15.16:49672) 2023-05-20T01:26:39.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.980+0000 7f0038a45700 1 -- 172.21.15.16:0/4201254467 learned_addr learned my addr 172.21.15.16:0/4201254467 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:26:39.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.981+0000 7f0033fff700 1 --2- 172.21.15.16:0/4201254467 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f00340ccec0 0x7f00340713c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:39.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.981+0000 7f0039246700 1 -- 172.21.15.16:0/4201254467 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f00340ccec0 msgr2=0x7f00340713c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:39.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.981+0000 7f0039246700 1 --2- 172.21.15.16:0/4201254467 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f00340ccec0 0x7f00340713c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:39.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.981+0000 7f0039246700 1 -- 172.21.15.16:0/4201254467 >> v1:172.21.15.16:6789/0 conn(0x7f0034157190 legacy=0x7f00340cc7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:39.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.981+0000 7f0039246700 1 -- 172.21.15.16:0/4201254467 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00340caf40 con 0x7f0034071900 2023-05-20T01:26:39.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.981+0000 7f00337fe700 1 -- 172.21.15.16:0/4201254467 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2435701216 0 0) 0x7f00340cb590 con 0x7f0034157190 2023-05-20T01:26:39.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.981+0000 7f0033fff700 1 --2- 172.21.15.16:0/4201254467 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f00340ccec0 0x7f00340713c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:26:39.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.981+0000 7f0039246700 1 --2- 172.21.15.16:0/4201254467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0034071900 0x7f00340ce740 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f002c00bfa0 tx=0x7f002c00d330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b673f88215d4fd9 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:39.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.981+0000 7f00337fe700 1 -- 172.21.15.16:0/4201254467 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f002c010070 con 0x7f0034071900 2023-05-20T01:26:39.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.981+0000 7f00337fe700 1 -- 172.21.15.16:0/4201254467 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f002c00b2e0 con 0x7f0034071900 2023-05-20T01:26:39.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.981+0000 7f00337fe700 1 -- 172.21.15.16:0/4201254467 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f002c00dcf0 con 0x7f0034071900 2023-05-20T01:26:39.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.982+0000 7f003aca9700 1 -- 172.21.15.16:0/4201254467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0034071900 msgr2=0x7f00340ce740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:39.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.982+0000 7f003aca9700 1 --2- 172.21.15.16:0/4201254467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0034071900 0x7f00340ce740 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f002c00bfa0 tx=0x7f002c00d330 comp rx=0 tx=0).stop 2023-05-20T01:26:39.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.982+0000 7f003aca9700 1 -- 172.21.15.16:0/4201254467 shutdown_connections 2023-05-20T01:26:39.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.982+0000 7f003aca9700 1 --2- 172.21.15.16:0/4201254467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0034071900 0x7f00340ce740 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:39.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.982+0000 7f003aca9700 1 --2- 172.21.15.16:0/4201254467 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f00340ccec0 0x7f00340713c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:39.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.982+0000 7f003aca9700 1 -- 172.21.15.16:0/4201254467 >> 172.21.15.16:0/4201254467 conn(0x7f00340cffb0 msgr2=0x7f00340d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:39.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.982+0000 7f003aca9700 1 -- 172.21.15.16:0/4201254467 shutdown_connections 2023-05-20T01:26:39.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.982+0000 7f003aca9700 1 -- 172.21.15.16:0/4201254467 wait complete. 2023-05-20T01:26:39.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.983+0000 7f003aca9700 1 Processor -- start 2023-05-20T01:26:39.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.983+0000 7f003aca9700 1 -- start start 2023-05-20T01:26:39.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.984+0000 7f003aca9700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0034071900 0x7f00340aefe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:39.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.984+0000 7f003aca9700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00340ccec0 0x7f00340b5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:39.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.984+0000 7f003aca9700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0034157190 0x7f003406ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:39.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.984+0000 7f003aca9700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00340b3620 con 0x7f00340ccec0 2023-05-20T01:26:39.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.984+0000 7f003aca9700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f003414c5d0 con 0x7f0034071900 2023-05-20T01:26:39.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.984+0000 7f003aca9700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00340b3340 con 0x7f0034157190 2023-05-20T01:26:39.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.984+0000 7f0033fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00340ccec0 0x7f00340b5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:39.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.984+0000 7f0033fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00340ccec0 0x7f00340b5550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38152/0 (socket says 172.21.15.16:38152) 2023-05-20T01:26:39.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.984+0000 7f0033fff700 1 -- 172.21.15.16:0/1127106070 learned_addr learned my addr 172.21.15.16:0/1127106070 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:26:39.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.984+0000 7f0033fff700 1 -- 172.21.15.16:0/1127106070 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0034157190 msgr2=0x7f003406ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:39.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.984+0000 7f0033fff700 1 --2- 172.21.15.16:0/1127106070 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0034157190 0x7f003406ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:39.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.984+0000 7f0033fff700 1 -- 172.21.15.16:0/1127106070 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0034071900 msgr2=0x7f00340aefe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:39.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.984+0000 7f0033fff700 1 --2- 172.21.15.16:0/1127106070 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0034071900 0x7f00340aefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:39.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.984+0000 7f0033fff700 1 -- 172.21.15.16:0/1127106070 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f002c00bc80 con 0x7f00340ccec0 2023-05-20T01:26:39.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.984+0000 7f0033fff700 1 --2- 172.21.15.16:0/1127106070 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00340ccec0 0x7f00340b5550 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f002000c750 tx=0x7f002000cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:39.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.984+0000 7f00317fa700 1 -- 172.21.15.16:0/1127106070 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0020011800 con 0x7f00340ccec0 2023-05-20T01:26:39.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.984+0000 7f00317fa700 1 -- 172.21.15.16:0/1127106070 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0020011960 con 0x7f00340ccec0 2023-05-20T01:26:39.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.984+0000 7f003aca9700 1 -- 172.21.15.16:0/1127106070 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0034157560 con 0x7f00340ccec0 2023-05-20T01:26:39.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.984+0000 7f00317fa700 1 -- 172.21.15.16:0/1127106070 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0020011cd0 con 0x7f00340ccec0 2023-05-20T01:26:39.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.985+0000 7f003aca9700 1 -- 172.21.15.16:0/1127106070 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00340cb220 con 0x7f00340ccec0 2023-05-20T01:26:39.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.985+0000 7f003aca9700 1 -- 172.21.15.16:0/1127106070 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00340cba20 con 0x7f00340ccec0 2023-05-20T01:26:40.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.986+0000 7f00317fa700 1 -- 172.21.15.16:0/1127106070 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0020023480 con 0x7f00340ccec0 2023-05-20T01:26:40.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.987+0000 7f00317fa700 1 --2- 172.21.15.16:0/1127106070 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f001806ace0 0x7f001806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:40.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.987+0000 7f00317fa700 1 -- 172.21.15.16:0/1127106070 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f00200898c0 con 0x7f00340ccec0 2023-05-20T01:26:40.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.987+0000 7f0038a45700 1 --2- 172.21.15.16:0/1127106070 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f001806ace0 0x7f001806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:40.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.987+0000 7f0038a45700 1 --2- 172.21.15.16:0/1127106070 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f001806ace0 0x7f001806d190 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f0028001610 tx=0x7f00280079f0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:40.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:39.990+0000 7f00317fa700 1 -- 172.21.15.16:0/1127106070 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f00200549a0 con 0x7f00340ccec0 2023-05-20T01:26:40.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.186+0000 7f003aca9700 1 -- 172.21.15.16:0/1127106070 --> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f003414c740 con 0x7f001806ace0 2023-05-20T01:26:40.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.194+0000 7f00317fa700 1 -- 172.21.15.16:0/1127106070 <== mgr.4104 v2:172.21.15.16:6832/3817237998 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+299805 (secure 0 0 0) 0x7f003414c740 con 0x7f001806ace0 2023-05-20T01:26:40.202 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:26:40.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.198+0000 7f003aca9700 1 -- 172.21.15.16:0/1127106070 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f001806ace0 msgr2=0x7f001806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:40.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.198+0000 7f003aca9700 1 --2- 172.21.15.16:0/1127106070 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f001806ace0 0x7f001806d190 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f0028001610 tx=0x7f00280079f0 comp rx=0 tx=0).stop 2023-05-20T01:26:40.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.198+0000 7f003aca9700 1 -- 172.21.15.16:0/1127106070 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00340ccec0 msgr2=0x7f00340b5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:40.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.198+0000 7f003aca9700 1 --2- 172.21.15.16:0/1127106070 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00340ccec0 0x7f00340b5550 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f002000c750 tx=0x7f002000cb10 comp rx=0 tx=0).stop 2023-05-20T01:26:40.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.198+0000 7f003aca9700 1 -- 172.21.15.16:0/1127106070 shutdown_connections 2023-05-20T01:26:40.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.198+0000 7f003aca9700 1 --2- 172.21.15.16:0/1127106070 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f001806ace0 0x7f001806d190 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:40.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.198+0000 7f003aca9700 1 --2- 172.21.15.16:0/1127106070 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0034157190 0x7f003406ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:40.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.198+0000 7f003aca9700 1 --2- 172.21.15.16:0/1127106070 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0034071900 0x7f00340aefe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:40.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.198+0000 7f003aca9700 1 --2- 172.21.15.16:0/1127106070 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00340ccec0 0x7f00340b5550 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:40.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.198+0000 7f003aca9700 1 -- 172.21.15.16:0/1127106070 >> 172.21.15.16:0/1127106070 conn(0x7f00340cffb0 msgr2=0x7f003406ad60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:40.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.198+0000 7f003aca9700 1 -- 172.21.15.16:0/1127106070 shutdown_connections 2023-05-20T01:26:40.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.198+0000 7f003aca9700 1 -- 172.21.15.16:0/1127106070 wait complete. 2023-05-20T01:26:40.206 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2023-05-20T01:26:40.214 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":415,"stamp":"2023-05-20T01:26:39.454792+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":30,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":110204,"kb_used_data":11644,"kb_used_omap":0,"kb_used_meta":98496,"kb_avail":754864516,"statfs":{"total":773094113280,"available":772981264384,"internally_reserved":0,"allocated":11923456,"data_stored":6674952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":100859904},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.819728"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-05-20T01:26:34.638110+0000","last_change":"2023-05-20T01:26:34.638110+0000","last_active":"2023-05-20T01:26:34.630120+0000","last_peered":"2023-05-20T01:26:34.630120+0000","last_clean":"2023-05-20T01:26:34.630120+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:34.638110+0000","last_undegraded":"2023-05-20T01:26:34.638110+0000","last_fullsized":"2023-05-20T01:26:34.638110+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-05-20T01:26:34.639497+0000","last_change":"2023-05-20T01:26:34.639497+0000","last_active":"2023-05-20T01:26:34.630120+0000","last_peered":"2023-05-20T01:26:34.630120+0000","last_clean":"2023-05-20T01:26:34.630120+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:34.639497+0000","last_undegraded":"2023-05-20T01:26:34.639497+0000","last_fullsized":"2023-05-20T01:26:34.639497+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-05-20T01:26:34.644602+0000","last_change":"2023-05-20T01:26:34.644602+0000","last_active":"2023-05-20T01:26:34.630120+0000","last_peered":"2023-05-20T01:26:34.630120+0000","last_clean":"2023-05-20T01:26:34.630120+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:34.644602+0000","last_undegraded":"2023-05-20T01:26:34.644602+0000","last_fullsized":"2023-05-20T01:26:34.644602+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-05-20T01:26:34.642067+0000","last_change":"2023-05-20T01:26:34.642067+0000","last_active":"2023-05-20T01:26:34.630120+0000","last_peered":"2023-05-20T01:26:34.630120+0000","last_clean":"2023-05-20T01:26:34.630120+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:34.642067+0000","last_undegraded":"2023-05-20T01:26:34.642067+0000","last_fullsized":"2023-05-20T01:26:34.642067+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-05-20T01:26:34.640655+0000","last_change":"2023-05-20T01:26:34.640655+0000","last_active":"2023-05-20T01:26:34.630120+0000","last_peered":"2023-05-20T01:26:34.630120+0000","last_clean":"2023-05-20T01:26:34.630120+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:34.640655+0000","last_undegraded":"2023-05-20T01:26:34.640655+0000","last_fullsized":"2023-05-20T01:26:34.640655+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-05-20T01:26:34.641569+0000","last_change":"2023-05-20T01:26:34.641569+0000","last_active":"2023-05-20T01:26:34.630120+0000","last_peered":"2023-05-20T01:26:34.630120+0000","last_clean":"2023-05-20T01:26:34.630120+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:34.641569+0000","last_undegraded":"2023-05-20T01:26:34.641569+0000","last_fullsized":"2023-05-20T01:26:34.641569+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-05-20T01:26:34.644015+0000","last_change":"2023-05-20T01:26:34.644015+0000","last_active":"2023-05-20T01:26:34.630120+0000","last_peered":"2023-05-20T01:26:34.630120+0000","last_clean":"2023-05-20T01:26:34.630120+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:34.644015+0000","last_undegraded":"2023-05-20T01:26:34.644015+0000","last_fullsized":"2023-05-20T01:26:34.644015+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-05-20T01:26:34.638480+0000","last_change":"2023-05-20T01:26:34.638480+0000","last_active":"2023-05-20T01:26:34.630120+0000","last_peered":"2023-05-20T01:26:34.630120+0000","last_clean":"2023-05-20T01:26:34.630120+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:34.638480+0000","last_undegraded":"2023-05-20T01:26:34.638480+0000","last_fullsized":"2023-05-20T01:26:34.638480+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-05-20T01:26:34.644724+0000","last_change":"2023-05-20T01:26:34.644724+0000","last_active":"2023-05-20T01:26:34.630120+0000","last_peered":"2023-05-20T01:26:34.630120+0000","last_clean":"2023-05-20T01:26:34.630120+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:34.644724+0000","last_undegraded":"2023-05-20T01:26:34.644724+0000","last_fullsized":"2023-05-20T01:26:34.644724+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-05-20T01:26:34.639019+0000","last_change":"2023-05-20T01:26:34.639019+0000","last_active":"2023-05-20T01:26:34.630120+0000","last_peered":"2023-05-20T01:26:34.630120+0000","last_clean":"2023-05-20T01:26:34.630120+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:34.639019+0000","last_undegraded":"2023-05-20T01:26:34.639019+0000","last_fullsized":"2023-05-20T01:26:34.639019+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-05-20T01:26:34.641727+0000","last_change":"2023-05-20T01:26:34.641727+0000","last_active":"2023-05-20T01:26:34.630120+0000","last_peered":"2023-05-20T01:26:34.630120+0000","last_clean":"2023-05-20T01:26:34.630120+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:34.641727+0000","last_undegraded":"2023-05-20T01:26:34.641727+0000","last_fullsized":"2023-05-20T01:26:34.641727+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-05-20T01:26:34.640629+0000","last_change":"2023-05-20T01:26:34.640629+0000","last_active":"2023-05-20T01:26:34.630120+0000","last_peered":"2023-05-20T01:26:34.630120+0000","last_clean":"2023-05-20T01:26:34.630120+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:34.640629+0000","last_undegraded":"2023-05-20T01:26:34.640629+0000","last_fullsized":"2023-05-20T01:26:34.640629+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-05-20T01:26:34.641472+0000","last_change":"2023-05-20T01:26:34.641472+0000","last_active":"2023-05-20T01:26:34.630120+0000","last_peered":"2023-05-20T01:26:34.630120+0000","last_clean":"2023-05-20T01:26:34.630120+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:34.641472+0000","last_undegraded":"2023-05-20T01:26:34.641472+0000","last_fullsized":"2023-05-20T01:26:34.641472+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-05-20T01:26:34.638486+0000","last_change":"2023-05-20T01:26:34.638486+0000","last_active":"2023-05-20T01:26:34.630120+0000","last_peered":"2023-05-20T01:26:34.630120+0000","last_clean":"2023-05-20T01:26:34.630120+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:34.638486+0000","last_undegraded":"2023-05-20T01:26:34.638486+0000","last_fullsized":"2023-05-20T01:26:34.638486+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-05-20T01:26:34.638266+0000","last_change":"2023-05-20T01:26:34.638266+0000","last_active":"2023-05-20T01:26:34.630120+0000","last_peered":"2023-05-20T01:26:34.630120+0000","last_clean":"2023-05-20T01:26:34.630120+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:34.638266+0000","last_undegraded":"2023-05-20T01:26:34.638266+0000","last_fullsized":"2023-05-20T01:26:34.638266+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-05-20T01:26:34.643913+0000","last_change":"2023-05-20T01:26:34.643913+0000","last_active":"2023-05-20T01:26:34.630120+0000","last_peered":"2023-05-20T01:26:34.630120+0000","last_clean":"2023-05-20T01:26:34.630120+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:34.643913+0000","last_undegraded":"2023-05-20T01:26:34.643913+0000","last_fullsized":"2023-05-20T01:26:34.643913+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-05-20T01:26:34.641466+0000","last_change":"2023-05-20T01:26:34.641466+0000","last_active":"2023-05-20T01:26:34.630120+0000","last_peered":"2023-05-20T01:26:34.630120+0000","last_clean":"2023-05-20T01:26:34.630120+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:34.641466+0000","last_undegraded":"2023-05-20T01:26:34.641466+0000","last_fullsized":"2023-05-20T01:26:34.641466+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-05-20T01:26:34.638503+0000","last_change":"2023-05-20T01:26:34.638503+0000","last_active":"2023-05-20T01:26:34.630120+0000","last_peered":"2023-05-20T01:26:34.630120+0000","last_clean":"2023-05-20T01:26:34.630120+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:34.638503+0000","last_undegraded":"2023-05-20T01:26:34.638503+0000","last_fullsized":"2023-05-20T01:26:34.638503+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-05-20T01:26:34.640567+0000","last_change":"2023-05-20T01:26:34.640567+0000","last_active":"2023-05-20T01:26:34.630120+0000","last_peered":"2023-05-20T01:26:34.630120+0000","last_clean":"2023-05-20T01:26:34.630120+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:34.640567+0000","last_undegraded":"2023-05-20T01:26:34.640567+0000","last_fullsized":"2023-05-20T01:26:34.640567+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-05-20T01:26:34.641446+0000","last_change":"2023-05-20T01:26:34.641446+0000","last_active":"2023-05-20T01:26:34.630120+0000","last_peered":"2023-05-20T01:26:34.630120+0000","last_clean":"2023-05-20T01:26:34.630120+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:34.641446+0000","last_undegraded":"2023-05-20T01:26:34.641446+0000","last_fullsized":"2023-05-20T01:26:34.641446+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-05-20T01:26:34.644021+0000","last_change":"2023-05-20T01:26:34.644021+0000","last_active":"2023-05-20T01:26:34.630120+0000","last_peered":"2023-05-20T01:26:34.630120+0000","last_clean":"2023-05-20T01:26:34.630120+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:34.644021+0000","last_undegraded":"2023-05-20T01:26:34.644021+0000","last_fullsized":"2023-05-20T01:26:34.644021+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-05-20T01:26:34.638401+0000","last_change":"2023-05-20T01:26:34.638401+0000","last_active":"2023-05-20T01:26:34.630120+0000","last_peered":"2023-05-20T01:26:34.630120+0000","last_clean":"2023-05-20T01:26:34.630120+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:34.638401+0000","last_undegraded":"2023-05-20T01:26:34.638401+0000","last_fullsized":"2023-05-20T01:26:34.638401+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-05-20T01:26:34.640200+0000","last_change":"2023-05-20T01:26:34.640200+0000","last_active":"2023-05-20T01:26:34.630120+0000","last_peered":"2023-05-20T01:26:34.630120+0000","last_clean":"2023-05-20T01:26:34.630120+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:34.640200+0000","last_undegraded":"2023-05-20T01:26:34.640200+0000","last_fullsized":"2023-05-20T01:26:34.640200+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-05-20T01:26:34.638339+0000","last_change":"2023-05-20T01:26:34.638339+0000","last_active":"2023-05-20T01:26:34.630120+0000","last_peered":"2023-05-20T01:26:34.630120+0000","last_clean":"2023-05-20T01:26:34.630120+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:34.638339+0000","last_undegraded":"2023-05-20T01:26:34.638339+0000","last_fullsized":"2023-05-20T01:26:34.638339+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-05-20T01:26:34.642905+0000","last_change":"2023-05-20T01:26:34.642905+0000","last_active":"2023-05-20T01:26:34.630120+0000","last_peered":"2023-05-20T01:26:34.630120+0000","last_clean":"2023-05-20T01:26:34.630120+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:34.642905+0000","last_undegraded":"2023-05-20T01:26:34.642905+0000","last_fullsized":"2023-05-20T01:26:34.642905+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":191,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-05-20T01:26:34.645291+0000","last_change":"2023-05-20T01:16:23.289068+0000","last_active":"2023-05-20T01:26:34.645291+0000","last_peered":"2023-05-20T01:26:34.645291+0000","last_clean":"2023-05-20T01:26:34.645291+0000","last_became_active":"2023-05-20T01:16:23.288781+0000","last_became_peered":"2023-05-20T01:16:23.288781+0000","last_unstale":"2023-05-20T01:26:34.645291+0000","last_undegraded":"2023-05-20T01:26:34.645291+0000","last_fullsized":"2023-05-20T01:26:34.645291+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:23:26.310250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-05-20T01:26:34.639380+0000","last_change":"2023-05-20T01:26:34.639380+0000","last_active":"2023-05-20T01:26:34.630120+0000","last_peered":"2023-05-20T01:26:34.630120+0000","last_clean":"2023-05-20T01:26:34.630120+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:34.639380+0000","last_undegraded":"2023-05-20T01:26:34.639380+0000","last_fullsized":"2023-05-20T01:26:34.639380+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":199,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-05-20T01:26:34.637011+0000","last_change":"2023-05-20T01:16:22.829324+0000","last_active":"2023-05-20T01:26:34.637011+0000","last_peered":"2023-05-20T01:26:34.637011+0000","last_clean":"2023-05-20T01:26:34.637011+0000","last_became_active":"2023-05-20T01:16:22.828964+0000","last_became_peered":"2023-05-20T01:16:22.828964+0000","last_unstale":"2023-05-20T01:26:34.637011+0000","last_undegraded":"2023-05-20T01:26:34.637011+0000","last_fullsized":"2023-05-20T01:26:34.637011+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:23:48.805679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-05-20T01:26:34.641191+0000","last_change":"2023-05-20T01:26:34.641191+0000","last_active":"2023-05-20T01:26:34.630120+0000","last_peered":"2023-05-20T01:26:34.630120+0000","last_clean":"2023-05-20T01:26:34.630120+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:34.641191+0000","last_undegraded":"2023-05-20T01:26:34.641191+0000","last_fullsized":"2023-05-20T01:26:34.641191+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":175,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-05-20T01:26:34.642808+0000","last_change":"2023-05-20T01:16:22.834754+0000","last_active":"2023-05-20T01:26:34.642808+0000","last_peered":"2023-05-20T01:26:34.642808+0000","last_clean":"2023-05-20T01:26:34.642808+0000","last_became_active":"2023-05-20T01:16:22.834437+0000","last_became_peered":"2023-05-20T01:16:22.834437+0000","last_unstale":"2023-05-20T01:26:34.642808+0000","last_undegraded":"2023-05-20T01:26:34.642808+0000","last_fullsized":"2023-05-20T01:26:34.642808+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T01:22:38.911204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":310,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-05-20T01:26:34.642091+0000","last_change":"2023-05-20T01:16:22.834920+0000","last_active":"2023-05-20T01:26:34.642091+0000","last_peered":"2023-05-20T01:26:34.642091+0000","last_clean":"2023-05-20T01:26:34.642091+0000","last_became_active":"2023-05-20T01:16:22.833924+0000","last_became_peered":"2023-05-20T01:16:22.833924+0000","last_unstale":"2023-05-20T01:26:34.642091+0000","last_undegraded":"2023-05-20T01:26:34.642091+0000","last_fullsized":"2023-05-20T01:26:34.642091+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:02:13.281571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-05-20T01:26:34.642061+0000","last_change":"2023-05-20T01:26:34.642061+0000","last_active":"2023-05-20T01:26:34.630120+0000","last_peered":"2023-05-20T01:26:34.630120+0000","last_clean":"2023-05-20T01:26:34.630120+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:34.642061+0000","last_undegraded":"2023-05-20T01:26:34.642061+0000","last_fullsized":"2023-05-20T01:26:34.642061+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":159,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-05-20T01:26:34.642202+0000","last_change":"2023-05-20T01:16:23.290259+0000","last_active":"2023-05-20T01:26:34.642202+0000","last_peered":"2023-05-20T01:26:34.642202+0000","last_clean":"2023-05-20T01:26:34.642202+0000","last_became_active":"2023-05-20T01:16:23.289911+0000","last_became_peered":"2023-05-20T01:16:23.289911+0000","last_unstale":"2023-05-20T01:26:34.642202+0000","last_undegraded":"2023-05-20T01:26:34.642202+0000","last_fullsized":"2023-05-20T01:26:34.642202+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:40:10.726971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-05-20T01:26:34.638510+0000","last_change":"2023-05-20T01:26:34.638510+0000","last_active":"2023-05-20T01:26:34.630120+0000","last_peered":"2023-05-20T01:26:34.630120+0000","last_clean":"2023-05-20T01:26:34.630120+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:34.638510+0000","last_undegraded":"2023-05-20T01:26:34.638510+0000","last_fullsized":"2023-05-20T01:26:34.638510+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":160,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-05-20T01:26:34.642159+0000","last_change":"2023-05-20T01:16:23.290207+0000","last_active":"2023-05-20T01:26:34.642159+0000","last_peered":"2023-05-20T01:26:34.642159+0000","last_clean":"2023-05-20T01:26:34.642159+0000","last_became_active":"2023-05-20T01:16:23.289928+0000","last_became_peered":"2023-05-20T01:16:23.289928+0000","last_unstale":"2023-05-20T01:26:34.642159+0000","last_undegraded":"2023-05-20T01:26:34.642159+0000","last_fullsized":"2023-05-20T01:26:34.642159+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:05:09.399167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-05-20T01:26:34.640347+0000","last_change":"2023-05-20T01:26:34.640347+0000","last_active":"2023-05-20T01:26:34.630120+0000","last_peered":"2023-05-20T01:26:34.630120+0000","last_clean":"2023-05-20T01:26:34.630120+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:34.640347+0000","last_undegraded":"2023-05-20T01:26:34.640347+0000","last_fullsized":"2023-05-20T01:26:34.640347+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":161,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-05-20T01:26:34.642350+0000","last_change":"2023-05-20T01:16:23.290538+0000","last_active":"2023-05-20T01:26:34.642350+0000","last_peered":"2023-05-20T01:26:34.642350+0000","last_clean":"2023-05-20T01:26:34.642350+0000","last_became_active":"2023-05-20T01:16:23.290230+0000","last_became_peered":"2023-05-20T01:16:23.290230+0000","last_unstale":"2023-05-20T01:26:34.642350+0000","last_undegraded":"2023-05-20T01:26:34.642350+0000","last_fullsized":"2023-05-20T01:26:34.642350+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:47:13.453162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-05-20T01:26:34.638985+0000","last_change":"2023-05-20T01:26:34.638985+0000","last_active":"2023-05-20T01:26:34.630120+0000","last_peered":"2023-05-20T01:26:34.630120+0000","last_clean":"2023-05-20T01:26:34.630120+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:34.638985+0000","last_undegraded":"2023-05-20T01:26:34.638985+0000","last_fullsized":"2023-05-20T01:26:34.638985+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":175,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-05-20T01:26:34.641407+0000","last_change":"2023-05-20T01:16:23.290341+0000","last_active":"2023-05-20T01:26:34.641407+0000","last_peered":"2023-05-20T01:26:34.641407+0000","last_clean":"2023-05-20T01:26:34.641407+0000","last_became_active":"2023-05-20T01:16:23.290053+0000","last_became_peered":"2023-05-20T01:16:23.290053+0000","last_unstale":"2023-05-20T01:26:34.641407+0000","last_undegraded":"2023-05-20T01:26:34.641407+0000","last_fullsized":"2023-05-20T01:26:34.641407+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:17:08.936381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-05-20T01:26:34.640797+0000","last_change":"2023-05-20T01:26:34.640797+0000","last_active":"2023-05-20T01:26:34.630120+0000","last_peered":"2023-05-20T01:26:34.630120+0000","last_clean":"2023-05-20T01:26:34.630120+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:34.640797+0000","last_undegraded":"2023-05-20T01:26:34.640797+0000","last_fullsized":"2023-05-20T01:26:34.640797+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":175,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-05-20T01:26:34.641451+0000","last_change":"2023-05-20T01:16:22.834330+0000","last_active":"2023-05-20T01:26:34.641451+0000","last_peered":"2023-05-20T01:26:34.641451+0000","last_clean":"2023-05-20T01:26:34.641451+0000","last_became_active":"2023-05-20T01:16:22.833946+0000","last_became_peered":"2023-05-20T01:16:22.833946+0000","last_unstale":"2023-05-20T01:26:34.641451+0000","last_undegraded":"2023-05-20T01:26:34.641451+0000","last_fullsized":"2023-05-20T01:26:34.641451+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:23:50.042871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574975,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":16432,"kb_used_data":1896,"kb_used_omap":0,"kb_used_meta":14528,"kb_avail":94355408,"statfs":{"total":96636764160,"available":96619937792,"internally_reserved":0,"allocated":1941504,"data_stored":1279819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":14876672},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:25:37 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.48599999999999999,"15min":0.51400000000000001},"min":{"1min":0.30199999999999999,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.88200000000000001},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.51000000000000001,"15min":0.54700000000000004},"min":{"1min":0.39200000000000002,"5min":0.309,"15min":0.30599999999999999},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.90000000000000002},"last":0.64500000000000002}]},{"osd":1,"last update":"Sat May 20 01:25:37 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.497,"15min":0.72199999999999998},"min":{"1min":0.371,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":8.8879999999999999},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.53200000000000003,"15min":0.76000000000000001},"min":{"1min":0.44800000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":8.875},"last":0.753}]},{"osd":2,"last update":"Sat May 20 01:25:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.502,"15min":0.70899999999999996},"min":{"1min":0.32600000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":8.9120000000000008},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.50800000000000001,"15min":0.71399999999999997},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":8.8520000000000003},"last":0.58799999999999997}]},{"osd":3,"last update":"Sat May 20 01:25:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.498,"15min":0.76100000000000001},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":10.884},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.55400000000000005,"15min":0.81699999999999995},"min":{"1min":0.40699999999999997,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":10.952},"last":0.65500000000000003}]},{"osd":4,"last update":"Sat May 20 01:25:37 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.51000000000000001,"15min":0.51200000000000001},"min":{"1min":0.35899999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.52900000000000003,"15min":0.53100000000000003},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.875},"last":0.53000000000000003}]},{"osd":5,"last update":"Sat May 20 01:25:37 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.56499999999999995,"15min":0.54700000000000004},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.56999999999999995,"15min":0.57799999999999996},"min":{"1min":0.39700000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.46500000000000002}]},{"osd":6,"last update":"Sat May 20 01:25:37 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.59499999999999997,"15min":0.59799999999999998},"min":{"1min":0.41999999999999998,"5min":0.41599999999999998,"15min":0.378},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.57199999999999995,"15min":0.58499999999999996},"min":{"1min":0.39000000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.47999999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574974,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":11116,"kb_used_data":1316,"kb_used_omap":0,"kb_used_meta":9792,"kb_avail":94360724,"statfs":{"total":96636764160,"available":96625381376,"internally_reserved":0,"allocated":1347584,"data_stored":689451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":10027008},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:25:42 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.44800000000000001,"15min":0.47699999999999998},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.26900000000000002},"max":{"1min":0.60999999999999999,"5min":0.72699999999999998,"15min":0.77200000000000002},"last":0.44},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.47899999999999998,"15min":0.51700000000000002},"min":{"1min":0.32900000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.504}]},{"osd":1,"last update":"Sat May 20 01:25:42 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.48699999999999999,"15min":0.51900000000000002},"min":{"1min":0.32800000000000001,"5min":0.29699999999999999,"15min":0.28000000000000003},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.83799999999999997},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.51700000000000002,"15min":0.52600000000000002},"min":{"1min":0.379,"5min":0.32600000000000001,"15min":0.318},"max":{"1min":0.66500000000000004,"5min":0.749,"15min":0.79200000000000004},"last":0.42199999999999999}]},{"osd":2,"last update":"Sat May 20 01:25:42 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.48399999999999999,"15min":0.503},"min":{"1min":0.35099999999999998,"5min":0.34399999999999997,"15min":0.33700000000000002},"max":{"1min":0.57099999999999995,"5min":0.72199999999999998,"15min":0.76600000000000001},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.51600000000000001,"15min":0.54900000000000004},"min":{"1min":0.37,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.745,"5min":0.76200000000000001,"15min":0.76800000000000002},"last":0.53800000000000003}]},{"osd":3,"last update":"Sat May 20 01:25:42 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.52500000000000002,"15min":0.53600000000000003},"min":{"1min":0.378,"5min":0.311,"15min":0.311},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.52000000000000002,"15min":0.53900000000000003},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.72699999999999998,"5min":0.73199999999999998,"15min":0.79500000000000004},"last":0.51600000000000001}]},{"osd":4,"last update":"Sat May 20 01:25:42 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.53100000000000003,"15min":0.54400000000000004},"min":{"1min":0.34799999999999998,"5min":0.33300000000000002,"15min":0.32600000000000001},"max":{"1min":0.70699999999999996,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.54000000000000004,"15min":0.56299999999999994},"min":{"1min":0.40500000000000003,"5min":0.38100000000000001,"15min":0.32700000000000001},"max":{"1min":0.71799999999999997,"5min":0.77400000000000002,"15min":0.80100000000000005},"last":0.58999999999999997}]},{"osd":5,"last update":"Sat May 20 01:25:42 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.53400000000000003,"15min":0.54000000000000004},"min":{"1min":0.38,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.63600000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.56499999999999995,"15min":0.59799999999999998},"min":{"1min":0.372,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.64400000000000002,"5min":0.81599999999999995,"15min":0.89400000000000002},"last":0.57599999999999996}]},{"osd":7,"last update":"Sat May 20 01:25:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.57099999999999995,"15min":0.59399999999999997},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.71299999999999997,"5min":0.80200000000000005,"15min":0.92900000000000005},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.57599999999999996,"15min":0.60499999999999998},"min":{"1min":0.45100000000000001,"5min":0.379,"15min":0.379},"max":{"1min":0.66600000000000004,"5min":0.82799999999999996,"15min":0.85199999999999998},"last":0.60299999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574974,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":17264,"kb_used_data":1896,"kb_used_omap":0,"kb_used_meta":15360,"kb_avail":94354576,"statfs":{"total":96636764160,"available":96619085824,"internally_reserved":0,"allocated":1941504,"data_stored":1279819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":15728640},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:25:36 2023","interfaces":[{"interface":"back","average":{"1min":0.30599999999999999,"5min":0.28299999999999997,"15min":0.311},"min":{"1min":0.18099999999999999,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":0.48799999999999999,"5min":0.57099999999999995,"15min":1.194},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.32400000000000001,"15min":0.31900000000000001},"min":{"1min":0.23899999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.75800000000000001},"last":0.40600000000000003}]},{"osd":2,"last update":"Sat May 20 01:25:36 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.307,"15min":0.31900000000000001},"min":{"1min":0.23100000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.66400000000000003},"last":0.442},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.374,"15min":0.40400000000000003},"min":{"1min":0.313,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":1.3580000000000001},"last":0.52400000000000002}]},{"osd":3,"last update":"Sat May 20 01:25:36 2023","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.35899999999999999,"15min":0.371},"min":{"1min":0.27400000000000002,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":1.619},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.35299999999999998,"15min":0.379},"min":{"1min":0.25800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.57699999999999996,"5min":0.60899999999999999,"15min":1.8400000000000001},"last":0.497}]},{"osd":4,"last update":"Sat May 20 01:25:36 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.55600000000000005,"15min":0.57899999999999996},"min":{"1min":0.36799999999999999,"5min":0.34899999999999998,"15min":0.32200000000000001},"max":{"1min":0.76100000000000001,"5min":0.81699999999999995,"15min":1.2769999999999999},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.57099999999999995,"15min":0.58299999999999996},"min":{"1min":0.38,"5min":0.312,"15min":0.312},"max":{"1min":0.81599999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.63600000000000001}]},{"osd":5,"last update":"Sat May 20 01:25:36 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.58699999999999997,"15min":0.59099999999999997},"min":{"1min":0.46600000000000003,"5min":0.37,"15min":0.37},"max":{"1min":0.78500000000000003,"5min":0.81899999999999995,"15min":1.4510000000000001},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.58199999999999996,"15min":0.60299999999999998},"min":{"1min":0.39700000000000002,"5min":0.39100000000000001,"15min":0.378},"max":{"1min":0.76800000000000002,"5min":0.78800000000000003,"15min":1.409},"last":0.59999999999999998}]},{"osd":6,"last update":"Sat May 20 01:25:36 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.60299999999999998},"min":{"1min":0.39000000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.82299999999999995,"5min":0.876,"15min":1.575},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.58199999999999996,"15min":0.59399999999999997},"min":{"1min":0.45400000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":1.0600000000000001},"last":0.61399999999999999}]},{"osd":7,"last update":"Sat May 20 01:25:36 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.60399999999999998},"min":{"1min":0.41099999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":1.496},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.63,"15min":0.64000000000000001},"min":{"1min":0.44500000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.5009999999999999,"5min":2.1709999999999998,"15min":2.1709999999999998},"last":0.66900000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574974,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":11176,"kb_used_data":1312,"kb_used_omap":0,"kb_used_meta":9856,"kb_avail":94360664,"statfs":{"total":96636764160,"available":96625319936,"internally_reserved":0,"allocated":1343488,"data_stored":688490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":10092544},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat May 20 01:25:35 2023","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.29799999999999999,"15min":0.32000000000000001},"min":{"1min":0.188,"5min":0.17999999999999999,"15min":0.17999999999999999},"max":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.314,"15min":0.34399999999999997},"min":{"1min":0.23999999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":1.502,"5min":1.502,"15min":1.502},"last":0.23400000000000001}]},{"osd":2,"last update":"Sat May 20 01:25:35 2023","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.30299999999999999,"15min":0.32800000000000001},"min":{"1min":0.20899999999999999,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.55700000000000005},"last":0.28299999999999997},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.36199999999999999,"15min":0.38500000000000001},"min":{"1min":0.29599999999999999,"5min":0.216,"15min":0.216},"max":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.314}]},{"osd":3,"last update":"Sat May 20 01:25:35 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.36399999999999999,"15min":0.37},"min":{"1min":0.23499999999999999,"5min":0.22,"15min":0.22},"max":{"1min":4.258,"5min":4.258,"15min":4.258},"last":0.27100000000000002},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.35199999999999998,"15min":0.36599999999999999},"min":{"1min":0.25700000000000001,"5min":0.245,"15min":0.245},"max":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.60899999999999999},"last":0.35899999999999999}]},{"osd":4,"last update":"Sat May 20 01:25:35 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.57199999999999995,"15min":0.55500000000000005},"min":{"1min":0.373,"5min":0.372,"15min":0.34399999999999997},"max":{"1min":1.4330000000000001,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.58499999999999996,"15min":0.57199999999999995},"min":{"1min":0.42799999999999999,"5min":0.39200000000000002,"15min":0.36299999999999999},"max":{"1min":1.298,"5min":1.298,"15min":1.298},"last":0.61499999999999999}]},{"osd":5,"last update":"Sat May 20 01:25:35 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.57399999999999995,"15min":0.57699999999999996},"min":{"1min":0.40600000000000003,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.79900000000000004,"5min":0.80500000000000005,"15min":0.88100000000000001},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.60099999999999998,"15min":0.59399999999999997},"min":{"1min":0.41299999999999998,"5min":0.374,"15min":0.374},"max":{"1min":1.4710000000000001,"5min":1.4710000000000001,"15min":4.4720000000000004},"last":0.65700000000000003}]},{"osd":6,"last update":"Sat May 20 01:25:35 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.68200000000000005,"15min":0.625},"min":{"1min":0.36899999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.79800000000000004,"5min":10.042999999999999,"15min":10.042999999999999},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.60599999999999998,"15min":0.61699999999999999},"min":{"1min":0.40200000000000002,"5min":0.377,"15min":0.377},"max":{"1min":1.367,"5min":1.367,"15min":1.367},"last":0.59099999999999997}]},{"osd":7,"last update":"Sat May 20 01:25:35 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.60699999999999998,"15min":0.59799999999999998},"min":{"1min":0.443,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.399,"5min":1.399,"15min":1.399},"last":0.628},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.72699999999999998,"15min":0.65800000000000003},"min":{"1min":0.439,"5min":0.439,"15min":0.42099999999999999},"max":{"1min":1.4490000000000001,"5min":10.085000000000001,"15min":10.085000000000001},"last":0.56699999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574974,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":11180,"kb_used_data":1316,"kb_used_omap":0,"kb_used_meta":9856,"kb_avail":94360660,"statfs":{"total":96636764160,"available":96625315840,"internally_reserved":0,"allocated":1347584,"data_stored":689451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":10092544},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.33700000000000002,"5min":0.28299999999999997,"15min":0.32300000000000001},"min":{"1min":0.224,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":0.50900000000000001,"5min":0.53400000000000003,"15min":1.038},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.32300000000000001,"5min":0.29899999999999999,"15min":0.33900000000000002},"min":{"1min":0.21199999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":1.079},"last":0.26200000000000001}]},{"osd":1,"last update":"Sat May 20 01:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.317,"15min":0.376},"min":{"1min":0.253,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.44500000000000001,"5min":0.61399999999999999,"15min":1.4950000000000001},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.35899999999999999,"15min":0.41299999999999998},"min":{"1min":0.28399999999999997,"5min":0.249,"15min":0.249},"max":{"1min":0.60799999999999998,"5min":0.629,"15min":1.5189999999999999},"last":0.33000000000000002}]},{"osd":3,"last update":"Sat May 20 01:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.32900000000000001,"15min":0.378},"min":{"1min":0.246,"5min":0.218,"15min":0.216},"max":{"1min":0.57899999999999996,"5min":0.60599999999999998,"15min":1.2090000000000001},"last":0.379},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.36899999999999999,"15min":0.42699999999999999},"min":{"1min":0.32400000000000001,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.59499999999999997,"5min":0.68100000000000005,"15min":1.5620000000000001},"last":0.35299999999999998}]},{"osd":4,"last update":"Sat May 20 01:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.56799999999999995,"15min":0.58099999999999996},"min":{"1min":0.46500000000000002,"5min":0.35899999999999999,"15min":0.34899999999999998},"max":{"1min":1.107,"5min":1.107,"15min":1.319},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.59399999999999997,"15min":0.61299999999999999},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":1.4319999999999999},"last":0.70999999999999996}]},{"osd":5,"last update":"Sat May 20 01:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.56799999999999995,"15min":0.61699999999999999},"min":{"1min":0.443,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.82199999999999995,"5min":0.95599999999999996,"15min":1.758},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.58899999999999997,"15min":0.63500000000000001},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.004,"5min":1.004,"15min":1.7110000000000001},"last":0.65400000000000003}]},{"osd":6,"last update":"Sat May 20 01:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.59099999999999997,"15min":0.63200000000000001},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.069,"5min":1.069,"15min":1.665},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.58499999999999996,"15min":0.628},"min":{"1min":0.39900000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.88,"5min":0.93500000000000005,"15min":1.819},"last":0.75}]},{"osd":7,"last update":"Sat May 20 01:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.59999999999999998,"15min":0.65200000000000002},"min":{"1min":0.39000000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":1.7310000000000001},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.60999999999999999,"15min":0.65100000000000002},"min":{"1min":0.47799999999999998,"5min":0.42399999999999999,"15min":0.42199999999999999},"max":{"1min":1.038,"5min":1.038,"15min":1.8460000000000001},"last":0.77700000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574974,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":15456,"kb_used_data":1304,"kb_used_omap":0,"kb_used_meta":14144,"kb_avail":94356384,"statfs":{"total":96636764160,"available":96620937216,"internally_reserved":0,"allocated":1335296,"data_stored":682961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":14483456},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:26:34 2023","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.316,"15min":0.31900000000000001},"min":{"1min":0.24099999999999999,"5min":0.20100000000000001,"15min":0.188},"max":{"1min":0.54100000000000004,"5min":0.65000000000000002,"15min":1.528},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.34200000000000003,"15min":0.40799999999999997},"min":{"1min":0.26500000000000001,"5min":0.20799999999999999,"15min":0.19500000000000001},"max":{"1min":0.54200000000000004,"5min":0.72399999999999998,"15min":3.1890000000000001},"last":0.54200000000000004}]},{"osd":1,"last update":"Sat May 20 01:26:34 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.32500000000000001,"15min":0.40000000000000002},"min":{"1min":0.22700000000000001,"5min":0.189,"15min":0.189},"max":{"1min":0.55900000000000005,"5min":0.73199999999999998,"15min":3.4260000000000002},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.437,"5min":0.372,"15min":0.36099999999999999},"min":{"1min":0.27700000000000002,"5min":0.23100000000000001,"15min":0.221},"max":{"1min":0.60999999999999999,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.60999999999999999}]},{"osd":2,"last update":"Sat May 20 01:26:34 2023","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.36399999999999999,"15min":0.38400000000000001},"min":{"1min":0.30399999999999999,"5min":0.24299999999999999,"15min":0.224},"max":{"1min":0.59399999999999997,"5min":0.67400000000000004,"15min":1.1619999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.40000000000000002,"15min":0.41299999999999998},"min":{"1min":0.26900000000000002,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.75,"5min":0.78500000000000003,"15min":1.413},"last":0.64900000000000002}]},{"osd":4,"last update":"Sat May 20 01:26:34 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.56999999999999995,"15min":0.58299999999999996},"min":{"1min":0.45800000000000002,"5min":0.38700000000000001,"15min":0.38200000000000001},"max":{"1min":0.77000000000000002,"5min":0.95399999999999996,"15min":1.0700000000000001},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.59199999999999997,"15min":0.66100000000000003},"min":{"1min":0.42399999999999999,"5min":0.40200000000000002,"15min":0.35899999999999999},"max":{"1min":0.88500000000000001,"5min":0.88600000000000001,"15min":3.4660000000000002},"last":0.879}]},{"osd":5,"last update":"Sat May 20 01:26:34 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.59099999999999997,"15min":0.66600000000000004},"min":{"1min":0.503,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":3.5379999999999998},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.60799999999999998,"15min":0.626},"min":{"1min":0.51400000000000001,"5min":0.41999999999999998,"15min":0.374},"max":{"1min":0.91300000000000003,"5min":1.026,"15min":1.385},"last":0.89700000000000002}]},{"osd":6,"last update":"Sat May 20 01:26:34 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.60699999999999998,"15min":0.629},"min":{"1min":0.44400000000000001,"5min":0.41199999999999998,"15min":0.38100000000000001},"max":{"1min":0.873,"5min":0.99199999999999999,"15min":1.347},"last":0.873},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.60899999999999999,"15min":0.626},"min":{"1min":0.438,"5min":0.374,"15min":0.374},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":1.107},"last":0.94999999999999996}]},{"osd":7,"last update":"Sat May 20 01:26:34 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.60999999999999999,"15min":0.63200000000000001},"min":{"1min":0.46700000000000003,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.92200000000000004,"5min":0.92900000000000005,"15min":1.1359999999999999},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.63700000000000001,"15min":0.70699999999999996},"min":{"1min":0.52300000000000002,"5min":0.436,"15min":0.436},"max":{"1min":0.92300000000000004,"5min":1.0069999999999999,"15min":3.5009999999999999},"last":0.91500000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574974,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":16544,"kb_used_data":1304,"kb_used_omap":0,"kb_used_meta":15232,"kb_avail":94355296,"statfs":{"total":96636764160,"available":96619823104,"internally_reserved":0,"allocated":1335296,"data_stored":682961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":15597568},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:25:33 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.47499999999999998,"15min":0.48199999999999998},"min":{"1min":0.372,"5min":0.247,"15min":0.247},"max":{"1min":0.68799999999999994,"5min":0.73699999999999999,"15min":0.75600000000000001},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.504,"5min":0.503,"15min":0.499},"min":{"1min":0.35299999999999998,"5min":0.30599999999999999,"15min":0.30099999999999999},"max":{"1min":0.71199999999999997,"5min":1.6830000000000001,"15min":1.7989999999999999},"last":0.67300000000000004}]},{"osd":1,"last update":"Sat May 20 01:25:33 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.499,"15min":0.51800000000000002},"min":{"1min":0.35499999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.68400000000000005,"5min":0.746,"15min":0.82799999999999996},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.51700000000000002,"15min":0.52800000000000002},"min":{"1min":0.36799999999999999,"5min":0.316,"15min":0.316},"max":{"1min":0.66500000000000004,"5min":1.7190000000000001,"15min":2.0739999999999998},"last":0.66100000000000003}]},{"osd":2,"last update":"Sat May 20 01:25:33 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.53300000000000003,"15min":0.51200000000000001},"min":{"1min":0.38800000000000001,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53800000000000003,"15min":0.54900000000000004},"min":{"1min":0.42799999999999999,"5min":0.36299999999999999,"15min":0.34000000000000002},"max":{"1min":0.73199999999999998,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.59299999999999997}]},{"osd":3,"last update":"Sat May 20 01:25:33 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.52000000000000002,"15min":0.52200000000000002},"min":{"1min":0.36499999999999999,"5min":0.34899999999999998,"15min":0.34399999999999997},"max":{"1min":0.63600000000000001,"5min":0.81399999999999995,"15min":1.827},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.57599999999999996,"15min":0.64700000000000002},"min":{"1min":0.433,"5min":0.38,"15min":0.38},"max":{"1min":1.093,"5min":1.093,"15min":3.1749999999999998},"last":0.55100000000000005}]},{"osd":5,"last update":"Sat May 20 01:25:33 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.53300000000000003,"15min":0.48999999999999999},"min":{"1min":0.32900000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.76700000000000002,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.505,"5min":0.54700000000000004,"15min":0.51400000000000001},"min":{"1min":0.34699999999999998,"5min":0.33400000000000002,"15min":0.29899999999999999},"max":{"1min":0.70099999999999996,"5min":0.80400000000000005,"15min":0.80500000000000005},"last":0.50800000000000001}]},{"osd":6,"last update":"Sat May 20 01:25:33 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58999999999999997,"15min":0.55100000000000005},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.32400000000000001},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.60699999999999998,"15min":0.59199999999999997},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.80100000000000005,"5min":0.84699999999999998,"15min":0.85899999999999999},"last":0.53100000000000003}]},{"osd":7,"last update":"Sat May 20 01:25:33 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.58099999999999996,"15min":0.55700000000000005},"min":{"1min":0.40899999999999997,"5min":0.36099999999999999,"15min":0.34200000000000003},"max":{"1min":0.67100000000000004,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.626,"5min":0.61799999999999999,"15min":0.59999999999999998},"min":{"1min":0.436,"5min":0.36399999999999999,"15min":0.34000000000000002},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.58299999999999996}]}]},{"osd":5,"up_from":13,"seq":55834574975,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":11036,"kb_used_data":1300,"kb_used_omap":0,"kb_used_meta":9728,"kb_avail":94360804,"statfs":{"total":96636764160,"available":96625463296,"internally_reserved":0,"allocated":1331200,"data_stored":682000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9961472},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:25:38 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.45500000000000002,"15min":0.63300000000000001},"min":{"1min":0.32500000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":6.3929999999999998},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.504,"5min":0.46899999999999997,"15min":0.65600000000000003},"min":{"1min":0.379,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":6.3339999999999996},"last":0.53900000000000003}]},{"osd":1,"last update":"Sat May 20 01:25:38 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.46700000000000003,"15min":0.57499999999999996},"min":{"1min":0.33800000000000002,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":3.2450000000000001},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.51100000000000001,"15min":0.53500000000000003},"min":{"1min":0.371,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":1.1819999999999999},"last":0.51100000000000001}]},{"osd":2,"last update":"Sat May 20 01:25:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.48899999999999999,"15min":0.53700000000000003},"min":{"1min":0.40999999999999998,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.751,"5min":0.751,"15min":1.1639999999999999},"last":0.624},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.49199999999999999,"15min":0.54200000000000004},"min":{"1min":0.39200000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":1.258},"last":0.61399999999999999}]},{"osd":3,"last update":"Sat May 20 01:25:38 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.51700000000000002,"15min":0.70199999999999996},"min":{"1min":0.44500000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":6.351},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.54300000000000004,"15min":0.73399999999999999},"min":{"1min":0.39100000000000001,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":6.3730000000000002},"last":0.58199999999999996}]},{"osd":4,"last update":"Sat May 20 01:25:38 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.51900000000000002,"15min":0.496},"min":{"1min":0.433,"5min":0.309,"15min":0.309},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.70199999999999996},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.41299999999999998,"5min":0.35399999999999998,"15min":0.34200000000000003},"max":{"1min":0.73999999999999999,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.59899999999999998}]},{"osd":6,"last update":"Sat May 20 01:25:38 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.29999999999999999},"max":{"1min":0.77600000000000002,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.55100000000000005,"15min":0.55800000000000005},"min":{"1min":0.432,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.76600000000000001,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.65500000000000003}]},{"osd":7,"last update":"Sat May 20 01:25:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.57799999999999996,"15min":0.58199999999999996},"min":{"1min":0.44,"5min":0.33400000000000002,"15min":0.33000000000000002},"max":{"1min":0.79200000000000004,"5min":0.82299999999999995,"15min":0.91000000000000003},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.59299999999999997,"15min":0.60799999999999998},"min":{"1min":0.53400000000000003,"5min":0.40999999999999998,"15min":0.39500000000000002},"max":{"1min":0.81100000000000005,"5min":0.81399999999999995,"15min":0.94499999999999995},"last":0.71099999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-05-20T01:26:40.221 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-05-20T01:26:40.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.375+0000 7f30c5f72700 1 Processor -- start 2023-05-20T01:26:40.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.376+0000 7f30c5f72700 1 -- start start 2023-05-20T01:26:40.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.376+0000 7f30c5f72700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f30c00ccec0 0x7f30c00713c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:40.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.376+0000 7f30c5f72700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30c00cb590 con 0x7f30c0071900 2023-05-20T01:26:40.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.376+0000 7f30c5f72700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30c00ca9a0 con 0x7f30c0157190 2023-05-20T01:26:40.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.376+0000 7f30c5f72700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30c00cac60 con 0x7f30c00ccec0 2023-05-20T01:26:40.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.376+0000 7f30beffd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f30c00ccec0 0x7f30c00713c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:40.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.376+0000 7f30beffd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f30c00ccec0 0x7f30c00713c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38160/0 (socket says 172.21.15.16:38160) 2023-05-20T01:26:40.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.376+0000 7f30beffd700 1 -- 172.21.15.16:0/1502839892 learned_addr learned my addr 172.21.15.16:0/1502839892 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:26:40.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.377+0000 7f30beffd700 1 -- 172.21.15.16:0/1502839892 >> v1:172.21.15.132:6789/0 conn(0x7f30c0157190 legacy=0x7f30c00cc7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:40.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.377+0000 7f30beffd700 1 -- 172.21.15.16:0/1502839892 >> v1:172.21.15.16:6789/0 conn(0x7f30c0071900 legacy=0x7f30c00ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:40.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.377+0000 7f30beffd700 1 -- 172.21.15.16:0/1502839892 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30c00caf40 con 0x7f30c00ccec0 2023-05-20T01:26:40.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.377+0000 7f30be7fc700 1 -- 172.21.15.16:0/1502839892 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3904649751 0 0) 0x7f30c00cb590 con 0x7f30c0071900 2023-05-20T01:26:40.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.377+0000 7f30beffd700 1 --2- 172.21.15.16:0/1502839892 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30c00ccec0 0x7f30c00713c0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f30b00098f0 tx=0x7f30b0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dbe5935a0ade2c7 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:40.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.377+0000 7f30be7fc700 1 -- 172.21.15.16:0/1502839892 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30b000f8a0 con 0x7f30c00ccec0 2023-05-20T01:26:40.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.377+0000 7f30be7fc700 1 -- 172.21.15.16:0/1502839892 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30b000fa00 con 0x7f30c00ccec0 2023-05-20T01:26:40.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.377+0000 7f30be7fc700 1 -- 172.21.15.16:0/1502839892 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30b000fd90 con 0x7f30c00ccec0 2023-05-20T01:26:40.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.378+0000 7f30c5f72700 1 -- 172.21.15.16:0/1502839892 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30c00ccec0 msgr2=0x7f30c00713c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:40.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.378+0000 7f30c5f72700 1 --2- 172.21.15.16:0/1502839892 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30c00ccec0 0x7f30c00713c0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f30b00098f0 tx=0x7f30b0009c00 comp rx=0 tx=0).stop 2023-05-20T01:26:40.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.378+0000 7f30c5f72700 1 -- 172.21.15.16:0/1502839892 shutdown_connections 2023-05-20T01:26:40.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.378+0000 7f30c5f72700 1 --2- 172.21.15.16:0/1502839892 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30c00ccec0 0x7f30c00713c0 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:40.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.378+0000 7f30c5f72700 1 -- 172.21.15.16:0/1502839892 >> 172.21.15.16:0/1502839892 conn(0x7f30c00cffb0 msgr2=0x7f30c00d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:40.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.378+0000 7f30c5f72700 1 -- 172.21.15.16:0/1502839892 shutdown_connections 2023-05-20T01:26:40.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.378+0000 7f30c5f72700 1 -- 172.21.15.16:0/1502839892 wait complete. 2023-05-20T01:26:40.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.379+0000 7f30c5f72700 1 Processor -- start 2023-05-20T01:26:40.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.379+0000 7f30c5f72700 1 -- start start 2023-05-20T01:26:40.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.380+0000 7f30c5f72700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30c0071900 0x7f30c0152600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:40.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.380+0000 7f30c5f72700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f30c00ccec0 0x7f30c00b5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:40.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.380+0000 7f30c5f72700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f30c0157190 0x7f30c0158740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:40.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.380+0000 7f30c5f72700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30c00c41b0 con 0x7f30c0071900 2023-05-20T01:26:40.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.380+0000 7f30c5f72700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30c00c4490 con 0x7f30c0157190 2023-05-20T01:26:40.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.380+0000 7f30c5f72700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30c00c4770 con 0x7f30c00ccec0 2023-05-20T01:26:40.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.380+0000 7f30bf7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30c0071900 0x7f30c0152600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:40.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.380+0000 7f30bf7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30c0071900 0x7f30c0152600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38164/0 (socket says 172.21.15.16:38164) 2023-05-20T01:26:40.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.380+0000 7f30bf7fe700 1 -- 172.21.15.16:0/2008629591 learned_addr learned my addr 172.21.15.16:0/2008629591 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:26:40.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.380+0000 7f30beffd700 1 --2- 172.21.15.16:0/2008629591 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f30c00ccec0 0x7f30c00b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:40.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.380+0000 7f30bffff700 1 --2- 172.21.15.16:0/2008629591 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f30c0157190 0x7f30c0158740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:40.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.380+0000 7f30bf7fe700 1 -- 172.21.15.16:0/2008629591 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f30c00ccec0 msgr2=0x7f30c00b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:40.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.380+0000 7f30bf7fe700 1 --2- 172.21.15.16:0/2008629591 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f30c00ccec0 0x7f30c00b5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:40.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.380+0000 7f30bf7fe700 1 -- 172.21.15.16:0/2008629591 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f30c0157190 msgr2=0x7f30c0158740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:40.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.380+0000 7f30bf7fe700 1 --2- 172.21.15.16:0/2008629591 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f30c0157190 0x7f30c0158740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:40.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.380+0000 7f30bf7fe700 1 -- 172.21.15.16:0/2008629591 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30b0009550 con 0x7f30c0071900 2023-05-20T01:26:40.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.380+0000 7f30beffd700 1 --2- 172.21.15.16:0/2008629591 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f30c00ccec0 0x7f30c00b5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:26:40.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.380+0000 7f30bf7fe700 1 --2- 172.21.15.16:0/2008629591 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30c0071900 0x7f30c0152600 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f30a800c450 tx=0x7f30a800c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:40.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.381+0000 7f30b7fff700 1 -- 172.21.15.16:0/2008629591 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30a80052d0 con 0x7f30c0071900 2023-05-20T01:26:40.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.381+0000 7f30b7fff700 1 -- 172.21.15.16:0/2008629591 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30a8005430 con 0x7f30c0071900 2023-05-20T01:26:40.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.381+0000 7f30c5f72700 1 -- 172.21.15.16:0/2008629591 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30c0157560 con 0x7f30c0071900 2023-05-20T01:26:40.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.381+0000 7f30b7fff700 1 -- 172.21.15.16:0/2008629591 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30a80038c0 con 0x7f30c0071900 2023-05-20T01:26:40.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.381+0000 7f30c5f72700 1 -- 172.21.15.16:0/2008629591 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30c00cb220 con 0x7f30c0071900 2023-05-20T01:26:40.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.382+0000 7f30c5f72700 1 -- 172.21.15.16:0/2008629591 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30c00cba20 con 0x7f30c0071900 2023-05-20T01:26:40.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.382+0000 7f30b7fff700 1 -- 172.21.15.16:0/2008629591 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f30a8003a20 con 0x7f30c0071900 2023-05-20T01:26:40.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.382+0000 7f30b7fff700 1 --2- 172.21.15.16:0/2008629591 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f30a006ace0 0x7f30a006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:40.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.383+0000 7f30b7fff700 1 -- 172.21.15.16:0/2008629591 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f30a80888d0 con 0x7f30c0071900 2023-05-20T01:26:40.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.383+0000 7f30beffd700 1 --2- 172.21.15.16:0/2008629591 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f30a006ace0 0x7f30a006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:40.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.383+0000 7f30beffd700 1 --2- 172.21.15.16:0/2008629591 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f30a006ace0 0x7f30a006d190 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f30b0009130 tx=0x7f30b001b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:40.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.387+0000 7f30b7fff700 1 -- 172.21.15.16:0/2008629591 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f30a80538d0 con 0x7f30c0071900 2023-05-20T01:26:40.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.580+0000 7f30c5f72700 1 -- 172.21.15.16:0/2008629591 --> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f30c015a8c0 con 0x7f30a006ace0 2023-05-20T01:26:40.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.589+0000 7f30b7fff700 1 -- 172.21.15.16:0/2008629591 <== mgr.4104 v2:172.21.15.16:6832/3817237998 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+299805 (secure 0 0 0) 0x7f30c015a8c0 con 0x7f30a006ace0 2023-05-20T01:26:40.596 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:26:40.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.592+0000 7f30c5f72700 1 -- 172.21.15.16:0/2008629591 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f30a006ace0 msgr2=0x7f30a006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:40.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.592+0000 7f30c5f72700 1 --2- 172.21.15.16:0/2008629591 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f30a006ace0 0x7f30a006d190 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f30b0009130 tx=0x7f30b001b040 comp rx=0 tx=0).stop 2023-05-20T01:26:40.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.592+0000 7f30c5f72700 1 -- 172.21.15.16:0/2008629591 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30c0071900 msgr2=0x7f30c0152600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:40.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.592+0000 7f30c5f72700 1 --2- 172.21.15.16:0/2008629591 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30c0071900 0x7f30c0152600 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f30a800c450 tx=0x7f30a800c810 comp rx=0 tx=0).stop 2023-05-20T01:26:40.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.592+0000 7f30c5f72700 1 -- 172.21.15.16:0/2008629591 shutdown_connections 2023-05-20T01:26:40.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.592+0000 7f30c5f72700 1 --2- 172.21.15.16:0/2008629591 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f30c0157190 0x7f30c0158740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:40.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.592+0000 7f30c5f72700 1 --2- 172.21.15.16:0/2008629591 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30c0071900 0x7f30c0152600 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:40.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.592+0000 7f30c5f72700 1 --2- 172.21.15.16:0/2008629591 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f30a006ace0 0x7f30a006d190 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:40.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.592+0000 7f30c5f72700 1 --2- 172.21.15.16:0/2008629591 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f30c00ccec0 0x7f30c00b5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:40.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.592+0000 7f30c5f72700 1 -- 172.21.15.16:0/2008629591 >> 172.21.15.16:0/2008629591 conn(0x7f30c00cffb0 msgr2=0x7f30c015a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:40.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.592+0000 7f30c5f72700 1 -- 172.21.15.16:0/2008629591 shutdown_connections 2023-05-20T01:26:40.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.592+0000 7f30c5f72700 1 -- 172.21.15.16:0/2008629591 wait complete. 2023-05-20T01:26:40.600 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2023-05-20T01:26:40.609 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":415,"stamp":"2023-05-20T01:26:39.454792+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":30,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":110204,"kb_used_data":11644,"kb_used_omap":0,"kb_used_meta":98496,"kb_avail":754864516,"statfs":{"total":773094113280,"available":772981264384,"internally_reserved":0,"allocated":11923456,"data_stored":6674952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":100859904},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.819728"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-05-20T01:26:34.638110+0000","last_change":"2023-05-20T01:26:34.638110+0000","last_active":"2023-05-20T01:26:34.630120+0000","last_peered":"2023-05-20T01:26:34.630120+0000","last_clean":"2023-05-20T01:26:34.630120+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:34.638110+0000","last_undegraded":"2023-05-20T01:26:34.638110+0000","last_fullsized":"2023-05-20T01:26:34.638110+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-05-20T01:26:34.639497+0000","last_change":"2023-05-20T01:26:34.639497+0000","last_active":"2023-05-20T01:26:34.630120+0000","last_peered":"2023-05-20T01:26:34.630120+0000","last_clean":"2023-05-20T01:26:34.630120+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:34.639497+0000","last_undegraded":"2023-05-20T01:26:34.639497+0000","last_fullsized":"2023-05-20T01:26:34.639497+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-05-20T01:26:34.644602+0000","last_change":"2023-05-20T01:26:34.644602+0000","last_active":"2023-05-20T01:26:34.630120+0000","last_peered":"2023-05-20T01:26:34.630120+0000","last_clean":"2023-05-20T01:26:34.630120+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:34.644602+0000","last_undegraded":"2023-05-20T01:26:34.644602+0000","last_fullsized":"2023-05-20T01:26:34.644602+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-05-20T01:26:34.642067+0000","last_change":"2023-05-20T01:26:34.642067+0000","last_active":"2023-05-20T01:26:34.630120+0000","last_peered":"2023-05-20T01:26:34.630120+0000","last_clean":"2023-05-20T01:26:34.630120+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:34.642067+0000","last_undegraded":"2023-05-20T01:26:34.642067+0000","last_fullsized":"2023-05-20T01:26:34.642067+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-05-20T01:26:34.640655+0000","last_change":"2023-05-20T01:26:34.640655+0000","last_active":"2023-05-20T01:26:34.630120+0000","last_peered":"2023-05-20T01:26:34.630120+0000","last_clean":"2023-05-20T01:26:34.630120+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:34.640655+0000","last_undegraded":"2023-05-20T01:26:34.640655+0000","last_fullsized":"2023-05-20T01:26:34.640655+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-05-20T01:26:34.641569+0000","last_change":"2023-05-20T01:26:34.641569+0000","last_active":"2023-05-20T01:26:34.630120+0000","last_peered":"2023-05-20T01:26:34.630120+0000","last_clean":"2023-05-20T01:26:34.630120+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:34.641569+0000","last_undegraded":"2023-05-20T01:26:34.641569+0000","last_fullsized":"2023-05-20T01:26:34.641569+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-05-20T01:26:34.644015+0000","last_change":"2023-05-20T01:26:34.644015+0000","last_active":"2023-05-20T01:26:34.630120+0000","last_peered":"2023-05-20T01:26:34.630120+0000","last_clean":"2023-05-20T01:26:34.630120+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:34.644015+0000","last_undegraded":"2023-05-20T01:26:34.644015+0000","last_fullsized":"2023-05-20T01:26:34.644015+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-05-20T01:26:34.638480+0000","last_change":"2023-05-20T01:26:34.638480+0000","last_active":"2023-05-20T01:26:34.630120+0000","last_peered":"2023-05-20T01:26:34.630120+0000","last_clean":"2023-05-20T01:26:34.630120+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:34.638480+0000","last_undegraded":"2023-05-20T01:26:34.638480+0000","last_fullsized":"2023-05-20T01:26:34.638480+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-05-20T01:26:34.644724+0000","last_change":"2023-05-20T01:26:34.644724+0000","last_active":"2023-05-20T01:26:34.630120+0000","last_peered":"2023-05-20T01:26:34.630120+0000","last_clean":"2023-05-20T01:26:34.630120+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:34.644724+0000","last_undegraded":"2023-05-20T01:26:34.644724+0000","last_fullsized":"2023-05-20T01:26:34.644724+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-05-20T01:26:34.639019+0000","last_change":"2023-05-20T01:26:34.639019+0000","last_active":"2023-05-20T01:26:34.630120+0000","last_peered":"2023-05-20T01:26:34.630120+0000","last_clean":"2023-05-20T01:26:34.630120+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:34.639019+0000","last_undegraded":"2023-05-20T01:26:34.639019+0000","last_fullsized":"2023-05-20T01:26:34.639019+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-05-20T01:26:34.641727+0000","last_change":"2023-05-20T01:26:34.641727+0000","last_active":"2023-05-20T01:26:34.630120+0000","last_peered":"2023-05-20T01:26:34.630120+0000","last_clean":"2023-05-20T01:26:34.630120+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:34.641727+0000","last_undegraded":"2023-05-20T01:26:34.641727+0000","last_fullsized":"2023-05-20T01:26:34.641727+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-05-20T01:26:34.640629+0000","last_change":"2023-05-20T01:26:34.640629+0000","last_active":"2023-05-20T01:26:34.630120+0000","last_peered":"2023-05-20T01:26:34.630120+0000","last_clean":"2023-05-20T01:26:34.630120+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:34.640629+0000","last_undegraded":"2023-05-20T01:26:34.640629+0000","last_fullsized":"2023-05-20T01:26:34.640629+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-05-20T01:26:34.641472+0000","last_change":"2023-05-20T01:26:34.641472+0000","last_active":"2023-05-20T01:26:34.630120+0000","last_peered":"2023-05-20T01:26:34.630120+0000","last_clean":"2023-05-20T01:26:34.630120+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:34.641472+0000","last_undegraded":"2023-05-20T01:26:34.641472+0000","last_fullsized":"2023-05-20T01:26:34.641472+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-05-20T01:26:34.638486+0000","last_change":"2023-05-20T01:26:34.638486+0000","last_active":"2023-05-20T01:26:34.630120+0000","last_peered":"2023-05-20T01:26:34.630120+0000","last_clean":"2023-05-20T01:26:34.630120+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:34.638486+0000","last_undegraded":"2023-05-20T01:26:34.638486+0000","last_fullsized":"2023-05-20T01:26:34.638486+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-05-20T01:26:34.638266+0000","last_change":"2023-05-20T01:26:34.638266+0000","last_active":"2023-05-20T01:26:34.630120+0000","last_peered":"2023-05-20T01:26:34.630120+0000","last_clean":"2023-05-20T01:26:34.630120+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:34.638266+0000","last_undegraded":"2023-05-20T01:26:34.638266+0000","last_fullsized":"2023-05-20T01:26:34.638266+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-05-20T01:26:34.643913+0000","last_change":"2023-05-20T01:26:34.643913+0000","last_active":"2023-05-20T01:26:34.630120+0000","last_peered":"2023-05-20T01:26:34.630120+0000","last_clean":"2023-05-20T01:26:34.630120+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:34.643913+0000","last_undegraded":"2023-05-20T01:26:34.643913+0000","last_fullsized":"2023-05-20T01:26:34.643913+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-05-20T01:26:34.641466+0000","last_change":"2023-05-20T01:26:34.641466+0000","last_active":"2023-05-20T01:26:34.630120+0000","last_peered":"2023-05-20T01:26:34.630120+0000","last_clean":"2023-05-20T01:26:34.630120+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:34.641466+0000","last_undegraded":"2023-05-20T01:26:34.641466+0000","last_fullsized":"2023-05-20T01:26:34.641466+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-05-20T01:26:34.638503+0000","last_change":"2023-05-20T01:26:34.638503+0000","last_active":"2023-05-20T01:26:34.630120+0000","last_peered":"2023-05-20T01:26:34.630120+0000","last_clean":"2023-05-20T01:26:34.630120+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:34.638503+0000","last_undegraded":"2023-05-20T01:26:34.638503+0000","last_fullsized":"2023-05-20T01:26:34.638503+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-05-20T01:26:34.640567+0000","last_change":"2023-05-20T01:26:34.640567+0000","last_active":"2023-05-20T01:26:34.630120+0000","last_peered":"2023-05-20T01:26:34.630120+0000","last_clean":"2023-05-20T01:26:34.630120+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:34.640567+0000","last_undegraded":"2023-05-20T01:26:34.640567+0000","last_fullsized":"2023-05-20T01:26:34.640567+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-05-20T01:26:34.641446+0000","last_change":"2023-05-20T01:26:34.641446+0000","last_active":"2023-05-20T01:26:34.630120+0000","last_peered":"2023-05-20T01:26:34.630120+0000","last_clean":"2023-05-20T01:26:34.630120+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:34.641446+0000","last_undegraded":"2023-05-20T01:26:34.641446+0000","last_fullsized":"2023-05-20T01:26:34.641446+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-05-20T01:26:34.644021+0000","last_change":"2023-05-20T01:26:34.644021+0000","last_active":"2023-05-20T01:26:34.630120+0000","last_peered":"2023-05-20T01:26:34.630120+0000","last_clean":"2023-05-20T01:26:34.630120+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:34.644021+0000","last_undegraded":"2023-05-20T01:26:34.644021+0000","last_fullsized":"2023-05-20T01:26:34.644021+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-05-20T01:26:34.638401+0000","last_change":"2023-05-20T01:26:34.638401+0000","last_active":"2023-05-20T01:26:34.630120+0000","last_peered":"2023-05-20T01:26:34.630120+0000","last_clean":"2023-05-20T01:26:34.630120+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:34.638401+0000","last_undegraded":"2023-05-20T01:26:34.638401+0000","last_fullsized":"2023-05-20T01:26:34.638401+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-05-20T01:26:34.640200+0000","last_change":"2023-05-20T01:26:34.640200+0000","last_active":"2023-05-20T01:26:34.630120+0000","last_peered":"2023-05-20T01:26:34.630120+0000","last_clean":"2023-05-20T01:26:34.630120+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:34.640200+0000","last_undegraded":"2023-05-20T01:26:34.640200+0000","last_fullsized":"2023-05-20T01:26:34.640200+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-05-20T01:26:34.638339+0000","last_change":"2023-05-20T01:26:34.638339+0000","last_active":"2023-05-20T01:26:34.630120+0000","last_peered":"2023-05-20T01:26:34.630120+0000","last_clean":"2023-05-20T01:26:34.630120+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:34.638339+0000","last_undegraded":"2023-05-20T01:26:34.638339+0000","last_fullsized":"2023-05-20T01:26:34.638339+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-05-20T01:26:34.642905+0000","last_change":"2023-05-20T01:26:34.642905+0000","last_active":"2023-05-20T01:26:34.630120+0000","last_peered":"2023-05-20T01:26:34.630120+0000","last_clean":"2023-05-20T01:26:34.630120+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:34.642905+0000","last_undegraded":"2023-05-20T01:26:34.642905+0000","last_fullsized":"2023-05-20T01:26:34.642905+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":191,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-05-20T01:26:34.645291+0000","last_change":"2023-05-20T01:16:23.289068+0000","last_active":"2023-05-20T01:26:34.645291+0000","last_peered":"2023-05-20T01:26:34.645291+0000","last_clean":"2023-05-20T01:26:34.645291+0000","last_became_active":"2023-05-20T01:16:23.288781+0000","last_became_peered":"2023-05-20T01:16:23.288781+0000","last_unstale":"2023-05-20T01:26:34.645291+0000","last_undegraded":"2023-05-20T01:26:34.645291+0000","last_fullsized":"2023-05-20T01:26:34.645291+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:23:26.310250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-05-20T01:26:34.639380+0000","last_change":"2023-05-20T01:26:34.639380+0000","last_active":"2023-05-20T01:26:34.630120+0000","last_peered":"2023-05-20T01:26:34.630120+0000","last_clean":"2023-05-20T01:26:34.630120+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:34.639380+0000","last_undegraded":"2023-05-20T01:26:34.639380+0000","last_fullsized":"2023-05-20T01:26:34.639380+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":199,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-05-20T01:26:34.637011+0000","last_change":"2023-05-20T01:16:22.829324+0000","last_active":"2023-05-20T01:26:34.637011+0000","last_peered":"2023-05-20T01:26:34.637011+0000","last_clean":"2023-05-20T01:26:34.637011+0000","last_became_active":"2023-05-20T01:16:22.828964+0000","last_became_peered":"2023-05-20T01:16:22.828964+0000","last_unstale":"2023-05-20T01:26:34.637011+0000","last_undegraded":"2023-05-20T01:26:34.637011+0000","last_fullsized":"2023-05-20T01:26:34.637011+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:23:48.805679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-05-20T01:26:34.641191+0000","last_change":"2023-05-20T01:26:34.641191+0000","last_active":"2023-05-20T01:26:34.630120+0000","last_peered":"2023-05-20T01:26:34.630120+0000","last_clean":"2023-05-20T01:26:34.630120+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:34.641191+0000","last_undegraded":"2023-05-20T01:26:34.641191+0000","last_fullsized":"2023-05-20T01:26:34.641191+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":175,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-05-20T01:26:34.642808+0000","last_change":"2023-05-20T01:16:22.834754+0000","last_active":"2023-05-20T01:26:34.642808+0000","last_peered":"2023-05-20T01:26:34.642808+0000","last_clean":"2023-05-20T01:26:34.642808+0000","last_became_active":"2023-05-20T01:16:22.834437+0000","last_became_peered":"2023-05-20T01:16:22.834437+0000","last_unstale":"2023-05-20T01:26:34.642808+0000","last_undegraded":"2023-05-20T01:26:34.642808+0000","last_fullsized":"2023-05-20T01:26:34.642808+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T01:22:38.911204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":310,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-05-20T01:26:34.642091+0000","last_change":"2023-05-20T01:16:22.834920+0000","last_active":"2023-05-20T01:26:34.642091+0000","last_peered":"2023-05-20T01:26:34.642091+0000","last_clean":"2023-05-20T01:26:34.642091+0000","last_became_active":"2023-05-20T01:16:22.833924+0000","last_became_peered":"2023-05-20T01:16:22.833924+0000","last_unstale":"2023-05-20T01:26:34.642091+0000","last_undegraded":"2023-05-20T01:26:34.642091+0000","last_fullsized":"2023-05-20T01:26:34.642091+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:02:13.281571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-05-20T01:26:34.642061+0000","last_change":"2023-05-20T01:26:34.642061+0000","last_active":"2023-05-20T01:26:34.630120+0000","last_peered":"2023-05-20T01:26:34.630120+0000","last_clean":"2023-05-20T01:26:34.630120+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:34.642061+0000","last_undegraded":"2023-05-20T01:26:34.642061+0000","last_fullsized":"2023-05-20T01:26:34.642061+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":159,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-05-20T01:26:34.642202+0000","last_change":"2023-05-20T01:16:23.290259+0000","last_active":"2023-05-20T01:26:34.642202+0000","last_peered":"2023-05-20T01:26:34.642202+0000","last_clean":"2023-05-20T01:26:34.642202+0000","last_became_active":"2023-05-20T01:16:23.289911+0000","last_became_peered":"2023-05-20T01:16:23.289911+0000","last_unstale":"2023-05-20T01:26:34.642202+0000","last_undegraded":"2023-05-20T01:26:34.642202+0000","last_fullsized":"2023-05-20T01:26:34.642202+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:40:10.726971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-05-20T01:26:34.638510+0000","last_change":"2023-05-20T01:26:34.638510+0000","last_active":"2023-05-20T01:26:34.630120+0000","last_peered":"2023-05-20T01:26:34.630120+0000","last_clean":"2023-05-20T01:26:34.630120+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:34.638510+0000","last_undegraded":"2023-05-20T01:26:34.638510+0000","last_fullsized":"2023-05-20T01:26:34.638510+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":160,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-05-20T01:26:34.642159+0000","last_change":"2023-05-20T01:16:23.290207+0000","last_active":"2023-05-20T01:26:34.642159+0000","last_peered":"2023-05-20T01:26:34.642159+0000","last_clean":"2023-05-20T01:26:34.642159+0000","last_became_active":"2023-05-20T01:16:23.289928+0000","last_became_peered":"2023-05-20T01:16:23.289928+0000","last_unstale":"2023-05-20T01:26:34.642159+0000","last_undegraded":"2023-05-20T01:26:34.642159+0000","last_fullsized":"2023-05-20T01:26:34.642159+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:05:09.399167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-05-20T01:26:34.640347+0000","last_change":"2023-05-20T01:26:34.640347+0000","last_active":"2023-05-20T01:26:34.630120+0000","last_peered":"2023-05-20T01:26:34.630120+0000","last_clean":"2023-05-20T01:26:34.630120+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:34.640347+0000","last_undegraded":"2023-05-20T01:26:34.640347+0000","last_fullsized":"2023-05-20T01:26:34.640347+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":161,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-05-20T01:26:34.642350+0000","last_change":"2023-05-20T01:16:23.290538+0000","last_active":"2023-05-20T01:26:34.642350+0000","last_peered":"2023-05-20T01:26:34.642350+0000","last_clean":"2023-05-20T01:26:34.642350+0000","last_became_active":"2023-05-20T01:16:23.290230+0000","last_became_peered":"2023-05-20T01:16:23.290230+0000","last_unstale":"2023-05-20T01:26:34.642350+0000","last_undegraded":"2023-05-20T01:26:34.642350+0000","last_fullsized":"2023-05-20T01:26:34.642350+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:47:13.453162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-05-20T01:26:34.638985+0000","last_change":"2023-05-20T01:26:34.638985+0000","last_active":"2023-05-20T01:26:34.630120+0000","last_peered":"2023-05-20T01:26:34.630120+0000","last_clean":"2023-05-20T01:26:34.630120+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:34.638985+0000","last_undegraded":"2023-05-20T01:26:34.638985+0000","last_fullsized":"2023-05-20T01:26:34.638985+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":175,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-05-20T01:26:34.641407+0000","last_change":"2023-05-20T01:16:23.290341+0000","last_active":"2023-05-20T01:26:34.641407+0000","last_peered":"2023-05-20T01:26:34.641407+0000","last_clean":"2023-05-20T01:26:34.641407+0000","last_became_active":"2023-05-20T01:16:23.290053+0000","last_became_peered":"2023-05-20T01:16:23.290053+0000","last_unstale":"2023-05-20T01:26:34.641407+0000","last_undegraded":"2023-05-20T01:26:34.641407+0000","last_fullsized":"2023-05-20T01:26:34.641407+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:17:08.936381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-05-20T01:26:34.640797+0000","last_change":"2023-05-20T01:26:34.640797+0000","last_active":"2023-05-20T01:26:34.630120+0000","last_peered":"2023-05-20T01:26:34.630120+0000","last_clean":"2023-05-20T01:26:34.630120+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:34.640797+0000","last_undegraded":"2023-05-20T01:26:34.640797+0000","last_fullsized":"2023-05-20T01:26:34.640797+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":175,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-05-20T01:26:34.641451+0000","last_change":"2023-05-20T01:16:22.834330+0000","last_active":"2023-05-20T01:26:34.641451+0000","last_peered":"2023-05-20T01:26:34.641451+0000","last_clean":"2023-05-20T01:26:34.641451+0000","last_became_active":"2023-05-20T01:16:22.833946+0000","last_became_peered":"2023-05-20T01:16:22.833946+0000","last_unstale":"2023-05-20T01:26:34.641451+0000","last_undegraded":"2023-05-20T01:26:34.641451+0000","last_fullsized":"2023-05-20T01:26:34.641451+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:23:50.042871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574975,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":16432,"kb_used_data":1896,"kb_used_omap":0,"kb_used_meta":14528,"kb_avail":94355408,"statfs":{"total":96636764160,"available":96619937792,"internally_reserved":0,"allocated":1941504,"data_stored":1279819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":14876672},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:25:37 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.48599999999999999,"15min":0.51400000000000001},"min":{"1min":0.30199999999999999,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.88200000000000001},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.51000000000000001,"15min":0.54700000000000004},"min":{"1min":0.39200000000000002,"5min":0.309,"15min":0.30599999999999999},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.90000000000000002},"last":0.64500000000000002}]},{"osd":1,"last update":"Sat May 20 01:25:37 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.497,"15min":0.72199999999999998},"min":{"1min":0.371,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":8.8879999999999999},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.53200000000000003,"15min":0.76000000000000001},"min":{"1min":0.44800000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":8.875},"last":0.753}]},{"osd":2,"last update":"Sat May 20 01:25:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.502,"15min":0.70899999999999996},"min":{"1min":0.32600000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":8.9120000000000008},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.50800000000000001,"15min":0.71399999999999997},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":8.8520000000000003},"last":0.58799999999999997}]},{"osd":3,"last update":"Sat May 20 01:25:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.498,"15min":0.76100000000000001},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":10.884},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.55400000000000005,"15min":0.81699999999999995},"min":{"1min":0.40699999999999997,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":10.952},"last":0.65500000000000003}]},{"osd":4,"last update":"Sat May 20 01:25:37 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.51000000000000001,"15min":0.51200000000000001},"min":{"1min":0.35899999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.52900000000000003,"15min":0.53100000000000003},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.875},"last":0.53000000000000003}]},{"osd":5,"last update":"Sat May 20 01:25:37 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.56499999999999995,"15min":0.54700000000000004},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.56999999999999995,"15min":0.57799999999999996},"min":{"1min":0.39700000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.46500000000000002}]},{"osd":6,"last update":"Sat May 20 01:25:37 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.59499999999999997,"15min":0.59799999999999998},"min":{"1min":0.41999999999999998,"5min":0.41599999999999998,"15min":0.378},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.57199999999999995,"15min":0.58499999999999996},"min":{"1min":0.39000000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.47999999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574974,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":11116,"kb_used_data":1316,"kb_used_omap":0,"kb_used_meta":9792,"kb_avail":94360724,"statfs":{"total":96636764160,"available":96625381376,"internally_reserved":0,"allocated":1347584,"data_stored":689451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":10027008},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:25:42 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.44800000000000001,"15min":0.47699999999999998},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.26900000000000002},"max":{"1min":0.60999999999999999,"5min":0.72699999999999998,"15min":0.77200000000000002},"last":0.44},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.47899999999999998,"15min":0.51700000000000002},"min":{"1min":0.32900000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.504}]},{"osd":1,"last update":"Sat May 20 01:25:42 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.48699999999999999,"15min":0.51900000000000002},"min":{"1min":0.32800000000000001,"5min":0.29699999999999999,"15min":0.28000000000000003},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.83799999999999997},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.51700000000000002,"15min":0.52600000000000002},"min":{"1min":0.379,"5min":0.32600000000000001,"15min":0.318},"max":{"1min":0.66500000000000004,"5min":0.749,"15min":0.79200000000000004},"last":0.42199999999999999}]},{"osd":2,"last update":"Sat May 20 01:25:42 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.48399999999999999,"15min":0.503},"min":{"1min":0.35099999999999998,"5min":0.34399999999999997,"15min":0.33700000000000002},"max":{"1min":0.57099999999999995,"5min":0.72199999999999998,"15min":0.76600000000000001},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.51600000000000001,"15min":0.54900000000000004},"min":{"1min":0.37,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.745,"5min":0.76200000000000001,"15min":0.76800000000000002},"last":0.53800000000000003}]},{"osd":3,"last update":"Sat May 20 01:25:42 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.52500000000000002,"15min":0.53600000000000003},"min":{"1min":0.378,"5min":0.311,"15min":0.311},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.52000000000000002,"15min":0.53900000000000003},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.72699999999999998,"5min":0.73199999999999998,"15min":0.79500000000000004},"last":0.51600000000000001}]},{"osd":4,"last update":"Sat May 20 01:25:42 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.53100000000000003,"15min":0.54400000000000004},"min":{"1min":0.34799999999999998,"5min":0.33300000000000002,"15min":0.32600000000000001},"max":{"1min":0.70699999999999996,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.54000000000000004,"15min":0.56299999999999994},"min":{"1min":0.40500000000000003,"5min":0.38100000000000001,"15min":0.32700000000000001},"max":{"1min":0.71799999999999997,"5min":0.77400000000000002,"15min":0.80100000000000005},"last":0.58999999999999997}]},{"osd":5,"last update":"Sat May 20 01:25:42 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.53400000000000003,"15min":0.54000000000000004},"min":{"1min":0.38,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.63600000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.56499999999999995,"15min":0.59799999999999998},"min":{"1min":0.372,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.64400000000000002,"5min":0.81599999999999995,"15min":0.89400000000000002},"last":0.57599999999999996}]},{"osd":7,"last update":"Sat May 20 01:25:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.57099999999999995,"15min":0.59399999999999997},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.71299999999999997,"5min":0.80200000000000005,"15min":0.92900000000000005},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.57599999999999996,"15min":0.60499999999999998},"min":{"1min":0.45100000000000001,"5min":0.379,"15min":0.379},"max":{"1min":0.66600000000000004,"5min":0.82799999999999996,"15min":0.85199999999999998},"last":0.60299999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574974,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":17264,"kb_used_data":1896,"kb_used_omap":0,"kb_used_meta":15360,"kb_avail":94354576,"statfs":{"total":96636764160,"available":96619085824,"internally_reserved":0,"allocated":1941504,"data_stored":1279819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":15728640},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:25:36 2023","interfaces":[{"interface":"back","average":{"1min":0.30599999999999999,"5min":0.28299999999999997,"15min":0.311},"min":{"1min":0.18099999999999999,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":0.48799999999999999,"5min":0.57099999999999995,"15min":1.194},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.32400000000000001,"15min":0.31900000000000001},"min":{"1min":0.23899999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.75800000000000001},"last":0.40600000000000003}]},{"osd":2,"last update":"Sat May 20 01:25:36 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.307,"15min":0.31900000000000001},"min":{"1min":0.23100000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.66400000000000003},"last":0.442},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.374,"15min":0.40400000000000003},"min":{"1min":0.313,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":1.3580000000000001},"last":0.52400000000000002}]},{"osd":3,"last update":"Sat May 20 01:25:36 2023","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.35899999999999999,"15min":0.371},"min":{"1min":0.27400000000000002,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":1.619},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.35299999999999998,"15min":0.379},"min":{"1min":0.25800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.57699999999999996,"5min":0.60899999999999999,"15min":1.8400000000000001},"last":0.497}]},{"osd":4,"last update":"Sat May 20 01:25:36 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.55600000000000005,"15min":0.57899999999999996},"min":{"1min":0.36799999999999999,"5min":0.34899999999999998,"15min":0.32200000000000001},"max":{"1min":0.76100000000000001,"5min":0.81699999999999995,"15min":1.2769999999999999},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.57099999999999995,"15min":0.58299999999999996},"min":{"1min":0.38,"5min":0.312,"15min":0.312},"max":{"1min":0.81599999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.63600000000000001}]},{"osd":5,"last update":"Sat May 20 01:25:36 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.58699999999999997,"15min":0.59099999999999997},"min":{"1min":0.46600000000000003,"5min":0.37,"15min":0.37},"max":{"1min":0.78500000000000003,"5min":0.81899999999999995,"15min":1.4510000000000001},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.58199999999999996,"15min":0.60299999999999998},"min":{"1min":0.39700000000000002,"5min":0.39100000000000001,"15min":0.378},"max":{"1min":0.76800000000000002,"5min":0.78800000000000003,"15min":1.409},"last":0.59999999999999998}]},{"osd":6,"last update":"Sat May 20 01:25:36 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.60299999999999998},"min":{"1min":0.39000000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.82299999999999995,"5min":0.876,"15min":1.575},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.58199999999999996,"15min":0.59399999999999997},"min":{"1min":0.45400000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":1.0600000000000001},"last":0.61399999999999999}]},{"osd":7,"last update":"Sat May 20 01:25:36 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.60399999999999998},"min":{"1min":0.41099999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":1.496},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.63,"15min":0.64000000000000001},"min":{"1min":0.44500000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.5009999999999999,"5min":2.1709999999999998,"15min":2.1709999999999998},"last":0.66900000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574974,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":11176,"kb_used_data":1312,"kb_used_omap":0,"kb_used_meta":9856,"kb_avail":94360664,"statfs":{"total":96636764160,"available":96625319936,"internally_reserved":0,"allocated":1343488,"data_stored":688490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":10092544},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat May 20 01:25:35 2023","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.29799999999999999,"15min":0.32000000000000001},"min":{"1min":0.188,"5min":0.17999999999999999,"15min":0.17999999999999999},"max":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.314,"15min":0.34399999999999997},"min":{"1min":0.23999999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":1.502,"5min":1.502,"15min":1.502},"last":0.23400000000000001}]},{"osd":2,"last update":"Sat May 20 01:25:35 2023","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.30299999999999999,"15min":0.32800000000000001},"min":{"1min":0.20899999999999999,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.55700000000000005},"last":0.28299999999999997},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.36199999999999999,"15min":0.38500000000000001},"min":{"1min":0.29599999999999999,"5min":0.216,"15min":0.216},"max":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.314}]},{"osd":3,"last update":"Sat May 20 01:25:35 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.36399999999999999,"15min":0.37},"min":{"1min":0.23499999999999999,"5min":0.22,"15min":0.22},"max":{"1min":4.258,"5min":4.258,"15min":4.258},"last":0.27100000000000002},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.35199999999999998,"15min":0.36599999999999999},"min":{"1min":0.25700000000000001,"5min":0.245,"15min":0.245},"max":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.60899999999999999},"last":0.35899999999999999}]},{"osd":4,"last update":"Sat May 20 01:25:35 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.57199999999999995,"15min":0.55500000000000005},"min":{"1min":0.373,"5min":0.372,"15min":0.34399999999999997},"max":{"1min":1.4330000000000001,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.58499999999999996,"15min":0.57199999999999995},"min":{"1min":0.42799999999999999,"5min":0.39200000000000002,"15min":0.36299999999999999},"max":{"1min":1.298,"5min":1.298,"15min":1.298},"last":0.61499999999999999}]},{"osd":5,"last update":"Sat May 20 01:25:35 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.57399999999999995,"15min":0.57699999999999996},"min":{"1min":0.40600000000000003,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.79900000000000004,"5min":0.80500000000000005,"15min":0.88100000000000001},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.60099999999999998,"15min":0.59399999999999997},"min":{"1min":0.41299999999999998,"5min":0.374,"15min":0.374},"max":{"1min":1.4710000000000001,"5min":1.4710000000000001,"15min":4.4720000000000004},"last":0.65700000000000003}]},{"osd":6,"last update":"Sat May 20 01:25:35 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.68200000000000005,"15min":0.625},"min":{"1min":0.36899999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.79800000000000004,"5min":10.042999999999999,"15min":10.042999999999999},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.60599999999999998,"15min":0.61699999999999999},"min":{"1min":0.40200000000000002,"5min":0.377,"15min":0.377},"max":{"1min":1.367,"5min":1.367,"15min":1.367},"last":0.59099999999999997}]},{"osd":7,"last update":"Sat May 20 01:25:35 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.60699999999999998,"15min":0.59799999999999998},"min":{"1min":0.443,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.399,"5min":1.399,"15min":1.399},"last":0.628},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.72699999999999998,"15min":0.65800000000000003},"min":{"1min":0.439,"5min":0.439,"15min":0.42099999999999999},"max":{"1min":1.4490000000000001,"5min":10.085000000000001,"15min":10.085000000000001},"last":0.56699999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574974,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":11180,"kb_used_data":1316,"kb_used_omap":0,"kb_used_meta":9856,"kb_avail":94360660,"statfs":{"total":96636764160,"available":96625315840,"internally_reserved":0,"allocated":1347584,"data_stored":689451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":10092544},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.33700000000000002,"5min":0.28299999999999997,"15min":0.32300000000000001},"min":{"1min":0.224,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":0.50900000000000001,"5min":0.53400000000000003,"15min":1.038},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.32300000000000001,"5min":0.29899999999999999,"15min":0.33900000000000002},"min":{"1min":0.21199999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":1.079},"last":0.26200000000000001}]},{"osd":1,"last update":"Sat May 20 01:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.317,"15min":0.376},"min":{"1min":0.253,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.44500000000000001,"5min":0.61399999999999999,"15min":1.4950000000000001},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.35899999999999999,"15min":0.41299999999999998},"min":{"1min":0.28399999999999997,"5min":0.249,"15min":0.249},"max":{"1min":0.60799999999999998,"5min":0.629,"15min":1.5189999999999999},"last":0.33000000000000002}]},{"osd":3,"last update":"Sat May 20 01:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.32900000000000001,"15min":0.378},"min":{"1min":0.246,"5min":0.218,"15min":0.216},"max":{"1min":0.57899999999999996,"5min":0.60599999999999998,"15min":1.2090000000000001},"last":0.379},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.36899999999999999,"15min":0.42699999999999999},"min":{"1min":0.32400000000000001,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.59499999999999997,"5min":0.68100000000000005,"15min":1.5620000000000001},"last":0.35299999999999998}]},{"osd":4,"last update":"Sat May 20 01:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.56799999999999995,"15min":0.58099999999999996},"min":{"1min":0.46500000000000002,"5min":0.35899999999999999,"15min":0.34899999999999998},"max":{"1min":1.107,"5min":1.107,"15min":1.319},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.59399999999999997,"15min":0.61299999999999999},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":1.4319999999999999},"last":0.70999999999999996}]},{"osd":5,"last update":"Sat May 20 01:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.56799999999999995,"15min":0.61699999999999999},"min":{"1min":0.443,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.82199999999999995,"5min":0.95599999999999996,"15min":1.758},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.58899999999999997,"15min":0.63500000000000001},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.004,"5min":1.004,"15min":1.7110000000000001},"last":0.65400000000000003}]},{"osd":6,"last update":"Sat May 20 01:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.59099999999999997,"15min":0.63200000000000001},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.069,"5min":1.069,"15min":1.665},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.58499999999999996,"15min":0.628},"min":{"1min":0.39900000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.88,"5min":0.93500000000000005,"15min":1.819},"last":0.75}]},{"osd":7,"last update":"Sat May 20 01:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.59999999999999998,"15min":0.65200000000000002},"min":{"1min":0.39000000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":1.7310000000000001},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.60999999999999999,"15min":0.65100000000000002},"min":{"1min":0.47799999999999998,"5min":0.42399999999999999,"15min":0.42199999999999999},"max":{"1min":1.038,"5min":1.038,"15min":1.8460000000000001},"last":0.77700000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574974,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":15456,"kb_used_data":1304,"kb_used_omap":0,"kb_used_meta":14144,"kb_avail":94356384,"statfs":{"total":96636764160,"available":96620937216,"internally_reserved":0,"allocated":1335296,"data_stored":682961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":14483456},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:26:34 2023","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.316,"15min":0.31900000000000001},"min":{"1min":0.24099999999999999,"5min":0.20100000000000001,"15min":0.188},"max":{"1min":0.54100000000000004,"5min":0.65000000000000002,"15min":1.528},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.34200000000000003,"15min":0.40799999999999997},"min":{"1min":0.26500000000000001,"5min":0.20799999999999999,"15min":0.19500000000000001},"max":{"1min":0.54200000000000004,"5min":0.72399999999999998,"15min":3.1890000000000001},"last":0.54200000000000004}]},{"osd":1,"last update":"Sat May 20 01:26:34 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.32500000000000001,"15min":0.40000000000000002},"min":{"1min":0.22700000000000001,"5min":0.189,"15min":0.189},"max":{"1min":0.55900000000000005,"5min":0.73199999999999998,"15min":3.4260000000000002},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.437,"5min":0.372,"15min":0.36099999999999999},"min":{"1min":0.27700000000000002,"5min":0.23100000000000001,"15min":0.221},"max":{"1min":0.60999999999999999,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.60999999999999999}]},{"osd":2,"last update":"Sat May 20 01:26:34 2023","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.36399999999999999,"15min":0.38400000000000001},"min":{"1min":0.30399999999999999,"5min":0.24299999999999999,"15min":0.224},"max":{"1min":0.59399999999999997,"5min":0.67400000000000004,"15min":1.1619999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.40000000000000002,"15min":0.41299999999999998},"min":{"1min":0.26900000000000002,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.75,"5min":0.78500000000000003,"15min":1.413},"last":0.64900000000000002}]},{"osd":4,"last update":"Sat May 20 01:26:34 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.56999999999999995,"15min":0.58299999999999996},"min":{"1min":0.45800000000000002,"5min":0.38700000000000001,"15min":0.38200000000000001},"max":{"1min":0.77000000000000002,"5min":0.95399999999999996,"15min":1.0700000000000001},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.59199999999999997,"15min":0.66100000000000003},"min":{"1min":0.42399999999999999,"5min":0.40200000000000002,"15min":0.35899999999999999},"max":{"1min":0.88500000000000001,"5min":0.88600000000000001,"15min":3.4660000000000002},"last":0.879}]},{"osd":5,"last update":"Sat May 20 01:26:34 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.59099999999999997,"15min":0.66600000000000004},"min":{"1min":0.503,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":3.5379999999999998},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.60799999999999998,"15min":0.626},"min":{"1min":0.51400000000000001,"5min":0.41999999999999998,"15min":0.374},"max":{"1min":0.91300000000000003,"5min":1.026,"15min":1.385},"last":0.89700000000000002}]},{"osd":6,"last update":"Sat May 20 01:26:34 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.60699999999999998,"15min":0.629},"min":{"1min":0.44400000000000001,"5min":0.41199999999999998,"15min":0.38100000000000001},"max":{"1min":0.873,"5min":0.99199999999999999,"15min":1.347},"last":0.873},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.60899999999999999,"15min":0.626},"min":{"1min":0.438,"5min":0.374,"15min":0.374},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":1.107},"last":0.94999999999999996}]},{"osd":7,"last update":"Sat May 20 01:26:34 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.60999999999999999,"15min":0.63200000000000001},"min":{"1min":0.46700000000000003,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.92200000000000004,"5min":0.92900000000000005,"15min":1.1359999999999999},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.63700000000000001,"15min":0.70699999999999996},"min":{"1min":0.52300000000000002,"5min":0.436,"15min":0.436},"max":{"1min":0.92300000000000004,"5min":1.0069999999999999,"15min":3.5009999999999999},"last":0.91500000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574974,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":16544,"kb_used_data":1304,"kb_used_omap":0,"kb_used_meta":15232,"kb_avail":94355296,"statfs":{"total":96636764160,"available":96619823104,"internally_reserved":0,"allocated":1335296,"data_stored":682961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":15597568},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:25:33 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.47499999999999998,"15min":0.48199999999999998},"min":{"1min":0.372,"5min":0.247,"15min":0.247},"max":{"1min":0.68799999999999994,"5min":0.73699999999999999,"15min":0.75600000000000001},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.504,"5min":0.503,"15min":0.499},"min":{"1min":0.35299999999999998,"5min":0.30599999999999999,"15min":0.30099999999999999},"max":{"1min":0.71199999999999997,"5min":1.6830000000000001,"15min":1.7989999999999999},"last":0.67300000000000004}]},{"osd":1,"last update":"Sat May 20 01:25:33 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.499,"15min":0.51800000000000002},"min":{"1min":0.35499999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.68400000000000005,"5min":0.746,"15min":0.82799999999999996},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.51700000000000002,"15min":0.52800000000000002},"min":{"1min":0.36799999999999999,"5min":0.316,"15min":0.316},"max":{"1min":0.66500000000000004,"5min":1.7190000000000001,"15min":2.0739999999999998},"last":0.66100000000000003}]},{"osd":2,"last update":"Sat May 20 01:25:33 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.53300000000000003,"15min":0.51200000000000001},"min":{"1min":0.38800000000000001,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53800000000000003,"15min":0.54900000000000004},"min":{"1min":0.42799999999999999,"5min":0.36299999999999999,"15min":0.34000000000000002},"max":{"1min":0.73199999999999998,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.59299999999999997}]},{"osd":3,"last update":"Sat May 20 01:25:33 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.52000000000000002,"15min":0.52200000000000002},"min":{"1min":0.36499999999999999,"5min":0.34899999999999998,"15min":0.34399999999999997},"max":{"1min":0.63600000000000001,"5min":0.81399999999999995,"15min":1.827},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.57599999999999996,"15min":0.64700000000000002},"min":{"1min":0.433,"5min":0.38,"15min":0.38},"max":{"1min":1.093,"5min":1.093,"15min":3.1749999999999998},"last":0.55100000000000005}]},{"osd":5,"last update":"Sat May 20 01:25:33 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.53300000000000003,"15min":0.48999999999999999},"min":{"1min":0.32900000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.76700000000000002,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.505,"5min":0.54700000000000004,"15min":0.51400000000000001},"min":{"1min":0.34699999999999998,"5min":0.33400000000000002,"15min":0.29899999999999999},"max":{"1min":0.70099999999999996,"5min":0.80400000000000005,"15min":0.80500000000000005},"last":0.50800000000000001}]},{"osd":6,"last update":"Sat May 20 01:25:33 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58999999999999997,"15min":0.55100000000000005},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.32400000000000001},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.60699999999999998,"15min":0.59199999999999997},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.80100000000000005,"5min":0.84699999999999998,"15min":0.85899999999999999},"last":0.53100000000000003}]},{"osd":7,"last update":"Sat May 20 01:25:33 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.58099999999999996,"15min":0.55700000000000005},"min":{"1min":0.40899999999999997,"5min":0.36099999999999999,"15min":0.34200000000000003},"max":{"1min":0.67100000000000004,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.626,"5min":0.61799999999999999,"15min":0.59999999999999998},"min":{"1min":0.436,"5min":0.36399999999999999,"15min":0.34000000000000002},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.58299999999999996}]}]},{"osd":5,"up_from":13,"seq":55834574975,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":11036,"kb_used_data":1300,"kb_used_omap":0,"kb_used_meta":9728,"kb_avail":94360804,"statfs":{"total":96636764160,"available":96625463296,"internally_reserved":0,"allocated":1331200,"data_stored":682000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9961472},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:25:38 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.45500000000000002,"15min":0.63300000000000001},"min":{"1min":0.32500000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":6.3929999999999998},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.504,"5min":0.46899999999999997,"15min":0.65600000000000003},"min":{"1min":0.379,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":6.3339999999999996},"last":0.53900000000000003}]},{"osd":1,"last update":"Sat May 20 01:25:38 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.46700000000000003,"15min":0.57499999999999996},"min":{"1min":0.33800000000000002,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":3.2450000000000001},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.51100000000000001,"15min":0.53500000000000003},"min":{"1min":0.371,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":1.1819999999999999},"last":0.51100000000000001}]},{"osd":2,"last update":"Sat May 20 01:25:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.48899999999999999,"15min":0.53700000000000003},"min":{"1min":0.40999999999999998,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.751,"5min":0.751,"15min":1.1639999999999999},"last":0.624},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.49199999999999999,"15min":0.54200000000000004},"min":{"1min":0.39200000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":1.258},"last":0.61399999999999999}]},{"osd":3,"last update":"Sat May 20 01:25:38 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.51700000000000002,"15min":0.70199999999999996},"min":{"1min":0.44500000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":6.351},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.54300000000000004,"15min":0.73399999999999999},"min":{"1min":0.39100000000000001,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":6.3730000000000002},"last":0.58199999999999996}]},{"osd":4,"last update":"Sat May 20 01:25:38 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.51900000000000002,"15min":0.496},"min":{"1min":0.433,"5min":0.309,"15min":0.309},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.70199999999999996},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.41299999999999998,"5min":0.35399999999999998,"15min":0.34200000000000003},"max":{"1min":0.73999999999999999,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.59899999999999998}]},{"osd":6,"last update":"Sat May 20 01:25:38 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.29999999999999999},"max":{"1min":0.77600000000000002,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.55100000000000005,"15min":0.55800000000000005},"min":{"1min":0.432,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.76600000000000001,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.65500000000000003}]},{"osd":7,"last update":"Sat May 20 01:25:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.57799999999999996,"15min":0.58199999999999996},"min":{"1min":0.44,"5min":0.33400000000000002,"15min":0.33000000000000002},"max":{"1min":0.79200000000000004,"5min":0.82299999999999995,"15min":0.91000000000000003},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.59299999999999997,"15min":0.60799999999999998},"min":{"1min":0.53400000000000003,"5min":0.40999999999999998,"15min":0.39500000000000002},"max":{"1min":0.81100000000000005,"5min":0.81399999999999995,"15min":0.94499999999999995},"last":0.71099999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-05-20T01:26:40.617 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-05-20T01:26:40.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.769+0000 7f04e62ac700 1 Processor -- start 2023-05-20T01:26:40.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.769+0000 7f04e62ac700 1 -- start start 2023-05-20T01:26:40.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.770+0000 7f04e62ac700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f04e00ccec0 0x7f04e00713c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:40.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.770+0000 7f04e62ac700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f04e0071900 0x7f04e00ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:40.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.770+0000 7f04e62ac700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04e00cb590 con 0x7f04e0157190 2023-05-20T01:26:40.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.770+0000 7f04e62ac700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04e00ca9a0 con 0x7f04e00ccec0 2023-05-20T01:26:40.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.770+0000 7f04e62ac700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04e00cac60 con 0x7f04e0071900 2023-05-20T01:26:40.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.770+0000 7f04df7fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f04e00ccec0 0x7f04e00713c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:40.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.770+0000 7f04df7fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f04e00ccec0 0x7f04e00713c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38176/0 (socket says 172.21.15.16:38176) 2023-05-20T01:26:40.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.770+0000 7f04dffff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f04e0157190 0x7f04e00cc7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49702/0 (socket says 172.21.15.16:49702) 2023-05-20T01:26:40.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.770+0000 7f04df7fe700 1 -- 172.21.15.16:0/2435750196 learned_addr learned my addr 172.21.15.16:0/2435750196 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:26:40.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.770+0000 7f04e4849700 1 --2- 172.21.15.16:0/2435750196 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f04e0071900 0x7f04e00ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:40.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.771+0000 7f04df7fe700 1 -- 172.21.15.16:0/2435750196 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f04e0071900 msgr2=0x7f04e00ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:40.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.771+0000 7f04df7fe700 1 --2- 172.21.15.16:0/2435750196 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f04e0071900 0x7f04e00ce740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:40.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.771+0000 7f04df7fe700 1 -- 172.21.15.16:0/2435750196 >> v1:172.21.15.16:6789/0 conn(0x7f04e0157190 legacy=0x7f04e00cc7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:40.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.771+0000 7f04df7fe700 1 -- 172.21.15.16:0/2435750196 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04e00caf40 con 0x7f04e00ccec0 2023-05-20T01:26:40.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.771+0000 7f04deffd700 1 -- 172.21.15.16:0/2435750196 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4094025538 0 0) 0x7f04e00cb590 con 0x7f04e0157190 2023-05-20T01:26:40.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.771+0000 7f04df7fe700 1 --2- 172.21.15.16:0/2435750196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04e00ccec0 0x7f04e00713c0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f04d400bb90 tx=0x7f04d400bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b3ef67b98571ea2 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:40.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.771+0000 7f04deffd700 1 -- 172.21.15.16:0/2435750196 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04d400dd50 con 0x7f04e00ccec0 2023-05-20T01:26:40.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.771+0000 7f04deffd700 1 -- 172.21.15.16:0/2435750196 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04d4010470 con 0x7f04e00ccec0 2023-05-20T01:26:40.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.771+0000 7f04deffd700 1 -- 172.21.15.16:0/2435750196 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04d4010800 con 0x7f04e00ccec0 2023-05-20T01:26:40.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.771+0000 7f04e62ac700 1 -- 172.21.15.16:0/2435750196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04e00ccec0 msgr2=0x7f04e00713c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:40.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.771+0000 7f04e62ac700 1 --2- 172.21.15.16:0/2435750196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04e00ccec0 0x7f04e00713c0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f04d400bb90 tx=0x7f04d400bea0 comp rx=0 tx=0).stop 2023-05-20T01:26:40.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.772+0000 7f04e62ac700 1 -- 172.21.15.16:0/2435750196 shutdown_connections 2023-05-20T01:26:40.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.772+0000 7f04e62ac700 1 --2- 172.21.15.16:0/2435750196 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f04e0071900 0x7f04e00ce740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:40.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.772+0000 7f04e62ac700 1 --2- 172.21.15.16:0/2435750196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04e00ccec0 0x7f04e00713c0 secure :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f04d400bb90 tx=0x7f04d400bea0 comp rx=0 tx=0).stop 2023-05-20T01:26:40.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.772+0000 7f04e62ac700 1 -- 172.21.15.16:0/2435750196 >> 172.21.15.16:0/2435750196 conn(0x7f04e00cffb0 msgr2=0x7f04e00d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:40.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.772+0000 7f04e62ac700 1 -- 172.21.15.16:0/2435750196 shutdown_connections 2023-05-20T01:26:40.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.772+0000 7f04e62ac700 1 -- 172.21.15.16:0/2435750196 wait complete. 2023-05-20T01:26:40.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.773+0000 7f04e62ac700 1 Processor -- start 2023-05-20T01:26:40.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.773+0000 7f04e62ac700 1 -- start start 2023-05-20T01:26:40.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.774+0000 7f04e62ac700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f04e0071900 0x7f04e0151f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:40.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.774+0000 7f04e62ac700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04e0157190 0x7f04e00b2560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:40.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.774+0000 7f04e62ac700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f04e0151840 0x7f04e00c7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:40.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.774+0000 7f04e62ac700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04e00afab0 con 0x7f04e0157190 2023-05-20T01:26:40.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.774+0000 7f04e62ac700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04e00b4020 con 0x7f04e0151840 2023-05-20T01:26:40.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.774+0000 7f04e62ac700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04e00b2aa0 con 0x7f04e0071900 2023-05-20T01:26:40.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.774+0000 7f04df7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04e0157190 0x7f04e00b2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:40.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.774+0000 7f04df7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04e0157190 0x7f04e00b2560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38182/0 (socket says 172.21.15.16:38182) 2023-05-20T01:26:40.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.774+0000 7f04df7fe700 1 -- 172.21.15.16:0/2269290689 learned_addr learned my addr 172.21.15.16:0/2269290689 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:26:40.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.774+0000 7f04dffff700 1 --2- 172.21.15.16:0/2269290689 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f04e0071900 0x7f04e0151f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:40.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.774+0000 7f04e4849700 1 --2- 172.21.15.16:0/2269290689 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f04e0151840 0x7f04e00c7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:40.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.774+0000 7f04df7fe700 1 -- 172.21.15.16:0/2269290689 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f04e0071900 msgr2=0x7f04e0151f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:40.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.774+0000 7f04df7fe700 1 --2- 172.21.15.16:0/2269290689 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f04e0071900 0x7f04e0151f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:40.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.774+0000 7f04df7fe700 1 -- 172.21.15.16:0/2269290689 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f04e0151840 msgr2=0x7f04e00c7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:40.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.774+0000 7f04df7fe700 1 --2- 172.21.15.16:0/2269290689 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f04e0151840 0x7f04e00c7080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:40.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.774+0000 7f04df7fe700 1 -- 172.21.15.16:0/2269290689 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04d400b7f0 con 0x7f04e0157190 2023-05-20T01:26:40.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.774+0000 7f04dffff700 1 --2- 172.21.15.16:0/2269290689 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f04e0071900 0x7f04e0151f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:26:40.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.774+0000 7f04e4849700 1 --2- 172.21.15.16:0/2269290689 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f04e0151840 0x7f04e00c7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:26:40.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.774+0000 7f04df7fe700 1 --2- 172.21.15.16:0/2269290689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04e0157190 0x7f04e00b2560 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f04d4000c00 tx=0x7f04d4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:40.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.774+0000 7f04dcff9700 1 -- 172.21.15.16:0/2269290689 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04d401f070 con 0x7f04e0157190 2023-05-20T01:26:40.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.775+0000 7f04dcff9700 1 -- 172.21.15.16:0/2269290689 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04d40038b0 con 0x7f04e0157190 2023-05-20T01:26:40.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.775+0000 7f04dcff9700 1 -- 172.21.15.16:0/2269290689 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04d4003c20 con 0x7f04e0157190 2023-05-20T01:26:40.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.775+0000 7f04e62ac700 1 -- 172.21.15.16:0/2269290689 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04e0157560 con 0x7f04e0157190 2023-05-20T01:26:40.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.775+0000 7f04e62ac700 1 -- 172.21.15.16:0/2269290689 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04e00cb220 con 0x7f04e0157190 2023-05-20T01:26:40.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.775+0000 7f04e62ac700 1 -- 172.21.15.16:0/2269290689 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04e00cba20 con 0x7f04e0157190 2023-05-20T01:26:40.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.776+0000 7f04dcff9700 1 -- 172.21.15.16:0/2269290689 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f04d4019400 con 0x7f04e0157190 2023-05-20T01:26:40.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.776+0000 7f04dcff9700 1 --2- 172.21.15.16:0/2269290689 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f04cc06ace0 0x7f04cc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:40.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.776+0000 7f04dcff9700 1 -- 172.21.15.16:0/2269290689 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f04d408c9f0 con 0x7f04e0157190 2023-05-20T01:26:40.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.776+0000 7f04dffff700 1 --2- 172.21.15.16:0/2269290689 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f04cc06ace0 0x7f04cc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:40.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.777+0000 7f04dffff700 1 --2- 172.21.15.16:0/2269290689 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f04cc06ace0 0x7f04cc06d190 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f04d0000f60 tx=0x7f04d0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:40.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.781+0000 7f04dcff9700 1 -- 172.21.15.16:0/2269290689 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f04d40579f0 con 0x7f04e0157190 2023-05-20T01:26:40.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.974+0000 7f04e62ac700 1 -- 172.21.15.16:0/2269290689 --> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f04e00747b0 con 0x7f04cc06ace0 2023-05-20T01:26:40.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.982+0000 7f04dcff9700 1 -- 172.21.15.16:0/2269290689 <== mgr.4104 v2:172.21.15.16:6832/3817237998 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+299805 (secure 0 0 0) 0x7f04e00747b0 con 0x7f04cc06ace0 2023-05-20T01:26:40.991 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:26:40.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.985+0000 7f04e62ac700 1 -- 172.21.15.16:0/2269290689 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f04cc06ace0 msgr2=0x7f04cc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:40.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.985+0000 7f04e62ac700 1 --2- 172.21.15.16:0/2269290689 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f04cc06ace0 0x7f04cc06d190 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f04d0000f60 tx=0x7f04d0009040 comp rx=0 tx=0).stop 2023-05-20T01:26:40.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.985+0000 7f04e62ac700 1 -- 172.21.15.16:0/2269290689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04e0157190 msgr2=0x7f04e00b2560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:40.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.985+0000 7f04e62ac700 1 --2- 172.21.15.16:0/2269290689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04e0157190 0x7f04e00b2560 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f04d4000c00 tx=0x7f04d4004750 comp rx=0 tx=0).stop 2023-05-20T01:26:40.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.985+0000 7f04e62ac700 1 -- 172.21.15.16:0/2269290689 shutdown_connections 2023-05-20T01:26:40.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.985+0000 7f04e62ac700 1 --2- 172.21.15.16:0/2269290689 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f04e0151840 0x7f04e00c7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:40.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.985+0000 7f04e62ac700 1 --2- 172.21.15.16:0/2269290689 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f04cc06ace0 0x7f04cc06d190 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:40.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.985+0000 7f04e62ac700 1 --2- 172.21.15.16:0/2269290689 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f04e0071900 0x7f04e0151f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:40.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.985+0000 7f04e62ac700 1 --2- 172.21.15.16:0/2269290689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04e0157190 0x7f04e00b2560 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:40.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.985+0000 7f04e62ac700 1 -- 172.21.15.16:0/2269290689 >> 172.21.15.16:0/2269290689 conn(0x7f04e00cffb0 msgr2=0x7f04e006ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:40.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.985+0000 7f04e62ac700 1 -- 172.21.15.16:0/2269290689 shutdown_connections 2023-05-20T01:26:40.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:40.985+0000 7f04e62ac700 1 -- 172.21.15.16:0/2269290689 wait complete. 2023-05-20T01:26:40.995 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2023-05-20T01:26:41.002 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":415,"stamp":"2023-05-20T01:26:39.454792+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":30,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":110204,"kb_used_data":11644,"kb_used_omap":0,"kb_used_meta":98496,"kb_avail":754864516,"statfs":{"total":773094113280,"available":772981264384,"internally_reserved":0,"allocated":11923456,"data_stored":6674952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":100859904},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.819728"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-05-20T01:26:34.638110+0000","last_change":"2023-05-20T01:26:34.638110+0000","last_active":"2023-05-20T01:26:34.630120+0000","last_peered":"2023-05-20T01:26:34.630120+0000","last_clean":"2023-05-20T01:26:34.630120+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:34.638110+0000","last_undegraded":"2023-05-20T01:26:34.638110+0000","last_fullsized":"2023-05-20T01:26:34.638110+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-05-20T01:26:34.639497+0000","last_change":"2023-05-20T01:26:34.639497+0000","last_active":"2023-05-20T01:26:34.630120+0000","last_peered":"2023-05-20T01:26:34.630120+0000","last_clean":"2023-05-20T01:26:34.630120+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:34.639497+0000","last_undegraded":"2023-05-20T01:26:34.639497+0000","last_fullsized":"2023-05-20T01:26:34.639497+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-05-20T01:26:34.644602+0000","last_change":"2023-05-20T01:26:34.644602+0000","last_active":"2023-05-20T01:26:34.630120+0000","last_peered":"2023-05-20T01:26:34.630120+0000","last_clean":"2023-05-20T01:26:34.630120+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:34.644602+0000","last_undegraded":"2023-05-20T01:26:34.644602+0000","last_fullsized":"2023-05-20T01:26:34.644602+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-05-20T01:26:34.642067+0000","last_change":"2023-05-20T01:26:34.642067+0000","last_active":"2023-05-20T01:26:34.630120+0000","last_peered":"2023-05-20T01:26:34.630120+0000","last_clean":"2023-05-20T01:26:34.630120+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:34.642067+0000","last_undegraded":"2023-05-20T01:26:34.642067+0000","last_fullsized":"2023-05-20T01:26:34.642067+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-05-20T01:26:34.640655+0000","last_change":"2023-05-20T01:26:34.640655+0000","last_active":"2023-05-20T01:26:34.630120+0000","last_peered":"2023-05-20T01:26:34.630120+0000","last_clean":"2023-05-20T01:26:34.630120+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:34.640655+0000","last_undegraded":"2023-05-20T01:26:34.640655+0000","last_fullsized":"2023-05-20T01:26:34.640655+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-05-20T01:26:34.641569+0000","last_change":"2023-05-20T01:26:34.641569+0000","last_active":"2023-05-20T01:26:34.630120+0000","last_peered":"2023-05-20T01:26:34.630120+0000","last_clean":"2023-05-20T01:26:34.630120+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:34.641569+0000","last_undegraded":"2023-05-20T01:26:34.641569+0000","last_fullsized":"2023-05-20T01:26:34.641569+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-05-20T01:26:34.644015+0000","last_change":"2023-05-20T01:26:34.644015+0000","last_active":"2023-05-20T01:26:34.630120+0000","last_peered":"2023-05-20T01:26:34.630120+0000","last_clean":"2023-05-20T01:26:34.630120+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:34.644015+0000","last_undegraded":"2023-05-20T01:26:34.644015+0000","last_fullsized":"2023-05-20T01:26:34.644015+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-05-20T01:26:34.638480+0000","last_change":"2023-05-20T01:26:34.638480+0000","last_active":"2023-05-20T01:26:34.630120+0000","last_peered":"2023-05-20T01:26:34.630120+0000","last_clean":"2023-05-20T01:26:34.630120+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:34.638480+0000","last_undegraded":"2023-05-20T01:26:34.638480+0000","last_fullsized":"2023-05-20T01:26:34.638480+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-05-20T01:26:34.644724+0000","last_change":"2023-05-20T01:26:34.644724+0000","last_active":"2023-05-20T01:26:34.630120+0000","last_peered":"2023-05-20T01:26:34.630120+0000","last_clean":"2023-05-20T01:26:34.630120+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:34.644724+0000","last_undegraded":"2023-05-20T01:26:34.644724+0000","last_fullsized":"2023-05-20T01:26:34.644724+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-05-20T01:26:34.639019+0000","last_change":"2023-05-20T01:26:34.639019+0000","last_active":"2023-05-20T01:26:34.630120+0000","last_peered":"2023-05-20T01:26:34.630120+0000","last_clean":"2023-05-20T01:26:34.630120+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:34.639019+0000","last_undegraded":"2023-05-20T01:26:34.639019+0000","last_fullsized":"2023-05-20T01:26:34.639019+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-05-20T01:26:34.641727+0000","last_change":"2023-05-20T01:26:34.641727+0000","last_active":"2023-05-20T01:26:34.630120+0000","last_peered":"2023-05-20T01:26:34.630120+0000","last_clean":"2023-05-20T01:26:34.630120+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:34.641727+0000","last_undegraded":"2023-05-20T01:26:34.641727+0000","last_fullsized":"2023-05-20T01:26:34.641727+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-05-20T01:26:34.640629+0000","last_change":"2023-05-20T01:26:34.640629+0000","last_active":"2023-05-20T01:26:34.630120+0000","last_peered":"2023-05-20T01:26:34.630120+0000","last_clean":"2023-05-20T01:26:34.630120+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:34.640629+0000","last_undegraded":"2023-05-20T01:26:34.640629+0000","last_fullsized":"2023-05-20T01:26:34.640629+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-05-20T01:26:34.641472+0000","last_change":"2023-05-20T01:26:34.641472+0000","last_active":"2023-05-20T01:26:34.630120+0000","last_peered":"2023-05-20T01:26:34.630120+0000","last_clean":"2023-05-20T01:26:34.630120+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:34.641472+0000","last_undegraded":"2023-05-20T01:26:34.641472+0000","last_fullsized":"2023-05-20T01:26:34.641472+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-05-20T01:26:34.638486+0000","last_change":"2023-05-20T01:26:34.638486+0000","last_active":"2023-05-20T01:26:34.630120+0000","last_peered":"2023-05-20T01:26:34.630120+0000","last_clean":"2023-05-20T01:26:34.630120+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:34.638486+0000","last_undegraded":"2023-05-20T01:26:34.638486+0000","last_fullsized":"2023-05-20T01:26:34.638486+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-05-20T01:26:34.638266+0000","last_change":"2023-05-20T01:26:34.638266+0000","last_active":"2023-05-20T01:26:34.630120+0000","last_peered":"2023-05-20T01:26:34.630120+0000","last_clean":"2023-05-20T01:26:34.630120+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:34.638266+0000","last_undegraded":"2023-05-20T01:26:34.638266+0000","last_fullsized":"2023-05-20T01:26:34.638266+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-05-20T01:26:34.643913+0000","last_change":"2023-05-20T01:26:34.643913+0000","last_active":"2023-05-20T01:26:34.630120+0000","last_peered":"2023-05-20T01:26:34.630120+0000","last_clean":"2023-05-20T01:26:34.630120+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:34.643913+0000","last_undegraded":"2023-05-20T01:26:34.643913+0000","last_fullsized":"2023-05-20T01:26:34.643913+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-05-20T01:26:34.641466+0000","last_change":"2023-05-20T01:26:34.641466+0000","last_active":"2023-05-20T01:26:34.630120+0000","last_peered":"2023-05-20T01:26:34.630120+0000","last_clean":"2023-05-20T01:26:34.630120+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:34.641466+0000","last_undegraded":"2023-05-20T01:26:34.641466+0000","last_fullsized":"2023-05-20T01:26:34.641466+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-05-20T01:26:34.638503+0000","last_change":"2023-05-20T01:26:34.638503+0000","last_active":"2023-05-20T01:26:34.630120+0000","last_peered":"2023-05-20T01:26:34.630120+0000","last_clean":"2023-05-20T01:26:34.630120+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:34.638503+0000","last_undegraded":"2023-05-20T01:26:34.638503+0000","last_fullsized":"2023-05-20T01:26:34.638503+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-05-20T01:26:34.640567+0000","last_change":"2023-05-20T01:26:34.640567+0000","last_active":"2023-05-20T01:26:34.630120+0000","last_peered":"2023-05-20T01:26:34.630120+0000","last_clean":"2023-05-20T01:26:34.630120+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:34.640567+0000","last_undegraded":"2023-05-20T01:26:34.640567+0000","last_fullsized":"2023-05-20T01:26:34.640567+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-05-20T01:26:34.641446+0000","last_change":"2023-05-20T01:26:34.641446+0000","last_active":"2023-05-20T01:26:34.630120+0000","last_peered":"2023-05-20T01:26:34.630120+0000","last_clean":"2023-05-20T01:26:34.630120+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:34.641446+0000","last_undegraded":"2023-05-20T01:26:34.641446+0000","last_fullsized":"2023-05-20T01:26:34.641446+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-05-20T01:26:34.644021+0000","last_change":"2023-05-20T01:26:34.644021+0000","last_active":"2023-05-20T01:26:34.630120+0000","last_peered":"2023-05-20T01:26:34.630120+0000","last_clean":"2023-05-20T01:26:34.630120+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:34.644021+0000","last_undegraded":"2023-05-20T01:26:34.644021+0000","last_fullsized":"2023-05-20T01:26:34.644021+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-05-20T01:26:34.638401+0000","last_change":"2023-05-20T01:26:34.638401+0000","last_active":"2023-05-20T01:26:34.630120+0000","last_peered":"2023-05-20T01:26:34.630120+0000","last_clean":"2023-05-20T01:26:34.630120+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:34.638401+0000","last_undegraded":"2023-05-20T01:26:34.638401+0000","last_fullsized":"2023-05-20T01:26:34.638401+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-05-20T01:26:34.640200+0000","last_change":"2023-05-20T01:26:34.640200+0000","last_active":"2023-05-20T01:26:34.630120+0000","last_peered":"2023-05-20T01:26:34.630120+0000","last_clean":"2023-05-20T01:26:34.630120+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:34.640200+0000","last_undegraded":"2023-05-20T01:26:34.640200+0000","last_fullsized":"2023-05-20T01:26:34.640200+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-05-20T01:26:34.638339+0000","last_change":"2023-05-20T01:26:34.638339+0000","last_active":"2023-05-20T01:26:34.630120+0000","last_peered":"2023-05-20T01:26:34.630120+0000","last_clean":"2023-05-20T01:26:34.630120+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:34.638339+0000","last_undegraded":"2023-05-20T01:26:34.638339+0000","last_fullsized":"2023-05-20T01:26:34.638339+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-05-20T01:26:34.642905+0000","last_change":"2023-05-20T01:26:34.642905+0000","last_active":"2023-05-20T01:26:34.630120+0000","last_peered":"2023-05-20T01:26:34.630120+0000","last_clean":"2023-05-20T01:26:34.630120+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:34.642905+0000","last_undegraded":"2023-05-20T01:26:34.642905+0000","last_fullsized":"2023-05-20T01:26:34.642905+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":191,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-05-20T01:26:34.645291+0000","last_change":"2023-05-20T01:16:23.289068+0000","last_active":"2023-05-20T01:26:34.645291+0000","last_peered":"2023-05-20T01:26:34.645291+0000","last_clean":"2023-05-20T01:26:34.645291+0000","last_became_active":"2023-05-20T01:16:23.288781+0000","last_became_peered":"2023-05-20T01:16:23.288781+0000","last_unstale":"2023-05-20T01:26:34.645291+0000","last_undegraded":"2023-05-20T01:26:34.645291+0000","last_fullsized":"2023-05-20T01:26:34.645291+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:23:26.310250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-05-20T01:26:34.639380+0000","last_change":"2023-05-20T01:26:34.639380+0000","last_active":"2023-05-20T01:26:34.630120+0000","last_peered":"2023-05-20T01:26:34.630120+0000","last_clean":"2023-05-20T01:26:34.630120+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:34.639380+0000","last_undegraded":"2023-05-20T01:26:34.639380+0000","last_fullsized":"2023-05-20T01:26:34.639380+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":199,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-05-20T01:26:34.637011+0000","last_change":"2023-05-20T01:16:22.829324+0000","last_active":"2023-05-20T01:26:34.637011+0000","last_peered":"2023-05-20T01:26:34.637011+0000","last_clean":"2023-05-20T01:26:34.637011+0000","last_became_active":"2023-05-20T01:16:22.828964+0000","last_became_peered":"2023-05-20T01:16:22.828964+0000","last_unstale":"2023-05-20T01:26:34.637011+0000","last_undegraded":"2023-05-20T01:26:34.637011+0000","last_fullsized":"2023-05-20T01:26:34.637011+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:23:48.805679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-05-20T01:26:34.641191+0000","last_change":"2023-05-20T01:26:34.641191+0000","last_active":"2023-05-20T01:26:34.630120+0000","last_peered":"2023-05-20T01:26:34.630120+0000","last_clean":"2023-05-20T01:26:34.630120+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:34.641191+0000","last_undegraded":"2023-05-20T01:26:34.641191+0000","last_fullsized":"2023-05-20T01:26:34.641191+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":175,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-05-20T01:26:34.642808+0000","last_change":"2023-05-20T01:16:22.834754+0000","last_active":"2023-05-20T01:26:34.642808+0000","last_peered":"2023-05-20T01:26:34.642808+0000","last_clean":"2023-05-20T01:26:34.642808+0000","last_became_active":"2023-05-20T01:16:22.834437+0000","last_became_peered":"2023-05-20T01:16:22.834437+0000","last_unstale":"2023-05-20T01:26:34.642808+0000","last_undegraded":"2023-05-20T01:26:34.642808+0000","last_fullsized":"2023-05-20T01:26:34.642808+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T01:22:38.911204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":310,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-05-20T01:26:34.642091+0000","last_change":"2023-05-20T01:16:22.834920+0000","last_active":"2023-05-20T01:26:34.642091+0000","last_peered":"2023-05-20T01:26:34.642091+0000","last_clean":"2023-05-20T01:26:34.642091+0000","last_became_active":"2023-05-20T01:16:22.833924+0000","last_became_peered":"2023-05-20T01:16:22.833924+0000","last_unstale":"2023-05-20T01:26:34.642091+0000","last_undegraded":"2023-05-20T01:26:34.642091+0000","last_fullsized":"2023-05-20T01:26:34.642091+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:02:13.281571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-05-20T01:26:34.642061+0000","last_change":"2023-05-20T01:26:34.642061+0000","last_active":"2023-05-20T01:26:34.630120+0000","last_peered":"2023-05-20T01:26:34.630120+0000","last_clean":"2023-05-20T01:26:34.630120+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:34.642061+0000","last_undegraded":"2023-05-20T01:26:34.642061+0000","last_fullsized":"2023-05-20T01:26:34.642061+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":159,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-05-20T01:26:34.642202+0000","last_change":"2023-05-20T01:16:23.290259+0000","last_active":"2023-05-20T01:26:34.642202+0000","last_peered":"2023-05-20T01:26:34.642202+0000","last_clean":"2023-05-20T01:26:34.642202+0000","last_became_active":"2023-05-20T01:16:23.289911+0000","last_became_peered":"2023-05-20T01:16:23.289911+0000","last_unstale":"2023-05-20T01:26:34.642202+0000","last_undegraded":"2023-05-20T01:26:34.642202+0000","last_fullsized":"2023-05-20T01:26:34.642202+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:40:10.726971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-05-20T01:26:34.638510+0000","last_change":"2023-05-20T01:26:34.638510+0000","last_active":"2023-05-20T01:26:34.630120+0000","last_peered":"2023-05-20T01:26:34.630120+0000","last_clean":"2023-05-20T01:26:34.630120+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:34.638510+0000","last_undegraded":"2023-05-20T01:26:34.638510+0000","last_fullsized":"2023-05-20T01:26:34.638510+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":160,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-05-20T01:26:34.642159+0000","last_change":"2023-05-20T01:16:23.290207+0000","last_active":"2023-05-20T01:26:34.642159+0000","last_peered":"2023-05-20T01:26:34.642159+0000","last_clean":"2023-05-20T01:26:34.642159+0000","last_became_active":"2023-05-20T01:16:23.289928+0000","last_became_peered":"2023-05-20T01:16:23.289928+0000","last_unstale":"2023-05-20T01:26:34.642159+0000","last_undegraded":"2023-05-20T01:26:34.642159+0000","last_fullsized":"2023-05-20T01:26:34.642159+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:05:09.399167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-05-20T01:26:34.640347+0000","last_change":"2023-05-20T01:26:34.640347+0000","last_active":"2023-05-20T01:26:34.630120+0000","last_peered":"2023-05-20T01:26:34.630120+0000","last_clean":"2023-05-20T01:26:34.630120+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:34.640347+0000","last_undegraded":"2023-05-20T01:26:34.640347+0000","last_fullsized":"2023-05-20T01:26:34.640347+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":161,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-05-20T01:26:34.642350+0000","last_change":"2023-05-20T01:16:23.290538+0000","last_active":"2023-05-20T01:26:34.642350+0000","last_peered":"2023-05-20T01:26:34.642350+0000","last_clean":"2023-05-20T01:26:34.642350+0000","last_became_active":"2023-05-20T01:16:23.290230+0000","last_became_peered":"2023-05-20T01:16:23.290230+0000","last_unstale":"2023-05-20T01:26:34.642350+0000","last_undegraded":"2023-05-20T01:26:34.642350+0000","last_fullsized":"2023-05-20T01:26:34.642350+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:47:13.453162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-05-20T01:26:34.638985+0000","last_change":"2023-05-20T01:26:34.638985+0000","last_active":"2023-05-20T01:26:34.630120+0000","last_peered":"2023-05-20T01:26:34.630120+0000","last_clean":"2023-05-20T01:26:34.630120+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:34.638985+0000","last_undegraded":"2023-05-20T01:26:34.638985+0000","last_fullsized":"2023-05-20T01:26:34.638985+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":175,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-05-20T01:26:34.641407+0000","last_change":"2023-05-20T01:16:23.290341+0000","last_active":"2023-05-20T01:26:34.641407+0000","last_peered":"2023-05-20T01:26:34.641407+0000","last_clean":"2023-05-20T01:26:34.641407+0000","last_became_active":"2023-05-20T01:16:23.290053+0000","last_became_peered":"2023-05-20T01:16:23.290053+0000","last_unstale":"2023-05-20T01:26:34.641407+0000","last_undegraded":"2023-05-20T01:26:34.641407+0000","last_fullsized":"2023-05-20T01:26:34.641407+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:17:08.936381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-05-20T01:26:35.634998+0000","last_change":"2023-05-20T01:26:35.634998+0000","last_active":"2023-05-20T01:26:35.634998+0000","last_peered":"2023-05-20T01:26:35.634998+0000","last_clean":"2023-05-20T01:26:35.634998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:35.634998+0000","last_undegraded":"2023-05-20T01:26:35.634998+0000","last_fullsized":"2023-05-20T01:26:35.634998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-05-20T01:26:34.640797+0000","last_change":"2023-05-20T01:26:34.640797+0000","last_active":"2023-05-20T01:26:34.630120+0000","last_peered":"2023-05-20T01:26:34.630120+0000","last_clean":"2023-05-20T01:26:34.630120+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-05-20T01:26:34.640797+0000","last_undegraded":"2023-05-20T01:26:34.640797+0000","last_fullsized":"2023-05-20T01:26:34.640797+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":175,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-05-20T01:26:34.641451+0000","last_change":"2023-05-20T01:16:22.834330+0000","last_active":"2023-05-20T01:26:34.641451+0000","last_peered":"2023-05-20T01:26:34.641451+0000","last_clean":"2023-05-20T01:26:34.641451+0000","last_became_active":"2023-05-20T01:16:22.833946+0000","last_became_peered":"2023-05-20T01:16:22.833946+0000","last_unstale":"2023-05-20T01:26:34.641451+0000","last_undegraded":"2023-05-20T01:26:34.641451+0000","last_fullsized":"2023-05-20T01:26:34.641451+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:23:50.042871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574975,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":16432,"kb_used_data":1896,"kb_used_omap":0,"kb_used_meta":14528,"kb_avail":94355408,"statfs":{"total":96636764160,"available":96619937792,"internally_reserved":0,"allocated":1941504,"data_stored":1279819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":14876672},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:25:37 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.48599999999999999,"15min":0.51400000000000001},"min":{"1min":0.30199999999999999,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.88200000000000001},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.51000000000000001,"15min":0.54700000000000004},"min":{"1min":0.39200000000000002,"5min":0.309,"15min":0.30599999999999999},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.90000000000000002},"last":0.64500000000000002}]},{"osd":1,"last update":"Sat May 20 01:25:37 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.497,"15min":0.72199999999999998},"min":{"1min":0.371,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":8.8879999999999999},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.53200000000000003,"15min":0.76000000000000001},"min":{"1min":0.44800000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":8.875},"last":0.753}]},{"osd":2,"last update":"Sat May 20 01:25:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.502,"15min":0.70899999999999996},"min":{"1min":0.32600000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":8.9120000000000008},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.50800000000000001,"15min":0.71399999999999997},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":8.8520000000000003},"last":0.58799999999999997}]},{"osd":3,"last update":"Sat May 20 01:25:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.498,"15min":0.76100000000000001},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":10.884},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.55400000000000005,"15min":0.81699999999999995},"min":{"1min":0.40699999999999997,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":10.952},"last":0.65500000000000003}]},{"osd":4,"last update":"Sat May 20 01:25:37 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.51000000000000001,"15min":0.51200000000000001},"min":{"1min":0.35899999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.52900000000000003,"15min":0.53100000000000003},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.875},"last":0.53000000000000003}]},{"osd":5,"last update":"Sat May 20 01:25:37 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.56499999999999995,"15min":0.54700000000000004},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.56999999999999995,"15min":0.57799999999999996},"min":{"1min":0.39700000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.46500000000000002}]},{"osd":6,"last update":"Sat May 20 01:25:37 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.59499999999999997,"15min":0.59799999999999998},"min":{"1min":0.41999999999999998,"5min":0.41599999999999998,"15min":0.378},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.57199999999999995,"15min":0.58499999999999996},"min":{"1min":0.39000000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.47999999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574974,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":11116,"kb_used_data":1316,"kb_used_omap":0,"kb_used_meta":9792,"kb_avail":94360724,"statfs":{"total":96636764160,"available":96625381376,"internally_reserved":0,"allocated":1347584,"data_stored":689451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":10027008},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:25:42 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.44800000000000001,"15min":0.47699999999999998},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.26900000000000002},"max":{"1min":0.60999999999999999,"5min":0.72699999999999998,"15min":0.77200000000000002},"last":0.44},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.47899999999999998,"15min":0.51700000000000002},"min":{"1min":0.32900000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.504}]},{"osd":1,"last update":"Sat May 20 01:25:42 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.48699999999999999,"15min":0.51900000000000002},"min":{"1min":0.32800000000000001,"5min":0.29699999999999999,"15min":0.28000000000000003},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.83799999999999997},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.51700000000000002,"15min":0.52600000000000002},"min":{"1min":0.379,"5min":0.32600000000000001,"15min":0.318},"max":{"1min":0.66500000000000004,"5min":0.749,"15min":0.79200000000000004},"last":0.42199999999999999}]},{"osd":2,"last update":"Sat May 20 01:25:42 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.48399999999999999,"15min":0.503},"min":{"1min":0.35099999999999998,"5min":0.34399999999999997,"15min":0.33700000000000002},"max":{"1min":0.57099999999999995,"5min":0.72199999999999998,"15min":0.76600000000000001},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.51600000000000001,"15min":0.54900000000000004},"min":{"1min":0.37,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.745,"5min":0.76200000000000001,"15min":0.76800000000000002},"last":0.53800000000000003}]},{"osd":3,"last update":"Sat May 20 01:25:42 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.52500000000000002,"15min":0.53600000000000003},"min":{"1min":0.378,"5min":0.311,"15min":0.311},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.52000000000000002,"15min":0.53900000000000003},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.72699999999999998,"5min":0.73199999999999998,"15min":0.79500000000000004},"last":0.51600000000000001}]},{"osd":4,"last update":"Sat May 20 01:25:42 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.53100000000000003,"15min":0.54400000000000004},"min":{"1min":0.34799999999999998,"5min":0.33300000000000002,"15min":0.32600000000000001},"max":{"1min":0.70699999999999996,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.54000000000000004,"15min":0.56299999999999994},"min":{"1min":0.40500000000000003,"5min":0.38100000000000001,"15min":0.32700000000000001},"max":{"1min":0.71799999999999997,"5min":0.77400000000000002,"15min":0.80100000000000005},"last":0.58999999999999997}]},{"osd":5,"last update":"Sat May 20 01:25:42 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.53400000000000003,"15min":0.54000000000000004},"min":{"1min":0.38,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.63600000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.56499999999999995,"15min":0.59799999999999998},"min":{"1min":0.372,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.64400000000000002,"5min":0.81599999999999995,"15min":0.89400000000000002},"last":0.57599999999999996}]},{"osd":7,"last update":"Sat May 20 01:25:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.57099999999999995,"15min":0.59399999999999997},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.71299999999999997,"5min":0.80200000000000005,"15min":0.92900000000000005},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.57599999999999996,"15min":0.60499999999999998},"min":{"1min":0.45100000000000001,"5min":0.379,"15min":0.379},"max":{"1min":0.66600000000000004,"5min":0.82799999999999996,"15min":0.85199999999999998},"last":0.60299999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574974,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":17264,"kb_used_data":1896,"kb_used_omap":0,"kb_used_meta":15360,"kb_avail":94354576,"statfs":{"total":96636764160,"available":96619085824,"internally_reserved":0,"allocated":1941504,"data_stored":1279819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":15728640},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:25:36 2023","interfaces":[{"interface":"back","average":{"1min":0.30599999999999999,"5min":0.28299999999999997,"15min":0.311},"min":{"1min":0.18099999999999999,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":0.48799999999999999,"5min":0.57099999999999995,"15min":1.194},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.32400000000000001,"15min":0.31900000000000001},"min":{"1min":0.23899999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.75800000000000001},"last":0.40600000000000003}]},{"osd":2,"last update":"Sat May 20 01:25:36 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.307,"15min":0.31900000000000001},"min":{"1min":0.23100000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.66400000000000003},"last":0.442},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.374,"15min":0.40400000000000003},"min":{"1min":0.313,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":1.3580000000000001},"last":0.52400000000000002}]},{"osd":3,"last update":"Sat May 20 01:25:36 2023","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.35899999999999999,"15min":0.371},"min":{"1min":0.27400000000000002,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":1.619},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.35299999999999998,"15min":0.379},"min":{"1min":0.25800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.57699999999999996,"5min":0.60899999999999999,"15min":1.8400000000000001},"last":0.497}]},{"osd":4,"last update":"Sat May 20 01:25:36 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.55600000000000005,"15min":0.57899999999999996},"min":{"1min":0.36799999999999999,"5min":0.34899999999999998,"15min":0.32200000000000001},"max":{"1min":0.76100000000000001,"5min":0.81699999999999995,"15min":1.2769999999999999},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.57099999999999995,"15min":0.58299999999999996},"min":{"1min":0.38,"5min":0.312,"15min":0.312},"max":{"1min":0.81599999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.63600000000000001}]},{"osd":5,"last update":"Sat May 20 01:25:36 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.58699999999999997,"15min":0.59099999999999997},"min":{"1min":0.46600000000000003,"5min":0.37,"15min":0.37},"max":{"1min":0.78500000000000003,"5min":0.81899999999999995,"15min":1.4510000000000001},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.58199999999999996,"15min":0.60299999999999998},"min":{"1min":0.39700000000000002,"5min":0.39100000000000001,"15min":0.378},"max":{"1min":0.76800000000000002,"5min":0.78800000000000003,"15min":1.409},"last":0.59999999999999998}]},{"osd":6,"last update":"Sat May 20 01:25:36 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.60299999999999998},"min":{"1min":0.39000000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.82299999999999995,"5min":0.876,"15min":1.575},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.58199999999999996,"15min":0.59399999999999997},"min":{"1min":0.45400000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":1.0600000000000001},"last":0.61399999999999999}]},{"osd":7,"last update":"Sat May 20 01:25:36 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.60399999999999998},"min":{"1min":0.41099999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":1.496},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.63,"15min":0.64000000000000001},"min":{"1min":0.44500000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.5009999999999999,"5min":2.1709999999999998,"15min":2.1709999999999998},"last":0.66900000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574974,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":11176,"kb_used_data":1312,"kb_used_omap":0,"kb_used_meta":9856,"kb_avail":94360664,"statfs":{"total":96636764160,"available":96625319936,"internally_reserved":0,"allocated":1343488,"data_stored":688490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":10092544},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat May 20 01:25:35 2023","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.29799999999999999,"15min":0.32000000000000001},"min":{"1min":0.188,"5min":0.17999999999999999,"15min":0.17999999999999999},"max":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.314,"15min":0.34399999999999997},"min":{"1min":0.23999999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":1.502,"5min":1.502,"15min":1.502},"last":0.23400000000000001}]},{"osd":2,"last update":"Sat May 20 01:25:35 2023","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.30299999999999999,"15min":0.32800000000000001},"min":{"1min":0.20899999999999999,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.55700000000000005},"last":0.28299999999999997},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.36199999999999999,"15min":0.38500000000000001},"min":{"1min":0.29599999999999999,"5min":0.216,"15min":0.216},"max":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.314}]},{"osd":3,"last update":"Sat May 20 01:25:35 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.36399999999999999,"15min":0.37},"min":{"1min":0.23499999999999999,"5min":0.22,"15min":0.22},"max":{"1min":4.258,"5min":4.258,"15min":4.258},"last":0.27100000000000002},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.35199999999999998,"15min":0.36599999999999999},"min":{"1min":0.25700000000000001,"5min":0.245,"15min":0.245},"max":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.60899999999999999},"last":0.35899999999999999}]},{"osd":4,"last update":"Sat May 20 01:25:35 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.57199999999999995,"15min":0.55500000000000005},"min":{"1min":0.373,"5min":0.372,"15min":0.34399999999999997},"max":{"1min":1.4330000000000001,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.58499999999999996,"15min":0.57199999999999995},"min":{"1min":0.42799999999999999,"5min":0.39200000000000002,"15min":0.36299999999999999},"max":{"1min":1.298,"5min":1.298,"15min":1.298},"last":0.61499999999999999}]},{"osd":5,"last update":"Sat May 20 01:25:35 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.57399999999999995,"15min":0.57699999999999996},"min":{"1min":0.40600000000000003,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.79900000000000004,"5min":0.80500000000000005,"15min":0.88100000000000001},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.60099999999999998,"15min":0.59399999999999997},"min":{"1min":0.41299999999999998,"5min":0.374,"15min":0.374},"max":{"1min":1.4710000000000001,"5min":1.4710000000000001,"15min":4.4720000000000004},"last":0.65700000000000003}]},{"osd":6,"last update":"Sat May 20 01:25:35 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.68200000000000005,"15min":0.625},"min":{"1min":0.36899999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.79800000000000004,"5min":10.042999999999999,"15min":10.042999999999999},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.60599999999999998,"15min":0.61699999999999999},"min":{"1min":0.40200000000000002,"5min":0.377,"15min":0.377},"max":{"1min":1.367,"5min":1.367,"15min":1.367},"last":0.59099999999999997}]},{"osd":7,"last update":"Sat May 20 01:25:35 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.60699999999999998,"15min":0.59799999999999998},"min":{"1min":0.443,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.399,"5min":1.399,"15min":1.399},"last":0.628},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.72699999999999998,"15min":0.65800000000000003},"min":{"1min":0.439,"5min":0.439,"15min":0.42099999999999999},"max":{"1min":1.4490000000000001,"5min":10.085000000000001,"15min":10.085000000000001},"last":0.56699999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574974,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":11180,"kb_used_data":1316,"kb_used_omap":0,"kb_used_meta":9856,"kb_avail":94360660,"statfs":{"total":96636764160,"available":96625315840,"internally_reserved":0,"allocated":1347584,"data_stored":689451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":10092544},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.33700000000000002,"5min":0.28299999999999997,"15min":0.32300000000000001},"min":{"1min":0.224,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":0.50900000000000001,"5min":0.53400000000000003,"15min":1.038},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.32300000000000001,"5min":0.29899999999999999,"15min":0.33900000000000002},"min":{"1min":0.21199999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":1.079},"last":0.26200000000000001}]},{"osd":1,"last update":"Sat May 20 01:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.317,"15min":0.376},"min":{"1min":0.253,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.44500000000000001,"5min":0.61399999999999999,"15min":1.4950000000000001},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.35899999999999999,"15min":0.41299999999999998},"min":{"1min":0.28399999999999997,"5min":0.249,"15min":0.249},"max":{"1min":0.60799999999999998,"5min":0.629,"15min":1.5189999999999999},"last":0.33000000000000002}]},{"osd":3,"last update":"Sat May 20 01:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.32900000000000001,"15min":0.378},"min":{"1min":0.246,"5min":0.218,"15min":0.216},"max":{"1min":0.57899999999999996,"5min":0.60599999999999998,"15min":1.2090000000000001},"last":0.379},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.36899999999999999,"15min":0.42699999999999999},"min":{"1min":0.32400000000000001,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.59499999999999997,"5min":0.68100000000000005,"15min":1.5620000000000001},"last":0.35299999999999998}]},{"osd":4,"last update":"Sat May 20 01:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.56799999999999995,"15min":0.58099999999999996},"min":{"1min":0.46500000000000002,"5min":0.35899999999999999,"15min":0.34899999999999998},"max":{"1min":1.107,"5min":1.107,"15min":1.319},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.59399999999999997,"15min":0.61299999999999999},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":1.4319999999999999},"last":0.70999999999999996}]},{"osd":5,"last update":"Sat May 20 01:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.56799999999999995,"15min":0.61699999999999999},"min":{"1min":0.443,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.82199999999999995,"5min":0.95599999999999996,"15min":1.758},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.58899999999999997,"15min":0.63500000000000001},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.004,"5min":1.004,"15min":1.7110000000000001},"last":0.65400000000000003}]},{"osd":6,"last update":"Sat May 20 01:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.59099999999999997,"15min":0.63200000000000001},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.069,"5min":1.069,"15min":1.665},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.58499999999999996,"15min":0.628},"min":{"1min":0.39900000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.88,"5min":0.93500000000000005,"15min":1.819},"last":0.75}]},{"osd":7,"last update":"Sat May 20 01:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.59999999999999998,"15min":0.65200000000000002},"min":{"1min":0.39000000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":1.7310000000000001},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.60999999999999999,"15min":0.65100000000000002},"min":{"1min":0.47799999999999998,"5min":0.42399999999999999,"15min":0.42199999999999999},"max":{"1min":1.038,"5min":1.038,"15min":1.8460000000000001},"last":0.77700000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574974,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":15456,"kb_used_data":1304,"kb_used_omap":0,"kb_used_meta":14144,"kb_avail":94356384,"statfs":{"total":96636764160,"available":96620937216,"internally_reserved":0,"allocated":1335296,"data_stored":682961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":14483456},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:26:34 2023","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.316,"15min":0.31900000000000001},"min":{"1min":0.24099999999999999,"5min":0.20100000000000001,"15min":0.188},"max":{"1min":0.54100000000000004,"5min":0.65000000000000002,"15min":1.528},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.34200000000000003,"15min":0.40799999999999997},"min":{"1min":0.26500000000000001,"5min":0.20799999999999999,"15min":0.19500000000000001},"max":{"1min":0.54200000000000004,"5min":0.72399999999999998,"15min":3.1890000000000001},"last":0.54200000000000004}]},{"osd":1,"last update":"Sat May 20 01:26:34 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.32500000000000001,"15min":0.40000000000000002},"min":{"1min":0.22700000000000001,"5min":0.189,"15min":0.189},"max":{"1min":0.55900000000000005,"5min":0.73199999999999998,"15min":3.4260000000000002},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.437,"5min":0.372,"15min":0.36099999999999999},"min":{"1min":0.27700000000000002,"5min":0.23100000000000001,"15min":0.221},"max":{"1min":0.60999999999999999,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.60999999999999999}]},{"osd":2,"last update":"Sat May 20 01:26:34 2023","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.36399999999999999,"15min":0.38400000000000001},"min":{"1min":0.30399999999999999,"5min":0.24299999999999999,"15min":0.224},"max":{"1min":0.59399999999999997,"5min":0.67400000000000004,"15min":1.1619999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.40000000000000002,"15min":0.41299999999999998},"min":{"1min":0.26900000000000002,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.75,"5min":0.78500000000000003,"15min":1.413},"last":0.64900000000000002}]},{"osd":4,"last update":"Sat May 20 01:26:34 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.56999999999999995,"15min":0.58299999999999996},"min":{"1min":0.45800000000000002,"5min":0.38700000000000001,"15min":0.38200000000000001},"max":{"1min":0.77000000000000002,"5min":0.95399999999999996,"15min":1.0700000000000001},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.59199999999999997,"15min":0.66100000000000003},"min":{"1min":0.42399999999999999,"5min":0.40200000000000002,"15min":0.35899999999999999},"max":{"1min":0.88500000000000001,"5min":0.88600000000000001,"15min":3.4660000000000002},"last":0.879}]},{"osd":5,"last update":"Sat May 20 01:26:34 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.59099999999999997,"15min":0.66600000000000004},"min":{"1min":0.503,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":3.5379999999999998},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.60799999999999998,"15min":0.626},"min":{"1min":0.51400000000000001,"5min":0.41999999999999998,"15min":0.374},"max":{"1min":0.91300000000000003,"5min":1.026,"15min":1.385},"last":0.89700000000000002}]},{"osd":6,"last update":"Sat May 20 01:26:34 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.60699999999999998,"15min":0.629},"min":{"1min":0.44400000000000001,"5min":0.41199999999999998,"15min":0.38100000000000001},"max":{"1min":0.873,"5min":0.99199999999999999,"15min":1.347},"last":0.873},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.60899999999999999,"15min":0.626},"min":{"1min":0.438,"5min":0.374,"15min":0.374},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":1.107},"last":0.94999999999999996}]},{"osd":7,"last update":"Sat May 20 01:26:34 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.60999999999999999,"15min":0.63200000000000001},"min":{"1min":0.46700000000000003,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.92200000000000004,"5min":0.92900000000000005,"15min":1.1359999999999999},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.63700000000000001,"15min":0.70699999999999996},"min":{"1min":0.52300000000000002,"5min":0.436,"15min":0.436},"max":{"1min":0.92300000000000004,"5min":1.0069999999999999,"15min":3.5009999999999999},"last":0.91500000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574974,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":16544,"kb_used_data":1304,"kb_used_omap":0,"kb_used_meta":15232,"kb_avail":94355296,"statfs":{"total":96636764160,"available":96619823104,"internally_reserved":0,"allocated":1335296,"data_stored":682961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":15597568},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:25:33 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.47499999999999998,"15min":0.48199999999999998},"min":{"1min":0.372,"5min":0.247,"15min":0.247},"max":{"1min":0.68799999999999994,"5min":0.73699999999999999,"15min":0.75600000000000001},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.504,"5min":0.503,"15min":0.499},"min":{"1min":0.35299999999999998,"5min":0.30599999999999999,"15min":0.30099999999999999},"max":{"1min":0.71199999999999997,"5min":1.6830000000000001,"15min":1.7989999999999999},"last":0.67300000000000004}]},{"osd":1,"last update":"Sat May 20 01:25:33 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.499,"15min":0.51800000000000002},"min":{"1min":0.35499999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.68400000000000005,"5min":0.746,"15min":0.82799999999999996},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.51700000000000002,"15min":0.52800000000000002},"min":{"1min":0.36799999999999999,"5min":0.316,"15min":0.316},"max":{"1min":0.66500000000000004,"5min":1.7190000000000001,"15min":2.0739999999999998},"last":0.66100000000000003}]},{"osd":2,"last update":"Sat May 20 01:25:33 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.53300000000000003,"15min":0.51200000000000001},"min":{"1min":0.38800000000000001,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53800000000000003,"15min":0.54900000000000004},"min":{"1min":0.42799999999999999,"5min":0.36299999999999999,"15min":0.34000000000000002},"max":{"1min":0.73199999999999998,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.59299999999999997}]},{"osd":3,"last update":"Sat May 20 01:25:33 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.52000000000000002,"15min":0.52200000000000002},"min":{"1min":0.36499999999999999,"5min":0.34899999999999998,"15min":0.34399999999999997},"max":{"1min":0.63600000000000001,"5min":0.81399999999999995,"15min":1.827},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.57599999999999996,"15min":0.64700000000000002},"min":{"1min":0.433,"5min":0.38,"15min":0.38},"max":{"1min":1.093,"5min":1.093,"15min":3.1749999999999998},"last":0.55100000000000005}]},{"osd":5,"last update":"Sat May 20 01:25:33 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.53300000000000003,"15min":0.48999999999999999},"min":{"1min":0.32900000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.76700000000000002,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.505,"5min":0.54700000000000004,"15min":0.51400000000000001},"min":{"1min":0.34699999999999998,"5min":0.33400000000000002,"15min":0.29899999999999999},"max":{"1min":0.70099999999999996,"5min":0.80400000000000005,"15min":0.80500000000000005},"last":0.50800000000000001}]},{"osd":6,"last update":"Sat May 20 01:25:33 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58999999999999997,"15min":0.55100000000000005},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.32400000000000001},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.60699999999999998,"15min":0.59199999999999997},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.80100000000000005,"5min":0.84699999999999998,"15min":0.85899999999999999},"last":0.53100000000000003}]},{"osd":7,"last update":"Sat May 20 01:25:33 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.58099999999999996,"15min":0.55700000000000005},"min":{"1min":0.40899999999999997,"5min":0.36099999999999999,"15min":0.34200000000000003},"max":{"1min":0.67100000000000004,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.626,"5min":0.61799999999999999,"15min":0.59999999999999998},"min":{"1min":0.436,"5min":0.36399999999999999,"15min":0.34000000000000002},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.58299999999999996}]}]},{"osd":5,"up_from":13,"seq":55834574975,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":11036,"kb_used_data":1300,"kb_used_omap":0,"kb_used_meta":9728,"kb_avail":94360804,"statfs":{"total":96636764160,"available":96625463296,"internally_reserved":0,"allocated":1331200,"data_stored":682000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9961472},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:25:38 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.45500000000000002,"15min":0.63300000000000001},"min":{"1min":0.32500000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":6.3929999999999998},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.504,"5min":0.46899999999999997,"15min":0.65600000000000003},"min":{"1min":0.379,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":6.3339999999999996},"last":0.53900000000000003}]},{"osd":1,"last update":"Sat May 20 01:25:38 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.46700000000000003,"15min":0.57499999999999996},"min":{"1min":0.33800000000000002,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":3.2450000000000001},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.51100000000000001,"15min":0.53500000000000003},"min":{"1min":0.371,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":1.1819999999999999},"last":0.51100000000000001}]},{"osd":2,"last update":"Sat May 20 01:25:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.48899999999999999,"15min":0.53700000000000003},"min":{"1min":0.40999999999999998,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.751,"5min":0.751,"15min":1.1639999999999999},"last":0.624},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.49199999999999999,"15min":0.54200000000000004},"min":{"1min":0.39200000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":1.258},"last":0.61399999999999999}]},{"osd":3,"last update":"Sat May 20 01:25:38 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.51700000000000002,"15min":0.70199999999999996},"min":{"1min":0.44500000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":6.351},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.54300000000000004,"15min":0.73399999999999999},"min":{"1min":0.39100000000000001,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":6.3730000000000002},"last":0.58199999999999996}]},{"osd":4,"last update":"Sat May 20 01:25:38 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.51900000000000002,"15min":0.496},"min":{"1min":0.433,"5min":0.309,"15min":0.309},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.70199999999999996},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.41299999999999998,"5min":0.35399999999999998,"15min":0.34200000000000003},"max":{"1min":0.73999999999999999,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.59899999999999998}]},{"osd":6,"last update":"Sat May 20 01:25:38 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.29999999999999999},"max":{"1min":0.77600000000000002,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.55100000000000005,"15min":0.55800000000000005},"min":{"1min":0.432,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.76600000000000001,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.65500000000000003}]},{"osd":7,"last update":"Sat May 20 01:25:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.57799999999999996,"15min":0.58199999999999996},"min":{"1min":0.44,"5min":0.33400000000000002,"15min":0.33000000000000002},"max":{"1min":0.79200000000000004,"5min":0.82299999999999995,"15min":0.91000000000000003},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.59299999999999997,"15min":0.60799999999999998},"min":{"1min":0.53400000000000003,"5min":0.40999999999999998,"15min":0.39500000000000002},"max":{"1min":0.81100000000000005,"5min":0.81399999999999995,"15min":0.94499999999999995},"last":0.71099999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-05-20T01:26:41.010 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.3f is not active+clean 2023-05-20T01:26:41.010 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:26:35.634998+0000', 'last_change': '2023-05-20T01:26:35.634998+0000', 'last_active': '2023-05-20T01:26:35.634998+0000', 'last_peered': '2023-05-20T01:26:35.634998+0000', 'last_clean': '2023-05-20T01:26:35.634998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:26:35.634998+0000', 'last_undegraded': '2023-05-20T01:26:35.634998+0000', 'last_fullsized': '2023-05-20T01:26:35.634998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_clean_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:26:41.011 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.3e is not active+clean 2023-05-20T01:26:41.011 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:26:35.634998+0000', 'last_change': '2023-05-20T01:26:35.634998+0000', 'last_active': '2023-05-20T01:26:35.634998+0000', 'last_peered': '2023-05-20T01:26:35.634998+0000', 'last_clean': '2023-05-20T01:26:35.634998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:26:35.634998+0000', 'last_undegraded': '2023-05-20T01:26:35.634998+0000', 'last_fullsized': '2023-05-20T01:26:35.634998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_clean_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:26:41.011 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.3d is not active+clean 2023-05-20T01:26:41.012 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:26:35.634998+0000', 'last_change': '2023-05-20T01:26:35.634998+0000', 'last_active': '2023-05-20T01:26:35.634998+0000', 'last_peered': '2023-05-20T01:26:35.634998+0000', 'last_clean': '2023-05-20T01:26:35.634998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:26:35.634998+0000', 'last_undegraded': '2023-05-20T01:26:35.634998+0000', 'last_fullsized': '2023-05-20T01:26:35.634998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_clean_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:26:41.012 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.3c is not active+clean 2023-05-20T01:26:41.012 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:26:35.634998+0000', 'last_change': '2023-05-20T01:26:35.634998+0000', 'last_active': '2023-05-20T01:26:35.634998+0000', 'last_peered': '2023-05-20T01:26:35.634998+0000', 'last_clean': '2023-05-20T01:26:35.634998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:26:35.634998+0000', 'last_undegraded': '2023-05-20T01:26:35.634998+0000', 'last_fullsized': '2023-05-20T01:26:35.634998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_clean_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:26:41.013 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.3b is not active+clean 2023-05-20T01:26:41.014 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:26:35.634998+0000', 'last_change': '2023-05-20T01:26:35.634998+0000', 'last_active': '2023-05-20T01:26:35.634998+0000', 'last_peered': '2023-05-20T01:26:35.634998+0000', 'last_clean': '2023-05-20T01:26:35.634998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:26:35.634998+0000', 'last_undegraded': '2023-05-20T01:26:35.634998+0000', 'last_fullsized': '2023-05-20T01:26:35.634998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_clean_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:26:41.014 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.3a is not active+clean 2023-05-20T01:26:41.015 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:26:35.634998+0000', 'last_change': '2023-05-20T01:26:35.634998+0000', 'last_active': '2023-05-20T01:26:35.634998+0000', 'last_peered': '2023-05-20T01:26:35.634998+0000', 'last_clean': '2023-05-20T01:26:35.634998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:26:35.634998+0000', 'last_undegraded': '2023-05-20T01:26:35.634998+0000', 'last_fullsized': '2023-05-20T01:26:35.634998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_clean_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:26:41.015 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.39 is not active+clean 2023-05-20T01:26:41.015 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:26:35.634998+0000', 'last_change': '2023-05-20T01:26:35.634998+0000', 'last_active': '2023-05-20T01:26:35.634998+0000', 'last_peered': '2023-05-20T01:26:35.634998+0000', 'last_clean': '2023-05-20T01:26:35.634998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:26:35.634998+0000', 'last_undegraded': '2023-05-20T01:26:35.634998+0000', 'last_fullsized': '2023-05-20T01:26:35.634998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_clean_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:26:41.016 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.38 is not active+clean 2023-05-20T01:26:41.016 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:26:35.634998+0000', 'last_change': '2023-05-20T01:26:35.634998+0000', 'last_active': '2023-05-20T01:26:35.634998+0000', 'last_peered': '2023-05-20T01:26:35.634998+0000', 'last_clean': '2023-05-20T01:26:35.634998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:26:35.634998+0000', 'last_undegraded': '2023-05-20T01:26:35.634998+0000', 'last_fullsized': '2023-05-20T01:26:35.634998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_clean_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:26:41.017 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.37 is not active+clean 2023-05-20T01:26:41.017 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:26:35.634998+0000', 'last_change': '2023-05-20T01:26:35.634998+0000', 'last_active': '2023-05-20T01:26:35.634998+0000', 'last_peered': '2023-05-20T01:26:35.634998+0000', 'last_clean': '2023-05-20T01:26:35.634998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:26:35.634998+0000', 'last_undegraded': '2023-05-20T01:26:35.634998+0000', 'last_fullsized': '2023-05-20T01:26:35.634998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_clean_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:26:41.017 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.36 is not active+clean 2023-05-20T01:26:41.017 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:26:35.634998+0000', 'last_change': '2023-05-20T01:26:35.634998+0000', 'last_active': '2023-05-20T01:26:35.634998+0000', 'last_peered': '2023-05-20T01:26:35.634998+0000', 'last_clean': '2023-05-20T01:26:35.634998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:26:35.634998+0000', 'last_undegraded': '2023-05-20T01:26:35.634998+0000', 'last_fullsized': '2023-05-20T01:26:35.634998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_clean_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:26:41.018 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.35 is not active+clean 2023-05-20T01:26:41.018 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:26:35.634998+0000', 'last_change': '2023-05-20T01:26:35.634998+0000', 'last_active': '2023-05-20T01:26:35.634998+0000', 'last_peered': '2023-05-20T01:26:35.634998+0000', 'last_clean': '2023-05-20T01:26:35.634998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:26:35.634998+0000', 'last_undegraded': '2023-05-20T01:26:35.634998+0000', 'last_fullsized': '2023-05-20T01:26:35.634998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_clean_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:26:41.018 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.34 is not active+clean 2023-05-20T01:26:41.019 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:26:35.634998+0000', 'last_change': '2023-05-20T01:26:35.634998+0000', 'last_active': '2023-05-20T01:26:35.634998+0000', 'last_peered': '2023-05-20T01:26:35.634998+0000', 'last_clean': '2023-05-20T01:26:35.634998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:26:35.634998+0000', 'last_undegraded': '2023-05-20T01:26:35.634998+0000', 'last_fullsized': '2023-05-20T01:26:35.634998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_clean_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:26:41.019 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.33 is not active+clean 2023-05-20T01:26:41.019 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:26:35.634998+0000', 'last_change': '2023-05-20T01:26:35.634998+0000', 'last_active': '2023-05-20T01:26:35.634998+0000', 'last_peered': '2023-05-20T01:26:35.634998+0000', 'last_clean': '2023-05-20T01:26:35.634998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:26:35.634998+0000', 'last_undegraded': '2023-05-20T01:26:35.634998+0000', 'last_fullsized': '2023-05-20T01:26:35.634998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_clean_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:26:41.020 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.32 is not active+clean 2023-05-20T01:26:41.020 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:26:35.634998+0000', 'last_change': '2023-05-20T01:26:35.634998+0000', 'last_active': '2023-05-20T01:26:35.634998+0000', 'last_peered': '2023-05-20T01:26:35.634998+0000', 'last_clean': '2023-05-20T01:26:35.634998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:26:35.634998+0000', 'last_undegraded': '2023-05-20T01:26:35.634998+0000', 'last_fullsized': '2023-05-20T01:26:35.634998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_clean_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:26:41.020 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.31 is not active+clean 2023-05-20T01:26:41.021 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:26:35.634998+0000', 'last_change': '2023-05-20T01:26:35.634998+0000', 'last_active': '2023-05-20T01:26:35.634998+0000', 'last_peered': '2023-05-20T01:26:35.634998+0000', 'last_clean': '2023-05-20T01:26:35.634998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:26:35.634998+0000', 'last_undegraded': '2023-05-20T01:26:35.634998+0000', 'last_fullsized': '2023-05-20T01:26:35.634998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_clean_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:26:41.021 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.30 is not active+clean 2023-05-20T01:26:41.021 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:26:35.634998+0000', 'last_change': '2023-05-20T01:26:35.634998+0000', 'last_active': '2023-05-20T01:26:35.634998+0000', 'last_peered': '2023-05-20T01:26:35.634998+0000', 'last_clean': '2023-05-20T01:26:35.634998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:26:35.634998+0000', 'last_undegraded': '2023-05-20T01:26:35.634998+0000', 'last_fullsized': '2023-05-20T01:26:35.634998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_clean_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:26:41.022 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.2f is not active+clean 2023-05-20T01:26:41.022 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:26:35.634998+0000', 'last_change': '2023-05-20T01:26:35.634998+0000', 'last_active': '2023-05-20T01:26:35.634998+0000', 'last_peered': '2023-05-20T01:26:35.634998+0000', 'last_clean': '2023-05-20T01:26:35.634998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:26:35.634998+0000', 'last_undegraded': '2023-05-20T01:26:35.634998+0000', 'last_fullsized': '2023-05-20T01:26:35.634998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_clean_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:26:41.022 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.2e is not active+clean 2023-05-20T01:26:41.023 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:26:35.634998+0000', 'last_change': '2023-05-20T01:26:35.634998+0000', 'last_active': '2023-05-20T01:26:35.634998+0000', 'last_peered': '2023-05-20T01:26:35.634998+0000', 'last_clean': '2023-05-20T01:26:35.634998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:26:35.634998+0000', 'last_undegraded': '2023-05-20T01:26:35.634998+0000', 'last_fullsized': '2023-05-20T01:26:35.634998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_clean_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:26:41.023 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.2d is not active+clean 2023-05-20T01:26:41.024 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:26:35.634998+0000', 'last_change': '2023-05-20T01:26:35.634998+0000', 'last_active': '2023-05-20T01:26:35.634998+0000', 'last_peered': '2023-05-20T01:26:35.634998+0000', 'last_clean': '2023-05-20T01:26:35.634998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:26:35.634998+0000', 'last_undegraded': '2023-05-20T01:26:35.634998+0000', 'last_fullsized': '2023-05-20T01:26:35.634998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_clean_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:26:41.024 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.2c is not active+clean 2023-05-20T01:26:41.024 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:26:35.634998+0000', 'last_change': '2023-05-20T01:26:35.634998+0000', 'last_active': '2023-05-20T01:26:35.634998+0000', 'last_peered': '2023-05-20T01:26:35.634998+0000', 'last_clean': '2023-05-20T01:26:35.634998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:26:35.634998+0000', 'last_undegraded': '2023-05-20T01:26:35.634998+0000', 'last_fullsized': '2023-05-20T01:26:35.634998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_clean_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:26:41.025 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.2b is not active+clean 2023-05-20T01:26:41.025 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:26:35.634998+0000', 'last_change': '2023-05-20T01:26:35.634998+0000', 'last_active': '2023-05-20T01:26:35.634998+0000', 'last_peered': '2023-05-20T01:26:35.634998+0000', 'last_clean': '2023-05-20T01:26:35.634998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:26:35.634998+0000', 'last_undegraded': '2023-05-20T01:26:35.634998+0000', 'last_fullsized': '2023-05-20T01:26:35.634998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_clean_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:26:41.026 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.2a is not active+clean 2023-05-20T01:26:41.026 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:26:35.634998+0000', 'last_change': '2023-05-20T01:26:35.634998+0000', 'last_active': '2023-05-20T01:26:35.634998+0000', 'last_peered': '2023-05-20T01:26:35.634998+0000', 'last_clean': '2023-05-20T01:26:35.634998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:26:35.634998+0000', 'last_undegraded': '2023-05-20T01:26:35.634998+0000', 'last_fullsized': '2023-05-20T01:26:35.634998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_clean_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:26:41.026 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.29 is not active+clean 2023-05-20T01:26:41.027 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:26:35.634998+0000', 'last_change': '2023-05-20T01:26:35.634998+0000', 'last_active': '2023-05-20T01:26:35.634998+0000', 'last_peered': '2023-05-20T01:26:35.634998+0000', 'last_clean': '2023-05-20T01:26:35.634998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:26:35.634998+0000', 'last_undegraded': '2023-05-20T01:26:35.634998+0000', 'last_fullsized': '2023-05-20T01:26:35.634998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_clean_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:26:41.027 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.28 is not active+clean 2023-05-20T01:26:41.027 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:26:35.634998+0000', 'last_change': '2023-05-20T01:26:35.634998+0000', 'last_active': '2023-05-20T01:26:35.634998+0000', 'last_peered': '2023-05-20T01:26:35.634998+0000', 'last_clean': '2023-05-20T01:26:35.634998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:26:35.634998+0000', 'last_undegraded': '2023-05-20T01:26:35.634998+0000', 'last_fullsized': '2023-05-20T01:26:35.634998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_clean_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:26:41.028 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.27 is not active+clean 2023-05-20T01:26:41.028 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:26:35.634998+0000', 'last_change': '2023-05-20T01:26:35.634998+0000', 'last_active': '2023-05-20T01:26:35.634998+0000', 'last_peered': '2023-05-20T01:26:35.634998+0000', 'last_clean': '2023-05-20T01:26:35.634998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:26:35.634998+0000', 'last_undegraded': '2023-05-20T01:26:35.634998+0000', 'last_fullsized': '2023-05-20T01:26:35.634998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_clean_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:26:41.028 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.26 is not active+clean 2023-05-20T01:26:41.029 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:26:35.634998+0000', 'last_change': '2023-05-20T01:26:35.634998+0000', 'last_active': '2023-05-20T01:26:35.634998+0000', 'last_peered': '2023-05-20T01:26:35.634998+0000', 'last_clean': '2023-05-20T01:26:35.634998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:26:35.634998+0000', 'last_undegraded': '2023-05-20T01:26:35.634998+0000', 'last_fullsized': '2023-05-20T01:26:35.634998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_clean_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:26:41.029 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.25 is not active+clean 2023-05-20T01:26:41.029 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:26:35.634998+0000', 'last_change': '2023-05-20T01:26:35.634998+0000', 'last_active': '2023-05-20T01:26:35.634998+0000', 'last_peered': '2023-05-20T01:26:35.634998+0000', 'last_clean': '2023-05-20T01:26:35.634998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:26:35.634998+0000', 'last_undegraded': '2023-05-20T01:26:35.634998+0000', 'last_fullsized': '2023-05-20T01:26:35.634998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_clean_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:26:41.030 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.24 is not active+clean 2023-05-20T01:26:41.030 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:26:35.634998+0000', 'last_change': '2023-05-20T01:26:35.634998+0000', 'last_active': '2023-05-20T01:26:35.634998+0000', 'last_peered': '2023-05-20T01:26:35.634998+0000', 'last_clean': '2023-05-20T01:26:35.634998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:26:35.634998+0000', 'last_undegraded': '2023-05-20T01:26:35.634998+0000', 'last_fullsized': '2023-05-20T01:26:35.634998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_clean_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:26:41.031 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.23 is not active+clean 2023-05-20T01:26:41.031 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:26:35.634998+0000', 'last_change': '2023-05-20T01:26:35.634998+0000', 'last_active': '2023-05-20T01:26:35.634998+0000', 'last_peered': '2023-05-20T01:26:35.634998+0000', 'last_clean': '2023-05-20T01:26:35.634998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:26:35.634998+0000', 'last_undegraded': '2023-05-20T01:26:35.634998+0000', 'last_fullsized': '2023-05-20T01:26:35.634998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_clean_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:26:41.032 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.22 is not active+clean 2023-05-20T01:26:41.032 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:26:35.634998+0000', 'last_change': '2023-05-20T01:26:35.634998+0000', 'last_active': '2023-05-20T01:26:35.634998+0000', 'last_peered': '2023-05-20T01:26:35.634998+0000', 'last_clean': '2023-05-20T01:26:35.634998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:26:35.634998+0000', 'last_undegraded': '2023-05-20T01:26:35.634998+0000', 'last_fullsized': '2023-05-20T01:26:35.634998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_clean_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:26:41.033 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.21 is not active+clean 2023-05-20T01:26:41.033 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:26:35.634998+0000', 'last_change': '2023-05-20T01:26:35.634998+0000', 'last_active': '2023-05-20T01:26:35.634998+0000', 'last_peered': '2023-05-20T01:26:35.634998+0000', 'last_clean': '2023-05-20T01:26:35.634998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:26:35.634998+0000', 'last_undegraded': '2023-05-20T01:26:35.634998+0000', 'last_fullsized': '2023-05-20T01:26:35.634998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_clean_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:26:41.033 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.20 is not active+clean 2023-05-20T01:26:41.034 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:26:35.634998+0000', 'last_change': '2023-05-20T01:26:35.634998+0000', 'last_active': '2023-05-20T01:26:35.634998+0000', 'last_peered': '2023-05-20T01:26:35.634998+0000', 'last_clean': '2023-05-20T01:26:35.634998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:26:35.634998+0000', 'last_undegraded': '2023-05-20T01:26:35.634998+0000', 'last_fullsized': '2023-05-20T01:26:35.634998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_clean_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:26:41.034 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.0 is not active+clean 2023-05-20T01:26:41.035 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:26:35.634998+0000', 'last_change': '2023-05-20T01:26:35.634998+0000', 'last_active': '2023-05-20T01:26:35.634998+0000', 'last_peered': '2023-05-20T01:26:35.634998+0000', 'last_clean': '2023-05-20T01:26:35.634998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:26:35.634998+0000', 'last_undegraded': '2023-05-20T01:26:35.634998+0000', 'last_fullsized': '2023-05-20T01:26:35.634998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_clean_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:26:41.035 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1f is not active+clean 2023-05-20T01:26:41.035 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2023-05-20T01:26:34.638110+0000', 'last_change': '2023-05-20T01:26:34.638110+0000', 'last_active': '2023-05-20T01:26:34.630120+0000', 'last_peered': '2023-05-20T01:26:34.630120+0000', 'last_clean': '2023-05-20T01:26:34.630120+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:26:34.638110+0000', 'last_undegraded': '2023-05-20T01:26:34.638110+0000', 'last_fullsized': '2023-05-20T01:26:34.638110+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:26:34.630120+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:26:34.630120+0000', 'last_clean_scrub_stamp': '2023-05-20T01:26:34.630120+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-05-20T01:26:41.036 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.1 is not active+clean 2023-05-20T01:26:41.036 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:26:35.634998+0000', 'last_change': '2023-05-20T01:26:35.634998+0000', 'last_active': '2023-05-20T01:26:35.634998+0000', 'last_peered': '2023-05-20T01:26:35.634998+0000', 'last_clean': '2023-05-20T01:26:35.634998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:26:35.634998+0000', 'last_undegraded': '2023-05-20T01:26:35.634998+0000', 'last_fullsized': '2023-05-20T01:26:35.634998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_clean_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:26:41.036 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1e is not active+clean 2023-05-20T01:26:41.037 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2023-05-20T01:26:34.639497+0000', 'last_change': '2023-05-20T01:26:34.639497+0000', 'last_active': '2023-05-20T01:26:34.630120+0000', 'last_peered': '2023-05-20T01:26:34.630120+0000', 'last_clean': '2023-05-20T01:26:34.630120+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:26:34.639497+0000', 'last_undegraded': '2023-05-20T01:26:34.639497+0000', 'last_fullsized': '2023-05-20T01:26:34.639497+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:26:34.630120+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:26:34.630120+0000', 'last_clean_scrub_stamp': '2023-05-20T01:26:34.630120+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-05-20T01:26:41.037 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.2 is not active+clean 2023-05-20T01:26:41.037 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:26:35.634998+0000', 'last_change': '2023-05-20T01:26:35.634998+0000', 'last_active': '2023-05-20T01:26:35.634998+0000', 'last_peered': '2023-05-20T01:26:35.634998+0000', 'last_clean': '2023-05-20T01:26:35.634998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:26:35.634998+0000', 'last_undegraded': '2023-05-20T01:26:35.634998+0000', 'last_fullsized': '2023-05-20T01:26:35.634998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_clean_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:26:41.038 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1d is not active+clean 2023-05-20T01:26:41.038 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2023-05-20T01:26:34.644602+0000', 'last_change': '2023-05-20T01:26:34.644602+0000', 'last_active': '2023-05-20T01:26:34.630120+0000', 'last_peered': '2023-05-20T01:26:34.630120+0000', 'last_clean': '2023-05-20T01:26:34.630120+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:26:34.644602+0000', 'last_undegraded': '2023-05-20T01:26:34.644602+0000', 'last_fullsized': '2023-05-20T01:26:34.644602+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:26:34.630120+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:26:34.630120+0000', 'last_clean_scrub_stamp': '2023-05-20T01:26:34.630120+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-05-20T01:26:41.038 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.3 is not active+clean 2023-05-20T01:26:41.039 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:26:35.634998+0000', 'last_change': '2023-05-20T01:26:35.634998+0000', 'last_active': '2023-05-20T01:26:35.634998+0000', 'last_peered': '2023-05-20T01:26:35.634998+0000', 'last_clean': '2023-05-20T01:26:35.634998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:26:35.634998+0000', 'last_undegraded': '2023-05-20T01:26:35.634998+0000', 'last_fullsized': '2023-05-20T01:26:35.634998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_clean_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:26:41.039 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1c is not active+clean 2023-05-20T01:26:41.039 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2023-05-20T01:26:34.642067+0000', 'last_change': '2023-05-20T01:26:34.642067+0000', 'last_active': '2023-05-20T01:26:34.630120+0000', 'last_peered': '2023-05-20T01:26:34.630120+0000', 'last_clean': '2023-05-20T01:26:34.630120+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:26:34.642067+0000', 'last_undegraded': '2023-05-20T01:26:34.642067+0000', 'last_fullsized': '2023-05-20T01:26:34.642067+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:26:34.630120+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:26:34.630120+0000', 'last_clean_scrub_stamp': '2023-05-20T01:26:34.630120+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-05-20T01:26:41.040 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.4 is not active+clean 2023-05-20T01:26:41.040 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:26:35.634998+0000', 'last_change': '2023-05-20T01:26:35.634998+0000', 'last_active': '2023-05-20T01:26:35.634998+0000', 'last_peered': '2023-05-20T01:26:35.634998+0000', 'last_clean': '2023-05-20T01:26:35.634998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:26:35.634998+0000', 'last_undegraded': '2023-05-20T01:26:35.634998+0000', 'last_fullsized': '2023-05-20T01:26:35.634998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_clean_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:26:41.040 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1b is not active+clean 2023-05-20T01:26:41.040 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2023-05-20T01:26:34.640655+0000', 'last_change': '2023-05-20T01:26:34.640655+0000', 'last_active': '2023-05-20T01:26:34.630120+0000', 'last_peered': '2023-05-20T01:26:34.630120+0000', 'last_clean': '2023-05-20T01:26:34.630120+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:26:34.640655+0000', 'last_undegraded': '2023-05-20T01:26:34.640655+0000', 'last_fullsized': '2023-05-20T01:26:34.640655+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:26:34.630120+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:26:34.630120+0000', 'last_clean_scrub_stamp': '2023-05-20T01:26:34.630120+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-05-20T01:26:41.041 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.5 is not active+clean 2023-05-20T01:26:41.041 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:26:35.634998+0000', 'last_change': '2023-05-20T01:26:35.634998+0000', 'last_active': '2023-05-20T01:26:35.634998+0000', 'last_peered': '2023-05-20T01:26:35.634998+0000', 'last_clean': '2023-05-20T01:26:35.634998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:26:35.634998+0000', 'last_undegraded': '2023-05-20T01:26:35.634998+0000', 'last_fullsized': '2023-05-20T01:26:35.634998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_clean_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:26:41.041 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1a is not active+clean 2023-05-20T01:26:41.042 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2023-05-20T01:26:34.641569+0000', 'last_change': '2023-05-20T01:26:34.641569+0000', 'last_active': '2023-05-20T01:26:34.630120+0000', 'last_peered': '2023-05-20T01:26:34.630120+0000', 'last_clean': '2023-05-20T01:26:34.630120+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:26:34.641569+0000', 'last_undegraded': '2023-05-20T01:26:34.641569+0000', 'last_fullsized': '2023-05-20T01:26:34.641569+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:26:34.630120+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:26:34.630120+0000', 'last_clean_scrub_stamp': '2023-05-20T01:26:34.630120+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-05-20T01:26:41.042 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.6 is not active+clean 2023-05-20T01:26:41.043 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:26:35.634998+0000', 'last_change': '2023-05-20T01:26:35.634998+0000', 'last_active': '2023-05-20T01:26:35.634998+0000', 'last_peered': '2023-05-20T01:26:35.634998+0000', 'last_clean': '2023-05-20T01:26:35.634998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:26:35.634998+0000', 'last_undegraded': '2023-05-20T01:26:35.634998+0000', 'last_fullsized': '2023-05-20T01:26:35.634998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_clean_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:26:41.043 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.19 is not active+clean 2023-05-20T01:26:41.044 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2023-05-20T01:26:34.644015+0000', 'last_change': '2023-05-20T01:26:34.644015+0000', 'last_active': '2023-05-20T01:26:34.630120+0000', 'last_peered': '2023-05-20T01:26:34.630120+0000', 'last_clean': '2023-05-20T01:26:34.630120+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:26:34.644015+0000', 'last_undegraded': '2023-05-20T01:26:34.644015+0000', 'last_fullsized': '2023-05-20T01:26:34.644015+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:26:34.630120+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:26:34.630120+0000', 'last_clean_scrub_stamp': '2023-05-20T01:26:34.630120+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-05-20T01:26:41.044 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.7 is not active+clean 2023-05-20T01:26:41.045 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:26:35.634998+0000', 'last_change': '2023-05-20T01:26:35.634998+0000', 'last_active': '2023-05-20T01:26:35.634998+0000', 'last_peered': '2023-05-20T01:26:35.634998+0000', 'last_clean': '2023-05-20T01:26:35.634998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:26:35.634998+0000', 'last_undegraded': '2023-05-20T01:26:35.634998+0000', 'last_fullsized': '2023-05-20T01:26:35.634998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_clean_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:26:41.045 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.18 is not active+clean 2023-05-20T01:26:41.045 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2023-05-20T01:26:34.638480+0000', 'last_change': '2023-05-20T01:26:34.638480+0000', 'last_active': '2023-05-20T01:26:34.630120+0000', 'last_peered': '2023-05-20T01:26:34.630120+0000', 'last_clean': '2023-05-20T01:26:34.630120+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:26:34.638480+0000', 'last_undegraded': '2023-05-20T01:26:34.638480+0000', 'last_fullsized': '2023-05-20T01:26:34.638480+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:26:34.630120+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:26:34.630120+0000', 'last_clean_scrub_stamp': '2023-05-20T01:26:34.630120+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-05-20T01:26:41.046 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.8 is not active+clean 2023-05-20T01:26:41.046 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:26:35.634998+0000', 'last_change': '2023-05-20T01:26:35.634998+0000', 'last_active': '2023-05-20T01:26:35.634998+0000', 'last_peered': '2023-05-20T01:26:35.634998+0000', 'last_clean': '2023-05-20T01:26:35.634998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:26:35.634998+0000', 'last_undegraded': '2023-05-20T01:26:35.634998+0000', 'last_fullsized': '2023-05-20T01:26:35.634998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_clean_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:26:41.047 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.17 is not active+clean 2023-05-20T01:26:41.047 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2023-05-20T01:26:34.644724+0000', 'last_change': '2023-05-20T01:26:34.644724+0000', 'last_active': '2023-05-20T01:26:34.630120+0000', 'last_peered': '2023-05-20T01:26:34.630120+0000', 'last_clean': '2023-05-20T01:26:34.630120+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:26:34.644724+0000', 'last_undegraded': '2023-05-20T01:26:34.644724+0000', 'last_fullsized': '2023-05-20T01:26:34.644724+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:26:34.630120+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:26:34.630120+0000', 'last_clean_scrub_stamp': '2023-05-20T01:26:34.630120+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-05-20T01:26:41.048 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.9 is not active+clean 2023-05-20T01:26:41.048 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:26:35.634998+0000', 'last_change': '2023-05-20T01:26:35.634998+0000', 'last_active': '2023-05-20T01:26:35.634998+0000', 'last_peered': '2023-05-20T01:26:35.634998+0000', 'last_clean': '2023-05-20T01:26:35.634998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:26:35.634998+0000', 'last_undegraded': '2023-05-20T01:26:35.634998+0000', 'last_fullsized': '2023-05-20T01:26:35.634998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_clean_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:26:41.049 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.16 is not active+clean 2023-05-20T01:26:41.049 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2023-05-20T01:26:34.639019+0000', 'last_change': '2023-05-20T01:26:34.639019+0000', 'last_active': '2023-05-20T01:26:34.630120+0000', 'last_peered': '2023-05-20T01:26:34.630120+0000', 'last_clean': '2023-05-20T01:26:34.630120+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:26:34.639019+0000', 'last_undegraded': '2023-05-20T01:26:34.639019+0000', 'last_fullsized': '2023-05-20T01:26:34.639019+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:26:34.630120+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:26:34.630120+0000', 'last_clean_scrub_stamp': '2023-05-20T01:26:34.630120+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-05-20T01:26:41.049 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.a is not active+clean 2023-05-20T01:26:41.049 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:26:35.634998+0000', 'last_change': '2023-05-20T01:26:35.634998+0000', 'last_active': '2023-05-20T01:26:35.634998+0000', 'last_peered': '2023-05-20T01:26:35.634998+0000', 'last_clean': '2023-05-20T01:26:35.634998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:26:35.634998+0000', 'last_undegraded': '2023-05-20T01:26:35.634998+0000', 'last_fullsized': '2023-05-20T01:26:35.634998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_clean_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:26:41.050 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.15 is not active+clean 2023-05-20T01:26:41.050 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2023-05-20T01:26:34.641727+0000', 'last_change': '2023-05-20T01:26:34.641727+0000', 'last_active': '2023-05-20T01:26:34.630120+0000', 'last_peered': '2023-05-20T01:26:34.630120+0000', 'last_clean': '2023-05-20T01:26:34.630120+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:26:34.641727+0000', 'last_undegraded': '2023-05-20T01:26:34.641727+0000', 'last_fullsized': '2023-05-20T01:26:34.641727+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:26:34.630120+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:26:34.630120+0000', 'last_clean_scrub_stamp': '2023-05-20T01:26:34.630120+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-05-20T01:26:41.051 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.b is not active+clean 2023-05-20T01:26:41.051 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:26:35.634998+0000', 'last_change': '2023-05-20T01:26:35.634998+0000', 'last_active': '2023-05-20T01:26:35.634998+0000', 'last_peered': '2023-05-20T01:26:35.634998+0000', 'last_clean': '2023-05-20T01:26:35.634998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:26:35.634998+0000', 'last_undegraded': '2023-05-20T01:26:35.634998+0000', 'last_fullsized': '2023-05-20T01:26:35.634998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_clean_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:26:41.051 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.14 is not active+clean 2023-05-20T01:26:41.051 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2023-05-20T01:26:34.640629+0000', 'last_change': '2023-05-20T01:26:34.640629+0000', 'last_active': '2023-05-20T01:26:34.630120+0000', 'last_peered': '2023-05-20T01:26:34.630120+0000', 'last_clean': '2023-05-20T01:26:34.630120+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:26:34.640629+0000', 'last_undegraded': '2023-05-20T01:26:34.640629+0000', 'last_fullsized': '2023-05-20T01:26:34.640629+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:26:34.630120+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:26:34.630120+0000', 'last_clean_scrub_stamp': '2023-05-20T01:26:34.630120+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-05-20T01:26:41.052 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.c is not active+clean 2023-05-20T01:26:41.052 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:26:35.634998+0000', 'last_change': '2023-05-20T01:26:35.634998+0000', 'last_active': '2023-05-20T01:26:35.634998+0000', 'last_peered': '2023-05-20T01:26:35.634998+0000', 'last_clean': '2023-05-20T01:26:35.634998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:26:35.634998+0000', 'last_undegraded': '2023-05-20T01:26:35.634998+0000', 'last_fullsized': '2023-05-20T01:26:35.634998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_clean_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:26:41.053 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.13 is not active+clean 2023-05-20T01:26:41.053 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2023-05-20T01:26:34.641472+0000', 'last_change': '2023-05-20T01:26:34.641472+0000', 'last_active': '2023-05-20T01:26:34.630120+0000', 'last_peered': '2023-05-20T01:26:34.630120+0000', 'last_clean': '2023-05-20T01:26:34.630120+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:26:34.641472+0000', 'last_undegraded': '2023-05-20T01:26:34.641472+0000', 'last_fullsized': '2023-05-20T01:26:34.641472+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:26:34.630120+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:26:34.630120+0000', 'last_clean_scrub_stamp': '2023-05-20T01:26:34.630120+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-05-20T01:26:41.054 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.d is not active+clean 2023-05-20T01:26:41.054 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:26:35.634998+0000', 'last_change': '2023-05-20T01:26:35.634998+0000', 'last_active': '2023-05-20T01:26:35.634998+0000', 'last_peered': '2023-05-20T01:26:35.634998+0000', 'last_clean': '2023-05-20T01:26:35.634998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:26:35.634998+0000', 'last_undegraded': '2023-05-20T01:26:35.634998+0000', 'last_fullsized': '2023-05-20T01:26:35.634998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_clean_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:26:41.055 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.12 is not active+clean 2023-05-20T01:26:41.055 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2023-05-20T01:26:34.638486+0000', 'last_change': '2023-05-20T01:26:34.638486+0000', 'last_active': '2023-05-20T01:26:34.630120+0000', 'last_peered': '2023-05-20T01:26:34.630120+0000', 'last_clean': '2023-05-20T01:26:34.630120+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:26:34.638486+0000', 'last_undegraded': '2023-05-20T01:26:34.638486+0000', 'last_fullsized': '2023-05-20T01:26:34.638486+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:26:34.630120+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:26:34.630120+0000', 'last_clean_scrub_stamp': '2023-05-20T01:26:34.630120+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-05-20T01:26:41.056 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.e is not active+clean 2023-05-20T01:26:41.057 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:26:35.634998+0000', 'last_change': '2023-05-20T01:26:35.634998+0000', 'last_active': '2023-05-20T01:26:35.634998+0000', 'last_peered': '2023-05-20T01:26:35.634998+0000', 'last_clean': '2023-05-20T01:26:35.634998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:26:35.634998+0000', 'last_undegraded': '2023-05-20T01:26:35.634998+0000', 'last_fullsized': '2023-05-20T01:26:35.634998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_clean_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:26:41.058 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.11 is not active+clean 2023-05-20T01:26:41.058 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2023-05-20T01:26:34.638266+0000', 'last_change': '2023-05-20T01:26:34.638266+0000', 'last_active': '2023-05-20T01:26:34.630120+0000', 'last_peered': '2023-05-20T01:26:34.630120+0000', 'last_clean': '2023-05-20T01:26:34.630120+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:26:34.638266+0000', 'last_undegraded': '2023-05-20T01:26:34.638266+0000', 'last_fullsized': '2023-05-20T01:26:34.638266+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:26:34.630120+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:26:34.630120+0000', 'last_clean_scrub_stamp': '2023-05-20T01:26:34.630120+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-05-20T01:26:41.059 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.f is not active+clean 2023-05-20T01:26:41.060 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:26:35.634998+0000', 'last_change': '2023-05-20T01:26:35.634998+0000', 'last_active': '2023-05-20T01:26:35.634998+0000', 'last_peered': '2023-05-20T01:26:35.634998+0000', 'last_clean': '2023-05-20T01:26:35.634998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:26:35.634998+0000', 'last_undegraded': '2023-05-20T01:26:35.634998+0000', 'last_fullsized': '2023-05-20T01:26:35.634998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_clean_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:26:41.060 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.10 is not active+clean 2023-05-20T01:26:41.061 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2023-05-20T01:26:34.643913+0000', 'last_change': '2023-05-20T01:26:34.643913+0000', 'last_active': '2023-05-20T01:26:34.630120+0000', 'last_peered': '2023-05-20T01:26:34.630120+0000', 'last_clean': '2023-05-20T01:26:34.630120+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:26:34.643913+0000', 'last_undegraded': '2023-05-20T01:26:34.643913+0000', 'last_fullsized': '2023-05-20T01:26:34.643913+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:26:34.630120+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:26:34.630120+0000', 'last_clean_scrub_stamp': '2023-05-20T01:26:34.630120+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-05-20T01:26:41.061 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.18 is not active+clean 2023-05-20T01:26:41.061 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:26:35.634998+0000', 'last_change': '2023-05-20T01:26:35.634998+0000', 'last_active': '2023-05-20T01:26:35.634998+0000', 'last_peered': '2023-05-20T01:26:35.634998+0000', 'last_clean': '2023-05-20T01:26:35.634998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:26:35.634998+0000', 'last_undegraded': '2023-05-20T01:26:35.634998+0000', 'last_fullsized': '2023-05-20T01:26:35.634998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_clean_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:26:41.062 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.7 is not active+clean 2023-05-20T01:26:41.062 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2023-05-20T01:26:34.641466+0000', 'last_change': '2023-05-20T01:26:34.641466+0000', 'last_active': '2023-05-20T01:26:34.630120+0000', 'last_peered': '2023-05-20T01:26:34.630120+0000', 'last_clean': '2023-05-20T01:26:34.630120+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:26:34.641466+0000', 'last_undegraded': '2023-05-20T01:26:34.641466+0000', 'last_fullsized': '2023-05-20T01:26:34.641466+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:26:34.630120+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:26:34.630120+0000', 'last_clean_scrub_stamp': '2023-05-20T01:26:34.630120+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-05-20T01:26:41.063 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.19 is not active+clean 2023-05-20T01:26:41.063 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:26:35.634998+0000', 'last_change': '2023-05-20T01:26:35.634998+0000', 'last_active': '2023-05-20T01:26:35.634998+0000', 'last_peered': '2023-05-20T01:26:35.634998+0000', 'last_clean': '2023-05-20T01:26:35.634998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:26:35.634998+0000', 'last_undegraded': '2023-05-20T01:26:35.634998+0000', 'last_fullsized': '2023-05-20T01:26:35.634998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_clean_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:26:41.064 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.6 is not active+clean 2023-05-20T01:26:41.065 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2023-05-20T01:26:34.638503+0000', 'last_change': '2023-05-20T01:26:34.638503+0000', 'last_active': '2023-05-20T01:26:34.630120+0000', 'last_peered': '2023-05-20T01:26:34.630120+0000', 'last_clean': '2023-05-20T01:26:34.630120+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:26:34.638503+0000', 'last_undegraded': '2023-05-20T01:26:34.638503+0000', 'last_fullsized': '2023-05-20T01:26:34.638503+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:26:34.630120+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:26:34.630120+0000', 'last_clean_scrub_stamp': '2023-05-20T01:26:34.630120+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-05-20T01:26:41.065 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.1a is not active+clean 2023-05-20T01:26:41.065 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:26:35.634998+0000', 'last_change': '2023-05-20T01:26:35.634998+0000', 'last_active': '2023-05-20T01:26:35.634998+0000', 'last_peered': '2023-05-20T01:26:35.634998+0000', 'last_clean': '2023-05-20T01:26:35.634998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:26:35.634998+0000', 'last_undegraded': '2023-05-20T01:26:35.634998+0000', 'last_fullsized': '2023-05-20T01:26:35.634998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_clean_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:26:41.066 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.5 is not active+clean 2023-05-20T01:26:41.066 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2023-05-20T01:26:34.640567+0000', 'last_change': '2023-05-20T01:26:34.640567+0000', 'last_active': '2023-05-20T01:26:34.630120+0000', 'last_peered': '2023-05-20T01:26:34.630120+0000', 'last_clean': '2023-05-20T01:26:34.630120+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:26:34.640567+0000', 'last_undegraded': '2023-05-20T01:26:34.640567+0000', 'last_fullsized': '2023-05-20T01:26:34.640567+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:26:34.630120+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:26:34.630120+0000', 'last_clean_scrub_stamp': '2023-05-20T01:26:34.630120+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-05-20T01:26:41.067 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.1b is not active+clean 2023-05-20T01:26:41.067 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:26:35.634998+0000', 'last_change': '2023-05-20T01:26:35.634998+0000', 'last_active': '2023-05-20T01:26:35.634998+0000', 'last_peered': '2023-05-20T01:26:35.634998+0000', 'last_clean': '2023-05-20T01:26:35.634998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:26:35.634998+0000', 'last_undegraded': '2023-05-20T01:26:35.634998+0000', 'last_fullsized': '2023-05-20T01:26:35.634998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_clean_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:26:41.067 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.4 is not active+clean 2023-05-20T01:26:41.067 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2023-05-20T01:26:34.641446+0000', 'last_change': '2023-05-20T01:26:34.641446+0000', 'last_active': '2023-05-20T01:26:34.630120+0000', 'last_peered': '2023-05-20T01:26:34.630120+0000', 'last_clean': '2023-05-20T01:26:34.630120+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:26:34.641446+0000', 'last_undegraded': '2023-05-20T01:26:34.641446+0000', 'last_fullsized': '2023-05-20T01:26:34.641446+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:26:34.630120+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:26:34.630120+0000', 'last_clean_scrub_stamp': '2023-05-20T01:26:34.630120+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-05-20T01:26:41.068 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.1c is not active+clean 2023-05-20T01:26:41.068 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:26:35.634998+0000', 'last_change': '2023-05-20T01:26:35.634998+0000', 'last_active': '2023-05-20T01:26:35.634998+0000', 'last_peered': '2023-05-20T01:26:35.634998+0000', 'last_clean': '2023-05-20T01:26:35.634998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:26:35.634998+0000', 'last_undegraded': '2023-05-20T01:26:35.634998+0000', 'last_fullsized': '2023-05-20T01:26:35.634998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_clean_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:26:41.068 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3 is not active+clean 2023-05-20T01:26:41.069 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2023-05-20T01:26:34.644021+0000', 'last_change': '2023-05-20T01:26:34.644021+0000', 'last_active': '2023-05-20T01:26:34.630120+0000', 'last_peered': '2023-05-20T01:26:34.630120+0000', 'last_clean': '2023-05-20T01:26:34.630120+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:26:34.644021+0000', 'last_undegraded': '2023-05-20T01:26:34.644021+0000', 'last_fullsized': '2023-05-20T01:26:34.644021+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:26:34.630120+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:26:34.630120+0000', 'last_clean_scrub_stamp': '2023-05-20T01:26:34.630120+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-05-20T01:26:41.069 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.1d is not active+clean 2023-05-20T01:26:41.069 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:26:35.634998+0000', 'last_change': '2023-05-20T01:26:35.634998+0000', 'last_active': '2023-05-20T01:26:35.634998+0000', 'last_peered': '2023-05-20T01:26:35.634998+0000', 'last_clean': '2023-05-20T01:26:35.634998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:26:35.634998+0000', 'last_undegraded': '2023-05-20T01:26:35.634998+0000', 'last_fullsized': '2023-05-20T01:26:35.634998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_clean_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:26:41.070 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2 is not active+clean 2023-05-20T01:26:41.070 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2023-05-20T01:26:34.638401+0000', 'last_change': '2023-05-20T01:26:34.638401+0000', 'last_active': '2023-05-20T01:26:34.630120+0000', 'last_peered': '2023-05-20T01:26:34.630120+0000', 'last_clean': '2023-05-20T01:26:34.630120+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:26:34.638401+0000', 'last_undegraded': '2023-05-20T01:26:34.638401+0000', 'last_fullsized': '2023-05-20T01:26:34.638401+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:26:34.630120+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:26:34.630120+0000', 'last_clean_scrub_stamp': '2023-05-20T01:26:34.630120+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-05-20T01:26:41.070 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.1e is not active+clean 2023-05-20T01:26:41.071 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:26:35.634998+0000', 'last_change': '2023-05-20T01:26:35.634998+0000', 'last_active': '2023-05-20T01:26:35.634998+0000', 'last_peered': '2023-05-20T01:26:35.634998+0000', 'last_clean': '2023-05-20T01:26:35.634998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:26:35.634998+0000', 'last_undegraded': '2023-05-20T01:26:35.634998+0000', 'last_fullsized': '2023-05-20T01:26:35.634998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_clean_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:26:41.071 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1 is not active+clean 2023-05-20T01:26:41.072 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2023-05-20T01:26:34.640200+0000', 'last_change': '2023-05-20T01:26:34.640200+0000', 'last_active': '2023-05-20T01:26:34.630120+0000', 'last_peered': '2023-05-20T01:26:34.630120+0000', 'last_clean': '2023-05-20T01:26:34.630120+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:26:34.640200+0000', 'last_undegraded': '2023-05-20T01:26:34.640200+0000', 'last_fullsized': '2023-05-20T01:26:34.640200+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:26:34.630120+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:26:34.630120+0000', 'last_clean_scrub_stamp': '2023-05-20T01:26:34.630120+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-05-20T01:26:41.072 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.1f is not active+clean 2023-05-20T01:26:41.073 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:26:35.634998+0000', 'last_change': '2023-05-20T01:26:35.634998+0000', 'last_active': '2023-05-20T01:26:35.634998+0000', 'last_peered': '2023-05-20T01:26:35.634998+0000', 'last_clean': '2023-05-20T01:26:35.634998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:26:35.634998+0000', 'last_undegraded': '2023-05-20T01:26:35.634998+0000', 'last_fullsized': '2023-05-20T01:26:35.634998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_clean_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:26:41.074 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.0 is not active+clean 2023-05-20T01:26:41.074 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2023-05-20T01:26:34.638339+0000', 'last_change': '2023-05-20T01:26:34.638339+0000', 'last_active': '2023-05-20T01:26:34.630120+0000', 'last_peered': '2023-05-20T01:26:34.630120+0000', 'last_clean': '2023-05-20T01:26:34.630120+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:26:34.638339+0000', 'last_undegraded': '2023-05-20T01:26:34.638339+0000', 'last_fullsized': '2023-05-20T01:26:34.638339+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:26:34.630120+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:26:34.630120+0000', 'last_clean_scrub_stamp': '2023-05-20T01:26:34.630120+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-05-20T01:26:41.075 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.14 is not active+clean 2023-05-20T01:26:41.076 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:26:35.634998+0000', 'last_change': '2023-05-20T01:26:35.634998+0000', 'last_active': '2023-05-20T01:26:35.634998+0000', 'last_peered': '2023-05-20T01:26:35.634998+0000', 'last_clean': '2023-05-20T01:26:35.634998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:26:35.634998+0000', 'last_undegraded': '2023-05-20T01:26:35.634998+0000', 'last_fullsized': '2023-05-20T01:26:35.634998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_clean_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:26:41.076 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.b is not active+clean 2023-05-20T01:26:41.077 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2023-05-20T01:26:34.642905+0000', 'last_change': '2023-05-20T01:26:34.642905+0000', 'last_active': '2023-05-20T01:26:34.630120+0000', 'last_peered': '2023-05-20T01:26:34.630120+0000', 'last_clean': '2023-05-20T01:26:34.630120+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:26:34.642905+0000', 'last_undegraded': '2023-05-20T01:26:34.642905+0000', 'last_fullsized': '2023-05-20T01:26:34.642905+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:26:34.630120+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:26:34.630120+0000', 'last_clean_scrub_stamp': '2023-05-20T01:26:34.630120+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-05-20T01:26:41.078 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.15 is not active+clean 2023-05-20T01:26:41.078 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:26:35.634998+0000', 'last_change': '2023-05-20T01:26:35.634998+0000', 'last_active': '2023-05-20T01:26:35.634998+0000', 'last_peered': '2023-05-20T01:26:35.634998+0000', 'last_clean': '2023-05-20T01:26:35.634998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:26:35.634998+0000', 'last_undegraded': '2023-05-20T01:26:35.634998+0000', 'last_fullsized': '2023-05-20T01:26:35.634998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_clean_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:26:41.079 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.a is not active+clean 2023-05-20T01:26:41.079 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2023-05-20T01:26:34.639380+0000', 'last_change': '2023-05-20T01:26:34.639380+0000', 'last_active': '2023-05-20T01:26:34.630120+0000', 'last_peered': '2023-05-20T01:26:34.630120+0000', 'last_clean': '2023-05-20T01:26:34.630120+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:26:34.639380+0000', 'last_undegraded': '2023-05-20T01:26:34.639380+0000', 'last_fullsized': '2023-05-20T01:26:34.639380+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:26:34.630120+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:26:34.630120+0000', 'last_clean_scrub_stamp': '2023-05-20T01:26:34.630120+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-05-20T01:26:41.080 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.12 is not active+clean 2023-05-20T01:26:41.081 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:26:35.634998+0000', 'last_change': '2023-05-20T01:26:35.634998+0000', 'last_active': '2023-05-20T01:26:35.634998+0000', 'last_peered': '2023-05-20T01:26:35.634998+0000', 'last_clean': '2023-05-20T01:26:35.634998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:26:35.634998+0000', 'last_undegraded': '2023-05-20T01:26:35.634998+0000', 'last_fullsized': '2023-05-20T01:26:35.634998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_clean_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:26:41.081 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.d is not active+clean 2023-05-20T01:26:41.081 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2023-05-20T01:26:34.641191+0000', 'last_change': '2023-05-20T01:26:34.641191+0000', 'last_active': '2023-05-20T01:26:34.630120+0000', 'last_peered': '2023-05-20T01:26:34.630120+0000', 'last_clean': '2023-05-20T01:26:34.630120+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:26:34.641191+0000', 'last_undegraded': '2023-05-20T01:26:34.641191+0000', 'last_fullsized': '2023-05-20T01:26:34.641191+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:26:34.630120+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:26:34.630120+0000', 'last_clean_scrub_stamp': '2023-05-20T01:26:34.630120+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-05-20T01:26:41.082 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.13 is not active+clean 2023-05-20T01:26:41.082 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:26:35.634998+0000', 'last_change': '2023-05-20T01:26:35.634998+0000', 'last_active': '2023-05-20T01:26:35.634998+0000', 'last_peered': '2023-05-20T01:26:35.634998+0000', 'last_clean': '2023-05-20T01:26:35.634998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:26:35.634998+0000', 'last_undegraded': '2023-05-20T01:26:35.634998+0000', 'last_fullsized': '2023-05-20T01:26:35.634998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_clean_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:26:41.082 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.c is not active+clean 2023-05-20T01:26:41.083 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2023-05-20T01:26:34.642061+0000', 'last_change': '2023-05-20T01:26:34.642061+0000', 'last_active': '2023-05-20T01:26:34.630120+0000', 'last_peered': '2023-05-20T01:26:34.630120+0000', 'last_clean': '2023-05-20T01:26:34.630120+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:26:34.642061+0000', 'last_undegraded': '2023-05-20T01:26:34.642061+0000', 'last_fullsized': '2023-05-20T01:26:34.642061+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:26:34.630120+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:26:34.630120+0000', 'last_clean_scrub_stamp': '2023-05-20T01:26:34.630120+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-05-20T01:26:41.083 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.11 is not active+clean 2023-05-20T01:26:41.083 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:26:35.634998+0000', 'last_change': '2023-05-20T01:26:35.634998+0000', 'last_active': '2023-05-20T01:26:35.634998+0000', 'last_peered': '2023-05-20T01:26:35.634998+0000', 'last_clean': '2023-05-20T01:26:35.634998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:26:35.634998+0000', 'last_undegraded': '2023-05-20T01:26:35.634998+0000', 'last_fullsized': '2023-05-20T01:26:35.634998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_clean_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:26:41.083 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.e is not active+clean 2023-05-20T01:26:41.084 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2023-05-20T01:26:34.638510+0000', 'last_change': '2023-05-20T01:26:34.638510+0000', 'last_active': '2023-05-20T01:26:34.630120+0000', 'last_peered': '2023-05-20T01:26:34.630120+0000', 'last_clean': '2023-05-20T01:26:34.630120+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:26:34.638510+0000', 'last_undegraded': '2023-05-20T01:26:34.638510+0000', 'last_fullsized': '2023-05-20T01:26:34.638510+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:26:34.630120+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:26:34.630120+0000', 'last_clean_scrub_stamp': '2023-05-20T01:26:34.630120+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-05-20T01:26:41.084 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.10 is not active+clean 2023-05-20T01:26:41.084 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:26:35.634998+0000', 'last_change': '2023-05-20T01:26:35.634998+0000', 'last_active': '2023-05-20T01:26:35.634998+0000', 'last_peered': '2023-05-20T01:26:35.634998+0000', 'last_clean': '2023-05-20T01:26:35.634998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:26:35.634998+0000', 'last_undegraded': '2023-05-20T01:26:35.634998+0000', 'last_fullsized': '2023-05-20T01:26:35.634998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_clean_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:26:41.085 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.f is not active+clean 2023-05-20T01:26:41.085 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2023-05-20T01:26:34.640347+0000', 'last_change': '2023-05-20T01:26:34.640347+0000', 'last_active': '2023-05-20T01:26:34.630120+0000', 'last_peered': '2023-05-20T01:26:34.630120+0000', 'last_clean': '2023-05-20T01:26:34.630120+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:26:34.640347+0000', 'last_undegraded': '2023-05-20T01:26:34.640347+0000', 'last_fullsized': '2023-05-20T01:26:34.640347+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:26:34.630120+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:26:34.630120+0000', 'last_clean_scrub_stamp': '2023-05-20T01:26:34.630120+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-05-20T01:26:41.085 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.17 is not active+clean 2023-05-20T01:26:41.086 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:26:35.634998+0000', 'last_change': '2023-05-20T01:26:35.634998+0000', 'last_active': '2023-05-20T01:26:35.634998+0000', 'last_peered': '2023-05-20T01:26:35.634998+0000', 'last_clean': '2023-05-20T01:26:35.634998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:26:35.634998+0000', 'last_undegraded': '2023-05-20T01:26:35.634998+0000', 'last_fullsized': '2023-05-20T01:26:35.634998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_clean_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:26:41.086 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.8 is not active+clean 2023-05-20T01:26:41.086 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2023-05-20T01:26:34.638985+0000', 'last_change': '2023-05-20T01:26:34.638985+0000', 'last_active': '2023-05-20T01:26:34.630120+0000', 'last_peered': '2023-05-20T01:26:34.630120+0000', 'last_clean': '2023-05-20T01:26:34.630120+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:26:34.638985+0000', 'last_undegraded': '2023-05-20T01:26:34.638985+0000', 'last_fullsized': '2023-05-20T01:26:34.638985+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:26:34.630120+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:26:34.630120+0000', 'last_clean_scrub_stamp': '2023-05-20T01:26:34.630120+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-05-20T01:26:41.087 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.16 is not active+clean 2023-05-20T01:26:41.087 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-05-20T01:26:35.634998+0000', 'last_change': '2023-05-20T01:26:35.634998+0000', 'last_active': '2023-05-20T01:26:35.634998+0000', 'last_peered': '2023-05-20T01:26:35.634998+0000', 'last_clean': '2023-05-20T01:26:35.634998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:26:35.634998+0000', 'last_undegraded': '2023-05-20T01:26:35.634998+0000', 'last_fullsized': '2023-05-20T01:26:35.634998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'last_clean_scrub_stamp': '2023-05-20T01:26:35.634998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-05-20T01:26:41.087 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.9 is not active+clean 2023-05-20T01:26:41.088 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2023-05-20T01:26:34.640797+0000', 'last_change': '2023-05-20T01:26:34.640797+0000', 'last_active': '2023-05-20T01:26:34.630120+0000', 'last_peered': '2023-05-20T01:26:34.630120+0000', 'last_clean': '2023-05-20T01:26:34.630120+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-05-20T01:26:34.640797+0000', 'last_undegraded': '2023-05-20T01:26:34.640797+0000', 'last_fullsized': '2023-05-20T01:26:34.640797+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-05-20T01:26:34.630120+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-05-20T01:26:34.630120+0000', 'last_clean_scrub_stamp': '2023-05-20T01:26:34.630120+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-05-20T01:26:41.089 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-05-20T01:26:41.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.209+0000 7fe77a502700 1 Processor -- start 2023-05-20T01:26:41.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.209+0000 7fe77a502700 1 -- start start 2023-05-20T01:26:41.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.210+0000 7fe77a502700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe77415e110 0x7fe774071140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:41.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.210+0000 7fe77a502700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe77406e6d0 0x7fe77406a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:41.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.210+0000 7fe77a502700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe77415f650 con 0x7fe7740b2200 2023-05-20T01:26:41.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.210+0000 7fe77a502700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe77415e4e0 con 0x7fe77406e6d0 2023-05-20T01:26:41.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.210+0000 7fe77a502700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe77415ead0 con 0x7fe77415e110 2023-05-20T01:26:41.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.210+0000 7fe778a9f700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe77406e6d0 0x7fe77406a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:41.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.210+0000 7fe778a9f700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe77406e6d0 0x7fe77406a880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38192/0 (socket says 172.21.15.16:38192) 2023-05-20T01:26:41.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.210+0000 7fe778a9f700 1 -- 172.21.15.16:0/1501576485 learned_addr learned my addr 172.21.15.16:0/1501576485 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:26:41.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.210+0000 7fe773fff700 1 --2- 172.21.15.16:0/1501576485 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe77415e110 0x7fe774071140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:41.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.210+0000 7fe778a9f700 1 -- 172.21.15.16:0/1501576485 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe77415e110 msgr2=0x7fe774071140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:41.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.210+0000 7fe778a9f700 1 --2- 172.21.15.16:0/1501576485 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe77415e110 0x7fe774071140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:41.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.210+0000 7fe778a9f700 1 -- 172.21.15.16:0/1501576485 >> v1:172.21.15.132:6789/0 conn(0x7fe7740b2200 legacy=0x7fe77406dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:41.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.210+0000 7fe778a9f700 1 -- 172.21.15.16:0/1501576485 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe77415edb0 con 0x7fe77406e6d0 2023-05-20T01:26:41.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.210+0000 7fe778a9f700 1 --2- 172.21.15.16:0/1501576485 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe77406e6d0 0x7fe77406a880 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7fe76c009e50 tx=0x7fe76c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=281ec42d57486875 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:41.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.210+0000 7fe772ffd700 1 -- 172.21.15.16:0/1501576485 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe76c00e070 con 0x7fe77406e6d0 2023-05-20T01:26:41.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.210+0000 7fe772ffd700 1 -- 172.21.15.16:0/1501576485 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe76c003a20 con 0x7fe77406e6d0 2023-05-20T01:26:41.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.210+0000 7fe772ffd700 1 -- 172.21.15.16:0/1501576485 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe76c003dd0 con 0x7fe77406e6d0 2023-05-20T01:26:41.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.211+0000 7fe77a502700 1 -- 172.21.15.16:0/1501576485 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe77406e6d0 msgr2=0x7fe77406a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:41.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.211+0000 7fe77a502700 1 --2- 172.21.15.16:0/1501576485 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe77406e6d0 0x7fe77406a880 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7fe76c009e50 tx=0x7fe76c00b330 comp rx=0 tx=0).stop 2023-05-20T01:26:41.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.211+0000 7fe77a502700 1 -- 172.21.15.16:0/1501576485 shutdown_connections 2023-05-20T01:26:41.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.211+0000 7fe77a502700 1 --2- 172.21.15.16:0/1501576485 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe77406e6d0 0x7fe77406a880 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:41.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.211+0000 7fe77a502700 1 --2- 172.21.15.16:0/1501576485 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe77415e110 0x7fe774071140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:41.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.211+0000 7fe77a502700 1 -- 172.21.15.16:0/1501576485 >> 172.21.15.16:0/1501576485 conn(0x7fe7740d0c70 msgr2=0x7fe7740b1af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:41.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.211+0000 7fe77a502700 1 -- 172.21.15.16:0/1501576485 shutdown_connections 2023-05-20T01:26:41.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.211+0000 7fe77a502700 1 -- 172.21.15.16:0/1501576485 wait complete. 2023-05-20T01:26:41.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.212+0000 7fe77a502700 1 Processor -- start 2023-05-20T01:26:41.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.212+0000 7fe77a502700 1 -- start start 2023-05-20T01:26:41.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.213+0000 7fe77a502700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe77406e6d0 0x7fe77415b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:41.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.213+0000 7fe77a502700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe7740b2200 0x7fe77406a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:41.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.213+0000 7fe77a502700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe77415e110 0x7fe77406acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:41.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.213+0000 7fe77a502700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7740c47f0 con 0x7fe77406e6d0 2023-05-20T01:26:41.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.213+0000 7fe77a502700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7740c3bb0 con 0x7fe7740b2200 2023-05-20T01:26:41.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.213+0000 7fe77a502700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe77415baa0 con 0x7fe77415e110 2023-05-20T01:26:41.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.213+0000 7fe773fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe77406e6d0 0x7fe77415b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:41.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.213+0000 7fe773fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe77406e6d0 0x7fe77415b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38194/0 (socket says 172.21.15.16:38194) 2023-05-20T01:26:41.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.213+0000 7fe773fff700 1 -- 172.21.15.16:0/3459577014 learned_addr learned my addr 172.21.15.16:0/3459577014 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:26:41.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.213+0000 7fe7737fe700 1 --2- 172.21.15.16:0/3459577014 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe7740b2200 0x7fe77406a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:41.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.213+0000 7fe778a9f700 1 --2- 172.21.15.16:0/3459577014 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe77415e110 0x7fe77406acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:41.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.214+0000 7fe773fff700 1 -- 172.21.15.16:0/3459577014 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe77415e110 msgr2=0x7fe77406acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:41.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.214+0000 7fe773fff700 1 --2- 172.21.15.16:0/3459577014 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe77415e110 0x7fe77406acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:41.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.214+0000 7fe773fff700 1 -- 172.21.15.16:0/3459577014 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe7740b2200 msgr2=0x7fe77406a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:41.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.214+0000 7fe773fff700 1 --2- 172.21.15.16:0/3459577014 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe7740b2200 0x7fe77406a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:41.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.214+0000 7fe773fff700 1 -- 172.21.15.16:0/3459577014 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe76c009b30 con 0x7fe77406e6d0 2023-05-20T01:26:41.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.214+0000 7fe778a9f700 1 --2- 172.21.15.16:0/3459577014 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe77415e110 0x7fe77406acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:26:41.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.214+0000 7fe773fff700 1 --2- 172.21.15.16:0/3459577014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe77406e6d0 0x7fe77415b560 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7fe76400c7f0 tx=0x7fe76400cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:41.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.214+0000 7fe770ff9700 1 -- 172.21.15.16:0/3459577014 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe764011800 con 0x7fe77406e6d0 2023-05-20T01:26:41.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.214+0000 7fe770ff9700 1 -- 172.21.15.16:0/3459577014 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe764011960 con 0x7fe77406e6d0 2023-05-20T01:26:41.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.214+0000 7fe77a502700 1 -- 172.21.15.16:0/3459577014 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe77415f370 con 0x7fe77406e6d0 2023-05-20T01:26:41.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.214+0000 7fe770ff9700 1 -- 172.21.15.16:0/3459577014 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe764011cd0 con 0x7fe77406e6d0 2023-05-20T01:26:41.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.214+0000 7fe77a502700 1 -- 172.21.15.16:0/3459577014 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe77415f090 con 0x7fe77406e6d0 2023-05-20T01:26:41.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.215+0000 7fe77a502700 1 -- 172.21.15.16:0/3459577014 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe774066580 con 0x7fe77406e6d0 2023-05-20T01:26:41.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.215+0000 7fe770ff9700 1 -- 172.21.15.16:0/3459577014 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe764010350 con 0x7fe77406e6d0 2023-05-20T01:26:41.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.216+0000 7fe770ff9700 1 --2- 172.21.15.16:0/3459577014 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe76006ace0 0x7fe76006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:41.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.216+0000 7fe770ff9700 1 -- 172.21.15.16:0/3459577014 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fe7640890a0 con 0x7fe77406e6d0 2023-05-20T01:26:41.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.216+0000 7fe7737fe700 1 --2- 172.21.15.16:0/3459577014 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe76006ace0 0x7fe76006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:41.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.216+0000 7fe7737fe700 1 --2- 172.21.15.16:0/3459577014 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe76006ace0 0x7fe76006d190 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fe77409ddf0 tx=0x7fe768007ec0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:41.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.220+0000 7fe770ff9700 1 -- 172.21.15.16:0/3459577014 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fe764054020 con 0x7fe77406e6d0 2023-05-20T01:26:41.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.502+0000 7fe77a502700 1 -- 172.21.15.16:0/3459577014 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fe7740747b0 con 0x7fe77406e6d0 2023-05-20T01:26:41.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.502+0000 7fe770ff9700 1 -- 172.21.15.16:0/3459577014 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1111 (secure 0 0 0) 0x7fe764057640 con 0x7fe77406e6d0 2023-05-20T01:26:41.507 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:26:41.507 INFO:teuthology.orchestra.run.smithi016.stdout:{"fsid":"c2245cc5-ecab-4087-a59b-bc81db901676","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":631,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":112,"num_osds":8,"num_up_osds":8,"osd_up_since":1684545379,"num_in_osds":8,"osd_in_since":1684545372,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":64},{"state_name":"creating+peering","count":32},{"state_name":"active+clean","count":9}],"num_pgs":105,"num_pools":4,"num_objects":2,"data_bytes":590368,"bytes_used":112848896,"bytes_avail":772981264384,"bytes_total":773094113280,"unknown_pgs_ratio":0.60952383279800415,"inactive_pgs_ratio":0.30476191639900208},"fsmap":{"epoch":78,"id":5,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":5,"rank":0,"name":"a","status":"up:active","gid":7896}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":16,"modified":"2023-05-20T01:23:41.412868+0000","services":{}},"progress_events":{}} 2023-05-20T01:26:41.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.504+0000 7fe77a502700 1 -- 172.21.15.16:0/3459577014 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe76006ace0 msgr2=0x7fe76006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:41.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.504+0000 7fe77a502700 1 --2- 172.21.15.16:0/3459577014 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe76006ace0 0x7fe76006d190 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fe77409ddf0 tx=0x7fe768007ec0 comp rx=0 tx=0).stop 2023-05-20T01:26:41.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.504+0000 7fe77a502700 1 -- 172.21.15.16:0/3459577014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe77406e6d0 msgr2=0x7fe77415b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:41.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.504+0000 7fe77a502700 1 --2- 172.21.15.16:0/3459577014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe77406e6d0 0x7fe77415b560 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7fe76400c7f0 tx=0x7fe76400cb00 comp rx=0 tx=0).stop 2023-05-20T01:26:41.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.504+0000 7fe77a502700 1 -- 172.21.15.16:0/3459577014 shutdown_connections 2023-05-20T01:26:41.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.504+0000 7fe77a502700 1 --2- 172.21.15.16:0/3459577014 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe76006ace0 0x7fe76006d190 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:41.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.504+0000 7fe77a502700 1 --2- 172.21.15.16:0/3459577014 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe77415e110 0x7fe77406acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:41.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.504+0000 7fe77a502700 1 --2- 172.21.15.16:0/3459577014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe77406e6d0 0x7fe77415b560 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:41.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.504+0000 7fe77a502700 1 --2- 172.21.15.16:0/3459577014 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe7740b2200 0x7fe77406a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:41.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.505+0000 7fe77a502700 1 -- 172.21.15.16:0/3459577014 >> 172.21.15.16:0/3459577014 conn(0x7fe7740d0c70 msgr2=0x7fe77406dfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:41.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.505+0000 7fe77a502700 1 -- 172.21.15.16:0/3459577014 shutdown_connections 2023-05-20T01:26:41.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.505+0000 7fe77a502700 1 -- 172.21.15.16:0/3459577014 wait complete. 2023-05-20T01:26:41.521 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-05-20T01:26:41.522 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-05-20T01:26:41.629 INFO:tasks.ceph.mds.a.smithi016.stderr:2023-05-20T01:26:41.625+0000 7fcb0f23e700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-05-20T01:26:41.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.642+0000 7f509cec6700 1 Processor -- start 2023-05-20T01:26:41.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.642+0000 7f509cec6700 1 -- start start 2023-05-20T01:26:41.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.643+0000 7f509cec6700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5098144020 0x7f50980d1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:41.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.643+0000 7f509cec6700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f50980cec60 0x7f50980cf0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:41.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.643+0000 7f509cec6700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50980cc500 con 0x7f50980b2760 2023-05-20T01:26:41.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.643+0000 7f509cec6700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50980cb6e0 con 0x7f5098144020 2023-05-20T01:26:41.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.643+0000 7f509cec6700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50980cf620 con 0x7f50980cec60 2023-05-20T01:26:41.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.643+0000 7f5095d9b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5098144020 0x7f50980d1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:41.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.643+0000 7f509659c700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f50980b2760 0x7f50980b0980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49730/0 (socket says 172.21.15.16:49730) 2023-05-20T01:26:41.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.643+0000 7f509659c700 1 -- 172.21.15.16:0/3693737627 learned_addr learned my addr 172.21.15.16:0/3693737627 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:26:41.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.643+0000 7f5096d9d700 1 --2- 172.21.15.16:0/3693737627 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f50980cec60 0x7f50980cf0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:41.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.643+0000 7f5095d9b700 1 -- 172.21.15.16:0/3693737627 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f50980cec60 msgr2=0x7f50980cf0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:41.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.643+0000 7f5095d9b700 1 --2- 172.21.15.16:0/3693737627 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f50980cec60 0x7f50980cf0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:41.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.643+0000 7f5095d9b700 1 -- 172.21.15.16:0/3693737627 >> v1:172.21.15.16:6789/0 conn(0x7f50980b2760 legacy=0x7f50980b0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:41.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.643+0000 7f509559a700 1 -- 172.21.15.16:0/3693737627 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1213998666 0 0) 0x7f50980cc500 con 0x7f50980b2760 2023-05-20T01:26:41.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.643+0000 7f5095d9b700 1 -- 172.21.15.16:0/3693737627 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50980cb8f0 con 0x7f5098144020 2023-05-20T01:26:41.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.644+0000 7f5095d9b700 1 --2- 172.21.15.16:0/3693737627 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5098144020 0x7f50980d1680 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f5088009840 tx=0x7f5088009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dcfda107aade7b0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:41.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.644+0000 7f509559a700 1 -- 172.21.15.16:0/3693737627 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5088011830 con 0x7f5098144020 2023-05-20T01:26:41.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.644+0000 7f509559a700 1 -- 172.21.15.16:0/3693737627 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5088011990 con 0x7f5098144020 2023-05-20T01:26:41.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.644+0000 7f509559a700 1 -- 172.21.15.16:0/3693737627 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5088011d20 con 0x7f5098144020 2023-05-20T01:26:41.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.644+0000 7f509cec6700 1 -- 172.21.15.16:0/3693737627 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5098144020 msgr2=0x7f50980d1680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:41.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.644+0000 7f509cec6700 1 --2- 172.21.15.16:0/3693737627 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5098144020 0x7f50980d1680 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f5088009840 tx=0x7f5088009b50 comp rx=0 tx=0).stop 2023-05-20T01:26:41.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.644+0000 7f509cec6700 1 -- 172.21.15.16:0/3693737627 shutdown_connections 2023-05-20T01:26:41.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.644+0000 7f509cec6700 1 --2- 172.21.15.16:0/3693737627 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f50980cec60 0x7f50980cf0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:41.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.644+0000 7f509cec6700 1 --2- 172.21.15.16:0/3693737627 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5098144020 0x7f50980d1680 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:41.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.644+0000 7f509cec6700 1 -- 172.21.15.16:0/3693737627 >> 172.21.15.16:0/3693737627 conn(0x7f509806ae30 msgr2=0x7f509809a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:41.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.644+0000 7f509cec6700 1 -- 172.21.15.16:0/3693737627 shutdown_connections 2023-05-20T01:26:41.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.644+0000 7f509cec6700 1 -- 172.21.15.16:0/3693737627 wait complete. 2023-05-20T01:26:41.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.646+0000 7f509cec6700 1 Processor -- start 2023-05-20T01:26:41.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.646+0000 7f509cec6700 1 -- start start 2023-05-20T01:26:41.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.647+0000 7f509cec6700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f50980b2760 0x7f5098158e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:41.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.647+0000 7f509cec6700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f50980cec60 0x7f50980caa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:41.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.647+0000 7f509cec6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5098144020 0x7f509815b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:41.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.647+0000 7f509cec6700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50981599a0 con 0x7f5098144020 2023-05-20T01:26:41.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.647+0000 7f509cec6700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f509806c8c0 con 0x7f50980b2760 2023-05-20T01:26:41.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.647+0000 7f509cec6700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f509806bc40 con 0x7f50980cec60 2023-05-20T01:26:41.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.647+0000 7f5096d9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5098144020 0x7f509815b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:41.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.647+0000 7f5096d9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5098144020 0x7f509815b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38212/0 (socket says 172.21.15.16:38212) 2023-05-20T01:26:41.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.647+0000 7f5096d9d700 1 -- 172.21.15.16:0/3572483599 learned_addr learned my addr 172.21.15.16:0/3572483599 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:26:41.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.648+0000 7f5095d9b700 1 --2- 172.21.15.16:0/3572483599 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f50980cec60 0x7f50980caa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:41.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.648+0000 7f509659c700 1 --2- 172.21.15.16:0/3572483599 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f50980b2760 0x7f5098158e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:41.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.648+0000 7f5096d9d700 1 -- 172.21.15.16:0/3572483599 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f50980cec60 msgr2=0x7f50980caa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:41.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.648+0000 7f5096d9d700 1 --2- 172.21.15.16:0/3572483599 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f50980cec60 0x7f50980caa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:41.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.648+0000 7f5096d9d700 1 -- 172.21.15.16:0/3572483599 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f50980b2760 msgr2=0x7f5098158e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:41.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.648+0000 7f5096d9d700 1 --2- 172.21.15.16:0/3572483599 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f50980b2760 0x7f5098158e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:41.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.648+0000 7f5096d9d700 1 -- 172.21.15.16:0/3572483599 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50880094a0 con 0x7f5098144020 2023-05-20T01:26:41.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.648+0000 7f5095d9b700 1 --2- 172.21.15.16:0/3572483599 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f50980cec60 0x7f50980caa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:26:41.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.648+0000 7f5096d9d700 1 --2- 172.21.15.16:0/3572483599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5098144020 0x7f509815b0d0 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f5090007e60 tx=0x7f509000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:41.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.648+0000 7f5086ffd700 1 -- 172.21.15.16:0/3572483599 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f509000ede0 con 0x7f5098144020 2023-05-20T01:26:41.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.648+0000 7f5086ffd700 1 -- 172.21.15.16:0/3572483599 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5090014420 con 0x7f5098144020 2023-05-20T01:26:41.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.648+0000 7f509cec6700 1 -- 172.21.15.16:0/3572483599 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50980cbeb0 con 0x7f5098144020 2023-05-20T01:26:41.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.648+0000 7f5086ffd700 1 -- 172.21.15.16:0/3572483599 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50900147d0 con 0x7f5098144020 2023-05-20T01:26:41.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.648+0000 7f509cec6700 1 -- 172.21.15.16:0/3572483599 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50980cbbd0 con 0x7f5098144020 2023-05-20T01:26:41.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.649+0000 7f509cec6700 1 -- 172.21.15.16:0/3572483599 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50980cc190 con 0x7f5098144020 2023-05-20T01:26:41.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.650+0000 7f5086ffd700 1 -- 172.21.15.16:0/3572483599 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5090014930 con 0x7f5098144020 2023-05-20T01:26:41.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.650+0000 7f5086ffd700 1 --2- 172.21.15.16:0/3572483599 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f507806ace0 0x7f507806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:41.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.650+0000 7f5086ffd700 1 -- 172.21.15.16:0/3572483599 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f5090089d10 con 0x7f5098144020 2023-05-20T01:26:41.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.650+0000 7f509659c700 1 --2- 172.21.15.16:0/3572483599 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f507806ace0 0x7f507806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:41.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.650+0000 7f509659c700 1 --2- 172.21.15.16:0/3572483599 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f507806ace0 0x7f507806d190 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f508c000f60 tx=0x7f508c007c90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:41.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.654+0000 7f5086ffd700 1 -- 172.21.15.16:0/3572483599 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f5090054dc0 con 0x7f5098144020 2023-05-20T01:26:41.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.848+0000 7f509cec6700 1 -- 172.21.15.16:0/3572483599 --> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f509806ca00 con 0x7f507806ace0 2023-05-20T01:26:41.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.857+0000 7f5086ffd700 1 -- 172.21.15.16:0/3572483599 <== mgr.4104 v2:172.21.15.16:6832/3817237998 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311986 (secure 0 0 0) 0x7f509806ca00 con 0x7f507806ace0 2023-05-20T01:26:41.866 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:26:41.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.861+0000 7f509cec6700 1 -- 172.21.15.16:0/3572483599 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f507806ace0 msgr2=0x7f507806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:41.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.861+0000 7f509cec6700 1 --2- 172.21.15.16:0/3572483599 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f507806ace0 0x7f507806d190 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f508c000f60 tx=0x7f508c007c90 comp rx=0 tx=0).stop 2023-05-20T01:26:41.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.861+0000 7f509cec6700 1 -- 172.21.15.16:0/3572483599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5098144020 msgr2=0x7f509815b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:41.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.861+0000 7f509cec6700 1 --2- 172.21.15.16:0/3572483599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5098144020 0x7f509815b0d0 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f5090007e60 tx=0x7f509000e3c0 comp rx=0 tx=0).stop 2023-05-20T01:26:41.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.861+0000 7f509cec6700 1 -- 172.21.15.16:0/3572483599 shutdown_connections 2023-05-20T01:26:41.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.861+0000 7f509cec6700 1 --2- 172.21.15.16:0/3572483599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5098144020 0x7f509815b0d0 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:41.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.861+0000 7f509cec6700 1 --2- 172.21.15.16:0/3572483599 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f50980b2760 0x7f5098158e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:41.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.861+0000 7f509cec6700 1 --2- 172.21.15.16:0/3572483599 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f507806ace0 0x7f507806d190 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:41.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.861+0000 7f509cec6700 1 --2- 172.21.15.16:0/3572483599 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f50980cec60 0x7f50980caa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:41.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.861+0000 7f509cec6700 1 -- 172.21.15.16:0/3572483599 >> 172.21.15.16:0/3572483599 conn(0x7f509806ae30 msgr2=0x7f50980af5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:41.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.861+0000 7f509cec6700 1 -- 172.21.15.16:0/3572483599 shutdown_connections 2023-05-20T01:26:41.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:41.861+0000 7f509cec6700 1 -- 172.21.15.16:0/3572483599 wait complete. 2023-05-20T01:26:41.869 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2023-05-20T01:26:41.877 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":416,"stamp":"2023-05-20T01:26:41.455164+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":197,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":111872,"kb_used_data":12032,"kb_used_omap":0,"kb_used_meta":99776,"kb_avail":754862848,"statfs":{"total":773094113280,"available":772979556352,"internally_reserved":0,"allocated":12320768,"data_stored":6870098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":102170624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001085"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.388334+0000","last_change":"2023-05-20T01:26:36.087663+0000","last_active":"2023-05-20T01:26:38.388334+0000","last_peered":"2023-05-20T01:26:38.388334+0000","last_clean":"2023-05-20T01:26:38.388334+0000","last_became_active":"2023-05-20T01:26:36.087500+0000","last_became_peered":"2023-05-20T01:26:36.087500+0000","last_unstale":"2023-05-20T01:26:38.388334+0000","last_undegraded":"2023-05-20T01:26:38.388334+0000","last_fullsized":"2023-05-20T01:26:38.388334+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:55:04.702228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:37.086071+0000","last_change":"2023-05-20T01:26:36.093490+0000","last_active":"2023-05-20T01:26:37.086071+0000","last_peered":"2023-05-20T01:26:37.086071+0000","last_clean":"2023-05-20T01:26:37.086071+0000","last_became_active":"2023-05-20T01:26:36.093189+0000","last_became_peered":"2023-05-20T01:26:36.093189+0000","last_unstale":"2023-05-20T01:26:37.086071+0000","last_undegraded":"2023-05-20T01:26:37.086071+0000","last_fullsized":"2023-05-20T01:26:37.086071+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:17:45.202356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.388054+0000","last_change":"2023-05-20T01:26:36.089906+0000","last_active":"2023-05-20T01:26:38.388054+0000","last_peered":"2023-05-20T01:26:38.388054+0000","last_clean":"2023-05-20T01:26:38.388054+0000","last_became_active":"2023-05-20T01:26:36.089609+0000","last_became_peered":"2023-05-20T01:26:36.089609+0000","last_unstale":"2023-05-20T01:26:38.388054+0000","last_undegraded":"2023-05-20T01:26:38.388054+0000","last_fullsized":"2023-05-20T01:26:38.388054+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:29:19.826889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:37.079825+0000","last_change":"2023-05-20T01:26:36.093090+0000","last_active":"2023-05-20T01:26:37.079825+0000","last_peered":"2023-05-20T01:26:37.079825+0000","last_clean":"2023-05-20T01:26:37.079825+0000","last_became_active":"2023-05-20T01:26:36.092763+0000","last_became_peered":"2023-05-20T01:26:36.092763+0000","last_unstale":"2023-05-20T01:26:37.079825+0000","last_undegraded":"2023-05-20T01:26:37.079825+0000","last_fullsized":"2023-05-20T01:26:37.079825+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:43:17.762338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.710862+0000","last_change":"2023-05-20T01:26:36.093707+0000","last_active":"2023-05-20T01:26:38.710862+0000","last_peered":"2023-05-20T01:26:38.710862+0000","last_clean":"2023-05-20T01:26:38.710862+0000","last_became_active":"2023-05-20T01:26:36.093448+0000","last_became_peered":"2023-05-20T01:26:36.093448+0000","last_unstale":"2023-05-20T01:26:38.710862+0000","last_undegraded":"2023-05-20T01:26:38.710862+0000","last_fullsized":"2023-05-20T01:26:38.710862+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:41:14.914426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:37.085434+0000","last_change":"2023-05-20T01:26:36.086895+0000","last_active":"2023-05-20T01:26:37.085434+0000","last_peered":"2023-05-20T01:26:37.085434+0000","last_clean":"2023-05-20T01:26:37.085434+0000","last_became_active":"2023-05-20T01:26:36.086612+0000","last_became_peered":"2023-05-20T01:26:36.086612+0000","last_unstale":"2023-05-20T01:26:37.085434+0000","last_undegraded":"2023-05-20T01:26:37.085434+0000","last_fullsized":"2023-05-20T01:26:37.085434+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:21:15.302621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:37.085177+0000","last_change":"2023-05-20T01:26:36.085521+0000","last_active":"2023-05-20T01:26:37.085177+0000","last_peered":"2023-05-20T01:26:37.085177+0000","last_clean":"2023-05-20T01:26:37.085177+0000","last_became_active":"2023-05-20T01:26:36.085179+0000","last_became_peered":"2023-05-20T01:26:36.085179+0000","last_unstale":"2023-05-20T01:26:37.085177+0000","last_undegraded":"2023-05-20T01:26:37.085177+0000","last_fullsized":"2023-05-20T01:26:37.085177+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:20:28.171785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.710990+0000","last_change":"2023-05-20T01:26:36.092594+0000","last_active":"2023-05-20T01:26:38.710990+0000","last_peered":"2023-05-20T01:26:38.710990+0000","last_clean":"2023-05-20T01:26:38.710990+0000","last_became_active":"2023-05-20T01:26:36.091906+0000","last_became_peered":"2023-05-20T01:26:36.091906+0000","last_unstale":"2023-05-20T01:26:38.710990+0000","last_undegraded":"2023-05-20T01:26:38.710990+0000","last_fullsized":"2023-05-20T01:26:38.710990+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:39:33.187319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:37.085906+0000","last_change":"2023-05-20T01:26:36.094841+0000","last_active":"2023-05-20T01:26:37.085906+0000","last_peered":"2023-05-20T01:26:37.085906+0000","last_clean":"2023-05-20T01:26:37.085906+0000","last_became_active":"2023-05-20T01:26:36.094546+0000","last_became_peered":"2023-05-20T01:26:36.094546+0000","last_unstale":"2023-05-20T01:26:37.085906+0000","last_undegraded":"2023-05-20T01:26:37.085906+0000","last_fullsized":"2023-05-20T01:26:37.085906+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:29:43.382263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.711007+0000","last_change":"2023-05-20T01:26:36.089291+0000","last_active":"2023-05-20T01:26:38.711007+0000","last_peered":"2023-05-20T01:26:38.711007+0000","last_clean":"2023-05-20T01:26:38.711007+0000","last_became_active":"2023-05-20T01:26:36.088987+0000","last_became_peered":"2023-05-20T01:26:36.088987+0000","last_unstale":"2023-05-20T01:26:38.711007+0000","last_undegraded":"2023-05-20T01:26:38.711007+0000","last_fullsized":"2023-05-20T01:26:38.711007+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:59:28.429861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.710513+0000","last_change":"2023-05-20T01:26:36.088457+0000","last_active":"2023-05-20T01:26:38.710513+0000","last_peered":"2023-05-20T01:26:38.710513+0000","last_clean":"2023-05-20T01:26:38.710513+0000","last_became_active":"2023-05-20T01:26:36.087957+0000","last_became_peered":"2023-05-20T01:26:36.087957+0000","last_unstale":"2023-05-20T01:26:38.710513+0000","last_undegraded":"2023-05-20T01:26:38.710513+0000","last_fullsized":"2023-05-20T01:26:38.710513+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T01:29:59.564067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:37.084652+0000","last_change":"2023-05-20T01:26:36.092100+0000","last_active":"2023-05-20T01:26:37.084652+0000","last_peered":"2023-05-20T01:26:37.084652+0000","last_clean":"2023-05-20T01:26:37.084652+0000","last_became_active":"2023-05-20T01:26:36.091816+0000","last_became_peered":"2023-05-20T01:26:36.091816+0000","last_unstale":"2023-05-20T01:26:37.084652+0000","last_undegraded":"2023-05-20T01:26:37.084652+0000","last_fullsized":"2023-05-20T01:26:37.084652+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:12:49.577601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:37.079617+0000","last_change":"2023-05-20T01:26:36.082387+0000","last_active":"2023-05-20T01:26:37.079617+0000","last_peered":"2023-05-20T01:26:37.079617+0000","last_clean":"2023-05-20T01:26:37.079617+0000","last_became_active":"2023-05-20T01:26:36.082097+0000","last_became_peered":"2023-05-20T01:26:36.082097+0000","last_unstale":"2023-05-20T01:26:37.079617+0000","last_undegraded":"2023-05-20T01:26:37.079617+0000","last_fullsized":"2023-05-20T01:26:37.079617+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:04:11.590675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.585216+0000","last_change":"2023-05-20T01:26:36.094589+0000","last_active":"2023-05-20T01:26:38.585216+0000","last_peered":"2023-05-20T01:26:38.585216+0000","last_clean":"2023-05-20T01:26:38.585216+0000","last_became_active":"2023-05-20T01:26:36.094326+0000","last_became_peered":"2023-05-20T01:26:36.094326+0000","last_unstale":"2023-05-20T01:26:38.585216+0000","last_undegraded":"2023-05-20T01:26:38.585216+0000","last_fullsized":"2023-05-20T01:26:38.585216+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:53:33.968042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.585244+0000","last_change":"2023-05-20T01:26:36.092887+0000","last_active":"2023-05-20T01:26:38.585244+0000","last_peered":"2023-05-20T01:26:38.585244+0000","last_clean":"2023-05-20T01:26:38.585244+0000","last_became_active":"2023-05-20T01:26:36.092572+0000","last_became_peered":"2023-05-20T01:26:36.092572+0000","last_unstale":"2023-05-20T01:26:38.585244+0000","last_undegraded":"2023-05-20T01:26:38.585244+0000","last_fullsized":"2023-05-20T01:26:38.585244+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:22:28.115116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.585139+0000","last_change":"2023-05-20T01:26:36.093210+0000","last_active":"2023-05-20T01:26:38.585139+0000","last_peered":"2023-05-20T01:26:38.585139+0000","last_clean":"2023-05-20T01:26:38.585139+0000","last_became_active":"2023-05-20T01:26:36.092935+0000","last_became_peered":"2023-05-20T01:26:36.092935+0000","last_unstale":"2023-05-20T01:26:38.585139+0000","last_undegraded":"2023-05-20T01:26:38.585139+0000","last_fullsized":"2023-05-20T01:26:38.585139+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T01:43:14.591660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.388357+0000","last_change":"2023-05-20T01:26:36.090126+0000","last_active":"2023-05-20T01:26:38.388357+0000","last_peered":"2023-05-20T01:26:38.388357+0000","last_clean":"2023-05-20T01:26:38.388357+0000","last_became_active":"2023-05-20T01:26:36.089858+0000","last_became_peered":"2023-05-20T01:26:36.089858+0000","last_unstale":"2023-05-20T01:26:38.388357+0000","last_undegraded":"2023-05-20T01:26:38.388357+0000","last_fullsized":"2023-05-20T01:26:38.388357+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:16:06.854941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:37.085974+0000","last_change":"2023-05-20T01:26:36.085121+0000","last_active":"2023-05-20T01:26:37.085974+0000","last_peered":"2023-05-20T01:26:37.085974+0000","last_clean":"2023-05-20T01:26:37.085974+0000","last_became_active":"2023-05-20T01:26:36.084610+0000","last_became_peered":"2023-05-20T01:26:36.084610+0000","last_unstale":"2023-05-20T01:26:37.085974+0000","last_undegraded":"2023-05-20T01:26:37.085974+0000","last_fullsized":"2023-05-20T01:26:37.085974+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:55:47.951254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.388149+0000","last_change":"2023-05-20T01:26:36.094178+0000","last_active":"2023-05-20T01:26:38.388149+0000","last_peered":"2023-05-20T01:26:38.388149+0000","last_clean":"2023-05-20T01:26:38.388149+0000","last_became_active":"2023-05-20T01:26:36.093750+0000","last_became_peered":"2023-05-20T01:26:36.093750+0000","last_unstale":"2023-05-20T01:26:38.388149+0000","last_undegraded":"2023-05-20T01:26:38.388149+0000","last_fullsized":"2023-05-20T01:26:38.388149+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:45:42.406317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:37.084486+0000","last_change":"2023-05-20T01:26:36.095210+0000","last_active":"2023-05-20T01:26:37.084486+0000","last_peered":"2023-05-20T01:26:37.084486+0000","last_clean":"2023-05-20T01:26:37.084486+0000","last_became_active":"2023-05-20T01:26:36.094887+0000","last_became_peered":"2023-05-20T01:26:36.094887+0000","last_unstale":"2023-05-20T01:26:37.084486+0000","last_undegraded":"2023-05-20T01:26:37.084486+0000","last_fullsized":"2023-05-20T01:26:37.084486+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:04:53.469731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.710431+0000","last_change":"2023-05-20T01:26:36.097965+0000","last_active":"2023-05-20T01:26:38.710431+0000","last_peered":"2023-05-20T01:26:38.710431+0000","last_clean":"2023-05-20T01:26:38.710431+0000","last_became_active":"2023-05-20T01:26:36.097743+0000","last_became_peered":"2023-05-20T01:26:36.097743+0000","last_unstale":"2023-05-20T01:26:38.710431+0000","last_undegraded":"2023-05-20T01:26:38.710431+0000","last_fullsized":"2023-05-20T01:26:38.710431+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:33:01.953967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.710901+0000","last_change":"2023-05-20T01:26:36.093120+0000","last_active":"2023-05-20T01:26:38.710901+0000","last_peered":"2023-05-20T01:26:38.710901+0000","last_clean":"2023-05-20T01:26:38.710901+0000","last_became_active":"2023-05-20T01:26:36.092842+0000","last_became_peered":"2023-05-20T01:26:36.092842+0000","last_unstale":"2023-05-20T01:26:38.710901+0000","last_undegraded":"2023-05-20T01:26:38.710901+0000","last_fullsized":"2023-05-20T01:26:38.710901+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:18:32.715040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.387928+0000","last_change":"2023-05-20T01:26:36.084528+0000","last_active":"2023-05-20T01:26:38.387928+0000","last_peered":"2023-05-20T01:26:38.387928+0000","last_clean":"2023-05-20T01:26:38.387928+0000","last_became_active":"2023-05-20T01:26:36.084261+0000","last_became_peered":"2023-05-20T01:26:36.084261+0000","last_unstale":"2023-05-20T01:26:38.387928+0000","last_undegraded":"2023-05-20T01:26:38.387928+0000","last_fullsized":"2023-05-20T01:26:38.387928+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T13:21:38.550913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.710170+0000","last_change":"2023-05-20T01:26:36.088685+0000","last_active":"2023-05-20T01:26:38.710170+0000","last_peered":"2023-05-20T01:26:38.710170+0000","last_clean":"2023-05-20T01:26:38.710170+0000","last_became_active":"2023-05-20T01:26:36.088420+0000","last_became_peered":"2023-05-20T01:26:36.088420+0000","last_unstale":"2023-05-20T01:26:38.710170+0000","last_undegraded":"2023-05-20T01:26:38.710170+0000","last_fullsized":"2023-05-20T01:26:38.710170+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:48:55.089787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:37.084874+0000","last_change":"2023-05-20T01:26:36.098577+0000","last_active":"2023-05-20T01:26:37.084874+0000","last_peered":"2023-05-20T01:26:37.084874+0000","last_clean":"2023-05-20T01:26:37.084874+0000","last_became_active":"2023-05-20T01:26:36.098167+0000","last_became_peered":"2023-05-20T01:26:36.098167+0000","last_unstale":"2023-05-20T01:26:37.084874+0000","last_undegraded":"2023-05-20T01:26:37.084874+0000","last_fullsized":"2023-05-20T01:26:37.084874+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:17:37.043790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:37.085072+0000","last_change":"2023-05-20T01:26:36.097405+0000","last_active":"2023-05-20T01:26:37.085072+0000","last_peered":"2023-05-20T01:26:37.085072+0000","last_clean":"2023-05-20T01:26:37.085072+0000","last_became_active":"2023-05-20T01:26:36.097120+0000","last_became_peered":"2023-05-20T01:26:36.097120+0000","last_unstale":"2023-05-20T01:26:37.085072+0000","last_undegraded":"2023-05-20T01:26:37.085072+0000","last_fullsized":"2023-05-20T01:26:37.085072+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:26:42.366377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.710531+0000","last_change":"2023-05-20T01:26:36.093252+0000","last_active":"2023-05-20T01:26:38.710531+0000","last_peered":"2023-05-20T01:26:38.710531+0000","last_clean":"2023-05-20T01:26:38.710531+0000","last_became_active":"2023-05-20T01:26:36.092989+0000","last_became_peered":"2023-05-20T01:26:36.092989+0000","last_unstale":"2023-05-20T01:26:38.710531+0000","last_undegraded":"2023-05-20T01:26:38.710531+0000","last_fullsized":"2023-05-20T01:26:38.710531+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:56:41.862467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:37.085582+0000","last_change":"2023-05-20T01:26:36.088025+0000","last_active":"2023-05-20T01:26:37.085582+0000","last_peered":"2023-05-20T01:26:37.085582+0000","last_clean":"2023-05-20T01:26:37.085582+0000","last_became_active":"2023-05-20T01:26:36.087683+0000","last_became_peered":"2023-05-20T01:26:36.087683+0000","last_unstale":"2023-05-20T01:26:37.085582+0000","last_undegraded":"2023-05-20T01:26:37.085582+0000","last_fullsized":"2023-05-20T01:26:37.085582+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:11:00.910242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.710443+0000","last_change":"2023-05-20T01:26:36.097332+0000","last_active":"2023-05-20T01:26:38.710443+0000","last_peered":"2023-05-20T01:26:38.710443+0000","last_clean":"2023-05-20T01:26:38.710443+0000","last_became_active":"2023-05-20T01:26:36.097091+0000","last_became_peered":"2023-05-20T01:26:36.097091+0000","last_unstale":"2023-05-20T01:26:38.710443+0000","last_undegraded":"2023-05-20T01:26:38.710443+0000","last_fullsized":"2023-05-20T01:26:38.710443+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T01:55:28.226587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:37.084366+0000","last_change":"2023-05-20T01:26:36.098206+0000","last_active":"2023-05-20T01:26:37.084366+0000","last_peered":"2023-05-20T01:26:37.084366+0000","last_clean":"2023-05-20T01:26:37.084366+0000","last_became_active":"2023-05-20T01:26:36.097871+0000","last_became_peered":"2023-05-20T01:26:36.097871+0000","last_unstale":"2023-05-20T01:26:37.084366+0000","last_undegraded":"2023-05-20T01:26:37.084366+0000","last_fullsized":"2023-05-20T01:26:37.084366+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:57:28.773432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:37.085356+0000","last_change":"2023-05-20T01:26:36.090284+0000","last_active":"2023-05-20T01:26:37.085356+0000","last_peered":"2023-05-20T01:26:37.085356+0000","last_clean":"2023-05-20T01:26:37.085356+0000","last_became_active":"2023-05-20T01:26:36.089977+0000","last_became_peered":"2023-05-20T01:26:36.089977+0000","last_unstale":"2023-05-20T01:26:37.085356+0000","last_undegraded":"2023-05-20T01:26:37.085356+0000","last_fullsized":"2023-05-20T01:26:37.085356+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T01:33:35.105606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.710420+0000","last_change":"2023-05-20T01:26:36.093275+0000","last_active":"2023-05-20T01:26:38.710420+0000","last_peered":"2023-05-20T01:26:38.710420+0000","last_clean":"2023-05-20T01:26:38.710420+0000","last_became_active":"2023-05-20T01:26:36.093007+0000","last_became_peered":"2023-05-20T01:26:36.093007+0000","last_unstale":"2023-05-20T01:26:38.710420+0000","last_undegraded":"2023-05-20T01:26:38.710420+0000","last_fullsized":"2023-05-20T01:26:38.710420+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:28:12.346951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.388141+0000","last_change":"2023-05-20T01:26:36.088499+0000","last_active":"2023-05-20T01:26:38.388141+0000","last_peered":"2023-05-20T01:26:38.388141+0000","last_clean":"2023-05-20T01:26:38.388141+0000","last_became_active":"2023-05-20T01:26:36.088216+0000","last_became_peered":"2023-05-20T01:26:36.088216+0000","last_unstale":"2023-05-20T01:26:38.388141+0000","last_undegraded":"2023-05-20T01:26:38.388141+0000","last_fullsized":"2023-05-20T01:26:38.388141+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:27:02.653342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"111'4","reported_seq":17,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:37.079844+0000","last_change":"2023-05-20T01:26:35.658045+0000","last_active":"2023-05-20T01:26:37.079844+0000","last_peered":"2023-05-20T01:26:37.079844+0000","last_clean":"2023-05-20T01:26:37.079844+0000","last_became_active":"2023-05-20T01:26:35.657827+0000","last_became_peered":"2023-05-20T01:26:35.657827+0000","last_unstale":"2023-05-20T01:26:37.079844+0000","last_undegraded":"2023-05-20T01:26:37.079844+0000","last_fullsized":"2023-05-20T01:26:37.079844+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:04:53.335119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.710845+0000","last_change":"2023-05-20T01:26:36.096394+0000","last_active":"2023-05-20T01:26:38.710845+0000","last_peered":"2023-05-20T01:26:38.710845+0000","last_clean":"2023-05-20T01:26:38.710845+0000","last_became_active":"2023-05-20T01:26:36.096130+0000","last_became_peered":"2023-05-20T01:26:36.096130+0000","last_unstale":"2023-05-20T01:26:38.710845+0000","last_undegraded":"2023-05-20T01:26:38.710845+0000","last_fullsized":"2023-05-20T01:26:38.710845+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:33:46.171949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"111'1","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.710699+0000","last_change":"2023-05-20T01:26:35.663547+0000","last_active":"2023-05-20T01:26:38.710699+0000","last_peered":"2023-05-20T01:26:38.710699+0000","last_clean":"2023-05-20T01:26:38.710699+0000","last_became_active":"2023-05-20T01:26:35.663282+0000","last_became_peered":"2023-05-20T01:26:35.663282+0000","last_unstale":"2023-05-20T01:26:38.710699+0000","last_undegraded":"2023-05-20T01:26:38.710699+0000","last_fullsized":"2023-05-20T01:26:38.710699+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:12:00.672741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:37.084432+0000","last_change":"2023-05-20T01:26:36.091809+0000","last_active":"2023-05-20T01:26:37.084432+0000","last_peered":"2023-05-20T01:26:37.084432+0000","last_clean":"2023-05-20T01:26:37.084432+0000","last_became_active":"2023-05-20T01:26:36.091435+0000","last_became_peered":"2023-05-20T01:26:36.091435+0000","last_unstale":"2023-05-20T01:26:37.084432+0000","last_undegraded":"2023-05-20T01:26:37.084432+0000","last_fullsized":"2023-05-20T01:26:37.084432+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:18:59.295525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"111'2","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:37.084653+0000","last_change":"2023-05-20T01:26:35.662991+0000","last_active":"2023-05-20T01:26:37.084653+0000","last_peered":"2023-05-20T01:26:37.084653+0000","last_clean":"2023-05-20T01:26:37.084653+0000","last_became_active":"2023-05-20T01:26:35.662055+0000","last_became_peered":"2023-05-20T01:26:35.662055+0000","last_unstale":"2023-05-20T01:26:37.084653+0000","last_undegraded":"2023-05-20T01:26:37.084653+0000","last_fullsized":"2023-05-20T01:26:37.084653+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:11:56.969558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.710254+0000","last_change":"2023-05-20T01:26:36.090727+0000","last_active":"2023-05-20T01:26:38.710254+0000","last_peered":"2023-05-20T01:26:38.710254+0000","last_clean":"2023-05-20T01:26:38.710254+0000","last_became_active":"2023-05-20T01:26:36.090471+0000","last_became_peered":"2023-05-20T01:26:36.090471+0000","last_unstale":"2023-05-20T01:26:38.710254+0000","last_undegraded":"2023-05-20T01:26:38.710254+0000","last_fullsized":"2023-05-20T01:26:38.710254+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:54:14.429799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"111'2","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.710939+0000","last_change":"2023-05-20T01:26:35.662610+0000","last_active":"2023-05-20T01:26:38.710939+0000","last_peered":"2023-05-20T01:26:38.710939+0000","last_clean":"2023-05-20T01:26:38.710939+0000","last_became_active":"2023-05-20T01:26:35.662325+0000","last_became_peered":"2023-05-20T01:26:35.662325+0000","last_unstale":"2023-05-20T01:26:38.710939+0000","last_undegraded":"2023-05-20T01:26:38.710939+0000","last_fullsized":"2023-05-20T01:26:38.710939+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:41:19.578105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:37.079965+0000","last_change":"2023-05-20T01:26:36.085271+0000","last_active":"2023-05-20T01:26:37.079965+0000","last_peered":"2023-05-20T01:26:37.079965+0000","last_clean":"2023-05-20T01:26:37.079965+0000","last_became_active":"2023-05-20T01:26:36.085000+0000","last_became_peered":"2023-05-20T01:26:36.085000+0000","last_unstale":"2023-05-20T01:26:37.079965+0000","last_undegraded":"2023-05-20T01:26:37.079965+0000","last_fullsized":"2023-05-20T01:26:37.079965+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:55:54.583693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"111'1","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.585261+0000","last_change":"2023-05-20T01:26:35.665455+0000","last_active":"2023-05-20T01:26:38.585261+0000","last_peered":"2023-05-20T01:26:38.585261+0000","last_clean":"2023-05-20T01:26:38.585261+0000","last_became_active":"2023-05-20T01:26:35.665131+0000","last_became_peered":"2023-05-20T01:26:35.665131+0000","last_unstale":"2023-05-20T01:26:38.585261+0000","last_undegraded":"2023-05-20T01:26:38.585261+0000","last_fullsized":"2023-05-20T01:26:38.585261+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:08:18.586264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.388276+0000","last_change":"2023-05-20T01:26:36.094004+0000","last_active":"2023-05-20T01:26:38.388276+0000","last_peered":"2023-05-20T01:26:38.388276+0000","last_clean":"2023-05-20T01:26:38.388276+0000","last_became_active":"2023-05-20T01:26:36.093393+0000","last_became_peered":"2023-05-20T01:26:36.093393+0000","last_unstale":"2023-05-20T01:26:38.388276+0000","last_undegraded":"2023-05-20T01:26:38.388276+0000","last_fullsized":"2023-05-20T01:26:38.388276+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:30:02.313003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"111'1","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.710035+0000","last_change":"2023-05-20T01:26:35.664025+0000","last_active":"2023-05-20T01:26:38.710035+0000","last_peered":"2023-05-20T01:26:38.710035+0000","last_clean":"2023-05-20T01:26:38.710035+0000","last_became_active":"2023-05-20T01:26:35.663803+0000","last_became_peered":"2023-05-20T01:26:35.663803+0000","last_unstale":"2023-05-20T01:26:38.710035+0000","last_undegraded":"2023-05-20T01:26:38.710035+0000","last_fullsized":"2023-05-20T01:26:38.710035+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:34:07.792503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:37.086053+0000","last_change":"2023-05-20T01:26:36.085317+0000","last_active":"2023-05-20T01:26:37.086053+0000","last_peered":"2023-05-20T01:26:37.086053+0000","last_clean":"2023-05-20T01:26:37.086053+0000","last_became_active":"2023-05-20T01:26:36.084792+0000","last_became_peered":"2023-05-20T01:26:36.084792+0000","last_unstale":"2023-05-20T01:26:37.086053+0000","last_undegraded":"2023-05-20T01:26:37.086053+0000","last_fullsized":"2023-05-20T01:26:37.086053+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:40:05.319863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:37.084283+0000","last_change":"2023-05-20T01:26:35.661709+0000","last_active":"2023-05-20T01:26:37.084283+0000","last_peered":"2023-05-20T01:26:37.084283+0000","last_clean":"2023-05-20T01:26:37.084283+0000","last_became_active":"2023-05-20T01:26:35.661425+0000","last_became_peered":"2023-05-20T01:26:35.661425+0000","last_unstale":"2023-05-20T01:26:37.084283+0000","last_undegraded":"2023-05-20T01:26:37.084283+0000","last_fullsized":"2023-05-20T01:26:37.084283+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:16:44.004923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:37.080031+0000","last_change":"2023-05-20T01:26:36.085287+0000","last_active":"2023-05-20T01:26:37.080031+0000","last_peered":"2023-05-20T01:26:37.080031+0000","last_clean":"2023-05-20T01:26:37.080031+0000","last_became_active":"2023-05-20T01:26:36.085013+0000","last_became_peered":"2023-05-20T01:26:36.085013+0000","last_unstale":"2023-05-20T01:26:37.080031+0000","last_undegraded":"2023-05-20T01:26:37.080031+0000","last_fullsized":"2023-05-20T01:26:37.080031+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:11:15.186887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:37.079515+0000","last_change":"2023-05-20T01:26:35.649191+0000","last_active":"2023-05-20T01:26:37.079515+0000","last_peered":"2023-05-20T01:26:37.079515+0000","last_clean":"2023-05-20T01:26:37.079515+0000","last_became_active":"2023-05-20T01:26:35.648091+0000","last_became_peered":"2023-05-20T01:26:35.648091+0000","last_unstale":"2023-05-20T01:26:37.079515+0000","last_undegraded":"2023-05-20T01:26:37.079515+0000","last_fullsized":"2023-05-20T01:26:37.079515+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:10:27.773443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.388142+0000","last_change":"2023-05-20T01:26:36.097545+0000","last_active":"2023-05-20T01:26:38.388142+0000","last_peered":"2023-05-20T01:26:38.388142+0000","last_clean":"2023-05-20T01:26:38.388142+0000","last_became_active":"2023-05-20T01:26:36.093102+0000","last_became_peered":"2023-05-20T01:26:36.093102+0000","last_unstale":"2023-05-20T01:26:38.388142+0000","last_undegraded":"2023-05-20T01:26:38.388142+0000","last_fullsized":"2023-05-20T01:26:38.388142+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:40:37.047579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:37.084969+0000","last_change":"2023-05-20T01:26:35.661860+0000","last_active":"2023-05-20T01:26:37.084969+0000","last_peered":"2023-05-20T01:26:37.084969+0000","last_clean":"2023-05-20T01:26:37.084969+0000","last_became_active":"2023-05-20T01:26:35.661520+0000","last_became_peered":"2023-05-20T01:26:35.661520+0000","last_unstale":"2023-05-20T01:26:37.084969+0000","last_undegraded":"2023-05-20T01:26:37.084969+0000","last_fullsized":"2023-05-20T01:26:37.084969+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:26:28.474417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.710779+0000","last_change":"2023-05-20T01:26:36.101022+0000","last_active":"2023-05-20T01:26:38.710779+0000","last_peered":"2023-05-20T01:26:38.710779+0000","last_clean":"2023-05-20T01:26:38.710779+0000","last_became_active":"2023-05-20T01:26:36.100820+0000","last_became_peered":"2023-05-20T01:26:36.100820+0000","last_unstale":"2023-05-20T01:26:38.710779+0000","last_undegraded":"2023-05-20T01:26:38.710779+0000","last_fullsized":"2023-05-20T01:26:38.710779+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:30:15.369472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.388507+0000","last_change":"2023-05-20T01:26:35.660602+0000","last_active":"2023-05-20T01:26:38.388507+0000","last_peered":"2023-05-20T01:26:38.388507+0000","last_clean":"2023-05-20T01:26:38.388507+0000","last_became_active":"2023-05-20T01:26:35.660324+0000","last_became_peered":"2023-05-20T01:26:35.660324+0000","last_unstale":"2023-05-20T01:26:38.388507+0000","last_undegraded":"2023-05-20T01:26:38.388507+0000","last_fullsized":"2023-05-20T01:26:38.388507+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:42:31.290518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.710399+0000","last_change":"2023-05-20T01:26:36.099492+0000","last_active":"2023-05-20T01:26:38.710399+0000","last_peered":"2023-05-20T01:26:38.710399+0000","last_clean":"2023-05-20T01:26:38.710399+0000","last_became_active":"2023-05-20T01:26:36.099226+0000","last_became_peered":"2023-05-20T01:26:36.099226+0000","last_unstale":"2023-05-20T01:26:38.710399+0000","last_undegraded":"2023-05-20T01:26:38.710399+0000","last_fullsized":"2023-05-20T01:26:38.710399+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:45:00.597228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"111'3","reported_seq":16,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.585343+0000","last_change":"2023-05-20T01:26:35.660335+0000","last_active":"2023-05-20T01:26:38.585343+0000","last_peered":"2023-05-20T01:26:38.585343+0000","last_clean":"2023-05-20T01:26:38.585343+0000","last_became_active":"2023-05-20T01:26:35.660069+0000","last_became_peered":"2023-05-20T01:26:35.660069+0000","last_unstale":"2023-05-20T01:26:38.585343+0000","last_undegraded":"2023-05-20T01:26:38.585343+0000","last_fullsized":"2023-05-20T01:26:38.585343+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T01:48:11.207779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.710588+0000","last_change":"2023-05-20T01:26:36.099329+0000","last_active":"2023-05-20T01:26:38.710588+0000","last_peered":"2023-05-20T01:26:38.710588+0000","last_clean":"2023-05-20T01:26:38.710588+0000","last_became_active":"2023-05-20T01:26:36.099062+0000","last_became_peered":"2023-05-20T01:26:36.099062+0000","last_unstale":"2023-05-20T01:26:38.710588+0000","last_undegraded":"2023-05-20T01:26:38.710588+0000","last_fullsized":"2023-05-20T01:26:38.710588+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:59:31.710005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"111'6","reported_seq":19,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:37.085649+0000","last_change":"2023-05-20T01:26:35.662782+0000","last_active":"2023-05-20T01:26:37.085649+0000","last_peered":"2023-05-20T01:26:37.085649+0000","last_clean":"2023-05-20T01:26:37.085649+0000","last_became_active":"2023-05-20T01:26:35.662462+0000","last_became_peered":"2023-05-20T01:26:35.662462+0000","last_unstale":"2023-05-20T01:26:37.085649+0000","last_undegraded":"2023-05-20T01:26:37.085649+0000","last_fullsized":"2023-05-20T01:26:37.085649+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:22:15.875557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.710253+0000","last_change":"2023-05-20T01:26:36.089582+0000","last_active":"2023-05-20T01:26:38.710253+0000","last_peered":"2023-05-20T01:26:38.710253+0000","last_clean":"2023-05-20T01:26:38.710253+0000","last_became_active":"2023-05-20T01:26:36.089301+0000","last_became_peered":"2023-05-20T01:26:36.089301+0000","last_unstale":"2023-05-20T01:26:38.710253+0000","last_undegraded":"2023-05-20T01:26:38.710253+0000","last_fullsized":"2023-05-20T01:26:38.710253+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:54:08.199336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"111'2","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.710957+0000","last_change":"2023-05-20T01:26:35.662523+0000","last_active":"2023-05-20T01:26:38.710957+0000","last_peered":"2023-05-20T01:26:38.710957+0000","last_clean":"2023-05-20T01:26:38.710957+0000","last_became_active":"2023-05-20T01:26:35.662205+0000","last_became_peered":"2023-05-20T01:26:35.662205+0000","last_unstale":"2023-05-20T01:26:38.710957+0000","last_undegraded":"2023-05-20T01:26:38.710957+0000","last_fullsized":"2023-05-20T01:26:38.710957+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:36:38.412172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:37.085851+0000","last_change":"2023-05-20T01:26:36.086463+0000","last_active":"2023-05-20T01:26:37.085851+0000","last_peered":"2023-05-20T01:26:37.085851+0000","last_clean":"2023-05-20T01:26:37.085851+0000","last_became_active":"2023-05-20T01:26:36.086154+0000","last_became_peered":"2023-05-20T01:26:36.086154+0000","last_unstale":"2023-05-20T01:26:37.085851+0000","last_undegraded":"2023-05-20T01:26:37.085851+0000","last_fullsized":"2023-05-20T01:26:37.085851+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:18:48.948205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:37.079815+0000","last_change":"2023-05-20T01:26:35.661955+0000","last_active":"2023-05-20T01:26:37.079815+0000","last_peered":"2023-05-20T01:26:37.079815+0000","last_clean":"2023-05-20T01:26:37.079815+0000","last_became_active":"2023-05-20T01:26:35.661556+0000","last_became_peered":"2023-05-20T01:26:35.661556+0000","last_unstale":"2023-05-20T01:26:37.079815+0000","last_undegraded":"2023-05-20T01:26:37.079815+0000","last_fullsized":"2023-05-20T01:26:37.079815+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:18:07.621593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.710599+0000","last_change":"2023-05-20T01:26:36.097947+0000","last_active":"2023-05-20T01:26:38.710599+0000","last_peered":"2023-05-20T01:26:38.710599+0000","last_clean":"2023-05-20T01:26:38.710599+0000","last_became_active":"2023-05-20T01:26:36.097689+0000","last_became_peered":"2023-05-20T01:26:36.097689+0000","last_unstale":"2023-05-20T01:26:38.710599+0000","last_undegraded":"2023-05-20T01:26:38.710599+0000","last_fullsized":"2023-05-20T01:26:38.710599+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:13:38.420510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.710554+0000","last_change":"2023-05-20T01:26:35.660215+0000","last_active":"2023-05-20T01:26:38.710554+0000","last_peered":"2023-05-20T01:26:38.710554+0000","last_clean":"2023-05-20T01:26:38.710554+0000","last_became_active":"2023-05-20T01:26:35.659873+0000","last_became_peered":"2023-05-20T01:26:35.659873+0000","last_unstale":"2023-05-20T01:26:38.710554+0000","last_undegraded":"2023-05-20T01:26:38.710554+0000","last_fullsized":"2023-05-20T01:26:38.710554+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:33:52.131861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.388408+0000","last_change":"2023-05-20T01:26:36.090068+0000","last_active":"2023-05-20T01:26:38.388408+0000","last_peered":"2023-05-20T01:26:38.388408+0000","last_clean":"2023-05-20T01:26:38.388408+0000","last_became_active":"2023-05-20T01:26:36.089776+0000","last_became_peered":"2023-05-20T01:26:36.089776+0000","last_unstale":"2023-05-20T01:26:38.388408+0000","last_undegraded":"2023-05-20T01:26:38.388408+0000","last_fullsized":"2023-05-20T01:26:38.388408+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:46:24.436985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"111'8","reported_seq":21,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:37.084312+0000","last_change":"2023-05-20T01:26:35.662613+0000","last_active":"2023-05-20T01:26:37.084312+0000","last_peered":"2023-05-20T01:26:37.084312+0000","last_clean":"2023-05-20T01:26:37.084312+0000","last_became_active":"2023-05-20T01:26:35.661401+0000","last_became_peered":"2023-05-20T01:26:35.661401+0000","last_unstale":"2023-05-20T01:26:37.084312+0000","last_undegraded":"2023-05-20T01:26:37.084312+0000","last_fullsized":"2023-05-20T01:26:37.084312+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:35:33.516158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.387962+0000","last_change":"2023-05-20T01:26:36.087191+0000","last_active":"2023-05-20T01:26:38.387962+0000","last_peered":"2023-05-20T01:26:38.387962+0000","last_clean":"2023-05-20T01:26:38.387962+0000","last_became_active":"2023-05-20T01:26:36.086899+0000","last_became_peered":"2023-05-20T01:26:36.086899+0000","last_unstale":"2023-05-20T01:26:38.387962+0000","last_undegraded":"2023-05-20T01:26:38.387962+0000","last_fullsized":"2023-05-20T01:26:38.387962+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:22:24.848309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"111'2","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.710629+0000","last_change":"2023-05-20T01:26:35.662291+0000","last_active":"2023-05-20T01:26:38.710629+0000","last_peered":"2023-05-20T01:26:38.710629+0000","last_clean":"2023-05-20T01:26:38.710629+0000","last_became_active":"2023-05-20T01:26:35.661970+0000","last_became_peered":"2023-05-20T01:26:35.661970+0000","last_unstale":"2023-05-20T01:26:38.710629+0000","last_undegraded":"2023-05-20T01:26:38.710629+0000","last_fullsized":"2023-05-20T01:26:38.710629+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:34:56.803848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:37.084339+0000","last_change":"2023-05-20T01:26:36.087917+0000","last_active":"2023-05-20T01:26:37.084339+0000","last_peered":"2023-05-20T01:26:37.084339+0000","last_clean":"2023-05-20T01:26:37.084339+0000","last_became_active":"2023-05-20T01:26:36.087647+0000","last_became_peered":"2023-05-20T01:26:36.087647+0000","last_unstale":"2023-05-20T01:26:37.084339+0000","last_undegraded":"2023-05-20T01:26:37.084339+0000","last_fullsized":"2023-05-20T01:26:37.084339+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:40:40.590526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.388290+0000","last_change":"2023-05-20T01:26:35.660495+0000","last_active":"2023-05-20T01:26:38.388290+0000","last_peered":"2023-05-20T01:26:38.388290+0000","last_clean":"2023-05-20T01:26:38.388290+0000","last_became_active":"2023-05-20T01:26:35.660218+0000","last_became_peered":"2023-05-20T01:26:35.660218+0000","last_unstale":"2023-05-20T01:26:38.388290+0000","last_undegraded":"2023-05-20T01:26:38.388290+0000","last_fullsized":"2023-05-20T01:26:38.388290+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T13:17:28.416406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:37.079885+0000","last_change":"2023-05-20T01:26:36.077930+0000","last_active":"2023-05-20T01:26:37.079885+0000","last_peered":"2023-05-20T01:26:37.079885+0000","last_clean":"2023-05-20T01:26:37.079885+0000","last_became_active":"2023-05-20T01:26:36.077655+0000","last_became_peered":"2023-05-20T01:26:36.077655+0000","last_unstale":"2023-05-20T01:26:37.079885+0000","last_undegraded":"2023-05-20T01:26:37.079885+0000","last_fullsized":"2023-05-20T01:26:37.079885+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:07:19.398609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:37.085847+0000","last_change":"2023-05-20T01:26:35.655521+0000","last_active":"2023-05-20T01:26:37.085847+0000","last_peered":"2023-05-20T01:26:37.085847+0000","last_clean":"2023-05-20T01:26:37.085847+0000","last_became_active":"2023-05-20T01:26:35.655217+0000","last_became_peered":"2023-05-20T01:26:35.655217+0000","last_unstale":"2023-05-20T01:26:37.085847+0000","last_undegraded":"2023-05-20T01:26:37.085847+0000","last_fullsized":"2023-05-20T01:26:37.085847+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:28:08.499442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.710124+0000","last_change":"2023-05-20T01:26:36.088402+0000","last_active":"2023-05-20T01:26:38.710124+0000","last_peered":"2023-05-20T01:26:38.710124+0000","last_clean":"2023-05-20T01:26:38.710124+0000","last_became_active":"2023-05-20T01:26:36.085409+0000","last_became_peered":"2023-05-20T01:26:36.085409+0000","last_unstale":"2023-05-20T01:26:38.710124+0000","last_undegraded":"2023-05-20T01:26:38.710124+0000","last_fullsized":"2023-05-20T01:26:38.710124+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:00:02.738451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.711040+0000","last_change":"2023-05-20T01:26:35.652937+0000","last_active":"2023-05-20T01:26:38.711040+0000","last_peered":"2023-05-20T01:26:38.711040+0000","last_clean":"2023-05-20T01:26:38.711040+0000","last_became_active":"2023-05-20T01:26:35.652635+0000","last_became_peered":"2023-05-20T01:26:35.652635+0000","last_unstale":"2023-05-20T01:26:38.711040+0000","last_undegraded":"2023-05-20T01:26:38.711040+0000","last_fullsized":"2023-05-20T01:26:38.711040+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:24:24.769004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.710820+0000","last_change":"2023-05-20T01:26:36.092027+0000","last_active":"2023-05-20T01:26:38.710820+0000","last_peered":"2023-05-20T01:26:38.710820+0000","last_clean":"2023-05-20T01:26:38.710820+0000","last_became_active":"2023-05-20T01:26:36.091712+0000","last_became_peered":"2023-05-20T01:26:36.091712+0000","last_unstale":"2023-05-20T01:26:38.710820+0000","last_undegraded":"2023-05-20T01:26:38.710820+0000","last_fullsized":"2023-05-20T01:26:38.710820+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:32:55.788621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:37.084437+0000","last_change":"2023-05-20T01:26:35.665478+0000","last_active":"2023-05-20T01:26:37.084437+0000","last_peered":"2023-05-20T01:26:37.084437+0000","last_clean":"2023-05-20T01:26:37.084437+0000","last_became_active":"2023-05-20T01:26:35.665164+0000","last_became_peered":"2023-05-20T01:26:35.665164+0000","last_unstale":"2023-05-20T01:26:37.084437+0000","last_undegraded":"2023-05-20T01:26:37.084437+0000","last_fullsized":"2023-05-20T01:26:37.084437+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:42:07.320423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.710624+0000","last_change":"2023-05-20T01:26:36.088743+0000","last_active":"2023-05-20T01:26:38.710624+0000","last_peered":"2023-05-20T01:26:38.710624+0000","last_clean":"2023-05-20T01:26:38.710624+0000","last_became_active":"2023-05-20T01:26:36.088466+0000","last_became_peered":"2023-05-20T01:26:36.088466+0000","last_unstale":"2023-05-20T01:26:38.710624+0000","last_undegraded":"2023-05-20T01:26:38.710624+0000","last_fullsized":"2023-05-20T01:26:38.710624+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:04:01.736880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.710626+0000","last_change":"2023-05-20T01:26:35.660517+0000","last_active":"2023-05-20T01:26:38.710626+0000","last_peered":"2023-05-20T01:26:38.710626+0000","last_clean":"2023-05-20T01:26:38.710626+0000","last_became_active":"2023-05-20T01:26:35.660240+0000","last_became_peered":"2023-05-20T01:26:35.660240+0000","last_unstale":"2023-05-20T01:26:38.710626+0000","last_undegraded":"2023-05-20T01:26:38.710626+0000","last_fullsized":"2023-05-20T01:26:38.710626+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:22:05.762439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.585497+0000","last_change":"2023-05-20T01:26:36.093031+0000","last_active":"2023-05-20T01:26:38.585497+0000","last_peered":"2023-05-20T01:26:38.585497+0000","last_clean":"2023-05-20T01:26:38.585497+0000","last_became_active":"2023-05-20T01:26:36.092732+0000","last_became_peered":"2023-05-20T01:26:36.092732+0000","last_unstale":"2023-05-20T01:26:38.585497+0000","last_undegraded":"2023-05-20T01:26:38.585497+0000","last_fullsized":"2023-05-20T01:26:38.585497+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:30:31.423277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:37.085471+0000","last_change":"2023-05-20T01:26:35.653958+0000","last_active":"2023-05-20T01:26:37.085471+0000","last_peered":"2023-05-20T01:26:37.085471+0000","last_clean":"2023-05-20T01:26:37.085471+0000","last_became_active":"2023-05-20T01:26:35.653285+0000","last_became_peered":"2023-05-20T01:26:35.653285+0000","last_unstale":"2023-05-20T01:26:37.085471+0000","last_undegraded":"2023-05-20T01:26:37.085471+0000","last_fullsized":"2023-05-20T01:26:37.085471+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:52:23.814568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.710424+0000","last_change":"2023-05-20T01:26:36.095384+0000","last_active":"2023-05-20T01:26:38.710424+0000","last_peered":"2023-05-20T01:26:38.710424+0000","last_clean":"2023-05-20T01:26:38.710424+0000","last_became_active":"2023-05-20T01:26:36.095143+0000","last_became_peered":"2023-05-20T01:26:36.095143+0000","last_unstale":"2023-05-20T01:26:38.710424+0000","last_undegraded":"2023-05-20T01:26:38.710424+0000","last_fullsized":"2023-05-20T01:26:38.710424+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:28:21.553443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.711154+0000","last_change":"2023-05-20T01:26:35.662061+0000","last_active":"2023-05-20T01:26:38.711154+0000","last_peered":"2023-05-20T01:26:38.711154+0000","last_clean":"2023-05-20T01:26:38.711154+0000","last_became_active":"2023-05-20T01:26:35.660711+0000","last_became_peered":"2023-05-20T01:26:35.660711+0000","last_unstale":"2023-05-20T01:26:38.711154+0000","last_undegraded":"2023-05-20T01:26:38.711154+0000","last_fullsized":"2023-05-20T01:26:38.711154+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:33:20.487893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.388177+0000","last_change":"2023-05-20T01:26:36.091123+0000","last_active":"2023-05-20T01:26:38.388177+0000","last_peered":"2023-05-20T01:26:38.388177+0000","last_clean":"2023-05-20T01:26:38.388177+0000","last_became_active":"2023-05-20T01:26:36.090841+0000","last_became_peered":"2023-05-20T01:26:36.090841+0000","last_unstale":"2023-05-20T01:26:38.388177+0000","last_undegraded":"2023-05-20T01:26:38.388177+0000","last_fullsized":"2023-05-20T01:26:38.388177+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:17:58.013423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"111'2","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.710624+0000","last_change":"2023-05-20T01:26:35.661288+0000","last_active":"2023-05-20T01:26:38.710624+0000","last_peered":"2023-05-20T01:26:38.710624+0000","last_clean":"2023-05-20T01:26:38.710624+0000","last_became_active":"2023-05-20T01:26:35.660976+0000","last_became_peered":"2023-05-20T01:26:35.660976+0000","last_unstale":"2023-05-20T01:26:38.710624+0000","last_undegraded":"2023-05-20T01:26:38.710624+0000","last_fullsized":"2023-05-20T01:26:38.710624+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:12:43.594752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":197,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:37.084799+0000","last_change":"2023-05-20T01:16:23.289068+0000","last_active":"2023-05-20T01:26:37.084799+0000","last_peered":"2023-05-20T01:26:37.084799+0000","last_clean":"2023-05-20T01:26:37.084799+0000","last_became_active":"2023-05-20T01:16:23.288781+0000","last_became_peered":"2023-05-20T01:16:23.288781+0000","last_unstale":"2023-05-20T01:26:37.084799+0000","last_undegraded":"2023-05-20T01:26:37.084799+0000","last_fullsized":"2023-05-20T01:26:37.084799+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:22:36.000137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.388316+0000","last_change":"2023-05-20T01:26:36.089408+0000","last_active":"2023-05-20T01:26:38.388316+0000","last_peered":"2023-05-20T01:26:38.388316+0000","last_clean":"2023-05-20T01:26:38.388316+0000","last_became_active":"2023-05-20T01:26:36.088124+0000","last_became_peered":"2023-05-20T01:26:36.088124+0000","last_unstale":"2023-05-20T01:26:38.388316+0000","last_undegraded":"2023-05-20T01:26:38.388316+0000","last_fullsized":"2023-05-20T01:26:38.388316+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:32:46.166136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.710489+0000","last_change":"2023-05-20T01:26:35.659858+0000","last_active":"2023-05-20T01:26:38.710489+0000","last_peered":"2023-05-20T01:26:38.710489+0000","last_clean":"2023-05-20T01:26:38.710489+0000","last_became_active":"2023-05-20T01:26:35.659432+0000","last_became_peered":"2023-05-20T01:26:35.659432+0000","last_unstale":"2023-05-20T01:26:38.710489+0000","last_undegraded":"2023-05-20T01:26:38.710489+0000","last_fullsized":"2023-05-20T01:26:38.710489+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:12:44.311239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":205,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:37.080044+0000","last_change":"2023-05-20T01:16:22.829324+0000","last_active":"2023-05-20T01:26:37.080044+0000","last_peered":"2023-05-20T01:26:37.080044+0000","last_clean":"2023-05-20T01:26:37.080044+0000","last_became_active":"2023-05-20T01:16:22.828964+0000","last_became_peered":"2023-05-20T01:16:22.828964+0000","last_unstale":"2023-05-20T01:26:37.080044+0000","last_undegraded":"2023-05-20T01:26:37.080044+0000","last_fullsized":"2023-05-20T01:26:37.080044+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:10:12.087621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:37.085468+0000","last_change":"2023-05-20T01:26:36.090093+0000","last_active":"2023-05-20T01:26:37.085468+0000","last_peered":"2023-05-20T01:26:37.085468+0000","last_clean":"2023-05-20T01:26:37.085468+0000","last_became_active":"2023-05-20T01:26:36.089787+0000","last_became_peered":"2023-05-20T01:26:36.089787+0000","last_unstale":"2023-05-20T01:26:37.085468+0000","last_undegraded":"2023-05-20T01:26:37.085468+0000","last_fullsized":"2023-05-20T01:26:37.085468+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:41:03.342972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"111'1","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.585424+0000","last_change":"2023-05-20T01:26:35.659399+0000","last_active":"2023-05-20T01:26:38.585424+0000","last_peered":"2023-05-20T01:26:38.585424+0000","last_clean":"2023-05-20T01:26:38.585424+0000","last_became_active":"2023-05-20T01:26:35.659081+0000","last_became_peered":"2023-05-20T01:26:35.659081+0000","last_unstale":"2023-05-20T01:26:38.585424+0000","last_undegraded":"2023-05-20T01:26:38.585424+0000","last_fullsized":"2023-05-20T01:26:38.585424+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:04:20.587342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":181,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.710416+0000","last_change":"2023-05-20T01:16:22.834754+0000","last_active":"2023-05-20T01:26:38.710416+0000","last_peered":"2023-05-20T01:26:38.710416+0000","last_clean":"2023-05-20T01:26:38.710416+0000","last_became_active":"2023-05-20T01:16:22.834437+0000","last_became_peered":"2023-05-20T01:16:22.834437+0000","last_unstale":"2023-05-20T01:26:38.710416+0000","last_undegraded":"2023-05-20T01:26:38.710416+0000","last_fullsized":"2023-05-20T01:26:38.710416+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:42:52.772764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":316,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.585048+0000","last_change":"2023-05-20T01:16:22.834920+0000","last_active":"2023-05-20T01:26:38.585048+0000","last_peered":"2023-05-20T01:26:38.585048+0000","last_clean":"2023-05-20T01:26:38.585048+0000","last_became_active":"2023-05-20T01:16:22.833924+0000","last_became_peered":"2023-05-20T01:16:22.833924+0000","last_unstale":"2023-05-20T01:26:38.585048+0000","last_undegraded":"2023-05-20T01:26:38.585048+0000","last_fullsized":"2023-05-20T01:26:38.585048+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:49:53.011248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.710724+0000","last_change":"2023-05-20T01:26:36.095844+0000","last_active":"2023-05-20T01:26:38.710724+0000","last_peered":"2023-05-20T01:26:38.710724+0000","last_clean":"2023-05-20T01:26:38.710724+0000","last_became_active":"2023-05-20T01:26:36.095578+0000","last_became_peered":"2023-05-20T01:26:36.095578+0000","last_unstale":"2023-05-20T01:26:38.710724+0000","last_undegraded":"2023-05-20T01:26:38.710724+0000","last_fullsized":"2023-05-20T01:26:38.710724+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:40:05.629844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.710414+0000","last_change":"2023-05-20T01:26:35.661736+0000","last_active":"2023-05-20T01:26:38.710414+0000","last_peered":"2023-05-20T01:26:38.710414+0000","last_clean":"2023-05-20T01:26:38.710414+0000","last_became_active":"2023-05-20T01:26:35.661426+0000","last_became_peered":"2023-05-20T01:26:35.661426+0000","last_unstale":"2023-05-20T01:26:38.710414+0000","last_undegraded":"2023-05-20T01:26:38.710414+0000","last_fullsized":"2023-05-20T01:26:38.710414+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:42:44.705803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":165,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.585229+0000","last_change":"2023-05-20T01:16:23.290259+0000","last_active":"2023-05-20T01:26:38.585229+0000","last_peered":"2023-05-20T01:26:38.585229+0000","last_clean":"2023-05-20T01:26:38.585229+0000","last_became_active":"2023-05-20T01:16:23.289911+0000","last_became_peered":"2023-05-20T01:16:23.289911+0000","last_unstale":"2023-05-20T01:26:38.585229+0000","last_undegraded":"2023-05-20T01:26:38.585229+0000","last_fullsized":"2023-05-20T01:26:38.585229+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:49:10.789562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.710366+0000","last_change":"2023-05-20T01:26:36.089519+0000","last_active":"2023-05-20T01:26:38.710366+0000","last_peered":"2023-05-20T01:26:38.710366+0000","last_clean":"2023-05-20T01:26:38.710366+0000","last_became_active":"2023-05-20T01:26:36.089108+0000","last_became_peered":"2023-05-20T01:26:36.089108+0000","last_unstale":"2023-05-20T01:26:38.710366+0000","last_undegraded":"2023-05-20T01:26:38.710366+0000","last_fullsized":"2023-05-20T01:26:38.710366+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:03:24.852613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"111'2","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.710758+0000","last_change":"2023-05-20T01:26:35.660034+0000","last_active":"2023-05-20T01:26:38.710758+0000","last_peered":"2023-05-20T01:26:38.710758+0000","last_clean":"2023-05-20T01:26:38.710758+0000","last_became_active":"2023-05-20T01:26:35.659396+0000","last_became_peered":"2023-05-20T01:26:35.659396+0000","last_unstale":"2023-05-20T01:26:38.710758+0000","last_undegraded":"2023-05-20T01:26:38.710758+0000","last_fullsized":"2023-05-20T01:26:38.710758+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:49:13.574029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":166,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.585265+0000","last_change":"2023-05-20T01:16:23.290207+0000","last_active":"2023-05-20T01:26:38.585265+0000","last_peered":"2023-05-20T01:26:38.585265+0000","last_clean":"2023-05-20T01:26:38.585265+0000","last_became_active":"2023-05-20T01:16:23.289928+0000","last_became_peered":"2023-05-20T01:16:23.289928+0000","last_unstale":"2023-05-20T01:26:38.585265+0000","last_undegraded":"2023-05-20T01:26:38.585265+0000","last_fullsized":"2023-05-20T01:26:38.585265+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:30:18.841320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.710991+0000","last_change":"2023-05-20T01:26:36.092735+0000","last_active":"2023-05-20T01:26:38.710991+0000","last_peered":"2023-05-20T01:26:38.710991+0000","last_clean":"2023-05-20T01:26:38.710991+0000","last_became_active":"2023-05-20T01:26:36.092299+0000","last_became_peered":"2023-05-20T01:26:36.092299+0000","last_unstale":"2023-05-20T01:26:38.710991+0000","last_undegraded":"2023-05-20T01:26:38.710991+0000","last_fullsized":"2023-05-20T01:26:38.710991+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:40:11.942170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"111'1","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:37.086008+0000","last_change":"2023-05-20T01:26:35.663015+0000","last_active":"2023-05-20T01:26:37.086008+0000","last_peered":"2023-05-20T01:26:37.086008+0000","last_clean":"2023-05-20T01:26:37.086008+0000","last_became_active":"2023-05-20T01:26:35.662730+0000","last_became_peered":"2023-05-20T01:26:35.662730+0000","last_unstale":"2023-05-20T01:26:37.086008+0000","last_undegraded":"2023-05-20T01:26:37.086008+0000","last_fullsized":"2023-05-20T01:26:37.086008+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:06:24.569009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":167,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.710042+0000","last_change":"2023-05-20T01:16:23.290538+0000","last_active":"2023-05-20T01:26:38.710042+0000","last_peered":"2023-05-20T01:26:38.710042+0000","last_clean":"2023-05-20T01:26:38.710042+0000","last_became_active":"2023-05-20T01:16:23.290230+0000","last_became_peered":"2023-05-20T01:16:23.290230+0000","last_unstale":"2023-05-20T01:26:38.710042+0000","last_undegraded":"2023-05-20T01:26:38.710042+0000","last_fullsized":"2023-05-20T01:26:38.710042+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:42:28.178103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.710890+0000","last_change":"2023-05-20T01:26:36.099640+0000","last_active":"2023-05-20T01:26:38.710890+0000","last_peered":"2023-05-20T01:26:38.710890+0000","last_clean":"2023-05-20T01:26:38.710890+0000","last_became_active":"2023-05-20T01:26:36.099373+0000","last_became_peered":"2023-05-20T01:26:36.099373+0000","last_unstale":"2023-05-20T01:26:38.710890+0000","last_undegraded":"2023-05-20T01:26:38.710890+0000","last_fullsized":"2023-05-20T01:26:38.710890+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:10:31.286889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.711081+0000","last_change":"2023-05-20T01:26:35.660641+0000","last_active":"2023-05-20T01:26:38.711081+0000","last_peered":"2023-05-20T01:26:38.711081+0000","last_clean":"2023-05-20T01:26:38.711081+0000","last_became_active":"2023-05-20T01:26:35.659463+0000","last_became_peered":"2023-05-20T01:26:35.659463+0000","last_unstale":"2023-05-20T01:26:38.711081+0000","last_undegraded":"2023-05-20T01:26:38.711081+0000","last_fullsized":"2023-05-20T01:26:38.711081+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:02:37.953812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":181,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:37.085836+0000","last_change":"2023-05-20T01:16:23.290341+0000","last_active":"2023-05-20T01:26:37.085836+0000","last_peered":"2023-05-20T01:26:37.085836+0000","last_clean":"2023-05-20T01:26:37.085836+0000","last_became_active":"2023-05-20T01:16:23.290053+0000","last_became_peered":"2023-05-20T01:16:23.290053+0000","last_unstale":"2023-05-20T01:26:37.085836+0000","last_undegraded":"2023-05-20T01:26:37.085836+0000","last_fullsized":"2023-05-20T01:26:37.085836+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:45:41.696310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:37.079710+0000","last_change":"2023-05-20T01:26:36.094364+0000","last_active":"2023-05-20T01:26:37.079710+0000","last_peered":"2023-05-20T01:26:37.079710+0000","last_clean":"2023-05-20T01:26:37.079710+0000","last_became_active":"2023-05-20T01:26:36.093797+0000","last_became_peered":"2023-05-20T01:26:36.093797+0000","last_unstale":"2023-05-20T01:26:37.079710+0000","last_undegraded":"2023-05-20T01:26:37.079710+0000","last_fullsized":"2023-05-20T01:26:37.079710+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:25:35.442465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:37.085298+0000","last_change":"2023-05-20T01:26:35.654858+0000","last_active":"2023-05-20T01:26:37.085298+0000","last_peered":"2023-05-20T01:26:37.085298+0000","last_clean":"2023-05-20T01:26:37.085298+0000","last_became_active":"2023-05-20T01:26:35.654547+0000","last_became_peered":"2023-05-20T01:26:35.654547+0000","last_unstale":"2023-05-20T01:26:37.085298+0000","last_undegraded":"2023-05-20T01:26:37.085298+0000","last_fullsized":"2023-05-20T01:26:37.085298+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:20:58.771107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":181,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:37.085991+0000","last_change":"2023-05-20T01:16:22.834330+0000","last_active":"2023-05-20T01:26:37.085991+0000","last_peered":"2023-05-20T01:26:37.085991+0000","last_clean":"2023-05-20T01:26:37.085991+0000","last_became_active":"2023-05-20T01:16:22.833946+0000","last_became_peered":"2023-05-20T01:16:22.833946+0000","last_unstale":"2023-05-20T01:26:37.085991+0000","last_undegraded":"2023-05-20T01:26:37.085991+0000","last_fullsized":"2023-05-20T01:26:37.085991+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:04:24.045534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574976,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":16604,"kb_used_data":1940,"kb_used_omap":0,"kb_used_meta":14656,"kb_avail":94355236,"statfs":{"total":96636764160,"available":96619761664,"internally_reserved":0,"allocated":1986560,"data_stored":1301348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":15007744},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:26:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.28000000000000003,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.748,"5min":0.80600000000000005,"15min":0.88200000000000001},"last":0.29399999999999998},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.52900000000000003,"15min":0.54800000000000004},"min":{"1min":0.38900000000000001,"5min":0.309,"15min":0.30599999999999999},"max":{"1min":0.71399999999999997,"5min":0.77100000000000002,"15min":0.90000000000000002},"last":0.44800000000000001}]},{"osd":1,"last update":"Sat May 20 01:26:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.53100000000000003,"15min":0.69899999999999995},"min":{"1min":0.41599999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.70699999999999996,"5min":0.95599999999999996,"15min":8.8879999999999999},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.54800000000000004,"15min":0.73099999999999998},"min":{"1min":0.36899999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.753,"5min":0.85399999999999998,"15min":8.875},"last":0.57099999999999995}]},{"osd":2,"last update":"Sat May 20 01:26:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.52200000000000002,"15min":0.68500000000000005},"min":{"1min":0.41099999999999998,"5min":0.315,"15min":0.315},"max":{"1min":0.75700000000000001,"5min":0.79200000000000004,"15min":8.9120000000000008},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.53100000000000003,"15min":0.68999999999999995},"min":{"1min":0.46200000000000002,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":8.8520000000000003},"last":0.41099999999999998}]},{"osd":3,"last update":"Sat May 20 01:26:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.51900000000000002,"15min":0.72999999999999998},"min":{"1min":0.441,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.65800000000000003,"5min":0.89900000000000002,"15min":10.884},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.56899999999999995,"15min":0.78200000000000003},"min":{"1min":0.39400000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.78000000000000003,"5min":0.88100000000000001,"15min":10.952},"last":0.52800000000000002}]},{"osd":4,"last update":"Sat May 20 01:26:38 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.502,"15min":0.51000000000000001},"min":{"1min":0.34999999999999998,"5min":0.313,"15min":0.313},"max":{"1min":0.77600000000000002,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.52700000000000002,"15min":0.53000000000000003},"min":{"1min":0.371,"5min":0.309,"15min":0.309},"max":{"1min":0.71399999999999997,"5min":0.82599999999999996,"15min":0.875},"last":0.432}]},{"osd":5,"last update":"Sat May 20 01:26:38 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.55100000000000005,"15min":0.54600000000000004},"min":{"1min":0.35199999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.72699999999999998,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.57099999999999995,"15min":0.57699999999999996},"min":{"1min":0.40400000000000003,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.83899999999999997,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.58799999999999997}]},{"osd":6,"last update":"Sat May 20 01:26:38 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.58999999999999997,"15min":0.59599999999999997},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.378},"max":{"1min":0.78800000000000003,"5min":1.028,"15min":1.028},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.56799999999999995,"15min":0.58299999999999996},"min":{"1min":0.42699999999999999,"5min":0.377,"15min":0.377},"max":{"1min":0.77100000000000002,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.55700000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574975,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":11300,"kb_used_data":1372,"kb_used_omap":0,"kb_used_meta":9920,"kb_avail":94360540,"statfs":{"total":96636764160,"available":96625192960,"internally_reserved":0,"allocated":1404928,"data_stored":712988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":10158080},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:25:42 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.44800000000000001,"15min":0.47699999999999998},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.26900000000000002},"max":{"1min":0.60999999999999999,"5min":0.72699999999999998,"15min":0.77200000000000002},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.47899999999999998,"15min":0.51700000000000002},"min":{"1min":0.32900000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.41099999999999998}]},{"osd":1,"last update":"Sat May 20 01:25:42 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.48699999999999999,"15min":0.51900000000000002},"min":{"1min":0.32800000000000001,"5min":0.29699999999999999,"15min":0.28000000000000003},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.83799999999999997},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.51700000000000002,"15min":0.52600000000000002},"min":{"1min":0.379,"5min":0.32600000000000001,"15min":0.318},"max":{"1min":0.66500000000000004,"5min":0.749,"15min":0.79200000000000004},"last":0.63400000000000001}]},{"osd":2,"last update":"Sat May 20 01:25:42 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.48399999999999999,"15min":0.503},"min":{"1min":0.35099999999999998,"5min":0.34399999999999997,"15min":0.33700000000000002},"max":{"1min":0.57099999999999995,"5min":0.72199999999999998,"15min":0.76600000000000001},"last":0.37},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.51600000000000001,"15min":0.54900000000000004},"min":{"1min":0.37,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.745,"5min":0.76200000000000001,"15min":0.76800000000000002},"last":0.54200000000000004}]},{"osd":3,"last update":"Sat May 20 01:25:42 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.52500000000000002,"15min":0.53600000000000003},"min":{"1min":0.378,"5min":0.311,"15min":0.311},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.52000000000000002,"15min":0.53900000000000003},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.72699999999999998,"5min":0.73199999999999998,"15min":0.79500000000000004},"last":0.52800000000000002}]},{"osd":4,"last update":"Sat May 20 01:25:42 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.53100000000000003,"15min":0.54400000000000004},"min":{"1min":0.34799999999999998,"5min":0.33300000000000002,"15min":0.32600000000000001},"max":{"1min":0.70699999999999996,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.54000000000000004,"15min":0.56299999999999994},"min":{"1min":0.40500000000000003,"5min":0.38100000000000001,"15min":0.32700000000000001},"max":{"1min":0.71799999999999997,"5min":0.77400000000000002,"15min":0.80100000000000005},"last":0.68200000000000005}]},{"osd":5,"last update":"Sat May 20 01:25:42 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.53400000000000003,"15min":0.54000000000000004},"min":{"1min":0.38,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.63600000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.56499999999999995,"15min":0.59799999999999998},"min":{"1min":0.372,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.64400000000000002,"5min":0.81599999999999995,"15min":0.89400000000000002},"last":0.61699999999999999}]},{"osd":7,"last update":"Sat May 20 01:25:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.57099999999999995,"15min":0.59399999999999997},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.71299999999999997,"5min":0.80200000000000005,"15min":0.92900000000000005},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.57599999999999996,"15min":0.60499999999999998},"min":{"1min":0.45100000000000001,"5min":0.379,"15min":0.379},"max":{"1min":0.66600000000000004,"5min":0.82799999999999996,"15min":0.85199999999999998},"last":0.56599999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574975,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":17496,"kb_used_data":1936,"kb_used_omap":0,"kb_used_meta":15552,"kb_avail":94354344,"statfs":{"total":96636764160,"available":96618848256,"internally_reserved":0,"allocated":1982464,"data_stored":1301222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":15925248},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:25:36 2023","interfaces":[{"interface":"back","average":{"1min":0.30599999999999999,"5min":0.28299999999999997,"15min":0.311},"min":{"1min":0.18099999999999999,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":0.48799999999999999,"5min":0.57099999999999995,"15min":1.194},"last":0.249},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.32400000000000001,"15min":0.31900000000000001},"min":{"1min":0.23899999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.75800000000000001},"last":0.33400000000000002}]},{"osd":2,"last update":"Sat May 20 01:25:36 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.307,"15min":0.31900000000000001},"min":{"1min":0.23100000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.66400000000000003},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.374,"15min":0.40400000000000003},"min":{"1min":0.313,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":1.3580000000000001},"last":0.372}]},{"osd":3,"last update":"Sat May 20 01:25:36 2023","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.35899999999999999,"15min":0.371},"min":{"1min":0.27400000000000002,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":1.619},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.35299999999999998,"15min":0.379},"min":{"1min":0.25800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.57699999999999996,"5min":0.60899999999999999,"15min":1.8400000000000001},"last":0.36399999999999999}]},{"osd":4,"last update":"Sat May 20 01:25:36 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.55600000000000005,"15min":0.57899999999999996},"min":{"1min":0.36799999999999999,"5min":0.34899999999999998,"15min":0.32200000000000001},"max":{"1min":0.76100000000000001,"5min":0.81699999999999995,"15min":1.2769999999999999},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.57099999999999995,"15min":0.58299999999999996},"min":{"1min":0.38,"5min":0.312,"15min":0.312},"max":{"1min":0.81599999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.44800000000000001}]},{"osd":5,"last update":"Sat May 20 01:25:36 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.58699999999999997,"15min":0.59099999999999997},"min":{"1min":0.46600000000000003,"5min":0.37,"15min":0.37},"max":{"1min":0.78500000000000003,"5min":0.81899999999999995,"15min":1.4510000000000001},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.58199999999999996,"15min":0.60299999999999998},"min":{"1min":0.39700000000000002,"5min":0.39100000000000001,"15min":0.378},"max":{"1min":0.76800000000000002,"5min":0.78800000000000003,"15min":1.409},"last":0.45900000000000002}]},{"osd":6,"last update":"Sat May 20 01:25:36 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.60299999999999998},"min":{"1min":0.39000000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.82299999999999995,"5min":0.876,"15min":1.575},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.58199999999999996,"15min":0.59399999999999997},"min":{"1min":0.45400000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":1.0600000000000001},"last":0.52700000000000002}]},{"osd":7,"last update":"Sat May 20 01:25:36 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.60399999999999998},"min":{"1min":0.41099999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":1.496},"last":0.435},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.63,"15min":0.64000000000000001},"min":{"1min":0.44500000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.5009999999999999,"5min":2.1709999999999998,"15min":2.1709999999999998},"last":0.51000000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574975,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":11356,"kb_used_data":1364,"kb_used_omap":0,"kb_used_meta":9984,"kb_avail":94360484,"statfs":{"total":96636764160,"available":96625135616,"internally_reserved":0,"allocated":1396736,"data_stored":711513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":10223616},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat May 20 01:26:36 2023","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.32700000000000001,"15min":0.32500000000000001},"min":{"1min":0.28699999999999998,"5min":0.17999999999999999,"15min":0.17999999999999999},"max":{"1min":0.90800000000000003,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.36299999999999999,"15min":0.35399999999999998},"min":{"1min":0.23400000000000001,"5min":0.20799999999999999,"15min":0.20399999999999999},"max":{"1min":2.7559999999999998,"5min":2.7559999999999998,"15min":2.7559999999999998},"last":0.38500000000000001}]},{"osd":2,"last update":"Sat May 20 01:26:36 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.35599999999999998,"15min":0.34200000000000003},"min":{"1min":0.252,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":3.3079999999999998,"5min":3.3079999999999998,"15min":3.3079999999999998},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.45000000000000001,"15min":0.40899999999999997},"min":{"1min":0.314,"5min":0.22500000000000001,"15min":0.216},"max":{"1min":5.7270000000000003,"5min":5.7270000000000003,"15min":5.7270000000000003},"last":0.29899999999999999}]},{"osd":3,"last update":"Sat May 20 01:26:36 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.39500000000000002,"15min":0.376},"min":{"1min":0.27100000000000002,"5min":0.22,"15min":0.22},"max":{"1min":0.82399999999999995,"5min":4.258,"15min":4.258},"last":0.437},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.40100000000000002,"15min":0.38},"min":{"1min":0.30199999999999999,"5min":0.245,"15min":0.245},"max":{"1min":2.714,"5min":2.714,"15min":2.714},"last":0.41999999999999998}]},{"osd":4,"last update":"Sat May 20 01:26:36 2023","interfaces":[{"interface":"back","average":{"1min":1.383,"5min":0.74099999999999999,"15min":0.61099999999999999},"min":{"1min":0.46100000000000002,"5min":0.372,"15min":0.34399999999999997},"max":{"1min":13.125,"5min":13.125,"15min":13.125},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.624,"15min":0.58699999999999997},"min":{"1min":0.50600000000000001,"5min":0.39200000000000002,"15min":0.36299999999999999},"max":{"1min":1.9930000000000001,"5min":1.9930000000000001,"15min":1.9930000000000001},"last":0.59899999999999998}]},{"osd":5,"last update":"Sat May 20 01:26:36 2023","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":0.64600000000000002,"15min":0.59999999999999998},"min":{"1min":0.49199999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":5.3620000000000001,"5min":5.3620000000000001,"15min":5.3620000000000001},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.68100000000000005,"15min":0.622},"min":{"1min":0.46899999999999997,"5min":0.374,"15min":0.374},"max":{"1min":5.7999999999999998,"5min":5.7999999999999998,"15min":5.7999999999999998},"last":0.67000000000000004}]},{"osd":6,"last update":"Sat May 20 01:26:36 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.73599999999999999,"15min":0.64100000000000001},"min":{"1min":0.45700000000000002,"5min":0.36899999999999999,"15min":0.35199999999999998},"max":{"1min":3.3460000000000001,"5min":10.042999999999999,"15min":10.042999999999999},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.63400000000000001,"15min":0.624},"min":{"1min":0.46400000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.94099999999999995,"5min":1.367,"15min":1.367},"last":0.78700000000000003}]},{"osd":7,"last update":"Sat May 20 01:26:36 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.65100000000000002,"15min":0.61299999999999999},"min":{"1min":0.51900000000000002,"5min":0.41099999999999998,"15min":0.39600000000000002},"max":{"1min":2.802,"5min":2.802,"15min":2.802},"last":0.62},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.76000000000000001,"15min":0.66800000000000004},"min":{"1min":0.55200000000000005,"5min":0.439,"15min":0.42099999999999999},"max":{"1min":1.948,"5min":10.085000000000001,"15min":10.085000000000001},"last":0.74399999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574975,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":11348,"kb_used_data":1356,"kb_used_omap":0,"kb_used_meta":9984,"kb_avail":94360492,"statfs":{"total":96636764160,"available":96625143808,"internally_reserved":0,"allocated":1388544,"data_stored":710854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":10223616},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.33700000000000002,"5min":0.28299999999999997,"15min":0.32300000000000001},"min":{"1min":0.224,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":0.50900000000000001,"5min":0.53400000000000003,"15min":1.038},"last":0.28899999999999998},{"interface":"front","average":{"1min":0.32300000000000001,"5min":0.29899999999999999,"15min":0.33900000000000002},"min":{"1min":0.21199999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":1.079},"last":0.312}]},{"osd":1,"last update":"Sat May 20 01:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.317,"15min":0.376},"min":{"1min":0.253,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.44500000000000001,"5min":0.61399999999999999,"15min":1.4950000000000001},"last":0.37},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.35899999999999999,"15min":0.41299999999999998},"min":{"1min":0.28399999999999997,"5min":0.249,"15min":0.249},"max":{"1min":0.60799999999999998,"5min":0.629,"15min":1.5189999999999999},"last":0.35699999999999998}]},{"osd":3,"last update":"Sat May 20 01:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.32900000000000001,"15min":0.378},"min":{"1min":0.246,"5min":0.218,"15min":0.216},"max":{"1min":0.57899999999999996,"5min":0.60599999999999998,"15min":1.2090000000000001},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.36899999999999999,"15min":0.42699999999999999},"min":{"1min":0.32400000000000001,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.59499999999999997,"5min":0.68100000000000005,"15min":1.5620000000000001},"last":0.443}]},{"osd":4,"last update":"Sat May 20 01:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.56799999999999995,"15min":0.58099999999999996},"min":{"1min":0.46500000000000002,"5min":0.35899999999999999,"15min":0.34899999999999998},"max":{"1min":1.107,"5min":1.107,"15min":1.319},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.59399999999999997,"15min":0.61299999999999999},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":1.4319999999999999},"last":0.56100000000000005}]},{"osd":5,"last update":"Sat May 20 01:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.56799999999999995,"15min":0.61699999999999999},"min":{"1min":0.443,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.82199999999999995,"5min":0.95599999999999996,"15min":1.758},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.58899999999999997,"15min":0.63500000000000001},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.004,"5min":1.004,"15min":1.7110000000000001},"last":0.54200000000000004}]},{"osd":6,"last update":"Sat May 20 01:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.59099999999999997,"15min":0.63200000000000001},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.069,"5min":1.069,"15min":1.665},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.58499999999999996,"15min":0.628},"min":{"1min":0.39900000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.88,"5min":0.93500000000000005,"15min":1.819},"last":0.63900000000000001}]},{"osd":7,"last update":"Sat May 20 01:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.59999999999999998,"15min":0.65200000000000002},"min":{"1min":0.39000000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":1.7310000000000001},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.60999999999999999,"15min":0.65100000000000002},"min":{"1min":0.47799999999999998,"5min":0.42399999999999999,"15min":0.42199999999999999},"max":{"1min":1.038,"5min":1.038,"15min":1.8460000000000001},"last":0.67400000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574975,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":15700,"kb_used_data":1356,"kb_used_omap":0,"kb_used_meta":14336,"kb_avail":94356140,"statfs":{"total":96636764160,"available":96620687360,"internally_reserved":0,"allocated":1388544,"data_stored":711402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":14680064},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:26:34 2023","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.316,"15min":0.31900000000000001},"min":{"1min":0.24099999999999999,"5min":0.20100000000000001,"15min":0.188},"max":{"1min":0.54100000000000004,"5min":0.65000000000000002,"15min":1.528},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.34200000000000003,"15min":0.40799999999999997},"min":{"1min":0.26500000000000001,"5min":0.20799999999999999,"15min":0.19500000000000001},"max":{"1min":0.54200000000000004,"5min":0.72399999999999998,"15min":3.1890000000000001},"last":0.42599999999999999}]},{"osd":1,"last update":"Sat May 20 01:26:34 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.32500000000000001,"15min":0.40000000000000002},"min":{"1min":0.22700000000000001,"5min":0.189,"15min":0.189},"max":{"1min":0.55900000000000005,"5min":0.73199999999999998,"15min":3.4260000000000002},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.437,"5min":0.372,"15min":0.36099999999999999},"min":{"1min":0.27700000000000002,"5min":0.23100000000000001,"15min":0.221},"max":{"1min":0.60999999999999999,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.59299999999999997}]},{"osd":2,"last update":"Sat May 20 01:26:34 2023","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.36399999999999999,"15min":0.38400000000000001},"min":{"1min":0.30399999999999999,"5min":0.24299999999999999,"15min":0.224},"max":{"1min":0.59399999999999997,"5min":0.67400000000000004,"15min":1.1619999999999999},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.40000000000000002,"15min":0.41299999999999998},"min":{"1min":0.26900000000000002,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.75,"5min":0.78500000000000003,"15min":1.413},"last":0.48699999999999999}]},{"osd":4,"last update":"Sat May 20 01:26:34 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.56999999999999995,"15min":0.58299999999999996},"min":{"1min":0.45800000000000002,"5min":0.38700000000000001,"15min":0.38200000000000001},"max":{"1min":0.77000000000000002,"5min":0.95399999999999996,"15min":1.0700000000000001},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.59199999999999997,"15min":0.66100000000000003},"min":{"1min":0.42399999999999999,"5min":0.40200000000000002,"15min":0.35899999999999999},"max":{"1min":0.88500000000000001,"5min":0.88600000000000001,"15min":3.4660000000000002},"last":1.339}]},{"osd":5,"last update":"Sat May 20 01:26:34 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.59099999999999997,"15min":0.66600000000000004},"min":{"1min":0.503,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":3.5379999999999998},"last":1.274},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.60799999999999998,"15min":0.626},"min":{"1min":0.51400000000000001,"5min":0.41999999999999998,"15min":0.374},"max":{"1min":0.91300000000000003,"5min":1.026,"15min":1.385},"last":0.64200000000000002}]},{"osd":6,"last update":"Sat May 20 01:26:34 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.60699999999999998,"15min":0.629},"min":{"1min":0.44400000000000001,"5min":0.41199999999999998,"15min":0.38100000000000001},"max":{"1min":0.873,"5min":0.99199999999999999,"15min":1.347},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.60899999999999999,"15min":0.626},"min":{"1min":0.438,"5min":0.374,"15min":0.374},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":1.107},"last":0.61299999999999999}]},{"osd":7,"last update":"Sat May 20 01:26:34 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.60999999999999999,"15min":0.63200000000000001},"min":{"1min":0.46700000000000003,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.92200000000000004,"5min":0.92900000000000005,"15min":1.1359999999999999},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.63700000000000001,"15min":0.70699999999999996},"min":{"1min":0.52300000000000002,"5min":0.436,"15min":0.436},"max":{"1min":0.92300000000000004,"5min":1.0069999999999999,"15min":3.5009999999999999},"last":1.3120000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574975,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":16792,"kb_used_data":1360,"kb_used_omap":0,"kb_used_meta":15424,"kb_avail":94355048,"statfs":{"total":96636764160,"available":96619569152,"internally_reserved":0,"allocated":1392640,"data_stored":710912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":15794176},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:26:36 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.497,"15min":0.48999999999999999},"min":{"1min":0.313,"5min":0.26900000000000002,"15min":0.247},"max":{"1min":0.71999999999999997,"5min":0.73699999999999999,"15min":0.75600000000000001},"last":0.313},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.52200000000000002,"15min":0.504},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.76900000000000002,"5min":1.6830000000000001,"15min":1.7989999999999999},"last":0.47299999999999998}]},{"osd":1,"last update":"Sat May 20 01:26:36 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.51700000000000002,"15min":0.52100000000000002},"min":{"1min":0.374,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.82799999999999996},"last":0.504},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.52800000000000002,"15min":0.53000000000000003},"min":{"1min":0.371,"5min":0.316,"15min":0.316},"max":{"1min":0.747,"5min":1.7190000000000001,"15min":2.0739999999999998},"last":0.371}]},{"osd":2,"last update":"Sat May 20 01:26:36 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.55000000000000004,"15min":0.51800000000000002},"min":{"1min":0.35799999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.71399999999999997,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.55000000000000004,"15min":0.55100000000000005},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.34000000000000002},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.94699999999999995}]},{"osd":3,"last update":"Sat May 20 01:26:36 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.53900000000000003,"15min":0.52700000000000002},"min":{"1min":0.39000000000000001,"5min":0.34899999999999998,"15min":0.34399999999999997},"max":{"1min":0.72199999999999998,"5min":0.81399999999999995,"15min":1.827},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.58599999999999997,"15min":0.63700000000000001},"min":{"1min":0.45800000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.78300000000000003,"5min":1.093,"15min":3.1749999999999998},"last":0.45800000000000002}]},{"osd":5,"last update":"Sat May 20 01:26:36 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.52000000000000002,"15min":0.49099999999999999},"min":{"1min":0.36899999999999999,"5min":0.308,"15min":0.29199999999999998},"max":{"1min":0.63100000000000001,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.499,"5min":0.53700000000000003,"15min":0.51400000000000001},"min":{"1min":0.34399999999999997,"5min":0.33400000000000002,"15min":0.29899999999999999},"max":{"1min":0.69399999999999995,"5min":0.80400000000000005,"15min":0.80500000000000005},"last":0.34399999999999997}]},{"osd":6,"last update":"Sat May 20 01:26:36 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.57999999999999996,"15min":0.55100000000000005},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.32400000000000001},"max":{"1min":0.64900000000000002,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.60299999999999998,"15min":0.59199999999999997},"min":{"1min":0.48099999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.81200000000000006,"5min":0.84699999999999998,"15min":0.85899999999999999},"last":0.48499999999999999}]},{"osd":7,"last update":"Sat May 20 01:26:36 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.57699999999999996,"15min":0.55800000000000005},"min":{"1min":0.436,"5min":0.36099999999999999,"15min":0.34200000000000003},"max":{"1min":0.70999999999999996,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.442},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.61499999999999999,"15min":0.59799999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.34000000000000002},"max":{"1min":0.69499999999999995,"5min":1.034,"15min":1.034},"last":0.38100000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574976,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":11276,"kb_used_data":1348,"kb_used_omap":0,"kb_used_meta":9920,"kb_avail":94360564,"statfs":{"total":96636764160,"available":96625217536,"internally_reserved":0,"allocated":1380352,"data_stored":709859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":10158080},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:26:40 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.47499999999999998,"15min":0.61299999999999999},"min":{"1min":0.33300000000000002,"5min":0.29999999999999999,"15min":0.23300000000000001},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":6.3929999999999998},"last":0.503},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.48899999999999999,"15min":0.63600000000000001},"min":{"1min":0.34799999999999998,"5min":0.315,"15min":0.26900000000000002},"max":{"1min":0.80900000000000005,"5min":0.83699999999999997,"15min":6.3339999999999996},"last":0.59799999999999998}]},{"osd":1,"last update":"Sat May 20 01:26:40 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.48999999999999999,"15min":0.56499999999999995},"min":{"1min":0.36099999999999999,"5min":0.313,"15min":0.29299999999999998},"max":{"1min":0.68000000000000005,"5min":0.72299999999999998,"15min":3.2450000000000001},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.52500000000000002,"15min":0.53400000000000003},"min":{"1min":0.33200000000000002,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":1.1819999999999999},"last":0.45500000000000002}]},{"osd":2,"last update":"Sat May 20 01:26:40 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.51000000000000001,"15min":0.53600000000000003},"min":{"1min":0.40000000000000002,"5min":0.33200000000000002,"15min":0.30399999999999999},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":1.1639999999999999},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.51700000000000002,"15min":0.54200000000000004},"min":{"1min":0.42499999999999999,"5min":0.308,"15min":0.29399999999999998},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":1.258},"last":0.57899999999999996}]},{"osd":3,"last update":"Sat May 20 01:26:40 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.54700000000000004,"15min":0.68200000000000005},"min":{"1min":0.378,"5min":0.35099999999999998,"15min":0.33000000000000002},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":6.351},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.56200000000000006,"15min":0.70999999999999996},"min":{"1min":0.42899999999999999,"5min":0.374,"15min":0.33400000000000002},"max":{"1min":0.745,"5min":0.79000000000000004,"15min":6.3730000000000002},"last":0.55700000000000005}]},{"osd":4,"last update":"Sat May 20 01:26:40 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.52100000000000002,"15min":0.497},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.55400000000000005,"15min":0.55100000000000005},"min":{"1min":0.35699999999999998,"5min":0.35399999999999998,"15min":0.34200000000000003},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.64700000000000002}]},{"osd":6,"last update":"Sat May 20 01:26:40 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.29999999999999999},"max":{"1min":0.77100000000000002,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.623},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.38700000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.71699999999999997}]},{"osd":7,"last update":"Sat May 20 01:26:40 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58299999999999996,"15min":0.58099999999999996},"min":{"1min":0.41599999999999998,"5min":0.33400000000000002,"15min":0.33000000000000002},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.91000000000000003},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.60099999999999998,"15min":0.60699999999999998},"min":{"1min":0.41499999999999998,"5min":0.40999999999999998,"15min":0.39500000000000002},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.94499999999999995},"last":0.65900000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-05-20T01:26:44.885 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-05-20T01:26:45.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.022+0000 7efbfac5c700 1 Processor -- start 2023-05-20T01:26:45.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.022+0000 7efbfac5c700 1 -- start start 2023-05-20T01:26:45.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.023+0000 7efbfac5c700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7efbf40b4750 0x7efbf40d2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:45.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.023+0000 7efbfac5c700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efbf40b3b50 con 0x7efbf40cd970 2023-05-20T01:26:45.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.023+0000 7efbfac5c700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efbf40d0b90 con 0x7efbf40cfc50 2023-05-20T01:26:45.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.023+0000 7efbfac5c700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efbf40cfad0 con 0x7efbf40b4750 2023-05-20T01:26:45.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.023+0000 7efbf91f9700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7efbf40cd970 0x7efbf40cdd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49748/0 (socket says 172.21.15.16:49748) 2023-05-20T01:26:45.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.023+0000 7efbf91f9700 1 -- 172.21.15.16:0/3109700332 learned_addr learned my addr 172.21.15.16:0/3109700332 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:26:45.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.023+0000 7efbf3fff700 1 --2- 172.21.15.16:0/3109700332 >> v2:172.21.15.132:3300/0 conn(0x7efbf40b4750 0x7efbf40d2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:45.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.023+0000 7efbf37fe700 1 -- 172.21.15.16:0/3109700332 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3406603223 0 0) 0x7efbf40b3b50 con 0x7efbf40cd970 2023-05-20T01:26:45.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.023+0000 7efbf37fe700 1 -- 172.21.15.16:0/3109700332 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efbe40035b0 con 0x7efbf40cd970 2023-05-20T01:26:45.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.023+0000 7efbf37fe700 1 -- 172.21.15.16:0/3109700332 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2369320484 0 0) 0x7efbe40035b0 con 0x7efbf40cd970 2023-05-20T01:26:45.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.023+0000 7efbf37fe700 1 -- 172.21.15.16:0/3109700332 >> v2:172.21.15.132:3300/0 conn(0x7efbf40b4750 msgr2=0x7efbf40d2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:45.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.023+0000 7efbf37fe700 1 --2- 172.21.15.16:0/3109700332 >> v2:172.21.15.132:3300/0 conn(0x7efbf40b4750 0x7efbf40d2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:45.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.023+0000 7efbf37fe700 1 -- 172.21.15.16:0/3109700332 >> v1:172.21.15.132:6789/0 conn(0x7efbf40cfc50 legacy=0x7efbf40d0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:45.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.023+0000 7efbf37fe700 1 -- 172.21.15.16:0/3109700332 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efbf415d0b0 con 0x7efbf40cd970 2023-05-20T01:26:45.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.023+0000 7efbf37fe700 1 -- 172.21.15.16:0/3109700332 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7efbec0019f0 con 0x7efbf40cd970 2023-05-20T01:26:45.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.023+0000 7efbf3fff700 1 --2- 172.21.15.16:0/3109700332 >> v2:172.21.15.132:3300/0 conn(0x7efbf40b4750 0x7efbf40d2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-05-20T01:26:45.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.023+0000 7efbf37fe700 1 -- 172.21.15.16:0/3109700332 >> v1:172.21.15.16:6789/0 conn(0x7efbf40cd970 legacy=0x7efbf40cdd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:45.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.023+0000 7efbf37fe700 1 --2- 172.21.15.16:0/3109700332 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efbe4003a70 0x7efbe4003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:45.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.024+0000 7efbf37fe700 1 --2- 172.21.15.16:0/3109700332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efbe4004850 0x7efbe4004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:45.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.024+0000 7efbf37fe700 1 --2- 172.21.15.16:0/3109700332 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efbe4005250 0x7efbe400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:45.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.024+0000 7efbf37fe700 1 -- 172.21.15.16:0/3109700332 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efbf40cfad0 con 0x7efbe4004850 2023-05-20T01:26:45.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.024+0000 7efbf37fe700 1 -- 172.21.15.16:0/3109700332 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efbe400c190 con 0x7efbe4005250 2023-05-20T01:26:45.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.024+0000 7efbf37fe700 1 -- 172.21.15.16:0/3109700332 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efbe400c350 con 0x7efbe4003a70 2023-05-20T01:26:45.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.024+0000 7efbf3fff700 1 --2- 172.21.15.16:0/3109700332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efbe4004850 0x7efbe4004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:45.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.024+0000 7efbf91f9700 1 --2- 172.21.15.16:0/3109700332 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efbe4005250 0x7efbe400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:45.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.024+0000 7efbf89f8700 1 --2- 172.21.15.16:0/3109700332 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efbe4003a70 0x7efbe4003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:45.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.024+0000 7efbf3fff700 1 -- 172.21.15.16:0/3109700332 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efbe4003a70 msgr2=0x7efbe4003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:45.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.024+0000 7efbf3fff700 1 --2- 172.21.15.16:0/3109700332 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efbe4003a70 0x7efbe4003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:45.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.024+0000 7efbf3fff700 1 -- 172.21.15.16:0/3109700332 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efbe4005250 msgr2=0x7efbe400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:45.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.024+0000 7efbf3fff700 1 --2- 172.21.15.16:0/3109700332 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efbe4005250 0x7efbe400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:45.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.024+0000 7efbf3fff700 1 -- 172.21.15.16:0/3109700332 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efbe40035b0 con 0x7efbe4004850 2023-05-20T01:26:45.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.024+0000 7efbf91f9700 1 --2- 172.21.15.16:0/3109700332 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efbe4005250 0x7efbe400ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:26:45.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.024+0000 7efbf91f9700 1 -- 172.21.15.16:0/3109700332 reap_dead start 2023-05-20T01:26:45.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.024+0000 7efbf3fff700 1 --2- 172.21.15.16:0/3109700332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efbe4004850 0x7efbe4004cc0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7efbe0009d00 tx=0x7efbe4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0f0023996876784 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:45.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.024+0000 7efbf37fe700 1 -- 172.21.15.16:0/3109700332 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbe0009e50 con 0x7efbe4004850 2023-05-20T01:26:45.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.024+0000 7efbf37fe700 1 -- 172.21.15.16:0/3109700332 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efbe000e9a0 con 0x7efbe4004850 2023-05-20T01:26:45.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.024+0000 7efbf37fe700 1 -- 172.21.15.16:0/3109700332 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbe000ed30 con 0x7efbe4004850 2023-05-20T01:26:45.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.025+0000 7efbfac5c700 1 -- 172.21.15.16:0/3109700332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efbe4004850 msgr2=0x7efbe4004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:45.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.025+0000 7efbfac5c700 1 --2- 172.21.15.16:0/3109700332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efbe4004850 0x7efbe4004cc0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7efbe0009d00 tx=0x7efbe4005880 comp rx=0 tx=0).stop 2023-05-20T01:26:45.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.025+0000 7efbfac5c700 1 -- 172.21.15.16:0/3109700332 shutdown_connections 2023-05-20T01:26:45.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.025+0000 7efbfac5c700 1 --2- 172.21.15.16:0/3109700332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efbe4004850 0x7efbe4004cc0 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:45.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.025+0000 7efbfac5c700 1 -- 172.21.15.16:0/3109700332 >> 172.21.15.16:0/3109700332 conn(0x7efbf409c590 msgr2=0x7efbf40d3240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:45.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.025+0000 7efbfac5c700 1 -- 172.21.15.16:0/3109700332 shutdown_connections 2023-05-20T01:26:45.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.025+0000 7efbfac5c700 1 -- 172.21.15.16:0/3109700332 wait complete. 2023-05-20T01:26:45.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.026+0000 7efbfac5c700 1 Processor -- start 2023-05-20T01:26:45.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.026+0000 7efbfac5c700 1 -- start start 2023-05-20T01:26:45.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.027+0000 7efbfac5c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efbe4004850 0x7efbf41f5a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:45.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.027+0000 7efbfac5c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efbf415dbc0 0x7efbf415f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:45.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.027+0000 7efbfac5c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efbf415e030 0x7efbf415fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:45.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.027+0000 7efbfac5c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efbf4153d80 con 0x7efbf415dbc0 2023-05-20T01:26:45.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.027+0000 7efbfac5c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efbf41601c0 con 0x7efbe4004850 2023-05-20T01:26:45.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.027+0000 7efbfac5c700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efbf40b8a10 con 0x7efbf415e030 2023-05-20T01:26:45.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.027+0000 7efbf3fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efbf415dbc0 0x7efbf415f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:45.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.027+0000 7efbf3fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efbf415dbc0 0x7efbf415f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38230/0 (socket says 172.21.15.16:38230) 2023-05-20T01:26:45.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.027+0000 7efbf3fff700 1 -- 172.21.15.16:0/611266531 learned_addr learned my addr 172.21.15.16:0/611266531 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:26:45.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.028+0000 7efbf91f9700 1 --2- 172.21.15.16:0/611266531 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efbf415e030 0x7efbf415fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:45.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.028+0000 7efbf3fff700 1 -- 172.21.15.16:0/611266531 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efbf415e030 msgr2=0x7efbf415fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:45.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.028+0000 7efbf89f8700 1 --2- 172.21.15.16:0/611266531 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efbe4004850 0x7efbf41f5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:45.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.028+0000 7efbf3fff700 1 --2- 172.21.15.16:0/611266531 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efbf415e030 0x7efbf415fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:45.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.028+0000 7efbf3fff700 1 -- 172.21.15.16:0/611266531 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efbe4004850 msgr2=0x7efbf41f5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:45.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.028+0000 7efbf3fff700 1 --2- 172.21.15.16:0/611266531 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efbe4004850 0x7efbf41f5a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:45.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.028+0000 7efbf3fff700 1 -- 172.21.15.16:0/611266531 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efbf40d4000 con 0x7efbf415dbc0 2023-05-20T01:26:45.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.028+0000 7efbf3fff700 1 --2- 172.21.15.16:0/611266531 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efbf415dbc0 0x7efbf415f740 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7efbe400c160 tx=0x7efbe00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:45.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.028+0000 7efbf17fa700 1 -- 172.21.15.16:0/611266531 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbe0003ab0 con 0x7efbf415dbc0 2023-05-20T01:26:45.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.028+0000 7efbf17fa700 1 -- 172.21.15.16:0/611266531 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efbe0003c50 con 0x7efbf415dbc0 2023-05-20T01:26:45.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.028+0000 7efbfac5c700 1 -- 172.21.15.16:0/611266531 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efbf40b4380 con 0x7efbf415dbc0 2023-05-20T01:26:45.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.028+0000 7efbf17fa700 1 -- 172.21.15.16:0/611266531 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbe0009e50 con 0x7efbf415dbc0 2023-05-20T01:26:45.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.028+0000 7efbfac5c700 1 -- 172.21.15.16:0/611266531 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efbf40b3da0 con 0x7efbf415dbc0 2023-05-20T01:26:45.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.029+0000 7efbfac5c700 1 -- 172.21.15.16:0/611266531 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efbf40747b0 con 0x7efbf415dbc0 2023-05-20T01:26:45.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.030+0000 7efbf17fa700 1 -- 172.21.15.16:0/611266531 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7efbe0013470 con 0x7efbf415dbc0 2023-05-20T01:26:45.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.030+0000 7efbf17fa700 1 --2- 172.21.15.16:0/611266531 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7efbd806ace0 0x7efbd806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:45.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.030+0000 7efbf17fa700 1 -- 172.21.15.16:0/611266531 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6382+0+0 (secure 0 0 0) 0x7efbe008aee0 con 0x7efbf415dbc0 2023-05-20T01:26:45.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.030+0000 7efbf89f8700 1 --2- 172.21.15.16:0/611266531 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7efbd806ace0 0x7efbd806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:45.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.031+0000 7efbf89f8700 1 --2- 172.21.15.16:0/611266531 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7efbd806ace0 0x7efbd806d190 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7efbe80016d0 tx=0x7efbe8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:45.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.035+0000 7efbf17fa700 1 -- 172.21.15.16:0/611266531 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7efbe0055ee0 con 0x7efbf415dbc0 2023-05-20T01:26:45.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.228+0000 7efbfac5c700 1 -- 172.21.15.16:0/611266531 --> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7efbf40b90a0 con 0x7efbd806ace0 2023-05-20T01:26:45.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.237+0000 7efbf17fa700 1 -- 172.21.15.16:0/611266531 <== mgr.4104 v2:172.21.15.16:6832/3817237998 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311986 (secure 0 0 0) 0x7efbf40b90a0 con 0x7efbd806ace0 2023-05-20T01:26:45.245 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:26:45.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.240+0000 7efbfac5c700 1 -- 172.21.15.16:0/611266531 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7efbd806ace0 msgr2=0x7efbd806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:45.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.240+0000 7efbfac5c700 1 --2- 172.21.15.16:0/611266531 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7efbd806ace0 0x7efbd806d190 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7efbe80016d0 tx=0x7efbe8009040 comp rx=0 tx=0).stop 2023-05-20T01:26:45.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.240+0000 7efbfac5c700 1 -- 172.21.15.16:0/611266531 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efbf415dbc0 msgr2=0x7efbf415f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:45.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.240+0000 7efbfac5c700 1 --2- 172.21.15.16:0/611266531 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efbf415dbc0 0x7efbf415f740 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7efbe400c160 tx=0x7efbe00047c0 comp rx=0 tx=0).stop 2023-05-20T01:26:45.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.240+0000 7efbfac5c700 1 -- 172.21.15.16:0/611266531 shutdown_connections 2023-05-20T01:26:45.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.240+0000 7efbfac5c700 1 --2- 172.21.15.16:0/611266531 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7efbd806ace0 0x7efbd806d190 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:45.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.240+0000 7efbfac5c700 1 --2- 172.21.15.16:0/611266531 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efbf415e030 0x7efbf415fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:45.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.241+0000 7efbfac5c700 1 --2- 172.21.15.16:0/611266531 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efbe4004850 0x7efbf41f5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:45.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.241+0000 7efbfac5c700 1 --2- 172.21.15.16:0/611266531 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efbf415dbc0 0x7efbf415f740 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:45.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.241+0000 7efbfac5c700 1 -- 172.21.15.16:0/611266531 >> 172.21.15.16:0/611266531 conn(0x7efbf409c590 msgr2=0x7efbf415c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:45.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.241+0000 7efbfac5c700 1 -- 172.21.15.16:0/611266531 shutdown_connections 2023-05-20T01:26:45.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.241+0000 7efbfac5c700 1 -- 172.21.15.16:0/611266531 wait complete. 2023-05-20T01:26:45.248 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2023-05-20T01:26:45.257 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":417,"stamp":"2023-05-20T01:26:43.455725+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":197,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":111872,"kb_used_data":12032,"kb_used_omap":0,"kb_used_meta":99776,"kb_avail":754862848,"statfs":{"total":773094113280,"available":772979556352,"internally_reserved":0,"allocated":12320768,"data_stored":6870098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":102170624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.814825"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.388334+0000","last_change":"2023-05-20T01:26:36.087663+0000","last_active":"2023-05-20T01:26:38.388334+0000","last_peered":"2023-05-20T01:26:38.388334+0000","last_clean":"2023-05-20T01:26:38.388334+0000","last_became_active":"2023-05-20T01:26:36.087500+0000","last_became_peered":"2023-05-20T01:26:36.087500+0000","last_unstale":"2023-05-20T01:26:38.388334+0000","last_undegraded":"2023-05-20T01:26:38.388334+0000","last_fullsized":"2023-05-20T01:26:38.388334+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:55:04.702228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:37.086071+0000","last_change":"2023-05-20T01:26:36.093490+0000","last_active":"2023-05-20T01:26:37.086071+0000","last_peered":"2023-05-20T01:26:37.086071+0000","last_clean":"2023-05-20T01:26:37.086071+0000","last_became_active":"2023-05-20T01:26:36.093189+0000","last_became_peered":"2023-05-20T01:26:36.093189+0000","last_unstale":"2023-05-20T01:26:37.086071+0000","last_undegraded":"2023-05-20T01:26:37.086071+0000","last_fullsized":"2023-05-20T01:26:37.086071+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:17:45.202356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.388054+0000","last_change":"2023-05-20T01:26:36.089906+0000","last_active":"2023-05-20T01:26:38.388054+0000","last_peered":"2023-05-20T01:26:38.388054+0000","last_clean":"2023-05-20T01:26:38.388054+0000","last_became_active":"2023-05-20T01:26:36.089609+0000","last_became_peered":"2023-05-20T01:26:36.089609+0000","last_unstale":"2023-05-20T01:26:38.388054+0000","last_undegraded":"2023-05-20T01:26:38.388054+0000","last_fullsized":"2023-05-20T01:26:38.388054+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:29:19.826889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:37.079825+0000","last_change":"2023-05-20T01:26:36.093090+0000","last_active":"2023-05-20T01:26:37.079825+0000","last_peered":"2023-05-20T01:26:37.079825+0000","last_clean":"2023-05-20T01:26:37.079825+0000","last_became_active":"2023-05-20T01:26:36.092763+0000","last_became_peered":"2023-05-20T01:26:36.092763+0000","last_unstale":"2023-05-20T01:26:37.079825+0000","last_undegraded":"2023-05-20T01:26:37.079825+0000","last_fullsized":"2023-05-20T01:26:37.079825+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:43:17.762338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.710862+0000","last_change":"2023-05-20T01:26:36.093707+0000","last_active":"2023-05-20T01:26:38.710862+0000","last_peered":"2023-05-20T01:26:38.710862+0000","last_clean":"2023-05-20T01:26:38.710862+0000","last_became_active":"2023-05-20T01:26:36.093448+0000","last_became_peered":"2023-05-20T01:26:36.093448+0000","last_unstale":"2023-05-20T01:26:38.710862+0000","last_undegraded":"2023-05-20T01:26:38.710862+0000","last_fullsized":"2023-05-20T01:26:38.710862+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:41:14.914426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:37.085434+0000","last_change":"2023-05-20T01:26:36.086895+0000","last_active":"2023-05-20T01:26:37.085434+0000","last_peered":"2023-05-20T01:26:37.085434+0000","last_clean":"2023-05-20T01:26:37.085434+0000","last_became_active":"2023-05-20T01:26:36.086612+0000","last_became_peered":"2023-05-20T01:26:36.086612+0000","last_unstale":"2023-05-20T01:26:37.085434+0000","last_undegraded":"2023-05-20T01:26:37.085434+0000","last_fullsized":"2023-05-20T01:26:37.085434+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:21:15.302621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:37.085177+0000","last_change":"2023-05-20T01:26:36.085521+0000","last_active":"2023-05-20T01:26:37.085177+0000","last_peered":"2023-05-20T01:26:37.085177+0000","last_clean":"2023-05-20T01:26:37.085177+0000","last_became_active":"2023-05-20T01:26:36.085179+0000","last_became_peered":"2023-05-20T01:26:36.085179+0000","last_unstale":"2023-05-20T01:26:37.085177+0000","last_undegraded":"2023-05-20T01:26:37.085177+0000","last_fullsized":"2023-05-20T01:26:37.085177+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:20:28.171785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.710990+0000","last_change":"2023-05-20T01:26:36.092594+0000","last_active":"2023-05-20T01:26:38.710990+0000","last_peered":"2023-05-20T01:26:38.710990+0000","last_clean":"2023-05-20T01:26:38.710990+0000","last_became_active":"2023-05-20T01:26:36.091906+0000","last_became_peered":"2023-05-20T01:26:36.091906+0000","last_unstale":"2023-05-20T01:26:38.710990+0000","last_undegraded":"2023-05-20T01:26:38.710990+0000","last_fullsized":"2023-05-20T01:26:38.710990+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:39:33.187319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:37.085906+0000","last_change":"2023-05-20T01:26:36.094841+0000","last_active":"2023-05-20T01:26:37.085906+0000","last_peered":"2023-05-20T01:26:37.085906+0000","last_clean":"2023-05-20T01:26:37.085906+0000","last_became_active":"2023-05-20T01:26:36.094546+0000","last_became_peered":"2023-05-20T01:26:36.094546+0000","last_unstale":"2023-05-20T01:26:37.085906+0000","last_undegraded":"2023-05-20T01:26:37.085906+0000","last_fullsized":"2023-05-20T01:26:37.085906+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:29:43.382263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.711007+0000","last_change":"2023-05-20T01:26:36.089291+0000","last_active":"2023-05-20T01:26:38.711007+0000","last_peered":"2023-05-20T01:26:38.711007+0000","last_clean":"2023-05-20T01:26:38.711007+0000","last_became_active":"2023-05-20T01:26:36.088987+0000","last_became_peered":"2023-05-20T01:26:36.088987+0000","last_unstale":"2023-05-20T01:26:38.711007+0000","last_undegraded":"2023-05-20T01:26:38.711007+0000","last_fullsized":"2023-05-20T01:26:38.711007+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:59:28.429861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.710513+0000","last_change":"2023-05-20T01:26:36.088457+0000","last_active":"2023-05-20T01:26:38.710513+0000","last_peered":"2023-05-20T01:26:38.710513+0000","last_clean":"2023-05-20T01:26:38.710513+0000","last_became_active":"2023-05-20T01:26:36.087957+0000","last_became_peered":"2023-05-20T01:26:36.087957+0000","last_unstale":"2023-05-20T01:26:38.710513+0000","last_undegraded":"2023-05-20T01:26:38.710513+0000","last_fullsized":"2023-05-20T01:26:38.710513+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T01:29:59.564067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:37.084652+0000","last_change":"2023-05-20T01:26:36.092100+0000","last_active":"2023-05-20T01:26:37.084652+0000","last_peered":"2023-05-20T01:26:37.084652+0000","last_clean":"2023-05-20T01:26:37.084652+0000","last_became_active":"2023-05-20T01:26:36.091816+0000","last_became_peered":"2023-05-20T01:26:36.091816+0000","last_unstale":"2023-05-20T01:26:37.084652+0000","last_undegraded":"2023-05-20T01:26:37.084652+0000","last_fullsized":"2023-05-20T01:26:37.084652+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:12:49.577601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:37.079617+0000","last_change":"2023-05-20T01:26:36.082387+0000","last_active":"2023-05-20T01:26:37.079617+0000","last_peered":"2023-05-20T01:26:37.079617+0000","last_clean":"2023-05-20T01:26:37.079617+0000","last_became_active":"2023-05-20T01:26:36.082097+0000","last_became_peered":"2023-05-20T01:26:36.082097+0000","last_unstale":"2023-05-20T01:26:37.079617+0000","last_undegraded":"2023-05-20T01:26:37.079617+0000","last_fullsized":"2023-05-20T01:26:37.079617+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:04:11.590675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.585216+0000","last_change":"2023-05-20T01:26:36.094589+0000","last_active":"2023-05-20T01:26:38.585216+0000","last_peered":"2023-05-20T01:26:38.585216+0000","last_clean":"2023-05-20T01:26:38.585216+0000","last_became_active":"2023-05-20T01:26:36.094326+0000","last_became_peered":"2023-05-20T01:26:36.094326+0000","last_unstale":"2023-05-20T01:26:38.585216+0000","last_undegraded":"2023-05-20T01:26:38.585216+0000","last_fullsized":"2023-05-20T01:26:38.585216+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:53:33.968042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.585244+0000","last_change":"2023-05-20T01:26:36.092887+0000","last_active":"2023-05-20T01:26:38.585244+0000","last_peered":"2023-05-20T01:26:38.585244+0000","last_clean":"2023-05-20T01:26:38.585244+0000","last_became_active":"2023-05-20T01:26:36.092572+0000","last_became_peered":"2023-05-20T01:26:36.092572+0000","last_unstale":"2023-05-20T01:26:38.585244+0000","last_undegraded":"2023-05-20T01:26:38.585244+0000","last_fullsized":"2023-05-20T01:26:38.585244+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:22:28.115116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.585139+0000","last_change":"2023-05-20T01:26:36.093210+0000","last_active":"2023-05-20T01:26:38.585139+0000","last_peered":"2023-05-20T01:26:38.585139+0000","last_clean":"2023-05-20T01:26:38.585139+0000","last_became_active":"2023-05-20T01:26:36.092935+0000","last_became_peered":"2023-05-20T01:26:36.092935+0000","last_unstale":"2023-05-20T01:26:38.585139+0000","last_undegraded":"2023-05-20T01:26:38.585139+0000","last_fullsized":"2023-05-20T01:26:38.585139+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T01:43:14.591660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.388357+0000","last_change":"2023-05-20T01:26:36.090126+0000","last_active":"2023-05-20T01:26:38.388357+0000","last_peered":"2023-05-20T01:26:38.388357+0000","last_clean":"2023-05-20T01:26:38.388357+0000","last_became_active":"2023-05-20T01:26:36.089858+0000","last_became_peered":"2023-05-20T01:26:36.089858+0000","last_unstale":"2023-05-20T01:26:38.388357+0000","last_undegraded":"2023-05-20T01:26:38.388357+0000","last_fullsized":"2023-05-20T01:26:38.388357+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:16:06.854941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:37.085974+0000","last_change":"2023-05-20T01:26:36.085121+0000","last_active":"2023-05-20T01:26:37.085974+0000","last_peered":"2023-05-20T01:26:37.085974+0000","last_clean":"2023-05-20T01:26:37.085974+0000","last_became_active":"2023-05-20T01:26:36.084610+0000","last_became_peered":"2023-05-20T01:26:36.084610+0000","last_unstale":"2023-05-20T01:26:37.085974+0000","last_undegraded":"2023-05-20T01:26:37.085974+0000","last_fullsized":"2023-05-20T01:26:37.085974+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:55:47.951254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.388149+0000","last_change":"2023-05-20T01:26:36.094178+0000","last_active":"2023-05-20T01:26:38.388149+0000","last_peered":"2023-05-20T01:26:38.388149+0000","last_clean":"2023-05-20T01:26:38.388149+0000","last_became_active":"2023-05-20T01:26:36.093750+0000","last_became_peered":"2023-05-20T01:26:36.093750+0000","last_unstale":"2023-05-20T01:26:38.388149+0000","last_undegraded":"2023-05-20T01:26:38.388149+0000","last_fullsized":"2023-05-20T01:26:38.388149+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:45:42.406317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:37.084486+0000","last_change":"2023-05-20T01:26:36.095210+0000","last_active":"2023-05-20T01:26:37.084486+0000","last_peered":"2023-05-20T01:26:37.084486+0000","last_clean":"2023-05-20T01:26:37.084486+0000","last_became_active":"2023-05-20T01:26:36.094887+0000","last_became_peered":"2023-05-20T01:26:36.094887+0000","last_unstale":"2023-05-20T01:26:37.084486+0000","last_undegraded":"2023-05-20T01:26:37.084486+0000","last_fullsized":"2023-05-20T01:26:37.084486+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:04:53.469731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.710431+0000","last_change":"2023-05-20T01:26:36.097965+0000","last_active":"2023-05-20T01:26:38.710431+0000","last_peered":"2023-05-20T01:26:38.710431+0000","last_clean":"2023-05-20T01:26:38.710431+0000","last_became_active":"2023-05-20T01:26:36.097743+0000","last_became_peered":"2023-05-20T01:26:36.097743+0000","last_unstale":"2023-05-20T01:26:38.710431+0000","last_undegraded":"2023-05-20T01:26:38.710431+0000","last_fullsized":"2023-05-20T01:26:38.710431+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:33:01.953967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.710901+0000","last_change":"2023-05-20T01:26:36.093120+0000","last_active":"2023-05-20T01:26:38.710901+0000","last_peered":"2023-05-20T01:26:38.710901+0000","last_clean":"2023-05-20T01:26:38.710901+0000","last_became_active":"2023-05-20T01:26:36.092842+0000","last_became_peered":"2023-05-20T01:26:36.092842+0000","last_unstale":"2023-05-20T01:26:38.710901+0000","last_undegraded":"2023-05-20T01:26:38.710901+0000","last_fullsized":"2023-05-20T01:26:38.710901+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:18:32.715040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.387928+0000","last_change":"2023-05-20T01:26:36.084528+0000","last_active":"2023-05-20T01:26:38.387928+0000","last_peered":"2023-05-20T01:26:38.387928+0000","last_clean":"2023-05-20T01:26:38.387928+0000","last_became_active":"2023-05-20T01:26:36.084261+0000","last_became_peered":"2023-05-20T01:26:36.084261+0000","last_unstale":"2023-05-20T01:26:38.387928+0000","last_undegraded":"2023-05-20T01:26:38.387928+0000","last_fullsized":"2023-05-20T01:26:38.387928+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T13:21:38.550913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.710170+0000","last_change":"2023-05-20T01:26:36.088685+0000","last_active":"2023-05-20T01:26:38.710170+0000","last_peered":"2023-05-20T01:26:38.710170+0000","last_clean":"2023-05-20T01:26:38.710170+0000","last_became_active":"2023-05-20T01:26:36.088420+0000","last_became_peered":"2023-05-20T01:26:36.088420+0000","last_unstale":"2023-05-20T01:26:38.710170+0000","last_undegraded":"2023-05-20T01:26:38.710170+0000","last_fullsized":"2023-05-20T01:26:38.710170+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:48:55.089787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:37.084874+0000","last_change":"2023-05-20T01:26:36.098577+0000","last_active":"2023-05-20T01:26:37.084874+0000","last_peered":"2023-05-20T01:26:37.084874+0000","last_clean":"2023-05-20T01:26:37.084874+0000","last_became_active":"2023-05-20T01:26:36.098167+0000","last_became_peered":"2023-05-20T01:26:36.098167+0000","last_unstale":"2023-05-20T01:26:37.084874+0000","last_undegraded":"2023-05-20T01:26:37.084874+0000","last_fullsized":"2023-05-20T01:26:37.084874+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:17:37.043790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:37.085072+0000","last_change":"2023-05-20T01:26:36.097405+0000","last_active":"2023-05-20T01:26:37.085072+0000","last_peered":"2023-05-20T01:26:37.085072+0000","last_clean":"2023-05-20T01:26:37.085072+0000","last_became_active":"2023-05-20T01:26:36.097120+0000","last_became_peered":"2023-05-20T01:26:36.097120+0000","last_unstale":"2023-05-20T01:26:37.085072+0000","last_undegraded":"2023-05-20T01:26:37.085072+0000","last_fullsized":"2023-05-20T01:26:37.085072+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:26:42.366377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.710531+0000","last_change":"2023-05-20T01:26:36.093252+0000","last_active":"2023-05-20T01:26:38.710531+0000","last_peered":"2023-05-20T01:26:38.710531+0000","last_clean":"2023-05-20T01:26:38.710531+0000","last_became_active":"2023-05-20T01:26:36.092989+0000","last_became_peered":"2023-05-20T01:26:36.092989+0000","last_unstale":"2023-05-20T01:26:38.710531+0000","last_undegraded":"2023-05-20T01:26:38.710531+0000","last_fullsized":"2023-05-20T01:26:38.710531+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:56:41.862467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:37.085582+0000","last_change":"2023-05-20T01:26:36.088025+0000","last_active":"2023-05-20T01:26:37.085582+0000","last_peered":"2023-05-20T01:26:37.085582+0000","last_clean":"2023-05-20T01:26:37.085582+0000","last_became_active":"2023-05-20T01:26:36.087683+0000","last_became_peered":"2023-05-20T01:26:36.087683+0000","last_unstale":"2023-05-20T01:26:37.085582+0000","last_undegraded":"2023-05-20T01:26:37.085582+0000","last_fullsized":"2023-05-20T01:26:37.085582+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:11:00.910242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.710443+0000","last_change":"2023-05-20T01:26:36.097332+0000","last_active":"2023-05-20T01:26:38.710443+0000","last_peered":"2023-05-20T01:26:38.710443+0000","last_clean":"2023-05-20T01:26:38.710443+0000","last_became_active":"2023-05-20T01:26:36.097091+0000","last_became_peered":"2023-05-20T01:26:36.097091+0000","last_unstale":"2023-05-20T01:26:38.710443+0000","last_undegraded":"2023-05-20T01:26:38.710443+0000","last_fullsized":"2023-05-20T01:26:38.710443+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T01:55:28.226587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:37.084366+0000","last_change":"2023-05-20T01:26:36.098206+0000","last_active":"2023-05-20T01:26:37.084366+0000","last_peered":"2023-05-20T01:26:37.084366+0000","last_clean":"2023-05-20T01:26:37.084366+0000","last_became_active":"2023-05-20T01:26:36.097871+0000","last_became_peered":"2023-05-20T01:26:36.097871+0000","last_unstale":"2023-05-20T01:26:37.084366+0000","last_undegraded":"2023-05-20T01:26:37.084366+0000","last_fullsized":"2023-05-20T01:26:37.084366+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:57:28.773432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:37.085356+0000","last_change":"2023-05-20T01:26:36.090284+0000","last_active":"2023-05-20T01:26:37.085356+0000","last_peered":"2023-05-20T01:26:37.085356+0000","last_clean":"2023-05-20T01:26:37.085356+0000","last_became_active":"2023-05-20T01:26:36.089977+0000","last_became_peered":"2023-05-20T01:26:36.089977+0000","last_unstale":"2023-05-20T01:26:37.085356+0000","last_undegraded":"2023-05-20T01:26:37.085356+0000","last_fullsized":"2023-05-20T01:26:37.085356+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T01:33:35.105606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.710420+0000","last_change":"2023-05-20T01:26:36.093275+0000","last_active":"2023-05-20T01:26:38.710420+0000","last_peered":"2023-05-20T01:26:38.710420+0000","last_clean":"2023-05-20T01:26:38.710420+0000","last_became_active":"2023-05-20T01:26:36.093007+0000","last_became_peered":"2023-05-20T01:26:36.093007+0000","last_unstale":"2023-05-20T01:26:38.710420+0000","last_undegraded":"2023-05-20T01:26:38.710420+0000","last_fullsized":"2023-05-20T01:26:38.710420+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:28:12.346951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.388141+0000","last_change":"2023-05-20T01:26:36.088499+0000","last_active":"2023-05-20T01:26:38.388141+0000","last_peered":"2023-05-20T01:26:38.388141+0000","last_clean":"2023-05-20T01:26:38.388141+0000","last_became_active":"2023-05-20T01:26:36.088216+0000","last_became_peered":"2023-05-20T01:26:36.088216+0000","last_unstale":"2023-05-20T01:26:38.388141+0000","last_undegraded":"2023-05-20T01:26:38.388141+0000","last_fullsized":"2023-05-20T01:26:38.388141+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:27:02.653342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"111'4","reported_seq":17,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:37.079844+0000","last_change":"2023-05-20T01:26:35.658045+0000","last_active":"2023-05-20T01:26:37.079844+0000","last_peered":"2023-05-20T01:26:37.079844+0000","last_clean":"2023-05-20T01:26:37.079844+0000","last_became_active":"2023-05-20T01:26:35.657827+0000","last_became_peered":"2023-05-20T01:26:35.657827+0000","last_unstale":"2023-05-20T01:26:37.079844+0000","last_undegraded":"2023-05-20T01:26:37.079844+0000","last_fullsized":"2023-05-20T01:26:37.079844+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:04:53.335119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.710845+0000","last_change":"2023-05-20T01:26:36.096394+0000","last_active":"2023-05-20T01:26:38.710845+0000","last_peered":"2023-05-20T01:26:38.710845+0000","last_clean":"2023-05-20T01:26:38.710845+0000","last_became_active":"2023-05-20T01:26:36.096130+0000","last_became_peered":"2023-05-20T01:26:36.096130+0000","last_unstale":"2023-05-20T01:26:38.710845+0000","last_undegraded":"2023-05-20T01:26:38.710845+0000","last_fullsized":"2023-05-20T01:26:38.710845+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:33:46.171949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"111'1","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.710699+0000","last_change":"2023-05-20T01:26:35.663547+0000","last_active":"2023-05-20T01:26:38.710699+0000","last_peered":"2023-05-20T01:26:38.710699+0000","last_clean":"2023-05-20T01:26:38.710699+0000","last_became_active":"2023-05-20T01:26:35.663282+0000","last_became_peered":"2023-05-20T01:26:35.663282+0000","last_unstale":"2023-05-20T01:26:38.710699+0000","last_undegraded":"2023-05-20T01:26:38.710699+0000","last_fullsized":"2023-05-20T01:26:38.710699+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:12:00.672741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:37.084432+0000","last_change":"2023-05-20T01:26:36.091809+0000","last_active":"2023-05-20T01:26:37.084432+0000","last_peered":"2023-05-20T01:26:37.084432+0000","last_clean":"2023-05-20T01:26:37.084432+0000","last_became_active":"2023-05-20T01:26:36.091435+0000","last_became_peered":"2023-05-20T01:26:36.091435+0000","last_unstale":"2023-05-20T01:26:37.084432+0000","last_undegraded":"2023-05-20T01:26:37.084432+0000","last_fullsized":"2023-05-20T01:26:37.084432+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:18:59.295525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"111'2","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:37.084653+0000","last_change":"2023-05-20T01:26:35.662991+0000","last_active":"2023-05-20T01:26:37.084653+0000","last_peered":"2023-05-20T01:26:37.084653+0000","last_clean":"2023-05-20T01:26:37.084653+0000","last_became_active":"2023-05-20T01:26:35.662055+0000","last_became_peered":"2023-05-20T01:26:35.662055+0000","last_unstale":"2023-05-20T01:26:37.084653+0000","last_undegraded":"2023-05-20T01:26:37.084653+0000","last_fullsized":"2023-05-20T01:26:37.084653+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:11:56.969558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.710254+0000","last_change":"2023-05-20T01:26:36.090727+0000","last_active":"2023-05-20T01:26:38.710254+0000","last_peered":"2023-05-20T01:26:38.710254+0000","last_clean":"2023-05-20T01:26:38.710254+0000","last_became_active":"2023-05-20T01:26:36.090471+0000","last_became_peered":"2023-05-20T01:26:36.090471+0000","last_unstale":"2023-05-20T01:26:38.710254+0000","last_undegraded":"2023-05-20T01:26:38.710254+0000","last_fullsized":"2023-05-20T01:26:38.710254+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:54:14.429799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"111'2","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.710939+0000","last_change":"2023-05-20T01:26:35.662610+0000","last_active":"2023-05-20T01:26:38.710939+0000","last_peered":"2023-05-20T01:26:38.710939+0000","last_clean":"2023-05-20T01:26:38.710939+0000","last_became_active":"2023-05-20T01:26:35.662325+0000","last_became_peered":"2023-05-20T01:26:35.662325+0000","last_unstale":"2023-05-20T01:26:38.710939+0000","last_undegraded":"2023-05-20T01:26:38.710939+0000","last_fullsized":"2023-05-20T01:26:38.710939+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:41:19.578105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:37.079965+0000","last_change":"2023-05-20T01:26:36.085271+0000","last_active":"2023-05-20T01:26:37.079965+0000","last_peered":"2023-05-20T01:26:37.079965+0000","last_clean":"2023-05-20T01:26:37.079965+0000","last_became_active":"2023-05-20T01:26:36.085000+0000","last_became_peered":"2023-05-20T01:26:36.085000+0000","last_unstale":"2023-05-20T01:26:37.079965+0000","last_undegraded":"2023-05-20T01:26:37.079965+0000","last_fullsized":"2023-05-20T01:26:37.079965+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:55:54.583693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"111'1","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.585261+0000","last_change":"2023-05-20T01:26:35.665455+0000","last_active":"2023-05-20T01:26:38.585261+0000","last_peered":"2023-05-20T01:26:38.585261+0000","last_clean":"2023-05-20T01:26:38.585261+0000","last_became_active":"2023-05-20T01:26:35.665131+0000","last_became_peered":"2023-05-20T01:26:35.665131+0000","last_unstale":"2023-05-20T01:26:38.585261+0000","last_undegraded":"2023-05-20T01:26:38.585261+0000","last_fullsized":"2023-05-20T01:26:38.585261+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:08:18.586264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.388276+0000","last_change":"2023-05-20T01:26:36.094004+0000","last_active":"2023-05-20T01:26:38.388276+0000","last_peered":"2023-05-20T01:26:38.388276+0000","last_clean":"2023-05-20T01:26:38.388276+0000","last_became_active":"2023-05-20T01:26:36.093393+0000","last_became_peered":"2023-05-20T01:26:36.093393+0000","last_unstale":"2023-05-20T01:26:38.388276+0000","last_undegraded":"2023-05-20T01:26:38.388276+0000","last_fullsized":"2023-05-20T01:26:38.388276+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:30:02.313003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"111'1","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.710035+0000","last_change":"2023-05-20T01:26:35.664025+0000","last_active":"2023-05-20T01:26:38.710035+0000","last_peered":"2023-05-20T01:26:38.710035+0000","last_clean":"2023-05-20T01:26:38.710035+0000","last_became_active":"2023-05-20T01:26:35.663803+0000","last_became_peered":"2023-05-20T01:26:35.663803+0000","last_unstale":"2023-05-20T01:26:38.710035+0000","last_undegraded":"2023-05-20T01:26:38.710035+0000","last_fullsized":"2023-05-20T01:26:38.710035+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:34:07.792503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:37.086053+0000","last_change":"2023-05-20T01:26:36.085317+0000","last_active":"2023-05-20T01:26:37.086053+0000","last_peered":"2023-05-20T01:26:37.086053+0000","last_clean":"2023-05-20T01:26:37.086053+0000","last_became_active":"2023-05-20T01:26:36.084792+0000","last_became_peered":"2023-05-20T01:26:36.084792+0000","last_unstale":"2023-05-20T01:26:37.086053+0000","last_undegraded":"2023-05-20T01:26:37.086053+0000","last_fullsized":"2023-05-20T01:26:37.086053+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:40:05.319863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:37.084283+0000","last_change":"2023-05-20T01:26:35.661709+0000","last_active":"2023-05-20T01:26:37.084283+0000","last_peered":"2023-05-20T01:26:37.084283+0000","last_clean":"2023-05-20T01:26:37.084283+0000","last_became_active":"2023-05-20T01:26:35.661425+0000","last_became_peered":"2023-05-20T01:26:35.661425+0000","last_unstale":"2023-05-20T01:26:37.084283+0000","last_undegraded":"2023-05-20T01:26:37.084283+0000","last_fullsized":"2023-05-20T01:26:37.084283+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:16:44.004923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:37.080031+0000","last_change":"2023-05-20T01:26:36.085287+0000","last_active":"2023-05-20T01:26:37.080031+0000","last_peered":"2023-05-20T01:26:37.080031+0000","last_clean":"2023-05-20T01:26:37.080031+0000","last_became_active":"2023-05-20T01:26:36.085013+0000","last_became_peered":"2023-05-20T01:26:36.085013+0000","last_unstale":"2023-05-20T01:26:37.080031+0000","last_undegraded":"2023-05-20T01:26:37.080031+0000","last_fullsized":"2023-05-20T01:26:37.080031+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:11:15.186887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:37.079515+0000","last_change":"2023-05-20T01:26:35.649191+0000","last_active":"2023-05-20T01:26:37.079515+0000","last_peered":"2023-05-20T01:26:37.079515+0000","last_clean":"2023-05-20T01:26:37.079515+0000","last_became_active":"2023-05-20T01:26:35.648091+0000","last_became_peered":"2023-05-20T01:26:35.648091+0000","last_unstale":"2023-05-20T01:26:37.079515+0000","last_undegraded":"2023-05-20T01:26:37.079515+0000","last_fullsized":"2023-05-20T01:26:37.079515+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:10:27.773443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.388142+0000","last_change":"2023-05-20T01:26:36.097545+0000","last_active":"2023-05-20T01:26:38.388142+0000","last_peered":"2023-05-20T01:26:38.388142+0000","last_clean":"2023-05-20T01:26:38.388142+0000","last_became_active":"2023-05-20T01:26:36.093102+0000","last_became_peered":"2023-05-20T01:26:36.093102+0000","last_unstale":"2023-05-20T01:26:38.388142+0000","last_undegraded":"2023-05-20T01:26:38.388142+0000","last_fullsized":"2023-05-20T01:26:38.388142+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:40:37.047579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:37.084969+0000","last_change":"2023-05-20T01:26:35.661860+0000","last_active":"2023-05-20T01:26:37.084969+0000","last_peered":"2023-05-20T01:26:37.084969+0000","last_clean":"2023-05-20T01:26:37.084969+0000","last_became_active":"2023-05-20T01:26:35.661520+0000","last_became_peered":"2023-05-20T01:26:35.661520+0000","last_unstale":"2023-05-20T01:26:37.084969+0000","last_undegraded":"2023-05-20T01:26:37.084969+0000","last_fullsized":"2023-05-20T01:26:37.084969+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:26:28.474417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.710779+0000","last_change":"2023-05-20T01:26:36.101022+0000","last_active":"2023-05-20T01:26:38.710779+0000","last_peered":"2023-05-20T01:26:38.710779+0000","last_clean":"2023-05-20T01:26:38.710779+0000","last_became_active":"2023-05-20T01:26:36.100820+0000","last_became_peered":"2023-05-20T01:26:36.100820+0000","last_unstale":"2023-05-20T01:26:38.710779+0000","last_undegraded":"2023-05-20T01:26:38.710779+0000","last_fullsized":"2023-05-20T01:26:38.710779+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:30:15.369472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.388507+0000","last_change":"2023-05-20T01:26:35.660602+0000","last_active":"2023-05-20T01:26:38.388507+0000","last_peered":"2023-05-20T01:26:38.388507+0000","last_clean":"2023-05-20T01:26:38.388507+0000","last_became_active":"2023-05-20T01:26:35.660324+0000","last_became_peered":"2023-05-20T01:26:35.660324+0000","last_unstale":"2023-05-20T01:26:38.388507+0000","last_undegraded":"2023-05-20T01:26:38.388507+0000","last_fullsized":"2023-05-20T01:26:38.388507+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:42:31.290518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.710399+0000","last_change":"2023-05-20T01:26:36.099492+0000","last_active":"2023-05-20T01:26:38.710399+0000","last_peered":"2023-05-20T01:26:38.710399+0000","last_clean":"2023-05-20T01:26:38.710399+0000","last_became_active":"2023-05-20T01:26:36.099226+0000","last_became_peered":"2023-05-20T01:26:36.099226+0000","last_unstale":"2023-05-20T01:26:38.710399+0000","last_undegraded":"2023-05-20T01:26:38.710399+0000","last_fullsized":"2023-05-20T01:26:38.710399+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:45:00.597228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"111'3","reported_seq":16,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.585343+0000","last_change":"2023-05-20T01:26:35.660335+0000","last_active":"2023-05-20T01:26:38.585343+0000","last_peered":"2023-05-20T01:26:38.585343+0000","last_clean":"2023-05-20T01:26:38.585343+0000","last_became_active":"2023-05-20T01:26:35.660069+0000","last_became_peered":"2023-05-20T01:26:35.660069+0000","last_unstale":"2023-05-20T01:26:38.585343+0000","last_undegraded":"2023-05-20T01:26:38.585343+0000","last_fullsized":"2023-05-20T01:26:38.585343+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T01:48:11.207779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.710588+0000","last_change":"2023-05-20T01:26:36.099329+0000","last_active":"2023-05-20T01:26:38.710588+0000","last_peered":"2023-05-20T01:26:38.710588+0000","last_clean":"2023-05-20T01:26:38.710588+0000","last_became_active":"2023-05-20T01:26:36.099062+0000","last_became_peered":"2023-05-20T01:26:36.099062+0000","last_unstale":"2023-05-20T01:26:38.710588+0000","last_undegraded":"2023-05-20T01:26:38.710588+0000","last_fullsized":"2023-05-20T01:26:38.710588+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:59:31.710005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"111'6","reported_seq":19,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:37.085649+0000","last_change":"2023-05-20T01:26:35.662782+0000","last_active":"2023-05-20T01:26:37.085649+0000","last_peered":"2023-05-20T01:26:37.085649+0000","last_clean":"2023-05-20T01:26:37.085649+0000","last_became_active":"2023-05-20T01:26:35.662462+0000","last_became_peered":"2023-05-20T01:26:35.662462+0000","last_unstale":"2023-05-20T01:26:37.085649+0000","last_undegraded":"2023-05-20T01:26:37.085649+0000","last_fullsized":"2023-05-20T01:26:37.085649+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:22:15.875557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.710253+0000","last_change":"2023-05-20T01:26:36.089582+0000","last_active":"2023-05-20T01:26:38.710253+0000","last_peered":"2023-05-20T01:26:38.710253+0000","last_clean":"2023-05-20T01:26:38.710253+0000","last_became_active":"2023-05-20T01:26:36.089301+0000","last_became_peered":"2023-05-20T01:26:36.089301+0000","last_unstale":"2023-05-20T01:26:38.710253+0000","last_undegraded":"2023-05-20T01:26:38.710253+0000","last_fullsized":"2023-05-20T01:26:38.710253+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:54:08.199336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"111'2","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.710957+0000","last_change":"2023-05-20T01:26:35.662523+0000","last_active":"2023-05-20T01:26:38.710957+0000","last_peered":"2023-05-20T01:26:38.710957+0000","last_clean":"2023-05-20T01:26:38.710957+0000","last_became_active":"2023-05-20T01:26:35.662205+0000","last_became_peered":"2023-05-20T01:26:35.662205+0000","last_unstale":"2023-05-20T01:26:38.710957+0000","last_undegraded":"2023-05-20T01:26:38.710957+0000","last_fullsized":"2023-05-20T01:26:38.710957+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:36:38.412172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:37.085851+0000","last_change":"2023-05-20T01:26:36.086463+0000","last_active":"2023-05-20T01:26:37.085851+0000","last_peered":"2023-05-20T01:26:37.085851+0000","last_clean":"2023-05-20T01:26:37.085851+0000","last_became_active":"2023-05-20T01:26:36.086154+0000","last_became_peered":"2023-05-20T01:26:36.086154+0000","last_unstale":"2023-05-20T01:26:37.085851+0000","last_undegraded":"2023-05-20T01:26:37.085851+0000","last_fullsized":"2023-05-20T01:26:37.085851+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:18:48.948205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:37.079815+0000","last_change":"2023-05-20T01:26:35.661955+0000","last_active":"2023-05-20T01:26:37.079815+0000","last_peered":"2023-05-20T01:26:37.079815+0000","last_clean":"2023-05-20T01:26:37.079815+0000","last_became_active":"2023-05-20T01:26:35.661556+0000","last_became_peered":"2023-05-20T01:26:35.661556+0000","last_unstale":"2023-05-20T01:26:37.079815+0000","last_undegraded":"2023-05-20T01:26:37.079815+0000","last_fullsized":"2023-05-20T01:26:37.079815+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:18:07.621593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.710599+0000","last_change":"2023-05-20T01:26:36.097947+0000","last_active":"2023-05-20T01:26:38.710599+0000","last_peered":"2023-05-20T01:26:38.710599+0000","last_clean":"2023-05-20T01:26:38.710599+0000","last_became_active":"2023-05-20T01:26:36.097689+0000","last_became_peered":"2023-05-20T01:26:36.097689+0000","last_unstale":"2023-05-20T01:26:38.710599+0000","last_undegraded":"2023-05-20T01:26:38.710599+0000","last_fullsized":"2023-05-20T01:26:38.710599+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:13:38.420510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.710554+0000","last_change":"2023-05-20T01:26:35.660215+0000","last_active":"2023-05-20T01:26:38.710554+0000","last_peered":"2023-05-20T01:26:38.710554+0000","last_clean":"2023-05-20T01:26:38.710554+0000","last_became_active":"2023-05-20T01:26:35.659873+0000","last_became_peered":"2023-05-20T01:26:35.659873+0000","last_unstale":"2023-05-20T01:26:38.710554+0000","last_undegraded":"2023-05-20T01:26:38.710554+0000","last_fullsized":"2023-05-20T01:26:38.710554+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:33:52.131861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.388408+0000","last_change":"2023-05-20T01:26:36.090068+0000","last_active":"2023-05-20T01:26:38.388408+0000","last_peered":"2023-05-20T01:26:38.388408+0000","last_clean":"2023-05-20T01:26:38.388408+0000","last_became_active":"2023-05-20T01:26:36.089776+0000","last_became_peered":"2023-05-20T01:26:36.089776+0000","last_unstale":"2023-05-20T01:26:38.388408+0000","last_undegraded":"2023-05-20T01:26:38.388408+0000","last_fullsized":"2023-05-20T01:26:38.388408+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:46:24.436985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"111'8","reported_seq":21,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:37.084312+0000","last_change":"2023-05-20T01:26:35.662613+0000","last_active":"2023-05-20T01:26:37.084312+0000","last_peered":"2023-05-20T01:26:37.084312+0000","last_clean":"2023-05-20T01:26:37.084312+0000","last_became_active":"2023-05-20T01:26:35.661401+0000","last_became_peered":"2023-05-20T01:26:35.661401+0000","last_unstale":"2023-05-20T01:26:37.084312+0000","last_undegraded":"2023-05-20T01:26:37.084312+0000","last_fullsized":"2023-05-20T01:26:37.084312+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:35:33.516158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.387962+0000","last_change":"2023-05-20T01:26:36.087191+0000","last_active":"2023-05-20T01:26:38.387962+0000","last_peered":"2023-05-20T01:26:38.387962+0000","last_clean":"2023-05-20T01:26:38.387962+0000","last_became_active":"2023-05-20T01:26:36.086899+0000","last_became_peered":"2023-05-20T01:26:36.086899+0000","last_unstale":"2023-05-20T01:26:38.387962+0000","last_undegraded":"2023-05-20T01:26:38.387962+0000","last_fullsized":"2023-05-20T01:26:38.387962+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:22:24.848309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"111'2","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.710629+0000","last_change":"2023-05-20T01:26:35.662291+0000","last_active":"2023-05-20T01:26:38.710629+0000","last_peered":"2023-05-20T01:26:38.710629+0000","last_clean":"2023-05-20T01:26:38.710629+0000","last_became_active":"2023-05-20T01:26:35.661970+0000","last_became_peered":"2023-05-20T01:26:35.661970+0000","last_unstale":"2023-05-20T01:26:38.710629+0000","last_undegraded":"2023-05-20T01:26:38.710629+0000","last_fullsized":"2023-05-20T01:26:38.710629+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:34:56.803848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:37.084339+0000","last_change":"2023-05-20T01:26:36.087917+0000","last_active":"2023-05-20T01:26:37.084339+0000","last_peered":"2023-05-20T01:26:37.084339+0000","last_clean":"2023-05-20T01:26:37.084339+0000","last_became_active":"2023-05-20T01:26:36.087647+0000","last_became_peered":"2023-05-20T01:26:36.087647+0000","last_unstale":"2023-05-20T01:26:37.084339+0000","last_undegraded":"2023-05-20T01:26:37.084339+0000","last_fullsized":"2023-05-20T01:26:37.084339+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:40:40.590526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.388290+0000","last_change":"2023-05-20T01:26:35.660495+0000","last_active":"2023-05-20T01:26:38.388290+0000","last_peered":"2023-05-20T01:26:38.388290+0000","last_clean":"2023-05-20T01:26:38.388290+0000","last_became_active":"2023-05-20T01:26:35.660218+0000","last_became_peered":"2023-05-20T01:26:35.660218+0000","last_unstale":"2023-05-20T01:26:38.388290+0000","last_undegraded":"2023-05-20T01:26:38.388290+0000","last_fullsized":"2023-05-20T01:26:38.388290+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T13:17:28.416406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:37.079885+0000","last_change":"2023-05-20T01:26:36.077930+0000","last_active":"2023-05-20T01:26:37.079885+0000","last_peered":"2023-05-20T01:26:37.079885+0000","last_clean":"2023-05-20T01:26:37.079885+0000","last_became_active":"2023-05-20T01:26:36.077655+0000","last_became_peered":"2023-05-20T01:26:36.077655+0000","last_unstale":"2023-05-20T01:26:37.079885+0000","last_undegraded":"2023-05-20T01:26:37.079885+0000","last_fullsized":"2023-05-20T01:26:37.079885+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:07:19.398609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:37.085847+0000","last_change":"2023-05-20T01:26:35.655521+0000","last_active":"2023-05-20T01:26:37.085847+0000","last_peered":"2023-05-20T01:26:37.085847+0000","last_clean":"2023-05-20T01:26:37.085847+0000","last_became_active":"2023-05-20T01:26:35.655217+0000","last_became_peered":"2023-05-20T01:26:35.655217+0000","last_unstale":"2023-05-20T01:26:37.085847+0000","last_undegraded":"2023-05-20T01:26:37.085847+0000","last_fullsized":"2023-05-20T01:26:37.085847+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:28:08.499442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.710124+0000","last_change":"2023-05-20T01:26:36.088402+0000","last_active":"2023-05-20T01:26:38.710124+0000","last_peered":"2023-05-20T01:26:38.710124+0000","last_clean":"2023-05-20T01:26:38.710124+0000","last_became_active":"2023-05-20T01:26:36.085409+0000","last_became_peered":"2023-05-20T01:26:36.085409+0000","last_unstale":"2023-05-20T01:26:38.710124+0000","last_undegraded":"2023-05-20T01:26:38.710124+0000","last_fullsized":"2023-05-20T01:26:38.710124+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:00:02.738451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.711040+0000","last_change":"2023-05-20T01:26:35.652937+0000","last_active":"2023-05-20T01:26:38.711040+0000","last_peered":"2023-05-20T01:26:38.711040+0000","last_clean":"2023-05-20T01:26:38.711040+0000","last_became_active":"2023-05-20T01:26:35.652635+0000","last_became_peered":"2023-05-20T01:26:35.652635+0000","last_unstale":"2023-05-20T01:26:38.711040+0000","last_undegraded":"2023-05-20T01:26:38.711040+0000","last_fullsized":"2023-05-20T01:26:38.711040+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:24:24.769004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.710820+0000","last_change":"2023-05-20T01:26:36.092027+0000","last_active":"2023-05-20T01:26:38.710820+0000","last_peered":"2023-05-20T01:26:38.710820+0000","last_clean":"2023-05-20T01:26:38.710820+0000","last_became_active":"2023-05-20T01:26:36.091712+0000","last_became_peered":"2023-05-20T01:26:36.091712+0000","last_unstale":"2023-05-20T01:26:38.710820+0000","last_undegraded":"2023-05-20T01:26:38.710820+0000","last_fullsized":"2023-05-20T01:26:38.710820+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:32:55.788621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:37.084437+0000","last_change":"2023-05-20T01:26:35.665478+0000","last_active":"2023-05-20T01:26:37.084437+0000","last_peered":"2023-05-20T01:26:37.084437+0000","last_clean":"2023-05-20T01:26:37.084437+0000","last_became_active":"2023-05-20T01:26:35.665164+0000","last_became_peered":"2023-05-20T01:26:35.665164+0000","last_unstale":"2023-05-20T01:26:37.084437+0000","last_undegraded":"2023-05-20T01:26:37.084437+0000","last_fullsized":"2023-05-20T01:26:37.084437+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:42:07.320423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.710624+0000","last_change":"2023-05-20T01:26:36.088743+0000","last_active":"2023-05-20T01:26:38.710624+0000","last_peered":"2023-05-20T01:26:38.710624+0000","last_clean":"2023-05-20T01:26:38.710624+0000","last_became_active":"2023-05-20T01:26:36.088466+0000","last_became_peered":"2023-05-20T01:26:36.088466+0000","last_unstale":"2023-05-20T01:26:38.710624+0000","last_undegraded":"2023-05-20T01:26:38.710624+0000","last_fullsized":"2023-05-20T01:26:38.710624+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:04:01.736880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.710626+0000","last_change":"2023-05-20T01:26:35.660517+0000","last_active":"2023-05-20T01:26:38.710626+0000","last_peered":"2023-05-20T01:26:38.710626+0000","last_clean":"2023-05-20T01:26:38.710626+0000","last_became_active":"2023-05-20T01:26:35.660240+0000","last_became_peered":"2023-05-20T01:26:35.660240+0000","last_unstale":"2023-05-20T01:26:38.710626+0000","last_undegraded":"2023-05-20T01:26:38.710626+0000","last_fullsized":"2023-05-20T01:26:38.710626+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:22:05.762439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.585497+0000","last_change":"2023-05-20T01:26:36.093031+0000","last_active":"2023-05-20T01:26:38.585497+0000","last_peered":"2023-05-20T01:26:38.585497+0000","last_clean":"2023-05-20T01:26:38.585497+0000","last_became_active":"2023-05-20T01:26:36.092732+0000","last_became_peered":"2023-05-20T01:26:36.092732+0000","last_unstale":"2023-05-20T01:26:38.585497+0000","last_undegraded":"2023-05-20T01:26:38.585497+0000","last_fullsized":"2023-05-20T01:26:38.585497+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:30:31.423277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:37.085471+0000","last_change":"2023-05-20T01:26:35.653958+0000","last_active":"2023-05-20T01:26:37.085471+0000","last_peered":"2023-05-20T01:26:37.085471+0000","last_clean":"2023-05-20T01:26:37.085471+0000","last_became_active":"2023-05-20T01:26:35.653285+0000","last_became_peered":"2023-05-20T01:26:35.653285+0000","last_unstale":"2023-05-20T01:26:37.085471+0000","last_undegraded":"2023-05-20T01:26:37.085471+0000","last_fullsized":"2023-05-20T01:26:37.085471+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:52:23.814568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.710424+0000","last_change":"2023-05-20T01:26:36.095384+0000","last_active":"2023-05-20T01:26:38.710424+0000","last_peered":"2023-05-20T01:26:38.710424+0000","last_clean":"2023-05-20T01:26:38.710424+0000","last_became_active":"2023-05-20T01:26:36.095143+0000","last_became_peered":"2023-05-20T01:26:36.095143+0000","last_unstale":"2023-05-20T01:26:38.710424+0000","last_undegraded":"2023-05-20T01:26:38.710424+0000","last_fullsized":"2023-05-20T01:26:38.710424+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:28:21.553443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.711154+0000","last_change":"2023-05-20T01:26:35.662061+0000","last_active":"2023-05-20T01:26:38.711154+0000","last_peered":"2023-05-20T01:26:38.711154+0000","last_clean":"2023-05-20T01:26:38.711154+0000","last_became_active":"2023-05-20T01:26:35.660711+0000","last_became_peered":"2023-05-20T01:26:35.660711+0000","last_unstale":"2023-05-20T01:26:38.711154+0000","last_undegraded":"2023-05-20T01:26:38.711154+0000","last_fullsized":"2023-05-20T01:26:38.711154+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:33:20.487893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.388177+0000","last_change":"2023-05-20T01:26:36.091123+0000","last_active":"2023-05-20T01:26:38.388177+0000","last_peered":"2023-05-20T01:26:38.388177+0000","last_clean":"2023-05-20T01:26:38.388177+0000","last_became_active":"2023-05-20T01:26:36.090841+0000","last_became_peered":"2023-05-20T01:26:36.090841+0000","last_unstale":"2023-05-20T01:26:38.388177+0000","last_undegraded":"2023-05-20T01:26:38.388177+0000","last_fullsized":"2023-05-20T01:26:38.388177+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:17:58.013423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"111'2","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.710624+0000","last_change":"2023-05-20T01:26:35.661288+0000","last_active":"2023-05-20T01:26:38.710624+0000","last_peered":"2023-05-20T01:26:38.710624+0000","last_clean":"2023-05-20T01:26:38.710624+0000","last_became_active":"2023-05-20T01:26:35.660976+0000","last_became_peered":"2023-05-20T01:26:35.660976+0000","last_unstale":"2023-05-20T01:26:38.710624+0000","last_undegraded":"2023-05-20T01:26:38.710624+0000","last_fullsized":"2023-05-20T01:26:38.710624+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:12:43.594752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":197,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:37.084799+0000","last_change":"2023-05-20T01:16:23.289068+0000","last_active":"2023-05-20T01:26:37.084799+0000","last_peered":"2023-05-20T01:26:37.084799+0000","last_clean":"2023-05-20T01:26:37.084799+0000","last_became_active":"2023-05-20T01:16:23.288781+0000","last_became_peered":"2023-05-20T01:16:23.288781+0000","last_unstale":"2023-05-20T01:26:37.084799+0000","last_undegraded":"2023-05-20T01:26:37.084799+0000","last_fullsized":"2023-05-20T01:26:37.084799+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:22:36.000137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.388316+0000","last_change":"2023-05-20T01:26:36.089408+0000","last_active":"2023-05-20T01:26:38.388316+0000","last_peered":"2023-05-20T01:26:38.388316+0000","last_clean":"2023-05-20T01:26:38.388316+0000","last_became_active":"2023-05-20T01:26:36.088124+0000","last_became_peered":"2023-05-20T01:26:36.088124+0000","last_unstale":"2023-05-20T01:26:38.388316+0000","last_undegraded":"2023-05-20T01:26:38.388316+0000","last_fullsized":"2023-05-20T01:26:38.388316+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:32:46.166136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.710489+0000","last_change":"2023-05-20T01:26:35.659858+0000","last_active":"2023-05-20T01:26:38.710489+0000","last_peered":"2023-05-20T01:26:38.710489+0000","last_clean":"2023-05-20T01:26:38.710489+0000","last_became_active":"2023-05-20T01:26:35.659432+0000","last_became_peered":"2023-05-20T01:26:35.659432+0000","last_unstale":"2023-05-20T01:26:38.710489+0000","last_undegraded":"2023-05-20T01:26:38.710489+0000","last_fullsized":"2023-05-20T01:26:38.710489+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:12:44.311239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":205,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:37.080044+0000","last_change":"2023-05-20T01:16:22.829324+0000","last_active":"2023-05-20T01:26:37.080044+0000","last_peered":"2023-05-20T01:26:37.080044+0000","last_clean":"2023-05-20T01:26:37.080044+0000","last_became_active":"2023-05-20T01:16:22.828964+0000","last_became_peered":"2023-05-20T01:16:22.828964+0000","last_unstale":"2023-05-20T01:26:37.080044+0000","last_undegraded":"2023-05-20T01:26:37.080044+0000","last_fullsized":"2023-05-20T01:26:37.080044+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:10:12.087621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:37.085468+0000","last_change":"2023-05-20T01:26:36.090093+0000","last_active":"2023-05-20T01:26:37.085468+0000","last_peered":"2023-05-20T01:26:37.085468+0000","last_clean":"2023-05-20T01:26:37.085468+0000","last_became_active":"2023-05-20T01:26:36.089787+0000","last_became_peered":"2023-05-20T01:26:36.089787+0000","last_unstale":"2023-05-20T01:26:37.085468+0000","last_undegraded":"2023-05-20T01:26:37.085468+0000","last_fullsized":"2023-05-20T01:26:37.085468+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:41:03.342972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"111'1","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.585424+0000","last_change":"2023-05-20T01:26:35.659399+0000","last_active":"2023-05-20T01:26:38.585424+0000","last_peered":"2023-05-20T01:26:38.585424+0000","last_clean":"2023-05-20T01:26:38.585424+0000","last_became_active":"2023-05-20T01:26:35.659081+0000","last_became_peered":"2023-05-20T01:26:35.659081+0000","last_unstale":"2023-05-20T01:26:38.585424+0000","last_undegraded":"2023-05-20T01:26:38.585424+0000","last_fullsized":"2023-05-20T01:26:38.585424+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:04:20.587342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":181,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.710416+0000","last_change":"2023-05-20T01:16:22.834754+0000","last_active":"2023-05-20T01:26:38.710416+0000","last_peered":"2023-05-20T01:26:38.710416+0000","last_clean":"2023-05-20T01:26:38.710416+0000","last_became_active":"2023-05-20T01:16:22.834437+0000","last_became_peered":"2023-05-20T01:16:22.834437+0000","last_unstale":"2023-05-20T01:26:38.710416+0000","last_undegraded":"2023-05-20T01:26:38.710416+0000","last_fullsized":"2023-05-20T01:26:38.710416+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:42:52.772764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":316,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.585048+0000","last_change":"2023-05-20T01:16:22.834920+0000","last_active":"2023-05-20T01:26:38.585048+0000","last_peered":"2023-05-20T01:26:38.585048+0000","last_clean":"2023-05-20T01:26:38.585048+0000","last_became_active":"2023-05-20T01:16:22.833924+0000","last_became_peered":"2023-05-20T01:16:22.833924+0000","last_unstale":"2023-05-20T01:26:38.585048+0000","last_undegraded":"2023-05-20T01:26:38.585048+0000","last_fullsized":"2023-05-20T01:26:38.585048+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:49:53.011248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.710724+0000","last_change":"2023-05-20T01:26:36.095844+0000","last_active":"2023-05-20T01:26:38.710724+0000","last_peered":"2023-05-20T01:26:38.710724+0000","last_clean":"2023-05-20T01:26:38.710724+0000","last_became_active":"2023-05-20T01:26:36.095578+0000","last_became_peered":"2023-05-20T01:26:36.095578+0000","last_unstale":"2023-05-20T01:26:38.710724+0000","last_undegraded":"2023-05-20T01:26:38.710724+0000","last_fullsized":"2023-05-20T01:26:38.710724+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:40:05.629844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.710414+0000","last_change":"2023-05-20T01:26:35.661736+0000","last_active":"2023-05-20T01:26:38.710414+0000","last_peered":"2023-05-20T01:26:38.710414+0000","last_clean":"2023-05-20T01:26:38.710414+0000","last_became_active":"2023-05-20T01:26:35.661426+0000","last_became_peered":"2023-05-20T01:26:35.661426+0000","last_unstale":"2023-05-20T01:26:38.710414+0000","last_undegraded":"2023-05-20T01:26:38.710414+0000","last_fullsized":"2023-05-20T01:26:38.710414+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:42:44.705803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":165,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.585229+0000","last_change":"2023-05-20T01:16:23.290259+0000","last_active":"2023-05-20T01:26:38.585229+0000","last_peered":"2023-05-20T01:26:38.585229+0000","last_clean":"2023-05-20T01:26:38.585229+0000","last_became_active":"2023-05-20T01:16:23.289911+0000","last_became_peered":"2023-05-20T01:16:23.289911+0000","last_unstale":"2023-05-20T01:26:38.585229+0000","last_undegraded":"2023-05-20T01:26:38.585229+0000","last_fullsized":"2023-05-20T01:26:38.585229+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:49:10.789562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.710366+0000","last_change":"2023-05-20T01:26:36.089519+0000","last_active":"2023-05-20T01:26:38.710366+0000","last_peered":"2023-05-20T01:26:38.710366+0000","last_clean":"2023-05-20T01:26:38.710366+0000","last_became_active":"2023-05-20T01:26:36.089108+0000","last_became_peered":"2023-05-20T01:26:36.089108+0000","last_unstale":"2023-05-20T01:26:38.710366+0000","last_undegraded":"2023-05-20T01:26:38.710366+0000","last_fullsized":"2023-05-20T01:26:38.710366+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:03:24.852613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"111'2","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.710758+0000","last_change":"2023-05-20T01:26:35.660034+0000","last_active":"2023-05-20T01:26:38.710758+0000","last_peered":"2023-05-20T01:26:38.710758+0000","last_clean":"2023-05-20T01:26:38.710758+0000","last_became_active":"2023-05-20T01:26:35.659396+0000","last_became_peered":"2023-05-20T01:26:35.659396+0000","last_unstale":"2023-05-20T01:26:38.710758+0000","last_undegraded":"2023-05-20T01:26:38.710758+0000","last_fullsized":"2023-05-20T01:26:38.710758+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:49:13.574029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":166,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.585265+0000","last_change":"2023-05-20T01:16:23.290207+0000","last_active":"2023-05-20T01:26:38.585265+0000","last_peered":"2023-05-20T01:26:38.585265+0000","last_clean":"2023-05-20T01:26:38.585265+0000","last_became_active":"2023-05-20T01:16:23.289928+0000","last_became_peered":"2023-05-20T01:16:23.289928+0000","last_unstale":"2023-05-20T01:26:38.585265+0000","last_undegraded":"2023-05-20T01:26:38.585265+0000","last_fullsized":"2023-05-20T01:26:38.585265+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:30:18.841320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.710991+0000","last_change":"2023-05-20T01:26:36.092735+0000","last_active":"2023-05-20T01:26:38.710991+0000","last_peered":"2023-05-20T01:26:38.710991+0000","last_clean":"2023-05-20T01:26:38.710991+0000","last_became_active":"2023-05-20T01:26:36.092299+0000","last_became_peered":"2023-05-20T01:26:36.092299+0000","last_unstale":"2023-05-20T01:26:38.710991+0000","last_undegraded":"2023-05-20T01:26:38.710991+0000","last_fullsized":"2023-05-20T01:26:38.710991+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:40:11.942170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"111'1","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:37.086008+0000","last_change":"2023-05-20T01:26:35.663015+0000","last_active":"2023-05-20T01:26:37.086008+0000","last_peered":"2023-05-20T01:26:37.086008+0000","last_clean":"2023-05-20T01:26:37.086008+0000","last_became_active":"2023-05-20T01:26:35.662730+0000","last_became_peered":"2023-05-20T01:26:35.662730+0000","last_unstale":"2023-05-20T01:26:37.086008+0000","last_undegraded":"2023-05-20T01:26:37.086008+0000","last_fullsized":"2023-05-20T01:26:37.086008+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:06:24.569009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":167,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.710042+0000","last_change":"2023-05-20T01:16:23.290538+0000","last_active":"2023-05-20T01:26:38.710042+0000","last_peered":"2023-05-20T01:26:38.710042+0000","last_clean":"2023-05-20T01:26:38.710042+0000","last_became_active":"2023-05-20T01:16:23.290230+0000","last_became_peered":"2023-05-20T01:16:23.290230+0000","last_unstale":"2023-05-20T01:26:38.710042+0000","last_undegraded":"2023-05-20T01:26:38.710042+0000","last_fullsized":"2023-05-20T01:26:38.710042+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:42:28.178103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.710890+0000","last_change":"2023-05-20T01:26:36.099640+0000","last_active":"2023-05-20T01:26:38.710890+0000","last_peered":"2023-05-20T01:26:38.710890+0000","last_clean":"2023-05-20T01:26:38.710890+0000","last_became_active":"2023-05-20T01:26:36.099373+0000","last_became_peered":"2023-05-20T01:26:36.099373+0000","last_unstale":"2023-05-20T01:26:38.710890+0000","last_undegraded":"2023-05-20T01:26:38.710890+0000","last_fullsized":"2023-05-20T01:26:38.710890+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:10:31.286889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:38.711081+0000","last_change":"2023-05-20T01:26:35.660641+0000","last_active":"2023-05-20T01:26:38.711081+0000","last_peered":"2023-05-20T01:26:38.711081+0000","last_clean":"2023-05-20T01:26:38.711081+0000","last_became_active":"2023-05-20T01:26:35.659463+0000","last_became_peered":"2023-05-20T01:26:35.659463+0000","last_unstale":"2023-05-20T01:26:38.711081+0000","last_undegraded":"2023-05-20T01:26:38.711081+0000","last_fullsized":"2023-05-20T01:26:38.711081+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:02:37.953812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":181,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:37.085836+0000","last_change":"2023-05-20T01:16:23.290341+0000","last_active":"2023-05-20T01:26:37.085836+0000","last_peered":"2023-05-20T01:26:37.085836+0000","last_clean":"2023-05-20T01:26:37.085836+0000","last_became_active":"2023-05-20T01:16:23.290053+0000","last_became_peered":"2023-05-20T01:16:23.290053+0000","last_unstale":"2023-05-20T01:26:37.085836+0000","last_undegraded":"2023-05-20T01:26:37.085836+0000","last_fullsized":"2023-05-20T01:26:37.085836+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:45:41.696310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:37.079710+0000","last_change":"2023-05-20T01:26:36.094364+0000","last_active":"2023-05-20T01:26:37.079710+0000","last_peered":"2023-05-20T01:26:37.079710+0000","last_clean":"2023-05-20T01:26:37.079710+0000","last_became_active":"2023-05-20T01:26:36.093797+0000","last_became_peered":"2023-05-20T01:26:36.093797+0000","last_unstale":"2023-05-20T01:26:37.079710+0000","last_undegraded":"2023-05-20T01:26:37.079710+0000","last_fullsized":"2023-05-20T01:26:37.079710+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:35.634998+0000","last_clean_scrub_stamp":"2023-05-20T01:26:35.634998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:25:35.442465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:37.085298+0000","last_change":"2023-05-20T01:26:35.654858+0000","last_active":"2023-05-20T01:26:37.085298+0000","last_peered":"2023-05-20T01:26:37.085298+0000","last_clean":"2023-05-20T01:26:37.085298+0000","last_became_active":"2023-05-20T01:26:35.654547+0000","last_became_peered":"2023-05-20T01:26:35.654547+0000","last_unstale":"2023-05-20T01:26:37.085298+0000","last_undegraded":"2023-05-20T01:26:37.085298+0000","last_fullsized":"2023-05-20T01:26:37.085298+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:26:34.630120+0000","last_clean_scrub_stamp":"2023-05-20T01:26:34.630120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:20:58.771107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":181,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-05-20T01:26:37.085991+0000","last_change":"2023-05-20T01:16:22.834330+0000","last_active":"2023-05-20T01:26:37.085991+0000","last_peered":"2023-05-20T01:26:37.085991+0000","last_clean":"2023-05-20T01:26:37.085991+0000","last_became_active":"2023-05-20T01:16:22.833946+0000","last_became_peered":"2023-05-20T01:16:22.833946+0000","last_unstale":"2023-05-20T01:26:37.085991+0000","last_undegraded":"2023-05-20T01:26:37.085991+0000","last_fullsized":"2023-05-20T01:26:37.085991+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:04:24.045534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574976,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":16604,"kb_used_data":1940,"kb_used_omap":0,"kb_used_meta":14656,"kb_avail":94355236,"statfs":{"total":96636764160,"available":96619761664,"internally_reserved":0,"allocated":1986560,"data_stored":1301348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":15007744},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:26:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.28000000000000003,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.748,"5min":0.80600000000000005,"15min":0.88200000000000001},"last":0.29399999999999998},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.52900000000000003,"15min":0.54800000000000004},"min":{"1min":0.38900000000000001,"5min":0.309,"15min":0.30599999999999999},"max":{"1min":0.71399999999999997,"5min":0.77100000000000002,"15min":0.90000000000000002},"last":0.44800000000000001}]},{"osd":1,"last update":"Sat May 20 01:26:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.53100000000000003,"15min":0.69899999999999995},"min":{"1min":0.41599999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.70699999999999996,"5min":0.95599999999999996,"15min":8.8879999999999999},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.54800000000000004,"15min":0.73099999999999998},"min":{"1min":0.36899999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.753,"5min":0.85399999999999998,"15min":8.875},"last":0.57099999999999995}]},{"osd":2,"last update":"Sat May 20 01:26:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.52200000000000002,"15min":0.68500000000000005},"min":{"1min":0.41099999999999998,"5min":0.315,"15min":0.315},"max":{"1min":0.75700000000000001,"5min":0.79200000000000004,"15min":8.9120000000000008},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.53100000000000003,"15min":0.68999999999999995},"min":{"1min":0.46200000000000002,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":8.8520000000000003},"last":0.41099999999999998}]},{"osd":3,"last update":"Sat May 20 01:26:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.51900000000000002,"15min":0.72999999999999998},"min":{"1min":0.441,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.65800000000000003,"5min":0.89900000000000002,"15min":10.884},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.56899999999999995,"15min":0.78200000000000003},"min":{"1min":0.39400000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.78000000000000003,"5min":0.88100000000000001,"15min":10.952},"last":0.52800000000000002}]},{"osd":4,"last update":"Sat May 20 01:26:38 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.502,"15min":0.51000000000000001},"min":{"1min":0.34999999999999998,"5min":0.313,"15min":0.313},"max":{"1min":0.77600000000000002,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.52700000000000002,"15min":0.53000000000000003},"min":{"1min":0.371,"5min":0.309,"15min":0.309},"max":{"1min":0.71399999999999997,"5min":0.82599999999999996,"15min":0.875},"last":0.432}]},{"osd":5,"last update":"Sat May 20 01:26:38 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.55100000000000005,"15min":0.54600000000000004},"min":{"1min":0.35199999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.72699999999999998,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.57099999999999995,"15min":0.57699999999999996},"min":{"1min":0.40400000000000003,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.83899999999999997,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.58799999999999997}]},{"osd":6,"last update":"Sat May 20 01:26:38 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.58999999999999997,"15min":0.59599999999999997},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.378},"max":{"1min":0.78800000000000003,"5min":1.028,"15min":1.028},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.56799999999999995,"15min":0.58299999999999996},"min":{"1min":0.42699999999999999,"5min":0.377,"15min":0.377},"max":{"1min":0.77100000000000002,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.55700000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574975,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":11300,"kb_used_data":1372,"kb_used_omap":0,"kb_used_meta":9920,"kb_avail":94360540,"statfs":{"total":96636764160,"available":96625192960,"internally_reserved":0,"allocated":1404928,"data_stored":712988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":10158080},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:25:42 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.44800000000000001,"15min":0.47699999999999998},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.26900000000000002},"max":{"1min":0.60999999999999999,"5min":0.72699999999999998,"15min":0.77200000000000002},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.47899999999999998,"15min":0.51700000000000002},"min":{"1min":0.32900000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.41099999999999998}]},{"osd":1,"last update":"Sat May 20 01:25:42 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.48699999999999999,"15min":0.51900000000000002},"min":{"1min":0.32800000000000001,"5min":0.29699999999999999,"15min":0.28000000000000003},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.83799999999999997},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.51700000000000002,"15min":0.52600000000000002},"min":{"1min":0.379,"5min":0.32600000000000001,"15min":0.318},"max":{"1min":0.66500000000000004,"5min":0.749,"15min":0.79200000000000004},"last":0.63400000000000001}]},{"osd":2,"last update":"Sat May 20 01:25:42 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.48399999999999999,"15min":0.503},"min":{"1min":0.35099999999999998,"5min":0.34399999999999997,"15min":0.33700000000000002},"max":{"1min":0.57099999999999995,"5min":0.72199999999999998,"15min":0.76600000000000001},"last":0.37},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.51600000000000001,"15min":0.54900000000000004},"min":{"1min":0.37,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.745,"5min":0.76200000000000001,"15min":0.76800000000000002},"last":0.54200000000000004}]},{"osd":3,"last update":"Sat May 20 01:25:42 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.52500000000000002,"15min":0.53600000000000003},"min":{"1min":0.378,"5min":0.311,"15min":0.311},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.52000000000000002,"15min":0.53900000000000003},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.72699999999999998,"5min":0.73199999999999998,"15min":0.79500000000000004},"last":0.52800000000000002}]},{"osd":4,"last update":"Sat May 20 01:25:42 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.53100000000000003,"15min":0.54400000000000004},"min":{"1min":0.34799999999999998,"5min":0.33300000000000002,"15min":0.32600000000000001},"max":{"1min":0.70699999999999996,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.54000000000000004,"15min":0.56299999999999994},"min":{"1min":0.40500000000000003,"5min":0.38100000000000001,"15min":0.32700000000000001},"max":{"1min":0.71799999999999997,"5min":0.77400000000000002,"15min":0.80100000000000005},"last":0.68200000000000005}]},{"osd":5,"last update":"Sat May 20 01:25:42 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.53400000000000003,"15min":0.54000000000000004},"min":{"1min":0.38,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.63600000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.56499999999999995,"15min":0.59799999999999998},"min":{"1min":0.372,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.64400000000000002,"5min":0.81599999999999995,"15min":0.89400000000000002},"last":0.61699999999999999}]},{"osd":7,"last update":"Sat May 20 01:25:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.57099999999999995,"15min":0.59399999999999997},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.71299999999999997,"5min":0.80200000000000005,"15min":0.92900000000000005},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.57599999999999996,"15min":0.60499999999999998},"min":{"1min":0.45100000000000001,"5min":0.379,"15min":0.379},"max":{"1min":0.66600000000000004,"5min":0.82799999999999996,"15min":0.85199999999999998},"last":0.56599999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574975,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":17496,"kb_used_data":1936,"kb_used_omap":0,"kb_used_meta":15552,"kb_avail":94354344,"statfs":{"total":96636764160,"available":96618848256,"internally_reserved":0,"allocated":1982464,"data_stored":1301222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":15925248},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:25:36 2023","interfaces":[{"interface":"back","average":{"1min":0.30599999999999999,"5min":0.28299999999999997,"15min":0.311},"min":{"1min":0.18099999999999999,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":0.48799999999999999,"5min":0.57099999999999995,"15min":1.194},"last":0.249},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.32400000000000001,"15min":0.31900000000000001},"min":{"1min":0.23899999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.75800000000000001},"last":0.33400000000000002}]},{"osd":2,"last update":"Sat May 20 01:25:36 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.307,"15min":0.31900000000000001},"min":{"1min":0.23100000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.66400000000000003},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.374,"15min":0.40400000000000003},"min":{"1min":0.313,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":1.3580000000000001},"last":0.372}]},{"osd":3,"last update":"Sat May 20 01:25:36 2023","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.35899999999999999,"15min":0.371},"min":{"1min":0.27400000000000002,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":1.619},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.35299999999999998,"15min":0.379},"min":{"1min":0.25800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.57699999999999996,"5min":0.60899999999999999,"15min":1.8400000000000001},"last":0.36399999999999999}]},{"osd":4,"last update":"Sat May 20 01:25:36 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.55600000000000005,"15min":0.57899999999999996},"min":{"1min":0.36799999999999999,"5min":0.34899999999999998,"15min":0.32200000000000001},"max":{"1min":0.76100000000000001,"5min":0.81699999999999995,"15min":1.2769999999999999},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.57099999999999995,"15min":0.58299999999999996},"min":{"1min":0.38,"5min":0.312,"15min":0.312},"max":{"1min":0.81599999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.44800000000000001}]},{"osd":5,"last update":"Sat May 20 01:25:36 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.58699999999999997,"15min":0.59099999999999997},"min":{"1min":0.46600000000000003,"5min":0.37,"15min":0.37},"max":{"1min":0.78500000000000003,"5min":0.81899999999999995,"15min":1.4510000000000001},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.58199999999999996,"15min":0.60299999999999998},"min":{"1min":0.39700000000000002,"5min":0.39100000000000001,"15min":0.378},"max":{"1min":0.76800000000000002,"5min":0.78800000000000003,"15min":1.409},"last":0.45900000000000002}]},{"osd":6,"last update":"Sat May 20 01:25:36 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.60299999999999998},"min":{"1min":0.39000000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.82299999999999995,"5min":0.876,"15min":1.575},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.58199999999999996,"15min":0.59399999999999997},"min":{"1min":0.45400000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":1.0600000000000001},"last":0.52700000000000002}]},{"osd":7,"last update":"Sat May 20 01:25:36 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.60399999999999998},"min":{"1min":0.41099999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":1.496},"last":0.435},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.63,"15min":0.64000000000000001},"min":{"1min":0.44500000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.5009999999999999,"5min":2.1709999999999998,"15min":2.1709999999999998},"last":0.51000000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574975,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":11356,"kb_used_data":1364,"kb_used_omap":0,"kb_used_meta":9984,"kb_avail":94360484,"statfs":{"total":96636764160,"available":96625135616,"internally_reserved":0,"allocated":1396736,"data_stored":711513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":10223616},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat May 20 01:26:36 2023","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.32700000000000001,"15min":0.32500000000000001},"min":{"1min":0.28699999999999998,"5min":0.17999999999999999,"15min":0.17999999999999999},"max":{"1min":0.90800000000000003,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.36299999999999999,"15min":0.35399999999999998},"min":{"1min":0.23400000000000001,"5min":0.20799999999999999,"15min":0.20399999999999999},"max":{"1min":2.7559999999999998,"5min":2.7559999999999998,"15min":2.7559999999999998},"last":0.38500000000000001}]},{"osd":2,"last update":"Sat May 20 01:26:36 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.35599999999999998,"15min":0.34200000000000003},"min":{"1min":0.252,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":3.3079999999999998,"5min":3.3079999999999998,"15min":3.3079999999999998},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.45000000000000001,"15min":0.40899999999999997},"min":{"1min":0.314,"5min":0.22500000000000001,"15min":0.216},"max":{"1min":5.7270000000000003,"5min":5.7270000000000003,"15min":5.7270000000000003},"last":0.29899999999999999}]},{"osd":3,"last update":"Sat May 20 01:26:36 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.39500000000000002,"15min":0.376},"min":{"1min":0.27100000000000002,"5min":0.22,"15min":0.22},"max":{"1min":0.82399999999999995,"5min":4.258,"15min":4.258},"last":0.437},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.40100000000000002,"15min":0.38},"min":{"1min":0.30199999999999999,"5min":0.245,"15min":0.245},"max":{"1min":2.714,"5min":2.714,"15min":2.714},"last":0.41999999999999998}]},{"osd":4,"last update":"Sat May 20 01:26:36 2023","interfaces":[{"interface":"back","average":{"1min":1.383,"5min":0.74099999999999999,"15min":0.61099999999999999},"min":{"1min":0.46100000000000002,"5min":0.372,"15min":0.34399999999999997},"max":{"1min":13.125,"5min":13.125,"15min":13.125},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.624,"15min":0.58699999999999997},"min":{"1min":0.50600000000000001,"5min":0.39200000000000002,"15min":0.36299999999999999},"max":{"1min":1.9930000000000001,"5min":1.9930000000000001,"15min":1.9930000000000001},"last":0.59899999999999998}]},{"osd":5,"last update":"Sat May 20 01:26:36 2023","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":0.64600000000000002,"15min":0.59999999999999998},"min":{"1min":0.49199999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":5.3620000000000001,"5min":5.3620000000000001,"15min":5.3620000000000001},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.68100000000000005,"15min":0.622},"min":{"1min":0.46899999999999997,"5min":0.374,"15min":0.374},"max":{"1min":5.7999999999999998,"5min":5.7999999999999998,"15min":5.7999999999999998},"last":0.67000000000000004}]},{"osd":6,"last update":"Sat May 20 01:26:36 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.73599999999999999,"15min":0.64100000000000001},"min":{"1min":0.45700000000000002,"5min":0.36899999999999999,"15min":0.35199999999999998},"max":{"1min":3.3460000000000001,"5min":10.042999999999999,"15min":10.042999999999999},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.63400000000000001,"15min":0.624},"min":{"1min":0.46400000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.94099999999999995,"5min":1.367,"15min":1.367},"last":0.78700000000000003}]},{"osd":7,"last update":"Sat May 20 01:26:36 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.65100000000000002,"15min":0.61299999999999999},"min":{"1min":0.51900000000000002,"5min":0.41099999999999998,"15min":0.39600000000000002},"max":{"1min":2.802,"5min":2.802,"15min":2.802},"last":0.62},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.76000000000000001,"15min":0.66800000000000004},"min":{"1min":0.55200000000000005,"5min":0.439,"15min":0.42099999999999999},"max":{"1min":1.948,"5min":10.085000000000001,"15min":10.085000000000001},"last":0.74399999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574975,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":11348,"kb_used_data":1356,"kb_used_omap":0,"kb_used_meta":9984,"kb_avail":94360492,"statfs":{"total":96636764160,"available":96625143808,"internally_reserved":0,"allocated":1388544,"data_stored":710854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":10223616},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.33700000000000002,"5min":0.28299999999999997,"15min":0.32300000000000001},"min":{"1min":0.224,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":0.50900000000000001,"5min":0.53400000000000003,"15min":1.038},"last":0.28899999999999998},{"interface":"front","average":{"1min":0.32300000000000001,"5min":0.29899999999999999,"15min":0.33900000000000002},"min":{"1min":0.21199999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":1.079},"last":0.312}]},{"osd":1,"last update":"Sat May 20 01:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.317,"15min":0.376},"min":{"1min":0.253,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.44500000000000001,"5min":0.61399999999999999,"15min":1.4950000000000001},"last":0.37},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.35899999999999999,"15min":0.41299999999999998},"min":{"1min":0.28399999999999997,"5min":0.249,"15min":0.249},"max":{"1min":0.60799999999999998,"5min":0.629,"15min":1.5189999999999999},"last":0.35699999999999998}]},{"osd":3,"last update":"Sat May 20 01:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.32900000000000001,"15min":0.378},"min":{"1min":0.246,"5min":0.218,"15min":0.216},"max":{"1min":0.57899999999999996,"5min":0.60599999999999998,"15min":1.2090000000000001},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.36899999999999999,"15min":0.42699999999999999},"min":{"1min":0.32400000000000001,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.59499999999999997,"5min":0.68100000000000005,"15min":1.5620000000000001},"last":0.443}]},{"osd":4,"last update":"Sat May 20 01:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.56799999999999995,"15min":0.58099999999999996},"min":{"1min":0.46500000000000002,"5min":0.35899999999999999,"15min":0.34899999999999998},"max":{"1min":1.107,"5min":1.107,"15min":1.319},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.59399999999999997,"15min":0.61299999999999999},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":1.4319999999999999},"last":0.56100000000000005}]},{"osd":5,"last update":"Sat May 20 01:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.56799999999999995,"15min":0.61699999999999999},"min":{"1min":0.443,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.82199999999999995,"5min":0.95599999999999996,"15min":1.758},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.58899999999999997,"15min":0.63500000000000001},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.004,"5min":1.004,"15min":1.7110000000000001},"last":0.54200000000000004}]},{"osd":6,"last update":"Sat May 20 01:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.59099999999999997,"15min":0.63200000000000001},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.069,"5min":1.069,"15min":1.665},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.58499999999999996,"15min":0.628},"min":{"1min":0.39900000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.88,"5min":0.93500000000000005,"15min":1.819},"last":0.63900000000000001}]},{"osd":7,"last update":"Sat May 20 01:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.59999999999999998,"15min":0.65200000000000002},"min":{"1min":0.39000000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":1.7310000000000001},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.60999999999999999,"15min":0.65100000000000002},"min":{"1min":0.47799999999999998,"5min":0.42399999999999999,"15min":0.42199999999999999},"max":{"1min":1.038,"5min":1.038,"15min":1.8460000000000001},"last":0.67400000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574975,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":15700,"kb_used_data":1356,"kb_used_omap":0,"kb_used_meta":14336,"kb_avail":94356140,"statfs":{"total":96636764160,"available":96620687360,"internally_reserved":0,"allocated":1388544,"data_stored":711402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":14680064},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:26:34 2023","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.316,"15min":0.31900000000000001},"min":{"1min":0.24099999999999999,"5min":0.20100000000000001,"15min":0.188},"max":{"1min":0.54100000000000004,"5min":0.65000000000000002,"15min":1.528},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.34200000000000003,"15min":0.40799999999999997},"min":{"1min":0.26500000000000001,"5min":0.20799999999999999,"15min":0.19500000000000001},"max":{"1min":0.54200000000000004,"5min":0.72399999999999998,"15min":3.1890000000000001},"last":0.42599999999999999}]},{"osd":1,"last update":"Sat May 20 01:26:34 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.32500000000000001,"15min":0.40000000000000002},"min":{"1min":0.22700000000000001,"5min":0.189,"15min":0.189},"max":{"1min":0.55900000000000005,"5min":0.73199999999999998,"15min":3.4260000000000002},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.437,"5min":0.372,"15min":0.36099999999999999},"min":{"1min":0.27700000000000002,"5min":0.23100000000000001,"15min":0.221},"max":{"1min":0.60999999999999999,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.59299999999999997}]},{"osd":2,"last update":"Sat May 20 01:26:34 2023","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.36399999999999999,"15min":0.38400000000000001},"min":{"1min":0.30399999999999999,"5min":0.24299999999999999,"15min":0.224},"max":{"1min":0.59399999999999997,"5min":0.67400000000000004,"15min":1.1619999999999999},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.40000000000000002,"15min":0.41299999999999998},"min":{"1min":0.26900000000000002,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.75,"5min":0.78500000000000003,"15min":1.413},"last":0.48699999999999999}]},{"osd":4,"last update":"Sat May 20 01:26:34 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.56999999999999995,"15min":0.58299999999999996},"min":{"1min":0.45800000000000002,"5min":0.38700000000000001,"15min":0.38200000000000001},"max":{"1min":0.77000000000000002,"5min":0.95399999999999996,"15min":1.0700000000000001},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.59199999999999997,"15min":0.66100000000000003},"min":{"1min":0.42399999999999999,"5min":0.40200000000000002,"15min":0.35899999999999999},"max":{"1min":0.88500000000000001,"5min":0.88600000000000001,"15min":3.4660000000000002},"last":1.339}]},{"osd":5,"last update":"Sat May 20 01:26:34 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.59099999999999997,"15min":0.66600000000000004},"min":{"1min":0.503,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":3.5379999999999998},"last":1.274},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.60799999999999998,"15min":0.626},"min":{"1min":0.51400000000000001,"5min":0.41999999999999998,"15min":0.374},"max":{"1min":0.91300000000000003,"5min":1.026,"15min":1.385},"last":0.64200000000000002}]},{"osd":6,"last update":"Sat May 20 01:26:34 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.60699999999999998,"15min":0.629},"min":{"1min":0.44400000000000001,"5min":0.41199999999999998,"15min":0.38100000000000001},"max":{"1min":0.873,"5min":0.99199999999999999,"15min":1.347},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.60899999999999999,"15min":0.626},"min":{"1min":0.438,"5min":0.374,"15min":0.374},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":1.107},"last":0.61299999999999999}]},{"osd":7,"last update":"Sat May 20 01:26:34 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.60999999999999999,"15min":0.63200000000000001},"min":{"1min":0.46700000000000003,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.92200000000000004,"5min":0.92900000000000005,"15min":1.1359999999999999},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.63700000000000001,"15min":0.70699999999999996},"min":{"1min":0.52300000000000002,"5min":0.436,"15min":0.436},"max":{"1min":0.92300000000000004,"5min":1.0069999999999999,"15min":3.5009999999999999},"last":1.3120000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574975,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":16792,"kb_used_data":1360,"kb_used_omap":0,"kb_used_meta":15424,"kb_avail":94355048,"statfs":{"total":96636764160,"available":96619569152,"internally_reserved":0,"allocated":1392640,"data_stored":710912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":15794176},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:26:36 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.497,"15min":0.48999999999999999},"min":{"1min":0.313,"5min":0.26900000000000002,"15min":0.247},"max":{"1min":0.71999999999999997,"5min":0.73699999999999999,"15min":0.75600000000000001},"last":0.313},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.52200000000000002,"15min":0.504},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.76900000000000002,"5min":1.6830000000000001,"15min":1.7989999999999999},"last":0.47299999999999998}]},{"osd":1,"last update":"Sat May 20 01:26:36 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.51700000000000002,"15min":0.52100000000000002},"min":{"1min":0.374,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.82799999999999996},"last":0.504},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.52800000000000002,"15min":0.53000000000000003},"min":{"1min":0.371,"5min":0.316,"15min":0.316},"max":{"1min":0.747,"5min":1.7190000000000001,"15min":2.0739999999999998},"last":0.371}]},{"osd":2,"last update":"Sat May 20 01:26:36 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.55000000000000004,"15min":0.51800000000000002},"min":{"1min":0.35799999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.71399999999999997,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.55000000000000004,"15min":0.55100000000000005},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.34000000000000002},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.94699999999999995}]},{"osd":3,"last update":"Sat May 20 01:26:36 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.53900000000000003,"15min":0.52700000000000002},"min":{"1min":0.39000000000000001,"5min":0.34899999999999998,"15min":0.34399999999999997},"max":{"1min":0.72199999999999998,"5min":0.81399999999999995,"15min":1.827},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.58599999999999997,"15min":0.63700000000000001},"min":{"1min":0.45800000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.78300000000000003,"5min":1.093,"15min":3.1749999999999998},"last":0.45800000000000002}]},{"osd":5,"last update":"Sat May 20 01:26:36 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.52000000000000002,"15min":0.49099999999999999},"min":{"1min":0.36899999999999999,"5min":0.308,"15min":0.29199999999999998},"max":{"1min":0.63100000000000001,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.499,"5min":0.53700000000000003,"15min":0.51400000000000001},"min":{"1min":0.34399999999999997,"5min":0.33400000000000002,"15min":0.29899999999999999},"max":{"1min":0.69399999999999995,"5min":0.80400000000000005,"15min":0.80500000000000005},"last":0.34399999999999997}]},{"osd":6,"last update":"Sat May 20 01:26:36 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.57999999999999996,"15min":0.55100000000000005},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.32400000000000001},"max":{"1min":0.64900000000000002,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.60299999999999998,"15min":0.59199999999999997},"min":{"1min":0.48099999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.81200000000000006,"5min":0.84699999999999998,"15min":0.85899999999999999},"last":0.48499999999999999}]},{"osd":7,"last update":"Sat May 20 01:26:36 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.57699999999999996,"15min":0.55800000000000005},"min":{"1min":0.436,"5min":0.36099999999999999,"15min":0.34200000000000003},"max":{"1min":0.70999999999999996,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.442},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.61499999999999999,"15min":0.59799999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.34000000000000002},"max":{"1min":0.69499999999999995,"5min":1.034,"15min":1.034},"last":0.38100000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574976,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":11276,"kb_used_data":1348,"kb_used_omap":0,"kb_used_meta":9920,"kb_avail":94360564,"statfs":{"total":96636764160,"available":96625217536,"internally_reserved":0,"allocated":1380352,"data_stored":709859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":10158080},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:26:40 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.47499999999999998,"15min":0.61299999999999999},"min":{"1min":0.33300000000000002,"5min":0.29999999999999999,"15min":0.23300000000000001},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":6.3929999999999998},"last":0.503},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.48899999999999999,"15min":0.63600000000000001},"min":{"1min":0.34799999999999998,"5min":0.315,"15min":0.26900000000000002},"max":{"1min":0.80900000000000005,"5min":0.83699999999999997,"15min":6.3339999999999996},"last":0.59799999999999998}]},{"osd":1,"last update":"Sat May 20 01:26:40 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.48999999999999999,"15min":0.56499999999999995},"min":{"1min":0.36099999999999999,"5min":0.313,"15min":0.29299999999999998},"max":{"1min":0.68000000000000005,"5min":0.72299999999999998,"15min":3.2450000000000001},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.52500000000000002,"15min":0.53400000000000003},"min":{"1min":0.33200000000000002,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":1.1819999999999999},"last":0.45500000000000002}]},{"osd":2,"last update":"Sat May 20 01:26:40 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.51000000000000001,"15min":0.53600000000000003},"min":{"1min":0.40000000000000002,"5min":0.33200000000000002,"15min":0.30399999999999999},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":1.1639999999999999},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.51700000000000002,"15min":0.54200000000000004},"min":{"1min":0.42499999999999999,"5min":0.308,"15min":0.29399999999999998},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":1.258},"last":0.57899999999999996}]},{"osd":3,"last update":"Sat May 20 01:26:40 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.54700000000000004,"15min":0.68200000000000005},"min":{"1min":0.378,"5min":0.35099999999999998,"15min":0.33000000000000002},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":6.351},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.56200000000000006,"15min":0.70999999999999996},"min":{"1min":0.42899999999999999,"5min":0.374,"15min":0.33400000000000002},"max":{"1min":0.745,"5min":0.79000000000000004,"15min":6.3730000000000002},"last":0.55700000000000005}]},{"osd":4,"last update":"Sat May 20 01:26:40 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.52100000000000002,"15min":0.497},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.55400000000000005,"15min":0.55100000000000005},"min":{"1min":0.35699999999999998,"5min":0.35399999999999998,"15min":0.34200000000000003},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.64700000000000002}]},{"osd":6,"last update":"Sat May 20 01:26:40 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.29999999999999999},"max":{"1min":0.77100000000000002,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.623},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.38700000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.71699999999999997}]},{"osd":7,"last update":"Sat May 20 01:26:40 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58299999999999996,"15min":0.58099999999999996},"min":{"1min":0.41599999999999998,"5min":0.33400000000000002,"15min":0.33000000000000002},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.91000000000000003},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.60099999999999998,"15min":0.60699999999999998},"min":{"1min":0.41499999999999998,"5min":0.40999999999999998,"15min":0.39500000000000002},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.94499999999999995},"last":0.65900000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-05-20T01:26:45.264 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-05-20T01:26:45.265 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-05-20T01:26:45.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.426+0000 7f858d1eb700 1 Processor -- start 2023-05-20T01:26:45.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.426+0000 7f858d1eb700 1 -- start start 2023-05-20T01:26:45.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.427+0000 7f858d1eb700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f85880cd8a0 0x7f8588071810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:45.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.427+0000 7f858d1eb700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85880cd410 con 0x7f858815d0b0 2023-05-20T01:26:45.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.427+0000 7f858d1eb700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85880cbd70 con 0x7f85880cc120 2023-05-20T01:26:45.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.427+0000 7f858d1eb700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8588071d50 con 0x7f85880cd8a0 2023-05-20T01:26:45.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.427+0000 7f858759e700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f858815d0b0 0x7f858815d550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49768/0 (socket says 172.21.15.16:49768) 2023-05-20T01:26:45.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.427+0000 7f858759e700 1 -- 172.21.15.16:0/3767513623 learned_addr learned my addr 172.21.15.16:0/3767513623 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:26:45.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.427+0000 7f858659c700 1 --2- 172.21.15.16:0/3767513623 >> v2:172.21.15.132:3300/0 conn(0x7f85880cd8a0 0x7f8588071810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:45.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.427+0000 7f8585d9b700 1 -- 172.21.15.16:0/3767513623 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 391754103 0 0) 0x7f85880cd410 con 0x7f858815d0b0 2023-05-20T01:26:45.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.427+0000 7f8585d9b700 1 -- 172.21.15.16:0/3767513623 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f85700035b0 con 0x7f858815d0b0 2023-05-20T01:26:45.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.427+0000 7f8585d9b700 1 -- 172.21.15.16:0/3767513623 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1499665861 0 0) 0x7f85700035b0 con 0x7f858815d0b0 2023-05-20T01:26:45.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.427+0000 7f8585d9b700 1 -- 172.21.15.16:0/3767513623 >> v2:172.21.15.132:3300/0 conn(0x7f85880cd8a0 msgr2=0x7f8588071810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:45.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.427+0000 7f8585d9b700 1 --2- 172.21.15.16:0/3767513623 >> v2:172.21.15.132:3300/0 conn(0x7f85880cd8a0 0x7f8588071810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:45.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.427+0000 7f8585d9b700 1 -- 172.21.15.16:0/3767513623 >> v1:172.21.15.132:6789/0 conn(0x7f85880cc120 legacy=0x7f85880cc4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:45.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.427+0000 7f8585d9b700 1 -- 172.21.15.16:0/3767513623 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85880cdfa0 con 0x7f858815d0b0 2023-05-20T01:26:45.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.427+0000 7f8585d9b700 1 -- 172.21.15.16:0/3767513623 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7f85800019f0 con 0x7f858815d0b0 2023-05-20T01:26:45.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.427+0000 7f8585d9b700 1 -- 172.21.15.16:0/3767513623 >> v1:172.21.15.16:6789/0 conn(0x7f858815d0b0 legacy=0x7f858815d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:45.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.427+0000 7f8585d9b700 1 --2- 172.21.15.16:0/3767513623 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8570003a00 0x7f8570003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:45.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.427+0000 7f8585d9b700 1 --2- 172.21.15.16:0/3767513623 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8570004850 0x7f8570004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:45.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.427+0000 7f8585d9b700 1 --2- 172.21.15.16:0/3767513623 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8570005200 0x7f857000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:45.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.427+0000 7f8585d9b700 1 -- 172.21.15.16:0/3767513623 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8588071d50 con 0x7f8570003a00 2023-05-20T01:26:45.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.427+0000 7f8585d9b700 1 -- 172.21.15.16:0/3767513623 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f857000c080 con 0x7f8570005200 2023-05-20T01:26:45.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.427+0000 7f8585d9b700 1 -- 172.21.15.16:0/3767513623 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f857000c240 con 0x7f8570004850 2023-05-20T01:26:45.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.428+0000 7f8586d9d700 1 --2- 172.21.15.16:0/3767513623 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8570003a00 0x7f8570003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:45.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.428+0000 7f858759e700 1 --2- 172.21.15.16:0/3767513623 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8570005200 0x7f857000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:45.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.428+0000 7f858659c700 1 --2- 172.21.15.16:0/3767513623 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8570004850 0x7f8570004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:45.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.428+0000 7f8586d9d700 1 -- 172.21.15.16:0/3767513623 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8570004850 msgr2=0x7f8570004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:45.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.428+0000 7f8586d9d700 1 --2- 172.21.15.16:0/3767513623 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8570004850 0x7f8570004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:45.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.428+0000 7f8586d9d700 1 -- 172.21.15.16:0/3767513623 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8570005200 msgr2=0x7f857000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:45.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.428+0000 7f8586d9d700 1 --2- 172.21.15.16:0/3767513623 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8570005200 0x7f857000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:45.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.428+0000 7f858759e700 1 -- 172.21.15.16:0/3767513623 reap_dead start 2023-05-20T01:26:45.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.428+0000 7f8586d9d700 1 -- 172.21.15.16:0/3767513623 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85700035b0 con 0x7f8570003a00 2023-05-20T01:26:45.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.428+0000 7f858659c700 1 --2- 172.21.15.16:0/3767513623 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8570004850 0x7f8570004cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:26:45.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.428+0000 7f8586d9d700 1 --2- 172.21.15.16:0/3767513623 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8570003a00 0x7f8570003e10 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f85780017c0 tx=0x7f8570005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bf5b3a36ff513c server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:45.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.428+0000 7f8585d9b700 1 -- 172.21.15.16:0/3767513623 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85780052d0 con 0x7f8570003a00 2023-05-20T01:26:45.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.428+0000 7f8585d9b700 1 -- 172.21.15.16:0/3767513623 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8578005450 con 0x7f8570003a00 2023-05-20T01:26:45.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.428+0000 7f8585d9b700 1 -- 172.21.15.16:0/3767513623 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85780038c0 con 0x7f8570003a00 2023-05-20T01:26:45.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.429+0000 7f858d1eb700 1 -- 172.21.15.16:0/3767513623 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8570003a00 msgr2=0x7f8570003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:45.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.429+0000 7f858d1eb700 1 --2- 172.21.15.16:0/3767513623 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8570003a00 0x7f8570003e10 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f85780017c0 tx=0x7f8570005880 comp rx=0 tx=0).stop 2023-05-20T01:26:45.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.429+0000 7f858d1eb700 1 -- 172.21.15.16:0/3767513623 shutdown_connections 2023-05-20T01:26:45.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.429+0000 7f858d1eb700 1 --2- 172.21.15.16:0/3767513623 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8570003a00 0x7f8570003e10 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:45.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.429+0000 7f858d1eb700 1 -- 172.21.15.16:0/3767513623 >> 172.21.15.16:0/3767513623 conn(0x7f858806cbb0 msgr2=0x7f858806cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:45.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.429+0000 7f858d1eb700 1 -- 172.21.15.16:0/3767513623 shutdown_connections 2023-05-20T01:26:45.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.429+0000 7f858d1eb700 1 -- 172.21.15.16:0/3767513623 wait complete. 2023-05-20T01:26:45.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.430+0000 7f858d1eb700 1 Processor -- start 2023-05-20T01:26:45.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.430+0000 7f858d1eb700 1 -- start start 2023-05-20T01:26:45.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.431+0000 7f858d1eb700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8570003a00 0x7f8588142120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:45.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.431+0000 7f858d1eb700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8588142660 0x7f8588140090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:45.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.431+0000 7f858d1eb700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f85881405d0 0x7f85881f2150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:45.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.431+0000 7f858d1eb700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f858813fe20 con 0x7f8588142660 2023-05-20T01:26:45.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.431+0000 7f858d1eb700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8588142ad0 con 0x7f85881405d0 2023-05-20T01:26:45.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.431+0000 7f858d1eb700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8588140a40 con 0x7f8570003a00 2023-05-20T01:26:45.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.431+0000 7f858659c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8588142660 0x7f8588140090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:45.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.431+0000 7f858659c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8588142660 0x7f8588140090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38250/0 (socket says 172.21.15.16:38250) 2023-05-20T01:26:45.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.431+0000 7f858659c700 1 -- 172.21.15.16:0/316906009 learned_addr learned my addr 172.21.15.16:0/316906009 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:26:45.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.431+0000 7f858659c700 1 -- 172.21.15.16:0/316906009 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8570003a00 msgr2=0x7f8588142120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:45.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.431+0000 7f858659c700 1 --2- 172.21.15.16:0/316906009 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8570003a00 0x7f8588142120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:45.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.431+0000 7f858759e700 1 --2- 172.21.15.16:0/316906009 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f85881405d0 0x7f85881f2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:45.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.431+0000 7f858659c700 1 -- 172.21.15.16:0/316906009 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f85881405d0 msgr2=0x7f85881f2150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:45.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.431+0000 7f858659c700 1 --2- 172.21.15.16:0/316906009 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f85881405d0 0x7f85881f2150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:45.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.431+0000 7f858659c700 1 -- 172.21.15.16:0/316906009 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8588072190 con 0x7f8588142660 2023-05-20T01:26:45.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.431+0000 7f858759e700 1 --2- 172.21.15.16:0/316906009 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f85881405d0 0x7f85881f2150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:26:45.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.432+0000 7f858659c700 1 --2- 172.21.15.16:0/316906009 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8588142660 0x7f8588140090 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f857c011570 tx=0x7f857c011930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:45.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.432+0000 7f856f7fe700 1 -- 172.21.15.16:0/316906009 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f857c004220 con 0x7f8588142660 2023-05-20T01:26:45.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.432+0000 7f856f7fe700 1 -- 172.21.15.16:0/316906009 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f857c004380 con 0x7f8588142660 2023-05-20T01:26:45.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.432+0000 7f858d1eb700 1 -- 172.21.15.16:0/316906009 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85880ce280 con 0x7f8588142660 2023-05-20T01:26:45.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.432+0000 7f856f7fe700 1 -- 172.21.15.16:0/316906009 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f857c015730 con 0x7f8588142660 2023-05-20T01:26:45.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.432+0000 7f858d1eb700 1 -- 172.21.15.16:0/316906009 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f858815aa60 con 0x7f8588142660 2023-05-20T01:26:45.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.433+0000 7f858d1eb700 1 -- 172.21.15.16:0/316906009 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85880747b0 con 0x7f8588142660 2023-05-20T01:26:45.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.433+0000 7f856f7fe700 1 -- 172.21.15.16:0/316906009 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f857c015890 con 0x7f8588142660 2023-05-20T01:26:45.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.434+0000 7f856f7fe700 1 --2- 172.21.15.16:0/316906009 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f857406ace0 0x7f857406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:45.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.434+0000 7f856f7fe700 1 -- 172.21.15.16:0/316906009 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f857c0897e0 con 0x7f8588142660 2023-05-20T01:26:45.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.434+0000 7f8586d9d700 1 --2- 172.21.15.16:0/316906009 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f857406ace0 0x7f857406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:45.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.434+0000 7f8586d9d700 1 --2- 172.21.15.16:0/316906009 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f857406ace0 0x7f857406d190 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f8578000c00 tx=0x7f8578011000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:45.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.438+0000 7f856f7fe700 1 -- 172.21.15.16:0/316906009 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f857c054750 con 0x7f8588142660 2023-05-20T01:26:45.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.715+0000 7f858d1eb700 1 -- 172.21.15.16:0/316906009 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f8588073af0 con 0x7f8588142660 2023-05-20T01:26:45.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.719+0000 7f856f7fe700 1 -- 172.21.15.16:0/316906009 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) v1 ==== 190+0+0 (secure 0 0 0) 0x7f857c057d70 con 0x7f8588142660 2023-05-20T01:26:45.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.721+0000 7f858d1eb700 1 -- 172.21.15.16:0/316906009 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f857406ace0 msgr2=0x7f857406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:45.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.721+0000 7f858d1eb700 1 --2- 172.21.15.16:0/316906009 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f857406ace0 0x7f857406d190 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f8578000c00 tx=0x7f8578011000 comp rx=0 tx=0).stop 2023-05-20T01:26:45.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.721+0000 7f858d1eb700 1 -- 172.21.15.16:0/316906009 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8588142660 msgr2=0x7f8588140090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:45.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.721+0000 7f858d1eb700 1 --2- 172.21.15.16:0/316906009 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8588142660 0x7f8588140090 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f857c011570 tx=0x7f857c011930 comp rx=0 tx=0).stop 2023-05-20T01:26:45.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.721+0000 7f858d1eb700 1 -- 172.21.15.16:0/316906009 shutdown_connections 2023-05-20T01:26:45.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.721+0000 7f858d1eb700 1 --2- 172.21.15.16:0/316906009 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f85881405d0 0x7f85881f2150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:45.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.721+0000 7f858d1eb700 1 --2- 172.21.15.16:0/316906009 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f857406ace0 0x7f857406d190 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:45.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.721+0000 7f858d1eb700 1 --2- 172.21.15.16:0/316906009 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8570003a00 0x7f8588142120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:45.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.721+0000 7f858d1eb700 1 --2- 172.21.15.16:0/316906009 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8588142660 0x7f8588140090 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:45.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.721+0000 7f858d1eb700 1 -- 172.21.15.16:0/316906009 >> 172.21.15.16:0/316906009 conn(0x7f858806cbb0 msgr2=0x7f85880cb180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:45.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.721+0000 7f858d1eb700 1 -- 172.21.15.16:0/316906009 shutdown_connections 2023-05-20T01:26:45.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.721+0000 7f858d1eb700 1 -- 172.21.15.16:0/316906009 wait complete. 2023-05-20T01:26:45.727 INFO:teuthology.orchestra.run.smithi016.stderr:updated caps for client.0 2023-05-20T01:26:45.737 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-20T01:26:45.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.858+0000 7fef46d0f700 1 Processor -- start 2023-05-20T01:26:45.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.858+0000 7fef46d0f700 1 -- start start 2023-05-20T01:26:45.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.859+0000 7fef46d0f700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fef4006f450 0x7fef400cf890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:45.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.859+0000 7fef46d0f700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fef4006a510 0x7fef4006a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:45.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.859+0000 7fef46d0f700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef400cc510 con 0x7fef4006d4b0 2023-05-20T01:26:45.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.859+0000 7fef46d0f700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef4006aef0 con 0x7fef4006a510 2023-05-20T01:26:45.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.859+0000 7fef46d0f700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef40071ed0 con 0x7fef4006f450 2023-05-20T01:26:45.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.859+0000 7fef452ac700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fef4006a510 0x7fef4006a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:45.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.859+0000 7fef3ffff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fef4006d4b0 0x7fef40161080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49786/0 (socket says 172.21.15.16:49786) 2023-05-20T01:26:45.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.859+0000 7fef452ac700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fef4006a510 0x7fef4006a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38260/0 (socket says 172.21.15.16:38260) 2023-05-20T01:26:45.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.859+0000 7fef3ffff700 1 -- 172.21.15.16:0/4151576993 learned_addr learned my addr 172.21.15.16:0/4151576993 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:26:45.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.859+0000 7fef44aab700 1 --2- 172.21.15.16:0/4151576993 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fef4006f450 0x7fef400cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:45.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.860+0000 7fef452ac700 1 -- 172.21.15.16:0/4151576993 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fef4006f450 msgr2=0x7fef400cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:45.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.860+0000 7fef452ac700 1 --2- 172.21.15.16:0/4151576993 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fef4006f450 0x7fef400cf890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:45.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.860+0000 7fef452ac700 1 -- 172.21.15.16:0/4151576993 >> v1:172.21.15.16:6789/0 conn(0x7fef4006d4b0 legacy=0x7fef40161080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:45.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.860+0000 7fef452ac700 1 -- 172.21.15.16:0/4151576993 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef400721b0 con 0x7fef4006a510 2023-05-20T01:26:45.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.860+0000 7fef452ac700 1 --2- 172.21.15.16:0/4151576993 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef4006a510 0x7fef4006a9b0 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7fef38009d90 tx=0x7fef3800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11c9f402fc3e65c4 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:45.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.860+0000 7fef3f7fe700 1 -- 172.21.15.16:0/4151576993 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef3800e070 con 0x7fef4006a510 2023-05-20T01:26:45.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.860+0000 7fef3f7fe700 1 -- 172.21.15.16:0/4151576993 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef3800bbd0 con 0x7fef4006a510 2023-05-20T01:26:45.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.860+0000 7fef3f7fe700 1 -- 172.21.15.16:0/4151576993 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef38012430 con 0x7fef4006a510 2023-05-20T01:26:45.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.860+0000 7fef46d0f700 1 -- 172.21.15.16:0/4151576993 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef4006a510 msgr2=0x7fef4006a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:45.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.860+0000 7fef46d0f700 1 --2- 172.21.15.16:0/4151576993 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef4006a510 0x7fef4006a9b0 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7fef38009d90 tx=0x7fef3800b330 comp rx=0 tx=0).stop 2023-05-20T01:26:45.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.861+0000 7fef46d0f700 1 -- 172.21.15.16:0/4151576993 shutdown_connections 2023-05-20T01:26:45.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.861+0000 7fef46d0f700 1 --2- 172.21.15.16:0/4151576993 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef4006a510 0x7fef4006a9b0 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:45.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.861+0000 7fef46d0f700 1 --2- 172.21.15.16:0/4151576993 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fef4006f450 0x7fef400cf890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:45.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.861+0000 7fef46d0f700 1 -- 172.21.15.16:0/4151576993 >> 172.21.15.16:0/4151576993 conn(0x7fef400cffb0 msgr2=0x7fef400d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:45.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.861+0000 7fef46d0f700 1 -- 172.21.15.16:0/4151576993 shutdown_connections 2023-05-20T01:26:45.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.861+0000 7fef46d0f700 1 -- 172.21.15.16:0/4151576993 wait complete. 2023-05-20T01:26:45.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.862+0000 7fef46d0f700 1 Processor -- start 2023-05-20T01:26:45.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.862+0000 7fef46d0f700 1 -- start start 2023-05-20T01:26:45.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.863+0000 7fef46d0f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef4006a510 0x7fef401e5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:45.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.863+0000 7fef46d0f700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fef4006d4b0 0x7fef40158760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:45.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.863+0000 7fef46d0f700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fef4006f450 0x7fef401e32f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:45.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.863+0000 7fef46d0f700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef401e5840 con 0x7fef4006a510 2023-05-20T01:26:45.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.863+0000 7fef46d0f700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef401e70d0 con 0x7fef4006f450 2023-05-20T01:26:45.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.863+0000 7fef46d0f700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef401e6800 con 0x7fef4006d4b0 2023-05-20T01:26:45.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.863+0000 7fef44aab700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef4006a510 0x7fef401e5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:45.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.863+0000 7fef44aab700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef4006a510 0x7fef401e5300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38262/0 (socket says 172.21.15.16:38262) 2023-05-20T01:26:45.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.863+0000 7fef44aab700 1 -- 172.21.15.16:0/266320082 learned_addr learned my addr 172.21.15.16:0/266320082 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:26:45.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.863+0000 7fef452ac700 1 --2- 172.21.15.16:0/266320082 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fef4006f450 0x7fef401e32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:45.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.863+0000 7fef44aab700 1 -- 172.21.15.16:0/266320082 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fef4006d4b0 msgr2=0x7fef40158760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:45.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.863+0000 7fef3ffff700 1 --2- 172.21.15.16:0/266320082 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fef4006d4b0 0x7fef40158760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:45.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.863+0000 7fef44aab700 1 --2- 172.21.15.16:0/266320082 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fef4006d4b0 0x7fef40158760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:45.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.863+0000 7fef44aab700 1 -- 172.21.15.16:0/266320082 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fef4006f450 msgr2=0x7fef401e32f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:45.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.863+0000 7fef44aab700 1 --2- 172.21.15.16:0/266320082 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fef4006f450 0x7fef401e32f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:45.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.863+0000 7fef44aab700 1 -- 172.21.15.16:0/266320082 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef38009a70 con 0x7fef4006a510 2023-05-20T01:26:45.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.863+0000 7fef452ac700 1 --2- 172.21.15.16:0/266320082 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fef4006f450 0x7fef401e32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:26:45.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.863+0000 7fef3ffff700 1 --2- 172.21.15.16:0/266320082 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fef4006d4b0 0x7fef40158760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:26:45.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.863+0000 7fef44aab700 1 --2- 172.21.15.16:0/266320082 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef4006a510 0x7fef401e5300 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7fef3400d830 tx=0x7fef3400dbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:45.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.863+0000 7fef3d7fa700 1 -- 172.21.15.16:0/266320082 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef340125f0 con 0x7fef4006a510 2023-05-20T01:26:45.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.864+0000 7fef3d7fa700 1 -- 172.21.15.16:0/266320082 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef34012750 con 0x7fef4006a510 2023-05-20T01:26:45.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.864+0000 7fef46d0f700 1 -- 172.21.15.16:0/266320082 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef400cbe50 con 0x7fef4006a510 2023-05-20T01:26:45.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.864+0000 7fef3d7fa700 1 -- 172.21.15.16:0/266320082 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef34012b00 con 0x7fef4006a510 2023-05-20T01:26:45.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.864+0000 7fef46d0f700 1 -- 172.21.15.16:0/266320082 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef400cc1a0 con 0x7fef4006a510 2023-05-20T01:26:45.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.865+0000 7fef46d0f700 1 -- 172.21.15.16:0/266320082 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef400cc9a0 con 0x7fef4006a510 2023-05-20T01:26:45.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.865+0000 7fef3d7fa700 1 -- 172.21.15.16:0/266320082 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fef34012c60 con 0x7fef4006a510 2023-05-20T01:26:45.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.865+0000 7fef3d7fa700 1 --2- 172.21.15.16:0/266320082 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fef2406ace0 0x7fef2406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:45.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.866+0000 7fef3d7fa700 1 -- 172.21.15.16:0/266320082 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fef34089970 con 0x7fef4006a510 2023-05-20T01:26:45.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.866+0000 7fef3ffff700 1 --2- 172.21.15.16:0/266320082 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fef2406ace0 0x7fef2406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:45.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.866+0000 7fef3ffff700 1 --2- 172.21.15.16:0/266320082 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fef2406ace0 0x7fef2406d190 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fef401e7c10 tx=0x7fef2c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:45.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:45.870+0000 7fef3d7fa700 1 -- 172.21.15.16:0/266320082 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fef340548e0 con 0x7fef4006a510 2023-05-20T01:26:46.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.113+0000 7fef46d0f700 1 -- 172.21.15.16:0/266320082 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fef400747b0 con 0x7fef4006a510 2023-05-20T01:26:46.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.113+0000 7fef3d7fa700 1 -- 172.21.15.16:0/266320082 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+4676 (secure 0 0 0) 0x7fef34057f00 con 0x7fef4006a510 2023-05-20T01:26:46.118 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:26:46.118 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":78,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7069,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/3619911175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":3619911175},{"type":"v1","addr":"172.21.15.16:6837","nonce":3619911175}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73},{"gid":7081,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/959638003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":959638003},{"type":"v1","addr":"172.21.15.132:6834","nonce":959638003}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73},{"gid":7084,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2596482253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":2596482253},{"type":"v1","addr":"172.21.15.132:6835","nonce":2596482253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-20T01:26:36.067393+0000","modified":"2023-05-20T01:26:39.083145+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7896},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7896":{"gid":7896,"name":"a","rank":0,"incarnation":74,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6836/3765848068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3765848068},{"type":"v1","addr":"172.21.15.16:6836","nonce":3765848068}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-05-20T01:26:46.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.115+0000 7fef46d0f700 1 -- 172.21.15.16:0/266320082 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fef2406ace0 msgr2=0x7fef2406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:46.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.115+0000 7fef46d0f700 1 --2- 172.21.15.16:0/266320082 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fef2406ace0 0x7fef2406d190 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fef401e7c10 tx=0x7fef2c009040 comp rx=0 tx=0).stop 2023-05-20T01:26:46.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.115+0000 7fef46d0f700 1 -- 172.21.15.16:0/266320082 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef4006a510 msgr2=0x7fef401e5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:46.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.115+0000 7fef46d0f700 1 --2- 172.21.15.16:0/266320082 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef4006a510 0x7fef401e5300 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7fef3400d830 tx=0x7fef3400dbf0 comp rx=0 tx=0).stop 2023-05-20T01:26:46.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.115+0000 7fef46d0f700 1 -- 172.21.15.16:0/266320082 shutdown_connections 2023-05-20T01:26:46.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.115+0000 7fef46d0f700 1 --2- 172.21.15.16:0/266320082 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fef4006f450 0x7fef401e32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:46.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.115+0000 7fef46d0f700 1 --2- 172.21.15.16:0/266320082 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef4006a510 0x7fef401e5300 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:46.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.115+0000 7fef46d0f700 1 --2- 172.21.15.16:0/266320082 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fef2406ace0 0x7fef2406d190 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:46.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.115+0000 7fef46d0f700 1 --2- 172.21.15.16:0/266320082 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fef4006d4b0 0x7fef40158760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:46.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.115+0000 7fef46d0f700 1 -- 172.21.15.16:0/266320082 >> 172.21.15.16:0/266320082 conn(0x7fef400cffb0 msgr2=0x7fef400c9930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:46.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.115+0000 7fef46d0f700 1 -- 172.21.15.16:0/266320082 shutdown_connections 2023-05-20T01:26:46.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.115+0000 7fef46d0f700 1 -- 172.21.15.16:0/266320082 wait complete. 2023-05-20T01:26:46.121 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 78 2023-05-20T01:26:46.534 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 78, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-05-20T01:26:36.067393+0000', 'modified': '2023-05-20T01:26:39.083145+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7896}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7896': {'gid': 7896, 'name': 'a', 'rank': 0, 'incarnation': 74, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.16:6836/3765848068', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6834', 'nonce': 3765848068}, {'type': 'v1', 'addr': '172.21.15.16:6836', 'nonce': 3765848068}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-05-20T01:26:46.535 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-05-20T01:26:46.535 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-05-20T01:26:46.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.661+0000 7fd539416700 1 Processor -- start 2023-05-20T01:26:46.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.661+0000 7fd539416700 1 -- start start 2023-05-20T01:26:46.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.661+0000 7fd539416700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd5340b4750 0x7fd5340d2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:46.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.661+0000 7fd539416700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd5340cd970 0x7fd5340cddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:46.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.661+0000 7fd539416700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5340b3b50 con 0x7fd5340cfc50 2023-05-20T01:26:46.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.661+0000 7fd539416700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5340cfad0 con 0x7fd5340cd970 2023-05-20T01:26:46.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.661+0000 7fd539416700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5340d3df0 con 0x7fd5340b4750 2023-05-20T01:26:46.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.662+0000 7fd5337fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd5340cd970 0x7fd5340cddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:46.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.662+0000 7fd532ffd700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fd5340cfc50 0x7fd5340d0020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49798/0 (socket says 172.21.15.16:49798) 2023-05-20T01:26:46.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.662+0000 7fd532ffd700 1 -- 172.21.15.16:0/4117198994 learned_addr learned my addr 172.21.15.16:0/4117198994 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:26:46.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.662+0000 7fd5337fe700 1 --2- 172.21.15.16:0/4117198994 >> v2:172.21.15.16:3300/0 conn(0x7fd5340cd970 0x7fd5340cddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38274/0 (socket says 172.21.15.16:38274) 2023-05-20T01:26:46.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.662+0000 7fd5327fc700 1 --2- 172.21.15.16:0/4117198994 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd5340b4750 0x7fd5340d2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:46.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.662+0000 7fd5337fe700 1 -- 172.21.15.16:0/4117198994 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd5340b4750 msgr2=0x7fd5340d2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:46.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.662+0000 7fd5337fe700 1 --2- 172.21.15.16:0/4117198994 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd5340b4750 0x7fd5340d2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:46.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.662+0000 7fd5337fe700 1 -- 172.21.15.16:0/4117198994 >> v1:172.21.15.16:6789/0 conn(0x7fd5340cfc50 legacy=0x7fd5340d0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:46.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.662+0000 7fd5337fe700 1 -- 172.21.15.16:0/4117198994 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5340d0b90 con 0x7fd5340cd970 2023-05-20T01:26:46.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.662+0000 7fd531ffb700 1 -- 172.21.15.16:0/4117198994 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3155474642 0 0) 0x7fd5340b3b50 con 0x7fd5340cfc50 2023-05-20T01:26:46.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.662+0000 7fd5337fe700 1 --2- 172.21.15.16:0/4117198994 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5340cd970 0x7fd5340cddf0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7fd52c009d90 tx=0x7fd52c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fdd8758524d347c server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:46.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.662+0000 7fd531ffb700 1 -- 172.21.15.16:0/4117198994 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd52c00e070 con 0x7fd5340cd970 2023-05-20T01:26:46.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.662+0000 7fd531ffb700 1 -- 172.21.15.16:0/4117198994 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd52c00bbd0 con 0x7fd5340cd970 2023-05-20T01:26:46.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.662+0000 7fd531ffb700 1 -- 172.21.15.16:0/4117198994 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd52c012430 con 0x7fd5340cd970 2023-05-20T01:26:46.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.663+0000 7fd539416700 1 -- 172.21.15.16:0/4117198994 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5340cd970 msgr2=0x7fd5340cddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:46.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.663+0000 7fd539416700 1 --2- 172.21.15.16:0/4117198994 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5340cd970 0x7fd5340cddf0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7fd52c009d90 tx=0x7fd52c00b330 comp rx=0 tx=0).stop 2023-05-20T01:26:46.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.663+0000 7fd539416700 1 -- 172.21.15.16:0/4117198994 shutdown_connections 2023-05-20T01:26:46.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.663+0000 7fd539416700 1 --2- 172.21.15.16:0/4117198994 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5340cd970 0x7fd5340cddf0 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:46.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.663+0000 7fd539416700 1 --2- 172.21.15.16:0/4117198994 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd5340b4750 0x7fd5340d2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:46.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.663+0000 7fd539416700 1 -- 172.21.15.16:0/4117198994 >> 172.21.15.16:0/4117198994 conn(0x7fd53409c590 msgr2=0x7fd5340d3240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:46.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.663+0000 7fd539416700 1 -- 172.21.15.16:0/4117198994 shutdown_connections 2023-05-20T01:26:46.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.663+0000 7fd539416700 1 -- 172.21.15.16:0/4117198994 wait complete. 2023-05-20T01:26:46.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.664+0000 7fd539416700 1 Processor -- start 2023-05-20T01:26:46.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.664+0000 7fd539416700 1 -- start start 2023-05-20T01:26:46.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.665+0000 7fd539416700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5340b4750 0x7fd53415a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:46.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.665+0000 7fd539416700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd5340cd970 0x7fd5340b0f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:46.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.665+0000 7fd539416700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd5340cfc50 0x7fd5340b1480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:46.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.665+0000 7fd539416700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5340ccf30 con 0x7fd5340b4750 2023-05-20T01:26:46.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.665+0000 7fd539416700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5340b6070 con 0x7fd5340cfc50 2023-05-20T01:26:46.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.665+0000 7fd539416700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5340b6350 con 0x7fd5340cd970 2023-05-20T01:26:46.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.665+0000 7fd532ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5340b4750 0x7fd53415a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:46.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.665+0000 7fd532ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5340b4750 0x7fd53415a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38276/0 (socket says 172.21.15.16:38276) 2023-05-20T01:26:46.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.665+0000 7fd532ffd700 1 -- 172.21.15.16:0/2119272104 learned_addr learned my addr 172.21.15.16:0/2119272104 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:26:46.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.665+0000 7fd5337fe700 1 --2- 172.21.15.16:0/2119272104 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd5340cfc50 0x7fd5340b1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:46.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.665+0000 7fd5327fc700 1 --2- 172.21.15.16:0/2119272104 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd5340cd970 0x7fd5340b0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:46.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.666+0000 7fd532ffd700 1 -- 172.21.15.16:0/2119272104 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd5340cd970 msgr2=0x7fd5340b0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:46.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.666+0000 7fd532ffd700 1 --2- 172.21.15.16:0/2119272104 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd5340cd970 0x7fd5340b0f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:46.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.666+0000 7fd532ffd700 1 -- 172.21.15.16:0/2119272104 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd5340cfc50 msgr2=0x7fd5340b1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:46.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.666+0000 7fd532ffd700 1 --2- 172.21.15.16:0/2119272104 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd5340cfc50 0x7fd5340b1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:46.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.666+0000 7fd532ffd700 1 -- 172.21.15.16:0/2119272104 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd52c009a70 con 0x7fd5340b4750 2023-05-20T01:26:46.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.666+0000 7fd5327fc700 1 --2- 172.21.15.16:0/2119272104 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd5340cd970 0x7fd5340b0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:26:46.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.666+0000 7fd532ffd700 1 --2- 172.21.15.16:0/2119272104 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5340b4750 0x7fd53415a7c0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7fd52400c550 tx=0x7fd52400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:46.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.666+0000 7fd51b7fe700 1 -- 172.21.15.16:0/2119272104 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd524004c80 con 0x7fd5340b4750 2023-05-20T01:26:46.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.666+0000 7fd51b7fe700 1 -- 172.21.15.16:0/2119272104 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd524004e00 con 0x7fd5340b4750 2023-05-20T01:26:46.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.666+0000 7fd539416700 1 -- 172.21.15.16:0/2119272104 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5340b4380 con 0x7fd5340b4750 2023-05-20T01:26:46.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.666+0000 7fd51b7fe700 1 -- 172.21.15.16:0/2119272104 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd524007310 con 0x7fd5340b4750 2023-05-20T01:26:46.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.666+0000 7fd539416700 1 -- 172.21.15.16:0/2119272104 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd53415d0b0 con 0x7fd5340b4750 2023-05-20T01:26:46.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.668+0000 7fd51b7fe700 1 -- 172.21.15.16:0/2119272104 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd524007470 con 0x7fd5340b4750 2023-05-20T01:26:46.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.668+0000 7fd51b7fe700 1 --2- 172.21.15.16:0/2119272104 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd52006ace0 0x7fd52006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:46.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.668+0000 7fd51b7fe700 1 -- 172.21.15.16:0/2119272104 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fd5240892e0 con 0x7fd5340b4750 2023-05-20T01:26:46.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.668+0000 7fd5327fc700 1 --2- 172.21.15.16:0/2119272104 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd52006ace0 0x7fd52006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:46.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.669+0000 7fd5327fc700 1 --2- 172.21.15.16:0/2119272104 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd52006ace0 0x7fd52006d190 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fd528007a50 tx=0x7fd528008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:46.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.670+0000 7fd5455a2b80 1 Processor -- start 2023-05-20T01:26:46.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.670+0000 7fd5455a2b80 1 -- start start 2023-05-20T01:26:46.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.671+0000 7fd5455a2b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564713160400 0x5647131607d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:46.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.671+0000 7fd5455a2b80 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56471315c700 0x564713135460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:46.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.671+0000 7fd5455a2b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564713133870 0x5647131359a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:46.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.671+0000 7fd5455a2b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x56471303b010 con 0x564713160400 2023-05-20T01:26:46.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.671+0000 7fd5455a2b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x56471308d850 con 0x564713133870 2023-05-20T01:26:46.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.671+0000 7fd5455a2b80 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x564713086ce0 con 0x56471315c700 2023-05-20T01:26:46.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.671+0000 7fd532ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564713160400 0x5647131607d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:46.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.671+0000 7fd532ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564713160400 0x5647131607d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38284/0 (socket says 172.21.15.16:38284) 2023-05-20T01:26:46.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.671+0000 7fd532ffd700 1 -- 172.21.15.16:0/2136867852 learned_addr learned my addr 172.21.15.16:0/2136867852 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:26:46.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.671+0000 7fd5327fc700 1 --2- 172.21.15.16:0/2136867852 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56471315c700 0x564713135460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:46.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.671+0000 7fd5337fe700 1 --2- 172.21.15.16:0/2136867852 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564713133870 0x5647131359a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:46.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.671+0000 7fd532ffd700 1 -- 172.21.15.16:0/2136867852 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56471315c700 msgr2=0x564713135460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:46.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.671+0000 7fd532ffd700 1 --2- 172.21.15.16:0/2136867852 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56471315c700 0x564713135460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:46.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.671+0000 7fd532ffd700 1 -- 172.21.15.16:0/2136867852 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564713133870 msgr2=0x5647131359a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:46.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.671+0000 7fd532ffd700 1 --2- 172.21.15.16:0/2136867852 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564713133870 0x5647131359a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:46.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.671+0000 7fd532ffd700 1 -- 172.21.15.16:0/2136867852 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564712f085e0 con 0x564713160400 2023-05-20T01:26:46.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.671+0000 7fd5327fc700 1 --2- 172.21.15.16:0/2136867852 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56471315c700 0x564713135460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:26:46.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.671+0000 7fd5337fe700 1 --2- 172.21.15.16:0/2136867852 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564713133870 0x5647131359a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:26:46.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.672+0000 7fd532ffd700 1 --2- 172.21.15.16:0/2136867852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564713160400 0x5647131607d0 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7fd524004fd0 tx=0x7fd524057ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68cbab0d0368cc7f server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:46.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.672+0000 7fd518ff9700 1 -- 172.21.15.16:0/2136867852 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5240890d0 con 0x564713160400 2023-05-20T01:26:46.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.672+0000 7fd518ff9700 1 -- 172.21.15.16:0/2136867852 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5240601c0 con 0x564713160400 2023-05-20T01:26:46.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.672+0000 7fd518ff9700 1 -- 172.21.15.16:0/2136867852 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd524057360 con 0x564713160400 2023-05-20T01:26:46.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.672+0000 7fd5455a2b80 1 -- 172.21.15.16:0/2136867852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564713160400 msgr2=0x5647131607d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:46.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.672+0000 7fd5455a2b80 1 --2- 172.21.15.16:0/2136867852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564713160400 0x5647131607d0 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7fd524004fd0 tx=0x7fd524057ab0 comp rx=0 tx=0).stop 2023-05-20T01:26:46.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.672+0000 7fd5455a2b80 1 -- 172.21.15.16:0/2136867852 shutdown_connections 2023-05-20T01:26:46.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.672+0000 7fd5455a2b80 1 --2- 172.21.15.16:0/2136867852 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564713133870 0x5647131359a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:46.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.672+0000 7fd5455a2b80 1 --2- 172.21.15.16:0/2136867852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564713160400 0x5647131607d0 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:46.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.672+0000 7fd5455a2b80 1 --2- 172.21.15.16:0/2136867852 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56471315c700 0x564713135460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:46.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.672+0000 7fd5455a2b80 1 -- 172.21.15.16:0/2136867852 >> 172.21.15.16:0/2136867852 conn(0x56471312af30 msgr2=0x56471317b210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:46.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.673+0000 7fd5455a2b80 1 -- 172.21.15.16:0/2136867852 shutdown_connections 2023-05-20T01:26:46.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.673+0000 7fd5455a2b80 1 -- 172.21.15.16:0/2136867852 wait complete. 2023-05-20T01:26:46.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.673+0000 7fd5455a2b80 1 -- start start 2023-05-20T01:26:46.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.673+0000 7fd5455a2b80 1 Processor -- start 2023-05-20T01:26:46.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.674+0000 7fd5455a2b80 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x564713133870 0x564713163d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:46.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.674+0000 7fd5455a2b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56471315c700 0x56471317e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:46.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.674+0000 7fd5455a2b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564713160400 0x56471317a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:46.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.674+0000 7fd5455a2b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x564713028890 con 0x56471315c700 2023-05-20T01:26:46.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.674+0000 7fd5455a2b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x564713028530 con 0x564713160400 2023-05-20T01:26:46.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.674+0000 7fd5455a2b80 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x564712f0a8d0 con 0x564713133870 2023-05-20T01:26:46.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.674+0000 7fd5455a2b80 10 client.0 _ll_register_callbacks cb 0x564713046730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-05-20T01:26:46.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.674+0000 7fd5327fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56471315c700 0x56471317e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:46.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.674+0000 7fd5327fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56471315c700 0x56471317e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38292/0 (socket says 172.21.15.16:38292) 2023-05-20T01:26:46.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.674+0000 7fd5327fc700 1 -- 172.21.15.16:0/2764794861 learned_addr learned my addr 172.21.15.16:0/2764794861 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:26:46.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.674+0000 7fd532ffd700 1 --2- 172.21.15.16:0/2764794861 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x564713133870 0x564713163d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:46.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.674+0000 7fd5327fc700 1 -- 172.21.15.16:0/2764794861 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x564713133870 msgr2=0x564713163d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:46.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.674+0000 7fd5337fe700 1 --2- 172.21.15.16:0/2764794861 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564713160400 0x56471317a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:46.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.674+0000 7fd5327fc700 1 --2- 172.21.15.16:0/2764794861 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x564713133870 0x564713163d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:46.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.674+0000 7fd5327fc700 1 -- 172.21.15.16:0/2764794861 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564713160400 msgr2=0x56471317a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:46.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.674+0000 7fd5327fc700 1 --2- 172.21.15.16:0/2764794861 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564713160400 0x56471317a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:46.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.674+0000 7fd5327fc700 1 -- 172.21.15.16:0/2764794861 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd53415d0b0 con 0x56471315c700 2023-05-20T01:26:46.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.674+0000 7fd5327fc700 1 --2- 172.21.15.16:0/2764794861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56471315c700 0x56471317e270 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7fd52800ffb0 tx=0x7fd5280047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2714c98e6f26da14 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:46.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.674+0000 7fd511ffb700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-05-20T01:26:46.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.675+0000 7fd511ffb700 1 -- 172.21.15.16:0/2764794861 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd528021610 con 0x56471315c700 2023-05-20T01:26:46.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.675+0000 7fd5455a2b80 1 -- 172.21.15.16:0/2764794861 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5647131286f0 con 0x56471315c700 2023-05-20T01:26:46.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.675+0000 7fd511ffb700 1 -- 172.21.15.16:0/2764794861 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fd528021770 con 0x56471315c700 2023-05-20T01:26:46.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.675+0000 7fd511ffb700 1 -- 172.21.15.16:0/2764794861 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd528021930 con 0x56471315c700 2023-05-20T01:26:46.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.675+0000 7fd5455a2b80 10 client.8088 fetch_fsmap learned FSMap version 78 2023-05-20T01:26:46.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.675+0000 7fd5455a2b80 1 -- 172.21.15.16:0/2764794861 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=2+}) v3 -- 0x56471304d140 con 0x56471315c700 2023-05-20T01:26:46.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.675+0000 7fd511ffb700 1 -- 172.21.15.16:0/2764794861 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5647131286f0 con 0x56471315c700 2023-05-20T01:26:46.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.675+0000 7fd511ffb700 1 -- 172.21.15.16:0/2764794861 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 78) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fd528021a90 con 0x56471315c700 2023-05-20T01:26:46.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.675+0000 7fd5455a2b80 10 client.8088 fetch_fsmap finished waiting for FSMap version 78 2023-05-20T01:26:46.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.675+0000 7fd5455a2b80 10 client.8088 resolve_mds: resolved name 'a' to daemon mds.a 2023-05-20T01:26:46.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.675+0000 7fd5455a2b80 20 client.8088 populate_metadata read hostname 'smithi016' 2023-05-20T01:26:46.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.675+0000 7fd5455a2b80 1 --2- 172.21.15.16:0/2764794861 >> [v2:172.21.15.16:6834/3765848068,v1:172.21.15.16:6836/3765848068] conn(0x5647131614d0 0x5647131618a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:46.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.675+0000 7fd5455a2b80 4 client.8088 mds_command: new command op to 7896 tid=0[{"prefix": "get_command_descriptions"}] 2023-05-20T01:26:46.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.675+0000 7fd5455a2b80 1 -- 172.21.15.16:0/2764794861 --> [v2:172.21.15.16:6834/3765848068,v1:172.21.15.16:6836/3765848068] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564713049050 con 0x5647131614d0 2023-05-20T01:26:46.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.675+0000 7fd532ffd700 1 --2- 172.21.15.16:0/2764794861 >> [v2:172.21.15.16:6834/3765848068,v1:172.21.15.16:6836/3765848068] conn(0x5647131614d0 0x5647131618a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:46.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.676+0000 7fd511ffb700 10 client.8088 ms_handle_connect on v2:172.21.15.16:6834/3765848068 2023-05-20T01:26:46.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.676+0000 7fd532ffd700 1 --2- 172.21.15.16:0/2764794861 >> [v2:172.21.15.16:6834/3765848068,v1:172.21.15.16:6836/3765848068] conn(0x5647131614d0 0x5647131618a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52d60b1ef9916f87 server_cookie=80f078c9d354568c in_seq=0 out_seq=0 2023-05-20T01:26:46.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.676+0000 7fd511ffb700 1 -- 172.21.15.16:0/2764794861 <== mds.0 v2:172.21.15.16:6834/3765848068 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x564713049050 con 0x5647131614d0 2023-05-20T01:26:46.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.676+0000 7fd511ffb700 10 client.8088 handle_command_reply: tid=0 2023-05-20T01:26:46.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.676+0000 7fd5455a2b80 1 client.8088 shutdown 2023-05-20T01:26:46.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.676+0000 7fd5455a2b80 1 -- 172.21.15.16:0/2764794861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56471315c700 msgr2=0x56471317e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:46.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.676+0000 7fd5455a2b80 1 --2- 172.21.15.16:0/2764794861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56471315c700 0x56471317e270 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7fd52800ffb0 tx=0x7fd5280047c0 comp rx=0 tx=0).stop 2023-05-20T01:26:46.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.676+0000 7fd5455a2b80 1 -- 172.21.15.16:0/2764794861 shutdown_connections 2023-05-20T01:26:46.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.676+0000 7fd5455a2b80 1 --2- 172.21.15.16:0/2764794861 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564713160400 0x56471317a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:46.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.676+0000 7fd5455a2b80 1 --2- 172.21.15.16:0/2764794861 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x564713133870 0x564713163d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:46.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.676+0000 7fd5455a2b80 1 --2- 172.21.15.16:0/2764794861 >> [v2:172.21.15.16:6834/3765848068,v1:172.21.15.16:6836/3765848068] conn(0x5647131614d0 0x5647131618a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:46.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.676+0000 7fd5455a2b80 1 --2- 172.21.15.16:0/2764794861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56471315c700 0x56471317e270 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:46.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.676+0000 7fd511ffb700 0 client.8088 ms_handle_reset on v2:172.21.15.16:6834/3765848068 2023-05-20T01:26:46.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.676+0000 7fd5455a2b80 1 -- 172.21.15.16:0/2764794861 >> 172.21.15.16:0/2764794861 conn(0x56471312af30 msgr2=0x564713130450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:46.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.676+0000 7fd5455a2b80 1 -- 172.21.15.16:0/2764794861 shutdown_connections 2023-05-20T01:26:46.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.677+0000 7fd5455a2b80 1 -- 172.21.15.16:0/2764794861 wait complete. 2023-05-20T01:26:46.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.677+0000 7fd5455a2b80 20 client.8088 trim_cache size 0 max 16384 2023-05-20T01:26:46.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.695+0000 7fd5455a2b80 1 Processor -- start 2023-05-20T01:26:46.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.695+0000 7fd5455a2b80 1 -- start start 2023-05-20T01:26:46.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.695+0000 7fd5455a2b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564713133870 0x56471312f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:46.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.695+0000 7fd5455a2b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56471315c700 0x56471317b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:46.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.695+0000 7fd5455a2b80 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x564713160400 0x56471317b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:46.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.695+0000 7fd5455a2b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x56471315cc00 con 0x56471315c700 2023-05-20T01:26:46.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.695+0000 7fd5455a2b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x56471317f560 con 0x564713133870 2023-05-20T01:26:46.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.695+0000 7fd5455a2b80 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x56471317a250 con 0x564713160400 2023-05-20T01:26:46.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.695+0000 7fd5327fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56471315c700 0x56471317b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:46.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.695+0000 7fd5327fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56471315c700 0x56471317b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38300/0 (socket says 172.21.15.16:38300) 2023-05-20T01:26:46.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.695+0000 7fd5327fc700 1 -- 172.21.15.16:0/1560481492 learned_addr learned my addr 172.21.15.16:0/1560481492 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:26:46.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.696+0000 7fd5327fc700 1 -- 172.21.15.16:0/1560481492 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x564713160400 msgr2=0x56471317b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:46.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.696+0000 7fd5327fc700 1 --2- 172.21.15.16:0/1560481492 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x564713160400 0x56471317b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:46.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.696+0000 7fd5327fc700 1 -- 172.21.15.16:0/1560481492 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564713133870 msgr2=0x56471312f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:46.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.696+0000 7fd532ffd700 1 --2- 172.21.15.16:0/1560481492 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564713133870 0x56471312f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:46.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.696+0000 7fd5327fc700 1 --2- 172.21.15.16:0/1560481492 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564713133870 0x56471312f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:46.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.696+0000 7fd5327fc700 1 -- 172.21.15.16:0/1560481492 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56471304ce90 con 0x56471315c700 2023-05-20T01:26:46.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.696+0000 7fd5327fc700 1 --2- 172.21.15.16:0/1560481492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56471315c700 0x56471317b210 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fd52800fbb0 tx=0x7fd52801f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd5c4579143c631d server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:46.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.696+0000 7fd5197fa700 1 -- 172.21.15.16:0/1560481492 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd528029550 con 0x56471315c700 2023-05-20T01:26:46.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.696+0000 7fd5197fa700 1 -- 172.21.15.16:0/1560481492 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd52801c050 con 0x56471315c700 2023-05-20T01:26:46.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.696+0000 7fd5197fa700 1 -- 172.21.15.16:0/1560481492 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5280298b0 con 0x56471315c700 2023-05-20T01:26:46.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.696+0000 7fd5455a2b80 1 -- 172.21.15.16:0/1560481492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56471315c700 msgr2=0x56471317b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:46.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.696+0000 7fd5455a2b80 1 --2- 172.21.15.16:0/1560481492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56471315c700 0x56471317b210 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fd52800fbb0 tx=0x7fd52801f000 comp rx=0 tx=0).stop 2023-05-20T01:26:46.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.696+0000 7fd5455a2b80 1 -- 172.21.15.16:0/1560481492 shutdown_connections 2023-05-20T01:26:46.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.696+0000 7fd5455a2b80 1 --2- 172.21.15.16:0/1560481492 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x564713160400 0x56471317b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:46.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.697+0000 7fd5455a2b80 1 --2- 172.21.15.16:0/1560481492 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564713133870 0x56471312f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:46.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.697+0000 7fd5455a2b80 1 --2- 172.21.15.16:0/1560481492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56471315c700 0x56471317b210 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:46.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.697+0000 7fd5455a2b80 1 -- 172.21.15.16:0/1560481492 >> 172.21.15.16:0/1560481492 conn(0x56471312af30 msgr2=0x56471312c400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:46.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.697+0000 7fd5455a2b80 1 -- 172.21.15.16:0/1560481492 shutdown_connections 2023-05-20T01:26:46.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.697+0000 7fd5455a2b80 1 -- 172.21.15.16:0/1560481492 wait complete. 2023-05-20T01:26:46.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.697+0000 7fd5455a2b80 1 -- start start 2023-05-20T01:26:46.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.697+0000 7fd5455a2b80 1 Processor -- start 2023-05-20T01:26:46.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.698+0000 7fd5455a2b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564713133870 0x5647131ca260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:46.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.698+0000 7fd5455a2b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56471315c700 0x56471318bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:46.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.698+0000 7fd5455a2b80 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x564713160400 0x56471317b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:46.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.698+0000 7fd5455a2b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x564713171070 con 0x564713133870 2023-05-20T01:26:46.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.698+0000 7fd5455a2b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x56471317e7b0 con 0x56471315c700 2023-05-20T01:26:46.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.698+0000 7fd5455a2b80 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x56471318c290 con 0x564713160400 2023-05-20T01:26:46.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.698+0000 7fd5455a2b80 10 client.0 _ll_register_callbacks cb 0x7fd52800fc60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-05-20T01:26:46.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.698+0000 7fd532ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564713133870 0x5647131ca260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:46.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.698+0000 7fd532ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564713133870 0x5647131ca260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38304/0 (socket says 172.21.15.16:38304) 2023-05-20T01:26:46.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.698+0000 7fd532ffd700 1 -- 172.21.15.16:0/3375063454 learned_addr learned my addr 172.21.15.16:0/3375063454 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:26:46.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.698+0000 7fd532ffd700 1 -- 172.21.15.16:0/3375063454 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x564713160400 msgr2=0x56471317b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:46.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.698+0000 7fd532ffd700 1 --2- 172.21.15.16:0/3375063454 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x564713160400 0x56471317b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:46.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.698+0000 7fd532ffd700 1 -- 172.21.15.16:0/3375063454 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56471315c700 msgr2=0x56471318bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:46.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.698+0000 7fd532ffd700 1 --2- 172.21.15.16:0/3375063454 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56471315c700 0x56471318bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:46.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.698+0000 7fd532ffd700 1 -- 172.21.15.16:0/3375063454 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56471304d140 con 0x564713133870 2023-05-20T01:26:46.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.699+0000 7fd532ffd700 1 --2- 172.21.15.16:0/3375063454 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564713133870 0x5647131ca260 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fd52405fc90 tx=0x7fd524088000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad9c0dd08a306f0a server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:46.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.699+0000 7fd511ffb700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-05-20T01:26:46.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.699+0000 7fd511ffb700 1 -- 172.21.15.16:0/3375063454 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd524056210 con 0x564713133870 2023-05-20T01:26:46.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.699+0000 7fd511ffb700 1 -- 172.21.15.16:0/3375063454 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fd5240767f0 con 0x564713133870 2023-05-20T01:26:46.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.699+0000 7fd5455a2b80 1 -- 172.21.15.16:0/3375063454 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564713130270 con 0x564713133870 2023-05-20T01:26:46.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.699+0000 7fd511ffb700 1 -- 172.21.15.16:0/3375063454 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd524057360 con 0x564713133870 2023-05-20T01:26:46.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.699+0000 7fd5455a2b80 10 client.8094 fetch_fsmap learned FSMap version 78 2023-05-20T01:26:46.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.699+0000 7fd5455a2b80 1 -- 172.21.15.16:0/3375063454 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=2+}) v3 -- 0x564712e9dc10 con 0x564713133870 2023-05-20T01:26:46.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.699+0000 7fd511ffb700 1 -- 172.21.15.16:0/3375063454 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564713130270 con 0x564713133870 2023-05-20T01:26:46.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.699+0000 7fd511ffb700 1 -- 172.21.15.16:0/3375063454 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 78) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fd52405fd30 con 0x564713133870 2023-05-20T01:26:46.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.699+0000 7fd5455a2b80 10 client.8094 fetch_fsmap finished waiting for FSMap version 78 2023-05-20T01:26:46.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.699+0000 7fd5455a2b80 10 client.8094 resolve_mds: resolved name 'a' to daemon mds.a 2023-05-20T01:26:46.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.699+0000 7fd5455a2b80 20 client.8094 populate_metadata read hostname 'smithi016' 2023-05-20T01:26:46.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.699+0000 7fd5455a2b80 1 --2- 172.21.15.16:0/3375063454 >> [v2:172.21.15.16:6834/3765848068,v1:172.21.15.16:6836/3765848068] conn(0x5647131e2120 0x5647131e2510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:46.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.699+0000 7fd5455a2b80 4 client.8094 mds_command: new command op to 7896 tid=0[{"prefix": "status"}] 2023-05-20T01:26:46.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.699+0000 7fd5455a2b80 1 -- 172.21.15.16:0/3375063454 --> [v2:172.21.15.16:6834/3765848068,v1:172.21.15.16:6836/3765848068] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7fd5240601c0 con 0x5647131e2120 2023-05-20T01:26:46.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.699+0000 7fd5327fc700 1 --2- 172.21.15.16:0/3375063454 >> [v2:172.21.15.16:6834/3765848068,v1:172.21.15.16:6836/3765848068] conn(0x5647131e2120 0x5647131e2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:46.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.700+0000 7fd511ffb700 10 client.8094 ms_handle_connect on v2:172.21.15.16:6834/3765848068 2023-05-20T01:26:46.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.700+0000 7fd5327fc700 1 --2- 172.21.15.16:0/3375063454 >> [v2:172.21.15.16:6834/3765848068,v1:172.21.15.16:6836/3765848068] conn(0x5647131e2120 0x5647131e2510 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a25732638c15fb2 server_cookie=3866b82eddf207bd in_seq=0 out_seq=0 2023-05-20T01:26:46.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.700+0000 7fd511ffb700 1 -- 172.21.15.16:0/3375063454 <== mds.0 v2:172.21.15.16:6834/3765848068 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x7fd5240601c0 con 0x5647131e2120 2023-05-20T01:26:46.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.700+0000 7fd511ffb700 10 client.8094 handle_command_reply: tid=0 2023-05-20T01:26:46.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.700+0000 7fd5455a2b80 1 client.8094 shutdown 2023-05-20T01:26:46.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.700+0000 7fd5455a2b80 1 -- 172.21.15.16:0/3375063454 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564713133870 msgr2=0x5647131ca260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:46.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.700+0000 7fd5455a2b80 1 --2- 172.21.15.16:0/3375063454 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564713133870 0x5647131ca260 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fd52405fc90 tx=0x7fd524088000 comp rx=0 tx=0).stop 2023-05-20T01:26:46.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.700+0000 7fd5455a2b80 1 -- 172.21.15.16:0/3375063454 shutdown_connections 2023-05-20T01:26:46.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.700+0000 7fd5455a2b80 1 --2- 172.21.15.16:0/3375063454 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x564713160400 0x56471317b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:46.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.700+0000 7fd5455a2b80 1 --2- 172.21.15.16:0/3375063454 >> [v2:172.21.15.16:6834/3765848068,v1:172.21.15.16:6836/3765848068] conn(0x5647131e2120 0x5647131e2510 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:46.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.700+0000 7fd5455a2b80 1 --2- 172.21.15.16:0/3375063454 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564713133870 0x5647131ca260 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:46.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.700+0000 7fd5455a2b80 1 --2- 172.21.15.16:0/3375063454 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56471315c700 0x56471318bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:46.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.700+0000 7fd511ffb700 0 client.8094 ms_handle_reset on v2:172.21.15.16:6834/3765848068 2023-05-20T01:26:46.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.700+0000 7fd5455a2b80 1 -- 172.21.15.16:0/3375063454 >> 172.21.15.16:0/3375063454 conn(0x56471312af30 msgr2=0x56471319d0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:46.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.700+0000 7fd5455a2b80 1 -- 172.21.15.16:0/3375063454 shutdown_connections 2023-05-20T01:26:46.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.700+0000 7fd5455a2b80 1 -- 172.21.15.16:0/3375063454 wait complete. 2023-05-20T01:26:46.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.700+0000 7fd5455a2b80 20 client.8094 trim_cache size 0 max 16384 2023-05-20T01:26:46.714 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-05-20T01:26:46.714 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "c2245cc5-ecab-4087-a59b-bc81db901676", 2023-05-20T01:26:46.714 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2023-05-20T01:26:46.714 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 7896, 2023-05-20T01:26:46.714 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2023-05-20T01:26:46.714 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2023-05-20T01:26:46.715 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2023-05-20T01:26:46.715 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 10.631784883, 2023-05-20T01:26:46.715 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 78, 2023-05-20T01:26:46.715 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 112, 2023-05-20T01:26:46.715 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 112, 2023-05-20T01:26:46.715 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 15.948664062000001 2023-05-20T01:26:46.715 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-05-20T01:26:46.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.700+0000 7fd539416700 1 -- 172.21.15.16:0/2119272104 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd52006ace0 msgr2=0x7fd52006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:46.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.701+0000 7fd539416700 1 --2- 172.21.15.16:0/2119272104 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd52006ace0 0x7fd52006d190 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fd528007a50 tx=0x7fd528008040 comp rx=0 tx=0).stop 2023-05-20T01:26:46.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.701+0000 7fd539416700 1 -- 172.21.15.16:0/2119272104 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5340b4750 msgr2=0x7fd53415a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:46.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.701+0000 7fd539416700 1 --2- 172.21.15.16:0/2119272104 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5340b4750 0x7fd53415a7c0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7fd52400c550 tx=0x7fd52400c860 comp rx=0 tx=0).stop 2023-05-20T01:26:46.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.701+0000 7fd539416700 1 -- 172.21.15.16:0/2119272104 shutdown_connections 2023-05-20T01:26:46.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.701+0000 7fd539416700 1 --2- 172.21.15.16:0/2119272104 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd5340cfc50 0x7fd5340b1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:46.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.701+0000 7fd539416700 1 --2- 172.21.15.16:0/2119272104 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5340b4750 0x7fd53415a7c0 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:46.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.701+0000 7fd539416700 1 --2- 172.21.15.16:0/2119272104 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd52006ace0 0x7fd52006d190 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:46.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.701+0000 7fd539416700 1 --2- 172.21.15.16:0/2119272104 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd5340cd970 0x7fd5340b0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:46.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.701+0000 7fd539416700 1 -- 172.21.15.16:0/2119272104 >> 172.21.15.16:0/2119272104 conn(0x7fd53409c590 msgr2=0x7fd53406b170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:46.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.701+0000 7fd539416700 1 -- 172.21.15.16:0/2119272104 shutdown_connections 2023-05-20T01:26:46.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.701+0000 7fd539416700 1 -- 172.21.15.16:0/2119272104 wait complete. 2023-05-20T01:26:46.719 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-05-20T01:26:46.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.839+0000 7f8bdc296700 1 Processor -- start 2023-05-20T01:26:46.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.839+0000 7f8bdc296700 1 -- start start 2023-05-20T01:26:46.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.840+0000 7f8bdc296700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f8bd4144020 0x7f8bd40d1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:46.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.840+0000 7f8bdc296700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8bd40cec60 0x7f8bd40cf0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:46.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.840+0000 7f8bdc296700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bd40cf600 con 0x7f8bd40b2760 2023-05-20T01:26:46.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.840+0000 7f8bdc296700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bd40cb6e0 con 0x7f8bd40cec60 2023-05-20T01:26:46.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.840+0000 7f8bdc296700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bd41448a0 con 0x7f8bd4144020 2023-05-20T01:26:46.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.840+0000 7f8bda833700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8bd40cec60 0x7f8bd40cf0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:46.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.840+0000 7f8bda833700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8bd40cec60 0x7f8bd40cf0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38316/0 (socket says 172.21.15.16:38316) 2023-05-20T01:26:46.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.840+0000 7f8bda833700 1 -- 172.21.15.16:0/3685367204 learned_addr learned my addr 172.21.15.16:0/3685367204 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:26:46.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.841+0000 7f8bd9831700 1 --2- 172.21.15.16:0/3685367204 >> v2:172.21.15.132:3300/0 conn(0x7f8bd4144020 0x7f8bd40d1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:46.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.841+0000 7f8bda833700 1 -- 172.21.15.16:0/3685367204 >> v2:172.21.15.132:3300/0 conn(0x7f8bd4144020 msgr2=0x7f8bd40d1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:46.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.841+0000 7f8bda833700 1 --2- 172.21.15.16:0/3685367204 >> v2:172.21.15.132:3300/0 conn(0x7f8bd4144020 0x7f8bd40d1680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:46.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.841+0000 7f8bda833700 1 -- 172.21.15.16:0/3685367204 >> v1:172.21.15.132:6789/0 conn(0x7f8bd40b2760 legacy=0x7f8bd40b0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:46.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.841+0000 7f8bda833700 1 -- 172.21.15.16:0/3685367204 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bd40cc500 con 0x7f8bd40cec60 2023-05-20T01:26:46.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.841+0000 7f8bda833700 1 --2- 172.21.15.16:0/3685367204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bd40cec60 0x7f8bd40cf0c0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f8bd40b3c80 tx=0x7f8bcc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8e395e967829cd6 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:46.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.841+0000 7f8bd9030700 1 -- 172.21.15.16:0/3685367204 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc00e070 con 0x7f8bd40cec60 2023-05-20T01:26:46.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.841+0000 7f8bd9030700 1 -- 172.21.15.16:0/3685367204 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bcc003a20 con 0x7f8bd40cec60 2023-05-20T01:26:46.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.841+0000 7f8bd9030700 1 -- 172.21.15.16:0/3685367204 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc003dd0 con 0x7f8bd40cec60 2023-05-20T01:26:46.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.841+0000 7f8bdc296700 1 -- 172.21.15.16:0/3685367204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bd40cec60 msgr2=0x7f8bd40cf0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:46.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.841+0000 7f8bdc296700 1 --2- 172.21.15.16:0/3685367204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bd40cec60 0x7f8bd40cf0c0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f8bd40b3c80 tx=0x7f8bcc00b330 comp rx=0 tx=0).stop 2023-05-20T01:26:46.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.842+0000 7f8bdc296700 1 -- 172.21.15.16:0/3685367204 shutdown_connections 2023-05-20T01:26:46.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.842+0000 7f8bdc296700 1 --2- 172.21.15.16:0/3685367204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bd40cec60 0x7f8bd40cf0c0 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:46.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.842+0000 7f8bdc296700 1 --2- 172.21.15.16:0/3685367204 >> v2:172.21.15.132:3300/0 conn(0x7f8bd4144020 0x7f8bd40d1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:46.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.842+0000 7f8bdc296700 1 -- 172.21.15.16:0/3685367204 >> 172.21.15.16:0/3685367204 conn(0x7f8bd406ae30 msgr2=0x7f8bd409a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:46.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.842+0000 7f8bdc296700 1 -- 172.21.15.16:0/3685367204 shutdown_connections 2023-05-20T01:26:46.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.842+0000 7f8bdc296700 1 -- 172.21.15.16:0/3685367204 wait complete. 2023-05-20T01:26:46.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.843+0000 7f8bdc296700 1 Processor -- start 2023-05-20T01:26:46.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.843+0000 7f8bdc296700 1 -- start start 2023-05-20T01:26:46.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.844+0000 7f8bdc296700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8bd40b2760 0x7f8bd41f3000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:46.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.844+0000 7f8bdc296700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bd40cec60 0x7f8bd40b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:46.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.844+0000 7f8bdc296700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8bd4144020 0x7f8bd406c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:46.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.844+0000 7f8bdc296700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd406ce20 con 0x7f8bd40cec60 2023-05-20T01:26:46.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.844+0000 7f8bdc296700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd40b5360 con 0x7f8bd4144020 2023-05-20T01:26:46.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.844+0000 7f8bdc296700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd40b4060 con 0x7f8bd40b2760 2023-05-20T01:26:46.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.844+0000 7f8bd9831700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bd40cec60 0x7f8bd40b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:46.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.844+0000 7f8bd9831700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bd40cec60 0x7f8bd40b4e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38320/0 (socket says 172.21.15.16:38320) 2023-05-20T01:26:46.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.844+0000 7f8bd9831700 1 -- 172.21.15.16:0/1288950776 learned_addr learned my addr 172.21.15.16:0/1288950776 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:26:46.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.844+0000 7f8bda833700 1 --2- 172.21.15.16:0/1288950776 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8bd4144020 0x7f8bd406c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:46.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.844+0000 7f8bda032700 1 --2- 172.21.15.16:0/1288950776 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8bd40b2760 0x7f8bd41f3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:46.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.844+0000 7f8bd9831700 1 -- 172.21.15.16:0/1288950776 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8bd40b2760 msgr2=0x7f8bd41f3000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:46.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.844+0000 7f8bd9831700 1 --2- 172.21.15.16:0/1288950776 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8bd40b2760 0x7f8bd41f3000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:46.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.844+0000 7f8bd9831700 1 -- 172.21.15.16:0/1288950776 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8bd4144020 msgr2=0x7f8bd406c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:46.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.844+0000 7f8bd9831700 1 --2- 172.21.15.16:0/1288950776 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8bd4144020 0x7f8bd406c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:46.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.844+0000 7f8bd9831700 1 -- 172.21.15.16:0/1288950776 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bcc009b30 con 0x7f8bd40cec60 2023-05-20T01:26:46.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.844+0000 7f8bda833700 1 --2- 172.21.15.16:0/1288950776 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8bd4144020 0x7f8bd406c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:26:46.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.844+0000 7f8bda032700 1 --2- 172.21.15.16:0/1288950776 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8bd40b2760 0x7f8bd41f3000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:26:46.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.844+0000 7f8bd9831700 1 --2- 172.21.15.16:0/1288950776 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bd40cec60 0x7f8bd40b4e20 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f8bd000c7f0 tx=0x7f8bd000cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:46.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.844+0000 7f8bcaffd700 1 -- 172.21.15.16:0/1288950776 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bd0011800 con 0x7f8bd40cec60 2023-05-20T01:26:46.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.844+0000 7f8bcaffd700 1 -- 172.21.15.16:0/1288950776 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bd0011960 con 0x7f8bd40cec60 2023-05-20T01:26:46.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.844+0000 7f8bdc296700 1 -- 172.21.15.16:0/1288950776 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bd40cbbd0 con 0x7f8bd40cec60 2023-05-20T01:26:46.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.844+0000 7f8bcaffd700 1 -- 172.21.15.16:0/1288950776 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bd0011cf0 con 0x7f8bd40cec60 2023-05-20T01:26:46.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.844+0000 7f8bdc296700 1 -- 172.21.15.16:0/1288950776 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bd40cb8f0 con 0x7f8bd40cec60 2023-05-20T01:26:46.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.845+0000 7f8bdc296700 1 -- 172.21.15.16:0/1288950776 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bd40cbeb0 con 0x7f8bd40cec60 2023-05-20T01:26:46.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.846+0000 7f8bcaffd700 1 -- 172.21.15.16:0/1288950776 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8bd0018360 con 0x7f8bd40cec60 2023-05-20T01:26:46.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.846+0000 7f8bcaffd700 1 --2- 172.21.15.16:0/1288950776 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f8bb806ace0 0x7f8bb806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:46.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.846+0000 7f8bcaffd700 1 -- 172.21.15.16:0/1288950776 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f8bd0089580 con 0x7f8bd40cec60 2023-05-20T01:26:46.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.846+0000 7f8bda032700 1 --2- 172.21.15.16:0/1288950776 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f8bb806ace0 0x7f8bb806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:46.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.847+0000 7f8bda032700 1 --2- 172.21.15.16:0/1288950776 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f8bb806ace0 0x7f8bb806d190 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f8bd41f5a20 tx=0x7f8bc4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:46.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:46.851+0000 7f8bcaffd700 1 -- 172.21.15.16:0/1288950776 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f8bd0054500 con 0x7f8bd40cec60 2023-05-20T01:26:47.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:47.096+0000 7f8bdc296700 1 -- 172.21.15.16:0/1288950776 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f8bd40b4c70 con 0x7f8bd40cec60 2023-05-20T01:26:47.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:47.096+0000 7f8bcaffd700 1 -- 172.21.15.16:0/1288950776 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v78) v1 ==== 53+0+73 (secure 0 0 0) 0x7f8bd0057b20 con 0x7f8bd40cec60 2023-05-20T01:26:47.102 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-05-20T01:26:47.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:47.098+0000 7f8bdc296700 1 -- 172.21.15.16:0/1288950776 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f8bb806ace0 msgr2=0x7f8bb806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:47.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:47.098+0000 7f8bdc296700 1 --2- 172.21.15.16:0/1288950776 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f8bb806ace0 0x7f8bb806d190 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f8bd41f5a20 tx=0x7f8bc4008040 comp rx=0 tx=0).stop 2023-05-20T01:26:47.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:47.098+0000 7f8bdc296700 1 -- 172.21.15.16:0/1288950776 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bd40cec60 msgr2=0x7f8bd40b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:47.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:47.098+0000 7f8bdc296700 1 --2- 172.21.15.16:0/1288950776 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bd40cec60 0x7f8bd40b4e20 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f8bd000c7f0 tx=0x7f8bd000cbb0 comp rx=0 tx=0).stop 2023-05-20T01:26:47.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:47.098+0000 7f8bdc296700 1 -- 172.21.15.16:0/1288950776 shutdown_connections 2023-05-20T01:26:47.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:47.098+0000 7f8bdc296700 1 --2- 172.21.15.16:0/1288950776 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8bd4144020 0x7f8bd406c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:47.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:47.098+0000 7f8bdc296700 1 --2- 172.21.15.16:0/1288950776 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f8bb806ace0 0x7f8bb806d190 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:47.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:47.098+0000 7f8bdc296700 1 --2- 172.21.15.16:0/1288950776 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8bd40b2760 0x7f8bd41f3000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:47.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:47.098+0000 7f8bdc296700 1 --2- 172.21.15.16:0/1288950776 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bd40cec60 0x7f8bd40b4e20 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:47.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:47.098+0000 7f8bdc296700 1 -- 172.21.15.16:0/1288950776 >> 172.21.15.16:0/1288950776 conn(0x7f8bd406ae30 msgr2=0x7f8bd40d0e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:47.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:47.098+0000 7f8bdc296700 1 -- 172.21.15.16:0/1288950776 shutdown_connections 2023-05-20T01:26:47.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:47.098+0000 7f8bdc296700 1 -- 172.21.15.16:0/1288950776 wait complete. 2023-05-20T01:26:47.114 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-05-20T01:26:47.115 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-05-20T01:26:47.115 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2023-05-20T01:26:47.115 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-05-20T01:26:47.116 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-05-20T01:26:47.116 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-05-20T01:26:47.116 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-05-20T01:26:47.116 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-05-20T01:26:47.117 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-05-20T01:26:47.117 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:26:47.118 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2023-05-20T01:26:47.134 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-05-20T01:26:47.135 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-05-20T01:26:47.135 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2023-05-20T01:26:47.135 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-05-20T01:26:47.135 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2023-05-20T01:26:47.135 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-05-20T01:26:47.135 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-05-20T01:26:47.136 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2023-05-20T01:26:47.136 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-05-20T01:26:47.136 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2023-05-20T01:26:47.136 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-05-20T01:26:47.136 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2023-05-20T01:26:47.136 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-05-20T01:26:47.137 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 41827sec preferred_lft 41827sec 2023-05-20T01:26:47.137 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link 2023-05-20T01:26:47.137 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-05-20T01:26:47.137 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-05-20T01:26:47.137 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2023-05-20T01:26:47.137 INFO:teuthology.orchestra.run.smithi016.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-05-20T01:26:47.138 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0a:2e:0c:6b:5d:42 brd ff:ff:ff:ff:ff:ff 2023-05-20T01:26:47.138 INFO:teuthology.orchestra.run.smithi016.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-05-20T01:26:47.138 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-05-20T01:26:47.138 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::7c00:b2ff:fef4:946/64 scope link 2023-05-20T01:26:47.139 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-05-20T01:26:47.139 INFO:teuthology.orchestra.run.smithi016.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-05-20T01:26:47.139 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether d2:e4:70:a7:b9:4b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-05-20T01:26:47.139 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::d0e4:70ff:fea7:b94b/64 scope link 2023-05-20T01:26:47.140 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-05-20T01:26:47.140 INFO:teuthology.orchestra.run.smithi016.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-05-20T01:26:47.140 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0a:2e:0c:6b:5d:42 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-05-20T01:26:47.141 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::82e:cff:fe6b:5d42/64 scope link 2023-05-20T01:26:47.141 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-05-20T01:26:47.141 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:26:47.141 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-05-20T01:26:47.200 INFO:teuthology.orchestra.run.smithi016.stdout:1 2023-05-20T01:26:47.201 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:26:47.202 DEBUG:teuthology.orchestra.run.smithi016:> route 2023-05-20T01:26:47.262 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2023-05-20T01:26:47.262 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-05-20T01:26:47.263 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-05-20T01:26:47.263 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-05-20T01:26:47.263 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-05-20T01:26:47.263 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:26:47.264 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2023-05-20T01:26:47.264 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2023-05-20T01:26:47.265 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-05-20T01:26:47.265 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-05-20T01:26:47.265 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-05-20T01:26:47.265 DEBUG:teuthology.orchestra.run.smithi016:> ') 2023-05-20T01:26:47.369 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:26:47.369 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2023-05-20T01:26:47.386 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-05-20T01:26:47.386 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-05-20T01:26:47.387 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-05-20T01:26:47.387 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:26:47.387 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.0 up 2023-05-20T01:26:47.453 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-05-20T01:26:47.453 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-05-20T01:26:47.509 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-05-20T01:26:47.509 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-05-20T01:26:47.510 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-05-20T01:26:47.565 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-05-20T01:26:47.567 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:26:47.567 DEBUG:teuthology.orchestra.run.smithi016:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-05-20T01:26:47.653 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-05-20T01:26:47.653 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-05-20T01:26:47.684 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-05-20T01:26:47.685 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-05-20T01:26:47.685 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.132:3300,172.21.15.132:3301" 2023-05-20T01:26:47.685 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: trying mount with new device syntax: 0@c2245cc5-ecab-4087-a59b-bc81db901676.cephfs=/ 2023-05-20T01:26:47.685 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.16:3300/172.21.15.132:3300/172.21.15.132:3301" will pass to kernel 2023-05-20T01:26:47.685 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: v2 mount support check returned 2 2023-05-20T01:26:47.685 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: kernel does not support v2 syntax 2023-05-20T01:26:47.685 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: switching to using v1 address with old syntax 2023-05-20T01:26:47.717 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-05-20T01:26:47.717 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-05-20T01:26:47.718 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.132:3300,172.21.15.132:3301" 2023-05-20T01:26:47.718 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: trying mount with new device syntax: 0@c2245cc5-ecab-4087-a59b-bc81db901676.cephfs=/ 2023-05-20T01:26:47.718 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.16:3300/172.21.15.132:3300/172.21.15.132:3301" will pass to kernel 2023-05-20T01:26:47.718 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: v2 mount support check returned 2 2023-05-20T01:26:47.718 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: kernel does not support v2 syntax 2023-05-20T01:26:47.718 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: switching to using v1 address with old syntax 2023-05-20T01:26:47.718 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:6789,172.21.15.132:6789,172.21.15.132:6790" 2023-05-20T01:26:47.719 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.16:6789,172.21.15.132:6789,172.21.15.132:6790:/ 2023-05-20T01:26:47.719 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=c2245cc5-ecab-4087-a59b-bc81db901676" will pass to kernel 2023-05-20T01:26:47.723 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-05-20T01:26:47.723 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:26:47.723 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-05-20T01:26:47.761 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:26:47.762 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-05-20T01:26:47.842 INFO:teuthology.orchestra.run.smithi016.stdout:client6644 2023-05-20T01:26:47.843 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-05-20T01:26:47.843 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2023-05-20T01:26:47.900 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_extra --pg_num_min 64 2023-05-20T01:26:48.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.065+0000 7f3598dd6700 1 Processor -- start 2023-05-20T01:26:48.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.066+0000 7f3598dd6700 1 -- start start 2023-05-20T01:26:48.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.066+0000 7f3598dd6700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f35940b1f60 0x7f359406b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:48.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.066+0000 7f3598dd6700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f359406bb60 0x7f35940b4c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:48.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.066+0000 7f3598dd6700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35940b4770 con 0x7f359415ced0 2023-05-20T01:26:48.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.066+0000 7f3598dd6700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f359406bfe0 con 0x7f359406bb60 2023-05-20T01:26:48.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.066+0000 7f3598dd6700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35940b2820 con 0x7f35940b1f60 2023-05-20T01:26:48.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.067+0000 7f359259c700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f359415ced0 0x7f35940d09b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49858/0 (socket says 172.21.15.16:49858) 2023-05-20T01:26:48.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.067+0000 7f359259c700 1 -- 172.21.15.16:0/1726505401 learned_addr learned my addr 172.21.15.16:0/1726505401 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:26:48.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.067+0000 7f3592d9d700 1 --2- 172.21.15.16:0/1726505401 >> v2:172.21.15.16:3300/0 conn(0x7f359406bb60 0x7f35940b4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:48.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.067+0000 7f3591d9b700 1 --2- 172.21.15.16:0/1726505401 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f35940b1f60 0x7f359406b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:48.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.067+0000 7f359159a700 1 -- 172.21.15.16:0/1726505401 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1426874667 0 0) 0x7f35940b4770 con 0x7f359415ced0 2023-05-20T01:26:48.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.067+0000 7f359159a700 1 -- 172.21.15.16:0/1726505401 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f357c0035b0 con 0x7f359415ced0 2023-05-20T01:26:48.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.067+0000 7f3592d9d700 1 -- 172.21.15.16:0/1726505401 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f35940b1f60 msgr2=0x7f359406b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:48.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.067+0000 7f3592d9d700 1 --2- 172.21.15.16:0/1726505401 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f35940b1f60 0x7f359406b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:48.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.067+0000 7f3592d9d700 1 -- 172.21.15.16:0/1726505401 >> v1:172.21.15.16:6789/0 conn(0x7f359415ced0 legacy=0x7f35940d09b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:48.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.067+0000 7f3592d9d700 1 -- 172.21.15.16:0/1726505401 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35940b2540 con 0x7f359406bb60 2023-05-20T01:26:48.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.067+0000 7f3592d9d700 1 --2- 172.21.15.16:0/1726505401 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f359406bb60 0x7f35940b4c00 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f358c009d90 tx=0x7f358c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e008bda0bb93a9d5 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:48.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.067+0000 7f359159a700 1 -- 172.21.15.16:0/1726505401 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f358c00e070 con 0x7f359406bb60 2023-05-20T01:26:48.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.067+0000 7f359159a700 1 -- 172.21.15.16:0/1726505401 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f358c00bb20 con 0x7f359406bb60 2023-05-20T01:26:48.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.067+0000 7f359159a700 1 -- 172.21.15.16:0/1726505401 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f358c012430 con 0x7f359406bb60 2023-05-20T01:26:48.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.068+0000 7f3598dd6700 1 -- 172.21.15.16:0/1726505401 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f359406bb60 msgr2=0x7f35940b4c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:48.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.068+0000 7f3598dd6700 1 --2- 172.21.15.16:0/1726505401 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f359406bb60 0x7f35940b4c00 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f358c009d90 tx=0x7f358c00b330 comp rx=0 tx=0).stop 2023-05-20T01:26:48.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.068+0000 7f3598dd6700 1 -- 172.21.15.16:0/1726505401 shutdown_connections 2023-05-20T01:26:48.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.068+0000 7f3598dd6700 1 --2- 172.21.15.16:0/1726505401 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f359406bb60 0x7f35940b4c00 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:48.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.068+0000 7f3598dd6700 1 --2- 172.21.15.16:0/1726505401 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f35940b1f60 0x7f359406b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:48.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.068+0000 7f3598dd6700 1 -- 172.21.15.16:0/1726505401 >> 172.21.15.16:0/1726505401 conn(0x7f359409a470 msgr2=0x7f35940d11a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:48.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.068+0000 7f3598dd6700 1 -- 172.21.15.16:0/1726505401 shutdown_connections 2023-05-20T01:26:48.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.068+0000 7f3598dd6700 1 -- 172.21.15.16:0/1726505401 wait complete. 2023-05-20T01:26:48.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.069+0000 7f3598dd6700 1 Processor -- start 2023-05-20T01:26:48.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.069+0000 7f3598dd6700 1 -- start start 2023-05-20T01:26:48.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.070+0000 7f3598dd6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f359406bb60 0x7f359415a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:48.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.070+0000 7f3598dd6700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f35940b1f60 0x7f35940c56e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:48.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.070+0000 7f3598dd6700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f359415ced0 0x7f3594154920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:48.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.070+0000 7f3598dd6700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3594159770 con 0x7f359406bb60 2023-05-20T01:26:48.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.070+0000 7f3598dd6700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3594159a50 con 0x7f35940b1f60 2023-05-20T01:26:48.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.070+0000 7f3598dd6700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3594159d30 con 0x7f359415ced0 2023-05-20T01:26:48.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.070+0000 7f359259c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f359406bb60 0x7f359415a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:48.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.070+0000 7f359259c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f359406bb60 0x7f359415a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38336/0 (socket says 172.21.15.16:38336) 2023-05-20T01:26:48.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.070+0000 7f359259c700 1 -- 172.21.15.16:0/2754919586 learned_addr learned my addr 172.21.15.16:0/2754919586 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:26:48.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.070+0000 7f3592d9d700 1 --2- 172.21.15.16:0/2754919586 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f359415ced0 0x7f3594154920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:48.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.070+0000 7f3591d9b700 1 --2- 172.21.15.16:0/2754919586 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f35940b1f60 0x7f35940c56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:48.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.071+0000 7f359259c700 1 -- 172.21.15.16:0/2754919586 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f359415ced0 msgr2=0x7f3594154920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:48.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.071+0000 7f359259c700 1 --2- 172.21.15.16:0/2754919586 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f359415ced0 0x7f3594154920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:48.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.071+0000 7f359259c700 1 -- 172.21.15.16:0/2754919586 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f35940b1f60 msgr2=0x7f35940c56e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:48.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.071+0000 7f359259c700 1 --2- 172.21.15.16:0/2754919586 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f35940b1f60 0x7f35940c56e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:48.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.071+0000 7f359259c700 1 -- 172.21.15.16:0/2754919586 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f358c009a70 con 0x7f359406bb60 2023-05-20T01:26:48.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.071+0000 7f3591d9b700 1 --2- 172.21.15.16:0/2754919586 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f35940b1f60 0x7f35940c56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:26:48.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.071+0000 7f359259c700 1 --2- 172.21.15.16:0/2754919586 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f359406bb60 0x7f359415a810 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f358400c700 tx=0x7f358400cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:48.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.071+0000 7f3582ffd700 1 -- 172.21.15.16:0/2754919586 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3584005d70 con 0x7f359406bb60 2023-05-20T01:26:48.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.071+0000 7f3582ffd700 1 -- 172.21.15.16:0/2754919586 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35840070d0 con 0x7f359406bb60 2023-05-20T01:26:48.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.071+0000 7f3598dd6700 1 -- 172.21.15.16:0/2754919586 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35940b4400 con 0x7f359406bb60 2023-05-20T01:26:48.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.071+0000 7f3582ffd700 1 -- 172.21.15.16:0/2754919586 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3584007460 con 0x7f359406bb60 2023-05-20T01:26:48.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.071+0000 7f3598dd6700 1 -- 172.21.15.16:0/2754919586 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35940b4120 con 0x7f359406bb60 2023-05-20T01:26:48.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.072+0000 7f3598dd6700 1 -- 172.21.15.16:0/2754919586 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f359415d2a0 con 0x7f359406bb60 2023-05-20T01:26:48.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.072+0000 7f3582ffd700 1 -- 172.21.15.16:0/2754919586 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f35840075c0 con 0x7f359406bb60 2023-05-20T01:26:48.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.073+0000 7f3582ffd700 1 --2- 172.21.15.16:0/2754919586 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f357406ace0 0x7f357406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:48.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.073+0000 7f3582ffd700 1 -- 172.21.15.16:0/2754919586 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f3584088bd0 con 0x7f359406bb60 2023-05-20T01:26:48.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.073+0000 7f3591d9b700 1 --2- 172.21.15.16:0/2754919586 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f357406ace0 0x7f357406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:48.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.073+0000 7f3591d9b700 1 --2- 172.21.15.16:0/2754919586 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f357406ace0 0x7f357406d190 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f359415b660 tx=0x7f3588008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:48.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.077+0000 7f3582ffd700 1 -- 172.21.15.16:0/2754919586 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f3584053b50 con 0x7f359406bb60 2023-05-20T01:26:48.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.278+0000 7f3598dd6700 1 -- 172.21.15.16:0/2754919586 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_extra", "pg_num_min": 64} v 0) v1 -- 0x7f35940747b0 con 0x7f359406bb60 2023-05-20T01:26:48.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.526+0000 7f3582ffd700 1 -- 172.21.15.16:0/2754919586 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_extra", "pg_num_min": 64}]=0 pool 'cephfs_data_extra' created v113) v1 ==== 142+0+0 (secure 0 0 0) 0x7f3584057170 con 0x7f359406bb60 2023-05-20T01:26:48.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.530+0000 7f3598dd6700 1 -- 172.21.15.16:0/2754919586 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f357406ace0 msgr2=0x7f357406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:48.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.530+0000 7f3598dd6700 1 --2- 172.21.15.16:0/2754919586 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f357406ace0 0x7f357406d190 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f359415b660 tx=0x7f3588008040 comp rx=0 tx=0).stop 2023-05-20T01:26:48.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.530+0000 7f3598dd6700 1 -- 172.21.15.16:0/2754919586 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f359406bb60 msgr2=0x7f359415a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:48.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.530+0000 7f3598dd6700 1 --2- 172.21.15.16:0/2754919586 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f359406bb60 0x7f359415a810 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f358400c700 tx=0x7f358400cac0 comp rx=0 tx=0).stop 2023-05-20T01:26:48.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.530+0000 7f3598dd6700 1 -- 172.21.15.16:0/2754919586 shutdown_connections 2023-05-20T01:26:48.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.530+0000 7f3598dd6700 1 --2- 172.21.15.16:0/2754919586 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f357406ace0 0x7f357406d190 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:48.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.530+0000 7f3598dd6700 1 --2- 172.21.15.16:0/2754919586 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f359415ced0 0x7f3594154920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:48.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.530+0000 7f3598dd6700 1 --2- 172.21.15.16:0/2754919586 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f359406bb60 0x7f359415a810 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:48.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.530+0000 7f3598dd6700 1 --2- 172.21.15.16:0/2754919586 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f35940b1f60 0x7f35940c56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:48.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.530+0000 7f3598dd6700 1 -- 172.21.15.16:0/2754919586 >> 172.21.15.16:0/2754919586 conn(0x7f359409a470 msgr2=0x7f359415bc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:48.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.530+0000 7f3598dd6700 1 -- 172.21.15.16:0/2754919586 shutdown_connections 2023-05-20T01:26:48.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.530+0000 7f3598dd6700 1 -- 172.21.15.16:0/2754919586 wait complete. 2023-05-20T01:26:48.542 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data_extra' created 2023-05-20T01:26:48.551 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_extra 2023-05-20T01:26:48.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.674+0000 7fbce86d2700 1 Processor -- start 2023-05-20T01:26:48.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.674+0000 7fbce86d2700 1 -- start start 2023-05-20T01:26:48.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.675+0000 7fbce86d2700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fbce015cec0 0x7fbce0071bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:48.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.675+0000 7fbce86d2700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbce0072130 0x7fbce00cf9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:48.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.675+0000 7fbce86d2700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbce00cd630 con 0x7fbce015d290 2023-05-20T01:26:48.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.675+0000 7fbce86d2700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbce0072570 con 0x7fbce015cec0 2023-05-20T01:26:48.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.675+0000 7fbce86d2700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbce00cca60 con 0x7fbce0072130 2023-05-20T01:26:48.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.675+0000 7fbce646e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fbce015cec0 0x7fbce0071bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:48.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.676+0000 7fbce6c6f700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbce0072130 0x7fbce00cf9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:48.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.676+0000 7fbce646e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fbce015cec0 0x7fbce0071bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.16:35366/0 (socket says 172.21.15.16:35366) 2023-05-20T01:26:48.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.676+0000 7fbce646e700 1 -- 172.21.15.16:0/1239661128 learned_addr learned my addr 172.21.15.16:0/1239661128 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:26:48.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.676+0000 7fbce646e700 1 -- 172.21.15.16:0/1239661128 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbce0072130 msgr2=0x7fbce00cf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:48.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.676+0000 7fbce646e700 1 --2- 172.21.15.16:0/1239661128 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbce0072130 0x7fbce00cf9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:48.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.676+0000 7fbce646e700 1 -- 172.21.15.16:0/1239661128 >> v1:172.21.15.132:6789/0 conn(0x7fbce015d290 legacy=0x7fbce00cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:48.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.676+0000 7fbce646e700 1 -- 172.21.15.16:0/1239661128 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbce00ccd00 con 0x7fbce015cec0 2023-05-20T01:26:48.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.676+0000 7fbce6c6f700 1 --2- 172.21.15.16:0/1239661128 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbce0072130 0x7fbce00cf9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-05-20T01:26:48.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.676+0000 7fbce646e700 1 --2- 172.21.15.16:0/1239661128 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbce015cec0 0x7fbce0071bf0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fbcd0006fd0 tx=0x7fbcd0009bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=80cd961bef3fc61c server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:48.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.676+0000 7fbce546c700 1 -- 172.21.15.16:0/1239661128 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcd0005e30 con 0x7fbce015cec0 2023-05-20T01:26:48.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.676+0000 7fbce546c700 1 -- 172.21.15.16:0/1239661128 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcd0004030 con 0x7fbce015cec0 2023-05-20T01:26:48.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.676+0000 7fbce546c700 1 -- 172.21.15.16:0/1239661128 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcd00043e0 con 0x7fbce015cec0 2023-05-20T01:26:48.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.677+0000 7fbce86d2700 1 -- 172.21.15.16:0/1239661128 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbce015cec0 msgr2=0x7fbce0071bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:48.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.677+0000 7fbce86d2700 1 --2- 172.21.15.16:0/1239661128 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbce015cec0 0x7fbce0071bf0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fbcd0006fd0 tx=0x7fbcd0009bd0 comp rx=0 tx=0).stop 2023-05-20T01:26:48.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.677+0000 7fbce86d2700 1 -- 172.21.15.16:0/1239661128 shutdown_connections 2023-05-20T01:26:48.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.677+0000 7fbce86d2700 1 --2- 172.21.15.16:0/1239661128 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbce0072130 0x7fbce00cf9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:48.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.677+0000 7fbce86d2700 1 --2- 172.21.15.16:0/1239661128 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbce015cec0 0x7fbce0071bf0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:48.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.677+0000 7fbce86d2700 1 -- 172.21.15.16:0/1239661128 >> 172.21.15.16:0/1239661128 conn(0x7fbce00cffd0 msgr2=0x7fbce00d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:48.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.677+0000 7fbce86d2700 1 -- 172.21.15.16:0/1239661128 shutdown_connections 2023-05-20T01:26:48.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.677+0000 7fbce86d2700 1 -- 172.21.15.16:0/1239661128 wait complete. 2023-05-20T01:26:48.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.678+0000 7fbce86d2700 1 Processor -- start 2023-05-20T01:26:48.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.678+0000 7fbce86d2700 1 -- start start 2023-05-20T01:26:48.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.679+0000 7fbce86d2700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbce0072130 0x7fbce006dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:48.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.679+0000 7fbce86d2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbce015cec0 0x7fbce01522b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:48.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.679+0000 7fbce86d2700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbce015d290 0x7fbce01527f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:48.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.679+0000 7fbce86d2700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbce00c6f40 con 0x7fbce015cec0 2023-05-20T01:26:48.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.679+0000 7fbce86d2700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbce00b5b60 con 0x7fbce015d290 2023-05-20T01:26:48.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.679+0000 7fbce86d2700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbce00b4ac0 con 0x7fbce0072130 2023-05-20T01:26:48.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.679+0000 7fbce5c6d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbce015cec0 0x7fbce01522b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:48.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.679+0000 7fbce5c6d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbce015cec0 0x7fbce01522b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38354/0 (socket says 172.21.15.16:38354) 2023-05-20T01:26:48.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.679+0000 7fbce5c6d700 1 -- 172.21.15.16:0/3062910880 learned_addr learned my addr 172.21.15.16:0/3062910880 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:26:48.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.679+0000 7fbce6c6f700 1 --2- 172.21.15.16:0/3062910880 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbce015d290 0x7fbce01527f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:48.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.679+0000 7fbce646e700 1 --2- 172.21.15.16:0/3062910880 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbce0072130 0x7fbce006dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:48.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.679+0000 7fbce5c6d700 1 -- 172.21.15.16:0/3062910880 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbce0072130 msgr2=0x7fbce006dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:48.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.679+0000 7fbce5c6d700 1 --2- 172.21.15.16:0/3062910880 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbce0072130 0x7fbce006dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:48.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.679+0000 7fbce5c6d700 1 -- 172.21.15.16:0/3062910880 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbce015d290 msgr2=0x7fbce01527f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:48.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.679+0000 7fbce5c6d700 1 --2- 172.21.15.16:0/3062910880 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbce015d290 0x7fbce01527f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:48.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.679+0000 7fbce5c6d700 1 -- 172.21.15.16:0/3062910880 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcd8009a70 con 0x7fbce015cec0 2023-05-20T01:26:48.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.680+0000 7fbce5c6d700 1 --2- 172.21.15.16:0/3062910880 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbce015cec0 0x7fbce01522b0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc00c450 tx=0x7fbcdc00c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:48.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.680+0000 7fbcd6ffd700 1 -- 172.21.15.16:0/3062910880 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcdc0052d0 con 0x7fbce015cec0 2023-05-20T01:26:48.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.680+0000 7fbcd6ffd700 1 -- 172.21.15.16:0/3062910880 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcdc005430 con 0x7fbce015cec0 2023-05-20T01:26:48.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.680+0000 7fbce86d2700 1 -- 172.21.15.16:0/3062910880 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcd0009500 con 0x7fbce015cec0 2023-05-20T01:26:48.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.680+0000 7fbcd6ffd700 1 -- 172.21.15.16:0/3062910880 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcdc0038c0 con 0x7fbce015cec0 2023-05-20T01:26:48.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.680+0000 7fbce86d2700 1 -- 172.21.15.16:0/3062910880 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbce00cd2c0 con 0x7fbce015cec0 2023-05-20T01:26:48.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.681+0000 7fbce86d2700 1 -- 172.21.15.16:0/3062910880 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbce00ccfe0 con 0x7fbce015cec0 2023-05-20T01:26:48.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.682+0000 7fbcd6ffd700 1 -- 172.21.15.16:0/3062910880 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fbcdc003a20 con 0x7fbce015cec0 2023-05-20T01:26:48.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.682+0000 7fbcd6ffd700 1 --2- 172.21.15.16:0/3062910880 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fbcc406ace0 0x7fbcc406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:48.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.682+0000 7fbcd6ffd700 1 -- 172.21.15.16:0/3062910880 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6754+0+0 (secure 0 0 0) 0x7fbcdc088db0 con 0x7fbce015cec0 2023-05-20T01:26:48.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.682+0000 7fbce646e700 1 --2- 172.21.15.16:0/3062910880 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fbcc406ace0 0x7fbcc406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:48.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.683+0000 7fbce646e700 1 --2- 172.21.15.16:0/3062910880 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fbcc406ace0 0x7fbcc406d190 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fbcd0006fd0 tx=0x7fbcd0008070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:48.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.686+0000 7fbcd6ffd700 1 -- 172.21.15.16:0/3062910880 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fbcdc053c80 con 0x7fbce015cec0 2023-05-20T01:26:48.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.933+0000 7fbce86d2700 1 -- 172.21.15.16:0/3062910880 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_extra"} v 0) v1 -- 0x7fbce0160890 con 0x7fbce015cec0 2023-05-20T01:26:48.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.938+0000 7fbcd6ffd700 1 -- 172.21.15.16:0/3062910880 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_extra"}]=0 added data pool 17 to fsmap v79) v1 ==== 141+0+0 (secure 0 0 0) 0x7fbcdc0572a0 con 0x7fbce015cec0 2023-05-20T01:26:48.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.941+0000 7fbce86d2700 1 -- 172.21.15.16:0/3062910880 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fbcc406ace0 msgr2=0x7fbcc406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:48.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.941+0000 7fbce86d2700 1 --2- 172.21.15.16:0/3062910880 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fbcc406ace0 0x7fbcc406d190 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fbcd0006fd0 tx=0x7fbcd0008070 comp rx=0 tx=0).stop 2023-05-20T01:26:48.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.941+0000 7fbce86d2700 1 -- 172.21.15.16:0/3062910880 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbce015cec0 msgr2=0x7fbce01522b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:48.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.941+0000 7fbce86d2700 1 --2- 172.21.15.16:0/3062910880 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbce015cec0 0x7fbce01522b0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc00c450 tx=0x7fbcdc00c760 comp rx=0 tx=0).stop 2023-05-20T01:26:48.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.941+0000 7fbce86d2700 1 -- 172.21.15.16:0/3062910880 shutdown_connections 2023-05-20T01:26:48.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.941+0000 7fbce86d2700 1 --2- 172.21.15.16:0/3062910880 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbce015d290 0x7fbce01527f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:48.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.941+0000 7fbce86d2700 1 --2- 172.21.15.16:0/3062910880 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fbcc406ace0 0x7fbcc406d190 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:48.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.941+0000 7fbce86d2700 1 --2- 172.21.15.16:0/3062910880 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbce0072130 0x7fbce006dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:48.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.941+0000 7fbce86d2700 1 --2- 172.21.15.16:0/3062910880 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbce015cec0 0x7fbce01522b0 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:48.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.941+0000 7fbce86d2700 1 -- 172.21.15.16:0/3062910880 >> 172.21.15.16:0/3062910880 conn(0x7fbce00cffd0 msgr2=0x7fbce015ef80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:48.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.942+0000 7fbce86d2700 1 -- 172.21.15.16:0/3062910880 shutdown_connections 2023-05-20T01:26:48.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:48.942+0000 7fbce86d2700 1 -- 172.21.15.16:0/3062910880 wait complete. 2023-05-20T01:26:48.953 INFO:teuthology.orchestra.run.smithi016.stderr:added data pool 17 to fsmap 2023-05-20T01:26:48.964 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-20T01:26:49.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.084+0000 7f6c67a92700 1 Processor -- start 2023-05-20T01:26:49.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.084+0000 7f6c67a92700 1 -- start start 2023-05-20T01:26:49.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.085+0000 7f6c67a92700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f6c6006ac20 0x7f6c6006e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:49.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.085+0000 7f6c67a92700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c600b1e30 con 0x7f6c60072040 2023-05-20T01:26:49.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.085+0000 7f6c67a92700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c600b2110 con 0x7f6c6006e170 2023-05-20T01:26:49.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.085+0000 7f6c67a92700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c6015d050 con 0x7f6c6006ac20 2023-05-20T01:26:49.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.085+0000 7f6c6582e700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f6c60072040 0x7f6c600cec20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49886/0 (socket says 172.21.15.16:49886) 2023-05-20T01:26:49.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.085+0000 7f6c6582e700 1 -- 172.21.15.16:0/3522606073 learned_addr learned my addr 172.21.15.16:0/3522606073 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:26:49.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.085+0000 7f6c6602f700 1 --2- 172.21.15.16:0/3522606073 >> v2:172.21.15.132:3300/0 conn(0x7f6c6006ac20 0x7f6c6006e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:49.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.085+0000 7f6c6482c700 1 -- 172.21.15.16:0/3522606073 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 908815879 0 0) 0x7f6c600b1e30 con 0x7f6c60072040 2023-05-20T01:26:49.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.086+0000 7f6c6482c700 1 -- 172.21.15.16:0/3522606073 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c4c0035b0 con 0x7f6c60072040 2023-05-20T01:26:49.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.086+0000 7f6c6482c700 1 -- 172.21.15.16:0/3522606073 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 343334168 0 0) 0x7f6c4c0035b0 con 0x7f6c60072040 2023-05-20T01:26:49.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.086+0000 7f6c6482c700 1 -- 172.21.15.16:0/3522606073 >> v2:172.21.15.132:3300/0 conn(0x7f6c6006ac20 msgr2=0x7f6c6006e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:49.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.086+0000 7f6c6482c700 1 --2- 172.21.15.16:0/3522606073 >> v2:172.21.15.132:3300/0 conn(0x7f6c6006ac20 0x7f6c6006e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:49.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.086+0000 7f6c6482c700 1 -- 172.21.15.16:0/3522606073 >> v1:172.21.15.132:6789/0 conn(0x7f6c6006e170 legacy=0x7f6c6006a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:49.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.086+0000 7f6c6482c700 1 -- 172.21.15.16:0/3522606073 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c600b23f0 con 0x7f6c60072040 2023-05-20T01:26:49.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.086+0000 7f6c6482c700 1 -- 172.21.15.16:0/3522606073 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7f6c50001670 con 0x7f6c60072040 2023-05-20T01:26:49.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.086+0000 7f6c6482c700 1 -- 172.21.15.16:0/3522606073 >> v1:172.21.15.16:6789/0 conn(0x7f6c60072040 legacy=0x7f6c600cec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:49.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.086+0000 7f6c6482c700 1 --2- 172.21.15.16:0/3522606073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c4c003a70 0x7f6c4c003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:49.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.086+0000 7f6c6482c700 1 --2- 172.21.15.16:0/3522606073 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6c4c004850 0x7f6c4c004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:49.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.086+0000 7f6c6482c700 1 --2- 172.21.15.16:0/3522606073 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6c4c005200 0x7f6c4c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:49.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.086+0000 7f6c6482c700 1 -- 172.21.15.16:0/3522606073 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c4c00c0e0 con 0x7f6c4c003a70 2023-05-20T01:26:49.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.086+0000 7f6c6482c700 1 -- 172.21.15.16:0/3522606073 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c4c00c250 con 0x7f6c4c004850 2023-05-20T01:26:49.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.086+0000 7f6c6482c700 1 -- 172.21.15.16:0/3522606073 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c4c00c410 con 0x7f6c4c005200 2023-05-20T01:26:49.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.086+0000 7f6c6582e700 1 --2- 172.21.15.16:0/3522606073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c4c003a70 0x7f6c4c003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:49.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.086+0000 7f6c6502d700 1 --2- 172.21.15.16:0/3522606073 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6c4c004850 0x7f6c4c004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:49.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.086+0000 7f6c6602f700 1 --2- 172.21.15.16:0/3522606073 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6c4c005200 0x7f6c4c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:49.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.086+0000 7f6c6582e700 1 -- 172.21.15.16:0/3522606073 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6c4c005200 msgr2=0x7f6c4c00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:49.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.086+0000 7f6c6582e700 1 --2- 172.21.15.16:0/3522606073 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6c4c005200 0x7f6c4c00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:49.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.086+0000 7f6c6582e700 1 -- 172.21.15.16:0/3522606073 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6c4c004850 msgr2=0x7f6c4c004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:49.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.086+0000 7f6c6582e700 1 --2- 172.21.15.16:0/3522606073 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6c4c004850 0x7f6c4c004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:49.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.086+0000 7f6c6582e700 1 -- 172.21.15.16:0/3522606073 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c4c0035b0 con 0x7f6c4c003a70 2023-05-20T01:26:49.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.086+0000 7f6c6602f700 1 -- 172.21.15.16:0/3522606073 reap_dead start 2023-05-20T01:26:49.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.086+0000 7f6c6582e700 1 --2- 172.21.15.16:0/3522606073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c4c003a70 0x7f6c4c003e40 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f6c50001eb0 tx=0x7f6c4c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=689802ed857214af server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:49.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.087+0000 7f6c6482c700 1 -- 172.21.15.16:0/3522606073 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c5000ce00 con 0x7f6c4c003a70 2023-05-20T01:26:49.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.087+0000 7f6c6482c700 1 -- 172.21.15.16:0/3522606073 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c50005960 con 0x7f6c4c003a70 2023-05-20T01:26:49.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.087+0000 7f6c6482c700 1 -- 172.21.15.16:0/3522606073 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c50005cf0 con 0x7f6c4c003a70 2023-05-20T01:26:49.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.087+0000 7f6c67a92700 1 -- 172.21.15.16:0/3522606073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c4c003a70 msgr2=0x7f6c4c003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:49.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.087+0000 7f6c67a92700 1 --2- 172.21.15.16:0/3522606073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c4c003a70 0x7f6c4c003e40 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f6c50001eb0 tx=0x7f6c4c005880 comp rx=0 tx=0).stop 2023-05-20T01:26:49.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.087+0000 7f6c67a92700 1 -- 172.21.15.16:0/3522606073 shutdown_connections 2023-05-20T01:26:49.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.087+0000 7f6c67a92700 1 --2- 172.21.15.16:0/3522606073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c4c003a70 0x7f6c4c003e40 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:49.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.087+0000 7f6c67a92700 1 -- 172.21.15.16:0/3522606073 >> 172.21.15.16:0/3522606073 conn(0x7f6c600cca90 msgr2=0x7f6c600cce90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:49.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.087+0000 7f6c67a92700 1 -- 172.21.15.16:0/3522606073 shutdown_connections 2023-05-20T01:26:49.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.087+0000 7f6c67a92700 1 -- 172.21.15.16:0/3522606073 wait complete. 2023-05-20T01:26:49.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.088+0000 7f6c67a92700 1 Processor -- start 2023-05-20T01:26:49.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.088+0000 7f6c67a92700 1 -- start start 2023-05-20T01:26:49.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.089+0000 7f6c67a92700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6c4c003a70 0x7f6c6014b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:49.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.089+0000 7f6c67a92700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6c6015e700 0x7f6c600c6850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:49.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.089+0000 7f6c67a92700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c6015eec0 0x7f6c6015f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:49.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.089+0000 7f6c67a92700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c6014ba10 con 0x7f6c6015eec0 2023-05-20T01:26:49.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.089+0000 7f6c67a92700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c6015c010 con 0x7f6c4c003a70 2023-05-20T01:26:49.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.089+0000 7f6c67a92700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c6015f8a0 con 0x7f6c6015e700 2023-05-20T01:26:49.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.089+0000 7f6c6602f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c6015eec0 0x7f6c6015f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:49.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.089+0000 7f6c6602f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c6015eec0 0x7f6c6015f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38374/0 (socket says 172.21.15.16:38374) 2023-05-20T01:26:49.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.089+0000 7f6c6602f700 1 -- 172.21.15.16:0/3196593349 learned_addr learned my addr 172.21.15.16:0/3196593349 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:26:49.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.089+0000 7f6c6502d700 1 --2- 172.21.15.16:0/3196593349 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6c6015e700 0x7f6c600c6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:49.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.089+0000 7f6c6582e700 1 --2- 172.21.15.16:0/3196593349 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6c4c003a70 0x7f6c6014b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:49.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.090+0000 7f6c6602f700 1 -- 172.21.15.16:0/3196593349 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6c6015e700 msgr2=0x7f6c600c6850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:49.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.090+0000 7f6c6602f700 1 --2- 172.21.15.16:0/3196593349 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6c6015e700 0x7f6c600c6850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:49.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.090+0000 7f6c6602f700 1 -- 172.21.15.16:0/3196593349 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6c4c003a70 msgr2=0x7f6c6014b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:49.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.090+0000 7f6c6602f700 1 --2- 172.21.15.16:0/3196593349 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6c4c003a70 0x7f6c6014b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:49.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.090+0000 7f6c6602f700 1 -- 172.21.15.16:0/3196593349 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c60072410 con 0x7f6c6015eec0 2023-05-20T01:26:49.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.090+0000 7f6c6502d700 1 --2- 172.21.15.16:0/3196593349 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6c6015e700 0x7f6c600c6850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:26:49.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.090+0000 7f6c6602f700 1 --2- 172.21.15.16:0/3196593349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c6015eec0 0x7f6c6015f360 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f6c600c3ae0 tx=0x7f6c580114b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:49.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.090+0000 7f6c567fc700 1 -- 172.21.15.16:0/3196593349 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c5800e070 con 0x7f6c6015eec0 2023-05-20T01:26:49.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.090+0000 7f6c567fc700 1 -- 172.21.15.16:0/3196593349 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c58003c30 con 0x7f6c6015eec0 2023-05-20T01:26:49.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.090+0000 7f6c67a92700 1 -- 172.21.15.16:0/3196593349 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c600b26d0 con 0x7f6c6015eec0 2023-05-20T01:26:49.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.090+0000 7f6c567fc700 1 -- 172.21.15.16:0/3196593349 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c58003de0 con 0x7f6c6015eec0 2023-05-20T01:26:49.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.090+0000 7f6c67a92700 1 -- 172.21.15.16:0/3196593349 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c600b1a90 con 0x7f6c6015eec0 2023-05-20T01:26:49.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.091+0000 7f6c67a92700 1 -- 172.21.15.16:0/3196593349 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c6004e8c0 con 0x7f6c6015eec0 2023-05-20T01:26:49.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.091+0000 7f6c567fc700 1 -- 172.21.15.16:0/3196593349 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6c58011a10 con 0x7f6c6015eec0 2023-05-20T01:26:49.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.092+0000 7f6c567fc700 1 --2- 172.21.15.16:0/3196593349 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f6c4406ace0 0x7f6c4406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:49.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.092+0000 7f6c567fc700 1 -- 172.21.15.16:0/3196593349 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6786+0+0 (secure 0 0 0) 0x7f6c58089d30 con 0x7f6c6015eec0 2023-05-20T01:26:49.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.092+0000 7f6c6582e700 1 --2- 172.21.15.16:0/3196593349 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f6c4406ace0 0x7f6c4406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:49.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.092+0000 7f6c6582e700 1 --2- 172.21.15.16:0/3196593349 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f6c4406ace0 0x7f6c4406d190 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f6c50000c00 tx=0x7f6c50007000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:49.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.096+0000 7f6c567fc700 1 -- 172.21.15.16:0/3196593349 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f6c58054b00 con 0x7f6c6015eec0 2023-05-20T01:26:49.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.337+0000 7f6c67a92700 1 -- 172.21.15.16:0/3196593349 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6c600cd920 con 0x7f6c6015eec0 2023-05-20T01:26:49.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.337+0000 7f6c567fc700 1 -- 172.21.15.16:0/3196593349 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) v1 ==== 94+0+4679 (secure 0 0 0) 0x7f6c58058120 con 0x7f6c6015eec0 2023-05-20T01:26:49.342 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:26:49.342 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":79,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7069,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/3619911175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":3619911175},{"type":"v1","addr":"172.21.15.16:6837","nonce":3619911175}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73},{"gid":7081,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/959638003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":959638003},{"type":"v1","addr":"172.21.15.132:6834","nonce":959638003}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73},{"gid":7084,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2596482253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":2596482253},{"type":"v1","addr":"172.21.15.132:6835","nonce":2596482253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-20T01:26:36.067393+0000","modified":"2023-05-20T01:26:48.935135+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7896},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7896":{"gid":7896,"name":"a","rank":0,"incarnation":74,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6836/3765848068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3765848068},{"type":"v1","addr":"172.21.15.16:6836","nonce":3765848068}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16,17],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-05-20T01:26:49.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.340+0000 7f6c67a92700 1 -- 172.21.15.16:0/3196593349 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f6c4406ace0 msgr2=0x7f6c4406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:49.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.340+0000 7f6c67a92700 1 --2- 172.21.15.16:0/3196593349 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f6c4406ace0 0x7f6c4406d190 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f6c50000c00 tx=0x7f6c50007000 comp rx=0 tx=0).stop 2023-05-20T01:26:49.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.340+0000 7f6c67a92700 1 -- 172.21.15.16:0/3196593349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c6015eec0 msgr2=0x7f6c6015f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:49.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.340+0000 7f6c67a92700 1 --2- 172.21.15.16:0/3196593349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c6015eec0 0x7f6c6015f360 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f6c600c3ae0 tx=0x7f6c580114b0 comp rx=0 tx=0).stop 2023-05-20T01:26:49.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.340+0000 7f6c67a92700 1 -- 172.21.15.16:0/3196593349 shutdown_connections 2023-05-20T01:26:49.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.340+0000 7f6c67a92700 1 --2- 172.21.15.16:0/3196593349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c6015eec0 0x7f6c6015f360 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:49.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.340+0000 7f6c67a92700 1 --2- 172.21.15.16:0/3196593349 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6c4c003a70 0x7f6c6014b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:49.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.340+0000 7f6c67a92700 1 --2- 172.21.15.16:0/3196593349 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f6c4406ace0 0x7f6c4406d190 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:49.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.340+0000 7f6c67a92700 1 --2- 172.21.15.16:0/3196593349 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6c6015e700 0x7f6c600c6850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:49.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.340+0000 7f6c67a92700 1 -- 172.21.15.16:0/3196593349 >> 172.21.15.16:0/3196593349 conn(0x7f6c600cca90 msgr2=0x7f6c6015c590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:49.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.340+0000 7f6c67a92700 1 -- 172.21.15.16:0/3196593349 shutdown_connections 2023-05-20T01:26:49.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.340+0000 7f6c67a92700 1 -- 172.21.15.16:0/3196593349 wait complete. 2023-05-20T01:26:49.346 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 79 2023-05-20T01:26:49.356 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-20T01:26:49.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.478+0000 7f348423e700 1 Processor -- start 2023-05-20T01:26:49.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.478+0000 7f348423e700 1 -- start start 2023-05-20T01:26:49.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.479+0000 7f348423e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f347c06f450 0x7f347c0cf890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:49.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.479+0000 7f348423e700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f347c06d4b0 0x7f347c161080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:49.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.479+0000 7f348423e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f347c06a510 0x7f347c06a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:49.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.479+0000 7f348423e700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f347c06aef0 con 0x7f347c06d4b0 2023-05-20T01:26:49.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.479+0000 7f348423e700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f347c0cc510 con 0x7f347c06a510 2023-05-20T01:26:49.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.479+0000 7f348423e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f347c1615c0 con 0x7f347c06f450 2023-05-20T01:26:49.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.479+0000 7f34817d9700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f347c06d4b0 0x7f347c161080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:49.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.480+0000 7f34817d9700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f347c06d4b0 0x7f347c161080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38378/0 (socket says 172.21.15.16:38378) 2023-05-20T01:26:49.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.480+0000 7f34817d9700 1 -- 172.21.15.16:0/3501103371 learned_addr learned my addr 172.21.15.16:0/3501103371 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:26:49.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.480+0000 7f34827db700 1 --2- 172.21.15.16:0/3501103371 >> v2:172.21.15.132:3300/0 conn(0x7f347c06a510 0x7f347c06a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:49.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.480+0000 7f3481fda700 1 --2- 172.21.15.16:0/3501103371 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f347c06f450 0x7f347c0cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:49.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.480+0000 7f34817d9700 1 -- 172.21.15.16:0/3501103371 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f347c06f450 msgr2=0x7f347c0cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:49.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.480+0000 7f34817d9700 1 --2- 172.21.15.16:0/3501103371 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f347c06f450 0x7f347c0cf890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:49.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.480+0000 7f34817d9700 1 -- 172.21.15.16:0/3501103371 >> v2:172.21.15.132:3300/0 conn(0x7f347c06a510 msgr2=0x7f347c06a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:26:49.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.480+0000 7f34817d9700 1 --2- 172.21.15.16:0/3501103371 >> v2:172.21.15.132:3300/0 conn(0x7f347c06a510 0x7f347c06a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:49.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.480+0000 7f34817d9700 1 -- 172.21.15.16:0/3501103371 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f347c071ed0 con 0x7f347c06d4b0 2023-05-20T01:26:49.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.480+0000 7f34827db700 1 --2- 172.21.15.16:0/3501103371 >> v2:172.21.15.132:3300/0 conn(0x7f347c06a510 0x7f347c06a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:26:49.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.480+0000 7f3481fda700 1 --2- 172.21.15.16:0/3501103371 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f347c06f450 0x7f347c0cf890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:26:49.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.480+0000 7f34817d9700 1 --2- 172.21.15.16:0/3501103371 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f347c06d4b0 0x7f347c161080 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f347c0c7010 tx=0x7f3478009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4498eab0144d33d6 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:49.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.480+0000 7f3480fd8700 1 -- 172.21.15.16:0/3501103371 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f347800b870 con 0x7f347c06d4b0 2023-05-20T01:26:49.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.480+0000 7f3480fd8700 1 -- 172.21.15.16:0/3501103371 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f347800b9d0 con 0x7f347c06d4b0 2023-05-20T01:26:49.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.480+0000 7f3480fd8700 1 -- 172.21.15.16:0/3501103371 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f347800bd60 con 0x7f347c06d4b0 2023-05-20T01:26:49.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.481+0000 7f348423e700 1 -- 172.21.15.16:0/3501103371 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f347c06d4b0 msgr2=0x7f347c161080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:49.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.481+0000 7f348423e700 1 --2- 172.21.15.16:0/3501103371 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f347c06d4b0 0x7f347c161080 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f347c0c7010 tx=0x7f3478009bd0 comp rx=0 tx=0).stop 2023-05-20T01:26:49.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.481+0000 7f348423e700 1 -- 172.21.15.16:0/3501103371 shutdown_connections 2023-05-20T01:26:49.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.481+0000 7f348423e700 1 --2- 172.21.15.16:0/3501103371 >> v2:172.21.15.132:3300/0 conn(0x7f347c06a510 0x7f347c06a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:49.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.481+0000 7f348423e700 1 --2- 172.21.15.16:0/3501103371 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f347c06f450 0x7f347c0cf890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:49.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.481+0000 7f348423e700 1 --2- 172.21.15.16:0/3501103371 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f347c06d4b0 0x7f347c161080 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:49.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.481+0000 7f348423e700 1 -- 172.21.15.16:0/3501103371 >> 172.21.15.16:0/3501103371 conn(0x7f347c0cffb0 msgr2=0x7f347c0d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:49.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.481+0000 7f348423e700 1 -- 172.21.15.16:0/3501103371 shutdown_connections 2023-05-20T01:26:49.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.481+0000 7f348423e700 1 -- 172.21.15.16:0/3501103371 wait complete. 2023-05-20T01:26:49.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.482+0000 7f348423e700 1 Processor -- start 2023-05-20T01:26:49.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.483+0000 7f348423e700 1 -- start start 2023-05-20T01:26:49.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.483+0000 7f348423e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f347c06a510 0x7f347c06daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:49.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.483+0000 7f348423e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f347c06d4b0 0x7f347c155730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:49.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.483+0000 7f348423e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f347c06f450 0x7f347c1f2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:49.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.483+0000 7f348423e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f347c1552a0 con 0x7f347c06d4b0 2023-05-20T01:26:49.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.483+0000 7f348423e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f347c154f50 con 0x7f347c06f450 2023-05-20T01:26:49.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.483+0000 7f348423e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f347c1f30d0 con 0x7f347c06a510 2023-05-20T01:26:49.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.484+0000 7f34817d9700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f347c06d4b0 0x7f347c155730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:49.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.484+0000 7f34817d9700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f347c06d4b0 0x7f347c155730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38384/0 (socket says 172.21.15.16:38384) 2023-05-20T01:26:49.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.484+0000 7f34817d9700 1 -- 172.21.15.16:0/892829189 learned_addr learned my addr 172.21.15.16:0/892829189 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:26:49.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.484+0000 7f34827db700 1 --2- 172.21.15.16:0/892829189 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f347c06f450 0x7f347c1f2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:49.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.484+0000 7f3481fda700 1 --2- 172.21.15.16:0/892829189 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f347c06a510 0x7f347c06daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:49.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.484+0000 7f34817d9700 1 -- 172.21.15.16:0/892829189 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f347c06a510 msgr2=0x7f347c06daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:49.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.484+0000 7f34817d9700 1 --2- 172.21.15.16:0/892829189 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f347c06a510 0x7f347c06daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:49.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.484+0000 7f34817d9700 1 -- 172.21.15.16:0/892829189 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f347c06f450 msgr2=0x7f347c1f2610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:49.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.484+0000 7f34817d9700 1 --2- 172.21.15.16:0/892829189 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f347c06f450 0x7f347c1f2610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:49.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.484+0000 7f34817d9700 1 -- 172.21.15.16:0/892829189 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3478009550 con 0x7f347c06d4b0 2023-05-20T01:26:49.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.484+0000 7f34817d9700 1 --2- 172.21.15.16:0/892829189 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f347c06d4b0 0x7f347c155730 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f3478000c00 tx=0x7f3478004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:49.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.484+0000 7f34727fc700 1 -- 172.21.15.16:0/892829189 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3478004540 con 0x7f347c06d4b0 2023-05-20T01:26:49.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.484+0000 7f34727fc700 1 -- 172.21.15.16:0/892829189 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34780036a0 con 0x7f347c06d4b0 2023-05-20T01:26:49.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.484+0000 7f34727fc700 1 -- 172.21.15.16:0/892829189 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3478003a50 con 0x7f347c06d4b0 2023-05-20T01:26:49.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.484+0000 7f348423e700 1 -- 172.21.15.16:0/892829189 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f347c0cc1a0 con 0x7f347c06d4b0 2023-05-20T01:26:49.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.484+0000 7f348423e700 1 -- 172.21.15.16:0/892829189 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f347c0721b0 con 0x7f347c06d4b0 2023-05-20T01:26:49.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.485+0000 7f348423e700 1 -- 172.21.15.16:0/892829189 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f347c0cbe50 con 0x7f347c06d4b0 2023-05-20T01:26:49.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.486+0000 7f34727fc700 1 -- 172.21.15.16:0/892829189 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3478021070 con 0x7f347c06d4b0 2023-05-20T01:26:49.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.486+0000 7f34727fc700 1 --2- 172.21.15.16:0/892829189 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f346006ace0 0x7f346006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:26:49.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.486+0000 7f34727fc700 1 -- 172.21.15.16:0/892829189 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6786+0+0 (secure 0 0 0) 0x7f347808ae40 con 0x7f347c06d4b0 2023-05-20T01:26:49.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.486+0000 7f3481fda700 1 --2- 172.21.15.16:0/892829189 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f346006ace0 0x7f346006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:26:49.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.487+0000 7f3481fda700 1 --2- 172.21.15.16:0/892829189 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f346006ace0 0x7f346006d190 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f347c0cb580 tx=0x7f346c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:26:49.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.491+0000 7f34727fc700 1 -- 172.21.15.16:0/892829189 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f3478055ca0 con 0x7f347c06d4b0 2023-05-20T01:26:49.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.691+0000 7f348423e700 1 -- 172.21.15.16:0/892829189 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f347c0747b0 con 0x7f347c06d4b0 2023-05-20T01:26:49.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.692+0000 7f34727fc700 1 -- 172.21.15.16:0/892829189 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v114) v1 ==== 74+0+20203 (secure 0 0 0) 0x7f3478015090 con 0x7f347c06d4b0 2023-05-20T01:26:49.697 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:26:49.697 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":114,"fsid":"c2245cc5-ecab-4087-a59b-bc81db901676","created":"2023-05-20T01:16:09.549230+0000","modified":"2023-05-20T01:26:48.934848+0000","last_up_change":"2023-05-20T01:16:19.788624+0000","last_in_change":"2023-05-20T01:16:12.092160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-05-20T01:16:20.934366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-05-20T01:16:21.372294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-05-20T01:26:34.067017+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-05-20T01:26:35.040602+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data_extra","create_time":"2023-05-20T01:26:48.309738+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"317a83cd-19c8-4958-a301-6087ef342289","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6825","nonce":3923634179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6827","nonce":3923634179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6831","nonce":3923634179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6829","nonce":3923634179}]},"public_addr":"172.21.15.16:6825/3923634179","cluster_addr":"172.21.15.16:6827/3923634179","heartbeat_back_addr":"172.21.15.16:6831/3923634179","heartbeat_front_addr":"172.21.15.16:6829/3923634179","state":["exists","up"]},{"osd":1,"uuid":"5581046f-8219-4089-8d7d-b7846399505e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6809","nonce":3860264220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6811","nonce":3860264220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6815","nonce":3860264220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6813","nonce":3860264220}]},"public_addr":"172.21.15.16:6809/3860264220","cluster_addr":"172.21.15.16:6811/3860264220","heartbeat_back_addr":"172.21.15.16:6815/3860264220","heartbeat_front_addr":"172.21.15.16:6813/3860264220","state":["exists","up"]},{"osd":2,"uuid":"072e9e66-4133-478e-8d4a-64d46c2963f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6817","nonce":293187030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6819","nonce":293187030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6823","nonce":293187030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6821","nonce":293187030}]},"public_addr":"172.21.15.16:6817/293187030","cluster_addr":"172.21.15.16:6819/293187030","heartbeat_back_addr":"172.21.15.16:6823/293187030","heartbeat_front_addr":"172.21.15.16:6821/293187030","state":["exists","up"]},{"osd":3,"uuid":"11e1bba6-00f8-48dd-9028-d8d716b7f145","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6801","nonce":3669781890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6803","nonce":3669781890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6807","nonce":3669781890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6805","nonce":3669781890}]},"public_addr":"172.21.15.16:6801/3669781890","cluster_addr":"172.21.15.16:6803/3669781890","heartbeat_back_addr":"172.21.15.16:6807/3669781890","heartbeat_front_addr":"172.21.15.16:6805/3669781890","state":["exists","up"]},{"osd":4,"uuid":"b8909c3a-cf70-46cb-8d0b-009c707cf9d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6819","nonce":506317223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6823","nonce":506317223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6829","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6831","nonce":506317223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6827","nonce":506317223}]},"public_addr":"172.21.15.132:6819/506317223","cluster_addr":"172.21.15.132:6823/506317223","heartbeat_back_addr":"172.21.15.132:6831/506317223","heartbeat_front_addr":"172.21.15.132:6827/506317223","state":["exists","up"]},{"osd":5,"uuid":"b1818304-4583-47bf-a6ec-0bfbb5166233","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6818","nonce":1844689995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6822","nonce":1844689995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6830","nonce":1844689995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6826","nonce":1844689995}]},"public_addr":"172.21.15.132:6818/1844689995","cluster_addr":"172.21.15.132:6822/1844689995","heartbeat_back_addr":"172.21.15.132:6830/1844689995","heartbeat_front_addr":"172.21.15.132:6826/1844689995","state":["exists","up"]},{"osd":6,"uuid":"d37f4691-eed1-4a93-8818-18354f03f4a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6801","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6802","nonce":727581123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6805","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6807","nonce":727581123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6813","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6815","nonce":727581123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6811","nonce":727581123}]},"public_addr":"172.21.15.132:6802/727581123","cluster_addr":"172.21.15.132:6807/727581123","heartbeat_back_addr":"172.21.15.132:6815/727581123","heartbeat_front_addr":"172.21.15.132:6811/727581123","state":["exists","up"]},{"osd":7,"uuid":"4f712f62-5d1d-4816-8042-03209144f8b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6803","nonce":466001149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6806","nonce":466001149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6814","nonce":466001149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6810","nonce":466001149}]},"public_addr":"172.21.15.132:6803/466001149","cluster_addr":"172.21.15.132:6806/466001149","heartbeat_back_addr":"172.21.15.132:6814/466001149","heartbeat_front_addr":"172.21.15.132:6810/466001149","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.338863+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.278680+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305694+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.246835+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.276237+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.304441+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305421+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.267050+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-20T01:26:49.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.695+0000 7f348423e700 1 -- 172.21.15.16:0/892829189 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f346006ace0 msgr2=0x7f346006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:49.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.695+0000 7f348423e700 1 --2- 172.21.15.16:0/892829189 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f346006ace0 0x7f346006d190 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f347c0cb580 tx=0x7f346c008040 comp rx=0 tx=0).stop 2023-05-20T01:26:49.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.695+0000 7f348423e700 1 -- 172.21.15.16:0/892829189 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f347c06d4b0 msgr2=0x7f347c155730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:26:49.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.695+0000 7f348423e700 1 --2- 172.21.15.16:0/892829189 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f347c06d4b0 0x7f347c155730 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f3478000c00 tx=0x7f3478004750 comp rx=0 tx=0).stop 2023-05-20T01:26:49.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.695+0000 7f348423e700 1 -- 172.21.15.16:0/892829189 shutdown_connections 2023-05-20T01:26:49.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.695+0000 7f348423e700 1 --2- 172.21.15.16:0/892829189 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f347c06f450 0x7f347c1f2610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:49.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.695+0000 7f348423e700 1 --2- 172.21.15.16:0/892829189 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f346006ace0 0x7f346006d190 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:49.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.695+0000 7f348423e700 1 --2- 172.21.15.16:0/892829189 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f347c06a510 0x7f347c06daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:49.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.695+0000 7f348423e700 1 --2- 172.21.15.16:0/892829189 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f347c06d4b0 0x7f347c155730 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:26:49.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.695+0000 7f348423e700 1 -- 172.21.15.16:0/892829189 >> 172.21.15.16:0/892829189 conn(0x7f347c0cffb0 msgr2=0x7f347c0c7130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:26:49.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.695+0000 7f348423e700 1 -- 172.21.15.16:0/892829189 shutdown_connections 2023-05-20T01:26:49.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:26:49.695+0000 7f348423e700 1 -- 172.21.15.16:0/892829189 wait complete. 2023-05-20T01:26:49.711 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:26:49.713 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir stripey) 2023-05-20T01:26:49.735 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:26:49.736 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.layout -v 'stripe_unit=65536 stripe_count=4 object_size=262144 pool=cephfs_data_extra' ./stripey) 2023-05-20T01:26:49.796 INFO:tasks.cephfs.mount:Writing 1048576 bytes to stripey/flushed_file_0 2023-05-20T01:26:49.797 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits daemon-helper kill python3 -c ' 2023-05-20T01:26:49.798 DEBUG:teuthology.orchestra.run.smithi016:> import zlib 2023-05-20T01:26:49.798 DEBUG:teuthology.orchestra.run.smithi016:> path = "/home/ubuntu/cephtest/mnt.0/stripey/flushed_file_0" 2023-05-20T01:26:49.799 DEBUG:teuthology.orchestra.run.smithi016:> with open(path, '"'"'w'"'"') as f: 2023-05-20T01:26:49.799 DEBUG:teuthology.orchestra.run.smithi016:> for i in range(0, 1048576): 2023-05-20T01:26:49.800 DEBUG:teuthology.orchestra.run.smithi016:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-05-20T01:26:49.800 DEBUG:teuthology.orchestra.run.smithi016:> f.write(chr(val)) 2023-05-20T01:26:49.801 DEBUG:teuthology.orchestra.run.smithi016:> ' 2023-05-20T01:26:50.692 INFO:tasks.cephfs.mount:Validating 1048576 bytes from stripey/flushed_file_0 2023-05-20T01:26:50.693 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-05-20T01:26:50.693 DEBUG:teuthology.orchestra.run.smithi016:> import zlib 2023-05-20T01:26:50.694 DEBUG:teuthology.orchestra.run.smithi016:> path = "/home/ubuntu/cephtest/mnt.0/stripey/flushed_file_0" 2023-05-20T01:26:50.694 DEBUG:teuthology.orchestra.run.smithi016:> with open(path, '"'"'r'"'"') as f: 2023-05-20T01:26:50.695 DEBUG:teuthology.orchestra.run.smithi016:> bytes = f.read() 2023-05-20T01:26:50.696 DEBUG:teuthology.orchestra.run.smithi016:> if len(bytes) != 1048576: 2023-05-20T01:26:50.696 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-05-20T01:26:50.696 DEBUG:teuthology.orchestra.run.smithi016:> len(bytes), 1048576 2023-05-20T01:26:50.697 DEBUG:teuthology.orchestra.run.smithi016:> )) 2023-05-20T01:26:50.697 DEBUG:teuthology.orchestra.run.smithi016:> for i, b in enumerate(bytes): 2023-05-20T01:26:50.697 DEBUG:teuthology.orchestra.run.smithi016:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-05-20T01:26:50.698 DEBUG:teuthology.orchestra.run.smithi016:> if b != chr(val): 2023-05-20T01:26:50.698 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-05-20T01:26:50.698 DEBUG:teuthology.orchestra.run.smithi016:> ' 2023-05-20T01:26:51.599 INFO:tasks.cephfs.mount:Writing 524288 bytes to stripey/flushed_file_1 2023-05-20T01:26:51.600 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits daemon-helper kill python3 -c ' 2023-05-20T01:26:51.600 DEBUG:teuthology.orchestra.run.smithi016:> import zlib 2023-05-20T01:26:51.600 DEBUG:teuthology.orchestra.run.smithi016:> path = "/home/ubuntu/cephtest/mnt.0/stripey/flushed_file_1" 2023-05-20T01:26:51.601 DEBUG:teuthology.orchestra.run.smithi016:> with open(path, '"'"'w'"'"') as f: 2023-05-20T01:26:51.601 DEBUG:teuthology.orchestra.run.smithi016:> for i in range(0, 524288): 2023-05-20T01:26:51.601 DEBUG:teuthology.orchestra.run.smithi016:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-05-20T01:26:51.602 DEBUG:teuthology.orchestra.run.smithi016:> f.write(chr(val)) 2023-05-20T01:26:51.602 DEBUG:teuthology.orchestra.run.smithi016:> ' 2023-05-20T01:26:52.296 INFO:tasks.cephfs.mount:Validating 524288 bytes from stripey/flushed_file_1 2023-05-20T01:26:52.297 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-05-20T01:26:52.297 DEBUG:teuthology.orchestra.run.smithi016:> import zlib 2023-05-20T01:26:52.297 DEBUG:teuthology.orchestra.run.smithi016:> path = "/home/ubuntu/cephtest/mnt.0/stripey/flushed_file_1" 2023-05-20T01:26:52.298 DEBUG:teuthology.orchestra.run.smithi016:> with open(path, '"'"'r'"'"') as f: 2023-05-20T01:26:52.298 DEBUG:teuthology.orchestra.run.smithi016:> bytes = f.read() 2023-05-20T01:26:52.298 DEBUG:teuthology.orchestra.run.smithi016:> if len(bytes) != 524288: 2023-05-20T01:26:52.299 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-05-20T01:26:52.299 DEBUG:teuthology.orchestra.run.smithi016:> len(bytes), 524288 2023-05-20T01:26:52.299 DEBUG:teuthology.orchestra.run.smithi016:> )) 2023-05-20T01:26:52.299 DEBUG:teuthology.orchestra.run.smithi016:> for i, b in enumerate(bytes): 2023-05-20T01:26:52.300 DEBUG:teuthology.orchestra.run.smithi016:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-05-20T01:26:52.300 DEBUG:teuthology.orchestra.run.smithi016:> if b != chr(val): 2023-05-20T01:26:52.300 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-05-20T01:26:52.301 DEBUG:teuthology.orchestra.run.smithi016:> ' 2023-05-20T01:26:52.804 INFO:tasks.cephfs.mount:Writing 917504 bytes to stripey/flushed_file_2 2023-05-20T01:26:52.805 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits daemon-helper kill python3 -c ' 2023-05-20T01:26:52.805 DEBUG:teuthology.orchestra.run.smithi016:> import zlib 2023-05-20T01:26:52.805 DEBUG:teuthology.orchestra.run.smithi016:> path = "/home/ubuntu/cephtest/mnt.0/stripey/flushed_file_2" 2023-05-20T01:26:52.805 DEBUG:teuthology.orchestra.run.smithi016:> with open(path, '"'"'w'"'"') as f: 2023-05-20T01:26:52.806 DEBUG:teuthology.orchestra.run.smithi016:> for i in range(0, 917504): 2023-05-20T01:26:52.806 DEBUG:teuthology.orchestra.run.smithi016:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-05-20T01:26:52.806 DEBUG:teuthology.orchestra.run.smithi016:> f.write(chr(val)) 2023-05-20T01:26:52.807 DEBUG:teuthology.orchestra.run.smithi016:> ' 2023-05-20T01:26:53.701 INFO:tasks.cephfs.mount:Validating 917504 bytes from stripey/flushed_file_2 2023-05-20T01:26:53.702 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-05-20T01:26:53.702 DEBUG:teuthology.orchestra.run.smithi016:> import zlib 2023-05-20T01:26:53.703 DEBUG:teuthology.orchestra.run.smithi016:> path = "/home/ubuntu/cephtest/mnt.0/stripey/flushed_file_2" 2023-05-20T01:26:53.703 DEBUG:teuthology.orchestra.run.smithi016:> with open(path, '"'"'r'"'"') as f: 2023-05-20T01:26:53.703 DEBUG:teuthology.orchestra.run.smithi016:> bytes = f.read() 2023-05-20T01:26:53.703 DEBUG:teuthology.orchestra.run.smithi016:> if len(bytes) != 917504: 2023-05-20T01:26:53.704 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-05-20T01:26:53.704 DEBUG:teuthology.orchestra.run.smithi016:> len(bytes), 917504 2023-05-20T01:26:53.704 DEBUG:teuthology.orchestra.run.smithi016:> )) 2023-05-20T01:26:53.705 DEBUG:teuthology.orchestra.run.smithi016:> for i, b in enumerate(bytes): 2023-05-20T01:26:53.705 DEBUG:teuthology.orchestra.run.smithi016:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-05-20T01:26:53.705 DEBUG:teuthology.orchestra.run.smithi016:> if b != chr(val): 2023-05-20T01:26:53.705 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-05-20T01:26:53.706 DEBUG:teuthology.orchestra.run.smithi016:> ' 2023-05-20T01:26:54.610 INFO:tasks.cephfs.mount:Writing 917503 bytes to stripey/flushed_file_3 2023-05-20T01:26:54.611 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits daemon-helper kill python3 -c ' 2023-05-20T01:26:54.611 DEBUG:teuthology.orchestra.run.smithi016:> import zlib 2023-05-20T01:26:54.612 DEBUG:teuthology.orchestra.run.smithi016:> path = "/home/ubuntu/cephtest/mnt.0/stripey/flushed_file_3" 2023-05-20T01:26:54.612 DEBUG:teuthology.orchestra.run.smithi016:> with open(path, '"'"'w'"'"') as f: 2023-05-20T01:26:54.613 DEBUG:teuthology.orchestra.run.smithi016:> for i in range(0, 917503): 2023-05-20T01:26:54.613 DEBUG:teuthology.orchestra.run.smithi016:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-05-20T01:26:54.614 DEBUG:teuthology.orchestra.run.smithi016:> f.write(chr(val)) 2023-05-20T01:26:54.614 DEBUG:teuthology.orchestra.run.smithi016:> ' 2023-05-20T01:26:55.506 INFO:tasks.cephfs.mount:Validating 917503 bytes from stripey/flushed_file_3 2023-05-20T01:26:55.507 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-05-20T01:26:55.507 DEBUG:teuthology.orchestra.run.smithi016:> import zlib 2023-05-20T01:26:55.508 DEBUG:teuthology.orchestra.run.smithi016:> path = "/home/ubuntu/cephtest/mnt.0/stripey/flushed_file_3" 2023-05-20T01:26:55.508 DEBUG:teuthology.orchestra.run.smithi016:> with open(path, '"'"'r'"'"') as f: 2023-05-20T01:26:55.508 DEBUG:teuthology.orchestra.run.smithi016:> bytes = f.read() 2023-05-20T01:26:55.509 DEBUG:teuthology.orchestra.run.smithi016:> if len(bytes) != 917503: 2023-05-20T01:26:55.509 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-05-20T01:26:55.509 DEBUG:teuthology.orchestra.run.smithi016:> len(bytes), 917503 2023-05-20T01:26:55.509 DEBUG:teuthology.orchestra.run.smithi016:> )) 2023-05-20T01:26:55.510 DEBUG:teuthology.orchestra.run.smithi016:> for i, b in enumerate(bytes): 2023-05-20T01:26:55.510 DEBUG:teuthology.orchestra.run.smithi016:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-05-20T01:26:55.510 DEBUG:teuthology.orchestra.run.smithi016:> if b != chr(val): 2023-05-20T01:26:55.510 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-05-20T01:26:55.511 DEBUG:teuthology.orchestra.run.smithi016:> ' 2023-05-20T01:26:56.414 INFO:tasks.cephfs.mount:Writing 1441792 bytes to stripey/flushed_file_4 2023-05-20T01:26:56.415 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits daemon-helper kill python3 -c ' 2023-05-20T01:26:56.415 DEBUG:teuthology.orchestra.run.smithi016:> import zlib 2023-05-20T01:26:56.415 DEBUG:teuthology.orchestra.run.smithi016:> path = "/home/ubuntu/cephtest/mnt.0/stripey/flushed_file_4" 2023-05-20T01:26:56.416 DEBUG:teuthology.orchestra.run.smithi016:> with open(path, '"'"'w'"'"') as f: 2023-05-20T01:26:56.416 DEBUG:teuthology.orchestra.run.smithi016:> for i in range(0, 1441792): 2023-05-20T01:26:56.416 DEBUG:teuthology.orchestra.run.smithi016:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-05-20T01:26:56.416 DEBUG:teuthology.orchestra.run.smithi016:> f.write(chr(val)) 2023-05-20T01:26:56.417 DEBUG:teuthology.orchestra.run.smithi016:> ' 2023-05-20T01:26:57.713 INFO:tasks.cephfs.mount:Validating 1441792 bytes from stripey/flushed_file_4 2023-05-20T01:26:57.714 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-05-20T01:26:57.714 DEBUG:teuthology.orchestra.run.smithi016:> import zlib 2023-05-20T01:26:57.715 DEBUG:teuthology.orchestra.run.smithi016:> path = "/home/ubuntu/cephtest/mnt.0/stripey/flushed_file_4" 2023-05-20T01:26:57.715 DEBUG:teuthology.orchestra.run.smithi016:> with open(path, '"'"'r'"'"') as f: 2023-05-20T01:26:57.716 DEBUG:teuthology.orchestra.run.smithi016:> bytes = f.read() 2023-05-20T01:26:57.716 DEBUG:teuthology.orchestra.run.smithi016:> if len(bytes) != 1441792: 2023-05-20T01:26:57.717 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-05-20T01:26:57.718 DEBUG:teuthology.orchestra.run.smithi016:> len(bytes), 1441792 2023-05-20T01:26:57.718 DEBUG:teuthology.orchestra.run.smithi016:> )) 2023-05-20T01:26:57.719 DEBUG:teuthology.orchestra.run.smithi016:> for i, b in enumerate(bytes): 2023-05-20T01:26:57.719 DEBUG:teuthology.orchestra.run.smithi016:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-05-20T01:26:57.720 DEBUG:teuthology.orchestra.run.smithi016:> if b != chr(val): 2023-05-20T01:26:57.720 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-05-20T01:26:57.721 DEBUG:teuthology.orchestra.run.smithi016:> ' 2023-05-20T01:26:58.821 INFO:tasks.cephfs.mount:Writing 1441793 bytes to stripey/flushed_file_5 2023-05-20T01:26:58.821 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits daemon-helper kill python3 -c ' 2023-05-20T01:26:58.822 DEBUG:teuthology.orchestra.run.smithi016:> import zlib 2023-05-20T01:26:58.822 DEBUG:teuthology.orchestra.run.smithi016:> path = "/home/ubuntu/cephtest/mnt.0/stripey/flushed_file_5" 2023-05-20T01:26:58.822 DEBUG:teuthology.orchestra.run.smithi016:> with open(path, '"'"'w'"'"') as f: 2023-05-20T01:26:58.823 DEBUG:teuthology.orchestra.run.smithi016:> for i in range(0, 1441793): 2023-05-20T01:26:58.823 DEBUG:teuthology.orchestra.run.smithi016:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-05-20T01:26:58.823 DEBUG:teuthology.orchestra.run.smithi016:> f.write(chr(val)) 2023-05-20T01:26:58.823 DEBUG:teuthology.orchestra.run.smithi016:> ' 2023-05-20T01:27:00.119 INFO:tasks.cephfs.mount:Validating 1441793 bytes from stripey/flushed_file_5 2023-05-20T01:27:00.120 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-05-20T01:27:00.120 DEBUG:teuthology.orchestra.run.smithi016:> import zlib 2023-05-20T01:27:00.121 DEBUG:teuthology.orchestra.run.smithi016:> path = "/home/ubuntu/cephtest/mnt.0/stripey/flushed_file_5" 2023-05-20T01:27:00.121 DEBUG:teuthology.orchestra.run.smithi016:> with open(path, '"'"'r'"'"') as f: 2023-05-20T01:27:00.122 DEBUG:teuthology.orchestra.run.smithi016:> bytes = f.read() 2023-05-20T01:27:00.122 DEBUG:teuthology.orchestra.run.smithi016:> if len(bytes) != 1441793: 2023-05-20T01:27:00.123 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-05-20T01:27:00.123 DEBUG:teuthology.orchestra.run.smithi016:> len(bytes), 1441793 2023-05-20T01:27:00.124 DEBUG:teuthology.orchestra.run.smithi016:> )) 2023-05-20T01:27:00.124 DEBUG:teuthology.orchestra.run.smithi016:> for i, b in enumerate(bytes): 2023-05-20T01:27:00.125 DEBUG:teuthology.orchestra.run.smithi016:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-05-20T01:27:00.125 DEBUG:teuthology.orchestra.run.smithi016:> if b != chr(val): 2023-05-20T01:27:00.126 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-05-20T01:27:00.126 DEBUG:teuthology.orchestra.run.smithi016:> ' 2023-05-20T01:27:01.226 INFO:tasks.cephfs.mount:Writing 1572864 bytes to stripey/flushed_file_6 2023-05-20T01:27:01.227 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits daemon-helper kill python3 -c ' 2023-05-20T01:27:01.227 DEBUG:teuthology.orchestra.run.smithi016:> import zlib 2023-05-20T01:27:01.228 DEBUG:teuthology.orchestra.run.smithi016:> path = "/home/ubuntu/cephtest/mnt.0/stripey/flushed_file_6" 2023-05-20T01:27:01.228 DEBUG:teuthology.orchestra.run.smithi016:> with open(path, '"'"'w'"'"') as f: 2023-05-20T01:27:01.228 DEBUG:teuthology.orchestra.run.smithi016:> for i in range(0, 1572864): 2023-05-20T01:27:01.229 DEBUG:teuthology.orchestra.run.smithi016:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-05-20T01:27:01.229 DEBUG:teuthology.orchestra.run.smithi016:> f.write(chr(val)) 2023-05-20T01:27:01.229 DEBUG:teuthology.orchestra.run.smithi016:> ' 2023-05-20T01:27:02.524 INFO:tasks.cephfs.mount:Validating 1572864 bytes from stripey/flushed_file_6 2023-05-20T01:27:02.524 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-05-20T01:27:02.525 DEBUG:teuthology.orchestra.run.smithi016:> import zlib 2023-05-20T01:27:02.525 DEBUG:teuthology.orchestra.run.smithi016:> path = "/home/ubuntu/cephtest/mnt.0/stripey/flushed_file_6" 2023-05-20T01:27:02.525 DEBUG:teuthology.orchestra.run.smithi016:> with open(path, '"'"'r'"'"') as f: 2023-05-20T01:27:02.526 DEBUG:teuthology.orchestra.run.smithi016:> bytes = f.read() 2023-05-20T01:27:02.526 DEBUG:teuthology.orchestra.run.smithi016:> if len(bytes) != 1572864: 2023-05-20T01:27:02.526 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-05-20T01:27:02.526 DEBUG:teuthology.orchestra.run.smithi016:> len(bytes), 1572864 2023-05-20T01:27:02.527 DEBUG:teuthology.orchestra.run.smithi016:> )) 2023-05-20T01:27:02.527 DEBUG:teuthology.orchestra.run.smithi016:> for i, b in enumerate(bytes): 2023-05-20T01:27:02.527 DEBUG:teuthology.orchestra.run.smithi016:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-05-20T01:27:02.528 DEBUG:teuthology.orchestra.run.smithi016:> if b != chr(val): 2023-05-20T01:27:02.528 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-05-20T01:27:02.528 DEBUG:teuthology.orchestra.run.smithi016:> ' 2023-05-20T01:27:03.832 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-20T01:27:03.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:03.993+0000 7ff5bcb35700 1 Processor -- start 2023-05-20T01:27:03.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:03.993+0000 7ff5bcb35700 1 -- start start 2023-05-20T01:27:04.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:03.994+0000 7ff5bcb35700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff5b80ccec0 0x7ff5b80713c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:04.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:03.994+0000 7ff5bcb35700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5b80cb590 con 0x7ff5b8071900 2023-05-20T01:27:04.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:03.994+0000 7ff5bcb35700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5b80ca9a0 con 0x7ff5b8157190 2023-05-20T01:27:04.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:03.994+0000 7ff5bcb35700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5b80cac60 con 0x7ff5b80ccec0 2023-05-20T01:27:04.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:03.994+0000 7ff5b6d9d700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ff5b8071900 0x7ff5b80ce740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49936/0 (socket says 172.21.15.16:49936) 2023-05-20T01:27:04.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:03.994+0000 7ff5b6d9d700 1 -- 172.21.15.16:0/355505484 learned_addr learned my addr 172.21.15.16:0/355505484 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:27:04.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:03.994+0000 7ff5b5d9b700 1 --2- 172.21.15.16:0/355505484 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff5b80ccec0 0x7ff5b80713c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:04.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:03.994+0000 7ff5b559a700 1 -- 172.21.15.16:0/355505484 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2827738472 0 0) 0x7ff5b80cb590 con 0x7ff5b8071900 2023-05-20T01:27:04.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:03.994+0000 7ff5b559a700 1 -- 172.21.15.16:0/355505484 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff5a00035b0 con 0x7ff5b8071900 2023-05-20T01:27:04.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:03.995+0000 7ff5b559a700 1 -- 172.21.15.16:0/355505484 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1013239083 0 0) 0x7ff5a00035b0 con 0x7ff5b8071900 2023-05-20T01:27:04.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:03.995+0000 7ff5b559a700 1 -- 172.21.15.16:0/355505484 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff5b80ccec0 msgr2=0x7ff5b80713c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:04.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:03.995+0000 7ff5b559a700 1 --2- 172.21.15.16:0/355505484 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff5b80ccec0 0x7ff5b80713c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:04.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:03.995+0000 7ff5b559a700 1 -- 172.21.15.16:0/355505484 >> v1:172.21.15.132:6789/0 conn(0x7ff5b8157190 legacy=0x7ff5b80cc7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:04.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:03.995+0000 7ff5b559a700 1 -- 172.21.15.16:0/355505484 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5b80caf40 con 0x7ff5b8071900 2023-05-20T01:27:04.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:03.995+0000 7ff5b559a700 1 -- 172.21.15.16:0/355505484 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7ff5b00019f0 con 0x7ff5b8071900 2023-05-20T01:27:04.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:03.995+0000 7ff5b559a700 1 -- 172.21.15.16:0/355505484 >> v1:172.21.15.16:6789/0 conn(0x7ff5b8071900 legacy=0x7ff5b80ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:04.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:03.995+0000 7ff5b559a700 1 --2- 172.21.15.16:0/355505484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5a0003aa0 0x7ff5a0003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:04.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:03.995+0000 7ff5b559a700 1 --2- 172.21.15.16:0/355505484 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff5b80ccec0 0x7ff5a0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:04.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:03.995+0000 7ff5b559a700 1 --2- 172.21.15.16:0/355505484 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff5a0004d90 0x7ff5a0005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:04.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:03.995+0000 7ff5b559a700 1 -- 172.21.15.16:0/355505484 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5b80cac60 con 0x7ff5a0003aa0 2023-05-20T01:27:04.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:03.995+0000 7ff5b559a700 1 -- 172.21.15.16:0/355505484 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a000bb10 con 0x7ff5a0004d90 2023-05-20T01:27:04.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:03.995+0000 7ff5b559a700 1 -- 172.21.15.16:0/355505484 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a000dee0 con 0x7ff5b80ccec0 2023-05-20T01:27:04.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:03.995+0000 7ff5b659c700 1 --2- 172.21.15.16:0/355505484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5a0003aa0 0x7ff5a0003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:04.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:03.995+0000 7ff5b5d9b700 1 --2- 172.21.15.16:0/355505484 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff5b80ccec0 0x7ff5a0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:04.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:03.995+0000 7ff5b6d9d700 1 --2- 172.21.15.16:0/355505484 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff5a0004d90 0x7ff5a0005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:04.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:03.995+0000 7ff5b659c700 1 -- 172.21.15.16:0/355505484 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff5b80ccec0 msgr2=0x7ff5a0004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:04.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:03.995+0000 7ff5b659c700 1 --2- 172.21.15.16:0/355505484 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff5b80ccec0 0x7ff5a0004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:04.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:03.995+0000 7ff5b659c700 1 -- 172.21.15.16:0/355505484 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff5a0004d90 msgr2=0x7ff5a0005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:04.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:03.995+0000 7ff5b659c700 1 --2- 172.21.15.16:0/355505484 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff5a0004d90 0x7ff5a0005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:04.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:03.995+0000 7ff5b659c700 1 -- 172.21.15.16:0/355505484 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5a00035b0 con 0x7ff5a0003aa0 2023-05-20T01:27:04.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:03.995+0000 7ff5b6d9d700 1 --2- 172.21.15.16:0/355505484 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff5a0004d90 0x7ff5a0005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:27:04.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:03.995+0000 7ff5b659c700 1 --2- 172.21.15.16:0/355505484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5a0003aa0 0x7ff5a0003e70 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac0018d0 tx=0x7ff5a0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7639afe8b1b2d5de server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:04.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:03.996+0000 7ff5b559a700 1 -- 172.21.15.16:0/355505484 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5ac00c690 con 0x7ff5a0003aa0 2023-05-20T01:27:04.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:03.996+0000 7ff5b559a700 1 -- 172.21.15.16:0/355505484 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5ac00c810 con 0x7ff5a0003aa0 2023-05-20T01:27:04.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:03.996+0000 7ff5b559a700 1 -- 172.21.15.16:0/355505484 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5ac00cbc0 con 0x7ff5a0003aa0 2023-05-20T01:27:04.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:03.996+0000 7ff5bcb35700 1 -- 172.21.15.16:0/355505484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5a0003aa0 msgr2=0x7ff5a0003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:04.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:03.996+0000 7ff5bcb35700 1 --2- 172.21.15.16:0/355505484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5a0003aa0 0x7ff5a0003e70 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac0018d0 tx=0x7ff5a0005880 comp rx=0 tx=0).stop 2023-05-20T01:27:04.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:03.996+0000 7ff5b6d9d700 1 -- 172.21.15.16:0/355505484 reap_dead start 2023-05-20T01:27:04.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:03.996+0000 7ff5bcb35700 1 -- 172.21.15.16:0/355505484 shutdown_connections 2023-05-20T01:27:04.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:03.996+0000 7ff5bcb35700 1 -- 172.21.15.16:0/355505484 >> 172.21.15.16:0/355505484 conn(0x7ff5b80cffb0 msgr2=0x7ff5b80d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:27:04.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:03.996+0000 7ff5bcb35700 1 -- 172.21.15.16:0/355505484 shutdown_connections 2023-05-20T01:27:04.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:03.996+0000 7ff5bcb35700 1 -- 172.21.15.16:0/355505484 wait complete. 2023-05-20T01:27:04.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:03.998+0000 7ff5bcb35700 1 Processor -- start 2023-05-20T01:27:04.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:03.998+0000 7ff5bcb35700 1 -- start start 2023-05-20T01:27:04.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:03.998+0000 7ff5bcb35700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff5a0003aa0 0x7ff5b815c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:04.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:03.998+0000 7ff5bcb35700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff5b814b7c0 0x7ff5b80b55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:04.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:03.998+0000 7ff5bcb35700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5b814ab10 0x7ff5b814af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:04.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:03.998+0000 7ff5bcb35700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5b814b4c0 con 0x7ff5b814ab10 2023-05-20T01:27:04.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:03.998+0000 7ff5bcb35700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5b80c5f30 con 0x7ff5b814b7c0 2023-05-20T01:27:04.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:03.998+0000 7ff5bcb35700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5b80c6210 con 0x7ff5a0003aa0 2023-05-20T01:27:04.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:03.998+0000 7ff5b6d9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5b814ab10 0x7ff5b814af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:04.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:03.998+0000 7ff5b6d9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5b814ab10 0x7ff5b814af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38420/0 (socket says 172.21.15.16:38420) 2023-05-20T01:27:04.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:03.998+0000 7ff5b6d9d700 1 -- 172.21.15.16:0/2724535249 learned_addr learned my addr 172.21.15.16:0/2724535249 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:27:04.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:03.999+0000 7ff5b5d9b700 1 --2- 172.21.15.16:0/2724535249 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff5b814b7c0 0x7ff5b80b55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:04.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:03.999+0000 7ff5b659c700 1 --2- 172.21.15.16:0/2724535249 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff5a0003aa0 0x7ff5b815c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:04.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:03.999+0000 7ff5b6d9d700 1 -- 172.21.15.16:0/2724535249 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff5a0003aa0 msgr2=0x7ff5b815c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:04.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:03.999+0000 7ff5b6d9d700 1 --2- 172.21.15.16:0/2724535249 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff5a0003aa0 0x7ff5b815c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:04.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:03.999+0000 7ff5b6d9d700 1 -- 172.21.15.16:0/2724535249 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff5b814b7c0 msgr2=0x7ff5b80b55b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:04.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:03.999+0000 7ff5b6d9d700 1 --2- 172.21.15.16:0/2724535249 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff5b814b7c0 0x7ff5b80b55b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:04.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:03.999+0000 7ff5b6d9d700 1 -- 172.21.15.16:0/2724535249 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5b8157560 con 0x7ff5b814ab10 2023-05-20T01:27:04.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:03.999+0000 7ff5b6d9d700 1 --2- 172.21.15.16:0/2724535249 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5b814ab10 0x7ff5b814af80 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7ff5b80d0a90 tx=0x7ff5b000e440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:04.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:03.999+0000 7ff5a6ffd700 1 -- 172.21.15.16:0/2724535249 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b000ebb0 con 0x7ff5b814ab10 2023-05-20T01:27:04.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:03.999+0000 7ff5a6ffd700 1 -- 172.21.15.16:0/2724535249 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5b000ed10 con 0x7ff5b814ab10 2023-05-20T01:27:04.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:03.999+0000 7ff5bcb35700 1 -- 172.21.15.16:0/2724535249 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5b80cb220 con 0x7ff5b814ab10 2023-05-20T01:27:04.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:03.999+0000 7ff5a6ffd700 1 -- 172.21.15.16:0/2724535249 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b0013660 con 0x7ff5b814ab10 2023-05-20T01:27:04.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:03.999+0000 7ff5bcb35700 1 -- 172.21.15.16:0/2724535249 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5b80cba20 con 0x7ff5b814ab10 2023-05-20T01:27:04.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:04.000+0000 7ff5bcb35700 1 -- 172.21.15.16:0/2724535249 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5b80747b0 con 0x7ff5b814ab10 2023-05-20T01:27:04.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:04.001+0000 7ff5a6ffd700 1 -- 172.21.15.16:0/2724535249 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff5b00137c0 con 0x7ff5b814ab10 2023-05-20T01:27:04.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:04.001+0000 7ff5a6ffd700 1 --2- 172.21.15.16:0/2724535249 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff59806ace0 0x7ff59806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:04.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:04.001+0000 7ff5a6ffd700 1 -- 172.21.15.16:0/2724535249 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6786+0+0 (secure 0 0 0) 0x7ff5b0089bd0 con 0x7ff5b814ab10 2023-05-20T01:27:04.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:04.001+0000 7ff5b659c700 1 --2- 172.21.15.16:0/2724535249 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff59806ace0 0x7ff59806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:04.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:04.002+0000 7ff5b659c700 1 --2- 172.21.15.16:0/2724535249 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff59806ace0 0x7ff59806d190 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac014040 tx=0x7ff5ac005040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:04.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:04.006+0000 7ff5a6ffd700 1 -- 172.21.15.16:0/2724535249 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7ff5b00549a0 con 0x7ff5b814ab10 2023-05-20T01:27:04.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:04.245+0000 7ff5bcb35700 1 -- 172.21.15.16:0/2724535249 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff5b806d510 con 0x7ff5b814ab10 2023-05-20T01:27:04.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:04.246+0000 7ff5a6ffd700 1 -- 172.21.15.16:0/2724535249 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) v1 ==== 94+0+4679 (secure 0 0 0) 0x7ff5b0001200 con 0x7ff5b814ab10 2023-05-20T01:27:04.251 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:27:04.251 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":79,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7069,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/3619911175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":3619911175},{"type":"v1","addr":"172.21.15.16:6837","nonce":3619911175}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73},{"gid":7081,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/959638003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":959638003},{"type":"v1","addr":"172.21.15.132:6834","nonce":959638003}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73},{"gid":7084,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2596482253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":2596482253},{"type":"v1","addr":"172.21.15.132:6835","nonce":2596482253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-20T01:26:36.067393+0000","modified":"2023-05-20T01:26:48.935135+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7896},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7896":{"gid":7896,"name":"a","rank":0,"incarnation":74,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6836/3765848068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3765848068},{"type":"v1","addr":"172.21.15.16:6836","nonce":3765848068}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16,17],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-05-20T01:27:04.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:04.248+0000 7ff5bcb35700 1 -- 172.21.15.16:0/2724535249 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff59806ace0 msgr2=0x7ff59806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:04.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:04.248+0000 7ff5bcb35700 1 --2- 172.21.15.16:0/2724535249 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff59806ace0 0x7ff59806d190 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac014040 tx=0x7ff5ac005040 comp rx=0 tx=0).stop 2023-05-20T01:27:04.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:04.248+0000 7ff5bcb35700 1 -- 172.21.15.16:0/2724535249 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5b814ab10 msgr2=0x7ff5b814af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:04.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:04.248+0000 7ff5bcb35700 1 --2- 172.21.15.16:0/2724535249 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5b814ab10 0x7ff5b814af80 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7ff5b80d0a90 tx=0x7ff5b000e440 comp rx=0 tx=0).stop 2023-05-20T01:27:04.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:04.248+0000 7ff5bcb35700 1 -- 172.21.15.16:0/2724535249 shutdown_connections 2023-05-20T01:27:04.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:04.248+0000 7ff5bcb35700 1 --2- 172.21.15.16:0/2724535249 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5b814ab10 0x7ff5b814af80 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:04.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:04.248+0000 7ff5bcb35700 1 --2- 172.21.15.16:0/2724535249 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff59806ace0 0x7ff59806d190 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:04.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:04.248+0000 7ff5bcb35700 1 --2- 172.21.15.16:0/2724535249 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff5a0003aa0 0x7ff5b815c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:04.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:04.248+0000 7ff5bcb35700 1 --2- 172.21.15.16:0/2724535249 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff5b814b7c0 0x7ff5b80b55b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:04.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:04.248+0000 7ff5bcb35700 1 -- 172.21.15.16:0/2724535249 >> 172.21.15.16:0/2724535249 conn(0x7ff5b80cffb0 msgr2=0x7ff5b806e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:27:04.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:04.248+0000 7ff5bcb35700 1 -- 172.21.15.16:0/2724535249 shutdown_connections 2023-05-20T01:27:04.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:04.248+0000 7ff5bcb35700 1 -- 172.21.15.16:0/2724535249 wait complete. 2023-05-20T01:27:04.254 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 79 2023-05-20T01:27:04.265 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2023-05-20T01:27:04.423 INFO:teuthology.orchestra.run.smithi016.stdout:{"message":"","return_code":0} 2023-05-20T01:27:04.424 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2023-05-20T01:27:04.424 INFO:tasks.cephfs.mount:Writing 524288 bytes to stripey/unflushed_file 2023-05-20T01:27:04.424 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits daemon-helper kill python3 -c ' 2023-05-20T01:27:04.425 DEBUG:teuthology.orchestra.run.smithi016:> import zlib 2023-05-20T01:27:04.425 DEBUG:teuthology.orchestra.run.smithi016:> path = "/home/ubuntu/cephtest/mnt.0/stripey/unflushed_file" 2023-05-20T01:27:04.425 DEBUG:teuthology.orchestra.run.smithi016:> with open(path, '"'"'w'"'"') as f: 2023-05-20T01:27:04.426 DEBUG:teuthology.orchestra.run.smithi016:> for i in range(0, 524288): 2023-05-20T01:27:04.426 DEBUG:teuthology.orchestra.run.smithi016:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-05-20T01:27:04.426 DEBUG:teuthology.orchestra.run.smithi016:> f.write(chr(val)) 2023-05-20T01:27:04.426 DEBUG:teuthology.orchestra.run.smithi016:> ' 2023-05-20T01:27:05.084 INFO:tasks.cephfs.mount:Validating 524288 bytes from stripey/unflushed_file 2023-05-20T01:27:05.084 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-05-20T01:27:05.084 DEBUG:teuthology.orchestra.run.smithi016:> import zlib 2023-05-20T01:27:05.085 DEBUG:teuthology.orchestra.run.smithi016:> path = "/home/ubuntu/cephtest/mnt.0/stripey/unflushed_file" 2023-05-20T01:27:05.085 DEBUG:teuthology.orchestra.run.smithi016:> with open(path, '"'"'r'"'"') as f: 2023-05-20T01:27:05.085 DEBUG:teuthology.orchestra.run.smithi016:> bytes = f.read() 2023-05-20T01:27:05.086 DEBUG:teuthology.orchestra.run.smithi016:> if len(bytes) != 524288: 2023-05-20T01:27:05.086 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-05-20T01:27:05.086 DEBUG:teuthology.orchestra.run.smithi016:> len(bytes), 524288 2023-05-20T01:27:05.086 DEBUG:teuthology.orchestra.run.smithi016:> )) 2023-05-20T01:27:05.087 DEBUG:teuthology.orchestra.run.smithi016:> for i, b in enumerate(bytes): 2023-05-20T01:27:05.087 DEBUG:teuthology.orchestra.run.smithi016:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-05-20T01:27:05.088 DEBUG:teuthology.orchestra.run.smithi016:> if b != chr(val): 2023-05-20T01:27:05.089 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-05-20T01:27:05.089 DEBUG:teuthology.orchestra.run.smithi016:> ' 2023-05-20T01:27:05.592 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits daemon-helper kill python3 -c ' 2023-05-20T01:27:05.593 DEBUG:teuthology.orchestra.run.smithi016:> import os 2023-05-20T01:27:05.593 DEBUG:teuthology.orchestra.run.smithi016:> import stat 2023-05-20T01:27:05.594 DEBUG:teuthology.orchestra.run.smithi016:> 2023-05-20T01:27:05.594 DEBUG:teuthology.orchestra.run.smithi016:> print(os.stat("/home/ubuntu/cephtest/mnt.0/stripey/unflushed_file").st_ino) 2023-05-20T01:27:05.594 DEBUG:teuthology.orchestra.run.smithi016:> ' 2023-05-20T01:27:05.699 INFO:teuthology.orchestra.run.smithi016.stdout:1099511628284 2023-05-20T01:27:05.889 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-05-20T01:27:05.889 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2023-05-20T01:27:05.947 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-05-20T01:27:05.948 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2023-05-20T01:27:06.005 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-20T01:27:06.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.165+0000 7f285afe2700 1 Processor -- start 2023-05-20T01:27:06.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.165+0000 7f285afe2700 1 -- start start 2023-05-20T01:27:06.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.166+0000 7f285afe2700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f285406f450 0x7f28540cf890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:06.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.166+0000 7f285afe2700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f285406d4b0 0x7f2854161080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:06.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.166+0000 7f285afe2700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28540cc510 con 0x7f285406a510 2023-05-20T01:27:06.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.166+0000 7f285afe2700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28541615c0 con 0x7f285406d4b0 2023-05-20T01:27:06.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.166+0000 7f285afe2700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2854071ed0 con 0x7f285406f450 2023-05-20T01:27:06.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.166+0000 7f2853fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f285406d4b0 0x7f2854161080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:06.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.166+0000 7f2853fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f285406d4b0 0x7f2854161080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38428/0 (socket says 172.21.15.16:38428) 2023-05-20T01:27:06.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.166+0000 7f2853fff700 1 -- 172.21.15.16:0/975355223 learned_addr learned my addr 172.21.15.16:0/975355223 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:27:06.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.167+0000 7f2858d7e700 1 --2- 172.21.15.16:0/975355223 >> v2:172.21.15.132:3300/0 conn(0x7f285406f450 0x7f28540cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:06.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.167+0000 7f2853fff700 1 -- 172.21.15.16:0/975355223 >> v2:172.21.15.132:3300/0 conn(0x7f285406f450 msgr2=0x7f28540cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:06.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.167+0000 7f2853fff700 1 --2- 172.21.15.16:0/975355223 >> v2:172.21.15.132:3300/0 conn(0x7f285406f450 0x7f28540cf890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:06.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.167+0000 7f2853fff700 1 -- 172.21.15.16:0/975355223 >> v1:172.21.15.132:6789/0 conn(0x7f285406a510 legacy=0x7f28540cdb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:06.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.167+0000 7f2853fff700 1 -- 172.21.15.16:0/975355223 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28540721b0 con 0x7f285406d4b0 2023-05-20T01:27:06.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.167+0000 7f2853fff700 1 --2- 172.21.15.16:0/975355223 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f285406d4b0 0x7f2854161080 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f28540cdb00 tx=0x7f2840009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e504bef1a134b8b server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:06.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.167+0000 7f28537fe700 1 -- 172.21.15.16:0/975355223 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2840004230 con 0x7f285406d4b0 2023-05-20T01:27:06.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.167+0000 7f28537fe700 1 -- 172.21.15.16:0/975355223 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2840004390 con 0x7f285406d4b0 2023-05-20T01:27:06.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.167+0000 7f28537fe700 1 -- 172.21.15.16:0/975355223 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28400036a0 con 0x7f285406d4b0 2023-05-20T01:27:06.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.167+0000 7f285afe2700 1 -- 172.21.15.16:0/975355223 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f285406d4b0 msgr2=0x7f2854161080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:06.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.167+0000 7f285afe2700 1 --2- 172.21.15.16:0/975355223 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f285406d4b0 0x7f2854161080 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f28540cdb00 tx=0x7f2840009bd0 comp rx=0 tx=0).stop 2023-05-20T01:27:06.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.168+0000 7f285afe2700 1 -- 172.21.15.16:0/975355223 shutdown_connections 2023-05-20T01:27:06.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.168+0000 7f285afe2700 1 --2- 172.21.15.16:0/975355223 >> v2:172.21.15.132:3300/0 conn(0x7f285406f450 0x7f28540cf890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:06.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.168+0000 7f285afe2700 1 --2- 172.21.15.16:0/975355223 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f285406d4b0 0x7f2854161080 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:06.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.168+0000 7f285afe2700 1 -- 172.21.15.16:0/975355223 >> 172.21.15.16:0/975355223 conn(0x7f28540cffb0 msgr2=0x7f28540d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:27:06.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.168+0000 7f285afe2700 1 -- 172.21.15.16:0/975355223 shutdown_connections 2023-05-20T01:27:06.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.168+0000 7f285afe2700 1 -- 172.21.15.16:0/975355223 wait complete. 2023-05-20T01:27:06.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.169+0000 7f285afe2700 1 Processor -- start 2023-05-20T01:27:06.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.169+0000 7f285afe2700 1 -- start start 2023-05-20T01:27:06.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.170+0000 7f285afe2700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f285406a510 0x7f2854153a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:06.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.170+0000 7f285afe2700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f285406d4b0 0x7f28540b3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:06.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.170+0000 7f285afe2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f285406f450 0x7f28540c85a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:06.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.170+0000 7f285afe2700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28540b3d40 con 0x7f285406f450 2023-05-20T01:27:06.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.170+0000 7f285afe2700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28541548f0 con 0x7f285406d4b0 2023-05-20T01:27:06.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.170+0000 7f285afe2700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28540b3f50 con 0x7f285406a510 2023-05-20T01:27:06.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.170+0000 7f285957f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f285406f450 0x7f28540c85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:06.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.171+0000 7f285957f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f285406f450 0x7f28540c85a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38434/0 (socket says 172.21.15.16:38434) 2023-05-20T01:27:06.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.171+0000 7f285957f700 1 -- 172.21.15.16:0/2090511477 learned_addr learned my addr 172.21.15.16:0/2090511477 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:27:06.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.171+0000 7f2853fff700 1 --2- 172.21.15.16:0/2090511477 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f285406d4b0 0x7f28540b3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:06.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.171+0000 7f2858d7e700 1 --2- 172.21.15.16:0/2090511477 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f285406a510 0x7f2854153a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:06.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.171+0000 7f285957f700 1 -- 172.21.15.16:0/2090511477 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f285406a510 msgr2=0x7f2854153a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:06.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.171+0000 7f285957f700 1 --2- 172.21.15.16:0/2090511477 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f285406a510 0x7f2854153a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:06.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.171+0000 7f285957f700 1 -- 172.21.15.16:0/2090511477 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f285406d4b0 msgr2=0x7f28540b3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:06.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.171+0000 7f285957f700 1 --2- 172.21.15.16:0/2090511477 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f285406d4b0 0x7f28540b3800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:06.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.171+0000 7f285957f700 1 -- 172.21.15.16:0/2090511477 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2840009550 con 0x7f285406f450 2023-05-20T01:27:06.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.171+0000 7f285957f700 1 --2- 172.21.15.16:0/2090511477 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f285406f450 0x7f28540c85a0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f284c00c7a0 tx=0x7f284c00cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:06.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.171+0000 7f28517fa700 1 -- 172.21.15.16:0/2090511477 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f284c003e10 con 0x7f285406f450 2023-05-20T01:27:06.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.171+0000 7f28517fa700 1 -- 172.21.15.16:0/2090511477 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f284c005070 con 0x7f285406f450 2023-05-20T01:27:06.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.171+0000 7f285afe2700 1 -- 172.21.15.16:0/2090511477 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28540cbe50 con 0x7f285406f450 2023-05-20T01:27:06.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.171+0000 7f28517fa700 1 -- 172.21.15.16:0/2090511477 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f284c005420 con 0x7f285406f450 2023-05-20T01:27:06.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.171+0000 7f285afe2700 1 -- 172.21.15.16:0/2090511477 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28540cc1a0 con 0x7f285406f450 2023-05-20T01:27:06.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.172+0000 7f285afe2700 1 -- 172.21.15.16:0/2090511477 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f285406a950 con 0x7f285406f450 2023-05-20T01:27:06.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.173+0000 7f28517fa700 1 -- 172.21.15.16:0/2090511477 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f284c005580 con 0x7f285406f450 2023-05-20T01:27:06.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.173+0000 7f28517fa700 1 --2- 172.21.15.16:0/2090511477 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f283806ace0 0x7f283806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:06.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.173+0000 7f28517fa700 1 -- 172.21.15.16:0/2090511477 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6786+0+0 (secure 0 0 0) 0x7f284c088db0 con 0x7f285406f450 2023-05-20T01:27:06.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.173+0000 7f2858d7e700 1 --2- 172.21.15.16:0/2090511477 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f283806ace0 0x7f283806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:06.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.174+0000 7f2858d7e700 1 --2- 172.21.15.16:0/2090511477 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f283806ace0 0x7f283806d190 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f2848006fd0 tx=0x7f2848008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:06.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.178+0000 7f28517fa700 1 -- 172.21.15.16:0/2090511477 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f284c053b80 con 0x7f285406f450 2023-05-20T01:27:06.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.376+0000 7f285afe2700 1 -- 172.21.15.16:0/2090511477 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f28540747b0 con 0x7f285406f450 2023-05-20T01:27:06.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.378+0000 7f28517fa700 1 -- 172.21.15.16:0/2090511477 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v115) v1 ==== 74+0+20190 (secure 0 0 0) 0x7f284c017020 con 0x7f285406f450 2023-05-20T01:27:06.382 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:27:06.383 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":115,"fsid":"c2245cc5-ecab-4087-a59b-bc81db901676","created":"2023-05-20T01:16:09.549230+0000","modified":"2023-05-20T01:26:49.936387+0000","last_up_change":"2023-05-20T01:16:19.788624+0000","last_in_change":"2023-05-20T01:16:12.092160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-05-20T01:16:20.934366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-05-20T01:16:21.372294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-05-20T01:26:34.067017+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-05-20T01:26:35.040602+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data_extra","create_time":"2023-05-20T01:26:48.309738+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"317a83cd-19c8-4958-a301-6087ef342289","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6825","nonce":3923634179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6827","nonce":3923634179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6831","nonce":3923634179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6829","nonce":3923634179}]},"public_addr":"172.21.15.16:6825/3923634179","cluster_addr":"172.21.15.16:6827/3923634179","heartbeat_back_addr":"172.21.15.16:6831/3923634179","heartbeat_front_addr":"172.21.15.16:6829/3923634179","state":["exists","up"]},{"osd":1,"uuid":"5581046f-8219-4089-8d7d-b7846399505e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6809","nonce":3860264220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6811","nonce":3860264220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6815","nonce":3860264220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6813","nonce":3860264220}]},"public_addr":"172.21.15.16:6809/3860264220","cluster_addr":"172.21.15.16:6811/3860264220","heartbeat_back_addr":"172.21.15.16:6815/3860264220","heartbeat_front_addr":"172.21.15.16:6813/3860264220","state":["exists","up"]},{"osd":2,"uuid":"072e9e66-4133-478e-8d4a-64d46c2963f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6817","nonce":293187030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6819","nonce":293187030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6823","nonce":293187030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6821","nonce":293187030}]},"public_addr":"172.21.15.16:6817/293187030","cluster_addr":"172.21.15.16:6819/293187030","heartbeat_back_addr":"172.21.15.16:6823/293187030","heartbeat_front_addr":"172.21.15.16:6821/293187030","state":["exists","up"]},{"osd":3,"uuid":"11e1bba6-00f8-48dd-9028-d8d716b7f145","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6801","nonce":3669781890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6803","nonce":3669781890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6807","nonce":3669781890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6805","nonce":3669781890}]},"public_addr":"172.21.15.16:6801/3669781890","cluster_addr":"172.21.15.16:6803/3669781890","heartbeat_back_addr":"172.21.15.16:6807/3669781890","heartbeat_front_addr":"172.21.15.16:6805/3669781890","state":["exists","up"]},{"osd":4,"uuid":"b8909c3a-cf70-46cb-8d0b-009c707cf9d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6819","nonce":506317223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6823","nonce":506317223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6829","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6831","nonce":506317223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6827","nonce":506317223}]},"public_addr":"172.21.15.132:6819/506317223","cluster_addr":"172.21.15.132:6823/506317223","heartbeat_back_addr":"172.21.15.132:6831/506317223","heartbeat_front_addr":"172.21.15.132:6827/506317223","state":["exists","up"]},{"osd":5,"uuid":"b1818304-4583-47bf-a6ec-0bfbb5166233","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6818","nonce":1844689995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6822","nonce":1844689995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6830","nonce":1844689995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6826","nonce":1844689995}]},"public_addr":"172.21.15.132:6818/1844689995","cluster_addr":"172.21.15.132:6822/1844689995","heartbeat_back_addr":"172.21.15.132:6830/1844689995","heartbeat_front_addr":"172.21.15.132:6826/1844689995","state":["exists","up"]},{"osd":6,"uuid":"d37f4691-eed1-4a93-8818-18354f03f4a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6801","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6802","nonce":727581123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6805","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6807","nonce":727581123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6813","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6815","nonce":727581123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6811","nonce":727581123}]},"public_addr":"172.21.15.132:6802/727581123","cluster_addr":"172.21.15.132:6807/727581123","heartbeat_back_addr":"172.21.15.132:6815/727581123","heartbeat_front_addr":"172.21.15.132:6811/727581123","state":["exists","up"]},{"osd":7,"uuid":"4f712f62-5d1d-4816-8042-03209144f8b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6803","nonce":466001149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6806","nonce":466001149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6814","nonce":466001149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6810","nonce":466001149}]},"public_addr":"172.21.15.132:6803/466001149","cluster_addr":"172.21.15.132:6806/466001149","heartbeat_back_addr":"172.21.15.132:6814/466001149","heartbeat_front_addr":"172.21.15.132:6810/466001149","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.338863+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.278680+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305694+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.246835+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.276237+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.304441+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305421+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.267050+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-20T01:27:06.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.380+0000 7f285afe2700 1 -- 172.21.15.16:0/2090511477 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f283806ace0 msgr2=0x7f283806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:06.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.380+0000 7f285afe2700 1 --2- 172.21.15.16:0/2090511477 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f283806ace0 0x7f283806d190 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f2848006fd0 tx=0x7f2848008040 comp rx=0 tx=0).stop 2023-05-20T01:27:06.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.380+0000 7f285afe2700 1 -- 172.21.15.16:0/2090511477 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f285406f450 msgr2=0x7f28540c85a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:06.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.380+0000 7f285afe2700 1 --2- 172.21.15.16:0/2090511477 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f285406f450 0x7f28540c85a0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f284c00c7a0 tx=0x7f284c00cab0 comp rx=0 tx=0).stop 2023-05-20T01:27:06.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.380+0000 7f285afe2700 1 -- 172.21.15.16:0/2090511477 shutdown_connections 2023-05-20T01:27:06.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.380+0000 7f285afe2700 1 --2- 172.21.15.16:0/2090511477 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f285406f450 0x7f28540c85a0 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:06.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.380+0000 7f285afe2700 1 --2- 172.21.15.16:0/2090511477 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f283806ace0 0x7f283806d190 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:06.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.380+0000 7f285afe2700 1 --2- 172.21.15.16:0/2090511477 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f285406a510 0x7f2854153a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:06.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.380+0000 7f285afe2700 1 --2- 172.21.15.16:0/2090511477 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f285406d4b0 0x7f28540b3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:06.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.380+0000 7f285afe2700 1 -- 172.21.15.16:0/2090511477 >> 172.21.15.16:0/2090511477 conn(0x7f28540cffb0 msgr2=0x7f285415ad80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:27:06.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.380+0000 7f285afe2700 1 -- 172.21.15.16:0/2090511477 shutdown_connections 2023-05-20T01:27:06.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.380+0000 7f285afe2700 1 -- 172.21.15.16:0/2090511477 wait complete. 2023-05-20T01:27:06.397 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-05-20T01:16:20.934366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-05-20T01:16:21.372294+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2023-05-20T01:26:34.067017+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '111', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2023-05-20T01:26:35.040602+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '112', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data_extra', 'create_time': '2023-05-20T01:26:48.309738+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '115', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-05-20T01:27:06.398 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-05-20T01:27:06.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.522+0000 7f12863eb700 1 Processor -- start 2023-05-20T01:27:06.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.522+0000 7f12863eb700 1 -- start start 2023-05-20T01:27:06.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.523+0000 7f12863eb700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f1280071880 0x7f1280071c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:06.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.523+0000 7f12863eb700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f12800cbfb0 0x7f12800b2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:06.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.523+0000 7f12863eb700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12800d0e10 con 0x7f12800cd620 2023-05-20T01:27:06.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.523+0000 7f12863eb700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12800cd200 con 0x7f1280071880 2023-05-20T01:27:06.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.523+0000 7f12863eb700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1280072190 con 0x7f12800cbfb0 2023-05-20T01:27:06.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.523+0000 7f127f7fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f12800cd620 0x7f12800cb8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49968/0 (socket says 172.21.15.16:49968) 2023-05-20T01:27:06.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.523+0000 7f127f7fe700 1 -- 172.21.15.16:0/359506343 learned_addr learned my addr 172.21.15.16:0/359506343 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:27:06.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.523+0000 7f127ffff700 1 --2- 172.21.15.16:0/359506343 >> v2:172.21.15.132:3300/0 conn(0x7f1280071880 0x7f1280071c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:06.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.523+0000 7f1284988700 1 --2- 172.21.15.16:0/359506343 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f12800cbfb0 0x7f12800b2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:06.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.523+0000 7f127effd700 1 -- 172.21.15.16:0/359506343 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1207055803 0 0) 0x7f12800d0e10 con 0x7f12800cd620 2023-05-20T01:27:06.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.523+0000 7f127effd700 1 -- 172.21.15.16:0/359506343 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f126c0035b0 con 0x7f12800cd620 2023-05-20T01:27:06.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.524+0000 7f127effd700 1 -- 172.21.15.16:0/359506343 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2967220728 0 0) 0x7f126c0035b0 con 0x7f12800cd620 2023-05-20T01:27:06.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.524+0000 7f127effd700 1 -- 172.21.15.16:0/359506343 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f12800cbfb0 msgr2=0x7f12800b2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:06.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.524+0000 7f127effd700 1 --2- 172.21.15.16:0/359506343 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f12800cbfb0 0x7f12800b2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:06.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.524+0000 7f127effd700 1 -- 172.21.15.16:0/359506343 >> v2:172.21.15.132:3300/0 conn(0x7f1280071880 msgr2=0x7f1280071c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:06.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.524+0000 7f127effd700 1 --2- 172.21.15.16:0/359506343 >> v2:172.21.15.132:3300/0 conn(0x7f1280071880 0x7f1280071c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:06.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.524+0000 7f127effd700 1 -- 172.21.15.16:0/359506343 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12800cc580 con 0x7f12800cd620 2023-05-20T01:27:06.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.524+0000 7f127effd700 1 -- 172.21.15.16:0/359506343 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7f1268001670 con 0x7f12800cd620 2023-05-20T01:27:06.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.524+0000 7f127ffff700 1 --2- 172.21.15.16:0/359506343 >> v2:172.21.15.132:3300/0 conn(0x7f1280071880 0x7f1280071c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-05-20T01:27:06.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.524+0000 7f127effd700 1 -- 172.21.15.16:0/359506343 >> v1:172.21.15.16:6789/0 conn(0x7f12800cd620 legacy=0x7f12800cb8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:06.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.524+0000 7f127effd700 1 --2- 172.21.15.16:0/359506343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f126c003a00 0x7f126c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:06.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.524+0000 7f127effd700 1 --2- 172.21.15.16:0/359506343 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f12800cbfb0 0x7f126c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:06.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.524+0000 7f127effd700 1 --2- 172.21.15.16:0/359506343 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f126c004d90 0x7f126c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:06.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.524+0000 7f127effd700 1 -- 172.21.15.16:0/359506343 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12800cd200 con 0x7f126c003a00 2023-05-20T01:27:06.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.524+0000 7f127effd700 1 -- 172.21.15.16:0/359506343 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1280072190 con 0x7f126c004d90 2023-05-20T01:27:06.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.524+0000 7f127effd700 1 -- 172.21.15.16:0/359506343 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f126c00ba10 con 0x7f12800cbfb0 2023-05-20T01:27:06.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.524+0000 7f127ffff700 1 --2- 172.21.15.16:0/359506343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f126c003a00 0x7f126c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:06.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.524+0000 7f127f7fe700 1 --2- 172.21.15.16:0/359506343 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f12800cbfb0 0x7f126c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:06.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.524+0000 7f1284988700 1 --2- 172.21.15.16:0/359506343 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f126c004d90 0x7f126c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:06.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.524+0000 7f127ffff700 1 -- 172.21.15.16:0/359506343 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f12800cbfb0 msgr2=0x7f126c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:06.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.524+0000 7f127ffff700 1 --2- 172.21.15.16:0/359506343 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f12800cbfb0 0x7f126c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:06.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.524+0000 7f127ffff700 1 -- 172.21.15.16:0/359506343 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f126c004d90 msgr2=0x7f126c005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:06.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.524+0000 7f127ffff700 1 --2- 172.21.15.16:0/359506343 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f126c004d90 0x7f126c005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:06.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.524+0000 7f127ffff700 1 -- 172.21.15.16:0/359506343 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f126c0035b0 con 0x7f126c003a00 2023-05-20T01:27:06.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.524+0000 7f1284988700 1 --2- 172.21.15.16:0/359506343 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f126c004d90 0x7f126c005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:27:06.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.524+0000 7f127ffff700 1 --2- 172.21.15.16:0/359506343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f126c003a00 0x7f126c003e10 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f1270009d10 tx=0x7f126c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d419477e3eea1f0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:06.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.524+0000 7f127effd700 1 -- 172.21.15.16:0/359506343 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1270009e60 con 0x7f126c003a00 2023-05-20T01:27:06.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.525+0000 7f127effd700 1 -- 172.21.15.16:0/359506343 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f127000e9a0 con 0x7f126c003a00 2023-05-20T01:27:06.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.525+0000 7f127effd700 1 -- 172.21.15.16:0/359506343 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f127000ed10 con 0x7f126c003a00 2023-05-20T01:27:06.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.525+0000 7f12863eb700 1 -- 172.21.15.16:0/359506343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f126c003a00 msgr2=0x7f126c003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:06.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.525+0000 7f12863eb700 1 --2- 172.21.15.16:0/359506343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f126c003a00 0x7f126c003e10 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f1270009d10 tx=0x7f126c005880 comp rx=0 tx=0).stop 2023-05-20T01:27:06.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.525+0000 7f1284988700 1 -- 172.21.15.16:0/359506343 reap_dead start 2023-05-20T01:27:06.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.525+0000 7f12863eb700 1 -- 172.21.15.16:0/359506343 shutdown_connections 2023-05-20T01:27:06.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.525+0000 7f12863eb700 1 -- 172.21.15.16:0/359506343 >> 172.21.15.16:0/359506343 conn(0x7f12800cffd0 msgr2=0x7f12800d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:27:06.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.525+0000 7f12863eb700 1 -- 172.21.15.16:0/359506343 shutdown_connections 2023-05-20T01:27:06.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.525+0000 7f12863eb700 1 -- 172.21.15.16:0/359506343 wait complete. 2023-05-20T01:27:06.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.527+0000 7f12863eb700 1 Processor -- start 2023-05-20T01:27:06.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.527+0000 7f12863eb700 1 -- start start 2023-05-20T01:27:06.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.528+0000 7f12863eb700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f126c003a00 0x7f12801534f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:06.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.528+0000 7f12863eb700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f128015c0a0 0x7f12800afb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:06.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.528+0000 7f12863eb700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f128015c510 0x7f12800b0060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:06.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.528+0000 7f12863eb700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12800c4130 con 0x7f128015c0a0 2023-05-20T01:27:06.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.528+0000 7f12863eb700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1280155d60 con 0x7f128015c510 2023-05-20T01:27:06.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.528+0000 7f12863eb700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1280156040 con 0x7f126c003a00 2023-05-20T01:27:06.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.528+0000 7f127f7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f128015c0a0 0x7f12800afb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:06.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.528+0000 7f127f7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f128015c0a0 0x7f12800afb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38452/0 (socket says 172.21.15.16:38452) 2023-05-20T01:27:06.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.528+0000 7f127f7fe700 1 -- 172.21.15.16:0/1336728014 learned_addr learned my addr 172.21.15.16:0/1336728014 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:27:06.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.528+0000 7f1284988700 1 --2- 172.21.15.16:0/1336728014 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f128015c510 0x7f12800b0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:06.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.528+0000 7f127ffff700 1 --2- 172.21.15.16:0/1336728014 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f126c003a00 0x7f12801534f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:06.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.528+0000 7f127f7fe700 1 -- 172.21.15.16:0/1336728014 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f126c003a00 msgr2=0x7f12801534f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:06.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.528+0000 7f127f7fe700 1 --2- 172.21.15.16:0/1336728014 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f126c003a00 0x7f12801534f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:06.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.528+0000 7f127f7fe700 1 -- 172.21.15.16:0/1336728014 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f128015c510 msgr2=0x7f12800b0060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:06.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.528+0000 7f127f7fe700 1 --2- 172.21.15.16:0/1336728014 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f128015c510 0x7f12800b0060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:06.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.528+0000 7f127f7fe700 1 -- 172.21.15.16:0/1336728014 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12800ccb40 con 0x7f128015c0a0 2023-05-20T01:27:06.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.528+0000 7f1284988700 1 --2- 172.21.15.16:0/1336728014 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f128015c510 0x7f12800b0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:27:06.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.529+0000 7f127f7fe700 1 --2- 172.21.15.16:0/1336728014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f128015c0a0 0x7f12800afb20 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f1268000c00 tx=0x7f126800ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:06.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.529+0000 7f127cff9700 1 -- 172.21.15.16:0/1336728014 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f126800c5f0 con 0x7f128015c0a0 2023-05-20T01:27:06.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.529+0000 7f127cff9700 1 -- 172.21.15.16:0/1336728014 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f126800c790 con 0x7f128015c0a0 2023-05-20T01:27:06.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.529+0000 7f127cff9700 1 -- 172.21.15.16:0/1336728014 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f126800cb40 con 0x7f128015c0a0 2023-05-20T01:27:06.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.529+0000 7f12863eb700 1 -- 172.21.15.16:0/1336728014 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12800cc860 con 0x7f128015c0a0 2023-05-20T01:27:06.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.529+0000 7f12863eb700 1 -- 172.21.15.16:0/1336728014 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12800cce20 con 0x7f128015c0a0 2023-05-20T01:27:06.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.530+0000 7f12863eb700 1 -- 172.21.15.16:0/1336728014 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12800c42a0 con 0x7f128015c0a0 2023-05-20T01:27:06.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.530+0000 7f127cff9700 1 -- 172.21.15.16:0/1336728014 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f126800cca0 con 0x7f128015c0a0 2023-05-20T01:27:06.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.531+0000 7f127cff9700 1 --2- 172.21.15.16:0/1336728014 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f126006ace0 0x7f126006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:06.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.531+0000 7f127cff9700 1 -- 172.21.15.16:0/1336728014 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6786+0+0 (secure 0 0 0) 0x7f1268089640 con 0x7f128015c0a0 2023-05-20T01:27:06.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.531+0000 7f127ffff700 1 --2- 172.21.15.16:0/1336728014 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f126006ace0 0x7f126006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:06.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.531+0000 7f127ffff700 1 --2- 172.21.15.16:0/1336728014 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f126006ace0 0x7f126006d190 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f1270000c00 tx=0x7f1270008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:06.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.535+0000 7f127cff9700 1 -- 172.21.15.16:0/1336728014 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f1268054410 con 0x7f128015c0a0 2023-05-20T01:27:06.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.731+0000 7f12863eb700 1 -- 172.21.15.16:0/1336728014 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f12800747b0 con 0x7f128015c0a0 2023-05-20T01:27:06.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.731+0000 7f127cff9700 1 -- 172.21.15.16:0/1336728014 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v115) v1 ==== 92+0+10 (secure 0 0 0) 0x7f1268057a30 con 0x7f128015c0a0 2023-05-20T01:27:06.736 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 8 2023-05-20T01:27:06.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.733+0000 7f12863eb700 1 -- 172.21.15.16:0/1336728014 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f126006ace0 msgr2=0x7f126006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:06.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.733+0000 7f12863eb700 1 --2- 172.21.15.16:0/1336728014 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f126006ace0 0x7f126006d190 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f1270000c00 tx=0x7f1270008030 comp rx=0 tx=0).stop 2023-05-20T01:27:06.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.733+0000 7f12863eb700 1 -- 172.21.15.16:0/1336728014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f128015c0a0 msgr2=0x7f12800afb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:06.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.733+0000 7f12863eb700 1 --2- 172.21.15.16:0/1336728014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f128015c0a0 0x7f12800afb20 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f1268000c00 tx=0x7f126800ebd0 comp rx=0 tx=0).stop 2023-05-20T01:27:06.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.733+0000 7f12863eb700 1 -- 172.21.15.16:0/1336728014 shutdown_connections 2023-05-20T01:27:06.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.733+0000 7f12863eb700 1 --2- 172.21.15.16:0/1336728014 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f128015c510 0x7f12800b0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:06.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.733+0000 7f12863eb700 1 --2- 172.21.15.16:0/1336728014 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f126006ace0 0x7f126006d190 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:06.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.733+0000 7f12863eb700 1 --2- 172.21.15.16:0/1336728014 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f126c003a00 0x7f12801534f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:06.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.733+0000 7f12863eb700 1 --2- 172.21.15.16:0/1336728014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f128015c0a0 0x7f12800afb20 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:06.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.733+0000 7f12863eb700 1 -- 172.21.15.16:0/1336728014 >> 172.21.15.16:0/1336728014 conn(0x7f12800cffd0 msgr2=0x7f128006a510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:27:06.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.733+0000 7f12863eb700 1 -- 172.21.15.16:0/1336728014 shutdown_connections 2023-05-20T01:27:06.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.733+0000 7f12863eb700 1 -- 172.21.15.16:0/1336728014 wait complete. 2023-05-20T01:27:06.750 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-05-20T01:27:06.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.872+0000 7f2cd829c700 1 Processor -- start 2023-05-20T01:27:06.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.873+0000 7f2cd829c700 1 -- start start 2023-05-20T01:27:06.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.873+0000 7f2cd829c700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f2cd0071880 0x7f2cd0071c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:06.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.873+0000 7f2cd829c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2cd00cd620 0x7f2cd00cb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:06.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.874+0000 7f2cd829c700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cd00d0e10 con 0x7f2cd00cbde0 2023-05-20T01:27:06.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.874+0000 7f2cd829c700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cd00cd200 con 0x7f2cd0071880 2023-05-20T01:27:06.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.874+0000 7f2cd829c700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd0072190 con 0x7f2cd00cd620 2023-05-20T01:27:06.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.874+0000 7f2cd6038700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f2cd0071880 0x7f2cd0071c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:06.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.874+0000 7f2cd5837700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2cd00cd620 0x7f2cd00cb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:06.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.874+0000 7f2cd6839700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f2cd00cbde0 0x7f2cd006d690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.16:59702/0 (socket says 172.21.15.16:59702) 2023-05-20T01:27:06.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.874+0000 7f2cd6038700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f2cd0071880 0x7f2cd0071c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.16:35480/0 (socket says 172.21.15.16:35480) 2023-05-20T01:27:06.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.874+0000 7f2cd5837700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2cd00cd620 0x7f2cd00cb8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.16:38496/0 (socket says 172.21.15.16:38496) 2023-05-20T01:27:06.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.874+0000 7f2cd6839700 1 -- 172.21.15.16:0/1644072428 learned_addr learned my addr 172.21.15.16:0/1644072428 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:27:06.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.874+0000 7f2cd6038700 1 -- 172.21.15.16:0/1644072428 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2cd00cd620 msgr2=0x7f2cd00cb8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:06.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.874+0000 7f2cd6038700 1 --2- 172.21.15.16:0/1644072428 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2cd00cd620 0x7f2cd00cb8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:06.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.874+0000 7f2cd6038700 1 -- 172.21.15.16:0/1644072428 >> v1:172.21.15.132:6789/0 conn(0x7f2cd00cbde0 legacy=0x7f2cd006d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:06.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.874+0000 7f2cd6038700 1 -- 172.21.15.16:0/1644072428 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cd00cc580 con 0x7f2cd0071880 2023-05-20T01:27:06.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.874+0000 7f2cd5036700 1 -- 172.21.15.16:0/1644072428 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 980520692 0 0) 0x7f2cd00d0e10 con 0x7f2cd00cbde0 2023-05-20T01:27:06.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.875+0000 7f2cd6038700 1 --2- 172.21.15.16:0/1644072428 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2cd0071880 0x7f2cd0071c50 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc009820 tx=0x7f2ccc009b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f035a4d062dc1e4d server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:06.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.875+0000 7f2cd5036700 1 -- 172.21.15.16:0/1644072428 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ccc005e30 con 0x7f2cd0071880 2023-05-20T01:27:06.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.875+0000 7f2cd5036700 1 -- 172.21.15.16:0/1644072428 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ccc006030 con 0x7f2cd0071880 2023-05-20T01:27:06.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.875+0000 7f2cd5036700 1 -- 172.21.15.16:0/1644072428 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ccc0063c0 con 0x7f2cd0071880 2023-05-20T01:27:06.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.875+0000 7f2cd829c700 1 -- 172.21.15.16:0/1644072428 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2cd0071880 msgr2=0x7f2cd0071c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:06.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.875+0000 7f2cd829c700 1 --2- 172.21.15.16:0/1644072428 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2cd0071880 0x7f2cd0071c50 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc009820 tx=0x7f2ccc009b30 comp rx=0 tx=0).stop 2023-05-20T01:27:06.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.875+0000 7f2cd829c700 1 -- 172.21.15.16:0/1644072428 shutdown_connections 2023-05-20T01:27:06.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.875+0000 7f2cd829c700 1 --2- 172.21.15.16:0/1644072428 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2cd0071880 0x7f2cd0071c50 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:06.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.875+0000 7f2cd829c700 1 --2- 172.21.15.16:0/1644072428 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2cd00cd620 0x7f2cd00cb8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:06.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.875+0000 7f2cd829c700 1 -- 172.21.15.16:0/1644072428 >> 172.21.15.16:0/1644072428 conn(0x7f2cd00cffd0 msgr2=0x7f2cd00d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:27:06.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.875+0000 7f2cd829c700 1 -- 172.21.15.16:0/1644072428 shutdown_connections 2023-05-20T01:27:06.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.875+0000 7f2cd829c700 1 -- 172.21.15.16:0/1644072428 wait complete. 2023-05-20T01:27:06.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.877+0000 7f2cd829c700 1 Processor -- start 2023-05-20T01:27:06.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.877+0000 7f2cd829c700 1 -- start start 2023-05-20T01:27:06.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.877+0000 7f2cd829c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2cd0071880 0x7f2cd014e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:06.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.877+0000 7f2cd829c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2cd00cbde0 0x7f2cd0158680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:06.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.877+0000 7f2cd829c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cd00cd620 0x7f2cd0158bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:06.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.877+0000 7f2cd829c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd01f3900 con 0x7f2cd00cd620 2023-05-20T01:27:06.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.877+0000 7f2cd829c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd01f3ba0 con 0x7f2cd00cbde0 2023-05-20T01:27:06.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.877+0000 7f2cd829c700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd01f3e80 con 0x7f2cd0071880 2023-05-20T01:27:06.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.877+0000 7f2cd6839700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cd00cd620 0x7f2cd0158bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:06.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.877+0000 7f2cd6839700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cd00cd620 0x7f2cd0158bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38466/0 (socket says 172.21.15.16:38466) 2023-05-20T01:27:06.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.877+0000 7f2cd6839700 1 -- 172.21.15.16:0/294803628 learned_addr learned my addr 172.21.15.16:0/294803628 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:27:06.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.877+0000 7f2cd6038700 1 --2- 172.21.15.16:0/294803628 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2cd0071880 0x7f2cd014e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:06.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.878+0000 7f2cd5837700 1 --2- 172.21.15.16:0/294803628 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2cd00cbde0 0x7f2cd0158680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:06.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.878+0000 7f2cd6839700 1 -- 172.21.15.16:0/294803628 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2cd0071880 msgr2=0x7f2cd014e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:06.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.878+0000 7f2cd6839700 1 --2- 172.21.15.16:0/294803628 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2cd0071880 0x7f2cd014e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:06.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.878+0000 7f2cd6839700 1 -- 172.21.15.16:0/294803628 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2cd00cbde0 msgr2=0x7f2cd0158680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:06.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.878+0000 7f2cd6839700 1 --2- 172.21.15.16:0/294803628 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2cd00cbde0 0x7f2cd0158680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:06.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.878+0000 7f2cd6839700 1 -- 172.21.15.16:0/294803628 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ccc009430 con 0x7f2cd00cd620 2023-05-20T01:27:06.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.878+0000 7f2cd6839700 1 --2- 172.21.15.16:0/294803628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cd00cd620 0x7f2cd0158bc0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f2cc800cb60 tx=0x7f2cc800ce70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:06.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.878+0000 7f2cc2ffd700 1 -- 172.21.15.16:0/294803628 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc800a9e0 con 0x7f2cd00cd620 2023-05-20T01:27:06.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.878+0000 7f2cc2ffd700 1 -- 172.21.15.16:0/294803628 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cc800ab40 con 0x7f2cd00cd620 2023-05-20T01:27:06.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.878+0000 7f2cc2ffd700 1 -- 172.21.15.16:0/294803628 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc80070d0 con 0x7f2cd00cd620 2023-05-20T01:27:06.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.878+0000 7f2cd829c700 1 -- 172.21.15.16:0/294803628 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cd00ccb40 con 0x7f2cd00cd620 2023-05-20T01:27:06.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.878+0000 7f2cd829c700 1 -- 172.21.15.16:0/294803628 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cd00cc860 con 0x7f2cd00cd620 2023-05-20T01:27:06.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.879+0000 7f2cd829c700 1 -- 172.21.15.16:0/294803628 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cd00cce20 con 0x7f2cd00cd620 2023-05-20T01:27:06.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.880+0000 7f2cc2ffd700 1 -- 172.21.15.16:0/294803628 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2cc8007230 con 0x7f2cd00cd620 2023-05-20T01:27:06.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.880+0000 7f2cc2ffd700 1 --2- 172.21.15.16:0/294803628 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f2cb406ace0 0x7f2cb406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:06.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.880+0000 7f2cc2ffd700 1 -- 172.21.15.16:0/294803628 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6786+0+0 (secure 0 0 0) 0x7f2cc8089140 con 0x7f2cd00cd620 2023-05-20T01:27:06.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.880+0000 7f2cd6038700 1 --2- 172.21.15.16:0/294803628 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f2cb406ace0 0x7f2cb406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:06.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.881+0000 7f2cd6038700 1 --2- 172.21.15.16:0/294803628 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f2cb406ace0 0x7f2cb406d190 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc000c00 tx=0x7f2ccc008070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:06.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:06.885+0000 7f2cc2ffd700 1 -- 172.21.15.16:0/294803628 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f2cc8053fa0 con 0x7f2cd00cd620 2023-05-20T01:27:07.038 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:27:07.040 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:27:07.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.081+0000 7f2cd829c700 1 -- 172.21.15.16:0/294803628 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2cd00747b0 con 0x7f2cd00cd620 2023-05-20T01:27:07.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.081+0000 7f2cc2ffd700 1 -- 172.21.15.16:0/294803628 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v115) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2cc80575c0 con 0x7f2cd00cd620 2023-05-20T01:27:07.086 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-05-20T01:27:07.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.083+0000 7f2cd829c700 1 -- 172.21.15.16:0/294803628 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f2cb406ace0 msgr2=0x7f2cb406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:07.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.083+0000 7f2cd829c700 1 --2- 172.21.15.16:0/294803628 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f2cb406ace0 0x7f2cb406d190 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc000c00 tx=0x7f2ccc008070 comp rx=0 tx=0).stop 2023-05-20T01:27:07.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.083+0000 7f2cd829c700 1 -- 172.21.15.16:0/294803628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cd00cd620 msgr2=0x7f2cd0158bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:07.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.083+0000 7f2cd829c700 1 --2- 172.21.15.16:0/294803628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cd00cd620 0x7f2cd0158bc0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f2cc800cb60 tx=0x7f2cc800ce70 comp rx=0 tx=0).stop 2023-05-20T01:27:07.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.083+0000 7f2cd829c700 1 -- 172.21.15.16:0/294803628 shutdown_connections 2023-05-20T01:27:07.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.083+0000 7f2cd829c700 1 --2- 172.21.15.16:0/294803628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cd00cd620 0x7f2cd0158bc0 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:07.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.083+0000 7f2cd829c700 1 --2- 172.21.15.16:0/294803628 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f2cb406ace0 0x7f2cb406d190 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:07.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.083+0000 7f2cd829c700 1 --2- 172.21.15.16:0/294803628 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2cd0071880 0x7f2cd014e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:07.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.083+0000 7f2cd829c700 1 --2- 172.21.15.16:0/294803628 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2cd00cbde0 0x7f2cd0158680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:07.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.083+0000 7f2cd829c700 1 -- 172.21.15.16:0/294803628 >> 172.21.15.16:0/294803628 conn(0x7f2cd00cffd0 msgr2=0x7f2cd00b5550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:27:07.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.083+0000 7f2cd829c700 1 -- 172.21.15.16:0/294803628 shutdown_connections 2023-05-20T01:27:07.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.083+0000 7f2cd829c700 1 -- 172.21.15.16:0/294803628 wait complete. 2023-05-20T01:27:07.101 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-05-20T01:27:07.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.265+0000 7ffb54f59700 1 Processor -- start 2023-05-20T01:27:07.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.265+0000 7ffb54f59700 1 -- start start 2023-05-20T01:27:07.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.266+0000 7ffb54f59700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffb50159b50 0x7ffb5006e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:07.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.266+0000 7ffb54f59700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ffb500cd1c0 0x7ffb500710b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:07.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.266+0000 7ffb54f59700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7ffb500cd630 0x7ffb500715f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:07.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.266+0000 7ffb54f59700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb500ccd30 con 0x7ffb500cd1c0 2023-05-20T01:27:07.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.266+0000 7ffb54f59700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb500ce990 con 0x7ffb500cd630 2023-05-20T01:27:07.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.266+0000 7ffb54f59700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb500cc6e0 con 0x7ffb50159b50 2023-05-20T01:27:07.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.266+0000 7ffb4dd9b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ffb500cd1c0 0x7ffb500710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:07.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.266+0000 7ffb4dd9b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ffb500cd1c0 0x7ffb500710b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38470/0 (socket says 172.21.15.16:38470) 2023-05-20T01:27:07.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.266+0000 7ffb4dd9b700 1 -- 172.21.15.16:0/2636908381 learned_addr learned my addr 172.21.15.16:0/2636908381 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:27:07.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.266+0000 7ffb4dd9b700 1 -- 172.21.15.16:0/2636908381 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffb50159b50 msgr2=0x7ffb5006e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:07.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.266+0000 7ffb4ed9d700 1 --2- 172.21.15.16:0/2636908381 >> v2:172.21.15.132:3300/0 conn(0x7ffb500cd630 0x7ffb500715f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:07.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.266+0000 7ffb4e59c700 1 --2- 172.21.15.16:0/2636908381 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffb50159b50 0x7ffb5006e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:07.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.266+0000 7ffb4dd9b700 1 --2- 172.21.15.16:0/2636908381 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffb50159b50 0x7ffb5006e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:07.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.266+0000 7ffb4dd9b700 1 -- 172.21.15.16:0/2636908381 >> v2:172.21.15.132:3300/0 conn(0x7ffb500cd630 msgr2=0x7ffb500715f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:07.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.266+0000 7ffb4dd9b700 1 --2- 172.21.15.16:0/2636908381 >> v2:172.21.15.132:3300/0 conn(0x7ffb500cd630 0x7ffb500715f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:07.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.266+0000 7ffb4dd9b700 1 -- 172.21.15.16:0/2636908381 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb500cc9c0 con 0x7ffb500cd1c0 2023-05-20T01:27:07.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.266+0000 7ffb4ed9d700 1 --2- 172.21.15.16:0/2636908381 >> v2:172.21.15.132:3300/0 conn(0x7ffb500cd630 0x7ffb500715f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:27:07.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.267+0000 7ffb4dd9b700 1 --2- 172.21.15.16:0/2636908381 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb500cd1c0 0x7ffb500710b0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7ffb5006cfd0 tx=0x7ffb40009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a17644c731646955 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:07.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.267+0000 7ffb4d59a700 1 -- 172.21.15.16:0/2636908381 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb4000f8a0 con 0x7ffb500cd1c0 2023-05-20T01:27:07.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.267+0000 7ffb4d59a700 1 -- 172.21.15.16:0/2636908381 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb4000fa00 con 0x7ffb500cd1c0 2023-05-20T01:27:07.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.267+0000 7ffb4d59a700 1 -- 172.21.15.16:0/2636908381 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb4000fd90 con 0x7ffb500cd1c0 2023-05-20T01:27:07.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.267+0000 7ffb54f59700 1 -- 172.21.15.16:0/2636908381 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb500cd1c0 msgr2=0x7ffb500710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:07.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.267+0000 7ffb54f59700 1 --2- 172.21.15.16:0/2636908381 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb500cd1c0 0x7ffb500710b0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7ffb5006cfd0 tx=0x7ffb40009bd0 comp rx=0 tx=0).stop 2023-05-20T01:27:07.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.267+0000 7ffb54f59700 1 -- 172.21.15.16:0/2636908381 shutdown_connections 2023-05-20T01:27:07.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.267+0000 7ffb54f59700 1 --2- 172.21.15.16:0/2636908381 >> v2:172.21.15.132:3300/0 conn(0x7ffb500cd630 0x7ffb500715f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:07.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.267+0000 7ffb54f59700 1 --2- 172.21.15.16:0/2636908381 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffb50159b50 0x7ffb5006e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:07.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.267+0000 7ffb54f59700 1 --2- 172.21.15.16:0/2636908381 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb500cd1c0 0x7ffb500710b0 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:07.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.267+0000 7ffb54f59700 1 -- 172.21.15.16:0/2636908381 >> 172.21.15.16:0/2636908381 conn(0x7ffb500cffd0 msgr2=0x7ffb500d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:27:07.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.267+0000 7ffb54f59700 1 -- 172.21.15.16:0/2636908381 shutdown_connections 2023-05-20T01:27:07.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.267+0000 7ffb54f59700 1 -- 172.21.15.16:0/2636908381 wait complete. 2023-05-20T01:27:07.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.269+0000 7ffb54f59700 1 Processor -- start 2023-05-20T01:27:07.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.269+0000 7ffb54f59700 1 -- start start 2023-05-20T01:27:07.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.270+0000 7ffb54f59700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffb500cd1c0 0x7ffb5014fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:07.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.270+0000 7ffb54f59700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb500cd630 0x7ffb500b57e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:07.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.270+0000 7ffb54f59700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffb50159b50 0x7ffb50154850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:07.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.270+0000 7ffb54f59700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb50150910 con 0x7ffb500cd630 2023-05-20T01:27:07.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.270+0000 7ffb54f59700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb501567f0 con 0x7ffb500cd1c0 2023-05-20T01:27:07.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.270+0000 7ffb54f59700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb50155be0 con 0x7ffb50159b50 2023-05-20T01:27:07.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.270+0000 7ffb4dd9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb500cd630 0x7ffb500b57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:07.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.270+0000 7ffb4dd9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb500cd630 0x7ffb500b57e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38478/0 (socket says 172.21.15.16:38478) 2023-05-20T01:27:07.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.270+0000 7ffb4dd9b700 1 -- 172.21.15.16:0/527881380 learned_addr learned my addr 172.21.15.16:0/527881380 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:27:07.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.270+0000 7ffb4e59c700 1 --2- 172.21.15.16:0/527881380 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffb500cd1c0 0x7ffb5014fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:07.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.270+0000 7ffb4ed9d700 1 --2- 172.21.15.16:0/527881380 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffb50159b50 0x7ffb50154850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:07.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.270+0000 7ffb4dd9b700 1 -- 172.21.15.16:0/527881380 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffb50159b50 msgr2=0x7ffb50154850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:07.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.270+0000 7ffb4dd9b700 1 --2- 172.21.15.16:0/527881380 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffb50159b50 0x7ffb50154850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:07.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.270+0000 7ffb4dd9b700 1 -- 172.21.15.16:0/527881380 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffb500cd1c0 msgr2=0x7ffb5014fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:07.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.270+0000 7ffb4dd9b700 1 --2- 172.21.15.16:0/527881380 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffb500cd1c0 0x7ffb5014fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:07.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.270+0000 7ffb4dd9b700 1 -- 172.21.15.16:0/527881380 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb40009550 con 0x7ffb500cd630 2023-05-20T01:27:07.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.270+0000 7ffb4dd9b700 1 --2- 172.21.15.16:0/527881380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb500cd630 0x7ffb500b57e0 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7ffb500675d0 tx=0x7ffb40004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:07.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.270+0000 7ffb3effd700 1 -- 172.21.15.16:0/527881380 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb4001e070 con 0x7ffb500cd630 2023-05-20T01:27:07.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.270+0000 7ffb3effd700 1 -- 172.21.15.16:0/527881380 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb40004260 con 0x7ffb500cd630 2023-05-20T01:27:07.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.270+0000 7ffb54f59700 1 -- 172.21.15.16:0/527881380 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb50071e80 con 0x7ffb500cd630 2023-05-20T01:27:07.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.270+0000 7ffb3effd700 1 -- 172.21.15.16:0/527881380 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb400045f0 con 0x7ffb500cd630 2023-05-20T01:27:07.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.271+0000 7ffb54f59700 1 -- 172.21.15.16:0/527881380 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb5006e5e0 con 0x7ffb500cd630 2023-05-20T01:27:07.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.272+0000 7ffb54f59700 1 -- 172.21.15.16:0/527881380 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb500cec70 con 0x7ffb500cd630 2023-05-20T01:27:07.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.272+0000 7ffb3effd700 1 -- 172.21.15.16:0/527881380 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ffb400036a0 con 0x7ffb500cd630 2023-05-20T01:27:07.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.272+0000 7ffb3effd700 1 --2- 172.21.15.16:0/527881380 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ffb3006ace0 0x7ffb3006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:07.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.272+0000 7ffb3effd700 1 -- 172.21.15.16:0/527881380 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6786+0+0 (secure 0 0 0) 0x7ffb4008c590 con 0x7ffb500cd630 2023-05-20T01:27:07.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.272+0000 7ffb4e59c700 1 --2- 172.21.15.16:0/527881380 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ffb3006ace0 0x7ffb3006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:07.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.273+0000 7ffb4e59c700 1 --2- 172.21.15.16:0/527881380 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ffb3006ace0 0x7ffb3006d190 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7ffb500b1700 tx=0x7ffb44008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:07.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.277+0000 7ffb3effd700 1 -- 172.21.15.16:0/527881380 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7ffb400573f0 con 0x7ffb500cd630 2023-05-20T01:27:07.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.478+0000 7ffb54f59700 1 -- 172.21.15.16:0/527881380 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ffb500747b0 con 0x7ffb500cd630 2023-05-20T01:27:07.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.479+0000 7ffb3effd700 1 -- 172.21.15.16:0/527881380 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v115) v1 ==== 104+0+11 (secure 0 0 0) 0x7ffb4005aa10 con 0x7ffb500cd630 2023-05-20T01:27:07.484 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-05-20T01:27:07.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.481+0000 7ffb54f59700 1 -- 172.21.15.16:0/527881380 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ffb3006ace0 msgr2=0x7ffb3006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:07.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.481+0000 7ffb54f59700 1 --2- 172.21.15.16:0/527881380 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ffb3006ace0 0x7ffb3006d190 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7ffb500b1700 tx=0x7ffb44008040 comp rx=0 tx=0).stop 2023-05-20T01:27:07.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.481+0000 7ffb54f59700 1 -- 172.21.15.16:0/527881380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb500cd630 msgr2=0x7ffb500b57e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:07.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.481+0000 7ffb54f59700 1 --2- 172.21.15.16:0/527881380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb500cd630 0x7ffb500b57e0 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7ffb500675d0 tx=0x7ffb40004750 comp rx=0 tx=0).stop 2023-05-20T01:27:07.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.481+0000 7ffb54f59700 1 -- 172.21.15.16:0/527881380 shutdown_connections 2023-05-20T01:27:07.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.481+0000 7ffb54f59700 1 --2- 172.21.15.16:0/527881380 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ffb3006ace0 0x7ffb3006d190 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:07.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.481+0000 7ffb54f59700 1 --2- 172.21.15.16:0/527881380 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffb50159b50 0x7ffb50154850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:07.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.481+0000 7ffb54f59700 1 --2- 172.21.15.16:0/527881380 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffb500cd1c0 0x7ffb5014fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:07.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.481+0000 7ffb54f59700 1 --2- 172.21.15.16:0/527881380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb500cd630 0x7ffb500b57e0 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:07.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.481+0000 7ffb54f59700 1 -- 172.21.15.16:0/527881380 >> 172.21.15.16:0/527881380 conn(0x7ffb500cffd0 msgr2=0x7ffb500cb800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:27:07.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.481+0000 7ffb54f59700 1 -- 172.21.15.16:0/527881380 shutdown_connections 2023-05-20T01:27:07.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.481+0000 7ffb54f59700 1 -- 172.21.15.16:0/527881380 wait complete. 2023-05-20T01:27:07.497 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-05-20T01:27:07.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.622+0000 7f0f5e855700 1 Processor -- start 2023-05-20T01:27:07.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.622+0000 7f0f5e855700 1 -- start start 2023-05-20T01:27:07.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.623+0000 7f0f5e855700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0f5815ab80 0x7f0f580cda80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:07.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.623+0000 7f0f5e855700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0f580cdfc0 0x7f0f580cec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:07.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.623+0000 7f0f5e855700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f580ce4d0 con 0x7f0f580cf1c0 2023-05-20T01:27:07.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.623+0000 7f0f5e855700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f580cb700 con 0x7f0f580cdfc0 2023-05-20T01:27:07.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.623+0000 7f0f5e855700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f580cf640 con 0x7f0f5815ab80 2023-05-20T01:27:07.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.623+0000 7f0f577fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0f580cdfc0 0x7f0f580cec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:07.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.623+0000 7f0f577fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0f580cdfc0 0x7f0f580cec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38486/0 (socket says 172.21.15.16:38486) 2023-05-20T01:27:07.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.623+0000 7f0f577fe700 1 -- 172.21.15.16:0/4173030526 learned_addr learned my addr 172.21.15.16:0/4173030526 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:27:07.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.623+0000 7f0f57fff700 1 --2- 172.21.15.16:0/4173030526 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0f5815ab80 0x7f0f580cda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:07.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.623+0000 7f0f577fe700 1 -- 172.21.15.16:0/4173030526 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0f5815ab80 msgr2=0x7f0f580cda80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:07.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.623+0000 7f0f577fe700 1 --2- 172.21.15.16:0/4173030526 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0f5815ab80 0x7f0f580cda80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:07.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.623+0000 7f0f577fe700 1 -- 172.21.15.16:0/4173030526 >> v1:172.21.15.132:6789/0 conn(0x7f0f580cf1c0 legacy=0x7f0f5815e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:07.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.623+0000 7f0f577fe700 1 -- 172.21.15.16:0/4173030526 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f580ce7b0 con 0x7f0f580cdfc0 2023-05-20T01:27:07.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.623+0000 7f0f57fff700 1 --2- 172.21.15.16:0/4173030526 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0f5815ab80 0x7f0f580cda80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:27:07.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.624+0000 7f0f577fe700 1 --2- 172.21.15.16:0/4173030526 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f580cdfc0 0x7f0f580cec80 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f0f480098f0 tx=0x7f0f48009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea9995d9e817a1cb server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:07.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.624+0000 7f0f56ffd700 1 -- 172.21.15.16:0/4173030526 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f4800b8a0 con 0x7f0f580cdfc0 2023-05-20T01:27:07.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.624+0000 7f0f56ffd700 1 -- 172.21.15.16:0/4173030526 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f4800ba00 con 0x7f0f580cdfc0 2023-05-20T01:27:07.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.624+0000 7f0f56ffd700 1 -- 172.21.15.16:0/4173030526 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f4800bd90 con 0x7f0f580cdfc0 2023-05-20T01:27:07.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.624+0000 7f0f5e855700 1 -- 172.21.15.16:0/4173030526 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f580cdfc0 msgr2=0x7f0f580cec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:07.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.624+0000 7f0f5e855700 1 --2- 172.21.15.16:0/4173030526 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f580cdfc0 0x7f0f580cec80 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f0f480098f0 tx=0x7f0f48009c00 comp rx=0 tx=0).stop 2023-05-20T01:27:07.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.624+0000 7f0f5e855700 1 -- 172.21.15.16:0/4173030526 shutdown_connections 2023-05-20T01:27:07.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.624+0000 7f0f5e855700 1 --2- 172.21.15.16:0/4173030526 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0f5815ab80 0x7f0f580cda80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:07.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.624+0000 7f0f5e855700 1 --2- 172.21.15.16:0/4173030526 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f580cdfc0 0x7f0f580cec80 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:07.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.624+0000 7f0f5e855700 1 -- 172.21.15.16:0/4173030526 >> 172.21.15.16:0/4173030526 conn(0x7f0f5806ae30 msgr2=0x7f0f5809a400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:27:07.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.624+0000 7f0f5e855700 1 -- 172.21.15.16:0/4173030526 shutdown_connections 2023-05-20T01:27:07.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.624+0000 7f0f5e855700 1 -- 172.21.15.16:0/4173030526 wait complete. 2023-05-20T01:27:07.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.626+0000 7f0f5e855700 1 Processor -- start 2023-05-20T01:27:07.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.626+0000 7f0f5e855700 1 -- start start 2023-05-20T01:27:07.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.626+0000 7f0f5e855700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0f580cdfc0 0x7f0f581f0270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:07.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.626+0000 7f0f5e855700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f580cf1c0 0x7f0f58157e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:07.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.626+0000 7f0f5e855700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0f5815ab80 0x7f0f580c7370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:07.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.626+0000 7f0f5e855700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f580c4be0 con 0x7f0f580cf1c0 2023-05-20T01:27:07.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.626+0000 7f0f5e855700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5815a470 con 0x7f0f580cdfc0 2023-05-20T01:27:07.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.626+0000 7f0f5e855700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f581f07b0 con 0x7f0f5815ab80 2023-05-20T01:27:07.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.627+0000 7f0f577fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f580cf1c0 0x7f0f58157e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:07.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.627+0000 7f0f577fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f580cf1c0 0x7f0f58157e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38492/0 (socket says 172.21.15.16:38492) 2023-05-20T01:27:07.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.627+0000 7f0f577fe700 1 -- 172.21.15.16:0/2325931833 learned_addr learned my addr 172.21.15.16:0/2325931833 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:27:07.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.627+0000 7f0f57fff700 1 --2- 172.21.15.16:0/2325931833 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0f580cdfc0 0x7f0f581f0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:07.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.627+0000 7f0f5cdf2700 1 --2- 172.21.15.16:0/2325931833 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0f5815ab80 0x7f0f580c7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:07.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.627+0000 7f0f577fe700 1 -- 172.21.15.16:0/2325931833 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0f5815ab80 msgr2=0x7f0f580c7370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:07.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.627+0000 7f0f577fe700 1 --2- 172.21.15.16:0/2325931833 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0f5815ab80 0x7f0f580c7370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:07.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.627+0000 7f0f577fe700 1 -- 172.21.15.16:0/2325931833 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0f580cdfc0 msgr2=0x7f0f581f0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:07.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.627+0000 7f0f577fe700 1 --2- 172.21.15.16:0/2325931833 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0f580cdfc0 0x7f0f581f0270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:07.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.627+0000 7f0f577fe700 1 -- 172.21.15.16:0/2325931833 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f48009550 con 0x7f0f580cf1c0 2023-05-20T01:27:07.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.627+0000 7f0f57fff700 1 --2- 172.21.15.16:0/2325931833 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0f580cdfc0 0x7f0f581f0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:27:07.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.627+0000 7f0f577fe700 1 --2- 172.21.15.16:0/2325931833 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f580cf1c0 0x7f0f58157e10 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f0f48014040 tx=0x7f0f48004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:07.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.627+0000 7f0f54ff9700 1 -- 172.21.15.16:0/2325931833 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f480036a0 con 0x7f0f580cf1c0 2023-05-20T01:27:07.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.627+0000 7f0f54ff9700 1 -- 172.21.15.16:0/2325931833 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f4801e070 con 0x7f0f580cf1c0 2023-05-20T01:27:07.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.627+0000 7f0f5e855700 1 -- 172.21.15.16:0/2325931833 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f580cbbf0 con 0x7f0f580cf1c0 2023-05-20T01:27:07.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.627+0000 7f0f54ff9700 1 -- 172.21.15.16:0/2325931833 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f48003a40 con 0x7f0f580cf1c0 2023-05-20T01:27:07.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.627+0000 7f0f5e855700 1 -- 172.21.15.16:0/2325931833 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f580cb910 con 0x7f0f580cf1c0 2023-05-20T01:27:07.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.628+0000 7f0f5e855700 1 -- 172.21.15.16:0/2325931833 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f580d0e30 con 0x7f0f580cf1c0 2023-05-20T01:27:07.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.629+0000 7f0f54ff9700 1 -- 172.21.15.16:0/2325931833 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0f48003ba0 con 0x7f0f580cf1c0 2023-05-20T01:27:07.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.629+0000 7f0f54ff9700 1 --2- 172.21.15.16:0/2325931833 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f0f3806ace0 0x7f0f3806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:07.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.629+0000 7f0f54ff9700 1 -- 172.21.15.16:0/2325931833 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6786+0+0 (secure 0 0 0) 0x7f0f48020070 con 0x7f0f580cf1c0 2023-05-20T01:27:07.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.629+0000 7f0f57fff700 1 --2- 172.21.15.16:0/2325931833 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f0f3806ace0 0x7f0f3806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:07.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.630+0000 7f0f57fff700 1 --2- 172.21.15.16:0/2325931833 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f0f3806ace0 0x7f0f3806d190 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f0f40009a00 tx=0x7f0f40008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:07.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.633+0000 7f0f54ff9700 1 -- 172.21.15.16:0/2325931833 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f0f48055ac0 con 0x7f0f580cf1c0 2023-05-20T01:27:07.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.831+0000 7f0f5e855700 1 -- 172.21.15.16:0/2325931833 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0f580747b0 con 0x7f0f580cf1c0 2023-05-20T01:27:07.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.832+0000 7f0f54ff9700 1 -- 172.21.15.16:0/2325931833 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v115) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0f480590e0 con 0x7f0f580cf1c0 2023-05-20T01:27:07.837 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2023-05-20T01:27:07.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.834+0000 7f0f5e855700 1 -- 172.21.15.16:0/2325931833 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f0f3806ace0 msgr2=0x7f0f3806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:07.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.834+0000 7f0f5e855700 1 --2- 172.21.15.16:0/2325931833 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f0f3806ace0 0x7f0f3806d190 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f0f40009a00 tx=0x7f0f40008040 comp rx=0 tx=0).stop 2023-05-20T01:27:07.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.834+0000 7f0f5e855700 1 -- 172.21.15.16:0/2325931833 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f580cf1c0 msgr2=0x7f0f58157e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:07.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.834+0000 7f0f5e855700 1 --2- 172.21.15.16:0/2325931833 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f580cf1c0 0x7f0f58157e10 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f0f48014040 tx=0x7f0f48004750 comp rx=0 tx=0).stop 2023-05-20T01:27:07.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.834+0000 7f0f5e855700 1 -- 172.21.15.16:0/2325931833 shutdown_connections 2023-05-20T01:27:07.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.834+0000 7f0f5e855700 1 --2- 172.21.15.16:0/2325931833 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f0f3806ace0 0x7f0f3806d190 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:07.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.834+0000 7f0f5e855700 1 --2- 172.21.15.16:0/2325931833 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0f5815ab80 0x7f0f580c7370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:07.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.834+0000 7f0f5e855700 1 --2- 172.21.15.16:0/2325931833 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0f580cdfc0 0x7f0f581f0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:07.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.834+0000 7f0f5e855700 1 --2- 172.21.15.16:0/2325931833 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f580cf1c0 0x7f0f58157e10 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:07.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.834+0000 7f0f5e855700 1 -- 172.21.15.16:0/2325931833 >> 172.21.15.16:0/2325931833 conn(0x7f0f5806ae30 msgr2=0x7f0f580b0d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:27:07.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.834+0000 7f0f5e855700 1 -- 172.21.15.16:0/2325931833 shutdown_connections 2023-05-20T01:27:07.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.834+0000 7f0f5e855700 1 -- 172.21.15.16:0/2325931833 wait complete. 2023-05-20T01:27:07.850 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_extra pg_num 2023-05-20T01:27:07.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.973+0000 7ffbb8d0d700 1 Processor -- start 2023-05-20T01:27:07.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.973+0000 7ffbb8d0d700 1 -- start start 2023-05-20T01:27:07.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.974+0000 7ffbb8d0d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ffbb4071880 0x7ffbb4071c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:07.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.974+0000 7ffbb8d0d700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbb40d0e10 con 0x7ffbb40cbfb0 2023-05-20T01:27:07.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.974+0000 7ffbb8d0d700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbb4072190 con 0x7ffbb40cd620 2023-05-20T01:27:07.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.974+0000 7ffbb8d0d700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbb40cd200 con 0x7ffbb4071880 2023-05-20T01:27:07.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.974+0000 7ffbb259c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ffbb4071880 0x7ffbb4071c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:07.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.974+0000 7ffbb2d9d700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ffbb40cbfb0 0x7ffbb40b2e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50028/0 (socket says 172.21.15.16:50028) 2023-05-20T01:27:07.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.974+0000 7ffbb2d9d700 1 -- 172.21.15.16:0/4115773876 learned_addr learned my addr 172.21.15.16:0/4115773876 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:27:07.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.974+0000 7ffbb259c700 1 -- 172.21.15.16:0/4115773876 >> v1:172.21.15.132:6789/0 conn(0x7ffbb40cd620 legacy=0x7ffbb40cb8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:07.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.974+0000 7ffbb259c700 1 -- 172.21.15.16:0/4115773876 >> v1:172.21.15.16:6789/0 conn(0x7ffbb40cbfb0 legacy=0x7ffbb40b2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:07.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.974+0000 7ffbb259c700 1 -- 172.21.15.16:0/4115773876 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbb40cc580 con 0x7ffbb4071880 2023-05-20T01:27:07.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.974+0000 7ffbb259c700 1 --2- 172.21.15.16:0/4115773876 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffbb4071880 0x7ffbb4071c50 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7ffba4009820 tx=0x7ffba4009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e32468c74a4b7ed server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:07.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.975+0000 7ffbb159a700 1 -- 172.21.15.16:0/4115773876 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba4005e30 con 0x7ffbb4071880 2023-05-20T01:27:07.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.975+0000 7ffbb159a700 1 -- 172.21.15.16:0/4115773876 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffba4006030 con 0x7ffbb4071880 2023-05-20T01:27:07.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.975+0000 7ffbb159a700 1 -- 172.21.15.16:0/4115773876 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba40063e0 con 0x7ffbb4071880 2023-05-20T01:27:07.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.975+0000 7ffbb8d0d700 1 -- 172.21.15.16:0/4115773876 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffbb4071880 msgr2=0x7ffbb4071c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:07.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.975+0000 7ffbb8d0d700 1 --2- 172.21.15.16:0/4115773876 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffbb4071880 0x7ffbb4071c50 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7ffba4009820 tx=0x7ffba4009b30 comp rx=0 tx=0).stop 2023-05-20T01:27:07.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.975+0000 7ffbb8d0d700 1 -- 172.21.15.16:0/4115773876 shutdown_connections 2023-05-20T01:27:07.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.975+0000 7ffbb8d0d700 1 --2- 172.21.15.16:0/4115773876 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffbb4071880 0x7ffbb4071c50 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:07.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.975+0000 7ffbb8d0d700 1 -- 172.21.15.16:0/4115773876 >> 172.21.15.16:0/4115773876 conn(0x7ffbb40cffd0 msgr2=0x7ffbb40d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:27:07.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.975+0000 7ffbb8d0d700 1 -- 172.21.15.16:0/4115773876 shutdown_connections 2023-05-20T01:27:07.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.975+0000 7ffbb8d0d700 1 -- 172.21.15.16:0/4115773876 wait complete. 2023-05-20T01:27:07.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.977+0000 7ffbb8d0d700 1 Processor -- start 2023-05-20T01:27:07.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.977+0000 7ffbb8d0d700 1 -- start start 2023-05-20T01:27:07.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.977+0000 7ffbb8d0d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffbb4071880 0x7ffbb4151cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:07.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.977+0000 7ffbb8d0d700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffbb40cbfb0 0x7ffbb40ac8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:07.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.977+0000 7ffbb8d0d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffbb40cd620 0x7ffbb40b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:07.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.977+0000 7ffbb8d0d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb4152210 con 0x7ffbb4071880 2023-05-20T01:27:07.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.977+0000 7ffbb8d0d700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb40c8190 con 0x7ffbb40cd620 2023-05-20T01:27:07.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.977+0000 7ffbb8d0d700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb40c8470 con 0x7ffbb40cbfb0 2023-05-20T01:27:07.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.978+0000 7ffbb259c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffbb4071880 0x7ffbb4151cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:07.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.978+0000 7ffbb259c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffbb4071880 0x7ffbb4151cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38502/0 (socket says 172.21.15.16:38502) 2023-05-20T01:27:07.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.978+0000 7ffbb259c700 1 -- 172.21.15.16:0/741359016 learned_addr learned my addr 172.21.15.16:0/741359016 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:27:07.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.978+0000 7ffbb2d9d700 1 --2- 172.21.15.16:0/741359016 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffbb40cd620 0x7ffbb40b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:07.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.978+0000 7ffbb1d9b700 1 --2- 172.21.15.16:0/741359016 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffbb40cbfb0 0x7ffbb40ac8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:07.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.978+0000 7ffbb259c700 1 -- 172.21.15.16:0/741359016 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffbb40cbfb0 msgr2=0x7ffbb40ac8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:07.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.978+0000 7ffbb259c700 1 --2- 172.21.15.16:0/741359016 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffbb40cbfb0 0x7ffbb40ac8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:07.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.978+0000 7ffbb259c700 1 -- 172.21.15.16:0/741359016 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffbb40cd620 msgr2=0x7ffbb40b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:07.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.978+0000 7ffbb259c700 1 --2- 172.21.15.16:0/741359016 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffbb40cd620 0x7ffbb40b05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:07.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.978+0000 7ffbb259c700 1 -- 172.21.15.16:0/741359016 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffba4009430 con 0x7ffbb4071880 2023-05-20T01:27:07.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.978+0000 7ffbb1d9b700 1 --2- 172.21.15.16:0/741359016 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffbb40cbfb0 0x7ffbb40ac8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:27:07.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.978+0000 7ffbb2d9d700 1 --2- 172.21.15.16:0/741359016 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffbb40cd620 0x7ffbb40b05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:27:07.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.978+0000 7ffbb259c700 1 --2- 172.21.15.16:0/741359016 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffbb4071880 0x7ffbb4151cd0 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7ffba4000c00 tx=0x7ffba4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:07.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.978+0000 7ffba2ffd700 1 -- 172.21.15.16:0/741359016 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba4004250 con 0x7ffbb4071880 2023-05-20T01:27:07.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.978+0000 7ffba2ffd700 1 -- 172.21.15.16:0/741359016 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffba40043d0 con 0x7ffbb4071880 2023-05-20T01:27:07.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.978+0000 7ffbb8d0d700 1 -- 172.21.15.16:0/741359016 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbb40ccb40 con 0x7ffbb4071880 2023-05-20T01:27:07.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.978+0000 7ffba2ffd700 1 -- 172.21.15.16:0/741359016 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba40036a0 con 0x7ffbb4071880 2023-05-20T01:27:07.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.978+0000 7ffbb8d0d700 1 -- 172.21.15.16:0/741359016 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbb40cc860 con 0x7ffbb4071880 2023-05-20T01:27:07.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.979+0000 7ffbb8d0d700 1 -- 172.21.15.16:0/741359016 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbb40cce20 con 0x7ffbb4071880 2023-05-20T01:27:07.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.980+0000 7ffba2ffd700 1 -- 172.21.15.16:0/741359016 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ffba4022070 con 0x7ffbb4071880 2023-05-20T01:27:07.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.980+0000 7ffba2ffd700 1 --2- 172.21.15.16:0/741359016 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ffb9406ace0 0x7ffb9406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:07.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.980+0000 7ffba2ffd700 1 -- 172.21.15.16:0/741359016 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6786+0+0 (secure 0 0 0) 0x7ffba408b0e0 con 0x7ffbb4071880 2023-05-20T01:27:07.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.980+0000 7ffbb1d9b700 1 --2- 172.21.15.16:0/741359016 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ffb9406ace0 0x7ffb9406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:07.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.980+0000 7ffbb1d9b700 1 --2- 172.21.15.16:0/741359016 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ffb9406ace0 0x7ffb9406d190 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7ffbb406ae50 tx=0x7ffba8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:07.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:07.984+0000 7ffba2ffd700 1 -- 172.21.15.16:0/741359016 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7ffba4055f40 con 0x7ffbb4071880 2023-05-20T01:27:08.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.186+0000 7ffbb8d0d700 1 -- 172.21.15.16:0/741359016 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"} v 0) v1 -- 0x7ffbb40747b0 con 0x7ffbb4071880 2023-05-20T01:27:08.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.186+0000 7ffba2ffd700 1 -- 172.21.15.16:0/741359016 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"}]=0 v115) v1 ==== 106+0+11 (secure 0 0 0) 0x7ffba4059560 con 0x7ffbb4071880 2023-05-20T01:27:08.191 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-05-20T01:27:08.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.188+0000 7ffbb8d0d700 1 -- 172.21.15.16:0/741359016 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ffb9406ace0 msgr2=0x7ffb9406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:08.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.188+0000 7ffbb8d0d700 1 --2- 172.21.15.16:0/741359016 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ffb9406ace0 0x7ffb9406d190 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7ffbb406ae50 tx=0x7ffba8009040 comp rx=0 tx=0).stop 2023-05-20T01:27:08.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.188+0000 7ffbb8d0d700 1 -- 172.21.15.16:0/741359016 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffbb4071880 msgr2=0x7ffbb4151cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:08.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.188+0000 7ffbb8d0d700 1 --2- 172.21.15.16:0/741359016 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffbb4071880 0x7ffbb4151cd0 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7ffba4000c00 tx=0x7ffba4004750 comp rx=0 tx=0).stop 2023-05-20T01:27:08.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.188+0000 7ffbb8d0d700 1 -- 172.21.15.16:0/741359016 shutdown_connections 2023-05-20T01:27:08.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.188+0000 7ffbb8d0d700 1 --2- 172.21.15.16:0/741359016 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffbb40cd620 0x7ffbb40b05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:08.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.188+0000 7ffbb8d0d700 1 --2- 172.21.15.16:0/741359016 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffbb4071880 0x7ffbb4151cd0 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:08.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.188+0000 7ffbb8d0d700 1 --2- 172.21.15.16:0/741359016 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ffb9406ace0 0x7ffb9406d190 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:08.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.188+0000 7ffbb8d0d700 1 --2- 172.21.15.16:0/741359016 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffbb40cbfb0 0x7ffbb40ac8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:08.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.188+0000 7ffbb8d0d700 1 -- 172.21.15.16:0/741359016 >> 172.21.15.16:0/741359016 conn(0x7ffbb40cffd0 msgr2=0x7ffbb4159fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:27:08.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.188+0000 7ffbb8d0d700 1 -- 172.21.15.16:0/741359016 shutdown_connections 2023-05-20T01:27:08.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.188+0000 7ffbb8d0d700 1 -- 172.21.15.16:0/741359016 wait complete. 2023-05-20T01:27:08.205 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-05-20T01:27:08.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.326+0000 7fe3a1d05700 1 Processor -- start 2023-05-20T01:27:08.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.327+0000 7fe3a1d05700 1 -- start start 2023-05-20T01:27:08.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.327+0000 7fe3a1d05700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fe39c06b620 0x7fe39c06ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:08.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.327+0000 7fe3a1d05700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe39c0cb6e0 0x7fe39c0cbb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:08.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.327+0000 7fe3a1d05700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe39c0ce3e0 con 0x7fe39c0d1000 2023-05-20T01:27:08.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.327+0000 7fe3a1d05700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe39c0cdab0 con 0x7fe39c06b620 2023-05-20T01:27:08.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.327+0000 7fe3a1d05700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe39c0cdd90 con 0x7fe39c0cb6e0 2023-05-20T01:27:08.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.327+0000 7fe39b7fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fe39c0d1000 0x7fe39c0d13d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50038/0 (socket says 172.21.15.16:50038) 2023-05-20T01:27:08.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.328+0000 7fe39b7fe700 1 -- 172.21.15.16:0/2748865277 learned_addr learned my addr 172.21.15.16:0/2748865277 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:27:08.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.328+0000 7fe39a7fc700 1 -- 172.21.15.16:0/2748865277 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4043811611 0 0) 0x7fe39c0ce3e0 con 0x7fe39c0d1000 2023-05-20T01:27:08.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.328+0000 7fe39a7fc700 1 -- 172.21.15.16:0/2748865277 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe3880035b0 con 0x7fe39c0d1000 2023-05-20T01:27:08.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.328+0000 7fe39affd700 1 --2- 172.21.15.16:0/2748865277 >> v2:172.21.15.132:3300/0 conn(0x7fe39c06b620 0x7fe39c06ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:08.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.328+0000 7fe39bfff700 1 --2- 172.21.15.16:0/2748865277 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe39c0cb6e0 0x7fe39c0cbb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:08.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.328+0000 7fe39a7fc700 1 -- 172.21.15.16:0/2748865277 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1450499974 0 0) 0x7fe3880035b0 con 0x7fe39c0d1000 2023-05-20T01:27:08.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.328+0000 7fe39a7fc700 1 -- 172.21.15.16:0/2748865277 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe39c0cb6e0 msgr2=0x7fe39c0cbb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:08.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.328+0000 7fe39a7fc700 1 --2- 172.21.15.16:0/2748865277 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe39c0cb6e0 0x7fe39c0cbb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:08.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.328+0000 7fe39a7fc700 1 -- 172.21.15.16:0/2748865277 >> v2:172.21.15.132:3300/0 conn(0x7fe39c06b620 msgr2=0x7fe39c06ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:08.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.328+0000 7fe39a7fc700 1 --2- 172.21.15.16:0/2748865277 >> v2:172.21.15.132:3300/0 conn(0x7fe39c06b620 0x7fe39c06ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:08.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.328+0000 7fe39a7fc700 1 -- 172.21.15.16:0/2748865277 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe39c0ce070 con 0x7fe39c0d1000 2023-05-20T01:27:08.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.328+0000 7fe39a7fc700 1 -- 172.21.15.16:0/2748865277 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7fe38c001670 con 0x7fe39c0d1000 2023-05-20T01:27:08.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.328+0000 7fe39affd700 1 --2- 172.21.15.16:0/2748865277 >> v2:172.21.15.132:3300/0 conn(0x7fe39c06b620 0x7fe39c06ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:27:08.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.328+0000 7fe39a7fc700 1 -- 172.21.15.16:0/2748865277 >> v1:172.21.15.16:6789/0 conn(0x7fe39c0d1000 legacy=0x7fe39c0d13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:08.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.328+0000 7fe39a7fc700 1 --2- 172.21.15.16:0/2748865277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe388003a70 0x7fe388003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:08.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.328+0000 7fe39a7fc700 1 --2- 172.21.15.16:0/2748865277 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe388004850 0x7fe388004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:08.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.328+0000 7fe39a7fc700 1 --2- 172.21.15.16:0/2748865277 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe39c0cb6e0 0x7fe388005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:08.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.328+0000 7fe39a7fc700 1 -- 172.21.15.16:0/2748865277 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe39c0cdab0 con 0x7fe388003a70 2023-05-20T01:27:08.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.328+0000 7fe39a7fc700 1 -- 172.21.15.16:0/2748865277 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe39c0cdd90 con 0x7fe388004850 2023-05-20T01:27:08.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.328+0000 7fe39a7fc700 1 -- 172.21.15.16:0/2748865277 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe38800bb40 con 0x7fe39c0cb6e0 2023-05-20T01:27:08.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.328+0000 7fe39b7fe700 1 --2- 172.21.15.16:0/2748865277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe388003a70 0x7fe388003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:08.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.328+0000 7fe39affd700 1 --2- 172.21.15.16:0/2748865277 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe388004850 0x7fe388004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:08.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.328+0000 7fe39bfff700 1 --2- 172.21.15.16:0/2748865277 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe39c0cb6e0 0x7fe388005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:08.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.329+0000 7fe39b7fe700 1 -- 172.21.15.16:0/2748865277 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe39c0cb6e0 msgr2=0x7fe388005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:08.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.329+0000 7fe39b7fe700 1 --2- 172.21.15.16:0/2748865277 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe39c0cb6e0 0x7fe388005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:08.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.329+0000 7fe39b7fe700 1 -- 172.21.15.16:0/2748865277 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe388004850 msgr2=0x7fe388004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:08.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.329+0000 7fe39b7fe700 1 --2- 172.21.15.16:0/2748865277 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe388004850 0x7fe388004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:08.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.329+0000 7fe39b7fe700 1 -- 172.21.15.16:0/2748865277 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3880035b0 con 0x7fe388003a70 2023-05-20T01:27:08.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.329+0000 7fe39bfff700 1 --2- 172.21.15.16:0/2748865277 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe39c0cb6e0 0x7fe388005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:27:08.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.329+0000 7fe39b7fe700 1 --2- 172.21.15.16:0/2748865277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe388003a70 0x7fe388003e80 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7fe38c00c320 tx=0x7fe388005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eebc2a7f4c28d85 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:08.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.329+0000 7fe39a7fc700 1 -- 172.21.15.16:0/2748865277 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe38c00ce60 con 0x7fe388003a70 2023-05-20T01:27:08.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.329+0000 7fe39a7fc700 1 -- 172.21.15.16:0/2748865277 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe38c005960 con 0x7fe388003a70 2023-05-20T01:27:08.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.329+0000 7fe39a7fc700 1 -- 172.21.15.16:0/2748865277 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe38c005d10 con 0x7fe388003a70 2023-05-20T01:27:08.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.329+0000 7fe3a1d05700 1 -- 172.21.15.16:0/2748865277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe388003a70 msgr2=0x7fe388003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:08.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.329+0000 7fe3a1d05700 1 --2- 172.21.15.16:0/2748865277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe388003a70 0x7fe388003e80 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7fe38c00c320 tx=0x7fe388005880 comp rx=0 tx=0).stop 2023-05-20T01:27:08.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.329+0000 7fe39bfff700 1 -- 172.21.15.16:0/2748865277 reap_dead start 2023-05-20T01:27:08.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.329+0000 7fe3a1d05700 1 -- 172.21.15.16:0/2748865277 shutdown_connections 2023-05-20T01:27:08.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.329+0000 7fe3a1d05700 1 -- 172.21.15.16:0/2748865277 >> 172.21.15.16:0/2748865277 conn(0x7fe39c06ae30 msgr2=0x7fe39c09a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:27:08.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.330+0000 7fe3a1d05700 1 -- 172.21.15.16:0/2748865277 shutdown_connections 2023-05-20T01:27:08.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.330+0000 7fe3a1d05700 1 -- 172.21.15.16:0/2748865277 wait complete. 2023-05-20T01:27:08.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.331+0000 7fe3a1d05700 1 Processor -- start 2023-05-20T01:27:08.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.331+0000 7fe3a1d05700 1 -- start start 2023-05-20T01:27:08.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.332+0000 7fe3a1d05700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe388003a70 0x7fe39c0aec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:08.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.332+0000 7fe3a1d05700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe39c0b6ab0 0x7fe39c15b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:08.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.332+0000 7fe3a1d05700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe39c0b4620 0x7fe39c0b4a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:08.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.332+0000 7fe3a1d05700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe39c0b4fd0 con 0x7fe388003a70 2023-05-20T01:27:08.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.332+0000 7fe3a1d05700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe39c0af310 con 0x7fe39c0b6ab0 2023-05-20T01:27:08.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.332+0000 7fe3a1d05700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe39c0af5f0 con 0x7fe39c0b4620 2023-05-20T01:27:08.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.332+0000 7fe39b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe388003a70 0x7fe39c0aec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:08.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.332+0000 7fe39b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe388003a70 0x7fe39c0aec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38522/0 (socket says 172.21.15.16:38522) 2023-05-20T01:27:08.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.332+0000 7fe39b7fe700 1 -- 172.21.15.16:0/4218182797 learned_addr learned my addr 172.21.15.16:0/4218182797 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:27:08.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.332+0000 7fe39b7fe700 1 -- 172.21.15.16:0/4218182797 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe39c0b4620 msgr2=0x7fe39c0b4a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:08.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.332+0000 7fe39b7fe700 1 --2- 172.21.15.16:0/4218182797 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe39c0b4620 0x7fe39c0b4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:08.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.332+0000 7fe39b7fe700 1 -- 172.21.15.16:0/4218182797 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe39c0b6ab0 msgr2=0x7fe39c15b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:08.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.332+0000 7fe39b7fe700 1 --2- 172.21.15.16:0/4218182797 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe39c0b6ab0 0x7fe39c15b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:08.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.332+0000 7fe39b7fe700 1 -- 172.21.15.16:0/4218182797 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe39c0ce870 con 0x7fe388003a70 2023-05-20T01:27:08.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.332+0000 7fe39b7fe700 1 --2- 172.21.15.16:0/4218182797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe388003a70 0x7fe39c0aec10 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7fe38c008fd0 tx=0x7fe38c001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:08.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.332+0000 7fe393fff700 1 -- 172.21.15.16:0/4218182797 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe38c004a80 con 0x7fe388003a70 2023-05-20T01:27:08.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.332+0000 7fe393fff700 1 -- 172.21.15.16:0/4218182797 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe38c004be0 con 0x7fe388003a70 2023-05-20T01:27:08.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.332+0000 7fe393fff700 1 -- 172.21.15.16:0/4218182797 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe38c004d90 con 0x7fe388003a70 2023-05-20T01:27:08.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.332+0000 7fe3a1d05700 1 -- 172.21.15.16:0/4218182797 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe39c0b28b0 con 0x7fe388003a70 2023-05-20T01:27:08.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.332+0000 7fe3a1d05700 1 -- 172.21.15.16:0/4218182797 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe39c0cd390 con 0x7fe388003a70 2023-05-20T01:27:08.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.333+0000 7fe3a1d05700 1 -- 172.21.15.16:0/4218182797 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe39c0747b0 con 0x7fe388003a70 2023-05-20T01:27:08.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.334+0000 7fe393fff700 1 -- 172.21.15.16:0/4218182797 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe38c0070d0 con 0x7fe388003a70 2023-05-20T01:27:08.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.334+0000 7fe393fff700 1 --2- 172.21.15.16:0/4218182797 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe37c06ad10 0x7fe37c06d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:08.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.335+0000 7fe393fff700 1 -- 172.21.15.16:0/4218182797 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6786+0+0 (secure 0 0 0) 0x7fe38c08ac40 con 0x7fe388003a70 2023-05-20T01:27:08.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.335+0000 7fe39affd700 1 --2- 172.21.15.16:0/4218182797 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe37c06ad10 0x7fe37c06d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:08.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.335+0000 7fe39affd700 1 --2- 172.21.15.16:0/4218182797 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe37c06ad10 0x7fe37c06d1c0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fe384006b20 tx=0x7fe384006ab0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:08.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.339+0000 7fe393fff700 1 -- 172.21.15.16:0/4218182797 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fe38c055ac0 con 0x7fe388003a70 2023-05-20T01:27:08.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.557+0000 7fe3a1d05700 1 -- 172.21.15.16:0/4218182797 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fe39c0b1640 con 0x7fe388003a70 2023-05-20T01:27:08.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.558+0000 7fe393fff700 1 -- 172.21.15.16:0/4218182797 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v115) v1 ==== 80+0+0 (secure 0 0 0) 0x7fe38c007da0 con 0x7fe388003a70 2023-05-20T01:27:08.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.560+0000 7fe3a1d05700 1 -- 172.21.15.16:0/4218182797 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe37c06ad10 msgr2=0x7fe37c06d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:08.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.560+0000 7fe3a1d05700 1 --2- 172.21.15.16:0/4218182797 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe37c06ad10 0x7fe37c06d1c0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fe384006b20 tx=0x7fe384006ab0 comp rx=0 tx=0).stop 2023-05-20T01:27:08.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.560+0000 7fe3a1d05700 1 -- 172.21.15.16:0/4218182797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe388003a70 msgr2=0x7fe39c0aec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:08.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.560+0000 7fe3a1d05700 1 --2- 172.21.15.16:0/4218182797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe388003a70 0x7fe39c0aec10 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7fe38c008fd0 tx=0x7fe38c001a00 comp rx=0 tx=0).stop 2023-05-20T01:27:08.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.560+0000 7fe3a1d05700 1 -- 172.21.15.16:0/4218182797 shutdown_connections 2023-05-20T01:27:08.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.560+0000 7fe3a1d05700 1 --2- 172.21.15.16:0/4218182797 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe37c06ad10 0x7fe37c06d1c0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:08.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.560+0000 7fe3a1d05700 1 --2- 172.21.15.16:0/4218182797 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe39c0b4620 0x7fe39c0b4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:08.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.560+0000 7fe3a1d05700 1 --2- 172.21.15.16:0/4218182797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe388003a70 0x7fe39c0aec10 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:08.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.560+0000 7fe3a1d05700 1 --2- 172.21.15.16:0/4218182797 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe39c0b6ab0 0x7fe39c15b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:08.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.560+0000 7fe3a1d05700 1 -- 172.21.15.16:0/4218182797 >> 172.21.15.16:0/4218182797 conn(0x7fe39c06ae30 msgr2=0x7fe39c0d13d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:27:08.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.560+0000 7fe3a1d05700 1 -- 172.21.15.16:0/4218182797 shutdown_connections 2023-05-20T01:27:08.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.560+0000 7fe3a1d05700 1 -- 172.21.15.16:0/4218182797 wait complete. 2023-05-20T01:27:08.568 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2023-05-20T01:27:08.576 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-05-20T01:27:08.577 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:27:08.578 DEBUG:teuthology.orchestra.run.smithi016:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-05-20T01:27:08.661 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2023-05-20T01:27:08.662 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:27:08.662 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-05-20T01:27:08.701 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-05-20T01:27:08.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.863+0000 7ff7ff4db700 1 Processor -- start 2023-05-20T01:27:08.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.864+0000 7ff7ff4db700 1 -- start start 2023-05-20T01:27:08.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.864+0000 7ff7ff4db700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7ff7f80cad50 0x7ff7f8071140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:08.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.864+0000 7ff7ff4db700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff7f8071680 0x7ff7f8071aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:08.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.864+0000 7ff7ff4db700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7f80ce920 con 0x7ff7f80ca980 2023-05-20T01:27:08.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.864+0000 7ff7ff4db700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7f80af8f0 con 0x7ff7f80cad50 2023-05-20T01:27:08.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.864+0000 7ff7ff4db700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f80725d0 con 0x7ff7f8071680 2023-05-20T01:27:08.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.865+0000 7ff7fda78700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff7f8071680 0x7ff7f8071aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:08.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.865+0000 7ff7fca76700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7ff7f80cad50 0x7ff7f8071140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:08.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.865+0000 7ff7fda78700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff7f8071680 0x7ff7f8071aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.16:38574/0 (socket says 172.21.15.16:38574) 2023-05-20T01:27:08.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.865+0000 7ff7fda78700 1 -- 172.21.15.16:0/2173223644 learned_addr learned my addr 172.21.15.16:0/2173223644 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:27:08.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.865+0000 7ff7fd277700 1 --1- 172.21.15.16:0/2173223644 >> v1:172.21.15.132:6789/0 conn(0x7ff7f80ca980 0x7ff7f80aef10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.16:59774/0 (socket says 172.21.15.16:59774) 2023-05-20T01:27:08.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.865+0000 7ff7fca76700 1 -- 172.21.15.16:0/2173223644 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff7f8071680 msgr2=0x7ff7f8071aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:08.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.865+0000 7ff7fca76700 1 --2- 172.21.15.16:0/2173223644 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff7f8071680 0x7ff7f8071aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:08.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.865+0000 7ff7fca76700 1 -- 172.21.15.16:0/2173223644 >> v1:172.21.15.132:6789/0 conn(0x7ff7f80ca980 legacy=0x7ff7f80aef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:08.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.865+0000 7ff7fca76700 1 -- 172.21.15.16:0/2173223644 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7f80af620 con 0x7ff7f80cad50 2023-05-20T01:27:08.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.865+0000 7ff7ebfff700 1 -- 172.21.15.16:0/2173223644 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1299605285 0 0) 0x7ff7f80ce920 con 0x7ff7f80ca980 2023-05-20T01:27:08.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.865+0000 7ff7fda78700 1 --2- 172.21.15.16:0/2173223644 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff7f8071680 0x7ff7f8071aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-05-20T01:27:08.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.865+0000 7ff7fca76700 1 --2- 172.21.15.16:0/2173223644 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff7f80cad50 0x7ff7f8071140 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec009840 tx=0x7ff7ec009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb73e9487fe455ed server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:08.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.865+0000 7ff7ebfff700 1 -- 172.21.15.16:0/2173223644 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7ec011830 con 0x7ff7f80cad50 2023-05-20T01:27:08.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.865+0000 7ff7ebfff700 1 -- 172.21.15.16:0/2173223644 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7ec011990 con 0x7ff7f80cad50 2023-05-20T01:27:08.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.865+0000 7ff7ebfff700 1 -- 172.21.15.16:0/2173223644 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7ec011d20 con 0x7ff7f80cad50 2023-05-20T01:27:08.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.866+0000 7ff7ff4db700 1 -- 172.21.15.16:0/2173223644 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff7f80cad50 msgr2=0x7ff7f8071140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:08.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.866+0000 7ff7ff4db700 1 --2- 172.21.15.16:0/2173223644 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff7f80cad50 0x7ff7f8071140 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec009840 tx=0x7ff7ec009b50 comp rx=0 tx=0).stop 2023-05-20T01:27:08.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.866+0000 7ff7ff4db700 1 -- 172.21.15.16:0/2173223644 shutdown_connections 2023-05-20T01:27:08.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.866+0000 7ff7ff4db700 1 --2- 172.21.15.16:0/2173223644 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff7f8071680 0x7ff7f8071aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:08.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.866+0000 7ff7ff4db700 1 --2- 172.21.15.16:0/2173223644 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff7f80cad50 0x7ff7f8071140 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:08.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.866+0000 7ff7ff4db700 1 -- 172.21.15.16:0/2173223644 >> 172.21.15.16:0/2173223644 conn(0x7ff7f80d0c70 msgr2=0x7ff7f80b1af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:27:08.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.866+0000 7ff7ff4db700 1 -- 172.21.15.16:0/2173223644 shutdown_connections 2023-05-20T01:27:08.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.866+0000 7ff7ff4db700 1 -- 172.21.15.16:0/2173223644 wait complete. 2023-05-20T01:27:08.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.867+0000 7ff7ff4db700 1 Processor -- start 2023-05-20T01:27:08.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.867+0000 7ff7ff4db700 1 -- start start 2023-05-20T01:27:08.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.868+0000 7ff7ff4db700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7f8071680 0x7ff7f806e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:08.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.868+0000 7ff7ff4db700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff7f80ca980 0x7ff7f81e3360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:08.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.868+0000 7ff7ff4db700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff7f80cad50 0x7ff7f81e5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:08.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.868+0000 7ff7ff4db700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f80c80f0 con 0x7ff7f8071680 2023-05-20T01:27:08.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.868+0000 7ff7ff4db700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f80c8280 con 0x7ff7f80ca980 2023-05-20T01:27:08.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.868+0000 7ff7ff4db700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f81e3e20 con 0x7ff7f80cad50 2023-05-20T01:27:08.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.868+0000 7ff7fd277700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7f8071680 0x7ff7f806e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:08.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.868+0000 7ff7fd277700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7f8071680 0x7ff7f806e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38538/0 (socket says 172.21.15.16:38538) 2023-05-20T01:27:08.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.868+0000 7ff7fd277700 1 -- 172.21.15.16:0/365997727 learned_addr learned my addr 172.21.15.16:0/365997727 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:27:08.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.868+0000 7ff7fca76700 1 --2- 172.21.15.16:0/365997727 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff7f80ca980 0x7ff7f81e3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:08.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.868+0000 7ff7fda78700 1 --2- 172.21.15.16:0/365997727 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff7f80cad50 0x7ff7f81e5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:08.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.869+0000 7ff7fd277700 1 -- 172.21.15.16:0/365997727 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff7f80cad50 msgr2=0x7ff7f81e5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:08.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.869+0000 7ff7fd277700 1 --2- 172.21.15.16:0/365997727 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff7f80cad50 0x7ff7f81e5060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:08.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.869+0000 7ff7fd277700 1 -- 172.21.15.16:0/365997727 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff7f80ca980 msgr2=0x7ff7f81e3360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:08.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.869+0000 7ff7fd277700 1 --2- 172.21.15.16:0/365997727 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff7f80ca980 0x7ff7f81e3360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:08.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.869+0000 7ff7fd277700 1 -- 172.21.15.16:0/365997727 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7f0009b30 con 0x7ff7f8071680 2023-05-20T01:27:08.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.869+0000 7ff7fda78700 1 --2- 172.21.15.16:0/365997727 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff7f80cad50 0x7ff7f81e5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:27:08.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.869+0000 7ff7fca76700 1 --2- 172.21.15.16:0/365997727 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff7f80ca980 0x7ff7f81e3360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:27:08.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.869+0000 7ff7fd277700 1 --2- 172.21.15.16:0/365997727 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7f8071680 0x7ff7f806e310 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7ff7f40011e0 tx=0x7ff7f400cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:08.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.869+0000 7ff7e9ffb700 1 -- 172.21.15.16:0/365997727 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f4004c80 con 0x7ff7f8071680 2023-05-20T01:27:08.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.869+0000 7ff7e9ffb700 1 -- 172.21.15.16:0/365997727 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7f4004e00 con 0x7ff7f8071680 2023-05-20T01:27:08.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.869+0000 7ff7ff4db700 1 -- 172.21.15.16:0/365997727 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7ec0094a0 con 0x7ff7f8071680 2023-05-20T01:27:08.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.869+0000 7ff7e9ffb700 1 -- 172.21.15.16:0/365997727 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f4007310 con 0x7ff7f8071680 2023-05-20T01:27:08.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.869+0000 7ff7ff4db700 1 -- 172.21.15.16:0/365997727 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7f80ce5b0 con 0x7ff7f8071680 2023-05-20T01:27:08.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.870+0000 7ff7ff4db700 1 -- 172.21.15.16:0/365997727 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7f80722f0 con 0x7ff7f8071680 2023-05-20T01:27:08.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.870+0000 7ff7e9ffb700 1 -- 172.21.15.16:0/365997727 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff7f4007470 con 0x7ff7f8071680 2023-05-20T01:27:08.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.871+0000 7ff7e9ffb700 1 --2- 172.21.15.16:0/365997727 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff7dc06ace0 0x7ff7dc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:08.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.871+0000 7ff7e9ffb700 1 -- 172.21.15.16:0/365997727 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6786+0+0 (secure 0 0 0) 0x7ff7f4089610 con 0x7ff7f8071680 2023-05-20T01:27:08.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.871+0000 7ff7fca76700 1 --2- 172.21.15.16:0/365997727 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff7dc06ace0 0x7ff7dc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:08.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.871+0000 7ff7fca76700 1 --2- 172.21.15.16:0/365997727 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff7dc06ace0 0x7ff7dc06d190 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7ff7f806ebf0 tx=0x7ff7ec008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:08.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:08.875+0000 7ff7e9ffb700 1 -- 172.21.15.16:0/365997727 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7ff7f4054360 con 0x7ff7f8071680 2023-05-20T01:27:09.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.118+0000 7ff7ff4db700 1 -- 172.21.15.16:0/365997727 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7ff7f8071fe0 con 0x7ff7f8071680 2023-05-20T01:27:09.531 INFO:tasks.ceph.mon.a.smithi016.stderr:2023-05-20T01:27:09.527+0000 7f49eb8ae700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-05-20T01:27:09.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.534+0000 7ff7e9ffb700 1 -- 172.21.15.16:0/365997727 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v80) v1 ==== 161+0+0 (secure 0 0 0) 0x7ff7f4057980 con 0x7ff7f8071680 2023-05-20T01:27:09.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.536+0000 7ff7ff4db700 1 -- 172.21.15.16:0/365997727 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff7dc06ace0 msgr2=0x7ff7dc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:09.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.536+0000 7ff7ff4db700 1 --2- 172.21.15.16:0/365997727 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff7dc06ace0 0x7ff7dc06d190 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7ff7f806ebf0 tx=0x7ff7ec008030 comp rx=0 tx=0).stop 2023-05-20T01:27:09.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.536+0000 7ff7ff4db700 1 -- 172.21.15.16:0/365997727 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7f8071680 msgr2=0x7ff7f806e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:09.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.536+0000 7ff7ff4db700 1 --2- 172.21.15.16:0/365997727 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7f8071680 0x7ff7f806e310 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7ff7f40011e0 tx=0x7ff7f400cae0 comp rx=0 tx=0).stop 2023-05-20T01:27:09.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.537+0000 7ff7ff4db700 1 -- 172.21.15.16:0/365997727 shutdown_connections 2023-05-20T01:27:09.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.537+0000 7ff7ff4db700 1 --2- 172.21.15.16:0/365997727 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff7dc06ace0 0x7ff7dc06d190 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:09.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.537+0000 7ff7ff4db700 1 --2- 172.21.15.16:0/365997727 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff7f80cad50 0x7ff7f81e5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:09.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.537+0000 7ff7ff4db700 1 --2- 172.21.15.16:0/365997727 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7f8071680 0x7ff7f806e310 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:09.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.537+0000 7ff7ff4db700 1 --2- 172.21.15.16:0/365997727 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff7f80ca980 0x7ff7f81e3360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:09.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.537+0000 7ff7ff4db700 1 -- 172.21.15.16:0/365997727 >> 172.21.15.16:0/365997727 conn(0x7ff7f80d0c70 msgr2=0x7ff7f80b4e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:27:09.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.537+0000 7ff7ff4db700 1 -- 172.21.15.16:0/365997727 shutdown_connections 2023-05-20T01:27:09.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.537+0000 7ff7ff4db700 1 -- 172.21.15.16:0/365997727 wait complete. 2023-05-20T01:27:09.546 INFO:teuthology.orchestra.run.smithi016.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-05-20T01:27:09.556 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-05-20T01:27:09.559 DEBUG:teuthology.orchestra.run.smithi132:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-05-20T01:27:09.578 INFO:tasks.ceph.mds.a.smithi016.stdout:starting mds.a at 2023-05-20T01:27:09.606 INFO:teuthology.orchestra.run.smithi016.stdout:[global] 2023-05-20T01:27:09.607 INFO:teuthology.orchestra.run.smithi016.stdout: chdir = "" 2023-05-20T01:27:09.607 INFO:teuthology.orchestra.run.smithi016.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-05-20T01:27:09.608 INFO:teuthology.orchestra.run.smithi016.stdout: auth supported = cephx 2023-05-20T01:27:09.608 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:27:09.608 INFO:teuthology.orchestra.run.smithi016.stdout: filestore xattr use omap = true 2023-05-20T01:27:09.609 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:27:09.609 INFO:teuthology.orchestra.run.smithi016.stdout: mon clock drift allowed = 1.000 2023-05-20T01:27:09.609 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:27:09.609 INFO:teuthology.orchestra.run.smithi016.stdout: osd crush chooseleaf type = 0 2023-05-20T01:27:09.609 INFO:teuthology.orchestra.run.smithi016.stdout: auth debug = true 2023-05-20T01:27:09.609 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:27:09.609 INFO:teuthology.orchestra.run.smithi016.stdout: ms die on old message = true 2023-05-20T01:27:09.610 INFO:teuthology.orchestra.run.smithi016.stdout: ms die on bug = true 2023-05-20T01:27:09.610 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:27:09.610 INFO:teuthology.orchestra.run.smithi016.stdout: mon max pg per osd = 10000 # >= luminous 2023-05-20T01:27:09.610 INFO:teuthology.orchestra.run.smithi016.stdout: mon pg warn max object skew = 0 2023-05-20T01:27:09.610 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:27:09.610 INFO:teuthology.orchestra.run.smithi016.stdout: # disable pg_autoscaler by default for new pools 2023-05-20T01:27:09.610 INFO:teuthology.orchestra.run.smithi016.stdout: osd_pool_default_pg_autoscale_mode = off 2023-05-20T01:27:09.610 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:27:09.611 INFO:teuthology.orchestra.run.smithi016.stdout: osd pool default size = 2 2023-05-20T01:27:09.611 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:27:09.611 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd allow primary affinity = true 2023-05-20T01:27:09.611 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd allow pg remap = true 2023-05-20T01:27:09.611 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on legacy crush tunables = false 2023-05-20T01:27:09.611 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on crush straw calc version zero = false 2023-05-20T01:27:09.611 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on no sortbitwise = false 2023-05-20T01:27:09.612 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on osd down out interval zero = false 2023-05-20T01:27:09.612 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on too few osds = false 2023-05-20T01:27:09.612 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-05-20T01:27:09.612 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_pool_no_redundancy = false 2023-05-20T01:27:09.612 INFO:teuthology.orchestra.run.smithi016.stdout: mon_allow_pool_size_one = true 2023-05-20T01:27:09.612 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:27:09.612 INFO:teuthology.orchestra.run.smithi016.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-05-20T01:27:09.612 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:27:09.613 INFO:teuthology.orchestra.run.smithi016.stdout: osd default data pool replay window = 5 2023-05-20T01:27:09.613 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:27:09.613 INFO:teuthology.orchestra.run.smithi016.stdout: mon allow pool delete = true 2023-05-20T01:27:09.613 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:27:09.613 INFO:teuthology.orchestra.run.smithi016.stdout: mon cluster log file level = debug 2023-05-20T01:27:09.613 INFO:teuthology.orchestra.run.smithi016.stdout: debug asserts on shutdown = true 2023-05-20T01:27:09.613 INFO:teuthology.orchestra.run.smithi016.stdout: mon health detail to clog = false 2023-05-20T01:27:09.614 INFO:teuthology.orchestra.run.smithi016.stdout: mon host = "172.21.15.16,172.21.15.132,[v2:172.21.15.132:3301,v1:172.21.15.132:6790]" 2023-05-20T01:27:09.614 INFO:teuthology.orchestra.run.smithi016.stdout: ms die on skipped message = False 2023-05-20T01:27:09.614 INFO:teuthology.orchestra.run.smithi016.stdout: fsid = c2245cc5-ecab-4087-a59b-bc81db901676 2023-05-20T01:27:09.614 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:27:09.614 INFO:teuthology.orchestra.run.smithi016.stdout:[osd] 2023-05-20T01:27:09.614 INFO:teuthology.orchestra.run.smithi016.stdout: osd journal size = 100 2023-05-20T01:27:09.614 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:27:09.614 INFO:teuthology.orchestra.run.smithi016.stdout: osd scrub load threshold = 5.0 2023-05-20T01:27:09.615 INFO:teuthology.orchestra.run.smithi016.stdout: osd scrub max interval = 600 2023-05-20T01:27:09.615 INFO:teuthology.orchestra.run.smithi016.stdout: osd mclock profile = high_recovery_ops 2023-05-20T01:27:09.615 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:27:09.615 INFO:teuthology.orchestra.run.smithi016.stdout: osd recover clone overlap = true 2023-05-20T01:27:09.615 INFO:teuthology.orchestra.run.smithi016.stdout: osd recovery max chunk = 1048576 2023-05-20T01:27:09.615 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:27:09.615 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug shutdown = true 2023-05-20T01:27:09.615 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug op order = true 2023-05-20T01:27:09.616 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug verify stray on activate = true 2023-05-20T01:27:09.616 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:27:09.616 INFO:teuthology.orchestra.run.smithi016.stdout: osd open classes on start = true 2023-05-20T01:27:09.616 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug pg log writeout = true 2023-05-20T01:27:09.616 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:27:09.616 INFO:teuthology.orchestra.run.smithi016.stdout: osd deep scrub update digest min age = 30 2023-05-20T01:27:09.616 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:27:09.617 INFO:teuthology.orchestra.run.smithi016.stdout: osd map max advance = 10 2023-05-20T01:27:09.617 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:27:09.617 INFO:teuthology.orchestra.run.smithi016.stdout: journal zero on create = true 2023-05-20T01:27:09.617 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:27:09.617 INFO:teuthology.orchestra.run.smithi016.stdout: filestore ondisk finisher threads = 3 2023-05-20T01:27:09.617 INFO:teuthology.orchestra.run.smithi016.stdout: filestore apply finisher threads = 3 2023-05-20T01:27:09.617 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:27:09.617 INFO:teuthology.orchestra.run.smithi016.stdout: bdev debug aio = true 2023-05-20T01:27:09.618 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug misdirected ops = true 2023-05-20T01:27:09.618 INFO:teuthology.orchestra.run.smithi016.stdout: bluestore block size = 96636764160 2023-05-20T01:27:09.618 INFO:teuthology.orchestra.run.smithi016.stdout: bluestore fsck on mount = True 2023-05-20T01:27:09.618 INFO:teuthology.orchestra.run.smithi016.stdout: debug bluefs = 20 2023-05-20T01:27:09.618 INFO:teuthology.orchestra.run.smithi016.stdout: debug bluestore = 20 2023-05-20T01:27:09.618 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2023-05-20T01:27:09.618 INFO:teuthology.orchestra.run.smithi016.stdout: debug osd = 20 2023-05-20T01:27:09.618 INFO:teuthology.orchestra.run.smithi016.stdout: debug rocksdb = 10 2023-05-20T01:27:09.619 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd backfillfull_ratio = 0.85 2023-05-20T01:27:09.619 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd full ratio = 0.9 2023-05-20T01:27:09.619 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd nearfull ratio = 0.8 2023-05-20T01:27:09.619 INFO:teuthology.orchestra.run.smithi016.stdout: osd failsafe full ratio = 0.95 2023-05-20T01:27:09.619 INFO:teuthology.orchestra.run.smithi016.stdout: osd objectstore = bluestore 2023-05-20T01:27:09.619 INFO:teuthology.orchestra.run.smithi016.stdout: osd op complaint time = 180 2023-05-20T01:27:09.619 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:27:09.619 INFO:teuthology.orchestra.run.smithi016.stdout:[mgr] 2023-05-20T01:27:09.620 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2023-05-20T01:27:09.620 INFO:teuthology.orchestra.run.smithi016.stdout: debug mgr = 20 2023-05-20T01:27:09.620 INFO:teuthology.orchestra.run.smithi016.stdout: debug mon = 20 2023-05-20T01:27:09.620 INFO:teuthology.orchestra.run.smithi016.stdout: debug auth = 20 2023-05-20T01:27:09.620 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min pgs per osd = 4 2023-05-20T01:27:09.620 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min bytes per osd = 10 2023-05-20T01:27:09.620 INFO:teuthology.orchestra.run.smithi016.stdout: mgr/telemetry/nag = false 2023-05-20T01:27:09.621 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:27:09.621 INFO:teuthology.orchestra.run.smithi016.stdout:[mon] 2023-05-20T01:27:09.621 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2023-05-20T01:27:09.621 INFO:teuthology.orchestra.run.smithi016.stdout: debug mon = 20 2023-05-20T01:27:09.621 INFO:teuthology.orchestra.run.smithi016.stdout: debug paxos = 20 2023-05-20T01:27:09.621 INFO:teuthology.orchestra.run.smithi016.stdout: debug auth = 20 2023-05-20T01:27:09.621 INFO:teuthology.orchestra.run.smithi016.stdout: mon data avail warn = 5 2023-05-20T01:27:09.622 INFO:teuthology.orchestra.run.smithi016.stdout: mon mgr mkfs grace = 240 2023-05-20T01:27:09.622 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min pgs per osd = 4 2023-05-20T01:27:09.622 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd reporter subtree level = osd 2023-05-20T01:27:09.622 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd prime pg temp = true 2023-05-20T01:27:09.622 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min bytes per osd = 10 2023-05-20T01:27:09.622 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:27:09.622 INFO:teuthology.orchestra.run.smithi016.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-05-20T01:27:09.622 INFO:teuthology.orchestra.run.smithi016.stdout: auth mon ticket ttl = 660 # 11m 2023-05-20T01:27:09.623 INFO:teuthology.orchestra.run.smithi016.stdout: auth service ticket ttl = 240 # 4m 2023-05-20T01:27:09.623 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:27:09.623 INFO:teuthology.orchestra.run.smithi016.stdout: # don't complain about insecure global_id in the test suite 2023-05-20T01:27:09.623 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-05-20T01:27:09.623 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-05-20T01:27:09.623 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:27:09.623 INFO:teuthology.orchestra.run.smithi016.stdout: # 1m isn't quite enough 2023-05-20T01:27:09.623 INFO:teuthology.orchestra.run.smithi016.stdout: mon_down_mkfs_grace = 2m 2023-05-20T01:27:09.624 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:27:09.624 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_filestore_osds = false 2023-05-20T01:27:09.624 INFO:teuthology.orchestra.run.smithi016.stdout: mon op complaint time = 120 2023-05-20T01:27:09.624 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:27:09.624 INFO:teuthology.orchestra.run.smithi016.stdout:[client] 2023-05-20T01:27:09.624 INFO:teuthology.orchestra.run.smithi016.stdout: rgw cache enabled = true 2023-05-20T01:27:09.624 INFO:teuthology.orchestra.run.smithi016.stdout: rgw enable ops log = true 2023-05-20T01:27:09.624 INFO:teuthology.orchestra.run.smithi016.stdout: rgw enable usage log = true 2023-05-20T01:27:09.625 INFO:teuthology.orchestra.run.smithi016.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-05-20T01:27:09.625 INFO:teuthology.orchestra.run.smithi016.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-05-20T01:27:09.625 INFO:teuthology.orchestra.run.smithi016.stdout: client mount timeout = 600 2023-05-20T01:27:09.625 INFO:teuthology.orchestra.run.smithi016.stdout: debug client = 20 2023-05-20T01:27:09.625 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2023-05-20T01:27:09.625 INFO:teuthology.orchestra.run.smithi016.stdout: pid file = "" 2023-05-20T01:27:09.625 INFO:teuthology.orchestra.run.smithi016.stdout: rados mon op timeout = 900 2023-05-20T01:27:09.626 INFO:teuthology.orchestra.run.smithi016.stdout: rados osd op timeout = 900 2023-05-20T01:27:09.626 INFO:teuthology.orchestra.run.smithi016.stdout:[mon.a] 2023-05-20T01:27:09.626 INFO:teuthology.orchestra.run.smithi016.stdout:[mon.b] 2023-05-20T01:27:09.626 INFO:teuthology.orchestra.run.smithi016.stdout:[mon.c] 2023-05-20T01:27:09.626 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.a] 2023-05-20T01:27:09.626 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.c] 2023-05-20T01:27:09.626 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.b] 2023-05-20T01:27:09.626 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.d] 2023-05-20T01:27:09.627 INFO:teuthology.orchestra.run.smithi016.stdout:[mds] 2023-05-20T01:27:09.627 INFO:teuthology.orchestra.run.smithi016.stdout: debug mds = 20 2023-05-20T01:27:09.627 INFO:teuthology.orchestra.run.smithi016.stdout: debug mds balancer = 20 2023-05-20T01:27:09.627 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2023-05-20T01:27:09.627 INFO:teuthology.orchestra.run.smithi016.stdout: mds debug frag = True 2023-05-20T01:27:09.627 INFO:teuthology.orchestra.run.smithi016.stdout: mds debug scatterstat = False 2023-05-20T01:27:09.627 INFO:teuthology.orchestra.run.smithi016.stdout: mds op complaint time = 180 2023-05-20T01:27:09.627 INFO:teuthology.orchestra.run.smithi016.stdout: mds verify scatter = False 2023-05-20T01:27:09.628 INFO:teuthology.orchestra.run.smithi016.stdout: osd op complaint time = 180 2023-05-20T01:27:09.628 INFO:teuthology.orchestra.run.smithi016.stdout: rados mon op timeout = 900 2023-05-20T01:27:09.628 INFO:teuthology.orchestra.run.smithi016.stdout: rados osd op timeout = 900 2023-05-20T01:27:09.629 INFO:teuthology.orchestra.run.smithi132.stdout:[global] 2023-05-20T01:27:09.629 INFO:teuthology.orchestra.run.smithi132.stdout: chdir = "" 2023-05-20T01:27:09.629 INFO:teuthology.orchestra.run.smithi132.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-05-20T01:27:09.630 INFO:teuthology.orchestra.run.smithi132.stdout: auth supported = cephx 2023-05-20T01:27:09.630 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:27:09.630 INFO:teuthology.orchestra.run.smithi132.stdout: filestore xattr use omap = true 2023-05-20T01:27:09.630 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:27:09.630 INFO:teuthology.orchestra.run.smithi132.stdout: mon clock drift allowed = 1.000 2023-05-20T01:27:09.630 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:27:09.630 INFO:teuthology.orchestra.run.smithi132.stdout: osd crush chooseleaf type = 0 2023-05-20T01:27:09.631 INFO:teuthology.orchestra.run.smithi132.stdout: auth debug = true 2023-05-20T01:27:09.631 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:27:09.631 INFO:teuthology.orchestra.run.smithi132.stdout: ms die on old message = true 2023-05-20T01:27:09.631 INFO:teuthology.orchestra.run.smithi132.stdout: ms die on bug = true 2023-05-20T01:27:09.631 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:27:09.631 INFO:teuthology.orchestra.run.smithi132.stdout: mon max pg per osd = 10000 # >= luminous 2023-05-20T01:27:09.631 INFO:teuthology.orchestra.run.smithi132.stdout: mon pg warn max object skew = 0 2023-05-20T01:27:09.631 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:27:09.632 INFO:teuthology.orchestra.run.smithi132.stdout: # disable pg_autoscaler by default for new pools 2023-05-20T01:27:09.632 INFO:teuthology.orchestra.run.smithi132.stdout: osd_pool_default_pg_autoscale_mode = off 2023-05-20T01:27:09.632 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:27:09.632 INFO:teuthology.orchestra.run.smithi132.stdout: osd pool default size = 2 2023-05-20T01:27:09.632 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:27:09.632 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd allow primary affinity = true 2023-05-20T01:27:09.632 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd allow pg remap = true 2023-05-20T01:27:09.633 INFO:teuthology.orchestra.run.smithi132.stdout: mon warn on legacy crush tunables = false 2023-05-20T01:27:09.633 INFO:teuthology.orchestra.run.smithi132.stdout: mon warn on crush straw calc version zero = false 2023-05-20T01:27:09.633 INFO:teuthology.orchestra.run.smithi132.stdout: mon warn on no sortbitwise = false 2023-05-20T01:27:09.633 INFO:teuthology.orchestra.run.smithi132.stdout: mon warn on osd down out interval zero = false 2023-05-20T01:27:09.633 INFO:teuthology.orchestra.run.smithi132.stdout: mon warn on too few osds = false 2023-05-20T01:27:09.633 INFO:teuthology.orchestra.run.smithi132.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-05-20T01:27:09.633 INFO:teuthology.orchestra.run.smithi132.stdout: mon_warn_on_pool_no_redundancy = false 2023-05-20T01:27:09.633 INFO:teuthology.orchestra.run.smithi132.stdout: mon_allow_pool_size_one = true 2023-05-20T01:27:09.634 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:27:09.634 INFO:teuthology.orchestra.run.smithi132.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-05-20T01:27:09.634 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:27:09.634 INFO:teuthology.orchestra.run.smithi132.stdout: osd default data pool replay window = 5 2023-05-20T01:27:09.634 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:27:09.634 INFO:teuthology.orchestra.run.smithi132.stdout: mon allow pool delete = true 2023-05-20T01:27:09.634 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:27:09.634 INFO:teuthology.orchestra.run.smithi132.stdout: mon cluster log file level = debug 2023-05-20T01:27:09.635 INFO:teuthology.orchestra.run.smithi132.stdout: debug asserts on shutdown = true 2023-05-20T01:27:09.635 INFO:teuthology.orchestra.run.smithi132.stdout: mon health detail to clog = false 2023-05-20T01:27:09.635 INFO:teuthology.orchestra.run.smithi132.stdout: mon host = "172.21.15.16,172.21.15.132,[v2:172.21.15.132:3301,v1:172.21.15.132:6790]" 2023-05-20T01:27:09.635 INFO:teuthology.orchestra.run.smithi132.stdout: ms die on skipped message = False 2023-05-20T01:27:09.635 INFO:teuthology.orchestra.run.smithi132.stdout: fsid = c2245cc5-ecab-4087-a59b-bc81db901676 2023-05-20T01:27:09.635 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:27:09.635 INFO:teuthology.orchestra.run.smithi132.stdout:[osd] 2023-05-20T01:27:09.635 INFO:teuthology.orchestra.run.smithi132.stdout: osd journal size = 100 2023-05-20T01:27:09.636 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:27:09.636 INFO:teuthology.orchestra.run.smithi132.stdout: osd scrub load threshold = 5.0 2023-05-20T01:27:09.636 INFO:teuthology.orchestra.run.smithi132.stdout: osd scrub max interval = 600 2023-05-20T01:27:09.636 INFO:teuthology.orchestra.run.smithi132.stdout: osd mclock profile = high_recovery_ops 2023-05-20T01:27:09.636 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:27:09.636 INFO:teuthology.orchestra.run.smithi132.stdout: osd recover clone overlap = true 2023-05-20T01:27:09.636 INFO:teuthology.orchestra.run.smithi132.stdout: osd recovery max chunk = 1048576 2023-05-20T01:27:09.637 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:27:09.637 INFO:teuthology.orchestra.run.smithi132.stdout: osd debug shutdown = true 2023-05-20T01:27:09.637 INFO:teuthology.orchestra.run.smithi132.stdout: osd debug op order = true 2023-05-20T01:27:09.637 INFO:teuthology.orchestra.run.smithi132.stdout: osd debug verify stray on activate = true 2023-05-20T01:27:09.637 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:27:09.637 INFO:teuthology.orchestra.run.smithi132.stdout: osd open classes on start = true 2023-05-20T01:27:09.637 INFO:teuthology.orchestra.run.smithi132.stdout: osd debug pg log writeout = true 2023-05-20T01:27:09.637 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:27:09.638 INFO:teuthology.orchestra.run.smithi132.stdout: osd deep scrub update digest min age = 30 2023-05-20T01:27:09.638 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:27:09.638 INFO:teuthology.orchestra.run.smithi132.stdout: osd map max advance = 10 2023-05-20T01:27:09.638 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:27:09.638 INFO:teuthology.orchestra.run.smithi132.stdout: journal zero on create = true 2023-05-20T01:27:09.638 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:27:09.638 INFO:teuthology.orchestra.run.smithi132.stdout: filestore ondisk finisher threads = 3 2023-05-20T01:27:09.638 INFO:teuthology.orchestra.run.smithi132.stdout: filestore apply finisher threads = 3 2023-05-20T01:27:09.639 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:27:09.639 INFO:teuthology.orchestra.run.smithi132.stdout: bdev debug aio = true 2023-05-20T01:27:09.639 INFO:teuthology.orchestra.run.smithi132.stdout: osd debug misdirected ops = true 2023-05-20T01:27:09.639 INFO:teuthology.orchestra.run.smithi132.stdout: bluestore block size = 96636764160 2023-05-20T01:27:09.639 INFO:teuthology.orchestra.run.smithi132.stdout: bluestore fsck on mount = True 2023-05-20T01:27:09.639 INFO:teuthology.orchestra.run.smithi132.stdout: debug bluefs = 20 2023-05-20T01:27:09.639 INFO:teuthology.orchestra.run.smithi132.stdout: debug bluestore = 20 2023-05-20T01:27:09.639 INFO:teuthology.orchestra.run.smithi132.stdout: debug ms = 1 2023-05-20T01:27:09.640 INFO:teuthology.orchestra.run.smithi132.stdout: debug osd = 20 2023-05-20T01:27:09.640 INFO:teuthology.orchestra.run.smithi132.stdout: debug rocksdb = 10 2023-05-20T01:27:09.640 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd backfillfull_ratio = 0.85 2023-05-20T01:27:09.640 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd full ratio = 0.9 2023-05-20T01:27:09.640 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd nearfull ratio = 0.8 2023-05-20T01:27:09.640 INFO:teuthology.orchestra.run.smithi132.stdout: osd failsafe full ratio = 0.95 2023-05-20T01:27:09.640 INFO:teuthology.orchestra.run.smithi132.stdout: osd objectstore = bluestore 2023-05-20T01:27:09.640 INFO:teuthology.orchestra.run.smithi132.stdout: osd op complaint time = 180 2023-05-20T01:27:09.641 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:27:09.641 INFO:teuthology.orchestra.run.smithi132.stdout:[mgr] 2023-05-20T01:27:09.641 INFO:teuthology.orchestra.run.smithi132.stdout: debug ms = 1 2023-05-20T01:27:09.641 INFO:teuthology.orchestra.run.smithi132.stdout: debug mgr = 20 2023-05-20T01:27:09.641 INFO:teuthology.orchestra.run.smithi132.stdout: debug mon = 20 2023-05-20T01:27:09.641 INFO:teuthology.orchestra.run.smithi132.stdout: debug auth = 20 2023-05-20T01:27:09.641 INFO:teuthology.orchestra.run.smithi132.stdout: mon reweight min pgs per osd = 4 2023-05-20T01:27:09.642 INFO:teuthology.orchestra.run.smithi132.stdout: mon reweight min bytes per osd = 10 2023-05-20T01:27:09.642 INFO:teuthology.orchestra.run.smithi132.stdout: mgr/telemetry/nag = false 2023-05-20T01:27:09.642 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:27:09.642 INFO:teuthology.orchestra.run.smithi132.stdout:[mon] 2023-05-20T01:27:09.642 INFO:teuthology.orchestra.run.smithi132.stdout: debug ms = 1 2023-05-20T01:27:09.642 INFO:teuthology.orchestra.run.smithi132.stdout: debug mon = 20 2023-05-20T01:27:09.642 INFO:teuthology.orchestra.run.smithi132.stdout: debug paxos = 20 2023-05-20T01:27:09.642 INFO:teuthology.orchestra.run.smithi132.stdout: debug auth = 20 2023-05-20T01:27:09.643 INFO:teuthology.orchestra.run.smithi132.stdout: mon data avail warn = 5 2023-05-20T01:27:09.643 INFO:teuthology.orchestra.run.smithi132.stdout: mon mgr mkfs grace = 240 2023-05-20T01:27:09.643 INFO:teuthology.orchestra.run.smithi132.stdout: mon reweight min pgs per osd = 4 2023-05-20T01:27:09.643 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd reporter subtree level = osd 2023-05-20T01:27:09.643 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd prime pg temp = true 2023-05-20T01:27:09.643 INFO:teuthology.orchestra.run.smithi132.stdout: mon reweight min bytes per osd = 10 2023-05-20T01:27:09.643 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:27:09.643 INFO:teuthology.orchestra.run.smithi132.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-05-20T01:27:09.644 INFO:teuthology.orchestra.run.smithi132.stdout: auth mon ticket ttl = 660 # 11m 2023-05-20T01:27:09.644 INFO:teuthology.orchestra.run.smithi132.stdout: auth service ticket ttl = 240 # 4m 2023-05-20T01:27:09.644 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:27:09.644 INFO:teuthology.orchestra.run.smithi132.stdout: # don't complain about insecure global_id in the test suite 2023-05-20T01:27:09.644 INFO:teuthology.orchestra.run.smithi132.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-05-20T01:27:09.644 INFO:teuthology.orchestra.run.smithi132.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-05-20T01:27:09.644 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:27:09.644 INFO:teuthology.orchestra.run.smithi132.stdout: # 1m isn't quite enough 2023-05-20T01:27:09.645 INFO:teuthology.orchestra.run.smithi132.stdout: mon_down_mkfs_grace = 2m 2023-05-20T01:27:09.645 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:27:09.645 INFO:teuthology.orchestra.run.smithi132.stdout: mon_warn_on_filestore_osds = false 2023-05-20T01:27:09.645 INFO:teuthology.orchestra.run.smithi132.stdout: mon op complaint time = 120 2023-05-20T01:27:09.645 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:27:09.645 INFO:teuthology.orchestra.run.smithi132.stdout:[client] 2023-05-20T01:27:09.645 INFO:teuthology.orchestra.run.smithi132.stdout: rgw cache enabled = true 2023-05-20T01:27:09.645 INFO:teuthology.orchestra.run.smithi132.stdout: rgw enable ops log = true 2023-05-20T01:27:09.646 INFO:teuthology.orchestra.run.smithi132.stdout: rgw enable usage log = true 2023-05-20T01:27:09.646 INFO:teuthology.orchestra.run.smithi132.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-05-20T01:27:09.646 INFO:teuthology.orchestra.run.smithi132.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-05-20T01:27:09.646 INFO:teuthology.orchestra.run.smithi132.stdout: client mount timeout = 600 2023-05-20T01:27:09.646 INFO:teuthology.orchestra.run.smithi132.stdout: debug client = 20 2023-05-20T01:27:09.646 INFO:teuthology.orchestra.run.smithi132.stdout: debug ms = 1 2023-05-20T01:27:09.646 INFO:teuthology.orchestra.run.smithi132.stdout: pid file = "" 2023-05-20T01:27:09.646 INFO:teuthology.orchestra.run.smithi132.stdout: rados mon op timeout = 900 2023-05-20T01:27:09.647 INFO:teuthology.orchestra.run.smithi132.stdout: rados osd op timeout = 900 2023-05-20T01:27:09.647 INFO:teuthology.orchestra.run.smithi132.stdout:[mon.a] 2023-05-20T01:27:09.647 INFO:teuthology.orchestra.run.smithi132.stdout:[mon.b] 2023-05-20T01:27:09.647 INFO:teuthology.orchestra.run.smithi132.stdout:[mon.c] 2023-05-20T01:27:09.647 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.a] 2023-05-20T01:27:09.647 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.c] 2023-05-20T01:27:09.647 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.b] 2023-05-20T01:27:09.648 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.d] 2023-05-20T01:27:09.648 INFO:teuthology.orchestra.run.smithi132.stdout:[mds] 2023-05-20T01:27:09.648 INFO:teuthology.orchestra.run.smithi132.stdout: debug mds = 20 2023-05-20T01:27:09.648 INFO:teuthology.orchestra.run.smithi132.stdout: debug mds balancer = 20 2023-05-20T01:27:09.648 INFO:teuthology.orchestra.run.smithi132.stdout: debug ms = 1 2023-05-20T01:27:09.648 INFO:teuthology.orchestra.run.smithi132.stdout: mds debug frag = True 2023-05-20T01:27:09.648 INFO:teuthology.orchestra.run.smithi132.stdout: mds debug scatterstat = False 2023-05-20T01:27:09.648 INFO:teuthology.orchestra.run.smithi132.stdout: mds op complaint time = 180 2023-05-20T01:27:09.649 INFO:teuthology.orchestra.run.smithi132.stdout: mds verify scatter = False 2023-05-20T01:27:09.649 INFO:teuthology.orchestra.run.smithi132.stdout: osd op complaint time = 180 2023-05-20T01:27:09.649 INFO:teuthology.orchestra.run.smithi132.stdout: rados mon op timeout = 900 2023-05-20T01:27:09.649 INFO:teuthology.orchestra.run.smithi132.stdout: rados osd op timeout = 900 2023-05-20T01:27:09.650 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-05-20T01:27:09.671 DEBUG:teuthology.orchestra.run.smithi132:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-05-20T01:27:09.716 INFO:teuthology.orchestra.run.smithi016.stdout:[global] 2023-05-20T01:27:09.717 INFO:teuthology.orchestra.run.smithi016.stdout: chdir = "" 2023-05-20T01:27:09.717 INFO:teuthology.orchestra.run.smithi016.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-05-20T01:27:09.717 INFO:teuthology.orchestra.run.smithi016.stdout: auth supported = cephx 2023-05-20T01:27:09.717 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:27:09.717 INFO:teuthology.orchestra.run.smithi016.stdout: filestore xattr use omap = true 2023-05-20T01:27:09.718 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:27:09.718 INFO:teuthology.orchestra.run.smithi016.stdout: mon clock drift allowed = 1.000 2023-05-20T01:27:09.718 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:27:09.718 INFO:teuthology.orchestra.run.smithi016.stdout: osd crush chooseleaf type = 0 2023-05-20T01:27:09.718 INFO:teuthology.orchestra.run.smithi016.stdout: auth debug = true 2023-05-20T01:27:09.719 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:27:09.719 INFO:teuthology.orchestra.run.smithi016.stdout: ms die on old message = true 2023-05-20T01:27:09.719 INFO:teuthology.orchestra.run.smithi016.stdout: ms die on bug = true 2023-05-20T01:27:09.719 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:27:09.719 INFO:teuthology.orchestra.run.smithi016.stdout: mon max pg per osd = 10000 # >= luminous 2023-05-20T01:27:09.720 INFO:teuthology.orchestra.run.smithi016.stdout: mon pg warn max object skew = 0 2023-05-20T01:27:09.720 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:27:09.720 INFO:teuthology.orchestra.run.smithi016.stdout: # disable pg_autoscaler by default for new pools 2023-05-20T01:27:09.720 INFO:teuthology.orchestra.run.smithi016.stdout: osd_pool_default_pg_autoscale_mode = off 2023-05-20T01:27:09.721 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:27:09.721 INFO:teuthology.orchestra.run.smithi016.stdout: osd pool default size = 2 2023-05-20T01:27:09.721 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:27:09.721 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd allow primary affinity = true 2023-05-20T01:27:09.722 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd allow pg remap = true 2023-05-20T01:27:09.723 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on legacy crush tunables = false 2023-05-20T01:27:09.723 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on crush straw calc version zero = false 2023-05-20T01:27:09.723 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on no sortbitwise = false 2023-05-20T01:27:09.723 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on osd down out interval zero = false 2023-05-20T01:27:09.724 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on too few osds = false 2023-05-20T01:27:09.724 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-05-20T01:27:09.724 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_pool_no_redundancy = false 2023-05-20T01:27:09.725 INFO:teuthology.orchestra.run.smithi016.stdout: mon_allow_pool_size_one = true 2023-05-20T01:27:09.725 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:27:09.725 INFO:teuthology.orchestra.run.smithi016.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-05-20T01:27:09.725 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:27:09.726 INFO:teuthology.orchestra.run.smithi016.stdout: osd default data pool replay window = 5 2023-05-20T01:27:09.726 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:27:09.727 INFO:teuthology.orchestra.run.smithi016.stdout: mon allow pool delete = true 2023-05-20T01:27:09.727 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:27:09.727 INFO:teuthology.orchestra.run.smithi016.stdout: mon cluster log file level = debug 2023-05-20T01:27:09.727 INFO:teuthology.orchestra.run.smithi016.stdout: debug asserts on shutdown = true 2023-05-20T01:27:09.728 INFO:teuthology.orchestra.run.smithi016.stdout: mon health detail to clog = false 2023-05-20T01:27:09.728 INFO:teuthology.orchestra.run.smithi016.stdout: mon host = "172.21.15.16,172.21.15.132,[v2:172.21.15.132:3301,v1:172.21.15.132:6790]" 2023-05-20T01:27:09.728 INFO:teuthology.orchestra.run.smithi016.stdout: ms die on skipped message = False 2023-05-20T01:27:09.728 INFO:teuthology.orchestra.run.smithi016.stdout: fsid = c2245cc5-ecab-4087-a59b-bc81db901676 2023-05-20T01:27:09.728 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:27:09.729 INFO:teuthology.orchestra.run.smithi016.stdout:[osd] 2023-05-20T01:27:09.729 INFO:teuthology.orchestra.run.smithi016.stdout: osd journal size = 100 2023-05-20T01:27:09.729 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:27:09.729 INFO:teuthology.orchestra.run.smithi016.stdout: osd scrub load threshold = 5.0 2023-05-20T01:27:09.729 INFO:teuthology.orchestra.run.smithi016.stdout: osd scrub max interval = 600 2023-05-20T01:27:09.730 INFO:teuthology.orchestra.run.smithi016.stdout: osd mclock profile = high_recovery_ops 2023-05-20T01:27:09.730 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:27:09.731 INFO:teuthology.orchestra.run.smithi016.stdout: osd recover clone overlap = true 2023-05-20T01:27:09.731 INFO:teuthology.orchestra.run.smithi016.stdout: osd recovery max chunk = 1048576 2023-05-20T01:27:09.732 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:27:09.732 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug shutdown = true 2023-05-20T01:27:09.732 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug op order = true 2023-05-20T01:27:09.733 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug verify stray on activate = true 2023-05-20T01:27:09.733 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:27:09.734 INFO:teuthology.orchestra.run.smithi016.stdout: osd open classes on start = true 2023-05-20T01:27:09.734 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug pg log writeout = true 2023-05-20T01:27:09.734 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:27:09.734 INFO:teuthology.orchestra.run.smithi016.stdout: osd deep scrub update digest min age = 30 2023-05-20T01:27:09.735 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:27:09.735 INFO:teuthology.orchestra.run.smithi016.stdout: osd map max advance = 10 2023-05-20T01:27:09.735 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:27:09.736 INFO:teuthology.orchestra.run.smithi016.stdout: journal zero on create = true 2023-05-20T01:27:09.736 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:27:09.736 INFO:teuthology.orchestra.run.smithi016.stdout: filestore ondisk finisher threads = 3 2023-05-20T01:27:09.737 INFO:teuthology.orchestra.run.smithi016.stdout: filestore apply finisher threads = 3 2023-05-20T01:27:09.737 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:27:09.738 INFO:teuthology.orchestra.run.smithi016.stdout: bdev debug aio = true 2023-05-20T01:27:09.738 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug misdirected ops = true 2023-05-20T01:27:09.738 INFO:teuthology.orchestra.run.smithi016.stdout: bluestore block size = 96636764160 2023-05-20T01:27:09.739 INFO:teuthology.orchestra.run.smithi016.stdout: bluestore fsck on mount = True 2023-05-20T01:27:09.739 INFO:teuthology.orchestra.run.smithi016.stdout: debug bluefs = 20 2023-05-20T01:27:09.739 INFO:teuthology.orchestra.run.smithi016.stdout: debug bluestore = 20 2023-05-20T01:27:09.740 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2023-05-20T01:27:09.740 INFO:teuthology.orchestra.run.smithi016.stdout: debug osd = 20 2023-05-20T01:27:09.740 INFO:teuthology.orchestra.run.smithi016.stdout: debug rocksdb = 10 2023-05-20T01:27:09.740 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd backfillfull_ratio = 0.85 2023-05-20T01:27:09.741 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd full ratio = 0.9 2023-05-20T01:27:09.741 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd nearfull ratio = 0.8 2023-05-20T01:27:09.741 INFO:teuthology.orchestra.run.smithi016.stdout: osd failsafe full ratio = 0.95 2023-05-20T01:27:09.741 INFO:teuthology.orchestra.run.smithi016.stdout: osd objectstore = bluestore 2023-05-20T01:27:09.742 INFO:teuthology.orchestra.run.smithi016.stdout: osd op complaint time = 180 2023-05-20T01:27:09.742 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:27:09.742 INFO:teuthology.orchestra.run.smithi016.stdout:[mgr] 2023-05-20T01:27:09.743 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2023-05-20T01:27:09.743 INFO:teuthology.orchestra.run.smithi016.stdout: debug mgr = 20 2023-05-20T01:27:09.744 INFO:teuthology.orchestra.run.smithi016.stdout: debug mon = 20 2023-05-20T01:27:09.744 INFO:teuthology.orchestra.run.smithi016.stdout: debug auth = 20 2023-05-20T01:27:09.745 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min pgs per osd = 4 2023-05-20T01:27:09.745 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min bytes per osd = 10 2023-05-20T01:27:09.745 INFO:teuthology.orchestra.run.smithi016.stdout: mgr/telemetry/nag = false 2023-05-20T01:27:09.745 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:27:09.746 INFO:teuthology.orchestra.run.smithi016.stdout:[mon] 2023-05-20T01:27:09.746 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2023-05-20T01:27:09.746 INFO:teuthology.orchestra.run.smithi016.stdout: debug mon = 20 2023-05-20T01:27:09.746 INFO:teuthology.orchestra.run.smithi016.stdout: debug paxos = 20 2023-05-20T01:27:09.746 INFO:teuthology.orchestra.run.smithi016.stdout: debug auth = 20 2023-05-20T01:27:09.746 INFO:teuthology.orchestra.run.smithi016.stdout: mon data avail warn = 5 2023-05-20T01:27:09.746 INFO:teuthology.orchestra.run.smithi016.stdout: mon mgr mkfs grace = 240 2023-05-20T01:27:09.747 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min pgs per osd = 4 2023-05-20T01:27:09.747 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd reporter subtree level = osd 2023-05-20T01:27:09.747 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd prime pg temp = true 2023-05-20T01:27:09.747 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min bytes per osd = 10 2023-05-20T01:27:09.747 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:27:09.747 INFO:teuthology.orchestra.run.smithi016.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-05-20T01:27:09.748 INFO:teuthology.orchestra.run.smithi016.stdout: auth mon ticket ttl = 660 # 11m 2023-05-20T01:27:09.748 INFO:teuthology.orchestra.run.smithi016.stdout: auth service ticket ttl = 240 # 4m 2023-05-20T01:27:09.748 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:27:09.748 INFO:teuthology.orchestra.run.smithi016.stdout: # don't complain about insecure global_id in the test suite 2023-05-20T01:27:09.748 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-05-20T01:27:09.748 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-05-20T01:27:09.748 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:27:09.749 INFO:teuthology.orchestra.run.smithi016.stdout: # 1m isn't quite enough 2023-05-20T01:27:09.749 INFO:teuthology.orchestra.run.smithi016.stdout: mon_down_mkfs_grace = 2m 2023-05-20T01:27:09.749 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:27:09.749 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_filestore_osds = false 2023-05-20T01:27:09.749 INFO:teuthology.orchestra.run.smithi016.stdout: mon op complaint time = 120 2023-05-20T01:27:09.749 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:27:09.750 INFO:teuthology.orchestra.run.smithi016.stdout:[client] 2023-05-20T01:27:09.750 INFO:teuthology.orchestra.run.smithi016.stdout: rgw cache enabled = true 2023-05-20T01:27:09.750 INFO:teuthology.orchestra.run.smithi016.stdout: rgw enable ops log = true 2023-05-20T01:27:09.750 INFO:teuthology.orchestra.run.smithi016.stdout: rgw enable usage log = true 2023-05-20T01:27:09.750 INFO:teuthology.orchestra.run.smithi016.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-05-20T01:27:09.751 INFO:teuthology.orchestra.run.smithi016.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-05-20T01:27:09.751 INFO:teuthology.orchestra.run.smithi016.stdout: client mount timeout = 600 2023-05-20T01:27:09.751 INFO:teuthology.orchestra.run.smithi016.stdout: debug client = 20 2023-05-20T01:27:09.751 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2023-05-20T01:27:09.751 INFO:teuthology.orchestra.run.smithi016.stdout: pid file = "" 2023-05-20T01:27:09.752 INFO:teuthology.orchestra.run.smithi016.stdout: rados mon op timeout = 900 2023-05-20T01:27:09.752 INFO:teuthology.orchestra.run.smithi016.stdout: rados osd op timeout = 900 2023-05-20T01:27:09.752 INFO:teuthology.orchestra.run.smithi016.stdout:[mon.a] 2023-05-20T01:27:09.752 INFO:teuthology.orchestra.run.smithi016.stdout:[mon.b] 2023-05-20T01:27:09.752 INFO:teuthology.orchestra.run.smithi016.stdout:[mon.c] 2023-05-20T01:27:09.752 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.a] 2023-05-20T01:27:09.752 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.c] 2023-05-20T01:27:09.753 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.b] 2023-05-20T01:27:09.753 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.d] 2023-05-20T01:27:09.753 INFO:teuthology.orchestra.run.smithi016.stdout:[mds] 2023-05-20T01:27:09.753 INFO:teuthology.orchestra.run.smithi016.stdout: debug mds = 20 2023-05-20T01:27:09.754 INFO:teuthology.orchestra.run.smithi016.stdout: debug mds balancer = 20 2023-05-20T01:27:09.754 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2023-05-20T01:27:09.754 INFO:teuthology.orchestra.run.smithi016.stdout: mds debug frag = True 2023-05-20T01:27:09.754 INFO:teuthology.orchestra.run.smithi016.stdout: mds debug scatterstat = False 2023-05-20T01:27:09.755 INFO:teuthology.orchestra.run.smithi016.stdout: mds op complaint time = 180 2023-05-20T01:27:09.755 INFO:teuthology.orchestra.run.smithi016.stdout: mds verify scatter = False 2023-05-20T01:27:09.755 INFO:teuthology.orchestra.run.smithi016.stdout: osd op complaint time = 180 2023-05-20T01:27:09.756 INFO:teuthology.orchestra.run.smithi016.stdout: rados mon op timeout = 900 2023-05-20T01:27:09.756 INFO:teuthology.orchestra.run.smithi016.stdout: rados osd op timeout = 900 2023-05-20T01:27:09.758 INFO:teuthology.orchestra.run.smithi132.stdout:[global] 2023-05-20T01:27:09.758 INFO:teuthology.orchestra.run.smithi132.stdout: chdir = "" 2023-05-20T01:27:09.758 INFO:teuthology.orchestra.run.smithi132.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-05-20T01:27:09.759 INFO:teuthology.orchestra.run.smithi132.stdout: auth supported = cephx 2023-05-20T01:27:09.759 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:27:09.759 INFO:teuthology.orchestra.run.smithi132.stdout: filestore xattr use omap = true 2023-05-20T01:27:09.759 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:27:09.759 INFO:teuthology.orchestra.run.smithi132.stdout: mon clock drift allowed = 1.000 2023-05-20T01:27:09.759 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:27:09.759 INFO:teuthology.orchestra.run.smithi132.stdout: osd crush chooseleaf type = 0 2023-05-20T01:27:09.760 INFO:teuthology.orchestra.run.smithi132.stdout: auth debug = true 2023-05-20T01:27:09.760 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:27:09.761 INFO:teuthology.orchestra.run.smithi132.stdout: ms die on old message = true 2023-05-20T01:27:09.761 INFO:teuthology.orchestra.run.smithi132.stdout: ms die on bug = true 2023-05-20T01:27:09.761 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:27:09.761 INFO:teuthology.orchestra.run.smithi132.stdout: mon max pg per osd = 10000 # >= luminous 2023-05-20T01:27:09.761 INFO:teuthology.orchestra.run.smithi132.stdout: mon pg warn max object skew = 0 2023-05-20T01:27:09.761 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:27:09.761 INFO:teuthology.orchestra.run.smithi132.stdout: # disable pg_autoscaler by default for new pools 2023-05-20T01:27:09.761 INFO:teuthology.orchestra.run.smithi132.stdout: osd_pool_default_pg_autoscale_mode = off 2023-05-20T01:27:09.762 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:27:09.762 INFO:teuthology.orchestra.run.smithi132.stdout: osd pool default size = 2 2023-05-20T01:27:09.762 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:27:09.762 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd allow primary affinity = true 2023-05-20T01:27:09.762 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd allow pg remap = true 2023-05-20T01:27:09.762 INFO:teuthology.orchestra.run.smithi132.stdout: mon warn on legacy crush tunables = false 2023-05-20T01:27:09.762 INFO:teuthology.orchestra.run.smithi132.stdout: mon warn on crush straw calc version zero = false 2023-05-20T01:27:09.763 INFO:teuthology.orchestra.run.smithi132.stdout: mon warn on no sortbitwise = false 2023-05-20T01:27:09.763 INFO:teuthology.orchestra.run.smithi132.stdout: mon warn on osd down out interval zero = false 2023-05-20T01:27:09.763 INFO:teuthology.orchestra.run.smithi132.stdout: mon warn on too few osds = false 2023-05-20T01:27:09.763 INFO:teuthology.orchestra.run.smithi132.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-05-20T01:27:09.763 INFO:teuthology.orchestra.run.smithi132.stdout: mon_warn_on_pool_no_redundancy = false 2023-05-20T01:27:09.763 INFO:teuthology.orchestra.run.smithi132.stdout: mon_allow_pool_size_one = true 2023-05-20T01:27:09.763 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:27:09.764 INFO:teuthology.orchestra.run.smithi132.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-05-20T01:27:09.764 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:27:09.764 INFO:teuthology.orchestra.run.smithi132.stdout: osd default data pool replay window = 5 2023-05-20T01:27:09.765 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:27:09.765 INFO:teuthology.orchestra.run.smithi132.stdout: mon allow pool delete = true 2023-05-20T01:27:09.765 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:27:09.766 INFO:teuthology.orchestra.run.smithi132.stdout: mon cluster log file level = debug 2023-05-20T01:27:09.766 INFO:teuthology.orchestra.run.smithi132.stdout: debug asserts on shutdown = true 2023-05-20T01:27:09.766 INFO:teuthology.orchestra.run.smithi132.stdout: mon health detail to clog = false 2023-05-20T01:27:09.766 INFO:teuthology.orchestra.run.smithi132.stdout: mon host = "172.21.15.16,172.21.15.132,[v2:172.21.15.132:3301,v1:172.21.15.132:6790]" 2023-05-20T01:27:09.766 INFO:teuthology.orchestra.run.smithi132.stdout: ms die on skipped message = False 2023-05-20T01:27:09.766 INFO:teuthology.orchestra.run.smithi132.stdout: fsid = c2245cc5-ecab-4087-a59b-bc81db901676 2023-05-20T01:27:09.766 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:27:09.767 INFO:teuthology.orchestra.run.smithi132.stdout:[osd] 2023-05-20T01:27:09.767 INFO:teuthology.orchestra.run.smithi132.stdout: osd journal size = 100 2023-05-20T01:27:09.767 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:27:09.767 INFO:teuthology.orchestra.run.smithi132.stdout: osd scrub load threshold = 5.0 2023-05-20T01:27:09.767 INFO:teuthology.orchestra.run.smithi132.stdout: osd scrub max interval = 600 2023-05-20T01:27:09.767 INFO:teuthology.orchestra.run.smithi132.stdout: osd mclock profile = high_recovery_ops 2023-05-20T01:27:09.767 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:27:09.768 INFO:teuthology.orchestra.run.smithi132.stdout: osd recover clone overlap = true 2023-05-20T01:27:09.768 INFO:teuthology.orchestra.run.smithi132.stdout: osd recovery max chunk = 1048576 2023-05-20T01:27:09.768 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:27:09.769 INFO:teuthology.orchestra.run.smithi132.stdout: osd debug shutdown = true 2023-05-20T01:27:09.769 INFO:teuthology.orchestra.run.smithi132.stdout: osd debug op order = true 2023-05-20T01:27:09.769 INFO:teuthology.orchestra.run.smithi132.stdout: osd debug verify stray on activate = true 2023-05-20T01:27:09.769 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:27:09.769 INFO:teuthology.orchestra.run.smithi132.stdout: osd open classes on start = true 2023-05-20T01:27:09.770 INFO:teuthology.orchestra.run.smithi132.stdout: osd debug pg log writeout = true 2023-05-20T01:27:09.770 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:27:09.770 INFO:teuthology.orchestra.run.smithi132.stdout: osd deep scrub update digest min age = 30 2023-05-20T01:27:09.770 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:27:09.770 INFO:teuthology.orchestra.run.smithi132.stdout: osd map max advance = 10 2023-05-20T01:27:09.770 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:27:09.770 INFO:teuthology.orchestra.run.smithi132.stdout: journal zero on create = true 2023-05-20T01:27:09.771 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:27:09.771 INFO:teuthology.orchestra.run.smithi132.stdout: filestore ondisk finisher threads = 3 2023-05-20T01:27:09.771 INFO:teuthology.orchestra.run.smithi132.stdout: filestore apply finisher threads = 3 2023-05-20T01:27:09.771 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:27:09.771 INFO:teuthology.orchestra.run.smithi132.stdout: bdev debug aio = true 2023-05-20T01:27:09.771 INFO:teuthology.orchestra.run.smithi132.stdout: osd debug misdirected ops = true 2023-05-20T01:27:09.772 INFO:teuthology.orchestra.run.smithi132.stdout: bluestore block size = 96636764160 2023-05-20T01:27:09.772 INFO:teuthology.orchestra.run.smithi132.stdout: bluestore fsck on mount = True 2023-05-20T01:27:09.772 INFO:teuthology.orchestra.run.smithi132.stdout: debug bluefs = 20 2023-05-20T01:27:09.772 INFO:teuthology.orchestra.run.smithi132.stdout: debug bluestore = 20 2023-05-20T01:27:09.772 INFO:teuthology.orchestra.run.smithi132.stdout: debug ms = 1 2023-05-20T01:27:09.772 INFO:teuthology.orchestra.run.smithi132.stdout: debug osd = 20 2023-05-20T01:27:09.772 INFO:teuthology.orchestra.run.smithi132.stdout: debug rocksdb = 10 2023-05-20T01:27:09.772 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd backfillfull_ratio = 0.85 2023-05-20T01:27:09.773 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd full ratio = 0.9 2023-05-20T01:27:09.773 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd nearfull ratio = 0.8 2023-05-20T01:27:09.773 INFO:teuthology.orchestra.run.smithi132.stdout: osd failsafe full ratio = 0.95 2023-05-20T01:27:09.773 INFO:teuthology.orchestra.run.smithi132.stdout: osd objectstore = bluestore 2023-05-20T01:27:09.773 INFO:teuthology.orchestra.run.smithi132.stdout: osd op complaint time = 180 2023-05-20T01:27:09.773 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:27:09.773 INFO:teuthology.orchestra.run.smithi132.stdout:[mgr] 2023-05-20T01:27:09.774 INFO:teuthology.orchestra.run.smithi132.stdout: debug ms = 1 2023-05-20T01:27:09.774 INFO:teuthology.orchestra.run.smithi132.stdout: debug mgr = 20 2023-05-20T01:27:09.774 INFO:teuthology.orchestra.run.smithi132.stdout: debug mon = 20 2023-05-20T01:27:09.774 INFO:teuthology.orchestra.run.smithi132.stdout: debug auth = 20 2023-05-20T01:27:09.774 INFO:teuthology.orchestra.run.smithi132.stdout: mon reweight min pgs per osd = 4 2023-05-20T01:27:09.774 INFO:teuthology.orchestra.run.smithi132.stdout: mon reweight min bytes per osd = 10 2023-05-20T01:27:09.774 INFO:teuthology.orchestra.run.smithi132.stdout: mgr/telemetry/nag = false 2023-05-20T01:27:09.774 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:27:09.775 INFO:teuthology.orchestra.run.smithi132.stdout:[mon] 2023-05-20T01:27:09.775 INFO:teuthology.orchestra.run.smithi132.stdout: debug ms = 1 2023-05-20T01:27:09.775 INFO:teuthology.orchestra.run.smithi132.stdout: debug mon = 20 2023-05-20T01:27:09.775 INFO:teuthology.orchestra.run.smithi132.stdout: debug paxos = 20 2023-05-20T01:27:09.775 INFO:teuthology.orchestra.run.smithi132.stdout: debug auth = 20 2023-05-20T01:27:09.775 INFO:teuthology.orchestra.run.smithi132.stdout: mon data avail warn = 5 2023-05-20T01:27:09.775 INFO:teuthology.orchestra.run.smithi132.stdout: mon mgr mkfs grace = 240 2023-05-20T01:27:09.775 INFO:teuthology.orchestra.run.smithi132.stdout: mon reweight min pgs per osd = 4 2023-05-20T01:27:09.776 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd reporter subtree level = osd 2023-05-20T01:27:09.776 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd prime pg temp = true 2023-05-20T01:27:09.776 INFO:teuthology.orchestra.run.smithi132.stdout: mon reweight min bytes per osd = 10 2023-05-20T01:27:09.776 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:27:09.776 INFO:teuthology.orchestra.run.smithi132.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-05-20T01:27:09.776 INFO:teuthology.orchestra.run.smithi132.stdout: auth mon ticket ttl = 660 # 11m 2023-05-20T01:27:09.776 INFO:teuthology.orchestra.run.smithi132.stdout: auth service ticket ttl = 240 # 4m 2023-05-20T01:27:09.777 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:27:09.777 INFO:teuthology.orchestra.run.smithi132.stdout: # don't complain about insecure global_id in the test suite 2023-05-20T01:27:09.777 INFO:teuthology.orchestra.run.smithi132.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-05-20T01:27:09.777 INFO:teuthology.orchestra.run.smithi132.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-05-20T01:27:09.777 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:27:09.777 INFO:teuthology.orchestra.run.smithi132.stdout: # 1m isn't quite enough 2023-05-20T01:27:09.777 INFO:teuthology.orchestra.run.smithi132.stdout: mon_down_mkfs_grace = 2m 2023-05-20T01:27:09.777 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:27:09.778 INFO:teuthology.orchestra.run.smithi132.stdout: mon_warn_on_filestore_osds = false 2023-05-20T01:27:09.778 INFO:teuthology.orchestra.run.smithi132.stdout: mon op complaint time = 120 2023-05-20T01:27:09.778 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:27:09.778 INFO:teuthology.orchestra.run.smithi132.stdout:[client] 2023-05-20T01:27:09.778 INFO:teuthology.orchestra.run.smithi132.stdout: rgw cache enabled = true 2023-05-20T01:27:09.778 INFO:teuthology.orchestra.run.smithi132.stdout: rgw enable ops log = true 2023-05-20T01:27:09.778 INFO:teuthology.orchestra.run.smithi132.stdout: rgw enable usage log = true 2023-05-20T01:27:09.779 INFO:teuthology.orchestra.run.smithi132.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-05-20T01:27:09.779 INFO:teuthology.orchestra.run.smithi132.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-05-20T01:27:09.779 INFO:teuthology.orchestra.run.smithi132.stdout: client mount timeout = 600 2023-05-20T01:27:09.779 INFO:teuthology.orchestra.run.smithi132.stdout: debug client = 20 2023-05-20T01:27:09.779 INFO:teuthology.orchestra.run.smithi132.stdout: debug ms = 1 2023-05-20T01:27:09.779 INFO:teuthology.orchestra.run.smithi132.stdout: pid file = "" 2023-05-20T01:27:09.779 INFO:teuthology.orchestra.run.smithi132.stdout: rados mon op timeout = 900 2023-05-20T01:27:09.780 INFO:teuthology.orchestra.run.smithi132.stdout: rados osd op timeout = 900 2023-05-20T01:27:09.780 INFO:teuthology.orchestra.run.smithi132.stdout:[mon.a] 2023-05-20T01:27:09.780 INFO:teuthology.orchestra.run.smithi132.stdout:[mon.b] 2023-05-20T01:27:09.780 INFO:teuthology.orchestra.run.smithi132.stdout:[mon.c] 2023-05-20T01:27:09.780 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.a] 2023-05-20T01:27:09.780 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.c] 2023-05-20T01:27:09.781 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.b] 2023-05-20T01:27:09.781 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.d] 2023-05-20T01:27:09.781 INFO:teuthology.orchestra.run.smithi132.stdout:[mds] 2023-05-20T01:27:09.781 INFO:teuthology.orchestra.run.smithi132.stdout: debug mds = 20 2023-05-20T01:27:09.781 INFO:teuthology.orchestra.run.smithi132.stdout: debug mds balancer = 20 2023-05-20T01:27:09.781 INFO:teuthology.orchestra.run.smithi132.stdout: debug ms = 1 2023-05-20T01:27:09.781 INFO:teuthology.orchestra.run.smithi132.stdout: mds debug frag = True 2023-05-20T01:27:09.782 INFO:teuthology.orchestra.run.smithi132.stdout: mds debug scatterstat = False 2023-05-20T01:27:09.782 INFO:teuthology.orchestra.run.smithi132.stdout: mds op complaint time = 180 2023-05-20T01:27:09.782 INFO:teuthology.orchestra.run.smithi132.stdout: mds verify scatter = False 2023-05-20T01:27:09.782 INFO:teuthology.orchestra.run.smithi132.stdout: osd op complaint time = 180 2023-05-20T01:27:09.782 INFO:teuthology.orchestra.run.smithi132.stdout: rados mon op timeout = 900 2023-05-20T01:27:09.782 INFO:teuthology.orchestra.run.smithi132.stdout: rados osd op timeout = 900 2023-05-20T01:27:09.783 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph purge cephfs_metadata --yes-i-really-really-mean-it 2023-05-20T01:27:09.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.821+0000 7f14efd9c080 1 Processor -- start 2023-05-20T01:27:09.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.821+0000 7f14efd9c080 1 -- start start 2023-05-20T01:27:09.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.822+0000 7f14efd9c080 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f726f3910 0x559f72661270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:09.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.822+0000 7f14efd9c080 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559f7270dbd0 0x559f7270ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:09.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.822+0000 7f14efd9c080 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559f7270e6b0 0x559f72709840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:09.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.822+0000 7f14efd9c080 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f72598e80 con 0x559f726f3910 2023-05-20T01:27:09.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.822+0000 7f14efd9c080 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f72598b20 con 0x559f7270e6b0 2023-05-20T01:27:09.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.822+0000 7f14efd9c080 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x559f7256efe0 con 0x559f7270dbd0 2023-05-20T01:27:09.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.822+0000 7f14e5e9c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f726f3910 0x559f72661270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:09.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.822+0000 7f14e5e9c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f726f3910 0x559f72661270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38564/0 (socket says 172.21.15.16:38564) 2023-05-20T01:27:09.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.822+0000 7f14e5e9c700 1 -- 172.21.15.16:0/1953258413 learned_addr learned my addr 172.21.15.16:0/1953258413 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:27:09.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.822+0000 7f14e569b700 1 --2- 172.21.15.16:0/1953258413 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559f7270dbd0 0x559f7270ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:09.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.822+0000 7f14e669d700 1 --2- 172.21.15.16:0/1953258413 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559f7270e6b0 0x559f72709840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:09.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.822+0000 7f14e5e9c700 1 -- 172.21.15.16:0/1953258413 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559f7270dbd0 msgr2=0x559f7270ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:09.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.822+0000 7f14e5e9c700 1 --2- 172.21.15.16:0/1953258413 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559f7270dbd0 0x559f7270ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:09.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.822+0000 7f14e5e9c700 1 -- 172.21.15.16:0/1953258413 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559f7270e6b0 msgr2=0x559f72709840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:09.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.822+0000 7f14e5e9c700 1 --2- 172.21.15.16:0/1953258413 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559f7270e6b0 0x559f72709840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:09.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.822+0000 7f14e5e9c700 1 -- 172.21.15.16:0/1953258413 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14e000a100 con 0x559f726f3910 2023-05-20T01:27:09.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.822+0000 7f14e5e9c700 1 --2- 172.21.15.16:0/1953258413 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f726f3910 0x559f72661270 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f14dc001cd0 tx=0x7f14dc00b5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdc424c4d986d380 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:09.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.822+0000 7f14d7fff700 1 -- 172.21.15.16:0/1953258413 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14dc00bd70 con 0x559f726f3910 2023-05-20T01:27:09.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.822+0000 7f14d7fff700 1 -- 172.21.15.16:0/1953258413 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14dc00e470 con 0x559f726f3910 2023-05-20T01:27:09.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.822+0000 7f14d7fff700 1 -- 172.21.15.16:0/1953258413 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14dc00e800 con 0x559f726f3910 2023-05-20T01:27:09.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.823+0000 7f14efd9c080 1 -- 172.21.15.16:0/1953258413 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f726f3910 msgr2=0x559f72661270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:09.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.823+0000 7f14efd9c080 1 --2- 172.21.15.16:0/1953258413 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f726f3910 0x559f72661270 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f14dc001cd0 tx=0x7f14dc00b5d0 comp rx=0 tx=0).stop 2023-05-20T01:27:09.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.823+0000 7f14efd9c080 1 -- 172.21.15.16:0/1953258413 shutdown_connections 2023-05-20T01:27:09.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.823+0000 7f14efd9c080 1 --2- 172.21.15.16:0/1953258413 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559f7270e6b0 0x559f72709840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:09.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.823+0000 7f14efd9c080 1 --2- 172.21.15.16:0/1953258413 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f726f3910 0x559f72661270 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:09.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.823+0000 7f14efd9c080 1 --2- 172.21.15.16:0/1953258413 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559f7270dbd0 0x559f7270ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:09.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.823+0000 7f14efd9c080 1 -- 172.21.15.16:0/1953258413 >> 172.21.15.16:0/1953258413 conn(0x559f7267c820 msgr2=0x559f7270dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:27:09.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.823+0000 7f14efd9c080 1 -- 172.21.15.16:0/1953258413 shutdown_connections 2023-05-20T01:27:09.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.823+0000 7f14efd9c080 1 -- 172.21.15.16:0/1953258413 wait complete. 2023-05-20T01:27:09.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.824+0000 7f14efd9c080 1 Processor -- start 2023-05-20T01:27:09.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.824+0000 7f14efd9c080 1 -- start start 2023-05-20T01:27:09.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.824+0000 7f14efd9c080 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f726f3910 0x559f72664260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:09.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.824+0000 7f14efd9c080 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559f7270dbd0 0x559f726f2720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:09.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.824+0000 7f14efd9c080 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559f7270e6b0 0x559f727089a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:09.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.824+0000 7f14efd9c080 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f72663e40 con 0x559f726f3910 2023-05-20T01:27:09.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.824+0000 7f14efd9c080 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f72679460 con 0x559f7270dbd0 2023-05-20T01:27:09.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.824+0000 7f14efd9c080 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x559f72663780 con 0x559f7270e6b0 2023-05-20T01:27:09.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.825+0000 7f14e5e9c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f726f3910 0x559f72664260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:09.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.825+0000 7f14e5e9c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f726f3910 0x559f72664260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38570/0 (socket says 172.21.15.16:38570) 2023-05-20T01:27:09.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.825+0000 7f14e5e9c700 1 -- 172.21.15.16:0/372300084 learned_addr learned my addr 172.21.15.16:0/372300084 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:27:09.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.825+0000 7f14e569b700 1 --2- 172.21.15.16:0/372300084 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559f7270dbd0 0x559f726f2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:09.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.825+0000 7f14e669d700 1 --2- 172.21.15.16:0/372300084 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559f7270e6b0 0x559f727089a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:09.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.825+0000 7f14e5e9c700 1 -- 172.21.15.16:0/372300084 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559f7270e6b0 msgr2=0x559f727089a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:09.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.825+0000 7f14e5e9c700 1 --2- 172.21.15.16:0/372300084 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559f7270e6b0 0x559f727089a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:09.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.825+0000 7f14e5e9c700 1 -- 172.21.15.16:0/372300084 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559f7270dbd0 msgr2=0x559f726f2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:09.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.825+0000 7f14e5e9c700 1 --2- 172.21.15.16:0/372300084 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559f7270dbd0 0x559f726f2720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:09.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.825+0000 7f14e5e9c700 1 -- 172.21.15.16:0/372300084 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14d0001e10 con 0x559f726f3910 2023-05-20T01:27:09.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.825+0000 7f14e5e9c700 1 --2- 172.21.15.16:0/372300084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f726f3910 0x559f72664260 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x559f72798f70 tx=0x7f14dc000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:09.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.825+0000 7f14d6ffd700 1 -- 172.21.15.16:0/372300084 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14dc01e070 con 0x559f726f3910 2023-05-20T01:27:09.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.825+0000 7f14d6ffd700 1 -- 172.21.15.16:0/372300084 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14dc000fe0 con 0x559f726f3910 2023-05-20T01:27:09.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.825+0000 7f14d6ffd700 1 -- 172.21.15.16:0/372300084 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14dc023640 con 0x559f726f3910 2023-05-20T01:27:09.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.825+0000 7f14efd9c080 1 -- 172.21.15.16:0/372300084 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x559f72586a80 con 0x559f726f3910 2023-05-20T01:27:09.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.825+0000 7f14efd9c080 1 -- 172.21.15.16:0/372300084 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14d8005d20 con 0x559f726f3910 2023-05-20T01:27:09.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.827+0000 7f14d6ffd700 1 -- 172.21.15.16:0/372300084 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f14dc02c440 con 0x559f726f3910 2023-05-20T01:27:09.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.827+0000 7f14d6ffd700 1 --2- 172.21.15.16:0/372300084 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f14c8069600 0x7f14c806bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:09.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.827+0000 7f14d6ffd700 1 -- 172.21.15.16:0/372300084 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f14dc089ad0 con 0x559f726f3910 2023-05-20T01:27:09.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.827+0000 7f14e569b700 1 --2- 172.21.15.16:0/372300084 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f14c8069600 0x7f14c806bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:09.840 INFO:teuthology.orchestra.run.smithi016.stdout:Warning: using slow linear search 2023-05-20T01:27:09.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.827+0000 7f14efd9c080 1 --2- 172.21.15.16:0/372300084 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x559f72709d50 0x559f727a3c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:09.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.827+0000 7f14efd9c080 1 -- 172.21.15.16:0/372300084 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:1 15.0 15:00000000:%01:::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) v8 -- 0x559f727a4170 con 0x559f72709d50 2023-05-20T01:27:09.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.827+0000 7f14e669d700 1 --2- 172.21.15.16:0/372300084 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x559f72709d50 0x559f727a3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:09.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.828+0000 7f14e569b700 1 --2- 172.21.15.16:0/372300084 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f14c8069600 0x7f14c806bab0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x559f7270eac0 tx=0x7f14d800f000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:09.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.828+0000 7f14e669d700 1 --2- 172.21.15.16:0/372300084 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x559f72709d50 0x559f727a3c30 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:09.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.828+0000 7f14e669d700 1 -- 172.21.15.16:0/372300084 <== osd.2 v2:172.21.15.16:6816/293187030 1 ==== osd_op_reply(1 [pgnls start_epoch 116 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f14e000ab70 con 0x559f72709d50 2023-05-20T01:27:09.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.828+0000 7f14e669d700 1 --2- 172.21.15.16:0/372300084 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x7f14e00133b0 0x7f14e0013820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:09.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.828+0000 7f14e669d700 1 -- 172.21.15.16:0/372300084 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:2 15.10 15:08000000:%01:::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) v8 -- 0x559f727a4170 con 0x7f14e00133b0 2023-05-20T01:27:09.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.828+0000 7f14e5e9c700 1 --2- 172.21.15.16:0/372300084 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x7f14e00133b0 0x7f14e0013820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:09.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.829+0000 7f14e5e9c700 1 --2- 172.21.15.16:0/372300084 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x7f14e00133b0 0x7f14e0013820 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:09.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.831+0000 7f14e5e9c700 1 -- 172.21.15.16:0/372300084 <== osd.4 v2:172.21.15.132:6817/506317223 1 ==== osd_op_reply(2 [pgnls start_epoch 116 out=177b] v115'10 uv10 ondisk = 1) v8 ==== 144+0+177 (crc 0 0 0) 0x7f14dc014070 con 0x7f14e00133b0 2023-05-20T01:27:09.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.831+0000 7f14e5e9c700 1 -- 172.21.15.16:0/372300084 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:3 15.8 15:10000000:%01:::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) v8 -- 0x559f727a4170 con 0x559f72709d50 2023-05-20T01:27:09.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.831+0000 7f14e669d700 1 -- 172.21.15.16:0/372300084 <== osd.2 v2:172.21.15.16:6816/293187030 2 ==== osd_op_reply(3 [pgnls start_epoch 116 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f14e000ab70 con 0x559f72709d50 2023-05-20T01:27:09.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.831+0000 7f14e669d700 1 --2- 172.21.15.16:0/372300084 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x7f14e0011840 0x7f14e00102c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:09.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.831+0000 7f14e669d700 1 -- 172.21.15.16:0/372300084 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:4 15.18 15:18000000:%01:::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) v8 -- 0x559f727a4170 con 0x7f14e0011840 2023-05-20T01:27:09.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.831+0000 7f14e569b700 1 --2- 172.21.15.16:0/372300084 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x7f14e0011840 0x7f14e00102c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:09.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.832+0000 7f14e569b700 1 --2- 172.21.15.16:0/372300084 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x7f14e0011840 0x7f14e00102c0 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:09.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.832+0000 7f14e569b700 1 -- 172.21.15.16:0/372300084 <== osd.3 v2:172.21.15.16:6800/3669781890 1 ==== osd_op_reply(4 [pgnls start_epoch 116 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f14d8019040 con 0x7f14e0011840 2023-05-20T01:27:09.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.832+0000 7f14e569b700 1 --2- 172.21.15.16:0/372300084 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x7f14d8009040 0x7f14d801e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:09.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.832+0000 7f14e569b700 1 -- 172.21.15.16:0/372300084 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:5 15.4 15:20000000:%01:::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) v8 -- 0x559f727a4170 con 0x7f14d8009040 2023-05-20T01:27:09.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.832+0000 7f14e669d700 1 --2- 172.21.15.16:0/372300084 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x7f14d8009040 0x7f14d801e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:09.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.832+0000 7f14e669d700 1 --2- 172.21.15.16:0/372300084 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x7f14d8009040 0x7f14d801e400 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:09.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.835+0000 7f14e669d700 1 -- 172.21.15.16:0/372300084 <== osd.1 v2:172.21.15.16:6808/3860264220 1 ==== osd_op_reply(5 [pgnls start_epoch 116 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f14e000ab70 con 0x7f14d8009040 2023-05-20T01:27:09.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.835+0000 7f14e669d700 1 --2- 172.21.15.16:0/372300084 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x7f14e0024040 0x7f14e0024920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:09.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.835+0000 7f14e669d700 1 -- 172.21.15.16:0/372300084 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:6 15.14 15:28000000:%01:::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) v8 -- 0x559f727a4170 con 0x7f14e0024040 2023-05-20T01:27:09.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.835+0000 7f14e5e9c700 1 --2- 172.21.15.16:0/372300084 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x7f14e0024040 0x7f14e0024920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:09.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.835+0000 7f14e5e9c700 1 --2- 172.21.15.16:0/372300084 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x7f14e0024040 0x7f14e0024920 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:09.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.836+0000 7f14e5e9c700 1 -- 172.21.15.16:0/372300084 <== osd.6 v2:172.21.15.132:6801/727581123 1 ==== osd_op_reply(6 [pgnls start_epoch 116 out=149b] v115'40 uv40 ondisk = 1) v8 ==== 144+0+149 (crc 0 0 0) 0x7f14dc06f040 con 0x7f14e0024040 2023-05-20T01:27:09.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.836+0000 7f14e5e9c700 1 --2- 172.21.15.16:0/372300084 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x7f14dc02c070 0x7f14dc057500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:09.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.836+0000 7f14e5e9c700 1 -- 172.21.15.16:0/372300084 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:7 15.c 15:30000000:%01:::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) v8 -- 0x559f727a4170 con 0x7f14dc02c070 2023-05-20T01:27:09.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.836+0000 7f14e569b700 1 --2- 172.21.15.16:0/372300084 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x7f14dc02c070 0x7f14dc057500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:09.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.837+0000 7f14e569b700 1 --2- 172.21.15.16:0/372300084 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x7f14dc02c070 0x7f14dc057500 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:09.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.837+0000 7f14e569b700 1 -- 172.21.15.16:0/372300084 <== osd.5 v2:172.21.15.132:6816/1844689995 1 ==== osd_op_reply(7 [pgnls start_epoch 116 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f14d8019040 con 0x7f14dc02c070 2023-05-20T01:27:09.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.837+0000 7f14e569b700 1 -- 172.21.15.16:0/372300084 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:8 15.1c 15:38000000:%01:::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) v8 -- 0x559f727a4170 con 0x7f14d8009040 2023-05-20T01:27:09.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.837+0000 7f14e669d700 1 -- 172.21.15.16:0/372300084 <== osd.1 v2:172.21.15.16:6808/3860264220 2 ==== osd_op_reply(8 [pgnls start_epoch 116 out=73b] v111'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x7f14e000ab70 con 0x7f14d8009040 2023-05-20T01:27:09.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.837+0000 7f14e669d700 1 -- 172.21.15.16:0/372300084 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:9 15.2 15:40000000:%01:::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) v8 -- 0x559f727a4170 con 0x559f72709d50 2023-05-20T01:27:09.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.837+0000 7f14e669d700 1 -- 172.21.15.16:0/372300084 <== osd.2 v2:172.21.15.16:6816/293187030 3 ==== osd_op_reply(9 [pgnls start_epoch 116 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f14e000ab70 con 0x559f72709d50 2023-05-20T01:27:09.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.837+0000 7f14e669d700 1 -- 172.21.15.16:0/372300084 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:10 15.12 15:48000000:%01:::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) v8 -- 0x559f727a4170 con 0x7f14e0011840 2023-05-20T01:27:09.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.838+0000 7f14e569b700 1 -- 172.21.15.16:0/372300084 <== osd.3 v2:172.21.15.16:6800/3669781890 2 ==== osd_op_reply(10 [pgnls start_epoch 116 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f14d8019040 con 0x7f14e0011840 2023-05-20T01:27:09.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.838+0000 7f14e569b700 1 -- 172.21.15.16:0/372300084 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:11 15.a 15:50000000:%01:::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) v8 -- 0x559f727a4170 con 0x559f72709d50 2023-05-20T01:27:09.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.838+0000 7f14e669d700 1 -- 172.21.15.16:0/372300084 <== osd.2 v2:172.21.15.16:6816/293187030 4 ==== osd_op_reply(11 [pgnls start_epoch 116 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f14e000ab70 con 0x559f72709d50 2023-05-20T01:27:09.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.838+0000 7f14e669d700 1 -- 172.21.15.16:0/372300084 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:12 15.1a 15:58000000:%01:::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) v8 -- 0x559f727a4170 con 0x7f14dc02c070 2023-05-20T01:27:09.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.838+0000 7f14e569b700 1 -- 172.21.15.16:0/372300084 <== osd.5 v2:172.21.15.132:6816/1844689995 2 ==== osd_op_reply(12 [pgnls start_epoch 116 out=49b] v111'1 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f14d8019040 con 0x7f14dc02c070 2023-05-20T01:27:09.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.838+0000 7f14e569b700 1 --2- 172.21.15.16:0/372300084 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x7f14d8038910 0x7f14d80283f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:09.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.838+0000 7f14e569b700 1 -- 172.21.15.16:0/372300084 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:13 15.6 15:60000000:%01:::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) v8 -- 0x559f727a4170 con 0x7f14d8038910 2023-05-20T01:27:09.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.838+0000 7f14e669d700 1 --2- 172.21.15.16:0/372300084 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x7f14d8038910 0x7f14d80283f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:09.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.839+0000 7f14e669d700 1 --2- 172.21.15.16:0/372300084 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x7f14d8038910 0x7f14d80283f0 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:09.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.841+0000 7f14e669d700 1 -- 172.21.15.16:0/372300084 <== osd.0 v2:172.21.15.16:6824/3923634179 1 ==== osd_op_reply(13 [pgnls start_epoch 116 out=77b] v115'6 uv6 ondisk = 1) v8 ==== 144+0+77 (crc 0 0 0) 0x7f14e000ab70 con 0x7f14d8038910 2023-05-20T01:27:09.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.841+0000 7f14e669d700 1 -- 172.21.15.16:0/372300084 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:14 15.16 15:68000000:%01:::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) v8 -- 0x559f727a4170 con 0x7f14d8038910 2023-05-20T01:27:09.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.841+0000 7f14e669d700 1 -- 172.21.15.16:0/372300084 <== osd.0 v2:172.21.15.16:6824/3923634179 2 ==== osd_op_reply(14 [pgnls start_epoch 116 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f14e000ab70 con 0x7f14d8038910 2023-05-20T01:27:09.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.841+0000 7f14e669d700 1 -- 172.21.15.16:0/372300084 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:15 15.e 15:70000000:%01:::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) v8 -- 0x559f727a4170 con 0x559f72709d50 2023-05-20T01:27:09.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.841+0000 7f14e669d700 1 -- 172.21.15.16:0/372300084 <== osd.2 v2:172.21.15.16:6816/293187030 5 ==== osd_op_reply(15 [pgnls start_epoch 116 out=73b] v111'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x7f14e000ab70 con 0x559f72709d50 2023-05-20T01:27:09.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.842+0000 7f14e669d700 1 -- 172.21.15.16:0/372300084 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:16 15.1e 15:78000000:%01:::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) v8 -- 0x559f727a4170 con 0x559f72709d50 2023-05-20T01:27:09.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.842+0000 7f14e669d700 1 -- 172.21.15.16:0/372300084 <== osd.2 v2:172.21.15.16:6816/293187030 6 ==== osd_op_reply(16 [pgnls start_epoch 116 out=49b] v111'1 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f14e000ab70 con 0x559f72709d50 2023-05-20T01:27:09.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.842+0000 7f14e669d700 1 -- 172.21.15.16:0/372300084 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:17 15.1 15:80000000:%01:::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) v8 -- 0x559f727a4170 con 0x7f14e0024040 2023-05-20T01:27:09.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.842+0000 7f14e5e9c700 1 -- 172.21.15.16:0/372300084 <== osd.6 v2:172.21.15.132:6801/727581123 2 ==== osd_op_reply(17 [pgnls start_epoch 116 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f14dc06f040 con 0x7f14e0024040 2023-05-20T01:27:09.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.842+0000 7f14e5e9c700 1 -- 172.21.15.16:0/372300084 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:18 15.11 15:88000000:%01:::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) v8 -- 0x559f727a4170 con 0x559f72709d50 2023-05-20T01:27:09.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.842+0000 7f14e669d700 1 -- 172.21.15.16:0/372300084 <== osd.2 v2:172.21.15.16:6816/293187030 7 ==== osd_op_reply(18 [pgnls start_epoch 116 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f14e000ab70 con 0x559f72709d50 2023-05-20T01:27:09.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.842+0000 7f14e669d700 1 -- 172.21.15.16:0/372300084 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:19 15.9 15:90000000:%01:::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) v8 -- 0x559f727a4170 con 0x7f14e0024040 2023-05-20T01:27:09.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.843+0000 7f14e5e9c700 1 -- 172.21.15.16:0/372300084 <== osd.6 v2:172.21.15.132:6801/727581123 3 ==== osd_op_reply(19 [pgnls start_epoch 116 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f14dc06f040 con 0x7f14e0024040 2023-05-20T01:27:09.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.843+0000 7f14e5e9c700 1 -- 172.21.15.16:0/372300084 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:20 15.19 15:98000000:%01:::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) v8 -- 0x559f727a4170 con 0x7f14e00133b0 2023-05-20T01:27:09.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.843+0000 7f14e5e9c700 1 -- 172.21.15.16:0/372300084 <== osd.4 v2:172.21.15.132:6817/506317223 2 ==== osd_op_reply(20 [pgnls start_epoch 116 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f14dc06f040 con 0x7f14e00133b0 2023-05-20T01:27:09.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.843+0000 7f14e5e9c700 1 -- 172.21.15.16:0/372300084 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:21 15.5 15:a0000000:%01:::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) v8 -- 0x559f727a4170 con 0x7f14e0024040 2023-05-20T01:27:09.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.843+0000 7f14e5e9c700 1 -- 172.21.15.16:0/372300084 <== osd.6 v2:172.21.15.132:6801/727581123 4 ==== osd_op_reply(21 [pgnls start_epoch 116 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f14dc06f040 con 0x7f14e0024040 2023-05-20T01:27:09.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.843+0000 7f14e5e9c700 1 --2- 172.21.15.16:0/372300084 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x7f14dc057a90 0x7f14dc0585a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:09.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.843+0000 7f14e5e9c700 1 -- 172.21.15.16:0/372300084 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:22 15.15 15:a8000000:%01:::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) v8 -- 0x559f727a4170 con 0x7f14dc057a90 2023-05-20T01:27:09.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.844+0000 7f14e5e9c700 1 --2- 172.21.15.16:0/372300084 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x7f14dc057a90 0x7f14dc0585a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:09.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.844+0000 7f14e5e9c700 1 --2- 172.21.15.16:0/372300084 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x7f14dc057a90 0x7f14dc0585a0 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:09.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.844+0000 7f14e5e9c700 1 -- 172.21.15.16:0/372300084 <== osd.7 v2:172.21.15.132:6800/466001149 1 ==== osd_op_reply(22 [pgnls start_epoch 116 out=97b] v111'3 uv3 ondisk = 1) v8 ==== 144+0+97 (crc 0 0 0) 0x7f14dc06f040 con 0x7f14dc057a90 2023-05-20T01:27:09.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.844+0000 7f14e5e9c700 1 -- 172.21.15.16:0/372300084 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:23 15.d 15:b0000000:%01:::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) v8 -- 0x559f727a4170 con 0x7f14dc057a90 2023-05-20T01:27:09.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.844+0000 7f14e5e9c700 1 -- 172.21.15.16:0/372300084 <== osd.7 v2:172.21.15.132:6800/466001149 2 ==== osd_op_reply(23 [pgnls start_epoch 116 out=74b] v111'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x7f14dc06f040 con 0x7f14dc057a90 2023-05-20T01:27:09.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.845+0000 7f14e5e9c700 1 -- 172.21.15.16:0/372300084 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:24 15.1d 15:b8000000:%01:::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) v8 -- 0x559f727a4170 con 0x7f14e00133b0 2023-05-20T01:27:09.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.845+0000 7f14e5e9c700 1 -- 172.21.15.16:0/372300084 <== osd.4 v2:172.21.15.132:6817/506317223 3 ==== osd_op_reply(24 [pgnls start_epoch 116 out=73b] v111'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x7f14dc06f040 con 0x7f14e00133b0 2023-05-20T01:27:09.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.845+0000 7f14e5e9c700 1 -- 172.21.15.16:0/372300084 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:25 15.3 15:c0000000:%01:::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) v8 -- 0x559f727a4170 con 0x7f14e00133b0 2023-05-20T01:27:09.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.845+0000 7f14e5e9c700 1 -- 172.21.15.16:0/372300084 <== osd.4 v2:172.21.15.132:6817/506317223 4 ==== osd_op_reply(25 [pgnls start_epoch 116 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f14dc06f040 con 0x7f14e00133b0 2023-05-20T01:27:09.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.845+0000 7f14e5e9c700 1 -- 172.21.15.16:0/372300084 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:26 15.13 15:c8000000:%01:::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) v8 -- 0x559f727a4170 con 0x7f14d8009040 2023-05-20T01:27:09.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.846+0000 7f14e669d700 1 -- 172.21.15.16:0/372300084 <== osd.1 v2:172.21.15.16:6808/3860264220 3 ==== osd_op_reply(26 [pgnls start_epoch 116 out=98b] v115'3 uv3 ondisk = 1) v8 ==== 144+0+98 (crc 0 0 0) 0x7f14e000ab70 con 0x7f14d8009040 2023-05-20T01:27:09.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.846+0000 7f14e669d700 1 -- 172.21.15.16:0/372300084 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:27 15.b 15:d0000000:%01:::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) v8 -- 0x559f727a4170 con 0x559f72709d50 2023-05-20T01:27:09.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.846+0000 7f14e669d700 1 -- 172.21.15.16:0/372300084 <== osd.2 v2:172.21.15.16:6816/293187030 8 ==== osd_op_reply(27 [pgnls start_epoch 116 out=76b] v115'3 uv3 ondisk = 1) v8 ==== 144+0+76 (crc 0 0 0) 0x7f14e000ab70 con 0x559f72709d50 2023-05-20T01:27:09.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.846+0000 7f14e669d700 1 -- 172.21.15.16:0/372300084 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:28 15.1b 15:d8000000:%01:::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) v8 -- 0x559f727a4170 con 0x7f14dc057a90 2023-05-20T01:27:09.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.846+0000 7f14e5e9c700 1 -- 172.21.15.16:0/372300084 <== osd.7 v2:172.21.15.132:6800/466001149 3 ==== osd_op_reply(28 [pgnls start_epoch 116 out=49b] v111'1 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f14dc06f040 con 0x7f14dc057a90 2023-05-20T01:27:09.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.846+0000 7f14e5e9c700 1 -- 172.21.15.16:0/372300084 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:29 15.7 15:e0000000:%01:::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) v8 -- 0x559f727a4170 con 0x7f14d8009040 2023-05-20T01:27:09.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.847+0000 7f14e669d700 1 -- 172.21.15.16:0/372300084 <== osd.1 v2:172.21.15.16:6808/3860264220 4 ==== osd_op_reply(29 [pgnls start_epoch 116 out=73b] v111'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x7f14e000ab70 con 0x7f14d8009040 2023-05-20T01:27:09.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.847+0000 7f14e669d700 1 -- 172.21.15.16:0/372300084 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:30 15.17 15:e8000000:%01:::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) v8 -- 0x559f727a4170 con 0x7f14e00133b0 2023-05-20T01:27:09.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.847+0000 7f14e5e9c700 1 -- 172.21.15.16:0/372300084 <== osd.4 v2:172.21.15.132:6817/506317223 5 ==== osd_op_reply(30 [pgnls start_epoch 116 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f14dc06f040 con 0x7f14e00133b0 2023-05-20T01:27:09.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.847+0000 7f14e5e9c700 1 -- 172.21.15.16:0/372300084 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:31 15.f 15:f0000000:%01:::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) v8 -- 0x559f727a4170 con 0x7f14e0024040 2023-05-20T01:27:09.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.847+0000 7f14e5e9c700 1 -- 172.21.15.16:0/372300084 <== osd.6 v2:172.21.15.132:6801/727581123 5 ==== osd_op_reply(31 [pgnls start_epoch 116 out=73b] v111'1 uv1 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x7f14dc06f040 con 0x7f14e0024040 2023-05-20T01:27:09.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.847+0000 7f14e5e9c700 1 -- 172.21.15.16:0/372300084 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:32 15.1f 15:f8000000:%01:::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) v8 -- 0x559f727a4170 con 0x7f14e0011840 2023-05-20T01:27:09.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.848+0000 7f14e569b700 1 -- 172.21.15.16:0/372300084 <== osd.3 v2:172.21.15.16:6800/3669781890 3 ==== osd_op_reply(32 [pgnls start_epoch 116 out=101b] v115'6 uv6 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x7f14d8019040 con 0x7f14e0011840 2023-05-20T01:27:09.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.848+0000 7f14efd9c080 1 -- 172.21.15.16:0/372300084 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:33 15.10 15:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e116) v8 -- 0x559f727a4580 con 0x7f14e00133b0 2023-05-20T01:27:09.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.848+0000 7f14efd9c080 1 -- 172.21.15.16:0/372300084 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:34 15.10 15:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e116) v8 -- 0x559f727ad630 con 0x7f14e00133b0 2023-05-20T01:27:09.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.848+0000 7f14efd9c080 1 -- 172.21.15.16:0/372300084 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:35 15.10 15:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e116) v8 -- 0x559f727b00c0 con 0x7f14e00133b0 2023-05-20T01:27:09.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.848+0000 7f14efd9c080 1 -- 172.21.15.16:0/372300084 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:36 15.10 15:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e116) v8 -- 0x559f727b08c0 con 0x7f14e00133b0 2023-05-20T01:27:09.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.848+0000 7f14efd9c080 1 -- 172.21.15.16:0/372300084 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:37 15.10 15:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e116) v8 -- 0x559f727b10c0 con 0x7f14e00133b0 2023-05-20T01:27:09.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.848+0000 7f14efd9c080 1 -- 172.21.15.16:0/372300084 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:38 15.14 15:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e116) v8 -- 0x559f727b18c0 con 0x7f14e0024040 2023-05-20T01:27:09.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.848+0000 7f14efd9c080 1 -- 172.21.15.16:0/372300084 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:39 15.14 15:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e116) v8 -- 0x559f727b20c0 con 0x7f14e0024040 2023-05-20T01:27:09.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.848+0000 7f14efd9c080 1 -- 172.21.15.16:0/372300084 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:40 15.14 15:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e116) v8 -- 0x559f727b28c0 con 0x7f14e0024040 2023-05-20T01:27:09.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.848+0000 7f14efd9c080 1 -- 172.21.15.16:0/372300084 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:41 15.14 15:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e116) v8 -- 0x559f727b30c0 con 0x7f14e0024040 2023-05-20T01:27:09.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.848+0000 7f14efd9c080 1 -- 172.21.15.16:0/372300084 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:42 15.1c 15:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e116) v8 -- 0x559f727b28c0 con 0x7f14d8009040 2023-05-20T01:27:09.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.848+0000 7f14efd9c080 1 -- 172.21.15.16:0/372300084 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:43 15.6 15:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e116) v8 -- 0x559f727b3d60 con 0x7f14d8038910 2023-05-20T01:27:09.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.848+0000 7f14efd9c080 1 -- 172.21.15.16:0/372300084 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:44 15.e 15:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e116) v8 -- 0x559f727b4660 con 0x559f72709d50 2023-05-20T01:27:09.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.848+0000 7f14efd9c080 1 -- 172.21.15.16:0/372300084 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:45 15.15 15:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e116) v8 -- 0x559f727b5150 con 0x7f14dc057a90 2023-05-20T01:27:09.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.848+0000 7f14efd9c080 1 -- 172.21.15.16:0/372300084 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:46 15.15 15:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e116) v8 -- 0x559f727b5c40 con 0x7f14dc057a90 2023-05-20T01:27:09.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.848+0000 7f14efd9c080 1 -- 172.21.15.16:0/372300084 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:47 15.d 15:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e116) v8 -- 0x559f727b6710 con 0x7f14dc057a90 2023-05-20T01:27:09.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.848+0000 7f14efd9c080 1 -- 172.21.15.16:0/372300084 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:48 15.1d 15:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e116) v8 -- 0x559f727b7200 con 0x7f14e00133b0 2023-05-20T01:27:09.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.849+0000 7f14e669d700 1 -- 172.21.15.16:0/372300084 <== osd.2 v2:172.21.15.16:6816/293187030 9 ==== osd_op_reply(44 608.00000000 [delete] v116'3 uv3 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f14e000ab70 con 0x559f72709d50 2023-05-20T01:27:09.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.849+0000 7f14efd9c080 1 -- 172.21.15.16:0/372300084 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:49 15.13 15:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e116) v8 -- 0x559f727b6710 con 0x7f14d8009040 2023-05-20T01:27:09.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.850+0000 7f14e5e9c700 1 -- 172.21.15.16:0/372300084 <== osd.6 v2:172.21.15.132:6801/727581123 6 ==== osd_op_reply(38 1.00000000.inode [delete] v116'41 uv41 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f14dc06f040 con 0x7f14e0024040 2023-05-20T01:27:09.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.850+0000 7f14efd9c080 1 -- 172.21.15.16:0/372300084 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:50 15.13 15:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e116) v8 -- 0x559f727b5c40 con 0x7f14d8009040 2023-05-20T01:27:09.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.850+0000 7f14e5e9c700 1 -- 172.21.15.16:0/372300084 <== osd.7 v2:172.21.15.132:6800/466001149 4 ==== osd_op_reply(45 604.00000000 [delete] v116'4 uv4 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f14dc06f040 con 0x7f14dc057a90 2023-05-20T01:27:09.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.850+0000 7f14efd9c080 1 -- 172.21.15.16:0/372300084 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:51 15.b 15:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e116) v8 -- 0x559f727b7200 con 0x559f72709d50 2023-05-20T01:27:09.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.851+0000 7f14e5e9c700 1 -- 172.21.15.16:0/372300084 <== osd.6 v2:172.21.15.132:6801/727581123 7 ==== osd_op_reply(39 200.00000000 [delete] v116'42 uv42 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f14dc06f040 con 0x7f14e0024040 2023-05-20T01:27:09.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.851+0000 7f14efd9c080 1 -- 172.21.15.16:0/372300084 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:52 15.7 15:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e116) v8 -- 0x559f727b7560 con 0x7f14d8009040 2023-05-20T01:27:09.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.851+0000 7f14e669d700 1 -- 172.21.15.16:0/372300084 <== osd.1 v2:172.21.15.16:6808/3860264220 5 ==== osd_op_reply(49 mds0_inotable [delete] v116'4 uv4 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f14e000ab70 con 0x7f14d8009040 2023-05-20T01:27:09.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.851+0000 7f14efd9c080 1 -- 172.21.15.16:0/372300084 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:53 15.f 15:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e116) v8 -- 0x559f727b7560 con 0x7f14e0024040 2023-05-20T01:27:09.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.852+0000 7f14e669d700 1 -- 172.21.15.16:0/372300084 <== osd.1 v2:172.21.15.16:6808/3860264220 6 ==== osd_op_reply(50 100.00000000 [delete] v116'5 uv5 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f14e000ab70 con 0x7f14d8009040 2023-05-20T01:27:09.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.852+0000 7f14e669d700 1 -- 172.21.15.16:0/372300084 <== osd.2 v2:172.21.15.16:6816/293187030 10 ==== osd_op_reply(51 mds0_sessionmap [delete] v116'4 uv4 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f14e000ab70 con 0x559f72709d50 2023-05-20T01:27:09.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.852+0000 7f14efd9c080 1 -- 172.21.15.16:0/372300084 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:54 15.1f 15:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e116) v8 -- 0x559f727b7560 con 0x7f14e0011840 2023-05-20T01:27:09.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.852+0000 7f14efd9c080 1 -- 172.21.15.16:0/372300084 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:55 15.1f 15:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e116) v8 -- 0x559f727b4c80 con 0x7f14e0011840 2023-05-20T01:27:09.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.852+0000 7f14e669d700 1 -- 172.21.15.16:0/372300084 <== osd.1 v2:172.21.15.16:6808/3860264220 7 ==== osd_op_reply(52 609.00000000 [delete] v116'3 uv3 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f14e000ab70 con 0x7f14d8009040 2023-05-20T01:27:09.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.853+0000 7f14e5e9c700 1 -- 172.21.15.16:0/372300084 <== osd.7 v2:172.21.15.132:6800/466001149 5 ==== osd_op_reply(46 500.00000000 [delete] v116'5 uv5 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f14dc06f040 con 0x7f14dc057a90 2023-05-20T01:27:09.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.853+0000 7f14e5e9c700 1 -- 172.21.15.16:0/372300084 <== osd.6 v2:172.21.15.132:6801/727581123 8 ==== osd_op_reply(40 200.00000001 [delete] v116'43 uv43 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f14dc06f040 con 0x7f14e0024040 2023-05-20T01:27:09.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.853+0000 7f14e669d700 1 -- 172.21.15.16:0/372300084 <== osd.0 v2:172.21.15.16:6824/3923634179 3 ==== osd_op_reply(43 mds0_openfiles.0 [delete] v116'7 uv7 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f14e000ab70 con 0x7f14d8038910 2023-05-20T01:27:09.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.853+0000 7f14e669d700 1 -- 172.21.15.16:0/372300084 <== osd.1 v2:172.21.15.16:6808/3860264220 8 ==== osd_op_reply(42 607.00000000 [delete] v116'3 uv3 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f14e000ab70 con 0x7f14d8009040 2023-05-20T01:27:09.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.853+0000 7f14e5e9c700 1 -- 172.21.15.16:0/372300084 <== osd.7 v2:172.21.15.132:6800/466001149 6 ==== osd_op_reply(47 mds_snaptable [delete] v116'2 uv2 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f14dc06f040 con 0x7f14dc057a90 2023-05-20T01:27:09.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.855+0000 7f14e5e9c700 1 -- 172.21.15.16:0/372300084 <== osd.6 v2:172.21.15.132:6801/727581123 9 ==== osd_op_reply(41 606.00000000 [delete] v116'44 uv44 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f14dc06f040 con 0x7f14e0024040 2023-05-20T01:27:09.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.856+0000 7f14e5e9c700 1 -- 172.21.15.16:0/372300084 <== osd.4 v2:172.21.15.132:6817/506317223 6 ==== osd_op_reply(33 601.00000000 [delete] v116'11 uv11 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f14dc06f040 con 0x7f14e00133b0 2023-05-20T01:27:09.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.856+0000 7f14e5e9c700 1 -- 172.21.15.16:0/372300084 <== osd.4 v2:172.21.15.132:6817/506317223 7 ==== osd_op_reply(48 605.00000000 [delete] v116'3 uv3 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f14dc06f040 con 0x7f14e00133b0 2023-05-20T01:27:09.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.856+0000 7f14e5e9c700 1 -- 172.21.15.16:0/372300084 <== osd.6 v2:172.21.15.132:6801/727581123 10 ==== osd_op_reply(53 400.00000000 [delete] v116'2 uv2 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f14dc06f040 con 0x7f14e0024040 2023-05-20T01:27:09.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.856+0000 7f14e5e9c700 1 -- 172.21.15.16:0/372300084 <== osd.4 v2:172.21.15.132:6817/506317223 8 ==== osd_op_reply(34 602.00000000 [delete] v116'12 uv12 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f14dc06f040 con 0x7f14e00133b0 2023-05-20T01:27:09.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.856+0000 7f14e5e9c700 1 -- 172.21.15.16:0/372300084 <== osd.4 v2:172.21.15.132:6817/506317223 9 ==== osd_op_reply(35 10000000000.00000000 [delete] v116'13 uv13 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f14dc06f040 con 0x7f14e00133b0 2023-05-20T01:27:09.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.856+0000 7f14e5e9c700 1 -- 172.21.15.16:0/372300084 <== osd.4 v2:172.21.15.132:6817/506317223 10 ==== osd_op_reply(36 600.00000000 [delete] v116'14 uv14 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f14dc06f040 con 0x7f14e00133b0 2023-05-20T01:27:09.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.856+0000 7f14e5e9c700 1 -- 172.21.15.16:0/372300084 <== osd.4 v2:172.21.15.132:6817/506317223 11 ==== osd_op_reply(37 603.00000000 [delete] v116'15 uv15 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f14dc06f040 con 0x7f14e00133b0 2023-05-20T01:27:09.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.856+0000 7f14e569b700 1 -- 172.21.15.16:0/372300084 <== osd.3 v2:172.21.15.16:6800/3669781890 4 ==== osd_op_reply(54 100.00000000.inode [delete] v116'7 uv7 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f14d8019040 con 0x7f14e0011840 2023-05-20T01:27:09.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.856+0000 7f14e569b700 1 -- 172.21.15.16:0/372300084 <== osd.3 v2:172.21.15.16:6800/3669781890 5 ==== osd_op_reply(55 1.00000000 [delete] v116'8 uv8 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f14d8019040 con 0x7f14e0011840 2023-05-20T01:27:09.867 INFO:teuthology.orchestra.run.smithi016.stdout:Removed 23 objects 2023-05-20T01:27:09.867 INFO:teuthology.orchestra.run.smithi016.stdout:successfully purged pool cephfs_metadata 2023-05-20T01:27:09.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.856+0000 7f14efd9c080 1 -- 172.21.15.16:0/372300084 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x7f14d8038910 msgr2=0x7f14d80283f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:09.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.856+0000 7f14efd9c080 1 --2- 172.21.15.16:0/372300084 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x7f14d8038910 0x7f14d80283f0 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:09.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.856+0000 7f14efd9c080 1 -- 172.21.15.16:0/372300084 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x7f14d8009040 msgr2=0x7f14d801e400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:09.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.856+0000 7f14efd9c080 1 --2- 172.21.15.16:0/372300084 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x7f14d8009040 0x7f14d801e400 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:09.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.856+0000 7f14efd9c080 1 -- 172.21.15.16:0/372300084 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x559f72709d50 msgr2=0x559f727a3c30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:09.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.856+0000 7f14efd9c080 1 --2- 172.21.15.16:0/372300084 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x559f72709d50 0x559f727a3c30 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:09.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.856+0000 7f14efd9c080 1 -- 172.21.15.16:0/372300084 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x7f14e0011840 msgr2=0x7f14e00102c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:09.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.856+0000 7f14efd9c080 1 --2- 172.21.15.16:0/372300084 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x7f14e0011840 0x7f14e00102c0 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:09.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.856+0000 7f14efd9c080 1 -- 172.21.15.16:0/372300084 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x7f14e00133b0 msgr2=0x7f14e0013820 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:09.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.856+0000 7f14efd9c080 1 --2- 172.21.15.16:0/372300084 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x7f14e00133b0 0x7f14e0013820 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:09.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.856+0000 7f14efd9c080 1 -- 172.21.15.16:0/372300084 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x7f14dc02c070 msgr2=0x7f14dc057500 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:09.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.856+0000 7f14efd9c080 1 --2- 172.21.15.16:0/372300084 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x7f14dc02c070 0x7f14dc057500 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:09.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.856+0000 7f14efd9c080 1 -- 172.21.15.16:0/372300084 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x7f14e0024040 msgr2=0x7f14e0024920 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:09.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.856+0000 7f14efd9c080 1 --2- 172.21.15.16:0/372300084 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x7f14e0024040 0x7f14e0024920 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:09.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.856+0000 7f14efd9c080 1 -- 172.21.15.16:0/372300084 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x7f14dc057a90 msgr2=0x7f14dc0585a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:09.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.856+0000 7f14efd9c080 1 --2- 172.21.15.16:0/372300084 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x7f14dc057a90 0x7f14dc0585a0 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:09.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.856+0000 7f14e669d700 1 -- 172.21.15.16:0/372300084 reap_dead start 2023-05-20T01:27:09.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.856+0000 7f14e669d700 1 -- 172.21.15.16:0/372300084 reap_dead start 2023-05-20T01:27:09.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.856+0000 7f14e669d700 1 -- 172.21.15.16:0/372300084 reap_dead start 2023-05-20T01:27:09.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.856+0000 7f14efd9c080 1 -- 172.21.15.16:0/372300084 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f14c8069600 msgr2=0x7f14c806bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:09.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.856+0000 7f14efd9c080 1 --2- 172.21.15.16:0/372300084 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f14c8069600 0x7f14c806bab0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x559f7270eac0 tx=0x7f14d800f000 comp rx=0 tx=0).stop 2023-05-20T01:27:09.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.856+0000 7f14efd9c080 1 -- 172.21.15.16:0/372300084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f726f3910 msgr2=0x559f72664260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:09.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.856+0000 7f14efd9c080 1 --2- 172.21.15.16:0/372300084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f726f3910 0x559f72664260 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x559f72798f70 tx=0x7f14dc000bc0 comp rx=0 tx=0).stop 2023-05-20T01:27:09.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.856+0000 7f14efd9c080 1 -- 172.21.15.16:0/372300084 shutdown_connections 2023-05-20T01:27:09.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.856+0000 7f14efd9c080 1 --2- 172.21.15.16:0/372300084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f726f3910 0x559f72664260 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:09.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.856+0000 7f14efd9c080 1 --2- 172.21.15.16:0/372300084 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f14c8069600 0x7f14c806bab0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:09.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.856+0000 7f14efd9c080 1 -- 172.21.15.16:0/372300084 >> 172.21.15.16:0/372300084 conn(0x559f7267c820 msgr2=0x559f7270bc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:27:09.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.857+0000 7f14efd9c080 1 -- 172.21.15.16:0/372300084 shutdown_connections 2023-05-20T01:27:09.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:09.857+0000 7f14efd9c080 1 -- 172.21.15.16:0/372300084 wait complete. 2023-05-20T01:27:09.873 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs reset cephfs --yes-i-really-mean-it 2023-05-20T01:27:10.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.031+0000 7f32a7e49700 1 Processor -- start 2023-05-20T01:27:10.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.031+0000 7f32a7e49700 1 -- start start 2023-05-20T01:27:10.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.032+0000 7f32a7e49700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f32a015cec0 0x7f32a0071bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:10.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.032+0000 7f32a7e49700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f32a015d290 0x7f32a0072130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:10.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.032+0000 7f32a7e49700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32a00cd630 con 0x7f32a00cf2c0 2023-05-20T01:27:10.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.032+0000 7f32a7e49700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32a00cca60 con 0x7f32a015cec0 2023-05-20T01:27:10.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.032+0000 7f32a7e49700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32a00cfdd0 con 0x7f32a015d290 2023-05-20T01:27:10.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.032+0000 7f32a5be5700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f32a015cec0 0x7f32a0071bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:10.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.032+0000 7f32a5be5700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f32a015cec0 0x7f32a0071bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38594/0 (socket says 172.21.15.16:38594) 2023-05-20T01:27:10.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.032+0000 7f32a5be5700 1 -- 172.21.15.16:0/2880395214 learned_addr learned my addr 172.21.15.16:0/2880395214 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:27:10.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.032+0000 7f32a53e4700 1 --2- 172.21.15.16:0/2880395214 >> v2:172.21.15.132:3300/0 conn(0x7f32a015d290 0x7f32a0072130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:10.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.032+0000 7f32a5be5700 1 -- 172.21.15.16:0/2880395214 >> v2:172.21.15.132:3300/0 conn(0x7f32a015d290 msgr2=0x7f32a0072130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:10.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.032+0000 7f32a5be5700 1 --2- 172.21.15.16:0/2880395214 >> v2:172.21.15.132:3300/0 conn(0x7f32a015d290 0x7f32a0072130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:10.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.033+0000 7f32a5be5700 1 -- 172.21.15.16:0/2880395214 >> v1:172.21.15.132:6789/0 conn(0x7f32a00cf2c0 legacy=0x7f32a00cf6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:10.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.033+0000 7f32a5be5700 1 -- 172.21.15.16:0/2880395214 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32a00ccd00 con 0x7f32a015cec0 2023-05-20T01:27:10.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.033+0000 7f32a53e4700 1 --2- 172.21.15.16:0/2880395214 >> v2:172.21.15.132:3300/0 conn(0x7f32a015d290 0x7f32a0072130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:27:10.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.033+0000 7f32a5be5700 1 --2- 172.21.15.16:0/2880395214 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32a015cec0 0x7f32a0071bf0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f32a00cc970 tx=0x7f3290009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ff473d6f3f135a3 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:10.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.033+0000 7f32a4be3700 1 -- 172.21.15.16:0/2880395214 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3290005e30 con 0x7f32a015cec0 2023-05-20T01:27:10.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.033+0000 7f32a4be3700 1 -- 172.21.15.16:0/2880395214 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3290004030 con 0x7f32a015cec0 2023-05-20T01:27:10.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.033+0000 7f32a4be3700 1 -- 172.21.15.16:0/2880395214 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32900043e0 con 0x7f32a015cec0 2023-05-20T01:27:10.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.033+0000 7f32a7e49700 1 -- 172.21.15.16:0/2880395214 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32a015cec0 msgr2=0x7f32a0071bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:10.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.033+0000 7f32a7e49700 1 --2- 172.21.15.16:0/2880395214 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32a015cec0 0x7f32a0071bf0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f32a00cc970 tx=0x7f3290009bd0 comp rx=0 tx=0).stop 2023-05-20T01:27:10.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.033+0000 7f32a7e49700 1 -- 172.21.15.16:0/2880395214 shutdown_connections 2023-05-20T01:27:10.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.033+0000 7f32a7e49700 1 --2- 172.21.15.16:0/2880395214 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32a015cec0 0x7f32a0071bf0 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:10.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.033+0000 7f32a7e49700 1 --2- 172.21.15.16:0/2880395214 >> v2:172.21.15.132:3300/0 conn(0x7f32a015d290 0x7f32a0072130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:10.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.033+0000 7f32a7e49700 1 -- 172.21.15.16:0/2880395214 >> 172.21.15.16:0/2880395214 conn(0x7f32a00cffd0 msgr2=0x7f32a00d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:27:10.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.034+0000 7f32a7e49700 1 -- 172.21.15.16:0/2880395214 shutdown_connections 2023-05-20T01:27:10.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.034+0000 7f32a7e49700 1 -- 172.21.15.16:0/2880395214 wait complete. 2023-05-20T01:27:10.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.035+0000 7f32a7e49700 1 Processor -- start 2023-05-20T01:27:10.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.035+0000 7f32a7e49700 1 -- start start 2023-05-20T01:27:10.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.035+0000 7f32a7e49700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f32a00cf2c0 0x7f32a01590f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:10.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.035+0000 7f32a7e49700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32a015cec0 0x7f32a00c70a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:10.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.036+0000 7f32a7e49700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f32a015d290 0x7f32a015ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:10.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.036+0000 7f32a7e49700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32a01532e0 con 0x7f32a015cec0 2023-05-20T01:27:10.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.036+0000 7f32a7e49700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32a00c7a60 con 0x7f32a015d290 2023-05-20T01:27:10.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.036+0000 7f32a7e49700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32a00c7d40 con 0x7f32a00cf2c0 2023-05-20T01:27:10.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.036+0000 7f32a53e4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32a015cec0 0x7f32a00c70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:10.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.036+0000 7f32a53e4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32a015cec0 0x7f32a00c70a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38602/0 (socket says 172.21.15.16:38602) 2023-05-20T01:27:10.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.036+0000 7f32a53e4700 1 -- 172.21.15.16:0/1457409334 learned_addr learned my addr 172.21.15.16:0/1457409334 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:27:10.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.036+0000 7f32a5be5700 1 --2- 172.21.15.16:0/1457409334 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f32a00cf2c0 0x7f32a01590f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:10.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.036+0000 7f32a63e6700 1 --2- 172.21.15.16:0/1457409334 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f32a015d290 0x7f32a015ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:10.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.036+0000 7f32a53e4700 1 -- 172.21.15.16:0/1457409334 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f32a00cf2c0 msgr2=0x7f32a01590f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:10.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.036+0000 7f32a53e4700 1 --2- 172.21.15.16:0/1457409334 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f32a00cf2c0 0x7f32a01590f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:10.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.036+0000 7f32a53e4700 1 -- 172.21.15.16:0/1457409334 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f32a015d290 msgr2=0x7f32a015ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:10.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.036+0000 7f32a53e4700 1 --2- 172.21.15.16:0/1457409334 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f32a015d290 0x7f32a015ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:10.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.036+0000 7f32a53e4700 1 -- 172.21.15.16:0/1457409334 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3290009500 con 0x7f32a015cec0 2023-05-20T01:27:10.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.036+0000 7f32a53e4700 1 --2- 172.21.15.16:0/1457409334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32a015cec0 0x7f32a00c70a0 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f329c00e810 tx=0x7f329c00eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:10.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.036+0000 7f32967fc700 1 -- 172.21.15.16:0/1457409334 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f329c00c020 con 0x7f32a015cec0 2023-05-20T01:27:10.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.036+0000 7f32967fc700 1 -- 172.21.15.16:0/1457409334 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f329c004250 con 0x7f32a015cec0 2023-05-20T01:27:10.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.036+0000 7f32967fc700 1 -- 172.21.15.16:0/1457409334 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f329c0036a0 con 0x7f32a015cec0 2023-05-20T01:27:10.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.036+0000 7f32a7e49700 1 -- 172.21.15.16:0/1457409334 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32a00cd2c0 con 0x7f32a015cec0 2023-05-20T01:27:10.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.036+0000 7f32a7e49700 1 -- 172.21.15.16:0/1457409334 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32a00ccfe0 con 0x7f32a015cec0 2023-05-20T01:27:10.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.037+0000 7f32a7e49700 1 -- 172.21.15.16:0/1457409334 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32a0160890 con 0x7f32a015cec0 2023-05-20T01:27:10.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.038+0000 7f32967fc700 1 -- 172.21.15.16:0/1457409334 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f329c003800 con 0x7f32a015cec0 2023-05-20T01:27:10.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.038+0000 7f32967fc700 1 --2- 172.21.15.16:0/1457409334 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f328406ace0 0x7f328406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:10.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.038+0000 7f32a5be5700 1 --2- 172.21.15.16:0/1457409334 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f328406ace0 0x7f328406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:10.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.038+0000 7f32967fc700 1 -- 172.21.15.16:0/1457409334 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f329c0890f0 con 0x7f32a015cec0 2023-05-20T01:27:10.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.039+0000 7f32a5be5700 1 --2- 172.21.15.16:0/1457409334 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f328406ace0 0x7f328406d190 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f32a00cc970 tx=0x7f3290008070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:10.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.043+0000 7f32967fc700 1 -- 172.21.15.16:0/1457409334 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f329c053e70 con 0x7f32a015cec0 2023-05-20T01:27:10.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.290+0000 7f32a7e49700 1 -- 172.21.15.16:0/1457409334 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs reset", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f32a00747b0 con 0x7f32a015cec0 2023-05-20T01:27:10.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.535+0000 7f32967fc700 1 -- 172.21.15.16:0/1457409334 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs reset", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v81) v1 ==== 107+0+0 (secure 0 0 0) 0x7f329c057490 con 0x7f32a015cec0 2023-05-20T01:27:10.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.538+0000 7f32a7e49700 1 -- 172.21.15.16:0/1457409334 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f328406ace0 msgr2=0x7f328406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:10.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.538+0000 7f32a7e49700 1 --2- 172.21.15.16:0/1457409334 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f328406ace0 0x7f328406d190 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f32a00cc970 tx=0x7f3290008070 comp rx=0 tx=0).stop 2023-05-20T01:27:10.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.538+0000 7f32a7e49700 1 -- 172.21.15.16:0/1457409334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32a015cec0 msgr2=0x7f32a00c70a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:10.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.538+0000 7f32a7e49700 1 --2- 172.21.15.16:0/1457409334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32a015cec0 0x7f32a00c70a0 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f329c00e810 tx=0x7f329c00eb20 comp rx=0 tx=0).stop 2023-05-20T01:27:10.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.538+0000 7f32a7e49700 1 -- 172.21.15.16:0/1457409334 shutdown_connections 2023-05-20T01:27:10.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.538+0000 7f32a7e49700 1 --2- 172.21.15.16:0/1457409334 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f32a015d290 0x7f32a015ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:10.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.538+0000 7f32a7e49700 1 --2- 172.21.15.16:0/1457409334 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f328406ace0 0x7f328406d190 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:10.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.538+0000 7f32a7e49700 1 --2- 172.21.15.16:0/1457409334 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f32a00cf2c0 0x7f32a01590f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:10.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.538+0000 7f32a7e49700 1 --2- 172.21.15.16:0/1457409334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32a015cec0 0x7f32a00c70a0 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:10.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.538+0000 7f32a7e49700 1 -- 172.21.15.16:0/1457409334 >> 172.21.15.16:0/1457409334 conn(0x7f32a00cffd0 msgr2=0x7f32a015c490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:27:10.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.538+0000 7f32a7e49700 1 -- 172.21.15.16:0/1457409334 shutdown_connections 2023-05-20T01:27:10.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.538+0000 7f32a7e49700 1 -- 172.21.15.16:0/1457409334 wait complete. 2023-05-20T01:27:10.546 INFO:tasks.ceph.mds.a.smithi016.stderr:2023-05-20T01:27:10.542+0000 7fa36895c700 -1 mds.0.inotable: load_2 could not read table: -2 2023-05-20T01:27:10.546 INFO:tasks.ceph.mds.a.smithi016.stderr:2023-05-20T01:27:10.542+0000 7fa36895c700 -1 log_channel(cluster) log [ERR] : error reading table object 'mds0_inotable' -2 ((2) No such file or directory) 2023-05-20T01:27:10.559 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-05-20T01:27:10.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.684+0000 7f8c753da700 1 Processor -- start 2023-05-20T01:27:10.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.684+0000 7f8c753da700 1 -- start start 2023-05-20T01:27:10.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.685+0000 7f8c753da700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8c70072300 0x7f8c700cf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:10.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.685+0000 7f8c753da700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f8c7015d290 0x7f8c700cf800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:10.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.685+0000 7f8c753da700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c700cd630 con 0x7f8c7015cec0 2023-05-20T01:27:10.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.685+0000 7f8c753da700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c700cca60 con 0x7f8c70072300 2023-05-20T01:27:10.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.685+0000 7f8c753da700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c700ccd00 con 0x7f8c7015d290 2023-05-20T01:27:10.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.685+0000 7f8c6e7fc700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8c70072300 0x7f8c700cf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:10.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.685+0000 7f8c6e7fc700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8c70072300 0x7f8c700cf2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38622/0 (socket says 172.21.15.16:38622) 2023-05-20T01:27:10.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.685+0000 7f8c6e7fc700 1 -- 172.21.15.16:0/1995932024 learned_addr learned my addr 172.21.15.16:0/1995932024 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:27:10.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.685+0000 7f8c6f7fe700 1 --2- 172.21.15.16:0/1995932024 >> v2:172.21.15.132:3300/0 conn(0x7f8c7015d290 0x7f8c700cf800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:10.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.685+0000 7f8c6e7fc700 1 -- 172.21.15.16:0/1995932024 >> v2:172.21.15.132:3300/0 conn(0x7f8c7015d290 msgr2=0x7f8c700cf800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:10.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.685+0000 7f8c6e7fc700 1 --2- 172.21.15.16:0/1995932024 >> v2:172.21.15.132:3300/0 conn(0x7f8c7015d290 0x7f8c700cf800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:10.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.685+0000 7f8c6e7fc700 1 -- 172.21.15.16:0/1995932024 >> v1:172.21.15.132:6789/0 conn(0x7f8c7015cec0 legacy=0x7f8c70071bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:10.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.685+0000 7f8c6e7fc700 1 -- 172.21.15.16:0/1995932024 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c700ccfe0 con 0x7f8c70072300 2023-05-20T01:27:10.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.686+0000 7f8c6e7fc700 1 --2- 172.21.15.16:0/1995932024 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c70072300 0x7f8c700cf2c0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f8c70070df0 tx=0x7f8c60009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0c72aab046a98f3 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:10.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.686+0000 7f8c6dffb700 1 -- 172.21.15.16:0/1995932024 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c6000b870 con 0x7f8c70072300 2023-05-20T01:27:10.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.686+0000 7f8c6dffb700 1 -- 172.21.15.16:0/1995932024 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c6000b9d0 con 0x7f8c70072300 2023-05-20T01:27:10.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.686+0000 7f8c6dffb700 1 -- 172.21.15.16:0/1995932024 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c6000bd60 con 0x7f8c70072300 2023-05-20T01:27:10.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.686+0000 7f8c753da700 1 -- 172.21.15.16:0/1995932024 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c70072300 msgr2=0x7f8c700cf2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:10.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.686+0000 7f8c753da700 1 --2- 172.21.15.16:0/1995932024 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c70072300 0x7f8c700cf2c0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f8c70070df0 tx=0x7f8c60009bd0 comp rx=0 tx=0).stop 2023-05-20T01:27:10.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.686+0000 7f8c753da700 1 -- 172.21.15.16:0/1995932024 shutdown_connections 2023-05-20T01:27:10.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.686+0000 7f8c753da700 1 --2- 172.21.15.16:0/1995932024 >> v2:172.21.15.132:3300/0 conn(0x7f8c7015d290 0x7f8c700cf800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:10.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.686+0000 7f8c753da700 1 --2- 172.21.15.16:0/1995932024 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c70072300 0x7f8c700cf2c0 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:10.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.686+0000 7f8c753da700 1 -- 172.21.15.16:0/1995932024 >> 172.21.15.16:0/1995932024 conn(0x7f8c700cffd0 msgr2=0x7f8c700d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:27:10.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.686+0000 7f8c753da700 1 -- 172.21.15.16:0/1995932024 shutdown_connections 2023-05-20T01:27:10.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.686+0000 7f8c753da700 1 -- 172.21.15.16:0/1995932024 wait complete. 2023-05-20T01:27:10.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.688+0000 7f8c753da700 1 Processor -- start 2023-05-20T01:27:10.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.688+0000 7f8c753da700 1 -- start start 2023-05-20T01:27:10.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.689+0000 7f8c753da700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8c70072300 0x7f8c700ad4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:10.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.689+0000 7f8c753da700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c7015cec0 0x7f8c700c94a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:10.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.689+0000 7f8c753da700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c7015d290 0x7f8c700c99e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:10.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.689+0000 7f8c753da700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c700adaa0 con 0x7f8c7015cec0 2023-05-20T01:27:10.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.689+0000 7f8c753da700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c700c68c0 con 0x7f8c7015d290 2023-05-20T01:27:10.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.689+0000 7f8c753da700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c700c9f20 con 0x7f8c70072300 2023-05-20T01:27:10.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.689+0000 7f8c6e7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c7015cec0 0x7f8c700c94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:10.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.689+0000 7f8c6e7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c7015cec0 0x7f8c700c94a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38628/0 (socket says 172.21.15.16:38628) 2023-05-20T01:27:10.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.689+0000 7f8c6e7fc700 1 -- 172.21.15.16:0/1685060705 learned_addr learned my addr 172.21.15.16:0/1685060705 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:27:10.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.689+0000 7f8c6f7fe700 1 --2- 172.21.15.16:0/1685060705 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c7015d290 0x7f8c700c99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:10.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.689+0000 7f8c6effd700 1 --2- 172.21.15.16:0/1685060705 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8c70072300 0x7f8c700ad4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:10.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.689+0000 7f8c6e7fc700 1 -- 172.21.15.16:0/1685060705 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8c70072300 msgr2=0x7f8c700ad4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:10.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.689+0000 7f8c6e7fc700 1 --2- 172.21.15.16:0/1685060705 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8c70072300 0x7f8c700ad4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:10.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.689+0000 7f8c6e7fc700 1 -- 172.21.15.16:0/1685060705 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c7015d290 msgr2=0x7f8c700c99e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:10.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.689+0000 7f8c6e7fc700 1 --2- 172.21.15.16:0/1685060705 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c7015d290 0x7f8c700c99e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:10.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.689+0000 7f8c6e7fc700 1 -- 172.21.15.16:0/1685060705 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c60009550 con 0x7f8c7015cec0 2023-05-20T01:27:10.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.689+0000 7f8c6effd700 1 --2- 172.21.15.16:0/1685060705 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8c70072300 0x7f8c700ad4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:27:10.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.689+0000 7f8c6e7fc700 1 --2- 172.21.15.16:0/1685060705 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c7015cec0 0x7f8c700c94a0 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f8c60000c00 tx=0x7f8c60004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:10.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.689+0000 7f8c577fe700 1 -- 172.21.15.16:0/1685060705 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c60004540 con 0x7f8c7015cec0 2023-05-20T01:27:10.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.689+0000 7f8c577fe700 1 -- 172.21.15.16:0/1685060705 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c600036a0 con 0x7f8c7015cec0 2023-05-20T01:27:10.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.689+0000 7f8c577fe700 1 -- 172.21.15.16:0/1685060705 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c60003a30 con 0x7f8c7015cec0 2023-05-20T01:27:10.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.689+0000 7f8c753da700 1 -- 172.21.15.16:0/1685060705 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c700cfd40 con 0x7f8c7015cec0 2023-05-20T01:27:10.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.689+0000 7f8c753da700 1 -- 172.21.15.16:0/1685060705 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c700cd2c0 con 0x7f8c7015cec0 2023-05-20T01:27:10.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.690+0000 7f8c753da700 1 -- 172.21.15.16:0/1685060705 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c70160890 con 0x7f8c7015cec0 2023-05-20T01:27:10.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.691+0000 7f8c577fe700 1 -- 172.21.15.16:0/1685060705 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8c60021070 con 0x7f8c7015cec0 2023-05-20T01:27:10.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.691+0000 7f8c577fe700 1 --2- 172.21.15.16:0/1685060705 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f8c5c06ace0 0x7f8c5c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:10.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.691+0000 7f8c577fe700 1 -- 172.21.15.16:0/1685060705 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6958+0+0 (secure 0 0 0) 0x7f8c6008af70 con 0x7f8c7015cec0 2023-05-20T01:27:10.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.691+0000 7f8c6effd700 1 --2- 172.21.15.16:0/1685060705 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f8c5c06ace0 0x7f8c5c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:10.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.692+0000 7f8c6effd700 1 --2- 172.21.15.16:0/1685060705 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f8c5c06ace0 0x7f8c5c06d190 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f8c64000fb0 tx=0x7f8c64009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:10.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.696+0000 7f8c577fe700 1 -- 172.21.15.16:0/1685060705 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f8c600bd2d0 con 0x7f8c7015cec0 2023-05-20T01:27:10.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:10.944+0000 7f8c753da700 1 -- 172.21.15.16:0/1685060705 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f8c700c6690 con 0x7f8c7015cec0 2023-05-20T01:27:11.542 INFO:tasks.ceph.mon.a.smithi016.stderr:2023-05-20T01:27:11.538+0000 7f49eb8ae700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-05-20T01:27:11.543 INFO:tasks.ceph.mon.a.smithi016.stderr:2023-05-20T01:27:11.538+0000 7f49eb8ae700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2023-05-20T01:27:11.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:11.542+0000 7f8c577fe700 1 -- 172.21.15.16:0/1685060705 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v83) v1 ==== 162+0+0 (secure 0 0 0) 0x7f8c700c6690 con 0x7f8c7015cec0 2023-05-20T01:27:11.548 INFO:tasks.ceph.mds.a.smithi016.stderr: -31> 2023-05-20T01:27:10.542+0000 7fa36895c700 -1 mds.0.inotable: load_2 could not read table: -2 2023-05-20T01:27:11.548 INFO:tasks.ceph.mds.a.smithi016.stderr: -30> 2023-05-20T01:27:10.542+0000 7fa36895c700 -1 log_channel(cluster) log [ERR] : error reading table object 'mds0_inotable' -2 ((2) No such file or directory) 2023-05-20T01:27:11.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:11.545+0000 7f8c753da700 1 -- 172.21.15.16:0/1685060705 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f8c5c06ace0 msgr2=0x7f8c5c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:11.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:11.545+0000 7f8c753da700 1 --2- 172.21.15.16:0/1685060705 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f8c5c06ace0 0x7f8c5c06d190 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f8c64000fb0 tx=0x7f8c64009040 comp rx=0 tx=0).stop 2023-05-20T01:27:11.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:11.545+0000 7f8c753da700 1 -- 172.21.15.16:0/1685060705 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c7015cec0 msgr2=0x7f8c700c94a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:11.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:11.545+0000 7f8c753da700 1 --2- 172.21.15.16:0/1685060705 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c7015cec0 0x7f8c700c94a0 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f8c60000c00 tx=0x7f8c60004750 comp rx=0 tx=0).stop 2023-05-20T01:27:11.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:11.545+0000 7f8c753da700 1 -- 172.21.15.16:0/1685060705 shutdown_connections 2023-05-20T01:27:11.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:11.545+0000 7f8c753da700 1 --2- 172.21.15.16:0/1685060705 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c7015d290 0x7f8c700c99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:11.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:11.545+0000 7f8c753da700 1 --2- 172.21.15.16:0/1685060705 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f8c5c06ace0 0x7f8c5c06d190 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:11.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:11.545+0000 7f8c753da700 1 --2- 172.21.15.16:0/1685060705 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8c70072300 0x7f8c700ad4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:11.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:11.545+0000 7f8c753da700 1 --2- 172.21.15.16:0/1685060705 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c7015cec0 0x7f8c700c94a0 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:11.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:11.545+0000 7f8c753da700 1 -- 172.21.15.16:0/1685060705 >> 172.21.15.16:0/1685060705 conn(0x7f8c700cffd0 msgr2=0x7f8c7015fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:27:11.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:11.545+0000 7f8c753da700 1 -- 172.21.15.16:0/1685060705 shutdown_connections 2023-05-20T01:27:11.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:11.545+0000 7f8c753da700 1 -- 172.21.15.16:0/1685060705 wait complete. 2023-05-20T01:27:11.552 INFO:teuthology.orchestra.run.smithi016.stderr:cephfs marked joinable; MDS may join as newly active. 2023-05-20T01:27:11.565 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-20T01:27:11.574 INFO:tasks.ceph.mds.a.smithi016.stdout:starting mds.a at 2023-05-20T01:27:11.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:11.692+0000 7f4c70034700 1 Processor -- start 2023-05-20T01:27:11.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:11.693+0000 7f4c70034700 1 -- start start 2023-05-20T01:27:11.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:11.693+0000 7f4c70034700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f4c681599c0 0x7f4c680caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:11.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:11.693+0000 7f4c70034700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c68159d90 con 0x7f4c680caaa0 2023-05-20T01:27:11.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:11.693+0000 7f4c70034700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c68071ef0 con 0x7f4c680cf350 2023-05-20T01:27:11.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:11.693+0000 7f4c70034700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c680cdf20 con 0x7f4c681599c0 2023-05-20T01:27:11.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:11.693+0000 7f4c6d5cf700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f4c680caaa0 0x7f4c680710d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50174/0 (socket says 172.21.15.16:50174) 2023-05-20T01:27:11.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:11.693+0000 7f4c6d5cf700 1 -- 172.21.15.16:0/2391377409 learned_addr learned my addr 172.21.15.16:0/2391377409 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:27:11.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:11.694+0000 7f4c6cdce700 1 -- 172.21.15.16:0/2391377409 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3755726633 0 0) 0x7f4c5c001110 con 0x7f4c680caaa0 2023-05-20T01:27:11.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:11.694+0000 7f4c6ddd0700 1 --2- 172.21.15.16:0/2391377409 >> v2:172.21.15.132:3300/0 conn(0x7f4c681599c0 0x7f4c680caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:11.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:11.694+0000 7f4c6cdce700 1 -- 172.21.15.16:0/2391377409 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c540035b0 con 0x7f4c680caaa0 2023-05-20T01:27:11.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:11.694+0000 7f4c6cdce700 1 -- 172.21.15.16:0/2391377409 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3416742452 0 0) 0x7f4c540035b0 con 0x7f4c680caaa0 2023-05-20T01:27:11.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:11.694+0000 7f4c6cdce700 1 -- 172.21.15.16:0/2391377409 >> v2:172.21.15.132:3300/0 conn(0x7f4c681599c0 msgr2=0x7f4c680caf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:11.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:11.694+0000 7f4c6cdce700 1 --2- 172.21.15.16:0/2391377409 >> v2:172.21.15.132:3300/0 conn(0x7f4c681599c0 0x7f4c680caf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:11.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:11.694+0000 7f4c6cdce700 1 -- 172.21.15.16:0/2391377409 >> v1:172.21.15.132:6789/0 conn(0x7f4c680cf350 legacy=0x7f4c680717e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:11.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:11.694+0000 7f4c6cdce700 1 -- 172.21.15.16:0/2391377409 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c680cdb80 con 0x7f4c680caaa0 2023-05-20T01:27:11.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:11.694+0000 7f4c6cdce700 1 -- 172.21.15.16:0/2391377409 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7f4c5c001800 con 0x7f4c680caaa0 2023-05-20T01:27:11.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:11.694+0000 7f4c6cdce700 1 -- 172.21.15.16:0/2391377409 >> v1:172.21.15.16:6789/0 conn(0x7f4c680caaa0 legacy=0x7f4c680710d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:11.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:11.694+0000 7f4c6cdce700 1 --2- 172.21.15.16:0/2391377409 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4c54003a70 0x7f4c54003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:11.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:11.694+0000 7f4c6cdce700 1 --2- 172.21.15.16:0/2391377409 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4c54004850 0x7f4c54004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:11.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:11.694+0000 7f4c6cdce700 1 --2- 172.21.15.16:0/2391377409 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c54005200 0x7f4c5400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:11.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:11.694+0000 7f4c6cdce700 1 -- 172.21.15.16:0/2391377409 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c680cdf20 con 0x7f4c54005200 2023-05-20T01:27:11.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:11.694+0000 7f4c6cdce700 1 -- 172.21.15.16:0/2391377409 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c5400c060 con 0x7f4c54003a70 2023-05-20T01:27:11.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:11.694+0000 7f4c6cdce700 1 -- 172.21.15.16:0/2391377409 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c5400c220 con 0x7f4c54004850 2023-05-20T01:27:11.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:11.694+0000 7f4c6e5d1700 1 --2- 172.21.15.16:0/2391377409 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c54005200 0x7f4c5400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:11.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:11.694+0000 7f4c6ddd0700 1 --2- 172.21.15.16:0/2391377409 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4c54003a70 0x7f4c54003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:11.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:11.694+0000 7f4c6d5cf700 1 --2- 172.21.15.16:0/2391377409 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4c54004850 0x7f4c54004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:11.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:11.694+0000 7f4c6e5d1700 1 -- 172.21.15.16:0/2391377409 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4c54004850 msgr2=0x7f4c54004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:11.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:11.694+0000 7f4c6e5d1700 1 --2- 172.21.15.16:0/2391377409 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4c54004850 0x7f4c54004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:11.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:11.694+0000 7f4c6e5d1700 1 -- 172.21.15.16:0/2391377409 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4c54003a70 msgr2=0x7f4c54003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:11.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:11.694+0000 7f4c6e5d1700 1 --2- 172.21.15.16:0/2391377409 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4c54003a70 0x7f4c54003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:11.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:11.695+0000 7f4c6e5d1700 1 -- 172.21.15.16:0/2391377409 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c540035b0 con 0x7f4c54005200 2023-05-20T01:27:11.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:11.695+0000 7f4c6d5cf700 1 --2- 172.21.15.16:0/2391377409 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4c54004850 0x7f4c54004cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:27:11.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:11.695+0000 7f4c6e5d1700 1 -- 172.21.15.16:0/2391377409 reap_dead start 2023-05-20T01:27:11.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:11.695+0000 7f4c6e5d1700 1 --2- 172.21.15.16:0/2391377409 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c54005200 0x7f4c5400ba10 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f4c6000c290 tx=0x7f4c54005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cd1f362e43cb4e4 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:11.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:11.695+0000 7f4c6cdce700 1 -- 172.21.15.16:0/2391377409 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c60001e70 con 0x7f4c54005200 2023-05-20T01:27:11.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:11.695+0000 7f4c6cdce700 1 -- 172.21.15.16:0/2391377409 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c6000ce70 con 0x7f4c54005200 2023-05-20T01:27:11.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:11.695+0000 7f4c6cdce700 1 -- 172.21.15.16:0/2391377409 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c60003c20 con 0x7f4c54005200 2023-05-20T01:27:11.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:11.695+0000 7f4c70034700 1 -- 172.21.15.16:0/2391377409 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c54005200 msgr2=0x7f4c5400ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:11.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:11.695+0000 7f4c70034700 1 --2- 172.21.15.16:0/2391377409 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c54005200 0x7f4c5400ba10 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f4c6000c290 tx=0x7f4c54005880 comp rx=0 tx=0).stop 2023-05-20T01:27:11.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:11.695+0000 7f4c70034700 1 -- 172.21.15.16:0/2391377409 shutdown_connections 2023-05-20T01:27:11.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:11.695+0000 7f4c70034700 1 --2- 172.21.15.16:0/2391377409 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c54005200 0x7f4c5400ba10 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:11.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:11.695+0000 7f4c70034700 1 -- 172.21.15.16:0/2391377409 >> 172.21.15.16:0/2391377409 conn(0x7f4c680d0c30 msgr2=0x7f4c680b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:27:11.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:11.696+0000 7f4c70034700 1 -- 172.21.15.16:0/2391377409 shutdown_connections 2023-05-20T01:27:11.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:11.696+0000 7f4c70034700 1 -- 172.21.15.16:0/2391377409 wait complete. 2023-05-20T01:27:11.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:11.697+0000 7f4c70034700 1 Processor -- start 2023-05-20T01:27:11.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:11.697+0000 7f4c70034700 1 -- start start 2023-05-20T01:27:11.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:11.697+0000 7f4c70034700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4c54005200 0x7f4c681f3240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:11.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:11.698+0000 7f4c70034700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4c68153e10 0x7f4c6815bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:11.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:11.698+0000 7f4c70034700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c6815c310 0x7f4c680aec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:11.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:11.698+0000 7f4c70034700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c6815b300 con 0x7f4c6815c310 2023-05-20T01:27:11.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:11.698+0000 7f4c70034700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c6815c780 con 0x7f4c68153e10 2023-05-20T01:27:11.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:11.698+0000 7f4c70034700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c681eccd0 con 0x7f4c54005200 2023-05-20T01:27:11.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:11.698+0000 7f4c6e5d1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c6815c310 0x7f4c680aec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:11.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:11.698+0000 7f4c6e5d1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c6815c310 0x7f4c680aec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38660/0 (socket says 172.21.15.16:38660) 2023-05-20T01:27:11.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:11.698+0000 7f4c6e5d1700 1 -- 172.21.15.16:0/2231008408 learned_addr learned my addr 172.21.15.16:0/2231008408 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:27:11.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:11.698+0000 7f4c6ddd0700 1 --2- 172.21.15.16:0/2231008408 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4c54005200 0x7f4c681f3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:11.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:11.698+0000 7f4c6d5cf700 1 --2- 172.21.15.16:0/2231008408 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4c68153e10 0x7f4c6815bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:11.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:11.698+0000 7f4c6e5d1700 1 -- 172.21.15.16:0/2231008408 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4c54005200 msgr2=0x7f4c681f3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:11.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:11.698+0000 7f4c6e5d1700 1 --2- 172.21.15.16:0/2231008408 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4c54005200 0x7f4c681f3240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:11.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:11.698+0000 7f4c6e5d1700 1 -- 172.21.15.16:0/2231008408 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4c68153e10 msgr2=0x7f4c6815bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:11.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:11.698+0000 7f4c6e5d1700 1 --2- 172.21.15.16:0/2231008408 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4c68153e10 0x7f4c6815bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:11.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:11.698+0000 7f4c6e5d1700 1 -- 172.21.15.16:0/2231008408 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c6806edd0 con 0x7f4c6815c310 2023-05-20T01:27:11.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:11.698+0000 7f4c6e5d1700 1 --2- 172.21.15.16:0/2231008408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c6815c310 0x7f4c680aec30 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f4c6000c880 tx=0x7f4c600015b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:11.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:11.698+0000 7f4c5a7fc700 1 -- 172.21.15.16:0/2231008408 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c60017070 con 0x7f4c6815c310 2023-05-20T01:27:11.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:11.698+0000 7f4c5a7fc700 1 -- 172.21.15.16:0/2231008408 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c60001250 con 0x7f4c6815c310 2023-05-20T01:27:11.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:11.698+0000 7f4c5a7fc700 1 -- 172.21.15.16:0/2231008408 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c60004ea0 con 0x7f4c6815c310 2023-05-20T01:27:11.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:11.698+0000 7f4c70034700 1 -- 172.21.15.16:0/2231008408 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c6806eaf0 con 0x7f4c6815c310 2023-05-20T01:27:11.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:11.698+0000 7f4c70034700 1 -- 172.21.15.16:0/2231008408 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c6806f0b0 con 0x7f4c6815c310 2023-05-20T01:27:11.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:11.699+0000 7f4c70034700 1 -- 172.21.15.16:0/2231008408 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c680747b0 con 0x7f4c6815c310 2023-05-20T01:27:11.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:11.700+0000 7f4c5a7fc700 1 -- 172.21.15.16:0/2231008408 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4c600054e0 con 0x7f4c6815c310 2023-05-20T01:27:11.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:11.700+0000 7f4c5a7fc700 1 --2- 172.21.15.16:0/2231008408 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f4c4c06ace0 0x7f4c4c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:11.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:11.700+0000 7f4c5a7fc700 1 -- 172.21.15.16:0/2231008408 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6958+0+0 (secure 0 0 0) 0x7f4c6001a040 con 0x7f4c6815c310 2023-05-20T01:27:11.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:11.700+0000 7f4c6ddd0700 1 --2- 172.21.15.16:0/2231008408 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f4c4c06ace0 0x7f4c4c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:11.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:11.701+0000 7f4c6ddd0700 1 --2- 172.21.15.16:0/2231008408 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f4c4c06ace0 0x7f4c4c06d190 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f4c680b0860 tx=0x7f4c64009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:11.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:11.705+0000 7f4c5a7fc700 1 -- 172.21.15.16:0/2231008408 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f4c60054fa0 con 0x7f4c6815c310 2023-05-20T01:27:11.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:11.945+0000 7f4c70034700 1 -- 172.21.15.16:0/2231008408 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4c6815a260 con 0x7f4c6815c310 2023-05-20T01:27:11.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:11.945+0000 7f4c5a7fc700 1 -- 172.21.15.16:0/2231008408 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) v1 ==== 94+0+3915 (secure 0 0 0) 0x7f4c600585c0 con 0x7f4c6815c310 2023-05-20T01:27:11.950 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:27:11.951 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":83,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7069,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/3619911175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":3619911175},{"type":"v1","addr":"172.21.15.16:6837","nonce":3619911175}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73},{"gid":7081,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/959638003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":959638003},{"type":"v1","addr":"172.21.15.132:6834","nonce":959638003}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73},{"gid":7084,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2596482253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":2596482253},{"type":"v1","addr":"172.21.15.132:6835","nonce":2596482253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-20T01:27:10.291795+0000","modified":"2023-05-20T01:27:11.539644+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":117,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[16,17],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-05-20T01:27:11.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:11.947+0000 7f4c70034700 1 -- 172.21.15.16:0/2231008408 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f4c4c06ace0 msgr2=0x7f4c4c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:11.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:11.947+0000 7f4c70034700 1 --2- 172.21.15.16:0/2231008408 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f4c4c06ace0 0x7f4c4c06d190 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f4c680b0860 tx=0x7f4c64009210 comp rx=0 tx=0).stop 2023-05-20T01:27:11.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:11.947+0000 7f4c70034700 1 -- 172.21.15.16:0/2231008408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c6815c310 msgr2=0x7f4c680aec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:11.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:11.947+0000 7f4c70034700 1 --2- 172.21.15.16:0/2231008408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c6815c310 0x7f4c680aec30 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f4c6000c880 tx=0x7f4c600015b0 comp rx=0 tx=0).stop 2023-05-20T01:27:11.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:11.947+0000 7f4c70034700 1 -- 172.21.15.16:0/2231008408 shutdown_connections 2023-05-20T01:27:11.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:11.947+0000 7f4c70034700 1 --2- 172.21.15.16:0/2231008408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c6815c310 0x7f4c680aec30 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:11.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:11.947+0000 7f4c70034700 1 --2- 172.21.15.16:0/2231008408 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f4c4c06ace0 0x7f4c4c06d190 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:11.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:11.947+0000 7f4c70034700 1 --2- 172.21.15.16:0/2231008408 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4c54005200 0x7f4c681f3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:11.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:11.947+0000 7f4c70034700 1 --2- 172.21.15.16:0/2231008408 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4c68153e10 0x7f4c6815bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:11.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:11.947+0000 7f4c70034700 1 -- 172.21.15.16:0/2231008408 >> 172.21.15.16:0/2231008408 conn(0x7f4c680d0c30 msgr2=0x7f4c680caf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:27:11.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:11.947+0000 7f4c70034700 1 -- 172.21.15.16:0/2231008408 shutdown_connections 2023-05-20T01:27:11.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:11.947+0000 7f4c70034700 1 -- 172.21.15.16:0/2231008408 wait complete. 2023-05-20T01:27:11.954 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 83 2023-05-20T01:27:11.964 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-05-20T01:27:11.964 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-20T01:27:12.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:12.124+0000 7f01b9ca2700 1 Processor -- start 2023-05-20T01:27:12.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:12.124+0000 7f01b9ca2700 1 -- start start 2023-05-20T01:27:12.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:12.125+0000 7f01b9ca2700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f01b406f450 0x7f01b40cf890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:12.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:12.125+0000 7f01b9ca2700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01b40cc510 con 0x7f01b406a510 2023-05-20T01:27:12.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:12.125+0000 7f01b9ca2700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01b4071ed0 con 0x7f01b406d4b0 2023-05-20T01:27:12.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:12.125+0000 7f01b9ca2700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01b40721b0 con 0x7f01b406f450 2023-05-20T01:27:12.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:12.125+0000 7f01b3fff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f01b406a510 0x7f01b40cc9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50196/0 (socket says 172.21.15.16:50196) 2023-05-20T01:27:12.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:12.125+0000 7f01b3fff700 1 -- 172.21.15.16:0/3551534291 learned_addr learned my addr 172.21.15.16:0/3551534291 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:27:12.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:12.125+0000 7f01b37fe700 1 --2- 172.21.15.16:0/3551534291 >> v2:172.21.15.132:3300/0 conn(0x7f01b406f450 0x7f01b40cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:12.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:12.125+0000 7f01b27fc700 1 -- 172.21.15.16:0/3551534291 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 321985757 0 0) 0x7f01b40cc510 con 0x7f01b406a510 2023-05-20T01:27:12.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:12.125+0000 7f01b27fc700 1 -- 172.21.15.16:0/3551534291 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f019c0035b0 con 0x7f01b406a510 2023-05-20T01:27:12.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:12.125+0000 7f01b27fc700 1 -- 172.21.15.16:0/3551534291 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1934210389 0 0) 0x7f019c0035b0 con 0x7f01b406a510 2023-05-20T01:27:12.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:12.125+0000 7f01b27fc700 1 -- 172.21.15.16:0/3551534291 >> v2:172.21.15.132:3300/0 conn(0x7f01b406f450 msgr2=0x7f01b40cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:12.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:12.125+0000 7f01b27fc700 1 --2- 172.21.15.16:0/3551534291 >> v2:172.21.15.132:3300/0 conn(0x7f01b406f450 0x7f01b40cf890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:12.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:12.125+0000 7f01b27fc700 1 -- 172.21.15.16:0/3551534291 >> v1:172.21.15.132:6789/0 conn(0x7f01b406d4b0 legacy=0x7f01b4161080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:12.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:12.125+0000 7f01b27fc700 1 -- 172.21.15.16:0/3551534291 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01b40cc1a0 con 0x7f01b406a510 2023-05-20T01:27:12.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:12.125+0000 7f01b27fc700 1 -- 172.21.15.16:0/3551534291 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7f01ac0019f0 con 0x7f01b406a510 2023-05-20T01:27:12.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:12.125+0000 7f01b27fc700 1 -- 172.21.15.16:0/3551534291 >> v1:172.21.15.16:6789/0 conn(0x7f01b406a510 legacy=0x7f01b40cc9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:12.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:12.125+0000 7f01b27fc700 1 --2- 172.21.15.16:0/3551534291 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f019c003a70 0x7f019c003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:12.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:12.125+0000 7f01b27fc700 1 --2- 172.21.15.16:0/3551534291 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f019c004850 0x7f019c004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:12.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:12.125+0000 7f01b27fc700 1 --2- 172.21.15.16:0/3551534291 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f019c005200 0x7f019c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:12.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:12.125+0000 7f01b27fc700 1 -- 172.21.15.16:0/3551534291 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01b40721b0 con 0x7f019c005200 2023-05-20T01:27:12.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:12.125+0000 7f01b27fc700 1 -- 172.21.15.16:0/3551534291 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f019c00c110 con 0x7f019c004850 2023-05-20T01:27:12.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:12.125+0000 7f01b27fc700 1 -- 172.21.15.16:0/3551534291 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f019c00c2d0 con 0x7f019c003a70 2023-05-20T01:27:12.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:12.126+0000 7f01b3fff700 1 --2- 172.21.15.16:0/3551534291 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f019c005200 0x7f019c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:12.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:12.126+0000 7f01b37fe700 1 --2- 172.21.15.16:0/3551534291 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f019c003a70 0x7f019c003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:12.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:12.126+0000 7f01b2ffd700 1 --2- 172.21.15.16:0/3551534291 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f019c004850 0x7f019c004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:12.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:12.126+0000 7f01b3fff700 1 -- 172.21.15.16:0/3551534291 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f019c003a70 msgr2=0x7f019c003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:12.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:12.126+0000 7f01b3fff700 1 --2- 172.21.15.16:0/3551534291 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f019c003a70 0x7f019c003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:12.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:12.126+0000 7f01b3fff700 1 -- 172.21.15.16:0/3551534291 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f019c004850 msgr2=0x7f019c004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:12.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:12.126+0000 7f01b3fff700 1 --2- 172.21.15.16:0/3551534291 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f019c004850 0x7f019c004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:12.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:12.126+0000 7f01b3fff700 1 -- 172.21.15.16:0/3551534291 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f019c0035b0 con 0x7f019c005200 2023-05-20T01:27:12.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:12.126+0000 7f01b3fff700 1 -- 172.21.15.16:0/3551534291 reap_dead start 2023-05-20T01:27:12.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:12.126+0000 7f01b3fff700 1 --2- 172.21.15.16:0/3551534291 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f019c005200 0x7f019c00ba10 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f01ac002230 tx=0x7f019c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=680da0dc8d430837 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:12.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:12.126+0000 7f01b27fc700 1 -- 172.21.15.16:0/3551534291 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01ac012600 con 0x7f019c005200 2023-05-20T01:27:12.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:12.126+0000 7f01b27fc700 1 -- 172.21.15.16:0/3551534291 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01ac012760 con 0x7f019c005200 2023-05-20T01:27:12.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:12.126+0000 7f01b27fc700 1 -- 172.21.15.16:0/3551534291 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01ac012b10 con 0x7f019c005200 2023-05-20T01:27:12.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:12.127+0000 7f01b9ca2700 1 -- 172.21.15.16:0/3551534291 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f019c005200 msgr2=0x7f019c00ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:12.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:12.127+0000 7f01b9ca2700 1 --2- 172.21.15.16:0/3551534291 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f019c005200 0x7f019c00ba10 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f01ac002230 tx=0x7f019c005880 comp rx=0 tx=0).stop 2023-05-20T01:27:12.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:12.127+0000 7f01b9ca2700 1 -- 172.21.15.16:0/3551534291 shutdown_connections 2023-05-20T01:27:12.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:12.127+0000 7f01b9ca2700 1 --2- 172.21.15.16:0/3551534291 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f019c005200 0x7f019c00ba10 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:12.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:12.127+0000 7f01b9ca2700 1 -- 172.21.15.16:0/3551534291 >> 172.21.15.16:0/3551534291 conn(0x7f01b40cffb0 msgr2=0x7f01b40d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:27:12.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:12.127+0000 7f01b9ca2700 1 -- 172.21.15.16:0/3551534291 shutdown_connections 2023-05-20T01:27:12.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:12.127+0000 7f01b9ca2700 1 -- 172.21.15.16:0/3551534291 wait complete. 2023-05-20T01:27:12.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:12.128+0000 7f01b9ca2700 1 Processor -- start 2023-05-20T01:27:12.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:12.128+0000 7f01b9ca2700 1 -- start start 2023-05-20T01:27:12.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:12.129+0000 7f01b9ca2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f019c005200 0x7f01b41e6780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:12.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:12.129+0000 7f01b9ca2700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f01b4159460 0x7f01b40c5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:12.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:12.129+0000 7f01b9ca2700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f01b40b5340 0x7f01b40b57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:12.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:12.129+0000 7f01b9ca2700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01b41e7040 con 0x7f019c005200 2023-05-20T01:27:12.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:12.129+0000 7f01b9ca2700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01b40b5d00 con 0x7f01b40b5340 2023-05-20T01:27:12.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:12.129+0000 7f01b9ca2700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01b41e6d60 con 0x7f01b4159460 2023-05-20T01:27:12.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:12.129+0000 7f01b37fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f019c005200 0x7f01b41e6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:12.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:12.129+0000 7f01b37fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f019c005200 0x7f01b41e6780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38676/0 (socket says 172.21.15.16:38676) 2023-05-20T01:27:12.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:12.129+0000 7f01b37fe700 1 -- 172.21.15.16:0/4088130405 learned_addr learned my addr 172.21.15.16:0/4088130405 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:27:12.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:12.129+0000 7f01b37fe700 1 -- 172.21.15.16:0/4088130405 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f01b4159460 msgr2=0x7f01b40c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:12.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:12.129+0000 7f01b37fe700 1 --2- 172.21.15.16:0/4088130405 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f01b4159460 0x7f01b40c5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:12.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:12.129+0000 7f01b37fe700 1 -- 172.21.15.16:0/4088130405 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f01b40b5340 msgr2=0x7f01b40b57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:12.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:12.129+0000 7f01b37fe700 1 --2- 172.21.15.16:0/4088130405 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f01b40b5340 0x7f01b40b57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:12.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:12.129+0000 7f01b37fe700 1 -- 172.21.15.16:0/4088130405 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01b40cf890 con 0x7f019c005200 2023-05-20T01:27:12.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:12.130+0000 7f01b37fe700 1 --2- 172.21.15.16:0/4088130405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f019c005200 0x7f01b41e6780 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f01a40119a0 tx=0x7f01a4011d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:12.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:12.130+0000 7f019bfff700 1 -- 172.21.15.16:0/4088130405 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a400f940 con 0x7f019c005200 2023-05-20T01:27:12.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:12.130+0000 7f019bfff700 1 -- 172.21.15.16:0/4088130405 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01a400faa0 con 0x7f019c005200 2023-05-20T01:27:12.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:12.130+0000 7f019bfff700 1 -- 172.21.15.16:0/4088130405 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a400fe30 con 0x7f019c005200 2023-05-20T01:27:12.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:12.130+0000 7f01b9ca2700 1 -- 172.21.15.16:0/4088130405 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01b40cbe50 con 0x7f019c005200 2023-05-20T01:27:12.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:12.130+0000 7f01b9ca2700 1 -- 172.21.15.16:0/4088130405 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01b40cfa20 con 0x7f019c005200 2023-05-20T01:27:12.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:12.131+0000 7f01b9ca2700 1 -- 172.21.15.16:0/4088130405 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01b40747b0 con 0x7f019c005200 2023-05-20T01:27:12.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:12.131+0000 7f019bfff700 1 -- 172.21.15.16:0/4088130405 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f01a4014440 con 0x7f019c005200 2023-05-20T01:27:12.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:12.131+0000 7f019bfff700 1 --2- 172.21.15.16:0/4088130405 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f01a006ace0 0x7f01a006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:12.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:12.132+0000 7f019bfff700 1 -- 172.21.15.16:0/4088130405 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6958+0+0 (secure 0 0 0) 0x7f01a4089ee0 con 0x7f019c005200 2023-05-20T01:27:12.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:12.132+0000 7f01b2ffd700 1 --2- 172.21.15.16:0/4088130405 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f01a006ace0 0x7f01a006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:12.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:12.132+0000 7f01b2ffd700 1 --2- 172.21.15.16:0/4088130405 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f01a006ace0 0x7f01a006d190 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f01a8007fd0 tx=0x7f01a8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:12.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:12.136+0000 7f019bfff700 1 -- 172.21.15.16:0/4088130405 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f01a40587e0 con 0x7f019c005200 2023-05-20T01:27:12.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:12.378+0000 7f01b9ca2700 1 -- 172.21.15.16:0/4088130405 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f01b4071c10 con 0x7f019c005200 2023-05-20T01:27:12.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:12.378+0000 7f019bfff700 1 -- 172.21.15.16:0/4088130405 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) v1 ==== 94+0+3915 (secure 0 0 0) 0x7f01a4058370 con 0x7f019c005200 2023-05-20T01:27:12.383 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:27:12.383 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":83,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7069,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/3619911175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":3619911175},{"type":"v1","addr":"172.21.15.16:6837","nonce":3619911175}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73},{"gid":7081,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/959638003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":959638003},{"type":"v1","addr":"172.21.15.132:6834","nonce":959638003}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73},{"gid":7084,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2596482253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":2596482253},{"type":"v1","addr":"172.21.15.132:6835","nonce":2596482253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-20T01:27:10.291795+0000","modified":"2023-05-20T01:27:11.539644+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":117,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[16,17],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-05-20T01:27:12.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:12.380+0000 7f01b9ca2700 1 -- 172.21.15.16:0/4088130405 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f01a006ace0 msgr2=0x7f01a006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:12.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:12.380+0000 7f01b9ca2700 1 --2- 172.21.15.16:0/4088130405 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f01a006ace0 0x7f01a006d190 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f01a8007fd0 tx=0x7f01a8009040 comp rx=0 tx=0).stop 2023-05-20T01:27:12.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:12.380+0000 7f01b9ca2700 1 -- 172.21.15.16:0/4088130405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f019c005200 msgr2=0x7f01b41e6780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:12.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:12.380+0000 7f01b9ca2700 1 --2- 172.21.15.16:0/4088130405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f019c005200 0x7f01b41e6780 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f01a40119a0 tx=0x7f01a4011d60 comp rx=0 tx=0).stop 2023-05-20T01:27:12.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:12.380+0000 7f01b9ca2700 1 -- 172.21.15.16:0/4088130405 shutdown_connections 2023-05-20T01:27:12.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:12.380+0000 7f01b9ca2700 1 --2- 172.21.15.16:0/4088130405 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f01b40b5340 0x7f01b40b57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:12.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:12.380+0000 7f01b9ca2700 1 --2- 172.21.15.16:0/4088130405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f019c005200 0x7f01b41e6780 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:12.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:12.380+0000 7f01b9ca2700 1 --2- 172.21.15.16:0/4088130405 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f01a006ace0 0x7f01a006d190 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:12.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:12.380+0000 7f01b9ca2700 1 --2- 172.21.15.16:0/4088130405 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f01b4159460 0x7f01b40c5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:12.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:12.380+0000 7f01b9ca2700 1 -- 172.21.15.16:0/4088130405 >> 172.21.15.16:0/4088130405 conn(0x7f01b40cffb0 msgr2=0x7f01b40c7900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:27:12.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:12.381+0000 7f01b9ca2700 1 -- 172.21.15.16:0/4088130405 shutdown_connections 2023-05-20T01:27:12.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:12.381+0000 7f01b9ca2700 1 -- 172.21.15.16:0/4088130405 wait complete. 2023-05-20T01:27:12.389 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 83 2023-05-20T01:27:12.398 DEBUG:tasks.ceph_test_case:wait_until_equal: None != up:standby, waiting (timeout=60)... 2023-05-20T01:27:17.400 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-20T01:27:17.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.535+0000 7f40c22ee700 1 Processor -- start 2023-05-20T01:27:17.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.535+0000 7f40c22ee700 1 -- start start 2023-05-20T01:27:17.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.536+0000 7f40c22ee700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f40bc0ccec0 0x7f40bc0713c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:17.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.536+0000 7f40c22ee700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40bc0cb590 con 0x7f40bc071900 2023-05-20T01:27:17.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.536+0000 7f40c22ee700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40bc0ca9a0 con 0x7f40bc157190 2023-05-20T01:27:17.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.536+0000 7f40c22ee700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40bc0cac60 con 0x7f40bc0ccec0 2023-05-20T01:27:17.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.536+0000 7f40c088b700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f40bc071900 0x7f40bc0ce740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50218/0 (socket says 172.21.15.16:50218) 2023-05-20T01:27:17.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.536+0000 7f40c088b700 1 -- 172.21.15.16:0/2492990239 learned_addr learned my addr 172.21.15.16:0/2492990239 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:27:17.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.536+0000 7f40bb7fe700 1 --2- 172.21.15.16:0/2492990239 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f40bc0ccec0 0x7f40bc0713c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:17.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.536+0000 7f40baffd700 1 -- 172.21.15.16:0/2492990239 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2269525183 0 0) 0x7f40bc0cb590 con 0x7f40bc071900 2023-05-20T01:27:17.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.536+0000 7f40baffd700 1 -- 172.21.15.16:0/2492990239 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f40a80035b0 con 0x7f40bc071900 2023-05-20T01:27:17.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.536+0000 7f40baffd700 1 -- 172.21.15.16:0/2492990239 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3399274380 0 0) 0x7f40a80035b0 con 0x7f40bc071900 2023-05-20T01:27:17.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.536+0000 7f40baffd700 1 -- 172.21.15.16:0/2492990239 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f40bc0ccec0 msgr2=0x7f40bc0713c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:17.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.536+0000 7f40baffd700 1 --2- 172.21.15.16:0/2492990239 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f40bc0ccec0 0x7f40bc0713c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:17.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.536+0000 7f40baffd700 1 -- 172.21.15.16:0/2492990239 >> v1:172.21.15.132:6789/0 conn(0x7f40bc157190 legacy=0x7f40bc0cc7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:17.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.537+0000 7f40baffd700 1 -- 172.21.15.16:0/2492990239 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40bc0caf40 con 0x7f40bc071900 2023-05-20T01:27:17.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.537+0000 7f40baffd700 1 -- 172.21.15.16:0/2492990239 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7f40b40019f0 con 0x7f40bc071900 2023-05-20T01:27:17.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.537+0000 7f40baffd700 1 -- 172.21.15.16:0/2492990239 >> v1:172.21.15.16:6789/0 conn(0x7f40bc071900 legacy=0x7f40bc0ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:17.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.537+0000 7f40baffd700 1 --2- 172.21.15.16:0/2492990239 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f40bc0ccec0 0x7f40a8003aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:17.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.537+0000 7f40baffd700 1 --2- 172.21.15.16:0/2492990239 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f40a8003fe0 0x7f40a8004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:17.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.537+0000 7f40baffd700 1 --2- 172.21.15.16:0/2492990239 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40a8004d90 0x7f40a8005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:17.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.537+0000 7f40baffd700 1 -- 172.21.15.16:0/2492990239 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40bc0cac60 con 0x7f40a8004d90 2023-05-20T01:27:17.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.537+0000 7f40baffd700 1 -- 172.21.15.16:0/2492990239 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40a800bba0 con 0x7f40a8003fe0 2023-05-20T01:27:17.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.537+0000 7f40baffd700 1 -- 172.21.15.16:0/2492990239 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40a800df40 con 0x7f40bc0ccec0 2023-05-20T01:27:17.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.537+0000 7f40c088b700 1 --2- 172.21.15.16:0/2492990239 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40a8004d90 0x7f40a8005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:17.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.537+0000 7f40bbfff700 1 --2- 172.21.15.16:0/2492990239 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f40bc0ccec0 0x7f40a8003aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:17.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.537+0000 7f40bb7fe700 1 --2- 172.21.15.16:0/2492990239 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f40a8003fe0 0x7f40a8004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:17.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.537+0000 7f40c088b700 1 -- 172.21.15.16:0/2492990239 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f40bc0ccec0 msgr2=0x7f40a8003aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:17.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.537+0000 7f40c088b700 1 --2- 172.21.15.16:0/2492990239 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f40bc0ccec0 0x7f40a8003aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:17.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.537+0000 7f40c088b700 1 -- 172.21.15.16:0/2492990239 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f40a8003fe0 msgr2=0x7f40a8004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:17.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.537+0000 7f40c088b700 1 --2- 172.21.15.16:0/2492990239 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f40a8003fe0 0x7f40a8004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:17.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.537+0000 7f40c088b700 1 -- 172.21.15.16:0/2492990239 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40a80035b0 con 0x7f40a8004d90 2023-05-20T01:27:17.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.537+0000 7f40bb7fe700 1 --2- 172.21.15.16:0/2492990239 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f40a8003fe0 0x7f40a8004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:27:17.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.537+0000 7f40c088b700 1 --2- 172.21.15.16:0/2492990239 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40a8004d90 0x7f40a8005200 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f40b400a7d0 tx=0x7f40a8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d70a4bc86dd6a05 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:17.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.537+0000 7f40baffd700 1 -- 172.21.15.16:0/2492990239 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b4012600 con 0x7f40a8004d90 2023-05-20T01:27:17.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.537+0000 7f40baffd700 1 -- 172.21.15.16:0/2492990239 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40b4012780 con 0x7f40a8004d90 2023-05-20T01:27:17.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.537+0000 7f40baffd700 1 -- 172.21.15.16:0/2492990239 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b4012b30 con 0x7f40a8004d90 2023-05-20T01:27:17.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.538+0000 7f40c22ee700 1 -- 172.21.15.16:0/2492990239 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40a8004d90 msgr2=0x7f40a8005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:17.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.538+0000 7f40c22ee700 1 --2- 172.21.15.16:0/2492990239 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40a8004d90 0x7f40a8005200 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f40b400a7d0 tx=0x7f40a8005880 comp rx=0 tx=0).stop 2023-05-20T01:27:17.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.538+0000 7f40c088b700 1 -- 172.21.15.16:0/2492990239 reap_dead start 2023-05-20T01:27:17.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.538+0000 7f40c22ee700 1 -- 172.21.15.16:0/2492990239 shutdown_connections 2023-05-20T01:27:17.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.538+0000 7f40c22ee700 1 -- 172.21.15.16:0/2492990239 >> 172.21.15.16:0/2492990239 conn(0x7f40bc0cffb0 msgr2=0x7f40bc0d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:27:17.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.538+0000 7f40c22ee700 1 -- 172.21.15.16:0/2492990239 shutdown_connections 2023-05-20T01:27:17.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.538+0000 7f40c22ee700 1 -- 172.21.15.16:0/2492990239 wait complete. 2023-05-20T01:27:17.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.539+0000 7f40c22ee700 1 Processor -- start 2023-05-20T01:27:17.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.539+0000 7f40c22ee700 1 -- start start 2023-05-20T01:27:17.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.540+0000 7f40c22ee700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40bc14b400 0x7f40bc0c3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:17.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.540+0000 7f40c22ee700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f40bc14b7d0 0x7f40bc0c4020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:17.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.540+0000 7f40c22ee700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f40bc0b4870 0x7f40bc0b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:17.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.540+0000 7f40c22ee700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40bc0b55d0 con 0x7f40bc14b400 2023-05-20T01:27:17.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.540+0000 7f40c22ee700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40bc0c7240 con 0x7f40bc14b7d0 2023-05-20T01:27:17.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.540+0000 7f40c22ee700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40bc0c7520 con 0x7f40bc0b4870 2023-05-20T01:27:17.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.540+0000 7f40bbfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40bc14b400 0x7f40bc0c3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:17.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.540+0000 7f40bbfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40bc14b400 0x7f40bc0c3ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38698/0 (socket says 172.21.15.16:38698) 2023-05-20T01:27:17.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.540+0000 7f40bbfff700 1 -- 172.21.15.16:0/1554403027 learned_addr learned my addr 172.21.15.16:0/1554403027 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:27:17.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.541+0000 7f40bb7fe700 1 --2- 172.21.15.16:0/1554403027 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f40bc14b7d0 0x7f40bc0c4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:17.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.541+0000 7f40c088b700 1 --2- 172.21.15.16:0/1554403027 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f40bc0b4870 0x7f40bc0b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:17.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.541+0000 7f40bbfff700 1 -- 172.21.15.16:0/1554403027 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f40bc0b4870 msgr2=0x7f40bc0b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:17.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.541+0000 7f40bbfff700 1 --2- 172.21.15.16:0/1554403027 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f40bc0b4870 0x7f40bc0b4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:17.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.541+0000 7f40bbfff700 1 -- 172.21.15.16:0/1554403027 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f40bc14b7d0 msgr2=0x7f40bc0c4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:17.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.541+0000 7f40bbfff700 1 --2- 172.21.15.16:0/1554403027 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f40bc14b7d0 0x7f40bc0c4020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:17.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.541+0000 7f40bbfff700 1 -- 172.21.15.16:0/1554403027 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40bc157560 con 0x7f40bc14b400 2023-05-20T01:27:17.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.541+0000 7f40bbfff700 1 --2- 172.21.15.16:0/1554403027 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40bc14b400 0x7f40bc0c3ae0 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f40ac00f840 tx=0x7f40ac00fc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:17.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.541+0000 7f40b8ff9700 1 -- 172.21.15.16:0/1554403027 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40ac005320 con 0x7f40bc14b400 2023-05-20T01:27:17.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.541+0000 7f40b8ff9700 1 -- 172.21.15.16:0/1554403027 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40ac0054a0 con 0x7f40bc14b400 2023-05-20T01:27:17.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.541+0000 7f40c22ee700 1 -- 172.21.15.16:0/1554403027 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40bc0cb220 con 0x7f40bc14b400 2023-05-20T01:27:17.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.541+0000 7f40b8ff9700 1 -- 172.21.15.16:0/1554403027 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40ac006aa0 con 0x7f40bc14b400 2023-05-20T01:27:17.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.541+0000 7f40c22ee700 1 -- 172.21.15.16:0/1554403027 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40bc0cba20 con 0x7f40bc14b400 2023-05-20T01:27:17.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.542+0000 7f40c22ee700 1 -- 172.21.15.16:0/1554403027 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40bc0c7080 con 0x7f40bc14b400 2023-05-20T01:27:17.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.542+0000 7f40b8ff9700 1 -- 172.21.15.16:0/1554403027 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f40ac006c00 con 0x7f40bc14b400 2023-05-20T01:27:17.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.543+0000 7f40b8ff9700 1 --2- 172.21.15.16:0/1554403027 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f409c06ace0 0x7f409c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:17.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.543+0000 7f40b8ff9700 1 -- 172.21.15.16:0/1554403027 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6987+0+0 (secure 0 0 0) 0x7f40ac0891c0 con 0x7f40bc14b400 2023-05-20T01:27:17.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.543+0000 7f40bb7fe700 1 --2- 172.21.15.16:0/1554403027 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f409c06ace0 0x7f409c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:17.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.543+0000 7f40bb7fe700 1 --2- 172.21.15.16:0/1554403027 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f409c06ace0 0x7f409c06d190 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f40bc0b5bd0 tx=0x7f40a4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:17.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.547+0000 7f40b8ff9700 1 -- 172.21.15.16:0/1554403027 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f40ac053f60 con 0x7f40bc14b400 2023-05-20T01:27:17.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.790+0000 7f40c22ee700 1 -- 172.21.15.16:0/1554403027 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f40bc0c79e0 con 0x7f40bc14b400 2023-05-20T01:27:17.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.790+0000 7f40b8ff9700 1 -- 172.21.15.16:0/1554403027 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 84 v84) v1 ==== 94+0+4672 (secure 0 0 0) 0x7f40ac057580 con 0x7f40bc14b400 2023-05-20T01:27:17.795 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:27:17.795 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":84,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7069,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/3619911175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":3619911175},{"type":"v1","addr":"172.21.15.16:6837","nonce":3619911175}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73},{"gid":7081,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/959638003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":959638003},{"type":"v1","addr":"172.21.15.132:6834","nonce":959638003}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73},{"gid":7084,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2596482253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":2596482253},{"type":"v1","addr":"172.21.15.132:6835","nonce":2596482253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73},{"gid":8205,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/1584803225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1584803225},{"type":"v1","addr":"172.21.15.16:6836","nonce":1584803225}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":84}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-20T01:27:10.291795+0000","modified":"2023-05-20T01:27:11.539644+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":117,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[16,17],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-05-20T01:27:17.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.793+0000 7f40c22ee700 1 -- 172.21.15.16:0/1554403027 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f409c06ace0 msgr2=0x7f409c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:17.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.793+0000 7f40c22ee700 1 --2- 172.21.15.16:0/1554403027 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f409c06ace0 0x7f409c06d190 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f40bc0b5bd0 tx=0x7f40a4008040 comp rx=0 tx=0).stop 2023-05-20T01:27:17.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.793+0000 7f40c22ee700 1 -- 172.21.15.16:0/1554403027 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40bc14b400 msgr2=0x7f40bc0c3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:17.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.793+0000 7f40c22ee700 1 --2- 172.21.15.16:0/1554403027 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40bc14b400 0x7f40bc0c3ae0 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f40ac00f840 tx=0x7f40ac00fc00 comp rx=0 tx=0).stop 2023-05-20T01:27:17.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.793+0000 7f40c22ee700 1 -- 172.21.15.16:0/1554403027 shutdown_connections 2023-05-20T01:27:17.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.793+0000 7f40c22ee700 1 --2- 172.21.15.16:0/1554403027 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f409c06ace0 0x7f409c06d190 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:17.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.793+0000 7f40c22ee700 1 --2- 172.21.15.16:0/1554403027 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f40bc0b4870 0x7f40bc0b4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:17.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.793+0000 7f40c22ee700 1 --2- 172.21.15.16:0/1554403027 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40bc14b400 0x7f40bc0c3ae0 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:17.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.793+0000 7f40c22ee700 1 --2- 172.21.15.16:0/1554403027 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f40bc14b7d0 0x7f40bc0c4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:17.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.793+0000 7f40c22ee700 1 -- 172.21.15.16:0/1554403027 >> 172.21.15.16:0/1554403027 conn(0x7f40bc0cffb0 msgr2=0x7f40bc06dcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:27:17.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.793+0000 7f40c22ee700 1 -- 172.21.15.16:0/1554403027 shutdown_connections 2023-05-20T01:27:17.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.793+0000 7f40c22ee700 1 -- 172.21.15.16:0/1554403027 wait complete. 2023-05-20T01:27:17.800 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 84 2023-05-20T01:27:17.809 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-20T01:27:17.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.932+0000 7f8c7933e700 1 Processor -- start 2023-05-20T01:27:17.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.932+0000 7f8c7933e700 1 -- start start 2023-05-20T01:27:17.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.933+0000 7f8c7933e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f8c740cfc50 0x7f8c740d2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:17.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.933+0000 7f8c7933e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8c740cd970 0x7f8c740cddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:17.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.933+0000 7f8c7933e700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c740b3b50 con 0x7f8c740b4750 2023-05-20T01:27:17.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.933+0000 7f8c7933e700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c740cfad0 con 0x7f8c740cfc50 2023-05-20T01:27:17.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.933+0000 7f8c7933e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c740d3df0 con 0x7f8c740cd970 2023-05-20T01:27:17.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.933+0000 7f8c727fc700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f8c740b4750 0x7f8c740d0020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50234/0 (socket says 172.21.15.16:50234) 2023-05-20T01:27:17.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.933+0000 7f8c727fc700 1 -- 172.21.15.16:0/282826249 learned_addr learned my addr 172.21.15.16:0/282826249 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:27:17.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.933+0000 7f8c72ffd700 1 --2- 172.21.15.16:0/282826249 >> v2:172.21.15.132:3300/0 conn(0x7f8c740cfc50 0x7f8c740d2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:17.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.933+0000 7f8c737fe700 1 --2- 172.21.15.16:0/282826249 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8c740cd970 0x7f8c740cddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:17.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.933+0000 7f8c71ffb700 1 -- 172.21.15.16:0/282826249 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2796609981 0 0) 0x7f8c740b3b50 con 0x7f8c740b4750 2023-05-20T01:27:17.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.933+0000 7f8c71ffb700 1 -- 172.21.15.16:0/282826249 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c5c0035b0 con 0x7f8c740b4750 2023-05-20T01:27:17.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.934+0000 7f8c71ffb700 1 -- 172.21.15.16:0/282826249 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2185877632 0 0) 0x7f8c5c0035b0 con 0x7f8c740b4750 2023-05-20T01:27:17.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.934+0000 7f8c71ffb700 1 -- 172.21.15.16:0/282826249 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8c740cd970 msgr2=0x7f8c740cddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:17.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.934+0000 7f8c71ffb700 1 --2- 172.21.15.16:0/282826249 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8c740cd970 0x7f8c740cddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:17.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.934+0000 7f8c71ffb700 1 -- 172.21.15.16:0/282826249 >> v2:172.21.15.132:3300/0 conn(0x7f8c740cfc50 msgr2=0x7f8c740d2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:17.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.934+0000 7f8c71ffb700 1 --2- 172.21.15.16:0/282826249 >> v2:172.21.15.132:3300/0 conn(0x7f8c740cfc50 0x7f8c740d2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:17.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.934+0000 7f8c71ffb700 1 -- 172.21.15.16:0/282826249 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c740d0b90 con 0x7f8c740b4750 2023-05-20T01:27:17.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.934+0000 7f8c71ffb700 1 -- 172.21.15.16:0/282826249 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7f8c64001670 con 0x7f8c740b4750 2023-05-20T01:27:17.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.934+0000 7f8c71ffb700 1 -- 172.21.15.16:0/282826249 >> v1:172.21.15.16:6789/0 conn(0x7f8c740b4750 legacy=0x7f8c740d0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:17.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.934+0000 7f8c71ffb700 1 --2- 172.21.15.16:0/282826249 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c5c003a70 0x7f8c5c003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:17.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.934+0000 7f8c71ffb700 1 --2- 172.21.15.16:0/282826249 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8c740cd970 0x7f8c5c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:17.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.934+0000 7f8c71ffb700 1 --2- 172.21.15.16:0/282826249 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c5c004d90 0x7f8c5c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:17.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.934+0000 7f8c71ffb700 1 -- 172.21.15.16:0/282826249 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c740cfad0 con 0x7f8c5c003a70 2023-05-20T01:27:17.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.934+0000 7f8c71ffb700 1 -- 172.21.15.16:0/282826249 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c740d3df0 con 0x7f8c5c004d90 2023-05-20T01:27:17.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.934+0000 7f8c71ffb700 1 -- 172.21.15.16:0/282826249 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c5c00ba60 con 0x7f8c740cd970 2023-05-20T01:27:17.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.934+0000 7f8c72ffd700 1 --2- 172.21.15.16:0/282826249 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c5c003a70 0x7f8c5c003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:17.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.934+0000 7f8c727fc700 1 --2- 172.21.15.16:0/282826249 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8c740cd970 0x7f8c5c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:17.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.934+0000 7f8c737fe700 1 --2- 172.21.15.16:0/282826249 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c5c004d90 0x7f8c5c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:17.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.934+0000 7f8c72ffd700 1 -- 172.21.15.16:0/282826249 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8c740cd970 msgr2=0x7f8c5c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:17.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.934+0000 7f8c72ffd700 1 --2- 172.21.15.16:0/282826249 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8c740cd970 0x7f8c5c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:17.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.934+0000 7f8c72ffd700 1 -- 172.21.15.16:0/282826249 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c5c004d90 msgr2=0x7f8c5c005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:17.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.934+0000 7f8c72ffd700 1 --2- 172.21.15.16:0/282826249 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c5c004d90 0x7f8c5c005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:17.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.934+0000 7f8c72ffd700 1 -- 172.21.15.16:0/282826249 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c5c0035b0 con 0x7f8c5c003a70 2023-05-20T01:27:17.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.934+0000 7f8c737fe700 1 --2- 172.21.15.16:0/282826249 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c5c004d90 0x7f8c5c005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:27:17.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.934+0000 7f8c72ffd700 1 --2- 172.21.15.16:0/282826249 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c5c003a70 0x7f8c5c003e80 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f8c68009b90 tx=0x7f8c5c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2f6292312378177 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:17.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.934+0000 7f8c71ffb700 1 -- 172.21.15.16:0/282826249 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c6800e6c0 con 0x7f8c5c003a70 2023-05-20T01:27:17.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.934+0000 7f8c71ffb700 1 -- 172.21.15.16:0/282826249 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c6800e820 con 0x7f8c5c003a70 2023-05-20T01:27:17.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.934+0000 7f8c71ffb700 1 -- 172.21.15.16:0/282826249 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c6800eb90 con 0x7f8c5c003a70 2023-05-20T01:27:17.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.935+0000 7f8c7933e700 1 -- 172.21.15.16:0/282826249 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c5c003a70 msgr2=0x7f8c5c003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:17.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.935+0000 7f8c7933e700 1 --2- 172.21.15.16:0/282826249 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c5c003a70 0x7f8c5c003e80 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f8c68009b90 tx=0x7f8c5c005880 comp rx=0 tx=0).stop 2023-05-20T01:27:17.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.935+0000 7f8c737fe700 1 -- 172.21.15.16:0/282826249 reap_dead start 2023-05-20T01:27:17.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.935+0000 7f8c7933e700 1 -- 172.21.15.16:0/282826249 shutdown_connections 2023-05-20T01:27:17.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.935+0000 7f8c7933e700 1 -- 172.21.15.16:0/282826249 >> 172.21.15.16:0/282826249 conn(0x7f8c7409c590 msgr2=0x7f8c740d3240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:27:17.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.935+0000 7f8c7933e700 1 -- 172.21.15.16:0/282826249 shutdown_connections 2023-05-20T01:27:17.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.935+0000 7f8c7933e700 1 -- 172.21.15.16:0/282826249 wait complete. 2023-05-20T01:27:17.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.936+0000 7f8c7933e700 1 Processor -- start 2023-05-20T01:27:17.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.936+0000 7f8c7933e700 1 -- start start 2023-05-20T01:27:17.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.937+0000 7f8c7933e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c5c003a70 0x7f8c740cbf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:17.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.937+0000 7f8c7933e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c741f64a0 0x7f8c7415ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:17.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.937+0000 7f8c7933e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8c740b4cc0 0x7f8c740b5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:17.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.937+0000 7f8c7933e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c740b56a0 con 0x7f8c741f64a0 2023-05-20T01:27:17.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.937+0000 7f8c7933e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c740cb270 con 0x7f8c5c003a70 2023-05-20T01:27:17.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.937+0000 7f8c7933e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c741f6910 con 0x7f8c740b4cc0 2023-05-20T01:27:17.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.937+0000 7f8c727fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c741f64a0 0x7f8c7415ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:17.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.937+0000 7f8c727fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c741f64a0 0x7f8c7415ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38720/0 (socket says 172.21.15.16:38720) 2023-05-20T01:27:17.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.937+0000 7f8c727fc700 1 -- 172.21.15.16:0/3198992694 learned_addr learned my addr 172.21.15.16:0/3198992694 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:27:17.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.938+0000 7f8c72ffd700 1 --2- 172.21.15.16:0/3198992694 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c5c003a70 0x7f8c740cbf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:17.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.938+0000 7f8c737fe700 1 --2- 172.21.15.16:0/3198992694 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8c740b4cc0 0x7f8c740b5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:17.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.938+0000 7f8c727fc700 1 -- 172.21.15.16:0/3198992694 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8c740b4cc0 msgr2=0x7f8c740b5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:17.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.938+0000 7f8c727fc700 1 --2- 172.21.15.16:0/3198992694 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8c740b4cc0 0x7f8c740b5160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:17.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.938+0000 7f8c727fc700 1 -- 172.21.15.16:0/3198992694 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c5c003a70 msgr2=0x7f8c740cbf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:17.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.938+0000 7f8c727fc700 1 --2- 172.21.15.16:0/3198992694 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c5c003a70 0x7f8c740cbf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:17.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.938+0000 7f8c727fc700 1 -- 172.21.15.16:0/3198992694 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c740b4380 con 0x7f8c741f64a0 2023-05-20T01:27:17.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.938+0000 7f8c72ffd700 1 --2- 172.21.15.16:0/3198992694 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c5c003a70 0x7f8c740cbf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:27:17.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.938+0000 7f8c727fc700 1 --2- 172.21.15.16:0/3198992694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c741f64a0 0x7f8c7415ec50 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f8c64007340 tx=0x7f8c64007700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:17.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.938+0000 7f8c5b7fe700 1 -- 172.21.15.16:0/3198992694 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c64007820 con 0x7f8c741f64a0 2023-05-20T01:27:17.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.938+0000 7f8c5b7fe700 1 -- 172.21.15.16:0/3198992694 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c6400f8d0 con 0x7f8c741f64a0 2023-05-20T01:27:17.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.938+0000 7f8c7933e700 1 -- 172.21.15.16:0/3198992694 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c7415d0b0 con 0x7f8c741f64a0 2023-05-20T01:27:17.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.938+0000 7f8c5b7fe700 1 -- 172.21.15.16:0/3198992694 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c6400fc60 con 0x7f8c741f64a0 2023-05-20T01:27:17.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.938+0000 7f8c7933e700 1 -- 172.21.15.16:0/3198992694 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c740d4000 con 0x7f8c741f64a0 2023-05-20T01:27:17.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.939+0000 7f8c7933e700 1 -- 172.21.15.16:0/3198992694 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c740cb0c0 con 0x7f8c741f64a0 2023-05-20T01:27:17.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.939+0000 7f8c5b7fe700 1 -- 172.21.15.16:0/3198992694 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8c6401d360 con 0x7f8c741f64a0 2023-05-20T01:27:17.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.940+0000 7f8c5b7fe700 1 --2- 172.21.15.16:0/3198992694 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f8c6006ace0 0x7f8c6006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:17.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.940+0000 7f8c72ffd700 1 --2- 172.21.15.16:0/3198992694 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f8c6006ace0 0x7f8c6006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:17.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.940+0000 7f8c5b7fe700 1 -- 172.21.15.16:0/3198992694 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6987+0+0 (secure 0 0 0) 0x7f8c64088fd0 con 0x7f8c741f64a0 2023-05-20T01:27:17.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.940+0000 7f8c72ffd700 1 --2- 172.21.15.16:0/3198992694 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f8c6006ace0 0x7f8c6006d190 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f8c68009b30 tx=0x7f8c68008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:17.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:17.944+0000 7f8c5b7fe700 1 -- 172.21.15.16:0/3198992694 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f8c64053ce0 con 0x7f8c741f64a0 2023-05-20T01:27:18.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.190+0000 7f8c7933e700 1 -- 172.21.15.16:0/3198992694 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8c740cf510 con 0x7f8c741f64a0 2023-05-20T01:27:18.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.190+0000 7f8c5b7fe700 1 -- 172.21.15.16:0/3198992694 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 84 v84) v1 ==== 94+0+4672 (secure 0 0 0) 0x7f8c64057300 con 0x7f8c741f64a0 2023-05-20T01:27:18.195 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:27:18.195 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":84,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7069,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/3619911175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":3619911175},{"type":"v1","addr":"172.21.15.16:6837","nonce":3619911175}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73},{"gid":7081,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/959638003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":959638003},{"type":"v1","addr":"172.21.15.132:6834","nonce":959638003}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73},{"gid":7084,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2596482253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":2596482253},{"type":"v1","addr":"172.21.15.132:6835","nonce":2596482253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73},{"gid":8205,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/1584803225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1584803225},{"type":"v1","addr":"172.21.15.16:6836","nonce":1584803225}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":84}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-20T01:27:10.291795+0000","modified":"2023-05-20T01:27:11.539644+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":117,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[16,17],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-05-20T01:27:18.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.193+0000 7f8c7933e700 1 -- 172.21.15.16:0/3198992694 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f8c6006ace0 msgr2=0x7f8c6006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:18.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.193+0000 7f8c7933e700 1 --2- 172.21.15.16:0/3198992694 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f8c6006ace0 0x7f8c6006d190 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f8c68009b30 tx=0x7f8c68008030 comp rx=0 tx=0).stop 2023-05-20T01:27:18.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.193+0000 7f8c7933e700 1 -- 172.21.15.16:0/3198992694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c741f64a0 msgr2=0x7f8c7415ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:18.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.193+0000 7f8c7933e700 1 --2- 172.21.15.16:0/3198992694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c741f64a0 0x7f8c7415ec50 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f8c64007340 tx=0x7f8c64007700 comp rx=0 tx=0).stop 2023-05-20T01:27:18.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.193+0000 7f8c7933e700 1 -- 172.21.15.16:0/3198992694 shutdown_connections 2023-05-20T01:27:18.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.193+0000 7f8c7933e700 1 --2- 172.21.15.16:0/3198992694 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f8c6006ace0 0x7f8c6006d190 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:18.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.193+0000 7f8c7933e700 1 --2- 172.21.15.16:0/3198992694 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8c740b4cc0 0x7f8c740b5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:18.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.193+0000 7f8c7933e700 1 --2- 172.21.15.16:0/3198992694 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c5c003a70 0x7f8c740cbf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:18.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.193+0000 7f8c7933e700 1 --2- 172.21.15.16:0/3198992694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c741f64a0 0x7f8c7415ec50 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:18.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.193+0000 7f8c7933e700 1 -- 172.21.15.16:0/3198992694 >> 172.21.15.16:0/3198992694 conn(0x7f8c7409c590 msgr2=0x7f8c7415c820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:27:18.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.193+0000 7f8c7933e700 1 -- 172.21.15.16:0/3198992694 shutdown_connections 2023-05-20T01:27:18.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.193+0000 7f8c7933e700 1 -- 172.21.15.16:0/3198992694 wait complete. 2023-05-20T01:27:18.199 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 84 2023-05-20T01:27:18.209 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-20T01:27:18.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.329+0000 7f3d001b6700 1 Processor -- start 2023-05-20T01:27:18.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.329+0000 7f3d001b6700 1 -- start start 2023-05-20T01:27:18.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.330+0000 7f3d001b6700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f3cf806f450 0x7f3cf80cf890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:18.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.330+0000 7f3d001b6700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cf80cc510 con 0x7f3cf806a510 2023-05-20T01:27:18.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.330+0000 7f3d001b6700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cf8071ed0 con 0x7f3cf806d4b0 2023-05-20T01:27:18.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.330+0000 7f3d001b6700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cf80721b0 con 0x7f3cf806f450 2023-05-20T01:27:18.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.330+0000 7f3cfe753700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f3cf806a510 0x7f3cf80cc9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50258/0 (socket says 172.21.15.16:50258) 2023-05-20T01:27:18.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.330+0000 7f3cfe753700 1 -- 172.21.15.16:0/287940429 learned_addr learned my addr 172.21.15.16:0/287940429 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:27:18.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.331+0000 7f3cfcf50700 1 -- 172.21.15.16:0/287940429 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3147244762 0 0) 0x7f3cf80cc510 con 0x7f3cf806a510 2023-05-20T01:27:18.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.331+0000 7f3cfdf52700 1 --2- 172.21.15.16:0/287940429 >> v2:172.21.15.132:3300/0 conn(0x7f3cf806f450 0x7f3cf80cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:18.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.331+0000 7f3cfcf50700 1 -- 172.21.15.16:0/287940429 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ce40035b0 con 0x7f3cf806a510 2023-05-20T01:27:18.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.331+0000 7f3cfcf50700 1 -- 172.21.15.16:0/287940429 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 190993234 0 0) 0x7f3ce40035b0 con 0x7f3cf806a510 2023-05-20T01:27:18.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.331+0000 7f3cfcf50700 1 -- 172.21.15.16:0/287940429 >> v2:172.21.15.132:3300/0 conn(0x7f3cf806f450 msgr2=0x7f3cf80cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:18.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.331+0000 7f3cfcf50700 1 --2- 172.21.15.16:0/287940429 >> v2:172.21.15.132:3300/0 conn(0x7f3cf806f450 0x7f3cf80cf890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:18.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.331+0000 7f3cfcf50700 1 -- 172.21.15.16:0/287940429 >> v1:172.21.15.132:6789/0 conn(0x7f3cf806d4b0 legacy=0x7f3cf8161080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:18.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.331+0000 7f3cfcf50700 1 -- 172.21.15.16:0/287940429 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cf80cc1a0 con 0x7f3cf806a510 2023-05-20T01:27:18.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.331+0000 7f3cfcf50700 1 -- 172.21.15.16:0/287940429 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7f3cf00019f0 con 0x7f3cf806a510 2023-05-20T01:27:18.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.331+0000 7f3cfcf50700 1 -- 172.21.15.16:0/287940429 >> v1:172.21.15.16:6789/0 conn(0x7f3cf806a510 legacy=0x7f3cf80cc9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:18.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.331+0000 7f3cfcf50700 1 --2- 172.21.15.16:0/287940429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ce4003a70 0x7f3ce4003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:18.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.331+0000 7f3cfcf50700 1 --2- 172.21.15.16:0/287940429 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3ce4004850 0x7f3ce4004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:18.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.331+0000 7f3cfcf50700 1 --2- 172.21.15.16:0/287940429 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3ce4005250 0x7f3ce400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:18.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.331+0000 7f3cfcf50700 1 -- 172.21.15.16:0/287940429 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cf80721b0 con 0x7f3ce4003a70 2023-05-20T01:27:18.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.331+0000 7f3cfcf50700 1 -- 172.21.15.16:0/287940429 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce400c190 con 0x7f3ce4004850 2023-05-20T01:27:18.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.331+0000 7f3cfcf50700 1 -- 172.21.15.16:0/287940429 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce400c350 con 0x7f3ce4005250 2023-05-20T01:27:18.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.331+0000 7f3cfdf52700 1 --2- 172.21.15.16:0/287940429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ce4003a70 0x7f3ce4003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:18.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.331+0000 7f3cfd751700 1 --2- 172.21.15.16:0/287940429 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3ce4004850 0x7f3ce4004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:18.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.331+0000 7f3cfe753700 1 --2- 172.21.15.16:0/287940429 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3ce4005250 0x7f3ce400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:18.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.331+0000 7f3cfdf52700 1 -- 172.21.15.16:0/287940429 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3ce4005250 msgr2=0x7f3ce400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:18.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.331+0000 7f3cfdf52700 1 --2- 172.21.15.16:0/287940429 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3ce4005250 0x7f3ce400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:18.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.331+0000 7f3cfdf52700 1 -- 172.21.15.16:0/287940429 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3ce4004850 msgr2=0x7f3ce4004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:18.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.331+0000 7f3cfdf52700 1 --2- 172.21.15.16:0/287940429 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3ce4004850 0x7f3ce4004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:18.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.331+0000 7f3cfdf52700 1 -- 172.21.15.16:0/287940429 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ce40035b0 con 0x7f3ce4003a70 2023-05-20T01:27:18.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.331+0000 7f3cfe753700 1 -- 172.21.15.16:0/287940429 reap_dead start 2023-05-20T01:27:18.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.332+0000 7f3cfdf52700 1 --2- 172.21.15.16:0/287940429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ce4003a70 0x7f3ce4003e80 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f3ce8009b30 tx=0x7f3ce4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=180d5ca7ca471ae3 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:18.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.332+0000 7f3cfcf50700 1 -- 172.21.15.16:0/287940429 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ce800e6c0 con 0x7f3ce4003a70 2023-05-20T01:27:18.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.332+0000 7f3cfcf50700 1 -- 172.21.15.16:0/287940429 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ce800e820 con 0x7f3ce4003a70 2023-05-20T01:27:18.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.332+0000 7f3cfcf50700 1 -- 172.21.15.16:0/287940429 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ce800ebb0 con 0x7f3ce4003a70 2023-05-20T01:27:18.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.332+0000 7f3d001b6700 1 -- 172.21.15.16:0/287940429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ce4003a70 msgr2=0x7f3ce4003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:18.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.332+0000 7f3d001b6700 1 --2- 172.21.15.16:0/287940429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ce4003a70 0x7f3ce4003e80 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f3ce8009b30 tx=0x7f3ce4005880 comp rx=0 tx=0).stop 2023-05-20T01:27:18.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.332+0000 7f3d001b6700 1 -- 172.21.15.16:0/287940429 shutdown_connections 2023-05-20T01:27:18.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.332+0000 7f3d001b6700 1 --2- 172.21.15.16:0/287940429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ce4003a70 0x7f3ce4003e80 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:18.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.332+0000 7f3d001b6700 1 -- 172.21.15.16:0/287940429 >> 172.21.15.16:0/287940429 conn(0x7f3cf80cffb0 msgr2=0x7f3cf80d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:27:18.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.332+0000 7f3d001b6700 1 -- 172.21.15.16:0/287940429 shutdown_connections 2023-05-20T01:27:18.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.332+0000 7f3d001b6700 1 -- 172.21.15.16:0/287940429 wait complete. 2023-05-20T01:27:18.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.334+0000 7f3d001b6700 1 Processor -- start 2023-05-20T01:27:18.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.334+0000 7f3d001b6700 1 -- start start 2023-05-20T01:27:18.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.335+0000 7f3d001b6700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3ce4003a70 0x7f3cf80c3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:18.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.335+0000 7f3d001b6700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3cf80b4af0 0x7f3cf814d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:18.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.335+0000 7f3d001b6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cf814db90 0x7f3cf814e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:18.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.335+0000 7f3d001b6700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cf80c6bb0 con 0x7f3cf814db90 2023-05-20T01:27:18.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.335+0000 7f3d001b6700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cf80b4660 con 0x7f3cf80b4af0 2023-05-20T01:27:18.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.335+0000 7f3d001b6700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cf80b3d30 con 0x7f3ce4003a70 2023-05-20T01:27:18.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.335+0000 7f3cfe753700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cf814db90 0x7f3cf814e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:18.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.335+0000 7f3cfe753700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cf814db90 0x7f3cf814e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38742/0 (socket says 172.21.15.16:38742) 2023-05-20T01:27:18.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.335+0000 7f3cfe753700 1 -- 172.21.15.16:0/1727976808 learned_addr learned my addr 172.21.15.16:0/1727976808 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:27:18.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.335+0000 7f3cfd751700 1 --2- 172.21.15.16:0/1727976808 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3cf80b4af0 0x7f3cf814d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:18.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.335+0000 7f3cfdf52700 1 --2- 172.21.15.16:0/1727976808 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3ce4003a70 0x7f3cf80c3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:18.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.335+0000 7f3cfe753700 1 -- 172.21.15.16:0/1727976808 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3ce4003a70 msgr2=0x7f3cf80c3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:18.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.335+0000 7f3cfe753700 1 --2- 172.21.15.16:0/1727976808 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3ce4003a70 0x7f3cf80c3ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:18.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.335+0000 7f3cfe753700 1 -- 172.21.15.16:0/1727976808 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3cf80b4af0 msgr2=0x7f3cf814d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:18.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.335+0000 7f3cfe753700 1 --2- 172.21.15.16:0/1727976808 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3cf80b4af0 0x7f3cf814d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:18.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.335+0000 7f3cfe753700 1 -- 172.21.15.16:0/1727976808 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cf80cf890 con 0x7f3cf814db90 2023-05-20T01:27:18.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.336+0000 7f3cfe753700 1 --2- 172.21.15.16:0/1727976808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cf814db90 0x7f3cf814e000 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f3cf80d0ff0 tx=0x7f3cf000e440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:18.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.336+0000 7f3cee7fc700 1 -- 172.21.15.16:0/1727976808 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf000eb40 con 0x7f3cf814db90 2023-05-20T01:27:18.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.336+0000 7f3cee7fc700 1 -- 172.21.15.16:0/1727976808 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cf000ecc0 con 0x7f3cf814db90 2023-05-20T01:27:18.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.336+0000 7f3cee7fc700 1 -- 172.21.15.16:0/1727976808 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf0013420 con 0x7f3cf814db90 2023-05-20T01:27:18.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.336+0000 7f3d001b6700 1 -- 172.21.15.16:0/1727976808 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cf80cbe50 con 0x7f3cf814db90 2023-05-20T01:27:18.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.336+0000 7f3d001b6700 1 -- 172.21.15.16:0/1727976808 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cf80cfa20 con 0x7f3cf814db90 2023-05-20T01:27:18.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.337+0000 7f3d001b6700 1 -- 172.21.15.16:0/1727976808 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cf80747b0 con 0x7f3cf814db90 2023-05-20T01:27:18.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.337+0000 7f3cee7fc700 1 -- 172.21.15.16:0/1727976808 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3cf00135a0 con 0x7f3cf814db90 2023-05-20T01:27:18.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.338+0000 7f3cee7fc700 1 --2- 172.21.15.16:0/1727976808 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f3cdc06ace0 0x7f3cdc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:18.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.338+0000 7f3cee7fc700 1 -- 172.21.15.16:0/1727976808 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6987+0+0 (secure 0 0 0) 0x7f3cf0089d30 con 0x7f3cf814db90 2023-05-20T01:27:18.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.338+0000 7f3cfdf52700 1 --2- 172.21.15.16:0/1727976808 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f3cdc06ace0 0x7f3cdc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:18.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.338+0000 7f3cfdf52700 1 --2- 172.21.15.16:0/1727976808 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f3cdc06ace0 0x7f3cdc06d190 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f3ce80090b0 tx=0x7f3ce801e040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:18.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.342+0000 7f3cee7fc700 1 -- 172.21.15.16:0/1727976808 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f3cf00549b0 con 0x7f3cf814db90 2023-05-20T01:27:18.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.586+0000 7f3d001b6700 1 -- 172.21.15.16:0/1727976808 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3cf8071c10 con 0x7f3cf814db90 2023-05-20T01:27:18.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.587+0000 7f3cee7fc700 1 -- 172.21.15.16:0/1727976808 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 84 v84) v1 ==== 94+0+4672 (secure 0 0 0) 0x7f3cf0057fd0 con 0x7f3cf814db90 2023-05-20T01:27:18.591 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:27:18.592 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":84,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7069,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/3619911175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":3619911175},{"type":"v1","addr":"172.21.15.16:6837","nonce":3619911175}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73},{"gid":7081,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/959638003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":959638003},{"type":"v1","addr":"172.21.15.132:6834","nonce":959638003}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73},{"gid":7084,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2596482253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":2596482253},{"type":"v1","addr":"172.21.15.132:6835","nonce":2596482253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73},{"gid":8205,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/1584803225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1584803225},{"type":"v1","addr":"172.21.15.16:6836","nonce":1584803225}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":84}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-20T01:27:10.291795+0000","modified":"2023-05-20T01:27:11.539644+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":117,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[16,17],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-05-20T01:27:18.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.589+0000 7f3d001b6700 1 -- 172.21.15.16:0/1727976808 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f3cdc06ace0 msgr2=0x7f3cdc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:18.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.589+0000 7f3d001b6700 1 --2- 172.21.15.16:0/1727976808 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f3cdc06ace0 0x7f3cdc06d190 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f3ce80090b0 tx=0x7f3ce801e040 comp rx=0 tx=0).stop 2023-05-20T01:27:18.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.589+0000 7f3d001b6700 1 -- 172.21.15.16:0/1727976808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cf814db90 msgr2=0x7f3cf814e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:18.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.589+0000 7f3d001b6700 1 --2- 172.21.15.16:0/1727976808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cf814db90 0x7f3cf814e000 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f3cf80d0ff0 tx=0x7f3cf000e440 comp rx=0 tx=0).stop 2023-05-20T01:27:18.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.589+0000 7f3d001b6700 1 -- 172.21.15.16:0/1727976808 shutdown_connections 2023-05-20T01:27:18.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.589+0000 7f3d001b6700 1 --2- 172.21.15.16:0/1727976808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cf814db90 0x7f3cf814e000 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:18.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.589+0000 7f3d001b6700 1 --2- 172.21.15.16:0/1727976808 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f3cdc06ace0 0x7f3cdc06d190 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:18.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.589+0000 7f3d001b6700 1 --2- 172.21.15.16:0/1727976808 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3ce4003a70 0x7f3cf80c3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:18.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.589+0000 7f3d001b6700 1 --2- 172.21.15.16:0/1727976808 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3cf80b4af0 0x7f3cf814d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:18.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.589+0000 7f3d001b6700 1 -- 172.21.15.16:0/1727976808 >> 172.21.15.16:0/1727976808 conn(0x7f3cf80cffb0 msgr2=0x7f3cf806d880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:27:18.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.589+0000 7f3d001b6700 1 -- 172.21.15.16:0/1727976808 shutdown_connections 2023-05-20T01:27:18.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.590+0000 7f3d001b6700 1 -- 172.21.15.16:0/1727976808 wait complete. 2023-05-20T01:27:18.595 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 84 2023-05-20T01:27:18.611 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-20T01:27:18.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.731+0000 7f683073f700 1 Processor -- start 2023-05-20T01:27:18.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.731+0000 7f683073f700 1 -- start start 2023-05-20T01:27:18.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.732+0000 7f683073f700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6828153eb0 0x7f68280ccbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:18.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.732+0000 7f683073f700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f68280cb000 0x7f682815a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:18.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.732+0000 7f683073f700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f682815ac80 con 0x7f682806cb90 2023-05-20T01:27:18.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.732+0000 7f683073f700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6828159d30 con 0x7f6828153eb0 2023-05-20T01:27:18.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.732+0000 7f683073f700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68280cb750 con 0x7f68280cb000 2023-05-20T01:27:18.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.733+0000 7f682e4db700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6828153eb0 0x7f68280ccbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:18.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.733+0000 7f682e4db700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6828153eb0 0x7f68280ccbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38746/0 (socket says 172.21.15.16:38746) 2023-05-20T01:27:18.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.733+0000 7f682e4db700 1 -- 172.21.15.16:0/1894299036 learned_addr learned my addr 172.21.15.16:0/1894299036 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:27:18.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.733+0000 7f682dcda700 1 --2- 172.21.15.16:0/1894299036 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f68280cb000 0x7f682815a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:18.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.733+0000 7f682e4db700 1 -- 172.21.15.16:0/1894299036 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f68280cb000 msgr2=0x7f682815a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:18.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.733+0000 7f682e4db700 1 --2- 172.21.15.16:0/1894299036 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f68280cb000 0x7f682815a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:18.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.733+0000 7f682e4db700 1 -- 172.21.15.16:0/1894299036 >> v1:172.21.15.132:6789/0 conn(0x7f682806cb90 legacy=0x7f682815d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:18.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.733+0000 7f682e4db700 1 -- 172.21.15.16:0/1894299036 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68280cba30 con 0x7f6828153eb0 2023-05-20T01:27:18.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.733+0000 7f682e4db700 1 --2- 172.21.15.16:0/1894299036 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6828153eb0 0x7f68280ccbd0 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f681c0098f0 tx=0x7f681c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84e7214430d7bd86 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:18.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.733+0000 7f682d4d9700 1 -- 172.21.15.16:0/1894299036 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f681c011830 con 0x7f6828153eb0 2023-05-20T01:27:18.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.733+0000 7f682d4d9700 1 -- 172.21.15.16:0/1894299036 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f681c011990 con 0x7f6828153eb0 2023-05-20T01:27:18.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.733+0000 7f682d4d9700 1 -- 172.21.15.16:0/1894299036 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f681c011d20 con 0x7f6828153eb0 2023-05-20T01:27:18.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.734+0000 7f683073f700 1 -- 172.21.15.16:0/1894299036 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6828153eb0 msgr2=0x7f68280ccbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:18.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.734+0000 7f683073f700 1 --2- 172.21.15.16:0/1894299036 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6828153eb0 0x7f68280ccbd0 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f681c0098f0 tx=0x7f681c009c00 comp rx=0 tx=0).stop 2023-05-20T01:27:18.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.734+0000 7f683073f700 1 -- 172.21.15.16:0/1894299036 shutdown_connections 2023-05-20T01:27:18.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.734+0000 7f683073f700 1 --2- 172.21.15.16:0/1894299036 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6828153eb0 0x7f68280ccbd0 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:18.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.734+0000 7f683073f700 1 --2- 172.21.15.16:0/1894299036 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f68280cb000 0x7f682815a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:18.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.734+0000 7f683073f700 1 -- 172.21.15.16:0/1894299036 >> 172.21.15.16:0/1894299036 conn(0x7f68280661b0 msgr2=0x7f6828066580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:27:18.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.734+0000 7f683073f700 1 -- 172.21.15.16:0/1894299036 shutdown_connections 2023-05-20T01:27:18.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.734+0000 7f683073f700 1 -- 172.21.15.16:0/1894299036 wait complete. 2023-05-20T01:27:18.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.735+0000 7f683073f700 1 Processor -- start 2023-05-20T01:27:18.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.735+0000 7f683073f700 1 -- start start 2023-05-20T01:27:18.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.736+0000 7f683073f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f682806cb90 0x7f682813fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:18.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.736+0000 7f683073f700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f68280cb000 0x7f68280c9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:18.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.736+0000 7f683073f700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6828153eb0 0x7f68281f3730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:18.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.736+0000 7f683073f700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f682815c7c0 con 0x7f682806cb90 2023-05-20T01:27:18.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.736+0000 7f683073f700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68281406a0 con 0x7f6828153eb0 2023-05-20T01:27:18.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.736+0000 7f683073f700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f682815be90 con 0x7f68280cb000 2023-05-20T01:27:18.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.736+0000 7f682e4db700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f682806cb90 0x7f682813fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:18.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.736+0000 7f682e4db700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f682806cb90 0x7f682813fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38752/0 (socket says 172.21.15.16:38752) 2023-05-20T01:27:18.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.736+0000 7f682e4db700 1 -- 172.21.15.16:0/1829755582 learned_addr learned my addr 172.21.15.16:0/1829755582 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:27:18.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.736+0000 7f682ecdc700 1 --2- 172.21.15.16:0/1829755582 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6828153eb0 0x7f68281f3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:18.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.736+0000 7f682dcda700 1 --2- 172.21.15.16:0/1829755582 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f68280cb000 0x7f68280c9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:18.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.736+0000 7f682e4db700 1 -- 172.21.15.16:0/1829755582 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f68280cb000 msgr2=0x7f68280c9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:18.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.736+0000 7f682e4db700 1 --2- 172.21.15.16:0/1829755582 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f68280cb000 0x7f68280c9ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:18.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.736+0000 7f682e4db700 1 -- 172.21.15.16:0/1829755582 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6828153eb0 msgr2=0x7f68281f3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:18.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.736+0000 7f682e4db700 1 --2- 172.21.15.16:0/1829755582 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6828153eb0 0x7f68281f3730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:18.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.736+0000 7f682e4db700 1 -- 172.21.15.16:0/1829755582 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f681c009550 con 0x7f682806cb90 2023-05-20T01:27:18.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.736+0000 7f682e4db700 1 --2- 172.21.15.16:0/1829755582 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f682806cb90 0x7f682813fdc0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f681c009c40 tx=0x7f681c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:18.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.737+0000 7f681affd700 1 -- 172.21.15.16:0/1829755582 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f681c0036a0 con 0x7f682806cb90 2023-05-20T01:27:18.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.737+0000 7f681affd700 1 -- 172.21.15.16:0/1829755582 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f681c01e070 con 0x7f682806cb90 2023-05-20T01:27:18.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.737+0000 7f683073f700 1 -- 172.21.15.16:0/1829755582 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6828159f40 con 0x7f682806cb90 2023-05-20T01:27:18.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.737+0000 7f681affd700 1 -- 172.21.15.16:0/1829755582 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f681c003a60 con 0x7f682806cb90 2023-05-20T01:27:18.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.737+0000 7f683073f700 1 -- 172.21.15.16:0/1829755582 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68280cbd10 con 0x7f682806cb90 2023-05-20T01:27:18.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.738+0000 7f6818ff9700 1 -- 172.21.15.16:0/1829755582 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68040052e0 con 0x7f682806cb90 2023-05-20T01:27:18.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.738+0000 7f681affd700 1 -- 172.21.15.16:0/1829755582 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f681c003bc0 con 0x7f682806cb90 2023-05-20T01:27:18.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.738+0000 7f681affd700 1 --2- 172.21.15.16:0/1829755582 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f680c06ace0 0x7f680c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:18.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.738+0000 7f681affd700 1 -- 172.21.15.16:0/1829755582 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6987+0+0 (secure 0 0 0) 0x7f681c08a740 con 0x7f682806cb90 2023-05-20T01:27:18.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.738+0000 7f682dcda700 1 --2- 172.21.15.16:0/1829755582 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f680c06ace0 0x7f680c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:18.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.739+0000 7f682dcda700 1 --2- 172.21.15.16:0/1829755582 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f680c06ace0 0x7f680c06d190 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f682814b3d0 tx=0x7f6824008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:18.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.743+0000 7f681affd700 1 -- 172.21.15.16:0/1829755582 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f681c055590 con 0x7f682806cb90 2023-05-20T01:27:18.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.983+0000 7f6818ff9700 1 -- 172.21.15.16:0/1829755582 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6804005590 con 0x7f682806cb90 2023-05-20T01:27:18.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.984+0000 7f681affd700 1 -- 172.21.15.16:0/1829755582 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 84 v84) v1 ==== 94+0+4672 (secure 0 0 0) 0x7f681c058bb0 con 0x7f682806cb90 2023-05-20T01:27:18.989 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:27:18.989 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":84,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7069,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/3619911175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":3619911175},{"type":"v1","addr":"172.21.15.16:6837","nonce":3619911175}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73},{"gid":7081,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/959638003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":959638003},{"type":"v1","addr":"172.21.15.132:6834","nonce":959638003}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73},{"gid":7084,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2596482253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":2596482253},{"type":"v1","addr":"172.21.15.132:6835","nonce":2596482253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73},{"gid":8205,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/1584803225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1584803225},{"type":"v1","addr":"172.21.15.16:6836","nonce":1584803225}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":84}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-20T01:27:10.291795+0000","modified":"2023-05-20T01:27:11.539644+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":117,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[16,17],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-05-20T01:27:18.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.986+0000 7f6818ff9700 1 -- 172.21.15.16:0/1829755582 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f680c06ace0 msgr2=0x7f680c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:18.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.986+0000 7f6818ff9700 1 --2- 172.21.15.16:0/1829755582 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f680c06ace0 0x7f680c06d190 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f682814b3d0 tx=0x7f6824008040 comp rx=0 tx=0).stop 2023-05-20T01:27:18.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.986+0000 7f6818ff9700 1 -- 172.21.15.16:0/1829755582 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f682806cb90 msgr2=0x7f682813fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:18.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.986+0000 7f6818ff9700 1 --2- 172.21.15.16:0/1829755582 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f682806cb90 0x7f682813fdc0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f681c009c40 tx=0x7f681c004750 comp rx=0 tx=0).stop 2023-05-20T01:27:18.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.986+0000 7f6818ff9700 1 -- 172.21.15.16:0/1829755582 shutdown_connections 2023-05-20T01:27:18.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.986+0000 7f6818ff9700 1 --2- 172.21.15.16:0/1829755582 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6828153eb0 0x7f68281f3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:18.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.986+0000 7f6818ff9700 1 --2- 172.21.15.16:0/1829755582 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f682806cb90 0x7f682813fdc0 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:18.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.986+0000 7f6818ff9700 1 --2- 172.21.15.16:0/1829755582 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f680c06ace0 0x7f680c06d190 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:18.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.986+0000 7f6818ff9700 1 --2- 172.21.15.16:0/1829755582 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f68280cb000 0x7f68280c9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:18.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.986+0000 7f6818ff9700 1 -- 172.21.15.16:0/1829755582 >> 172.21.15.16:0/1829755582 conn(0x7f68280661b0 msgr2=0x7f6828071d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:27:18.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.986+0000 7f6818ff9700 1 -- 172.21.15.16:0/1829755582 shutdown_connections 2023-05-20T01:27:18.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:18.986+0000 7f6818ff9700 1 -- 172.21.15.16:0/1829755582 wait complete. 2023-05-20T01:27:18.992 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 84 2023-05-20T01:27:19.002 DEBUG:teuthology.orchestra.run.smithi016:> cephfs-table-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 cephfs:0 reset session 2023-05-20T01:27:19.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.040+0000 7fa532a4e2c0 1 -- start start 2023-05-20T01:27:19.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.040+0000 7fa532a4e2c0 1 Processor -- start 2023-05-20T01:27:19.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.040+0000 7fa532a4e2c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a1ff379000 0x55a1ff413b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:19.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.040+0000 7fa532a4e2c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a1ff378c00 0x55a1ff413600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:19.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.040+0000 7fa532a4e2c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a1ff378800 0x55a1ff413080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:19.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.040+0000 7fa532a4e2c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1ff3b5cc0 con 0x55a1ff379000 2023-05-20T01:27:19.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.040+0000 7fa532a4e2c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1ff3b5400 con 0x55a1ff378c00 2023-05-20T01:27:19.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.040+0000 7fa532a4e2c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a1ff3b4a00 con 0x55a1ff378800 2023-05-20T01:27:19.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.040+0000 7fa5309fa700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a1ff379000 0x55a1ff413b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:19.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.040+0000 7fa5309fa700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a1ff379000 0x55a1ff413b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38766/0 (socket says 172.21.15.16:38766) 2023-05-20T01:27:19.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.040+0000 7fa5309fa700 1 -- 172.21.15.16:0/1748762220 learned_addr learned my addr 172.21.15.16:0/1748762220 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:27:19.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.040+0000 7fa5301f9700 1 --2- 172.21.15.16:0/1748762220 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a1ff378c00 0x55a1ff413600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:19.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.041+0000 7fa5311fb700 1 --2- 172.21.15.16:0/1748762220 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a1ff378800 0x55a1ff413080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:19.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.041+0000 7fa5309fa700 1 -- 172.21.15.16:0/1748762220 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a1ff378800 msgr2=0x55a1ff413080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:19.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.041+0000 7fa5309fa700 1 --2- 172.21.15.16:0/1748762220 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a1ff378800 0x55a1ff413080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.041+0000 7fa5309fa700 1 -- 172.21.15.16:0/1748762220 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a1ff378c00 msgr2=0x55a1ff413600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:19.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.041+0000 7fa5309fa700 1 --2- 172.21.15.16:0/1748762220 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a1ff378c00 0x55a1ff413600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.041+0000 7fa5309fa700 1 -- 172.21.15.16:0/1748762220 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a1ff39cd00 con 0x55a1ff379000 2023-05-20T01:27:19.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.041+0000 7fa5311fb700 1 --2- 172.21.15.16:0/1748762220 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a1ff378800 0x55a1ff413080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:27:19.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.041+0000 7fa5309fa700 1 --2- 172.21.15.16:0/1748762220 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a1ff379000 0x55a1ff413b80 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x55a2001ea630 tx=0x55a200111320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0248fe4a0374e03 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:19.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.041+0000 7fa52e1f5700 1 -- 172.21.15.16:0/1748762220 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55a1ff3ad080 con 0x55a1ff379000 2023-05-20T01:27:19.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.041+0000 7fa52e1f5700 1 -- 172.21.15.16:0/1748762220 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a1ff3cb800 con 0x55a1ff379000 2023-05-20T01:27:19.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.041+0000 7fa52e1f5700 1 -- 172.21.15.16:0/1748762220 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55a1ff3ad600 con 0x55a1ff379000 2023-05-20T01:27:19.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.041+0000 7fa532a4e2c0 1 -- 172.21.15.16:0/1748762220 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a1ff39d6c0 con 0x55a1ff379000 2023-05-20T01:27:19.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.041+0000 7fa532a4e2c0 1 -- 172.21.15.16:0/1748762220 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55a1ff39d520 con 0x55a1ff379000 2023-05-20T01:27:19.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.041+0000 7fa532a4e2c0 4 waiting for MDS map... 2023-05-20T01:27:19.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.041+0000 7fa52e1f5700 1 -- 172.21.15.16:0/1748762220 <== mon.0 v2:172.21.15.16:3300/0 4 ==== osd_map(119..119 src has 1..119) v4 ==== 6987+0+0 (secure 0 0 0) 0x55a1ff3bac40 con 0x55a1ff379000 2023-05-20T01:27:19.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.041+0000 7fa52e1f5700 1 -- 172.21.15.16:0/1748762220 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 84) v1 ==== 3126+0+0 (secure 0 0 0) 0x55a1ff3bef00 con 0x55a1ff379000 2023-05-20T01:27:19.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.041+0000 7fa532a4e2c0 4 Got MDS map 84 2023-05-20T01:27:19.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.041+0000 7fa532a4e2c0 10 main: main 2023-05-20T01:27:19.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.041+0000 7fa532a4e2c0 4 main: connecting to RADOS... 2023-05-20T01:27:19.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.042+0000 7fa532a4e2c0 1 Processor -- start 2023-05-20T01:27:19.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.042+0000 7fa532a4e2c0 1 -- start start 2023-05-20T01:27:19.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.042+0000 7fa532a4e2c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a1ff379c00 0x55a1ff412b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:19.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.042+0000 7fa532a4e2c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a1ff68a000 0x55a1ff412580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:19.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.042+0000 7fa532a4e2c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a1ff68a400 0x55a1ff414100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:19.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.042+0000 7fa532a4e2c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1ff3b4b40 con 0x55a1ff379c00 2023-05-20T01:27:19.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.042+0000 7fa532a4e2c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1ff3b4a00 con 0x55a1ff68a000 2023-05-20T01:27:19.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.042+0000 7fa532a4e2c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a1ff3b5400 con 0x55a1ff68a400 2023-05-20T01:27:19.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.043+0000 7fa5311fb700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a1ff379c00 0x55a1ff412b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:19.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.043+0000 7fa5311fb700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a1ff379c00 0x55a1ff412b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38772/0 (socket says 172.21.15.16:38772) 2023-05-20T01:27:19.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.043+0000 7fa5311fb700 1 -- 172.21.15.16:0/803036556 learned_addr learned my addr 172.21.15.16:0/803036556 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:27:19.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.043+0000 7fa5301f9700 1 --2- 172.21.15.16:0/803036556 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a1ff68a400 0x55a1ff414100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:19.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.043+0000 7fa5309fa700 1 --2- 172.21.15.16:0/803036556 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a1ff68a000 0x55a1ff412580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:19.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.043+0000 7fa5311fb700 1 -- 172.21.15.16:0/803036556 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a1ff68a400 msgr2=0x55a1ff414100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:19.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.043+0000 7fa5311fb700 1 --2- 172.21.15.16:0/803036556 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a1ff68a400 0x55a1ff414100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.043+0000 7fa5311fb700 1 -- 172.21.15.16:0/803036556 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a1ff68a000 msgr2=0x55a1ff412580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:19.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.043+0000 7fa5311fb700 1 --2- 172.21.15.16:0/803036556 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a1ff68a000 0x55a1ff412580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.043+0000 7fa5311fb700 1 -- 172.21.15.16:0/803036556 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a1ff39d380 con 0x55a1ff379c00 2023-05-20T01:27:19.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.043+0000 7fa5311fb700 1 --2- 172.21.15.16:0/803036556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a1ff379c00 0x55a1ff412b00 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x55a2001ebb60 tx=0x55a2001fac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b2c36561a976d09 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:19.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.043+0000 7fa52c1f1700 1 -- 172.21.15.16:0/803036556 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55a1ff3ad080 con 0x55a1ff379c00 2023-05-20T01:27:19.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.043+0000 7fa52c1f1700 1 -- 172.21.15.16:0/803036556 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a1ff3ca480 con 0x55a1ff379c00 2023-05-20T01:27:19.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.043+0000 7fa52c1f1700 1 -- 172.21.15.16:0/803036556 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55a1ff3ad600 con 0x55a1ff379c00 2023-05-20T01:27:19.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.044+0000 7fa532a4e2c0 1 -- 172.21.15.16:0/803036556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a1ff379c00 msgr2=0x55a1ff412b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:19.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.044+0000 7fa532a4e2c0 1 --2- 172.21.15.16:0/803036556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a1ff379c00 0x55a1ff412b00 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x55a2001ebb60 tx=0x55a2001fac00 comp rx=0 tx=0).stop 2023-05-20T01:27:19.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.044+0000 7fa532a4e2c0 1 -- 172.21.15.16:0/803036556 shutdown_connections 2023-05-20T01:27:19.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.044+0000 7fa532a4e2c0 1 --2- 172.21.15.16:0/803036556 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a1ff68a400 0x55a1ff414100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.044+0000 7fa532a4e2c0 1 --2- 172.21.15.16:0/803036556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a1ff379c00 0x55a1ff412b00 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.044+0000 7fa532a4e2c0 1 --2- 172.21.15.16:0/803036556 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a1ff68a000 0x55a1ff412580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.044+0000 7fa532a4e2c0 1 -- 172.21.15.16:0/803036556 >> 172.21.15.16:0/803036556 conn(0x55a1ff378400 msgr2=0x55a2000f5800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:27:19.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.044+0000 7fa532a4e2c0 1 -- 172.21.15.16:0/803036556 shutdown_connections 2023-05-20T01:27:19.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.044+0000 7fa532a4e2c0 1 -- 172.21.15.16:0/803036556 wait complete. 2023-05-20T01:27:19.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.044+0000 7fa532a4e2c0 1 Processor -- start 2023-05-20T01:27:19.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.044+0000 7fa532a4e2c0 1 -- start start 2023-05-20T01:27:19.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.045+0000 7fa532a4e2c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a1ff378400 0x55a1ff412b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:19.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.045+0000 7fa532a4e2c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a1ff6ae000 0x55a1ff412580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:19.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.045+0000 7fa532a4e2c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a1ff6ae400 0x55a1ff414100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:19.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.045+0000 7fa532a4e2c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1ff3b4c80 con 0x55a1ff6ae400 2023-05-20T01:27:19.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.045+0000 7fa532a4e2c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1ff3b4000 con 0x55a1ff6ae000 2023-05-20T01:27:19.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.045+0000 7fa532a4e2c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a1ff3b5180 con 0x55a1ff378400 2023-05-20T01:27:19.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.045+0000 7fa5301f9700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a1ff6ae400 0x55a1ff414100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:19.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.045+0000 7fa5301f9700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a1ff6ae400 0x55a1ff414100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38782/0 (socket says 172.21.15.16:38782) 2023-05-20T01:27:19.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.045+0000 7fa5301f9700 1 -- 172.21.15.16:0/781585009 learned_addr learned my addr 172.21.15.16:0/781585009 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:27:19.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.045+0000 7fa5309fa700 1 --2- 172.21.15.16:0/781585009 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a1ff6ae000 0x55a1ff412580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:19.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.045+0000 7fa5311fb700 1 --2- 172.21.15.16:0/781585009 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a1ff378400 0x55a1ff412b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:19.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.045+0000 7fa5301f9700 1 -- 172.21.15.16:0/781585009 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a1ff378400 msgr2=0x55a1ff412b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:19.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.045+0000 7fa5301f9700 1 --2- 172.21.15.16:0/781585009 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a1ff378400 0x55a1ff412b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.045+0000 7fa5301f9700 1 -- 172.21.15.16:0/781585009 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a1ff6ae000 msgr2=0x55a1ff412580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:19.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.045+0000 7fa5301f9700 1 --2- 172.21.15.16:0/781585009 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a1ff6ae000 0x55a1ff412580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.045+0000 7fa5301f9700 1 -- 172.21.15.16:0/781585009 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a1ff39d1e0 con 0x55a1ff6ae400 2023-05-20T01:27:19.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.046+0000 7fa5301f9700 1 --2- 172.21.15.16:0/781585009 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a1ff6ae400 0x55a1ff414100 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x55a1ff68a6c0 tx=0x55a2001fad20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:19.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.046+0000 7fa52a9ee700 1 -- 172.21.15.16:0/781585009 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55a1ff3ad080 con 0x55a1ff6ae400 2023-05-20T01:27:19.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.046+0000 7fa52a9ee700 1 -- 172.21.15.16:0/781585009 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a2001f6780 con 0x55a1ff6ae400 2023-05-20T01:27:19.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.046+0000 7fa52a9ee700 1 -- 172.21.15.16:0/781585009 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55a1ff3ad600 con 0x55a1ff6ae400 2023-05-20T01:27:19.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.046+0000 7fa532a4e2c0 1 -- 172.21.15.16:0/781585009 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55a1ff39dd40 con 0x55a1ff6ae400 2023-05-20T01:27:19.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.046+0000 7fa532a4e2c0 1 -- 172.21.15.16:0/781585009 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a1ff39cd00 con 0x55a1ff6ae400 2023-05-20T01:27:19.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.046+0000 7fa532a4e2c0 4 main: resolving pool 15 2023-05-20T01:27:19.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.047+0000 7fa52a9ee700 1 -- 172.21.15.16:0/781585009 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x55a1ff3da840 con 0x55a1ff6ae400 2023-05-20T01:27:19.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.047+0000 7fa52a9ee700 1 --2- 172.21.15.16:0/781585009 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x55a1ff6b0400 0x55a1ff7a2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:19.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.047+0000 7fa52a9ee700 1 -- 172.21.15.16:0/781585009 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6987+0+0 (secure 0 0 0) 0x55a1ff3baa80 con 0x55a1ff6ae400 2023-05-20T01:27:19.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.047+0000 7fa5311fb700 1 --2- 172.21.15.16:0/781585009 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x55a1ff6b0400 0x55a1ff7a2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:19.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.047+0000 7fa532a4e2c0 4 main: creating IoCtx.. 2023-05-20T01:27:19.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.047+0000 7fa532a4e2c0 1 --2- 172.21.15.16:0/781585009 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x55a1ff6ae800 0x55a1ff414680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:19.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.047+0000 7fa532a4e2c0 1 -- 172.21.15.16:0/781585009 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:1 15.b 15:d0630e4c:::mds0_sessionmap:head [omap-clear,omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x55a200150700 con 0x55a1ff6ae800 2023-05-20T01:27:19.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.048+0000 7fa5309fa700 1 --2- 172.21.15.16:0/781585009 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x55a1ff6ae800 0x55a1ff414680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:19.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.048+0000 7fa5311fb700 1 --2- 172.21.15.16:0/781585009 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x55a1ff6b0400 0x55a1ff7a2000 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x55a2001ebb60 tx=0x55a200111440 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:19.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.048+0000 7fa5309fa700 1 --2- 172.21.15.16:0/781585009 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x55a1ff6ae800 0x55a1ff414680 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:19.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.049+0000 7fa5309fa700 1 -- 172.21.15.16:0/781585009 <== osd.2 v2:172.21.15.16:6816/293187030 1 ==== osd_op_reply(1 mds0_sessionmap [omap-clear,omap-set-header] v119'5 uv5 ondisk = 0) v8 ==== 201+0+0 (crc 0 0 0) 0x55a200142fc0 con 0x55a1ff6ae800 2023-05-20T01:27:19.071 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-05-20T01:27:19.071 INFO:teuthology.orchestra.run.smithi016.stdout: "0": { 2023-05-20T01:27:19.071 INFO:teuthology.orchestra.run.smithi016.stdout: "data": {}, 2023-05-20T01:27:19.071 INFO:teuthology.orchestra.run.smithi016.stdout: "result": 0 2023-05-20T01:27:19.071 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:27:19.072 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-05-20T01:27:19.072 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:27:19.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.049+0000 7fa532a4e2c0 1 -- 172.21.15.16:0/781585009 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x55a1ff6ae800 msgr2=0x55a1ff414680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:19.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.049+0000 7fa532a4e2c0 1 --2- 172.21.15.16:0/781585009 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x55a1ff6ae800 0x55a1ff414680 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.049+0000 7fa532a4e2c0 1 -- 172.21.15.16:0/781585009 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x55a1ff6b0400 msgr2=0x55a1ff7a2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:19.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.049+0000 7fa532a4e2c0 1 --2- 172.21.15.16:0/781585009 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x55a1ff6b0400 0x55a1ff7a2000 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x55a2001ebb60 tx=0x55a200111440 comp rx=0 tx=0).stop 2023-05-20T01:27:19.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.049+0000 7fa532a4e2c0 1 -- 172.21.15.16:0/781585009 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a1ff6ae400 msgr2=0x55a1ff414100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:19.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.049+0000 7fa532a4e2c0 1 --2- 172.21.15.16:0/781585009 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a1ff6ae400 0x55a1ff414100 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x55a1ff68a6c0 tx=0x55a2001fad20 comp rx=0 tx=0).stop 2023-05-20T01:27:19.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.049+0000 7fa5301f9700 1 -- 172.21.15.16:0/781585009 reap_dead start 2023-05-20T01:27:19.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.049+0000 7fa532a4e2c0 1 -- 172.21.15.16:0/781585009 shutdown_connections 2023-05-20T01:27:19.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.049+0000 7fa532a4e2c0 1 -- 172.21.15.16:0/781585009 >> 172.21.15.16:0/781585009 conn(0x55a1ff379c00 msgr2=0x55a2000f5800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:27:19.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.050+0000 7fa532a4e2c0 1 -- 172.21.15.16:0/781585009 shutdown_connections 2023-05-20T01:27:19.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.050+0000 7fa532a4e2c0 1 -- 172.21.15.16:0/781585009 wait complete. 2023-05-20T01:27:19.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.050+0000 7fa532a4e2c0 1 -- 172.21.15.16:0/1748762220 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a1ff379000 msgr2=0x55a1ff413b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:19.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.050+0000 7fa532a4e2c0 1 --2- 172.21.15.16:0/1748762220 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a1ff379000 0x55a1ff413b80 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x55a2001ea630 tx=0x55a200111320 comp rx=0 tx=0).stop 2023-05-20T01:27:19.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.050+0000 7fa532a4e2c0 1 -- 172.21.15.16:0/1748762220 shutdown_connections 2023-05-20T01:27:19.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.050+0000 7fa532a4e2c0 1 --2- 172.21.15.16:0/1748762220 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a1ff378800 0x55a1ff413080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.050+0000 7fa532a4e2c0 1 --2- 172.21.15.16:0/1748762220 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a1ff379000 0x55a1ff413b80 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.050+0000 7fa532a4e2c0 1 --2- 172.21.15.16:0/1748762220 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a1ff378c00 0x55a1ff413600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.050+0000 7fa532a4e2c0 1 -- 172.21.15.16:0/1748762220 >> 172.21.15.16:0/1748762220 conn(0x55a1ff379400 msgr2=0x55a2000f5000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:27:19.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.050+0000 7fa532a4e2c0 1 -- 172.21.15.16:0/1748762220 shutdown_connections 2023-05-20T01:27:19.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.050+0000 7fa532a4e2c0 1 -- 172.21.15.16:0/1748762220 wait complete. 2023-05-20T01:27:19.076 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-table-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'cephfs:0', 'reset', 'session'] in time 0:00:00.073953, result: { "0": { "data": {}, "result": 0 } } 2023-05-20T01:27:19.077 DEBUG:teuthology.orchestra.run.smithi016:> cephfs-table-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 cephfs:0 reset snap 2023-05-20T01:27:19.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.152+0000 7fc2700ee2c0 1 -- start start 2023-05-20T01:27:19.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.152+0000 7fc2700ee2c0 1 Processor -- start 2023-05-20T01:27:19.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.153+0000 7fc2700ee2c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55b022c2b400 0x55b022cc5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:19.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.153+0000 7fc2700ee2c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b022c2ac00 0x55b022cc5080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:19.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.153+0000 7fc2700ee2c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b022c2a800 0x55b022cc4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:19.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.153+0000 7fc2700ee2c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b022c67cc0 con 0x55b022c2ac00 2023-05-20T01:27:19.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.153+0000 7fc2700ee2c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b022c66000 con 0x55b022c2a800 2023-05-20T01:27:19.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.153+0000 7fc2700ee2c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b022c66c80 con 0x55b022c2b400 2023-05-20T01:27:19.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.153+0000 7fc26d899700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b022c2ac00 0x55b022cc5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:19.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.153+0000 7fc26e09a700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55b022c2b400 0x55b022cc5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:19.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.153+0000 7fc26d899700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b022c2ac00 0x55b022cc5080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38800/0 (socket says 172.21.15.16:38800) 2023-05-20T01:27:19.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.153+0000 7fc26d899700 1 -- 172.21.15.16:0/2407379919 learned_addr learned my addr 172.21.15.16:0/2407379919 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:27:19.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.153+0000 7fc26e09a700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55b022c2b400 0x55b022cc5b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.16:38836/0 (socket says 172.21.15.16:38836) 2023-05-20T01:27:19.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.153+0000 7fc26e89b700 1 --2- 172.21.15.16:0/2407379919 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b022c2a800 0x55b022cc4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:19.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.153+0000 7fc26d899700 1 -- 172.21.15.16:0/2407379919 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55b022c2b400 msgr2=0x55b022cc5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:19.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.153+0000 7fc26d899700 1 --2- 172.21.15.16:0/2407379919 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55b022c2b400 0x55b022cc5b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.153+0000 7fc26d899700 1 -- 172.21.15.16:0/2407379919 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b022c2a800 msgr2=0x55b022cc4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:19.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.153+0000 7fc26d899700 1 --2- 172.21.15.16:0/2407379919 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b022c2a800 0x55b022cc4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.153+0000 7fc26d899700 1 -- 172.21.15.16:0/2407379919 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b022c4f380 con 0x55b022c2ac00 2023-05-20T01:27:19.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.153+0000 7fc26d899700 1 --2- 172.21.15.16:0/2407379919 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b022c2ac00 0x55b022cc5080 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x55b0239f4720 tx=0x55b023a010e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6224d89f3981a7e0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:19.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.154+0000 7fc26b895700 1 -- 172.21.15.16:0/2407379919 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55b022c5f080 con 0x55b022c2ac00 2023-05-20T01:27:19.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.154+0000 7fc26b895700 1 -- 172.21.15.16:0/2407379919 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b022c7d800 con 0x55b022c2ac00 2023-05-20T01:27:19.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.154+0000 7fc26b895700 1 -- 172.21.15.16:0/2407379919 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55b022c5f600 con 0x55b022c2ac00 2023-05-20T01:27:19.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.154+0000 7fc2700ee2c0 1 -- 172.21.15.16:0/2407379919 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b022c4fd40 con 0x55b022c2ac00 2023-05-20T01:27:19.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.154+0000 7fc2700ee2c0 1 -- 172.21.15.16:0/2407379919 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55b0239fe000 con 0x55b022c2ac00 2023-05-20T01:27:19.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.154+0000 7fc2700ee2c0 4 waiting for MDS map... 2023-05-20T01:27:19.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.154+0000 7fc26b895700 1 -- 172.21.15.16:0/2407379919 <== mon.0 v2:172.21.15.16:3300/0 4 ==== osd_map(119..119 src has 1..119) v4 ==== 6987+0+0 (secure 0 0 0) 0x55b022c6cc40 con 0x55b022c2ac00 2023-05-20T01:27:19.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.154+0000 7fc26b895700 1 -- 172.21.15.16:0/2407379919 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 84) v1 ==== 3126+0+0 (secure 0 0 0) 0x55b023ab2900 con 0x55b022c2ac00 2023-05-20T01:27:19.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.154+0000 7fc2700ee2c0 4 Got MDS map 84 2023-05-20T01:27:19.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.154+0000 7fc2700ee2c0 10 main: main 2023-05-20T01:27:19.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.154+0000 7fc2700ee2c0 4 main: connecting to RADOS... 2023-05-20T01:27:19.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.154+0000 7fc2700ee2c0 1 Processor -- start 2023-05-20T01:27:19.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.155+0000 7fc2700ee2c0 1 -- start start 2023-05-20T01:27:19.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.155+0000 7fc2700ee2c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b022c2b800 0x55b022cc6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:19.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.155+0000 7fc2700ee2c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55b022f46000 0x55b022cc6680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:19.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.155+0000 7fc2700ee2c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b022f46400 0x55b022cc6c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:19.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.155+0000 7fc2700ee2c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b022c66b40 con 0x55b022c2b800 2023-05-20T01:27:19.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.155+0000 7fc2700ee2c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b022c67e00 con 0x55b022f46400 2023-05-20T01:27:19.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.155+0000 7fc2700ee2c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b022cb8f00 con 0x55b022f46000 2023-05-20T01:27:19.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.155+0000 7fc26e89b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b022c2b800 0x55b022cc6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:19.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.155+0000 7fc26e89b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b022c2b800 0x55b022cc6100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38804/0 (socket says 172.21.15.16:38804) 2023-05-20T01:27:19.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.155+0000 7fc26e89b700 1 -- 172.21.15.16:0/2981610077 learned_addr learned my addr 172.21.15.16:0/2981610077 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:27:19.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.156+0000 7fc26e09a700 1 --2- 172.21.15.16:0/2981610077 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55b022f46000 0x55b022cc6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:19.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.156+0000 7fc26d899700 1 --2- 172.21.15.16:0/2981610077 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b022f46400 0x55b022cc6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:19.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.156+0000 7fc26e89b700 1 -- 172.21.15.16:0/2981610077 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55b022f46000 msgr2=0x55b022cc6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:19.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.156+0000 7fc26e89b700 1 --2- 172.21.15.16:0/2981610077 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55b022f46000 0x55b022cc6680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.156+0000 7fc26e89b700 1 -- 172.21.15.16:0/2981610077 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b022f46400 msgr2=0x55b022cc6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:19.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.156+0000 7fc26e89b700 1 --2- 172.21.15.16:0/2981610077 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b022f46400 0x55b022cc6c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.156+0000 7fc26e89b700 1 -- 172.21.15.16:0/2981610077 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b0239fe1a0 con 0x55b022c2b800 2023-05-20T01:27:19.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.156+0000 7fc26e89b700 1 --2- 172.21.15.16:0/2981610077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b022c2b800 0x55b022cc6100 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x55b0239f5c80 tx=0x55b023a00180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20206aa45ff163a8 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:19.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.156+0000 7fc269891700 1 -- 172.21.15.16:0/2981610077 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55b022c5ef20 con 0x55b022c2b800 2023-05-20T01:27:19.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.156+0000 7fc269891700 1 -- 172.21.15.16:0/2981610077 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b022c7c480 con 0x55b022c2b800 2023-05-20T01:27:19.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.156+0000 7fc269891700 1 -- 172.21.15.16:0/2981610077 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55b022c5f080 con 0x55b022c2b800 2023-05-20T01:27:19.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.156+0000 7fc2700ee2c0 1 -- 172.21.15.16:0/2981610077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b022c2b800 msgr2=0x55b022cc6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:19.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.156+0000 7fc2700ee2c0 1 --2- 172.21.15.16:0/2981610077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b022c2b800 0x55b022cc6100 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x55b0239f5c80 tx=0x55b023a00180 comp rx=0 tx=0).stop 2023-05-20T01:27:19.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.156+0000 7fc2700ee2c0 1 -- 172.21.15.16:0/2981610077 shutdown_connections 2023-05-20T01:27:19.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.156+0000 7fc2700ee2c0 1 --2- 172.21.15.16:0/2981610077 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b022f46400 0x55b022cc6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.156+0000 7fc2700ee2c0 1 --2- 172.21.15.16:0/2981610077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b022c2b800 0x55b022cc6100 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.156+0000 7fc2700ee2c0 1 --2- 172.21.15.16:0/2981610077 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55b022f46000 0x55b022cc6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.156+0000 7fc2700ee2c0 1 -- 172.21.15.16:0/2981610077 >> 172.21.15.16:0/2981610077 conn(0x55b022c2bc00 msgr2=0x55b022d08000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:27:19.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.157+0000 7fc2700ee2c0 1 -- 172.21.15.16:0/2981610077 shutdown_connections 2023-05-20T01:27:19.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.157+0000 7fc2700ee2c0 1 -- 172.21.15.16:0/2981610077 wait complete. 2023-05-20T01:27:19.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.157+0000 7fc2700ee2c0 1 Processor -- start 2023-05-20T01:27:19.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.157+0000 7fc2700ee2c0 1 -- start start 2023-05-20T01:27:19.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.158+0000 7fc2700ee2c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b022c2bc00 0x55b022cc6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:19.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.158+0000 7fc2700ee2c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55b022f64000 0x55b022cc6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:19.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.158+0000 7fc2700ee2c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b022f64400 0x55b022cc6c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:19.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.158+0000 7fc2700ee2c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b022c66000 con 0x55b022f64400 2023-05-20T01:27:19.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.158+0000 7fc2700ee2c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b022c67180 con 0x55b022c2bc00 2023-05-20T01:27:19.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.158+0000 7fc2700ee2c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b022c67040 con 0x55b022f64000 2023-05-20T01:27:19.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.158+0000 7fc26d899700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b022f64400 0x55b022cc6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:19.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.158+0000 7fc26d899700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b022f64400 0x55b022cc6c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38814/0 (socket says 172.21.15.16:38814) 2023-05-20T01:27:19.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.158+0000 7fc26d899700 1 -- 172.21.15.16:0/2220937230 learned_addr learned my addr 172.21.15.16:0/2220937230 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:27:19.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.158+0000 7fc26e89b700 1 --2- 172.21.15.16:0/2220937230 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b022c2bc00 0x55b022cc6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:19.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.158+0000 7fc26e09a700 1 --2- 172.21.15.16:0/2220937230 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55b022f64000 0x55b022cc6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:19.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.158+0000 7fc26d899700 1 -- 172.21.15.16:0/2220937230 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55b022f64000 msgr2=0x55b022cc6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:19.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.158+0000 7fc26d899700 1 --2- 172.21.15.16:0/2220937230 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55b022f64000 0x55b022cc6680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.158+0000 7fc26d899700 1 -- 172.21.15.16:0/2220937230 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b022c2bc00 msgr2=0x55b022cc6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:19.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.158+0000 7fc26d899700 1 --2- 172.21.15.16:0/2220937230 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b022c2bc00 0x55b022cc6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.158+0000 7fc26d899700 1 -- 172.21.15.16:0/2220937230 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b0239fe4e0 con 0x55b022f64400 2023-05-20T01:27:19.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.158+0000 7fc26d899700 1 --2- 172.21.15.16:0/2220937230 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b022f64400 0x55b022cc6c00 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x55b022f62c00 tx=0x55b023a00060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:19.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.158+0000 7fc26808e700 1 -- 172.21.15.16:0/2220937230 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55b022c5f1e0 con 0x55b022f64400 2023-05-20T01:27:19.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.158+0000 7fc26808e700 1 -- 172.21.15.16:0/2220937230 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b022c7de00 con 0x55b022f64400 2023-05-20T01:27:19.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.158+0000 7fc2700ee2c0 1 -- 172.21.15.16:0/2220937230 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55b0239fe340 con 0x55b022f64400 2023-05-20T01:27:19.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.158+0000 7fc26808e700 1 -- 172.21.15.16:0/2220937230 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55b022c5ef20 con 0x55b022f64400 2023-05-20T01:27:19.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.159+0000 7fc2700ee2c0 1 -- 172.21.15.16:0/2220937230 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b0239fe680 con 0x55b022f64400 2023-05-20T01:27:19.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.159+0000 7fc2700ee2c0 4 main: resolving pool 15 2023-05-20T01:27:19.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.160+0000 7fc26808e700 1 -- 172.21.15.16:0/2220937230 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x55b022c8cb00 con 0x55b022f64400 2023-05-20T01:27:19.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.160+0000 7fc26808e700 1 --2- 172.21.15.16:0/2220937230 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x55b022f66800 0x55b022cc7700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:19.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.160+0000 7fc26808e700 1 -- 172.21.15.16:0/2220937230 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6987+0+0 (secure 0 0 0) 0x55b022c6ca80 con 0x55b022f64400 2023-05-20T01:27:19.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.160+0000 7fc2700ee2c0 4 main: creating IoCtx.. 2023-05-20T01:27:19.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.160+0000 7fc26e89b700 1 --2- 172.21.15.16:0/2220937230 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x55b022f66800 0x55b022cc7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:19.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.160+0000 7fc2700ee2c0 1 --2- 172.21.15.16:0/2220937230 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x55b022f64800 0x55b022cc7180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:19.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.160+0000 7fc2700ee2c0 1 -- 172.21.15.16:0/2220937230 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:1 15.d 15:b50e409b:::mds_snaptable:head [writefull 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x55b023abc700 con 0x55b022f64800 2023-05-20T01:27:19.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.160+0000 7fc26e09a700 1 --2- 172.21.15.16:0/2220937230 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x55b022f64800 0x55b022cc7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:19.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.160+0000 7fc26e89b700 1 --2- 172.21.15.16:0/2220937230 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x55b022f66800 0x55b022cc7700 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x55b0239f5c80 tx=0x55b023a00420 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:19.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.161+0000 7fc26e09a700 1 --2- 172.21.15.16:0/2220937230 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x55b022f64800 0x55b022cc7180 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:19.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.163+0000 7fc26e09a700 1 -- 172.21.15.16:0/2220937230 <== osd.7 v2:172.21.15.132:6800/466001149 1 ==== osd_op_reply(1 mds_snaptable [writefull 0~70] v119'3 uv3 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x55b022d90fc0 con 0x55b022f64800 2023-05-20T01:27:19.186 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-05-20T01:27:19.186 INFO:teuthology.orchestra.run.smithi016.stdout: "result": 0 2023-05-20T01:27:19.186 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-05-20T01:27:19.186 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:27:19.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.163+0000 7fc2700ee2c0 1 -- 172.21.15.16:0/2220937230 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x55b022f64800 msgr2=0x55b022cc7180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:19.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.163+0000 7fc2700ee2c0 1 --2- 172.21.15.16:0/2220937230 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x55b022f64800 0x55b022cc7180 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.163+0000 7fc2700ee2c0 1 -- 172.21.15.16:0/2220937230 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x55b022f66800 msgr2=0x55b022cc7700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:19.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.163+0000 7fc2700ee2c0 1 --2- 172.21.15.16:0/2220937230 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x55b022f66800 0x55b022cc7700 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x55b0239f5c80 tx=0x55b023a00420 comp rx=0 tx=0).stop 2023-05-20T01:27:19.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.163+0000 7fc2700ee2c0 1 -- 172.21.15.16:0/2220937230 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b022f64400 msgr2=0x55b022cc6c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:19.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.163+0000 7fc2700ee2c0 1 --2- 172.21.15.16:0/2220937230 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b022f64400 0x55b022cc6c00 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x55b022f62c00 tx=0x55b023a00060 comp rx=0 tx=0).stop 2023-05-20T01:27:19.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.163+0000 7fc26e09a700 1 -- 172.21.15.16:0/2220937230 reap_dead start 2023-05-20T01:27:19.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.163+0000 7fc2700ee2c0 1 -- 172.21.15.16:0/2220937230 shutdown_connections 2023-05-20T01:27:19.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.163+0000 7fc2700ee2c0 1 -- 172.21.15.16:0/2220937230 >> 172.21.15.16:0/2220937230 conn(0x55b022c2b800 msgr2=0x55b022d08000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:27:19.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.163+0000 7fc2700ee2c0 1 -- 172.21.15.16:0/2220937230 shutdown_connections 2023-05-20T01:27:19.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.163+0000 7fc2700ee2c0 1 -- 172.21.15.16:0/2220937230 wait complete. 2023-05-20T01:27:19.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.163+0000 7fc2700ee2c0 1 -- 172.21.15.16:0/2407379919 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b022c2ac00 msgr2=0x55b022cc5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:19.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.163+0000 7fc2700ee2c0 1 --2- 172.21.15.16:0/2407379919 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b022c2ac00 0x55b022cc5080 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x55b0239f4720 tx=0x55b023a010e0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.163+0000 7fc2700ee2c0 1 -- 172.21.15.16:0/2407379919 shutdown_connections 2023-05-20T01:27:19.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.163+0000 7fc2700ee2c0 1 --2- 172.21.15.16:0/2407379919 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b022c2a800 0x55b022cc4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.163+0000 7fc2700ee2c0 1 --2- 172.21.15.16:0/2407379919 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55b022c2b400 0x55b022cc5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.163+0000 7fc2700ee2c0 1 --2- 172.21.15.16:0/2407379919 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b022c2ac00 0x55b022cc5080 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.163+0000 7fc2700ee2c0 1 -- 172.21.15.16:0/2407379919 >> 172.21.15.16:0/2407379919 conn(0x55b022c2a400 msgr2=0x55b022d07800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:27:19.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.163+0000 7fc2700ee2c0 1 -- 172.21.15.16:0/2407379919 shutdown_connections 2023-05-20T01:27:19.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.163+0000 7fc2700ee2c0 1 -- 172.21.15.16:0/2407379919 wait complete. 2023-05-20T01:27:19.191 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-table-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'cephfs:0', 'reset', 'snap'] in time 0:00:00.114805, result: { "result": 0 } 2023-05-20T01:27:19.192 DEBUG:teuthology.orchestra.run.smithi016:> cephfs-table-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 cephfs:0 reset inode 2023-05-20T01:27:19.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.266+0000 7f45aa17d2c0 1 -- start start 2023-05-20T01:27:19.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.266+0000 7f45aa17d2c0 1 Processor -- start 2023-05-20T01:27:19.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.267+0000 7f45aa17d2c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x564d2698f800 0x564d26a29080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:19.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.267+0000 7f45aa17d2c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d2698f400 0x564d26a28b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:19.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.267+0000 7f45aa17d2c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564d2698ec00 0x564d26a29600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:19.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.267+0000 7f45aa17d2c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d269cbcc0 con 0x564d2698f400 2023-05-20T01:27:19.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.267+0000 7f45aa17d2c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d269cb400 con 0x564d2698ec00 2023-05-20T01:27:19.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.267+0000 7f45aa17d2c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x564d269ca000 con 0x564d2698f800 2023-05-20T01:27:19.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.267+0000 7f45a7928700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d2698f400 0x564d26a28b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:19.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.267+0000 7f45a7928700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d2698f400 0x564d26a28b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38834/0 (socket says 172.21.15.16:38834) 2023-05-20T01:27:19.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.267+0000 7f45a7928700 1 -- 172.21.15.16:0/2803160304 learned_addr learned my addr 172.21.15.16:0/2803160304 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:27:19.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.267+0000 7f45a8129700 1 --2- 172.21.15.16:0/2803160304 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x564d2698f800 0x564d26a29080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:19.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.267+0000 7f45a892a700 1 --2- 172.21.15.16:0/2803160304 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564d2698ec00 0x564d26a29600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:19.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.267+0000 7f45a7928700 1 -- 172.21.15.16:0/2803160304 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x564d2698f800 msgr2=0x564d26a29080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:19.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.267+0000 7f45a7928700 1 --2- 172.21.15.16:0/2803160304 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x564d2698f800 0x564d26a29080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.267+0000 7f45a7928700 1 -- 172.21.15.16:0/2803160304 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564d2698ec00 msgr2=0x564d26a29600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:19.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.267+0000 7f45a7928700 1 --2- 172.21.15.16:0/2803160304 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564d2698ec00 0x564d26a29600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.267+0000 7f45a7928700 1 -- 172.21.15.16:0/2803160304 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564d269b3520 con 0x564d2698f400 2023-05-20T01:27:19.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.267+0000 7f45a892a700 1 --2- 172.21.15.16:0/2803160304 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564d2698ec00 0x564d26a29600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:27:19.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.267+0000 7f45a7928700 1 --2- 172.21.15.16:0/2803160304 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d2698f400 0x564d26a28b00 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x564d26aee6c0 tx=0x564d2776cea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b65e4886766092a2 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:19.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.267+0000 7f45a5924700 1 -- 172.21.15.16:0/2803160304 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x564d269c3080 con 0x564d2698f400 2023-05-20T01:27:19.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.267+0000 7f45a5924700 1 -- 172.21.15.16:0/2803160304 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564d269e0600 con 0x564d2698f400 2023-05-20T01:27:19.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.267+0000 7f45a5924700 1 -- 172.21.15.16:0/2803160304 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x564d269c3600 con 0x564d2698f400 2023-05-20T01:27:19.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.267+0000 7f45aa17d2c0 1 -- 172.21.15.16:0/2803160304 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564d269b3d40 con 0x564d2698f400 2023-05-20T01:27:19.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.267+0000 7f45aa17d2c0 1 -- 172.21.15.16:0/2803160304 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x564d269b3520 con 0x564d2698f400 2023-05-20T01:27:19.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.268+0000 7f45aa17d2c0 4 waiting for MDS map... 2023-05-20T01:27:19.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.268+0000 7f45a5924700 1 -- 172.21.15.16:0/2803160304 <== mon.0 v2:172.21.15.16:3300/0 4 ==== osd_map(119..119 src has 1..119) v4 ==== 6987+0+0 (secure 0 0 0) 0x564d269d0c40 con 0x564d2698f400 2023-05-20T01:27:19.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.268+0000 7f45a5924700 1 -- 172.21.15.16:0/2803160304 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 84) v1 ==== 3126+0+0 (secure 0 0 0) 0x564d2780f200 con 0x564d2698f400 2023-05-20T01:27:19.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.268+0000 7f45aa17d2c0 4 Got MDS map 84 2023-05-20T01:27:19.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.268+0000 7f45aa17d2c0 10 main: main 2023-05-20T01:27:19.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.268+0000 7f45aa17d2c0 4 main: connecting to RADOS... 2023-05-20T01:27:19.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.268+0000 7f45aa17d2c0 1 Processor -- start 2023-05-20T01:27:19.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.268+0000 7f45aa17d2c0 1 -- start start 2023-05-20T01:27:19.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.269+0000 7f45aa17d2c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564d2698fc00 0x564d26a28580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:19.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.269+0000 7f45aa17d2c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x564d2698f000 0x564d26a2a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:19.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.269+0000 7f45aa17d2c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d27760400 0x564d26a2a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:19.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.269+0000 7f45aa17d2c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d269cac80 con 0x564d27760400 2023-05-20T01:27:19.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.269+0000 7f45aa17d2c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d269cab40 con 0x564d2698fc00 2023-05-20T01:27:19.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.269+0000 7f45aa17d2c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x564d269cb400 con 0x564d2698f000 2023-05-20T01:27:19.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.269+0000 7f45a7928700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d27760400 0x564d26a2a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:19.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.269+0000 7f45a7928700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d27760400 0x564d26a2a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38842/0 (socket says 172.21.15.16:38842) 2023-05-20T01:27:19.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.269+0000 7f45a7928700 1 -- 172.21.15.16:0/337233635 learned_addr learned my addr 172.21.15.16:0/337233635 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:27:19.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.269+0000 7f45a8129700 1 --2- 172.21.15.16:0/337233635 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x564d2698f000 0x564d26a2a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:19.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.269+0000 7f45a892a700 1 --2- 172.21.15.16:0/337233635 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564d2698fc00 0x564d26a28580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:19.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.269+0000 7f45a7928700 1 -- 172.21.15.16:0/337233635 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x564d2698f000 msgr2=0x564d26a2a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:19.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.269+0000 7f45a7928700 1 --2- 172.21.15.16:0/337233635 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x564d2698f000 0x564d26a2a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.269+0000 7f45a7928700 1 -- 172.21.15.16:0/337233635 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564d2698fc00 msgr2=0x564d26a28580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:19.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.269+0000 7f45a7928700 1 --2- 172.21.15.16:0/337233635 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564d2698fc00 0x564d26a28580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.269+0000 7f45a7928700 1 -- 172.21.15.16:0/337233635 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564d269b3d40 con 0x564d27760400 2023-05-20T01:27:19.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.269+0000 7f45a892a700 1 --2- 172.21.15.16:0/337233635 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564d2698fc00 0x564d26a28580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:27:19.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.270+0000 7f45a7928700 1 --2- 172.21.15.16:0/337233635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d27760400 0x564d26a2a680 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x564d26aefb90 tx=0x564d277272c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=449c013129b5b823 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:19.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.270+0000 7f45a3920700 1 -- 172.21.15.16:0/337233635 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x564d269c31e0 con 0x564d27760400 2023-05-20T01:27:19.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.270+0000 7f45a3920700 1 -- 172.21.15.16:0/337233635 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564d269e1e00 con 0x564d27760400 2023-05-20T01:27:19.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.270+0000 7f45a3920700 1 -- 172.21.15.16:0/337233635 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x564d269c3080 con 0x564d27760400 2023-05-20T01:27:19.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.270+0000 7f45aa17d2c0 1 -- 172.21.15.16:0/337233635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d27760400 msgr2=0x564d26a2a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:19.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.270+0000 7f45aa17d2c0 1 --2- 172.21.15.16:0/337233635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d27760400 0x564d26a2a680 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x564d26aefb90 tx=0x564d277272c0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.270+0000 7f45aa17d2c0 1 -- 172.21.15.16:0/337233635 shutdown_connections 2023-05-20T01:27:19.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.270+0000 7f45aa17d2c0 1 --2- 172.21.15.16:0/337233635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d27760400 0x564d26a2a680 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.270+0000 7f45aa17d2c0 1 --2- 172.21.15.16:0/337233635 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564d2698fc00 0x564d26a28580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.270+0000 7f45aa17d2c0 1 --2- 172.21.15.16:0/337233635 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x564d2698f000 0x564d26a2a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.270+0000 7f45aa17d2c0 1 -- 172.21.15.16:0/337233635 >> 172.21.15.16:0/337233635 conn(0x564d2698e800 msgr2=0x564d26a6b000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:27:19.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.270+0000 7f45aa17d2c0 1 -- 172.21.15.16:0/337233635 shutdown_connections 2023-05-20T01:27:19.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.270+0000 7f45aa17d2c0 1 -- 172.21.15.16:0/337233635 wait complete. 2023-05-20T01:27:19.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.271+0000 7f45aa17d2c0 1 Processor -- start 2023-05-20T01:27:19.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.271+0000 7f45aa17d2c0 1 -- start start 2023-05-20T01:27:19.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.271+0000 7f45aa17d2c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564d27761400 0x564d26a2b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:19.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.271+0000 7f45aa17d2c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d27761000 0x564d26a2b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:19.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.271+0000 7f45aa17d2c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x564d27760c00 0x564d26a2ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:19.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.271+0000 7f45aa17d2c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d269caa00 con 0x564d27761000 2023-05-20T01:27:19.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.271+0000 7f45aa17d2c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d269cb180 con 0x564d27761400 2023-05-20T01:27:19.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.271+0000 7f45aa17d2c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x564d269cb040 con 0x564d27760c00 2023-05-20T01:27:19.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.271+0000 7f45a8129700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d27761000 0x564d26a2b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:19.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.271+0000 7f45a8129700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d27761000 0x564d26a2b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38846/0 (socket says 172.21.15.16:38846) 2023-05-20T01:27:19.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.271+0000 7f45a8129700 1 -- 172.21.15.16:0/1618647443 learned_addr learned my addr 172.21.15.16:0/1618647443 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:27:19.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.271+0000 7f45a892a700 1 --2- 172.21.15.16:0/1618647443 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564d27761400 0x564d26a2b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:19.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.271+0000 7f45a7928700 1 --2- 172.21.15.16:0/1618647443 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x564d27760c00 0x564d26a2ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:19.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.272+0000 7f45a8129700 1 -- 172.21.15.16:0/1618647443 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x564d27760c00 msgr2=0x564d26a2ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:19.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.272+0000 7f45a8129700 1 --2- 172.21.15.16:0/1618647443 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x564d27760c00 0x564d26a2ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.272+0000 7f45a8129700 1 -- 172.21.15.16:0/1618647443 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564d27761400 msgr2=0x564d26a2b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:19.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.272+0000 7f45a8129700 1 --2- 172.21.15.16:0/1618647443 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564d27761400 0x564d26a2b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.272+0000 7f45a8129700 1 -- 172.21.15.16:0/1618647443 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564d26af04e0 con 0x564d27761000 2023-05-20T01:27:19.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.272+0000 7f45a8129700 1 --2- 172.21.15.16:0/1618647443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d27761000 0x564d26a2b180 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x564d27708780 tx=0x564d2776c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:19.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.272+0000 7f45a211d700 1 -- 172.21.15.16:0/1618647443 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x564d269c31e0 con 0x564d27761000 2023-05-20T01:27:19.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.272+0000 7f45a211d700 1 -- 172.21.15.16:0/1618647443 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564d269e1800 con 0x564d27761000 2023-05-20T01:27:19.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.272+0000 7f45a211d700 1 -- 172.21.15.16:0/1618647443 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x564d269c3080 con 0x564d27761000 2023-05-20T01:27:19.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.272+0000 7f45aa17d2c0 1 -- 172.21.15.16:0/1618647443 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x564d269b24e0 con 0x564d27761000 2023-05-20T01:27:19.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.272+0000 7f45aa17d2c0 1 -- 172.21.15.16:0/1618647443 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564d26af0680 con 0x564d27761000 2023-05-20T01:27:19.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.272+0000 7f45aa17d2c0 4 main: resolving pool 15 2023-05-20T01:27:19.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.273+0000 7f45a211d700 1 -- 172.21.15.16:0/1618647443 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x564d269f0840 con 0x564d27761000 2023-05-20T01:27:19.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.274+0000 7f45a211d700 1 --2- 172.21.15.16:0/1618647443 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x564d27761c00 0x564d26db4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:19.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.274+0000 7f45a211d700 1 -- 172.21.15.16:0/1618647443 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6987+0+0 (secure 0 0 0) 0x564d269d0a80 con 0x564d27761000 2023-05-20T01:27:19.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.274+0000 7f45a892a700 1 --2- 172.21.15.16:0/1618647443 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x564d27761c00 0x564d26db4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:19.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.274+0000 7f45aa17d2c0 4 main: creating IoCtx.. 2023-05-20T01:27:19.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.274+0000 7f45aa17d2c0 1 --2- 172.21.15.16:0/1618647443 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x564d27760800 0x564d26a2a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:19.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.274+0000 7f45aa17d2c0 1 -- 172.21.15.16:0/1618647443 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:1 15.13 15:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x564d2781a700 con 0x564d27760800 2023-05-20T01:27:19.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.274+0000 7f45a7928700 1 --2- 172.21.15.16:0/1618647443 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x564d27760800 0x564d26a2a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:19.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.274+0000 7f45a892a700 1 --2- 172.21.15.16:0/1618647443 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x564d27761c00 0x564d26db4000 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x564d27709ce0 tx=0x564d26dbc000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:19.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.274+0000 7f45a7928700 1 --2- 172.21.15.16:0/1618647443 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x564d27760800 0x564d26a2a680 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:19.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.276+0000 7f45a7928700 1 -- 172.21.15.16:0/1618647443 <== osd.1 v2:172.21.15.16:6808/3860264220 1 ==== osd_op_reply(1 mds0_inotable [writefull 0~34] v119'6 uv6 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x564d26aeafc0 con 0x564d27760800 2023-05-20T01:27:19.289 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-05-20T01:27:19.289 INFO:teuthology.orchestra.run.smithi016.stdout: "0": { 2023-05-20T01:27:19.289 INFO:teuthology.orchestra.run.smithi016.stdout: "data": {}, 2023-05-20T01:27:19.289 INFO:teuthology.orchestra.run.smithi016.stdout: "result": 0 2023-05-20T01:27:19.289 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:27:19.290 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-05-20T01:27:19.290 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:27:19.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.276+0000 7f45aa17d2c0 1 -- 172.21.15.16:0/1618647443 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x564d27760800 msgr2=0x564d26a2a680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:19.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.276+0000 7f45aa17d2c0 1 --2- 172.21.15.16:0/1618647443 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x564d27760800 0x564d26a2a680 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.276+0000 7f45aa17d2c0 1 -- 172.21.15.16:0/1618647443 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x564d27761c00 msgr2=0x564d26db4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:19.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.276+0000 7f45aa17d2c0 1 --2- 172.21.15.16:0/1618647443 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x564d27761c00 0x564d26db4000 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x564d27709ce0 tx=0x564d26dbc000 comp rx=0 tx=0).stop 2023-05-20T01:27:19.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.276+0000 7f45aa17d2c0 1 -- 172.21.15.16:0/1618647443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d27761000 msgr2=0x564d26a2b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:19.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.276+0000 7f45aa17d2c0 1 --2- 172.21.15.16:0/1618647443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d27761000 0x564d26a2b180 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x564d27708780 tx=0x564d2776c000 comp rx=0 tx=0).stop 2023-05-20T01:27:19.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.276+0000 7f45a8129700 1 -- 172.21.15.16:0/1618647443 reap_dead start 2023-05-20T01:27:19.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.276+0000 7f45aa17d2c0 1 -- 172.21.15.16:0/1618647443 shutdown_connections 2023-05-20T01:27:19.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.276+0000 7f45aa17d2c0 1 -- 172.21.15.16:0/1618647443 >> 172.21.15.16:0/1618647443 conn(0x564d2698e800 msgr2=0x564d26a6b000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:27:19.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.276+0000 7f45aa17d2c0 1 -- 172.21.15.16:0/1618647443 shutdown_connections 2023-05-20T01:27:19.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.276+0000 7f45aa17d2c0 1 -- 172.21.15.16:0/1618647443 wait complete. 2023-05-20T01:27:19.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.276+0000 7f45aa17d2c0 1 -- 172.21.15.16:0/2803160304 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d2698f400 msgr2=0x564d26a28b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:19.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.276+0000 7f45aa17d2c0 1 --2- 172.21.15.16:0/2803160304 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d2698f400 0x564d26a28b00 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x564d26aee6c0 tx=0x564d2776cea0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.276+0000 7f45aa17d2c0 1 -- 172.21.15.16:0/2803160304 shutdown_connections 2023-05-20T01:27:19.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.276+0000 7f45aa17d2c0 1 --2- 172.21.15.16:0/2803160304 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564d2698ec00 0x564d26a29600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.276+0000 7f45aa17d2c0 1 --2- 172.21.15.16:0/2803160304 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x564d2698f800 0x564d26a29080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.276+0000 7f45aa17d2c0 1 --2- 172.21.15.16:0/2803160304 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d2698f400 0x564d26a28b00 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.276+0000 7f45aa17d2c0 1 -- 172.21.15.16:0/2803160304 >> 172.21.15.16:0/2803160304 conn(0x564d2698e400 msgr2=0x564d26a6c000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:27:19.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.277+0000 7f45aa17d2c0 1 -- 172.21.15.16:0/2803160304 shutdown_connections 2023-05-20T01:27:19.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.277+0000 7f45aa17d2c0 1 -- 172.21.15.16:0/2803160304 wait complete. 2023-05-20T01:27:19.294 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-table-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'cephfs:0', 'reset', 'inode'] in time 0:00:00.102379, result: { "0": { "data": {}, "result": 0 } } 2023-05-20T01:27:19.294 DEBUG:teuthology.orchestra.run.smithi016:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 journal reset --force 2023-05-20T01:27:19.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.371+0000 7f3e144692c0 1 -- start start 2023-05-20T01:27:19.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.371+0000 7f3e144692c0 1 Processor -- start 2023-05-20T01:27:19.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.371+0000 7f3e144692c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5653c039b000 0x5653c0435b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:19.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.371+0000 7f3e144692c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5653c039ac00 0x5653c0435600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:19.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.371+0000 7f3e144692c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653c039a800 0x5653c0435080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:19.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.371+0000 7f3e144692c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653c03d7cc0 con 0x5653c039a800 2023-05-20T01:27:19.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.371+0000 7f3e144692c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653c03d7400 con 0x5653c039ac00 2023-05-20T01:27:19.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.371+0000 7f3e144692c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x5653c03d6a00 con 0x5653c039b000 2023-05-20T01:27:19.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.371+0000 7f3e12c16700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653c039a800 0x5653c0435080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:19.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.371+0000 7f3e12c16700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653c039a800 0x5653c0435080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38864/0 (socket says 172.21.15.16:38864) 2023-05-20T01:27:19.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.371+0000 7f3e12c16700 1 -- 172.21.15.16:0/315426902 learned_addr learned my addr 172.21.15.16:0/315426902 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:27:19.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.371+0000 7f3e12415700 1 --2- 172.21.15.16:0/315426902 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5653c039b000 0x5653c0435b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:19.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.371+0000 7f3e11c14700 1 --2- 172.21.15.16:0/315426902 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5653c039ac00 0x5653c0435600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:19.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.372+0000 7f3e12c16700 1 -- 172.21.15.16:0/315426902 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5653c039b000 msgr2=0x5653c0435b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:19.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.372+0000 7f3e12c16700 1 --2- 172.21.15.16:0/315426902 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5653c039b000 0x5653c0435b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.372+0000 7f3e12c16700 1 -- 172.21.15.16:0/315426902 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5653c039ac00 msgr2=0x5653c0435600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:19.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.372+0000 7f3e12c16700 1 --2- 172.21.15.16:0/315426902 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5653c039ac00 0x5653c0435600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.372+0000 7f3e12c16700 1 -- 172.21.15.16:0/315426902 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5653c03bed00 con 0x5653c039a800 2023-05-20T01:27:19.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.372+0000 7f3e12c16700 1 --2- 172.21.15.16:0/315426902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653c039a800 0x5653c0435080 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x5653c1162960 tx=0x5653c11332c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a756e8be583b39c2 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:19.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.372+0000 7f3e0fc10700 1 -- 172.21.15.16:0/315426902 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5653c03cf080 con 0x5653c039a800 2023-05-20T01:27:19.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.372+0000 7f3e0fc10700 1 -- 172.21.15.16:0/315426902 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5653c03ed800 con 0x5653c039a800 2023-05-20T01:27:19.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.372+0000 7f3e0fc10700 1 -- 172.21.15.16:0/315426902 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5653c03cf600 con 0x5653c039a800 2023-05-20T01:27:19.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.372+0000 7f3e144692c0 1 -- 172.21.15.16:0/315426902 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5653c03bf6c0 con 0x5653c039a800 2023-05-20T01:27:19.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.372+0000 7f3e144692c0 1 -- 172.21.15.16:0/315426902 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x5653c03bf520 con 0x5653c039a800 2023-05-20T01:27:19.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.372+0000 7f3e144692c0 4 waiting for MDS map... 2023-05-20T01:27:19.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.372+0000 7f3e0fc10700 1 -- 172.21.15.16:0/315426902 <== mon.0 v2:172.21.15.16:3300/0 4 ==== osd_map(119..119 src has 1..119) v4 ==== 6987+0+0 (secure 0 0 0) 0x5653c03dcc40 con 0x5653c039a800 2023-05-20T01:27:19.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.372+0000 7f3e0fc10700 1 -- 172.21.15.16:0/315426902 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 84) v1 ==== 3126+0+0 (secure 0 0 0) 0x5653c03e0f00 con 0x5653c039a800 2023-05-20T01:27:19.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.372+0000 7f3e144692c0 4 Got MDS map 84 2023-05-20T01:27:19.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.372+0000 7f3e144692c0 10 main: JournalTool::main 2023-05-20T01:27:19.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.372+0000 7f3e144692c0 4 main: JournalTool: connecting to RADOS... 2023-05-20T01:27:19.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.373+0000 7f3e144692c0 1 Processor -- start 2023-05-20T01:27:19.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.373+0000 7f3e144692c0 1 -- start start 2023-05-20T01:27:19.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.374+0000 7f3e144692c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5653c039bc00 0x5653c0434b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:19.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.374+0000 7f3e144692c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5653c06a2000 0x5653c0434580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:19.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.374+0000 7f3e144692c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653c06a2400 0x5653c0436100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:19.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.374+0000 7f3e144692c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653c03d6b40 con 0x5653c06a2400 2023-05-20T01:27:19.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.374+0000 7f3e144692c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653c03d6a00 con 0x5653c039bc00 2023-05-20T01:27:19.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.374+0000 7f3e144692c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x5653c03d7400 con 0x5653c06a2000 2023-05-20T01:27:19.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.374+0000 7f3e11c14700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653c06a2400 0x5653c0436100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:19.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.374+0000 7f3e11c14700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653c06a2400 0x5653c0436100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38870/0 (socket says 172.21.15.16:38870) 2023-05-20T01:27:19.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.374+0000 7f3e11c14700 1 -- 172.21.15.16:0/2917652793 learned_addr learned my addr 172.21.15.16:0/2917652793 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:27:19.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.374+0000 7f3e12415700 1 --2- 172.21.15.16:0/2917652793 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5653c06a2000 0x5653c0434580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:19.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.374+0000 7f3e11c14700 1 --2- 172.21.15.16:0/2917652793 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5653c039bc00 0x5653c0434b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:19.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.374+0000 7f3e11c14700 1 -- 172.21.15.16:0/2917652793 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5653c06a2000 msgr2=0x5653c0434580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:19.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.374+0000 7f3e11c14700 1 --2- 172.21.15.16:0/2917652793 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5653c06a2000 0x5653c0434580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.374+0000 7f3e11c14700 1 -- 172.21.15.16:0/2917652793 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5653c039bc00 msgr2=0x5653c0434b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:19.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.374+0000 7f3e11c14700 1 --2- 172.21.15.16:0/2917652793 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5653c039bc00 0x5653c0434b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.374+0000 7f3e11c14700 1 -- 172.21.15.16:0/2917652793 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5653c03bf380 con 0x5653c06a2400 2023-05-20T01:27:19.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.374+0000 7f3e11c14700 1 --2- 172.21.15.16:0/2917652793 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653c06a2400 0x5653c0436100 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x5653c06b4060 tx=0x5653c121ade0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=384e616f6284f5c9 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:19.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.375+0000 7f3e0dc0c700 1 -- 172.21.15.16:0/2917652793 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5653c03cf080 con 0x5653c06a2400 2023-05-20T01:27:19.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.375+0000 7f3e0dc0c700 1 -- 172.21.15.16:0/2917652793 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5653c03ec480 con 0x5653c06a2400 2023-05-20T01:27:19.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.375+0000 7f3e0dc0c700 1 -- 172.21.15.16:0/2917652793 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5653c03cf600 con 0x5653c06a2400 2023-05-20T01:27:19.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.375+0000 7f3e144692c0 1 -- 172.21.15.16:0/2917652793 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653c06a2400 msgr2=0x5653c0436100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:19.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.375+0000 7f3e144692c0 1 --2- 172.21.15.16:0/2917652793 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653c06a2400 0x5653c0436100 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x5653c06b4060 tx=0x5653c121ade0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.375+0000 7f3e144692c0 1 -- 172.21.15.16:0/2917652793 shutdown_connections 2023-05-20T01:27:19.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.375+0000 7f3e144692c0 1 --2- 172.21.15.16:0/2917652793 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653c06a2400 0x5653c0436100 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.375+0000 7f3e144692c0 1 --2- 172.21.15.16:0/2917652793 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5653c039bc00 0x5653c0434b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.375+0000 7f3e144692c0 1 --2- 172.21.15.16:0/2917652793 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5653c06a2000 0x5653c0434580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.375+0000 7f3e144692c0 1 -- 172.21.15.16:0/2917652793 >> 172.21.15.16:0/2917652793 conn(0x5653c039a400 msgr2=0x5653c046b800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:27:19.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.375+0000 7f3e144692c0 1 -- 172.21.15.16:0/2917652793 shutdown_connections 2023-05-20T01:27:19.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.375+0000 7f3e144692c0 1 -- 172.21.15.16:0/2917652793 wait complete. 2023-05-20T01:27:19.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.376+0000 7f3e144692c0 1 Processor -- start 2023-05-20T01:27:19.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.376+0000 7f3e144692c0 1 -- start start 2023-05-20T01:27:19.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.376+0000 7f3e144692c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653c039a400 0x5653c0434b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:19.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.376+0000 7f3e144692c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5653c06d2000 0x5653c0434580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:19.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.376+0000 7f3e144692c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5653c06d2400 0x5653c0436100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:19.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.376+0000 7f3e144692c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653c03d6c80 con 0x5653c039a400 2023-05-20T01:27:19.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.376+0000 7f3e144692c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653c03d6000 con 0x5653c06d2000 2023-05-20T01:27:19.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.376+0000 7f3e144692c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x5653c03d7180 con 0x5653c06d2400 2023-05-20T01:27:19.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.376+0000 7f3e11c14700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653c039a400 0x5653c0434b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:19.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.376+0000 7f3e11c14700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653c039a400 0x5653c0434b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38872/0 (socket says 172.21.15.16:38872) 2023-05-20T01:27:19.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.376+0000 7f3e11c14700 1 -- 172.21.15.16:0/1327828722 learned_addr learned my addr 172.21.15.16:0/1327828722 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:27:19.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.376+0000 7f3e12415700 1 --2- 172.21.15.16:0/1327828722 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5653c06d2000 0x5653c0434580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:19.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.377+0000 7f3e11c14700 1 -- 172.21.15.16:0/1327828722 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5653c06d2400 msgr2=0x5653c0436100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:19.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.377+0000 7f3e11c14700 1 --2- 172.21.15.16:0/1327828722 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5653c06d2400 0x5653c0436100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.377+0000 7f3e11c14700 1 -- 172.21.15.16:0/1327828722 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5653c06d2000 msgr2=0x5653c0434580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:19.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.377+0000 7f3e11c14700 1 --2- 172.21.15.16:0/1327828722 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5653c06d2000 0x5653c0434580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.377+0000 7f3e11c14700 1 -- 172.21.15.16:0/1327828722 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5653c03bf1e0 con 0x5653c039a400 2023-05-20T01:27:19.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.377+0000 7f3e11c14700 1 --2- 172.21.15.16:0/1327828722 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653c039a400 0x5653c0434b00 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x5653c06b4d20 tx=0x5653c121acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:19.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.377+0000 7f3e0c409700 1 -- 172.21.15.16:0/1327828722 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5653c03cedc0 con 0x5653c039a400 2023-05-20T01:27:19.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.377+0000 7f3e0c409700 1 -- 172.21.15.16:0/1327828722 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5653c03ed980 con 0x5653c039a400 2023-05-20T01:27:19.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.377+0000 7f3e144692c0 1 -- 172.21.15.16:0/1327828722 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5653c03bfd40 con 0x5653c039a400 2023-05-20T01:27:19.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.377+0000 7f3e0c409700 1 -- 172.21.15.16:0/1327828722 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5653c03cf080 con 0x5653c039a400 2023-05-20T01:27:19.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.377+0000 7f3e144692c0 1 -- 172.21.15.16:0/1327828722 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5653c03bed00 con 0x5653c039a400 2023-05-20T01:27:19.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.377+0000 7f3e144692c0 4 main: JournalTool: resolving pool 15 2023-05-20T01:27:19.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.378+0000 7f3e0c409700 1 -- 172.21.15.16:0/1327828722 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x5653c03fc840 con 0x5653c039a400 2023-05-20T01:27:19.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.378+0000 7f3e0c409700 1 --2- 172.21.15.16:0/1327828722 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x5653c06d4400 0x5653c07be000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:19.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.378+0000 7f3e0c409700 1 -- 172.21.15.16:0/1327828722 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6987+0+0 (secure 0 0 0) 0x5653c03dca80 con 0x5653c039a400 2023-05-20T01:27:19.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.379+0000 7f3e12415700 1 --2- 172.21.15.16:0/1327828722 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x5653c06d4400 0x5653c07be000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:19.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.379+0000 7f3e144692c0 4 main: JournalTool: creating IoCtx.. 2023-05-20T01:27:19.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.379+0000 7f3e144692c0 4 main: Executing for rank 0 2023-05-20T01:27:19.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.379+0000 7f3e12415700 1 --2- 172.21.15.16:0/1327828722 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x5653c06d4400 0x5653c07be000 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x5653c0679e90 tx=0x5653c121bc20 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:19.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.379+0000 7f3e144692c0 1 -- start start 2023-05-20T01:27:19.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.379+0000 7f3e144692c0 1 Processor -- start 2023-05-20T01:27:19.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.380+0000 7f3e144692c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5653c06d2c00 0x5653c0436680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:19.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.380+0000 7f3e144692c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653c06d3000 0x5653c0436c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:19.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.380+0000 7f3e144692c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5653c06d3400 0x5653c0437180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:19.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.380+0000 7f3e144692c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653c03d7180 con 0x5653c06d3000 2023-05-20T01:27:19.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.380+0000 7f3e144692c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653c03d6000 con 0x5653c06d3400 2023-05-20T01:27:19.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.380+0000 7f3e144692c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x5653c03d7400 con 0x5653c06d2c00 2023-05-20T01:27:19.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.380+0000 7f3e12415700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653c06d3000 0x5653c0436c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:19.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.380+0000 7f3e12415700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653c06d3000 0x5653c0436c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38882/0 (socket says 172.21.15.16:38882) 2023-05-20T01:27:19.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.380+0000 7f3e12415700 1 -- 172.21.15.16:0/2232981928 learned_addr learned my addr 172.21.15.16:0/2232981928 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:27:19.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.380+0000 7f3e12c16700 1 --2- 172.21.15.16:0/2232981928 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5653c06d2c00 0x5653c0436680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:19.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.380+0000 7f3e11c14700 1 --2- 172.21.15.16:0/2232981928 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5653c06d3400 0x5653c0437180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:19.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.380+0000 7f3e12415700 1 -- 172.21.15.16:0/2232981928 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5653c06d2c00 msgr2=0x5653c0436680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:19.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.380+0000 7f3e12415700 1 --2- 172.21.15.16:0/2232981928 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5653c06d2c00 0x5653c0436680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.380+0000 7f3e12415700 1 -- 172.21.15.16:0/2232981928 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5653c06d3400 msgr2=0x5653c0437180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:19.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.380+0000 7f3e12415700 1 --2- 172.21.15.16:0/2232981928 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5653c06d3400 0x5653c0437180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.380+0000 7f3e12415700 1 -- 172.21.15.16:0/2232981928 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5653c0748b60 con 0x5653c06d3000 2023-05-20T01:27:19.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.380+0000 7f3e11c14700 1 --2- 172.21.15.16:0/2232981928 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5653c06d3400 0x5653c0437180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:27:19.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.380+0000 7f3e12415700 1 --2- 172.21.15.16:0/2232981928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653c06d3000 0x5653c0436c00 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x5653c0793170 tx=0x5653c116e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bda9420a6fc753c server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:19.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.381+0000 7f3e09403700 1 -- 172.21.15.16:0/2232981928 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5653c03ce6e0 con 0x5653c06d3000 2023-05-20T01:27:19.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.381+0000 7f3e09403700 1 -- 172.21.15.16:0/2232981928 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5653c120e780 con 0x5653c06d3000 2023-05-20T01:27:19.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.381+0000 7f3e09403700 1 -- 172.21.15.16:0/2232981928 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5653c03ce9a0 con 0x5653c06d3000 2023-05-20T01:27:19.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.381+0000 7f3e144692c0 1 -- 172.21.15.16:0/2232981928 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5653c07489c0 con 0x5653c06d3000 2023-05-20T01:27:19.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.381+0000 7f3e144692c0 1 -- 172.21.15.16:0/2232981928 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x5653c07031e0 con 0x5653c06d3000 2023-05-20T01:27:19.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.381+0000 7f3e144692c0 4 waiting for MDS map... 2023-05-20T01:27:19.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.381+0000 7f3e09403700 1 -- 172.21.15.16:0/2232981928 <== mon.0 v2:172.21.15.16:3300/0 4 ==== osd_map(119..119 src has 1..119) v4 ==== 6987+0+0 (secure 0 0 0) 0x5653c03dc8c0 con 0x5653c06d3000 2023-05-20T01:27:19.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.381+0000 7f3e09403700 1 -- 172.21.15.16:0/2232981928 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 84) v1 ==== 3126+0+0 (secure 0 0 0) 0x5653c081d200 con 0x5653c06d3000 2023-05-20T01:27:19.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.381+0000 7f3e144692c0 4 Got MDS map 84 2023-05-20T01:27:19.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.381+0000 7f3e144692c0 1 --2- 172.21.15.16:0/2232981928 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x5653c06d3800 0x5653c0437700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:19.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.381+0000 7f3e144692c0 1 -- 172.21.15.16:0/2232981928 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:1 15.f 15:f1f69726:::400.00000000:head [writefull 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x5653c115ee00 con 0x5653c06d3800 2023-05-20T01:27:19.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.381+0000 7f3e12c16700 1 --2- 172.21.15.16:0/2232981928 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x5653c06d3800 0x5653c0437700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:19.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.382+0000 7f3e12c16700 1 --2- 172.21.15.16:0/2232981928 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x5653c06d3800 0x5653c0437700 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=f4b04924f069b30 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:19.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.383+0000 7f3e12c16700 1 -- 172.21.15.16:0/2232981928 <== osd.6 v2:172.21.15.132:6801/727581123 1 ==== osd_op_reply(1 400.00000000 [writefull 0~22] v119'4 uv4 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x5653c120d200 con 0x5653c06d3800 2023-05-20T01:27:19.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.383+0000 7f3e144692c0 1 -- 172.21.15.16:0/2232981928 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:2 15.14 15:292cf221:::200.00000000:head [writefull 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x5653c115f180 con 0x5653c06d3800 2023-05-20T01:27:19.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.385+0000 7f3e12c16700 1 -- 172.21.15.16:0/2232981928 <== osd.6 v2:172.21.15.132:6801/727581123 2 ==== osd_op_reply(2 200.00000000 [writefull 0~90] v119'45 uv45 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x5653c120d200 con 0x5653c06d3800 2023-05-20T01:27:19.400 INFO:teuthology.orchestra.run.smithi016.stdout:writing EResetJournal entry 2023-05-20T01:27:19.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.385+0000 7f3e144692c0 1 -- 172.21.15.16:0/2232981928 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:3 15.14 15:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x5653c115f500 con 0x5653c06d3800 2023-05-20T01:27:19.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.385+0000 7f3e144692c0 1 --2- 172.21.15.16:0/2232981928 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x5653c06d3c00 0x5653c07be580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:19.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.385+0000 7f3e144692c0 1 -- 172.21.15.16:0/2232981928 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:4 15.1b 15:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x5653c115f880 con 0x5653c06d3c00 2023-05-20T01:27:19.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.385+0000 7f3e144692c0 1 --2- 172.21.15.16:0/2232981928 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x5653c06d4c00 0x5653c07beb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:19.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.385+0000 7f3e144692c0 1 -- 172.21.15.16:0/2232981928 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:5 15.b 15:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x5653c115fc00 con 0x5653c06d4c00 2023-05-20T01:27:19.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.385+0000 7f3e144692c0 1 --2- 172.21.15.16:0/2232981928 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x5653c06d5000 0x5653c07bf080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:19.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.385+0000 7f3e144692c0 1 -- 172.21.15.16:0/2232981928 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:6 15.1f 15:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x5653c115ee00 con 0x5653c06d5000 2023-05-20T01:27:19.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.385+0000 7f3e144692c0 1 --2- 172.21.15.16:0/2232981928 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x5653c06d5400 0x5653c07bf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:19.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.385+0000 7f3e144692c0 1 -- 172.21.15.16:0/2232981928 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:7 15.1a 15:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x5653c115f180 con 0x5653c06d5400 2023-05-20T01:27:19.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.385+0000 7f3e144692c0 1 -- 172.21.15.16:0/2232981928 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:8 15.1e 15:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x5653c115f500 con 0x5653c06d4c00 2023-05-20T01:27:19.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.385+0000 7f3e12415700 1 --2- 172.21.15.16:0/2232981928 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x5653c06d5000 0x5653c07bf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:19.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.385+0000 7f3e12c16700 1 --2- 172.21.15.16:0/2232981928 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x5653c06d4c00 0x5653c07beb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:19.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.385+0000 7f3e11c14700 1 --2- 172.21.15.16:0/2232981928 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x5653c06d3c00 0x5653c07be580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:19.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.385+0000 7f3e11c14700 1 --2- 172.21.15.16:0/2232981928 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x5653c06d5400 0x5653c07bf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:19.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.385+0000 7f3e12415700 1 --2- 172.21.15.16:0/2232981928 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x5653c06d5000 0x5653c07bf080 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=a9563b39933e4602 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:19.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.385+0000 7f3e12c16700 1 --2- 172.21.15.16:0/2232981928 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x5653c06d4c00 0x5653c07beb00 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=ce4cd9e9f5bd1fbe server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:19.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.385+0000 7f3e11c14700 1 --2- 172.21.15.16:0/2232981928 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x5653c06d3c00 0x5653c07be580 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=1f2d161d955a204e server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:19.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.386+0000 7f3e11c14700 1 --2- 172.21.15.16:0/2232981928 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x5653c06d5400 0x5653c07bf600 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=762798ae6a952f62 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:19.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.386+0000 7f3e12c16700 1 -- 172.21.15.16:0/2232981928 <== osd.6 v2:172.21.15.132:6801/727581123 3 ==== osd_op_reply(3 200.00000001 [delete] v119'46 uv45 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x5653c120d200 con 0x5653c06d3800 2023-05-20T01:27:19.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.387+0000 7f3e12c16700 1 -- 172.21.15.16:0/2232981928 <== osd.2 v2:172.21.15.16:6816/293187030 1 ==== osd_op_reply(5 200.00000003 [delete] v119'6 uv5 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x5653c120d200 con 0x5653c06d4c00 2023-05-20T01:27:19.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.388+0000 7f3e12c16700 1 -- 172.21.15.16:0/2232981928 <== osd.2 v2:172.21.15.16:6816/293187030 2 ==== osd_op_reply(8 200.00000006 [delete] v119'2 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x5653c120d200 con 0x5653c06d4c00 2023-05-20T01:27:19.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.388+0000 7f3e11c14700 1 -- 172.21.15.16:0/2232981928 <== osd.5 v2:172.21.15.132:6816/1844689995 1 ==== osd_op_reply(7 200.00000005 [delete] v119'2 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x5653c120d8c0 con 0x5653c06d5400 2023-05-20T01:27:19.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.391+0000 7f3e12415700 1 -- 172.21.15.16:0/2232981928 <== osd.3 v2:172.21.15.16:6800/3669781890 1 ==== osd_op_reply(6 200.00000004 [delete] v119'9 uv8 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x5653c085e000 con 0x5653c06d5000 2023-05-20T01:27:19.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.391+0000 7f3e11c14700 1 -- 172.21.15.16:0/2232981928 <== osd.7 v2:172.21.15.132:6800/466001149 1 ==== osd_op_reply(4 200.00000002 [delete] v119'2 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x5653c120d8c0 con 0x5653c06d3c00 2023-05-20T01:27:19.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.391+0000 7f3e07c00700 1 -- 172.21.15.16:0/2232981928 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:9 15.14 15:2e2fa760:::200.00000001:head [write 0~48 in=48b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x5653c115f880 con 0x5653c06d3800 2023-05-20T01:27:19.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.394+0000 7f3e12c16700 1 -- 172.21.15.16:0/2232981928 <== osd.6 v2:172.21.15.132:6801/727581123 4 ==== osd_op_reply(9 200.00000001 [write 0~48] v119'47 uv47 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x5653c120d200 con 0x5653c06d3800 2023-05-20T01:27:19.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.394+0000 7f3e144692c0 4 Successfully wrote new journal pointer and header for rank 5:0 2023-05-20T01:27:19.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.394+0000 7f3e144692c0 1 -- 172.21.15.16:0/2232981928 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x5653c06d4c00 msgr2=0x5653c07beb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:19.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.394+0000 7f3e144692c0 1 --2- 172.21.15.16:0/2232981928 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x5653c06d4c00 0x5653c07beb00 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.394+0000 7f3e144692c0 1 -- 172.21.15.16:0/2232981928 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x5653c06d5000 msgr2=0x5653c07bf080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:19.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.394+0000 7f3e144692c0 1 --2- 172.21.15.16:0/2232981928 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x5653c06d5000 0x5653c07bf080 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.394+0000 7f3e144692c0 1 -- 172.21.15.16:0/2232981928 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x5653c06d5400 msgr2=0x5653c07bf600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:19.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.394+0000 7f3e144692c0 1 --2- 172.21.15.16:0/2232981928 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x5653c06d5400 0x5653c07bf600 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.394+0000 7f3e144692c0 1 -- 172.21.15.16:0/2232981928 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x5653c06d3800 msgr2=0x5653c0437700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:19.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.394+0000 7f3e144692c0 1 --2- 172.21.15.16:0/2232981928 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x5653c06d3800 0x5653c0437700 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.394+0000 7f3e144692c0 1 -- 172.21.15.16:0/2232981928 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x5653c06d3c00 msgr2=0x5653c07be580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:19.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.394+0000 7f3e144692c0 1 --2- 172.21.15.16:0/2232981928 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x5653c06d3c00 0x5653c07be580 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.394+0000 7f3e11c14700 1 -- 172.21.15.16:0/2232981928 reap_dead start 2023-05-20T01:27:19.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.394+0000 7f3e11c14700 1 -- 172.21.15.16:0/2232981928 reap_dead start 2023-05-20T01:27:19.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.394+0000 7f3e11c14700 1 -- 172.21.15.16:0/2232981928 reap_dead start 2023-05-20T01:27:19.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.394+0000 7f3e144692c0 1 -- 172.21.15.16:0/2232981928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653c06d3000 msgr2=0x5653c0436c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:19.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.394+0000 7f3e144692c0 1 --2- 172.21.15.16:0/2232981928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653c06d3000 0x5653c0436c00 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x5653c0793170 tx=0x5653c116e000 comp rx=0 tx=0).stop 2023-05-20T01:27:19.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.394+0000 7f3e144692c0 1 -- 172.21.15.16:0/2232981928 shutdown_connections 2023-05-20T01:27:19.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.394+0000 7f3e144692c0 1 --2- 172.21.15.16:0/2232981928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653c06d3000 0x5653c0436c00 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.394+0000 7f3e144692c0 1 -- 172.21.15.16:0/2232981928 >> 172.21.15.16:0/2232981928 conn(0x5653c06d2800 msgr2=0x5653c046c000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:27:19.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.394+0000 7f3e144692c0 1 -- 172.21.15.16:0/2232981928 shutdown_connections 2023-05-20T01:27:19.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.394+0000 7f3e144692c0 1 -- 172.21.15.16:0/2232981928 wait complete. 2023-05-20T01:27:19.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.394+0000 7f3e144692c0 1 -- 172.21.15.16:0/1327828722 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x5653c06d4400 msgr2=0x5653c07be000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:19.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.394+0000 7f3e144692c0 1 --2- 172.21.15.16:0/1327828722 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x5653c06d4400 0x5653c07be000 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x5653c0679e90 tx=0x5653c121bc20 comp rx=0 tx=0).stop 2023-05-20T01:27:19.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.394+0000 7f3e144692c0 1 -- 172.21.15.16:0/1327828722 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653c039a400 msgr2=0x5653c0434b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:19.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.394+0000 7f3e144692c0 1 --2- 172.21.15.16:0/1327828722 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653c039a400 0x5653c0434b00 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x5653c06b4d20 tx=0x5653c121acc0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.394+0000 7f3e144692c0 1 -- 172.21.15.16:0/1327828722 shutdown_connections 2023-05-20T01:27:19.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.394+0000 7f3e144692c0 1 --2- 172.21.15.16:0/1327828722 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x5653c06d4400 0x5653c07be000 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.394+0000 7f3e144692c0 1 --2- 172.21.15.16:0/1327828722 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5653c06d2400 0x5653c0436100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.394+0000 7f3e144692c0 1 --2- 172.21.15.16:0/1327828722 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653c039a400 0x5653c0434b00 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.394+0000 7f3e144692c0 1 --2- 172.21.15.16:0/1327828722 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5653c06d2000 0x5653c0434580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.394+0000 7f3e144692c0 1 -- 172.21.15.16:0/1327828722 >> 172.21.15.16:0/1327828722 conn(0x5653c039bc00 msgr2=0x5653c046b800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:27:19.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.394+0000 7f3e144692c0 1 -- 172.21.15.16:0/1327828722 shutdown_connections 2023-05-20T01:27:19.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.394+0000 7f3e144692c0 1 -- 172.21.15.16:0/1327828722 wait complete. 2023-05-20T01:27:19.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.395+0000 7f3e144692c0 1 -- 172.21.15.16:0/315426902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653c039a800 msgr2=0x5653c0435080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:19.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.395+0000 7f3e144692c0 1 --2- 172.21.15.16:0/315426902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653c039a800 0x5653c0435080 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x5653c1162960 tx=0x5653c11332c0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.395+0000 7f3e144692c0 1 -- 172.21.15.16:0/315426902 shutdown_connections 2023-05-20T01:27:19.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.395+0000 7f3e144692c0 1 --2- 172.21.15.16:0/315426902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653c039a800 0x5653c0435080 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.395+0000 7f3e144692c0 1 --2- 172.21.15.16:0/315426902 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5653c039b000 0x5653c0435b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.395+0000 7f3e144692c0 1 --2- 172.21.15.16:0/315426902 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5653c039ac00 0x5653c0435600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.395+0000 7f3e144692c0 1 -- 172.21.15.16:0/315426902 >> 172.21.15.16:0/315426902 conn(0x5653c039b400 msgr2=0x5653c046b000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:27:19.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.395+0000 7f3e144692c0 1 -- 172.21.15.16:0/315426902 shutdown_connections 2023-05-20T01:27:19.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.395+0000 7f3e144692c0 1 -- 172.21.15.16:0/315426902 wait complete. 2023-05-20T01:27:19.412 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'journal', 'reset', '--force'] in time 0:00:00.118105, result: writing EResetJournal entry 2023-05-20T01:27:19.413 DEBUG:teuthology.orchestra.run.smithi016:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 init 2023-05-20T01:27:19.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.448+0000 7f80e31bf7c0 1 -- start start 2023-05-20T01:27:19.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.449+0000 7f80e31bf7c0 1 Processor -- start 2023-05-20T01:27:19.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.449+0000 7f80e31bf7c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5569e9cfb000 0x5569e9d94580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:19.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.449+0000 7f80e31bf7c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5569e9cfac00 0x5569e9d94b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:19.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.449+0000 7f80e31bf7c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5569e9cfa800 0x5569e9d95080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:19.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.449+0000 7f80e31bf7c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569e9d37cc0 con 0x5569e9cfb000 2023-05-20T01:27:19.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.449+0000 7f80e31bf7c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569e9d37400 con 0x5569e9cfa800 2023-05-20T01:27:19.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.449+0000 7f80e31bf7c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x5569e9d37e00 con 0x5569e9cfac00 2023-05-20T01:27:19.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.449+0000 7f80e116d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5569e9cfb000 0x5569e9d94580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:19.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.449+0000 7f80e116d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5569e9cfb000 0x5569e9d94580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38898/0 (socket says 172.21.15.16:38898) 2023-05-20T01:27:19.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.449+0000 7f80e116d700 1 -- 172.21.15.16:0/1469824962 learned_addr learned my addr 172.21.15.16:0/1469824962 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:27:19.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.449+0000 7f80e196e700 1 --2- 172.21.15.16:0/1469824962 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5569e9cfa800 0x5569e9d95080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:19.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.449+0000 7f80e096c700 1 --2- 172.21.15.16:0/1469824962 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5569e9cfac00 0x5569e9d94b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:19.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.450+0000 7f80e116d700 1 -- 172.21.15.16:0/1469824962 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5569e9cfac00 msgr2=0x5569e9d94b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:19.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.450+0000 7f80e116d700 1 --2- 172.21.15.16:0/1469824962 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5569e9cfac00 0x5569e9d94b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.450+0000 7f80e116d700 1 -- 172.21.15.16:0/1469824962 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5569e9cfa800 msgr2=0x5569e9d95080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:19.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.450+0000 7f80e116d700 1 --2- 172.21.15.16:0/1469824962 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5569e9cfa800 0x5569e9d95080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.450+0000 7f80e116d700 1 -- 172.21.15.16:0/1469824962 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5569e9d1ed00 con 0x5569e9cfb000 2023-05-20T01:27:19.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.450+0000 7f80e116d700 1 --2- 172.21.15.16:0/1469824962 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5569e9cfb000 0x5569e9d94580 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x5569e9e5adb0 tx=0x5569e9e64ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b210e4df9e034af2 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:19.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.450+0000 7f80de968700 1 -- 172.21.15.16:0/1469824962 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5569e9d2f600 con 0x5569e9cfb000 2023-05-20T01:27:19.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.450+0000 7f80de968700 1 -- 172.21.15.16:0/1469824962 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5569e9d4c480 con 0x5569e9cfb000 2023-05-20T01:27:19.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.450+0000 7f80de968700 1 -- 172.21.15.16:0/1469824962 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5569e9d2f1e0 con 0x5569e9cfb000 2023-05-20T01:27:19.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.450+0000 7f80e31bf7c0 1 -- 172.21.15.16:0/1469824962 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5569e9d1f6c0 con 0x5569e9cfb000 2023-05-20T01:27:19.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.450+0000 7f80e31bf7c0 1 -- 172.21.15.16:0/1469824962 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x5569e9d1f520 con 0x5569e9cfb000 2023-05-20T01:27:19.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.450+0000 7f80e31bf7c0 4 waiting for MDS map... 2023-05-20T01:27:19.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.450+0000 7f80de968700 1 -- 172.21.15.16:0/1469824962 <== mon.0 v2:172.21.15.16:3300/0 4 ==== osd_map(119..119 src has 1..119) v4 ==== 6987+0+0 (secure 0 0 0) 0x5569e9d3cc40 con 0x5569e9cfb000 2023-05-20T01:27:19.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.450+0000 7f80de968700 1 -- 172.21.15.16:0/1469824962 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 84) v1 ==== 3126+0+0 (secure 0 0 0) 0x5569eab62900 con 0x5569e9cfb000 2023-05-20T01:27:19.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.450+0000 7f80e31bf7c0 4 Got MDS map 84 2023-05-20T01:27:19.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.450+0000 7f80e31bf7c0 4 datascan.main: Using metadata pool output 2023-05-20T01:27:19.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.450+0000 7f80e31bf7c0 4 datascan.main: connecting to RADOS... 2023-05-20T01:27:19.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.451+0000 7f80e31bf7c0 1 Processor -- start 2023-05-20T01:27:19.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.451+0000 7f80e31bf7c0 1 -- start start 2023-05-20T01:27:19.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.451+0000 7f80e31bf7c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5569e9cfbc00 0x5569e9d95600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:19.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.452+0000 7f80e31bf7c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5569ea014000 0x5569e9d95b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:19.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.452+0000 7f80e31bf7c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5569ea014400 0x5569e9d96100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:19.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.452+0000 7f80e31bf7c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569e9d36a00 con 0x5569e9cfbc00 2023-05-20T01:27:19.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.452+0000 7f80e31bf7c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569e9d37400 con 0x5569ea014000 2023-05-20T01:27:19.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.452+0000 7f80e31bf7c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x5569e9d36b40 con 0x5569ea014400 2023-05-20T01:27:19.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.452+0000 7f80e196e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5569e9cfbc00 0x5569e9d95600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:19.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.452+0000 7f80e196e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5569e9cfbc00 0x5569e9d95600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38904/0 (socket says 172.21.15.16:38904) 2023-05-20T01:27:19.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.452+0000 7f80e196e700 1 -- 172.21.15.16:0/3062868418 learned_addr learned my addr 172.21.15.16:0/3062868418 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:27:19.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.452+0000 7f80e096c700 1 --2- 172.21.15.16:0/3062868418 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5569ea014400 0x5569e9d96100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:19.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.452+0000 7f80e116d700 1 --2- 172.21.15.16:0/3062868418 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5569ea014000 0x5569e9d95b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:19.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.452+0000 7f80e196e700 1 -- 172.21.15.16:0/3062868418 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5569ea014400 msgr2=0x5569e9d96100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:19.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.452+0000 7f80e196e700 1 --2- 172.21.15.16:0/3062868418 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5569ea014400 0x5569e9d96100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.452+0000 7f80e196e700 1 -- 172.21.15.16:0/3062868418 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5569ea014000 msgr2=0x5569e9d95b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:19.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.452+0000 7f80e196e700 1 --2- 172.21.15.16:0/3062868418 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5569ea014000 0x5569e9d95b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.452+0000 7f80e196e700 1 -- 172.21.15.16:0/3062868418 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5569e9d1f380 con 0x5569e9cfbc00 2023-05-20T01:27:19.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.452+0000 7f80e096c700 1 --2- 172.21.15.16:0/3062868418 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5569ea014400 0x5569e9d96100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:27:19.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.452+0000 7f80e196e700 1 --2- 172.21.15.16:0/3062868418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5569e9cfbc00 0x5569e9d95600 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x5569ea01a0f0 tx=0x5569eab86c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdc96eeb8ffff626 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:19.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.452+0000 7f80dc964700 1 -- 172.21.15.16:0/3062868418 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5569e9d2f600 con 0x5569e9cfbc00 2023-05-20T01:27:19.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.452+0000 7f80dc964700 1 -- 172.21.15.16:0/3062868418 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5569e9d4de00 con 0x5569e9cfbc00 2023-05-20T01:27:19.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.452+0000 7f80dc964700 1 -- 172.21.15.16:0/3062868418 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5569e9d2f1e0 con 0x5569e9cfbc00 2023-05-20T01:27:19.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.453+0000 7f80e31bf7c0 1 -- 172.21.15.16:0/3062868418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5569e9cfbc00 msgr2=0x5569e9d95600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:19.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.453+0000 7f80e31bf7c0 1 --2- 172.21.15.16:0/3062868418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5569e9cfbc00 0x5569e9d95600 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x5569ea01a0f0 tx=0x5569eab86c00 comp rx=0 tx=0).stop 2023-05-20T01:27:19.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.453+0000 7f80e31bf7c0 1 -- 172.21.15.16:0/3062868418 shutdown_connections 2023-05-20T01:27:19.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.453+0000 7f80e31bf7c0 1 --2- 172.21.15.16:0/3062868418 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5569ea014400 0x5569e9d96100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.453+0000 7f80e31bf7c0 1 --2- 172.21.15.16:0/3062868418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5569e9cfbc00 0x5569e9d95600 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.453+0000 7f80e31bf7c0 1 --2- 172.21.15.16:0/3062868418 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5569ea014000 0x5569e9d95b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.453+0000 7f80e31bf7c0 1 -- 172.21.15.16:0/3062868418 >> 172.21.15.16:0/3062868418 conn(0x5569e9cfa400 msgr2=0x5569e9dd7800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:27:19.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.453+0000 7f80e31bf7c0 1 -- 172.21.15.16:0/3062868418 shutdown_connections 2023-05-20T01:27:19.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.453+0000 7f80e31bf7c0 1 -- 172.21.15.16:0/3062868418 wait complete. 2023-05-20T01:27:19.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.453+0000 7f80e31bf7c0 1 Processor -- start 2023-05-20T01:27:19.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.453+0000 7f80e31bf7c0 1 -- start start 2023-05-20T01:27:19.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.454+0000 7f80e31bf7c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5569e9cfa400 0x5569e9d97180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:19.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.454+0000 7f80e31bf7c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5569ea014000 0x5569e9d96c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:19.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.454+0000 7f80e31bf7c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5569ea014400 0x5569e9d96680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:19.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.454+0000 7f80e31bf7c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569e9d36c80 con 0x5569ea014000 2023-05-20T01:27:19.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.454+0000 7f80e31bf7c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569e9d36000 con 0x5569ea014400 2023-05-20T01:27:19.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.454+0000 7f80e31bf7c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x5569e9d37180 con 0x5569e9cfa400 2023-05-20T01:27:19.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.454+0000 7f80e116d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5569ea014000 0x5569e9d96c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:19.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.454+0000 7f80e116d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5569ea014000 0x5569e9d96c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38912/0 (socket says 172.21.15.16:38912) 2023-05-20T01:27:19.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.454+0000 7f80e116d700 1 -- 172.21.15.16:0/1148140233 learned_addr learned my addr 172.21.15.16:0/1148140233 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:27:19.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.454+0000 7f80e196e700 1 --2- 172.21.15.16:0/1148140233 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5569e9cfa400 0x5569e9d97180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:19.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.454+0000 7f80e096c700 1 --2- 172.21.15.16:0/1148140233 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5569ea014400 0x5569e9d96680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:19.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.454+0000 7f80e116d700 1 -- 172.21.15.16:0/1148140233 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5569e9cfa400 msgr2=0x5569e9d97180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:19.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.454+0000 7f80e116d700 1 --2- 172.21.15.16:0/1148140233 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5569e9cfa400 0x5569e9d97180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.454+0000 7f80e116d700 1 -- 172.21.15.16:0/1148140233 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5569ea014400 msgr2=0x5569e9d96680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:19.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.454+0000 7f80e116d700 1 --2- 172.21.15.16:0/1148140233 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5569ea014400 0x5569e9d96680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.454+0000 7f80e116d700 1 -- 172.21.15.16:0/1148140233 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5569e9d1ed00 con 0x5569ea014000 2023-05-20T01:27:19.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.455+0000 7f80e116d700 1 --2- 172.21.15.16:0/1148140233 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5569ea014000 0x5569e9d96c00 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x5569ea01b2f0 tx=0x5569e9e64060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:19.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.455+0000 7f80db161700 1 -- 172.21.15.16:0/1148140233 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5569e9d2ef20 con 0x5569ea014000 2023-05-20T01:27:19.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.455+0000 7f80db161700 1 -- 172.21.15.16:0/1148140233 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5569eab7e300 con 0x5569ea014000 2023-05-20T01:27:19.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.455+0000 7f80e31bf7c0 1 -- 172.21.15.16:0/1148140233 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5569e9d1fd40 con 0x5569ea014000 2023-05-20T01:27:19.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.455+0000 7f80db161700 1 -- 172.21.15.16:0/1148140233 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5569e9d2f600 con 0x5569ea014000 2023-05-20T01:27:19.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.455+0000 7f80e31bf7c0 1 -- 172.21.15.16:0/1148140233 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5569e9d1f6c0 con 0x5569ea014000 2023-05-20T01:27:19.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.455+0000 7f80e31bf7c0 4 datascan.init: resolving metadata pool 15 2023-05-20T01:27:19.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.456+0000 7f80db161700 1 -- 172.21.15.16:0/1148140233 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x5569e9d5c840 con 0x5569ea014000 2023-05-20T01:27:19.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.456+0000 7f80db161700 1 --2- 172.21.15.16:0/1148140233 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x5569ea032800 0x5569e9d97700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:19.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.456+0000 7f80db161700 1 -- 172.21.15.16:0/1148140233 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6987+0+0 (secure 0 0 0) 0x5569e9d3ca80 con 0x5569ea014000 2023-05-20T01:27:19.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.456+0000 7f80e31bf7c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-05-20T01:27:19.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.456+0000 7f80e196e700 1 --2- 172.21.15.16:0/1148140233 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x5569ea032800 0x5569e9d97700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:19.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.456+0000 7f80e31bf7c0 1 --2- 172.21.15.16:0/1148140233 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x5569ea014800 0x5569e9d96100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:19.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.456+0000 7f80e31bf7c0 1 -- 172.21.15.16:0/1148140233 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:1 15.14 15:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x5569eab66700 con 0x5569ea014800 2023-05-20T01:27:19.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.457+0000 7f80e096c700 1 --2- 172.21.15.16:0/1148140233 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x5569ea014800 0x5569e9d96100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:19.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.457+0000 7f80e196e700 1 --2- 172.21.15.16:0/1148140233 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x5569ea032800 0x5569e9d97700 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x5569ea01a0f0 tx=0x5569e9e64e40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:19.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.457+0000 7f80e096c700 1 --2- 172.21.15.16:0/1148140233 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x5569ea014800 0x5569e9d96100 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:19.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.457+0000 7f80e096c700 1 -- 172.21.15.16:0/1148140233 <== osd.6 v2:172.21.15.132:6801/727581123 1 ==== osd_op_reply(1 1.00000000.inode [stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 160+0+0 (crc 0 0 0) 0x5569ea005440 con 0x5569ea014800 2023-05-20T01:27:19.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.458+0000 7f80e31bf7c0 1 -- 172.21.15.16:0/1148140233 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:2 15.14 15:287034c4:::1.00000000.inode:head [writefull 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x5569eab66a80 con 0x5569ea014800 2023-05-20T01:27:19.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.459+0000 7f80e096c700 1 -- 172.21.15.16:0/1148140233 <== osd.6 v2:172.21.15.132:6801/727581123 2 ==== osd_op_reply(2 1.00000000.inode [writefull 0~582] v119'48 uv48 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x5569ea005440 con 0x5569ea014800 2023-05-20T01:27:19.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.459+0000 7f80e31bf7c0 1 --2- 172.21.15.16:0/1148140233 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x5569ea014c00 0x5569e9d95b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:19.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.459+0000 7f80e31bf7c0 1 -- 172.21.15.16:0/1148140233 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:3 15.1f 15:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x5569eab66e00 con 0x5569ea014c00 2023-05-20T01:27:19.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.459+0000 7f80e196e700 1 --2- 172.21.15.16:0/1148140233 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x5569ea014c00 0x5569e9d95b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:19.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.460+0000 7f80e196e700 1 --2- 172.21.15.16:0/1148140233 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x5569ea014c00 0x5569e9d95b80 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:19.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.460+0000 7f80e196e700 1 -- 172.21.15.16:0/1148140233 <== osd.3 v2:172.21.15.16:6800/3669781890 1 ==== osd_op_reply(3 100.00000000.inode [stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 162+0+0 (crc 0 0 0) 0x5569ea005b00 con 0x5569ea014c00 2023-05-20T01:27:19.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.460+0000 7f80e31bf7c0 1 -- 172.21.15.16:0/1148140233 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:4 15.1f 15:fe07bba1:::100.00000000.inode:head [writefull 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x5569eab66700 con 0x5569ea014c00 2023-05-20T01:27:19.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.461+0000 7f80e196e700 1 -- 172.21.15.16:0/1148140233 <== osd.3 v2:172.21.15.16:6800/3669781890 2 ==== osd_op_reply(4 100.00000000.inode [writefull 0~582] v119'10 uv10 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x5569ea005b00 con 0x5569ea014c00 2023-05-20T01:27:19.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.461+0000 7f80e31bf7c0 1 --2- 172.21.15.16:0/1148140233 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x5569ea015000 0x5569e9d95600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:19.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.461+0000 7f80e31bf7c0 1 -- 172.21.15.16:0/1148140233 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:5 15.13 15:cd5a64a3:::100.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x5569eab66a80 con 0x5569ea015000 2023-05-20T01:27:19.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.461+0000 7f80e116d700 1 --2- 172.21.15.16:0/1148140233 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x5569ea015000 0x5569e9d95600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:19.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.462+0000 7f80e116d700 1 --2- 172.21.15.16:0/1148140233 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x5569ea015000 0x5569e9d95600 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:19.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.462+0000 7f80e116d700 1 -- 172.21.15.16:0/1148140233 <== osd.1 v2:172.21.15.16:6808/3860264220 1 ==== osd_op_reply(5 100.00000000 [omap-get-header] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x5569ea13c000 con 0x5569ea015000 2023-05-20T01:27:19.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.462+0000 7f80e31bf7c0 10 datascan.find_or_create_dirfrag: read_version = 0 2023-05-20T01:27:19.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.462+0000 7f80e31bf7c0 1 -- 172.21.15.16:0/1148140233 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:6 15.13 15:cd5a64a3:::100.00000000:head [create,omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x5569eab67180 con 0x5569ea015000 2023-05-20T01:27:19.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.463+0000 7f80e116d700 1 -- 172.21.15.16:0/1148140233 <== osd.1 v2:172.21.15.16:6808/3860264220 2 ==== osd_op_reply(6 100.00000000 [create,omap-set-header] v119'7 uv7 ondisk = 0) v8 ==== 198+0+0 (crc 0 0 0) 0x5569ea13c000 con 0x5569ea015000 2023-05-20T01:27:19.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.463+0000 7f80e31bf7c0 10 datascan.find_or_create_dirfrag: Created dirfrag: 0x0x100 2023-05-20T01:27:19.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.463+0000 7f80e31bf7c0 1 -- 172.21.15.16:0/1148140233 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x5569ea015000 msgr2=0x5569e9d95600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:19.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.463+0000 7f80e31bf7c0 1 --2- 172.21.15.16:0/1148140233 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x5569ea015000 0x5569e9d95600 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.463+0000 7f80e31bf7c0 1 -- 172.21.15.16:0/1148140233 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x5569ea014c00 msgr2=0x5569e9d95b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:19.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.463+0000 7f80e31bf7c0 1 --2- 172.21.15.16:0/1148140233 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x5569ea014c00 0x5569e9d95b80 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.463+0000 7f80e31bf7c0 1 -- 172.21.15.16:0/1148140233 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x5569ea014800 msgr2=0x5569e9d96100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:19.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.463+0000 7f80e31bf7c0 1 --2- 172.21.15.16:0/1148140233 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x5569ea014800 0x5569e9d96100 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.463+0000 7f80e096c700 1 -- 172.21.15.16:0/1148140233 reap_dead start 2023-05-20T01:27:19.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.463+0000 7f80e31bf7c0 1 -- 172.21.15.16:0/1148140233 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x5569ea032800 msgr2=0x5569e9d97700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:19.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.463+0000 7f80e31bf7c0 1 --2- 172.21.15.16:0/1148140233 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x5569ea032800 0x5569e9d97700 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x5569ea01a0f0 tx=0x5569e9e64e40 comp rx=0 tx=0).stop 2023-05-20T01:27:19.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.463+0000 7f80e31bf7c0 1 -- 172.21.15.16:0/1148140233 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5569ea014000 msgr2=0x5569e9d96c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:19.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.463+0000 7f80e31bf7c0 1 --2- 172.21.15.16:0/1148140233 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5569ea014000 0x5569e9d96c00 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x5569ea01b2f0 tx=0x5569e9e64060 comp rx=0 tx=0).stop 2023-05-20T01:27:19.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.464+0000 7f80e31bf7c0 1 -- 172.21.15.16:0/1148140233 shutdown_connections 2023-05-20T01:27:19.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.464+0000 7f80e31bf7c0 1 --2- 172.21.15.16:0/1148140233 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5569ea014000 0x5569e9d96c00 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.464+0000 7f80e31bf7c0 1 --2- 172.21.15.16:0/1148140233 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x5569ea032800 0x5569e9d97700 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.464+0000 7f80e31bf7c0 1 -- 172.21.15.16:0/1148140233 >> 172.21.15.16:0/1148140233 conn(0x5569e9cfbc00 msgr2=0x5569e9dd7800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:27:19.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.464+0000 7f80e31bf7c0 1 -- 172.21.15.16:0/1148140233 shutdown_connections 2023-05-20T01:27:19.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.464+0000 7f80e31bf7c0 1 -- 172.21.15.16:0/1148140233 wait complete. 2023-05-20T01:27:19.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.464+0000 7f80e31bf7c0 1 -- 172.21.15.16:0/1469824962 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5569e9cfb000 msgr2=0x5569e9d94580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:19.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.464+0000 7f80e31bf7c0 1 --2- 172.21.15.16:0/1469824962 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5569e9cfb000 0x5569e9d94580 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x5569e9e5adb0 tx=0x5569e9e64ea0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.464+0000 7f80e31bf7c0 1 -- 172.21.15.16:0/1469824962 shutdown_connections 2023-05-20T01:27:19.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.464+0000 7f80e31bf7c0 1 --2- 172.21.15.16:0/1469824962 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5569e9cfa800 0x5569e9d95080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.464+0000 7f80e31bf7c0 1 --2- 172.21.15.16:0/1469824962 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5569e9cfb000 0x5569e9d94580 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.464+0000 7f80e31bf7c0 1 --2- 172.21.15.16:0/1469824962 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5569e9cfac00 0x5569e9d94b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.464+0000 7f80e31bf7c0 1 -- 172.21.15.16:0/1469824962 >> 172.21.15.16:0/1469824962 conn(0x5569e9cfb400 msgr2=0x5569e9dd7000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:27:19.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.464+0000 7f80e31bf7c0 1 -- 172.21.15.16:0/1469824962 shutdown_connections 2023-05-20T01:27:19.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.464+0000 7f80e31bf7c0 1 -- 172.21.15.16:0/1469824962 wait complete. 2023-05-20T01:27:19.498 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'init'] in time 0:00:00.085296, result: 2023-05-20T01:27:19.499 DEBUG:teuthology.orchestra.run.smithi016:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_extents 2023-05-20T01:27:19.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.575+0000 7fcfbe0ce7c0 1 -- start start 2023-05-20T01:27:19.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.575+0000 7fcfbe0ce7c0 1 Processor -- start 2023-05-20T01:27:19.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.576+0000 7fcfbe0ce7c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b5159b000 0x563b51634580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:19.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.576+0000 7fcfbe0ce7c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563b5159ac00 0x563b51634b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:19.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.576+0000 7fcfbe0ce7c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563b5159a800 0x563b51635080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:19.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.576+0000 7fcfbe0ce7c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b515d7cc0 con 0x563b5159b000 2023-05-20T01:27:19.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.576+0000 7fcfbe0ce7c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b515d7400 con 0x563b5159ac00 2023-05-20T01:27:19.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.576+0000 7fcfbe0ce7c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x563b515d7e00 con 0x563b5159a800 2023-05-20T01:27:19.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.576+0000 7fcfbc07c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b5159b000 0x563b51634580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:19.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.576+0000 7fcfbc07c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b5159b000 0x563b51634580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38930/0 (socket says 172.21.15.16:38930) 2023-05-20T01:27:19.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.576+0000 7fcfbc07c700 1 -- 172.21.15.16:0/796601313 learned_addr learned my addr 172.21.15.16:0/796601313 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:27:19.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.576+0000 7fcfbb87b700 1 --2- 172.21.15.16:0/796601313 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563b5159ac00 0x563b51634b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:19.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.576+0000 7fcfbc87d700 1 --2- 172.21.15.16:0/796601313 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563b5159a800 0x563b51635080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:19.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.576+0000 7fcfbc07c700 1 -- 172.21.15.16:0/796601313 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563b5159a800 msgr2=0x563b51635080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:19.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.576+0000 7fcfbc07c700 1 --2- 172.21.15.16:0/796601313 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563b5159a800 0x563b51635080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.576+0000 7fcfbc07c700 1 -- 172.21.15.16:0/796601313 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563b5159ac00 msgr2=0x563b51634b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:19.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.576+0000 7fcfbc07c700 1 --2- 172.21.15.16:0/796601313 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563b5159ac00 0x563b51634b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.576+0000 7fcfbc07c700 1 -- 172.21.15.16:0/796601313 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563b515be4e0 con 0x563b5159b000 2023-05-20T01:27:19.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.576+0000 7fcfbb87b700 1 --2- 172.21.15.16:0/796601313 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563b5159ac00 0x563b51634b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:27:19.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.576+0000 7fcfbc87d700 1 --2- 172.21.15.16:0/796601313 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563b5159a800 0x563b51635080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:27:19.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.576+0000 7fcfbc07c700 1 --2- 172.21.15.16:0/796601313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b5159b000 0x563b51634580 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x563b516fae10 tx=0x563b51704f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57051d9dc5a3c988 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:19.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.576+0000 7fcfb9877700 1 -- 172.21.15.16:0/796601313 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x563b515cf600 con 0x563b5159b000 2023-05-20T01:27:19.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.577+0000 7fcfb9877700 1 -- 172.21.15.16:0/796601313 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x563b515ec480 con 0x563b5159b000 2023-05-20T01:27:19.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.577+0000 7fcfb9877700 1 -- 172.21.15.16:0/796601313 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x563b515cf1e0 con 0x563b5159b000 2023-05-20T01:27:19.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.577+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/796601313 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563b515bf520 con 0x563b5159b000 2023-05-20T01:27:19.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.577+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/796601313 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x563b515bf380 con 0x563b5159b000 2023-05-20T01:27:19.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.577+0000 7fcfbe0ce7c0 4 waiting for MDS map... 2023-05-20T01:27:19.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.577+0000 7fcfb9877700 1 -- 172.21.15.16:0/796601313 <== mon.0 v2:172.21.15.16:3300/0 4 ==== osd_map(119..119 src has 1..119) v4 ==== 6987+0+0 (secure 0 0 0) 0x563b515dcc40 con 0x563b5159b000 2023-05-20T01:27:19.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.577+0000 7fcfb9877700 1 -- 172.21.15.16:0/796601313 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 84) v1 ==== 3126+0+0 (secure 0 0 0) 0x563b52418300 con 0x563b5159b000 2023-05-20T01:27:19.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.577+0000 7fcfbe0ce7c0 4 Got MDS map 84 2023-05-20T01:27:19.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.577+0000 7fcfbe0ce7c0 4 datascan.main: Using metadata pool output 2023-05-20T01:27:19.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.577+0000 7fcfbe0ce7c0 4 datascan.main: connecting to RADOS... 2023-05-20T01:27:19.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.577+0000 7fcfbe0ce7c0 1 Processor -- start 2023-05-20T01:27:19.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.577+0000 7fcfbe0ce7c0 1 -- start start 2023-05-20T01:27:19.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.578+0000 7fcfbe0ce7c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b5159bc00 0x563b51635600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:19.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.578+0000 7fcfbe0ce7c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563b518b6000 0x563b51635b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:19.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.578+0000 7fcfbe0ce7c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563b518b6400 0x563b51636100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:19.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.578+0000 7fcfbe0ce7c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b515d6a00 con 0x563b5159bc00 2023-05-20T01:27:19.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.578+0000 7fcfbe0ce7c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b515d6b40 con 0x563b518b6400 2023-05-20T01:27:19.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.578+0000 7fcfbe0ce7c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x563b515d7400 con 0x563b518b6000 2023-05-20T01:27:19.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.578+0000 7fcfbc87d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b5159bc00 0x563b51635600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:19.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.578+0000 7fcfbc87d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b5159bc00 0x563b51635600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38936/0 (socket says 172.21.15.16:38936) 2023-05-20T01:27:19.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.578+0000 7fcfbc87d700 1 -- 172.21.15.16:0/2131001497 learned_addr learned my addr 172.21.15.16:0/2131001497 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:27:19.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.578+0000 7fcfbc07c700 1 --2- 172.21.15.16:0/2131001497 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563b518b6000 0x563b51635b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:19.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.578+0000 7fcfbc87d700 1 -- 172.21.15.16:0/2131001497 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563b518b6000 msgr2=0x563b51635b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:19.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.578+0000 7fcfbc87d700 1 --2- 172.21.15.16:0/2131001497 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563b518b6000 0x563b51635b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.578+0000 7fcfbc87d700 1 -- 172.21.15.16:0/2131001497 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563b518b6400 msgr2=0x563b51636100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:19.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.578+0000 7fcfbc87d700 1 --2- 172.21.15.16:0/2131001497 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563b518b6400 0x563b51636100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.578+0000 7fcfbc87d700 1 -- 172.21.15.16:0/2131001497 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563b515bfd40 con 0x563b5159bc00 2023-05-20T01:27:19.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.578+0000 7fcfbc87d700 1 --2- 172.21.15.16:0/2131001497 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b5159bc00 0x563b51635600 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x563b518c0090 tx=0x563b51704000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b098f1eb5009959 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:19.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.579+0000 7fcfb7873700 1 -- 172.21.15.16:0/2131001497 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x563b515cf080 con 0x563b5159bc00 2023-05-20T01:27:19.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.579+0000 7fcfb7873700 1 -- 172.21.15.16:0/2131001497 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x563b52424d80 con 0x563b5159bc00 2023-05-20T01:27:19.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.579+0000 7fcfb7873700 1 -- 172.21.15.16:0/2131001497 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x563b515cf600 con 0x563b5159bc00 2023-05-20T01:27:19.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.579+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/2131001497 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b5159bc00 msgr2=0x563b51635600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:19.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.579+0000 7fcfbe0ce7c0 1 --2- 172.21.15.16:0/2131001497 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b5159bc00 0x563b51635600 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x563b518c0090 tx=0x563b51704000 comp rx=0 tx=0).stop 2023-05-20T01:27:19.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.579+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/2131001497 shutdown_connections 2023-05-20T01:27:19.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.579+0000 7fcfbe0ce7c0 1 --2- 172.21.15.16:0/2131001497 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563b518b6400 0x563b51636100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.579+0000 7fcfbe0ce7c0 1 --2- 172.21.15.16:0/2131001497 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b5159bc00 0x563b51635600 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.579+0000 7fcfbe0ce7c0 1 --2- 172.21.15.16:0/2131001497 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563b518b6000 0x563b51635b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.579+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/2131001497 >> 172.21.15.16:0/2131001497 conn(0x563b5159a400 msgr2=0x563b51677800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:27:19.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.579+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/2131001497 shutdown_connections 2023-05-20T01:27:19.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.579+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/2131001497 wait complete. 2023-05-20T01:27:19.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.580+0000 7fcfbe0ce7c0 1 Processor -- start 2023-05-20T01:27:19.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.580+0000 7fcfbe0ce7c0 1 -- start start 2023-05-20T01:27:19.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.580+0000 7fcfbe0ce7c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b5159a400 0x563b51637180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:19.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.580+0000 7fcfbe0ce7c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563b518c8000 0x563b51636c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:19.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.580+0000 7fcfbe0ce7c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563b518c8400 0x563b51636680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:19.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.580+0000 7fcfbe0ce7c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b515d6c80 con 0x563b5159a400 2023-05-20T01:27:19.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.580+0000 7fcfbe0ce7c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b515d6000 con 0x563b518c8000 2023-05-20T01:27:19.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.580+0000 7fcfbe0ce7c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x563b515d7180 con 0x563b518c8400 2023-05-20T01:27:19.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.580+0000 7fcfbc87d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b5159a400 0x563b51637180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:19.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.580+0000 7fcfbc87d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b5159a400 0x563b51637180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38942/0 (socket says 172.21.15.16:38942) 2023-05-20T01:27:19.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.580+0000 7fcfbc87d700 1 -- 172.21.15.16:0/488708207 learned_addr learned my addr 172.21.15.16:0/488708207 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:27:19.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.581+0000 7fcfbc07c700 1 --2- 172.21.15.16:0/488708207 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563b518c8000 0x563b51636c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:19.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.581+0000 7fcfbb87b700 1 --2- 172.21.15.16:0/488708207 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563b518c8400 0x563b51636680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:19.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.581+0000 7fcfbc87d700 1 -- 172.21.15.16:0/488708207 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563b518c8400 msgr2=0x563b51636680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:19.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.581+0000 7fcfbc87d700 1 --2- 172.21.15.16:0/488708207 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563b518c8400 0x563b51636680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.581+0000 7fcfbc87d700 1 -- 172.21.15.16:0/488708207 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563b518c8000 msgr2=0x563b51636c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:19.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.581+0000 7fcfbc87d700 1 --2- 172.21.15.16:0/488708207 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563b518c8000 0x563b51636c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.581+0000 7fcfbc87d700 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563b515bf1e0 con 0x563b5159a400 2023-05-20T01:27:19.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.581+0000 7fcfbc87d700 1 --2- 172.21.15.16:0/488708207 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b5159a400 0x563b51637180 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x563b518c0300 tx=0x563b517041e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:19.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.581+0000 7fcfb6070700 1 -- 172.21.15.16:0/488708207 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x563b515cf1e0 con 0x563b5159a400 2023-05-20T01:27:19.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.581+0000 7fcfb6070700 1 -- 172.21.15.16:0/488708207 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x563b52424c00 con 0x563b5159a400 2023-05-20T01:27:19.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.581+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x563b515bfd40 con 0x563b5159a400 2023-05-20T01:27:19.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.581+0000 7fcfb6070700 1 -- 172.21.15.16:0/488708207 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x563b515cedc0 con 0x563b5159a400 2023-05-20T01:27:19.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.581+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563b518d6000 con 0x563b5159a400 2023-05-20T01:27:19.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.581+0000 7fcfbe0ce7c0 4 datascan.init: resolving metadata pool 15 2023-05-20T01:27:19.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.582+0000 7fcfb6070700 1 -- 172.21.15.16:0/488708207 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x563b515fc840 con 0x563b5159a400 2023-05-20T01:27:19.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.583+0000 7fcfb6070700 1 --2- 172.21.15.16:0/488708207 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x563b518d2400 0x563b51637700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:19.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.583+0000 7fcfb6070700 1 -- 172.21.15.16:0/488708207 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6987+0+0 (secure 0 0 0) 0x563b515dca80 con 0x563b5159a400 2023-05-20T01:27:19.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.583+0000 7fcfbc07c700 1 --2- 172.21.15.16:0/488708207 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x563b518d2400 0x563b51637700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:19.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.583+0000 7fcfbe0ce7c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-05-20T01:27:19.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.583+0000 7fcfbe0ce7c0 4 datascan.main: data pool 'cephfs_data' has ID 16 2023-05-20T01:27:19.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.583+0000 7fcfbe0ce7c0 4 datascan.main: opening data pool 'cephfs_data' 2023-05-20T01:27:19.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.583+0000 7fcfbe0ce7c0 4 datascan.main: data pool 'cephfs_data_extra' has ID 17 2023-05-20T01:27:19.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.583+0000 7fcfbe0ce7c0 4 datascan.main: opening data pool 'cephfs_data_extra' 2023-05-20T01:27:19.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.583+0000 7fcfbe0ce7c0 1 --2- 172.21.15.16:0/488708207 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x563b518c8800 0x563b51636100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:19.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.583+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:1 16.0 16:00000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b52420700 con 0x563b518c8800 2023-05-20T01:27:19.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.583+0000 7fcfbb87b700 1 --2- 172.21.15.16:0/488708207 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x563b518c8800 0x563b51636100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:19.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.583+0000 7fcfbc07c700 1 --2- 172.21.15.16:0/488708207 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x563b518d2400 0x563b51637700 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x563b516fb350 tx=0x563b52333440 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:19.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.583+0000 7fcfbb87b700 1 --2- 172.21.15.16:0/488708207 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x563b518c8800 0x563b51636100 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:19.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.584+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 <== osd.0 v2:172.21.15.16:6824/3923634179 1 ==== osd_op_reply(1 [pgnls start_epoch 119 out=49b] v115'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563b518ab440 con 0x563b518c8800 2023-05-20T01:27:19.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.584+0000 7fcfb7072700 1 --2- 172.21.15.16:0/488708207 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x563b518d2800 0x563b519de000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:19.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.584+0000 7fcfb7072700 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:2 16.20 16:04000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b52421180 con 0x563b518d2800 2023-05-20T01:27:19.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.584+0000 7fcfbc87d700 1 --2- 172.21.15.16:0/488708207 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x563b518d2800 0x563b519de000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:19.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.584+0000 7fcfbc87d700 1 --2- 172.21.15.16:0/488708207 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x563b518d2800 0x563b519de000 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:19.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.584+0000 7fcfbc87d700 1 -- 172.21.15.16:0/488708207 <== osd.2 v2:172.21.15.16:6816/293187030 1 ==== osd_op_reply(2 [pgnls start_epoch 119 out=49b] v115'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563b518ab680 con 0x563b518d2800 2023-05-20T01:27:19.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.584+0000 7fcfb7873700 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:3 16.10 16:08000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b52421500 con 0x563b518d2800 2023-05-20T01:27:19.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.585+0000 7fcfbc87d700 1 -- 172.21.15.16:0/488708207 <== osd.2 v2:172.21.15.16:6816/293187030 2 ==== osd_op_reply(3 [pgnls start_epoch 119 out=49b] v115'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563b518ab680 con 0x563b518d2800 2023-05-20T01:27:19.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.585+0000 7fcfb7072700 1 --2- 172.21.15.16:0/488708207 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x563b518d2c00 0x563b519de580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:19.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.585+0000 7fcfb7072700 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:4 16.30 16:0c000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b52421180 con 0x563b518d2c00 2023-05-20T01:27:19.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.585+0000 7fcfbc07c700 1 --2- 172.21.15.16:0/488708207 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x563b518d2c00 0x563b519de580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:19.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.585+0000 7fcfbc07c700 1 --2- 172.21.15.16:0/488708207 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x563b518d2c00 0x563b519de580 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:19.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.586+0000 7fcfbc07c700 1 -- 172.21.15.16:0/488708207 <== osd.7 v2:172.21.15.132:6800/466001149 1 ==== osd_op_reply(4 [pgnls start_epoch 119 out=49b] v115'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563b518ab8c0 con 0x563b518d2c00 2023-05-20T01:27:19.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.586+0000 7fcfb7873700 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:5 16.8 16:10000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b52421880 con 0x563b518c8800 2023-05-20T01:27:19.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.586+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 <== osd.0 v2:172.21.15.16:6824/3923634179 2 ==== osd_op_reply(5 [pgnls start_epoch 119 out=49b] v115'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563b518ab440 con 0x563b518c8800 2023-05-20T01:27:19.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.586+0000 7fcfb7072700 1 --2- 172.21.15.16:0/488708207 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x563b518d3000 0x563b519deb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:19.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.586+0000 7fcfb7072700 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:6 16.28 16:14000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b52421500 con 0x563b518d3000 2023-05-20T01:27:19.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.586+0000 7fcfbb87b700 1 --2- 172.21.15.16:0/488708207 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x563b518d3000 0x563b519deb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:19.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.586+0000 7fcfbb87b700 1 --2- 172.21.15.16:0/488708207 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x563b518d3000 0x563b519deb00 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:19.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.587+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 <== osd.1 v2:172.21.15.16:6808/3860264220 1 ==== osd_op_reply(6 [pgnls start_epoch 119 out=49b] v115'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563b518ab440 con 0x563b518d3000 2023-05-20T01:27:19.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.587+0000 7fcfb7873700 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:7 16.18 16:18000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b52421c00 con 0x563b518c8800 2023-05-20T01:27:19.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.587+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 <== osd.0 v2:172.21.15.16:6824/3923634179 3 ==== osd_op_reply(7 [pgnls start_epoch 119 out=49b] v115'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563b518ab440 con 0x563b518c8800 2023-05-20T01:27:19.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.587+0000 7fcfb7072700 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:8 16.38 16:1c000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b52420700 con 0x563b518d2800 2023-05-20T01:27:19.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.587+0000 7fcfbc87d700 1 -- 172.21.15.16:0/488708207 <== osd.2 v2:172.21.15.16:6816/293187030 3 ==== osd_op_reply(8 [pgnls start_epoch 119 out=49b] v115'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563b518ab680 con 0x563b518d2800 2023-05-20T01:27:19.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.587+0000 7fcfb7873700 1 --2- 172.21.15.16:0/488708207 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x563b518d3400 0x563b519df080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:19.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.587+0000 7fcfb7873700 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:9 16.4 16:20000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b519f8000 con 0x563b518d3400 2023-05-20T01:27:19.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.587+0000 7fcfbc87d700 1 --2- 172.21.15.16:0/488708207 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x563b518d3400 0x563b519df080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:19.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.588+0000 7fcfbc87d700 1 --2- 172.21.15.16:0/488708207 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x563b518d3400 0x563b519df080 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:19.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.588+0000 7fcfbc87d700 1 -- 172.21.15.16:0/488708207 <== osd.3 v2:172.21.15.16:6800/3669781890 1 ==== osd_op_reply(9 [pgnls start_epoch 119 out=49b] v115'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563b518ab680 con 0x563b518d3400 2023-05-20T01:27:19.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.588+0000 7fcfb7072700 1 --2- 172.21.15.16:0/488708207 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x563b518d3800 0x563b519df600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:19.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.588+0000 7fcfb7072700 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:10 16.24 16:24000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b52421880 con 0x563b518d3800 2023-05-20T01:27:19.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.588+0000 7fcfbc07c700 1 --2- 172.21.15.16:0/488708207 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x563b518d3800 0x563b519df600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:19.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.589+0000 7fcfbc07c700 1 --2- 172.21.15.16:0/488708207 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x563b518d3800 0x563b519df600 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:19.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.589+0000 7fcfbc07c700 1 -- 172.21.15.16:0/488708207 <== osd.6 v2:172.21.15.132:6801/727581123 1 ==== osd_op_reply(10 [pgnls start_epoch 119 out=49b] v115'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563b518ab8c0 con 0x563b518d3800 2023-05-20T01:27:19.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.589+0000 7fcfb7873700 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:11 16.14 16:28000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b519f8380 con 0x563b518c8800 2023-05-20T01:27:19.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.589+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 <== osd.0 v2:172.21.15.16:6824/3923634179 4 ==== osd_op_reply(11 [pgnls start_epoch 119 out=49b] v115'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563b518ab440 con 0x563b518c8800 2023-05-20T01:27:19.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.589+0000 7fcfb7072700 1 --2- 172.21.15.16:0/488708207 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x563b518d3c00 0x563b519dfb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:19.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.589+0000 7fcfb7072700 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:12 16.34 16:2c000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b519f8380 con 0x563b518d3c00 2023-05-20T01:27:19.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.589+0000 7fcfbb87b700 1 --2- 172.21.15.16:0/488708207 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x563b518d3c00 0x563b519dfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:19.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.590+0000 7fcfbb87b700 1 --2- 172.21.15.16:0/488708207 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x563b518d3c00 0x563b519dfb80 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:19.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.590+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 <== osd.4 v2:172.21.15.132:6817/506317223 1 ==== osd_op_reply(12 [pgnls start_epoch 119 out=49b] v115'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563b518ab440 con 0x563b518d3c00 2023-05-20T01:27:19.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.590+0000 7fcfb7873700 1 --2- 172.21.15.16:0/488708207 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x563b51a06000 0x563b519e0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:19.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.590+0000 7fcfb7873700 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:13 16.c 16:30000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b519f8700 con 0x563b51a06000 2023-05-20T01:27:19.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.590+0000 7fcfbc87d700 1 --2- 172.21.15.16:0/488708207 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x563b51a06000 0x563b519e0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:19.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.591+0000 7fcfbc87d700 1 --2- 172.21.15.16:0/488708207 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x563b51a06000 0x563b519e0680 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:19.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.591+0000 7fcfbc87d700 1 -- 172.21.15.16:0/488708207 <== osd.5 v2:172.21.15.132:6816/1844689995 1 ==== osd_op_reply(13 [pgnls start_epoch 119 out=49b] v115'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563b518ab680 con 0x563b51a06000 2023-05-20T01:27:19.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.591+0000 7fcfb7072700 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:14 16.2c 16:34000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b519f8a80 con 0x563b518d3c00 2023-05-20T01:27:19.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.591+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 <== osd.4 v2:172.21.15.132:6817/506317223 2 ==== osd_op_reply(14 [pgnls start_epoch 119 out=49b] v115'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563b518ab440 con 0x563b518d3c00 2023-05-20T01:27:19.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.591+0000 7fcfb7873700 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:15 16.1c 16:38000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b519f8000 con 0x563b518d2800 2023-05-20T01:27:19.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.592+0000 7fcfbc87d700 1 -- 172.21.15.16:0/488708207 <== osd.2 v2:172.21.15.16:6816/293187030 4 ==== osd_op_reply(15 [pgnls start_epoch 119 out=49b] v115'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563b518ab680 con 0x563b518d2800 2023-05-20T01:27:19.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.592+0000 7fcfb7072700 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:16 16.3c 16:3c000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b52421500 con 0x563b518d3400 2023-05-20T01:27:19.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.592+0000 7fcfbc87d700 1 -- 172.21.15.16:0/488708207 <== osd.3 v2:172.21.15.16:6800/3669781890 2 ==== osd_op_reply(16 [pgnls start_epoch 119 out=49b] v115'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563b518ab680 con 0x563b518d3400 2023-05-20T01:27:19.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.592+0000 7fcfb7873700 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:17 16.2 16:40000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b519f8700 con 0x563b518d3c00 2023-05-20T01:27:19.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.592+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 <== osd.4 v2:172.21.15.132:6817/506317223 3 ==== osd_op_reply(17 [pgnls start_epoch 119 out=49b] v115'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563b518ab440 con 0x563b518d3c00 2023-05-20T01:27:19.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.592+0000 7fcfb7072700 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:18 16.22 16:44000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b52421c00 con 0x563b518d3c00 2023-05-20T01:27:19.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.593+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 <== osd.4 v2:172.21.15.132:6817/506317223 4 ==== osd_op_reply(18 [pgnls start_epoch 119 out=49b] v115'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563b518ab440 con 0x563b518d3c00 2023-05-20T01:27:19.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.593+0000 7fcfb7873700 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:19 16.12 16:48000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b519f8e00 con 0x563b518d3800 2023-05-20T01:27:19.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.593+0000 7fcfbc07c700 1 -- 172.21.15.16:0/488708207 <== osd.6 v2:172.21.15.132:6801/727581123 2 ==== osd_op_reply(19 [pgnls start_epoch 119 out=49b] v115'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563b518ab8c0 con 0x563b518d3800 2023-05-20T01:27:19.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.593+0000 7fcfb7072700 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:20 16.32 16:4c000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b519f8e00 con 0x563b518d2c00 2023-05-20T01:27:19.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.594+0000 7fcfbc07c700 1 -- 172.21.15.16:0/488708207 <== osd.7 v2:172.21.15.132:6800/466001149 2 ==== osd_op_reply(20 [pgnls start_epoch 119 out=49b] v115'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563b518ab8c0 con 0x563b518d2c00 2023-05-20T01:27:19.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.594+0000 7fcfb7873700 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:21 16.a 16:50000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b52420700 con 0x563b518d2800 2023-05-20T01:27:19.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.594+0000 7fcfbc87d700 1 -- 172.21.15.16:0/488708207 <== osd.2 v2:172.21.15.16:6816/293187030 5 ==== osd_op_reply(21 [pgnls start_epoch 119 out=49b] v115'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563b518ab680 con 0x563b518d2800 2023-05-20T01:27:19.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.594+0000 7fcfb7072700 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:22 16.2a 16:54000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b519f8380 con 0x563b518d3000 2023-05-20T01:27:19.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.594+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 <== osd.1 v2:172.21.15.16:6808/3860264220 2 ==== osd_op_reply(22 [pgnls start_epoch 119 out=81b] v115'10 uv1 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x563b518ab440 con 0x563b518d3000 2023-05-20T01:27:19.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.594+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:23 16.2a 16:57fe7535:::100000001f6.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b52420a80 con 0x563b518d3000 2023-05-20T01:27:19.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.595+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 <== osd.1 v2:172.21.15.16:6808/3860264220 3 ==== osd_op_reply(23 100000001f6.00000000 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b518ab440 con 0x563b518d3000 2023-05-20T01:27:19.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.595+0000 7fcfbe0ce7c0 10 datascan.operator(): handling object 1099511628278.0 2023-05-20T01:27:19.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.595+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:24 16.2a 16:57fe7535:::100000001f6.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b52420e00 con 0x563b518d3000 2023-05-20T01:27:19.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.596+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 <== osd.1 v2:172.21.15.16:6808/3860264220 4 ==== osd_op_reply(24 100000001f6.00000000 [call] v119'11 uv11 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563b518ab440 con 0x563b518d3000 2023-05-20T01:27:19.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.596+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:25 16.1a 16:58000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b519f8a80 con 0x563b518d3400 2023-05-20T01:27:19.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.596+0000 7fcfbc87d700 1 -- 172.21.15.16:0/488708207 <== osd.3 v2:172.21.15.16:6800/3669781890 3 ==== osd_op_reply(25 [pgnls start_epoch 119 out=49b] v115'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563b518ab680 con 0x563b518d3400 2023-05-20T01:27:19.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.596+0000 7fcfb7072700 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:26 16.3a 16:5c000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b52421180 con 0x563b518d3800 2023-05-20T01:27:19.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.597+0000 7fcfbc07c700 1 -- 172.21.15.16:0/488708207 <== osd.6 v2:172.21.15.132:6801/727581123 3 ==== osd_op_reply(26 [pgnls start_epoch 119 out=49b] v115'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563b518ab8c0 con 0x563b518d3800 2023-05-20T01:27:19.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.597+0000 7fcfb7873700 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:27 16.6 16:60000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b519f8000 con 0x563b518d3800 2023-05-20T01:27:19.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.597+0000 7fcfbc07c700 1 -- 172.21.15.16:0/488708207 <== osd.6 v2:172.21.15.132:6801/727581123 4 ==== osd_op_reply(27 [pgnls start_epoch 119 out=49b] v115'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563b518ab8c0 con 0x563b518d3800 2023-05-20T01:27:19.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.597+0000 7fcfb7072700 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:28 16.26 16:64000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b52421880 con 0x563b518d3c00 2023-05-20T01:27:19.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.598+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 <== osd.4 v2:172.21.15.132:6817/506317223 5 ==== osd_op_reply(28 [pgnls start_epoch 119 out=49b] v115'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563b518ab440 con 0x563b518d3c00 2023-05-20T01:27:19.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.598+0000 7fcfb7873700 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:29 16.16 16:68000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b519f8a80 con 0x563b518d3400 2023-05-20T01:27:19.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.598+0000 7fcfbc87d700 1 -- 172.21.15.16:0/488708207 <== osd.3 v2:172.21.15.16:6800/3669781890 4 ==== osd_op_reply(29 [pgnls start_epoch 119 out=49b] v115'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563b518ab680 con 0x563b518d3400 2023-05-20T01:27:19.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.598+0000 7fcfb7072700 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:30 16.36 16:6c000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b52421c00 con 0x563b518d3000 2023-05-20T01:27:19.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.598+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 <== osd.1 v2:172.21.15.16:6808/3860264220 5 ==== osd_op_reply(30 [pgnls start_epoch 119 out=101b] v115'17 uv2 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x563b518ab440 con 0x563b518d3000 2023-05-20T01:27:19.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.598+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:31 16.36 16:6ee5e2c4:::100000001f9.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b519f8700 con 0x563b518d3000 2023-05-20T01:27:19.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.598+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 <== osd.1 v2:172.21.15.16:6808/3860264220 6 ==== osd_op_reply(31 100000001f9.00000000 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b518ab440 con 0x563b518d3000 2023-05-20T01:27:19.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.598+0000 7fcfbe0ce7c0 10 datascan.operator(): handling object 1099511628281.0 2023-05-20T01:27:19.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.598+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:32 16.36 16:6ee5e2c4:::100000001f9.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b519f9180 con 0x563b518d3000 2023-05-20T01:27:19.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.600+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 <== osd.1 v2:172.21.15.16:6808/3860264220 7 ==== osd_op_reply(32 100000001f9.00000000 [call] v119'18 uv18 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563b518ab440 con 0x563b518d3000 2023-05-20T01:27:19.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.600+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:33 16.36 16:6fe38dc2::::head [pgnls start_epoch 119 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b519f9500 con 0x563b518d3000 2023-05-20T01:27:19.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.600+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 <== osd.1 v2:172.21.15.16:6808/3860264220 8 ==== osd_op_reply(33 [pgnls start_epoch 119 out=81b] v119'18 uv18 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x563b518ab440 con 0x563b518d3000 2023-05-20T01:27:19.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.600+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:34 16.36 16:6fe38dc2:::100000001f7.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b519f8380 con 0x563b518d3000 2023-05-20T01:27:19.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.600+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 <== osd.1 v2:172.21.15.16:6808/3860264220 9 ==== osd_op_reply(34 100000001f7.00000000 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b518ab440 con 0x563b518d3000 2023-05-20T01:27:19.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.600+0000 7fcfbe0ce7c0 10 datascan.operator(): handling object 1099511628279.0 2023-05-20T01:27:19.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.600+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:35 16.36 16:6fe38dc2:::100000001f7.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b52420a80 con 0x563b518d3000 2023-05-20T01:27:19.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.601+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 <== osd.1 v2:172.21.15.16:6808/3860264220 10 ==== osd_op_reply(35 100000001f7.00000000 [call] v119'19 uv19 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563b518ab440 con 0x563b518d3000 2023-05-20T01:27:19.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.601+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:36 16.e 16:70000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b52420e00 con 0x563b518d2800 2023-05-20T01:27:19.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.602+0000 7fcfbc87d700 1 -- 172.21.15.16:0/488708207 <== osd.2 v2:172.21.15.16:6816/293187030 6 ==== osd_op_reply(36 [pgnls start_epoch 119 out=49b] v115'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563b518ab680 con 0x563b518d2800 2023-05-20T01:27:19.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.602+0000 7fcfb7873700 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:37 16.2e 16:74000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b519f9880 con 0x563b518d3800 2023-05-20T01:27:19.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.602+0000 7fcfbc07c700 1 -- 172.21.15.16:0/488708207 <== osd.6 v2:172.21.15.132:6801/727581123 5 ==== osd_op_reply(37 [pgnls start_epoch 119 out=49b] v115'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563b518ab8c0 con 0x563b518d3800 2023-05-20T01:27:19.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.602+0000 7fcfb7072700 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:38 16.1e 16:78000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b519f8e00 con 0x563b518d2c00 2023-05-20T01:27:19.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.602+0000 7fcfbc07c700 1 -- 172.21.15.16:0/488708207 <== osd.7 v2:172.21.15.132:6800/466001149 3 ==== osd_op_reply(38 [pgnls start_epoch 119 out=49b] v115'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563b518ab8c0 con 0x563b518d2c00 2023-05-20T01:27:19.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.602+0000 7fcfb7873700 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:39 16.3e 16:7c000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b52421500 con 0x563b518d3800 2023-05-20T01:27:19.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.603+0000 7fcfbc07c700 1 -- 172.21.15.16:0/488708207 <== osd.6 v2:172.21.15.132:6801/727581123 6 ==== osd_op_reply(39 [pgnls start_epoch 119 out=49b] v115'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563b518ab8c0 con 0x563b518d3800 2023-05-20T01:27:19.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.603+0000 7fcfb7072700 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:40 16.1 16:80000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b519f8000 con 0x563b518d2800 2023-05-20T01:27:19.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.603+0000 7fcfbc87d700 1 -- 172.21.15.16:0/488708207 <== osd.2 v2:172.21.15.16:6816/293187030 7 ==== osd_op_reply(40 [pgnls start_epoch 119 out=49b] v115'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563b518ab680 con 0x563b518d2800 2023-05-20T01:27:19.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.603+0000 7fcfb7873700 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:41 16.21 16:84000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b52420700 con 0x563b518d3800 2023-05-20T01:27:19.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.603+0000 7fcfbc07c700 1 -- 172.21.15.16:0/488708207 <== osd.6 v2:172.21.15.132:6801/727581123 7 ==== osd_op_reply(41 [pgnls start_epoch 119 out=49b] v115'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563b518ab8c0 con 0x563b518d3800 2023-05-20T01:27:19.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.603+0000 7fcfb7072700 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:42 16.11 16:88000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b519f9880 con 0x563b51a06000 2023-05-20T01:27:19.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.604+0000 7fcfbc87d700 1 -- 172.21.15.16:0/488708207 <== osd.5 v2:172.21.15.132:6816/1844689995 2 ==== osd_op_reply(42 [pgnls start_epoch 119 out=49b] v115'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563b518ab680 con 0x563b51a06000 2023-05-20T01:27:19.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.604+0000 7fcfb7873700 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:43 16.31 16:8c000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b51a00000 con 0x563b518d2c00 2023-05-20T01:27:19.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.604+0000 7fcfbc07c700 1 -- 172.21.15.16:0/488708207 <== osd.7 v2:172.21.15.132:6800/466001149 4 ==== osd_op_reply(43 [pgnls start_epoch 119 out=49b] v115'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563b518ab8c0 con 0x563b518d2c00 2023-05-20T01:27:19.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.604+0000 7fcfb7072700 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:44 16.9 16:90000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b519f9c00 con 0x563b518d3000 2023-05-20T01:27:19.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.604+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 <== osd.1 v2:172.21.15.16:6808/3860264220 11 ==== osd_op_reply(44 [pgnls start_epoch 119 out=49b] v115'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563b518ab440 con 0x563b518d3000 2023-05-20T01:27:19.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.604+0000 7fcfb7873700 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:45 16.29 16:94000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b51a00380 con 0x563b518c8800 2023-05-20T01:27:19.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.605+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 <== osd.0 v2:172.21.15.16:6824/3923634179 5 ==== osd_op_reply(45 [pgnls start_epoch 119 out=49b] v115'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563b518ab440 con 0x563b518c8800 2023-05-20T01:27:19.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.605+0000 7fcfb7072700 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:46 16.19 16:98000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b52421180 con 0x563b518d3c00 2023-05-20T01:27:19.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.605+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 <== osd.4 v2:172.21.15.132:6817/506317223 6 ==== osd_op_reply(46 [pgnls start_epoch 119 out=81b] v115'11 uv1 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x563b518ab440 con 0x563b518d3c00 2023-05-20T01:27:19.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.605+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:47 16.19 16:997b7129:::100000001f8.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b52421880 con 0x563b518d3c00 2023-05-20T01:27:19.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.605+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 <== osd.4 v2:172.21.15.132:6817/506317223 7 ==== osd_op_reply(47 100000001f8.00000000 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b518ab440 con 0x563b518d3c00 2023-05-20T01:27:19.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.605+0000 7fcfbe0ce7c0 10 datascan.operator(): handling object 1099511628280.0 2023-05-20T01:27:19.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.605+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:48 16.19 16:997b7129:::100000001f8.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b52421c00 con 0x563b518d3c00 2023-05-20T01:27:19.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.607+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 <== osd.4 v2:172.21.15.132:6817/506317223 8 ==== osd_op_reply(48 100000001f8.00000000 [call] v119'12 uv12 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563b518ab440 con 0x563b518d3c00 2023-05-20T01:27:19.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.607+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:49 16.39 16:9c000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b519f8700 con 0x563b518d3800 2023-05-20T01:27:19.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.607+0000 7fcfbc07c700 1 -- 172.21.15.16:0/488708207 <== osd.6 v2:172.21.15.132:6801/727581123 8 ==== osd_op_reply(49 [pgnls start_epoch 119 out=49b] v115'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563b518ab8c0 con 0x563b518d3800 2023-05-20T01:27:19.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.607+0000 7fcfb7072700 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:50 16.5 16:a0000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b51a00000 con 0x563b518c8800 2023-05-20T01:27:19.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.608+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 <== osd.0 v2:172.21.15.16:6824/3923634179 6 ==== osd_op_reply(50 [pgnls start_epoch 119 out=49b] v115'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563b518ab440 con 0x563b518c8800 2023-05-20T01:27:19.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.608+0000 7fcfb7873700 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:51 16.25 16:a4000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b51a00700 con 0x563b51a06000 2023-05-20T01:27:19.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.608+0000 7fcfbc87d700 1 -- 172.21.15.16:0/488708207 <== osd.5 v2:172.21.15.132:6816/1844689995 3 ==== osd_op_reply(51 [pgnls start_epoch 119 out=49b] v115'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563b518ab680 con 0x563b51a06000 2023-05-20T01:27:19.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.608+0000 7fcfb7072700 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:52 16.15 16:a8000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b51a01500 con 0x563b518c8800 2023-05-20T01:27:19.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.608+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 <== osd.0 v2:172.21.15.16:6824/3923634179 7 ==== osd_op_reply(52 [pgnls start_epoch 119 out=49b] v115'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563b518ab440 con 0x563b518c8800 2023-05-20T01:27:19.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.608+0000 7fcfb7873700 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:53 16.35 16:ac000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b51a00a80 con 0x563b51a06000 2023-05-20T01:27:19.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.609+0000 7fcfbc87d700 1 -- 172.21.15.16:0/488708207 <== osd.5 v2:172.21.15.132:6816/1844689995 4 ==== osd_op_reply(53 [pgnls start_epoch 119 out=49b] v115'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563b518ab680 con 0x563b51a06000 2023-05-20T01:27:19.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.609+0000 7fcfb7072700 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:54 16.d 16:b0000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b51a01880 con 0x563b518d3800 2023-05-20T01:27:19.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.609+0000 7fcfbc07c700 1 -- 172.21.15.16:0/488708207 <== osd.6 v2:172.21.15.132:6801/727581123 9 ==== osd_op_reply(54 [pgnls start_epoch 119 out=49b] v115'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563b518ab8c0 con 0x563b518d3800 2023-05-20T01:27:19.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.609+0000 7fcfb7873700 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:55 16.2d 16:b4000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b51a00e00 con 0x563b518c8800 2023-05-20T01:27:19.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.609+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 <== osd.0 v2:172.21.15.16:6824/3923634179 8 ==== osd_op_reply(55 [pgnls start_epoch 119 out=49b] v115'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563b518ab440 con 0x563b518c8800 2023-05-20T01:27:19.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.609+0000 7fcfb7072700 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:56 16.1d 16:b8000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b519f8e00 con 0x563b518d2800 2023-05-20T01:27:19.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.610+0000 7fcfbc87d700 1 -- 172.21.15.16:0/488708207 <== osd.2 v2:172.21.15.16:6816/293187030 8 ==== osd_op_reply(56 [pgnls start_epoch 119 out=49b] v115'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563b518ab680 con 0x563b518d2800 2023-05-20T01:27:19.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.610+0000 7fcfb7873700 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:57 16.3d 16:bc000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b51a01180 con 0x563b518c8800 2023-05-20T01:27:19.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.610+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 <== osd.0 v2:172.21.15.16:6824/3923634179 9 ==== osd_op_reply(57 [pgnls start_epoch 119 out=49b] v115'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563b518ab440 con 0x563b518c8800 2023-05-20T01:27:19.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.610+0000 7fcfb7072700 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:58 16.3 16:c0000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b519f8700 con 0x563b51a06000 2023-05-20T01:27:19.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.610+0000 7fcfbc87d700 1 -- 172.21.15.16:0/488708207 <== osd.5 v2:172.21.15.132:6816/1844689995 5 ==== osd_op_reply(58 [pgnls start_epoch 119 out=49b] v115'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563b518ab680 con 0x563b51a06000 2023-05-20T01:27:19.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.610+0000 7fcfb7873700 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:59 16.23 16:c4000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b519f8a80 con 0x563b518d2800 2023-05-20T01:27:19.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.611+0000 7fcfbc87d700 1 -- 172.21.15.16:0/488708207 <== osd.2 v2:172.21.15.16:6816/293187030 9 ==== osd_op_reply(59 [pgnls start_epoch 119 out=49b] v115'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563b518ab680 con 0x563b518d2800 2023-05-20T01:27:19.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.611+0000 7fcfb7072700 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:60 16.13 16:c8000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b52421500 con 0x563b518d3000 2023-05-20T01:27:19.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.611+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 <== osd.1 v2:172.21.15.16:6808/3860264220 12 ==== osd_op_reply(60 [pgnls start_epoch 119 out=81b] v115'16 uv1 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x563b518ab440 con 0x563b518d3000 2023-05-20T01:27:19.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.611+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:61 16.13 16:ca8fdbba:::100000001fb.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b519f9180 con 0x563b518d3000 2023-05-20T01:27:19.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.611+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 <== osd.1 v2:172.21.15.16:6808/3860264220 13 ==== osd_op_reply(61 100000001fb.00000000 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b518ab440 con 0x563b518d3000 2023-05-20T01:27:19.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.611+0000 7fcfbe0ce7c0 10 datascan.operator(): handling object 1099511628283.0 2023-05-20T01:27:19.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.611+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:62 16.13 16:ca8fdbba:::100000001fb.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b519f9500 con 0x563b518d3000 2023-05-20T01:27:19.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.613+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 <== osd.1 v2:172.21.15.16:6808/3860264220 14 ==== osd_op_reply(62 100000001fb.00000000 [call] v119'17 uv17 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563b518ab440 con 0x563b518d3000 2023-05-20T01:27:19.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.613+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:63 16.33 16:cc000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b519f8380 con 0x563b518d3400 2023-05-20T01:27:19.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.613+0000 7fcfbc87d700 1 -- 172.21.15.16:0/488708207 <== osd.3 v2:172.21.15.16:6800/3669781890 5 ==== osd_op_reply(63 [pgnls start_epoch 119 out=49b] v115'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563b518ab680 con 0x563b518d3400 2023-05-20T01:27:19.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.613+0000 7fcfb7072700 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:64 16.b 16:d0000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b52420700 con 0x563b518d3000 2023-05-20T01:27:19.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.613+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 <== osd.1 v2:172.21.15.16:6808/3860264220 15 ==== osd_op_reply(64 [pgnls start_epoch 119 out=49b] v115'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563b518ab440 con 0x563b518d3000 2023-05-20T01:27:19.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.613+0000 7fcfb7873700 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:65 16.2b 16:d4000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b519f8000 con 0x563b518d2800 2023-05-20T01:27:19.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.613+0000 7fcfbc87d700 1 -- 172.21.15.16:0/488708207 <== osd.2 v2:172.21.15.16:6816/293187030 10 ==== osd_op_reply(65 [pgnls start_epoch 119 out=49b] v115'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563b518ab680 con 0x563b518d2800 2023-05-20T01:27:19.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.614+0000 7fcfb7072700 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:66 16.1b 16:d8000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b51a00a80 con 0x563b51a06000 2023-05-20T01:27:19.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.614+0000 7fcfbc87d700 1 -- 172.21.15.16:0/488708207 <== osd.5 v2:172.21.15.132:6816/1844689995 6 ==== osd_op_reply(66 [pgnls start_epoch 119 out=49b] v115'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563b518ab680 con 0x563b51a06000 2023-05-20T01:27:19.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.614+0000 7fcfb7873700 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:67 16.3b 16:dc000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b519f9880 con 0x563b518d3000 2023-05-20T01:27:19.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.614+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 <== osd.1 v2:172.21.15.16:6808/3860264220 16 ==== osd_op_reply(67 [pgnls start_epoch 119 out=49b] v115'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563b518ab440 con 0x563b518d3000 2023-05-20T01:27:19.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.614+0000 7fcfb7072700 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:68 16.7 16:e0000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b51a00380 con 0x563b518d3400 2023-05-20T01:27:19.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.614+0000 7fcfbc87d700 1 -- 172.21.15.16:0/488708207 <== osd.3 v2:172.21.15.16:6800/3669781890 6 ==== osd_op_reply(68 [pgnls start_epoch 119 out=81b] v115'6 uv1 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x563b518ab680 con 0x563b518d3400 2023-05-20T01:27:19.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.614+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:69 16.7 16:e2e07930:::10000000001.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b519f9c00 con 0x563b518d3400 2023-05-20T01:27:19.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.615+0000 7fcfbc87d700 1 -- 172.21.15.16:0/488708207 <== osd.3 v2:172.21.15.16:6800/3669781890 7 ==== osd_op_reply(69 10000000001.00000000 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b518ab680 con 0x563b518d3400 2023-05-20T01:27:19.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.615+0000 7fcfbe0ce7c0 10 datascan.operator(): handling object 1099511627777.0 2023-05-20T01:27:19.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.615+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:70 16.7 16:e2e07930:::10000000001.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b52420a80 con 0x563b518d3400 2023-05-20T01:27:19.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.616+0000 7fcfbc87d700 1 -- 172.21.15.16:0/488708207 <== osd.3 v2:172.21.15.16:6800/3669781890 8 ==== osd_op_reply(70 10000000001.00000000 [call] v119'7 uv7 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563b518ab680 con 0x563b518d3400 2023-05-20T01:27:19.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.616+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:71 16.27 16:e4000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b51a01500 con 0x563b518d3c00 2023-05-20T01:27:19.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.616+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 <== osd.4 v2:172.21.15.132:6817/506317223 9 ==== osd_op_reply(71 [pgnls start_epoch 119 out=49b] v115'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563b518ab440 con 0x563b518d3c00 2023-05-20T01:27:19.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.616+0000 7fcfb7072700 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:72 16.17 16:e8000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b51a00000 con 0x563b518d2800 2023-05-20T01:27:19.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.617+0000 7fcfbc87d700 1 -- 172.21.15.16:0/488708207 <== osd.2 v2:172.21.15.16:6816/293187030 11 ==== osd_op_reply(72 [pgnls start_epoch 119 out=49b] v115'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563b518ab680 con 0x563b518d2800 2023-05-20T01:27:19.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.617+0000 7fcfb7873700 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:73 16.37 16:ec000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b52421180 con 0x563b518d3800 2023-05-20T01:27:19.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.617+0000 7fcfbc07c700 1 -- 172.21.15.16:0/488708207 <== osd.6 v2:172.21.15.132:6801/727581123 10 ==== osd_op_reply(73 [pgnls start_epoch 119 out=49b] v115'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563b518ab8c0 con 0x563b518d3800 2023-05-20T01:27:19.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.617+0000 7fcfb7072700 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:74 16.f 16:f0000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b519f8e00 con 0x563b518c8800 2023-05-20T01:27:19.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.617+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 <== osd.0 v2:172.21.15.16:6824/3923634179 10 ==== osd_op_reply(74 [pgnls start_epoch 119 out=49b] v115'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563b518ab440 con 0x563b518c8800 2023-05-20T01:27:19.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.617+0000 7fcfb7873700 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:75 16.2f 16:f4000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b52421880 con 0x563b518c8800 2023-05-20T01:27:19.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.618+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 <== osd.0 v2:172.21.15.16:6824/3923634179 11 ==== osd_op_reply(75 [pgnls start_epoch 119 out=49b] v115'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563b518ab440 con 0x563b518c8800 2023-05-20T01:27:19.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.618+0000 7fcfb7072700 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:76 16.1f 16:f8000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b519f8700 con 0x563b51a06000 2023-05-20T01:27:19.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.618+0000 7fcfbc87d700 1 -- 172.21.15.16:0/488708207 <== osd.5 v2:172.21.15.132:6816/1844689995 7 ==== osd_op_reply(76 [pgnls start_epoch 119 out=49b] v115'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563b518ab680 con 0x563b51a06000 2023-05-20T01:27:19.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.618+0000 7fcfb7873700 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:77 16.3f 16:fc000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b52421c00 con 0x563b518c8800 2023-05-20T01:27:19.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.618+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 <== osd.0 v2:172.21.15.16:6824/3923634179 12 ==== osd_op_reply(77 [pgnls start_epoch 119 out=81b] v115'9 uv1 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x563b518ab440 con 0x563b518c8800 2023-05-20T01:27:19.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.618+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:78 16.3f 16:feb1c803:::100000001fa.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b51a00e00 con 0x563b518c8800 2023-05-20T01:27:19.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.619+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 <== osd.0 v2:172.21.15.16:6824/3923634179 13 ==== osd_op_reply(78 100000001fa.00000000 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b518ab440 con 0x563b518c8800 2023-05-20T01:27:19.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.619+0000 7fcfbe0ce7c0 10 datascan.operator(): handling object 1099511628282.0 2023-05-20T01:27:19.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.619+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:79 16.3f 16:feb1c803:::100000001fa.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b51a00700 con 0x563b518c8800 2023-05-20T01:27:19.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.620+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 <== osd.0 v2:172.21.15.16:6824/3923634179 14 ==== osd_op_reply(79 100000001fa.00000000 [call] v119'10 uv10 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563b518ab440 con 0x563b518c8800 2023-05-20T01:27:19.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.620+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:80 17.0 17:00000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b51a01c00 con 0x563b518d2c00 2023-05-20T01:27:19.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.620+0000 7fcfbc07c700 1 -- 172.21.15.16:0/488708207 <== osd.7 v2:172.21.15.132:6800/466001149 5 ==== osd_op_reply(80 [pgnls start_epoch 119 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563b518ab8c0 con 0x563b518d2c00 2023-05-20T01:27:19.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.620+0000 7fcfb7873700 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:81 17.10 17:08000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b52420e00 con 0x563b518d3400 2023-05-20T01:27:19.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.621+0000 7fcfbc87d700 1 -- 172.21.15.16:0/488708207 <== osd.3 v2:172.21.15.16:6800/3669781890 9 ==== osd_op_reply(81 [pgnls start_epoch 119 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563b518ab680 con 0x563b518d3400 2023-05-20T01:27:19.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.621+0000 7fcfb7072700 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:82 17.8 17:10000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b519f8a80 con 0x563b518d2c00 2023-05-20T01:27:19.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.621+0000 7fcfbc07c700 1 -- 172.21.15.16:0/488708207 <== osd.7 v2:172.21.15.132:6800/466001149 6 ==== osd_op_reply(82 [pgnls start_epoch 119 out=81b] v115'4 uv4 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x563b518ab8c0 con 0x563b518d2c00 2023-05-20T01:27:19.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.621+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:83 17.8 17:1247e94b:::100000001f9.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b51a00a80 con 0x563b518d2c00 2023-05-20T01:27:19.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.621+0000 7fcfbc07c700 1 -- 172.21.15.16:0/488708207 <== osd.7 v2:172.21.15.132:6800/466001149 7 ==== osd_op_reply(83 100000001f9.00000001 [stat out=16b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b518ab8c0 con 0x563b518d2c00 2023-05-20T01:27:19.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.621+0000 7fcfbe0ce7c0 10 datascan.operator(): handling object 1099511628281.1 2023-05-20T01:27:19.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.621+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:84 16.36 16:6ee5e2c4:::100000001f9.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b51a00380 con 0x563b518d3000 2023-05-20T01:27:19.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.625+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 <== osd.1 v2:172.21.15.16:6808/3860264220 17 ==== osd_op_reply(84 100000001f9.00000000 [call,setxattr (8)] v119'20 uv20 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x563b518ab440 con 0x563b518d3000 2023-05-20T01:27:19.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.625+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:85 17.18 17:18000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b51a00000 con 0x563b518c8800 2023-05-20T01:27:19.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.626+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 <== osd.0 v2:172.21.15.16:6824/3923634179 15 ==== osd_op_reply(85 [pgnls start_epoch 119 out=81b] v115'4 uv4 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x563b518ab440 con 0x563b518c8800 2023-05-20T01:27:19.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.626+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:86 17.18 17:1cd44203:::100000001fb.00000002:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b51a01180 con 0x563b518c8800 2023-05-20T01:27:19.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.626+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 <== osd.0 v2:172.21.15.16:6824/3923634179 16 ==== osd_op_reply(86 100000001fb.00000002 [stat out=16b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b518ab440 con 0x563b518c8800 2023-05-20T01:27:19.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.626+0000 7fcfbe0ce7c0 10 datascan.operator(): handling object 1099511628283.2 2023-05-20T01:27:19.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.626+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:87 16.13 16:ca8fdbba:::100000001fb.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b52420a80 con 0x563b518d3000 2023-05-20T01:27:19.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.627+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 <== osd.1 v2:172.21.15.16:6808/3860264220 18 ==== osd_op_reply(87 100000001fb.00000000 [call,setxattr (8)] v119'18 uv18 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x563b518ab440 con 0x563b518d3000 2023-05-20T01:27:19.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.627+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:88 17.4 17:20000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b52420e00 con 0x563b518c8800 2023-05-20T01:27:19.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.628+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 <== osd.0 v2:172.21.15.16:6824/3923634179 17 ==== osd_op_reply(88 [pgnls start_epoch 119 out=81b] v115'4 uv4 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x563b518ab440 con 0x563b518c8800 2023-05-20T01:27:19.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.628+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:89 17.4 17:24ae92ec:::100000001fb.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b52421500 con 0x563b518c8800 2023-05-20T01:27:19.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.628+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 <== osd.0 v2:172.21.15.16:6824/3923634179 18 ==== osd_op_reply(89 100000001fb.00000001 [stat out=16b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b518ab440 con 0x563b518c8800 2023-05-20T01:27:19.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.628+0000 7fcfbe0ce7c0 10 datascan.operator(): handling object 1099511628283.1 2023-05-20T01:27:19.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.628+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:90 16.13 16:ca8fdbba:::100000001fb.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b52420700 con 0x563b518d3000 2023-05-20T01:27:19.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.629+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 <== osd.1 v2:172.21.15.16:6808/3860264220 19 ==== osd_op_reply(90 100000001fb.00000000 [call,setxattr (8)] v119'19 uv19 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x563b518ab440 con 0x563b518d3000 2023-05-20T01:27:19.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.629+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:91 17.14 17:28000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b51a00e00 con 0x563b518d3400 2023-05-20T01:27:19.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.629+0000 7fcfbc87d700 1 -- 172.21.15.16:0/488708207 <== osd.3 v2:172.21.15.16:6800/3669781890 10 ==== osd_op_reply(91 [pgnls start_epoch 119 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563b518ab680 con 0x563b518d3400 2023-05-20T01:27:19.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.629+0000 7fcfb7072700 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:92 17.c 17:30000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b519f8380 con 0x563b518d2c00 2023-05-20T01:27:19.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.630+0000 7fcfbc07c700 1 -- 172.21.15.16:0/488708207 <== osd.7 v2:172.21.15.132:6800/466001149 8 ==== osd_op_reply(92 [pgnls start_epoch 119 out=81b] v115'2 uv2 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x563b518ab8c0 con 0x563b518d2c00 2023-05-20T01:27:19.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.630+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:93 17.c 17:30604137:::100000001fc.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b51a00700 con 0x563b518d2c00 2023-05-20T01:27:19.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.630+0000 7fcfbc07c700 1 -- 172.21.15.16:0/488708207 <== osd.7 v2:172.21.15.132:6800/466001149 9 ==== osd_op_reply(93 100000001fc.00000000 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b518ab8c0 con 0x563b518d2c00 2023-05-20T01:27:19.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.630+0000 7fcfbe0ce7c0 10 datascan.operator(): handling object 1099511628284.0 2023-05-20T01:27:19.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.630+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:94 16.c 16:30604137:::100000001fc.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b51a00380 con 0x563b51a06000 2023-05-20T01:27:19.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.632+0000 7fcfbc87d700 1 -- 172.21.15.16:0/488708207 <== osd.5 v2:172.21.15.132:6816/1844689995 8 ==== osd_op_reply(94 100000001fc.00000000 [call,setxattr (8)] v119'7 uv7 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x563b518ab680 con 0x563b51a06000 2023-05-20T01:27:19.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.632+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:95 17.1c 17:38000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b51a00000 con 0x563b518d3000 2023-05-20T01:27:19.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.632+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 <== osd.1 v2:172.21.15.16:6808/3860264220 20 ==== osd_op_reply(95 [pgnls start_epoch 119 out=101b] v115'7 uv7 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x563b518ab440 con 0x563b518d3000 2023-05-20T01:27:19.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.632+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:96 17.1c 17:398da958:::100000001f7.00000002:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b51a01180 con 0x563b518d3000 2023-05-20T01:27:19.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.632+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 <== osd.1 v2:172.21.15.16:6808/3860264220 21 ==== osd_op_reply(96 100000001f7.00000002 [stat out=16b] v0'0 uv5 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b518ab440 con 0x563b518d3000 2023-05-20T01:27:19.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.632+0000 7fcfbe0ce7c0 10 datascan.operator(): handling object 1099511628279.2 2023-05-20T01:27:19.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.632+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:97 16.36 16:6fe38dc2:::100000001f7.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b52421880 con 0x563b518d3000 2023-05-20T01:27:19.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.633+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 <== osd.1 v2:172.21.15.16:6808/3860264220 22 ==== osd_op_reply(97 100000001f7.00000000 [call,setxattr (8)] v119'21 uv21 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x563b518ab440 con 0x563b518d3000 2023-05-20T01:27:19.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.633+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:98 17.1c 17:3c0114ac::::head [pgnls start_epoch 119 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b52421c00 con 0x563b518d3000 2023-05-20T01:27:19.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.633+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 <== osd.1 v2:172.21.15.16:6808/3860264220 23 ==== osd_op_reply(98 [pgnls start_epoch 119 out=101b] v115'7 uv7 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x563b518ab440 con 0x563b518d3000 2023-05-20T01:27:19.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.633+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:99 17.1c 17:3c0114ac:::100000001fa.00000004:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b52420a80 con 0x563b518d3000 2023-05-20T01:27:19.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.634+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 <== osd.1 v2:172.21.15.16:6808/3860264220 24 ==== osd_op_reply(99 100000001fa.00000004 [stat out=16b] v0'0 uv7 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b518ab440 con 0x563b518d3000 2023-05-20T01:27:19.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.634+0000 7fcfbe0ce7c0 10 datascan.operator(): handling object 1099511628282.4 2023-05-20T01:27:19.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.634+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:100 16.3f 16:feb1c803:::100000001fa.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b52420e00 con 0x563b518c8800 2023-05-20T01:27:19.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.635+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 <== osd.0 v2:172.21.15.16:6824/3923634179 19 ==== osd_op_reply(100 100000001fa.00000000 [call,setxattr (8)] v119'11 uv11 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x563b518ab440 con 0x563b518c8800 2023-05-20T01:27:19.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.635+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:101 17.1c 17:3d6db04b::::head [pgnls start_epoch 119 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b519f8a80 con 0x563b518d3000 2023-05-20T01:27:19.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.635+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 <== osd.1 v2:172.21.15.16:6808/3860264220 25 ==== osd_op_reply(101 [pgnls start_epoch 119 out=81b] v115'7 uv7 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x563b518ab440 con 0x563b518d3000 2023-05-20T01:27:19.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.635+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:102 17.1c 17:3d6db04b:::100000001f6.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b519f8380 con 0x563b518d3000 2023-05-20T01:27:19.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.635+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 <== osd.1 v2:172.21.15.16:6808/3860264220 26 ==== osd_op_reply(102 100000001f6.00000001 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b518ab440 con 0x563b518d3000 2023-05-20T01:27:19.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.635+0000 7fcfbe0ce7c0 10 datascan.operator(): handling object 1099511628278.1 2023-05-20T01:27:19.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.635+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:103 16.2a 16:57fe7535:::100000001f6.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b519f9c00 con 0x563b518d3000 2023-05-20T01:27:19.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.637+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 <== osd.1 v2:172.21.15.16:6808/3860264220 27 ==== osd_op_reply(103 100000001f6.00000000 [call,setxattr (8)] v119'12 uv12 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x563b518ab440 con 0x563b518d3000 2023-05-20T01:27:19.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.637+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:104 17.2 17:40000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b519f8700 con 0x563b518d3c00 2023-05-20T01:27:19.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.637+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 <== osd.4 v2:172.21.15.132:6817/506317223 10 ==== osd_op_reply(104 [pgnls start_epoch 119 out=101b] v115'4 uv4 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x563b518ab440 con 0x563b518d3c00 2023-05-20T01:27:19.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.637+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:105 17.2 17:40d56c48:::100000001fc.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b519f9180 con 0x563b518d3c00 2023-05-20T01:27:19.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.638+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 <== osd.4 v2:172.21.15.132:6817/506317223 11 ==== osd_op_reply(105 100000001fc.00000001 [stat out=16b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b518ab440 con 0x563b518d3c00 2023-05-20T01:27:19.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.638+0000 7fcfbe0ce7c0 10 datascan.operator(): handling object 1099511628284.1 2023-05-20T01:27:19.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.638+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:106 16.c 16:30604137:::100000001fc.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b51a01880 con 0x563b51a06000 2023-05-20T01:27:19.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.640+0000 7fcfbc87d700 1 -- 172.21.15.16:0/488708207 <== osd.5 v2:172.21.15.132:6816/1844689995 9 ==== osd_op_reply(106 100000001fc.00000000 [call,setxattr (8)] v119'8 uv8 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x563b518ab680 con 0x563b51a06000 2023-05-20T01:27:19.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.640+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:107 17.2 17:46f89f8a::::head [pgnls start_epoch 119 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b51a01c00 con 0x563b518d3c00 2023-05-20T01:27:19.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.640+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 <== osd.4 v2:172.21.15.132:6817/506317223 12 ==== osd_op_reply(107 [pgnls start_epoch 119 out=81b] v115'4 uv4 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x563b518ab440 con 0x563b518d3c00 2023-05-20T01:27:19.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.640+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:108 17.2 17:46f89f8a:::100000001f9.00000005:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b51a00a80 con 0x563b518d3c00 2023-05-20T01:27:19.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.640+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 <== osd.4 v2:172.21.15.132:6817/506317223 13 ==== osd_op_reply(108 100000001f9.00000005 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b518ab440 con 0x563b518d3c00 2023-05-20T01:27:19.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.640+0000 7fcfbe0ce7c0 10 datascan.operator(): handling object 1099511628281.5 2023-05-20T01:27:19.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.640+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:109 16.36 16:6ee5e2c4:::100000001f9.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b51a01500 con 0x563b518d3000 2023-05-20T01:27:19.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.642+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 <== osd.1 v2:172.21.15.16:6808/3860264220 28 ==== osd_op_reply(109 100000001f9.00000000 [call,setxattr (8)] v119'22 uv22 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x563b518ab440 con 0x563b518d3000 2023-05-20T01:27:19.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.642+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:110 17.12 17:48000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b52421180 con 0x563b518d3400 2023-05-20T01:27:19.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.642+0000 7fcfbc87d700 1 -- 172.21.15.16:0/488708207 <== osd.3 v2:172.21.15.16:6800/3669781890 11 ==== osd_op_reply(110 [pgnls start_epoch 119 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563b518ab680 con 0x563b518d3400 2023-05-20T01:27:19.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.642+0000 7fcfb7873700 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:111 17.a 17:50000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b52421500 con 0x563b518d2c00 2023-05-20T01:27:19.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.643+0000 7fcfbc07c700 1 -- 172.21.15.16:0/488708207 <== osd.7 v2:172.21.15.132:6800/466001149 10 ==== osd_op_reply(111 [pgnls start_epoch 119 out=101b] v115'7 uv7 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x563b518ab8c0 con 0x563b518d2c00 2023-05-20T01:27:19.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.643+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:112 17.a 17:544c5ef0:::100000001fc.00000003:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b519f8a80 con 0x563b518d2c00 2023-05-20T01:27:19.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.643+0000 7fcfbc07c700 1 -- 172.21.15.16:0/488708207 <== osd.7 v2:172.21.15.132:6800/466001149 11 ==== osd_op_reply(112 100000001fc.00000003 [stat out=16b] v0'0 uv7 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b518ab8c0 con 0x563b518d2c00 2023-05-20T01:27:19.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.643+0000 7fcfbe0ce7c0 10 datascan.operator(): handling object 1099511628284.3 2023-05-20T01:27:19.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.643+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:113 16.c 16:30604137:::100000001fc.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b519f8380 con 0x563b51a06000 2023-05-20T01:27:19.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.644+0000 7fcfbc87d700 1 -- 172.21.15.16:0/488708207 <== osd.5 v2:172.21.15.132:6816/1844689995 10 ==== osd_op_reply(113 100000001fc.00000000 [call,setxattr (8)] v119'9 uv9 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x563b518ab680 con 0x563b51a06000 2023-05-20T01:27:19.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.644+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:114 17.a 17:56c65d28::::head [pgnls start_epoch 119 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b519f9500 con 0x563b518d2c00 2023-05-20T01:27:19.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.645+0000 7fcfbc07c700 1 -- 172.21.15.16:0/488708207 <== osd.7 v2:172.21.15.132:6800/466001149 12 ==== osd_op_reply(114 [pgnls start_epoch 119 out=101b] v115'7 uv7 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x563b518ab8c0 con 0x563b518d2c00 2023-05-20T01:27:19.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.645+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:115 17.a 17:56c65d28:::100000001fa.00000006:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b519f9880 con 0x563b518d2c00 2023-05-20T01:27:19.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.645+0000 7fcfbc07c700 1 -- 172.21.15.16:0/488708207 <== osd.7 v2:172.21.15.132:6800/466001149 13 ==== osd_op_reply(115 100000001fa.00000006 [stat out=16b] v0'0 uv5 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b518ab8c0 con 0x563b518d2c00 2023-05-20T01:27:19.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.645+0000 7fcfbe0ce7c0 10 datascan.operator(): handling object 1099511628282.6 2023-05-20T01:27:19.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.645+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:116 16.3f 16:feb1c803:::100000001fa.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b519f8e00 con 0x563b518c8800 2023-05-20T01:27:19.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.646+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 <== osd.0 v2:172.21.15.16:6824/3923634179 20 ==== osd_op_reply(116 100000001fa.00000000 [call,setxattr (8)] v119'12 uv12 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x563b518ab440 con 0x563b518c8800 2023-05-20T01:27:19.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.646+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:117 17.a 17:57fe7535::::head [pgnls start_epoch 119 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b51a34000 con 0x563b518d2c00 2023-05-20T01:27:19.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.647+0000 7fcfbc07c700 1 -- 172.21.15.16:0/488708207 <== osd.7 v2:172.21.15.132:6800/466001149 14 ==== osd_op_reply(117 [pgnls start_epoch 119 out=81b] v115'7 uv7 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x563b518ab8c0 con 0x563b518d2c00 2023-05-20T01:27:19.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.647+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:118 17.a 17:57fe7535:::100000001f6.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b51a34380 con 0x563b518d2c00 2023-05-20T01:27:19.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.647+0000 7fcfbc07c700 1 -- 172.21.15.16:0/488708207 <== osd.7 v2:172.21.15.132:6800/466001149 15 ==== osd_op_reply(118 100000001f6.00000000 [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b518ab8c0 con 0x563b518d2c00 2023-05-20T01:27:19.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.647+0000 7fcfbe0ce7c0 10 datascan.operator(): handling object 1099511628278.0 2023-05-20T01:27:19.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.647+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:119 16.2a 16:57fe7535:::100000001f6.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b51a34700 con 0x563b518d3000 2023-05-20T01:27:19.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.648+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 <== osd.1 v2:172.21.15.16:6808/3860264220 29 ==== osd_op_reply(119 100000001f6.00000000 [call,setxattr (8)] v119'13 uv13 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x563b518ab440 con 0x563b518d3000 2023-05-20T01:27:19.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.648+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:120 17.1a 17:58000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b51a34a80 con 0x563b518c8800 2023-05-20T01:27:19.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.649+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 <== osd.0 v2:172.21.15.16:6824/3923634179 21 ==== osd_op_reply(120 [pgnls start_epoch 119 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563b518ab440 con 0x563b518c8800 2023-05-20T01:27:19.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.649+0000 7fcfb7873700 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:121 17.6 17:60000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b52420700 con 0x563b518d3800 2023-05-20T01:27:19.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.649+0000 7fcfbc07c700 1 -- 172.21.15.16:0/488708207 <== osd.6 v2:172.21.15.132:6801/727581123 11 ==== osd_op_reply(121 [pgnls start_epoch 119 out=81b] v115'3 uv3 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x563b518ab8c0 con 0x563b518d3800 2023-05-20T01:27:19.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.649+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:122 17.6 17:613bf181:::100000001f7.00000003:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b51a34e00 con 0x563b518d3800 2023-05-20T01:27:19.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.649+0000 7fcfbc07c700 1 -- 172.21.15.16:0/488708207 <== osd.6 v2:172.21.15.132:6801/727581123 12 ==== osd_op_reply(122 100000001f7.00000003 [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b518ab8c0 con 0x563b518d3800 2023-05-20T01:27:19.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.649+0000 7fcfbe0ce7c0 10 datascan.operator(): handling object 1099511628279.3 2023-05-20T01:27:19.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.649+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:123 16.36 16:6fe38dc2:::100000001f7.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b51a35180 con 0x563b518d3000 2023-05-20T01:27:19.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.651+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 <== osd.1 v2:172.21.15.16:6808/3860264220 30 ==== osd_op_reply(123 100000001f7.00000000 [call,setxattr (8)] v119'23 uv23 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x563b518ab440 con 0x563b518d3000 2023-05-20T01:27:19.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.651+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:124 17.16 17:68000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b51a35500 con 0x563b51a06000 2023-05-20T01:27:19.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.651+0000 7fcfbc87d700 1 -- 172.21.15.16:0/488708207 <== osd.5 v2:172.21.15.132:6816/1844689995 11 ==== osd_op_reply(124 [pgnls start_epoch 119 out=101b] v115'11 uv11 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x563b518ab680 con 0x563b51a06000 2023-05-20T01:27:19.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.651+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:125 17.16 17:6df493a7:::100000001f9.00000006:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b51a35880 con 0x563b51a06000 2023-05-20T01:27:19.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.651+0000 7fcfbc87d700 1 -- 172.21.15.16:0/488708207 <== osd.5 v2:172.21.15.132:6816/1844689995 12 ==== osd_op_reply(125 100000001f9.00000006 [stat out=16b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b518ab680 con 0x563b51a06000 2023-05-20T01:27:19.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.651+0000 7fcfbe0ce7c0 10 datascan.operator(): handling object 1099511628281.6 2023-05-20T01:27:19.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.651+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:126 16.36 16:6ee5e2c4:::100000001f9.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b51a35c00 con 0x563b518d3000 2023-05-20T01:27:19.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.652+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 <== osd.1 v2:172.21.15.16:6808/3860264220 31 ==== osd_op_reply(126 100000001f9.00000000 [call,setxattr (8)] v119'24 uv24 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x563b518ab440 con 0x563b518d3000 2023-05-20T01:27:19.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.652+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:127 17.16 17:6ee5e2c4::::head [pgnls start_epoch 119 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b51a34700 con 0x563b51a06000 2023-05-20T01:27:19.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.653+0000 7fcfbc87d700 1 -- 172.21.15.16:0/488708207 <== osd.5 v2:172.21.15.132:6816/1844689995 13 ==== osd_op_reply(127 [pgnls start_epoch 119 out=101b] v115'11 uv11 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x563b518ab680 con 0x563b51a06000 2023-05-20T01:27:19.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.653+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:128 17.16 17:6ee5e2c4:::100000001f9.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b519f9c00 con 0x563b51a06000 2023-05-20T01:27:19.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.653+0000 7fcfbc87d700 1 -- 172.21.15.16:0/488708207 <== osd.5 v2:172.21.15.132:6816/1844689995 14 ==== osd_op_reply(128 100000001f9.00000000 [stat out=16b] v0'0 uv10 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b518ab680 con 0x563b51a06000 2023-05-20T01:27:19.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.653+0000 7fcfbe0ce7c0 10 datascan.operator(): handling object 1099511628281.0 2023-05-20T01:27:19.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.653+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:129 16.36 16:6ee5e2c4:::100000001f9.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b519f8700 con 0x563b518d3000 2023-05-20T01:27:19.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.654+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 <== osd.1 v2:172.21.15.16:6808/3860264220 32 ==== osd_op_reply(129 100000001f9.00000000 [call,setxattr (8)] v119'25 uv25 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x563b518ab440 con 0x563b518d3000 2023-05-20T01:27:19.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.654+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:130 17.16 17:6fe38dc2::::head [pgnls start_epoch 119 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b519f9180 con 0x563b51a06000 2023-05-20T01:27:19.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.654+0000 7fcfbc87d700 1 -- 172.21.15.16:0/488708207 <== osd.5 v2:172.21.15.132:6816/1844689995 15 ==== osd_op_reply(130 [pgnls start_epoch 119 out=81b] v115'11 uv11 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x563b518ab680 con 0x563b51a06000 2023-05-20T01:27:19.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.655+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:131 17.16 17:6fe38dc2:::100000001f7.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b519f8e00 con 0x563b51a06000 2023-05-20T01:27:19.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.655+0000 7fcfbc87d700 1 -- 172.21.15.16:0/488708207 <== osd.5 v2:172.21.15.132:6816/1844689995 16 ==== osd_op_reply(131 100000001f7.00000000 [stat out=16b] v0'0 uv5 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b518ab680 con 0x563b51a06000 2023-05-20T01:27:19.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.655+0000 7fcfbe0ce7c0 10 datascan.operator(): handling object 1099511628279.0 2023-05-20T01:27:19.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.655+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:132 16.36 16:6fe38dc2:::100000001f7.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b519f8a80 con 0x563b518d3000 2023-05-20T01:27:19.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.656+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 <== osd.1 v2:172.21.15.16:6808/3860264220 33 ==== osd_op_reply(132 100000001f7.00000000 [call,setxattr (8)] v119'26 uv26 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x563b518ab440 con 0x563b518d3000 2023-05-20T01:27:19.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.656+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:133 17.e 17:70000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b519f9500 con 0x563b518d2800 2023-05-20T01:27:19.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.656+0000 7fcfbc87d700 1 -- 172.21.15.16:0/488708207 <== osd.2 v2:172.21.15.16:6816/293187030 12 ==== osd_op_reply(133 [pgnls start_epoch 119 out=101b] v115'10 uv10 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x563b518ab680 con 0x563b518d2800 2023-05-20T01:27:19.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.656+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:134 17.e 17:72395991:::10000000001.00000003:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b519f9880 con 0x563b518d2800 2023-05-20T01:27:19.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.656+0000 7fcfbc87d700 1 -- 172.21.15.16:0/488708207 <== osd.2 v2:172.21.15.16:6816/293187030 13 ==== osd_op_reply(134 10000000001.00000003 [stat out=16b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b518ab680 con 0x563b518d2800 2023-05-20T01:27:19.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.656+0000 7fcfbe0ce7c0 10 datascan.operator(): handling object 1099511627777.3 2023-05-20T01:27:19.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.656+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:135 16.7 16:e2e07930:::10000000001.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b52421500 con 0x563b518d3400 2023-05-20T01:27:19.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.658+0000 7fcfbc87d700 1 -- 172.21.15.16:0/488708207 <== osd.3 v2:172.21.15.16:6800/3669781890 12 ==== osd_op_reply(135 10000000001.00000000 [call,setxattr (8)] v119'8 uv8 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x563b518ab680 con 0x563b518d3400 2023-05-20T01:27:19.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.658+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:136 17.e 17:728aa123::::head [pgnls start_epoch 119 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b51a01c00 con 0x563b518d2800 2023-05-20T01:27:19.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.658+0000 7fcfbc87d700 1 -- 172.21.15.16:0/488708207 <== osd.2 v2:172.21.15.16:6816/293187030 14 ==== osd_op_reply(136 [pgnls start_epoch 119 out=101b] v115'10 uv10 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x563b518ab680 con 0x563b518d2800 2023-05-20T01:27:19.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.658+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:137 17.e 17:728aa123:::100000001f6.00000003:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b51a00a80 con 0x563b518d2800 2023-05-20T01:27:19.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.658+0000 7fcfbc87d700 1 -- 172.21.15.16:0/488708207 <== osd.2 v2:172.21.15.16:6816/293187030 15 ==== osd_op_reply(137 100000001f6.00000003 [stat out=16b] v0'0 uv6 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b518ab680 con 0x563b518d2800 2023-05-20T01:27:19.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.658+0000 7fcfbe0ce7c0 10 datascan.operator(): handling object 1099511628278.3 2023-05-20T01:27:19.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.658+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:138 16.2a 16:57fe7535:::100000001f6.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b51a01500 con 0x563b518d3000 2023-05-20T01:27:19.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.660+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 <== osd.1 v2:172.21.15.16:6808/3860264220 34 ==== osd_op_reply(138 100000001f6.00000000 [call,setxattr (8)] v119'14 uv14 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x563b518ab440 con 0x563b518d3000 2023-05-20T01:27:19.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.660+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:139 17.e 17:7780a862::::head [pgnls start_epoch 119 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b51a00700 con 0x563b518d2800 2023-05-20T01:27:19.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.660+0000 7fcfbc87d700 1 -- 172.21.15.16:0/488708207 <== osd.2 v2:172.21.15.16:6816/293187030 16 ==== osd_op_reply(139 [pgnls start_epoch 119 out=81b] v115'10 uv10 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x563b518ab680 con 0x563b518d2800 2023-05-20T01:27:19.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.660+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:140 17.e 17:7780a862:::100000001fa.00000003:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b519f9c00 con 0x563b518d2800 2023-05-20T01:27:19.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.660+0000 7fcfbc87d700 1 -- 172.21.15.16:0/488708207 <== osd.2 v2:172.21.15.16:6816/293187030 17 ==== osd_op_reply(140 100000001fa.00000003 [stat out=16b] v0'0 uv10 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b518ab680 con 0x563b518d2800 2023-05-20T01:27:19.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.660+0000 7fcfbe0ce7c0 10 datascan.operator(): handling object 1099511628282.3 2023-05-20T01:27:19.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.660+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:141 16.3f 16:feb1c803:::100000001fa.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b519f9180 con 0x563b518c8800 2023-05-20T01:27:19.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.661+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 <== osd.0 v2:172.21.15.16:6824/3923634179 22 ==== osd_op_reply(141 100000001fa.00000000 [call,setxattr (8)] v119'13 uv13 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x563b518ab440 con 0x563b518c8800 2023-05-20T01:27:19.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.661+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:142 17.1e 17:78000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b519f8e00 con 0x563b518d3000 2023-05-20T01:27:19.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.662+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 <== osd.1 v2:172.21.15.16:6808/3860264220 35 ==== osd_op_reply(142 [pgnls start_epoch 119 out=101b] v115'6 uv6 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x563b518ab440 con 0x563b518d3000 2023-05-20T01:27:19.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.662+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:143 17.1e 17:7e4d8c81:::10000000001.00000002:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b519f9500 con 0x563b518d3000 2023-05-20T01:27:19.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.662+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 <== osd.1 v2:172.21.15.16:6808/3860264220 36 ==== osd_op_reply(143 10000000001.00000002 [stat out=16b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b518ab440 con 0x563b518d3000 2023-05-20T01:27:19.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.662+0000 7fcfbe0ce7c0 10 datascan.operator(): handling object 1099511627777.2 2023-05-20T01:27:19.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.662+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:144 16.7 16:e2e07930:::10000000001.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b519f9880 con 0x563b518d3400 2023-05-20T01:27:19.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.663+0000 7fcfbc87d700 1 -- 172.21.15.16:0/488708207 <== osd.3 v2:172.21.15.16:6800/3669781890 13 ==== osd_op_reply(144 10000000001.00000000 [call,setxattr (8)] v119'9 uv9 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x563b518ab680 con 0x563b518d3400 2023-05-20T01:27:19.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.663+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:145 17.1e 17:7e8133fb::::head [pgnls start_epoch 119 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b519f8380 con 0x563b518d3000 2023-05-20T01:27:19.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.663+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 <== osd.1 v2:172.21.15.16:6808/3860264220 37 ==== osd_op_reply(145 [pgnls start_epoch 119 out=81b] v115'6 uv6 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x563b518ab440 con 0x563b518d3000 2023-05-20T01:27:19.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.663+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:146 17.1e 17:7e8133fb:::100000001fb.00000006:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b52421500 con 0x563b518d3000 2023-05-20T01:27:19.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.663+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 <== osd.1 v2:172.21.15.16:6808/3860264220 38 ==== osd_op_reply(146 100000001fb.00000006 [stat out=16b] v0'0 uv6 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b518ab440 con 0x563b518d3000 2023-05-20T01:27:19.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.663+0000 7fcfbe0ce7c0 10 datascan.operator(): handling object 1099511628283.6 2023-05-20T01:27:19.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.663+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:147 16.13 16:ca8fdbba:::100000001fb.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b52421180 con 0x563b518d3000 2023-05-20T01:27:19.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.665+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 <== osd.1 v2:172.21.15.16:6808/3860264220 39 ==== osd_op_reply(147 100000001fb.00000000 [call,setxattr (8)] v119'20 uv20 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x563b518ab440 con 0x563b518d3000 2023-05-20T01:27:19.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.665+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:148 17.1 17:80000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b51a01c00 con 0x563b518c8800 2023-05-20T01:27:19.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.665+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 <== osd.0 v2:172.21.15.16:6824/3923634179 23 ==== osd_op_reply(148 [pgnls start_epoch 119 out=81b] v115'4 uv4 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x563b518ab440 con 0x563b518c8800 2023-05-20T01:27:19.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.665+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:149 17.1 17:848ebb0b:::100000001f7.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b51a00a80 con 0x563b518c8800 2023-05-20T01:27:19.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.665+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 <== osd.0 v2:172.21.15.16:6824/3923634179 24 ==== osd_op_reply(149 100000001f7.00000001 [stat out=16b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b518ab440 con 0x563b518c8800 2023-05-20T01:27:19.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.665+0000 7fcfbe0ce7c0 10 datascan.operator(): handling object 1099511628279.1 2023-05-20T01:27:19.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.665+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:150 16.36 16:6fe38dc2:::100000001f7.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b51a00700 con 0x563b518d3000 2023-05-20T01:27:19.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.667+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 <== osd.1 v2:172.21.15.16:6808/3860264220 40 ==== osd_op_reply(150 100000001f7.00000000 [call,setxattr (8)] v119'27 uv27 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x563b518ab440 con 0x563b518d3000 2023-05-20T01:27:19.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.667+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:151 17.11 17:88000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b51a00e00 con 0x563b518d3400 2023-05-20T01:27:19.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.667+0000 7fcfbc87d700 1 -- 172.21.15.16:0/488708207 <== osd.3 v2:172.21.15.16:6800/3669781890 14 ==== osd_op_reply(151 [pgnls start_epoch 119 out=81b] v115'4 uv4 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x563b518ab680 con 0x563b518d3400 2023-05-20T01:27:19.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.667+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:152 17.11 17:893be98d:::100000001f8.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b51a00380 con 0x563b518d3400 2023-05-20T01:27:19.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.667+0000 7fcfbc87d700 1 -- 172.21.15.16:0/488708207 <== osd.3 v2:172.21.15.16:6800/3669781890 15 ==== osd_op_reply(152 100000001f8.00000001 [stat out=16b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b518ab680 con 0x563b518d3400 2023-05-20T01:27:19.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.667+0000 7fcfbe0ce7c0 10 datascan.operator(): handling object 1099511628280.1 2023-05-20T01:27:19.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.667+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:153 16.19 16:997b7129:::100000001f8.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b51a01880 con 0x563b518d3c00 2023-05-20T01:27:19.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.668+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 <== osd.4 v2:172.21.15.132:6817/506317223 14 ==== osd_op_reply(153 100000001f8.00000000 [call,setxattr (8)] v119'13 uv13 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x563b518ab440 con 0x563b518d3c00 2023-05-20T01:27:19.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.669+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:154 17.9 17:90000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b51a35500 con 0x563b518d3400 2023-05-20T01:27:19.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.669+0000 7fcfbc87d700 1 -- 172.21.15.16:0/488708207 <== osd.3 v2:172.21.15.16:6800/3669781890 16 ==== osd_op_reply(154 [pgnls start_epoch 119 out=81b] v115'4 uv4 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x563b518ab680 con 0x563b518d3400 2023-05-20T01:27:19.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.669+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:155 17.9 17:94f3e7b4:::100000001fa.00000002:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b51a35880 con 0x563b518d3400 2023-05-20T01:27:19.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.669+0000 7fcfbc87d700 1 -- 172.21.15.16:0/488708207 <== osd.3 v2:172.21.15.16:6800/3669781890 17 ==== osd_op_reply(155 100000001fa.00000002 [stat out=16b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b518ab680 con 0x563b518d3400 2023-05-20T01:27:19.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.669+0000 7fcfbe0ce7c0 10 datascan.operator(): handling object 1099511628282.2 2023-05-20T01:27:19.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.669+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:156 16.3f 16:feb1c803:::100000001fa.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b51a34700 con 0x563b518c8800 2023-05-20T01:27:19.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.670+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 <== osd.0 v2:172.21.15.16:6824/3923634179 25 ==== osd_op_reply(156 100000001fa.00000000 [call,setxattr (8)] v119'14 uv14 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x563b518ab440 con 0x563b518c8800 2023-05-20T01:27:19.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.670+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:157 17.19 17:98000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b51a01500 con 0x563b51a06000 2023-05-20T01:27:19.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.670+0000 7fcfbc87d700 1 -- 172.21.15.16:0/488708207 <== osd.5 v2:172.21.15.132:6816/1844689995 17 ==== osd_op_reply(157 [pgnls start_epoch 119 out=81b] v115'5 uv5 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x563b518ab680 con 0x563b51a06000 2023-05-20T01:27:19.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.671+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:158 17.19 17:997b7129:::100000001f8.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b51a01c00 con 0x563b51a06000 2023-05-20T01:27:19.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.671+0000 7fcfbc87d700 1 -- 172.21.15.16:0/488708207 <== osd.5 v2:172.21.15.132:6816/1844689995 18 ==== osd_op_reply(158 100000001f8.00000000 [stat out=16b] v0'0 uv5 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b518ab680 con 0x563b51a06000 2023-05-20T01:27:19.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.671+0000 7fcfbe0ce7c0 10 datascan.operator(): handling object 1099511628280.0 2023-05-20T01:27:19.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.671+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:159 16.19 16:997b7129:::100000001f8.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b51a00a80 con 0x563b518d3c00 2023-05-20T01:27:19.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.672+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 <== osd.4 v2:172.21.15.132:6817/506317223 15 ==== osd_op_reply(159 100000001f8.00000000 [call,setxattr (8)] v119'14 uv14 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x563b518ab440 con 0x563b518d3c00 2023-05-20T01:27:19.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.672+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:160 17.5 17:a0000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b51a00700 con 0x563b518d3800 2023-05-20T01:27:19.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.673+0000 7fcfbc07c700 1 -- 172.21.15.16:0/488708207 <== osd.6 v2:172.21.15.132:6801/727581123 13 ==== osd_op_reply(160 [pgnls start_epoch 119 out=81b] v115'4 uv4 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x563b518ab8c0 con 0x563b518d3800 2023-05-20T01:27:19.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.673+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:161 17.5 17:a238cdfc:::100000001f9.00000002:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b52421500 con 0x563b518d3800 2023-05-20T01:27:19.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.673+0000 7fcfbc07c700 1 -- 172.21.15.16:0/488708207 <== osd.6 v2:172.21.15.132:6801/727581123 14 ==== osd_op_reply(161 100000001f9.00000002 [stat out=16b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b518ab8c0 con 0x563b518d3800 2023-05-20T01:27:19.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.673+0000 7fcfbe0ce7c0 10 datascan.operator(): handling object 1099511628281.2 2023-05-20T01:27:19.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.673+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:162 16.36 16:6ee5e2c4:::100000001f9.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b52421180 con 0x563b518d3000 2023-05-20T01:27:19.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.674+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 <== osd.1 v2:172.21.15.16:6808/3860264220 41 ==== osd_op_reply(162 100000001f9.00000000 [call,setxattr (8)] v119'28 uv28 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x563b518ab440 con 0x563b518d3000 2023-05-20T01:27:19.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.674+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:163 17.15 17:a8000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b519f8700 con 0x563b51a06000 2023-05-20T01:27:19.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.674+0000 7fcfbc87d700 1 -- 172.21.15.16:0/488708207 <== osd.5 v2:172.21.15.132:6816/1844689995 19 ==== osd_op_reply(163 [pgnls start_epoch 119 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563b518ab680 con 0x563b51a06000 2023-05-20T01:27:19.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.674+0000 7fcfb7072700 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:164 17.d 17:b0000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b519f8000 con 0x563b518d2c00 2023-05-20T01:27:19.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.675+0000 7fcfbc07c700 1 -- 172.21.15.16:0/488708207 <== osd.7 v2:172.21.15.132:6800/466001149 16 ==== osd_op_reply(164 [pgnls start_epoch 119 out=81b] v115'2 uv2 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x563b518ab8c0 con 0x563b518d2c00 2023-05-20T01:27:19.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.675+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:165 17.d 17:b0a6c804:::100000001fc.00000002:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b519f8a80 con 0x563b518d2c00 2023-05-20T01:27:19.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.675+0000 7fcfbc07c700 1 -- 172.21.15.16:0/488708207 <== osd.7 v2:172.21.15.132:6800/466001149 17 ==== osd_op_reply(165 100000001fc.00000002 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b518ab8c0 con 0x563b518d2c00 2023-05-20T01:27:19.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.675+0000 7fcfbe0ce7c0 10 datascan.operator(): handling object 1099511628284.2 2023-05-20T01:27:19.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.675+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:166 16.c 16:30604137:::100000001fc.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b519f9180 con 0x563b51a06000 2023-05-20T01:27:19.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.676+0000 7fcfbc87d700 1 -- 172.21.15.16:0/488708207 <== osd.5 v2:172.21.15.132:6816/1844689995 20 ==== osd_op_reply(166 100000001fc.00000000 [call,setxattr (8)] v119'10 uv10 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x563b518ab680 con 0x563b51a06000 2023-05-20T01:27:19.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.676+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:167 17.1d 17:b8000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b519f8e00 con 0x563b518d2c00 2023-05-20T01:27:19.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.677+0000 7fcfbc07c700 1 -- 172.21.15.16:0/488708207 <== osd.7 v2:172.21.15.132:6800/466001149 18 ==== osd_op_reply(167 [pgnls start_epoch 119 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563b518ab8c0 con 0x563b518d2c00 2023-05-20T01:27:19.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.677+0000 7fcfb7072700 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:168 17.3 17:c0000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b51a35500 con 0x563b518d3c00 2023-05-20T01:27:19.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.677+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 <== osd.4 v2:172.21.15.132:6817/506317223 16 ==== osd_op_reply(168 [pgnls start_epoch 119 out=101b] v115'4 uv4 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x563b518ab440 con 0x563b518d3c00 2023-05-20T01:27:19.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.677+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:169 17.3 17:c0957981:::100000001fa.00000005:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b519f9500 con 0x563b518d3c00 2023-05-20T01:27:19.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.677+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 <== osd.4 v2:172.21.15.132:6817/506317223 17 ==== osd_op_reply(169 100000001fa.00000005 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b518ab440 con 0x563b518d3c00 2023-05-20T01:27:19.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.677+0000 7fcfbe0ce7c0 10 datascan.operator(): handling object 1099511628282.5 2023-05-20T01:27:19.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.677+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:170 16.3f 16:feb1c803:::100000001fa.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b519f8380 con 0x563b518c8800 2023-05-20T01:27:19.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.678+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 <== osd.0 v2:172.21.15.16:6824/3923634179 26 ==== osd_op_reply(170 100000001fa.00000000 [call,setxattr (8)] v119'15 uv15 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x563b518ab440 con 0x563b518c8800 2023-05-20T01:27:19.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.679+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:171 17.3 17:c5d8c51d::::head [pgnls start_epoch 119 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b51a35880 con 0x563b518d3c00 2023-05-20T01:27:19.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.679+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 <== osd.4 v2:172.21.15.132:6817/506317223 18 ==== osd_op_reply(171 [pgnls start_epoch 119 out=81b] v115'4 uv4 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x563b518ab440 con 0x563b518d3c00 2023-05-20T01:27:19.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.679+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:172 17.3 17:c5d8c51d:::100000001fb.00000004:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b51a34a80 con 0x563b518d3c00 2023-05-20T01:27:19.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.679+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 <== osd.4 v2:172.21.15.132:6817/506317223 19 ==== osd_op_reply(172 100000001fb.00000004 [stat out=16b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b518ab440 con 0x563b518d3c00 2023-05-20T01:27:19.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.679+0000 7fcfbe0ce7c0 10 datascan.operator(): handling object 1099511628283.4 2023-05-20T01:27:19.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.679+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:173 16.13 16:ca8fdbba:::100000001fb.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b51a35180 con 0x563b518d3000 2023-05-20T01:27:19.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.680+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 <== osd.1 v2:172.21.15.16:6808/3860264220 42 ==== osd_op_reply(173 100000001fb.00000000 [call,setxattr (8)] v119'21 uv21 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x563b518ab440 con 0x563b518d3000 2023-05-20T01:27:19.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.680+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:174 17.13 17:c8000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b51a35c00 con 0x563b518d3400 2023-05-20T01:27:19.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.680+0000 7fcfbc87d700 1 -- 172.21.15.16:0/488708207 <== osd.3 v2:172.21.15.16:6800/3669781890 18 ==== osd_op_reply(174 [pgnls start_epoch 119 out=101b] v115'13 uv13 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x563b518ab680 con 0x563b518d3400 2023-05-20T01:27:19.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.681+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:175 17.13 17:c9e7fe79:::100000001f9.00000003:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b51a34000 con 0x563b518d3400 2023-05-20T01:27:19.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.681+0000 7fcfbc87d700 1 -- 172.21.15.16:0/488708207 <== osd.3 v2:172.21.15.16:6800/3669781890 19 ==== osd_op_reply(175 100000001f9.00000003 [stat out=16b] v0'0 uv5 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b518ab680 con 0x563b518d3400 2023-05-20T01:27:19.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.681+0000 7fcfbe0ce7c0 10 datascan.operator(): handling object 1099511628281.3 2023-05-20T01:27:19.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.681+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:176 16.36 16:6ee5e2c4:::100000001f9.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b51a34380 con 0x563b518d3000 2023-05-20T01:27:19.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.682+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 <== osd.1 v2:172.21.15.16:6808/3860264220 43 ==== osd_op_reply(176 100000001f9.00000000 [call,setxattr (8)] v119'29 uv29 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x563b518ab440 con 0x563b518d3000 2023-05-20T01:27:19.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.682+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:177 17.13 17:ca8fdbba::::head [pgnls start_epoch 119 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b51a3e000 con 0x563b518d3400 2023-05-20T01:27:19.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.682+0000 7fcfbc87d700 1 -- 172.21.15.16:0/488708207 <== osd.3 v2:172.21.15.16:6800/3669781890 20 ==== osd_op_reply(177 [pgnls start_epoch 119 out=101b] v115'13 uv13 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x563b518ab680 con 0x563b518d3400 2023-05-20T01:27:19.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.682+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:178 17.13 17:ca8fdbba:::100000001fb.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b51a3e380 con 0x563b518d3400 2023-05-20T01:27:19.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.682+0000 7fcfbc87d700 1 -- 172.21.15.16:0/488708207 <== osd.3 v2:172.21.15.16:6800/3669781890 21 ==== osd_op_reply(178 100000001fb.00000000 [stat out=16b] v0'0 uv13 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b518ab680 con 0x563b518d3400 2023-05-20T01:27:19.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.682+0000 7fcfbe0ce7c0 10 datascan.operator(): handling object 1099511628283.0 2023-05-20T01:27:19.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.682+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:179 16.13 16:ca8fdbba:::100000001fb.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b51a3e700 con 0x563b518d3000 2023-05-20T01:27:19.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.683+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 <== osd.1 v2:172.21.15.16:6808/3860264220 44 ==== osd_op_reply(179 100000001fb.00000000 [call,setxattr (8)] v119'22 uv22 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x563b518ab440 con 0x563b518d3000 2023-05-20T01:27:19.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.683+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:180 17.13 17:cb24bee2::::head [pgnls start_epoch 119 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b51a3ea80 con 0x563b518d3400 2023-05-20T01:27:19.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.684+0000 7fcfbc87d700 1 -- 172.21.15.16:0/488708207 <== osd.3 v2:172.21.15.16:6800/3669781890 22 ==== osd_op_reply(180 [pgnls start_epoch 119 out=81b] v115'13 uv13 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x563b518ab680 con 0x563b518d3400 2023-05-20T01:27:19.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.684+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:181 17.13 17:cb24bee2:::100000001fa.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b51a3ee00 con 0x563b518d3400 2023-05-20T01:27:19.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.684+0000 7fcfbc87d700 1 -- 172.21.15.16:0/488708207 <== osd.3 v2:172.21.15.16:6800/3669781890 23 ==== osd_op_reply(181 100000001fa.00000001 [stat out=16b] v0'0 uv9 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b518ab680 con 0x563b518d3400 2023-05-20T01:27:19.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.684+0000 7fcfbe0ce7c0 10 datascan.operator(): handling object 1099511628282.1 2023-05-20T01:27:19.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.684+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:182 16.3f 16:feb1c803:::100000001fa.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b51a3f180 con 0x563b518c8800 2023-05-20T01:27:19.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.685+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 <== osd.0 v2:172.21.15.16:6824/3923634179 27 ==== osd_op_reply(182 100000001fa.00000000 [call,setxattr (8)] v119'16 uv16 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x563b518ab440 con 0x563b518c8800 2023-05-20T01:27:19.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.685+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:183 17.b 17:d0000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b51a3f500 con 0x563b518c8800 2023-05-20T01:27:19.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.685+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 <== osd.0 v2:172.21.15.16:6824/3923634179 28 ==== osd_op_reply(183 [pgnls start_epoch 119 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563b518ab440 con 0x563b518c8800 2023-05-20T01:27:19.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.685+0000 7fcfb7072700 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:184 17.1b 17:d8000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b519f9c00 con 0x563b518d3c00 2023-05-20T01:27:19.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.686+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 <== osd.4 v2:172.21.15.132:6817/506317223 20 ==== osd_op_reply(184 [pgnls start_epoch 119 out=81b] v115'4 uv4 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x563b518ab440 con 0x563b518d3c00 2023-05-20T01:27:19.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.686+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:185 17.1b 17:de1cf3c0:::100000001fb.00000003:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b51a3f880 con 0x563b518d3c00 2023-05-20T01:27:19.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.686+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 <== osd.4 v2:172.21.15.132:6817/506317223 21 ==== osd_op_reply(185 100000001fb.00000003 [stat out=16b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b518ab440 con 0x563b518d3c00 2023-05-20T01:27:19.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.686+0000 7fcfbe0ce7c0 10 datascan.operator(): handling object 1099511628283.3 2023-05-20T01:27:19.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.686+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:186 16.13 16:ca8fdbba:::100000001fb.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b51a3fc00 con 0x563b518d3000 2023-05-20T01:27:19.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.687+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 <== osd.1 v2:172.21.15.16:6808/3860264220 45 ==== osd_op_reply(186 100000001fb.00000000 [call,setxattr (8)] v119'23 uv23 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x563b518ab440 con 0x563b518d3000 2023-05-20T01:27:19.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.687+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:187 17.7 17:e0000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b51a3e700 con 0x563b518c8800 2023-05-20T01:27:19.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.687+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 <== osd.0 v2:172.21.15.16:6824/3923634179 29 ==== osd_op_reply(187 [pgnls start_epoch 119 out=101b] v115'8 uv8 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x563b518ab440 con 0x563b518c8800 2023-05-20T01:27:19.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.687+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:188 17.7 17:e2e07930:::10000000001.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b51a3f180 con 0x563b518c8800 2023-05-20T01:27:19.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.688+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 <== osd.0 v2:172.21.15.16:6824/3923634179 30 ==== osd_op_reply(188 10000000001.00000000 [stat out=16b] v0'0 uv6 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b518ab440 con 0x563b518c8800 2023-05-20T01:27:19.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.688+0000 7fcfbe0ce7c0 10 datascan.operator(): handling object 1099511627777.0 2023-05-20T01:27:19.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.688+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:189 16.7 16:e2e07930:::10000000001.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b51a34700 con 0x563b518d3400 2023-05-20T01:27:19.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.689+0000 7fcfbc87d700 1 -- 172.21.15.16:0/488708207 <== osd.3 v2:172.21.15.16:6800/3669781890 24 ==== osd_op_reply(189 10000000001.00000000 [call,setxattr (8)] v119'10 uv10 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x563b518ab680 con 0x563b518d3400 2023-05-20T01:27:19.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.689+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:190 17.7 17:e7ecf421::::head [pgnls start_epoch 119 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b51a35500 con 0x563b518c8800 2023-05-20T01:27:19.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.689+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 <== osd.0 v2:172.21.15.16:6824/3923634179 31 ==== osd_op_reply(190 [pgnls start_epoch 119 out=81b] v115'8 uv8 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x563b518ab440 con 0x563b518c8800 2023-05-20T01:27:19.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.689+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:191 17.7 17:e7ecf421:::100000001f9.00000004:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b51a35880 con 0x563b518c8800 2023-05-20T01:27:19.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.689+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 <== osd.0 v2:172.21.15.16:6824/3923634179 32 ==== osd_op_reply(191 100000001f9.00000004 [stat out=16b] v0'0 uv8 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b518ab440 con 0x563b518c8800 2023-05-20T01:27:19.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.689+0000 7fcfbe0ce7c0 10 datascan.operator(): handling object 1099511628281.4 2023-05-20T01:27:19.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.689+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:192 16.36 16:6ee5e2c4:::100000001f9.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b51a34a80 con 0x563b518d3000 2023-05-20T01:27:19.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.690+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 <== osd.1 v2:172.21.15.16:6808/3860264220 46 ==== osd_op_reply(192 100000001f9.00000000 [call,setxattr (8)] v119'30 uv30 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x563b518ab440 con 0x563b518d3000 2023-05-20T01:27:19.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.690+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:193 17.17 17:e8000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b51a35180 con 0x563b518d3800 2023-05-20T01:27:19.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.691+0000 7fcfbc07c700 1 -- 172.21.15.16:0/488708207 <== osd.6 v2:172.21.15.132:6801/727581123 15 ==== osd_op_reply(193 [pgnls start_epoch 119 out=101b] v115'4 uv4 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x563b518ab8c0 con 0x563b518d3800 2023-05-20T01:27:19.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.691+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:194 17.17 17:ee7e3069:::100000001f9.00000007:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b51a34380 con 0x563b518d3800 2023-05-20T01:27:19.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.691+0000 7fcfbc07c700 1 -- 172.21.15.16:0/488708207 <== osd.6 v2:172.21.15.132:6801/727581123 16 ==== osd_op_reply(194 100000001f9.00000007 [stat out=16b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b518ab8c0 con 0x563b518d3800 2023-05-20T01:27:19.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.691+0000 7fcfbe0ce7c0 10 datascan.operator(): handling object 1099511628281.7 2023-05-20T01:27:19.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.691+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:195 16.36 16:6ee5e2c4:::100000001f9.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b519f9500 con 0x563b518d3000 2023-05-20T01:27:19.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.692+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 <== osd.1 v2:172.21.15.16:6808/3860264220 47 ==== osd_op_reply(195 100000001f9.00000000 [call,setxattr (8)] v119'31 uv31 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x563b518ab440 con 0x563b518d3000 2023-05-20T01:27:19.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.692+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:196 17.17 17:eff9002a::::head [pgnls start_epoch 119 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b519f8380 con 0x563b518d3800 2023-05-20T01:27:19.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.693+0000 7fcfbc07c700 1 -- 172.21.15.16:0/488708207 <== osd.6 v2:172.21.15.132:6801/727581123 17 ==== osd_op_reply(196 [pgnls start_epoch 119 out=81b] v115'4 uv4 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x563b518ab8c0 con 0x563b518d3800 2023-05-20T01:27:19.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.693+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:197 17.17 17:eff9002a:::100000001f8.00000003:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b52421180 con 0x563b518d3800 2023-05-20T01:27:19.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.693+0000 7fcfbc07c700 1 -- 172.21.15.16:0/488708207 <== osd.6 v2:172.21.15.132:6801/727581123 18 ==== osd_op_reply(197 100000001f8.00000003 [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b518ab8c0 con 0x563b518d3800 2023-05-20T01:27:19.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.693+0000 7fcfbe0ce7c0 10 datascan.operator(): handling object 1099511628280.3 2023-05-20T01:27:19.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.693+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:198 16.19 16:997b7129:::100000001f8.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b51a01880 con 0x563b518d3c00 2023-05-20T01:27:19.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.694+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 <== osd.4 v2:172.21.15.132:6817/506317223 22 ==== osd_op_reply(198 100000001f8.00000000 [call,setxattr (8)] v119'15 uv15 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x563b518ab440 con 0x563b518d3c00 2023-05-20T01:27:19.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.694+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:199 17.f 17:f0000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b51a00a80 con 0x563b518d3000 2023-05-20T01:27:19.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.694+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 <== osd.1 v2:172.21.15.16:6808/3860264220 48 ==== osd_op_reply(199 [pgnls start_epoch 119 out=101b] v115'11 uv11 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x563b518ab440 con 0x563b518d3000 2023-05-20T01:27:19.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.695+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:200 17.f 17:f26fdfa9:::100000001f8.00000002:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b51a34e00 con 0x563b518d3000 2023-05-20T01:27:19.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.695+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 <== osd.1 v2:172.21.15.16:6808/3860264220 49 ==== osd_op_reply(200 100000001f8.00000002 [stat out=16b] v0'0 uv7 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b518ab440 con 0x563b518d3000 2023-05-20T01:27:19.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.695+0000 7fcfbe0ce7c0 10 datascan.operator(): handling object 1099511628280.2 2023-05-20T01:27:19.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.695+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:201 16.19 16:997b7129:::100000001f8.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b51a35180 con 0x563b518d3c00 2023-05-20T01:27:19.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.696+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 <== osd.4 v2:172.21.15.132:6817/506317223 23 ==== osd_op_reply(201 100000001f8.00000000 [call,setxattr (8)] v119'16 uv16 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x563b518ab440 con 0x563b518d3c00 2023-05-20T01:27:19.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.696+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:202 17.f 17:f376f0dc::::head [pgnls start_epoch 119 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b519f8000 con 0x563b518d3000 2023-05-20T01:27:19.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.696+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 <== osd.1 v2:172.21.15.16:6808/3860264220 50 ==== osd_op_reply(202 [pgnls start_epoch 119 out=101b] v115'11 uv11 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x563b518ab440 con 0x563b518d3000 2023-05-20T01:27:19.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.696+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:203 17.f 17:f376f0dc:::100000001fb.00000005:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b519f8a80 con 0x563b518d3000 2023-05-20T01:27:19.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.697+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 <== osd.1 v2:172.21.15.16:6808/3860264220 51 ==== osd_op_reply(203 100000001fb.00000005 [stat out=16b] v0'0 uv10 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b518ab440 con 0x563b518d3000 2023-05-20T01:27:19.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.697+0000 7fcfbe0ce7c0 10 datascan.operator(): handling object 1099511628283.5 2023-05-20T01:27:19.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.697+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:204 16.13 16:ca8fdbba:::100000001fb.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b519f8e00 con 0x563b518d3000 2023-05-20T01:27:19.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.698+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 <== osd.1 v2:172.21.15.16:6808/3860264220 52 ==== osd_op_reply(204 100000001fb.00000000 [call,setxattr (8)] v119'24 uv24 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x563b518ab440 con 0x563b518d3000 2023-05-20T01:27:19.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.698+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:205 17.f 17:f526278d::::head [pgnls start_epoch 119 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b52420700 con 0x563b518d3000 2023-05-20T01:27:19.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.698+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 <== osd.1 v2:172.21.15.16:6808/3860264220 53 ==== osd_op_reply(205 [pgnls start_epoch 119 out=101b] v115'11 uv11 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x563b518ab440 con 0x563b518d3000 2023-05-20T01:27:19.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.698+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:206 17.f 17:f526278d:::10000000001.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b52421500 con 0x563b518d3000 2023-05-20T01:27:19.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.698+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 <== osd.1 v2:172.21.15.16:6808/3860264220 54 ==== osd_op_reply(206 10000000001.00000001 [stat out=16b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b518ab440 con 0x563b518d3000 2023-05-20T01:27:19.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.698+0000 7fcfbe0ce7c0 10 datascan.operator(): handling object 1099511627777.1 2023-05-20T01:27:19.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.698+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:207 16.7 16:e2e07930:::10000000001.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b51a00700 con 0x563b518d3400 2023-05-20T01:27:19.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.699+0000 7fcfbc87d700 1 -- 172.21.15.16:0/488708207 <== osd.3 v2:172.21.15.16:6800/3669781890 25 ==== osd_op_reply(207 10000000001.00000000 [call,setxattr (8)] v119'11 uv11 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x563b518ab680 con 0x563b518d3400 2023-05-20T01:27:19.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.699+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:208 17.f 17:f776b802::::head [pgnls start_epoch 119 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b51a35500 con 0x563b518d3000 2023-05-20T01:27:19.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.699+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 <== osd.1 v2:172.21.15.16:6808/3860264220 55 ==== osd_op_reply(208 [pgnls start_epoch 119 out=81b] v115'11 uv11 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x563b518ab440 con 0x563b518d3000 2023-05-20T01:27:19.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.700+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:209 17.f 17:f776b802:::100000001fb.00000007:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b51a35880 con 0x563b518d3000 2023-05-20T01:27:19.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.700+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 <== osd.1 v2:172.21.15.16:6808/3860264220 56 ==== osd_op_reply(209 100000001fb.00000007 [stat out=16b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b518ab440 con 0x563b518d3000 2023-05-20T01:27:19.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.700+0000 7fcfbe0ce7c0 10 datascan.operator(): handling object 1099511628283.7 2023-05-20T01:27:19.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.700+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:210 16.13 16:ca8fdbba:::100000001fb.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b51a34a80 con 0x563b518d3000 2023-05-20T01:27:19.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.701+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 <== osd.1 v2:172.21.15.16:6808/3860264220 57 ==== osd_op_reply(210 100000001fb.00000000 [call,setxattr (8)] v119'25 uv25 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x563b518ab440 con 0x563b518d3000 2023-05-20T01:27:19.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.701+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:211 17.1f 17:f8000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b51a34e00 con 0x563b518d3800 2023-05-20T01:27:19.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.701+0000 7fcfbc07c700 1 -- 172.21.15.16:0/488708207 <== osd.6 v2:172.21.15.132:6801/727581123 19 ==== osd_op_reply(211 [pgnls start_epoch 119 out=101b] v115'8 uv8 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x563b518ab8c0 con 0x563b518d3800 2023-05-20T01:27:19.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.701+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:212 17.1f 17:faedc8eb:::100000001f6.00000002:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b51a35180 con 0x563b518d3800 2023-05-20T01:27:19.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.702+0000 7fcfbc07c700 1 -- 172.21.15.16:0/488708207 <== osd.6 v2:172.21.15.132:6801/727581123 20 ==== osd_op_reply(212 100000001f6.00000002 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b518ab8c0 con 0x563b518d3800 2023-05-20T01:27:19.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.702+0000 7fcfbe0ce7c0 10 datascan.operator(): handling object 1099511628278.2 2023-05-20T01:27:19.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.702+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:213 16.2a 16:57fe7535:::100000001f6.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b51a01880 con 0x563b518d3000 2023-05-20T01:27:19.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.703+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 <== osd.1 v2:172.21.15.16:6808/3860264220 58 ==== osd_op_reply(213 100000001f6.00000000 [call,setxattr (8)] v119'15 uv15 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x563b518ab440 con 0x563b518d3000 2023-05-20T01:27:19.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.703+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:214 17.1f 17:fb032386::::head [pgnls start_epoch 119 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b51a00a80 con 0x563b518d3800 2023-05-20T01:27:19.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.703+0000 7fcfbc07c700 1 -- 172.21.15.16:0/488708207 <== osd.6 v2:172.21.15.132:6801/727581123 21 ==== osd_op_reply(214 [pgnls start_epoch 119 out=101b] v115'8 uv8 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x563b518ab8c0 con 0x563b518d3800 2023-05-20T01:27:19.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.703+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:215 17.1f 17:fb032386:::100000001fa.00000007:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b519f9c00 con 0x563b518d3800 2023-05-20T01:27:19.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.703+0000 7fcfbc07c700 1 -- 172.21.15.16:0/488708207 <== osd.6 v2:172.21.15.132:6801/727581123 22 ==== osd_op_reply(215 100000001fa.00000007 [stat out=16b] v0'0 uv8 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b518ab8c0 con 0x563b518d3800 2023-05-20T01:27:19.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.703+0000 7fcfbe0ce7c0 10 datascan.operator(): handling object 1099511628282.7 2023-05-20T01:27:19.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.703+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:216 16.3f 16:feb1c803:::100000001fa.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b519f9500 con 0x563b518c8800 2023-05-20T01:27:19.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.704+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 <== osd.0 v2:172.21.15.16:6824/3923634179 33 ==== osd_op_reply(216 100000001fa.00000000 [call,setxattr (8)] v119'17 uv17 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x563b518ab440 con 0x563b518c8800 2023-05-20T01:27:19.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.704+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:217 17.1f 17:feb1c803::::head [pgnls start_epoch 119 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b51a3f500 con 0x563b518d3800 2023-05-20T01:27:19.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.705+0000 7fcfbc07c700 1 -- 172.21.15.16:0/488708207 <== osd.6 v2:172.21.15.132:6801/727581123 23 ==== osd_op_reply(217 [pgnls start_epoch 119 out=81b] v115'8 uv8 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x563b518ab8c0 con 0x563b518d3800 2023-05-20T01:27:19.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.705+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:218 17.1f 17:feb1c803:::100000001fa.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b51a3f880 con 0x563b518d3800 2023-05-20T01:27:19.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.705+0000 7fcfbc07c700 1 -- 172.21.15.16:0/488708207 <== osd.6 v2:172.21.15.132:6801/727581123 24 ==== osd_op_reply(218 100000001fa.00000000 [stat out=16b] v0'0 uv7 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b518ab8c0 con 0x563b518d3800 2023-05-20T01:27:19.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.705+0000 7fcfbe0ce7c0 10 datascan.operator(): handling object 1099511628282.0 2023-05-20T01:27:19.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.705+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:219 16.3f 16:feb1c803:::100000001fa.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x563b51a3fc00 con 0x563b518c8800 2023-05-20T01:27:19.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.706+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 <== osd.0 v2:172.21.15.16:6824/3923634179 34 ==== osd_op_reply(219 100000001fa.00000000 [call,setxattr (8)] v119'18 uv18 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x563b518ab440 con 0x563b518c8800 2023-05-20T01:27:19.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.706+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x563b518c8800 msgr2=0x563b51636100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:19.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.706+0000 7fcfbe0ce7c0 1 --2- 172.21.15.16:0/488708207 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x563b518c8800 0x563b51636100 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.706+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x563b518d3000 msgr2=0x563b519deb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:19.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.706+0000 7fcfbe0ce7c0 1 --2- 172.21.15.16:0/488708207 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x563b518d3000 0x563b519deb00 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.706+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x563b518d2800 msgr2=0x563b519de000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:19.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.706+0000 7fcfbe0ce7c0 1 --2- 172.21.15.16:0/488708207 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x563b518d2800 0x563b519de000 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.706+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x563b518d3400 msgr2=0x563b519df080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:19.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.706+0000 7fcfbe0ce7c0 1 --2- 172.21.15.16:0/488708207 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x563b518d3400 0x563b519df080 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.706+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x563b518d3c00 msgr2=0x563b519dfb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:19.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.706+0000 7fcfbe0ce7c0 1 --2- 172.21.15.16:0/488708207 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x563b518d3c00 0x563b519dfb80 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.706+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x563b51a06000 msgr2=0x563b519e0680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:19.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.706+0000 7fcfbe0ce7c0 1 --2- 172.21.15.16:0/488708207 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x563b51a06000 0x563b519e0680 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.706+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x563b518d3800 msgr2=0x563b519df600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:19.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.706+0000 7fcfbe0ce7c0 1 --2- 172.21.15.16:0/488708207 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x563b518d3800 0x563b519df600 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.706+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 reap_dead start 2023-05-20T01:27:19.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.706+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x563b518d2c00 msgr2=0x563b519de580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:19.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.706+0000 7fcfbe0ce7c0 1 --2- 172.21.15.16:0/488708207 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x563b518d2c00 0x563b519de580 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.706+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 reap_dead start 2023-05-20T01:27:19.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.706+0000 7fcfbb87b700 1 -- 172.21.15.16:0/488708207 reap_dead start 2023-05-20T01:27:19.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.706+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x563b518d2400 msgr2=0x563b51637700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:19.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.706+0000 7fcfbe0ce7c0 1 --2- 172.21.15.16:0/488708207 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x563b518d2400 0x563b51637700 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x563b516fb350 tx=0x563b52333440 comp rx=0 tx=0).stop 2023-05-20T01:27:19.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.706+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b5159a400 msgr2=0x563b51637180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:19.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.706+0000 7fcfbe0ce7c0 1 --2- 172.21.15.16:0/488708207 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b5159a400 0x563b51637180 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x563b518c0300 tx=0x563b517041e0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.706+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 shutdown_connections 2023-05-20T01:27:19.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.706+0000 7fcfbe0ce7c0 1 --2- 172.21.15.16:0/488708207 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x563b518d2400 0x563b51637700 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.706+0000 7fcfbe0ce7c0 1 --2- 172.21.15.16:0/488708207 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b5159a400 0x563b51637180 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.706+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 >> 172.21.15.16:0/488708207 conn(0x563b5159bc00 msgr2=0x563b51678000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:27:19.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.706+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 shutdown_connections 2023-05-20T01:27:19.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.706+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/488708207 wait complete. 2023-05-20T01:27:19.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.707+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/796601313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b5159b000 msgr2=0x563b51634580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:19.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.707+0000 7fcfbe0ce7c0 1 --2- 172.21.15.16:0/796601313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b5159b000 0x563b51634580 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x563b516fae10 tx=0x563b51704f00 comp rx=0 tx=0).stop 2023-05-20T01:27:19.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.707+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/796601313 shutdown_connections 2023-05-20T01:27:19.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.707+0000 7fcfbe0ce7c0 1 --2- 172.21.15.16:0/796601313 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563b5159a800 0x563b51635080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.707+0000 7fcfbe0ce7c0 1 --2- 172.21.15.16:0/796601313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b5159b000 0x563b51634580 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.707+0000 7fcfbe0ce7c0 1 --2- 172.21.15.16:0/796601313 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563b5159ac00 0x563b51634b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.707+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/796601313 >> 172.21.15.16:0/796601313 conn(0x563b5159b400 msgr2=0x563b51677000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:27:19.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.707+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/796601313 shutdown_connections 2023-05-20T01:27:19.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.707+0000 7fcfbe0ce7c0 1 -- 172.21.15.16:0/796601313 wait complete. 2023-05-20T01:27:19.718 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_extents'] in time 0:00:00.219421, result: 2023-05-20T01:27:19.718 DEBUG:teuthology.orchestra.run.smithi016:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_inodes 2023-05-20T01:27:19.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.794+0000 7fccfe5dc7c0 1 -- start start 2023-05-20T01:27:19.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.794+0000 7fccfe5dc7c0 1 Processor -- start 2023-05-20T01:27:19.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.795+0000 7fccfe5dc7c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564e09f12800 0x564e09fac580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:19.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.795+0000 7fccfe5dc7c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x564e09f12c00 0x564e09facb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:19.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.795+0000 7fccfe5dc7c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e09f13000 0x564e09fad080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:19.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.795+0000 7fccfe5dc7c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e09f4f400 con 0x564e09f13000 2023-05-20T01:27:19.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.795+0000 7fccfe5dc7c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e09f4fcc0 con 0x564e09f12800 2023-05-20T01:27:19.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.795+0000 7fccfe5dc7c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x564e09f4fe00 con 0x564e09f12c00 2023-05-20T01:27:19.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.795+0000 7fccfcd8b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e09f13000 0x564e09fad080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:19.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.795+0000 7fccfcd8b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e09f13000 0x564e09fad080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38976/0 (socket says 172.21.15.16:38976) 2023-05-20T01:27:19.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.795+0000 7fccfcd8b700 1 -- 172.21.15.16:0/4046560703 learned_addr learned my addr 172.21.15.16:0/4046560703 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:27:19.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.795+0000 7fccfc58a700 1 --2- 172.21.15.16:0/4046560703 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564e09f12800 0x564e09fac580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:19.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.795+0000 7fccfbd89700 1 --2- 172.21.15.16:0/4046560703 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x564e09f12c00 0x564e09facb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:19.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.795+0000 7fccfcd8b700 1 -- 172.21.15.16:0/4046560703 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x564e09f12c00 msgr2=0x564e09facb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:19.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.795+0000 7fccfcd8b700 1 --2- 172.21.15.16:0/4046560703 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x564e09f12c00 0x564e09facb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.795+0000 7fccfcd8b700 1 -- 172.21.15.16:0/4046560703 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564e09f12800 msgr2=0x564e09fac580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:19.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.795+0000 7fccfcd8b700 1 --2- 172.21.15.16:0/4046560703 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564e09f12800 0x564e09fac580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.795+0000 7fccfcd8b700 1 -- 172.21.15.16:0/4046560703 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564e09f36d00 con 0x564e09f13000 2023-05-20T01:27:19.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.795+0000 7fccfcd8b700 1 --2- 172.21.15.16:0/4046560703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e09f13000 0x564e09fad080 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x564e0acdee10 tx=0x564e0ace8480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b051d8bad9af4e7 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:19.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.796+0000 7fccf9d85700 1 -- 172.21.15.16:0/4046560703 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x564e09f47600 con 0x564e09f13000 2023-05-20T01:27:19.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.796+0000 7fccf9d85700 1 -- 172.21.15.16:0/4046560703 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564e09f64c00 con 0x564e09f13000 2023-05-20T01:27:19.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.796+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/4046560703 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564e09f376c0 con 0x564e09f13000 2023-05-20T01:27:19.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.796+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/4046560703 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x564e09f37520 con 0x564e09f13000 2023-05-20T01:27:19.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.796+0000 7fccfe5dc7c0 4 waiting for MDS map... 2023-05-20T01:27:19.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.796+0000 7fccf9d85700 1 -- 172.21.15.16:0/4046560703 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x564e09f471e0 con 0x564e09f13000 2023-05-20T01:27:19.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.796+0000 7fccf9d85700 1 -- 172.21.15.16:0/4046560703 <== mon.0 v2:172.21.15.16:3300/0 4 ==== osd_map(119..119 src has 1..119) v4 ==== 6987+0+0 (secure 0 0 0) 0x564e09f54c40 con 0x564e09f13000 2023-05-20T01:27:19.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.796+0000 7fccf9d85700 1 -- 172.21.15.16:0/4046560703 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 84) v1 ==== 3126+0+0 (secure 0 0 0) 0x564e0ad8d200 con 0x564e09f13000 2023-05-20T01:27:19.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.796+0000 7fccfe5dc7c0 4 Got MDS map 84 2023-05-20T01:27:19.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.796+0000 7fccfe5dc7c0 4 datascan.main: Using metadata pool output 2023-05-20T01:27:19.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.796+0000 7fccfe5dc7c0 4 datascan.main: connecting to RADOS... 2023-05-20T01:27:19.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.797+0000 7fccfe5dc7c0 1 Processor -- start 2023-05-20T01:27:19.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.797+0000 7fccfe5dc7c0 1 -- start start 2023-05-20T01:27:19.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.797+0000 7fccfe5dc7c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x564e09f13c00 0x564e09fad600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:19.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.797+0000 7fccfe5dc7c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e0a22c000 0x564e09fadb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:19.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.797+0000 7fccfe5dc7c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564e0a22c400 0x564e09fae100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:19.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.797+0000 7fccfe5dc7c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e09f4ea00 con 0x564e0a22c000 2023-05-20T01:27:19.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.797+0000 7fccfe5dc7c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e09f4fcc0 con 0x564e0a22c400 2023-05-20T01:27:19.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.797+0000 7fccfe5dc7c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x564e09f4eb40 con 0x564e09f13c00 2023-05-20T01:27:19.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.798+0000 7fccfc58a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e0a22c000 0x564e09fadb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:19.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.798+0000 7fccfbd89700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x564e09f13c00 0x564e09fad600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:19.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.798+0000 7fccfc58a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e0a22c000 0x564e09fadb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38980/0 (socket says 172.21.15.16:38980) 2023-05-20T01:27:19.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.798+0000 7fccfc58a700 1 -- 172.21.15.16:0/1513570731 learned_addr learned my addr 172.21.15.16:0/1513570731 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:27:19.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.798+0000 7fccfbd89700 1 --2- 172.21.15.16:0/1513570731 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564e0a22c400 0x564e09fae100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:19.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.798+0000 7fccfbd89700 1 -- 172.21.15.16:0/1513570731 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564e0a22c400 msgr2=0x564e09fae100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:19.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.798+0000 7fccfbd89700 1 --2- 172.21.15.16:0/1513570731 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564e0a22c400 0x564e09fae100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.798+0000 7fccfbd89700 1 -- 172.21.15.16:0/1513570731 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e0a22c000 msgr2=0x564e09fadb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:19.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.798+0000 7fccfbd89700 1 --2- 172.21.15.16:0/1513570731 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e0a22c000 0x564e09fadb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.798+0000 7fccfbd89700 1 -- 172.21.15.16:0/1513570731 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564e09f37380 con 0x564e09f13c00 2023-05-20T01:27:19.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.798+0000 7fccfc58a700 1 --2- 172.21.15.16:0/1513570731 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e0a22c000 0x564e09fadb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-05-20T01:27:19.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.798+0000 7fccfbd89700 1 --2- 172.21.15.16:0/1513570731 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x564e09f13c00 0x564e09fad600 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x564e0a238060 tx=0x564e0ada4cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ca0df3c78be973 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:19.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.798+0000 7fccf7d81700 1 -- 172.21.15.16:0/1513570731 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x564e09f47600 con 0x564e09f13c00 2023-05-20T01:27:19.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.798+0000 7fccf7d81700 1 -- 172.21.15.16:0/1513570731 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564e09f65e00 con 0x564e09f13c00 2023-05-20T01:27:19.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.798+0000 7fccf7d81700 1 -- 172.21.15.16:0/1513570731 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x564e09f471e0 con 0x564e09f13c00 2023-05-20T01:27:19.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.798+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/1513570731 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x564e09f13c00 msgr2=0x564e09fad600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:19.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.798+0000 7fccfe5dc7c0 1 --2- 172.21.15.16:0/1513570731 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x564e09f13c00 0x564e09fad600 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x564e0a238060 tx=0x564e0ada4cc0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.799+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/1513570731 shutdown_connections 2023-05-20T01:27:19.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.799+0000 7fccfe5dc7c0 1 --2- 172.21.15.16:0/1513570731 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564e0a22c400 0x564e09fae100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.799+0000 7fccfe5dc7c0 1 --2- 172.21.15.16:0/1513570731 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x564e09f13c00 0x564e09fad600 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.799+0000 7fccfe5dc7c0 1 --2- 172.21.15.16:0/1513570731 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e0a22c000 0x564e09fadb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.799+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/1513570731 >> 172.21.15.16:0/1513570731 conn(0x564e09f13800 msgr2=0x564e09fef800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:27:19.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.799+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/1513570731 shutdown_connections 2023-05-20T01:27:19.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.799+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/1513570731 wait complete. 2023-05-20T01:27:19.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.799+0000 7fccfe5dc7c0 1 Processor -- start 2023-05-20T01:27:19.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.799+0000 7fccfe5dc7c0 1 -- start start 2023-05-20T01:27:19.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.800+0000 7fccfe5dc7c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x564e09f13800 0x564e09faf180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:19.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.800+0000 7fccfe5dc7c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e0a22c000 0x564e09faec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:19.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.800+0000 7fccfe5dc7c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564e0a22c400 0x564e09fae680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:19.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.800+0000 7fccfe5dc7c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e09f4ec80 con 0x564e0a22c000 2023-05-20T01:27:19.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.800+0000 7fccfe5dc7c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e09f4e000 con 0x564e0a22c400 2023-05-20T01:27:19.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.800+0000 7fccfe5dc7c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x564e09f4f180 con 0x564e09f13800 2023-05-20T01:27:19.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.800+0000 7fccfc58a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e0a22c000 0x564e09faec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:19.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.800+0000 7fccfc58a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e0a22c000 0x564e09faec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38986/0 (socket says 172.21.15.16:38986) 2023-05-20T01:27:19.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.800+0000 7fccfc58a700 1 -- 172.21.15.16:0/2853136122 learned_addr learned my addr 172.21.15.16:0/2853136122 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:27:19.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.800+0000 7fccfbd89700 1 --2- 172.21.15.16:0/2853136122 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x564e09f13800 0x564e09faf180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:19.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.800+0000 7fccfbd89700 1 --2- 172.21.15.16:0/2853136122 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564e0a22c400 0x564e09fae680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:19.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.801+0000 7fccfc58a700 1 -- 172.21.15.16:0/2853136122 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x564e09f13800 msgr2=0x564e09faf180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:19.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.801+0000 7fccfc58a700 1 --2- 172.21.15.16:0/2853136122 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x564e09f13800 0x564e09faf180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.801+0000 7fccfc58a700 1 -- 172.21.15.16:0/2853136122 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564e0a22c400 msgr2=0x564e09fae680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:19.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.801+0000 7fccfc58a700 1 --2- 172.21.15.16:0/2853136122 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564e0a22c400 0x564e09fae680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.801+0000 7fccfc58a700 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564e0a250000 con 0x564e0a22c000 2023-05-20T01:27:19.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.801+0000 7fccfc58a700 1 --2- 172.21.15.16:0/2853136122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e0a22c000 0x564e09faec00 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x564e0a239410 tx=0x564e0ace8060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:19.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.801+0000 7fccf657e700 1 -- 172.21.15.16:0/2853136122 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x564e09f47080 con 0x564e0a22c000 2023-05-20T01:27:19.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.801+0000 7fccf657e700 1 -- 172.21.15.16:0/2853136122 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564e0ada0a80 con 0x564e0a22c000 2023-05-20T01:27:19.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.801+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x564e09f37d40 con 0x564e0a22c000 2023-05-20T01:27:19.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.801+0000 7fccf657e700 1 -- 172.21.15.16:0/2853136122 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x564e09f47600 con 0x564e0a22c000 2023-05-20T01:27:19.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.801+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564e0a2501a0 con 0x564e0a22c000 2023-05-20T01:27:19.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.801+0000 7fccfe5dc7c0 4 datascan.init: resolving metadata pool 15 2023-05-20T01:27:19.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.802+0000 7fccf657e700 1 -- 172.21.15.16:0/2853136122 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x564e09f74840 con 0x564e0a22c000 2023-05-20T01:27:19.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.802+0000 7fccf657e700 1 --2- 172.21.15.16:0/2853136122 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x564e0a258400 0x564e09faf700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:19.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.803+0000 7fccf657e700 1 -- 172.21.15.16:0/2853136122 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6987+0+0 (secure 0 0 0) 0x564e09f54a80 con 0x564e0a22c000 2023-05-20T01:27:19.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.803+0000 7fccfbd89700 1 --2- 172.21.15.16:0/2853136122 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x564e0a258400 0x564e09faf700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:19.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.803+0000 7fccfe5dc7c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-05-20T01:27:19.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.803+0000 7fccfe5dc7c0 4 datascan.main: data pool 'cephfs_data' has ID 16 2023-05-20T01:27:19.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.803+0000 7fccfe5dc7c0 4 datascan.main: opening data pool 'cephfs_data' 2023-05-20T01:27:19.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.803+0000 7fccfe5dc7c0 1 --2- 172.21.15.16:0/2853136122 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x564e0a22c800 0x564e09fae100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:19.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.803+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:1 15.14 15:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0ad9c700 con 0x564e0a22c800 2023-05-20T01:27:19.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.803+0000 7fccfbd89700 1 --2- 172.21.15.16:0/2853136122 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x564e0a22c800 0x564e09fae100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:19.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.803+0000 7fccfbd89700 1 --2- 172.21.15.16:0/2853136122 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x564e0a258400 0x564e09faf700 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x564e0a2382d0 tx=0x564e0ada4c00 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:19.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.803+0000 7fccfbd89700 1 --2- 172.21.15.16:0/2853136122 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x564e0a22c800 0x564e09fae100 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:19.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.804+0000 7fccfbd89700 1 -- 172.21.15.16:0/2853136122 <== osd.6 v2:172.21.15.132:6801/727581123 1 ==== osd_op_reply(1 1.00000000.inode [stat out=16b] v0'0 uv48 ondisk = 0) v8 ==== 160+0+16 (crc 0 0 0) 0x564e0a22b440 con 0x564e0a22c800 2023-05-20T01:27:19.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.804+0000 7fccfe5dc7c0 1 --2- 172.21.15.16:0/2853136122 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x564e0a22cc00 0x564e09fadb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:19.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.804+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:2 15.1f 15:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0ad9ca80 con 0x564e0a22cc00 2023-05-20T01:27:19.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.804+0000 7fccfcd8b700 1 --2- 172.21.15.16:0/2853136122 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x564e0a22cc00 0x564e09fadb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:19.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.804+0000 7fccfcd8b700 1 --2- 172.21.15.16:0/2853136122 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x564e0a22cc00 0x564e09fadb80 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:19.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.804+0000 7fccfcd8b700 1 -- 172.21.15.16:0/2853136122 <== osd.3 v2:172.21.15.16:6800/3669781890 1 ==== osd_op_reply(2 100.00000000.inode [stat out=16b] v0'0 uv10 ondisk = 0) v8 ==== 162+0+16 (crc 0 0 0) 0x564e0a22b680 con 0x564e0a22cc00 2023-05-20T01:27:19.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.804+0000 7fccfe5dc7c0 1 --2- 172.21.15.16:0/2853136122 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x564e0a22d000 0x564e09fad600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:19.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.804+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:3 16.0 16:00000000::::head [pgnls-filter start_epoch 119 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0ad9ce00 con 0x564e0a22d000 2023-05-20T01:27:19.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.804+0000 7fccfc58a700 1 --2- 172.21.15.16:0/2853136122 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x564e0a22d000 0x564e09fad600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:19.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.805+0000 7fccfc58a700 1 --2- 172.21.15.16:0/2853136122 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x564e0a22d000 0x564e09fad600 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:19.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.805+0000 7fccfc58a700 1 -- 172.21.15.16:0/2853136122 <== osd.0 v2:172.21.15.16:6824/3923634179 1 ==== osd_op_reply(3 [pgnls-filter start_epoch 119 out=49b] v115'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564e0a22b8c0 con 0x564e0a22d000 2023-05-20T01:27:19.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.805+0000 7fccf7580700 1 --2- 172.21.15.16:0/2853136122 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x564e0a258800 0x564e0a370000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:19.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.805+0000 7fccf7580700 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:4 16.20 16:04000000::::head [pgnls-filter start_epoch 119 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0ad9d180 con 0x564e0a258800 2023-05-20T01:27:19.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.805+0000 7fccfbd89700 1 --2- 172.21.15.16:0/2853136122 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x564e0a258800 0x564e0a370000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:19.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.805+0000 7fccfbd89700 1 --2- 172.21.15.16:0/2853136122 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x564e0a258800 0x564e0a370000 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:19.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.806+0000 7fccfbd89700 1 -- 172.21.15.16:0/2853136122 <== osd.2 v2:172.21.15.16:6816/293187030 1 ==== osd_op_reply(4 [pgnls-filter start_epoch 119 out=49b] v115'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564e0a22b440 con 0x564e0a258800 2023-05-20T01:27:19.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.806+0000 7fccf7d81700 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:5 16.10 16:08000000::::head [pgnls-filter start_epoch 119 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0ad9c700 con 0x564e0a258800 2023-05-20T01:27:19.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.806+0000 7fccfbd89700 1 -- 172.21.15.16:0/2853136122 <== osd.2 v2:172.21.15.16:6816/293187030 2 ==== osd_op_reply(5 [pgnls-filter start_epoch 119 out=49b] v115'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564e0a22b440 con 0x564e0a258800 2023-05-20T01:27:19.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.806+0000 7fccf7580700 1 --2- 172.21.15.16:0/2853136122 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x564e0a258c00 0x564e0a370580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:19.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.806+0000 7fccf7580700 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:6 16.30 16:0c000000::::head [pgnls-filter start_epoch 119 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0ad9d180 con 0x564e0a258c00 2023-05-20T01:27:19.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.806+0000 7fccfcd8b700 1 --2- 172.21.15.16:0/2853136122 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x564e0a258c00 0x564e0a370580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:19.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.807+0000 7fccfcd8b700 1 --2- 172.21.15.16:0/2853136122 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x564e0a258c00 0x564e0a370580 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:19.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.807+0000 7fccfcd8b700 1 -- 172.21.15.16:0/2853136122 <== osd.7 v2:172.21.15.132:6800/466001149 1 ==== osd_op_reply(6 [pgnls-filter start_epoch 119 out=49b] v115'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564e0a22b680 con 0x564e0a258c00 2023-05-20T01:27:19.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.807+0000 7fccf7d81700 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:7 16.8 16:10000000::::head [pgnls-filter start_epoch 119 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0ad9d880 con 0x564e0a22d000 2023-05-20T01:27:19.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.807+0000 7fccfc58a700 1 -- 172.21.15.16:0/2853136122 <== osd.0 v2:172.21.15.16:6824/3923634179 2 ==== osd_op_reply(7 [pgnls-filter start_epoch 119 out=49b] v115'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564e0a22b8c0 con 0x564e0a22d000 2023-05-20T01:27:19.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.807+0000 7fccf7580700 1 --2- 172.21.15.16:0/2853136122 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x564e0a259000 0x564e0a370b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:19.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.807+0000 7fccf7580700 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:8 16.28 16:14000000::::head [pgnls-filter start_epoch 119 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0ad9d500 con 0x564e0a259000 2023-05-20T01:27:19.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.807+0000 7fccfc58a700 1 --2- 172.21.15.16:0/2853136122 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x564e0a259000 0x564e0a370b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:19.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.808+0000 7fccfc58a700 1 --2- 172.21.15.16:0/2853136122 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x564e0a259000 0x564e0a370b00 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:19.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.808+0000 7fccfc58a700 1 -- 172.21.15.16:0/2853136122 <== osd.1 v2:172.21.15.16:6808/3860264220 1 ==== osd_op_reply(8 [pgnls-filter start_epoch 119 out=49b] v115'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564e0a22b8c0 con 0x564e0a259000 2023-05-20T01:27:19.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.808+0000 7fccf7d81700 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:9 16.18 16:18000000::::head [pgnls-filter start_epoch 119 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0ad9dc00 con 0x564e0a22d000 2023-05-20T01:27:19.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.808+0000 7fccfc58a700 1 -- 172.21.15.16:0/2853136122 <== osd.0 v2:172.21.15.16:6824/3923634179 3 ==== osd_op_reply(9 [pgnls-filter start_epoch 119 out=49b] v115'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564e0a22b8c0 con 0x564e0a22d000 2023-05-20T01:27:19.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.808+0000 7fccf7580700 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:10 16.38 16:1c000000::::head [pgnls-filter start_epoch 119 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0ad9ce00 con 0x564e0a258800 2023-05-20T01:27:19.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.808+0000 7fccfbd89700 1 -- 172.21.15.16:0/2853136122 <== osd.2 v2:172.21.15.16:6816/293187030 3 ==== osd_op_reply(10 [pgnls-filter start_epoch 119 out=49b] v115'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564e0a22b440 con 0x564e0a258800 2023-05-20T01:27:19.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.808+0000 7fccf7d81700 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:11 16.4 16:20000000::::head [pgnls-filter start_epoch 119 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0a37c000 con 0x564e0a22cc00 2023-05-20T01:27:19.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.809+0000 7fccfcd8b700 1 -- 172.21.15.16:0/2853136122 <== osd.3 v2:172.21.15.16:6800/3669781890 2 ==== osd_op_reply(11 [pgnls-filter start_epoch 119 out=49b] v115'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564e0a22b680 con 0x564e0a22cc00 2023-05-20T01:27:19.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.809+0000 7fccf7580700 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:12 16.24 16:24000000::::head [pgnls-filter start_epoch 119 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0ad9d880 con 0x564e0a22c800 2023-05-20T01:27:19.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.809+0000 7fccfbd89700 1 -- 172.21.15.16:0/2853136122 <== osd.6 v2:172.21.15.132:6801/727581123 2 ==== osd_op_reply(12 [pgnls-filter start_epoch 119 out=49b] v115'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564e0a22b440 con 0x564e0a22c800 2023-05-20T01:27:19.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.809+0000 7fccf7d81700 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:13 16.14 16:28000000::::head [pgnls-filter start_epoch 119 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0a37c380 con 0x564e0a22d000 2023-05-20T01:27:19.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.809+0000 7fccfc58a700 1 -- 172.21.15.16:0/2853136122 <== osd.0 v2:172.21.15.16:6824/3923634179 4 ==== osd_op_reply(13 [pgnls-filter start_epoch 119 out=49b] v115'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564e0a22b8c0 con 0x564e0a22d000 2023-05-20T01:27:19.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.809+0000 7fccf7580700 1 --2- 172.21.15.16:0/2853136122 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x564e0a259400 0x564e0a371080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:19.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.809+0000 7fccf7580700 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:14 16.34 16:2c000000::::head [pgnls-filter start_epoch 119 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0ad9d500 con 0x564e0a259400 2023-05-20T01:27:19.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.810+0000 7fccfbd89700 1 --2- 172.21.15.16:0/2853136122 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x564e0a259400 0x564e0a371080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:19.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.810+0000 7fccfbd89700 1 --2- 172.21.15.16:0/2853136122 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x564e0a259400 0x564e0a371080 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:19.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.810+0000 7fccfbd89700 1 -- 172.21.15.16:0/2853136122 <== osd.4 v2:172.21.15.132:6817/506317223 1 ==== osd_op_reply(14 [pgnls-filter start_epoch 119 out=49b] v115'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564e0a22b440 con 0x564e0a259400 2023-05-20T01:27:19.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.810+0000 7fccf7d81700 1 --2- 172.21.15.16:0/2853136122 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x564e0a380000 0x564e0a372100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:19.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.810+0000 7fccf7d81700 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:15 16.c 16:30000000::::head [pgnls-filter start_epoch 119 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0a37c700 con 0x564e0a380000 2023-05-20T01:27:19.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.811+0000 7fccfcd8b700 1 --2- 172.21.15.16:0/2853136122 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x564e0a380000 0x564e0a372100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:19.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.811+0000 7fccfcd8b700 1 --2- 172.21.15.16:0/2853136122 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x564e0a380000 0x564e0a372100 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:19.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.811+0000 7fccfcd8b700 1 -- 172.21.15.16:0/2853136122 <== osd.5 v2:172.21.15.132:6816/1844689995 1 ==== osd_op_reply(15 [pgnls-filter start_epoch 119 out=81b] v119'10 uv10 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x564e0a22b680 con 0x564e0a380000 2023-05-20T01:27:19.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.811+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:16 16.0 16:00000000::::head [pgnls-filter start_epoch 119 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0a37c380 con 0x564e0a22d000 2023-05-20T01:27:19.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.812+0000 7fccfc58a700 1 -- 172.21.15.16:0/2853136122 <== osd.0 v2:172.21.15.16:6824/3923634179 5 ==== osd_op_reply(16 [pgnls-filter start_epoch 119 out=49b] v115'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564e0a22b8c0 con 0x564e0a22d000 2023-05-20T01:27:19.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.812+0000 7fccf7d81700 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:17 16.20 16:04000000::::head [pgnls-filter start_epoch 119 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0a37c000 con 0x564e0a258800 2023-05-20T01:27:19.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.812+0000 7fccfbd89700 1 -- 172.21.15.16:0/2853136122 <== osd.2 v2:172.21.15.16:6816/293187030 4 ==== osd_op_reply(17 [pgnls-filter start_epoch 119 out=49b] v115'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564e0a22b440 con 0x564e0a258800 2023-05-20T01:27:19.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.812+0000 7fccf7580700 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:18 16.10 16:08000000::::head [pgnls-filter start_epoch 119 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0ad9dc00 con 0x564e0a258800 2023-05-20T01:27:19.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.812+0000 7fccfbd89700 1 -- 172.21.15.16:0/2853136122 <== osd.2 v2:172.21.15.16:6816/293187030 5 ==== osd_op_reply(18 [pgnls-filter start_epoch 119 out=49b] v115'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564e0a22b440 con 0x564e0a258800 2023-05-20T01:27:19.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.812+0000 7fccf7d81700 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:19 16.30 16:0c000000::::head [pgnls-filter start_epoch 119 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0a37ca80 con 0x564e0a258c00 2023-05-20T01:27:19.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.813+0000 7fccfcd8b700 1 -- 172.21.15.16:0/2853136122 <== osd.7 v2:172.21.15.132:6800/466001149 2 ==== osd_op_reply(19 [pgnls-filter start_epoch 119 out=49b] v115'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564e0a22b680 con 0x564e0a258c00 2023-05-20T01:27:19.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.813+0000 7fccf7580700 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:20 16.8 16:10000000::::head [pgnls-filter start_epoch 119 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0ad9c700 con 0x564e0a22d000 2023-05-20T01:27:19.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.813+0000 7fccfc58a700 1 -- 172.21.15.16:0/2853136122 <== osd.0 v2:172.21.15.16:6824/3923634179 6 ==== osd_op_reply(20 [pgnls-filter start_epoch 119 out=49b] v115'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564e0a22b8c0 con 0x564e0a22d000 2023-05-20T01:27:19.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.813+0000 7fccf7d81700 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:21 16.28 16:14000000::::head [pgnls-filter start_epoch 119 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0a37ce00 con 0x564e0a259000 2023-05-20T01:27:19.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.813+0000 7fccfc58a700 1 -- 172.21.15.16:0/2853136122 <== osd.1 v2:172.21.15.16:6808/3860264220 2 ==== osd_op_reply(21 [pgnls-filter start_epoch 119 out=49b] v115'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564e0a22b8c0 con 0x564e0a259000 2023-05-20T01:27:19.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.813+0000 7fccf7580700 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:22 16.18 16:18000000::::head [pgnls-filter start_epoch 119 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0ad9ce00 con 0x564e0a22d000 2023-05-20T01:27:19.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.813+0000 7fccfc58a700 1 -- 172.21.15.16:0/2853136122 <== osd.0 v2:172.21.15.16:6824/3923634179 7 ==== osd_op_reply(22 [pgnls-filter start_epoch 119 out=49b] v115'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564e0a22b8c0 con 0x564e0a22d000 2023-05-20T01:27:19.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.813+0000 7fccf7d81700 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:23 16.38 16:1c000000::::head [pgnls-filter start_epoch 119 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0a37d180 con 0x564e0a258800 2023-05-20T01:27:19.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.814+0000 7fccfbd89700 1 -- 172.21.15.16:0/2853136122 <== osd.2 v2:172.21.15.16:6816/293187030 6 ==== osd_op_reply(23 [pgnls-filter start_epoch 119 out=49b] v115'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564e0a22b440 con 0x564e0a258800 2023-05-20T01:27:19.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.814+0000 7fccf7580700 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:24 16.4 16:20000000::::head [pgnls-filter start_epoch 119 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0a37c000 con 0x564e0a22cc00 2023-05-20T01:27:19.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.814+0000 7fccfcd8b700 1 -- 172.21.15.16:0/2853136122 <== osd.3 v2:172.21.15.16:6800/3669781890 3 ==== osd_op_reply(24 [pgnls-filter start_epoch 119 out=49b] v115'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564e0a22b680 con 0x564e0a22cc00 2023-05-20T01:27:19.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.814+0000 7fccf7d81700 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:25 16.24 16:24000000::::head [pgnls-filter start_epoch 119 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0a37d180 con 0x564e0a22c800 2023-05-20T01:27:19.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.814+0000 7fccfbd89700 1 -- 172.21.15.16:0/2853136122 <== osd.6 v2:172.21.15.132:6801/727581123 3 ==== osd_op_reply(25 [pgnls-filter start_epoch 119 out=49b] v115'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564e0a22b440 con 0x564e0a22c800 2023-05-20T01:27:19.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.814+0000 7fccf7580700 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:26 16.14 16:28000000::::head [pgnls-filter start_epoch 119 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0ad9d500 con 0x564e0a22d000 2023-05-20T01:27:19.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.814+0000 7fccfc58a700 1 -- 172.21.15.16:0/2853136122 <== osd.0 v2:172.21.15.16:6824/3923634179 8 ==== osd_op_reply(26 [pgnls-filter start_epoch 119 out=49b] v115'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564e0a22b8c0 con 0x564e0a22d000 2023-05-20T01:27:19.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.815+0000 7fccf7d81700 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:27 16.34 16:2c000000::::head [pgnls-filter start_epoch 119 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0a37c380 con 0x564e0a259400 2023-05-20T01:27:19.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.815+0000 7fccfbd89700 1 -- 172.21.15.16:0/2853136122 <== osd.4 v2:172.21.15.132:6817/506317223 2 ==== osd_op_reply(27 [pgnls-filter start_epoch 119 out=49b] v115'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564e0a22b440 con 0x564e0a259400 2023-05-20T01:27:19.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.815+0000 7fccf7580700 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:28 16.c 16:30000000::::head [pgnls-filter start_epoch 119 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0ad9dc00 con 0x564e0a380000 2023-05-20T01:27:19.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.815+0000 7fccfcd8b700 1 -- 172.21.15.16:0/2853136122 <== osd.5 v2:172.21.15.132:6816/1844689995 2 ==== osd_op_reply(28 [pgnls-filter start_epoch 119 out=81b] v119'10 uv10 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x564e0a22b680 con 0x564e0a380000 2023-05-20T01:27:19.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.815+0000 7fccfe5dc7c0 20 datascan.forall_objects: OSD matched oid 100000001fc.00000000 2023-05-20T01:27:19.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.815+0000 7fccfe5dc7c0 10 datascan.operator(): handling object 100000001fc.0 2023-05-20T01:27:19.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.815+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:29 16.c 16:30604137:::100000001fc.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0ad9d880 con 0x564e0a380000 2023-05-20T01:27:19.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.816+0000 7fccfcd8b700 1 -- 172.21.15.16:0/2853136122 <== osd.5 v2:172.21.15.132:6816/1844689995 3 ==== osd_op_reply(29 100000001fc.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr,getxattr] v0'0 uv10 ondisk = 0) v8 ==== 416+0+46 (crc 0 0 0) 0x564e0a22b680 con 0x564e0a380000 2023-05-20T01:27:19.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.816+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:30 17.c 17:30604137:::100000001fc.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0ad9d180 con 0x564e0a258c00 2023-05-20T01:27:19.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.816+0000 7fccfcd8b700 1 -- 172.21.15.16:0/2853136122 <== osd.7 v2:172.21.15.132:6800/466001149 3 ==== osd_op_reply(30 100000001fc.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x564e0a22b680 con 0x564e0a258c00 2023-05-20T01:27:19.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.816+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:31 15.1f 15:ff5b34d6:::1.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0a37c700 con 0x564e0a22cc00 2023-05-20T01:27:19.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.816+0000 7fccfcd8b700 1 -- 172.21.15.16:0/2853136122 <== osd.3 v2:172.21.15.16:6800/3669781890 4 ==== osd_op_reply(31 1.00000000 [omap-get-header] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x564e0a22b680 con 0x564e0a22cc00 2023-05-20T01:27:19.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.816+0000 7fccfe5dc7c0 10 datascan.find_or_create_dirfrag: read_version = 0 2023-05-20T01:27:19.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.816+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:32 15.1f 15:ff5b34d6:::1.00000000:head [create,omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0a37ca80 con 0x564e0a22cc00 2023-05-20T01:27:19.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.817+0000 7fccfcd8b700 1 -- 172.21.15.16:0/2853136122 <== osd.3 v2:172.21.15.16:6800/3669781890 5 ==== osd_op_reply(32 1.00000000 [create,omap-set-header] v119'11 uv11 ondisk = 0) v8 ==== 196+0+0 (crc 0 0 0) 0x564e0a22b680 con 0x564e0a22cc00 2023-05-20T01:27:19.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.817+0000 7fccfe5dc7c0 10 datascan.find_or_create_dirfrag: Created dirfrag: 0x0x1 2023-05-20T01:27:19.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.817+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:33 15.1f 15:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=23b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0a37c000 con 0x564e0a22cc00 2023-05-20T01:27:19.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.817+0000 7fccfcd8b700 1 -- 172.21.15.16:0/2853136122 <== osd.3 v2:172.21.15.16:6800/3669781890 6 ==== osd_op_reply(33 1.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv11 ondisk = 0) v8 ==== 154+0+4 (crc 0 0 0) 0x564e0a22b680 con 0x564e0a22cc00 2023-05-20T01:27:19.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.817+0000 7fccfe5dc7c0 20 datascan.read_dentry: oid=1.00000000 dname=lost+found frag=*, r=0 2023-05-20T01:27:19.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.817+0000 7fccfe5dc7c0 20 datascan.read_dentry: lost+found_head not found in result 2023-05-20T01:27:19.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.818+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:34 15.1f 15:ff5b34d6:::1.00000000:head [omap-set-vals in=511b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0a37dc00 con 0x564e0a22cc00 2023-05-20T01:27:19.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.819+0000 7fccfcd8b700 1 -- 172.21.15.16:0/2853136122 <== osd.3 v2:172.21.15.16:6800/3669781890 7 ==== osd_op_reply(34 1.00000000 [omap-set-vals] v119'12 uv12 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x564e0a22b680 con 0x564e0a22cc00 2023-05-20T01:27:19.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.819+0000 7fccfe5dc7c0 20 datascan.inject_linkage: Injected dentry 0x0x1/lost+found pointing to 0x0x4 2023-05-20T01:27:19.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.819+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:35 15.3 15:c0ce80d4:::4.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0ad9dc00 con 0x564e0a259400 2023-05-20T01:27:19.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.819+0000 7fccfbd89700 1 -- 172.21.15.16:0/2853136122 <== osd.4 v2:172.21.15.132:6817/506317223 3 ==== osd_op_reply(35 4.00000000 [omap-get-header] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x564e0a22b440 con 0x564e0a259400 2023-05-20T01:27:19.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.819+0000 7fccfe5dc7c0 10 datascan.find_or_create_dirfrag: read_version = 0 2023-05-20T01:27:19.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.819+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:36 15.3 15:c0ce80d4:::4.00000000:head [create,omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0a37c700 con 0x564e0a259400 2023-05-20T01:27:19.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.820+0000 7fccfbd89700 1 -- 172.21.15.16:0/2853136122 <== osd.4 v2:172.21.15.132:6817/506317223 4 ==== osd_op_reply(36 4.00000000 [create,omap-set-header] v119'1 uv1 ondisk = 0) v8 ==== 196+0+0 (crc 0 0 0) 0x564e0a22b440 con 0x564e0a259400 2023-05-20T01:27:19.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.820+0000 7fccfe5dc7c0 10 datascan.find_or_create_dirfrag: Created dirfrag: 0x0x4 2023-05-20T01:27:19.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.820+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:37 15.3 15:c0ce80d4:::4.00000000:head [omap-set-vals in=512b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0a37ca80 con 0x564e0a259400 2023-05-20T01:27:19.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.821+0000 7fccfbd89700 1 -- 172.21.15.16:0/2853136122 <== osd.4 v2:172.21.15.132:6817/506317223 5 ==== osd_op_reply(37 4.00000000 [omap-set-vals] v119'2 uv2 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x564e0a22b440 con 0x564e0a259400 2023-05-20T01:27:19.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.821+0000 7fccfe5dc7c0 20 datascan.inject_linkage: Injected dentry 0x0x4/100000001fc pointing to 0x0x100000001fc 2023-05-20T01:27:19.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.821+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:38 16.2c 16:34000000::::head [pgnls-filter start_epoch 119 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0ad9ca80 con 0x564e0a259400 2023-05-20T01:27:19.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.821+0000 7fccfbd89700 1 -- 172.21.15.16:0/2853136122 <== osd.4 v2:172.21.15.132:6817/506317223 6 ==== osd_op_reply(38 [pgnls-filter start_epoch 119 out=49b] v115'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564e0a22b440 con 0x564e0a259400 2023-05-20T01:27:19.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.821+0000 7fccf7580700 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:39 16.1c 16:38000000::::head [pgnls-filter start_epoch 119 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0ad9c700 con 0x564e0a258800 2023-05-20T01:27:19.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.822+0000 7fccfbd89700 1 -- 172.21.15.16:0/2853136122 <== osd.2 v2:172.21.15.16:6816/293187030 7 ==== osd_op_reply(39 [pgnls-filter start_epoch 119 out=49b] v115'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564e0a22b440 con 0x564e0a258800 2023-05-20T01:27:19.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.822+0000 7fccf7d81700 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:40 16.3c 16:3c000000::::head [pgnls-filter start_epoch 119 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0a37ce00 con 0x564e0a22cc00 2023-05-20T01:27:19.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.822+0000 7fccfcd8b700 1 -- 172.21.15.16:0/2853136122 <== osd.3 v2:172.21.15.16:6800/3669781890 8 ==== osd_op_reply(40 [pgnls-filter start_epoch 119 out=49b] v115'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564e0a22b680 con 0x564e0a22cc00 2023-05-20T01:27:19.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.822+0000 7fccf7580700 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:41 16.2 16:40000000::::head [pgnls-filter start_epoch 119 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0ad9ce00 con 0x564e0a259400 2023-05-20T01:27:19.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.822+0000 7fccfbd89700 1 -- 172.21.15.16:0/2853136122 <== osd.4 v2:172.21.15.132:6817/506317223 7 ==== osd_op_reply(41 [pgnls-filter start_epoch 119 out=49b] v115'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564e0a22b440 con 0x564e0a259400 2023-05-20T01:27:19.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.822+0000 7fccf7d81700 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:42 16.22 16:44000000::::head [pgnls-filter start_epoch 119 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0a37d500 con 0x564e0a259400 2023-05-20T01:27:19.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.823+0000 7fccfbd89700 1 -- 172.21.15.16:0/2853136122 <== osd.4 v2:172.21.15.132:6817/506317223 8 ==== osd_op_reply(42 [pgnls-filter start_epoch 119 out=49b] v115'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564e0a22b440 con 0x564e0a259400 2023-05-20T01:27:19.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.823+0000 7fccf7580700 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:43 16.12 16:48000000::::head [pgnls-filter start_epoch 119 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0a37d180 con 0x564e0a22c800 2023-05-20T01:27:19.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.823+0000 7fccfbd89700 1 -- 172.21.15.16:0/2853136122 <== osd.6 v2:172.21.15.132:6801/727581123 4 ==== osd_op_reply(43 [pgnls-filter start_epoch 119 out=49b] v115'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564e0a22b440 con 0x564e0a22c800 2023-05-20T01:27:19.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.823+0000 7fccf7d81700 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:44 16.32 16:4c000000::::head [pgnls-filter start_epoch 119 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0a37d880 con 0x564e0a258c00 2023-05-20T01:27:19.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.823+0000 7fccfcd8b700 1 -- 172.21.15.16:0/2853136122 <== osd.7 v2:172.21.15.132:6800/466001149 4 ==== osd_op_reply(44 [pgnls-filter start_epoch 119 out=49b] v115'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564e0a22b680 con 0x564e0a258c00 2023-05-20T01:27:19.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.823+0000 7fccf7580700 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:45 16.a 16:50000000::::head [pgnls-filter start_epoch 119 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0a37c380 con 0x564e0a258800 2023-05-20T01:27:19.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.823+0000 7fccfbd89700 1 -- 172.21.15.16:0/2853136122 <== osd.2 v2:172.21.15.16:6816/293187030 8 ==== osd_op_reply(45 [pgnls-filter start_epoch 119 out=49b] v115'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564e0a22b440 con 0x564e0a258800 2023-05-20T01:27:19.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.823+0000 7fccf7d81700 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:46 16.2a 16:54000000::::head [pgnls-filter start_epoch 119 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0a37d500 con 0x564e0a259000 2023-05-20T01:27:19.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.824+0000 7fccfc58a700 1 -- 172.21.15.16:0/2853136122 <== osd.1 v2:172.21.15.16:6808/3860264220 3 ==== osd_op_reply(46 [pgnls-filter start_epoch 119 out=81b] v119'15 uv15 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x564e0a22b8c0 con 0x564e0a259000 2023-05-20T01:27:19.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.824+0000 7fccfe5dc7c0 20 datascan.forall_objects: OSD matched oid 100000001f6.00000000 2023-05-20T01:27:19.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.824+0000 7fccfe5dc7c0 10 datascan.operator(): handling object 100000001f6.0 2023-05-20T01:27:19.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.824+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:47 16.2a 16:57fe7535:::100000001f6.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0ad9d880 con 0x564e0a259000 2023-05-20T01:27:19.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.824+0000 7fccfc58a700 1 -- 172.21.15.16:0/2853136122 <== osd.1 v2:172.21.15.16:6808/3860264220 4 ==== osd_op_reply(47 100000001f6.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (111) out=111b,getxattr,getxattr] v0'0 uv15 ondisk = 0) v8 ==== 416+0+157 (crc 0 0 0) 0x564e0a22b8c0 con 0x564e0a259000 2023-05-20T01:27:19.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.824+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:48 17.a 17:57fe7535:::100000001f6.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0a392000 con 0x564e0a258c00 2023-05-20T01:27:19.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.825+0000 7fccfcd8b700 1 -- 172.21.15.16:0/2853136122 <== osd.7 v2:172.21.15.132:6800/466001149 5 ==== osd_op_reply(48 100000001f6.00000000 [getxattr (30) out=30b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x564e0a22b680 con 0x564e0a258c00 2023-05-20T01:27:19.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.825+0000 7fccfe5dc7c0 20 datascan.operator(): loaded layout from xattr: pi: 17 os: 262144 sc: 4 su: 65536 2023-05-20T01:27:19.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.825+0000 7fccfe5dc7c0 10 datascan.operator(): calculating striped size from complete objs: 0, partial objs: 4 2023-05-20T01:27:19.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.825+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:49 17.a 17:57fe7535:::100000001f6.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0a392380 con 0x564e0a258c00 2023-05-20T01:27:19.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.825+0000 7fccfcd8b700 1 -- 172.21.15.16:0/2853136122 <== osd.7 v2:172.21.15.132:6800/466001149 6 ==== osd_op_reply(49 100000001f6.00000000 [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x564e0a22b680 con 0x564e0a258c00 2023-05-20T01:27:19.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.825+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:50 17.1c 17:3d6db04b:::100000001f6.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0a392700 con 0x564e0a259000 2023-05-20T01:27:19.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.825+0000 7fccfc58a700 1 -- 172.21.15.16:0/2853136122 <== osd.1 v2:172.21.15.16:6808/3860264220 5 ==== osd_op_reply(50 100000001f6.00000001 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x564e0a22b8c0 con 0x564e0a259000 2023-05-20T01:27:19.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.825+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:51 17.1f 17:faedc8eb:::100000001f6.00000002:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0a392a80 con 0x564e0a22c800 2023-05-20T01:27:19.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.825+0000 7fccfbd89700 1 -- 172.21.15.16:0/2853136122 <== osd.6 v2:172.21.15.132:6801/727581123 5 ==== osd_op_reply(51 100000001f6.00000002 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x564e0a22b440 con 0x564e0a22c800 2023-05-20T01:27:19.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.825+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:52 17.e 17:728aa123:::100000001f6.00000003:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0a392e00 con 0x564e0a258800 2023-05-20T01:27:19.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.826+0000 7fccfbd89700 1 -- 172.21.15.16:0/2853136122 <== osd.2 v2:172.21.15.16:6816/293187030 9 ==== osd_op_reply(52 100000001f6.00000003 [stat out=16b] v0'0 uv6 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x564e0a22b440 con 0x564e0a258800 2023-05-20T01:27:19.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.826+0000 7fccfe5dc7c0 10 datascan.inject_with_backtrace: inode: 0x0x100000001f6 2023-05-20T01:27:19.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.826+0000 7fccfe5dc7c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/flushed_file_1 2023-05-20T01:27:19.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.826+0000 7fccfe5dc7c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=flushed_file_1 2023-05-20T01:27:19.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.826+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:53 15.10 15:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0a393180 con 0x564e0a259400 2023-05-20T01:27:19.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.826+0000 7fccfbd89700 1 -- 172.21.15.16:0/2853136122 <== osd.4 v2:172.21.15.132:6817/506317223 9 ==== osd_op_reply(53 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 164+0+0 (crc 0 0 0) 0x564e0a22b440 con 0x564e0a259400 2023-05-20T01:27:19.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.826+0000 7fccfe5dc7c0 4 datascan.get_frag_of: Unexpected error on '10000000000.00000000': (2) No such file or directory 2023-05-20T01:27:19.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.826+0000 7fccfe5dc7c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/flushed_file_1, will insert to root 2023-05-20T01:27:19.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.826+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:54 15.10 15:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0a393500 con 0x564e0a259400 2023-05-20T01:27:19.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.826+0000 7fccfbd89700 1 -- 172.21.15.16:0/2853136122 <== osd.4 v2:172.21.15.132:6817/506317223 10 ==== osd_op_reply(54 10000000000.00000000 [omap-get-header] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 164+0+0 (crc 0 0 0) 0x564e0a22b440 con 0x564e0a259400 2023-05-20T01:27:19.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.826+0000 7fccfe5dc7c0 10 datascan.find_or_create_dirfrag: read_version = 0 2023-05-20T01:27:19.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.826+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:55 15.10 15:0cf6ad0f:::10000000000.00000000:head [create,omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0a37d500 con 0x564e0a259400 2023-05-20T01:27:19.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.827+0000 7fccfbd89700 1 -- 172.21.15.16:0/2853136122 <== osd.4 v2:172.21.15.132:6817/506317223 11 ==== osd_op_reply(55 10000000000.00000000 [create,omap-set-header] v119'16 uv16 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x564e0a22b440 con 0x564e0a259400 2023-05-20T01:27:19.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.827+0000 7fccfe5dc7c0 10 datascan.find_or_create_dirfrag: Created dirfrag: 0x0x10000000000 2023-05-20T01:27:19.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.827+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:56 15.10 15:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=27b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0a37c000 con 0x564e0a259400 2023-05-20T01:27:19.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.828+0000 7fccfbd89700 1 -- 172.21.15.16:0/2853136122 <== osd.4 v2:172.21.15.132:6817/506317223 12 ==== osd_op_reply(56 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv16 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x564e0a22b440 con 0x564e0a259400 2023-05-20T01:27:19.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.828+0000 7fccfe5dc7c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=flushed_file_1 frag=*, r=0 2023-05-20T01:27:19.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.828+0000 7fccfe5dc7c0 20 datascan.read_dentry: flushed_file_1_head not found in result 2023-05-20T01:27:19.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.828+0000 7fccfe5dc7c0 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001f6 at 0x0x10000000000/flushed_file_1 with size=524288 bytes 2023-05-20T01:27:19.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.828+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:57 15.10 15:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=515b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0a37dc00 con 0x564e0a259400 2023-05-20T01:27:19.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.829+0000 7fccfbd89700 1 -- 172.21.15.16:0/2853136122 <== osd.4 v2:172.21.15.132:6817/506317223 13 ==== osd_op_reply(57 10000000000.00000000 [omap-set-vals] v119'17 uv17 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564e0a22b440 con 0x564e0a259400 2023-05-20T01:27:19.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.829+0000 7fccfe5dc7c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/flushed_file_1 pointing to 0x0x100000001f6 2023-05-20T01:27:19.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.829+0000 7fccfe5dc7c0 10 datascan.inject_with_backtrace: backptr: 0x0x1/stripey 2023-05-20T01:27:19.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.829+0000 7fccfe5dc7c0 20 datascan.get_frag_of: dirino=0x1 target_dname=stripey 2023-05-20T01:27:19.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.829+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:58 15.1f 15:ff5b34d6:::1.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0a37ce00 con 0x564e0a22cc00 2023-05-20T01:27:19.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.829+0000 7fccfcd8b700 1 -- 172.21.15.16:0/2853136122 <== osd.3 v2:172.21.15.16:6800/3669781890 9 ==== osd_op_reply(58 1.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 154+0+0 (crc 0 0 0) 0x564e0a22b680 con 0x564e0a22cc00 2023-05-20T01:27:19.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.829+0000 7fccfe5dc7c0 10 datascan.get_frag_of: No backtrace on '1.00000000' 2023-05-20T01:27:19.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.829+0000 7fccfe5dc7c0 4 datascan.get_frag_of: No backtrace on '1.00000000': cannot determine fragtree 2023-05-20T01:27:19.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.829+0000 7fccfe5dc7c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x1/stripey, will insert to root 2023-05-20T01:27:19.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.829+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:59 15.1f 15:ff5b34d6:::1.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0a37d880 con 0x564e0a22cc00 2023-05-20T01:27:19.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.829+0000 7fccfcd8b700 1 -- 172.21.15.16:0/2853136122 <== osd.3 v2:172.21.15.16:6800/3669781890 10 ==== osd_op_reply(59 1.00000000 [omap-get-header out=274b] v0'0 uv12 ondisk = 0) v8 ==== 154+0+274 (crc 0 0 0) 0x564e0a22b680 con 0x564e0a22cc00 2023-05-20T01:27:19.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.829+0000 7fccfe5dc7c0 10 datascan.find_or_create_dirfrag: read_version = 12 2023-05-20T01:27:19.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.829+0000 7fccfe5dc7c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x1 * 2023-05-20T01:27:19.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.829+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:60 15.1f 15:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0ad9d500 con 0x564e0a22cc00 2023-05-20T01:27:19.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.829+0000 7fccfcd8b700 1 -- 172.21.15.16:0/2853136122 <== osd.3 v2:172.21.15.16:6800/3669781890 11 ==== osd_op_reply(60 1.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv12 ondisk = 0) v8 ==== 154+0+4 (crc 0 0 0) 0x564e0a22b680 con 0x564e0a22cc00 2023-05-20T01:27:19.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.829+0000 7fccfe5dc7c0 20 datascan.read_dentry: oid=1.00000000 dname=stripey frag=*, r=0 2023-05-20T01:27:19.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.829+0000 7fccfe5dc7c0 20 datascan.read_dentry: stripey_head not found in result 2023-05-20T01:27:19.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.829+0000 7fccfe5dc7c0 10 datascan.inject_with_backtrace: Linking ancestor directory of inode 0x0x10000000000 at 0x0x1:stripey 2023-05-20T01:27:19.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.829+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:61 15.1f 15:ff5b34d6:::1.00000000:head [omap-set-vals in=508b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0ad9d180 con 0x564e0a22cc00 2023-05-20T01:27:19.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.830+0000 7fccfcd8b700 1 -- 172.21.15.16:0/2853136122 <== osd.3 v2:172.21.15.16:6800/3669781890 12 ==== osd_op_reply(61 1.00000000 [omap-set-vals] v119'13 uv13 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x564e0a22b680 con 0x564e0a22cc00 2023-05-20T01:27:19.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.830+0000 7fccfe5dc7c0 20 datascan.inject_linkage: Injected dentry 0x0x1/stripey pointing to 0x0x10000000000 2023-05-20T01:27:19.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.830+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:62 16.1a 16:58000000::::head [pgnls-filter start_epoch 119 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0ad9d880 con 0x564e0a22cc00 2023-05-20T01:27:19.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.831+0000 7fccfcd8b700 1 -- 172.21.15.16:0/2853136122 <== osd.3 v2:172.21.15.16:6800/3669781890 13 ==== osd_op_reply(62 [pgnls-filter start_epoch 119 out=49b] v115'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564e0a22b680 con 0x564e0a22cc00 2023-05-20T01:27:19.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.831+0000 7fccf7d81700 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:63 16.3a 16:5c000000::::head [pgnls-filter start_epoch 119 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0a37d180 con 0x564e0a22c800 2023-05-20T01:27:19.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.831+0000 7fccfbd89700 1 -- 172.21.15.16:0/2853136122 <== osd.6 v2:172.21.15.132:6801/727581123 6 ==== osd_op_reply(63 [pgnls-filter start_epoch 119 out=49b] v115'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564e0a22b440 con 0x564e0a22c800 2023-05-20T01:27:19.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.831+0000 7fccf7580700 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:64 16.6 16:60000000::::head [pgnls-filter start_epoch 119 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0a37c700 con 0x564e0a22c800 2023-05-20T01:27:19.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.831+0000 7fccfbd89700 1 -- 172.21.15.16:0/2853136122 <== osd.6 v2:172.21.15.132:6801/727581123 7 ==== osd_op_reply(64 [pgnls-filter start_epoch 119 out=49b] v115'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564e0a22b440 con 0x564e0a22c800 2023-05-20T01:27:19.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.831+0000 7fccf7d81700 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:65 16.26 16:64000000::::head [pgnls-filter start_epoch 119 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0a37c380 con 0x564e0a259400 2023-05-20T01:27:19.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.832+0000 7fccfbd89700 1 -- 172.21.15.16:0/2853136122 <== osd.4 v2:172.21.15.132:6817/506317223 14 ==== osd_op_reply(65 [pgnls-filter start_epoch 119 out=49b] v115'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564e0a22b440 con 0x564e0a259400 2023-05-20T01:27:19.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.832+0000 7fccf7580700 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:66 16.16 16:68000000::::head [pgnls-filter start_epoch 119 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0a37ca80 con 0x564e0a22cc00 2023-05-20T01:27:19.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.832+0000 7fccfcd8b700 1 -- 172.21.15.16:0/2853136122 <== osd.3 v2:172.21.15.16:6800/3669781890 14 ==== osd_op_reply(66 [pgnls-filter start_epoch 119 out=49b] v115'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564e0a22b680 con 0x564e0a22cc00 2023-05-20T01:27:19.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.832+0000 7fccf7d81700 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:67 16.36 16:6c000000::::head [pgnls-filter start_epoch 119 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0ad9ca80 con 0x564e0a259000 2023-05-20T01:27:19.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.832+0000 7fccfc58a700 1 -- 172.21.15.16:0/2853136122 <== osd.1 v2:172.21.15.16:6808/3860264220 6 ==== osd_op_reply(67 [pgnls-filter start_epoch 119 out=101b] v119'31 uv31 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x564e0a22b8c0 con 0x564e0a259000 2023-05-20T01:27:19.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.832+0000 7fccfe5dc7c0 20 datascan.forall_objects: OSD matched oid 100000001f9.00000000 2023-05-20T01:27:19.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.832+0000 7fccfe5dc7c0 10 datascan.operator(): handling object 100000001f9.0 2023-05-20T01:27:19.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.832+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:68 16.36 16:6ee5e2c4:::100000001f9.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0a37d500 con 0x564e0a259000 2023-05-20T01:27:19.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.832+0000 7fccfc58a700 1 -- 172.21.15.16:0/2853136122 <== osd.1 v2:172.21.15.16:6808/3860264220 7 ==== osd_op_reply(68 100000001f9.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (111) out=111b,getxattr,getxattr] v0'0 uv31 ondisk = 0) v8 ==== 416+0+157 (crc 0 0 0) 0x564e0a22b8c0 con 0x564e0a259000 2023-05-20T01:27:19.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.832+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:69 17.16 17:6ee5e2c4:::100000001f9.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0a37dc00 con 0x564e0a380000 2023-05-20T01:27:19.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.833+0000 7fccfcd8b700 1 -- 172.21.15.16:0/2853136122 <== osd.5 v2:172.21.15.132:6816/1844689995 4 ==== osd_op_reply(69 100000001f9.00000000 [getxattr (30) out=30b] v0'0 uv10 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x564e0a22b680 con 0x564e0a380000 2023-05-20T01:27:19.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.833+0000 7fccfe5dc7c0 20 datascan.operator(): loaded layout from xattr: pi: 17 os: 262144 sc: 4 su: 65536 2023-05-20T01:27:19.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.833+0000 7fccfe5dc7c0 10 datascan.operator(): calculating striped size from complete objs: 4, partial objs: 4 2023-05-20T01:27:19.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.833+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:70 17.7 17:e7ecf421:::100000001f9.00000004:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0a37ce00 con 0x564e0a22d000 2023-05-20T01:27:19.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.833+0000 7fccfc58a700 1 -- 172.21.15.16:0/2853136122 <== osd.0 v2:172.21.15.16:6824/3923634179 9 ==== osd_op_reply(70 100000001f9.00000004 [stat out=16b] v0'0 uv8 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x564e0a22b8c0 con 0x564e0a22d000 2023-05-20T01:27:19.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.833+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:71 17.2 17:46f89f8a:::100000001f9.00000005:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0a37d880 con 0x564e0a259400 2023-05-20T01:27:19.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.833+0000 7fccfbd89700 1 -- 172.21.15.16:0/2853136122 <== osd.4 v2:172.21.15.132:6817/506317223 15 ==== osd_op_reply(71 100000001f9.00000005 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x564e0a22b440 con 0x564e0a259400 2023-05-20T01:27:19.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.833+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:72 17.16 17:6df493a7:::100000001f9.00000006:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0a37ca80 con 0x564e0a380000 2023-05-20T01:27:19.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.834+0000 7fccfcd8b700 1 -- 172.21.15.16:0/2853136122 <== osd.5 v2:172.21.15.132:6816/1844689995 5 ==== osd_op_reply(72 100000001f9.00000006 [stat out=16b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x564e0a22b680 con 0x564e0a380000 2023-05-20T01:27:19.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.834+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:73 17.17 17:ee7e3069:::100000001f9.00000007:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0a37dc00 con 0x564e0a22c800 2023-05-20T01:27:19.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.834+0000 7fccfbd89700 1 -- 172.21.15.16:0/2853136122 <== osd.6 v2:172.21.15.132:6801/727581123 8 ==== osd_op_reply(73 100000001f9.00000007 [stat out=16b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x564e0a22b440 con 0x564e0a22c800 2023-05-20T01:27:19.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.834+0000 7fccfe5dc7c0 10 datascan.inject_with_backtrace: inode: 0x0x100000001f9 2023-05-20T01:27:19.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.834+0000 7fccfe5dc7c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/flushed_file_4 2023-05-20T01:27:19.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.834+0000 7fccfe5dc7c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=flushed_file_4 2023-05-20T01:27:19.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.834+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:74 15.10 15:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0ad9d500 con 0x564e0a259400 2023-05-20T01:27:19.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.834+0000 7fccfbd89700 1 -- 172.21.15.16:0/2853136122 <== osd.4 v2:172.21.15.132:6817/506317223 16 ==== osd_op_reply(74 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x564e0a22b440 con 0x564e0a259400 2023-05-20T01:27:19.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.834+0000 7fccfe5dc7c0 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-05-20T01:27:19.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.834+0000 7fccfe5dc7c0 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-05-20T01:27:19.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.834+0000 7fccfe5dc7c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/flushed_file_4, will insert to root 2023-05-20T01:27:19.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.834+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:75 15.10 15:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0ad9d180 con 0x564e0a259400 2023-05-20T01:27:19.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.835+0000 7fccfbd89700 1 -- 172.21.15.16:0/2853136122 <== osd.4 v2:172.21.15.132:6817/506317223 17 ==== osd_op_reply(75 10000000000.00000000 [omap-get-header out=274b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x564e0a22b440 con 0x564e0a259400 2023-05-20T01:27:19.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.835+0000 7fccfe5dc7c0 10 datascan.find_or_create_dirfrag: read_version = 17 2023-05-20T01:27:19.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.835+0000 7fccfe5dc7c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-05-20T01:27:19.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.835+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:76 15.10 15:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=27b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0ad9d880 con 0x564e0a259400 2023-05-20T01:27:19.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.835+0000 7fccfbd89700 1 -- 172.21.15.16:0/2853136122 <== osd.4 v2:172.21.15.132:6817/506317223 18 ==== osd_op_reply(76 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x564e0a22b440 con 0x564e0a259400 2023-05-20T01:27:19.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.835+0000 7fccfe5dc7c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=flushed_file_4 frag=*, r=0 2023-05-20T01:27:19.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.835+0000 7fccfe5dc7c0 20 datascan.read_dentry: flushed_file_4_head not found in result 2023-05-20T01:27:19.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.835+0000 7fccfe5dc7c0 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001f9 at 0x0x10000000000/flushed_file_4 with size=1441792 bytes 2023-05-20T01:27:19.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.835+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:77 15.10 15:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=515b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0ad9d500 con 0x564e0a259400 2023-05-20T01:27:19.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.836+0000 7fccfbd89700 1 -- 172.21.15.16:0/2853136122 <== osd.4 v2:172.21.15.132:6817/506317223 19 ==== osd_op_reply(77 10000000000.00000000 [omap-set-vals] v119'18 uv18 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564e0a22b440 con 0x564e0a259400 2023-05-20T01:27:19.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.836+0000 7fccfe5dc7c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/flushed_file_4 pointing to 0x0x100000001f9 2023-05-20T01:27:19.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.836+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:78 16.36 16:6fe38dc2::::head [pgnls-filter start_epoch 119 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0ad9d180 con 0x564e0a259000 2023-05-20T01:27:19.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.836+0000 7fccfc58a700 1 -- 172.21.15.16:0/2853136122 <== osd.1 v2:172.21.15.16:6808/3860264220 8 ==== osd_op_reply(78 [pgnls-filter start_epoch 119 out=81b] v119'31 uv31 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x564e0a22b8c0 con 0x564e0a259000 2023-05-20T01:27:19.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.836+0000 7fccfe5dc7c0 20 datascan.forall_objects: OSD matched oid 100000001f7.00000000 2023-05-20T01:27:19.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.836+0000 7fccfe5dc7c0 10 datascan.operator(): handling object 100000001f7.0 2023-05-20T01:27:19.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.836+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:79 16.36 16:6fe38dc2:::100000001f7.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0ad9d880 con 0x564e0a259000 2023-05-20T01:27:19.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.836+0000 7fccfc58a700 1 -- 172.21.15.16:0/2853136122 <== osd.1 v2:172.21.15.16:6808/3860264220 9 ==== osd_op_reply(79 100000001f7.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (111) out=111b,getxattr,getxattr] v0'0 uv27 ondisk = 0) v8 ==== 416+0+157 (crc 0 0 0) 0x564e0a22b8c0 con 0x564e0a259000 2023-05-20T01:27:19.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.836+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:80 17.16 17:6fe38dc2:::100000001f7.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0a37c700 con 0x564e0a380000 2023-05-20T01:27:19.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.837+0000 7fccfcd8b700 1 -- 172.21.15.16:0/2853136122 <== osd.5 v2:172.21.15.132:6816/1844689995 6 ==== osd_op_reply(80 100000001f7.00000000 [getxattr (30) out=30b] v0'0 uv5 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x564e0a22b680 con 0x564e0a380000 2023-05-20T01:27:19.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.837+0000 7fccfe5dc7c0 20 datascan.operator(): loaded layout from xattr: pi: 17 os: 262144 sc: 4 su: 65536 2023-05-20T01:27:19.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.837+0000 7fccfe5dc7c0 10 datascan.operator(): calculating striped size from complete objs: 0, partial objs: 4 2023-05-20T01:27:19.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.837+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:81 17.16 17:6fe38dc2:::100000001f7.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0a37c380 con 0x564e0a380000 2023-05-20T01:27:19.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.837+0000 7fccfcd8b700 1 -- 172.21.15.16:0/2853136122 <== osd.5 v2:172.21.15.132:6816/1844689995 7 ==== osd_op_reply(81 100000001f7.00000000 [stat out=16b] v0'0 uv5 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x564e0a22b680 con 0x564e0a380000 2023-05-20T01:27:19.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.837+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:82 17.1 17:848ebb0b:::100000001f7.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0a37d880 con 0x564e0a22d000 2023-05-20T01:27:19.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.837+0000 7fccfc58a700 1 -- 172.21.15.16:0/2853136122 <== osd.0 v2:172.21.15.16:6824/3923634179 10 ==== osd_op_reply(82 100000001f7.00000001 [stat out=16b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x564e0a22b8c0 con 0x564e0a22d000 2023-05-20T01:27:19.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.837+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:83 17.1c 17:398da958:::100000001f7.00000002:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0a37dc00 con 0x564e0a259000 2023-05-20T01:27:19.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.837+0000 7fccfc58a700 1 -- 172.21.15.16:0/2853136122 <== osd.1 v2:172.21.15.16:6808/3860264220 10 ==== osd_op_reply(83 100000001f7.00000002 [stat out=16b] v0'0 uv5 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x564e0a22b8c0 con 0x564e0a259000 2023-05-20T01:27:19.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.837+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:84 17.6 17:613bf181:::100000001f7.00000003:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0a37c000 con 0x564e0a22c800 2023-05-20T01:27:19.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.838+0000 7fccfbd89700 1 -- 172.21.15.16:0/2853136122 <== osd.6 v2:172.21.15.132:6801/727581123 9 ==== osd_op_reply(84 100000001f7.00000003 [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x564e0a22b440 con 0x564e0a22c800 2023-05-20T01:27:19.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.838+0000 7fccfe5dc7c0 10 datascan.inject_with_backtrace: inode: 0x0x100000001f7 2023-05-20T01:27:19.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.838+0000 7fccfe5dc7c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/flushed_file_2 2023-05-20T01:27:19.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.838+0000 7fccfe5dc7c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=flushed_file_2 2023-05-20T01:27:19.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.838+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:85 15.10 15:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0a37d500 con 0x564e0a259400 2023-05-20T01:27:19.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.838+0000 7fccfbd89700 1 -- 172.21.15.16:0/2853136122 <== osd.4 v2:172.21.15.132:6817/506317223 20 ==== osd_op_reply(85 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x564e0a22b440 con 0x564e0a259400 2023-05-20T01:27:19.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.838+0000 7fccfe5dc7c0 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-05-20T01:27:19.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.838+0000 7fccfe5dc7c0 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-05-20T01:27:19.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.838+0000 7fccfe5dc7c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/flushed_file_2, will insert to root 2023-05-20T01:27:19.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.838+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:86 15.10 15:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0a37ce00 con 0x564e0a259400 2023-05-20T01:27:19.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.838+0000 7fccfbd89700 1 -- 172.21.15.16:0/2853136122 <== osd.4 v2:172.21.15.132:6817/506317223 21 ==== osd_op_reply(86 10000000000.00000000 [omap-get-header out=274b] v0'0 uv18 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x564e0a22b440 con 0x564e0a259400 2023-05-20T01:27:19.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.838+0000 7fccfe5dc7c0 10 datascan.find_or_create_dirfrag: read_version = 18 2023-05-20T01:27:19.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.838+0000 7fccfe5dc7c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-05-20T01:27:19.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.838+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:87 15.10 15:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=27b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0a37d180 con 0x564e0a259400 2023-05-20T01:27:19.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.838+0000 7fccfbd89700 1 -- 172.21.15.16:0/2853136122 <== osd.4 v2:172.21.15.132:6817/506317223 22 ==== osd_op_reply(87 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv18 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x564e0a22b440 con 0x564e0a259400 2023-05-20T01:27:19.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.838+0000 7fccfe5dc7c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=flushed_file_2 frag=*, r=0 2023-05-20T01:27:19.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.838+0000 7fccfe5dc7c0 20 datascan.read_dentry: flushed_file_2_head not found in result 2023-05-20T01:27:19.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.838+0000 7fccfe5dc7c0 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001f7 at 0x0x10000000000/flushed_file_2 with size=917504 bytes 2023-05-20T01:27:19.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.838+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:88 15.10 15:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=515b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0ad9ca80 con 0x564e0a259400 2023-05-20T01:27:19.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.839+0000 7fccfbd89700 1 -- 172.21.15.16:0/2853136122 <== osd.4 v2:172.21.15.132:6817/506317223 23 ==== osd_op_reply(88 10000000000.00000000 [omap-set-vals] v119'19 uv19 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564e0a22b440 con 0x564e0a259400 2023-05-20T01:27:19.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.839+0000 7fccfe5dc7c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/flushed_file_2 pointing to 0x0x100000001f7 2023-05-20T01:27:19.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.839+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:89 16.e 16:70000000::::head [pgnls-filter start_epoch 119 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0ad9d180 con 0x564e0a258800 2023-05-20T01:27:19.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.840+0000 7fccfbd89700 1 -- 172.21.15.16:0/2853136122 <== osd.2 v2:172.21.15.16:6816/293187030 10 ==== osd_op_reply(89 [pgnls-filter start_epoch 119 out=49b] v115'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564e0a22b440 con 0x564e0a258800 2023-05-20T01:27:19.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.840+0000 7fccf7580700 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:90 16.2e 16:74000000::::head [pgnls-filter start_epoch 119 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0ad9dc00 con 0x564e0a22c800 2023-05-20T01:27:19.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.840+0000 7fccfbd89700 1 -- 172.21.15.16:0/2853136122 <== osd.6 v2:172.21.15.132:6801/727581123 10 ==== osd_op_reply(90 [pgnls-filter start_epoch 119 out=49b] v115'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564e0a22b440 con 0x564e0a22c800 2023-05-20T01:27:19.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.840+0000 7fccf7d81700 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:91 16.1e 16:78000000::::head [pgnls-filter start_epoch 119 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0ad9c700 con 0x564e0a258c00 2023-05-20T01:27:19.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.840+0000 7fccfcd8b700 1 -- 172.21.15.16:0/2853136122 <== osd.7 v2:172.21.15.132:6800/466001149 7 ==== osd_op_reply(91 [pgnls-filter start_epoch 119 out=49b] v115'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564e0a22b680 con 0x564e0a258c00 2023-05-20T01:27:19.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.840+0000 7fccf7580700 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:92 16.3e 16:7c000000::::head [pgnls-filter start_epoch 119 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0a392000 con 0x564e0a22c800 2023-05-20T01:27:19.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.841+0000 7fccfbd89700 1 -- 172.21.15.16:0/2853136122 <== osd.6 v2:172.21.15.132:6801/727581123 11 ==== osd_op_reply(92 [pgnls-filter start_epoch 119 out=49b] v115'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564e0a22b440 con 0x564e0a22c800 2023-05-20T01:27:19.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.841+0000 7fccf7d81700 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:93 16.1 16:80000000::::head [pgnls-filter start_epoch 119 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0ad9ce00 con 0x564e0a258800 2023-05-20T01:27:19.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.841+0000 7fccfbd89700 1 -- 172.21.15.16:0/2853136122 <== osd.2 v2:172.21.15.16:6816/293187030 11 ==== osd_op_reply(93 [pgnls-filter start_epoch 119 out=49b] v115'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564e0a22b440 con 0x564e0a258800 2023-05-20T01:27:19.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.841+0000 7fccf7580700 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:94 16.21 16:84000000::::head [pgnls-filter start_epoch 119 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0a392380 con 0x564e0a22c800 2023-05-20T01:27:19.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.841+0000 7fccfbd89700 1 -- 172.21.15.16:0/2853136122 <== osd.6 v2:172.21.15.132:6801/727581123 12 ==== osd_op_reply(94 [pgnls-filter start_epoch 119 out=49b] v115'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564e0a22b440 con 0x564e0a22c800 2023-05-20T01:27:19.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.841+0000 7fccf7d81700 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:95 16.11 16:88000000::::head [pgnls-filter start_epoch 119 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0ad9d500 con 0x564e0a380000 2023-05-20T01:27:19.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.842+0000 7fccfcd8b700 1 -- 172.21.15.16:0/2853136122 <== osd.5 v2:172.21.15.132:6816/1844689995 8 ==== osd_op_reply(95 [pgnls-filter start_epoch 119 out=49b] v115'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564e0a22b680 con 0x564e0a380000 2023-05-20T01:27:19.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.842+0000 7fccf7580700 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:96 16.31 16:8c000000::::head [pgnls-filter start_epoch 119 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0a392700 con 0x564e0a258c00 2023-05-20T01:27:19.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.842+0000 7fccfcd8b700 1 -- 172.21.15.16:0/2853136122 <== osd.7 v2:172.21.15.132:6800/466001149 8 ==== osd_op_reply(96 [pgnls-filter start_epoch 119 out=49b] v115'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564e0a22b680 con 0x564e0a258c00 2023-05-20T01:27:19.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.842+0000 7fccf7d81700 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:97 16.9 16:90000000::::head [pgnls-filter start_epoch 119 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0ad9ca80 con 0x564e0a259000 2023-05-20T01:27:19.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.843+0000 7fccfc58a700 1 -- 172.21.15.16:0/2853136122 <== osd.1 v2:172.21.15.16:6808/3860264220 11 ==== osd_op_reply(97 [pgnls-filter start_epoch 119 out=49b] v115'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564e0a22b8c0 con 0x564e0a259000 2023-05-20T01:27:19.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.843+0000 7fccf7580700 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:98 16.29 16:94000000::::head [pgnls-filter start_epoch 119 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0a392a80 con 0x564e0a22d000 2023-05-20T01:27:19.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.843+0000 7fccfc58a700 1 -- 172.21.15.16:0/2853136122 <== osd.0 v2:172.21.15.16:6824/3923634179 11 ==== osd_op_reply(98 [pgnls-filter start_epoch 119 out=49b] v115'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564e0a22b8c0 con 0x564e0a22d000 2023-05-20T01:27:19.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.843+0000 7fccf7d81700 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:99 16.19 16:98000000::::head [pgnls-filter start_epoch 119 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0ad9d180 con 0x564e0a259400 2023-05-20T01:27:19.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.843+0000 7fccfbd89700 1 -- 172.21.15.16:0/2853136122 <== osd.4 v2:172.21.15.132:6817/506317223 24 ==== osd_op_reply(99 [pgnls-filter start_epoch 119 out=81b] v119'16 uv16 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x564e0a22b440 con 0x564e0a259400 2023-05-20T01:27:19.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.843+0000 7fccfe5dc7c0 20 datascan.forall_objects: OSD matched oid 100000001f8.00000000 2023-05-20T01:27:19.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.843+0000 7fccfe5dc7c0 10 datascan.operator(): handling object 100000001f8.0 2023-05-20T01:27:19.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.843+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:100 16.19 16:997b7129:::100000001f8.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0a392000 con 0x564e0a259400 2023-05-20T01:27:19.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.844+0000 7fccfbd89700 1 -- 172.21.15.16:0/2853136122 <== osd.4 v2:172.21.15.132:6817/506317223 25 ==== osd_op_reply(100 100000001f8.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (111) out=111b,getxattr,getxattr] v0'0 uv16 ondisk = 0) v8 ==== 416+0+157 (crc 0 0 0) 0x564e0a22b440 con 0x564e0a259400 2023-05-20T01:27:19.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.844+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:101 17.19 17:997b7129:::100000001f8.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0a393c00 con 0x564e0a380000 2023-05-20T01:27:19.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.844+0000 7fccfcd8b700 1 -- 172.21.15.16:0/2853136122 <== osd.5 v2:172.21.15.132:6816/1844689995 9 ==== osd_op_reply(101 100000001f8.00000000 [getxattr (30) out=30b] v0'0 uv5 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x564e0a22b680 con 0x564e0a380000 2023-05-20T01:27:19.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.844+0000 7fccfe5dc7c0 20 datascan.operator(): loaded layout from xattr: pi: 17 os: 262144 sc: 4 su: 65536 2023-05-20T01:27:19.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.844+0000 7fccfe5dc7c0 10 datascan.operator(): calculating striped size from complete objs: 0, partial objs: 4 2023-05-20T01:27:19.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.844+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:102 17.19 17:997b7129:::100000001f8.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0ad9dc00 con 0x564e0a380000 2023-05-20T01:27:19.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.845+0000 7fccfcd8b700 1 -- 172.21.15.16:0/2853136122 <== osd.5 v2:172.21.15.132:6816/1844689995 10 ==== osd_op_reply(102 100000001f8.00000000 [stat out=16b] v0'0 uv5 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x564e0a22b680 con 0x564e0a380000 2023-05-20T01:27:19.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.845+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:103 17.11 17:893be98d:::100000001f8.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0ad9c700 con 0x564e0a22cc00 2023-05-20T01:27:19.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.845+0000 7fccfcd8b700 1 -- 172.21.15.16:0/2853136122 <== osd.3 v2:172.21.15.16:6800/3669781890 15 ==== osd_op_reply(103 100000001f8.00000001 [stat out=16b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x564e0a22b680 con 0x564e0a22cc00 2023-05-20T01:27:19.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.845+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:104 17.f 17:f26fdfa9:::100000001f8.00000002:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0ad9d500 con 0x564e0a259000 2023-05-20T01:27:19.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.845+0000 7fccfc58a700 1 -- 172.21.15.16:0/2853136122 <== osd.1 v2:172.21.15.16:6808/3860264220 12 ==== osd_op_reply(104 100000001f8.00000002 [stat out=16b] v0'0 uv7 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x564e0a22b8c0 con 0x564e0a259000 2023-05-20T01:27:19.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.845+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:105 17.17 17:eff9002a:::100000001f8.00000003:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0ad9dc00 con 0x564e0a22c800 2023-05-20T01:27:19.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.846+0000 7fccfbd89700 1 -- 172.21.15.16:0/2853136122 <== osd.6 v2:172.21.15.132:6801/727581123 13 ==== osd_op_reply(105 100000001f8.00000003 [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x564e0a22b440 con 0x564e0a22c800 2023-05-20T01:27:19.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.846+0000 7fccfe5dc7c0 10 datascan.inject_with_backtrace: inode: 0x0x100000001f8 2023-05-20T01:27:19.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.846+0000 7fccfe5dc7c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/flushed_file_3 2023-05-20T01:27:19.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.846+0000 7fccfe5dc7c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=flushed_file_3 2023-05-20T01:27:19.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.846+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:106 15.10 15:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0a392700 con 0x564e0a259400 2023-05-20T01:27:19.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.846+0000 7fccfbd89700 1 -- 172.21.15.16:0/2853136122 <== osd.4 v2:172.21.15.132:6817/506317223 26 ==== osd_op_reply(106 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x564e0a22b440 con 0x564e0a259400 2023-05-20T01:27:19.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.846+0000 7fccfe5dc7c0 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-05-20T01:27:19.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.846+0000 7fccfe5dc7c0 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-05-20T01:27:19.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.846+0000 7fccfe5dc7c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/flushed_file_3, will insert to root 2023-05-20T01:27:19.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.846+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:107 15.10 15:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0a393c00 con 0x564e0a259400 2023-05-20T01:27:19.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.846+0000 7fccfbd89700 1 -- 172.21.15.16:0/2853136122 <== osd.4 v2:172.21.15.132:6817/506317223 27 ==== osd_op_reply(107 10000000000.00000000 [omap-get-header out=274b] v0'0 uv19 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x564e0a22b440 con 0x564e0a259400 2023-05-20T01:27:19.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.846+0000 7fccfe5dc7c0 10 datascan.find_or_create_dirfrag: read_version = 19 2023-05-20T01:27:19.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.846+0000 7fccfe5dc7c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-05-20T01:27:19.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.846+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:108 15.10 15:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=27b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0a37ca80 con 0x564e0a259400 2023-05-20T01:27:19.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.846+0000 7fccfbd89700 1 -- 172.21.15.16:0/2853136122 <== osd.4 v2:172.21.15.132:6817/506317223 28 ==== osd_op_reply(108 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv19 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x564e0a22b440 con 0x564e0a259400 2023-05-20T01:27:19.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.847+0000 7fccfe5dc7c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=flushed_file_3 frag=*, r=0 2023-05-20T01:27:19.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.847+0000 7fccfe5dc7c0 20 datascan.read_dentry: flushed_file_3_head not found in result 2023-05-20T01:27:19.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.847+0000 7fccfe5dc7c0 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001f8 at 0x0x10000000000/flushed_file_3 with size=917503 bytes 2023-05-20T01:27:19.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.847+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:109 15.10 15:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=515b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0a37c700 con 0x564e0a259400 2023-05-20T01:27:19.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.848+0000 7fccfbd89700 1 -- 172.21.15.16:0/2853136122 <== osd.4 v2:172.21.15.132:6817/506317223 29 ==== osd_op_reply(109 10000000000.00000000 [omap-set-vals] v119'20 uv20 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564e0a22b440 con 0x564e0a259400 2023-05-20T01:27:19.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.848+0000 7fccfe5dc7c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/flushed_file_3 pointing to 0x0x100000001f8 2023-05-20T01:27:19.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.848+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:110 16.39 16:9c000000::::head [pgnls-filter start_epoch 119 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0a37c380 con 0x564e0a22c800 2023-05-20T01:27:19.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.848+0000 7fccfbd89700 1 -- 172.21.15.16:0/2853136122 <== osd.6 v2:172.21.15.132:6801/727581123 14 ==== osd_op_reply(110 [pgnls-filter start_epoch 119 out=49b] v115'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564e0a22b440 con 0x564e0a22c800 2023-05-20T01:27:19.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.848+0000 7fccf7d81700 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:111 16.5 16:a0000000::::head [pgnls-filter start_epoch 119 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0a37c000 con 0x564e0a22d000 2023-05-20T01:27:19.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.848+0000 7fccfc58a700 1 -- 172.21.15.16:0/2853136122 <== osd.0 v2:172.21.15.16:6824/3923634179 12 ==== osd_op_reply(111 [pgnls-filter start_epoch 119 out=49b] v115'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564e0a22b8c0 con 0x564e0a22d000 2023-05-20T01:27:19.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.848+0000 7fccf7580700 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:112 16.25 16:a4000000::::head [pgnls-filter start_epoch 119 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0a392e00 con 0x564e0a380000 2023-05-20T01:27:19.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.849+0000 7fccfcd8b700 1 -- 172.21.15.16:0/2853136122 <== osd.5 v2:172.21.15.132:6816/1844689995 11 ==== osd_op_reply(112 [pgnls-filter start_epoch 119 out=49b] v115'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564e0a22b680 con 0x564e0a380000 2023-05-20T01:27:19.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.849+0000 7fccf7d81700 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:113 16.15 16:a8000000::::head [pgnls-filter start_epoch 119 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0a37d500 con 0x564e0a22d000 2023-05-20T01:27:19.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.849+0000 7fccfc58a700 1 -- 172.21.15.16:0/2853136122 <== osd.0 v2:172.21.15.16:6824/3923634179 13 ==== osd_op_reply(113 [pgnls-filter start_epoch 119 out=49b] v115'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564e0a22b8c0 con 0x564e0a22d000 2023-05-20T01:27:19.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.849+0000 7fccf7580700 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:114 16.35 16:ac000000::::head [pgnls-filter start_epoch 119 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0a393180 con 0x564e0a380000 2023-05-20T01:27:19.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.849+0000 7fccfcd8b700 1 -- 172.21.15.16:0/2853136122 <== osd.5 v2:172.21.15.132:6816/1844689995 12 ==== osd_op_reply(114 [pgnls-filter start_epoch 119 out=49b] v115'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564e0a22b680 con 0x564e0a380000 2023-05-20T01:27:19.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.849+0000 7fccf7d81700 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:115 16.d 16:b0000000::::head [pgnls-filter start_epoch 119 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0a37ce00 con 0x564e0a22c800 2023-05-20T01:27:19.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.849+0000 7fccfbd89700 1 -- 172.21.15.16:0/2853136122 <== osd.6 v2:172.21.15.132:6801/727581123 15 ==== osd_op_reply(115 [pgnls-filter start_epoch 119 out=49b] v115'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564e0a22b440 con 0x564e0a22c800 2023-05-20T01:27:19.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.849+0000 7fccf7580700 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:116 16.2d 16:b4000000::::head [pgnls-filter start_epoch 119 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0a393500 con 0x564e0a22d000 2023-05-20T01:27:19.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.850+0000 7fccfc58a700 1 -- 172.21.15.16:0/2853136122 <== osd.0 v2:172.21.15.16:6824/3923634179 14 ==== osd_op_reply(116 [pgnls-filter start_epoch 119 out=49b] v115'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564e0a22b8c0 con 0x564e0a22d000 2023-05-20T01:27:19.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.850+0000 7fccf7d81700 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:117 16.1d 16:b8000000::::head [pgnls-filter start_epoch 119 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0a37d180 con 0x564e0a258800 2023-05-20T01:27:19.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.850+0000 7fccfbd89700 1 -- 172.21.15.16:0/2853136122 <== osd.2 v2:172.21.15.16:6816/293187030 12 ==== osd_op_reply(117 [pgnls-filter start_epoch 119 out=49b] v115'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564e0a22b440 con 0x564e0a258800 2023-05-20T01:27:19.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.850+0000 7fccf7580700 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:118 16.3d 16:bc000000::::head [pgnls-filter start_epoch 119 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0a392a80 con 0x564e0a22d000 2023-05-20T01:27:19.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.850+0000 7fccfc58a700 1 -- 172.21.15.16:0/2853136122 <== osd.0 v2:172.21.15.16:6824/3923634179 15 ==== osd_op_reply(118 [pgnls-filter start_epoch 119 out=49b] v115'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564e0a22b8c0 con 0x564e0a22d000 2023-05-20T01:27:19.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.850+0000 7fccf7d81700 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:119 16.3 16:c0000000::::head [pgnls-filter start_epoch 119 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0a37d880 con 0x564e0a380000 2023-05-20T01:27:19.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.851+0000 7fccfcd8b700 1 -- 172.21.15.16:0/2853136122 <== osd.5 v2:172.21.15.132:6816/1844689995 13 ==== osd_op_reply(119 [pgnls-filter start_epoch 119 out=49b] v115'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564e0a22b680 con 0x564e0a380000 2023-05-20T01:27:19.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.851+0000 7fccf7580700 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:120 16.23 16:c4000000::::head [pgnls-filter start_epoch 119 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0a392380 con 0x564e0a258800 2023-05-20T01:27:19.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.851+0000 7fccfbd89700 1 -- 172.21.15.16:0/2853136122 <== osd.2 v2:172.21.15.16:6816/293187030 13 ==== osd_op_reply(120 [pgnls-filter start_epoch 119 out=49b] v115'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564e0a22b440 con 0x564e0a258800 2023-05-20T01:27:19.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.851+0000 7fccf7d81700 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:121 16.13 16:c8000000::::head [pgnls-filter start_epoch 119 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0a37dc00 con 0x564e0a259000 2023-05-20T01:27:19.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.851+0000 7fccfc58a700 1 -- 172.21.15.16:0/2853136122 <== osd.1 v2:172.21.15.16:6808/3860264220 13 ==== osd_op_reply(121 [pgnls-filter start_epoch 119 out=81b] v119'25 uv25 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x564e0a22b8c0 con 0x564e0a259000 2023-05-20T01:27:19.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.851+0000 7fccfe5dc7c0 20 datascan.forall_objects: OSD matched oid 100000001fb.00000000 2023-05-20T01:27:19.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.851+0000 7fccfe5dc7c0 10 datascan.operator(): handling object 100000001fb.0 2023-05-20T01:27:19.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.851+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:122 16.13 16:ca8fdbba:::100000001fb.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0a37c700 con 0x564e0a259000 2023-05-20T01:27:19.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.851+0000 7fccfc58a700 1 -- 172.21.15.16:0/2853136122 <== osd.1 v2:172.21.15.16:6808/3860264220 14 ==== osd_op_reply(122 100000001fb.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (111) out=111b,getxattr,getxattr] v0'0 uv25 ondisk = 0) v8 ==== 416+0+157 (crc 0 0 0) 0x564e0a22b8c0 con 0x564e0a259000 2023-05-20T01:27:19.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.851+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:123 17.13 17:ca8fdbba:::100000001fb.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0a39c380 con 0x564e0a22cc00 2023-05-20T01:27:19.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.852+0000 7fccfcd8b700 1 -- 172.21.15.16:0/2853136122 <== osd.3 v2:172.21.15.16:6800/3669781890 16 ==== osd_op_reply(123 100000001fb.00000000 [getxattr (30) out=30b] v0'0 uv13 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x564e0a22b680 con 0x564e0a22cc00 2023-05-20T01:27:19.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.852+0000 7fccfe5dc7c0 20 datascan.operator(): loaded layout from xattr: pi: 17 os: 262144 sc: 4 su: 65536 2023-05-20T01:27:19.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.852+0000 7fccfe5dc7c0 10 datascan.operator(): calculating striped size from complete objs: 4, partial objs: 4 2023-05-20T01:27:19.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.852+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:124 17.3 17:c5d8c51d:::100000001fb.00000004:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0a39c700 con 0x564e0a259400 2023-05-20T01:27:19.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.852+0000 7fccfbd89700 1 -- 172.21.15.16:0/2853136122 <== osd.4 v2:172.21.15.132:6817/506317223 30 ==== osd_op_reply(124 100000001fb.00000004 [stat out=16b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x564e0a22b440 con 0x564e0a259400 2023-05-20T01:27:19.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.852+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:125 17.f 17:f376f0dc:::100000001fb.00000005:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0a39ca80 con 0x564e0a259000 2023-05-20T01:27:19.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.852+0000 7fccfc58a700 1 -- 172.21.15.16:0/2853136122 <== osd.1 v2:172.21.15.16:6808/3860264220 15 ==== osd_op_reply(125 100000001fb.00000005 [stat out=16b] v0'0 uv10 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x564e0a22b8c0 con 0x564e0a259000 2023-05-20T01:27:19.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.852+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:126 17.1e 17:7e8133fb:::100000001fb.00000006:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0a39ce00 con 0x564e0a259000 2023-05-20T01:27:19.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.853+0000 7fccfc58a700 1 -- 172.21.15.16:0/2853136122 <== osd.1 v2:172.21.15.16:6808/3860264220 16 ==== osd_op_reply(126 100000001fb.00000006 [stat out=16b] v0'0 uv6 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x564e0a22b8c0 con 0x564e0a259000 2023-05-20T01:27:19.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.853+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:127 17.f 17:f776b802:::100000001fb.00000007:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0a39d180 con 0x564e0a259000 2023-05-20T01:27:19.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.853+0000 7fccfc58a700 1 -- 172.21.15.16:0/2853136122 <== osd.1 v2:172.21.15.16:6808/3860264220 17 ==== osd_op_reply(127 100000001fb.00000007 [stat out=16b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x564e0a22b8c0 con 0x564e0a259000 2023-05-20T01:27:19.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.853+0000 7fccfe5dc7c0 10 datascan.inject_with_backtrace: inode: 0x0x100000001fb 2023-05-20T01:27:19.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.853+0000 7fccfe5dc7c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/flushed_file_6 2023-05-20T01:27:19.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.853+0000 7fccfe5dc7c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=flushed_file_6 2023-05-20T01:27:19.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.853+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:128 15.10 15:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0a39d500 con 0x564e0a259400 2023-05-20T01:27:19.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.853+0000 7fccfbd89700 1 -- 172.21.15.16:0/2853136122 <== osd.4 v2:172.21.15.132:6817/506317223 31 ==== osd_op_reply(128 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x564e0a22b440 con 0x564e0a259400 2023-05-20T01:27:19.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.853+0000 7fccfe5dc7c0 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-05-20T01:27:19.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.853+0000 7fccfe5dc7c0 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-05-20T01:27:19.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.853+0000 7fccfe5dc7c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/flushed_file_6, will insert to root 2023-05-20T01:27:19.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.853+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:129 15.10 15:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0a39d880 con 0x564e0a259400 2023-05-20T01:27:19.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.853+0000 7fccfbd89700 1 -- 172.21.15.16:0/2853136122 <== osd.4 v2:172.21.15.132:6817/506317223 32 ==== osd_op_reply(129 10000000000.00000000 [omap-get-header out=274b] v0'0 uv20 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x564e0a22b440 con 0x564e0a259400 2023-05-20T01:27:19.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.853+0000 7fccfe5dc7c0 10 datascan.find_or_create_dirfrag: read_version = 20 2023-05-20T01:27:19.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.853+0000 7fccfe5dc7c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-05-20T01:27:19.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.853+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:130 15.10 15:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=27b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0a39dc00 con 0x564e0a259400 2023-05-20T01:27:19.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.854+0000 7fccfbd89700 1 -- 172.21.15.16:0/2853136122 <== osd.4 v2:172.21.15.132:6817/506317223 33 ==== osd_op_reply(130 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv20 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x564e0a22b440 con 0x564e0a259400 2023-05-20T01:27:19.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.854+0000 7fccfe5dc7c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=flushed_file_6 frag=*, r=0 2023-05-20T01:27:19.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.854+0000 7fccfe5dc7c0 20 datascan.read_dentry: flushed_file_6_head not found in result 2023-05-20T01:27:19.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.854+0000 7fccfe5dc7c0 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001fb at 0x0x10000000000/flushed_file_6 with size=1572864 bytes 2023-05-20T01:27:19.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.854+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:131 15.10 15:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=515b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0a39c000 con 0x564e0a259400 2023-05-20T01:27:19.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.855+0000 7fccfbd89700 1 -- 172.21.15.16:0/2853136122 <== osd.4 v2:172.21.15.132:6817/506317223 34 ==== osd_op_reply(131 10000000000.00000000 [omap-set-vals] v119'21 uv21 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564e0a22b440 con 0x564e0a259400 2023-05-20T01:27:19.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.855+0000 7fccfe5dc7c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/flushed_file_6 pointing to 0x0x100000001fb 2023-05-20T01:27:19.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.855+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:132 16.33 16:cc000000::::head [pgnls-filter start_epoch 119 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0a39ca80 con 0x564e0a22cc00 2023-05-20T01:27:19.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.855+0000 7fccfcd8b700 1 -- 172.21.15.16:0/2853136122 <== osd.3 v2:172.21.15.16:6800/3669781890 17 ==== osd_op_reply(132 [pgnls-filter start_epoch 119 out=49b] v115'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564e0a22b680 con 0x564e0a22cc00 2023-05-20T01:27:19.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.855+0000 7fccf7d81700 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:133 16.b 16:d0000000::::head [pgnls-filter start_epoch 119 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0a37c000 con 0x564e0a259000 2023-05-20T01:27:19.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.855+0000 7fccfc58a700 1 -- 172.21.15.16:0/2853136122 <== osd.1 v2:172.21.15.16:6808/3860264220 18 ==== osd_op_reply(133 [pgnls-filter start_epoch 119 out=49b] v115'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564e0a22b8c0 con 0x564e0a259000 2023-05-20T01:27:19.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.855+0000 7fccf7580700 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:134 16.2b 16:d4000000::::head [pgnls-filter start_epoch 119 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0ad9d880 con 0x564e0a258800 2023-05-20T01:27:19.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.855+0000 7fccfbd89700 1 -- 172.21.15.16:0/2853136122 <== osd.2 v2:172.21.15.16:6816/293187030 14 ==== osd_op_reply(134 [pgnls-filter start_epoch 119 out=49b] v115'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564e0a22b440 con 0x564e0a258800 2023-05-20T01:27:19.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.855+0000 7fccf7d81700 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:135 16.1b 16:d8000000::::head [pgnls-filter start_epoch 119 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0a37ca80 con 0x564e0a380000 2023-05-20T01:27:19.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.856+0000 7fccfcd8b700 1 -- 172.21.15.16:0/2853136122 <== osd.5 v2:172.21.15.132:6816/1844689995 14 ==== osd_op_reply(135 [pgnls-filter start_epoch 119 out=49b] v115'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564e0a22b680 con 0x564e0a380000 2023-05-20T01:27:19.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.856+0000 7fccf7580700 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:136 16.3b 16:dc000000::::head [pgnls-filter start_epoch 119 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0ad9ca80 con 0x564e0a259000 2023-05-20T01:27:19.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.856+0000 7fccfc58a700 1 -- 172.21.15.16:0/2853136122 <== osd.1 v2:172.21.15.16:6808/3860264220 19 ==== osd_op_reply(136 [pgnls-filter start_epoch 119 out=49b] v115'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564e0a22b8c0 con 0x564e0a259000 2023-05-20T01:27:19.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.856+0000 7fccf7d81700 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:137 16.7 16:e0000000::::head [pgnls-filter start_epoch 119 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0a393880 con 0x564e0a22cc00 2023-05-20T01:27:19.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.856+0000 7fccfcd8b700 1 -- 172.21.15.16:0/2853136122 <== osd.3 v2:172.21.15.16:6800/3669781890 18 ==== osd_op_reply(137 [pgnls-filter start_epoch 119 out=81b] v119'11 uv11 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x564e0a22b680 con 0x564e0a22cc00 2023-05-20T01:27:19.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.856+0000 7fccfe5dc7c0 20 datascan.forall_objects: OSD matched oid 10000000001.00000000 2023-05-20T01:27:19.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.856+0000 7fccfe5dc7c0 10 datascan.operator(): handling object 10000000001.0 2023-05-20T01:27:19.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.856+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:138 16.7 16:e2e07930:::10000000001.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0a39ce00 con 0x564e0a22cc00 2023-05-20T01:27:19.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.857+0000 7fccfcd8b700 1 -- 172.21.15.16:0/2853136122 <== osd.3 v2:172.21.15.16:6800/3669781890 19 ==== osd_op_reply(138 10000000001.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (111) out=111b,getxattr,getxattr] v0'0 uv11 ondisk = 0) v8 ==== 416+0+157 (crc 0 0 0) 0x564e0a22b680 con 0x564e0a22cc00 2023-05-20T01:27:19.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.857+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:139 17.7 17:e2e07930:::10000000001.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0a37dc00 con 0x564e0a22d000 2023-05-20T01:27:19.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.857+0000 7fccfc58a700 1 -- 172.21.15.16:0/2853136122 <== osd.0 v2:172.21.15.16:6824/3923634179 16 ==== osd_op_reply(139 10000000001.00000000 [getxattr (30) out=30b] v0'0 uv6 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x564e0a22b8c0 con 0x564e0a22d000 2023-05-20T01:27:19.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.857+0000 7fccfe5dc7c0 20 datascan.operator(): loaded layout from xattr: pi: 17 os: 262144 sc: 4 su: 65536 2023-05-20T01:27:19.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.857+0000 7fccfe5dc7c0 10 datascan.operator(): calculating striped size from complete objs: 0, partial objs: 4 2023-05-20T01:27:19.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.857+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:140 17.7 17:e2e07930:::10000000001.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0a37c700 con 0x564e0a22d000 2023-05-20T01:27:19.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.857+0000 7fccfc58a700 1 -- 172.21.15.16:0/2853136122 <== osd.0 v2:172.21.15.16:6824/3923634179 17 ==== osd_op_reply(140 10000000001.00000000 [stat out=16b] v0'0 uv6 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x564e0a22b8c0 con 0x564e0a22d000 2023-05-20T01:27:19.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.857+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:141 17.f 17:f526278d:::10000000001.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0a393500 con 0x564e0a259000 2023-05-20T01:27:19.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.858+0000 7fccfc58a700 1 -- 172.21.15.16:0/2853136122 <== osd.1 v2:172.21.15.16:6808/3860264220 20 ==== osd_op_reply(141 10000000001.00000001 [stat out=16b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x564e0a22b8c0 con 0x564e0a259000 2023-05-20T01:27:19.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.858+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:142 17.1e 17:7e4d8c81:::10000000001.00000002:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0a392a80 con 0x564e0a259000 2023-05-20T01:27:19.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.858+0000 7fccfc58a700 1 -- 172.21.15.16:0/2853136122 <== osd.1 v2:172.21.15.16:6808/3860264220 21 ==== osd_op_reply(142 10000000001.00000002 [stat out=16b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x564e0a22b8c0 con 0x564e0a259000 2023-05-20T01:27:19.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.858+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:143 17.e 17:72395991:::10000000001.00000003:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0a37d880 con 0x564e0a258800 2023-05-20T01:27:19.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.858+0000 7fccfbd89700 1 -- 172.21.15.16:0/2853136122 <== osd.2 v2:172.21.15.16:6816/293187030 15 ==== osd_op_reply(143 10000000001.00000003 [stat out=16b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x564e0a22b440 con 0x564e0a258800 2023-05-20T01:27:19.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.858+0000 7fccfe5dc7c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000001 2023-05-20T01:27:19.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.858+0000 7fccfe5dc7c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/flushed_file_0 2023-05-20T01:27:19.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.858+0000 7fccfe5dc7c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=flushed_file_0 2023-05-20T01:27:19.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.858+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:144 15.10 15:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0a37ca80 con 0x564e0a259400 2023-05-20T01:27:19.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.858+0000 7fccfbd89700 1 -- 172.21.15.16:0/2853136122 <== osd.4 v2:172.21.15.132:6817/506317223 35 ==== osd_op_reply(144 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x564e0a22b440 con 0x564e0a259400 2023-05-20T01:27:19.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.858+0000 7fccfe5dc7c0 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-05-20T01:27:19.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.858+0000 7fccfe5dc7c0 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-05-20T01:27:19.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.858+0000 7fccfe5dc7c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/flushed_file_0, will insert to root 2023-05-20T01:27:19.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.858+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:145 15.10 15:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0a37d500 con 0x564e0a259400 2023-05-20T01:27:19.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.859+0000 7fccfbd89700 1 -- 172.21.15.16:0/2853136122 <== osd.4 v2:172.21.15.132:6817/506317223 36 ==== osd_op_reply(145 10000000000.00000000 [omap-get-header out=274b] v0'0 uv21 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x564e0a22b440 con 0x564e0a259400 2023-05-20T01:27:19.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.859+0000 7fccfe5dc7c0 10 datascan.find_or_create_dirfrag: read_version = 21 2023-05-20T01:27:19.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.859+0000 7fccfe5dc7c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-05-20T01:27:19.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.859+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:146 15.10 15:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=27b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0a392e00 con 0x564e0a259400 2023-05-20T01:27:19.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.859+0000 7fccfbd89700 1 -- 172.21.15.16:0/2853136122 <== osd.4 v2:172.21.15.132:6817/506317223 37 ==== osd_op_reply(146 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv21 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x564e0a22b440 con 0x564e0a259400 2023-05-20T01:27:19.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.859+0000 7fccfe5dc7c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=flushed_file_0 frag=*, r=0 2023-05-20T01:27:19.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.859+0000 7fccfe5dc7c0 20 datascan.read_dentry: flushed_file_0_head not found in result 2023-05-20T01:27:19.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.859+0000 7fccfe5dc7c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000001 at 0x0x10000000000/flushed_file_0 with size=1048576 bytes 2023-05-20T01:27:19.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.859+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:147 15.10 15:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=515b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0a393180 con 0x564e0a259400 2023-05-20T01:27:19.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.860+0000 7fccfbd89700 1 -- 172.21.15.16:0/2853136122 <== osd.4 v2:172.21.15.132:6817/506317223 38 ==== osd_op_reply(147 10000000000.00000000 [omap-set-vals] v119'22 uv22 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564e0a22b440 con 0x564e0a259400 2023-05-20T01:27:19.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.860+0000 7fccfe5dc7c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/flushed_file_0 pointing to 0x0x10000000001 2023-05-20T01:27:19.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.860+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:148 16.27 16:e4000000::::head [pgnls-filter start_epoch 119 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0a393880 con 0x564e0a259400 2023-05-20T01:27:19.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.861+0000 7fccfbd89700 1 -- 172.21.15.16:0/2853136122 <== osd.4 v2:172.21.15.132:6817/506317223 39 ==== osd_op_reply(148 [pgnls-filter start_epoch 119 out=49b] v115'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564e0a22b440 con 0x564e0a259400 2023-05-20T01:27:19.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.861+0000 7fccf7d81700 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:149 16.17 16:e8000000::::head [pgnls-filter start_epoch 119 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0a392000 con 0x564e0a258800 2023-05-20T01:27:19.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.861+0000 7fccfbd89700 1 -- 172.21.15.16:0/2853136122 <== osd.2 v2:172.21.15.16:6816/293187030 16 ==== osd_op_reply(149 [pgnls-filter start_epoch 119 out=49b] v115'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564e0a22b440 con 0x564e0a258800 2023-05-20T01:27:19.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.861+0000 7fccf7580700 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:150 16.37 16:ec000000::::head [pgnls-filter start_epoch 119 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0ad9d500 con 0x564e0a22c800 2023-05-20T01:27:19.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.861+0000 7fccfbd89700 1 -- 172.21.15.16:0/2853136122 <== osd.6 v2:172.21.15.132:6801/727581123 16 ==== osd_op_reply(150 [pgnls-filter start_epoch 119 out=49b] v115'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564e0a22b440 con 0x564e0a22c800 2023-05-20T01:27:19.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.861+0000 7fccf7d81700 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:151 16.f 16:f0000000::::head [pgnls-filter start_epoch 119 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0a392700 con 0x564e0a22d000 2023-05-20T01:27:19.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.861+0000 7fccfc58a700 1 -- 172.21.15.16:0/2853136122 <== osd.0 v2:172.21.15.16:6824/3923634179 18 ==== osd_op_reply(151 [pgnls-filter start_epoch 119 out=49b] v115'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564e0a22b8c0 con 0x564e0a22d000 2023-05-20T01:27:19.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.862+0000 7fccf7580700 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:152 16.2f 16:f4000000::::head [pgnls-filter start_epoch 119 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0ad9ce00 con 0x564e0a22d000 2023-05-20T01:27:19.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.862+0000 7fccfc58a700 1 -- 172.21.15.16:0/2853136122 <== osd.0 v2:172.21.15.16:6824/3923634179 19 ==== osd_op_reply(152 [pgnls-filter start_epoch 119 out=49b] v115'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564e0a22b8c0 con 0x564e0a22d000 2023-05-20T01:27:19.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.862+0000 7fccf7d81700 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:153 16.1f 16:f8000000::::head [pgnls-filter start_epoch 119 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0a393c00 con 0x564e0a380000 2023-05-20T01:27:19.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.862+0000 7fccfcd8b700 1 -- 172.21.15.16:0/2853136122 <== osd.5 v2:172.21.15.132:6816/1844689995 15 ==== osd_op_reply(153 [pgnls-filter start_epoch 119 out=49b] v115'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564e0a22b680 con 0x564e0a380000 2023-05-20T01:27:19.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.862+0000 7fccf7580700 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:154 16.3f 16:fc000000::::head [pgnls-filter start_epoch 119 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0ad9d180 con 0x564e0a22d000 2023-05-20T01:27:19.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.862+0000 7fccfc58a700 1 -- 172.21.15.16:0/2853136122 <== osd.0 v2:172.21.15.16:6824/3923634179 20 ==== osd_op_reply(154 [pgnls-filter start_epoch 119 out=81b] v119'18 uv18 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x564e0a22b8c0 con 0x564e0a22d000 2023-05-20T01:27:19.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.862+0000 7fccfe5dc7c0 20 datascan.forall_objects: OSD matched oid 100000001fa.00000000 2023-05-20T01:27:19.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.862+0000 7fccfe5dc7c0 10 datascan.operator(): handling object 100000001fa.0 2023-05-20T01:27:19.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.862+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:155 16.3f 16:feb1c803:::100000001fa.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0a39c380 con 0x564e0a22d000 2023-05-20T01:27:19.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.863+0000 7fccfc58a700 1 -- 172.21.15.16:0/2853136122 <== osd.0 v2:172.21.15.16:6824/3923634179 21 ==== osd_op_reply(155 100000001fa.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (111) out=111b,getxattr,getxattr] v0'0 uv18 ondisk = 0) v8 ==== 416+0+157 (crc 0 0 0) 0x564e0a22b8c0 con 0x564e0a22d000 2023-05-20T01:27:19.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.863+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:156 17.1f 17:feb1c803:::100000001fa.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0ad9c700 con 0x564e0a22c800 2023-05-20T01:27:19.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.863+0000 7fccfbd89700 1 -- 172.21.15.16:0/2853136122 <== osd.6 v2:172.21.15.132:6801/727581123 17 ==== osd_op_reply(156 100000001fa.00000000 [getxattr (30) out=30b] v0'0 uv7 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x564e0a22b440 con 0x564e0a22c800 2023-05-20T01:27:19.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.863+0000 7fccfe5dc7c0 20 datascan.operator(): loaded layout from xattr: pi: 17 os: 262144 sc: 4 su: 65536 2023-05-20T01:27:19.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.863+0000 7fccfe5dc7c0 10 datascan.operator(): calculating striped size from complete objs: 4, partial objs: 4 2023-05-20T01:27:19.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.863+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:157 17.1c 17:3c0114ac:::100000001fa.00000004:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0a37c000 con 0x564e0a259000 2023-05-20T01:27:19.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.863+0000 7fccfc58a700 1 -- 172.21.15.16:0/2853136122 <== osd.1 v2:172.21.15.16:6808/3860264220 22 ==== osd_op_reply(157 100000001fa.00000004 [stat out=16b] v0'0 uv7 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x564e0a22b8c0 con 0x564e0a259000 2023-05-20T01:27:19.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.863+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:158 17.3 17:c0957981:::100000001fa.00000005:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0a37dc00 con 0x564e0a259400 2023-05-20T01:27:19.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.864+0000 7fccfbd89700 1 -- 172.21.15.16:0/2853136122 <== osd.4 v2:172.21.15.132:6817/506317223 40 ==== osd_op_reply(158 100000001fa.00000005 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x564e0a22b440 con 0x564e0a259400 2023-05-20T01:27:19.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.864+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:159 17.a 17:56c65d28:::100000001fa.00000006:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0a37c700 con 0x564e0a258c00 2023-05-20T01:27:19.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.864+0000 7fccfcd8b700 1 -- 172.21.15.16:0/2853136122 <== osd.7 v2:172.21.15.132:6800/466001149 9 ==== osd_op_reply(159 100000001fa.00000006 [stat out=16b] v0'0 uv5 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x564e0a22b680 con 0x564e0a258c00 2023-05-20T01:27:19.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.864+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:160 17.1f 17:fb032386:::100000001fa.00000007:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0a37c380 con 0x564e0a22c800 2023-05-20T01:27:19.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.864+0000 7fccfbd89700 1 -- 172.21.15.16:0/2853136122 <== osd.6 v2:172.21.15.132:6801/727581123 18 ==== osd_op_reply(160 100000001fa.00000007 [stat out=16b] v0'0 uv8 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x564e0a22b440 con 0x564e0a22c800 2023-05-20T01:27:19.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.864+0000 7fccfe5dc7c0 10 datascan.inject_with_backtrace: inode: 0x0x100000001fa 2023-05-20T01:27:19.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.864+0000 7fccfe5dc7c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/flushed_file_5 2023-05-20T01:27:19.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.864+0000 7fccfe5dc7c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=flushed_file_5 2023-05-20T01:27:19.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.864+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:161 15.10 15:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0a37ce00 con 0x564e0a259400 2023-05-20T01:27:19.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.865+0000 7fccfbd89700 1 -- 172.21.15.16:0/2853136122 <== osd.4 v2:172.21.15.132:6817/506317223 41 ==== osd_op_reply(161 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x564e0a22b440 con 0x564e0a259400 2023-05-20T01:27:19.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.865+0000 7fccfe5dc7c0 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-05-20T01:27:19.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.865+0000 7fccfe5dc7c0 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-05-20T01:27:19.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.865+0000 7fccfe5dc7c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/flushed_file_5, will insert to root 2023-05-20T01:27:19.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.865+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:162 15.10 15:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0a37d180 con 0x564e0a259400 2023-05-20T01:27:19.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.865+0000 7fccfbd89700 1 -- 172.21.15.16:0/2853136122 <== osd.4 v2:172.21.15.132:6817/506317223 42 ==== osd_op_reply(162 10000000000.00000000 [omap-get-header out=274b] v0'0 uv22 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x564e0a22b440 con 0x564e0a259400 2023-05-20T01:27:19.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.865+0000 7fccfe5dc7c0 10 datascan.find_or_create_dirfrag: read_version = 22 2023-05-20T01:27:19.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.865+0000 7fccfe5dc7c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-05-20T01:27:19.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.865+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:163 15.10 15:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=27b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0ad9ca80 con 0x564e0a259400 2023-05-20T01:27:19.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.865+0000 7fccfbd89700 1 -- 172.21.15.16:0/2853136122 <== osd.4 v2:172.21.15.132:6817/506317223 43 ==== osd_op_reply(163 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv22 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x564e0a22b440 con 0x564e0a259400 2023-05-20T01:27:19.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.865+0000 7fccfe5dc7c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=flushed_file_5 frag=*, r=0 2023-05-20T01:27:19.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.865+0000 7fccfe5dc7c0 20 datascan.read_dentry: flushed_file_5_head not found in result 2023-05-20T01:27:19.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.865+0000 7fccfe5dc7c0 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001fa at 0x0x10000000000/flushed_file_5 with size=1441793 bytes 2023-05-20T01:27:19.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.865+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/2853136122 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:164 15.10 15:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=515b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x564e0ad9ce00 con 0x564e0a259400 2023-05-20T01:27:19.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.866+0000 7fccfbd89700 1 -- 172.21.15.16:0/2853136122 <== osd.4 v2:172.21.15.132:6817/506317223 44 ==== osd_op_reply(164 10000000000.00000000 [omap-set-vals] v119'23 uv23 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564e0a22b440 con 0x564e0a259400 2023-05-20T01:27:19.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.866+0000 7fccfe5dc7c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/flushed_file_5 pointing to 0x0x100000001fa 2023-05-20T01:27:19.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.866+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/2853136122 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x564e0a22d000 msgr2=0x564e09fad600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:19.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.866+0000 7fccfe5dc7c0 1 --2- 172.21.15.16:0/2853136122 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x564e0a22d000 0x564e09fad600 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.866+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/2853136122 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x564e0a259000 msgr2=0x564e0a370b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:19.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.866+0000 7fccfe5dc7c0 1 --2- 172.21.15.16:0/2853136122 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x564e0a259000 0x564e0a370b00 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.866+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/2853136122 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x564e0a258800 msgr2=0x564e0a370000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:19.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.866+0000 7fccfe5dc7c0 1 --2- 172.21.15.16:0/2853136122 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x564e0a258800 0x564e0a370000 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.866+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/2853136122 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x564e0a22cc00 msgr2=0x564e09fadb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:19.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.867+0000 7fccfe5dc7c0 1 --2- 172.21.15.16:0/2853136122 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x564e0a22cc00 0x564e09fadb80 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.867+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/2853136122 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x564e0a259400 msgr2=0x564e0a371080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:19.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.867+0000 7fccfe5dc7c0 1 --2- 172.21.15.16:0/2853136122 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x564e0a259400 0x564e0a371080 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.867+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/2853136122 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x564e0a380000 msgr2=0x564e0a372100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:19.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.867+0000 7fccfe5dc7c0 1 --2- 172.21.15.16:0/2853136122 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x564e0a380000 0x564e0a372100 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.867+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/2853136122 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x564e0a22c800 msgr2=0x564e09fae100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:19.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.867+0000 7fccfe5dc7c0 1 --2- 172.21.15.16:0/2853136122 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x564e0a22c800 0x564e09fae100 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.867+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/2853136122 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x564e0a258c00 msgr2=0x564e0a370580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:19.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.867+0000 7fccfe5dc7c0 1 --2- 172.21.15.16:0/2853136122 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x564e0a258c00 0x564e0a370580 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.867+0000 7fccfc58a700 1 -- 172.21.15.16:0/2853136122 reap_dead start 2023-05-20T01:27:19.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.867+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/2853136122 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x564e0a258400 msgr2=0x564e09faf700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:19.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.867+0000 7fccfe5dc7c0 1 --2- 172.21.15.16:0/2853136122 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x564e0a258400 0x564e09faf700 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x564e0a2382d0 tx=0x564e0ada4c00 comp rx=0 tx=0).stop 2023-05-20T01:27:19.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.867+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/2853136122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e0a22c000 msgr2=0x564e09faec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:19.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.867+0000 7fccfe5dc7c0 1 --2- 172.21.15.16:0/2853136122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e0a22c000 0x564e09faec00 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x564e0a239410 tx=0x564e0ace8060 comp rx=0 tx=0).stop 2023-05-20T01:27:19.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.867+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/2853136122 shutdown_connections 2023-05-20T01:27:19.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.867+0000 7fccfe5dc7c0 1 --2- 172.21.15.16:0/2853136122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e0a22c000 0x564e09faec00 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.867+0000 7fccfe5dc7c0 1 --2- 172.21.15.16:0/2853136122 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x564e0a258400 0x564e09faf700 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.867+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/2853136122 >> 172.21.15.16:0/2853136122 conn(0x564e09f13c00 msgr2=0x564e09fef800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:27:19.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.867+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/2853136122 shutdown_connections 2023-05-20T01:27:19.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.867+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/2853136122 wait complete. 2023-05-20T01:27:19.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.867+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/4046560703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e09f13000 msgr2=0x564e09fad080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:19.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.867+0000 7fccfe5dc7c0 1 --2- 172.21.15.16:0/4046560703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e09f13000 0x564e09fad080 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x564e0acdee10 tx=0x564e0ace8480 comp rx=0 tx=0).stop 2023-05-20T01:27:19.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.867+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/4046560703 shutdown_connections 2023-05-20T01:27:19.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.867+0000 7fccfe5dc7c0 1 --2- 172.21.15.16:0/4046560703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e09f13000 0x564e09fad080 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.867+0000 7fccfe5dc7c0 1 --2- 172.21.15.16:0/4046560703 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564e09f12800 0x564e09fac580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.867+0000 7fccfe5dc7c0 1 --2- 172.21.15.16:0/4046560703 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x564e09f12c00 0x564e09facb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.867+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/4046560703 >> 172.21.15.16:0/4046560703 conn(0x564e09f12400 msgr2=0x564e09fef000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:27:19.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.867+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/4046560703 shutdown_connections 2023-05-20T01:27:19.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.867+0000 7fccfe5dc7c0 1 -- 172.21.15.16:0/4046560703 wait complete. 2023-05-20T01:27:19.928 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_inodes'] in time 0:00:00.209917, result: 2023-05-20T01:27:19.928 DEBUG:teuthology.orchestra.run.smithi016:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_links 2023-05-20T01:27:19.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.966+0000 7f6b24d3d7c0 1 -- start start 2023-05-20T01:27:19.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.966+0000 7f6b24d3d7c0 1 Processor -- start 2023-05-20T01:27:19.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.966+0000 7f6b24d3d7c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564bc30c1800 0x564bc315b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:19.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.966+0000 7f6b24d3d7c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x564bc30c1400 0x564bc315ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:19.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.966+0000 7f6b24d3d7c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bc30c0c00 0x564bc315b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:19.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.966+0000 7f6b24d3d7c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bc30fdcc0 con 0x564bc30c0c00 2023-05-20T01:27:19.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.966+0000 7f6b24d3d7c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bc30fd400 con 0x564bc30c1800 2023-05-20T01:27:19.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.966+0000 7f6b24d3d7c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x564bc30fc000 con 0x564bc30c1400 2023-05-20T01:27:19.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.967+0000 7f6b234ec700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bc30c0c00 0x564bc315b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:19.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.967+0000 7f6b234ec700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bc30c0c00 0x564bc315b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39022/0 (socket says 172.21.15.16:39022) 2023-05-20T01:27:19.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.967+0000 7f6b234ec700 1 -- 172.21.15.16:0/1713335594 learned_addr learned my addr 172.21.15.16:0/1713335594 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:27:19.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.967+0000 7f6b22ceb700 1 --2- 172.21.15.16:0/1713335594 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564bc30c1800 0x564bc315b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:19.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.967+0000 7f6b224ea700 1 --2- 172.21.15.16:0/1713335594 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x564bc30c1400 0x564bc315ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:19.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.967+0000 7f6b234ec700 1 -- 172.21.15.16:0/1713335594 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x564bc30c1400 msgr2=0x564bc315ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:19.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.967+0000 7f6b234ec700 1 --2- 172.21.15.16:0/1713335594 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x564bc30c1400 0x564bc315ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.967+0000 7f6b234ec700 1 -- 172.21.15.16:0/1713335594 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564bc30c1800 msgr2=0x564bc315b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:19.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.967+0000 7f6b234ec700 1 --2- 172.21.15.16:0/1713335594 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564bc30c1800 0x564bc315b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.967+0000 7f6b234ec700 1 -- 172.21.15.16:0/1713335594 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564bc30e5d40 con 0x564bc30c0c00 2023-05-20T01:27:19.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.967+0000 7f6b234ec700 1 --2- 172.21.15.16:0/1713335594 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bc30c0c00 0x564bc315b600 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x564bc321cc60 tx=0x564bc3eb6fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42882d216a763961 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:19.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.967+0000 7f6b204e6700 1 -- 172.21.15.16:0/1713335594 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x564bc30f5600 con 0x564bc30c0c00 2023-05-20T01:27:19.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.967+0000 7f6b204e6700 1 -- 172.21.15.16:0/1713335594 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564bc3112c00 con 0x564bc30c0c00 2023-05-20T01:27:19.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.967+0000 7f6b204e6700 1 -- 172.21.15.16:0/1713335594 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x564bc30f5080 con 0x564bc30c0c00 2023-05-20T01:27:19.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.967+0000 7f6b24d3d7c0 1 -- 172.21.15.16:0/1713335594 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564bc3eb0000 con 0x564bc30c0c00 2023-05-20T01:27:19.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.967+0000 7f6b24d3d7c0 1 -- 172.21.15.16:0/1713335594 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x564bc3eb01a0 con 0x564bc30c0c00 2023-05-20T01:27:19.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.967+0000 7f6b24d3d7c0 4 waiting for MDS map... 2023-05-20T01:27:19.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.967+0000 7f6b204e6700 1 -- 172.21.15.16:0/1713335594 <== mon.0 v2:172.21.15.16:3300/0 4 ==== osd_map(119..119 src has 1..119) v4 ==== 6987+0+0 (secure 0 0 0) 0x564bc3102c40 con 0x564bc30c0c00 2023-05-20T01:27:19.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.967+0000 7f6b204e6700 1 -- 172.21.15.16:0/1713335594 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 84) v1 ==== 3126+0+0 (secure 0 0 0) 0x564bc3f54300 con 0x564bc30c0c00 2023-05-20T01:27:19.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.967+0000 7f6b24d3d7c0 4 Got MDS map 84 2023-05-20T01:27:19.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.968+0000 7f6b24d3d7c0 4 datascan.main: Using metadata pool output 2023-05-20T01:27:19.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.968+0000 7f6b24d3d7c0 4 datascan.main: connecting to RADOS... 2023-05-20T01:27:19.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.968+0000 7f6b24d3d7c0 1 Processor -- start 2023-05-20T01:27:19.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.968+0000 7f6b24d3d7c0 1 -- start start 2023-05-20T01:27:19.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.969+0000 7f6b24d3d7c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564bc30c1c00 0x564bc315a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:19.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.969+0000 7f6b24d3d7c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bc30c1000 0x564bc315c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:19.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.969+0000 7f6b24d3d7c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x564bc322ac00 0x564bc315c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:19.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.969+0000 7f6b24d3d7c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bc30fcc80 con 0x564bc30c1000 2023-05-20T01:27:19.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.969+0000 7f6b24d3d7c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bc30fde00 con 0x564bc30c1c00 2023-05-20T01:27:19.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.969+0000 7f6b24d3d7c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x564bc30fd400 con 0x564bc322ac00 2023-05-20T01:27:19.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.969+0000 7f6b22ceb700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bc30c1000 0x564bc315c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:19.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.969+0000 7f6b22ceb700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bc30c1000 0x564bc315c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39028/0 (socket says 172.21.15.16:39028) 2023-05-20T01:27:19.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.969+0000 7f6b22ceb700 1 -- 172.21.15.16:0/474443773 learned_addr learned my addr 172.21.15.16:0/474443773 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:27:19.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.969+0000 7f6b224ea700 1 --2- 172.21.15.16:0/474443773 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564bc30c1c00 0x564bc315a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:19.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.969+0000 7f6b224ea700 1 --2- 172.21.15.16:0/474443773 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x564bc322ac00 0x564bc315c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:19.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.969+0000 7f6b22ceb700 1 -- 172.21.15.16:0/474443773 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x564bc322ac00 msgr2=0x564bc315c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:19.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.969+0000 7f6b22ceb700 1 --2- 172.21.15.16:0/474443773 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x564bc322ac00 0x564bc315c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.969+0000 7f6b22ceb700 1 -- 172.21.15.16:0/474443773 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564bc30c1c00 msgr2=0x564bc315a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:19.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.969+0000 7f6b22ceb700 1 --2- 172.21.15.16:0/474443773 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564bc30c1c00 0x564bc315a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.969+0000 7f6b22ceb700 1 -- 172.21.15.16:0/474443773 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564bc3eb0340 con 0x564bc30c1000 2023-05-20T01:27:19.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.969+0000 7f6b22ceb700 1 --2- 172.21.15.16:0/474443773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bc30c1000 0x564bc315c100 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x564bc33f8030 tx=0x564bc3eb6000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb85089fe7e3497 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:19.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.969+0000 7f6b1e4e2700 1 -- 172.21.15.16:0/474443773 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x564bc30f51e0 con 0x564bc30c1000 2023-05-20T01:27:19.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.969+0000 7f6b1e4e2700 1 -- 172.21.15.16:0/474443773 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564bc3113980 con 0x564bc30c1000 2023-05-20T01:27:19.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.969+0000 7f6b1e4e2700 1 -- 172.21.15.16:0/474443773 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x564bc30f5600 con 0x564bc30c1000 2023-05-20T01:27:19.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.970+0000 7f6b24d3d7c0 1 -- 172.21.15.16:0/474443773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bc30c1000 msgr2=0x564bc315c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:19.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.970+0000 7f6b24d3d7c0 1 --2- 172.21.15.16:0/474443773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bc30c1000 0x564bc315c100 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x564bc33f8030 tx=0x564bc3eb6000 comp rx=0 tx=0).stop 2023-05-20T01:27:19.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.970+0000 7f6b24d3d7c0 1 -- 172.21.15.16:0/474443773 shutdown_connections 2023-05-20T01:27:19.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.970+0000 7f6b24d3d7c0 1 --2- 172.21.15.16:0/474443773 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x564bc322ac00 0x564bc315c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.970+0000 7f6b24d3d7c0 1 --2- 172.21.15.16:0/474443773 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564bc30c1c00 0x564bc315a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.970+0000 7f6b24d3d7c0 1 --2- 172.21.15.16:0/474443773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bc30c1000 0x564bc315c100 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.970+0000 7f6b24d3d7c0 1 -- 172.21.15.16:0/474443773 >> 172.21.15.16:0/474443773 conn(0x564bc30c0800 msgr2=0x564bc319d000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:27:19.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.970+0000 7f6b24d3d7c0 1 -- 172.21.15.16:0/474443773 shutdown_connections 2023-05-20T01:27:19.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.970+0000 7f6b24d3d7c0 1 -- 172.21.15.16:0/474443773 wait complete. 2023-05-20T01:27:19.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.970+0000 7f6b24d3d7c0 1 Processor -- start 2023-05-20T01:27:19.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.970+0000 7f6b24d3d7c0 1 -- start start 2023-05-20T01:27:19.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.971+0000 7f6b24d3d7c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564bc30c1c00 0x564bc315d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:19.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.971+0000 7f6b24d3d7c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x564bc30c1000 0x564bc315d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:19.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.971+0000 7f6b24d3d7c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bc322bc00 0x564bc315cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:19.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.971+0000 7f6b24d3d7c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bc30fdcc0 con 0x564bc322bc00 2023-05-20T01:27:19.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.971+0000 7f6b24d3d7c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bc30fd180 con 0x564bc30c1c00 2023-05-20T01:27:19.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.971+0000 7f6b24d3d7c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x564bc30fd040 con 0x564bc30c1000 2023-05-20T01:27:19.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.971+0000 7f6b224ea700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bc322bc00 0x564bc315cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:19.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.971+0000 7f6b224ea700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bc322bc00 0x564bc315cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39036/0 (socket says 172.21.15.16:39036) 2023-05-20T01:27:19.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.971+0000 7f6b224ea700 1 -- 172.21.15.16:0/3121821003 learned_addr learned my addr 172.21.15.16:0/3121821003 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:27:19.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.971+0000 7f6b224ea700 1 --2- 172.21.15.16:0/3121821003 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564bc30c1c00 0x564bc315d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:19.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.971+0000 7f6b22ceb700 1 --2- 172.21.15.16:0/3121821003 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x564bc30c1000 0x564bc315d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:19.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.971+0000 7f6b224ea700 1 -- 172.21.15.16:0/3121821003 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x564bc30c1000 msgr2=0x564bc315d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:19.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.971+0000 7f6b224ea700 1 --2- 172.21.15.16:0/3121821003 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x564bc30c1000 0x564bc315d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.971+0000 7f6b224ea700 1 -- 172.21.15.16:0/3121821003 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564bc30c1c00 msgr2=0x564bc315d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:19.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.971+0000 7f6b224ea700 1 --2- 172.21.15.16:0/3121821003 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564bc30c1c00 0x564bc315d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:19.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.971+0000 7f6b224ea700 1 -- 172.21.15.16:0/3121821003 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564bc3eb0680 con 0x564bc322bc00 2023-05-20T01:27:19.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.972+0000 7f6b224ea700 1 --2- 172.21.15.16:0/3121821003 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bc322bc00 0x564bc315cc00 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x564bc33f9410 tx=0x564bc3f62de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:19.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.972+0000 7f6b1ccdf700 1 -- 172.21.15.16:0/3121821003 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x564bc30f51e0 con 0x564bc322bc00 2023-05-20T01:27:19.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.972+0000 7f6b1ccdf700 1 -- 172.21.15.16:0/3121821003 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564bc3113800 con 0x564bc322bc00 2023-05-20T01:27:19.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.972+0000 7f6b24d3d7c0 1 -- 172.21.15.16:0/3121821003 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x564bc3eb04e0 con 0x564bc322bc00 2023-05-20T01:27:19.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.972+0000 7f6b1ccdf700 1 -- 172.21.15.16:0/3121821003 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x564bc30f5600 con 0x564bc322bc00 2023-05-20T01:27:19.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.972+0000 7f6b24d3d7c0 1 -- 172.21.15.16:0/3121821003 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564bc3eb0820 con 0x564bc322bc00 2023-05-20T01:27:19.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.972+0000 7f6b24d3d7c0 4 datascan.init: resolving metadata pool 15 2023-05-20T01:27:19.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.973+0000 7f6b1ccdf700 1 -- 172.21.15.16:0/3121821003 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x564bc3122840 con 0x564bc322bc00 2023-05-20T01:27:19.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.973+0000 7f6b1ccdf700 1 --2- 172.21.15.16:0/3121821003 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x564bc33ee400 0x564bc34f8000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:19.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.973+0000 7f6b1ccdf700 1 -- 172.21.15.16:0/3121821003 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6987+0+0 (secure 0 0 0) 0x564bc3102a80 con 0x564bc322bc00 2023-05-20T01:27:19.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.973+0000 7f6b22ceb700 1 --2- 172.21.15.16:0/3121821003 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x564bc33ee400 0x564bc34f8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:19.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.974+0000 7f6b24d3d7c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-05-20T01:27:19.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.974+0000 7f6b24d3d7c0 4 datascan.main: resolving metadata pool 15 2023-05-20T01:27:19.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.974+0000 7f6b24d3d7c0 1 --2- 172.21.15.16:0/3121821003 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x564bc322b800 0x564bc315c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:19.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.974+0000 7f6b24d3d7c0 1 -- 172.21.15.16:0/3121821003 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:1 15.0 15:00000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564bc3f58700 con 0x564bc322b800 2023-05-20T01:27:19.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.974+0000 7f6b224ea700 1 --2- 172.21.15.16:0/3121821003 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x564bc322b800 0x564bc315c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:19.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.974+0000 7f6b22ceb700 1 --2- 172.21.15.16:0/3121821003 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x564bc33ee400 0x564bc34f8000 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x564bc33f8030 tx=0x564bc3e59440 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:19.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.974+0000 7f6b224ea700 1 --2- 172.21.15.16:0/3121821003 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x564bc322b800 0x564bc315c680 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:19.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.974+0000 7f6b224ea700 1 -- 172.21.15.16:0/3121821003 <== osd.2 v2:172.21.15.16:6816/293187030 1 ==== osd_op_reply(1 [pgnls start_epoch 119 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564bc33dd440 con 0x564bc322b800 2023-05-20T01:27:19.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.974+0000 7f6b224ea700 1 --2- 172.21.15.16:0/3121821003 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x564bc33ee800 0x564bc34f8580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:19.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.974+0000 7f6b224ea700 1 -- 172.21.15.16:0/3121821003 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:2 15.10 15:08000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564bc3f58700 con 0x564bc33ee800 2023-05-20T01:27:19.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.975+0000 7f6b234ec700 1 --2- 172.21.15.16:0/3121821003 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x564bc33ee800 0x564bc34f8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:19.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.975+0000 7f6b234ec700 1 --2- 172.21.15.16:0/3121821003 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x564bc33ee800 0x564bc34f8580 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:19.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.975+0000 7f6b234ec700 1 -- 172.21.15.16:0/3121821003 <== osd.4 v2:172.21.15.132:6817/506317223 1 ==== osd_op_reply(2 [pgnls start_epoch 119 out=81b] v119'23 uv23 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x564bc33dd680 con 0x564bc33ee800 2023-05-20T01:27:19.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.975+0000 7f6b234ec700 1 -- 172.21.15.16:0/3121821003 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:3 15.8 15:10000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564bc3f58700 con 0x564bc322b800 2023-05-20T01:27:19.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.976+0000 7f6b224ea700 1 -- 172.21.15.16:0/3121821003 <== osd.2 v2:172.21.15.16:6816/293187030 2 ==== osd_op_reply(3 [pgnls start_epoch 119 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564bc33dd440 con 0x564bc322b800 2023-05-20T01:27:19.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.976+0000 7f6b224ea700 1 --2- 172.21.15.16:0/3121821003 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x564bc33eec00 0x564bc34f8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:19.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.976+0000 7f6b224ea700 1 -- 172.21.15.16:0/3121821003 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:4 15.18 15:18000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564bc3f58700 con 0x564bc33eec00 2023-05-20T01:27:19.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.976+0000 7f6b22ceb700 1 --2- 172.21.15.16:0/3121821003 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x564bc33eec00 0x564bc34f8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:19.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.976+0000 7f6b22ceb700 1 --2- 172.21.15.16:0/3121821003 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x564bc33eec00 0x564bc34f8b00 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:20.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.976+0000 7f6b22ceb700 1 -- 172.21.15.16:0/3121821003 <== osd.3 v2:172.21.15.16:6800/3669781890 1 ==== osd_op_reply(4 [pgnls start_epoch 119 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564bc33dd8c0 con 0x564bc33eec00 2023-05-20T01:27:20.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.976+0000 7f6b22ceb700 1 --2- 172.21.15.16:0/3121821003 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x564bc33ef000 0x564bc34f9600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:20.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.976+0000 7f6b22ceb700 1 -- 172.21.15.16:0/3121821003 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:5 15.4 15:20000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564bc3f58700 con 0x564bc33ef000 2023-05-20T01:27:20.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.977+0000 7f6b224ea700 1 --2- 172.21.15.16:0/3121821003 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x564bc33ef000 0x564bc34f9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:20.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.977+0000 7f6b224ea700 1 --2- 172.21.15.16:0/3121821003 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x564bc33ef000 0x564bc34f9600 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:20.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.977+0000 7f6b224ea700 1 -- 172.21.15.16:0/3121821003 <== osd.1 v2:172.21.15.16:6808/3860264220 1 ==== osd_op_reply(5 [pgnls start_epoch 119 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564bc33dd440 con 0x564bc33ef000 2023-05-20T01:27:20.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.977+0000 7f6b224ea700 1 --2- 172.21.15.16:0/3121821003 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x564bc33ef400 0x564bc34f9080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:20.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.977+0000 7f6b224ea700 1 -- 172.21.15.16:0/3121821003 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:6 15.14 15:28000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564bc3f58700 con 0x564bc33ef400 2023-05-20T01:27:20.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.977+0000 7f6b234ec700 1 --2- 172.21.15.16:0/3121821003 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x564bc33ef400 0x564bc34f9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:20.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.978+0000 7f6b234ec700 1 --2- 172.21.15.16:0/3121821003 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x564bc33ef400 0x564bc34f9080 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:20.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.978+0000 7f6b234ec700 1 -- 172.21.15.16:0/3121821003 <== osd.6 v2:172.21.15.132:6801/727581123 1 ==== osd_op_reply(6 [pgnls start_epoch 119 out=125b] v119'48 uv48 ondisk = 1) v8 ==== 144+0+125 (crc 0 0 0) 0x564bc33dd680 con 0x564bc33ef400 2023-05-20T01:27:20.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.978+0000 7f6b234ec700 1 --2- 172.21.15.16:0/3121821003 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x564bc322a400 0x564bc315bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:20.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.978+0000 7f6b234ec700 1 -- 172.21.15.16:0/3121821003 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:7 15.c 15:30000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564bc3f58700 con 0x564bc322a400 2023-05-20T01:27:20.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.978+0000 7f6b22ceb700 1 --2- 172.21.15.16:0/3121821003 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x564bc322a400 0x564bc315bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:20.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.978+0000 7f6b22ceb700 1 --2- 172.21.15.16:0/3121821003 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x564bc322a400 0x564bc315bb80 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:20.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.979+0000 7f6b22ceb700 1 -- 172.21.15.16:0/3121821003 <== osd.5 v2:172.21.15.132:6816/1844689995 1 ==== osd_op_reply(7 [pgnls start_epoch 119 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564bc33dd8c0 con 0x564bc322a400 2023-05-20T01:27:20.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.979+0000 7f6b22ceb700 1 -- 172.21.15.16:0/3121821003 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:8 15.1c 15:38000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564bc3f58700 con 0x564bc33ef000 2023-05-20T01:27:20.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.979+0000 7f6b224ea700 1 -- 172.21.15.16:0/3121821003 <== osd.1 v2:172.21.15.16:6808/3860264220 2 ==== osd_op_reply(8 [pgnls start_epoch 119 out=49b] v116'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564bc33dd440 con 0x564bc33ef000 2023-05-20T01:27:20.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.979+0000 7f6b224ea700 1 -- 172.21.15.16:0/3121821003 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:9 15.2 15:40000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564bc3f58700 con 0x564bc322b800 2023-05-20T01:27:20.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.979+0000 7f6b224ea700 1 -- 172.21.15.16:0/3121821003 <== osd.2 v2:172.21.15.16:6816/293187030 3 ==== osd_op_reply(9 [pgnls start_epoch 119 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564bc33dd440 con 0x564bc322b800 2023-05-20T01:27:20.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.979+0000 7f6b224ea700 1 -- 172.21.15.16:0/3121821003 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:10 15.12 15:48000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564bc3f58700 con 0x564bc33eec00 2023-05-20T01:27:20.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.979+0000 7f6b22ceb700 1 -- 172.21.15.16:0/3121821003 <== osd.3 v2:172.21.15.16:6800/3669781890 2 ==== osd_op_reply(10 [pgnls start_epoch 119 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564bc33dd8c0 con 0x564bc33eec00 2023-05-20T01:27:20.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.979+0000 7f6b22ceb700 1 -- 172.21.15.16:0/3121821003 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:11 15.a 15:50000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564bc3f58700 con 0x564bc322b800 2023-05-20T01:27:20.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.980+0000 7f6b224ea700 1 -- 172.21.15.16:0/3121821003 <== osd.2 v2:172.21.15.16:6816/293187030 4 ==== osd_op_reply(11 [pgnls start_epoch 119 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564bc33dd440 con 0x564bc322b800 2023-05-20T01:27:20.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.980+0000 7f6b224ea700 1 -- 172.21.15.16:0/3121821003 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:12 15.1a 15:58000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564bc3f58700 con 0x564bc322a400 2023-05-20T01:27:20.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.980+0000 7f6b22ceb700 1 -- 172.21.15.16:0/3121821003 <== osd.5 v2:172.21.15.132:6816/1844689995 2 ==== osd_op_reply(12 [pgnls start_epoch 119 out=49b] v119'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564bc33dd8c0 con 0x564bc322a400 2023-05-20T01:27:20.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.980+0000 7f6b22ceb700 1 --2- 172.21.15.16:0/3121821003 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x564bc3516000 0x564bc34f9b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:20.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.980+0000 7f6b22ceb700 1 -- 172.21.15.16:0/3121821003 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:13 15.6 15:60000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564bc3f58700 con 0x564bc3516000 2023-05-20T01:27:20.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.980+0000 7f6b224ea700 1 --2- 172.21.15.16:0/3121821003 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x564bc3516000 0x564bc34f9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:20.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.980+0000 7f6b224ea700 1 --2- 172.21.15.16:0/3121821003 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x564bc3516000 0x564bc34f9b80 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:20.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.981+0000 7f6b224ea700 1 -- 172.21.15.16:0/3121821003 <== osd.0 v2:172.21.15.16:6824/3923634179 1 ==== osd_op_reply(13 [pgnls start_epoch 119 out=49b] v116'7 uv7 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564bc33dd440 con 0x564bc3516000 2023-05-20T01:27:20.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.981+0000 7f6b224ea700 1 -- 172.21.15.16:0/3121821003 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:14 15.16 15:68000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564bc3f58700 con 0x564bc3516000 2023-05-20T01:27:20.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.981+0000 7f6b224ea700 1 -- 172.21.15.16:0/3121821003 <== osd.0 v2:172.21.15.16:6824/3923634179 2 ==== osd_op_reply(14 [pgnls start_epoch 119 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564bc33dd440 con 0x564bc3516000 2023-05-20T01:27:20.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.981+0000 7f6b224ea700 1 -- 172.21.15.16:0/3121821003 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:15 15.e 15:70000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564bc3f58700 con 0x564bc322b800 2023-05-20T01:27:20.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.981+0000 7f6b224ea700 1 -- 172.21.15.16:0/3121821003 <== osd.2 v2:172.21.15.16:6816/293187030 5 ==== osd_op_reply(15 [pgnls start_epoch 119 out=49b] v116'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564bc33dd440 con 0x564bc322b800 2023-05-20T01:27:20.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.981+0000 7f6b224ea700 1 -- 172.21.15.16:0/3121821003 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:16 15.1e 15:78000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564bc3f58700 con 0x564bc322b800 2023-05-20T01:27:20.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.981+0000 7f6b224ea700 1 -- 172.21.15.16:0/3121821003 <== osd.2 v2:172.21.15.16:6816/293187030 6 ==== osd_op_reply(16 [pgnls start_epoch 119 out=49b] v119'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564bc33dd440 con 0x564bc322b800 2023-05-20T01:27:20.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.981+0000 7f6b224ea700 1 -- 172.21.15.16:0/3121821003 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:17 15.1 15:80000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564bc3f58700 con 0x564bc33ef400 2023-05-20T01:27:20.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.982+0000 7f6b234ec700 1 -- 172.21.15.16:0/3121821003 <== osd.6 v2:172.21.15.132:6801/727581123 2 ==== osd_op_reply(17 [pgnls start_epoch 119 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564bc33dd680 con 0x564bc33ef400 2023-05-20T01:27:20.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.982+0000 7f6b234ec700 1 -- 172.21.15.16:0/3121821003 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:18 15.11 15:88000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564bc3f58700 con 0x564bc322b800 2023-05-20T01:27:20.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.982+0000 7f6b224ea700 1 -- 172.21.15.16:0/3121821003 <== osd.2 v2:172.21.15.16:6816/293187030 7 ==== osd_op_reply(18 [pgnls start_epoch 119 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564bc33dd440 con 0x564bc322b800 2023-05-20T01:27:20.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.982+0000 7f6b224ea700 1 -- 172.21.15.16:0/3121821003 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:19 15.9 15:90000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564bc3f58700 con 0x564bc33ef400 2023-05-20T01:27:20.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.982+0000 7f6b234ec700 1 -- 172.21.15.16:0/3121821003 <== osd.6 v2:172.21.15.132:6801/727581123 3 ==== osd_op_reply(19 [pgnls start_epoch 119 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564bc33dd680 con 0x564bc33ef400 2023-05-20T01:27:20.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.982+0000 7f6b234ec700 1 -- 172.21.15.16:0/3121821003 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:20 15.19 15:98000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564bc3f58700 con 0x564bc33ee800 2023-05-20T01:27:20.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.983+0000 7f6b234ec700 1 -- 172.21.15.16:0/3121821003 <== osd.4 v2:172.21.15.132:6817/506317223 2 ==== osd_op_reply(20 [pgnls start_epoch 119 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564bc33dd680 con 0x564bc33ee800 2023-05-20T01:27:20.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.983+0000 7f6b234ec700 1 -- 172.21.15.16:0/3121821003 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:21 15.5 15:a0000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564bc3f58700 con 0x564bc33ef400 2023-05-20T01:27:20.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.983+0000 7f6b234ec700 1 -- 172.21.15.16:0/3121821003 <== osd.6 v2:172.21.15.132:6801/727581123 4 ==== osd_op_reply(21 [pgnls start_epoch 119 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564bc33dd680 con 0x564bc33ef400 2023-05-20T01:27:20.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.983+0000 7f6b234ec700 1 --2- 172.21.15.16:0/3121821003 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x564bc322a800 0x564bc34fa680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:20.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.983+0000 7f6b234ec700 1 -- 172.21.15.16:0/3121821003 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:22 15.15 15:a8000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564bc3f58700 con 0x564bc322a800 2023-05-20T01:27:20.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.983+0000 7f6b234ec700 1 --2- 172.21.15.16:0/3121821003 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x564bc322a800 0x564bc34fa680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:20.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.983+0000 7f6b234ec700 1 --2- 172.21.15.16:0/3121821003 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x564bc322a800 0x564bc34fa680 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:20.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.984+0000 7f6b234ec700 1 -- 172.21.15.16:0/3121821003 <== osd.7 v2:172.21.15.132:6800/466001149 1 ==== osd_op_reply(22 [pgnls start_epoch 119 out=73b] v116'6 uv6 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x564bc33dd680 con 0x564bc322a800 2023-05-20T01:27:20.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.984+0000 7f6b234ec700 1 -- 172.21.15.16:0/3121821003 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:23 15.d 15:b0000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564bc3f58700 con 0x564bc322a800 2023-05-20T01:27:20.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.984+0000 7f6b234ec700 1 -- 172.21.15.16:0/3121821003 <== osd.7 v2:172.21.15.132:6800/466001149 2 ==== osd_op_reply(23 [pgnls start_epoch 119 out=74b] v119'3 uv3 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x564bc33dd680 con 0x564bc322a800 2023-05-20T01:27:20.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.984+0000 7f6b234ec700 1 -- 172.21.15.16:0/3121821003 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:24 15.1d 15:b8000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564bc3f58700 con 0x564bc33ee800 2023-05-20T01:27:20.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.984+0000 7f6b234ec700 1 -- 172.21.15.16:0/3121821003 <== osd.4 v2:172.21.15.132:6817/506317223 3 ==== osd_op_reply(24 [pgnls start_epoch 119 out=49b] v116'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564bc33dd680 con 0x564bc33ee800 2023-05-20T01:27:20.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.984+0000 7f6b234ec700 1 -- 172.21.15.16:0/3121821003 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:25 15.3 15:c0000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564bc3f58700 con 0x564bc33ee800 2023-05-20T01:27:20.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.984+0000 7f6b234ec700 1 -- 172.21.15.16:0/3121821003 <== osd.4 v2:172.21.15.132:6817/506317223 4 ==== osd_op_reply(25 [pgnls start_epoch 119 out=71b] v119'2 uv2 ondisk = 1) v8 ==== 144+0+71 (crc 0 0 0) 0x564bc33dd680 con 0x564bc33ee800 2023-05-20T01:27:20.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.984+0000 7f6b234ec700 1 -- 172.21.15.16:0/3121821003 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:26 15.13 15:c8000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564bc3f58700 con 0x564bc33ef000 2023-05-20T01:27:20.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.985+0000 7f6b224ea700 1 -- 172.21.15.16:0/3121821003 <== osd.1 v2:172.21.15.16:6808/3860264220 3 ==== osd_op_reply(26 [pgnls start_epoch 119 out=98b] v119'7 uv7 ondisk = 1) v8 ==== 144+0+98 (crc 0 0 0) 0x564bc33dd440 con 0x564bc33ef000 2023-05-20T01:27:20.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.985+0000 7f6b224ea700 1 -- 172.21.15.16:0/3121821003 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:27 15.b 15:d0000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564bc3f58700 con 0x564bc322b800 2023-05-20T01:27:20.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.985+0000 7f6b224ea700 1 -- 172.21.15.16:0/3121821003 <== osd.2 v2:172.21.15.16:6816/293187030 8 ==== osd_op_reply(27 [pgnls start_epoch 119 out=76b] v119'6 uv5 ondisk = 1) v8 ==== 144+0+76 (crc 0 0 0) 0x564bc33dd440 con 0x564bc322b800 2023-05-20T01:27:20.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.985+0000 7f6b224ea700 1 -- 172.21.15.16:0/3121821003 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:28 15.1b 15:d8000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564bc3f58700 con 0x564bc322a800 2023-05-20T01:27:20.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.985+0000 7f6b234ec700 1 -- 172.21.15.16:0/3121821003 <== osd.7 v2:172.21.15.132:6800/466001149 3 ==== osd_op_reply(28 [pgnls start_epoch 119 out=49b] v119'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564bc33dd680 con 0x564bc322a800 2023-05-20T01:27:20.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.985+0000 7f6b234ec700 1 -- 172.21.15.16:0/3121821003 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:29 15.7 15:e0000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564bc3f58700 con 0x564bc33ef000 2023-05-20T01:27:20.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.986+0000 7f6b224ea700 1 -- 172.21.15.16:0/3121821003 <== osd.1 v2:172.21.15.16:6808/3860264220 4 ==== osd_op_reply(29 [pgnls start_epoch 119 out=49b] v116'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564bc33dd440 con 0x564bc33ef000 2023-05-20T01:27:20.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.986+0000 7f6b224ea700 1 -- 172.21.15.16:0/3121821003 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:30 15.17 15:e8000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564bc3f58700 con 0x564bc33ee800 2023-05-20T01:27:20.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.986+0000 7f6b234ec700 1 -- 172.21.15.16:0/3121821003 <== osd.4 v2:172.21.15.132:6817/506317223 5 ==== osd_op_reply(30 [pgnls start_epoch 119 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564bc33dd680 con 0x564bc33ee800 2023-05-20T01:27:20.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.986+0000 7f6b234ec700 1 -- 172.21.15.16:0/3121821003 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:31 15.f 15:f0000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564bc3f58700 con 0x564bc33ef400 2023-05-20T01:27:20.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.986+0000 7f6b234ec700 1 -- 172.21.15.16:0/3121821003 <== osd.6 v2:172.21.15.132:6801/727581123 5 ==== osd_op_reply(31 [pgnls start_epoch 119 out=73b] v119'4 uv4 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x564bc33dd680 con 0x564bc33ef400 2023-05-20T01:27:20.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.986+0000 7f6b234ec700 1 -- 172.21.15.16:0/3121821003 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:32 15.1f 15:f8000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564bc3f58700 con 0x564bc33eec00 2023-05-20T01:27:20.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.987+0000 7f6b22ceb700 1 -- 172.21.15.16:0/3121821003 <== osd.3 v2:172.21.15.16:6800/3669781890 3 ==== osd_op_reply(32 [pgnls start_epoch 119 out=101b] v119'13 uv13 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x564bc33dd8c0 con 0x564bc33eec00 2023-05-20T01:27:20.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.987+0000 7f6b24d3d7c0 10 datascan.scan_links: step 1: handling object 10000000000.00000000 2023-05-20T01:27:20.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.987+0000 7f6b24d3d7c0 1 -- 172.21.15.16:0/3121821003 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:33 15.10 15:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x564bc3f58a80 con 0x564bc33ee800 2023-05-20T01:27:20.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.987+0000 7f6b234ec700 1 -- 172.21.15.16:0/3121821003 <== osd.4 v2:172.21.15.132:6817/506317223 6 ==== osd_op_reply(33 10000000000.00000000 [omap-get-vals out=3582b] v0'0 uv23 ondisk = 0) v8 ==== 164+0+3582 (crc 0 0 0) 0x564bc33dd680 con 0x564bc33ee800 2023-05-20T01:27:20.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.987+0000 7f6b24d3d7c0 20 datascan.scan_links: injected ino detected 2023-05-20T01:27:20.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.987+0000 7f6b24d3d7c0 20 datascan.scan_links: injected ino detected 2023-05-20T01:27:20.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.987+0000 7f6b24d3d7c0 20 datascan.scan_links: injected ino detected 2023-05-20T01:27:20.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.987+0000 7f6b24d3d7c0 20 datascan.scan_links: injected ino detected 2023-05-20T01:27:20.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.987+0000 7f6b24d3d7c0 20 datascan.scan_links: injected ino detected 2023-05-20T01:27:20.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.987+0000 7f6b24d3d7c0 20 datascan.scan_links: injected ino detected 2023-05-20T01:27:20.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.987+0000 7f6b24d3d7c0 20 datascan.scan_links: injected ino detected 2023-05-20T01:27:20.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.987+0000 7f6b24d3d7c0 10 datascan.scan_links: step 1: handling object 1.00000000.inode 2023-05-20T01:27:20.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.987+0000 7f6b24d3d7c0 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2023-05-20T01:27:20.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.987+0000 7f6b24d3d7c0 10 datascan.scan_links: step 1: handling object 200.00000000 2023-05-20T01:27:20.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.987+0000 7f6b24d3d7c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2023-05-20T01:27:20.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.987+0000 7f6b24d3d7c0 10 datascan.scan_links: step 1: handling object 200.00000001 2023-05-20T01:27:20.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.987+0000 7f6b24d3d7c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2023-05-20T01:27:20.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.987+0000 7f6b24d3d7c0 10 datascan.scan_links: step 1: handling object 500.00000000 2023-05-20T01:27:20.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.987+0000 7f6b24d3d7c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2023-05-20T01:27:20.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.987+0000 7f6b24d3d7c0 10 datascan.scan_links: step 1: handling object mds_snaptable 2023-05-20T01:27:20.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.987+0000 7f6b24d3d7c0 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2023-05-20T01:27:20.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.987+0000 7f6b24d3d7c0 10 datascan.scan_links: step 1: handling object 4.00000000 2023-05-20T01:27:20.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.987+0000 7f6b24d3d7c0 1 -- 172.21.15.16:0/3121821003 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:34 15.3 15:c0ce80d4:::4.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x564bc3f58e00 con 0x564bc33ee800 2023-05-20T01:27:20.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.988+0000 7f6b234ec700 1 -- 172.21.15.16:0/3121821003 <== osd.4 v2:172.21.15.132:6817/506317223 7 ==== osd_op_reply(34 4.00000000 [omap-get-vals out=513b] v0'0 uv2 ondisk = 0) v8 ==== 154+0+513 (crc 0 0 0) 0x564bc33dd680 con 0x564bc33ee800 2023-05-20T01:27:20.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.988+0000 7f6b24d3d7c0 20 datascan.scan_links: injected ino detected 2023-05-20T01:27:20.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.988+0000 7f6b24d3d7c0 10 datascan.scan_links: step 1: handling object mds0_inotable 2023-05-20T01:27:20.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.988+0000 7f6b24d3d7c0 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2023-05-20T01:27:20.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.988+0000 7f6b24d3d7c0 10 datascan.scan_links: step 1: handling object 100.00000000 2023-05-20T01:27:20.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.988+0000 7f6b24d3d7c0 1 -- 172.21.15.16:0/3121821003 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:35 15.13 15:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x564bc3f59180 con 0x564bc33ef000 2023-05-20T01:27:20.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.988+0000 7f6b224ea700 1 -- 172.21.15.16:0/3121821003 <== osd.1 v2:172.21.15.16:6808/3860264220 5 ==== osd_op_reply(35 100.00000000 [omap-get-vals out=5b] v0'0 uv7 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x564bc33dd440 con 0x564bc33ef000 2023-05-20T01:27:20.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.988+0000 7f6b24d3d7c0 10 datascan.scan_links: step 1: handling object mds0_sessionmap 2023-05-20T01:27:20.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.988+0000 7f6b24d3d7c0 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2023-05-20T01:27:20.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.988+0000 7f6b24d3d7c0 10 datascan.scan_links: step 1: handling object 400.00000000 2023-05-20T01:27:20.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.988+0000 7f6b24d3d7c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2023-05-20T01:27:20.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.988+0000 7f6b24d3d7c0 10 datascan.scan_links: step 1: handling object 100.00000000.inode 2023-05-20T01:27:20.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.988+0000 7f6b24d3d7c0 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2023-05-20T01:27:20.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.988+0000 7f6b24d3d7c0 10 datascan.scan_links: step 1: handling object 1.00000000 2023-05-20T01:27:20.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.988+0000 7f6b24d3d7c0 1 -- 172.21.15.16:0/3121821003 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:36 15.1f 15:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x564bc3f59500 con 0x564bc33eec00 2023-05-20T01:27:20.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.988+0000 7f6b22ceb700 1 -- 172.21.15.16:0/3121821003 <== osd.3 v2:172.21.15.16:6800/3669781890 4 ==== osd_op_reply(36 1.00000000 [omap-get-vals out=1016b] v0'0 uv13 ondisk = 0) v8 ==== 154+0+1016 (crc 0 0 0) 0x564bc33dd8c0 con 0x564bc33eec00 2023-05-20T01:27:20.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.988+0000 7f6b24d3d7c0 20 datascan.scan_links: injected ino detected 2023-05-20T01:27:20.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.988+0000 7f6b24d3d7c0 20 datascan.scan_links: injected ino detected 2023-05-20T01:27:20.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.988+0000 7f6b24d3d7c0 1 -- 172.21.15.16:0/3121821003 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:37 15.0 15:00000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564bc3f59880 con 0x564bc322b800 2023-05-20T01:27:20.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.988+0000 7f6b224ea700 1 -- 172.21.15.16:0/3121821003 <== osd.2 v2:172.21.15.16:6816/293187030 9 ==== osd_op_reply(37 [pgnls start_epoch 119 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564bc33dd440 con 0x564bc322b800 2023-05-20T01:27:20.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.988+0000 7f6b224ea700 1 -- 172.21.15.16:0/3121821003 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:38 15.10 15:08000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564bc3f59180 con 0x564bc33ee800 2023-05-20T01:27:20.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.989+0000 7f6b234ec700 1 -- 172.21.15.16:0/3121821003 <== osd.4 v2:172.21.15.132:6817/506317223 8 ==== osd_op_reply(38 [pgnls start_epoch 119 out=81b] v119'23 uv23 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x564bc33dd680 con 0x564bc33ee800 2023-05-20T01:27:20.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.989+0000 7f6b234ec700 1 -- 172.21.15.16:0/3121821003 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:39 15.8 15:10000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564bc3f58a80 con 0x564bc322b800 2023-05-20T01:27:20.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.989+0000 7f6b224ea700 1 -- 172.21.15.16:0/3121821003 <== osd.2 v2:172.21.15.16:6816/293187030 10 ==== osd_op_reply(39 [pgnls start_epoch 119 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564bc33dd440 con 0x564bc322b800 2023-05-20T01:27:20.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.989+0000 7f6b224ea700 1 -- 172.21.15.16:0/3121821003 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:40 15.18 15:18000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564bc3f58a80 con 0x564bc33eec00 2023-05-20T01:27:20.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.989+0000 7f6b22ceb700 1 -- 172.21.15.16:0/3121821003 <== osd.3 v2:172.21.15.16:6800/3669781890 5 ==== osd_op_reply(40 [pgnls start_epoch 119 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564bc33dd8c0 con 0x564bc33eec00 2023-05-20T01:27:20.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.989+0000 7f6b22ceb700 1 -- 172.21.15.16:0/3121821003 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:41 15.4 15:20000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564bc3f58a80 con 0x564bc33ef000 2023-05-20T01:27:20.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.990+0000 7f6b224ea700 1 -- 172.21.15.16:0/3121821003 <== osd.1 v2:172.21.15.16:6808/3860264220 6 ==== osd_op_reply(41 [pgnls start_epoch 119 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564bc33dd440 con 0x564bc33ef000 2023-05-20T01:27:20.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.990+0000 7f6b224ea700 1 -- 172.21.15.16:0/3121821003 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:42 15.14 15:28000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564bc3f58a80 con 0x564bc33ef400 2023-05-20T01:27:20.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.990+0000 7f6b234ec700 1 -- 172.21.15.16:0/3121821003 <== osd.6 v2:172.21.15.132:6801/727581123 6 ==== osd_op_reply(42 [pgnls start_epoch 119 out=125b] v119'48 uv48 ondisk = 1) v8 ==== 144+0+125 (crc 0 0 0) 0x564bc33dd680 con 0x564bc33ef400 2023-05-20T01:27:20.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.990+0000 7f6b234ec700 1 -- 172.21.15.16:0/3121821003 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:43 15.c 15:30000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564bc3f58a80 con 0x564bc322a400 2023-05-20T01:27:20.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.990+0000 7f6b22ceb700 1 -- 172.21.15.16:0/3121821003 <== osd.5 v2:172.21.15.132:6816/1844689995 3 ==== osd_op_reply(43 [pgnls start_epoch 119 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564bc33dd8c0 con 0x564bc322a400 2023-05-20T01:27:20.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.990+0000 7f6b22ceb700 1 -- 172.21.15.16:0/3121821003 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:44 15.1c 15:38000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564bc3f58a80 con 0x564bc33ef000 2023-05-20T01:27:20.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.990+0000 7f6b224ea700 1 -- 172.21.15.16:0/3121821003 <== osd.1 v2:172.21.15.16:6808/3860264220 7 ==== osd_op_reply(44 [pgnls start_epoch 119 out=49b] v116'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564bc33dd440 con 0x564bc33ef000 2023-05-20T01:27:20.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.990+0000 7f6b224ea700 1 -- 172.21.15.16:0/3121821003 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:45 15.2 15:40000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564bc3f58a80 con 0x564bc322b800 2023-05-20T01:27:20.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.991+0000 7f6b224ea700 1 -- 172.21.15.16:0/3121821003 <== osd.2 v2:172.21.15.16:6816/293187030 11 ==== osd_op_reply(45 [pgnls start_epoch 119 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564bc33dd440 con 0x564bc322b800 2023-05-20T01:27:20.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.991+0000 7f6b224ea700 1 -- 172.21.15.16:0/3121821003 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:46 15.12 15:48000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564bc3f58a80 con 0x564bc33eec00 2023-05-20T01:27:20.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.991+0000 7f6b22ceb700 1 -- 172.21.15.16:0/3121821003 <== osd.3 v2:172.21.15.16:6800/3669781890 6 ==== osd_op_reply(46 [pgnls start_epoch 119 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564bc33dd8c0 con 0x564bc33eec00 2023-05-20T01:27:20.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.991+0000 7f6b22ceb700 1 -- 172.21.15.16:0/3121821003 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:47 15.a 15:50000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564bc3f58a80 con 0x564bc322b800 2023-05-20T01:27:20.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.991+0000 7f6b224ea700 1 -- 172.21.15.16:0/3121821003 <== osd.2 v2:172.21.15.16:6816/293187030 12 ==== osd_op_reply(47 [pgnls start_epoch 119 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564bc33dd440 con 0x564bc322b800 2023-05-20T01:27:20.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.991+0000 7f6b224ea700 1 -- 172.21.15.16:0/3121821003 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:48 15.1a 15:58000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564bc3f58a80 con 0x564bc322a400 2023-05-20T01:27:20.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.991+0000 7f6b22ceb700 1 -- 172.21.15.16:0/3121821003 <== osd.5 v2:172.21.15.132:6816/1844689995 4 ==== osd_op_reply(48 [pgnls start_epoch 119 out=49b] v119'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564bc33dd8c0 con 0x564bc322a400 2023-05-20T01:27:20.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.991+0000 7f6b22ceb700 1 -- 172.21.15.16:0/3121821003 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:49 15.6 15:60000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564bc3f58a80 con 0x564bc3516000 2023-05-20T01:27:20.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.992+0000 7f6b224ea700 1 -- 172.21.15.16:0/3121821003 <== osd.0 v2:172.21.15.16:6824/3923634179 3 ==== osd_op_reply(49 [pgnls start_epoch 119 out=49b] v116'7 uv7 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564bc33dd440 con 0x564bc3516000 2023-05-20T01:27:20.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.992+0000 7f6b224ea700 1 -- 172.21.15.16:0/3121821003 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:50 15.16 15:68000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564bc3f58a80 con 0x564bc3516000 2023-05-20T01:27:20.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.992+0000 7f6b224ea700 1 -- 172.21.15.16:0/3121821003 <== osd.0 v2:172.21.15.16:6824/3923634179 4 ==== osd_op_reply(50 [pgnls start_epoch 119 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564bc33dd440 con 0x564bc3516000 2023-05-20T01:27:20.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.992+0000 7f6b224ea700 1 -- 172.21.15.16:0/3121821003 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:51 15.e 15:70000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564bc3f58a80 con 0x564bc322b800 2023-05-20T01:27:20.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.992+0000 7f6b224ea700 1 -- 172.21.15.16:0/3121821003 <== osd.2 v2:172.21.15.16:6816/293187030 13 ==== osd_op_reply(51 [pgnls start_epoch 119 out=49b] v116'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564bc33dd440 con 0x564bc322b800 2023-05-20T01:27:20.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.992+0000 7f6b224ea700 1 -- 172.21.15.16:0/3121821003 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:52 15.1e 15:78000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564bc3f58a80 con 0x564bc322b800 2023-05-20T01:27:20.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.992+0000 7f6b224ea700 1 -- 172.21.15.16:0/3121821003 <== osd.2 v2:172.21.15.16:6816/293187030 14 ==== osd_op_reply(52 [pgnls start_epoch 119 out=49b] v119'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564bc33dd440 con 0x564bc322b800 2023-05-20T01:27:20.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.992+0000 7f6b224ea700 1 -- 172.21.15.16:0/3121821003 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:53 15.1 15:80000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564bc3f58a80 con 0x564bc33ef400 2023-05-20T01:27:20.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.993+0000 7f6b234ec700 1 -- 172.21.15.16:0/3121821003 <== osd.6 v2:172.21.15.132:6801/727581123 7 ==== osd_op_reply(53 [pgnls start_epoch 119 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564bc33dd680 con 0x564bc33ef400 2023-05-20T01:27:20.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.993+0000 7f6b234ec700 1 -- 172.21.15.16:0/3121821003 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:54 15.11 15:88000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564bc3f58a80 con 0x564bc322b800 2023-05-20T01:27:20.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.993+0000 7f6b224ea700 1 -- 172.21.15.16:0/3121821003 <== osd.2 v2:172.21.15.16:6816/293187030 15 ==== osd_op_reply(54 [pgnls start_epoch 119 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564bc33dd440 con 0x564bc322b800 2023-05-20T01:27:20.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.993+0000 7f6b224ea700 1 -- 172.21.15.16:0/3121821003 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:55 15.9 15:90000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564bc3f58a80 con 0x564bc33ef400 2023-05-20T01:27:20.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.994+0000 7f6b234ec700 1 -- 172.21.15.16:0/3121821003 <== osd.6 v2:172.21.15.132:6801/727581123 8 ==== osd_op_reply(55 [pgnls start_epoch 119 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564bc33dd680 con 0x564bc33ef400 2023-05-20T01:27:20.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.994+0000 7f6b234ec700 1 -- 172.21.15.16:0/3121821003 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:56 15.19 15:98000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564bc3f58a80 con 0x564bc33ee800 2023-05-20T01:27:20.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.994+0000 7f6b234ec700 1 -- 172.21.15.16:0/3121821003 <== osd.4 v2:172.21.15.132:6817/506317223 9 ==== osd_op_reply(56 [pgnls start_epoch 119 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564bc33dd680 con 0x564bc33ee800 2023-05-20T01:27:20.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.994+0000 7f6b234ec700 1 -- 172.21.15.16:0/3121821003 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:57 15.5 15:a0000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564bc3f58a80 con 0x564bc33ef400 2023-05-20T01:27:20.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.994+0000 7f6b234ec700 1 -- 172.21.15.16:0/3121821003 <== osd.6 v2:172.21.15.132:6801/727581123 9 ==== osd_op_reply(57 [pgnls start_epoch 119 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564bc33dd680 con 0x564bc33ef400 2023-05-20T01:27:20.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.994+0000 7f6b234ec700 1 -- 172.21.15.16:0/3121821003 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:58 15.15 15:a8000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564bc3f58a80 con 0x564bc322a800 2023-05-20T01:27:20.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.994+0000 7f6b234ec700 1 -- 172.21.15.16:0/3121821003 <== osd.7 v2:172.21.15.132:6800/466001149 4 ==== osd_op_reply(58 [pgnls start_epoch 119 out=73b] v116'6 uv6 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x564bc33dd680 con 0x564bc322a800 2023-05-20T01:27:20.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.994+0000 7f6b234ec700 1 -- 172.21.15.16:0/3121821003 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:59 15.d 15:b0000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564bc3f58a80 con 0x564bc322a800 2023-05-20T01:27:20.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.995+0000 7f6b234ec700 1 -- 172.21.15.16:0/3121821003 <== osd.7 v2:172.21.15.132:6800/466001149 5 ==== osd_op_reply(59 [pgnls start_epoch 119 out=74b] v119'3 uv3 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x564bc33dd680 con 0x564bc322a800 2023-05-20T01:27:20.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.995+0000 7f6b234ec700 1 -- 172.21.15.16:0/3121821003 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:60 15.1d 15:b8000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564bc3f58a80 con 0x564bc33ee800 2023-05-20T01:27:20.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.995+0000 7f6b234ec700 1 -- 172.21.15.16:0/3121821003 <== osd.4 v2:172.21.15.132:6817/506317223 10 ==== osd_op_reply(60 [pgnls start_epoch 119 out=49b] v116'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564bc33dd680 con 0x564bc33ee800 2023-05-20T01:27:20.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.995+0000 7f6b234ec700 1 -- 172.21.15.16:0/3121821003 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:61 15.3 15:c0000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564bc3f58a80 con 0x564bc33ee800 2023-05-20T01:27:20.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.995+0000 7f6b234ec700 1 -- 172.21.15.16:0/3121821003 <== osd.4 v2:172.21.15.132:6817/506317223 11 ==== osd_op_reply(61 [pgnls start_epoch 119 out=71b] v119'2 uv2 ondisk = 1) v8 ==== 144+0+71 (crc 0 0 0) 0x564bc33dd680 con 0x564bc33ee800 2023-05-20T01:27:20.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.995+0000 7f6b234ec700 1 -- 172.21.15.16:0/3121821003 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:62 15.13 15:c8000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564bc3f58a80 con 0x564bc33ef000 2023-05-20T01:27:20.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.995+0000 7f6b224ea700 1 -- 172.21.15.16:0/3121821003 <== osd.1 v2:172.21.15.16:6808/3860264220 8 ==== osd_op_reply(62 [pgnls start_epoch 119 out=98b] v119'7 uv7 ondisk = 1) v8 ==== 144+0+98 (crc 0 0 0) 0x564bc33dd440 con 0x564bc33ef000 2023-05-20T01:27:20.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.995+0000 7f6b224ea700 1 -- 172.21.15.16:0/3121821003 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:63 15.b 15:d0000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564bc3f58a80 con 0x564bc322b800 2023-05-20T01:27:20.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.996+0000 7f6b224ea700 1 -- 172.21.15.16:0/3121821003 <== osd.2 v2:172.21.15.16:6816/293187030 16 ==== osd_op_reply(63 [pgnls start_epoch 119 out=76b] v119'6 uv5 ondisk = 1) v8 ==== 144+0+76 (crc 0 0 0) 0x564bc33dd440 con 0x564bc322b800 2023-05-20T01:27:20.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.996+0000 7f6b224ea700 1 -- 172.21.15.16:0/3121821003 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:64 15.1b 15:d8000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564bc3f58a80 con 0x564bc322a800 2023-05-20T01:27:20.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.996+0000 7f6b234ec700 1 -- 172.21.15.16:0/3121821003 <== osd.7 v2:172.21.15.132:6800/466001149 6 ==== osd_op_reply(64 [pgnls start_epoch 119 out=49b] v119'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564bc33dd680 con 0x564bc322a800 2023-05-20T01:27:20.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.996+0000 7f6b234ec700 1 -- 172.21.15.16:0/3121821003 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:65 15.7 15:e0000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564bc3f58a80 con 0x564bc33ef000 2023-05-20T01:27:20.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.996+0000 7f6b224ea700 1 -- 172.21.15.16:0/3121821003 <== osd.1 v2:172.21.15.16:6808/3860264220 9 ==== osd_op_reply(65 [pgnls start_epoch 119 out=49b] v116'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564bc33dd440 con 0x564bc33ef000 2023-05-20T01:27:20.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.996+0000 7f6b224ea700 1 -- 172.21.15.16:0/3121821003 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:66 15.17 15:e8000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564bc3f58a80 con 0x564bc33ee800 2023-05-20T01:27:20.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.997+0000 7f6b234ec700 1 -- 172.21.15.16:0/3121821003 <== osd.4 v2:172.21.15.132:6817/506317223 12 ==== osd_op_reply(66 [pgnls start_epoch 119 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564bc33dd680 con 0x564bc33ee800 2023-05-20T01:27:20.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.997+0000 7f6b234ec700 1 -- 172.21.15.16:0/3121821003 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:67 15.f 15:f0000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564bc3f58a80 con 0x564bc33ef400 2023-05-20T01:27:20.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.997+0000 7f6b234ec700 1 -- 172.21.15.16:0/3121821003 <== osd.6 v2:172.21.15.132:6801/727581123 10 ==== osd_op_reply(67 [pgnls start_epoch 119 out=73b] v119'4 uv4 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x564bc33dd680 con 0x564bc33ef400 2023-05-20T01:27:20.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.997+0000 7f6b234ec700 1 -- 172.21.15.16:0/3121821003 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:68 15.1f 15:f8000000::::head [pgnls start_epoch 119 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e119) v8 -- 0x564bc3f58a80 con 0x564bc33eec00 2023-05-20T01:27:20.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.997+0000 7f6b22ceb700 1 -- 172.21.15.16:0/3121821003 <== osd.3 v2:172.21.15.16:6800/3669781890 7 ==== osd_op_reply(68 [pgnls start_epoch 119 out=101b] v119'13 uv13 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x564bc33dd8c0 con 0x564bc33eec00 2023-05-20T01:27:20.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.997+0000 7f6b24d3d7c0 10 datascan.scan_links: step 2: handling object 10000000000.00000000 2023-05-20T01:27:20.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.997+0000 7f6b24d3d7c0 1 -- 172.21.15.16:0/3121821003 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:69 15.10 15:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x564bc3f59c00 con 0x564bc33ee800 2023-05-20T01:27:20.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.998+0000 7f6b234ec700 1 -- 172.21.15.16:0/3121821003 <== osd.4 v2:172.21.15.132:6817/506317223 13 ==== osd_op_reply(69 10000000000.00000000 [omap-get-vals out=3582b] v0'0 uv23 ondisk = 0) v8 ==== 164+0+3582 (crc 0 0 0) 0x564bc33dd680 con 0x564bc33ee800 2023-05-20T01:27:20.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.998+0000 7f6b24d3d7c0 20 datascan.scan_links: injected ino detected 2023-05-20T01:27:20.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.998+0000 7f6b24d3d7c0 20 datascan.scan_links: adding 0x10000000001 for future processing to fix dnfirst 2023-05-20T01:27:20.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.998+0000 7f6b24d3d7c0 20 datascan.scan_links: injected ino detected 2023-05-20T01:27:20.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.998+0000 7f6b24d3d7c0 20 datascan.scan_links: adding 0x100000001f6 for future processing to fix dnfirst 2023-05-20T01:27:20.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.998+0000 7f6b24d3d7c0 20 datascan.scan_links: injected ino detected 2023-05-20T01:27:20.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.998+0000 7f6b24d3d7c0 20 datascan.scan_links: adding 0x100000001f7 for future processing to fix dnfirst 2023-05-20T01:27:20.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.998+0000 7f6b24d3d7c0 20 datascan.scan_links: injected ino detected 2023-05-20T01:27:20.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.998+0000 7f6b24d3d7c0 20 datascan.scan_links: adding 0x100000001f8 for future processing to fix dnfirst 2023-05-20T01:27:20.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.998+0000 7f6b24d3d7c0 20 datascan.scan_links: injected ino detected 2023-05-20T01:27:20.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.998+0000 7f6b24d3d7c0 20 datascan.scan_links: adding 0x100000001f9 for future processing to fix dnfirst 2023-05-20T01:27:20.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.998+0000 7f6b24d3d7c0 20 datascan.scan_links: injected ino detected 2023-05-20T01:27:20.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.998+0000 7f6b24d3d7c0 20 datascan.scan_links: adding 0x100000001fa for future processing to fix dnfirst 2023-05-20T01:27:20.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.998+0000 7f6b24d3d7c0 20 datascan.scan_links: injected ino detected 2023-05-20T01:27:20.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.998+0000 7f6b24d3d7c0 20 datascan.scan_links: adding 0x100000001fb for future processing to fix dnfirst 2023-05-20T01:27:20.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.998+0000 7f6b24d3d7c0 10 datascan.scan_links: step 2: handling object 1.00000000.inode 2023-05-20T01:27:20.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.998+0000 7f6b24d3d7c0 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2023-05-20T01:27:20.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.998+0000 7f6b24d3d7c0 10 datascan.scan_links: step 2: handling object 200.00000000 2023-05-20T01:27:20.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.998+0000 7f6b24d3d7c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2023-05-20T01:27:20.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.998+0000 7f6b24d3d7c0 10 datascan.scan_links: step 2: handling object 200.00000001 2023-05-20T01:27:20.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.998+0000 7f6b24d3d7c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2023-05-20T01:27:20.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.998+0000 7f6b24d3d7c0 10 datascan.scan_links: step 2: handling object 500.00000000 2023-05-20T01:27:20.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.998+0000 7f6b24d3d7c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2023-05-20T01:27:20.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.998+0000 7f6b24d3d7c0 10 datascan.scan_links: step 2: handling object mds_snaptable 2023-05-20T01:27:20.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.998+0000 7f6b24d3d7c0 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2023-05-20T01:27:20.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.998+0000 7f6b24d3d7c0 10 datascan.scan_links: step 2: handling object 4.00000000 2023-05-20T01:27:20.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.998+0000 7f6b24d3d7c0 1 -- 172.21.15.16:0/3121821003 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:70 15.3 15:c0ce80d4:::4.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x564bc3f58700 con 0x564bc33ee800 2023-05-20T01:27:20.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.998+0000 7f6b234ec700 1 -- 172.21.15.16:0/3121821003 <== osd.4 v2:172.21.15.132:6817/506317223 14 ==== osd_op_reply(70 4.00000000 [omap-get-vals out=513b] v0'0 uv2 ondisk = 0) v8 ==== 154+0+513 (crc 0 0 0) 0x564bc33dd680 con 0x564bc33ee800 2023-05-20T01:27:20.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.998+0000 7f6b24d3d7c0 20 datascan.scan_links: injected ino detected 2023-05-20T01:27:20.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.998+0000 7f6b24d3d7c0 20 datascan.scan_links: adding 0x100000001fc for future processing to fix dnfirst 2023-05-20T01:27:20.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.998+0000 7f6b24d3d7c0 10 datascan.scan_links: step 2: handling object mds0_inotable 2023-05-20T01:27:20.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.998+0000 7f6b24d3d7c0 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2023-05-20T01:27:20.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.998+0000 7f6b24d3d7c0 10 datascan.scan_links: step 2: handling object 100.00000000 2023-05-20T01:27:20.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.998+0000 7f6b24d3d7c0 1 -- 172.21.15.16:0/3121821003 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:71 15.13 15:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x564bc3f59500 con 0x564bc33ef000 2023-05-20T01:27:20.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.998+0000 7f6b224ea700 1 -- 172.21.15.16:0/3121821003 <== osd.1 v2:172.21.15.16:6808/3860264220 10 ==== osd_op_reply(71 100.00000000 [omap-get-vals out=5b] v0'0 uv7 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x564bc33dd440 con 0x564bc33ef000 2023-05-20T01:27:20.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.998+0000 7f6b24d3d7c0 10 datascan.scan_links: step 2: handling object mds0_sessionmap 2023-05-20T01:27:20.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.998+0000 7f6b24d3d7c0 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2023-05-20T01:27:20.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.998+0000 7f6b24d3d7c0 10 datascan.scan_links: step 2: handling object 400.00000000 2023-05-20T01:27:20.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.998+0000 7f6b24d3d7c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2023-05-20T01:27:20.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.998+0000 7f6b24d3d7c0 10 datascan.scan_links: step 2: handling object 100.00000000.inode 2023-05-20T01:27:20.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.998+0000 7f6b24d3d7c0 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2023-05-20T01:27:20.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.998+0000 7f6b24d3d7c0 10 datascan.scan_links: step 2: handling object 1.00000000 2023-05-20T01:27:20.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.998+0000 7f6b24d3d7c0 1 -- 172.21.15.16:0/3121821003 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:72 15.1f 15:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x564bc3528000 con 0x564bc33eec00 2023-05-20T01:27:20.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.998+0000 7f6b22ceb700 1 -- 172.21.15.16:0/3121821003 <== osd.3 v2:172.21.15.16:6800/3669781890 8 ==== osd_op_reply(72 1.00000000 [omap-get-vals out=1016b] v0'0 uv13 ondisk = 0) v8 ==== 154+0+1016 (crc 0 0 0) 0x564bc33dd8c0 con 0x564bc33eec00 2023-05-20T01:27:20.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.998+0000 7f6b24d3d7c0 20 datascan.scan_links: injected ino detected 2023-05-20T01:27:20.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.998+0000 7f6b24d3d7c0 20 datascan.scan_links: adding 0x4 for future processing to fix dnfirst 2023-05-20T01:27:20.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.998+0000 7f6b24d3d7c0 20 datascan.scan_links: injected ino detected 2023-05-20T01:27:20.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.998+0000 7f6b24d3d7c0 20 datascan.scan_links: adding 0x10000000000 for future processing to fix dnfirst 2023-05-20T01:27:20.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.998+0000 7f6b24d3d7c0 10 datascan.scan_links: processing 0 dup_primaries, 0 remote_links 2023-05-20T01:27:20.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.998+0000 7f6b24d3d7c0 10 datascan.scan_links: removing dup dentries from 0 objects 2023-05-20T01:27:20.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.998+0000 7f6b24d3d7c0 10 datascan.scan_links: processing 0 bad_nlink_inos 2023-05-20T01:27:20.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.998+0000 7f6b24d3d7c0 10 datascan.scan_links: processing 10 injected_inos 2023-05-20T01:27:20.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.998+0000 7f6b24d3d7c0 10 datascan.scan_links: handling injected_ino 0x4 2023-05-20T01:27:20.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.998+0000 7f6b24d3d7c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x1:lost+found 2023-05-20T01:27:20.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.998+0000 7f6b24d3d7c0 1 -- 172.21.15.16:0/3121821003 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:73 15.1f 15:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=23b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x564bc3528380 con 0x564bc33eec00 2023-05-20T01:27:20.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.999+0000 7f6b22ceb700 1 -- 172.21.15.16:0/3121821003 <== osd.3 v2:172.21.15.16:6800/3669781890 9 ==== osd_op_reply(73 1.00000000 [omap-get-vals-by-keys out=511b] v0'0 uv13 ondisk = 0) v8 ==== 154+0+511 (crc 0 0 0) 0x564bc33dd8c0 con 0x564bc33eec00 2023-05-20T01:27:20.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.999+0000 7f6b24d3d7c0 20 datascan.read_dentry: oid=1.00000000 dname=lost+found frag=*, r=0 2023-05-20T01:27:20.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.999+0000 7f6b24d3d7c0 20 datascan.scan_links: first is now 2 2023-05-20T01:27:20.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:19.999+0000 7f6b24d3d7c0 1 -- 172.21.15.16:0/3121821003 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:74 15.1f 15:ff5b34d6:::1.00000000:head [omap-set-vals in=511b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x564bc3528700 con 0x564bc33eec00 2023-05-20T01:27:20.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.000+0000 7f6b22ceb700 1 -- 172.21.15.16:0/3121821003 <== osd.3 v2:172.21.15.16:6800/3669781890 10 ==== osd_op_reply(74 1.00000000 [omap-set-vals] v119'14 uv14 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x564bc33dd8c0 con 0x564bc33eec00 2023-05-20T01:27:20.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.000+0000 7f6b24d3d7c0 20 datascan.inject_linkage: Injected dentry 0x0x1/lost+found pointing to 0x0x4 2023-05-20T01:27:20.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.000+0000 7f6b24d3d7c0 10 datascan.scan_links: handling injected_ino 0x10000000000 2023-05-20T01:27:20.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.000+0000 7f6b24d3d7c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x1:stripey 2023-05-20T01:27:20.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.000+0000 7f6b24d3d7c0 1 -- 172.21.15.16:0/3121821003 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:75 15.1f 15:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x564bc3528a80 con 0x564bc33eec00 2023-05-20T01:27:20.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.000+0000 7f6b22ceb700 1 -- 172.21.15.16:0/3121821003 <== osd.3 v2:172.21.15.16:6800/3669781890 11 ==== osd_op_reply(75 1.00000000 [omap-get-vals-by-keys out=508b] v0'0 uv14 ondisk = 0) v8 ==== 154+0+508 (crc 0 0 0) 0x564bc33dd8c0 con 0x564bc33eec00 2023-05-20T01:27:20.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.000+0000 7f6b24d3d7c0 20 datascan.read_dentry: oid=1.00000000 dname=stripey frag=*, r=0 2023-05-20T01:27:20.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.000+0000 7f6b24d3d7c0 20 datascan.scan_links: first is now 2 2023-05-20T01:27:20.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.000+0000 7f6b24d3d7c0 1 -- 172.21.15.16:0/3121821003 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:76 15.1f 15:ff5b34d6:::1.00000000:head [omap-set-vals in=508b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x564bc3528e00 con 0x564bc33eec00 2023-05-20T01:27:20.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.001+0000 7f6b22ceb700 1 -- 172.21.15.16:0/3121821003 <== osd.3 v2:172.21.15.16:6800/3669781890 12 ==== osd_op_reply(76 1.00000000 [omap-set-vals] v119'15 uv15 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x564bc33dd8c0 con 0x564bc33eec00 2023-05-20T01:27:20.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.001+0000 7f6b24d3d7c0 20 datascan.inject_linkage: Injected dentry 0x0x1/stripey pointing to 0x0x10000000000 2023-05-20T01:27:20.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.001+0000 7f6b24d3d7c0 10 datascan.scan_links: handling injected_ino 0x10000000001 2023-05-20T01:27:20.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.001+0000 7f6b24d3d7c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:flushed_file_0 2023-05-20T01:27:20.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.001+0000 7f6b24d3d7c0 1 -- 172.21.15.16:0/3121821003 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:77 15.10 15:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=27b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x564bc3529180 con 0x564bc33ee800 2023-05-20T01:27:20.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.001+0000 7f6b234ec700 1 -- 172.21.15.16:0/3121821003 <== osd.4 v2:172.21.15.132:6817/506317223 15 ==== osd_op_reply(77 10000000000.00000000 [omap-get-vals-by-keys out=515b] v0'0 uv23 ondisk = 0) v8 ==== 164+0+515 (crc 0 0 0) 0x564bc33dd680 con 0x564bc33ee800 2023-05-20T01:27:20.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.001+0000 7f6b24d3d7c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=flushed_file_0 frag=*, r=0 2023-05-20T01:27:20.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.001+0000 7f6b24d3d7c0 20 datascan.scan_links: first is now 2 2023-05-20T01:27:20.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.001+0000 7f6b24d3d7c0 1 -- 172.21.15.16:0/3121821003 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:78 15.10 15:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=515b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x564bc3529180 con 0x564bc33ee800 2023-05-20T01:27:20.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.003+0000 7f6b234ec700 1 -- 172.21.15.16:0/3121821003 <== osd.4 v2:172.21.15.132:6817/506317223 16 ==== osd_op_reply(78 10000000000.00000000 [omap-set-vals] v119'24 uv24 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564bc33dd680 con 0x564bc33ee800 2023-05-20T01:27:20.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.003+0000 7f6b24d3d7c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/flushed_file_0 pointing to 0x0x10000000001 2023-05-20T01:27:20.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.003+0000 7f6b24d3d7c0 10 datascan.scan_links: handling injected_ino 0x100000001f6 2023-05-20T01:27:20.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.003+0000 7f6b24d3d7c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:flushed_file_1 2023-05-20T01:27:20.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.003+0000 7f6b24d3d7c0 1 -- 172.21.15.16:0/3121821003 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:79 15.10 15:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=27b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x564bc3528000 con 0x564bc33ee800 2023-05-20T01:27:20.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.003+0000 7f6b234ec700 1 -- 172.21.15.16:0/3121821003 <== osd.4 v2:172.21.15.132:6817/506317223 17 ==== osd_op_reply(79 10000000000.00000000 [omap-get-vals-by-keys out=515b] v0'0 uv24 ondisk = 0) v8 ==== 164+0+515 (crc 0 0 0) 0x564bc33dd680 con 0x564bc33ee800 2023-05-20T01:27:20.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.003+0000 7f6b24d3d7c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=flushed_file_1 frag=*, r=0 2023-05-20T01:27:20.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.003+0000 7f6b24d3d7c0 20 datascan.scan_links: first is now 2 2023-05-20T01:27:20.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.003+0000 7f6b24d3d7c0 1 -- 172.21.15.16:0/3121821003 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:80 15.10 15:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=515b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x564bc3f59180 con 0x564bc33ee800 2023-05-20T01:27:20.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.006+0000 7f6b234ec700 1 -- 172.21.15.16:0/3121821003 <== osd.4 v2:172.21.15.132:6817/506317223 18 ==== osd_op_reply(80 10000000000.00000000 [omap-set-vals] v119'25 uv25 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564bc33dd680 con 0x564bc33ee800 2023-05-20T01:27:20.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.006+0000 7f6b24d3d7c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/flushed_file_1 pointing to 0x0x100000001f6 2023-05-20T01:27:20.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.006+0000 7f6b24d3d7c0 10 datascan.scan_links: handling injected_ino 0x100000001f7 2023-05-20T01:27:20.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.006+0000 7f6b24d3d7c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:flushed_file_2 2023-05-20T01:27:20.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.006+0000 7f6b24d3d7c0 1 -- 172.21.15.16:0/3121821003 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:81 15.10 15:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=27b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x564bc3f58e00 con 0x564bc33ee800 2023-05-20T01:27:20.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.006+0000 7f6b234ec700 1 -- 172.21.15.16:0/3121821003 <== osd.4 v2:172.21.15.132:6817/506317223 19 ==== osd_op_reply(81 10000000000.00000000 [omap-get-vals-by-keys out=515b] v0'0 uv25 ondisk = 0) v8 ==== 164+0+515 (crc 0 0 0) 0x564bc33dd680 con 0x564bc33ee800 2023-05-20T01:27:20.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.006+0000 7f6b24d3d7c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=flushed_file_2 frag=*, r=0 2023-05-20T01:27:20.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.006+0000 7f6b24d3d7c0 20 datascan.scan_links: first is now 2 2023-05-20T01:27:20.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.006+0000 7f6b24d3d7c0 1 -- 172.21.15.16:0/3121821003 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:82 15.10 15:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=515b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x564bc3f59c00 con 0x564bc33ee800 2023-05-20T01:27:20.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.008+0000 7f6b234ec700 1 -- 172.21.15.16:0/3121821003 <== osd.4 v2:172.21.15.132:6817/506317223 20 ==== osd_op_reply(82 10000000000.00000000 [omap-set-vals] v119'26 uv26 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564bc33dd680 con 0x564bc33ee800 2023-05-20T01:27:20.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.008+0000 7f6b24d3d7c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/flushed_file_2 pointing to 0x0x100000001f7 2023-05-20T01:27:20.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.008+0000 7f6b24d3d7c0 10 datascan.scan_links: handling injected_ino 0x100000001f8 2023-05-20T01:27:20.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.008+0000 7f6b24d3d7c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:flushed_file_3 2023-05-20T01:27:20.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.008+0000 7f6b24d3d7c0 1 -- 172.21.15.16:0/3121821003 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:83 15.10 15:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=27b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x564bc3f58700 con 0x564bc33ee800 2023-05-20T01:27:20.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.008+0000 7f6b234ec700 1 -- 172.21.15.16:0/3121821003 <== osd.4 v2:172.21.15.132:6817/506317223 21 ==== osd_op_reply(83 10000000000.00000000 [omap-get-vals-by-keys out=515b] v0'0 uv26 ondisk = 0) v8 ==== 164+0+515 (crc 0 0 0) 0x564bc33dd680 con 0x564bc33ee800 2023-05-20T01:27:20.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.008+0000 7f6b24d3d7c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=flushed_file_3 frag=*, r=0 2023-05-20T01:27:20.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.008+0000 7f6b24d3d7c0 20 datascan.scan_links: first is now 2 2023-05-20T01:27:20.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.008+0000 7f6b24d3d7c0 1 -- 172.21.15.16:0/3121821003 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:84 15.10 15:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=515b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x564bc3f58a80 con 0x564bc33ee800 2023-05-20T01:27:20.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.011+0000 7f6b234ec700 1 -- 172.21.15.16:0/3121821003 <== osd.4 v2:172.21.15.132:6817/506317223 22 ==== osd_op_reply(84 10000000000.00000000 [omap-set-vals] v119'27 uv27 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564bc33dd680 con 0x564bc33ee800 2023-05-20T01:27:20.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.011+0000 7f6b24d3d7c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/flushed_file_3 pointing to 0x0x100000001f8 2023-05-20T01:27:20.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.011+0000 7f6b24d3d7c0 10 datascan.scan_links: handling injected_ino 0x100000001f9 2023-05-20T01:27:20.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.011+0000 7f6b24d3d7c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:flushed_file_4 2023-05-20T01:27:20.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.011+0000 7f6b24d3d7c0 1 -- 172.21.15.16:0/3121821003 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:85 15.10 15:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=27b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x564bc3529180 con 0x564bc33ee800 2023-05-20T01:27:20.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.011+0000 7f6b234ec700 1 -- 172.21.15.16:0/3121821003 <== osd.4 v2:172.21.15.132:6817/506317223 23 ==== osd_op_reply(85 10000000000.00000000 [omap-get-vals-by-keys out=515b] v0'0 uv27 ondisk = 0) v8 ==== 164+0+515 (crc 0 0 0) 0x564bc33dd680 con 0x564bc33ee800 2023-05-20T01:27:20.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.011+0000 7f6b24d3d7c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=flushed_file_4 frag=*, r=0 2023-05-20T01:27:20.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.011+0000 7f6b24d3d7c0 20 datascan.scan_links: first is now 2 2023-05-20T01:27:20.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.011+0000 7f6b24d3d7c0 1 -- 172.21.15.16:0/3121821003 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:86 15.10 15:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=515b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x564bc3528000 con 0x564bc33ee800 2023-05-20T01:27:20.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.012+0000 7f6b234ec700 1 -- 172.21.15.16:0/3121821003 <== osd.4 v2:172.21.15.132:6817/506317223 24 ==== osd_op_reply(86 10000000000.00000000 [omap-set-vals] v119'28 uv28 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564bc33dd680 con 0x564bc33ee800 2023-05-20T01:27:20.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.012+0000 7f6b24d3d7c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/flushed_file_4 pointing to 0x0x100000001f9 2023-05-20T01:27:20.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.012+0000 7f6b24d3d7c0 10 datascan.scan_links: handling injected_ino 0x100000001fa 2023-05-20T01:27:20.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.012+0000 7f6b24d3d7c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:flushed_file_5 2023-05-20T01:27:20.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.012+0000 7f6b24d3d7c0 1 -- 172.21.15.16:0/3121821003 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:87 15.10 15:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=27b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x564bc3528380 con 0x564bc33ee800 2023-05-20T01:27:20.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.012+0000 7f6b234ec700 1 -- 172.21.15.16:0/3121821003 <== osd.4 v2:172.21.15.132:6817/506317223 25 ==== osd_op_reply(87 10000000000.00000000 [omap-get-vals-by-keys out=515b] v0'0 uv28 ondisk = 0) v8 ==== 164+0+515 (crc 0 0 0) 0x564bc33dd680 con 0x564bc33ee800 2023-05-20T01:27:20.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.013+0000 7f6b24d3d7c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=flushed_file_5 frag=*, r=0 2023-05-20T01:27:20.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.013+0000 7f6b24d3d7c0 20 datascan.scan_links: first is now 2 2023-05-20T01:27:20.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.013+0000 7f6b24d3d7c0 1 -- 172.21.15.16:0/3121821003 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:88 15.10 15:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=515b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x564bc3529500 con 0x564bc33ee800 2023-05-20T01:27:20.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.015+0000 7f6b234ec700 1 -- 172.21.15.16:0/3121821003 <== osd.4 v2:172.21.15.132:6817/506317223 26 ==== osd_op_reply(88 10000000000.00000000 [omap-set-vals] v119'29 uv29 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564bc33dd680 con 0x564bc33ee800 2023-05-20T01:27:20.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.015+0000 7f6b24d3d7c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/flushed_file_5 pointing to 0x0x100000001fa 2023-05-20T01:27:20.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.015+0000 7f6b24d3d7c0 10 datascan.scan_links: handling injected_ino 0x100000001fb 2023-05-20T01:27:20.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.015+0000 7f6b24d3d7c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:flushed_file_6 2023-05-20T01:27:20.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.015+0000 7f6b24d3d7c0 1 -- 172.21.15.16:0/3121821003 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:89 15.10 15:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=27b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x564bc3f59180 con 0x564bc33ee800 2023-05-20T01:27:20.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.015+0000 7f6b234ec700 1 -- 172.21.15.16:0/3121821003 <== osd.4 v2:172.21.15.132:6817/506317223 27 ==== osd_op_reply(89 10000000000.00000000 [omap-get-vals-by-keys out=515b] v0'0 uv29 ondisk = 0) v8 ==== 164+0+515 (crc 0 0 0) 0x564bc33dd680 con 0x564bc33ee800 2023-05-20T01:27:20.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.015+0000 7f6b24d3d7c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=flushed_file_6 frag=*, r=0 2023-05-20T01:27:20.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.015+0000 7f6b24d3d7c0 20 datascan.scan_links: first is now 2 2023-05-20T01:27:20.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.015+0000 7f6b24d3d7c0 1 -- 172.21.15.16:0/3121821003 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:90 15.10 15:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=515b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x564bc3f58e00 con 0x564bc33ee800 2023-05-20T01:27:20.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.018+0000 7f6b234ec700 1 -- 172.21.15.16:0/3121821003 <== osd.4 v2:172.21.15.132:6817/506317223 28 ==== osd_op_reply(90 10000000000.00000000 [omap-set-vals] v119'30 uv30 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564bc33dd680 con 0x564bc33ee800 2023-05-20T01:27:20.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.018+0000 7f6b24d3d7c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/flushed_file_6 pointing to 0x0x100000001fb 2023-05-20T01:27:20.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.018+0000 7f6b24d3d7c0 10 datascan.scan_links: handling injected_ino 0x100000001fc 2023-05-20T01:27:20.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.018+0000 7f6b24d3d7c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x4:100000001fc 2023-05-20T01:27:20.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.018+0000 7f6b24d3d7c0 1 -- 172.21.15.16:0/3121821003 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:91 15.3 15:c0ce80d4:::4.00000000:head [omap-get-vals-by-keys in=24b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x564bc3f59c00 con 0x564bc33ee800 2023-05-20T01:27:20.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.018+0000 7f6b234ec700 1 -- 172.21.15.16:0/3121821003 <== osd.4 v2:172.21.15.132:6817/506317223 29 ==== osd_op_reply(91 4.00000000 [omap-get-vals-by-keys out=512b] v0'0 uv2 ondisk = 0) v8 ==== 154+0+512 (crc 0 0 0) 0x564bc33dd680 con 0x564bc33ee800 2023-05-20T01:27:20.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.018+0000 7f6b24d3d7c0 20 datascan.read_dentry: oid=4.00000000 dname=100000001fc frag=*, r=0 2023-05-20T01:27:20.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.018+0000 7f6b24d3d7c0 20 datascan.scan_links: first is now 2 2023-05-20T01:27:20.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.018+0000 7f6b24d3d7c0 1 -- 172.21.15.16:0/3121821003 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:92 15.3 15:c0ce80d4:::4.00000000:head [omap-set-vals in=512b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x564bc3f58700 con 0x564bc33ee800 2023-05-20T01:27:20.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.020+0000 7f6b234ec700 1 -- 172.21.15.16:0/3121821003 <== osd.4 v2:172.21.15.132:6817/506317223 30 ==== osd_op_reply(92 4.00000000 [omap-set-vals] v119'3 uv3 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x564bc33dd680 con 0x564bc33ee800 2023-05-20T01:27:20.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.020+0000 7f6b24d3d7c0 20 datascan.inject_linkage: Injected dentry 0x0x4/100000001fc pointing to 0x0x100000001fc 2023-05-20T01:27:20.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.020+0000 7f6b24d3d7c0 10 datascan.scan_links: updating inotable 2023-05-20T01:27:20.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.020+0000 7f6b24d3d7c0 1 -- 172.21.15.16:0/3121821003 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:93 15.13 15:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x564bc3529180 con 0x564bc33ef000 2023-05-20T01:27:20.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.021+0000 7f6b224ea700 1 -- 172.21.15.16:0/3121821003 <== osd.1 v2:172.21.15.16:6808/3860264220 11 ==== osd_op_reply(93 mds0_inotable [read 0~34 out=34b] v0'0 uv6 ondisk = 0) v8 ==== 157+0+34 (crc 0 0 0) 0x564bc33dd440 con 0x564bc33ef000 2023-05-20T01:27:20.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.021+0000 7f6b24d3d7c0 10 mds.0.inotable: skip_inos was [0x10000000000~0x10000000000] 2023-05-20T01:27:20.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.021+0000 7f6b24d3d7c0 10 mds.0.inotable: skip_inos now [0x100000001fd~0xfffffffe03] 2023-05-20T01:27:20.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.021+0000 7f6b24d3d7c0 1 -- 172.21.15.16:0/3121821003 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:94 15.13 15:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x564bc3528000 con 0x564bc33ef000 2023-05-20T01:27:20.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.022+0000 7f6b224ea700 1 -- 172.21.15.16:0/3121821003 <== osd.1 v2:172.21.15.16:6808/3860264220 12 ==== osd_op_reply(94 mds0_inotable [writefull 0~34] v119'8 uv8 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x564bc33dd440 con 0x564bc33ef000 2023-05-20T01:27:20.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.022+0000 7f6b24d3d7c0 10 datascan.scan_links: updating snaptable 2023-05-20T01:27:20.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.022+0000 7f6b24d3d7c0 1 -- 172.21.15.16:0/3121821003 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:95 15.d 15:b50e409b:::mds_snaptable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) v8 -- 0x564bc3528380 con 0x564bc322a800 2023-05-20T01:27:20.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.022+0000 7f6b234ec700 1 -- 172.21.15.16:0/3121821003 <== osd.7 v2:172.21.15.132:6800/466001149 7 ==== osd_op_reply(95 mds_snaptable [read 0~70 out=70b] v0'0 uv3 ondisk = 0) v8 ==== 157+0+70 (crc 0 0 0) 0x564bc33dd680 con 0x564bc322a800 2023-05-20T01:27:20.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.022+0000 7f6b24d3d7c0 1 -- 172.21.15.16:0/3121821003 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x564bc3516000 msgr2=0x564bc34f9b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:20.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.022+0000 7f6b24d3d7c0 1 --2- 172.21.15.16:0/3121821003 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x564bc3516000 0x564bc34f9b80 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:20.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.022+0000 7f6b24d3d7c0 1 -- 172.21.15.16:0/3121821003 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x564bc33ef000 msgr2=0x564bc34f9600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:20.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.022+0000 7f6b24d3d7c0 1 --2- 172.21.15.16:0/3121821003 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x564bc33ef000 0x564bc34f9600 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:20.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.022+0000 7f6b24d3d7c0 1 -- 172.21.15.16:0/3121821003 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x564bc322b800 msgr2=0x564bc315c680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:20.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.022+0000 7f6b24d3d7c0 1 --2- 172.21.15.16:0/3121821003 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x564bc322b800 0x564bc315c680 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:20.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.022+0000 7f6b24d3d7c0 1 -- 172.21.15.16:0/3121821003 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x564bc33eec00 msgr2=0x564bc34f8b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:20.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.022+0000 7f6b24d3d7c0 1 --2- 172.21.15.16:0/3121821003 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x564bc33eec00 0x564bc34f8b00 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:20.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.022+0000 7f6b22ceb700 1 -- 172.21.15.16:0/3121821003 reap_dead start 2023-05-20T01:27:20.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.022+0000 7f6b24d3d7c0 1 -- 172.21.15.16:0/3121821003 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x564bc33ee800 msgr2=0x564bc34f8580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:20.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.022+0000 7f6b24d3d7c0 1 --2- 172.21.15.16:0/3121821003 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x564bc33ee800 0x564bc34f8580 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:20.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.022+0000 7f6b24d3d7c0 1 -- 172.21.15.16:0/3121821003 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x564bc322a400 msgr2=0x564bc315bb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:20.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.022+0000 7f6b22ceb700 1 -- 172.21.15.16:0/3121821003 reap_dead start 2023-05-20T01:27:20.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.022+0000 7f6b24d3d7c0 1 --2- 172.21.15.16:0/3121821003 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x564bc322a400 0x564bc315bb80 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:20.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.022+0000 7f6b24d3d7c0 1 -- 172.21.15.16:0/3121821003 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x564bc33ef400 msgr2=0x564bc34f9080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:20.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.022+0000 7f6b24d3d7c0 1 --2- 172.21.15.16:0/3121821003 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x564bc33ef400 0x564bc34f9080 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:20.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.023+0000 7f6b24d3d7c0 1 -- 172.21.15.16:0/3121821003 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x564bc322a800 msgr2=0x564bc34fa680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:20.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.023+0000 7f6b24d3d7c0 1 --2- 172.21.15.16:0/3121821003 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x564bc322a800 0x564bc34fa680 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:20.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.023+0000 7f6b24d3d7c0 1 -- 172.21.15.16:0/3121821003 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x564bc33ee400 msgr2=0x564bc34f8000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:20.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.023+0000 7f6b24d3d7c0 1 --2- 172.21.15.16:0/3121821003 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x564bc33ee400 0x564bc34f8000 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x564bc33f8030 tx=0x564bc3e59440 comp rx=0 tx=0).stop 2023-05-20T01:27:20.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.023+0000 7f6b24d3d7c0 1 -- 172.21.15.16:0/3121821003 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bc322bc00 msgr2=0x564bc315cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:20.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.023+0000 7f6b24d3d7c0 1 --2- 172.21.15.16:0/3121821003 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bc322bc00 0x564bc315cc00 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x564bc33f9410 tx=0x564bc3f62de0 comp rx=0 tx=0).stop 2023-05-20T01:27:20.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.023+0000 7f6b24d3d7c0 1 -- 172.21.15.16:0/3121821003 shutdown_connections 2023-05-20T01:27:20.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.023+0000 7f6b22ceb700 1 -- 172.21.15.16:0/3121821003 reap_dead start 2023-05-20T01:27:20.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.023+0000 7f6b24d3d7c0 1 --2- 172.21.15.16:0/3121821003 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x564bc33ef400 0x564bc34f9080 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:20.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.023+0000 7f6b24d3d7c0 1 --2- 172.21.15.16:0/3121821003 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x564bc322a800 0x564bc34fa680 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:20.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.023+0000 7f6b24d3d7c0 1 --2- 172.21.15.16:0/3121821003 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x564bc33ee400 0x564bc34f8000 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:20.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.023+0000 7f6b24d3d7c0 1 --2- 172.21.15.16:0/3121821003 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x564bc322a400 0x564bc315bb80 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:20.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.023+0000 7f6b24d3d7c0 1 --2- 172.21.15.16:0/3121821003 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bc322bc00 0x564bc315cc00 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:20.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.023+0000 7f6b24d3d7c0 1 -- 172.21.15.16:0/3121821003 >> 172.21.15.16:0/3121821003 conn(0x564bc30c0800 msgr2=0x564bc319d000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:27:20.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.023+0000 7f6b24d3d7c0 1 -- 172.21.15.16:0/3121821003 shutdown_connections 2023-05-20T01:27:20.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.023+0000 7f6b24d3d7c0 1 -- 172.21.15.16:0/3121821003 wait complete. 2023-05-20T01:27:20.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.023+0000 7f6b24d3d7c0 1 -- 172.21.15.16:0/1713335594 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bc30c0c00 msgr2=0x564bc315b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:20.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.023+0000 7f6b24d3d7c0 1 --2- 172.21.15.16:0/1713335594 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bc30c0c00 0x564bc315b600 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x564bc321cc60 tx=0x564bc3eb6fc0 comp rx=0 tx=0).stop 2023-05-20T01:27:20.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.023+0000 7f6b24d3d7c0 1 -- 172.21.15.16:0/1713335594 shutdown_connections 2023-05-20T01:27:20.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.023+0000 7f6b24d3d7c0 1 --2- 172.21.15.16:0/1713335594 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bc30c0c00 0x564bc315b600 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:20.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.023+0000 7f6b24d3d7c0 1 --2- 172.21.15.16:0/1713335594 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564bc30c1800 0x564bc315b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:20.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.023+0000 7f6b24d3d7c0 1 --2- 172.21.15.16:0/1713335594 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x564bc30c1400 0x564bc315ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:20.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.023+0000 7f6b24d3d7c0 1 -- 172.21.15.16:0/1713335594 >> 172.21.15.16:0/1713335594 conn(0x564bc30c0400 msgr2=0x564bc319e000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:27:20.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.023+0000 7f6b24d3d7c0 1 -- 172.21.15.16:0/1713335594 shutdown_connections 2023-05-20T01:27:20.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.023+0000 7f6b24d3d7c0 1 -- 172.21.15.16:0/1713335594 wait complete. 2023-05-20T01:27:20.057 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_links'] in time 0:00:00.129066, result: 2023-05-20T01:27:20.058 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2023-05-20T01:27:20.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.192+0000 7f4a6dc65700 1 Processor -- start 2023-05-20T01:27:20.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.192+0000 7f4a6dc65700 1 -- start start 2023-05-20T01:27:20.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.193+0000 7f4a6dc65700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4a680cb670 0x7f4a680ce340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:20.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.193+0000 7f4a6dc65700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a68159d90 con 0x7f4a681599c0 2023-05-20T01:27:20.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.193+0000 7f4a6dc65700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a680cdb80 con 0x7f4a680caaa0 2023-05-20T01:27:20.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.193+0000 7f4a6dc65700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a680cdf20 con 0x7f4a680cb670 2023-05-20T01:27:20.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.193+0000 7f4a66ffd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4a680cb670 0x7f4a680ce340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:20.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.193+0000 7f4a66ffd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4a680cb670 0x7f4a680ce340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39058/0 (socket says 172.21.15.16:39058) 2023-05-20T01:27:20.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.193+0000 7f4a66ffd700 1 -- 172.21.15.16:0/3408161040 learned_addr learned my addr 172.21.15.16:0/3408161040 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:27:20.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.193+0000 7f4a66ffd700 1 -- 172.21.15.16:0/3408161040 >> v1:172.21.15.132:6789/0 conn(0x7f4a680caaa0 legacy=0x7f4a680ce880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:20.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.193+0000 7f4a66ffd700 1 -- 172.21.15.16:0/3408161040 >> v1:172.21.15.16:6789/0 conn(0x7f4a681599c0 legacy=0x7f4a680caf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:20.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.193+0000 7f4a66ffd700 1 -- 172.21.15.16:0/3408161040 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a6806eaf0 con 0x7f4a680cb670 2023-05-20T01:27:20.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.193+0000 7f4a667fc700 1 -- 172.21.15.16:0/3408161040 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2395505034 0 0) 0x7f4a58001110 con 0x7f4a681599c0 2023-05-20T01:27:20.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.194+0000 7f4a66ffd700 1 --2- 172.21.15.16:0/3408161040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a680cb670 0x7f4a680ce340 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f4a500098f0 tx=0x7f4a50009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=562ddc9c612739bc server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:20.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.194+0000 7f4a667fc700 1 -- 172.21.15.16:0/3408161040 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a5000b8a0 con 0x7f4a680cb670 2023-05-20T01:27:20.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.194+0000 7f4a667fc700 1 -- 172.21.15.16:0/3408161040 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a5000ba00 con 0x7f4a680cb670 2023-05-20T01:27:20.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.194+0000 7f4a667fc700 1 -- 172.21.15.16:0/3408161040 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a5000bd90 con 0x7f4a680cb670 2023-05-20T01:27:20.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.194+0000 7f4a6dc65700 1 -- 172.21.15.16:0/3408161040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a680cb670 msgr2=0x7f4a680ce340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:20.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.194+0000 7f4a6dc65700 1 --2- 172.21.15.16:0/3408161040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a680cb670 0x7f4a680ce340 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f4a500098f0 tx=0x7f4a50009c00 comp rx=0 tx=0).stop 2023-05-20T01:27:20.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.194+0000 7f4a6dc65700 1 -- 172.21.15.16:0/3408161040 shutdown_connections 2023-05-20T01:27:20.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.194+0000 7f4a6dc65700 1 --2- 172.21.15.16:0/3408161040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a680cb670 0x7f4a680ce340 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:20.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.194+0000 7f4a6dc65700 1 -- 172.21.15.16:0/3408161040 >> 172.21.15.16:0/3408161040 conn(0x7f4a680d0c30 msgr2=0x7f4a680b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:27:20.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.194+0000 7f4a6dc65700 1 -- 172.21.15.16:0/3408161040 shutdown_connections 2023-05-20T01:27:20.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.194+0000 7f4a6dc65700 1 -- 172.21.15.16:0/3408161040 wait complete. 2023-05-20T01:27:20.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.196+0000 7f4a6dc65700 1 Processor -- start 2023-05-20T01:27:20.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.196+0000 7f4a6dc65700 1 -- start start 2023-05-20T01:27:20.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.197+0000 7f4a6dc65700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a680caaa0 0x7f4a68149450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:20.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.197+0000 7f4a6dc65700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4a680cb670 0x7f4a68157540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:20.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.197+0000 7f4a6dc65700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4a681599c0 0x7f4a6815c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:20.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.197+0000 7f4a6dc65700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a68149990 con 0x7f4a680caaa0 2023-05-20T01:27:20.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.197+0000 7f4a6dc65700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a68157a80 con 0x7f4a681599c0 2023-05-20T01:27:20.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.197+0000 7f4a6dc65700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a681570b0 con 0x7f4a680cb670 2023-05-20T01:27:20.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.197+0000 7f4a677fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a680caaa0 0x7f4a68149450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:20.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.197+0000 7f4a677fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a680caaa0 0x7f4a68149450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39062/0 (socket says 172.21.15.16:39062) 2023-05-20T01:27:20.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.197+0000 7f4a677fe700 1 -- 172.21.15.16:0/3076318158 learned_addr learned my addr 172.21.15.16:0/3076318158 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:27:20.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.197+0000 7f4a67fff700 1 --2- 172.21.15.16:0/3076318158 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4a681599c0 0x7f4a6815c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:20.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.197+0000 7f4a66ffd700 1 --2- 172.21.15.16:0/3076318158 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4a680cb670 0x7f4a68157540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:20.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.197+0000 7f4a677fe700 1 -- 172.21.15.16:0/3076318158 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4a680cb670 msgr2=0x7f4a68157540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:20.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.197+0000 7f4a677fe700 1 --2- 172.21.15.16:0/3076318158 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4a680cb670 0x7f4a68157540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:20.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.197+0000 7f4a677fe700 1 -- 172.21.15.16:0/3076318158 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4a681599c0 msgr2=0x7f4a6815c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:20.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.197+0000 7f4a677fe700 1 --2- 172.21.15.16:0/3076318158 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4a681599c0 0x7f4a6815c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:20.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.197+0000 7f4a677fe700 1 -- 172.21.15.16:0/3076318158 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a50009550 con 0x7f4a680caaa0 2023-05-20T01:27:20.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.197+0000 7f4a677fe700 1 --2- 172.21.15.16:0/3076318158 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a680caaa0 0x7f4a68149450 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f4a5800c480 tx=0x7f4a5800c790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:20.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.197+0000 7f4a5ffff700 1 -- 172.21.15.16:0/3076318158 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a580054e0 con 0x7f4a680caaa0 2023-05-20T01:27:20.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.197+0000 7f4a5ffff700 1 -- 172.21.15.16:0/3076318158 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a5800ce50 con 0x7f4a680caaa0 2023-05-20T01:27:20.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.197+0000 7f4a6dc65700 1 -- 172.21.15.16:0/3076318158 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a6806f0b0 con 0x7f4a680caaa0 2023-05-20T01:27:20.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.197+0000 7f4a5ffff700 1 -- 172.21.15.16:0/3076318158 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a580038c0 con 0x7f4a680caaa0 2023-05-20T01:27:20.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.197+0000 7f4a6dc65700 1 -- 172.21.15.16:0/3076318158 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a6806edd0 con 0x7f4a680caaa0 2023-05-20T01:27:20.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.198+0000 7f4a6dc65700 1 -- 172.21.15.16:0/3076318158 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a6806e750 con 0x7f4a680caaa0 2023-05-20T01:27:20.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.199+0000 7f4a5ffff700 1 -- 172.21.15.16:0/3076318158 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4a58003a20 con 0x7f4a680caaa0 2023-05-20T01:27:20.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.199+0000 7f4a5ffff700 1 --2- 172.21.15.16:0/3076318158 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f4a4806ace0 0x7f4a4806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:20.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.199+0000 7f4a5ffff700 1 -- 172.21.15.16:0/3076318158 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6987+0+0 (secure 0 0 0) 0x7f4a58088980 con 0x7f4a680caaa0 2023-05-20T01:27:20.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.199+0000 7f4a66ffd700 1 --2- 172.21.15.16:0/3076318158 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f4a4806ace0 0x7f4a4806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:20.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.200+0000 7f4a66ffd700 1 --2- 172.21.15.16:0/3076318158 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f4a4806ace0 0x7f4a4806d190 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f4a50000c00 tx=0x7f4a50008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:20.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.204+0000 7f4a5ffff700 1 -- 172.21.15.16:0/3076318158 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f4a58053720 con 0x7f4a680caaa0 2023-05-20T01:27:20.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:20.491+0000 7f4a6dc65700 1 -- 172.21.15.16:0/3076318158 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) v1 -- 0x7f4a68159d90 con 0x7f4a680caaa0 2023-05-20T01:27:21.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:21.426+0000 7f4a5ffff700 1 -- 172.21.15.16:0/3076318158 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 5:0 v85) v1 ==== 101+0+0 (secure 0 0 0) 0x7f4a68159d90 con 0x7f4a680caaa0 2023-05-20T01:27:21.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:21.429+0000 7f4a6dc65700 1 -- 172.21.15.16:0/3076318158 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f4a4806ace0 msgr2=0x7f4a4806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:21.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:21.429+0000 7f4a6dc65700 1 --2- 172.21.15.16:0/3076318158 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f4a4806ace0 0x7f4a4806d190 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f4a50000c00 tx=0x7f4a50008030 comp rx=0 tx=0).stop 2023-05-20T01:27:21.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:21.429+0000 7f4a6dc65700 1 -- 172.21.15.16:0/3076318158 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a680caaa0 msgr2=0x7f4a68149450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:21.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:21.429+0000 7f4a6dc65700 1 --2- 172.21.15.16:0/3076318158 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a680caaa0 0x7f4a68149450 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f4a5800c480 tx=0x7f4a5800c790 comp rx=0 tx=0).stop 2023-05-20T01:27:21.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:21.429+0000 7f4a6dc65700 1 -- 172.21.15.16:0/3076318158 shutdown_connections 2023-05-20T01:27:21.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:21.429+0000 7f4a6dc65700 1 --2- 172.21.15.16:0/3076318158 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4a681599c0 0x7f4a6815c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:21.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:21.429+0000 7f4a6dc65700 1 --2- 172.21.15.16:0/3076318158 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a680caaa0 0x7f4a68149450 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:21.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:21.429+0000 7f4a6dc65700 1 --2- 172.21.15.16:0/3076318158 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f4a4806ace0 0x7f4a4806d190 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:21.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:21.429+0000 7f4a6dc65700 1 --2- 172.21.15.16:0/3076318158 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4a680cb670 0x7f4a68157540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:21.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:21.429+0000 7f4a6dc65700 1 -- 172.21.15.16:0/3076318158 >> 172.21.15.16:0/3076318158 conn(0x7f4a680d0c30 msgr2=0x7f4a680b41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:27:21.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:21.429+0000 7f4a6dc65700 1 -- 172.21.15.16:0/3076318158 shutdown_connections 2023-05-20T01:27:21.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:21.429+0000 7f4a6dc65700 1 -- 172.21.15.16:0/3076318158 wait complete. 2023-05-20T01:27:21.436 INFO:teuthology.orchestra.run.smithi016.stderr:repaired: restoring rank 5:0 2023-05-20T01:27:21.437 INFO:tasks.ceph.mds.a.smithi016.stderr:2023-05-20T01:27:21.433+0000 7f0bc87f9700 -1 mds.0.openfiles _load_finish got (2) No such file or directory 2023-05-20T01:27:21.451 INFO:tasks.ceph.mds.a:Restarting daemon 2023-05-20T01:27:21.451 INFO:tasks.ceph.mds.a:Stopping old one... 2023-05-20T01:27:21.451 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-05-20T01:27:21.451 INFO:teuthology.orchestra.run:waiting for 300 2023-05-20T01:27:21.452 INFO:tasks.ceph.mds.c:Restarting daemon 2023-05-20T01:27:21.452 INFO:tasks.ceph.mds.c:Stopping old one... 2023-05-20T01:27:21.452 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-05-20T01:27:21.452 INFO:teuthology.orchestra.run:waiting for 300 2023-05-20T01:27:21.453 INFO:tasks.ceph.mds.b:Restarting daemon 2023-05-20T01:27:21.453 INFO:tasks.ceph.mds.b:Stopping old one... 2023-05-20T01:27:21.453 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-05-20T01:27:21.453 INFO:teuthology.orchestra.run:waiting for 300 2023-05-20T01:27:21.454 INFO:tasks.ceph.mds.d:Restarting daemon 2023-05-20T01:27:21.454 INFO:tasks.ceph.mds.d:Stopping old one... 2023-05-20T01:27:21.454 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-05-20T01:27:21.455 INFO:teuthology.orchestra.run:waiting for 300 2023-05-20T01:27:21.464 INFO:tasks.ceph.mds.b:Stopped 2023-05-20T01:27:21.464 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-05-20T01:27:21.467 INFO:tasks.ceph.mds.a:Stopped 2023-05-20T01:27:21.467 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-05-20T01:27:21.468 INFO:tasks.ceph.mds.d:Stopped 2023-05-20T01:27:21.468 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-05-20T01:27:21.468 INFO:tasks.ceph.mds.c:Stopped 2023-05-20T01:27:21.469 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-05-20T01:27:21.509 INFO:tasks.ceph.mds.b:Started 2023-05-20T01:27:21.510 DEBUG:teuthology.parallel:result is None 2023-05-20T01:27:21.511 INFO:tasks.ceph.mds.a:Started 2023-05-20T01:27:21.512 INFO:tasks.ceph.mds.d:Started 2023-05-20T01:27:21.513 INFO:tasks.ceph.mds.c:Started 2023-05-20T01:27:21.513 DEBUG:teuthology.parallel:result is None 2023-05-20T01:27:21.514 DEBUG:teuthology.parallel:result is None 2023-05-20T01:27:21.515 DEBUG:teuthology.parallel:result is None 2023-05-20T01:27:21.515 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-20T01:27:21.595 INFO:tasks.ceph.mds.b.smithi132.stdout:starting mds.b at 2023-05-20T01:27:21.595 INFO:tasks.ceph.mds.d.smithi132.stdout:starting mds.d at 2023-05-20T01:27:21.597 INFO:tasks.ceph.mds.a.smithi016.stdout:starting mds.a at 2023-05-20T01:27:21.597 INFO:tasks.ceph.mds.c.smithi016.stdout:starting mds.c at 2023-05-20T01:27:21.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:21.653+0000 7f6b8ac06700 1 Processor -- start 2023-05-20T01:27:21.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:21.653+0000 7f6b8ac06700 1 -- start start 2023-05-20T01:27:21.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:21.654+0000 7f6b8ac06700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f6b840b4750 0x7f6b840d2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:21.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:21.654+0000 7f6b8ac06700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6b840cd970 0x7f6b840cddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:21.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:21.654+0000 7f6b8ac06700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b840b3b50 con 0x7f6b840cfc50 2023-05-20T01:27:21.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:21.654+0000 7f6b8ac06700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b840cfad0 con 0x7f6b840b4750 2023-05-20T01:27:21.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:21.654+0000 7f6b8ac06700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b840d3df0 con 0x7f6b840cd970 2023-05-20T01:27:21.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:21.654+0000 7f6b889a2700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f6b840cfc50 0x7f6b840d0020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50636/0 (socket says 172.21.15.16:50636) 2023-05-20T01:27:21.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:21.654+0000 7f6b889a2700 1 -- 172.21.15.16:0/3867999779 learned_addr learned my addr 172.21.15.16:0/3867999779 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:27:21.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:21.654+0000 7f6b83fff700 1 --2- 172.21.15.16:0/3867999779 >> v2:172.21.15.132:3300/0 conn(0x7f6b840b4750 0x7f6b840d2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:21.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:21.654+0000 7f6b891a3700 1 --2- 172.21.15.16:0/3867999779 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6b840cd970 0x7f6b840cddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:21.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:21.655+0000 7f6b837fe700 1 -- 172.21.15.16:0/3867999779 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3686077322 0 0) 0x7f6b840b3b50 con 0x7f6b840cfc50 2023-05-20T01:27:21.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:21.655+0000 7f6b837fe700 1 -- 172.21.15.16:0/3867999779 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b740035b0 con 0x7f6b840cfc50 2023-05-20T01:27:21.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:21.655+0000 7f6b83fff700 1 -- 172.21.15.16:0/3867999779 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6b840cd970 msgr2=0x7f6b840cddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:21.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:21.655+0000 7f6b83fff700 1 --2- 172.21.15.16:0/3867999779 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6b840cd970 0x7f6b840cddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:21.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:21.655+0000 7f6b83fff700 1 -- 172.21.15.16:0/3867999779 >> v1:172.21.15.16:6789/0 conn(0x7f6b840cfc50 legacy=0x7f6b840d0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:21.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:21.655+0000 7f6b83fff700 1 -- 172.21.15.16:0/3867999779 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b840d0b90 con 0x7f6b840b4750 2023-05-20T01:27:21.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:21.655+0000 7f6b891a3700 1 --2- 172.21.15.16:0/3867999779 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6b840cd970 0x7f6b840cddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-05-20T01:27:21.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:21.655+0000 7f6b83fff700 1 --2- 172.21.15.16:0/3867999779 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6b840b4750 0x7f6b840d2c10 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f6b70009840 tx=0x7f6b70009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=197d2607e8a3721c server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:21.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:21.655+0000 7f6b837fe700 1 -- 172.21.15.16:0/3867999779 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b70011830 con 0x7f6b840b4750 2023-05-20T01:27:21.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:21.655+0000 7f6b837fe700 1 -- 172.21.15.16:0/3867999779 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b70011990 con 0x7f6b840b4750 2023-05-20T01:27:21.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:21.655+0000 7f6b837fe700 1 -- 172.21.15.16:0/3867999779 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b70011d20 con 0x7f6b840b4750 2023-05-20T01:27:21.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:21.656+0000 7f6b8ac06700 1 -- 172.21.15.16:0/3867999779 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6b840b4750 msgr2=0x7f6b840d2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:21.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:21.656+0000 7f6b8ac06700 1 --2- 172.21.15.16:0/3867999779 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6b840b4750 0x7f6b840d2c10 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f6b70009840 tx=0x7f6b70009b50 comp rx=0 tx=0).stop 2023-05-20T01:27:21.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:21.656+0000 7f6b8ac06700 1 -- 172.21.15.16:0/3867999779 shutdown_connections 2023-05-20T01:27:21.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:21.656+0000 7f6b8ac06700 1 --2- 172.21.15.16:0/3867999779 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6b840cd970 0x7f6b840cddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:21.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:21.656+0000 7f6b8ac06700 1 --2- 172.21.15.16:0/3867999779 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6b840b4750 0x7f6b840d2c10 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:21.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:21.656+0000 7f6b8ac06700 1 -- 172.21.15.16:0/3867999779 >> 172.21.15.16:0/3867999779 conn(0x7f6b8409c590 msgr2=0x7f6b840d3240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:27:21.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:21.656+0000 7f6b8ac06700 1 -- 172.21.15.16:0/3867999779 shutdown_connections 2023-05-20T01:27:21.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:21.656+0000 7f6b8ac06700 1 -- 172.21.15.16:0/3867999779 wait complete. 2023-05-20T01:27:21.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:21.657+0000 7f6b8ac06700 1 Processor -- start 2023-05-20T01:27:21.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:21.657+0000 7f6b8ac06700 1 -- start start 2023-05-20T01:27:21.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:21.658+0000 7f6b8ac06700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6b840b4750 0x7f6b8415ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:21.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:21.658+0000 7f6b8ac06700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6b840cd970 0x7f6b840b6a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:21.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:21.658+0000 7f6b8ac06700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b840cfc50 0x7f6b840b03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:21.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:21.658+0000 7f6b8ac06700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b840ca030 con 0x7f6b840cfc50 2023-05-20T01:27:21.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:21.658+0000 7f6b8ac06700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b840b7180 con 0x7f6b840cd970 2023-05-20T01:27:21.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:21.658+0000 7f6b8ac06700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b840b7460 con 0x7f6b840b4750 2023-05-20T01:27:21.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:21.658+0000 7f6b891a3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b840cfc50 0x7f6b840b03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:21.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:21.658+0000 7f6b891a3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b840cfc50 0x7f6b840b03e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39118/0 (socket says 172.21.15.16:39118) 2023-05-20T01:27:21.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:21.658+0000 7f6b891a3700 1 -- 172.21.15.16:0/2612993135 learned_addr learned my addr 172.21.15.16:0/2612993135 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:27:21.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:21.658+0000 7f6b889a2700 1 --2- 172.21.15.16:0/2612993135 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6b840b4750 0x7f6b8415ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:21.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:21.658+0000 7f6b83fff700 1 --2- 172.21.15.16:0/2612993135 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6b840cd970 0x7f6b840b6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:21.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:21.658+0000 7f6b891a3700 1 -- 172.21.15.16:0/2612993135 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6b840b4750 msgr2=0x7f6b8415ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:21.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:21.658+0000 7f6b891a3700 1 --2- 172.21.15.16:0/2612993135 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6b840b4750 0x7f6b8415ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:21.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:21.658+0000 7f6b891a3700 1 -- 172.21.15.16:0/2612993135 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6b840cd970 msgr2=0x7f6b840b6a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:21.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:21.658+0000 7f6b891a3700 1 --2- 172.21.15.16:0/2612993135 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6b840cd970 0x7f6b840b6a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:21.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:21.658+0000 7f6b891a3700 1 -- 172.21.15.16:0/2612993135 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b7c009a70 con 0x7f6b840cfc50 2023-05-20T01:27:21.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:21.658+0000 7f6b891a3700 1 --2- 172.21.15.16:0/2612993135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b840cfc50 0x7f6b840b03e0 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c00efc0 tx=0x7f6b7c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:21.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:21.659+0000 7f6b817fa700 1 -- 172.21.15.16:0/2612993135 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b7c00ca90 con 0x7f6b840cfc50 2023-05-20T01:27:21.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:21.659+0000 7f6b817fa700 1 -- 172.21.15.16:0/2612993135 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b7c00f040 con 0x7f6b840cfc50 2023-05-20T01:27:21.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:21.659+0000 7f6b8ac06700 1 -- 172.21.15.16:0/2612993135 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b700094a0 con 0x7f6b840cfc50 2023-05-20T01:27:21.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:21.659+0000 7f6b817fa700 1 -- 172.21.15.16:0/2612993135 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b7c00ce30 con 0x7f6b840cfc50 2023-05-20T01:27:21.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:21.659+0000 7f6b8ac06700 1 -- 172.21.15.16:0/2612993135 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b840b4380 con 0x7f6b840cfc50 2023-05-20T01:27:21.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:21.660+0000 7f6b8ac06700 1 -- 172.21.15.16:0/2612993135 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b840d4000 con 0x7f6b840cfc50 2023-05-20T01:27:21.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:21.660+0000 7f6b817fa700 1 -- 172.21.15.16:0/2612993135 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6b7c014480 con 0x7f6b840cfc50 2023-05-20T01:27:21.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:21.660+0000 7f6b817fa700 1 --2- 172.21.15.16:0/2612993135 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f6b6806ace0 0x7f6b6806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:21.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:21.661+0000 7f6b817fa700 1 -- 172.21.15.16:0/2612993135 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 7073+0+0 (secure 0 0 0) 0x7f6b7c089f50 con 0x7f6b840cfc50 2023-05-20T01:27:21.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:21.661+0000 7f6b889a2700 1 --2- 172.21.15.16:0/2612993135 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f6b6806ace0 0x7f6b6806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:21.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:21.661+0000 7f6b889a2700 1 --2- 172.21.15.16:0/2612993135 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f6b6806ace0 0x7f6b6806d190 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f6b78007fd0 tx=0x7f6b78007dc0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:21.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:21.665+0000 7f6b817fa700 1 -- 172.21.15.16:0/2612993135 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f6b7c054b70 con 0x7f6b840cfc50 2023-05-20T01:27:21.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:21.915+0000 7f6b8ac06700 1 -- 172.21.15.16:0/2612993135 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6b8415d090 con 0x7f6b840cfc50 2023-05-20T01:27:21.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:21.915+0000 7f6b817fa700 1 -- 172.21.15.16:0/2612993135 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) v1 ==== 94+0+4681 (secure 0 0 0) 0x7f6b7c018090 con 0x7f6b840cfc50 2023-05-20T01:27:21.920 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:27:21.920 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":86,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7069,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/3619911175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":3619911175},{"type":"v1","addr":"172.21.15.16:6837","nonce":3619911175}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73},{"gid":7081,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/959638003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":959638003},{"type":"v1","addr":"172.21.15.132:6834","nonce":959638003}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73},{"gid":7084,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2596482253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":2596482253},{"type":"v1","addr":"172.21.15.132:6835","nonce":2596482253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-20T01:27:10.291795+0000","modified":"2023-05-20T01:27:21.427302+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":117,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8205},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8205":{"gid":8205,"name":"a","rank":0,"incarnation":86,"state":"up:replay","state_seq":1,"addr":"172.21.15.16:6836/1584803225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1584803225},{"type":"v1","addr":"172.21.15.16:6836","nonce":1584803225}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16,17],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-05-20T01:27:21.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:21.917+0000 7f6b8ac06700 1 -- 172.21.15.16:0/2612993135 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f6b6806ace0 msgr2=0x7f6b6806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:21.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:21.917+0000 7f6b8ac06700 1 --2- 172.21.15.16:0/2612993135 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f6b6806ace0 0x7f6b6806d190 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f6b78007fd0 tx=0x7f6b78007dc0 comp rx=0 tx=0).stop 2023-05-20T01:27:21.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:21.917+0000 7f6b8ac06700 1 -- 172.21.15.16:0/2612993135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b840cfc50 msgr2=0x7f6b840b03e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:21.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:21.917+0000 7f6b8ac06700 1 --2- 172.21.15.16:0/2612993135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b840cfc50 0x7f6b840b03e0 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c00efc0 tx=0x7f6b7c00c3a0 comp rx=0 tx=0).stop 2023-05-20T01:27:21.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:21.917+0000 7f6b8ac06700 1 -- 172.21.15.16:0/2612993135 shutdown_connections 2023-05-20T01:27:21.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:21.917+0000 7f6b8ac06700 1 --2- 172.21.15.16:0/2612993135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b840cfc50 0x7f6b840b03e0 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:21.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:21.917+0000 7f6b8ac06700 1 --2- 172.21.15.16:0/2612993135 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f6b6806ace0 0x7f6b6806d190 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:21.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:21.917+0000 7f6b8ac06700 1 --2- 172.21.15.16:0/2612993135 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6b840b4750 0x7f6b8415ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:21.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:21.917+0000 7f6b8ac06700 1 --2- 172.21.15.16:0/2612993135 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6b840cd970 0x7f6b840b6a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:21.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:21.917+0000 7f6b8ac06700 1 -- 172.21.15.16:0/2612993135 >> 172.21.15.16:0/2612993135 conn(0x7f6b8409c590 msgr2=0x7f6b8415c820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:27:21.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:21.918+0000 7f6b8ac06700 1 -- 172.21.15.16:0/2612993135 shutdown_connections 2023-05-20T01:27:21.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:21.918+0000 7f6b8ac06700 1 -- 172.21.15.16:0/2612993135 wait complete. 2023-05-20T01:27:21.924 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 86 2023-05-20T01:27:22.336 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 86, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-05-20T01:27:10.291795+0000', 'modified': '2023-05-20T01:27:21.427302+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 117, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8205}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8205': {'gid': 8205, 'name': 'a', 'rank': 0, 'incarnation': 86, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.16:6836/1584803225', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6834', 'nonce': 1584803225}, {'type': 'v1', 'addr': '172.21.15.16:6836', 'nonce': 1584803225}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [16, 17], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-05-20T01:27:22.336 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8205:up:replay 2023-05-20T01:27:22.432 INFO:tasks.ceph.mon.a.smithi016.stderr:2023-05-20T01:27:22.428+0000 7f49eb8ae700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-05-20T01:27:22.448 INFO:tasks.ceph.mds.c.smithi016.stderr:2023-05-20T01:27:22.444+0000 7efe54384700 -1 mds.0.openfiles _load_finish got (2) No such file or directory 2023-05-20T01:27:23.338 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-20T01:27:23.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:23.473+0000 7f3dafd9e700 1 Processor -- start 2023-05-20T01:27:23.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:23.474+0000 7f3dafd9e700 1 -- start start 2023-05-20T01:27:23.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:23.474+0000 7f3dafd9e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3da806f450 0x7f3da80cf890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:23.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:23.474+0000 7f3dafd9e700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3da806d4b0 0x7f3da8161080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:23.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:23.474+0000 7f3dafd9e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3da80cc510 con 0x7f3da806a510 2023-05-20T01:27:23.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:23.474+0000 7f3dafd9e700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3da81615c0 con 0x7f3da806d4b0 2023-05-20T01:27:23.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:23.474+0000 7f3dafd9e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3da8071ed0 con 0x7f3da806f450 2023-05-20T01:27:23.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:23.475+0000 7f3dad339700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3da806d4b0 0x7f3da8161080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:23.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:23.475+0000 7f3dad339700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3da806d4b0 0x7f3da8161080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39140/0 (socket says 172.21.15.16:39140) 2023-05-20T01:27:23.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:23.475+0000 7f3dad339700 1 -- 172.21.15.16:0/2213371257 learned_addr learned my addr 172.21.15.16:0/2213371257 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:27:23.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:23.475+0000 7f3dadb3a700 1 --2- 172.21.15.16:0/2213371257 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3da806f450 0x7f3da80cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:23.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:23.475+0000 7f3dad339700 1 -- 172.21.15.16:0/2213371257 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3da806f450 msgr2=0x7f3da80cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:23.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:23.475+0000 7f3dad339700 1 --2- 172.21.15.16:0/2213371257 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3da806f450 0x7f3da80cf890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:23.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:23.475+0000 7f3dad339700 1 -- 172.21.15.16:0/2213371257 >> v1:172.21.15.132:6789/0 conn(0x7f3da806a510 legacy=0x7f3da80cdb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:23.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:23.475+0000 7f3dad339700 1 -- 172.21.15.16:0/2213371257 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3da80721b0 con 0x7f3da806d4b0 2023-05-20T01:27:23.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:23.475+0000 7f3dad339700 1 --2- 172.21.15.16:0/2213371257 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3da806d4b0 0x7f3da8161080 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f3da40098f0 tx=0x7f3da4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f1301fd7585165e server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:23.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:23.475+0000 7f3dacb38700 1 -- 172.21.15.16:0/2213371257 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3da4004230 con 0x7f3da806d4b0 2023-05-20T01:27:23.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:23.475+0000 7f3dacb38700 1 -- 172.21.15.16:0/2213371257 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3da4004390 con 0x7f3da806d4b0 2023-05-20T01:27:23.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:23.475+0000 7f3dacb38700 1 -- 172.21.15.16:0/2213371257 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3da40036a0 con 0x7f3da806d4b0 2023-05-20T01:27:23.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:23.476+0000 7f3dafd9e700 1 -- 172.21.15.16:0/2213371257 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3da806d4b0 msgr2=0x7f3da8161080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:23.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:23.476+0000 7f3dafd9e700 1 --2- 172.21.15.16:0/2213371257 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3da806d4b0 0x7f3da8161080 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f3da40098f0 tx=0x7f3da4009c00 comp rx=0 tx=0).stop 2023-05-20T01:27:23.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:23.476+0000 7f3dafd9e700 1 -- 172.21.15.16:0/2213371257 shutdown_connections 2023-05-20T01:27:23.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:23.476+0000 7f3dafd9e700 1 --2- 172.21.15.16:0/2213371257 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3da806f450 0x7f3da80cf890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:23.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:23.476+0000 7f3dafd9e700 1 --2- 172.21.15.16:0/2213371257 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3da806d4b0 0x7f3da8161080 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:23.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:23.476+0000 7f3dafd9e700 1 -- 172.21.15.16:0/2213371257 >> 172.21.15.16:0/2213371257 conn(0x7f3da80cffb0 msgr2=0x7f3da80d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:27:23.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:23.476+0000 7f3dafd9e700 1 -- 172.21.15.16:0/2213371257 shutdown_connections 2023-05-20T01:27:23.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:23.476+0000 7f3dafd9e700 1 -- 172.21.15.16:0/2213371257 wait complete. 2023-05-20T01:27:23.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:23.477+0000 7f3dafd9e700 1 Processor -- start 2023-05-20T01:27:23.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:23.477+0000 7f3dafd9e700 1 -- start start 2023-05-20T01:27:23.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:23.478+0000 7f3dafd9e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3da806a510 0x7f3da80c5e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:23.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:23.478+0000 7f3dafd9e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3da806d4b0 0x7f3da815ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:23.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:23.478+0000 7f3dafd9e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3da806f450 0x7f3da815b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:23.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:23.478+0000 7f3dafd9e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3da81e74e0 con 0x7f3da806d4b0 2023-05-20T01:27:23.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:23.478+0000 7f3dafd9e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3da80c63a0 con 0x7f3da806a510 2023-05-20T01:27:23.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:23.478+0000 7f3dafd9e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3da81e6860 con 0x7f3da806f450 2023-05-20T01:27:23.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:23.478+0000 7f3dad339700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3da806d4b0 0x7f3da815ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:23.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:23.478+0000 7f3dad339700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3da806d4b0 0x7f3da815ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39146/0 (socket says 172.21.15.16:39146) 2023-05-20T01:27:23.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:23.478+0000 7f3dad339700 1 -- 172.21.15.16:0/1107875503 learned_addr learned my addr 172.21.15.16:0/1107875503 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:27:23.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:23.478+0000 7f3dae33b700 1 --2- 172.21.15.16:0/1107875503 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3da806f450 0x7f3da815b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:23.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:23.478+0000 7f3dadb3a700 1 --2- 172.21.15.16:0/1107875503 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3da806a510 0x7f3da80c5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:23.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:23.478+0000 7f3dad339700 1 -- 172.21.15.16:0/1107875503 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3da806f450 msgr2=0x7f3da815b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:23.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:23.478+0000 7f3dad339700 1 --2- 172.21.15.16:0/1107875503 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3da806f450 0x7f3da815b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:23.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:23.478+0000 7f3dad339700 1 -- 172.21.15.16:0/1107875503 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3da806a510 msgr2=0x7f3da80c5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:23.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:23.478+0000 7f3dad339700 1 --2- 172.21.15.16:0/1107875503 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3da806a510 0x7f3da80c5e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:23.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:23.478+0000 7f3dad339700 1 -- 172.21.15.16:0/1107875503 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3da4009550 con 0x7f3da806d4b0 2023-05-20T01:27:23.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:23.478+0000 7f3dae33b700 1 --2- 172.21.15.16:0/1107875503 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3da806f450 0x7f3da815b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:27:23.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:23.478+0000 7f3dadb3a700 1 --2- 172.21.15.16:0/1107875503 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3da806a510 0x7f3da80c5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:27:23.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:23.478+0000 7f3dad339700 1 --2- 172.21.15.16:0/1107875503 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3da806d4b0 0x7f3da815ae10 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f3da4009130 tx=0x7f3da4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:23.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:23.479+0000 7f3d9a7fc700 1 -- 172.21.15.16:0/1107875503 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3da4003c40 con 0x7f3da806d4b0 2023-05-20T01:27:23.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:23.479+0000 7f3d9a7fc700 1 -- 172.21.15.16:0/1107875503 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3da401f070 con 0x7f3da806d4b0 2023-05-20T01:27:23.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:23.479+0000 7f3dafd9e700 1 -- 172.21.15.16:0/1107875503 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3da80cbe50 con 0x7f3da806d4b0 2023-05-20T01:27:23.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:23.479+0000 7f3d9a7fc700 1 -- 172.21.15.16:0/1107875503 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3da400b660 con 0x7f3da806d4b0 2023-05-20T01:27:23.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:23.479+0000 7f3dafd9e700 1 -- 172.21.15.16:0/1107875503 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3da80cc1a0 con 0x7f3da806d4b0 2023-05-20T01:27:23.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:23.480+0000 7f3dafd9e700 1 -- 172.21.15.16:0/1107875503 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3da806a9b0 con 0x7f3da806d4b0 2023-05-20T01:27:23.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:23.480+0000 7f3d9a7fc700 1 -- 172.21.15.16:0/1107875503 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3da4021030 con 0x7f3da806d4b0 2023-05-20T01:27:23.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:23.480+0000 7f3d9a7fc700 1 --2- 172.21.15.16:0/1107875503 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f3d8c06ace0 0x7f3d8c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:23.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:23.481+0000 7f3d9a7fc700 1 -- 172.21.15.16:0/1107875503 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 7073+0+0 (secure 0 0 0) 0x7f3da408aeb0 con 0x7f3da806d4b0 2023-05-20T01:27:23.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:23.481+0000 7f3dadb3a700 1 --2- 172.21.15.16:0/1107875503 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f3d8c06ace0 0x7f3d8c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:23.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:23.481+0000 7f3dadb3a700 1 --2- 172.21.15.16:0/1107875503 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f3d8c06ace0 0x7f3d8c06d190 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c009820 tx=0x7f3d9c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:23.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:23.485+0000 7f3d9a7fc700 1 -- 172.21.15.16:0/1107875503 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f3da4055bf0 con 0x7f3da806d4b0 2023-05-20T01:27:23.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:23.728+0000 7f3dafd9e700 1 -- 172.21.15.16:0/1107875503 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3da806a9b0 con 0x7f3da806d4b0 2023-05-20T01:27:23.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:23.728+0000 7f3d9a7fc700 1 -- 172.21.15.16:0/1107875503 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) v1 ==== 94+0+4684 (secure 0 0 0) 0x7f3da4059210 con 0x7f3da806d4b0 2023-05-20T01:27:23.733 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:27:23.734 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":89,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6845,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/1435235487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1435235487},{"type":"v1","addr":"172.21.15.132:6834","nonce":1435235487}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":87},{"gid":6848,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/933893285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":933893285},{"type":"v1","addr":"172.21.15.132:6835","nonce":933893285}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":87},{"gid":8373,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/1220815342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1220815342},{"type":"v1","addr":"172.21.15.16:6836","nonce":1220815342}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":87}],"filesystems":[{"mdsmap":{"epoch":89,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-20T01:27:10.291795+0000","modified":"2023-05-20T01:27:23.439877+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":121,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8376},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8376":{"gid":8376,"name":"c","rank":0,"incarnation":88,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.16:6837/1927078882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":1927078882},{"type":"v1","addr":"172.21.15.16:6837","nonce":1927078882}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16,17],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-05-20T01:27:23.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:23.731+0000 7f3dafd9e700 1 -- 172.21.15.16:0/1107875503 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f3d8c06ace0 msgr2=0x7f3d8c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:23.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:23.731+0000 7f3dafd9e700 1 --2- 172.21.15.16:0/1107875503 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f3d8c06ace0 0x7f3d8c06d190 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c009820 tx=0x7f3d9c008040 comp rx=0 tx=0).stop 2023-05-20T01:27:23.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:23.731+0000 7f3dafd9e700 1 -- 172.21.15.16:0/1107875503 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3da806d4b0 msgr2=0x7f3da815ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:23.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:23.731+0000 7f3dafd9e700 1 --2- 172.21.15.16:0/1107875503 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3da806d4b0 0x7f3da815ae10 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f3da4009130 tx=0x7f3da4004750 comp rx=0 tx=0).stop 2023-05-20T01:27:23.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:23.731+0000 7f3dafd9e700 1 -- 172.21.15.16:0/1107875503 shutdown_connections 2023-05-20T01:27:23.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:23.731+0000 7f3dafd9e700 1 --2- 172.21.15.16:0/1107875503 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f3d8c06ace0 0x7f3d8c06d190 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:23.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:23.731+0000 7f3dafd9e700 1 --2- 172.21.15.16:0/1107875503 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3da806f450 0x7f3da815b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:23.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:23.731+0000 7f3dafd9e700 1 --2- 172.21.15.16:0/1107875503 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3da806a510 0x7f3da80c5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:23.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:23.731+0000 7f3dafd9e700 1 --2- 172.21.15.16:0/1107875503 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3da806d4b0 0x7f3da815ae10 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:23.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:23.731+0000 7f3dafd9e700 1 -- 172.21.15.16:0/1107875503 >> 172.21.15.16:0/1107875503 conn(0x7f3da80cffb0 msgr2=0x7f3da80b1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:27:23.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:23.731+0000 7f3dafd9e700 1 -- 172.21.15.16:0/1107875503 shutdown_connections 2023-05-20T01:27:23.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:23.731+0000 7f3dafd9e700 1 -- 172.21.15.16:0/1107875503 wait complete. 2023-05-20T01:27:23.737 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 89 2023-05-20T01:27:24.150 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 89, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-05-20T01:27:10.291795+0000', 'modified': '2023-05-20T01:27:23.439877+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 121, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8376}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8376': {'gid': 8376, 'name': 'c', 'rank': 0, 'incarnation': 88, 'state': 'up:reconnect', 'state_seq': 2, 'addr': '172.21.15.16:6837/1927078882', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6835', 'nonce': 1927078882}, {'type': 'v1', 'addr': '172.21.15.16:6837', 'nonce': 1927078882}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [16, 17], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-05-20T01:27:24.150 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8376:up:reconnect 2023-05-20T01:27:25.151 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-20T01:27:25.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:25.286+0000 7f5cffc45700 1 Processor -- start 2023-05-20T01:27:25.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:25.287+0000 7f5cffc45700 1 -- start start 2023-05-20T01:27:25.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:25.287+0000 7f5cffc45700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5cf81599c0 0x7f5cf80caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:25.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:25.287+0000 7f5cffc45700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f5cf80caaa0 0x7f5cf80cb4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:25.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:25.287+0000 7f5cffc45700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5cf806f3f0 0x7f5cf80710d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:25.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:25.287+0000 7f5cffc45700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cf80cdf20 con 0x7f5cf81599c0 2023-05-20T01:27:25.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:25.287+0000 7f5cffc45700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cf8159d90 con 0x7f5cf80caaa0 2023-05-20T01:27:25.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:25.287+0000 7f5cffc45700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cf80cdb80 con 0x7f5cf806f3f0 2023-05-20T01:27:25.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:25.288+0000 7f5cfd9e1700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5cf81599c0 0x7f5cf80caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:25.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:25.288+0000 7f5cfd9e1700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5cf81599c0 0x7f5cf80caf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39152/0 (socket says 172.21.15.16:39152) 2023-05-20T01:27:25.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:25.288+0000 7f5cfd9e1700 1 -- 172.21.15.16:0/1658698033 learned_addr learned my addr 172.21.15.16:0/1658698033 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:27:25.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:25.288+0000 7f5cfe1e2700 1 --2- 172.21.15.16:0/1658698033 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5cf806f3f0 0x7f5cf80710d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:25.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:25.288+0000 7f5cfd1e0700 1 --2- 172.21.15.16:0/1658698033 >> v2:172.21.15.132:3300/0 conn(0x7f5cf80caaa0 0x7f5cf80cb4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:25.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:25.288+0000 7f5cfd9e1700 1 -- 172.21.15.16:0/1658698033 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5cf806f3f0 msgr2=0x7f5cf80710d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:25.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:25.288+0000 7f5cfd9e1700 1 --2- 172.21.15.16:0/1658698033 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5cf806f3f0 0x7f5cf80710d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:25.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:25.288+0000 7f5cfd9e1700 1 -- 172.21.15.16:0/1658698033 >> v2:172.21.15.132:3300/0 conn(0x7f5cf80caaa0 msgr2=0x7f5cf80cb4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:25.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:25.288+0000 7f5cfd9e1700 1 --2- 172.21.15.16:0/1658698033 >> v2:172.21.15.132:3300/0 conn(0x7f5cf80caaa0 0x7f5cf80cb4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:25.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:25.288+0000 7f5cfd9e1700 1 -- 172.21.15.16:0/1658698033 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cf806eaf0 con 0x7f5cf81599c0 2023-05-20T01:27:25.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:25.288+0000 7f5cfd1e0700 1 --2- 172.21.15.16:0/1658698033 >> v2:172.21.15.132:3300/0 conn(0x7f5cf80caaa0 0x7f5cf80cb4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:27:25.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:25.288+0000 7f5cfd9e1700 1 --2- 172.21.15.16:0/1658698033 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5cf81599c0 0x7f5cf80caf60 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f5cf80cc990 tx=0x7f5cf4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71c39dd7fc55b79 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:25.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:25.288+0000 7f5cfc9df700 1 -- 172.21.15.16:0/1658698033 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf400b870 con 0x7f5cf81599c0 2023-05-20T01:27:25.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:25.288+0000 7f5cfc9df700 1 -- 172.21.15.16:0/1658698033 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cf400b9d0 con 0x7f5cf81599c0 2023-05-20T01:27:25.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:25.288+0000 7f5cfc9df700 1 -- 172.21.15.16:0/1658698033 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf400bd60 con 0x7f5cf81599c0 2023-05-20T01:27:25.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:25.289+0000 7f5cffc45700 1 -- 172.21.15.16:0/1658698033 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5cf81599c0 msgr2=0x7f5cf80caf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:25.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:25.289+0000 7f5cffc45700 1 --2- 172.21.15.16:0/1658698033 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5cf81599c0 0x7f5cf80caf60 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f5cf80cc990 tx=0x7f5cf4009bd0 comp rx=0 tx=0).stop 2023-05-20T01:27:25.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:25.289+0000 7f5cffc45700 1 -- 172.21.15.16:0/1658698033 shutdown_connections 2023-05-20T01:27:25.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:25.289+0000 7f5cffc45700 1 --2- 172.21.15.16:0/1658698033 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5cf806f3f0 0x7f5cf80710d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:25.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:25.289+0000 7f5cffc45700 1 --2- 172.21.15.16:0/1658698033 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5cf81599c0 0x7f5cf80caf60 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:25.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:25.289+0000 7f5cffc45700 1 --2- 172.21.15.16:0/1658698033 >> v2:172.21.15.132:3300/0 conn(0x7f5cf80caaa0 0x7f5cf80cb4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:25.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:25.289+0000 7f5cffc45700 1 -- 172.21.15.16:0/1658698033 >> 172.21.15.16:0/1658698033 conn(0x7f5cf80d0c30 msgr2=0x7f5cf80b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:27:25.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:25.289+0000 7f5cffc45700 1 -- 172.21.15.16:0/1658698033 shutdown_connections 2023-05-20T01:27:25.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:25.289+0000 7f5cffc45700 1 -- 172.21.15.16:0/1658698033 wait complete. 2023-05-20T01:27:25.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:25.291+0000 7f5cffc45700 1 Processor -- start 2023-05-20T01:27:25.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:25.291+0000 7f5cffc45700 1 -- start start 2023-05-20T01:27:25.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:25.292+0000 7f5cffc45700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5cf806f3f0 0x7f5cf81f13e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:25.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:25.292+0000 7f5cffc45700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5cf80caaa0 0x7f5cf80b57e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:25.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:25.292+0000 7f5cffc45700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5cf81599c0 0x7f5cf815c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:25.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:25.292+0000 7f5cffc45700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cf81f2310 con 0x7f5cf80caaa0 2023-05-20T01:27:25.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:25.292+0000 7f5cffc45700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cf81f1a70 con 0x7f5cf81599c0 2023-05-20T01:27:25.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:25.292+0000 7f5cffc45700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cf81f1d50 con 0x7f5cf806f3f0 2023-05-20T01:27:25.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:25.292+0000 7f5cfd1e0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5cf80caaa0 0x7f5cf80b57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:25.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:25.292+0000 7f5cfd1e0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5cf80caaa0 0x7f5cf80b57e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39160/0 (socket says 172.21.15.16:39160) 2023-05-20T01:27:25.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:25.292+0000 7f5cfd1e0700 1 -- 172.21.15.16:0/2443884097 learned_addr learned my addr 172.21.15.16:0/2443884097 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:27:25.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:25.292+0000 7f5cfd9e1700 1 --2- 172.21.15.16:0/2443884097 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5cf806f3f0 0x7f5cf81f13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:25.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:25.292+0000 7f5cfe1e2700 1 --2- 172.21.15.16:0/2443884097 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5cf81599c0 0x7f5cf815c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:25.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:25.292+0000 7f5cfd1e0700 1 -- 172.21.15.16:0/2443884097 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5cf806f3f0 msgr2=0x7f5cf81f13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:25.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:25.292+0000 7f5cfd1e0700 1 --2- 172.21.15.16:0/2443884097 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5cf806f3f0 0x7f5cf81f13e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:25.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:25.292+0000 7f5cfd1e0700 1 -- 172.21.15.16:0/2443884097 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5cf81599c0 msgr2=0x7f5cf815c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:25.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:25.292+0000 7f5cfd1e0700 1 --2- 172.21.15.16:0/2443884097 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5cf81599c0 0x7f5cf815c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:25.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:25.292+0000 7f5cfd1e0700 1 -- 172.21.15.16:0/2443884097 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cf4009550 con 0x7f5cf80caaa0 2023-05-20T01:27:25.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:25.292+0000 7f5cfd9e1700 1 --2- 172.21.15.16:0/2443884097 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5cf806f3f0 0x7f5cf81f13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:27:25.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:25.292+0000 7f5cfd1e0700 1 --2- 172.21.15.16:0/2443884097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5cf80caaa0 0x7f5cf80b57e0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f5cec00c7f0 tx=0x7f5cec00cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:25.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:25.293+0000 7f5cea7fc700 1 -- 172.21.15.16:0/2443884097 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cec011800 con 0x7f5cf80caaa0 2023-05-20T01:27:25.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:25.293+0000 7f5cea7fc700 1 -- 172.21.15.16:0/2443884097 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cec011960 con 0x7f5cf80caaa0 2023-05-20T01:27:25.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:25.293+0000 7f5cea7fc700 1 -- 172.21.15.16:0/2443884097 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cec011cf0 con 0x7f5cf80caaa0 2023-05-20T01:27:25.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:25.293+0000 7f5cffc45700 1 -- 172.21.15.16:0/2443884097 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cf806f0b0 con 0x7f5cf80caaa0 2023-05-20T01:27:25.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:25.293+0000 7f5cffc45700 1 -- 172.21.15.16:0/2443884097 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cf806edd0 con 0x7f5cf80caaa0 2023-05-20T01:27:25.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:25.294+0000 7f5cffc45700 1 -- 172.21.15.16:0/2443884097 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cf806e750 con 0x7f5cf80caaa0 2023-05-20T01:27:25.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:25.294+0000 7f5cea7fc700 1 -- 172.21.15.16:0/2443884097 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5cec018360 con 0x7f5cf80caaa0 2023-05-20T01:27:25.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:25.295+0000 7f5cea7fc700 1 --2- 172.21.15.16:0/2443884097 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f5cdc06ace0 0x7f5cdc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:25.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:25.295+0000 7f5cea7fc700 1 -- 172.21.15.16:0/2443884097 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 7073+0+0 (secure 0 0 0) 0x7f5cec089860 con 0x7f5cf80caaa0 2023-05-20T01:27:25.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:25.295+0000 7f5cfd9e1700 1 --2- 172.21.15.16:0/2443884097 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f5cdc06ace0 0x7f5cdc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:25.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:25.295+0000 7f5cfd9e1700 1 --2- 172.21.15.16:0/2443884097 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f5cdc06ace0 0x7f5cdc06d190 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f5cf4009130 tx=0x7f5cf4008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:25.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:25.299+0000 7f5cea7fc700 1 -- 172.21.15.16:0/2443884097 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f5cec0545a0 con 0x7f5cf80caaa0 2023-05-20T01:27:25.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:25.543+0000 7f5cffc45700 1 -- 172.21.15.16:0/2443884097 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5cf80747b0 con 0x7f5cf80caaa0 2023-05-20T01:27:25.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:25.543+0000 7f5cea7fc700 1 -- 172.21.15.16:0/2443884097 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+4681 (secure 0 0 0) 0x7f5cec057bc0 con 0x7f5cf80caaa0 2023-05-20T01:27:25.548 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:27:25.548 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":91,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6845,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/1435235487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1435235487},{"type":"v1","addr":"172.21.15.132:6834","nonce":1435235487}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":87},{"gid":6848,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/933893285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":933893285},{"type":"v1","addr":"172.21.15.132:6835","nonce":933893285}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":87},{"gid":8373,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/1220815342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1220815342},{"type":"v1","addr":"172.21.15.16:6836","nonce":1220815342}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":87}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-20T01:27:10.291795+0000","modified":"2023-05-20T01:27:25.446248+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":121,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8376},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8376":{"gid":8376,"name":"c","rank":0,"incarnation":88,"state":"up:active","state_seq":4,"addr":"172.21.15.16:6837/1927078882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":1927078882},{"type":"v1","addr":"172.21.15.16:6837","nonce":1927078882}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16,17],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-05-20T01:27:25.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:25.545+0000 7f5cffc45700 1 -- 172.21.15.16:0/2443884097 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f5cdc06ace0 msgr2=0x7f5cdc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:25.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:25.545+0000 7f5cffc45700 1 --2- 172.21.15.16:0/2443884097 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f5cdc06ace0 0x7f5cdc06d190 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f5cf4009130 tx=0x7f5cf4008030 comp rx=0 tx=0).stop 2023-05-20T01:27:25.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:25.545+0000 7f5cffc45700 1 -- 172.21.15.16:0/2443884097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5cf80caaa0 msgr2=0x7f5cf80b57e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:25.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:25.545+0000 7f5cffc45700 1 --2- 172.21.15.16:0/2443884097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5cf80caaa0 0x7f5cf80b57e0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f5cec00c7f0 tx=0x7f5cec00cbb0 comp rx=0 tx=0).stop 2023-05-20T01:27:25.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:25.545+0000 7f5cffc45700 1 -- 172.21.15.16:0/2443884097 shutdown_connections 2023-05-20T01:27:25.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:25.545+0000 7f5cffc45700 1 --2- 172.21.15.16:0/2443884097 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5cf81599c0 0x7f5cf815c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:25.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:25.545+0000 7f5cffc45700 1 --2- 172.21.15.16:0/2443884097 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f5cdc06ace0 0x7f5cdc06d190 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:25.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:25.545+0000 7f5cffc45700 1 --2- 172.21.15.16:0/2443884097 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5cf806f3f0 0x7f5cf81f13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:25.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:25.545+0000 7f5cffc45700 1 --2- 172.21.15.16:0/2443884097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5cf80caaa0 0x7f5cf80b57e0 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:25.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:25.545+0000 7f5cffc45700 1 -- 172.21.15.16:0/2443884097 >> 172.21.15.16:0/2443884097 conn(0x7f5cf80d0c30 msgr2=0x7f5cf80afab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:27:25.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:25.545+0000 7f5cffc45700 1 -- 172.21.15.16:0/2443884097 shutdown_connections 2023-05-20T01:27:25.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:25.545+0000 7f5cffc45700 1 -- 172.21.15.16:0/2443884097 wait complete. 2023-05-20T01:27:25.552 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 91 2023-05-20T01:27:25.965 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 91, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-05-20T01:27:10.291795+0000', 'modified': '2023-05-20T01:27:25.446248+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 121, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8376}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8376': {'gid': 8376, 'name': 'c', 'rank': 0, 'incarnation': 88, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.16:6837/1927078882', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6835', 'nonce': 1927078882}, {'type': 'v1', 'addr': '172.21.15.16:6837', 'nonce': 1927078882}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [16, 17], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-05-20T01:27:25.965 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-05-20T01:27:25.966 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-05-20T01:27:26.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.092+0000 7f6a0b9c5700 1 Processor -- start 2023-05-20T01:27:26.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.092+0000 7f6a0b9c5700 1 -- start start 2023-05-20T01:27:26.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.093+0000 7f6a0b9c5700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6a04161190 0x7f6a0406de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:26.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.093+0000 7f6a0b9c5700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f6a0406ac20 0x7f6a04071140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:26.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.093+0000 7f6a0b9c5700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a0415d020 con 0x7f6a0406e370 2023-05-20T01:27:26.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.093+0000 7f6a0b9c5700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a0406e7c0 con 0x7f6a0406ac20 2023-05-20T01:27:26.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.093+0000 7f6a0b9c5700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a040cebf0 con 0x7f6a04161190 2023-05-20T01:27:26.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.093+0000 7f6a08f60700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f6a0406e370 0x7f6a0406a510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50694/0 (socket says 172.21.15.16:50694) 2023-05-20T01:27:26.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.093+0000 7f6a08f60700 1 -- 172.21.15.16:0/2356257986 learned_addr learned my addr 172.21.15.16:0/2356257986 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:27:26.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.093+0000 7f6a09f62700 1 --2- 172.21.15.16:0/2356257986 >> v2:172.21.15.132:3300/0 conn(0x7f6a0406ac20 0x7f6a04071140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:26.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.093+0000 7f6a09761700 1 --2- 172.21.15.16:0/2356257986 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6a04161190 0x7f6a0406de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:26.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.094+0000 7f69fbfff700 1 -- 172.21.15.16:0/2356257986 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3817754645 0 0) 0x7f6a0415d020 con 0x7f6a0406e370 2023-05-20T01:27:26.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.094+0000 7f69fbfff700 1 -- 172.21.15.16:0/2356257986 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f69f00035b0 con 0x7f6a0406e370 2023-05-20T01:27:26.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.094+0000 7f69fbfff700 1 -- 172.21.15.16:0/2356257986 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1875380358 0 0) 0x7f69f00035b0 con 0x7f6a0406e370 2023-05-20T01:27:26.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.094+0000 7f69fbfff700 1 -- 172.21.15.16:0/2356257986 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6a04161190 msgr2=0x7f6a0406de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:26.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.094+0000 7f69fbfff700 1 --2- 172.21.15.16:0/2356257986 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6a04161190 0x7f6a0406de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:26.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.094+0000 7f69fbfff700 1 -- 172.21.15.16:0/2356257986 >> v2:172.21.15.132:3300/0 conn(0x7f6a0406ac20 msgr2=0x7f6a04071140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:26.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.094+0000 7f69fbfff700 1 --2- 172.21.15.16:0/2356257986 >> v2:172.21.15.132:3300/0 conn(0x7f6a0406ac20 0x7f6a04071140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:26.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.094+0000 7f69fbfff700 1 -- 172.21.15.16:0/2356257986 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a0406e9a0 con 0x7f6a0406e370 2023-05-20T01:27:26.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.094+0000 7f69fbfff700 1 -- 172.21.15.16:0/2356257986 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7f6a00001670 con 0x7f6a0406e370 2023-05-20T01:27:26.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.094+0000 7f69fbfff700 1 -- 172.21.15.16:0/2356257986 >> v1:172.21.15.16:6789/0 conn(0x7f6a0406e370 legacy=0x7f6a0406a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:26.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.094+0000 7f69fbfff700 1 --2- 172.21.15.16:0/2356257986 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f69f0003a70 0x7f69f0003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:26.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.094+0000 7f69fbfff700 1 --2- 172.21.15.16:0/2356257986 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6a04161190 0x7f69f0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:26.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.094+0000 7f69fbfff700 1 --2- 172.21.15.16:0/2356257986 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69f0004d90 0x7f69f0005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:26.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.094+0000 7f69fbfff700 1 -- 172.21.15.16:0/2356257986 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a0406e7c0 con 0x7f69f0004d90 2023-05-20T01:27:26.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.094+0000 7f69fbfff700 1 -- 172.21.15.16:0/2356257986 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a040cebf0 con 0x7f69f0003a70 2023-05-20T01:27:26.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.094+0000 7f69fbfff700 1 -- 172.21.15.16:0/2356257986 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69f000ba40 con 0x7f6a04161190 2023-05-20T01:27:26.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.094+0000 7f6a09761700 1 --2- 172.21.15.16:0/2356257986 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f69f0003a70 0x7f69f0003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:26.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.094+0000 7f6a09f62700 1 --2- 172.21.15.16:0/2356257986 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69f0004d90 0x7f69f0005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:26.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.094+0000 7f6a08f60700 1 --2- 172.21.15.16:0/2356257986 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6a04161190 0x7f69f0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:26.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.094+0000 7f6a09f62700 1 -- 172.21.15.16:0/2356257986 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6a04161190 msgr2=0x7f69f0004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:26.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.094+0000 7f6a09f62700 1 --2- 172.21.15.16:0/2356257986 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6a04161190 0x7f69f0004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:26.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.094+0000 7f6a09f62700 1 -- 172.21.15.16:0/2356257986 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f69f0003a70 msgr2=0x7f69f0003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:26.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.094+0000 7f6a09f62700 1 --2- 172.21.15.16:0/2356257986 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f69f0003a70 0x7f69f0003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:26.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.094+0000 7f6a09f62700 1 -- 172.21.15.16:0/2356257986 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69f00035b0 con 0x7f69f0004d90 2023-05-20T01:27:26.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.095+0000 7f6a09f62700 1 --2- 172.21.15.16:0/2356257986 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69f0004d90 0x7f69f0005200 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f69fc009f90 tx=0x7f69f0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=666000b8b8205016 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:26.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.095+0000 7f69fbfff700 1 -- 172.21.15.16:0/2356257986 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69fc00ebd0 con 0x7f69f0004d90 2023-05-20T01:27:26.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.095+0000 7f69fbfff700 1 -- 172.21.15.16:0/2356257986 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69fc00f040 con 0x7f69f0004d90 2023-05-20T01:27:26.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.095+0000 7f69fbfff700 1 -- 172.21.15.16:0/2356257986 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69fc014480 con 0x7f69f0004d90 2023-05-20T01:27:26.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.095+0000 7f6a0b9c5700 1 -- 172.21.15.16:0/2356257986 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69f0004d90 msgr2=0x7f69f0005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:26.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.095+0000 7f6a0b9c5700 1 --2- 172.21.15.16:0/2356257986 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69f0004d90 0x7f69f0005200 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f69fc009f90 tx=0x7f69f0005880 comp rx=0 tx=0).stop 2023-05-20T01:27:26.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.095+0000 7f6a09f62700 1 -- 172.21.15.16:0/2356257986 reap_dead start 2023-05-20T01:27:26.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.095+0000 7f6a0b9c5700 1 -- 172.21.15.16:0/2356257986 shutdown_connections 2023-05-20T01:27:26.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.095+0000 7f6a0b9c5700 1 -- 172.21.15.16:0/2356257986 >> 172.21.15.16:0/2356257986 conn(0x7f6a040d0c70 msgr2=0x7f6a040b1af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:27:26.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.095+0000 7f6a0b9c5700 1 -- 172.21.15.16:0/2356257986 shutdown_connections 2023-05-20T01:27:26.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.095+0000 7f6a0b9c5700 1 -- 172.21.15.16:0/2356257986 wait complete. 2023-05-20T01:27:26.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.097+0000 7f6a0b9c5700 1 Processor -- start 2023-05-20T01:27:26.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.097+0000 7f6a0b9c5700 1 -- start start 2023-05-20T01:27:26.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.098+0000 7f6a0b9c5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a040aec70 0x7f6a04153fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:26.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.098+0000 7f6a0b9c5700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6a040cc190 0x7f6a041544f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:26.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.098+0000 7f6a0b9c5700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6a040b1870 0x7f6a041f5a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:26.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.098+0000 7f6a0b9c5700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a040b0a40 con 0x7f6a040aec70 2023-05-20T01:27:26.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.098+0000 7f6a0b9c5700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a040c3be0 con 0x7f6a040b1870 2023-05-20T01:27:26.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.098+0000 7f6a0b9c5700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a040b0780 con 0x7f6a040cc190 2023-05-20T01:27:26.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.098+0000 7f6a09761700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a040aec70 0x7f6a04153fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:26.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.098+0000 7f6a09761700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a040aec70 0x7f6a04153fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39176/0 (socket says 172.21.15.16:39176) 2023-05-20T01:27:26.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.098+0000 7f6a09761700 1 -- 172.21.15.16:0/2903548783 learned_addr learned my addr 172.21.15.16:0/2903548783 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:27:26.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.098+0000 7f6a09761700 1 -- 172.21.15.16:0/2903548783 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6a040cc190 msgr2=0x7f6a041544f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:26.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.098+0000 7f6a09761700 1 --2- 172.21.15.16:0/2903548783 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6a040cc190 0x7f6a041544f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:26.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.098+0000 7f6a09761700 1 -- 172.21.15.16:0/2903548783 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6a040b1870 msgr2=0x7f6a041f5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:26.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.098+0000 7f6a09761700 1 --2- 172.21.15.16:0/2903548783 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6a040b1870 0x7f6a041f5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:26.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.098+0000 7f6a09761700 1 -- 172.21.15.16:0/2903548783 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a040cf170 con 0x7f6a040aec70 2023-05-20T01:27:26.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.098+0000 7f6a09761700 1 --2- 172.21.15.16:0/2903548783 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a040aec70 0x7f6a04153fb0 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f69f40119a0 tx=0x7f69f4011d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:26.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.098+0000 7f69f9ffb700 1 -- 172.21.15.16:0/2903548783 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69f400f940 con 0x7f6a040aec70 2023-05-20T01:27:26.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.098+0000 7f69f9ffb700 1 -- 172.21.15.16:0/2903548783 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69f400faa0 con 0x7f6a040aec70 2023-05-20T01:27:26.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.098+0000 7f6a0b9c5700 1 -- 172.21.15.16:0/2903548783 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a040cee90 con 0x7f6a040aec70 2023-05-20T01:27:26.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.098+0000 7f69f9ffb700 1 -- 172.21.15.16:0/2903548783 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69f400fe50 con 0x7f6a040aec70 2023-05-20T01:27:26.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.099+0000 7f6a0b9c5700 1 -- 172.21.15.16:0/2903548783 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a040cf450 con 0x7f6a040aec70 2023-05-20T01:27:26.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.100+0000 7f69f9ffb700 1 -- 172.21.15.16:0/2903548783 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f69f401d400 con 0x7f6a040aec70 2023-05-20T01:27:26.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.100+0000 7f69f9ffb700 1 --2- 172.21.15.16:0/2903548783 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f69e806ace0 0x7f69e806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:26.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.101+0000 7f69f9ffb700 1 -- 172.21.15.16:0/2903548783 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 7073+0+0 (secure 0 0 0) 0x7f69f4089f70 con 0x7f6a040aec70 2023-05-20T01:27:26.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.101+0000 7f6a08f60700 1 --2- 172.21.15.16:0/2903548783 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f69e806ace0 0x7f69e806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:26.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.101+0000 7f6a08f60700 1 --2- 172.21.15.16:0/2903548783 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f69e806ace0 0x7f69e806d190 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f6a000018a0 tx=0x7f6a00001810 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:26.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.102+0000 7f6a17b51b80 1 Processor -- start 2023-05-20T01:27:26.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.102+0000 7f6a17b51b80 1 -- start start 2023-05-20T01:27:26.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.103+0000 7f6a17b51b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5598e5811400 0x5598e58117d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:26.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.103+0000 7f6a17b51b80 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5598e580d700 0x5598e57e6460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:26.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.103+0000 7f6a17b51b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598e57e4870 0x5598e57e69a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:26.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.103+0000 7f6a17b51b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598e56ec010 con 0x5598e57e4870 2023-05-20T01:27:26.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.103+0000 7f6a17b51b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598e573e850 con 0x5598e5811400 2023-05-20T01:27:26.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.103+0000 7f6a17b51b80 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x5598e5737ce0 con 0x5598e580d700 2023-05-20T01:27:26.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.103+0000 7f6a09f62700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598e57e4870 0x5598e57e69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:26.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.103+0000 7f6a09f62700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598e57e4870 0x5598e57e69a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39188/0 (socket says 172.21.15.16:39188) 2023-05-20T01:27:26.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.103+0000 7f6a09f62700 1 -- 172.21.15.16:0/833767059 learned_addr learned my addr 172.21.15.16:0/833767059 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:27:26.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.104+0000 7f6a08f60700 1 --2- 172.21.15.16:0/833767059 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5598e580d700 0x5598e57e6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:26.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.104+0000 7f6a09f62700 1 -- 172.21.15.16:0/833767059 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5598e580d700 msgr2=0x5598e57e6460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:26.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.104+0000 7f6a09761700 1 --2- 172.21.15.16:0/833767059 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5598e5811400 0x5598e58117d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:26.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.104+0000 7f6a09f62700 1 --2- 172.21.15.16:0/833767059 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5598e580d700 0x5598e57e6460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:26.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.104+0000 7f6a09f62700 1 -- 172.21.15.16:0/833767059 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5598e5811400 msgr2=0x5598e58117d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:26.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.104+0000 7f6a09f62700 1 --2- 172.21.15.16:0/833767059 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5598e5811400 0x5598e58117d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:26.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.104+0000 7f6a09f62700 1 -- 172.21.15.16:0/833767059 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5598e55b95e0 con 0x5598e57e4870 2023-05-20T01:27:26.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.104+0000 7f6a09f62700 1 --2- 172.21.15.16:0/833767059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598e57e4870 0x5598e57e69a0 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f69fc009e30 tx=0x7f69fc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=320dfcc6206519e0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:26.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.104+0000 7f69eeffd700 1 -- 172.21.15.16:0/833767059 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69fc007000 con 0x5598e57e4870 2023-05-20T01:27:26.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.104+0000 7f69eeffd700 1 -- 172.21.15.16:0/833767059 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69fc003c40 con 0x5598e57e4870 2023-05-20T01:27:26.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.104+0000 7f69eeffd700 1 -- 172.21.15.16:0/833767059 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69fc00e9d0 con 0x5598e57e4870 2023-05-20T01:27:26.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.105+0000 7f6a17b51b80 1 -- 172.21.15.16:0/833767059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598e57e4870 msgr2=0x5598e57e69a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:26.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.105+0000 7f6a17b51b80 1 --2- 172.21.15.16:0/833767059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598e57e4870 0x5598e57e69a0 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f69fc009e30 tx=0x7f69fc004ad0 comp rx=0 tx=0).stop 2023-05-20T01:27:26.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.105+0000 7f6a17b51b80 1 -- 172.21.15.16:0/833767059 shutdown_connections 2023-05-20T01:27:26.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.105+0000 7f6a17b51b80 1 --2- 172.21.15.16:0/833767059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598e57e4870 0x5598e57e69a0 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:26.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.105+0000 7f6a17b51b80 1 --2- 172.21.15.16:0/833767059 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5598e5811400 0x5598e58117d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:26.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.105+0000 7f6a17b51b80 1 --2- 172.21.15.16:0/833767059 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5598e580d700 0x5598e57e6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:26.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.105+0000 7f6a17b51b80 1 -- 172.21.15.16:0/833767059 >> 172.21.15.16:0/833767059 conn(0x5598e57dbf30 msgr2=0x5598e582c210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:27:26.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.105+0000 7f6a17b51b80 1 -- 172.21.15.16:0/833767059 shutdown_connections 2023-05-20T01:27:26.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.105+0000 7f6a17b51b80 1 -- 172.21.15.16:0/833767059 wait complete. 2023-05-20T01:27:26.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.105+0000 7f6a17b51b80 1 -- start start 2023-05-20T01:27:26.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.105+0000 7f6a17b51b80 1 Processor -- start 2023-05-20T01:27:26.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.106+0000 7f6a17b51b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5598e57e4870 0x5598e5806100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:26.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.106+0000 7f6a17b51b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598e580d700 0x5598e582d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:26.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.106+0000 7f6a17b51b80 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5598e5811400 0x5598e582dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:26.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.106+0000 7f6a17b51b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598e56d9890 con 0x5598e580d700 2023-05-20T01:27:26.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.106+0000 7f6a17b51b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598e56d9530 con 0x5598e57e4870 2023-05-20T01:27:26.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.106+0000 7f6a17b51b80 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x5598e55bb8d0 con 0x5598e5811400 2023-05-20T01:27:26.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.106+0000 7f6a17b51b80 10 client.0 _ll_register_callbacks cb 0x5598e56f7730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-05-20T01:27:26.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.106+0000 7f6a08f60700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598e580d700 0x5598e582d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:26.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.106+0000 7f6a08f60700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598e580d700 0x5598e582d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39192/0 (socket says 172.21.15.16:39192) 2023-05-20T01:27:26.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.106+0000 7f6a08f60700 1 -- 172.21.15.16:0/3736814049 learned_addr learned my addr 172.21.15.16:0/3736814049 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:27:26.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.106+0000 7f6a09761700 1 --2- 172.21.15.16:0/3736814049 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5598e57e4870 0x5598e5806100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:26.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.106+0000 7f6a09f62700 1 --2- 172.21.15.16:0/3736814049 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5598e5811400 0x5598e582dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:26.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.107+0000 7f6a08f60700 1 -- 172.21.15.16:0/3736814049 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5598e5811400 msgr2=0x5598e582dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:26.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.107+0000 7f6a08f60700 1 --2- 172.21.15.16:0/3736814049 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5598e5811400 0x5598e582dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:26.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.107+0000 7f6a08f60700 1 -- 172.21.15.16:0/3736814049 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5598e57e4870 msgr2=0x5598e5806100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:26.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.107+0000 7f6a08f60700 1 --2- 172.21.15.16:0/3736814049 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5598e57e4870 0x5598e5806100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:26.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.107+0000 7f6a08f60700 1 -- 172.21.15.16:0/3736814049 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69f00035b0 con 0x5598e580d700 2023-05-20T01:27:26.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.107+0000 7f6a09f62700 1 --2- 172.21.15.16:0/3736814049 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5598e5811400 0x5598e582dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:27:26.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.107+0000 7f6a08f60700 1 --2- 172.21.15.16:0/3736814049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598e580d700 0x5598e582d690 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f6a00001050 tx=0x7f6a00001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44b3f41af1efade0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:26.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.107+0000 7f69e3fff700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-05-20T01:27:26.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.107+0000 7f69e3fff700 1 -- 172.21.15.16:0/3736814049 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a00001680 con 0x5598e580d700 2023-05-20T01:27:26.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.107+0000 7f6a17b51b80 1 -- 172.21.15.16:0/3736814049 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5598e57d96f0 con 0x5598e580d700 2023-05-20T01:27:26.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.107+0000 7f69e3fff700 1 -- 172.21.15.16:0/3736814049 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(123..123 src has 1..123) v4 ==== 7073+0+0 (secure 0 0 0) 0x7f6a0001b040 con 0x5598e580d700 2023-05-20T01:27:26.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.107+0000 7f69e3fff700 1 -- 172.21.15.16:0/3736814049 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6a000077a0 con 0x5598e580d700 2023-05-20T01:27:26.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.107+0000 7f6a17b51b80 10 client.8406 fetch_fsmap learned FSMap version 91 2023-05-20T01:27:26.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.107+0000 7f6a17b51b80 1 -- 172.21.15.16:0/3736814049 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) v3 -- 0x5598e56fe140 con 0x5598e580d700 2023-05-20T01:27:26.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.107+0000 7f69e3fff700 1 -- 172.21.15.16:0/3736814049 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5598e57d96f0 con 0x5598e580d700 2023-05-20T01:27:26.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.107+0000 7f69e3fff700 1 -- 172.21.15.16:0/3736814049 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 91) v1 ==== 3122+0+0 (secure 0 0 0) 0x7f6a00027260 con 0x5598e580d700 2023-05-20T01:27:26.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.107+0000 7f6a17b51b80 10 client.8406 fetch_fsmap finished waiting for FSMap version 91 2023-05-20T01:27:26.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.107+0000 7f6a17b51b80 10 client.8406 resolve_mds: resolved name 'c' to daemon mds.c 2023-05-20T01:27:26.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.107+0000 7f6a17b51b80 20 client.8406 populate_metadata read hostname 'smithi016' 2023-05-20T01:27:26.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.107+0000 7f6a17b51b80 1 --2- 172.21.15.16:0/3736814049 >> [v2:172.21.15.16:6835/1927078882,v1:172.21.15.16:6837/1927078882] conn(0x5598e5808de0 0x5598e58091b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:26.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.107+0000 7f6a17b51b80 4 client.8406 mds_command: new command op to 8376 tid=0[{"prefix": "get_command_descriptions"}] 2023-05-20T01:27:26.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.107+0000 7f6a17b51b80 1 -- 172.21.15.16:0/3736814049 --> [v2:172.21.15.16:6835/1927078882,v1:172.21.15.16:6837/1927078882] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5598e56fa050 con 0x5598e5808de0 2023-05-20T01:27:26.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.108+0000 7f6a09761700 1 --2- 172.21.15.16:0/3736814049 >> [v2:172.21.15.16:6835/1927078882,v1:172.21.15.16:6837/1927078882] conn(0x5598e5808de0 0x5598e58091b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:26.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.108+0000 7f69e3fff700 10 client.8406 ms_handle_connect on v2:172.21.15.16:6835/1927078882 2023-05-20T01:27:26.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.108+0000 7f6a09761700 1 --2- 172.21.15.16:0/3736814049 >> [v2:172.21.15.16:6835/1927078882,v1:172.21.15.16:6837/1927078882] conn(0x5598e5808de0 0x5598e58091b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0f567bb5b080378 server_cookie=458fdc96b97814f4 in_seq=0 out_seq=0 2023-05-20T01:27:26.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.108+0000 7f69e3fff700 1 -- 172.21.15.16:0/3736814049 <== mds.0 v2:172.21.15.16:6835/1927078882 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5598e56fa050 con 0x5598e5808de0 2023-05-20T01:27:26.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.108+0000 7f69e3fff700 10 client.8406 handle_command_reply: tid=0 2023-05-20T01:27:26.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.108+0000 7f6a17b51b80 1 client.8406 shutdown 2023-05-20T01:27:26.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.109+0000 7f6a17b51b80 1 -- 172.21.15.16:0/3736814049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598e580d700 msgr2=0x5598e582d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:26.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.109+0000 7f6a17b51b80 1 --2- 172.21.15.16:0/3736814049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598e580d700 0x5598e582d690 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f6a00001050 tx=0x7f6a00001a00 comp rx=0 tx=0).stop 2023-05-20T01:27:26.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.109+0000 7f6a17b51b80 1 -- 172.21.15.16:0/3736814049 shutdown_connections 2023-05-20T01:27:26.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.109+0000 7f6a17b51b80 1 --2- 172.21.15.16:0/3736814049 >> [v2:172.21.15.16:6835/1927078882,v1:172.21.15.16:6837/1927078882] conn(0x5598e5808de0 0x5598e58091b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:26.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.109+0000 7f6a17b51b80 1 --2- 172.21.15.16:0/3736814049 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5598e5811400 0x5598e582dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:26.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.109+0000 7f6a17b51b80 1 --2- 172.21.15.16:0/3736814049 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5598e57e4870 0x5598e5806100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:26.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.109+0000 7f69e3fff700 0 client.8406 ms_handle_reset on v2:172.21.15.16:6835/1927078882 2023-05-20T01:27:26.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.109+0000 7f6a17b51b80 1 --2- 172.21.15.16:0/3736814049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598e580d700 0x5598e582d690 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:26.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.109+0000 7f6a17b51b80 1 -- 172.21.15.16:0/3736814049 >> 172.21.15.16:0/3736814049 conn(0x5598e57dbf30 msgr2=0x5598e57e0120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:27:26.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.109+0000 7f6a17b51b80 1 -- 172.21.15.16:0/3736814049 shutdown_connections 2023-05-20T01:27:26.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.109+0000 7f6a17b51b80 1 -- 172.21.15.16:0/3736814049 wait complete. 2023-05-20T01:27:26.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.109+0000 7f6a17b51b80 20 client.8406 trim_cache size 0 max 16384 2023-05-20T01:27:26.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.127+0000 7f6a17b51b80 1 Processor -- start 2023-05-20T01:27:26.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.127+0000 7f6a17b51b80 1 -- start start 2023-05-20T01:27:26.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.128+0000 7f6a17b51b80 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5598e57e4870 0x5598e57e1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:26.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.128+0000 7f6a17b51b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5598e580d700 0x5598e580dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:26.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.128+0000 7f6a17b51b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598e5811400 0x5598e580e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:26.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.128+0000 7f6a17b51b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598e57e29c0 con 0x5598e5811400 2023-05-20T01:27:26.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.128+0000 7f6a17b51b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598e582ba30 con 0x5598e580d700 2023-05-20T01:27:26.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.128+0000 7f6a17b51b80 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x5598e582c700 con 0x5598e57e4870 2023-05-20T01:27:26.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.128+0000 7f6a09f62700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598e5811400 0x5598e580e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:26.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.128+0000 7f6a09f62700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598e5811400 0x5598e580e010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39202/0 (socket says 172.21.15.16:39202) 2023-05-20T01:27:26.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.128+0000 7f6a09f62700 1 -- 172.21.15.16:0/2980463625 learned_addr learned my addr 172.21.15.16:0/2980463625 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:27:26.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.128+0000 7f6a08f60700 1 --2- 172.21.15.16:0/2980463625 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5598e580d700 0x5598e580dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:26.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.128+0000 7f6a09761700 1 --2- 172.21.15.16:0/2980463625 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5598e57e4870 0x5598e57e1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:26.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.128+0000 7f6a09f62700 1 -- 172.21.15.16:0/2980463625 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5598e57e4870 msgr2=0x5598e57e1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:26.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.128+0000 7f6a09f62700 1 --2- 172.21.15.16:0/2980463625 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5598e57e4870 0x5598e57e1c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:26.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.128+0000 7f6a09f62700 1 -- 172.21.15.16:0/2980463625 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5598e580d700 msgr2=0x5598e580dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:26.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.128+0000 7f6a09f62700 1 --2- 172.21.15.16:0/2980463625 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5598e580d700 0x5598e580dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:26.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.128+0000 7f6a09f62700 1 -- 172.21.15.16:0/2980463625 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5598e56fde90 con 0x5598e5811400 2023-05-20T01:27:26.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.128+0000 7f6a09761700 1 --2- 172.21.15.16:0/2980463625 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5598e57e4870 0x5598e57e1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:27:26.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.128+0000 7f6a08f60700 1 --2- 172.21.15.16:0/2980463625 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5598e580d700 0x5598e580dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:27:26.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.128+0000 7f6a09f62700 1 --2- 172.21.15.16:0/2980463625 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598e5811400 0x5598e580e010 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f69fc009fd0 tx=0x7f69fc013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fceaef7b5484070 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:26.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.129+0000 7f69ef7fe700 1 -- 172.21.15.16:0/2980463625 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69fc003e70 con 0x5598e5811400 2023-05-20T01:27:26.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.129+0000 7f69ef7fe700 1 -- 172.21.15.16:0/2980463625 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69fc00e9d0 con 0x5598e5811400 2023-05-20T01:27:26.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.129+0000 7f69ef7fe700 1 -- 172.21.15.16:0/2980463625 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69fc00ed80 con 0x5598e5811400 2023-05-20T01:27:26.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.129+0000 7f6a17b51b80 1 -- 172.21.15.16:0/2980463625 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598e5811400 msgr2=0x5598e580e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:26.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.129+0000 7f6a17b51b80 1 --2- 172.21.15.16:0/2980463625 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598e5811400 0x5598e580e010 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f69fc009fd0 tx=0x7f69fc013000 comp rx=0 tx=0).stop 2023-05-20T01:27:26.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.129+0000 7f6a17b51b80 1 -- 172.21.15.16:0/2980463625 shutdown_connections 2023-05-20T01:27:26.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.129+0000 7f6a17b51b80 1 --2- 172.21.15.16:0/2980463625 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598e5811400 0x5598e580e010 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:26.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.129+0000 7f6a17b51b80 1 --2- 172.21.15.16:0/2980463625 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5598e57e4870 0x5598e57e1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:26.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.129+0000 7f6a17b51b80 1 --2- 172.21.15.16:0/2980463625 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5598e580d700 0x5598e580dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:26.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.129+0000 7f6a17b51b80 1 -- 172.21.15.16:0/2980463625 >> 172.21.15.16:0/2980463625 conn(0x5598e57dbf30 msgr2=0x5598e57dd400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:27:26.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.129+0000 7f6a17b51b80 1 -- 172.21.15.16:0/2980463625 shutdown_connections 2023-05-20T01:27:26.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.129+0000 7f6a17b51b80 1 -- 172.21.15.16:0/2980463625 wait complete. 2023-05-20T01:27:26.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.130+0000 7f6a17b51b80 1 -- start start 2023-05-20T01:27:26.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.130+0000 7f6a17b51b80 1 Processor -- start 2023-05-20T01:27:26.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.131+0000 7f6a17b51b80 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5598e57e4870 0x5598e58920b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:26.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.131+0000 7f6a17b51b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598e580d700 0x5598e580dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:26.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.131+0000 7f6a17b51b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5598e5811400 0x5598e580e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:26.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.131+0000 7f6a17b51b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598e5828a80 con 0x5598e580d700 2023-05-20T01:27:26.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.131+0000 7f6a17b51b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598e5896320 con 0x5598e5811400 2023-05-20T01:27:26.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.131+0000 7f6a17b51b80 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x5598e580cf80 con 0x5598e57e4870 2023-05-20T01:27:26.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.131+0000 7f6a17b51b80 10 client.0 _ll_register_callbacks cb 0x7f6a000076d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-05-20T01:27:26.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.131+0000 7f6a08f60700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598e580d700 0x5598e580dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:26.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.131+0000 7f6a08f60700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598e580d700 0x5598e580dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39206/0 (socket says 172.21.15.16:39206) 2023-05-20T01:27:26.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.131+0000 7f6a08f60700 1 -- 172.21.15.16:0/268585257 learned_addr learned my addr 172.21.15.16:0/268585257 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:27:26.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.131+0000 7f6a09761700 1 --2- 172.21.15.16:0/268585257 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5598e57e4870 0x5598e58920b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:26.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.131+0000 7f6a09f62700 1 --2- 172.21.15.16:0/268585257 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5598e5811400 0x5598e580e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:26.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.131+0000 7f6a08f60700 1 -- 172.21.15.16:0/268585257 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5598e57e4870 msgr2=0x5598e58920b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:26.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.131+0000 7f6a08f60700 1 --2- 172.21.15.16:0/268585257 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5598e57e4870 0x5598e58920b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:26.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.131+0000 7f6a08f60700 1 -- 172.21.15.16:0/268585257 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5598e5811400 msgr2=0x5598e580e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:26.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.131+0000 7f6a08f60700 1 --2- 172.21.15.16:0/268585257 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5598e5811400 0x5598e580e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:26.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.131+0000 7f6a08f60700 1 -- 172.21.15.16:0/268585257 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5598e55b95e0 con 0x5598e580d700 2023-05-20T01:27:26.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.131+0000 7f6a08f60700 1 --2- 172.21.15.16:0/268585257 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598e580d700 0x5598e580dad0 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x5598e57e1900 tx=0x7f6a00014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e3073e905532390 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:26.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.131+0000 7f69e3fff700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-05-20T01:27:26.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.131+0000 7f69e3fff700 1 -- 172.21.15.16:0/268585257 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a00007e60 con 0x5598e580d700 2023-05-20T01:27:26.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.132+0000 7f6a17b51b80 1 -- 172.21.15.16:0/268585257 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f69fc003c40 con 0x5598e580d700 2023-05-20T01:27:26.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.132+0000 7f69e3fff700 1 -- 172.21.15.16:0/268585257 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(123..123 src has 1..123) v4 ==== 7073+0+0 (secure 0 0 0) 0x7f6a00013d90 con 0x5598e580d700 2023-05-20T01:27:26.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.132+0000 7f69e3fff700 1 -- 172.21.15.16:0/268585257 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6a00027260 con 0x5598e580d700 2023-05-20T01:27:26.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.132+0000 7f6a17b51b80 10 client.8412 fetch_fsmap learned FSMap version 91 2023-05-20T01:27:26.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.132+0000 7f6a17b51b80 1 -- 172.21.15.16:0/268585257 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) v3 -- 0x5598e554ec10 con 0x5598e580d700 2023-05-20T01:27:26.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.132+0000 7f69e3fff700 1 -- 172.21.15.16:0/268585257 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a0003f070 con 0x5598e580d700 2023-05-20T01:27:26.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.132+0000 7f69e3fff700 1 -- 172.21.15.16:0/268585257 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 91) v1 ==== 3122+0+0 (secure 0 0 0) 0x7f6a00007780 con 0x5598e580d700 2023-05-20T01:27:26.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.132+0000 7f6a17b51b80 10 client.8412 fetch_fsmap finished waiting for FSMap version 91 2023-05-20T01:27:26.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.132+0000 7f6a17b51b80 10 client.8412 resolve_mds: resolved name 'c' to daemon mds.c 2023-05-20T01:27:26.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.132+0000 7f6a17b51b80 20 client.8412 populate_metadata read hostname 'smithi016' 2023-05-20T01:27:26.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.132+0000 7f6a17b51b80 1 --2- 172.21.15.16:0/268585257 >> [v2:172.21.15.16:6835/1927078882,v1:172.21.15.16:6837/1927078882] conn(0x5598e582d8b0 0x5598e582dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:26.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.132+0000 7f6a17b51b80 4 client.8412 mds_command: new command op to 8376 tid=0[{"prefix": "status"}] 2023-05-20T01:27:26.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.132+0000 7f6a17b51b80 1 -- 172.21.15.16:0/268585257 --> [v2:172.21.15.16:6835/1927078882,v1:172.21.15.16:6837/1927078882] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5598e5706610 con 0x5598e582d8b0 2023-05-20T01:27:26.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.132+0000 7f6a09761700 1 --2- 172.21.15.16:0/268585257 >> [v2:172.21.15.16:6835/1927078882,v1:172.21.15.16:6837/1927078882] conn(0x5598e582d8b0 0x5598e582dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:26.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.132+0000 7f69e3fff700 10 client.8412 ms_handle_connect on v2:172.21.15.16:6835/1927078882 2023-05-20T01:27:26.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.132+0000 7f6a09761700 1 --2- 172.21.15.16:0/268585257 >> [v2:172.21.15.16:6835/1927078882,v1:172.21.15.16:6837/1927078882] conn(0x5598e582d8b0 0x5598e582dc80 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6dc6e2f901a4bfd3 server_cookie=74bbaa5046682cc1 in_seq=0 out_seq=0 2023-05-20T01:27:26.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.133+0000 7f69e3fff700 1 -- 172.21.15.16:0/268585257 <== mds.0 v2:172.21.15.16:6835/1927078882 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x5598e5706610 con 0x5598e582d8b0 2023-05-20T01:27:26.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.133+0000 7f69e3fff700 10 client.8412 handle_command_reply: tid=0 2023-05-20T01:27:26.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.133+0000 7f6a17b51b80 1 client.8412 shutdown 2023-05-20T01:27:26.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.133+0000 7f6a17b51b80 1 -- 172.21.15.16:0/268585257 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598e580d700 msgr2=0x5598e580dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:26.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.133+0000 7f6a17b51b80 1 --2- 172.21.15.16:0/268585257 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598e580d700 0x5598e580dad0 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x5598e57e1900 tx=0x7f6a00014000 comp rx=0 tx=0).stop 2023-05-20T01:27:26.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.133+0000 7f6a17b51b80 1 -- 172.21.15.16:0/268585257 shutdown_connections 2023-05-20T01:27:26.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.133+0000 7f6a17b51b80 1 --2- 172.21.15.16:0/268585257 >> [v2:172.21.15.16:6835/1927078882,v1:172.21.15.16:6837/1927078882] conn(0x5598e582d8b0 0x5598e582dc80 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:26.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.133+0000 7f6a17b51b80 1 --2- 172.21.15.16:0/268585257 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5598e5811400 0x5598e580e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:26.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.133+0000 7f6a17b51b80 1 --2- 172.21.15.16:0/268585257 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5598e57e4870 0x5598e58920b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:26.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.133+0000 7f69e3fff700 0 client.8412 ms_handle_reset on v2:172.21.15.16:6835/1927078882 2023-05-20T01:27:26.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.133+0000 7f6a17b51b80 1 --2- 172.21.15.16:0/268585257 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598e580d700 0x5598e580dad0 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:26.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.133+0000 7f6a17b51b80 1 -- 172.21.15.16:0/268585257 >> 172.21.15.16:0/268585257 conn(0x5598e57dbf30 msgr2=0x5598e57e1c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:27:26.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.133+0000 7f6a17b51b80 1 -- 172.21.15.16:0/268585257 shutdown_connections 2023-05-20T01:27:26.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.133+0000 7f6a17b51b80 1 -- 172.21.15.16:0/268585257 wait complete. 2023-05-20T01:27:26.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.133+0000 7f6a17b51b80 20 client.8412 trim_cache size 0 max 16384 2023-05-20T01:27:26.152 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-05-20T01:27:26.152 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "c2245cc5-ecab-4087-a59b-bc81db901676", 2023-05-20T01:27:26.152 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2023-05-20T01:27:26.152 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 8376, 2023-05-20T01:27:26.152 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2023-05-20T01:27:26.153 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2023-05-20T01:27:26.153 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2023-05-20T01:27:26.153 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 3.6917029010000002, 2023-05-20T01:27:26.153 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 91, 2023-05-20T01:27:26.153 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 123, 2023-05-20T01:27:26.153 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 123, 2023-05-20T01:27:26.153 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 4.5378662429999999 2023-05-20T01:27:26.153 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-05-20T01:27:26.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.134+0000 7f6a0b9c5700 1 -- 172.21.15.16:0/2903548783 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f69e806ace0 msgr2=0x7f69e806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:26.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.134+0000 7f6a0b9c5700 1 --2- 172.21.15.16:0/2903548783 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f69e806ace0 0x7f69e806d190 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f6a000018a0 tx=0x7f6a00001810 comp rx=0 tx=0).stop 2023-05-20T01:27:26.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.134+0000 7f6a0b9c5700 1 -- 172.21.15.16:0/2903548783 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a040aec70 msgr2=0x7f6a04153fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:26.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.134+0000 7f6a0b9c5700 1 --2- 172.21.15.16:0/2903548783 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a040aec70 0x7f6a04153fb0 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f69f40119a0 tx=0x7f69f4011d60 comp rx=0 tx=0).stop 2023-05-20T01:27:26.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.134+0000 7f6a0b9c5700 1 -- 172.21.15.16:0/2903548783 shutdown_connections 2023-05-20T01:27:26.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.134+0000 7f6a0b9c5700 1 --2- 172.21.15.16:0/2903548783 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6a040b1870 0x7f6a041f5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:26.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.134+0000 7f6a0b9c5700 1 --2- 172.21.15.16:0/2903548783 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a040aec70 0x7f6a04153fb0 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:26.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.134+0000 7f6a0b9c5700 1 --2- 172.21.15.16:0/2903548783 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f69e806ace0 0x7f69e806d190 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:26.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.134+0000 7f6a0b9c5700 1 --2- 172.21.15.16:0/2903548783 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6a040cc190 0x7f6a041544f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:26.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.134+0000 7f6a0b9c5700 1 -- 172.21.15.16:0/2903548783 >> 172.21.15.16:0/2903548783 conn(0x7f6a040d0c70 msgr2=0x7f6a04071140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:27:26.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.134+0000 7f6a0b9c5700 1 -- 172.21.15.16:0/2903548783 shutdown_connections 2023-05-20T01:27:26.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.134+0000 7f6a0b9c5700 1 -- 172.21.15.16:0/2903548783 wait complete. 2023-05-20T01:27:26.157 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-20T01:27:26.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.277+0000 7f010cb1f700 1 Processor -- start 2023-05-20T01:27:26.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.277+0000 7f010cb1f700 1 -- start start 2023-05-20T01:27:26.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.278+0000 7f010cb1f700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f010806d2a0 0x7f0108071020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:26.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.278+0000 7f010cb1f700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01080ca350 con 0x7f01080cb320 2023-05-20T01:27:26.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.278+0000 7f010cb1f700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f010815cac0 con 0x7f0108159000 2023-05-20T01:27:26.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.278+0000 7f010cb1f700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f010815cda0 con 0x7f010806d2a0 2023-05-20T01:27:26.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.278+0000 7f0105d9b700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f01080cb320 0x7f0108070910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50742/0 (socket says 172.21.15.16:50742) 2023-05-20T01:27:26.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.278+0000 7f0105d9b700 1 -- 172.21.15.16:0/2021695989 learned_addr learned my addr 172.21.15.16:0/2021695989 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:27:26.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.278+0000 7f0106d9d700 1 --2- 172.21.15.16:0/2021695989 >> v2:172.21.15.132:3300/0 conn(0x7f010806d2a0 0x7f0108071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:26.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.278+0000 7f010559a700 1 -- 172.21.15.16:0/2021695989 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2694776834 0 0) 0x7f01080ca350 con 0x7f01080cb320 2023-05-20T01:27:26.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.278+0000 7f010559a700 1 -- 172.21.15.16:0/2021695989 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00f40046c0 con 0x7f01080cb320 2023-05-20T01:27:26.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.279+0000 7f010559a700 1 -- 172.21.15.16:0/2021695989 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 815615772 0 0) 0x7f00f40046c0 con 0x7f01080cb320 2023-05-20T01:27:26.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.279+0000 7f010559a700 1 -- 172.21.15.16:0/2021695989 >> v2:172.21.15.132:3300/0 conn(0x7f010806d2a0 msgr2=0x7f0108071020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:26.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.279+0000 7f010559a700 1 --2- 172.21.15.16:0/2021695989 >> v2:172.21.15.132:3300/0 conn(0x7f010806d2a0 0x7f0108071020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:26.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.279+0000 7f010559a700 1 -- 172.21.15.16:0/2021695989 >> v1:172.21.15.132:6789/0 conn(0x7f0108159000 legacy=0x7f010806cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:26.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.279+0000 7f010559a700 1 -- 172.21.15.16:0/2021695989 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f010815d080 con 0x7f01080cb320 2023-05-20T01:27:26.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.279+0000 7f010559a700 1 -- 172.21.15.16:0/2021695989 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7f00f8001670 con 0x7f01080cb320 2023-05-20T01:27:26.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.279+0000 7f010559a700 1 -- 172.21.15.16:0/2021695989 >> v1:172.21.15.16:6789/0 conn(0x7f01080cb320 legacy=0x7f0108070910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:26.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.279+0000 7f010559a700 1 --2- 172.21.15.16:0/2021695989 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f00f4004b80 0x7f00f4004f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:26.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.279+0000 7f010559a700 1 --2- 172.21.15.16:0/2021695989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00f4005960 0x7f00f4005dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:26.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.279+0000 7f010559a700 1 --2- 172.21.15.16:0/2021695989 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f00f4006310 0x7f00f400cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:26.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.279+0000 7f010559a700 1 -- 172.21.15.16:0/2021695989 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f010815cda0 con 0x7f00f4005960 2023-05-20T01:27:26.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.279+0000 7f010559a700 1 -- 172.21.15.16:0/2021695989 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00f400d1f0 con 0x7f00f4004b80 2023-05-20T01:27:26.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.279+0000 7f010559a700 1 -- 172.21.15.16:0/2021695989 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00f400d3b0 con 0x7f00f4006310 2023-05-20T01:27:26.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.279+0000 7f0105d9b700 1 --2- 172.21.15.16:0/2021695989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00f4005960 0x7f00f4005dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:26.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.279+0000 7f010659c700 1 --2- 172.21.15.16:0/2021695989 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f00f4004b80 0x7f00f4004f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:26.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.279+0000 7f0106d9d700 1 --2- 172.21.15.16:0/2021695989 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f00f4006310 0x7f00f400cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:26.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.279+0000 7f0105d9b700 1 -- 172.21.15.16:0/2021695989 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f00f4006310 msgr2=0x7f00f400cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:26.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.279+0000 7f0105d9b700 1 --2- 172.21.15.16:0/2021695989 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f00f4006310 0x7f00f400cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:26.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.279+0000 7f0105d9b700 1 -- 172.21.15.16:0/2021695989 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f00f4004b80 msgr2=0x7f00f4004f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:26.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.279+0000 7f0105d9b700 1 --2- 172.21.15.16:0/2021695989 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f00f4004b80 0x7f00f4004f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:26.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.279+0000 7f0105d9b700 1 -- 172.21.15.16:0/2021695989 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00f40046c0 con 0x7f00f4005960 2023-05-20T01:27:26.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.279+0000 7f0106d9d700 1 -- 172.21.15.16:0/2021695989 reap_dead start 2023-05-20T01:27:26.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.279+0000 7f0105d9b700 1 --2- 172.21.15.16:0/2021695989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00f4005960 0x7f00f4005dd0 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f00f800c210 tx=0x7f00f4006990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8cef00f70acb173 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:26.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.279+0000 7f010559a700 1 -- 172.21.15.16:0/2021695989 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f8007c10 con 0x7f00f4005960 2023-05-20T01:27:26.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.279+0000 7f010559a700 1 -- 172.21.15.16:0/2021695989 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00f800cdd0 con 0x7f00f4005960 2023-05-20T01:27:26.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.279+0000 7f010559a700 1 -- 172.21.15.16:0/2021695989 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f8005ba0 con 0x7f00f4005960 2023-05-20T01:27:26.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.280+0000 7f010cb1f700 1 -- 172.21.15.16:0/2021695989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00f4005960 msgr2=0x7f00f4005dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:26.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.280+0000 7f010cb1f700 1 --2- 172.21.15.16:0/2021695989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00f4005960 0x7f00f4005dd0 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f00f800c210 tx=0x7f00f4006990 comp rx=0 tx=0).stop 2023-05-20T01:27:26.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.280+0000 7f010cb1f700 1 -- 172.21.15.16:0/2021695989 shutdown_connections 2023-05-20T01:27:26.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.280+0000 7f010cb1f700 1 --2- 172.21.15.16:0/2021695989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00f4005960 0x7f00f4005dd0 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:26.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.280+0000 7f010cb1f700 1 -- 172.21.15.16:0/2021695989 >> 172.21.15.16:0/2021695989 conn(0x7f01080661b0 msgr2=0x7f0108066580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:27:26.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.280+0000 7f010cb1f700 1 -- 172.21.15.16:0/2021695989 shutdown_connections 2023-05-20T01:27:26.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.280+0000 7f010cb1f700 1 -- 172.21.15.16:0/2021695989 wait complete. 2023-05-20T01:27:26.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.281+0000 7f010cb1f700 1 Processor -- start 2023-05-20T01:27:26.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.282+0000 7f010cb1f700 1 -- start start 2023-05-20T01:27:26.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.282+0000 7f010cb1f700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f00f4005960 0x7f010815b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:26.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.282+0000 7f010cb1f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f010814faf0 0x7f0108157be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:26.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.282+0000 7f010cb1f700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0108143e60 0x7f01081442d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:26.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.282+0000 7f010cb1f700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0108144810 con 0x7f010814faf0 2023-05-20T01:27:26.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.282+0000 7f010cb1f700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0108150ca0 con 0x7f00f4005960 2023-05-20T01:27:26.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.282+0000 7f010cb1f700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f010815bb20 con 0x7f0108143e60 2023-05-20T01:27:26.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.282+0000 7f0105d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f010814faf0 0x7f0108157be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:26.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.282+0000 7f0105d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f010814faf0 0x7f0108157be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39226/0 (socket says 172.21.15.16:39226) 2023-05-20T01:27:26.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.282+0000 7f0105d9b700 1 -- 172.21.15.16:0/3372143848 learned_addr learned my addr 172.21.15.16:0/3372143848 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:27:26.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.283+0000 7f010659c700 1 --2- 172.21.15.16:0/3372143848 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f00f4005960 0x7f010815b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:26.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.283+0000 7f0106d9d700 1 --2- 172.21.15.16:0/3372143848 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0108143e60 0x7f01081442d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:26.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.283+0000 7f0105d9b700 1 -- 172.21.15.16:0/3372143848 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0108143e60 msgr2=0x7f01081442d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:26.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.283+0000 7f0105d9b700 1 --2- 172.21.15.16:0/3372143848 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0108143e60 0x7f01081442d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:26.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.283+0000 7f0105d9b700 1 -- 172.21.15.16:0/3372143848 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f00f4005960 msgr2=0x7f010815b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:26.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.283+0000 7f0105d9b700 1 --2- 172.21.15.16:0/3372143848 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f00f4005960 0x7f010815b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:26.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.283+0000 7f0105d9b700 1 -- 172.21.15.16:0/3372143848 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f010815d880 con 0x7f010814faf0 2023-05-20T01:27:26.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.283+0000 7f010659c700 1 --2- 172.21.15.16:0/3372143848 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f00f4005960 0x7f010815b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:27:26.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.283+0000 7f0105d9b700 1 --2- 172.21.15.16:0/3372143848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f010814faf0 0x7f0108157be0 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f00f800c800 tx=0x7f00f8001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:26.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.283+0000 7f00feffd700 1 -- 172.21.15.16:0/3372143848 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f8001630 con 0x7f010814faf0 2023-05-20T01:27:26.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.283+0000 7f00feffd700 1 -- 172.21.15.16:0/3372143848 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00f80070d0 con 0x7f010814faf0 2023-05-20T01:27:26.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.283+0000 7f00feffd700 1 -- 172.21.15.16:0/3372143848 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f8007440 con 0x7f010814faf0 2023-05-20T01:27:26.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.283+0000 7f010cb1f700 1 -- 172.21.15.16:0/3372143848 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f010815d360 con 0x7f010814faf0 2023-05-20T01:27:26.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.283+0000 7f010cb1f700 1 -- 172.21.15.16:0/3372143848 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01080b1b00 con 0x7f010814faf0 2023-05-20T01:27:26.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.284+0000 7f010cb1f700 1 -- 172.21.15.16:0/3372143848 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01081531c0 con 0x7f010814faf0 2023-05-20T01:27:26.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.285+0000 7f00feffd700 1 -- 172.21.15.16:0/3372143848 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f00f80075a0 con 0x7f010814faf0 2023-05-20T01:27:26.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.285+0000 7f00feffd700 1 --2- 172.21.15.16:0/3372143848 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f00e806ace0 0x7f00e806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:26.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.285+0000 7f00feffd700 1 -- 172.21.15.16:0/3372143848 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 7073+0+0 (secure 0 0 0) 0x7f00f808a850 con 0x7f010814faf0 2023-05-20T01:27:26.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.285+0000 7f010659c700 1 --2- 172.21.15.16:0/3372143848 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f00e806ace0 0x7f00e806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:26.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.286+0000 7f010659c700 1 --2- 172.21.15.16:0/3372143848 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f00e806ace0 0x7f00e806d190 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f00f0007fd0 tx=0x7f00f0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:26.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.289+0000 7f00feffd700 1 -- 172.21.15.16:0/3372143848 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f00f8055590 con 0x7f010814faf0 2023-05-20T01:27:26.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.531+0000 7f010cb1f700 1 -- 172.21.15.16:0/3372143848 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f01080747b0 con 0x7f010814faf0 2023-05-20T01:27:26.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.531+0000 7f00feffd700 1 -- 172.21.15.16:0/3372143848 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+4681 (secure 0 0 0) 0x7f00f8058bb0 con 0x7f010814faf0 2023-05-20T01:27:26.536 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:27:26.537 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":91,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6845,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/1435235487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1435235487},{"type":"v1","addr":"172.21.15.132:6834","nonce":1435235487}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":87},{"gid":6848,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/933893285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":933893285},{"type":"v1","addr":"172.21.15.132:6835","nonce":933893285}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":87},{"gid":8373,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/1220815342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1220815342},{"type":"v1","addr":"172.21.15.16:6836","nonce":1220815342}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":87}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-20T01:27:10.291795+0000","modified":"2023-05-20T01:27:25.446248+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":121,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8376},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8376":{"gid":8376,"name":"c","rank":0,"incarnation":88,"state":"up:active","state_seq":4,"addr":"172.21.15.16:6837/1927078882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":1927078882},{"type":"v1","addr":"172.21.15.16:6837","nonce":1927078882}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16,17],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-05-20T01:27:26.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.533+0000 7f010cb1f700 1 -- 172.21.15.16:0/3372143848 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f00e806ace0 msgr2=0x7f00e806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:26.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.533+0000 7f010cb1f700 1 --2- 172.21.15.16:0/3372143848 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f00e806ace0 0x7f00e806d190 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f00f0007fd0 tx=0x7f00f0009040 comp rx=0 tx=0).stop 2023-05-20T01:27:26.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.533+0000 7f010cb1f700 1 -- 172.21.15.16:0/3372143848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f010814faf0 msgr2=0x7f0108157be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:26.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.533+0000 7f010cb1f700 1 --2- 172.21.15.16:0/3372143848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f010814faf0 0x7f0108157be0 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f00f800c800 tx=0x7f00f8001a00 comp rx=0 tx=0).stop 2023-05-20T01:27:26.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.533+0000 7f010cb1f700 1 -- 172.21.15.16:0/3372143848 shutdown_connections 2023-05-20T01:27:26.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.533+0000 7f010cb1f700 1 --2- 172.21.15.16:0/3372143848 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f00e806ace0 0x7f00e806d190 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:26.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.533+0000 7f010cb1f700 1 --2- 172.21.15.16:0/3372143848 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0108143e60 0x7f01081442d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:26.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.533+0000 7f010cb1f700 1 --2- 172.21.15.16:0/3372143848 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f00f4005960 0x7f010815b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:26.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.533+0000 7f010cb1f700 1 --2- 172.21.15.16:0/3372143848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f010814faf0 0x7f0108157be0 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:26.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.533+0000 7f010cb1f700 1 -- 172.21.15.16:0/3372143848 >> 172.21.15.16:0/3372143848 conn(0x7f01080661b0 msgr2=0x7f0108073bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:27:26.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.534+0000 7f010cb1f700 1 -- 172.21.15.16:0/3372143848 shutdown_connections 2023-05-20T01:27:26.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.534+0000 7f010cb1f700 1 -- 172.21.15.16:0/3372143848 wait complete. 2023-05-20T01:27:26.541 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 91 2023-05-20T01:27:26.551 INFO:tasks.cephfs.test_data_scan:{ "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 5, "epoch": 91, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 5, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-05-20T01:27:10.291795+0000", "damaged": [], "data_pools": [ 16, 17 ], "enabled": true, "epoch": 91, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_8376": { "addr": "172.21.15.16:6837/1927078882", "addrs": { "addrvec": [ { "addr": "172.21.15.16:6835", "nonce": 1927078882, "type": "v2" }, { "addr": "172.21.15.16:6837", "nonce": 1927078882, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 8376, "incarnation": 88, "join_fscid": -1, "name": "c", "rank": 0, "state": "up:active", "state_seq": 4 } }, "last_failure": 0, "last_failure_osd_epoch": 121, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 15, "modified": "2023-05-20T01:27:25.446248+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 60, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 8376 } } } ], "standbys": [ { "addr": "172.21.15.132:6834/1435235487", "addrs": { "addrvec": [ { "addr": "172.21.15.132:6832", "nonce": 1435235487, "type": "v2" }, { "addr": "172.21.15.132:6834", "nonce": 1435235487, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 87, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6845, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.132:6835/933893285", "addrs": { "addrvec": [ { "addr": "172.21.15.132:6833", "nonce": 933893285, "type": "v2" }, { "addr": "172.21.15.132:6835", "nonce": 933893285, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 87, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6848, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.16:6836/1220815342", "addrs": { "addrvec": [ { "addr": "172.21.15.16:6834", "nonce": 1220815342, "type": "v2" }, { "addr": "172.21.15.16:6836", "nonce": 1220815342, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 87, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 8373, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-05-20T01:27:26.552 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-05-20T01:27:26.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.673+0000 7f5861ac5700 1 Processor -- start 2023-05-20T01:27:26.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.673+0000 7f5861ac5700 1 -- start start 2023-05-20T01:27:26.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.674+0000 7f5861ac5700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f585c1599c0 0x7f585c0caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:26.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.674+0000 7f5861ac5700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f585c0caaa0 0x7f585c0cb4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:26.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.674+0000 7f5861ac5700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f585c159d90 con 0x7f585c06f3f0 2023-05-20T01:27:26.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.674+0000 7f5861ac5700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f585c0cdf20 con 0x7f585c0caaa0 2023-05-20T01:27:26.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.674+0000 7f5861ac5700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f585c0cdb80 con 0x7f585c1599c0 2023-05-20T01:27:26.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.674+0000 7f585affd700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f585c0caaa0 0x7f585c0cb4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:26.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.674+0000 7f585b7fe700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f585c1599c0 0x7f585c0caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:26.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.674+0000 7f585affd700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f585c0caaa0 0x7f585c0cb4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.16:36256/0 (socket says 172.21.15.16:36256) 2023-05-20T01:27:26.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.674+0000 7f585affd700 1 -- 172.21.15.16:0/3964260016 learned_addr learned my addr 172.21.15.16:0/3964260016 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:27:26.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.674+0000 7f585affd700 1 -- 172.21.15.16:0/3964260016 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f585c1599c0 msgr2=0x7f585c0caf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:26.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.674+0000 7f585affd700 1 --2- 172.21.15.16:0/3964260016 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f585c1599c0 0x7f585c0caf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:26.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.674+0000 7f585affd700 1 -- 172.21.15.16:0/3964260016 >> v1:172.21.15.132:6789/0 conn(0x7f585c06f3f0 legacy=0x7f585c0710d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:26.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.674+0000 7f585affd700 1 -- 172.21.15.16:0/3964260016 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f585c06eaf0 con 0x7f585c0caaa0 2023-05-20T01:27:26.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.674+0000 7f585b7fe700 1 --2- 172.21.15.16:0/3964260016 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f585c1599c0 0x7f585c0caf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-05-20T01:27:26.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.675+0000 7f585affd700 1 --2- 172.21.15.16:0/3964260016 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f585c0caaa0 0x7f585c0cb4a0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f584c0098f0 tx=0x7f584c009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9cb3aed53e6a367f server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:26.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.675+0000 7f585a7fc700 1 -- 172.21.15.16:0/3964260016 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f584c011830 con 0x7f585c0caaa0 2023-05-20T01:27:26.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.675+0000 7f585a7fc700 1 -- 172.21.15.16:0/3964260016 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f584c011990 con 0x7f585c0caaa0 2023-05-20T01:27:26.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.675+0000 7f585a7fc700 1 -- 172.21.15.16:0/3964260016 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f584c011d20 con 0x7f585c0caaa0 2023-05-20T01:27:26.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.675+0000 7f5861ac5700 1 -- 172.21.15.16:0/3964260016 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f585c0caaa0 msgr2=0x7f585c0cb4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:26.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.675+0000 7f5861ac5700 1 --2- 172.21.15.16:0/3964260016 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f585c0caaa0 0x7f585c0cb4a0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f584c0098f0 tx=0x7f584c009c00 comp rx=0 tx=0).stop 2023-05-20T01:27:26.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.675+0000 7f5861ac5700 1 -- 172.21.15.16:0/3964260016 shutdown_connections 2023-05-20T01:27:26.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.675+0000 7f5861ac5700 1 --2- 172.21.15.16:0/3964260016 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f585c1599c0 0x7f585c0caf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:26.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.675+0000 7f5861ac5700 1 --2- 172.21.15.16:0/3964260016 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f585c0caaa0 0x7f585c0cb4a0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:26.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.675+0000 7f5861ac5700 1 -- 172.21.15.16:0/3964260016 >> 172.21.15.16:0/3964260016 conn(0x7f585c0d0c30 msgr2=0x7f585c0b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:27:26.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.675+0000 7f5861ac5700 1 -- 172.21.15.16:0/3964260016 shutdown_connections 2023-05-20T01:27:26.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.675+0000 7f5861ac5700 1 -- 172.21.15.16:0/3964260016 wait complete. 2023-05-20T01:27:26.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.677+0000 7f5861ac5700 1 Processor -- start 2023-05-20T01:27:26.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.677+0000 7f5861ac5700 1 -- start start 2023-05-20T01:27:26.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.677+0000 7f5861ac5700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f585c06f3f0 0x7f585c0b5680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:26.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.677+0000 7f5861ac5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f585c0caaa0 0x7f585c1f16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:26.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.677+0000 7f5861ac5700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f585c1599c0 0x7f585c1f1c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:26.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.677+0000 7f5861ac5700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f585c0b3530 con 0x7f585c0caaa0 2023-05-20T01:27:26.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.677+0000 7f5861ac5700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f585c1ef2b0 con 0x7f585c1599c0 2023-05-20T01:27:26.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.677+0000 7f5861ac5700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f585c0c5990 con 0x7f585c06f3f0 2023-05-20T01:27:26.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.678+0000 7f585affd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f585c0caaa0 0x7f585c1f16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:26.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.678+0000 7f585affd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f585c0caaa0 0x7f585c1f16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39240/0 (socket says 172.21.15.16:39240) 2023-05-20T01:27:26.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.678+0000 7f585affd700 1 -- 172.21.15.16:0/1357523818 learned_addr learned my addr 172.21.15.16:0/1357523818 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:27:26.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.678+0000 7f585b7fe700 1 --2- 172.21.15.16:0/1357523818 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f585c06f3f0 0x7f585c0b5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:26.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.678+0000 7f585bfff700 1 --2- 172.21.15.16:0/1357523818 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f585c1599c0 0x7f585c1f1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:26.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.678+0000 7f585affd700 1 -- 172.21.15.16:0/1357523818 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f585c06f3f0 msgr2=0x7f585c0b5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:26.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.678+0000 7f585affd700 1 --2- 172.21.15.16:0/1357523818 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f585c06f3f0 0x7f585c0b5680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:26.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.678+0000 7f585affd700 1 -- 172.21.15.16:0/1357523818 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f585c1599c0 msgr2=0x7f585c1f1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:26.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.678+0000 7f585affd700 1 --2- 172.21.15.16:0/1357523818 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f585c1599c0 0x7f585c1f1c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:26.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.678+0000 7f585affd700 1 -- 172.21.15.16:0/1357523818 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58500094a0 con 0x7f585c0caaa0 2023-05-20T01:27:26.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.678+0000 7f585bfff700 1 --2- 172.21.15.16:0/1357523818 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f585c1599c0 0x7f585c1f1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:27:26.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.678+0000 7f585affd700 1 --2- 172.21.15.16:0/1357523818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f585c0caaa0 0x7f585c1f16e0 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f584c000c00 tx=0x7f584c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:26.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.678+0000 7f5843fff700 1 -- 172.21.15.16:0/1357523818 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f584c004480 con 0x7f585c0caaa0 2023-05-20T01:27:26.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.678+0000 7f5843fff700 1 -- 172.21.15.16:0/1357523818 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f584c01f070 con 0x7f585c0caaa0 2023-05-20T01:27:26.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.678+0000 7f5861ac5700 1 -- 172.21.15.16:0/1357523818 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f584c009550 con 0x7f585c0caaa0 2023-05-20T01:27:26.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.678+0000 7f5843fff700 1 -- 172.21.15.16:0/1357523818 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f584c0038c0 con 0x7f585c0caaa0 2023-05-20T01:27:26.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.678+0000 7f5861ac5700 1 -- 172.21.15.16:0/1357523818 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f585c06f0b0 con 0x7f585c0caaa0 2023-05-20T01:27:26.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.679+0000 7f5861ac5700 1 -- 172.21.15.16:0/1357523818 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f585c06edd0 con 0x7f585c0caaa0 2023-05-20T01:27:26.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.680+0000 7f5843fff700 1 -- 172.21.15.16:0/1357523818 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f584c021030 con 0x7f585c0caaa0 2023-05-20T01:27:26.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.680+0000 7f5843fff700 1 --2- 172.21.15.16:0/1357523818 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f584806ace0 0x7f584806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:26.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.680+0000 7f5843fff700 1 -- 172.21.15.16:0/1357523818 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 7073+0+0 (secure 0 0 0) 0x7f584c08ad30 con 0x7f585c0caaa0 2023-05-20T01:27:26.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.680+0000 7f585b7fe700 1 --2- 172.21.15.16:0/1357523818 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f584806ace0 0x7f584806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:26.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.681+0000 7f585b7fe700 1 --2- 172.21.15.16:0/1357523818 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f584806ace0 0x7f584806d190 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f585c0af720 tx=0x7f5850009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:26.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.685+0000 7f5843fff700 1 -- 172.21.15.16:0/1357523818 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f584c0559f0 con 0x7f585c0caaa0 2023-05-20T01:27:26.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.928+0000 7f5861ac5700 1 -- 172.21.15.16:0/1357523818 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f585c06e750 con 0x7f585c0caaa0 2023-05-20T01:27:26.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.928+0000 7f5843fff700 1 -- 172.21.15.16:0/1357523818 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v91) v1 ==== 53+0+91 (secure 0 0 0) 0x7f584c059010 con 0x7f585c0caaa0 2023-05-20T01:27:26.934 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_extra ] 2023-05-20T01:27:26.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.931+0000 7f5861ac5700 1 -- 172.21.15.16:0/1357523818 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f584806ace0 msgr2=0x7f584806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:26.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.931+0000 7f5861ac5700 1 --2- 172.21.15.16:0/1357523818 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f584806ace0 0x7f584806d190 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f585c0af720 tx=0x7f5850009210 comp rx=0 tx=0).stop 2023-05-20T01:27:26.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.931+0000 7f5861ac5700 1 -- 172.21.15.16:0/1357523818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f585c0caaa0 msgr2=0x7f585c1f16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:26.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.931+0000 7f5861ac5700 1 --2- 172.21.15.16:0/1357523818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f585c0caaa0 0x7f585c1f16e0 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f584c000c00 tx=0x7f584c004750 comp rx=0 tx=0).stop 2023-05-20T01:27:26.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.931+0000 7f5861ac5700 1 -- 172.21.15.16:0/1357523818 shutdown_connections 2023-05-20T01:27:26.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.931+0000 7f5861ac5700 1 --2- 172.21.15.16:0/1357523818 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f585c1599c0 0x7f585c1f1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:26.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.931+0000 7f5861ac5700 1 --2- 172.21.15.16:0/1357523818 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f584806ace0 0x7f584806d190 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:26.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.931+0000 7f5861ac5700 1 --2- 172.21.15.16:0/1357523818 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f585c06f3f0 0x7f585c0b5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:26.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.931+0000 7f5861ac5700 1 --2- 172.21.15.16:0/1357523818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f585c0caaa0 0x7f585c1f16e0 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:26.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.931+0000 7f5861ac5700 1 -- 172.21.15.16:0/1357523818 >> 172.21.15.16:0/1357523818 conn(0x7f585c0d0c30 msgr2=0x7f585c0b04a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:27:26.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.931+0000 7f5861ac5700 1 -- 172.21.15.16:0/1357523818 shutdown_connections 2023-05-20T01:27:26.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:26.931+0000 7f5861ac5700 1 -- 172.21.15.16:0/1357523818 wait complete. 2023-05-20T01:27:26.947 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-05-20T01:27:26.947 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-05-20T01:27:26.948 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2023-05-20T01:27:26.948 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-05-20T01:27:26.948 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-05-20T01:27:26.949 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-05-20T01:27:26.949 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-05-20T01:27:26.949 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-05-20T01:27:26.949 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-05-20T01:27:26.950 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:27:26.950 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2023-05-20T01:27:26.967 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-05-20T01:27:26.968 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-05-20T01:27:26.968 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2023-05-20T01:27:26.968 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-05-20T01:27:26.968 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2023-05-20T01:27:26.968 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-05-20T01:27:26.969 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-05-20T01:27:26.969 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2023-05-20T01:27:26.969 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-05-20T01:27:26.969 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2023-05-20T01:27:26.969 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-05-20T01:27:26.969 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2023-05-20T01:27:26.969 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-05-20T01:27:26.970 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 41787sec preferred_lft 41787sec 2023-05-20T01:27:26.970 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link 2023-05-20T01:27:26.970 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-05-20T01:27:26.970 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-05-20T01:27:26.970 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2023-05-20T01:27:26.970 INFO:teuthology.orchestra.run.smithi016.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-05-20T01:27:26.970 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0a:2e:0c:6b:5d:42 brd ff:ff:ff:ff:ff:ff 2023-05-20T01:27:26.971 INFO:teuthology.orchestra.run.smithi016.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-05-20T01:27:26.971 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-05-20T01:27:26.971 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::7c00:b2ff:fef4:946/64 scope link 2023-05-20T01:27:26.971 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-05-20T01:27:26.971 INFO:teuthology.orchestra.run.smithi016.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-05-20T01:27:26.971 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether d2:e4:70:a7:b9:4b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-05-20T01:27:26.972 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::d0e4:70ff:fea7:b94b/64 scope link 2023-05-20T01:27:26.972 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-05-20T01:27:26.972 INFO:teuthology.orchestra.run.smithi016.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-05-20T01:27:26.972 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0a:2e:0c:6b:5d:42 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-05-20T01:27:26.972 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::82e:cff:fe6b:5d42/64 scope link 2023-05-20T01:27:26.972 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-05-20T01:27:26.973 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:27:26.973 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-05-20T01:27:27.034 INFO:teuthology.orchestra.run.smithi016.stdout:1 2023-05-20T01:27:27.035 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:27:27.036 DEBUG:teuthology.orchestra.run.smithi016:> route 2023-05-20T01:27:27.094 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2023-05-20T01:27:27.095 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-05-20T01:27:27.095 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-05-20T01:27:27.095 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-05-20T01:27:27.095 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-05-20T01:27:27.096 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:27:27.096 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2023-05-20T01:27:27.096 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2023-05-20T01:27:27.097 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-05-20T01:27:27.097 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-05-20T01:27:27.098 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-05-20T01:27:27.098 DEBUG:teuthology.orchestra.run.smithi016:> ') 2023-05-20T01:27:27.202 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:27:27.202 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2023-05-20T01:27:27.220 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-05-20T01:27:27.220 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-05-20T01:27:27.220 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-05-20T01:27:27.221 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:27:27.221 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.0 up 2023-05-20T01:27:27.288 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-05-20T01:27:27.288 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-05-20T01:27:27.345 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-05-20T01:27:27.345 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-05-20T01:27:27.346 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-05-20T01:27:27.402 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-05-20T01:27:27.403 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:27:27.404 DEBUG:teuthology.orchestra.run.smithi016:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-05-20T01:27:27.446 INFO:tasks.ceph.mds.c.smithi016.stderr:2023-05-20T01:27:27.442+0000 7efe57b8b700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-05-20T01:27:27.489 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-05-20T01:27:27.489 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-05-20T01:27:27.518 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-05-20T01:27:27.519 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-05-20T01:27:27.519 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.132:3300,172.21.15.132:3301" 2023-05-20T01:27:27.519 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: trying mount with new device syntax: 0@c2245cc5-ecab-4087-a59b-bc81db901676.cephfs=/ 2023-05-20T01:27:27.519 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.16:3300/172.21.15.132:3300/172.21.15.132:3301" will pass to kernel 2023-05-20T01:27:27.520 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: v2 mount support check returned 2 2023-05-20T01:27:27.520 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: kernel does not support v2 syntax 2023-05-20T01:27:27.520 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: switching to using v1 address with old syntax 2023-05-20T01:27:27.593 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-05-20T01:27:27.594 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-05-20T01:27:27.594 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.132:3300,172.21.15.132:3301" 2023-05-20T01:27:27.594 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: trying mount with new device syntax: 0@c2245cc5-ecab-4087-a59b-bc81db901676.cephfs=/ 2023-05-20T01:27:27.595 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.16:3300/172.21.15.132:3300/172.21.15.132:3301" will pass to kernel 2023-05-20T01:27:27.595 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: v2 mount support check returned 2 2023-05-20T01:27:27.595 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: kernel does not support v2 syntax 2023-05-20T01:27:27.596 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: switching to using v1 address with old syntax 2023-05-20T01:27:27.596 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:6789,172.21.15.132:6789,172.21.15.132:6790" 2023-05-20T01:27:27.596 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.16:6789,172.21.15.132:6789,172.21.15.132:6790:/ 2023-05-20T01:27:27.596 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=c2245cc5-ecab-4087-a59b-bc81db901676" will pass to kernel 2023-05-20T01:27:27.598 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-05-20T01:27:27.598 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:27:27.599 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-05-20T01:27:27.666 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:27:27.667 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-05-20T01:27:27.748 INFO:teuthology.orchestra.run.smithi016.stdout:client6884 2023-05-20T01:27:27.748 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-05-20T01:27:27.749 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2023-05-20T01:27:27.806 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.5:0 scrub start / repair,recursive 2023-05-20T01:27:27.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.971+0000 7f47bd234700 1 Processor -- start 2023-05-20T01:27:27.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.971+0000 7f47bd234700 1 -- start start 2023-05-20T01:27:27.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.972+0000 7f47bd234700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f47b815c830 0x7f47b8161070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:27.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.972+0000 7f47bd234700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f47b80c6b50 0x7f47b806a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:27.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.972+0000 7f47bd234700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47b80cf750 con 0x7f47b815cc00 2023-05-20T01:27:27.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.972+0000 7f47bd234700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47b80d0730 con 0x7f47b80c6b50 2023-05-20T01:27:27.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.972+0000 7f47bd234700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47b81615b0 con 0x7f47b815c830 2023-05-20T01:27:27.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.972+0000 7f47b659c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f47b80c6b50 0x7f47b806a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:27.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.972+0000 7f47b759e700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f47b815cc00 0x7f47b80712a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50778/0 (socket says 172.21.15.16:50778) 2023-05-20T01:27:27.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.972+0000 7f47b659c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f47b80c6b50 0x7f47b806a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39250/0 (socket says 172.21.15.16:39250) 2023-05-20T01:27:27.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.972+0000 7f47b759e700 1 -- 172.21.15.16:0/3620583231 learned_addr learned my addr 172.21.15.16:0/3620583231 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:27:27.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.972+0000 7f47b6d9d700 1 --2- 172.21.15.16:0/3620583231 >> v2:172.21.15.132:3300/0 conn(0x7f47b815c830 0x7f47b8161070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:27.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.973+0000 7f47b659c700 1 -- 172.21.15.16:0/3620583231 >> v2:172.21.15.132:3300/0 conn(0x7f47b815c830 msgr2=0x7f47b8161070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:27.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.973+0000 7f47b659c700 1 --2- 172.21.15.16:0/3620583231 >> v2:172.21.15.132:3300/0 conn(0x7f47b815c830 0x7f47b8161070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:27.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.973+0000 7f47b659c700 1 -- 172.21.15.16:0/3620583231 >> v1:172.21.15.16:6789/0 conn(0x7f47b815cc00 legacy=0x7f47b80712a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:27.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.973+0000 7f47b659c700 1 -- 172.21.15.16:0/3620583231 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47b80cf980 con 0x7f47b80c6b50 2023-05-20T01:27:27.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.973+0000 7f47b5d9b700 1 -- 172.21.15.16:0/3620583231 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3103020574 0 0) 0x7f47b80cf750 con 0x7f47b815cc00 2023-05-20T01:27:27.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.973+0000 7f47b659c700 1 --2- 172.21.15.16:0/3620583231 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47b80c6b50 0x7f47b806a510 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f47b8072650 tx=0x7f47a8009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=431a45c0574ab4ed server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:27.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.973+0000 7f47b5d9b700 1 -- 172.21.15.16:0/3620583231 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a8011830 con 0x7f47b80c6b50 2023-05-20T01:27:27.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.973+0000 7f47b5d9b700 1 -- 172.21.15.16:0/3620583231 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47a8011990 con 0x7f47b80c6b50 2023-05-20T01:27:27.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.973+0000 7f47b5d9b700 1 -- 172.21.15.16:0/3620583231 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a8011d20 con 0x7f47b80c6b50 2023-05-20T01:27:27.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.973+0000 7f47bd234700 1 -- 172.21.15.16:0/3620583231 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47b80c6b50 msgr2=0x7f47b806a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:27.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.973+0000 7f47bd234700 1 --2- 172.21.15.16:0/3620583231 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47b80c6b50 0x7f47b806a510 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f47b8072650 tx=0x7f47a8009b20 comp rx=0 tx=0).stop 2023-05-20T01:27:27.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.973+0000 7f47bd234700 1 -- 172.21.15.16:0/3620583231 shutdown_connections 2023-05-20T01:27:27.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.973+0000 7f47bd234700 1 --2- 172.21.15.16:0/3620583231 >> v2:172.21.15.132:3300/0 conn(0x7f47b815c830 0x7f47b8161070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:27.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.973+0000 7f47bd234700 1 --2- 172.21.15.16:0/3620583231 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47b80c6b50 0x7f47b806a510 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:27.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.973+0000 7f47bd234700 1 -- 172.21.15.16:0/3620583231 >> 172.21.15.16:0/3620583231 conn(0x7f47b80b2920 msgr2=0x7f47b80d0020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:27:27.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.974+0000 7f47bd234700 1 -- 172.21.15.16:0/3620583231 shutdown_connections 2023-05-20T01:27:27.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.974+0000 7f47bd234700 1 -- 172.21.15.16:0/3620583231 wait complete. 2023-05-20T01:27:27.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.975+0000 7f47bd234700 1 Processor -- start 2023-05-20T01:27:27.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.975+0000 7f47bd234700 1 -- start start 2023-05-20T01:27:27.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.976+0000 7f47bd234700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f47b80c6b50 0x7f47b815bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:27.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.976+0000 7f47bd234700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f47b815c830 0x7f47b80accc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:27.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.976+0000 7f47bd234700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47b815cc00 0x7f47b8150bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:27.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.976+0000 7f47bd234700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47b8159e20 con 0x7f47b815cc00 2023-05-20T01:27:27.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.976+0000 7f47bd234700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47b80b2f90 con 0x7f47b80c6b50 2023-05-20T01:27:27.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.976+0000 7f47bd234700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47b8151100 con 0x7f47b815c830 2023-05-20T01:27:27.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.976+0000 7f47b759e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47b815cc00 0x7f47b8150bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:27.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.976+0000 7f47b759e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47b815cc00 0x7f47b8150bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39258/0 (socket says 172.21.15.16:39258) 2023-05-20T01:27:27.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.976+0000 7f47b759e700 1 -- 172.21.15.16:0/1608136344 learned_addr learned my addr 172.21.15.16:0/1608136344 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:27:27.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.976+0000 7f47b659c700 1 --2- 172.21.15.16:0/1608136344 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f47b815c830 0x7f47b80accc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:27.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.976+0000 7f47b6d9d700 1 --2- 172.21.15.16:0/1608136344 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f47b80c6b50 0x7f47b815bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:27.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.976+0000 7f47b759e700 1 -- 172.21.15.16:0/1608136344 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f47b815c830 msgr2=0x7f47b80accc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:27.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.976+0000 7f47b759e700 1 --2- 172.21.15.16:0/1608136344 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f47b815c830 0x7f47b80accc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:27.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.976+0000 7f47b759e700 1 -- 172.21.15.16:0/1608136344 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f47b80c6b50 msgr2=0x7f47b815bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:27.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.976+0000 7f47b759e700 1 --2- 172.21.15.16:0/1608136344 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f47b80c6b50 0x7f47b815bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:27.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.976+0000 7f47b759e700 1 -- 172.21.15.16:0/1608136344 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47a80094a0 con 0x7f47b815cc00 2023-05-20T01:27:27.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.976+0000 7f47b6d9d700 1 --2- 172.21.15.16:0/1608136344 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f47b80c6b50 0x7f47b815bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:27:27.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.976+0000 7f47b659c700 1 --2- 172.21.15.16:0/1608136344 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f47b815c830 0x7f47b80accc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:27:27.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.976+0000 7f47b759e700 1 --2- 172.21.15.16:0/1608136344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47b815cc00 0x7f47b8150bc0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f47b0002230 tx=0x7f47b000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:27.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.976+0000 7f479f7fe700 1 -- 172.21.15.16:0/1608136344 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b000ac20 con 0x7f47b815cc00 2023-05-20T01:27:27.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.977+0000 7f479f7fe700 1 -- 172.21.15.16:0/1608136344 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47b000ada0 con 0x7f47b815cc00 2023-05-20T01:27:27.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.977+0000 7f479f7fe700 1 -- 172.21.15.16:0/1608136344 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b0007310 con 0x7f47b815cc00 2023-05-20T01:27:27.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.977+0000 7f47bd234700 1 -- 172.21.15.16:0/1608136344 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47b80d0980 con 0x7f47b815cc00 2023-05-20T01:27:27.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.977+0000 7f47bd234700 1 -- 172.21.15.16:0/1608136344 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47b80cfc60 con 0x7f47b815cc00 2023-05-20T01:27:27.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.978+0000 7f479f7fe700 1 -- 172.21.15.16:0/1608136344 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f47b0007470 con 0x7f47b815cc00 2023-05-20T01:27:27.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.978+0000 7f479f7fe700 1 --2- 172.21.15.16:0/1608136344 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f47a406ace0 0x7f47a406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:27.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.979+0000 7f479f7fe700 1 -- 172.21.15.16:0/1608136344 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 7073+0+0 (secure 0 0 0) 0x7f47b00893f0 con 0x7f47b815cc00 2023-05-20T01:27:27.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.979+0000 7f47b6d9d700 1 --2- 172.21.15.16:0/1608136344 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f47a406ace0 0x7f47a406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:27.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.979+0000 7f47b6d9d700 1 --2- 172.21.15.16:0/1608136344 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f47a406ace0 0x7f47a406d190 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f47ac009b60 tx=0x7f47ac008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:27.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.980+0000 7f47c93c0b80 1 Processor -- start 2023-05-20T01:27:27.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.981+0000 7f47c93c0b80 1 -- start start 2023-05-20T01:27:27.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.981+0000 7f47c93c0b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559feb3984d0 0x559feb3988a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:27.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.981+0000 7f47c93c0b80 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559feb3947d0 0x559feb36d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:27.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.981+0000 7f47c93c0b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559feb36b940 0x559feb36da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:27.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.981+0000 7f47c93c0b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x559feb2730e0 con 0x559feb3984d0 2023-05-20T01:27:27.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.981+0000 7f47c93c0b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x559feb2c5920 con 0x559feb36b940 2023-05-20T01:27:27.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.981+0000 7f47c93c0b80 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x559feb2bedb0 con 0x559feb3947d0 2023-05-20T01:27:27.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.982+0000 7f47b6d9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559feb3984d0 0x559feb3988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:27.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.982+0000 7f47b6d9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559feb3984d0 0x559feb3988a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39262/0 (socket says 172.21.15.16:39262) 2023-05-20T01:27:27.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.982+0000 7f47b6d9d700 1 -- 172.21.15.16:0/742436565 learned_addr learned my addr 172.21.15.16:0/742436565 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:27:27.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.982+0000 7f47b659c700 1 --2- 172.21.15.16:0/742436565 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559feb3947d0 0x559feb36d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:27.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.982+0000 7f47b759e700 1 --2- 172.21.15.16:0/742436565 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559feb36b940 0x559feb36da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:27.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.982+0000 7f47b6d9d700 1 -- 172.21.15.16:0/742436565 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559feb3947d0 msgr2=0x559feb36d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:27.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.982+0000 7f47b6d9d700 1 --2- 172.21.15.16:0/742436565 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559feb3947d0 0x559feb36d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:27.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.982+0000 7f47b6d9d700 1 -- 172.21.15.16:0/742436565 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559feb36b940 msgr2=0x559feb36da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:27.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.982+0000 7f47b6d9d700 1 --2- 172.21.15.16:0/742436565 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559feb36b940 0x559feb36da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:27.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.982+0000 7f47b6d9d700 1 -- 172.21.15.16:0/742436565 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559feb1406b0 con 0x559feb3984d0 2023-05-20T01:27:27.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.982+0000 7f47b759e700 1 --2- 172.21.15.16:0/742436565 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559feb36b940 0x559feb36da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:27:27.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.982+0000 7f47b659c700 1 --2- 172.21.15.16:0/742436565 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559feb3947d0 0x559feb36d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:27:27.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.982+0000 7f47b6d9d700 1 --2- 172.21.15.16:0/742436565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559feb3984d0 0x559feb3988a0 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f47ac010a80 tx=0x7f47ac013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b90040109adf8cd server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:27.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.982+0000 7f479cff9700 1 -- 172.21.15.16:0/742436565 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47ac015b10 con 0x559feb3984d0 2023-05-20T01:27:27.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.982+0000 7f479cff9700 1 -- 172.21.15.16:0/742436565 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47ac01a070 con 0x559feb3984d0 2023-05-20T01:27:27.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.982+0000 7f479cff9700 1 -- 172.21.15.16:0/742436565 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47ac015e70 con 0x559feb3984d0 2023-05-20T01:27:27.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.983+0000 7f47c93c0b80 1 -- 172.21.15.16:0/742436565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559feb3984d0 msgr2=0x559feb3988a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:27.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.983+0000 7f47c93c0b80 1 --2- 172.21.15.16:0/742436565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559feb3984d0 0x559feb3988a0 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f47ac010a80 tx=0x7f47ac013000 comp rx=0 tx=0).stop 2023-05-20T01:27:27.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.983+0000 7f47c93c0b80 1 -- 172.21.15.16:0/742436565 shutdown_connections 2023-05-20T01:27:28.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.983+0000 7f47c93c0b80 1 --2- 172.21.15.16:0/742436565 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559feb36b940 0x559feb36da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:28.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.983+0000 7f47c93c0b80 1 --2- 172.21.15.16:0/742436565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559feb3984d0 0x559feb3988a0 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:28.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.983+0000 7f47c93c0b80 1 --2- 172.21.15.16:0/742436565 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559feb3947d0 0x559feb36d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:28.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.983+0000 7f47c93c0b80 1 -- 172.21.15.16:0/742436565 >> 172.21.15.16:0/742436565 conn(0x559feb363000 msgr2=0x559feb3b32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:27:28.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.983+0000 7f47c93c0b80 1 -- 172.21.15.16:0/742436565 shutdown_connections 2023-05-20T01:27:28.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.983+0000 7f47c93c0b80 1 -- 172.21.15.16:0/742436565 wait complete. 2023-05-20T01:27:28.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.984+0000 7f47c93c0b80 1 -- start start 2023-05-20T01:27:28.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.984+0000 7f47c93c0b80 1 Processor -- start 2023-05-20T01:27:28.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.985+0000 7f47c93c0b80 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559feb36b940 0x559feb38fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:28.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.985+0000 7f47c93c0b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559feb3947d0 0x559feb36bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:28.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.985+0000 7f47c93c0b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559feb3984d0 0x559feb36c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:28.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.985+0000 7f47c93c0b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x559feb260960 con 0x559feb3947d0 2023-05-20T01:27:28.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.985+0000 7f47c93c0b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x559feb260600 con 0x559feb3984d0 2023-05-20T01:27:28.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.985+0000 7f47c93c0b80 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x559feb1429a0 con 0x559feb36b940 2023-05-20T01:27:28.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.985+0000 7f47c93c0b80 10 client.0 _ll_register_callbacks cb 0x559feb27e800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-05-20T01:27:28.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.985+0000 7f47b659c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559feb3947d0 0x559feb36bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:28.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.985+0000 7f47b659c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559feb3947d0 0x559feb36bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39270/0 (socket says 172.21.15.16:39270) 2023-05-20T01:27:28.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.985+0000 7f47b659c700 1 -- 172.21.15.16:0/751931758 learned_addr learned my addr 172.21.15.16:0/751931758 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:27:28.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.985+0000 7f47b759e700 1 --2- 172.21.15.16:0/751931758 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559feb3984d0 0x559feb36c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:28.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.985+0000 7f47b6d9d700 1 --2- 172.21.15.16:0/751931758 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559feb36b940 0x559feb38fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:28.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.986+0000 7f47b659c700 1 -- 172.21.15.16:0/751931758 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559feb36b940 msgr2=0x559feb38fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:28.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.986+0000 7f47b659c700 1 --2- 172.21.15.16:0/751931758 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559feb36b940 0x559feb38fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:28.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.986+0000 7f47b659c700 1 -- 172.21.15.16:0/751931758 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559feb3984d0 msgr2=0x559feb36c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:28.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.986+0000 7f47b659c700 1 --2- 172.21.15.16:0/751931758 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559feb3984d0 0x559feb36c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:28.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.986+0000 7f47b659c700 1 -- 172.21.15.16:0/751931758 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47ac00e050 con 0x559feb3947d0 2023-05-20T01:27:28.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.986+0000 7f47b759e700 1 --2- 172.21.15.16:0/751931758 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559feb3984d0 0x559feb36c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:27:28.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.986+0000 7f47b659c700 1 --2- 172.21.15.16:0/751931758 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559feb3947d0 0x559feb36bdb0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f47a80090b0 tx=0x7f47a800e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b52b0ab9e4f10eca server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:28.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.986+0000 7f4795ffb700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-05-20T01:27:28.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.986+0000 7f4795ffb700 1 -- 172.21.15.16:0/751931758 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a8004450 con 0x559feb3947d0 2023-05-20T01:27:28.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.986+0000 7f4795ffb700 1 -- 172.21.15.16:0/751931758 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(123..123 src has 1..123) v4 ==== 7073+0+0 (secure 0 0 0) 0x7f47a801f070 con 0x559feb3947d0 2023-05-20T01:27:28.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.986+0000 7f47c93c0b80 1 -- 172.21.15.16:0/751931758 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559feb3607c0 con 0x559feb3947d0 2023-05-20T01:27:28.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.986+0000 7f4795ffb700 1 -- 172.21.15.16:0/751931758 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f47a80045b0 con 0x559feb3947d0 2023-05-20T01:27:28.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.986+0000 7f47c93c0b80 10 client.8436 fetch_fsmap learned FSMap version 91 2023-05-20T01:27:28.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.986+0000 7f47c93c0b80 1 -- 172.21.15.16:0/751931758 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) v3 -- 0x559feb285210 con 0x559feb3947d0 2023-05-20T01:27:28.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.986+0000 7f4795ffb700 1 -- 172.21.15.16:0/751931758 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559feb3607c0 con 0x559feb3947d0 2023-05-20T01:27:28.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.986+0000 7f4795ffb700 1 -- 172.21.15.16:0/751931758 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 91) v1 ==== 3122+0+0 (secure 0 0 0) 0x7f47a80036a0 con 0x559feb3947d0 2023-05-20T01:27:28.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.986+0000 7f47c93c0b80 10 client.8436 fetch_fsmap finished waiting for FSMap version 91 2023-05-20T01:27:28.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.986+0000 7f47c93c0b80 10 client.8436 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2023-05-20T01:27:28.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.986+0000 7f47c93c0b80 20 client.8436 populate_metadata read hostname 'smithi016' 2023-05-20T01:27:28.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.986+0000 7f47c93c0b80 1 --2- 172.21.15.16:0/751931758 >> [v2:172.21.15.16:6835/1927078882,v1:172.21.15.16:6837/1927078882] conn(0x559feb38ca80 0x559feb38ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:28.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.987+0000 7f47c93c0b80 4 client.8436 mds_command: new command op to 8376 tid=0[{"prefix": "get_command_descriptions"}] 2023-05-20T01:27:28.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.987+0000 7f47c93c0b80 1 -- 172.21.15.16:0/751931758 --> [v2:172.21.15.16:6835/1927078882,v1:172.21.15.16:6837/1927078882] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559feb281120 con 0x559feb38ca80 2023-05-20T01:27:28.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.987+0000 7f47b6d9d700 1 --2- 172.21.15.16:0/751931758 >> [v2:172.21.15.16:6835/1927078882,v1:172.21.15.16:6837/1927078882] conn(0x559feb38ca80 0x559feb38ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:28.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.987+0000 7f47b6d9d700 1 --2- 172.21.15.16:0/751931758 >> [v2:172.21.15.16:6835/1927078882,v1:172.21.15.16:6837/1927078882] conn(0x559feb38ca80 0x559feb38ce80 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42719b16d734a2c1 server_cookie=a48a94a37a2157bb in_seq=0 out_seq=0 2023-05-20T01:27:28.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.987+0000 7f4795ffb700 10 client.8436 ms_handle_connect on v2:172.21.15.16:6835/1927078882 2023-05-20T01:27:28.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.987+0000 7f4795ffb700 1 -- 172.21.15.16:0/751931758 <== mds.0 v2:172.21.15.16:6835/1927078882 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x559feb281120 con 0x559feb38ca80 2023-05-20T01:27:28.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.987+0000 7f4795ffb700 10 client.8436 handle_command_reply: tid=0 2023-05-20T01:27:28.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.987+0000 7f47c93c0b80 1 client.8436 shutdown 2023-05-20T01:27:28.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.988+0000 7f47c93c0b80 1 -- 172.21.15.16:0/751931758 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559feb3947d0 msgr2=0x559feb36bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:28.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.988+0000 7f47c93c0b80 1 --2- 172.21.15.16:0/751931758 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559feb3947d0 0x559feb36bdb0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f47a80090b0 tx=0x7f47a800e000 comp rx=0 tx=0).stop 2023-05-20T01:27:28.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.988+0000 7f47c93c0b80 1 -- 172.21.15.16:0/751931758 shutdown_connections 2023-05-20T01:27:28.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.988+0000 7f47c93c0b80 1 --2- 172.21.15.16:0/751931758 >> [v2:172.21.15.16:6835/1927078882,v1:172.21.15.16:6837/1927078882] conn(0x559feb38ca80 0x559feb38ce80 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:28.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.988+0000 7f47c93c0b80 1 --2- 172.21.15.16:0/751931758 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559feb3984d0 0x559feb36c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:28.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.988+0000 7f47c93c0b80 1 --2- 172.21.15.16:0/751931758 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559feb36b940 0x559feb38fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:28.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.988+0000 7f4795ffb700 0 client.8436 ms_handle_reset on v2:172.21.15.16:6835/1927078882 2023-05-20T01:27:28.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.988+0000 7f47c93c0b80 1 --2- 172.21.15.16:0/751931758 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559feb3947d0 0x559feb36bdb0 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:28.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.988+0000 7f47c93c0b80 1 -- 172.21.15.16:0/751931758 >> 172.21.15.16:0/751931758 conn(0x559feb363000 msgr2=0x559feb3684b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:27:28.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.988+0000 7f47c93c0b80 1 -- 172.21.15.16:0/751931758 shutdown_connections 2023-05-20T01:27:28.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.988+0000 7f47c93c0b80 1 -- 172.21.15.16:0/751931758 wait complete. 2023-05-20T01:27:28.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:27.988+0000 7f47c93c0b80 20 client.8436 trim_cache size 0 max 16384 2023-05-20T01:27:28.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.005+0000 7f47c93c0b80 1 Processor -- start 2023-05-20T01:27:28.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.005+0000 7f47c93c0b80 1 -- start start 2023-05-20T01:27:28.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.005+0000 7f47c93c0b80 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559feb36b940 0x559feb369700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:28.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.005+0000 7f47c93c0b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559feb3947d0 0x559feb3b34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:28.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.006+0000 7f47c93c0b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559feb3984d0 0x559feb3b25a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:28.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.006+0000 7f47c93c0b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x559feb38f850 con 0x559feb3984d0 2023-05-20T01:27:28.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.006+0000 7f47c93c0b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x559feb3603a0 con 0x559feb3947d0 2023-05-20T01:27:28.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.006+0000 7f47c93c0b80 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x559feb36f8c0 con 0x559feb36b940 2023-05-20T01:27:28.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.006+0000 7f47b759e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559feb3984d0 0x559feb3b25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:28.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.006+0000 7f47b759e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559feb3984d0 0x559feb3b25a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39280/0 (socket says 172.21.15.16:39280) 2023-05-20T01:27:28.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.006+0000 7f47b759e700 1 -- 172.21.15.16:0/4258525570 learned_addr learned my addr 172.21.15.16:0/4258525570 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:27:28.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.006+0000 7f47b6d9d700 1 --2- 172.21.15.16:0/4258525570 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559feb36b940 0x559feb369700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:28.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.006+0000 7f47b659c700 1 --2- 172.21.15.16:0/4258525570 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559feb3947d0 0x559feb3b34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:28.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.006+0000 7f47b759e700 1 -- 172.21.15.16:0/4258525570 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559feb36b940 msgr2=0x559feb369700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:28.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.006+0000 7f47b759e700 1 --2- 172.21.15.16:0/4258525570 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559feb36b940 0x559feb369700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:28.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.006+0000 7f47b759e700 1 -- 172.21.15.16:0/4258525570 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559feb3947d0 msgr2=0x559feb3b34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:28.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.006+0000 7f47b759e700 1 --2- 172.21.15.16:0/4258525570 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559feb3947d0 0x559feb3b34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:28.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.006+0000 7f47b759e700 1 -- 172.21.15.16:0/4258525570 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559feb284f60 con 0x559feb3984d0 2023-05-20T01:27:28.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.006+0000 7f47b759e700 1 --2- 172.21.15.16:0/4258525570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559feb3984d0 0x559feb3b25a0 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f47b000a460 tx=0x7f47b005be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be708a9d23797a90 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:28.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.006+0000 7f479d7fa700 1 -- 172.21.15.16:0/4258525570 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b0056190 con 0x559feb3984d0 2023-05-20T01:27:28.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.006+0000 7f479d7fa700 1 -- 172.21.15.16:0/4258525570 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47b00572e0 con 0x559feb3984d0 2023-05-20T01:27:28.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.006+0000 7f479d7fa700 1 -- 172.21.15.16:0/4258525570 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b005fcb0 con 0x559feb3984d0 2023-05-20T01:27:28.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.007+0000 7f47c93c0b80 1 -- 172.21.15.16:0/4258525570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559feb3984d0 msgr2=0x559feb3b25a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:28.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.007+0000 7f47c93c0b80 1 --2- 172.21.15.16:0/4258525570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559feb3984d0 0x559feb3b25a0 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f47b000a460 tx=0x7f47b005be30 comp rx=0 tx=0).stop 2023-05-20T01:27:28.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.007+0000 7f47c93c0b80 1 -- 172.21.15.16:0/4258525570 shutdown_connections 2023-05-20T01:27:28.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.007+0000 7f47c93c0b80 1 --2- 172.21.15.16:0/4258525570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559feb3984d0 0x559feb3b25a0 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:28.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.007+0000 7f47c93c0b80 1 --2- 172.21.15.16:0/4258525570 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559feb36b940 0x559feb369700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:28.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.007+0000 7f47c93c0b80 1 --2- 172.21.15.16:0/4258525570 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559feb3947d0 0x559feb3b34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:28.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.007+0000 7f47c93c0b80 1 -- 172.21.15.16:0/4258525570 >> 172.21.15.16:0/4258525570 conn(0x559feb363000 msgr2=0x559feb3d4fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:27:28.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.007+0000 7f47c93c0b80 1 -- 172.21.15.16:0/4258525570 shutdown_connections 2023-05-20T01:27:28.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.007+0000 7f47c93c0b80 1 -- 172.21.15.16:0/4258525570 wait complete. 2023-05-20T01:27:28.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.007+0000 7f47c93c0b80 1 -- start start 2023-05-20T01:27:28.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.007+0000 7f47c93c0b80 1 Processor -- start 2023-05-20T01:27:28.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.008+0000 7f47c93c0b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559feb36b940 0x559feb41d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:28.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.008+0000 7f47c93c0b80 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559feb3947d0 0x559feb3b2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:28.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.008+0000 7f47c93c0b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559feb3984d0 0x559feb3b2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:28.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.008+0000 7f47c93c0b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x559feb38a570 con 0x559feb3984d0 2023-05-20T01:27:28.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.008+0000 7f47c93c0b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x559feb41d8b0 con 0x559feb36b940 2023-05-20T01:27:28.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.008+0000 7f47c93c0b80 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x559feb36c380 con 0x559feb3947d0 2023-05-20T01:27:28.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.008+0000 7f47c93c0b80 10 client.0 _ll_register_callbacks cb 0x7f47a8009380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-05-20T01:27:28.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.008+0000 7f47b759e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559feb3984d0 0x559feb3b2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:28.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.008+0000 7f47b759e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559feb3984d0 0x559feb3b2a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39286/0 (socket says 172.21.15.16:39286) 2023-05-20T01:27:28.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.008+0000 7f47b759e700 1 -- 172.21.15.16:0/2677728091 learned_addr learned my addr 172.21.15.16:0/2677728091 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:27:28.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.008+0000 7f47b6d9d700 1 --2- 172.21.15.16:0/2677728091 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559feb36b940 0x559feb41d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:28.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.008+0000 7f47b659c700 1 --2- 172.21.15.16:0/2677728091 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559feb3947d0 0x559feb3b2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:28.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.009+0000 7f47b759e700 1 -- 172.21.15.16:0/2677728091 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559feb3947d0 msgr2=0x559feb3b2540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:28.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.009+0000 7f47b759e700 1 --2- 172.21.15.16:0/2677728091 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559feb3947d0 0x559feb3b2540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:28.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.009+0000 7f47b759e700 1 -- 172.21.15.16:0/2677728091 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559feb36b940 msgr2=0x559feb41d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:28.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.009+0000 7f47b759e700 1 --2- 172.21.15.16:0/2677728091 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559feb36b940 0x559feb41d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:28.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.009+0000 7f47b759e700 1 -- 172.21.15.16:0/2677728091 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47b80cfc60 con 0x559feb3984d0 2023-05-20T01:27:28.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.009+0000 7f47b759e700 1 --2- 172.21.15.16:0/2677728091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559feb3984d0 0x559feb3b2a80 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f47b00874a0 tx=0x7f47b0001460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad491b12ffff4b6e server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:28.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.009+0000 7f4795ffb700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-05-20T01:27:28.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.009+0000 7f4795ffb700 1 -- 172.21.15.16:0/2677728091 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b0057450 con 0x559feb3984d0 2023-05-20T01:27:28.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.009+0000 7f4795ffb700 1 -- 172.21.15.16:0/2677728091 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(123..123 src has 1..123) v4 ==== 7073+0+0 (secure 0 0 0) 0x7f47b0007720 con 0x559feb3984d0 2023-05-20T01:27:28.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.009+0000 7f47c93c0b80 1 -- 172.21.15.16:0/2677728091 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f47ac01a070 con 0x559feb3984d0 2023-05-20T01:27:28.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.009+0000 7f4795ffb700 1 -- 172.21.15.16:0/2677728091 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f47b00575b0 con 0x559feb3984d0 2023-05-20T01:27:28.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.009+0000 7f47c93c0b80 10 client.8442 fetch_fsmap learned FSMap version 91 2023-05-20T01:27:28.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.009+0000 7f47c93c0b80 1 -- 172.21.15.16:0/2677728091 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) v3 -- 0x559feb0d5ce0 con 0x559feb3984d0 2023-05-20T01:27:28.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.009+0000 7f4795ffb700 1 -- 172.21.15.16:0/2677728091 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47ac01a070 con 0x559feb3984d0 2023-05-20T01:27:28.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.009+0000 7f4795ffb700 1 -- 172.21.15.16:0/2677728091 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 91) v1 ==== 3122+0+0 (secure 0 0 0) 0x7f47b00078e0 con 0x559feb3984d0 2023-05-20T01:27:28.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.009+0000 7f47c93c0b80 10 client.8442 fetch_fsmap finished waiting for FSMap version 91 2023-05-20T01:27:28.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.009+0000 7f47c93c0b80 10 client.8442 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2023-05-20T01:27:28.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.009+0000 7f47c93c0b80 20 client.8442 populate_metadata read hostname 'smithi016' 2023-05-20T01:27:28.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.009+0000 7f47c93c0b80 1 --2- 172.21.15.16:0/2677728091 >> [v2:172.21.15.16:6835/1927078882,v1:172.21.15.16:6837/1927078882] conn(0x559feb392900 0x559feb3a8c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:28.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.009+0000 7f47c93c0b80 4 client.8442 mds_command: new command op to 8376 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2023-05-20T01:27:28.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.009+0000 7f47c93c0b80 1 -- 172.21.15.16:0/2677728091 --> [v2:172.21.15.16:6835/1927078882,v1:172.21.15.16:6837/1927078882] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) v1 -- 0x559feb28d6e0 con 0x559feb392900 2023-05-20T01:27:28.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.009+0000 7f47b6d9d700 1 --2- 172.21.15.16:0/2677728091 >> [v2:172.21.15.16:6835/1927078882,v1:172.21.15.16:6837/1927078882] conn(0x559feb392900 0x559feb3a8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:28.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.010+0000 7f4795ffb700 10 client.8442 ms_handle_connect on v2:172.21.15.16:6835/1927078882 2023-05-20T01:27:28.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.010+0000 7f47b6d9d700 1 --2- 172.21.15.16:0/2677728091 >> [v2:172.21.15.16:6835/1927078882,v1:172.21.15.16:6837/1927078882] conn(0x559feb392900 0x559feb3a8c70 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f370f5cb2c8f38 server_cookie=a56917c6978e29bd in_seq=0 out_seq=0 2023-05-20T01:27:28.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.010+0000 7f4795ffb700 1 -- 172.21.15.16:0/2677728091 <== mds.0 v2:172.21.15.16:6835/1927078882 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x559feb28d6e0 con 0x559feb392900 2023-05-20T01:27:28.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.010+0000 7f4795ffb700 10 client.8442 handle_command_reply: tid=0 2023-05-20T01:27:28.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.010+0000 7f47c93c0b80 1 client.8442 shutdown 2023-05-20T01:27:28.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.011+0000 7f47c93c0b80 1 -- 172.21.15.16:0/2677728091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559feb3984d0 msgr2=0x559feb3b2a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:28.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.011+0000 7f47c93c0b80 1 --2- 172.21.15.16:0/2677728091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559feb3984d0 0x559feb3b2a80 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f47b00874a0 tx=0x7f47b0001460 comp rx=0 tx=0).stop 2023-05-20T01:27:28.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.011+0000 7f47c93c0b80 1 -- 172.21.15.16:0/2677728091 shutdown_connections 2023-05-20T01:27:28.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.011+0000 7f47c93c0b80 1 --2- 172.21.15.16:0/2677728091 >> [v2:172.21.15.16:6835/1927078882,v1:172.21.15.16:6837/1927078882] conn(0x559feb392900 0x559feb3a8c70 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:28.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.011+0000 7f47c93c0b80 1 --2- 172.21.15.16:0/2677728091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559feb3984d0 0x559feb3b2a80 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:28.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.011+0000 7f47c93c0b80 1 --2- 172.21.15.16:0/2677728091 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559feb36b940 0x559feb41d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:28.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.011+0000 7f47c93c0b80 1 --2- 172.21.15.16:0/2677728091 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559feb3947d0 0x559feb3b2540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:28.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.011+0000 7f4795ffb700 0 client.8442 ms_handle_reset on v2:172.21.15.16:6835/1927078882 2023-05-20T01:27:28.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.011+0000 7f47c93c0b80 1 -- 172.21.15.16:0/2677728091 >> 172.21.15.16:0/2677728091 conn(0x559feb363000 msgr2=0x559feb398b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:27:28.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.011+0000 7f47c93c0b80 1 -- 172.21.15.16:0/2677728091 shutdown_connections 2023-05-20T01:27:28.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.011+0000 7f47c93c0b80 1 -- 172.21.15.16:0/2677728091 wait complete. 2023-05-20T01:27:28.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.011+0000 7f47c93c0b80 20 client.8442 trim_cache size 0 max 16384 2023-05-20T01:27:28.025 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-05-20T01:27:28.025 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2023-05-20T01:27:28.025 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "ead1accb-88a4-49bd-9edb-9c6e858f3eb4", 2023-05-20T01:27:28.025 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2023-05-20T01:27:28.025 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-05-20T01:27:28.026 INFO:tasks.ceph.mds.c.smithi016.stderr:2023-05-20T01:27:28.011+0000 7efe54384700 -1 log_channel(cluster) log [ERR] : bad backtrace on directory inode 0x10000000000 2023-05-20T01:27:28.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.011+0000 7f47bd234700 1 -- 172.21.15.16:0/1608136344 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f47a406ace0 msgr2=0x7f47a406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:28.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.011+0000 7f47bd234700 1 --2- 172.21.15.16:0/1608136344 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f47a406ace0 0x7f47a406d190 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f47ac009b60 tx=0x7f47ac008040 comp rx=0 tx=0).stop 2023-05-20T01:27:28.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.011+0000 7f47bd234700 1 -- 172.21.15.16:0/1608136344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47b815cc00 msgr2=0x7f47b8150bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:28.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.011+0000 7f47bd234700 1 --2- 172.21.15.16:0/1608136344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47b815cc00 0x7f47b8150bc0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f47b0002230 tx=0x7f47b000a060 comp rx=0 tx=0).stop 2023-05-20T01:27:28.027 INFO:tasks.ceph.mds.c.smithi016.stderr:2023-05-20T01:27:28.011+0000 7efe54384700 -1 log_channel(cluster) log [ERR] : bad backtrace on directory inode 0x4 2023-05-20T01:27:28.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.012+0000 7f47bd234700 1 -- 172.21.15.16:0/1608136344 shutdown_connections 2023-05-20T01:27:28.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.012+0000 7f47bd234700 1 --2- 172.21.15.16:0/1608136344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47b815cc00 0x7f47b8150bc0 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:28.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.012+0000 7f47bd234700 1 --2- 172.21.15.16:0/1608136344 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f47b80c6b50 0x7f47b815bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:28.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.012+0000 7f47bd234700 1 --2- 172.21.15.16:0/1608136344 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f47a406ace0 0x7f47a406d190 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:28.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.012+0000 7f47bd234700 1 --2- 172.21.15.16:0/1608136344 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f47b815c830 0x7f47b80accc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:28.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.012+0000 7f47bd234700 1 -- 172.21.15.16:0/1608136344 >> 172.21.15.16:0/1608136344 conn(0x7f47b80b2920 msgr2=0x7f47b80c6230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:27:28.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.012+0000 7f47bd234700 1 -- 172.21.15.16:0/1608136344 shutdown_connections 2023-05-20T01:27:28.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.012+0000 7f47bd234700 1 -- 172.21.15.16:0/1608136344 wait complete. 2023-05-20T01:27:28.030 INFO:tasks.ceph.mds.c.smithi016.stderr:2023-05-20T01:27:28.012+0000 7efe54384700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /stripey/ auth v1 pv4 ap=3 DIRTYPARENT f(v0 1=1+0) n() (inest lock dirty) (ifile lock->sync w=1 flushing) (iversion lock) | dirtyscattered=2 lock=1 dirfrag=1 dirtyparent=1 waiter=1 authpin=1 scrubqueue=0 0x564d4a432580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(15)0x10000000000:[<0x1/stripey v1>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 1=1+0)","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 1=1+0)","memory_value.rstat":"n()","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-61} 2023-05-20T01:27:28.030 INFO:tasks.ceph.mds.c.smithi016.stderr:2023-05-20T01:27:28.013+0000 7efe54384700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x4 [...2,head] /lost+found/ auth v1 pv6 ap=3 DIRTYPARENT f(v0 1=1+0) n() (inest lock dirty) (ifile lock->sync w=1 flushing) (iversion lock) | dirtyscattered=2 lock=1 dirfrag=1 dirtyparent=1 waiter=1 authpin=1 scrubqueue=0 0x564d4a432b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(15)0x4:[<0x1/lost+found v1>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 1=1+0)","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 1=1+0)","memory_value.rstat":"n()","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-61} 2023-05-20T01:27:28.030 INFO:tasks.ceph.mds.c.smithi016.stderr:2023-05-20T01:27:28.013+0000 7efe54384700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001fb [2,head] /stripey/flushed_file_6 auth v1 ap=1 DIRTYPARENT s=1572864 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x564d4a485180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(17)0x100000001fb:[<0x10000000000/flushed_file_6 v54>,<0x1/stripey v59>]//[16]","memoryvalue":"(17)0x100000001fb:[<0x10000000000/flushed_file_6 v1>,<0x1/stripey v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-05-20T01:27:28.030 INFO:tasks.ceph.mds.c.smithi016.stderr:2023-05-20T01:27:28.013+0000 7efe54384700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001f9 [2,head] /stripey/flushed_file_4 auth v1 ap=1 DIRTYPARENT s=1441792 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x564d4a4b3080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(17)0x100000001f9:[<0x10000000000/flushed_file_4 v38>,<0x1/stripey v59>]//[16]","memoryvalue":"(17)0x100000001f9:[<0x10000000000/flushed_file_4 v1>,<0x1/stripey v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-05-20T01:27:28.030 INFO:tasks.ceph.mds.c.smithi016.stderr:2023-05-20T01:27:28.014+0000 7efe54384700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001f8 [2,head] /stripey/flushed_file_3 auth v1 ap=1 DIRTYPARENT s=917503 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x564d4a4b3600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(17)0x100000001f8:[<0x10000000000/flushed_file_3 v24>,<0x1/stripey v59>]//[16]","memoryvalue":"(17)0x100000001f8:[<0x10000000000/flushed_file_3 v1>,<0x1/stripey v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-05-20T01:27:28.030 INFO:tasks.ceph.mds.c.smithi016.stderr:2023-05-20T01:27:28.014+0000 7efe54384700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001f7 [2,head] /stripey/flushed_file_2 auth v1 ap=1 DIRTYPARENT s=917504 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x564d4a4b3b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(17)0x100000001f7:[<0x10000000000/flushed_file_2 v28>,<0x1/stripey v59>]//[16]","memoryvalue":"(17)0x100000001f7:[<0x10000000000/flushed_file_2 v1>,<0x1/stripey v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-05-20T01:27:28.030 INFO:tasks.ceph.mds.c.smithi016.stderr:2023-05-20T01:27:28.014+0000 7efe54384700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001f6 [2,head] /stripey/flushed_file_1 auth v1 ap=1 DIRTYPARENT s=524288 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x564d4a4b4100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(17)0x100000001f6:[<0x10000000000/flushed_file_1 v26>,<0x1/stripey v59>]//[16]","memoryvalue":"(17)0x100000001f6:[<0x10000000000/flushed_file_1 v1>,<0x1/stripey v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-05-20T01:27:28.031 INFO:tasks.ceph.mds.c.smithi016.stderr:2023-05-20T01:27:28.014+0000 7efe54384700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001fa [2,head] /stripey/flushed_file_5 auth v1 ap=1 DIRTYPARENT s=1441793 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x564d4a485700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(17)0x100000001fa:[<0x10000000000/flushed_file_5 v46>,<0x1/stripey v59>]//[16]","memoryvalue":"(17)0x100000001fa:[<0x10000000000/flushed_file_5 v1>,<0x1/stripey v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-05-20T01:27:28.031 INFO:tasks.ceph.mds.c.smithi016.stderr:2023-05-20T01:27:28.014+0000 7efe54384700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /stripey/flushed_file_0 auth v1 ap=1 DIRTYPARENT s=1048576 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x564d4a4b4680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(17)0x10000000001:[<0x10000000000/flushed_file_0 v30>,<0x1/stripey v59>]//[16]","memoryvalue":"(17)0x10000000001:[<0x10000000000/flushed_file_0 v1>,<0x1/stripey v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-05-20T01:27:28.031 INFO:tasks.ceph.mds.c.smithi016.stderr:2023-05-20T01:27:28.014+0000 7efe54384700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001fc [2,head] /lost+found/100000001fc auth v1 ap=1 DIRTYPARENT s=524288 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x564d4a4b4c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(17)0x100000001fc:[<0x4/100000001fc v1>,<0x1/lost+found v1>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":-61} 2023-05-20T01:27:28.031 INFO:tasks.ceph.mds.c.smithi016.stderr:2023-05-20T01:27:28.023+0000 7efe54384700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1 [...2,head] / auth v7 ap=5 snaprealm=0x564d4a3b4d80 DIRTYPARENT f(v1 2=0+2) n(v0 rc2023-05-20T01:27:27.660582+0000 2=0+2)/n() (idft sync r=1) (inest sync r=1) (ifile sync->mix r=1) (iversion lock) caps={6884=pLsXs/-@45} | dirtyscattered=0 request=0 lock=3 dirfrag=1 caps=1 openingsnapparents=0 dirtyparent=1 dirty=1 waiter=1 authpin=1 scrubqueue=0 0x564d4a434100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(15)0x1:[]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 2=0+2)","ondisk_value.rstat":"n(v0 3=0+3)","memory_value.dirstat":"f(v1 2=0+2)","memory_value.rstat":"n(v0 rc2023-05-20T01:27:27.660582+0000 2=0+2)","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-61} 2023-05-20T01:27:28.032 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.5:0 scrub status 2023-05-20T01:27:28.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.154+0000 7fe515ced700 1 Processor -- start 2023-05-20T01:27:28.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.154+0000 7fe515ced700 1 -- start start 2023-05-20T01:27:28.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.155+0000 7fe515ced700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe51015cec0 0x7fe510071bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:28.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.155+0000 7fe515ced700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fe510072130 0x7fe5100cf9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:28.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.155+0000 7fe515ced700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe510072590 con 0x7fe51015d290 2023-05-20T01:27:28.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.155+0000 7fe515ced700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5100cd630 con 0x7fe510072130 2023-05-20T01:27:28.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.155+0000 7fe515ced700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5100cca60 con 0x7fe51015cec0 2023-05-20T01:27:28.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.155+0000 7fe50ffff700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fe510072130 0x7fe5100cf9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:28.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.155+0000 7fe50f7fe700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe51015cec0 0x7fe510071bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:28.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.155+0000 7fe50effd700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7fe51015d290 0x7fe5100cf2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.16:60534/0 (socket says 172.21.15.16:60534) 2023-05-20T01:27:28.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.155+0000 7fe50effd700 1 -- 172.21.15.16:0/1543512756 learned_addr learned my addr 172.21.15.16:0/1543512756 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:27:28.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.155+0000 7fe50ffff700 1 -- 172.21.15.16:0/1543512756 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe51015cec0 msgr2=0x7fe510071bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:28.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.155+0000 7fe50ffff700 1 --2- 172.21.15.16:0/1543512756 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe51015cec0 0x7fe510071bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:28.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.155+0000 7fe50ffff700 1 -- 172.21.15.16:0/1543512756 >> v1:172.21.15.132:6789/0 conn(0x7fe51015d290 legacy=0x7fe5100cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:28.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.155+0000 7fe50ffff700 1 -- 172.21.15.16:0/1543512756 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5100ccd00 con 0x7fe510072130 2023-05-20T01:27:28.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.155+0000 7fe50e7fc700 1 -- 172.21.15.16:0/1543512756 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3052892343 0 0) 0x7fe510072590 con 0x7fe51015d290 2023-05-20T01:27:28.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.155+0000 7fe50f7fe700 1 --2- 172.21.15.16:0/1543512756 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe51015cec0 0x7fe510071bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-05-20T01:27:28.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.156+0000 7fe50ffff700 1 --2- 172.21.15.16:0/1543512756 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe510072130 0x7fe5100cf9d0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fe508009d90 tx=0x7fe50800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d193bbd52dd9cc7 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:28.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.156+0000 7fe50e7fc700 1 -- 172.21.15.16:0/1543512756 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe50800e070 con 0x7fe510072130 2023-05-20T01:27:28.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.156+0000 7fe50e7fc700 1 -- 172.21.15.16:0/1543512756 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe50800bb20 con 0x7fe510072130 2023-05-20T01:27:28.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.156+0000 7fe50e7fc700 1 -- 172.21.15.16:0/1543512756 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe508012430 con 0x7fe510072130 2023-05-20T01:27:28.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.156+0000 7fe515ced700 1 -- 172.21.15.16:0/1543512756 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe510072130 msgr2=0x7fe5100cf9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:28.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.156+0000 7fe515ced700 1 --2- 172.21.15.16:0/1543512756 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe510072130 0x7fe5100cf9d0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fe508009d90 tx=0x7fe50800b330 comp rx=0 tx=0).stop 2023-05-20T01:27:28.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.156+0000 7fe515ced700 1 -- 172.21.15.16:0/1543512756 shutdown_connections 2023-05-20T01:27:28.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.156+0000 7fe515ced700 1 --2- 172.21.15.16:0/1543512756 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe51015cec0 0x7fe510071bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:28.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.156+0000 7fe515ced700 1 --2- 172.21.15.16:0/1543512756 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe510072130 0x7fe5100cf9d0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:28.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.156+0000 7fe515ced700 1 -- 172.21.15.16:0/1543512756 >> 172.21.15.16:0/1543512756 conn(0x7fe5100cffd0 msgr2=0x7fe5100d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:27:28.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.156+0000 7fe515ced700 1 -- 172.21.15.16:0/1543512756 shutdown_connections 2023-05-20T01:27:28.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.156+0000 7fe515ced700 1 -- 172.21.15.16:0/1543512756 wait complete. 2023-05-20T01:27:28.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.158+0000 7fe515ced700 1 Processor -- start 2023-05-20T01:27:28.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.158+0000 7fe515ced700 1 -- start start 2023-05-20T01:27:28.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.158+0000 7fe515ced700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe510072130 0x7fe5101483f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:28.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.158+0000 7fe515ced700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe51015cec0 0x7fe5100c9580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:28.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.158+0000 7fe515ced700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe51015d290 0x7fe51015d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:28.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.159+0000 7fe515ced700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe510148930 con 0x7fe51015d290 2023-05-20T01:27:28.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.159+0000 7fe515ced700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5101f2970 con 0x7fe51015cec0 2023-05-20T01:27:28.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.159+0000 7fe515ced700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5100ca040 con 0x7fe510072130 2023-05-20T01:27:28.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.159+0000 7fe50ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe51015d290 0x7fe51015d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:28.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.159+0000 7fe50ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe51015d290 0x7fe51015d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39302/0 (socket says 172.21.15.16:39302) 2023-05-20T01:27:28.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.159+0000 7fe50ffff700 1 -- 172.21.15.16:0/2191048470 learned_addr learned my addr 172.21.15.16:0/2191048470 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:27:28.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.159+0000 7fe50effd700 1 --2- 172.21.15.16:0/2191048470 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe51015cec0 0x7fe5100c9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:28.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.159+0000 7fe50f7fe700 1 --2- 172.21.15.16:0/2191048470 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe510072130 0x7fe5101483f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:28.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.159+0000 7fe50ffff700 1 -- 172.21.15.16:0/2191048470 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe510072130 msgr2=0x7fe5101483f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:28.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.159+0000 7fe50ffff700 1 --2- 172.21.15.16:0/2191048470 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe510072130 0x7fe5101483f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:28.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.159+0000 7fe50ffff700 1 -- 172.21.15.16:0/2191048470 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe51015cec0 msgr2=0x7fe5100c9580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:28.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.159+0000 7fe50ffff700 1 --2- 172.21.15.16:0/2191048470 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe51015cec0 0x7fe5100c9580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:28.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.159+0000 7fe50ffff700 1 -- 172.21.15.16:0/2191048470 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe500009430 con 0x7fe51015d290 2023-05-20T01:27:28.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.159+0000 7fe50ffff700 1 --2- 172.21.15.16:0/2191048470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe51015d290 0x7fe51015d700 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7fe5100cdb50 tx=0x7fe508004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:28.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.159+0000 7fe4f7fff700 1 -- 172.21.15.16:0/2191048470 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe508012430 con 0x7fe51015d290 2023-05-20T01:27:28.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.159+0000 7fe4f7fff700 1 -- 172.21.15.16:0/2191048470 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe508020070 con 0x7fe51015d290 2023-05-20T01:27:28.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.159+0000 7fe515ced700 1 -- 172.21.15.16:0/2191048470 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe508009a70 con 0x7fe51015d290 2023-05-20T01:27:28.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.159+0000 7fe4f7fff700 1 -- 172.21.15.16:0/2191048470 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe508004280 con 0x7fe51015d290 2023-05-20T01:27:28.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.159+0000 7fe515ced700 1 -- 172.21.15.16:0/2191048470 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5100cd2c0 con 0x7fe51015d290 2023-05-20T01:27:28.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.161+0000 7fe4f7fff700 1 -- 172.21.15.16:0/2191048470 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe5080043e0 con 0x7fe51015d290 2023-05-20T01:27:28.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.161+0000 7fe4f7fff700 1 --2- 172.21.15.16:0/2191048470 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe4fc06ace0 0x7fe4fc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:28.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.161+0000 7fe4f7fff700 1 -- 172.21.15.16:0/2191048470 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 7073+0+0 (secure 0 0 0) 0x7fe508022070 con 0x7fe51015d290 2023-05-20T01:27:28.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.161+0000 7fe50f7fe700 1 --2- 172.21.15.16:0/2191048470 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe4fc06ace0 0x7fe4fc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:28.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.162+0000 7fe50f7fe700 1 --2- 172.21.15.16:0/2191048470 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe4fc06ace0 0x7fe4fc06d190 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fe51015a730 tx=0x7fe5000091c0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:28.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.163+0000 7fe521e79b80 1 Processor -- start 2023-05-20T01:27:28.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.163+0000 7fe521e79b80 1 -- start start 2023-05-20T01:27:28.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.164+0000 7fe521e79b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f669fb7450 0x55f669fb7820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:28.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.164+0000 7fe521e79b80 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55f669fb3750 0x55f669f8c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:28.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.164+0000 7fe521e79b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f669f8a8c0 0x55f669f8c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:28.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.164+0000 7fe521e79b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f669e92060 con 0x55f669f8a8c0 2023-05-20T01:27:28.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.164+0000 7fe521e79b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f669ee48a0 con 0x55f669fb7450 2023-05-20T01:27:28.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.164+0000 7fe521e79b80 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f669eddd30 con 0x55f669fb3750 2023-05-20T01:27:28.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.164+0000 7fe50ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f669f8a8c0 0x55f669f8c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:28.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.164+0000 7fe50ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f669f8a8c0 0x55f669f8c9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39310/0 (socket says 172.21.15.16:39310) 2023-05-20T01:27:28.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.164+0000 7fe50ffff700 1 -- 172.21.15.16:0/3486317218 learned_addr learned my addr 172.21.15.16:0/3486317218 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:27:28.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.165+0000 7fe50effd700 1 --2- 172.21.15.16:0/3486317218 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55f669fb3750 0x55f669f8c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:28.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.165+0000 7fe50f7fe700 1 --2- 172.21.15.16:0/3486317218 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f669fb7450 0x55f669fb7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:28.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.165+0000 7fe50ffff700 1 -- 172.21.15.16:0/3486317218 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55f669fb3750 msgr2=0x55f669f8c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:28.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.165+0000 7fe50ffff700 1 --2- 172.21.15.16:0/3486317218 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55f669fb3750 0x55f669f8c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:28.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.165+0000 7fe50ffff700 1 -- 172.21.15.16:0/3486317218 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f669fb7450 msgr2=0x55f669fb7820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:28.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.165+0000 7fe50ffff700 1 --2- 172.21.15.16:0/3486317218 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f669fb7450 0x55f669fb7820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:28.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.165+0000 7fe50ffff700 1 -- 172.21.15.16:0/3486317218 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f669d5f630 con 0x55f669f8a8c0 2023-05-20T01:27:28.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.165+0000 7fe50ffff700 1 --2- 172.21.15.16:0/3486317218 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f669f8a8c0 0x55f669f8c9f0 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7fe5080894c0 tx=0x7fe5080597a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cb0c17ebe1e57fb server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:28.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.165+0000 7fe4f57fa700 1 -- 172.21.15.16:0/3486317218 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe50805de00 con 0x55f669f8a8c0 2023-05-20T01:27:28.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.165+0000 7fe4f57fa700 1 -- 172.21.15.16:0/3486317218 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe508062190 con 0x55f669f8a8c0 2023-05-20T01:27:28.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.165+0000 7fe4f57fa700 1 -- 172.21.15.16:0/3486317218 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe50808b0b0 con 0x55f669f8a8c0 2023-05-20T01:27:28.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.166+0000 7fe521e79b80 1 -- 172.21.15.16:0/3486317218 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f669f8a8c0 msgr2=0x55f669f8c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:28.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.166+0000 7fe521e79b80 1 --2- 172.21.15.16:0/3486317218 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f669f8a8c0 0x55f669f8c9f0 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7fe5080894c0 tx=0x7fe5080597a0 comp rx=0 tx=0).stop 2023-05-20T01:27:28.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.166+0000 7fe521e79b80 1 -- 172.21.15.16:0/3486317218 shutdown_connections 2023-05-20T01:27:28.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.166+0000 7fe521e79b80 1 --2- 172.21.15.16:0/3486317218 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f669f8a8c0 0x55f669f8c9f0 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:28.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.166+0000 7fe521e79b80 1 --2- 172.21.15.16:0/3486317218 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f669fb7450 0x55f669fb7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:28.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.166+0000 7fe521e79b80 1 --2- 172.21.15.16:0/3486317218 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55f669fb3750 0x55f669f8c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:28.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.166+0000 7fe521e79b80 1 -- 172.21.15.16:0/3486317218 >> 172.21.15.16:0/3486317218 conn(0x55f669f81f80 msgr2=0x55f669fd2260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:27:28.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.166+0000 7fe521e79b80 1 -- 172.21.15.16:0/3486317218 shutdown_connections 2023-05-20T01:27:28.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.166+0000 7fe521e79b80 1 -- 172.21.15.16:0/3486317218 wait complete. 2023-05-20T01:27:28.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.166+0000 7fe521e79b80 1 -- start start 2023-05-20T01:27:28.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.167+0000 7fe521e79b80 1 Processor -- start 2023-05-20T01:27:28.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.167+0000 7fe521e79b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f669f8a8c0 0x55f669fd0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:28.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.167+0000 7fe521e79b80 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55f669fb3750 0x55f669fb0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:28.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.167+0000 7fe521e79b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f669fb7450 0x55f669fd4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:28.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.167+0000 7fe521e79b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f669e7f8e0 con 0x55f669f8a8c0 2023-05-20T01:27:28.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.167+0000 7fe521e79b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f669e7f580 con 0x55f669fb7450 2023-05-20T01:27:28.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.167+0000 7fe521e79b80 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f669d61920 con 0x55f669fb3750 2023-05-20T01:27:28.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.167+0000 7fe521e79b80 10 client.0 _ll_register_callbacks cb 0x55f669e9d780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-05-20T01:27:28.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.167+0000 7fe50f7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f669f8a8c0 0x55f669fd0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:28.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.167+0000 7fe50f7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f669f8a8c0 0x55f669fd0a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39312/0 (socket says 172.21.15.16:39312) 2023-05-20T01:27:28.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.167+0000 7fe50f7fe700 1 -- 172.21.15.16:0/1393001661 learned_addr learned my addr 172.21.15.16:0/1393001661 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:27:28.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.167+0000 7fe50ffff700 1 --2- 172.21.15.16:0/1393001661 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f669fb7450 0x55f669fd4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:28.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.168+0000 7fe50effd700 1 --2- 172.21.15.16:0/1393001661 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55f669fb3750 0x55f669fb0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:28.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.168+0000 7fe50f7fe700 1 -- 172.21.15.16:0/1393001661 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55f669fb3750 msgr2=0x55f669fb0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:28.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.168+0000 7fe50f7fe700 1 --2- 172.21.15.16:0/1393001661 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55f669fb3750 0x55f669fb0540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:28.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.168+0000 7fe50f7fe700 1 -- 172.21.15.16:0/1393001661 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f669fb7450 msgr2=0x55f669fd4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:28.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.168+0000 7fe50f7fe700 1 --2- 172.21.15.16:0/1393001661 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f669fb7450 0x55f669fd4900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:28.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.168+0000 7fe50f7fe700 1 -- 172.21.15.16:0/1393001661 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5100cd2c0 con 0x55f669f8a8c0 2023-05-20T01:27:28.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.168+0000 7fe50f7fe700 1 --2- 172.21.15.16:0/1393001661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f669f8a8c0 0x55f669fd0a20 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7fe500006350 tx=0x7fe500004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c8ac8b024d167fc server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:28.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.168+0000 7fe4ee7fc700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-05-20T01:27:28.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.168+0000 7fe4ee7fc700 1 -- 172.21.15.16:0/1393001661 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe50001c040 con 0x55f669f8a8c0 2023-05-20T01:27:28.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.168+0000 7fe4ee7fc700 1 -- 172.21.15.16:0/1393001661 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(123..123 src has 1..123) v4 ==== 7073+0+0 (secure 0 0 0) 0x7fe500004030 con 0x55f669f8a8c0 2023-05-20T01:27:28.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.168+0000 7fe521e79b80 1 -- 172.21.15.16:0/1393001661 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f669f7f740 con 0x55f669f8a8c0 2023-05-20T01:27:28.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.168+0000 7fe4ee7fc700 1 -- 172.21.15.16:0/1393001661 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe5000041f0 con 0x55f669f8a8c0 2023-05-20T01:27:28.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.168+0000 7fe521e79b80 10 client.8451 fetch_fsmap learned FSMap version 91 2023-05-20T01:27:28.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.168+0000 7fe521e79b80 1 -- 172.21.15.16:0/1393001661 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) v3 -- 0x55f669ea4190 con 0x55f669f8a8c0 2023-05-20T01:27:28.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.168+0000 7fe4ee7fc700 1 -- 172.21.15.16:0/1393001661 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f669f7f740 con 0x55f669f8a8c0 2023-05-20T01:27:28.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.168+0000 7fe4ee7fc700 1 -- 172.21.15.16:0/1393001661 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 91) v1 ==== 3122+0+0 (secure 0 0 0) 0x7fe500004390 con 0x55f669f8a8c0 2023-05-20T01:27:28.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.169+0000 7fe521e79b80 10 client.8451 fetch_fsmap finished waiting for FSMap version 91 2023-05-20T01:27:28.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.169+0000 7fe521e79b80 10 client.8451 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2023-05-20T01:27:28.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.169+0000 7fe521e79b80 20 client.8451 populate_metadata read hostname 'smithi016' 2023-05-20T01:27:28.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.169+0000 7fe521e79b80 1 --2- 172.21.15.16:0/1393001661 >> [v2:172.21.15.16:6835/1927078882,v1:172.21.15.16:6837/1927078882] conn(0x55f669fb2130 0x55f669fd36e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:28.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.169+0000 7fe521e79b80 4 client.8451 mds_command: new command op to 8376 tid=0[{"prefix": "get_command_descriptions"}] 2023-05-20T01:27:28.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.169+0000 7fe521e79b80 1 -- 172.21.15.16:0/1393001661 --> [v2:172.21.15.16:6835/1927078882,v1:172.21.15.16:6837/1927078882] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f669ea00a0 con 0x55f669fb2130 2023-05-20T01:27:28.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.169+0000 7fe50effd700 1 --2- 172.21.15.16:0/1393001661 >> [v2:172.21.15.16:6835/1927078882,v1:172.21.15.16:6837/1927078882] conn(0x55f669fb2130 0x55f669fd36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:28.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.169+0000 7fe4ee7fc700 10 client.8451 ms_handle_connect on v2:172.21.15.16:6835/1927078882 2023-05-20T01:27:28.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.169+0000 7fe50effd700 1 --2- 172.21.15.16:0/1393001661 >> [v2:172.21.15.16:6835/1927078882,v1:172.21.15.16:6837/1927078882] conn(0x55f669fb2130 0x55f669fd36e0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ac875ded244932e server_cookie=c0cee6169b8f5c44 in_seq=0 out_seq=0 2023-05-20T01:27:28.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.169+0000 7fe4ee7fc700 1 -- 172.21.15.16:0/1393001661 <== mds.0 v2:172.21.15.16:6835/1927078882 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55f669ea00a0 con 0x55f669fb2130 2023-05-20T01:27:28.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.169+0000 7fe4ee7fc700 10 client.8451 handle_command_reply: tid=0 2023-05-20T01:27:28.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.170+0000 7fe521e79b80 1 client.8451 shutdown 2023-05-20T01:27:28.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.170+0000 7fe521e79b80 1 -- 172.21.15.16:0/1393001661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f669f8a8c0 msgr2=0x55f669fd0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:28.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.170+0000 7fe521e79b80 1 --2- 172.21.15.16:0/1393001661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f669f8a8c0 0x55f669fd0a20 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7fe500006350 tx=0x7fe500004750 comp rx=0 tx=0).stop 2023-05-20T01:27:28.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.170+0000 7fe521e79b80 1 -- 172.21.15.16:0/1393001661 shutdown_connections 2023-05-20T01:27:28.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.170+0000 7fe521e79b80 1 --2- 172.21.15.16:0/1393001661 >> [v2:172.21.15.16:6835/1927078882,v1:172.21.15.16:6837/1927078882] conn(0x55f669fb2130 0x55f669fd36e0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:28.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.170+0000 7fe521e79b80 1 --2- 172.21.15.16:0/1393001661 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f669fb7450 0x55f669fd4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:28.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.170+0000 7fe521e79b80 1 --2- 172.21.15.16:0/1393001661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f669f8a8c0 0x55f669fd0a20 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:28.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.170+0000 7fe4ee7fc700 0 client.8451 ms_handle_reset on v2:172.21.15.16:6835/1927078882 2023-05-20T01:27:28.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.170+0000 7fe521e79b80 1 --2- 172.21.15.16:0/1393001661 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55f669fb3750 0x55f669fb0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:28.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.170+0000 7fe521e79b80 1 -- 172.21.15.16:0/1393001661 >> 172.21.15.16:0/1393001661 conn(0x55f669f81f80 msgr2=0x55f669f886f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:27:28.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.170+0000 7fe521e79b80 1 -- 172.21.15.16:0/1393001661 shutdown_connections 2023-05-20T01:27:28.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.170+0000 7fe521e79b80 1 -- 172.21.15.16:0/1393001661 wait complete. 2023-05-20T01:27:28.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.170+0000 7fe521e79b80 20 client.8451 trim_cache size 0 max 16384 2023-05-20T01:27:28.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.187+0000 7fe521e79b80 1 Processor -- start 2023-05-20T01:27:28.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.187+0000 7fe521e79b80 1 -- start start 2023-05-20T01:27:28.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.188+0000 7fe521e79b80 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55f669f81f80 0x55f669fb0450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:28.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.188+0000 7fe521e79b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f669f8a8c0 0x55f669f86170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:28.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.188+0000 7fe521e79b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f669fb3750 0x55f669f866b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:28.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.188+0000 7fe521e79b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f669fb3b20 con 0x55f669fb3750 2023-05-20T01:27:28.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.188+0000 7fe521e79b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f669feaf20 con 0x55f669f8a8c0 2023-05-20T01:27:28.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.188+0000 7fe521e79b80 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f669feac00 con 0x55f669f81f80 2023-05-20T01:27:28.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.188+0000 7fe50ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f669fb3750 0x55f669f866b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:28.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.188+0000 7fe50ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f669fb3750 0x55f669f866b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39322/0 (socket says 172.21.15.16:39322) 2023-05-20T01:27:28.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.188+0000 7fe50ffff700 1 -- 172.21.15.16:0/691636267 learned_addr learned my addr 172.21.15.16:0/691636267 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:27:28.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.188+0000 7fe50ffff700 1 -- 172.21.15.16:0/691636267 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55f669f81f80 msgr2=0x55f669fb0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:28.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.188+0000 7fe50ffff700 1 --2- 172.21.15.16:0/691636267 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55f669f81f80 0x55f669fb0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:28.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.188+0000 7fe50ffff700 1 -- 172.21.15.16:0/691636267 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f669f8a8c0 msgr2=0x55f669f86170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:28.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.188+0000 7fe50ffff700 1 --2- 172.21.15.16:0/691636267 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f669f8a8c0 0x55f669f86170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:28.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.188+0000 7fe50ffff700 1 -- 172.21.15.16:0/691636267 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f669ea3ee0 con 0x55f669fb3750 2023-05-20T01:27:28.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.189+0000 7fe50ffff700 1 --2- 172.21.15.16:0/691636267 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f669fb3750 0x55f669f866b0 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7fe508089570 tx=0x7fe50806f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53a6b4bafcad4b75 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:28.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.189+0000 7fe4f5ffb700 1 -- 172.21.15.16:0/691636267 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe50805de00 con 0x55f669fb3750 2023-05-20T01:27:28.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.189+0000 7fe4f5ffb700 1 -- 172.21.15.16:0/691636267 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5080581e0 con 0x55f669fb3750 2023-05-20T01:27:28.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.189+0000 7fe4f5ffb700 1 -- 172.21.15.16:0/691636267 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe508061f40 con 0x55f669fb3750 2023-05-20T01:27:28.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.189+0000 7fe521e79b80 1 -- 172.21.15.16:0/691636267 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f669fb3750 msgr2=0x55f669f866b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:28.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.189+0000 7fe521e79b80 1 --2- 172.21.15.16:0/691636267 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f669fb3750 0x55f669f866b0 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7fe508089570 tx=0x7fe50806f040 comp rx=0 tx=0).stop 2023-05-20T01:27:28.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.189+0000 7fe521e79b80 1 -- 172.21.15.16:0/691636267 shutdown_connections 2023-05-20T01:27:28.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.189+0000 7fe521e79b80 1 --2- 172.21.15.16:0/691636267 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f669fb3750 0x55f669f866b0 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:28.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.189+0000 7fe521e79b80 1 --2- 172.21.15.16:0/691636267 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55f669f81f80 0x55f669fb0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:28.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.189+0000 7fe521e79b80 1 --2- 172.21.15.16:0/691636267 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f669f8a8c0 0x55f669f86170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:28.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.189+0000 7fe521e79b80 1 -- 172.21.15.16:0/691636267 >> 172.21.15.16:0/691636267 conn(0x55f669fb7f80 msgr2=0x55f669fd29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:27:28.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.189+0000 7fe521e79b80 1 -- 172.21.15.16:0/691636267 shutdown_connections 2023-05-20T01:27:28.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.189+0000 7fe521e79b80 1 -- 172.21.15.16:0/691636267 wait complete. 2023-05-20T01:27:28.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.190+0000 7fe521e79b80 1 -- start start 2023-05-20T01:27:28.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.190+0000 7fe521e79b80 1 Processor -- start 2023-05-20T01:27:28.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.191+0000 7fe521e79b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f669f81f80 0x55f66a031d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:28.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.191+0000 7fe521e79b80 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55f669f8a8c0 0x55f669fc86d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:28.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.191+0000 7fe521e79b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f669fb3750 0x55f669fba410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:28.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.191+0000 7fe521e79b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f669fbb020 con 0x55f669f81f80 2023-05-20T01:27:28.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.191+0000 7fe521e79b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f669fd1c70 con 0x55f669fb3750 2023-05-20T01:27:28.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.191+0000 7fe521e79b80 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f66a0322c0 con 0x55f669f8a8c0 2023-05-20T01:27:28.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.191+0000 7fe521e79b80 10 client.0 _ll_register_callbacks cb 0x7fe500005d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-05-20T01:27:28.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.191+0000 7fe50f7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f669f81f80 0x55f66a031d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:28.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.191+0000 7fe50f7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f669f81f80 0x55f66a031d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39326/0 (socket says 172.21.15.16:39326) 2023-05-20T01:27:28.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.191+0000 7fe50f7fe700 1 -- 172.21.15.16:0/2269131046 learned_addr learned my addr 172.21.15.16:0/2269131046 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:27:28.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.191+0000 7fe50ffff700 1 --2- 172.21.15.16:0/2269131046 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f669fb3750 0x55f669fba410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:28.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.191+0000 7fe50effd700 1 --2- 172.21.15.16:0/2269131046 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55f669f8a8c0 0x55f669fc86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:28.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.191+0000 7fe50f7fe700 1 -- 172.21.15.16:0/2269131046 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55f669f8a8c0 msgr2=0x55f669fc86d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:28.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.191+0000 7fe50f7fe700 1 --2- 172.21.15.16:0/2269131046 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55f669f8a8c0 0x55f669fc86d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:28.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.191+0000 7fe50f7fe700 1 -- 172.21.15.16:0/2269131046 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f669fb3750 msgr2=0x55f669fba410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:28.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.191+0000 7fe50f7fe700 1 --2- 172.21.15.16:0/2269131046 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f669fb3750 0x55f669fba410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:28.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.191+0000 7fe50f7fe700 1 -- 172.21.15.16:0/2269131046 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f669d5f630 con 0x55f669f81f80 2023-05-20T01:27:28.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.191+0000 7fe50ffff700 1 --2- 172.21.15.16:0/2269131046 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f669fb3750 0x55f669fba410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:27:28.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.191+0000 7fe50f7fe700 1 --2- 172.21.15.16:0/2269131046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f669f81f80 0x55f66a031d80 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7fe500004790 tx=0x7fe500016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f842d444f5f47bbc server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:28.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.191+0000 7fe4ee7fc700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-05-20T01:27:28.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.191+0000 7fe4ee7fc700 1 -- 172.21.15.16:0/2269131046 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe500004240 con 0x55f669f81f80 2023-05-20T01:27:28.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.192+0000 7fe4ee7fc700 1 -- 172.21.15.16:0/2269131046 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(123..123 src has 1..123) v4 ==== 7073+0+0 (secure 0 0 0) 0x7fe500030d80 con 0x55f669f81f80 2023-05-20T01:27:28.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.192+0000 7fe521e79b80 1 -- 172.21.15.16:0/2269131046 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe508062190 con 0x55f669f81f80 2023-05-20T01:27:28.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.192+0000 7fe4ee7fc700 1 -- 172.21.15.16:0/2269131046 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe500006940 con 0x55f669f81f80 2023-05-20T01:27:28.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.192+0000 7fe521e79b80 10 client.8457 fetch_fsmap learned FSMap version 91 2023-05-20T01:27:28.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.192+0000 7fe521e79b80 1 -- 172.21.15.16:0/2269131046 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) v3 -- 0x55f669cf4c60 con 0x55f669f81f80 2023-05-20T01:27:28.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.192+0000 7fe4ee7fc700 1 -- 172.21.15.16:0/2269131046 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe508062190 con 0x55f669f81f80 2023-05-20T01:27:28.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.192+0000 7fe4ee7fc700 1 -- 172.21.15.16:0/2269131046 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 91) v1 ==== 3122+0+0 (secure 0 0 0) 0x7fe500014650 con 0x55f669f81f80 2023-05-20T01:27:28.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.192+0000 7fe521e79b80 10 client.8457 fetch_fsmap finished waiting for FSMap version 91 2023-05-20T01:27:28.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.192+0000 7fe521e79b80 10 client.8457 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2023-05-20T01:27:28.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.192+0000 7fe521e79b80 20 client.8457 populate_metadata read hostname 'smithi016' 2023-05-20T01:27:28.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.192+0000 7fe521e79b80 1 --2- 172.21.15.16:0/2269131046 >> [v2:172.21.15.16:6835/1927078882,v1:172.21.15.16:6837/1927078882] conn(0x55f669fb7450 0x55f669fcb7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:28.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.192+0000 7fe521e79b80 4 client.8457 mds_command: new command op to 8376 tid=0[{"prefix": "scrub status"}] 2023-05-20T01:27:28.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.192+0000 7fe521e79b80 1 -- 172.21.15.16:0/2269131046 --> [v2:172.21.15.16:6835/1927078882,v1:172.21.15.16:6837/1927078882] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f669eac660 con 0x55f669fb7450 2023-05-20T01:27:28.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.192+0000 7fe50effd700 1 --2- 172.21.15.16:0/2269131046 >> [v2:172.21.15.16:6835/1927078882,v1:172.21.15.16:6837/1927078882] conn(0x55f669fb7450 0x55f669fcb7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:28.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.192+0000 7fe4ee7fc700 10 client.8457 ms_handle_connect on v2:172.21.15.16:6835/1927078882 2023-05-20T01:27:28.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.192+0000 7fe50effd700 1 --2- 172.21.15.16:0/2269131046 >> [v2:172.21.15.16:6835/1927078882,v1:172.21.15.16:6837/1927078882] conn(0x55f669fb7450 0x55f669fcb7b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d424681b73cf948b server_cookie=5153e00c6de68322 in_seq=0 out_seq=0 2023-05-20T01:27:28.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.193+0000 7fe4ee7fc700 1 -- 172.21.15.16:0/2269131046 <== mds.0 v2:172.21.15.16:6835/1927078882 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+260 (crc 0 0 0) 0x55f669eac660 con 0x55f669fb7450 2023-05-20T01:27:28.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.193+0000 7fe4ee7fc700 10 client.8457 handle_command_reply: tid=0 2023-05-20T01:27:28.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.193+0000 7fe521e79b80 1 client.8457 shutdown 2023-05-20T01:27:28.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.193+0000 7fe521e79b80 1 -- 172.21.15.16:0/2269131046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f669f81f80 msgr2=0x55f66a031d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:28.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.193+0000 7fe521e79b80 1 --2- 172.21.15.16:0/2269131046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f669f81f80 0x55f66a031d80 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7fe500004790 tx=0x7fe500016000 comp rx=0 tx=0).stop 2023-05-20T01:27:28.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.193+0000 7fe521e79b80 1 -- 172.21.15.16:0/2269131046 shutdown_connections 2023-05-20T01:27:28.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.193+0000 7fe521e79b80 1 --2- 172.21.15.16:0/2269131046 >> [v2:172.21.15.16:6835/1927078882,v1:172.21.15.16:6837/1927078882] conn(0x55f669fb7450 0x55f669fcb7b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:28.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.193+0000 7fe521e79b80 1 --2- 172.21.15.16:0/2269131046 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f669fb3750 0x55f669fba410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:28.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.193+0000 7fe521e79b80 1 --2- 172.21.15.16:0/2269131046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f669f81f80 0x55f66a031d80 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:28.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.193+0000 7fe4ee7fc700 0 client.8457 ms_handle_reset on v2:172.21.15.16:6835/1927078882 2023-05-20T01:27:28.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.193+0000 7fe521e79b80 1 --2- 172.21.15.16:0/2269131046 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55f669f8a8c0 0x55f669fc86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:28.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.193+0000 7fe521e79b80 1 -- 172.21.15.16:0/2269131046 >> 172.21.15.16:0/2269131046 conn(0x55f669fb7f80 msgr2=0x55f669fe0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:27:28.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.193+0000 7fe521e79b80 1 -- 172.21.15.16:0/2269131046 shutdown_connections 2023-05-20T01:27:28.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.193+0000 7fe521e79b80 1 -- 172.21.15.16:0/2269131046 wait complete. 2023-05-20T01:27:28.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.193+0000 7fe521e79b80 20 client.8457 trim_cache size 0 max 16384 2023-05-20T01:27:28.217 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-05-20T01:27:28.218 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2023-05-20T01:27:28.218 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2023-05-20T01:27:28.218 INFO:teuthology.orchestra.run.smithi016.stdout: "ead1accb-88a4-49bd-9edb-9c6e858f3eb4": { 2023-05-20T01:27:28.218 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2023-05-20T01:27:28.219 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "ead1accb-88a4-49bd-9edb-9c6e858f3eb4", 2023-05-20T01:27:28.219 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,repair" 2023-05-20T01:27:28.219 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:27:28.219 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:27:28.220 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-05-20T01:27:28.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.193+0000 7fe515ced700 1 -- 172.21.15.16:0/2191048470 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe4fc06ace0 msgr2=0x7fe4fc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:28.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.193+0000 7fe515ced700 1 --2- 172.21.15.16:0/2191048470 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe4fc06ace0 0x7fe4fc06d190 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fe51015a730 tx=0x7fe5000091c0 comp rx=0 tx=0).stop 2023-05-20T01:27:28.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.193+0000 7fe515ced700 1 -- 172.21.15.16:0/2191048470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe51015d290 msgr2=0x7fe51015d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:28.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.193+0000 7fe515ced700 1 --2- 172.21.15.16:0/2191048470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe51015d290 0x7fe51015d700 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7fe5100cdb50 tx=0x7fe508004ad0 comp rx=0 tx=0).stop 2023-05-20T01:27:28.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.194+0000 7fe515ced700 1 -- 172.21.15.16:0/2191048470 shutdown_connections 2023-05-20T01:27:28.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.194+0000 7fe515ced700 1 --2- 172.21.15.16:0/2191048470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe51015d290 0x7fe51015d700 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:28.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.194+0000 7fe515ced700 1 --2- 172.21.15.16:0/2191048470 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe4fc06ace0 0x7fe4fc06d190 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:28.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.194+0000 7fe515ced700 1 --2- 172.21.15.16:0/2191048470 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe510072130 0x7fe5101483f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:28.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.194+0000 7fe515ced700 1 --2- 172.21.15.16:0/2191048470 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe51015cec0 0x7fe5100c9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:28.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.194+0000 7fe515ced700 1 -- 172.21.15.16:0/2191048470 >> 172.21.15.16:0/2191048470 conn(0x7fe5100cffd0 msgr2=0x7fe51015f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:27:28.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.194+0000 7fe515ced700 1 -- 172.21.15.16:0/2191048470 shutdown_connections 2023-05-20T01:27:28.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:28.194+0000 7fe515ced700 1 -- 172.21.15.16:0/2191048470 wait complete. 2023-05-20T01:27:28.222 INFO:tasks.cephfs.filesystem:scrub status for tag:ead1accb-88a4-49bd-9edb-9c6e858f3eb4 - {'path': '/', 'tag': 'ead1accb-88a4-49bd-9edb-9c6e858f3eb4', 'options': 'recursive,repair'} 2023-05-20T01:27:37.112 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:27:37.115 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:27:58.224 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.5:0 scrub status 2023-05-20T01:27:58.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.361+0000 7f128c166700 1 Processor -- start 2023-05-20T01:27:58.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.362+0000 7f128c166700 1 -- start start 2023-05-20T01:27:58.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.362+0000 7f128c166700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f128406bd30 0x7f1284167980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:58.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.362+0000 7f128c166700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12840cb700 con 0x7f12840b1e50 2023-05-20T01:27:58.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.362+0000 7f128c166700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12840b2430 con 0x7f128415cd20 2023-05-20T01:27:58.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.362+0000 7f128c166700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12840b2710 con 0x7f128406bd30 2023-05-20T01:27:58.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.362+0000 7f128a703700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f128406bd30 0x7f1284167980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:58.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.362+0000 7f1289701700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f12840b1e50 0x7f128406b620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50862/0 (socket says 172.21.15.16:50862) 2023-05-20T01:27:58.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.362+0000 7f1289701700 1 -- 172.21.15.16:0/1834725377 learned_addr learned my addr 172.21.15.16:0/1834725377 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:27:58.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.363+0000 7f128a703700 1 -- 172.21.15.16:0/1834725377 >> v1:172.21.15.132:6789/0 conn(0x7f128415cd20 legacy=0x7f12840b14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:58.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.363+0000 7f128a703700 1 -- 172.21.15.16:0/1834725377 >> v1:172.21.15.16:6789/0 conn(0x7f12840b1e50 legacy=0x7f128406b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:58.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.363+0000 7f128a703700 1 -- 172.21.15.16:0/1834725377 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12840cb950 con 0x7f128406bd30 2023-05-20T01:27:58.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.363+0000 7f1288f00700 1 -- 172.21.15.16:0/1834725377 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 137533796 0 0) 0x7f12840cb700 con 0x7f12840b1e50 2023-05-20T01:27:58.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.363+0000 7f128a703700 1 --2- 172.21.15.16:0/1834725377 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f128406bd30 0x7f1284167980 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f127c009d90 tx=0x7f127c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd42183587812f49 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:58.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.363+0000 7f1288f00700 1 -- 172.21.15.16:0/1834725377 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f127c00e070 con 0x7f128406bd30 2023-05-20T01:27:58.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.363+0000 7f1288f00700 1 -- 172.21.15.16:0/1834725377 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f127c00bb90 con 0x7f128406bd30 2023-05-20T01:27:58.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.363+0000 7f1288f00700 1 -- 172.21.15.16:0/1834725377 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f127c003a20 con 0x7f128406bd30 2023-05-20T01:27:58.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.364+0000 7f128c166700 1 -- 172.21.15.16:0/1834725377 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f128406bd30 msgr2=0x7f1284167980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:58.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.364+0000 7f128c166700 1 --2- 172.21.15.16:0/1834725377 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f128406bd30 0x7f1284167980 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f127c009d90 tx=0x7f127c00b330 comp rx=0 tx=0).stop 2023-05-20T01:27:58.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.364+0000 7f128c166700 1 -- 172.21.15.16:0/1834725377 shutdown_connections 2023-05-20T01:27:58.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.364+0000 7f128c166700 1 --2- 172.21.15.16:0/1834725377 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f128406bd30 0x7f1284167980 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:58.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.364+0000 7f128c166700 1 -- 172.21.15.16:0/1834725377 >> 172.21.15.16:0/1834725377 conn(0x7f128406ae30 msgr2=0x7f128409a400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:27:58.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.364+0000 7f128c166700 1 -- 172.21.15.16:0/1834725377 shutdown_connections 2023-05-20T01:27:58.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.364+0000 7f128c166700 1 -- 172.21.15.16:0/1834725377 wait complete. 2023-05-20T01:27:58.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.365+0000 7f128c166700 1 Processor -- start 2023-05-20T01:27:58.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.365+0000 7f128c166700 1 -- start start 2023-05-20T01:27:58.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.366+0000 7f128c166700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f128406bd30 0x7f128415b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:58.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.366+0000 7f128c166700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f12840b1e50 0x7f12840cc150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:58.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.366+0000 7f128c166700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f128415cd20 0x7f12840cc690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:58.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.366+0000 7f128c166700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12840b6b10 con 0x7f128406bd30 2023-05-20T01:27:58.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.366+0000 7f128c166700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12840cd160 con 0x7f12840b1e50 2023-05-20T01:27:58.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.366+0000 7f128c166700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f128415dff0 con 0x7f128415cd20 2023-05-20T01:27:58.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.366+0000 7f1289f02700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f128406bd30 0x7f128415b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:58.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.366+0000 7f1289f02700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f128406bd30 0x7f128415b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39340/0 (socket says 172.21.15.16:39340) 2023-05-20T01:27:58.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.366+0000 7f1289f02700 1 -- 172.21.15.16:0/2252741050 learned_addr learned my addr 172.21.15.16:0/2252741050 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:27:58.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.366+0000 7f128a703700 1 --2- 172.21.15.16:0/2252741050 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f128415cd20 0x7f12840cc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:58.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.366+0000 7f1289701700 1 --2- 172.21.15.16:0/2252741050 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f12840b1e50 0x7f12840cc150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:58.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.366+0000 7f1289f02700 1 -- 172.21.15.16:0/2252741050 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f128415cd20 msgr2=0x7f12840cc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:58.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.366+0000 7f1289f02700 1 --2- 172.21.15.16:0/2252741050 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f128415cd20 0x7f12840cc690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:58.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.366+0000 7f1289f02700 1 -- 172.21.15.16:0/2252741050 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f12840b1e50 msgr2=0x7f12840cc150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:58.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.366+0000 7f1289f02700 1 --2- 172.21.15.16:0/2252741050 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f12840b1e50 0x7f12840cc150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:58.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.366+0000 7f1289f02700 1 -- 172.21.15.16:0/2252741050 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f127c009a70 con 0x7f128406bd30 2023-05-20T01:27:58.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.366+0000 7f128a703700 1 --2- 172.21.15.16:0/2252741050 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f128415cd20 0x7f12840cc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:27:58.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.366+0000 7f1289701700 1 --2- 172.21.15.16:0/2252741050 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f12840b1e50 0x7f12840cc150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:27:58.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.366+0000 7f1289f02700 1 --2- 172.21.15.16:0/2252741050 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f128406bd30 0x7f128415b5d0 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f127400c4a0 tx=0x7f127400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:58.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.367+0000 7f127a7fc700 1 -- 172.21.15.16:0/2252741050 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12740054e0 con 0x7f128406bd30 2023-05-20T01:27:58.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.367+0000 7f127a7fc700 1 -- 172.21.15.16:0/2252741050 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1274003680 con 0x7f128406bd30 2023-05-20T01:27:58.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.367+0000 7f128c166700 1 -- 172.21.15.16:0/2252741050 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f128415d0f0 con 0x7f128406bd30 2023-05-20T01:27:58.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.367+0000 7f127a7fc700 1 -- 172.21.15.16:0/2252741050 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1274003a30 con 0x7f128406bd30 2023-05-20T01:27:58.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.367+0000 7f128c166700 1 -- 172.21.15.16:0/2252741050 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12840cbc30 con 0x7f128406bd30 2023-05-20T01:27:58.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.368+0000 7f127a7fc700 1 -- 172.21.15.16:0/2252741050 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1274003b90 con 0x7f128406bd30 2023-05-20T01:27:58.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.369+0000 7f127a7fc700 1 --2- 172.21.15.16:0/2252741050 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f126806ace0 0x7f126806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:58.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.369+0000 7f127a7fc700 1 -- 172.21.15.16:0/2252741050 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 7073+0+0 (secure 0 0 0) 0x7f1274088d80 con 0x7f128406bd30 2023-05-20T01:27:58.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.369+0000 7f1289701700 1 --2- 172.21.15.16:0/2252741050 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f126806ace0 0x7f126806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:58.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.369+0000 7f1289701700 1 --2- 172.21.15.16:0/2252741050 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f126806ace0 0x7f126806d190 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f12840ccd10 tx=0x7f1280009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:58.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.371+0000 7f12982f2b80 1 Processor -- start 2023-05-20T01:27:58.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.371+0000 7f12982f2b80 1 -- start start 2023-05-20T01:27:58.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.372+0000 7f12982f2b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee12ae6450 0x55ee12ae6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:58.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.372+0000 7f12982f2b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ee12ae2750 0x55ee12abb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:58.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.372+0000 7f12982f2b80 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ee12ab98c0 0x55ee12abb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:58.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.372+0000 7f12982f2b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee129c1060 con 0x55ee12ae6450 2023-05-20T01:27:58.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.372+0000 7f12982f2b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee12a138a0 con 0x55ee12ae2750 2023-05-20T01:27:58.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.372+0000 7f12982f2b80 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ee12a0cd30 con 0x55ee12ab98c0 2023-05-20T01:27:58.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.372+0000 7f1289f02700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee12ae6450 0x55ee12ae6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:58.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.372+0000 7f1289f02700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee12ae6450 0x55ee12ae6820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39348/0 (socket says 172.21.15.16:39348) 2023-05-20T01:27:58.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.372+0000 7f1289f02700 1 -- 172.21.15.16:0/4144900576 learned_addr learned my addr 172.21.15.16:0/4144900576 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:27:58.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.372+0000 7f1289701700 1 --2- 172.21.15.16:0/4144900576 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ee12ae2750 0x55ee12abb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:58.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.372+0000 7f128a703700 1 --2- 172.21.15.16:0/4144900576 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ee12ab98c0 0x55ee12abb9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:58.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.372+0000 7f1289f02700 1 -- 172.21.15.16:0/4144900576 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ee12ab98c0 msgr2=0x55ee12abb9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:58.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.372+0000 7f1289f02700 1 --2- 172.21.15.16:0/4144900576 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ee12ab98c0 0x55ee12abb9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:58.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.372+0000 7f1289f02700 1 -- 172.21.15.16:0/4144900576 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ee12ae2750 msgr2=0x55ee12abb4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:58.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.372+0000 7f1289f02700 1 --2- 172.21.15.16:0/4144900576 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ee12ae2750 0x55ee12abb4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:58.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.372+0000 7f1289f02700 1 -- 172.21.15.16:0/4144900576 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ee1288e630 con 0x55ee12ae6450 2023-05-20T01:27:58.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.372+0000 7f1289701700 1 --2- 172.21.15.16:0/4144900576 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ee12ae2750 0x55ee12abb4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:27:58.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.372+0000 7f128a703700 1 --2- 172.21.15.16:0/4144900576 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ee12ab98c0 0x55ee12abb9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:27:58.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.372+0000 7f1289f02700 1 --2- 172.21.15.16:0/4144900576 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee12ae6450 0x55ee12ae6820 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f127400cfd0 tx=0x7f1274003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a36c1f4f915c28b server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:58.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.372+0000 7f12677fe700 1 -- 172.21.15.16:0/4144900576 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1274055b00 con 0x55ee12ae6450 2023-05-20T01:27:58.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.373+0000 7f12677fe700 1 -- 172.21.15.16:0/4144900576 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f127405f620 con 0x55ee12ae6450 2023-05-20T01:27:58.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.373+0000 7f12677fe700 1 -- 172.21.15.16:0/4144900576 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1274058b50 con 0x55ee12ae6450 2023-05-20T01:27:58.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.373+0000 7f12982f2b80 1 -- 172.21.15.16:0/4144900576 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee12ae6450 msgr2=0x55ee12ae6820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:58.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.373+0000 7f12982f2b80 1 --2- 172.21.15.16:0/4144900576 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee12ae6450 0x55ee12ae6820 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f127400cfd0 tx=0x7f1274003f10 comp rx=0 tx=0).stop 2023-05-20T01:27:58.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.373+0000 7f12982f2b80 1 -- 172.21.15.16:0/4144900576 shutdown_connections 2023-05-20T01:27:58.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.373+0000 7f12982f2b80 1 --2- 172.21.15.16:0/4144900576 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ee12ab98c0 0x55ee12abb9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:58.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.373+0000 7f12982f2b80 1 --2- 172.21.15.16:0/4144900576 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee12ae6450 0x55ee12ae6820 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:58.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.373+0000 7f12982f2b80 1 --2- 172.21.15.16:0/4144900576 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ee12ae2750 0x55ee12abb4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:58.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.373+0000 7f12982f2b80 1 -- 172.21.15.16:0/4144900576 >> 172.21.15.16:0/4144900576 conn(0x55ee12ab0f80 msgr2=0x55ee12b01260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:27:58.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.373+0000 7f12982f2b80 1 -- 172.21.15.16:0/4144900576 shutdown_connections 2023-05-20T01:27:58.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.373+0000 7f12982f2b80 1 -- 172.21.15.16:0/4144900576 wait complete. 2023-05-20T01:27:58.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.374+0000 7f12982f2b80 1 -- start start 2023-05-20T01:27:58.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.374+0000 7f12982f2b80 1 Processor -- start 2023-05-20T01:27:58.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.375+0000 7f12982f2b80 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ee12ab98c0 0x55ee12adec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:58.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.375+0000 7f12982f2b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ee12ae2750 0x55ee12aff440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:58.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.375+0000 7f12982f2b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee12ae6450 0x55ee12aff980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:58.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.375+0000 7f12982f2b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee129ae8e0 con 0x55ee12ae6450 2023-05-20T01:27:58.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.375+0000 7f12982f2b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee129ae580 con 0x55ee12ae2750 2023-05-20T01:27:58.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.375+0000 7f12982f2b80 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ee12890920 con 0x55ee12ab98c0 2023-05-20T01:27:58.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.375+0000 7f12982f2b80 10 client.0 _ll_register_callbacks cb 0x55ee129cc780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-05-20T01:27:58.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.375+0000 7f128a703700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee12ae6450 0x55ee12aff980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:58.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.375+0000 7f128a703700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee12ae6450 0x55ee12aff980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39358/0 (socket says 172.21.15.16:39358) 2023-05-20T01:27:58.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.375+0000 7f128a703700 1 -- 172.21.15.16:0/498474744 learned_addr learned my addr 172.21.15.16:0/498474744 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:27:58.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.375+0000 7f1289f02700 1 --2- 172.21.15.16:0/498474744 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ee12ab98c0 0x55ee12adec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:58.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.375+0000 7f1289701700 1 --2- 172.21.15.16:0/498474744 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ee12ae2750 0x55ee12aff440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:58.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.375+0000 7f128a703700 1 -- 172.21.15.16:0/498474744 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ee12ab98c0 msgr2=0x55ee12adec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:58.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.375+0000 7f128a703700 1 --2- 172.21.15.16:0/498474744 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ee12ab98c0 0x55ee12adec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:58.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.375+0000 7f128a703700 1 -- 172.21.15.16:0/498474744 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ee12ae2750 msgr2=0x55ee12aff440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:58.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.375+0000 7f128a703700 1 --2- 172.21.15.16:0/498474744 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ee12ae2750 0x55ee12aff440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:58.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.375+0000 7f128a703700 1 -- 172.21.15.16:0/498474744 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12840cbc30 con 0x55ee12ae6450 2023-05-20T01:27:58.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.376+0000 7f128a703700 1 --2- 172.21.15.16:0/498474744 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee12ae6450 0x55ee12aff980 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f127c004000 tx=0x7f127c004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e90d7bc744405db6 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:58.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.376+0000 7f1264ff9700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-05-20T01:27:58.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.376+0000 7f1264ff9700 1 -- 172.21.15.16:0/498474744 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f127c004560 con 0x55ee12ae6450 2023-05-20T01:27:58.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.376+0000 7f1264ff9700 1 -- 172.21.15.16:0/498474744 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(123..123 src has 1..123) v4 ==== 7073+0+0 (secure 0 0 0) 0x7f127c0046c0 con 0x55ee12ae6450 2023-05-20T01:27:58.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.376+0000 7f12982f2b80 1 -- 172.21.15.16:0/498474744 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ee12aae740 con 0x55ee12ae6450 2023-05-20T01:27:58.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.376+0000 7f1264ff9700 1 -- 172.21.15.16:0/498474744 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f127c004880 con 0x55ee12ae6450 2023-05-20T01:27:58.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.376+0000 7f12982f2b80 10 client.8472 fetch_fsmap learned FSMap version 91 2023-05-20T01:27:58.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.376+0000 7f12982f2b80 1 -- 172.21.15.16:0/498474744 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) v3 -- 0x55ee129d3190 con 0x55ee12ae6450 2023-05-20T01:27:58.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.376+0000 7f1264ff9700 1 -- 172.21.15.16:0/498474744 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ee12aae740 con 0x55ee12ae6450 2023-05-20T01:27:58.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.376+0000 7f1264ff9700 1 -- 172.21.15.16:0/498474744 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 91) v1 ==== 3122+0+0 (secure 0 0 0) 0x7f127c00b050 con 0x55ee12ae6450 2023-05-20T01:27:58.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.376+0000 7f12982f2b80 10 client.8472 fetch_fsmap finished waiting for FSMap version 91 2023-05-20T01:27:58.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.376+0000 7f12982f2b80 10 client.8472 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2023-05-20T01:27:58.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.376+0000 7f12982f2b80 20 client.8472 populate_metadata read hostname 'smithi016' 2023-05-20T01:27:58.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.376+0000 7f12982f2b80 1 --2- 172.21.15.16:0/498474744 >> [v2:172.21.15.16:6835/1927078882,v1:172.21.15.16:6837/1927078882] conn(0x55ee12abbe00 0x55ee12ada920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:58.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.376+0000 7f12982f2b80 4 client.8472 mds_command: new command op to 8376 tid=0[{"prefix": "get_command_descriptions"}] 2023-05-20T01:27:58.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.376+0000 7f12982f2b80 1 -- 172.21.15.16:0/498474744 --> [v2:172.21.15.16:6835/1927078882,v1:172.21.15.16:6837/1927078882] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ee129cf0a0 con 0x55ee12abbe00 2023-05-20T01:27:58.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.376+0000 7f1289f02700 1 --2- 172.21.15.16:0/498474744 >> [v2:172.21.15.16:6835/1927078882,v1:172.21.15.16:6837/1927078882] conn(0x55ee12abbe00 0x55ee12ada920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:58.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.377+0000 7f1289f02700 1 --2- 172.21.15.16:0/498474744 >> [v2:172.21.15.16:6835/1927078882,v1:172.21.15.16:6837/1927078882] conn(0x55ee12abbe00 0x55ee12ada920 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d7ab82291b4ec13 server_cookie=6273c60fb3d2dc36 in_seq=0 out_seq=0 2023-05-20T01:27:58.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.377+0000 7f1264ff9700 10 client.8472 ms_handle_connect on v2:172.21.15.16:6835/1927078882 2023-05-20T01:27:58.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.377+0000 7f1264ff9700 1 -- 172.21.15.16:0/498474744 <== mds.0 v2:172.21.15.16:6835/1927078882 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ee129cf0a0 con 0x55ee12abbe00 2023-05-20T01:27:58.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.377+0000 7f1264ff9700 10 client.8472 handle_command_reply: tid=0 2023-05-20T01:27:58.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.377+0000 7f12982f2b80 1 client.8472 shutdown 2023-05-20T01:27:58.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.377+0000 7f12982f2b80 1 -- 172.21.15.16:0/498474744 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee12ae6450 msgr2=0x55ee12aff980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:58.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.377+0000 7f12982f2b80 1 --2- 172.21.15.16:0/498474744 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee12ae6450 0x55ee12aff980 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f127c004000 tx=0x7f127c004b40 comp rx=0 tx=0).stop 2023-05-20T01:27:58.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.377+0000 7f12982f2b80 1 -- 172.21.15.16:0/498474744 shutdown_connections 2023-05-20T01:27:58.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.377+0000 7f12982f2b80 1 --2- 172.21.15.16:0/498474744 >> [v2:172.21.15.16:6835/1927078882,v1:172.21.15.16:6837/1927078882] conn(0x55ee12abbe00 0x55ee12ada920 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:58.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.377+0000 7f12982f2b80 1 --2- 172.21.15.16:0/498474744 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee12ae6450 0x55ee12aff980 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:58.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.377+0000 7f12982f2b80 1 --2- 172.21.15.16:0/498474744 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ee12ab98c0 0x55ee12adec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:58.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.377+0000 7f1264ff9700 0 client.8472 ms_handle_reset on v2:172.21.15.16:6835/1927078882 2023-05-20T01:27:58.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.377+0000 7f12982f2b80 1 --2- 172.21.15.16:0/498474744 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ee12ae2750 0x55ee12aff440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:58.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.377+0000 7f12982f2b80 1 -- 172.21.15.16:0/498474744 >> 172.21.15.16:0/498474744 conn(0x55ee12ab0f80 msgr2=0x55ee12ab6430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:27:58.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.378+0000 7f12982f2b80 1 -- 172.21.15.16:0/498474744 shutdown_connections 2023-05-20T01:27:58.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.378+0000 7f12982f2b80 1 -- 172.21.15.16:0/498474744 wait complete. 2023-05-20T01:27:58.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.378+0000 7f12982f2b80 20 client.8472 trim_cache size 0 max 16384 2023-05-20T01:27:58.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.395+0000 7f12982f2b80 1 Processor -- start 2023-05-20T01:27:58.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.395+0000 7f12982f2b80 1 -- start start 2023-05-20T01:27:58.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.396+0000 7f12982f2b80 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ee12ab98c0 0x55ee12b01680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:58.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.396+0000 7f12982f2b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ee12ae2750 0x55ee12ab5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:58.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.396+0000 7f12982f2b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee12ae6450 0x55ee12ab6350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:58.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.396+0000 7f12982f2b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee12afb4b0 con 0x55ee12ae6450 2023-05-20T01:27:58.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.396+0000 7f12982f2b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee12afaf50 con 0x55ee12ae2750 2023-05-20T01:27:58.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.396+0000 7f12982f2b80 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ee12adddc0 con 0x55ee12ab98c0 2023-05-20T01:27:58.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.396+0000 7f128a703700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee12ae6450 0x55ee12ab6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:58.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.396+0000 7f128a703700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee12ae6450 0x55ee12ab6350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39366/0 (socket says 172.21.15.16:39366) 2023-05-20T01:27:58.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.396+0000 7f128a703700 1 -- 172.21.15.16:0/3845287198 learned_addr learned my addr 172.21.15.16:0/3845287198 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:27:58.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.396+0000 7f1289f02700 1 --2- 172.21.15.16:0/3845287198 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ee12ab98c0 0x55ee12b01680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:58.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.396+0000 7f1289701700 1 --2- 172.21.15.16:0/3845287198 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ee12ae2750 0x55ee12ab5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:58.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.396+0000 7f128a703700 1 -- 172.21.15.16:0/3845287198 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ee12ab98c0 msgr2=0x55ee12b01680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:58.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.396+0000 7f128a703700 1 --2- 172.21.15.16:0/3845287198 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ee12ab98c0 0x55ee12b01680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:58.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.396+0000 7f128a703700 1 -- 172.21.15.16:0/3845287198 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ee12ae2750 msgr2=0x55ee12ab5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:58.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.396+0000 7f128a703700 1 --2- 172.21.15.16:0/3845287198 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ee12ae2750 0x55ee12ab5e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:58.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.396+0000 7f128a703700 1 -- 172.21.15.16:0/3845287198 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ee129d2ee0 con 0x55ee12ae6450 2023-05-20T01:27:58.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.396+0000 7f128a703700 1 --2- 172.21.15.16:0/3845287198 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee12ae6450 0x55ee12ab6350 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f127c00d010 tx=0x7f127c01f070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83df98e209e02a24 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:58.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.396+0000 7f1267fff700 1 -- 172.21.15.16:0/3845287198 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f127c034070 con 0x55ee12ae6450 2023-05-20T01:27:58.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.396+0000 7f1267fff700 1 -- 172.21.15.16:0/3845287198 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f127c021070 con 0x55ee12ae6450 2023-05-20T01:27:58.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.396+0000 7f1267fff700 1 -- 172.21.15.16:0/3845287198 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f127c004440 con 0x55ee12ae6450 2023-05-20T01:27:58.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.397+0000 7f12982f2b80 1 -- 172.21.15.16:0/3845287198 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee12ae6450 msgr2=0x55ee12ab6350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:58.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.397+0000 7f12982f2b80 1 --2- 172.21.15.16:0/3845287198 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee12ae6450 0x55ee12ab6350 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f127c00d010 tx=0x7f127c01f070 comp rx=0 tx=0).stop 2023-05-20T01:27:58.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.397+0000 7f12982f2b80 1 -- 172.21.15.16:0/3845287198 shutdown_connections 2023-05-20T01:27:58.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.397+0000 7f12982f2b80 1 --2- 172.21.15.16:0/3845287198 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee12ae6450 0x55ee12ab6350 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:58.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.397+0000 7f12982f2b80 1 --2- 172.21.15.16:0/3845287198 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ee12ab98c0 0x55ee12b01680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:58.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.397+0000 7f12982f2b80 1 --2- 172.21.15.16:0/3845287198 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ee12ae2750 0x55ee12ab5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:58.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.397+0000 7f12982f2b80 1 -- 172.21.15.16:0/3845287198 >> 172.21.15.16:0/3845287198 conn(0x55ee12ab0f80 msgr2=0x55ee12abb290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:27:58.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.397+0000 7f12982f2b80 1 -- 172.21.15.16:0/3845287198 shutdown_connections 2023-05-20T01:27:58.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.397+0000 7f12982f2b80 1 -- 172.21.15.16:0/3845287198 wait complete. 2023-05-20T01:27:58.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.398+0000 7f12982f2b80 1 -- start start 2023-05-20T01:27:58.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.398+0000 7f12982f2b80 1 Processor -- start 2023-05-20T01:27:58.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.398+0000 7f12982f2b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee12ab98c0 0x55ee12b677f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:58.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.398+0000 7f12982f2b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ee12ae2750 0x55ee12abc250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:58.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.398+0000 7f12982f2b80 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ee12ae6450 0x55ee12abc790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:58.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.399+0000 7f12982f2b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee12ae1f20 con 0x55ee12ab98c0 2023-05-20T01:27:58.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.399+0000 7f12982f2b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee12ab6850 con 0x55ee12ae2750 2023-05-20T01:27:58.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.399+0000 7f12982f2b80 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ee12b692e0 con 0x55ee12ae6450 2023-05-20T01:27:58.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.399+0000 7f12982f2b80 10 client.0 _ll_register_callbacks cb 0x7f127c0094b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-05-20T01:27:58.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.399+0000 7f1289f02700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee12ab98c0 0x55ee12b677f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:58.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.399+0000 7f1289f02700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee12ab98c0 0x55ee12b677f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39368/0 (socket says 172.21.15.16:39368) 2023-05-20T01:27:58.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.399+0000 7f1289f02700 1 -- 172.21.15.16:0/2733109003 learned_addr learned my addr 172.21.15.16:0/2733109003 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:27:58.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.399+0000 7f128a703700 1 --2- 172.21.15.16:0/2733109003 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ee12ae6450 0x55ee12abc790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:58.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.399+0000 7f1289701700 1 --2- 172.21.15.16:0/2733109003 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ee12ae2750 0x55ee12abc250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:58.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.399+0000 7f1289f02700 1 -- 172.21.15.16:0/2733109003 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ee12ae6450 msgr2=0x55ee12abc790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:58.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.399+0000 7f1289f02700 1 --2- 172.21.15.16:0/2733109003 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ee12ae6450 0x55ee12abc790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:58.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.399+0000 7f1289f02700 1 -- 172.21.15.16:0/2733109003 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ee12ae2750 msgr2=0x55ee12abc250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:27:58.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.399+0000 7f1289f02700 1 --2- 172.21.15.16:0/2733109003 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ee12ae2750 0x55ee12abc250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:58.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.399+0000 7f1289f02700 1 -- 172.21.15.16:0/2733109003 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ee129d3190 con 0x55ee12ab98c0 2023-05-20T01:27:58.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.399+0000 7f1289f02700 1 --2- 172.21.15.16:0/2733109003 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee12ab98c0 0x55ee12b677f0 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f1274057460 tx=0x7f127405b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf0a5cc7a67c4d07 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:27:58.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.399+0000 7f1264ff9700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-05-20T01:27:58.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.399+0000 7f1264ff9700 1 -- 172.21.15.16:0/2733109003 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1274070070 con 0x55ee12ab98c0 2023-05-20T01:27:58.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.399+0000 7f1264ff9700 1 -- 172.21.15.16:0/2733109003 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(123..123 src has 1..123) v4 ==== 7073+0+0 (secure 0 0 0) 0x7f1274055bc0 con 0x55ee12ab98c0 2023-05-20T01:27:58.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.399+0000 7f12982f2b80 1 -- 172.21.15.16:0/2733109003 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f127405f620 con 0x55ee12ab98c0 2023-05-20T01:27:58.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.399+0000 7f1264ff9700 1 -- 172.21.15.16:0/2733109003 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1274074780 con 0x55ee12ab98c0 2023-05-20T01:27:58.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.399+0000 7f12982f2b80 10 client.8478 fetch_fsmap learned FSMap version 91 2023-05-20T01:27:58.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.399+0000 7f12982f2b80 1 -- 172.21.15.16:0/2733109003 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) v3 -- 0x55ee12823c60 con 0x55ee12ab98c0 2023-05-20T01:27:58.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.400+0000 7f1264ff9700 1 -- 172.21.15.16:0/2733109003 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f127405f620 con 0x55ee12ab98c0 2023-05-20T01:27:58.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.400+0000 7f1264ff9700 1 -- 172.21.15.16:0/2733109003 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 91) v1 ==== 3122+0+0 (secure 0 0 0) 0x7f12740742e0 con 0x55ee12ab98c0 2023-05-20T01:27:58.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.400+0000 7f12982f2b80 10 client.8478 fetch_fsmap finished waiting for FSMap version 91 2023-05-20T01:27:58.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.400+0000 7f12982f2b80 10 client.8478 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2023-05-20T01:27:58.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.400+0000 7f12982f2b80 20 client.8478 populate_metadata read hostname 'smithi016' 2023-05-20T01:27:58.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.400+0000 7f12982f2b80 1 --2- 172.21.15.16:0/2733109003 >> [v2:172.21.15.16:6835/1927078882,v1:172.21.15.16:6837/1927078882] conn(0x55ee12ad8e50 0x55ee12adc930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:27:58.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.400+0000 7f12982f2b80 4 client.8478 mds_command: new command op to 8376 tid=0[{"prefix": "scrub status"}] 2023-05-20T01:27:58.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.400+0000 7f12982f2b80 1 -- 172.21.15.16:0/2733109003 --> [v2:172.21.15.16:6835/1927078882,v1:172.21.15.16:6837/1927078882] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ee129db660 con 0x55ee12ad8e50 2023-05-20T01:27:58.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.400+0000 7f1289701700 1 --2- 172.21.15.16:0/2733109003 >> [v2:172.21.15.16:6835/1927078882,v1:172.21.15.16:6837/1927078882] conn(0x55ee12ad8e50 0x55ee12adc930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:27:58.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.400+0000 7f1289701700 1 --2- 172.21.15.16:0/2733109003 >> [v2:172.21.15.16:6835/1927078882,v1:172.21.15.16:6837/1927078882] conn(0x55ee12ad8e50 0x55ee12adc930 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6919bdc68d92ba1 server_cookie=de791840c66dba7b in_seq=0 out_seq=0 2023-05-20T01:27:58.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.400+0000 7f1264ff9700 10 client.8478 ms_handle_connect on v2:172.21.15.16:6835/1927078882 2023-05-20T01:27:58.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.400+0000 7f1264ff9700 1 -- 172.21.15.16:0/2733109003 <== mds.0 v2:172.21.15.16:6835/1927078882 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55ee129db660 con 0x55ee12ad8e50 2023-05-20T01:27:58.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.400+0000 7f1264ff9700 10 client.8478 handle_command_reply: tid=0 2023-05-20T01:27:58.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.400+0000 7f12982f2b80 1 client.8478 shutdown 2023-05-20T01:27:58.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.400+0000 7f12982f2b80 1 -- 172.21.15.16:0/2733109003 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee12ab98c0 msgr2=0x55ee12b677f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:58.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.400+0000 7f12982f2b80 1 --2- 172.21.15.16:0/2733109003 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee12ab98c0 0x55ee12b677f0 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f1274057460 tx=0x7f127405b040 comp rx=0 tx=0).stop 2023-05-20T01:27:58.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.400+0000 7f12982f2b80 1 -- 172.21.15.16:0/2733109003 shutdown_connections 2023-05-20T01:27:58.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.400+0000 7f12982f2b80 1 --2- 172.21.15.16:0/2733109003 >> [v2:172.21.15.16:6835/1927078882,v1:172.21.15.16:6837/1927078882] conn(0x55ee12ad8e50 0x55ee12adc930 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:58.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.400+0000 7f12982f2b80 1 --2- 172.21.15.16:0/2733109003 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ee12ae6450 0x55ee12abc790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:58.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.400+0000 7f12982f2b80 1 --2- 172.21.15.16:0/2733109003 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee12ab98c0 0x55ee12b677f0 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:58.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.400+0000 7f1264ff9700 0 client.8478 ms_handle_reset on v2:172.21.15.16:6835/1927078882 2023-05-20T01:27:58.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.400+0000 7f12982f2b80 1 --2- 172.21.15.16:0/2733109003 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ee12ae2750 0x55ee12abc250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:58.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.400+0000 7f12982f2b80 1 -- 172.21.15.16:0/2733109003 >> 172.21.15.16:0/2733109003 conn(0x55ee12ab0f80 msgr2=0x55ee12ab5200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:27:58.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.401+0000 7f12982f2b80 1 -- 172.21.15.16:0/2733109003 shutdown_connections 2023-05-20T01:27:58.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.401+0000 7f12982f2b80 1 -- 172.21.15.16:0/2733109003 wait complete. 2023-05-20T01:27:58.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.401+0000 7f12982f2b80 20 client.8478 trim_cache size 0 max 16384 2023-05-20T01:27:58.430 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-05-20T01:27:58.430 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2023-05-20T01:27:58.430 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2023-05-20T01:27:58.431 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-05-20T01:27:58.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.401+0000 7f128c166700 1 -- 172.21.15.16:0/2252741050 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f126806ace0 msgr2=0x7f126806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:58.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.401+0000 7f128c166700 1 --2- 172.21.15.16:0/2252741050 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f126806ace0 0x7f126806d190 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f12840ccd10 tx=0x7f1280009040 comp rx=0 tx=0).stop 2023-05-20T01:27:58.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.401+0000 7f128c166700 1 -- 172.21.15.16:0/2252741050 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f128406bd30 msgr2=0x7f128415b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:27:58.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.401+0000 7f128c166700 1 --2- 172.21.15.16:0/2252741050 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f128406bd30 0x7f128415b5d0 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f127400c4a0 tx=0x7f127400c860 comp rx=0 tx=0).stop 2023-05-20T01:27:58.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.401+0000 7f128c166700 1 -- 172.21.15.16:0/2252741050 shutdown_connections 2023-05-20T01:27:58.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.401+0000 7f128c166700 1 --2- 172.21.15.16:0/2252741050 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f126806ace0 0x7f126806d190 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:58.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.401+0000 7f128c166700 1 --2- 172.21.15.16:0/2252741050 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f128415cd20 0x7f12840cc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:58.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.401+0000 7f128c166700 1 --2- 172.21.15.16:0/2252741050 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f128406bd30 0x7f128415b5d0 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:58.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.401+0000 7f128c166700 1 --2- 172.21.15.16:0/2252741050 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f12840b1e50 0x7f12840cc150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:27:58.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.401+0000 7f128c166700 1 -- 172.21.15.16:0/2252741050 >> 172.21.15.16:0/2252741050 conn(0x7f128406ae30 msgr2=0x7f128409a400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:27:58.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.401+0000 7f128c166700 1 -- 172.21.15.16:0/2252741050 shutdown_connections 2023-05-20T01:27:58.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:27:58.401+0000 7f128c166700 1 -- 172.21.15.16:0/2252741050 wait complete. 2023-05-20T01:27:58.436 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-05-20T01:27:58.437 INFO:tasks.cephfs.mount:Validating 1048576 bytes from stripey/flushed_file_0 2023-05-20T01:27:58.438 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-05-20T01:27:58.438 DEBUG:teuthology.orchestra.run.smithi016:> import zlib 2023-05-20T01:27:58.438 DEBUG:teuthology.orchestra.run.smithi016:> path = "/home/ubuntu/cephtest/mnt.0/stripey/flushed_file_0" 2023-05-20T01:27:58.439 DEBUG:teuthology.orchestra.run.smithi016:> with open(path, '"'"'r'"'"') as f: 2023-05-20T01:27:58.439 DEBUG:teuthology.orchestra.run.smithi016:> bytes = f.read() 2023-05-20T01:27:58.439 DEBUG:teuthology.orchestra.run.smithi016:> if len(bytes) != 1048576: 2023-05-20T01:27:58.439 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-05-20T01:27:58.440 DEBUG:teuthology.orchestra.run.smithi016:> len(bytes), 1048576 2023-05-20T01:27:58.440 DEBUG:teuthology.orchestra.run.smithi016:> )) 2023-05-20T01:27:58.440 DEBUG:teuthology.orchestra.run.smithi016:> for i, b in enumerate(bytes): 2023-05-20T01:27:58.441 DEBUG:teuthology.orchestra.run.smithi016:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-05-20T01:27:58.441 DEBUG:teuthology.orchestra.run.smithi016:> if b != chr(val): 2023-05-20T01:27:58.441 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-05-20T01:27:58.441 DEBUG:teuthology.orchestra.run.smithi016:> ' 2023-05-20T01:27:59.308 INFO:tasks.cephfs.mount:Validating 524288 bytes from stripey/flushed_file_1 2023-05-20T01:27:59.309 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-05-20T01:27:59.309 DEBUG:teuthology.orchestra.run.smithi016:> import zlib 2023-05-20T01:27:59.309 DEBUG:teuthology.orchestra.run.smithi016:> path = "/home/ubuntu/cephtest/mnt.0/stripey/flushed_file_1" 2023-05-20T01:27:59.310 DEBUG:teuthology.orchestra.run.smithi016:> with open(path, '"'"'r'"'"') as f: 2023-05-20T01:27:59.310 DEBUG:teuthology.orchestra.run.smithi016:> bytes = f.read() 2023-05-20T01:27:59.310 DEBUG:teuthology.orchestra.run.smithi016:> if len(bytes) != 524288: 2023-05-20T01:27:59.310 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-05-20T01:27:59.311 DEBUG:teuthology.orchestra.run.smithi016:> len(bytes), 524288 2023-05-20T01:27:59.311 DEBUG:teuthology.orchestra.run.smithi016:> )) 2023-05-20T01:27:59.311 DEBUG:teuthology.orchestra.run.smithi016:> for i, b in enumerate(bytes): 2023-05-20T01:27:59.311 DEBUG:teuthology.orchestra.run.smithi016:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-05-20T01:27:59.312 DEBUG:teuthology.orchestra.run.smithi016:> if b != chr(val): 2023-05-20T01:27:59.312 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-05-20T01:27:59.312 DEBUG:teuthology.orchestra.run.smithi016:> ' 2023-05-20T01:28:00.016 INFO:tasks.cephfs.mount:Validating 917504 bytes from stripey/flushed_file_2 2023-05-20T01:28:00.016 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-05-20T01:28:00.017 DEBUG:teuthology.orchestra.run.smithi016:> import zlib 2023-05-20T01:28:00.017 DEBUG:teuthology.orchestra.run.smithi016:> path = "/home/ubuntu/cephtest/mnt.0/stripey/flushed_file_2" 2023-05-20T01:28:00.017 DEBUG:teuthology.orchestra.run.smithi016:> with open(path, '"'"'r'"'"') as f: 2023-05-20T01:28:00.018 DEBUG:teuthology.orchestra.run.smithi016:> bytes = f.read() 2023-05-20T01:28:00.018 DEBUG:teuthology.orchestra.run.smithi016:> if len(bytes) != 917504: 2023-05-20T01:28:00.018 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-05-20T01:28:00.018 DEBUG:teuthology.orchestra.run.smithi016:> len(bytes), 917504 2023-05-20T01:28:00.019 DEBUG:teuthology.orchestra.run.smithi016:> )) 2023-05-20T01:28:00.020 DEBUG:teuthology.orchestra.run.smithi016:> for i, b in enumerate(bytes): 2023-05-20T01:28:00.021 DEBUG:teuthology.orchestra.run.smithi016:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-05-20T01:28:00.022 DEBUG:teuthology.orchestra.run.smithi016:> if b != chr(val): 2023-05-20T01:28:00.023 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-05-20T01:28:00.023 DEBUG:teuthology.orchestra.run.smithi016:> ' 2023-05-20T01:28:00.925 INFO:tasks.cephfs.mount:Validating 917503 bytes from stripey/flushed_file_3 2023-05-20T01:28:00.926 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-05-20T01:28:00.926 DEBUG:teuthology.orchestra.run.smithi016:> import zlib 2023-05-20T01:28:00.926 DEBUG:teuthology.orchestra.run.smithi016:> path = "/home/ubuntu/cephtest/mnt.0/stripey/flushed_file_3" 2023-05-20T01:28:00.927 DEBUG:teuthology.orchestra.run.smithi016:> with open(path, '"'"'r'"'"') as f: 2023-05-20T01:28:00.927 DEBUG:teuthology.orchestra.run.smithi016:> bytes = f.read() 2023-05-20T01:28:00.927 DEBUG:teuthology.orchestra.run.smithi016:> if len(bytes) != 917503: 2023-05-20T01:28:00.928 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-05-20T01:28:00.928 DEBUG:teuthology.orchestra.run.smithi016:> len(bytes), 917503 2023-05-20T01:28:00.929 DEBUG:teuthology.orchestra.run.smithi016:> )) 2023-05-20T01:28:00.929 DEBUG:teuthology.orchestra.run.smithi016:> for i, b in enumerate(bytes): 2023-05-20T01:28:00.929 DEBUG:teuthology.orchestra.run.smithi016:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-05-20T01:28:00.929 DEBUG:teuthology.orchestra.run.smithi016:> if b != chr(val): 2023-05-20T01:28:00.930 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-05-20T01:28:00.930 DEBUG:teuthology.orchestra.run.smithi016:> ' 2023-05-20T01:28:01.834 INFO:tasks.cephfs.mount:Validating 1441792 bytes from stripey/flushed_file_4 2023-05-20T01:28:01.834 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-05-20T01:28:01.834 DEBUG:teuthology.orchestra.run.smithi016:> import zlib 2023-05-20T01:28:01.835 DEBUG:teuthology.orchestra.run.smithi016:> path = "/home/ubuntu/cephtest/mnt.0/stripey/flushed_file_4" 2023-05-20T01:28:01.835 DEBUG:teuthology.orchestra.run.smithi016:> with open(path, '"'"'r'"'"') as f: 2023-05-20T01:28:01.835 DEBUG:teuthology.orchestra.run.smithi016:> bytes = f.read() 2023-05-20T01:28:01.836 DEBUG:teuthology.orchestra.run.smithi016:> if len(bytes) != 1441792: 2023-05-20T01:28:01.836 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-05-20T01:28:01.836 DEBUG:teuthology.orchestra.run.smithi016:> len(bytes), 1441792 2023-05-20T01:28:01.837 DEBUG:teuthology.orchestra.run.smithi016:> )) 2023-05-20T01:28:01.837 DEBUG:teuthology.orchestra.run.smithi016:> for i, b in enumerate(bytes): 2023-05-20T01:28:01.837 DEBUG:teuthology.orchestra.run.smithi016:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-05-20T01:28:01.838 DEBUG:teuthology.orchestra.run.smithi016:> if b != chr(val): 2023-05-20T01:28:01.838 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-05-20T01:28:01.838 DEBUG:teuthology.orchestra.run.smithi016:> ' 2023-05-20T01:28:03.143 INFO:tasks.cephfs.mount:Validating 1441793 bytes from stripey/flushed_file_5 2023-05-20T01:28:03.143 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-05-20T01:28:03.143 DEBUG:teuthology.orchestra.run.smithi016:> import zlib 2023-05-20T01:28:03.144 DEBUG:teuthology.orchestra.run.smithi016:> path = "/home/ubuntu/cephtest/mnt.0/stripey/flushed_file_5" 2023-05-20T01:28:03.144 DEBUG:teuthology.orchestra.run.smithi016:> with open(path, '"'"'r'"'"') as f: 2023-05-20T01:28:03.144 DEBUG:teuthology.orchestra.run.smithi016:> bytes = f.read() 2023-05-20T01:28:03.145 DEBUG:teuthology.orchestra.run.smithi016:> if len(bytes) != 1441793: 2023-05-20T01:28:03.145 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-05-20T01:28:03.146 DEBUG:teuthology.orchestra.run.smithi016:> len(bytes), 1441793 2023-05-20T01:28:03.146 DEBUG:teuthology.orchestra.run.smithi016:> )) 2023-05-20T01:28:03.146 DEBUG:teuthology.orchestra.run.smithi016:> for i, b in enumerate(bytes): 2023-05-20T01:28:03.147 DEBUG:teuthology.orchestra.run.smithi016:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-05-20T01:28:03.147 DEBUG:teuthology.orchestra.run.smithi016:> if b != chr(val): 2023-05-20T01:28:03.147 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-05-20T01:28:03.147 DEBUG:teuthology.orchestra.run.smithi016:> ' 2023-05-20T01:28:04.452 INFO:tasks.cephfs.mount:Validating 1572864 bytes from stripey/flushed_file_6 2023-05-20T01:28:04.452 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-05-20T01:28:04.453 DEBUG:teuthology.orchestra.run.smithi016:> import zlib 2023-05-20T01:28:04.453 DEBUG:teuthology.orchestra.run.smithi016:> path = "/home/ubuntu/cephtest/mnt.0/stripey/flushed_file_6" 2023-05-20T01:28:04.453 DEBUG:teuthology.orchestra.run.smithi016:> with open(path, '"'"'r'"'"') as f: 2023-05-20T01:28:04.453 DEBUG:teuthology.orchestra.run.smithi016:> bytes = f.read() 2023-05-20T01:28:04.454 DEBUG:teuthology.orchestra.run.smithi016:> if len(bytes) != 1572864: 2023-05-20T01:28:04.454 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-05-20T01:28:04.454 DEBUG:teuthology.orchestra.run.smithi016:> len(bytes), 1572864 2023-05-20T01:28:04.455 DEBUG:teuthology.orchestra.run.smithi016:> )) 2023-05-20T01:28:04.455 DEBUG:teuthology.orchestra.run.smithi016:> for i, b in enumerate(bytes): 2023-05-20T01:28:04.455 DEBUG:teuthology.orchestra.run.smithi016:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-05-20T01:28:04.455 DEBUG:teuthology.orchestra.run.smithi016:> if b != chr(val): 2023-05-20T01:28:04.456 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-05-20T01:28:04.456 DEBUG:teuthology.orchestra.run.smithi016:> ' 2023-05-20T01:28:05.762 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:28:05.763 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo ls lost+found) 2023-05-20T01:28:05.847 INFO:teuthology.orchestra.run.smithi016.stdout:100000001fc 2023-05-20T01:28:05.852 INFO:tasks.cephfs.mount:Validating 524288 bytes from lost+found/100000001fc 2023-05-20T01:28:05.853 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-05-20T01:28:05.853 DEBUG:teuthology.orchestra.run.smithi016:> import zlib 2023-05-20T01:28:05.854 DEBUG:teuthology.orchestra.run.smithi016:> path = "/home/ubuntu/cephtest/mnt.0/lost+found/100000001fc" 2023-05-20T01:28:05.854 DEBUG:teuthology.orchestra.run.smithi016:> with open(path, '"'"'r'"'"') as f: 2023-05-20T01:28:05.854 DEBUG:teuthology.orchestra.run.smithi016:> bytes = f.read() 2023-05-20T01:28:05.855 DEBUG:teuthology.orchestra.run.smithi016:> if len(bytes) != 524288: 2023-05-20T01:28:05.855 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-05-20T01:28:05.855 DEBUG:teuthology.orchestra.run.smithi016:> len(bytes), 524288 2023-05-20T01:28:05.856 DEBUG:teuthology.orchestra.run.smithi016:> )) 2023-05-20T01:28:05.856 DEBUG:teuthology.orchestra.run.smithi016:> for i, b in enumerate(bytes): 2023-05-20T01:28:05.856 DEBUG:teuthology.orchestra.run.smithi016:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-05-20T01:28:05.857 DEBUG:teuthology.orchestra.run.smithi016:> if b != chr(val): 2023-05-20T01:28:05.857 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-05-20T01:28:05.858 DEBUG:teuthology.orchestra.run.smithi016:> ' 2023-05-20T01:28:06.009 INFO:teuthology.orchestra.run.smithi016.stderr:Traceback (most recent call last): 2023-05-20T01:28:06.010 INFO:teuthology.orchestra.run.smithi016.stderr: File "", line 13, in 2023-05-20T01:28:06.010 INFO:teuthology.orchestra.run.smithi016.stderr:RuntimeError: Bad data at offset 65536 2023-05-20T01:28:06.118 INFO:teuthology.orchestra.run.smithi016.stderr:daemon-helper: command failed with exit status 1 2023-05-20T01:28:06.124 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-05-20T01:28:06.125 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-05-20T01:28:06.126 DEBUG:teuthology.orchestra.run.smithi016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-05-20T01:28:06.208 DEBUG:teuthology.orchestra.run.smithi132:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-05-20T01:28:06.259 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-05-20T01:28:06.259 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-05-20T01:28:06.259 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2023-05-20T01:28:06.277 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-05-20T01:28:06.277 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2023-05-20T01:28:06.335 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-20T01:28:06.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.495+0000 7f62d5578700 1 Processor -- start 2023-05-20T01:28:06.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.495+0000 7f62d5578700 1 -- start start 2023-05-20T01:28:06.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.496+0000 7f62d5578700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f62d006cb90 0x7f62d015ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:06.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.496+0000 7f62d5578700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f62d00b54f0 0x7f62d00b29e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:06.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.496+0000 7f62d5578700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62d00b2f20 con 0x7f62d006cf60 2023-05-20T01:28:06.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.496+0000 7f62d5578700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62d00cc990 con 0x7f62d006cb90 2023-05-20T01:28:06.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.496+0000 7f62d5578700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62d00b31a0 con 0x7f62d00b54f0 2023-05-20T01:28:06.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.496+0000 7f62cf7fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f62d006cf60 0x7f62d0070910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50924/0 (socket says 172.21.15.16:50924) 2023-05-20T01:28:06.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.496+0000 7f62cf7fe700 1 -- 172.21.15.16:0/2847503621 learned_addr learned my addr 172.21.15.16:0/2847503621 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:28:06.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.496+0000 7f62ce7fc700 1 --2- 172.21.15.16:0/2847503621 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f62d00b54f0 0x7f62d00b29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:06.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.496+0000 7f62ceffd700 1 --2- 172.21.15.16:0/2847503621 >> v2:172.21.15.132:3300/0 conn(0x7f62d006cb90 0x7f62d015ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:06.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.497+0000 7f62cdffb700 1 -- 172.21.15.16:0/2847503621 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2089501251 0 0) 0x7f62d00b2f20 con 0x7f62d006cf60 2023-05-20T01:28:06.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.497+0000 7f62cdffb700 1 -- 172.21.15.16:0/2847503621 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f62b80035b0 con 0x7f62d006cf60 2023-05-20T01:28:06.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.497+0000 7f62cdffb700 1 -- 172.21.15.16:0/2847503621 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 269247236 0 0) 0x7f62b80035b0 con 0x7f62d006cf60 2023-05-20T01:28:06.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.497+0000 7f62cdffb700 1 -- 172.21.15.16:0/2847503621 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f62d00b54f0 msgr2=0x7f62d00b29e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:06.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.497+0000 7f62cdffb700 1 --2- 172.21.15.16:0/2847503621 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f62d00b54f0 0x7f62d00b29e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:06.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.497+0000 7f62cdffb700 1 -- 172.21.15.16:0/2847503621 >> v2:172.21.15.132:3300/0 conn(0x7f62d006cb90 msgr2=0x7f62d015ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:06.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.497+0000 7f62cdffb700 1 --2- 172.21.15.16:0/2847503621 >> v2:172.21.15.132:3300/0 conn(0x7f62d006cb90 0x7f62d015ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:06.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.497+0000 7f62cdffb700 1 -- 172.21.15.16:0/2847503621 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62d00b3480 con 0x7f62d006cf60 2023-05-20T01:28:06.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.497+0000 7f62cdffb700 1 -- 172.21.15.16:0/2847503621 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7f62c80019f0 con 0x7f62d006cf60 2023-05-20T01:28:06.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.497+0000 7f62cdffb700 1 -- 172.21.15.16:0/2847503621 >> v1:172.21.15.16:6789/0 conn(0x7f62d006cf60 legacy=0x7f62d0070910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:06.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.497+0000 7f62cdffb700 1 --2- 172.21.15.16:0/2847503621 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f62b8003a70 0x7f62b8003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:06.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.497+0000 7f62cdffb700 1 --2- 172.21.15.16:0/2847503621 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62b8004850 0x7f62b8004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:06.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.497+0000 7f62cdffb700 1 --2- 172.21.15.16:0/2847503621 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f62b8005200 0x7f62b800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:06.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.497+0000 7f62cdffb700 1 -- 172.21.15.16:0/2847503621 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62d00cc990 con 0x7f62b8004850 2023-05-20T01:28:06.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.497+0000 7f62cdffb700 1 -- 172.21.15.16:0/2847503621 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62d00b31a0 con 0x7f62b8003a70 2023-05-20T01:28:06.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.497+0000 7f62cdffb700 1 -- 172.21.15.16:0/2847503621 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62b800bf50 con 0x7f62b8005200 2023-05-20T01:28:06.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.497+0000 7f62ceffd700 1 --2- 172.21.15.16:0/2847503621 >> v2:172.21.15.132:3300/0 conn(0x7f62d006cb90 0x7f62d015ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-05-20T01:28:06.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.497+0000 7f62ce7fc700 1 --2- 172.21.15.16:0/2847503621 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f62d00b54f0 0x7f62d00b29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-05-20T01:28:06.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.497+0000 7f62ce7fc700 1 --2- 172.21.15.16:0/2847503621 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62b8004850 0x7f62b8004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:06.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.497+0000 7f62cf7fe700 1 --2- 172.21.15.16:0/2847503621 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f62b8005200 0x7f62b800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:06.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.497+0000 7f62ceffd700 1 --2- 172.21.15.16:0/2847503621 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f62b8003a70 0x7f62b8003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:06.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.497+0000 7f62ce7fc700 1 -- 172.21.15.16:0/2847503621 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f62b8005200 msgr2=0x7f62b800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:06.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.497+0000 7f62ce7fc700 1 --2- 172.21.15.16:0/2847503621 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f62b8005200 0x7f62b800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:06.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.497+0000 7f62ce7fc700 1 -- 172.21.15.16:0/2847503621 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f62b8003a70 msgr2=0x7f62b8003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:06.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.497+0000 7f62ce7fc700 1 --2- 172.21.15.16:0/2847503621 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f62b8003a70 0x7f62b8003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:06.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.498+0000 7f62ce7fc700 1 -- 172.21.15.16:0/2847503621 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62b80035b0 con 0x7f62b8004850 2023-05-20T01:28:06.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.498+0000 7f62ceffd700 1 --2- 172.21.15.16:0/2847503621 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f62b8003a70 0x7f62b8003e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:28:06.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.498+0000 7f62ce7fc700 1 --2- 172.21.15.16:0/2847503621 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62b8004850 0x7f62b8004cc0 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f62d00b5960 tx=0x7f62b8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6537a31cbac6533 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:06.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.498+0000 7f62cdffb700 1 -- 172.21.15.16:0/2847503621 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c0004030 con 0x7f62b8004850 2023-05-20T01:28:06.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.498+0000 7f62cdffb700 1 -- 172.21.15.16:0/2847503621 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62c0004190 con 0x7f62b8004850 2023-05-20T01:28:06.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.498+0000 7f62cdffb700 1 -- 172.21.15.16:0/2847503621 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c0004520 con 0x7f62b8004850 2023-05-20T01:28:06.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.498+0000 7f62d5578700 1 -- 172.21.15.16:0/2847503621 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62b8004850 msgr2=0x7f62b8004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:06.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.498+0000 7f62d5578700 1 --2- 172.21.15.16:0/2847503621 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62b8004850 0x7f62b8004cc0 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f62d00b5960 tx=0x7f62b8005880 comp rx=0 tx=0).stop 2023-05-20T01:28:06.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.498+0000 7f62cf7fe700 1 -- 172.21.15.16:0/2847503621 reap_dead start 2023-05-20T01:28:06.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.498+0000 7f62d5578700 1 -- 172.21.15.16:0/2847503621 shutdown_connections 2023-05-20T01:28:06.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.498+0000 7f62d5578700 1 -- 172.21.15.16:0/2847503621 >> 172.21.15.16:0/2847503621 conn(0x7f62d00661b0 msgr2=0x7f62d0066580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:06.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.498+0000 7f62d5578700 1 -- 172.21.15.16:0/2847503621 shutdown_connections 2023-05-20T01:28:06.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.498+0000 7f62d5578700 1 -- 172.21.15.16:0/2847503621 wait complete. 2023-05-20T01:28:06.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.500+0000 7f62d5578700 1 Processor -- start 2023-05-20T01:28:06.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.500+0000 7f62d5578700 1 -- start start 2023-05-20T01:28:06.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.501+0000 7f62d5578700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62b8004850 0x7f62d0140530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:06.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.501+0000 7f62d5578700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f62d01e6040 0x7f62d01e64b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:06.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.501+0000 7f62d5578700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f62d0140e50 0x7f62d01412c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:06.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.501+0000 7f62d5578700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62d01e69f0 con 0x7f62b8004850 2023-05-20T01:28:06.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.501+0000 7f62d5578700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62d0141800 con 0x7f62d0140e50 2023-05-20T01:28:06.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.501+0000 7f62d5578700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62d0154280 con 0x7f62d01e6040 2023-05-20T01:28:06.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.501+0000 7f62ceffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62b8004850 0x7f62d0140530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:06.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.501+0000 7f62ceffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62b8004850 0x7f62d0140530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39404/0 (socket says 172.21.15.16:39404) 2023-05-20T01:28:06.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.501+0000 7f62ceffd700 1 -- 172.21.15.16:0/1424101527 learned_addr learned my addr 172.21.15.16:0/1424101527 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:06.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.501+0000 7f62ce7fc700 1 --2- 172.21.15.16:0/1424101527 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f62d01e6040 0x7f62d01e64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:06.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.501+0000 7f62cf7fe700 1 --2- 172.21.15.16:0/1424101527 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f62d0140e50 0x7f62d01412c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:06.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.501+0000 7f62ceffd700 1 -- 172.21.15.16:0/1424101527 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f62d01e6040 msgr2=0x7f62d01e64b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:06.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.501+0000 7f62ceffd700 1 --2- 172.21.15.16:0/1424101527 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f62d01e6040 0x7f62d01e64b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:06.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.501+0000 7f62ceffd700 1 -- 172.21.15.16:0/1424101527 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f62d0140e50 msgr2=0x7f62d01412c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:06.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.501+0000 7f62ceffd700 1 --2- 172.21.15.16:0/1424101527 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f62d0140e50 0x7f62d01412c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:06.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.501+0000 7f62ceffd700 1 -- 172.21.15.16:0/1424101527 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62d015b240 con 0x7f62b8004850 2023-05-20T01:28:06.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.501+0000 7f62cf7fe700 1 --2- 172.21.15.16:0/1424101527 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f62d0140e50 0x7f62d01412c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:28:06.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.501+0000 7f62ceffd700 1 --2- 172.21.15.16:0/1424101527 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62b8004850 0x7f62d0140530 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f62c4010c60 tx=0x7f62c4010f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:06.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.501+0000 7f62b77fe700 1 -- 172.21.15.16:0/1424101527 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c400e8d0 con 0x7f62b8004850 2023-05-20T01:28:06.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.501+0000 7f62b77fe700 1 -- 172.21.15.16:0/1424101527 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62c400ea30 con 0x7f62b8004850 2023-05-20T01:28:06.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.501+0000 7f62d5578700 1 -- 172.21.15.16:0/1424101527 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62d015af60 con 0x7f62b8004850 2023-05-20T01:28:06.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.501+0000 7f62b77fe700 1 -- 172.21.15.16:0/1424101527 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c400eda0 con 0x7f62b8004850 2023-05-20T01:28:06.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.501+0000 7f62d5578700 1 -- 172.21.15.16:0/1424101527 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62d015b520 con 0x7f62b8004850 2023-05-20T01:28:06.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.502+0000 7f62d5578700 1 -- 172.21.15.16:0/1424101527 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62d00747b0 con 0x7f62b8004850 2023-05-20T01:28:06.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.503+0000 7f62b77fe700 1 -- 172.21.15.16:0/1424101527 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f62c4016460 con 0x7f62b8004850 2023-05-20T01:28:06.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.503+0000 7f62b77fe700 1 --2- 172.21.15.16:0/1424101527 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f62bc06ad10 0x7f62bc06d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:06.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.503+0000 7f62b77fe700 1 -- 172.21.15.16:0/1424101527 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 7073+0+0 (secure 0 0 0) 0x7f62c4089e10 con 0x7f62b8004850 2023-05-20T01:28:06.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.503+0000 7f62ce7fc700 1 --2- 172.21.15.16:0/1424101527 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f62bc06ad10 0x7f62bc06d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:06.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.504+0000 7f62ce7fc700 1 --2- 172.21.15.16:0/1424101527 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f62bc06ad10 0x7f62bc06d1c0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f62c0016040 tx=0x7f62c001f040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:06.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.508+0000 7f62b77fe700 1 -- 172.21.15.16:0/1424101527 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f62c4054a30 con 0x7f62b8004850 2023-05-20T01:28:06.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.715+0000 7f62d5578700 1 -- 172.21.15.16:0/1424101527 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f62d00b2e90 con 0x7f62b8004850 2023-05-20T01:28:06.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.716+0000 7f62b77fe700 1 -- 172.21.15.16:0/1424101527 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v123) v1 ==== 74+0+20629 (secure 0 0 0) 0x7f62c4024090 con 0x7f62b8004850 2023-05-20T01:28:06.721 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:28:06.722 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":123,"fsid":"c2245cc5-ecab-4087-a59b-bc81db901676","created":"2023-05-20T01:16:09.549230+0000","modified":"2023-05-20T01:27:21.615250+0000","last_up_change":"2023-05-20T01:16:19.788624+0000","last_in_change":"2023-05-20T01:16:12.092160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-05-20T01:16:20.934366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-05-20T01:16:21.372294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-05-20T01:26:34.067017+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-05-20T01:26:35.040602+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data_extra","create_time":"2023-05-20T01:26:48.309738+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"317a83cd-19c8-4958-a301-6087ef342289","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6825","nonce":3923634179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6827","nonce":3923634179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6831","nonce":3923634179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6829","nonce":3923634179}]},"public_addr":"172.21.15.16:6825/3923634179","cluster_addr":"172.21.15.16:6827/3923634179","heartbeat_back_addr":"172.21.15.16:6831/3923634179","heartbeat_front_addr":"172.21.15.16:6829/3923634179","state":["exists","up"]},{"osd":1,"uuid":"5581046f-8219-4089-8d7d-b7846399505e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6809","nonce":3860264220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6811","nonce":3860264220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6815","nonce":3860264220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6813","nonce":3860264220}]},"public_addr":"172.21.15.16:6809/3860264220","cluster_addr":"172.21.15.16:6811/3860264220","heartbeat_back_addr":"172.21.15.16:6815/3860264220","heartbeat_front_addr":"172.21.15.16:6813/3860264220","state":["exists","up"]},{"osd":2,"uuid":"072e9e66-4133-478e-8d4a-64d46c2963f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6817","nonce":293187030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6819","nonce":293187030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6823","nonce":293187030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6821","nonce":293187030}]},"public_addr":"172.21.15.16:6817/293187030","cluster_addr":"172.21.15.16:6819/293187030","heartbeat_back_addr":"172.21.15.16:6823/293187030","heartbeat_front_addr":"172.21.15.16:6821/293187030","state":["exists","up"]},{"osd":3,"uuid":"11e1bba6-00f8-48dd-9028-d8d716b7f145","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6801","nonce":3669781890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6803","nonce":3669781890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6807","nonce":3669781890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6805","nonce":3669781890}]},"public_addr":"172.21.15.16:6801/3669781890","cluster_addr":"172.21.15.16:6803/3669781890","heartbeat_back_addr":"172.21.15.16:6807/3669781890","heartbeat_front_addr":"172.21.15.16:6805/3669781890","state":["exists","up"]},{"osd":4,"uuid":"b8909c3a-cf70-46cb-8d0b-009c707cf9d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6819","nonce":506317223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6823","nonce":506317223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6829","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6831","nonce":506317223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6827","nonce":506317223}]},"public_addr":"172.21.15.132:6819/506317223","cluster_addr":"172.21.15.132:6823/506317223","heartbeat_back_addr":"172.21.15.132:6831/506317223","heartbeat_front_addr":"172.21.15.132:6827/506317223","state":["exists","up"]},{"osd":5,"uuid":"b1818304-4583-47bf-a6ec-0bfbb5166233","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6818","nonce":1844689995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6822","nonce":1844689995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6830","nonce":1844689995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6826","nonce":1844689995}]},"public_addr":"172.21.15.132:6818/1844689995","cluster_addr":"172.21.15.132:6822/1844689995","heartbeat_back_addr":"172.21.15.132:6830/1844689995","heartbeat_front_addr":"172.21.15.132:6826/1844689995","state":["exists","up"]},{"osd":6,"uuid":"d37f4691-eed1-4a93-8818-18354f03f4a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6801","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6802","nonce":727581123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6805","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6807","nonce":727581123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6813","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6815","nonce":727581123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6811","nonce":727581123}]},"public_addr":"172.21.15.132:6802/727581123","cluster_addr":"172.21.15.132:6807/727581123","heartbeat_back_addr":"172.21.15.132:6815/727581123","heartbeat_front_addr":"172.21.15.132:6811/727581123","state":["exists","up"]},{"osd":7,"uuid":"4f712f62-5d1d-4816-8042-03209144f8b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6803","nonce":466001149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6806","nonce":466001149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6814","nonce":466001149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6810","nonce":466001149}]},"public_addr":"172.21.15.132:6803/466001149","cluster_addr":"172.21.15.132:6806/466001149","heartbeat_back_addr":"172.21.15.132:6814/466001149","heartbeat_front_addr":"172.21.15.132:6810/466001149","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.338863+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.278680+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305694+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.246835+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.276237+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.304441+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305421+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.267050+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.33","mappings":[{"from":5,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6836/1584803225":"2023-05-21T01:27:21.602047+0000","172.21.15.16:6834/1584803225":"2023-05-21T01:27:21.602047+0000","172.21.15.16:6836/1281052551":"2023-05-21T01:27:10.544120+0000","172.21.15.16:6834/1281052551":"2023-05-21T01:27:10.544120+0000","172.21.15.16:6836/3765848068":"2023-05-21T01:27:09.119676+0000","172.21.15.16:6834/3765848068":"2023-05-21T01:27:09.119676+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-20T01:28:06.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.718+0000 7f62d5578700 1 -- 172.21.15.16:0/1424101527 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f62bc06ad10 msgr2=0x7f62bc06d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:06.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.718+0000 7f62d5578700 1 --2- 172.21.15.16:0/1424101527 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f62bc06ad10 0x7f62bc06d1c0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f62c0016040 tx=0x7f62c001f040 comp rx=0 tx=0).stop 2023-05-20T01:28:06.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.718+0000 7f62d5578700 1 -- 172.21.15.16:0/1424101527 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62b8004850 msgr2=0x7f62d0140530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:06.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.718+0000 7f62d5578700 1 --2- 172.21.15.16:0/1424101527 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62b8004850 0x7f62d0140530 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f62c4010c60 tx=0x7f62c4010f70 comp rx=0 tx=0).stop 2023-05-20T01:28:06.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.718+0000 7f62d5578700 1 -- 172.21.15.16:0/1424101527 shutdown_connections 2023-05-20T01:28:06.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.718+0000 7f62d5578700 1 --2- 172.21.15.16:0/1424101527 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f62d0140e50 0x7f62d01412c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:06.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.718+0000 7f62d5578700 1 --2- 172.21.15.16:0/1424101527 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62b8004850 0x7f62d0140530 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:06.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.718+0000 7f62d5578700 1 --2- 172.21.15.16:0/1424101527 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f62bc06ad10 0x7f62bc06d1c0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:06.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.718+0000 7f62d5578700 1 --2- 172.21.15.16:0/1424101527 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f62d01e6040 0x7f62d01e64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:06.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.718+0000 7f62d5578700 1 -- 172.21.15.16:0/1424101527 >> 172.21.15.16:0/1424101527 conn(0x7f62d00661b0 msgr2=0x7f62d00ce490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:06.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.719+0000 7f62d5578700 1 -- 172.21.15.16:0/1424101527 shutdown_connections 2023-05-20T01:28:06.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.719+0000 7f62d5578700 1 -- 172.21.15.16:0/1424101527 wait complete. 2023-05-20T01:28:06.735 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-05-20T01:16:20.934366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-05-20T01:16:21.372294+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2023-05-20T01:26:34.067017+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '111', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2023-05-20T01:26:35.040602+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '112', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data_extra', 'create_time': '2023-05-20T01:26:48.309738+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '115', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-05-20T01:28:06.736 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-05-20T01:28:06.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.859+0000 7fc81f605700 1 Processor -- start 2023-05-20T01:28:06.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.860+0000 7fc81f605700 1 -- start start 2023-05-20T01:28:06.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.860+0000 7fc81f605700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fc8180cd4f0 0x7fc8180cf1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:06.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.860+0000 7fc81f605700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc81806a510 0x7fc81806a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:06.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.860+0000 7fc81f605700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc81806dd00 con 0x7fc81815cec0 2023-05-20T01:28:06.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.860+0000 7fc81f605700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8180cd940 con 0x7fc8180cd4f0 2023-05-20T01:28:06.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.860+0000 7fc81f605700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc81806aed0 con 0x7fc81806a510 2023-05-20T01:28:06.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.861+0000 7fc81cba0700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fc8180cd4f0 0x7fc8180cf1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:06.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.861+0000 7fc81dba2700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc81806a510 0x7fc81806a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:06.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.861+0000 7fc81cba0700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fc8180cd4f0 0x7fc8180cf1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.16:36436/0 (socket says 172.21.15.16:36436) 2023-05-20T01:28:06.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.861+0000 7fc81d3a1700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7fc81815cec0 0x7fc81806e190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.16:60654/0 (socket says 172.21.15.16:60654) 2023-05-20T01:28:06.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.861+0000 7fc81dba2700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc81806a510 0x7fc81806a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.16:39454/0 (socket says 172.21.15.16:39454) 2023-05-20T01:28:06.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.861+0000 7fc81cba0700 1 -- 172.21.15.16:0/3756574546 learned_addr learned my addr 172.21.15.16:0/3756574546 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:06.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.861+0000 7fc81dba2700 1 -- 172.21.15.16:0/3756574546 >> v2:172.21.15.132:3300/0 conn(0x7fc8180cd4f0 msgr2=0x7fc8180cf1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:06.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.861+0000 7fc81dba2700 1 --2- 172.21.15.16:0/3756574546 >> v2:172.21.15.132:3300/0 conn(0x7fc8180cd4f0 0x7fc8180cf1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:06.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.861+0000 7fc81dba2700 1 -- 172.21.15.16:0/3756574546 >> v1:172.21.15.132:6789/0 conn(0x7fc81815cec0 legacy=0x7fc81806e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:06.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.861+0000 7fc81dba2700 1 -- 172.21.15.16:0/3756574546 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8180cfa60 con 0x7fc81806a510 2023-05-20T01:28:06.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.861+0000 7fc81cba0700 1 --2- 172.21.15.16:0/3756574546 >> v2:172.21.15.132:3300/0 conn(0x7fc8180cd4f0 0x7fc8180cf1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-05-20T01:28:06.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.862+0000 7fc81dba2700 1 --2- 172.21.15.16:0/3756574546 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc81806a510 0x7fc81806a990 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc810009d90 tx=0x7fc81000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b978ea1efcc45566 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:06.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.862+0000 7fc80ffff700 1 -- 172.21.15.16:0/3756574546 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc81000e070 con 0x7fc81806a510 2023-05-20T01:28:06.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.862+0000 7fc80ffff700 1 -- 172.21.15.16:0/3756574546 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc810003a20 con 0x7fc81806a510 2023-05-20T01:28:06.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.862+0000 7fc80ffff700 1 -- 172.21.15.16:0/3756574546 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc810003dd0 con 0x7fc81806a510 2023-05-20T01:28:06.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.862+0000 7fc81f605700 1 -- 172.21.15.16:0/3756574546 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc81806a510 msgr2=0x7fc81806a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:06.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.862+0000 7fc81f605700 1 --2- 172.21.15.16:0/3756574546 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc81806a510 0x7fc81806a990 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc810009d90 tx=0x7fc81000b330 comp rx=0 tx=0).stop 2023-05-20T01:28:06.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.862+0000 7fc81f605700 1 -- 172.21.15.16:0/3756574546 shutdown_connections 2023-05-20T01:28:06.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.862+0000 7fc81f605700 1 --2- 172.21.15.16:0/3756574546 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc81806a510 0x7fc81806a990 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:06.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.862+0000 7fc81f605700 1 --2- 172.21.15.16:0/3756574546 >> v2:172.21.15.132:3300/0 conn(0x7fc8180cd4f0 0x7fc8180cf1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:06.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.862+0000 7fc81f605700 1 -- 172.21.15.16:0/3756574546 >> 172.21.15.16:0/3756574546 conn(0x7fc81806c260 msgr2=0x7fc8180cffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:06.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.862+0000 7fc81f605700 1 -- 172.21.15.16:0/3756574546 shutdown_connections 2023-05-20T01:28:06.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.862+0000 7fc81f605700 1 -- 172.21.15.16:0/3756574546 wait complete. 2023-05-20T01:28:06.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.864+0000 7fc81f605700 1 Processor -- start 2023-05-20T01:28:06.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.864+0000 7fc81f605700 1 -- start start 2023-05-20T01:28:06.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.864+0000 7fc81f605700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc81806a510 0x7fc81815a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:06.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.864+0000 7fc81f605700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8180cd4f0 0x7fc8181f3090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:06.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.864+0000 7fc81f605700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc81815cec0 0x7fc81815b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:06.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.864+0000 7fc81f605700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc81815a770 con 0x7fc8180cd4f0 2023-05-20T01:28:06.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.864+0000 7fc81f605700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8180ad120 con 0x7fc81815cec0 2023-05-20T01:28:06.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.864+0000 7fc81f605700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8180ad3e0 con 0x7fc81806a510 2023-05-20T01:28:06.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.864+0000 7fc81cba0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8180cd4f0 0x7fc8181f3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:06.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.865+0000 7fc81d3a1700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc81806a510 0x7fc81815a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:06.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.865+0000 7fc81cba0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8180cd4f0 0x7fc8181f3090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39420/0 (socket says 172.21.15.16:39420) 2023-05-20T01:28:06.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.865+0000 7fc81cba0700 1 -- 172.21.15.16:0/290228326 learned_addr learned my addr 172.21.15.16:0/290228326 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:06.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.865+0000 7fc81dba2700 1 --2- 172.21.15.16:0/290228326 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc81815cec0 0x7fc81815b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:06.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.865+0000 7fc81cba0700 1 -- 172.21.15.16:0/290228326 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc81806a510 msgr2=0x7fc81815a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:06.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.865+0000 7fc81cba0700 1 --2- 172.21.15.16:0/290228326 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc81806a510 0x7fc81815a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:06.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.865+0000 7fc81cba0700 1 -- 172.21.15.16:0/290228326 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc81815cec0 msgr2=0x7fc81815b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:06.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.865+0000 7fc81cba0700 1 --2- 172.21.15.16:0/290228326 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc81815cec0 0x7fc81815b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:06.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.865+0000 7fc81cba0700 1 -- 172.21.15.16:0/290228326 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc814009550 con 0x7fc8180cd4f0 2023-05-20T01:28:06.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.865+0000 7fc81cba0700 1 --2- 172.21.15.16:0/290228326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8180cd4f0 0x7fc8181f3090 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7fc81400ebe0 tx=0x7fc81400eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:06.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.865+0000 7fc80dffb700 1 -- 172.21.15.16:0/290228326 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc81400c850 con 0x7fc8180cd4f0 2023-05-20T01:28:06.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.865+0000 7fc80dffb700 1 -- 172.21.15.16:0/290228326 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc81400c9b0 con 0x7fc8180cd4f0 2023-05-20T01:28:06.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.865+0000 7fc80dffb700 1 -- 172.21.15.16:0/290228326 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc81400cd40 con 0x7fc8180cd4f0 2023-05-20T01:28:06.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.865+0000 7fc81f605700 1 -- 172.21.15.16:0/290228326 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc810009a70 con 0x7fc8180cd4f0 2023-05-20T01:28:06.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.865+0000 7fc81f605700 1 -- 172.21.15.16:0/290228326 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc81806d990 con 0x7fc8180cd4f0 2023-05-20T01:28:06.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.866+0000 7fc81f605700 1 -- 172.21.15.16:0/290228326 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc81806d6b0 con 0x7fc8180cd4f0 2023-05-20T01:28:06.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.867+0000 7fc80dffb700 1 -- 172.21.15.16:0/290228326 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc814013470 con 0x7fc8180cd4f0 2023-05-20T01:28:06.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.867+0000 7fc80dffb700 1 --2- 172.21.15.16:0/290228326 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc7fc06ace0 0x7fc7fc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:06.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.867+0000 7fc80dffb700 1 -- 172.21.15.16:0/290228326 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 7073+0+0 (secure 0 0 0) 0x7fc8140899a0 con 0x7fc8180cd4f0 2023-05-20T01:28:06.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.867+0000 7fc81d3a1700 1 --2- 172.21.15.16:0/290228326 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc7fc06ace0 0x7fc7fc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:06.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.868+0000 7fc81d3a1700 1 --2- 172.21.15.16:0/290228326 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc7fc06ace0 0x7fc7fc06d190 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fc808000f60 tx=0x7fc808004a50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:06.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:06.871+0000 7fc80dffb700 1 -- 172.21.15.16:0/290228326 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fc814054650 con 0x7fc8180cd4f0 2023-05-20T01:28:07.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.066+0000 7fc81f605700 1 -- 172.21.15.16:0/290228326 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fc81815d290 con 0x7fc8180cd4f0 2023-05-20T01:28:07.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.066+0000 7fc80dffb700 1 -- 172.21.15.16:0/290228326 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v123) v1 ==== 92+0+10 (secure 0 0 0) 0x7fc814057c70 con 0x7fc8180cd4f0 2023-05-20T01:28:07.071 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 8 2023-05-20T01:28:07.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.068+0000 7fc81f605700 1 -- 172.21.15.16:0/290228326 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc7fc06ace0 msgr2=0x7fc7fc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:07.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.068+0000 7fc81f605700 1 --2- 172.21.15.16:0/290228326 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc7fc06ace0 0x7fc7fc06d190 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fc808000f60 tx=0x7fc808004a50 comp rx=0 tx=0).stop 2023-05-20T01:28:07.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.068+0000 7fc81f605700 1 -- 172.21.15.16:0/290228326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8180cd4f0 msgr2=0x7fc8181f3090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:07.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.068+0000 7fc81f605700 1 --2- 172.21.15.16:0/290228326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8180cd4f0 0x7fc8181f3090 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7fc81400ebe0 tx=0x7fc81400eef0 comp rx=0 tx=0).stop 2023-05-20T01:28:07.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.068+0000 7fc81f605700 1 -- 172.21.15.16:0/290228326 shutdown_connections 2023-05-20T01:28:07.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.068+0000 7fc81f605700 1 --2- 172.21.15.16:0/290228326 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc81815cec0 0x7fc81815b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:07.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.068+0000 7fc81f605700 1 --2- 172.21.15.16:0/290228326 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc7fc06ace0 0x7fc7fc06d190 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:07.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.068+0000 7fc81f605700 1 --2- 172.21.15.16:0/290228326 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc81806a510 0x7fc81815a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:07.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.068+0000 7fc81f605700 1 --2- 172.21.15.16:0/290228326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8180cd4f0 0x7fc8181f3090 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:07.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.068+0000 7fc81f605700 1 -- 172.21.15.16:0/290228326 >> 172.21.15.16:0/290228326 conn(0x7fc81806c260 msgr2=0x7fc8180b3120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:07.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.069+0000 7fc81f605700 1 -- 172.21.15.16:0/290228326 shutdown_connections 2023-05-20T01:28:07.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.069+0000 7fc81f605700 1 -- 172.21.15.16:0/290228326 wait complete. 2023-05-20T01:28:07.085 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-05-20T01:28:07.176 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:28:07.179 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:28:07.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.210+0000 7f39a0987700 1 Processor -- start 2023-05-20T01:28:07.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.210+0000 7f39a0987700 1 -- start start 2023-05-20T01:28:07.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.211+0000 7f39a0987700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f399c0cd1c0 0x7f399c0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:07.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.211+0000 7f39a0987700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f399c0cd630 0x7f399c0715f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:07.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.211+0000 7f39a0987700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f399c0ccd30 con 0x7f399c159b50 2023-05-20T01:28:07.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.211+0000 7f39a0987700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f399c0ce990 con 0x7f399c0cd1c0 2023-05-20T01:28:07.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.211+0000 7f39a0987700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f399c0cc6e0 con 0x7f399c0cd630 2023-05-20T01:28:07.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.211+0000 7f3999d9b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f399c0cd1c0 0x7f399c0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:07.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.211+0000 7f3999d9b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f399c0cd1c0 0x7f399c0710b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39428/0 (socket says 172.21.15.16:39428) 2023-05-20T01:28:07.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.211+0000 7f3999d9b700 1 -- 172.21.15.16:0/1985727098 learned_addr learned my addr 172.21.15.16:0/1985727098 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:07.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.211+0000 7f399ad9d700 1 --2- 172.21.15.16:0/1985727098 >> v2:172.21.15.132:3300/0 conn(0x7f399c0cd630 0x7f399c0715f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:07.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.211+0000 7f3999d9b700 1 -- 172.21.15.16:0/1985727098 >> v2:172.21.15.132:3300/0 conn(0x7f399c0cd630 msgr2=0x7f399c0715f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:07.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.211+0000 7f3999d9b700 1 --2- 172.21.15.16:0/1985727098 >> v2:172.21.15.132:3300/0 conn(0x7f399c0cd630 0x7f399c0715f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:07.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.211+0000 7f3999d9b700 1 -- 172.21.15.16:0/1985727098 >> v1:172.21.15.132:6789/0 conn(0x7f399c159b50 legacy=0x7f399c06e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:07.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.211+0000 7f3999d9b700 1 -- 172.21.15.16:0/1985727098 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f399c0cc9c0 con 0x7f399c0cd1c0 2023-05-20T01:28:07.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.211+0000 7f3999d9b700 1 --2- 172.21.15.16:0/1985727098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f399c0cd1c0 0x7f399c0710b0 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f399c06c340 tx=0x7f399000be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5950bac62113ba server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:07.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.211+0000 7f399959a700 1 -- 172.21.15.16:0/1985727098 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f399000dcf0 con 0x7f399c0cd1c0 2023-05-20T01:28:07.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.211+0000 7f399959a700 1 -- 172.21.15.16:0/1985727098 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f399000de50 con 0x7f399c0cd1c0 2023-05-20T01:28:07.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.211+0000 7f399959a700 1 -- 172.21.15.16:0/1985727098 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3990004250 con 0x7f399c0cd1c0 2023-05-20T01:28:07.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.212+0000 7f39a0987700 1 -- 172.21.15.16:0/1985727098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f399c0cd1c0 msgr2=0x7f399c0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:07.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.212+0000 7f39a0987700 1 --2- 172.21.15.16:0/1985727098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f399c0cd1c0 0x7f399c0710b0 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f399c06c340 tx=0x7f399000be70 comp rx=0 tx=0).stop 2023-05-20T01:28:07.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.212+0000 7f39a0987700 1 -- 172.21.15.16:0/1985727098 shutdown_connections 2023-05-20T01:28:07.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.212+0000 7f39a0987700 1 --2- 172.21.15.16:0/1985727098 >> v2:172.21.15.132:3300/0 conn(0x7f399c0cd630 0x7f399c0715f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:07.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.212+0000 7f39a0987700 1 --2- 172.21.15.16:0/1985727098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f399c0cd1c0 0x7f399c0710b0 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:07.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.212+0000 7f39a0987700 1 -- 172.21.15.16:0/1985727098 >> 172.21.15.16:0/1985727098 conn(0x7f399c0cffd0 msgr2=0x7f399c0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:07.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.212+0000 7f39a0987700 1 -- 172.21.15.16:0/1985727098 shutdown_connections 2023-05-20T01:28:07.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.212+0000 7f39a0987700 1 -- 172.21.15.16:0/1985727098 wait complete. 2023-05-20T01:28:07.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.213+0000 7f39a0987700 1 Processor -- start 2023-05-20T01:28:07.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.214+0000 7f39a0987700 1 -- start start 2023-05-20T01:28:07.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.214+0000 7f39a0987700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f399c0cd1c0 0x7f399c148580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:07.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.214+0000 7f39a0987700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f399c0cd630 0x7f399c1f0c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:07.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.214+0000 7f39a0987700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f399c159b50 0x7f399c1588e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:07.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.214+0000 7f39a0987700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f399c148ac0 con 0x7f399c0cd630 2023-05-20T01:28:07.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.214+0000 7f39a0987700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f399c1efbd0 con 0x7f399c0cd1c0 2023-05-20T01:28:07.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.214+0000 7f39a0987700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f399c1efeb0 con 0x7f399c159b50 2023-05-20T01:28:07.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.214+0000 7f3999d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f399c0cd630 0x7f399c1f0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:07.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.214+0000 7f3999d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f399c0cd630 0x7f399c1f0c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39434/0 (socket says 172.21.15.16:39434) 2023-05-20T01:28:07.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.214+0000 7f3999d9b700 1 -- 172.21.15.16:0/2452568893 learned_addr learned my addr 172.21.15.16:0/2452568893 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:07.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.214+0000 7f399a59c700 1 --2- 172.21.15.16:0/2452568893 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f399c0cd1c0 0x7f399c148580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:07.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.215+0000 7f399ad9d700 1 --2- 172.21.15.16:0/2452568893 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f399c159b50 0x7f399c1588e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:07.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.215+0000 7f3999d9b700 1 -- 172.21.15.16:0/2452568893 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f399c159b50 msgr2=0x7f399c1588e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:07.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.215+0000 7f3999d9b700 1 --2- 172.21.15.16:0/2452568893 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f399c159b50 0x7f399c1588e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:07.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.215+0000 7f3999d9b700 1 -- 172.21.15.16:0/2452568893 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f399c0cd1c0 msgr2=0x7f399c148580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:07.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.215+0000 7f3999d9b700 1 --2- 172.21.15.16:0/2452568893 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f399c0cd1c0 0x7f399c148580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:07.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.215+0000 7f3999d9b700 1 -- 172.21.15.16:0/2452568893 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f399000b7f0 con 0x7f399c0cd630 2023-05-20T01:28:07.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.215+0000 7f399a59c700 1 --2- 172.21.15.16:0/2452568893 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f399c0cd1c0 0x7f399c148580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:28:07.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.215+0000 7f3999d9b700 1 --2- 172.21.15.16:0/2452568893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f399c0cd630 0x7f399c1f0c70 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f3990000c00 tx=0x7f3990004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:07.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.215+0000 7f398affd700 1 -- 172.21.15.16:0/2452568893 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3990020070 con 0x7f399c0cd630 2023-05-20T01:28:07.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.215+0000 7f398affd700 1 -- 172.21.15.16:0/2452568893 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3990004030 con 0x7f399c0cd630 2023-05-20T01:28:07.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.215+0000 7f39a0987700 1 -- 172.21.15.16:0/2452568893 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f399c0cec70 con 0x7f399c0cd630 2023-05-20T01:28:07.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.215+0000 7f398affd700 1 -- 172.21.15.16:0/2452568893 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3990010470 con 0x7f399c0cd630 2023-05-20T01:28:07.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.215+0000 7f39a0987700 1 -- 172.21.15.16:0/2452568893 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f399c071e80 con 0x7f399c0cd630 2023-05-20T01:28:07.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.216+0000 7f39a0987700 1 -- 172.21.15.16:0/2452568893 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f399c0747b0 con 0x7f399c0cd630 2023-05-20T01:28:07.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.216+0000 7f398affd700 1 -- 172.21.15.16:0/2452568893 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f39900193f0 con 0x7f399c0cd630 2023-05-20T01:28:07.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.217+0000 7f398affd700 1 --2- 172.21.15.16:0/2452568893 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f397c06ace0 0x7f397c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:07.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.217+0000 7f398affd700 1 -- 172.21.15.16:0/2452568893 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 7073+0+0 (secure 0 0 0) 0x7f399008cd60 con 0x7f399c0cd630 2023-05-20T01:28:07.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.217+0000 7f399a59c700 1 --2- 172.21.15.16:0/2452568893 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f397c06ace0 0x7f397c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:07.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.217+0000 7f399a59c700 1 --2- 172.21.15.16:0/2452568893 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f397c06ace0 0x7f397c06d190 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f398c007fd0 tx=0x7f398c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:07.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.221+0000 7f398affd700 1 -- 172.21.15.16:0/2452568893 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f3990057aa0 con 0x7f399c0cd630 2023-05-20T01:28:07.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.417+0000 7f39a0987700 1 -- 172.21.15.16:0/2452568893 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f399c0d04d0 con 0x7f399c0cd630 2023-05-20T01:28:07.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.417+0000 7f398affd700 1 -- 172.21.15.16:0/2452568893 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v123) v1 ==== 93+0+10 (secure 0 0 0) 0x7f399005b0c0 con 0x7f399c0cd630 2023-05-20T01:28:07.422 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-05-20T01:28:07.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.419+0000 7f39a0987700 1 -- 172.21.15.16:0/2452568893 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f397c06ace0 msgr2=0x7f397c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:07.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.419+0000 7f39a0987700 1 --2- 172.21.15.16:0/2452568893 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f397c06ace0 0x7f397c06d190 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f398c007fd0 tx=0x7f398c009040 comp rx=0 tx=0).stop 2023-05-20T01:28:07.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.419+0000 7f39a0987700 1 -- 172.21.15.16:0/2452568893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f399c0cd630 msgr2=0x7f399c1f0c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:07.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.419+0000 7f39a0987700 1 --2- 172.21.15.16:0/2452568893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f399c0cd630 0x7f399c1f0c70 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f3990000c00 tx=0x7f3990004750 comp rx=0 tx=0).stop 2023-05-20T01:28:07.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.419+0000 7f39a0987700 1 -- 172.21.15.16:0/2452568893 shutdown_connections 2023-05-20T01:28:07.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.419+0000 7f39a0987700 1 --2- 172.21.15.16:0/2452568893 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f397c06ace0 0x7f397c06d190 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:07.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.419+0000 7f39a0987700 1 --2- 172.21.15.16:0/2452568893 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f399c159b50 0x7f399c1588e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:07.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.419+0000 7f39a0987700 1 --2- 172.21.15.16:0/2452568893 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f399c0cd1c0 0x7f399c148580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:07.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.419+0000 7f39a0987700 1 --2- 172.21.15.16:0/2452568893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f399c0cd630 0x7f399c1f0c70 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:07.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.419+0000 7f39a0987700 1 -- 172.21.15.16:0/2452568893 >> 172.21.15.16:0/2452568893 conn(0x7f399c0cffd0 msgr2=0x7f399c0cba40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:07.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.419+0000 7f39a0987700 1 -- 172.21.15.16:0/2452568893 shutdown_connections 2023-05-20T01:28:07.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.419+0000 7f39a0987700 1 -- 172.21.15.16:0/2452568893 wait complete. 2023-05-20T01:28:07.435 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-05-20T01:28:07.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.561+0000 7f6543a64700 1 Processor -- start 2023-05-20T01:28:07.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.561+0000 7f6543a64700 1 -- start start 2023-05-20T01:28:07.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.562+0000 7f6543a64700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f653c06e020 0x7f653c0710d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:07.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.562+0000 7f6543a64700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f653c072430 con 0x7f653c0cd070 2023-05-20T01:28:07.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.562+0000 7f6543a64700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f653c06e460 con 0x7f653c15cf40 2023-05-20T01:28:07.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.562+0000 7f6543a64700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f653c071610 con 0x7f653c06e020 2023-05-20T01:28:07.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.562+0000 7f6540fff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f653c0cd070 0x7f653c0cabb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50970/0 (socket says 172.21.15.16:50970) 2023-05-20T01:28:07.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.562+0000 7f6540fff700 1 -- 172.21.15.16:0/2882844930 learned_addr learned my addr 172.21.15.16:0/2882844930 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:28:07.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.562+0000 7f6542001700 1 --2- 172.21.15.16:0/2882844930 >> v2:172.21.15.16:3300/0 conn(0x7f653c06e020 0x7f653c0710d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:07.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.563+0000 7f6533fff700 1 -- 172.21.15.16:0/2882844930 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1011171600 0 0) 0x7f653c072430 con 0x7f653c0cd070 2023-05-20T01:28:07.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.563+0000 7f6533fff700 1 -- 172.21.15.16:0/2882844930 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f65280035b0 con 0x7f653c0cd070 2023-05-20T01:28:07.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.563+0000 7f6542001700 1 -- 172.21.15.16:0/2882844930 >> v1:172.21.15.132:6789/0 conn(0x7f653c15cf40 legacy=0x7f653c0cc960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:07.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.563+0000 7f6542001700 1 -- 172.21.15.16:0/2882844930 >> v1:172.21.15.16:6789/0 conn(0x7f653c0cd070 legacy=0x7f653c0cabb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:07.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.563+0000 7f6542001700 1 -- 172.21.15.16:0/2882844930 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f653c071820 con 0x7f653c06e020 2023-05-20T01:28:07.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.563+0000 7f6542001700 1 --2- 172.21.15.16:0/2882844930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f653c06e020 0x7f653c0710d0 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f6534009d90 tx=0x7f653400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b80dc1023dcce9b server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:07.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.563+0000 7f6533fff700 1 -- 172.21.15.16:0/2882844930 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f653400e070 con 0x7f653c06e020 2023-05-20T01:28:07.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.563+0000 7f6533fff700 1 -- 172.21.15.16:0/2882844930 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f653400bb20 con 0x7f653c06e020 2023-05-20T01:28:07.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.563+0000 7f6533fff700 1 -- 172.21.15.16:0/2882844930 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6534012430 con 0x7f653c06e020 2023-05-20T01:28:07.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.564+0000 7f6543a64700 1 -- 172.21.15.16:0/2882844930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f653c06e020 msgr2=0x7f653c0710d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:07.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.564+0000 7f6543a64700 1 --2- 172.21.15.16:0/2882844930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f653c06e020 0x7f653c0710d0 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f6534009d90 tx=0x7f653400b330 comp rx=0 tx=0).stop 2023-05-20T01:28:07.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.564+0000 7f6543a64700 1 -- 172.21.15.16:0/2882844930 shutdown_connections 2023-05-20T01:28:07.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.564+0000 7f6543a64700 1 --2- 172.21.15.16:0/2882844930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f653c06e020 0x7f653c0710d0 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:07.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.564+0000 7f6543a64700 1 -- 172.21.15.16:0/2882844930 >> 172.21.15.16:0/2882844930 conn(0x7f653c06c260 msgr2=0x7f653c0cffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:07.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.564+0000 7f6543a64700 1 -- 172.21.15.16:0/2882844930 shutdown_connections 2023-05-20T01:28:07.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.564+0000 7f6543a64700 1 -- 172.21.15.16:0/2882844930 wait complete. 2023-05-20T01:28:07.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.566+0000 7f6543a64700 1 Processor -- start 2023-05-20T01:28:07.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.566+0000 7f6543a64700 1 -- start start 2023-05-20T01:28:07.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.567+0000 7f6543a64700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f653c06e020 0x7f653c0c3b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:07.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.567+0000 7f6543a64700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f653c0cd070 0x7f653c0c4040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:07.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.567+0000 7f6543a64700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f653c15cf40 0x7f653c0c90b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:07.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.567+0000 7f6543a64700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f653c150d20 con 0x7f653c15cf40 2023-05-20T01:28:07.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.567+0000 7f6543a64700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f653c1e38f0 con 0x7f653c0cd070 2023-05-20T01:28:07.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.567+0000 7f6543a64700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f653c0c4630 con 0x7f653c06e020 2023-05-20T01:28:07.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.567+0000 7f6542001700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f653c15cf40 0x7f653c0c90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:07.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.567+0000 7f6542001700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f653c15cf40 0x7f653c0c90b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39450/0 (socket says 172.21.15.16:39450) 2023-05-20T01:28:07.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.567+0000 7f6542001700 1 -- 172.21.15.16:0/174553612 learned_addr learned my addr 172.21.15.16:0/174553612 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:07.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.567+0000 7f6541800700 1 --2- 172.21.15.16:0/174553612 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f653c06e020 0x7f653c0c3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:07.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.567+0000 7f6540fff700 1 --2- 172.21.15.16:0/174553612 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f653c0cd070 0x7f653c0c4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:07.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.567+0000 7f6542001700 1 -- 172.21.15.16:0/174553612 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f653c06e020 msgr2=0x7f653c0c3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:07.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.567+0000 7f6542001700 1 --2- 172.21.15.16:0/174553612 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f653c06e020 0x7f653c0c3b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:07.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.567+0000 7f6542001700 1 -- 172.21.15.16:0/174553612 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f653c0cd070 msgr2=0x7f653c0c4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:07.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.567+0000 7f6542001700 1 --2- 172.21.15.16:0/174553612 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f653c0cd070 0x7f653c0c4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:07.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.567+0000 7f6542001700 1 -- 172.21.15.16:0/174553612 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6534009a70 con 0x7f653c15cf40 2023-05-20T01:28:07.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.567+0000 7f6540fff700 1 --2- 172.21.15.16:0/174553612 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f653c0cd070 0x7f653c0c4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:28:07.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.567+0000 7f6542001700 1 --2- 172.21.15.16:0/174553612 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f653c15cf40 0x7f653c0c90b0 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f653c067250 tx=0x7f6534004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:07.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.567+0000 7f6531ffb700 1 -- 172.21.15.16:0/174553612 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6534012430 con 0x7f653c15cf40 2023-05-20T01:28:07.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.567+0000 7f6531ffb700 1 -- 172.21.15.16:0/174553612 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6534020070 con 0x7f653c15cf40 2023-05-20T01:28:07.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.567+0000 7f6531ffb700 1 -- 172.21.15.16:0/174553612 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6534004280 con 0x7f653c15cf40 2023-05-20T01:28:07.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.567+0000 7f6543a64700 1 -- 172.21.15.16:0/174553612 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f653c071de0 con 0x7f653c15cf40 2023-05-20T01:28:07.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.568+0000 7f6543a64700 1 -- 172.21.15.16:0/174553612 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f653c071b00 con 0x7f653c15cf40 2023-05-20T01:28:07.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.569+0000 7f6543a64700 1 -- 172.21.15.16:0/174553612 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f653c0720c0 con 0x7f653c15cf40 2023-05-20T01:28:07.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.569+0000 7f6531ffb700 1 -- 172.21.15.16:0/174553612 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f65340043e0 con 0x7f653c15cf40 2023-05-20T01:28:07.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.569+0000 7f6531ffb700 1 --2- 172.21.15.16:0/174553612 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f652006ace0 0x7f652006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:07.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.569+0000 7f6531ffb700 1 -- 172.21.15.16:0/174553612 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 7073+0+0 (secure 0 0 0) 0x7f6534022070 con 0x7f653c15cf40 2023-05-20T01:28:07.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.569+0000 7f6541800700 1 --2- 172.21.15.16:0/174553612 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f652006ace0 0x7f652006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:07.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.570+0000 7f6541800700 1 --2- 172.21.15.16:0/174553612 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f652006ace0 0x7f652006d190 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f653c070f70 tx=0x7f6538009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:07.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.574+0000 7f6531ffb700 1 -- 172.21.15.16:0/174553612 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f6534056250 con 0x7f653c15cf40 2023-05-20T01:28:07.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.770+0000 7f6543a64700 1 -- 172.21.15.16:0/174553612 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f653c0747b0 con 0x7f653c15cf40 2023-05-20T01:28:07.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.771+0000 7f6531ffb700 1 -- 172.21.15.16:0/174553612 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v123) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6534059870 con 0x7f653c15cf40 2023-05-20T01:28:07.775 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-05-20T01:28:07.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.773+0000 7f6543a64700 1 -- 172.21.15.16:0/174553612 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f652006ace0 msgr2=0x7f652006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:07.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.773+0000 7f6543a64700 1 --2- 172.21.15.16:0/174553612 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f652006ace0 0x7f652006d190 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f653c070f70 tx=0x7f6538009040 comp rx=0 tx=0).stop 2023-05-20T01:28:07.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.773+0000 7f6543a64700 1 -- 172.21.15.16:0/174553612 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f653c15cf40 msgr2=0x7f653c0c90b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:07.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.773+0000 7f6543a64700 1 --2- 172.21.15.16:0/174553612 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f653c15cf40 0x7f653c0c90b0 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f653c067250 tx=0x7f6534004ad0 comp rx=0 tx=0).stop 2023-05-20T01:28:07.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.773+0000 7f6543a64700 1 -- 172.21.15.16:0/174553612 shutdown_connections 2023-05-20T01:28:07.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.773+0000 7f6543a64700 1 --2- 172.21.15.16:0/174553612 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f653c15cf40 0x7f653c0c90b0 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:07.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.773+0000 7f6543a64700 1 --2- 172.21.15.16:0/174553612 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f652006ace0 0x7f652006d190 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:07.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.773+0000 7f6543a64700 1 --2- 172.21.15.16:0/174553612 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f653c06e020 0x7f653c0c3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:07.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.773+0000 7f6543a64700 1 --2- 172.21.15.16:0/174553612 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f653c0cd070 0x7f653c0c4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:07.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.773+0000 7f6543a64700 1 -- 172.21.15.16:0/174553612 >> 172.21.15.16:0/174553612 conn(0x7f653c06c260 msgr2=0x7f653c0cffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:07.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.773+0000 7f6543a64700 1 -- 172.21.15.16:0/174553612 shutdown_connections 2023-05-20T01:28:07.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.773+0000 7f6543a64700 1 -- 172.21.15.16:0/174553612 wait complete. 2023-05-20T01:28:07.789 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-05-20T01:28:07.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.914+0000 7f418260b700 1 Processor -- start 2023-05-20T01:28:07.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.914+0000 7f418260b700 1 -- start start 2023-05-20T01:28:07.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.915+0000 7f418260b700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f417c15cec0 0x7f417c071bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:07.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.915+0000 7f418260b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f417c15d290 0x7f417c072130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:07.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.915+0000 7f418260b700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f417c0cd630 con 0x7f417c0cf2c0 2023-05-20T01:28:07.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.915+0000 7f418260b700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f417c0cca60 con 0x7f417c15d290 2023-05-20T01:28:07.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.915+0000 7f418260b700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f417c0cfdd0 con 0x7f417c15cec0 2023-05-20T01:28:07.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.915+0000 7f417b7fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f417c15d290 0x7f417c072130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:07.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.915+0000 7f4180ba8700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f417c0cf2c0 0x7f417c0cf6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50986/0 (socket says 172.21.15.16:50986) 2023-05-20T01:28:07.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.915+0000 7f4180ba8700 1 -- 172.21.15.16:0/1420281462 learned_addr learned my addr 172.21.15.16:0/1420281462 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:28:07.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.916+0000 7f417bfff700 1 --2- 172.21.15.16:0/1420281462 >> v2:172.21.15.132:3300/0 conn(0x7f417c15cec0 0x7f417c071bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:07.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.916+0000 7f417b7fe700 1 -- 172.21.15.16:0/1420281462 >> v2:172.21.15.132:3300/0 conn(0x7f417c15cec0 msgr2=0x7f417c071bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:07.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.916+0000 7f417b7fe700 1 --2- 172.21.15.16:0/1420281462 >> v2:172.21.15.132:3300/0 conn(0x7f417c15cec0 0x7f417c071bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:07.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.916+0000 7f417b7fe700 1 -- 172.21.15.16:0/1420281462 >> v1:172.21.15.16:6789/0 conn(0x7f417c0cf2c0 legacy=0x7f417c0cf6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:07.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.916+0000 7f417b7fe700 1 -- 172.21.15.16:0/1420281462 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f417c0ccd00 con 0x7f417c15d290 2023-05-20T01:28:07.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.916+0000 7f417affd700 1 -- 172.21.15.16:0/1420281462 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3577813585 0 0) 0x7f417c0cd630 con 0x7f417c0cf2c0 2023-05-20T01:28:07.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.916+0000 7f417b7fe700 1 --2- 172.21.15.16:0/1420281462 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f417c15d290 0x7f417c072130 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f417c072770 tx=0x7f4164009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fa2f33d87936c07 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:07.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.916+0000 7f417affd700 1 -- 172.21.15.16:0/1420281462 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4164011830 con 0x7f417c15d290 2023-05-20T01:28:07.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.916+0000 7f417affd700 1 -- 172.21.15.16:0/1420281462 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4164011990 con 0x7f417c15d290 2023-05-20T01:28:07.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.916+0000 7f417affd700 1 -- 172.21.15.16:0/1420281462 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4164011d20 con 0x7f417c15d290 2023-05-20T01:28:07.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.917+0000 7f418260b700 1 -- 172.21.15.16:0/1420281462 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f417c15d290 msgr2=0x7f417c072130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:07.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.917+0000 7f418260b700 1 --2- 172.21.15.16:0/1420281462 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f417c15d290 0x7f417c072130 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f417c072770 tx=0x7f4164009b20 comp rx=0 tx=0).stop 2023-05-20T01:28:07.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.917+0000 7f418260b700 1 -- 172.21.15.16:0/1420281462 shutdown_connections 2023-05-20T01:28:07.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.917+0000 7f418260b700 1 --2- 172.21.15.16:0/1420281462 >> v2:172.21.15.132:3300/0 conn(0x7f417c15cec0 0x7f417c071bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:07.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.917+0000 7f418260b700 1 --2- 172.21.15.16:0/1420281462 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f417c15d290 0x7f417c072130 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:07.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.917+0000 7f418260b700 1 -- 172.21.15.16:0/1420281462 >> 172.21.15.16:0/1420281462 conn(0x7f417c0cffd0 msgr2=0x7f417c0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:07.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.917+0000 7f418260b700 1 -- 172.21.15.16:0/1420281462 shutdown_connections 2023-05-20T01:28:07.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.917+0000 7f418260b700 1 -- 172.21.15.16:0/1420281462 wait complete. 2023-05-20T01:28:07.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.918+0000 7f418260b700 1 Processor -- start 2023-05-20T01:28:07.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.919+0000 7f418260b700 1 -- start start 2023-05-20T01:28:07.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.919+0000 7f418260b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f417c0cf2c0 0x7f417c150690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:07.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.919+0000 7f418260b700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f417c15cec0 0x7f417c1f3380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:07.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.919+0000 7f418260b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f417c15d290 0x7f417c0c8780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:07.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.919+0000 7f418260b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f417c0b3430 con 0x7f417c15d290 2023-05-20T01:28:07.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.919+0000 7f418260b700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f417c0b2e40 con 0x7f417c0cf2c0 2023-05-20T01:28:07.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.919+0000 7f418260b700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f417c1f38c0 con 0x7f417c15cec0 2023-05-20T01:28:07.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.920+0000 7f4180ba8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f417c15d290 0x7f417c0c8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:07.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.920+0000 7f4180ba8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f417c15d290 0x7f417c0c8780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39464/0 (socket says 172.21.15.16:39464) 2023-05-20T01:28:07.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.920+0000 7f4180ba8700 1 -- 172.21.15.16:0/1888225797 learned_addr learned my addr 172.21.15.16:0/1888225797 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:07.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.920+0000 7f417bfff700 1 --2- 172.21.15.16:0/1888225797 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f417c0cf2c0 0x7f417c150690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:07.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.920+0000 7f417b7fe700 1 --2- 172.21.15.16:0/1888225797 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f417c15cec0 0x7f417c1f3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:07.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.920+0000 7f4180ba8700 1 -- 172.21.15.16:0/1888225797 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f417c15cec0 msgr2=0x7f417c1f3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:07.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.920+0000 7f4180ba8700 1 --2- 172.21.15.16:0/1888225797 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f417c15cec0 0x7f417c1f3380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:07.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.920+0000 7f4180ba8700 1 -- 172.21.15.16:0/1888225797 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f417c0cf2c0 msgr2=0x7f417c150690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:07.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.920+0000 7f4180ba8700 1 --2- 172.21.15.16:0/1888225797 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f417c0cf2c0 0x7f417c150690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:07.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.920+0000 7f4180ba8700 1 -- 172.21.15.16:0/1888225797 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41640094a0 con 0x7f417c15d290 2023-05-20T01:28:07.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.920+0000 7f417b7fe700 1 --2- 172.21.15.16:0/1888225797 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f417c15cec0 0x7f417c1f3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:28:07.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.920+0000 7f4180ba8700 1 --2- 172.21.15.16:0/1888225797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f417c15d290 0x7f417c0c8780 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f4174002230 tx=0x7f417400cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:07.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.920+0000 7f4178ff9700 1 -- 172.21.15.16:0/1888225797 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f417400ab70 con 0x7f417c15d290 2023-05-20T01:28:07.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.920+0000 7f4178ff9700 1 -- 172.21.15.16:0/1888225797 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f417400acf0 con 0x7f417c15d290 2023-05-20T01:28:07.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.920+0000 7f4178ff9700 1 -- 172.21.15.16:0/1888225797 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4174007310 con 0x7f417c15d290 2023-05-20T01:28:07.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.920+0000 7f418260b700 1 -- 172.21.15.16:0/1888225797 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f417c0cd2c0 con 0x7f417c15d290 2023-05-20T01:28:07.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.920+0000 7f418260b700 1 -- 172.21.15.16:0/1888225797 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f417c0ccfe0 con 0x7f417c15d290 2023-05-20T01:28:07.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.921+0000 7f418260b700 1 -- 172.21.15.16:0/1888225797 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f417c160890 con 0x7f417c15d290 2023-05-20T01:28:07.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.922+0000 7f4178ff9700 1 -- 172.21.15.16:0/1888225797 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4174007470 con 0x7f417c15d290 2023-05-20T01:28:07.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.922+0000 7f4178ff9700 1 --2- 172.21.15.16:0/1888225797 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f415c06ace0 0x7f415c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:07.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.922+0000 7f4178ff9700 1 -- 172.21.15.16:0/1888225797 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 7073+0+0 (secure 0 0 0) 0x7f41740892b0 con 0x7f417c15d290 2023-05-20T01:28:07.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.922+0000 7f417bfff700 1 --2- 172.21.15.16:0/1888225797 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f415c06ace0 0x7f415c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:07.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.922+0000 7f417bfff700 1 --2- 172.21.15.16:0/1888225797 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f415c06ace0 0x7f415c06d190 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f417c0ca320 tx=0x7f416c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:07.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:07.926+0000 7f4178ff9700 1 -- 172.21.15.16:0/1888225797 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f4174053ff0 con 0x7f417c15d290 2023-05-20T01:28:08.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.123+0000 7f418260b700 1 -- 172.21.15.16:0/1888225797 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f417c0b3570 con 0x7f417c15d290 2023-05-20T01:28:08.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.124+0000 7f4178ff9700 1 -- 172.21.15.16:0/1888225797 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v123) v1 ==== 100+0+11 (secure 0 0 0) 0x7f417400ae60 con 0x7f417c15d290 2023-05-20T01:28:08.128 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2023-05-20T01:28:08.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.126+0000 7f418260b700 1 -- 172.21.15.16:0/1888225797 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f415c06ace0 msgr2=0x7f415c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:08.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.126+0000 7f418260b700 1 --2- 172.21.15.16:0/1888225797 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f415c06ace0 0x7f415c06d190 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f417c0ca320 tx=0x7f416c008040 comp rx=0 tx=0).stop 2023-05-20T01:28:08.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.126+0000 7f418260b700 1 -- 172.21.15.16:0/1888225797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f417c15d290 msgr2=0x7f417c0c8780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:08.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.126+0000 7f418260b700 1 --2- 172.21.15.16:0/1888225797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f417c15d290 0x7f417c0c8780 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f4174002230 tx=0x7f417400cf50 comp rx=0 tx=0).stop 2023-05-20T01:28:08.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.126+0000 7f418260b700 1 -- 172.21.15.16:0/1888225797 shutdown_connections 2023-05-20T01:28:08.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.126+0000 7f418260b700 1 --2- 172.21.15.16:0/1888225797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f417c15d290 0x7f417c0c8780 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:08.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.126+0000 7f418260b700 1 --2- 172.21.15.16:0/1888225797 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f417c0cf2c0 0x7f417c150690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:08.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.126+0000 7f418260b700 1 --2- 172.21.15.16:0/1888225797 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f415c06ace0 0x7f415c06d190 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:08.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.126+0000 7f418260b700 1 --2- 172.21.15.16:0/1888225797 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f417c15cec0 0x7f417c1f3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:08.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.126+0000 7f418260b700 1 -- 172.21.15.16:0/1888225797 >> 172.21.15.16:0/1888225797 conn(0x7f417c0cffd0 msgr2=0x7f417c0cac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:08.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.126+0000 7f418260b700 1 -- 172.21.15.16:0/1888225797 shutdown_connections 2023-05-20T01:28:08.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.126+0000 7f418260b700 1 -- 172.21.15.16:0/1888225797 wait complete. 2023-05-20T01:28:08.142 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_extra pg_num 2023-05-20T01:28:08.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.265+0000 7f81fceaf700 1 Processor -- start 2023-05-20T01:28:08.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.265+0000 7f81fceaf700 1 -- start start 2023-05-20T01:28:08.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.266+0000 7f81fceaf700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f81f80cd4f0 0x7f81f80cf1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:08.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.266+0000 7f81fceaf700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f81f806a510 0x7f81f806a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:08.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.266+0000 7f81fceaf700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81f806dd00 con 0x7f81f815cec0 2023-05-20T01:28:08.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.266+0000 7f81fceaf700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81f80cd940 con 0x7f81f806a510 2023-05-20T01:28:08.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.266+0000 7f81fceaf700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81f806aed0 con 0x7f81f80cd4f0 2023-05-20T01:28:08.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.266+0000 7f81f6d9d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f81f806a510 0x7f81f806a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:08.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.266+0000 7f81f659c700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f81f815cec0 0x7f81f806e190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50996/0 (socket says 172.21.15.16:50996) 2023-05-20T01:28:08.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.266+0000 7f81f659c700 1 -- 172.21.15.16:0/4293695472 learned_addr learned my addr 172.21.15.16:0/4293695472 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:28:08.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.266+0000 7f81f5d9b700 1 --2- 172.21.15.16:0/4293695472 >> v2:172.21.15.132:3300/0 conn(0x7f81f80cd4f0 0x7f81f80cf1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:08.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.267+0000 7f81f6d9d700 1 -- 172.21.15.16:0/4293695472 >> v2:172.21.15.132:3300/0 conn(0x7f81f80cd4f0 msgr2=0x7f81f80cf1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:08.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.267+0000 7f81f6d9d700 1 --2- 172.21.15.16:0/4293695472 >> v2:172.21.15.132:3300/0 conn(0x7f81f80cd4f0 0x7f81f80cf1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:08.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.267+0000 7f81f6d9d700 1 -- 172.21.15.16:0/4293695472 >> v1:172.21.15.16:6789/0 conn(0x7f81f815cec0 legacy=0x7f81f806e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:08.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.267+0000 7f81f6d9d700 1 -- 172.21.15.16:0/4293695472 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81f80cfa60 con 0x7f81f806a510 2023-05-20T01:28:08.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.267+0000 7f81f559a700 1 -- 172.21.15.16:0/4293695472 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3939744804 0 0) 0x7f81f806dd00 con 0x7f81f815cec0 2023-05-20T01:28:08.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.267+0000 7f81f6d9d700 1 --2- 172.21.15.16:0/4293695472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81f806a510 0x7f81f806a990 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f81f80797f0 tx=0x7f81f000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12735bc5dd4f2dd0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:08.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.267+0000 7f81f559a700 1 -- 172.21.15.16:0/4293695472 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81f000e070 con 0x7f81f806a510 2023-05-20T01:28:08.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.267+0000 7f81f559a700 1 -- 172.21.15.16:0/4293695472 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81f000bbd0 con 0x7f81f806a510 2023-05-20T01:28:08.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.267+0000 7f81f559a700 1 -- 172.21.15.16:0/4293695472 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81f0012430 con 0x7f81f806a510 2023-05-20T01:28:08.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.267+0000 7f81fceaf700 1 -- 172.21.15.16:0/4293695472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81f806a510 msgr2=0x7f81f806a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:08.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.267+0000 7f81fceaf700 1 --2- 172.21.15.16:0/4293695472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81f806a510 0x7f81f806a990 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f81f80797f0 tx=0x7f81f000b330 comp rx=0 tx=0).stop 2023-05-20T01:28:08.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.267+0000 7f81fceaf700 1 -- 172.21.15.16:0/4293695472 shutdown_connections 2023-05-20T01:28:08.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.267+0000 7f81fceaf700 1 --2- 172.21.15.16:0/4293695472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81f806a510 0x7f81f806a990 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:08.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.267+0000 7f81fceaf700 1 --2- 172.21.15.16:0/4293695472 >> v2:172.21.15.132:3300/0 conn(0x7f81f80cd4f0 0x7f81f80cf1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:08.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.267+0000 7f81fceaf700 1 -- 172.21.15.16:0/4293695472 >> 172.21.15.16:0/4293695472 conn(0x7f81f806c260 msgr2=0x7f81f80cffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:08.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.267+0000 7f81fceaf700 1 -- 172.21.15.16:0/4293695472 shutdown_connections 2023-05-20T01:28:08.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.267+0000 7f81fceaf700 1 -- 172.21.15.16:0/4293695472 wait complete. 2023-05-20T01:28:08.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.269+0000 7f81fceaf700 1 Processor -- start 2023-05-20T01:28:08.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.269+0000 7f81fceaf700 1 -- start start 2023-05-20T01:28:08.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.270+0000 7f81fceaf700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f81f806a510 0x7f81f80c80f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:08.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.270+0000 7f81fceaf700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81f80cd4f0 0x7f81f81f1e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:08.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.270+0000 7f81fceaf700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f81f815cec0 0x7f81f80c49f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:08.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.270+0000 7f81fceaf700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81f815ed60 con 0x7f81f80cd4f0 2023-05-20T01:28:08.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.270+0000 7f81fceaf700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81f815f000 con 0x7f81f806a510 2023-05-20T01:28:08.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.270+0000 7f81fceaf700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81f815f2e0 con 0x7f81f815cec0 2023-05-20T01:28:08.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.270+0000 7f81f5d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81f80cd4f0 0x7f81f81f1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:08.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.270+0000 7f81f5d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81f80cd4f0 0x7f81f81f1e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39476/0 (socket says 172.21.15.16:39476) 2023-05-20T01:28:08.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.270+0000 7f81f5d9b700 1 -- 172.21.15.16:0/989241366 learned_addr learned my addr 172.21.15.16:0/989241366 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:08.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.271+0000 7f81f659c700 1 --2- 172.21.15.16:0/989241366 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f81f806a510 0x7f81f80c80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:08.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.271+0000 7f81f6d9d700 1 --2- 172.21.15.16:0/989241366 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f81f815cec0 0x7f81f80c49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:08.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.271+0000 7f81f5d9b700 1 -- 172.21.15.16:0/989241366 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f81f815cec0 msgr2=0x7f81f80c49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:08.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.271+0000 7f81f5d9b700 1 --2- 172.21.15.16:0/989241366 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f81f815cec0 0x7f81f80c49f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:08.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.271+0000 7f81f5d9b700 1 -- 172.21.15.16:0/989241366 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f81f806a510 msgr2=0x7f81f80c80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:08.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.271+0000 7f81f5d9b700 1 --2- 172.21.15.16:0/989241366 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f81f806a510 0x7f81f80c80f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:08.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.271+0000 7f81f5d9b700 1 -- 172.21.15.16:0/989241366 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81f0009a70 con 0x7f81f80cd4f0 2023-05-20T01:28:08.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.271+0000 7f81f6d9d700 1 --2- 172.21.15.16:0/989241366 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f81f815cec0 0x7f81f80c49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:28:08.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.271+0000 7f81f5d9b700 1 --2- 172.21.15.16:0/989241366 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81f80cd4f0 0x7f81f81f1e30 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f81e800e810 tx=0x7f81e800ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:08.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.271+0000 7f81e6ffd700 1 -- 172.21.15.16:0/989241366 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81e8013610 con 0x7f81f80cd4f0 2023-05-20T01:28:08.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.271+0000 7f81e6ffd700 1 -- 172.21.15.16:0/989241366 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81e8013770 con 0x7f81f80cd4f0 2023-05-20T01:28:08.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.271+0000 7f81fceaf700 1 -- 172.21.15.16:0/989241366 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81f806d990 con 0x7f81f80cd4f0 2023-05-20T01:28:08.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.271+0000 7f81e6ffd700 1 -- 172.21.15.16:0/989241366 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81e8013b20 con 0x7f81f80cd4f0 2023-05-20T01:28:08.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.271+0000 7f81fceaf700 1 -- 172.21.15.16:0/989241366 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81f806d6b0 con 0x7f81f80cd4f0 2023-05-20T01:28:08.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.272+0000 7f81fceaf700 1 -- 172.21.15.16:0/989241366 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81f815d290 con 0x7f81f80cd4f0 2023-05-20T01:28:08.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.273+0000 7f81e6ffd700 1 -- 172.21.15.16:0/989241366 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f81e8013c80 con 0x7f81f80cd4f0 2023-05-20T01:28:08.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.273+0000 7f81e6ffd700 1 --2- 172.21.15.16:0/989241366 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f81d806ace0 0x7f81d806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:08.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.273+0000 7f81e6ffd700 1 -- 172.21.15.16:0/989241366 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 7073+0+0 (secure 0 0 0) 0x7f81e80895f0 con 0x7f81f80cd4f0 2023-05-20T01:28:08.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.273+0000 7f81f659c700 1 --2- 172.21.15.16:0/989241366 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f81d806ace0 0x7f81d806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:08.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.273+0000 7f81f659c700 1 --2- 172.21.15.16:0/989241366 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f81d806ace0 0x7f81d806d190 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f81ec000f60 tx=0x7f81ec007be0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:08.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.277+0000 7f81e6ffd700 1 -- 172.21.15.16:0/989241366 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f81e8054210 con 0x7f81f80cd4f0 2023-05-20T01:28:08.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.477+0000 7f81fceaf700 1 -- 172.21.15.16:0/989241366 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"} v 0) v1 -- 0x7f81f80747b0 con 0x7f81f80cd4f0 2023-05-20T01:28:08.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.477+0000 7f81e6ffd700 1 -- 172.21.15.16:0/989241366 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"}]=0 v123) v1 ==== 106+0+11 (secure 0 0 0) 0x7f81e8057830 con 0x7f81f80cd4f0 2023-05-20T01:28:08.482 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-05-20T01:28:08.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.479+0000 7f81fceaf700 1 -- 172.21.15.16:0/989241366 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f81d806ace0 msgr2=0x7f81d806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:08.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.479+0000 7f81fceaf700 1 --2- 172.21.15.16:0/989241366 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f81d806ace0 0x7f81d806d190 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f81ec000f60 tx=0x7f81ec007be0 comp rx=0 tx=0).stop 2023-05-20T01:28:08.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.479+0000 7f81fceaf700 1 -- 172.21.15.16:0/989241366 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81f80cd4f0 msgr2=0x7f81f81f1e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:08.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.479+0000 7f81fceaf700 1 --2- 172.21.15.16:0/989241366 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81f80cd4f0 0x7f81f81f1e30 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f81e800e810 tx=0x7f81e800ebd0 comp rx=0 tx=0).stop 2023-05-20T01:28:08.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.479+0000 7f81fceaf700 1 -- 172.21.15.16:0/989241366 shutdown_connections 2023-05-20T01:28:08.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.479+0000 7f81fceaf700 1 --2- 172.21.15.16:0/989241366 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f81d806ace0 0x7f81d806d190 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:08.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.479+0000 7f81fceaf700 1 --2- 172.21.15.16:0/989241366 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f81f815cec0 0x7f81f80c49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:08.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.479+0000 7f81fceaf700 1 --2- 172.21.15.16:0/989241366 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f81f806a510 0x7f81f80c80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:08.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.479+0000 7f81fceaf700 1 --2- 172.21.15.16:0/989241366 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81f80cd4f0 0x7f81f81f1e30 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:08.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.479+0000 7f81fceaf700 1 -- 172.21.15.16:0/989241366 >> 172.21.15.16:0/989241366 conn(0x7f81f806c260 msgr2=0x7f81f806e190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:08.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.479+0000 7f81fceaf700 1 -- 172.21.15.16:0/989241366 shutdown_connections 2023-05-20T01:28:08.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.479+0000 7f81fceaf700 1 -- 172.21.15.16:0/989241366 wait complete. 2023-05-20T01:28:08.495 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-05-20T01:28:08.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.617+0000 7fde1b555700 1 Processor -- start 2023-05-20T01:28:08.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.617+0000 7fde1b555700 1 -- start start 2023-05-20T01:28:08.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.618+0000 7fde1b555700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fde1406a510 0x7fde1415e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:08.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.618+0000 7fde1b555700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fde1406a8e0 0x7fde140cf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:08.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.618+0000 7fde1b555700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fde140cfb30 0x7fde14071340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:08.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.618+0000 7fde1b555700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde1415e060 con 0x7fde140cfb30 2023-05-20T01:28:08.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.618+0000 7fde1b555700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde1415d450 con 0x7fde1406a8e0 2023-05-20T01:28:08.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.618+0000 7fde1b555700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde1415d730 con 0x7fde1406a510 2023-05-20T01:28:08.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.618+0000 7fde19af2700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fde140cfb30 0x7fde14071340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:08.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.618+0000 7fde19af2700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fde140cfb30 0x7fde14071340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39486/0 (socket says 172.21.15.16:39486) 2023-05-20T01:28:08.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.618+0000 7fde19af2700 1 -- 172.21.15.16:0/2336393717 learned_addr learned my addr 172.21.15.16:0/2336393717 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:08.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.618+0000 7fde192f1700 1 --2- 172.21.15.16:0/2336393717 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fde1406a510 0x7fde1415e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:08.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.618+0000 7fde18af0700 1 --2- 172.21.15.16:0/2336393717 >> v2:172.21.15.132:3300/0 conn(0x7fde1406a8e0 0x7fde140cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:08.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.619+0000 7fde19af2700 1 -- 172.21.15.16:0/2336393717 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fde1406a510 msgr2=0x7fde1415e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:08.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.619+0000 7fde19af2700 1 --2- 172.21.15.16:0/2336393717 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fde1406a510 0x7fde1415e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:08.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.619+0000 7fde19af2700 1 -- 172.21.15.16:0/2336393717 >> v2:172.21.15.132:3300/0 conn(0x7fde1406a8e0 msgr2=0x7fde140cf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:08.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.619+0000 7fde19af2700 1 --2- 172.21.15.16:0/2336393717 >> v2:172.21.15.132:3300/0 conn(0x7fde1406a8e0 0x7fde140cf5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:08.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.619+0000 7fde19af2700 1 -- 172.21.15.16:0/2336393717 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde1415da10 con 0x7fde140cfb30 2023-05-20T01:28:08.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.619+0000 7fde19af2700 1 --2- 172.21.15.16:0/2336393717 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde140cfb30 0x7fde14071340 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7fde14072c70 tx=0x7fde0c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c7f481a7eb1181a server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:08.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.619+0000 7fde0bfff700 1 -- 172.21.15.16:0/2336393717 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde0c00e070 con 0x7fde140cfb30 2023-05-20T01:28:08.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.619+0000 7fde0bfff700 1 -- 172.21.15.16:0/2336393717 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde0c00b9c0 con 0x7fde140cfb30 2023-05-20T01:28:08.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.619+0000 7fde0bfff700 1 -- 172.21.15.16:0/2336393717 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde0c00bd70 con 0x7fde140cfb30 2023-05-20T01:28:08.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.619+0000 7fde1b555700 1 -- 172.21.15.16:0/2336393717 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde140cfb30 msgr2=0x7fde14071340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:08.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.619+0000 7fde1b555700 1 --2- 172.21.15.16:0/2336393717 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde140cfb30 0x7fde14071340 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7fde14072c70 tx=0x7fde0c00b330 comp rx=0 tx=0).stop 2023-05-20T01:28:08.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.619+0000 7fde1b555700 1 -- 172.21.15.16:0/2336393717 shutdown_connections 2023-05-20T01:28:08.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.619+0000 7fde1b555700 1 --2- 172.21.15.16:0/2336393717 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde140cfb30 0x7fde14071340 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:08.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.619+0000 7fde1b555700 1 --2- 172.21.15.16:0/2336393717 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fde1406a510 0x7fde1415e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:08.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.619+0000 7fde1b555700 1 --2- 172.21.15.16:0/2336393717 >> v2:172.21.15.132:3300/0 conn(0x7fde1406a8e0 0x7fde140cf5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:08.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.619+0000 7fde1b555700 1 -- 172.21.15.16:0/2336393717 >> 172.21.15.16:0/2336393717 conn(0x7fde140d0c30 msgr2=0x7fde140b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:08.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.620+0000 7fde1b555700 1 -- 172.21.15.16:0/2336393717 shutdown_connections 2023-05-20T01:28:08.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.620+0000 7fde1b555700 1 -- 172.21.15.16:0/2336393717 wait complete. 2023-05-20T01:28:08.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.621+0000 7fde1b555700 1 Processor -- start 2023-05-20T01:28:08.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.621+0000 7fde1b555700 1 -- start start 2023-05-20T01:28:08.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.621+0000 7fde1b555700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fde1406a510 0x7fde140c6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:08.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.621+0000 7fde1b555700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fde1406a8e0 0x7fde1415ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:08.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.621+0000 7fde1b555700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde140cfb30 0x7fde1415b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:08.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.621+0000 7fde1b555700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde140b1790 con 0x7fde140cfb30 2023-05-20T01:28:08.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.622+0000 7fde1b555700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde140c4c60 con 0x7fde1406a510 2023-05-20T01:28:08.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.622+0000 7fde1b555700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde1415b690 con 0x7fde1406a8e0 2023-05-20T01:28:08.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.622+0000 7fde19af2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde140cfb30 0x7fde1415b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:08.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.622+0000 7fde19af2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde140cfb30 0x7fde1415b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39492/0 (socket says 172.21.15.16:39492) 2023-05-20T01:28:08.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.622+0000 7fde19af2700 1 -- 172.21.15.16:0/2091985043 learned_addr learned my addr 172.21.15.16:0/2091985043 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:08.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.622+0000 7fde192f1700 1 --2- 172.21.15.16:0/2091985043 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fde1406a510 0x7fde140c6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:08.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.622+0000 7fde18af0700 1 --2- 172.21.15.16:0/2091985043 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fde1406a8e0 0x7fde1415ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:08.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.622+0000 7fde19af2700 1 -- 172.21.15.16:0/2091985043 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fde1406a8e0 msgr2=0x7fde1415ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:08.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.622+0000 7fde19af2700 1 --2- 172.21.15.16:0/2091985043 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fde1406a8e0 0x7fde1415ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:08.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.622+0000 7fde19af2700 1 -- 172.21.15.16:0/2091985043 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fde1406a510 msgr2=0x7fde140c6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:08.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.622+0000 7fde19af2700 1 --2- 172.21.15.16:0/2091985043 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fde1406a510 0x7fde140c6160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:08.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.622+0000 7fde19af2700 1 -- 172.21.15.16:0/2091985043 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde0c009b30 con 0x7fde140cfb30 2023-05-20T01:28:08.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.622+0000 7fde18af0700 1 --2- 172.21.15.16:0/2091985043 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fde1406a8e0 0x7fde1415ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:28:08.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.622+0000 7fde19af2700 1 --2- 172.21.15.16:0/2091985043 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde140cfb30 0x7fde1415b150 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7fde0c00b730 tx=0x7fde0c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:08.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.622+0000 7fde09ffb700 1 -- 172.21.15.16:0/2091985043 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde0c0099b0 con 0x7fde140cfb30 2023-05-20T01:28:08.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.622+0000 7fde09ffb700 1 -- 172.21.15.16:0/2091985043 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde0c004250 con 0x7fde140cfb30 2023-05-20T01:28:08.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.622+0000 7fde1b555700 1 -- 172.21.15.16:0/2091985043 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde1406ad50 con 0x7fde140cfb30 2023-05-20T01:28:08.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.622+0000 7fde09ffb700 1 -- 172.21.15.16:0/2091985043 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde0c004600 con 0x7fde140cfb30 2023-05-20T01:28:08.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.622+0000 7fde1b555700 1 -- 172.21.15.16:0/2091985043 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde1415dcf0 con 0x7fde140cfb30 2023-05-20T01:28:08.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.623+0000 7fde1b555700 1 -- 172.21.15.16:0/2091985043 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde1415d080 con 0x7fde140cfb30 2023-05-20T01:28:08.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.624+0000 7fde09ffb700 1 -- 172.21.15.16:0/2091985043 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fde0c004760 con 0x7fde140cfb30 2023-05-20T01:28:08.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.625+0000 7fde09ffb700 1 --2- 172.21.15.16:0/2091985043 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fddf806ace0 0x7fddf806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:08.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.625+0000 7fde09ffb700 1 -- 172.21.15.16:0/2091985043 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 7073+0+0 (secure 0 0 0) 0x7fde0c08b640 con 0x7fde140cfb30 2023-05-20T01:28:08.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.625+0000 7fde192f1700 1 --2- 172.21.15.16:0/2091985043 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fddf806ace0 0x7fddf806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:08.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.625+0000 7fde192f1700 1 --2- 172.21.15.16:0/2091985043 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fddf806ace0 0x7fddf806d190 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fde040045f0 tx=0x7fde04008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:08.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.628+0000 7fde09ffb700 1 -- 172.21.15.16:0/2091985043 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fde0c056580 con 0x7fde140cfb30 2023-05-20T01:28:08.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.843+0000 7fde1b555700 1 -- 172.21.15.16:0/2091985043 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fde140c4e10 con 0x7fde140cfb30 2023-05-20T01:28:08.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.844+0000 7fde09ffb700 1 -- 172.21.15.16:0/2091985043 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 6 entries v123) v1 ==== 80+0+366 (secure 0 0 0) 0x7fde140c4e10 con 0x7fde140cfb30 2023-05-20T01:28:08.848 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6836/1584803225 2023-05-21T01:27:21.602047+0000 2023-05-20T01:28:08.849 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6834/1584803225 2023-05-21T01:27:21.602047+0000 2023-05-20T01:28:08.849 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6836/1281052551 2023-05-21T01:27:10.544120+0000 2023-05-20T01:28:08.849 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6834/1281052551 2023-05-21T01:27:10.544120+0000 2023-05-20T01:28:08.849 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6836/3765848068 2023-05-21T01:27:09.119676+0000 2023-05-20T01:28:08.849 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6834/3765848068 2023-05-21T01:27:09.119676+0000 2023-05-20T01:28:08.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.846+0000 7fde1b555700 1 -- 172.21.15.16:0/2091985043 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fddf806ace0 msgr2=0x7fddf806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:08.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.846+0000 7fde1b555700 1 --2- 172.21.15.16:0/2091985043 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fddf806ace0 0x7fddf806d190 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fde040045f0 tx=0x7fde04008040 comp rx=0 tx=0).stop 2023-05-20T01:28:08.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.846+0000 7fde1b555700 1 -- 172.21.15.16:0/2091985043 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde140cfb30 msgr2=0x7fde1415b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:08.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.846+0000 7fde1b555700 1 --2- 172.21.15.16:0/2091985043 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde140cfb30 0x7fde1415b150 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7fde0c00b730 tx=0x7fde0c004ad0 comp rx=0 tx=0).stop 2023-05-20T01:28:08.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.846+0000 7fde1b555700 1 -- 172.21.15.16:0/2091985043 shutdown_connections 2023-05-20T01:28:08.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.846+0000 7fde1b555700 1 --2- 172.21.15.16:0/2091985043 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde140cfb30 0x7fde1415b150 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:08.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.846+0000 7fde1b555700 1 --2- 172.21.15.16:0/2091985043 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fde1406a510 0x7fde140c6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:08.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.846+0000 7fde1b555700 1 --2- 172.21.15.16:0/2091985043 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fddf806ace0 0x7fddf806d190 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:08.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.846+0000 7fde1b555700 1 --2- 172.21.15.16:0/2091985043 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fde1406a8e0 0x7fde1415ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:08.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.846+0000 7fde1b555700 1 -- 172.21.15.16:0/2091985043 >> 172.21.15.16:0/2091985043 conn(0x7fde140d0c30 msgr2=0x7fde140ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:08.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.846+0000 7fde1b555700 1 -- 172.21.15.16:0/2091985043 shutdown_connections 2023-05-20T01:28:08.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:08.846+0000 7fde1b555700 1 -- 172.21.15.16:0/2091985043 wait complete. 2023-05-20T01:28:08.852 INFO:teuthology.orchestra.run.smithi016.stderr:listed 6 entries 2023-05-20T01:28:08.862 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-05-20T01:28:08.862 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:28:08.863 DEBUG:teuthology.orchestra.run.smithi016:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-05-20T01:28:08.903 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2023-05-20T01:28:08.904 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:28:08.905 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-05-20T01:28:08.961 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2023-05-20T01:28:08.961 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/proc/self/mounts of=/dev/stdout 2023-05-20T01:28:08.982 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284284k,nr_inodes=4071071,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=45,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16175 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260556k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-05-20T01:28:08.983 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-05-20T01:28:08.983 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2023-05-20T01:28:09.018 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16287364k,nr_inodes=4071841,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=38,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16696 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3261172k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-05-20T01:28:09.018 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2023-05-20T01:28:09.019 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/proc/self/mounts of=/dev/stdout 2023-05-20T01:28:09.046 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284284k,nr_inodes=4071071,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=45,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16175 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260556k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-05-20T01:28:09.047 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-20T01:28:09.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.178+0000 7effbb9ce700 1 Processor -- start 2023-05-20T01:28:09.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.178+0000 7effbb9ce700 1 -- start start 2023-05-20T01:28:09.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.179+0000 7effbb9ce700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7effb406d4b0 0x7effb40cf890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:09.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.179+0000 7effbb9ce700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7effb406a510 0x7effb406a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:09.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.179+0000 7effbb9ce700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effb40cc510 con 0x7effb406f450 2023-05-20T01:28:09.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.179+0000 7effbb9ce700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effb406aef0 con 0x7effb406a510 2023-05-20T01:28:09.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.179+0000 7effbb9ce700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effb4071ed0 con 0x7effb406d4b0 2023-05-20T01:28:09.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.179+0000 7effb9f6b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7effb406a510 0x7effb406a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:09.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.179+0000 7effb976a700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7effb406f450 0x7effb4161080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51024/0 (socket says 172.21.15.16:51024) 2023-05-20T01:28:09.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.179+0000 7effb976a700 1 -- 172.21.15.16:0/660864485 learned_addr learned my addr 172.21.15.16:0/660864485 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:28:09.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.179+0000 7effb8f69700 1 --2- 172.21.15.16:0/660864485 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7effb406d4b0 0x7effb40cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:09.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.179+0000 7effb9f6b700 1 -- 172.21.15.16:0/660864485 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7effb406d4b0 msgr2=0x7effb40cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:09.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.179+0000 7effb9f6b700 1 --2- 172.21.15.16:0/660864485 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7effb406d4b0 0x7effb40cf890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:09.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.179+0000 7effb9f6b700 1 -- 172.21.15.16:0/660864485 >> v1:172.21.15.16:6789/0 conn(0x7effb406f450 legacy=0x7effb4161080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:09.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.179+0000 7effb9f6b700 1 -- 172.21.15.16:0/660864485 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effb40721b0 con 0x7effb406a510 2023-05-20T01:28:09.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.179+0000 7effabfff700 1 -- 172.21.15.16:0/660864485 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1848295428 0 0) 0x7effb40cc510 con 0x7effb406f450 2023-05-20T01:28:09.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.179+0000 7effb8f69700 1 --2- 172.21.15.16:0/660864485 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7effb406d4b0 0x7effb40cf890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:28:09.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.179+0000 7effb9f6b700 1 --2- 172.21.15.16:0/660864485 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7effb406a510 0x7effb406a9b0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7effac009d90 tx=0x7effac00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9eb4eb3c74df9fe server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:09.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.180+0000 7effabfff700 1 -- 172.21.15.16:0/660864485 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effac00e070 con 0x7effb406a510 2023-05-20T01:28:09.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.180+0000 7effabfff700 1 -- 172.21.15.16:0/660864485 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effac00bbd0 con 0x7effb406a510 2023-05-20T01:28:09.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.180+0000 7effabfff700 1 -- 172.21.15.16:0/660864485 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effac012430 con 0x7effb406a510 2023-05-20T01:28:09.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.180+0000 7effbb9ce700 1 -- 172.21.15.16:0/660864485 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7effb406a510 msgr2=0x7effb406a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:09.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.180+0000 7effbb9ce700 1 --2- 172.21.15.16:0/660864485 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7effb406a510 0x7effb406a9b0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7effac009d90 tx=0x7effac00b330 comp rx=0 tx=0).stop 2023-05-20T01:28:09.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.180+0000 7effbb9ce700 1 -- 172.21.15.16:0/660864485 shutdown_connections 2023-05-20T01:28:09.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.180+0000 7effbb9ce700 1 --2- 172.21.15.16:0/660864485 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7effb406a510 0x7effb406a9b0 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:09.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.180+0000 7effbb9ce700 1 --2- 172.21.15.16:0/660864485 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7effb406d4b0 0x7effb40cf890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:09.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.180+0000 7effbb9ce700 1 -- 172.21.15.16:0/660864485 >> 172.21.15.16:0/660864485 conn(0x7effb40cffb0 msgr2=0x7effb40d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:09.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.180+0000 7effbb9ce700 1 -- 172.21.15.16:0/660864485 shutdown_connections 2023-05-20T01:28:09.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.180+0000 7effbb9ce700 1 -- 172.21.15.16:0/660864485 wait complete. 2023-05-20T01:28:09.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.182+0000 7effbb9ce700 1 Processor -- start 2023-05-20T01:28:09.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.182+0000 7effbb9ce700 1 -- start start 2023-05-20T01:28:09.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.183+0000 7effbb9ce700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7effb406a510 0x7effb414e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:09.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.183+0000 7effbb9ce700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7effb406d4b0 0x7effb41f24d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:09.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.183+0000 7effbb9ce700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7effb406f450 0x7effb40b4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:09.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.183+0000 7effbb9ce700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effb415b9e0 con 0x7effb406d4b0 2023-05-20T01:28:09.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.183+0000 7effbb9ce700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effb414ea70 con 0x7effb406a510 2023-05-20T01:28:09.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.183+0000 7effbb9ce700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effb41e3d50 con 0x7effb406f450 2023-05-20T01:28:09.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.183+0000 7effb8f69700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7effb406d4b0 0x7effb41f24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:09.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.183+0000 7effb8f69700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7effb406d4b0 0x7effb41f24d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39504/0 (socket says 172.21.15.16:39504) 2023-05-20T01:28:09.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.183+0000 7effb8f69700 1 -- 172.21.15.16:0/3397421719 learned_addr learned my addr 172.21.15.16:0/3397421719 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:09.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.183+0000 7effb9f6b700 1 --2- 172.21.15.16:0/3397421719 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7effb406f450 0x7effb40b4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:09.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.183+0000 7effb976a700 1 --2- 172.21.15.16:0/3397421719 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7effb406a510 0x7effb414e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:09.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.183+0000 7effb8f69700 1 -- 172.21.15.16:0/3397421719 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7effb406f450 msgr2=0x7effb40b4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:09.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.183+0000 7effb8f69700 1 --2- 172.21.15.16:0/3397421719 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7effb406f450 0x7effb40b4880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:09.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.183+0000 7effb8f69700 1 -- 172.21.15.16:0/3397421719 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7effb406a510 msgr2=0x7effb414e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:09.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.183+0000 7effb8f69700 1 --2- 172.21.15.16:0/3397421719 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7effb406a510 0x7effb414e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:09.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.183+0000 7effb8f69700 1 -- 172.21.15.16:0/3397421719 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effac009a70 con 0x7effb406d4b0 2023-05-20T01:28:09.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.183+0000 7effb976a700 1 --2- 172.21.15.16:0/3397421719 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7effb406a510 0x7effb414e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:28:09.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.183+0000 7effb8f69700 1 --2- 172.21.15.16:0/3397421719 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7effb406d4b0 0x7effb41f24d0 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7effa400e810 tx=0x7effa400ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:09.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.184+0000 7effa9ffb700 1 -- 172.21.15.16:0/3397421719 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effa4004230 con 0x7effb406d4b0 2023-05-20T01:28:09.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.184+0000 7effa9ffb700 1 -- 172.21.15.16:0/3397421719 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effa4004390 con 0x7effb406d4b0 2023-05-20T01:28:09.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.184+0000 7effa9ffb700 1 -- 172.21.15.16:0/3397421719 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effa40036a0 con 0x7effb406d4b0 2023-05-20T01:28:09.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.184+0000 7effbb9ce700 1 -- 172.21.15.16:0/3397421719 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effb40cbe50 con 0x7effb406d4b0 2023-05-20T01:28:09.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.184+0000 7effbb9ce700 1 -- 172.21.15.16:0/3397421719 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effb40cc1a0 con 0x7effb406d4b0 2023-05-20T01:28:09.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.185+0000 7effbb9ce700 1 -- 172.21.15.16:0/3397421719 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effb40cc9a0 con 0x7effb406d4b0 2023-05-20T01:28:09.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.185+0000 7effa9ffb700 1 -- 172.21.15.16:0/3397421719 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7effa4003800 con 0x7effb406d4b0 2023-05-20T01:28:09.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.185+0000 7effa9ffb700 1 --2- 172.21.15.16:0/3397421719 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7eff9806ace0 0x7eff9806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:09.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.186+0000 7effa9ffb700 1 -- 172.21.15.16:0/3397421719 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 7073+0+0 (secure 0 0 0) 0x7effa40893c0 con 0x7effb406d4b0 2023-05-20T01:28:09.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.186+0000 7effb976a700 1 --2- 172.21.15.16:0/3397421719 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7eff9806ace0 0x7eff9806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:09.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.186+0000 7effb976a700 1 --2- 172.21.15.16:0/3397421719 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7eff9806ace0 0x7eff9806d190 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7effb0007fd0 tx=0x7effb0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:09.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.190+0000 7effa9ffb700 1 -- 172.21.15.16:0/3397421719 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7effa4054070 con 0x7effb406d4b0 2023-05-20T01:28:09.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.430+0000 7effbb9ce700 1 -- 172.21.15.16:0/3397421719 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7effb41e4c40 con 0x7effb406d4b0 2023-05-20T01:28:09.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.430+0000 7effa9ffb700 1 -- 172.21.15.16:0/3397421719 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+4681 (secure 0 0 0) 0x7effa4057690 con 0x7effb406d4b0 2023-05-20T01:28:09.435 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:28:09.435 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":91,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6845,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/1435235487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1435235487},{"type":"v1","addr":"172.21.15.132:6834","nonce":1435235487}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":87},{"gid":6848,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/933893285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":933893285},{"type":"v1","addr":"172.21.15.132:6835","nonce":933893285}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":87},{"gid":8373,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/1220815342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1220815342},{"type":"v1","addr":"172.21.15.16:6836","nonce":1220815342}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":87}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-20T01:27:10.291795+0000","modified":"2023-05-20T01:27:25.446248+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":121,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8376},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8376":{"gid":8376,"name":"c","rank":0,"incarnation":88,"state":"up:active","state_seq":4,"addr":"172.21.15.16:6837/1927078882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":1927078882},{"type":"v1","addr":"172.21.15.16:6837","nonce":1927078882}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16,17],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-05-20T01:28:09.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.432+0000 7effbb9ce700 1 -- 172.21.15.16:0/3397421719 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7eff9806ace0 msgr2=0x7eff9806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:09.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.432+0000 7effbb9ce700 1 --2- 172.21.15.16:0/3397421719 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7eff9806ace0 0x7eff9806d190 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7effb0007fd0 tx=0x7effb0009040 comp rx=0 tx=0).stop 2023-05-20T01:28:09.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.432+0000 7effbb9ce700 1 -- 172.21.15.16:0/3397421719 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7effb406d4b0 msgr2=0x7effb41f24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:09.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.432+0000 7effbb9ce700 1 --2- 172.21.15.16:0/3397421719 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7effb406d4b0 0x7effb41f24d0 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7effa400e810 tx=0x7effa400ebd0 comp rx=0 tx=0).stop 2023-05-20T01:28:09.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.432+0000 7effbb9ce700 1 -- 172.21.15.16:0/3397421719 shutdown_connections 2023-05-20T01:28:09.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.432+0000 7effbb9ce700 1 --2- 172.21.15.16:0/3397421719 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7eff9806ace0 0x7eff9806d190 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:09.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.432+0000 7effbb9ce700 1 --2- 172.21.15.16:0/3397421719 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7effb406f450 0x7effb40b4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:09.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.432+0000 7effbb9ce700 1 --2- 172.21.15.16:0/3397421719 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7effb406a510 0x7effb414e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:09.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.432+0000 7effbb9ce700 1 --2- 172.21.15.16:0/3397421719 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7effb406d4b0 0x7effb41f24d0 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:09.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.432+0000 7effbb9ce700 1 -- 172.21.15.16:0/3397421719 >> 172.21.15.16:0/3397421719 conn(0x7effb40cffb0 msgr2=0x7effb40c98e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:09.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.433+0000 7effbb9ce700 1 -- 172.21.15.16:0/3397421719 shutdown_connections 2023-05-20T01:28:09.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.433+0000 7effbb9ce700 1 -- 172.21.15.16:0/3397421719 wait complete. 2023-05-20T01:28:09.439 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 91 2023-05-20T01:28:09.449 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-20T01:28:09.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.570+0000 7f8ac20a4700 1 Processor -- start 2023-05-20T01:28:09.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.570+0000 7f8ac20a4700 1 -- start start 2023-05-20T01:28:09.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.571+0000 7f8ac20a4700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8abc144020 0x7f8abc0d1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:09.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.571+0000 7f8ac20a4700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8abc0cc500 con 0x7f8abc0cec60 2023-05-20T01:28:09.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.571+0000 7f8ac20a4700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8abc0cb8f0 con 0x7f8abc0b2760 2023-05-20T01:28:09.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.571+0000 7f8ac20a4700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8abc0cb6e0 con 0x7f8abc144020 2023-05-20T01:28:09.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.571+0000 7f8abbfff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f8abc0cec60 0x7f8abc0cc990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51042/0 (socket says 172.21.15.16:51042) 2023-05-20T01:28:09.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.571+0000 7f8abbfff700 1 -- 172.21.15.16:0/1000226206 learned_addr learned my addr 172.21.15.16:0/1000226206 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:28:09.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.571+0000 7f8abaffd700 1 --2- 172.21.15.16:0/1000226206 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8abc144020 0x7f8abc0d1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:09.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.571+0000 7f8aba7fc700 1 -- 172.21.15.16:0/1000226206 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3114631410 0 0) 0x7f8abc0cc500 con 0x7f8abc0cec60 2023-05-20T01:28:09.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.571+0000 7f8aba7fc700 1 -- 172.21.15.16:0/1000226206 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8aa80035b0 con 0x7f8abc0cec60 2023-05-20T01:28:09.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.571+0000 7f8aba7fc700 1 -- 172.21.15.16:0/1000226206 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 874354861 0 0) 0x7f8aa80035b0 con 0x7f8abc0cec60 2023-05-20T01:28:09.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.571+0000 7f8aba7fc700 1 -- 172.21.15.16:0/1000226206 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8abc144020 msgr2=0x7f8abc0d1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:09.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.571+0000 7f8aba7fc700 1 --2- 172.21.15.16:0/1000226206 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8abc144020 0x7f8abc0d1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:09.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.571+0000 7f8aba7fc700 1 -- 172.21.15.16:0/1000226206 >> v1:172.21.15.132:6789/0 conn(0x7f8abc0b2760 legacy=0x7f8abc0b0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:09.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.572+0000 7f8aba7fc700 1 -- 172.21.15.16:0/1000226206 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8abc0cbbd0 con 0x7f8abc0cec60 2023-05-20T01:28:09.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.572+0000 7f8aba7fc700 1 -- 172.21.15.16:0/1000226206 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7f8ab40019f0 con 0x7f8abc0cec60 2023-05-20T01:28:09.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.572+0000 7f8aba7fc700 1 -- 172.21.15.16:0/1000226206 >> v1:172.21.15.16:6789/0 conn(0x7f8abc0cec60 legacy=0x7f8abc0cc990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:09.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.572+0000 7f8aba7fc700 1 --2- 172.21.15.16:0/1000226206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8aa8003aa0 0x7f8aa8003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:09.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.572+0000 7f8aba7fc700 1 --2- 172.21.15.16:0/1000226206 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8aa8004850 0x7f8aa8004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:09.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.572+0000 7f8aba7fc700 1 --2- 172.21.15.16:0/1000226206 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8abc144020 0x7f8aa8005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:09.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.572+0000 7f8aba7fc700 1 -- 172.21.15.16:0/1000226206 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8abc0cb6e0 con 0x7f8aa8003aa0 2023-05-20T01:28:09.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.572+0000 7f8aba7fc700 1 -- 172.21.15.16:0/1000226206 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa800bac0 con 0x7f8aa8004850 2023-05-20T01:28:09.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.572+0000 7f8aba7fc700 1 -- 172.21.15.16:0/1000226206 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa800bc30 con 0x7f8abc144020 2023-05-20T01:28:09.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.572+0000 7f8abb7fe700 1 --2- 172.21.15.16:0/1000226206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8aa8003aa0 0x7f8aa8003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:09.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.572+0000 7f8abaffd700 1 --2- 172.21.15.16:0/1000226206 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8aa8004850 0x7f8aa8004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:09.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.572+0000 7f8abbfff700 1 --2- 172.21.15.16:0/1000226206 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8abc144020 0x7f8aa8005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:09.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.572+0000 7f8abb7fe700 1 -- 172.21.15.16:0/1000226206 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8abc144020 msgr2=0x7f8aa8005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:09.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.572+0000 7f8abb7fe700 1 --2- 172.21.15.16:0/1000226206 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8abc144020 0x7f8aa8005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:09.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.572+0000 7f8abb7fe700 1 -- 172.21.15.16:0/1000226206 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8aa8004850 msgr2=0x7f8aa8004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:09.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.572+0000 7f8abb7fe700 1 --2- 172.21.15.16:0/1000226206 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8aa8004850 0x7f8aa8004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:09.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.572+0000 7f8abb7fe700 1 -- 172.21.15.16:0/1000226206 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8aa80035b0 con 0x7f8aa8003aa0 2023-05-20T01:28:09.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.572+0000 7f8abbfff700 1 --2- 172.21.15.16:0/1000226206 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8abc144020 0x7f8aa8005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:28:09.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.572+0000 7f8abb7fe700 1 --2- 172.21.15.16:0/1000226206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8aa8003aa0 0x7f8aa8003e70 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f8aac0018d0 tx=0x7f8aa8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=909f1955b7f6731a server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:09.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.572+0000 7f8aba7fc700 1 -- 172.21.15.16:0/1000226206 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aac00c690 con 0x7f8aa8003aa0 2023-05-20T01:28:09.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.572+0000 7f8aba7fc700 1 -- 172.21.15.16:0/1000226206 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8aac00c810 con 0x7f8aa8003aa0 2023-05-20T01:28:09.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.572+0000 7f8aba7fc700 1 -- 172.21.15.16:0/1000226206 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aac00cbc0 con 0x7f8aa8003aa0 2023-05-20T01:28:09.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.573+0000 7f8ac20a4700 1 -- 172.21.15.16:0/1000226206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8aa8003aa0 msgr2=0x7f8aa8003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:09.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.573+0000 7f8ac20a4700 1 --2- 172.21.15.16:0/1000226206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8aa8003aa0 0x7f8aa8003e70 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f8aac0018d0 tx=0x7f8aa8005880 comp rx=0 tx=0).stop 2023-05-20T01:28:09.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.573+0000 7f8abbfff700 1 -- 172.21.15.16:0/1000226206 reap_dead start 2023-05-20T01:28:09.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.573+0000 7f8ac20a4700 1 -- 172.21.15.16:0/1000226206 shutdown_connections 2023-05-20T01:28:09.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.573+0000 7f8ac20a4700 1 -- 172.21.15.16:0/1000226206 >> 172.21.15.16:0/1000226206 conn(0x7f8abc06ae30 msgr2=0x7f8abc09a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:09.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.573+0000 7f8ac20a4700 1 -- 172.21.15.16:0/1000226206 shutdown_connections 2023-05-20T01:28:09.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.573+0000 7f8ac20a4700 1 -- 172.21.15.16:0/1000226206 wait complete. 2023-05-20T01:28:09.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.575+0000 7f8ac20a4700 1 Processor -- start 2023-05-20T01:28:09.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.575+0000 7f8ac20a4700 1 -- start start 2023-05-20T01:28:09.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.576+0000 7f8ac20a4700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8aa8003aa0 0x7f8abc15b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:09.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.576+0000 7f8ac20a4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8abc0b3ec0 0x7f8abc1f3280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:09.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.576+0000 7f8ac20a4700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8abc15be70 0x7f8abc15c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:09.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.576+0000 7f8ac20a4700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8abc15c850 con 0x7f8abc0b3ec0 2023-05-20T01:28:09.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.576+0000 7f8ac20a4700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8abc1f37c0 con 0x7f8abc15be70 2023-05-20T01:28:09.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.576+0000 7f8ac20a4700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8abc15b3f0 con 0x7f8aa8003aa0 2023-05-20T01:28:09.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.576+0000 7f8abaffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8abc0b3ec0 0x7f8abc1f3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:09.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.576+0000 7f8abaffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8abc0b3ec0 0x7f8abc1f3280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39524/0 (socket says 172.21.15.16:39524) 2023-05-20T01:28:09.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.576+0000 7f8abaffd700 1 -- 172.21.15.16:0/683900198 learned_addr learned my addr 172.21.15.16:0/683900198 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:09.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.576+0000 7f8abb7fe700 1 --2- 172.21.15.16:0/683900198 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8aa8003aa0 0x7f8abc15b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:09.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.576+0000 7f8abbfff700 1 --2- 172.21.15.16:0/683900198 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8abc15be70 0x7f8abc15c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:09.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.576+0000 7f8abaffd700 1 -- 172.21.15.16:0/683900198 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8aa8003aa0 msgr2=0x7f8abc15b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:09.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.576+0000 7f8abaffd700 1 --2- 172.21.15.16:0/683900198 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8aa8003aa0 0x7f8abc15b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:09.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.576+0000 7f8abaffd700 1 -- 172.21.15.16:0/683900198 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8abc15be70 msgr2=0x7f8abc15c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:09.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.576+0000 7f8abaffd700 1 --2- 172.21.15.16:0/683900198 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8abc15be70 0x7f8abc15c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:09.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.576+0000 7f8abaffd700 1 -- 172.21.15.16:0/683900198 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8abc0cc190 con 0x7f8abc0b3ec0 2023-05-20T01:28:09.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.576+0000 7f8abaffd700 1 --2- 172.21.15.16:0/683900198 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8abc0b3ec0 0x7f8abc1f3280 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f8aa4011ad0 tx=0x7f8aa4011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:09.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.576+0000 7f8ab3fff700 1 -- 172.21.15.16:0/683900198 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa400fa00 con 0x7f8abc0b3ec0 2023-05-20T01:28:09.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.577+0000 7f8ab3fff700 1 -- 172.21.15.16:0/683900198 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8aa400fb60 con 0x7f8abc0b3ec0 2023-05-20T01:28:09.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.577+0000 7f8ac20a4700 1 -- 172.21.15.16:0/683900198 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8abc0cbeb0 con 0x7f8abc0b3ec0 2023-05-20T01:28:09.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.577+0000 7f8ab3fff700 1 -- 172.21.15.16:0/683900198 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa4014440 con 0x7f8abc0b3ec0 2023-05-20T01:28:09.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.577+0000 7f8ac20a4700 1 -- 172.21.15.16:0/683900198 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8abc0d1680 con 0x7f8abc0b3ec0 2023-05-20T01:28:09.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.578+0000 7f8ac20a4700 1 -- 172.21.15.16:0/683900198 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8abc0b54a0 con 0x7f8abc0b3ec0 2023-05-20T01:28:09.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.578+0000 7f8ab3fff700 1 -- 172.21.15.16:0/683900198 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8aa40145a0 con 0x7f8abc0b3ec0 2023-05-20T01:28:09.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.578+0000 7f8ab3fff700 1 --2- 172.21.15.16:0/683900198 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f8a9c06ace0 0x7f8a9c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:09.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.578+0000 7f8ab3fff700 1 -- 172.21.15.16:0/683900198 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 7073+0+0 (secure 0 0 0) 0x7f8aa4089f10 con 0x7f8abc0b3ec0 2023-05-20T01:28:09.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.578+0000 7f8abb7fe700 1 --2- 172.21.15.16:0/683900198 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f8a9c06ace0 0x7f8a9c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:09.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.579+0000 7f8abb7fe700 1 --2- 172.21.15.16:0/683900198 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f8a9c06ace0 0x7f8a9c06d190 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f8aac014040 tx=0x7f8aac005040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:09.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.583+0000 7f8ab3fff700 1 -- 172.21.15.16:0/683900198 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f8abc0b54a0 con 0x7f8abc0b3ec0 2023-05-20T01:28:09.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.784+0000 7f8ac20a4700 1 -- 172.21.15.16:0/683900198 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8abc0b54a0 con 0x7f8abc0b3ec0 2023-05-20T01:28:09.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.786+0000 7f8ab3fff700 1 -- 172.21.15.16:0/683900198 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v123) v1 ==== 74+0+20629 (secure 0 0 0) 0x7f8abc0b54a0 con 0x7f8abc0b3ec0 2023-05-20T01:28:09.790 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:28:09.791 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":123,"fsid":"c2245cc5-ecab-4087-a59b-bc81db901676","created":"2023-05-20T01:16:09.549230+0000","modified":"2023-05-20T01:27:21.615250+0000","last_up_change":"2023-05-20T01:16:19.788624+0000","last_in_change":"2023-05-20T01:16:12.092160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-05-20T01:16:20.934366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-05-20T01:16:21.372294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-05-20T01:26:34.067017+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-05-20T01:26:35.040602+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data_extra","create_time":"2023-05-20T01:26:48.309738+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"317a83cd-19c8-4958-a301-6087ef342289","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6825","nonce":3923634179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6827","nonce":3923634179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6831","nonce":3923634179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6829","nonce":3923634179}]},"public_addr":"172.21.15.16:6825/3923634179","cluster_addr":"172.21.15.16:6827/3923634179","heartbeat_back_addr":"172.21.15.16:6831/3923634179","heartbeat_front_addr":"172.21.15.16:6829/3923634179","state":["exists","up"]},{"osd":1,"uuid":"5581046f-8219-4089-8d7d-b7846399505e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6809","nonce":3860264220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6811","nonce":3860264220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6815","nonce":3860264220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6813","nonce":3860264220}]},"public_addr":"172.21.15.16:6809/3860264220","cluster_addr":"172.21.15.16:6811/3860264220","heartbeat_back_addr":"172.21.15.16:6815/3860264220","heartbeat_front_addr":"172.21.15.16:6813/3860264220","state":["exists","up"]},{"osd":2,"uuid":"072e9e66-4133-478e-8d4a-64d46c2963f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6817","nonce":293187030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6819","nonce":293187030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6823","nonce":293187030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6821","nonce":293187030}]},"public_addr":"172.21.15.16:6817/293187030","cluster_addr":"172.21.15.16:6819/293187030","heartbeat_back_addr":"172.21.15.16:6823/293187030","heartbeat_front_addr":"172.21.15.16:6821/293187030","state":["exists","up"]},{"osd":3,"uuid":"11e1bba6-00f8-48dd-9028-d8d716b7f145","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6801","nonce":3669781890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6803","nonce":3669781890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6807","nonce":3669781890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6805","nonce":3669781890}]},"public_addr":"172.21.15.16:6801/3669781890","cluster_addr":"172.21.15.16:6803/3669781890","heartbeat_back_addr":"172.21.15.16:6807/3669781890","heartbeat_front_addr":"172.21.15.16:6805/3669781890","state":["exists","up"]},{"osd":4,"uuid":"b8909c3a-cf70-46cb-8d0b-009c707cf9d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6819","nonce":506317223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6823","nonce":506317223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6829","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6831","nonce":506317223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6827","nonce":506317223}]},"public_addr":"172.21.15.132:6819/506317223","cluster_addr":"172.21.15.132:6823/506317223","heartbeat_back_addr":"172.21.15.132:6831/506317223","heartbeat_front_addr":"172.21.15.132:6827/506317223","state":["exists","up"]},{"osd":5,"uuid":"b1818304-4583-47bf-a6ec-0bfbb5166233","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6818","nonce":1844689995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6822","nonce":1844689995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6830","nonce":1844689995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6826","nonce":1844689995}]},"public_addr":"172.21.15.132:6818/1844689995","cluster_addr":"172.21.15.132:6822/1844689995","heartbeat_back_addr":"172.21.15.132:6830/1844689995","heartbeat_front_addr":"172.21.15.132:6826/1844689995","state":["exists","up"]},{"osd":6,"uuid":"d37f4691-eed1-4a93-8818-18354f03f4a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6801","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6802","nonce":727581123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6805","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6807","nonce":727581123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6813","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6815","nonce":727581123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6811","nonce":727581123}]},"public_addr":"172.21.15.132:6802/727581123","cluster_addr":"172.21.15.132:6807/727581123","heartbeat_back_addr":"172.21.15.132:6815/727581123","heartbeat_front_addr":"172.21.15.132:6811/727581123","state":["exists","up"]},{"osd":7,"uuid":"4f712f62-5d1d-4816-8042-03209144f8b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6803","nonce":466001149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6806","nonce":466001149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6814","nonce":466001149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6810","nonce":466001149}]},"public_addr":"172.21.15.132:6803/466001149","cluster_addr":"172.21.15.132:6806/466001149","heartbeat_back_addr":"172.21.15.132:6814/466001149","heartbeat_front_addr":"172.21.15.132:6810/466001149","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.338863+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.278680+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305694+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.246835+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.276237+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.304441+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305421+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.267050+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.33","mappings":[{"from":5,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6836/1584803225":"2023-05-21T01:27:21.602047+0000","172.21.15.16:6834/1584803225":"2023-05-21T01:27:21.602047+0000","172.21.15.16:6836/1281052551":"2023-05-21T01:27:10.544120+0000","172.21.15.16:6834/1281052551":"2023-05-21T01:27:10.544120+0000","172.21.15.16:6836/3765848068":"2023-05-21T01:27:09.119676+0000","172.21.15.16:6834/3765848068":"2023-05-21T01:27:09.119676+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-20T01:28:09.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.788+0000 7f8ac20a4700 1 -- 172.21.15.16:0/683900198 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f8a9c06ace0 msgr2=0x7f8a9c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:09.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.788+0000 7f8ac20a4700 1 --2- 172.21.15.16:0/683900198 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f8a9c06ace0 0x7f8a9c06d190 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f8aac014040 tx=0x7f8aac005040 comp rx=0 tx=0).stop 2023-05-20T01:28:09.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.788+0000 7f8ac20a4700 1 -- 172.21.15.16:0/683900198 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8abc0b3ec0 msgr2=0x7f8abc1f3280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:09.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.788+0000 7f8ac20a4700 1 --2- 172.21.15.16:0/683900198 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8abc0b3ec0 0x7f8abc1f3280 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f8aa4011ad0 tx=0x7f8aa4011e90 comp rx=0 tx=0).stop 2023-05-20T01:28:09.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.788+0000 7f8ac20a4700 1 -- 172.21.15.16:0/683900198 shutdown_connections 2023-05-20T01:28:09.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.788+0000 7f8ac20a4700 1 --2- 172.21.15.16:0/683900198 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8abc15be70 0x7f8abc15c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:09.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.788+0000 7f8ac20a4700 1 --2- 172.21.15.16:0/683900198 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f8a9c06ace0 0x7f8a9c06d190 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:09.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.788+0000 7f8ac20a4700 1 --2- 172.21.15.16:0/683900198 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8aa8003aa0 0x7f8abc15b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:09.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.788+0000 7f8ac20a4700 1 --2- 172.21.15.16:0/683900198 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8abc0b3ec0 0x7f8abc1f3280 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:09.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.788+0000 7f8ac20a4700 1 -- 172.21.15.16:0/683900198 >> 172.21.15.16:0/683900198 conn(0x7f8abc06ae30 msgr2=0x7f8abc06fac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:09.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.788+0000 7f8ac20a4700 1 -- 172.21.15.16:0/683900198 shutdown_connections 2023-05-20T01:28:09.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.788+0000 7f8ac20a4700 1 -- 172.21.15.16:0/683900198 wait complete. 2023-05-20T01:28:09.804 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-05-20T01:16:20.934366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-05-20T01:16:21.372294+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2023-05-20T01:26:34.067017+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '111', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2023-05-20T01:26:35.040602+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '112', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data_extra', 'create_time': '2023-05-20T01:26:48.309738+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '115', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-05-20T01:28:09.805 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-05-20T01:28:09.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.929+0000 7f7630020700 1 Processor -- start 2023-05-20T01:28:09.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.929+0000 7f7630020700 1 -- start start 2023-05-20T01:28:09.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.930+0000 7f7630020700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f7628157450 0x7f7628159e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:09.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.930+0000 7f7630020700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f762815d5b0 con 0x7f7628071e10 2023-05-20T01:28:09.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.930+0000 7f7630020700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7628157820 con 0x7f76280cb700 2023-05-20T01:28:09.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.930+0000 7f7630020700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f762815a4c0 con 0x7f7628157450 2023-05-20T01:28:09.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.930+0000 7f762d5bb700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f7628071e10 0x7f76280caff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51058/0 (socket says 172.21.15.16:51058) 2023-05-20T01:28:09.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.930+0000 7f762d5bb700 1 -- 172.21.15.16:0/1638284685 learned_addr learned my addr 172.21.15.16:0/1638284685 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:28:09.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.930+0000 7f762ddbc700 1 --2- 172.21.15.16:0/1638284685 >> v2:172.21.15.132:3300/0 conn(0x7f7628157450 0x7f7628159e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:09.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.930+0000 7f762cdba700 1 -- 172.21.15.16:0/1638284685 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4105025262 0 0) 0x7f762815d5b0 con 0x7f7628071e10 2023-05-20T01:28:09.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.930+0000 7f762cdba700 1 -- 172.21.15.16:0/1638284685 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f76140035b0 con 0x7f7628071e10 2023-05-20T01:28:09.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.930+0000 7f762ddbc700 1 -- 172.21.15.16:0/1638284685 >> v1:172.21.15.132:6789/0 conn(0x7f76280cb700 legacy=0x7f762815da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:09.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.930+0000 7f762ddbc700 1 -- 172.21.15.16:0/1638284685 >> v1:172.21.15.16:6789/0 conn(0x7f7628071e10 legacy=0x7f76280caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:09.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.930+0000 7f762ddbc700 1 -- 172.21.15.16:0/1638284685 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f762815a7a0 con 0x7f7628157450 2023-05-20T01:28:09.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.931+0000 7f762ddbc700 1 --2- 172.21.15.16:0/1638284685 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7628157450 0x7f7628159e40 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f7624009870 tx=0x7f7624009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2432315c25ed858 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:09.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.931+0000 7f762cdba700 1 -- 172.21.15.16:0/1638284685 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7624011830 con 0x7f7628157450 2023-05-20T01:28:09.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.931+0000 7f762cdba700 1 -- 172.21.15.16:0/1638284685 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7624011990 con 0x7f7628157450 2023-05-20T01:28:09.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.931+0000 7f762cdba700 1 -- 172.21.15.16:0/1638284685 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7624011d20 con 0x7f7628157450 2023-05-20T01:28:09.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.931+0000 7f7630020700 1 -- 172.21.15.16:0/1638284685 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7628157450 msgr2=0x7f7628159e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:09.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.931+0000 7f7630020700 1 --2- 172.21.15.16:0/1638284685 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7628157450 0x7f7628159e40 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f7624009870 tx=0x7f7624009b80 comp rx=0 tx=0).stop 2023-05-20T01:28:09.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.931+0000 7f7630020700 1 -- 172.21.15.16:0/1638284685 shutdown_connections 2023-05-20T01:28:09.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.931+0000 7f7630020700 1 --2- 172.21.15.16:0/1638284685 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7628157450 0x7f7628159e40 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:09.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.931+0000 7f7630020700 1 -- 172.21.15.16:0/1638284685 >> 172.21.15.16:0/1638284685 conn(0x7f762806cbb0 msgr2=0x7f762806cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:09.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.931+0000 7f7630020700 1 -- 172.21.15.16:0/1638284685 shutdown_connections 2023-05-20T01:28:09.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.931+0000 7f7630020700 1 -- 172.21.15.16:0/1638284685 wait complete. 2023-05-20T01:28:09.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.933+0000 7f7630020700 1 Processor -- start 2023-05-20T01:28:09.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.933+0000 7f7630020700 1 -- start start 2023-05-20T01:28:09.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.933+0000 7f7630020700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7628071e10 0x7f76280c9230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:09.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.933+0000 7f7630020700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76280cb700 0x7f762814d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:09.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.933+0000 7f7630020700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7628157450 0x7f762806e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:09.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.933+0000 7f7630020700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76281f3470 con 0x7f76280cb700 2023-05-20T01:28:09.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.933+0000 7f7630020700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76280c9770 con 0x7f7628071e10 2023-05-20T01:28:09.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.933+0000 7f7630020700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76280c8190 con 0x7f7628157450 2023-05-20T01:28:09.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.933+0000 7f762d5bb700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76280cb700 0x7f762814d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:09.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.933+0000 7f762d5bb700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76280cb700 0x7f762814d180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39536/0 (socket says 172.21.15.16:39536) 2023-05-20T01:28:09.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.933+0000 7f762d5bb700 1 -- 172.21.15.16:0/3603776085 learned_addr learned my addr 172.21.15.16:0/3603776085 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:09.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.934+0000 7f762ddbc700 1 --2- 172.21.15.16:0/3603776085 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7628071e10 0x7f76280c9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:09.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.934+0000 7f762e5bd700 1 --2- 172.21.15.16:0/3603776085 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7628157450 0x7f762806e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:09.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.934+0000 7f762d5bb700 1 -- 172.21.15.16:0/3603776085 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7628157450 msgr2=0x7f762806e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:09.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.934+0000 7f762d5bb700 1 --2- 172.21.15.16:0/3603776085 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7628157450 0x7f762806e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:09.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.934+0000 7f762d5bb700 1 -- 172.21.15.16:0/3603776085 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7628071e10 msgr2=0x7f76280c9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:09.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.934+0000 7f762d5bb700 1 --2- 172.21.15.16:0/3603776085 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7628071e10 0x7f76280c9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:09.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.934+0000 7f762d5bb700 1 -- 172.21.15.16:0/3603776085 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76240094a0 con 0x7f76280cb700 2023-05-20T01:28:09.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.934+0000 7f762e5bd700 1 --2- 172.21.15.16:0/3603776085 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7628157450 0x7f762806e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:28:09.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.934+0000 7f762d5bb700 1 --2- 172.21.15.16:0/3603776085 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76280cb700 0x7f762814d180 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f7618001eb0 tx=0x7f761800c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:09.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.934+0000 7f761e7fc700 1 -- 172.21.15.16:0/3603776085 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7618005d70 con 0x7f76280cb700 2023-05-20T01:28:09.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.934+0000 7f761e7fc700 1 -- 172.21.15.16:0/3603776085 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76180070d0 con 0x7f76280cb700 2023-05-20T01:28:09.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.934+0000 7f7630020700 1 -- 172.21.15.16:0/3603776085 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f762815ad60 con 0x7f76280cb700 2023-05-20T01:28:09.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.934+0000 7f761e7fc700 1 -- 172.21.15.16:0/3603776085 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7618007460 con 0x7f76280cb700 2023-05-20T01:28:09.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.934+0000 7f7630020700 1 -- 172.21.15.16:0/3603776085 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f762815aa80 con 0x7f76280cb700 2023-05-20T01:28:09.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.935+0000 7f7630020700 1 -- 172.21.15.16:0/3603776085 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7628073b00 con 0x7f76280cb700 2023-05-20T01:28:09.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.935+0000 7f761e7fc700 1 -- 172.21.15.16:0/3603776085 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f76180075c0 con 0x7f76280cb700 2023-05-20T01:28:09.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.936+0000 7f761e7fc700 1 --2- 172.21.15.16:0/3603776085 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f760c06ace0 0x7f760c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:09.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.936+0000 7f761e7fc700 1 -- 172.21.15.16:0/3603776085 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 7073+0+0 (secure 0 0 0) 0x7f7618088d30 con 0x7f76280cb700 2023-05-20T01:28:09.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.936+0000 7f762ddbc700 1 --2- 172.21.15.16:0/3603776085 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f760c06ace0 0x7f760c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:09.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.936+0000 7f762ddbc700 1 --2- 172.21.15.16:0/3603776085 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f760c06ace0 0x7f760c06d190 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f76240090b0 tx=0x7f7624008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:09.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:09.940+0000 7f761e7fc700 1 -- 172.21.15.16:0/3603776085 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f76180539e0 con 0x7f76280cb700 2023-05-20T01:28:10.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.135+0000 7f7630020700 1 -- 172.21.15.16:0/3603776085 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f76280747b0 con 0x7f76280cb700 2023-05-20T01:28:10.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.135+0000 7f761e7fc700 1 -- 172.21.15.16:0/3603776085 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v123) v1 ==== 92+0+10 (secure 0 0 0) 0x7f7618057000 con 0x7f76280cb700 2023-05-20T01:28:10.140 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 8 2023-05-20T01:28:10.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.138+0000 7f7630020700 1 -- 172.21.15.16:0/3603776085 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f760c06ace0 msgr2=0x7f760c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:10.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.138+0000 7f7630020700 1 --2- 172.21.15.16:0/3603776085 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f760c06ace0 0x7f760c06d190 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f76240090b0 tx=0x7f7624008030 comp rx=0 tx=0).stop 2023-05-20T01:28:10.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.138+0000 7f7630020700 1 -- 172.21.15.16:0/3603776085 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76280cb700 msgr2=0x7f762814d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:10.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.138+0000 7f7630020700 1 --2- 172.21.15.16:0/3603776085 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76280cb700 0x7f762814d180 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f7618001eb0 tx=0x7f761800c7b0 comp rx=0 tx=0).stop 2023-05-20T01:28:10.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.138+0000 7f7630020700 1 -- 172.21.15.16:0/3603776085 shutdown_connections 2023-05-20T01:28:10.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.138+0000 7f7630020700 1 --2- 172.21.15.16:0/3603776085 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f760c06ace0 0x7f760c06d190 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:10.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.138+0000 7f7630020700 1 --2- 172.21.15.16:0/3603776085 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7628157450 0x7f762806e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:10.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.138+0000 7f7630020700 1 --2- 172.21.15.16:0/3603776085 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7628071e10 0x7f76280c9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:10.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.138+0000 7f7630020700 1 --2- 172.21.15.16:0/3603776085 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76280cb700 0x7f762814d180 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:10.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.138+0000 7f7630020700 1 -- 172.21.15.16:0/3603776085 >> 172.21.15.16:0/3603776085 conn(0x7f762806cbb0 msgr2=0x7f762815af80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:10.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.138+0000 7f7630020700 1 -- 172.21.15.16:0/3603776085 shutdown_connections 2023-05-20T01:28:10.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.138+0000 7f7630020700 1 -- 172.21.15.16:0/3603776085 wait complete. 2023-05-20T01:28:10.154 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-05-20T01:28:10.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.277+0000 7f58ab7e1700 1 Processor -- start 2023-05-20T01:28:10.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.277+0000 7f58ab7e1700 1 -- start start 2023-05-20T01:28:10.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.278+0000 7f58ab7e1700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f58a415cec0 0x7f58a4071bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:10.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.278+0000 7f58ab7e1700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f58a4072130 0x7f58a40cf9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:10.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.278+0000 7f58ab7e1700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58a40cd630 con 0x7f58a415d290 2023-05-20T01:28:10.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.278+0000 7f58ab7e1700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58a4072530 con 0x7f58a415cec0 2023-05-20T01:28:10.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.278+0000 7f58ab7e1700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58a40cca60 con 0x7f58a4072130 2023-05-20T01:28:10.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.278+0000 7f58a957d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f58a415cec0 0x7f58a4071bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:10.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.278+0000 7f58a8d7c700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f58a415d290 0x7f58a40cf2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51070/0 (socket says 172.21.15.16:51070) 2023-05-20T01:28:10.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.278+0000 7f58a8d7c700 1 -- 172.21.15.16:0/1542975335 learned_addr learned my addr 172.21.15.16:0/1542975335 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:28:10.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.279+0000 7f58a957d700 1 -- 172.21.15.16:0/1542975335 >> v2:172.21.15.132:3300/0 conn(0x7f58a4072130 msgr2=0x7f58a40cf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:10.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.279+0000 7f58a957d700 1 --2- 172.21.15.16:0/1542975335 >> v2:172.21.15.132:3300/0 conn(0x7f58a4072130 0x7f58a40cf9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:10.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.279+0000 7f58a957d700 1 -- 172.21.15.16:0/1542975335 >> v1:172.21.15.16:6789/0 conn(0x7f58a415d290 legacy=0x7f58a40cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:10.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.279+0000 7f58a957d700 1 -- 172.21.15.16:0/1542975335 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58a40ccd00 con 0x7f58a415cec0 2023-05-20T01:28:10.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.279+0000 7f58a957d700 1 --2- 172.21.15.16:0/1542975335 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58a415cec0 0x7f58a4071bf0 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f58a40b6120 tx=0x7f5894009b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d46eb6278f3a484f server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:10.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.279+0000 7f589bfff700 1 -- 172.21.15.16:0/1542975335 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5894005e30 con 0x7f58a415cec0 2023-05-20T01:28:10.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.279+0000 7f589bfff700 1 -- 172.21.15.16:0/1542975335 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5894006030 con 0x7f58a415cec0 2023-05-20T01:28:10.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.279+0000 7f589bfff700 1 -- 172.21.15.16:0/1542975335 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58940063c0 con 0x7f58a415cec0 2023-05-20T01:28:10.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.279+0000 7f58ab7e1700 1 -- 172.21.15.16:0/1542975335 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58a415cec0 msgr2=0x7f58a4071bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:10.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.279+0000 7f58ab7e1700 1 --2- 172.21.15.16:0/1542975335 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58a415cec0 0x7f58a4071bf0 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f58a40b6120 tx=0x7f5894009b00 comp rx=0 tx=0).stop 2023-05-20T01:28:10.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.279+0000 7f58ab7e1700 1 -- 172.21.15.16:0/1542975335 shutdown_connections 2023-05-20T01:28:10.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.279+0000 7f58ab7e1700 1 --2- 172.21.15.16:0/1542975335 >> v2:172.21.15.132:3300/0 conn(0x7f58a4072130 0x7f58a40cf9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:10.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.279+0000 7f58ab7e1700 1 --2- 172.21.15.16:0/1542975335 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58a415cec0 0x7f58a4071bf0 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:10.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.279+0000 7f58ab7e1700 1 -- 172.21.15.16:0/1542975335 >> 172.21.15.16:0/1542975335 conn(0x7f58a40cffd0 msgr2=0x7f58a40d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:10.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.280+0000 7f58ab7e1700 1 -- 172.21.15.16:0/1542975335 shutdown_connections 2023-05-20T01:28:10.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.280+0000 7f58ab7e1700 1 -- 172.21.15.16:0/1542975335 wait complete. 2023-05-20T01:28:10.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.281+0000 7f58ab7e1700 1 Processor -- start 2023-05-20T01:28:10.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.281+0000 7f58ab7e1700 1 -- start start 2023-05-20T01:28:10.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.282+0000 7f58ab7e1700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58a4072130 0x7f58a415b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:10.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.282+0000 7f58ab7e1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58a415cec0 0x7f58a40c8530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:10.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.282+0000 7f58ab7e1700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f58a415d290 0x7f58a40c8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:10.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.282+0000 7f58ab7e1700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58a41f39b0 con 0x7f58a415cec0 2023-05-20T01:28:10.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.282+0000 7f58ab7e1700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58a40adc20 con 0x7f58a4072130 2023-05-20T01:28:10.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.282+0000 7f58ab7e1700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58a40adf00 con 0x7f58a415d290 2023-05-20T01:28:10.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.282+0000 7f58a8d7c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58a415cec0 0x7f58a40c8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:10.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.282+0000 7f58a8d7c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58a415cec0 0x7f58a40c8530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39550/0 (socket says 172.21.15.16:39550) 2023-05-20T01:28:10.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.282+0000 7f58a8d7c700 1 -- 172.21.15.16:0/1782423860 learned_addr learned my addr 172.21.15.16:0/1782423860 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:10.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.282+0000 7f58a957d700 1 --2- 172.21.15.16:0/1782423860 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58a4072130 0x7f58a415b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:10.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.282+0000 7f58a9d7e700 1 --2- 172.21.15.16:0/1782423860 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f58a415d290 0x7f58a40c8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:10.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.282+0000 7f58a8d7c700 1 -- 172.21.15.16:0/1782423860 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f58a415d290 msgr2=0x7f58a40c8a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:10.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.282+0000 7f58a8d7c700 1 --2- 172.21.15.16:0/1782423860 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f58a415d290 0x7f58a40c8a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:10.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.282+0000 7f58a8d7c700 1 -- 172.21.15.16:0/1782423860 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58a4072130 msgr2=0x7f58a415b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:10.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.282+0000 7f58a8d7c700 1 --2- 172.21.15.16:0/1782423860 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58a4072130 0x7f58a415b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:10.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.282+0000 7f58a8d7c700 1 -- 172.21.15.16:0/1782423860 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5894009430 con 0x7f58a415cec0 2023-05-20T01:28:10.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.282+0000 7f58a9d7e700 1 --2- 172.21.15.16:0/1782423860 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f58a415d290 0x7f58a40c8a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:28:10.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.282+0000 7f58a957d700 1 --2- 172.21.15.16:0/1782423860 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58a4072130 0x7f58a415b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:28:10.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.282+0000 7f58a8d7c700 1 --2- 172.21.15.16:0/1782423860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58a415cec0 0x7f58a40c8530 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f58a000c480 tx=0x7f58a000c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:10.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.282+0000 7f5899ffb700 1 -- 172.21.15.16:0/1782423860 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58a0005d70 con 0x7f58a415cec0 2023-05-20T01:28:10.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.282+0000 7f5899ffb700 1 -- 172.21.15.16:0/1782423860 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58a00070d0 con 0x7f58a415cec0 2023-05-20T01:28:10.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.282+0000 7f58ab7e1700 1 -- 172.21.15.16:0/1782423860 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58a40cd2c0 con 0x7f58a415cec0 2023-05-20T01:28:10.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.282+0000 7f5899ffb700 1 -- 172.21.15.16:0/1782423860 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58a0007440 con 0x7f58a415cec0 2023-05-20T01:28:10.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.283+0000 7f58ab7e1700 1 -- 172.21.15.16:0/1782423860 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58a40ccfe0 con 0x7f58a415cec0 2023-05-20T01:28:10.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.284+0000 7f58ab7e1700 1 -- 172.21.15.16:0/1782423860 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58a4160890 con 0x7f58a415cec0 2023-05-20T01:28:10.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.284+0000 7f5899ffb700 1 -- 172.21.15.16:0/1782423860 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f58a00075a0 con 0x7f58a415cec0 2023-05-20T01:28:10.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.284+0000 7f5899ffb700 1 --2- 172.21.15.16:0/1782423860 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f588806ace0 0x7f588806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:10.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.284+0000 7f5899ffb700 1 -- 172.21.15.16:0/1782423860 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 7073+0+0 (secure 0 0 0) 0x7f58a0088c50 con 0x7f58a415cec0 2023-05-20T01:28:10.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.284+0000 7f58a957d700 1 --2- 172.21.15.16:0/1782423860 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f588806ace0 0x7f588806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:10.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.285+0000 7f58a957d700 1 --2- 172.21.15.16:0/1782423860 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f588806ace0 0x7f588806d190 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f5894013040 tx=0x7f5894008070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:10.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.289+0000 7f5899ffb700 1 -- 172.21.15.16:0/1782423860 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f58a008e020 con 0x7f58a415cec0 2023-05-20T01:28:10.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.485+0000 7f58ab7e1700 1 -- 172.21.15.16:0/1782423860 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f58a40747b0 con 0x7f58a415cec0 2023-05-20T01:28:10.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.486+0000 7f5899ffb700 1 -- 172.21.15.16:0/1782423860 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v123) v1 ==== 93+0+10 (secure 0 0 0) 0x7f58a0053a40 con 0x7f58a415cec0 2023-05-20T01:28:10.490 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-05-20T01:28:10.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.488+0000 7f58ab7e1700 1 -- 172.21.15.16:0/1782423860 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f588806ace0 msgr2=0x7f588806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:10.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.488+0000 7f58ab7e1700 1 --2- 172.21.15.16:0/1782423860 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f588806ace0 0x7f588806d190 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f5894013040 tx=0x7f5894008070 comp rx=0 tx=0).stop 2023-05-20T01:28:10.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.488+0000 7f58ab7e1700 1 -- 172.21.15.16:0/1782423860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58a415cec0 msgr2=0x7f58a40c8530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:10.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.488+0000 7f58ab7e1700 1 --2- 172.21.15.16:0/1782423860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58a415cec0 0x7f58a40c8530 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f58a000c480 tx=0x7f58a000c840 comp rx=0 tx=0).stop 2023-05-20T01:28:10.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.488+0000 7f58ab7e1700 1 -- 172.21.15.16:0/1782423860 shutdown_connections 2023-05-20T01:28:10.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.488+0000 7f58ab7e1700 1 --2- 172.21.15.16:0/1782423860 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f588806ace0 0x7f588806d190 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:10.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.488+0000 7f58ab7e1700 1 --2- 172.21.15.16:0/1782423860 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f58a415d290 0x7f58a40c8a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:10.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.488+0000 7f58ab7e1700 1 --2- 172.21.15.16:0/1782423860 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58a4072130 0x7f58a415b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:10.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.488+0000 7f58ab7e1700 1 --2- 172.21.15.16:0/1782423860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58a415cec0 0x7f58a40c8530 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:10.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.488+0000 7f58ab7e1700 1 -- 172.21.15.16:0/1782423860 >> 172.21.15.16:0/1782423860 conn(0x7f58a40cffd0 msgr2=0x7f58a40cbe70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:10.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.488+0000 7f58ab7e1700 1 -- 172.21.15.16:0/1782423860 shutdown_connections 2023-05-20T01:28:10.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.488+0000 7f58ab7e1700 1 -- 172.21.15.16:0/1782423860 wait complete. 2023-05-20T01:28:10.505 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-05-20T01:28:10.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.628+0000 7f2705b1b700 1 Processor -- start 2023-05-20T01:28:10.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.628+0000 7f2705b1b700 1 -- start start 2023-05-20T01:28:10.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.629+0000 7f2705b1b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2700146710 0x7f270006b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:10.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.629+0000 7f2705b1b700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f270006bb60 0x7f27000cdad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:10.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.629+0000 7f2705b1b700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27000cb6c0 con 0x7f27000ce010 2023-05-20T01:28:10.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.629+0000 7f2705b1b700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f270006bfe0 con 0x7f2700146710 2023-05-20T01:28:10.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.629+0000 7f2705b1b700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27000ce840 con 0x7f270006bb60 2023-05-20T01:28:10.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.629+0000 7f26ff7fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2700146710 0x7f270006b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:10.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.629+0000 7f26ff7fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2700146710 0x7f270006b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39556/0 (socket says 172.21.15.16:39556) 2023-05-20T01:28:10.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.629+0000 7f26ff7fe700 1 -- 172.21.15.16:0/2216887818 learned_addr learned my addr 172.21.15.16:0/2216887818 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:10.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.629+0000 7f26feffd700 1 --2- 172.21.15.16:0/2216887818 >> v2:172.21.15.132:3300/0 conn(0x7f270006bb60 0x7f27000cdad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:10.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.630+0000 7f26ff7fe700 1 -- 172.21.15.16:0/2216887818 >> v2:172.21.15.132:3300/0 conn(0x7f270006bb60 msgr2=0x7f27000cdad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:10.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.630+0000 7f26ff7fe700 1 --2- 172.21.15.16:0/2216887818 >> v2:172.21.15.132:3300/0 conn(0x7f270006bb60 0x7f27000cdad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:10.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.630+0000 7f26ff7fe700 1 -- 172.21.15.16:0/2216887818 >> v1:172.21.15.132:6789/0 conn(0x7f27000ce010 legacy=0x7f2700160730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:10.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.630+0000 7f26ff7fe700 1 -- 172.21.15.16:0/2216887818 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27000cb910 con 0x7f2700146710 2023-05-20T01:28:10.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.630+0000 7f26ff7fe700 1 --2- 172.21.15.16:0/2216887818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2700146710 0x7f270006b620 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f270006de00 tx=0x7f26f0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9762a3bf3c50baaf server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:10.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.630+0000 7f26fe7fc700 1 -- 172.21.15.16:0/2216887818 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f000b870 con 0x7f2700146710 2023-05-20T01:28:10.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.630+0000 7f26fe7fc700 1 -- 172.21.15.16:0/2216887818 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26f000b9d0 con 0x7f2700146710 2023-05-20T01:28:10.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.630+0000 7f26fe7fc700 1 -- 172.21.15.16:0/2216887818 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f000bd60 con 0x7f2700146710 2023-05-20T01:28:10.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.630+0000 7f2705b1b700 1 -- 172.21.15.16:0/2216887818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2700146710 msgr2=0x7f270006b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:10.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.630+0000 7f2705b1b700 1 --2- 172.21.15.16:0/2216887818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2700146710 0x7f270006b620 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f270006de00 tx=0x7f26f0009bd0 comp rx=0 tx=0).stop 2023-05-20T01:28:10.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.631+0000 7f2705b1b700 1 -- 172.21.15.16:0/2216887818 shutdown_connections 2023-05-20T01:28:10.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.631+0000 7f2705b1b700 1 --2- 172.21.15.16:0/2216887818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2700146710 0x7f270006b620 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:10.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.631+0000 7f2705b1b700 1 --2- 172.21.15.16:0/2216887818 >> v2:172.21.15.132:3300/0 conn(0x7f270006bb60 0x7f27000cdad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:10.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.631+0000 7f2705b1b700 1 -- 172.21.15.16:0/2216887818 >> 172.21.15.16:0/2216887818 conn(0x7f270006ae30 msgr2=0x7f270009a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:10.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.631+0000 7f2705b1b700 1 -- 172.21.15.16:0/2216887818 shutdown_connections 2023-05-20T01:28:10.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.631+0000 7f2705b1b700 1 -- 172.21.15.16:0/2216887818 wait complete. 2023-05-20T01:28:10.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.632+0000 7f2705b1b700 1 Processor -- start 2023-05-20T01:28:10.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.632+0000 7f2705b1b700 1 -- start start 2023-05-20T01:28:10.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.633+0000 7f2705b1b700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f270006bb60 0x7f2700153850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:10.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.633+0000 7f2705b1b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27000ce010 0x7f27000c9730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:10.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.633+0000 7f2705b1b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2700146710 0x7f27000c9c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:10.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.633+0000 7f2705b1b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27000ca8c0 con 0x7f27000ce010 2023-05-20T01:28:10.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.633+0000 7f2705b1b700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27000af600 con 0x7f2700146710 2023-05-20T01:28:10.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.633+0000 7f2705b1b700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27000b34a0 con 0x7f270006bb60 2023-05-20T01:28:10.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.633+0000 7f26feffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27000ce010 0x7f27000c9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:10.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.633+0000 7f26feffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27000ce010 0x7f27000c9730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39564/0 (socket says 172.21.15.16:39564) 2023-05-20T01:28:10.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.633+0000 7f26feffd700 1 -- 172.21.15.16:0/1553314052 learned_addr learned my addr 172.21.15.16:0/1553314052 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:10.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.633+0000 7f26feffd700 1 -- 172.21.15.16:0/1553314052 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f270006bb60 msgr2=0x7f2700153850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:10.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.633+0000 7f26feffd700 1 --2- 172.21.15.16:0/1553314052 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f270006bb60 0x7f2700153850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:10.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.633+0000 7f26feffd700 1 -- 172.21.15.16:0/1553314052 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2700146710 msgr2=0x7f27000c9c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:10.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.633+0000 7f26fffff700 1 --2- 172.21.15.16:0/1553314052 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2700146710 0x7f27000c9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:10.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.633+0000 7f26feffd700 1 --2- 172.21.15.16:0/1553314052 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2700146710 0x7f27000c9c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:10.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.633+0000 7f26feffd700 1 -- 172.21.15.16:0/1553314052 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26f0009550 con 0x7f27000ce010 2023-05-20T01:28:10.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.633+0000 7f26fffff700 1 --2- 172.21.15.16:0/1553314052 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2700146710 0x7f27000c9c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:28:10.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.633+0000 7f26feffd700 1 --2- 172.21.15.16:0/1553314052 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27000ce010 0x7f27000c9730 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f26f400d800 tx=0x7f26f400dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:10.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.633+0000 7f26e7fff700 1 -- 172.21.15.16:0/1553314052 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f40125f0 con 0x7f27000ce010 2023-05-20T01:28:10.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.634+0000 7f26e7fff700 1 -- 172.21.15.16:0/1553314052 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26f4012750 con 0x7f27000ce010 2023-05-20T01:28:10.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.634+0000 7f2705b1b700 1 -- 172.21.15.16:0/1553314052 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27000cbed0 con 0x7f27000ce010 2023-05-20T01:28:10.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.634+0000 7f26e7fff700 1 -- 172.21.15.16:0/1553314052 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f4012b00 con 0x7f27000ce010 2023-05-20T01:28:10.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.634+0000 7f2705b1b700 1 -- 172.21.15.16:0/1553314052 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27000cbbf0 con 0x7f27000ce010 2023-05-20T01:28:10.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.635+0000 7f2705b1b700 1 -- 172.21.15.16:0/1553314052 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27000ce470 con 0x7f27000ce010 2023-05-20T01:28:10.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.635+0000 7f26e7fff700 1 -- 172.21.15.16:0/1553314052 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f26f4012c60 con 0x7f27000ce010 2023-05-20T01:28:10.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.635+0000 7f26e7fff700 1 --2- 172.21.15.16:0/1553314052 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f26ec06ace0 0x7f26ec06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:10.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.636+0000 7f26e7fff700 1 -- 172.21.15.16:0/1553314052 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 7073+0+0 (secure 0 0 0) 0x7f26f4089be0 con 0x7f27000ce010 2023-05-20T01:28:10.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.636+0000 7f26ff7fe700 1 --2- 172.21.15.16:0/1553314052 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f26ec06ace0 0x7f26ec06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:10.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.636+0000 7f26ff7fe700 1 --2- 172.21.15.16:0/1553314052 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f26ec06ace0 0x7f26ec06d190 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f26f0009c10 tx=0x7f26f0010000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:10.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.640+0000 7f26e7fff700 1 -- 172.21.15.16:0/1553314052 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f26f4054920 con 0x7f27000ce010 2023-05-20T01:28:10.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.838+0000 7f2705b1b700 1 -- 172.21.15.16:0/1553314052 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f27000747b0 con 0x7f27000ce010 2023-05-20T01:28:10.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.838+0000 7f26e7fff700 1 -- 172.21.15.16:0/1553314052 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v123) v1 ==== 104+0+11 (secure 0 0 0) 0x7f26f4057f40 con 0x7f27000ce010 2023-05-20T01:28:10.843 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-05-20T01:28:10.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.840+0000 7f2705b1b700 1 -- 172.21.15.16:0/1553314052 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f26ec06ace0 msgr2=0x7f26ec06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:10.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.840+0000 7f2705b1b700 1 --2- 172.21.15.16:0/1553314052 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f26ec06ace0 0x7f26ec06d190 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f26f0009c10 tx=0x7f26f0010000 comp rx=0 tx=0).stop 2023-05-20T01:28:10.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.840+0000 7f2705b1b700 1 -- 172.21.15.16:0/1553314052 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27000ce010 msgr2=0x7f27000c9730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:10.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.840+0000 7f2705b1b700 1 --2- 172.21.15.16:0/1553314052 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27000ce010 0x7f27000c9730 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f26f400d800 tx=0x7f26f400dbc0 comp rx=0 tx=0).stop 2023-05-20T01:28:10.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.840+0000 7f2705b1b700 1 -- 172.21.15.16:0/1553314052 shutdown_connections 2023-05-20T01:28:10.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.840+0000 7f2705b1b700 1 --2- 172.21.15.16:0/1553314052 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2700146710 0x7f27000c9c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:10.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.840+0000 7f2705b1b700 1 --2- 172.21.15.16:0/1553314052 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f26ec06ace0 0x7f26ec06d190 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:10.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.840+0000 7f2705b1b700 1 --2- 172.21.15.16:0/1553314052 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f270006bb60 0x7f2700153850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:10.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.840+0000 7f2705b1b700 1 --2- 172.21.15.16:0/1553314052 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27000ce010 0x7f27000c9730 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:10.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.840+0000 7f2705b1b700 1 -- 172.21.15.16:0/1553314052 >> 172.21.15.16:0/1553314052 conn(0x7f270006ae30 msgr2=0x7f27000add20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:10.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.840+0000 7f2705b1b700 1 -- 172.21.15.16:0/1553314052 shutdown_connections 2023-05-20T01:28:10.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.840+0000 7f2705b1b700 1 -- 172.21.15.16:0/1553314052 wait complete. 2023-05-20T01:28:10.856 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-05-20T01:28:10.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.981+0000 7fd7d8805700 1 Processor -- start 2023-05-20T01:28:10.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.981+0000 7fd7d8805700 1 -- start start 2023-05-20T01:28:10.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.982+0000 7fd7d8805700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd7d415cf40 0x7fd7d406e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:10.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.982+0000 7fd7d8805700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fd7d40710d0 0x7fd7d40cc960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:10.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.982+0000 7fd7d8805700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd7d40ccea0 0x7fd7d40cded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:10.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.982+0000 7fd7d8805700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7d4072430 con 0x7fd7d415cf40 2023-05-20T01:28:10.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.982+0000 7fd7d8805700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7d40cd2e0 con 0x7fd7d40710d0 2023-05-20T01:28:10.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.982+0000 7fd7d8805700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d4071820 con 0x7fd7d40ccea0 2023-05-20T01:28:10.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.982+0000 7fd7d259c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd7d415cf40 0x7fd7d406e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:10.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.982+0000 7fd7d259c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd7d415cf40 0x7fd7d406e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39570/0 (socket says 172.21.15.16:39570) 2023-05-20T01:28:10.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.982+0000 7fd7d259c700 1 -- 172.21.15.16:0/2181033860 learned_addr learned my addr 172.21.15.16:0/2181033860 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:10.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.982+0000 7fd7d2d9d700 1 --2- 172.21.15.16:0/2181033860 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd7d40ccea0 0x7fd7d40cded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:10.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.982+0000 7fd7d1d9b700 1 --2- 172.21.15.16:0/2181033860 >> v2:172.21.15.132:3300/0 conn(0x7fd7d40710d0 0x7fd7d40cc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:10.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.982+0000 7fd7d259c700 1 -- 172.21.15.16:0/2181033860 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd7d40ccea0 msgr2=0x7fd7d40cded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:10.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.982+0000 7fd7d259c700 1 --2- 172.21.15.16:0/2181033860 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd7d40ccea0 0x7fd7d40cded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:10.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.982+0000 7fd7d259c700 1 -- 172.21.15.16:0/2181033860 >> v2:172.21.15.132:3300/0 conn(0x7fd7d40710d0 msgr2=0x7fd7d40cc960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:10.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.982+0000 7fd7d259c700 1 --2- 172.21.15.16:0/2181033860 >> v2:172.21.15.132:3300/0 conn(0x7fd7d40710d0 0x7fd7d40cc960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:10.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.982+0000 7fd7d259c700 1 -- 172.21.15.16:0/2181033860 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7d4071b00 con 0x7fd7d415cf40 2023-05-20T01:28:10.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.983+0000 7fd7d259c700 1 --2- 172.21.15.16:0/2181033860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7d415cf40 0x7fd7d406e020 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7fd7d40d0c30 tx=0x7fd7c8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb074a24cb103153 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:10.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.983+0000 7fd7d159a700 1 -- 172.21.15.16:0/2181033860 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c800b870 con 0x7fd7d415cf40 2023-05-20T01:28:10.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.983+0000 7fd7d159a700 1 -- 172.21.15.16:0/2181033860 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7c800b9d0 con 0x7fd7d415cf40 2023-05-20T01:28:10.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.983+0000 7fd7d159a700 1 -- 172.21.15.16:0/2181033860 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c800bd60 con 0x7fd7d415cf40 2023-05-20T01:28:10.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.983+0000 7fd7d8805700 1 -- 172.21.15.16:0/2181033860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7d415cf40 msgr2=0x7fd7d406e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:10.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.983+0000 7fd7d8805700 1 --2- 172.21.15.16:0/2181033860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7d415cf40 0x7fd7d406e020 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7fd7d40d0c30 tx=0x7fd7c8009bd0 comp rx=0 tx=0).stop 2023-05-20T01:28:10.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.983+0000 7fd7d8805700 1 -- 172.21.15.16:0/2181033860 shutdown_connections 2023-05-20T01:28:10.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.983+0000 7fd7d8805700 1 --2- 172.21.15.16:0/2181033860 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd7d40ccea0 0x7fd7d40cded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:10.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.983+0000 7fd7d8805700 1 --2- 172.21.15.16:0/2181033860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7d415cf40 0x7fd7d406e020 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:10.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.983+0000 7fd7d8805700 1 --2- 172.21.15.16:0/2181033860 >> v2:172.21.15.132:3300/0 conn(0x7fd7d40710d0 0x7fd7d40cc960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:10.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.983+0000 7fd7d8805700 1 -- 172.21.15.16:0/2181033860 >> 172.21.15.16:0/2181033860 conn(0x7fd7d406c260 msgr2=0x7fd7d40cffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:10.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.983+0000 7fd7d8805700 1 -- 172.21.15.16:0/2181033860 shutdown_connections 2023-05-20T01:28:10.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.983+0000 7fd7d8805700 1 -- 172.21.15.16:0/2181033860 wait complete. 2023-05-20T01:28:10.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.985+0000 7fd7d8805700 1 Processor -- start 2023-05-20T01:28:10.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.985+0000 7fd7d8805700 1 -- start start 2023-05-20T01:28:10.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.985+0000 7fd7d8805700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7d40710d0 0x7fd7d40af710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:10.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.985+0000 7fd7d8805700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd7d40ccea0 0x7fd7d40b1320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:10.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.985+0000 7fd7d8805700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd7d415cf40 0x7fd7d40b1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:10.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.986+0000 7fd7d8805700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d4150a80 con 0x7fd7d40710d0 2023-05-20T01:28:10.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.986+0000 7fd7d8805700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d40b0e90 con 0x7fd7d40ccea0 2023-05-20T01:28:10.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.986+0000 7fd7d8805700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d40b2e40 con 0x7fd7d415cf40 2023-05-20T01:28:10.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.986+0000 7fd7d259c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7d40710d0 0x7fd7d40af710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:10.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.986+0000 7fd7d259c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7d40710d0 0x7fd7d40af710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39576/0 (socket says 172.21.15.16:39576) 2023-05-20T01:28:10.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.986+0000 7fd7d259c700 1 -- 172.21.15.16:0/2082941167 learned_addr learned my addr 172.21.15.16:0/2082941167 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:10.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.986+0000 7fd7d1d9b700 1 --2- 172.21.15.16:0/2082941167 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd7d40ccea0 0x7fd7d40b1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:10.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.986+0000 7fd7d2d9d700 1 --2- 172.21.15.16:0/2082941167 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd7d415cf40 0x7fd7d40b1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:10.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.986+0000 7fd7d259c700 1 -- 172.21.15.16:0/2082941167 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd7d415cf40 msgr2=0x7fd7d40b1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:10.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.986+0000 7fd7d259c700 1 --2- 172.21.15.16:0/2082941167 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd7d415cf40 0x7fd7d40b1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:10.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.986+0000 7fd7d259c700 1 -- 172.21.15.16:0/2082941167 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd7d40ccea0 msgr2=0x7fd7d40b1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:10.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.986+0000 7fd7d259c700 1 --2- 172.21.15.16:0/2082941167 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd7d40ccea0 0x7fd7d40b1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:10.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.986+0000 7fd7d259c700 1 -- 172.21.15.16:0/2082941167 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7c8009550 con 0x7fd7d40710d0 2023-05-20T01:28:10.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.986+0000 7fd7d259c700 1 --2- 172.21.15.16:0/2082941167 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7d40710d0 0x7fd7d40af710 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7fd7c8000c00 tx=0x7fd7c8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:10.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.986+0000 7fd7c2ffd700 1 -- 172.21.15.16:0/2082941167 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c80129c0 con 0x7fd7d40710d0 2023-05-20T01:28:10.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.986+0000 7fd7c2ffd700 1 -- 172.21.15.16:0/2082941167 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7c801e070 con 0x7fd7d40710d0 2023-05-20T01:28:10.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.986+0000 7fd7c2ffd700 1 -- 172.21.15.16:0/2082941167 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c8012d40 con 0x7fd7d40710d0 2023-05-20T01:28:10.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.986+0000 7fd7d8805700 1 -- 172.21.15.16:0/2082941167 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7d40720c0 con 0x7fd7d40710d0 2023-05-20T01:28:10.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.986+0000 7fd7d8805700 1 -- 172.21.15.16:0/2082941167 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7d4071de0 con 0x7fd7d40710d0 2023-05-20T01:28:10.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.987+0000 7fd7d8805700 1 -- 172.21.15.16:0/2082941167 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7d4071520 con 0x7fd7d40710d0 2023-05-20T01:28:10.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.988+0000 7fd7c2ffd700 1 -- 172.21.15.16:0/2082941167 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd7c802d450 con 0x7fd7d40710d0 2023-05-20T01:28:10.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.988+0000 7fd7c2ffd700 1 --2- 172.21.15.16:0/2082941167 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd7b406ad10 0x7fd7b406d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:10.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.988+0000 7fd7c2ffd700 1 -- 172.21.15.16:0/2082941167 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 7073+0+0 (secure 0 0 0) 0x7fd7c808a950 con 0x7fd7d40710d0 2023-05-20T01:28:10.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.988+0000 7fd7d1d9b700 1 --2- 172.21.15.16:0/2082941167 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd7b406ad10 0x7fd7b406d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:10.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.989+0000 7fd7d1d9b700 1 --2- 172.21.15.16:0/2082941167 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd7b406ad10 0x7fd7b406d1c0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fd7d4157f70 tx=0x7fd7c4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:11.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:10.993+0000 7fd7c2ffd700 1 -- 172.21.15.16:0/2082941167 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fd7c8055690 con 0x7fd7d40710d0 2023-05-20T01:28:11.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.190+0000 7fd7d8805700 1 -- 172.21.15.16:0/2082941167 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd7d40747b0 con 0x7fd7d40710d0 2023-05-20T01:28:11.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.190+0000 7fd7c2ffd700 1 -- 172.21.15.16:0/2082941167 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v123) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd7c8058cb0 con 0x7fd7d40710d0 2023-05-20T01:28:11.195 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2023-05-20T01:28:11.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.192+0000 7fd7d8805700 1 -- 172.21.15.16:0/2082941167 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd7b406ad10 msgr2=0x7fd7b406d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:11.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.192+0000 7fd7d8805700 1 --2- 172.21.15.16:0/2082941167 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd7b406ad10 0x7fd7b406d1c0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fd7d4157f70 tx=0x7fd7c4008040 comp rx=0 tx=0).stop 2023-05-20T01:28:11.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.192+0000 7fd7d8805700 1 -- 172.21.15.16:0/2082941167 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7d40710d0 msgr2=0x7fd7d40af710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:11.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.192+0000 7fd7d8805700 1 --2- 172.21.15.16:0/2082941167 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7d40710d0 0x7fd7d40af710 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7fd7c8000c00 tx=0x7fd7c8004750 comp rx=0 tx=0).stop 2023-05-20T01:28:11.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.192+0000 7fd7d8805700 1 -- 172.21.15.16:0/2082941167 shutdown_connections 2023-05-20T01:28:11.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.192+0000 7fd7d8805700 1 --2- 172.21.15.16:0/2082941167 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd7b406ad10 0x7fd7b406d1c0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:11.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.192+0000 7fd7d8805700 1 --2- 172.21.15.16:0/2082941167 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd7d415cf40 0x7fd7d40b1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:11.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.192+0000 7fd7d8805700 1 --2- 172.21.15.16:0/2082941167 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7d40710d0 0x7fd7d40af710 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:11.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.192+0000 7fd7d8805700 1 --2- 172.21.15.16:0/2082941167 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd7d40ccea0 0x7fd7d40b1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:11.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.192+0000 7fd7d8805700 1 -- 172.21.15.16:0/2082941167 >> 172.21.15.16:0/2082941167 conn(0x7fd7d406c260 msgr2=0x7fd7d40cc3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:11.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.193+0000 7fd7d8805700 1 -- 172.21.15.16:0/2082941167 shutdown_connections 2023-05-20T01:28:11.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.193+0000 7fd7d8805700 1 -- 172.21.15.16:0/2082941167 wait complete. 2023-05-20T01:28:11.209 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_extra pg_num 2023-05-20T01:28:11.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.332+0000 7f4567854700 1 Processor -- start 2023-05-20T01:28:11.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.332+0000 7f4567854700 1 -- start start 2023-05-20T01:28:11.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.333+0000 7f4567854700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f45600ce030 0x7f45600d0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:11.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.333+0000 7f4567854700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45600d1c10 con 0x7f45600cfcb0 2023-05-20T01:28:11.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.333+0000 7f4567854700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f456015d120 con 0x7f456015cd50 2023-05-20T01:28:11.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.333+0000 7f4567854700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45600d0660 con 0x7f45600ce030 2023-05-20T01:28:11.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.333+0000 7f4565df1700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f45600ce030 0x7f45600d0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:11.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.333+0000 7f4564def700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f45600cfcb0 0x7f45600cd920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51114/0 (socket says 172.21.15.16:51114) 2023-05-20T01:28:11.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.333+0000 7f4564def700 1 -- 172.21.15.16:0/3755891890 learned_addr learned my addr 172.21.15.16:0/3755891890 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:28:11.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.333+0000 7f4565df1700 1 -- 172.21.15.16:0/3755891890 >> v1:172.21.15.132:6789/0 conn(0x7f456015cd50 legacy=0x7f45600d20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:11.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.333+0000 7f4565df1700 1 -- 172.21.15.16:0/3755891890 >> v1:172.21.15.16:6789/0 conn(0x7f45600cfcb0 legacy=0x7f45600cd920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:11.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.333+0000 7f4565df1700 1 -- 172.21.15.16:0/3755891890 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45600d12e0 con 0x7f45600ce030 2023-05-20T01:28:11.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.333+0000 7f4553fff700 1 -- 172.21.15.16:0/3755891890 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3440459537 0 0) 0x7f45600d1c10 con 0x7f45600cfcb0 2023-05-20T01:28:11.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.333+0000 7f4565df1700 1 --2- 172.21.15.16:0/3755891890 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45600ce030 0x7f45600d0120 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f4558009d90 tx=0x7f455800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1250773a7880b60 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:11.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.333+0000 7f4553fff700 1 -- 172.21.15.16:0/3755891890 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f455800e070 con 0x7f45600ce030 2023-05-20T01:28:11.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.333+0000 7f4553fff700 1 -- 172.21.15.16:0/3755891890 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f455800bb90 con 0x7f45600ce030 2023-05-20T01:28:11.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.333+0000 7f4553fff700 1 -- 172.21.15.16:0/3755891890 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4558003a20 con 0x7f45600ce030 2023-05-20T01:28:11.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.334+0000 7f4567854700 1 -- 172.21.15.16:0/3755891890 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45600ce030 msgr2=0x7f45600d0120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:11.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.334+0000 7f4567854700 1 --2- 172.21.15.16:0/3755891890 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45600ce030 0x7f45600d0120 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f4558009d90 tx=0x7f455800b330 comp rx=0 tx=0).stop 2023-05-20T01:28:11.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.334+0000 7f4567854700 1 -- 172.21.15.16:0/3755891890 shutdown_connections 2023-05-20T01:28:11.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.334+0000 7f4567854700 1 --2- 172.21.15.16:0/3755891890 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45600ce030 0x7f45600d0120 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:11.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.334+0000 7f4567854700 1 -- 172.21.15.16:0/3755891890 >> 172.21.15.16:0/3755891890 conn(0x7f456006b070 msgr2=0x7f456006b470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:11.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.334+0000 7f4567854700 1 -- 172.21.15.16:0/3755891890 shutdown_connections 2023-05-20T01:28:11.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.334+0000 7f4567854700 1 -- 172.21.15.16:0/3755891890 wait complete. 2023-05-20T01:28:11.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.335+0000 7f4567854700 1 Processor -- start 2023-05-20T01:28:11.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.335+0000 7f4567854700 1 -- start start 2023-05-20T01:28:11.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.336+0000 7f4567854700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45600ce030 0x7f45600c93f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:11.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.336+0000 7f4567854700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f45600cfcb0 0x7f45600cc570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:11.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.336+0000 7f4567854700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f456015cd50 0x7f45600ccab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:11.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.336+0000 7f4567854700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45601467d0 con 0x7f456015cd50 2023-05-20T01:28:11.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.336+0000 7f4567854700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45600c9930 con 0x7f45600ce030 2023-05-20T01:28:11.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.336+0000 7f4567854700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4560156ab0 con 0x7f45600cfcb0 2023-05-20T01:28:11.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.336+0000 7f4565df1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f456015cd50 0x7f45600ccab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:11.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.336+0000 7f4565df1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f456015cd50 0x7f45600ccab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39594/0 (socket says 172.21.15.16:39594) 2023-05-20T01:28:11.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.336+0000 7f4565df1700 1 -- 172.21.15.16:0/776323410 learned_addr learned my addr 172.21.15.16:0/776323410 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:11.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.336+0000 7f4564def700 1 --2- 172.21.15.16:0/776323410 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f45600cfcb0 0x7f45600cc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:11.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.336+0000 7f45655f0700 1 --2- 172.21.15.16:0/776323410 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45600ce030 0x7f45600c93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:11.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.337+0000 7f4565df1700 1 -- 172.21.15.16:0/776323410 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f45600cfcb0 msgr2=0x7f45600cc570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:11.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.337+0000 7f4565df1700 1 --2- 172.21.15.16:0/776323410 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f45600cfcb0 0x7f45600cc570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:11.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.337+0000 7f4565df1700 1 -- 172.21.15.16:0/776323410 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45600ce030 msgr2=0x7f45600c93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:11.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.337+0000 7f4565df1700 1 --2- 172.21.15.16:0/776323410 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45600ce030 0x7f45600c93f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:11.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.337+0000 7f4565df1700 1 -- 172.21.15.16:0/776323410 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4558009a70 con 0x7f456015cd50 2023-05-20T01:28:11.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.337+0000 7f4564def700 1 --2- 172.21.15.16:0/776323410 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f45600cfcb0 0x7f45600cc570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:28:11.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.337+0000 7f4565df1700 1 --2- 172.21.15.16:0/776323410 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f456015cd50 0x7f45600ccab0 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f4558004000 tx=0x7f4558004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:11.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.337+0000 7f4551ffb700 1 -- 172.21.15.16:0/776323410 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4558012630 con 0x7f456015cd50 2023-05-20T01:28:11.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.337+0000 7f4551ffb700 1 -- 172.21.15.16:0/776323410 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45580127b0 con 0x7f456015cd50 2023-05-20T01:28:11.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.337+0000 7f4567854700 1 -- 172.21.15.16:0/776323410 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45600d18a0 con 0x7f456015cd50 2023-05-20T01:28:11.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.337+0000 7f4551ffb700 1 -- 172.21.15.16:0/776323410 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4558012b60 con 0x7f456015cd50 2023-05-20T01:28:11.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.337+0000 7f4567854700 1 -- 172.21.15.16:0/776323410 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45600d15c0 con 0x7f456015cd50 2023-05-20T01:28:11.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.338+0000 7f4567854700 1 -- 172.21.15.16:0/776323410 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4560161680 con 0x7f456015cd50 2023-05-20T01:28:11.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.338+0000 7f4551ffb700 1 -- 172.21.15.16:0/776323410 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4558012cc0 con 0x7f456015cd50 2023-05-20T01:28:11.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.339+0000 7f4551ffb700 1 --2- 172.21.15.16:0/776323410 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f454406ace0 0x7f454406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:11.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.339+0000 7f4551ffb700 1 -- 172.21.15.16:0/776323410 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 7073+0+0 (secure 0 0 0) 0x7f455808ba70 con 0x7f456015cd50 2023-05-20T01:28:11.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.339+0000 7f45655f0700 1 --2- 172.21.15.16:0/776323410 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f454406ace0 0x7f454406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:11.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.339+0000 7f45655f0700 1 --2- 172.21.15.16:0/776323410 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f454406ace0 0x7f454406d190 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f455c006fd0 tx=0x7f455c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:11.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.343+0000 7f4551ffb700 1 -- 172.21.15.16:0/776323410 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f455805a240 con 0x7f456015cd50 2023-05-20T01:28:11.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.540+0000 7f4567854700 1 -- 172.21.15.16:0/776323410 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"} v 0) v1 -- 0x7f4560146910 con 0x7f456015cd50 2023-05-20T01:28:11.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.541+0000 7f4551ffb700 1 -- 172.21.15.16:0/776323410 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"}]=0 v123) v1 ==== 106+0+11 (secure 0 0 0) 0x7f4560146910 con 0x7f456015cd50 2023-05-20T01:28:11.546 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-05-20T01:28:11.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.543+0000 7f4567854700 1 -- 172.21.15.16:0/776323410 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f454406ace0 msgr2=0x7f454406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:11.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.543+0000 7f4567854700 1 --2- 172.21.15.16:0/776323410 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f454406ace0 0x7f454406d190 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f455c006fd0 tx=0x7f455c008040 comp rx=0 tx=0).stop 2023-05-20T01:28:11.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.543+0000 7f4567854700 1 -- 172.21.15.16:0/776323410 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f456015cd50 msgr2=0x7f45600ccab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:11.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.543+0000 7f4567854700 1 --2- 172.21.15.16:0/776323410 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f456015cd50 0x7f45600ccab0 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f4558004000 tx=0x7f4558004ad0 comp rx=0 tx=0).stop 2023-05-20T01:28:11.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.543+0000 7f4567854700 1 -- 172.21.15.16:0/776323410 shutdown_connections 2023-05-20T01:28:11.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.543+0000 7f4567854700 1 --2- 172.21.15.16:0/776323410 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f456015cd50 0x7f45600ccab0 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:11.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.543+0000 7f4567854700 1 --2- 172.21.15.16:0/776323410 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45600ce030 0x7f45600c93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:11.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.543+0000 7f4567854700 1 --2- 172.21.15.16:0/776323410 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f454406ace0 0x7f454406d190 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:11.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.543+0000 7f4567854700 1 --2- 172.21.15.16:0/776323410 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f45600cfcb0 0x7f45600cc570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:11.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.543+0000 7f4567854700 1 -- 172.21.15.16:0/776323410 >> 172.21.15.16:0/776323410 conn(0x7f456006b070 msgr2=0x7f456015e530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:11.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.543+0000 7f4567854700 1 -- 172.21.15.16:0/776323410 shutdown_connections 2023-05-20T01:28:11.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.543+0000 7f4567854700 1 -- 172.21.15.16:0/776323410 wait complete. 2023-05-20T01:28:11.560 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-20T01:28:11.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.682+0000 7fda08d49700 1 Processor -- start 2023-05-20T01:28:11.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.682+0000 7fda08d49700 1 -- start start 2023-05-20T01:28:11.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.683+0000 7fda08d49700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fda0406f450 0x7fda040cf890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:11.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.683+0000 7fda08d49700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fda0406a510 0x7fda0406a950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:11.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.683+0000 7fda08d49700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda040cc510 con 0x7fda0406d4b0 2023-05-20T01:28:11.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.683+0000 7fda08d49700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda0406ae90 con 0x7fda0406f450 2023-05-20T01:28:11.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.683+0000 7fda08d49700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda04071ed0 con 0x7fda0406a510 2023-05-20T01:28:11.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.683+0000 7fda0259c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fda0406f450 0x7fda040cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:11.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.683+0000 7fda01d9b700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fda0406d4b0 0x7fda04161080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51126/0 (socket says 172.21.15.16:51126) 2023-05-20T01:28:11.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.683+0000 7fda01d9b700 1 -- 172.21.15.16:0/1119034155 learned_addr learned my addr 172.21.15.16:0/1119034155 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:28:11.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.684+0000 7fda02d9d700 1 --2- 172.21.15.16:0/1119034155 >> v2:172.21.15.132:3300/0 conn(0x7fda0406a510 0x7fda0406a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:11.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.684+0000 7fda0259c700 1 -- 172.21.15.16:0/1119034155 >> v2:172.21.15.132:3300/0 conn(0x7fda0406a510 msgr2=0x7fda0406a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:11.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.684+0000 7fda0259c700 1 --2- 172.21.15.16:0/1119034155 >> v2:172.21.15.132:3300/0 conn(0x7fda0406a510 0x7fda0406a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:11.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.684+0000 7fda0259c700 1 -- 172.21.15.16:0/1119034155 >> v1:172.21.15.16:6789/0 conn(0x7fda0406d4b0 legacy=0x7fda04161080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:11.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.684+0000 7fda0259c700 1 -- 172.21.15.16:0/1119034155 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda040721b0 con 0x7fda0406f450 2023-05-20T01:28:11.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.684+0000 7fda0159a700 1 -- 172.21.15.16:0/1119034155 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3512002984 0 0) 0x7fda040cc510 con 0x7fda0406d4b0 2023-05-20T01:28:11.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.684+0000 7fda0259c700 1 --2- 172.21.15.16:0/1119034155 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda0406f450 0x7fda040cf890 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7fda0406f9f0 tx=0x7fd9f8009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=163283cbc40bd564 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:11.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.684+0000 7fda0159a700 1 -- 172.21.15.16:0/1119034155 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f8011830 con 0x7fda0406f450 2023-05-20T01:28:11.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.684+0000 7fda0159a700 1 -- 172.21.15.16:0/1119034155 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9f8011990 con 0x7fda0406f450 2023-05-20T01:28:11.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.684+0000 7fda0159a700 1 -- 172.21.15.16:0/1119034155 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f8011d20 con 0x7fda0406f450 2023-05-20T01:28:11.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.685+0000 7fda08d49700 1 -- 172.21.15.16:0/1119034155 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda0406f450 msgr2=0x7fda040cf890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:11.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.685+0000 7fda08d49700 1 --2- 172.21.15.16:0/1119034155 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda0406f450 0x7fda040cf890 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7fda0406f9f0 tx=0x7fd9f8009b20 comp rx=0 tx=0).stop 2023-05-20T01:28:11.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.685+0000 7fda08d49700 1 -- 172.21.15.16:0/1119034155 shutdown_connections 2023-05-20T01:28:11.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.685+0000 7fda08d49700 1 --2- 172.21.15.16:0/1119034155 >> v2:172.21.15.132:3300/0 conn(0x7fda0406a510 0x7fda0406a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:11.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.685+0000 7fda08d49700 1 --2- 172.21.15.16:0/1119034155 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda0406f450 0x7fda040cf890 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:11.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.685+0000 7fda08d49700 1 -- 172.21.15.16:0/1119034155 >> 172.21.15.16:0/1119034155 conn(0x7fda040cffb0 msgr2=0x7fda040d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:11.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.685+0000 7fda08d49700 1 -- 172.21.15.16:0/1119034155 shutdown_connections 2023-05-20T01:28:11.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.685+0000 7fda08d49700 1 -- 172.21.15.16:0/1119034155 wait complete. 2023-05-20T01:28:11.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.686+0000 7fda08d49700 1 Processor -- start 2023-05-20T01:28:11.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.686+0000 7fda08d49700 1 -- start start 2023-05-20T01:28:11.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.687+0000 7fda08d49700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda0406a510 0x7fda040c72f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:11.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.687+0000 7fda08d49700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fda0406d4b0 0x7fda040c41d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:11.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.687+0000 7fda08d49700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fda0406f450 0x7fda040c4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:11.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.687+0000 7fda08d49700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda040c97d0 con 0x7fda0406a510 2023-05-20T01:28:11.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.687+0000 7fda08d49700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda040c8b50 con 0x7fda0406d4b0 2023-05-20T01:28:11.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.687+0000 7fda08d49700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda040c8e30 con 0x7fda0406f450 2023-05-20T01:28:11.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.687+0000 7fda0259c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda0406a510 0x7fda040c72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:11.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.687+0000 7fda0259c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda0406a510 0x7fda040c72f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39604/0 (socket says 172.21.15.16:39604) 2023-05-20T01:28:11.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.687+0000 7fda0259c700 1 -- 172.21.15.16:0/2814200635 learned_addr learned my addr 172.21.15.16:0/2814200635 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:11.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.687+0000 7fda02d9d700 1 --2- 172.21.15.16:0/2814200635 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fda0406f450 0x7fda040c4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:11.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.687+0000 7fda01d9b700 1 --2- 172.21.15.16:0/2814200635 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fda0406d4b0 0x7fda040c41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:11.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.687+0000 7fda0259c700 1 -- 172.21.15.16:0/2814200635 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fda0406f450 msgr2=0x7fda040c4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:11.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.687+0000 7fda0259c700 1 --2- 172.21.15.16:0/2814200635 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fda0406f450 0x7fda040c4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:11.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.687+0000 7fda0259c700 1 -- 172.21.15.16:0/2814200635 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fda0406d4b0 msgr2=0x7fda040c41d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:11.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.687+0000 7fda0259c700 1 --2- 172.21.15.16:0/2814200635 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fda0406d4b0 0x7fda040c41d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:11.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.687+0000 7fda0259c700 1 -- 172.21.15.16:0/2814200635 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9f80094a0 con 0x7fda0406a510 2023-05-20T01:28:11.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.687+0000 7fda02d9d700 1 --2- 172.21.15.16:0/2814200635 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fda0406f450 0x7fda040c4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:28:11.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.687+0000 7fda01d9b700 1 --2- 172.21.15.16:0/2814200635 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fda0406d4b0 0x7fda040c41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:28:11.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.688+0000 7fda0259c700 1 --2- 172.21.15.16:0/2814200635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda0406a510 0x7fda040c72f0 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7fd9f8014040 tx=0x7fd9f8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:11.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.688+0000 7fd9f2ffd700 1 -- 172.21.15.16:0/2814200635 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f80038a0 con 0x7fda0406a510 2023-05-20T01:28:11.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.688+0000 7fd9f2ffd700 1 -- 172.21.15.16:0/2814200635 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9f801e070 con 0x7fda0406a510 2023-05-20T01:28:11.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.688+0000 7fda08d49700 1 -- 172.21.15.16:0/2814200635 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda040cbe50 con 0x7fda0406a510 2023-05-20T01:28:11.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.688+0000 7fd9f2ffd700 1 -- 172.21.15.16:0/2814200635 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f8003c40 con 0x7fda0406a510 2023-05-20T01:28:11.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.688+0000 7fda08d49700 1 -- 172.21.15.16:0/2814200635 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda040cc1a0 con 0x7fda0406a510 2023-05-20T01:28:11.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.689+0000 7fda08d49700 1 -- 172.21.15.16:0/2814200635 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda040cc9a0 con 0x7fda0406a510 2023-05-20T01:28:11.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.689+0000 7fd9f2ffd700 1 -- 172.21.15.16:0/2814200635 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd9f802d450 con 0x7fda0406a510 2023-05-20T01:28:11.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.690+0000 7fd9f2ffd700 1 --2- 172.21.15.16:0/2814200635 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd9e406ace0 0x7fd9e406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:11.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.690+0000 7fda01d9b700 1 --2- 172.21.15.16:0/2814200635 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd9e406ace0 0x7fd9e406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:11.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.690+0000 7fd9f2ffd700 1 -- 172.21.15.16:0/2814200635 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 7073+0+0 (secure 0 0 0) 0x7fd9f808aa20 con 0x7fda0406a510 2023-05-20T01:28:11.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.690+0000 7fda01d9b700 1 --2- 172.21.15.16:0/2814200635 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd9e406ace0 0x7fd9e406d190 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fda0414dcd0 tx=0x7fd9f4007ca0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:11.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.694+0000 7fd9f2ffd700 1 -- 172.21.15.16:0/2814200635 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fd9f8055760 con 0x7fda0406a510 2023-05-20T01:28:11.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.934+0000 7fda08d49700 1 -- 172.21.15.16:0/2814200635 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fda040747b0 con 0x7fda0406a510 2023-05-20T01:28:11.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.935+0000 7fd9f2ffd700 1 -- 172.21.15.16:0/2814200635 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+4681 (secure 0 0 0) 0x7fd9f8020090 con 0x7fda0406a510 2023-05-20T01:28:11.939 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:28:11.940 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":91,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6845,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/1435235487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1435235487},{"type":"v1","addr":"172.21.15.132:6834","nonce":1435235487}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":87},{"gid":6848,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/933893285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":933893285},{"type":"v1","addr":"172.21.15.132:6835","nonce":933893285}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":87},{"gid":8373,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/1220815342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1220815342},{"type":"v1","addr":"172.21.15.16:6836","nonce":1220815342}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":87}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-20T01:27:10.291795+0000","modified":"2023-05-20T01:27:25.446248+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":121,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8376},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8376":{"gid":8376,"name":"c","rank":0,"incarnation":88,"state":"up:active","state_seq":4,"addr":"172.21.15.16:6837/1927078882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":1927078882},{"type":"v1","addr":"172.21.15.16:6837","nonce":1927078882}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16,17],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-05-20T01:28:11.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.937+0000 7fda08d49700 1 -- 172.21.15.16:0/2814200635 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd9e406ace0 msgr2=0x7fd9e406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:11.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.937+0000 7fda08d49700 1 --2- 172.21.15.16:0/2814200635 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd9e406ace0 0x7fd9e406d190 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fda0414dcd0 tx=0x7fd9f4007ca0 comp rx=0 tx=0).stop 2023-05-20T01:28:11.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.937+0000 7fda08d49700 1 -- 172.21.15.16:0/2814200635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda0406a510 msgr2=0x7fda040c72f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:11.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.937+0000 7fda08d49700 1 --2- 172.21.15.16:0/2814200635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda0406a510 0x7fda040c72f0 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7fd9f8014040 tx=0x7fd9f8004750 comp rx=0 tx=0).stop 2023-05-20T01:28:11.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.937+0000 7fda08d49700 1 -- 172.21.15.16:0/2814200635 shutdown_connections 2023-05-20T01:28:11.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.937+0000 7fda08d49700 1 --2- 172.21.15.16:0/2814200635 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd9e406ace0 0x7fd9e406d190 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:11.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.937+0000 7fda08d49700 1 --2- 172.21.15.16:0/2814200635 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fda0406f450 0x7fda040c4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:11.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.937+0000 7fda08d49700 1 --2- 172.21.15.16:0/2814200635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda0406a510 0x7fda040c72f0 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:11.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.937+0000 7fda08d49700 1 --2- 172.21.15.16:0/2814200635 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fda0406d4b0 0x7fda040c41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:11.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.937+0000 7fda08d49700 1 -- 172.21.15.16:0/2814200635 >> 172.21.15.16:0/2814200635 conn(0x7fda040cffb0 msgr2=0x7fda040ccda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:11.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.937+0000 7fda08d49700 1 -- 172.21.15.16:0/2814200635 shutdown_connections 2023-05-20T01:28:11.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:11.937+0000 7fda08d49700 1 -- 172.21.15.16:0/2814200635 wait complete. 2023-05-20T01:28:11.943 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 91 2023-05-20T01:28:11.953 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-20T01:28:12.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.073+0000 7f08430be700 1 Processor -- start 2023-05-20T01:28:12.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.074+0000 7f08430be700 1 -- start start 2023-05-20T01:28:12.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.074+0000 7f08430be700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f083c0ccec0 0x7f083c0713c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:12.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.074+0000 7f08430be700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f083c071900 0x7f083c0ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:12.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.074+0000 7f08430be700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f083c0cb590 con 0x7f083c157190 2023-05-20T01:28:12.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.074+0000 7f08430be700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f083c0ca9a0 con 0x7f083c0ccec0 2023-05-20T01:28:12.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.074+0000 7f08430be700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f083c0cac60 con 0x7f083c071900 2023-05-20T01:28:12.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.074+0000 7f083bfff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f083c0ccec0 0x7f083c0713c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:12.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.074+0000 7f0840e5a700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f083c157190 0x7f083c0cc7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51140/0 (socket says 172.21.15.16:51140) 2023-05-20T01:28:12.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.075+0000 7f0840e5a700 1 -- 172.21.15.16:0/2217345499 learned_addr learned my addr 172.21.15.16:0/2217345499 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:28:12.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.075+0000 7f084165b700 1 --2- 172.21.15.16:0/2217345499 >> v2:172.21.15.132:3300/0 conn(0x7f083c071900 0x7f083c0ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:12.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.075+0000 7f083bfff700 1 -- 172.21.15.16:0/2217345499 >> v2:172.21.15.132:3300/0 conn(0x7f083c071900 msgr2=0x7f083c0ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:12.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.075+0000 7f083bfff700 1 --2- 172.21.15.16:0/2217345499 >> v2:172.21.15.132:3300/0 conn(0x7f083c071900 0x7f083c0ce740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:12.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.075+0000 7f083bfff700 1 -- 172.21.15.16:0/2217345499 >> v1:172.21.15.16:6789/0 conn(0x7f083c157190 legacy=0x7f083c0cc7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:12.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.075+0000 7f083bfff700 1 -- 172.21.15.16:0/2217345499 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f083c0caf40 con 0x7f083c0ccec0 2023-05-20T01:28:12.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.075+0000 7f083b7fe700 1 -- 172.21.15.16:0/2217345499 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2237441927 0 0) 0x7f083c0cb590 con 0x7f083c157190 2023-05-20T01:28:12.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.075+0000 7f084165b700 1 --2- 172.21.15.16:0/2217345499 >> v2:172.21.15.132:3300/0 conn(0x7f083c071900 0x7f083c0ce740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:28:12.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.075+0000 7f083bfff700 1 --2- 172.21.15.16:0/2217345499 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f083c0ccec0 0x7f083c0713c0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f083c0afa80 tx=0x7f0828009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef255811bce030ff server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:12.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.075+0000 7f083b7fe700 1 -- 172.21.15.16:0/2217345499 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0828011830 con 0x7f083c0ccec0 2023-05-20T01:28:12.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.075+0000 7f083b7fe700 1 -- 172.21.15.16:0/2217345499 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0828011990 con 0x7f083c0ccec0 2023-05-20T01:28:12.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.075+0000 7f083b7fe700 1 -- 172.21.15.16:0/2217345499 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0828011d20 con 0x7f083c0ccec0 2023-05-20T01:28:12.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.076+0000 7f08430be700 1 -- 172.21.15.16:0/2217345499 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f083c0ccec0 msgr2=0x7f083c0713c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:12.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.076+0000 7f08430be700 1 --2- 172.21.15.16:0/2217345499 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f083c0ccec0 0x7f083c0713c0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f083c0afa80 tx=0x7f0828009b20 comp rx=0 tx=0).stop 2023-05-20T01:28:12.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.076+0000 7f08430be700 1 -- 172.21.15.16:0/2217345499 shutdown_connections 2023-05-20T01:28:12.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.076+0000 7f08430be700 1 --2- 172.21.15.16:0/2217345499 >> v2:172.21.15.132:3300/0 conn(0x7f083c071900 0x7f083c0ce740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:12.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.076+0000 7f08430be700 1 --2- 172.21.15.16:0/2217345499 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f083c0ccec0 0x7f083c0713c0 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:12.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.076+0000 7f08430be700 1 -- 172.21.15.16:0/2217345499 >> 172.21.15.16:0/2217345499 conn(0x7f083c0cffb0 msgr2=0x7f083c0d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:12.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.076+0000 7f08430be700 1 -- 172.21.15.16:0/2217345499 shutdown_connections 2023-05-20T01:28:12.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.076+0000 7f08430be700 1 -- 172.21.15.16:0/2217345499 wait complete. 2023-05-20T01:28:12.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.077+0000 7f08430be700 1 Processor -- start 2023-05-20T01:28:12.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.077+0000 7f08430be700 1 -- start start 2023-05-20T01:28:12.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.078+0000 7f08430be700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f083c071900 0x7f083c149790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:12.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.078+0000 7f08430be700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f083c0ccec0 0x7f083c0b18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:12.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.078+0000 7f08430be700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f083c157190 0x7f083c155680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:12.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.078+0000 7f08430be700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f083c149cd0 con 0x7f083c071900 2023-05-20T01:28:12.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.078+0000 7f08430be700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f083c149f80 con 0x7f083c157190 2023-05-20T01:28:12.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.078+0000 7f08430be700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f083c14a260 con 0x7f083c0ccec0 2023-05-20T01:28:12.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.078+0000 7f0840e5a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f083c071900 0x7f083c149790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:12.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.078+0000 7f0840e5a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f083c071900 0x7f083c149790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39618/0 (socket says 172.21.15.16:39618) 2023-05-20T01:28:12.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.078+0000 7f0840e5a700 1 -- 172.21.15.16:0/1208744367 learned_addr learned my addr 172.21.15.16:0/1208744367 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:12.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.078+0000 7f0840e5a700 1 -- 172.21.15.16:0/1208744367 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f083c0ccec0 msgr2=0x7f083c0b18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:12.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.078+0000 7f0840e5a700 1 --2- 172.21.15.16:0/1208744367 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f083c0ccec0 0x7f083c0b18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:12.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.078+0000 7f0840e5a700 1 -- 172.21.15.16:0/1208744367 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f083c157190 msgr2=0x7f083c155680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:12.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.078+0000 7f0840e5a700 1 --2- 172.21.15.16:0/1208744367 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f083c157190 0x7f083c155680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:12.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.078+0000 7f0840e5a700 1 -- 172.21.15.16:0/1208744367 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08280094a0 con 0x7f083c071900 2023-05-20T01:28:12.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.078+0000 7f0840e5a700 1 --2- 172.21.15.16:0/1208744367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f083c071900 0x7f083c149790 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f083000c550 tx=0x7f083000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:12.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.079+0000 7f08397fa700 1 -- 172.21.15.16:0/1208744367 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0830004c80 con 0x7f083c071900 2023-05-20T01:28:12.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.079+0000 7f08397fa700 1 -- 172.21.15.16:0/1208744367 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0830004e00 con 0x7f083c071900 2023-05-20T01:28:12.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.079+0000 7f08430be700 1 -- 172.21.15.16:0/1208744367 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f083c157560 con 0x7f083c071900 2023-05-20T01:28:12.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.079+0000 7f08397fa700 1 -- 172.21.15.16:0/1208744367 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0830007310 con 0x7f083c071900 2023-05-20T01:28:12.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.079+0000 7f08430be700 1 -- 172.21.15.16:0/1208744367 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f083c0cb220 con 0x7f083c071900 2023-05-20T01:28:12.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.080+0000 7f08430be700 1 -- 172.21.15.16:0/1208744367 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f083c0cba00 con 0x7f083c071900 2023-05-20T01:28:12.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.080+0000 7f08397fa700 1 -- 172.21.15.16:0/1208744367 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0830007470 con 0x7f083c071900 2023-05-20T01:28:12.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.080+0000 7f08397fa700 1 --2- 172.21.15.16:0/1208744367 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f082006ace0 0x7f082006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:12.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.080+0000 7f08397fa700 1 -- 172.21.15.16:0/1208744367 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 7073+0+0 (secure 0 0 0) 0x7f08300895c0 con 0x7f083c071900 2023-05-20T01:28:12.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.080+0000 7f083bfff700 1 --2- 172.21.15.16:0/1208744367 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f082006ace0 0x7f082006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:12.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.081+0000 7f083bfff700 1 --2- 172.21.15.16:0/1208744367 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f082006ace0 0x7f082006d190 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f083c1f5c20 tx=0x7f0828010000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:12.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.085+0000 7f08397fa700 1 -- 172.21.15.16:0/1208744367 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f0830054270 con 0x7f083c071900 2023-05-20T01:28:12.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.283+0000 7f08430be700 1 -- 172.21.15.16:0/1208744367 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f083c0747b0 con 0x7f083c071900 2023-05-20T01:28:12.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.284+0000 7f08397fa700 1 -- 172.21.15.16:0/1208744367 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v123) v1 ==== 74+0+20629 (secure 0 0 0) 0x7f0830057890 con 0x7f083c071900 2023-05-20T01:28:12.289 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:28:12.289 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":123,"fsid":"c2245cc5-ecab-4087-a59b-bc81db901676","created":"2023-05-20T01:16:09.549230+0000","modified":"2023-05-20T01:27:21.615250+0000","last_up_change":"2023-05-20T01:16:19.788624+0000","last_in_change":"2023-05-20T01:16:12.092160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-05-20T01:16:20.934366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-05-20T01:16:21.372294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-05-20T01:26:34.067017+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-05-20T01:26:35.040602+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data_extra","create_time":"2023-05-20T01:26:48.309738+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"317a83cd-19c8-4958-a301-6087ef342289","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6825","nonce":3923634179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6827","nonce":3923634179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6831","nonce":3923634179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6829","nonce":3923634179}]},"public_addr":"172.21.15.16:6825/3923634179","cluster_addr":"172.21.15.16:6827/3923634179","heartbeat_back_addr":"172.21.15.16:6831/3923634179","heartbeat_front_addr":"172.21.15.16:6829/3923634179","state":["exists","up"]},{"osd":1,"uuid":"5581046f-8219-4089-8d7d-b7846399505e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6809","nonce":3860264220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6811","nonce":3860264220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6815","nonce":3860264220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6813","nonce":3860264220}]},"public_addr":"172.21.15.16:6809/3860264220","cluster_addr":"172.21.15.16:6811/3860264220","heartbeat_back_addr":"172.21.15.16:6815/3860264220","heartbeat_front_addr":"172.21.15.16:6813/3860264220","state":["exists","up"]},{"osd":2,"uuid":"072e9e66-4133-478e-8d4a-64d46c2963f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6817","nonce":293187030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6819","nonce":293187030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6823","nonce":293187030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6821","nonce":293187030}]},"public_addr":"172.21.15.16:6817/293187030","cluster_addr":"172.21.15.16:6819/293187030","heartbeat_back_addr":"172.21.15.16:6823/293187030","heartbeat_front_addr":"172.21.15.16:6821/293187030","state":["exists","up"]},{"osd":3,"uuid":"11e1bba6-00f8-48dd-9028-d8d716b7f145","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6801","nonce":3669781890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6803","nonce":3669781890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6807","nonce":3669781890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6805","nonce":3669781890}]},"public_addr":"172.21.15.16:6801/3669781890","cluster_addr":"172.21.15.16:6803/3669781890","heartbeat_back_addr":"172.21.15.16:6807/3669781890","heartbeat_front_addr":"172.21.15.16:6805/3669781890","state":["exists","up"]},{"osd":4,"uuid":"b8909c3a-cf70-46cb-8d0b-009c707cf9d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6819","nonce":506317223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6823","nonce":506317223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6829","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6831","nonce":506317223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6827","nonce":506317223}]},"public_addr":"172.21.15.132:6819/506317223","cluster_addr":"172.21.15.132:6823/506317223","heartbeat_back_addr":"172.21.15.132:6831/506317223","heartbeat_front_addr":"172.21.15.132:6827/506317223","state":["exists","up"]},{"osd":5,"uuid":"b1818304-4583-47bf-a6ec-0bfbb5166233","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6818","nonce":1844689995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6822","nonce":1844689995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6830","nonce":1844689995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6826","nonce":1844689995}]},"public_addr":"172.21.15.132:6818/1844689995","cluster_addr":"172.21.15.132:6822/1844689995","heartbeat_back_addr":"172.21.15.132:6830/1844689995","heartbeat_front_addr":"172.21.15.132:6826/1844689995","state":["exists","up"]},{"osd":6,"uuid":"d37f4691-eed1-4a93-8818-18354f03f4a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6801","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6802","nonce":727581123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6805","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6807","nonce":727581123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6813","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6815","nonce":727581123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6811","nonce":727581123}]},"public_addr":"172.21.15.132:6802/727581123","cluster_addr":"172.21.15.132:6807/727581123","heartbeat_back_addr":"172.21.15.132:6815/727581123","heartbeat_front_addr":"172.21.15.132:6811/727581123","state":["exists","up"]},{"osd":7,"uuid":"4f712f62-5d1d-4816-8042-03209144f8b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6803","nonce":466001149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6806","nonce":466001149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6814","nonce":466001149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6810","nonce":466001149}]},"public_addr":"172.21.15.132:6803/466001149","cluster_addr":"172.21.15.132:6806/466001149","heartbeat_back_addr":"172.21.15.132:6814/466001149","heartbeat_front_addr":"172.21.15.132:6810/466001149","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.338863+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.278680+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305694+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.246835+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.276237+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.304441+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305421+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.267050+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.33","mappings":[{"from":5,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6836/1584803225":"2023-05-21T01:27:21.602047+0000","172.21.15.16:6834/1584803225":"2023-05-21T01:27:21.602047+0000","172.21.15.16:6836/1281052551":"2023-05-21T01:27:10.544120+0000","172.21.15.16:6834/1281052551":"2023-05-21T01:27:10.544120+0000","172.21.15.16:6836/3765848068":"2023-05-21T01:27:09.119676+0000","172.21.15.16:6834/3765848068":"2023-05-21T01:27:09.119676+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-20T01:28:12.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.286+0000 7f08430be700 1 -- 172.21.15.16:0/1208744367 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f082006ace0 msgr2=0x7f082006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:12.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.286+0000 7f08430be700 1 --2- 172.21.15.16:0/1208744367 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f082006ace0 0x7f082006d190 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f083c1f5c20 tx=0x7f0828010000 comp rx=0 tx=0).stop 2023-05-20T01:28:12.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.286+0000 7f08430be700 1 -- 172.21.15.16:0/1208744367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f083c071900 msgr2=0x7f083c149790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:12.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.286+0000 7f08430be700 1 --2- 172.21.15.16:0/1208744367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f083c071900 0x7f083c149790 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f083000c550 tx=0x7f083000c910 comp rx=0 tx=0).stop 2023-05-20T01:28:12.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.287+0000 7f08430be700 1 -- 172.21.15.16:0/1208744367 shutdown_connections 2023-05-20T01:28:12.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.287+0000 7f08430be700 1 --2- 172.21.15.16:0/1208744367 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f083c157190 0x7f083c155680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:12.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.287+0000 7f08430be700 1 --2- 172.21.15.16:0/1208744367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f083c071900 0x7f083c149790 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:12.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.287+0000 7f08430be700 1 --2- 172.21.15.16:0/1208744367 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f082006ace0 0x7f082006d190 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:12.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.287+0000 7f08430be700 1 --2- 172.21.15.16:0/1208744367 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f083c0ccec0 0x7f083c0b18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:12.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.287+0000 7f08430be700 1 -- 172.21.15.16:0/1208744367 >> 172.21.15.16:0/1208744367 conn(0x7f083c0cffb0 msgr2=0x7f083c159700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:12.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.287+0000 7f08430be700 1 -- 172.21.15.16:0/1208744367 shutdown_connections 2023-05-20T01:28:12.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.287+0000 7f08430be700 1 -- 172.21.15.16:0/1208744367 wait complete. 2023-05-20T01:28:12.303 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-05-20T01:28:12.304 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-20T01:28:12.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.424+0000 7fe2ae2e1700 1 Processor -- start 2023-05-20T01:28:12.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.424+0000 7fe2ae2e1700 1 -- start start 2023-05-20T01:28:12.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.424+0000 7fe2ae2e1700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fe2a806f450 0x7fe2a80cf890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:12.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.424+0000 7fe2ae2e1700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe2a806d4b0 0x7fe2a8161080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:12.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.424+0000 7fe2ae2e1700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe2a806a510 0x7fe2a806a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:12.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.424+0000 7fe2ae2e1700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2a806aed0 con 0x7fe2a806d4b0 2023-05-20T01:28:12.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.424+0000 7fe2ae2e1700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2a80cc510 con 0x7fe2a806f450 2023-05-20T01:28:12.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.424+0000 7fe2ae2e1700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2a81615c0 con 0x7fe2a806a510 2023-05-20T01:28:12.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.425+0000 7fe2a77fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe2a806d4b0 0x7fe2a8161080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:12.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.425+0000 7fe2a77fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe2a806d4b0 0x7fe2a8161080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39628/0 (socket says 172.21.15.16:39628) 2023-05-20T01:28:12.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.425+0000 7fe2a77fe700 1 -- 172.21.15.16:0/3829497899 learned_addr learned my addr 172.21.15.16:0/3829497899 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:12.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.425+0000 7fe2ac87e700 1 --2- 172.21.15.16:0/3829497899 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe2a806a510 0x7fe2a806a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:12.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.425+0000 7fe2a7fff700 1 --2- 172.21.15.16:0/3829497899 >> v2:172.21.15.132:3300/0 conn(0x7fe2a806f450 0x7fe2a80cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:12.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.425+0000 7fe2a77fe700 1 -- 172.21.15.16:0/3829497899 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe2a806a510 msgr2=0x7fe2a806a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:12.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.425+0000 7fe2a77fe700 1 --2- 172.21.15.16:0/3829497899 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe2a806a510 0x7fe2a806a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:12.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.425+0000 7fe2a77fe700 1 -- 172.21.15.16:0/3829497899 >> v2:172.21.15.132:3300/0 conn(0x7fe2a806f450 msgr2=0x7fe2a80cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:12.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.425+0000 7fe2a77fe700 1 --2- 172.21.15.16:0/3829497899 >> v2:172.21.15.132:3300/0 conn(0x7fe2a806f450 0x7fe2a80cf890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:12.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.425+0000 7fe2a77fe700 1 -- 172.21.15.16:0/3829497899 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2a8071ed0 con 0x7fe2a806d4b0 2023-05-20T01:28:12.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.425+0000 7fe2ac87e700 1 --2- 172.21.15.16:0/3829497899 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe2a806a510 0x7fe2a806a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:28:12.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.425+0000 7fe2a7fff700 1 --2- 172.21.15.16:0/3829497899 >> v2:172.21.15.132:3300/0 conn(0x7fe2a806f450 0x7fe2a80cf890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:28:12.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.425+0000 7fe2a77fe700 1 --2- 172.21.15.16:0/3829497899 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2a806d4b0 0x7fe2a8161080 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7fe2a80cdb00 tx=0x7fe298009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=398a090cc4890314 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:12.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.425+0000 7fe2a6ffd700 1 -- 172.21.15.16:0/3829497899 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe29800b870 con 0x7fe2a806d4b0 2023-05-20T01:28:12.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.425+0000 7fe2a6ffd700 1 -- 172.21.15.16:0/3829497899 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe29800b9d0 con 0x7fe2a806d4b0 2023-05-20T01:28:12.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.425+0000 7fe2a6ffd700 1 -- 172.21.15.16:0/3829497899 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe29800bd60 con 0x7fe2a806d4b0 2023-05-20T01:28:12.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.426+0000 7fe2ae2e1700 1 -- 172.21.15.16:0/3829497899 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2a806d4b0 msgr2=0x7fe2a8161080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:12.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.426+0000 7fe2ae2e1700 1 --2- 172.21.15.16:0/3829497899 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2a806d4b0 0x7fe2a8161080 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7fe2a80cdb00 tx=0x7fe298009bd0 comp rx=0 tx=0).stop 2023-05-20T01:28:12.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.426+0000 7fe2ae2e1700 1 -- 172.21.15.16:0/3829497899 shutdown_connections 2023-05-20T01:28:12.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.426+0000 7fe2ae2e1700 1 --2- 172.21.15.16:0/3829497899 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe2a806a510 0x7fe2a806a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:12.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.426+0000 7fe2ae2e1700 1 --2- 172.21.15.16:0/3829497899 >> v2:172.21.15.132:3300/0 conn(0x7fe2a806f450 0x7fe2a80cf890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:12.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.426+0000 7fe2ae2e1700 1 --2- 172.21.15.16:0/3829497899 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2a806d4b0 0x7fe2a8161080 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:12.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.426+0000 7fe2ae2e1700 1 -- 172.21.15.16:0/3829497899 >> 172.21.15.16:0/3829497899 conn(0x7fe2a80cffb0 msgr2=0x7fe2a80d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:12.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.426+0000 7fe2ae2e1700 1 -- 172.21.15.16:0/3829497899 shutdown_connections 2023-05-20T01:28:12.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.426+0000 7fe2ae2e1700 1 -- 172.21.15.16:0/3829497899 wait complete. 2023-05-20T01:28:12.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.427+0000 7fe2ae2e1700 1 Processor -- start 2023-05-20T01:28:12.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.427+0000 7fe2ae2e1700 1 -- start start 2023-05-20T01:28:12.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.428+0000 7fe2ae2e1700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2a806a510 0x7fe2a814c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:12.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.428+0000 7fe2ae2e1700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe2a806d4b0 0x7fe2a80c8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:12.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.428+0000 7fe2ae2e1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2a806f450 0x7fe2a80c85d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:12.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.428+0000 7fe2ae2e1700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2a815ffd0 con 0x7fe2a806f450 2023-05-20T01:28:12.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.428+0000 7fe2ae2e1700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2a81602b0 con 0x7fe2a806a510 2023-05-20T01:28:12.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.428+0000 7fe2ae2e1700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2a8160590 con 0x7fe2a806d4b0 2023-05-20T01:28:12.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.428+0000 7fe2ac87e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2a806f450 0x7fe2a80c85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:12.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.428+0000 7fe2ac87e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2a806f450 0x7fe2a80c85d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39636/0 (socket says 172.21.15.16:39636) 2023-05-20T01:28:12.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.428+0000 7fe2ac87e700 1 -- 172.21.15.16:0/1710849808 learned_addr learned my addr 172.21.15.16:0/1710849808 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:12.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.428+0000 7fe2a77fe700 1 --2- 172.21.15.16:0/1710849808 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe2a806d4b0 0x7fe2a80c8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:12.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.428+0000 7fe2a7fff700 1 --2- 172.21.15.16:0/1710849808 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2a806a510 0x7fe2a814c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:12.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.428+0000 7fe2ac87e700 1 -- 172.21.15.16:0/1710849808 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe2a806d4b0 msgr2=0x7fe2a80c8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:12.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.428+0000 7fe2ac87e700 1 --2- 172.21.15.16:0/1710849808 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe2a806d4b0 0x7fe2a80c8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:12.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.428+0000 7fe2ac87e700 1 -- 172.21.15.16:0/1710849808 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2a806a510 msgr2=0x7fe2a814c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:12.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.428+0000 7fe2ac87e700 1 --2- 172.21.15.16:0/1710849808 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2a806a510 0x7fe2a814c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:12.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.428+0000 7fe2ac87e700 1 -- 172.21.15.16:0/1710849808 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe298009550 con 0x7fe2a806f450 2023-05-20T01:28:12.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.428+0000 7fe2a77fe700 1 --2- 172.21.15.16:0/1710849808 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe2a806d4b0 0x7fe2a80c8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:28:12.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.428+0000 7fe2a7fff700 1 --2- 172.21.15.16:0/1710849808 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2a806a510 0x7fe2a814c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:28:12.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.429+0000 7fe2ac87e700 1 --2- 172.21.15.16:0/1710849808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2a806f450 0x7fe2a80c85d0 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7fe2a000cef0 tx=0x7fe2a000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:12.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.429+0000 7fe2a4ff9700 1 -- 172.21.15.16:0/1710849808 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a000add0 con 0x7fe2a806f450 2023-05-20T01:28:12.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.429+0000 7fe2a4ff9700 1 -- 172.21.15.16:0/1710849808 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2a0015440 con 0x7fe2a806f450 2023-05-20T01:28:12.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.429+0000 7fe2ae2e1700 1 -- 172.21.15.16:0/1710849808 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2a80cc1a0 con 0x7fe2a806f450 2023-05-20T01:28:12.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.429+0000 7fe2a4ff9700 1 -- 172.21.15.16:0/1710849808 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a00157f0 con 0x7fe2a806f450 2023-05-20T01:28:12.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.429+0000 7fe2ae2e1700 1 -- 172.21.15.16:0/1710849808 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2a80721b0 con 0x7fe2a806f450 2023-05-20T01:28:12.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.430+0000 7fe2ae2e1700 1 -- 172.21.15.16:0/1710849808 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2a80cbe50 con 0x7fe2a806f450 2023-05-20T01:28:12.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.430+0000 7fe2a4ff9700 1 -- 172.21.15.16:0/1710849808 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe2a000d430 con 0x7fe2a806f450 2023-05-20T01:28:12.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.431+0000 7fe2a4ff9700 1 --2- 172.21.15.16:0/1710849808 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe28806ace0 0x7fe28806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:12.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.431+0000 7fe2a7fff700 1 --2- 172.21.15.16:0/1710849808 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe28806ace0 0x7fe28806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:12.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.431+0000 7fe2a4ff9700 1 -- 172.21.15.16:0/1710849808 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 7073+0+0 (secure 0 0 0) 0x7fe2a0089e00 con 0x7fe2a806f450 2023-05-20T01:28:12.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.431+0000 7fe2a7fff700 1 --2- 172.21.15.16:0/1710849808 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe28806ace0 0x7fe28806d190 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fe29000aa20 tx=0x7fe290008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:12.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.435+0000 7fe2a4ff9700 1 -- 172.21.15.16:0/1710849808 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fe2a0054ab0 con 0x7fe2a806f450 2023-05-20T01:28:12.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.676+0000 7fe2ae2e1700 1 -- 172.21.15.16:0/1710849808 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe2a8160a50 con 0x7fe2a806f450 2023-05-20T01:28:12.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.676+0000 7fe2a4ff9700 1 -- 172.21.15.16:0/1710849808 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+4681 (secure 0 0 0) 0x7fe2a001a090 con 0x7fe2a806f450 2023-05-20T01:28:12.681 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:28:12.681 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":91,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6845,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/1435235487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1435235487},{"type":"v1","addr":"172.21.15.132:6834","nonce":1435235487}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":87},{"gid":6848,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/933893285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":933893285},{"type":"v1","addr":"172.21.15.132:6835","nonce":933893285}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":87},{"gid":8373,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/1220815342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1220815342},{"type":"v1","addr":"172.21.15.16:6836","nonce":1220815342}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":87}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-20T01:27:10.291795+0000","modified":"2023-05-20T01:27:25.446248+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":121,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8376},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8376":{"gid":8376,"name":"c","rank":0,"incarnation":88,"state":"up:active","state_seq":4,"addr":"172.21.15.16:6837/1927078882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":1927078882},{"type":"v1","addr":"172.21.15.16:6837","nonce":1927078882}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16,17],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-05-20T01:28:12.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.679+0000 7fe2ae2e1700 1 -- 172.21.15.16:0/1710849808 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe28806ace0 msgr2=0x7fe28806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:12.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.679+0000 7fe2ae2e1700 1 --2- 172.21.15.16:0/1710849808 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe28806ace0 0x7fe28806d190 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fe29000aa20 tx=0x7fe290008040 comp rx=0 tx=0).stop 2023-05-20T01:28:12.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.679+0000 7fe2ae2e1700 1 -- 172.21.15.16:0/1710849808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2a806f450 msgr2=0x7fe2a80c85d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:12.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.679+0000 7fe2ae2e1700 1 --2- 172.21.15.16:0/1710849808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2a806f450 0x7fe2a80c85d0 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7fe2a000cef0 tx=0x7fe2a000a340 comp rx=0 tx=0).stop 2023-05-20T01:28:12.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.679+0000 7fe2ae2e1700 1 -- 172.21.15.16:0/1710849808 shutdown_connections 2023-05-20T01:28:12.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.679+0000 7fe2ae2e1700 1 --2- 172.21.15.16:0/1710849808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2a806f450 0x7fe2a80c85d0 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:12.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.679+0000 7fe2ae2e1700 1 --2- 172.21.15.16:0/1710849808 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2a806a510 0x7fe2a814c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:12.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.679+0000 7fe2ae2e1700 1 --2- 172.21.15.16:0/1710849808 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe28806ace0 0x7fe28806d190 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:12.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.679+0000 7fe2ae2e1700 1 --2- 172.21.15.16:0/1710849808 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe2a806d4b0 0x7fe2a80c8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:12.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.679+0000 7fe2ae2e1700 1 -- 172.21.15.16:0/1710849808 >> 172.21.15.16:0/1710849808 conn(0x7fe2a80cffb0 msgr2=0x7fe2a815c320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:12.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.679+0000 7fe2ae2e1700 1 -- 172.21.15.16:0/1710849808 shutdown_connections 2023-05-20T01:28:12.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.679+0000 7fe2ae2e1700 1 -- 172.21.15.16:0/1710849808 wait complete. 2023-05-20T01:28:12.685 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 91 2023-05-20T01:28:12.695 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-20T01:28:12.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.815+0000 7f4b27d96700 1 Processor -- start 2023-05-20T01:28:12.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.815+0000 7f4b27d96700 1 -- start start 2023-05-20T01:28:12.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.816+0000 7f4b27d96700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f4b2015e2a0 0x7f4b200cf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:12.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.816+0000 7f4b27d96700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b2015e060 con 0x7f4b2006a510 2023-05-20T01:28:12.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.816+0000 7f4b27d96700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b2015d450 con 0x7f4b200cfb30 2023-05-20T01:28:12.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.816+0000 7f4b27d96700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b2015d730 con 0x7f4b2015e2a0 2023-05-20T01:28:12.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.816+0000 7f4b25b32700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f4b2006a510 0x7f4b2006a8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51168/0 (socket says 172.21.15.16:51168) 2023-05-20T01:28:12.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.816+0000 7f4b25b32700 1 -- 172.21.15.16:0/1883693905 learned_addr learned my addr 172.21.15.16:0/1883693905 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:28:12.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.817+0000 7f4b25331700 1 --2- 172.21.15.16:0/1883693905 >> v2:172.21.15.132:3300/0 conn(0x7f4b2015e2a0 0x7f4b200cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:12.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.817+0000 7f4b24b30700 1 -- 172.21.15.16:0/1883693905 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2348902459 0 0) 0x7f4b2015e060 con 0x7f4b2006a510 2023-05-20T01:28:12.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.817+0000 7f4b24b30700 1 -- 172.21.15.16:0/1883693905 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b0c0035b0 con 0x7f4b2006a510 2023-05-20T01:28:12.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.817+0000 7f4b24b30700 1 -- 172.21.15.16:0/1883693905 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4104894415 0 0) 0x7f4b0c0035b0 con 0x7f4b2006a510 2023-05-20T01:28:12.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.817+0000 7f4b24b30700 1 -- 172.21.15.16:0/1883693905 >> v2:172.21.15.132:3300/0 conn(0x7f4b2015e2a0 msgr2=0x7f4b200cf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:12.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.817+0000 7f4b24b30700 1 --2- 172.21.15.16:0/1883693905 >> v2:172.21.15.132:3300/0 conn(0x7f4b2015e2a0 0x7f4b200cf5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:12.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.817+0000 7f4b24b30700 1 -- 172.21.15.16:0/1883693905 >> v1:172.21.15.132:6789/0 conn(0x7f4b200cfb30 legacy=0x7f4b20071340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:12.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.817+0000 7f4b24b30700 1 -- 172.21.15.16:0/1883693905 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b2015da10 con 0x7f4b2006a510 2023-05-20T01:28:12.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.817+0000 7f4b24b30700 1 -- 172.21.15.16:0/1883693905 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7f4b10001670 con 0x7f4b2006a510 2023-05-20T01:28:12.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.817+0000 7f4b24b30700 1 -- 172.21.15.16:0/1883693905 >> v1:172.21.15.16:6789/0 conn(0x7f4b2006a510 legacy=0x7f4b2006a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:12.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.817+0000 7f4b24b30700 1 --2- 172.21.15.16:0/1883693905 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b0c003a70 0x7f4b0c003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:12.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.817+0000 7f4b24b30700 1 --2- 172.21.15.16:0/1883693905 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b0c004850 0x7f4b0c004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:12.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.817+0000 7f4b24b30700 1 --2- 172.21.15.16:0/1883693905 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b0c005200 0x7f4b0c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:12.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.817+0000 7f4b24b30700 1 -- 172.21.15.16:0/1883693905 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b2015d730 con 0x7f4b0c005200 2023-05-20T01:28:12.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.817+0000 7f4b24b30700 1 -- 172.21.15.16:0/1883693905 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b0c00c0e0 con 0x7f4b0c004850 2023-05-20T01:28:12.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.817+0000 7f4b24b30700 1 -- 172.21.15.16:0/1883693905 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b0c00c2a0 con 0x7f4b0c003a70 2023-05-20T01:28:12.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.817+0000 7f4b26333700 1 --2- 172.21.15.16:0/1883693905 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b0c005200 0x7f4b0c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:12.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.817+0000 7f4b25331700 1 --2- 172.21.15.16:0/1883693905 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b0c004850 0x7f4b0c004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:12.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.817+0000 7f4b25b32700 1 --2- 172.21.15.16:0/1883693905 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b0c003a70 0x7f4b0c003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:12.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.817+0000 7f4b26333700 1 -- 172.21.15.16:0/1883693905 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b0c003a70 msgr2=0x7f4b0c003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:12.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.817+0000 7f4b26333700 1 --2- 172.21.15.16:0/1883693905 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b0c003a70 0x7f4b0c003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:12.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.817+0000 7f4b26333700 1 -- 172.21.15.16:0/1883693905 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b0c004850 msgr2=0x7f4b0c004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:12.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.817+0000 7f4b26333700 1 --2- 172.21.15.16:0/1883693905 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b0c004850 0x7f4b0c004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:12.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.817+0000 7f4b26333700 1 -- 172.21.15.16:0/1883693905 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b0c0035b0 con 0x7f4b0c005200 2023-05-20T01:28:12.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.818+0000 7f4b26333700 1 -- 172.21.15.16:0/1883693905 reap_dead start 2023-05-20T01:28:12.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.818+0000 7f4b26333700 1 --2- 172.21.15.16:0/1883693905 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b0c005200 0x7f4b0c00ba10 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f4b1800c190 tx=0x7f4b0c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fedab939982e54 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:12.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.818+0000 7f4b24b30700 1 -- 172.21.15.16:0/1883693905 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b18001db0 con 0x7f4b0c005200 2023-05-20T01:28:12.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.818+0000 7f4b24b30700 1 -- 172.21.15.16:0/1883693905 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b1800ccf0 con 0x7f4b0c005200 2023-05-20T01:28:12.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.818+0000 7f4b24b30700 1 -- 172.21.15.16:0/1883693905 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b18003c40 con 0x7f4b0c005200 2023-05-20T01:28:12.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.818+0000 7f4b27d96700 1 -- 172.21.15.16:0/1883693905 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b0c005200 msgr2=0x7f4b0c00ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:12.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.818+0000 7f4b27d96700 1 --2- 172.21.15.16:0/1883693905 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b0c005200 0x7f4b0c00ba10 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f4b1800c190 tx=0x7f4b0c005880 comp rx=0 tx=0).stop 2023-05-20T01:28:12.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.818+0000 7f4b27d96700 1 -- 172.21.15.16:0/1883693905 shutdown_connections 2023-05-20T01:28:12.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.818+0000 7f4b27d96700 1 --2- 172.21.15.16:0/1883693905 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b0c005200 0x7f4b0c00ba10 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:12.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.818+0000 7f4b27d96700 1 -- 172.21.15.16:0/1883693905 >> 172.21.15.16:0/1883693905 conn(0x7f4b200d0c30 msgr2=0x7f4b200b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:12.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.818+0000 7f4b27d96700 1 -- 172.21.15.16:0/1883693905 shutdown_connections 2023-05-20T01:28:12.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.818+0000 7f4b27d96700 1 -- 172.21.15.16:0/1883693905 wait complete. 2023-05-20T01:28:12.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.820+0000 7f4b27d96700 1 Processor -- start 2023-05-20T01:28:12.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.820+0000 7f4b27d96700 1 -- start start 2023-05-20T01:28:12.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.820+0000 7f4b27d96700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b0c005200 0x7f4b200b3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:12.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.820+0000 7f4b27d96700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b200b2e40 0x7f4b200c4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:12.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.820+0000 7f4b27d96700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b200c5360 0x7f4b200c9090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:12.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.820+0000 7f4b27d96700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b201f6030 con 0x7f4b200c5360 2023-05-20T01:28:12.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.820+0000 7f4b27d96700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b201f62d0 con 0x7f4b0c005200 2023-05-20T01:28:12.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.820+0000 7f4b27d96700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b201f65b0 con 0x7f4b200b2e40 2023-05-20T01:28:12.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.821+0000 7f4b26333700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b200c5360 0x7f4b200c9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:12.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.821+0000 7f4b26333700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b200c5360 0x7f4b200c9090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39656/0 (socket says 172.21.15.16:39656) 2023-05-20T01:28:12.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.821+0000 7f4b26333700 1 -- 172.21.15.16:0/3158332944 learned_addr learned my addr 172.21.15.16:0/3158332944 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:12.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.821+0000 7f4b25b32700 1 --2- 172.21.15.16:0/3158332944 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b0c005200 0x7f4b200b3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:12.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.821+0000 7f4b25331700 1 --2- 172.21.15.16:0/3158332944 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b200b2e40 0x7f4b200c4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:12.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.821+0000 7f4b26333700 1 -- 172.21.15.16:0/3158332944 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b200b2e40 msgr2=0x7f4b200c4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:12.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.821+0000 7f4b26333700 1 --2- 172.21.15.16:0/3158332944 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b200b2e40 0x7f4b200c4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:12.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.821+0000 7f4b26333700 1 -- 172.21.15.16:0/3158332944 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b0c005200 msgr2=0x7f4b200b3930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:12.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.821+0000 7f4b26333700 1 --2- 172.21.15.16:0/3158332944 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b0c005200 0x7f4b200b3930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:12.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.821+0000 7f4b26333700 1 -- 172.21.15.16:0/3158332944 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b2015d080 con 0x7f4b200c5360 2023-05-20T01:28:12.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.821+0000 7f4b26333700 1 --2- 172.21.15.16:0/3158332944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b200c5360 0x7f4b200c9090 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f4b18005040 tx=0x7f4b180014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:12.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.821+0000 7f4b167fc700 1 -- 172.21.15.16:0/3158332944 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b180052d0 con 0x7f4b200c5360 2023-05-20T01:28:12.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.821+0000 7f4b167fc700 1 -- 172.21.15.16:0/3158332944 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b18005430 con 0x7f4b200c5360 2023-05-20T01:28:12.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.821+0000 7f4b27d96700 1 -- 172.21.15.16:0/3158332944 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b2015dcf0 con 0x7f4b200c5360 2023-05-20T01:28:12.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.821+0000 7f4b167fc700 1 -- 172.21.15.16:0/3158332944 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b180057a0 con 0x7f4b200c5360 2023-05-20T01:28:12.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.821+0000 7f4b27d96700 1 -- 172.21.15.16:0/3158332944 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b201f6b70 con 0x7f4b200c5360 2023-05-20T01:28:12.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.822+0000 7f4b27d96700 1 -- 172.21.15.16:0/3158332944 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b2015d210 con 0x7f4b200c5360 2023-05-20T01:28:12.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.823+0000 7f4b167fc700 1 -- 172.21.15.16:0/3158332944 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4b1801d480 con 0x7f4b200c5360 2023-05-20T01:28:12.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.823+0000 7f4b167fc700 1 --2- 172.21.15.16:0/3158332944 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f4b0406ace0 0x7f4b0406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:12.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.823+0000 7f4b167fc700 1 -- 172.21.15.16:0/3158332944 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 7073+0+0 (secure 0 0 0) 0x7f4b1808a3c0 con 0x7f4b200c5360 2023-05-20T01:28:12.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.823+0000 7f4b25b32700 1 --2- 172.21.15.16:0/3158332944 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f4b0406ace0 0x7f4b0406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:12.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.824+0000 7f4b25b32700 1 --2- 172.21.15.16:0/3158332944 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f4b0406ace0 0x7f4b0406d190 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f4b10000c00 tx=0x7f4b10007980 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:12.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:12.828+0000 7f4b167fc700 1 -- 172.21.15.16:0/3158332944 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f4b2015d210 con 0x7f4b200c5360 2023-05-20T01:28:13.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.072+0000 7f4b27d96700 1 -- 172.21.15.16:0/3158332944 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4b2015d210 con 0x7f4b200c5360 2023-05-20T01:28:13.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.073+0000 7f4b167fc700 1 -- 172.21.15.16:0/3158332944 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+4681 (secure 0 0 0) 0x7f4b2015d210 con 0x7f4b200c5360 2023-05-20T01:28:13.079 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:28:13.079 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":91,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6845,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/1435235487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1435235487},{"type":"v1","addr":"172.21.15.132:6834","nonce":1435235487}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":87},{"gid":6848,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/933893285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":933893285},{"type":"v1","addr":"172.21.15.132:6835","nonce":933893285}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":87},{"gid":8373,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/1220815342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1220815342},{"type":"v1","addr":"172.21.15.16:6836","nonce":1220815342}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":87}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-20T01:27:10.291795+0000","modified":"2023-05-20T01:27:25.446248+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":121,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8376},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8376":{"gid":8376,"name":"c","rank":0,"incarnation":88,"state":"up:active","state_seq":4,"addr":"172.21.15.16:6837/1927078882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":1927078882},{"type":"v1","addr":"172.21.15.16:6837","nonce":1927078882}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16,17],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-05-20T01:28:13.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.075+0000 7f4b27d96700 1 -- 172.21.15.16:0/3158332944 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f4b0406ace0 msgr2=0x7f4b0406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:13.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.075+0000 7f4b27d96700 1 --2- 172.21.15.16:0/3158332944 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f4b0406ace0 0x7f4b0406d190 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f4b10000c00 tx=0x7f4b10007980 comp rx=0 tx=0).stop 2023-05-20T01:28:13.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.075+0000 7f4b27d96700 1 -- 172.21.15.16:0/3158332944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b200c5360 msgr2=0x7f4b200c9090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:13.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.075+0000 7f4b27d96700 1 --2- 172.21.15.16:0/3158332944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b200c5360 0x7f4b200c9090 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f4b18005040 tx=0x7f4b180014a0 comp rx=0 tx=0).stop 2023-05-20T01:28:13.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.075+0000 7f4b27d96700 1 -- 172.21.15.16:0/3158332944 shutdown_connections 2023-05-20T01:28:13.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.075+0000 7f4b27d96700 1 --2- 172.21.15.16:0/3158332944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b200c5360 0x7f4b200c9090 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:13.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.075+0000 7f4b27d96700 1 --2- 172.21.15.16:0/3158332944 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b0c005200 0x7f4b200b3930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:13.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.075+0000 7f4b27d96700 1 --2- 172.21.15.16:0/3158332944 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f4b0406ace0 0x7f4b0406d190 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:13.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.075+0000 7f4b27d96700 1 --2- 172.21.15.16:0/3158332944 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b200b2e40 0x7f4b200c4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:13.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.075+0000 7f4b27d96700 1 -- 172.21.15.16:0/3158332944 >> 172.21.15.16:0/3158332944 conn(0x7f4b200d0c30 msgr2=0x7f4b2015e670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:13.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.075+0000 7f4b27d96700 1 -- 172.21.15.16:0/3158332944 shutdown_connections 2023-05-20T01:28:13.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.075+0000 7f4b27d96700 1 -- 172.21.15.16:0/3158332944 wait complete. 2023-05-20T01:28:13.082 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 91 2023-05-20T01:28:13.091 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-20T01:28:13.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.212+0000 7f67b9398700 1 Processor -- start 2023-05-20T01:28:13.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.212+0000 7f67b9398700 1 -- start start 2023-05-20T01:28:13.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.213+0000 7f67b9398700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f67b40d16c0 0x7f67b4144330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:13.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.213+0000 7f67b9398700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f67b40b1990 0x7f67b40cf0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:13.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.213+0000 7f67b9398700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67b4144870 con 0x7f67b40cec20 2023-05-20T01:28:13.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.213+0000 7f67b9398700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67b40b1e10 con 0x7f67b40d16c0 2023-05-20T01:28:13.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.213+0000 7f67b9398700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67b40cf600 con 0x7f67b40b1990 2023-05-20T01:28:13.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.213+0000 7f67b2ffd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f67b40d16c0 0x7f67b4144330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:13.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.213+0000 7f67b27fc700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f67b40cec20 0x7f67b40ae2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51190/0 (socket says 172.21.15.16:51190) 2023-05-20T01:28:13.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.213+0000 7f67b27fc700 1 -- 172.21.15.16:0/3736059226 learned_addr learned my addr 172.21.15.16:0/3736059226 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:28:13.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.213+0000 7f67b37fe700 1 --2- 172.21.15.16:0/3736059226 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f67b40b1990 0x7f67b40cf0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:13.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.214+0000 7f67b2ffd700 1 -- 172.21.15.16:0/3736059226 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f67b40b1990 msgr2=0x7f67b40cf0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:13.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.214+0000 7f67b2ffd700 1 --2- 172.21.15.16:0/3736059226 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f67b40b1990 0x7f67b40cf0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:13.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.214+0000 7f67b2ffd700 1 -- 172.21.15.16:0/3736059226 >> v1:172.21.15.16:6789/0 conn(0x7f67b40cec20 legacy=0x7f67b40ae2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:13.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.214+0000 7f67b2ffd700 1 -- 172.21.15.16:0/3736059226 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67b40b2000 con 0x7f67b40d16c0 2023-05-20T01:28:13.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.214+0000 7f67b1ffb700 1 -- 172.21.15.16:0/3736059226 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 634071465 0 0) 0x7f67b4144870 con 0x7f67b40cec20 2023-05-20T01:28:13.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.214+0000 7f67b2ffd700 1 --2- 172.21.15.16:0/3736059226 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67b40d16c0 0x7f67b4144330 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f67a4009870 tx=0x7f67a4009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c7ea9e6ad8fc7a9 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:13.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.214+0000 7f67b1ffb700 1 -- 172.21.15.16:0/3736059226 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a4011830 con 0x7f67b40d16c0 2023-05-20T01:28:13.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.214+0000 7f67b1ffb700 1 -- 172.21.15.16:0/3736059226 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67a4011990 con 0x7f67b40d16c0 2023-05-20T01:28:13.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.214+0000 7f67b1ffb700 1 -- 172.21.15.16:0/3736059226 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a4011d20 con 0x7f67b40d16c0 2023-05-20T01:28:13.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.214+0000 7f67b9398700 1 -- 172.21.15.16:0/3736059226 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67b40d16c0 msgr2=0x7f67b4144330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:13.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.214+0000 7f67b9398700 1 --2- 172.21.15.16:0/3736059226 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67b40d16c0 0x7f67b4144330 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f67a4009870 tx=0x7f67a4009b80 comp rx=0 tx=0).stop 2023-05-20T01:28:13.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.214+0000 7f67b9398700 1 -- 172.21.15.16:0/3736059226 shutdown_connections 2023-05-20T01:28:13.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.214+0000 7f67b9398700 1 --2- 172.21.15.16:0/3736059226 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f67b40b1990 0x7f67b40cf0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:13.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.214+0000 7f67b9398700 1 --2- 172.21.15.16:0/3736059226 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67b40d16c0 0x7f67b4144330 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:13.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.214+0000 7f67b9398700 1 -- 172.21.15.16:0/3736059226 >> 172.21.15.16:0/3736059226 conn(0x7f67b406ae30 msgr2=0x7f67b409a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:13.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.215+0000 7f67b9398700 1 -- 172.21.15.16:0/3736059226 shutdown_connections 2023-05-20T01:28:13.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.215+0000 7f67b9398700 1 -- 172.21.15.16:0/3736059226 wait complete. 2023-05-20T01:28:13.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.216+0000 7f67b9398700 1 Processor -- start 2023-05-20T01:28:13.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.216+0000 7f67b9398700 1 -- start start 2023-05-20T01:28:13.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.217+0000 7f67b9398700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f67b40b1990 0x7f67b40cc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:13.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.217+0000 7f67b9398700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f67b40cec20 0x7f67b41599e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:13.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.217+0000 7f67b9398700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67b40d16c0 0x7f67b40cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:13.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.217+0000 7f67b9398700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67b40cc030 con 0x7f67b40d16c0 2023-05-20T01:28:13.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.217+0000 7f67b9398700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67b40cb700 con 0x7f67b40cec20 2023-05-20T01:28:13.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.217+0000 7f67b9398700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67b40cb9e0 con 0x7f67b40b1990 2023-05-20T01:28:13.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.217+0000 7f67b37fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67b40d16c0 0x7f67b40cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:13.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.217+0000 7f67b37fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67b40d16c0 0x7f67b40cccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39670/0 (socket says 172.21.15.16:39670) 2023-05-20T01:28:13.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.217+0000 7f67b37fe700 1 -- 172.21.15.16:0/1482291616 learned_addr learned my addr 172.21.15.16:0/1482291616 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:13.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.218+0000 7f67b2ffd700 1 --2- 172.21.15.16:0/1482291616 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f67b40b1990 0x7f67b40cc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:13.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.218+0000 7f67b27fc700 1 --2- 172.21.15.16:0/1482291616 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f67b40cec20 0x7f67b41599e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:13.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.218+0000 7f67b37fe700 1 -- 172.21.15.16:0/1482291616 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f67b40b1990 msgr2=0x7f67b40cc4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:13.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.218+0000 7f67b37fe700 1 --2- 172.21.15.16:0/1482291616 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f67b40b1990 0x7f67b40cc4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:13.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.218+0000 7f67b37fe700 1 -- 172.21.15.16:0/1482291616 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f67b40cec20 msgr2=0x7f67b41599e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:13.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.218+0000 7f67b37fe700 1 --2- 172.21.15.16:0/1482291616 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f67b40cec20 0x7f67b41599e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:13.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.218+0000 7f67b37fe700 1 -- 172.21.15.16:0/1482291616 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67a40094a0 con 0x7f67b40d16c0 2023-05-20T01:28:13.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.218+0000 7f67b27fc700 1 --2- 172.21.15.16:0/1482291616 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f67b40cec20 0x7f67b41599e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:28:13.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.218+0000 7f67b2ffd700 1 --2- 172.21.15.16:0/1482291616 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f67b40b1990 0x7f67b40cc4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:28:13.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.218+0000 7f67b37fe700 1 --2- 172.21.15.16:0/1482291616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67b40d16c0 0x7f67b40cccb0 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f67ac007e60 tx=0x7f67ac00e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:13.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.218+0000 7f679b7fe700 1 -- 172.21.15.16:0/1482291616 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67ac00ede0 con 0x7f67b40d16c0 2023-05-20T01:28:13.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.218+0000 7f679b7fe700 1 -- 172.21.15.16:0/1482291616 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67ac014420 con 0x7f67b40d16c0 2023-05-20T01:28:13.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.218+0000 7f67b9398700 1 -- 172.21.15.16:0/1482291616 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67b40b25c0 con 0x7f67b40d16c0 2023-05-20T01:28:13.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.218+0000 7f679b7fe700 1 -- 172.21.15.16:0/1482291616 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67ac0147d0 con 0x7f67b40d16c0 2023-05-20T01:28:13.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.218+0000 7f67b9398700 1 -- 172.21.15.16:0/1482291616 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67b40b22e0 con 0x7f67b40d16c0 2023-05-20T01:28:13.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.219+0000 7f67b9398700 1 -- 172.21.15.16:0/1482291616 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67b40adda0 con 0x7f67b40d16c0 2023-05-20T01:28:13.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.220+0000 7f679b7fe700 1 -- 172.21.15.16:0/1482291616 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f67ac014930 con 0x7f67b40d16c0 2023-05-20T01:28:13.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.220+0000 7f679b7fe700 1 --2- 172.21.15.16:0/1482291616 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f67a006ace0 0x7f67a006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:13.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.220+0000 7f679b7fe700 1 -- 172.21.15.16:0/1482291616 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 7073+0+0 (secure 0 0 0) 0x7f67ac08a060 con 0x7f67b40d16c0 2023-05-20T01:28:13.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.220+0000 7f67b2ffd700 1 --2- 172.21.15.16:0/1482291616 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f67a006ace0 0x7f67a006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:13.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.221+0000 7f67b2ffd700 1 --2- 172.21.15.16:0/1482291616 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f67a006ace0 0x7f67a006d190 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f67a40090b0 tx=0x7f67a4008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:13.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.225+0000 7f679b7fe700 1 -- 172.21.15.16:0/1482291616 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f67ac054dc0 con 0x7f67b40d16c0 2023-05-20T01:28:13.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.423+0000 7f67b9398700 1 -- 172.21.15.16:0/1482291616 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f67b40747b0 con 0x7f67b40d16c0 2023-05-20T01:28:13.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.425+0000 7f679b7fe700 1 -- 172.21.15.16:0/1482291616 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v123) v1 ==== 74+0+20629 (secure 0 0 0) 0x7f67ac0583e0 con 0x7f67b40d16c0 2023-05-20T01:28:13.429 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:28:13.430 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":123,"fsid":"c2245cc5-ecab-4087-a59b-bc81db901676","created":"2023-05-20T01:16:09.549230+0000","modified":"2023-05-20T01:27:21.615250+0000","last_up_change":"2023-05-20T01:16:19.788624+0000","last_in_change":"2023-05-20T01:16:12.092160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-05-20T01:16:20.934366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-05-20T01:16:21.372294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-05-20T01:26:34.067017+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-05-20T01:26:35.040602+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data_extra","create_time":"2023-05-20T01:26:48.309738+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"317a83cd-19c8-4958-a301-6087ef342289","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6825","nonce":3923634179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6827","nonce":3923634179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6831","nonce":3923634179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6829","nonce":3923634179}]},"public_addr":"172.21.15.16:6825/3923634179","cluster_addr":"172.21.15.16:6827/3923634179","heartbeat_back_addr":"172.21.15.16:6831/3923634179","heartbeat_front_addr":"172.21.15.16:6829/3923634179","state":["exists","up"]},{"osd":1,"uuid":"5581046f-8219-4089-8d7d-b7846399505e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6809","nonce":3860264220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6811","nonce":3860264220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6815","nonce":3860264220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6813","nonce":3860264220}]},"public_addr":"172.21.15.16:6809/3860264220","cluster_addr":"172.21.15.16:6811/3860264220","heartbeat_back_addr":"172.21.15.16:6815/3860264220","heartbeat_front_addr":"172.21.15.16:6813/3860264220","state":["exists","up"]},{"osd":2,"uuid":"072e9e66-4133-478e-8d4a-64d46c2963f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6817","nonce":293187030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6819","nonce":293187030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6823","nonce":293187030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6821","nonce":293187030}]},"public_addr":"172.21.15.16:6817/293187030","cluster_addr":"172.21.15.16:6819/293187030","heartbeat_back_addr":"172.21.15.16:6823/293187030","heartbeat_front_addr":"172.21.15.16:6821/293187030","state":["exists","up"]},{"osd":3,"uuid":"11e1bba6-00f8-48dd-9028-d8d716b7f145","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6801","nonce":3669781890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6803","nonce":3669781890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6807","nonce":3669781890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6805","nonce":3669781890}]},"public_addr":"172.21.15.16:6801/3669781890","cluster_addr":"172.21.15.16:6803/3669781890","heartbeat_back_addr":"172.21.15.16:6807/3669781890","heartbeat_front_addr":"172.21.15.16:6805/3669781890","state":["exists","up"]},{"osd":4,"uuid":"b8909c3a-cf70-46cb-8d0b-009c707cf9d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6819","nonce":506317223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6823","nonce":506317223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6829","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6831","nonce":506317223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6827","nonce":506317223}]},"public_addr":"172.21.15.132:6819/506317223","cluster_addr":"172.21.15.132:6823/506317223","heartbeat_back_addr":"172.21.15.132:6831/506317223","heartbeat_front_addr":"172.21.15.132:6827/506317223","state":["exists","up"]},{"osd":5,"uuid":"b1818304-4583-47bf-a6ec-0bfbb5166233","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6818","nonce":1844689995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6822","nonce":1844689995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6830","nonce":1844689995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6826","nonce":1844689995}]},"public_addr":"172.21.15.132:6818/1844689995","cluster_addr":"172.21.15.132:6822/1844689995","heartbeat_back_addr":"172.21.15.132:6830/1844689995","heartbeat_front_addr":"172.21.15.132:6826/1844689995","state":["exists","up"]},{"osd":6,"uuid":"d37f4691-eed1-4a93-8818-18354f03f4a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6801","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6802","nonce":727581123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6805","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6807","nonce":727581123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6813","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6815","nonce":727581123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6811","nonce":727581123}]},"public_addr":"172.21.15.132:6802/727581123","cluster_addr":"172.21.15.132:6807/727581123","heartbeat_back_addr":"172.21.15.132:6815/727581123","heartbeat_front_addr":"172.21.15.132:6811/727581123","state":["exists","up"]},{"osd":7,"uuid":"4f712f62-5d1d-4816-8042-03209144f8b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6803","nonce":466001149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6806","nonce":466001149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6814","nonce":466001149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6810","nonce":466001149}]},"public_addr":"172.21.15.132:6803/466001149","cluster_addr":"172.21.15.132:6806/466001149","heartbeat_back_addr":"172.21.15.132:6814/466001149","heartbeat_front_addr":"172.21.15.132:6810/466001149","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.338863+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.278680+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305694+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.246835+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.276237+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.304441+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305421+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.267050+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.33","mappings":[{"from":5,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6836/1584803225":"2023-05-21T01:27:21.602047+0000","172.21.15.16:6834/1584803225":"2023-05-21T01:27:21.602047+0000","172.21.15.16:6836/1281052551":"2023-05-21T01:27:10.544120+0000","172.21.15.16:6834/1281052551":"2023-05-21T01:27:10.544120+0000","172.21.15.16:6836/3765848068":"2023-05-21T01:27:09.119676+0000","172.21.15.16:6834/3765848068":"2023-05-21T01:27:09.119676+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-20T01:28:13.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.426+0000 7f67b9398700 1 -- 172.21.15.16:0/1482291616 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f67a006ace0 msgr2=0x7f67a006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:13.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.426+0000 7f67b9398700 1 --2- 172.21.15.16:0/1482291616 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f67a006ace0 0x7f67a006d190 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f67a40090b0 tx=0x7f67a4008030 comp rx=0 tx=0).stop 2023-05-20T01:28:13.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.426+0000 7f67b9398700 1 -- 172.21.15.16:0/1482291616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67b40d16c0 msgr2=0x7f67b40cccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:13.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.426+0000 7f67b9398700 1 --2- 172.21.15.16:0/1482291616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67b40d16c0 0x7f67b40cccb0 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f67ac007e60 tx=0x7f67ac00e3c0 comp rx=0 tx=0).stop 2023-05-20T01:28:13.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.427+0000 7f67b9398700 1 -- 172.21.15.16:0/1482291616 shutdown_connections 2023-05-20T01:28:13.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.427+0000 7f67b9398700 1 --2- 172.21.15.16:0/1482291616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67b40d16c0 0x7f67b40cccb0 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:13.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.427+0000 7f67b9398700 1 --2- 172.21.15.16:0/1482291616 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f67a006ace0 0x7f67a006d190 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:13.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.427+0000 7f67b9398700 1 --2- 172.21.15.16:0/1482291616 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f67b40b1990 0x7f67b40cc4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:13.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.427+0000 7f67b9398700 1 --2- 172.21.15.16:0/1482291616 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f67b40cec20 0x7f67b41599e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:13.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.427+0000 7f67b9398700 1 -- 172.21.15.16:0/1482291616 >> 172.21.15.16:0/1482291616 conn(0x7f67b406ae30 msgr2=0x7f67b415d130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:13.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.427+0000 7f67b9398700 1 -- 172.21.15.16:0/1482291616 shutdown_connections 2023-05-20T01:28:13.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.427+0000 7f67b9398700 1 -- 172.21.15.16:0/1482291616 wait complete. 2023-05-20T01:28:13.443 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-05-20T01:28:13.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.564+0000 7ff460ee4700 1 Processor -- start 2023-05-20T01:28:13.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.564+0000 7ff460ee4700 1 -- start start 2023-05-20T01:28:13.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.565+0000 7ff460ee4700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7ff45c0cb670 0x7ff45c0ce340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:13.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.565+0000 7ff460ee4700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff45c159d90 con 0x7ff45c1599c0 2023-05-20T01:28:13.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.565+0000 7ff460ee4700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff45c0cdb80 con 0x7ff45c0caaa0 2023-05-20T01:28:13.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.565+0000 7ff460ee4700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff45c0cdf20 con 0x7ff45c0cb670 2023-05-20T01:28:13.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.566+0000 7ff45a59c700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ff45c1599c0 0x7ff45c0caf60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51202/0 (socket says 172.21.15.16:51202) 2023-05-20T01:28:13.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.566+0000 7ff45a59c700 1 -- 172.21.15.16:0/3529851768 learned_addr learned my addr 172.21.15.16:0/3529851768 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:28:13.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.566+0000 7ff459d9b700 1 --2- 172.21.15.16:0/3529851768 >> v2:172.21.15.132:3300/0 conn(0x7ff45c0cb670 0x7ff45c0ce340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:13.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.566+0000 7ff45959a700 1 -- 172.21.15.16:0/3529851768 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2273973131 0 0) 0x7ff450001110 con 0x7ff45c1599c0 2023-05-20T01:28:13.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.566+0000 7ff45959a700 1 -- 172.21.15.16:0/3529851768 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff4440035b0 con 0x7ff45c1599c0 2023-05-20T01:28:13.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.566+0000 7ff45959a700 1 -- 172.21.15.16:0/3529851768 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2547376366 0 0) 0x7ff4440035b0 con 0x7ff45c1599c0 2023-05-20T01:28:13.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.566+0000 7ff45959a700 1 -- 172.21.15.16:0/3529851768 >> v2:172.21.15.132:3300/0 conn(0x7ff45c0cb670 msgr2=0x7ff45c0ce340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:13.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.566+0000 7ff45959a700 1 --2- 172.21.15.16:0/3529851768 >> v2:172.21.15.132:3300/0 conn(0x7ff45c0cb670 0x7ff45c0ce340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:13.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.566+0000 7ff45959a700 1 -- 172.21.15.16:0/3529851768 >> v1:172.21.15.132:6789/0 conn(0x7ff45c0caaa0 legacy=0x7ff45c0ce880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:13.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.566+0000 7ff45959a700 1 -- 172.21.15.16:0/3529851768 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff45c06eaf0 con 0x7ff45c1599c0 2023-05-20T01:28:13.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.566+0000 7ff45959a700 1 -- 172.21.15.16:0/3529851768 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7ff450001800 con 0x7ff45c1599c0 2023-05-20T01:28:13.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.566+0000 7ff45959a700 1 -- 172.21.15.16:0/3529851768 >> v1:172.21.15.16:6789/0 conn(0x7ff45c1599c0 legacy=0x7ff45c0caf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:13.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.566+0000 7ff45959a700 1 --2- 172.21.15.16:0/3529851768 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff444003a70 0x7ff444003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:13.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.566+0000 7ff45959a700 1 --2- 172.21.15.16:0/3529851768 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff444004850 0x7ff444004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:13.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.566+0000 7ff45959a700 1 --2- 172.21.15.16:0/3529851768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff444005200 0x7ff44400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:13.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.566+0000 7ff45959a700 1 -- 172.21.15.16:0/3529851768 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff45c0cdf20 con 0x7ff444005200 2023-05-20T01:28:13.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.566+0000 7ff45959a700 1 -- 172.21.15.16:0/3529851768 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff44400c0e0 con 0x7ff444004850 2023-05-20T01:28:13.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.566+0000 7ff45959a700 1 -- 172.21.15.16:0/3529851768 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff44400c2a0 con 0x7ff444003a70 2023-05-20T01:28:13.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.566+0000 7ff45ad9d700 1 --2- 172.21.15.16:0/3529851768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff444005200 0x7ff44400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:13.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.566+0000 7ff459d9b700 1 --2- 172.21.15.16:0/3529851768 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff444004850 0x7ff444004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:13.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.566+0000 7ff45a59c700 1 --2- 172.21.15.16:0/3529851768 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff444003a70 0x7ff444003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:13.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.567+0000 7ff45ad9d700 1 -- 172.21.15.16:0/3529851768 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff444003a70 msgr2=0x7ff444003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:13.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.567+0000 7ff45ad9d700 1 --2- 172.21.15.16:0/3529851768 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff444003a70 0x7ff444003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:13.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.567+0000 7ff45ad9d700 1 -- 172.21.15.16:0/3529851768 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff444004850 msgr2=0x7ff444004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:13.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.567+0000 7ff45ad9d700 1 --2- 172.21.15.16:0/3529851768 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff444004850 0x7ff444004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:13.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.567+0000 7ff45ad9d700 1 -- 172.21.15.16:0/3529851768 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4440035b0 con 0x7ff444005200 2023-05-20T01:28:13.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.567+0000 7ff45ad9d700 1 -- 172.21.15.16:0/3529851768 reap_dead start 2023-05-20T01:28:13.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.567+0000 7ff45ad9d700 1 --2- 172.21.15.16:0/3529851768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff444005200 0x7ff44400ba10 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7ff45400e3a0 tx=0x7ff444005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fb3595af092ce11 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:13.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.567+0000 7ff45959a700 1 -- 172.21.15.16:0/3529851768 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff454001db0 con 0x7ff444005200 2023-05-20T01:28:13.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.567+0000 7ff45959a700 1 -- 172.21.15.16:0/3529851768 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff454003a00 con 0x7ff444005200 2023-05-20T01:28:13.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.567+0000 7ff45959a700 1 -- 172.21.15.16:0/3529851768 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff454003db0 con 0x7ff444005200 2023-05-20T01:28:13.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.567+0000 7ff460ee4700 1 -- 172.21.15.16:0/3529851768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff444005200 msgr2=0x7ff44400ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:13.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.567+0000 7ff460ee4700 1 --2- 172.21.15.16:0/3529851768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff444005200 0x7ff44400ba10 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7ff45400e3a0 tx=0x7ff444005880 comp rx=0 tx=0).stop 2023-05-20T01:28:13.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.567+0000 7ff460ee4700 1 -- 172.21.15.16:0/3529851768 shutdown_connections 2023-05-20T01:28:13.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.567+0000 7ff460ee4700 1 --2- 172.21.15.16:0/3529851768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff444005200 0x7ff44400ba10 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:13.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.567+0000 7ff460ee4700 1 -- 172.21.15.16:0/3529851768 >> 172.21.15.16:0/3529851768 conn(0x7ff45c0d0c30 msgr2=0x7ff45c0b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:13.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.568+0000 7ff460ee4700 1 -- 172.21.15.16:0/3529851768 shutdown_connections 2023-05-20T01:28:13.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.568+0000 7ff460ee4700 1 -- 172.21.15.16:0/3529851768 wait complete. 2023-05-20T01:28:13.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.569+0000 7ff460ee4700 1 Processor -- start 2023-05-20T01:28:13.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.569+0000 7ff460ee4700 1 -- start start 2023-05-20T01:28:13.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.570+0000 7ff460ee4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff444005200 0x7ff45c0c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:13.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.570+0000 7ff460ee4700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff45c1e49a0 0x7ff45c158180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:13.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.570+0000 7ff460ee4700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff45c1e3120 0x7ff45c15bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:13.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.570+0000 7ff460ee4700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff45c1e4580 con 0x7ff444005200 2023-05-20T01:28:13.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.570+0000 7ff460ee4700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff45c1586c0 con 0x7ff45c1e49a0 2023-05-20T01:28:13.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.570+0000 7ff460ee4700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff45c1e4e10 con 0x7ff45c1e3120 2023-05-20T01:28:13.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.570+0000 7ff45a59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff444005200 0x7ff45c0c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:13.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.570+0000 7ff45a59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff444005200 0x7ff45c0c9bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39686/0 (socket says 172.21.15.16:39686) 2023-05-20T01:28:13.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.570+0000 7ff45a59c700 1 -- 172.21.15.16:0/2563089843 learned_addr learned my addr 172.21.15.16:0/2563089843 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:13.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.570+0000 7ff45ad9d700 1 --2- 172.21.15.16:0/2563089843 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff45c1e3120 0x7ff45c15bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:13.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.570+0000 7ff45a59c700 1 -- 172.21.15.16:0/2563089843 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff45c1e3120 msgr2=0x7ff45c15bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:13.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.570+0000 7ff45a59c700 1 --2- 172.21.15.16:0/2563089843 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff45c1e3120 0x7ff45c15bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:13.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.570+0000 7ff45a59c700 1 -- 172.21.15.16:0/2563089843 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff45c1e49a0 msgr2=0x7ff45c158180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:13.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.570+0000 7ff45a59c700 1 --2- 172.21.15.16:0/2563089843 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff45c1e49a0 0x7ff45c158180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:13.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.570+0000 7ff45a59c700 1 -- 172.21.15.16:0/2563089843 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff45c06f0b0 con 0x7ff444005200 2023-05-20T01:28:13.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.570+0000 7ff45a59c700 1 --2- 172.21.15.16:0/2563089843 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff444005200 0x7ff45c0c9bb0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7ff450007370 tx=0x7ff450007680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:13.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.570+0000 7ff44affd700 1 -- 172.21.15.16:0/2563089843 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4500077a0 con 0x7ff444005200 2023-05-20T01:28:13.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.570+0000 7ff44affd700 1 -- 172.21.15.16:0/2563089843 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff45000f870 con 0x7ff444005200 2023-05-20T01:28:13.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.570+0000 7ff460ee4700 1 -- 172.21.15.16:0/2563089843 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff45c06edd0 con 0x7ff444005200 2023-05-20T01:28:13.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.570+0000 7ff44affd700 1 -- 172.21.15.16:0/2563089843 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff45000fc20 con 0x7ff444005200 2023-05-20T01:28:13.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.571+0000 7ff460ee4700 1 -- 172.21.15.16:0/2563089843 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff45c06e750 con 0x7ff444005200 2023-05-20T01:28:13.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.571+0000 7ff460ee4700 1 -- 172.21.15.16:0/2563089843 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff45c06f3f0 con 0x7ff444005200 2023-05-20T01:28:13.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.572+0000 7ff44affd700 1 -- 172.21.15.16:0/2563089843 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff45000fd80 con 0x7ff444005200 2023-05-20T01:28:13.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.572+0000 7ff44affd700 1 --2- 172.21.15.16:0/2563089843 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff43c06ace0 0x7ff43c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:13.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.572+0000 7ff44affd700 1 -- 172.21.15.16:0/2563089843 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 7073+0+0 (secure 0 0 0) 0x7ff450089030 con 0x7ff444005200 2023-05-20T01:28:13.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.572+0000 7ff459d9b700 1 --2- 172.21.15.16:0/2563089843 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff43c06ace0 0x7ff43c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:13.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.573+0000 7ff459d9b700 1 --2- 172.21.15.16:0/2563089843 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff43c06ace0 0x7ff43c06d190 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7ff45c0cd5a0 tx=0x7ff44c0093c0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:13.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.577+0000 7ff44affd700 1 -- 172.21.15.16:0/2563089843 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7ff450053d70 con 0x7ff444005200 2023-05-20T01:28:13.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:13.825+0000 7ff460ee4700 1 -- 172.21.15.16:0/2563089843 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7ff45c159d90 con 0x7ff444005200 2023-05-20T01:28:14.558 INFO:tasks.ceph.mon.a.smithi016.stderr:2023-05-20T01:28:14.554+0000 7f49eb8ae700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-05-20T01:28:14.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:14.558+0000 7ff44affd700 1 -- 172.21.15.16:0/2563089843 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v92) v1 ==== 161+0+0 (secure 0 0 0) 0x7ff45c159d90 con 0x7ff444005200 2023-05-20T01:28:14.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:14.561+0000 7ff460ee4700 1 -- 172.21.15.16:0/2563089843 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff43c06ace0 msgr2=0x7ff43c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:14.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:14.561+0000 7ff460ee4700 1 --2- 172.21.15.16:0/2563089843 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff43c06ace0 0x7ff43c06d190 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7ff45c0cd5a0 tx=0x7ff44c0093c0 comp rx=0 tx=0).stop 2023-05-20T01:28:14.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:14.561+0000 7ff460ee4700 1 -- 172.21.15.16:0/2563089843 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff444005200 msgr2=0x7ff45c0c9bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:14.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:14.561+0000 7ff460ee4700 1 --2- 172.21.15.16:0/2563089843 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff444005200 0x7ff45c0c9bb0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7ff450007370 tx=0x7ff450007680 comp rx=0 tx=0).stop 2023-05-20T01:28:14.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:14.562+0000 7ff460ee4700 1 -- 172.21.15.16:0/2563089843 shutdown_connections 2023-05-20T01:28:14.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:14.562+0000 7ff460ee4700 1 --2- 172.21.15.16:0/2563089843 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff43c06ace0 0x7ff43c06d190 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:14.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:14.562+0000 7ff460ee4700 1 --2- 172.21.15.16:0/2563089843 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff45c1e3120 0x7ff45c15bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:14.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:14.562+0000 7ff460ee4700 1 --2- 172.21.15.16:0/2563089843 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff444005200 0x7ff45c0c9bb0 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:14.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:14.562+0000 7ff460ee4700 1 --2- 172.21.15.16:0/2563089843 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff45c1e49a0 0x7ff45c158180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:14.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:14.562+0000 7ff460ee4700 1 -- 172.21.15.16:0/2563089843 >> 172.21.15.16:0/2563089843 conn(0x7ff45c0d0c30 msgr2=0x7ff45c0cef00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:14.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:14.562+0000 7ff460ee4700 1 -- 172.21.15.16:0/2563089843 shutdown_connections 2023-05-20T01:28:14.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:14.562+0000 7ff460ee4700 1 -- 172.21.15.16:0/2563089843 wait complete. 2023-05-20T01:28:14.569 INFO:tasks.ceph.mds.c.smithi016.stderr: -9221> 2023-05-20T01:27:22.444+0000 7efe54384700 -1 mds.0.openfiles _load_finish got (2) No such file or directory 2023-05-20T01:28:14.569 INFO:teuthology.orchestra.run.smithi016.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-05-20T01:28:14.569 INFO:tasks.ceph.mds.c.smithi016.stderr: -7750> 2023-05-20T01:27:27.442+0000 7efe57b8b700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-05-20T01:28:14.571 INFO:tasks.ceph.mds.c.smithi016.stderr: -5186> 2023-05-20T01:27:28.011+0000 7efe54384700 -1 log_channel(cluster) log [ERR] : bad backtrace on directory inode 0x10000000000 2023-05-20T01:28:14.572 INFO:tasks.ceph.mds.c.smithi016.stderr: -5151> 2023-05-20T01:27:28.011+0000 7efe54384700 -1 log_channel(cluster) log [ERR] : bad backtrace on directory inode 0x4 2023-05-20T01:28:14.572 INFO:tasks.ceph.mds.c.smithi016.stderr: -5014> 2023-05-20T01:27:28.012+0000 7efe54384700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /stripey/ auth v1 pv4 ap=3 DIRTYPARENT f(v0 1=1+0) n() (inest lock dirty) (ifile lock->sync w=1 flushing) (iversion lock) | dirtyscattered=2 lock=1 dirfrag=1 dirtyparent=1 waiter=1 authpin=1 scrubqueue=0 0x564d4a432580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(15)0x10000000000:[<0x1/stripey v1>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 1=1+0)","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 1=1+0)","memory_value.rstat":"n()","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-61} 2023-05-20T01:28:14.572 INFO:tasks.ceph.mds.c.smithi016.stderr: -4905> 2023-05-20T01:27:28.013+0000 7efe54384700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x4 [...2,head] /lost+found/ auth v1 pv6 ap=3 DIRTYPARENT f(v0 1=1+0) n() (inest lock dirty) (ifile lock->sync w=1 flushing) (iversion lock) | dirtyscattered=2 lock=1 dirfrag=1 dirtyparent=1 waiter=1 authpin=1 scrubqueue=0 0x564d4a432b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(15)0x4:[<0x1/lost+found v1>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 1=1+0)","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 1=1+0)","memory_value.rstat":"n()","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-61} 2023-05-20T01:28:14.573 INFO:tasks.ceph.mds.c.smithi016.stderr: -4716> 2023-05-20T01:27:28.013+0000 7efe54384700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001fb [2,head] /stripey/flushed_file_6 auth v1 ap=1 DIRTYPARENT s=1572864 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x564d4a485180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(17)0x100000001fb:[<0x10000000000/flushed_file_6 v54>,<0x1/stripey v59>]//[16]","memoryvalue":"(17)0x100000001fb:[<0x10000000000/flushed_file_6 v1>,<0x1/stripey v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-05-20T01:28:14.573 INFO:tasks.ceph.mds.c.smithi016.stderr: -4692> 2023-05-20T01:27:28.013+0000 7efe54384700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001f9 [2,head] /stripey/flushed_file_4 auth v1 ap=1 DIRTYPARENT s=1441792 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x564d4a4b3080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(17)0x100000001f9:[<0x10000000000/flushed_file_4 v38>,<0x1/stripey v59>]//[16]","memoryvalue":"(17)0x100000001f9:[<0x10000000000/flushed_file_4 v1>,<0x1/stripey v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-05-20T01:28:14.574 INFO:tasks.ceph.mds.c.smithi016.stderr: -4669> 2023-05-20T01:27:28.014+0000 7efe54384700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001f8 [2,head] /stripey/flushed_file_3 auth v1 ap=1 DIRTYPARENT s=917503 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x564d4a4b3600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(17)0x100000001f8:[<0x10000000000/flushed_file_3 v24>,<0x1/stripey v59>]//[16]","memoryvalue":"(17)0x100000001f8:[<0x10000000000/flushed_file_3 v1>,<0x1/stripey v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-05-20T01:28:14.574 INFO:tasks.ceph.mds.c.smithi016.stderr: -4647> 2023-05-20T01:27:28.014+0000 7efe54384700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001f7 [2,head] /stripey/flushed_file_2 auth v1 ap=1 DIRTYPARENT s=917504 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x564d4a4b3b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(17)0x100000001f7:[<0x10000000000/flushed_file_2 v28>,<0x1/stripey v59>]//[16]","memoryvalue":"(17)0x100000001f7:[<0x10000000000/flushed_file_2 v1>,<0x1/stripey v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-05-20T01:28:14.574 INFO:tasks.ceph.mds.c.smithi016.stderr: -4622> 2023-05-20T01:27:28.014+0000 7efe54384700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001f6 [2,head] /stripey/flushed_file_1 auth v1 ap=1 DIRTYPARENT s=524288 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x564d4a4b4100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(17)0x100000001f6:[<0x10000000000/flushed_file_1 v26>,<0x1/stripey v59>]//[16]","memoryvalue":"(17)0x100000001f6:[<0x10000000000/flushed_file_1 v1>,<0x1/stripey v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-05-20T01:28:14.574 INFO:tasks.ceph.mds.c.smithi016.stderr: -4606> 2023-05-20T01:27:28.014+0000 7efe54384700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001fa [2,head] /stripey/flushed_file_5 auth v1 ap=1 DIRTYPARENT s=1441793 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x564d4a485700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(17)0x100000001fa:[<0x10000000000/flushed_file_5 v46>,<0x1/stripey v59>]//[16]","memoryvalue":"(17)0x100000001fa:[<0x10000000000/flushed_file_5 v1>,<0x1/stripey v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-05-20T01:28:14.574 INFO:tasks.ceph.mds.c.smithi016.stderr: -4587> 2023-05-20T01:27:28.014+0000 7efe54384700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /stripey/flushed_file_0 auth v1 ap=1 DIRTYPARENT s=1048576 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x564d4a4b4680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(17)0x10000000001:[<0x10000000000/flushed_file_0 v30>,<0x1/stripey v59>]//[16]","memoryvalue":"(17)0x10000000001:[<0x10000000000/flushed_file_0 v1>,<0x1/stripey v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-05-20T01:28:14.574 INFO:tasks.ceph.mds.c.smithi016.stderr: -4572> 2023-05-20T01:27:28.014+0000 7efe54384700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001fc [2,head] /lost+found/100000001fc auth v1 ap=1 DIRTYPARENT s=524288 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x564d4a4b4c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(17)0x100000001fc:[<0x4/100000001fc v1>,<0x1/lost+found v1>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":-61} 2023-05-20T01:28:14.574 INFO:tasks.ceph.mds.c.smithi016.stderr: -4055> 2023-05-20T01:27:28.023+0000 7efe54384700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1 [...2,head] / auth v7 ap=5 snaprealm=0x564d4a3b4d80 DIRTYPARENT f(v1 2=0+2) n(v0 rc2023-05-20T01:27:27.660582+0000 2=0+2)/n() (idft sync r=1) (inest sync r=1) (ifile sync->mix r=1) (iversion lock) caps={6884=pLsXs/-@45} | dirtyscattered=0 request=0 lock=3 dirfrag=1 caps=1 openingsnapparents=0 dirtyparent=1 dirty=1 waiter=1 authpin=1 scrubqueue=0 0x564d4a434100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(15)0x1:[]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 2=0+2)","ondisk_value.rstat":"n(v0 3=0+3)","memory_value.dirstat":"f(v1 2=0+2)","memory_value.rstat":"n(v0 rc2023-05-20T01:27:27.660582+0000 2=0+2)","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-61} 2023-05-20T01:28:14.584 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-05-20T01:28:14.605 INFO:tasks.ceph.mds.c.smithi016.stdout:starting mds.c at 2023-05-20T01:28:14.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:14.715+0000 7f4b289e8700 1 Processor -- start 2023-05-20T01:28:14.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:14.715+0000 7f4b289e8700 1 -- start start 2023-05-20T01:28:14.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:14.716+0000 7f4b289e8700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b2415e150 0x7f4b24159e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:14.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:14.716+0000 7f4b289e8700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b2415d5b0 con 0x7f4b240cb700 2023-05-20T01:28:14.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:14.716+0000 7f4b289e8700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b24157820 con 0x7f4b24157450 2023-05-20T01:28:14.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:14.716+0000 7f4b289e8700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b2415a4c0 con 0x7f4b2415e150 2023-05-20T01:28:14.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:14.716+0000 7f4b21d9b700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f4b240cb700 0x7f4b2415da40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51236/0 (socket says 172.21.15.16:51236) 2023-05-20T01:28:14.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:14.716+0000 7f4b21d9b700 1 -- 172.21.15.16:0/3696421117 learned_addr learned my addr 172.21.15.16:0/3696421117 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:28:14.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:14.716+0000 7f4b22d9d700 1 --2- 172.21.15.16:0/3696421117 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b2415e150 0x7f4b24159e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:14.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:14.716+0000 7f4b2159a700 1 -- 172.21.15.16:0/3696421117 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1828802734 0 0) 0x7f4b2415d5b0 con 0x7f4b240cb700 2023-05-20T01:28:14.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:14.716+0000 7f4b2159a700 1 -- 172.21.15.16:0/3696421117 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b0c0057c0 con 0x7f4b240cb700 2023-05-20T01:28:14.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:14.717+0000 7f4b2159a700 1 -- 172.21.15.16:0/3696421117 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3525388875 0 0) 0x7f4b0c0057c0 con 0x7f4b240cb700 2023-05-20T01:28:14.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:14.717+0000 7f4b2159a700 1 -- 172.21.15.16:0/3696421117 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b2415e150 msgr2=0x7f4b24159e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:14.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:14.717+0000 7f4b2159a700 1 --2- 172.21.15.16:0/3696421117 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b2415e150 0x7f4b24159e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:14.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:14.717+0000 7f4b2159a700 1 -- 172.21.15.16:0/3696421117 >> v1:172.21.15.132:6789/0 conn(0x7f4b24157450 legacy=0x7f4b240caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:14.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:14.717+0000 7f4b2159a700 1 -- 172.21.15.16:0/3696421117 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b2415a7a0 con 0x7f4b240cb700 2023-05-20T01:28:14.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:14.717+0000 7f4b2159a700 1 -- 172.21.15.16:0/3696421117 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7f4b14001670 con 0x7f4b240cb700 2023-05-20T01:28:14.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:14.717+0000 7f4b2159a700 1 -- 172.21.15.16:0/3696421117 >> v1:172.21.15.16:6789/0 conn(0x7f4b240cb700 legacy=0x7f4b2415da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:14.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:14.717+0000 7f4b2159a700 1 --2- 172.21.15.16:0/3696421117 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b0c005c40 0x7f4b0c006050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:14.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:14.717+0000 7f4b2159a700 1 --2- 172.21.15.16:0/3696421117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b0c006a60 0x7f4b0c006ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:14.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:14.717+0000 7f4b2159a700 1 --2- 172.21.15.16:0/3696421117 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b2415e150 0x7f4b0c007410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:14.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:14.717+0000 7f4b2159a700 1 -- 172.21.15.16:0/3696421117 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b2415a4c0 con 0x7f4b0c006a60 2023-05-20T01:28:14.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:14.717+0000 7f4b2159a700 1 -- 172.21.15.16:0/3696421117 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b0c00dcd0 con 0x7f4b0c005c40 2023-05-20T01:28:14.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:14.717+0000 7f4b2159a700 1 -- 172.21.15.16:0/3696421117 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b0c00de60 con 0x7f4b2415e150 2023-05-20T01:28:14.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:14.717+0000 7f4b21d9b700 1 --2- 172.21.15.16:0/3696421117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b0c006a60 0x7f4b0c006ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:14.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:14.717+0000 7f4b2259c700 1 --2- 172.21.15.16:0/3696421117 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b0c005c40 0x7f4b0c006050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:14.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:14.717+0000 7f4b22d9d700 1 --2- 172.21.15.16:0/3696421117 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b2415e150 0x7f4b0c007410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:14.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:14.717+0000 7f4b21d9b700 1 -- 172.21.15.16:0/3696421117 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b2415e150 msgr2=0x7f4b0c007410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:14.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:14.717+0000 7f4b21d9b700 1 --2- 172.21.15.16:0/3696421117 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b2415e150 0x7f4b0c007410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:14.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:14.717+0000 7f4b21d9b700 1 -- 172.21.15.16:0/3696421117 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b0c005c40 msgr2=0x7f4b0c006050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:14.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:14.717+0000 7f4b21d9b700 1 --2- 172.21.15.16:0/3696421117 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b0c005c40 0x7f4b0c006050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:14.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:14.717+0000 7f4b21d9b700 1 -- 172.21.15.16:0/3696421117 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b0c0057c0 con 0x7f4b0c006a60 2023-05-20T01:28:14.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:14.717+0000 7f4b21d9b700 1 --2- 172.21.15.16:0/3696421117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b0c006a60 0x7f4b0c006ed0 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f4b1400c240 tx=0x7f4b0c007a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52a5956f9f569c1a server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:14.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:14.717+0000 7f4b2159a700 1 -- 172.21.15.16:0/3696421117 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b14007c10 con 0x7f4b0c006a60 2023-05-20T01:28:14.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:14.718+0000 7f4b2159a700 1 -- 172.21.15.16:0/3696421117 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b1400cde0 con 0x7f4b0c006a60 2023-05-20T01:28:14.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:14.718+0000 7f4b2159a700 1 -- 172.21.15.16:0/3696421117 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b14005ba0 con 0x7f4b0c006a60 2023-05-20T01:28:14.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:14.718+0000 7f4b289e8700 1 -- 172.21.15.16:0/3696421117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b0c006a60 msgr2=0x7f4b0c006ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:14.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:14.718+0000 7f4b289e8700 1 --2- 172.21.15.16:0/3696421117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b0c006a60 0x7f4b0c006ed0 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f4b1400c240 tx=0x7f4b0c007a90 comp rx=0 tx=0).stop 2023-05-20T01:28:14.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:14.718+0000 7f4b22d9d700 1 -- 172.21.15.16:0/3696421117 reap_dead start 2023-05-20T01:28:14.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:14.718+0000 7f4b289e8700 1 -- 172.21.15.16:0/3696421117 shutdown_connections 2023-05-20T01:28:14.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:14.718+0000 7f4b289e8700 1 -- 172.21.15.16:0/3696421117 >> 172.21.15.16:0/3696421117 conn(0x7f4b2406cbb0 msgr2=0x7f4b2406cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:14.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:14.718+0000 7f4b289e8700 1 -- 172.21.15.16:0/3696421117 shutdown_connections 2023-05-20T01:28:14.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:14.718+0000 7f4b289e8700 1 -- 172.21.15.16:0/3696421117 wait complete. 2023-05-20T01:28:14.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:14.720+0000 7f4b289e8700 1 Processor -- start 2023-05-20T01:28:14.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:14.720+0000 7f4b289e8700 1 -- start start 2023-05-20T01:28:14.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:14.720+0000 7f4b289e8700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b0c006a60 0x7f4b240ce260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:14.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:14.721+0000 7f4b289e8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b24152b40 0x7f4b24154270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:14.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:14.721+0000 7f4b289e8700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b240cc4a0 0x7f4b2414adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:14.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:14.721+0000 7f4b289e8700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b24152fb0 con 0x7f4b24152b40 2023-05-20T01:28:14.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:14.721+0000 7f4b289e8700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b24153de0 con 0x7f4b0c006a60 2023-05-20T01:28:14.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:14.721+0000 7f4b289e8700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b241531d0 con 0x7f4b240cc4a0 2023-05-20T01:28:14.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:14.721+0000 7f4b21d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b24152b40 0x7f4b24154270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:14.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:14.721+0000 7f4b21d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b24152b40 0x7f4b24154270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39720/0 (socket says 172.21.15.16:39720) 2023-05-20T01:28:14.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:14.721+0000 7f4b21d9b700 1 -- 172.21.15.16:0/122166753 learned_addr learned my addr 172.21.15.16:0/122166753 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:14.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:14.721+0000 7f4b22d9d700 1 --2- 172.21.15.16:0/122166753 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b240cc4a0 0x7f4b2414adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:14.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:14.721+0000 7f4b2259c700 1 --2- 172.21.15.16:0/122166753 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b0c006a60 0x7f4b240ce260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:14.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:14.721+0000 7f4b21d9b700 1 -- 172.21.15.16:0/122166753 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b240cc4a0 msgr2=0x7f4b2414adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:14.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:14.721+0000 7f4b21d9b700 1 --2- 172.21.15.16:0/122166753 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b240cc4a0 0x7f4b2414adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:14.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:14.721+0000 7f4b21d9b700 1 -- 172.21.15.16:0/122166753 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b0c006a60 msgr2=0x7f4b240ce260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:14.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:14.721+0000 7f4b21d9b700 1 --2- 172.21.15.16:0/122166753 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b0c006a60 0x7f4b240ce260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:14.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:14.721+0000 7f4b21d9b700 1 -- 172.21.15.16:0/122166753 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b2415ad60 con 0x7f4b24152b40 2023-05-20T01:28:14.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:14.721+0000 7f4b21d9b700 1 --2- 172.21.15.16:0/122166753 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b24152b40 0x7f4b24154270 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f4b1400c830 tx=0x7f4b14001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:14.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:14.721+0000 7f4b12ffd700 1 -- 172.21.15.16:0/122166753 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b14001640 con 0x7f4b24152b40 2023-05-20T01:28:14.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:14.721+0000 7f4b12ffd700 1 -- 172.21.15.16:0/122166753 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b140070d0 con 0x7f4b24152b40 2023-05-20T01:28:14.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:14.721+0000 7f4b289e8700 1 -- 172.21.15.16:0/122166753 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b2415aa80 con 0x7f4b24152b40 2023-05-20T01:28:14.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:14.721+0000 7f4b12ffd700 1 -- 172.21.15.16:0/122166753 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b140074a0 con 0x7f4b24152b40 2023-05-20T01:28:14.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:14.721+0000 7f4b289e8700 1 -- 172.21.15.16:0/122166753 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b24073b00 con 0x7f4b24152b40 2023-05-20T01:28:14.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:14.722+0000 7f4b289e8700 1 -- 172.21.15.16:0/122166753 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b240747b0 con 0x7f4b24152b40 2023-05-20T01:28:14.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:14.723+0000 7f4b12ffd700 1 -- 172.21.15.16:0/122166753 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4b14019030 con 0x7f4b24152b40 2023-05-20T01:28:14.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:14.723+0000 7f4b12ffd700 1 --2- 172.21.15.16:0/122166753 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f4b0406ace0 0x7f4b0406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:14.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:14.723+0000 7f4b12ffd700 1 -- 172.21.15.16:0/122166753 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 7159+0+0 (secure 0 0 0) 0x7f4b1408a6f0 con 0x7f4b24152b40 2023-05-20T01:28:14.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:14.723+0000 7f4b2259c700 1 --2- 172.21.15.16:0/122166753 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f4b0406ace0 0x7f4b0406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:14.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:14.724+0000 7f4b2259c700 1 --2- 172.21.15.16:0/122166753 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f4b0406ace0 0x7f4b0406d190 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f4b18007fd0 tx=0x7f4b18009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:14.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:14.728+0000 7f4b12ffd700 1 -- 172.21.15.16:0/122166753 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f4b14055360 con 0x7f4b24152b40 2023-05-20T01:28:14.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:14.975+0000 7f4b289e8700 1 -- 172.21.15.16:0/122166753 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f4b240c5c50 con 0x7f4b24152b40 2023-05-20T01:28:15.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:15.568+0000 7f4b12ffd700 1 -- 172.21.15.16:0/122166753 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v93) v1 ==== 104+0+0 (secure 0 0 0) 0x7f4b14058980 con 0x7f4b24152b40 2023-05-20T01:28:15.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:15.571+0000 7f4b289e8700 1 -- 172.21.15.16:0/122166753 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f4b0406ace0 msgr2=0x7f4b0406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:15.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:15.571+0000 7f4b289e8700 1 --2- 172.21.15.16:0/122166753 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f4b0406ace0 0x7f4b0406d190 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f4b18007fd0 tx=0x7f4b18009040 comp rx=0 tx=0).stop 2023-05-20T01:28:15.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:15.571+0000 7f4b289e8700 1 -- 172.21.15.16:0/122166753 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b24152b40 msgr2=0x7f4b24154270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:15.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:15.571+0000 7f4b289e8700 1 --2- 172.21.15.16:0/122166753 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b24152b40 0x7f4b24154270 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f4b1400c830 tx=0x7f4b14001a00 comp rx=0 tx=0).stop 2023-05-20T01:28:15.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:15.571+0000 7f4b289e8700 1 -- 172.21.15.16:0/122166753 shutdown_connections 2023-05-20T01:28:15.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:15.571+0000 7f4b289e8700 1 --2- 172.21.15.16:0/122166753 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f4b0406ace0 0x7f4b0406d190 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:15.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:15.571+0000 7f4b289e8700 1 --2- 172.21.15.16:0/122166753 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b240cc4a0 0x7f4b2414adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:15.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:15.571+0000 7f4b289e8700 1 --2- 172.21.15.16:0/122166753 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b0c006a60 0x7f4b240ce260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:15.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:15.571+0000 7f4b289e8700 1 --2- 172.21.15.16:0/122166753 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b24152b40 0x7f4b24154270 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:15.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:15.571+0000 7f4b289e8700 1 -- 172.21.15.16:0/122166753 >> 172.21.15.16:0/122166753 conn(0x7f4b2406cbb0 msgr2=0x7f4b240c5f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:15.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:15.571+0000 7f4b289e8700 1 -- 172.21.15.16:0/122166753 shutdown_connections 2023-05-20T01:28:15.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:15.571+0000 7f4b289e8700 1 -- 172.21.15.16:0/122166753 wait complete. 2023-05-20T01:28:15.589 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-05-20T01:28:15.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:15.756+0000 7fb4e88fb700 1 Processor -- start 2023-05-20T01:28:15.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:15.756+0000 7fb4e88fb700 1 -- start start 2023-05-20T01:28:15.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:15.757+0000 7fb4e88fb700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb4e415ca30 0x7fb4e415db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:15.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:15.757+0000 7fb4e88fb700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb4e406cbf0 0x7fb4e406d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:15.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:15.757+0000 7fb4e88fb700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4e406d590 con 0x7fb4e40cddd0 2023-05-20T01:28:15.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:15.757+0000 7fb4e88fb700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4e415e090 con 0x7fb4e406cbf0 2023-05-20T01:28:15.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:15.757+0000 7fb4e88fb700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e40cd3c0 con 0x7fb4e415ca30 2023-05-20T01:28:15.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:15.757+0000 7fb4e2d9d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb4e406cbf0 0x7fb4e406d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:15.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:15.757+0000 7fb4e2d9d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb4e406cbf0 0x7fb4e406d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39732/0 (socket says 172.21.15.16:39732) 2023-05-20T01:28:15.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:15.757+0000 7fb4e2d9d700 1 -- 172.21.15.16:0/2967283831 learned_addr learned my addr 172.21.15.16:0/2967283831 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:15.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:15.757+0000 7fb4e259c700 1 --2- 172.21.15.16:0/2967283831 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb4e415ca30 0x7fb4e415db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:15.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:15.758+0000 7fb4e2d9d700 1 -- 172.21.15.16:0/2967283831 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb4e415ca30 msgr2=0x7fb4e415db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:15.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:15.758+0000 7fb4e2d9d700 1 --2- 172.21.15.16:0/2967283831 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb4e415ca30 0x7fb4e415db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:15.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:15.758+0000 7fb4e2d9d700 1 -- 172.21.15.16:0/2967283831 >> v1:172.21.15.132:6789/0 conn(0x7fb4e40cddd0 legacy=0x7fb4e4070d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:15.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:15.758+0000 7fb4e2d9d700 1 -- 172.21.15.16:0/2967283831 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4e41606c0 con 0x7fb4e406cbf0 2023-05-20T01:28:15.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:15.758+0000 7fb4e2d9d700 1 --2- 172.21.15.16:0/2967283831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4e406cbf0 0x7fb4e406d050 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc009e50 tx=0x7fb4dc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e188e8b9c0adeb99 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:15.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:15.758+0000 7fb4e159a700 1 -- 172.21.15.16:0/2967283831 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4dc00e070 con 0x7fb4e406cbf0 2023-05-20T01:28:15.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:15.758+0000 7fb4e159a700 1 -- 172.21.15.16:0/2967283831 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4dc00b9c0 con 0x7fb4e406cbf0 2023-05-20T01:28:15.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:15.758+0000 7fb4e159a700 1 -- 172.21.15.16:0/2967283831 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4dc00bd70 con 0x7fb4e406cbf0 2023-05-20T01:28:15.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:15.758+0000 7fb4e88fb700 1 -- 172.21.15.16:0/2967283831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4e406cbf0 msgr2=0x7fb4e406d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:15.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:15.758+0000 7fb4e88fb700 1 --2- 172.21.15.16:0/2967283831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4e406cbf0 0x7fb4e406d050 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc009e50 tx=0x7fb4dc00b330 comp rx=0 tx=0).stop 2023-05-20T01:28:15.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:15.758+0000 7fb4e88fb700 1 -- 172.21.15.16:0/2967283831 shutdown_connections 2023-05-20T01:28:15.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:15.759+0000 7fb4e88fb700 1 --2- 172.21.15.16:0/2967283831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4e406cbf0 0x7fb4e406d050 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:15.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:15.759+0000 7fb4e88fb700 1 --2- 172.21.15.16:0/2967283831 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb4e415ca30 0x7fb4e415db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:15.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:15.759+0000 7fb4e88fb700 1 -- 172.21.15.16:0/2967283831 >> 172.21.15.16:0/2967283831 conn(0x7fb4e40661b0 msgr2=0x7fb4e40665b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:15.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:15.759+0000 7fb4e88fb700 1 -- 172.21.15.16:0/2967283831 shutdown_connections 2023-05-20T01:28:15.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:15.759+0000 7fb4e88fb700 1 -- 172.21.15.16:0/2967283831 wait complete. 2023-05-20T01:28:15.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:15.760+0000 7fb4e88fb700 1 Processor -- start 2023-05-20T01:28:15.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:15.760+0000 7fb4e88fb700 1 -- start start 2023-05-20T01:28:15.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:15.761+0000 7fb4e88fb700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb4e406cbf0 0x7fb4e415f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:15.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:15.761+0000 7fb4e88fb700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4e40cddd0 0x7fb4e4149cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:15.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:15.761+0000 7fb4e88fb700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb4e415ca30 0x7fb4e41f2630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:15.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:15.761+0000 7fb4e88fb700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e415fcf0 con 0x7fb4e40cddd0 2023-05-20T01:28:15.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:15.761+0000 7fb4e88fb700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e4146920 con 0x7fb4e415ca30 2023-05-20T01:28:15.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:15.761+0000 7fb4e88fb700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e4143b00 con 0x7fb4e406cbf0 2023-05-20T01:28:15.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:15.761+0000 7fb4e1d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4e40cddd0 0x7fb4e4149cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:15.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:15.761+0000 7fb4e1d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4e40cddd0 0x7fb4e4149cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39736/0 (socket says 172.21.15.16:39736) 2023-05-20T01:28:15.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:15.761+0000 7fb4e1d9b700 1 -- 172.21.15.16:0/1359863077 learned_addr learned my addr 172.21.15.16:0/1359863077 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:15.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:15.761+0000 7fb4e259c700 1 --2- 172.21.15.16:0/1359863077 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb4e406cbf0 0x7fb4e415f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:15.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:15.761+0000 7fb4e2d9d700 1 --2- 172.21.15.16:0/1359863077 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb4e415ca30 0x7fb4e41f2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:15.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:15.761+0000 7fb4e1d9b700 1 -- 172.21.15.16:0/1359863077 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb4e406cbf0 msgr2=0x7fb4e415f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:15.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:15.761+0000 7fb4e1d9b700 1 --2- 172.21.15.16:0/1359863077 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb4e406cbf0 0x7fb4e415f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:15.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:15.761+0000 7fb4e1d9b700 1 -- 172.21.15.16:0/1359863077 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb4e415ca30 msgr2=0x7fb4e41f2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:15.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:15.761+0000 7fb4e1d9b700 1 --2- 172.21.15.16:0/1359863077 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb4e415ca30 0x7fb4e41f2630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:15.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:15.761+0000 7fb4e1d9b700 1 -- 172.21.15.16:0/1359863077 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4dc009b30 con 0x7fb4e40cddd0 2023-05-20T01:28:15.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:15.761+0000 7fb4e1d9b700 1 --2- 172.21.15.16:0/1359863077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4e40cddd0 0x7fb4e4149cd0 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7fb4d400c4e0 tx=0x7fb4d400c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:15.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:15.761+0000 7fb4d2ffd700 1 -- 172.21.15.16:0/1359863077 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4d40052d0 con 0x7fb4e40cddd0 2023-05-20T01:28:15.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:15.761+0000 7fb4d2ffd700 1 -- 172.21.15.16:0/1359863077 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4d4005430 con 0x7fb4e40cddd0 2023-05-20T01:28:15.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:15.761+0000 7fb4e88fb700 1 -- 172.21.15.16:0/1359863077 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4e40cd610 con 0x7fb4e40cddd0 2023-05-20T01:28:15.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:15.761+0000 7fb4d2ffd700 1 -- 172.21.15.16:0/1359863077 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4d40038c0 con 0x7fb4e40cddd0 2023-05-20T01:28:15.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:15.762+0000 7fb4e88fb700 1 -- 172.21.15.16:0/1359863077 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4e40ce240 con 0x7fb4e40cddd0 2023-05-20T01:28:15.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:15.762+0000 7fb4e88fb700 1 -- 172.21.15.16:0/1359863077 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4e40cd8f0 con 0x7fb4e40cddd0 2023-05-20T01:28:15.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:15.763+0000 7fb4d2ffd700 1 -- 172.21.15.16:0/1359863077 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb4d4003a20 con 0x7fb4e40cddd0 2023-05-20T01:28:15.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:15.763+0000 7fb4d2ffd700 1 --2- 172.21.15.16:0/1359863077 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb4c406ace0 0x7fb4c406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:15.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:15.764+0000 7fb4e259c700 1 --2- 172.21.15.16:0/1359863077 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb4c406ace0 0x7fb4c406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:15.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:15.764+0000 7fb4d2ffd700 1 -- 172.21.15.16:0/1359863077 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 7159+0+0 (secure 0 0 0) 0x7fb4d4088c30 con 0x7fb4e40cddd0 2023-05-20T01:28:15.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:15.764+0000 7fb4e259c700 1 --2- 172.21.15.16:0/1359863077 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb4c406ace0 0x7fb4c406d190 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fb4e41585e0 tx=0x7fb4d8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:15.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:15.768+0000 7fb4d2ffd700 1 -- 172.21.15.16:0/1359863077 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fb4d4053890 con 0x7fb4e40cddd0 2023-05-20T01:28:15.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:15.966+0000 7fb4e88fb700 1 -- 172.21.15.16:0/1359863077 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb4e40cec60 con 0x7fb4e40cddd0 2023-05-20T01:28:16.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:16.628+0000 7fb4d2ffd700 1 -- 172.21.15.16:0/1359863077 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v125) v1 ==== 181+0+0 (secure 0 0 0) 0x7fb4d4056eb0 con 0x7fb4e40cddd0 2023-05-20T01:28:16.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:16.632+0000 7fb4e88fb700 1 -- 172.21.15.16:0/1359863077 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb4c406ace0 msgr2=0x7fb4c406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:16.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:16.632+0000 7fb4e88fb700 1 --2- 172.21.15.16:0/1359863077 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb4c406ace0 0x7fb4c406d190 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fb4e41585e0 tx=0x7fb4d8008040 comp rx=0 tx=0).stop 2023-05-20T01:28:16.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:16.632+0000 7fb4e88fb700 1 -- 172.21.15.16:0/1359863077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4e40cddd0 msgr2=0x7fb4e4149cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:16.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:16.632+0000 7fb4e88fb700 1 --2- 172.21.15.16:0/1359863077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4e40cddd0 0x7fb4e4149cd0 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7fb4d400c4e0 tx=0x7fb4d400c7f0 comp rx=0 tx=0).stop 2023-05-20T01:28:16.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:16.632+0000 7fb4e88fb700 1 -- 172.21.15.16:0/1359863077 shutdown_connections 2023-05-20T01:28:16.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:16.632+0000 7fb4e88fb700 1 --2- 172.21.15.16:0/1359863077 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb4e415ca30 0x7fb4e41f2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:16.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:16.632+0000 7fb4e88fb700 1 --2- 172.21.15.16:0/1359863077 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb4c406ace0 0x7fb4c406d190 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:16.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:16.632+0000 7fb4e88fb700 1 --2- 172.21.15.16:0/1359863077 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb4e406cbf0 0x7fb4e415f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:16.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:16.632+0000 7fb4e88fb700 1 --2- 172.21.15.16:0/1359863077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4e40cddd0 0x7fb4e4149cd0 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:16.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:16.632+0000 7fb4e88fb700 1 -- 172.21.15.16:0/1359863077 >> 172.21.15.16:0/1359863077 conn(0x7fb4e40661b0 msgr2=0x7fb4e40716d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:16.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:16.632+0000 7fb4e88fb700 1 -- 172.21.15.16:0/1359863077 shutdown_connections 2023-05-20T01:28:16.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:16.632+0000 7fb4e88fb700 1 -- 172.21.15.16:0/1359863077 wait complete. 2023-05-20T01:28:16.640 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' removed 2023-05-20T01:28:16.654 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-05-20T01:28:16.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:16.786+0000 7f1e0ba1a700 1 Processor -- start 2023-05-20T01:28:16.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:16.786+0000 7f1e0ba1a700 1 -- start start 2023-05-20T01:28:16.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:16.787+0000 7f1e0ba1a700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1e04159880 0x7f1e0415dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:16.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:16.787+0000 7f1e0ba1a700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1e04159c50 0x7f1e0415a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:16.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:16.787+0000 7f1e0ba1a700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e0415adc0 con 0x7f1e040cdc00 2023-05-20T01:28:16.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:16.787+0000 7f1e0ba1a700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e0415a600 con 0x7f1e04159c50 2023-05-20T01:28:16.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:16.787+0000 7f1e0ba1a700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0415a800 con 0x7f1e04159880 2023-05-20T01:28:16.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:16.787+0000 7f1e08fb5700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1e04159c50 0x7f1e0415a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:16.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:16.787+0000 7f1e08fb5700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1e04159c50 0x7f1e0415a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39746/0 (socket says 172.21.15.16:39746) 2023-05-20T01:28:16.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:16.787+0000 7f1e08fb5700 1 -- 172.21.15.16:0/1894314477 learned_addr learned my addr 172.21.15.16:0/1894314477 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:16.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:16.787+0000 7f1e097b6700 1 --2- 172.21.15.16:0/1894314477 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1e04159880 0x7f1e0415dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:16.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:16.787+0000 7f1e08fb5700 1 -- 172.21.15.16:0/1894314477 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1e04159880 msgr2=0x7f1e0415dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:16.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:16.787+0000 7f1e08fb5700 1 --2- 172.21.15.16:0/1894314477 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1e04159880 0x7f1e0415dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:16.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:16.787+0000 7f1e08fb5700 1 -- 172.21.15.16:0/1894314477 >> v1:172.21.15.132:6789/0 conn(0x7f1e040cdc00 legacy=0x7f1e0406cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:16.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:16.787+0000 7f1e08fb5700 1 -- 172.21.15.16:0/1894314477 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e0415aae0 con 0x7f1e04159c50 2023-05-20T01:28:16.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:16.787+0000 7f1e08fb5700 1 --2- 172.21.15.16:0/1894314477 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e04159c50 0x7f1e0415a0c0 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f1e0000bb90 tx=0x7f1e0000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8487ac58bc6b6e91 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:16.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:16.787+0000 7f1dfbfff700 1 -- 172.21.15.16:0/1894314477 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e0000dd10 con 0x7f1e04159c50 2023-05-20T01:28:16.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:16.788+0000 7f1dfbfff700 1 -- 172.21.15.16:0/1894314477 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e0000de70 con 0x7f1e04159c50 2023-05-20T01:28:16.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:16.788+0000 7f1dfbfff700 1 -- 172.21.15.16:0/1894314477 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e00004230 con 0x7f1e04159c50 2023-05-20T01:28:16.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:16.788+0000 7f1e0ba1a700 1 -- 172.21.15.16:0/1894314477 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e04159c50 msgr2=0x7f1e0415a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:16.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:16.788+0000 7f1e0ba1a700 1 --2- 172.21.15.16:0/1894314477 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e04159c50 0x7f1e0415a0c0 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f1e0000bb90 tx=0x7f1e0000bea0 comp rx=0 tx=0).stop 2023-05-20T01:28:16.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:16.788+0000 7f1e0ba1a700 1 -- 172.21.15.16:0/1894314477 shutdown_connections 2023-05-20T01:28:16.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:16.788+0000 7f1e0ba1a700 1 --2- 172.21.15.16:0/1894314477 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1e04159880 0x7f1e0415dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:16.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:16.788+0000 7f1e0ba1a700 1 --2- 172.21.15.16:0/1894314477 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e04159c50 0x7f1e0415a0c0 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:16.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:16.788+0000 7f1e0ba1a700 1 -- 172.21.15.16:0/1894314477 >> 172.21.15.16:0/1894314477 conn(0x7f1e040ccab0 msgr2=0x7f1e040cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:16.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:16.788+0000 7f1e0ba1a700 1 -- 172.21.15.16:0/1894314477 shutdown_connections 2023-05-20T01:28:16.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:16.788+0000 7f1e0ba1a700 1 -- 172.21.15.16:0/1894314477 wait complete. 2023-05-20T01:28:16.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:16.789+0000 7f1e0ba1a700 1 Processor -- start 2023-05-20T01:28:16.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:16.789+0000 7f1e0ba1a700 1 -- start start 2023-05-20T01:28:16.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:16.790+0000 7f1e0ba1a700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1e040cdc00 0x7f1e040c5d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:16.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:16.790+0000 7f1e0ba1a700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1e04159880 0x7f1e04147790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:16.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:16.790+0000 7f1e0ba1a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e04159c50 0x7f1e040c96e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:16.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:16.790+0000 7f1e0ba1a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e040c6960 con 0x7f1e04159c50 2023-05-20T01:28:16.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:16.790+0000 7f1e0ba1a700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e040c63a0 con 0x7f1e040cdc00 2023-05-20T01:28:16.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:16.790+0000 7f1e0ba1a700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e040c6680 con 0x7f1e04159880 2023-05-20T01:28:16.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:16.790+0000 7f1e09fb7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e04159c50 0x7f1e040c96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:16.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:16.790+0000 7f1e09fb7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e04159c50 0x7f1e040c96e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39754/0 (socket says 172.21.15.16:39754) 2023-05-20T01:28:16.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:16.790+0000 7f1e09fb7700 1 -- 172.21.15.16:0/2477823219 learned_addr learned my addr 172.21.15.16:0/2477823219 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:16.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:16.790+0000 7f1e08fb5700 1 --2- 172.21.15.16:0/2477823219 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1e04159880 0x7f1e04147790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:16.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:16.790+0000 7f1e097b6700 1 --2- 172.21.15.16:0/2477823219 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1e040cdc00 0x7f1e040c5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:16.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:16.791+0000 7f1e09fb7700 1 -- 172.21.15.16:0/2477823219 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1e04159880 msgr2=0x7f1e04147790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:16.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:16.791+0000 7f1e09fb7700 1 --2- 172.21.15.16:0/2477823219 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1e04159880 0x7f1e04147790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:16.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:16.791+0000 7f1e09fb7700 1 -- 172.21.15.16:0/2477823219 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1e040cdc00 msgr2=0x7f1e040c5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:16.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:16.791+0000 7f1e09fb7700 1 --2- 172.21.15.16:0/2477823219 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1e040cdc00 0x7f1e040c5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:16.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:16.791+0000 7f1e09fb7700 1 -- 172.21.15.16:0/2477823219 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e0000b7f0 con 0x7f1e04159c50 2023-05-20T01:28:16.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:16.791+0000 7f1e097b6700 1 --2- 172.21.15.16:0/2477823219 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1e040cdc00 0x7f1e040c5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:28:16.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:16.791+0000 7f1e09fb7700 1 --2- 172.21.15.16:0/2477823219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e04159c50 0x7f1e040c96e0 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc00b970 tx=0x7f1dfc00bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:16.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:16.791+0000 7f1df9ffb700 1 -- 172.21.15.16:0/2477823219 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dfc00c930 con 0x7f1e04159c50 2023-05-20T01:28:16.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:16.791+0000 7f1df9ffb700 1 -- 172.21.15.16:0/2477823219 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dfc00cad0 con 0x7f1e04159c50 2023-05-20T01:28:16.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:16.791+0000 7f1e0ba1a700 1 -- 172.21.15.16:0/2477823219 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e0415d820 con 0x7f1e04159c50 2023-05-20T01:28:16.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:16.791+0000 7f1df9ffb700 1 -- 172.21.15.16:0/2477823219 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dfc00ce80 con 0x7f1e04159c50 2023-05-20T01:28:16.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:16.791+0000 7f1e0ba1a700 1 -- 172.21.15.16:0/2477823219 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e0415d540 con 0x7f1e04159c50 2023-05-20T01:28:16.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:16.792+0000 7f1e0ba1a700 1 -- 172.21.15.16:0/2477823219 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e0415e280 con 0x7f1e04159c50 2023-05-20T01:28:16.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:16.792+0000 7f1df9ffb700 1 -- 172.21.15.16:0/2477823219 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1dfc014440 con 0x7f1e04159c50 2023-05-20T01:28:16.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:16.793+0000 7f1df9ffb700 1 --2- 172.21.15.16:0/2477823219 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f1de806ad10 0x7f1de806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:16.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:16.793+0000 7f1df9ffb700 1 -- 172.21.15.16:0/2477823219 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f1dfc089230 con 0x7f1e04159c50 2023-05-20T01:28:16.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:16.793+0000 7f1e097b6700 1 --2- 172.21.15.16:0/2477823219 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f1de806ad10 0x7f1de806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:16.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:16.793+0000 7f1e097b6700 1 --2- 172.21.15.16:0/2477823219 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f1de806ad10 0x7f1de806d1c0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f1df400aa50 tx=0x7f1df4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:16.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:16.797+0000 7f1df9ffb700 1 -- 172.21.15.16:0/2477823219 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f1dfc0540f0 con 0x7f1e04159c50 2023-05-20T01:28:16.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:16.994+0000 7f1e0ba1a700 1 -- 172.21.15.16:0/2477823219 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f1e040747b0 con 0x7f1e04159c50 2023-05-20T01:28:17.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:17.634+0000 7f1df9ffb700 1 -- 172.21.15.16:0/2477823219 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v126) v1 ==== 169+0+0 (secure 0 0 0) 0x7f1dfc057710 con 0x7f1e04159c50 2023-05-20T01:28:17.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:17.639+0000 7f1e0ba1a700 1 -- 172.21.15.16:0/2477823219 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f1de806ad10 msgr2=0x7f1de806d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:17.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:17.639+0000 7f1e0ba1a700 1 --2- 172.21.15.16:0/2477823219 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f1de806ad10 0x7f1de806d1c0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f1df400aa50 tx=0x7f1df4008040 comp rx=0 tx=0).stop 2023-05-20T01:28:17.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:17.639+0000 7f1e0ba1a700 1 -- 172.21.15.16:0/2477823219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e04159c50 msgr2=0x7f1e040c96e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:17.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:17.639+0000 7f1e0ba1a700 1 --2- 172.21.15.16:0/2477823219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e04159c50 0x7f1e040c96e0 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc00b970 tx=0x7f1dfc00bc80 comp rx=0 tx=0).stop 2023-05-20T01:28:17.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:17.639+0000 7f1e0ba1a700 1 -- 172.21.15.16:0/2477823219 shutdown_connections 2023-05-20T01:28:17.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:17.639+0000 7f1e0ba1a700 1 --2- 172.21.15.16:0/2477823219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e04159c50 0x7f1e040c96e0 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:17.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:17.639+0000 7f1e0ba1a700 1 --2- 172.21.15.16:0/2477823219 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1e040cdc00 0x7f1e040c5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:17.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:17.639+0000 7f1e0ba1a700 1 --2- 172.21.15.16:0/2477823219 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f1de806ad10 0x7f1de806d1c0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:17.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:17.639+0000 7f1e0ba1a700 1 --2- 172.21.15.16:0/2477823219 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1e04159880 0x7f1e04147790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:17.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:17.639+0000 7f1e0ba1a700 1 -- 172.21.15.16:0/2477823219 >> 172.21.15.16:0/2477823219 conn(0x7f1e040ccab0 msgr2=0x7f1e040718c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:17.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:17.640+0000 7f1e0ba1a700 1 -- 172.21.15.16:0/2477823219 shutdown_connections 2023-05-20T01:28:17.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:17.640+0000 7f1e0ba1a700 1 -- 172.21.15.16:0/2477823219 wait complete. 2023-05-20T01:28:17.646 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' removed 2023-05-20T01:28:17.659 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_extra cephfs_data_extra --yes-i-really-really-mean-it 2023-05-20T01:28:17.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:17.791+0000 7f899bc1e700 1 Processor -- start 2023-05-20T01:28:17.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:17.791+0000 7f899bc1e700 1 -- start start 2023-05-20T01:28:17.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:17.792+0000 7f899bc1e700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8994159c50 0x7f899415a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:17.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:17.792+0000 7f899bc1e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f89940cdc00 0x7f89940cdff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:17.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:17.792+0000 7f899bc1e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f899415adc0 con 0x7f8994159880 2023-05-20T01:28:17.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:17.792+0000 7f899bc1e700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f899415a610 con 0x7f8994159c50 2023-05-20T01:28:17.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:17.792+0000 7f899bc1e700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89940ce530 con 0x7f89940cdc00 2023-05-20T01:28:17.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:17.792+0000 7f89991b9700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8994159c50 0x7f899415a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:17.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:17.792+0000 7f89991b9700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8994159c50 0x7f899415a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39760/0 (socket says 172.21.15.16:39760) 2023-05-20T01:28:17.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:17.792+0000 7f89991b9700 1 -- 172.21.15.16:0/2327617253 learned_addr learned my addr 172.21.15.16:0/2327617253 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:17.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:17.792+0000 7f89991b9700 1 -- 172.21.15.16:0/2327617253 >> v2:172.21.15.132:3300/0 conn(0x7f89940cdc00 msgr2=0x7f89940cdff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:17.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:17.793+0000 7f899a1bb700 1 --2- 172.21.15.16:0/2327617253 >> v2:172.21.15.132:3300/0 conn(0x7f89940cdc00 0x7f89940cdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:17.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:17.793+0000 7f89991b9700 1 --2- 172.21.15.16:0/2327617253 >> v2:172.21.15.132:3300/0 conn(0x7f89940cdc00 0x7f89940cdff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:17.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:17.793+0000 7f89991b9700 1 -- 172.21.15.16:0/2327617253 >> v1:172.21.15.132:6789/0 conn(0x7f8994159880 legacy=0x7f899415dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:17.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:17.793+0000 7f89991b9700 1 -- 172.21.15.16:0/2327617253 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f899415a800 con 0x7f8994159c50 2023-05-20T01:28:17.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:17.793+0000 7f89991b9700 1 --2- 172.21.15.16:0/2327617253 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8994159c50 0x7f899415a0d0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f8994070ff0 tx=0x7f8990009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef1484829bc0dc98 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:17.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:17.793+0000 7f89989b8700 1 -- 172.21.15.16:0/2327617253 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8990004230 con 0x7f8994159c50 2023-05-20T01:28:17.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:17.793+0000 7f89989b8700 1 -- 172.21.15.16:0/2327617253 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8990004390 con 0x7f8994159c50 2023-05-20T01:28:17.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:17.793+0000 7f89989b8700 1 -- 172.21.15.16:0/2327617253 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89900036a0 con 0x7f8994159c50 2023-05-20T01:28:17.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:17.793+0000 7f899bc1e700 1 -- 172.21.15.16:0/2327617253 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8994159c50 msgr2=0x7f899415a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:17.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:17.793+0000 7f899bc1e700 1 --2- 172.21.15.16:0/2327617253 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8994159c50 0x7f899415a0d0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f8994070ff0 tx=0x7f8990009bd0 comp rx=0 tx=0).stop 2023-05-20T01:28:17.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:17.794+0000 7f899bc1e700 1 -- 172.21.15.16:0/2327617253 shutdown_connections 2023-05-20T01:28:17.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:17.794+0000 7f899bc1e700 1 --2- 172.21.15.16:0/2327617253 >> v2:172.21.15.132:3300/0 conn(0x7f89940cdc00 0x7f89940cdff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:17.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:17.794+0000 7f899bc1e700 1 --2- 172.21.15.16:0/2327617253 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8994159c50 0x7f899415a0d0 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:17.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:17.794+0000 7f899bc1e700 1 -- 172.21.15.16:0/2327617253 >> 172.21.15.16:0/2327617253 conn(0x7f89940ccab0 msgr2=0x7f89940cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:17.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:17.794+0000 7f899bc1e700 1 -- 172.21.15.16:0/2327617253 shutdown_connections 2023-05-20T01:28:17.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:17.794+0000 7f899bc1e700 1 -- 172.21.15.16:0/2327617253 wait complete. 2023-05-20T01:28:17.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:17.795+0000 7f899bc1e700 1 Processor -- start 2023-05-20T01:28:17.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:17.795+0000 7f899bc1e700 1 -- start start 2023-05-20T01:28:17.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:17.796+0000 7f899bc1e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f89940cdc00 0x7f89941564f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:17.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:17.796+0000 7f899bc1e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8994159880 0x7f89940c72b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:17.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:17.796+0000 7f899bc1e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8994159c50 0x7f899406a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:17.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:17.796+0000 7f899bc1e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89940c9c90 con 0x7f8994159c50 2023-05-20T01:28:17.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:17.796+0000 7f899bc1e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89940c9080 con 0x7f8994159880 2023-05-20T01:28:17.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:17.796+0000 7f899bc1e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89940c9360 con 0x7f89940cdc00 2023-05-20T01:28:17.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:17.796+0000 7f899a1bb700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8994159c50 0x7f899406a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:17.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:17.796+0000 7f899a1bb700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8994159c50 0x7f899406a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39768/0 (socket says 172.21.15.16:39768) 2023-05-20T01:28:17.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:17.796+0000 7f899a1bb700 1 -- 172.21.15.16:0/3959379921 learned_addr learned my addr 172.21.15.16:0/3959379921 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:17.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:17.796+0000 7f89999ba700 1 --2- 172.21.15.16:0/3959379921 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f89940cdc00 0x7f89941564f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:17.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:17.796+0000 7f89991b9700 1 --2- 172.21.15.16:0/3959379921 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8994159880 0x7f89940c72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:17.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:17.796+0000 7f899a1bb700 1 -- 172.21.15.16:0/3959379921 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f89940cdc00 msgr2=0x7f89941564f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:17.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:17.796+0000 7f899a1bb700 1 --2- 172.21.15.16:0/3959379921 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f89940cdc00 0x7f89941564f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:17.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:17.796+0000 7f899a1bb700 1 -- 172.21.15.16:0/3959379921 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8994159880 msgr2=0x7f89940c72b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:17.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:17.796+0000 7f899a1bb700 1 --2- 172.21.15.16:0/3959379921 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8994159880 0x7f89940c72b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:17.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:17.796+0000 7f899a1bb700 1 -- 172.21.15.16:0/3959379921 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8990009550 con 0x7f8994159c50 2023-05-20T01:28:17.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:17.796+0000 7f899a1bb700 1 --2- 172.21.15.16:0/3959379921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8994159c50 0x7f899406a510 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f898c00cc30 tx=0x7f898c00cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:17.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:17.796+0000 7f898a7fc700 1 -- 172.21.15.16:0/3959379921 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f898c00aa50 con 0x7f8994159c50 2023-05-20T01:28:17.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:17.796+0000 7f898a7fc700 1 -- 172.21.15.16:0/3959379921 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f898c00abb0 con 0x7f8994159c50 2023-05-20T01:28:17.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:17.796+0000 7f899bc1e700 1 -- 172.21.15.16:0/3959379921 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f899415d540 con 0x7f8994159c50 2023-05-20T01:28:17.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:17.796+0000 7f898a7fc700 1 -- 172.21.15.16:0/3959379921 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f898c003a20 con 0x7f8994159c50 2023-05-20T01:28:17.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:17.796+0000 7f899bc1e700 1 -- 172.21.15.16:0/3959379921 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f899415aae0 con 0x7f8994159c50 2023-05-20T01:28:17.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:17.797+0000 7f899bc1e700 1 -- 172.21.15.16:0/3959379921 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f899415d820 con 0x7f8994159c50 2023-05-20T01:28:17.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:17.798+0000 7f898a7fc700 1 -- 172.21.15.16:0/3959379921 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f898c00f3f0 con 0x7f8994159c50 2023-05-20T01:28:17.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:17.798+0000 7f898a7fc700 1 --2- 172.21.15.16:0/3959379921 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f897806ace0 0x7f897806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:17.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:17.798+0000 7f898a7fc700 1 -- 172.21.15.16:0/3959379921 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6278+0+0 (secure 0 0 0) 0x7f898c089670 con 0x7f8994159c50 2023-05-20T01:28:17.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:17.798+0000 7f89999ba700 1 --2- 172.21.15.16:0/3959379921 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f897806ace0 0x7f897806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:17.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:17.799+0000 7f89999ba700 1 --2- 172.21.15.16:0/3959379921 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f897806ace0 0x7f897806d190 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f899406e690 tx=0x7f89840090e0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:17.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:17.806+0000 7f898a7fc700 1 -- 172.21.15.16:0/3959379921 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f898c054640 con 0x7f8994159c50 2023-05-20T01:28:18.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:18.007+0000 7f899bc1e700 1 -- 172.21.15.16:0/3959379921 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_extra", "pool2": "cephfs_data_extra", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f89940747b0 con 0x7f8994159c50 2023-05-20T01:28:18.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:18.636+0000 7f898a7fc700 1 -- 172.21.15.16:0/3959379921 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_extra", "pool2": "cephfs_data_extra", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_extra' removed v127) v1 ==== 187+0+0 (secure 0 0 0) 0x7f898c057c60 con 0x7f8994159c50 2023-05-20T01:28:18.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:18.640+0000 7f899bc1e700 1 -- 172.21.15.16:0/3959379921 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f897806ace0 msgr2=0x7f897806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:18.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:18.640+0000 7f899bc1e700 1 --2- 172.21.15.16:0/3959379921 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f897806ace0 0x7f897806d190 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f899406e690 tx=0x7f89840090e0 comp rx=0 tx=0).stop 2023-05-20T01:28:18.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:18.640+0000 7f899bc1e700 1 -- 172.21.15.16:0/3959379921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8994159c50 msgr2=0x7f899406a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:18.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:18.640+0000 7f899bc1e700 1 --2- 172.21.15.16:0/3959379921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8994159c50 0x7f899406a510 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f898c00cc30 tx=0x7f898c00cf40 comp rx=0 tx=0).stop 2023-05-20T01:28:18.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:18.640+0000 7f899bc1e700 1 -- 172.21.15.16:0/3959379921 shutdown_connections 2023-05-20T01:28:18.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:18.640+0000 7f899bc1e700 1 --2- 172.21.15.16:0/3959379921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8994159c50 0x7f899406a510 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:18.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:18.640+0000 7f899bc1e700 1 --2- 172.21.15.16:0/3959379921 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f897806ace0 0x7f897806d190 secure :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f899406e690 tx=0x7f89840090e0 comp rx=0 tx=0).stop 2023-05-20T01:28:18.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:18.640+0000 7f899bc1e700 1 --2- 172.21.15.16:0/3959379921 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f89940cdc00 0x7f89941564f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:18.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:18.640+0000 7f899bc1e700 1 --2- 172.21.15.16:0/3959379921 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8994159880 0x7f89940c72b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:18.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:18.640+0000 7f899bc1e700 1 -- 172.21.15.16:0/3959379921 >> 172.21.15.16:0/3959379921 conn(0x7f89940ccab0 msgr2=0x7f8994071200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:18.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:18.643+0000 7f899bc1e700 1 -- 172.21.15.16:0/3959379921 shutdown_connections 2023-05-20T01:28:18.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:18.643+0000 7f899bc1e700 1 -- 172.21.15.16:0/3959379921 wait complete. 2023-05-20T01:28:18.649 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data_extra' removed 2023-05-20T01:28:18.666 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-05-20T01:28:18.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:18.793+0000 7f83af955700 1 Processor -- start 2023-05-20T01:28:18.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:18.794+0000 7f83af955700 1 -- start start 2023-05-20T01:28:18.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:18.794+0000 7f83af955700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f83a815cec0 0x7f83a80cd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:18.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:18.795+0000 7f83af955700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83a806dd00 con 0x7f83a806e190 2023-05-20T01:28:18.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:18.795+0000 7f83af955700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83a80cfa60 con 0x7f83a806a510 2023-05-20T01:28:18.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:18.795+0000 7f83af955700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83a806d6b0 con 0x7f83a815cec0 2023-05-20T01:28:18.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:18.795+0000 7f83acef0700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f83a806e190 0x7f83a80cf1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51302/0 (socket says 172.21.15.16:51302) 2023-05-20T01:28:18.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:18.795+0000 7f83acef0700 1 -- 172.21.15.16:0/1843175241 learned_addr learned my addr 172.21.15.16:0/1843175241 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:28:18.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:18.795+0000 7f83ad6f1700 1 --2- 172.21.15.16:0/1843175241 >> v2:172.21.15.132:3300/0 conn(0x7f83a815cec0 0x7f83a80cd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:18.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:18.795+0000 7f839ffff700 1 -- 172.21.15.16:0/1843175241 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3380241776 0 0) 0x7f83a806dd00 con 0x7f83a806e190 2023-05-20T01:28:18.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:18.795+0000 7f839ffff700 1 -- 172.21.15.16:0/1843175241 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f83940035b0 con 0x7f83a806e190 2023-05-20T01:28:18.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:18.795+0000 7f839ffff700 1 -- 172.21.15.16:0/1843175241 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3282214167 0 0) 0x7f83940035b0 con 0x7f83a806e190 2023-05-20T01:28:18.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:18.795+0000 7f839ffff700 1 -- 172.21.15.16:0/1843175241 >> v2:172.21.15.132:3300/0 conn(0x7f83a815cec0 msgr2=0x7f83a80cd4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:18.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:18.795+0000 7f839ffff700 1 --2- 172.21.15.16:0/1843175241 >> v2:172.21.15.132:3300/0 conn(0x7f83a815cec0 0x7f83a80cd4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:18.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:18.795+0000 7f839ffff700 1 -- 172.21.15.16:0/1843175241 >> v1:172.21.15.132:6789/0 conn(0x7f83a806a510 legacy=0x7f83a80710d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:18.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:18.795+0000 7f839ffff700 1 -- 172.21.15.16:0/1843175241 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83a806d990 con 0x7f83a806e190 2023-05-20T01:28:18.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:18.795+0000 7f839ffff700 1 -- 172.21.15.16:0/1843175241 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7f8398001670 con 0x7f83a806e190 2023-05-20T01:28:18.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:18.795+0000 7f839ffff700 1 -- 172.21.15.16:0/1843175241 >> v1:172.21.15.16:6789/0 conn(0x7f83a806e190 legacy=0x7f83a80cf1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:18.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:18.795+0000 7f839ffff700 1 --2- 172.21.15.16:0/1843175241 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8394003a00 0x7f8394003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:18.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:18.795+0000 7f839ffff700 1 --2- 172.21.15.16:0/1843175241 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8394004850 0x7f8394004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:18.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:18.795+0000 7f839ffff700 1 --2- 172.21.15.16:0/1843175241 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8394005200 0x7f839400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:18.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:18.795+0000 7f839ffff700 1 -- 172.21.15.16:0/1843175241 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83a806d6b0 con 0x7f8394005200 2023-05-20T01:28:18.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:18.795+0000 7f839ffff700 1 -- 172.21.15.16:0/1843175241 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f839400c080 con 0x7f8394003a00 2023-05-20T01:28:18.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:18.795+0000 7f839ffff700 1 -- 172.21.15.16:0/1843175241 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f839400c240 con 0x7f8394004850 2023-05-20T01:28:18.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:18.795+0000 7f83adef2700 1 --2- 172.21.15.16:0/1843175241 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8394005200 0x7f839400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:18.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:18.795+0000 7f83ad6f1700 1 --2- 172.21.15.16:0/1843175241 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8394003a00 0x7f8394003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:18.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:18.796+0000 7f83acef0700 1 --2- 172.21.15.16:0/1843175241 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8394004850 0x7f8394004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:18.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:18.796+0000 7f83adef2700 1 -- 172.21.15.16:0/1843175241 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8394004850 msgr2=0x7f8394004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:18.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:18.796+0000 7f83adef2700 1 --2- 172.21.15.16:0/1843175241 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8394004850 0x7f8394004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:18.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:18.796+0000 7f83adef2700 1 -- 172.21.15.16:0/1843175241 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8394003a00 msgr2=0x7f8394003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:18.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:18.796+0000 7f83adef2700 1 --2- 172.21.15.16:0/1843175241 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8394003a00 0x7f8394003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:18.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:18.796+0000 7f83adef2700 1 -- 172.21.15.16:0/1843175241 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83940035b0 con 0x7f8394005200 2023-05-20T01:28:18.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:18.796+0000 7f83ad6f1700 1 --2- 172.21.15.16:0/1843175241 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8394003a00 0x7f8394003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-05-20T01:28:18.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:18.796+0000 7f83adef2700 1 -- 172.21.15.16:0/1843175241 reap_dead start 2023-05-20T01:28:18.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:18.796+0000 7f83adef2700 1 --2- 172.21.15.16:0/1843175241 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8394005200 0x7f839400ba10 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f83a40017c0 tx=0x7f8394005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dab8131b6affc7b server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:18.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:18.796+0000 7f839ffff700 1 -- 172.21.15.16:0/1843175241 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a40052d0 con 0x7f8394005200 2023-05-20T01:28:18.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:18.796+0000 7f839ffff700 1 -- 172.21.15.16:0/1843175241 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83a4005430 con 0x7f8394005200 2023-05-20T01:28:18.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:18.796+0000 7f839ffff700 1 -- 172.21.15.16:0/1843175241 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a40038c0 con 0x7f8394005200 2023-05-20T01:28:18.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:18.797+0000 7f83af955700 1 -- 172.21.15.16:0/1843175241 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8394005200 msgr2=0x7f839400ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:18.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:18.797+0000 7f83af955700 1 --2- 172.21.15.16:0/1843175241 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8394005200 0x7f839400ba10 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f83a40017c0 tx=0x7f8394005880 comp rx=0 tx=0).stop 2023-05-20T01:28:18.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:18.797+0000 7f83af955700 1 -- 172.21.15.16:0/1843175241 shutdown_connections 2023-05-20T01:28:18.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:18.797+0000 7f83af955700 1 --2- 172.21.15.16:0/1843175241 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8394005200 0x7f839400ba10 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:18.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:18.797+0000 7f83af955700 1 -- 172.21.15.16:0/1843175241 >> 172.21.15.16:0/1843175241 conn(0x7f83a806c260 msgr2=0x7f83a80cffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:18.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:18.797+0000 7f83af955700 1 -- 172.21.15.16:0/1843175241 shutdown_connections 2023-05-20T01:28:18.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:18.797+0000 7f83af955700 1 -- 172.21.15.16:0/1843175241 wait complete. 2023-05-20T01:28:18.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:18.798+0000 7f83af955700 1 Processor -- start 2023-05-20T01:28:18.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:18.798+0000 7f83af955700 1 -- start start 2023-05-20T01:28:18.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:18.799+0000 7f83af955700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8394005200 0x7f83a81e67e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:18.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:18.799+0000 7f83af955700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83a81e7f60 0x7f83a81548e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:18.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:18.799+0000 7f83af955700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f83a81550e0 0x7f83a81e8d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:18.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:18.799+0000 7f83af955700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83a8155980 con 0x7f83a81e7f60 2023-05-20T01:28:18.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:18.799+0000 7f83af955700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83a81e6ec0 con 0x7f83a81550e0 2023-05-20T01:28:18.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:18.799+0000 7f83af955700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83a81e71a0 con 0x7f8394005200 2023-05-20T01:28:18.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:18.799+0000 7f83acef0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83a81e7f60 0x7f83a81548e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:18.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:18.799+0000 7f83acef0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83a81e7f60 0x7f83a81548e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39786/0 (socket says 172.21.15.16:39786) 2023-05-20T01:28:18.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:18.799+0000 7f83acef0700 1 -- 172.21.15.16:0/2800492814 learned_addr learned my addr 172.21.15.16:0/2800492814 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:18.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:18.800+0000 7f83ad6f1700 1 --2- 172.21.15.16:0/2800492814 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8394005200 0x7f83a81e67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:18.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:18.800+0000 7f83adef2700 1 --2- 172.21.15.16:0/2800492814 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f83a81550e0 0x7f83a81e8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:18.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:18.800+0000 7f83acef0700 1 -- 172.21.15.16:0/2800492814 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8394005200 msgr2=0x7f83a81e67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:18.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:18.800+0000 7f83acef0700 1 --2- 172.21.15.16:0/2800492814 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8394005200 0x7f83a81e67e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:18.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:18.800+0000 7f83acef0700 1 -- 172.21.15.16:0/2800492814 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f83a81550e0 msgr2=0x7f83a81e8d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:18.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:18.800+0000 7f83acef0700 1 --2- 172.21.15.16:0/2800492814 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f83a81550e0 0x7f83a81e8d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:18.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:18.800+0000 7f83acef0700 1 -- 172.21.15.16:0/2800492814 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83a80cfd40 con 0x7f83a81e7f60 2023-05-20T01:28:18.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:18.800+0000 7f83acef0700 1 --2- 172.21.15.16:0/2800492814 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83a81e7f60 0x7f83a81548e0 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f83980072c0 tx=0x7f83980075d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:18.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:18.800+0000 7f839dffb700 1 -- 172.21.15.16:0/2800492814 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f839800f6c0 con 0x7f83a81e7f60 2023-05-20T01:28:18.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:18.800+0000 7f839dffb700 1 -- 172.21.15.16:0/2800492814 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f839800f840 con 0x7f83a81e7f60 2023-05-20T01:28:18.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:18.800+0000 7f83af955700 1 -- 172.21.15.16:0/2800492814 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83a815d290 con 0x7f83a81e7f60 2023-05-20T01:28:18.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:18.800+0000 7f839dffb700 1 -- 172.21.15.16:0/2800492814 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f839800fbf0 con 0x7f83a81e7f60 2023-05-20T01:28:18.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:18.800+0000 7f83af955700 1 -- 172.21.15.16:0/2800492814 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83a806e600 con 0x7f83a81e7f60 2023-05-20T01:28:18.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:18.801+0000 7f83af955700 1 -- 172.21.15.16:0/2800492814 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83a80747b0 con 0x7f83a81e7f60 2023-05-20T01:28:18.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:18.801+0000 7f839dffb700 1 -- 172.21.15.16:0/2800492814 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f839800fd50 con 0x7f83a81e7f60 2023-05-20T01:28:18.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:18.802+0000 7f839dffb700 1 --2- 172.21.15.16:0/2800492814 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f838c06ace0 0x7f838c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:18.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:18.802+0000 7f839dffb700 1 -- 172.21.15.16:0/2800492814 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5874+0+0 (secure 0 0 0) 0x7f83980892f0 con 0x7f83a81e7f60 2023-05-20T01:28:18.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:18.802+0000 7f83ad6f1700 1 --2- 172.21.15.16:0/2800492814 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f838c06ace0 0x7f838c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:18.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:18.802+0000 7f83ad6f1700 1 --2- 172.21.15.16:0/2800492814 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f838c06ace0 0x7f838c06d190 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f83a0009fd0 tx=0x7f83a000b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:18.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:18.806+0000 7f839dffb700 1 -- 172.21.15.16:0/2800492814 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f8398054450 con 0x7f83a81e7f60 2023-05-20T01:28:19.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.004+0000 7f83af955700 1 -- 172.21.15.16:0/2800492814 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f83a8070ca0 con 0x7f83a81e7f60 2023-05-20T01:28:19.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.007+0000 7f839dffb700 1 -- 172.21.15.16:0/2800492814 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v17) v1 ==== 104+0+0 (secure 0 0 0) 0x7f8398057a70 con 0x7f83a81e7f60 2023-05-20T01:28:19.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.010+0000 7f83af955700 1 -- 172.21.15.16:0/2800492814 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f838c06ace0 msgr2=0x7f838c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:19.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.010+0000 7f83af955700 1 --2- 172.21.15.16:0/2800492814 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f838c06ace0 0x7f838c06d190 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f83a0009fd0 tx=0x7f83a000b040 comp rx=0 tx=0).stop 2023-05-20T01:28:19.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.010+0000 7f83af955700 1 -- 172.21.15.16:0/2800492814 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83a81e7f60 msgr2=0x7f83a81548e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:19.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.010+0000 7f83af955700 1 --2- 172.21.15.16:0/2800492814 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83a81e7f60 0x7f83a81548e0 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f83980072c0 tx=0x7f83980075d0 comp rx=0 tx=0).stop 2023-05-20T01:28:19.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.010+0000 7f83af955700 1 -- 172.21.15.16:0/2800492814 shutdown_connections 2023-05-20T01:28:19.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.010+0000 7f83af955700 1 --2- 172.21.15.16:0/2800492814 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f83a81550e0 0x7f83a81e8d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:19.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.010+0000 7f83af955700 1 --2- 172.21.15.16:0/2800492814 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f838c06ace0 0x7f838c06d190 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:19.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.010+0000 7f83af955700 1 --2- 172.21.15.16:0/2800492814 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8394005200 0x7f83a81e67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:19.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.010+0000 7f83af955700 1 --2- 172.21.15.16:0/2800492814 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83a81e7f60 0x7f83a81548e0 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:19.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.010+0000 7f83af955700 1 -- 172.21.15.16:0/2800492814 >> 172.21.15.16:0/2800492814 conn(0x7f83a806c260 msgr2=0x7f83a8072fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:19.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.010+0000 7f83af955700 1 -- 172.21.15.16:0/2800492814 shutdown_connections 2023-05-20T01:28:19.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.010+0000 7f83af955700 1 -- 172.21.15.16:0/2800492814 wait complete. 2023-05-20T01:28:19.026 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_data_scan.TestDataScan.test_extra_data_pool_stashed_layout' 2023-05-20T01:28:19.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.148+0000 7f015bcdc700 1 Processor -- start 2023-05-20T01:28:19.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.148+0000 7f015bcdc700 1 -- start start 2023-05-20T01:28:19.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.149+0000 7f015bcdc700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f015406d4b0 0x7f01540cf890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:19.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.149+0000 7f015bcdc700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01540cc510 con 0x7f015406f450 2023-05-20T01:28:19.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.149+0000 7f015bcdc700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0154071ed0 con 0x7f015406a510 2023-05-20T01:28:19.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.149+0000 7f015bcdc700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01540721b0 con 0x7f015406d4b0 2023-05-20T01:28:19.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.149+0000 7f0159a78700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f015406f450 0x7f0154161080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51320/0 (socket says 172.21.15.16:51320) 2023-05-20T01:28:19.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.149+0000 7f0159a78700 1 -- 172.21.15.16:0/477942226 learned_addr learned my addr 172.21.15.16:0/477942226 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:28:19.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.149+0000 7f0159277700 1 --2- 172.21.15.16:0/477942226 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f015406d4b0 0x7f01540cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:19.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.149+0000 7f0158a76700 1 -- 172.21.15.16:0/477942226 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2030426340 0 0) 0x7f01540cc510 con 0x7f015406f450 2023-05-20T01:28:19.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.149+0000 7f0158a76700 1 -- 172.21.15.16:0/477942226 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01400035b0 con 0x7f015406f450 2023-05-20T01:28:19.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.149+0000 7f0158a76700 1 -- 172.21.15.16:0/477942226 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 103961669 0 0) 0x7f0154071ed0 con 0x7f015406a510 2023-05-20T01:28:19.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.149+0000 7f0158a76700 1 -- 172.21.15.16:0/477942226 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01540cc510 con 0x7f015406a510 2023-05-20T01:28:19.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.150+0000 7f0159277700 1 -- 172.21.15.16:0/477942226 >> v1:172.21.15.132:6789/0 conn(0x7f015406a510 legacy=0x7f01540cc9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:19.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.150+0000 7f0158a76700 1 -- 172.21.15.16:0/477942226 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1953295971 0 0) 0x7f01400035b0 con 0x7f015406f450 2023-05-20T01:28:19.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.150+0000 7f0159277700 1 -- 172.21.15.16:0/477942226 >> v1:172.21.15.16:6789/0 conn(0x7f015406f450 legacy=0x7f0154161080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:19.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.150+0000 7f0159277700 1 -- 172.21.15.16:0/477942226 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01540cc1a0 con 0x7f015406d4b0 2023-05-20T01:28:19.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.150+0000 7f0159277700 1 --2- 172.21.15.16:0/477942226 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f015406d4b0 0x7f01540cf890 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0150009870 tx=0x7f0150009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9064a2743c3ab12f server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:19.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.150+0000 7f0158a76700 1 -- 172.21.15.16:0/477942226 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0144001670 con 0x7f015406d4b0 2023-05-20T01:28:19.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.150+0000 7f0158a76700 1 -- 172.21.15.16:0/477942226 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0150004030 con 0x7f015406d4b0 2023-05-20T01:28:19.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.150+0000 7f0158a76700 1 -- 172.21.15.16:0/477942226 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01500043a0 con 0x7f015406d4b0 2023-05-20T01:28:19.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.150+0000 7f015bcdc700 1 -- 172.21.15.16:0/477942226 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f015406d4b0 msgr2=0x7f01540cf890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:19.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.150+0000 7f015bcdc700 1 --2- 172.21.15.16:0/477942226 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f015406d4b0 0x7f01540cf890 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0150009870 tx=0x7f0150009b80 comp rx=0 tx=0).stop 2023-05-20T01:28:19.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.150+0000 7f015bcdc700 1 -- 172.21.15.16:0/477942226 shutdown_connections 2023-05-20T01:28:19.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.150+0000 7f015bcdc700 1 --2- 172.21.15.16:0/477942226 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f015406d4b0 0x7f01540cf890 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:19.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.150+0000 7f015bcdc700 1 -- 172.21.15.16:0/477942226 >> 172.21.15.16:0/477942226 conn(0x7f01540cffb0 msgr2=0x7f01540d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:19.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.151+0000 7f015bcdc700 1 -- 172.21.15.16:0/477942226 shutdown_connections 2023-05-20T01:28:19.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.151+0000 7f015bcdc700 1 -- 172.21.15.16:0/477942226 wait complete. 2023-05-20T01:28:19.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.152+0000 7f015bcdc700 1 Processor -- start 2023-05-20T01:28:19.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.152+0000 7f015bcdc700 1 -- start start 2023-05-20T01:28:19.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.152+0000 7f015bcdc700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f015406a510 0x7f01541587f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:19.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.152+0000 7f015bcdc700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f015406d4b0 0x7f015406e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:19.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.152+0000 7f015bcdc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f015406f450 0x7f015406e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:19.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.152+0000 7f015bcdc700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01540ad250 con 0x7f015406f450 2023-05-20T01:28:19.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.152+0000 7f015bcdc700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01540b1870 con 0x7f015406a510 2023-05-20T01:28:19.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.152+0000 7f015bcdc700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01541f6fd0 con 0x7f015406d4b0 2023-05-20T01:28:19.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.153+0000 7f015a279700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f015406f450 0x7f015406e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:19.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.153+0000 7f015a279700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f015406f450 0x7f015406e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39802/0 (socket says 172.21.15.16:39802) 2023-05-20T01:28:19.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.153+0000 7f015a279700 1 -- 172.21.15.16:0/2338323826 learned_addr learned my addr 172.21.15.16:0/2338323826 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:19.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.153+0000 7f0159277700 1 --2- 172.21.15.16:0/2338323826 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f015406d4b0 0x7f015406e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:19.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.153+0000 7f0159a78700 1 --2- 172.21.15.16:0/2338323826 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f015406a510 0x7f01541587f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:19.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.153+0000 7f015a279700 1 -- 172.21.15.16:0/2338323826 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f015406d4b0 msgr2=0x7f015406e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:19.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.153+0000 7f015a279700 1 --2- 172.21.15.16:0/2338323826 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f015406d4b0 0x7f015406e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:19.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.153+0000 7f015a279700 1 -- 172.21.15.16:0/2338323826 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f015406a510 msgr2=0x7f01541587f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:19.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.153+0000 7f015a279700 1 --2- 172.21.15.16:0/2338323826 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f015406a510 0x7f01541587f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:19.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.153+0000 7f015a279700 1 -- 172.21.15.16:0/2338323826 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01500094a0 con 0x7f015406f450 2023-05-20T01:28:19.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.153+0000 7f0159a78700 1 --2- 172.21.15.16:0/2338323826 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f015406a510 0x7f01541587f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:28:19.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.153+0000 7f015a279700 1 --2- 172.21.15.16:0/2338323826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f015406f450 0x7f015406e6f0 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f014c00cc20 tx=0x7f014c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:19.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.153+0000 7f014a7fc700 1 -- 172.21.15.16:0/2338323826 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f014c00ac20 con 0x7f015406f450 2023-05-20T01:28:19.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.153+0000 7f014a7fc700 1 -- 172.21.15.16:0/2338323826 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f014c00ad80 con 0x7f015406f450 2023-05-20T01:28:19.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.153+0000 7f015bcdc700 1 -- 172.21.15.16:0/2338323826 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f015406a9b0 con 0x7f015406f450 2023-05-20T01:28:19.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.153+0000 7f014a7fc700 1 -- 172.21.15.16:0/2338323826 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f014c003c20 con 0x7f015406f450 2023-05-20T01:28:19.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.153+0000 7f015bcdc700 1 -- 172.21.15.16:0/2338323826 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01540cbe50 con 0x7f015406f450 2023-05-20T01:28:19.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.154+0000 7f015bcdc700 1 -- 172.21.15.16:0/2338323826 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f015406ab40 con 0x7f015406f450 2023-05-20T01:28:19.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.155+0000 7f014a7fc700 1 -- 172.21.15.16:0/2338323826 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f014c0050d0 con 0x7f015406f450 2023-05-20T01:28:19.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.155+0000 7f014a7fc700 1 --2- 172.21.15.16:0/2338323826 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f013806ace0 0x7f013806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:19.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.155+0000 7f014a7fc700 1 -- 172.21.15.16:0/2338323826 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5874+0+0 (secure 0 0 0) 0x7f014c089580 con 0x7f015406f450 2023-05-20T01:28:19.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.155+0000 7f0159a78700 1 --2- 172.21.15.16:0/2338323826 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f013806ace0 0x7f013806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:19.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.156+0000 7f0159a78700 1 --2- 172.21.15.16:0/2338323826 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f013806ace0 0x7f013806d190 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f01440011e0 tx=0x7f0144009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:19.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.159+0000 7f014a7fc700 1 -- 172.21.15.16:0/2338323826 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f014c0546f0 con 0x7f015406f450 2023-05-20T01:28:19.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.446+0000 7f015bcdc700 1 -- 172.21.15.16:0/2338323826 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_data_scan.TestDataScan.test_extra_data_pool_stashed_layout"]} v 0) v1 -- 0x7f01540747b0 con 0x7f015406f450 2023-05-20T01:28:19.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.645+0000 7f014a7fc700 1 -- 172.21.15.16:0/2338323826 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_data_scan.TestDataScan.test_extra_data_pool_stashed_layout"]}]=0 v508) v1 ==== 155+0+0 (secure 0 0 0) 0x7f014c057d10 con 0x7f015406f450 2023-05-20T01:28:19.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.647+0000 7f015bcdc700 1 -- 172.21.15.16:0/2338323826 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f013806ace0 msgr2=0x7f013806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:19.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.647+0000 7f015bcdc700 1 --2- 172.21.15.16:0/2338323826 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f013806ace0 0x7f013806d190 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f01440011e0 tx=0x7f0144009040 comp rx=0 tx=0).stop 2023-05-20T01:28:19.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.647+0000 7f015bcdc700 1 -- 172.21.15.16:0/2338323826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f015406f450 msgr2=0x7f015406e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:19.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.647+0000 7f015bcdc700 1 --2- 172.21.15.16:0/2338323826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f015406f450 0x7f015406e6f0 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f014c00cc20 tx=0x7f014c00a060 comp rx=0 tx=0).stop 2023-05-20T01:28:19.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.647+0000 7f015bcdc700 1 -- 172.21.15.16:0/2338323826 shutdown_connections 2023-05-20T01:28:19.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.647+0000 7f015bcdc700 1 --2- 172.21.15.16:0/2338323826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f015406f450 0x7f015406e6f0 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:19.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.647+0000 7f015bcdc700 1 --2- 172.21.15.16:0/2338323826 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f015406a510 0x7f01541587f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:19.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.647+0000 7f015bcdc700 1 --2- 172.21.15.16:0/2338323826 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f013806ace0 0x7f013806d190 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:19.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.647+0000 7f015bcdc700 1 --2- 172.21.15.16:0/2338323826 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f015406d4b0 0x7f015406e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:19.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.647+0000 7f015bcdc700 1 -- 172.21.15.16:0/2338323826 >> 172.21.15.16:0/2338323826 conn(0x7f01540cffb0 msgr2=0x7f0154159690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:19.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.647+0000 7f015bcdc700 1 -- 172.21.15.16:0/2338323826 shutdown_connections 2023-05-20T01:28:19.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.647+0000 7f015bcdc700 1 -- 172.21.15.16:0/2338323826 wait complete. 2023-05-20T01:28:19.664 INFO:tasks.cephfs_test_runner:test_extra_data_pool_stashed_layout (tasks.cephfs.test_data_scan.TestDataScan) ... ok 2023-05-20T01:28:19.665 INFO:tasks.cephfs_test_runner:Starting test: test_fragmented_injection (tasks.cephfs.test_data_scan.TestDataScan) 2023-05-20T01:28:19.665 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_data_scan.TestDataScan.test_fragmented_injection' 2023-05-20T01:28:19.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.787+0000 7f1271f65700 1 Processor -- start 2023-05-20T01:28:19.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.787+0000 7f1271f65700 1 -- start start 2023-05-20T01:28:19.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.788+0000 7f1271f65700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f126c06a510 0x7f126c15e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:19.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.788+0000 7f1271f65700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f126c06a8e0 0x7f126c0cf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:19.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.788+0000 7f1271f65700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f126c15e060 con 0x7f126c0cfb30 2023-05-20T01:28:19.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.788+0000 7f1271f65700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f126c15d450 con 0x7f126c06a8e0 2023-05-20T01:28:19.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.788+0000 7f1271f65700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f126c15d730 con 0x7f126c06a510 2023-05-20T01:28:19.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.788+0000 7f126affd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f126c06a8e0 0x7f126c0cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:19.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.788+0000 7f126affd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f126c06a8e0 0x7f126c0cf5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39808/0 (socket says 172.21.15.16:39808) 2023-05-20T01:28:19.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.788+0000 7f126affd700 1 -- 172.21.15.16:0/4090382652 learned_addr learned my addr 172.21.15.16:0/4090382652 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:19.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.788+0000 7f126b7fe700 1 --2- 172.21.15.16:0/4090382652 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f126c06a510 0x7f126c15e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:19.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.788+0000 7f126affd700 1 -- 172.21.15.16:0/4090382652 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f126c06a510 msgr2=0x7f126c15e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:19.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.788+0000 7f126affd700 1 --2- 172.21.15.16:0/4090382652 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f126c06a510 0x7f126c15e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:19.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.788+0000 7f126affd700 1 -- 172.21.15.16:0/4090382652 >> v1:172.21.15.132:6789/0 conn(0x7f126c0cfb30 legacy=0x7f126c071340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:19.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.788+0000 7f126affd700 1 -- 172.21.15.16:0/4090382652 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f126c15da10 con 0x7f126c06a8e0 2023-05-20T01:28:19.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.788+0000 7f126affd700 1 --2- 172.21.15.16:0/4090382652 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f126c06a8e0 0x7f126c0cf5f0 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f125c0098f0 tx=0x7f125c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c3f715f615c6bcc server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:19.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.788+0000 7f126a7fc700 1 -- 172.21.15.16:0/4090382652 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f125c004230 con 0x7f126c06a8e0 2023-05-20T01:28:19.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.789+0000 7f126a7fc700 1 -- 172.21.15.16:0/4090382652 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f125c004390 con 0x7f126c06a8e0 2023-05-20T01:28:19.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.789+0000 7f126a7fc700 1 -- 172.21.15.16:0/4090382652 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f125c0036a0 con 0x7f126c06a8e0 2023-05-20T01:28:19.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.789+0000 7f1271f65700 1 -- 172.21.15.16:0/4090382652 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f126c06a8e0 msgr2=0x7f126c0cf5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:19.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.789+0000 7f1271f65700 1 --2- 172.21.15.16:0/4090382652 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f126c06a8e0 0x7f126c0cf5f0 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f125c0098f0 tx=0x7f125c009c00 comp rx=0 tx=0).stop 2023-05-20T01:28:19.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.789+0000 7f1271f65700 1 -- 172.21.15.16:0/4090382652 shutdown_connections 2023-05-20T01:28:19.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.789+0000 7f1271f65700 1 --2- 172.21.15.16:0/4090382652 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f126c06a510 0x7f126c15e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:19.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.789+0000 7f1271f65700 1 --2- 172.21.15.16:0/4090382652 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f126c06a8e0 0x7f126c0cf5f0 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:19.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.789+0000 7f1271f65700 1 -- 172.21.15.16:0/4090382652 >> 172.21.15.16:0/4090382652 conn(0x7f126c0d0c40 msgr2=0x7f126c0b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:19.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.789+0000 7f1271f65700 1 -- 172.21.15.16:0/4090382652 shutdown_connections 2023-05-20T01:28:19.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.789+0000 7f1271f65700 1 -- 172.21.15.16:0/4090382652 wait complete. 2023-05-20T01:28:19.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.790+0000 7f1271f65700 1 Processor -- start 2023-05-20T01:28:19.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.791+0000 7f1271f65700 1 -- start start 2023-05-20T01:28:19.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.791+0000 7f1271f65700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f126c06a510 0x7f126c152240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:19.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.791+0000 7f1271f65700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f126c06a8e0 0x7f126c0c9700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:19.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.791+0000 7f1271f65700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f126c0cfb30 0x7f126c0c9c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:19.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.791+0000 7f1271f65700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f126c0ca180 con 0x7f126c06a8e0 2023-05-20T01:28:19.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.791+0000 7f1271f65700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f126c0b5580 con 0x7f126c06a510 2023-05-20T01:28:19.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.791+0000 7f1271f65700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f126c0b5860 con 0x7f126c0cfb30 2023-05-20T01:28:19.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.791+0000 7f126affd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f126c06a8e0 0x7f126c0c9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:19.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.791+0000 7f126affd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f126c06a8e0 0x7f126c0c9700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39816/0 (socket says 172.21.15.16:39816) 2023-05-20T01:28:19.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.791+0000 7f126affd700 1 -- 172.21.15.16:0/1380860300 learned_addr learned my addr 172.21.15.16:0/1380860300 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:19.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.792+0000 7f126bfff700 1 --2- 172.21.15.16:0/1380860300 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f126c0cfb30 0x7f126c0c9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:19.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.792+0000 7f126b7fe700 1 --2- 172.21.15.16:0/1380860300 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f126c06a510 0x7f126c152240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:19.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.792+0000 7f126affd700 1 -- 172.21.15.16:0/1380860300 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f126c0cfb30 msgr2=0x7f126c0c9c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:19.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.792+0000 7f126affd700 1 --2- 172.21.15.16:0/1380860300 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f126c0cfb30 0x7f126c0c9c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:19.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.792+0000 7f126affd700 1 -- 172.21.15.16:0/1380860300 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f126c06a510 msgr2=0x7f126c152240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:19.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.792+0000 7f126affd700 1 --2- 172.21.15.16:0/1380860300 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f126c06a510 0x7f126c152240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:19.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.792+0000 7f126affd700 1 -- 172.21.15.16:0/1380860300 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f125c009550 con 0x7f126c06a8e0 2023-05-20T01:28:19.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.792+0000 7f126affd700 1 --2- 172.21.15.16:0/1380860300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f126c06a8e0 0x7f126c0c9700 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f126c066b10 tx=0x7f125c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:19.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.792+0000 7f1263fff700 1 -- 172.21.15.16:0/1380860300 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f125c003b60 con 0x7f126c06a8e0 2023-05-20T01:28:19.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.792+0000 7f1263fff700 1 -- 172.21.15.16:0/1380860300 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f125c01f070 con 0x7f126c06a8e0 2023-05-20T01:28:19.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.792+0000 7f1271f65700 1 -- 172.21.15.16:0/1380860300 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f126c06ad50 con 0x7f126c06a8e0 2023-05-20T01:28:19.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.792+0000 7f1263fff700 1 -- 172.21.15.16:0/1380860300 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f125c00b660 con 0x7f126c06a8e0 2023-05-20T01:28:19.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.792+0000 7f1271f65700 1 -- 172.21.15.16:0/1380860300 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f126c15dcf0 con 0x7f126c06a8e0 2023-05-20T01:28:19.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.793+0000 7f1271f65700 1 -- 172.21.15.16:0/1380860300 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f126c15d080 con 0x7f126c06a8e0 2023-05-20T01:28:19.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.794+0000 7f1263fff700 1 -- 172.21.15.16:0/1380860300 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f125c021030 con 0x7f126c06a8e0 2023-05-20T01:28:19.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.794+0000 7f1263fff700 1 --2- 172.21.15.16:0/1380860300 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f124c06ace0 0x7f124c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:19.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.794+0000 7f1263fff700 1 -- 172.21.15.16:0/1380860300 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5874+0+0 (secure 0 0 0) 0x7f125c08a850 con 0x7f126c06a8e0 2023-05-20T01:28:19.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.794+0000 7f126b7fe700 1 --2- 172.21.15.16:0/1380860300 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f124c06ace0 0x7f124c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:19.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.795+0000 7f126b7fe700 1 --2- 172.21.15.16:0/1380860300 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f124c06ace0 0x7f124c06d190 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f1254006b20 tx=0x7f1254008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:19.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:19.799+0000 7f1263fff700 1 -- 172.21.15.16:0/1380860300 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f125c055b60 con 0x7f126c06a8e0 2023-05-20T01:28:20.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:20.082+0000 7f1271f65700 1 -- 172.21.15.16:0/1380860300 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_data_scan.TestDataScan.test_fragmented_injection"]} v 0) v1 -- 0x7f126c15d080 con 0x7f126c06a8e0 2023-05-20T01:28:20.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:20.647+0000 7f1263fff700 1 -- 172.21.15.16:0/1380860300 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_data_scan.TestDataScan.test_fragmented_injection"]}]=0 v509) v1 ==== 148+0+0 (secure 0 0 0) 0x7f125c059180 con 0x7f126c06a8e0 2023-05-20T01:28:20.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:20.650+0000 7f1271f65700 1 -- 172.21.15.16:0/1380860300 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f124c06ace0 msgr2=0x7f124c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:20.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:20.650+0000 7f1271f65700 1 --2- 172.21.15.16:0/1380860300 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f124c06ace0 0x7f124c06d190 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f1254006b20 tx=0x7f1254008040 comp rx=0 tx=0).stop 2023-05-20T01:28:20.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:20.650+0000 7f1271f65700 1 -- 172.21.15.16:0/1380860300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f126c06a8e0 msgr2=0x7f126c0c9700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:20.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:20.650+0000 7f1271f65700 1 --2- 172.21.15.16:0/1380860300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f126c06a8e0 0x7f126c0c9700 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f126c066b10 tx=0x7f125c004750 comp rx=0 tx=0).stop 2023-05-20T01:28:20.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:20.650+0000 7f1271f65700 1 -- 172.21.15.16:0/1380860300 shutdown_connections 2023-05-20T01:28:20.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:20.650+0000 7f1271f65700 1 --2- 172.21.15.16:0/1380860300 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f124c06ace0 0x7f124c06d190 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:20.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:20.650+0000 7f1271f65700 1 --2- 172.21.15.16:0/1380860300 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f126c0cfb30 0x7f126c0c9c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:20.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:20.650+0000 7f1271f65700 1 --2- 172.21.15.16:0/1380860300 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f126c06a510 0x7f126c152240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:20.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:20.650+0000 7f1271f65700 1 --2- 172.21.15.16:0/1380860300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f126c06a8e0 0x7f126c0c9700 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:20.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:20.650+0000 7f1271f65700 1 -- 172.21.15.16:0/1380860300 >> 172.21.15.16:0/1380860300 conn(0x7f126c0d0c40 msgr2=0x7f126c0cc0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:20.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:20.651+0000 7f1271f65700 1 -- 172.21.15.16:0/1380860300 shutdown_connections 2023-05-20T01:28:20.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:20.651+0000 7f1271f65700 1 -- 172.21.15.16:0/1380860300 wait complete. 2023-05-20T01:28:20.670 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-05-20T01:28:20.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:20.801+0000 7f31f45a0700 1 Processor -- start 2023-05-20T01:28:20.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:20.801+0000 7f31f45a0700 1 -- start start 2023-05-20T01:28:20.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:20.802+0000 7f31f45a0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f31ec15cec0 0x7f31ec071bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:20.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:20.802+0000 7f31f45a0700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31ec072590 con 0x7f31ec15d290 2023-05-20T01:28:20.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:20.802+0000 7f31f45a0700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31ec0cd630 con 0x7f31ec072130 2023-05-20T01:28:20.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:20.802+0000 7f31f45a0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31ec0cca60 con 0x7f31ec15cec0 2023-05-20T01:28:20.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:20.802+0000 7f31f1b3b700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f31ec15d290 0x7f31ec0cf2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51348/0 (socket says 172.21.15.16:51348) 2023-05-20T01:28:20.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:20.802+0000 7f31f1b3b700 1 -- 172.21.15.16:0/584607356 learned_addr learned my addr 172.21.15.16:0/584607356 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:28:20.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:20.803+0000 7f31f233c700 1 --2- 172.21.15.16:0/584607356 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f31ec15cec0 0x7f31ec071bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:20.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:20.803+0000 7f31f133a700 1 -- 172.21.15.16:0/584607356 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3943150878 0 0) 0x7f31ec072590 con 0x7f31ec15d290 2023-05-20T01:28:20.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:20.803+0000 7f31f133a700 1 -- 172.21.15.16:0/584607356 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f31d80035b0 con 0x7f31ec15d290 2023-05-20T01:28:20.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:20.803+0000 7f31f133a700 1 -- 172.21.15.16:0/584607356 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3672594285 0 0) 0x7f31d80035b0 con 0x7f31ec15d290 2023-05-20T01:28:20.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:20.803+0000 7f31f133a700 1 -- 172.21.15.16:0/584607356 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f31ec15cec0 msgr2=0x7f31ec071bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:20.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:20.803+0000 7f31f133a700 1 --2- 172.21.15.16:0/584607356 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f31ec15cec0 0x7f31ec071bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:20.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:20.803+0000 7f31f133a700 1 -- 172.21.15.16:0/584607356 >> v1:172.21.15.132:6789/0 conn(0x7f31ec072130 legacy=0x7f31ec160890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:20.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:20.803+0000 7f31f133a700 1 -- 172.21.15.16:0/584607356 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31ec0ccd00 con 0x7f31ec15d290 2023-05-20T01:28:20.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:20.803+0000 7f31f133a700 1 -- 172.21.15.16:0/584607356 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7f31dc001670 con 0x7f31ec15d290 2023-05-20T01:28:20.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:20.803+0000 7f31f133a700 1 -- 172.21.15.16:0/584607356 >> v1:172.21.15.16:6789/0 conn(0x7f31ec15d290 legacy=0x7f31ec0cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:20.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:20.803+0000 7f31f133a700 1 --2- 172.21.15.16:0/584607356 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f31d8003a30 0x7f31d8003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:20.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:20.803+0000 7f31f133a700 1 --2- 172.21.15.16:0/584607356 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31d8004850 0x7f31d8004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:20.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:20.803+0000 7f31f133a700 1 --2- 172.21.15.16:0/584607356 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f31ec15cec0 0x7f31d8005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:20.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:20.803+0000 7f31f133a700 1 -- 172.21.15.16:0/584607356 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31ec0cca60 con 0x7f31d8004850 2023-05-20T01:28:20.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:20.803+0000 7f31f133a700 1 -- 172.21.15.16:0/584607356 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31d800bb10 con 0x7f31d8003a30 2023-05-20T01:28:20.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:20.803+0000 7f31f133a700 1 -- 172.21.15.16:0/584607356 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31d800bcd0 con 0x7f31ec15cec0 2023-05-20T01:28:20.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:20.803+0000 7f31f1b3b700 1 --2- 172.21.15.16:0/584607356 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31d8004850 0x7f31d8004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:20.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:20.803+0000 7f31f233c700 1 --2- 172.21.15.16:0/584607356 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f31d8003a30 0x7f31d8003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:20.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:20.803+0000 7f31f2b3d700 1 --2- 172.21.15.16:0/584607356 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f31ec15cec0 0x7f31d8005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:20.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:20.803+0000 7f31f1b3b700 1 -- 172.21.15.16:0/584607356 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f31ec15cec0 msgr2=0x7f31d8005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:20.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:20.803+0000 7f31f1b3b700 1 --2- 172.21.15.16:0/584607356 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f31ec15cec0 0x7f31d8005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:20.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:20.803+0000 7f31f1b3b700 1 -- 172.21.15.16:0/584607356 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f31d8003a30 msgr2=0x7f31d8003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:20.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:20.803+0000 7f31f1b3b700 1 --2- 172.21.15.16:0/584607356 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f31d8003a30 0x7f31d8003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:20.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:20.803+0000 7f31f1b3b700 1 -- 172.21.15.16:0/584607356 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31d80035b0 con 0x7f31d8004850 2023-05-20T01:28:20.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:20.804+0000 7f31f1b3b700 1 --2- 172.21.15.16:0/584607356 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31d8004850 0x7f31d8004cc0 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f31dc00c240 tx=0x7f31d8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e045e5ca98ea764 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:20.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:20.804+0000 7f31f133a700 1 -- 172.21.15.16:0/584607356 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31dc007c10 con 0x7f31d8004850 2023-05-20T01:28:20.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:20.804+0000 7f31f133a700 1 -- 172.21.15.16:0/584607356 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31dc00cde0 con 0x7f31d8004850 2023-05-20T01:28:20.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:20.804+0000 7f31f133a700 1 -- 172.21.15.16:0/584607356 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31dc005ba0 con 0x7f31d8004850 2023-05-20T01:28:20.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:20.804+0000 7f31f45a0700 1 -- 172.21.15.16:0/584607356 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31d8004850 msgr2=0x7f31d8004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:20.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:20.804+0000 7f31f45a0700 1 --2- 172.21.15.16:0/584607356 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31d8004850 0x7f31d8004cc0 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f31dc00c240 tx=0x7f31d8005880 comp rx=0 tx=0).stop 2023-05-20T01:28:20.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:20.804+0000 7f31f2b3d700 1 -- 172.21.15.16:0/584607356 reap_dead start 2023-05-20T01:28:20.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:20.804+0000 7f31f45a0700 1 -- 172.21.15.16:0/584607356 shutdown_connections 2023-05-20T01:28:20.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:20.804+0000 7f31f45a0700 1 -- 172.21.15.16:0/584607356 >> 172.21.15.16:0/584607356 conn(0x7f31ec0cffd0 msgr2=0x7f31ec0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:20.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:20.804+0000 7f31f45a0700 1 -- 172.21.15.16:0/584607356 shutdown_connections 2023-05-20T01:28:20.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:20.804+0000 7f31f45a0700 1 -- 172.21.15.16:0/584607356 wait complete. 2023-05-20T01:28:20.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:20.806+0000 7f31f45a0700 1 Processor -- start 2023-05-20T01:28:20.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:20.806+0000 7f31f45a0700 1 -- start start 2023-05-20T01:28:20.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:20.806+0000 7f31f45a0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f31d8004850 0x7f31ec0b1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:20.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:20.806+0000 7f31f45a0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f31ec1f5e00 0x7f31ec152490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:20.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:20.806+0000 7f31f45a0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31ec0b0270 0x7f31ec0b06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:20.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:20.806+0000 7f31f45a0700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31ec0afab0 con 0x7f31ec0b0270 2023-05-20T01:28:20.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:20.806+0000 7f31f45a0700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31ec1f65e0 con 0x7f31ec1f5e00 2023-05-20T01:28:20.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:20.806+0000 7f31f45a0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31ec1f68c0 con 0x7f31d8004850 2023-05-20T01:28:20.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:20.806+0000 7f31f2b3d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31ec0b0270 0x7f31ec0b06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:20.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:20.806+0000 7f31f2b3d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31ec0b0270 0x7f31ec0b06d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39836/0 (socket says 172.21.15.16:39836) 2023-05-20T01:28:20.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:20.806+0000 7f31f2b3d700 1 -- 172.21.15.16:0/1691183302 learned_addr learned my addr 172.21.15.16:0/1691183302 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:20.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:20.807+0000 7f31f1b3b700 1 --2- 172.21.15.16:0/1691183302 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f31ec1f5e00 0x7f31ec152490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:20.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:20.807+0000 7f31f233c700 1 --2- 172.21.15.16:0/1691183302 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f31d8004850 0x7f31ec0b1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:20.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:20.807+0000 7f31f2b3d700 1 -- 172.21.15.16:0/1691183302 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f31d8004850 msgr2=0x7f31ec0b1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:20.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:20.807+0000 7f31f2b3d700 1 --2- 172.21.15.16:0/1691183302 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f31d8004850 0x7f31ec0b1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:20.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:20.807+0000 7f31f2b3d700 1 -- 172.21.15.16:0/1691183302 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f31ec1f5e00 msgr2=0x7f31ec152490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:20.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:20.807+0000 7f31f2b3d700 1 --2- 172.21.15.16:0/1691183302 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f31ec1f5e00 0x7f31ec152490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:20.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:20.807+0000 7f31f2b3d700 1 -- 172.21.15.16:0/1691183302 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31ec0cd2c0 con 0x7f31ec0b0270 2023-05-20T01:28:20.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:20.807+0000 7f31f2b3d700 1 --2- 172.21.15.16:0/1691183302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31ec0b0270 0x7f31ec0b06d0 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f31ec072cf0 tx=0x7f31e400d150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:20.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:20.807+0000 7f31e2ffd700 1 -- 172.21.15.16:0/1691183302 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e4003c10 con 0x7f31ec0b0270 2023-05-20T01:28:20.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:20.807+0000 7f31e2ffd700 1 -- 172.21.15.16:0/1691183302 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31e4003d90 con 0x7f31ec0b0270 2023-05-20T01:28:20.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:20.807+0000 7f31f45a0700 1 -- 172.21.15.16:0/1691183302 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31ec0ccfe0 con 0x7f31ec0b0270 2023-05-20T01:28:20.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:20.807+0000 7f31e2ffd700 1 -- 172.21.15.16:0/1691183302 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e400d950 con 0x7f31ec0b0270 2023-05-20T01:28:20.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:20.807+0000 7f31f45a0700 1 -- 172.21.15.16:0/1691183302 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31ec071bf0 con 0x7f31ec0b0270 2023-05-20T01:28:20.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:20.808+0000 7f31f45a0700 1 -- 172.21.15.16:0/1691183302 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31ec0b00c0 con 0x7f31ec0b0270 2023-05-20T01:28:20.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:20.809+0000 7f31e2ffd700 1 -- 172.21.15.16:0/1691183302 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f31e400dab0 con 0x7f31ec0b0270 2023-05-20T01:28:20.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:20.809+0000 7f31e2ffd700 1 --2- 172.21.15.16:0/1691183302 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f31d006ace0 0x7f31d006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:20.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:20.809+0000 7f31e2ffd700 1 -- 172.21.15.16:0/1691183302 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5874+0+0 (secure 0 0 0) 0x7f31e40894e0 con 0x7f31ec0b0270 2023-05-20T01:28:20.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:20.809+0000 7f31f233c700 1 --2- 172.21.15.16:0/1691183302 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f31d006ace0 0x7f31d006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:20.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:20.809+0000 7f31f233c700 1 --2- 172.21.15.16:0/1691183302 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f31d006ace0 0x7f31d006d190 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f31e8009c40 tx=0x7f31e80091c0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:20.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:20.813+0000 7f31e2ffd700 1 -- 172.21.15.16:0/1691183302 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f31e4054640 con 0x7f31ec0b0270 2023-05-20T01:28:21.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.010+0000 7f31f45a0700 1 -- 172.21.15.16:0/1691183302 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f31ec0747b0 con 0x7f31ec0b0270 2023-05-20T01:28:21.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.014+0000 7f31e2ffd700 1 -- 172.21.15.16:0/1691183302 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) v1 ==== 122+0+0 (secure 0 0 0) 0x7f31e4057c60 con 0x7f31ec0b0270 2023-05-20T01:28:21.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.016+0000 7f31f45a0700 1 -- 172.21.15.16:0/1691183302 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f31d006ace0 msgr2=0x7f31d006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:21.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.016+0000 7f31f45a0700 1 --2- 172.21.15.16:0/1691183302 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f31d006ace0 0x7f31d006d190 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f31e8009c40 tx=0x7f31e80091c0 comp rx=0 tx=0).stop 2023-05-20T01:28:21.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.016+0000 7f31f45a0700 1 -- 172.21.15.16:0/1691183302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31ec0b0270 msgr2=0x7f31ec0b06d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:21.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.016+0000 7f31f45a0700 1 --2- 172.21.15.16:0/1691183302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31ec0b0270 0x7f31ec0b06d0 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f31ec072cf0 tx=0x7f31e400d150 comp rx=0 tx=0).stop 2023-05-20T01:28:21.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.016+0000 7f31f45a0700 1 -- 172.21.15.16:0/1691183302 shutdown_connections 2023-05-20T01:28:21.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.016+0000 7f31f45a0700 1 --2- 172.21.15.16:0/1691183302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31ec0b0270 0x7f31ec0b06d0 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:21.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.016+0000 7f31f45a0700 1 --2- 172.21.15.16:0/1691183302 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f31d006ace0 0x7f31d006d190 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:21.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.016+0000 7f31f45a0700 1 --2- 172.21.15.16:0/1691183302 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f31d8004850 0x7f31ec0b1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:21.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.016+0000 7f31f45a0700 1 --2- 172.21.15.16:0/1691183302 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f31ec1f5e00 0x7f31ec152490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:21.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.016+0000 7f31f45a0700 1 -- 172.21.15.16:0/1691183302 >> 172.21.15.16:0/1691183302 conn(0x7f31ec0cffd0 msgr2=0x7f31ec160020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:21.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.016+0000 7f31f45a0700 1 -- 172.21.15.16:0/1691183302 shutdown_connections 2023-05-20T01:28:21.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.016+0000 7f31f45a0700 1 -- 172.21.15.16:0/1691183302 wait complete. 2023-05-20T01:28:21.033 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-05-20T01:28:21.033 DEBUG:teuthology.orchestra.run.smithi016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-05-20T01:28:21.083 DEBUG:teuthology.orchestra.run.smithi132:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-05-20T01:28:21.126 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-05-20T01:28:21.127 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-05-20T01:28:21.127 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2023-05-20T01:28:21.146 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16287364k,nr_inodes=4071841,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=38,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16696 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3261172k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-05-20T01:28:21.147 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2023-05-20T01:28:21.147 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/proc/self/mounts of=/dev/stdout 2023-05-20T01:28:21.187 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284284k,nr_inodes=4071071,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=45,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16175 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260556k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-05-20T01:28:21.187 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-05-20T01:28:21.187 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2023-05-20T01:28:21.204 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16287364k,nr_inodes=4071841,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=38,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16696 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3261172k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-05-20T01:28:21.205 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2023-05-20T01:28:21.205 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/proc/self/mounts of=/dev/stdout 2023-05-20T01:28:21.247 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284284k,nr_inodes=4071071,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=45,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16175 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260556k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-05-20T01:28:21.248 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-05-20T01:28:21.248 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-20T01:28:21.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.369+0000 7fe40c390700 1 Processor -- start 2023-05-20T01:28:21.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.369+0000 7fe40c390700 1 -- start start 2023-05-20T01:28:21.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.370+0000 7fe40c390700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe404144020 0x7fe4040d1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:21.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.370+0000 7fe40c390700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fe4040cec60 0x7fe4040cf0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:21.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.370+0000 7fe40c390700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4040cf600 con 0x7fe4040b2760 2023-05-20T01:28:21.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.370+0000 7fe40c390700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4040cb6e0 con 0x7fe404144020 2023-05-20T01:28:21.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.370+0000 7fe40c390700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4041448a0 con 0x7fe4040cec60 2023-05-20T01:28:21.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.370+0000 7fe40992b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe404144020 0x7fe4040d1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:21.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.370+0000 7fe40992b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe404144020 0x7fe4040d1680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39842/0 (socket says 172.21.15.16:39842) 2023-05-20T01:28:21.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.370+0000 7fe40992b700 1 -- 172.21.15.16:0/2042859267 learned_addr learned my addr 172.21.15.16:0/2042859267 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:21.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.371+0000 7fe40a92d700 1 --2- 172.21.15.16:0/2042859267 >> v2:172.21.15.132:3300/0 conn(0x7fe4040cec60 0x7fe4040cf0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:21.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.371+0000 7fe40992b700 1 -- 172.21.15.16:0/2042859267 >> v2:172.21.15.132:3300/0 conn(0x7fe4040cec60 msgr2=0x7fe4040cf0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:21.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.371+0000 7fe40992b700 1 --2- 172.21.15.16:0/2042859267 >> v2:172.21.15.132:3300/0 conn(0x7fe4040cec60 0x7fe4040cf0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:21.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.371+0000 7fe40992b700 1 -- 172.21.15.16:0/2042859267 >> v1:172.21.15.132:6789/0 conn(0x7fe4040b2760 legacy=0x7fe4040b0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:21.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.371+0000 7fe40992b700 1 -- 172.21.15.16:0/2042859267 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4040cc500 con 0x7fe404144020 2023-05-20T01:28:21.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.371+0000 7fe40992b700 1 --2- 172.21.15.16:0/2042859267 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe404144020 0x7fe4040d1680 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7fe404073b10 tx=0x7fe400009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=203d5613b4c78081 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:21.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.371+0000 7fe40912a700 1 -- 172.21.15.16:0/2042859267 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe400004230 con 0x7fe404144020 2023-05-20T01:28:21.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.371+0000 7fe40912a700 1 -- 172.21.15.16:0/2042859267 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe400004390 con 0x7fe404144020 2023-05-20T01:28:21.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.371+0000 7fe40912a700 1 -- 172.21.15.16:0/2042859267 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4000036a0 con 0x7fe404144020 2023-05-20T01:28:21.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.371+0000 7fe40c390700 1 -- 172.21.15.16:0/2042859267 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe404144020 msgr2=0x7fe4040d1680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:21.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.371+0000 7fe40c390700 1 --2- 172.21.15.16:0/2042859267 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe404144020 0x7fe4040d1680 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7fe404073b10 tx=0x7fe400009bd0 comp rx=0 tx=0).stop 2023-05-20T01:28:21.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.372+0000 7fe40c390700 1 -- 172.21.15.16:0/2042859267 shutdown_connections 2023-05-20T01:28:21.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.372+0000 7fe40c390700 1 --2- 172.21.15.16:0/2042859267 >> v2:172.21.15.132:3300/0 conn(0x7fe4040cec60 0x7fe4040cf0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:21.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.372+0000 7fe40c390700 1 --2- 172.21.15.16:0/2042859267 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe404144020 0x7fe4040d1680 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:21.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.372+0000 7fe40c390700 1 -- 172.21.15.16:0/2042859267 >> 172.21.15.16:0/2042859267 conn(0x7fe40406ae30 msgr2=0x7fe40409a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:21.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.372+0000 7fe40c390700 1 -- 172.21.15.16:0/2042859267 shutdown_connections 2023-05-20T01:28:21.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.372+0000 7fe40c390700 1 -- 172.21.15.16:0/2042859267 wait complete. 2023-05-20T01:28:21.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.373+0000 7fe40c390700 1 Processor -- start 2023-05-20T01:28:21.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.373+0000 7fe40c390700 1 -- start start 2023-05-20T01:28:21.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.374+0000 7fe40c390700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe4040b2760 0x7fe4040b60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:21.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.374+0000 7fe40c390700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4040cec60 0x7fe4040b66b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:21.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.374+0000 7fe40c390700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe404144020 0x7fe4040cdcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:21.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.374+0000 7fe40c390700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4040b5cf0 con 0x7fe4040cec60 2023-05-20T01:28:21.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.374+0000 7fe40c390700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4040b50b0 con 0x7fe404144020 2023-05-20T01:28:21.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.374+0000 7fe40c390700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4040b5350 con 0x7fe4040b2760 2023-05-20T01:28:21.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.374+0000 7fe40992b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4040cec60 0x7fe4040b66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:21.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.374+0000 7fe40992b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4040cec60 0x7fe4040b66b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39848/0 (socket says 172.21.15.16:39848) 2023-05-20T01:28:21.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.374+0000 7fe40992b700 1 -- 172.21.15.16:0/2377291196 learned_addr learned my addr 172.21.15.16:0/2377291196 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:21.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.375+0000 7fe40a92d700 1 --2- 172.21.15.16:0/2377291196 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe404144020 0x7fe4040cdcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:21.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.375+0000 7fe40a12c700 1 --2- 172.21.15.16:0/2377291196 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe4040b2760 0x7fe4040b60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:21.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.375+0000 7fe40992b700 1 -- 172.21.15.16:0/2377291196 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe4040b2760 msgr2=0x7fe4040b60f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:21.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.375+0000 7fe40992b700 1 --2- 172.21.15.16:0/2377291196 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe4040b2760 0x7fe4040b60f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:21.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.375+0000 7fe40992b700 1 -- 172.21.15.16:0/2377291196 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe404144020 msgr2=0x7fe4040cdcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:21.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.375+0000 7fe40992b700 1 --2- 172.21.15.16:0/2377291196 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe404144020 0x7fe4040cdcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:21.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.375+0000 7fe40992b700 1 -- 172.21.15.16:0/2377291196 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe400009550 con 0x7fe4040cec60 2023-05-20T01:28:21.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.375+0000 7fe40a92d700 1 --2- 172.21.15.16:0/2377291196 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe404144020 0x7fe4040cdcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:28:21.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.375+0000 7fe40992b700 1 --2- 172.21.15.16:0/2377291196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4040cec60 0x7fe4040b66b0 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7fe400009130 tx=0x7fe400004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:21.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.375+0000 7fe3faffd700 1 -- 172.21.15.16:0/2377291196 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe400011630 con 0x7fe4040cec60 2023-05-20T01:28:21.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.375+0000 7fe3faffd700 1 -- 172.21.15.16:0/2377291196 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe400011790 con 0x7fe4040cec60 2023-05-20T01:28:21.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.375+0000 7fe3faffd700 1 -- 172.21.15.16:0/2377291196 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe400011b00 con 0x7fe4040cec60 2023-05-20T01:28:21.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.375+0000 7fe40c390700 1 -- 172.21.15.16:0/2377291196 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4040cbbd0 con 0x7fe4040cec60 2023-05-20T01:28:21.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.375+0000 7fe40c390700 1 -- 172.21.15.16:0/2377291196 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4040cb8f0 con 0x7fe4040cec60 2023-05-20T01:28:21.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.376+0000 7fe40c390700 1 -- 172.21.15.16:0/2377291196 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4040cbeb0 con 0x7fe4040cec60 2023-05-20T01:28:21.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.377+0000 7fe3faffd700 1 -- 172.21.15.16:0/2377291196 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe400021070 con 0x7fe4040cec60 2023-05-20T01:28:21.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.377+0000 7fe3faffd700 1 --2- 172.21.15.16:0/2377291196 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe3e806ace0 0x7fe3e806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:21.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.377+0000 7fe3faffd700 1 -- 172.21.15.16:0/2377291196 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5874+0+0 (secure 0 0 0) 0x7fe40008aa80 con 0x7fe4040cec60 2023-05-20T01:28:21.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.377+0000 7fe40a12c700 1 --2- 172.21.15.16:0/2377291196 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe3e806ace0 0x7fe3e806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:21.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.377+0000 7fe40a12c700 1 --2- 172.21.15.16:0/2377291196 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe3e806ace0 0x7fe3e806d190 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fe3f4009870 tx=0x7fe3f4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:21.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.381+0000 7fe3faffd700 1 -- 172.21.15.16:0/2377291196 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fe400055c70 con 0x7fe4040cec60 2023-05-20T01:28:21.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.627+0000 7fe40c390700 1 -- 172.21.15.16:0/2377291196 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe4040747b0 con 0x7fe4040cec60 2023-05-20T01:28:21.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.628+0000 7fe3faffd700 1 -- 172.21.15.16:0/2377291196 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) v1 ==== 94+0+3529 (secure 0 0 0) 0x7fe400028de0 con 0x7fe4040cec60 2023-05-20T01:28:21.632 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:28:21.632 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":93,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6845,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/1435235487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1435235487},{"type":"v1","addr":"172.21.15.132:6834","nonce":1435235487}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":87},{"gid":6848,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/933893285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":933893285},{"type":"v1","addr":"172.21.15.132:6835","nonce":933893285}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":87},{"gid":8373,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/1220815342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1220815342},{"type":"v1","addr":"172.21.15.16:6836","nonce":1220815342}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":87},{"gid":8631,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/726740923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":726740923},{"type":"v1","addr":"172.21.15.16:6837","nonce":726740923}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":93}],"filesystems":[]} 2023-05-20T01:28:21.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.630+0000 7fe40c390700 1 -- 172.21.15.16:0/2377291196 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe3e806ace0 msgr2=0x7fe3e806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:21.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.630+0000 7fe40c390700 1 --2- 172.21.15.16:0/2377291196 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe3e806ace0 0x7fe3e806d190 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fe3f4009870 tx=0x7fe3f4008040 comp rx=0 tx=0).stop 2023-05-20T01:28:21.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.630+0000 7fe40c390700 1 -- 172.21.15.16:0/2377291196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4040cec60 msgr2=0x7fe4040b66b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:21.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.630+0000 7fe40c390700 1 --2- 172.21.15.16:0/2377291196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4040cec60 0x7fe4040b66b0 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7fe400009130 tx=0x7fe400004750 comp rx=0 tx=0).stop 2023-05-20T01:28:21.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.630+0000 7fe40c390700 1 -- 172.21.15.16:0/2377291196 shutdown_connections 2023-05-20T01:28:21.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.630+0000 7fe40c390700 1 --2- 172.21.15.16:0/2377291196 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe404144020 0x7fe4040cdcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:21.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.630+0000 7fe40c390700 1 --2- 172.21.15.16:0/2377291196 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fe3e806ace0 0x7fe3e806d190 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:21.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.630+0000 7fe40c390700 1 --2- 172.21.15.16:0/2377291196 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe4040b2760 0x7fe4040b60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:21.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.630+0000 7fe40c390700 1 --2- 172.21.15.16:0/2377291196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4040cec60 0x7fe4040b66b0 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:21.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.630+0000 7fe40c390700 1 -- 172.21.15.16:0/2377291196 >> 172.21.15.16:0/2377291196 conn(0x7fe40406ae30 msgr2=0x7fe4040b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:21.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.630+0000 7fe40c390700 1 -- 172.21.15.16:0/2377291196 shutdown_connections 2023-05-20T01:28:21.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.630+0000 7fe40c390700 1 -- 172.21.15.16:0/2377291196 wait complete. 2023-05-20T01:28:21.636 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 93 2023-05-20T01:28:21.647 INFO:tasks.ceph.mds.a:Restarting daemon 2023-05-20T01:28:21.647 INFO:tasks.ceph.mds.a:Stopping old one... 2023-05-20T01:28:21.648 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-05-20T01:28:21.649 INFO:teuthology.orchestra.run:waiting for 300 2023-05-20T01:28:21.649 INFO:tasks.ceph.mds.c:Restarting daemon 2023-05-20T01:28:21.649 INFO:tasks.ceph.mds.c:Stopping old one... 2023-05-20T01:28:21.649 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-05-20T01:28:21.650 INFO:teuthology.orchestra.run:waiting for 300 2023-05-20T01:28:21.650 INFO:tasks.ceph.mds.b:Restarting daemon 2023-05-20T01:28:21.650 INFO:tasks.ceph.mds.b:Stopping old one... 2023-05-20T01:28:21.650 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-05-20T01:28:21.651 INFO:teuthology.orchestra.run:waiting for 300 2023-05-20T01:28:21.651 INFO:tasks.ceph.mds.d:Restarting daemon 2023-05-20T01:28:21.651 INFO:tasks.ceph.mds.d:Stopping old one... 2023-05-20T01:28:21.652 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-05-20T01:28:21.652 INFO:teuthology.orchestra.run:waiting for 300 2023-05-20T01:28:21.659 INFO:tasks.ceph.mds.a:Stopped 2023-05-20T01:28:21.660 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-05-20T01:28:21.662 INFO:tasks.ceph.mds.b:Stopped 2023-05-20T01:28:21.663 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-05-20T01:28:21.664 INFO:tasks.ceph.mds.c:Stopped 2023-05-20T01:28:21.664 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-05-20T01:28:21.665 INFO:tasks.ceph.mds.d:Stopped 2023-05-20T01:28:21.666 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-05-20T01:28:21.703 INFO:tasks.ceph.mds.a:Started 2023-05-20T01:28:21.704 DEBUG:teuthology.parallel:result is None 2023-05-20T01:28:21.705 INFO:tasks.ceph.mds.b:Started 2023-05-20T01:28:21.705 INFO:tasks.ceph.mds.d:Started 2023-05-20T01:28:21.705 DEBUG:teuthology.parallel:result is None 2023-05-20T01:28:21.706 DEBUG:teuthology.parallel:result is None 2023-05-20T01:28:21.706 INFO:tasks.ceph.mds.c:Started 2023-05-20T01:28:21.707 DEBUG:teuthology.parallel:result is None 2023-05-20T01:28:21.707 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-05-20T01:28:21.786 INFO:tasks.ceph.mds.c.smithi016.stdout:starting mds.c at 2023-05-20T01:28:21.787 INFO:tasks.ceph.mds.a.smithi016.stdout:starting mds.a at 2023-05-20T01:28:21.787 INFO:tasks.ceph.mds.a.smithi016.stderr:2023-05-20T01:28:21.783+0000 7f90caed8700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2023-05-20T01:28:21.790 INFO:tasks.ceph.mds.b.smithi132.stdout:starting mds.b at 2023-05-20T01:28:21.791 INFO:tasks.ceph.mds.d.smithi132.stdout:starting mds.d at 2023-05-20T01:28:21.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.838+0000 7fa272a63700 1 Processor -- start 2023-05-20T01:28:21.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.838+0000 7fa272a63700 1 -- start start 2023-05-20T01:28:21.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.839+0000 7fa272a63700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa26c0cb670 0x7fa26c0ce340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:21.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.839+0000 7fa272a63700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa26c159d90 con 0x7fa26c0caaa0 2023-05-20T01:28:21.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.839+0000 7fa272a63700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa26c0cdb80 con 0x7fa26c1599c0 2023-05-20T01:28:21.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.839+0000 7fa272a63700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa26c0cdf20 con 0x7fa26c0cb670 2023-05-20T01:28:21.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.839+0000 7fa26b7fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa26c0cb670 0x7fa26c0ce340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:21.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.839+0000 7fa271000700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fa26c0caaa0 0x7fa26c0ce880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51410/0 (socket says 172.21.15.16:51410) 2023-05-20T01:28:21.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.839+0000 7fa271000700 1 -- 172.21.15.16:0/631832473 learned_addr learned my addr 172.21.15.16:0/631832473 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:28:21.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.839+0000 7fa26b7fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa26c0cb670 0x7fa26c0ce340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39880/0 (socket says 172.21.15.16:39880) 2023-05-20T01:28:21.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.839+0000 7fa26b7fe700 1 -- 172.21.15.16:0/631832473 >> v1:172.21.15.132:6789/0 conn(0x7fa26c1599c0 legacy=0x7fa26c0caf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:21.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.839+0000 7fa26b7fe700 1 -- 172.21.15.16:0/631832473 >> v1:172.21.15.16:6789/0 conn(0x7fa26c0caaa0 legacy=0x7fa26c0ce880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:21.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.839+0000 7fa26b7fe700 1 -- 172.21.15.16:0/631832473 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa26c06eaf0 con 0x7fa26c0cb670 2023-05-20T01:28:21.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.839+0000 7fa26b7fe700 1 --2- 172.21.15.16:0/631832473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa26c0cb670 0x7fa26c0ce340 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7fa260009840 tx=0x7fa260009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db1b85966b54123e server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:21.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.839+0000 7fa26affd700 1 -- 172.21.15.16:0/631832473 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa260004230 con 0x7fa26c0cb670 2023-05-20T01:28:21.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.839+0000 7fa26affd700 1 -- 172.21.15.16:0/631832473 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa260004390 con 0x7fa26c0cb670 2023-05-20T01:28:21.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.839+0000 7fa26affd700 1 -- 172.21.15.16:0/631832473 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2600036a0 con 0x7fa26c0cb670 2023-05-20T01:28:21.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.840+0000 7fa272a63700 1 -- 172.21.15.16:0/631832473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa26c0cb670 msgr2=0x7fa26c0ce340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:21.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.840+0000 7fa272a63700 1 --2- 172.21.15.16:0/631832473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa26c0cb670 0x7fa26c0ce340 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7fa260009840 tx=0x7fa260009b50 comp rx=0 tx=0).stop 2023-05-20T01:28:21.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.840+0000 7fa272a63700 1 -- 172.21.15.16:0/631832473 shutdown_connections 2023-05-20T01:28:21.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.840+0000 7fa272a63700 1 --2- 172.21.15.16:0/631832473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa26c0cb670 0x7fa26c0ce340 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:21.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.840+0000 7fa272a63700 1 -- 172.21.15.16:0/631832473 >> 172.21.15.16:0/631832473 conn(0x7fa26c0d0c30 msgr2=0x7fa26c0b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:21.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.840+0000 7fa272a63700 1 -- 172.21.15.16:0/631832473 shutdown_connections 2023-05-20T01:28:21.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.840+0000 7fa272a63700 1 -- 172.21.15.16:0/631832473 wait complete. 2023-05-20T01:28:21.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.841+0000 7fa272a63700 1 Processor -- start 2023-05-20T01:28:21.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.842+0000 7fa272a63700 1 -- start start 2023-05-20T01:28:21.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.842+0000 7fa272a63700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa26c0caaa0 0x7fa26c14b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:21.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.842+0000 7fa272a63700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa26c0cb670 0x7fa26c1f3f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:21.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.842+0000 7fa272a63700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa26c1599c0 0x7fa26c0c9250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:21.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.842+0000 7fa272a63700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa26c1f3580 con 0x7fa26c1599c0 2023-05-20T01:28:21.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.842+0000 7fa272a63700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa26c1f44d0 con 0x7fa26c0cb670 2023-05-20T01:28:21.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.842+0000 7fa272a63700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa26c0c9790 con 0x7fa26c0caaa0 2023-05-20T01:28:21.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.842+0000 7fa271000700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa26c1599c0 0x7fa26c0c9250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:21.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.842+0000 7fa271000700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa26c1599c0 0x7fa26c0c9250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39888/0 (socket says 172.21.15.16:39888) 2023-05-20T01:28:21.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.842+0000 7fa271000700 1 -- 172.21.15.16:0/475742616 learned_addr learned my addr 172.21.15.16:0/475742616 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:21.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.842+0000 7fa26bfff700 1 --2- 172.21.15.16:0/475742616 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa26c0caaa0 0x7fa26c14b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:21.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.842+0000 7fa26b7fe700 1 --2- 172.21.15.16:0/475742616 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa26c0cb670 0x7fa26c1f3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:21.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.843+0000 7fa271000700 1 -- 172.21.15.16:0/475742616 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa26c0caaa0 msgr2=0x7fa26c14b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:21.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.843+0000 7fa271000700 1 --2- 172.21.15.16:0/475742616 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa26c0caaa0 0x7fa26c14b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:21.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.843+0000 7fa271000700 1 -- 172.21.15.16:0/475742616 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa26c0cb670 msgr2=0x7fa26c1f3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:21.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.843+0000 7fa271000700 1 --2- 172.21.15.16:0/475742616 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa26c0cb670 0x7fa26c1f3f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:21.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.843+0000 7fa271000700 1 -- 172.21.15.16:0/475742616 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2600094a0 con 0x7fa26c1599c0 2023-05-20T01:28:21.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.843+0000 7fa271000700 1 --2- 172.21.15.16:0/475742616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa26c1599c0 0x7fa26c0c9250 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7fa26400cbc0 tx=0x7fa26400ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:21.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.843+0000 7fa268ff9700 1 -- 172.21.15.16:0/475742616 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa26400a9e0 con 0x7fa26c1599c0 2023-05-20T01:28:21.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.843+0000 7fa268ff9700 1 -- 172.21.15.16:0/475742616 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa26400ab40 con 0x7fa26c1599c0 2023-05-20T01:28:21.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.843+0000 7fa268ff9700 1 -- 172.21.15.16:0/475742616 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa264012400 con 0x7fa26c1599c0 2023-05-20T01:28:21.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.843+0000 7fa272a63700 1 -- 172.21.15.16:0/475742616 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa26c06f0b0 con 0x7fa26c1599c0 2023-05-20T01:28:21.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.843+0000 7fa272a63700 1 -- 172.21.15.16:0/475742616 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa26c06edd0 con 0x7fa26c1599c0 2023-05-20T01:28:21.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.844+0000 7fa272a63700 1 -- 172.21.15.16:0/475742616 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa26c06e750 con 0x7fa26c1599c0 2023-05-20T01:28:21.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.844+0000 7fa268ff9700 1 -- 172.21.15.16:0/475742616 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa26401b3b0 con 0x7fa26c1599c0 2023-05-20T01:28:21.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.845+0000 7fa268ff9700 1 --2- 172.21.15.16:0/475742616 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa25806ace0 0x7fa25806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:21.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.845+0000 7fa268ff9700 1 -- 172.21.15.16:0/475742616 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5874+0+0 (secure 0 0 0) 0x7fa264088d00 con 0x7fa26c1599c0 2023-05-20T01:28:21.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.845+0000 7fa26bfff700 1 --2- 172.21.15.16:0/475742616 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa25806ace0 0x7fa25806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:21.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.845+0000 7fa26bfff700 1 --2- 172.21.15.16:0/475742616 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa25806ace0 0x7fa25806d190 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fa25c009960 tx=0x7fa25c0090e0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:21.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:21.849+0000 7fa268ff9700 1 -- 172.21.15.16:0/475742616 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fa264053f10 con 0x7fa26c1599c0 2023-05-20T01:28:22.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:22.061+0000 7fa272a63700 1 -- 172.21.15.16:0/475742616 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fa26c0747b0 con 0x7fa26c1599c0 2023-05-20T01:28:23.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.018+0000 7fa268ff9700 1 -- 172.21.15.16:0/475742616 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v132) v1 ==== 97+0+0 (secure 0 0 0) 0x7fa264057530 con 0x7fa26c1599c0 2023-05-20T01:28:23.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.020+0000 7fa272a63700 1 -- 172.21.15.16:0/475742616 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa25806ace0 msgr2=0x7fa25806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:23.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.020+0000 7fa272a63700 1 --2- 172.21.15.16:0/475742616 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa25806ace0 0x7fa25806d190 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fa25c009960 tx=0x7fa25c0090e0 comp rx=0 tx=0).stop 2023-05-20T01:28:23.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.020+0000 7fa272a63700 1 -- 172.21.15.16:0/475742616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa26c1599c0 msgr2=0x7fa26c0c9250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:23.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.020+0000 7fa272a63700 1 --2- 172.21.15.16:0/475742616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa26c1599c0 0x7fa26c0c9250 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7fa26400cbc0 tx=0x7fa26400ced0 comp rx=0 tx=0).stop 2023-05-20T01:28:23.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.021+0000 7fa272a63700 1 -- 172.21.15.16:0/475742616 shutdown_connections 2023-05-20T01:28:23.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.021+0000 7fa272a63700 1 --2- 172.21.15.16:0/475742616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa26c1599c0 0x7fa26c0c9250 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:23.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.021+0000 7fa272a63700 1 --2- 172.21.15.16:0/475742616 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa25806ace0 0x7fa25806d190 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:23.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.021+0000 7fa272a63700 1 --2- 172.21.15.16:0/475742616 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa26c0caaa0 0x7fa26c14b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:23.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.021+0000 7fa272a63700 1 --2- 172.21.15.16:0/475742616 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa26c0cb670 0x7fa26c1f3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:23.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.021+0000 7fa272a63700 1 -- 172.21.15.16:0/475742616 >> 172.21.15.16:0/475742616 conn(0x7fa26c0d0c30 msgr2=0x7fa26c0ca2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:23.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.021+0000 7fa272a63700 1 -- 172.21.15.16:0/475742616 shutdown_connections 2023-05-20T01:28:23.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.021+0000 7fa272a63700 1 -- 172.21.15.16:0/475742616 wait complete. 2023-05-20T01:28:23.027 INFO:teuthology.orchestra.run.smithi016.stderr: removed all blocklist entries 2023-05-20T01:28:23.041 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-05-20T01:28:23.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.168+0000 7f72638b3700 1 Processor -- start 2023-05-20T01:28:23.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.168+0000 7f72638b3700 1 -- start start 2023-05-20T01:28:23.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.169+0000 7f72638b3700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f725c0cb320 0x7f725c070910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:23.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.169+0000 7f72638b3700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f725c0ca350 con 0x7f725c06d2a0 2023-05-20T01:28:23.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.169+0000 7f72638b3700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f725c15cac0 con 0x7f725c159000 2023-05-20T01:28:23.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.169+0000 7f72638b3700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f725c15cda0 con 0x7f725c0cb320 2023-05-20T01:28:23.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.169+0000 7f7261e50700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f725c06d2a0 0x7f725c070e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51428/0 (socket says 172.21.15.16:51428) 2023-05-20T01:28:23.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.169+0000 7f7261e50700 1 -- 172.21.15.16:0/364165399 learned_addr learned my addr 172.21.15.16:0/364165399 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:28:23.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.169+0000 7f7253fff700 1 -- 172.21.15.16:0/364165399 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 82731974 0 0) 0x7f725c0ca350 con 0x7f725c06d2a0 2023-05-20T01:28:23.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.169+0000 7f7260e4e700 1 --2- 172.21.15.16:0/364165399 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f725c0cb320 0x7f725c070910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:23.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.169+0000 7f7253fff700 1 -- 172.21.15.16:0/364165399 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f72480035b0 con 0x7f725c06d2a0 2023-05-20T01:28:23.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.170+0000 7f7253fff700 1 -- 172.21.15.16:0/364165399 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1327198777 0 0) 0x7f72480035b0 con 0x7f725c06d2a0 2023-05-20T01:28:23.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.170+0000 7f7253fff700 1 -- 172.21.15.16:0/364165399 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f725c0cb320 msgr2=0x7f725c070910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:23.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.170+0000 7f7253fff700 1 --2- 172.21.15.16:0/364165399 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f725c0cb320 0x7f725c070910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:23.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.170+0000 7f7253fff700 1 -- 172.21.15.16:0/364165399 >> v1:172.21.15.132:6789/0 conn(0x7f725c159000 legacy=0x7f725c06cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:23.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.170+0000 7f7253fff700 1 -- 172.21.15.16:0/364165399 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f725c15d080 con 0x7f725c06d2a0 2023-05-20T01:28:23.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.170+0000 7f7253fff700 1 -- 172.21.15.16:0/364165399 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7f72540019f0 con 0x7f725c06d2a0 2023-05-20T01:28:23.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.170+0000 7f7253fff700 1 -- 172.21.15.16:0/364165399 >> v1:172.21.15.16:6789/0 conn(0x7f725c06d2a0 legacy=0x7f725c070e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:23.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.170+0000 7f7253fff700 1 --2- 172.21.15.16:0/364165399 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7248003aa0 0x7f7248003eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:23.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.170+0000 7f7253fff700 1 --2- 172.21.15.16:0/364165399 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f725c0cb320 0x7f7248004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:23.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.170+0000 7f7253fff700 1 --2- 172.21.15.16:0/364165399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7248004d90 0x7f7248005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:23.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.170+0000 7f7253fff700 1 -- 172.21.15.16:0/364165399 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f725c15cda0 con 0x7f7248004d90 2023-05-20T01:28:23.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.170+0000 7f7253fff700 1 -- 172.21.15.16:0/364165399 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f724800baf0 con 0x7f7248003aa0 2023-05-20T01:28:23.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.170+0000 7f7253fff700 1 -- 172.21.15.16:0/364165399 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f724800bcb0 con 0x7f725c0cb320 2023-05-20T01:28:23.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.170+0000 7f7261e50700 1 --2- 172.21.15.16:0/364165399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7248004d90 0x7f7248005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:23.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.170+0000 7f7260e4e700 1 --2- 172.21.15.16:0/364165399 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f725c0cb320 0x7f7248004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:23.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.170+0000 7f726164f700 1 --2- 172.21.15.16:0/364165399 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7248003aa0 0x7f7248003eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:23.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.170+0000 7f7261e50700 1 -- 172.21.15.16:0/364165399 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f725c0cb320 msgr2=0x7f7248004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:23.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.170+0000 7f7261e50700 1 --2- 172.21.15.16:0/364165399 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f725c0cb320 0x7f7248004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:23.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.170+0000 7f7261e50700 1 -- 172.21.15.16:0/364165399 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7248003aa0 msgr2=0x7f7248003eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:23.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.170+0000 7f7261e50700 1 --2- 172.21.15.16:0/364165399 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7248003aa0 0x7f7248003eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:23.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.170+0000 7f7261e50700 1 -- 172.21.15.16:0/364165399 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72480035b0 con 0x7f7248004d90 2023-05-20T01:28:23.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.170+0000 7f726164f700 1 --2- 172.21.15.16:0/364165399 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7248003aa0 0x7f7248003eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:28:23.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.170+0000 7f7260e4e700 1 --2- 172.21.15.16:0/364165399 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f725c0cb320 0x7f7248004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:28:23.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.170+0000 7f7261e50700 1 --2- 172.21.15.16:0/364165399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7248004d90 0x7f7248005200 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f72540013d0 tx=0x7f7248005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ad2937be2596fc9 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:23.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.171+0000 7f7253fff700 1 -- 172.21.15.16:0/364165399 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7254012600 con 0x7f7248004d90 2023-05-20T01:28:23.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.171+0000 7f7253fff700 1 -- 172.21.15.16:0/364165399 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72540127a0 con 0x7f7248004d90 2023-05-20T01:28:23.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.171+0000 7f7253fff700 1 -- 172.21.15.16:0/364165399 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7254012b50 con 0x7f7248004d90 2023-05-20T01:28:23.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.171+0000 7f72638b3700 1 -- 172.21.15.16:0/364165399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7248004d90 msgr2=0x7f7248005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:23.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.171+0000 7f72638b3700 1 --2- 172.21.15.16:0/364165399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7248004d90 0x7f7248005200 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f72540013d0 tx=0x7f7248005880 comp rx=0 tx=0).stop 2023-05-20T01:28:23.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.171+0000 7f7261e50700 1 -- 172.21.15.16:0/364165399 reap_dead start 2023-05-20T01:28:23.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.171+0000 7f72638b3700 1 -- 172.21.15.16:0/364165399 shutdown_connections 2023-05-20T01:28:23.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.171+0000 7f72638b3700 1 -- 172.21.15.16:0/364165399 >> 172.21.15.16:0/364165399 conn(0x7f725c0661b0 msgr2=0x7f725c066580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:23.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.171+0000 7f72638b3700 1 -- 172.21.15.16:0/364165399 shutdown_connections 2023-05-20T01:28:23.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.171+0000 7f72638b3700 1 -- 172.21.15.16:0/364165399 wait complete. 2023-05-20T01:28:23.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.172+0000 7f72638b3700 1 Processor -- start 2023-05-20T01:28:23.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.173+0000 7f72638b3700 1 -- start start 2023-05-20T01:28:23.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.173+0000 7f72638b3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7248004d90 0x7f725c1593d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:23.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.173+0000 7f72638b3700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f725c1f1ed0 0x7f725c0b41d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:23.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.173+0000 7f72638b3700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f725c0b4710 0x7f725c0c8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:23.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.173+0000 7f72638b3700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f725c14f8e0 con 0x7f7248004d90 2023-05-20T01:28:23.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.173+0000 7f72638b3700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f725c0cdbe0 con 0x7f725c0b4710 2023-05-20T01:28:23.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.173+0000 7f72638b3700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f725c0cdec0 con 0x7f725c1f1ed0 2023-05-20T01:28:23.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.173+0000 7f726164f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7248004d90 0x7f725c1593d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:23.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.173+0000 7f726164f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7248004d90 0x7f725c1593d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39908/0 (socket says 172.21.15.16:39908) 2023-05-20T01:28:23.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.173+0000 7f726164f700 1 -- 172.21.15.16:0/3813771154 learned_addr learned my addr 172.21.15.16:0/3813771154 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:23.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.174+0000 7f7261e50700 1 --2- 172.21.15.16:0/3813771154 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f725c0b4710 0x7f725c0c8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:23.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.174+0000 7f7260e4e700 1 --2- 172.21.15.16:0/3813771154 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f725c1f1ed0 0x7f725c0b41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:23.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.174+0000 7f726164f700 1 -- 172.21.15.16:0/3813771154 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f725c1f1ed0 msgr2=0x7f725c0b41d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:23.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.174+0000 7f726164f700 1 --2- 172.21.15.16:0/3813771154 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f725c1f1ed0 0x7f725c0b41d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:23.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.174+0000 7f726164f700 1 -- 172.21.15.16:0/3813771154 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f725c0b4710 msgr2=0x7f725c0c8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:23.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.174+0000 7f726164f700 1 --2- 172.21.15.16:0/3813771154 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f725c0b4710 0x7f725c0c8aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:23.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.174+0000 7f726164f700 1 -- 172.21.15.16:0/3813771154 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f725c15d880 con 0x7f7248004d90 2023-05-20T01:28:23.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.174+0000 7f726164f700 1 --2- 172.21.15.16:0/3813771154 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7248004d90 0x7f725c1593d0 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f724c00e550 tx=0x7f724c00e910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:23.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.174+0000 7f7251ffb700 1 -- 172.21.15.16:0/3813771154 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f724c0052e0 con 0x7f7248004d90 2023-05-20T01:28:23.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.174+0000 7f7251ffb700 1 -- 172.21.15.16:0/3813771154 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f724c005440 con 0x7f7248004d90 2023-05-20T01:28:23.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.174+0000 7f72638b3700 1 -- 172.21.15.16:0/3813771154 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f725c15d360 con 0x7f7248004d90 2023-05-20T01:28:23.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.174+0000 7f7251ffb700 1 -- 172.21.15.16:0/3813771154 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f724c0038a0 con 0x7f7248004d90 2023-05-20T01:28:23.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.174+0000 7f72638b3700 1 -- 172.21.15.16:0/3813771154 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f725c0b1b00 con 0x7f7248004d90 2023-05-20T01:28:23.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.175+0000 7f72638b3700 1 -- 172.21.15.16:0/3813771154 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f725c0747b0 con 0x7f7248004d90 2023-05-20T01:28:23.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.176+0000 7f7251ffb700 1 -- 172.21.15.16:0/3813771154 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f724c003a00 con 0x7f7248004d90 2023-05-20T01:28:23.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.176+0000 7f7251ffb700 1 --2- 172.21.15.16:0/3813771154 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f724006ace0 0x7f724006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:23.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.176+0000 7f7251ffb700 1 -- 172.21.15.16:0/3813771154 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f724c088960 con 0x7f7248004d90 2023-05-20T01:28:23.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.176+0000 7f7260e4e700 1 --2- 172.21.15.16:0/3813771154 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f724006ace0 0x7f724006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:23.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.177+0000 7f7260e4e700 1 --2- 172.21.15.16:0/3813771154 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f724006ace0 0x7f724006d190 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f725c1f5b70 tx=0x7f7258008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:23.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.181+0000 7f7251ffb700 1 -- 172.21.15.16:0/3813771154 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f724c053d90 con 0x7f7248004d90 2023-05-20T01:28:23.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.458+0000 7f72638b3700 1 -- 172.21.15.16:0/3813771154 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f725c1f5c10 con 0x7f7248004d90 2023-05-20T01:28:23.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.459+0000 7f7251ffb700 1 -- 172.21.15.16:0/3813771154 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f724c019090 con 0x7f7248004d90 2023-05-20T01:28:23.463 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:28:23.463 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-05-20T01:28:23.464 INFO:teuthology.orchestra.run.smithi016.stdout: "auth_dump": [ 2023-05-20T01:28:23.464 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-05-20T01:28:23.464 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.a", 2023-05-20T01:28:23.464 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQA5H2hkq5FEJBAA1qUUPSLmwo43hTQS7GSENA==", 2023-05-20T01:28:23.464 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-05-20T01:28:23.464 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-05-20T01:28:23.464 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2023-05-20T01:28:23.465 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2023-05-20T01:28:23.465 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-05-20T01:28:23.465 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:28:23.465 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-05-20T01:28:23.465 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-05-20T01:28:23.465 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.b", 2023-05-20T01:28:23.465 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQA5H2hk/2XSMBAAnGCIm7TJZsO9HyXwOWxO7w==", 2023-05-20T01:28:23.466 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-05-20T01:28:23.466 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-05-20T01:28:23.466 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2023-05-20T01:28:23.466 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2023-05-20T01:28:23.466 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-05-20T01:28:23.466 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:28:23.466 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-05-20T01:28:23.466 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-05-20T01:28:23.467 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.c", 2023-05-20T01:28:23.467 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQA5H2hkBVK8KxAAgxy9+gbttxZeygwrAX+PHQ==", 2023-05-20T01:28:23.467 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-05-20T01:28:23.467 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-05-20T01:28:23.467 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2023-05-20T01:28:23.467 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2023-05-20T01:28:23.467 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-05-20T01:28:23.468 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:28:23.468 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-05-20T01:28:23.468 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-05-20T01:28:23.468 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.d", 2023-05-20T01:28:23.468 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQA5H2hkFxEMOBAAMPvue7F+/qqOFUD/C//U+g==", 2023-05-20T01:28:23.468 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-05-20T01:28:23.468 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-05-20T01:28:23.469 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2023-05-20T01:28:23.469 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2023-05-20T01:28:23.469 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-05-20T01:28:23.469 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:28:23.469 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-05-20T01:28:23.469 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-05-20T01:28:23.469 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.0", 2023-05-20T01:28:23.469 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQA7H2hkC4WwIRAA5KMQFqldcekzEuH92rZYHA==", 2023-05-20T01:28:23.470 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-05-20T01:28:23.470 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-05-20T01:28:23.470 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-05-20T01:28:23.470 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-05-20T01:28:23.470 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:28:23.470 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-05-20T01:28:23.470 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-05-20T01:28:23.471 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.1", 2023-05-20T01:28:23.471 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQA+H2hkvMH1IBAAhOQOjCiv3r2Pl3HcSS6eAA==", 2023-05-20T01:28:23.471 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-05-20T01:28:23.471 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-05-20T01:28:23.471 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-05-20T01:28:23.471 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-05-20T01:28:23.471 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:28:23.471 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-05-20T01:28:23.472 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-05-20T01:28:23.472 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.2", 2023-05-20T01:28:23.472 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQBBH2hkSJU5IBAAdFd4IhXxISVybFyT24gb0g==", 2023-05-20T01:28:23.472 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-05-20T01:28:23.472 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-05-20T01:28:23.472 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-05-20T01:28:23.472 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-05-20T01:28:23.473 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:28:23.473 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-05-20T01:28:23.473 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-05-20T01:28:23.473 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.3", 2023-05-20T01:28:23.473 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQBEH2hkrcH8HxAANlLWDyIYycLl1TrYCBN1hQ==", 2023-05-20T01:28:23.473 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-05-20T01:28:23.473 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-05-20T01:28:23.473 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-05-20T01:28:23.474 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-05-20T01:28:23.474 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:28:23.474 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-05-20T01:28:23.474 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-05-20T01:28:23.474 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.4", 2023-05-20T01:28:23.474 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQBKH2hkZr+4JBAAzbzKd4ulGI8kVGqXpjCDog==", 2023-05-20T01:28:23.474 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-05-20T01:28:23.474 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-05-20T01:28:23.475 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-05-20T01:28:23.475 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-05-20T01:28:23.475 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:28:23.475 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-05-20T01:28:23.475 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-05-20T01:28:23.475 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.5", 2023-05-20T01:28:23.475 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQBNH2hkNV02JBAALrC7eOTnD1dQM9njk+BgLQ==", 2023-05-20T01:28:23.475 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-05-20T01:28:23.476 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-05-20T01:28:23.476 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-05-20T01:28:23.476 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-05-20T01:28:23.476 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:28:23.476 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-05-20T01:28:23.476 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-05-20T01:28:23.476 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.6", 2023-05-20T01:28:23.477 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQBQH2hkGOfBIxAABCTVPLBi5+drKpqOC8xqdw==", 2023-05-20T01:28:23.477 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-05-20T01:28:23.477 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-05-20T01:28:23.477 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-05-20T01:28:23.477 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-05-20T01:28:23.477 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:28:23.477 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-05-20T01:28:23.477 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-05-20T01:28:23.478 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.7", 2023-05-20T01:28:23.478 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQBTH2hkpwEdIxAAaI3X5ZAVJ9MfVG0MjFJvSQ==", 2023-05-20T01:28:23.478 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-05-20T01:28:23.478 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-05-20T01:28:23.478 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-05-20T01:28:23.478 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-05-20T01:28:23.478 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:28:23.478 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-05-20T01:28:23.479 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-05-20T01:28:23.479 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.0", 2023-05-20T01:28:23.479 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQA6H2hkph3EABAA0277JHCqplw1LklDfidPWQ==", 2023-05-20T01:28:23.479 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-05-20T01:28:23.479 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-05-20T01:28:23.479 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow r", 2023-05-20T01:28:23.479 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-05-20T01:28:23.480 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:28:23.480 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-05-20T01:28:23.480 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-05-20T01:28:23.480 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.1", 2023-05-20T01:28:23.480 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQA6H2hk43WQBxAApSfBSu1qEaKSGWaYAvz/Jg==", 2023-05-20T01:28:23.480 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-05-20T01:28:23.480 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-05-20T01:28:23.480 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2023-05-20T01:28:23.481 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2023-05-20T01:28:23.481 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2023-05-20T01:28:23.481 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:28:23.481 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-05-20T01:28:23.481 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-05-20T01:28:23.481 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.2", 2023-05-20T01:28:23.481 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQA6H2hk+TgtBBAAc4+W2eOVgJfNd3xXjGvz4Q==", 2023-05-20T01:28:23.481 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-05-20T01:28:23.482 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-05-20T01:28:23.482 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2023-05-20T01:28:23.482 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2023-05-20T01:28:23.482 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2023-05-20T01:28:23.482 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:28:23.482 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-05-20T01:28:23.482 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-05-20T01:28:23.482 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.3", 2023-05-20T01:28:23.483 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQA6H2hkifVADRAAQP6vtdVNYyeNPV0d+agOCw==", 2023-05-20T01:28:23.483 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-05-20T01:28:23.483 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-05-20T01:28:23.483 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2023-05-20T01:28:23.483 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2023-05-20T01:28:23.483 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2023-05-20T01:28:23.483 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:28:23.484 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-05-20T01:28:23.484 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-05-20T01:28:23.484 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.admin", 2023-05-20T01:28:23.484 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQA5H2hkb4BMBhAAw7zpfDAb7x5szkLY+zTL7Q==", 2023-05-20T01:28:23.484 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-05-20T01:28:23.484 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2023-05-20T01:28:23.484 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2023-05-20T01:28:23.484 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2023-05-20T01:28:23.485 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-05-20T01:28:23.485 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:28:23.485 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-05-20T01:28:23.485 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-05-20T01:28:23.485 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mgr.x", 2023-05-20T01:28:23.485 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQA5H2hksQbMIBAAQFm/pY5stJHNudg6zcPBdQ==", 2023-05-20T01:28:23.485 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-05-20T01:28:23.485 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2023-05-20T01:28:23.486 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile mgr", 2023-05-20T01:28:23.486 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-05-20T01:28:23.486 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:28:23.486 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-05-20T01:28:23.486 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-05-20T01:28:23.486 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mgr.y", 2023-05-20T01:28:23.487 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQA5H2hkBQg7HBAAbOJZ+55rRACVBMd2eyjqtg==", 2023-05-20T01:28:23.487 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-05-20T01:28:23.487 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2023-05-20T01:28:23.487 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile mgr", 2023-05-20T01:28:23.487 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-05-20T01:28:23.487 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:28:23.487 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:28:23.487 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2023-05-20T01:28:23.488 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-05-20T01:28:23.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.461+0000 7f72638b3700 1 -- 172.21.15.16:0/3813771154 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f724006ace0 msgr2=0x7f724006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:23.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.461+0000 7f72638b3700 1 --2- 172.21.15.16:0/3813771154 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f724006ace0 0x7f724006d190 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f725c1f5b70 tx=0x7f7258008040 comp rx=0 tx=0).stop 2023-05-20T01:28:23.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.461+0000 7f72638b3700 1 -- 172.21.15.16:0/3813771154 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7248004d90 msgr2=0x7f725c1593d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:23.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.461+0000 7f72638b3700 1 --2- 172.21.15.16:0/3813771154 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7248004d90 0x7f725c1593d0 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f724c00e550 tx=0x7f724c00e910 comp rx=0 tx=0).stop 2023-05-20T01:28:23.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.461+0000 7f72638b3700 1 -- 172.21.15.16:0/3813771154 shutdown_connections 2023-05-20T01:28:23.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.461+0000 7f72638b3700 1 --2- 172.21.15.16:0/3813771154 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f725c0b4710 0x7f725c0c8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:23.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.461+0000 7f72638b3700 1 --2- 172.21.15.16:0/3813771154 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7248004d90 0x7f725c1593d0 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:23.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.461+0000 7f72638b3700 1 --2- 172.21.15.16:0/3813771154 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f724006ace0 0x7f724006d190 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:23.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.461+0000 7f72638b3700 1 --2- 172.21.15.16:0/3813771154 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f725c1f1ed0 0x7f725c0b41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:23.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.461+0000 7f72638b3700 1 -- 172.21.15.16:0/3813771154 >> 172.21.15.16:0/3813771154 conn(0x7f725c0661b0 msgr2=0x7f725c14d540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:23.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.461+0000 7f72638b3700 1 -- 172.21.15.16:0/3813771154 shutdown_connections 2023-05-20T01:28:23.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.461+0000 7f72638b3700 1 -- 172.21.15.16:0/3813771154 wait complete. 2023-05-20T01:28:23.491 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-20T01:28:23.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.611+0000 7fa9ad6aa700 1 Processor -- start 2023-05-20T01:28:23.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.611+0000 7fa9ad6aa700 1 -- start start 2023-05-20T01:28:23.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.612+0000 7fa9ad6aa700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa9a80b4750 0x7fa9a80d2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:23.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.612+0000 7fa9ad6aa700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa9a80cd970 0x7fa9a80cddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:23.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.612+0000 7fa9ad6aa700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9a80b3b50 con 0x7fa9a80cfc50 2023-05-20T01:28:23.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.612+0000 7fa9ad6aa700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9a80cfad0 con 0x7fa9a80b4750 2023-05-20T01:28:23.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.612+0000 7fa9ad6aa700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9a80d3df0 con 0x7fa9a80cd970 2023-05-20T01:28:23.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.612+0000 7fa9a67fc700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa9a80b4750 0x7fa9a80d2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:23.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.612+0000 7fa9a6ffd700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fa9a80cfc50 0x7fa9a80d0020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51444/0 (socket says 172.21.15.16:51444) 2023-05-20T01:28:23.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.612+0000 7fa9a6ffd700 1 -- 172.21.15.16:0/13106528 learned_addr learned my addr 172.21.15.16:0/13106528 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:28:23.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.612+0000 7fa9a5ffb700 1 -- 172.21.15.16:0/13106528 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 720501223 0 0) 0x7fa9a80b3b50 con 0x7fa9a80cfc50 2023-05-20T01:28:23.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.612+0000 7fa9a5ffb700 1 -- 172.21.15.16:0/13106528 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa9900035b0 con 0x7fa9a80cfc50 2023-05-20T01:28:23.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.613+0000 7fa9a67fc700 1 -- 172.21.15.16:0/13106528 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa9a80cd970 msgr2=0x7fa9a80cddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:23.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.613+0000 7fa9a77fe700 1 --2- 172.21.15.16:0/13106528 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa9a80cd970 0x7fa9a80cddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:23.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.613+0000 7fa9a67fc700 1 --2- 172.21.15.16:0/13106528 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa9a80cd970 0x7fa9a80cddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:23.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.613+0000 7fa9a67fc700 1 -- 172.21.15.16:0/13106528 >> v1:172.21.15.16:6789/0 conn(0x7fa9a80cfc50 legacy=0x7fa9a80d0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:23.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.613+0000 7fa9a67fc700 1 -- 172.21.15.16:0/13106528 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9a80d0b90 con 0x7fa9a80b4750 2023-05-20T01:28:23.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.613+0000 7fa9a77fe700 1 --2- 172.21.15.16:0/13106528 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa9a80cd970 0x7fa9a80cddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:28:23.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.613+0000 7fa9a67fc700 1 --2- 172.21.15.16:0/13106528 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9a80b4750 0x7fa9a80d2c10 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7fa998009840 tx=0x7fa998009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c24940a1a55f21f server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:23.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.613+0000 7fa9a5ffb700 1 -- 172.21.15.16:0/13106528 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa998011830 con 0x7fa9a80b4750 2023-05-20T01:28:23.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.613+0000 7fa9a5ffb700 1 -- 172.21.15.16:0/13106528 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa998011990 con 0x7fa9a80b4750 2023-05-20T01:28:23.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.613+0000 7fa9a5ffb700 1 -- 172.21.15.16:0/13106528 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa998011d20 con 0x7fa9a80b4750 2023-05-20T01:28:23.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.613+0000 7fa9ad6aa700 1 -- 172.21.15.16:0/13106528 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9a80b4750 msgr2=0x7fa9a80d2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:23.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.613+0000 7fa9ad6aa700 1 --2- 172.21.15.16:0/13106528 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9a80b4750 0x7fa9a80d2c10 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7fa998009840 tx=0x7fa998009b50 comp rx=0 tx=0).stop 2023-05-20T01:28:23.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.613+0000 7fa9ad6aa700 1 -- 172.21.15.16:0/13106528 shutdown_connections 2023-05-20T01:28:23.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.613+0000 7fa9ad6aa700 1 --2- 172.21.15.16:0/13106528 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa9a80cd970 0x7fa9a80cddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:23.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.613+0000 7fa9ad6aa700 1 --2- 172.21.15.16:0/13106528 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9a80b4750 0x7fa9a80d2c10 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:23.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.613+0000 7fa9ad6aa700 1 -- 172.21.15.16:0/13106528 >> 172.21.15.16:0/13106528 conn(0x7fa9a809c590 msgr2=0x7fa9a80d3240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:23.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.613+0000 7fa9ad6aa700 1 -- 172.21.15.16:0/13106528 shutdown_connections 2023-05-20T01:28:23.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.614+0000 7fa9ad6aa700 1 -- 172.21.15.16:0/13106528 wait complete. 2023-05-20T01:28:23.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.615+0000 7fa9ad6aa700 1 Processor -- start 2023-05-20T01:28:23.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.615+0000 7fa9ad6aa700 1 -- start start 2023-05-20T01:28:23.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.616+0000 7fa9ad6aa700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9a80b4750 0x7fa9a81e42f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:23.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.616+0000 7fa9ad6aa700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9a80cd970 0x7fa9a80b0880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:23.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.616+0000 7fa9ad6aa700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa9a80cfc50 0x7fa9a81e6d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:23.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.616+0000 7fa9ad6aa700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9a80b6a70 con 0x7fa9a80cd970 2023-05-20T01:28:23.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.616+0000 7fa9ad6aa700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9a81e72d0 con 0x7fa9a80b4750 2023-05-20T01:28:23.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.616+0000 7fa9ad6aa700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9a80b6110 con 0x7fa9a80cfc50 2023-05-20T01:28:23.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.616+0000 7fa9a67fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9a80cd970 0x7fa9a80b0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:23.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.616+0000 7fa9a67fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9a80cd970 0x7fa9a80b0880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39922/0 (socket says 172.21.15.16:39922) 2023-05-20T01:28:23.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.616+0000 7fa9a67fc700 1 -- 172.21.15.16:0/3291112427 learned_addr learned my addr 172.21.15.16:0/3291112427 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:23.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.616+0000 7fa9a67fc700 1 -- 172.21.15.16:0/3291112427 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa9a80cfc50 msgr2=0x7fa9a81e6d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:23.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.616+0000 7fa9a77fe700 1 --2- 172.21.15.16:0/3291112427 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa9a80cfc50 0x7fa9a81e6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:23.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.616+0000 7fa9a6ffd700 1 --2- 172.21.15.16:0/3291112427 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9a80b4750 0x7fa9a81e42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:23.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.616+0000 7fa9a67fc700 1 --2- 172.21.15.16:0/3291112427 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa9a80cfc50 0x7fa9a81e6d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:23.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.616+0000 7fa9a67fc700 1 -- 172.21.15.16:0/3291112427 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9a80b4750 msgr2=0x7fa9a81e42f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:23.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.616+0000 7fa9a67fc700 1 --2- 172.21.15.16:0/3291112427 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9a80b4750 0x7fa9a81e42f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:23.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.616+0000 7fa9a67fc700 1 -- 172.21.15.16:0/3291112427 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9980094a0 con 0x7fa9a80cd970 2023-05-20T01:28:23.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.616+0000 7fa9a67fc700 1 --2- 172.21.15.16:0/3291112427 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9a80cd970 0x7fa9a80b0880 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7fa998009fd0 tx=0x7fa998004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:23.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.616+0000 7fa98f7fe700 1 -- 172.21.15.16:0/3291112427 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9980036a0 con 0x7fa9a80cd970 2023-05-20T01:28:23.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.616+0000 7fa98f7fe700 1 -- 172.21.15.16:0/3291112427 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa99801e070 con 0x7fa9a80cd970 2023-05-20T01:28:23.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.616+0000 7fa9ad6aa700 1 -- 172.21.15.16:0/3291112427 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9a80b4380 con 0x7fa9a80cd970 2023-05-20T01:28:23.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.616+0000 7fa98f7fe700 1 -- 172.21.15.16:0/3291112427 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa998003a60 con 0x7fa9a80cd970 2023-05-20T01:28:23.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.617+0000 7fa9ad6aa700 1 -- 172.21.15.16:0/3291112427 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9a815d0b0 con 0x7fa9a80cd970 2023-05-20T01:28:23.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.617+0000 7fa9ad6aa700 1 -- 172.21.15.16:0/3291112427 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9a80d4000 con 0x7fa9a80cd970 2023-05-20T01:28:23.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.618+0000 7fa98f7fe700 1 -- 172.21.15.16:0/3291112427 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa998003bc0 con 0x7fa9a80cd970 2023-05-20T01:28:23.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.618+0000 7fa98f7fe700 1 --2- 172.21.15.16:0/3291112427 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa99406ace0 0x7fa99406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:23.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.618+0000 7fa98f7fe700 1 -- 172.21.15.16:0/3291112427 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5530+0+0 (secure 0 0 0) 0x7fa99808a230 con 0x7fa9a80cd970 2023-05-20T01:28:23.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.618+0000 7fa9a6ffd700 1 --2- 172.21.15.16:0/3291112427 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa99406ace0 0x7fa99406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:23.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.619+0000 7fa9a6ffd700 1 --2- 172.21.15.16:0/3291112427 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa99406ace0 0x7fa99406d190 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fa99c0011e0 tx=0x7fa99c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:23.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.623+0000 7fa98f7fe700 1 -- 172.21.15.16:0/3291112427 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fa998055580 con 0x7fa9a80cd970 2023-05-20T01:28:23.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.823+0000 7fa9ad6aa700 1 -- 172.21.15.16:0/3291112427 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa9a804e8c0 con 0x7fa9a80cd970 2023-05-20T01:28:23.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.824+0000 7fa98f7fe700 1 -- 172.21.15.16:0/3291112427 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v132) v1 ==== 74+0+14760 (secure 0 0 0) 0x7fa998058ba0 con 0x7fa9a80cd970 2023-05-20T01:28:23.829 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:28:23.829 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":132,"fsid":"c2245cc5-ecab-4087-a59b-bc81db901676","created":"2023-05-20T01:16:09.549230+0000","modified":"2023-05-20T01:28:23.014618+0000","last_up_change":"2023-05-20T01:16:19.788624+0000","last_in_change":"2023-05-20T01:16:12.092160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-05-20T01:16:20.934366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-05-20T01:16:21.372294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"317a83cd-19c8-4958-a301-6087ef342289","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6825","nonce":3923634179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6827","nonce":3923634179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6831","nonce":3923634179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6829","nonce":3923634179}]},"public_addr":"172.21.15.16:6825/3923634179","cluster_addr":"172.21.15.16:6827/3923634179","heartbeat_back_addr":"172.21.15.16:6831/3923634179","heartbeat_front_addr":"172.21.15.16:6829/3923634179","state":["exists","up"]},{"osd":1,"uuid":"5581046f-8219-4089-8d7d-b7846399505e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6809","nonce":3860264220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6811","nonce":3860264220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6815","nonce":3860264220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6813","nonce":3860264220}]},"public_addr":"172.21.15.16:6809/3860264220","cluster_addr":"172.21.15.16:6811/3860264220","heartbeat_back_addr":"172.21.15.16:6815/3860264220","heartbeat_front_addr":"172.21.15.16:6813/3860264220","state":["exists","up"]},{"osd":2,"uuid":"072e9e66-4133-478e-8d4a-64d46c2963f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6817","nonce":293187030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6819","nonce":293187030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6823","nonce":293187030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6821","nonce":293187030}]},"public_addr":"172.21.15.16:6817/293187030","cluster_addr":"172.21.15.16:6819/293187030","heartbeat_back_addr":"172.21.15.16:6823/293187030","heartbeat_front_addr":"172.21.15.16:6821/293187030","state":["exists","up"]},{"osd":3,"uuid":"11e1bba6-00f8-48dd-9028-d8d716b7f145","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6801","nonce":3669781890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6803","nonce":3669781890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6807","nonce":3669781890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6805","nonce":3669781890}]},"public_addr":"172.21.15.16:6801/3669781890","cluster_addr":"172.21.15.16:6803/3669781890","heartbeat_back_addr":"172.21.15.16:6807/3669781890","heartbeat_front_addr":"172.21.15.16:6805/3669781890","state":["exists","up"]},{"osd":4,"uuid":"b8909c3a-cf70-46cb-8d0b-009c707cf9d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6819","nonce":506317223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6823","nonce":506317223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6829","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6831","nonce":506317223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6827","nonce":506317223}]},"public_addr":"172.21.15.132:6819/506317223","cluster_addr":"172.21.15.132:6823/506317223","heartbeat_back_addr":"172.21.15.132:6831/506317223","heartbeat_front_addr":"172.21.15.132:6827/506317223","state":["exists","up"]},{"osd":5,"uuid":"b1818304-4583-47bf-a6ec-0bfbb5166233","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6818","nonce":1844689995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6822","nonce":1844689995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6830","nonce":1844689995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6826","nonce":1844689995}]},"public_addr":"172.21.15.132:6818/1844689995","cluster_addr":"172.21.15.132:6822/1844689995","heartbeat_back_addr":"172.21.15.132:6830/1844689995","heartbeat_front_addr":"172.21.15.132:6826/1844689995","state":["exists","up"]},{"osd":6,"uuid":"d37f4691-eed1-4a93-8818-18354f03f4a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6801","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6802","nonce":727581123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6805","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6807","nonce":727581123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6813","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6815","nonce":727581123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6811","nonce":727581123}]},"public_addr":"172.21.15.132:6802/727581123","cluster_addr":"172.21.15.132:6807/727581123","heartbeat_back_addr":"172.21.15.132:6815/727581123","heartbeat_front_addr":"172.21.15.132:6811/727581123","state":["exists","up"]},{"osd":7,"uuid":"4f712f62-5d1d-4816-8042-03209144f8b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6803","nonce":466001149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6806","nonce":466001149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6814","nonce":466001149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6810","nonce":466001149}]},"public_addr":"172.21.15.132:6803/466001149","cluster_addr":"172.21.15.132:6806/466001149","heartbeat_back_addr":"172.21.15.132:6814/466001149","heartbeat_front_addr":"172.21.15.132:6810/466001149","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.338863+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.278680+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305694+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.246835+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.276237+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.304441+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305421+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.267050+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-20T01:28:23.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.826+0000 7fa9ad6aa700 1 -- 172.21.15.16:0/3291112427 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa99406ace0 msgr2=0x7fa99406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:23.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.826+0000 7fa9ad6aa700 1 --2- 172.21.15.16:0/3291112427 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa99406ace0 0x7fa99406d190 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fa99c0011e0 tx=0x7fa99c009040 comp rx=0 tx=0).stop 2023-05-20T01:28:23.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.826+0000 7fa9ad6aa700 1 -- 172.21.15.16:0/3291112427 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9a80cd970 msgr2=0x7fa9a80b0880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:23.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.826+0000 7fa9ad6aa700 1 --2- 172.21.15.16:0/3291112427 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9a80cd970 0x7fa9a80b0880 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7fa998009fd0 tx=0x7fa998004750 comp rx=0 tx=0).stop 2023-05-20T01:28:23.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.827+0000 7fa9ad6aa700 1 -- 172.21.15.16:0/3291112427 shutdown_connections 2023-05-20T01:28:23.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.827+0000 7fa9ad6aa700 1 --2- 172.21.15.16:0/3291112427 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa99406ace0 0x7fa99406d190 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:23.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.827+0000 7fa9ad6aa700 1 --2- 172.21.15.16:0/3291112427 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa9a80cfc50 0x7fa9a81e6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:23.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.827+0000 7fa9ad6aa700 1 --2- 172.21.15.16:0/3291112427 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9a80b4750 0x7fa9a81e42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:23.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.827+0000 7fa9ad6aa700 1 --2- 172.21.15.16:0/3291112427 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9a80cd970 0x7fa9a80b0880 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:23.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.827+0000 7fa9ad6aa700 1 -- 172.21.15.16:0/3291112427 >> 172.21.15.16:0/3291112427 conn(0x7fa9a809c590 msgr2=0x7fa9a806b190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:23.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.827+0000 7fa9ad6aa700 1 -- 172.21.15.16:0/3291112427 shutdown_connections 2023-05-20T01:28:23.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.827+0000 7fa9ad6aa700 1 -- 172.21.15.16:0/3291112427 wait complete. 2023-05-20T01:28:23.843 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-05-20T01:16:20.934366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-05-20T01:16:21.372294+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-05-20T01:28:23.844 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-05-20T01:28:23.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.969+0000 7feaa3fac700 1 Processor -- start 2023-05-20T01:28:23.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.969+0000 7feaa3fac700 1 -- start start 2023-05-20T01:28:23.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.969+0000 7feaa3fac700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fea9c15cec0 0x7fea9c071bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:23.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.970+0000 7feaa3fac700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fea9c072130 0x7fea9c0cf9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:23.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.970+0000 7feaa3fac700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea9c0cd630 con 0x7fea9c15d290 2023-05-20T01:28:23.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.970+0000 7feaa3fac700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea9c072570 con 0x7fea9c15cec0 2023-05-20T01:28:23.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.970+0000 7feaa3fac700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea9c0cca60 con 0x7fea9c072130 2023-05-20T01:28:23.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.970+0000 7feaa1d48700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fea9c15cec0 0x7fea9c071bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:23.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.970+0000 7feaa2549700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fea9c072130 0x7fea9c0cf9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:23.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.970+0000 7feaa1d48700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fea9c15cec0 0x7fea9c071bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.16:36950/0 (socket says 172.21.15.16:36950) 2023-05-20T01:28:23.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.970+0000 7feaa1547700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7fea9c15d290 0x7fea9c0cf2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.16:32938/0 (socket says 172.21.15.16:32938) 2023-05-20T01:28:23.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.970+0000 7feaa1d48700 1 -- 172.21.15.16:0/4149218584 learned_addr learned my addr 172.21.15.16:0/4149218584 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:23.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.970+0000 7feaa1d48700 1 -- 172.21.15.16:0/4149218584 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fea9c072130 msgr2=0x7fea9c0cf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:23.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.970+0000 7feaa1d48700 1 --2- 172.21.15.16:0/4149218584 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fea9c072130 0x7fea9c0cf9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:23.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.970+0000 7feaa1d48700 1 -- 172.21.15.16:0/4149218584 >> v1:172.21.15.132:6789/0 conn(0x7fea9c15d290 legacy=0x7fea9c0cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:23.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.970+0000 7feaa0d46700 1 -- 172.21.15.16:0/4149218584 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 610063035 0 0) 0x7fea9c0cd630 con 0x7fea9c15d290 2023-05-20T01:28:23.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.970+0000 7feaa1d48700 1 -- 172.21.15.16:0/4149218584 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea9c0ccd00 con 0x7fea9c15cec0 2023-05-20T01:28:23.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.970+0000 7feaa2549700 1 --2- 172.21.15.16:0/4149218584 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fea9c072130 0x7fea9c0cf9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-05-20T01:28:23.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.971+0000 7feaa1d48700 1 --2- 172.21.15.16:0/4149218584 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fea9c15cec0 0x7fea9c071bf0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fea98006fd0 tx=0x7fea98009bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5bdebb9608616031 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:23.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.971+0000 7feaa0d46700 1 -- 172.21.15.16:0/4149218584 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea98005e30 con 0x7fea9c15cec0 2023-05-20T01:28:23.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.971+0000 7feaa0d46700 1 -- 172.21.15.16:0/4149218584 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea98004030 con 0x7fea9c15cec0 2023-05-20T01:28:23.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.971+0000 7feaa0d46700 1 -- 172.21.15.16:0/4149218584 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea980043e0 con 0x7fea9c15cec0 2023-05-20T01:28:23.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.971+0000 7feaa3fac700 1 -- 172.21.15.16:0/4149218584 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fea9c15cec0 msgr2=0x7fea9c071bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:23.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.971+0000 7feaa3fac700 1 --2- 172.21.15.16:0/4149218584 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fea9c15cec0 0x7fea9c071bf0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fea98006fd0 tx=0x7fea98009bd0 comp rx=0 tx=0).stop 2023-05-20T01:28:23.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.971+0000 7feaa3fac700 1 -- 172.21.15.16:0/4149218584 shutdown_connections 2023-05-20T01:28:23.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.971+0000 7feaa3fac700 1 --2- 172.21.15.16:0/4149218584 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fea9c072130 0x7fea9c0cf9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:23.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.971+0000 7feaa3fac700 1 --2- 172.21.15.16:0/4149218584 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fea9c15cec0 0x7fea9c071bf0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:23.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.971+0000 7feaa3fac700 1 -- 172.21.15.16:0/4149218584 >> 172.21.15.16:0/4149218584 conn(0x7fea9c0cffd0 msgr2=0x7fea9c0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:23.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.972+0000 7feaa3fac700 1 -- 172.21.15.16:0/4149218584 shutdown_connections 2023-05-20T01:28:23.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.972+0000 7feaa3fac700 1 -- 172.21.15.16:0/4149218584 wait complete. 2023-05-20T01:28:23.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.973+0000 7feaa3fac700 1 Processor -- start 2023-05-20T01:28:23.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.973+0000 7feaa3fac700 1 -- start start 2023-05-20T01:28:23.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.974+0000 7feaa3fac700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fea9c072130 0x7fea9c0b0490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:23.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.974+0000 7feaa3fac700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea9c15cec0 0x7fea9c1f22e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:23.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.974+0000 7feaa3fac700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fea9c15d290 0x7fea9c1e6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:23.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.974+0000 7feaa3fac700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea9c150f70 con 0x7fea9c15cec0 2023-05-20T01:28:23.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.974+0000 7feaa3fac700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea9c155550 con 0x7fea9c15d290 2023-05-20T01:28:23.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.974+0000 7feaa3fac700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea9c1559e0 con 0x7fea9c072130 2023-05-20T01:28:23.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.974+0000 7feaa1547700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea9c15cec0 0x7fea9c1f22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:23.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.974+0000 7feaa1547700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea9c15cec0 0x7fea9c1f22e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39934/0 (socket says 172.21.15.16:39934) 2023-05-20T01:28:23.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.974+0000 7feaa1547700 1 -- 172.21.15.16:0/4291785271 learned_addr learned my addr 172.21.15.16:0/4291785271 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:23.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.974+0000 7feaa1d48700 1 --2- 172.21.15.16:0/4291785271 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fea9c072130 0x7fea9c0b0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:23.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.974+0000 7feaa2549700 1 --2- 172.21.15.16:0/4291785271 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fea9c15d290 0x7fea9c1e6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:23.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.974+0000 7feaa1547700 1 -- 172.21.15.16:0/4291785271 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fea9c072130 msgr2=0x7fea9c0b0490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:23.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.974+0000 7feaa1547700 1 --2- 172.21.15.16:0/4291785271 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fea9c072130 0x7fea9c0b0490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:23.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.974+0000 7feaa1547700 1 -- 172.21.15.16:0/4291785271 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fea9c15d290 msgr2=0x7fea9c1e6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:23.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.974+0000 7feaa1547700 1 --2- 172.21.15.16:0/4291785271 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fea9c15d290 0x7fea9c1e6ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:23.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.974+0000 7feaa1547700 1 -- 172.21.15.16:0/4291785271 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea94009a70 con 0x7fea9c15cec0 2023-05-20T01:28:23.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.974+0000 7feaa1547700 1 --2- 172.21.15.16:0/4291785271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea9c15cec0 0x7fea9c1f22e0 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7fea8c00c550 tx=0x7fea8c00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:23.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.974+0000 7fea927fc700 1 -- 172.21.15.16:0/4291785271 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea8c004c80 con 0x7fea9c15cec0 2023-05-20T01:28:23.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.974+0000 7fea927fc700 1 -- 172.21.15.16:0/4291785271 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea8c004e00 con 0x7fea9c15cec0 2023-05-20T01:28:23.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.974+0000 7feaa3fac700 1 -- 172.21.15.16:0/4291785271 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea98009500 con 0x7fea9c15cec0 2023-05-20T01:28:23.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.974+0000 7fea927fc700 1 -- 172.21.15.16:0/4291785271 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea8c007310 con 0x7fea9c15cec0 2023-05-20T01:28:23.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.975+0000 7feaa3fac700 1 -- 172.21.15.16:0/4291785271 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea9c0cd2c0 con 0x7fea9c15cec0 2023-05-20T01:28:23.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.975+0000 7feaa3fac700 1 -- 172.21.15.16:0/4291785271 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea9c0ccfe0 con 0x7fea9c15cec0 2023-05-20T01:28:23.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.976+0000 7fea927fc700 1 -- 172.21.15.16:0/4291785271 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fea8c007470 con 0x7fea9c15cec0 2023-05-20T01:28:23.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.976+0000 7fea927fc700 1 --2- 172.21.15.16:0/4291785271 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fea8006ace0 0x7fea8006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:23.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.976+0000 7fea927fc700 1 -- 172.21.15.16:0/4291785271 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5530+0+0 (secure 0 0 0) 0x7fea8c088e70 con 0x7fea9c15cec0 2023-05-20T01:28:23.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.976+0000 7feaa1d48700 1 --2- 172.21.15.16:0/4291785271 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fea8006ace0 0x7fea8006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:23.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.977+0000 7feaa1d48700 1 --2- 172.21.15.16:0/4291785271 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fea8006ace0 0x7fea8006d190 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fea98006fd0 tx=0x7fea98008070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:23.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:23.981+0000 7fea927fc700 1 -- 172.21.15.16:0/4291785271 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fea8c0540b0 con 0x7fea9c15cec0 2023-05-20T01:28:24.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.177+0000 7feaa3fac700 1 -- 172.21.15.16:0/4291785271 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fea9c160870 con 0x7fea9c15cec0 2023-05-20T01:28:24.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.178+0000 7fea927fc700 1 -- 172.21.15.16:0/4291785271 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v132) v1 ==== 92+0+10 (secure 0 0 0) 0x7fea8c0576d0 con 0x7fea9c15cec0 2023-05-20T01:28:24.182 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 8 2023-05-20T01:28:24.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.180+0000 7feaa3fac700 1 -- 172.21.15.16:0/4291785271 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fea8006ace0 msgr2=0x7fea8006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:24.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.180+0000 7feaa3fac700 1 --2- 172.21.15.16:0/4291785271 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fea8006ace0 0x7fea8006d190 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fea98006fd0 tx=0x7fea98008070 comp rx=0 tx=0).stop 2023-05-20T01:28:24.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.180+0000 7feaa3fac700 1 -- 172.21.15.16:0/4291785271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea9c15cec0 msgr2=0x7fea9c1f22e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:24.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.180+0000 7feaa3fac700 1 --2- 172.21.15.16:0/4291785271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea9c15cec0 0x7fea9c1f22e0 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7fea8c00c550 tx=0x7fea8c00c860 comp rx=0 tx=0).stop 2023-05-20T01:28:24.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.180+0000 7feaa3fac700 1 -- 172.21.15.16:0/4291785271 shutdown_connections 2023-05-20T01:28:24.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.180+0000 7feaa3fac700 1 --2- 172.21.15.16:0/4291785271 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fea9c15d290 0x7fea9c1e6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:24.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.180+0000 7feaa3fac700 1 --2- 172.21.15.16:0/4291785271 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fea8006ace0 0x7fea8006d190 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:24.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.180+0000 7feaa3fac700 1 --2- 172.21.15.16:0/4291785271 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fea9c072130 0x7fea9c0b0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:24.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.180+0000 7feaa3fac700 1 --2- 172.21.15.16:0/4291785271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea9c15cec0 0x7fea9c1f22e0 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:24.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.180+0000 7feaa3fac700 1 -- 172.21.15.16:0/4291785271 >> 172.21.15.16:0/4291785271 conn(0x7fea9c0cffd0 msgr2=0x7fea9c15d730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:24.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.180+0000 7feaa3fac700 1 -- 172.21.15.16:0/4291785271 shutdown_connections 2023-05-20T01:28:24.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.180+0000 7feaa3fac700 1 -- 172.21.15.16:0/4291785271 wait complete. 2023-05-20T01:28:24.196 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-05-20T01:28:24.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.321+0000 7f57c50d8700 1 Processor -- start 2023-05-20T01:28:24.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.321+0000 7f57c50d8700 1 -- start start 2023-05-20T01:28:24.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.322+0000 7f57c50d8700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f57c00cec80 0x7f57c00cf0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:24.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.322+0000 7f57c50d8700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f57c00cc110 0x7f57c00cc590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:24.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.322+0000 7f57c50d8700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57c00ce4d0 con 0x7f57c015ab80 2023-05-20T01:28:24.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.322+0000 7f57c50d8700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57c00cb700 con 0x7f57c00cec80 2023-05-20T01:28:24.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.322+0000 7f57c50d8700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57c00cf630 con 0x7f57c00cc110 2023-05-20T01:28:24.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.323+0000 7f57bf59e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f57c00cc110 0x7f57c00cc590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:24.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.323+0000 7f57be59c700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f57c00cec80 0x7f57c00cf0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:24.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.323+0000 7f57bed9d700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f57c015ab80 0x7f57c00cda80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.16:32950/0 (socket says 172.21.15.16:32950) 2023-05-20T01:28:24.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.323+0000 7f57bed9d700 1 -- 172.21.15.16:0/1298843255 learned_addr learned my addr 172.21.15.16:0/1298843255 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:28:24.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.323+0000 7f57be59c700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f57c00cec80 0x7f57c00cf0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.16:36962/0 (socket says 172.21.15.16:36962) 2023-05-20T01:28:24.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.323+0000 7f57bf59e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f57c00cc110 0x7f57c00cc590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.16:39982/0 (socket says 172.21.15.16:39982) 2023-05-20T01:28:24.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.323+0000 7f57bdd9b700 1 -- 172.21.15.16:0/1298843255 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2975408809 0 0) 0x7f57c00ce4d0 con 0x7f57c015ab80 2023-05-20T01:28:24.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.323+0000 7f57bdd9b700 1 -- 172.21.15.16:0/1298843255 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57a80035b0 con 0x7f57c015ab80 2023-05-20T01:28:24.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.323+0000 7f57be59c700 1 -- 172.21.15.16:0/1298843255 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f57c00cc110 msgr2=0x7f57c00cc590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:24.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.323+0000 7f57be59c700 1 --2- 172.21.15.16:0/1298843255 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f57c00cc110 0x7f57c00cc590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:24.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.323+0000 7f57be59c700 1 -- 172.21.15.16:0/1298843255 >> v1:172.21.15.132:6789/0 conn(0x7f57c015ab80 legacy=0x7f57c00cda80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:24.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.323+0000 7f57be59c700 1 -- 172.21.15.16:0/1298843255 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57c00ce7b0 con 0x7f57c00cec80 2023-05-20T01:28:24.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.323+0000 7f57bf59e700 1 --2- 172.21.15.16:0/1298843255 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f57c00cc110 0x7f57c00cc590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-05-20T01:28:24.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.323+0000 7f57be59c700 1 --2- 172.21.15.16:0/1298843255 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f57c00cec80 0x7f57c00cf0f0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f57b0009840 tx=0x7f57b0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=25f64f0603a393f9 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:24.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.323+0000 7f57bdd9b700 1 -- 172.21.15.16:0/1298843255 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b0011830 con 0x7f57c00cec80 2023-05-20T01:28:24.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.323+0000 7f57bdd9b700 1 -- 172.21.15.16:0/1298843255 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57b0011990 con 0x7f57c00cec80 2023-05-20T01:28:24.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.323+0000 7f57bdd9b700 1 -- 172.21.15.16:0/1298843255 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b0011d00 con 0x7f57c00cec80 2023-05-20T01:28:24.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.324+0000 7f57c50d8700 1 -- 172.21.15.16:0/1298843255 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f57c00cec80 msgr2=0x7f57c00cf0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:24.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.324+0000 7f57c50d8700 1 --2- 172.21.15.16:0/1298843255 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f57c00cec80 0x7f57c00cf0f0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f57b0009840 tx=0x7f57b0009b50 comp rx=0 tx=0).stop 2023-05-20T01:28:24.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.324+0000 7f57c50d8700 1 -- 172.21.15.16:0/1298843255 shutdown_connections 2023-05-20T01:28:24.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.324+0000 7f57c50d8700 1 --2- 172.21.15.16:0/1298843255 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f57c00cc110 0x7f57c00cc590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:24.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.324+0000 7f57c50d8700 1 --2- 172.21.15.16:0/1298843255 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f57c00cec80 0x7f57c00cf0f0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:24.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.324+0000 7f57c50d8700 1 -- 172.21.15.16:0/1298843255 >> 172.21.15.16:0/1298843255 conn(0x7f57c006ae30 msgr2=0x7f57c009a400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:24.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.324+0000 7f57c50d8700 1 -- 172.21.15.16:0/1298843255 shutdown_connections 2023-05-20T01:28:24.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.324+0000 7f57c50d8700 1 -- 172.21.15.16:0/1298843255 wait complete. 2023-05-20T01:28:24.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.326+0000 7f57c50d8700 1 Processor -- start 2023-05-20T01:28:24.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.326+0000 7f57c50d8700 1 -- start start 2023-05-20T01:28:24.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.327+0000 7f57c50d8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57c00cc110 0x7f57c01f0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:24.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.327+0000 7f57c50d8700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f57c00cec80 0x7f57c00cd0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:24.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.327+0000 7f57c50d8700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f57c015ab80 0x7f57c00cd630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:24.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.327+0000 7f57c50d8700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57c00cb370 con 0x7f57c00cc110 2023-05-20T01:28:24.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.327+0000 7f57c50d8700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57c01f06d0 con 0x7f57c015ab80 2023-05-20T01:28:24.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.327+0000 7f57c50d8700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57c00c8f60 con 0x7f57c00cec80 2023-05-20T01:28:24.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.327+0000 7f57bed9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57c00cc110 0x7f57c01f0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:24.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.327+0000 7f57bed9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57c00cc110 0x7f57c01f0190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39948/0 (socket says 172.21.15.16:39948) 2023-05-20T01:28:24.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.327+0000 7f57bed9d700 1 -- 172.21.15.16:0/1854270869 learned_addr learned my addr 172.21.15.16:0/1854270869 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:24.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.327+0000 7f57bf59e700 1 --2- 172.21.15.16:0/1854270869 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f57c015ab80 0x7f57c00cd630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:24.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.327+0000 7f57be59c700 1 --2- 172.21.15.16:0/1854270869 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f57c00cec80 0x7f57c00cd0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:24.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.327+0000 7f57bed9d700 1 -- 172.21.15.16:0/1854270869 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f57c00cec80 msgr2=0x7f57c00cd0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:24.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.327+0000 7f57bed9d700 1 --2- 172.21.15.16:0/1854270869 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f57c00cec80 0x7f57c00cd0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:24.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.327+0000 7f57bed9d700 1 -- 172.21.15.16:0/1854270869 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f57c015ab80 msgr2=0x7f57c00cd630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:24.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.327+0000 7f57bed9d700 1 --2- 172.21.15.16:0/1854270869 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f57c015ab80 0x7f57c00cd630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:24.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.327+0000 7f57bed9d700 1 -- 172.21.15.16:0/1854270869 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57b8009a70 con 0x7f57c00cc110 2023-05-20T01:28:24.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.328+0000 7f57bed9d700 1 --2- 172.21.15.16:0/1854270869 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57c00cc110 0x7f57c01f0190 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f57b400c4d0 tx=0x7f57b400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:24.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.328+0000 7f57a77fe700 1 -- 172.21.15.16:0/1854270869 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b4005d70 con 0x7f57c00cc110 2023-05-20T01:28:24.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.328+0000 7f57a77fe700 1 -- 172.21.15.16:0/1854270869 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57b40070d0 con 0x7f57c00cc110 2023-05-20T01:28:24.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.328+0000 7f57c50d8700 1 -- 172.21.15.16:0/1854270869 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57b00094a0 con 0x7f57c00cc110 2023-05-20T01:28:24.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.328+0000 7f57a77fe700 1 -- 172.21.15.16:0/1854270869 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b4007440 con 0x7f57c00cc110 2023-05-20T01:28:24.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.328+0000 7f57c50d8700 1 -- 172.21.15.16:0/1854270869 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57c00cbbf0 con 0x7f57c00cc110 2023-05-20T01:28:24.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.329+0000 7f57c50d8700 1 -- 172.21.15.16:0/1854270869 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57c00cb910 con 0x7f57c00cc110 2023-05-20T01:28:24.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.329+0000 7f57a77fe700 1 -- 172.21.15.16:0/1854270869 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f57b40075a0 con 0x7f57c00cc110 2023-05-20T01:28:24.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.330+0000 7f57a77fe700 1 --2- 172.21.15.16:0/1854270869 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f57ac06ace0 0x7f57ac06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:24.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.330+0000 7f57a77fe700 1 -- 172.21.15.16:0/1854270869 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f57b40886c0 con 0x7f57c00cc110 2023-05-20T01:28:24.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.330+0000 7f57be59c700 1 --2- 172.21.15.16:0/1854270869 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f57ac06ace0 0x7f57ac06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:24.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.330+0000 7f57be59c700 1 --2- 172.21.15.16:0/1854270869 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f57ac06ace0 0x7f57ac06d190 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f57b0009fd0 tx=0x7f57b001c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:24.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.334+0000 7f57a77fe700 1 -- 172.21.15.16:0/1854270869 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f57b4053a30 con 0x7f57c00cc110 2023-05-20T01:28:24.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.533+0000 7f57c50d8700 1 -- 172.21.15.16:0/1854270869 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f57c00ce190 con 0x7f57c00cc110 2023-05-20T01:28:24.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.534+0000 7f57a77fe700 1 -- 172.21.15.16:0/1854270869 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v132) v1 ==== 93+0+10 (secure 0 0 0) 0x7f57b4017020 con 0x7f57c00cc110 2023-05-20T01:28:24.538 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-05-20T01:28:24.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.536+0000 7f57c50d8700 1 -- 172.21.15.16:0/1854270869 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f57ac06ace0 msgr2=0x7f57ac06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:24.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.536+0000 7f57c50d8700 1 --2- 172.21.15.16:0/1854270869 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f57ac06ace0 0x7f57ac06d190 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f57b0009fd0 tx=0x7f57b001c040 comp rx=0 tx=0).stop 2023-05-20T01:28:24.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.536+0000 7f57c50d8700 1 -- 172.21.15.16:0/1854270869 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57c00cc110 msgr2=0x7f57c01f0190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:24.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.536+0000 7f57c50d8700 1 --2- 172.21.15.16:0/1854270869 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57c00cc110 0x7f57c01f0190 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f57b400c4d0 tx=0x7f57b400c890 comp rx=0 tx=0).stop 2023-05-20T01:28:24.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.536+0000 7f57c50d8700 1 -- 172.21.15.16:0/1854270869 shutdown_connections 2023-05-20T01:28:24.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.536+0000 7f57c50d8700 1 --2- 172.21.15.16:0/1854270869 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f57c015ab80 0x7f57c00cd630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:24.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.536+0000 7f57c50d8700 1 --2- 172.21.15.16:0/1854270869 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57c00cc110 0x7f57c01f0190 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:24.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.536+0000 7f57c50d8700 1 --2- 172.21.15.16:0/1854270869 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f57ac06ace0 0x7f57ac06d190 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:24.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.536+0000 7f57c50d8700 1 --2- 172.21.15.16:0/1854270869 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f57c00cec80 0x7f57c00cd0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:24.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.536+0000 7f57c50d8700 1 -- 172.21.15.16:0/1854270869 >> 172.21.15.16:0/1854270869 conn(0x7f57c006ae30 msgr2=0x7f57c00b0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:24.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.536+0000 7f57c50d8700 1 -- 172.21.15.16:0/1854270869 shutdown_connections 2023-05-20T01:28:24.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.536+0000 7f57c50d8700 1 -- 172.21.15.16:0/1854270869 wait complete. 2023-05-20T01:28:24.553 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-05-20T01:28:24.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.674+0000 7fce3491d700 1 Processor -- start 2023-05-20T01:28:24.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.674+0000 7fce3491d700 1 -- start start 2023-05-20T01:28:24.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.675+0000 7fce3491d700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fce300ccf00 0x7fce300cb530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:24.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.675+0000 7fce3491d700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce3015cec0 con 0x7fce3006d2a0 2023-05-20T01:28:24.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.675+0000 7fce3491d700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce300d0490 con 0x7fce3015caf0 2023-05-20T01:28:24.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.675+0000 7fce3491d700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce30160610 con 0x7fce300ccf00 2023-05-20T01:28:24.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.675+0000 7fce2dd9b700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fce3006d2a0 0x7fce300cae20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51482/0 (socket says 172.21.15.16:51482) 2023-05-20T01:28:24.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.675+0000 7fce2dd9b700 1 -- 172.21.15.16:0/2209374478 learned_addr learned my addr 172.21.15.16:0/2209374478 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:28:24.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.675+0000 7fce2d59a700 1 -- 172.21.15.16:0/2209374478 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2269391137 0 0) 0x7fce3015cec0 con 0x7fce3006d2a0 2023-05-20T01:28:24.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.676+0000 7fce2ed9d700 1 --2- 172.21.15.16:0/2209374478 >> v2:172.21.15.132:3300/0 conn(0x7fce300ccf00 0x7fce300cb530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:24.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.676+0000 7fce2d59a700 1 -- 172.21.15.16:0/2209374478 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce180035b0 con 0x7fce3006d2a0 2023-05-20T01:28:24.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.676+0000 7fce2d59a700 1 -- 172.21.15.16:0/2209374478 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3603170542 0 0) 0x7fce180035b0 con 0x7fce3006d2a0 2023-05-20T01:28:24.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.676+0000 7fce2d59a700 1 -- 172.21.15.16:0/2209374478 >> v2:172.21.15.132:3300/0 conn(0x7fce300ccf00 msgr2=0x7fce300cb530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:24.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.676+0000 7fce2d59a700 1 --2- 172.21.15.16:0/2209374478 >> v2:172.21.15.132:3300/0 conn(0x7fce300ccf00 0x7fce300cb530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:24.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.676+0000 7fce2d59a700 1 -- 172.21.15.16:0/2209374478 >> v1:172.21.15.132:6789/0 conn(0x7fce3015caf0 legacy=0x7fce3006cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:24.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.676+0000 7fce2d59a700 1 -- 172.21.15.16:0/2209374478 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce301608f0 con 0x7fce3006d2a0 2023-05-20T01:28:24.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.676+0000 7fce2d59a700 1 -- 172.21.15.16:0/2209374478 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7fce20001670 con 0x7fce3006d2a0 2023-05-20T01:28:24.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.676+0000 7fce2d59a700 1 -- 172.21.15.16:0/2209374478 >> v1:172.21.15.16:6789/0 conn(0x7fce3006d2a0 legacy=0x7fce300cae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:24.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.676+0000 7fce2d59a700 1 --2- 172.21.15.16:0/2209374478 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fce18003a70 0x7fce18003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:24.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.676+0000 7fce2d59a700 1 --2- 172.21.15.16:0/2209374478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce18004850 0x7fce18004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:24.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.676+0000 7fce2d59a700 1 --2- 172.21.15.16:0/2209374478 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fce18005270 0x7fce1800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:24.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.676+0000 7fce2d59a700 1 -- 172.21.15.16:0/2209374478 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce30160610 con 0x7fce18004850 2023-05-20T01:28:24.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.676+0000 7fce2d59a700 1 -- 172.21.15.16:0/2209374478 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce1800c1b0 con 0x7fce18003a70 2023-05-20T01:28:24.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.676+0000 7fce2d59a700 1 -- 172.21.15.16:0/2209374478 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce1800c370 con 0x7fce18005270 2023-05-20T01:28:24.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.676+0000 7fce2dd9b700 1 --2- 172.21.15.16:0/2209374478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce18004850 0x7fce18004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:24.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.676+0000 7fce2ed9d700 1 --2- 172.21.15.16:0/2209374478 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fce18005270 0x7fce1800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:24.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.676+0000 7fce2e59c700 1 --2- 172.21.15.16:0/2209374478 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fce18003a70 0x7fce18003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:24.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.676+0000 7fce2dd9b700 1 -- 172.21.15.16:0/2209374478 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fce18005270 msgr2=0x7fce1800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:24.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.676+0000 7fce2dd9b700 1 --2- 172.21.15.16:0/2209374478 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fce18005270 0x7fce1800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:24.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.676+0000 7fce2dd9b700 1 -- 172.21.15.16:0/2209374478 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fce18003a70 msgr2=0x7fce18003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:24.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.676+0000 7fce2dd9b700 1 --2- 172.21.15.16:0/2209374478 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fce18003a70 0x7fce18003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:24.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.676+0000 7fce2ed9d700 1 -- 172.21.15.16:0/2209374478 reap_dead start 2023-05-20T01:28:24.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.676+0000 7fce2dd9b700 1 -- 172.21.15.16:0/2209374478 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce180035b0 con 0x7fce18004850 2023-05-20T01:28:24.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.676+0000 7fce2dd9b700 1 --2- 172.21.15.16:0/2209374478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce18004850 0x7fce18004cc0 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7fce2000c210 tx=0x7fce18005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a70932f3fdc3701 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:24.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.677+0000 7fce2d59a700 1 -- 172.21.15.16:0/2209374478 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce20007c10 con 0x7fce18004850 2023-05-20T01:28:24.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.677+0000 7fce2d59a700 1 -- 172.21.15.16:0/2209374478 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce2000cd50 con 0x7fce18004850 2023-05-20T01:28:24.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.677+0000 7fce2d59a700 1 -- 172.21.15.16:0/2209374478 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce20005ba0 con 0x7fce18004850 2023-05-20T01:28:24.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.677+0000 7fce3491d700 1 -- 172.21.15.16:0/2209374478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce18004850 msgr2=0x7fce18004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:24.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.677+0000 7fce3491d700 1 --2- 172.21.15.16:0/2209374478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce18004850 0x7fce18004cc0 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7fce2000c210 tx=0x7fce18005880 comp rx=0 tx=0).stop 2023-05-20T01:28:24.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.677+0000 7fce3491d700 1 -- 172.21.15.16:0/2209374478 shutdown_connections 2023-05-20T01:28:24.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.677+0000 7fce3491d700 1 --2- 172.21.15.16:0/2209374478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce18004850 0x7fce18004cc0 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:24.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.677+0000 7fce3491d700 1 -- 172.21.15.16:0/2209374478 >> 172.21.15.16:0/2209374478 conn(0x7fce300661b0 msgr2=0x7fce30066580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:24.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.677+0000 7fce3491d700 1 -- 172.21.15.16:0/2209374478 shutdown_connections 2023-05-20T01:28:24.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.677+0000 7fce3491d700 1 -- 172.21.15.16:0/2209374478 wait complete. 2023-05-20T01:28:24.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.679+0000 7fce3491d700 1 Processor -- start 2023-05-20T01:28:24.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.679+0000 7fce3491d700 1 -- start start 2023-05-20T01:28:24.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.679+0000 7fce3491d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fce18004850 0x7fce300ce9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:24.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.679+0000 7fce3491d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce3015d5a0 0x7fce3013fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:24.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.679+0000 7fce3491d700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fce301402d0 0x7fce30142cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:24.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.679+0000 7fce3491d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce30143a30 con 0x7fce3015d5a0 2023-05-20T01:28:24.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.679+0000 7fce3491d700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce30140740 con 0x7fce18004850 2023-05-20T01:28:24.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.679+0000 7fce3491d700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce30140ba0 con 0x7fce301402d0 2023-05-20T01:28:24.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.679+0000 7fce2dd9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce3015d5a0 0x7fce3013fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:24.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.679+0000 7fce2dd9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce3015d5a0 0x7fce3013fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39968/0 (socket says 172.21.15.16:39968) 2023-05-20T01:28:24.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.680+0000 7fce2dd9b700 1 -- 172.21.15.16:0/1473004956 learned_addr learned my addr 172.21.15.16:0/1473004956 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:24.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.680+0000 7fce2e59c700 1 --2- 172.21.15.16:0/1473004956 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fce18004850 0x7fce300ce9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:24.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.680+0000 7fce2ed9d700 1 --2- 172.21.15.16:0/1473004956 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fce301402d0 0x7fce30142cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:24.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.680+0000 7fce2dd9b700 1 -- 172.21.15.16:0/1473004956 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fce301402d0 msgr2=0x7fce30142cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:24.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.680+0000 7fce2dd9b700 1 --2- 172.21.15.16:0/1473004956 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fce301402d0 0x7fce30142cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:24.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.680+0000 7fce2dd9b700 1 -- 172.21.15.16:0/1473004956 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fce18004850 msgr2=0x7fce300ce9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:24.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.680+0000 7fce2dd9b700 1 --2- 172.21.15.16:0/1473004956 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fce18004850 0x7fce300ce9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:24.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.680+0000 7fce2dd9b700 1 -- 172.21.15.16:0/1473004956 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce30160eb0 con 0x7fce3015d5a0 2023-05-20T01:28:24.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.680+0000 7fce2ed9d700 1 --2- 172.21.15.16:0/1473004956 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fce301402d0 0x7fce30142cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:28:24.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.680+0000 7fce2dd9b700 1 --2- 172.21.15.16:0/1473004956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce3015d5a0 0x7fce3013fd90 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7fce2000c800 tx=0x7fce20001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:24.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.680+0000 7fce1effd700 1 -- 172.21.15.16:0/1473004956 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce20001630 con 0x7fce3015d5a0 2023-05-20T01:28:24.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.680+0000 7fce1effd700 1 -- 172.21.15.16:0/1473004956 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce2000cec0 con 0x7fce3015d5a0 2023-05-20T01:28:24.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.680+0000 7fce1effd700 1 -- 172.21.15.16:0/1473004956 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce20007310 con 0x7fce3015d5a0 2023-05-20T01:28:24.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.680+0000 7fce3491d700 1 -- 172.21.15.16:0/1473004956 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce30160bd0 con 0x7fce3015d5a0 2023-05-20T01:28:24.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.680+0000 7fce3491d700 1 -- 172.21.15.16:0/1473004956 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce300d0110 con 0x7fce3015d5a0 2023-05-20T01:28:24.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.681+0000 7fce3491d700 1 -- 172.21.15.16:0/1473004956 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce300747b0 con 0x7fce3015d5a0 2023-05-20T01:28:24.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.682+0000 7fce1effd700 1 -- 172.21.15.16:0/1473004956 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fce20007470 con 0x7fce3015d5a0 2023-05-20T01:28:24.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.682+0000 7fce1effd700 1 --2- 172.21.15.16:0/1473004956 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fce1006ace0 0x7fce1006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:24.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.682+0000 7fce1effd700 1 -- 172.21.15.16:0/1473004956 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5530+0+0 (secure 0 0 0) 0x7fce2008ab70 con 0x7fce3015d5a0 2023-05-20T01:28:24.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.682+0000 7fce2e59c700 1 --2- 172.21.15.16:0/1473004956 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fce1006ace0 0x7fce1006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:24.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.683+0000 7fce2e59c700 1 --2- 172.21.15.16:0/1473004956 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fce1006ace0 0x7fce1006d190 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fce24000c00 tx=0x7fce24009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:24.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.686+0000 7fce1effd700 1 -- 172.21.15.16:0/1473004956 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fce20055fa0 con 0x7fce3015d5a0 2023-05-20T01:28:24.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.887+0000 7fce3491d700 1 -- 172.21.15.16:0/1473004956 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fce30143a30 con 0x7fce3015d5a0 2023-05-20T01:28:24.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.887+0000 7fce1effd700 1 -- 172.21.15.16:0/1473004956 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v132) v1 ==== 84+0+126 (secure 0 0 0) 0x7fce20017090 con 0x7fce3015d5a0 2023-05-20T01:28:24.892 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:28:24.892 INFO:teuthology.orchestra.run.smithi016.stdout:[ 2023-05-20T01:28:24.892 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-05-20T01:28:24.892 INFO:teuthology.orchestra.run.smithi016.stdout: "poolnum": 1, 2023-05-20T01:28:24.893 INFO:teuthology.orchestra.run.smithi016.stdout: "poolname": "rbd" 2023-05-20T01:28:24.893 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-05-20T01:28:24.893 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-05-20T01:28:24.893 INFO:teuthology.orchestra.run.smithi016.stdout: "poolnum": 2, 2023-05-20T01:28:24.893 INFO:teuthology.orchestra.run.smithi016.stdout: "poolname": ".mgr" 2023-05-20T01:28:24.893 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:28:24.893 INFO:teuthology.orchestra.run.smithi016.stdout:] 2023-05-20T01:28:24.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.889+0000 7fce3491d700 1 -- 172.21.15.16:0/1473004956 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fce1006ace0 msgr2=0x7fce1006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:24.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.889+0000 7fce3491d700 1 --2- 172.21.15.16:0/1473004956 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fce1006ace0 0x7fce1006d190 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fce24000c00 tx=0x7fce24009040 comp rx=0 tx=0).stop 2023-05-20T01:28:24.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.889+0000 7fce3491d700 1 -- 172.21.15.16:0/1473004956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce3015d5a0 msgr2=0x7fce3013fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:24.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.889+0000 7fce3491d700 1 --2- 172.21.15.16:0/1473004956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce3015d5a0 0x7fce3013fd90 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7fce2000c800 tx=0x7fce20001a00 comp rx=0 tx=0).stop 2023-05-20T01:28:24.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.889+0000 7fce3491d700 1 -- 172.21.15.16:0/1473004956 shutdown_connections 2023-05-20T01:28:24.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.889+0000 7fce3491d700 1 --2- 172.21.15.16:0/1473004956 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fce1006ace0 0x7fce1006d190 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:24.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.889+0000 7fce3491d700 1 --2- 172.21.15.16:0/1473004956 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fce301402d0 0x7fce30142cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:24.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.889+0000 7fce3491d700 1 --2- 172.21.15.16:0/1473004956 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fce18004850 0x7fce300ce9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:24.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.889+0000 7fce3491d700 1 --2- 172.21.15.16:0/1473004956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce3015d5a0 0x7fce3013fd90 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:24.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.889+0000 7fce3491d700 1 -- 172.21.15.16:0/1473004956 >> 172.21.15.16:0/1473004956 conn(0x7fce300661b0 msgr2=0x7fce30071bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:24.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.889+0000 7fce3491d700 1 -- 172.21.15.16:0/1473004956 shutdown_connections 2023-05-20T01:28:24.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:24.889+0000 7fce3491d700 1 -- 172.21.15.16:0/1473004956 wait complete. 2023-05-20T01:28:24.906 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-05-20T01:28:24.906 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-05-20T01:28:25.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.031+0000 7efc9c564700 1 Processor -- start 2023-05-20T01:28:25.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.031+0000 7efc9c564700 1 -- start start 2023-05-20T01:28:25.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.032+0000 7efc9c564700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7efc94159880 0x7efc9415dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:25.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.032+0000 7efc9c564700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efc940ce310 0x7efc9415a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:25.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.032+0000 7efc9c564700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc9415adc0 con 0x7efc94159c50 2023-05-20T01:28:25.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.032+0000 7efc9c564700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc9415a610 con 0x7efc94159880 2023-05-20T01:28:25.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.032+0000 7efc9c564700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc9415a800 con 0x7efc940ce310 2023-05-20T01:28:25.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.033+0000 7efc99aff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7efc94159c50 0x7efc940cdc00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51504/0 (socket says 172.21.15.16:51504) 2023-05-20T01:28:25.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.033+0000 7efc99aff700 1 -- 172.21.15.16:0/2020816056 learned_addr learned my addr 172.21.15.16:0/2020816056 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:28:25.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.033+0000 7efc9a300700 1 --2- 172.21.15.16:0/2020816056 >> v2:172.21.15.132:3300/0 conn(0x7efc94159880 0x7efc9415dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:25.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.033+0000 7efc9ab01700 1 --2- 172.21.15.16:0/2020816056 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efc940ce310 0x7efc9415a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:25.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.033+0000 7efc992fe700 1 -- 172.21.15.16:0/2020816056 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2963688450 0 0) 0x7efc9415adc0 con 0x7efc94159c50 2023-05-20T01:28:25.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.033+0000 7efc992fe700 1 -- 172.21.15.16:0/2020816056 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc800035b0 con 0x7efc94159c50 2023-05-20T01:28:25.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.033+0000 7efc9ab01700 1 -- 172.21.15.16:0/2020816056 >> v2:172.21.15.132:3300/0 conn(0x7efc94159880 msgr2=0x7efc9415dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:25.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.033+0000 7efc9ab01700 1 --2- 172.21.15.16:0/2020816056 >> v2:172.21.15.132:3300/0 conn(0x7efc94159880 0x7efc9415dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:25.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.033+0000 7efc9ab01700 1 -- 172.21.15.16:0/2020816056 >> v1:172.21.15.16:6789/0 conn(0x7efc94159c50 legacy=0x7efc940cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:25.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.033+0000 7efc9ab01700 1 -- 172.21.15.16:0/2020816056 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc9415aae0 con 0x7efc940ce310 2023-05-20T01:28:25.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.033+0000 7efc9a300700 1 --2- 172.21.15.16:0/2020816056 >> v2:172.21.15.132:3300/0 conn(0x7efc94159880 0x7efc9415dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-05-20T01:28:25.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.033+0000 7efc9ab01700 1 --2- 172.21.15.16:0/2020816056 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efc940ce310 0x7efc9415a0d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7efc8c009d90 tx=0x7efc8c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d5da1c926dba2fe server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:25.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.033+0000 7efc992fe700 1 -- 172.21.15.16:0/2020816056 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc8c00e070 con 0x7efc940ce310 2023-05-20T01:28:25.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.033+0000 7efc992fe700 1 -- 172.21.15.16:0/2020816056 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc8c00bb20 con 0x7efc940ce310 2023-05-20T01:28:25.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.033+0000 7efc992fe700 1 -- 172.21.15.16:0/2020816056 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc8c00be90 con 0x7efc940ce310 2023-05-20T01:28:25.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.034+0000 7efc9c564700 1 -- 172.21.15.16:0/2020816056 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efc940ce310 msgr2=0x7efc9415a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:25.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.034+0000 7efc9c564700 1 --2- 172.21.15.16:0/2020816056 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efc940ce310 0x7efc9415a0d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7efc8c009d90 tx=0x7efc8c00b330 comp rx=0 tx=0).stop 2023-05-20T01:28:25.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.034+0000 7efc9c564700 1 -- 172.21.15.16:0/2020816056 shutdown_connections 2023-05-20T01:28:25.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.034+0000 7efc9c564700 1 --2- 172.21.15.16:0/2020816056 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efc940ce310 0x7efc9415a0d0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:25.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.034+0000 7efc9c564700 1 --2- 172.21.15.16:0/2020816056 >> v2:172.21.15.132:3300/0 conn(0x7efc94159880 0x7efc9415dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:25.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.034+0000 7efc9c564700 1 -- 172.21.15.16:0/2020816056 >> 172.21.15.16:0/2020816056 conn(0x7efc940ccab0 msgr2=0x7efc940cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:25.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.034+0000 7efc9c564700 1 -- 172.21.15.16:0/2020816056 shutdown_connections 2023-05-20T01:28:25.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.034+0000 7efc9c564700 1 -- 172.21.15.16:0/2020816056 wait complete. 2023-05-20T01:28:25.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.035+0000 7efc9c564700 1 Processor -- start 2023-05-20T01:28:25.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.035+0000 7efc9c564700 1 -- start start 2023-05-20T01:28:25.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.036+0000 7efc9c564700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efc940ce310 0x7efc941550d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:25.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.036+0000 7efc9c564700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efc94159880 0x7efc940b0360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:25.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.036+0000 7efc9c564700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc94159c50 0x7efc940cb510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:25.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.036+0000 7efc9c564700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc9409cc80 con 0x7efc94159c50 2023-05-20T01:28:25.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.036+0000 7efc9c564700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc940b3db0 con 0x7efc94159880 2023-05-20T01:28:25.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.036+0000 7efc9c564700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc940b4090 con 0x7efc940ce310 2023-05-20T01:28:25.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.036+0000 7efc9ab01700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc94159c50 0x7efc940cb510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:25.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.036+0000 7efc9ab01700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc94159c50 0x7efc940cb510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39984/0 (socket says 172.21.15.16:39984) 2023-05-20T01:28:25.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.036+0000 7efc9ab01700 1 -- 172.21.15.16:0/1244607796 learned_addr learned my addr 172.21.15.16:0/1244607796 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:25.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.036+0000 7efc9a300700 1 --2- 172.21.15.16:0/1244607796 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efc940ce310 0x7efc941550d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:25.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.036+0000 7efc9ab01700 1 -- 172.21.15.16:0/1244607796 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efc940ce310 msgr2=0x7efc941550d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:25.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.036+0000 7efc9ab01700 1 --2- 172.21.15.16:0/1244607796 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efc940ce310 0x7efc941550d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:25.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.036+0000 7efc99aff700 1 --2- 172.21.15.16:0/1244607796 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efc94159880 0x7efc940b0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:25.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.036+0000 7efc9ab01700 1 -- 172.21.15.16:0/1244607796 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efc94159880 msgr2=0x7efc940b0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:25.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.036+0000 7efc9ab01700 1 --2- 172.21.15.16:0/1244607796 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efc94159880 0x7efc940b0360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:25.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.036+0000 7efc9ab01700 1 -- 172.21.15.16:0/1244607796 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc840094a0 con 0x7efc94159c50 2023-05-20T01:28:25.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.037+0000 7efc9ab01700 1 --2- 172.21.15.16:0/1244607796 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc94159c50 0x7efc940cb510 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7efc8c014040 tx=0x7efc8c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:25.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.037+0000 7efc8affd700 1 -- 172.21.15.16:0/1244607796 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc8c004960 con 0x7efc94159c50 2023-05-20T01:28:25.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.037+0000 7efc8affd700 1 -- 172.21.15.16:0/1244607796 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc8c020070 con 0x7efc94159c50 2023-05-20T01:28:25.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.037+0000 7efc9c564700 1 -- 172.21.15.16:0/1244607796 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc8c009a70 con 0x7efc94159c50 2023-05-20T01:28:25.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.037+0000 7efc8affd700 1 -- 172.21.15.16:0/1244607796 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc8c0276c0 con 0x7efc94159c50 2023-05-20T01:28:25.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.037+0000 7efc9c564700 1 -- 172.21.15.16:0/1244607796 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc9415d820 con 0x7efc94159c50 2023-05-20T01:28:25.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.038+0000 7efc9c564700 1 -- 172.21.15.16:0/1244607796 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc9415d540 con 0x7efc94159c50 2023-05-20T01:28:25.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.038+0000 7efc8affd700 1 -- 172.21.15.16:0/1244607796 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7efc8c012430 con 0x7efc94159c50 2023-05-20T01:28:25.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.039+0000 7efc8affd700 1 --2- 172.21.15.16:0/1244607796 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7efc7806ace0 0x7efc7806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:25.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.039+0000 7efc8affd700 1 -- 172.21.15.16:0/1244607796 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5530+0+0 (secure 0 0 0) 0x7efc8c022070 con 0x7efc94159c50 2023-05-20T01:28:25.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.039+0000 7efc9a300700 1 --2- 172.21.15.16:0/1244607796 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7efc7806ace0 0x7efc7806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:25.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.039+0000 7efc9a300700 1 --2- 172.21.15.16:0/1244607796 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7efc7806ace0 0x7efc7806d190 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7efc84009b80 tx=0x7efc84008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:25.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.043+0000 7efc8affd700 1 -- 172.21.15.16:0/1244607796 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7efc8c0561b0 con 0x7efc94159c50 2023-05-20T01:28:25.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.243+0000 7efc9c564700 1 -- 172.21.15.16:0/1244607796 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7efc940747b0 con 0x7efc94159c50 2023-05-20T01:28:25.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.665+0000 7efc8affd700 1 -- 172.21.15.16:0/1244607796 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v133) v1 ==== 138+0+0 (secure 0 0 0) 0x7efc8c0597d0 con 0x7efc94159c50 2023-05-20T01:28:25.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.670+0000 7efc9c564700 1 -- 172.21.15.16:0/1244607796 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7efc7806ace0 msgr2=0x7efc7806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:25.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.670+0000 7efc9c564700 1 --2- 172.21.15.16:0/1244607796 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7efc7806ace0 0x7efc7806d190 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7efc84009b80 tx=0x7efc84008040 comp rx=0 tx=0).stop 2023-05-20T01:28:25.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.670+0000 7efc9c564700 1 -- 172.21.15.16:0/1244607796 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc94159c50 msgr2=0x7efc940cb510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:25.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.670+0000 7efc9c564700 1 --2- 172.21.15.16:0/1244607796 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc94159c50 0x7efc940cb510 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7efc8c014040 tx=0x7efc8c004ad0 comp rx=0 tx=0).stop 2023-05-20T01:28:25.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.671+0000 7efc9c564700 1 -- 172.21.15.16:0/1244607796 shutdown_connections 2023-05-20T01:28:25.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.671+0000 7efc9c564700 1 --2- 172.21.15.16:0/1244607796 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc94159c50 0x7efc940cb510 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:25.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.671+0000 7efc9c564700 1 --2- 172.21.15.16:0/1244607796 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7efc7806ace0 0x7efc7806d190 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:25.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.671+0000 7efc9c564700 1 --2- 172.21.15.16:0/1244607796 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efc940ce310 0x7efc941550d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:25.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.671+0000 7efc9c564700 1 --2- 172.21.15.16:0/1244607796 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efc94159880 0x7efc940b0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:25.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.671+0000 7efc9c564700 1 -- 172.21.15.16:0/1244607796 >> 172.21.15.16:0/1244607796 conn(0x7efc940ccab0 msgr2=0x7efc9406de00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:25.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.671+0000 7efc9c564700 1 -- 172.21.15.16:0/1244607796 shutdown_connections 2023-05-20T01:28:25.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.672+0000 7efc9c564700 1 -- 172.21.15.16:0/1244607796 wait complete. 2023-05-20T01:28:25.677 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' created 2023-05-20T01:28:25.690 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-05-20T01:28:25.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.823+0000 7f6ec241a700 1 Processor -- start 2023-05-20T01:28:25.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.824+0000 7f6ec241a700 1 -- start start 2023-05-20T01:28:25.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.824+0000 7f6ec241a700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6ebc157450 0x7f6ebc159e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:25.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.824+0000 7f6ec241a700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f6ebc071e10 0x7f6ebc0caff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:25.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.824+0000 7f6ec241a700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6ebc0cb530 0x7f6ebc15da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:25.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.824+0000 7f6ec241a700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ebc0cb9b0 con 0x7f6ebc157450 2023-05-20T01:28:25.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.824+0000 7f6ec241a700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ebc15d5b0 con 0x7f6ebc071e10 2023-05-20T01:28:25.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.824+0000 7f6ec241a700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ebc157820 con 0x7f6ebc0cb530 2023-05-20T01:28:25.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.824+0000 7f6ebbfff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6ebc157450 0x7f6ebc159e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:25.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.824+0000 7f6ebbfff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6ebc157450 0x7f6ebc159e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39988/0 (socket says 172.21.15.16:39988) 2023-05-20T01:28:25.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.824+0000 7f6ebbfff700 1 -- 172.21.15.16:0/3461867218 learned_addr learned my addr 172.21.15.16:0/3461867218 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:25.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.825+0000 7f6ebb7fe700 1 --2- 172.21.15.16:0/3461867218 >> v2:172.21.15.132:3300/0 conn(0x7f6ebc071e10 0x7f6ebc0caff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:25.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.825+0000 7f6ec09b7700 1 --2- 172.21.15.16:0/3461867218 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6ebc0cb530 0x7f6ebc15da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:25.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.825+0000 7f6ebbfff700 1 -- 172.21.15.16:0/3461867218 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6ebc0cb530 msgr2=0x7f6ebc15da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:25.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.825+0000 7f6ebbfff700 1 --2- 172.21.15.16:0/3461867218 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6ebc0cb530 0x7f6ebc15da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:25.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.825+0000 7f6ebbfff700 1 -- 172.21.15.16:0/3461867218 >> v2:172.21.15.132:3300/0 conn(0x7f6ebc071e10 msgr2=0x7f6ebc0caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:25.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.825+0000 7f6ebbfff700 1 --2- 172.21.15.16:0/3461867218 >> v2:172.21.15.132:3300/0 conn(0x7f6ebc071e10 0x7f6ebc0caff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:25.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.825+0000 7f6ebbfff700 1 -- 172.21.15.16:0/3461867218 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ebc15a4c0 con 0x7f6ebc157450 2023-05-20T01:28:25.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.825+0000 7f6ebbfff700 1 --2- 172.21.15.16:0/3461867218 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ebc157450 0x7f6ebc159e40 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f6eac0098f0 tx=0x7f6eac009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c33cf1959f40ae78 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:25.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.825+0000 7f6ebaffd700 1 -- 172.21.15.16:0/3461867218 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eac00b860 con 0x7f6ebc157450 2023-05-20T01:28:25.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.825+0000 7f6ebaffd700 1 -- 172.21.15.16:0/3461867218 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6eac00b9c0 con 0x7f6ebc157450 2023-05-20T01:28:25.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.825+0000 7f6ebaffd700 1 -- 172.21.15.16:0/3461867218 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eac00bd50 con 0x7f6ebc157450 2023-05-20T01:28:25.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.826+0000 7f6ec241a700 1 -- 172.21.15.16:0/3461867218 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ebc157450 msgr2=0x7f6ebc159e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:25.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.826+0000 7f6ec241a700 1 --2- 172.21.15.16:0/3461867218 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ebc157450 0x7f6ebc159e40 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f6eac0098f0 tx=0x7f6eac009c00 comp rx=0 tx=0).stop 2023-05-20T01:28:25.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.826+0000 7f6ec241a700 1 -- 172.21.15.16:0/3461867218 shutdown_connections 2023-05-20T01:28:25.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.826+0000 7f6ec241a700 1 --2- 172.21.15.16:0/3461867218 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6ebc0cb530 0x7f6ebc15da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:25.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.826+0000 7f6ec241a700 1 --2- 172.21.15.16:0/3461867218 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ebc157450 0x7f6ebc159e40 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:25.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.826+0000 7f6ec241a700 1 --2- 172.21.15.16:0/3461867218 >> v2:172.21.15.132:3300/0 conn(0x7f6ebc071e10 0x7f6ebc0caff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:25.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.826+0000 7f6ec241a700 1 -- 172.21.15.16:0/3461867218 >> 172.21.15.16:0/3461867218 conn(0x7f6ebc06cbb0 msgr2=0x7f6ebc06cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:25.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.826+0000 7f6ec241a700 1 -- 172.21.15.16:0/3461867218 shutdown_connections 2023-05-20T01:28:25.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.826+0000 7f6ec241a700 1 -- 172.21.15.16:0/3461867218 wait complete. 2023-05-20T01:28:25.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.827+0000 7f6ec241a700 1 Processor -- start 2023-05-20T01:28:25.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.827+0000 7f6ec241a700 1 -- start start 2023-05-20T01:28:25.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.828+0000 7f6ec241a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ebc071e10 0x7f6ebc14c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:25.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.828+0000 7f6ec241a700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6ebc0cb530 0x7f6ebc0cc230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:25.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.828+0000 7f6ec241a700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ebc157450 0x7f6ebc0cc770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:25.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.828+0000 7f6ec241a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ebc14c8e0 con 0x7f6ebc071e10 2023-05-20T01:28:25.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.828+0000 7f6ec241a700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ebc0c6a90 con 0x7f6ebc157450 2023-05-20T01:28:25.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.828+0000 7f6ec241a700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ebc0b2930 con 0x7f6ebc0cb530 2023-05-20T01:28:25.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.828+0000 7f6ebbfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ebc071e10 0x7f6ebc14c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:25.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.828+0000 7f6ebbfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ebc071e10 0x7f6ebc14c3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39994/0 (socket says 172.21.15.16:39994) 2023-05-20T01:28:25.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.828+0000 7f6ebbfff700 1 -- 172.21.15.16:0/2833032371 learned_addr learned my addr 172.21.15.16:0/2833032371 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:25.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.828+0000 7f6ebb7fe700 1 --2- 172.21.15.16:0/2833032371 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6ebc0cb530 0x7f6ebc0cc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:25.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.828+0000 7f6ec09b7700 1 --2- 172.21.15.16:0/2833032371 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ebc157450 0x7f6ebc0cc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:25.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.828+0000 7f6ebbfff700 1 -- 172.21.15.16:0/2833032371 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6ebc0cb530 msgr2=0x7f6ebc0cc230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:25.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.828+0000 7f6ebbfff700 1 --2- 172.21.15.16:0/2833032371 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6ebc0cb530 0x7f6ebc0cc230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:25.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.828+0000 7f6ebbfff700 1 -- 172.21.15.16:0/2833032371 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ebc157450 msgr2=0x7f6ebc0cc770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:25.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.828+0000 7f6ebbfff700 1 --2- 172.21.15.16:0/2833032371 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ebc157450 0x7f6ebc0cc770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:25.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.828+0000 7f6ebbfff700 1 -- 172.21.15.16:0/2833032371 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6eac009550 con 0x7f6ebc071e10 2023-05-20T01:28:25.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.828+0000 7f6ec09b7700 1 --2- 172.21.15.16:0/2833032371 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ebc157450 0x7f6ebc0cc770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:28:25.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.828+0000 7f6ebbfff700 1 --2- 172.21.15.16:0/2833032371 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ebc071e10 0x7f6ebc14c3a0 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f6eac014040 tx=0x7f6eac004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:25.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.829+0000 7f6eb8ff9700 1 -- 172.21.15.16:0/2833032371 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eac004450 con 0x7f6ebc071e10 2023-05-20T01:28:25.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.829+0000 7f6eb8ff9700 1 -- 172.21.15.16:0/2833032371 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6eac01e070 con 0x7f6ebc071e10 2023-05-20T01:28:25.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.829+0000 7f6eb8ff9700 1 -- 172.21.15.16:0/2833032371 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eac0038e0 con 0x7f6ebc071e10 2023-05-20T01:28:25.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.829+0000 7f6ec241a700 1 -- 172.21.15.16:0/2833032371 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ebc15aa80 con 0x7f6ebc071e10 2023-05-20T01:28:25.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.829+0000 7f6ec241a700 1 -- 172.21.15.16:0/2833032371 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ebc15a7a0 con 0x7f6ebc071e10 2023-05-20T01:28:25.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.830+0000 7f6eb27fc700 1 -- 172.21.15.16:0/2833032371 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e940052e0 con 0x7f6ebc071e10 2023-05-20T01:28:25.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.830+0000 7f6eb8ff9700 1 -- 172.21.15.16:0/2833032371 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6eac003a40 con 0x7f6ebc071e10 2023-05-20T01:28:25.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.831+0000 7f6eb8ff9700 1 --2- 172.21.15.16:0/2833032371 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f6e9c06ace0 0x7f6e9c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:25.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.831+0000 7f6eb8ff9700 1 -- 172.21.15.16:0/2833032371 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 5900+0+0 (secure 0 0 0) 0x7f6eac020070 con 0x7f6ebc071e10 2023-05-20T01:28:25.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.831+0000 7f6ebb7fe700 1 --2- 172.21.15.16:0/2833032371 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f6e9c06ace0 0x7f6e9c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:25.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.831+0000 7f6ebb7fe700 1 --2- 172.21.15.16:0/2833032371 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f6e9c06ace0 0x7f6e9c06d190 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc153b70 tx=0x7f6ea4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:25.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:25.835+0000 7f6eb8ff9700 1 -- 172.21.15.16:0/2833032371 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f6eac0558e0 con 0x7f6ebc071e10 2023-05-20T01:28:26.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:26.036+0000 7f6eb27fc700 1 -- 172.21.15.16:0/2833032371 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f6e94005590 con 0x7f6ebc071e10 2023-05-20T01:28:26.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:26.671+0000 7f6eb8ff9700 1 -- 172.21.15.16:0/2833032371 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v134) v1 ==== 170+0+0 (secure 0 0 0) 0x7f6eac0243e0 con 0x7f6ebc071e10 2023-05-20T01:28:26.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:26.676+0000 7f6eb27fc700 1 -- 172.21.15.16:0/2833032371 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f6e9c06ace0 msgr2=0x7f6e9c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:26.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:26.676+0000 7f6eb27fc700 1 --2- 172.21.15.16:0/2833032371 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f6e9c06ace0 0x7f6e9c06d190 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc153b70 tx=0x7f6ea4008040 comp rx=0 tx=0).stop 2023-05-20T01:28:26.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:26.676+0000 7f6eb27fc700 1 -- 172.21.15.16:0/2833032371 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ebc071e10 msgr2=0x7f6ebc14c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:26.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:26.676+0000 7f6eb27fc700 1 --2- 172.21.15.16:0/2833032371 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ebc071e10 0x7f6ebc14c3a0 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f6eac014040 tx=0x7f6eac004750 comp rx=0 tx=0).stop 2023-05-20T01:28:26.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:26.676+0000 7f6eb27fc700 1 -- 172.21.15.16:0/2833032371 shutdown_connections 2023-05-20T01:28:26.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:26.676+0000 7f6eb27fc700 1 --2- 172.21.15.16:0/2833032371 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ebc157450 0x7f6ebc0cc770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:26.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:26.676+0000 7f6eb27fc700 1 --2- 172.21.15.16:0/2833032371 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ebc071e10 0x7f6ebc14c3a0 secure :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f6eac014040 tx=0x7f6eac004750 comp rx=0 tx=0).stop 2023-05-20T01:28:26.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:26.676+0000 7f6eb27fc700 1 --2- 172.21.15.16:0/2833032371 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f6e9c06ace0 0x7f6e9c06d190 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:26.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:26.676+0000 7f6eb27fc700 1 --2- 172.21.15.16:0/2833032371 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6ebc0cb530 0x7f6ebc0cc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:26.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:26.677+0000 7f6eb27fc700 1 -- 172.21.15.16:0/2833032371 >> 172.21.15.16:0/2833032371 conn(0x7f6ebc06cbb0 msgr2=0x7f6ebc06e150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:26.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:26.680+0000 7f6eb27fc700 1 -- 172.21.15.16:0/2833032371 shutdown_connections 2023-05-20T01:28:26.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:26.680+0000 7f6eb27fc700 1 -- 172.21.15.16:0/2833032371 wait complete. 2023-05-20T01:28:26.700 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' created 2023-05-20T01:28:26.709 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-05-20T01:28:26.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:26.838+0000 7f0885326700 1 Processor -- start 2023-05-20T01:28:26.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:26.838+0000 7f0885326700 1 -- start start 2023-05-20T01:28:26.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:26.839+0000 7f0885326700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f08800cd890 0x7f08800b6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:26.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:26.839+0000 7f0885326700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f08800b65d0 0x7f08800cf570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:26.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:26.839+0000 7f0885326700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08800d1820 con 0x7f088015ca90 2023-05-20T01:28:26.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:26.839+0000 7f0885326700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08800d0c50 con 0x7f08800b65d0 2023-05-20T01:28:26.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:26.839+0000 7f0885326700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08800cdce0 con 0x7f08800cd890 2023-05-20T01:28:26.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:26.839+0000 7f087effd700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f088015ca90 0x7f08800d2550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51530/0 (socket says 172.21.15.16:51530) 2023-05-20T01:28:26.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:26.839+0000 7f087effd700 1 -- 172.21.15.16:0/590415283 learned_addr learned my addr 172.21.15.16:0/590415283 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:28:26.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:26.839+0000 7f087f7fe700 1 --2- 172.21.15.16:0/590415283 >> v2:172.21.15.16:3300/0 conn(0x7f08800b65d0 0x7f08800cf570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:26.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:26.839+0000 7f087e7fc700 1 --2- 172.21.15.16:0/590415283 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f08800cd890 0x7f08800b6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:26.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:26.839+0000 7f087dffb700 1 -- 172.21.15.16:0/590415283 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1880565495 0 0) 0x7f08800d1820 con 0x7f088015ca90 2023-05-20T01:28:26.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:26.839+0000 7f087dffb700 1 -- 172.21.15.16:0/590415283 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f08680035b0 con 0x7f088015ca90 2023-05-20T01:28:26.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:26.839+0000 7f087f7fe700 1 -- 172.21.15.16:0/590415283 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f08800cd890 msgr2=0x7f08800b6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:26.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:26.839+0000 7f087f7fe700 1 --2- 172.21.15.16:0/590415283 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f08800cd890 0x7f08800b6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:26.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:26.839+0000 7f087f7fe700 1 -- 172.21.15.16:0/590415283 >> v1:172.21.15.16:6789/0 conn(0x7f088015ca90 legacy=0x7f08800d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:26.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:26.839+0000 7f087f7fe700 1 -- 172.21.15.16:0/590415283 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08800d2c60 con 0x7f08800b65d0 2023-05-20T01:28:26.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:26.839+0000 7f087e7fc700 1 --2- 172.21.15.16:0/590415283 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f08800cd890 0x7f08800b6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:28:26.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:26.839+0000 7f087f7fe700 1 --2- 172.21.15.16:0/590415283 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08800b65d0 0x7f08800cf570 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f0878009d90 tx=0x7f087800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bf0cdd7acf21b49 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:26.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:26.840+0000 7f087dffb700 1 -- 172.21.15.16:0/590415283 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f087800e070 con 0x7f08800b65d0 2023-05-20T01:28:26.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:26.840+0000 7f087dffb700 1 -- 172.21.15.16:0/590415283 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f087800bb20 con 0x7f08800b65d0 2023-05-20T01:28:26.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:26.840+0000 7f087dffb700 1 -- 172.21.15.16:0/590415283 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f087800be90 con 0x7f08800b65d0 2023-05-20T01:28:26.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:26.840+0000 7f0885326700 1 -- 172.21.15.16:0/590415283 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08800b65d0 msgr2=0x7f08800cf570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:26.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:26.840+0000 7f0885326700 1 --2- 172.21.15.16:0/590415283 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08800b65d0 0x7f08800cf570 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f0878009d90 tx=0x7f087800b330 comp rx=0 tx=0).stop 2023-05-20T01:28:26.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:26.840+0000 7f0885326700 1 -- 172.21.15.16:0/590415283 shutdown_connections 2023-05-20T01:28:26.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:26.840+0000 7f0885326700 1 --2- 172.21.15.16:0/590415283 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08800b65d0 0x7f08800cf570 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:26.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:26.840+0000 7f0885326700 1 --2- 172.21.15.16:0/590415283 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f08800cd890 0x7f08800b6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:26.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:26.840+0000 7f0885326700 1 -- 172.21.15.16:0/590415283 >> 172.21.15.16:0/590415283 conn(0x7f088006b000 msgr2=0x7f088006b400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:26.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:26.840+0000 7f0885326700 1 -- 172.21.15.16:0/590415283 shutdown_connections 2023-05-20T01:28:26.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:26.840+0000 7f0885326700 1 -- 172.21.15.16:0/590415283 wait complete. 2023-05-20T01:28:26.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:26.842+0000 7f0885326700 1 Processor -- start 2023-05-20T01:28:26.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:26.842+0000 7f0885326700 1 -- start start 2023-05-20T01:28:26.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:26.843+0000 7f0885326700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08800b65d0 0x7f08801f1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:26.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:26.843+0000 7f0885326700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f08800cd890 0x7f08800c97d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:26.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:26.843+0000 7f0885326700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f088015ca90 0x7f08800b3d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:26.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:26.843+0000 7f0885326700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08800c8730 con 0x7f08800b65d0 2023-05-20T01:28:26.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:26.843+0000 7f0885326700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08800c8a10 con 0x7f088015ca90 2023-05-20T01:28:26.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:26.843+0000 7f0885326700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08800c8cf0 con 0x7f08800cd890 2023-05-20T01:28:26.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:26.843+0000 7f087effd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08800b65d0 0x7f08801f1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:26.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:26.843+0000 7f087effd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08800b65d0 0x7f08801f1bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40008/0 (socket says 172.21.15.16:40008) 2023-05-20T01:28:26.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:26.843+0000 7f087effd700 1 -- 172.21.15.16:0/3401686050 learned_addr learned my addr 172.21.15.16:0/3401686050 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:26.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:26.843+0000 7f087f7fe700 1 --2- 172.21.15.16:0/3401686050 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f088015ca90 0x7f08800b3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:26.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:26.843+0000 7f087e7fc700 1 --2- 172.21.15.16:0/3401686050 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f08800cd890 0x7f08800c97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:26.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:26.843+0000 7f087effd700 1 -- 172.21.15.16:0/3401686050 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f08800cd890 msgr2=0x7f08800c97d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:26.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:26.843+0000 7f087effd700 1 --2- 172.21.15.16:0/3401686050 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f08800cd890 0x7f08800c97d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:26.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:26.843+0000 7f087effd700 1 -- 172.21.15.16:0/3401686050 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f088015ca90 msgr2=0x7f08800b3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:26.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:26.843+0000 7f087effd700 1 --2- 172.21.15.16:0/3401686050 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f088015ca90 0x7f08800b3d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:26.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:26.843+0000 7f087effd700 1 -- 172.21.15.16:0/3401686050 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0878009a70 con 0x7f08800b65d0 2023-05-20T01:28:26.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:26.844+0000 7f087effd700 1 --2- 172.21.15.16:0/3401686050 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08800b65d0 0x7f08801f1bd0 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f087000c4d0 tx=0x7f087000c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:26.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:26.844+0000 7f08677fe700 1 -- 172.21.15.16:0/3401686050 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0870005d70 con 0x7f08800b65d0 2023-05-20T01:28:26.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:26.844+0000 7f08677fe700 1 -- 172.21.15.16:0/3401686050 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08700070d0 con 0x7f08800b65d0 2023-05-20T01:28:26.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:26.844+0000 7f0885326700 1 -- 172.21.15.16:0/3401686050 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08800cdf00 con 0x7f08800b65d0 2023-05-20T01:28:26.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:26.844+0000 7f08677fe700 1 -- 172.21.15.16:0/3401686050 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0870007440 con 0x7f08800b65d0 2023-05-20T01:28:26.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:26.844+0000 7f0885326700 1 -- 172.21.15.16:0/3401686050 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08800b6a50 con 0x7f08800b65d0 2023-05-20T01:28:26.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:26.845+0000 7f0885326700 1 -- 172.21.15.16:0/3401686050 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08800d1a70 con 0x7f08800b65d0 2023-05-20T01:28:26.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:26.845+0000 7f08677fe700 1 -- 172.21.15.16:0/3401686050 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f08700075a0 con 0x7f08800b65d0 2023-05-20T01:28:26.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:26.846+0000 7f08677fe700 1 --2- 172.21.15.16:0/3401686050 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f086c06ace0 0x7f086c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:26.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:26.846+0000 7f087e7fc700 1 --2- 172.21.15.16:0/3401686050 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f086c06ace0 0x7f086c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:26.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:26.846+0000 7f08677fe700 1 -- 172.21.15.16:0/3401686050 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6282+0+0 (secure 0 0 0) 0x7f08700887c0 con 0x7f08800b65d0 2023-05-20T01:28:26.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:26.846+0000 7f087e7fc700 1 --2- 172.21.15.16:0/3401686050 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f086c06ace0 0x7f086c06d190 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f08800b83f0 tx=0x7f0874008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:26.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:26.851+0000 7f08677fe700 1 -- 172.21.15.16:0/3401686050 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f08700538d0 con 0x7f08800b65d0 2023-05-20T01:28:27.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.092+0000 7f0885326700 1 -- 172.21.15.16:0/3401686050 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f088004e8c0 con 0x7f08800b65d0 2023-05-20T01:28:27.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.098+0000 7f08677fe700 1 -- 172.21.15.16:0/3401686050 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 18 and data pool 19 v95) v1 ==== 174+0+0 (secure 0 0 0) 0x7f0870056ef0 con 0x7f08800b65d0 2023-05-20T01:28:27.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.106+0000 7f0885326700 1 -- 172.21.15.16:0/3401686050 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f086c06ace0 msgr2=0x7f086c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:27.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.106+0000 7f0885326700 1 --2- 172.21.15.16:0/3401686050 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f086c06ace0 0x7f086c06d190 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f08800b83f0 tx=0x7f0874008040 comp rx=0 tx=0).stop 2023-05-20T01:28:27.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.106+0000 7f0885326700 1 -- 172.21.15.16:0/3401686050 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08800b65d0 msgr2=0x7f08801f1bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:27.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.106+0000 7f0885326700 1 --2- 172.21.15.16:0/3401686050 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08800b65d0 0x7f08801f1bd0 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f087000c4d0 tx=0x7f087000c7e0 comp rx=0 tx=0).stop 2023-05-20T01:28:27.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.108+0000 7f0885326700 1 -- 172.21.15.16:0/3401686050 shutdown_connections 2023-05-20T01:28:27.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.108+0000 7f0885326700 1 --2- 172.21.15.16:0/3401686050 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f088015ca90 0x7f08800b3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:27.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.108+0000 7f0885326700 1 --2- 172.21.15.16:0/3401686050 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08800b65d0 0x7f08801f1bd0 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:27.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.108+0000 7f0885326700 1 --2- 172.21.15.16:0/3401686050 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f086c06ace0 0x7f086c06d190 secure :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f08800b83f0 tx=0x7f0874008040 comp rx=0 tx=0).stop 2023-05-20T01:28:27.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.108+0000 7f0885326700 1 --2- 172.21.15.16:0/3401686050 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f08800cd890 0x7f08800c97d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:27.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.108+0000 7f0885326700 1 -- 172.21.15.16:0/3401686050 >> 172.21.15.16:0/3401686050 conn(0x7f088006b000 msgr2=0x7f088015a120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:27.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.109+0000 7f0885326700 1 -- 172.21.15.16:0/3401686050 shutdown_connections 2023-05-20T01:28:27.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.115+0000 7f0885326700 1 -- 172.21.15.16:0/3401686050 wait complete. 2023-05-20T01:28:27.125 INFO:teuthology.orchestra.run.smithi016.stderr:new fs with metadata pool 18 and data pool 19 2023-05-20T01:28:27.139 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-20T01:28:27.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.259+0000 7fd2d3052700 1 Processor -- start 2023-05-20T01:28:27.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.259+0000 7fd2d3052700 1 -- start start 2023-05-20T01:28:27.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.260+0000 7fd2d3052700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fd2cc06d4b0 0x7fd2cc0cf890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:27.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.260+0000 7fd2d3052700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd2cc06a510 0x7fd2cc06a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:27.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.260+0000 7fd2d3052700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2cc0cc510 con 0x7fd2cc06f450 2023-05-20T01:28:27.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.260+0000 7fd2d3052700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2cc06aed0 con 0x7fd2cc06d4b0 2023-05-20T01:28:27.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.260+0000 7fd2d3052700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2cc071ed0 con 0x7fd2cc06a510 2023-05-20T01:28:27.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.260+0000 7fd2d0dee700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fd2cc06f450 0x7fd2cc161080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51560/0 (socket says 172.21.15.16:51560) 2023-05-20T01:28:27.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.260+0000 7fd2d0dee700 1 -- 172.21.15.16:0/3702499940 learned_addr learned my addr 172.21.15.16:0/3702499940 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:28:27.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.261+0000 7fd2cbfff700 1 --2- 172.21.15.16:0/3702499940 >> v2:172.21.15.132:3300/0 conn(0x7fd2cc06d4b0 0x7fd2cc0cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:27.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.261+0000 7fd2d15ef700 1 --2- 172.21.15.16:0/3702499940 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd2cc06a510 0x7fd2cc06a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:27.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.261+0000 7fd2cb7fe700 1 -- 172.21.15.16:0/3702499940 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2339687379 0 0) 0x7fd2cc0cc510 con 0x7fd2cc06f450 2023-05-20T01:28:27.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.261+0000 7fd2cb7fe700 1 -- 172.21.15.16:0/3702499940 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2bc0035b0 con 0x7fd2cc06f450 2023-05-20T01:28:27.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.261+0000 7fd2d15ef700 1 -- 172.21.15.16:0/3702499940 >> v2:172.21.15.132:3300/0 conn(0x7fd2cc06d4b0 msgr2=0x7fd2cc0cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:27.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.261+0000 7fd2d15ef700 1 --2- 172.21.15.16:0/3702499940 >> v2:172.21.15.132:3300/0 conn(0x7fd2cc06d4b0 0x7fd2cc0cf890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:27.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.261+0000 7fd2d15ef700 1 -- 172.21.15.16:0/3702499940 >> v1:172.21.15.16:6789/0 conn(0x7fd2cc06f450 legacy=0x7fd2cc161080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:27.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.261+0000 7fd2cb7fe700 1 -- 172.21.15.16:0/3702499940 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3456840850 0 0) 0x7fd2bc0035b0 con 0x7fd2cc06f450 2023-05-20T01:28:27.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.261+0000 7fd2d15ef700 1 -- 172.21.15.16:0/3702499940 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2cc0721b0 con 0x7fd2cc06a510 2023-05-20T01:28:27.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.261+0000 7fd2d15ef700 1 --2- 172.21.15.16:0/3702499940 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd2cc06a510 0x7fd2cc06a990 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd2c4009d90 tx=0x7fd2c400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73011859cdc8d63a server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:27.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.261+0000 7fd2cb7fe700 1 -- 172.21.15.16:0/3702499940 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c400e070 con 0x7fd2cc06a510 2023-05-20T01:28:27.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.261+0000 7fd2cb7fe700 1 -- 172.21.15.16:0/3702499940 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2c4003a20 con 0x7fd2cc06a510 2023-05-20T01:28:27.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.261+0000 7fd2cb7fe700 1 -- 172.21.15.16:0/3702499940 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c0001670 con 0x7fd2cc06a510 2023-05-20T01:28:27.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.262+0000 7fd2d3052700 1 -- 172.21.15.16:0/3702499940 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd2cc06a510 msgr2=0x7fd2cc06a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:27.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.262+0000 7fd2d3052700 1 --2- 172.21.15.16:0/3702499940 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd2cc06a510 0x7fd2cc06a990 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd2c4009d90 tx=0x7fd2c400b330 comp rx=0 tx=0).stop 2023-05-20T01:28:27.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.262+0000 7fd2d3052700 1 -- 172.21.15.16:0/3702499940 shutdown_connections 2023-05-20T01:28:27.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.262+0000 7fd2d3052700 1 --2- 172.21.15.16:0/3702499940 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd2cc06a510 0x7fd2cc06a990 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:27.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.262+0000 7fd2d3052700 1 --2- 172.21.15.16:0/3702499940 >> v2:172.21.15.132:3300/0 conn(0x7fd2cc06d4b0 0x7fd2cc0cf890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:27.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.262+0000 7fd2d3052700 1 -- 172.21.15.16:0/3702499940 >> 172.21.15.16:0/3702499940 conn(0x7fd2cc0cffb0 msgr2=0x7fd2cc0d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:27.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.262+0000 7fd2d3052700 1 -- 172.21.15.16:0/3702499940 shutdown_connections 2023-05-20T01:28:27.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.262+0000 7fd2d3052700 1 -- 172.21.15.16:0/3702499940 wait complete. 2023-05-20T01:28:27.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.263+0000 7fd2d3052700 1 Processor -- start 2023-05-20T01:28:27.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.263+0000 7fd2d3052700 1 -- start start 2023-05-20T01:28:27.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.264+0000 7fd2d3052700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd2cc06a510 0x7fd2cc0c6560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:27.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.264+0000 7fd2d3052700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd2cc06d4b0 0x7fd2cc151330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:27.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.264+0000 7fd2d3052700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2cc06f450 0x7fd2cc0c44f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:27.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.264+0000 7fd2d3052700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2cc0c54c0 con 0x7fd2cc06f450 2023-05-20T01:28:27.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.264+0000 7fd2d3052700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2cc0c57a0 con 0x7fd2cc06d4b0 2023-05-20T01:28:27.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.264+0000 7fd2d3052700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2cc0c5a80 con 0x7fd2cc06a510 2023-05-20T01:28:27.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.264+0000 7fd2d15ef700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2cc06f450 0x7fd2cc0c44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:27.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.264+0000 7fd2d15ef700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2cc06f450 0x7fd2cc0c44f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40042/0 (socket says 172.21.15.16:40042) 2023-05-20T01:28:27.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.264+0000 7fd2d15ef700 1 -- 172.21.15.16:0/2375205 learned_addr learned my addr 172.21.15.16:0/2375205 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:27.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.264+0000 7fd2d0dee700 1 --2- 172.21.15.16:0/2375205 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd2cc06a510 0x7fd2cc0c6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:27.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.264+0000 7fd2d15ef700 1 -- 172.21.15.16:0/2375205 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd2cc06a510 msgr2=0x7fd2cc0c6560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:27.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.264+0000 7fd2d15ef700 1 --2- 172.21.15.16:0/2375205 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd2cc06a510 0x7fd2cc0c6560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:27.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.264+0000 7fd2cbfff700 1 --2- 172.21.15.16:0/2375205 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd2cc06d4b0 0x7fd2cc151330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:27.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.264+0000 7fd2d15ef700 1 -- 172.21.15.16:0/2375205 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd2cc06d4b0 msgr2=0x7fd2cc151330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:27.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.264+0000 7fd2d15ef700 1 --2- 172.21.15.16:0/2375205 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd2cc06d4b0 0x7fd2cc151330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:27.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.264+0000 7fd2d15ef700 1 -- 172.21.15.16:0/2375205 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2c4009a70 con 0x7fd2cc06f450 2023-05-20T01:28:27.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.265+0000 7fd2d15ef700 1 --2- 172.21.15.16:0/2375205 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2cc06f450 0x7fd2cc0c44f0 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7fd2cc06b010 tx=0x7fd2c4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:27.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.265+0000 7fd2c97fa700 1 -- 172.21.15.16:0/2375205 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c40045c0 con 0x7fd2cc06f450 2023-05-20T01:28:27.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.265+0000 7fd2c97fa700 1 -- 172.21.15.16:0/2375205 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2c4004740 con 0x7fd2cc06f450 2023-05-20T01:28:27.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.265+0000 7fd2d3052700 1 -- 172.21.15.16:0/2375205 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2cc0cbe50 con 0x7fd2cc06f450 2023-05-20T01:28:27.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.265+0000 7fd2c97fa700 1 -- 172.21.15.16:0/2375205 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c4012430 con 0x7fd2cc06f450 2023-05-20T01:28:27.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.265+0000 7fd2d3052700 1 -- 172.21.15.16:0/2375205 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2cc0cc1a0 con 0x7fd2cc06f450 2023-05-20T01:28:27.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.266+0000 7fd2d3052700 1 -- 172.21.15.16:0/2375205 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2cc0cc9a0 con 0x7fd2cc06f450 2023-05-20T01:28:27.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.266+0000 7fd2c97fa700 1 -- 172.21.15.16:0/2375205 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd2c40125b0 con 0x7fd2cc06f450 2023-05-20T01:28:27.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.267+0000 7fd2c97fa700 1 --2- 172.21.15.16:0/2375205 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd2b006ace0 0x7fd2b006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:27.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.267+0000 7fd2c97fa700 1 -- 172.21.15.16:0/2375205 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fd2c408b430 con 0x7fd2cc06f450 2023-05-20T01:28:27.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.267+0000 7fd2d0dee700 1 --2- 172.21.15.16:0/2375205 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd2b006ace0 0x7fd2b006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:27.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.267+0000 7fd2d0dee700 1 --2- 172.21.15.16:0/2375205 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd2b006ace0 0x7fd2b006d190 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fd2c0001eb0 tx=0x7fd2c0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:27.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.271+0000 7fd2c97fa700 1 -- 172.21.15.16:0/2375205 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fd2c4059ec0 con 0x7fd2cc06f450 2023-05-20T01:28:27.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.471+0000 7fd2d3052700 1 -- 172.21.15.16:0/2375205 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd2cc0747b0 con 0x7fd2cc06f450 2023-05-20T01:28:27.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.472+0000 7fd2c97fa700 1 -- 172.21.15.16:0/2375205 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v135) v1 ==== 74+0+18485 (secure 0 0 0) 0x7fd2c4059a50 con 0x7fd2cc06f450 2023-05-20T01:28:27.477 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:28:27.477 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":135,"fsid":"c2245cc5-ecab-4087-a59b-bc81db901676","created":"2023-05-20T01:16:09.549230+0000","modified":"2023-05-20T01:28:27.093637+0000","last_up_change":"2023-05-20T01:16:19.788624+0000","last_in_change":"2023-05-20T01:16:12.092160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-05-20T01:16:20.934366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-05-20T01:16:21.372294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-05-20T01:28:25.274827+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"135","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-05-20T01:28:26.067827+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"135","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"317a83cd-19c8-4958-a301-6087ef342289","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6825","nonce":3923634179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6827","nonce":3923634179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6831","nonce":3923634179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6829","nonce":3923634179}]},"public_addr":"172.21.15.16:6825/3923634179","cluster_addr":"172.21.15.16:6827/3923634179","heartbeat_back_addr":"172.21.15.16:6831/3923634179","heartbeat_front_addr":"172.21.15.16:6829/3923634179","state":["exists","up"]},{"osd":1,"uuid":"5581046f-8219-4089-8d7d-b7846399505e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6809","nonce":3860264220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6811","nonce":3860264220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6815","nonce":3860264220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6813","nonce":3860264220}]},"public_addr":"172.21.15.16:6809/3860264220","cluster_addr":"172.21.15.16:6811/3860264220","heartbeat_back_addr":"172.21.15.16:6815/3860264220","heartbeat_front_addr":"172.21.15.16:6813/3860264220","state":["exists","up"]},{"osd":2,"uuid":"072e9e66-4133-478e-8d4a-64d46c2963f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6817","nonce":293187030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6819","nonce":293187030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6823","nonce":293187030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6821","nonce":293187030}]},"public_addr":"172.21.15.16:6817/293187030","cluster_addr":"172.21.15.16:6819/293187030","heartbeat_back_addr":"172.21.15.16:6823/293187030","heartbeat_front_addr":"172.21.15.16:6821/293187030","state":["exists","up"]},{"osd":3,"uuid":"11e1bba6-00f8-48dd-9028-d8d716b7f145","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6801","nonce":3669781890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6803","nonce":3669781890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6807","nonce":3669781890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6805","nonce":3669781890}]},"public_addr":"172.21.15.16:6801/3669781890","cluster_addr":"172.21.15.16:6803/3669781890","heartbeat_back_addr":"172.21.15.16:6807/3669781890","heartbeat_front_addr":"172.21.15.16:6805/3669781890","state":["exists","up"]},{"osd":4,"uuid":"b8909c3a-cf70-46cb-8d0b-009c707cf9d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6819","nonce":506317223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6823","nonce":506317223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6829","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6831","nonce":506317223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6827","nonce":506317223}]},"public_addr":"172.21.15.132:6819/506317223","cluster_addr":"172.21.15.132:6823/506317223","heartbeat_back_addr":"172.21.15.132:6831/506317223","heartbeat_front_addr":"172.21.15.132:6827/506317223","state":["exists","up"]},{"osd":5,"uuid":"b1818304-4583-47bf-a6ec-0bfbb5166233","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6818","nonce":1844689995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6822","nonce":1844689995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6830","nonce":1844689995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6826","nonce":1844689995}]},"public_addr":"172.21.15.132:6818/1844689995","cluster_addr":"172.21.15.132:6822/1844689995","heartbeat_back_addr":"172.21.15.132:6830/1844689995","heartbeat_front_addr":"172.21.15.132:6826/1844689995","state":["exists","up"]},{"osd":6,"uuid":"d37f4691-eed1-4a93-8818-18354f03f4a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6801","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6802","nonce":727581123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6805","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6807","nonce":727581123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6813","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6815","nonce":727581123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6811","nonce":727581123}]},"public_addr":"172.21.15.132:6802/727581123","cluster_addr":"172.21.15.132:6807/727581123","heartbeat_back_addr":"172.21.15.132:6815/727581123","heartbeat_front_addr":"172.21.15.132:6811/727581123","state":["exists","up"]},{"osd":7,"uuid":"4f712f62-5d1d-4816-8042-03209144f8b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6803","nonce":466001149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6806","nonce":466001149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6814","nonce":466001149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6810","nonce":466001149}]},"public_addr":"172.21.15.132:6803/466001149","cluster_addr":"172.21.15.132:6806/466001149","heartbeat_back_addr":"172.21.15.132:6814/466001149","heartbeat_front_addr":"172.21.15.132:6810/466001149","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.338863+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.278680+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305694+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.246835+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.276237+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.304441+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305421+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.267050+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-20T01:28:27.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.474+0000 7fd2d3052700 1 -- 172.21.15.16:0/2375205 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd2b006ace0 msgr2=0x7fd2b006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:27.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.474+0000 7fd2d3052700 1 --2- 172.21.15.16:0/2375205 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd2b006ace0 0x7fd2b006d190 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fd2c0001eb0 tx=0x7fd2c0009040 comp rx=0 tx=0).stop 2023-05-20T01:28:27.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.475+0000 7fd2d3052700 1 -- 172.21.15.16:0/2375205 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2cc06f450 msgr2=0x7fd2cc0c44f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:27.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.475+0000 7fd2d3052700 1 --2- 172.21.15.16:0/2375205 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2cc06f450 0x7fd2cc0c44f0 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7fd2cc06b010 tx=0x7fd2c4004ad0 comp rx=0 tx=0).stop 2023-05-20T01:28:27.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.475+0000 7fd2d3052700 1 -- 172.21.15.16:0/2375205 shutdown_connections 2023-05-20T01:28:27.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.475+0000 7fd2d3052700 1 --2- 172.21.15.16:0/2375205 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2cc06f450 0x7fd2cc0c44f0 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:27.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.475+0000 7fd2d3052700 1 --2- 172.21.15.16:0/2375205 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd2b006ace0 0x7fd2b006d190 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:27.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.475+0000 7fd2d3052700 1 --2- 172.21.15.16:0/2375205 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd2cc06a510 0x7fd2cc0c6560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:27.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.475+0000 7fd2d3052700 1 --2- 172.21.15.16:0/2375205 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd2cc06d4b0 0x7fd2cc151330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:27.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.475+0000 7fd2d3052700 1 -- 172.21.15.16:0/2375205 >> 172.21.15.16:0/2375205 conn(0x7fd2cc0cffb0 msgr2=0x7fd2cc0d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:27.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.475+0000 7fd2d3052700 1 -- 172.21.15.16:0/2375205 shutdown_connections 2023-05-20T01:28:27.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.475+0000 7fd2d3052700 1 -- 172.21.15.16:0/2375205 wait complete. 2023-05-20T01:28:27.492 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-20T01:28:27.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.614+0000 7f30fbaa0700 1 Processor -- start 2023-05-20T01:28:27.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.615+0000 7f30fbaa0700 1 -- start start 2023-05-20T01:28:27.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.615+0000 7f30fbaa0700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f30f40d16c0 0x7f30f4144330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:27.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.615+0000 7f30fbaa0700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30f40b1dd0 con 0x7f30f40b1990 2023-05-20T01:28:27.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.615+0000 7f30fbaa0700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30f4144870 con 0x7f30f40cec20 2023-05-20T01:28:27.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.615+0000 7f30fbaa0700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30f40b2000 con 0x7f30f40d16c0 2023-05-20T01:28:27.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.616+0000 7f30fa03d700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f30f40b1990 0x7f30f40ae9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51578/0 (socket says 172.21.15.16:51578) 2023-05-20T01:28:27.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.616+0000 7f30fa03d700 1 -- 172.21.15.16:0/4042502043 learned_addr learned my addr 172.21.15.16:0/4042502043 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:28:27.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.616+0000 7f30f983c700 1 --2- 172.21.15.16:0/4042502043 >> v2:172.21.15.132:3300/0 conn(0x7f30f40d16c0 0x7f30f4144330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:27.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.616+0000 7f30f883a700 1 -- 172.21.15.16:0/4042502043 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3665854616 0 0) 0x7f30f40b1dd0 con 0x7f30f40b1990 2023-05-20T01:28:27.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.616+0000 7f30f883a700 1 -- 172.21.15.16:0/4042502043 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f30e00035b0 con 0x7f30f40b1990 2023-05-20T01:28:27.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.616+0000 7f30f883a700 1 -- 172.21.15.16:0/4042502043 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1435300887 0 0) 0x7f30e00035b0 con 0x7f30f40b1990 2023-05-20T01:28:27.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.616+0000 7f30f983c700 1 -- 172.21.15.16:0/4042502043 >> v1:172.21.15.132:6789/0 conn(0x7f30f40cec20 legacy=0x7f30f40ae2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:27.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.616+0000 7f30f983c700 1 -- 172.21.15.16:0/4042502043 >> v1:172.21.15.16:6789/0 conn(0x7f30f40b1990 legacy=0x7f30f40ae9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:27.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.616+0000 7f30f983c700 1 -- 172.21.15.16:0/4042502043 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30f40b22e0 con 0x7f30f40d16c0 2023-05-20T01:28:27.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.616+0000 7f30f983c700 1 --2- 172.21.15.16:0/4042502043 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f30f40d16c0 0x7f30f4144330 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f30e8009870 tx=0x7f30e8009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e0775941f536b4b server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:27.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.616+0000 7f30f883a700 1 -- 172.21.15.16:0/4042502043 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30ec0019f0 con 0x7f30f40d16c0 2023-05-20T01:28:27.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.616+0000 7f30f883a700 1 -- 172.21.15.16:0/4042502043 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30e8004030 con 0x7f30f40d16c0 2023-05-20T01:28:27.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.616+0000 7f30f883a700 1 -- 172.21.15.16:0/4042502043 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30e80043c0 con 0x7f30f40d16c0 2023-05-20T01:28:27.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.617+0000 7f30fbaa0700 1 -- 172.21.15.16:0/4042502043 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f30f40d16c0 msgr2=0x7f30f4144330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:27.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.617+0000 7f30fbaa0700 1 --2- 172.21.15.16:0/4042502043 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f30f40d16c0 0x7f30f4144330 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f30e8009870 tx=0x7f30e8009b80 comp rx=0 tx=0).stop 2023-05-20T01:28:27.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.617+0000 7f30fbaa0700 1 -- 172.21.15.16:0/4042502043 shutdown_connections 2023-05-20T01:28:27.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.617+0000 7f30fbaa0700 1 --2- 172.21.15.16:0/4042502043 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f30f40d16c0 0x7f30f4144330 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:27.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.617+0000 7f30fbaa0700 1 -- 172.21.15.16:0/4042502043 >> 172.21.15.16:0/4042502043 conn(0x7f30f406ae30 msgr2=0x7f30f409a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:27.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.617+0000 7f30fbaa0700 1 -- 172.21.15.16:0/4042502043 shutdown_connections 2023-05-20T01:28:27.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.617+0000 7f30fbaa0700 1 -- 172.21.15.16:0/4042502043 wait complete. 2023-05-20T01:28:27.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.618+0000 7f30fbaa0700 1 Processor -- start 2023-05-20T01:28:27.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.618+0000 7f30fbaa0700 1 -- start start 2023-05-20T01:28:27.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.619+0000 7f30fbaa0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30f40b1990 0x7f30f40c44d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:27.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.619+0000 7f30fbaa0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f30f40cec20 0x7f30f40c8120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:27.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.619+0000 7f30fbaa0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f30f40d16c0 0x7f30f40c8660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:27.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.619+0000 7f30fbaa0700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30f40b6610 con 0x7f30f40b1990 2023-05-20T01:28:27.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.619+0000 7f30fbaa0700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30f40c51a0 con 0x7f30f40d16c0 2023-05-20T01:28:27.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.619+0000 7f30fbaa0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30f40c5480 con 0x7f30f40cec20 2023-05-20T01:28:27.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.619+0000 7f30f983c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30f40b1990 0x7f30f40c44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:27.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.619+0000 7f30f983c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30f40b1990 0x7f30f40c44d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40052/0 (socket says 172.21.15.16:40052) 2023-05-20T01:28:27.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.619+0000 7f30f983c700 1 -- 172.21.15.16:0/3330942912 learned_addr learned my addr 172.21.15.16:0/3330942912 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:27.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.619+0000 7f30f903b700 1 --2- 172.21.15.16:0/3330942912 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f30f40cec20 0x7f30f40c8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:27.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.619+0000 7f30fa03d700 1 --2- 172.21.15.16:0/3330942912 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f30f40d16c0 0x7f30f40c8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:27.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.619+0000 7f30f983c700 1 -- 172.21.15.16:0/3330942912 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f30f40cec20 msgr2=0x7f30f40c8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:27.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.619+0000 7f30f983c700 1 --2- 172.21.15.16:0/3330942912 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f30f40cec20 0x7f30f40c8120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:27.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.619+0000 7f30f983c700 1 -- 172.21.15.16:0/3330942912 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f30f40d16c0 msgr2=0x7f30f40c8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:27.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.619+0000 7f30f983c700 1 --2- 172.21.15.16:0/3330942912 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f30f40d16c0 0x7f30f40c8660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:27.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.619+0000 7f30f983c700 1 -- 172.21.15.16:0/3330942912 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30e80094a0 con 0x7f30f40b1990 2023-05-20T01:28:27.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.619+0000 7f30fa03d700 1 --2- 172.21.15.16:0/3330942912 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f30f40d16c0 0x7f30f40c8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:28:27.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.620+0000 7f30f983c700 1 --2- 172.21.15.16:0/3330942912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30f40b1990 0x7f30f40c44d0 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f30e8000c00 tx=0x7f30e80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:27.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.620+0000 7f30e67fc700 1 -- 172.21.15.16:0/3330942912 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30e8011830 con 0x7f30f40b1990 2023-05-20T01:28:27.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.620+0000 7f30e67fc700 1 -- 172.21.15.16:0/3330942912 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30e80119b0 con 0x7f30f40b1990 2023-05-20T01:28:27.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.620+0000 7f30fbaa0700 1 -- 172.21.15.16:0/3330942912 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30f40adda0 con 0x7f30f40b1990 2023-05-20T01:28:27.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.620+0000 7f30e67fc700 1 -- 172.21.15.16:0/3330942912 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30e8011d60 con 0x7f30f40b1990 2023-05-20T01:28:27.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.620+0000 7f30fbaa0700 1 -- 172.21.15.16:0/3330942912 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30f40b25c0 con 0x7f30f40b1990 2023-05-20T01:28:27.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.621+0000 7f30fbaa0700 1 -- 172.21.15.16:0/3330942912 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30f40b28a0 con 0x7f30f40b1990 2023-05-20T01:28:27.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.621+0000 7f30e67fc700 1 -- 172.21.15.16:0/3330942912 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f30e801b440 con 0x7f30f40b1990 2023-05-20T01:28:27.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.622+0000 7f30e67fc700 1 --2- 172.21.15.16:0/3330942912 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f30d806ace0 0x7f30d806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:27.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.622+0000 7f30f903b700 1 --2- 172.21.15.16:0/3330942912 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f30d806ace0 0x7f30d806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:27.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.622+0000 7f30e67fc700 1 -- 172.21.15.16:0/3330942912 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f30e808ac90 con 0x7f30f40b1990 2023-05-20T01:28:27.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.622+0000 7f30f903b700 1 --2- 172.21.15.16:0/3330942912 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f30d806ace0 0x7f30d806d190 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f30f40c76f0 tx=0x7f30f0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:27.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.626+0000 7f30e67fc700 1 -- 172.21.15.16:0/3330942912 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f30e8055c90 con 0x7f30f40b1990 2023-05-20T01:28:27.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.825+0000 7f30fbaa0700 1 -- 172.21.15.16:0/3330942912 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f30f40747b0 con 0x7f30f40b1990 2023-05-20T01:28:27.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.826+0000 7f30e67fc700 1 -- 172.21.15.16:0/3330942912 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v135) v1 ==== 74+0+18485 (secure 0 0 0) 0x7f30e80592b0 con 0x7f30f40b1990 2023-05-20T01:28:27.831 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:28:27.831 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":135,"fsid":"c2245cc5-ecab-4087-a59b-bc81db901676","created":"2023-05-20T01:16:09.549230+0000","modified":"2023-05-20T01:28:27.093637+0000","last_up_change":"2023-05-20T01:16:19.788624+0000","last_in_change":"2023-05-20T01:16:12.092160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-05-20T01:16:20.934366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-05-20T01:16:21.372294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-05-20T01:28:25.274827+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"135","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-05-20T01:28:26.067827+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"135","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"317a83cd-19c8-4958-a301-6087ef342289","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6825","nonce":3923634179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6827","nonce":3923634179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6831","nonce":3923634179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6829","nonce":3923634179}]},"public_addr":"172.21.15.16:6825/3923634179","cluster_addr":"172.21.15.16:6827/3923634179","heartbeat_back_addr":"172.21.15.16:6831/3923634179","heartbeat_front_addr":"172.21.15.16:6829/3923634179","state":["exists","up"]},{"osd":1,"uuid":"5581046f-8219-4089-8d7d-b7846399505e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6809","nonce":3860264220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6811","nonce":3860264220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6815","nonce":3860264220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6813","nonce":3860264220}]},"public_addr":"172.21.15.16:6809/3860264220","cluster_addr":"172.21.15.16:6811/3860264220","heartbeat_back_addr":"172.21.15.16:6815/3860264220","heartbeat_front_addr":"172.21.15.16:6813/3860264220","state":["exists","up"]},{"osd":2,"uuid":"072e9e66-4133-478e-8d4a-64d46c2963f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6817","nonce":293187030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6819","nonce":293187030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6823","nonce":293187030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6821","nonce":293187030}]},"public_addr":"172.21.15.16:6817/293187030","cluster_addr":"172.21.15.16:6819/293187030","heartbeat_back_addr":"172.21.15.16:6823/293187030","heartbeat_front_addr":"172.21.15.16:6821/293187030","state":["exists","up"]},{"osd":3,"uuid":"11e1bba6-00f8-48dd-9028-d8d716b7f145","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6801","nonce":3669781890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6803","nonce":3669781890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6807","nonce":3669781890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6805","nonce":3669781890}]},"public_addr":"172.21.15.16:6801/3669781890","cluster_addr":"172.21.15.16:6803/3669781890","heartbeat_back_addr":"172.21.15.16:6807/3669781890","heartbeat_front_addr":"172.21.15.16:6805/3669781890","state":["exists","up"]},{"osd":4,"uuid":"b8909c3a-cf70-46cb-8d0b-009c707cf9d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6819","nonce":506317223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6823","nonce":506317223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6829","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6831","nonce":506317223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6827","nonce":506317223}]},"public_addr":"172.21.15.132:6819/506317223","cluster_addr":"172.21.15.132:6823/506317223","heartbeat_back_addr":"172.21.15.132:6831/506317223","heartbeat_front_addr":"172.21.15.132:6827/506317223","state":["exists","up"]},{"osd":5,"uuid":"b1818304-4583-47bf-a6ec-0bfbb5166233","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6818","nonce":1844689995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6822","nonce":1844689995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6830","nonce":1844689995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6826","nonce":1844689995}]},"public_addr":"172.21.15.132:6818/1844689995","cluster_addr":"172.21.15.132:6822/1844689995","heartbeat_back_addr":"172.21.15.132:6830/1844689995","heartbeat_front_addr":"172.21.15.132:6826/1844689995","state":["exists","up"]},{"osd":6,"uuid":"d37f4691-eed1-4a93-8818-18354f03f4a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6801","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6802","nonce":727581123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6805","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6807","nonce":727581123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6813","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6815","nonce":727581123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6811","nonce":727581123}]},"public_addr":"172.21.15.132:6802/727581123","cluster_addr":"172.21.15.132:6807/727581123","heartbeat_back_addr":"172.21.15.132:6815/727581123","heartbeat_front_addr":"172.21.15.132:6811/727581123","state":["exists","up"]},{"osd":7,"uuid":"4f712f62-5d1d-4816-8042-03209144f8b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6803","nonce":466001149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6806","nonce":466001149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6814","nonce":466001149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6810","nonce":466001149}]},"public_addr":"172.21.15.132:6803/466001149","cluster_addr":"172.21.15.132:6806/466001149","heartbeat_back_addr":"172.21.15.132:6814/466001149","heartbeat_front_addr":"172.21.15.132:6810/466001149","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.338863+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.278680+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305694+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.246835+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.276237+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.304441+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305421+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.267050+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-20T01:28:27.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.828+0000 7f30fbaa0700 1 -- 172.21.15.16:0/3330942912 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f30d806ace0 msgr2=0x7f30d806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:27.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.828+0000 7f30fbaa0700 1 --2- 172.21.15.16:0/3330942912 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f30d806ace0 0x7f30d806d190 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f30f40c76f0 tx=0x7f30f0009040 comp rx=0 tx=0).stop 2023-05-20T01:28:27.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.828+0000 7f30fbaa0700 1 -- 172.21.15.16:0/3330942912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30f40b1990 msgr2=0x7f30f40c44d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:27.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.828+0000 7f30fbaa0700 1 --2- 172.21.15.16:0/3330942912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30f40b1990 0x7f30f40c44d0 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f30e8000c00 tx=0x7f30e80047c0 comp rx=0 tx=0).stop 2023-05-20T01:28:27.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.828+0000 7f30fbaa0700 1 -- 172.21.15.16:0/3330942912 shutdown_connections 2023-05-20T01:28:27.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.828+0000 7f30fbaa0700 1 --2- 172.21.15.16:0/3330942912 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f30f40d16c0 0x7f30f40c8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:27.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.828+0000 7f30fbaa0700 1 --2- 172.21.15.16:0/3330942912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30f40b1990 0x7f30f40c44d0 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:27.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.828+0000 7f30fbaa0700 1 --2- 172.21.15.16:0/3330942912 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f30d806ace0 0x7f30d806d190 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:27.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.828+0000 7f30fbaa0700 1 --2- 172.21.15.16:0/3330942912 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f30f40cec20 0x7f30f40c8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:27.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.828+0000 7f30fbaa0700 1 -- 172.21.15.16:0/3330942912 >> 172.21.15.16:0/3330942912 conn(0x7f30f406ae30 msgr2=0x7f30f409a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:27.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.828+0000 7f30fbaa0700 1 -- 172.21.15.16:0/3330942912 shutdown_connections 2023-05-20T01:28:27.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.828+0000 7f30fbaa0700 1 -- 172.21.15.16:0/3330942912 wait complete. 2023-05-20T01:28:27.845 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-05-20T01:28:27.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.970+0000 7fbf3699a700 1 Processor -- start 2023-05-20T01:28:27.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.970+0000 7fbf3699a700 1 -- start start 2023-05-20T01:28:27.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.971+0000 7fbf3699a700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbf30072300 0x7fbf300cf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:27.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.971+0000 7fbf3699a700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fbf3015d290 0x7fbf300cf800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:27.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.971+0000 7fbf3699a700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf300cd630 con 0x7fbf3015cec0 2023-05-20T01:28:27.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.971+0000 7fbf3699a700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf300cca60 con 0x7fbf3015d290 2023-05-20T01:28:27.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.971+0000 7fbf3699a700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf300ccd00 con 0x7fbf30072300 2023-05-20T01:28:27.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.971+0000 7fbf2ffff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fbf3015cec0 0x7fbf30071bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51588/0 (socket says 172.21.15.16:51588) 2023-05-20T01:28:27.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.971+0000 7fbf2ffff700 1 -- 172.21.15.16:0/266856234 learned_addr learned my addr 172.21.15.16:0/266856234 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:28:27.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.971+0000 7fbf2f7fe700 1 --2- 172.21.15.16:0/266856234 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbf30072300 0x7fbf300cf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:27.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.971+0000 7fbf34f37700 1 --2- 172.21.15.16:0/266856234 >> v2:172.21.15.132:3300/0 conn(0x7fbf3015d290 0x7fbf300cf800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:27.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.971+0000 7fbf2effd700 1 -- 172.21.15.16:0/266856234 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2237719671 0 0) 0x7fbf300cd630 con 0x7fbf3015cec0 2023-05-20T01:28:27.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.971+0000 7fbf2effd700 1 -- 172.21.15.16:0/266856234 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf180035b0 con 0x7fbf3015cec0 2023-05-20T01:28:27.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.971+0000 7fbf2effd700 1 -- 172.21.15.16:0/266856234 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4155533075 0 0) 0x7fbf180035b0 con 0x7fbf3015cec0 2023-05-20T01:28:27.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.971+0000 7fbf2effd700 1 -- 172.21.15.16:0/266856234 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbf30072300 msgr2=0x7fbf300cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:27.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.971+0000 7fbf2effd700 1 --2- 172.21.15.16:0/266856234 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbf30072300 0x7fbf300cf2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:27.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.971+0000 7fbf2effd700 1 -- 172.21.15.16:0/266856234 >> v2:172.21.15.132:3300/0 conn(0x7fbf3015d290 msgr2=0x7fbf300cf800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:27.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.971+0000 7fbf2effd700 1 --2- 172.21.15.16:0/266856234 >> v2:172.21.15.132:3300/0 conn(0x7fbf3015d290 0x7fbf300cf800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:27.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.971+0000 7fbf2effd700 1 -- 172.21.15.16:0/266856234 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf300ccfe0 con 0x7fbf3015cec0 2023-05-20T01:28:27.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.971+0000 7fbf2effd700 1 -- 172.21.15.16:0/266856234 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7fbf200017f0 con 0x7fbf3015cec0 2023-05-20T01:28:27.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.972+0000 7fbf2effd700 1 -- 172.21.15.16:0/266856234 >> v1:172.21.15.16:6789/0 conn(0x7fbf3015cec0 legacy=0x7fbf30071bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:27.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.972+0000 7fbf2effd700 1 --2- 172.21.15.16:0/266856234 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbf30072300 0x7fbf18003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:27.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.972+0000 7fbf2effd700 1 --2- 172.21.15.16:0/266856234 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbf18003f80 0x7fbf18004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:27.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.972+0000 7fbf2effd700 1 --2- 172.21.15.16:0/266856234 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf18004d90 0x7fbf18005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:27.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.972+0000 7fbf2effd700 1 -- 172.21.15.16:0/266856234 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf300cca60 con 0x7fbf18004d90 2023-05-20T01:28:27.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.972+0000 7fbf2effd700 1 -- 172.21.15.16:0/266856234 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf300ccd00 con 0x7fbf18003f80 2023-05-20T01:28:27.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.972+0000 7fbf2effd700 1 -- 172.21.15.16:0/266856234 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf1800ba40 con 0x7fbf30072300 2023-05-20T01:28:27.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.972+0000 7fbf34f37700 1 --2- 172.21.15.16:0/266856234 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf18004d90 0x7fbf18005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:27.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.972+0000 7fbf2ffff700 1 --2- 172.21.15.16:0/266856234 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbf30072300 0x7fbf18003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:27.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.972+0000 7fbf2f7fe700 1 --2- 172.21.15.16:0/266856234 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbf18003f80 0x7fbf18004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:27.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.972+0000 7fbf34f37700 1 -- 172.21.15.16:0/266856234 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbf30072300 msgr2=0x7fbf18003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:27.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.972+0000 7fbf34f37700 1 --2- 172.21.15.16:0/266856234 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbf30072300 0x7fbf18003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:27.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.972+0000 7fbf34f37700 1 -- 172.21.15.16:0/266856234 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbf18003f80 msgr2=0x7fbf18004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:27.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.972+0000 7fbf34f37700 1 --2- 172.21.15.16:0/266856234 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbf18003f80 0x7fbf18004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:27.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.972+0000 7fbf34f37700 1 -- 172.21.15.16:0/266856234 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf180035b0 con 0x7fbf18004d90 2023-05-20T01:28:27.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.972+0000 7fbf34f37700 1 --2- 172.21.15.16:0/266856234 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf18004d90 0x7fbf18005200 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7fbf28009fc0 tx=0x7fbf18005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf8c1e1111e6773b server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:27.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.972+0000 7fbf2effd700 1 -- 172.21.15.16:0/266856234 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf2800ebd0 con 0x7fbf18004d90 2023-05-20T01:28:27.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.972+0000 7fbf2effd700 1 -- 172.21.15.16:0/266856234 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf2800f040 con 0x7fbf18004d90 2023-05-20T01:28:27.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.972+0000 7fbf2effd700 1 -- 172.21.15.16:0/266856234 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf28014480 con 0x7fbf18004d90 2023-05-20T01:28:27.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.973+0000 7fbf3699a700 1 -- 172.21.15.16:0/266856234 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf18004d90 msgr2=0x7fbf18005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:27.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.973+0000 7fbf3699a700 1 --2- 172.21.15.16:0/266856234 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf18004d90 0x7fbf18005200 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7fbf28009fc0 tx=0x7fbf18005880 comp rx=0 tx=0).stop 2023-05-20T01:28:27.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.973+0000 7fbf34f37700 1 -- 172.21.15.16:0/266856234 reap_dead start 2023-05-20T01:28:27.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.973+0000 7fbf3699a700 1 -- 172.21.15.16:0/266856234 shutdown_connections 2023-05-20T01:28:27.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.973+0000 7fbf3699a700 1 -- 172.21.15.16:0/266856234 >> 172.21.15.16:0/266856234 conn(0x7fbf300cffd0 msgr2=0x7fbf300d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:27.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.973+0000 7fbf3699a700 1 -- 172.21.15.16:0/266856234 shutdown_connections 2023-05-20T01:28:27.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.973+0000 7fbf3699a700 1 -- 172.21.15.16:0/266856234 wait complete. 2023-05-20T01:28:27.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.974+0000 7fbf3699a700 1 Processor -- start 2023-05-20T01:28:27.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.974+0000 7fbf3699a700 1 -- start start 2023-05-20T01:28:27.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.975+0000 7fbf3699a700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbf30152950 0x7fbf300b42a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:27.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.975+0000 7fbf3699a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf30152d20 0x7fbf300b5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:27.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.975+0000 7fbf3699a700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbf300ac8a0 0x7fbf300acd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:27.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.975+0000 7fbf3699a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf300af350 con 0x7fbf30152d20 2023-05-20T01:28:27.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.975+0000 7fbf3699a700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf300b47e0 con 0x7fbf300ac8a0 2023-05-20T01:28:27.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.975+0000 7fbf3699a700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf300b3820 con 0x7fbf30152950 2023-05-20T01:28:27.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.975+0000 7fbf2f7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf30152d20 0x7fbf300b5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:27.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.975+0000 7fbf2f7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf30152d20 0x7fbf300b5790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40074/0 (socket says 172.21.15.16:40074) 2023-05-20T01:28:27.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.975+0000 7fbf2f7fe700 1 -- 172.21.15.16:0/2919857587 learned_addr learned my addr 172.21.15.16:0/2919857587 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:27.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.975+0000 7fbf34f37700 1 --2- 172.21.15.16:0/2919857587 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbf300ac8a0 0x7fbf300acd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:27.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.975+0000 7fbf2ffff700 1 --2- 172.21.15.16:0/2919857587 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbf30152950 0x7fbf300b42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:27.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.976+0000 7fbf2f7fe700 1 -- 172.21.15.16:0/2919857587 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbf30152950 msgr2=0x7fbf300b42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:27.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.976+0000 7fbf2f7fe700 1 --2- 172.21.15.16:0/2919857587 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbf30152950 0x7fbf300b42a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:27.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.976+0000 7fbf2f7fe700 1 -- 172.21.15.16:0/2919857587 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbf300ac8a0 msgr2=0x7fbf300acd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:27.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.976+0000 7fbf2f7fe700 1 --2- 172.21.15.16:0/2919857587 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbf300ac8a0 0x7fbf300acd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:27.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.976+0000 7fbf2f7fe700 1 -- 172.21.15.16:0/2919857587 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf300cfd40 con 0x7fbf30152d20 2023-05-20T01:28:27.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.976+0000 7fbf2f7fe700 1 --2- 172.21.15.16:0/2919857587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf30152d20 0x7fbf300b5790 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7fbf24011940 tx=0x7fbf24011d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:27.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.976+0000 7fbf2cff9700 1 -- 172.21.15.16:0/2919857587 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf24004240 con 0x7fbf30152d20 2023-05-20T01:28:27.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.976+0000 7fbf2cff9700 1 -- 172.21.15.16:0/2919857587 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf240043a0 con 0x7fbf30152d20 2023-05-20T01:28:27.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.976+0000 7fbf3699a700 1 -- 172.21.15.16:0/2919857587 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf300cd2c0 con 0x7fbf30152d20 2023-05-20T01:28:27.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.976+0000 7fbf2cff9700 1 -- 172.21.15.16:0/2919857587 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf240038c0 con 0x7fbf30152d20 2023-05-20T01:28:27.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.976+0000 7fbf3699a700 1 -- 172.21.15.16:0/2919857587 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf300cf2c0 con 0x7fbf30152d20 2023-05-20T01:28:27.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.977+0000 7fbf3699a700 1 -- 172.21.15.16:0/2919857587 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf300adb90 con 0x7fbf30152d20 2023-05-20T01:28:27.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.977+0000 7fbf2cff9700 1 -- 172.21.15.16:0/2919857587 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fbf24003a20 con 0x7fbf30152d20 2023-05-20T01:28:27.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.978+0000 7fbf2cff9700 1 --2- 172.21.15.16:0/2919857587 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fbf1c06ace0 0x7fbf1c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:27.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.978+0000 7fbf2cff9700 1 -- 172.21.15.16:0/2919857587 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fbf24089b10 con 0x7fbf30152d20 2023-05-20T01:28:27.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.978+0000 7fbf2ffff700 1 --2- 172.21.15.16:0/2919857587 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fbf1c06ace0 0x7fbf1c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:27.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.978+0000 7fbf2ffff700 1 --2- 172.21.15.16:0/2919857587 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fbf1c06ace0 0x7fbf1c06d190 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fbf20001130 tx=0x7fbf20009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:27.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:27.982+0000 7fbf2cff9700 1 -- 172.21.15.16:0/2919857587 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fbf24054a80 con 0x7fbf30152d20 2023-05-20T01:28:28.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:28.230+0000 7fbf3699a700 1 -- 172.21.15.16:0/2919857587 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fbf300747b0 con 0x7fbf30152d20 2023-05-20T01:28:29.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:29.102+0000 7fbf2cff9700 1 -- 172.21.15.16:0/2919857587 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v98) v1 ==== 118+0+0 (secure 0 0 0) 0x7fbf240190b0 con 0x7fbf30152d20 2023-05-20T01:28:29.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:29.105+0000 7fbf3699a700 1 -- 172.21.15.16:0/2919857587 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fbf1c06ace0 msgr2=0x7fbf1c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:29.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:29.105+0000 7fbf3699a700 1 --2- 172.21.15.16:0/2919857587 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fbf1c06ace0 0x7fbf1c06d190 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fbf20001130 tx=0x7fbf20009040 comp rx=0 tx=0).stop 2023-05-20T01:28:29.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:29.105+0000 7fbf3699a700 1 -- 172.21.15.16:0/2919857587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf30152d20 msgr2=0x7fbf300b5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:29.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:29.105+0000 7fbf3699a700 1 --2- 172.21.15.16:0/2919857587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf30152d20 0x7fbf300b5790 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7fbf24011940 tx=0x7fbf24011d00 comp rx=0 tx=0).stop 2023-05-20T01:28:29.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:29.105+0000 7fbf3699a700 1 -- 172.21.15.16:0/2919857587 shutdown_connections 2023-05-20T01:28:29.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:29.105+0000 7fbf3699a700 1 --2- 172.21.15.16:0/2919857587 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbf300ac8a0 0x7fbf300acd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:29.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:29.105+0000 7fbf3699a700 1 --2- 172.21.15.16:0/2919857587 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fbf1c06ace0 0x7fbf1c06d190 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:29.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:29.105+0000 7fbf3699a700 1 --2- 172.21.15.16:0/2919857587 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbf30152950 0x7fbf300b42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:29.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:29.105+0000 7fbf3699a700 1 --2- 172.21.15.16:0/2919857587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf30152d20 0x7fbf300b5790 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:29.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:29.105+0000 7fbf3699a700 1 -- 172.21.15.16:0/2919857587 >> 172.21.15.16:0/2919857587 conn(0x7fbf300cffd0 msgr2=0x7fbf3015b4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:29.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:29.105+0000 7fbf3699a700 1 -- 172.21.15.16:0/2919857587 shutdown_connections 2023-05-20T01:28:29.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:29.106+0000 7fbf3699a700 1 -- 172.21.15.16:0/2919857587 wait complete. 2023-05-20T01:28:29.125 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-05-20T01:28:29.126 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-05-20T01:28:29.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:29.257+0000 7f14e7e3f700 1 Processor -- start 2023-05-20T01:28:29.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:29.257+0000 7f14e7e3f700 1 -- start start 2023-05-20T01:28:29.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:29.258+0000 7f14e7e3f700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f14e015d290 0x7f14e00cf9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:29.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:29.258+0000 7f14e7e3f700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14e00cd630 con 0x7f14e015cec0 2023-05-20T01:28:29.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:29.258+0000 7f14e7e3f700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14e00cca60 con 0x7f14e0072300 2023-05-20T01:28:29.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:29.258+0000 7f14e7e3f700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14e00ccd00 con 0x7f14e015d290 2023-05-20T01:28:29.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:29.258+0000 7f14e5bdb700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f14e015cec0 0x7f14e0071bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51608/0 (socket says 172.21.15.16:51608) 2023-05-20T01:28:29.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:29.258+0000 7f14e5bdb700 1 -- 172.21.15.16:0/576137632 learned_addr learned my addr 172.21.15.16:0/576137632 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:28:29.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:29.258+0000 7f14e63dc700 1 --2- 172.21.15.16:0/576137632 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f14e015d290 0x7f14e00cf9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:29.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:29.258+0000 7f14e4bd9700 1 -- 172.21.15.16:0/576137632 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3962742575 0 0) 0x7f14e00cd630 con 0x7f14e015cec0 2023-05-20T01:28:29.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:29.258+0000 7f14e4bd9700 1 -- 172.21.15.16:0/576137632 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f14cc0035b0 con 0x7f14e015cec0 2023-05-20T01:28:29.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:29.259+0000 7f14e4bd9700 1 -- 172.21.15.16:0/576137632 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3257053386 0 0) 0x7f14cc0035b0 con 0x7f14e015cec0 2023-05-20T01:28:29.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:29.259+0000 7f14e4bd9700 1 -- 172.21.15.16:0/576137632 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f14e015d290 msgr2=0x7f14e00cf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:29.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:29.259+0000 7f14e4bd9700 1 --2- 172.21.15.16:0/576137632 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f14e015d290 0x7f14e00cf9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:29.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:29.259+0000 7f14e4bd9700 1 -- 172.21.15.16:0/576137632 >> v1:172.21.15.132:6789/0 conn(0x7f14e0072300 legacy=0x7f14e00cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:29.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:29.259+0000 7f14e4bd9700 1 -- 172.21.15.16:0/576137632 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14e00ccfe0 con 0x7f14e015cec0 2023-05-20T01:28:29.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:29.259+0000 7f14e4bd9700 1 -- 172.21.15.16:0/576137632 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7f14d40017f0 con 0x7f14e015cec0 2023-05-20T01:28:29.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:29.259+0000 7f14e4bd9700 1 -- 172.21.15.16:0/576137632 >> v1:172.21.15.16:6789/0 conn(0x7f14e015cec0 legacy=0x7f14e0071bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:29.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:29.259+0000 7f14e4bd9700 1 --2- 172.21.15.16:0/576137632 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14cc003a30 0x7f14cc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:29.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:29.259+0000 7f14e4bd9700 1 --2- 172.21.15.16:0/576137632 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f14cc0043d0 0x7f14cc004890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:29.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:29.259+0000 7f14e4bd9700 1 --2- 172.21.15.16:0/576137632 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f14e015d290 0x7f14cc005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:29.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:29.259+0000 7f14e4bd9700 1 -- 172.21.15.16:0/576137632 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14e00ccd00 con 0x7f14cc003a30 2023-05-20T01:28:29.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:29.259+0000 7f14e4bd9700 1 -- 172.21.15.16:0/576137632 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14cc00bac0 con 0x7f14cc0043d0 2023-05-20T01:28:29.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:29.259+0000 7f14e4bd9700 1 -- 172.21.15.16:0/576137632 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14cc00bc50 con 0x7f14e015d290 2023-05-20T01:28:29.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:29.259+0000 7f14e5bdb700 1 --2- 172.21.15.16:0/576137632 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14cc003a30 0x7f14cc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:29.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:29.259+0000 7f14e63dc700 1 --2- 172.21.15.16:0/576137632 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f14e015d290 0x7f14cc005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:29.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:29.259+0000 7f14e53da700 1 --2- 172.21.15.16:0/576137632 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f14cc0043d0 0x7f14cc004890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:29.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:29.259+0000 7f14e5bdb700 1 -- 172.21.15.16:0/576137632 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f14e015d290 msgr2=0x7f14cc005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:29.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:29.259+0000 7f14e5bdb700 1 --2- 172.21.15.16:0/576137632 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f14e015d290 0x7f14cc005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:29.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:29.259+0000 7f14e5bdb700 1 -- 172.21.15.16:0/576137632 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f14cc0043d0 msgr2=0x7f14cc004890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:29.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:29.259+0000 7f14e5bdb700 1 --2- 172.21.15.16:0/576137632 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f14cc0043d0 0x7f14cc004890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:29.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:29.259+0000 7f14e5bdb700 1 -- 172.21.15.16:0/576137632 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14cc0035b0 con 0x7f14cc003a30 2023-05-20T01:28:29.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:29.259+0000 7f14e5bdb700 1 --2- 172.21.15.16:0/576137632 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14cc003a30 0x7f14cc003e00 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f14d400c270 tx=0x7f14cc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdcd48c70998421d server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:29.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:29.259+0000 7f14e4bd9700 1 -- 172.21.15.16:0/576137632 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14d400cdd0 con 0x7f14cc003a30 2023-05-20T01:28:29.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:29.259+0000 7f14e4bd9700 1 -- 172.21.15.16:0/576137632 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14d4007ba0 con 0x7f14cc003a30 2023-05-20T01:28:29.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:29.259+0000 7f14e4bd9700 1 -- 172.21.15.16:0/576137632 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14d40072f0 con 0x7f14cc003a30 2023-05-20T01:28:29.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:29.260+0000 7f14e7e3f700 1 -- 172.21.15.16:0/576137632 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14cc003a30 msgr2=0x7f14cc003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:29.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:29.260+0000 7f14e7e3f700 1 --2- 172.21.15.16:0/576137632 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14cc003a30 0x7f14cc003e00 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f14d400c270 tx=0x7f14cc005880 comp rx=0 tx=0).stop 2023-05-20T01:28:29.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:29.260+0000 7f14e63dc700 1 -- 172.21.15.16:0/576137632 reap_dead start 2023-05-20T01:28:29.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:29.260+0000 7f14e7e3f700 1 -- 172.21.15.16:0/576137632 shutdown_connections 2023-05-20T01:28:29.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:29.260+0000 7f14e7e3f700 1 -- 172.21.15.16:0/576137632 >> 172.21.15.16:0/576137632 conn(0x7f14e00cffd0 msgr2=0x7f14e00d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:29.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:29.260+0000 7f14e7e3f700 1 -- 172.21.15.16:0/576137632 shutdown_connections 2023-05-20T01:28:29.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:29.260+0000 7f14e7e3f700 1 -- 172.21.15.16:0/576137632 wait complete. 2023-05-20T01:28:29.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:29.261+0000 7f14e7e3f700 1 Processor -- start 2023-05-20T01:28:29.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:29.261+0000 7f14e7e3f700 1 -- start start 2023-05-20T01:28:29.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:29.262+0000 7f14e7e3f700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f14cc003a30 0x7f14e00b1da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:29.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:29.262+0000 7f14e7e3f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14e00af2a0 0x7f14e00afa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:29.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:29.262+0000 7f14e7e3f700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f14e01f1fa0 0x7f14e00affd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:29.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:29.262+0000 7f14e7e3f700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14e015adc0 con 0x7f14e00af2a0 2023-05-20T01:28:29.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:29.262+0000 7f14e7e3f700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14e00c9ab0 con 0x7f14e01f1fa0 2023-05-20T01:28:29.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:29.262+0000 7f14e7e3f700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14e00c9530 con 0x7f14cc003a30 2023-05-20T01:28:29.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:29.262+0000 7f14e53da700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14e00af2a0 0x7f14e00afa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:29.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:29.262+0000 7f14e53da700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14e00af2a0 0x7f14e00afa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40094/0 (socket says 172.21.15.16:40094) 2023-05-20T01:28:29.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:29.262+0000 7f14e53da700 1 -- 172.21.15.16:0/1170019630 learned_addr learned my addr 172.21.15.16:0/1170019630 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:29.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:29.262+0000 7f14e5bdb700 1 --2- 172.21.15.16:0/1170019630 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f14cc003a30 0x7f14e00b1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:29.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:29.263+0000 7f14e63dc700 1 --2- 172.21.15.16:0/1170019630 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f14e01f1fa0 0x7f14e00affd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:29.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:29.263+0000 7f14e53da700 1 -- 172.21.15.16:0/1170019630 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f14cc003a30 msgr2=0x7f14e00b1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:29.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:29.263+0000 7f14e53da700 1 --2- 172.21.15.16:0/1170019630 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f14cc003a30 0x7f14e00b1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:29.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:29.263+0000 7f14e53da700 1 -- 172.21.15.16:0/1170019630 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f14e01f1fa0 msgr2=0x7f14e00affd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:29.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:29.263+0000 7f14e53da700 1 --2- 172.21.15.16:0/1170019630 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f14e01f1fa0 0x7f14e00affd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:29.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:29.263+0000 7f14e53da700 1 -- 172.21.15.16:0/1170019630 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14e00cf9d0 con 0x7f14e00af2a0 2023-05-20T01:28:29.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:29.263+0000 7f14e5bdb700 1 --2- 172.21.15.16:0/1170019630 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f14cc003a30 0x7f14e00b1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:28:29.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:29.263+0000 7f14e53da700 1 --2- 172.21.15.16:0/1170019630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14e00af2a0 0x7f14e00afa90 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f14dc00f7f0 tx=0x7f14dc00fbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:29.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:29.263+0000 7f14d27fc700 1 -- 172.21.15.16:0/1170019630 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14dc0052e0 con 0x7f14e00af2a0 2023-05-20T01:28:29.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:29.263+0000 7f14d27fc700 1 -- 172.21.15.16:0/1170019630 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14dc005460 con 0x7f14e00af2a0 2023-05-20T01:28:29.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:29.263+0000 7f14e7e3f700 1 -- 172.21.15.16:0/1170019630 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14e00cd2c0 con 0x7f14e00af2a0 2023-05-20T01:28:29.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:29.263+0000 7f14d27fc700 1 -- 172.21.15.16:0/1170019630 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14dc0038c0 con 0x7f14e00af2a0 2023-05-20T01:28:29.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:29.263+0000 7f14e7e3f700 1 -- 172.21.15.16:0/1170019630 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14e00cfb60 con 0x7f14e00af2a0 2023-05-20T01:28:29.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:29.264+0000 7f14e7e3f700 1 -- 172.21.15.16:0/1170019630 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14e00747b0 con 0x7f14e00af2a0 2023-05-20T01:28:29.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:29.264+0000 7f14d27fc700 1 -- 172.21.15.16:0/1170019630 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f14dc003a20 con 0x7f14e00af2a0 2023-05-20T01:28:29.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:29.265+0000 7f14d27fc700 1 --2- 172.21.15.16:0/1170019630 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f14c406ace0 0x7f14c406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:29.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:29.265+0000 7f14d27fc700 1 -- 172.21.15.16:0/1170019630 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f14dc0894d0 con 0x7f14e00af2a0 2023-05-20T01:28:29.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:29.265+0000 7f14e5bdb700 1 --2- 172.21.15.16:0/1170019630 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f14c406ace0 0x7f14c406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:29.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:29.265+0000 7f14e5bdb700 1 --2- 172.21.15.16:0/1170019630 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f14c406ace0 0x7f14c406d190 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f14d4007d50 tx=0x7f14d4007000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:29.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:29.269+0000 7f14d27fc700 1 -- 172.21.15.16:0/1170019630 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f14dc0543b0 con 0x7f14e00af2a0 2023-05-20T01:28:29.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:29.516+0000 7f14e7e3f700 1 -- 172.21.15.16:0/1170019630 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f14e00c9bf0 con 0x7f14e00af2a0 2023-05-20T01:28:30.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.446+0000 7f14d27fc700 1 -- 172.21.15.16:0/1170019630 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v99) v1 ==== 122+0+0 (secure 0 0 0) 0x7f14e00c9bf0 con 0x7f14e00af2a0 2023-05-20T01:28:30.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.449+0000 7f14e7e3f700 1 -- 172.21.15.16:0/1170019630 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f14c406ace0 msgr2=0x7f14c406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:30.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.449+0000 7f14e7e3f700 1 --2- 172.21.15.16:0/1170019630 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f14c406ace0 0x7f14c406d190 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f14d4007d50 tx=0x7f14d4007000 comp rx=0 tx=0).stop 2023-05-20T01:28:30.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.449+0000 7f14e7e3f700 1 -- 172.21.15.16:0/1170019630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14e00af2a0 msgr2=0x7f14e00afa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:30.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.449+0000 7f14e7e3f700 1 --2- 172.21.15.16:0/1170019630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14e00af2a0 0x7f14e00afa90 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f14dc00f7f0 tx=0x7f14dc00fbb0 comp rx=0 tx=0).stop 2023-05-20T01:28:30.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.449+0000 7f14e7e3f700 1 -- 172.21.15.16:0/1170019630 shutdown_connections 2023-05-20T01:28:30.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.449+0000 7f14e7e3f700 1 --2- 172.21.15.16:0/1170019630 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f14e01f1fa0 0x7f14e00affd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:30.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.449+0000 7f14e7e3f700 1 --2- 172.21.15.16:0/1170019630 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f14c406ace0 0x7f14c406d190 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:30.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.449+0000 7f14e7e3f700 1 --2- 172.21.15.16:0/1170019630 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f14cc003a30 0x7f14e00b1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:30.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.449+0000 7f14e7e3f700 1 --2- 172.21.15.16:0/1170019630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14e00af2a0 0x7f14e00afa90 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:30.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.449+0000 7f14e7e3f700 1 -- 172.21.15.16:0/1170019630 >> 172.21.15.16:0/1170019630 conn(0x7f14e00cffd0 msgr2=0x7f14e0160ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:30.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.449+0000 7f14e7e3f700 1 -- 172.21.15.16:0/1170019630 shutdown_connections 2023-05-20T01:28:30.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.449+0000 7f14e7e3f700 1 -- 172.21.15.16:0/1170019630 wait complete. 2023-05-20T01:28:30.469 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-05-20T01:28:30.470 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-20T01:28:30.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.597+0000 7f60271cd700 1 Processor -- start 2023-05-20T01:28:30.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.597+0000 7f60271cd700 1 -- start start 2023-05-20T01:28:30.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.598+0000 7f60271cd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6020157190 0x7f60200cc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:30.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.598+0000 7f60271cd700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6020071ad0 0x7f60200ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:30.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.598+0000 7f60271cd700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60200cb590 con 0x7f60200cccf0 2023-05-20T01:28:30.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.598+0000 7f60271cd700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6020071f50 con 0x7f6020157190 2023-05-20T01:28:30.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.598+0000 7f60271cd700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60200ca9a0 con 0x7f6020071ad0 2023-05-20T01:28:30.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.598+0000 7f6024f69700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6020157190 0x7f60200cc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:30.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.598+0000 7f6024f69700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6020157190 0x7f60200cc7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40100/0 (socket says 172.21.15.16:40100) 2023-05-20T01:28:30.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.598+0000 7f6024f69700 1 -- 172.21.15.16:0/3956767432 learned_addr learned my addr 172.21.15.16:0/3956767432 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:30.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.599+0000 7f602576a700 1 --2- 172.21.15.16:0/3956767432 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6020071ad0 0x7f60200ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:30.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.599+0000 7f6024f69700 1 -- 172.21.15.16:0/3956767432 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6020071ad0 msgr2=0x7f60200ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:30.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.599+0000 7f6024f69700 1 --2- 172.21.15.16:0/3956767432 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6020071ad0 0x7f60200ce740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:30.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.599+0000 7f6024f69700 1 -- 172.21.15.16:0/3956767432 >> v1:172.21.15.132:6789/0 conn(0x7f60200cccf0 legacy=0x7f60200713c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:30.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.599+0000 7f6024f69700 1 -- 172.21.15.16:0/3956767432 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60200cac60 con 0x7f6020157190 2023-05-20T01:28:30.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.599+0000 7f6024f69700 1 --2- 172.21.15.16:0/3956767432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6020157190 0x7f60200cc7b0 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f601400bb90 tx=0x7f601400bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d86fde214f04c8af server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:30.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.599+0000 7f601f7fe700 1 -- 172.21.15.16:0/3956767432 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f601400dd50 con 0x7f6020157190 2023-05-20T01:28:30.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.599+0000 7f601f7fe700 1 -- 172.21.15.16:0/3956767432 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6014010470 con 0x7f6020157190 2023-05-20T01:28:30.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.599+0000 7f601f7fe700 1 -- 172.21.15.16:0/3956767432 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6014010800 con 0x7f6020157190 2023-05-20T01:28:30.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.600+0000 7f60271cd700 1 -- 172.21.15.16:0/3956767432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6020157190 msgr2=0x7f60200cc7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:30.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.600+0000 7f60271cd700 1 --2- 172.21.15.16:0/3956767432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6020157190 0x7f60200cc7b0 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f601400bb90 tx=0x7f601400bea0 comp rx=0 tx=0).stop 2023-05-20T01:28:30.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.600+0000 7f60271cd700 1 -- 172.21.15.16:0/3956767432 shutdown_connections 2023-05-20T01:28:30.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.600+0000 7f60271cd700 1 --2- 172.21.15.16:0/3956767432 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6020071ad0 0x7f60200ce740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:30.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.600+0000 7f60271cd700 1 --2- 172.21.15.16:0/3956767432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6020157190 0x7f60200cc7b0 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:30.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.600+0000 7f60271cd700 1 -- 172.21.15.16:0/3956767432 >> 172.21.15.16:0/3956767432 conn(0x7f60200cffb0 msgr2=0x7f60200d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:30.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.600+0000 7f60271cd700 1 -- 172.21.15.16:0/3956767432 shutdown_connections 2023-05-20T01:28:30.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.600+0000 7f60271cd700 1 -- 172.21.15.16:0/3956767432 wait complete. 2023-05-20T01:28:30.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.601+0000 7f60271cd700 1 Processor -- start 2023-05-20T01:28:30.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.601+0000 7f60271cd700 1 -- start start 2023-05-20T01:28:30.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.602+0000 7f60271cd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6020071ad0 0x7f6020152500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:30.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.602+0000 7f60271cd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f60200cccf0 0x7f6020158d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:30.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.602+0000 7f60271cd700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6020157190 0x7f60201592b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:30.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.602+0000 7f60271cd700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60200b3ea0 con 0x7f6020071ad0 2023-05-20T01:28:30.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.602+0000 7f60271cd700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6020152a40 con 0x7f60200cccf0 2023-05-20T01:28:30.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.602+0000 7f60271cd700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60200ac8a0 con 0x7f6020157190 2023-05-20T01:28:30.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.602+0000 7f6024f69700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6020071ad0 0x7f6020152500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:30.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.602+0000 7f6024f69700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6020071ad0 0x7f6020152500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40104/0 (socket says 172.21.15.16:40104) 2023-05-20T01:28:30.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.602+0000 7f6024f69700 1 -- 172.21.15.16:0/86668702 learned_addr learned my addr 172.21.15.16:0/86668702 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:30.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.602+0000 7f6024f69700 1 -- 172.21.15.16:0/86668702 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6020157190 msgr2=0x7f60201592b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:30.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.602+0000 7f6024f69700 1 --2- 172.21.15.16:0/86668702 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6020157190 0x7f60201592b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:30.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.602+0000 7f6024f69700 1 -- 172.21.15.16:0/86668702 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f60200cccf0 msgr2=0x7f6020158d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:30.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.602+0000 7f6024f69700 1 --2- 172.21.15.16:0/86668702 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f60200cccf0 0x7f6020158d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:30.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.602+0000 7f6024f69700 1 -- 172.21.15.16:0/86668702 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f601400b7f0 con 0x7f6020071ad0 2023-05-20T01:28:30.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.602+0000 7f6024f69700 1 --2- 172.21.15.16:0/86668702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6020071ad0 0x7f6020152500 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f601400bee0 tx=0x7f6014004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:30.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.602+0000 7f601d7fa700 1 -- 172.21.15.16:0/86668702 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6014003b30 con 0x7f6020071ad0 2023-05-20T01:28:30.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.602+0000 7f601d7fa700 1 -- 172.21.15.16:0/86668702 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6014003c90 con 0x7f6020071ad0 2023-05-20T01:28:30.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.602+0000 7f601d7fa700 1 -- 172.21.15.16:0/86668702 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60140105e0 con 0x7f6020071ad0 2023-05-20T01:28:30.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.603+0000 7f60271cd700 1 -- 172.21.15.16:0/86668702 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60200cb220 con 0x7f6020071ad0 2023-05-20T01:28:30.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.603+0000 7f60271cd700 1 -- 172.21.15.16:0/86668702 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60200caf40 con 0x7f6020071ad0 2023-05-20T01:28:30.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.603+0000 7f60271cd700 1 -- 172.21.15.16:0/86668702 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6020157560 con 0x7f6020071ad0 2023-05-20T01:28:30.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.604+0000 7f601d7fa700 1 -- 172.21.15.16:0/86668702 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6014019400 con 0x7f6020071ad0 2023-05-20T01:28:30.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.604+0000 7f601d7fa700 1 --2- 172.21.15.16:0/86668702 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f600406ace0 0x7f600406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:30.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.604+0000 7f601d7fa700 1 -- 172.21.15.16:0/86668702 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f601408a950 con 0x7f6020071ad0 2023-05-20T01:28:30.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.604+0000 7f601ffff700 1 --2- 172.21.15.16:0/86668702 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f600406ace0 0x7f600406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:30.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.605+0000 7f601ffff700 1 --2- 172.21.15.16:0/86668702 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f600406ace0 0x7f600406d190 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f600c000f60 tx=0x7f600c007a40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:30.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.609+0000 7f601d7fa700 1 -- 172.21.15.16:0/86668702 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f6014055950 con 0x7f6020071ad0 2023-05-20T01:28:30.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.854+0000 7f60271cd700 1 -- 172.21.15.16:0/86668702 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f60200747b0 con 0x7f6020071ad0 2023-05-20T01:28:30.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.855+0000 7f601d7fa700 1 -- 172.21.15.16:0/86668702 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) v1 ==== 94+0+4679 (secure 0 0 0) 0x7f6014058f70 con 0x7f6020071ad0 2023-05-20T01:28:30.860 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:28:30.860 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":99,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7043,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/1859755557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1859755557},{"type":"v1","addr":"172.21.15.16:6836","nonce":1859755557}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94},{"gid":7046,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/2654783598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":2654783598},{"type":"v1","addr":"172.21.15.132:6834","nonce":2654783598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94},{"gid":7049,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2042939723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":2042939723},{"type":"v1","addr":"172.21.15.132:6835","nonce":2042939723}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-20T01:28:27.093957+0000","modified":"2023-05-20T01:28:30.436870+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8703},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8703":{"gid":8703,"name":"a","rank":0,"incarnation":95,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6837/2295684907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":2295684907},{"type":"v1","addr":"172.21.15.16:6837","nonce":2295684907}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-05-20T01:28:30.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.857+0000 7f60271cd700 1 -- 172.21.15.16:0/86668702 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f600406ace0 msgr2=0x7f600406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:30.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.857+0000 7f60271cd700 1 --2- 172.21.15.16:0/86668702 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f600406ace0 0x7f600406d190 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f600c000f60 tx=0x7f600c007a40 comp rx=0 tx=0).stop 2023-05-20T01:28:30.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.857+0000 7f60271cd700 1 -- 172.21.15.16:0/86668702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6020071ad0 msgr2=0x7f6020152500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:30.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.857+0000 7f60271cd700 1 --2- 172.21.15.16:0/86668702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6020071ad0 0x7f6020152500 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f601400bee0 tx=0x7f6014004750 comp rx=0 tx=0).stop 2023-05-20T01:28:30.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.857+0000 7f60271cd700 1 -- 172.21.15.16:0/86668702 shutdown_connections 2023-05-20T01:28:30.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.857+0000 7f60271cd700 1 --2- 172.21.15.16:0/86668702 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f600406ace0 0x7f600406d190 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:30.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.857+0000 7f60271cd700 1 --2- 172.21.15.16:0/86668702 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6020157190 0x7f60201592b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:30.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.857+0000 7f60271cd700 1 --2- 172.21.15.16:0/86668702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6020071ad0 0x7f6020152500 unknown :-1 s=CLOSED pgs=1622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:30.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.857+0000 7f60271cd700 1 --2- 172.21.15.16:0/86668702 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f60200cccf0 0x7f6020158d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:30.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.857+0000 7f60271cd700 1 -- 172.21.15.16:0/86668702 >> 172.21.15.16:0/86668702 conn(0x7f60200cffb0 msgr2=0x7f60200cf5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:30.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.857+0000 7f60271cd700 1 -- 172.21.15.16:0/86668702 shutdown_connections 2023-05-20T01:28:30.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.857+0000 7f60271cd700 1 -- 172.21.15.16:0/86668702 wait complete. 2023-05-20T01:28:30.864 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 99 2023-05-20T01:28:30.873 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-20T01:28:30.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.994+0000 7f52f13bd700 1 Processor -- start 2023-05-20T01:28:30.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.995+0000 7f52f13bd700 1 -- start start 2023-05-20T01:28:31.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.995+0000 7f52f13bd700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f52ec0b3880 0x7f52ec0b3c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:31.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.995+0000 7f52f13bd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f52ec0d1500 0x7f52ec15d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:31.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.995+0000 7f52f13bd700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52ec0b4520 con 0x7f52ec0d2da0 2023-05-20T01:28:31.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.995+0000 7f52f13bd700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52ec0b7a60 con 0x7f52ec0d1500 2023-05-20T01:28:31.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.995+0000 7f52f13bd700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52ec0b7c70 con 0x7f52ec0b3880 2023-05-20T01:28:31.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.995+0000 7f52eb7fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f52ec0d1500 0x7f52ec15d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:31.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.996+0000 7f52eb7fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f52ec0d1500 0x7f52ec15d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40114/0 (socket says 172.21.15.16:40114) 2023-05-20T01:28:31.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.996+0000 7f52eb7fe700 1 -- 172.21.15.16:0/4042984970 learned_addr learned my addr 172.21.15.16:0/4042984970 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:31.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.996+0000 7f52eaffd700 1 --2- 172.21.15.16:0/4042984970 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f52ec0b3880 0x7f52ec0b3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:31.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.996+0000 7f52eb7fe700 1 -- 172.21.15.16:0/4042984970 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f52ec0b3880 msgr2=0x7f52ec0b3c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:31.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.996+0000 7f52eb7fe700 1 --2- 172.21.15.16:0/4042984970 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f52ec0b3880 0x7f52ec0b3c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:31.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.996+0000 7f52eb7fe700 1 -- 172.21.15.16:0/4042984970 >> v1:172.21.15.132:6789/0 conn(0x7f52ec0d2da0 legacy=0x7f52ec0d0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:31.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.996+0000 7f52eb7fe700 1 -- 172.21.15.16:0/4042984970 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52ec0b7f50 con 0x7f52ec0d1500 2023-05-20T01:28:31.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.996+0000 7f52eb7fe700 1 --2- 172.21.15.16:0/4042984970 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52ec0d1500 0x7f52ec15d2c0 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f52e4009e50 tx=0x7f52e400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47e31582fba79134 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:31.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.996+0000 7f52e9ffb700 1 -- 172.21.15.16:0/4042984970 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52e400e070 con 0x7f52ec0d1500 2023-05-20T01:28:31.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.996+0000 7f52e9ffb700 1 -- 172.21.15.16:0/4042984970 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52e4003a20 con 0x7f52ec0d1500 2023-05-20T01:28:31.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.996+0000 7f52e9ffb700 1 -- 172.21.15.16:0/4042984970 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52e4003dd0 con 0x7f52ec0d1500 2023-05-20T01:28:31.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.997+0000 7f52f13bd700 1 -- 172.21.15.16:0/4042984970 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52ec0d1500 msgr2=0x7f52ec15d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:31.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.997+0000 7f52f13bd700 1 --2- 172.21.15.16:0/4042984970 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52ec0d1500 0x7f52ec15d2c0 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f52e4009e50 tx=0x7f52e400b330 comp rx=0 tx=0).stop 2023-05-20T01:28:31.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.997+0000 7f52f13bd700 1 -- 172.21.15.16:0/4042984970 shutdown_connections 2023-05-20T01:28:31.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.997+0000 7f52f13bd700 1 --2- 172.21.15.16:0/4042984970 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52ec0d1500 0x7f52ec15d2c0 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:31.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.997+0000 7f52f13bd700 1 --2- 172.21.15.16:0/4042984970 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f52ec0b3880 0x7f52ec0b3c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:31.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.997+0000 7f52f13bd700 1 -- 172.21.15.16:0/4042984970 >> 172.21.15.16:0/4042984970 conn(0x7f52ec09c590 msgr2=0x7f52ec0d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:31.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.997+0000 7f52f13bd700 1 -- 172.21.15.16:0/4042984970 shutdown_connections 2023-05-20T01:28:31.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.997+0000 7f52f13bd700 1 -- 172.21.15.16:0/4042984970 wait complete. 2023-05-20T01:28:31.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.998+0000 7f52f13bd700 1 Processor -- start 2023-05-20T01:28:31.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.999+0000 7f52f13bd700 1 -- start start 2023-05-20T01:28:31.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.999+0000 7f52f13bd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52ec0b3880 0x7f52ec15b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:31.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.999+0000 7f52f13bd700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f52ec0d1500 0x7f52ec09dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:31.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.999+0000 7f52f13bd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f52ec0d2da0 0x7f52ec09e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:31.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.999+0000 7f52f13bd700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52ec0a1070 con 0x7f52ec0b3880 2023-05-20T01:28:31.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.999+0000 7f52f13bd700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52ec0b00b0 con 0x7f52ec0d2da0 2023-05-20T01:28:31.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.999+0000 7f52f13bd700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52ec0a07d0 con 0x7f52ec0d1500 2023-05-20T01:28:31.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.999+0000 7f52eaffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52ec0b3880 0x7f52ec15b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:31.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.999+0000 7f52eaffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52ec0b3880 0x7f52ec15b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40116/0 (socket says 172.21.15.16:40116) 2023-05-20T01:28:31.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:30.999+0000 7f52eaffd700 1 -- 172.21.15.16:0/1711204291 learned_addr learned my addr 172.21.15.16:0/1711204291 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:31.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.000+0000 7f52eb7fe700 1 --2- 172.21.15.16:0/1711204291 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f52ec0d2da0 0x7f52ec09e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:31.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.000+0000 7f52ea7fc700 1 --2- 172.21.15.16:0/1711204291 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f52ec0d1500 0x7f52ec09dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:31.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.000+0000 7f52eaffd700 1 -- 172.21.15.16:0/1711204291 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f52ec0d1500 msgr2=0x7f52ec09dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:31.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.000+0000 7f52eaffd700 1 --2- 172.21.15.16:0/1711204291 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f52ec0d1500 0x7f52ec09dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:31.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.000+0000 7f52eaffd700 1 -- 172.21.15.16:0/1711204291 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f52ec0d2da0 msgr2=0x7f52ec09e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:31.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.000+0000 7f52eaffd700 1 --2- 172.21.15.16:0/1711204291 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f52ec0d2da0 0x7f52ec09e0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:31.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.000+0000 7f52eaffd700 1 -- 172.21.15.16:0/1711204291 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52e4009b30 con 0x7f52ec0b3880 2023-05-20T01:28:31.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.000+0000 7f52eaffd700 1 --2- 172.21.15.16:0/1711204291 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52ec0b3880 0x7f52ec15b2d0 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f52dc00c820 tx=0x7f52dc00cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:31.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.000+0000 7f52e37fe700 1 -- 172.21.15.16:0/1711204291 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52dc011800 con 0x7f52ec0b3880 2023-05-20T01:28:31.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.000+0000 7f52e37fe700 1 -- 172.21.15.16:0/1711204291 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52dc011960 con 0x7f52ec0b3880 2023-05-20T01:28:31.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.000+0000 7f52f13bd700 1 -- 172.21.15.16:0/1711204291 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52ec0b4190 con 0x7f52ec0b3880 2023-05-20T01:28:31.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.000+0000 7f52e37fe700 1 -- 172.21.15.16:0/1711204291 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52dc011cf0 con 0x7f52ec0b3880 2023-05-20T01:28:31.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.000+0000 7f52f13bd700 1 -- 172.21.15.16:0/1711204291 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52ec0b8230 con 0x7f52ec0b3880 2023-05-20T01:28:31.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.001+0000 7f52f13bd700 1 -- 172.21.15.16:0/1711204291 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52ec09c990 con 0x7f52ec0b3880 2023-05-20T01:28:31.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.002+0000 7f52e37fe700 1 -- 172.21.15.16:0/1711204291 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f52dc018360 con 0x7f52ec0b3880 2023-05-20T01:28:31.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.002+0000 7f52e37fe700 1 --2- 172.21.15.16:0/1711204291 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f52cc06ace0 0x7f52cc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:31.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.002+0000 7f52e37fe700 1 -- 172.21.15.16:0/1711204291 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f52dc089510 con 0x7f52ec0b3880 2023-05-20T01:28:31.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.002+0000 7f52ea7fc700 1 --2- 172.21.15.16:0/1711204291 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f52cc06ace0 0x7f52cc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:31.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.003+0000 7f52ea7fc700 1 --2- 172.21.15.16:0/1711204291 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f52cc06ace0 0x7f52cc06d190 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f52ec0c6390 tx=0x7f52d4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:31.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.007+0000 7f52e37fe700 1 -- 172.21.15.16:0/1711204291 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f52dc054510 con 0x7f52ec0b3880 2023-05-20T01:28:31.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.204+0000 7f52f13bd700 1 -- 172.21.15.16:0/1711204291 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f52ec04e8c0 con 0x7f52ec0b3880 2023-05-20T01:28:31.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.205+0000 7f52e37fe700 1 -- 172.21.15.16:0/1711204291 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v136) v1 ==== 74+0+18472 (secure 0 0 0) 0x7f52dc057b30 con 0x7f52ec0b3880 2023-05-20T01:28:31.210 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:28:31.210 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":136,"fsid":"c2245cc5-ecab-4087-a59b-bc81db901676","created":"2023-05-20T01:16:09.549230+0000","modified":"2023-05-20T01:28:28.096014+0000","last_up_change":"2023-05-20T01:16:19.788624+0000","last_in_change":"2023-05-20T01:16:12.092160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-05-20T01:16:20.934366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-05-20T01:16:21.372294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-05-20T01:28:25.274827+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"135","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-05-20T01:28:26.067827+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"317a83cd-19c8-4958-a301-6087ef342289","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6825","nonce":3923634179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6827","nonce":3923634179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6831","nonce":3923634179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6829","nonce":3923634179}]},"public_addr":"172.21.15.16:6825/3923634179","cluster_addr":"172.21.15.16:6827/3923634179","heartbeat_back_addr":"172.21.15.16:6831/3923634179","heartbeat_front_addr":"172.21.15.16:6829/3923634179","state":["exists","up"]},{"osd":1,"uuid":"5581046f-8219-4089-8d7d-b7846399505e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6809","nonce":3860264220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6811","nonce":3860264220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6815","nonce":3860264220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6813","nonce":3860264220}]},"public_addr":"172.21.15.16:6809/3860264220","cluster_addr":"172.21.15.16:6811/3860264220","heartbeat_back_addr":"172.21.15.16:6815/3860264220","heartbeat_front_addr":"172.21.15.16:6813/3860264220","state":["exists","up"]},{"osd":2,"uuid":"072e9e66-4133-478e-8d4a-64d46c2963f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6817","nonce":293187030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6819","nonce":293187030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6823","nonce":293187030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6821","nonce":293187030}]},"public_addr":"172.21.15.16:6817/293187030","cluster_addr":"172.21.15.16:6819/293187030","heartbeat_back_addr":"172.21.15.16:6823/293187030","heartbeat_front_addr":"172.21.15.16:6821/293187030","state":["exists","up"]},{"osd":3,"uuid":"11e1bba6-00f8-48dd-9028-d8d716b7f145","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6801","nonce":3669781890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6803","nonce":3669781890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6807","nonce":3669781890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6805","nonce":3669781890}]},"public_addr":"172.21.15.16:6801/3669781890","cluster_addr":"172.21.15.16:6803/3669781890","heartbeat_back_addr":"172.21.15.16:6807/3669781890","heartbeat_front_addr":"172.21.15.16:6805/3669781890","state":["exists","up"]},{"osd":4,"uuid":"b8909c3a-cf70-46cb-8d0b-009c707cf9d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6819","nonce":506317223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6823","nonce":506317223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6829","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6831","nonce":506317223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6827","nonce":506317223}]},"public_addr":"172.21.15.132:6819/506317223","cluster_addr":"172.21.15.132:6823/506317223","heartbeat_back_addr":"172.21.15.132:6831/506317223","heartbeat_front_addr":"172.21.15.132:6827/506317223","state":["exists","up"]},{"osd":5,"uuid":"b1818304-4583-47bf-a6ec-0bfbb5166233","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6818","nonce":1844689995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6822","nonce":1844689995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6830","nonce":1844689995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6826","nonce":1844689995}]},"public_addr":"172.21.15.132:6818/1844689995","cluster_addr":"172.21.15.132:6822/1844689995","heartbeat_back_addr":"172.21.15.132:6830/1844689995","heartbeat_front_addr":"172.21.15.132:6826/1844689995","state":["exists","up"]},{"osd":6,"uuid":"d37f4691-eed1-4a93-8818-18354f03f4a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6801","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6802","nonce":727581123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6805","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6807","nonce":727581123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6813","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6815","nonce":727581123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6811","nonce":727581123}]},"public_addr":"172.21.15.132:6802/727581123","cluster_addr":"172.21.15.132:6807/727581123","heartbeat_back_addr":"172.21.15.132:6815/727581123","heartbeat_front_addr":"172.21.15.132:6811/727581123","state":["exists","up"]},{"osd":7,"uuid":"4f712f62-5d1d-4816-8042-03209144f8b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6803","nonce":466001149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6806","nonce":466001149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6814","nonce":466001149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6810","nonce":466001149}]},"public_addr":"172.21.15.132:6803/466001149","cluster_addr":"172.21.15.132:6806/466001149","heartbeat_back_addr":"172.21.15.132:6814/466001149","heartbeat_front_addr":"172.21.15.132:6810/466001149","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.338863+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.278680+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305694+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.246835+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.276237+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.304441+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305421+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.267050+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-20T01:28:31.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.207+0000 7f52f13bd700 1 -- 172.21.15.16:0/1711204291 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f52cc06ace0 msgr2=0x7f52cc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:31.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.207+0000 7f52f13bd700 1 --2- 172.21.15.16:0/1711204291 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f52cc06ace0 0x7f52cc06d190 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f52ec0c6390 tx=0x7f52d4008040 comp rx=0 tx=0).stop 2023-05-20T01:28:31.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.207+0000 7f52f13bd700 1 -- 172.21.15.16:0/1711204291 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52ec0b3880 msgr2=0x7f52ec15b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:31.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.207+0000 7f52f13bd700 1 --2- 172.21.15.16:0/1711204291 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52ec0b3880 0x7f52ec15b2d0 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f52dc00c820 tx=0x7f52dc00cbe0 comp rx=0 tx=0).stop 2023-05-20T01:28:31.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.207+0000 7f52f13bd700 1 -- 172.21.15.16:0/1711204291 shutdown_connections 2023-05-20T01:28:31.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.207+0000 7f52f13bd700 1 --2- 172.21.15.16:0/1711204291 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f52ec0d2da0 0x7f52ec09e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:31.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.207+0000 7f52f13bd700 1 --2- 172.21.15.16:0/1711204291 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52ec0b3880 0x7f52ec15b2d0 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:31.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.207+0000 7f52f13bd700 1 --2- 172.21.15.16:0/1711204291 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f52cc06ace0 0x7f52cc06d190 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:31.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.207+0000 7f52f13bd700 1 --2- 172.21.15.16:0/1711204291 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f52ec0d1500 0x7f52ec09dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:31.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.207+0000 7f52f13bd700 1 -- 172.21.15.16:0/1711204291 >> 172.21.15.16:0/1711204291 conn(0x7f52ec09c590 msgr2=0x7f52ec0b6c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:31.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.207+0000 7f52f13bd700 1 -- 172.21.15.16:0/1711204291 shutdown_connections 2023-05-20T01:28:31.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.207+0000 7f52f13bd700 1 -- 172.21.15.16:0/1711204291 wait complete. 2023-05-20T01:28:31.226 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-05-20T01:28:31.226 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-05-20T01:28:31.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.346+0000 7f885cd0c700 1 Processor -- start 2023-05-20T01:28:31.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.346+0000 7f885cd0c700 1 -- start start 2023-05-20T01:28:31.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.347+0000 7f885cd0c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f88580d1000 0x7f88580d13d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:31.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.347+0000 7f885cd0c700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88580ce3e0 con 0x7f885806b620 2023-05-20T01:28:31.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.347+0000 7f885cd0c700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88580b17f0 con 0x7f885806ba90 2023-05-20T01:28:31.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.347+0000 7f885cd0c700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f885806bef0 con 0x7f88580d1000 2023-05-20T01:28:31.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.348+0000 7f885659c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f88580d1000 0x7f88580d13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:31.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.348+0000 7f8855d9b700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f885806b620 0x7f88580b09d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51652/0 (socket says 172.21.15.16:51652) 2023-05-20T01:28:31.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.348+0000 7f8855d9b700 1 -- 172.21.15.16:0/1420142917 learned_addr learned my addr 172.21.15.16:0/1420142917 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:28:31.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.348+0000 7f885659c700 1 -- 172.21.15.16:0/1420142917 >> v1:172.21.15.132:6789/0 conn(0x7f885806ba90 legacy=0x7f88580b10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:31.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.348+0000 7f885659c700 1 -- 172.21.15.16:0/1420142917 >> v1:172.21.15.16:6789/0 conn(0x7f885806b620 legacy=0x7f88580b09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:31.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.348+0000 7f885559a700 1 -- 172.21.15.16:0/1420142917 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3155087725 0 0) 0x7f88580ce3e0 con 0x7f885806b620 2023-05-20T01:28:31.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.348+0000 7f885659c700 1 -- 172.21.15.16:0/1420142917 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88580d1910 con 0x7f88580d1000 2023-05-20T01:28:31.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.348+0000 7f885659c700 1 --2- 172.21.15.16:0/1420142917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88580d1000 0x7f88580d13d0 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f8848009840 tx=0x7f8848009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7dd29e45260d357 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:31.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.348+0000 7f885559a700 1 -- 172.21.15.16:0/1420142917 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8848011830 con 0x7f88580d1000 2023-05-20T01:28:31.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.348+0000 7f885559a700 1 -- 172.21.15.16:0/1420142917 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8848011990 con 0x7f88580d1000 2023-05-20T01:28:31.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.348+0000 7f885559a700 1 -- 172.21.15.16:0/1420142917 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8848011d20 con 0x7f88580d1000 2023-05-20T01:28:31.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.349+0000 7f885cd0c700 1 -- 172.21.15.16:0/1420142917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88580d1000 msgr2=0x7f88580d13d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:31.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.349+0000 7f885cd0c700 1 --2- 172.21.15.16:0/1420142917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88580d1000 0x7f88580d13d0 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f8848009840 tx=0x7f8848009b50 comp rx=0 tx=0).stop 2023-05-20T01:28:31.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.349+0000 7f885cd0c700 1 -- 172.21.15.16:0/1420142917 shutdown_connections 2023-05-20T01:28:31.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.349+0000 7f885cd0c700 1 --2- 172.21.15.16:0/1420142917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88580d1000 0x7f88580d13d0 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:31.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.349+0000 7f885cd0c700 1 -- 172.21.15.16:0/1420142917 >> 172.21.15.16:0/1420142917 conn(0x7f885806ae30 msgr2=0x7f885809a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:31.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.349+0000 7f885cd0c700 1 -- 172.21.15.16:0/1420142917 shutdown_connections 2023-05-20T01:28:31.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.349+0000 7f885cd0c700 1 -- 172.21.15.16:0/1420142917 wait complete. 2023-05-20T01:28:31.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.350+0000 7f885cd0c700 1 Processor -- start 2023-05-20T01:28:31.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.350+0000 7f885cd0c700 1 -- start start 2023-05-20T01:28:31.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.351+0000 7f885cd0c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f885806b620 0x7f88580c6750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:31.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.351+0000 7f885cd0c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f885806ba90 0x7f8858156110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:31.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.351+0000 7f885cd0c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88580d1000 0x7f8858156650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:31.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.351+0000 7f885cd0c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88580b5270 con 0x7f88580d1000 2023-05-20T01:28:31.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.351+0000 7f885cd0c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88580c55a0 con 0x7f885806ba90 2023-05-20T01:28:31.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.351+0000 7f885cd0c700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88580c5880 con 0x7f885806b620 2023-05-20T01:28:31.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.351+0000 7f8856d9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88580d1000 0x7f8858156650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:31.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.351+0000 7f8856d9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88580d1000 0x7f8858156650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40134/0 (socket says 172.21.15.16:40134) 2023-05-20T01:28:31.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.351+0000 7f8856d9d700 1 -- 172.21.15.16:0/206079764 learned_addr learned my addr 172.21.15.16:0/206079764 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:31.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.351+0000 7f885659c700 1 --2- 172.21.15.16:0/206079764 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f885806b620 0x7f88580c6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:31.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.351+0000 7f8855d9b700 1 --2- 172.21.15.16:0/206079764 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f885806ba90 0x7f8858156110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:31.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.351+0000 7f8856d9d700 1 -- 172.21.15.16:0/206079764 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f885806b620 msgr2=0x7f88580c6750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:31.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.351+0000 7f8856d9d700 1 --2- 172.21.15.16:0/206079764 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f885806b620 0x7f88580c6750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:31.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.351+0000 7f8856d9d700 1 -- 172.21.15.16:0/206079764 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f885806ba90 msgr2=0x7f8858156110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:31.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.351+0000 7f8856d9d700 1 --2- 172.21.15.16:0/206079764 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f885806ba90 0x7f8858156110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:31.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.351+0000 7f8856d9d700 1 -- 172.21.15.16:0/206079764 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88480094a0 con 0x7f88580d1000 2023-05-20T01:28:31.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.352+0000 7f8856d9d700 1 --2- 172.21.15.16:0/206079764 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88580d1000 0x7f8858156650 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f885000cb50 tx=0x7f885000cf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:31.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.352+0000 7f884effd700 1 -- 172.21.15.16:0/206079764 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8850003c00 con 0x7f88580d1000 2023-05-20T01:28:31.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.352+0000 7f884effd700 1 -- 172.21.15.16:0/206079764 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8850003d60 con 0x7f88580d1000 2023-05-20T01:28:31.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.352+0000 7f884effd700 1 -- 172.21.15.16:0/206079764 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f885000ab70 con 0x7f88580d1000 2023-05-20T01:28:31.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.352+0000 7f885cd0c700 1 -- 172.21.15.16:0/206079764 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88580cdab0 con 0x7f88580d1000 2023-05-20T01:28:31.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.352+0000 7f885cd0c700 1 -- 172.21.15.16:0/206079764 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88580cc6f0 con 0x7f88580d1000 2023-05-20T01:28:31.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.353+0000 7f885cd0c700 1 -- 172.21.15.16:0/206079764 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88580cdd90 con 0x7f88580d1000 2023-05-20T01:28:31.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.353+0000 7f884effd700 1 -- 172.21.15.16:0/206079764 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f885000acd0 con 0x7f88580d1000 2023-05-20T01:28:31.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.354+0000 7f884effd700 1 --2- 172.21.15.16:0/206079764 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f883806ace0 0x7f883806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:31.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.354+0000 7f885659c700 1 --2- 172.21.15.16:0/206079764 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f883806ace0 0x7f883806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:31.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.354+0000 7f884effd700 1 -- 172.21.15.16:0/206079764 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f8850089590 con 0x7f88580d1000 2023-05-20T01:28:31.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.354+0000 7f885659c700 1 --2- 172.21.15.16:0/206079764 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f883806ace0 0x7f883806d190 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f885815b600 tx=0x7f884801c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:31.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.358+0000 7f884effd700 1 -- 172.21.15.16:0/206079764 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f8850054590 con 0x7f88580d1000 2023-05-20T01:28:31.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.550+0000 7f885cd0c700 1 -- 172.21.15.16:0/206079764 --> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f88580747b0 con 0x7f883806ace0 2023-05-20T01:28:31.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.559+0000 7f884effd700 1 -- 172.21.15.16:0/206079764 <== mgr.4104 v2:172.21.15.16:6832/3817237998 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+312007 (secure 0 0 0) 0x7f88580747b0 con 0x7f883806ace0 2023-05-20T01:28:31.566 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:28:31.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.563+0000 7f885cd0c700 1 -- 172.21.15.16:0/206079764 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f883806ace0 msgr2=0x7f883806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:31.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.563+0000 7f885cd0c700 1 --2- 172.21.15.16:0/206079764 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f883806ace0 0x7f883806d190 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f885815b600 tx=0x7f884801c040 comp rx=0 tx=0).stop 2023-05-20T01:28:31.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.563+0000 7f885cd0c700 1 -- 172.21.15.16:0/206079764 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88580d1000 msgr2=0x7f8858156650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:31.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.563+0000 7f885cd0c700 1 --2- 172.21.15.16:0/206079764 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88580d1000 0x7f8858156650 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f885000cb50 tx=0x7f885000cf10 comp rx=0 tx=0).stop 2023-05-20T01:28:31.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.563+0000 7f885cd0c700 1 -- 172.21.15.16:0/206079764 shutdown_connections 2023-05-20T01:28:31.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.563+0000 7f885cd0c700 1 --2- 172.21.15.16:0/206079764 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88580d1000 0x7f8858156650 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:31.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.563+0000 7f885cd0c700 1 --2- 172.21.15.16:0/206079764 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f883806ace0 0x7f883806d190 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:31.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.563+0000 7f885cd0c700 1 --2- 172.21.15.16:0/206079764 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f885806b620 0x7f88580c6750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:31.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.563+0000 7f885cd0c700 1 --2- 172.21.15.16:0/206079764 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f885806ba90 0x7f8858156110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:31.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.563+0000 7f885cd0c700 1 -- 172.21.15.16:0/206079764 >> 172.21.15.16:0/206079764 conn(0x7f885806ae30 msgr2=0x7f88580b30b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:31.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.563+0000 7f885cd0c700 1 -- 172.21.15.16:0/206079764 shutdown_connections 2023-05-20T01:28:31.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.563+0000 7f885cd0c700 1 -- 172.21.15.16:0/206079764 wait complete. 2023-05-20T01:28:31.570 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2023-05-20T01:28:31.579 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":495,"stamp":"2023-05-20T01:28:31.480405+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":121,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":135856,"kb_used_data":14384,"kb_used_omap":0,"kb_used_meta":121408,"kb_avail":754838864,"statfs":{"total":773094113280,"available":772954996736,"internally_reserved":0,"allocated":14729216,"data_stored":8175166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":124321792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.815906"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-05-20T01:28:27.127769+0000","last_change":"2023-05-20T01:28:27.121288+0000","last_active":"2023-05-20T01:28:27.127769+0000","last_peered":"2023-05-20T01:28:27.127769+0000","last_clean":"2023-05-20T01:28:27.127769+0000","last_became_active":"2023-05-20T01:28:27.121001+0000","last_became_peered":"2023-05-20T01:28:27.121001+0000","last_unstale":"2023-05-20T01:28:27.127769+0000","last_undegraded":"2023-05-20T01:28:27.127769+0000","last_fullsized":"2023-05-20T01:28:27.127769+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:23:30.339851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:28.104881+0000","last_change":"2023-05-20T01:28:27.114001+0000","last_active":"2023-05-20T01:28:28.104881+0000","last_peered":"2023-05-20T01:28:28.104881+0000","last_clean":"2023-05-20T01:28:28.104881+0000","last_became_active":"2023-05-20T01:28:27.113740+0000","last_became_peered":"2023-05-20T01:28:27.113740+0000","last_unstale":"2023-05-20T01:28:28.104881+0000","last_undegraded":"2023-05-20T01:28:28.104881+0000","last_fullsized":"2023-05-20T01:28:28.104881+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:31:46.370507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:28.291232+0000","last_change":"2023-05-20T01:28:27.112397+0000","last_active":"2023-05-20T01:28:28.291232+0000","last_peered":"2023-05-20T01:28:28.291232+0000","last_clean":"2023-05-20T01:28:28.291232+0000","last_became_active":"2023-05-20T01:28:27.112116+0000","last_became_peered":"2023-05-20T01:28:27.112116+0000","last_unstale":"2023-05-20T01:28:28.291232+0000","last_undegraded":"2023-05-20T01:28:28.291232+0000","last_fullsized":"2023-05-20T01:28:28.291232+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T01:29:56.213301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-05-20T01:28:27.128556+0000","last_change":"2023-05-20T01:28:27.123975+0000","last_active":"2023-05-20T01:28:27.128556+0000","last_peered":"2023-05-20T01:28:27.128556+0000","last_clean":"2023-05-20T01:28:27.128556+0000","last_became_active":"2023-05-20T01:28:27.122875+0000","last_became_peered":"2023-05-20T01:28:27.122875+0000","last_unstale":"2023-05-20T01:28:27.128556+0000","last_undegraded":"2023-05-20T01:28:27.128556+0000","last_fullsized":"2023-05-20T01:28:27.128556+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:31:54.303518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-05-20T01:28:27.125756+0000","last_change":"2023-05-20T01:28:27.122001+0000","last_active":"2023-05-20T01:28:27.125756+0000","last_peered":"2023-05-20T01:28:27.125756+0000","last_clean":"2023-05-20T01:28:27.125756+0000","last_became_active":"2023-05-20T01:28:27.121585+0000","last_became_peered":"2023-05-20T01:28:27.121585+0000","last_unstale":"2023-05-20T01:28:27.125756+0000","last_undegraded":"2023-05-20T01:28:27.125756+0000","last_fullsized":"2023-05-20T01:28:27.125756+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:41:48.008633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:28.104654+0000","last_change":"2023-05-20T01:28:27.123814+0000","last_active":"2023-05-20T01:28:28.104654+0000","last_peered":"2023-05-20T01:28:28.104654+0000","last_clean":"2023-05-20T01:28:28.104654+0000","last_became_active":"2023-05-20T01:28:27.123520+0000","last_became_peered":"2023-05-20T01:28:27.123520+0000","last_unstale":"2023-05-20T01:28:28.104654+0000","last_undegraded":"2023-05-20T01:28:28.104654+0000","last_fullsized":"2023-05-20T01:28:28.104654+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:20:27.718807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:28.104687+0000","last_change":"2023-05-20T01:28:27.118737+0000","last_active":"2023-05-20T01:28:28.104687+0000","last_peered":"2023-05-20T01:28:28.104687+0000","last_clean":"2023-05-20T01:28:28.104687+0000","last_became_active":"2023-05-20T01:28:27.117977+0000","last_became_peered":"2023-05-20T01:28:27.117977+0000","last_unstale":"2023-05-20T01:28:28.104687+0000","last_undegraded":"2023-05-20T01:28:28.104687+0000","last_fullsized":"2023-05-20T01:28:28.104687+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:33:25.915827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:28.291040+0000","last_change":"2023-05-20T01:28:27.117875+0000","last_active":"2023-05-20T01:28:28.291040+0000","last_peered":"2023-05-20T01:28:28.291040+0000","last_clean":"2023-05-20T01:28:28.291040+0000","last_became_active":"2023-05-20T01:28:27.117606+0000","last_became_peered":"2023-05-20T01:28:27.117606+0000","last_unstale":"2023-05-20T01:28:28.291040+0000","last_undegraded":"2023-05-20T01:28:28.291040+0000","last_fullsized":"2023-05-20T01:28:28.291040+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:51:35.106788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:28.291195+0000","last_change":"2023-05-20T01:28:27.117145+0000","last_active":"2023-05-20T01:28:28.291195+0000","last_peered":"2023-05-20T01:28:28.291195+0000","last_clean":"2023-05-20T01:28:28.291195+0000","last_became_active":"2023-05-20T01:28:27.116894+0000","last_became_peered":"2023-05-20T01:28:27.116894+0000","last_unstale":"2023-05-20T01:28:28.291195+0000","last_undegraded":"2023-05-20T01:28:28.291195+0000","last_fullsized":"2023-05-20T01:28:28.291195+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:10:42.339813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:28.104858+0000","last_change":"2023-05-20T01:28:27.113665+0000","last_active":"2023-05-20T01:28:28.104858+0000","last_peered":"2023-05-20T01:28:28.104858+0000","last_clean":"2023-05-20T01:28:28.104858+0000","last_became_active":"2023-05-20T01:28:27.112664+0000","last_became_peered":"2023-05-20T01:28:27.112664+0000","last_unstale":"2023-05-20T01:28:28.104858+0000","last_undegraded":"2023-05-20T01:28:28.104858+0000","last_fullsized":"2023-05-20T01:28:28.104858+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:58:35.022389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:29.601805+0000","last_change":"2023-05-20T01:28:27.118216+0000","last_active":"2023-05-20T01:28:29.601805+0000","last_peered":"2023-05-20T01:28:29.601805+0000","last_clean":"2023-05-20T01:28:29.601805+0000","last_became_active":"2023-05-20T01:28:27.117601+0000","last_became_peered":"2023-05-20T01:28:27.117601+0000","last_unstale":"2023-05-20T01:28:29.601805+0000","last_undegraded":"2023-05-20T01:28:29.601805+0000","last_fullsized":"2023-05-20T01:28:29.601805+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:12:34.227931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:28.291211+0000","last_change":"2023-05-20T01:28:27.116573+0000","last_active":"2023-05-20T01:28:28.291211+0000","last_peered":"2023-05-20T01:28:28.291211+0000","last_clean":"2023-05-20T01:28:28.291211+0000","last_became_active":"2023-05-20T01:28:27.116264+0000","last_became_peered":"2023-05-20T01:28:27.116264+0000","last_unstale":"2023-05-20T01:28:28.291211+0000","last_undegraded":"2023-05-20T01:28:28.291211+0000","last_fullsized":"2023-05-20T01:28:28.291211+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:03:13.890512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-05-20T01:28:27.126125+0000","last_change":"2023-05-20T01:28:27.117009+0000","last_active":"2023-05-20T01:28:27.126125+0000","last_peered":"2023-05-20T01:28:27.126125+0000","last_clean":"2023-05-20T01:28:27.126125+0000","last_became_active":"2023-05-20T01:28:27.116774+0000","last_became_peered":"2023-05-20T01:28:27.116774+0000","last_unstale":"2023-05-20T01:28:27.126125+0000","last_undegraded":"2023-05-20T01:28:27.126125+0000","last_fullsized":"2023-05-20T01:28:27.126125+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:30:11.299773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:28.104607+0000","last_change":"2023-05-20T01:28:27.124550+0000","last_active":"2023-05-20T01:28:28.104607+0000","last_peered":"2023-05-20T01:28:28.104607+0000","last_clean":"2023-05-20T01:28:28.104607+0000","last_became_active":"2023-05-20T01:28:27.124270+0000","last_became_peered":"2023-05-20T01:28:27.124270+0000","last_unstale":"2023-05-20T01:28:28.104607+0000","last_undegraded":"2023-05-20T01:28:28.104607+0000","last_fullsized":"2023-05-20T01:28:28.104607+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:36:22.392704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-05-20T01:28:27.128618+0000","last_change":"2023-05-20T01:28:27.124483+0000","last_active":"2023-05-20T01:28:27.128618+0000","last_peered":"2023-05-20T01:28:27.128618+0000","last_clean":"2023-05-20T01:28:27.128618+0000","last_became_active":"2023-05-20T01:28:27.124084+0000","last_became_peered":"2023-05-20T01:28:27.124084+0000","last_unstale":"2023-05-20T01:28:27.128618+0000","last_undegraded":"2023-05-20T01:28:27.128618+0000","last_fullsized":"2023-05-20T01:28:27.128618+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T13:11:30.428632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:28.102153+0000","last_change":"2023-05-20T01:28:27.117982+0000","last_active":"2023-05-20T01:28:28.102153+0000","last_peered":"2023-05-20T01:28:28.102153+0000","last_clean":"2023-05-20T01:28:28.102153+0000","last_became_active":"2023-05-20T01:28:27.117726+0000","last_became_peered":"2023-05-20T01:28:27.117726+0000","last_unstale":"2023-05-20T01:28:28.102153+0000","last_undegraded":"2023-05-20T01:28:28.102153+0000","last_fullsized":"2023-05-20T01:28:28.102153+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:22:54.730939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:28.105155+0000","last_change":"2023-05-20T01:28:27.109900+0000","last_active":"2023-05-20T01:28:28.105155+0000","last_peered":"2023-05-20T01:28:28.105155+0000","last_clean":"2023-05-20T01:28:28.105155+0000","last_became_active":"2023-05-20T01:28:27.108667+0000","last_became_peered":"2023-05-20T01:28:27.108667+0000","last_unstale":"2023-05-20T01:28:28.105155+0000","last_undegraded":"2023-05-20T01:28:28.105155+0000","last_fullsized":"2023-05-20T01:28:28.105155+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:31:38.867656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:28.102853+0000","last_change":"2023-05-20T01:28:27.109262+0000","last_active":"2023-05-20T01:28:28.102853+0000","last_peered":"2023-05-20T01:28:28.102853+0000","last_clean":"2023-05-20T01:28:28.102853+0000","last_became_active":"2023-05-20T01:28:27.108995+0000","last_became_peered":"2023-05-20T01:28:27.108995+0000","last_unstale":"2023-05-20T01:28:28.102853+0000","last_undegraded":"2023-05-20T01:28:28.102853+0000","last_fullsized":"2023-05-20T01:28:28.102853+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:00:05.587896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:29.601842+0000","last_change":"2023-05-20T01:28:27.117867+0000","last_active":"2023-05-20T01:28:29.601842+0000","last_peered":"2023-05-20T01:28:29.601842+0000","last_clean":"2023-05-20T01:28:29.601842+0000","last_became_active":"2023-05-20T01:28:27.116175+0000","last_became_peered":"2023-05-20T01:28:27.116175+0000","last_unstale":"2023-05-20T01:28:29.601842+0000","last_undegraded":"2023-05-20T01:28:29.601842+0000","last_fullsized":"2023-05-20T01:28:29.601842+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:38:01.636748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:28.102531+0000","last_change":"2023-05-20T01:28:27.111932+0000","last_active":"2023-05-20T01:28:28.102531+0000","last_peered":"2023-05-20T01:28:28.102531+0000","last_clean":"2023-05-20T01:28:28.102531+0000","last_became_active":"2023-05-20T01:28:27.111699+0000","last_became_peered":"2023-05-20T01:28:27.111699+0000","last_unstale":"2023-05-20T01:28:28.102531+0000","last_undegraded":"2023-05-20T01:28:28.102531+0000","last_fullsized":"2023-05-20T01:28:28.102531+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:00:02.108926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-05-20T01:28:27.125727+0000","last_change":"2023-05-20T01:28:27.121833+0000","last_active":"2023-05-20T01:28:27.125727+0000","last_peered":"2023-05-20T01:28:27.125727+0000","last_clean":"2023-05-20T01:28:27.125727+0000","last_became_active":"2023-05-20T01:28:27.121226+0000","last_became_peered":"2023-05-20T01:28:27.121226+0000","last_unstale":"2023-05-20T01:28:27.125727+0000","last_undegraded":"2023-05-20T01:28:27.125727+0000","last_fullsized":"2023-05-20T01:28:27.125727+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:21:32.484022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:29.601551+0000","last_change":"2023-05-20T01:28:27.114932+0000","last_active":"2023-05-20T01:28:29.601551+0000","last_peered":"2023-05-20T01:28:29.601551+0000","last_clean":"2023-05-20T01:28:29.601551+0000","last_became_active":"2023-05-20T01:28:27.114682+0000","last_became_peered":"2023-05-20T01:28:27.114682+0000","last_unstale":"2023-05-20T01:28:29.601551+0000","last_undegraded":"2023-05-20T01:28:29.601551+0000","last_fullsized":"2023-05-20T01:28:29.601551+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:23:28.303320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:28.291109+0000","last_change":"2023-05-20T01:28:27.121610+0000","last_active":"2023-05-20T01:28:28.291109+0000","last_peered":"2023-05-20T01:28:28.291109+0000","last_clean":"2023-05-20T01:28:28.291109+0000","last_became_active":"2023-05-20T01:28:27.121294+0000","last_became_peered":"2023-05-20T01:28:27.121294+0000","last_unstale":"2023-05-20T01:28:28.291109+0000","last_undegraded":"2023-05-20T01:28:28.291109+0000","last_fullsized":"2023-05-20T01:28:28.291109+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:54:47.187098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:29.601506+0000","last_change":"2023-05-20T01:28:27.120994+0000","last_active":"2023-05-20T01:28:29.601506+0000","last_peered":"2023-05-20T01:28:29.601506+0000","last_clean":"2023-05-20T01:28:29.601506+0000","last_became_active":"2023-05-20T01:28:27.120782+0000","last_became_peered":"2023-05-20T01:28:27.120782+0000","last_unstale":"2023-05-20T01:28:29.601506+0000","last_undegraded":"2023-05-20T01:28:29.601506+0000","last_fullsized":"2023-05-20T01:28:29.601506+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:41:56.296593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:28.104791+0000","last_change":"2023-05-20T01:28:27.114072+0000","last_active":"2023-05-20T01:28:28.104791+0000","last_peered":"2023-05-20T01:28:28.104791+0000","last_clean":"2023-05-20T01:28:28.104791+0000","last_became_active":"2023-05-20T01:28:27.113786+0000","last_became_peered":"2023-05-20T01:28:27.113786+0000","last_unstale":"2023-05-20T01:28:28.104791+0000","last_undegraded":"2023-05-20T01:28:28.104791+0000","last_fullsized":"2023-05-20T01:28:28.104791+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:44:13.745200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:28.102859+0000","last_change":"2023-05-20T01:28:27.123864+0000","last_active":"2023-05-20T01:28:28.102859+0000","last_peered":"2023-05-20T01:28:28.102859+0000","last_clean":"2023-05-20T01:28:28.102859+0000","last_became_active":"2023-05-20T01:28:27.123444+0000","last_became_peered":"2023-05-20T01:28:27.123444+0000","last_unstale":"2023-05-20T01:28:28.102859+0000","last_undegraded":"2023-05-20T01:28:28.102859+0000","last_fullsized":"2023-05-20T01:28:28.102859+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:18:45.524372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-05-20T01:28:27.123919+0000","last_change":"2023-05-20T01:28:27.117032+0000","last_active":"2023-05-20T01:28:27.123919+0000","last_peered":"2023-05-20T01:28:27.123919+0000","last_clean":"2023-05-20T01:28:27.123919+0000","last_became_active":"2023-05-20T01:28:27.116801+0000","last_became_peered":"2023-05-20T01:28:27.116801+0000","last_unstale":"2023-05-20T01:28:27.123919+0000","last_undegraded":"2023-05-20T01:28:27.123919+0000","last_fullsized":"2023-05-20T01:28:27.123919+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:07:48.100900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-05-20T01:28:27.123922+0000","last_change":"2023-05-20T01:28:27.115783+0000","last_active":"2023-05-20T01:28:27.123922+0000","last_peered":"2023-05-20T01:28:27.123922+0000","last_clean":"2023-05-20T01:28:27.123922+0000","last_became_active":"2023-05-20T01:28:27.115175+0000","last_became_peered":"2023-05-20T01:28:27.115175+0000","last_unstale":"2023-05-20T01:28:27.123922+0000","last_undegraded":"2023-05-20T01:28:27.123922+0000","last_fullsized":"2023-05-20T01:28:27.123922+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:49:00.841918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-05-20T01:28:27.128758+0000","last_change":"2023-05-20T01:28:27.125691+0000","last_active":"2023-05-20T01:28:27.128758+0000","last_peered":"2023-05-20T01:28:27.128758+0000","last_clean":"2023-05-20T01:28:27.128758+0000","last_became_active":"2023-05-20T01:28:27.124527+0000","last_became_peered":"2023-05-20T01:28:27.124527+0000","last_unstale":"2023-05-20T01:28:27.128758+0000","last_undegraded":"2023-05-20T01:28:27.128758+0000","last_fullsized":"2023-05-20T01:28:27.128758+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:50:46.978921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-05-20T01:28:27.128565+0000","last_change":"2023-05-20T01:28:27.124126+0000","last_active":"2023-05-20T01:28:27.128565+0000","last_peered":"2023-05-20T01:28:27.128565+0000","last_clean":"2023-05-20T01:28:27.128565+0000","last_became_active":"2023-05-20T01:28:27.123394+0000","last_became_peered":"2023-05-20T01:28:27.123394+0000","last_unstale":"2023-05-20T01:28:27.128565+0000","last_undegraded":"2023-05-20T01:28:27.128565+0000","last_fullsized":"2023-05-20T01:28:27.128565+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:22:07.190005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-05-20T01:28:27.125857+0000","last_change":"2023-05-20T01:28:27.122974+0000","last_active":"2023-05-20T01:28:27.125857+0000","last_peered":"2023-05-20T01:28:27.125857+0000","last_clean":"2023-05-20T01:28:27.125857+0000","last_became_active":"2023-05-20T01:28:27.122699+0000","last_became_peered":"2023-05-20T01:28:27.122699+0000","last_unstale":"2023-05-20T01:28:27.125857+0000","last_undegraded":"2023-05-20T01:28:27.125857+0000","last_fullsized":"2023-05-20T01:28:27.125857+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:34:30.960285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:28.104659+0000","last_change":"2023-05-20T01:28:27.102528+0000","last_active":"2023-05-20T01:28:28.104659+0000","last_peered":"2023-05-20T01:28:28.104659+0000","last_clean":"2023-05-20T01:28:28.104659+0000","last_became_active":"2023-05-20T01:28:27.102222+0000","last_became_peered":"2023-05-20T01:28:27.102222+0000","last_unstale":"2023-05-20T01:28:28.104659+0000","last_undegraded":"2023-05-20T01:28:28.104659+0000","last_fullsized":"2023-05-20T01:28:28.104659+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:18:45.524372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:28.102725+0000","last_change":"2023-05-20T01:28:27.121554+0000","last_active":"2023-05-20T01:28:28.102725+0000","last_peered":"2023-05-20T01:28:28.102725+0000","last_clean":"2023-05-20T01:28:28.102725+0000","last_became_active":"2023-05-20T01:28:27.121253+0000","last_became_peered":"2023-05-20T01:28:27.121253+0000","last_unstale":"2023-05-20T01:28:28.102725+0000","last_undegraded":"2023-05-20T01:28:28.102725+0000","last_fullsized":"2023-05-20T01:28:28.102725+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:41:15.368853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"135'4","reported_seq":15,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-05-20T01:28:27.124968+0000","last_change":"2023-05-20T01:28:26.694591+0000","last_active":"2023-05-20T01:28:27.124968+0000","last_peered":"2023-05-20T01:28:27.124968+0000","last_clean":"2023-05-20T01:28:27.124968+0000","last_became_active":"2023-05-20T01:28:26.694306+0000","last_became_peered":"2023-05-20T01:28:26.694306+0000","last_unstale":"2023-05-20T01:28:27.124968+0000","last_undegraded":"2023-05-20T01:28:27.124968+0000","last_fullsized":"2023-05-20T01:28:27.124968+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_clean_scrub_stamp":"2023-05-20T01:28:25.659712+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:23:38.881118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:28.291368+0000","last_change":"2023-05-20T01:28:27.124985+0000","last_active":"2023-05-20T01:28:28.291368+0000","last_peered":"2023-05-20T01:28:28.291368+0000","last_clean":"2023-05-20T01:28:28.291368+0000","last_became_active":"2023-05-20T01:28:27.124719+0000","last_became_peered":"2023-05-20T01:28:27.124719+0000","last_unstale":"2023-05-20T01:28:28.291368+0000","last_undegraded":"2023-05-20T01:28:28.291368+0000","last_fullsized":"2023-05-20T01:28:28.291368+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:59:30.774683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"135'1","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-05-20T01:28:27.120222+0000","last_change":"2023-05-20T01:28:26.696838+0000","last_active":"2023-05-20T01:28:27.120222+0000","last_peered":"2023-05-20T01:28:27.120222+0000","last_clean":"2023-05-20T01:28:27.120222+0000","last_became_active":"2023-05-20T01:28:26.696442+0000","last_became_peered":"2023-05-20T01:28:26.696442+0000","last_unstale":"2023-05-20T01:28:27.120222+0000","last_undegraded":"2023-05-20T01:28:27.120222+0000","last_fullsized":"2023-05-20T01:28:27.120222+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_clean_scrub_stamp":"2023-05-20T01:28:25.659712+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:18:12.147035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-05-20T01:28:27.120887+0000","last_change":"2023-05-20T01:28:27.113350+0000","last_active":"2023-05-20T01:28:27.120887+0000","last_peered":"2023-05-20T01:28:27.120887+0000","last_clean":"2023-05-20T01:28:27.120887+0000","last_became_active":"2023-05-20T01:28:27.113076+0000","last_became_peered":"2023-05-20T01:28:27.113076+0000","last_unstale":"2023-05-20T01:28:27.120887+0000","last_undegraded":"2023-05-20T01:28:27.120887+0000","last_fullsized":"2023-05-20T01:28:27.120887+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:28:01.116965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1d","version":"135'2","reported_seq":15,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:29.601890+0000","last_change":"2023-05-20T01:28:26.694478+0000","last_active":"2023-05-20T01:28:29.601890+0000","last_peered":"2023-05-20T01:28:29.601890+0000","last_clean":"2023-05-20T01:28:29.601890+0000","last_became_active":"2023-05-20T01:28:26.694122+0000","last_became_peered":"2023-05-20T01:28:26.694122+0000","last_unstale":"2023-05-20T01:28:29.601890+0000","last_undegraded":"2023-05-20T01:28:29.601890+0000","last_fullsized":"2023-05-20T01:28:29.601890+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_clean_scrub_stamp":"2023-05-20T01:28:25.659712+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:27:55.665824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-05-20T01:28:27.120724+0000","last_change":"2023-05-20T01:28:27.113613+0000","last_active":"2023-05-20T01:28:27.120724+0000","last_peered":"2023-05-20T01:28:27.120724+0000","last_clean":"2023-05-20T01:28:27.120724+0000","last_became_active":"2023-05-20T01:28:27.113336+0000","last_became_peered":"2023-05-20T01:28:27.113336+0000","last_unstale":"2023-05-20T01:28:27.120724+0000","last_undegraded":"2023-05-20T01:28:27.120724+0000","last_fullsized":"2023-05-20T01:28:27.120724+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:55:27.219103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"135'2","reported_seq":15,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:28.104768+0000","last_change":"2023-05-20T01:28:26.693985+0000","last_active":"2023-05-20T01:28:28.104768+0000","last_peered":"2023-05-20T01:28:28.104768+0000","last_clean":"2023-05-20T01:28:28.104768+0000","last_became_active":"2023-05-20T01:28:26.693718+0000","last_became_peered":"2023-05-20T01:28:26.693718+0000","last_unstale":"2023-05-20T01:28:28.104768+0000","last_undegraded":"2023-05-20T01:28:28.104768+0000","last_fullsized":"2023-05-20T01:28:28.104768+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_clean_scrub_stamp":"2023-05-20T01:28:25.659712+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:58:19.162137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:28.102303+0000","last_change":"2023-05-20T01:28:27.104775+0000","last_active":"2023-05-20T01:28:28.102303+0000","last_peered":"2023-05-20T01:28:28.102303+0000","last_clean":"2023-05-20T01:28:28.102303+0000","last_became_active":"2023-05-20T01:28:27.104519+0000","last_became_peered":"2023-05-20T01:28:27.104519+0000","last_unstale":"2023-05-20T01:28:28.102303+0000","last_undegraded":"2023-05-20T01:28:28.102303+0000","last_fullsized":"2023-05-20T01:28:28.102303+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:39:32.802521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"135'1","reported_seq":14,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:28.104547+0000","last_change":"2023-05-20T01:28:26.692296+0000","last_active":"2023-05-20T01:28:28.104547+0000","last_peered":"2023-05-20T01:28:28.104547+0000","last_clean":"2023-05-20T01:28:28.104547+0000","last_became_active":"2023-05-20T01:28:26.691906+0000","last_became_peered":"2023-05-20T01:28:26.691906+0000","last_unstale":"2023-05-20T01:28:28.104547+0000","last_undegraded":"2023-05-20T01:28:28.104547+0000","last_fullsized":"2023-05-20T01:28:28.104547+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_clean_scrub_stamp":"2023-05-20T01:28:25.659712+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:03:59.465771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:28.105123+0000","last_change":"2023-05-20T01:28:27.107663+0000","last_active":"2023-05-20T01:28:28.105123+0000","last_peered":"2023-05-20T01:28:28.105123+0000","last_clean":"2023-05-20T01:28:28.105123+0000","last_became_active":"2023-05-20T01:28:27.107357+0000","last_became_peered":"2023-05-20T01:28:27.107357+0000","last_unstale":"2023-05-20T01:28:28.105123+0000","last_undegraded":"2023-05-20T01:28:28.105123+0000","last_fullsized":"2023-05-20T01:28:28.105123+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:51:28.678736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"135'1","reported_seq":14,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:28.102442+0000","last_change":"2023-05-20T01:28:26.691992+0000","last_active":"2023-05-20T01:28:28.102442+0000","last_peered":"2023-05-20T01:28:28.102442+0000","last_clean":"2023-05-20T01:28:28.102442+0000","last_became_active":"2023-05-20T01:28:26.690664+0000","last_became_peered":"2023-05-20T01:28:26.690664+0000","last_unstale":"2023-05-20T01:28:28.102442+0000","last_undegraded":"2023-05-20T01:28:28.102442+0000","last_fullsized":"2023-05-20T01:28:28.102442+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_clean_scrub_stamp":"2023-05-20T01:28:25.659712+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:05:40.675125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:28.290789+0000","last_change":"2023-05-20T01:28:27.104819+0000","last_active":"2023-05-20T01:28:28.290789+0000","last_peered":"2023-05-20T01:28:28.290789+0000","last_clean":"2023-05-20T01:28:28.290789+0000","last_became_active":"2023-05-20T01:28:27.104547+0000","last_became_peered":"2023-05-20T01:28:27.104547+0000","last_unstale":"2023-05-20T01:28:28.290789+0000","last_undegraded":"2023-05-20T01:28:28.290789+0000","last_fullsized":"2023-05-20T01:28:28.290789+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:10:08.036660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":13,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:28.104630+0000","last_change":"2023-05-20T01:28:26.690350+0000","last_active":"2023-05-20T01:28:28.104630+0000","last_peered":"2023-05-20T01:28:28.104630+0000","last_clean":"2023-05-20T01:28:28.104630+0000","last_became_active":"2023-05-20T01:28:26.690011+0000","last_became_peered":"2023-05-20T01:28:26.690011+0000","last_unstale":"2023-05-20T01:28:28.104630+0000","last_undegraded":"2023-05-20T01:28:28.104630+0000","last_fullsized":"2023-05-20T01:28:28.104630+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_clean_scrub_stamp":"2023-05-20T01:28:25.659712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:41:14.364064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:28.104438+0000","last_change":"2023-05-20T01:28:27.110577+0000","last_active":"2023-05-20T01:28:28.104438+0000","last_peered":"2023-05-20T01:28:28.104438+0000","last_clean":"2023-05-20T01:28:28.104438+0000","last_became_active":"2023-05-20T01:28:27.110309+0000","last_became_peered":"2023-05-20T01:28:27.110309+0000","last_unstale":"2023-05-20T01:28:28.104438+0000","last_undegraded":"2023-05-20T01:28:28.104438+0000","last_fullsized":"2023-05-20T01:28:28.104438+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:34:31.788215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":13,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:28.104511+0000","last_change":"2023-05-20T01:28:26.693970+0000","last_active":"2023-05-20T01:28:28.104511+0000","last_peered":"2023-05-20T01:28:28.104511+0000","last_clean":"2023-05-20T01:28:28.104511+0000","last_became_active":"2023-05-20T01:28:26.693345+0000","last_became_peered":"2023-05-20T01:28:26.693345+0000","last_unstale":"2023-05-20T01:28:28.104511+0000","last_undegraded":"2023-05-20T01:28:28.104511+0000","last_fullsized":"2023-05-20T01:28:28.104511+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_clean_scrub_stamp":"2023-05-20T01:28:25.659712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:11:44.990260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:28.291179+0000","last_change":"2023-05-20T01:28:27.123808+0000","last_active":"2023-05-20T01:28:28.291179+0000","last_peered":"2023-05-20T01:28:28.291179+0000","last_clean":"2023-05-20T01:28:28.291179+0000","last_became_active":"2023-05-20T01:28:27.123511+0000","last_became_peered":"2023-05-20T01:28:27.123511+0000","last_unstale":"2023-05-20T01:28:28.291179+0000","last_undegraded":"2023-05-20T01:28:28.291179+0000","last_fullsized":"2023-05-20T01:28:28.291179+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:50:56.574275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.f","version":"135'1","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-05-20T01:28:27.122329+0000","last_change":"2023-05-20T01:28:26.692018+0000","last_active":"2023-05-20T01:28:27.122329+0000","last_peered":"2023-05-20T01:28:27.122329+0000","last_clean":"2023-05-20T01:28:27.122329+0000","last_became_active":"2023-05-20T01:28:26.691764+0000","last_became_peered":"2023-05-20T01:28:26.691764+0000","last_unstale":"2023-05-20T01:28:27.122329+0000","last_undegraded":"2023-05-20T01:28:27.122329+0000","last_fullsized":"2023-05-20T01:28:27.122329+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_clean_scrub_stamp":"2023-05-20T01:28:25.659712+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:04:39.314176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-05-20T01:28:27.124077+0000","last_change":"2023-05-20T01:28:27.116069+0000","last_active":"2023-05-20T01:28:27.124077+0000","last_peered":"2023-05-20T01:28:27.124077+0000","last_clean":"2023-05-20T01:28:27.124077+0000","last_became_active":"2023-05-20T01:28:27.115799+0000","last_became_peered":"2023-05-20T01:28:27.115799+0000","last_unstale":"2023-05-20T01:28:27.124077+0000","last_undegraded":"2023-05-20T01:28:27.124077+0000","last_fullsized":"2023-05-20T01:28:27.124077+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:50:24.438722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.e","version":"135'2","reported_seq":15,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:29.601849+0000","last_change":"2023-05-20T01:28:26.695191+0000","last_active":"2023-05-20T01:28:29.601849+0000","last_peered":"2023-05-20T01:28:29.601849+0000","last_clean":"2023-05-20T01:28:29.601849+0000","last_became_active":"2023-05-20T01:28:26.694855+0000","last_became_peered":"2023-05-20T01:28:26.694855+0000","last_unstale":"2023-05-20T01:28:29.601849+0000","last_undegraded":"2023-05-20T01:28:29.601849+0000","last_fullsized":"2023-05-20T01:28:29.601849+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_clean_scrub_stamp":"2023-05-20T01:28:25.659712+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:41:37.683888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:28.102533+0000","last_change":"2023-05-20T01:28:27.123776+0000","last_active":"2023-05-20T01:28:28.102533+0000","last_peered":"2023-05-20T01:28:28.102533+0000","last_clean":"2023-05-20T01:28:28.102533+0000","last_became_active":"2023-05-20T01:28:27.123484+0000","last_became_peered":"2023-05-20T01:28:27.123484+0000","last_unstale":"2023-05-20T01:28:28.102533+0000","last_undegraded":"2023-05-20T01:28:28.102533+0000","last_fullsized":"2023-05-20T01:28:28.102533+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:21:01.822967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.d","version":"135'1","reported_seq":14,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:28.105223+0000","last_change":"2023-05-20T01:28:26.683534+0000","last_active":"2023-05-20T01:28:28.105223+0000","last_peered":"2023-05-20T01:28:28.105223+0000","last_clean":"2023-05-20T01:28:28.105223+0000","last_became_active":"2023-05-20T01:28:26.682870+0000","last_became_peered":"2023-05-20T01:28:26.682870+0000","last_unstale":"2023-05-20T01:28:28.105223+0000","last_undegraded":"2023-05-20T01:28:28.105223+0000","last_fullsized":"2023-05-20T01:28:28.105223+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_clean_scrub_stamp":"2023-05-20T01:28:25.659712+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:11:10.553608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:28.104863+0000","last_change":"2023-05-20T01:28:27.112662+0000","last_active":"2023-05-20T01:28:28.104863+0000","last_peered":"2023-05-20T01:28:28.104863+0000","last_clean":"2023-05-20T01:28:28.104863+0000","last_became_active":"2023-05-20T01:28:27.112398+0000","last_became_peered":"2023-05-20T01:28:27.112398+0000","last_unstale":"2023-05-20T01:28:28.104863+0000","last_undegraded":"2023-05-20T01:28:28.104863+0000","last_fullsized":"2023-05-20T01:28:28.104863+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:32:38.424065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-05-20T01:28:27.104438+0000","last_change":"2023-05-20T01:28:26.697014+0000","last_active":"2023-05-20T01:28:27.104438+0000","last_peered":"2023-05-20T01:28:27.104438+0000","last_clean":"2023-05-20T01:28:27.104438+0000","last_became_active":"2023-05-20T01:28:26.696580+0000","last_became_peered":"2023-05-20T01:28:26.696580+0000","last_unstale":"2023-05-20T01:28:27.104438+0000","last_undegraded":"2023-05-20T01:28:27.104438+0000","last_fullsized":"2023-05-20T01:28:27.104438+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_clean_scrub_stamp":"2023-05-20T01:28:25.659712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:34:15.371890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-05-20T01:28:27.123671+0000","last_change":"2023-05-20T01:28:27.117167+0000","last_active":"2023-05-20T01:28:27.123671+0000","last_peered":"2023-05-20T01:28:27.123671+0000","last_clean":"2023-05-20T01:28:27.123671+0000","last_became_active":"2023-05-20T01:28:27.116889+0000","last_became_peered":"2023-05-20T01:28:27.116889+0000","last_unstale":"2023-05-20T01:28:27.123671+0000","last_undegraded":"2023-05-20T01:28:27.123671+0000","last_fullsized":"2023-05-20T01:28:27.123671+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:31:04.819577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.b","version":"135'2","reported_seq":13,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-05-20T01:28:27.126078+0000","last_change":"2023-05-20T01:28:26.687062+0000","last_active":"2023-05-20T01:28:27.126078+0000","last_peered":"2023-05-20T01:28:27.126078+0000","last_clean":"2023-05-20T01:28:27.126078+0000","last_became_active":"2023-05-20T01:28:26.686794+0000","last_became_peered":"2023-05-20T01:28:26.686794+0000","last_unstale":"2023-05-20T01:28:27.126078+0000","last_undegraded":"2023-05-20T01:28:27.126078+0000","last_fullsized":"2023-05-20T01:28:27.126078+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_clean_scrub_stamp":"2023-05-20T01:28:25.659712+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:06:53.372054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:28.105013+0000","last_change":"2023-05-20T01:28:27.103203+0000","last_active":"2023-05-20T01:28:28.105013+0000","last_peered":"2023-05-20T01:28:28.105013+0000","last_clean":"2023-05-20T01:28:28.105013+0000","last_became_active":"2023-05-20T01:28:27.102938+0000","last_became_peered":"2023-05-20T01:28:27.102938+0000","last_unstale":"2023-05-20T01:28:28.105013+0000","last_undegraded":"2023-05-20T01:28:28.105013+0000","last_fullsized":"2023-05-20T01:28:28.105013+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:12:37.307273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":13,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:28.102630+0000","last_change":"2023-05-20T01:28:26.692534+0000","last_active":"2023-05-20T01:28:28.102630+0000","last_peered":"2023-05-20T01:28:28.102630+0000","last_clean":"2023-05-20T01:28:28.102630+0000","last_became_active":"2023-05-20T01:28:26.692259+0000","last_became_peered":"2023-05-20T01:28:26.692259+0000","last_unstale":"2023-05-20T01:28:28.102630+0000","last_undegraded":"2023-05-20T01:28:28.102630+0000","last_fullsized":"2023-05-20T01:28:28.102630+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_clean_scrub_stamp":"2023-05-20T01:28:25.659712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:48:51.119722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:28.104366+0000","last_change":"2023-05-20T01:28:27.114722+0000","last_active":"2023-05-20T01:28:28.104366+0000","last_peered":"2023-05-20T01:28:28.104366+0000","last_clean":"2023-05-20T01:28:28.104366+0000","last_became_active":"2023-05-20T01:28:27.114464+0000","last_became_peered":"2023-05-20T01:28:27.114464+0000","last_unstale":"2023-05-20T01:28:28.104366+0000","last_undegraded":"2023-05-20T01:28:28.104366+0000","last_fullsized":"2023-05-20T01:28:28.104366+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:39:54.356057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":13,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:28.102089+0000","last_change":"2023-05-20T01:28:26.683304+0000","last_active":"2023-05-20T01:28:28.102089+0000","last_peered":"2023-05-20T01:28:28.102089+0000","last_clean":"2023-05-20T01:28:28.102089+0000","last_became_active":"2023-05-20T01:28:26.682988+0000","last_became_peered":"2023-05-20T01:28:26.682988+0000","last_unstale":"2023-05-20T01:28:28.102089+0000","last_undegraded":"2023-05-20T01:28:28.102089+0000","last_fullsized":"2023-05-20T01:28:28.102089+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_clean_scrub_stamp":"2023-05-20T01:28:25.659712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:17:14.758553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-05-20T01:28:27.128467+0000","last_change":"2023-05-20T01:28:27.125052+0000","last_active":"2023-05-20T01:28:27.128467+0000","last_peered":"2023-05-20T01:28:27.128467+0000","last_clean":"2023-05-20T01:28:27.128467+0000","last_became_active":"2023-05-20T01:28:27.124750+0000","last_became_peered":"2023-05-20T01:28:27.124750+0000","last_unstale":"2023-05-20T01:28:27.128467+0000","last_undegraded":"2023-05-20T01:28:27.128467+0000","last_fullsized":"2023-05-20T01:28:27.128467+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:12:34.227931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":13,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:29.601585+0000","last_change":"2023-05-20T01:28:26.696466+0000","last_active":"2023-05-20T01:28:29.601585+0000","last_peered":"2023-05-20T01:28:29.601585+0000","last_clean":"2023-05-20T01:28:29.601585+0000","last_became_active":"2023-05-20T01:28:26.696005+0000","last_became_peered":"2023-05-20T01:28:26.696005+0000","last_unstale":"2023-05-20T01:28:29.601585+0000","last_undegraded":"2023-05-20T01:28:29.601585+0000","last_fullsized":"2023-05-20T01:28:29.601585+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_clean_scrub_stamp":"2023-05-20T01:28:25.659712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:55:26.214314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:28.105316+0000","last_change":"2023-05-20T01:28:27.121067+0000","last_active":"2023-05-20T01:28:28.105316+0000","last_peered":"2023-05-20T01:28:28.105316+0000","last_clean":"2023-05-20T01:28:28.105316+0000","last_became_active":"2023-05-20T01:28:27.120742+0000","last_became_peered":"2023-05-20T01:28:27.120742+0000","last_unstale":"2023-05-20T01:28:28.105316+0000","last_undegraded":"2023-05-20T01:28:28.105316+0000","last_fullsized":"2023-05-20T01:28:28.105316+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:17:25.249433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.7","version":"135'2","reported_seq":13,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-05-20T01:28:27.123699+0000","last_change":"2023-05-20T01:28:26.684920+0000","last_active":"2023-05-20T01:28:27.123699+0000","last_peered":"2023-05-20T01:28:27.123699+0000","last_clean":"2023-05-20T01:28:27.123699+0000","last_became_active":"2023-05-20T01:28:26.684628+0000","last_became_peered":"2023-05-20T01:28:26.684628+0000","last_unstale":"2023-05-20T01:28:27.123699+0000","last_undegraded":"2023-05-20T01:28:27.123699+0000","last_fullsized":"2023-05-20T01:28:27.123699+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_clean_scrub_stamp":"2023-05-20T01:28:25.659712+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:34:33.232021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-05-20T01:28:27.128534+0000","last_change":"2023-05-20T01:28:27.119165+0000","last_active":"2023-05-20T01:28:27.128534+0000","last_peered":"2023-05-20T01:28:27.128534+0000","last_clean":"2023-05-20T01:28:27.128534+0000","last_became_active":"2023-05-20T01:28:27.118928+0000","last_became_peered":"2023-05-20T01:28:27.118928+0000","last_unstale":"2023-05-20T01:28:27.128534+0000","last_undegraded":"2023-05-20T01:28:27.128534+0000","last_fullsized":"2023-05-20T01:28:27.128534+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:27:16.782676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-05-20T01:28:27.103441+0000","last_change":"2023-05-20T01:28:26.693198+0000","last_active":"2023-05-20T01:28:27.103441+0000","last_peered":"2023-05-20T01:28:27.103441+0000","last_clean":"2023-05-20T01:28:27.103441+0000","last_became_active":"2023-05-20T01:28:26.692911+0000","last_became_peered":"2023-05-20T01:28:26.692911+0000","last_unstale":"2023-05-20T01:28:27.103441+0000","last_undegraded":"2023-05-20T01:28:27.103441+0000","last_fullsized":"2023-05-20T01:28:27.103441+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_clean_scrub_stamp":"2023-05-20T01:28:25.659712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:41:13.080238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:28.104742+0000","last_change":"2023-05-20T01:28:27.114743+0000","last_active":"2023-05-20T01:28:28.104742+0000","last_peered":"2023-05-20T01:28:28.104742+0000","last_clean":"2023-05-20T01:28:28.104742+0000","last_became_active":"2023-05-20T01:28:27.114454+0000","last_became_peered":"2023-05-20T01:28:27.114454+0000","last_unstale":"2023-05-20T01:28:28.104742+0000","last_undegraded":"2023-05-20T01:28:28.104742+0000","last_fullsized":"2023-05-20T01:28:28.104742+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:24:34.647958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":13,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:28.104903+0000","last_change":"2023-05-20T01:28:26.695359+0000","last_active":"2023-05-20T01:28:28.104903+0000","last_peered":"2023-05-20T01:28:28.104903+0000","last_clean":"2023-05-20T01:28:28.104903+0000","last_became_active":"2023-05-20T01:28:26.695082+0000","last_became_peered":"2023-05-20T01:28:26.695082+0000","last_unstale":"2023-05-20T01:28:28.104903+0000","last_undegraded":"2023-05-20T01:28:28.104903+0000","last_fullsized":"2023-05-20T01:28:28.104903+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_clean_scrub_stamp":"2023-05-20T01:28:25.659712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:22:23.906600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:28.105155+0000","last_change":"2023-05-20T01:28:27.124168+0000","last_active":"2023-05-20T01:28:28.105155+0000","last_peered":"2023-05-20T01:28:28.105155+0000","last_clean":"2023-05-20T01:28:28.105155+0000","last_became_active":"2023-05-20T01:28:27.123907+0000","last_became_peered":"2023-05-20T01:28:27.123907+0000","last_unstale":"2023-05-20T01:28:28.105155+0000","last_undegraded":"2023-05-20T01:28:28.105155+0000","last_fullsized":"2023-05-20T01:28:28.105155+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:57:28.027837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":13,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:29.601711+0000","last_change":"2023-05-20T01:28:26.694256+0000","last_active":"2023-05-20T01:28:29.601711+0000","last_peered":"2023-05-20T01:28:29.601711+0000","last_clean":"2023-05-20T01:28:29.601711+0000","last_became_active":"2023-05-20T01:28:26.693870+0000","last_became_peered":"2023-05-20T01:28:26.693870+0000","last_unstale":"2023-05-20T01:28:29.601711+0000","last_undegraded":"2023-05-20T01:28:29.601711+0000","last_fullsized":"2023-05-20T01:28:29.601711+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_clean_scrub_stamp":"2023-05-20T01:28:25.659712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:34:29.955497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-05-20T01:28:27.120640+0000","last_change":"2023-05-20T01:28:27.112949+0000","last_active":"2023-05-20T01:28:27.120640+0000","last_peered":"2023-05-20T01:28:27.120640+0000","last_clean":"2023-05-20T01:28:27.120640+0000","last_became_active":"2023-05-20T01:28:27.112651+0000","last_became_peered":"2023-05-20T01:28:27.112651+0000","last_unstale":"2023-05-20T01:28:27.120640+0000","last_undegraded":"2023-05-20T01:28:27.120640+0000","last_fullsized":"2023-05-20T01:28:27.120640+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:23:28.303320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":13,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:28.104928+0000","last_change":"2023-05-20T01:28:26.692720+0000","last_active":"2023-05-20T01:28:28.104928+0000","last_peered":"2023-05-20T01:28:28.104928+0000","last_clean":"2023-05-20T01:28:28.104928+0000","last_became_active":"2023-05-20T01:28:26.692433+0000","last_became_peered":"2023-05-20T01:28:26.692433+0000","last_unstale":"2023-05-20T01:28:28.104928+0000","last_undegraded":"2023-05-20T01:28:28.104928+0000","last_fullsized":"2023-05-20T01:28:28.104928+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_clean_scrub_stamp":"2023-05-20T01:28:25.659712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:07:59.020182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-05-20T01:28:27.130732+0000","last_change":"2023-05-20T01:28:27.126550+0000","last_active":"2023-05-20T01:28:27.130732+0000","last_peered":"2023-05-20T01:28:27.130732+0000","last_clean":"2023-05-20T01:28:27.130732+0000","last_became_active":"2023-05-20T01:28:27.126233+0000","last_became_peered":"2023-05-20T01:28:27.126233+0000","last_unstale":"2023-05-20T01:28:27.130732+0000","last_undegraded":"2023-05-20T01:28:27.130732+0000","last_fullsized":"2023-05-20T01:28:27.130732+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:42:47.941689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-05-20T01:28:27.104612+0000","last_change":"2023-05-20T01:28:26.696774+0000","last_active":"2023-05-20T01:28:27.104612+0000","last_peered":"2023-05-20T01:28:27.104612+0000","last_clean":"2023-05-20T01:28:27.104612+0000","last_became_active":"2023-05-20T01:28:26.696485+0000","last_became_peered":"2023-05-20T01:28:26.696485+0000","last_unstale":"2023-05-20T01:28:27.104612+0000","last_undegraded":"2023-05-20T01:28:27.104612+0000","last_fullsized":"2023-05-20T01:28:27.104612+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_clean_scrub_stamp":"2023-05-20T01:28:25.659712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:59:07.152477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:28.102190+0000","last_change":"2023-05-20T01:28:27.120405+0000","last_active":"2023-05-20T01:28:28.102190+0000","last_peered":"2023-05-20T01:28:28.102190+0000","last_clean":"2023-05-20T01:28:28.102190+0000","last_became_active":"2023-05-20T01:28:27.120117+0000","last_became_peered":"2023-05-20T01:28:27.120117+0000","last_unstale":"2023-05-20T01:28:28.102190+0000","last_undegraded":"2023-05-20T01:28:28.102190+0000","last_fullsized":"2023-05-20T01:28:28.102190+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:21:33.984873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":13,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:28.102087+0000","last_change":"2023-05-20T01:28:26.682752+0000","last_active":"2023-05-20T01:28:28.102087+0000","last_peered":"2023-05-20T01:28:28.102087+0000","last_clean":"2023-05-20T01:28:28.102087+0000","last_became_active":"2023-05-20T01:28:26.682472+0000","last_became_peered":"2023-05-20T01:28:26.682472+0000","last_unstale":"2023-05-20T01:28:28.102087+0000","last_undegraded":"2023-05-20T01:28:28.102087+0000","last_fullsized":"2023-05-20T01:28:28.102087+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_clean_scrub_stamp":"2023-05-20T01:28:25.659712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:40:05.403429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:28.104922+0000","last_change":"2023-05-20T01:28:27.111490+0000","last_active":"2023-05-20T01:28:28.104922+0000","last_peered":"2023-05-20T01:28:28.104922+0000","last_clean":"2023-05-20T01:28:28.104922+0000","last_became_active":"2023-05-20T01:28:27.111220+0000","last_became_peered":"2023-05-20T01:28:27.111220+0000","last_unstale":"2023-05-20T01:28:28.104922+0000","last_undegraded":"2023-05-20T01:28:28.104922+0000","last_fullsized":"2023-05-20T01:28:28.104922+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:50:40.180843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":13,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:29.601644+0000","last_change":"2023-05-20T01:28:26.694747+0000","last_active":"2023-05-20T01:28:29.601644+0000","last_peered":"2023-05-20T01:28:29.601644+0000","last_clean":"2023-05-20T01:28:29.601644+0000","last_became_active":"2023-05-20T01:28:26.694300+0000","last_became_peered":"2023-05-20T01:28:26.694300+0000","last_unstale":"2023-05-20T01:28:29.601644+0000","last_undegraded":"2023-05-20T01:28:29.601644+0000","last_fullsized":"2023-05-20T01:28:29.601644+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_clean_scrub_stamp":"2023-05-20T01:28:25.659712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:51:45.694079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-05-20T01:28:27.128403+0000","last_change":"2023-05-20T01:28:27.122177+0000","last_active":"2023-05-20T01:28:27.128403+0000","last_peered":"2023-05-20T01:28:27.128403+0000","last_clean":"2023-05-20T01:28:27.128403+0000","last_became_active":"2023-05-20T01:28:27.120626+0000","last_became_peered":"2023-05-20T01:28:27.120626+0000","last_unstale":"2023-05-20T01:28:27.128403+0000","last_undegraded":"2023-05-20T01:28:27.128403+0000","last_fullsized":"2023-05-20T01:28:27.128403+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:02:44.921661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":13,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:28.102956+0000","last_change":"2023-05-20T01:28:26.691759+0000","last_active":"2023-05-20T01:28:28.102956+0000","last_peered":"2023-05-20T01:28:28.102956+0000","last_clean":"2023-05-20T01:28:28.102956+0000","last_became_active":"2023-05-20T01:28:26.691412+0000","last_became_peered":"2023-05-20T01:28:26.691412+0000","last_unstale":"2023-05-20T01:28:28.102956+0000","last_undegraded":"2023-05-20T01:28:28.102956+0000","last_fullsized":"2023-05-20T01:28:28.102956+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_clean_scrub_stamp":"2023-05-20T01:28:25.659712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:20:26.714018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":242,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:28.105217+0000","last_change":"2023-05-20T01:16:23.289068+0000","last_active":"2023-05-20T01:28:28.105217+0000","last_peered":"2023-05-20T01:28:28.105217+0000","last_clean":"2023-05-20T01:28:28.105217+0000","last_became_active":"2023-05-20T01:16:23.288781+0000","last_became_peered":"2023-05-20T01:16:23.288781+0000","last_unstale":"2023-05-20T01:28:28.105217+0000","last_undegraded":"2023-05-20T01:28:28.105217+0000","last_fullsized":"2023-05-20T01:28:28.105217+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T01:29:53.661129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-05-20T01:28:27.128648+0000","last_change":"2023-05-20T01:28:27.124712+0000","last_active":"2023-05-20T01:28:27.128648+0000","last_peered":"2023-05-20T01:28:27.128648+0000","last_clean":"2023-05-20T01:28:27.128648+0000","last_became_active":"2023-05-20T01:28:27.122733+0000","last_became_peered":"2023-05-20T01:28:27.122733+0000","last_unstale":"2023-05-20T01:28:27.128648+0000","last_undegraded":"2023-05-20T01:28:27.128648+0000","last_fullsized":"2023-05-20T01:28:27.128648+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:38:46.482786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-05-20T01:28:27.108063+0000","last_change":"2023-05-20T01:28:26.686983+0000","last_active":"2023-05-20T01:28:27.108063+0000","last_peered":"2023-05-20T01:28:27.108063+0000","last_clean":"2023-05-20T01:28:27.108063+0000","last_became_active":"2023-05-20T01:28:26.686713+0000","last_became_peered":"2023-05-20T01:28:26.686713+0000","last_unstale":"2023-05-20T01:28:27.108063+0000","last_undegraded":"2023-05-20T01:28:27.108063+0000","last_fullsized":"2023-05-20T01:28:27.108063+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_clean_scrub_stamp":"2023-05-20T01:28:25.659712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:20:30.818187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":254,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:28.102610+0000","last_change":"2023-05-20T01:16:22.829324+0000","last_active":"2023-05-20T01:28:28.102610+0000","last_peered":"2023-05-20T01:28:28.102610+0000","last_clean":"2023-05-20T01:28:28.102610+0000","last_became_active":"2023-05-20T01:16:22.828964+0000","last_became_peered":"2023-05-20T01:16:22.828964+0000","last_unstale":"2023-05-20T01:28:28.102610+0000","last_undegraded":"2023-05-20T01:28:28.102610+0000","last_fullsized":"2023-05-20T01:28:28.102610+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:02:43.785285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:28.290913+0000","last_change":"2023-05-20T01:28:27.116232+0000","last_active":"2023-05-20T01:28:28.290913+0000","last_peered":"2023-05-20T01:28:28.290913+0000","last_clean":"2023-05-20T01:28:28.290913+0000","last_became_active":"2023-05-20T01:28:27.115905+0000","last_became_peered":"2023-05-20T01:28:27.115905+0000","last_unstale":"2023-05-20T01:28:28.290913+0000","last_undegraded":"2023-05-20T01:28:28.290913+0000","last_fullsized":"2023-05-20T01:28:28.290913+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:04:43.248337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.10","version":"135'8","reported_seq":21,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:28.290755+0000","last_change":"2023-05-20T01:28:26.689073+0000","last_active":"2023-05-20T01:28:28.290755+0000","last_peered":"2023-05-20T01:28:28.290755+0000","last_clean":"2023-05-20T01:28:28.290755+0000","last_became_active":"2023-05-20T01:28:26.688542+0000","last_became_peered":"2023-05-20T01:28:26.688542+0000","last_unstale":"2023-05-20T01:28:28.290755+0000","last_undegraded":"2023-05-20T01:28:28.290755+0000","last_fullsized":"2023-05-20T01:28:28.290755+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_clean_scrub_stamp":"2023-05-20T01:28:25.659712+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:23:27.047072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":220,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:28.104742+0000","last_change":"2023-05-20T01:16:22.834754+0000","last_active":"2023-05-20T01:28:28.104742+0000","last_peered":"2023-05-20T01:28:28.104742+0000","last_clean":"2023-05-20T01:28:28.104742+0000","last_became_active":"2023-05-20T01:16:22.834437+0000","last_became_peered":"2023-05-20T01:16:22.834437+0000","last_unstale":"2023-05-20T01:28:28.104742+0000","last_undegraded":"2023-05-20T01:28:28.104742+0000","last_fullsized":"2023-05-20T01:28:28.104742+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:41:13.799210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":351,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-05-20T01:28:27.103303+0000","last_change":"2023-05-20T01:16:22.834920+0000","last_active":"2023-05-20T01:28:27.103303+0000","last_peered":"2023-05-20T01:28:27.103303+0000","last_clean":"2023-05-20T01:28:27.103303+0000","last_became_active":"2023-05-20T01:16:22.833924+0000","last_became_peered":"2023-05-20T01:16:22.833924+0000","last_unstale":"2023-05-20T01:28:27.103303+0000","last_undegraded":"2023-05-20T01:28:27.103303+0000","last_fullsized":"2023-05-20T01:28:27.103303+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:22:16.802805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:28.104394+0000","last_change":"2023-05-20T01:28:27.119031+0000","last_active":"2023-05-20T01:28:28.104394+0000","last_peered":"2023-05-20T01:28:28.104394+0000","last_clean":"2023-05-20T01:28:28.104394+0000","last_became_active":"2023-05-20T01:28:27.118764+0000","last_became_peered":"2023-05-20T01:28:27.118764+0000","last_unstale":"2023-05-20T01:28:28.104394+0000","last_undegraded":"2023-05-20T01:28:28.104394+0000","last_fullsized":"2023-05-20T01:28:28.104394+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:55:54.880516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":13,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:28.104630+0000","last_change":"2023-05-20T01:28:26.691756+0000","last_active":"2023-05-20T01:28:28.104630+0000","last_peered":"2023-05-20T01:28:28.104630+0000","last_clean":"2023-05-20T01:28:28.104630+0000","last_became_active":"2023-05-20T01:28:26.691446+0000","last_became_peered":"2023-05-20T01:28:26.691446+0000","last_unstale":"2023-05-20T01:28:28.104630+0000","last_undegraded":"2023-05-20T01:28:28.104630+0000","last_fullsized":"2023-05-20T01:28:28.104630+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_clean_scrub_stamp":"2023-05-20T01:28:25.659712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:38:23.028213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":200,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-05-20T01:28:27.105137+0000","last_change":"2023-05-20T01:16:23.290259+0000","last_active":"2023-05-20T01:28:27.105137+0000","last_peered":"2023-05-20T01:28:27.105137+0000","last_clean":"2023-05-20T01:28:27.105137+0000","last_became_active":"2023-05-20T01:16:23.289911+0000","last_became_peered":"2023-05-20T01:16:23.289911+0000","last_unstale":"2023-05-20T01:28:27.105137+0000","last_undegraded":"2023-05-20T01:28:27.105137+0000","last_fullsized":"2023-05-20T01:28:27.105137+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:29:51.437617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-05-20T01:28:27.128124+0000","last_change":"2023-05-20T01:28:27.124801+0000","last_active":"2023-05-20T01:28:27.128124+0000","last_peered":"2023-05-20T01:28:27.128124+0000","last_clean":"2023-05-20T01:28:27.128124+0000","last_became_active":"2023-05-20T01:28:27.123965+0000","last_became_peered":"2023-05-20T01:28:27.123965+0000","last_unstale":"2023-05-20T01:28:27.128124+0000","last_undegraded":"2023-05-20T01:28:27.128124+0000","last_fullsized":"2023-05-20T01:28:27.128124+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:03:35.578428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"135'2","reported_seq":13,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-05-20T01:28:27.127518+0000","last_change":"2023-05-20T01:28:26.693285+0000","last_active":"2023-05-20T01:28:27.127518+0000","last_peered":"2023-05-20T01:28:27.127518+0000","last_clean":"2023-05-20T01:28:27.127518+0000","last_became_active":"2023-05-20T01:28:26.693057+0000","last_became_peered":"2023-05-20T01:28:26.693057+0000","last_unstale":"2023-05-20T01:28:27.127518+0000","last_undegraded":"2023-05-20T01:28:27.127518+0000","last_fullsized":"2023-05-20T01:28:27.127518+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_clean_scrub_stamp":"2023-05-20T01:28:25.659712+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:52:40.552794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":201,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-05-20T01:28:27.103443+0000","last_change":"2023-05-20T01:16:23.290207+0000","last_active":"2023-05-20T01:28:27.103443+0000","last_peered":"2023-05-20T01:28:27.103443+0000","last_clean":"2023-05-20T01:28:27.103443+0000","last_became_active":"2023-05-20T01:16:23.289928+0000","last_became_peered":"2023-05-20T01:16:23.289928+0000","last_unstale":"2023-05-20T01:28:27.103443+0000","last_undegraded":"2023-05-20T01:28:27.103443+0000","last_fullsized":"2023-05-20T01:28:27.103443+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:48:20.917850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-05-20T01:28:27.128691+0000","last_change":"2023-05-20T01:28:27.125501+0000","last_active":"2023-05-20T01:28:27.128691+0000","last_peered":"2023-05-20T01:28:27.128691+0000","last_clean":"2023-05-20T01:28:27.128691+0000","last_became_active":"2023-05-20T01:28:27.124194+0000","last_became_peered":"2023-05-20T01:28:27.124194+0000","last_unstale":"2023-05-20T01:28:27.128691+0000","last_undegraded":"2023-05-20T01:28:27.128691+0000","last_fullsized":"2023-05-20T01:28:27.128691+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T13:12:53.827858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-05-20T01:28:27.114846+0000","last_change":"2023-05-20T01:28:26.685301+0000","last_active":"2023-05-20T01:28:27.114846+0000","last_peered":"2023-05-20T01:28:27.114846+0000","last_clean":"2023-05-20T01:28:27.114846+0000","last_became_active":"2023-05-20T01:28:26.685007+0000","last_became_peered":"2023-05-20T01:28:26.685007+0000","last_unstale":"2023-05-20T01:28:27.114846+0000","last_undegraded":"2023-05-20T01:28:27.114846+0000","last_fullsized":"2023-05-20T01:28:27.114846+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_clean_scrub_stamp":"2023-05-20T01:28:25.659712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T01:52:41.398890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":202,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-05-20T01:28:27.104385+0000","last_change":"2023-05-20T01:16:23.290538+0000","last_active":"2023-05-20T01:28:27.104385+0000","last_peered":"2023-05-20T01:28:27.104385+0000","last_clean":"2023-05-20T01:28:27.104385+0000","last_became_active":"2023-05-20T01:16:23.290230+0000","last_became_peered":"2023-05-20T01:16:23.290230+0000","last_unstale":"2023-05-20T01:28:27.104385+0000","last_undegraded":"2023-05-20T01:28:27.104385+0000","last_fullsized":"2023-05-20T01:28:27.104385+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:45:05.815733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-05-20T01:28:27.124655+0000","last_change":"2023-05-20T01:28:27.120192+0000","last_active":"2023-05-20T01:28:27.124655+0000","last_peered":"2023-05-20T01:28:27.124655+0000","last_clean":"2023-05-20T01:28:27.124655+0000","last_became_active":"2023-05-20T01:28:27.119859+0000","last_became_peered":"2023-05-20T01:28:27.119859+0000","last_unstale":"2023-05-20T01:28:27.124655+0000","last_undegraded":"2023-05-20T01:28:27.124655+0000","last_fullsized":"2023-05-20T01:28:27.124655+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:11:21.663266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"135'3","reported_seq":14,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-05-20T01:28:27.123533+0000","last_change":"2023-05-20T01:28:26.692131+0000","last_active":"2023-05-20T01:28:27.123533+0000","last_peered":"2023-05-20T01:28:27.123533+0000","last_clean":"2023-05-20T01:28:27.123533+0000","last_became_active":"2023-05-20T01:28:26.691813+0000","last_became_peered":"2023-05-20T01:28:26.691813+0000","last_unstale":"2023-05-20T01:28:27.123533+0000","last_undegraded":"2023-05-20T01:28:27.123533+0000","last_fullsized":"2023-05-20T01:28:27.123533+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_clean_scrub_stamp":"2023-05-20T01:28:25.659712+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:55:03.697685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":226,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:28.291185+0000","last_change":"2023-05-20T01:16:23.290341+0000","last_active":"2023-05-20T01:28:28.291185+0000","last_peered":"2023-05-20T01:28:28.291185+0000","last_clean":"2023-05-20T01:28:28.291185+0000","last_became_active":"2023-05-20T01:16:23.290053+0000","last_became_peered":"2023-05-20T01:16:23.290053+0000","last_unstale":"2023-05-20T01:28:28.291185+0000","last_undegraded":"2023-05-20T01:28:28.291185+0000","last_fullsized":"2023-05-20T01:28:28.291185+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:57:33.990802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-05-20T01:28:27.120856+0000","last_change":"2023-05-20T01:28:27.118519+0000","last_active":"2023-05-20T01:28:27.120856+0000","last_peered":"2023-05-20T01:28:27.120856+0000","last_clean":"2023-05-20T01:28:27.120856+0000","last_became_active":"2023-05-20T01:28:27.118248+0000","last_became_peered":"2023-05-20T01:28:27.118248+0000","last_unstale":"2023-05-20T01:28:27.120856+0000","last_undegraded":"2023-05-20T01:28:27.120856+0000","last_fullsized":"2023-05-20T01:28:27.120856+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:51:46.698868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"135'6","reported_seq":19,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:28.291211+0000","last_change":"2023-05-20T01:28:26.689861+0000","last_active":"2023-05-20T01:28:28.291211+0000","last_peered":"2023-05-20T01:28:28.291211+0000","last_clean":"2023-05-20T01:28:28.291211+0000","last_became_active":"2023-05-20T01:28:26.689504+0000","last_became_peered":"2023-05-20T01:28:26.689504+0000","last_unstale":"2023-05-20T01:28:28.291211+0000","last_undegraded":"2023-05-20T01:28:28.291211+0000","last_fullsized":"2023-05-20T01:28:28.291211+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_clean_scrub_stamp":"2023-05-20T01:28:25.659712+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:04:55.358540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":226,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:28.291316+0000","last_change":"2023-05-20T01:16:22.834330+0000","last_active":"2023-05-20T01:28:28.291316+0000","last_peered":"2023-05-20T01:28:28.291316+0000","last_clean":"2023-05-20T01:28:28.291316+0000","last_became_active":"2023-05-20T01:16:22.833946+0000","last_became_peered":"2023-05-20T01:16:22.833946+0000","last_unstale":"2023-05-20T01:28:28.291316+0000","last_undegraded":"2023-05-20T01:28:28.291316+0000","last_fullsized":"2023-05-20T01:28:28.291316+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:27:45.076913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574998,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":17420,"kb_used_data":2244,"kb_used_omap":0,"kb_used_meta":15168,"kb_avail":94354420,"statfs":{"total":96636764160,"available":96618926080,"internally_reserved":0,"allocated":2297856,"data_stored":1463037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":15532032},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:27:41 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.53500000000000003,"15min":0.51700000000000002},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.76200000000000001,"5min":0.80600000000000005,"15min":0.88200000000000001},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.54600000000000004,"15min":0.54700000000000004},"min":{"1min":0.31,"5min":0.31,"15min":0.30599999999999999},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.90000000000000002},"last":0.58099999999999996}]},{"osd":1,"last update":"Sat May 20 01:27:41 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54400000000000004,"15min":0.67100000000000004},"min":{"1min":0.33900000000000002,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.81899999999999995,"5min":0.95599999999999996,"15min":8.8879999999999999},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.56399999999999995,"15min":0.70399999999999996},"min":{"1min":0.40000000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.80700000000000005,"5min":0.85399999999999998,"15min":8.875},"last":0.505}]},{"osd":2,"last update":"Sat May 20 01:27:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.54300000000000004,"15min":0.66000000000000003},"min":{"1min":0.38900000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":8.9120000000000008},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54300000000000004,"15min":0.66300000000000003},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.75,"5min":0.75,"15min":8.8520000000000003},"last":0.46800000000000003}]},{"osd":3,"last update":"Sat May 20 01:27:41 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.53900000000000003,"15min":0.69699999999999995},"min":{"1min":0.40899999999999997,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.77400000000000002,"5min":0.89900000000000002,"15min":10.884},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.58099999999999996,"15min":0.748},"min":{"1min":0.441,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.80100000000000005,"5min":0.88100000000000001,"15min":10.952},"last":0.55300000000000005}]},{"osd":4,"last update":"Sat May 20 01:27:41 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.51000000000000001,"15min":0.51100000000000001},"min":{"1min":0.38900000000000001,"5min":0.34999999999999998,"15min":0.313},"max":{"1min":0.79800000000000004,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.52500000000000002,"15min":0.53000000000000003},"min":{"1min":0.39400000000000002,"5min":0.309,"15min":0.309},"max":{"1min":0.66100000000000003,"5min":0.82599999999999996,"15min":0.875},"last":0.34999999999999998}]},{"osd":5,"last update":"Sat May 20 01:27:41 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.54800000000000004},"min":{"1min":0.35199999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.82399999999999995,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.57399999999999995,"15min":0.57799999999999996},"min":{"1min":0.42699999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.755,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.436}]},{"osd":6,"last update":"Sat May 20 01:27:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59099999999999997,"15min":0.59699999999999998},"min":{"1min":0.46200000000000002,"5min":0.39600000000000002,"15min":0.378},"max":{"1min":0.83299999999999996,"5min":1.028,"15min":1.028},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56599999999999995,"15min":0.58099999999999996},"min":{"1min":0.46100000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.71599999999999997,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.48699999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574997,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":16648,"kb_used_data":1664,"kb_used_omap":0,"kb_used_meta":14976,"kb_avail":94355192,"statfs":{"total":96636764160,"available":96619716608,"internally_reserved":0,"allocated":1703936,"data_stored":879004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":15335424},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:27:50 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.49199999999999999,"15min":0.48499999999999999},"min":{"1min":0.29399999999999998,"5min":0.28299999999999997,"15min":0.26900000000000002},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.50900000000000001,"15min":0.51800000000000002},"min":{"1min":0.34599999999999997,"5min":0.309,"15min":0.25800000000000001},"max":{"1min":0.78400000000000003,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.46700000000000003}]},{"osd":1,"last update":"Sat May 20 01:27:50 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.52600000000000002,"15min":0.52300000000000002},"min":{"1min":0.36399999999999999,"5min":0.312,"15min":0.28000000000000003},"max":{"1min":0.77400000000000002,"5min":0.79300000000000004,"15min":0.83799999999999997},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.53300000000000003},"min":{"1min":0.35899999999999999,"5min":0.33000000000000002,"15min":0.318},"max":{"1min":0.71499999999999997,"5min":0.77800000000000002,"15min":0.79200000000000004},"last":0.52500000000000002}]},{"osd":2,"last update":"Sat May 20 01:27:50 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.504,"15min":0.505},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.76600000000000001},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.55400000000000005,"15min":0.55200000000000005},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.28699999999999998},"max":{"1min":0.72699999999999998,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.51400000000000001}]},{"osd":3,"last update":"Sat May 20 01:27:50 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.54900000000000004,"15min":0.53800000000000003},"min":{"1min":0.40300000000000002,"5min":0.32400000000000001,"15min":0.311},"max":{"1min":0.753,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.54400000000000004,"15min":0.54300000000000004},"min":{"1min":0.378,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.71699999999999997,"5min":0.78400000000000003,"15min":0.79500000000000004},"last":0.57699999999999996}]},{"osd":4,"last update":"Sat May 20 01:27:50 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.53400000000000003},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.68300000000000005,"5min":0.754,"15min":0.77600000000000002},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.54500000000000004,"15min":0.55600000000000005},"min":{"1min":0.40000000000000002,"5min":0.38100000000000001,"15min":0.32700000000000001},"max":{"1min":0.69299999999999995,"5min":0.77400000000000002,"15min":0.80100000000000005},"last":0.55400000000000005}]},{"osd":5,"last update":"Sat May 20 01:27:50 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.51900000000000002,"15min":0.53600000000000003},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":1.2310000000000001},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56799999999999995,"15min":0.59099999999999997},"min":{"1min":0.41999999999999998,"5min":0.372,"15min":0.34399999999999997},"max":{"1min":0.72399999999999998,"5min":0.83499999999999996,"15min":0.89400000000000002},"last":0.44600000000000001}]},{"osd":7,"last update":"Sat May 20 01:27:50 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56999999999999995,"15min":0.58699999999999997},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.73299999999999998,"5min":0.80200000000000005,"15min":0.92900000000000005},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.58399999999999996,"15min":0.59999999999999998},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.379},"max":{"1min":0.747,"5min":0.78500000000000003,"15min":0.85199999999999998},"last":0.49099999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574997,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":18368,"kb_used_data":2232,"kb_used_omap":0,"kb_used_meta":16128,"kb_avail":94353472,"statfs":{"total":96636764160,"available":96617955328,"internally_reserved":0,"allocated":2285568,"data_stored":1467808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":16515072},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:27:42 2023","interfaces":[{"interface":"back","average":{"1min":0.32800000000000001,"5min":0.311,"15min":0.313},"min":{"1min":0.22800000000000001,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":0.54500000000000004,"5min":0.60799999999999998,"15min":1.194},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.34799999999999998,"15min":0.32700000000000001},"min":{"1min":0.21299999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.59399999999999997,"5min":0.73199999999999998,"15min":0.75800000000000001},"last":0.29199999999999998}]},{"osd":2,"last update":"Sat May 20 01:27:42 2023","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.33600000000000002,"15min":0.32500000000000001},"min":{"1min":0.20200000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.49199999999999999,"5min":0.56799999999999995,"15min":0.66400000000000003},"last":0.27600000000000002},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"min":{"1min":0.30499999999999999,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.59799999999999998,"5min":0.68200000000000005,"15min":1.3580000000000001},"last":0.39700000000000002}]},{"osd":3,"last update":"Sat May 20 01:27:42 2023","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.39300000000000002,"15min":0.377},"min":{"1min":0.24399999999999999,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.54400000000000004,"5min":0.755,"15min":1.619},"last":0.377},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.38200000000000001,"15min":0.38100000000000001},"min":{"1min":0.26700000000000002,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.56499999999999995,"5min":0.63100000000000001,"15min":1.8400000000000001},"last":0.33000000000000002}]},{"osd":4,"last update":"Sat May 20 01:27:42 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.58599999999999997,"15min":0.58299999999999996},"min":{"1min":0.41699999999999998,"5min":0.34899999999999998,"15min":0.32200000000000001},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":1.2769999999999999},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.59599999999999997,"15min":0.58699999999999997},"min":{"1min":0.44900000000000001,"5min":0.38,"15min":0.312},"max":{"1min":0.76600000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.59199999999999997}]},{"osd":5,"last update":"Sat May 20 01:27:42 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.60499999999999998,"15min":0.59599999999999997},"min":{"1min":0.374,"5min":0.37,"15min":0.37},"max":{"1min":0.86299999999999999,"5min":0.96799999999999997,"15min":1.4510000000000001},"last":0.622},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.59499999999999997,"15min":0.60399999999999998},"min":{"1min":0.439,"5min":0.38900000000000001,"15min":0.378},"max":{"1min":0.89700000000000002,"5min":1.1299999999999999,"15min":1.409},"last":0.60399999999999998}]},{"osd":6,"last update":"Sat May 20 01:27:42 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.61299999999999999,"15min":0.60799999999999998},"min":{"1min":0.38800000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.92200000000000004,"5min":1.1499999999999999,"15min":1.575},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.60099999999999998,"15min":0.59799999999999998},"min":{"1min":0.442,"5min":0.41299999999999998,"15min":0.33200000000000002},"max":{"1min":0.88200000000000001,"5min":1.002,"15min":1.0600000000000001},"last":0.51000000000000001}]},{"osd":7,"last update":"Sat May 20 01:27:42 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.60699999999999998,"15min":0.60799999999999998},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.36499999999999999},"max":{"1min":0.84999999999999998,"5min":1.0229999999999999,"15min":1.496},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65600000000000003,"15min":0.64200000000000002},"min":{"1min":0.443,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.79400000000000004,"5min":2.1709999999999998,"15min":2.1709999999999998},"last":0.51900000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574997,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":16684,"kb_used_data":1636,"kb_used_omap":0,"kb_used_meta":15040,"kb_avail":94355156,"statfs":{"total":96636764160,"available":96619679744,"internally_reserved":0,"allocated":1675264,"data_stored":869506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":15400960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat May 20 01:27:38 2023","interfaces":[{"interface":"back","average":{"1min":0.313,"5min":0.33300000000000002,"15min":0.32300000000000001},"min":{"1min":0.20399999999999999,"5min":0.17999999999999999,"15min":0.17999999999999999},"max":{"1min":0.51200000000000001,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.379,"15min":0.35299999999999998},"min":{"1min":0.26100000000000001,"5min":0.21099999999999999,"15min":0.20399999999999999},"max":{"1min":0.59199999999999997,"5min":2.7559999999999998,"15min":2.7559999999999998},"last":0.45500000000000002}]},{"osd":2,"last update":"Sat May 20 01:27:38 2023","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.36399999999999999,"15min":0.34200000000000003},"min":{"1min":0.26000000000000001,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":0.45800000000000002,"5min":3.3079999999999998,"15min":3.3079999999999998},"last":0.24399999999999999},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.46000000000000002,"15min":0.40799999999999997},"min":{"1min":0.27500000000000002,"5min":0.26200000000000001,"15min":0.216},"max":{"1min":0.54700000000000004,"5min":5.7270000000000003,"15min":5.7270000000000003},"last":0.35599999999999998}]},{"osd":3,"last update":"Sat May 20 01:27:38 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.40300000000000002,"15min":0.375},"min":{"1min":0.26600000000000001,"5min":0.22,"15min":0.22},"max":{"1min":0.52400000000000002,"5min":4.258,"15min":4.258},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.40999999999999998,"15min":0.38},"min":{"1min":0.28299999999999997,"5min":0.245,"15min":0.245},"max":{"1min":0.505,"5min":2.714,"15min":2.714},"last":0.29499999999999998}]},{"osd":4,"last update":"Sat May 20 01:27:38 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.73799999999999999,"15min":0.61399999999999999},"min":{"1min":0.41699999999999998,"5min":0.372,"15min":0.34399999999999997},"max":{"1min":0.70699999999999996,"5min":13.125,"15min":13.125},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.61599999999999999,"15min":0.58899999999999997},"min":{"1min":0.44400000000000001,"5min":0.39200000000000002,"15min":0.36299999999999999},"max":{"1min":0.73399999999999999,"5min":1.9930000000000001,"15min":1.9930000000000001},"last":0.54700000000000004}]},{"osd":5,"last update":"Sat May 20 01:27:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.64600000000000002,"15min":0.60099999999999998},"min":{"1min":0.40799999999999997,"5min":0.379,"15min":0.36099999999999999},"max":{"1min":0.80600000000000005,"5min":5.3620000000000001,"15min":5.3620000000000001},"last":0.442},{"interface":"front","average":{"1min":0.62,"5min":0.68000000000000005,"15min":0.626},"min":{"1min":0.48499999999999999,"5min":0.374,"15min":0.374},"max":{"1min":0.752,"5min":5.7999999999999998,"15min":5.7999999999999998},"last":0.56100000000000005}]},{"osd":6,"last update":"Sat May 20 01:27:38 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.74199999999999999,"15min":0.64400000000000002},"min":{"1min":0.43099999999999999,"5min":0.36899999999999999,"15min":0.35199999999999998},"max":{"1min":0.81999999999999995,"5min":10.042999999999999,"15min":10.042999999999999},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.628,"15min":0.623},"min":{"1min":0.48599999999999999,"5min":0.377,"15min":0.377},"max":{"1min":0.78700000000000003,"5min":1.367,"15min":1.367},"last":0.503}]},{"osd":7,"last update":"Sat May 20 01:27:38 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.65900000000000003,"15min":0.61599999999999999},"min":{"1min":0.54000000000000004,"5min":0.433,"15min":0.39600000000000002},"max":{"1min":0.85499999999999998,"5min":2.802,"15min":2.802},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.75800000000000001,"15min":0.66800000000000004},"min":{"1min":0.46400000000000002,"5min":0.439,"15min":0.42099999999999999},"max":{"1min":0.77900000000000003,"5min":10.085000000000001,"15min":10.085000000000001},"last":0.46700000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574997,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":16248,"kb_used_data":1648,"kb_used_omap":0,"kb_used_meta":14592,"kb_avail":94355592,"statfs":{"total":96636764160,"available":96620126208,"internally_reserved":0,"allocated":1687552,"data_stored":871083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":14942208},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.30399999999999999,"5min":0.38100000000000001,"15min":0.34499999999999997},"min":{"1min":0.246,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":0.44600000000000001,"5min":10.262,"15min":10.262},"last":0.27000000000000002},{"interface":"front","average":{"1min":0.33600000000000002,"5min":0.40999999999999998,"15min":0.36599999999999999},"min":{"1min":0.27300000000000002,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.39500000000000002,"5min":12.146000000000001,"15min":12.146000000000001},"last":0.32500000000000001}]},{"osd":1,"last update":"Sat May 20 01:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.38500000000000001,"15min":0.38200000000000001},"min":{"1min":0.29799999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.53200000000000003,"5min":5.4939999999999998,"15min":5.4939999999999998},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.42399999999999999,"15min":0.42099999999999999},"min":{"1min":0.33400000000000002,"5min":0.27400000000000002,"15min":0.249},"max":{"1min":0.51200000000000001,"5min":6.2859999999999996,"15min":6.2859999999999996},"last":0.37}]},{"osd":3,"last update":"Sat May 20 01:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.39700000000000002,"15min":0.38800000000000001},"min":{"1min":0.25900000000000001,"5min":0.218,"15min":0.216},"max":{"1min":0.57799999999999996,"5min":6.351,"15min":6.351},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.45100000000000001,"15min":0.44},"min":{"1min":0.36499999999999999,"5min":0.28000000000000003,"15min":0.27600000000000002},"max":{"1min":0.629,"5min":7.2999999999999998,"15min":7.2999999999999998},"last":0.38400000000000001}]},{"osd":4,"last update":"Sat May 20 01:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.64400000000000002,"15min":0.60099999999999998},"min":{"1min":0.46999999999999997,"5min":0.38800000000000001,"15min":0.34899999999999998},"max":{"1min":0.76900000000000002,"5min":7.2629999999999999,"15min":7.2629999999999999},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.64900000000000002,"15min":0.627},"min":{"1min":0.48899999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.79400000000000004,"5min":6.3159999999999998,"15min":6.3159999999999998},"last":0.57999999999999996}]},{"osd":5,"last update":"Sat May 20 01:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64000000000000001,"15min":0.628},"min":{"1min":0.48799999999999999,"5min":0.36299999999999999,"15min":0.33100000000000002},"max":{"1min":0.83599999999999997,"5min":5.524,"15min":5.524},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.628,"5min":0.65000000000000002,"15min":0.64400000000000002},"min":{"1min":0.50800000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.85399999999999998,"5min":5.5860000000000003,"15min":5.5860000000000003},"last":0.55400000000000005}]},{"osd":6,"last update":"Sat May 20 01:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.65200000000000002,"15min":0.64200000000000002},"min":{"1min":0.48499999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.80200000000000005,"5min":6.1920000000000002,"15min":6.1920000000000002},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.65000000000000002,"15min":0.64000000000000001},"min":{"1min":0.497,"5min":0.39800000000000002,"15min":0.36399999999999999},"max":{"1min":0.84699999999999998,"5min":6.069,"15min":6.069},"last":0.78500000000000003}]},{"osd":7,"last update":"Sat May 20 01:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.65400000000000003,"15min":0.65800000000000003},"min":{"1min":0.50900000000000001,"5min":0.39000000000000001,"15min":0.38900000000000001},"max":{"1min":0.79300000000000004,"5min":6.2370000000000001,"15min":6.2370000000000001},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.65200000000000002},"min":{"1min":0.50900000000000001,"5min":0.42399999999999999,"15min":0.42199999999999999},"max":{"1min":0.80900000000000005,"5min":2.9359999999999999,"15min":2.9359999999999999},"last":0.76700000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574997,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":16580,"kb_used_data":1660,"kb_used_omap":0,"kb_used_meta":14912,"kb_avail":94355260,"statfs":{"total":96636764160,"available":96619786240,"internally_reserved":0,"allocated":1699840,"data_stored":877600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":15269888},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:27:34 2023","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.32700000000000001,"15min":0.32000000000000001},"min":{"1min":0.20200000000000001,"5min":0.20100000000000001,"15min":0.188},"max":{"1min":0.60299999999999998,"5min":0.65000000000000002,"15min":1.528},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.35199999999999998,"15min":0.39900000000000002},"min":{"1min":0.215,"5min":0.20799999999999999,"15min":0.19500000000000001},"max":{"1min":0.623,"5min":0.72399999999999998,"15min":3.1890000000000001},"last":0.30299999999999999}]},{"osd":1,"last update":"Sat May 20 01:27:34 2023","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.34200000000000003,"15min":0.39100000000000001},"min":{"1min":0.24299999999999999,"5min":0.189,"15min":0.189},"max":{"1min":0.56399999999999995,"5min":0.73199999999999998,"15min":3.4260000000000002},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.377,"5min":0.378,"15min":0.36199999999999999},"min":{"1min":0.249,"5min":0.23100000000000001,"15min":0.221},"max":{"1min":0.59299999999999997,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.35199999999999998}]},{"osd":2,"last update":"Sat May 20 01:27:34 2023","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.374,"15min":0.38200000000000001},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.224},"max":{"1min":0.58699999999999997,"5min":0.67400000000000004,"15min":1.1619999999999999},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.41399999999999998,"15min":0.41099999999999998},"min":{"1min":0.28899999999999998,"5min":0.26400000000000001,"15min":0.24299999999999999},"max":{"1min":0.67900000000000005,"5min":0.78500000000000003,"15min":1.413},"last":0.38500000000000001}]},{"osd":4,"last update":"Sat May 20 01:27:34 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.57899999999999996,"15min":0.58199999999999996},"min":{"1min":0.40500000000000003,"5min":0.38700000000000001,"15min":0.38200000000000001},"max":{"1min":0.81299999999999994,"5min":0.95399999999999996,"15min":1.0700000000000001},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.60899999999999999,"15min":0.65500000000000003},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.35899999999999999},"max":{"1min":1.339,"5min":1.339,"15min":3.4660000000000002},"last":0.55900000000000005}]},{"osd":5,"last update":"Sat May 20 01:27:34 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.60699999999999998,"15min":0.65700000000000003},"min":{"1min":0.36499999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.274,"5min":1.274,"15min":3.5379999999999998},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61199999999999999,"15min":0.623},"min":{"1min":0.438,"5min":0.41999999999999998,"15min":0.374},"max":{"1min":0.78600000000000003,"5min":1.026,"15min":1.385},"last":0.47699999999999998}]},{"osd":6,"last update":"Sat May 20 01:27:34 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61699999999999999,"15min":0.626},"min":{"1min":0.41799999999999998,"5min":0.41199999999999998,"15min":0.38100000000000001},"max":{"1min":0.81299999999999994,"5min":0.99199999999999999,"15min":1.347},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.61599999999999999,"15min":0.623},"min":{"1min":0.45300000000000001,"5min":0.39500000000000002,"15min":0.374},"max":{"1min":0.88400000000000001,"5min":0.94999999999999996,"15min":1.107},"last":0.53900000000000003}]},{"osd":7,"last update":"Sat May 20 01:27:34 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.61599999999999999,"15min":0.629},"min":{"1min":0.40699999999999997,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.85099999999999998,"5min":0.92900000000000005,"15min":1.1359999999999999},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.65600000000000003,"15min":0.69999999999999996},"min":{"1min":0.45100000000000001,"5min":0.436,"15min":0.436},"max":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":3.5009999999999999},"last":0.61799999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574997,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":17664,"kb_used_data":1656,"kb_used_omap":0,"kb_used_meta":16000,"kb_avail":94354176,"statfs":{"total":96636764160,"available":96618676224,"internally_reserved":0,"allocated":1695744,"data_stored":877510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":16384000},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:27:37 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.51900000000000002,"15min":0.49399999999999999},"min":{"1min":0.34699999999999998,"5min":0.309,"15min":0.247},"max":{"1min":0.67300000000000004,"5min":0.73699999999999999,"15min":0.75600000000000001},"last":0.503},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.52200000000000002,"15min":0.50800000000000001},"min":{"1min":0.36199999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.746,"5min":0.81799999999999995,"15min":1.7989999999999999},"last":0.52800000000000002}]},{"osd":1,"last update":"Sat May 20 01:27:37 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.52100000000000002,"15min":0.52000000000000002},"min":{"1min":0.37,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.66200000000000003,"5min":0.75800000000000001,"15min":0.82799999999999996},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.53300000000000003,"15min":0.53400000000000003},"min":{"1min":0.38400000000000001,"5min":0.316,"15min":0.316},"max":{"1min":0.77400000000000002,"5min":0.78100000000000003,"15min":2.0739999999999998},"last":0.38400000000000001}]},{"osd":2,"last update":"Sat May 20 01:27:37 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.55300000000000005,"15min":0.52200000000000002},"min":{"1min":0.30499999999999999,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.73699999999999999,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.56499999999999995,"15min":0.55300000000000005},"min":{"1min":0.38400000000000001,"5min":0.35799999999999998,"15min":0.34000000000000002},"max":{"1min":0.80300000000000005,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.44900000000000001}]},{"osd":3,"last update":"Sat May 20 01:27:37 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.55000000000000004,"15min":0.53100000000000003},"min":{"1min":0.36099999999999999,"5min":0.34899999999999998,"15min":0.34399999999999997},"max":{"1min":0.76900000000000002,"5min":0.81399999999999995,"15min":1.827},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.58999999999999997,"15min":0.629},"min":{"1min":0.45900000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.77200000000000002,"5min":1.093,"15min":3.1749999999999998},"last":0.56000000000000005}]},{"osd":5,"last update":"Sat May 20 01:27:37 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51600000000000001,"15min":0.49399999999999999},"min":{"1min":0.32500000000000001,"5min":0.308,"15min":0.29199999999999998},"max":{"1min":0.73499999999999999,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.51700000000000002},"min":{"1min":0.33800000000000002,"5min":0.33400000000000002,"15min":0.29899999999999999},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.40999999999999998}]},{"osd":6,"last update":"Sat May 20 01:27:37 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.57199999999999995,"15min":0.55200000000000005},"min":{"1min":0.376,"5min":0.33000000000000002,"15min":0.32400000000000001},"max":{"1min":0.871,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60499999999999998,"15min":0.59399999999999997},"min":{"1min":0.41199999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.76000000000000001,"5min":0.84699999999999998,"15min":0.85899999999999999},"last":0.57699999999999996}]},{"osd":7,"last update":"Sat May 20 01:27:37 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.56000000000000005},"min":{"1min":0.39800000000000002,"5min":0.36099999999999999,"15min":0.34200000000000003},"max":{"1min":0.79300000000000004,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61499999999999999,"15min":0.59899999999999998},"min":{"1min":0.45100000000000001,"5min":0.38100000000000001,"15min":0.34000000000000002},"max":{"1min":0.82899999999999996,"5min":1.034,"15min":1.034},"last":0.434}]}]},{"osd":5,"up_from":13,"seq":55834574998,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":16244,"kb_used_data":1644,"kb_used_omap":0,"kb_used_meta":14592,"kb_avail":94355596,"statfs":{"total":96636764160,"available":96620130304,"internally_reserved":0,"allocated":1683456,"data_stored":869618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":14942208},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:27:42 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.48899999999999999,"15min":0.59299999999999997},"min":{"1min":0.36199999999999999,"5min":0.29999999999999999,"15min":0.23300000000000001},"max":{"1min":0.70199999999999996,"5min":0.79000000000000004,"15min":6.3929999999999998},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.498,"15min":0.61299999999999999},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.26900000000000002},"max":{"1min":0.76200000000000001,"5min":0.83699999999999997,"15min":6.3339999999999996},"last":0.501}]},{"osd":1,"last update":"Sat May 20 01:27:42 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.49299999999999999,"15min":0.55300000000000005},"min":{"1min":0.33900000000000002,"5min":0.313,"15min":0.29299999999999998},"max":{"1min":0.65300000000000002,"5min":0.72299999999999998,"15min":3.2450000000000001},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.53100000000000003,"15min":0.53400000000000003},"min":{"1min":0.36799999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.69399999999999995,"5min":0.84599999999999997,"15min":1.1819999999999999},"last":0.70599999999999996}]},{"osd":2,"last update":"Sat May 20 01:27:42 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.51700000000000002,"15min":0.53100000000000003},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.30399999999999999},"max":{"1min":0.72399999999999998,"5min":0.84799999999999998,"15min":1.1639999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.52700000000000002,"15min":0.53800000000000003},"min":{"1min":0.39600000000000002,"5min":0.308,"15min":0.29399999999999998},"max":{"1min":0.63,"5min":0.78600000000000003,"15min":1.258},"last":0.52600000000000002}]},{"osd":3,"last update":"Sat May 20 01:27:42 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.55700000000000005,"15min":0.65900000000000003},"min":{"1min":0.44500000000000001,"5min":0.35099999999999998,"15min":0.33000000000000002},"max":{"1min":0.71799999999999997,"5min":0.91000000000000003,"15min":6.351},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.56999999999999995,"15min":0.68500000000000005},"min":{"1min":0.41799999999999998,"5min":0.374,"15min":0.33400000000000002},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":6.3730000000000002},"last":0.65200000000000002}]},{"osd":4,"last update":"Sat May 20 01:27:42 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.51500000000000001,"15min":0.497},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.62},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.55800000000000005,"15min":0.54900000000000004},"min":{"1min":0.36799999999999999,"5min":0.35699999999999998,"15min":0.34200000000000003},"max":{"1min":0.77400000000000002,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.66600000000000004}]},{"osd":6,"last update":"Sat May 20 01:27:42 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.55000000000000004,"15min":0.54600000000000004},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.55100000000000005,"15min":0.55400000000000005},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.77300000000000002,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.63800000000000001}]},{"osd":7,"last update":"Sat May 20 01:27:42 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.58099999999999996,"15min":0.57799999999999996},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33000000000000002},"max":{"1min":0.83399999999999996,"5min":0.83999999999999997,"15min":0.91000000000000003},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.60199999999999998,"15min":0.60299999999999998},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.872,"5min":0.89400000000000002,"15min":0.94499999999999995},"last":0.69399999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-05-20T01:28:31.589 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-05-20T01:28:31.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.741+0000 7faae11aa700 1 Processor -- start 2023-05-20T01:28:31.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.741+0000 7faae11aa700 1 -- start start 2023-05-20T01:28:31.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.742+0000 7faae11aa700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7faadc06f450 0x7faadc0cf890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:31.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.742+0000 7faae11aa700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7faadc06d4b0 0x7faadc161080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:31.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.742+0000 7faae11aa700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faadc06a510 0x7faadc06a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:31.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.742+0000 7faae11aa700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faadc06aed0 con 0x7faadc06d4b0 2023-05-20T01:28:31.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.742+0000 7faae11aa700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faadc0cc510 con 0x7faadc06f450 2023-05-20T01:28:31.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.742+0000 7faae11aa700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faadc1615c0 con 0x7faadc06a510 2023-05-20T01:28:31.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.742+0000 7faada59c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7faadc06d4b0 0x7faadc161080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:31.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.742+0000 7faada59c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7faadc06d4b0 0x7faadc161080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40138/0 (socket says 172.21.15.16:40138) 2023-05-20T01:28:31.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.742+0000 7faada59c700 1 -- 172.21.15.16:0/98114738 learned_addr learned my addr 172.21.15.16:0/98114738 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:31.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.742+0000 7faadad9d700 1 --2- 172.21.15.16:0/98114738 >> v2:172.21.15.132:3300/0 conn(0x7faadc06f450 0x7faadc0cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:31.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.742+0000 7faadb59e700 1 --2- 172.21.15.16:0/98114738 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faadc06a510 0x7faadc06a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:31.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.742+0000 7faada59c700 1 -- 172.21.15.16:0/98114738 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faadc06a510 msgr2=0x7faadc06a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:31.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.742+0000 7faada59c700 1 --2- 172.21.15.16:0/98114738 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faadc06a510 0x7faadc06a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:31.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.742+0000 7faada59c700 1 -- 172.21.15.16:0/98114738 >> v2:172.21.15.132:3300/0 conn(0x7faadc06f450 msgr2=0x7faadc0cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:31.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.742+0000 7faada59c700 1 --2- 172.21.15.16:0/98114738 >> v2:172.21.15.132:3300/0 conn(0x7faadc06f450 0x7faadc0cf890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:31.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.742+0000 7faada59c700 1 -- 172.21.15.16:0/98114738 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faadc071ed0 con 0x7faadc06d4b0 2023-05-20T01:28:31.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.742+0000 7faadad9d700 1 --2- 172.21.15.16:0/98114738 >> v2:172.21.15.132:3300/0 conn(0x7faadc06f450 0x7faadc0cf890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:28:31.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.743+0000 7faada59c700 1 --2- 172.21.15.16:0/98114738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faadc06d4b0 0x7faadc161080 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7faadc0cdb00 tx=0x7faacc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af58fec73a1c3b93 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:31.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.743+0000 7faad9d9b700 1 -- 172.21.15.16:0/98114738 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faacc00b870 con 0x7faadc06d4b0 2023-05-20T01:28:31.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.743+0000 7faad9d9b700 1 -- 172.21.15.16:0/98114738 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faacc00b9d0 con 0x7faadc06d4b0 2023-05-20T01:28:31.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.743+0000 7faad9d9b700 1 -- 172.21.15.16:0/98114738 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faacc00bd60 con 0x7faadc06d4b0 2023-05-20T01:28:31.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.743+0000 7faae11aa700 1 -- 172.21.15.16:0/98114738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faadc06d4b0 msgr2=0x7faadc161080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:31.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.743+0000 7faae11aa700 1 --2- 172.21.15.16:0/98114738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faadc06d4b0 0x7faadc161080 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7faadc0cdb00 tx=0x7faacc009bd0 comp rx=0 tx=0).stop 2023-05-20T01:28:31.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.743+0000 7faae11aa700 1 -- 172.21.15.16:0/98114738 shutdown_connections 2023-05-20T01:28:31.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.743+0000 7faae11aa700 1 --2- 172.21.15.16:0/98114738 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faadc06a510 0x7faadc06a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:31.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.743+0000 7faae11aa700 1 --2- 172.21.15.16:0/98114738 >> v2:172.21.15.132:3300/0 conn(0x7faadc06f450 0x7faadc0cf890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:31.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.743+0000 7faae11aa700 1 --2- 172.21.15.16:0/98114738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faadc06d4b0 0x7faadc161080 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:31.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.743+0000 7faae11aa700 1 -- 172.21.15.16:0/98114738 >> 172.21.15.16:0/98114738 conn(0x7faadc0cffb0 msgr2=0x7faadc0d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:31.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.743+0000 7faae11aa700 1 -- 172.21.15.16:0/98114738 shutdown_connections 2023-05-20T01:28:31.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.743+0000 7faae11aa700 1 -- 172.21.15.16:0/98114738 wait complete. 2023-05-20T01:28:31.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.745+0000 7faae11aa700 1 Processor -- start 2023-05-20T01:28:31.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.745+0000 7faae11aa700 1 -- start start 2023-05-20T01:28:31.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.746+0000 7faae11aa700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faadc06a510 0x7faadc159670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:31.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.746+0000 7faae11aa700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faadc06d4b0 0x7faadc159e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:31.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.746+0000 7faae11aa700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faadc06f450 0x7faadc0c7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:31.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.746+0000 7faae11aa700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faadc0c4900 con 0x7faadc06d4b0 2023-05-20T01:28:31.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.746+0000 7faae11aa700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faadc15aa70 con 0x7faadc06a510 2023-05-20T01:28:31.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.746+0000 7faae11aa700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faadc159bb0 con 0x7faadc06f450 2023-05-20T01:28:31.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.746+0000 7faada59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faadc06d4b0 0x7faadc159e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:31.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.746+0000 7faada59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faadc06d4b0 0x7faadc159e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40144/0 (socket says 172.21.15.16:40144) 2023-05-20T01:28:31.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.746+0000 7faada59c700 1 -- 172.21.15.16:0/1519045386 learned_addr learned my addr 172.21.15.16:0/1519045386 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:31.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.746+0000 7faadb59e700 1 --2- 172.21.15.16:0/1519045386 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faadc06f450 0x7faadc0c7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:31.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.746+0000 7faadad9d700 1 --2- 172.21.15.16:0/1519045386 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faadc06a510 0x7faadc159670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:31.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.746+0000 7faada59c700 1 -- 172.21.15.16:0/1519045386 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faadc06f450 msgr2=0x7faadc0c7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:31.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.746+0000 7faada59c700 1 --2- 172.21.15.16:0/1519045386 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faadc06f450 0x7faadc0c7080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:31.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.746+0000 7faada59c700 1 -- 172.21.15.16:0/1519045386 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faadc06a510 msgr2=0x7faadc159670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:31.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.746+0000 7faada59c700 1 --2- 172.21.15.16:0/1519045386 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faadc06a510 0x7faadc159670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:31.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.746+0000 7faada59c700 1 -- 172.21.15.16:0/1519045386 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faacc009550 con 0x7faadc06d4b0 2023-05-20T01:28:31.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.746+0000 7faada59c700 1 --2- 172.21.15.16:0/1519045386 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faadc06d4b0 0x7faadc159e00 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7faacc000c00 tx=0x7faacc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:31.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.746+0000 7faac37fe700 1 -- 172.21.15.16:0/1519045386 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faacc004540 con 0x7faadc06d4b0 2023-05-20T01:28:31.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.746+0000 7faac37fe700 1 -- 172.21.15.16:0/1519045386 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faacc0036a0 con 0x7faadc06d4b0 2023-05-20T01:28:31.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.747+0000 7faae11aa700 1 -- 172.21.15.16:0/1519045386 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faadc0cc1a0 con 0x7faadc06d4b0 2023-05-20T01:28:31.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.747+0000 7faac37fe700 1 -- 172.21.15.16:0/1519045386 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faacc003a50 con 0x7faadc06d4b0 2023-05-20T01:28:31.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.747+0000 7faae11aa700 1 -- 172.21.15.16:0/1519045386 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faadc0721b0 con 0x7faadc06d4b0 2023-05-20T01:28:31.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.748+0000 7faae11aa700 1 -- 172.21.15.16:0/1519045386 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faadc0cbe50 con 0x7faadc06d4b0 2023-05-20T01:28:31.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.748+0000 7faac37fe700 1 -- 172.21.15.16:0/1519045386 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7faacc021070 con 0x7faadc06d4b0 2023-05-20T01:28:31.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.748+0000 7faac37fe700 1 --2- 172.21.15.16:0/1519045386 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7faac806ace0 0x7faac806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:31.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.749+0000 7faadad9d700 1 --2- 172.21.15.16:0/1519045386 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7faac806ace0 0x7faac806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:31.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.749+0000 7faac37fe700 1 -- 172.21.15.16:0/1519045386 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6382+0+0 (secure 0 0 0) 0x7faacc08ac90 con 0x7faadc06d4b0 2023-05-20T01:28:31.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.749+0000 7faadad9d700 1 --2- 172.21.15.16:0/1519045386 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7faac806ace0 0x7faac806d190 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7faad0006fd0 tx=0x7faad0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:31.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.753+0000 7faac37fe700 1 -- 172.21.15.16:0/1519045386 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7faacc055db0 con 0x7faadc06d4b0 2023-05-20T01:28:31.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.946+0000 7faae11aa700 1 -- 172.21.15.16:0/1519045386 --> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7faadc0c4a40 con 0x7faac806ace0 2023-05-20T01:28:31.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.955+0000 7faac37fe700 1 -- 172.21.15.16:0/1519045386 <== mgr.4104 v2:172.21.15.16:6832/3817237998 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+312007 (secure 0 0 0) 0x7faadc0c4a40 con 0x7faac806ace0 2023-05-20T01:28:31.963 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:28:31.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.959+0000 7faae11aa700 1 -- 172.21.15.16:0/1519045386 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7faac806ace0 msgr2=0x7faac806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:31.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.959+0000 7faae11aa700 1 --2- 172.21.15.16:0/1519045386 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7faac806ace0 0x7faac806d190 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7faad0006fd0 tx=0x7faad0008040 comp rx=0 tx=0).stop 2023-05-20T01:28:31.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.959+0000 7faae11aa700 1 -- 172.21.15.16:0/1519045386 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faadc06d4b0 msgr2=0x7faadc159e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:31.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.959+0000 7faae11aa700 1 --2- 172.21.15.16:0/1519045386 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faadc06d4b0 0x7faadc159e00 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7faacc000c00 tx=0x7faacc004750 comp rx=0 tx=0).stop 2023-05-20T01:28:31.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.959+0000 7faae11aa700 1 -- 172.21.15.16:0/1519045386 shutdown_connections 2023-05-20T01:28:31.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.959+0000 7faae11aa700 1 --2- 172.21.15.16:0/1519045386 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7faac806ace0 0x7faac806d190 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:31.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.959+0000 7faae11aa700 1 --2- 172.21.15.16:0/1519045386 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faadc06f450 0x7faadc0c7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:31.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.959+0000 7faae11aa700 1 --2- 172.21.15.16:0/1519045386 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faadc06a510 0x7faadc159670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:31.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.959+0000 7faae11aa700 1 --2- 172.21.15.16:0/1519045386 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faadc06d4b0 0x7faadc159e00 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:31.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.959+0000 7faae11aa700 1 -- 172.21.15.16:0/1519045386 >> 172.21.15.16:0/1519045386 conn(0x7faadc0cffb0 msgr2=0x7faadc15b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:31.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.959+0000 7faae11aa700 1 -- 172.21.15.16:0/1519045386 shutdown_connections 2023-05-20T01:28:31.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:31.959+0000 7faae11aa700 1 -- 172.21.15.16:0/1519045386 wait complete. 2023-05-20T01:28:31.966 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2023-05-20T01:28:31.975 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":495,"stamp":"2023-05-20T01:28:31.480405+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":121,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":135856,"kb_used_data":14384,"kb_used_omap":0,"kb_used_meta":121408,"kb_avail":754838864,"statfs":{"total":773094113280,"available":772954996736,"internally_reserved":0,"allocated":14729216,"data_stored":8175166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":124321792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.815906"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-05-20T01:28:27.127769+0000","last_change":"2023-05-20T01:28:27.121288+0000","last_active":"2023-05-20T01:28:27.127769+0000","last_peered":"2023-05-20T01:28:27.127769+0000","last_clean":"2023-05-20T01:28:27.127769+0000","last_became_active":"2023-05-20T01:28:27.121001+0000","last_became_peered":"2023-05-20T01:28:27.121001+0000","last_unstale":"2023-05-20T01:28:27.127769+0000","last_undegraded":"2023-05-20T01:28:27.127769+0000","last_fullsized":"2023-05-20T01:28:27.127769+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:23:30.339851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:28.104881+0000","last_change":"2023-05-20T01:28:27.114001+0000","last_active":"2023-05-20T01:28:28.104881+0000","last_peered":"2023-05-20T01:28:28.104881+0000","last_clean":"2023-05-20T01:28:28.104881+0000","last_became_active":"2023-05-20T01:28:27.113740+0000","last_became_peered":"2023-05-20T01:28:27.113740+0000","last_unstale":"2023-05-20T01:28:28.104881+0000","last_undegraded":"2023-05-20T01:28:28.104881+0000","last_fullsized":"2023-05-20T01:28:28.104881+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:31:46.370507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:28.291232+0000","last_change":"2023-05-20T01:28:27.112397+0000","last_active":"2023-05-20T01:28:28.291232+0000","last_peered":"2023-05-20T01:28:28.291232+0000","last_clean":"2023-05-20T01:28:28.291232+0000","last_became_active":"2023-05-20T01:28:27.112116+0000","last_became_peered":"2023-05-20T01:28:27.112116+0000","last_unstale":"2023-05-20T01:28:28.291232+0000","last_undegraded":"2023-05-20T01:28:28.291232+0000","last_fullsized":"2023-05-20T01:28:28.291232+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T01:29:56.213301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-05-20T01:28:27.128556+0000","last_change":"2023-05-20T01:28:27.123975+0000","last_active":"2023-05-20T01:28:27.128556+0000","last_peered":"2023-05-20T01:28:27.128556+0000","last_clean":"2023-05-20T01:28:27.128556+0000","last_became_active":"2023-05-20T01:28:27.122875+0000","last_became_peered":"2023-05-20T01:28:27.122875+0000","last_unstale":"2023-05-20T01:28:27.128556+0000","last_undegraded":"2023-05-20T01:28:27.128556+0000","last_fullsized":"2023-05-20T01:28:27.128556+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:31:54.303518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-05-20T01:28:27.125756+0000","last_change":"2023-05-20T01:28:27.122001+0000","last_active":"2023-05-20T01:28:27.125756+0000","last_peered":"2023-05-20T01:28:27.125756+0000","last_clean":"2023-05-20T01:28:27.125756+0000","last_became_active":"2023-05-20T01:28:27.121585+0000","last_became_peered":"2023-05-20T01:28:27.121585+0000","last_unstale":"2023-05-20T01:28:27.125756+0000","last_undegraded":"2023-05-20T01:28:27.125756+0000","last_fullsized":"2023-05-20T01:28:27.125756+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:41:48.008633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:28.104654+0000","last_change":"2023-05-20T01:28:27.123814+0000","last_active":"2023-05-20T01:28:28.104654+0000","last_peered":"2023-05-20T01:28:28.104654+0000","last_clean":"2023-05-20T01:28:28.104654+0000","last_became_active":"2023-05-20T01:28:27.123520+0000","last_became_peered":"2023-05-20T01:28:27.123520+0000","last_unstale":"2023-05-20T01:28:28.104654+0000","last_undegraded":"2023-05-20T01:28:28.104654+0000","last_fullsized":"2023-05-20T01:28:28.104654+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:20:27.718807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:28.104687+0000","last_change":"2023-05-20T01:28:27.118737+0000","last_active":"2023-05-20T01:28:28.104687+0000","last_peered":"2023-05-20T01:28:28.104687+0000","last_clean":"2023-05-20T01:28:28.104687+0000","last_became_active":"2023-05-20T01:28:27.117977+0000","last_became_peered":"2023-05-20T01:28:27.117977+0000","last_unstale":"2023-05-20T01:28:28.104687+0000","last_undegraded":"2023-05-20T01:28:28.104687+0000","last_fullsized":"2023-05-20T01:28:28.104687+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:33:25.915827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:28.291040+0000","last_change":"2023-05-20T01:28:27.117875+0000","last_active":"2023-05-20T01:28:28.291040+0000","last_peered":"2023-05-20T01:28:28.291040+0000","last_clean":"2023-05-20T01:28:28.291040+0000","last_became_active":"2023-05-20T01:28:27.117606+0000","last_became_peered":"2023-05-20T01:28:27.117606+0000","last_unstale":"2023-05-20T01:28:28.291040+0000","last_undegraded":"2023-05-20T01:28:28.291040+0000","last_fullsized":"2023-05-20T01:28:28.291040+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:51:35.106788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:28.291195+0000","last_change":"2023-05-20T01:28:27.117145+0000","last_active":"2023-05-20T01:28:28.291195+0000","last_peered":"2023-05-20T01:28:28.291195+0000","last_clean":"2023-05-20T01:28:28.291195+0000","last_became_active":"2023-05-20T01:28:27.116894+0000","last_became_peered":"2023-05-20T01:28:27.116894+0000","last_unstale":"2023-05-20T01:28:28.291195+0000","last_undegraded":"2023-05-20T01:28:28.291195+0000","last_fullsized":"2023-05-20T01:28:28.291195+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:10:42.339813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:28.104858+0000","last_change":"2023-05-20T01:28:27.113665+0000","last_active":"2023-05-20T01:28:28.104858+0000","last_peered":"2023-05-20T01:28:28.104858+0000","last_clean":"2023-05-20T01:28:28.104858+0000","last_became_active":"2023-05-20T01:28:27.112664+0000","last_became_peered":"2023-05-20T01:28:27.112664+0000","last_unstale":"2023-05-20T01:28:28.104858+0000","last_undegraded":"2023-05-20T01:28:28.104858+0000","last_fullsized":"2023-05-20T01:28:28.104858+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:58:35.022389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:29.601805+0000","last_change":"2023-05-20T01:28:27.118216+0000","last_active":"2023-05-20T01:28:29.601805+0000","last_peered":"2023-05-20T01:28:29.601805+0000","last_clean":"2023-05-20T01:28:29.601805+0000","last_became_active":"2023-05-20T01:28:27.117601+0000","last_became_peered":"2023-05-20T01:28:27.117601+0000","last_unstale":"2023-05-20T01:28:29.601805+0000","last_undegraded":"2023-05-20T01:28:29.601805+0000","last_fullsized":"2023-05-20T01:28:29.601805+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:12:34.227931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:28.291211+0000","last_change":"2023-05-20T01:28:27.116573+0000","last_active":"2023-05-20T01:28:28.291211+0000","last_peered":"2023-05-20T01:28:28.291211+0000","last_clean":"2023-05-20T01:28:28.291211+0000","last_became_active":"2023-05-20T01:28:27.116264+0000","last_became_peered":"2023-05-20T01:28:27.116264+0000","last_unstale":"2023-05-20T01:28:28.291211+0000","last_undegraded":"2023-05-20T01:28:28.291211+0000","last_fullsized":"2023-05-20T01:28:28.291211+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:03:13.890512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-05-20T01:28:27.126125+0000","last_change":"2023-05-20T01:28:27.117009+0000","last_active":"2023-05-20T01:28:27.126125+0000","last_peered":"2023-05-20T01:28:27.126125+0000","last_clean":"2023-05-20T01:28:27.126125+0000","last_became_active":"2023-05-20T01:28:27.116774+0000","last_became_peered":"2023-05-20T01:28:27.116774+0000","last_unstale":"2023-05-20T01:28:27.126125+0000","last_undegraded":"2023-05-20T01:28:27.126125+0000","last_fullsized":"2023-05-20T01:28:27.126125+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:30:11.299773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:28.104607+0000","last_change":"2023-05-20T01:28:27.124550+0000","last_active":"2023-05-20T01:28:28.104607+0000","last_peered":"2023-05-20T01:28:28.104607+0000","last_clean":"2023-05-20T01:28:28.104607+0000","last_became_active":"2023-05-20T01:28:27.124270+0000","last_became_peered":"2023-05-20T01:28:27.124270+0000","last_unstale":"2023-05-20T01:28:28.104607+0000","last_undegraded":"2023-05-20T01:28:28.104607+0000","last_fullsized":"2023-05-20T01:28:28.104607+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:36:22.392704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-05-20T01:28:27.128618+0000","last_change":"2023-05-20T01:28:27.124483+0000","last_active":"2023-05-20T01:28:27.128618+0000","last_peered":"2023-05-20T01:28:27.128618+0000","last_clean":"2023-05-20T01:28:27.128618+0000","last_became_active":"2023-05-20T01:28:27.124084+0000","last_became_peered":"2023-05-20T01:28:27.124084+0000","last_unstale":"2023-05-20T01:28:27.128618+0000","last_undegraded":"2023-05-20T01:28:27.128618+0000","last_fullsized":"2023-05-20T01:28:27.128618+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T13:11:30.428632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:28.102153+0000","last_change":"2023-05-20T01:28:27.117982+0000","last_active":"2023-05-20T01:28:28.102153+0000","last_peered":"2023-05-20T01:28:28.102153+0000","last_clean":"2023-05-20T01:28:28.102153+0000","last_became_active":"2023-05-20T01:28:27.117726+0000","last_became_peered":"2023-05-20T01:28:27.117726+0000","last_unstale":"2023-05-20T01:28:28.102153+0000","last_undegraded":"2023-05-20T01:28:28.102153+0000","last_fullsized":"2023-05-20T01:28:28.102153+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:22:54.730939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:28.105155+0000","last_change":"2023-05-20T01:28:27.109900+0000","last_active":"2023-05-20T01:28:28.105155+0000","last_peered":"2023-05-20T01:28:28.105155+0000","last_clean":"2023-05-20T01:28:28.105155+0000","last_became_active":"2023-05-20T01:28:27.108667+0000","last_became_peered":"2023-05-20T01:28:27.108667+0000","last_unstale":"2023-05-20T01:28:28.105155+0000","last_undegraded":"2023-05-20T01:28:28.105155+0000","last_fullsized":"2023-05-20T01:28:28.105155+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:31:38.867656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:28.102853+0000","last_change":"2023-05-20T01:28:27.109262+0000","last_active":"2023-05-20T01:28:28.102853+0000","last_peered":"2023-05-20T01:28:28.102853+0000","last_clean":"2023-05-20T01:28:28.102853+0000","last_became_active":"2023-05-20T01:28:27.108995+0000","last_became_peered":"2023-05-20T01:28:27.108995+0000","last_unstale":"2023-05-20T01:28:28.102853+0000","last_undegraded":"2023-05-20T01:28:28.102853+0000","last_fullsized":"2023-05-20T01:28:28.102853+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:00:05.587896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:29.601842+0000","last_change":"2023-05-20T01:28:27.117867+0000","last_active":"2023-05-20T01:28:29.601842+0000","last_peered":"2023-05-20T01:28:29.601842+0000","last_clean":"2023-05-20T01:28:29.601842+0000","last_became_active":"2023-05-20T01:28:27.116175+0000","last_became_peered":"2023-05-20T01:28:27.116175+0000","last_unstale":"2023-05-20T01:28:29.601842+0000","last_undegraded":"2023-05-20T01:28:29.601842+0000","last_fullsized":"2023-05-20T01:28:29.601842+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:38:01.636748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:28.102531+0000","last_change":"2023-05-20T01:28:27.111932+0000","last_active":"2023-05-20T01:28:28.102531+0000","last_peered":"2023-05-20T01:28:28.102531+0000","last_clean":"2023-05-20T01:28:28.102531+0000","last_became_active":"2023-05-20T01:28:27.111699+0000","last_became_peered":"2023-05-20T01:28:27.111699+0000","last_unstale":"2023-05-20T01:28:28.102531+0000","last_undegraded":"2023-05-20T01:28:28.102531+0000","last_fullsized":"2023-05-20T01:28:28.102531+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:00:02.108926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-05-20T01:28:27.125727+0000","last_change":"2023-05-20T01:28:27.121833+0000","last_active":"2023-05-20T01:28:27.125727+0000","last_peered":"2023-05-20T01:28:27.125727+0000","last_clean":"2023-05-20T01:28:27.125727+0000","last_became_active":"2023-05-20T01:28:27.121226+0000","last_became_peered":"2023-05-20T01:28:27.121226+0000","last_unstale":"2023-05-20T01:28:27.125727+0000","last_undegraded":"2023-05-20T01:28:27.125727+0000","last_fullsized":"2023-05-20T01:28:27.125727+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:21:32.484022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:29.601551+0000","last_change":"2023-05-20T01:28:27.114932+0000","last_active":"2023-05-20T01:28:29.601551+0000","last_peered":"2023-05-20T01:28:29.601551+0000","last_clean":"2023-05-20T01:28:29.601551+0000","last_became_active":"2023-05-20T01:28:27.114682+0000","last_became_peered":"2023-05-20T01:28:27.114682+0000","last_unstale":"2023-05-20T01:28:29.601551+0000","last_undegraded":"2023-05-20T01:28:29.601551+0000","last_fullsized":"2023-05-20T01:28:29.601551+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:23:28.303320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:28.291109+0000","last_change":"2023-05-20T01:28:27.121610+0000","last_active":"2023-05-20T01:28:28.291109+0000","last_peered":"2023-05-20T01:28:28.291109+0000","last_clean":"2023-05-20T01:28:28.291109+0000","last_became_active":"2023-05-20T01:28:27.121294+0000","last_became_peered":"2023-05-20T01:28:27.121294+0000","last_unstale":"2023-05-20T01:28:28.291109+0000","last_undegraded":"2023-05-20T01:28:28.291109+0000","last_fullsized":"2023-05-20T01:28:28.291109+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:54:47.187098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:29.601506+0000","last_change":"2023-05-20T01:28:27.120994+0000","last_active":"2023-05-20T01:28:29.601506+0000","last_peered":"2023-05-20T01:28:29.601506+0000","last_clean":"2023-05-20T01:28:29.601506+0000","last_became_active":"2023-05-20T01:28:27.120782+0000","last_became_peered":"2023-05-20T01:28:27.120782+0000","last_unstale":"2023-05-20T01:28:29.601506+0000","last_undegraded":"2023-05-20T01:28:29.601506+0000","last_fullsized":"2023-05-20T01:28:29.601506+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:41:56.296593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:28.104791+0000","last_change":"2023-05-20T01:28:27.114072+0000","last_active":"2023-05-20T01:28:28.104791+0000","last_peered":"2023-05-20T01:28:28.104791+0000","last_clean":"2023-05-20T01:28:28.104791+0000","last_became_active":"2023-05-20T01:28:27.113786+0000","last_became_peered":"2023-05-20T01:28:27.113786+0000","last_unstale":"2023-05-20T01:28:28.104791+0000","last_undegraded":"2023-05-20T01:28:28.104791+0000","last_fullsized":"2023-05-20T01:28:28.104791+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:44:13.745200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:28.102859+0000","last_change":"2023-05-20T01:28:27.123864+0000","last_active":"2023-05-20T01:28:28.102859+0000","last_peered":"2023-05-20T01:28:28.102859+0000","last_clean":"2023-05-20T01:28:28.102859+0000","last_became_active":"2023-05-20T01:28:27.123444+0000","last_became_peered":"2023-05-20T01:28:27.123444+0000","last_unstale":"2023-05-20T01:28:28.102859+0000","last_undegraded":"2023-05-20T01:28:28.102859+0000","last_fullsized":"2023-05-20T01:28:28.102859+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:18:45.524372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-05-20T01:28:27.123919+0000","last_change":"2023-05-20T01:28:27.117032+0000","last_active":"2023-05-20T01:28:27.123919+0000","last_peered":"2023-05-20T01:28:27.123919+0000","last_clean":"2023-05-20T01:28:27.123919+0000","last_became_active":"2023-05-20T01:28:27.116801+0000","last_became_peered":"2023-05-20T01:28:27.116801+0000","last_unstale":"2023-05-20T01:28:27.123919+0000","last_undegraded":"2023-05-20T01:28:27.123919+0000","last_fullsized":"2023-05-20T01:28:27.123919+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:07:48.100900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-05-20T01:28:27.123922+0000","last_change":"2023-05-20T01:28:27.115783+0000","last_active":"2023-05-20T01:28:27.123922+0000","last_peered":"2023-05-20T01:28:27.123922+0000","last_clean":"2023-05-20T01:28:27.123922+0000","last_became_active":"2023-05-20T01:28:27.115175+0000","last_became_peered":"2023-05-20T01:28:27.115175+0000","last_unstale":"2023-05-20T01:28:27.123922+0000","last_undegraded":"2023-05-20T01:28:27.123922+0000","last_fullsized":"2023-05-20T01:28:27.123922+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:49:00.841918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-05-20T01:28:27.128758+0000","last_change":"2023-05-20T01:28:27.125691+0000","last_active":"2023-05-20T01:28:27.128758+0000","last_peered":"2023-05-20T01:28:27.128758+0000","last_clean":"2023-05-20T01:28:27.128758+0000","last_became_active":"2023-05-20T01:28:27.124527+0000","last_became_peered":"2023-05-20T01:28:27.124527+0000","last_unstale":"2023-05-20T01:28:27.128758+0000","last_undegraded":"2023-05-20T01:28:27.128758+0000","last_fullsized":"2023-05-20T01:28:27.128758+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:50:46.978921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-05-20T01:28:27.128565+0000","last_change":"2023-05-20T01:28:27.124126+0000","last_active":"2023-05-20T01:28:27.128565+0000","last_peered":"2023-05-20T01:28:27.128565+0000","last_clean":"2023-05-20T01:28:27.128565+0000","last_became_active":"2023-05-20T01:28:27.123394+0000","last_became_peered":"2023-05-20T01:28:27.123394+0000","last_unstale":"2023-05-20T01:28:27.128565+0000","last_undegraded":"2023-05-20T01:28:27.128565+0000","last_fullsized":"2023-05-20T01:28:27.128565+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:22:07.190005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-05-20T01:28:27.125857+0000","last_change":"2023-05-20T01:28:27.122974+0000","last_active":"2023-05-20T01:28:27.125857+0000","last_peered":"2023-05-20T01:28:27.125857+0000","last_clean":"2023-05-20T01:28:27.125857+0000","last_became_active":"2023-05-20T01:28:27.122699+0000","last_became_peered":"2023-05-20T01:28:27.122699+0000","last_unstale":"2023-05-20T01:28:27.125857+0000","last_undegraded":"2023-05-20T01:28:27.125857+0000","last_fullsized":"2023-05-20T01:28:27.125857+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:34:30.960285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:28.104659+0000","last_change":"2023-05-20T01:28:27.102528+0000","last_active":"2023-05-20T01:28:28.104659+0000","last_peered":"2023-05-20T01:28:28.104659+0000","last_clean":"2023-05-20T01:28:28.104659+0000","last_became_active":"2023-05-20T01:28:27.102222+0000","last_became_peered":"2023-05-20T01:28:27.102222+0000","last_unstale":"2023-05-20T01:28:28.104659+0000","last_undegraded":"2023-05-20T01:28:28.104659+0000","last_fullsized":"2023-05-20T01:28:28.104659+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:18:45.524372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:28.102725+0000","last_change":"2023-05-20T01:28:27.121554+0000","last_active":"2023-05-20T01:28:28.102725+0000","last_peered":"2023-05-20T01:28:28.102725+0000","last_clean":"2023-05-20T01:28:28.102725+0000","last_became_active":"2023-05-20T01:28:27.121253+0000","last_became_peered":"2023-05-20T01:28:27.121253+0000","last_unstale":"2023-05-20T01:28:28.102725+0000","last_undegraded":"2023-05-20T01:28:28.102725+0000","last_fullsized":"2023-05-20T01:28:28.102725+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:41:15.368853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"135'4","reported_seq":15,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-05-20T01:28:27.124968+0000","last_change":"2023-05-20T01:28:26.694591+0000","last_active":"2023-05-20T01:28:27.124968+0000","last_peered":"2023-05-20T01:28:27.124968+0000","last_clean":"2023-05-20T01:28:27.124968+0000","last_became_active":"2023-05-20T01:28:26.694306+0000","last_became_peered":"2023-05-20T01:28:26.694306+0000","last_unstale":"2023-05-20T01:28:27.124968+0000","last_undegraded":"2023-05-20T01:28:27.124968+0000","last_fullsized":"2023-05-20T01:28:27.124968+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_clean_scrub_stamp":"2023-05-20T01:28:25.659712+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:23:38.881118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:28.291368+0000","last_change":"2023-05-20T01:28:27.124985+0000","last_active":"2023-05-20T01:28:28.291368+0000","last_peered":"2023-05-20T01:28:28.291368+0000","last_clean":"2023-05-20T01:28:28.291368+0000","last_became_active":"2023-05-20T01:28:27.124719+0000","last_became_peered":"2023-05-20T01:28:27.124719+0000","last_unstale":"2023-05-20T01:28:28.291368+0000","last_undegraded":"2023-05-20T01:28:28.291368+0000","last_fullsized":"2023-05-20T01:28:28.291368+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:59:30.774683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"135'1","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-05-20T01:28:27.120222+0000","last_change":"2023-05-20T01:28:26.696838+0000","last_active":"2023-05-20T01:28:27.120222+0000","last_peered":"2023-05-20T01:28:27.120222+0000","last_clean":"2023-05-20T01:28:27.120222+0000","last_became_active":"2023-05-20T01:28:26.696442+0000","last_became_peered":"2023-05-20T01:28:26.696442+0000","last_unstale":"2023-05-20T01:28:27.120222+0000","last_undegraded":"2023-05-20T01:28:27.120222+0000","last_fullsized":"2023-05-20T01:28:27.120222+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_clean_scrub_stamp":"2023-05-20T01:28:25.659712+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:18:12.147035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-05-20T01:28:27.120887+0000","last_change":"2023-05-20T01:28:27.113350+0000","last_active":"2023-05-20T01:28:27.120887+0000","last_peered":"2023-05-20T01:28:27.120887+0000","last_clean":"2023-05-20T01:28:27.120887+0000","last_became_active":"2023-05-20T01:28:27.113076+0000","last_became_peered":"2023-05-20T01:28:27.113076+0000","last_unstale":"2023-05-20T01:28:27.120887+0000","last_undegraded":"2023-05-20T01:28:27.120887+0000","last_fullsized":"2023-05-20T01:28:27.120887+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:28:01.116965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1d","version":"135'2","reported_seq":15,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:29.601890+0000","last_change":"2023-05-20T01:28:26.694478+0000","last_active":"2023-05-20T01:28:29.601890+0000","last_peered":"2023-05-20T01:28:29.601890+0000","last_clean":"2023-05-20T01:28:29.601890+0000","last_became_active":"2023-05-20T01:28:26.694122+0000","last_became_peered":"2023-05-20T01:28:26.694122+0000","last_unstale":"2023-05-20T01:28:29.601890+0000","last_undegraded":"2023-05-20T01:28:29.601890+0000","last_fullsized":"2023-05-20T01:28:29.601890+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_clean_scrub_stamp":"2023-05-20T01:28:25.659712+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:27:55.665824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-05-20T01:28:27.120724+0000","last_change":"2023-05-20T01:28:27.113613+0000","last_active":"2023-05-20T01:28:27.120724+0000","last_peered":"2023-05-20T01:28:27.120724+0000","last_clean":"2023-05-20T01:28:27.120724+0000","last_became_active":"2023-05-20T01:28:27.113336+0000","last_became_peered":"2023-05-20T01:28:27.113336+0000","last_unstale":"2023-05-20T01:28:27.120724+0000","last_undegraded":"2023-05-20T01:28:27.120724+0000","last_fullsized":"2023-05-20T01:28:27.120724+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:55:27.219103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"135'2","reported_seq":15,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:28.104768+0000","last_change":"2023-05-20T01:28:26.693985+0000","last_active":"2023-05-20T01:28:28.104768+0000","last_peered":"2023-05-20T01:28:28.104768+0000","last_clean":"2023-05-20T01:28:28.104768+0000","last_became_active":"2023-05-20T01:28:26.693718+0000","last_became_peered":"2023-05-20T01:28:26.693718+0000","last_unstale":"2023-05-20T01:28:28.104768+0000","last_undegraded":"2023-05-20T01:28:28.104768+0000","last_fullsized":"2023-05-20T01:28:28.104768+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_clean_scrub_stamp":"2023-05-20T01:28:25.659712+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:58:19.162137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:28.102303+0000","last_change":"2023-05-20T01:28:27.104775+0000","last_active":"2023-05-20T01:28:28.102303+0000","last_peered":"2023-05-20T01:28:28.102303+0000","last_clean":"2023-05-20T01:28:28.102303+0000","last_became_active":"2023-05-20T01:28:27.104519+0000","last_became_peered":"2023-05-20T01:28:27.104519+0000","last_unstale":"2023-05-20T01:28:28.102303+0000","last_undegraded":"2023-05-20T01:28:28.102303+0000","last_fullsized":"2023-05-20T01:28:28.102303+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:39:32.802521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"135'1","reported_seq":14,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:28.104547+0000","last_change":"2023-05-20T01:28:26.692296+0000","last_active":"2023-05-20T01:28:28.104547+0000","last_peered":"2023-05-20T01:28:28.104547+0000","last_clean":"2023-05-20T01:28:28.104547+0000","last_became_active":"2023-05-20T01:28:26.691906+0000","last_became_peered":"2023-05-20T01:28:26.691906+0000","last_unstale":"2023-05-20T01:28:28.104547+0000","last_undegraded":"2023-05-20T01:28:28.104547+0000","last_fullsized":"2023-05-20T01:28:28.104547+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_clean_scrub_stamp":"2023-05-20T01:28:25.659712+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:03:59.465771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:28.105123+0000","last_change":"2023-05-20T01:28:27.107663+0000","last_active":"2023-05-20T01:28:28.105123+0000","last_peered":"2023-05-20T01:28:28.105123+0000","last_clean":"2023-05-20T01:28:28.105123+0000","last_became_active":"2023-05-20T01:28:27.107357+0000","last_became_peered":"2023-05-20T01:28:27.107357+0000","last_unstale":"2023-05-20T01:28:28.105123+0000","last_undegraded":"2023-05-20T01:28:28.105123+0000","last_fullsized":"2023-05-20T01:28:28.105123+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:51:28.678736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"135'1","reported_seq":14,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:28.102442+0000","last_change":"2023-05-20T01:28:26.691992+0000","last_active":"2023-05-20T01:28:28.102442+0000","last_peered":"2023-05-20T01:28:28.102442+0000","last_clean":"2023-05-20T01:28:28.102442+0000","last_became_active":"2023-05-20T01:28:26.690664+0000","last_became_peered":"2023-05-20T01:28:26.690664+0000","last_unstale":"2023-05-20T01:28:28.102442+0000","last_undegraded":"2023-05-20T01:28:28.102442+0000","last_fullsized":"2023-05-20T01:28:28.102442+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_clean_scrub_stamp":"2023-05-20T01:28:25.659712+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:05:40.675125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:28.290789+0000","last_change":"2023-05-20T01:28:27.104819+0000","last_active":"2023-05-20T01:28:28.290789+0000","last_peered":"2023-05-20T01:28:28.290789+0000","last_clean":"2023-05-20T01:28:28.290789+0000","last_became_active":"2023-05-20T01:28:27.104547+0000","last_became_peered":"2023-05-20T01:28:27.104547+0000","last_unstale":"2023-05-20T01:28:28.290789+0000","last_undegraded":"2023-05-20T01:28:28.290789+0000","last_fullsized":"2023-05-20T01:28:28.290789+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:10:08.036660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":13,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:28.104630+0000","last_change":"2023-05-20T01:28:26.690350+0000","last_active":"2023-05-20T01:28:28.104630+0000","last_peered":"2023-05-20T01:28:28.104630+0000","last_clean":"2023-05-20T01:28:28.104630+0000","last_became_active":"2023-05-20T01:28:26.690011+0000","last_became_peered":"2023-05-20T01:28:26.690011+0000","last_unstale":"2023-05-20T01:28:28.104630+0000","last_undegraded":"2023-05-20T01:28:28.104630+0000","last_fullsized":"2023-05-20T01:28:28.104630+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_clean_scrub_stamp":"2023-05-20T01:28:25.659712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:41:14.364064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:28.104438+0000","last_change":"2023-05-20T01:28:27.110577+0000","last_active":"2023-05-20T01:28:28.104438+0000","last_peered":"2023-05-20T01:28:28.104438+0000","last_clean":"2023-05-20T01:28:28.104438+0000","last_became_active":"2023-05-20T01:28:27.110309+0000","last_became_peered":"2023-05-20T01:28:27.110309+0000","last_unstale":"2023-05-20T01:28:28.104438+0000","last_undegraded":"2023-05-20T01:28:28.104438+0000","last_fullsized":"2023-05-20T01:28:28.104438+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:34:31.788215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":13,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:28.104511+0000","last_change":"2023-05-20T01:28:26.693970+0000","last_active":"2023-05-20T01:28:28.104511+0000","last_peered":"2023-05-20T01:28:28.104511+0000","last_clean":"2023-05-20T01:28:28.104511+0000","last_became_active":"2023-05-20T01:28:26.693345+0000","last_became_peered":"2023-05-20T01:28:26.693345+0000","last_unstale":"2023-05-20T01:28:28.104511+0000","last_undegraded":"2023-05-20T01:28:28.104511+0000","last_fullsized":"2023-05-20T01:28:28.104511+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_clean_scrub_stamp":"2023-05-20T01:28:25.659712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:11:44.990260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:28.291179+0000","last_change":"2023-05-20T01:28:27.123808+0000","last_active":"2023-05-20T01:28:28.291179+0000","last_peered":"2023-05-20T01:28:28.291179+0000","last_clean":"2023-05-20T01:28:28.291179+0000","last_became_active":"2023-05-20T01:28:27.123511+0000","last_became_peered":"2023-05-20T01:28:27.123511+0000","last_unstale":"2023-05-20T01:28:28.291179+0000","last_undegraded":"2023-05-20T01:28:28.291179+0000","last_fullsized":"2023-05-20T01:28:28.291179+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:50:56.574275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.f","version":"135'1","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-05-20T01:28:27.122329+0000","last_change":"2023-05-20T01:28:26.692018+0000","last_active":"2023-05-20T01:28:27.122329+0000","last_peered":"2023-05-20T01:28:27.122329+0000","last_clean":"2023-05-20T01:28:27.122329+0000","last_became_active":"2023-05-20T01:28:26.691764+0000","last_became_peered":"2023-05-20T01:28:26.691764+0000","last_unstale":"2023-05-20T01:28:27.122329+0000","last_undegraded":"2023-05-20T01:28:27.122329+0000","last_fullsized":"2023-05-20T01:28:27.122329+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_clean_scrub_stamp":"2023-05-20T01:28:25.659712+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:04:39.314176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-05-20T01:28:27.124077+0000","last_change":"2023-05-20T01:28:27.116069+0000","last_active":"2023-05-20T01:28:27.124077+0000","last_peered":"2023-05-20T01:28:27.124077+0000","last_clean":"2023-05-20T01:28:27.124077+0000","last_became_active":"2023-05-20T01:28:27.115799+0000","last_became_peered":"2023-05-20T01:28:27.115799+0000","last_unstale":"2023-05-20T01:28:27.124077+0000","last_undegraded":"2023-05-20T01:28:27.124077+0000","last_fullsized":"2023-05-20T01:28:27.124077+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:50:24.438722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.e","version":"135'2","reported_seq":15,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:29.601849+0000","last_change":"2023-05-20T01:28:26.695191+0000","last_active":"2023-05-20T01:28:29.601849+0000","last_peered":"2023-05-20T01:28:29.601849+0000","last_clean":"2023-05-20T01:28:29.601849+0000","last_became_active":"2023-05-20T01:28:26.694855+0000","last_became_peered":"2023-05-20T01:28:26.694855+0000","last_unstale":"2023-05-20T01:28:29.601849+0000","last_undegraded":"2023-05-20T01:28:29.601849+0000","last_fullsized":"2023-05-20T01:28:29.601849+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_clean_scrub_stamp":"2023-05-20T01:28:25.659712+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:41:37.683888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:28.102533+0000","last_change":"2023-05-20T01:28:27.123776+0000","last_active":"2023-05-20T01:28:28.102533+0000","last_peered":"2023-05-20T01:28:28.102533+0000","last_clean":"2023-05-20T01:28:28.102533+0000","last_became_active":"2023-05-20T01:28:27.123484+0000","last_became_peered":"2023-05-20T01:28:27.123484+0000","last_unstale":"2023-05-20T01:28:28.102533+0000","last_undegraded":"2023-05-20T01:28:28.102533+0000","last_fullsized":"2023-05-20T01:28:28.102533+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:21:01.822967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.d","version":"135'1","reported_seq":14,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:28.105223+0000","last_change":"2023-05-20T01:28:26.683534+0000","last_active":"2023-05-20T01:28:28.105223+0000","last_peered":"2023-05-20T01:28:28.105223+0000","last_clean":"2023-05-20T01:28:28.105223+0000","last_became_active":"2023-05-20T01:28:26.682870+0000","last_became_peered":"2023-05-20T01:28:26.682870+0000","last_unstale":"2023-05-20T01:28:28.105223+0000","last_undegraded":"2023-05-20T01:28:28.105223+0000","last_fullsized":"2023-05-20T01:28:28.105223+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_clean_scrub_stamp":"2023-05-20T01:28:25.659712+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:11:10.553608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:28.104863+0000","last_change":"2023-05-20T01:28:27.112662+0000","last_active":"2023-05-20T01:28:28.104863+0000","last_peered":"2023-05-20T01:28:28.104863+0000","last_clean":"2023-05-20T01:28:28.104863+0000","last_became_active":"2023-05-20T01:28:27.112398+0000","last_became_peered":"2023-05-20T01:28:27.112398+0000","last_unstale":"2023-05-20T01:28:28.104863+0000","last_undegraded":"2023-05-20T01:28:28.104863+0000","last_fullsized":"2023-05-20T01:28:28.104863+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:32:38.424065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-05-20T01:28:27.104438+0000","last_change":"2023-05-20T01:28:26.697014+0000","last_active":"2023-05-20T01:28:27.104438+0000","last_peered":"2023-05-20T01:28:27.104438+0000","last_clean":"2023-05-20T01:28:27.104438+0000","last_became_active":"2023-05-20T01:28:26.696580+0000","last_became_peered":"2023-05-20T01:28:26.696580+0000","last_unstale":"2023-05-20T01:28:27.104438+0000","last_undegraded":"2023-05-20T01:28:27.104438+0000","last_fullsized":"2023-05-20T01:28:27.104438+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_clean_scrub_stamp":"2023-05-20T01:28:25.659712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:34:15.371890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-05-20T01:28:27.123671+0000","last_change":"2023-05-20T01:28:27.117167+0000","last_active":"2023-05-20T01:28:27.123671+0000","last_peered":"2023-05-20T01:28:27.123671+0000","last_clean":"2023-05-20T01:28:27.123671+0000","last_became_active":"2023-05-20T01:28:27.116889+0000","last_became_peered":"2023-05-20T01:28:27.116889+0000","last_unstale":"2023-05-20T01:28:27.123671+0000","last_undegraded":"2023-05-20T01:28:27.123671+0000","last_fullsized":"2023-05-20T01:28:27.123671+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:31:04.819577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.b","version":"135'2","reported_seq":13,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-05-20T01:28:27.126078+0000","last_change":"2023-05-20T01:28:26.687062+0000","last_active":"2023-05-20T01:28:27.126078+0000","last_peered":"2023-05-20T01:28:27.126078+0000","last_clean":"2023-05-20T01:28:27.126078+0000","last_became_active":"2023-05-20T01:28:26.686794+0000","last_became_peered":"2023-05-20T01:28:26.686794+0000","last_unstale":"2023-05-20T01:28:27.126078+0000","last_undegraded":"2023-05-20T01:28:27.126078+0000","last_fullsized":"2023-05-20T01:28:27.126078+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_clean_scrub_stamp":"2023-05-20T01:28:25.659712+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:06:53.372054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:28.105013+0000","last_change":"2023-05-20T01:28:27.103203+0000","last_active":"2023-05-20T01:28:28.105013+0000","last_peered":"2023-05-20T01:28:28.105013+0000","last_clean":"2023-05-20T01:28:28.105013+0000","last_became_active":"2023-05-20T01:28:27.102938+0000","last_became_peered":"2023-05-20T01:28:27.102938+0000","last_unstale":"2023-05-20T01:28:28.105013+0000","last_undegraded":"2023-05-20T01:28:28.105013+0000","last_fullsized":"2023-05-20T01:28:28.105013+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:12:37.307273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":13,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:28.102630+0000","last_change":"2023-05-20T01:28:26.692534+0000","last_active":"2023-05-20T01:28:28.102630+0000","last_peered":"2023-05-20T01:28:28.102630+0000","last_clean":"2023-05-20T01:28:28.102630+0000","last_became_active":"2023-05-20T01:28:26.692259+0000","last_became_peered":"2023-05-20T01:28:26.692259+0000","last_unstale":"2023-05-20T01:28:28.102630+0000","last_undegraded":"2023-05-20T01:28:28.102630+0000","last_fullsized":"2023-05-20T01:28:28.102630+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_clean_scrub_stamp":"2023-05-20T01:28:25.659712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:48:51.119722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:28.104366+0000","last_change":"2023-05-20T01:28:27.114722+0000","last_active":"2023-05-20T01:28:28.104366+0000","last_peered":"2023-05-20T01:28:28.104366+0000","last_clean":"2023-05-20T01:28:28.104366+0000","last_became_active":"2023-05-20T01:28:27.114464+0000","last_became_peered":"2023-05-20T01:28:27.114464+0000","last_unstale":"2023-05-20T01:28:28.104366+0000","last_undegraded":"2023-05-20T01:28:28.104366+0000","last_fullsized":"2023-05-20T01:28:28.104366+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:39:54.356057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":13,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:28.102089+0000","last_change":"2023-05-20T01:28:26.683304+0000","last_active":"2023-05-20T01:28:28.102089+0000","last_peered":"2023-05-20T01:28:28.102089+0000","last_clean":"2023-05-20T01:28:28.102089+0000","last_became_active":"2023-05-20T01:28:26.682988+0000","last_became_peered":"2023-05-20T01:28:26.682988+0000","last_unstale":"2023-05-20T01:28:28.102089+0000","last_undegraded":"2023-05-20T01:28:28.102089+0000","last_fullsized":"2023-05-20T01:28:28.102089+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_clean_scrub_stamp":"2023-05-20T01:28:25.659712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:17:14.758553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-05-20T01:28:27.128467+0000","last_change":"2023-05-20T01:28:27.125052+0000","last_active":"2023-05-20T01:28:27.128467+0000","last_peered":"2023-05-20T01:28:27.128467+0000","last_clean":"2023-05-20T01:28:27.128467+0000","last_became_active":"2023-05-20T01:28:27.124750+0000","last_became_peered":"2023-05-20T01:28:27.124750+0000","last_unstale":"2023-05-20T01:28:27.128467+0000","last_undegraded":"2023-05-20T01:28:27.128467+0000","last_fullsized":"2023-05-20T01:28:27.128467+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:12:34.227931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":13,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:29.601585+0000","last_change":"2023-05-20T01:28:26.696466+0000","last_active":"2023-05-20T01:28:29.601585+0000","last_peered":"2023-05-20T01:28:29.601585+0000","last_clean":"2023-05-20T01:28:29.601585+0000","last_became_active":"2023-05-20T01:28:26.696005+0000","last_became_peered":"2023-05-20T01:28:26.696005+0000","last_unstale":"2023-05-20T01:28:29.601585+0000","last_undegraded":"2023-05-20T01:28:29.601585+0000","last_fullsized":"2023-05-20T01:28:29.601585+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_clean_scrub_stamp":"2023-05-20T01:28:25.659712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:55:26.214314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:28.105316+0000","last_change":"2023-05-20T01:28:27.121067+0000","last_active":"2023-05-20T01:28:28.105316+0000","last_peered":"2023-05-20T01:28:28.105316+0000","last_clean":"2023-05-20T01:28:28.105316+0000","last_became_active":"2023-05-20T01:28:27.120742+0000","last_became_peered":"2023-05-20T01:28:27.120742+0000","last_unstale":"2023-05-20T01:28:28.105316+0000","last_undegraded":"2023-05-20T01:28:28.105316+0000","last_fullsized":"2023-05-20T01:28:28.105316+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:17:25.249433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.7","version":"135'2","reported_seq":13,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-05-20T01:28:27.123699+0000","last_change":"2023-05-20T01:28:26.684920+0000","last_active":"2023-05-20T01:28:27.123699+0000","last_peered":"2023-05-20T01:28:27.123699+0000","last_clean":"2023-05-20T01:28:27.123699+0000","last_became_active":"2023-05-20T01:28:26.684628+0000","last_became_peered":"2023-05-20T01:28:26.684628+0000","last_unstale":"2023-05-20T01:28:27.123699+0000","last_undegraded":"2023-05-20T01:28:27.123699+0000","last_fullsized":"2023-05-20T01:28:27.123699+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_clean_scrub_stamp":"2023-05-20T01:28:25.659712+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:34:33.232021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-05-20T01:28:27.128534+0000","last_change":"2023-05-20T01:28:27.119165+0000","last_active":"2023-05-20T01:28:27.128534+0000","last_peered":"2023-05-20T01:28:27.128534+0000","last_clean":"2023-05-20T01:28:27.128534+0000","last_became_active":"2023-05-20T01:28:27.118928+0000","last_became_peered":"2023-05-20T01:28:27.118928+0000","last_unstale":"2023-05-20T01:28:27.128534+0000","last_undegraded":"2023-05-20T01:28:27.128534+0000","last_fullsized":"2023-05-20T01:28:27.128534+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:27:16.782676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-05-20T01:28:27.103441+0000","last_change":"2023-05-20T01:28:26.693198+0000","last_active":"2023-05-20T01:28:27.103441+0000","last_peered":"2023-05-20T01:28:27.103441+0000","last_clean":"2023-05-20T01:28:27.103441+0000","last_became_active":"2023-05-20T01:28:26.692911+0000","last_became_peered":"2023-05-20T01:28:26.692911+0000","last_unstale":"2023-05-20T01:28:27.103441+0000","last_undegraded":"2023-05-20T01:28:27.103441+0000","last_fullsized":"2023-05-20T01:28:27.103441+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_clean_scrub_stamp":"2023-05-20T01:28:25.659712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:41:13.080238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:28.104742+0000","last_change":"2023-05-20T01:28:27.114743+0000","last_active":"2023-05-20T01:28:28.104742+0000","last_peered":"2023-05-20T01:28:28.104742+0000","last_clean":"2023-05-20T01:28:28.104742+0000","last_became_active":"2023-05-20T01:28:27.114454+0000","last_became_peered":"2023-05-20T01:28:27.114454+0000","last_unstale":"2023-05-20T01:28:28.104742+0000","last_undegraded":"2023-05-20T01:28:28.104742+0000","last_fullsized":"2023-05-20T01:28:28.104742+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:24:34.647958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":13,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:28.104903+0000","last_change":"2023-05-20T01:28:26.695359+0000","last_active":"2023-05-20T01:28:28.104903+0000","last_peered":"2023-05-20T01:28:28.104903+0000","last_clean":"2023-05-20T01:28:28.104903+0000","last_became_active":"2023-05-20T01:28:26.695082+0000","last_became_peered":"2023-05-20T01:28:26.695082+0000","last_unstale":"2023-05-20T01:28:28.104903+0000","last_undegraded":"2023-05-20T01:28:28.104903+0000","last_fullsized":"2023-05-20T01:28:28.104903+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_clean_scrub_stamp":"2023-05-20T01:28:25.659712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:22:23.906600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:28.105155+0000","last_change":"2023-05-20T01:28:27.124168+0000","last_active":"2023-05-20T01:28:28.105155+0000","last_peered":"2023-05-20T01:28:28.105155+0000","last_clean":"2023-05-20T01:28:28.105155+0000","last_became_active":"2023-05-20T01:28:27.123907+0000","last_became_peered":"2023-05-20T01:28:27.123907+0000","last_unstale":"2023-05-20T01:28:28.105155+0000","last_undegraded":"2023-05-20T01:28:28.105155+0000","last_fullsized":"2023-05-20T01:28:28.105155+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:57:28.027837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":13,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:29.601711+0000","last_change":"2023-05-20T01:28:26.694256+0000","last_active":"2023-05-20T01:28:29.601711+0000","last_peered":"2023-05-20T01:28:29.601711+0000","last_clean":"2023-05-20T01:28:29.601711+0000","last_became_active":"2023-05-20T01:28:26.693870+0000","last_became_peered":"2023-05-20T01:28:26.693870+0000","last_unstale":"2023-05-20T01:28:29.601711+0000","last_undegraded":"2023-05-20T01:28:29.601711+0000","last_fullsized":"2023-05-20T01:28:29.601711+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_clean_scrub_stamp":"2023-05-20T01:28:25.659712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:34:29.955497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-05-20T01:28:27.120640+0000","last_change":"2023-05-20T01:28:27.112949+0000","last_active":"2023-05-20T01:28:27.120640+0000","last_peered":"2023-05-20T01:28:27.120640+0000","last_clean":"2023-05-20T01:28:27.120640+0000","last_became_active":"2023-05-20T01:28:27.112651+0000","last_became_peered":"2023-05-20T01:28:27.112651+0000","last_unstale":"2023-05-20T01:28:27.120640+0000","last_undegraded":"2023-05-20T01:28:27.120640+0000","last_fullsized":"2023-05-20T01:28:27.120640+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:23:28.303320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":13,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:28.104928+0000","last_change":"2023-05-20T01:28:26.692720+0000","last_active":"2023-05-20T01:28:28.104928+0000","last_peered":"2023-05-20T01:28:28.104928+0000","last_clean":"2023-05-20T01:28:28.104928+0000","last_became_active":"2023-05-20T01:28:26.692433+0000","last_became_peered":"2023-05-20T01:28:26.692433+0000","last_unstale":"2023-05-20T01:28:28.104928+0000","last_undegraded":"2023-05-20T01:28:28.104928+0000","last_fullsized":"2023-05-20T01:28:28.104928+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_clean_scrub_stamp":"2023-05-20T01:28:25.659712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:07:59.020182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-05-20T01:28:27.130732+0000","last_change":"2023-05-20T01:28:27.126550+0000","last_active":"2023-05-20T01:28:27.130732+0000","last_peered":"2023-05-20T01:28:27.130732+0000","last_clean":"2023-05-20T01:28:27.130732+0000","last_became_active":"2023-05-20T01:28:27.126233+0000","last_became_peered":"2023-05-20T01:28:27.126233+0000","last_unstale":"2023-05-20T01:28:27.130732+0000","last_undegraded":"2023-05-20T01:28:27.130732+0000","last_fullsized":"2023-05-20T01:28:27.130732+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:42:47.941689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-05-20T01:28:27.104612+0000","last_change":"2023-05-20T01:28:26.696774+0000","last_active":"2023-05-20T01:28:27.104612+0000","last_peered":"2023-05-20T01:28:27.104612+0000","last_clean":"2023-05-20T01:28:27.104612+0000","last_became_active":"2023-05-20T01:28:26.696485+0000","last_became_peered":"2023-05-20T01:28:26.696485+0000","last_unstale":"2023-05-20T01:28:27.104612+0000","last_undegraded":"2023-05-20T01:28:27.104612+0000","last_fullsized":"2023-05-20T01:28:27.104612+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_clean_scrub_stamp":"2023-05-20T01:28:25.659712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:59:07.152477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:28.102190+0000","last_change":"2023-05-20T01:28:27.120405+0000","last_active":"2023-05-20T01:28:28.102190+0000","last_peered":"2023-05-20T01:28:28.102190+0000","last_clean":"2023-05-20T01:28:28.102190+0000","last_became_active":"2023-05-20T01:28:27.120117+0000","last_became_peered":"2023-05-20T01:28:27.120117+0000","last_unstale":"2023-05-20T01:28:28.102190+0000","last_undegraded":"2023-05-20T01:28:28.102190+0000","last_fullsized":"2023-05-20T01:28:28.102190+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:21:33.984873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":13,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:28.102087+0000","last_change":"2023-05-20T01:28:26.682752+0000","last_active":"2023-05-20T01:28:28.102087+0000","last_peered":"2023-05-20T01:28:28.102087+0000","last_clean":"2023-05-20T01:28:28.102087+0000","last_became_active":"2023-05-20T01:28:26.682472+0000","last_became_peered":"2023-05-20T01:28:26.682472+0000","last_unstale":"2023-05-20T01:28:28.102087+0000","last_undegraded":"2023-05-20T01:28:28.102087+0000","last_fullsized":"2023-05-20T01:28:28.102087+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_clean_scrub_stamp":"2023-05-20T01:28:25.659712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:40:05.403429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:28.104922+0000","last_change":"2023-05-20T01:28:27.111490+0000","last_active":"2023-05-20T01:28:28.104922+0000","last_peered":"2023-05-20T01:28:28.104922+0000","last_clean":"2023-05-20T01:28:28.104922+0000","last_became_active":"2023-05-20T01:28:27.111220+0000","last_became_peered":"2023-05-20T01:28:27.111220+0000","last_unstale":"2023-05-20T01:28:28.104922+0000","last_undegraded":"2023-05-20T01:28:28.104922+0000","last_fullsized":"2023-05-20T01:28:28.104922+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:50:40.180843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":13,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:29.601644+0000","last_change":"2023-05-20T01:28:26.694747+0000","last_active":"2023-05-20T01:28:29.601644+0000","last_peered":"2023-05-20T01:28:29.601644+0000","last_clean":"2023-05-20T01:28:29.601644+0000","last_became_active":"2023-05-20T01:28:26.694300+0000","last_became_peered":"2023-05-20T01:28:26.694300+0000","last_unstale":"2023-05-20T01:28:29.601644+0000","last_undegraded":"2023-05-20T01:28:29.601644+0000","last_fullsized":"2023-05-20T01:28:29.601644+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_clean_scrub_stamp":"2023-05-20T01:28:25.659712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:51:45.694079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-05-20T01:28:27.128403+0000","last_change":"2023-05-20T01:28:27.122177+0000","last_active":"2023-05-20T01:28:27.128403+0000","last_peered":"2023-05-20T01:28:27.128403+0000","last_clean":"2023-05-20T01:28:27.128403+0000","last_became_active":"2023-05-20T01:28:27.120626+0000","last_became_peered":"2023-05-20T01:28:27.120626+0000","last_unstale":"2023-05-20T01:28:27.128403+0000","last_undegraded":"2023-05-20T01:28:27.128403+0000","last_fullsized":"2023-05-20T01:28:27.128403+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:02:44.921661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":13,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:28.102956+0000","last_change":"2023-05-20T01:28:26.691759+0000","last_active":"2023-05-20T01:28:28.102956+0000","last_peered":"2023-05-20T01:28:28.102956+0000","last_clean":"2023-05-20T01:28:28.102956+0000","last_became_active":"2023-05-20T01:28:26.691412+0000","last_became_peered":"2023-05-20T01:28:26.691412+0000","last_unstale":"2023-05-20T01:28:28.102956+0000","last_undegraded":"2023-05-20T01:28:28.102956+0000","last_fullsized":"2023-05-20T01:28:28.102956+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_clean_scrub_stamp":"2023-05-20T01:28:25.659712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T02:20:26.714018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":242,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:28.105217+0000","last_change":"2023-05-20T01:16:23.289068+0000","last_active":"2023-05-20T01:28:28.105217+0000","last_peered":"2023-05-20T01:28:28.105217+0000","last_clean":"2023-05-20T01:28:28.105217+0000","last_became_active":"2023-05-20T01:16:23.288781+0000","last_became_peered":"2023-05-20T01:16:23.288781+0000","last_unstale":"2023-05-20T01:28:28.105217+0000","last_undegraded":"2023-05-20T01:28:28.105217+0000","last_fullsized":"2023-05-20T01:28:28.105217+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T01:29:53.661129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-05-20T01:28:27.128648+0000","last_change":"2023-05-20T01:28:27.124712+0000","last_active":"2023-05-20T01:28:27.128648+0000","last_peered":"2023-05-20T01:28:27.128648+0000","last_clean":"2023-05-20T01:28:27.128648+0000","last_became_active":"2023-05-20T01:28:27.122733+0000","last_became_peered":"2023-05-20T01:28:27.122733+0000","last_unstale":"2023-05-20T01:28:27.128648+0000","last_undegraded":"2023-05-20T01:28:27.128648+0000","last_fullsized":"2023-05-20T01:28:27.128648+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:38:46.482786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-05-20T01:28:27.108063+0000","last_change":"2023-05-20T01:28:26.686983+0000","last_active":"2023-05-20T01:28:27.108063+0000","last_peered":"2023-05-20T01:28:27.108063+0000","last_clean":"2023-05-20T01:28:27.108063+0000","last_became_active":"2023-05-20T01:28:26.686713+0000","last_became_peered":"2023-05-20T01:28:26.686713+0000","last_unstale":"2023-05-20T01:28:27.108063+0000","last_undegraded":"2023-05-20T01:28:27.108063+0000","last_fullsized":"2023-05-20T01:28:27.108063+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_clean_scrub_stamp":"2023-05-20T01:28:25.659712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:20:30.818187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":254,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:28.102610+0000","last_change":"2023-05-20T01:16:22.829324+0000","last_active":"2023-05-20T01:28:28.102610+0000","last_peered":"2023-05-20T01:28:28.102610+0000","last_clean":"2023-05-20T01:28:28.102610+0000","last_became_active":"2023-05-20T01:16:22.828964+0000","last_became_peered":"2023-05-20T01:16:22.828964+0000","last_unstale":"2023-05-20T01:28:28.102610+0000","last_undegraded":"2023-05-20T01:28:28.102610+0000","last_fullsized":"2023-05-20T01:28:28.102610+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:02:43.785285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:28.290913+0000","last_change":"2023-05-20T01:28:27.116232+0000","last_active":"2023-05-20T01:28:28.290913+0000","last_peered":"2023-05-20T01:28:28.290913+0000","last_clean":"2023-05-20T01:28:28.290913+0000","last_became_active":"2023-05-20T01:28:27.115905+0000","last_became_peered":"2023-05-20T01:28:27.115905+0000","last_unstale":"2023-05-20T01:28:28.290913+0000","last_undegraded":"2023-05-20T01:28:28.290913+0000","last_fullsized":"2023-05-20T01:28:28.290913+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T10:04:43.248337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.10","version":"135'8","reported_seq":21,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:28.290755+0000","last_change":"2023-05-20T01:28:26.689073+0000","last_active":"2023-05-20T01:28:28.290755+0000","last_peered":"2023-05-20T01:28:28.290755+0000","last_clean":"2023-05-20T01:28:28.290755+0000","last_became_active":"2023-05-20T01:28:26.688542+0000","last_became_peered":"2023-05-20T01:28:26.688542+0000","last_unstale":"2023-05-20T01:28:28.290755+0000","last_undegraded":"2023-05-20T01:28:28.290755+0000","last_fullsized":"2023-05-20T01:28:28.290755+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_clean_scrub_stamp":"2023-05-20T01:28:25.659712+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T11:23:27.047072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":220,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:28.104742+0000","last_change":"2023-05-20T01:16:22.834754+0000","last_active":"2023-05-20T01:28:28.104742+0000","last_peered":"2023-05-20T01:28:28.104742+0000","last_clean":"2023-05-20T01:28:28.104742+0000","last_became_active":"2023-05-20T01:16:22.834437+0000","last_became_peered":"2023-05-20T01:16:22.834437+0000","last_unstale":"2023-05-20T01:28:28.104742+0000","last_undegraded":"2023-05-20T01:28:28.104742+0000","last_fullsized":"2023-05-20T01:28:28.104742+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:41:13.799210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":351,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-05-20T01:28:27.103303+0000","last_change":"2023-05-20T01:16:22.834920+0000","last_active":"2023-05-20T01:28:27.103303+0000","last_peered":"2023-05-20T01:28:27.103303+0000","last_clean":"2023-05-20T01:28:27.103303+0000","last_became_active":"2023-05-20T01:16:22.833924+0000","last_became_peered":"2023-05-20T01:16:22.833924+0000","last_unstale":"2023-05-20T01:28:27.103303+0000","last_undegraded":"2023-05-20T01:28:27.103303+0000","last_fullsized":"2023-05-20T01:28:27.103303+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:22:16.802805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:28.104394+0000","last_change":"2023-05-20T01:28:27.119031+0000","last_active":"2023-05-20T01:28:28.104394+0000","last_peered":"2023-05-20T01:28:28.104394+0000","last_clean":"2023-05-20T01:28:28.104394+0000","last_became_active":"2023-05-20T01:28:27.118764+0000","last_became_peered":"2023-05-20T01:28:27.118764+0000","last_unstale":"2023-05-20T01:28:28.104394+0000","last_undegraded":"2023-05-20T01:28:28.104394+0000","last_fullsized":"2023-05-20T01:28:28.104394+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:55:54.880516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":13,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:28.104630+0000","last_change":"2023-05-20T01:28:26.691756+0000","last_active":"2023-05-20T01:28:28.104630+0000","last_peered":"2023-05-20T01:28:28.104630+0000","last_clean":"2023-05-20T01:28:28.104630+0000","last_became_active":"2023-05-20T01:28:26.691446+0000","last_became_peered":"2023-05-20T01:28:26.691446+0000","last_unstale":"2023-05-20T01:28:28.104630+0000","last_undegraded":"2023-05-20T01:28:28.104630+0000","last_fullsized":"2023-05-20T01:28:28.104630+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_clean_scrub_stamp":"2023-05-20T01:28:25.659712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:38:23.028213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":200,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-05-20T01:28:27.105137+0000","last_change":"2023-05-20T01:16:23.290259+0000","last_active":"2023-05-20T01:28:27.105137+0000","last_peered":"2023-05-20T01:28:27.105137+0000","last_clean":"2023-05-20T01:28:27.105137+0000","last_became_active":"2023-05-20T01:16:23.289911+0000","last_became_peered":"2023-05-20T01:16:23.289911+0000","last_unstale":"2023-05-20T01:28:27.105137+0000","last_undegraded":"2023-05-20T01:28:27.105137+0000","last_fullsized":"2023-05-20T01:28:27.105137+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T12:29:51.437617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-05-20T01:28:27.128124+0000","last_change":"2023-05-20T01:28:27.124801+0000","last_active":"2023-05-20T01:28:27.128124+0000","last_peered":"2023-05-20T01:28:27.128124+0000","last_clean":"2023-05-20T01:28:27.128124+0000","last_became_active":"2023-05-20T01:28:27.123965+0000","last_became_peered":"2023-05-20T01:28:27.123965+0000","last_unstale":"2023-05-20T01:28:27.128124+0000","last_undegraded":"2023-05-20T01:28:27.128124+0000","last_fullsized":"2023-05-20T01:28:27.128124+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:03:35.578428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"135'2","reported_seq":13,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-05-20T01:28:27.127518+0000","last_change":"2023-05-20T01:28:26.693285+0000","last_active":"2023-05-20T01:28:27.127518+0000","last_peered":"2023-05-20T01:28:27.127518+0000","last_clean":"2023-05-20T01:28:27.127518+0000","last_became_active":"2023-05-20T01:28:26.693057+0000","last_became_peered":"2023-05-20T01:28:26.693057+0000","last_unstale":"2023-05-20T01:28:27.127518+0000","last_undegraded":"2023-05-20T01:28:27.127518+0000","last_fullsized":"2023-05-20T01:28:27.127518+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_clean_scrub_stamp":"2023-05-20T01:28:25.659712+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T09:52:40.552794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":201,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-05-20T01:28:27.103443+0000","last_change":"2023-05-20T01:16:23.290207+0000","last_active":"2023-05-20T01:28:27.103443+0000","last_peered":"2023-05-20T01:28:27.103443+0000","last_clean":"2023-05-20T01:28:27.103443+0000","last_became_active":"2023-05-20T01:16:23.289928+0000","last_became_peered":"2023-05-20T01:16:23.289928+0000","last_unstale":"2023-05-20T01:28:27.103443+0000","last_undegraded":"2023-05-20T01:28:27.103443+0000","last_fullsized":"2023-05-20T01:28:27.103443+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T06:48:20.917850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-05-20T01:28:27.128691+0000","last_change":"2023-05-20T01:28:27.125501+0000","last_active":"2023-05-20T01:28:27.128691+0000","last_peered":"2023-05-20T01:28:27.128691+0000","last_clean":"2023-05-20T01:28:27.128691+0000","last_became_active":"2023-05-20T01:28:27.124194+0000","last_became_peered":"2023-05-20T01:28:27.124194+0000","last_unstale":"2023-05-20T01:28:27.128691+0000","last_undegraded":"2023-05-20T01:28:27.128691+0000","last_fullsized":"2023-05-20T01:28:27.128691+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T13:12:53.827858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-05-20T01:28:27.114846+0000","last_change":"2023-05-20T01:28:26.685301+0000","last_active":"2023-05-20T01:28:27.114846+0000","last_peered":"2023-05-20T01:28:27.114846+0000","last_clean":"2023-05-20T01:28:27.114846+0000","last_became_active":"2023-05-20T01:28:26.685007+0000","last_became_peered":"2023-05-20T01:28:26.685007+0000","last_unstale":"2023-05-20T01:28:27.114846+0000","last_undegraded":"2023-05-20T01:28:27.114846+0000","last_fullsized":"2023-05-20T01:28:27.114846+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_clean_scrub_stamp":"2023-05-20T01:28:25.659712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T01:52:41.398890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":202,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-05-20T01:28:27.104385+0000","last_change":"2023-05-20T01:16:23.290538+0000","last_active":"2023-05-20T01:28:27.104385+0000","last_peered":"2023-05-20T01:28:27.104385+0000","last_clean":"2023-05-20T01:28:27.104385+0000","last_became_active":"2023-05-20T01:16:23.290230+0000","last_became_peered":"2023-05-20T01:16:23.290230+0000","last_unstale":"2023-05-20T01:28:27.104385+0000","last_undegraded":"2023-05-20T01:28:27.104385+0000","last_fullsized":"2023-05-20T01:28:27.104385+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:45:05.815733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-05-20T01:28:27.124655+0000","last_change":"2023-05-20T01:28:27.120192+0000","last_active":"2023-05-20T01:28:27.124655+0000","last_peered":"2023-05-20T01:28:27.124655+0000","last_clean":"2023-05-20T01:28:27.124655+0000","last_became_active":"2023-05-20T01:28:27.119859+0000","last_became_peered":"2023-05-20T01:28:27.119859+0000","last_unstale":"2023-05-20T01:28:27.124655+0000","last_undegraded":"2023-05-20T01:28:27.124655+0000","last_fullsized":"2023-05-20T01:28:27.124655+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T04:11:21.663266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"135'3","reported_seq":14,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-05-20T01:28:27.123533+0000","last_change":"2023-05-20T01:28:26.692131+0000","last_active":"2023-05-20T01:28:27.123533+0000","last_peered":"2023-05-20T01:28:27.123533+0000","last_clean":"2023-05-20T01:28:27.123533+0000","last_became_active":"2023-05-20T01:28:26.691813+0000","last_became_peered":"2023-05-20T01:28:26.691813+0000","last_unstale":"2023-05-20T01:28:27.123533+0000","last_undegraded":"2023-05-20T01:28:27.123533+0000","last_fullsized":"2023-05-20T01:28:27.123533+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_clean_scrub_stamp":"2023-05-20T01:28:25.659712+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:55:03.697685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":226,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:28.291185+0000","last_change":"2023-05-20T01:16:23.290341+0000","last_active":"2023-05-20T01:28:28.291185+0000","last_peered":"2023-05-20T01:28:28.291185+0000","last_clean":"2023-05-20T01:28:28.291185+0000","last_became_active":"2023-05-20T01:16:23.290053+0000","last_became_peered":"2023-05-20T01:16:23.290053+0000","last_unstale":"2023-05-20T01:28:28.291185+0000","last_undegraded":"2023-05-20T01:28:28.291185+0000","last_fullsized":"2023-05-20T01:28:28.291185+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T05:57:33.990802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-05-20T01:28:27.120856+0000","last_change":"2023-05-20T01:28:27.118519+0000","last_active":"2023-05-20T01:28:27.120856+0000","last_peered":"2023-05-20T01:28:27.120856+0000","last_clean":"2023-05-20T01:28:27.120856+0000","last_became_active":"2023-05-20T01:28:27.118248+0000","last_became_peered":"2023-05-20T01:28:27.118248+0000","last_unstale":"2023-05-20T01:28:27.120856+0000","last_undegraded":"2023-05-20T01:28:27.120856+0000","last_fullsized":"2023-05-20T01:28:27.120856+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:26.664501+0000","last_clean_scrub_stamp":"2023-05-20T01:28:26.664501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T03:51:46.698868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"135'6","reported_seq":19,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:28.291211+0000","last_change":"2023-05-20T01:28:26.689861+0000","last_active":"2023-05-20T01:28:28.291211+0000","last_peered":"2023-05-20T01:28:28.291211+0000","last_clean":"2023-05-20T01:28:28.291211+0000","last_became_active":"2023-05-20T01:28:26.689504+0000","last_became_peered":"2023-05-20T01:28:26.689504+0000","last_unstale":"2023-05-20T01:28:28.291211+0000","last_undegraded":"2023-05-20T01:28:28.291211+0000","last_fullsized":"2023-05-20T01:28:28.291211+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:28:25.659712+0000","last_clean_scrub_stamp":"2023-05-20T01:28:25.659712+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T07:04:55.358540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":226,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-05-20T01:28:28.291316+0000","last_change":"2023-05-20T01:16:22.834330+0000","last_active":"2023-05-20T01:28:28.291316+0000","last_peered":"2023-05-20T01:28:28.291316+0000","last_clean":"2023-05-20T01:28:28.291316+0000","last_became_active":"2023-05-20T01:16:22.833946+0000","last_became_peered":"2023-05-20T01:16:22.833946+0000","last_unstale":"2023-05-20T01:28:28.291316+0000","last_undegraded":"2023-05-20T01:28:28.291316+0000","last_fullsized":"2023-05-20T01:28:28.291316+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-05-20T01:16:21.805525+0000","last_clean_scrub_stamp":"2023-05-20T01:16:21.805525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-05-21T08:27:45.076913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574998,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":17420,"kb_used_data":2244,"kb_used_omap":0,"kb_used_meta":15168,"kb_avail":94354420,"statfs":{"total":96636764160,"available":96618926080,"internally_reserved":0,"allocated":2297856,"data_stored":1463037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":15532032},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:27:41 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.53500000000000003,"15min":0.51700000000000002},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.76200000000000001,"5min":0.80600000000000005,"15min":0.88200000000000001},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.54600000000000004,"15min":0.54700000000000004},"min":{"1min":0.31,"5min":0.31,"15min":0.30599999999999999},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.90000000000000002},"last":0.58099999999999996}]},{"osd":1,"last update":"Sat May 20 01:27:41 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54400000000000004,"15min":0.67100000000000004},"min":{"1min":0.33900000000000002,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.81899999999999995,"5min":0.95599999999999996,"15min":8.8879999999999999},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.56399999999999995,"15min":0.70399999999999996},"min":{"1min":0.40000000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.80700000000000005,"5min":0.85399999999999998,"15min":8.875},"last":0.505}]},{"osd":2,"last update":"Sat May 20 01:27:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.54300000000000004,"15min":0.66000000000000003},"min":{"1min":0.38900000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":8.9120000000000008},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54300000000000004,"15min":0.66300000000000003},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.75,"5min":0.75,"15min":8.8520000000000003},"last":0.46800000000000003}]},{"osd":3,"last update":"Sat May 20 01:27:41 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.53900000000000003,"15min":0.69699999999999995},"min":{"1min":0.40899999999999997,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.77400000000000002,"5min":0.89900000000000002,"15min":10.884},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.58099999999999996,"15min":0.748},"min":{"1min":0.441,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.80100000000000005,"5min":0.88100000000000001,"15min":10.952},"last":0.55300000000000005}]},{"osd":4,"last update":"Sat May 20 01:27:41 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.51000000000000001,"15min":0.51100000000000001},"min":{"1min":0.38900000000000001,"5min":0.34999999999999998,"15min":0.313},"max":{"1min":0.79800000000000004,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.52500000000000002,"15min":0.53000000000000003},"min":{"1min":0.39400000000000002,"5min":0.309,"15min":0.309},"max":{"1min":0.66100000000000003,"5min":0.82599999999999996,"15min":0.875},"last":0.34999999999999998}]},{"osd":5,"last update":"Sat May 20 01:27:41 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.54800000000000004},"min":{"1min":0.35199999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.82399999999999995,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.57399999999999995,"15min":0.57799999999999996},"min":{"1min":0.42699999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.755,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.436}]},{"osd":6,"last update":"Sat May 20 01:27:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59099999999999997,"15min":0.59699999999999998},"min":{"1min":0.46200000000000002,"5min":0.39600000000000002,"15min":0.378},"max":{"1min":0.83299999999999996,"5min":1.028,"15min":1.028},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56599999999999995,"15min":0.58099999999999996},"min":{"1min":0.46100000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.71599999999999997,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.48699999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574997,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":16648,"kb_used_data":1664,"kb_used_omap":0,"kb_used_meta":14976,"kb_avail":94355192,"statfs":{"total":96636764160,"available":96619716608,"internally_reserved":0,"allocated":1703936,"data_stored":879004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":15335424},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:27:50 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.49199999999999999,"15min":0.48499999999999999},"min":{"1min":0.29399999999999998,"5min":0.28299999999999997,"15min":0.26900000000000002},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.50900000000000001,"15min":0.51800000000000002},"min":{"1min":0.34599999999999997,"5min":0.309,"15min":0.25800000000000001},"max":{"1min":0.78400000000000003,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.46700000000000003}]},{"osd":1,"last update":"Sat May 20 01:27:50 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.52600000000000002,"15min":0.52300000000000002},"min":{"1min":0.36399999999999999,"5min":0.312,"15min":0.28000000000000003},"max":{"1min":0.77400000000000002,"5min":0.79300000000000004,"15min":0.83799999999999997},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.53300000000000003},"min":{"1min":0.35899999999999999,"5min":0.33000000000000002,"15min":0.318},"max":{"1min":0.71499999999999997,"5min":0.77800000000000002,"15min":0.79200000000000004},"last":0.52500000000000002}]},{"osd":2,"last update":"Sat May 20 01:27:50 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.504,"15min":0.505},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.76600000000000001},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.55400000000000005,"15min":0.55200000000000005},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.28699999999999998},"max":{"1min":0.72699999999999998,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.51400000000000001}]},{"osd":3,"last update":"Sat May 20 01:27:50 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.54900000000000004,"15min":0.53800000000000003},"min":{"1min":0.40300000000000002,"5min":0.32400000000000001,"15min":0.311},"max":{"1min":0.753,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.54400000000000004,"15min":0.54300000000000004},"min":{"1min":0.378,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.71699999999999997,"5min":0.78400000000000003,"15min":0.79500000000000004},"last":0.57699999999999996}]},{"osd":4,"last update":"Sat May 20 01:27:50 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.53400000000000003},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.68300000000000005,"5min":0.754,"15min":0.77600000000000002},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.54500000000000004,"15min":0.55600000000000005},"min":{"1min":0.40000000000000002,"5min":0.38100000000000001,"15min":0.32700000000000001},"max":{"1min":0.69299999999999995,"5min":0.77400000000000002,"15min":0.80100000000000005},"last":0.55400000000000005}]},{"osd":5,"last update":"Sat May 20 01:27:50 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.51900000000000002,"15min":0.53600000000000003},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":1.2310000000000001},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56799999999999995,"15min":0.59099999999999997},"min":{"1min":0.41999999999999998,"5min":0.372,"15min":0.34399999999999997},"max":{"1min":0.72399999999999998,"5min":0.83499999999999996,"15min":0.89400000000000002},"last":0.44600000000000001}]},{"osd":7,"last update":"Sat May 20 01:27:50 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56999999999999995,"15min":0.58699999999999997},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.73299999999999998,"5min":0.80200000000000005,"15min":0.92900000000000005},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.58399999999999996,"15min":0.59999999999999998},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.379},"max":{"1min":0.747,"5min":0.78500000000000003,"15min":0.85199999999999998},"last":0.49099999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574997,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":18368,"kb_used_data":2232,"kb_used_omap":0,"kb_used_meta":16128,"kb_avail":94353472,"statfs":{"total":96636764160,"available":96617955328,"internally_reserved":0,"allocated":2285568,"data_stored":1467808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":16515072},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:27:42 2023","interfaces":[{"interface":"back","average":{"1min":0.32800000000000001,"5min":0.311,"15min":0.313},"min":{"1min":0.22800000000000001,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":0.54500000000000004,"5min":0.60799999999999998,"15min":1.194},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.34799999999999998,"15min":0.32700000000000001},"min":{"1min":0.21299999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.59399999999999997,"5min":0.73199999999999998,"15min":0.75800000000000001},"last":0.29199999999999998}]},{"osd":2,"last update":"Sat May 20 01:27:42 2023","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.33600000000000002,"15min":0.32500000000000001},"min":{"1min":0.20200000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.49199999999999999,"5min":0.56799999999999995,"15min":0.66400000000000003},"last":0.27600000000000002},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"min":{"1min":0.30499999999999999,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.59799999999999998,"5min":0.68200000000000005,"15min":1.3580000000000001},"last":0.39700000000000002}]},{"osd":3,"last update":"Sat May 20 01:27:42 2023","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.39300000000000002,"15min":0.377},"min":{"1min":0.24399999999999999,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.54400000000000004,"5min":0.755,"15min":1.619},"last":0.377},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.38200000000000001,"15min":0.38100000000000001},"min":{"1min":0.26700000000000002,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.56499999999999995,"5min":0.63100000000000001,"15min":1.8400000000000001},"last":0.33000000000000002}]},{"osd":4,"last update":"Sat May 20 01:27:42 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.58599999999999997,"15min":0.58299999999999996},"min":{"1min":0.41699999999999998,"5min":0.34899999999999998,"15min":0.32200000000000001},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":1.2769999999999999},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.59599999999999997,"15min":0.58699999999999997},"min":{"1min":0.44900000000000001,"5min":0.38,"15min":0.312},"max":{"1min":0.76600000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.59199999999999997}]},{"osd":5,"last update":"Sat May 20 01:27:42 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.60499999999999998,"15min":0.59599999999999997},"min":{"1min":0.374,"5min":0.37,"15min":0.37},"max":{"1min":0.86299999999999999,"5min":0.96799999999999997,"15min":1.4510000000000001},"last":0.622},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.59499999999999997,"15min":0.60399999999999998},"min":{"1min":0.439,"5min":0.38900000000000001,"15min":0.378},"max":{"1min":0.89700000000000002,"5min":1.1299999999999999,"15min":1.409},"last":0.60399999999999998}]},{"osd":6,"last update":"Sat May 20 01:27:42 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.61299999999999999,"15min":0.60799999999999998},"min":{"1min":0.38800000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.92200000000000004,"5min":1.1499999999999999,"15min":1.575},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.60099999999999998,"15min":0.59799999999999998},"min":{"1min":0.442,"5min":0.41299999999999998,"15min":0.33200000000000002},"max":{"1min":0.88200000000000001,"5min":1.002,"15min":1.0600000000000001},"last":0.51000000000000001}]},{"osd":7,"last update":"Sat May 20 01:27:42 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.60699999999999998,"15min":0.60799999999999998},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.36499999999999999},"max":{"1min":0.84999999999999998,"5min":1.0229999999999999,"15min":1.496},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65600000000000003,"15min":0.64200000000000002},"min":{"1min":0.443,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.79400000000000004,"5min":2.1709999999999998,"15min":2.1709999999999998},"last":0.51900000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574997,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":16684,"kb_used_data":1636,"kb_used_omap":0,"kb_used_meta":15040,"kb_avail":94355156,"statfs":{"total":96636764160,"available":96619679744,"internally_reserved":0,"allocated":1675264,"data_stored":869506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":15400960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat May 20 01:27:38 2023","interfaces":[{"interface":"back","average":{"1min":0.313,"5min":0.33300000000000002,"15min":0.32300000000000001},"min":{"1min":0.20399999999999999,"5min":0.17999999999999999,"15min":0.17999999999999999},"max":{"1min":0.51200000000000001,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.379,"15min":0.35299999999999998},"min":{"1min":0.26100000000000001,"5min":0.21099999999999999,"15min":0.20399999999999999},"max":{"1min":0.59199999999999997,"5min":2.7559999999999998,"15min":2.7559999999999998},"last":0.45500000000000002}]},{"osd":2,"last update":"Sat May 20 01:27:38 2023","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.36399999999999999,"15min":0.34200000000000003},"min":{"1min":0.26000000000000001,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":0.45800000000000002,"5min":3.3079999999999998,"15min":3.3079999999999998},"last":0.24399999999999999},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.46000000000000002,"15min":0.40799999999999997},"min":{"1min":0.27500000000000002,"5min":0.26200000000000001,"15min":0.216},"max":{"1min":0.54700000000000004,"5min":5.7270000000000003,"15min":5.7270000000000003},"last":0.35599999999999998}]},{"osd":3,"last update":"Sat May 20 01:27:38 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.40300000000000002,"15min":0.375},"min":{"1min":0.26600000000000001,"5min":0.22,"15min":0.22},"max":{"1min":0.52400000000000002,"5min":4.258,"15min":4.258},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.40999999999999998,"15min":0.38},"min":{"1min":0.28299999999999997,"5min":0.245,"15min":0.245},"max":{"1min":0.505,"5min":2.714,"15min":2.714},"last":0.29499999999999998}]},{"osd":4,"last update":"Sat May 20 01:27:38 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.73799999999999999,"15min":0.61399999999999999},"min":{"1min":0.41699999999999998,"5min":0.372,"15min":0.34399999999999997},"max":{"1min":0.70699999999999996,"5min":13.125,"15min":13.125},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.61599999999999999,"15min":0.58899999999999997},"min":{"1min":0.44400000000000001,"5min":0.39200000000000002,"15min":0.36299999999999999},"max":{"1min":0.73399999999999999,"5min":1.9930000000000001,"15min":1.9930000000000001},"last":0.54700000000000004}]},{"osd":5,"last update":"Sat May 20 01:27:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.64600000000000002,"15min":0.60099999999999998},"min":{"1min":0.40799999999999997,"5min":0.379,"15min":0.36099999999999999},"max":{"1min":0.80600000000000005,"5min":5.3620000000000001,"15min":5.3620000000000001},"last":0.442},{"interface":"front","average":{"1min":0.62,"5min":0.68000000000000005,"15min":0.626},"min":{"1min":0.48499999999999999,"5min":0.374,"15min":0.374},"max":{"1min":0.752,"5min":5.7999999999999998,"15min":5.7999999999999998},"last":0.56100000000000005}]},{"osd":6,"last update":"Sat May 20 01:27:38 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.74199999999999999,"15min":0.64400000000000002},"min":{"1min":0.43099999999999999,"5min":0.36899999999999999,"15min":0.35199999999999998},"max":{"1min":0.81999999999999995,"5min":10.042999999999999,"15min":10.042999999999999},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.628,"15min":0.623},"min":{"1min":0.48599999999999999,"5min":0.377,"15min":0.377},"max":{"1min":0.78700000000000003,"5min":1.367,"15min":1.367},"last":0.503}]},{"osd":7,"last update":"Sat May 20 01:27:38 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.65900000000000003,"15min":0.61599999999999999},"min":{"1min":0.54000000000000004,"5min":0.433,"15min":0.39600000000000002},"max":{"1min":0.85499999999999998,"5min":2.802,"15min":2.802},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.75800000000000001,"15min":0.66800000000000004},"min":{"1min":0.46400000000000002,"5min":0.439,"15min":0.42099999999999999},"max":{"1min":0.77900000000000003,"5min":10.085000000000001,"15min":10.085000000000001},"last":0.46700000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574997,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":16248,"kb_used_data":1648,"kb_used_omap":0,"kb_used_meta":14592,"kb_avail":94355592,"statfs":{"total":96636764160,"available":96620126208,"internally_reserved":0,"allocated":1687552,"data_stored":871083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":14942208},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.30399999999999999,"5min":0.38100000000000001,"15min":0.34499999999999997},"min":{"1min":0.246,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":0.44600000000000001,"5min":10.262,"15min":10.262},"last":0.27000000000000002},{"interface":"front","average":{"1min":0.33600000000000002,"5min":0.40999999999999998,"15min":0.36599999999999999},"min":{"1min":0.27300000000000002,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.39500000000000002,"5min":12.146000000000001,"15min":12.146000000000001},"last":0.32500000000000001}]},{"osd":1,"last update":"Sat May 20 01:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.38500000000000001,"15min":0.38200000000000001},"min":{"1min":0.29799999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.53200000000000003,"5min":5.4939999999999998,"15min":5.4939999999999998},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.42399999999999999,"15min":0.42099999999999999},"min":{"1min":0.33400000000000002,"5min":0.27400000000000002,"15min":0.249},"max":{"1min":0.51200000000000001,"5min":6.2859999999999996,"15min":6.2859999999999996},"last":0.37}]},{"osd":3,"last update":"Sat May 20 01:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.39700000000000002,"15min":0.38800000000000001},"min":{"1min":0.25900000000000001,"5min":0.218,"15min":0.216},"max":{"1min":0.57799999999999996,"5min":6.351,"15min":6.351},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.45100000000000001,"15min":0.44},"min":{"1min":0.36499999999999999,"5min":0.28000000000000003,"15min":0.27600000000000002},"max":{"1min":0.629,"5min":7.2999999999999998,"15min":7.2999999999999998},"last":0.38400000000000001}]},{"osd":4,"last update":"Sat May 20 01:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.64400000000000002,"15min":0.60099999999999998},"min":{"1min":0.46999999999999997,"5min":0.38800000000000001,"15min":0.34899999999999998},"max":{"1min":0.76900000000000002,"5min":7.2629999999999999,"15min":7.2629999999999999},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.64900000000000002,"15min":0.627},"min":{"1min":0.48899999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.79400000000000004,"5min":6.3159999999999998,"15min":6.3159999999999998},"last":0.57999999999999996}]},{"osd":5,"last update":"Sat May 20 01:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64000000000000001,"15min":0.628},"min":{"1min":0.48799999999999999,"5min":0.36299999999999999,"15min":0.33100000000000002},"max":{"1min":0.83599999999999997,"5min":5.524,"15min":5.524},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.628,"5min":0.65000000000000002,"15min":0.64400000000000002},"min":{"1min":0.50800000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.85399999999999998,"5min":5.5860000000000003,"15min":5.5860000000000003},"last":0.55400000000000005}]},{"osd":6,"last update":"Sat May 20 01:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.65200000000000002,"15min":0.64200000000000002},"min":{"1min":0.48499999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.80200000000000005,"5min":6.1920000000000002,"15min":6.1920000000000002},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.65000000000000002,"15min":0.64000000000000001},"min":{"1min":0.497,"5min":0.39800000000000002,"15min":0.36399999999999999},"max":{"1min":0.84699999999999998,"5min":6.069,"15min":6.069},"last":0.78500000000000003}]},{"osd":7,"last update":"Sat May 20 01:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.65400000000000003,"15min":0.65800000000000003},"min":{"1min":0.50900000000000001,"5min":0.39000000000000001,"15min":0.38900000000000001},"max":{"1min":0.79300000000000004,"5min":6.2370000000000001,"15min":6.2370000000000001},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.65200000000000002},"min":{"1min":0.50900000000000001,"5min":0.42399999999999999,"15min":0.42199999999999999},"max":{"1min":0.80900000000000005,"5min":2.9359999999999999,"15min":2.9359999999999999},"last":0.76700000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574997,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":16580,"kb_used_data":1660,"kb_used_omap":0,"kb_used_meta":14912,"kb_avail":94355260,"statfs":{"total":96636764160,"available":96619786240,"internally_reserved":0,"allocated":1699840,"data_stored":877600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":15269888},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:27:34 2023","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.32700000000000001,"15min":0.32000000000000001},"min":{"1min":0.20200000000000001,"5min":0.20100000000000001,"15min":0.188},"max":{"1min":0.60299999999999998,"5min":0.65000000000000002,"15min":1.528},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.35199999999999998,"15min":0.39900000000000002},"min":{"1min":0.215,"5min":0.20799999999999999,"15min":0.19500000000000001},"max":{"1min":0.623,"5min":0.72399999999999998,"15min":3.1890000000000001},"last":0.30299999999999999}]},{"osd":1,"last update":"Sat May 20 01:27:34 2023","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.34200000000000003,"15min":0.39100000000000001},"min":{"1min":0.24299999999999999,"5min":0.189,"15min":0.189},"max":{"1min":0.56399999999999995,"5min":0.73199999999999998,"15min":3.4260000000000002},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.377,"5min":0.378,"15min":0.36199999999999999},"min":{"1min":0.249,"5min":0.23100000000000001,"15min":0.221},"max":{"1min":0.59299999999999997,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.35199999999999998}]},{"osd":2,"last update":"Sat May 20 01:27:34 2023","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.374,"15min":0.38200000000000001},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.224},"max":{"1min":0.58699999999999997,"5min":0.67400000000000004,"15min":1.1619999999999999},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.41399999999999998,"15min":0.41099999999999998},"min":{"1min":0.28899999999999998,"5min":0.26400000000000001,"15min":0.24299999999999999},"max":{"1min":0.67900000000000005,"5min":0.78500000000000003,"15min":1.413},"last":0.38500000000000001}]},{"osd":4,"last update":"Sat May 20 01:27:34 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.57899999999999996,"15min":0.58199999999999996},"min":{"1min":0.40500000000000003,"5min":0.38700000000000001,"15min":0.38200000000000001},"max":{"1min":0.81299999999999994,"5min":0.95399999999999996,"15min":1.0700000000000001},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.60899999999999999,"15min":0.65500000000000003},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.35899999999999999},"max":{"1min":1.339,"5min":1.339,"15min":3.4660000000000002},"last":0.55900000000000005}]},{"osd":5,"last update":"Sat May 20 01:27:34 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.60699999999999998,"15min":0.65700000000000003},"min":{"1min":0.36499999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.274,"5min":1.274,"15min":3.5379999999999998},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61199999999999999,"15min":0.623},"min":{"1min":0.438,"5min":0.41999999999999998,"15min":0.374},"max":{"1min":0.78600000000000003,"5min":1.026,"15min":1.385},"last":0.47699999999999998}]},{"osd":6,"last update":"Sat May 20 01:27:34 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61699999999999999,"15min":0.626},"min":{"1min":0.41799999999999998,"5min":0.41199999999999998,"15min":0.38100000000000001},"max":{"1min":0.81299999999999994,"5min":0.99199999999999999,"15min":1.347},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.61599999999999999,"15min":0.623},"min":{"1min":0.45300000000000001,"5min":0.39500000000000002,"15min":0.374},"max":{"1min":0.88400000000000001,"5min":0.94999999999999996,"15min":1.107},"last":0.53900000000000003}]},{"osd":7,"last update":"Sat May 20 01:27:34 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.61599999999999999,"15min":0.629},"min":{"1min":0.40699999999999997,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.85099999999999998,"5min":0.92900000000000005,"15min":1.1359999999999999},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.65600000000000003,"15min":0.69999999999999996},"min":{"1min":0.45100000000000001,"5min":0.436,"15min":0.436},"max":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":3.5009999999999999},"last":0.61799999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574997,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":17664,"kb_used_data":1656,"kb_used_omap":0,"kb_used_meta":16000,"kb_avail":94354176,"statfs":{"total":96636764160,"available":96618676224,"internally_reserved":0,"allocated":1695744,"data_stored":877510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":16384000},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:27:37 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.51900000000000002,"15min":0.49399999999999999},"min":{"1min":0.34699999999999998,"5min":0.309,"15min":0.247},"max":{"1min":0.67300000000000004,"5min":0.73699999999999999,"15min":0.75600000000000001},"last":0.503},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.52200000000000002,"15min":0.50800000000000001},"min":{"1min":0.36199999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.746,"5min":0.81799999999999995,"15min":1.7989999999999999},"last":0.52800000000000002}]},{"osd":1,"last update":"Sat May 20 01:27:37 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.52100000000000002,"15min":0.52000000000000002},"min":{"1min":0.37,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.66200000000000003,"5min":0.75800000000000001,"15min":0.82799999999999996},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.53300000000000003,"15min":0.53400000000000003},"min":{"1min":0.38400000000000001,"5min":0.316,"15min":0.316},"max":{"1min":0.77400000000000002,"5min":0.78100000000000003,"15min":2.0739999999999998},"last":0.38400000000000001}]},{"osd":2,"last update":"Sat May 20 01:27:37 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.55300000000000005,"15min":0.52200000000000002},"min":{"1min":0.30499999999999999,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.73699999999999999,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.56499999999999995,"15min":0.55300000000000005},"min":{"1min":0.38400000000000001,"5min":0.35799999999999998,"15min":0.34000000000000002},"max":{"1min":0.80300000000000005,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.44900000000000001}]},{"osd":3,"last update":"Sat May 20 01:27:37 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.55000000000000004,"15min":0.53100000000000003},"min":{"1min":0.36099999999999999,"5min":0.34899999999999998,"15min":0.34399999999999997},"max":{"1min":0.76900000000000002,"5min":0.81399999999999995,"15min":1.827},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.58999999999999997,"15min":0.629},"min":{"1min":0.45900000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.77200000000000002,"5min":1.093,"15min":3.1749999999999998},"last":0.56000000000000005}]},{"osd":5,"last update":"Sat May 20 01:27:37 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51600000000000001,"15min":0.49399999999999999},"min":{"1min":0.32500000000000001,"5min":0.308,"15min":0.29199999999999998},"max":{"1min":0.73499999999999999,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.51700000000000002},"min":{"1min":0.33800000000000002,"5min":0.33400000000000002,"15min":0.29899999999999999},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.40999999999999998}]},{"osd":6,"last update":"Sat May 20 01:27:37 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.57199999999999995,"15min":0.55200000000000005},"min":{"1min":0.376,"5min":0.33000000000000002,"15min":0.32400000000000001},"max":{"1min":0.871,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60499999999999998,"15min":0.59399999999999997},"min":{"1min":0.41199999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.76000000000000001,"5min":0.84699999999999998,"15min":0.85899999999999999},"last":0.57699999999999996}]},{"osd":7,"last update":"Sat May 20 01:27:37 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.56000000000000005},"min":{"1min":0.39800000000000002,"5min":0.36099999999999999,"15min":0.34200000000000003},"max":{"1min":0.79300000000000004,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61499999999999999,"15min":0.59899999999999998},"min":{"1min":0.45100000000000001,"5min":0.38100000000000001,"15min":0.34000000000000002},"max":{"1min":0.82899999999999996,"5min":1.034,"15min":1.034},"last":0.434}]}]},{"osd":5,"up_from":13,"seq":55834574998,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":16244,"kb_used_data":1644,"kb_used_omap":0,"kb_used_meta":14592,"kb_avail":94355596,"statfs":{"total":96636764160,"available":96620130304,"internally_reserved":0,"allocated":1683456,"data_stored":869618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":14942208},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat May 20 01:27:42 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.48899999999999999,"15min":0.59299999999999997},"min":{"1min":0.36199999999999999,"5min":0.29999999999999999,"15min":0.23300000000000001},"max":{"1min":0.70199999999999996,"5min":0.79000000000000004,"15min":6.3929999999999998},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.498,"15min":0.61299999999999999},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.26900000000000002},"max":{"1min":0.76200000000000001,"5min":0.83699999999999997,"15min":6.3339999999999996},"last":0.501}]},{"osd":1,"last update":"Sat May 20 01:27:42 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.49299999999999999,"15min":0.55300000000000005},"min":{"1min":0.33900000000000002,"5min":0.313,"15min":0.29299999999999998},"max":{"1min":0.65300000000000002,"5min":0.72299999999999998,"15min":3.2450000000000001},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.53100000000000003,"15min":0.53400000000000003},"min":{"1min":0.36799999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.69399999999999995,"5min":0.84599999999999997,"15min":1.1819999999999999},"last":0.70599999999999996}]},{"osd":2,"last update":"Sat May 20 01:27:42 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.51700000000000002,"15min":0.53100000000000003},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.30399999999999999},"max":{"1min":0.72399999999999998,"5min":0.84799999999999998,"15min":1.1639999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.52700000000000002,"15min":0.53800000000000003},"min":{"1min":0.39600000000000002,"5min":0.308,"15min":0.29399999999999998},"max":{"1min":0.63,"5min":0.78600000000000003,"15min":1.258},"last":0.52600000000000002}]},{"osd":3,"last update":"Sat May 20 01:27:42 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.55700000000000005,"15min":0.65900000000000003},"min":{"1min":0.44500000000000001,"5min":0.35099999999999998,"15min":0.33000000000000002},"max":{"1min":0.71799999999999997,"5min":0.91000000000000003,"15min":6.351},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.56999999999999995,"15min":0.68500000000000005},"min":{"1min":0.41799999999999998,"5min":0.374,"15min":0.33400000000000002},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":6.3730000000000002},"last":0.65200000000000002}]},{"osd":4,"last update":"Sat May 20 01:27:42 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.51500000000000001,"15min":0.497},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.62},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.55800000000000005,"15min":0.54900000000000004},"min":{"1min":0.36799999999999999,"5min":0.35699999999999998,"15min":0.34200000000000003},"max":{"1min":0.77400000000000002,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.66600000000000004}]},{"osd":6,"last update":"Sat May 20 01:27:42 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.55000000000000004,"15min":0.54600000000000004},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.55100000000000005,"15min":0.55400000000000005},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.77300000000000002,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.63800000000000001}]},{"osd":7,"last update":"Sat May 20 01:27:42 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.58099999999999996,"15min":0.57799999999999996},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33000000000000002},"max":{"1min":0.83399999999999996,"5min":0.83999999999999997,"15min":0.91000000000000003},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.60199999999999998,"15min":0.60299999999999998},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.872,"5min":0.89400000000000002,"15min":0.94499999999999995},"last":0.69399999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-05-20T01:28:31.983 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-05-20T01:28:31.983 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-05-20T01:28:32.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.143+0000 7f1037b96700 1 Processor -- start 2023-05-20T01:28:32.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.143+0000 7f1037b96700 1 -- start start 2023-05-20T01:28:32.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.144+0000 7f1037b96700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f103015cd50 0x7f10300d20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:32.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.144+0000 7f1037b96700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f10300ce030 0x7f10300d0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:32.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.144+0000 7f1037b96700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10300d1c10 con 0x7f10300cfcb0 2023-05-20T01:28:32.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.144+0000 7f1037b96700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10300d0660 con 0x7f10300ce030 2023-05-20T01:28:32.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.144+0000 7f1037b96700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f103015d120 con 0x7f103015cd50 2023-05-20T01:28:32.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.144+0000 7f1036133700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f10300ce030 0x7f10300d0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:32.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.144+0000 7f1035131700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f10300cfcb0 0x7f10300cd920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51680/0 (socket says 172.21.15.16:51680) 2023-05-20T01:28:32.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.144+0000 7f1035131700 1 -- 172.21.15.16:0/1203297404 learned_addr learned my addr 172.21.15.16:0/1203297404 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:28:32.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.144+0000 7f1035932700 1 --2- 172.21.15.16:0/1203297404 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f103015cd50 0x7f10300d20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:32.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.144+0000 7f1036133700 1 -- 172.21.15.16:0/1203297404 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f103015cd50 msgr2=0x7f10300d20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:32.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.144+0000 7f1036133700 1 --2- 172.21.15.16:0/1203297404 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f103015cd50 0x7f10300d20a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:32.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.144+0000 7f1036133700 1 -- 172.21.15.16:0/1203297404 >> v1:172.21.15.16:6789/0 conn(0x7f10300cfcb0 legacy=0x7f10300cd920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:32.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.144+0000 7f1036133700 1 -- 172.21.15.16:0/1203297404 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10300d12e0 con 0x7f10300ce030 2023-05-20T01:28:32.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.144+0000 7f1034930700 1 -- 172.21.15.16:0/1203297404 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3270111568 0 0) 0x7f10300d1c10 con 0x7f10300cfcb0 2023-05-20T01:28:32.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.144+0000 7f1036133700 1 --2- 172.21.15.16:0/1203297404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10300ce030 0x7f10300d0120 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f1028009d90 tx=0x7f102800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=677a2dcd6ec3184a server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:32.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.144+0000 7f1034930700 1 -- 172.21.15.16:0/1203297404 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f102800e070 con 0x7f10300ce030 2023-05-20T01:28:32.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.144+0000 7f1034930700 1 -- 172.21.15.16:0/1203297404 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f102800bb20 con 0x7f10300ce030 2023-05-20T01:28:32.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.144+0000 7f1034930700 1 -- 172.21.15.16:0/1203297404 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1028012430 con 0x7f10300ce030 2023-05-20T01:28:32.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.145+0000 7f1037b96700 1 -- 172.21.15.16:0/1203297404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10300ce030 msgr2=0x7f10300d0120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:32.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.145+0000 7f1037b96700 1 --2- 172.21.15.16:0/1203297404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10300ce030 0x7f10300d0120 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f1028009d90 tx=0x7f102800b330 comp rx=0 tx=0).stop 2023-05-20T01:28:32.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.145+0000 7f1037b96700 1 -- 172.21.15.16:0/1203297404 shutdown_connections 2023-05-20T01:28:32.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.145+0000 7f1037b96700 1 --2- 172.21.15.16:0/1203297404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10300ce030 0x7f10300d0120 unknown :-1 s=CLOSED pgs=1631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:32.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.145+0000 7f1037b96700 1 --2- 172.21.15.16:0/1203297404 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f103015cd50 0x7f10300d20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:32.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.145+0000 7f1037b96700 1 -- 172.21.15.16:0/1203297404 >> 172.21.15.16:0/1203297404 conn(0x7f103006b070 msgr2=0x7f103006b470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:32.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.145+0000 7f1037b96700 1 -- 172.21.15.16:0/1203297404 shutdown_connections 2023-05-20T01:28:32.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.145+0000 7f1037b96700 1 -- 172.21.15.16:0/1203297404 wait complete. 2023-05-20T01:28:32.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.147+0000 7f1037b96700 1 Processor -- start 2023-05-20T01:28:32.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.147+0000 7f1037b96700 1 -- start start 2023-05-20T01:28:32.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.148+0000 7f1037b96700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f10300ce030 0x7f103015ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:32.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.148+0000 7f1037b96700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10300cfcb0 0x7f103015a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:32.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.148+0000 7f1037b96700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f103015cd50 0x7f103015aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:32.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.148+0000 7f1037b96700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f103015b560 con 0x7f10300cfcb0 2023-05-20T01:28:32.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.148+0000 7f1037b96700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f103015b840 con 0x7f10300ce030 2023-05-20T01:28:32.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.148+0000 7f1037b96700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f103015bb20 con 0x7f103015cd50 2023-05-20T01:28:32.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.148+0000 7f1035131700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10300cfcb0 0x7f103015a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:32.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.148+0000 7f1035131700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10300cfcb0 0x7f103015a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40158/0 (socket says 172.21.15.16:40158) 2023-05-20T01:28:32.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.148+0000 7f1035131700 1 -- 172.21.15.16:0/2436656110 learned_addr learned my addr 172.21.15.16:0/2436656110 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:32.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.148+0000 7f1035932700 1 --2- 172.21.15.16:0/2436656110 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f10300ce030 0x7f103015ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:32.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.148+0000 7f1036133700 1 --2- 172.21.15.16:0/2436656110 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f103015cd50 0x7f103015aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:32.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.148+0000 7f1035131700 1 -- 172.21.15.16:0/2436656110 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f103015cd50 msgr2=0x7f103015aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:32.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.148+0000 7f1035131700 1 --2- 172.21.15.16:0/2436656110 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f103015cd50 0x7f103015aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:32.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.148+0000 7f1035131700 1 -- 172.21.15.16:0/2436656110 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f10300ce030 msgr2=0x7f103015ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:32.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.148+0000 7f1035131700 1 --2- 172.21.15.16:0/2436656110 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f10300ce030 0x7f103015ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:32.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.148+0000 7f1035131700 1 -- 172.21.15.16:0/2436656110 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1028009a70 con 0x7f10300cfcb0 2023-05-20T01:28:32.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.148+0000 7f1035932700 1 --2- 172.21.15.16:0/2436656110 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f10300ce030 0x7f103015ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:28:32.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.148+0000 7f1035131700 1 --2- 172.21.15.16:0/2436656110 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10300cfcb0 0x7f103015a9a0 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f10200011e0 tx=0x7f102000cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:32.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.148+0000 7f10267fc700 1 -- 172.21.15.16:0/2436656110 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1020004c80 con 0x7f10300cfcb0 2023-05-20T01:28:32.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.148+0000 7f10267fc700 1 -- 172.21.15.16:0/2436656110 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1020004e00 con 0x7f10300cfcb0 2023-05-20T01:28:32.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.148+0000 7f1037b96700 1 -- 172.21.15.16:0/2436656110 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10300d18a0 con 0x7f10300cfcb0 2023-05-20T01:28:32.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.148+0000 7f10267fc700 1 -- 172.21.15.16:0/2436656110 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1020007310 con 0x7f10300cfcb0 2023-05-20T01:28:32.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.148+0000 7f1037b96700 1 -- 172.21.15.16:0/2436656110 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10300d15c0 con 0x7f10300cfcb0 2023-05-20T01:28:32.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.149+0000 7f1037b96700 1 -- 172.21.15.16:0/2436656110 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1030161680 con 0x7f10300cfcb0 2023-05-20T01:28:32.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.150+0000 7f10267fc700 1 -- 172.21.15.16:0/2436656110 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1020007470 con 0x7f10300cfcb0 2023-05-20T01:28:32.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.150+0000 7f10267fc700 1 --2- 172.21.15.16:0/2436656110 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f101406ace0 0x7f101406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:32.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.150+0000 7f10267fc700 1 -- 172.21.15.16:0/2436656110 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f1020089400 con 0x7f10300cfcb0 2023-05-20T01:28:32.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.150+0000 7f1035932700 1 --2- 172.21.15.16:0/2436656110 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f101406ace0 0x7f101406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:32.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.151+0000 7f1035932700 1 --2- 172.21.15.16:0/2436656110 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f101406ace0 0x7f101406d190 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f102c009b70 tx=0x7f102c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:32.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.155+0000 7f10267fc700 1 -- 172.21.15.16:0/2436656110 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f10200542d0 con 0x7f10300cfcb0 2023-05-20T01:28:32.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.433+0000 7f1037b96700 1 -- 172.21.15.16:0/2436656110 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f10300d0aa0 con 0x7f10300cfcb0 2023-05-20T01:28:32.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.437+0000 7f10267fc700 1 -- 172.21.15.16:0/2436656110 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) v1 ==== 190+0+0 (secure 0 0 0) 0x7f10200578f0 con 0x7f10300cfcb0 2023-05-20T01:28:32.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.440+0000 7f1037b96700 1 -- 172.21.15.16:0/2436656110 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f101406ace0 msgr2=0x7f101406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:32.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.440+0000 7f1037b96700 1 --2- 172.21.15.16:0/2436656110 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f101406ace0 0x7f101406d190 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f102c009b70 tx=0x7f102c008040 comp rx=0 tx=0).stop 2023-05-20T01:28:32.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.440+0000 7f1037b96700 1 -- 172.21.15.16:0/2436656110 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10300cfcb0 msgr2=0x7f103015a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:32.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.440+0000 7f1037b96700 1 --2- 172.21.15.16:0/2436656110 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10300cfcb0 0x7f103015a9a0 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f10200011e0 tx=0x7f102000cae0 comp rx=0 tx=0).stop 2023-05-20T01:28:32.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.440+0000 7f1037b96700 1 -- 172.21.15.16:0/2436656110 shutdown_connections 2023-05-20T01:28:32.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.440+0000 7f1037b96700 1 --2- 172.21.15.16:0/2436656110 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f101406ace0 0x7f101406d190 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:32.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.440+0000 7f1037b96700 1 --2- 172.21.15.16:0/2436656110 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f103015cd50 0x7f103015aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:32.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.440+0000 7f1037b96700 1 --2- 172.21.15.16:0/2436656110 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f10300ce030 0x7f103015ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:32.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.440+0000 7f1037b96700 1 --2- 172.21.15.16:0/2436656110 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10300cfcb0 0x7f103015a9a0 unknown :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:32.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.440+0000 7f1037b96700 1 -- 172.21.15.16:0/2436656110 >> 172.21.15.16:0/2436656110 conn(0x7f103006b070 msgr2=0x7f103015fae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:32.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.440+0000 7f1037b96700 1 -- 172.21.15.16:0/2436656110 shutdown_connections 2023-05-20T01:28:32.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.440+0000 7f1037b96700 1 -- 172.21.15.16:0/2436656110 wait complete. 2023-05-20T01:28:32.446 INFO:teuthology.orchestra.run.smithi016.stderr:updated caps for client.0 2023-05-20T01:28:32.457 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-20T01:28:32.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.577+0000 7f716fcf4700 1 Processor -- start 2023-05-20T01:28:32.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.577+0000 7f716fcf4700 1 -- start start 2023-05-20T01:28:32.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.578+0000 7f716fcf4700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7168144020 0x7f71680d1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:32.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.578+0000 7f716fcf4700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71680cc500 con 0x7f71680cec60 2023-05-20T01:28:32.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.578+0000 7f716fcf4700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71680cb8f0 con 0x7f71680b2760 2023-05-20T01:28:32.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.578+0000 7f716fcf4700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71680cb6e0 con 0x7f7168144020 2023-05-20T01:28:32.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.578+0000 7f716d28f700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7168144020 0x7f71680d1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:32.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.578+0000 7f716e291700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f71680cec60 0x7f71680cc990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51696/0 (socket says 172.21.15.16:51696) 2023-05-20T01:28:32.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.578+0000 7f716e291700 1 -- 172.21.15.16:0/834814714 learned_addr learned my addr 172.21.15.16:0/834814714 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:28:32.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.579+0000 7f716d28f700 1 -- 172.21.15.16:0/834814714 >> v1:172.21.15.132:6789/0 conn(0x7f71680b2760 legacy=0x7f71680b0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:32.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.579+0000 7f716d28f700 1 -- 172.21.15.16:0/834814714 >> v1:172.21.15.16:6789/0 conn(0x7f71680cec60 legacy=0x7f71680cc990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:32.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.579+0000 7f716d28f700 1 -- 172.21.15.16:0/834814714 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71680cbbd0 con 0x7f7168144020 2023-05-20T01:28:32.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.579+0000 7f716ca8e700 1 -- 172.21.15.16:0/834814714 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 146256195 0 0) 0x7f71680cc500 con 0x7f71680cec60 2023-05-20T01:28:32.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.579+0000 7f716d28f700 1 --2- 172.21.15.16:0/834814714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7168144020 0x7f71680d1680 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f7164009840 tx=0x7f7164009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82b492fe2f6cc909 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:32.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.579+0000 7f716ca8e700 1 -- 172.21.15.16:0/834814714 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7164011830 con 0x7f7168144020 2023-05-20T01:28:32.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.579+0000 7f716ca8e700 1 -- 172.21.15.16:0/834814714 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7164011990 con 0x7f7168144020 2023-05-20T01:28:32.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.579+0000 7f716ca8e700 1 -- 172.21.15.16:0/834814714 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7164011d20 con 0x7f7168144020 2023-05-20T01:28:32.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.579+0000 7f716fcf4700 1 -- 172.21.15.16:0/834814714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7168144020 msgr2=0x7f71680d1680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:32.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.579+0000 7f716fcf4700 1 --2- 172.21.15.16:0/834814714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7168144020 0x7f71680d1680 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f7164009840 tx=0x7f7164009b50 comp rx=0 tx=0).stop 2023-05-20T01:28:32.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.579+0000 7f716fcf4700 1 -- 172.21.15.16:0/834814714 shutdown_connections 2023-05-20T01:28:32.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.579+0000 7f716fcf4700 1 --2- 172.21.15.16:0/834814714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7168144020 0x7f71680d1680 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:32.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.579+0000 7f716fcf4700 1 -- 172.21.15.16:0/834814714 >> 172.21.15.16:0/834814714 conn(0x7f716806ae30 msgr2=0x7f716809a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:32.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.580+0000 7f716fcf4700 1 -- 172.21.15.16:0/834814714 shutdown_connections 2023-05-20T01:28:32.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.580+0000 7f716fcf4700 1 -- 172.21.15.16:0/834814714 wait complete. 2023-05-20T01:28:32.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.581+0000 7f716fcf4700 1 Processor -- start 2023-05-20T01:28:32.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.581+0000 7f716fcf4700 1 -- start start 2023-05-20T01:28:32.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.581+0000 7f716fcf4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71680b2760 0x7f71680cddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:32.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.581+0000 7f716fcf4700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f71680cec60 0x7f716806d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:32.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.581+0000 7f716fcf4700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7168144020 0x7f71680b57e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:32.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.581+0000 7f716fcf4700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7168158e00 con 0x7f71680b2760 2023-05-20T01:28:32.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.581+0000 7f716fcf4700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71680ce330 con 0x7f71680cec60 2023-05-20T01:28:32.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.581+0000 7f716fcf4700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f716806c750 con 0x7f7168144020 2023-05-20T01:28:32.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.582+0000 7f716da90700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71680b2760 0x7f71680cddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:32.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.582+0000 7f716da90700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71680b2760 0x7f71680cddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40170/0 (socket says 172.21.15.16:40170) 2023-05-20T01:28:32.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.582+0000 7f716da90700 1 -- 172.21.15.16:0/2458174318 learned_addr learned my addr 172.21.15.16:0/2458174318 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:32.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.582+0000 7f716d28f700 1 --2- 172.21.15.16:0/2458174318 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f71680cec60 0x7f716806d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:32.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.582+0000 7f716e291700 1 --2- 172.21.15.16:0/2458174318 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7168144020 0x7f71680b57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:32.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.582+0000 7f716da90700 1 -- 172.21.15.16:0/2458174318 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7168144020 msgr2=0x7f71680b57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:32.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.582+0000 7f716da90700 1 --2- 172.21.15.16:0/2458174318 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7168144020 0x7f71680b57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:32.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.582+0000 7f716da90700 1 -- 172.21.15.16:0/2458174318 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f71680cec60 msgr2=0x7f716806d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:32.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.582+0000 7f716da90700 1 --2- 172.21.15.16:0/2458174318 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f71680cec60 0x7f716806d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:32.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.582+0000 7f716da90700 1 -- 172.21.15.16:0/2458174318 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71640094a0 con 0x7f71680b2760 2023-05-20T01:28:32.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.582+0000 7f716e291700 1 --2- 172.21.15.16:0/2458174318 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7168144020 0x7f71680b57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:28:32.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.582+0000 7f716d28f700 1 --2- 172.21.15.16:0/2458174318 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f71680cec60 0x7f716806d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:28:32.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.582+0000 7f716da90700 1 --2- 172.21.15.16:0/2458174318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71680b2760 0x7f71680cddf0 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f715800c4e0 tx=0x7f715800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:32.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.582+0000 7f715e7fc700 1 -- 172.21.15.16:0/2458174318 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71580052d0 con 0x7f71680b2760 2023-05-20T01:28:32.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.582+0000 7f715e7fc700 1 -- 172.21.15.16:0/2458174318 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7158005430 con 0x7f71680b2760 2023-05-20T01:28:32.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.582+0000 7f715e7fc700 1 -- 172.21.15.16:0/2458174318 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71580038c0 con 0x7f71680b2760 2023-05-20T01:28:32.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.582+0000 7f716fcf4700 1 -- 172.21.15.16:0/2458174318 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71680cc190 con 0x7f71680b2760 2023-05-20T01:28:32.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.582+0000 7f716fcf4700 1 -- 172.21.15.16:0/2458174318 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71680cbeb0 con 0x7f71680b2760 2023-05-20T01:28:32.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.583+0000 7f716fcf4700 1 -- 172.21.15.16:0/2458174318 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71680cf0c0 con 0x7f71680b2760 2023-05-20T01:28:32.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.584+0000 7f715e7fc700 1 -- 172.21.15.16:0/2458174318 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7158003a20 con 0x7f71680b2760 2023-05-20T01:28:32.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.584+0000 7f715e7fc700 1 --2- 172.21.15.16:0/2458174318 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f714c06ace0 0x7f714c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:32.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.584+0000 7f715e7fc700 1 -- 172.21.15.16:0/2458174318 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f7158088b10 con 0x7f71680b2760 2023-05-20T01:28:32.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.584+0000 7f716d28f700 1 --2- 172.21.15.16:0/2458174318 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f714c06ace0 0x7f714c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:32.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.585+0000 7f716d28f700 1 --2- 172.21.15.16:0/2458174318 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f714c06ace0 0x7f714c06d190 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f71640090b0 tx=0x7f716401c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:32.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.588+0000 7f715e7fc700 1 -- 172.21.15.16:0/2458174318 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f7158053b10 con 0x7f71680b2760 2023-05-20T01:28:32.666 INFO:tasks.ceph.mds.a.smithi016.stderr:2023-05-20T01:28:32.662+0000 7f90c4ecc700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-05-20T01:28:32.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.835+0000 7f716fcf4700 1 -- 172.21.15.16:0/2458174318 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f71680cf4a0 con 0x7f71680b2760 2023-05-20T01:28:32.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.836+0000 7f715e7fc700 1 -- 172.21.15.16:0/2458174318 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) v1 ==== 94+0+4679 (secure 0 0 0) 0x7f7158057130 con 0x7f71680b2760 2023-05-20T01:28:32.841 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:28:32.841 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":99,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7043,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/1859755557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1859755557},{"type":"v1","addr":"172.21.15.16:6836","nonce":1859755557}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94},{"gid":7046,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/2654783598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":2654783598},{"type":"v1","addr":"172.21.15.132:6834","nonce":2654783598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94},{"gid":7049,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2042939723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":2042939723},{"type":"v1","addr":"172.21.15.132:6835","nonce":2042939723}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-20T01:28:27.093957+0000","modified":"2023-05-20T01:28:30.436870+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8703},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8703":{"gid":8703,"name":"a","rank":0,"incarnation":95,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6837/2295684907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":2295684907},{"type":"v1","addr":"172.21.15.16:6837","nonce":2295684907}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-05-20T01:28:32.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.838+0000 7f716fcf4700 1 -- 172.21.15.16:0/2458174318 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f714c06ace0 msgr2=0x7f714c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:32.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.838+0000 7f716fcf4700 1 --2- 172.21.15.16:0/2458174318 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f714c06ace0 0x7f714c06d190 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f71640090b0 tx=0x7f716401c040 comp rx=0 tx=0).stop 2023-05-20T01:28:32.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.838+0000 7f716fcf4700 1 -- 172.21.15.16:0/2458174318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71680b2760 msgr2=0x7f71680cddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:32.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.838+0000 7f716fcf4700 1 --2- 172.21.15.16:0/2458174318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71680b2760 0x7f71680cddf0 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f715800c4e0 tx=0x7f715800c8a0 comp rx=0 tx=0).stop 2023-05-20T01:28:32.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.838+0000 7f716fcf4700 1 -- 172.21.15.16:0/2458174318 shutdown_connections 2023-05-20T01:28:32.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.838+0000 7f716fcf4700 1 --2- 172.21.15.16:0/2458174318 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f714c06ace0 0x7f714c06d190 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:32.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.838+0000 7f716fcf4700 1 --2- 172.21.15.16:0/2458174318 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7168144020 0x7f71680b57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:32.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.838+0000 7f716fcf4700 1 --2- 172.21.15.16:0/2458174318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71680b2760 0x7f71680cddf0 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:32.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.838+0000 7f716fcf4700 1 --2- 172.21.15.16:0/2458174318 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f71680cec60 0x7f716806d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:32.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.838+0000 7f716fcf4700 1 -- 172.21.15.16:0/2458174318 >> 172.21.15.16:0/2458174318 conn(0x7f716806ae30 msgr2=0x7f71680af6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:32.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.838+0000 7f716fcf4700 1 -- 172.21.15.16:0/2458174318 shutdown_connections 2023-05-20T01:28:32.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:32.838+0000 7f716fcf4700 1 -- 172.21.15.16:0/2458174318 wait complete. 2023-05-20T01:28:32.844 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 99 2023-05-20T01:28:33.258 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 99, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-05-20T01:28:27.093957+0000', 'modified': '2023-05-20T01:28:30.436870+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8703}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8703': {'gid': 8703, 'name': 'a', 'rank': 0, 'incarnation': 95, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.16:6837/2295684907', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6835', 'nonce': 2295684907}, {'type': 'v1', 'addr': '172.21.15.16:6837', 'nonce': 2295684907}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [19], 'metadata_pool': 18, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-05-20T01:28:33.258 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-05-20T01:28:33.259 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-05-20T01:28:33.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.384+0000 7f84e5baf700 1 Processor -- start 2023-05-20T01:28:33.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.384+0000 7f84e5baf700 1 -- start start 2023-05-20T01:28:33.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.385+0000 7f84e5baf700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f84e015caf0 0x7f84e006cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:33.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.385+0000 7f84e5baf700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f84e00b1e00 0x7f84e006d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:33.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.385+0000 7f84e5baf700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84e015cec0 con 0x7f84e00ccf00 2023-05-20T01:28:33.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.385+0000 7f84e5baf700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84e00d0490 con 0x7f84e015caf0 2023-05-20T01:28:33.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.385+0000 7f84e5baf700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84e0160610 con 0x7f84e00b1e00 2023-05-20T01:28:33.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.385+0000 7f84df7fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f84e015caf0 0x7f84e006cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:33.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.385+0000 7f84deffd700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f84e00ccf00 0x7f84e00c8aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51708/0 (socket says 172.21.15.16:51708) 2023-05-20T01:28:33.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.385+0000 7f84deffd700 1 -- 172.21.15.16:0/2481668579 learned_addr learned my addr 172.21.15.16:0/2481668579 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:28:33.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.385+0000 7f84dffff700 1 --2- 172.21.15.16:0/2481668579 >> v2:172.21.15.132:3300/0 conn(0x7f84e00b1e00 0x7f84e006d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:33.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.385+0000 7f84df7fe700 1 -- 172.21.15.16:0/2481668579 >> v2:172.21.15.132:3300/0 conn(0x7f84e00b1e00 msgr2=0x7f84e006d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:33.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.385+0000 7f84df7fe700 1 --2- 172.21.15.16:0/2481668579 >> v2:172.21.15.132:3300/0 conn(0x7f84e00b1e00 0x7f84e006d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:33.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.385+0000 7f84df7fe700 1 -- 172.21.15.16:0/2481668579 >> v1:172.21.15.16:6789/0 conn(0x7f84e00ccf00 legacy=0x7f84e00c8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:33.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.385+0000 7f84df7fe700 1 -- 172.21.15.16:0/2481668579 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84e01608f0 con 0x7f84e015caf0 2023-05-20T01:28:33.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.385+0000 7f84de7fc700 1 -- 172.21.15.16:0/2481668579 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 924550001 0 0) 0x7f84e015cec0 con 0x7f84e00ccf00 2023-05-20T01:28:33.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.385+0000 7f84df7fe700 1 --2- 172.21.15.16:0/2481668579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84e015caf0 0x7f84e006cb90 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f84e0070760 tx=0x7f84d0009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8da42b197384deb4 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:33.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.385+0000 7f84de7fc700 1 -- 172.21.15.16:0/2481668579 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84d0011830 con 0x7f84e015caf0 2023-05-20T01:28:33.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.385+0000 7f84de7fc700 1 -- 172.21.15.16:0/2481668579 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84d0011990 con 0x7f84e015caf0 2023-05-20T01:28:33.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.385+0000 7f84de7fc700 1 -- 172.21.15.16:0/2481668579 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84d0011d20 con 0x7f84e015caf0 2023-05-20T01:28:33.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.386+0000 7f84e5baf700 1 -- 172.21.15.16:0/2481668579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84e015caf0 msgr2=0x7f84e006cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:33.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.386+0000 7f84e5baf700 1 --2- 172.21.15.16:0/2481668579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84e015caf0 0x7f84e006cb90 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f84e0070760 tx=0x7f84d0009b20 comp rx=0 tx=0).stop 2023-05-20T01:28:33.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.386+0000 7f84e5baf700 1 -- 172.21.15.16:0/2481668579 shutdown_connections 2023-05-20T01:28:33.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.386+0000 7f84e5baf700 1 --2- 172.21.15.16:0/2481668579 >> v2:172.21.15.132:3300/0 conn(0x7f84e00b1e00 0x7f84e006d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:33.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.386+0000 7f84e5baf700 1 --2- 172.21.15.16:0/2481668579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84e015caf0 0x7f84e006cb90 unknown :-1 s=CLOSED pgs=1637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:33.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.386+0000 7f84e5baf700 1 -- 172.21.15.16:0/2481668579 >> 172.21.15.16:0/2481668579 conn(0x7f84e00661b0 msgr2=0x7f84e0066580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:33.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.386+0000 7f84e5baf700 1 -- 172.21.15.16:0/2481668579 shutdown_connections 2023-05-20T01:28:33.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.386+0000 7f84e5baf700 1 -- 172.21.15.16:0/2481668579 wait complete. 2023-05-20T01:28:33.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.387+0000 7f84e5baf700 1 Processor -- start 2023-05-20T01:28:33.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.387+0000 7f84e5baf700 1 -- start start 2023-05-20T01:28:33.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.388+0000 7f84e5baf700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84e00b1e00 0x7f84e0156490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:33.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.388+0000 7f84e5baf700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f84e00ccf00 0x7f84e0153f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:33.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.388+0000 7f84e5baf700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f84e015caf0 0x7f84e01e60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:33.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.388+0000 7f84e5baf700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84e01544b0 con 0x7f84e00b1e00 2023-05-20T01:28:33.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.388+0000 7f84e5baf700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84e013fd20 con 0x7f84e015caf0 2023-05-20T01:28:33.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.388+0000 7f84e5baf700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84e0154c40 con 0x7f84e00ccf00 2023-05-20T01:28:33.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.388+0000 7f84df7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84e00b1e00 0x7f84e0156490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:33.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.388+0000 7f84df7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84e00b1e00 0x7f84e0156490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40184/0 (socket says 172.21.15.16:40184) 2023-05-20T01:28:33.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.388+0000 7f84df7fe700 1 -- 172.21.15.16:0/1418481781 learned_addr learned my addr 172.21.15.16:0/1418481781 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:33.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.388+0000 7f84dffff700 1 --2- 172.21.15.16:0/1418481781 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f84e015caf0 0x7f84e01e60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:33.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.388+0000 7f84deffd700 1 --2- 172.21.15.16:0/1418481781 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f84e00ccf00 0x7f84e0153f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:33.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.389+0000 7f84df7fe700 1 -- 172.21.15.16:0/1418481781 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f84e00ccf00 msgr2=0x7f84e0153f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:33.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.389+0000 7f84df7fe700 1 --2- 172.21.15.16:0/1418481781 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f84e00ccf00 0x7f84e0153f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:33.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.389+0000 7f84df7fe700 1 -- 172.21.15.16:0/1418481781 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f84e015caf0 msgr2=0x7f84e01e60e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:33.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.389+0000 7f84df7fe700 1 --2- 172.21.15.16:0/1418481781 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f84e015caf0 0x7f84e01e60e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:33.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.389+0000 7f84df7fe700 1 -- 172.21.15.16:0/1418481781 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84d00094a0 con 0x7f84e00b1e00 2023-05-20T01:28:33.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.389+0000 7f84deffd700 1 --2- 172.21.15.16:0/1418481781 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f84e00ccf00 0x7f84e0153f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:28:33.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.389+0000 7f84df7fe700 1 --2- 172.21.15.16:0/1418481781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84e00b1e00 0x7f84e0156490 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f84d00090b0 tx=0x7f84d0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:33.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.389+0000 7f84c7fff700 1 -- 172.21.15.16:0/1418481781 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84d00036a0 con 0x7f84e00b1e00 2023-05-20T01:28:33.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.389+0000 7f84c7fff700 1 -- 172.21.15.16:0/1418481781 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84d001e070 con 0x7f84e00b1e00 2023-05-20T01:28:33.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.389+0000 7f84e5baf700 1 -- 172.21.15.16:0/1418481781 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84e0160eb0 con 0x7f84e00b1e00 2023-05-20T01:28:33.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.389+0000 7f84c7fff700 1 -- 172.21.15.16:0/1418481781 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84d0003a00 con 0x7f84e00b1e00 2023-05-20T01:28:33.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.389+0000 7f84e5baf700 1 -- 172.21.15.16:0/1418481781 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84e0160bd0 con 0x7f84e00b1e00 2023-05-20T01:28:33.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.390+0000 7f84c7fff700 1 -- 172.21.15.16:0/1418481781 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f84d0003b60 con 0x7f84e00b1e00 2023-05-20T01:28:33.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.391+0000 7f84c7fff700 1 --2- 172.21.15.16:0/1418481781 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f84cc06ace0 0x7f84cc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:33.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.391+0000 7f84c7fff700 1 -- 172.21.15.16:0/1418481781 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f84d008a680 con 0x7f84e00b1e00 2023-05-20T01:28:33.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.391+0000 7f84deffd700 1 --2- 172.21.15.16:0/1418481781 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f84cc06ace0 0x7f84cc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:33.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.391+0000 7f84deffd700 1 --2- 172.21.15.16:0/1418481781 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f84cc06ace0 0x7f84cc06d190 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f84d4000f60 tx=0x7f84d4007ca0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:33.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.393+0000 7f84f1d3bb80 1 Processor -- start 2023-05-20T01:28:33.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.393+0000 7f84f1d3bb80 1 -- start start 2023-05-20T01:28:33.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.394+0000 7f84f1d3bb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee6c726400 0x55ee6c7267d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:33.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.394+0000 7f84f1d3bb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ee6c722700 0x55ee6c6fb460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:33.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.394+0000 7f84f1d3bb80 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ee6c6f9870 0x55ee6c6fb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:33.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.394+0000 7f84f1d3bb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee6c601010 con 0x55ee6c726400 2023-05-20T01:28:33.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.394+0000 7f84f1d3bb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee6c653850 con 0x55ee6c722700 2023-05-20T01:28:33.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.394+0000 7f84f1d3bb80 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ee6c64cce0 con 0x55ee6c6f9870 2023-05-20T01:28:33.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.394+0000 7f84df7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee6c726400 0x55ee6c7267d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:33.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.394+0000 7f84df7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee6c726400 0x55ee6c7267d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40192/0 (socket says 172.21.15.16:40192) 2023-05-20T01:28:33.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.394+0000 7f84df7fe700 1 -- 172.21.15.16:0/2817732954 learned_addr learned my addr 172.21.15.16:0/2817732954 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:33.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.394+0000 7f84dffff700 1 --2- 172.21.15.16:0/2817732954 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ee6c6f9870 0x55ee6c6fb9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:33.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.394+0000 7f84deffd700 1 --2- 172.21.15.16:0/2817732954 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ee6c722700 0x55ee6c6fb460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:33.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.394+0000 7f84df7fe700 1 -- 172.21.15.16:0/2817732954 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ee6c6f9870 msgr2=0x55ee6c6fb9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:33.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.394+0000 7f84df7fe700 1 --2- 172.21.15.16:0/2817732954 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ee6c6f9870 0x55ee6c6fb9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:33.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.394+0000 7f84df7fe700 1 -- 172.21.15.16:0/2817732954 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ee6c722700 msgr2=0x55ee6c6fb460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:33.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.394+0000 7f84df7fe700 1 --2- 172.21.15.16:0/2817732954 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ee6c722700 0x55ee6c6fb460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:33.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.394+0000 7f84df7fe700 1 -- 172.21.15.16:0/2817732954 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ee6c4ce5e0 con 0x55ee6c726400 2023-05-20T01:28:33.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.394+0000 7f84df7fe700 1 --2- 172.21.15.16:0/2817732954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee6c726400 0x55ee6c7267d0 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f84d00888e0 tx=0x7f84d00557b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e20b8f5741b4ae26 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:33.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.394+0000 7f84c57fa700 1 -- 172.21.15.16:0/2817732954 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84d005d300 con 0x55ee6c726400 2023-05-20T01:28:33.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.395+0000 7f84c57fa700 1 -- 172.21.15.16:0/2817732954 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84d0061690 con 0x55ee6c726400 2023-05-20T01:28:33.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.395+0000 7f84c57fa700 1 -- 172.21.15.16:0/2817732954 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84d0003e10 con 0x55ee6c726400 2023-05-20T01:28:33.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.395+0000 7f84f1d3bb80 1 -- 172.21.15.16:0/2817732954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee6c726400 msgr2=0x55ee6c7267d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:33.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.395+0000 7f84f1d3bb80 1 --2- 172.21.15.16:0/2817732954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee6c726400 0x55ee6c7267d0 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f84d00888e0 tx=0x7f84d00557b0 comp rx=0 tx=0).stop 2023-05-20T01:28:33.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.395+0000 7f84f1d3bb80 1 -- 172.21.15.16:0/2817732954 shutdown_connections 2023-05-20T01:28:33.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.395+0000 7f84f1d3bb80 1 --2- 172.21.15.16:0/2817732954 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ee6c6f9870 0x55ee6c6fb9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:33.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.395+0000 7f84f1d3bb80 1 --2- 172.21.15.16:0/2817732954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee6c726400 0x55ee6c7267d0 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:33.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.395+0000 7f84f1d3bb80 1 --2- 172.21.15.16:0/2817732954 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ee6c722700 0x55ee6c6fb460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:33.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.395+0000 7f84f1d3bb80 1 -- 172.21.15.16:0/2817732954 >> 172.21.15.16:0/2817732954 conn(0x55ee6c6f0f30 msgr2=0x55ee6c741210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:33.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.395+0000 7f84f1d3bb80 1 -- 172.21.15.16:0/2817732954 shutdown_connections 2023-05-20T01:28:33.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.395+0000 7f84f1d3bb80 1 -- 172.21.15.16:0/2817732954 wait complete. 2023-05-20T01:28:33.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.396+0000 7f84f1d3bb80 1 -- start start 2023-05-20T01:28:33.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.396+0000 7f84f1d3bb80 1 Processor -- start 2023-05-20T01:28:33.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.397+0000 7f84f1d3bb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ee6c6f9870 0x55ee6c744f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:33.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.397+0000 7f84f1d3bb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee6c722700 0x55ee6c720cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:33.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.397+0000 7f84f1d3bb80 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ee6c726400 0x55ee6c721200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:33.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.397+0000 7f84f1d3bb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee6c5ee890 con 0x55ee6c722700 2023-05-20T01:28:33.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.397+0000 7f84f1d3bb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee6c5ee530 con 0x55ee6c6f9870 2023-05-20T01:28:33.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.397+0000 7f84f1d3bb80 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ee6c4d08d0 con 0x55ee6c726400 2023-05-20T01:28:33.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.397+0000 7f84f1d3bb80 10 client.0 _ll_register_callbacks cb 0x55ee6c60c730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-05-20T01:28:33.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.397+0000 7f84deffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee6c722700 0x55ee6c720cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:33.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.397+0000 7f84deffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee6c722700 0x55ee6c720cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40200/0 (socket says 172.21.15.16:40200) 2023-05-20T01:28:33.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.397+0000 7f84deffd700 1 -- 172.21.15.16:0/119232800 learned_addr learned my addr 172.21.15.16:0/119232800 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:33.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.397+0000 7f84df7fe700 1 --2- 172.21.15.16:0/119232800 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ee6c6f9870 0x55ee6c744f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:33.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.397+0000 7f84dffff700 1 --2- 172.21.15.16:0/119232800 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ee6c726400 0x55ee6c721200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:33.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.397+0000 7f84deffd700 1 -- 172.21.15.16:0/119232800 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ee6c726400 msgr2=0x55ee6c721200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:33.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.397+0000 7f84deffd700 1 --2- 172.21.15.16:0/119232800 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ee6c726400 0x55ee6c721200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:33.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.397+0000 7f84deffd700 1 -- 172.21.15.16:0/119232800 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ee6c6f9870 msgr2=0x55ee6c744f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:33.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.397+0000 7f84deffd700 1 --2- 172.21.15.16:0/119232800 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ee6c6f9870 0x55ee6c744f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:33.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.397+0000 7f84deffd700 1 -- 172.21.15.16:0/119232800 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84e0160bd0 con 0x55ee6c722700 2023-05-20T01:28:33.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.397+0000 7f84deffd700 1 --2- 172.21.15.16:0/119232800 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee6c722700 0x55ee6c720cc0 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f84d40011e0 tx=0x7f84d4011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e480f76b8be74419 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:33.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.397+0000 7f84be7fc700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-05-20T01:28:33.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.397+0000 7f84be7fc700 1 -- 172.21.15.16:0/119232800 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84d4001820 con 0x55ee6c722700 2023-05-20T01:28:33.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.397+0000 7f84f1d3bb80 1 -- 172.21.15.16:0/119232800 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ee6c6ee6f0 con 0x55ee6c722700 2023-05-20T01:28:33.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.397+0000 7f84be7fc700 1 -- 172.21.15.16:0/119232800 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(136..136 src has 1..136) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f84d4022d30 con 0x55ee6c722700 2023-05-20T01:28:33.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.398+0000 7f84be7fc700 1 -- 172.21.15.16:0/119232800 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=99) v2 ==== 24+0+0 (secure 0 0 0) 0x7f84d4001980 con 0x55ee6c722700 2023-05-20T01:28:33.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.398+0000 7f84f1d3bb80 10 client.8850 fetch_fsmap learned FSMap version 99 2023-05-20T01:28:33.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.398+0000 7f84f1d3bb80 1 -- 172.21.15.16:0/119232800 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=99,monmap=2+}) v3 -- 0x55ee6c613140 con 0x55ee6c722700 2023-05-20T01:28:33.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.398+0000 7f84be7fc700 1 -- 172.21.15.16:0/119232800 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ee6c6ee6f0 con 0x55ee6c722700 2023-05-20T01:28:33.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.398+0000 7f84be7fc700 1 -- 172.21.15.16:0/119232800 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 99) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f84d4001ae0 con 0x55ee6c722700 2023-05-20T01:28:33.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.398+0000 7f84f1d3bb80 10 client.8850 fetch_fsmap finished waiting for FSMap version 99 2023-05-20T01:28:33.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.398+0000 7f84f1d3bb80 10 client.8850 resolve_mds: resolved name 'a' to daemon mds.a 2023-05-20T01:28:33.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.398+0000 7f84f1d3bb80 20 client.8850 populate_metadata read hostname 'smithi016' 2023-05-20T01:28:33.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.398+0000 7f84f1d3bb80 1 --2- 172.21.15.16:0/119232800 >> [v2:172.21.15.16:6835/2295684907,v1:172.21.15.16:6837/2295684907] conn(0x55ee6c7405f0 0x55ee6c7409c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:33.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.398+0000 7f84f1d3bb80 4 client.8850 mds_command: new command op to 8703 tid=0[{"prefix": "get_command_descriptions"}] 2023-05-20T01:28:33.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.398+0000 7f84f1d3bb80 1 -- 172.21.15.16:0/119232800 --> [v2:172.21.15.16:6835/2295684907,v1:172.21.15.16:6837/2295684907] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ee6c60f050 con 0x55ee6c7405f0 2023-05-20T01:28:33.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.399+0000 7f84df7fe700 1 --2- 172.21.15.16:0/119232800 >> [v2:172.21.15.16:6835/2295684907,v1:172.21.15.16:6837/2295684907] conn(0x55ee6c7405f0 0x55ee6c7409c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:33.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.399+0000 7f84be7fc700 10 client.8850 ms_handle_connect on v2:172.21.15.16:6835/2295684907 2023-05-20T01:28:33.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.399+0000 7f84df7fe700 1 --2- 172.21.15.16:0/119232800 >> [v2:172.21.15.16:6835/2295684907,v1:172.21.15.16:6837/2295684907] conn(0x55ee6c7405f0 0x55ee6c7409c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f60b34ad5b35d12b server_cookie=ddd0761ccab66f66 in_seq=0 out_seq=0 2023-05-20T01:28:33.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.399+0000 7f84be7fc700 1 -- 172.21.15.16:0/119232800 <== mds.0 v2:172.21.15.16:6835/2295684907 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ee6c60f050 con 0x55ee6c7405f0 2023-05-20T01:28:33.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.399+0000 7f84be7fc700 10 client.8850 handle_command_reply: tid=0 2023-05-20T01:28:33.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.399+0000 7f84f1d3bb80 1 client.8850 shutdown 2023-05-20T01:28:33.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.399+0000 7f84f1d3bb80 1 -- 172.21.15.16:0/119232800 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee6c722700 msgr2=0x55ee6c720cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:33.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.399+0000 7f84f1d3bb80 1 --2- 172.21.15.16:0/119232800 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee6c722700 0x55ee6c720cc0 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f84d40011e0 tx=0x7f84d4011000 comp rx=0 tx=0).stop 2023-05-20T01:28:33.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.400+0000 7f84f1d3bb80 1 -- 172.21.15.16:0/119232800 shutdown_connections 2023-05-20T01:28:33.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.400+0000 7f84f1d3bb80 1 --2- 172.21.15.16:0/119232800 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ee6c726400 0x55ee6c721200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:33.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.400+0000 7f84f1d3bb80 1 --2- 172.21.15.16:0/119232800 >> [v2:172.21.15.16:6835/2295684907,v1:172.21.15.16:6837/2295684907] conn(0x55ee6c7405f0 0x55ee6c7409c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:33.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.400+0000 7f84f1d3bb80 1 --2- 172.21.15.16:0/119232800 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ee6c6f9870 0x55ee6c744f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:33.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.400+0000 7f84f1d3bb80 1 --2- 172.21.15.16:0/119232800 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee6c722700 0x55ee6c720cc0 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:33.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.400+0000 7f84be7fc700 0 client.8850 ms_handle_reset on v2:172.21.15.16:6835/2295684907 2023-05-20T01:28:33.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.400+0000 7f84f1d3bb80 1 -- 172.21.15.16:0/119232800 >> 172.21.15.16:0/119232800 conn(0x55ee6c6f0f30 msgr2=0x55ee6c6f63e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:33.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.400+0000 7f84f1d3bb80 1 -- 172.21.15.16:0/119232800 shutdown_connections 2023-05-20T01:28:33.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.400+0000 7f84f1d3bb80 1 -- 172.21.15.16:0/119232800 wait complete. 2023-05-20T01:28:33.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.400+0000 7f84f1d3bb80 20 client.8850 trim_cache size 0 max 16384 2023-05-20T01:28:33.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.418+0000 7f84f1d3bb80 1 Processor -- start 2023-05-20T01:28:33.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.418+0000 7f84f1d3bb80 1 -- start start 2023-05-20T01:28:33.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.418+0000 7f84f1d3bb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ee6c6f9870 0x55ee6c744950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:33.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.418+0000 7f84f1d3bb80 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ee6c722700 0x55ee6c744e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:33.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.418+0000 7f84f1d3bb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee6c726400 0x55ee6c6f5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:33.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.418+0000 7f84f1d3bb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee6c73fb50 con 0x55ee6c726400 2023-05-20T01:28:33.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.418+0000 7f84f1d3bb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee6c6f7310 con 0x55ee6c6f9870 2023-05-20T01:28:33.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.418+0000 7f84f1d3bb80 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ee6c6f0b00 con 0x55ee6c722700 2023-05-20T01:28:33.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.419+0000 7f84dffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee6c726400 0x55ee6c6f5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:33.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.419+0000 7f84dffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee6c726400 0x55ee6c6f5cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40208/0 (socket says 172.21.15.16:40208) 2023-05-20T01:28:33.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.419+0000 7f84dffff700 1 -- 172.21.15.16:0/3134828037 learned_addr learned my addr 172.21.15.16:0/3134828037 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:33.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.419+0000 7f84deffd700 1 --2- 172.21.15.16:0/3134828037 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ee6c722700 0x55ee6c744e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:33.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.419+0000 7f84df7fe700 1 --2- 172.21.15.16:0/3134828037 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ee6c6f9870 0x55ee6c744950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:33.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.419+0000 7f84dffff700 1 -- 172.21.15.16:0/3134828037 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ee6c722700 msgr2=0x55ee6c744e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:33.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.419+0000 7f84dffff700 1 --2- 172.21.15.16:0/3134828037 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ee6c722700 0x55ee6c744e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:33.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.419+0000 7f84dffff700 1 -- 172.21.15.16:0/3134828037 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ee6c6f9870 msgr2=0x55ee6c744950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:33.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.419+0000 7f84dffff700 1 --2- 172.21.15.16:0/3134828037 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ee6c6f9870 0x55ee6c744950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:33.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.419+0000 7f84dffff700 1 -- 172.21.15.16:0/3134828037 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ee6c612e90 con 0x55ee6c726400 2023-05-20T01:28:33.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.419+0000 7f84dffff700 1 --2- 172.21.15.16:0/3134828037 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee6c726400 0x55ee6c6f5cd0 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f84d8003b00 tx=0x7f84d8003e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1695919510ee31b server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:33.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.419+0000 7f84c5ffb700 1 -- 172.21.15.16:0/3134828037 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84d8004430 con 0x55ee6c726400 2023-05-20T01:28:33.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.419+0000 7f84c5ffb700 1 -- 172.21.15.16:0/3134828037 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84d80045b0 con 0x55ee6c726400 2023-05-20T01:28:33.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.419+0000 7f84c5ffb700 1 -- 172.21.15.16:0/3134828037 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84d8016400 con 0x55ee6c726400 2023-05-20T01:28:33.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.420+0000 7f84f1d3bb80 1 -- 172.21.15.16:0/3134828037 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee6c726400 msgr2=0x55ee6c6f5cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:33.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.420+0000 7f84f1d3bb80 1 --2- 172.21.15.16:0/3134828037 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee6c726400 0x55ee6c6f5cd0 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f84d8003b00 tx=0x7f84d8003e10 comp rx=0 tx=0).stop 2023-05-20T01:28:33.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.420+0000 7f84f1d3bb80 1 -- 172.21.15.16:0/3134828037 shutdown_connections 2023-05-20T01:28:33.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.420+0000 7f84f1d3bb80 1 --2- 172.21.15.16:0/3134828037 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee6c726400 0x55ee6c6f5cd0 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:33.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.420+0000 7f84f1d3bb80 1 --2- 172.21.15.16:0/3134828037 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ee6c6f9870 0x55ee6c744950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:33.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.420+0000 7f84f1d3bb80 1 --2- 172.21.15.16:0/3134828037 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ee6c722700 0x55ee6c744e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:33.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.420+0000 7f84f1d3bb80 1 -- 172.21.15.16:0/3134828037 >> 172.21.15.16:0/3134828037 conn(0x55ee6c6f0f30 msgr2=0x55ee6c722eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:33.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.420+0000 7f84f1d3bb80 1 -- 172.21.15.16:0/3134828037 shutdown_connections 2023-05-20T01:28:33.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.420+0000 7f84f1d3bb80 1 -- 172.21.15.16:0/3134828037 wait complete. 2023-05-20T01:28:33.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.421+0000 7f84f1d3bb80 1 -- start start 2023-05-20T01:28:33.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.421+0000 7f84f1d3bb80 1 Processor -- start 2023-05-20T01:28:33.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.421+0000 7f84f1d3bb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ee6c6f9870 0x55ee6c7a7840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:33.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.421+0000 7f84f1d3bb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee6c722700 0x55ee6c6f5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:33.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.421+0000 7f84f1d3bb80 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ee6c726400 0x55ee6c6f6210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:33.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.421+0000 7f84f1d3bb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee6c6fbf30 con 0x55ee6c722700 2023-05-20T01:28:33.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.421+0000 7f84f1d3bb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee6c7a7d80 con 0x55ee6c6f9870 2023-05-20T01:28:33.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.421+0000 7f84f1d3bb80 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ee6c721ef0 con 0x55ee6c726400 2023-05-20T01:28:33.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.421+0000 7f84f1d3bb80 10 client.0 _ll_register_callbacks cb 0x7f84d40158b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-05-20T01:28:33.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.421+0000 7f84deffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee6c722700 0x55ee6c6f5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:33.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.422+0000 7f84deffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee6c722700 0x55ee6c6f5cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40214/0 (socket says 172.21.15.16:40214) 2023-05-20T01:28:33.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.422+0000 7f84deffd700 1 -- 172.21.15.16:0/4082339045 learned_addr learned my addr 172.21.15.16:0/4082339045 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:33.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.422+0000 7f84df7fe700 1 --2- 172.21.15.16:0/4082339045 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ee6c6f9870 0x55ee6c7a7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:33.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.422+0000 7f84dffff700 1 --2- 172.21.15.16:0/4082339045 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ee6c726400 0x55ee6c6f6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:33.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.422+0000 7f84deffd700 1 -- 172.21.15.16:0/4082339045 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ee6c726400 msgr2=0x55ee6c6f6210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:33.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.422+0000 7f84deffd700 1 --2- 172.21.15.16:0/4082339045 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ee6c726400 0x55ee6c6f6210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:33.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.422+0000 7f84deffd700 1 -- 172.21.15.16:0/4082339045 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ee6c6f9870 msgr2=0x55ee6c7a7840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:33.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.422+0000 7f84deffd700 1 --2- 172.21.15.16:0/4082339045 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ee6c6f9870 0x55ee6c7a7840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:33.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.422+0000 7f84deffd700 1 -- 172.21.15.16:0/4082339045 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84d8011040 con 0x55ee6c722700 2023-05-20T01:28:33.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.422+0000 7f84deffd700 1 --2- 172.21.15.16:0/4082339045 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee6c722700 0x55ee6c6f5cd0 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f84d40011e0 tx=0x7f84d400a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecaa49018c98a3d4 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:33.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.422+0000 7f84be7fc700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-05-20T01:28:33.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.422+0000 7f84be7fc700 1 -- 172.21.15.16:0/4082339045 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84d4013770 con 0x55ee6c722700 2023-05-20T01:28:33.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.422+0000 7f84be7fc700 1 -- 172.21.15.16:0/4082339045 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(136..136 src has 1..136) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f84d4013a80 con 0x55ee6c722700 2023-05-20T01:28:33.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.422+0000 7f84f1d3bb80 1 -- 172.21.15.16:0/4082339045 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f84d0061690 con 0x55ee6c722700 2023-05-20T01:28:33.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.422+0000 7f84be7fc700 1 -- 172.21.15.16:0/4082339045 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=99) v2 ==== 24+0+0 (secure 0 0 0) 0x7f84d4013c40 con 0x55ee6c722700 2023-05-20T01:28:33.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.422+0000 7f84f1d3bb80 10 client.8856 fetch_fsmap learned FSMap version 99 2023-05-20T01:28:33.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.422+0000 7f84f1d3bb80 1 -- 172.21.15.16:0/4082339045 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=99,monmap=2+}) v3 -- 0x55ee6c463c10 con 0x55ee6c722700 2023-05-20T01:28:33.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.422+0000 7f84be7fc700 1 -- 172.21.15.16:0/4082339045 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84d0061690 con 0x55ee6c722700 2023-05-20T01:28:33.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.423+0000 7f84be7fc700 1 -- 172.21.15.16:0/4082339045 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 99) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f84d40016a0 con 0x55ee6c722700 2023-05-20T01:28:33.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.423+0000 7f84f1d3bb80 10 client.8856 fetch_fsmap finished waiting for FSMap version 99 2023-05-20T01:28:33.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.423+0000 7f84f1d3bb80 10 client.8856 resolve_mds: resolved name 'a' to daemon mds.a 2023-05-20T01:28:33.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.423+0000 7f84f1d3bb80 20 client.8856 populate_metadata read hostname 'smithi016' 2023-05-20T01:28:33.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.423+0000 7f84f1d3bb80 1 --2- 172.21.15.16:0/4082339045 >> [v2:172.21.15.16:6835/2295684907,v1:172.21.15.16:6837/2295684907] conn(0x55ee6c759ce0 0x55ee6c75a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:33.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.423+0000 7f84f1d3bb80 4 client.8856 mds_command: new command op to 8703 tid=0[{"prefix": "status"}] 2023-05-20T01:28:33.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.423+0000 7f84f1d3bb80 1 -- 172.21.15.16:0/4082339045 --> [v2:172.21.15.16:6835/2295684907,v1:172.21.15.16:6837/2295684907] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55ee6c61b610 con 0x55ee6c759ce0 2023-05-20T01:28:33.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.423+0000 7f84df7fe700 1 --2- 172.21.15.16:0/4082339045 >> [v2:172.21.15.16:6835/2295684907,v1:172.21.15.16:6837/2295684907] conn(0x55ee6c759ce0 0x55ee6c75a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:33.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.423+0000 7f84be7fc700 10 client.8856 ms_handle_connect on v2:172.21.15.16:6835/2295684907 2023-05-20T01:28:33.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.423+0000 7f84df7fe700 1 --2- 172.21.15.16:0/4082339045 >> [v2:172.21.15.16:6835/2295684907,v1:172.21.15.16:6837/2295684907] conn(0x55ee6c759ce0 0x55ee6c75a0b0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2a7a35b0d102176 server_cookie=2efb9efc5717ddfb in_seq=0 out_seq=0 2023-05-20T01:28:33.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.423+0000 7f84be7fc700 1 -- 172.21.15.16:0/4082339045 <== mds.0 v2:172.21.15.16:6835/2295684907 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x55ee6c61b610 con 0x55ee6c759ce0 2023-05-20T01:28:33.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.423+0000 7f84be7fc700 10 client.8856 handle_command_reply: tid=0 2023-05-20T01:28:33.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.423+0000 7f84f1d3bb80 1 client.8856 shutdown 2023-05-20T01:28:33.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.423+0000 7f84f1d3bb80 1 -- 172.21.15.16:0/4082339045 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee6c722700 msgr2=0x55ee6c6f5cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:33.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.423+0000 7f84f1d3bb80 1 --2- 172.21.15.16:0/4082339045 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee6c722700 0x55ee6c6f5cd0 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f84d40011e0 tx=0x7f84d400a000 comp rx=0 tx=0).stop 2023-05-20T01:28:33.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.423+0000 7f84f1d3bb80 1 -- 172.21.15.16:0/4082339045 shutdown_connections 2023-05-20T01:28:33.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.423+0000 7f84f1d3bb80 1 --2- 172.21.15.16:0/4082339045 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ee6c726400 0x55ee6c6f6210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:33.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.423+0000 7f84f1d3bb80 1 --2- 172.21.15.16:0/4082339045 >> [v2:172.21.15.16:6835/2295684907,v1:172.21.15.16:6837/2295684907] conn(0x55ee6c759ce0 0x55ee6c75a0b0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:33.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.423+0000 7f84f1d3bb80 1 --2- 172.21.15.16:0/4082339045 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ee6c6f9870 0x55ee6c7a7840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:33.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.423+0000 7f84be7fc700 0 client.8856 ms_handle_reset on v2:172.21.15.16:6835/2295684907 2023-05-20T01:28:33.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.423+0000 7f84f1d3bb80 1 --2- 172.21.15.16:0/4082339045 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee6c722700 0x55ee6c6f5cd0 secure :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f84d40011e0 tx=0x7f84d400a000 comp rx=0 tx=0).stop 2023-05-20T01:28:33.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.423+0000 7f84f1d3bb80 1 -- 172.21.15.16:0/4082339045 >> 172.21.15.16:0/4082339045 conn(0x55ee6c6f0f30 msgr2=0x55ee6c73e230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:33.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.424+0000 7f84f1d3bb80 1 -- 172.21.15.16:0/4082339045 shutdown_connections 2023-05-20T01:28:33.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.424+0000 7f84f1d3bb80 1 -- 172.21.15.16:0/4082339045 wait complete. 2023-05-20T01:28:33.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.424+0000 7f84f1d3bb80 20 client.8856 trim_cache size 0 max 16384 2023-05-20T01:28:33.438 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-05-20T01:28:33.438 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "c2245cc5-ecab-4087-a59b-bc81db901676", 2023-05-20T01:28:33.438 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2023-05-20T01:28:33.439 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 8703, 2023-05-20T01:28:33.439 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2023-05-20T01:28:33.439 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2023-05-20T01:28:33.439 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2023-05-20T01:28:33.439 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 6.3286967719999998, 2023-05-20T01:28:33.439 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 99, 2023-05-20T01:28:33.439 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 136, 2023-05-20T01:28:33.440 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 136, 2023-05-20T01:28:33.440 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 11.639229781999999 2023-05-20T01:28:33.440 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-05-20T01:28:33.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.424+0000 7f84e5baf700 1 -- 172.21.15.16:0/1418481781 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f84cc06ace0 msgr2=0x7f84cc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:33.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.424+0000 7f84e5baf700 1 --2- 172.21.15.16:0/1418481781 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f84cc06ace0 0x7f84cc06d190 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f84d4000f60 tx=0x7f84d4007ca0 comp rx=0 tx=0).stop 2023-05-20T01:28:33.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.424+0000 7f84e5baf700 1 -- 172.21.15.16:0/1418481781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84e00b1e00 msgr2=0x7f84e0156490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:33.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.424+0000 7f84e5baf700 1 --2- 172.21.15.16:0/1418481781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84e00b1e00 0x7f84e0156490 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f84d00090b0 tx=0x7f84d0004750 comp rx=0 tx=0).stop 2023-05-20T01:28:33.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.424+0000 7f84e5baf700 1 -- 172.21.15.16:0/1418481781 shutdown_connections 2023-05-20T01:28:33.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.424+0000 7f84e5baf700 1 --2- 172.21.15.16:0/1418481781 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f84e015caf0 0x7f84e01e60e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:33.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.424+0000 7f84e5baf700 1 --2- 172.21.15.16:0/1418481781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84e00b1e00 0x7f84e0156490 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:33.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.424+0000 7f84e5baf700 1 --2- 172.21.15.16:0/1418481781 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f84cc06ace0 0x7f84cc06d190 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:33.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.424+0000 7f84e5baf700 1 --2- 172.21.15.16:0/1418481781 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f84e00ccf00 0x7f84e0153f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:33.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.424+0000 7f84e5baf700 1 -- 172.21.15.16:0/1418481781 >> 172.21.15.16:0/1418481781 conn(0x7f84e00661b0 msgr2=0x7f84e00710a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:33.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.424+0000 7f84e5baf700 1 -- 172.21.15.16:0/1418481781 shutdown_connections 2023-05-20T01:28:33.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.424+0000 7f84e5baf700 1 -- 172.21.15.16:0/1418481781 wait complete. 2023-05-20T01:28:33.443 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-05-20T01:28:33.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.564+0000 7f147e210700 1 Processor -- start 2023-05-20T01:28:33.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.564+0000 7f147e210700 1 -- start start 2023-05-20T01:28:33.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.565+0000 7f147e210700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f147806cb90 0x7f147815ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:33.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.565+0000 7f147e210700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f147806cf60 0x7f1478070910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:33.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.565+0000 7f147e210700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14780b31a0 con 0x7f14780b54f0 2023-05-20T01:28:33.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.565+0000 7f147e210700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14780cc990 con 0x7f147806cf60 2023-05-20T01:28:33.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.565+0000 7f147e210700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14780b3480 con 0x7f147806cb90 2023-05-20T01:28:33.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.565+0000 7f1477fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f147806cf60 0x7f1478070910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:33.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.565+0000 7f1477fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f147806cf60 0x7f1478070910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40224/0 (socket says 172.21.15.16:40224) 2023-05-20T01:28:33.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.565+0000 7f1477fff700 1 -- 172.21.15.16:0/3308332470 learned_addr learned my addr 172.21.15.16:0/3308332470 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:33.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.565+0000 7f14777fe700 1 --2- 172.21.15.16:0/3308332470 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f147806cb90 0x7f147815ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:33.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.565+0000 7f1477fff700 1 -- 172.21.15.16:0/3308332470 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f147806cb90 msgr2=0x7f147815ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:33.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.565+0000 7f1477fff700 1 --2- 172.21.15.16:0/3308332470 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f147806cb90 0x7f147815ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:33.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.565+0000 7f1477fff700 1 -- 172.21.15.16:0/3308332470 >> v1:172.21.15.132:6789/0 conn(0x7f14780b54f0 legacy=0x7f14780b29e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:33.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.565+0000 7f1477fff700 1 -- 172.21.15.16:0/3308332470 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f147815af60 con 0x7f147806cf60 2023-05-20T01:28:33.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.565+0000 7f1477fff700 1 --2- 172.21.15.16:0/3308332470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f147806cf60 0x7f1478070910 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f1470009e50 tx=0x7f147000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1eb5d228eb1a1c01 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:33.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.565+0000 7f14767fc700 1 -- 172.21.15.16:0/3308332470 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f147000e070 con 0x7f147806cf60 2023-05-20T01:28:33.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.566+0000 7f14767fc700 1 -- 172.21.15.16:0/3308332470 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1470003a20 con 0x7f147806cf60 2023-05-20T01:28:33.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.566+0000 7f14767fc700 1 -- 172.21.15.16:0/3308332470 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1470003dd0 con 0x7f147806cf60 2023-05-20T01:28:33.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.566+0000 7f147e210700 1 -- 172.21.15.16:0/3308332470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f147806cf60 msgr2=0x7f1478070910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:33.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.566+0000 7f147e210700 1 --2- 172.21.15.16:0/3308332470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f147806cf60 0x7f1478070910 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f1470009e50 tx=0x7f147000b330 comp rx=0 tx=0).stop 2023-05-20T01:28:33.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.566+0000 7f147e210700 1 -- 172.21.15.16:0/3308332470 shutdown_connections 2023-05-20T01:28:33.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.566+0000 7f147e210700 1 --2- 172.21.15.16:0/3308332470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f147806cf60 0x7f1478070910 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:33.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.566+0000 7f147e210700 1 --2- 172.21.15.16:0/3308332470 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f147806cb90 0x7f147815ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:33.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.566+0000 7f147e210700 1 -- 172.21.15.16:0/3308332470 >> 172.21.15.16:0/3308332470 conn(0x7f14780661b0 msgr2=0x7f1478066580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:33.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.566+0000 7f147e210700 1 -- 172.21.15.16:0/3308332470 shutdown_connections 2023-05-20T01:28:33.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.566+0000 7f147e210700 1 -- 172.21.15.16:0/3308332470 wait complete. 2023-05-20T01:28:33.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.567+0000 7f147e210700 1 Processor -- start 2023-05-20T01:28:33.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.567+0000 7f147e210700 1 -- start start 2023-05-20T01:28:33.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.568+0000 7f147e210700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f147806cb90 0x7f147814e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:33.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.568+0000 7f147e210700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f147806cf60 0x7f1478156ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:33.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.568+0000 7f147e210700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f14780b54f0 0x7f1478157ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:33.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.568+0000 7f147e210700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1478155a10 con 0x7f147806cb90 2023-05-20T01:28:33.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.568+0000 7f147e210700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1478155cf0 con 0x7f14780b54f0 2023-05-20T01:28:33.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.568+0000 7f147e210700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1478155fd0 con 0x7f147806cf60 2023-05-20T01:28:33.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.568+0000 7f14777fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f147806cb90 0x7f147814e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:33.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.568+0000 7f14777fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f147806cb90 0x7f147814e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40226/0 (socket says 172.21.15.16:40226) 2023-05-20T01:28:33.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.568+0000 7f14777fe700 1 -- 172.21.15.16:0/1312072910 learned_addr learned my addr 172.21.15.16:0/1312072910 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:33.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.569+0000 7f14777fe700 1 -- 172.21.15.16:0/1312072910 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f147806cf60 msgr2=0x7f1478156ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:33.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.569+0000 7f1477fff700 1 --2- 172.21.15.16:0/1312072910 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f14780b54f0 0x7f1478157ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:33.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.569+0000 7f1476ffd700 1 --2- 172.21.15.16:0/1312072910 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f147806cf60 0x7f1478156ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:33.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.569+0000 7f14777fe700 1 --2- 172.21.15.16:0/1312072910 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f147806cf60 0x7f1478156ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:33.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.569+0000 7f14777fe700 1 -- 172.21.15.16:0/1312072910 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f14780b54f0 msgr2=0x7f1478157ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:33.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.569+0000 7f14777fe700 1 --2- 172.21.15.16:0/1312072910 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f14780b54f0 0x7f1478157ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:33.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.569+0000 7f14777fe700 1 -- 172.21.15.16:0/1312072910 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1470009b30 con 0x7f147806cb90 2023-05-20T01:28:33.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.569+0000 7f14777fe700 1 --2- 172.21.15.16:0/1312072910 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f147806cb90 0x7f147814e880 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f146800c7f0 tx=0x7f146800cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:33.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.569+0000 7f145ffff700 1 -- 172.21.15.16:0/1312072910 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1468011800 con 0x7f147806cb90 2023-05-20T01:28:33.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.569+0000 7f145ffff700 1 -- 172.21.15.16:0/1312072910 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1468011960 con 0x7f147806cb90 2023-05-20T01:28:33.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.569+0000 7f147e210700 1 -- 172.21.15.16:0/1312072910 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f147815b520 con 0x7f147806cb90 2023-05-20T01:28:33.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.569+0000 7f145ffff700 1 -- 172.21.15.16:0/1312072910 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1468011d10 con 0x7f147806cb90 2023-05-20T01:28:33.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.569+0000 7f147e210700 1 -- 172.21.15.16:0/1312072910 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f147815b240 con 0x7f147806cb90 2023-05-20T01:28:33.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.570+0000 7f147e210700 1 -- 172.21.15.16:0/1312072910 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f147806d400 con 0x7f147806cb90 2023-05-20T01:28:33.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.570+0000 7f145ffff700 1 -- 172.21.15.16:0/1312072910 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1468018360 con 0x7f147806cb90 2023-05-20T01:28:33.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.571+0000 7f145ffff700 1 --2- 172.21.15.16:0/1312072910 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f146406ace0 0x7f146406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:33.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.571+0000 7f145ffff700 1 -- 172.21.15.16:0/1312072910 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f14680894e0 con 0x7f147806cb90 2023-05-20T01:28:33.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.571+0000 7f1476ffd700 1 --2- 172.21.15.16:0/1312072910 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f146406ace0 0x7f146406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:33.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.571+0000 7f1476ffd700 1 --2- 172.21.15.16:0/1312072910 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f146406ace0 0x7f146406d190 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f14780b5960 tx=0x7f146c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:33.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.575+0000 7f145ffff700 1 -- 172.21.15.16:0/1312072910 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f14680544e0 con 0x7f147806cb90 2023-05-20T01:28:33.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.819+0000 7f147e210700 1 -- 172.21.15.16:0/1312072910 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f14780747b0 con 0x7f147806cb90 2023-05-20T01:28:33.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.819+0000 7f145ffff700 1 -- 172.21.15.16:0/1312072910 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v99) v1 ==== 53+0+73 (secure 0 0 0) 0x7f1468057b00 con 0x7f147806cb90 2023-05-20T01:28:33.823 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-05-20T01:28:33.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.821+0000 7f147e210700 1 -- 172.21.15.16:0/1312072910 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f146406ace0 msgr2=0x7f146406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:33.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.821+0000 7f147e210700 1 --2- 172.21.15.16:0/1312072910 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f146406ace0 0x7f146406d190 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f14780b5960 tx=0x7f146c008040 comp rx=0 tx=0).stop 2023-05-20T01:28:33.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.821+0000 7f147e210700 1 -- 172.21.15.16:0/1312072910 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f147806cb90 msgr2=0x7f147814e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:33.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.821+0000 7f147e210700 1 --2- 172.21.15.16:0/1312072910 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f147806cb90 0x7f147814e880 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f146800c7f0 tx=0x7f146800cbb0 comp rx=0 tx=0).stop 2023-05-20T01:28:33.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.821+0000 7f147e210700 1 -- 172.21.15.16:0/1312072910 shutdown_connections 2023-05-20T01:28:33.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.821+0000 7f147e210700 1 --2- 172.21.15.16:0/1312072910 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f14780b54f0 0x7f1478157ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:33.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.821+0000 7f147e210700 1 --2- 172.21.15.16:0/1312072910 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f147806cb90 0x7f147814e880 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:33.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.821+0000 7f147e210700 1 --2- 172.21.15.16:0/1312072910 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f146406ace0 0x7f146406d190 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:33.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.821+0000 7f147e210700 1 --2- 172.21.15.16:0/1312072910 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f147806cf60 0x7f1478156ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:33.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.821+0000 7f147e210700 1 -- 172.21.15.16:0/1312072910 >> 172.21.15.16:0/1312072910 conn(0x7f14780661b0 msgr2=0x7f1478158a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:33.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.821+0000 7f147e210700 1 -- 172.21.15.16:0/1312072910 shutdown_connections 2023-05-20T01:28:33.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:33.821+0000 7f147e210700 1 -- 172.21.15.16:0/1312072910 wait complete. 2023-05-20T01:28:33.837 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-05-20T01:28:33.838 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-05-20T01:28:33.838 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2023-05-20T01:28:33.838 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-05-20T01:28:33.839 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-05-20T01:28:33.839 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-05-20T01:28:33.839 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-05-20T01:28:33.839 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-05-20T01:28:33.840 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-05-20T01:28:33.840 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:28:33.840 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2023-05-20T01:28:33.858 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-05-20T01:28:33.858 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-05-20T01:28:33.858 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2023-05-20T01:28:33.859 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-05-20T01:28:33.859 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2023-05-20T01:28:33.859 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-05-20T01:28:33.859 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-05-20T01:28:33.859 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2023-05-20T01:28:33.859 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-05-20T01:28:33.860 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2023-05-20T01:28:33.860 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-05-20T01:28:33.860 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2023-05-20T01:28:33.860 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-05-20T01:28:33.860 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 41720sec preferred_lft 41720sec 2023-05-20T01:28:33.860 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link 2023-05-20T01:28:33.861 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-05-20T01:28:33.861 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-05-20T01:28:33.861 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2023-05-20T01:28:33.861 INFO:teuthology.orchestra.run.smithi016.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-05-20T01:28:33.861 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0a:2e:0c:6b:5d:42 brd ff:ff:ff:ff:ff:ff 2023-05-20T01:28:33.861 INFO:teuthology.orchestra.run.smithi016.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-05-20T01:28:33.861 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-05-20T01:28:33.862 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::7c00:b2ff:fef4:946/64 scope link 2023-05-20T01:28:33.862 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-05-20T01:28:33.862 INFO:teuthology.orchestra.run.smithi016.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-05-20T01:28:33.862 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether d2:e4:70:a7:b9:4b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-05-20T01:28:33.862 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::d0e4:70ff:fea7:b94b/64 scope link 2023-05-20T01:28:33.862 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-05-20T01:28:33.862 INFO:teuthology.orchestra.run.smithi016.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-05-20T01:28:33.862 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0a:2e:0c:6b:5d:42 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-05-20T01:28:33.863 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::82e:cff:fe6b:5d42/64 scope link 2023-05-20T01:28:33.863 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-05-20T01:28:33.863 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:28:33.864 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-05-20T01:28:33.924 INFO:teuthology.orchestra.run.smithi016.stdout:1 2023-05-20T01:28:33.925 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:28:33.925 DEBUG:teuthology.orchestra.run.smithi016:> route 2023-05-20T01:28:33.983 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2023-05-20T01:28:33.984 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-05-20T01:28:33.984 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-05-20T01:28:33.984 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-05-20T01:28:33.984 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-05-20T01:28:33.985 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:28:33.985 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2023-05-20T01:28:33.986 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2023-05-20T01:28:33.986 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-05-20T01:28:33.986 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-05-20T01:28:33.987 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-05-20T01:28:33.987 DEBUG:teuthology.orchestra.run.smithi016:> ') 2023-05-20T01:28:34.090 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:28:34.090 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2023-05-20T01:28:34.107 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-05-20T01:28:34.108 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-05-20T01:28:34.108 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-05-20T01:28:34.108 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:28:34.109 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.0 up 2023-05-20T01:28:34.175 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-05-20T01:28:34.176 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-05-20T01:28:34.232 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-05-20T01:28:34.232 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-05-20T01:28:34.233 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-05-20T01:28:34.289 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-05-20T01:28:34.289 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:28:34.290 DEBUG:teuthology.orchestra.run.smithi016:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-05-20T01:28:34.375 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-05-20T01:28:34.376 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-05-20T01:28:34.407 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-05-20T01:28:34.407 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-05-20T01:28:34.407 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.132:3300,172.21.15.132:3301" 2023-05-20T01:28:34.407 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: trying mount with new device syntax: 0@c2245cc5-ecab-4087-a59b-bc81db901676.cephfs=/ 2023-05-20T01:28:34.408 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.16:3300/172.21.15.132:3300/172.21.15.132:3301" will pass to kernel 2023-05-20T01:28:34.408 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: v2 mount support check returned 2 2023-05-20T01:28:34.408 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: kernel does not support v2 syntax 2023-05-20T01:28:34.408 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: switching to using v1 address with old syntax 2023-05-20T01:28:34.447 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-05-20T01:28:34.447 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-05-20T01:28:34.447 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.132:3300,172.21.15.132:3301" 2023-05-20T01:28:34.448 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: trying mount with new device syntax: 0@c2245cc5-ecab-4087-a59b-bc81db901676.cephfs=/ 2023-05-20T01:28:34.448 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.16:3300/172.21.15.132:3300/172.21.15.132:3301" will pass to kernel 2023-05-20T01:28:34.448 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: v2 mount support check returned 2 2023-05-20T01:28:34.448 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: kernel does not support v2 syntax 2023-05-20T01:28:34.448 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: switching to using v1 address with old syntax 2023-05-20T01:28:34.448 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:6789,172.21.15.132:6789,172.21.15.132:6790" 2023-05-20T01:28:34.449 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.16:6789,172.21.15.132:6789,172.21.15.132:6790:/ 2023-05-20T01:28:34.449 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=c2245cc5-ecab-4087-a59b-bc81db901676" will pass to kernel 2023-05-20T01:28:34.452 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-05-20T01:28:34.452 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:28:34.453 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-05-20T01:28:34.487 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:28:34.488 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-05-20T01:28:34.568 INFO:teuthology.orchestra.run.smithi016.stdout:client8865 2023-05-20T01:28:34.568 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-05-20T01:28:34.569 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2023-05-20T01:28:34.626 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-05-20T01:28:34.668 DEBUG:teuthology.orchestra.run.smithi132:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-05-20T01:28:34.714 INFO:teuthology.orchestra.run.smithi016.stdout:[global] 2023-05-20T01:28:34.715 INFO:teuthology.orchestra.run.smithi016.stdout: chdir = "" 2023-05-20T01:28:34.715 INFO:teuthology.orchestra.run.smithi016.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-05-20T01:28:34.715 INFO:teuthology.orchestra.run.smithi016.stdout: auth supported = cephx 2023-05-20T01:28:34.715 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:28:34.715 INFO:teuthology.orchestra.run.smithi016.stdout: filestore xattr use omap = true 2023-05-20T01:28:34.715 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:28:34.716 INFO:teuthology.orchestra.run.smithi016.stdout: mon clock drift allowed = 1.000 2023-05-20T01:28:34.716 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:28:34.716 INFO:teuthology.orchestra.run.smithi016.stdout: osd crush chooseleaf type = 0 2023-05-20T01:28:34.716 INFO:teuthology.orchestra.run.smithi016.stdout: auth debug = true 2023-05-20T01:28:34.716 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:28:34.716 INFO:teuthology.orchestra.run.smithi016.stdout: ms die on old message = true 2023-05-20T01:28:34.716 INFO:teuthology.orchestra.run.smithi016.stdout: ms die on bug = true 2023-05-20T01:28:34.717 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:28:34.717 INFO:teuthology.orchestra.run.smithi016.stdout: mon max pg per osd = 10000 # >= luminous 2023-05-20T01:28:34.717 INFO:teuthology.orchestra.run.smithi016.stdout: mon pg warn max object skew = 0 2023-05-20T01:28:34.717 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:28:34.717 INFO:teuthology.orchestra.run.smithi016.stdout: # disable pg_autoscaler by default for new pools 2023-05-20T01:28:34.717 INFO:teuthology.orchestra.run.smithi016.stdout: osd_pool_default_pg_autoscale_mode = off 2023-05-20T01:28:34.717 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:28:34.717 INFO:teuthology.orchestra.run.smithi016.stdout: osd pool default size = 2 2023-05-20T01:28:34.718 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:28:34.718 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd allow primary affinity = true 2023-05-20T01:28:34.718 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd allow pg remap = true 2023-05-20T01:28:34.718 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on legacy crush tunables = false 2023-05-20T01:28:34.718 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on crush straw calc version zero = false 2023-05-20T01:28:34.718 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on no sortbitwise = false 2023-05-20T01:28:34.719 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on osd down out interval zero = false 2023-05-20T01:28:34.719 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on too few osds = false 2023-05-20T01:28:34.720 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-05-20T01:28:34.720 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_pool_no_redundancy = false 2023-05-20T01:28:34.720 INFO:teuthology.orchestra.run.smithi016.stdout: mon_allow_pool_size_one = true 2023-05-20T01:28:34.721 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:28:34.721 INFO:teuthology.orchestra.run.smithi016.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-05-20T01:28:34.721 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:28:34.721 INFO:teuthology.orchestra.run.smithi016.stdout: osd default data pool replay window = 5 2023-05-20T01:28:34.721 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:28:34.721 INFO:teuthology.orchestra.run.smithi016.stdout: mon allow pool delete = true 2023-05-20T01:28:34.721 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:28:34.721 INFO:teuthology.orchestra.run.smithi016.stdout: mon cluster log file level = debug 2023-05-20T01:28:34.722 INFO:teuthology.orchestra.run.smithi016.stdout: debug asserts on shutdown = true 2023-05-20T01:28:34.722 INFO:teuthology.orchestra.run.smithi016.stdout: mon health detail to clog = false 2023-05-20T01:28:34.722 INFO:teuthology.orchestra.run.smithi016.stdout: mon host = "172.21.15.16,172.21.15.132,[v2:172.21.15.132:3301,v1:172.21.15.132:6790]" 2023-05-20T01:28:34.722 INFO:teuthology.orchestra.run.smithi016.stdout: ms die on skipped message = False 2023-05-20T01:28:34.722 INFO:teuthology.orchestra.run.smithi016.stdout: fsid = c2245cc5-ecab-4087-a59b-bc81db901676 2023-05-20T01:28:34.722 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:28:34.722 INFO:teuthology.orchestra.run.smithi016.stdout:[osd] 2023-05-20T01:28:34.723 INFO:teuthology.orchestra.run.smithi016.stdout: osd journal size = 100 2023-05-20T01:28:34.723 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:28:34.723 INFO:teuthology.orchestra.run.smithi016.stdout: osd scrub load threshold = 5.0 2023-05-20T01:28:34.723 INFO:teuthology.orchestra.run.smithi016.stdout: osd scrub max interval = 600 2023-05-20T01:28:34.723 INFO:teuthology.orchestra.run.smithi016.stdout: osd mclock profile = high_recovery_ops 2023-05-20T01:28:34.723 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:28:34.723 INFO:teuthology.orchestra.run.smithi016.stdout: osd recover clone overlap = true 2023-05-20T01:28:34.724 INFO:teuthology.orchestra.run.smithi016.stdout: osd recovery max chunk = 1048576 2023-05-20T01:28:34.724 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:28:34.724 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug shutdown = true 2023-05-20T01:28:34.724 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug op order = true 2023-05-20T01:28:34.724 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug verify stray on activate = true 2023-05-20T01:28:34.724 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:28:34.725 INFO:teuthology.orchestra.run.smithi016.stdout: osd open classes on start = true 2023-05-20T01:28:34.725 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug pg log writeout = true 2023-05-20T01:28:34.725 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:28:34.725 INFO:teuthology.orchestra.run.smithi016.stdout: osd deep scrub update digest min age = 30 2023-05-20T01:28:34.725 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:28:34.725 INFO:teuthology.orchestra.run.smithi016.stdout: osd map max advance = 10 2023-05-20T01:28:34.725 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:28:34.725 INFO:teuthology.orchestra.run.smithi016.stdout: journal zero on create = true 2023-05-20T01:28:34.726 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:28:34.726 INFO:teuthology.orchestra.run.smithi016.stdout: filestore ondisk finisher threads = 3 2023-05-20T01:28:34.726 INFO:teuthology.orchestra.run.smithi016.stdout: filestore apply finisher threads = 3 2023-05-20T01:28:34.726 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:28:34.726 INFO:teuthology.orchestra.run.smithi016.stdout: bdev debug aio = true 2023-05-20T01:28:34.726 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug misdirected ops = true 2023-05-20T01:28:34.726 INFO:teuthology.orchestra.run.smithi016.stdout: bluestore block size = 96636764160 2023-05-20T01:28:34.727 INFO:teuthology.orchestra.run.smithi016.stdout: bluestore fsck on mount = True 2023-05-20T01:28:34.727 INFO:teuthology.orchestra.run.smithi016.stdout: debug bluefs = 20 2023-05-20T01:28:34.727 INFO:teuthology.orchestra.run.smithi016.stdout: debug bluestore = 20 2023-05-20T01:28:34.727 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2023-05-20T01:28:34.727 INFO:teuthology.orchestra.run.smithi016.stdout: debug osd = 20 2023-05-20T01:28:34.727 INFO:teuthology.orchestra.run.smithi016.stdout: debug rocksdb = 10 2023-05-20T01:28:34.727 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd backfillfull_ratio = 0.85 2023-05-20T01:28:34.728 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd full ratio = 0.9 2023-05-20T01:28:34.728 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd nearfull ratio = 0.8 2023-05-20T01:28:34.728 INFO:teuthology.orchestra.run.smithi016.stdout: osd failsafe full ratio = 0.95 2023-05-20T01:28:34.728 INFO:teuthology.orchestra.run.smithi016.stdout: osd objectstore = bluestore 2023-05-20T01:28:34.728 INFO:teuthology.orchestra.run.smithi016.stdout: osd op complaint time = 180 2023-05-20T01:28:34.728 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:28:34.728 INFO:teuthology.orchestra.run.smithi016.stdout:[mgr] 2023-05-20T01:28:34.728 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2023-05-20T01:28:34.729 INFO:teuthology.orchestra.run.smithi016.stdout: debug mgr = 20 2023-05-20T01:28:34.729 INFO:teuthology.orchestra.run.smithi016.stdout: debug mon = 20 2023-05-20T01:28:34.729 INFO:teuthology.orchestra.run.smithi016.stdout: debug auth = 20 2023-05-20T01:28:34.729 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min pgs per osd = 4 2023-05-20T01:28:34.729 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min bytes per osd = 10 2023-05-20T01:28:34.729 INFO:teuthology.orchestra.run.smithi016.stdout: mgr/telemetry/nag = false 2023-05-20T01:28:34.729 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:28:34.730 INFO:teuthology.orchestra.run.smithi016.stdout:[mon] 2023-05-20T01:28:34.730 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2023-05-20T01:28:34.730 INFO:teuthology.orchestra.run.smithi016.stdout: debug mon = 20 2023-05-20T01:28:34.730 INFO:teuthology.orchestra.run.smithi016.stdout: debug paxos = 20 2023-05-20T01:28:34.730 INFO:teuthology.orchestra.run.smithi016.stdout: debug auth = 20 2023-05-20T01:28:34.730 INFO:teuthology.orchestra.run.smithi016.stdout: mon data avail warn = 5 2023-05-20T01:28:34.730 INFO:teuthology.orchestra.run.smithi016.stdout: mon mgr mkfs grace = 240 2023-05-20T01:28:34.730 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min pgs per osd = 4 2023-05-20T01:28:34.731 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd reporter subtree level = osd 2023-05-20T01:28:34.731 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd prime pg temp = true 2023-05-20T01:28:34.731 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min bytes per osd = 10 2023-05-20T01:28:34.731 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:28:34.731 INFO:teuthology.orchestra.run.smithi016.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-05-20T01:28:34.732 INFO:teuthology.orchestra.run.smithi016.stdout: auth mon ticket ttl = 660 # 11m 2023-05-20T01:28:34.732 INFO:teuthology.orchestra.run.smithi016.stdout: auth service ticket ttl = 240 # 4m 2023-05-20T01:28:34.732 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:28:34.732 INFO:teuthology.orchestra.run.smithi016.stdout: # don't complain about insecure global_id in the test suite 2023-05-20T01:28:34.732 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-05-20T01:28:34.732 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-05-20T01:28:34.732 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:28:34.732 INFO:teuthology.orchestra.run.smithi016.stdout: # 1m isn't quite enough 2023-05-20T01:28:34.733 INFO:teuthology.orchestra.run.smithi016.stdout: mon_down_mkfs_grace = 2m 2023-05-20T01:28:34.733 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:28:34.733 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_filestore_osds = false 2023-05-20T01:28:34.733 INFO:teuthology.orchestra.run.smithi016.stdout: mon op complaint time = 120 2023-05-20T01:28:34.733 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:28:34.733 INFO:teuthology.orchestra.run.smithi016.stdout:[client] 2023-05-20T01:28:34.733 INFO:teuthology.orchestra.run.smithi016.stdout: rgw cache enabled = true 2023-05-20T01:28:34.733 INFO:teuthology.orchestra.run.smithi016.stdout: rgw enable ops log = true 2023-05-20T01:28:34.734 INFO:teuthology.orchestra.run.smithi016.stdout: rgw enable usage log = true 2023-05-20T01:28:34.734 INFO:teuthology.orchestra.run.smithi016.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-05-20T01:28:34.734 INFO:teuthology.orchestra.run.smithi016.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-05-20T01:28:34.734 INFO:teuthology.orchestra.run.smithi016.stdout: client mount timeout = 600 2023-05-20T01:28:34.734 INFO:teuthology.orchestra.run.smithi016.stdout: debug client = 20 2023-05-20T01:28:34.734 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2023-05-20T01:28:34.734 INFO:teuthology.orchestra.run.smithi016.stdout: pid file = "" 2023-05-20T01:28:34.734 INFO:teuthology.orchestra.run.smithi016.stdout: rados mon op timeout = 900 2023-05-20T01:28:34.735 INFO:teuthology.orchestra.run.smithi016.stdout: rados osd op timeout = 900 2023-05-20T01:28:34.735 INFO:teuthology.orchestra.run.smithi016.stdout:[mon.a] 2023-05-20T01:28:34.735 INFO:teuthology.orchestra.run.smithi016.stdout:[mon.b] 2023-05-20T01:28:34.735 INFO:teuthology.orchestra.run.smithi016.stdout:[mon.c] 2023-05-20T01:28:34.735 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.a] 2023-05-20T01:28:34.735 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.c] 2023-05-20T01:28:34.735 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.b] 2023-05-20T01:28:34.736 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.d] 2023-05-20T01:28:34.736 INFO:teuthology.orchestra.run.smithi016.stdout:[mds] 2023-05-20T01:28:34.736 INFO:teuthology.orchestra.run.smithi016.stdout: debug mds = 20 2023-05-20T01:28:34.736 INFO:teuthology.orchestra.run.smithi016.stdout: debug mds balancer = 20 2023-05-20T01:28:34.736 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2023-05-20T01:28:34.736 INFO:teuthology.orchestra.run.smithi016.stdout: mds debug frag = True 2023-05-20T01:28:34.736 INFO:teuthology.orchestra.run.smithi016.stdout: mds debug scatterstat = False 2023-05-20T01:28:34.736 INFO:teuthology.orchestra.run.smithi016.stdout: mds op complaint time = 180 2023-05-20T01:28:34.737 INFO:teuthology.orchestra.run.smithi016.stdout: mds verify scatter = False 2023-05-20T01:28:34.737 INFO:teuthology.orchestra.run.smithi016.stdout: osd op complaint time = 180 2023-05-20T01:28:34.737 INFO:teuthology.orchestra.run.smithi016.stdout: rados mon op timeout = 900 2023-05-20T01:28:34.737 INFO:teuthology.orchestra.run.smithi016.stdout: rados osd op timeout = 900 2023-05-20T01:28:34.737 INFO:teuthology.orchestra.run.smithi016.stdout: mds bal merge size = 0 2023-05-20T01:28:34.739 INFO:teuthology.orchestra.run.smithi132.stdout:[global] 2023-05-20T01:28:34.739 INFO:teuthology.orchestra.run.smithi132.stdout: chdir = "" 2023-05-20T01:28:34.739 INFO:teuthology.orchestra.run.smithi132.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-05-20T01:28:34.739 INFO:teuthology.orchestra.run.smithi132.stdout: auth supported = cephx 2023-05-20T01:28:34.739 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:28:34.739 INFO:teuthology.orchestra.run.smithi132.stdout: filestore xattr use omap = true 2023-05-20T01:28:34.740 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:28:34.740 INFO:teuthology.orchestra.run.smithi132.stdout: mon clock drift allowed = 1.000 2023-05-20T01:28:34.740 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:28:34.740 INFO:teuthology.orchestra.run.smithi132.stdout: osd crush chooseleaf type = 0 2023-05-20T01:28:34.740 INFO:teuthology.orchestra.run.smithi132.stdout: auth debug = true 2023-05-20T01:28:34.740 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:28:34.741 INFO:teuthology.orchestra.run.smithi132.stdout: ms die on old message = true 2023-05-20T01:28:34.741 INFO:teuthology.orchestra.run.smithi132.stdout: ms die on bug = true 2023-05-20T01:28:34.741 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:28:34.741 INFO:teuthology.orchestra.run.smithi132.stdout: mon max pg per osd = 10000 # >= luminous 2023-05-20T01:28:34.741 INFO:teuthology.orchestra.run.smithi132.stdout: mon pg warn max object skew = 0 2023-05-20T01:28:34.741 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:28:34.741 INFO:teuthology.orchestra.run.smithi132.stdout: # disable pg_autoscaler by default for new pools 2023-05-20T01:28:34.741 INFO:teuthology.orchestra.run.smithi132.stdout: osd_pool_default_pg_autoscale_mode = off 2023-05-20T01:28:34.742 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:28:34.742 INFO:teuthology.orchestra.run.smithi132.stdout: osd pool default size = 2 2023-05-20T01:28:34.742 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:28:34.742 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd allow primary affinity = true 2023-05-20T01:28:34.742 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd allow pg remap = true 2023-05-20T01:28:34.742 INFO:teuthology.orchestra.run.smithi132.stdout: mon warn on legacy crush tunables = false 2023-05-20T01:28:34.742 INFO:teuthology.orchestra.run.smithi132.stdout: mon warn on crush straw calc version zero = false 2023-05-20T01:28:34.742 INFO:teuthology.orchestra.run.smithi132.stdout: mon warn on no sortbitwise = false 2023-05-20T01:28:34.743 INFO:teuthology.orchestra.run.smithi132.stdout: mon warn on osd down out interval zero = false 2023-05-20T01:28:34.743 INFO:teuthology.orchestra.run.smithi132.stdout: mon warn on too few osds = false 2023-05-20T01:28:34.743 INFO:teuthology.orchestra.run.smithi132.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-05-20T01:28:34.743 INFO:teuthology.orchestra.run.smithi132.stdout: mon_warn_on_pool_no_redundancy = false 2023-05-20T01:28:34.743 INFO:teuthology.orchestra.run.smithi132.stdout: mon_allow_pool_size_one = true 2023-05-20T01:28:34.743 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:28:34.743 INFO:teuthology.orchestra.run.smithi132.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-05-20T01:28:34.743 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:28:34.744 INFO:teuthology.orchestra.run.smithi132.stdout: osd default data pool replay window = 5 2023-05-20T01:28:34.744 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:28:34.744 INFO:teuthology.orchestra.run.smithi132.stdout: mon allow pool delete = true 2023-05-20T01:28:34.744 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:28:34.744 INFO:teuthology.orchestra.run.smithi132.stdout: mon cluster log file level = debug 2023-05-20T01:28:34.744 INFO:teuthology.orchestra.run.smithi132.stdout: debug asserts on shutdown = true 2023-05-20T01:28:34.744 INFO:teuthology.orchestra.run.smithi132.stdout: mon health detail to clog = false 2023-05-20T01:28:34.744 INFO:teuthology.orchestra.run.smithi132.stdout: mon host = "172.21.15.16,172.21.15.132,[v2:172.21.15.132:3301,v1:172.21.15.132:6790]" 2023-05-20T01:28:34.745 INFO:teuthology.orchestra.run.smithi132.stdout: ms die on skipped message = False 2023-05-20T01:28:34.745 INFO:teuthology.orchestra.run.smithi132.stdout: fsid = c2245cc5-ecab-4087-a59b-bc81db901676 2023-05-20T01:28:34.745 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:28:34.745 INFO:teuthology.orchestra.run.smithi132.stdout:[osd] 2023-05-20T01:28:34.745 INFO:teuthology.orchestra.run.smithi132.stdout: osd journal size = 100 2023-05-20T01:28:34.745 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:28:34.745 INFO:teuthology.orchestra.run.smithi132.stdout: osd scrub load threshold = 5.0 2023-05-20T01:28:34.746 INFO:teuthology.orchestra.run.smithi132.stdout: osd scrub max interval = 600 2023-05-20T01:28:34.746 INFO:teuthology.orchestra.run.smithi132.stdout: osd mclock profile = high_recovery_ops 2023-05-20T01:28:34.746 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:28:34.746 INFO:teuthology.orchestra.run.smithi132.stdout: osd recover clone overlap = true 2023-05-20T01:28:34.746 INFO:teuthology.orchestra.run.smithi132.stdout: osd recovery max chunk = 1048576 2023-05-20T01:28:34.746 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:28:34.746 INFO:teuthology.orchestra.run.smithi132.stdout: osd debug shutdown = true 2023-05-20T01:28:34.746 INFO:teuthology.orchestra.run.smithi132.stdout: osd debug op order = true 2023-05-20T01:28:34.747 INFO:teuthology.orchestra.run.smithi132.stdout: osd debug verify stray on activate = true 2023-05-20T01:28:34.747 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:28:34.747 INFO:teuthology.orchestra.run.smithi132.stdout: osd open classes on start = true 2023-05-20T01:28:34.747 INFO:teuthology.orchestra.run.smithi132.stdout: osd debug pg log writeout = true 2023-05-20T01:28:34.747 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:28:34.747 INFO:teuthology.orchestra.run.smithi132.stdout: osd deep scrub update digest min age = 30 2023-05-20T01:28:34.747 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:28:34.747 INFO:teuthology.orchestra.run.smithi132.stdout: osd map max advance = 10 2023-05-20T01:28:34.748 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:28:34.748 INFO:teuthology.orchestra.run.smithi132.stdout: journal zero on create = true 2023-05-20T01:28:34.748 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:28:34.748 INFO:teuthology.orchestra.run.smithi132.stdout: filestore ondisk finisher threads = 3 2023-05-20T01:28:34.748 INFO:teuthology.orchestra.run.smithi132.stdout: filestore apply finisher threads = 3 2023-05-20T01:28:34.748 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:28:34.748 INFO:teuthology.orchestra.run.smithi132.stdout: bdev debug aio = true 2023-05-20T01:28:34.749 INFO:teuthology.orchestra.run.smithi132.stdout: osd debug misdirected ops = true 2023-05-20T01:28:34.749 INFO:teuthology.orchestra.run.smithi132.stdout: bluestore block size = 96636764160 2023-05-20T01:28:34.749 INFO:teuthology.orchestra.run.smithi132.stdout: bluestore fsck on mount = True 2023-05-20T01:28:34.749 INFO:teuthology.orchestra.run.smithi132.stdout: debug bluefs = 20 2023-05-20T01:28:34.749 INFO:teuthology.orchestra.run.smithi132.stdout: debug bluestore = 20 2023-05-20T01:28:34.749 INFO:teuthology.orchestra.run.smithi132.stdout: debug ms = 1 2023-05-20T01:28:34.749 INFO:teuthology.orchestra.run.smithi132.stdout: debug osd = 20 2023-05-20T01:28:34.749 INFO:teuthology.orchestra.run.smithi132.stdout: debug rocksdb = 10 2023-05-20T01:28:34.750 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd backfillfull_ratio = 0.85 2023-05-20T01:28:34.750 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd full ratio = 0.9 2023-05-20T01:28:34.750 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd nearfull ratio = 0.8 2023-05-20T01:28:34.750 INFO:teuthology.orchestra.run.smithi132.stdout: osd failsafe full ratio = 0.95 2023-05-20T01:28:34.750 INFO:teuthology.orchestra.run.smithi132.stdout: osd objectstore = bluestore 2023-05-20T01:28:34.750 INFO:teuthology.orchestra.run.smithi132.stdout: osd op complaint time = 180 2023-05-20T01:28:34.750 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:28:34.750 INFO:teuthology.orchestra.run.smithi132.stdout:[mgr] 2023-05-20T01:28:34.751 INFO:teuthology.orchestra.run.smithi132.stdout: debug ms = 1 2023-05-20T01:28:34.751 INFO:teuthology.orchestra.run.smithi132.stdout: debug mgr = 20 2023-05-20T01:28:34.751 INFO:teuthology.orchestra.run.smithi132.stdout: debug mon = 20 2023-05-20T01:28:34.751 INFO:teuthology.orchestra.run.smithi132.stdout: debug auth = 20 2023-05-20T01:28:34.751 INFO:teuthology.orchestra.run.smithi132.stdout: mon reweight min pgs per osd = 4 2023-05-20T01:28:34.751 INFO:teuthology.orchestra.run.smithi132.stdout: mon reweight min bytes per osd = 10 2023-05-20T01:28:34.751 INFO:teuthology.orchestra.run.smithi132.stdout: mgr/telemetry/nag = false 2023-05-20T01:28:34.752 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:28:34.752 INFO:teuthology.orchestra.run.smithi132.stdout:[mon] 2023-05-20T01:28:34.752 INFO:teuthology.orchestra.run.smithi132.stdout: debug ms = 1 2023-05-20T01:28:34.752 INFO:teuthology.orchestra.run.smithi132.stdout: debug mon = 20 2023-05-20T01:28:34.752 INFO:teuthology.orchestra.run.smithi132.stdout: debug paxos = 20 2023-05-20T01:28:34.752 INFO:teuthology.orchestra.run.smithi132.stdout: debug auth = 20 2023-05-20T01:28:34.752 INFO:teuthology.orchestra.run.smithi132.stdout: mon data avail warn = 5 2023-05-20T01:28:34.752 INFO:teuthology.orchestra.run.smithi132.stdout: mon mgr mkfs grace = 240 2023-05-20T01:28:34.753 INFO:teuthology.orchestra.run.smithi132.stdout: mon reweight min pgs per osd = 4 2023-05-20T01:28:34.753 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd reporter subtree level = osd 2023-05-20T01:28:34.753 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd prime pg temp = true 2023-05-20T01:28:34.753 INFO:teuthology.orchestra.run.smithi132.stdout: mon reweight min bytes per osd = 10 2023-05-20T01:28:34.753 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:28:34.753 INFO:teuthology.orchestra.run.smithi132.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-05-20T01:28:34.753 INFO:teuthology.orchestra.run.smithi132.stdout: auth mon ticket ttl = 660 # 11m 2023-05-20T01:28:34.753 INFO:teuthology.orchestra.run.smithi132.stdout: auth service ticket ttl = 240 # 4m 2023-05-20T01:28:34.754 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:28:34.754 INFO:teuthology.orchestra.run.smithi132.stdout: # don't complain about insecure global_id in the test suite 2023-05-20T01:28:34.754 INFO:teuthology.orchestra.run.smithi132.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-05-20T01:28:34.754 INFO:teuthology.orchestra.run.smithi132.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-05-20T01:28:34.754 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:28:34.754 INFO:teuthology.orchestra.run.smithi132.stdout: # 1m isn't quite enough 2023-05-20T01:28:34.754 INFO:teuthology.orchestra.run.smithi132.stdout: mon_down_mkfs_grace = 2m 2023-05-20T01:28:34.754 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:28:34.755 INFO:teuthology.orchestra.run.smithi132.stdout: mon_warn_on_filestore_osds = false 2023-05-20T01:28:34.755 INFO:teuthology.orchestra.run.smithi132.stdout: mon op complaint time = 120 2023-05-20T01:28:34.755 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:28:34.755 INFO:teuthology.orchestra.run.smithi132.stdout:[client] 2023-05-20T01:28:34.755 INFO:teuthology.orchestra.run.smithi132.stdout: rgw cache enabled = true 2023-05-20T01:28:34.755 INFO:teuthology.orchestra.run.smithi132.stdout: rgw enable ops log = true 2023-05-20T01:28:34.755 INFO:teuthology.orchestra.run.smithi132.stdout: rgw enable usage log = true 2023-05-20T01:28:34.755 INFO:teuthology.orchestra.run.smithi132.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-05-20T01:28:34.756 INFO:teuthology.orchestra.run.smithi132.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-05-20T01:28:34.756 INFO:teuthology.orchestra.run.smithi132.stdout: client mount timeout = 600 2023-05-20T01:28:34.756 INFO:teuthology.orchestra.run.smithi132.stdout: debug client = 20 2023-05-20T01:28:34.756 INFO:teuthology.orchestra.run.smithi132.stdout: debug ms = 1 2023-05-20T01:28:34.756 INFO:teuthology.orchestra.run.smithi132.stdout: pid file = "" 2023-05-20T01:28:34.756 INFO:teuthology.orchestra.run.smithi132.stdout: rados mon op timeout = 900 2023-05-20T01:28:34.756 INFO:teuthology.orchestra.run.smithi132.stdout: rados osd op timeout = 900 2023-05-20T01:28:34.757 INFO:teuthology.orchestra.run.smithi132.stdout:[mon.a] 2023-05-20T01:28:34.757 INFO:teuthology.orchestra.run.smithi132.stdout:[mon.b] 2023-05-20T01:28:34.757 INFO:teuthology.orchestra.run.smithi132.stdout:[mon.c] 2023-05-20T01:28:34.757 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.a] 2023-05-20T01:28:34.757 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.c] 2023-05-20T01:28:34.757 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.b] 2023-05-20T01:28:34.757 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.d] 2023-05-20T01:28:34.757 INFO:teuthology.orchestra.run.smithi132.stdout:[mds] 2023-05-20T01:28:34.758 INFO:teuthology.orchestra.run.smithi132.stdout: debug mds = 20 2023-05-20T01:28:34.758 INFO:teuthology.orchestra.run.smithi132.stdout: debug mds balancer = 20 2023-05-20T01:28:34.758 INFO:teuthology.orchestra.run.smithi132.stdout: debug ms = 1 2023-05-20T01:28:34.758 INFO:teuthology.orchestra.run.smithi132.stdout: mds debug frag = True 2023-05-20T01:28:34.758 INFO:teuthology.orchestra.run.smithi132.stdout: mds debug scatterstat = False 2023-05-20T01:28:34.758 INFO:teuthology.orchestra.run.smithi132.stdout: mds op complaint time = 180 2023-05-20T01:28:34.758 INFO:teuthology.orchestra.run.smithi132.stdout: mds verify scatter = False 2023-05-20T01:28:34.758 INFO:teuthology.orchestra.run.smithi132.stdout: osd op complaint time = 180 2023-05-20T01:28:34.759 INFO:teuthology.orchestra.run.smithi132.stdout: rados mon op timeout = 900 2023-05-20T01:28:34.759 INFO:teuthology.orchestra.run.smithi132.stdout: rados osd op timeout = 900 2023-05-20T01:28:34.759 INFO:teuthology.orchestra.run.smithi132.stdout: mds bal merge size = 0 2023-05-20T01:28:34.760 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-05-20T01:28:34.762 DEBUG:teuthology.orchestra.run.smithi132:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-05-20T01:28:34.807 INFO:teuthology.orchestra.run.smithi016.stdout:[global] 2023-05-20T01:28:34.808 INFO:teuthology.orchestra.run.smithi016.stdout: chdir = "" 2023-05-20T01:28:34.808 INFO:teuthology.orchestra.run.smithi016.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-05-20T01:28:34.808 INFO:teuthology.orchestra.run.smithi016.stdout: auth supported = cephx 2023-05-20T01:28:34.808 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:28:34.808 INFO:teuthology.orchestra.run.smithi016.stdout: filestore xattr use omap = true 2023-05-20T01:28:34.808 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:28:34.808 INFO:teuthology.orchestra.run.smithi016.stdout: mon clock drift allowed = 1.000 2023-05-20T01:28:34.809 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:28:34.809 INFO:teuthology.orchestra.run.smithi016.stdout: osd crush chooseleaf type = 0 2023-05-20T01:28:34.809 INFO:teuthology.orchestra.run.smithi016.stdout: auth debug = true 2023-05-20T01:28:34.809 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:28:34.809 INFO:teuthology.orchestra.run.smithi016.stdout: ms die on old message = true 2023-05-20T01:28:34.809 INFO:teuthology.orchestra.run.smithi016.stdout: ms die on bug = true 2023-05-20T01:28:34.809 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:28:34.810 INFO:teuthology.orchestra.run.smithi016.stdout: mon max pg per osd = 10000 # >= luminous 2023-05-20T01:28:34.810 INFO:teuthology.orchestra.run.smithi016.stdout: mon pg warn max object skew = 0 2023-05-20T01:28:34.810 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:28:34.810 INFO:teuthology.orchestra.run.smithi016.stdout: # disable pg_autoscaler by default for new pools 2023-05-20T01:28:34.810 INFO:teuthology.orchestra.run.smithi016.stdout: osd_pool_default_pg_autoscale_mode = off 2023-05-20T01:28:34.810 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:28:34.810 INFO:teuthology.orchestra.run.smithi016.stdout: osd pool default size = 2 2023-05-20T01:28:34.810 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:28:34.811 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd allow primary affinity = true 2023-05-20T01:28:34.811 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd allow pg remap = true 2023-05-20T01:28:34.811 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on legacy crush tunables = false 2023-05-20T01:28:34.811 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on crush straw calc version zero = false 2023-05-20T01:28:34.811 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on no sortbitwise = false 2023-05-20T01:28:34.811 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on osd down out interval zero = false 2023-05-20T01:28:34.811 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on too few osds = false 2023-05-20T01:28:34.811 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-05-20T01:28:34.812 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_pool_no_redundancy = false 2023-05-20T01:28:34.812 INFO:teuthology.orchestra.run.smithi016.stdout: mon_allow_pool_size_one = true 2023-05-20T01:28:34.812 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:28:34.812 INFO:teuthology.orchestra.run.smithi016.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-05-20T01:28:34.812 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:28:34.812 INFO:teuthology.orchestra.run.smithi016.stdout: osd default data pool replay window = 5 2023-05-20T01:28:34.812 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:28:34.813 INFO:teuthology.orchestra.run.smithi016.stdout: mon allow pool delete = true 2023-05-20T01:28:34.813 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:28:34.813 INFO:teuthology.orchestra.run.smithi016.stdout: mon cluster log file level = debug 2023-05-20T01:28:34.813 INFO:teuthology.orchestra.run.smithi016.stdout: debug asserts on shutdown = true 2023-05-20T01:28:34.813 INFO:teuthology.orchestra.run.smithi016.stdout: mon health detail to clog = false 2023-05-20T01:28:34.813 INFO:teuthology.orchestra.run.smithi016.stdout: mon host = "172.21.15.16,172.21.15.132,[v2:172.21.15.132:3301,v1:172.21.15.132:6790]" 2023-05-20T01:28:34.813 INFO:teuthology.orchestra.run.smithi016.stdout: ms die on skipped message = False 2023-05-20T01:28:34.813 INFO:teuthology.orchestra.run.smithi016.stdout: fsid = c2245cc5-ecab-4087-a59b-bc81db901676 2023-05-20T01:28:34.814 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:28:34.814 INFO:teuthology.orchestra.run.smithi016.stdout:[osd] 2023-05-20T01:28:34.814 INFO:teuthology.orchestra.run.smithi016.stdout: osd journal size = 100 2023-05-20T01:28:34.814 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:28:34.814 INFO:teuthology.orchestra.run.smithi016.stdout: osd scrub load threshold = 5.0 2023-05-20T01:28:34.814 INFO:teuthology.orchestra.run.smithi016.stdout: osd scrub max interval = 600 2023-05-20T01:28:34.814 INFO:teuthology.orchestra.run.smithi016.stdout: osd mclock profile = high_recovery_ops 2023-05-20T01:28:34.815 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:28:34.815 INFO:teuthology.orchestra.run.smithi016.stdout: osd recover clone overlap = true 2023-05-20T01:28:34.815 INFO:teuthology.orchestra.run.smithi016.stdout: osd recovery max chunk = 1048576 2023-05-20T01:28:34.815 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:28:34.815 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug shutdown = true 2023-05-20T01:28:34.815 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug op order = true 2023-05-20T01:28:34.815 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug verify stray on activate = true 2023-05-20T01:28:34.815 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:28:34.816 INFO:teuthology.orchestra.run.smithi016.stdout: osd open classes on start = true 2023-05-20T01:28:34.816 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug pg log writeout = true 2023-05-20T01:28:34.816 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:28:34.816 INFO:teuthology.orchestra.run.smithi016.stdout: osd deep scrub update digest min age = 30 2023-05-20T01:28:34.816 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:28:34.816 INFO:teuthology.orchestra.run.smithi016.stdout: osd map max advance = 10 2023-05-20T01:28:34.816 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:28:34.816 INFO:teuthology.orchestra.run.smithi016.stdout: journal zero on create = true 2023-05-20T01:28:34.817 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:28:34.817 INFO:teuthology.orchestra.run.smithi016.stdout: filestore ondisk finisher threads = 3 2023-05-20T01:28:34.817 INFO:teuthology.orchestra.run.smithi016.stdout: filestore apply finisher threads = 3 2023-05-20T01:28:34.817 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:28:34.817 INFO:teuthology.orchestra.run.smithi016.stdout: bdev debug aio = true 2023-05-20T01:28:34.817 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug misdirected ops = true 2023-05-20T01:28:34.817 INFO:teuthology.orchestra.run.smithi016.stdout: bluestore block size = 96636764160 2023-05-20T01:28:34.818 INFO:teuthology.orchestra.run.smithi016.stdout: bluestore fsck on mount = True 2023-05-20T01:28:34.818 INFO:teuthology.orchestra.run.smithi016.stdout: debug bluefs = 20 2023-05-20T01:28:34.818 INFO:teuthology.orchestra.run.smithi016.stdout: debug bluestore = 20 2023-05-20T01:28:34.818 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2023-05-20T01:28:34.818 INFO:teuthology.orchestra.run.smithi016.stdout: debug osd = 20 2023-05-20T01:28:34.818 INFO:teuthology.orchestra.run.smithi016.stdout: debug rocksdb = 10 2023-05-20T01:28:34.818 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd backfillfull_ratio = 0.85 2023-05-20T01:28:34.818 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd full ratio = 0.9 2023-05-20T01:28:34.819 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd nearfull ratio = 0.8 2023-05-20T01:28:34.819 INFO:teuthology.orchestra.run.smithi016.stdout: osd failsafe full ratio = 0.95 2023-05-20T01:28:34.819 INFO:teuthology.orchestra.run.smithi016.stdout: osd objectstore = bluestore 2023-05-20T01:28:34.819 INFO:teuthology.orchestra.run.smithi016.stdout: osd op complaint time = 180 2023-05-20T01:28:34.819 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:28:34.819 INFO:teuthology.orchestra.run.smithi016.stdout:[mgr] 2023-05-20T01:28:34.819 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2023-05-20T01:28:34.819 INFO:teuthology.orchestra.run.smithi016.stdout: debug mgr = 20 2023-05-20T01:28:34.820 INFO:teuthology.orchestra.run.smithi016.stdout: debug mon = 20 2023-05-20T01:28:34.820 INFO:teuthology.orchestra.run.smithi016.stdout: debug auth = 20 2023-05-20T01:28:34.820 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min pgs per osd = 4 2023-05-20T01:28:34.820 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min bytes per osd = 10 2023-05-20T01:28:34.820 INFO:teuthology.orchestra.run.smithi016.stdout: mgr/telemetry/nag = false 2023-05-20T01:28:34.820 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:28:34.820 INFO:teuthology.orchestra.run.smithi016.stdout:[mon] 2023-05-20T01:28:34.820 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2023-05-20T01:28:34.821 INFO:teuthology.orchestra.run.smithi016.stdout: debug mon = 20 2023-05-20T01:28:34.821 INFO:teuthology.orchestra.run.smithi016.stdout: debug paxos = 20 2023-05-20T01:28:34.821 INFO:teuthology.orchestra.run.smithi016.stdout: debug auth = 20 2023-05-20T01:28:34.821 INFO:teuthology.orchestra.run.smithi016.stdout: mon data avail warn = 5 2023-05-20T01:28:34.821 INFO:teuthology.orchestra.run.smithi016.stdout: mon mgr mkfs grace = 240 2023-05-20T01:28:34.821 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min pgs per osd = 4 2023-05-20T01:28:34.821 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd reporter subtree level = osd 2023-05-20T01:28:34.821 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd prime pg temp = true 2023-05-20T01:28:34.822 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min bytes per osd = 10 2023-05-20T01:28:34.822 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:28:34.822 INFO:teuthology.orchestra.run.smithi016.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-05-20T01:28:34.822 INFO:teuthology.orchestra.run.smithi016.stdout: auth mon ticket ttl = 660 # 11m 2023-05-20T01:28:34.822 INFO:teuthology.orchestra.run.smithi016.stdout: auth service ticket ttl = 240 # 4m 2023-05-20T01:28:34.822 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:28:34.822 INFO:teuthology.orchestra.run.smithi016.stdout: # don't complain about insecure global_id in the test suite 2023-05-20T01:28:34.823 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-05-20T01:28:34.823 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-05-20T01:28:34.823 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:28:34.823 INFO:teuthology.orchestra.run.smithi016.stdout: # 1m isn't quite enough 2023-05-20T01:28:34.823 INFO:teuthology.orchestra.run.smithi016.stdout: mon_down_mkfs_grace = 2m 2023-05-20T01:28:34.823 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:28:34.823 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_filestore_osds = false 2023-05-20T01:28:34.823 INFO:teuthology.orchestra.run.smithi016.stdout: mon op complaint time = 120 2023-05-20T01:28:34.824 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:28:34.824 INFO:teuthology.orchestra.run.smithi016.stdout:[client] 2023-05-20T01:28:34.824 INFO:teuthology.orchestra.run.smithi016.stdout: rgw cache enabled = true 2023-05-20T01:28:34.824 INFO:teuthology.orchestra.run.smithi016.stdout: rgw enable ops log = true 2023-05-20T01:28:34.824 INFO:teuthology.orchestra.run.smithi016.stdout: rgw enable usage log = true 2023-05-20T01:28:34.824 INFO:teuthology.orchestra.run.smithi016.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-05-20T01:28:34.824 INFO:teuthology.orchestra.run.smithi016.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-05-20T01:28:34.825 INFO:teuthology.orchestra.run.smithi016.stdout: client mount timeout = 600 2023-05-20T01:28:34.825 INFO:teuthology.orchestra.run.smithi016.stdout: debug client = 20 2023-05-20T01:28:34.825 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2023-05-20T01:28:34.825 INFO:teuthology.orchestra.run.smithi016.stdout: pid file = "" 2023-05-20T01:28:34.825 INFO:teuthology.orchestra.run.smithi016.stdout: rados mon op timeout = 900 2023-05-20T01:28:34.825 INFO:teuthology.orchestra.run.smithi016.stdout: rados osd op timeout = 900 2023-05-20T01:28:34.825 INFO:teuthology.orchestra.run.smithi016.stdout:[mon.a] 2023-05-20T01:28:34.825 INFO:teuthology.orchestra.run.smithi016.stdout:[mon.b] 2023-05-20T01:28:34.826 INFO:teuthology.orchestra.run.smithi016.stdout:[mon.c] 2023-05-20T01:28:34.826 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.a] 2023-05-20T01:28:34.826 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.c] 2023-05-20T01:28:34.826 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.b] 2023-05-20T01:28:34.826 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.d] 2023-05-20T01:28:34.826 INFO:teuthology.orchestra.run.smithi016.stdout:[mds] 2023-05-20T01:28:34.826 INFO:teuthology.orchestra.run.smithi016.stdout: debug mds = 20 2023-05-20T01:28:34.827 INFO:teuthology.orchestra.run.smithi016.stdout: debug mds balancer = 20 2023-05-20T01:28:34.827 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2023-05-20T01:28:34.827 INFO:teuthology.orchestra.run.smithi016.stdout: mds debug frag = True 2023-05-20T01:28:34.827 INFO:teuthology.orchestra.run.smithi016.stdout: mds debug scatterstat = False 2023-05-20T01:28:34.827 INFO:teuthology.orchestra.run.smithi016.stdout: mds op complaint time = 180 2023-05-20T01:28:34.827 INFO:teuthology.orchestra.run.smithi016.stdout: mds verify scatter = False 2023-05-20T01:28:34.827 INFO:teuthology.orchestra.run.smithi016.stdout: osd op complaint time = 180 2023-05-20T01:28:34.827 INFO:teuthology.orchestra.run.smithi016.stdout: rados mon op timeout = 900 2023-05-20T01:28:34.828 INFO:teuthology.orchestra.run.smithi016.stdout: rados osd op timeout = 900 2023-05-20T01:28:34.828 INFO:teuthology.orchestra.run.smithi016.stdout: mds bal merge size = 0 2023-05-20T01:28:34.828 INFO:teuthology.orchestra.run.smithi016.stdout: mds bal split size = 10000 2023-05-20T01:28:34.829 INFO:teuthology.orchestra.run.smithi132.stdout:[global] 2023-05-20T01:28:34.829 INFO:teuthology.orchestra.run.smithi132.stdout: chdir = "" 2023-05-20T01:28:34.829 INFO:teuthology.orchestra.run.smithi132.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-05-20T01:28:34.830 INFO:teuthology.orchestra.run.smithi132.stdout: auth supported = cephx 2023-05-20T01:28:34.830 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:28:34.830 INFO:teuthology.orchestra.run.smithi132.stdout: filestore xattr use omap = true 2023-05-20T01:28:34.830 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:28:34.830 INFO:teuthology.orchestra.run.smithi132.stdout: mon clock drift allowed = 1.000 2023-05-20T01:28:34.830 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:28:34.830 INFO:teuthology.orchestra.run.smithi132.stdout: osd crush chooseleaf type = 0 2023-05-20T01:28:34.831 INFO:teuthology.orchestra.run.smithi132.stdout: auth debug = true 2023-05-20T01:28:34.831 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:28:34.831 INFO:teuthology.orchestra.run.smithi132.stdout: ms die on old message = true 2023-05-20T01:28:34.831 INFO:teuthology.orchestra.run.smithi132.stdout: ms die on bug = true 2023-05-20T01:28:34.831 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:28:34.831 INFO:teuthology.orchestra.run.smithi132.stdout: mon max pg per osd = 10000 # >= luminous 2023-05-20T01:28:34.832 INFO:teuthology.orchestra.run.smithi132.stdout: mon pg warn max object skew = 0 2023-05-20T01:28:34.832 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:28:34.832 INFO:teuthology.orchestra.run.smithi132.stdout: # disable pg_autoscaler by default for new pools 2023-05-20T01:28:34.832 INFO:teuthology.orchestra.run.smithi132.stdout: osd_pool_default_pg_autoscale_mode = off 2023-05-20T01:28:34.832 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:28:34.833 INFO:teuthology.orchestra.run.smithi132.stdout: osd pool default size = 2 2023-05-20T01:28:34.833 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:28:34.833 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd allow primary affinity = true 2023-05-20T01:28:34.833 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd allow pg remap = true 2023-05-20T01:28:34.833 INFO:teuthology.orchestra.run.smithi132.stdout: mon warn on legacy crush tunables = false 2023-05-20T01:28:34.833 INFO:teuthology.orchestra.run.smithi132.stdout: mon warn on crush straw calc version zero = false 2023-05-20T01:28:34.833 INFO:teuthology.orchestra.run.smithi132.stdout: mon warn on no sortbitwise = false 2023-05-20T01:28:34.833 INFO:teuthology.orchestra.run.smithi132.stdout: mon warn on osd down out interval zero = false 2023-05-20T01:28:34.834 INFO:teuthology.orchestra.run.smithi132.stdout: mon warn on too few osds = false 2023-05-20T01:28:34.834 INFO:teuthology.orchestra.run.smithi132.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-05-20T01:28:34.834 INFO:teuthology.orchestra.run.smithi132.stdout: mon_warn_on_pool_no_redundancy = false 2023-05-20T01:28:34.834 INFO:teuthology.orchestra.run.smithi132.stdout: mon_allow_pool_size_one = true 2023-05-20T01:28:34.834 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:28:34.834 INFO:teuthology.orchestra.run.smithi132.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-05-20T01:28:34.834 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:28:34.834 INFO:teuthology.orchestra.run.smithi132.stdout: osd default data pool replay window = 5 2023-05-20T01:28:34.835 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:28:34.835 INFO:teuthology.orchestra.run.smithi132.stdout: mon allow pool delete = true 2023-05-20T01:28:34.835 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:28:34.835 INFO:teuthology.orchestra.run.smithi132.stdout: mon cluster log file level = debug 2023-05-20T01:28:34.835 INFO:teuthology.orchestra.run.smithi132.stdout: debug asserts on shutdown = true 2023-05-20T01:28:34.835 INFO:teuthology.orchestra.run.smithi132.stdout: mon health detail to clog = false 2023-05-20T01:28:34.835 INFO:teuthology.orchestra.run.smithi132.stdout: mon host = "172.21.15.16,172.21.15.132,[v2:172.21.15.132:3301,v1:172.21.15.132:6790]" 2023-05-20T01:28:34.835 INFO:teuthology.orchestra.run.smithi132.stdout: ms die on skipped message = False 2023-05-20T01:28:34.836 INFO:teuthology.orchestra.run.smithi132.stdout: fsid = c2245cc5-ecab-4087-a59b-bc81db901676 2023-05-20T01:28:34.836 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:28:34.836 INFO:teuthology.orchestra.run.smithi132.stdout:[osd] 2023-05-20T01:28:34.836 INFO:teuthology.orchestra.run.smithi132.stdout: osd journal size = 100 2023-05-20T01:28:34.836 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:28:34.836 INFO:teuthology.orchestra.run.smithi132.stdout: osd scrub load threshold = 5.0 2023-05-20T01:28:34.836 INFO:teuthology.orchestra.run.smithi132.stdout: osd scrub max interval = 600 2023-05-20T01:28:34.837 INFO:teuthology.orchestra.run.smithi132.stdout: osd mclock profile = high_recovery_ops 2023-05-20T01:28:34.837 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:28:34.837 INFO:teuthology.orchestra.run.smithi132.stdout: osd recover clone overlap = true 2023-05-20T01:28:34.837 INFO:teuthology.orchestra.run.smithi132.stdout: osd recovery max chunk = 1048576 2023-05-20T01:28:34.837 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:28:34.837 INFO:teuthology.orchestra.run.smithi132.stdout: osd debug shutdown = true 2023-05-20T01:28:34.837 INFO:teuthology.orchestra.run.smithi132.stdout: osd debug op order = true 2023-05-20T01:28:34.838 INFO:teuthology.orchestra.run.smithi132.stdout: osd debug verify stray on activate = true 2023-05-20T01:28:34.838 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:28:34.838 INFO:teuthology.orchestra.run.smithi132.stdout: osd open classes on start = true 2023-05-20T01:28:34.838 INFO:teuthology.orchestra.run.smithi132.stdout: osd debug pg log writeout = true 2023-05-20T01:28:34.838 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:28:34.838 INFO:teuthology.orchestra.run.smithi132.stdout: osd deep scrub update digest min age = 30 2023-05-20T01:28:34.838 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:28:34.838 INFO:teuthology.orchestra.run.smithi132.stdout: osd map max advance = 10 2023-05-20T01:28:34.839 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:28:34.839 INFO:teuthology.orchestra.run.smithi132.stdout: journal zero on create = true 2023-05-20T01:28:34.839 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:28:34.839 INFO:teuthology.orchestra.run.smithi132.stdout: filestore ondisk finisher threads = 3 2023-05-20T01:28:34.839 INFO:teuthology.orchestra.run.smithi132.stdout: filestore apply finisher threads = 3 2023-05-20T01:28:34.839 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:28:34.839 INFO:teuthology.orchestra.run.smithi132.stdout: bdev debug aio = true 2023-05-20T01:28:34.839 INFO:teuthology.orchestra.run.smithi132.stdout: osd debug misdirected ops = true 2023-05-20T01:28:34.840 INFO:teuthology.orchestra.run.smithi132.stdout: bluestore block size = 96636764160 2023-05-20T01:28:34.840 INFO:teuthology.orchestra.run.smithi132.stdout: bluestore fsck on mount = True 2023-05-20T01:28:34.840 INFO:teuthology.orchestra.run.smithi132.stdout: debug bluefs = 20 2023-05-20T01:28:34.840 INFO:teuthology.orchestra.run.smithi132.stdout: debug bluestore = 20 2023-05-20T01:28:34.840 INFO:teuthology.orchestra.run.smithi132.stdout: debug ms = 1 2023-05-20T01:28:34.840 INFO:teuthology.orchestra.run.smithi132.stdout: debug osd = 20 2023-05-20T01:28:34.840 INFO:teuthology.orchestra.run.smithi132.stdout: debug rocksdb = 10 2023-05-20T01:28:34.840 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd backfillfull_ratio = 0.85 2023-05-20T01:28:34.841 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd full ratio = 0.9 2023-05-20T01:28:34.841 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd nearfull ratio = 0.8 2023-05-20T01:28:34.841 INFO:teuthology.orchestra.run.smithi132.stdout: osd failsafe full ratio = 0.95 2023-05-20T01:28:34.841 INFO:teuthology.orchestra.run.smithi132.stdout: osd objectstore = bluestore 2023-05-20T01:28:34.841 INFO:teuthology.orchestra.run.smithi132.stdout: osd op complaint time = 180 2023-05-20T01:28:34.841 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:28:34.841 INFO:teuthology.orchestra.run.smithi132.stdout:[mgr] 2023-05-20T01:28:34.841 INFO:teuthology.orchestra.run.smithi132.stdout: debug ms = 1 2023-05-20T01:28:34.842 INFO:teuthology.orchestra.run.smithi132.stdout: debug mgr = 20 2023-05-20T01:28:34.842 INFO:teuthology.orchestra.run.smithi132.stdout: debug mon = 20 2023-05-20T01:28:34.842 INFO:teuthology.orchestra.run.smithi132.stdout: debug auth = 20 2023-05-20T01:28:34.842 INFO:teuthology.orchestra.run.smithi132.stdout: mon reweight min pgs per osd = 4 2023-05-20T01:28:34.842 INFO:teuthology.orchestra.run.smithi132.stdout: mon reweight min bytes per osd = 10 2023-05-20T01:28:34.842 INFO:teuthology.orchestra.run.smithi132.stdout: mgr/telemetry/nag = false 2023-05-20T01:28:34.842 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:28:34.842 INFO:teuthology.orchestra.run.smithi132.stdout:[mon] 2023-05-20T01:28:34.843 INFO:teuthology.orchestra.run.smithi132.stdout: debug ms = 1 2023-05-20T01:28:34.843 INFO:teuthology.orchestra.run.smithi132.stdout: debug mon = 20 2023-05-20T01:28:34.843 INFO:teuthology.orchestra.run.smithi132.stdout: debug paxos = 20 2023-05-20T01:28:34.843 INFO:teuthology.orchestra.run.smithi132.stdout: debug auth = 20 2023-05-20T01:28:34.843 INFO:teuthology.orchestra.run.smithi132.stdout: mon data avail warn = 5 2023-05-20T01:28:34.843 INFO:teuthology.orchestra.run.smithi132.stdout: mon mgr mkfs grace = 240 2023-05-20T01:28:34.843 INFO:teuthology.orchestra.run.smithi132.stdout: mon reweight min pgs per osd = 4 2023-05-20T01:28:34.843 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd reporter subtree level = osd 2023-05-20T01:28:34.844 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd prime pg temp = true 2023-05-20T01:28:34.844 INFO:teuthology.orchestra.run.smithi132.stdout: mon reweight min bytes per osd = 10 2023-05-20T01:28:34.844 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:28:34.844 INFO:teuthology.orchestra.run.smithi132.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-05-20T01:28:34.844 INFO:teuthology.orchestra.run.smithi132.stdout: auth mon ticket ttl = 660 # 11m 2023-05-20T01:28:34.844 INFO:teuthology.orchestra.run.smithi132.stdout: auth service ticket ttl = 240 # 4m 2023-05-20T01:28:34.844 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:28:34.845 INFO:teuthology.orchestra.run.smithi132.stdout: # don't complain about insecure global_id in the test suite 2023-05-20T01:28:34.845 INFO:teuthology.orchestra.run.smithi132.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-05-20T01:28:34.845 INFO:teuthology.orchestra.run.smithi132.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-05-20T01:28:34.845 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:28:34.845 INFO:teuthology.orchestra.run.smithi132.stdout: # 1m isn't quite enough 2023-05-20T01:28:34.845 INFO:teuthology.orchestra.run.smithi132.stdout: mon_down_mkfs_grace = 2m 2023-05-20T01:28:34.845 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:28:34.845 INFO:teuthology.orchestra.run.smithi132.stdout: mon_warn_on_filestore_osds = false 2023-05-20T01:28:34.846 INFO:teuthology.orchestra.run.smithi132.stdout: mon op complaint time = 120 2023-05-20T01:28:34.846 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-05-20T01:28:34.846 INFO:teuthology.orchestra.run.smithi132.stdout:[client] 2023-05-20T01:28:34.846 INFO:teuthology.orchestra.run.smithi132.stdout: rgw cache enabled = true 2023-05-20T01:28:34.846 INFO:teuthology.orchestra.run.smithi132.stdout: rgw enable ops log = true 2023-05-20T01:28:34.846 INFO:teuthology.orchestra.run.smithi132.stdout: rgw enable usage log = true 2023-05-20T01:28:34.846 INFO:teuthology.orchestra.run.smithi132.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-05-20T01:28:34.846 INFO:teuthology.orchestra.run.smithi132.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-05-20T01:28:34.847 INFO:teuthology.orchestra.run.smithi132.stdout: client mount timeout = 600 2023-05-20T01:28:34.847 INFO:teuthology.orchestra.run.smithi132.stdout: debug client = 20 2023-05-20T01:28:34.847 INFO:teuthology.orchestra.run.smithi132.stdout: debug ms = 1 2023-05-20T01:28:34.847 INFO:teuthology.orchestra.run.smithi132.stdout: pid file = "" 2023-05-20T01:28:34.847 INFO:teuthology.orchestra.run.smithi132.stdout: rados mon op timeout = 900 2023-05-20T01:28:34.847 INFO:teuthology.orchestra.run.smithi132.stdout: rados osd op timeout = 900 2023-05-20T01:28:34.847 INFO:teuthology.orchestra.run.smithi132.stdout:[mon.a] 2023-05-20T01:28:34.847 INFO:teuthology.orchestra.run.smithi132.stdout:[mon.b] 2023-05-20T01:28:34.848 INFO:teuthology.orchestra.run.smithi132.stdout:[mon.c] 2023-05-20T01:28:34.848 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.a] 2023-05-20T01:28:34.848 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.c] 2023-05-20T01:28:34.848 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.b] 2023-05-20T01:28:34.848 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.d] 2023-05-20T01:28:34.848 INFO:teuthology.orchestra.run.smithi132.stdout:[mds] 2023-05-20T01:28:34.848 INFO:teuthology.orchestra.run.smithi132.stdout: debug mds = 20 2023-05-20T01:28:34.848 INFO:teuthology.orchestra.run.smithi132.stdout: debug mds balancer = 20 2023-05-20T01:28:34.849 INFO:teuthology.orchestra.run.smithi132.stdout: debug ms = 1 2023-05-20T01:28:34.849 INFO:teuthology.orchestra.run.smithi132.stdout: mds debug frag = True 2023-05-20T01:28:34.849 INFO:teuthology.orchestra.run.smithi132.stdout: mds debug scatterstat = False 2023-05-20T01:28:34.849 INFO:teuthology.orchestra.run.smithi132.stdout: mds op complaint time = 180 2023-05-20T01:28:34.849 INFO:teuthology.orchestra.run.smithi132.stdout: mds verify scatter = False 2023-05-20T01:28:34.849 INFO:teuthology.orchestra.run.smithi132.stdout: osd op complaint time = 180 2023-05-20T01:28:34.849 INFO:teuthology.orchestra.run.smithi132.stdout: rados mon op timeout = 900 2023-05-20T01:28:34.850 INFO:teuthology.orchestra.run.smithi132.stdout: rados osd op timeout = 900 2023-05-20T01:28:34.850 INFO:teuthology.orchestra.run.smithi132.stdout: mds bal merge size = 0 2023-05-20T01:28:34.850 INFO:teuthology.orchestra.run.smithi132.stdout: mds bal split size = 10000 2023-05-20T01:28:34.850 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits daemon-helper kill python3 -c ' 2023-05-20T01:28:34.851 DEBUG:teuthology.orchestra.run.smithi016:> import os 2023-05-20T01:28:34.851 DEBUG:teuthology.orchestra.run.smithi016:> path = os.path.join("/home/ubuntu/cephtest/mnt.0", "subdir") 2023-05-20T01:28:34.851 DEBUG:teuthology.orchestra.run.smithi016:> os.mkdir(path) 2023-05-20T01:28:34.851 DEBUG:teuthology.orchestra.run.smithi016:> for n in range(0, 100): 2023-05-20T01:28:34.852 DEBUG:teuthology.orchestra.run.smithi016:> open(os.path.join(path, "%s" % n), '"'"'w'"'"').write("%s" % n) 2023-05-20T01:28:34.852 DEBUG:teuthology.orchestra.run.smithi016:> ' 2023-05-20T01:28:35.108 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits daemon-helper kill python3 -c ' 2023-05-20T01:28:35.109 DEBUG:teuthology.orchestra.run.smithi016:> import os 2023-05-20T01:28:35.109 DEBUG:teuthology.orchestra.run.smithi016:> import stat 2023-05-20T01:28:35.109 DEBUG:teuthology.orchestra.run.smithi016:> 2023-05-20T01:28:35.110 DEBUG:teuthology.orchestra.run.smithi016:> print(os.stat("/home/ubuntu/cephtest/mnt.0/subdir").st_ino) 2023-05-20T01:28:35.110 DEBUG:teuthology.orchestra.run.smithi016:> ' 2023-05-20T01:28:35.216 INFO:teuthology.orchestra.run.smithi016.stdout:1099511627776 2023-05-20T01:28:35.405 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-20T01:28:35.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.567+0000 7f0579c4a700 1 Processor -- start 2023-05-20T01:28:35.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.568+0000 7f0579c4a700 1 -- start start 2023-05-20T01:28:35.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.568+0000 7f0579c4a700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f0574072460 0x7f057406a460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:35.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.569+0000 7f0579c4a700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0574071fd0 con 0x7f057406a9a0 2023-05-20T01:28:35.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.569+0000 7f0579c4a700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05740713c0 con 0x7f057415cb40 2023-05-20T01:28:35.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.569+0000 7f0579c4a700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f057406ae20 con 0x7f0574072460 2023-05-20T01:28:35.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.569+0000 7f0573fff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f057406a9a0 0x7f05740c3c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51772/0 (socket says 172.21.15.16:51772) 2023-05-20T01:28:35.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.569+0000 7f0573fff700 1 -- 172.21.15.16:0/2979308667 learned_addr learned my addr 172.21.15.16:0/2979308667 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:28:35.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.569+0000 7f0572ffd700 1 --2- 172.21.15.16:0/2979308667 >> v2:172.21.15.132:3300/0 conn(0x7f0574072460 0x7f057406a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:35.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.569+0000 7f05727fc700 1 -- 172.21.15.16:0/2979308667 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1806344397 0 0) 0x7f0574071fd0 con 0x7f057406a9a0 2023-05-20T01:28:35.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.569+0000 7f05727fc700 1 -- 172.21.15.16:0/2979308667 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f055c0035b0 con 0x7f057406a9a0 2023-05-20T01:28:35.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.569+0000 7f05727fc700 1 -- 172.21.15.16:0/2979308667 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3309356272 0 0) 0x7f05740713c0 con 0x7f057415cb40 2023-05-20T01:28:35.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.569+0000 7f05727fc700 1 -- 172.21.15.16:0/2979308667 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0574071fd0 con 0x7f057415cb40 2023-05-20T01:28:35.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.569+0000 7f0572ffd700 1 -- 172.21.15.16:0/2979308667 >> v1:172.21.15.132:6789/0 conn(0x7f057415cb40 legacy=0x7f057406f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:35.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.569+0000 7f0572ffd700 1 -- 172.21.15.16:0/2979308667 >> v1:172.21.15.16:6789/0 conn(0x7f057406a9a0 legacy=0x7f05740c3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:35.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.569+0000 7f0572ffd700 1 -- 172.21.15.16:0/2979308667 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05740716a0 con 0x7f0574072460 2023-05-20T01:28:35.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.569+0000 7f0572ffd700 1 --2- 172.21.15.16:0/2979308667 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0574072460 0x7f057406a460 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f0568009840 tx=0x7f0568009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f9301ad3a5b5b3a server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:35.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.569+0000 7f05727fc700 1 -- 172.21.15.16:0/2979308667 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0568011830 con 0x7f0574072460 2023-05-20T01:28:35.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.570+0000 7f05727fc700 1 -- 172.21.15.16:0/2979308667 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0568011990 con 0x7f0574072460 2023-05-20T01:28:35.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.570+0000 7f05727fc700 1 -- 172.21.15.16:0/2979308667 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0568011d20 con 0x7f0574072460 2023-05-20T01:28:35.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.570+0000 7f0579c4a700 1 -- 172.21.15.16:0/2979308667 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0574072460 msgr2=0x7f057406a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:35.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.570+0000 7f0579c4a700 1 --2- 172.21.15.16:0/2979308667 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0574072460 0x7f057406a460 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f0568009840 tx=0x7f0568009b50 comp rx=0 tx=0).stop 2023-05-20T01:28:35.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.570+0000 7f0579c4a700 1 -- 172.21.15.16:0/2979308667 shutdown_connections 2023-05-20T01:28:35.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.570+0000 7f0579c4a700 1 --2- 172.21.15.16:0/2979308667 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0574072460 0x7f057406a460 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:35.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.570+0000 7f0579c4a700 1 -- 172.21.15.16:0/2979308667 >> 172.21.15.16:0/2979308667 conn(0x7f05740b1c90 msgr2=0x7f05740b2060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:35.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.570+0000 7f0579c4a700 1 -- 172.21.15.16:0/2979308667 shutdown_connections 2023-05-20T01:28:35.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.570+0000 7f0579c4a700 1 -- 172.21.15.16:0/2979308667 wait complete. 2023-05-20T01:28:35.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.572+0000 7f0579c4a700 1 Processor -- start 2023-05-20T01:28:35.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.572+0000 7f0579c4a700 1 -- start start 2023-05-20T01:28:35.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.573+0000 7f0579c4a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f057406a9a0 0x7f05741567b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:35.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.573+0000 7f0579c4a700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0574072460 0x7f05740c5c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:35.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.573+0000 7f0579c4a700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f057415cb40 0x7f05741f3960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:35.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.573+0000 7f0579c4a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05741586f0 con 0x7f057406a9a0 2023-05-20T01:28:35.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.573+0000 7f0579c4a700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0574158990 con 0x7f057415cb40 2023-05-20T01:28:35.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.573+0000 7f0579c4a700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0574158c70 con 0x7f0574072460 2023-05-20T01:28:35.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.573+0000 7f05737fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f057406a9a0 0x7f05741567b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:35.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.573+0000 7f05737fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f057406a9a0 0x7f05741567b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40246/0 (socket says 172.21.15.16:40246) 2023-05-20T01:28:35.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.573+0000 7f05737fe700 1 -- 172.21.15.16:0/2056758273 learned_addr learned my addr 172.21.15.16:0/2056758273 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:35.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.573+0000 7f0573fff700 1 --2- 172.21.15.16:0/2056758273 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f057415cb40 0x7f05741f3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:35.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.573+0000 7f0572ffd700 1 --2- 172.21.15.16:0/2056758273 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0574072460 0x7f05740c5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:35.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.573+0000 7f05737fe700 1 -- 172.21.15.16:0/2056758273 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0574072460 msgr2=0x7f05740c5c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:35.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.573+0000 7f05737fe700 1 --2- 172.21.15.16:0/2056758273 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0574072460 0x7f05740c5c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:35.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.573+0000 7f05737fe700 1 -- 172.21.15.16:0/2056758273 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f057415cb40 msgr2=0x7f05741f3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:35.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.573+0000 7f05737fe700 1 --2- 172.21.15.16:0/2056758273 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f057415cb40 0x7f05741f3960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:35.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.573+0000 7f05737fe700 1 -- 172.21.15.16:0/2056758273 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05680094a0 con 0x7f057406a9a0 2023-05-20T01:28:35.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.573+0000 7f0573fff700 1 --2- 172.21.15.16:0/2056758273 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f057415cb40 0x7f05741f3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:28:35.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.573+0000 7f05737fe700 1 --2- 172.21.15.16:0/2056758273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f057406a9a0 0x7f05741567b0 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f056400c520 tx=0x7f056400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:35.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.574+0000 7f055bfff700 1 -- 172.21.15.16:0/2056758273 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05640052d0 con 0x7f057406a9a0 2023-05-20T01:28:35.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.574+0000 7f055bfff700 1 -- 172.21.15.16:0/2056758273 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0564005430 con 0x7f057406a9a0 2023-05-20T01:28:35.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.574+0000 7f0579c4a700 1 -- 172.21.15.16:0/2056758273 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0574071c60 con 0x7f057406a9a0 2023-05-20T01:28:35.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.574+0000 7f055bfff700 1 -- 172.21.15.16:0/2056758273 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05640038c0 con 0x7f057406a9a0 2023-05-20T01:28:35.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.574+0000 7f0579c4a700 1 -- 172.21.15.16:0/2056758273 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0574071980 con 0x7f057406a9a0 2023-05-20T01:28:35.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.575+0000 7f0579c4a700 1 -- 172.21.15.16:0/2056758273 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05740cfea0 con 0x7f057406a9a0 2023-05-20T01:28:35.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.575+0000 7f055bfff700 1 -- 172.21.15.16:0/2056758273 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0564003a20 con 0x7f057406a9a0 2023-05-20T01:28:35.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.575+0000 7f055bfff700 1 --2- 172.21.15.16:0/2056758273 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f056006ace0 0x7f056006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:35.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.576+0000 7f055bfff700 1 -- 172.21.15.16:0/2056758273 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f0564088b80 con 0x7f057406a9a0 2023-05-20T01:28:35.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.576+0000 7f0572ffd700 1 --2- 172.21.15.16:0/2056758273 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f056006ace0 0x7f056006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:35.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.576+0000 7f0572ffd700 1 --2- 172.21.15.16:0/2056758273 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f056006ace0 0x7f056006d190 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f0568009b90 tx=0x7f056801c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:35.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.580+0000 7f055bfff700 1 -- 172.21.15.16:0/2056758273 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f0564053b00 con 0x7f057406a9a0 2023-05-20T01:28:35.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.821+0000 7f0579c4a700 1 -- 172.21.15.16:0/2056758273 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f057404e8c0 con 0x7f057406a9a0 2023-05-20T01:28:35.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.822+0000 7f055bfff700 1 -- 172.21.15.16:0/2056758273 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) v1 ==== 94+0+4679 (secure 0 0 0) 0x7f0564057120 con 0x7f057406a9a0 2023-05-20T01:28:35.827 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:28:35.827 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":99,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7043,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/1859755557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1859755557},{"type":"v1","addr":"172.21.15.16:6836","nonce":1859755557}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94},{"gid":7046,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/2654783598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":2654783598},{"type":"v1","addr":"172.21.15.132:6834","nonce":2654783598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94},{"gid":7049,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2042939723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":2042939723},{"type":"v1","addr":"172.21.15.132:6835","nonce":2042939723}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-20T01:28:27.093957+0000","modified":"2023-05-20T01:28:30.436870+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8703},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8703":{"gid":8703,"name":"a","rank":0,"incarnation":95,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6837/2295684907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":2295684907},{"type":"v1","addr":"172.21.15.16:6837","nonce":2295684907}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-05-20T01:28:35.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.824+0000 7f0579c4a700 1 -- 172.21.15.16:0/2056758273 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f056006ace0 msgr2=0x7f056006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:35.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.824+0000 7f0579c4a700 1 --2- 172.21.15.16:0/2056758273 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f056006ace0 0x7f056006d190 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f0568009b90 tx=0x7f056801c040 comp rx=0 tx=0).stop 2023-05-20T01:28:35.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.824+0000 7f0579c4a700 1 -- 172.21.15.16:0/2056758273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f057406a9a0 msgr2=0x7f05741567b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:35.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.824+0000 7f0579c4a700 1 --2- 172.21.15.16:0/2056758273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f057406a9a0 0x7f05741567b0 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f056400c520 tx=0x7f056400c8e0 comp rx=0 tx=0).stop 2023-05-20T01:28:35.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.824+0000 7f0579c4a700 1 -- 172.21.15.16:0/2056758273 shutdown_connections 2023-05-20T01:28:35.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.824+0000 7f0579c4a700 1 --2- 172.21.15.16:0/2056758273 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f057415cb40 0x7f05741f3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:35.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.824+0000 7f0579c4a700 1 --2- 172.21.15.16:0/2056758273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f057406a9a0 0x7f05741567b0 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:35.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.824+0000 7f0579c4a700 1 --2- 172.21.15.16:0/2056758273 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f056006ace0 0x7f056006d190 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:35.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.824+0000 7f0579c4a700 1 --2- 172.21.15.16:0/2056758273 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0574072460 0x7f05740c5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:35.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.824+0000 7f0579c4a700 1 -- 172.21.15.16:0/2056758273 >> 172.21.15.16:0/2056758273 conn(0x7f05740b1c90 msgr2=0x7f05740cf110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:35.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.825+0000 7f0579c4a700 1 -- 172.21.15.16:0/2056758273 shutdown_connections 2023-05-20T01:28:35.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.825+0000 7f0579c4a700 1 -- 172.21.15.16:0/2056758273 wait complete. 2023-05-20T01:28:35.831 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 99 2023-05-20T01:28:35.841 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-20T01:28:35.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.962+0000 7f5ffd843700 1 Processor -- start 2023-05-20T01:28:35.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.962+0000 7f5ffd843700 1 -- start start 2023-05-20T01:28:35.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.963+0000 7f5ffd843700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5ff815ce80 0x7f5ff80d2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:35.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.963+0000 7f5ffd843700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f5ff80d3fe0 0x7f5ff80cf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:35.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.963+0000 7f5ffd843700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ff80d2b50 con 0x7f5ff80b4fb0 2023-05-20T01:28:35.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.963+0000 7f5ffd843700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ff80d4470 con 0x7f5ff815ce80 2023-05-20T01:28:35.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.963+0000 7f5ffd843700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ff815d250 con 0x7f5ff80d3fe0 2023-05-20T01:28:35.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.963+0000 7f5ff6ffd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5ff815ce80 0x7f5ff80d2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:35.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.963+0000 7f5ff77fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f5ff80b4fb0 0x7f5ff80b6380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51786/0 (socket says 172.21.15.16:51786) 2023-05-20T01:28:35.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.963+0000 7f5ff77fe700 1 -- 172.21.15.16:0/2414505446 learned_addr learned my addr 172.21.15.16:0/2414505446 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:28:35.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.963+0000 7f5ff67fc700 1 --2- 172.21.15.16:0/2414505446 >> v2:172.21.15.132:3300/0 conn(0x7f5ff80d3fe0 0x7f5ff80cf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:35.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.963+0000 7f5ff5ffb700 1 -- 172.21.15.16:0/2414505446 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1068082572 0 0) 0x7f5ff80d2b50 con 0x7f5ff80b4fb0 2023-05-20T01:28:35.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.963+0000 7f5ff5ffb700 1 -- 172.21.15.16:0/2414505446 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5fe00035b0 con 0x7f5ff80b4fb0 2023-05-20T01:28:35.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.963+0000 7f5ff6ffd700 1 -- 172.21.15.16:0/2414505446 >> v2:172.21.15.132:3300/0 conn(0x7f5ff80d3fe0 msgr2=0x7f5ff80cf980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:35.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.963+0000 7f5ff6ffd700 1 --2- 172.21.15.16:0/2414505446 >> v2:172.21.15.132:3300/0 conn(0x7f5ff80d3fe0 0x7f5ff80cf980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:35.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.963+0000 7f5ff6ffd700 1 -- 172.21.15.16:0/2414505446 >> v1:172.21.15.16:6789/0 conn(0x7f5ff80b4fb0 legacy=0x7f5ff80b6380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:35.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.963+0000 7f5ff6ffd700 1 -- 172.21.15.16:0/2414505446 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ff80d2220 con 0x7f5ff815ce80 2023-05-20T01:28:35.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.964+0000 7f5ff6ffd700 1 --2- 172.21.15.16:0/2414505446 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ff815ce80 0x7f5ff80d2fe0 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f5ff81608d0 tx=0x7f5fe8009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76a3562b6050ed03 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:35.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.964+0000 7f5ff5ffb700 1 -- 172.21.15.16:0/2414505446 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe8011830 con 0x7f5ff815ce80 2023-05-20T01:28:35.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.964+0000 7f5ff5ffb700 1 -- 172.21.15.16:0/2414505446 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fe8011990 con 0x7f5ff815ce80 2023-05-20T01:28:35.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.964+0000 7f5ff5ffb700 1 -- 172.21.15.16:0/2414505446 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe8011d20 con 0x7f5ff815ce80 2023-05-20T01:28:35.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.964+0000 7f5ffd843700 1 -- 172.21.15.16:0/2414505446 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ff815ce80 msgr2=0x7f5ff80d2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:35.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.964+0000 7f5ffd843700 1 --2- 172.21.15.16:0/2414505446 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ff815ce80 0x7f5ff80d2fe0 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f5ff81608d0 tx=0x7f5fe8009b20 comp rx=0 tx=0).stop 2023-05-20T01:28:35.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.964+0000 7f5ffd843700 1 -- 172.21.15.16:0/2414505446 shutdown_connections 2023-05-20T01:28:35.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.964+0000 7f5ffd843700 1 --2- 172.21.15.16:0/2414505446 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ff815ce80 0x7f5ff80d2fe0 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:35.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.964+0000 7f5ffd843700 1 --2- 172.21.15.16:0/2414505446 >> v2:172.21.15.132:3300/0 conn(0x7f5ff80d3fe0 0x7f5ff80cf980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:35.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.964+0000 7f5ffd843700 1 -- 172.21.15.16:0/2414505446 >> 172.21.15.16:0/2414505446 conn(0x7f5ff809c800 msgr2=0x7f5ff80d3530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:35.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.964+0000 7f5ffd843700 1 -- 172.21.15.16:0/2414505446 shutdown_connections 2023-05-20T01:28:35.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.964+0000 7f5ffd843700 1 -- 172.21.15.16:0/2414505446 wait complete. 2023-05-20T01:28:35.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.966+0000 7f5ffd843700 1 Processor -- start 2023-05-20T01:28:35.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.966+0000 7f5ffd843700 1 -- start start 2023-05-20T01:28:35.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.966+0000 7f5ffd843700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5ff80b4fb0 0x7f5ff81f9b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:35.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.966+0000 7f5ffd843700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ff80d3fe0 0x7f5ff80b8980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:35.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.966+0000 7f5ffd843700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5ff815ce80 0x7f5ff80b8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:35.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.967+0000 7f5ffd843700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff81fa040 con 0x7f5ff80d3fe0 2023-05-20T01:28:35.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.967+0000 7f5ffd843700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff81f6fb0 con 0x7f5ff80b4fb0 2023-05-20T01:28:35.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.967+0000 7f5ffd843700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff80b6f40 con 0x7f5ff815ce80 2023-05-20T01:28:35.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.967+0000 7f5ff67fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ff80d3fe0 0x7f5ff80b8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:35.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.967+0000 7f5ff67fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ff80d3fe0 0x7f5ff80b8980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40262/0 (socket says 172.21.15.16:40262) 2023-05-20T01:28:35.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.967+0000 7f5ff67fc700 1 -- 172.21.15.16:0/1907003900 learned_addr learned my addr 172.21.15.16:0/1907003900 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:35.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.967+0000 7f5ff67fc700 1 -- 172.21.15.16:0/1907003900 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5ff815ce80 msgr2=0x7f5ff80b8ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:35.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.967+0000 7f5ff67fc700 1 --2- 172.21.15.16:0/1907003900 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5ff815ce80 0x7f5ff80b8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:35.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.967+0000 7f5ff67fc700 1 -- 172.21.15.16:0/1907003900 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5ff80b4fb0 msgr2=0x7f5ff81f9b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:35.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.967+0000 7f5ff67fc700 1 --2- 172.21.15.16:0/1907003900 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5ff80b4fb0 0x7f5ff81f9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:35.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.967+0000 7f5ff67fc700 1 -- 172.21.15.16:0/1907003900 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fe80094a0 con 0x7f5ff80d3fe0 2023-05-20T01:28:35.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.967+0000 7f5ff67fc700 1 --2- 172.21.15.16:0/1907003900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ff80d3fe0 0x7f5ff80b8980 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f5fec00e9b0 tx=0x7f5fec00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:35.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.967+0000 7f5fdffff700 1 -- 172.21.15.16:0/1907003900 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fec009710 con 0x7f5ff80d3fe0 2023-05-20T01:28:35.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.967+0000 7f5fdffff700 1 -- 172.21.15.16:0/1907003900 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fec00c850 con 0x7f5ff80d3fe0 2023-05-20T01:28:35.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.967+0000 7f5ffd843700 1 -- 172.21.15.16:0/1907003900 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ff80d27e0 con 0x7f5ff80d3fe0 2023-05-20T01:28:35.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.967+0000 7f5fdffff700 1 -- 172.21.15.16:0/1907003900 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fec00cbc0 con 0x7f5ff80d3fe0 2023-05-20T01:28:35.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.967+0000 7f5ffd843700 1 -- 172.21.15.16:0/1907003900 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ff80d2500 con 0x7f5ff80d3fe0 2023-05-20T01:28:35.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.968+0000 7f5ffd843700 1 -- 172.21.15.16:0/1907003900 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ff80b4a00 con 0x7f5ff80d3fe0 2023-05-20T01:28:35.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.969+0000 7f5fdffff700 1 -- 172.21.15.16:0/1907003900 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5fec00cd20 con 0x7f5ff80d3fe0 2023-05-20T01:28:35.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.969+0000 7f5fdffff700 1 --2- 172.21.15.16:0/1907003900 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f5fe406ace0 0x7f5fe406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:35.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.969+0000 7f5fdffff700 1 -- 172.21.15.16:0/1907003900 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f5fec0896a0 con 0x7f5ff80d3fe0 2023-05-20T01:28:35.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.969+0000 7f5ff6ffd700 1 --2- 172.21.15.16:0/1907003900 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f5fe406ace0 0x7f5fe406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:35.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.970+0000 7f5ff6ffd700 1 --2- 172.21.15.16:0/1907003900 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f5fe406ace0 0x7f5fe406d190 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f5fe80090b0 tx=0x7f5fe8010000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:35.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:35.974+0000 7f5fdffff700 1 -- 172.21.15.16:0/1907003900 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f5fec058130 con 0x7f5ff80d3fe0 2023-05-20T01:28:36.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.214+0000 7f5ffd843700 1 -- 172.21.15.16:0/1907003900 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5ff8066ce0 con 0x7f5ff80d3fe0 2023-05-20T01:28:36.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.214+0000 7f5fdffff700 1 -- 172.21.15.16:0/1907003900 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) v1 ==== 94+0+4679 (secure 0 0 0) 0x7f5fec057cc0 con 0x7f5ff80d3fe0 2023-05-20T01:28:36.219 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:28:36.219 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":99,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7043,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/1859755557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1859755557},{"type":"v1","addr":"172.21.15.16:6836","nonce":1859755557}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94},{"gid":7046,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/2654783598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":2654783598},{"type":"v1","addr":"172.21.15.132:6834","nonce":2654783598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94},{"gid":7049,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2042939723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":2042939723},{"type":"v1","addr":"172.21.15.132:6835","nonce":2042939723}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-20T01:28:27.093957+0000","modified":"2023-05-20T01:28:30.436870+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8703},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8703":{"gid":8703,"name":"a","rank":0,"incarnation":95,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6837/2295684907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":2295684907},{"type":"v1","addr":"172.21.15.16:6837","nonce":2295684907}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-05-20T01:28:36.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.216+0000 7f5ffd843700 1 -- 172.21.15.16:0/1907003900 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f5fe406ace0 msgr2=0x7f5fe406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:36.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.216+0000 7f5ffd843700 1 --2- 172.21.15.16:0/1907003900 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f5fe406ace0 0x7f5fe406d190 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f5fe80090b0 tx=0x7f5fe8010000 comp rx=0 tx=0).stop 2023-05-20T01:28:36.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.216+0000 7f5ffd843700 1 -- 172.21.15.16:0/1907003900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ff80d3fe0 msgr2=0x7f5ff80b8980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:36.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.216+0000 7f5ffd843700 1 --2- 172.21.15.16:0/1907003900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ff80d3fe0 0x7f5ff80b8980 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f5fec00e9b0 tx=0x7f5fec00ed70 comp rx=0 tx=0).stop 2023-05-20T01:28:36.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.216+0000 7f5ffd843700 1 -- 172.21.15.16:0/1907003900 shutdown_connections 2023-05-20T01:28:36.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.216+0000 7f5ffd843700 1 --2- 172.21.15.16:0/1907003900 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f5fe406ace0 0x7f5fe406d190 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:36.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.216+0000 7f5ffd843700 1 --2- 172.21.15.16:0/1907003900 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5ff815ce80 0x7f5ff80b8ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:36.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.216+0000 7f5ffd843700 1 --2- 172.21.15.16:0/1907003900 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5ff80b4fb0 0x7f5ff81f9b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:36.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.216+0000 7f5ffd843700 1 --2- 172.21.15.16:0/1907003900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ff80d3fe0 0x7f5ff80b8980 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:36.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.216+0000 7f5ffd843700 1 -- 172.21.15.16:0/1907003900 >> 172.21.15.16:0/1907003900 conn(0x7f5ff809c800 msgr2=0x7f5ff8159020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:36.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.217+0000 7f5ffd843700 1 -- 172.21.15.16:0/1907003900 shutdown_connections 2023-05-20T01:28:36.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.217+0000 7f5ffd843700 1 -- 172.21.15.16:0/1907003900 wait complete. 2023-05-20T01:28:36.224 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 99 2023-05-20T01:28:36.233 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json dirfrag split /subdir 0/0 1 2023-05-20T01:28:36.374 DEBUG:tasks.cephfs.filesystem:_json_asok output empty 2023-05-20T01:28:36.375 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-05-20T01:28:36.375 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2023-05-20T01:28:36.393 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-05-20T01:28:36.394 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2023-05-20T01:28:36.450 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-20T01:28:36.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.612+0000 7f2d72ebf700 1 Processor -- start 2023-05-20T01:28:36.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.612+0000 7f2d72ebf700 1 -- start start 2023-05-20T01:28:36.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.613+0000 7f2d72ebf700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f2d6c072460 0x7f2d6c06a460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:36.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.613+0000 7f2d72ebf700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2d6c06a9a0 0x7f2d6c0c3c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:36.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.613+0000 7f2d72ebf700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d6c071fd0 con 0x7f2d6c15cb40 2023-05-20T01:28:36.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.613+0000 7f2d72ebf700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d6c06ae20 con 0x7f2d6c06a9a0 2023-05-20T01:28:36.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.613+0000 7f2d72ebf700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d6c0713c0 con 0x7f2d6c072460 2023-05-20T01:28:36.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.613+0000 7f2d7145c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2d6c06a9a0 0x7f2d6c0c3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:36.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.613+0000 7f2d7145c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2d6c06a9a0 0x7f2d6c0c3c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40272/0 (socket says 172.21.15.16:40272) 2023-05-20T01:28:36.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.613+0000 7f2d70c5b700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f2d6c15cb40 0x7f2d6c06f3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51796/0 (socket says 172.21.15.16:51796) 2023-05-20T01:28:36.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.613+0000 7f2d7145c700 1 -- 172.21.15.16:0/57210952 learned_addr learned my addr 172.21.15.16:0/57210952 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:36.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.614+0000 7f2d6bfff700 1 --2- 172.21.15.16:0/57210952 >> v2:172.21.15.132:3300/0 conn(0x7f2d6c072460 0x7f2d6c06a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:36.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.614+0000 7f2d7145c700 1 -- 172.21.15.16:0/57210952 >> v2:172.21.15.132:3300/0 conn(0x7f2d6c072460 msgr2=0x7f2d6c06a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:36.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.614+0000 7f2d7145c700 1 --2- 172.21.15.16:0/57210952 >> v2:172.21.15.132:3300/0 conn(0x7f2d6c072460 0x7f2d6c06a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:36.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.614+0000 7f2d7145c700 1 -- 172.21.15.16:0/57210952 >> v1:172.21.15.16:6789/0 conn(0x7f2d6c15cb40 legacy=0x7f2d6c06f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:36.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.614+0000 7f2d7145c700 1 -- 172.21.15.16:0/57210952 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d6c0716a0 con 0x7f2d6c06a9a0 2023-05-20T01:28:36.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.614+0000 7f2d6b7fe700 1 -- 172.21.15.16:0/57210952 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1213539095 0 0) 0x7f2d6c071fd0 con 0x7f2d6c15cb40 2023-05-20T01:28:36.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.614+0000 7f2d7145c700 1 --2- 172.21.15.16:0/57210952 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d6c06a9a0 0x7f2d6c0c3c70 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c0b2cf0 tx=0x7f2d6400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2419ba1219a7a788 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:36.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.614+0000 7f2d6b7fe700 1 -- 172.21.15.16:0/57210952 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d6400e070 con 0x7f2d6c06a9a0 2023-05-20T01:28:36.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.614+0000 7f2d6b7fe700 1 -- 172.21.15.16:0/57210952 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d6400b9c0 con 0x7f2d6c06a9a0 2023-05-20T01:28:36.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.614+0000 7f2d6b7fe700 1 -- 172.21.15.16:0/57210952 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d6400bd70 con 0x7f2d6c06a9a0 2023-05-20T01:28:36.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.614+0000 7f2d72ebf700 1 -- 172.21.15.16:0/57210952 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d6c06a9a0 msgr2=0x7f2d6c0c3c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:36.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.614+0000 7f2d72ebf700 1 --2- 172.21.15.16:0/57210952 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d6c06a9a0 0x7f2d6c0c3c70 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c0b2cf0 tx=0x7f2d6400b330 comp rx=0 tx=0).stop 2023-05-20T01:28:36.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.615+0000 7f2d72ebf700 1 -- 172.21.15.16:0/57210952 shutdown_connections 2023-05-20T01:28:36.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.615+0000 7f2d72ebf700 1 --2- 172.21.15.16:0/57210952 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d6c06a9a0 0x7f2d6c0c3c70 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:36.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.615+0000 7f2d72ebf700 1 --2- 172.21.15.16:0/57210952 >> v2:172.21.15.132:3300/0 conn(0x7f2d6c072460 0x7f2d6c06a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:36.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.615+0000 7f2d72ebf700 1 -- 172.21.15.16:0/57210952 >> 172.21.15.16:0/57210952 conn(0x7f2d6c0b1c90 msgr2=0x7f2d6c0b2060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:36.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.615+0000 7f2d72ebf700 1 -- 172.21.15.16:0/57210952 shutdown_connections 2023-05-20T01:28:36.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.615+0000 7f2d72ebf700 1 -- 172.21.15.16:0/57210952 wait complete. 2023-05-20T01:28:36.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.616+0000 7f2d72ebf700 1 Processor -- start 2023-05-20T01:28:36.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.616+0000 7f2d72ebf700 1 -- start start 2023-05-20T01:28:36.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.617+0000 7f2d72ebf700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2d6c06a9a0 0x7f2d6c15d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:36.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.617+0000 7f2d72ebf700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2d6c072460 0x7f2d6c1474c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:36.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.617+0000 7f2d72ebf700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d6c15cb40 0x7f2d6c0ad8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:36.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.617+0000 7f2d72ebf700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d6c15d240 con 0x7f2d6c15cb40 2023-05-20T01:28:36.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.617+0000 7f2d72ebf700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d6c1600d0 con 0x7f2d6c06a9a0 2023-05-20T01:28:36.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.617+0000 7f2d72ebf700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d6c1603b0 con 0x7f2d6c072460 2023-05-20T01:28:36.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.617+0000 7f2d7145c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d6c15cb40 0x7f2d6c0ad8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:36.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.617+0000 7f2d7145c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d6c15cb40 0x7f2d6c0ad8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40278/0 (socket says 172.21.15.16:40278) 2023-05-20T01:28:36.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.617+0000 7f2d7145c700 1 -- 172.21.15.16:0/2664103940 learned_addr learned my addr 172.21.15.16:0/2664103940 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:36.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.617+0000 7f2d70c5b700 1 --2- 172.21.15.16:0/2664103940 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2d6c06a9a0 0x7f2d6c15d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:36.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.617+0000 7f2d6bfff700 1 --2- 172.21.15.16:0/2664103940 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2d6c072460 0x7f2d6c1474c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:36.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.617+0000 7f2d7145c700 1 -- 172.21.15.16:0/2664103940 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2d6c072460 msgr2=0x7f2d6c1474c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:36.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.617+0000 7f2d7145c700 1 --2- 172.21.15.16:0/2664103940 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2d6c072460 0x7f2d6c1474c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:36.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.617+0000 7f2d7145c700 1 -- 172.21.15.16:0/2664103940 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2d6c06a9a0 msgr2=0x7f2d6c15d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:36.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.617+0000 7f2d7145c700 1 --2- 172.21.15.16:0/2664103940 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2d6c06a9a0 0x7f2d6c15d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:36.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.617+0000 7f2d7145c700 1 -- 172.21.15.16:0/2664103940 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d64009b30 con 0x7f2d6c15cb40 2023-05-20T01:28:36.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.617+0000 7f2d6bfff700 1 --2- 172.21.15.16:0/2664103940 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2d6c072460 0x7f2d6c1474c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:28:36.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.617+0000 7f2d70c5b700 1 --2- 172.21.15.16:0/2664103940 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2d6c06a9a0 0x7f2d6c15d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:28:36.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.617+0000 7f2d7145c700 1 --2- 172.21.15.16:0/2664103940 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d6c15cb40 0x7f2d6c0ad8f0 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c0cdca0 tx=0x7f2d64004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:36.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.617+0000 7f2d697fa700 1 -- 172.21.15.16:0/2664103940 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d640099b0 con 0x7f2d6c15cb40 2023-05-20T01:28:36.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.618+0000 7f2d697fa700 1 -- 172.21.15.16:0/2664103940 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d640092e0 con 0x7f2d6c15cb40 2023-05-20T01:28:36.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.618+0000 7f2d72ebf700 1 -- 172.21.15.16:0/2664103940 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d6c071c60 con 0x7f2d6c15cb40 2023-05-20T01:28:36.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.618+0000 7f2d697fa700 1 -- 172.21.15.16:0/2664103940 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d6401b650 con 0x7f2d6c15cb40 2023-05-20T01:28:36.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.618+0000 7f2d72ebf700 1 -- 172.21.15.16:0/2664103940 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d6c071980 con 0x7f2d6c15cb40 2023-05-20T01:28:36.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.619+0000 7f2d72ebf700 1 -- 172.21.15.16:0/2664103940 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d6c0cfea0 con 0x7f2d6c15cb40 2023-05-20T01:28:36.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.619+0000 7f2d697fa700 1 -- 172.21.15.16:0/2664103940 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2d6401b7b0 con 0x7f2d6c15cb40 2023-05-20T01:28:36.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.619+0000 7f2d697fa700 1 --2- 172.21.15.16:0/2664103940 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f2d5006ace0 0x7f2d5006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:36.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.619+0000 7f2d697fa700 1 -- 172.21.15.16:0/2664103940 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f2d6408aba0 con 0x7f2d6c15cb40 2023-05-20T01:28:36.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.619+0000 7f2d70c5b700 1 --2- 172.21.15.16:0/2664103940 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f2d5006ace0 0x7f2d5006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:36.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.620+0000 7f2d70c5b700 1 --2- 172.21.15.16:0/2664103940 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f2d5006ace0 0x7f2d5006d190 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f2d60000f60 tx=0x7f2d60007ca0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:36.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.624+0000 7f2d697fa700 1 -- 172.21.15.16:0/2664103940 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f2d64055ba0 con 0x7f2d6c15cb40 2023-05-20T01:28:36.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.826+0000 7f2d72ebf700 1 -- 172.21.15.16:0/2664103940 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2d6c04e8c0 con 0x7f2d6c15cb40 2023-05-20T01:28:36.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.827+0000 7f2d697fa700 1 -- 172.21.15.16:0/2664103940 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v136) v1 ==== 74+0+18472 (secure 0 0 0) 0x7f2d640591c0 con 0x7f2d6c15cb40 2023-05-20T01:28:36.832 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:28:36.832 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":136,"fsid":"c2245cc5-ecab-4087-a59b-bc81db901676","created":"2023-05-20T01:16:09.549230+0000","modified":"2023-05-20T01:28:28.096014+0000","last_up_change":"2023-05-20T01:16:19.788624+0000","last_in_change":"2023-05-20T01:16:12.092160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-05-20T01:16:20.934366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-05-20T01:16:21.372294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-05-20T01:28:25.274827+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"135","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-05-20T01:28:26.067827+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"317a83cd-19c8-4958-a301-6087ef342289","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6825","nonce":3923634179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6827","nonce":3923634179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6831","nonce":3923634179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6829","nonce":3923634179}]},"public_addr":"172.21.15.16:6825/3923634179","cluster_addr":"172.21.15.16:6827/3923634179","heartbeat_back_addr":"172.21.15.16:6831/3923634179","heartbeat_front_addr":"172.21.15.16:6829/3923634179","state":["exists","up"]},{"osd":1,"uuid":"5581046f-8219-4089-8d7d-b7846399505e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6809","nonce":3860264220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6811","nonce":3860264220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6815","nonce":3860264220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6813","nonce":3860264220}]},"public_addr":"172.21.15.16:6809/3860264220","cluster_addr":"172.21.15.16:6811/3860264220","heartbeat_back_addr":"172.21.15.16:6815/3860264220","heartbeat_front_addr":"172.21.15.16:6813/3860264220","state":["exists","up"]},{"osd":2,"uuid":"072e9e66-4133-478e-8d4a-64d46c2963f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6817","nonce":293187030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6819","nonce":293187030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6823","nonce":293187030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6821","nonce":293187030}]},"public_addr":"172.21.15.16:6817/293187030","cluster_addr":"172.21.15.16:6819/293187030","heartbeat_back_addr":"172.21.15.16:6823/293187030","heartbeat_front_addr":"172.21.15.16:6821/293187030","state":["exists","up"]},{"osd":3,"uuid":"11e1bba6-00f8-48dd-9028-d8d716b7f145","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6801","nonce":3669781890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6803","nonce":3669781890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6807","nonce":3669781890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6805","nonce":3669781890}]},"public_addr":"172.21.15.16:6801/3669781890","cluster_addr":"172.21.15.16:6803/3669781890","heartbeat_back_addr":"172.21.15.16:6807/3669781890","heartbeat_front_addr":"172.21.15.16:6805/3669781890","state":["exists","up"]},{"osd":4,"uuid":"b8909c3a-cf70-46cb-8d0b-009c707cf9d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6819","nonce":506317223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6823","nonce":506317223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6829","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6831","nonce":506317223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6827","nonce":506317223}]},"public_addr":"172.21.15.132:6819/506317223","cluster_addr":"172.21.15.132:6823/506317223","heartbeat_back_addr":"172.21.15.132:6831/506317223","heartbeat_front_addr":"172.21.15.132:6827/506317223","state":["exists","up"]},{"osd":5,"uuid":"b1818304-4583-47bf-a6ec-0bfbb5166233","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6818","nonce":1844689995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6822","nonce":1844689995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6830","nonce":1844689995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6826","nonce":1844689995}]},"public_addr":"172.21.15.132:6818/1844689995","cluster_addr":"172.21.15.132:6822/1844689995","heartbeat_back_addr":"172.21.15.132:6830/1844689995","heartbeat_front_addr":"172.21.15.132:6826/1844689995","state":["exists","up"]},{"osd":6,"uuid":"d37f4691-eed1-4a93-8818-18354f03f4a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6801","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6802","nonce":727581123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6805","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6807","nonce":727581123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6813","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6815","nonce":727581123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6811","nonce":727581123}]},"public_addr":"172.21.15.132:6802/727581123","cluster_addr":"172.21.15.132:6807/727581123","heartbeat_back_addr":"172.21.15.132:6815/727581123","heartbeat_front_addr":"172.21.15.132:6811/727581123","state":["exists","up"]},{"osd":7,"uuid":"4f712f62-5d1d-4816-8042-03209144f8b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6803","nonce":466001149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6806","nonce":466001149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6814","nonce":466001149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6810","nonce":466001149}]},"public_addr":"172.21.15.132:6803/466001149","cluster_addr":"172.21.15.132:6806/466001149","heartbeat_back_addr":"172.21.15.132:6814/466001149","heartbeat_front_addr":"172.21.15.132:6810/466001149","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.338863+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.278680+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305694+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.246835+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.276237+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.304441+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305421+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.267050+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-20T01:28:36.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.829+0000 7f2d72ebf700 1 -- 172.21.15.16:0/2664103940 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f2d5006ace0 msgr2=0x7f2d5006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:36.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.829+0000 7f2d72ebf700 1 --2- 172.21.15.16:0/2664103940 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f2d5006ace0 0x7f2d5006d190 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f2d60000f60 tx=0x7f2d60007ca0 comp rx=0 tx=0).stop 2023-05-20T01:28:36.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.829+0000 7f2d72ebf700 1 -- 172.21.15.16:0/2664103940 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d6c15cb40 msgr2=0x7f2d6c0ad8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:36.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.829+0000 7f2d72ebf700 1 --2- 172.21.15.16:0/2664103940 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d6c15cb40 0x7f2d6c0ad8f0 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c0cdca0 tx=0x7f2d64004ad0 comp rx=0 tx=0).stop 2023-05-20T01:28:36.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.829+0000 7f2d72ebf700 1 -- 172.21.15.16:0/2664103940 shutdown_connections 2023-05-20T01:28:36.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.829+0000 7f2d72ebf700 1 --2- 172.21.15.16:0/2664103940 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d6c15cb40 0x7f2d6c0ad8f0 unknown :-1 s=CLOSED pgs=1653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:36.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.829+0000 7f2d72ebf700 1 --2- 172.21.15.16:0/2664103940 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2d6c06a9a0 0x7f2d6c15d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:36.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.829+0000 7f2d72ebf700 1 --2- 172.21.15.16:0/2664103940 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f2d5006ace0 0x7f2d5006d190 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:36.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.829+0000 7f2d72ebf700 1 --2- 172.21.15.16:0/2664103940 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2d6c072460 0x7f2d6c1474c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:36.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.829+0000 7f2d72ebf700 1 -- 172.21.15.16:0/2664103940 >> 172.21.15.16:0/2664103940 conn(0x7f2d6c0b1c90 msgr2=0x7f2d6c0b3550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:36.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.829+0000 7f2d72ebf700 1 -- 172.21.15.16:0/2664103940 shutdown_connections 2023-05-20T01:28:36.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.829+0000 7f2d72ebf700 1 -- 172.21.15.16:0/2664103940 wait complete. 2023-05-20T01:28:36.846 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-05-20T01:16:20.934366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-05-20T01:16:21.372294+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_metadata', 'create_time': '2023-05-20T01:28:25.274827+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '135', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 19, 'pool_name': 'cephfs_data', 'create_time': '2023-05-20T01:28:26.067827+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '136', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-05-20T01:28:36.847 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-05-20T01:28:36.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.970+0000 7fa917f12700 1 Processor -- start 2023-05-20T01:28:36.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.971+0000 7fa917f12700 1 -- start start 2023-05-20T01:28:36.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.971+0000 7fa917f12700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa9100cc440 0x7fa910071da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:36.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.971+0000 7fa917f12700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa91006e860 con 0x7fa9100b2350 2023-05-20T01:28:36.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.971+0000 7fa917f12700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa91015d600 con 0x7fa910159a80 2023-05-20T01:28:36.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.971+0000 7fa917f12700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa910159e50 con 0x7fa9100cc440 2023-05-20T01:28:36.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.972+0000 7fa9154ad700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fa9100b2350 0x7fa9100caf10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51812/0 (socket says 172.21.15.16:51812) 2023-05-20T01:28:36.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.972+0000 7fa9154ad700 1 -- 172.21.15.16:0/3545443746 learned_addr learned my addr 172.21.15.16:0/3545443746 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:28:36.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.972+0000 7fa9164af700 1 --2- 172.21.15.16:0/3545443746 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa9100cc440 0x7fa910071da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:36.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.972+0000 7fa914cac700 1 -- 172.21.15.16:0/3545443746 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2913154430 0 0) 0x7fa91006e860 con 0x7fa9100b2350 2023-05-20T01:28:36.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.972+0000 7fa914cac700 1 -- 172.21.15.16:0/3545443746 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa8fc0057c0 con 0x7fa9100b2350 2023-05-20T01:28:36.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.972+0000 7fa914cac700 1 -- 172.21.15.16:0/3545443746 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 157473158 0 0) 0x7fa91015d600 con 0x7fa910159a80 2023-05-20T01:28:36.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.972+0000 7fa914cac700 1 -- 172.21.15.16:0/3545443746 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa91006e860 con 0x7fa910159a80 2023-05-20T01:28:36.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.972+0000 7fa9164af700 1 -- 172.21.15.16:0/3545443746 >> v1:172.21.15.132:6789/0 conn(0x7fa910159a80 legacy=0x7fa9100b1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:36.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.972+0000 7fa9164af700 1 -- 172.21.15.16:0/3545443746 >> v1:172.21.15.16:6789/0 conn(0x7fa9100b2350 legacy=0x7fa9100caf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:36.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.972+0000 7fa914cac700 1 -- 172.21.15.16:0/3545443746 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3839776043 0 0) 0x7fa8fc0057c0 con 0x7fa9100b2350 2023-05-20T01:28:36.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.972+0000 7fa9164af700 1 -- 172.21.15.16:0/3545443746 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa91015d850 con 0x7fa9100cc440 2023-05-20T01:28:36.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.972+0000 7fa9164af700 1 --2- 172.21.15.16:0/3545443746 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa9100cc440 0x7fa910071da0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa908009dc0 tx=0x7fa90800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=160b88ef46687bff server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:36.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.973+0000 7fa914cac700 1 -- 172.21.15.16:0/3545443746 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa90800e070 con 0x7fa9100cc440 2023-05-20T01:28:36.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.973+0000 7fa914cac700 1 -- 172.21.15.16:0/3545443746 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa908003a20 con 0x7fa9100cc440 2023-05-20T01:28:36.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.973+0000 7fa914cac700 1 -- 172.21.15.16:0/3545443746 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa904001670 con 0x7fa9100cc440 2023-05-20T01:28:36.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.973+0000 7fa917f12700 1 -- 172.21.15.16:0/3545443746 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa9100cc440 msgr2=0x7fa910071da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:36.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.973+0000 7fa917f12700 1 --2- 172.21.15.16:0/3545443746 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa9100cc440 0x7fa910071da0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa908009dc0 tx=0x7fa90800b330 comp rx=0 tx=0).stop 2023-05-20T01:28:36.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.973+0000 7fa917f12700 1 -- 172.21.15.16:0/3545443746 shutdown_connections 2023-05-20T01:28:36.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.973+0000 7fa917f12700 1 --2- 172.21.15.16:0/3545443746 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa9100cc440 0x7fa910071da0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:36.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.973+0000 7fa917f12700 1 -- 172.21.15.16:0/3545443746 >> 172.21.15.16:0/3545443746 conn(0x7fa9100ccc40 msgr2=0x7fa9100cd040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:36.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.973+0000 7fa917f12700 1 -- 172.21.15.16:0/3545443746 shutdown_connections 2023-05-20T01:28:36.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.973+0000 7fa917f12700 1 -- 172.21.15.16:0/3545443746 wait complete. 2023-05-20T01:28:36.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.975+0000 7fa917f12700 1 Processor -- start 2023-05-20T01:28:36.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.975+0000 7fa917f12700 1 -- start start 2023-05-20T01:28:36.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.976+0000 7fa917f12700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa9100b2350 0x7fa910149660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:36.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.976+0000 7fa917f12700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9100cc440 0x7fa9100b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:36.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.976+0000 7fa917f12700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa910159a80 0x7fa9100c8140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:36.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.976+0000 7fa917f12700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa910149ba0 con 0x7fa910159a80 2023-05-20T01:28:36.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.976+0000 7fa917f12700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9100b5610 con 0x7fa9100cc440 2023-05-20T01:28:36.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.976+0000 7fa917f12700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9100c73c0 con 0x7fa9100b2350 2023-05-20T01:28:36.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.976+0000 7fa9164af700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa910159a80 0x7fa9100c8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:36.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.976+0000 7fa9164af700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa910159a80 0x7fa9100c8140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40292/0 (socket says 172.21.15.16:40292) 2023-05-20T01:28:36.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.976+0000 7fa9164af700 1 -- 172.21.15.16:0/1618944086 learned_addr learned my addr 172.21.15.16:0/1618944086 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:36.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.976+0000 7fa915cae700 1 --2- 172.21.15.16:0/1618944086 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa9100b2350 0x7fa910149660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:36.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.976+0000 7fa9154ad700 1 --2- 172.21.15.16:0/1618944086 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9100cc440 0x7fa9100b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:36.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.976+0000 7fa9164af700 1 -- 172.21.15.16:0/1618944086 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa9100b2350 msgr2=0x7fa910149660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:36.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.976+0000 7fa9164af700 1 --2- 172.21.15.16:0/1618944086 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa9100b2350 0x7fa910149660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:36.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.976+0000 7fa9164af700 1 -- 172.21.15.16:0/1618944086 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9100cc440 msgr2=0x7fa9100b4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:36.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.976+0000 7fa9164af700 1 --2- 172.21.15.16:0/1618944086 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9100cc440 0x7fa9100b4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:36.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.976+0000 7fa9164af700 1 -- 172.21.15.16:0/1618944086 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa908009a70 con 0x7fa910159a80 2023-05-20T01:28:36.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.976+0000 7fa9164af700 1 --2- 172.21.15.16:0/1618944086 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa910159a80 0x7fa9100c8140 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7fa908004000 tx=0x7fa908004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:36.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.976+0000 7fa9027fc700 1 -- 172.21.15.16:0/1618944086 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa908004600 con 0x7fa910159a80 2023-05-20T01:28:36.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.977+0000 7fa9027fc700 1 -- 172.21.15.16:0/1618944086 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa908004780 con 0x7fa910159a80 2023-05-20T01:28:36.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.977+0000 7fa917f12700 1 -- 172.21.15.16:0/1618944086 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa91015de10 con 0x7fa910159a80 2023-05-20T01:28:36.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.977+0000 7fa9027fc700 1 -- 172.21.15.16:0/1618944086 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa90801d6b0 con 0x7fa910159a80 2023-05-20T01:28:36.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.977+0000 7fa917f12700 1 -- 172.21.15.16:0/1618944086 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa91015db30 con 0x7fa910159a80 2023-05-20T01:28:36.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.978+0000 7fa917f12700 1 -- 172.21.15.16:0/1618944086 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa91015e0f0 con 0x7fa910159a80 2023-05-20T01:28:36.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.978+0000 7fa9027fc700 1 -- 172.21.15.16:0/1618944086 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa90801d810 con 0x7fa910159a80 2023-05-20T01:28:36.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.978+0000 7fa9027fc700 1 --2- 172.21.15.16:0/1618944086 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa8f406ace0 0x7fa8f406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:36.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.979+0000 7fa915cae700 1 --2- 172.21.15.16:0/1618944086 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa8f406ace0 0x7fa8f406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:36.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.979+0000 7fa9027fc700 1 -- 172.21.15.16:0/1618944086 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fa90808b310 con 0x7fa910159a80 2023-05-20T01:28:36.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.979+0000 7fa915cae700 1 --2- 172.21.15.16:0/1618944086 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa8f406ace0 0x7fa8f406d190 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fa90c007fd0 tx=0x7fa90c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:36.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:36.983+0000 7fa9027fc700 1 -- 172.21.15.16:0/1618944086 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fa908059d00 con 0x7fa910159a80 2023-05-20T01:28:37.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.181+0000 7fa917f12700 1 -- 172.21.15.16:0/1618944086 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fa91004e8c0 con 0x7fa910159a80 2023-05-20T01:28:37.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.182+0000 7fa9027fc700 1 -- 172.21.15.16:0/1618944086 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v136) v1 ==== 92+0+10 (secure 0 0 0) 0x7fa908059890 con 0x7fa910159a80 2023-05-20T01:28:37.186 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 8 2023-05-20T01:28:37.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.184+0000 7fa917f12700 1 -- 172.21.15.16:0/1618944086 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa8f406ace0 msgr2=0x7fa8f406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:37.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.184+0000 7fa917f12700 1 --2- 172.21.15.16:0/1618944086 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa8f406ace0 0x7fa8f406d190 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fa90c007fd0 tx=0x7fa90c009040 comp rx=0 tx=0).stop 2023-05-20T01:28:37.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.184+0000 7fa917f12700 1 -- 172.21.15.16:0/1618944086 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa910159a80 msgr2=0x7fa9100c8140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:37.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.184+0000 7fa917f12700 1 --2- 172.21.15.16:0/1618944086 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa910159a80 0x7fa9100c8140 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7fa908004000 tx=0x7fa908004ad0 comp rx=0 tx=0).stop 2023-05-20T01:28:37.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.184+0000 7fa917f12700 1 -- 172.21.15.16:0/1618944086 shutdown_connections 2023-05-20T01:28:37.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.184+0000 7fa917f12700 1 --2- 172.21.15.16:0/1618944086 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa910159a80 0x7fa9100c8140 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:37.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.184+0000 7fa917f12700 1 --2- 172.21.15.16:0/1618944086 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fa8f406ace0 0x7fa8f406d190 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:37.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.184+0000 7fa917f12700 1 --2- 172.21.15.16:0/1618944086 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa9100b2350 0x7fa910149660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:37.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.184+0000 7fa917f12700 1 --2- 172.21.15.16:0/1618944086 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9100cc440 0x7fa9100b4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:37.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.184+0000 7fa917f12700 1 -- 172.21.15.16:0/1618944086 >> 172.21.15.16:0/1618944086 conn(0x7fa9100ccc40 msgr2=0x7fa9100d01c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:37.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.184+0000 7fa917f12700 1 -- 172.21.15.16:0/1618944086 shutdown_connections 2023-05-20T01:28:37.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.184+0000 7fa917f12700 1 -- 172.21.15.16:0/1618944086 wait complete. 2023-05-20T01:28:37.200 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-05-20T01:28:37.255 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:28:37.257 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:28:37.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.326+0000 7feec6f4c700 1 Processor -- start 2023-05-20T01:28:37.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.326+0000 7feec6f4c700 1 -- start start 2023-05-20T01:28:37.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.326+0000 7feec6f4c700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7feec00b2350 0x7feec0071da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:37.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.326+0000 7feec6f4c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7feec00cc440 0x7feec00caf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:37.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.326+0000 7feec6f4c700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feec006e860 con 0x7feec0159a80 2023-05-20T01:28:37.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.326+0000 7feec6f4c700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feec015d600 con 0x7feec00b2350 2023-05-20T01:28:37.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.326+0000 7feec6f4c700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feec0159e50 con 0x7feec00cc440 2023-05-20T01:28:37.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.327+0000 7feec54e9700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7feec00cc440 0x7feec00caf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:37.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.327+0000 7feebffff700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7feec00b2350 0x7feec0071da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:37.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.327+0000 7feec54e9700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7feec00cc440 0x7feec00caf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.16:40338/0 (socket says 172.21.15.16:40338) 2023-05-20T01:28:37.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.327+0000 7feec54e9700 1 -- 172.21.15.16:0/4090222882 learned_addr learned my addr 172.21.15.16:0/4090222882 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:37.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.327+0000 7feec4ce8700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7feec0159a80 0x7feec00b1c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.16:33306/0 (socket says 172.21.15.16:33306) 2023-05-20T01:28:37.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.327+0000 7feebffff700 1 --2- 172.21.15.16:0/4090222882 >> v2:172.21.15.132:3300/0 conn(0x7feec00b2350 0x7feec0071da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.16:37320/0 (socket says 172.21.15.16:37320) 2023-05-20T01:28:37.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.327+0000 7feec54e9700 1 -- 172.21.15.16:0/4090222882 >> v2:172.21.15.132:3300/0 conn(0x7feec00b2350 msgr2=0x7feec0071da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:37.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.327+0000 7feec54e9700 1 --2- 172.21.15.16:0/4090222882 >> v2:172.21.15.132:3300/0 conn(0x7feec00b2350 0x7feec0071da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:37.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.327+0000 7feec54e9700 1 -- 172.21.15.16:0/4090222882 >> v1:172.21.15.132:6789/0 conn(0x7feec0159a80 legacy=0x7feec00b1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:37.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.327+0000 7feec54e9700 1 -- 172.21.15.16:0/4090222882 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feec015d850 con 0x7feec00cc440 2023-05-20T01:28:37.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.327+0000 7feebffff700 1 --2- 172.21.15.16:0/4090222882 >> v2:172.21.15.132:3300/0 conn(0x7feec00b2350 0x7feec0071da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-05-20T01:28:37.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.327+0000 7feec54e9700 1 --2- 172.21.15.16:0/4090222882 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7feec00cc440 0x7feec00caf10 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7feeb800d150 tx=0x7feeb800d460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d03fb8284dab5b4f server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:37.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.327+0000 7feebf7fe700 1 -- 172.21.15.16:0/4090222882 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeb8010070 con 0x7feec00cc440 2023-05-20T01:28:37.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.328+0000 7feebf7fe700 1 -- 172.21.15.16:0/4090222882 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feeb800dd10 con 0x7feec00cc440 2023-05-20T01:28:37.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.328+0000 7feebf7fe700 1 -- 172.21.15.16:0/4090222882 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeb800b310 con 0x7feec00cc440 2023-05-20T01:28:37.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.328+0000 7feec6f4c700 1 -- 172.21.15.16:0/4090222882 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7feec00cc440 msgr2=0x7feec00caf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:37.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.328+0000 7feec6f4c700 1 --2- 172.21.15.16:0/4090222882 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7feec00cc440 0x7feec00caf10 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7feeb800d150 tx=0x7feeb800d460 comp rx=0 tx=0).stop 2023-05-20T01:28:37.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.328+0000 7feec6f4c700 1 -- 172.21.15.16:0/4090222882 shutdown_connections 2023-05-20T01:28:37.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.328+0000 7feec6f4c700 1 --2- 172.21.15.16:0/4090222882 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7feec00cc440 0x7feec00caf10 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:37.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.328+0000 7feec6f4c700 1 --2- 172.21.15.16:0/4090222882 >> v2:172.21.15.132:3300/0 conn(0x7feec00b2350 0x7feec0071da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:37.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.328+0000 7feec6f4c700 1 -- 172.21.15.16:0/4090222882 >> 172.21.15.16:0/4090222882 conn(0x7feec00ccc40 msgr2=0x7feec00cd040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:37.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.328+0000 7feec6f4c700 1 -- 172.21.15.16:0/4090222882 shutdown_connections 2023-05-20T01:28:37.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.328+0000 7feec6f4c700 1 -- 172.21.15.16:0/4090222882 wait complete. 2023-05-20T01:28:37.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.329+0000 7feec6f4c700 1 Processor -- start 2023-05-20T01:28:37.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.329+0000 7feec6f4c700 1 -- start start 2023-05-20T01:28:37.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.330+0000 7feec6f4c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feec00b2350 0x7feec015a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:37.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.330+0000 7feec6f4c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7feec00cc440 0x7feec00c82e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:37.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.330+0000 7feec6f4c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feec0159a80 0x7feec00c8820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:37.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.330+0000 7feec6f4c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feec00c9b40 con 0x7feec00b2350 2023-05-20T01:28:37.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.330+0000 7feec6f4c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feec00c8f30 con 0x7feec0159a80 2023-05-20T01:28:37.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.330+0000 7feec6f4c700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feec00c9210 con 0x7feec00cc440 2023-05-20T01:28:37.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.330+0000 7feec4ce8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feec00b2350 0x7feec015a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:37.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.330+0000 7feec4ce8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feec00b2350 0x7feec015a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40302/0 (socket says 172.21.15.16:40302) 2023-05-20T01:28:37.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.330+0000 7feec4ce8700 1 -- 172.21.15.16:0/577732248 learned_addr learned my addr 172.21.15.16:0/577732248 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:37.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.330+0000 7feebffff700 1 --2- 172.21.15.16:0/577732248 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7feec00cc440 0x7feec00c82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:37.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.330+0000 7feec54e9700 1 --2- 172.21.15.16:0/577732248 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feec0159a80 0x7feec00c8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:37.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.331+0000 7feec4ce8700 1 -- 172.21.15.16:0/577732248 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7feec00cc440 msgr2=0x7feec00c82e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:37.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.331+0000 7feec4ce8700 1 --2- 172.21.15.16:0/577732248 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7feec00cc440 0x7feec00c82e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:37.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.331+0000 7feec4ce8700 1 -- 172.21.15.16:0/577732248 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feec0159a80 msgr2=0x7feec00c8820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:37.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.331+0000 7feec4ce8700 1 --2- 172.21.15.16:0/577732248 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feec0159a80 0x7feec00c8820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:37.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.331+0000 7feec4ce8700 1 -- 172.21.15.16:0/577732248 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeac0094a0 con 0x7feec00b2350 2023-05-20T01:28:37.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.331+0000 7feec4ce8700 1 --2- 172.21.15.16:0/577732248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feec00b2350 0x7feec015a290 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7feeb400c520 tx=0x7feeb400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:37.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.331+0000 7feebd7fa700 1 -- 172.21.15.16:0/577732248 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeb4005cb0 con 0x7feec00b2350 2023-05-20T01:28:37.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.331+0000 7feebd7fa700 1 -- 172.21.15.16:0/577732248 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feeb4005e10 con 0x7feec00b2350 2023-05-20T01:28:37.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.331+0000 7feec6f4c700 1 -- 172.21.15.16:0/577732248 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feeb800bd40 con 0x7feec00b2350 2023-05-20T01:28:37.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.331+0000 7feebd7fa700 1 -- 172.21.15.16:0/577732248 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeb40072d0 con 0x7feec00b2350 2023-05-20T01:28:37.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.331+0000 7feec6f4c700 1 -- 172.21.15.16:0/577732248 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feec015de10 con 0x7feec00b2350 2023-05-20T01:28:37.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.332+0000 7feec6f4c700 1 -- 172.21.15.16:0/577732248 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feec015db30 con 0x7feec00b2350 2023-05-20T01:28:37.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.333+0000 7feebd7fa700 1 -- 172.21.15.16:0/577732248 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7feeb4007430 con 0x7feec00b2350 2023-05-20T01:28:37.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.333+0000 7feebd7fa700 1 --2- 172.21.15.16:0/577732248 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7feea406ad10 0x7feea406d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:37.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.333+0000 7feebd7fa700 1 -- 172.21.15.16:0/577732248 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6382+0+0 (secure 0 0 0) 0x7feeb40888e0 con 0x7feec00b2350 2023-05-20T01:28:37.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.333+0000 7feebffff700 1 --2- 172.21.15.16:0/577732248 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7feea406ad10 0x7feea406d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:37.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.334+0000 7feebffff700 1 --2- 172.21.15.16:0/577732248 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7feea406ad10 0x7feea406d1c0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7feec00ca020 tx=0x7feeac009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:37.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.338+0000 7feebd7fa700 1 -- 172.21.15.16:0/577732248 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7feeb4053870 con 0x7feec00b2350 2023-05-20T01:28:37.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.536+0000 7feec6f4c700 1 -- 172.21.15.16:0/577732248 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7feec015e0f0 con 0x7feec00b2350 2023-05-20T01:28:37.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.537+0000 7feebd7fa700 1 -- 172.21.15.16:0/577732248 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v136) v1 ==== 93+0+10 (secure 0 0 0) 0x7feeb4017020 con 0x7feec00b2350 2023-05-20T01:28:37.541 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-05-20T01:28:37.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.539+0000 7feec6f4c700 1 -- 172.21.15.16:0/577732248 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7feea406ad10 msgr2=0x7feea406d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:37.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.539+0000 7feec6f4c700 1 --2- 172.21.15.16:0/577732248 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7feea406ad10 0x7feea406d1c0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7feec00ca020 tx=0x7feeac009210 comp rx=0 tx=0).stop 2023-05-20T01:28:37.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.539+0000 7feec6f4c700 1 -- 172.21.15.16:0/577732248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feec00b2350 msgr2=0x7feec015a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:37.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.539+0000 7feec6f4c700 1 --2- 172.21.15.16:0/577732248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feec00b2350 0x7feec015a290 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7feeb400c520 tx=0x7feeb400c830 comp rx=0 tx=0).stop 2023-05-20T01:28:37.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.539+0000 7feec6f4c700 1 -- 172.21.15.16:0/577732248 shutdown_connections 2023-05-20T01:28:37.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.539+0000 7feec6f4c700 1 --2- 172.21.15.16:0/577732248 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feec0159a80 0x7feec00c8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:37.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.539+0000 7feec6f4c700 1 --2- 172.21.15.16:0/577732248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feec00b2350 0x7feec015a290 unknown :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:37.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.539+0000 7feec6f4c700 1 --2- 172.21.15.16:0/577732248 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7feea406ad10 0x7feea406d1c0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:37.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.539+0000 7feec6f4c700 1 --2- 172.21.15.16:0/577732248 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7feec00cc440 0x7feec00c82e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:37.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.539+0000 7feec6f4c700 1 -- 172.21.15.16:0/577732248 >> 172.21.15.16:0/577732248 conn(0x7feec00ccc40 msgr2=0x7feec00caf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:37.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.539+0000 7feec6f4c700 1 -- 172.21.15.16:0/577732248 shutdown_connections 2023-05-20T01:28:37.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.539+0000 7feec6f4c700 1 -- 172.21.15.16:0/577732248 wait complete. 2023-05-20T01:28:37.556 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-05-20T01:28:37.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.720+0000 7f2b25e1d700 1 Processor -- start 2023-05-20T01:28:37.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.720+0000 7f2b25e1d700 1 -- start start 2023-05-20T01:28:37.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.720+0000 7f2b25e1d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2b200b5780 0x7f2b2015b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:37.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.720+0000 7f2b25e1d700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f2b2006ab70 0x7f2b2015e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:37.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.720+0000 7f2b25e1d700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b2015b440 con 0x7f2b2015be10 2023-05-20T01:28:37.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.720+0000 7f2b25e1d700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b200cd410 con 0x7f2b200b5780 2023-05-20T01:28:37.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.720+0000 7f2b25e1d700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b200cd6f0 con 0x7f2b2006ab70 2023-05-20T01:28:37.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.721+0000 7f2b1f7fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2b200b5780 0x7f2b2015b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:37.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.721+0000 7f2b1effd700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f2b2015be10 0x7f2b2006a460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51838/0 (socket says 172.21.15.16:51838) 2023-05-20T01:28:37.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.721+0000 7f2b1effd700 1 -- 172.21.15.16:0/2943694956 learned_addr learned my addr 172.21.15.16:0/2943694956 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:28:37.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.721+0000 7f2b1ffff700 1 --2- 172.21.15.16:0/2943694956 >> v2:172.21.15.132:3300/0 conn(0x7f2b2006ab70 0x7f2b2015e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:37.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.721+0000 7f2b1f7fe700 1 -- 172.21.15.16:0/2943694956 >> v2:172.21.15.132:3300/0 conn(0x7f2b2006ab70 msgr2=0x7f2b2015e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:37.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.721+0000 7f2b1f7fe700 1 --2- 172.21.15.16:0/2943694956 >> v2:172.21.15.132:3300/0 conn(0x7f2b2006ab70 0x7f2b2015e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:37.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.721+0000 7f2b1f7fe700 1 -- 172.21.15.16:0/2943694956 >> v1:172.21.15.16:6789/0 conn(0x7f2b2015be10 legacy=0x7f2b2006a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:37.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.721+0000 7f2b1e7fc700 1 -- 172.21.15.16:0/2943694956 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4237181930 0 0) 0x7f2b2015b440 con 0x7f2b2015be10 2023-05-20T01:28:37.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.721+0000 7f2b1f7fe700 1 -- 172.21.15.16:0/2943694956 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b200cd9d0 con 0x7f2b200b5780 2023-05-20T01:28:37.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.721+0000 7f2b1f7fe700 1 --2- 172.21.15.16:0/2943694956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b200b5780 0x7f2b2015b8d0 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f2b2015b3b0 tx=0x7f2b14009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=389121d09e4276bd server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:37.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.721+0000 7f2b1e7fc700 1 -- 172.21.15.16:0/2943694956 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b14011830 con 0x7f2b200b5780 2023-05-20T01:28:37.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.721+0000 7f2b1e7fc700 1 -- 172.21.15.16:0/2943694956 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b14011990 con 0x7f2b200b5780 2023-05-20T01:28:37.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.721+0000 7f2b1e7fc700 1 -- 172.21.15.16:0/2943694956 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b14011d20 con 0x7f2b200b5780 2023-05-20T01:28:37.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.722+0000 7f2b25e1d700 1 -- 172.21.15.16:0/2943694956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b200b5780 msgr2=0x7f2b2015b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:37.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.722+0000 7f2b25e1d700 1 --2- 172.21.15.16:0/2943694956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b200b5780 0x7f2b2015b8d0 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f2b2015b3b0 tx=0x7f2b14009b20 comp rx=0 tx=0).stop 2023-05-20T01:28:37.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.722+0000 7f2b25e1d700 1 -- 172.21.15.16:0/2943694956 shutdown_connections 2023-05-20T01:28:37.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.722+0000 7f2b25e1d700 1 --2- 172.21.15.16:0/2943694956 >> v2:172.21.15.132:3300/0 conn(0x7f2b2006ab70 0x7f2b2015e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:37.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.722+0000 7f2b25e1d700 1 --2- 172.21.15.16:0/2943694956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b200b5780 0x7f2b2015b8d0 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:37.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.722+0000 7f2b25e1d700 1 -- 172.21.15.16:0/2943694956 >> 172.21.15.16:0/2943694956 conn(0x7f2b200b1d90 msgr2=0x7f2b200b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:37.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.722+0000 7f2b25e1d700 1 -- 172.21.15.16:0/2943694956 shutdown_connections 2023-05-20T01:28:37.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.722+0000 7f2b25e1d700 1 -- 172.21.15.16:0/2943694956 wait complete. 2023-05-20T01:28:37.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.723+0000 7f2b25e1d700 1 Processor -- start 2023-05-20T01:28:37.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.724+0000 7f2b25e1d700 1 -- start start 2023-05-20T01:28:37.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.725+0000 7f2b25e1d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2b2006ab70 0x7f2b200b33b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:37.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.725+0000 7f2b25e1d700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2b200b5780 0x7f2b2014f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:37.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.725+0000 7f2b25e1d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b2015be10 0x7f2b200b3e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:37.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.725+0000 7f2b25e1d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b200cfe50 con 0x7f2b2015be10 2023-05-20T01:28:37.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.725+0000 7f2b25e1d700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b200ad630 con 0x7f2b2006ab70 2023-05-20T01:28:37.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.725+0000 7f2b25e1d700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b20158580 con 0x7f2b200b5780 2023-05-20T01:28:37.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.725+0000 7f2b1ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b2015be10 0x7f2b200b3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:37.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.725+0000 7f2b1ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b2015be10 0x7f2b200b3e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40320/0 (socket says 172.21.15.16:40320) 2023-05-20T01:28:37.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.725+0000 7f2b1ffff700 1 -- 172.21.15.16:0/726266800 learned_addr learned my addr 172.21.15.16:0/726266800 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:37.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.725+0000 7f2b1effd700 1 --2- 172.21.15.16:0/726266800 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2b200b5780 0x7f2b2014f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:37.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.725+0000 7f2b1f7fe700 1 --2- 172.21.15.16:0/726266800 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2b2006ab70 0x7f2b200b33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:37.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.725+0000 7f2b1ffff700 1 -- 172.21.15.16:0/726266800 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2b200b5780 msgr2=0x7f2b2014f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:37.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.725+0000 7f2b1ffff700 1 --2- 172.21.15.16:0/726266800 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2b200b5780 0x7f2b2014f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:37.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.725+0000 7f2b1ffff700 1 -- 172.21.15.16:0/726266800 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2b2006ab70 msgr2=0x7f2b200b33b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:37.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.725+0000 7f2b1ffff700 1 --2- 172.21.15.16:0/726266800 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2b2006ab70 0x7f2b200b33b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:37.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.725+0000 7f2b1ffff700 1 -- 172.21.15.16:0/726266800 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b140094a0 con 0x7f2b2015be10 2023-05-20T01:28:37.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.725+0000 7f2b1ffff700 1 --2- 172.21.15.16:0/726266800 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b2015be10 0x7f2b200b3e50 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f2b18007e30 tx=0x7f2b1800e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:37.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.725+0000 7f2b07fff700 1 -- 172.21.15.16:0/726266800 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b1800ed30 con 0x7f2b2015be10 2023-05-20T01:28:37.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.725+0000 7f2b07fff700 1 -- 172.21.15.16:0/726266800 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b1800ee90 con 0x7f2b2015be10 2023-05-20T01:28:37.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.725+0000 7f2b25e1d700 1 -- 172.21.15.16:0/726266800 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b2015b030 con 0x7f2b2015be10 2023-05-20T01:28:37.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.725+0000 7f2b07fff700 1 -- 172.21.15.16:0/726266800 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b18014660 con 0x7f2b2015be10 2023-05-20T01:28:37.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.726+0000 7f2b25e1d700 1 -- 172.21.15.16:0/726266800 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b200b5b50 con 0x7f2b2015be10 2023-05-20T01:28:37.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.726+0000 7f2b25e1d700 1 -- 172.21.15.16:0/726266800 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b2015eb70 con 0x7f2b2015be10 2023-05-20T01:28:37.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.727+0000 7f2b07fff700 1 -- 172.21.15.16:0/726266800 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2b180147c0 con 0x7f2b2015be10 2023-05-20T01:28:37.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.727+0000 7f2b07fff700 1 --2- 172.21.15.16:0/726266800 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f2b0c06ace0 0x7f2b0c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:37.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.727+0000 7f2b07fff700 1 -- 172.21.15.16:0/726266800 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f2b18089be0 con 0x7f2b2015be10 2023-05-20T01:28:37.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.727+0000 7f2b1f7fe700 1 --2- 172.21.15.16:0/726266800 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f2b0c06ace0 0x7f2b0c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:37.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.728+0000 7f2b1f7fe700 1 --2- 172.21.15.16:0/726266800 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f2b0c06ace0 0x7f2b0c06d190 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f2b140090b0 tx=0x7f2b14008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:37.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.732+0000 7f2b07fff700 1 -- 172.21.15.16:0/726266800 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f2b18058670 con 0x7f2b2015be10 2023-05-20T01:28:37.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.932+0000 7f2b25e1d700 1 -- 172.21.15.16:0/726266800 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2b200ad7a0 con 0x7f2b2015be10 2023-05-20T01:28:37.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.933+0000 7f2b07fff700 1 -- 172.21.15.16:0/726266800 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v136) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2b18058200 con 0x7f2b2015be10 2023-05-20T01:28:37.937 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-05-20T01:28:37.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.935+0000 7f2b25e1d700 1 -- 172.21.15.16:0/726266800 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f2b0c06ace0 msgr2=0x7f2b0c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:37.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.935+0000 7f2b25e1d700 1 --2- 172.21.15.16:0/726266800 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f2b0c06ace0 0x7f2b0c06d190 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f2b140090b0 tx=0x7f2b14008030 comp rx=0 tx=0).stop 2023-05-20T01:28:37.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.935+0000 7f2b25e1d700 1 -- 172.21.15.16:0/726266800 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b2015be10 msgr2=0x7f2b200b3e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:37.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.935+0000 7f2b25e1d700 1 --2- 172.21.15.16:0/726266800 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b2015be10 0x7f2b200b3e50 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f2b18007e30 tx=0x7f2b1800e3c0 comp rx=0 tx=0).stop 2023-05-20T01:28:37.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.935+0000 7f2b25e1d700 1 -- 172.21.15.16:0/726266800 shutdown_connections 2023-05-20T01:28:37.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.935+0000 7f2b25e1d700 1 --2- 172.21.15.16:0/726266800 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b2015be10 0x7f2b200b3e50 unknown :-1 s=CLOSED pgs=1659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:37.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.935+0000 7f2b25e1d700 1 --2- 172.21.15.16:0/726266800 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2b2006ab70 0x7f2b200b33b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:37.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.935+0000 7f2b25e1d700 1 --2- 172.21.15.16:0/726266800 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f2b0c06ace0 0x7f2b0c06d190 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:37.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.935+0000 7f2b25e1d700 1 --2- 172.21.15.16:0/726266800 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2b200b5780 0x7f2b2014f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:37.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.935+0000 7f2b25e1d700 1 -- 172.21.15.16:0/726266800 >> 172.21.15.16:0/726266800 conn(0x7f2b200b1d90 msgr2=0x7f2b200b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:37.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.935+0000 7f2b25e1d700 1 -- 172.21.15.16:0/726266800 shutdown_connections 2023-05-20T01:28:37.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:37.935+0000 7f2b25e1d700 1 -- 172.21.15.16:0/726266800 wait complete. 2023-05-20T01:28:37.951 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-05-20T01:28:38.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.075+0000 7f87a6b1f700 1 Processor -- start 2023-05-20T01:28:38.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.075+0000 7f87a6b1f700 1 -- start start 2023-05-20T01:28:38.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.076+0000 7f87a6b1f700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f87a00cf370 0x7f87a0161260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:38.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.076+0000 7f87a6b1f700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87a006a460 0x7f87a006a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:38.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.076+0000 7f87a6b1f700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87a006fe20 con 0x7f87a0072670 2023-05-20T01:28:38.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.076+0000 7f87a6b1f700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87a00d0080 con 0x7f87a00cf370 2023-05-20T01:28:38.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.076+0000 7f87a6b1f700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87a015d200 con 0x7f87a006a460 2023-05-20T01:28:38.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.076+0000 7f879ffff700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f87a00cf370 0x7f87a0161260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:38.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.076+0000 7f87a50bc700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87a006a460 0x7f87a006a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:38.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.077+0000 7f879ffff700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f87a00cf370 0x7f87a0161260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.16:37348/0 (socket says 172.21.15.16:37348) 2023-05-20T01:28:38.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.077+0000 7f87a50bc700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87a006a460 0x7f87a006a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.16:40364/0 (socket says 172.21.15.16:40364) 2023-05-20T01:28:38.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.077+0000 7f87a48bb700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f87a0072670 0x7f87a006f710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.16:33334/0 (socket says 172.21.15.16:33334) 2023-05-20T01:28:38.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.077+0000 7f879ffff700 1 -- 172.21.15.16:0/76421592 learned_addr learned my addr 172.21.15.16:0/76421592 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:38.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.077+0000 7f87a50bc700 1 -- 172.21.15.16:0/76421592 >> v2:172.21.15.132:3300/0 conn(0x7f87a00cf370 msgr2=0x7f87a0161260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:38.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.077+0000 7f87a50bc700 1 --2- 172.21.15.16:0/76421592 >> v2:172.21.15.132:3300/0 conn(0x7f87a00cf370 0x7f87a0161260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:38.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.077+0000 7f87a50bc700 1 -- 172.21.15.16:0/76421592 >> v1:172.21.15.132:6789/0 conn(0x7f87a0072670 legacy=0x7f87a006f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:38.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.077+0000 7f879f7fe700 1 -- 172.21.15.16:0/76421592 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1407713778 0 0) 0x7f8794001110 con 0x7f87a0072670 2023-05-20T01:28:38.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.077+0000 7f87a50bc700 1 -- 172.21.15.16:0/76421592 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87a01617a0 con 0x7f87a006a460 2023-05-20T01:28:38.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.077+0000 7f87a50bc700 1 --2- 172.21.15.16:0/76421592 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87a006a460 0x7f87a006a860 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8798009d90 tx=0x7f879800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b328b29b9d522e1 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:38.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.077+0000 7f879f7fe700 1 -- 172.21.15.16:0/76421592 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f879800e070 con 0x7f87a006a460 2023-05-20T01:28:38.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.077+0000 7f879f7fe700 1 -- 172.21.15.16:0/76421592 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f879800bb20 con 0x7f87a006a460 2023-05-20T01:28:38.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.077+0000 7f879f7fe700 1 -- 172.21.15.16:0/76421592 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8798012430 con 0x7f87a006a460 2023-05-20T01:28:38.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.078+0000 7f87a6b1f700 1 -- 172.21.15.16:0/76421592 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87a006a460 msgr2=0x7f87a006a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:38.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.078+0000 7f87a6b1f700 1 --2- 172.21.15.16:0/76421592 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87a006a460 0x7f87a006a860 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8798009d90 tx=0x7f879800b330 comp rx=0 tx=0).stop 2023-05-20T01:28:38.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.078+0000 7f87a6b1f700 1 -- 172.21.15.16:0/76421592 shutdown_connections 2023-05-20T01:28:38.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.078+0000 7f87a6b1f700 1 --2- 172.21.15.16:0/76421592 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87a006a460 0x7f87a006a860 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:38.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.078+0000 7f87a6b1f700 1 --2- 172.21.15.16:0/76421592 >> v2:172.21.15.132:3300/0 conn(0x7f87a00cf370 0x7f87a0161260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:38.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.078+0000 7f87a6b1f700 1 -- 172.21.15.16:0/76421592 >> 172.21.15.16:0/76421592 conn(0x7f87a00ccc40 msgr2=0x7f87a00cd040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:38.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.078+0000 7f87a6b1f700 1 -- 172.21.15.16:0/76421592 shutdown_connections 2023-05-20T01:28:38.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.078+0000 7f87a6b1f700 1 -- 172.21.15.16:0/76421592 wait complete. 2023-05-20T01:28:38.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.079+0000 7f87a6b1f700 1 Processor -- start 2023-05-20T01:28:38.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.079+0000 7f87a6b1f700 1 -- start start 2023-05-20T01:28:38.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.080+0000 7f87a6b1f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87a006a460 0x7f87a00c69e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:38.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.080+0000 7f87a6b1f700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f87a0072670 0x7f87a00aef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:38.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.080+0000 7f87a6b1f700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87a00cf370 0x7f87a014d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:38.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.080+0000 7f87a6b1f700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87a014e230 con 0x7f87a006a460 2023-05-20T01:28:38.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.080+0000 7f87a6b1f700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87a00c5c20 con 0x7f87a0072670 2023-05-20T01:28:38.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.080+0000 7f87a6b1f700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87a00c5f00 con 0x7f87a00cf370 2023-05-20T01:28:38.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.080+0000 7f87a48bb700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87a006a460 0x7f87a00c69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:38.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.080+0000 7f87a48bb700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87a006a460 0x7f87a00c69e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40330/0 (socket says 172.21.15.16:40330) 2023-05-20T01:28:38.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.080+0000 7f87a48bb700 1 -- 172.21.15.16:0/4059199478 learned_addr learned my addr 172.21.15.16:0/4059199478 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:38.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.080+0000 7f879ffff700 1 --2- 172.21.15.16:0/4059199478 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f87a0072670 0x7f87a00aef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:38.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.080+0000 7f87a50bc700 1 --2- 172.21.15.16:0/4059199478 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87a00cf370 0x7f87a014d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:38.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.080+0000 7f87a48bb700 1 -- 172.21.15.16:0/4059199478 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87a00cf370 msgr2=0x7f87a014d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:38.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.080+0000 7f87a48bb700 1 --2- 172.21.15.16:0/4059199478 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87a00cf370 0x7f87a014d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:38.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.080+0000 7f87a48bb700 1 -- 172.21.15.16:0/4059199478 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f87a0072670 msgr2=0x7f87a00aef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:38.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.080+0000 7f87a48bb700 1 --2- 172.21.15.16:0/4059199478 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f87a0072670 0x7f87a00aef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:38.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.080+0000 7f87a48bb700 1 -- 172.21.15.16:0/4059199478 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8798009a70 con 0x7f87a006a460 2023-05-20T01:28:38.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.081+0000 7f87a48bb700 1 --2- 172.21.15.16:0/4059199478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87a006a460 0x7f87a00c69e0 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f879400c730 tx=0x7f879400caf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:38.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.081+0000 7f879d7fa700 1 -- 172.21.15.16:0/4059199478 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8794004c80 con 0x7f87a006a460 2023-05-20T01:28:38.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.081+0000 7f879d7fa700 1 -- 172.21.15.16:0/4059199478 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8794004e00 con 0x7f87a006a460 2023-05-20T01:28:38.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.081+0000 7f87a6b1f700 1 -- 172.21.15.16:0/4059199478 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87a00cf810 con 0x7f87a006a460 2023-05-20T01:28:38.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.081+0000 7f879d7fa700 1 -- 172.21.15.16:0/4059199478 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8794007310 con 0x7f87a006a460 2023-05-20T01:28:38.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.081+0000 7f87a6b1f700 1 -- 172.21.15.16:0/4059199478 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87a006ada0 con 0x7f87a006a460 2023-05-20T01:28:38.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.082+0000 7f87a6b1f700 1 -- 172.21.15.16:0/4059199478 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87a00cfac0 con 0x7f87a006a460 2023-05-20T01:28:38.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.082+0000 7f879d7fa700 1 -- 172.21.15.16:0/4059199478 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8794007470 con 0x7f87a006a460 2023-05-20T01:28:38.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.082+0000 7f879d7fa700 1 --2- 172.21.15.16:0/4059199478 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f878406ace0 0x7f878406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:38.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.083+0000 7f879d7fa700 1 -- 172.21.15.16:0/4059199478 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f8794089210 con 0x7f87a006a460 2023-05-20T01:28:38.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.083+0000 7f879ffff700 1 --2- 172.21.15.16:0/4059199478 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f878406ace0 0x7f878406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:38.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.083+0000 7f879ffff700 1 --2- 172.21.15.16:0/4059199478 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f878406ace0 0x7f878406d190 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f87a006f680 tx=0x7f878c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:38.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.087+0000 7f879d7fa700 1 -- 172.21.15.16:0/4059199478 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f8794054190 con 0x7f87a006a460 2023-05-20T01:28:38.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.284+0000 7f87a6b1f700 1 -- 172.21.15.16:0/4059199478 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f87a006fe20 con 0x7f87a006a460 2023-05-20T01:28:38.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.285+0000 7f879d7fa700 1 -- 172.21.15.16:0/4059199478 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v136) v1 ==== 100+0+11 (secure 0 0 0) 0x7f87a006fe20 con 0x7f87a006a460 2023-05-20T01:28:38.289 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2023-05-20T01:28:38.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.286+0000 7f87a6b1f700 1 -- 172.21.15.16:0/4059199478 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f878406ace0 msgr2=0x7f878406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:38.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.286+0000 7f87a6b1f700 1 --2- 172.21.15.16:0/4059199478 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f878406ace0 0x7f878406d190 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f87a006f680 tx=0x7f878c008040 comp rx=0 tx=0).stop 2023-05-20T01:28:38.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.286+0000 7f87a6b1f700 1 -- 172.21.15.16:0/4059199478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87a006a460 msgr2=0x7f87a00c69e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:38.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.286+0000 7f87a6b1f700 1 --2- 172.21.15.16:0/4059199478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87a006a460 0x7f87a00c69e0 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f879400c730 tx=0x7f879400caf0 comp rx=0 tx=0).stop 2023-05-20T01:28:38.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.287+0000 7f87a6b1f700 1 -- 172.21.15.16:0/4059199478 shutdown_connections 2023-05-20T01:28:38.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.287+0000 7f87a6b1f700 1 --2- 172.21.15.16:0/4059199478 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f878406ace0 0x7f878406d190 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:38.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.287+0000 7f87a6b1f700 1 --2- 172.21.15.16:0/4059199478 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87a00cf370 0x7f87a014d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:38.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.287+0000 7f87a6b1f700 1 --2- 172.21.15.16:0/4059199478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87a006a460 0x7f87a00c69e0 unknown :-1 s=CLOSED pgs=1660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:38.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.287+0000 7f87a6b1f700 1 --2- 172.21.15.16:0/4059199478 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f87a0072670 0x7f87a00aef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:38.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.287+0000 7f87a6b1f700 1 -- 172.21.15.16:0/4059199478 >> 172.21.15.16:0/4059199478 conn(0x7f87a00ccc40 msgr2=0x7f87a00adac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:38.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.287+0000 7f87a6b1f700 1 -- 172.21.15.16:0/4059199478 shutdown_connections 2023-05-20T01:28:38.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.287+0000 7f87a6b1f700 1 -- 172.21.15.16:0/4059199478 wait complete. 2023-05-20T01:28:38.303 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-05-20T01:28:38.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.424+0000 7f088d37d700 1 Processor -- start 2023-05-20T01:28:38.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.424+0000 7f088d37d700 1 -- start start 2023-05-20T01:28:38.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.425+0000 7f088d37d700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f088815ce80 0x7f08880d2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:38.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.425+0000 7f088d37d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f08880d3fe0 0x7f08880cf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:38.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.425+0000 7f088d37d700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08880d2b50 con 0x7f08880b4fb0 2023-05-20T01:28:38.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.425+0000 7f088d37d700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08880d4470 con 0x7f08880d3fe0 2023-05-20T01:28:38.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.425+0000 7f088d37d700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f088815d250 con 0x7f088815ce80 2023-05-20T01:28:38.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.425+0000 7f08867fc700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f08880d3fe0 0x7f08880cf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:38.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.425+0000 7f08877fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f08880b4fb0 0x7f08880b6380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51870/0 (socket says 172.21.15.16:51870) 2023-05-20T01:28:38.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.425+0000 7f08877fe700 1 -- 172.21.15.16:0/3248479303 learned_addr learned my addr 172.21.15.16:0/3248479303 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:28:38.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.425+0000 7f08867fc700 1 -- 172.21.15.16:0/3248479303 >> v2:172.21.15.132:3300/0 conn(0x7f088815ce80 msgr2=0x7f08880d2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:38.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.425+0000 7f08867fc700 1 --2- 172.21.15.16:0/3248479303 >> v2:172.21.15.132:3300/0 conn(0x7f088815ce80 0x7f08880d2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:38.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.425+0000 7f08867fc700 1 -- 172.21.15.16:0/3248479303 >> v1:172.21.15.16:6789/0 conn(0x7f08880b4fb0 legacy=0x7f08880b6380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:38.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.425+0000 7f08867fc700 1 -- 172.21.15.16:0/3248479303 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08880d2220 con 0x7f08880d3fe0 2023-05-20T01:28:38.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.425+0000 7f0885ffb700 1 -- 172.21.15.16:0/3248479303 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2486292502 0 0) 0x7f08880d2b50 con 0x7f08880b4fb0 2023-05-20T01:28:38.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.426+0000 7f08867fc700 1 --2- 172.21.15.16:0/3248479303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08880d3fe0 0x7f08880cf980 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f0888066f20 tx=0x7f0878009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b5d8e2028c6c927 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:38.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.426+0000 7f0885ffb700 1 -- 172.21.15.16:0/3248479303 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0878011830 con 0x7f08880d3fe0 2023-05-20T01:28:38.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.426+0000 7f0885ffb700 1 -- 172.21.15.16:0/3248479303 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0878011990 con 0x7f08880d3fe0 2023-05-20T01:28:38.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.426+0000 7f0885ffb700 1 -- 172.21.15.16:0/3248479303 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0878011d20 con 0x7f08880d3fe0 2023-05-20T01:28:38.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.426+0000 7f088d37d700 1 -- 172.21.15.16:0/3248479303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08880d3fe0 msgr2=0x7f08880cf980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:38.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.426+0000 7f088d37d700 1 --2- 172.21.15.16:0/3248479303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08880d3fe0 0x7f08880cf980 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f0888066f20 tx=0x7f0878009b20 comp rx=0 tx=0).stop 2023-05-20T01:28:38.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.426+0000 7f088d37d700 1 -- 172.21.15.16:0/3248479303 shutdown_connections 2023-05-20T01:28:38.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.426+0000 7f088d37d700 1 --2- 172.21.15.16:0/3248479303 >> v2:172.21.15.132:3300/0 conn(0x7f088815ce80 0x7f08880d2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:38.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.426+0000 7f088d37d700 1 --2- 172.21.15.16:0/3248479303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08880d3fe0 0x7f08880cf980 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:38.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.426+0000 7f088d37d700 1 -- 172.21.15.16:0/3248479303 >> 172.21.15.16:0/3248479303 conn(0x7f088809c800 msgr2=0x7f08880d3530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:38.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.426+0000 7f088d37d700 1 -- 172.21.15.16:0/3248479303 shutdown_connections 2023-05-20T01:28:38.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.426+0000 7f088d37d700 1 -- 172.21.15.16:0/3248479303 wait complete. 2023-05-20T01:28:38.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.428+0000 7f088d37d700 1 Processor -- start 2023-05-20T01:28:38.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.428+0000 7f088d37d700 1 -- start start 2023-05-20T01:28:38.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.428+0000 7f088d37d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08880b4fb0 0x7f0888153f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:38.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.428+0000 7f088d37d700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f08880d3fe0 0x7f08880c8950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:38.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.428+0000 7f088d37d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f088815ce80 0x7f08880cc650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:38.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.428+0000 7f088d37d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08880c84c0 con 0x7f08880b4fb0 2023-05-20T01:28:38.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.428+0000 7f088d37d700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0888154460 con 0x7f088815ce80 2023-05-20T01:28:38.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.428+0000 7f088d37d700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08880c78b0 con 0x7f08880d3fe0 2023-05-20T01:28:38.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.428+0000 7f0886ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08880b4fb0 0x7f0888153f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:38.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.428+0000 7f0886ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08880b4fb0 0x7f0888153f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40344/0 (socket says 172.21.15.16:40344) 2023-05-20T01:28:38.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.428+0000 7f0886ffd700 1 -- 172.21.15.16:0/2309733399 learned_addr learned my addr 172.21.15.16:0/2309733399 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:38.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.429+0000 7f08877fe700 1 --2- 172.21.15.16:0/2309733399 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f088815ce80 0x7f08880cc650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:38.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.429+0000 7f08867fc700 1 --2- 172.21.15.16:0/2309733399 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f08880d3fe0 0x7f08880c8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:38.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.429+0000 7f0886ffd700 1 -- 172.21.15.16:0/2309733399 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f08880d3fe0 msgr2=0x7f08880c8950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:38.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.429+0000 7f0886ffd700 1 --2- 172.21.15.16:0/2309733399 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f08880d3fe0 0x7f08880c8950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:38.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.429+0000 7f0886ffd700 1 -- 172.21.15.16:0/2309733399 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f088815ce80 msgr2=0x7f08880cc650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:38.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.429+0000 7f0886ffd700 1 --2- 172.21.15.16:0/2309733399 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f088815ce80 0x7f08880cc650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:38.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.429+0000 7f0886ffd700 1 -- 172.21.15.16:0/2309733399 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08780094a0 con 0x7f08880b4fb0 2023-05-20T01:28:38.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.429+0000 7f0886ffd700 1 --2- 172.21.15.16:0/2309733399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08880b4fb0 0x7f0888153f20 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f087c00b4a0 tx=0x7f087c00b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:38.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.429+0000 7f086f7fe700 1 -- 172.21.15.16:0/2309733399 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f087c011830 con 0x7f08880b4fb0 2023-05-20T01:28:38.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.429+0000 7f086f7fe700 1 -- 172.21.15.16:0/2309733399 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f087c011990 con 0x7f08880b4fb0 2023-05-20T01:28:38.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.429+0000 7f086f7fe700 1 -- 172.21.15.16:0/2309733399 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f087c011d40 con 0x7f08880b4fb0 2023-05-20T01:28:38.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.429+0000 7f088d37d700 1 -- 172.21.15.16:0/2309733399 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08880d27e0 con 0x7f08880b4fb0 2023-05-20T01:28:38.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.429+0000 7f088d37d700 1 -- 172.21.15.16:0/2309733399 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08880d2500 con 0x7f08880b4fb0 2023-05-20T01:28:38.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.430+0000 7f088d37d700 1 -- 172.21.15.16:0/2309733399 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08880b4a00 con 0x7f08880b4fb0 2023-05-20T01:28:38.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.431+0000 7f086f7fe700 1 -- 172.21.15.16:0/2309733399 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f087c01a400 con 0x7f08880b4fb0 2023-05-20T01:28:38.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.431+0000 7f086f7fe700 1 --2- 172.21.15.16:0/2309733399 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f087406ace0 0x7f087406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:38.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.431+0000 7f086f7fe700 1 -- 172.21.15.16:0/2309733399 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f087c089660 con 0x7f08880b4fb0 2023-05-20T01:28:38.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.431+0000 7f08867fc700 1 --2- 172.21.15.16:0/2309733399 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f087406ace0 0x7f087406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:38.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.432+0000 7f08867fc700 1 --2- 172.21.15.16:0/2309733399 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f087406ace0 0x7f087406d190 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f08780090b0 tx=0x7f0878008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:38.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.436+0000 7f086f7fe700 1 -- 172.21.15.16:0/2309733399 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f087c0545e0 con 0x7f08880b4fb0 2023-05-20T01:28:38.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.650+0000 7f088d37d700 1 -- 172.21.15.16:0/2309733399 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f0888066ce0 con 0x7f08880b4fb0 2023-05-20T01:28:38.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.651+0000 7f086f7fe700 1 -- 172.21.15.16:0/2309733399 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v136) v1 ==== 80+0+0 (secure 0 0 0) 0x7f087c057c00 con 0x7f08880b4fb0 2023-05-20T01:28:38.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.653+0000 7f088d37d700 1 -- 172.21.15.16:0/2309733399 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f087406ace0 msgr2=0x7f087406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:38.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.653+0000 7f088d37d700 1 --2- 172.21.15.16:0/2309733399 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f087406ace0 0x7f087406d190 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f08780090b0 tx=0x7f0878008030 comp rx=0 tx=0).stop 2023-05-20T01:28:38.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.653+0000 7f088d37d700 1 -- 172.21.15.16:0/2309733399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08880b4fb0 msgr2=0x7f0888153f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:38.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.653+0000 7f088d37d700 1 --2- 172.21.15.16:0/2309733399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08880b4fb0 0x7f0888153f20 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f087c00b4a0 tx=0x7f087c00b860 comp rx=0 tx=0).stop 2023-05-20T01:28:38.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.653+0000 7f088d37d700 1 -- 172.21.15.16:0/2309733399 shutdown_connections 2023-05-20T01:28:38.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.653+0000 7f088d37d700 1 --2- 172.21.15.16:0/2309733399 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f088815ce80 0x7f08880cc650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:38.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.653+0000 7f088d37d700 1 --2- 172.21.15.16:0/2309733399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08880b4fb0 0x7f0888153f20 unknown :-1 s=CLOSED pgs=1663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:38.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.653+0000 7f088d37d700 1 --2- 172.21.15.16:0/2309733399 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f087406ace0 0x7f087406d190 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:38.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.653+0000 7f088d37d700 1 --2- 172.21.15.16:0/2309733399 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f08880d3fe0 0x7f08880c8950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:38.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.653+0000 7f088d37d700 1 -- 172.21.15.16:0/2309733399 >> 172.21.15.16:0/2309733399 conn(0x7f088809c800 msgr2=0x7f08880c9e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:38.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.653+0000 7f088d37d700 1 -- 172.21.15.16:0/2309733399 shutdown_connections 2023-05-20T01:28:38.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:38.653+0000 7f088d37d700 1 -- 172.21.15.16:0/2309733399 wait complete. 2023-05-20T01:28:38.659 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2023-05-20T01:28:38.669 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-05-20T01:28:38.669 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:28:38.670 DEBUG:teuthology.orchestra.run.smithi016:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-05-20T01:28:38.823 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2023-05-20T01:28:38.824 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:28:38.824 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-05-20T01:28:38.842 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2023-05-20T01:28:39.051 INFO:teuthology.orchestra.run.smithi016.stdout:{"message":"","return_code":0} 2023-05-20T01:28:39.052 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2023-05-20T01:28:39.053 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-05-20T01:28:39.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.173+0000 7f76ac60f700 1 Processor -- start 2023-05-20T01:28:39.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.173+0000 7f76ac60f700 1 -- start start 2023-05-20T01:28:39.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.174+0000 7f76ac60f700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f76a40cdc40 0x7f76a40d0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:39.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.174+0000 7f76ac60f700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76a40d09b0 con 0x7f76a40cd710 2023-05-20T01:28:39.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.174+0000 7f76ac60f700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76a40d0c10 con 0x7f76a40ce010 2023-05-20T01:28:39.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.174+0000 7f76ac60f700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76a4070e70 con 0x7f76a40cdc40 2023-05-20T01:28:39.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.174+0000 7f76aa3ab700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f76a40cdc40 0x7f76a40d0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:39.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.174+0000 7f76a9baa700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f76a40cd710 0x7f76a40cf8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51894/0 (socket says 172.21.15.16:51894) 2023-05-20T01:28:39.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.174+0000 7f76aa3ab700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f76a40cdc40 0x7f76a40d0470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40366/0 (socket says 172.21.15.16:40366) 2023-05-20T01:28:39.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.174+0000 7f76a9baa700 1 -- 172.21.15.16:0/174650653 learned_addr learned my addr 172.21.15.16:0/174650653 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:28:39.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.175+0000 7f76aa3ab700 1 -- 172.21.15.16:0/174650653 >> v1:172.21.15.132:6789/0 conn(0x7f76a40ce010 legacy=0x7f76a40c4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:39.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.175+0000 7f76aa3ab700 1 -- 172.21.15.16:0/174650653 >> v1:172.21.15.16:6789/0 conn(0x7f76a40cd710 legacy=0x7f76a40cf8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:39.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.175+0000 7f76aa3ab700 1 -- 172.21.15.16:0/174650653 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76a40d0ef0 con 0x7f76a40cdc40 2023-05-20T01:28:39.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.175+0000 7f76a93a9700 1 -- 172.21.15.16:0/174650653 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3927217324 0 0) 0x7f76a40d09b0 con 0x7f76a40cd710 2023-05-20T01:28:39.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.175+0000 7f76aa3ab700 1 --2- 172.21.15.16:0/174650653 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76a40cdc40 0x7f76a40d0470 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f7694009840 tx=0x7f7694009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98302fc126603a16 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:39.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.175+0000 7f76a93a9700 1 -- 172.21.15.16:0/174650653 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7694011830 con 0x7f76a40cdc40 2023-05-20T01:28:39.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.175+0000 7f76a93a9700 1 -- 172.21.15.16:0/174650653 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7694011990 con 0x7f76a40cdc40 2023-05-20T01:28:39.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.175+0000 7f76a93a9700 1 -- 172.21.15.16:0/174650653 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7694011d20 con 0x7f76a40cdc40 2023-05-20T01:28:39.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.175+0000 7f76ac60f700 1 -- 172.21.15.16:0/174650653 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76a40cdc40 msgr2=0x7f76a40d0470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:39.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.175+0000 7f76ac60f700 1 --2- 172.21.15.16:0/174650653 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76a40cdc40 0x7f76a40d0470 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f7694009840 tx=0x7f7694009b50 comp rx=0 tx=0).stop 2023-05-20T01:28:39.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.176+0000 7f76ac60f700 1 -- 172.21.15.16:0/174650653 shutdown_connections 2023-05-20T01:28:39.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.176+0000 7f76ac60f700 1 --2- 172.21.15.16:0/174650653 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76a40cdc40 0x7f76a40d0470 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:39.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.176+0000 7f76ac60f700 1 -- 172.21.15.16:0/174650653 >> 172.21.15.16:0/174650653 conn(0x7f76a406e4c0 msgr2=0x7f76a406e8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:39.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.176+0000 7f76ac60f700 1 -- 172.21.15.16:0/174650653 shutdown_connections 2023-05-20T01:28:39.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.176+0000 7f76ac60f700 1 -- 172.21.15.16:0/174650653 wait complete. 2023-05-20T01:28:39.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.177+0000 7f76ac60f700 1 Processor -- start 2023-05-20T01:28:39.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.177+0000 7f76ac60f700 1 -- start start 2023-05-20T01:28:39.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.178+0000 7f76ac60f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76a40cd710 0x7f76a41518f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:39.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.178+0000 7f76ac60f700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f76a40cdc40 0x7f76a406bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:39.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.178+0000 7f76ac60f700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f76a40ce010 0x7f76a41e1810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:39.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.178+0000 7f76ac60f700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76a41e1d50 con 0x7f76a40cd710 2023-05-20T01:28:39.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.178+0000 7f76ac60f700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76a41e21e0 con 0x7f76a40cdc40 2023-05-20T01:28:39.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.178+0000 7f76ac60f700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76a406c660 con 0x7f76a40ce010 2023-05-20T01:28:39.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.178+0000 7f76aa3ab700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76a40cd710 0x7f76a41518f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:39.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.178+0000 7f76aa3ab700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76a40cd710 0x7f76a41518f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40372/0 (socket says 172.21.15.16:40372) 2023-05-20T01:28:39.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.178+0000 7f76aa3ab700 1 -- 172.21.15.16:0/758412713 learned_addr learned my addr 172.21.15.16:0/758412713 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:39.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.178+0000 7f76a9baa700 1 --2- 172.21.15.16:0/758412713 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f76a40cdc40 0x7f76a406bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:39.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.178+0000 7f76aabac700 1 --2- 172.21.15.16:0/758412713 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f76a40ce010 0x7f76a41e1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:39.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.178+0000 7f76aa3ab700 1 -- 172.21.15.16:0/758412713 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f76a40ce010 msgr2=0x7f76a41e1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:39.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.178+0000 7f76aa3ab700 1 --2- 172.21.15.16:0/758412713 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f76a40ce010 0x7f76a41e1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:39.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.178+0000 7f76aa3ab700 1 -- 172.21.15.16:0/758412713 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f76a40cdc40 msgr2=0x7f76a406bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:39.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.178+0000 7f76aa3ab700 1 --2- 172.21.15.16:0/758412713 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f76a40cdc40 0x7f76a406bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:39.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.178+0000 7f76aa3ab700 1 -- 172.21.15.16:0/758412713 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76940094a0 con 0x7f76a40cd710 2023-05-20T01:28:39.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.178+0000 7f76a9baa700 1 --2- 172.21.15.16:0/758412713 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f76a40cdc40 0x7f76a406bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:28:39.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.178+0000 7f76aa3ab700 1 --2- 172.21.15.16:0/758412713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76a40cd710 0x7f76a41518f0 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f7694014040 tx=0x7f7694004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:39.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.178+0000 7f769affd700 1 -- 172.21.15.16:0/758412713 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f769401a900 con 0x7f76a40cd710 2023-05-20T01:28:39.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.178+0000 7f769affd700 1 -- 172.21.15.16:0/758412713 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f769401aa60 con 0x7f76a40cd710 2023-05-20T01:28:39.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.178+0000 7f76ac60f700 1 -- 172.21.15.16:0/758412713 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76a40cf0e0 con 0x7f76a40cd710 2023-05-20T01:28:39.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.178+0000 7f769affd700 1 -- 172.21.15.16:0/758412713 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f769401add0 con 0x7f76a40cd710 2023-05-20T01:28:39.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.178+0000 7f76ac60f700 1 -- 172.21.15.16:0/758412713 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76a40cee00 con 0x7f76a40cd710 2023-05-20T01:28:39.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.179+0000 7f76ac60f700 1 -- 172.21.15.16:0/758412713 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76a40cf3c0 con 0x7f76a40cd710 2023-05-20T01:28:39.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.180+0000 7f769affd700 1 -- 172.21.15.16:0/758412713 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7694022070 con 0x7f76a40cd710 2023-05-20T01:28:39.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.180+0000 7f769affd700 1 --2- 172.21.15.16:0/758412713 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f768806ace0 0x7f768806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:39.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.180+0000 7f769affd700 1 -- 172.21.15.16:0/758412713 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f769408a9a0 con 0x7f76a40cd710 2023-05-20T01:28:39.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.180+0000 7f76a9baa700 1 --2- 172.21.15.16:0/758412713 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f768806ace0 0x7f768806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:39.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.181+0000 7f76a9baa700 1 --2- 172.21.15.16:0/758412713 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f768806ace0 0x7f768806d190 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f76a4151e30 tx=0x7f76a0007e50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:39.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.185+0000 7f769affd700 1 -- 172.21.15.16:0/758412713 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f7694055880 con 0x7f76a40cd710 2023-05-20T01:28:39.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.429+0000 7f76ac60f700 1 -- 172.21.15.16:0/758412713 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f76a40675f0 con 0x7f76a40cd710 2023-05-20T01:28:39.546 INFO:tasks.ceph.mon.a.smithi016.stderr:2023-05-20T01:28:39.542+0000 7f49eb8ae700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-05-20T01:28:39.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.548+0000 7f769affd700 1 -- 172.21.15.16:0/758412713 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v100) v1 ==== 161+0+0 (secure 0 0 0) 0x7f7694029020 con 0x7f76a40cd710 2023-05-20T01:28:39.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.550+0000 7f76ac60f700 1 -- 172.21.15.16:0/758412713 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f768806ace0 msgr2=0x7f768806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:39.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.550+0000 7f76ac60f700 1 --2- 172.21.15.16:0/758412713 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f768806ace0 0x7f768806d190 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f76a4151e30 tx=0x7f76a0007e50 comp rx=0 tx=0).stop 2023-05-20T01:28:39.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.550+0000 7f76ac60f700 1 -- 172.21.15.16:0/758412713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76a40cd710 msgr2=0x7f76a41518f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:39.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.550+0000 7f76ac60f700 1 --2- 172.21.15.16:0/758412713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76a40cd710 0x7f76a41518f0 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f7694014040 tx=0x7f7694004750 comp rx=0 tx=0).stop 2023-05-20T01:28:39.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.550+0000 7f76ac60f700 1 -- 172.21.15.16:0/758412713 shutdown_connections 2023-05-20T01:28:39.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.550+0000 7f76ac60f700 1 --2- 172.21.15.16:0/758412713 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f768806ace0 0x7f768806d190 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:39.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.550+0000 7f76ac60f700 1 --2- 172.21.15.16:0/758412713 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f76a40ce010 0x7f76a41e1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:39.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.550+0000 7f76ac60f700 1 --2- 172.21.15.16:0/758412713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76a40cd710 0x7f76a41518f0 unknown :-1 s=CLOSED pgs=1666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:39.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.550+0000 7f76ac60f700 1 --2- 172.21.15.16:0/758412713 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f76a40cdc40 0x7f76a406bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:39.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.550+0000 7f76ac60f700 1 -- 172.21.15.16:0/758412713 >> 172.21.15.16:0/758412713 conn(0x7f76a406e4c0 msgr2=0x7f76a40c84e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:39.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.550+0000 7f76ac60f700 1 -- 172.21.15.16:0/758412713 shutdown_connections 2023-05-20T01:28:39.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.550+0000 7f76ac60f700 1 -- 172.21.15.16:0/758412713 wait complete. 2023-05-20T01:28:39.557 INFO:teuthology.orchestra.run.smithi016.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-05-20T01:28:39.567 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys 10000000000.01000000 2023-05-20T01:28:39.591 INFO:tasks.ceph.mds.a.smithi016.stdout:starting mds.a at 2023-05-20T01:28:39.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.608+0000 7fbe3f94b080 1 Processor -- start 2023-05-20T01:28:39.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.608+0000 7fbe3f94b080 1 -- start start 2023-05-20T01:28:39.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.609+0000 7fbe3f94b080 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55d2908a5af0 0x55d29094ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:39.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.609+0000 7fbe3f94b080 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2908a5ec0 0x55d29093c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:39.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.609+0000 7fbe3f94b080 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d29094ef80 0x55d290874100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:39.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.609+0000 7fbe3f94b080 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d29081ca80 con 0x55d2908a5ec0 2023-05-20T01:28:39.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.609+0000 7fbe3f94b080 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2907d8e80 con 0x55d29094ef80 2023-05-20T01:28:39.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.609+0000 7fbe3f94b080 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d2907d8b20 con 0x55d2908a5af0 2023-05-20T01:28:39.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.609+0000 7fbe3524a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2908a5ec0 0x55d29093c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:39.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.609+0000 7fbe3524a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2908a5ec0 0x55d29093c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40400/0 (socket says 172.21.15.16:40400) 2023-05-20T01:28:39.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.609+0000 7fbe3524a700 1 -- 172.21.15.16:0/1237457918 learned_addr learned my addr 172.21.15.16:0/1237457918 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:39.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.609+0000 7fbe35a4b700 1 --2- 172.21.15.16:0/1237457918 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55d2908a5af0 0x55d29094ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:39.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.609+0000 7fbe3624c700 1 --2- 172.21.15.16:0/1237457918 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d29094ef80 0x55d290874100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:39.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.609+0000 7fbe3524a700 1 -- 172.21.15.16:0/1237457918 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55d2908a5af0 msgr2=0x55d29094ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:39.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.609+0000 7fbe3524a700 1 --2- 172.21.15.16:0/1237457918 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55d2908a5af0 0x55d29094ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:39.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.609+0000 7fbe3524a700 1 -- 172.21.15.16:0/1237457918 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d29094ef80 msgr2=0x55d290874100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:39.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.609+0000 7fbe3524a700 1 --2- 172.21.15.16:0/1237457918 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d29094ef80 0x55d290874100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:39.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.609+0000 7fbe3524a700 1 -- 172.21.15.16:0/1237457918 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe3000a100 con 0x55d2908a5ec0 2023-05-20T01:28:39.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.610+0000 7fbe3524a700 1 --2- 172.21.15.16:0/1237457918 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2908a5ec0 0x55d29093c430 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c001e30 tx=0x7fbe2c00b4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=608a24fa98dc8b2e server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:39.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.610+0000 7fbe27fff700 1 -- 172.21.15.16:0/1237457918 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe2c014070 con 0x55d2908a5ec0 2023-05-20T01:28:39.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.610+0000 7fbe27fff700 1 -- 172.21.15.16:0/1237457918 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe2c008030 con 0x55d2908a5ec0 2023-05-20T01:28:39.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.610+0000 7fbe27fff700 1 -- 172.21.15.16:0/1237457918 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe2c00bd20 con 0x55d2908a5ec0 2023-05-20T01:28:39.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.610+0000 7fbe3f94b080 1 -- 172.21.15.16:0/1237457918 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2908a5ec0 msgr2=0x55d29093c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:39.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.610+0000 7fbe3f94b080 1 --2- 172.21.15.16:0/1237457918 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2908a5ec0 0x55d29093c430 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c001e30 tx=0x7fbe2c00b4c0 comp rx=0 tx=0).stop 2023-05-20T01:28:39.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.610+0000 7fbe3f94b080 1 -- 172.21.15.16:0/1237457918 shutdown_connections 2023-05-20T01:28:39.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.610+0000 7fbe3f94b080 1 --2- 172.21.15.16:0/1237457918 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d29094ef80 0x55d290874100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:39.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.610+0000 7fbe3f94b080 1 --2- 172.21.15.16:0/1237457918 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55d2908a5af0 0x55d29094ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:39.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.610+0000 7fbe3f94b080 1 --2- 172.21.15.16:0/1237457918 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2908a5ec0 0x55d29093c430 unknown :-1 s=CLOSED pgs=1671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:39.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.610+0000 7fbe3f94b080 1 -- 172.21.15.16:0/1237457918 >> 172.21.15.16:0/1237457918 conn(0x55d290873d30 msgr2=0x55d2908c0cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:39.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.610+0000 7fbe3f94b080 1 -- 172.21.15.16:0/1237457918 shutdown_connections 2023-05-20T01:28:39.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.610+0000 7fbe3f94b080 1 -- 172.21.15.16:0/1237457918 wait complete. 2023-05-20T01:28:39.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.611+0000 7fbe3f94b080 1 Processor -- start 2023-05-20T01:28:39.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.611+0000 7fbe3f94b080 1 -- start start 2023-05-20T01:28:39.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.612+0000 7fbe3f94b080 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2908a5af0 0x55d290937580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:39.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.612+0000 7fbe3f94b080 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d2908a5ec0 0x55d29094e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:39.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.612+0000 7fbe3f94b080 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55d29094ef80 0x55d29093f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:39.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.612+0000 7fbe3f94b080 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2907aefe0 con 0x55d2908a5af0 2023-05-20T01:28:39.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.612+0000 7fbe3f94b080 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d29093bc00 con 0x55d2908a5ec0 2023-05-20T01:28:39.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.612+0000 7fbe3f94b080 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d29093cd30 con 0x55d29094ef80 2023-05-20T01:28:39.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.612+0000 7fbe35a4b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2908a5af0 0x55d290937580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:39.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.612+0000 7fbe35a4b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2908a5af0 0x55d290937580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40404/0 (socket says 172.21.15.16:40404) 2023-05-20T01:28:39.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.612+0000 7fbe35a4b700 1 -- 172.21.15.16:0/1119969562 learned_addr learned my addr 172.21.15.16:0/1119969562 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:39.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.612+0000 7fbe3524a700 1 --2- 172.21.15.16:0/1119969562 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d2908a5ec0 0x55d29094e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:39.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.612+0000 7fbe3624c700 1 --2- 172.21.15.16:0/1119969562 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55d29094ef80 0x55d29093f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:39.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.612+0000 7fbe35a4b700 1 -- 172.21.15.16:0/1119969562 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55d29094ef80 msgr2=0x55d29093f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:39.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.612+0000 7fbe35a4b700 1 --2- 172.21.15.16:0/1119969562 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55d29094ef80 0x55d29093f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:39.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.612+0000 7fbe35a4b700 1 -- 172.21.15.16:0/1119969562 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d2908a5ec0 msgr2=0x55d29094e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:39.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.612+0000 7fbe35a4b700 1 --2- 172.21.15.16:0/1119969562 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d2908a5ec0 0x55d29094e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:39.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.612+0000 7fbe35a4b700 1 -- 172.21.15.16:0/1119969562 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe2c00a040 con 0x55d2908a5af0 2023-05-20T01:28:39.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.612+0000 7fbe35a4b700 1 --2- 172.21.15.16:0/1119969562 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2908a5af0 0x55d290937580 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7fbe2800c580 tx=0x7fbe2800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:39.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.612+0000 7fbe26ffd700 1 -- 172.21.15.16:0/1119969562 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe28016030 con 0x55d2908a5af0 2023-05-20T01:28:39.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.612+0000 7fbe26ffd700 1 -- 172.21.15.16:0/1119969562 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe28005070 con 0x55d2908a5af0 2023-05-20T01:28:39.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.612+0000 7fbe3f94b080 1 -- 172.21.15.16:0/1119969562 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55d2907c6a80 con 0x55d2908a5af0 2023-05-20T01:28:39.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.612+0000 7fbe26ffd700 1 -- 172.21.15.16:0/1119969562 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe280116a0 con 0x55d2908a5af0 2023-05-20T01:28:39.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.612+0000 7fbe3f94b080 1 -- 172.21.15.16:0/1119969562 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d2907ba830 con 0x55d2908a5af0 2023-05-20T01:28:39.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.614+0000 7fbe26ffd700 1 -- 172.21.15.16:0/1119969562 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fbe28019070 con 0x55d2908a5af0 2023-05-20T01:28:39.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.614+0000 7fbe26ffd700 1 --2- 172.21.15.16:0/1119969562 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fbe18069600 0x7fbe1806bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:39.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.614+0000 7fbe26ffd700 1 -- 172.21.15.16:0/1119969562 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6468+0+0 (secure 0 0 0) 0x7fbe28017070 con 0x55d2908a5af0 2023-05-20T01:28:39.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.614+0000 7fbe3524a700 1 --2- 172.21.15.16:0/1119969562 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fbe18069600 0x7fbe1806bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:39.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.614+0000 7fbe3f94b080 1 --2- 172.21.15.16:0/1119969562 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x55d29093fbc0 0x55d290937df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:39.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.614+0000 7fbe3f94b080 1 -- 172.21.15.16:0/1119969562 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:1 18.5 18:a40620de:::10000000000.01000000:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e137) v8 -- 0x55d29094e830 con 0x55d29093fbc0 2023-05-20T01:28:39.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.614+0000 7fbe3624c700 1 --2- 172.21.15.16:0/1119969562 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x55d29093fbc0 0x55d290937df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:39.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.615+0000 7fbe3524a700 1 --2- 172.21.15.16:0/1119969562 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fbe18069600 0x7fbe1806bab0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c001e30 tx=0x7fbe2c009000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:39.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.615+0000 7fbe3624c700 1 --2- 172.21.15.16:0/1119969562 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x55d29093fbc0 0x55d290937df0 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:39.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.617+0000 7fbe3624c700 1 -- 172.21.15.16:0/1119969562 <== osd.1 v2:172.21.15.16:6808/3860264220 1 ==== osd_op_reply(1 10000000000.01000000 [omap-get-keys out=560b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+560 (crc 0 0 0) 0x7fbe30017040 con 0x55d29093fbc0 2023-05-20T01:28:39.629 INFO:teuthology.orchestra.run.smithi016.stdout:10_head 2023-05-20T01:28:39.629 INFO:teuthology.orchestra.run.smithi016.stdout:11_head 2023-05-20T01:28:39.629 INFO:teuthology.orchestra.run.smithi016.stdout:12_head 2023-05-20T01:28:39.629 INFO:teuthology.orchestra.run.smithi016.stdout:15_head 2023-05-20T01:28:39.629 INFO:teuthology.orchestra.run.smithi016.stdout:19_head 2023-05-20T01:28:39.630 INFO:teuthology.orchestra.run.smithi016.stdout:1_head 2023-05-20T01:28:39.630 INFO:teuthology.orchestra.run.smithi016.stdout:20_head 2023-05-20T01:28:39.630 INFO:teuthology.orchestra.run.smithi016.stdout:21_head 2023-05-20T01:28:39.630 INFO:teuthology.orchestra.run.smithi016.stdout:27_head 2023-05-20T01:28:39.630 INFO:teuthology.orchestra.run.smithi016.stdout:28_head 2023-05-20T01:28:39.630 INFO:teuthology.orchestra.run.smithi016.stdout:2_head 2023-05-20T01:28:39.630 INFO:teuthology.orchestra.run.smithi016.stdout:31_head 2023-05-20T01:28:39.631 INFO:teuthology.orchestra.run.smithi016.stdout:34_head 2023-05-20T01:28:39.631 INFO:teuthology.orchestra.run.smithi016.stdout:36_head 2023-05-20T01:28:39.631 INFO:teuthology.orchestra.run.smithi016.stdout:38_head 2023-05-20T01:28:39.631 INFO:teuthology.orchestra.run.smithi016.stdout:39_head 2023-05-20T01:28:39.631 INFO:teuthology.orchestra.run.smithi016.stdout:3_head 2023-05-20T01:28:39.631 INFO:teuthology.orchestra.run.smithi016.stdout:42_head 2023-05-20T01:28:39.631 INFO:teuthology.orchestra.run.smithi016.stdout:43_head 2023-05-20T01:28:39.632 INFO:teuthology.orchestra.run.smithi016.stdout:46_head 2023-05-20T01:28:39.632 INFO:teuthology.orchestra.run.smithi016.stdout:4_head 2023-05-20T01:28:39.632 INFO:teuthology.orchestra.run.smithi016.stdout:50_head 2023-05-20T01:28:39.632 INFO:teuthology.orchestra.run.smithi016.stdout:52_head 2023-05-20T01:28:39.632 INFO:teuthology.orchestra.run.smithi016.stdout:53_head 2023-05-20T01:28:39.632 INFO:teuthology.orchestra.run.smithi016.stdout:55_head 2023-05-20T01:28:39.632 INFO:teuthology.orchestra.run.smithi016.stdout:59_head 2023-05-20T01:28:39.633 INFO:teuthology.orchestra.run.smithi016.stdout:61_head 2023-05-20T01:28:39.633 INFO:teuthology.orchestra.run.smithi016.stdout:63_head 2023-05-20T01:28:39.633 INFO:teuthology.orchestra.run.smithi016.stdout:64_head 2023-05-20T01:28:39.633 INFO:teuthology.orchestra.run.smithi016.stdout:65_head 2023-05-20T01:28:39.633 INFO:teuthology.orchestra.run.smithi016.stdout:66_head 2023-05-20T01:28:39.633 INFO:teuthology.orchestra.run.smithi016.stdout:69_head 2023-05-20T01:28:39.633 INFO:teuthology.orchestra.run.smithi016.stdout:70_head 2023-05-20T01:28:39.633 INFO:teuthology.orchestra.run.smithi016.stdout:73_head 2023-05-20T01:28:39.634 INFO:teuthology.orchestra.run.smithi016.stdout:74_head 2023-05-20T01:28:39.634 INFO:teuthology.orchestra.run.smithi016.stdout:75_head 2023-05-20T01:28:39.634 INFO:teuthology.orchestra.run.smithi016.stdout:77_head 2023-05-20T01:28:39.634 INFO:teuthology.orchestra.run.smithi016.stdout:80_head 2023-05-20T01:28:39.634 INFO:teuthology.orchestra.run.smithi016.stdout:83_head 2023-05-20T01:28:39.634 INFO:teuthology.orchestra.run.smithi016.stdout:84_head 2023-05-20T01:28:39.634 INFO:teuthology.orchestra.run.smithi016.stdout:85_head 2023-05-20T01:28:39.634 INFO:teuthology.orchestra.run.smithi016.stdout:86_head 2023-05-20T01:28:39.635 INFO:teuthology.orchestra.run.smithi016.stdout:87_head 2023-05-20T01:28:39.635 INFO:teuthology.orchestra.run.smithi016.stdout:88_head 2023-05-20T01:28:39.635 INFO:teuthology.orchestra.run.smithi016.stdout:8_head 2023-05-20T01:28:39.635 INFO:teuthology.orchestra.run.smithi016.stdout:90_head 2023-05-20T01:28:39.635 INFO:teuthology.orchestra.run.smithi016.stdout:92_head 2023-05-20T01:28:39.635 INFO:teuthology.orchestra.run.smithi016.stdout:95_head 2023-05-20T01:28:39.635 INFO:teuthology.orchestra.run.smithi016.stdout:96_head 2023-05-20T01:28:39.635 INFO:teuthology.orchestra.run.smithi016.stdout:97_head 2023-05-20T01:28:39.636 INFO:teuthology.orchestra.run.smithi016.stdout:9_head 2023-05-20T01:28:39.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.618+0000 7fbe3f94b080 1 -- 172.21.15.16:0/1119969562 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x55d29093fbc0 msgr2=0x55d290937df0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:39.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.618+0000 7fbe3f94b080 1 --2- 172.21.15.16:0/1119969562 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x55d29093fbc0 0x55d290937df0 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:39.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.618+0000 7fbe3f94b080 1 -- 172.21.15.16:0/1119969562 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fbe18069600 msgr2=0x7fbe1806bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:39.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.618+0000 7fbe3f94b080 1 --2- 172.21.15.16:0/1119969562 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fbe18069600 0x7fbe1806bab0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c001e30 tx=0x7fbe2c009000 comp rx=0 tx=0).stop 2023-05-20T01:28:39.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.618+0000 7fbe3f94b080 1 -- 172.21.15.16:0/1119969562 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2908a5af0 msgr2=0x55d290937580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:39.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.618+0000 7fbe3f94b080 1 --2- 172.21.15.16:0/1119969562 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2908a5af0 0x55d290937580 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7fbe2800c580 tx=0x7fbe2800c890 comp rx=0 tx=0).stop 2023-05-20T01:28:39.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.618+0000 7fbe3624c700 1 -- 172.21.15.16:0/1119969562 reap_dead start 2023-05-20T01:28:39.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.618+0000 7fbe3f94b080 1 -- 172.21.15.16:0/1119969562 shutdown_connections 2023-05-20T01:28:39.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.618+0000 7fbe3f94b080 1 -- 172.21.15.16:0/1119969562 >> 172.21.15.16:0/1119969562 conn(0x55d290873d30 msgr2=0x55d29094c530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:39.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.618+0000 7fbe3f94b080 1 -- 172.21.15.16:0/1119969562 shutdown_connections 2023-05-20T01:28:39.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.618+0000 7fbe3f94b080 1 -- 172.21.15.16:0/1119969562 wait complete. 2023-05-20T01:28:39.639 INFO:tasks.cephfs.test_data_scan:victim_key=21_head 2023-05-20T01:28:39.639 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmomapkey 10000000000.01000000 21_head 2023-05-20T01:28:39.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.717+0000 7f4b8afc2080 1 Processor -- start 2023-05-20T01:28:39.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.717+0000 7f4b8afc2080 1 -- start start 2023-05-20T01:28:39.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.718+0000 7f4b8afc2080 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e28edb2af0 0x55e28ee580f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:39.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.718+0000 7f4b8afc2080 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e28edb2ec0 0x55e28ee58630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:39.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.718+0000 7f4b8afc2080 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e28ee5bf80 0x55e28ee59aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:39.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.718+0000 7f4b8afc2080 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e28ece5e80 con 0x55e28edb2ec0 2023-05-20T01:28:39.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.718+0000 7f4b8afc2080 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e28ece5b20 con 0x55e28edb2af0 2023-05-20T01:28:39.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.718+0000 7f4b8afc2080 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e28ecbbfe0 con 0x55e28ee5bf80 2023-05-20T01:28:39.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.718+0000 7f4b808c1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e28edb2ec0 0x55e28ee58630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:39.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.718+0000 7f4b808c1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e28edb2ec0 0x55e28ee58630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40422/0 (socket says 172.21.15.16:40422) 2023-05-20T01:28:39.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.718+0000 7f4b810c2700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e28edb2af0 0x55e28ee580f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:39.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.718+0000 7f4b808c1700 1 -- 172.21.15.16:0/1143795166 learned_addr learned my addr 172.21.15.16:0/1143795166 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:39.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.718+0000 7f4b818c3700 1 --2- 172.21.15.16:0/1143795166 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e28ee5bf80 0x55e28ee59aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:39.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.718+0000 7f4b808c1700 1 -- 172.21.15.16:0/1143795166 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e28ee5bf80 msgr2=0x55e28ee59aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:39.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.718+0000 7f4b808c1700 1 --2- 172.21.15.16:0/1143795166 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e28ee5bf80 0x55e28ee59aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:39.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.718+0000 7f4b808c1700 1 -- 172.21.15.16:0/1143795166 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e28edb2af0 msgr2=0x55e28ee580f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:39.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.718+0000 7f4b808c1700 1 --2- 172.21.15.16:0/1143795166 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e28edb2af0 0x55e28ee580f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:39.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.719+0000 7f4b808c1700 1 -- 172.21.15.16:0/1143795166 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b78005de0 con 0x55e28edb2ec0 2023-05-20T01:28:39.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.719+0000 7f4b808c1700 1 --2- 172.21.15.16:0/1143795166 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e28edb2ec0 0x55e28ee58630 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7f4b74002e10 tx=0x7f4b7400b4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bea92847ac97c45 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:39.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.719+0000 7f4b737fe700 1 -- 172.21.15.16:0/1143795166 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b74002ea0 con 0x55e28edb2ec0 2023-05-20T01:28:39.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.719+0000 7f4b737fe700 1 -- 172.21.15.16:0/1143795166 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b74008030 con 0x55e28edb2ec0 2023-05-20T01:28:39.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.719+0000 7f4b737fe700 1 -- 172.21.15.16:0/1143795166 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b7400f400 con 0x55e28edb2ec0 2023-05-20T01:28:39.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.719+0000 7f4b8afc2080 1 -- 172.21.15.16:0/1143795166 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e28edb2ec0 msgr2=0x55e28ee58630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:39.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.719+0000 7f4b8afc2080 1 --2- 172.21.15.16:0/1143795166 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e28edb2ec0 0x55e28ee58630 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7f4b74002e10 tx=0x7f4b7400b4c0 comp rx=0 tx=0).stop 2023-05-20T01:28:39.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.719+0000 7f4b8afc2080 1 -- 172.21.15.16:0/1143795166 shutdown_connections 2023-05-20T01:28:39.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.719+0000 7f4b8afc2080 1 --2- 172.21.15.16:0/1143795166 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e28ee5bf80 0x55e28ee59aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:39.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.719+0000 7f4b8afc2080 1 --2- 172.21.15.16:0/1143795166 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e28edb2af0 0x55e28ee580f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:39.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.719+0000 7f4b8afc2080 1 --2- 172.21.15.16:0/1143795166 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e28edb2ec0 0x55e28ee58630 unknown :-1 s=CLOSED pgs=1674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:39.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.719+0000 7f4b8afc2080 1 -- 172.21.15.16:0/1143795166 >> 172.21.15.16:0/1143795166 conn(0x55e28ed80d30 msgr2=0x55e28ed81100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:39.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.719+0000 7f4b8afc2080 1 -- 172.21.15.16:0/1143795166 shutdown_connections 2023-05-20T01:28:39.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.719+0000 7f4b8afc2080 1 -- 172.21.15.16:0/1143795166 wait complete. 2023-05-20T01:28:39.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.720+0000 7f4b8afc2080 1 Processor -- start 2023-05-20T01:28:39.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.720+0000 7f4b8afc2080 1 -- start start 2023-05-20T01:28:39.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.721+0000 7f4b8afc2080 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e28edb2af0 0x55e28ee3aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:39.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.721+0000 7f4b8afc2080 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e28edb2ec0 0x55e28ee56d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:39.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.721+0000 7f4b8afc2080 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e28ee5bf80 0x55e28ee57290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:39.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.721+0000 7f4b8afc2080 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e28eeefcc0 con 0x55e28edb2ec0 2023-05-20T01:28:39.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.721+0000 7f4b8afc2080 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e28ed82460 con 0x55e28edb2af0 2023-05-20T01:28:39.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.721+0000 7f4b8afc2080 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e28ee518c0 con 0x55e28ee5bf80 2023-05-20T01:28:39.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.721+0000 7f4b810c2700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e28edb2af0 0x55e28ee3aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:39.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.721+0000 7f4b808c1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e28edb2ec0 0x55e28ee56d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:39.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.721+0000 7f4b810c2700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e28edb2af0 0x55e28ee3aec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.16:37448/0 (socket says 172.21.15.16:37448) 2023-05-20T01:28:39.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.721+0000 7f4b810c2700 1 -- 172.21.15.16:0/2452444137 learned_addr learned my addr 172.21.15.16:0/2452444137 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:39.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.721+0000 7f4b818c3700 1 --2- 172.21.15.16:0/2452444137 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e28ee5bf80 0x55e28ee57290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:39.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.721+0000 7f4b808c1700 1 -- 172.21.15.16:0/2452444137 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e28ee5bf80 msgr2=0x55e28ee57290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:39.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.721+0000 7f4b808c1700 1 --2- 172.21.15.16:0/2452444137 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e28ee5bf80 0x55e28ee57290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:39.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.721+0000 7f4b808c1700 1 -- 172.21.15.16:0/2452444137 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e28edb2af0 msgr2=0x55e28ee3aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:39.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.721+0000 7f4b808c1700 1 --2- 172.21.15.16:0/2452444137 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e28edb2af0 0x55e28ee3aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:39.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.721+0000 7f4b808c1700 1 -- 172.21.15.16:0/2452444137 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b7400a040 con 0x55e28edb2ec0 2023-05-20T01:28:39.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.721+0000 7f4b808c1700 1 --2- 172.21.15.16:0/2452444137 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e28edb2ec0 0x55e28ee56d50 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f4b74002de0 tx=0x7f4b74001fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:39.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.722+0000 7f4b727fc700 1 -- 172.21.15.16:0/2452444137 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b74002ea0 con 0x55e28edb2ec0 2023-05-20T01:28:39.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.722+0000 7f4b727fc700 1 -- 172.21.15.16:0/2452444137 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b74002a20 con 0x55e28edb2ec0 2023-05-20T01:28:39.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.722+0000 7f4b8afc2080 1 -- 172.21.15.16:0/2452444137 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55e28ecc74a0 con 0x55e28edb2ec0 2023-05-20T01:28:39.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.722+0000 7f4b727fc700 1 -- 172.21.15.16:0/2452444137 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b74024680 con 0x55e28edb2ec0 2023-05-20T01:28:39.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.722+0000 7f4b8afc2080 1 -- 172.21.15.16:0/2452444137 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e28ecd3a80 con 0x55e28edb2ec0 2023-05-20T01:28:39.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.723+0000 7f4b727fc700 1 -- 172.21.15.16:0/2452444137 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4b74018420 con 0x55e28edb2ec0 2023-05-20T01:28:39.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.723+0000 7f4b727fc700 1 --2- 172.21.15.16:0/2452444137 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f4b64069600 0x7f4b6406bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:39.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.724+0000 7f4b810c2700 1 --2- 172.21.15.16:0/2452444137 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f4b64069600 0x7f4b6406bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:39.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.724+0000 7f4b727fc700 1 -- 172.21.15.16:0/2452444137 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6468+0+0 (secure 0 0 0) 0x7f4b740893a0 con 0x55e28edb2ec0 2023-05-20T01:28:39.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.724+0000 7f4b8afc2080 1 --2- 172.21.15.16:0/2452444137 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x55e28ee4de80 0x55e28ee4e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:39.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.724+0000 7f4b8afc2080 1 -- 172.21.15.16:0/2452444137 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:1 18.5 18:a40620de:::10000000000.01000000:head [omap-rm-keys in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137) v8 -- 0x55e28ee3b400 con 0x55e28ee4de80 2023-05-20T01:28:39.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.724+0000 7f4b818c3700 1 --2- 172.21.15.16:0/2452444137 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x55e28ee4de80 0x55e28ee4e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:39.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.724+0000 7f4b810c2700 1 --2- 172.21.15.16:0/2452444137 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f4b64069600 0x7f4b6406bab0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f4b7800a4f0 tx=0x7f4b7800f000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:39.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.724+0000 7f4b818c3700 1 --2- 172.21.15.16:0/2452444137 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x55e28ee4de80 0x55e28ee4e280 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:39.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.726+0000 7f4b818c3700 1 -- 172.21.15.16:0/2452444137 <== osd.1 v2:172.21.15.16:6808/3860264220 1 ==== osd_op_reply(1 10000000000.01000000 [omap-rm-keys] v137'3 uv3 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f4b7c00d8c0 con 0x55e28ee4de80 2023-05-20T01:28:39.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.726+0000 7f4b8afc2080 1 -- 172.21.15.16:0/2452444137 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x55e28ee4de80 msgr2=0x55e28ee4e280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:39.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.726+0000 7f4b8afc2080 1 --2- 172.21.15.16:0/2452444137 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x55e28ee4de80 0x55e28ee4e280 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:39.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.726+0000 7f4b8afc2080 1 -- 172.21.15.16:0/2452444137 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f4b64069600 msgr2=0x7f4b6406bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:39.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.726+0000 7f4b8afc2080 1 --2- 172.21.15.16:0/2452444137 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f4b64069600 0x7f4b6406bab0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f4b7800a4f0 tx=0x7f4b7800f000 comp rx=0 tx=0).stop 2023-05-20T01:28:39.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.726+0000 7f4b8afc2080 1 -- 172.21.15.16:0/2452444137 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e28edb2ec0 msgr2=0x55e28ee56d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:39.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.726+0000 7f4b8afc2080 1 --2- 172.21.15.16:0/2452444137 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e28edb2ec0 0x55e28ee56d50 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f4b74002de0 tx=0x7f4b74001fe0 comp rx=0 tx=0).stop 2023-05-20T01:28:39.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.726+0000 7f4b818c3700 1 -- 172.21.15.16:0/2452444137 reap_dead start 2023-05-20T01:28:39.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.726+0000 7f4b8afc2080 1 -- 172.21.15.16:0/2452444137 shutdown_connections 2023-05-20T01:28:39.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.726+0000 7f4b8afc2080 1 -- 172.21.15.16:0/2452444137 >> 172.21.15.16:0/2452444137 conn(0x55e28ed80d30 msgr2=0x55e28edafab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:39.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.726+0000 7f4b8afc2080 1 -- 172.21.15.16:0/2452444137 shutdown_connections 2023-05-20T01:28:39.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.726+0000 7f4b8afc2080 1 -- 172.21.15.16:0/2452444137 wait complete. 2023-05-20T01:28:39.743 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-05-20T01:28:39.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.868+0000 7f2702220700 1 Processor -- start 2023-05-20T01:28:39.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.868+0000 7f2702220700 1 -- start start 2023-05-20T01:28:39.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.869+0000 7f2702220700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f26fc06f3e0 0x7f26fc06f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:39.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.869+0000 7f2702220700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f26fc0713c0 0x7f26fc071840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:39.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.869+0000 7f2702220700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26fc1614a0 con 0x7f26fc0cf2e0 2023-05-20T01:28:39.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.869+0000 7f2702220700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26fc15d160 con 0x7f26fc06f3e0 2023-05-20T01:28:39.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.869+0000 7f2702220700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26fc0cfe60 con 0x7f26fc0713c0 2023-05-20T01:28:39.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.869+0000 7f26fb7fe700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f26fc06f3e0 0x7f26fc06f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:39.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.869+0000 7f26fbfff700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f26fc0713c0 0x7f26fc071840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:39.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.869+0000 7f26faffd700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f26fc0cf2e0 0x7f26fc0cf750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.16:33448/0 (socket says 172.21.15.16:33448) 2023-05-20T01:28:39.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.869+0000 7f26faffd700 1 -- 172.21.15.16:0/969896898 learned_addr learned my addr 172.21.15.16:0/969896898 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:28:39.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.870+0000 7f26fbfff700 1 -- 172.21.15.16:0/969896898 >> v2:172.21.15.132:3300/0 conn(0x7f26fc06f3e0 msgr2=0x7f26fc06f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:39.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.870+0000 7f26fbfff700 1 --2- 172.21.15.16:0/969896898 >> v2:172.21.15.132:3300/0 conn(0x7f26fc06f3e0 0x7f26fc06f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:39.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.870+0000 7f26fbfff700 1 -- 172.21.15.16:0/969896898 >> v1:172.21.15.132:6789/0 conn(0x7f26fc0cf2e0 legacy=0x7f26fc0cf750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:39.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.870+0000 7f26fbfff700 1 -- 172.21.15.16:0/969896898 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26fc06fcf0 con 0x7f26fc0713c0 2023-05-20T01:28:39.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.870+0000 7f26fb7fe700 1 --2- 172.21.15.16:0/969896898 >> v2:172.21.15.132:3300/0 conn(0x7f26fc06f3e0 0x7f26fc06f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-05-20T01:28:39.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.870+0000 7f26fbfff700 1 --2- 172.21.15.16:0/969896898 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f26fc0713c0 0x7f26fc071840 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f26f4009d90 tx=0x7f26f400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=506db17c61895c96 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:39.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.870+0000 7f26fa7fc700 1 -- 172.21.15.16:0/969896898 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f400e070 con 0x7f26fc0713c0 2023-05-20T01:28:39.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.870+0000 7f26fa7fc700 1 -- 172.21.15.16:0/969896898 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26f400bb20 con 0x7f26fc0713c0 2023-05-20T01:28:39.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.870+0000 7f26fa7fc700 1 -- 172.21.15.16:0/969896898 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f4012430 con 0x7f26fc0713c0 2023-05-20T01:28:39.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.871+0000 7f2702220700 1 -- 172.21.15.16:0/969896898 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f26fc0713c0 msgr2=0x7f26fc071840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:39.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.871+0000 7f2702220700 1 --2- 172.21.15.16:0/969896898 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f26fc0713c0 0x7f26fc071840 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f26f4009d90 tx=0x7f26f400b330 comp rx=0 tx=0).stop 2023-05-20T01:28:39.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.871+0000 7f2702220700 1 -- 172.21.15.16:0/969896898 shutdown_connections 2023-05-20T01:28:39.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.871+0000 7f2702220700 1 --2- 172.21.15.16:0/969896898 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f26fc0713c0 0x7f26fc071840 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:39.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.871+0000 7f2702220700 1 --2- 172.21.15.16:0/969896898 >> v2:172.21.15.132:3300/0 conn(0x7f26fc06f3e0 0x7f26fc06f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:39.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.871+0000 7f2702220700 1 -- 172.21.15.16:0/969896898 >> 172.21.15.16:0/969896898 conn(0x7f26fc0b2920 msgr2=0x7f26fc0d0120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:39.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.871+0000 7f2702220700 1 -- 172.21.15.16:0/969896898 shutdown_connections 2023-05-20T01:28:39.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.871+0000 7f2702220700 1 -- 172.21.15.16:0/969896898 wait complete. 2023-05-20T01:28:39.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.872+0000 7f2702220700 1 Processor -- start 2023-05-20T01:28:39.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.872+0000 7f2702220700 1 -- start start 2023-05-20T01:28:39.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.873+0000 7f2702220700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f26fc06f3e0 0x7f26fc154770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:39.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.873+0000 7f2702220700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f26fc0713c0 0x7f26fc0b5680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:39.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.873+0000 7f2702220700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26fc0cf2e0 0x7f26fc15efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:39.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.873+0000 7f2702220700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26fc0aed30 con 0x7f26fc0cf2e0 2023-05-20T01:28:39.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.873+0000 7f2702220700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26fc0ae490 con 0x7f26fc0713c0 2023-05-20T01:28:39.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.873+0000 7f2702220700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26fc0ae770 con 0x7f26fc06f3e0 2023-05-20T01:28:39.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.873+0000 7f26fbfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26fc0cf2e0 0x7f26fc15efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:39.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.873+0000 7f26fbfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26fc0cf2e0 0x7f26fc15efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40446/0 (socket says 172.21.15.16:40446) 2023-05-20T01:28:39.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.873+0000 7f26fbfff700 1 -- 172.21.15.16:0/1187988084 learned_addr learned my addr 172.21.15.16:0/1187988084 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:39.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.873+0000 7f26fb7fe700 1 --2- 172.21.15.16:0/1187988084 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f26fc06f3e0 0x7f26fc154770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:39.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.873+0000 7f26faffd700 1 --2- 172.21.15.16:0/1187988084 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f26fc0713c0 0x7f26fc0b5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:39.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.873+0000 7f26fbfff700 1 -- 172.21.15.16:0/1187988084 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f26fc06f3e0 msgr2=0x7f26fc154770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:39.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.873+0000 7f26fbfff700 1 --2- 172.21.15.16:0/1187988084 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f26fc06f3e0 0x7f26fc154770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:39.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.873+0000 7f26fbfff700 1 -- 172.21.15.16:0/1187988084 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f26fc0713c0 msgr2=0x7f26fc0b5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:39.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.873+0000 7f26fbfff700 1 --2- 172.21.15.16:0/1187988084 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f26fc0713c0 0x7f26fc0b5680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:39.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.873+0000 7f26fbfff700 1 -- 172.21.15.16:0/1187988084 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26f00094a0 con 0x7f26fc0cf2e0 2023-05-20T01:28:39.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.874+0000 7f26fbfff700 1 --2- 172.21.15.16:0/1187988084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26fc0cf2e0 0x7f26fc15efb0 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f26f4014040 tx=0x7f26f4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:39.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.874+0000 7f26e3fff700 1 -- 172.21.15.16:0/1187988084 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f4003ea0 con 0x7f26fc0cf2e0 2023-05-20T01:28:39.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.874+0000 7f26e3fff700 1 -- 172.21.15.16:0/1187988084 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26f4020070 con 0x7f26fc0cf2e0 2023-05-20T01:28:39.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.874+0000 7f2702220700 1 -- 172.21.15.16:0/1187988084 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26f4009a70 con 0x7f26fc0cf2e0 2023-05-20T01:28:39.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.874+0000 7f26e3fff700 1 -- 172.21.15.16:0/1187988084 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f40098f0 con 0x7f26fc0cf2e0 2023-05-20T01:28:39.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.874+0000 7f2702220700 1 -- 172.21.15.16:0/1187988084 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26fc160b70 con 0x7f26fc0cf2e0 2023-05-20T01:28:39.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.875+0000 7f2702220700 1 -- 172.21.15.16:0/1187988084 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26fc160890 con 0x7f26fc0cf2e0 2023-05-20T01:28:39.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.875+0000 7f26e3fff700 1 -- 172.21.15.16:0/1187988084 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f26f4012430 con 0x7f26fc0cf2e0 2023-05-20T01:28:39.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.876+0000 7f26e3fff700 1 --2- 172.21.15.16:0/1187988084 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f26e806ace0 0x7f26e806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:39.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.876+0000 7f26e3fff700 1 -- 172.21.15.16:0/1187988084 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6468+0+0 (secure 0 0 0) 0x7f26f4022070 con 0x7f26fc0cf2e0 2023-05-20T01:28:39.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.876+0000 7f26fb7fe700 1 --2- 172.21.15.16:0/1187988084 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f26e806ace0 0x7f26e806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:39.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.876+0000 7f26fb7fe700 1 --2- 172.21.15.16:0/1187988084 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f26e806ace0 0x7f26e806d190 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f26fc1602b0 tx=0x7f26f0009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:39.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:39.880+0000 7f26e3fff700 1 -- 172.21.15.16:0/1187988084 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f26f4059fc0 con 0x7f26fc0cf2e0 2023-05-20T01:28:40.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:40.125+0000 7f2702220700 1 -- 172.21.15.16:0/1187988084 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f26fc160e50 con 0x7f26fc0cf2e0 2023-05-20T01:28:40.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:40.554+0000 7f26e3fff700 1 -- 172.21.15.16:0/1187988084 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v101) v1 ==== 162+0+0 (secure 0 0 0) 0x7f26f4059b50 con 0x7f26fc0cf2e0 2023-05-20T01:28:40.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:40.557+0000 7f2702220700 1 -- 172.21.15.16:0/1187988084 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f26e806ace0 msgr2=0x7f26e806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:40.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:40.557+0000 7f2702220700 1 --2- 172.21.15.16:0/1187988084 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f26e806ace0 0x7f26e806d190 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f26fc1602b0 tx=0x7f26f0009210 comp rx=0 tx=0).stop 2023-05-20T01:28:40.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:40.557+0000 7f2702220700 1 -- 172.21.15.16:0/1187988084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26fc0cf2e0 msgr2=0x7f26fc15efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:40.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:40.557+0000 7f2702220700 1 --2- 172.21.15.16:0/1187988084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26fc0cf2e0 0x7f26fc15efb0 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f26f4014040 tx=0x7f26f4004ad0 comp rx=0 tx=0).stop 2023-05-20T01:28:40.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:40.557+0000 7f2702220700 1 -- 172.21.15.16:0/1187988084 shutdown_connections 2023-05-20T01:28:40.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:40.557+0000 7f2702220700 1 --2- 172.21.15.16:0/1187988084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26fc0cf2e0 0x7f26fc15efb0 unknown :-1 s=CLOSED pgs=1676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:40.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:40.557+0000 7f2702220700 1 --2- 172.21.15.16:0/1187988084 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f26e806ace0 0x7f26e806d190 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:40.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:40.557+0000 7f2702220700 1 --2- 172.21.15.16:0/1187988084 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f26fc06f3e0 0x7f26fc154770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:40.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:40.557+0000 7f2702220700 1 --2- 172.21.15.16:0/1187988084 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f26fc0713c0 0x7f26fc0b5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:40.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:40.557+0000 7f2702220700 1 -- 172.21.15.16:0/1187988084 >> 172.21.15.16:0/1187988084 conn(0x7f26fc0b2920 msgr2=0x7f26fc0cf750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:40.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:40.557+0000 7f2702220700 1 -- 172.21.15.16:0/1187988084 shutdown_connections 2023-05-20T01:28:40.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:40.557+0000 7f2702220700 1 -- 172.21.15.16:0/1187988084 wait complete. 2023-05-20T01:28:40.564 INFO:teuthology.orchestra.run.smithi016.stderr:cephfs marked joinable; MDS may join as newly active. 2023-05-20T01:28:40.579 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-20T01:28:40.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:40.705+0000 7f9561acc700 1 Processor -- start 2023-05-20T01:28:40.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:40.705+0000 7f9561acc700 1 -- start start 2023-05-20T01:28:40.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:40.705+0000 7f9561acc700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f955c0713e0 0x7f955c0cf370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:40.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:40.705+0000 7f9561acc700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f955c072400 0x7f955c15dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:40.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:40.705+0000 7f9561acc700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f955c15d7f0 con 0x7f955c0717b0 2023-05-20T01:28:40.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:40.705+0000 7f9561acc700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f955c0d0080 con 0x7f955c072400 2023-05-20T01:28:40.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:40.705+0000 7f9561acc700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f955c0cf8b0 con 0x7f955c0713e0 2023-05-20T01:28:40.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:40.706+0000 7f955bfff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f955c0717b0 0x7f955c160820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51996/0 (socket says 172.21.15.16:51996) 2023-05-20T01:28:40.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:40.706+0000 7f955bfff700 1 -- 172.21.15.16:0/572942721 learned_addr learned my addr 172.21.15.16:0/572942721 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:28:40.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:40.706+0000 7f955affd700 1 --2- 172.21.15.16:0/572942721 >> v2:172.21.15.132:3300/0 conn(0x7f955c072400 0x7f955c15dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:40.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:40.706+0000 7f955b7fe700 1 --2- 172.21.15.16:0/572942721 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f955c0713e0 0x7f955c0cf370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:40.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:40.706+0000 7f955a7fc700 1 -- 172.21.15.16:0/572942721 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3714042500 0 0) 0x7f955c15d7f0 con 0x7f955c0717b0 2023-05-20T01:28:40.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:40.706+0000 7f955a7fc700 1 -- 172.21.15.16:0/572942721 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95440035b0 con 0x7f955c0717b0 2023-05-20T01:28:40.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:40.706+0000 7f955b7fe700 1 -- 172.21.15.16:0/572942721 >> v2:172.21.15.132:3300/0 conn(0x7f955c072400 msgr2=0x7f955c15dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:40.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:40.706+0000 7f955b7fe700 1 --2- 172.21.15.16:0/572942721 >> v2:172.21.15.132:3300/0 conn(0x7f955c072400 0x7f955c15dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:40.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:40.706+0000 7f955b7fe700 1 -- 172.21.15.16:0/572942721 >> v1:172.21.15.16:6789/0 conn(0x7f955c0717b0 legacy=0x7f955c160820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:40.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:40.706+0000 7f955b7fe700 1 -- 172.21.15.16:0/572942721 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f955c15d200 con 0x7f955c0713e0 2023-05-20T01:28:40.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:40.706+0000 7f955affd700 1 --2- 172.21.15.16:0/572942721 >> v2:172.21.15.132:3300/0 conn(0x7f955c072400 0x7f955c15dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-05-20T01:28:40.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:40.706+0000 7f955b7fe700 1 --2- 172.21.15.16:0/572942721 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f955c0713e0 0x7f955c0cf370 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f954c009d90 tx=0x7f954c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1468d2c78a295ac1 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:40.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:40.706+0000 7f955a7fc700 1 -- 172.21.15.16:0/572942721 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f954c00e070 con 0x7f955c0713e0 2023-05-20T01:28:40.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:40.707+0000 7f955a7fc700 1 -- 172.21.15.16:0/572942721 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f954c00bb20 con 0x7f955c0713e0 2023-05-20T01:28:40.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:40.707+0000 7f955a7fc700 1 -- 172.21.15.16:0/572942721 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f954c012430 con 0x7f955c0713e0 2023-05-20T01:28:40.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:40.707+0000 7f9561acc700 1 -- 172.21.15.16:0/572942721 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f955c0713e0 msgr2=0x7f955c0cf370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:40.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:40.707+0000 7f9561acc700 1 --2- 172.21.15.16:0/572942721 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f955c0713e0 0x7f955c0cf370 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f954c009d90 tx=0x7f954c00b330 comp rx=0 tx=0).stop 2023-05-20T01:28:40.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:40.707+0000 7f9561acc700 1 -- 172.21.15.16:0/572942721 shutdown_connections 2023-05-20T01:28:40.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:40.707+0000 7f9561acc700 1 --2- 172.21.15.16:0/572942721 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f955c0713e0 0x7f955c0cf370 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:40.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:40.707+0000 7f9561acc700 1 --2- 172.21.15.16:0/572942721 >> v2:172.21.15.132:3300/0 conn(0x7f955c072400 0x7f955c15dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:40.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:40.707+0000 7f9561acc700 1 -- 172.21.15.16:0/572942721 >> 172.21.15.16:0/572942721 conn(0x7f955c06e250 msgr2=0x7f955c0ccc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:40.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:40.707+0000 7f9561acc700 1 -- 172.21.15.16:0/572942721 shutdown_connections 2023-05-20T01:28:40.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:40.707+0000 7f9561acc700 1 -- 172.21.15.16:0/572942721 wait complete. 2023-05-20T01:28:40.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:40.708+0000 7f9561acc700 1 Processor -- start 2023-05-20T01:28:40.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:40.708+0000 7f9561acc700 1 -- start start 2023-05-20T01:28:40.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:40.709+0000 7f9561acc700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f955c0713e0 0x7f955c0c5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:40.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:40.709+0000 7f9561acc700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f955c0717b0 0x7f955c159f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:40.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:40.709+0000 7f9561acc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f955c072400 0x7f955c159980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:40.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:40.709+0000 7f9561acc700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f955c0c8c70 con 0x7f955c072400 2023-05-20T01:28:40.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:40.709+0000 7f9561acc700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f955c0c8060 con 0x7f955c0713e0 2023-05-20T01:28:40.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:40.709+0000 7f9561acc700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f955c0c8340 con 0x7f955c0717b0 2023-05-20T01:28:40.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:40.709+0000 7f955bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f955c072400 0x7f955c159980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:40.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:40.709+0000 7f955bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f955c072400 0x7f955c159980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40474/0 (socket says 172.21.15.16:40474) 2023-05-20T01:28:40.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:40.709+0000 7f955bfff700 1 -- 172.21.15.16:0/1646845298 learned_addr learned my addr 172.21.15.16:0/1646845298 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:40.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:40.709+0000 7f955affd700 1 --2- 172.21.15.16:0/1646845298 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f955c0717b0 0x7f955c159f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:40.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:40.710+0000 7f955bfff700 1 -- 172.21.15.16:0/1646845298 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f955c0717b0 msgr2=0x7f955c159f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:40.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:40.710+0000 7f955bfff700 1 --2- 172.21.15.16:0/1646845298 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f955c0717b0 0x7f955c159f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:40.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:40.710+0000 7f955bfff700 1 -- 172.21.15.16:0/1646845298 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f955c0713e0 msgr2=0x7f955c0c5370 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-05-20T01:28:40.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:40.710+0000 7f955bfff700 1 --2- 172.21.15.16:0/1646845298 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f955c0713e0 0x7f955c0c5370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:40.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:40.710+0000 7f955bfff700 1 -- 172.21.15.16:0/1646845298 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95500094a0 con 0x7f955c072400 2023-05-20T01:28:40.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:40.710+0000 7f955affd700 1 --2- 172.21.15.16:0/1646845298 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f955c0717b0 0x7f955c159f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:28:40.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:40.710+0000 7f955bfff700 1 --2- 172.21.15.16:0/1646845298 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f955c072400 0x7f955c159980 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f955400c4d0 tx=0x7f955400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:40.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:40.710+0000 7f9543fff700 1 -- 172.21.15.16:0/1646845298 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9554005d70 con 0x7f955c072400 2023-05-20T01:28:40.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:40.710+0000 7f9543fff700 1 -- 172.21.15.16:0/1646845298 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95540070d0 con 0x7f955c072400 2023-05-20T01:28:40.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:40.710+0000 7f9561acc700 1 -- 172.21.15.16:0/1646845298 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f954c009a70 con 0x7f955c072400 2023-05-20T01:28:40.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:40.710+0000 7f9543fff700 1 -- 172.21.15.16:0/1646845298 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9554007440 con 0x7f955c072400 2023-05-20T01:28:40.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:40.710+0000 7f9561acc700 1 -- 172.21.15.16:0/1646845298 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f955c06b000 con 0x7f955c072400 2023-05-20T01:28:40.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:40.711+0000 7f9561acc700 1 -- 172.21.15.16:0/1646845298 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f955c15e1c0 con 0x7f955c072400 2023-05-20T01:28:40.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:40.712+0000 7f9543fff700 1 -- 172.21.15.16:0/1646845298 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f95540075a0 con 0x7f955c072400 2023-05-20T01:28:40.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:40.712+0000 7f9543fff700 1 --2- 172.21.15.16:0/1646845298 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f954806ace0 0x7f954806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:40.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:40.712+0000 7f9543fff700 1 -- 172.21.15.16:0/1646845298 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6468+0+0 (secure 0 0 0) 0x7f95540889a0 con 0x7f955c072400 2023-05-20T01:28:40.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:40.712+0000 7f955b7fe700 1 --2- 172.21.15.16:0/1646845298 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f954806ace0 0x7f954806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:40.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:40.712+0000 7f955b7fe700 1 --2- 172.21.15.16:0/1646845298 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f954806ace0 0x7f954806d190 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f955c0ca9e0 tx=0x7f954c011000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:40.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:40.716+0000 7f9543fff700 1 -- 172.21.15.16:0/1646845298 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f95540539f0 con 0x7f955c072400 2023-05-20T01:28:40.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:40.960+0000 7f9561acc700 1 -- 172.21.15.16:0/1646845298 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f955c0cfac0 con 0x7f955c072400 2023-05-20T01:28:40.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:40.960+0000 7f9543fff700 1 -- 172.21.15.16:0/1646845298 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 102 v102) v1 ==== 95+0+4684 (secure 0 0 0) 0x7f9554057010 con 0x7f955c072400 2023-05-20T01:28:40.965 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:28:40.965 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":102,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7043,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/1859755557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1859755557},{"type":"v1","addr":"172.21.15.16:6836","nonce":1859755557}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94},{"gid":7046,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/2654783598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":2654783598},{"type":"v1","addr":"172.21.15.132:6834","nonce":2654783598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94},{"gid":7049,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2042939723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":2042939723},{"type":"v1","addr":"172.21.15.132:6835","nonce":2042939723}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":102,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-20T01:28:27.093957+0000","modified":"2023-05-20T01:28:40.555927+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":137,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8937},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8937":{"gid":8937,"name":"a","rank":0,"incarnation":102,"state":"up:replay","state_seq":1,"addr":"172.21.15.16:6837/2869411815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":2869411815},{"type":"v1","addr":"172.21.15.16:6837","nonce":2869411815}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-05-20T01:28:40.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:40.962+0000 7f9561acc700 1 -- 172.21.15.16:0/1646845298 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f954806ace0 msgr2=0x7f954806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:40.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:40.962+0000 7f9561acc700 1 --2- 172.21.15.16:0/1646845298 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f954806ace0 0x7f954806d190 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f955c0ca9e0 tx=0x7f954c011000 comp rx=0 tx=0).stop 2023-05-20T01:28:40.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:40.962+0000 7f9561acc700 1 -- 172.21.15.16:0/1646845298 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f955c072400 msgr2=0x7f955c159980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:40.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:40.962+0000 7f9561acc700 1 --2- 172.21.15.16:0/1646845298 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f955c072400 0x7f955c159980 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f955400c4d0 tx=0x7f955400c890 comp rx=0 tx=0).stop 2023-05-20T01:28:40.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:40.962+0000 7f9561acc700 1 -- 172.21.15.16:0/1646845298 shutdown_connections 2023-05-20T01:28:40.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:40.962+0000 7f9561acc700 1 --2- 172.21.15.16:0/1646845298 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f955c072400 0x7f955c159980 unknown :-1 s=CLOSED pgs=1678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:40.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:40.962+0000 7f9561acc700 1 --2- 172.21.15.16:0/1646845298 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f955c0713e0 0x7f955c0c5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:40.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:40.962+0000 7f9561acc700 1 --2- 172.21.15.16:0/1646845298 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f954806ace0 0x7f954806d190 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:40.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:40.962+0000 7f9561acc700 1 --2- 172.21.15.16:0/1646845298 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f955c0717b0 0x7f955c159f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:40.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:40.962+0000 7f9561acc700 1 -- 172.21.15.16:0/1646845298 >> 172.21.15.16:0/1646845298 conn(0x7f955c06e250 msgr2=0x7f955c0caf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:40.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:40.962+0000 7f9561acc700 1 -- 172.21.15.16:0/1646845298 shutdown_connections 2023-05-20T01:28:40.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:40.962+0000 7f9561acc700 1 -- 172.21.15.16:0/1646845298 wait complete. 2023-05-20T01:28:40.970 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 102 2023-05-20T01:28:41.382 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 102, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-05-20T01:28:27.093957+0000', 'modified': '2023-05-20T01:28:40.555927+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 137, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8937}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8937': {'gid': 8937, 'name': 'a', 'rank': 0, 'incarnation': 102, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.16:6837/2869411815', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6835', 'nonce': 2869411815}, {'type': 'v1', 'addr': '172.21.15.16:6837', 'nonce': 2869411815}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [19], 'metadata_pool': 18, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-05-20T01:28:41.382 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8937:up:replay 2023-05-20T01:28:42.383 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-20T01:28:42.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:42.518+0000 7fbb5ee7d700 1 Processor -- start 2023-05-20T01:28:42.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:42.518+0000 7fbb5ee7d700 1 -- start start 2023-05-20T01:28:42.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:42.519+0000 7fbb5ee7d700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fbb58158f80 0x7fbb580cc2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:42.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:42.519+0000 7fbb5ee7d700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb580af340 con 0x7fbb580ae9a0 2023-05-20T01:28:42.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:42.519+0000 7fbb5ee7d700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb5815d520 con 0x7fbb580ca870 2023-05-20T01:28:42.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:42.519+0000 7fbb5ee7d700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb5815da50 con 0x7fbb58158f80 2023-05-20T01:28:42.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:42.519+0000 7fbb5d41a700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fbb580ae9a0 0x7fbb58071240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:52010/0 (socket says 172.21.15.16:52010) 2023-05-20T01:28:42.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:42.519+0000 7fbb5d41a700 1 -- 172.21.15.16:0/3666691006 learned_addr learned my addr 172.21.15.16:0/3666691006 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:28:42.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:42.519+0000 7fbb5cc19700 1 --2- 172.21.15.16:0/3666691006 >> v2:172.21.15.132:3300/0 conn(0x7fbb58158f80 0x7fbb580cc2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:42.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:42.520+0000 7fbb577fe700 1 -- 172.21.15.16:0/3666691006 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3871633325 0 0) 0x7fbb580af340 con 0x7fbb580ae9a0 2023-05-20T01:28:42.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:42.520+0000 7fbb577fe700 1 -- 172.21.15.16:0/3666691006 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb480035b0 con 0x7fbb580ae9a0 2023-05-20T01:28:42.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:42.520+0000 7fbb577fe700 1 -- 172.21.15.16:0/3666691006 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2419593606 0 0) 0x7fbb480035b0 con 0x7fbb580ae9a0 2023-05-20T01:28:42.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:42.520+0000 7fbb577fe700 1 -- 172.21.15.16:0/3666691006 >> v2:172.21.15.132:3300/0 conn(0x7fbb58158f80 msgr2=0x7fbb580cc2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:42.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:42.520+0000 7fbb577fe700 1 --2- 172.21.15.16:0/3666691006 >> v2:172.21.15.132:3300/0 conn(0x7fbb58158f80 0x7fbb580cc2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:42.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:42.520+0000 7fbb577fe700 1 -- 172.21.15.16:0/3666691006 >> v1:172.21.15.132:6789/0 conn(0x7fbb580ca870 legacy=0x7fbb58070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:42.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:42.520+0000 7fbb577fe700 1 -- 172.21.15.16:0/3666691006 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb58071950 con 0x7fbb580ae9a0 2023-05-20T01:28:42.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:42.520+0000 7fbb577fe700 1 -- 172.21.15.16:0/3666691006 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7fbb500019f0 con 0x7fbb580ae9a0 2023-05-20T01:28:42.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:42.520+0000 7fbb5cc19700 1 --2- 172.21.15.16:0/3666691006 >> v2:172.21.15.132:3300/0 conn(0x7fbb58158f80 0x7fbb580cc2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-05-20T01:28:42.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:42.520+0000 7fbb577fe700 1 -- 172.21.15.16:0/3666691006 >> v1:172.21.15.16:6789/0 conn(0x7fbb580ae9a0 legacy=0x7fbb58071240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:42.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:42.520+0000 7fbb577fe700 1 --2- 172.21.15.16:0/3666691006 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb48003a00 0x7fbb48003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:42.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:42.520+0000 7fbb577fe700 1 --2- 172.21.15.16:0/3666691006 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbb48004850 0x7fbb48004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:42.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:42.520+0000 7fbb577fe700 1 --2- 172.21.15.16:0/3666691006 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbb48005200 0x7fbb4800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:42.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:42.520+0000 7fbb577fe700 1 -- 172.21.15.16:0/3666691006 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb5815da50 con 0x7fbb48003a00 2023-05-20T01:28:42.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:42.520+0000 7fbb577fe700 1 -- 172.21.15.16:0/3666691006 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb4800c110 con 0x7fbb48005200 2023-05-20T01:28:42.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:42.520+0000 7fbb577fe700 1 -- 172.21.15.16:0/3666691006 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb4800c2d0 con 0x7fbb48004850 2023-05-20T01:28:42.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:42.520+0000 7fbb5cc19700 1 --2- 172.21.15.16:0/3666691006 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb48003a00 0x7fbb48003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:42.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:42.520+0000 7fbb5d41a700 1 --2- 172.21.15.16:0/3666691006 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbb48005200 0x7fbb4800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:42.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:42.520+0000 7fbb57fff700 1 --2- 172.21.15.16:0/3666691006 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbb48004850 0x7fbb48004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:42.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:42.520+0000 7fbb5cc19700 1 -- 172.21.15.16:0/3666691006 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbb48004850 msgr2=0x7fbb48004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:42.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:42.520+0000 7fbb5cc19700 1 --2- 172.21.15.16:0/3666691006 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbb48004850 0x7fbb48004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:42.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:42.520+0000 7fbb5cc19700 1 -- 172.21.15.16:0/3666691006 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbb48005200 msgr2=0x7fbb4800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:42.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:42.520+0000 7fbb5cc19700 1 --2- 172.21.15.16:0/3666691006 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbb48005200 0x7fbb4800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:42.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:42.520+0000 7fbb5cc19700 1 -- 172.21.15.16:0/3666691006 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb480035b0 con 0x7fbb48003a00 2023-05-20T01:28:42.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:42.520+0000 7fbb5d41a700 1 -- 172.21.15.16:0/3666691006 reap_dead start 2023-05-20T01:28:42.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:42.520+0000 7fbb57fff700 1 --2- 172.21.15.16:0/3666691006 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbb48004850 0x7fbb48004cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:28:42.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:42.521+0000 7fbb5cc19700 1 --2- 172.21.15.16:0/3666691006 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb48003a00 0x7fbb48003e10 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c0098c0 tx=0x7fbb48005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=422681cf538fbd49 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:42.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:42.521+0000 7fbb577fe700 1 -- 172.21.15.16:0/3666691006 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb4c009e50 con 0x7fbb48003a00 2023-05-20T01:28:42.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:42.521+0000 7fbb577fe700 1 -- 172.21.15.16:0/3666691006 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb4c00e9a0 con 0x7fbb48003a00 2023-05-20T01:28:42.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:42.521+0000 7fbb577fe700 1 -- 172.21.15.16:0/3666691006 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb4c00ed30 con 0x7fbb48003a00 2023-05-20T01:28:42.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:42.521+0000 7fbb5ee7d700 1 -- 172.21.15.16:0/3666691006 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb48003a00 msgr2=0x7fbb48003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:42.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:42.521+0000 7fbb5ee7d700 1 --2- 172.21.15.16:0/3666691006 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb48003a00 0x7fbb48003e10 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c0098c0 tx=0x7fbb48005880 comp rx=0 tx=0).stop 2023-05-20T01:28:42.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:42.521+0000 7fbb5ee7d700 1 -- 172.21.15.16:0/3666691006 shutdown_connections 2023-05-20T01:28:42.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:42.521+0000 7fbb5ee7d700 1 --2- 172.21.15.16:0/3666691006 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb48003a00 0x7fbb48003e10 unknown :-1 s=CLOSED pgs=1680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:42.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:42.521+0000 7fbb5ee7d700 1 -- 172.21.15.16:0/3666691006 >> 172.21.15.16:0/3666691006 conn(0x7fbb580b1590 msgr2=0x7fbb580b1960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:42.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:42.521+0000 7fbb5ee7d700 1 -- 172.21.15.16:0/3666691006 shutdown_connections 2023-05-20T01:28:42.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:42.521+0000 7fbb5ee7d700 1 -- 172.21.15.16:0/3666691006 wait complete. 2023-05-20T01:28:42.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:42.523+0000 7fbb5ee7d700 1 Processor -- start 2023-05-20T01:28:42.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:42.523+0000 7fbb5ee7d700 1 -- start start 2023-05-20T01:28:42.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:42.524+0000 7fbb5ee7d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb48003a00 0x7fbb581412e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:42.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:42.524+0000 7fbb5ee7d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbb58147cf0 0x7fbb58143b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:42.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:42.524+0000 7fbb5ee7d700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbb581440d0 0x7fbb580cc210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:42.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:42.524+0000 7fbb5ee7d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb58144b30 con 0x7fbb48003a00 2023-05-20T01:28:42.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:42.524+0000 7fbb5ee7d700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb58142010 con 0x7fbb58147cf0 2023-05-20T01:28:42.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:42.524+0000 7fbb5ee7d700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb58141820 con 0x7fbb581440d0 2023-05-20T01:28:42.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:42.524+0000 7fbb5cc19700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb48003a00 0x7fbb581412e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:42.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:42.524+0000 7fbb5cc19700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb48003a00 0x7fbb581412e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40490/0 (socket says 172.21.15.16:40490) 2023-05-20T01:28:42.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:42.524+0000 7fbb5cc19700 1 -- 172.21.15.16:0/1244678628 learned_addr learned my addr 172.21.15.16:0/1244678628 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:42.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:42.525+0000 7fbb57fff700 1 --2- 172.21.15.16:0/1244678628 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbb58147cf0 0x7fbb58143b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:42.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:42.525+0000 7fbb5d41a700 1 --2- 172.21.15.16:0/1244678628 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbb581440d0 0x7fbb580cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:42.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:42.525+0000 7fbb5cc19700 1 -- 172.21.15.16:0/1244678628 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbb581440d0 msgr2=0x7fbb580cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:42.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:42.525+0000 7fbb5cc19700 1 --2- 172.21.15.16:0/1244678628 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbb581440d0 0x7fbb580cc210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:42.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:42.525+0000 7fbb5cc19700 1 -- 172.21.15.16:0/1244678628 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbb58147cf0 msgr2=0x7fbb58143b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:42.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:42.525+0000 7fbb5cc19700 1 --2- 172.21.15.16:0/1244678628 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbb58147cf0 0x7fbb58143b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:42.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:42.525+0000 7fbb5cc19700 1 -- 172.21.15.16:0/1244678628 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb58159350 con 0x7fbb48003a00 2023-05-20T01:28:42.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:42.525+0000 7fbb5d41a700 1 --2- 172.21.15.16:0/1244678628 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbb581440d0 0x7fbb580cc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:28:42.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:42.525+0000 7fbb57fff700 1 --2- 172.21.15.16:0/1244678628 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbb58147cf0 0x7fbb58143b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:28:42.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:42.525+0000 7fbb5cc19700 1 --2- 172.21.15.16:0/1244678628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb48003a00 0x7fbb581412e0 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c00e790 tx=0x7fbb4c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:42.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:42.525+0000 7fbb557fa700 1 -- 172.21.15.16:0/1244678628 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb4c011db0 con 0x7fbb48003a00 2023-05-20T01:28:42.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:42.525+0000 7fbb557fa700 1 -- 172.21.15.16:0/1244678628 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb4c009e50 con 0x7fbb48003a00 2023-05-20T01:28:42.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:42.525+0000 7fbb5ee7d700 1 -- 172.21.15.16:0/1244678628 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb580cd8e0 con 0x7fbb48003a00 2023-05-20T01:28:42.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:42.525+0000 7fbb557fa700 1 -- 172.21.15.16:0/1244678628 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb4c019680 con 0x7fbb48003a00 2023-05-20T01:28:42.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:42.525+0000 7fbb5ee7d700 1 -- 172.21.15.16:0/1244678628 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb5815d770 con 0x7fbb48003a00 2023-05-20T01:28:42.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:42.526+0000 7fbb5ee7d700 1 -- 172.21.15.16:0/1244678628 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb581422f0 con 0x7fbb48003a00 2023-05-20T01:28:42.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:42.527+0000 7fbb557fa700 1 -- 172.21.15.16:0/1244678628 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fbb4c020040 con 0x7fbb48003a00 2023-05-20T01:28:42.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:42.527+0000 7fbb557fa700 1 --2- 172.21.15.16:0/1244678628 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fbb3c06ace0 0x7fbb3c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:42.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:42.527+0000 7fbb557fa700 1 -- 172.21.15.16:0/1244678628 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6468+0+0 (secure 0 0 0) 0x7fbb4c08cac0 con 0x7fbb48003a00 2023-05-20T01:28:42.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:42.527+0000 7fbb57fff700 1 --2- 172.21.15.16:0/1244678628 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fbb3c06ace0 0x7fbb3c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:42.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:42.527+0000 7fbb57fff700 1 --2- 172.21.15.16:0/1244678628 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fbb3c06ace0 0x7fbb3c06d190 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fbb44000c00 tx=0x7fbb44009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:42.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:42.531+0000 7fbb557fa700 1 -- 172.21.15.16:0/1244678628 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fbb4c057b80 con 0x7fbb48003a00 2023-05-20T01:28:42.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:42.775+0000 7fbb5ee7d700 1 -- 172.21.15.16:0/1244678628 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbb5804e8c0 con 0x7fbb48003a00 2023-05-20T01:28:42.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:42.775+0000 7fbb557fa700 1 -- 172.21.15.16:0/1244678628 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 104 v104) v1 ==== 95+0+4684 (secure 0 0 0) 0x7fbb4c026090 con 0x7fbb48003a00 2023-05-20T01:28:42.780 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:28:42.780 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":104,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7043,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/1859755557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1859755557},{"type":"v1","addr":"172.21.15.16:6836","nonce":1859755557}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94},{"gid":7046,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/2654783598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":2654783598},{"type":"v1","addr":"172.21.15.132:6834","nonce":2654783598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94},{"gid":7049,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2042939723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":2042939723},{"type":"v1","addr":"172.21.15.132:6835","nonce":2042939723}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":104,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-20T01:28:27.093957+0000","modified":"2023-05-20T01:28:42.571292+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":137,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8937},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8937":{"gid":8937,"name":"a","rank":0,"incarnation":102,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.16:6837/2869411815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":2869411815},{"type":"v1","addr":"172.21.15.16:6837","nonce":2869411815}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-05-20T01:28:42.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:42.778+0000 7fbb5ee7d700 1 -- 172.21.15.16:0/1244678628 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fbb3c06ace0 msgr2=0x7fbb3c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:42.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:42.778+0000 7fbb5ee7d700 1 --2- 172.21.15.16:0/1244678628 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fbb3c06ace0 0x7fbb3c06d190 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fbb44000c00 tx=0x7fbb44009040 comp rx=0 tx=0).stop 2023-05-20T01:28:42.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:42.778+0000 7fbb5ee7d700 1 -- 172.21.15.16:0/1244678628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb48003a00 msgr2=0x7fbb581412e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:42.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:42.778+0000 7fbb5ee7d700 1 --2- 172.21.15.16:0/1244678628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb48003a00 0x7fbb581412e0 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c00e790 tx=0x7fbb4c0047c0 comp rx=0 tx=0).stop 2023-05-20T01:28:42.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:42.778+0000 7fbb5ee7d700 1 -- 172.21.15.16:0/1244678628 shutdown_connections 2023-05-20T01:28:42.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:42.778+0000 7fbb5ee7d700 1 --2- 172.21.15.16:0/1244678628 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fbb3c06ace0 0x7fbb3c06d190 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:42.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:42.778+0000 7fbb5ee7d700 1 --2- 172.21.15.16:0/1244678628 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbb581440d0 0x7fbb580cc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:42.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:42.778+0000 7fbb5ee7d700 1 --2- 172.21.15.16:0/1244678628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb48003a00 0x7fbb581412e0 unknown :-1 s=CLOSED pgs=1681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:42.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:42.778+0000 7fbb5ee7d700 1 --2- 172.21.15.16:0/1244678628 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbb58147cf0 0x7fbb58143b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:42.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:42.778+0000 7fbb5ee7d700 1 -- 172.21.15.16:0/1244678628 >> 172.21.15.16:0/1244678628 conn(0x7fbb580b1590 msgr2=0x7fbb58154690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:42.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:42.778+0000 7fbb5ee7d700 1 -- 172.21.15.16:0/1244678628 shutdown_connections 2023-05-20T01:28:42.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:42.778+0000 7fbb5ee7d700 1 -- 172.21.15.16:0/1244678628 wait complete. 2023-05-20T01:28:42.785 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 104 2023-05-20T01:28:43.196 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 104, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-05-20T01:28:27.093957+0000', 'modified': '2023-05-20T01:28:42.571292+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 137, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8937}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8937': {'gid': 8937, 'name': 'a', 'rank': 0, 'incarnation': 102, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.16:6837/2869411815', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6835', 'nonce': 2869411815}, {'type': 'v1', 'addr': '172.21.15.16:6837', 'nonce': 2869411815}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [19], 'metadata_pool': 18, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-05-20T01:28:43.197 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8937:up:rejoin 2023-05-20T01:28:44.198 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-20T01:28:44.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:44.332+0000 7f2817a4c700 1 Processor -- start 2023-05-20T01:28:44.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:44.332+0000 7f2817a4c700 1 -- start start 2023-05-20T01:28:44.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:44.333+0000 7f2817a4c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f28100b2860 0x7f28100cdae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:44.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:44.333+0000 7f2817a4c700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28100cf770 con 0x7f28100b4f10 2023-05-20T01:28:44.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:44.333+0000 7f2817a4c700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28100cfcf0 con 0x7f28101613f0 2023-05-20T01:28:44.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:44.333+0000 7f2817a4c700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f281015e1e0 con 0x7f28100b2860 2023-05-20T01:28:44.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:44.333+0000 7f2815fe9700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f28100b4f10 0x7f281006b420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:52034/0 (socket says 172.21.15.16:52034) 2023-05-20T01:28:44.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:44.333+0000 7f2815fe9700 1 -- 172.21.15.16:0/433858028 learned_addr learned my addr 172.21.15.16:0/433858028 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:28:44.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:44.333+0000 7f28157e8700 1 --2- 172.21.15.16:0/433858028 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f28100b2860 0x7f28100cdae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:44.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:44.333+0000 7f2803fff700 1 -- 172.21.15.16:0/433858028 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2028179051 0 0) 0x7f28100cf770 con 0x7f28100b4f10 2023-05-20T01:28:44.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:44.334+0000 7f2803fff700 1 -- 172.21.15.16:0/433858028 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27fc0035b0 con 0x7f28100b4f10 2023-05-20T01:28:44.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:44.334+0000 7f2803fff700 1 -- 172.21.15.16:0/433858028 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1965002069 0 0) 0x7f27fc0035b0 con 0x7f28100b4f10 2023-05-20T01:28:44.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:44.334+0000 7f2803fff700 1 -- 172.21.15.16:0/433858028 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f28100b2860 msgr2=0x7f28100cdae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:44.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:44.334+0000 7f2803fff700 1 --2- 172.21.15.16:0/433858028 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f28100b2860 0x7f28100cdae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:44.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:44.334+0000 7f2803fff700 1 -- 172.21.15.16:0/433858028 >> v1:172.21.15.132:6789/0 conn(0x7f28101613f0 legacy=0x7f281015dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:44.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:44.334+0000 7f2803fff700 1 -- 172.21.15.16:0/433858028 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28100cfa10 con 0x7f28100b4f10 2023-05-20T01:28:44.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:44.334+0000 7f2803fff700 1 -- 172.21.15.16:0/433858028 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7f28080019f0 con 0x7f28100b4f10 2023-05-20T01:28:44.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:44.334+0000 7f2803fff700 1 -- 172.21.15.16:0/433858028 >> v1:172.21.15.16:6789/0 conn(0x7f28100b4f10 legacy=0x7f281006b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:44.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:44.334+0000 7f2803fff700 1 --2- 172.21.15.16:0/433858028 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f28100b2860 0x7f27fc003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:44.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:44.334+0000 7f2803fff700 1 --2- 172.21.15.16:0/433858028 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f27fc003f70 0x7f27fc004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:44.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:44.334+0000 7f2803fff700 1 --2- 172.21.15.16:0/433858028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27fc004d90 0x7f27fc005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:44.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:44.334+0000 7f2803fff700 1 -- 172.21.15.16:0/433858028 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f281015e1e0 con 0x7f27fc004d90 2023-05-20T01:28:44.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:44.334+0000 7f2803fff700 1 -- 172.21.15.16:0/433858028 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27fc00ba40 con 0x7f27fc003f70 2023-05-20T01:28:44.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:44.334+0000 7f2803fff700 1 -- 172.21.15.16:0/433858028 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27fc00bc00 con 0x7f28100b2860 2023-05-20T01:28:44.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:44.334+0000 7f2815fe9700 1 --2- 172.21.15.16:0/433858028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27fc004d90 0x7f27fc005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:44.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:44.334+0000 7f28157e8700 1 --2- 172.21.15.16:0/433858028 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f28100b2860 0x7f27fc003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:44.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:44.334+0000 7f2814fe7700 1 --2- 172.21.15.16:0/433858028 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f27fc003f70 0x7f27fc004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:44.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:44.334+0000 7f2815fe9700 1 -- 172.21.15.16:0/433858028 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f28100b2860 msgr2=0x7f27fc003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:44.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:44.334+0000 7f2815fe9700 1 --2- 172.21.15.16:0/433858028 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f28100b2860 0x7f27fc003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:44.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:44.334+0000 7f2815fe9700 1 -- 172.21.15.16:0/433858028 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f27fc003f70 msgr2=0x7f27fc004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:44.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:44.334+0000 7f2815fe9700 1 --2- 172.21.15.16:0/433858028 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f27fc003f70 0x7f27fc004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:44.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:44.334+0000 7f2815fe9700 1 -- 172.21.15.16:0/433858028 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27fc0035b0 con 0x7f27fc004d90 2023-05-20T01:28:44.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:44.334+0000 7f2815fe9700 1 --2- 172.21.15.16:0/433858028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27fc004d90 0x7f27fc005200 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f280800a830 tx=0x7f27fc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f77e7fc34610f7c server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:44.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:44.335+0000 7f2803fff700 1 -- 172.21.15.16:0/433858028 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2808012600 con 0x7f27fc004d90 2023-05-20T01:28:44.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:44.335+0000 7f2803fff700 1 -- 172.21.15.16:0/433858028 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2808012780 con 0x7f27fc004d90 2023-05-20T01:28:44.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:44.335+0000 7f2803fff700 1 -- 172.21.15.16:0/433858028 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2808012b30 con 0x7f27fc004d90 2023-05-20T01:28:44.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:44.335+0000 7f2817a4c700 1 -- 172.21.15.16:0/433858028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27fc004d90 msgr2=0x7f27fc005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:44.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:44.335+0000 7f2817a4c700 1 --2- 172.21.15.16:0/433858028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27fc004d90 0x7f27fc005200 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f280800a830 tx=0x7f27fc005880 comp rx=0 tx=0).stop 2023-05-20T01:28:44.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:44.335+0000 7f2815fe9700 1 -- 172.21.15.16:0/433858028 reap_dead start 2023-05-20T01:28:44.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:44.335+0000 7f2817a4c700 1 -- 172.21.15.16:0/433858028 shutdown_connections 2023-05-20T01:28:44.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:44.335+0000 7f2817a4c700 1 -- 172.21.15.16:0/433858028 >> 172.21.15.16:0/433858028 conn(0x7f28100727e0 msgr2=0x7f28100cff10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:44.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:44.335+0000 7f2817a4c700 1 -- 172.21.15.16:0/433858028 shutdown_connections 2023-05-20T01:28:44.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:44.335+0000 7f2817a4c700 1 -- 172.21.15.16:0/433858028 wait complete. 2023-05-20T01:28:44.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:44.337+0000 7f2817a4c700 1 Processor -- start 2023-05-20T01:28:44.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:44.337+0000 7f2817a4c700 1 -- start start 2023-05-20T01:28:44.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:44.337+0000 7f2817a4c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2810146eb0 0x7f28100cbf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:44.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:44.337+0000 7f2817a4c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2810147280 0x7f2810159930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:44.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:44.337+0000 7f2817a4c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f28101476f0 0x7f2810159e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:44.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:44.337+0000 7f2817a4c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28100cd3d0 con 0x7f2810147280 2023-05-20T01:28:44.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:44.337+0000 7f2817a4c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28100cc4b0 con 0x7f2810146eb0 2023-05-20T01:28:44.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:44.337+0000 7f2817a4c700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28100cc750 con 0x7f28101476f0 2023-05-20T01:28:44.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:44.338+0000 7f2814fe7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2810147280 0x7f2810159930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:44.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:44.338+0000 7f2814fe7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2810147280 0x7f2810159930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40516/0 (socket says 172.21.15.16:40516) 2023-05-20T01:28:44.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:44.338+0000 7f2814fe7700 1 -- 172.21.15.16:0/3173526707 learned_addr learned my addr 172.21.15.16:0/3173526707 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:44.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:44.338+0000 7f28157e8700 1 --2- 172.21.15.16:0/3173526707 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2810146eb0 0x7f28100cbf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:44.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:44.338+0000 7f2815fe9700 1 --2- 172.21.15.16:0/3173526707 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f28101476f0 0x7f2810159e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:44.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:44.338+0000 7f2814fe7700 1 -- 172.21.15.16:0/3173526707 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f28101476f0 msgr2=0x7f2810159e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:44.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:44.338+0000 7f2814fe7700 1 --2- 172.21.15.16:0/3173526707 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f28101476f0 0x7f2810159e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:44.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:44.338+0000 7f2814fe7700 1 -- 172.21.15.16:0/3173526707 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2810146eb0 msgr2=0x7f28100cbf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:44.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:44.338+0000 7f2814fe7700 1 --2- 172.21.15.16:0/3173526707 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2810146eb0 0x7f28100cbf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:44.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:44.338+0000 7f2814fe7700 1 -- 172.21.15.16:0/3173526707 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f281015d5b0 con 0x7f2810147280 2023-05-20T01:28:44.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:44.338+0000 7f28157e8700 1 --2- 172.21.15.16:0/3173526707 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2810146eb0 0x7f28100cbf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:28:44.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:44.338+0000 7f2815fe9700 1 --2- 172.21.15.16:0/3173526707 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f28101476f0 0x7f2810159e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:28:44.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:44.338+0000 7f2814fe7700 1 --2- 172.21.15.16:0/3173526707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2810147280 0x7f2810159930 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f280c00f830 tx=0x7f280c00fbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:44.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:44.338+0000 7f2801ffb700 1 -- 172.21.15.16:0/3173526707 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f280c0052e0 con 0x7f2810147280 2023-05-20T01:28:44.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:44.338+0000 7f2801ffb700 1 -- 172.21.15.16:0/3173526707 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f280c005480 con 0x7f2810147280 2023-05-20T01:28:44.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:44.338+0000 7f2817a4c700 1 -- 172.21.15.16:0/3173526707 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f281015d2d0 con 0x7f2810147280 2023-05-20T01:28:44.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:44.338+0000 7f2801ffb700 1 -- 172.21.15.16:0/3173526707 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f280c0038c0 con 0x7f2810147280 2023-05-20T01:28:44.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:44.338+0000 7f2817a4c700 1 -- 172.21.15.16:0/3173526707 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28100b2c30 con 0x7f2810147280 2023-05-20T01:28:44.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:44.339+0000 7f2817a4c700 1 -- 172.21.15.16:0/3173526707 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28100b62c0 con 0x7f2810147280 2023-05-20T01:28:44.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:44.340+0000 7f2801ffb700 1 -- 172.21.15.16:0/3173526707 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f280c003a20 con 0x7f2810147280 2023-05-20T01:28:44.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:44.340+0000 7f2801ffb700 1 --2- 172.21.15.16:0/3173526707 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f27f406ace0 0x7f27f406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:44.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:44.340+0000 7f2801ffb700 1 -- 172.21.15.16:0/3173526707 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6468+0+0 (secure 0 0 0) 0x7f280c089520 con 0x7f2810147280 2023-05-20T01:28:44.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:44.340+0000 7f28157e8700 1 --2- 172.21.15.16:0/3173526707 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f27f406ace0 0x7f27f406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:44.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:44.341+0000 7f28157e8700 1 --2- 172.21.15.16:0/3173526707 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f27f406ace0 0x7f27f406d190 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f2804009c60 tx=0x7f2804008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:44.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:44.345+0000 7f2801ffb700 1 -- 172.21.15.16:0/3173526707 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f28100b62c0 con 0x7f2810147280 2023-05-20T01:28:44.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:44.590+0000 7f2817a4c700 1 -- 172.21.15.16:0/3173526707 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f28100b62c0 con 0x7f2810147280 2023-05-20T01:28:44.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:44.591+0000 7f2801ffb700 1 -- 172.21.15.16:0/3173526707 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) v1 ==== 95+0+4684 (secure 0 0 0) 0x7f28100b62c0 con 0x7f2810147280 2023-05-20T01:28:44.596 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:28:44.597 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":105,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7043,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/1859755557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1859755557},{"type":"v1","addr":"172.21.15.16:6836","nonce":1859755557}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94},{"gid":7046,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/2654783598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":2654783598},{"type":"v1","addr":"172.21.15.132:6834","nonce":2654783598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94},{"gid":7049,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2042939723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":2042939723},{"type":"v1","addr":"172.21.15.132:6835","nonce":2042939723}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-20T01:28:27.093957+0000","modified":"2023-05-20T01:28:43.576200+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":137,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8937},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8937":{"gid":8937,"name":"a","rank":0,"incarnation":102,"state":"up:active","state_seq":4,"addr":"172.21.15.16:6837/2869411815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":2869411815},{"type":"v1","addr":"172.21.15.16:6837","nonce":2869411815}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-05-20T01:28:44.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:44.593+0000 7f2817a4c700 1 -- 172.21.15.16:0/3173526707 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f27f406ace0 msgr2=0x7f27f406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:44.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:44.593+0000 7f2817a4c700 1 --2- 172.21.15.16:0/3173526707 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f27f406ace0 0x7f27f406d190 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f2804009c60 tx=0x7f2804008040 comp rx=0 tx=0).stop 2023-05-20T01:28:44.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:44.593+0000 7f2817a4c700 1 -- 172.21.15.16:0/3173526707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2810147280 msgr2=0x7f2810159930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:44.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:44.593+0000 7f2817a4c700 1 --2- 172.21.15.16:0/3173526707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2810147280 0x7f2810159930 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f280c00f830 tx=0x7f280c00fbf0 comp rx=0 tx=0).stop 2023-05-20T01:28:44.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:44.593+0000 7f2817a4c700 1 -- 172.21.15.16:0/3173526707 shutdown_connections 2023-05-20T01:28:44.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:44.593+0000 7f2817a4c700 1 --2- 172.21.15.16:0/3173526707 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f27f406ace0 0x7f27f406d190 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:44.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:44.593+0000 7f2817a4c700 1 --2- 172.21.15.16:0/3173526707 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f28101476f0 0x7f2810159e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:44.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:44.593+0000 7f2817a4c700 1 --2- 172.21.15.16:0/3173526707 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2810146eb0 0x7f28100cbf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:44.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:44.593+0000 7f2817a4c700 1 --2- 172.21.15.16:0/3173526707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2810147280 0x7f2810159930 unknown :-1 s=CLOSED pgs=1684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:44.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:44.593+0000 7f2817a4c700 1 -- 172.21.15.16:0/3173526707 >> 172.21.15.16:0/3173526707 conn(0x7f28100b3e00 msgr2=0x7f2810073990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:44.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:44.593+0000 7f2817a4c700 1 -- 172.21.15.16:0/3173526707 shutdown_connections 2023-05-20T01:28:44.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:44.593+0000 7f2817a4c700 1 -- 172.21.15.16:0/3173526707 wait complete. 2023-05-20T01:28:44.602 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 105 2023-05-20T01:28:45.012 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 105, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-05-20T01:28:27.093957+0000', 'modified': '2023-05-20T01:28:43.576200+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 137, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8937}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8937': {'gid': 8937, 'name': 'a', 'rank': 0, 'incarnation': 102, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.16:6837/2869411815', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6835', 'nonce': 2869411815}, {'type': 'v1', 'addr': '172.21.15.16:6837', 'nonce': 2869411815}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [19], 'metadata_pool': 18, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-05-20T01:28:45.013 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-05-20T01:28:45.014 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-05-20T01:28:45.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.139+0000 7f78cbf9b700 1 Processor -- start 2023-05-20T01:28:45.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.139+0000 7f78cbf9b700 1 -- start start 2023-05-20T01:28:45.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.140+0000 7f78cbf9b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f78c40713e0 0x7f78c40cf370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:45.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.140+0000 7f78cbf9b700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f78c4072400 0x7f78c415dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:45.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.140+0000 7f78cbf9b700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f78c40717b0 0x7f78c4160820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:45.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.140+0000 7f78cbf9b700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78c40d0080 con 0x7f78c40713e0 2023-05-20T01:28:45.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.140+0000 7f78cbf9b700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78c40cf8b0 con 0x7f78c40717b0 2023-05-20T01:28:45.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.140+0000 7f78cbf9b700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78c415d7f0 con 0x7f78c4072400 2023-05-20T01:28:45.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.140+0000 7f78c9d37700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f78c40713e0 0x7f78c40cf370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:45.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.140+0000 7f78c9d37700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f78c40713e0 0x7f78c40cf370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40522/0 (socket says 172.21.15.16:40522) 2023-05-20T01:28:45.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.140+0000 7f78c9d37700 1 -- 172.21.15.16:0/4266822667 learned_addr learned my addr 172.21.15.16:0/4266822667 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:45.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.141+0000 7f78c9d37700 1 -- 172.21.15.16:0/4266822667 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f78c4072400 msgr2=0x7f78c415dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:45.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.141+0000 7f78c9d37700 1 --2- 172.21.15.16:0/4266822667 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f78c4072400 0x7f78c415dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:45.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.141+0000 7f78ca538700 1 --2- 172.21.15.16:0/4266822667 >> v2:172.21.15.132:3300/0 conn(0x7f78c40717b0 0x7f78c4160820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:45.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.141+0000 7f78c9d37700 1 -- 172.21.15.16:0/4266822667 >> v2:172.21.15.132:3300/0 conn(0x7f78c40717b0 msgr2=0x7f78c4160820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:45.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.141+0000 7f78c9d37700 1 --2- 172.21.15.16:0/4266822667 >> v2:172.21.15.132:3300/0 conn(0x7f78c40717b0 0x7f78c4160820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:45.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.141+0000 7f78c9d37700 1 -- 172.21.15.16:0/4266822667 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78c415d200 con 0x7f78c40713e0 2023-05-20T01:28:45.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.141+0000 7f78c9d37700 1 --2- 172.21.15.16:0/4266822667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78c40713e0 0x7f78c40cf370 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f78c4161940 tx=0x7f78b4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5f89d3fb257aa4a server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:45.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.141+0000 7f78c8d35700 1 -- 172.21.15.16:0/4266822667 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78b400b870 con 0x7f78c40713e0 2023-05-20T01:28:45.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.141+0000 7f78c8d35700 1 -- 172.21.15.16:0/4266822667 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78b400b9d0 con 0x7f78c40713e0 2023-05-20T01:28:45.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.141+0000 7f78c8d35700 1 -- 172.21.15.16:0/4266822667 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78b400bd60 con 0x7f78c40713e0 2023-05-20T01:28:45.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.142+0000 7f78cbf9b700 1 -- 172.21.15.16:0/4266822667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78c40713e0 msgr2=0x7f78c40cf370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:45.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.142+0000 7f78cbf9b700 1 --2- 172.21.15.16:0/4266822667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78c40713e0 0x7f78c40cf370 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f78c4161940 tx=0x7f78b4009bd0 comp rx=0 tx=0).stop 2023-05-20T01:28:45.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.142+0000 7f78cbf9b700 1 -- 172.21.15.16:0/4266822667 shutdown_connections 2023-05-20T01:28:45.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.142+0000 7f78cbf9b700 1 --2- 172.21.15.16:0/4266822667 >> v2:172.21.15.132:3300/0 conn(0x7f78c40717b0 0x7f78c4160820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:45.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.142+0000 7f78cbf9b700 1 --2- 172.21.15.16:0/4266822667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78c40713e0 0x7f78c40cf370 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:45.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.142+0000 7f78cbf9b700 1 --2- 172.21.15.16:0/4266822667 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f78c4072400 0x7f78c415dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:45.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.142+0000 7f78cbf9b700 1 -- 172.21.15.16:0/4266822667 >> 172.21.15.16:0/4266822667 conn(0x7f78c406e250 msgr2=0x7f78c40ccc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:45.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.142+0000 7f78cbf9b700 1 -- 172.21.15.16:0/4266822667 shutdown_connections 2023-05-20T01:28:45.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.142+0000 7f78cbf9b700 1 -- 172.21.15.16:0/4266822667 wait complete. 2023-05-20T01:28:45.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.143+0000 7f78cbf9b700 1 Processor -- start 2023-05-20T01:28:45.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.143+0000 7f78cbf9b700 1 -- start start 2023-05-20T01:28:45.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.144+0000 7f78cbf9b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f78c40713e0 0x7f78c4154ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:45.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.144+0000 7f78cbf9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78c40717b0 0x7f78c40c4930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:45.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.144+0000 7f78cbf9b700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f78c4072400 0x7f78c415b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:45.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.144+0000 7f78cbf9b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78c40c6930 con 0x7f78c40717b0 2023-05-20T01:28:45.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.144+0000 7f78cbf9b700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78c40c5890 con 0x7f78c40713e0 2023-05-20T01:28:45.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.144+0000 7f78cbf9b700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78c40c5b70 con 0x7f78c4072400 2023-05-20T01:28:45.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.144+0000 7f78c9536700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78c40717b0 0x7f78c40c4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:45.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.144+0000 7f78c9536700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78c40717b0 0x7f78c40c4930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40532/0 (socket says 172.21.15.16:40532) 2023-05-20T01:28:45.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.145+0000 7f78c9536700 1 -- 172.21.15.16:0/2057119296 learned_addr learned my addr 172.21.15.16:0/2057119296 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:45.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.145+0000 7f78ca538700 1 --2- 172.21.15.16:0/2057119296 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f78c4072400 0x7f78c415b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:45.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.145+0000 7f78c9d37700 1 --2- 172.21.15.16:0/2057119296 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f78c40713e0 0x7f78c4154ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:45.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.145+0000 7f78c9536700 1 -- 172.21.15.16:0/2057119296 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f78c4072400 msgr2=0x7f78c415b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:45.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.145+0000 7f78c9536700 1 --2- 172.21.15.16:0/2057119296 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f78c4072400 0x7f78c415b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:45.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.145+0000 7f78c9536700 1 -- 172.21.15.16:0/2057119296 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f78c40713e0 msgr2=0x7f78c4154ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:45.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.145+0000 7f78c9536700 1 --2- 172.21.15.16:0/2057119296 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f78c40713e0 0x7f78c4154ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:45.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.145+0000 7f78c9536700 1 -- 172.21.15.16:0/2057119296 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78b4009550 con 0x7f78c40717b0 2023-05-20T01:28:45.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.145+0000 7f78c9d37700 1 --2- 172.21.15.16:0/2057119296 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f78c40713e0 0x7f78c4154ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:28:45.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.145+0000 7f78c9536700 1 --2- 172.21.15.16:0/2057119296 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78c40717b0 0x7f78c40c4930 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f78c000b4a0 tx=0x7f78c000b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:45.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.145+0000 7f78ba7fc700 1 -- 172.21.15.16:0/2057119296 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78c0011830 con 0x7f78c40717b0 2023-05-20T01:28:45.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.145+0000 7f78ba7fc700 1 -- 172.21.15.16:0/2057119296 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78c0011990 con 0x7f78c40717b0 2023-05-20T01:28:45.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.145+0000 7f78ba7fc700 1 -- 172.21.15.16:0/2057119296 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78c0011d40 con 0x7f78c40717b0 2023-05-20T01:28:45.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.145+0000 7f78cbf9b700 1 -- 172.21.15.16:0/2057119296 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78c406b000 con 0x7f78c40717b0 2023-05-20T01:28:45.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.145+0000 7f78cbf9b700 1 -- 172.21.15.16:0/2057119296 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78c415e1c0 con 0x7f78c40717b0 2023-05-20T01:28:45.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.147+0000 7f78ba7fc700 1 -- 172.21.15.16:0/2057119296 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f78c001a400 con 0x7f78c40717b0 2023-05-20T01:28:45.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.147+0000 7f78ba7fc700 1 --2- 172.21.15.16:0/2057119296 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f78a806ace0 0x7f78a806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:45.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.147+0000 7f78ba7fc700 1 -- 172.21.15.16:0/2057119296 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6468+0+0 (secure 0 0 0) 0x7f78c00895c0 con 0x7f78c40717b0 2023-05-20T01:28:45.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.147+0000 7f78c9d37700 1 --2- 172.21.15.16:0/2057119296 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f78a806ace0 0x7f78a806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:45.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.147+0000 7f78c9d37700 1 --2- 172.21.15.16:0/2057119296 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f78a806ace0 0x7f78a806d190 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f78b4000c00 tx=0x7f78b4008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:45.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.149+0000 7f78d8127b80 1 Processor -- start 2023-05-20T01:28:45.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.149+0000 7f78d8127b80 1 -- start start 2023-05-20T01:28:45.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.150+0000 7f78d8127b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a597e8d400 0x55a597e8d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:45.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.150+0000 7f78d8127b80 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a597e89700 0x55a597e62460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:45.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.150+0000 7f78d8127b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a597e60870 0x55a597e629a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:45.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.150+0000 7f78d8127b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a597d68010 con 0x55a597e60870 2023-05-20T01:28:45.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.150+0000 7f78d8127b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a597dba850 con 0x55a597e8d400 2023-05-20T01:28:45.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.150+0000 7f78d8127b80 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a597db3ce0 con 0x55a597e89700 2023-05-20T01:28:45.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.150+0000 7f78ca538700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a597e60870 0x55a597e629a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:45.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.150+0000 7f78ca538700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a597e60870 0x55a597e629a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40540/0 (socket says 172.21.15.16:40540) 2023-05-20T01:28:45.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.150+0000 7f78ca538700 1 -- 172.21.15.16:0/3918953775 learned_addr learned my addr 172.21.15.16:0/3918953775 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:45.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.150+0000 7f78c9d37700 1 --2- 172.21.15.16:0/3918953775 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a597e8d400 0x55a597e8d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:45.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.150+0000 7f78c9536700 1 --2- 172.21.15.16:0/3918953775 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a597e89700 0x55a597e62460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:45.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.150+0000 7f78ca538700 1 -- 172.21.15.16:0/3918953775 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a597e89700 msgr2=0x55a597e62460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:45.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.150+0000 7f78ca538700 1 --2- 172.21.15.16:0/3918953775 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a597e89700 0x55a597e62460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:45.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.150+0000 7f78ca538700 1 -- 172.21.15.16:0/3918953775 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a597e8d400 msgr2=0x55a597e8d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:45.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.150+0000 7f78ca538700 1 --2- 172.21.15.16:0/3918953775 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a597e8d400 0x55a597e8d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:45.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.150+0000 7f78ca538700 1 -- 172.21.15.16:0/3918953775 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a597c355e0 con 0x55a597e60870 2023-05-20T01:28:45.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.150+0000 7f78c9d37700 1 --2- 172.21.15.16:0/3918953775 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a597e8d400 0x55a597e8d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:28:45.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.150+0000 7f78ca538700 1 --2- 172.21.15.16:0/3918953775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a597e60870 0x55a597e629a0 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f78bc00f5e0 tx=0x7f78bc00f8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be4b1fc5f2fc519e server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:45.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.151+0000 7f78a77fe700 1 -- 172.21.15.16:0/3918953775 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78bc007c20 con 0x55a597e60870 2023-05-20T01:28:45.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.151+0000 7f78a77fe700 1 -- 172.21.15.16:0/3918953775 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78bc00a780 con 0x55a597e60870 2023-05-20T01:28:45.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.151+0000 7f78a77fe700 1 -- 172.21.15.16:0/3918953775 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78bc00ab10 con 0x55a597e60870 2023-05-20T01:28:45.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.151+0000 7f78d8127b80 1 -- 172.21.15.16:0/3918953775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a597e60870 msgr2=0x55a597e629a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:45.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.151+0000 7f78d8127b80 1 --2- 172.21.15.16:0/3918953775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a597e60870 0x55a597e629a0 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f78bc00f5e0 tx=0x7f78bc00f8f0 comp rx=0 tx=0).stop 2023-05-20T01:28:45.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.151+0000 7f78d8127b80 1 -- 172.21.15.16:0/3918953775 shutdown_connections 2023-05-20T01:28:45.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.151+0000 7f78d8127b80 1 --2- 172.21.15.16:0/3918953775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a597e60870 0x55a597e629a0 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:45.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.151+0000 7f78d8127b80 1 --2- 172.21.15.16:0/3918953775 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a597e8d400 0x55a597e8d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:45.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.151+0000 7f78d8127b80 1 --2- 172.21.15.16:0/3918953775 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a597e89700 0x55a597e62460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:45.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.151+0000 7f78d8127b80 1 -- 172.21.15.16:0/3918953775 >> 172.21.15.16:0/3918953775 conn(0x55a597e57f30 msgr2=0x55a597ea8210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:45.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.151+0000 7f78d8127b80 1 -- 172.21.15.16:0/3918953775 shutdown_connections 2023-05-20T01:28:45.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.151+0000 7f78d8127b80 1 -- 172.21.15.16:0/3918953775 wait complete. 2023-05-20T01:28:45.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.152+0000 7f78d8127b80 1 -- start start 2023-05-20T01:28:45.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.152+0000 7f78d8127b80 1 Processor -- start 2023-05-20T01:28:45.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.153+0000 7f78d8127b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a597e60870 0x55a597e88fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:45.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.153+0000 7f78d8127b80 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a597e89700 0x55a597ea7a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:45.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.153+0000 7f78d8127b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a597e8d400 0x55a597e87cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:45.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.153+0000 7f78d8127b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a597d55890 con 0x55a597e60870 2023-05-20T01:28:45.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.153+0000 7f78d8127b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a597d55530 con 0x55a597e8d400 2023-05-20T01:28:45.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.153+0000 7f78d8127b80 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a597c378d0 con 0x55a597e89700 2023-05-20T01:28:45.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.153+0000 7f78d8127b80 10 client.0 _ll_register_callbacks cb 0x55a597d73730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-05-20T01:28:45.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.153+0000 7f78c9d37700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a597e60870 0x55a597e88fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:45.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.153+0000 7f78c9d37700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a597e60870 0x55a597e88fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40542/0 (socket says 172.21.15.16:40542) 2023-05-20T01:28:45.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.153+0000 7f78c9d37700 1 -- 172.21.15.16:0/3349002437 learned_addr learned my addr 172.21.15.16:0/3349002437 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:45.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.153+0000 7f78ca538700 1 --2- 172.21.15.16:0/3349002437 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a597e8d400 0x55a597e87cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:45.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.153+0000 7f78c9536700 1 --2- 172.21.15.16:0/3349002437 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a597e89700 0x55a597ea7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:45.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.153+0000 7f78c9d37700 1 -- 172.21.15.16:0/3349002437 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a597e89700 msgr2=0x55a597ea7a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:45.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.153+0000 7f78c9d37700 1 --2- 172.21.15.16:0/3349002437 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a597e89700 0x55a597ea7a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:45.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.153+0000 7f78c9d37700 1 -- 172.21.15.16:0/3349002437 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a597e8d400 msgr2=0x55a597e87cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:45.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.153+0000 7f78c9d37700 1 --2- 172.21.15.16:0/3349002437 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a597e8d400 0x55a597e87cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:45.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.153+0000 7f78c9d37700 1 -- 172.21.15.16:0/3349002437 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78bc007d80 con 0x55a597e60870 2023-05-20T01:28:45.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.153+0000 7f78ca538700 1 --2- 172.21.15.16:0/3349002437 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a597e8d400 0x55a597e87cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:28:45.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.154+0000 7f78c9d37700 1 --2- 172.21.15.16:0/3349002437 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a597e60870 0x55a597e88fe0 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f78c406ccc0 tx=0x7f78b40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96129d7467f686a8 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:45.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.154+0000 7f78a4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-05-20T01:28:45.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.154+0000 7f78a4ff9700 1 -- 172.21.15.16:0/3349002437 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78b402c070 con 0x55a597e60870 2023-05-20T01:28:45.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.154+0000 7f78a4ff9700 1 -- 172.21.15.16:0/3349002437 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(137..137 src has 1..137) v4 ==== 6468+0+0 (secure 0 0 0) 0x7f78b4032dd0 con 0x55a597e60870 2023-05-20T01:28:45.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.154+0000 7f78d8127b80 1 -- 172.21.15.16:0/3349002437 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a597e556f0 con 0x55a597e60870 2023-05-20T01:28:45.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.154+0000 7f78a4ff9700 1 -- 172.21.15.16:0/3349002437 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=105) v2 ==== 24+0+0 (secure 0 0 0) 0x7f78b400bee0 con 0x55a597e60870 2023-05-20T01:28:45.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.154+0000 7f78d8127b80 10 client.8988 fetch_fsmap learned FSMap version 105 2023-05-20T01:28:45.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.154+0000 7f78d8127b80 1 -- 172.21.15.16:0/3349002437 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=105,monmap=2+}) v3 -- 0x55a597d7a140 con 0x55a597e60870 2023-05-20T01:28:45.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.154+0000 7f78a4ff9700 1 -- 172.21.15.16:0/3349002437 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a597e556f0 con 0x55a597e60870 2023-05-20T01:28:45.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.154+0000 7f78a4ff9700 1 -- 172.21.15.16:0/3349002437 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 105) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f78b4003d10 con 0x55a597e60870 2023-05-20T01:28:45.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.154+0000 7f78d8127b80 10 client.8988 fetch_fsmap finished waiting for FSMap version 105 2023-05-20T01:28:45.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.154+0000 7f78d8127b80 10 client.8988 resolve_mds: resolved name 'a' to daemon mds.a 2023-05-20T01:28:45.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.154+0000 7f78d8127b80 20 client.8988 populate_metadata read hostname 'smithi016' 2023-05-20T01:28:45.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.154+0000 7f78d8127b80 1 --2- 172.21.15.16:0/3349002437 >> [v2:172.21.15.16:6835/2869411815,v1:172.21.15.16:6837/2869411815] conn(0x55a597e82900 0x55a597e82cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:45.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.154+0000 7f78d8127b80 4 client.8988 mds_command: new command op to 8937 tid=0[{"prefix": "get_command_descriptions"}] 2023-05-20T01:28:45.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.154+0000 7f78d8127b80 1 -- 172.21.15.16:0/3349002437 --> [v2:172.21.15.16:6835/2869411815,v1:172.21.15.16:6837/2869411815] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a597d76050 con 0x55a597e82900 2023-05-20T01:28:45.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.154+0000 7f78c9536700 1 --2- 172.21.15.16:0/3349002437 >> [v2:172.21.15.16:6835/2869411815,v1:172.21.15.16:6837/2869411815] conn(0x55a597e82900 0x55a597e82cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:45.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.155+0000 7f78c9536700 1 --2- 172.21.15.16:0/3349002437 >> [v2:172.21.15.16:6835/2869411815,v1:172.21.15.16:6837/2869411815] conn(0x55a597e82900 0x55a597e82cd0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14e64514dfea0e29 server_cookie=ea70d74682cb107e in_seq=0 out_seq=0 2023-05-20T01:28:45.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.155+0000 7f78a4ff9700 10 client.8988 ms_handle_connect on v2:172.21.15.16:6835/2869411815 2023-05-20T01:28:45.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.155+0000 7f78a4ff9700 1 -- 172.21.15.16:0/3349002437 <== mds.0 v2:172.21.15.16:6835/2869411815 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55a597d76050 con 0x55a597e82900 2023-05-20T01:28:45.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.155+0000 7f78a4ff9700 10 client.8988 handle_command_reply: tid=0 2023-05-20T01:28:45.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.155+0000 7f78d8127b80 1 client.8988 shutdown 2023-05-20T01:28:45.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.155+0000 7f78d8127b80 1 -- 172.21.15.16:0/3349002437 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a597e60870 msgr2=0x55a597e88fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:45.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.155+0000 7f78d8127b80 1 --2- 172.21.15.16:0/3349002437 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a597e60870 0x55a597e88fe0 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f78c406ccc0 tx=0x7f78b40047c0 comp rx=0 tx=0).stop 2023-05-20T01:28:45.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.155+0000 7f78d8127b80 1 -- 172.21.15.16:0/3349002437 shutdown_connections 2023-05-20T01:28:45.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.155+0000 7f78d8127b80 1 --2- 172.21.15.16:0/3349002437 >> [v2:172.21.15.16:6835/2869411815,v1:172.21.15.16:6837/2869411815] conn(0x55a597e82900 0x55a597e82cd0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:45.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.155+0000 7f78d8127b80 1 --2- 172.21.15.16:0/3349002437 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a597e8d400 0x55a597e87cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:45.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.155+0000 7f78d8127b80 1 --2- 172.21.15.16:0/3349002437 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a597e60870 0x55a597e88fe0 unknown :-1 s=CLOSED pgs=1688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:45.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.155+0000 7f78a4ff9700 0 client.8988 ms_handle_reset on v2:172.21.15.16:6835/2869411815 2023-05-20T01:28:45.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.155+0000 7f78d8127b80 1 --2- 172.21.15.16:0/3349002437 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a597e89700 0x55a597ea7a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:45.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.156+0000 7f78d8127b80 1 -- 172.21.15.16:0/3349002437 >> 172.21.15.16:0/3349002437 conn(0x55a597e57f30 msgr2=0x55a597e5d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:45.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.156+0000 7f78d8127b80 1 -- 172.21.15.16:0/3349002437 shutdown_connections 2023-05-20T01:28:45.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.156+0000 7f78d8127b80 1 -- 172.21.15.16:0/3349002437 wait complete. 2023-05-20T01:28:45.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.156+0000 7f78d8127b80 20 client.8988 trim_cache size 0 max 16384 2023-05-20T01:28:45.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.174+0000 7f78d8127b80 1 Processor -- start 2023-05-20T01:28:45.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.174+0000 7f78d8127b80 1 -- start start 2023-05-20T01:28:45.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.174+0000 7f78d8127b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a597e60870 0x55a597e5f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:45.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.174+0000 7f78d8127b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a597e89700 0x55a597e5d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:45.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.174+0000 7f78d8127b80 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a597e8d400 0x55a597ec0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:45.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.174+0000 7f78d8127b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a597e852b0 con 0x55a597e60870 2023-05-20T01:28:45.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.174+0000 7f78d8127b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a597e630f0 con 0x55a597e89700 2023-05-20T01:28:45.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.174+0000 7f78d8127b80 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a597e63ff0 con 0x55a597e8d400 2023-05-20T01:28:45.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.175+0000 7f78c9d37700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a597e60870 0x55a597e5f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:45.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.175+0000 7f78c9d37700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a597e60870 0x55a597e5f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40552/0 (socket says 172.21.15.16:40552) 2023-05-20T01:28:45.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.175+0000 7f78c9d37700 1 -- 172.21.15.16:0/374779926 learned_addr learned my addr 172.21.15.16:0/374779926 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:45.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.175+0000 7f78ca538700 1 --2- 172.21.15.16:0/374779926 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a597e8d400 0x55a597ec0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:45.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.175+0000 7f78c9536700 1 --2- 172.21.15.16:0/374779926 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a597e89700 0x55a597e5d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:45.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.175+0000 7f78c9d37700 1 -- 172.21.15.16:0/374779926 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a597e8d400 msgr2=0x55a597ec0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:45.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.175+0000 7f78c9d37700 1 --2- 172.21.15.16:0/374779926 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a597e8d400 0x55a597ec0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:45.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.175+0000 7f78c9d37700 1 -- 172.21.15.16:0/374779926 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a597e89700 msgr2=0x55a597e5d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:45.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.175+0000 7f78c9d37700 1 --2- 172.21.15.16:0/374779926 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a597e89700 0x55a597e5d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:45.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.175+0000 7f78c9d37700 1 -- 172.21.15.16:0/374779926 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a597d79e90 con 0x55a597e60870 2023-05-20T01:28:45.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.175+0000 7f78c9536700 1 --2- 172.21.15.16:0/374779926 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a597e89700 0x55a597e5d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:28:45.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.175+0000 7f78c9d37700 1 --2- 172.21.15.16:0/374779926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a597e60870 0x55a597e5f6d0 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7f78b4006d10 tx=0x7f78b402c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a03a7fdd4059be6d server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:45.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.175+0000 7f78a7fff700 1 -- 172.21.15.16:0/374779926 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78b4025d10 con 0x55a597e60870 2023-05-20T01:28:45.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.175+0000 7f78a7fff700 1 -- 172.21.15.16:0/374779926 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78b4025e70 con 0x55a597e60870 2023-05-20T01:28:45.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.175+0000 7f78a7fff700 1 -- 172.21.15.16:0/374779926 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78b40259e0 con 0x55a597e60870 2023-05-20T01:28:45.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.176+0000 7f78d8127b80 1 -- 172.21.15.16:0/374779926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a597e60870 msgr2=0x55a597e5f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:45.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.176+0000 7f78d8127b80 1 --2- 172.21.15.16:0/374779926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a597e60870 0x55a597e5f6d0 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7f78b4006d10 tx=0x7f78b402c000 comp rx=0 tx=0).stop 2023-05-20T01:28:45.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.176+0000 7f78d8127b80 1 -- 172.21.15.16:0/374779926 shutdown_connections 2023-05-20T01:28:45.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.176+0000 7f78d8127b80 1 --2- 172.21.15.16:0/374779926 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a597e8d400 0x55a597ec0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:45.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.176+0000 7f78d8127b80 1 --2- 172.21.15.16:0/374779926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a597e60870 0x55a597e5f6d0 unknown :-1 s=CLOSED pgs=1689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:45.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.176+0000 7f78d8127b80 1 --2- 172.21.15.16:0/374779926 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a597e89700 0x55a597e5d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:45.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.176+0000 7f78d8127b80 1 -- 172.21.15.16:0/374779926 >> 172.21.15.16:0/374779926 conn(0x55a597e57f30 msgr2=0x55a597e59400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:45.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.176+0000 7f78d8127b80 1 -- 172.21.15.16:0/374779926 shutdown_connections 2023-05-20T01:28:45.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.176+0000 7f78d8127b80 1 -- 172.21.15.16:0/374779926 wait complete. 2023-05-20T01:28:45.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.176+0000 7f78d8127b80 1 -- start start 2023-05-20T01:28:45.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.176+0000 7f78d8127b80 1 Processor -- start 2023-05-20T01:28:45.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.177+0000 7f78d8127b80 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a597e60870 0x55a597ec6120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:45.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.177+0000 7f78d8127b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a597e89700 0x55a597ec4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:45.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.177+0000 7f78d8127b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a597e8d400 0x55a597e87210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:45.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.177+0000 7f78d8127b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a597eb9b90 con 0x55a597e89700 2023-05-20T01:28:45.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.177+0000 7f78d8127b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a597ea8750 con 0x55a597e8d400 2023-05-20T01:28:45.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.177+0000 7f78d8127b80 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a597ec6af0 con 0x55a597e60870 2023-05-20T01:28:45.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.177+0000 7f78d8127b80 10 client.0 _ll_register_callbacks cb 0x7f78b400be10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-05-20T01:28:45.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.177+0000 7f78c9536700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a597e89700 0x55a597ec4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:45.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.177+0000 7f78c9536700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a597e89700 0x55a597ec4430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40558/0 (socket says 172.21.15.16:40558) 2023-05-20T01:28:45.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.177+0000 7f78c9536700 1 -- 172.21.15.16:0/4207852689 learned_addr learned my addr 172.21.15.16:0/4207852689 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:45.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.178+0000 7f78c9536700 1 -- 172.21.15.16:0/4207852689 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a597e60870 msgr2=0x55a597ec6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:45.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.178+0000 7f78c9536700 1 --2- 172.21.15.16:0/4207852689 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a597e60870 0x55a597ec6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:45.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.178+0000 7f78c9536700 1 -- 172.21.15.16:0/4207852689 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a597e8d400 msgr2=0x55a597e87210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:45.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.178+0000 7f78c9536700 1 --2- 172.21.15.16:0/4207852689 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a597e8d400 0x55a597e87210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:45.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.178+0000 7f78c9536700 1 -- 172.21.15.16:0/4207852689 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a597d7a140 con 0x55a597e89700 2023-05-20T01:28:45.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.178+0000 7f78c9536700 1 --2- 172.21.15.16:0/4207852689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a597e89700 0x55a597ec4430 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f78c000bbb0 tx=0x7f78c0011ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7cec565d0cbbfc0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:45.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.178+0000 7f78a4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-05-20T01:28:45.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.178+0000 7f78a4ff9700 1 -- 172.21.15.16:0/4207852689 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78c0056600 con 0x55a597e89700 2023-05-20T01:28:45.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.178+0000 7f78a4ff9700 1 -- 172.21.15.16:0/4207852689 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(137..137 src has 1..137) v4 ==== 6468+0+0 (secure 0 0 0) 0x7f78c0017b90 con 0x55a597e89700 2023-05-20T01:28:45.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.178+0000 7f78d8127b80 1 -- 172.21.15.16:0/4207852689 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f78bc00a780 con 0x55a597e89700 2023-05-20T01:28:45.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.178+0000 7f78a4ff9700 1 -- 172.21.15.16:0/4207852689 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=105) v2 ==== 24+0+0 (secure 0 0 0) 0x7f78c0017d50 con 0x55a597e89700 2023-05-20T01:28:45.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.178+0000 7f78d8127b80 10 client.8994 fetch_fsmap learned FSMap version 105 2023-05-20T01:28:45.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.178+0000 7f78d8127b80 1 -- 172.21.15.16:0/4207852689 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=105,monmap=2+}) v3 -- 0x55a597bcac10 con 0x55a597e89700 2023-05-20T01:28:45.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.178+0000 7f78a4ff9700 1 -- 172.21.15.16:0/4207852689 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78bc00a780 con 0x55a597e89700 2023-05-20T01:28:45.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.178+0000 7f78a4ff9700 1 -- 172.21.15.16:0/4207852689 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 105) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f78c006fb70 con 0x55a597e89700 2023-05-20T01:28:45.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.178+0000 7f78d8127b80 10 client.8994 fetch_fsmap finished waiting for FSMap version 105 2023-05-20T01:28:45.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.178+0000 7f78d8127b80 10 client.8994 resolve_mds: resolved name 'a' to daemon mds.a 2023-05-20T01:28:45.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.178+0000 7f78d8127b80 20 client.8994 populate_metadata read hostname 'smithi016' 2023-05-20T01:28:45.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.178+0000 7f78d8127b80 1 --2- 172.21.15.16:0/4207852689 >> [v2:172.21.15.16:6835/2869411815,v1:172.21.15.16:6837/2869411815] conn(0x55a597f0e810 0x55a597f0ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:45.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.178+0000 7f78d8127b80 4 client.8994 mds_command: new command op to 8937 tid=0[{"prefix": "status"}] 2023-05-20T01:28:45.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.178+0000 7f78d8127b80 1 -- 172.21.15.16:0/4207852689 --> [v2:172.21.15.16:6835/2869411815,v1:172.21.15.16:6837/2869411815] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55a597d82610 con 0x55a597f0e810 2023-05-20T01:28:45.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.179+0000 7f78c9d37700 1 --2- 172.21.15.16:0/4207852689 >> [v2:172.21.15.16:6835/2869411815,v1:172.21.15.16:6837/2869411815] conn(0x55a597f0e810 0x55a597f0ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:45.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.179+0000 7f78a4ff9700 10 client.8994 ms_handle_connect on v2:172.21.15.16:6835/2869411815 2023-05-20T01:28:45.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.179+0000 7f78c9d37700 1 --2- 172.21.15.16:0/4207852689 >> [v2:172.21.15.16:6835/2869411815,v1:172.21.15.16:6837/2869411815] conn(0x55a597f0e810 0x55a597f0ebe0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a26a5d9ee0a9df1a server_cookie=a2c69aca2ea0d08c in_seq=0 out_seq=0 2023-05-20T01:28:45.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.179+0000 7f78a4ff9700 1 -- 172.21.15.16:0/4207852689 <== mds.0 v2:172.21.15.16:6835/2869411815 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+334 (crc 0 0 0) 0x55a597d82610 con 0x55a597f0e810 2023-05-20T01:28:45.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.179+0000 7f78a4ff9700 10 client.8994 handle_command_reply: tid=0 2023-05-20T01:28:45.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.179+0000 7f78d8127b80 1 client.8994 shutdown 2023-05-20T01:28:45.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.179+0000 7f78d8127b80 1 -- 172.21.15.16:0/4207852689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a597e89700 msgr2=0x55a597ec4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:45.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.179+0000 7f78d8127b80 1 --2- 172.21.15.16:0/4207852689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a597e89700 0x55a597ec4430 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f78c000bbb0 tx=0x7f78c0011ea0 comp rx=0 tx=0).stop 2023-05-20T01:28:45.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.179+0000 7f78d8127b80 1 -- 172.21.15.16:0/4207852689 shutdown_connections 2023-05-20T01:28:45.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.179+0000 7f78d8127b80 1 --2- 172.21.15.16:0/4207852689 >> [v2:172.21.15.16:6835/2869411815,v1:172.21.15.16:6837/2869411815] conn(0x55a597f0e810 0x55a597f0ebe0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:45.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.179+0000 7f78d8127b80 1 --2- 172.21.15.16:0/4207852689 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a597e8d400 0x55a597e87210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:45.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.179+0000 7f78a4ff9700 0 client.8994 ms_handle_reset on v2:172.21.15.16:6835/2869411815 2023-05-20T01:28:45.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.179+0000 7f78d8127b80 1 --2- 172.21.15.16:0/4207852689 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a597e60870 0x55a597ec6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:45.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.179+0000 7f78d8127b80 1 --2- 172.21.15.16:0/4207852689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a597e89700 0x55a597ec4430 unknown :-1 s=CLOSED pgs=1690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:45.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.179+0000 7f78d8127b80 1 -- 172.21.15.16:0/4207852689 >> 172.21.15.16:0/4207852689 conn(0x55a597e57f30 msgr2=0x55a597e5d200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:45.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.180+0000 7f78d8127b80 1 -- 172.21.15.16:0/4207852689 shutdown_connections 2023-05-20T01:28:45.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.180+0000 7f78d8127b80 1 -- 172.21.15.16:0/4207852689 wait complete. 2023-05-20T01:28:45.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.180+0000 7f78d8127b80 20 client.8994 trim_cache size 0 max 16384 2023-05-20T01:28:45.193 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-05-20T01:28:45.193 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "c2245cc5-ecab-4087-a59b-bc81db901676", 2023-05-20T01:28:45.194 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2023-05-20T01:28:45.194 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 8937, 2023-05-20T01:28:45.194 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2023-05-20T01:28:45.194 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2023-05-20T01:28:45.194 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2023-05-20T01:28:45.194 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 4.6195756030000004, 2023-05-20T01:28:45.194 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 105, 2023-05-20T01:28:45.194 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 137, 2023-05-20T01:28:45.195 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 137, 2023-05-20T01:28:45.195 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 5.5907862579999996 2023-05-20T01:28:45.195 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-05-20T01:28:45.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.180+0000 7f78cbf9b700 1 -- 172.21.15.16:0/2057119296 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f78a806ace0 msgr2=0x7f78a806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:45.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.180+0000 7f78cbf9b700 1 --2- 172.21.15.16:0/2057119296 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f78a806ace0 0x7f78a806d190 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f78b4000c00 tx=0x7f78b4008030 comp rx=0 tx=0).stop 2023-05-20T01:28:45.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.180+0000 7f78cbf9b700 1 -- 172.21.15.16:0/2057119296 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78c40717b0 msgr2=0x7f78c40c4930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:45.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.180+0000 7f78cbf9b700 1 --2- 172.21.15.16:0/2057119296 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78c40717b0 0x7f78c40c4930 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f78c000b4a0 tx=0x7f78c000b7b0 comp rx=0 tx=0).stop 2023-05-20T01:28:45.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.180+0000 7f78cbf9b700 1 -- 172.21.15.16:0/2057119296 shutdown_connections 2023-05-20T01:28:45.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.180+0000 7f78cbf9b700 1 --2- 172.21.15.16:0/2057119296 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f78a806ace0 0x7f78a806d190 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:45.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.180+0000 7f78cbf9b700 1 --2- 172.21.15.16:0/2057119296 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f78c4072400 0x7f78c415b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:45.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.180+0000 7f78cbf9b700 1 --2- 172.21.15.16:0/2057119296 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f78c40713e0 0x7f78c4154ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:45.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.180+0000 7f78cbf9b700 1 --2- 172.21.15.16:0/2057119296 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78c40717b0 0x7f78c40c4930 unknown :-1 s=CLOSED pgs=1686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:45.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.180+0000 7f78cbf9b700 1 -- 172.21.15.16:0/2057119296 >> 172.21.15.16:0/2057119296 conn(0x7f78c406e250 msgr2=0x7f78c406a5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:45.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.180+0000 7f78cbf9b700 1 -- 172.21.15.16:0/2057119296 shutdown_connections 2023-05-20T01:28:45.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.180+0000 7f78cbf9b700 1 -- 172.21.15.16:0/2057119296 wait complete. 2023-05-20T01:28:45.198 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-05-20T01:28:45.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.318+0000 7f4c60ea1700 1 Processor -- start 2023-05-20T01:28:45.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.318+0000 7f4c60ea1700 1 -- start start 2023-05-20T01:28:45.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.319+0000 7f4c60ea1700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4c5c0ce7a0 0x7f4c5c15dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:45.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.319+0000 7f4c60ea1700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f4c5c0cab20 0x7f4c5c0caf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:45.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.319+0000 7f4c60ea1700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c5c06f730 con 0x7f4c5c15cd60 2023-05-20T01:28:45.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.319+0000 7f4c60ea1700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c5c15e2a0 con 0x7f4c5c0ce7a0 2023-05-20T01:28:45.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.319+0000 7f4c60ea1700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c5c15d130 con 0x7f4c5c0cab20 2023-05-20T01:28:45.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.319+0000 7f4c59d9b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4c5c0ce7a0 0x7f4c5c15dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:45.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.319+0000 7f4c59d9b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4c5c0ce7a0 0x7f4c5c15dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40566/0 (socket says 172.21.15.16:40566) 2023-05-20T01:28:45.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.319+0000 7f4c59d9b700 1 -- 172.21.15.16:0/2097151229 learned_addr learned my addr 172.21.15.16:0/2097151229 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:45.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.319+0000 7f4c5a59c700 1 --1- 172.21.15.16:0/2097151229 >> v1:172.21.15.16:6789/0 conn(0x7f4c5c15cd60 0x7f4c5c0ce090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:52092/0 (socket says 172.21.15.16:52092) 2023-05-20T01:28:45.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.319+0000 7f4c5ad9d700 1 --2- 172.21.15.16:0/2097151229 >> v2:172.21.15.132:3300/0 conn(0x7f4c5c0cab20 0x7f4c5c0caf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:45.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.319+0000 7f4c59d9b700 1 -- 172.21.15.16:0/2097151229 >> v2:172.21.15.132:3300/0 conn(0x7f4c5c0cab20 msgr2=0x7f4c5c0caf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:45.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.319+0000 7f4c59d9b700 1 --2- 172.21.15.16:0/2097151229 >> v2:172.21.15.132:3300/0 conn(0x7f4c5c0cab20 0x7f4c5c0caf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:45.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.319+0000 7f4c59d9b700 1 -- 172.21.15.16:0/2097151229 >> v1:172.21.15.16:6789/0 conn(0x7f4c5c15cd60 legacy=0x7f4c5c0ce090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:45.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.319+0000 7f4c59d9b700 1 -- 172.21.15.16:0/2097151229 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c5c06e840 con 0x7f4c5c0ce7a0 2023-05-20T01:28:45.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.319+0000 7f4c5959a700 1 -- 172.21.15.16:0/2097151229 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1806586564 0 0) 0x7f4c5c06f730 con 0x7f4c5c15cd60 2023-05-20T01:28:45.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.319+0000 7f4c59d9b700 1 --2- 172.21.15.16:0/2097151229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c5c0ce7a0 0x7f4c5c15dd60 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c071ed0 tx=0x7f4c5000be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad8fdedacaa3bee server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:45.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.319+0000 7f4c5959a700 1 -- 172.21.15.16:0/2097151229 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c5000dd30 con 0x7f4c5c0ce7a0 2023-05-20T01:28:45.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.320+0000 7f4c5959a700 1 -- 172.21.15.16:0/2097151229 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c5000de90 con 0x7f4c5c0ce7a0 2023-05-20T01:28:45.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.320+0000 7f4c5959a700 1 -- 172.21.15.16:0/2097151229 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c50010690 con 0x7f4c5c0ce7a0 2023-05-20T01:28:45.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.320+0000 7f4c60ea1700 1 -- 172.21.15.16:0/2097151229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c5c0ce7a0 msgr2=0x7f4c5c15dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:45.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.320+0000 7f4c60ea1700 1 --2- 172.21.15.16:0/2097151229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c5c0ce7a0 0x7f4c5c15dd60 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c071ed0 tx=0x7f4c5000be70 comp rx=0 tx=0).stop 2023-05-20T01:28:45.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.320+0000 7f4c60ea1700 1 -- 172.21.15.16:0/2097151229 shutdown_connections 2023-05-20T01:28:45.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.320+0000 7f4c60ea1700 1 --2- 172.21.15.16:0/2097151229 >> v2:172.21.15.132:3300/0 conn(0x7f4c5c0cab20 0x7f4c5c0caf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:45.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.320+0000 7f4c60ea1700 1 --2- 172.21.15.16:0/2097151229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c5c0ce7a0 0x7f4c5c15dd60 unknown :-1 s=CLOSED pgs=1692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:45.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.320+0000 7f4c60ea1700 1 -- 172.21.15.16:0/2097151229 >> 172.21.15.16:0/2097151229 conn(0x7f4c5c0b1c90 msgr2=0x7f4c5c0b2060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:45.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.320+0000 7f4c60ea1700 1 -- 172.21.15.16:0/2097151229 shutdown_connections 2023-05-20T01:28:45.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.320+0000 7f4c60ea1700 1 -- 172.21.15.16:0/2097151229 wait complete. 2023-05-20T01:28:45.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.322+0000 7f4c60ea1700 1 Processor -- start 2023-05-20T01:28:45.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.322+0000 7f4c60ea1700 1 -- start start 2023-05-20T01:28:45.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.322+0000 7f4c60ea1700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4c5c0cab20 0x7f4c5c155580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:45.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.322+0000 7f4c60ea1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c5c0ce7a0 0x7f4c5c0cbfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:45.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.322+0000 7f4c60ea1700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4c5c15cd60 0x7f4c5c0cc500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:45.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.322+0000 7f4c60ea1700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c5c0b4d40 con 0x7f4c5c0ce7a0 2023-05-20T01:28:45.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.322+0000 7f4c60ea1700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c5c0b5020 con 0x7f4c5c15cd60 2023-05-20T01:28:45.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.322+0000 7f4c60ea1700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c5c0b5300 con 0x7f4c5c0cab20 2023-05-20T01:28:45.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.322+0000 7f4c59d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c5c0ce7a0 0x7f4c5c0cbfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:45.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.323+0000 7f4c59d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c5c0ce7a0 0x7f4c5c0cbfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40572/0 (socket says 172.21.15.16:40572) 2023-05-20T01:28:45.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.323+0000 7f4c59d9b700 1 -- 172.21.15.16:0/3321347566 learned_addr learned my addr 172.21.15.16:0/3321347566 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:45.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.323+0000 7f4c5ad9d700 1 --2- 172.21.15.16:0/3321347566 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4c5c15cd60 0x7f4c5c0cc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:45.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.323+0000 7f4c5a59c700 1 --2- 172.21.15.16:0/3321347566 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4c5c0cab20 0x7f4c5c155580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:45.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.323+0000 7f4c59d9b700 1 -- 172.21.15.16:0/3321347566 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4c5c0cab20 msgr2=0x7f4c5c155580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:45.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.323+0000 7f4c59d9b700 1 --2- 172.21.15.16:0/3321347566 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4c5c0cab20 0x7f4c5c155580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:45.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.323+0000 7f4c59d9b700 1 -- 172.21.15.16:0/3321347566 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4c5c15cd60 msgr2=0x7f4c5c0cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:45.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.323+0000 7f4c59d9b700 1 --2- 172.21.15.16:0/3321347566 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4c5c15cd60 0x7f4c5c0cc500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:45.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.323+0000 7f4c59d9b700 1 -- 172.21.15.16:0/3321347566 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c5000b7f0 con 0x7f4c5c0ce7a0 2023-05-20T01:28:45.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.323+0000 7f4c5a59c700 1 --2- 172.21.15.16:0/3321347566 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4c5c0cab20 0x7f4c5c155580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:28:45.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.323+0000 7f4c5ad9d700 1 --2- 172.21.15.16:0/3321347566 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4c5c15cd60 0x7f4c5c0cc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:28:45.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.323+0000 7f4c59d9b700 1 --2- 172.21.15.16:0/3321347566 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c5c0ce7a0 0x7f4c5c0cbfc0 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f4c5000b7c0 tx=0x7f4c50004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:45.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.323+0000 7f4c4affd700 1 -- 172.21.15.16:0/3321347566 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c5001f070 con 0x7f4c5c0ce7a0 2023-05-20T01:28:45.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.323+0000 7f4c4affd700 1 -- 172.21.15.16:0/3321347566 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c50003ba0 con 0x7f4c5c0ce7a0 2023-05-20T01:28:45.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.323+0000 7f4c4affd700 1 -- 172.21.15.16:0/3321347566 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c50024470 con 0x7f4c5c0ce7a0 2023-05-20T01:28:45.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.323+0000 7f4c60ea1700 1 -- 172.21.15.16:0/3321347566 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c5c06ee00 con 0x7f4c5c0ce7a0 2023-05-20T01:28:45.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.323+0000 7f4c60ea1700 1 -- 172.21.15.16:0/3321347566 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c5c06eb20 con 0x7f4c5c0ce7a0 2023-05-20T01:28:45.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.324+0000 7f4c60ea1700 1 -- 172.21.15.16:0/3321347566 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c5c06f0e0 con 0x7f4c5c0ce7a0 2023-05-20T01:28:45.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.325+0000 7f4c4affd700 1 -- 172.21.15.16:0/3321347566 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4c5002d430 con 0x7f4c5c0ce7a0 2023-05-20T01:28:45.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.325+0000 7f4c4affd700 1 --2- 172.21.15.16:0/3321347566 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f4c3c06ace0 0x7f4c3c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:45.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.325+0000 7f4c4affd700 1 -- 172.21.15.16:0/3321347566 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6468+0+0 (secure 0 0 0) 0x7f4c5008cb00 con 0x7f4c5c0ce7a0 2023-05-20T01:28:45.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.325+0000 7f4c5a59c700 1 --2- 172.21.15.16:0/3321347566 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f4c3c06ace0 0x7f4c3c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:45.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.325+0000 7f4c5a59c700 1 --2- 172.21.15.16:0/3321347566 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f4c3c06ace0 0x7f4c3c06d190 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c000f60 tx=0x7f4c4c007d30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:45.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.329+0000 7f4c4affd700 1 -- 172.21.15.16:0/3321347566 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f4c50057aa0 con 0x7f4c5c0ce7a0 2023-05-20T01:28:45.563 INFO:tasks.ceph.mds.a.smithi016.stderr:2023-05-20T01:28:45.559+0000 7fa7fec3c700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-05-20T01:28:45.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.571+0000 7f4c60ea1700 1 -- 172.21.15.16:0/3321347566 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f4c5c067710 con 0x7f4c5c0ce7a0 2023-05-20T01:28:45.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.571+0000 7f4c4affd700 1 -- 172.21.15.16:0/3321347566 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v105) v1 ==== 53+0+73 (secure 0 0 0) 0x7f4c5005b0c0 con 0x7f4c5c0ce7a0 2023-05-20T01:28:45.577 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-05-20T01:28:45.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.573+0000 7f4c60ea1700 1 -- 172.21.15.16:0/3321347566 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f4c3c06ace0 msgr2=0x7f4c3c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:45.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.573+0000 7f4c60ea1700 1 --2- 172.21.15.16:0/3321347566 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f4c3c06ace0 0x7f4c3c06d190 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c000f60 tx=0x7f4c4c007d30 comp rx=0 tx=0).stop 2023-05-20T01:28:45.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.573+0000 7f4c60ea1700 1 -- 172.21.15.16:0/3321347566 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c5c0ce7a0 msgr2=0x7f4c5c0cbfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:45.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.573+0000 7f4c60ea1700 1 --2- 172.21.15.16:0/3321347566 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c5c0ce7a0 0x7f4c5c0cbfc0 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f4c5000b7c0 tx=0x7f4c50004750 comp rx=0 tx=0).stop 2023-05-20T01:28:45.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.573+0000 7f4c60ea1700 1 -- 172.21.15.16:0/3321347566 shutdown_connections 2023-05-20T01:28:45.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.573+0000 7f4c60ea1700 1 --2- 172.21.15.16:0/3321347566 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4c5c15cd60 0x7f4c5c0cc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:45.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.573+0000 7f4c60ea1700 1 --2- 172.21.15.16:0/3321347566 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f4c3c06ace0 0x7f4c3c06d190 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:45.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.573+0000 7f4c60ea1700 1 --2- 172.21.15.16:0/3321347566 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4c5c0cab20 0x7f4c5c155580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:45.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.573+0000 7f4c60ea1700 1 --2- 172.21.15.16:0/3321347566 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c5c0ce7a0 0x7f4c5c0cbfc0 unknown :-1 s=CLOSED pgs=1693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:45.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.573+0000 7f4c60ea1700 1 -- 172.21.15.16:0/3321347566 >> 172.21.15.16:0/3321347566 conn(0x7f4c5c0b1c90 msgr2=0x7f4c5c06a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:45.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.573+0000 7f4c60ea1700 1 -- 172.21.15.16:0/3321347566 shutdown_connections 2023-05-20T01:28:45.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:45.573+0000 7f4c60ea1700 1 -- 172.21.15.16:0/3321347566 wait complete. 2023-05-20T01:28:45.590 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-05-20T01:28:45.591 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-05-20T01:28:45.591 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2023-05-20T01:28:45.591 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-05-20T01:28:45.591 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-05-20T01:28:45.592 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-05-20T01:28:45.592 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-05-20T01:28:45.593 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-05-20T01:28:45.593 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-05-20T01:28:45.593 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:28:45.594 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2023-05-20T01:28:45.611 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-05-20T01:28:45.611 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-05-20T01:28:45.612 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2023-05-20T01:28:45.612 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-05-20T01:28:45.612 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2023-05-20T01:28:45.612 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-05-20T01:28:45.612 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-05-20T01:28:45.612 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2023-05-20T01:28:45.613 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-05-20T01:28:45.613 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2023-05-20T01:28:45.613 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-05-20T01:28:45.613 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2023-05-20T01:28:45.613 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-05-20T01:28:45.613 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 41708sec preferred_lft 41708sec 2023-05-20T01:28:45.613 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link 2023-05-20T01:28:45.613 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-05-20T01:28:45.614 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-05-20T01:28:45.614 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2023-05-20T01:28:45.614 INFO:teuthology.orchestra.run.smithi016.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-05-20T01:28:45.614 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0a:2e:0c:6b:5d:42 brd ff:ff:ff:ff:ff:ff 2023-05-20T01:28:45.614 INFO:teuthology.orchestra.run.smithi016.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-05-20T01:28:45.614 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-05-20T01:28:45.614 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::7c00:b2ff:fef4:946/64 scope link 2023-05-20T01:28:45.614 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-05-20T01:28:45.615 INFO:teuthology.orchestra.run.smithi016.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-05-20T01:28:45.615 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether d2:e4:70:a7:b9:4b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-05-20T01:28:45.615 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::d0e4:70ff:fea7:b94b/64 scope link 2023-05-20T01:28:45.615 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-05-20T01:28:45.615 INFO:teuthology.orchestra.run.smithi016.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-05-20T01:28:45.615 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0a:2e:0c:6b:5d:42 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-05-20T01:28:45.615 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::82e:cff:fe6b:5d42/64 scope link 2023-05-20T01:28:45.616 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-05-20T01:28:45.616 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:28:45.616 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-05-20T01:28:45.677 INFO:teuthology.orchestra.run.smithi016.stdout:1 2023-05-20T01:28:45.678 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:28:45.679 DEBUG:teuthology.orchestra.run.smithi016:> route 2023-05-20T01:28:45.738 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2023-05-20T01:28:45.738 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-05-20T01:28:45.739 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-05-20T01:28:45.739 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-05-20T01:28:45.739 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-05-20T01:28:45.739 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:28:45.740 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2023-05-20T01:28:45.740 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2023-05-20T01:28:45.741 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-05-20T01:28:45.741 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-05-20T01:28:45.741 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-05-20T01:28:45.742 DEBUG:teuthology.orchestra.run.smithi016:> ') 2023-05-20T01:28:45.845 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:28:45.846 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2023-05-20T01:28:45.863 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-05-20T01:28:45.863 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-05-20T01:28:45.864 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-05-20T01:28:45.864 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:28:45.865 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.0 up 2023-05-20T01:28:45.930 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-05-20T01:28:45.931 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-05-20T01:28:45.988 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-05-20T01:28:45.989 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-05-20T01:28:45.989 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-05-20T01:28:46.045 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-05-20T01:28:46.045 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:28:46.045 DEBUG:teuthology.orchestra.run.smithi016:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-05-20T01:28:46.132 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-05-20T01:28:46.132 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-05-20T01:28:46.162 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-05-20T01:28:46.162 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-05-20T01:28:46.163 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.132:3300,172.21.15.132:3301" 2023-05-20T01:28:46.163 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: trying mount with new device syntax: 0@c2245cc5-ecab-4087-a59b-bc81db901676.cephfs=/ 2023-05-20T01:28:46.163 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.16:3300/172.21.15.132:3300/172.21.15.132:3301" will pass to kernel 2023-05-20T01:28:46.163 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: v2 mount support check returned 2 2023-05-20T01:28:46.163 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: kernel does not support v2 syntax 2023-05-20T01:28:46.163 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: switching to using v1 address with old syntax 2023-05-20T01:28:46.195 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-05-20T01:28:46.196 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-05-20T01:28:46.196 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.132:3300,172.21.15.132:3301" 2023-05-20T01:28:46.196 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: trying mount with new device syntax: 0@c2245cc5-ecab-4087-a59b-bc81db901676.cephfs=/ 2023-05-20T01:28:46.196 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.16:3300/172.21.15.132:3300/172.21.15.132:3301" will pass to kernel 2023-05-20T01:28:46.196 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: v2 mount support check returned 2 2023-05-20T01:28:46.196 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: kernel does not support v2 syntax 2023-05-20T01:28:46.197 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: switching to using v1 address with old syntax 2023-05-20T01:28:46.197 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:6789,172.21.15.132:6789,172.21.15.132:6790" 2023-05-20T01:28:46.197 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.16:6789,172.21.15.132:6789,172.21.15.132:6790:/ 2023-05-20T01:28:46.197 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=c2245cc5-ecab-4087-a59b-bc81db901676" will pass to kernel 2023-05-20T01:28:46.203 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-05-20T01:28:46.204 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:28:46.204 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-05-20T01:28:46.240 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:28:46.240 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-05-20T01:28:46.312 INFO:teuthology.orchestra.run.smithi016.stdout:client9006 2023-05-20T01:28:46.313 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-05-20T01:28:46.313 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2023-05-20T01:28:46.370 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:28:46.371 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls subdir/) 2023-05-20T01:28:46.432 INFO:teuthology.orchestra.run.smithi016.stdout:0 2023-05-20T01:28:46.433 INFO:teuthology.orchestra.run.smithi016.stdout:1 2023-05-20T01:28:46.433 INFO:teuthology.orchestra.run.smithi016.stdout:10 2023-05-20T01:28:46.433 INFO:teuthology.orchestra.run.smithi016.stdout:11 2023-05-20T01:28:46.433 INFO:teuthology.orchestra.run.smithi016.stdout:12 2023-05-20T01:28:46.433 INFO:teuthology.orchestra.run.smithi016.stdout:13 2023-05-20T01:28:46.433 INFO:teuthology.orchestra.run.smithi016.stdout:14 2023-05-20T01:28:46.433 INFO:teuthology.orchestra.run.smithi016.stdout:15 2023-05-20T01:28:46.434 INFO:teuthology.orchestra.run.smithi016.stdout:16 2023-05-20T01:28:46.434 INFO:teuthology.orchestra.run.smithi016.stdout:17 2023-05-20T01:28:46.434 INFO:teuthology.orchestra.run.smithi016.stdout:18 2023-05-20T01:28:46.434 INFO:teuthology.orchestra.run.smithi016.stdout:19 2023-05-20T01:28:46.434 INFO:teuthology.orchestra.run.smithi016.stdout:2 2023-05-20T01:28:46.434 INFO:teuthology.orchestra.run.smithi016.stdout:20 2023-05-20T01:28:46.434 INFO:teuthology.orchestra.run.smithi016.stdout:22 2023-05-20T01:28:46.435 INFO:teuthology.orchestra.run.smithi016.stdout:23 2023-05-20T01:28:46.435 INFO:teuthology.orchestra.run.smithi016.stdout:24 2023-05-20T01:28:46.435 INFO:teuthology.orchestra.run.smithi016.stdout:25 2023-05-20T01:28:46.435 INFO:teuthology.orchestra.run.smithi016.stdout:26 2023-05-20T01:28:46.435 INFO:teuthology.orchestra.run.smithi016.stdout:27 2023-05-20T01:28:46.435 INFO:teuthology.orchestra.run.smithi016.stdout:28 2023-05-20T01:28:46.435 INFO:teuthology.orchestra.run.smithi016.stdout:29 2023-05-20T01:28:46.435 INFO:teuthology.orchestra.run.smithi016.stdout:3 2023-05-20T01:28:46.436 INFO:teuthology.orchestra.run.smithi016.stdout:30 2023-05-20T01:28:46.436 INFO:teuthology.orchestra.run.smithi016.stdout:31 2023-05-20T01:28:46.436 INFO:teuthology.orchestra.run.smithi016.stdout:32 2023-05-20T01:28:46.436 INFO:teuthology.orchestra.run.smithi016.stdout:33 2023-05-20T01:28:46.436 INFO:teuthology.orchestra.run.smithi016.stdout:34 2023-05-20T01:28:46.436 INFO:teuthology.orchestra.run.smithi016.stdout:35 2023-05-20T01:28:46.436 INFO:teuthology.orchestra.run.smithi016.stdout:36 2023-05-20T01:28:46.436 INFO:teuthology.orchestra.run.smithi016.stdout:37 2023-05-20T01:28:46.437 INFO:teuthology.orchestra.run.smithi016.stdout:38 2023-05-20T01:28:46.437 INFO:teuthology.orchestra.run.smithi016.stdout:39 2023-05-20T01:28:46.437 INFO:teuthology.orchestra.run.smithi016.stdout:4 2023-05-20T01:28:46.437 INFO:teuthology.orchestra.run.smithi016.stdout:40 2023-05-20T01:28:46.437 INFO:teuthology.orchestra.run.smithi016.stdout:41 2023-05-20T01:28:46.437 INFO:teuthology.orchestra.run.smithi016.stdout:42 2023-05-20T01:28:46.437 INFO:teuthology.orchestra.run.smithi016.stdout:43 2023-05-20T01:28:46.438 INFO:teuthology.orchestra.run.smithi016.stdout:44 2023-05-20T01:28:46.438 INFO:teuthology.orchestra.run.smithi016.stdout:45 2023-05-20T01:28:46.438 INFO:teuthology.orchestra.run.smithi016.stdout:46 2023-05-20T01:28:46.438 INFO:teuthology.orchestra.run.smithi016.stdout:47 2023-05-20T01:28:46.438 INFO:teuthology.orchestra.run.smithi016.stdout:48 2023-05-20T01:28:46.438 INFO:teuthology.orchestra.run.smithi016.stdout:49 2023-05-20T01:28:46.438 INFO:teuthology.orchestra.run.smithi016.stdout:5 2023-05-20T01:28:46.438 INFO:teuthology.orchestra.run.smithi016.stdout:50 2023-05-20T01:28:46.439 INFO:teuthology.orchestra.run.smithi016.stdout:51 2023-05-20T01:28:46.439 INFO:teuthology.orchestra.run.smithi016.stdout:52 2023-05-20T01:28:46.439 INFO:teuthology.orchestra.run.smithi016.stdout:53 2023-05-20T01:28:46.439 INFO:teuthology.orchestra.run.smithi016.stdout:54 2023-05-20T01:28:46.439 INFO:teuthology.orchestra.run.smithi016.stdout:55 2023-05-20T01:28:46.439 INFO:teuthology.orchestra.run.smithi016.stdout:56 2023-05-20T01:28:46.439 INFO:teuthology.orchestra.run.smithi016.stdout:57 2023-05-20T01:28:46.439 INFO:teuthology.orchestra.run.smithi016.stdout:58 2023-05-20T01:28:46.440 INFO:teuthology.orchestra.run.smithi016.stdout:59 2023-05-20T01:28:46.440 INFO:teuthology.orchestra.run.smithi016.stdout:6 2023-05-20T01:28:46.440 INFO:teuthology.orchestra.run.smithi016.stdout:60 2023-05-20T01:28:46.440 INFO:teuthology.orchestra.run.smithi016.stdout:61 2023-05-20T01:28:46.440 INFO:teuthology.orchestra.run.smithi016.stdout:62 2023-05-20T01:28:46.440 INFO:teuthology.orchestra.run.smithi016.stdout:63 2023-05-20T01:28:46.440 INFO:teuthology.orchestra.run.smithi016.stdout:64 2023-05-20T01:28:46.440 INFO:teuthology.orchestra.run.smithi016.stdout:65 2023-05-20T01:28:46.441 INFO:teuthology.orchestra.run.smithi016.stdout:66 2023-05-20T01:28:46.441 INFO:teuthology.orchestra.run.smithi016.stdout:67 2023-05-20T01:28:46.441 INFO:teuthology.orchestra.run.smithi016.stdout:68 2023-05-20T01:28:46.441 INFO:teuthology.orchestra.run.smithi016.stdout:69 2023-05-20T01:28:46.441 INFO:teuthology.orchestra.run.smithi016.stdout:7 2023-05-20T01:28:46.441 INFO:teuthology.orchestra.run.smithi016.stdout:70 2023-05-20T01:28:46.441 INFO:teuthology.orchestra.run.smithi016.stdout:71 2023-05-20T01:28:46.441 INFO:teuthology.orchestra.run.smithi016.stdout:72 2023-05-20T01:28:46.442 INFO:teuthology.orchestra.run.smithi016.stdout:73 2023-05-20T01:28:46.442 INFO:teuthology.orchestra.run.smithi016.stdout:74 2023-05-20T01:28:46.442 INFO:teuthology.orchestra.run.smithi016.stdout:75 2023-05-20T01:28:46.442 INFO:teuthology.orchestra.run.smithi016.stdout:76 2023-05-20T01:28:46.442 INFO:teuthology.orchestra.run.smithi016.stdout:77 2023-05-20T01:28:46.442 INFO:teuthology.orchestra.run.smithi016.stdout:78 2023-05-20T01:28:46.442 INFO:teuthology.orchestra.run.smithi016.stdout:79 2023-05-20T01:28:46.443 INFO:teuthology.orchestra.run.smithi016.stdout:8 2023-05-20T01:28:46.443 INFO:teuthology.orchestra.run.smithi016.stdout:80 2023-05-20T01:28:46.443 INFO:teuthology.orchestra.run.smithi016.stdout:81 2023-05-20T01:28:46.443 INFO:teuthology.orchestra.run.smithi016.stdout:82 2023-05-20T01:28:46.443 INFO:teuthology.orchestra.run.smithi016.stdout:83 2023-05-20T01:28:46.444 INFO:teuthology.orchestra.run.smithi016.stdout:84 2023-05-20T01:28:46.444 INFO:teuthology.orchestra.run.smithi016.stdout:85 2023-05-20T01:28:46.444 INFO:teuthology.orchestra.run.smithi016.stdout:86 2023-05-20T01:28:46.444 INFO:teuthology.orchestra.run.smithi016.stdout:87 2023-05-20T01:28:46.444 INFO:teuthology.orchestra.run.smithi016.stdout:88 2023-05-20T01:28:46.444 INFO:teuthology.orchestra.run.smithi016.stdout:89 2023-05-20T01:28:46.444 INFO:teuthology.orchestra.run.smithi016.stdout:9 2023-05-20T01:28:46.445 INFO:teuthology.orchestra.run.smithi016.stdout:90 2023-05-20T01:28:46.445 INFO:teuthology.orchestra.run.smithi016.stdout:91 2023-05-20T01:28:46.445 INFO:teuthology.orchestra.run.smithi016.stdout:92 2023-05-20T01:28:46.445 INFO:teuthology.orchestra.run.smithi016.stdout:93 2023-05-20T01:28:46.445 INFO:teuthology.orchestra.run.smithi016.stdout:94 2023-05-20T01:28:46.445 INFO:teuthology.orchestra.run.smithi016.stdout:95 2023-05-20T01:28:46.445 INFO:teuthology.orchestra.run.smithi016.stdout:96 2023-05-20T01:28:46.445 INFO:teuthology.orchestra.run.smithi016.stdout:97 2023-05-20T01:28:46.446 INFO:teuthology.orchestra.run.smithi016.stdout:98 2023-05-20T01:28:46.446 INFO:teuthology.orchestra.run.smithi016.stdout:99 2023-05-20T01:28:46.446 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-05-20T01:28:46.446 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2023-05-20T01:28:46.490 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-05-20T01:28:46.491 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2023-05-20T01:28:46.549 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-05-20T01:28:46.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:46.708+0000 7f1c5fea6700 1 Processor -- start 2023-05-20T01:28:46.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:46.709+0000 7f1c5fea6700 1 -- start start 2023-05-20T01:28:46.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:46.709+0000 7f1c5fea6700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f1c580d0470 0x7f1c580c4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:46.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:46.709+0000 7f1c5fea6700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c580d0c10 con 0x7f1c580cdc40 2023-05-20T01:28:46.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:46.709+0000 7f1c5fea6700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c580d0ef0 con 0x7f1c580cd710 2023-05-20T01:28:46.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:46.709+0000 7f1c5fea6700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c58070e70 con 0x7f1c580d0470 2023-05-20T01:28:46.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:46.710+0000 7f1c5dc42700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f1c580cdc40 0x7f1c580cf8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:52110/0 (socket says 172.21.15.16:52110) 2023-05-20T01:28:46.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:46.710+0000 7f1c5dc42700 1 -- 172.21.15.16:0/4026171074 learned_addr learned my addr 172.21.15.16:0/4026171074 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:28:46.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:46.710+0000 7f1c5e443700 1 --2- 172.21.15.16:0/4026171074 >> v2:172.21.15.132:3300/0 conn(0x7f1c580d0470 0x7f1c580c4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:46.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:46.710+0000 7f1c5cc40700 1 -- 172.21.15.16:0/4026171074 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3853478998 0 0) 0x7f1c580d0c10 con 0x7f1c580cdc40 2023-05-20T01:28:46.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:46.710+0000 7f1c5cc40700 1 -- 172.21.15.16:0/4026171074 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1c440035b0 con 0x7f1c580cdc40 2023-05-20T01:28:46.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:46.710+0000 7f1c5e443700 1 -- 172.21.15.16:0/4026171074 >> v1:172.21.15.132:6789/0 conn(0x7f1c580cd710 legacy=0x7f1c580ce010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:46.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:46.710+0000 7f1c5e443700 1 -- 172.21.15.16:0/4026171074 >> v1:172.21.15.16:6789/0 conn(0x7f1c580cdc40 legacy=0x7f1c580cf8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:46.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:46.710+0000 7f1c5e443700 1 -- 172.21.15.16:0/4026171074 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c580cee00 con 0x7f1c580d0470 2023-05-20T01:28:46.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:46.710+0000 7f1c5e443700 1 --2- 172.21.15.16:0/4026171074 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c580d0470 0x7f1c580c4d80 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f1c50009d90 tx=0x7f1c5000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c9f2b3d19240bd3 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:46.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:46.710+0000 7f1c5cc40700 1 -- 172.21.15.16:0/4026171074 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c5000e070 con 0x7f1c580d0470 2023-05-20T01:28:46.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:46.710+0000 7f1c5cc40700 1 -- 172.21.15.16:0/4026171074 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c5000bb20 con 0x7f1c580d0470 2023-05-20T01:28:46.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:46.710+0000 7f1c5cc40700 1 -- 172.21.15.16:0/4026171074 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c5000be90 con 0x7f1c580d0470 2023-05-20T01:28:46.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:46.711+0000 7f1c5fea6700 1 -- 172.21.15.16:0/4026171074 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c580d0470 msgr2=0x7f1c580c4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:46.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:46.711+0000 7f1c5fea6700 1 --2- 172.21.15.16:0/4026171074 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c580d0470 0x7f1c580c4d80 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f1c50009d90 tx=0x7f1c5000b330 comp rx=0 tx=0).stop 2023-05-20T01:28:46.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:46.711+0000 7f1c5fea6700 1 -- 172.21.15.16:0/4026171074 shutdown_connections 2023-05-20T01:28:46.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:46.711+0000 7f1c5fea6700 1 --2- 172.21.15.16:0/4026171074 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c580d0470 0x7f1c580c4d80 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:46.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:46.711+0000 7f1c5fea6700 1 -- 172.21.15.16:0/4026171074 >> 172.21.15.16:0/4026171074 conn(0x7f1c5806e4c0 msgr2=0x7f1c5806e8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:46.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:46.711+0000 7f1c5fea6700 1 -- 172.21.15.16:0/4026171074 shutdown_connections 2023-05-20T01:28:46.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:46.711+0000 7f1c5fea6700 1 -- 172.21.15.16:0/4026171074 wait complete. 2023-05-20T01:28:46.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:46.712+0000 7f1c5fea6700 1 Processor -- start 2023-05-20T01:28:46.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:46.712+0000 7f1c5fea6700 1 -- start start 2023-05-20T01:28:46.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:46.713+0000 7f1c5fea6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c580cd710 0x7f1c581e6a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:46.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:46.713+0000 7f1c5fea6700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c580cdc40 0x7f1c581dfe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:46.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:46.713+0000 7f1c5fea6700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c580d0470 0x7f1c581e0350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:46.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:46.713+0000 7f1c5fea6700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c581e22a0 con 0x7f1c580cd710 2023-05-20T01:28:46.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:46.713+0000 7f1c5fea6700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c581e0980 con 0x7f1c580cdc40 2023-05-20T01:28:46.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:46.713+0000 7f1c5fea6700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c581e0c60 con 0x7f1c580d0470 2023-05-20T01:28:46.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:46.713+0000 7f1c5dc42700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c580cd710 0x7f1c581e6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:46.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:46.713+0000 7f1c5dc42700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c580cd710 0x7f1c581e6a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40588/0 (socket says 172.21.15.16:40588) 2023-05-20T01:28:46.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:46.713+0000 7f1c5dc42700 1 -- 172.21.15.16:0/3253070727 learned_addr learned my addr 172.21.15.16:0/3253070727 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:46.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:46.713+0000 7f1c5e443700 1 --2- 172.21.15.16:0/3253070727 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c580d0470 0x7f1c581e0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:46.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:46.713+0000 7f1c5d441700 1 --2- 172.21.15.16:0/3253070727 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c580cdc40 0x7f1c581dfe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:46.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:46.713+0000 7f1c5dc42700 1 -- 172.21.15.16:0/3253070727 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c580d0470 msgr2=0x7f1c581e0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:46.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:46.713+0000 7f1c5dc42700 1 --2- 172.21.15.16:0/3253070727 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c580d0470 0x7f1c581e0350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:46.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:46.713+0000 7f1c5dc42700 1 -- 172.21.15.16:0/3253070727 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c580cdc40 msgr2=0x7f1c581dfe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:46.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:46.713+0000 7f1c5dc42700 1 --2- 172.21.15.16:0/3253070727 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c580cdc40 0x7f1c581dfe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:46.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:46.713+0000 7f1c5dc42700 1 -- 172.21.15.16:0/3253070727 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c50009a70 con 0x7f1c580cd710 2023-05-20T01:28:46.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:46.714+0000 7f1c5dc42700 1 --2- 172.21.15.16:0/3253070727 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c580cd710 0x7f1c581e6a90 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f1c54001eb0 tx=0x7f1c5400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:46.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:46.714+0000 7f1c4a7fc700 1 -- 172.21.15.16:0/3253070727 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c54005d70 con 0x7f1c580cd710 2023-05-20T01:28:46.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:46.714+0000 7f1c4a7fc700 1 -- 172.21.15.16:0/3253070727 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c540070d0 con 0x7f1c580cd710 2023-05-20T01:28:46.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:46.714+0000 7f1c5fea6700 1 -- 172.21.15.16:0/3253070727 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c580cf3c0 con 0x7f1c580cd710 2023-05-20T01:28:46.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:46.714+0000 7f1c4a7fc700 1 -- 172.21.15.16:0/3253070727 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c54007460 con 0x7f1c580cd710 2023-05-20T01:28:46.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:46.714+0000 7f1c5fea6700 1 -- 172.21.15.16:0/3253070727 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c580cf0e0 con 0x7f1c580cd710 2023-05-20T01:28:46.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:46.715+0000 7f1c5fea6700 1 -- 172.21.15.16:0/3253070727 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c580d08d0 con 0x7f1c580cd710 2023-05-20T01:28:46.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:46.715+0000 7f1c4a7fc700 1 -- 172.21.15.16:0/3253070727 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1c540075c0 con 0x7f1c580cd710 2023-05-20T01:28:46.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:46.716+0000 7f1c4a7fc700 1 --2- 172.21.15.16:0/3253070727 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f1c3c06ace0 0x7f1c3c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:46.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:46.716+0000 7f1c4a7fc700 1 -- 172.21.15.16:0/3253070727 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6468+0+0 (secure 0 0 0) 0x7f1c54088be0 con 0x7f1c580cd710 2023-05-20T01:28:46.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:46.716+0000 7f1c5d441700 1 --2- 172.21.15.16:0/3253070727 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f1c3c06ace0 0x7f1c3c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:46.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:46.716+0000 7f1c5d441700 1 --2- 172.21.15.16:0/3253070727 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f1c3c06ace0 0x7f1c3c06d190 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f1c5806d720 tx=0x7f1c4c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:46.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:46.720+0000 7f1c4a7fc700 1 -- 172.21.15.16:0/3253070727 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f1c54053af0 con 0x7f1c580cd710 2023-05-20T01:28:46.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:46.916+0000 7f1c5fea6700 1 -- 172.21.15.16:0/3253070727 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1c5804e8c0 con 0x7f1c580cd710 2023-05-20T01:28:46.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:46.917+0000 7f1c4a7fc700 1 -- 172.21.15.16:0/3253070727 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v137) v1 ==== 74+0+18601 (secure 0 0 0) 0x7f1c54057110 con 0x7f1c580cd710 2023-05-20T01:28:46.922 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:28:46.922 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":137,"fsid":"c2245cc5-ecab-4087-a59b-bc81db901676","created":"2023-05-20T01:16:09.549230+0000","modified":"2023-05-20T01:28:39.430665+0000","last_up_change":"2023-05-20T01:16:19.788624+0000","last_in_change":"2023-05-20T01:16:12.092160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-05-20T01:16:20.934366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-05-20T01:16:21.372294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-05-20T01:28:25.274827+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"135","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-05-20T01:28:26.067827+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"317a83cd-19c8-4958-a301-6087ef342289","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6825","nonce":3923634179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6827","nonce":3923634179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6831","nonce":3923634179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3923634179},{"type":"v1","addr":"172.21.15.16:6829","nonce":3923634179}]},"public_addr":"172.21.15.16:6825/3923634179","cluster_addr":"172.21.15.16:6827/3923634179","heartbeat_back_addr":"172.21.15.16:6831/3923634179","heartbeat_front_addr":"172.21.15.16:6829/3923634179","state":["exists","up"]},{"osd":1,"uuid":"5581046f-8219-4089-8d7d-b7846399505e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6809","nonce":3860264220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6811","nonce":3860264220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6815","nonce":3860264220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3860264220},{"type":"v1","addr":"172.21.15.16:6813","nonce":3860264220}]},"public_addr":"172.21.15.16:6809/3860264220","cluster_addr":"172.21.15.16:6811/3860264220","heartbeat_back_addr":"172.21.15.16:6815/3860264220","heartbeat_front_addr":"172.21.15.16:6813/3860264220","state":["exists","up"]},{"osd":2,"uuid":"072e9e66-4133-478e-8d4a-64d46c2963f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6817","nonce":293187030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6819","nonce":293187030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6823","nonce":293187030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":293187030},{"type":"v1","addr":"172.21.15.16:6821","nonce":293187030}]},"public_addr":"172.21.15.16:6817/293187030","cluster_addr":"172.21.15.16:6819/293187030","heartbeat_back_addr":"172.21.15.16:6823/293187030","heartbeat_front_addr":"172.21.15.16:6821/293187030","state":["exists","up"]},{"osd":3,"uuid":"11e1bba6-00f8-48dd-9028-d8d716b7f145","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6801","nonce":3669781890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6803","nonce":3669781890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6807","nonce":3669781890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3669781890},{"type":"v1","addr":"172.21.15.16:6805","nonce":3669781890}]},"public_addr":"172.21.15.16:6801/3669781890","cluster_addr":"172.21.15.16:6803/3669781890","heartbeat_back_addr":"172.21.15.16:6807/3669781890","heartbeat_front_addr":"172.21.15.16:6805/3669781890","state":["exists","up"]},{"osd":4,"uuid":"b8909c3a-cf70-46cb-8d0b-009c707cf9d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6819","nonce":506317223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6823","nonce":506317223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6829","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6831","nonce":506317223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":506317223},{"type":"v1","addr":"172.21.15.132:6827","nonce":506317223}]},"public_addr":"172.21.15.132:6819/506317223","cluster_addr":"172.21.15.132:6823/506317223","heartbeat_back_addr":"172.21.15.132:6831/506317223","heartbeat_front_addr":"172.21.15.132:6827/506317223","state":["exists","up"]},{"osd":5,"uuid":"b1818304-4583-47bf-a6ec-0bfbb5166233","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6818","nonce":1844689995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6822","nonce":1844689995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6830","nonce":1844689995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":1844689995},{"type":"v1","addr":"172.21.15.132:6826","nonce":1844689995}]},"public_addr":"172.21.15.132:6818/1844689995","cluster_addr":"172.21.15.132:6822/1844689995","heartbeat_back_addr":"172.21.15.132:6830/1844689995","heartbeat_front_addr":"172.21.15.132:6826/1844689995","state":["exists","up"]},{"osd":6,"uuid":"d37f4691-eed1-4a93-8818-18354f03f4a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6801","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6802","nonce":727581123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6805","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6807","nonce":727581123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6813","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6815","nonce":727581123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":727581123},{"type":"v1","addr":"172.21.15.132:6811","nonce":727581123}]},"public_addr":"172.21.15.132:6802/727581123","cluster_addr":"172.21.15.132:6807/727581123","heartbeat_back_addr":"172.21.15.132:6815/727581123","heartbeat_front_addr":"172.21.15.132:6811/727581123","state":["exists","up"]},{"osd":7,"uuid":"4f712f62-5d1d-4816-8042-03209144f8b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6803","nonce":466001149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6806","nonce":466001149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6814","nonce":466001149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":466001149},{"type":"v1","addr":"172.21.15.132:6810","nonce":466001149}]},"public_addr":"172.21.15.132:6803/466001149","cluster_addr":"172.21.15.132:6806/466001149","heartbeat_back_addr":"172.21.15.132:6814/466001149","heartbeat_front_addr":"172.21.15.132:6810/466001149","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.338863+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.278680+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305694+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.246835+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.276237+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.304441+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.305421+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-05-20T01:16:17.267050+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6837/2295684907":"2023-05-21T01:28:39.430636+0000","172.21.15.16:6835/2295684907":"2023-05-21T01:28:39.430636+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-05-20T01:28:46.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:46.919+0000 7f1c5fea6700 1 -- 172.21.15.16:0/3253070727 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f1c3c06ace0 msgr2=0x7f1c3c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:46.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:46.919+0000 7f1c5fea6700 1 --2- 172.21.15.16:0/3253070727 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f1c3c06ace0 0x7f1c3c06d190 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f1c5806d720 tx=0x7f1c4c009040 comp rx=0 tx=0).stop 2023-05-20T01:28:46.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:46.919+0000 7f1c5fea6700 1 -- 172.21.15.16:0/3253070727 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c580cd710 msgr2=0x7f1c581e6a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:46.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:46.919+0000 7f1c5fea6700 1 --2- 172.21.15.16:0/3253070727 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c580cd710 0x7f1c581e6a90 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f1c54001eb0 tx=0x7f1c5400c860 comp rx=0 tx=0).stop 2023-05-20T01:28:46.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:46.919+0000 7f1c5fea6700 1 -- 172.21.15.16:0/3253070727 shutdown_connections 2023-05-20T01:28:46.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:46.919+0000 7f1c5fea6700 1 --2- 172.21.15.16:0/3253070727 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f1c3c06ace0 0x7f1c3c06d190 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:46.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:46.919+0000 7f1c5fea6700 1 --2- 172.21.15.16:0/3253070727 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c580d0470 0x7f1c581e0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:46.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:46.919+0000 7f1c5fea6700 1 --2- 172.21.15.16:0/3253070727 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c580cd710 0x7f1c581e6a90 unknown :-1 s=CLOSED pgs=1696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:46.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:46.919+0000 7f1c5fea6700 1 --2- 172.21.15.16:0/3253070727 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c580cdc40 0x7f1c581dfe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:46.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:46.919+0000 7f1c5fea6700 1 -- 172.21.15.16:0/3253070727 >> 172.21.15.16:0/3253070727 conn(0x7f1c5806e4c0 msgr2=0x7f1c580b5c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:46.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:46.919+0000 7f1c5fea6700 1 -- 172.21.15.16:0/3253070727 shutdown_connections 2023-05-20T01:28:46.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:46.919+0000 7f1c5fea6700 1 -- 172.21.15.16:0/3253070727 wait complete. 2023-05-20T01:28:46.936 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-05-20T01:16:20.934366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-05-20T01:16:21.372294+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_metadata', 'create_time': '2023-05-20T01:28:25.274827+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '135', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 19, 'pool_name': 'cephfs_data', 'create_time': '2023-05-20T01:28:26.067827+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '136', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-05-20T01:28:46.938 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-05-20T01:28:47.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.062+0000 7f41362af700 1 Processor -- start 2023-05-20T01:28:47.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.062+0000 7f41362af700 1 -- start start 2023-05-20T01:28:47.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.063+0000 7f41362af700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f4130070cf0 0x7f4130160770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:47.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.063+0000 7f41362af700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41300708a0 con 0x7f41300b2d00 2023-05-20T01:28:47.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.063+0000 7f41362af700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41300d0720 con 0x7f413006ff30 2023-05-20T01:28:47.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.063+0000 7f41362af700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f413006d5a0 con 0x7f4130070cf0 2023-05-20T01:28:47.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.063+0000 7f412ffff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f41300b2d00 0x7f41300b30d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:52122/0 (socket says 172.21.15.16:52122) 2023-05-20T01:28:47.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.063+0000 7f412ffff700 1 -- 172.21.15.16:0/3524850025 learned_addr learned my addr 172.21.15.16:0/3524850025 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:28:47.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.064+0000 7f412f7fe700 1 --2- 172.21.15.16:0/3524850025 >> v2:172.21.15.132:3300/0 conn(0x7f4130070cf0 0x7f4130160770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:47.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.064+0000 7f412effd700 1 -- 172.21.15.16:0/3524850025 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2288870177 0 0) 0x7f4120001110 con 0x7f41300b2d00 2023-05-20T01:28:47.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.064+0000 7f412effd700 1 -- 172.21.15.16:0/3524850025 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f411c0035b0 con 0x7f41300b2d00 2023-05-20T01:28:47.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.064+0000 7f412effd700 1 -- 172.21.15.16:0/3524850025 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1733739148 0 0) 0x7f411c0035b0 con 0x7f41300b2d00 2023-05-20T01:28:47.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.064+0000 7f412effd700 1 -- 172.21.15.16:0/3524850025 >> v2:172.21.15.132:3300/0 conn(0x7f4130070cf0 msgr2=0x7f4130160770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:47.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.064+0000 7f412effd700 1 --2- 172.21.15.16:0/3524850025 >> v2:172.21.15.132:3300/0 conn(0x7f4130070cf0 0x7f4130160770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:47.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.064+0000 7f412effd700 1 -- 172.21.15.16:0/3524850025 >> v1:172.21.15.132:6789/0 conn(0x7f413006ff30 legacy=0x7f413006eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:47.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.064+0000 7f412effd700 1 -- 172.21.15.16:0/3524850025 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41300d0a00 con 0x7f41300b2d00 2023-05-20T01:28:47.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.064+0000 7f412effd700 1 -- 172.21.15.16:0/3524850025 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7f4120001800 con 0x7f41300b2d00 2023-05-20T01:28:47.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.064+0000 7f412f7fe700 1 --2- 172.21.15.16:0/3524850025 >> v2:172.21.15.132:3300/0 conn(0x7f4130070cf0 0x7f4130160770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-05-20T01:28:47.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.064+0000 7f412effd700 1 -- 172.21.15.16:0/3524850025 >> v1:172.21.15.16:6789/0 conn(0x7f41300b2d00 legacy=0x7f41300b30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:47.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.064+0000 7f412effd700 1 --2- 172.21.15.16:0/3524850025 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f411c003a00 0x7f411c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:47.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.064+0000 7f412effd700 1 --2- 172.21.15.16:0/3524850025 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f411c0043f0 0x7f411c0048b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:47.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.064+0000 7f412effd700 1 --2- 172.21.15.16:0/3524850025 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f411c0050b0 0x7f411c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:47.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.064+0000 7f412effd700 1 -- 172.21.15.16:0/3524850025 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f413006d5a0 con 0x7f411c0043f0 2023-05-20T01:28:47.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.064+0000 7f412effd700 1 -- 172.21.15.16:0/3524850025 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f411c00c000 con 0x7f411c0050b0 2023-05-20T01:28:47.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.064+0000 7f412effd700 1 -- 172.21.15.16:0/3524850025 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f411c00c1c0 con 0x7f411c003a00 2023-05-20T01:28:47.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.064+0000 7f412f7fe700 1 --2- 172.21.15.16:0/3524850025 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f411c0043f0 0x7f411c0048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:47.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.064+0000 7f412ffff700 1 --2- 172.21.15.16:0/3524850025 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f411c003a00 0x7f411c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:47.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.064+0000 7f413484c700 1 --2- 172.21.15.16:0/3524850025 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f411c0050b0 0x7f411c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:47.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.064+0000 7f412f7fe700 1 -- 172.21.15.16:0/3524850025 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f411c003a00 msgr2=0x7f411c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:47.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.064+0000 7f412f7fe700 1 --2- 172.21.15.16:0/3524850025 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f411c003a00 0x7f411c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:47.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.064+0000 7f412f7fe700 1 -- 172.21.15.16:0/3524850025 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f411c0050b0 msgr2=0x7f411c00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:47.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.064+0000 7f412f7fe700 1 --2- 172.21.15.16:0/3524850025 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f411c0050b0 0x7f411c00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:47.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.064+0000 7f412f7fe700 1 -- 172.21.15.16:0/3524850025 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f411c0035b0 con 0x7f411c0043f0 2023-05-20T01:28:47.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.064+0000 7f413484c700 1 -- 172.21.15.16:0/3524850025 reap_dead start 2023-05-20T01:28:47.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.065+0000 7f412f7fe700 1 --2- 172.21.15.16:0/3524850025 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f411c0043f0 0x7f411c0048b0 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f4118009d30 tx=0x7f411c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f817abba53bbe407 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:47.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.065+0000 7f412effd700 1 -- 172.21.15.16:0/3524850025 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4118009e80 con 0x7f411c0043f0 2023-05-20T01:28:47.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.065+0000 7f412effd700 1 -- 172.21.15.16:0/3524850025 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f411800e9a0 con 0x7f411c0043f0 2023-05-20T01:28:47.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.065+0000 7f412effd700 1 -- 172.21.15.16:0/3524850025 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f411800ed10 con 0x7f411c0043f0 2023-05-20T01:28:47.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.065+0000 7f41362af700 1 -- 172.21.15.16:0/3524850025 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f411c0043f0 msgr2=0x7f411c0048b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:47.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.065+0000 7f41362af700 1 --2- 172.21.15.16:0/3524850025 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f411c0043f0 0x7f411c0048b0 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f4118009d30 tx=0x7f411c005880 comp rx=0 tx=0).stop 2023-05-20T01:28:47.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.065+0000 7f41362af700 1 -- 172.21.15.16:0/3524850025 shutdown_connections 2023-05-20T01:28:47.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.065+0000 7f41362af700 1 --2- 172.21.15.16:0/3524850025 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f411c0043f0 0x7f411c0048b0 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:47.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.065+0000 7f41362af700 1 -- 172.21.15.16:0/3524850025 >> 172.21.15.16:0/3524850025 conn(0x7f41300cdd00 msgr2=0x7f41300ce100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:47.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.065+0000 7f41362af700 1 -- 172.21.15.16:0/3524850025 shutdown_connections 2023-05-20T01:28:47.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.065+0000 7f41362af700 1 -- 172.21.15.16:0/3524850025 wait complete. 2023-05-20T01:28:47.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.067+0000 7f41362af700 1 Processor -- start 2023-05-20T01:28:47.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.067+0000 7f41362af700 1 -- start start 2023-05-20T01:28:47.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.067+0000 7f41362af700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f411c0043f0 0x7f4130151a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:47.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.067+0000 7f41362af700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41300d1150 0x7f41300d15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:47.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.067+0000 7f41362af700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f41300b5da0 0x7f41300b6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:47.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.068+0000 7f41362af700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41301f38f0 con 0x7f41300d1150 2023-05-20T01:28:47.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.068+0000 7f41362af700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41300b6ad0 con 0x7f41300b5da0 2023-05-20T01:28:47.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.068+0000 7f41362af700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41300d1b00 con 0x7f411c0043f0 2023-05-20T01:28:47.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.068+0000 7f412f7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41300d1150 0x7f41300d15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:47.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.068+0000 7f412f7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41300d1150 0x7f41300d15c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40608/0 (socket says 172.21.15.16:40608) 2023-05-20T01:28:47.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.068+0000 7f412f7fe700 1 -- 172.21.15.16:0/2830566598 learned_addr learned my addr 172.21.15.16:0/2830566598 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:47.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.068+0000 7f412ffff700 1 --2- 172.21.15.16:0/2830566598 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f411c0043f0 0x7f4130151a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:47.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.068+0000 7f413484c700 1 --2- 172.21.15.16:0/2830566598 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f41300b5da0 0x7f41300b6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:47.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.068+0000 7f412f7fe700 1 -- 172.21.15.16:0/2830566598 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f411c0043f0 msgr2=0x7f4130151a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:47.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.068+0000 7f412f7fe700 1 --2- 172.21.15.16:0/2830566598 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f411c0043f0 0x7f4130151a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:47.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.068+0000 7f412f7fe700 1 -- 172.21.15.16:0/2830566598 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f41300b5da0 msgr2=0x7f41300b6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:47.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.068+0000 7f412f7fe700 1 --2- 172.21.15.16:0/2830566598 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f41300b5da0 0x7f41300b6240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:47.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.068+0000 7f412f7fe700 1 -- 172.21.15.16:0/2830566598 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41300d0fc0 con 0x7f41300d1150 2023-05-20T01:28:47.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.068+0000 7f412f7fe700 1 --2- 172.21.15.16:0/2830566598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41300d1150 0x7f41300d15c0 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f41180044c0 tx=0x7f41180047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:47.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.068+0000 7f412cff9700 1 -- 172.21.15.16:0/2830566598 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4118003ab0 con 0x7f41300d1150 2023-05-20T01:28:47.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.068+0000 7f412cff9700 1 -- 172.21.15.16:0/2830566598 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4118003c10 con 0x7f41300d1150 2023-05-20T01:28:47.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.068+0000 7f41362af700 1 -- 172.21.15.16:0/2830566598 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41300d0ce0 con 0x7f41300d1150 2023-05-20T01:28:47.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.068+0000 7f412cff9700 1 -- 172.21.15.16:0/2830566598 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4118009e80 con 0x7f41300d1150 2023-05-20T01:28:47.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.068+0000 7f41362af700 1 -- 172.21.15.16:0/2830566598 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41300703b0 con 0x7f41300d1150 2023-05-20T01:28:47.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.069+0000 7f41362af700 1 -- 172.21.15.16:0/2830566598 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41301f2910 con 0x7f41300d1150 2023-05-20T01:28:47.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.070+0000 7f412cff9700 1 -- 172.21.15.16:0/2830566598 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4118013470 con 0x7f41300d1150 2023-05-20T01:28:47.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.070+0000 7f412cff9700 1 --2- 172.21.15.16:0/2830566598 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f411006ace0 0x7f411006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:47.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.070+0000 7f412cff9700 1 -- 172.21.15.16:0/2830566598 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6468+0+0 (secure 0 0 0) 0x7f411808aea0 con 0x7f41300d1150 2023-05-20T01:28:47.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.070+0000 7f412ffff700 1 --2- 172.21.15.16:0/2830566598 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f411006ace0 0x7f411006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:47.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.071+0000 7f412ffff700 1 --2- 172.21.15.16:0/2830566598 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f411006ace0 0x7f411006d190 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f41200018a0 tx=0x7f4120001170 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:47.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.075+0000 7f412cff9700 1 -- 172.21.15.16:0/2830566598 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f4118055e40 con 0x7f41300d1150 2023-05-20T01:28:47.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.272+0000 7f41362af700 1 -- 172.21.15.16:0/2830566598 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f413004e8c0 con 0x7f41300d1150 2023-05-20T01:28:47.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.272+0000 7f412cff9700 1 -- 172.21.15.16:0/2830566598 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v137) v1 ==== 92+0+10 (secure 0 0 0) 0x7f4118059460 con 0x7f41300d1150 2023-05-20T01:28:47.277 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 8 2023-05-20T01:28:47.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.274+0000 7f41362af700 1 -- 172.21.15.16:0/2830566598 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f411006ace0 msgr2=0x7f411006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:47.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.274+0000 7f41362af700 1 --2- 172.21.15.16:0/2830566598 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f411006ace0 0x7f411006d190 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f41200018a0 tx=0x7f4120001170 comp rx=0 tx=0).stop 2023-05-20T01:28:47.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.274+0000 7f41362af700 1 -- 172.21.15.16:0/2830566598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41300d1150 msgr2=0x7f41300d15c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:47.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.274+0000 7f41362af700 1 --2- 172.21.15.16:0/2830566598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41300d1150 0x7f41300d15c0 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f41180044c0 tx=0x7f41180047c0 comp rx=0 tx=0).stop 2023-05-20T01:28:47.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.274+0000 7f41362af700 1 -- 172.21.15.16:0/2830566598 shutdown_connections 2023-05-20T01:28:47.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.274+0000 7f41362af700 1 --2- 172.21.15.16:0/2830566598 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f41300b5da0 0x7f41300b6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:47.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.274+0000 7f41362af700 1 --2- 172.21.15.16:0/2830566598 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f411006ace0 0x7f411006d190 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:47.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.274+0000 7f41362af700 1 --2- 172.21.15.16:0/2830566598 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f411c0043f0 0x7f4130151a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:47.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.274+0000 7f41362af700 1 --2- 172.21.15.16:0/2830566598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41300d1150 0x7f41300d15c0 unknown :-1 s=CLOSED pgs=1699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:47.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.274+0000 7f41362af700 1 -- 172.21.15.16:0/2830566598 >> 172.21.15.16:0/2830566598 conn(0x7f41300cdd00 msgr2=0x7f4130152300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:47.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.274+0000 7f41362af700 1 -- 172.21.15.16:0/2830566598 shutdown_connections 2023-05-20T01:28:47.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.274+0000 7f41362af700 1 -- 172.21.15.16:0/2830566598 wait complete. 2023-05-20T01:28:47.292 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-05-20T01:28:47.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.415+0000 7f8c9b697700 1 Processor -- start 2023-05-20T01:28:47.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.416+0000 7f8c9b697700 1 -- start start 2023-05-20T01:28:47.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.416+0000 7f8c9b697700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f8c940b2d00 0x7f8c940b30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:47.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.416+0000 7f8c9b697700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8c9406ff30 0x7f8c9406eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:47.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.416+0000 7f8c9b697700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c940708a0 con 0x7f8c94070cf0 2023-05-20T01:28:47.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.416+0000 7f8c9b697700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c9406d5a0 con 0x7f8c9406ff30 2023-05-20T01:28:47.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.416+0000 7f8c9b697700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c9415d080 con 0x7f8c940b2d00 2023-05-20T01:28:47.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.417+0000 7f8c99c34700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8c9406ff30 0x7f8c9406eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:47.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.417+0000 7f8c99c34700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8c9406ff30 0x7f8c9406eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40618/0 (socket says 172.21.15.16:40618) 2023-05-20T01:28:47.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.417+0000 7f8c99c34700 1 -- 172.21.15.16:0/3338711431 learned_addr learned my addr 172.21.15.16:0/3338711431 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:47.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.417+0000 7f8c99433700 1 --2- 172.21.15.16:0/3338711431 >> v2:172.21.15.132:3300/0 conn(0x7f8c940b2d00 0x7f8c940b30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:47.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.417+0000 7f8c99c34700 1 -- 172.21.15.16:0/3338711431 >> v2:172.21.15.132:3300/0 conn(0x7f8c940b2d00 msgr2=0x7f8c940b30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:47.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.417+0000 7f8c99c34700 1 --2- 172.21.15.16:0/3338711431 >> v2:172.21.15.132:3300/0 conn(0x7f8c940b2d00 0x7f8c940b30d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:47.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.417+0000 7f8c99c34700 1 -- 172.21.15.16:0/3338711431 >> v1:172.21.15.132:6789/0 conn(0x7f8c94070cf0 legacy=0x7f8c94160770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:47.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.417+0000 7f8c99c34700 1 -- 172.21.15.16:0/3338711431 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c940b3610 con 0x7f8c9406ff30 2023-05-20T01:28:47.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.417+0000 7f8c99c34700 1 --2- 172.21.15.16:0/3338711431 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c9406ff30 0x7f8c9406eae0 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f8c940ce9a0 tx=0x7f8c8c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e14b4f93a981345e server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:47.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.417+0000 7f8c8bfff700 1 -- 172.21.15.16:0/3338711431 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c8c00e070 con 0x7f8c9406ff30 2023-05-20T01:28:47.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.417+0000 7f8c8bfff700 1 -- 172.21.15.16:0/3338711431 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c8c00b910 con 0x7f8c9406ff30 2023-05-20T01:28:47.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.417+0000 7f8c8bfff700 1 -- 172.21.15.16:0/3338711431 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c8c00bcc0 con 0x7f8c9406ff30 2023-05-20T01:28:47.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.418+0000 7f8c9b697700 1 -- 172.21.15.16:0/3338711431 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c9406ff30 msgr2=0x7f8c9406eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:47.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.418+0000 7f8c9b697700 1 --2- 172.21.15.16:0/3338711431 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c9406ff30 0x7f8c9406eae0 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f8c940ce9a0 tx=0x7f8c8c00b330 comp rx=0 tx=0).stop 2023-05-20T01:28:47.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.418+0000 7f8c9b697700 1 -- 172.21.15.16:0/3338711431 shutdown_connections 2023-05-20T01:28:47.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.418+0000 7f8c9b697700 1 --2- 172.21.15.16:0/3338711431 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c9406ff30 0x7f8c9406eae0 unknown :-1 s=CLOSED pgs=1700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:47.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.418+0000 7f8c9b697700 1 --2- 172.21.15.16:0/3338711431 >> v2:172.21.15.132:3300/0 conn(0x7f8c940b2d00 0x7f8c940b30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:47.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.418+0000 7f8c9b697700 1 -- 172.21.15.16:0/3338711431 >> 172.21.15.16:0/3338711431 conn(0x7f8c940cdd00 msgr2=0x7f8c940ce100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:47.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.418+0000 7f8c9b697700 1 -- 172.21.15.16:0/3338711431 shutdown_connections 2023-05-20T01:28:47.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.418+0000 7f8c9b697700 1 -- 172.21.15.16:0/3338711431 wait complete. 2023-05-20T01:28:47.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.419+0000 7f8c9b697700 1 Processor -- start 2023-05-20T01:28:47.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.419+0000 7f8c9b697700 1 -- start start 2023-05-20T01:28:47.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.420+0000 7f8c9b697700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c9406ff30 0x7f8c940c5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:47.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.420+0000 7f8c9b697700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8c94070cf0 0x7f8c941537c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:47.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.420+0000 7f8c9b697700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c940b2d00 0x7f8c9413f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:47.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.420+0000 7f8c9b697700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c940c7220 con 0x7f8c940b2d00 2023-05-20T01:28:47.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.420+0000 7f8c9b697700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c940c6320 con 0x7f8c9406ff30 2023-05-20T01:28:47.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.420+0000 7f8c9b697700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c940c65a0 con 0x7f8c94070cf0 2023-05-20T01:28:47.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.420+0000 7f8c99c34700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c940b2d00 0x7f8c9413f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:47.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.420+0000 7f8c99c34700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c940b2d00 0x7f8c9413f3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40624/0 (socket says 172.21.15.16:40624) 2023-05-20T01:28:47.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.420+0000 7f8c99c34700 1 -- 172.21.15.16:0/1029141351 learned_addr learned my addr 172.21.15.16:0/1029141351 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:47.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.420+0000 7f8c99433700 1 --2- 172.21.15.16:0/1029141351 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c9406ff30 0x7f8c940c5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:47.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.420+0000 7f8c98c32700 1 --2- 172.21.15.16:0/1029141351 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8c94070cf0 0x7f8c941537c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:47.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.420+0000 7f8c99c34700 1 -- 172.21.15.16:0/1029141351 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8c94070cf0 msgr2=0x7f8c941537c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:47.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.420+0000 7f8c99c34700 1 --2- 172.21.15.16:0/1029141351 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8c94070cf0 0x7f8c941537c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:47.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.420+0000 7f8c99c34700 1 -- 172.21.15.16:0/1029141351 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c9406ff30 msgr2=0x7f8c940c5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:47.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.420+0000 7f8c99c34700 1 --2- 172.21.15.16:0/1029141351 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c9406ff30 0x7f8c940c5de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:47.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.420+0000 7f8c99c34700 1 -- 172.21.15.16:0/1029141351 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c8c009b30 con 0x7f8c940b2d00 2023-05-20T01:28:47.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.420+0000 7f8c99433700 1 --2- 172.21.15.16:0/1029141351 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c9406ff30 0x7f8c940c5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:28:47.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.420+0000 7f8c99c34700 1 --2- 172.21.15.16:0/1029141351 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c940b2d00 0x7f8c9413f3d0 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c00b370 tx=0x7f8c8c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:47.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.421+0000 7f8c89ffb700 1 -- 172.21.15.16:0/1029141351 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c8c0099b0 con 0x7f8c940b2d00 2023-05-20T01:28:47.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.421+0000 7f8c89ffb700 1 -- 172.21.15.16:0/1029141351 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c8c0092e0 con 0x7f8c940b2d00 2023-05-20T01:28:47.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.421+0000 7f8c9b697700 1 -- 172.21.15.16:0/1029141351 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c940d0a00 con 0x7f8c940b2d00 2023-05-20T01:28:47.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.421+0000 7f8c89ffb700 1 -- 172.21.15.16:0/1029141351 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c8c01c640 con 0x7f8c940b2d00 2023-05-20T01:28:47.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.421+0000 7f8c9b697700 1 -- 172.21.15.16:0/1029141351 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c940d0720 con 0x7f8c940b2d00 2023-05-20T01:28:47.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.422+0000 7f8c9b697700 1 -- 172.21.15.16:0/1029141351 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c940d0ce0 con 0x7f8c940b2d00 2023-05-20T01:28:47.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.422+0000 7f8c89ffb700 1 -- 172.21.15.16:0/1029141351 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8c8c01c7a0 con 0x7f8c940b2d00 2023-05-20T01:28:47.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.423+0000 7f8c89ffb700 1 --2- 172.21.15.16:0/1029141351 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f8c7806ace0 0x7f8c7806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:47.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.423+0000 7f8c99433700 1 --2- 172.21.15.16:0/1029141351 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f8c7806ace0 0x7f8c7806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:47.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.423+0000 7f8c89ffb700 1 -- 172.21.15.16:0/1029141351 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6468+0+0 (secure 0 0 0) 0x7f8c8c08af90 con 0x7f8c940b2d00 2023-05-20T01:28:47.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.423+0000 7f8c99433700 1 --2- 172.21.15.16:0/1029141351 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f8c7806ace0 0x7f8c7806d190 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f8c84009b60 tx=0x7f8c84008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:47.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.427+0000 7f8c89ffb700 1 -- 172.21.15.16:0/1029141351 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f8c8c055f30 con 0x7f8c940b2d00 2023-05-20T01:28:47.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.623+0000 7f8c9b697700 1 -- 172.21.15.16:0/1029141351 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8c9404e8c0 con 0x7f8c940b2d00 2023-05-20T01:28:47.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.623+0000 7f8c89ffb700 1 -- 172.21.15.16:0/1029141351 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v137) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8c8c059550 con 0x7f8c940b2d00 2023-05-20T01:28:47.628 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-05-20T01:28:47.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.625+0000 7f8c9b697700 1 -- 172.21.15.16:0/1029141351 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f8c7806ace0 msgr2=0x7f8c7806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:47.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.625+0000 7f8c9b697700 1 --2- 172.21.15.16:0/1029141351 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f8c7806ace0 0x7f8c7806d190 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f8c84009b60 tx=0x7f8c84008040 comp rx=0 tx=0).stop 2023-05-20T01:28:47.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.625+0000 7f8c9b697700 1 -- 172.21.15.16:0/1029141351 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c940b2d00 msgr2=0x7f8c9413f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:47.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.625+0000 7f8c9b697700 1 --2- 172.21.15.16:0/1029141351 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c940b2d00 0x7f8c9413f3d0 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c00b370 tx=0x7f8c8c004ad0 comp rx=0 tx=0).stop 2023-05-20T01:28:47.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.625+0000 7f8c9b697700 1 -- 172.21.15.16:0/1029141351 shutdown_connections 2023-05-20T01:28:47.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.625+0000 7f8c9b697700 1 --2- 172.21.15.16:0/1029141351 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c940b2d00 0x7f8c9413f3d0 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:47.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.625+0000 7f8c9b697700 1 --2- 172.21.15.16:0/1029141351 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c9406ff30 0x7f8c940c5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:47.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.625+0000 7f8c9b697700 1 --2- 172.21.15.16:0/1029141351 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f8c7806ace0 0x7f8c7806d190 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:47.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.625+0000 7f8c9b697700 1 --2- 172.21.15.16:0/1029141351 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8c94070cf0 0x7f8c941537c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:47.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.625+0000 7f8c9b697700 1 -- 172.21.15.16:0/1029141351 >> 172.21.15.16:0/1029141351 conn(0x7f8c940cdd00 msgr2=0x7f8c940d0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:47.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.625+0000 7f8c9b697700 1 -- 172.21.15.16:0/1029141351 shutdown_connections 2023-05-20T01:28:47.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.625+0000 7f8c9b697700 1 -- 172.21.15.16:0/1029141351 wait complete. 2023-05-20T01:28:47.642 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-05-20T01:28:47.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.764+0000 7ffbbb896700 1 Processor -- start 2023-05-20T01:28:47.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.765+0000 7ffbbb896700 1 -- start start 2023-05-20T01:28:47.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.765+0000 7ffbbb896700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffbb40b3410 0x7ffbb40ccc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:47.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.765+0000 7ffbbb896700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7ffbb40cd1c0 0x7ffbb406e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:47.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.765+0000 7ffbbb896700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbb40701f0 con 0x7ffbb415f140 2023-05-20T01:28:47.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.765+0000 7ffbbb896700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbb4070f20 con 0x7ffbb40cd1c0 2023-05-20T01:28:47.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.765+0000 7ffbbb896700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb4070a90 con 0x7ffbb40b3410 2023-05-20T01:28:47.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.766+0000 7ffbb9632700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ffbb415f140 0x7ffbb40b2d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:52156/0 (socket says 172.21.15.16:52156) 2023-05-20T01:28:47.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.766+0000 7ffbb9632700 1 -- 172.21.15.16:0/4129120208 learned_addr learned my addr 172.21.15.16:0/4129120208 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:28:47.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.766+0000 7ffbb8e31700 1 --2- 172.21.15.16:0/4129120208 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffbb40b3410 0x7ffbb40ccc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:47.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.766+0000 7ffbb9e33700 1 --2- 172.21.15.16:0/4129120208 >> v2:172.21.15.132:3300/0 conn(0x7ffbb40cd1c0 0x7ffbb406e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:47.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.766+0000 7ffba7fff700 1 -- 172.21.15.16:0/4129120208 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2735742713 0 0) 0x7ffbb40701f0 con 0x7ffbb415f140 2023-05-20T01:28:47.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.766+0000 7ffba7fff700 1 -- 172.21.15.16:0/4129120208 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffba00035b0 con 0x7ffbb415f140 2023-05-20T01:28:47.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.766+0000 7ffba7fff700 1 -- 172.21.15.16:0/4129120208 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2981284722 0 0) 0x7ffba00035b0 con 0x7ffbb415f140 2023-05-20T01:28:47.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.766+0000 7ffba7fff700 1 -- 172.21.15.16:0/4129120208 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffbb40b3410 msgr2=0x7ffbb40ccc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:47.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.766+0000 7ffba7fff700 1 --2- 172.21.15.16:0/4129120208 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffbb40b3410 0x7ffbb40ccc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:47.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.766+0000 7ffba7fff700 1 -- 172.21.15.16:0/4129120208 >> v2:172.21.15.132:3300/0 conn(0x7ffbb40cd1c0 msgr2=0x7ffbb406e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:47.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.766+0000 7ffba7fff700 1 --2- 172.21.15.16:0/4129120208 >> v2:172.21.15.132:3300/0 conn(0x7ffbb40cd1c0 0x7ffbb406e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:47.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.766+0000 7ffba7fff700 1 -- 172.21.15.16:0/4129120208 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbb415f510 con 0x7ffbb415f140 2023-05-20T01:28:47.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.766+0000 7ffba7fff700 1 -- 172.21.15.16:0/4129120208 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7ffbb0001670 con 0x7ffbb415f140 2023-05-20T01:28:47.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.766+0000 7ffba7fff700 1 -- 172.21.15.16:0/4129120208 >> v1:172.21.15.16:6789/0 conn(0x7ffbb415f140 legacy=0x7ffbb40b2d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:47.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.766+0000 7ffba7fff700 1 --2- 172.21.15.16:0/4129120208 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffbb40b3410 0x7ffba0003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:47.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.766+0000 7ffba7fff700 1 --2- 172.21.15.16:0/4129120208 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffba0003f80 0x7ffba0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:47.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.766+0000 7ffba7fff700 1 --2- 172.21.15.16:0/4129120208 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffba0004d90 0x7ffba0005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:47.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.766+0000 7ffba7fff700 1 -- 172.21.15.16:0/4129120208 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb4070f20 con 0x7ffba0003f80 2023-05-20T01:28:47.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.766+0000 7ffba7fff700 1 -- 172.21.15.16:0/4129120208 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb4070a90 con 0x7ffba0004d90 2023-05-20T01:28:47.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.766+0000 7ffba7fff700 1 -- 172.21.15.16:0/4129120208 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffba000ba40 con 0x7ffbb40b3410 2023-05-20T01:28:47.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.766+0000 7ffbb8e31700 1 --2- 172.21.15.16:0/4129120208 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffba0003f80 0x7ffba0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:47.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.767+0000 7ffbb9632700 1 --2- 172.21.15.16:0/4129120208 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffbb40b3410 0x7ffba0003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:47.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.767+0000 7ffbb9e33700 1 --2- 172.21.15.16:0/4129120208 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffba0004d90 0x7ffba0005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:47.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.767+0000 7ffbb8e31700 1 -- 172.21.15.16:0/4129120208 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffbb40b3410 msgr2=0x7ffba0003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:47.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.767+0000 7ffbb8e31700 1 --2- 172.21.15.16:0/4129120208 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffbb40b3410 0x7ffba0003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:47.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.767+0000 7ffbb8e31700 1 -- 172.21.15.16:0/4129120208 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffba0004d90 msgr2=0x7ffba0005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:47.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.767+0000 7ffbb8e31700 1 --2- 172.21.15.16:0/4129120208 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffba0004d90 0x7ffba0005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:47.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.767+0000 7ffbb8e31700 1 -- 172.21.15.16:0/4129120208 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffba00035b0 con 0x7ffba0003f80 2023-05-20T01:28:47.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.767+0000 7ffbb8e31700 1 --2- 172.21.15.16:0/4129120208 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffba0003f80 0x7ffba0004850 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7ffba8009b60 tx=0x7ffba0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e793b46d862e59f server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:47.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.767+0000 7ffba7fff700 1 -- 172.21.15.16:0/4129120208 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba800e660 con 0x7ffba0003f80 2023-05-20T01:28:47.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.767+0000 7ffba7fff700 1 -- 172.21.15.16:0/4129120208 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffba800e7c0 con 0x7ffba0003f80 2023-05-20T01:28:47.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.767+0000 7ffba7fff700 1 -- 172.21.15.16:0/4129120208 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba800eb30 con 0x7ffba0003f80 2023-05-20T01:28:47.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.768+0000 7ffbbb896700 1 -- 172.21.15.16:0/4129120208 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffba0003f80 msgr2=0x7ffba0004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:47.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.768+0000 7ffbbb896700 1 --2- 172.21.15.16:0/4129120208 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffba0003f80 0x7ffba0004850 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7ffba8009b60 tx=0x7ffba0005880 comp rx=0 tx=0).stop 2023-05-20T01:28:47.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.768+0000 7ffbb9e33700 1 -- 172.21.15.16:0/4129120208 reap_dead start 2023-05-20T01:28:47.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.768+0000 7ffbbb896700 1 -- 172.21.15.16:0/4129120208 shutdown_connections 2023-05-20T01:28:47.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.768+0000 7ffbbb896700 1 -- 172.21.15.16:0/4129120208 >> 172.21.15.16:0/4129120208 conn(0x7ffbb40cdd00 msgr2=0x7ffbb40ce100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:47.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.768+0000 7ffbbb896700 1 -- 172.21.15.16:0/4129120208 shutdown_connections 2023-05-20T01:28:47.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.768+0000 7ffbbb896700 1 -- 172.21.15.16:0/4129120208 wait complete. 2023-05-20T01:28:47.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.769+0000 7ffbbb896700 1 Processor -- start 2023-05-20T01:28:47.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.769+0000 7ffbbb896700 1 -- start start 2023-05-20T01:28:47.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.770+0000 7ffbbb896700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffba0003f80 0x7ffbb40c78c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:47.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.770+0000 7ffbbb896700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffbb415d100 0x7ffbb414e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:47.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.770+0000 7ffbbb896700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffbb40b5300 0x7ffbb40b5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:47.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.770+0000 7ffbbb896700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb415de60 con 0x7ffbb415d100 2023-05-20T01:28:47.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.770+0000 7ffbbb896700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb40c7e00 con 0x7ffbb40b5300 2023-05-20T01:28:47.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.770+0000 7ffbbb896700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb40b5cc0 con 0x7ffba0003f80 2023-05-20T01:28:47.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.770+0000 7ffbb8e31700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffbb415d100 0x7ffbb414e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:47.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.770+0000 7ffbb8e31700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffbb415d100 0x7ffbb414e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40642/0 (socket says 172.21.15.16:40642) 2023-05-20T01:28:47.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.770+0000 7ffbb8e31700 1 -- 172.21.15.16:0/173084509 learned_addr learned my addr 172.21.15.16:0/173084509 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:47.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.770+0000 7ffbb9632700 1 --2- 172.21.15.16:0/173084509 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffba0003f80 0x7ffbb40c78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:47.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.770+0000 7ffbb9e33700 1 --2- 172.21.15.16:0/173084509 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffbb40b5300 0x7ffbb40b5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:47.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.770+0000 7ffbb8e31700 1 -- 172.21.15.16:0/173084509 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffba0003f80 msgr2=0x7ffbb40c78c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:47.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.770+0000 7ffbb8e31700 1 --2- 172.21.15.16:0/173084509 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffba0003f80 0x7ffbb40c78c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:47.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.770+0000 7ffbb8e31700 1 -- 172.21.15.16:0/173084509 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffbb40b5300 msgr2=0x7ffbb40b5780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:47.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.770+0000 7ffbb8e31700 1 --2- 172.21.15.16:0/173084509 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffbb40b5300 0x7ffbb40b5780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:47.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.770+0000 7ffbb8e31700 1 -- 172.21.15.16:0/173084509 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbb40cd8f0 con 0x7ffbb415d100 2023-05-20T01:28:47.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.770+0000 7ffbb9e33700 1 --2- 172.21.15.16:0/173084509 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffbb40b5300 0x7ffbb40b5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:28:47.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.770+0000 7ffbb8e31700 1 --2- 172.21.15.16:0/173084509 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffbb415d100 0x7ffbb414e780 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7ffba0004770 tx=0x7ffba80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:47.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.770+0000 7ffba5ffb700 1 -- 172.21.15.16:0/173084509 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba8021070 con 0x7ffbb415d100 2023-05-20T01:28:47.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.770+0000 7ffba5ffb700 1 -- 172.21.15.16:0/173084509 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffba80045d0 con 0x7ffbb415d100 2023-05-20T01:28:47.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.770+0000 7ffbbb896700 1 -- 172.21.15.16:0/173084509 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbb40b71e0 con 0x7ffbb415d100 2023-05-20T01:28:47.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.771+0000 7ffba5ffb700 1 -- 172.21.15.16:0/173084509 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba800e660 con 0x7ffbb415d100 2023-05-20T01:28:47.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.771+0000 7ffbbb896700 1 -- 172.21.15.16:0/173084509 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbb4070440 con 0x7ffbb415d100 2023-05-20T01:28:47.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.772+0000 7ffbbb896700 1 -- 172.21.15.16:0/173084509 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbb404e8c0 con 0x7ffbb415d100 2023-05-20T01:28:47.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.772+0000 7ffba5ffb700 1 -- 172.21.15.16:0/173084509 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ffba8025070 con 0x7ffbb415d100 2023-05-20T01:28:47.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.772+0000 7ffba5ffb700 1 --2- 172.21.15.16:0/173084509 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ffb9806ace0 0x7ffb9806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:47.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.773+0000 7ffba5ffb700 1 -- 172.21.15.16:0/173084509 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6468+0+0 (secure 0 0 0) 0x7ffba808a7f0 con 0x7ffbb415d100 2023-05-20T01:28:47.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.773+0000 7ffbb9632700 1 --2- 172.21.15.16:0/173084509 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ffb9806ace0 0x7ffb9806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:47.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.773+0000 7ffbb9632700 1 --2- 172.21.15.16:0/173084509 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ffb9806ace0 0x7ffb9806d190 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7ffbb0000c00 tx=0x7ffbb0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:47.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.777+0000 7ffba5ffb700 1 -- 172.21.15.16:0/173084509 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7ffba80558b0 con 0x7ffbb415d100 2023-05-20T01:28:47.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.974+0000 7ffbbb896700 1 -- 172.21.15.16:0/173084509 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ffbb406fd10 con 0x7ffbb415d100 2023-05-20T01:28:47.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.974+0000 7ffba5ffb700 1 -- 172.21.15.16:0/173084509 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v137) v1 ==== 104+0+11 (secure 0 0 0) 0x7ffba8058ed0 con 0x7ffbb415d100 2023-05-20T01:28:47.979 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-05-20T01:28:47.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.976+0000 7ffbbb896700 1 -- 172.21.15.16:0/173084509 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ffb9806ace0 msgr2=0x7ffb9806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:47.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.976+0000 7ffbbb896700 1 --2- 172.21.15.16:0/173084509 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ffb9806ace0 0x7ffb9806d190 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7ffbb0000c00 tx=0x7ffbb0009040 comp rx=0 tx=0).stop 2023-05-20T01:28:47.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.976+0000 7ffbbb896700 1 -- 172.21.15.16:0/173084509 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffbb415d100 msgr2=0x7ffbb414e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:47.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.976+0000 7ffbbb896700 1 --2- 172.21.15.16:0/173084509 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffbb415d100 0x7ffbb414e780 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7ffba0004770 tx=0x7ffba80047c0 comp rx=0 tx=0).stop 2023-05-20T01:28:47.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.977+0000 7ffbbb896700 1 -- 172.21.15.16:0/173084509 shutdown_connections 2023-05-20T01:28:47.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.977+0000 7ffbbb896700 1 --2- 172.21.15.16:0/173084509 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffbb40b5300 0x7ffbb40b5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:47.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.977+0000 7ffbbb896700 1 --2- 172.21.15.16:0/173084509 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ffb9806ace0 0x7ffb9806d190 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:47.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.977+0000 7ffbbb896700 1 --2- 172.21.15.16:0/173084509 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffba0003f80 0x7ffbb40c78c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:47.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.977+0000 7ffbbb896700 1 --2- 172.21.15.16:0/173084509 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffbb415d100 0x7ffbb414e780 unknown :-1 s=CLOSED pgs=1704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:47.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.977+0000 7ffbbb896700 1 -- 172.21.15.16:0/173084509 >> 172.21.15.16:0/173084509 conn(0x7ffbb40cdd00 msgr2=0x7ffbb40b1d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:47.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.977+0000 7ffbbb896700 1 -- 172.21.15.16:0/173084509 shutdown_connections 2023-05-20T01:28:47.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:47.977+0000 7ffbbb896700 1 -- 172.21.15.16:0/173084509 wait complete. 2023-05-20T01:28:47.993 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-05-20T01:28:48.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.117+0000 7fba0ec15700 1 Processor -- start 2023-05-20T01:28:48.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.117+0000 7fba0ec15700 1 -- start start 2023-05-20T01:28:48.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.118+0000 7fba0ec15700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fba08159850 0x7fba080ce120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:48.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.118+0000 7fba0ec15700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fba08072480 0x7fba080ce660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:48.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.118+0000 7fba0ec15700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fba0806f740 0x7fba08071450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:48.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.118+0000 7fba0ec15700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba0806e380 con 0x7fba0806f740 2023-05-20T01:28:48.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.118+0000 7fba0ec15700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba0806de50 con 0x7fba08159850 2023-05-20T01:28:48.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.118+0000 7fba0ec15700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba0815d450 con 0x7fba08072480 2023-05-20T01:28:48.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.118+0000 7fba0d1b2700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fba0806f740 0x7fba08071450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:48.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.118+0000 7fba0d1b2700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fba0806f740 0x7fba08071450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40650/0 (socket says 172.21.15.16:40650) 2023-05-20T01:28:48.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.118+0000 7fba0d1b2700 1 -- 172.21.15.16:0/895028332 learned_addr learned my addr 172.21.15.16:0/895028332 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:48.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.118+0000 7fba07fff700 1 --2- 172.21.15.16:0/895028332 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fba08072480 0x7fba080ce660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:48.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.118+0000 7fba0c9b1700 1 --2- 172.21.15.16:0/895028332 >> v2:172.21.15.132:3300/0 conn(0x7fba08159850 0x7fba080ce120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:48.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.119+0000 7fba0d1b2700 1 -- 172.21.15.16:0/895028332 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fba08072480 msgr2=0x7fba080ce660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:48.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.119+0000 7fba0d1b2700 1 --2- 172.21.15.16:0/895028332 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fba08072480 0x7fba080ce660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:48.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.119+0000 7fba0d1b2700 1 -- 172.21.15.16:0/895028332 >> v2:172.21.15.132:3300/0 conn(0x7fba08159850 msgr2=0x7fba080ce120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:48.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.119+0000 7fba0d1b2700 1 --2- 172.21.15.16:0/895028332 >> v2:172.21.15.132:3300/0 conn(0x7fba08159850 0x7fba080ce120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:48.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.119+0000 7fba0d1b2700 1 -- 172.21.15.16:0/895028332 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba0806e0a0 con 0x7fba0806f740 2023-05-20T01:28:48.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.119+0000 7fba0d1b2700 1 --2- 172.21.15.16:0/895028332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba0806f740 0x7fba08071450 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7fba080cd890 tx=0x7fba0000d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7df38a3008211d8 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:48.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.119+0000 7fba077fe700 1 -- 172.21.15.16:0/895028332 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba00010070 con 0x7fba0806f740 2023-05-20T01:28:48.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.119+0000 7fba077fe700 1 -- 172.21.15.16:0/895028332 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba0000dce0 con 0x7fba0806f740 2023-05-20T01:28:48.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.119+0000 7fba077fe700 1 -- 172.21.15.16:0/895028332 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba0000b310 con 0x7fba0806f740 2023-05-20T01:28:48.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.120+0000 7fba0ec15700 1 -- 172.21.15.16:0/895028332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba0806f740 msgr2=0x7fba08071450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:48.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.120+0000 7fba0ec15700 1 --2- 172.21.15.16:0/895028332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba0806f740 0x7fba08071450 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7fba080cd890 tx=0x7fba0000d430 comp rx=0 tx=0).stop 2023-05-20T01:28:48.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.120+0000 7fba0ec15700 1 -- 172.21.15.16:0/895028332 shutdown_connections 2023-05-20T01:28:48.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.120+0000 7fba0ec15700 1 --2- 172.21.15.16:0/895028332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba0806f740 0x7fba08071450 unknown :-1 s=CLOSED pgs=1705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:48.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.120+0000 7fba0ec15700 1 --2- 172.21.15.16:0/895028332 >> v2:172.21.15.132:3300/0 conn(0x7fba08159850 0x7fba080ce120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:48.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.120+0000 7fba0ec15700 1 --2- 172.21.15.16:0/895028332 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fba08072480 0x7fba080ce660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:48.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.120+0000 7fba0ec15700 1 -- 172.21.15.16:0/895028332 >> 172.21.15.16:0/895028332 conn(0x7fba080b1d90 msgr2=0x7fba080b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:48.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.120+0000 7fba0ec15700 1 -- 172.21.15.16:0/895028332 shutdown_connections 2023-05-20T01:28:48.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.120+0000 7fba0ec15700 1 -- 172.21.15.16:0/895028332 wait complete. 2023-05-20T01:28:48.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.121+0000 7fba0ec15700 1 Processor -- start 2023-05-20T01:28:48.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.121+0000 7fba0ec15700 1 -- start start 2023-05-20T01:28:48.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.122+0000 7fba0ec15700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba0806f740 0x7fba080c7d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:48.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.122+0000 7fba0ec15700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fba08072480 0x7fba080c65f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:48.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.122+0000 7fba0ec15700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fba08159850 0x7fba080cbc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:48.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.122+0000 7fba0ec15700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba080c82e0 con 0x7fba0806f740 2023-05-20T01:28:48.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.122+0000 7fba0ec15700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba080c85c0 con 0x7fba08072480 2023-05-20T01:28:48.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.122+0000 7fba0ec15700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba080c88a0 con 0x7fba08159850 2023-05-20T01:28:48.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.122+0000 7fba0c9b1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba0806f740 0x7fba080c7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:48.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.122+0000 7fba0c9b1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba0806f740 0x7fba080c7d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40652/0 (socket says 172.21.15.16:40652) 2023-05-20T01:28:48.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.122+0000 7fba0c9b1700 1 -- 172.21.15.16:0/1853030634 learned_addr learned my addr 172.21.15.16:0/1853030634 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:48.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.122+0000 7fba07fff700 1 --2- 172.21.15.16:0/1853030634 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fba08072480 0x7fba080c65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:48.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.122+0000 7fba0d1b2700 1 --2- 172.21.15.16:0/1853030634 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fba08159850 0x7fba080cbc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:48.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.122+0000 7fba0c9b1700 1 -- 172.21.15.16:0/1853030634 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fba08159850 msgr2=0x7fba080cbc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:48.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.122+0000 7fba0c9b1700 1 --2- 172.21.15.16:0/1853030634 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fba08159850 0x7fba080cbc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:48.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.122+0000 7fba0c9b1700 1 -- 172.21.15.16:0/1853030634 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fba08072480 msgr2=0x7fba080c65f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:48.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.122+0000 7fba0c9b1700 1 --2- 172.21.15.16:0/1853030634 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fba08072480 0x7fba080c65f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:48.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.122+0000 7fba0c9b1700 1 -- 172.21.15.16:0/1853030634 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba0000bd40 con 0x7fba0806f740 2023-05-20T01:28:48.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.122+0000 7fba0c9b1700 1 --2- 172.21.15.16:0/1853030634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba0806f740 0x7fba080c7d00 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc00e870 tx=0x7fb9fc00eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:48.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.123+0000 7fba057fa700 1 -- 172.21.15.16:0/1853030634 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9fc009590 con 0x7fba0806f740 2023-05-20T01:28:48.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.123+0000 7fba057fa700 1 -- 172.21.15.16:0/1853030634 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9fc013680 con 0x7fba0806f740 2023-05-20T01:28:48.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.123+0000 7fba0ec15700 1 -- 172.21.15.16:0/1853030634 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba0815d980 con 0x7fba0806f740 2023-05-20T01:28:48.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.123+0000 7fba057fa700 1 -- 172.21.15.16:0/1853030634 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9fc0139f0 con 0x7fba0806f740 2023-05-20T01:28:48.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.123+0000 7fba0ec15700 1 -- 172.21.15.16:0/1853030634 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba0815d6a0 con 0x7fba0806f740 2023-05-20T01:28:48.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.124+0000 7fba0ec15700 1 -- 172.21.15.16:0/1853030634 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba0815dc60 con 0x7fba0806f740 2023-05-20T01:28:48.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.124+0000 7fba057fa700 1 -- 172.21.15.16:0/1853030634 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb9fc013b50 con 0x7fba0806f740 2023-05-20T01:28:48.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.125+0000 7fba057fa700 1 --2- 172.21.15.16:0/1853030634 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb9ec06ace0 0x7fb9ec06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:48.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.125+0000 7fba07fff700 1 --2- 172.21.15.16:0/1853030634 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb9ec06ace0 0x7fb9ec06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:48.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.125+0000 7fba057fa700 1 -- 172.21.15.16:0/1853030634 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6468+0+0 (secure 0 0 0) 0x7fb9fc089810 con 0x7fba0806f740 2023-05-20T01:28:48.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.125+0000 7fba07fff700 1 --2- 172.21.15.16:0/1853030634 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb9ec06ace0 0x7fb9ec06d190 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fba08156ea0 tx=0x7fb9f4009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:48.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.129+0000 7fba057fa700 1 -- 172.21.15.16:0/1853030634 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fb9fc0547b0 con 0x7fba0806f740 2023-05-20T01:28:48.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.326+0000 7fba0ec15700 1 -- 172.21.15.16:0/1853030634 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fba0804e8c0 con 0x7fba0806f740 2023-05-20T01:28:48.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.327+0000 7fba057fa700 1 -- 172.21.15.16:0/1853030634 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v137) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb9fc087e60 con 0x7fba0806f740 2023-05-20T01:28:48.335 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2023-05-20T01:28:48.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.329+0000 7fba0ec15700 1 -- 172.21.15.16:0/1853030634 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb9ec06ace0 msgr2=0x7fb9ec06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:48.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.329+0000 7fba0ec15700 1 --2- 172.21.15.16:0/1853030634 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb9ec06ace0 0x7fb9ec06d190 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fba08156ea0 tx=0x7fb9f4009210 comp rx=0 tx=0).stop 2023-05-20T01:28:48.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.329+0000 7fba0ec15700 1 -- 172.21.15.16:0/1853030634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba0806f740 msgr2=0x7fba080c7d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:48.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.329+0000 7fba0ec15700 1 --2- 172.21.15.16:0/1853030634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba0806f740 0x7fba080c7d00 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc00e870 tx=0x7fb9fc00eb80 comp rx=0 tx=0).stop 2023-05-20T01:28:48.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.329+0000 7fba0ec15700 1 -- 172.21.15.16:0/1853030634 shutdown_connections 2023-05-20T01:28:48.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.329+0000 7fba0ec15700 1 --2- 172.21.15.16:0/1853030634 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb9ec06ace0 0x7fb9ec06d190 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:48.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.329+0000 7fba0ec15700 1 --2- 172.21.15.16:0/1853030634 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fba08159850 0x7fba080cbc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:48.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.329+0000 7fba0ec15700 1 --2- 172.21.15.16:0/1853030634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba0806f740 0x7fba080c7d00 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:48.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.329+0000 7fba0ec15700 1 --2- 172.21.15.16:0/1853030634 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fba08072480 0x7fba080c65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:48.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.329+0000 7fba0ec15700 1 -- 172.21.15.16:0/1853030634 >> 172.21.15.16:0/1853030634 conn(0x7fba080b1d90 msgr2=0x7fba08156690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:48.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.329+0000 7fba0ec15700 1 -- 172.21.15.16:0/1853030634 shutdown_connections 2023-05-20T01:28:48.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.329+0000 7fba0ec15700 1 -- 172.21.15.16:0/1853030634 wait complete. 2023-05-20T01:28:48.345 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-05-20T01:28:48.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.467+0000 7ff88b395700 1 Processor -- start 2023-05-20T01:28:48.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.467+0000 7ff88b395700 1 -- start start 2023-05-20T01:28:48.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.468+0000 7ff88b395700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff88415cb30 0x7ff88406c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:48.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.468+0000 7ff88b395700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7ff8840d0a50 0x7ff884161150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:48.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.468+0000 7ff88b395700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff884161690 con 0x7ff88406cc50 2023-05-20T01:28:48.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.468+0000 7ff88b395700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff88406d4f0 con 0x7ff8840d0a50 2023-05-20T01:28:48.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.468+0000 7ff88b395700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff88406d210 con 0x7ff88415cb30 2023-05-20T01:28:48.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.469+0000 7ff889932700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7ff8840d0a50 0x7ff884161150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:48.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.469+0000 7ff889131700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff88415cb30 0x7ff88406c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:48.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.469+0000 7ff888930700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7ff88406cc50 0x7ff8840d0340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.16:33670/0 (socket says 172.21.15.16:33670) 2023-05-20T01:28:48.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.469+0000 7ff888930700 1 -- 172.21.15.16:0/863834347 learned_addr learned my addr 172.21.15.16:0/863834347 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:28:48.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.469+0000 7ff889932700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7ff8840d0a50 0x7ff884161150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.16:37686/0 (socket says 172.21.15.16:37686) 2023-05-20T01:28:48.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.469+0000 7ff889131700 1 -- 172.21.15.16:0/863834347 >> v2:172.21.15.132:3300/0 conn(0x7ff8840d0a50 msgr2=0x7ff884161150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:48.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.469+0000 7ff889131700 1 --2- 172.21.15.16:0/863834347 >> v2:172.21.15.132:3300/0 conn(0x7ff8840d0a50 0x7ff884161150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:48.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.469+0000 7ff889131700 1 -- 172.21.15.16:0/863834347 >> v1:172.21.15.132:6789/0 conn(0x7ff88406cc50 legacy=0x7ff8840d0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:48.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.469+0000 7ff889131700 1 -- 172.21.15.16:0/863834347 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff88406b6a0 con 0x7ff88415cb30 2023-05-20T01:28:48.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.469+0000 7ff889932700 1 --2- 172.21.15.16:0/863834347 >> v2:172.21.15.132:3300/0 conn(0x7ff8840d0a50 0x7ff884161150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-05-20T01:28:48.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.469+0000 7ff889131700 1 --2- 172.21.15.16:0/863834347 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff88415cb30 0x7ff88406c180 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff874009840 tx=0x7ff874009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9c17b8374256e5d server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:48.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.469+0000 7ff87bfff700 1 -- 172.21.15.16:0/863834347 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff874011830 con 0x7ff88415cb30 2023-05-20T01:28:48.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.470+0000 7ff87bfff700 1 -- 172.21.15.16:0/863834347 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff874011990 con 0x7ff88415cb30 2023-05-20T01:28:48.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.470+0000 7ff87bfff700 1 -- 172.21.15.16:0/863834347 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff874011d20 con 0x7ff88415cb30 2023-05-20T01:28:48.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.470+0000 7ff88b395700 1 -- 172.21.15.16:0/863834347 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff88415cb30 msgr2=0x7ff88406c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:48.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.470+0000 7ff88b395700 1 --2- 172.21.15.16:0/863834347 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff88415cb30 0x7ff88406c180 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff874009840 tx=0x7ff874009b50 comp rx=0 tx=0).stop 2023-05-20T01:28:48.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.470+0000 7ff88b395700 1 -- 172.21.15.16:0/863834347 shutdown_connections 2023-05-20T01:28:48.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.470+0000 7ff88b395700 1 --2- 172.21.15.16:0/863834347 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff88415cb30 0x7ff88406c180 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:48.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.470+0000 7ff88b395700 1 --2- 172.21.15.16:0/863834347 >> v2:172.21.15.132:3300/0 conn(0x7ff8840d0a50 0x7ff884161150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:48.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.470+0000 7ff88b395700 1 -- 172.21.15.16:0/863834347 >> 172.21.15.16:0/863834347 conn(0x7ff88406e4c0 msgr2=0x7ff88406e8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:48.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.470+0000 7ff88b395700 1 -- 172.21.15.16:0/863834347 shutdown_connections 2023-05-20T01:28:48.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.470+0000 7ff88b395700 1 -- 172.21.15.16:0/863834347 wait complete. 2023-05-20T01:28:48.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.471+0000 7ff88b395700 1 Processor -- start 2023-05-20T01:28:48.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.471+0000 7ff88b395700 1 -- start start 2023-05-20T01:28:48.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.472+0000 7ff88b395700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff88406cc50 0x7ff8841435f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:48.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.472+0000 7ff88b395700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8840d0a50 0x7ff88415f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:48.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.472+0000 7ff88b395700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff88415cb30 0x7ff8840b5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:48.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.472+0000 7ff88b395700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff884150f00 con 0x7ff8840d0a50 2023-05-20T01:28:48.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.472+0000 7ff88b395700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8840b51e0 con 0x7ff88415cb30 2023-05-20T01:28:48.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.472+0000 7ff88b395700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8840b45d0 con 0x7ff88406cc50 2023-05-20T01:28:48.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.472+0000 7ff888930700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8840d0a50 0x7ff88415f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:48.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.472+0000 7ff888930700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8840d0a50 0x7ff88415f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40668/0 (socket says 172.21.15.16:40668) 2023-05-20T01:28:48.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.472+0000 7ff888930700 1 -- 172.21.15.16:0/3386282128 learned_addr learned my addr 172.21.15.16:0/3386282128 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:48.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.472+0000 7ff889932700 1 --2- 172.21.15.16:0/3386282128 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff88415cb30 0x7ff8840b5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:48.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.472+0000 7ff889131700 1 --2- 172.21.15.16:0/3386282128 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff88406cc50 0x7ff8841435f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:48.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.472+0000 7ff888930700 1 -- 172.21.15.16:0/3386282128 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff88406cc50 msgr2=0x7ff8841435f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:48.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.472+0000 7ff888930700 1 --2- 172.21.15.16:0/3386282128 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff88406cc50 0x7ff8841435f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:48.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.472+0000 7ff888930700 1 -- 172.21.15.16:0/3386282128 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff88415cb30 msgr2=0x7ff8840b5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:48.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.472+0000 7ff888930700 1 --2- 172.21.15.16:0/3386282128 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff88415cb30 0x7ff8840b5670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:48.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.472+0000 7ff888930700 1 -- 172.21.15.16:0/3386282128 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff87c009a70 con 0x7ff8840d0a50 2023-05-20T01:28:48.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.472+0000 7ff889131700 1 --2- 172.21.15.16:0/3386282128 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff88406cc50 0x7ff8841435f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:28:48.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.473+0000 7ff888930700 1 --2- 172.21.15.16:0/3386282128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8840d0a50 0x7ff88415f310 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7ff88000c520 tx=0x7ff88000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:48.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.473+0000 7ff879ffb700 1 -- 172.21.15.16:0/3386282128 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff880005d70 con 0x7ff8840d0a50 2023-05-20T01:28:48.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.473+0000 7ff879ffb700 1 -- 172.21.15.16:0/3386282128 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8800070d0 con 0x7ff8840d0a50 2023-05-20T01:28:48.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.473+0000 7ff88b395700 1 -- 172.21.15.16:0/3386282128 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8740094a0 con 0x7ff8840d0a50 2023-05-20T01:28:48.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.473+0000 7ff879ffb700 1 -- 172.21.15.16:0/3386282128 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff880007440 con 0x7ff8840d0a50 2023-05-20T01:28:48.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.473+0000 7ff88b395700 1 -- 172.21.15.16:0/3386282128 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff88406bc60 con 0x7ff8840d0a50 2023-05-20T01:28:48.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.474+0000 7ff88b395700 1 -- 172.21.15.16:0/3386282128 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff88406b980 con 0x7ff8840d0a50 2023-05-20T01:28:48.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.474+0000 7ff879ffb700 1 -- 172.21.15.16:0/3386282128 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff8800075a0 con 0x7ff8840d0a50 2023-05-20T01:28:48.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.475+0000 7ff879ffb700 1 --2- 172.21.15.16:0/3386282128 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff86806ace0 0x7ff86806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:48.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.475+0000 7ff879ffb700 1 -- 172.21.15.16:0/3386282128 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6468+0+0 (secure 0 0 0) 0x7ff8800887b0 con 0x7ff8840d0a50 2023-05-20T01:28:48.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.475+0000 7ff889131700 1 --2- 172.21.15.16:0/3386282128 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff86806ace0 0x7ff86806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:48.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.475+0000 7ff889131700 1 --2- 172.21.15.16:0/3386282128 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff86806ace0 0x7ff86806d190 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7ff8740090b0 tx=0x7ff874008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:48.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.479+0000 7ff879ffb700 1 -- 172.21.15.16:0/3386282128 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7ff880053800 con 0x7ff8840d0a50 2023-05-20T01:28:48.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.693+0000 7ff88b395700 1 -- 172.21.15.16:0/3386282128 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ff88415cf00 con 0x7ff8840d0a50 2023-05-20T01:28:48.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.693+0000 7ff879ffb700 1 -- 172.21.15.16:0/3386282128 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v137) v1 ==== 80+0+122 (secure 0 0 0) 0x7ff880056e20 con 0x7ff8840d0a50 2023-05-20T01:28:48.698 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6837/2295684907 2023-05-21T01:28:39.430636+0000 2023-05-20T01:28:48.698 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6835/2295684907 2023-05-21T01:28:39.430636+0000 2023-05-20T01:28:48.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.696+0000 7ff88b395700 1 -- 172.21.15.16:0/3386282128 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff86806ace0 msgr2=0x7ff86806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:48.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.696+0000 7ff88b395700 1 --2- 172.21.15.16:0/3386282128 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff86806ace0 0x7ff86806d190 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7ff8740090b0 tx=0x7ff874008030 comp rx=0 tx=0).stop 2023-05-20T01:28:48.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.696+0000 7ff88b395700 1 -- 172.21.15.16:0/3386282128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8840d0a50 msgr2=0x7ff88415f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:48.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.696+0000 7ff88b395700 1 --2- 172.21.15.16:0/3386282128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8840d0a50 0x7ff88415f310 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7ff88000c520 tx=0x7ff88000c8e0 comp rx=0 tx=0).stop 2023-05-20T01:28:48.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.696+0000 7ff88b395700 1 -- 172.21.15.16:0/3386282128 shutdown_connections 2023-05-20T01:28:48.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.696+0000 7ff88b395700 1 --2- 172.21.15.16:0/3386282128 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff88415cb30 0x7ff8840b5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:48.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.696+0000 7ff88b395700 1 --2- 172.21.15.16:0/3386282128 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff86806ace0 0x7ff86806d190 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:48.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.696+0000 7ff88b395700 1 --2- 172.21.15.16:0/3386282128 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff88406cc50 0x7ff8841435f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:48.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.696+0000 7ff88b395700 1 --2- 172.21.15.16:0/3386282128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8840d0a50 0x7ff88415f310 unknown :-1 s=CLOSED pgs=1707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:48.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.696+0000 7ff88b395700 1 -- 172.21.15.16:0/3386282128 >> 172.21.15.16:0/3386282128 conn(0x7ff88406e4c0 msgr2=0x7ff8841586c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:48.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.696+0000 7ff88b395700 1 -- 172.21.15.16:0/3386282128 shutdown_connections 2023-05-20T01:28:48.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.696+0000 7ff88b395700 1 -- 172.21.15.16:0/3386282128 wait complete. 2023-05-20T01:28:48.702 INFO:teuthology.orchestra.run.smithi016.stderr:listed 2 entries 2023-05-20T01:28:48.712 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-05-20T01:28:48.712 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:28:48.713 DEBUG:teuthology.orchestra.run.smithi016:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-05-20T01:28:48.758 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2023-05-20T01:28:48.759 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:28:48.759 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-05-20T01:28:48.777 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-05-20T01:28:48.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.937+0000 7f51f51fa700 1 Processor -- start 2023-05-20T01:28:48.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.937+0000 7f51f51fa700 1 -- start start 2023-05-20T01:28:48.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.938+0000 7f51f51fa700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f51f015cc70 0x7f51f00718b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:48.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.938+0000 7f51f51fa700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f51f00cc7c0 0x7f51f0161230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:48.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.938+0000 7f51f51fa700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51f0071040 con 0x7f51f0070b30 2023-05-20T01:28:48.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.938+0000 7f51f51fa700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51f0160790 con 0x7f51f015cc70 2023-05-20T01:28:48.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.938+0000 7f51f51fa700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51f0071290 con 0x7f51f00cc7c0 2023-05-20T01:28:48.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.938+0000 7f51eed9d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f51f015cc70 0x7f51f00718b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:48.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.938+0000 7f51eed9d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f51f015cc70 0x7f51f00718b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40674/0 (socket says 172.21.15.16:40674) 2023-05-20T01:28:48.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.938+0000 7f51ee59c700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f51f0070b30 0x7f51f00cc0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:52204/0 (socket says 172.21.15.16:52204) 2023-05-20T01:28:48.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.938+0000 7f51eed9d700 1 -- 172.21.15.16:0/538162743 learned_addr learned my addr 172.21.15.16:0/538162743 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:48.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.938+0000 7f51eed9d700 1 -- 172.21.15.16:0/538162743 >> v2:172.21.15.132:3300/0 conn(0x7f51f00cc7c0 msgr2=0x7f51f0161230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:48.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.938+0000 7f51ef59e700 1 --2- 172.21.15.16:0/538162743 >> v2:172.21.15.132:3300/0 conn(0x7f51f00cc7c0 0x7f51f0161230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:48.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.938+0000 7f51eed9d700 1 --2- 172.21.15.16:0/538162743 >> v2:172.21.15.132:3300/0 conn(0x7f51f00cc7c0 0x7f51f0161230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:48.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.938+0000 7f51edd9b700 1 -- 172.21.15.16:0/538162743 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 426448609 0 0) 0x7f51f0071040 con 0x7f51f0070b30 2023-05-20T01:28:48.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.938+0000 7f51eed9d700 1 -- 172.21.15.16:0/538162743 >> v1:172.21.15.16:6789/0 conn(0x7f51f0070b30 legacy=0x7f51f00cc0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:48.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.938+0000 7f51eed9d700 1 -- 172.21.15.16:0/538162743 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51f0071570 con 0x7f51f015cc70 2023-05-20T01:28:48.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.938+0000 7f51ef59e700 1 --2- 172.21.15.16:0/538162743 >> v2:172.21.15.132:3300/0 conn(0x7f51f00cc7c0 0x7f51f0161230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:28:48.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.938+0000 7f51eed9d700 1 --2- 172.21.15.16:0/538162743 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51f015cc70 0x7f51f00718b0 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f51f0070910 tx=0x7f51e4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=520bc24abbd3e570 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:48.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.939+0000 7f51edd9b700 1 -- 172.21.15.16:0/538162743 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51e400b870 con 0x7f51f015cc70 2023-05-20T01:28:48.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.939+0000 7f51edd9b700 1 -- 172.21.15.16:0/538162743 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51e400b9d0 con 0x7f51f015cc70 2023-05-20T01:28:48.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.939+0000 7f51edd9b700 1 -- 172.21.15.16:0/538162743 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51e400bd60 con 0x7f51f015cc70 2023-05-20T01:28:48.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.939+0000 7f51f51fa700 1 -- 172.21.15.16:0/538162743 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51f015cc70 msgr2=0x7f51f00718b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:48.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.939+0000 7f51f51fa700 1 --2- 172.21.15.16:0/538162743 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51f015cc70 0x7f51f00718b0 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f51f0070910 tx=0x7f51e4009bd0 comp rx=0 tx=0).stop 2023-05-20T01:28:48.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.939+0000 7f51f51fa700 1 -- 172.21.15.16:0/538162743 shutdown_connections 2023-05-20T01:28:48.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.939+0000 7f51f51fa700 1 --2- 172.21.15.16:0/538162743 >> v2:172.21.15.132:3300/0 conn(0x7f51f00cc7c0 0x7f51f0161230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:48.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.939+0000 7f51f51fa700 1 --2- 172.21.15.16:0/538162743 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51f015cc70 0x7f51f00718b0 unknown :-1 s=CLOSED pgs=1709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:48.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.939+0000 7f51f51fa700 1 -- 172.21.15.16:0/538162743 >> 172.21.15.16:0/538162743 conn(0x7f51f00b1590 msgr2=0x7f51f00b1960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:48.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.939+0000 7f51f51fa700 1 -- 172.21.15.16:0/538162743 shutdown_connections 2023-05-20T01:28:48.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.939+0000 7f51f51fa700 1 -- 172.21.15.16:0/538162743 wait complete. 2023-05-20T01:28:48.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.941+0000 7f51f51fa700 1 Processor -- start 2023-05-20T01:28:48.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.941+0000 7f51f51fa700 1 -- start start 2023-05-20T01:28:48.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.941+0000 7f51f51fa700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f51f0070b30 0x7f51f0151690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:48.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.941+0000 7f51f51fa700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51f00cc7c0 0x7f51f01f3e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:48.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.941+0000 7f51f51fa700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f51f015cc70 0x7f51f015f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:48.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.941+0000 7f51f51fa700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51f01f3cc0 con 0x7f51f00cc7c0 2023-05-20T01:28:48.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.941+0000 7f51f51fa700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51f0151bd0 con 0x7f51f015cc70 2023-05-20T01:28:48.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.941+0000 7f51f51fa700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51f015dd30 con 0x7f51f0070b30 2023-05-20T01:28:48.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.942+0000 7f51ee59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51f00cc7c0 0x7f51f01f3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:48.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.942+0000 7f51ee59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51f00cc7c0 0x7f51f01f3e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40682/0 (socket says 172.21.15.16:40682) 2023-05-20T01:28:48.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.942+0000 7f51ee59c700 1 -- 172.21.15.16:0/37430570 learned_addr learned my addr 172.21.15.16:0/37430570 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:48.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.942+0000 7f51eed9d700 1 --2- 172.21.15.16:0/37430570 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f51f0070b30 0x7f51f0151690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:48.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.942+0000 7f51ef59e700 1 --2- 172.21.15.16:0/37430570 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f51f015cc70 0x7f51f015f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:48.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.942+0000 7f51ee59c700 1 -- 172.21.15.16:0/37430570 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f51f0070b30 msgr2=0x7f51f0151690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:48.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.942+0000 7f51ee59c700 1 --2- 172.21.15.16:0/37430570 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f51f0070b30 0x7f51f0151690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:48.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.942+0000 7f51ee59c700 1 -- 172.21.15.16:0/37430570 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f51f015cc70 msgr2=0x7f51f015f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:48.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.942+0000 7f51ee59c700 1 --2- 172.21.15.16:0/37430570 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f51f015cc70 0x7f51f015f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:48.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.942+0000 7f51ee59c700 1 -- 172.21.15.16:0/37430570 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51e4009550 con 0x7f51f00cc7c0 2023-05-20T01:28:48.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.942+0000 7f51ef59e700 1 --2- 172.21.15.16:0/37430570 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f51f015cc70 0x7f51f015f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:28:48.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.942+0000 7f51ee59c700 1 --2- 172.21.15.16:0/37430570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51f00cc7c0 0x7f51f01f3e70 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f51e000c520 tx=0x7f51e000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:48.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.942+0000 7f51d77fe700 1 -- 172.21.15.16:0/37430570 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51e0004c80 con 0x7f51f00cc7c0 2023-05-20T01:28:48.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.942+0000 7f51d77fe700 1 -- 172.21.15.16:0/37430570 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51e0004de0 con 0x7f51f00cc7c0 2023-05-20T01:28:48.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.942+0000 7f51f51fa700 1 -- 172.21.15.16:0/37430570 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51f0160d10 con 0x7f51f00cc7c0 2023-05-20T01:28:48.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.942+0000 7f51d77fe700 1 -- 172.21.15.16:0/37430570 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51e0007310 con 0x7f51f00cc7c0 2023-05-20T01:28:48.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.942+0000 7f51f51fa700 1 -- 172.21.15.16:0/37430570 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51f0160a30 con 0x7f51f00cc7c0 2023-05-20T01:28:48.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.943+0000 7f51f51fa700 1 -- 172.21.15.16:0/37430570 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51f015d040 con 0x7f51f00cc7c0 2023-05-20T01:28:48.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.944+0000 7f51d77fe700 1 -- 172.21.15.16:0/37430570 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f51e0007470 con 0x7f51f00cc7c0 2023-05-20T01:28:48.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.944+0000 7f51d77fe700 1 --2- 172.21.15.16:0/37430570 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f51dc06ace0 0x7f51dc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:48.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.944+0000 7f51d77fe700 1 -- 172.21.15.16:0/37430570 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6468+0+0 (secure 0 0 0) 0x7f51e0089340 con 0x7f51f00cc7c0 2023-05-20T01:28:48.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.944+0000 7f51eed9d700 1 --2- 172.21.15.16:0/37430570 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f51dc06ace0 0x7f51dc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:48.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.945+0000 7f51eed9d700 1 --2- 172.21.15.16:0/37430570 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f51dc06ace0 0x7f51dc06d190 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f51e4014040 tx=0x7f51e4008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:48.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:48.949+0000 7f51d77fe700 1 -- 172.21.15.16:0/37430570 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f51e00541b0 con 0x7f51f00cc7c0 2023-05-20T01:28:49.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.194+0000 7f51f51fa700 1 -- 172.21.15.16:0/37430570 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f51f004e8c0 con 0x7f51f00cc7c0 2023-05-20T01:28:49.598 INFO:tasks.ceph.mon.a.smithi016.stderr:2023-05-20T01:28:49.594+0000 7f49eb8ae700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-05-20T01:28:49.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.602+0000 7f51d77fe700 1 -- 172.21.15.16:0/37430570 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v106) v1 ==== 161+0+0 (secure 0 0 0) 0x7f51e00577d0 con 0x7f51f00cc7c0 2023-05-20T01:28:49.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.604+0000 7f51f51fa700 1 -- 172.21.15.16:0/37430570 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f51dc06ace0 msgr2=0x7f51dc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:49.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.604+0000 7f51f51fa700 1 --2- 172.21.15.16:0/37430570 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f51dc06ace0 0x7f51dc06d190 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f51e4014040 tx=0x7f51e4008030 comp rx=0 tx=0).stop 2023-05-20T01:28:49.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.604+0000 7f51f51fa700 1 -- 172.21.15.16:0/37430570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51f00cc7c0 msgr2=0x7f51f01f3e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:49.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.604+0000 7f51f51fa700 1 --2- 172.21.15.16:0/37430570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51f00cc7c0 0x7f51f01f3e70 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f51e000c520 tx=0x7f51e000c830 comp rx=0 tx=0).stop 2023-05-20T01:28:49.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.605+0000 7f51f51fa700 1 -- 172.21.15.16:0/37430570 shutdown_connections 2023-05-20T01:28:49.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.605+0000 7f51f51fa700 1 --2- 172.21.15.16:0/37430570 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f51f015cc70 0x7f51f015f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:49.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.605+0000 7f51f51fa700 1 --2- 172.21.15.16:0/37430570 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f51dc06ace0 0x7f51dc06d190 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:49.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.605+0000 7f51f51fa700 1 --2- 172.21.15.16:0/37430570 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f51f0070b30 0x7f51f0151690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:49.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.605+0000 7f51f51fa700 1 --2- 172.21.15.16:0/37430570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51f00cc7c0 0x7f51f01f3e70 unknown :-1 s=CLOSED pgs=1710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:49.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.605+0000 7f51f51fa700 1 -- 172.21.15.16:0/37430570 >> 172.21.15.16:0/37430570 conn(0x7f51f00b1590 msgr2=0x7f51f00cb550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:49.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.605+0000 7f51f51fa700 1 -- 172.21.15.16:0/37430570 shutdown_connections 2023-05-20T01:28:49.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.605+0000 7f51f51fa700 1 -- 172.21.15.16:0/37430570 wait complete. 2023-05-20T01:28:49.611 INFO:tasks.ceph.mds.a.smithi016.stderr: -2411> 2023-05-20T01:28:45.559+0000 7fa7fec3c700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-05-20T01:28:49.611 INFO:teuthology.orchestra.run.smithi016.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-05-20T01:28:49.624 DEBUG:teuthology.orchestra.run.smithi016:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_extents 2023-05-20T01:28:49.638 INFO:tasks.ceph.mds.a.smithi016.stdout:starting mds.a at 2023-05-20T01:28:49.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.663+0000 7fdd59b7c7c0 1 -- start start 2023-05-20T01:28:49.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.663+0000 7fdd59b7c7c0 1 Processor -- start 2023-05-20T01:28:49.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.663+0000 7fdd59b7c7c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563b4b4c4800 0x563b4b55e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:49.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.663+0000 7fdd59b7c7c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b4b4c4c00 0x563b4b55eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:49.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.663+0000 7fdd59b7c7c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563b4b4c5000 0x563b4b55f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:49.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.663+0000 7fdd59b7c7c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b4b501cc0 con 0x563b4b4c4c00 2023-05-20T01:28:49.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.663+0000 7fdd59b7c7c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b4b501400 con 0x563b4b4c5000 2023-05-20T01:28:49.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.663+0000 7fdd59b7c7c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x563b4b501e00 con 0x563b4b4c4800 2023-05-20T01:28:49.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.663+0000 7fdd57329700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b4b4c4c00 0x563b4b55eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:49.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.663+0000 7fdd57329700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b4b4c4c00 0x563b4b55eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40708/0 (socket says 172.21.15.16:40708) 2023-05-20T01:28:49.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.663+0000 7fdd57329700 1 -- 172.21.15.16:0/1784838100 learned_addr learned my addr 172.21.15.16:0/1784838100 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:49.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.664+0000 7fdd57b2a700 1 --2- 172.21.15.16:0/1784838100 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563b4b4c4800 0x563b4b55e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:49.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.664+0000 7fdd5832b700 1 --2- 172.21.15.16:0/1784838100 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563b4b4c5000 0x563b4b55f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:49.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.664+0000 7fdd57329700 1 -- 172.21.15.16:0/1784838100 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563b4b4c4800 msgr2=0x563b4b55e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:49.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.664+0000 7fdd57329700 1 --2- 172.21.15.16:0/1784838100 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563b4b4c4800 0x563b4b55e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:49.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.664+0000 7fdd57329700 1 -- 172.21.15.16:0/1784838100 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563b4b4c5000 msgr2=0x563b4b55f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:49.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.664+0000 7fdd57329700 1 --2- 172.21.15.16:0/1784838100 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563b4b4c5000 0x563b4b55f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:49.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.664+0000 7fdd57329700 1 -- 172.21.15.16:0/1784838100 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563b4b4e8d00 con 0x563b4b4c4c00 2023-05-20T01:28:49.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.664+0000 7fdd57b2a700 1 --2- 172.21.15.16:0/1784838100 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563b4b4c4800 0x563b4b55e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:28:49.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.664+0000 7fdd5832b700 1 --2- 172.21.15.16:0/1784838100 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563b4b4c5000 0x563b4b55f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:28:49.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.664+0000 7fdd57329700 1 --2- 172.21.15.16:0/1784838100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b4b4c4c00 0x563b4b55eb00 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x563b4c290e70 tx=0x563b4c29a3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7df2047ec0441325 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:49.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.664+0000 7fdd55325700 1 -- 172.21.15.16:0/1784838100 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x563b4b4f9600 con 0x563b4b4c4c00 2023-05-20T01:28:49.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.664+0000 7fdd55325700 1 -- 172.21.15.16:0/1784838100 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x563b4b516480 con 0x563b4b4c4c00 2023-05-20T01:28:49.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.664+0000 7fdd55325700 1 -- 172.21.15.16:0/1784838100 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x563b4b4f91e0 con 0x563b4b4c4c00 2023-05-20T01:28:49.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.664+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1784838100 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563b4b4e96c0 con 0x563b4b4c4c00 2023-05-20T01:28:49.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.664+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1784838100 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x563b4b4e9520 con 0x563b4b4c4c00 2023-05-20T01:28:49.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.664+0000 7fdd59b7c7c0 4 waiting for MDS map... 2023-05-20T01:28:49.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.664+0000 7fdd55325700 1 -- 172.21.15.16:0/1784838100 <== mon.0 v2:172.21.15.16:3300/0 4 ==== osd_map(138..138 src has 1..138) v4 ==== 6554+0+0 (secure 0 0 0) 0x563b4b506c40 con 0x563b4b4c4c00 2023-05-20T01:28:49.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.664+0000 7fdd55325700 1 -- 172.21.15.16:0/1784838100 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 106) v1 ==== 2573+0+0 (secure 0 0 0) 0x563b4c33c000 con 0x563b4b4c4c00 2023-05-20T01:28:49.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.664+0000 7fdd59b7c7c0 4 Got MDS map 106 2023-05-20T01:28:49.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.665+0000 7fdd59b7c7c0 4 datascan.main: Using metadata pool output 2023-05-20T01:28:49.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.665+0000 7fdd59b7c7c0 4 datascan.main: connecting to RADOS... 2023-05-20T01:28:49.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.665+0000 7fdd59b7c7c0 1 Processor -- start 2023-05-20T01:28:49.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.665+0000 7fdd59b7c7c0 1 -- start start 2023-05-20T01:28:49.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.666+0000 7fdd59b7c7c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563b4b4c5c00 0x563b4b55f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:49.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.666+0000 7fdd59b7c7c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b4b7e8000 0x563b4b55fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:49.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.666+0000 7fdd59b7c7c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563b4b7e8400 0x563b4b560100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:49.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.666+0000 7fdd59b7c7c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b4b500a00 con 0x563b4b7e8000 2023-05-20T01:28:49.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.666+0000 7fdd59b7c7c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b4b501400 con 0x563b4b4c5c00 2023-05-20T01:28:49.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.666+0000 7fdd59b7c7c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x563b4b500b40 con 0x563b4b7e8400 2023-05-20T01:28:49.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.666+0000 7fdd57b2a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b4b7e8000 0x563b4b55fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:49.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.666+0000 7fdd57b2a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b4b7e8000 0x563b4b55fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40714/0 (socket says 172.21.15.16:40714) 2023-05-20T01:28:49.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.666+0000 7fdd57b2a700 1 -- 172.21.15.16:0/129746055 learned_addr learned my addr 172.21.15.16:0/129746055 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:49.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.666+0000 7fdd5832b700 1 --2- 172.21.15.16:0/129746055 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563b4b4c5c00 0x563b4b55f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:49.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.666+0000 7fdd57329700 1 --2- 172.21.15.16:0/129746055 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563b4b7e8400 0x563b4b560100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:49.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.666+0000 7fdd57b2a700 1 -- 172.21.15.16:0/129746055 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563b4b7e8400 msgr2=0x563b4b560100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:49.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.666+0000 7fdd57b2a700 1 --2- 172.21.15.16:0/129746055 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563b4b7e8400 0x563b4b560100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:49.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.666+0000 7fdd57b2a700 1 -- 172.21.15.16:0/129746055 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563b4b4c5c00 msgr2=0x563b4b55f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:49.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.666+0000 7fdd57b2a700 1 --2- 172.21.15.16:0/129746055 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563b4b4c5c00 0x563b4b55f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:49.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.666+0000 7fdd57b2a700 1 -- 172.21.15.16:0/129746055 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563b4b4e9380 con 0x563b4b7e8000 2023-05-20T01:28:49.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.666+0000 7fdd57b2a700 1 --2- 172.21.15.16:0/129746055 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b4b7e8000 0x563b4b55fb80 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x563b4b7f00f0 tx=0x563b4c35ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ffeedbb53585f3b server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:49.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.666+0000 7fdd53321700 1 -- 172.21.15.16:0/129746055 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x563b4b4f9600 con 0x563b4b7e8000 2023-05-20T01:28:49.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.666+0000 7fdd53321700 1 -- 172.21.15.16:0/129746055 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x563b4c356900 con 0x563b4b7e8000 2023-05-20T01:28:49.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.667+0000 7fdd53321700 1 -- 172.21.15.16:0/129746055 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x563b4b4f91e0 con 0x563b4b7e8000 2023-05-20T01:28:49.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.667+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/129746055 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b4b7e8000 msgr2=0x563b4b55fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:49.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.667+0000 7fdd59b7c7c0 1 --2- 172.21.15.16:0/129746055 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b4b7e8000 0x563b4b55fb80 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x563b4b7f00f0 tx=0x563b4c35ecc0 comp rx=0 tx=0).stop 2023-05-20T01:28:49.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.667+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/129746055 shutdown_connections 2023-05-20T01:28:49.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.667+0000 7fdd59b7c7c0 1 --2- 172.21.15.16:0/129746055 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563b4b7e8400 0x563b4b560100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:49.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.667+0000 7fdd59b7c7c0 1 --2- 172.21.15.16:0/129746055 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563b4b4c5c00 0x563b4b55f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:49.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.667+0000 7fdd59b7c7c0 1 --2- 172.21.15.16:0/129746055 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b4b7e8000 0x563b4b55fb80 unknown :-1 s=CLOSED pgs=1716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:49.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.667+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/129746055 >> 172.21.15.16:0/129746055 conn(0x563b4b4c5800 msgr2=0x563b4b5a1800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:49.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.667+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/129746055 shutdown_connections 2023-05-20T01:28:49.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.667+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/129746055 wait complete. 2023-05-20T01:28:49.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.668+0000 7fdd59b7c7c0 1 Processor -- start 2023-05-20T01:28:49.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.668+0000 7fdd59b7c7c0 1 -- start start 2023-05-20T01:28:49.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.669+0000 7fdd59b7c7c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b4b4c5800 0x563b4b561180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:49.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.669+0000 7fdd59b7c7c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563b4b7e8000 0x563b4b560c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:49.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.669+0000 7fdd59b7c7c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563b4b7e8400 0x563b4b560680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:49.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.669+0000 7fdd59b7c7c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b4b500c80 con 0x563b4b4c5800 2023-05-20T01:28:49.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.669+0000 7fdd59b7c7c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b4b500000 con 0x563b4b7e8400 2023-05-20T01:28:49.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.669+0000 7fdd59b7c7c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x563b4b501180 con 0x563b4b7e8000 2023-05-20T01:28:49.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.669+0000 7fdd5832b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b4b4c5800 0x563b4b561180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:49.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.669+0000 7fdd5832b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b4b4c5800 0x563b4b561180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40718/0 (socket says 172.21.15.16:40718) 2023-05-20T01:28:49.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.669+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 learned_addr learned my addr 172.21.15.16:0/1962333830 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:49.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.669+0000 7fdd57b2a700 1 --2- 172.21.15.16:0/1962333830 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563b4b7e8000 0x563b4b560c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:49.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.669+0000 7fdd57329700 1 --2- 172.21.15.16:0/1962333830 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563b4b7e8400 0x563b4b560680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:49.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.669+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563b4b7e8000 msgr2=0x563b4b560c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:49.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.669+0000 7fdd5832b700 1 --2- 172.21.15.16:0/1962333830 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563b4b7e8000 0x563b4b560c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:49.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.669+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563b4b7e8400 msgr2=0x563b4b560680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:49.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.669+0000 7fdd5832b700 1 --2- 172.21.15.16:0/1962333830 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563b4b7e8400 0x563b4b560680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:49.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.669+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563b4b8041a0 con 0x563b4b4c5800 2023-05-20T01:28:49.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.669+0000 7fdd57329700 1 --2- 172.21.15.16:0/1962333830 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563b4b7e8400 0x563b4b560680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:28:49.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.669+0000 7fdd5832b700 1 --2- 172.21.15.16:0/1962333830 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b4b4c5800 0x563b4b561180 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x563b4b7f13e0 tx=0x563b4c29a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:49.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.669+0000 7fdd51b1e700 1 -- 172.21.15.16:0/1962333830 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x563b4b4f9080 con 0x563b4b4c5800 2023-05-20T01:28:49.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.670+0000 7fdd51b1e700 1 -- 172.21.15.16:0/1962333830 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x563b4b517980 con 0x563b4b4c5800 2023-05-20T01:28:49.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.670+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x563b4b804000 con 0x563b4b4c5800 2023-05-20T01:28:49.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.670+0000 7fdd51b1e700 1 -- 172.21.15.16:0/1962333830 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x563b4b4f9600 con 0x563b4b4c5800 2023-05-20T01:28:49.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.670+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563b4b804340 con 0x563b4b4c5800 2023-05-20T01:28:49.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.670+0000 7fdd59b7c7c0 4 datascan.init: resolving metadata pool 18 2023-05-20T01:28:49.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.671+0000 7fdd51b1e700 1 -- 172.21.15.16:0/1962333830 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x563b4b526840 con 0x563b4b4c5800 2023-05-20T01:28:49.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.671+0000 7fdd51b1e700 1 --2- 172.21.15.16:0/1962333830 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x563b4b80a400 0x563b4b561700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:49.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.671+0000 7fdd51b1e700 1 -- 172.21.15.16:0/1962333830 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6554+0+0 (secure 0 0 0) 0x563b4b506a80 con 0x563b4b4c5800 2023-05-20T01:28:49.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.671+0000 7fdd59b7c7c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-05-20T01:28:49.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.671+0000 7fdd59b7c7c0 4 datascan.main: data pool 'cephfs_data' has ID 19 2023-05-20T01:28:49.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.671+0000 7fdd59b7c7c0 4 datascan.main: opening data pool 'cephfs_data' 2023-05-20T01:28:49.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.671+0000 7fdd57b2a700 1 --2- 172.21.15.16:0/1962333830 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x563b4b80a400 0x563b4b561700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:49.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.671+0000 7fdd59b7c7c0 1 --2- 172.21.15.16:0/1962333830 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x563b4b7e8800 0x563b4b560100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:49.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.671+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:1 19.0 19:00000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4c33e700 con 0x563b4b7e8800 2023-05-20T01:28:49.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.671+0000 7fdd57329700 1 --2- 172.21.15.16:0/1962333830 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x563b4b7e8800 0x563b4b560100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:49.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.672+0000 7fdd57b2a700 1 --2- 172.21.15.16:0/1962333830 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x563b4b80a400 0x563b4b561700 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x563b4b7f00f0 tx=0x563b4c35ec00 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:49.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.672+0000 7fdd57329700 1 --2- 172.21.15.16:0/1962333830 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x563b4b7e8800 0x563b4b560100 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:49.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.672+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.3 v2:172.21.15.16:6800/3669781890 1 ==== osd_op_reply(1 [pgnls start_epoch 138 out=81b] v136'10 uv10 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b7e8800 2023-05-20T01:28:49.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.672+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:2 19.0 19:012cdbc7:::10000000206.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4c33ea80 con 0x563b4b7e8800 2023-05-20T01:28:49.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.673+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.3 v2:172.21.15.16:6800/3669781890 2 ==== osd_op_reply(2 10000000206.00000000 [stat out=16b] v0'0 uv10 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b7e8800 2023-05-20T01:28:49.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.673+0000 7fdd59b7c7c0 10 datascan.operator(): handling object 1099511628294.0 2023-05-20T01:28:49.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.673+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:3 19.0 19:012cdbc7:::10000000206.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4c33ee00 con 0x563b4b7e8800 2023-05-20T01:28:49.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.674+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.3 v2:172.21.15.16:6800/3669781890 3 ==== osd_op_reply(3 10000000206.00000000 [call] v138'11 uv11 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b7e8800 2023-05-20T01:28:49.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.674+0000 7fdd59b7c7c0 1 --2- 172.21.15.16:0/1962333830 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x563b4b7e8c00 0x563b4b55fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:49.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.674+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:4 19.20 19:04000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4c33e700 con 0x563b4b7e8c00 2023-05-20T01:28:49.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.675+0000 7fdd5832b700 1 --2- 172.21.15.16:0/1962333830 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x563b4b7e8c00 0x563b4b55fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:49.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.675+0000 7fdd5832b700 1 --2- 172.21.15.16:0/1962333830 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x563b4b7e8c00 0x563b4b55fb80 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:49.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.676+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.4 v2:172.21.15.132:6817/506317223 1 ==== osd_op_reply(4 [pgnls start_epoch 138 out=81b] v136'11 uv11 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b7e8c00 2023-05-20T01:28:49.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.676+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:5 19.20 19:05552577:::1000000020b.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4c33ea80 con 0x563b4b7e8c00 2023-05-20T01:28:49.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.676+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.4 v2:172.21.15.132:6817/506317223 2 ==== osd_op_reply(5 1000000020b.00000000 [stat out=16b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b7e8c00 2023-05-20T01:28:49.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.676+0000 7fdd59b7c7c0 10 datascan.operator(): handling object 1099511628299.0 2023-05-20T01:28:49.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.676+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:6 19.20 19:05552577:::1000000020b.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4c33ee00 con 0x563b4b7e8c00 2023-05-20T01:28:49.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.678+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.4 v2:172.21.15.132:6817/506317223 3 ==== osd_op_reply(6 1000000020b.00000000 [call] v138'12 uv12 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b7e8c00 2023-05-20T01:28:49.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.678+0000 7fdd59b7c7c0 1 --2- 172.21.15.16:0/1962333830 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x563b4b7e9000 0x563b4b55f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:49.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.678+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:7 19.10 19:08000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4c33f180 con 0x563b4b7e9000 2023-05-20T01:28:49.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.678+0000 7fdd57b2a700 1 --2- 172.21.15.16:0/1962333830 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x563b4b7e9000 0x563b4b55f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:49.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.678+0000 7fdd57b2a700 1 --2- 172.21.15.16:0/1962333830 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x563b4b7e9000 0x563b4b55f600 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:49.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.678+0000 7fdd57b2a700 1 -- 172.21.15.16:0/1962333830 <== osd.1 v2:172.21.15.16:6808/3860264220 1 ==== osd_op_reply(7 [pgnls start_epoch 138 out=101b] v136'14 uv14 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x563b4b7e58c0 con 0x563b4b7e9000 2023-05-20T01:28:49.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.678+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:8 19.10 19:083d5826:::1000000022a.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4c33e700 con 0x563b4b7e9000 2023-05-20T01:28:49.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.679+0000 7fdd57b2a700 1 -- 172.21.15.16:0/1962333830 <== osd.1 v2:172.21.15.16:6808/3860264220 2 ==== osd_op_reply(8 1000000022a.00000000 [stat out=16b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b4b7e58c0 con 0x563b4b7e9000 2023-05-20T01:28:49.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.679+0000 7fdd59b7c7c0 10 datascan.operator(): handling object 1099511628330.0 2023-05-20T01:28:49.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.679+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:9 19.10 19:083d5826:::1000000022a.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4c33ea80 con 0x563b4b7e9000 2023-05-20T01:28:49.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.680+0000 7fdd57b2a700 1 -- 172.21.15.16:0/1962333830 <== osd.1 v2:172.21.15.16:6808/3860264220 3 ==== osd_op_reply(9 1000000022a.00000000 [call] v138'15 uv15 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563b4b7e58c0 con 0x563b4b7e9000 2023-05-20T01:28:49.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.680+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:10 19.10 19:08500202::::head [pgnls start_epoch 138 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4c33f500 con 0x563b4b7e9000 2023-05-20T01:28:49.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.680+0000 7fdd57b2a700 1 -- 172.21.15.16:0/1962333830 <== osd.1 v2:172.21.15.16:6808/3860264220 4 ==== osd_op_reply(10 [pgnls start_epoch 138 out=101b] v138'15 uv15 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x563b4b7e58c0 con 0x563b4b7e9000 2023-05-20T01:28:49.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.680+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:11 19.10 19:08500202:::1000000023e.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4c33f880 con 0x563b4b7e9000 2023-05-20T01:28:49.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.680+0000 7fdd57b2a700 1 -- 172.21.15.16:0/1962333830 <== osd.1 v2:172.21.15.16:6808/3860264220 5 ==== osd_op_reply(11 1000000023e.00000000 [stat out=16b] v0'0 uv13 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b4b7e58c0 con 0x563b4b7e9000 2023-05-20T01:28:49.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.680+0000 7fdd59b7c7c0 10 datascan.operator(): handling object 1099511628350.0 2023-05-20T01:28:49.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.681+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:12 19.10 19:08500202:::1000000023e.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4c33fc00 con 0x563b4b7e9000 2023-05-20T01:28:49.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.683+0000 7fdd57b2a700 1 -- 172.21.15.16:0/1962333830 <== osd.1 v2:172.21.15.16:6808/3860264220 6 ==== osd_op_reply(12 1000000023e.00000000 [call] v138'16 uv16 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563b4b7e58c0 con 0x563b4b7e9000 2023-05-20T01:28:49.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.683+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:13 19.10 19:08b99fea::::head [pgnls start_epoch 138 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4c33ea80 con 0x563b4b7e9000 2023-05-20T01:28:49.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.683+0000 7fdd57b2a700 1 -- 172.21.15.16:0/1962333830 <== osd.1 v2:172.21.15.16:6808/3860264220 7 ==== osd_op_reply(13 [pgnls start_epoch 138 out=101b] v138'16 uv16 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x563b4b7e58c0 con 0x563b4b7e9000 2023-05-20T01:28:49.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.683+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:14 19.10 19:08b99fea:::100000001fd.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4c33f500 con 0x563b4b7e9000 2023-05-20T01:28:49.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.683+0000 7fdd57b2a700 1 -- 172.21.15.16:0/1962333830 <== osd.1 v2:172.21.15.16:6808/3860264220 8 ==== osd_op_reply(14 100000001fd.00000000 [stat out=16b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b4b7e58c0 con 0x563b4b7e9000 2023-05-20T01:28:49.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.683+0000 7fdd59b7c7c0 10 datascan.operator(): handling object 1099511628285.0 2023-05-20T01:28:49.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.683+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:15 19.10 19:08b99fea:::100000001fd.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4c33f880 con 0x563b4b7e9000 2023-05-20T01:28:49.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.685+0000 7fdd57b2a700 1 -- 172.21.15.16:0/1962333830 <== osd.1 v2:172.21.15.16:6808/3860264220 9 ==== osd_op_reply(15 100000001fd.00000000 [call] v138'17 uv17 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563b4b7e58c0 con 0x563b4b7e9000 2023-05-20T01:28:49.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.685+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:16 19.10 19:08e6f8ca::::head [pgnls start_epoch 138 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4c33f180 con 0x563b4b7e9000 2023-05-20T01:28:49.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.686+0000 7fdd57b2a700 1 -- 172.21.15.16:0/1962333830 <== osd.1 v2:172.21.15.16:6808/3860264220 10 ==== osd_op_reply(16 [pgnls start_epoch 138 out=81b] v138'17 uv17 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x563b4b7e58c0 con 0x563b4b7e9000 2023-05-20T01:28:49.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.686+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:17 19.10 19:08e6f8ca:::10000000257.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4c33e700 con 0x563b4b7e9000 2023-05-20T01:28:49.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.686+0000 7fdd57b2a700 1 -- 172.21.15.16:0/1962333830 <== osd.1 v2:172.21.15.16:6808/3860264220 11 ==== osd_op_reply(17 10000000257.00000000 [stat out=16b] v0'0 uv14 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b4b7e58c0 con 0x563b4b7e9000 2023-05-20T01:28:49.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.686+0000 7fdd59b7c7c0 10 datascan.operator(): handling object 1099511628375.0 2023-05-20T01:28:49.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.686+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:18 19.10 19:08e6f8ca:::10000000257.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4c33fc00 con 0x563b4b7e9000 2023-05-20T01:28:49.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.688+0000 7fdd57b2a700 1 -- 172.21.15.16:0/1962333830 <== osd.1 v2:172.21.15.16:6808/3860264220 12 ==== osd_op_reply(18 10000000257.00000000 [call] v138'18 uv18 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563b4b7e58c0 con 0x563b4b7e9000 2023-05-20T01:28:49.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.688+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:19 19.30 19:0c000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4c33ea80 con 0x563b4b7e8800 2023-05-20T01:28:49.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.688+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.3 v2:172.21.15.16:6800/3669781890 4 ==== osd_op_reply(19 [pgnls start_epoch 138 out=101b] v136'17 uv17 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b7e8800 2023-05-20T01:28:49.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.688+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:20 19.30 19:0c00cb85:::1000000021d.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4c33f500 con 0x563b4b7e8800 2023-05-20T01:28:49.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.689+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.3 v2:172.21.15.16:6800/3669781890 5 ==== osd_op_reply(20 1000000021d.00000000 [stat out=16b] v0'0 uv14 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b7e8800 2023-05-20T01:28:49.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.689+0000 7fdd59b7c7c0 10 datascan.operator(): handling object 1099511628317.0 2023-05-20T01:28:49.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.689+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:21 19.30 19:0c00cb85:::1000000021d.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4c33f880 con 0x563b4b7e8800 2023-05-20T01:28:49.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.690+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.3 v2:172.21.15.16:6800/3669781890 6 ==== osd_op_reply(21 1000000021d.00000000 [call] v138'18 uv18 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b7e8800 2023-05-20T01:28:49.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.690+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:22 19.30 19:0c28cce1::::head [pgnls start_epoch 138 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91e000 con 0x563b4b7e8800 2023-05-20T01:28:49.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.690+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.3 v2:172.21.15.16:6800/3669781890 7 ==== osd_op_reply(22 [pgnls start_epoch 138 out=101b] v138'18 uv18 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b7e8800 2023-05-20T01:28:49.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.690+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:23 19.30 19:0c28cce1:::10000000244.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91e380 con 0x563b4b7e8800 2023-05-20T01:28:49.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.690+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.3 v2:172.21.15.16:6800/3669781890 8 ==== osd_op_reply(23 10000000244.00000000 [stat out=16b] v0'0 uv16 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b7e8800 2023-05-20T01:28:49.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.690+0000 7fdd59b7c7c0 10 datascan.operator(): handling object 1099511628356.0 2023-05-20T01:28:49.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.690+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:24 19.30 19:0c28cce1:::10000000244.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91e700 con 0x563b4b7e8800 2023-05-20T01:28:49.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.691+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.3 v2:172.21.15.16:6800/3669781890 9 ==== osd_op_reply(24 10000000244.00000000 [call] v138'19 uv19 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b7e8800 2023-05-20T01:28:49.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.692+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:25 19.30 19:0cc9e906::::head [pgnls start_epoch 138 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91ea80 con 0x563b4b7e8800 2023-05-20T01:28:49.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.692+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.3 v2:172.21.15.16:6800/3669781890 10 ==== osd_op_reply(25 [pgnls start_epoch 138 out=101b] v138'19 uv19 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b7e8800 2023-05-20T01:28:49.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.692+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:26 19.30 19:0cc9e906:::10000000227.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91ee00 con 0x563b4b7e8800 2023-05-20T01:28:49.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.692+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.3 v2:172.21.15.16:6800/3669781890 11 ==== osd_op_reply(26 10000000227.00000000 [stat out=16b] v0'0 uv15 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b7e8800 2023-05-20T01:28:49.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.692+0000 7fdd59b7c7c0 10 datascan.operator(): handling object 1099511628327.0 2023-05-20T01:28:49.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.692+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:27 19.30 19:0cc9e906:::10000000227.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91f180 con 0x563b4b7e8800 2023-05-20T01:28:49.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.693+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.3 v2:172.21.15.16:6800/3669781890 12 ==== osd_op_reply(27 10000000227.00000000 [call] v138'20 uv20 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b7e8800 2023-05-20T01:28:49.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.693+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:28 19.30 19:0e3d5a27::::head [pgnls start_epoch 138 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91f500 con 0x563b4b7e8800 2023-05-20T01:28:49.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.693+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.3 v2:172.21.15.16:6800/3669781890 13 ==== osd_op_reply(28 [pgnls start_epoch 138 out=81b] v138'20 uv20 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b7e8800 2023-05-20T01:28:49.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.693+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:29 19.30 19:0e3d5a27:::10000000250.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91f880 con 0x563b4b7e8800 2023-05-20T01:28:49.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.694+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.3 v2:172.21.15.16:6800/3669781890 14 ==== osd_op_reply(29 10000000250.00000000 [stat out=16b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b7e8800 2023-05-20T01:28:49.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.694+0000 7fdd59b7c7c0 10 datascan.operator(): handling object 1099511628368.0 2023-05-20T01:28:49.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.694+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:30 19.30 19:0e3d5a27:::10000000250.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91e380 con 0x563b4b7e8800 2023-05-20T01:28:49.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.695+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.3 v2:172.21.15.16:6800/3669781890 15 ==== osd_op_reply(30 10000000250.00000000 [call] v138'21 uv21 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b7e8800 2023-05-20T01:28:49.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.695+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:31 19.8 19:10000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91e700 con 0x563b4b7e9000 2023-05-20T01:28:49.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.695+0000 7fdd57b2a700 1 -- 172.21.15.16:0/1962333830 <== osd.1 v2:172.21.15.16:6808/3860264220 13 ==== osd_op_reply(31 [pgnls start_epoch 138 out=49b] v136'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563b4b7e58c0 con 0x563b4b7e9000 2023-05-20T01:28:49.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.695+0000 7fdd52b20700 1 --2- 172.21.15.16:0/1962333830 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x563b4b80a800 0x563b4b924000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:49.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.695+0000 7fdd52b20700 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:32 19.28 19:14000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91fc00 con 0x563b4b80a800 2023-05-20T01:28:49.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.695+0000 7fdd57329700 1 --2- 172.21.15.16:0/1962333830 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x563b4b80a800 0x563b4b924000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:49.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.695+0000 7fdd57329700 1 --2- 172.21.15.16:0/1962333830 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x563b4b80a800 0x563b4b924000 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:49.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.696+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.0 v2:172.21.15.16:6824/3923634179 1 ==== osd_op_reply(32 [pgnls start_epoch 138 out=101b] v136'14 uv14 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b80a800 2023-05-20T01:28:49.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.696+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:33 19.28 19:14ce3b86:::10000000233.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91ea80 con 0x563b4b80a800 2023-05-20T01:28:49.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.696+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.0 v2:172.21.15.16:6824/3923634179 2 ==== osd_op_reply(33 10000000233.00000000 [stat out=16b] v0'0 uv13 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b80a800 2023-05-20T01:28:49.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.696+0000 7fdd59b7c7c0 10 datascan.operator(): handling object 1099511628339.0 2023-05-20T01:28:49.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.696+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:34 19.28 19:14ce3b86:::10000000233.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91ee00 con 0x563b4b80a800 2023-05-20T01:28:49.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.698+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.0 v2:172.21.15.16:6824/3923634179 3 ==== osd_op_reply(34 10000000233.00000000 [call] v138'15 uv15 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b80a800 2023-05-20T01:28:49.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.698+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:35 19.28 19:15246602::::head [pgnls start_epoch 138 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91f180 con 0x563b4b80a800 2023-05-20T01:28:49.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.698+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.0 v2:172.21.15.16:6824/3923634179 4 ==== osd_op_reply(35 [pgnls start_epoch 138 out=101b] v138'15 uv15 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b80a800 2023-05-20T01:28:49.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.698+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:36 19.28 19:15246602:::10000000202.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91e000 con 0x563b4b80a800 2023-05-20T01:28:49.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.698+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.0 v2:172.21.15.16:6824/3923634179 5 ==== osd_op_reply(36 10000000202.00000000 [stat out=16b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b80a800 2023-05-20T01:28:49.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.698+0000 7fdd59b7c7c0 10 datascan.operator(): handling object 1099511628290.0 2023-05-20T01:28:49.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.698+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:37 19.28 19:15246602:::10000000202.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4c33ea80 con 0x563b4b80a800 2023-05-20T01:28:49.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.699+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.0 v2:172.21.15.16:6824/3923634179 6 ==== osd_op_reply(37 10000000202.00000000 [call] v138'16 uv16 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b80a800 2023-05-20T01:28:49.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.699+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:38 19.28 19:1537fa38::::head [pgnls start_epoch 138 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4c33f500 con 0x563b4b80a800 2023-05-20T01:28:49.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.700+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.0 v2:172.21.15.16:6824/3923634179 7 ==== osd_op_reply(38 [pgnls start_epoch 138 out=101b] v138'16 uv16 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b80a800 2023-05-20T01:28:49.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.700+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:39 19.28 19:1537fa38:::10000000236.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4c33f880 con 0x563b4b80a800 2023-05-20T01:28:49.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.700+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.0 v2:172.21.15.16:6824/3923634179 8 ==== osd_op_reply(39 10000000236.00000000 [stat out=16b] v0'0 uv14 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b80a800 2023-05-20T01:28:49.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.700+0000 7fdd59b7c7c0 10 datascan.operator(): handling object 1099511628342.0 2023-05-20T01:28:49.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.700+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:40 19.28 19:1537fa38:::10000000236.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91f500 con 0x563b4b80a800 2023-05-20T01:28:49.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.701+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.0 v2:172.21.15.16:6824/3923634179 9 ==== osd_op_reply(40 10000000236.00000000 [call] v138'17 uv17 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b80a800 2023-05-20T01:28:49.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.701+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:41 19.28 19:177a762d::::head [pgnls start_epoch 138 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91f880 con 0x563b4b80a800 2023-05-20T01:28:49.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.701+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.0 v2:172.21.15.16:6824/3923634179 10 ==== osd_op_reply(41 [pgnls start_epoch 138 out=81b] v138'17 uv17 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b80a800 2023-05-20T01:28:49.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.701+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:42 19.28 19:177a762d:::10000000201.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91e380 con 0x563b4b80a800 2023-05-20T01:28:49.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.702+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.0 v2:172.21.15.16:6824/3923634179 11 ==== osd_op_reply(42 10000000201.00000000 [stat out=16b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b80a800 2023-05-20T01:28:49.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.702+0000 7fdd59b7c7c0 10 datascan.operator(): handling object 1099511628289.0 2023-05-20T01:28:49.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.702+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:43 19.28 19:177a762d:::10000000201.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91fc00 con 0x563b4b80a800 2023-05-20T01:28:49.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.703+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.0 v2:172.21.15.16:6824/3923634179 12 ==== osd_op_reply(43 10000000201.00000000 [call] v138'18 uv18 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b80a800 2023-05-20T01:28:49.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.703+0000 7fdd59b7c7c0 1 --2- 172.21.15.16:0/1962333830 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x563b4b7e9400 0x563b4b924580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:49.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.703+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:44 19.18 19:18000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91ea80 con 0x563b4b7e9400 2023-05-20T01:28:49.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.703+0000 7fdd5832b700 1 --2- 172.21.15.16:0/1962333830 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x563b4b7e9400 0x563b4b924580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:49.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.704+0000 7fdd5832b700 1 --2- 172.21.15.16:0/1962333830 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x563b4b7e9400 0x563b4b924580 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:49.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.704+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.6 v2:172.21.15.132:6801/727581123 1 ==== osd_op_reply(44 [pgnls start_epoch 138 out=49b] v136'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b7e9400 2023-05-20T01:28:49.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.704+0000 7fdd53321700 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:45 19.38 19:1c000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4c33ea80 con 0x563b4b7e9400 2023-05-20T01:28:49.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.704+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.6 v2:172.21.15.132:6801/727581123 2 ==== osd_op_reply(45 [pgnls start_epoch 138 out=49b] v136'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b7e9400 2023-05-20T01:28:49.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.704+0000 7fdd52b20700 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:46 19.4 19:20000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4c33ee00 con 0x563b4b7e9000 2023-05-20T01:28:49.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.705+0000 7fdd57b2a700 1 -- 172.21.15.16:0/1962333830 <== osd.1 v2:172.21.15.16:6808/3860264220 14 ==== osd_op_reply(46 [pgnls start_epoch 138 out=81b] v136'11 uv11 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x563b4b7e58c0 con 0x563b4b7e9000 2023-05-20T01:28:49.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.705+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:47 19.4 19:2231e764:::10000000220.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91ee00 con 0x563b4b7e9000 2023-05-20T01:28:49.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.705+0000 7fdd57b2a700 1 -- 172.21.15.16:0/1962333830 <== osd.1 v2:172.21.15.16:6808/3860264220 15 ==== osd_op_reply(47 10000000220.00000000 [stat out=16b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b4b7e58c0 con 0x563b4b7e9000 2023-05-20T01:28:49.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.705+0000 7fdd59b7c7c0 10 datascan.operator(): handling object 1099511628320.0 2023-05-20T01:28:49.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.705+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:48 19.4 19:2231e764:::10000000220.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91e700 con 0x563b4b7e9000 2023-05-20T01:28:49.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.706+0000 7fdd57b2a700 1 -- 172.21.15.16:0/1962333830 <== osd.1 v2:172.21.15.16:6808/3860264220 16 ==== osd_op_reply(48 10000000220.00000000 [call] v138'12 uv12 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563b4b7e58c0 con 0x563b4b7e9000 2023-05-20T01:28:49.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.706+0000 7fdd59b7c7c0 1 --2- 172.21.15.16:0/1962333830 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x563b4b7e9800 0x563b4b924b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:49.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.706+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:49 19.24 19:24000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91ea80 con 0x563b4b7e9800 2023-05-20T01:28:49.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.707+0000 7fdd57b2a700 1 --2- 172.21.15.16:0/1962333830 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x563b4b7e9800 0x563b4b924b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:49.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.707+0000 7fdd57b2a700 1 --2- 172.21.15.16:0/1962333830 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x563b4b7e9800 0x563b4b924b00 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:49.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.707+0000 7fdd57b2a700 1 -- 172.21.15.16:0/1962333830 <== osd.5 v2:172.21.15.132:6816/1844689995 1 ==== osd_op_reply(49 [pgnls start_epoch 138 out=101b] v136'8 uv8 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x563b4b7e58c0 con 0x563b4b7e9800 2023-05-20T01:28:49.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.707+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:50 19.24 19:242ba651:::10000000234.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91f180 con 0x563b4b7e9800 2023-05-20T01:28:49.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.708+0000 7fdd57b2a700 1 -- 172.21.15.16:0/1962333830 <== osd.5 v2:172.21.15.132:6816/1844689995 2 ==== osd_op_reply(50 10000000234.00000000 [stat out=16b] v0'0 uv8 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b4b7e58c0 con 0x563b4b7e9800 2023-05-20T01:28:49.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.708+0000 7fdd59b7c7c0 10 datascan.operator(): handling object 1099511628340.0 2023-05-20T01:28:49.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.708+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:51 19.24 19:242ba651:::10000000234.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91e000 con 0x563b4b7e9800 2023-05-20T01:28:49.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.709+0000 7fdd57b2a700 1 -- 172.21.15.16:0/1962333830 <== osd.5 v2:172.21.15.132:6816/1844689995 3 ==== osd_op_reply(51 10000000234.00000000 [call] v138'9 uv9 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563b4b7e58c0 con 0x563b4b7e9800 2023-05-20T01:28:49.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.709+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:52 19.24 19:25b5fce8::::head [pgnls start_epoch 138 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91f500 con 0x563b4b7e9800 2023-05-20T01:28:49.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.710+0000 7fdd57b2a700 1 -- 172.21.15.16:0/1962333830 <== osd.5 v2:172.21.15.132:6816/1844689995 4 ==== osd_op_reply(52 [pgnls start_epoch 138 out=81b] v138'9 uv9 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x563b4b7e58c0 con 0x563b4b7e9800 2023-05-20T01:28:49.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.710+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:53 19.24 19:25b5fce8:::10000000231.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4c33f180 con 0x563b4b7e9800 2023-05-20T01:28:49.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.710+0000 7fdd57b2a700 1 -- 172.21.15.16:0/1962333830 <== osd.5 v2:172.21.15.132:6816/1844689995 5 ==== osd_op_reply(53 10000000231.00000000 [stat out=16b] v0'0 uv7 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b4b7e58c0 con 0x563b4b7e9800 2023-05-20T01:28:49.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.710+0000 7fdd59b7c7c0 10 datascan.operator(): handling object 1099511628337.0 2023-05-20T01:28:49.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.710+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:54 19.24 19:25b5fce8:::10000000231.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4c33e700 con 0x563b4b7e9800 2023-05-20T01:28:49.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.712+0000 7fdd57b2a700 1 -- 172.21.15.16:0/1962333830 <== osd.5 v2:172.21.15.132:6816/1844689995 6 ==== osd_op_reply(54 10000000231.00000000 [call] v138'10 uv10 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563b4b7e58c0 con 0x563b4b7e9800 2023-05-20T01:28:49.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.712+0000 7fdd59b7c7c0 1 --2- 172.21.15.16:0/1962333830 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x563b4b7e9c00 0x563b4b925080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:49.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.712+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:55 19.14 19:28000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4c33fc00 con 0x563b4b7e9c00 2023-05-20T01:28:49.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.712+0000 7fdd57329700 1 --2- 172.21.15.16:0/1962333830 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x563b4b7e9c00 0x563b4b925080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:49.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.713+0000 7fdd57329700 1 --2- 172.21.15.16:0/1962333830 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x563b4b7e9c00 0x563b4b925080 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:49.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.713+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.7 v2:172.21.15.132:6800/466001149 1 ==== osd_op_reply(55 [pgnls start_epoch 138 out=101b] v136'17 uv17 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b7e9c00 2023-05-20T01:28:49.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.713+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:56 19.14 19:29971f97:::1000000020e.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4c33ee00 con 0x563b4b7e9c00 2023-05-20T01:28:49.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.713+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.7 v2:172.21.15.132:6800/466001149 2 ==== osd_op_reply(56 1000000020e.00000000 [stat out=16b] v0'0 uv15 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b7e9c00 2023-05-20T01:28:49.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.713+0000 7fdd59b7c7c0 10 datascan.operator(): handling object 1099511628302.0 2023-05-20T01:28:49.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.713+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:57 19.14 19:29971f97:::1000000020e.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4c33f500 con 0x563b4b7e9c00 2023-05-20T01:28:49.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.715+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.7 v2:172.21.15.132:6800/466001149 3 ==== osd_op_reply(57 1000000020e.00000000 [call] v138'18 uv18 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b7e9c00 2023-05-20T01:28:49.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.715+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:58 19.14 19:2ab28be7::::head [pgnls start_epoch 138 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4c33f880 con 0x563b4b7e9c00 2023-05-20T01:28:49.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.715+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.7 v2:172.21.15.132:6800/466001149 4 ==== osd_op_reply(58 [pgnls start_epoch 138 out=101b] v138'18 uv18 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b7e9c00 2023-05-20T01:28:49.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.715+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:59 19.14 19:2ab28be7:::10000000211.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91ee00 con 0x563b4b7e9c00 2023-05-20T01:28:49.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.716+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.7 v2:172.21.15.132:6800/466001149 5 ==== osd_op_reply(59 10000000211.00000000 [stat out=16b] v0'0 uv16 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b7e9c00 2023-05-20T01:28:49.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.716+0000 7fdd59b7c7c0 10 datascan.operator(): handling object 1099511628305.0 2023-05-20T01:28:49.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.716+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:60 19.14 19:2ab28be7:::10000000211.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91e700 con 0x563b4b7e9c00 2023-05-20T01:28:49.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.717+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.7 v2:172.21.15.132:6800/466001149 6 ==== osd_op_reply(60 10000000211.00000000 [call] v138'19 uv19 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b7e9c00 2023-05-20T01:28:49.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.717+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:61 19.14 19:2b89f3ba::::head [pgnls start_epoch 138 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91ea80 con 0x563b4b7e9c00 2023-05-20T01:28:49.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.717+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.7 v2:172.21.15.132:6800/466001149 7 ==== osd_op_reply(61 [pgnls start_epoch 138 out=81b] v138'19 uv19 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b7e9c00 2023-05-20T01:28:49.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.717+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:62 19.14 19:2b89f3ba:::10000000221.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91f180 con 0x563b4b7e9c00 2023-05-20T01:28:49.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.718+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.7 v2:172.21.15.132:6800/466001149 8 ==== osd_op_reply(62 10000000221.00000000 [stat out=16b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b7e9c00 2023-05-20T01:28:49.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.718+0000 7fdd59b7c7c0 10 datascan.operator(): handling object 1099511628321.0 2023-05-20T01:28:49.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.718+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:63 19.14 19:2b89f3ba:::10000000221.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91e000 con 0x563b4b7e9c00 2023-05-20T01:28:49.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.720+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.7 v2:172.21.15.132:6800/466001149 9 ==== osd_op_reply(63 10000000221.00000000 [call] v138'20 uv20 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b7e9c00 2023-05-20T01:28:49.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.720+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:64 19.34 19:2c000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91f500 con 0x563b4b7e9400 2023-05-20T01:28:49.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.720+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.6 v2:172.21.15.132:6801/727581123 3 ==== osd_op_reply(64 [pgnls start_epoch 138 out=49b] v136'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b7e9400 2023-05-20T01:28:49.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.720+0000 7fdd53321700 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:65 19.c 19:30000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4c33fc00 con 0x563b4b7e8800 2023-05-20T01:28:49.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.720+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.3 v2:172.21.15.16:6800/3669781890 16 ==== osd_op_reply(65 [pgnls start_epoch 138 out=81b] v136'8 uv8 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b7e8800 2023-05-20T01:28:49.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.720+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:66 19.c 19:30604137:::100000001fc.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91f880 con 0x563b4b7e8800 2023-05-20T01:28:49.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.721+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.3 v2:172.21.15.16:6800/3669781890 17 ==== osd_op_reply(66 100000001fc.00000000 [stat out=16b] v0'0 uv8 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b7e8800 2023-05-20T01:28:49.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.721+0000 7fdd59b7c7c0 10 datascan.operator(): handling object 1099511628284.0 2023-05-20T01:28:49.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.721+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:67 19.c 19:30604137:::100000001fc.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91e380 con 0x563b4b7e8800 2023-05-20T01:28:49.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.722+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.3 v2:172.21.15.16:6800/3669781890 18 ==== osd_op_reply(67 100000001fc.00000000 [call] v138'9 uv9 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b7e8800 2023-05-20T01:28:49.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.722+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:68 19.2c 19:34000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91fc00 con 0x563b4b7e8800 2023-05-20T01:28:49.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.722+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.3 v2:172.21.15.16:6800/3669781890 19 ==== osd_op_reply(68 [pgnls start_epoch 138 out=101b] v136'12 uv12 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b7e8800 2023-05-20T01:28:49.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.722+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:69 19.2c 19:35964134:::10000000223.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91ee00 con 0x563b4b7e8800 2023-05-20T01:28:49.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.722+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.3 v2:172.21.15.16:6800/3669781890 20 ==== osd_op_reply(69 10000000223.00000000 [stat out=16b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b7e8800 2023-05-20T01:28:49.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.722+0000 7fdd59b7c7c0 10 datascan.operator(): handling object 1099511628323.0 2023-05-20T01:28:49.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.722+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:70 19.2c 19:35964134:::10000000223.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91e700 con 0x563b4b7e8800 2023-05-20T01:28:49.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.723+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.3 v2:172.21.15.16:6800/3669781890 21 ==== osd_op_reply(70 10000000223.00000000 [call] v138'13 uv13 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b7e8800 2023-05-20T01:28:49.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.723+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:71 19.2c 19:37389ade::::head [pgnls start_epoch 138 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4c33f500 con 0x563b4b7e8800 2023-05-20T01:28:49.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.724+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.3 v2:172.21.15.16:6800/3669781890 22 ==== osd_op_reply(71 [pgnls start_epoch 138 out=81b] v138'13 uv13 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b7e8800 2023-05-20T01:28:49.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.724+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:72 19.2c 19:37389ade:::10000000209.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4c33f880 con 0x563b4b7e8800 2023-05-20T01:28:49.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.724+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.3 v2:172.21.15.16:6800/3669781890 23 ==== osd_op_reply(72 10000000209.00000000 [stat out=16b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b7e8800 2023-05-20T01:28:49.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.724+0000 7fdd59b7c7c0 10 datascan.operator(): handling object 1099511628297.0 2023-05-20T01:28:49.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.724+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:73 19.2c 19:37389ade:::10000000209.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91ea80 con 0x563b4b7e8800 2023-05-20T01:28:49.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.725+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.3 v2:172.21.15.16:6800/3669781890 24 ==== osd_op_reply(73 10000000209.00000000 [call] v138'14 uv14 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b7e8800 2023-05-20T01:28:49.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.725+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:74 19.1c 19:38000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91f180 con 0x563b4b7e9c00 2023-05-20T01:28:49.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.726+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.7 v2:172.21.15.132:6800/466001149 10 ==== osd_op_reply(74 [pgnls start_epoch 138 out=49b] v136'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b7e9c00 2023-05-20T01:28:49.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.726+0000 7fdd53321700 1 --2- 172.21.15.16:0/1962333830 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x563b4b80ac00 0x563b4b926c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:49.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.726+0000 7fdd53321700 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:75 19.3c 19:3c000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4c33ee00 con 0x563b4b80ac00 2023-05-20T01:28:49.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.726+0000 7fdd5832b700 1 --2- 172.21.15.16:0/1962333830 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x563b4b80ac00 0x563b4b926c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:49.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.726+0000 7fdd5832b700 1 --2- 172.21.15.16:0/1962333830 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x563b4b80ac00 0x563b4b926c00 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:49.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.726+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.2 v2:172.21.15.16:6816/293187030 1 ==== osd_op_reply(75 [pgnls start_epoch 138 out=49b] v136'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b80ac00 2023-05-20T01:28:49.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.726+0000 7fdd52b20700 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:76 19.2 19:40000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4c33ea80 con 0x563b4b7e9c00 2023-05-20T01:28:49.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.727+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.7 v2:172.21.15.132:6800/466001149 11 ==== osd_op_reply(76 [pgnls start_epoch 138 out=49b] v136'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b7e9c00 2023-05-20T01:28:49.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.727+0000 7fdd53321700 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:77 19.22 19:44000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b948000 con 0x563b4b80ac00 2023-05-20T01:28:49.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.727+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.2 v2:172.21.15.16:6816/293187030 2 ==== osd_op_reply(77 [pgnls start_epoch 138 out=101b] v136'14 uv14 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b80ac00 2023-05-20T01:28:49.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.727+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:78 19.22 19:450d4ca5:::1000000024c.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91e000 con 0x563b4b80ac00 2023-05-20T01:28:49.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.727+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.2 v2:172.21.15.16:6816/293187030 3 ==== osd_op_reply(78 1000000024c.00000000 [stat out=16b] v0'0 uv14 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b80ac00 2023-05-20T01:28:49.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.727+0000 7fdd59b7c7c0 10 datascan.operator(): handling object 1099511628364.0 2023-05-20T01:28:49.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.727+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:79 19.22 19:450d4ca5:::1000000024c.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91f880 con 0x563b4b80ac00 2023-05-20T01:28:49.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.729+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.2 v2:172.21.15.16:6816/293187030 4 ==== osd_op_reply(79 1000000024c.00000000 [call] v138'15 uv15 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b80ac00 2023-05-20T01:28:49.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.729+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:80 19.22 19:45b8cac8::::head [pgnls start_epoch 138 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91e380 con 0x563b4b80ac00 2023-05-20T01:28:49.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.729+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.2 v2:172.21.15.16:6816/293187030 5 ==== osd_op_reply(80 [pgnls start_epoch 138 out=101b] v138'15 uv15 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b80ac00 2023-05-20T01:28:49.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.729+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:81 19.22 19:45b8cac8:::1000000023d.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91fc00 con 0x563b4b80ac00 2023-05-20T01:28:49.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.729+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.2 v2:172.21.15.16:6816/293187030 6 ==== osd_op_reply(81 1000000023d.00000000 [stat out=16b] v0'0 uv13 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b80ac00 2023-05-20T01:28:49.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.729+0000 7fdd59b7c7c0 10 datascan.operator(): handling object 1099511628349.0 2023-05-20T01:28:49.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.729+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:82 19.22 19:45b8cac8:::1000000023d.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91ee00 con 0x563b4b80ac00 2023-05-20T01:28:49.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.730+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.2 v2:172.21.15.16:6816/293187030 7 ==== osd_op_reply(82 1000000023d.00000000 [call] v138'16 uv16 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b80ac00 2023-05-20T01:28:49.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.731+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:83 19.22 19:47efb22f::::head [pgnls start_epoch 138 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91e700 con 0x563b4b80ac00 2023-05-20T01:28:49.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.731+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.2 v2:172.21.15.16:6816/293187030 8 ==== osd_op_reply(83 [pgnls start_epoch 138 out=81b] v138'16 uv16 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b80ac00 2023-05-20T01:28:49.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.731+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:84 19.22 19:47efb22f:::10000000235.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4c33fc00 con 0x563b4b80ac00 2023-05-20T01:28:49.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.731+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.2 v2:172.21.15.16:6816/293187030 9 ==== osd_op_reply(84 10000000235.00000000 [stat out=16b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b80ac00 2023-05-20T01:28:49.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.731+0000 7fdd59b7c7c0 10 datascan.operator(): handling object 1099511628341.0 2023-05-20T01:28:49.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.731+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:85 19.22 19:47efb22f:::10000000235.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b948000 con 0x563b4b80ac00 2023-05-20T01:28:49.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.732+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.2 v2:172.21.15.16:6816/293187030 10 ==== osd_op_reply(85 10000000235.00000000 [call] v138'17 uv17 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b80ac00 2023-05-20T01:28:49.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.732+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:86 19.12 19:48000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b948a80 con 0x563b4b7e9c00 2023-05-20T01:28:49.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.733+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.7 v2:172.21.15.132:6800/466001149 12 ==== osd_op_reply(86 [pgnls start_epoch 138 out=49b] v136'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b7e9c00 2023-05-20T01:28:49.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.733+0000 7fdd53321700 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:87 19.32 19:4c000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b948380 con 0x563b4b7e9000 2023-05-20T01:28:49.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.733+0000 7fdd57b2a700 1 -- 172.21.15.16:0/1962333830 <== osd.1 v2:172.21.15.16:6808/3860264220 17 ==== osd_op_reply(87 [pgnls start_epoch 138 out=81b] v136'10 uv10 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x563b4b7e58c0 con 0x563b4b7e9000 2023-05-20T01:28:49.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.733+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:88 19.32 19:4e8c6251:::10000000252.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b948e00 con 0x563b4b7e9000 2023-05-20T01:28:49.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.733+0000 7fdd57b2a700 1 -- 172.21.15.16:0/1962333830 <== osd.1 v2:172.21.15.16:6808/3860264220 18 ==== osd_op_reply(88 10000000252.00000000 [stat out=16b] v0'0 uv10 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b4b7e58c0 con 0x563b4b7e9000 2023-05-20T01:28:49.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.733+0000 7fdd59b7c7c0 10 datascan.operator(): handling object 1099511628370.0 2023-05-20T01:28:49.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.733+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:89 19.32 19:4e8c6251:::10000000252.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b949180 con 0x563b4b7e9000 2023-05-20T01:28:49.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.734+0000 7fdd57b2a700 1 -- 172.21.15.16:0/1962333830 <== osd.1 v2:172.21.15.16:6808/3860264220 19 ==== osd_op_reply(89 10000000252.00000000 [call] v138'11 uv11 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563b4b7e58c0 con 0x563b4b7e9000 2023-05-20T01:28:49.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.734+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:90 19.a 19:50000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b949500 con 0x563b4b80ac00 2023-05-20T01:28:49.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.735+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.2 v2:172.21.15.16:6816/293187030 11 ==== osd_op_reply(90 [pgnls start_epoch 138 out=101b] v136'9 uv9 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b80ac00 2023-05-20T01:28:49.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.735+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:91 19.a 19:52dcfedc:::1000000023f.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b949880 con 0x563b4b80ac00 2023-05-20T01:28:49.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.735+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.2 v2:172.21.15.16:6816/293187030 12 ==== osd_op_reply(91 1000000023f.00000000 [stat out=16b] v0'0 uv9 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b80ac00 2023-05-20T01:28:49.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.735+0000 7fdd59b7c7c0 10 datascan.operator(): handling object 1099511628351.0 2023-05-20T01:28:49.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.735+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:92 19.a 19:52dcfedc:::1000000023f.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b949c00 con 0x563b4b80ac00 2023-05-20T01:28:49.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.736+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.2 v2:172.21.15.16:6816/293187030 13 ==== osd_op_reply(92 1000000023f.00000000 [call] v138'10 uv10 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b80ac00 2023-05-20T01:28:49.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.736+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:93 19.a 19:53d16f76::::head [pgnls start_epoch 138 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91f880 con 0x563b4b80ac00 2023-05-20T01:28:49.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.737+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.2 v2:172.21.15.16:6816/293187030 14 ==== osd_op_reply(93 [pgnls start_epoch 138 out=81b] v138'10 uv10 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b80ac00 2023-05-20T01:28:49.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.737+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:94 19.a 19:53d16f76:::1000000023b.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91e380 con 0x563b4b80ac00 2023-05-20T01:28:49.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.737+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.2 v2:172.21.15.16:6816/293187030 15 ==== osd_op_reply(94 1000000023b.00000000 [stat out=16b] v0'0 uv8 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b80ac00 2023-05-20T01:28:49.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.737+0000 7fdd59b7c7c0 10 datascan.operator(): handling object 1099511628347.0 2023-05-20T01:28:49.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.737+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:95 19.a 19:53d16f76:::1000000023b.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91fc00 con 0x563b4b80ac00 2023-05-20T01:28:49.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.738+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.2 v2:172.21.15.16:6816/293187030 16 ==== osd_op_reply(95 1000000023b.00000000 [call] v138'11 uv11 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b80ac00 2023-05-20T01:28:49.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.738+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:96 19.2a 19:54000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91ee00 con 0x563b4b80a800 2023-05-20T01:28:49.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.738+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.0 v2:172.21.15.16:6824/3923634179 13 ==== osd_op_reply(96 [pgnls start_epoch 138 out=101b] v136'17 uv17 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b80a800 2023-05-20T01:28:49.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.739+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:97 19.2a 19:55a6cd83:::1000000020a.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91e700 con 0x563b4b80a800 2023-05-20T01:28:49.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.739+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.0 v2:172.21.15.16:6824/3923634179 14 ==== osd_op_reply(97 1000000020a.00000000 [stat out=16b] v0'0 uv15 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b80a800 2023-05-20T01:28:49.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.739+0000 7fdd59b7c7c0 10 datascan.operator(): handling object 1099511628298.0 2023-05-20T01:28:49.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.739+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:98 19.2a 19:55a6cd83:::1000000020a.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91f500 con 0x563b4b80a800 2023-05-20T01:28:49.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.740+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.0 v2:172.21.15.16:6824/3923634179 15 ==== osd_op_reply(98 1000000020a.00000000 [call] v138'18 uv18 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b80a800 2023-05-20T01:28:49.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.740+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:99 19.2a 19:56889a45::::head [pgnls start_epoch 138 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91e000 con 0x563b4b80a800 2023-05-20T01:28:49.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.740+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.0 v2:172.21.15.16:6824/3923634179 16 ==== osd_op_reply(99 [pgnls start_epoch 138 out=101b] v138'18 uv18 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b80a800 2023-05-20T01:28:49.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.740+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:100 19.2a 19:56889a45:::10000000213.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4c33ee00 con 0x563b4b80a800 2023-05-20T01:28:49.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.740+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.0 v2:172.21.15.16:6824/3923634179 17 ==== osd_op_reply(100 10000000213.00000000 [stat out=16b] v0'0 uv16 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b80a800 2023-05-20T01:28:49.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.740+0000 7fdd59b7c7c0 10 datascan.operator(): handling object 1099511628307.0 2023-05-20T01:28:49.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.740+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:101 19.2a 19:56889a45:::10000000213.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91ea80 con 0x563b4b80a800 2023-05-20T01:28:49.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.742+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.0 v2:172.21.15.16:6824/3923634179 18 ==== osd_op_reply(101 10000000213.00000000 [call] v138'19 uv19 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b80a800 2023-05-20T01:28:49.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.742+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:102 19.2a 19:576ad4f4::::head [pgnls start_epoch 138 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91f180 con 0x563b4b80a800 2023-05-20T01:28:49.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.742+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.0 v2:172.21.15.16:6824/3923634179 19 ==== osd_op_reply(102 [pgnls start_epoch 138 out=101b] v138'19 uv19 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b80a800 2023-05-20T01:28:49.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.742+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:103 19.2a 19:576ad4f4:::10000000245.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91ee00 con 0x563b4b80a800 2023-05-20T01:28:49.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.742+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.0 v2:172.21.15.16:6824/3923634179 20 ==== osd_op_reply(103 10000000245.00000000 [stat out=16b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b80a800 2023-05-20T01:28:49.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.742+0000 7fdd59b7c7c0 10 datascan.operator(): handling object 1099511628357.0 2023-05-20T01:28:49.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.742+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:104 19.2a 19:576ad4f4:::10000000245.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91e700 con 0x563b4b80a800 2023-05-20T01:28:49.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.743+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.0 v2:172.21.15.16:6824/3923634179 21 ==== osd_op_reply(104 10000000245.00000000 [call] v138'20 uv20 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b80a800 2023-05-20T01:28:49.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.743+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:105 19.2a 19:57fe7535::::head [pgnls start_epoch 138 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91f500 con 0x563b4b80a800 2023-05-20T01:28:49.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.743+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.0 v2:172.21.15.16:6824/3923634179 22 ==== osd_op_reply(105 [pgnls start_epoch 138 out=81b] v138'20 uv20 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b80a800 2023-05-20T01:28:49.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.743+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:106 19.2a 19:57fe7535:::100000001f6.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91e000 con 0x563b4b80a800 2023-05-20T01:28:49.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.744+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.0 v2:172.21.15.16:6824/3923634179 23 ==== osd_op_reply(106 100000001f6.00000000 [stat out=16b] v0'0 uv14 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b80a800 2023-05-20T01:28:49.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.744+0000 7fdd59b7c7c0 10 datascan.operator(): handling object 1099511628278.0 2023-05-20T01:28:49.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.744+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:107 19.2a 19:57fe7535:::100000001f6.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91f880 con 0x563b4b80a800 2023-05-20T01:28:49.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.745+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.0 v2:172.21.15.16:6824/3923634179 24 ==== osd_op_reply(107 100000001f6.00000000 [call] v138'21 uv21 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b80a800 2023-05-20T01:28:49.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.745+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:108 19.1a 19:58000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b948a80 con 0x563b4b7e8800 2023-05-20T01:28:49.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.745+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.3 v2:172.21.15.16:6800/3669781890 25 ==== osd_op_reply(108 [pgnls start_epoch 138 out=101b] v136'11 uv11 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b7e8800 2023-05-20T01:28:49.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.745+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:109 19.1a 19:582f7b2d:::10000000226.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b948000 con 0x563b4b7e8800 2023-05-20T01:28:49.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.745+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.3 v2:172.21.15.16:6800/3669781890 26 ==== osd_op_reply(109 10000000226.00000000 [stat out=16b] v0'0 uv10 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b7e8800 2023-05-20T01:28:49.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.745+0000 7fdd59b7c7c0 10 datascan.operator(): handling object 1099511628326.0 2023-05-20T01:28:49.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.745+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:110 19.1a 19:582f7b2d:::10000000226.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b949500 con 0x563b4b7e8800 2023-05-20T01:28:49.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.747+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.3 v2:172.21.15.16:6800/3669781890 27 ==== osd_op_reply(110 10000000226.00000000 [call] v138'12 uv12 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b7e8800 2023-05-20T01:28:49.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.747+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:111 19.1a 19:5a2a1294::::head [pgnls start_epoch 138 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b949880 con 0x563b4b7e8800 2023-05-20T01:28:49.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.747+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.3 v2:172.21.15.16:6800/3669781890 28 ==== osd_op_reply(111 [pgnls start_epoch 138 out=81b] v138'12 uv12 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b7e8800 2023-05-20T01:28:49.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.747+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:112 19.1a 19:5a2a1294:::1000000023a.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b949c00 con 0x563b4b7e8800 2023-05-20T01:28:49.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.748+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.3 v2:172.21.15.16:6800/3669781890 29 ==== osd_op_reply(112 1000000023a.00000000 [stat out=16b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b7e8800 2023-05-20T01:28:49.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.748+0000 7fdd59b7c7c0 10 datascan.operator(): handling object 1099511628346.0 2023-05-20T01:28:49.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.748+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:113 19.1a 19:5a2a1294:::1000000023a.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4c33f500 con 0x563b4b7e8800 2023-05-20T01:28:49.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.749+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.3 v2:172.21.15.16:6800/3669781890 30 ==== osd_op_reply(113 1000000023a.00000000 [call] v138'13 uv13 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b7e8800 2023-05-20T01:28:49.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.749+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:114 19.3a 19:5c000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4c33f880 con 0x563b4b7e8c00 2023-05-20T01:28:49.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.750+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.4 v2:172.21.15.132:6817/506317223 4 ==== osd_op_reply(114 [pgnls start_epoch 138 out=49b] v136'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b7e8c00 2023-05-20T01:28:49.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.750+0000 7fdd53321700 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:115 19.6 19:60000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b948700 con 0x563b4b7e8c00 2023-05-20T01:28:49.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.750+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.4 v2:172.21.15.132:6817/506317223 5 ==== osd_op_reply(115 [pgnls start_epoch 138 out=81b] v136'9 uv9 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b7e8c00 2023-05-20T01:28:49.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.750+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:116 19.6 19:614797eb:::10000000228.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4c33ea80 con 0x563b4b7e8c00 2023-05-20T01:28:49.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.750+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.4 v2:172.21.15.132:6817/506317223 6 ==== osd_op_reply(116 10000000228.00000000 [stat out=16b] v0'0 uv9 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b7e8c00 2023-05-20T01:28:49.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.750+0000 7fdd59b7c7c0 10 datascan.operator(): handling object 1099511628328.0 2023-05-20T01:28:49.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.750+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:117 19.6 19:614797eb:::10000000228.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4c33fc00 con 0x563b4b7e8c00 2023-05-20T01:28:49.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.752+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.4 v2:172.21.15.132:6817/506317223 7 ==== osd_op_reply(117 10000000228.00000000 [call] v138'10 uv10 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b7e8c00 2023-05-20T01:28:49.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.752+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:118 19.26 19:64000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b948a80 con 0x563b4b7e8800 2023-05-20T01:28:49.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.752+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.3 v2:172.21.15.16:6800/3669781890 31 ==== osd_op_reply(118 [pgnls start_epoch 138 out=101b] v136'21 uv21 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b7e8800 2023-05-20T01:28:49.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.752+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:119 19.26 19:658f2455:::100000001ff.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b948000 con 0x563b4b7e8800 2023-05-20T01:28:49.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.752+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.3 v2:172.21.15.16:6800/3669781890 32 ==== osd_op_reply(119 100000001ff.00000000 [stat out=16b] v0'0 uv19 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b7e8800 2023-05-20T01:28:49.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.752+0000 7fdd59b7c7c0 10 datascan.operator(): handling object 1099511628287.0 2023-05-20T01:28:49.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.752+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:120 19.26 19:658f2455:::100000001ff.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b949500 con 0x563b4b7e8800 2023-05-20T01:28:49.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.753+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.3 v2:172.21.15.16:6800/3669781890 33 ==== osd_op_reply(120 100000001ff.00000000 [call] v138'22 uv22 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b7e8800 2023-05-20T01:28:49.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.753+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:121 19.26 19:65d05a5c::::head [pgnls start_epoch 138 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91ea80 con 0x563b4b7e8800 2023-05-20T01:28:49.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.754+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.3 v2:172.21.15.16:6800/3669781890 34 ==== osd_op_reply(121 [pgnls start_epoch 138 out=101b] v138'22 uv22 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b7e8800 2023-05-20T01:28:49.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.754+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:122 19.26 19:65d05a5c:::10000000258.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91f180 con 0x563b4b7e8800 2023-05-20T01:28:49.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.754+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.3 v2:172.21.15.16:6800/3669781890 35 ==== osd_op_reply(122 10000000258.00000000 [stat out=16b] v0'0 uv21 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b7e8800 2023-05-20T01:28:49.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.754+0000 7fdd59b7c7c0 10 datascan.operator(): handling object 1099511628376.0 2023-05-20T01:28:49.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.754+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:123 19.26 19:65d05a5c:::10000000258.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91ee00 con 0x563b4b7e8800 2023-05-20T01:28:49.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.755+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.3 v2:172.21.15.16:6800/3669781890 36 ==== osd_op_reply(123 10000000258.00000000 [call] v138'23 uv23 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b7e8800 2023-05-20T01:28:49.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.755+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:124 19.26 19:65e03fee::::head [pgnls start_epoch 138 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91e700 con 0x563b4b7e8800 2023-05-20T01:28:49.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.755+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.3 v2:172.21.15.16:6800/3669781890 37 ==== osd_op_reply(124 [pgnls start_epoch 138 out=81b] v138'23 uv23 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b7e8800 2023-05-20T01:28:49.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.755+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:125 19.26 19:65e03fee:::10000000240.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91f500 con 0x563b4b7e8800 2023-05-20T01:28:49.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.755+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.3 v2:172.21.15.16:6800/3669781890 38 ==== osd_op_reply(125 10000000240.00000000 [stat out=16b] v0'0 uv20 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b7e8800 2023-05-20T01:28:49.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.755+0000 7fdd59b7c7c0 10 datascan.operator(): handling object 1099511628352.0 2023-05-20T01:28:49.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.755+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:126 19.26 19:65e03fee:::10000000240.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91e000 con 0x563b4b7e8800 2023-05-20T01:28:49.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.757+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.3 v2:172.21.15.16:6800/3669781890 39 ==== osd_op_reply(126 10000000240.00000000 [call] v138'24 uv24 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b7e8800 2023-05-20T01:28:49.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.757+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:127 19.16 19:68000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91f880 con 0x563b4b80ac00 2023-05-20T01:28:49.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.757+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.2 v2:172.21.15.16:6816/293187030 17 ==== osd_op_reply(127 [pgnls start_epoch 138 out=49b] v136'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b80ac00 2023-05-20T01:28:49.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.757+0000 7fdd52b20700 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:128 19.36 19:6c000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4c33f880 con 0x563b4b7e9000 2023-05-20T01:28:49.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.757+0000 7fdd57b2a700 1 -- 172.21.15.16:0/1962333830 <== osd.1 v2:172.21.15.16:6808/3860264220 20 ==== osd_op_reply(128 [pgnls start_epoch 138 out=101b] v136'23 uv23 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x563b4b7e58c0 con 0x563b4b7e9000 2023-05-20T01:28:49.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.757+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:129 19.36 19:6cdfec98:::10000000255.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4c33ee00 con 0x563b4b7e9000 2023-05-20T01:28:49.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.757+0000 7fdd57b2a700 1 -- 172.21.15.16:0/1962333830 <== osd.1 v2:172.21.15.16:6808/3860264220 21 ==== osd_op_reply(129 10000000255.00000000 [stat out=16b] v0'0 uv23 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b4b7e58c0 con 0x563b4b7e9000 2023-05-20T01:28:49.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.757+0000 7fdd59b7c7c0 10 datascan.operator(): handling object 1099511628373.0 2023-05-20T01:28:49.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.757+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:130 19.36 19:6cdfec98:::10000000255.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4c33f180 con 0x563b4b7e9000 2023-05-20T01:28:49.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.760+0000 7fdd57b2a700 1 -- 172.21.15.16:0/1962333830 <== osd.1 v2:172.21.15.16:6808/3860264220 22 ==== osd_op_reply(130 10000000255.00000000 [call] v138'24 uv24 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563b4b7e58c0 con 0x563b4b7e9000 2023-05-20T01:28:49.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.760+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:131 19.36 19:6ee5e2c4::::head [pgnls start_epoch 138 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4c33e700 con 0x563b4b7e9000 2023-05-20T01:28:49.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.760+0000 7fdd57b2a700 1 -- 172.21.15.16:0/1962333830 <== osd.1 v2:172.21.15.16:6808/3860264220 23 ==== osd_op_reply(131 [pgnls start_epoch 138 out=101b] v138'24 uv24 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x563b4b7e58c0 con 0x563b4b7e9000 2023-05-20T01:28:49.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.760+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:132 19.36 19:6ee5e2c4:::100000001f9.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4c33f880 con 0x563b4b7e9000 2023-05-20T01:28:49.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.760+0000 7fdd57b2a700 1 -- 172.21.15.16:0/1962333830 <== osd.1 v2:172.21.15.16:6808/3860264220 24 ==== osd_op_reply(132 100000001f9.00000000 [stat out=16b] v0'0 uv21 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b4b7e58c0 con 0x563b4b7e9000 2023-05-20T01:28:49.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.760+0000 7fdd59b7c7c0 10 datascan.operator(): handling object 1099511628281.0 2023-05-20T01:28:49.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.760+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:133 19.36 19:6ee5e2c4:::100000001f9.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4c33ee00 con 0x563b4b7e9000 2023-05-20T01:28:49.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.761+0000 7fdd57b2a700 1 -- 172.21.15.16:0/1962333830 <== osd.1 v2:172.21.15.16:6808/3860264220 25 ==== osd_op_reply(133 100000001f9.00000000 [call] v138'25 uv25 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563b4b7e58c0 con 0x563b4b7e9000 2023-05-20T01:28:49.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.761+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:134 19.36 19:6f746908::::head [pgnls start_epoch 138 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4c33f500 con 0x563b4b7e9000 2023-05-20T01:28:49.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.762+0000 7fdd57b2a700 1 -- 172.21.15.16:0/1962333830 <== osd.1 v2:172.21.15.16:6808/3860264220 26 ==== osd_op_reply(134 [pgnls start_epoch 138 out=101b] v138'25 uv25 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x563b4b7e58c0 con 0x563b4b7e9000 2023-05-20T01:28:49.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.762+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:135 19.36 19:6f746908:::1000000022c.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b948000 con 0x563b4b7e9000 2023-05-20T01:28:49.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.762+0000 7fdd57b2a700 1 -- 172.21.15.16:0/1962333830 <== osd.1 v2:172.21.15.16:6808/3860264220 27 ==== osd_op_reply(135 1000000022c.00000000 [stat out=16b] v0'0 uv22 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b4b7e58c0 con 0x563b4b7e9000 2023-05-20T01:28:49.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.762+0000 7fdd59b7c7c0 10 datascan.operator(): handling object 1099511628332.0 2023-05-20T01:28:49.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.762+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:136 19.36 19:6f746908:::1000000022c.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b948380 con 0x563b4b7e9000 2023-05-20T01:28:49.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.763+0000 7fdd57b2a700 1 -- 172.21.15.16:0/1962333830 <== osd.1 v2:172.21.15.16:6808/3860264220 28 ==== osd_op_reply(136 1000000022c.00000000 [call] v138'26 uv26 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563b4b7e58c0 con 0x563b4b7e9000 2023-05-20T01:28:49.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.763+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:137 19.36 19:6fe38dc2::::head [pgnls start_epoch 138 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b948700 con 0x563b4b7e9000 2023-05-20T01:28:49.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.763+0000 7fdd57b2a700 1 -- 172.21.15.16:0/1962333830 <== osd.1 v2:172.21.15.16:6808/3860264220 29 ==== osd_op_reply(137 [pgnls start_epoch 138 out=81b] v138'26 uv26 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x563b4b7e58c0 con 0x563b4b7e9000 2023-05-20T01:28:49.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.763+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:138 19.36 19:6fe38dc2:::100000001f7.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b948a80 con 0x563b4b7e9000 2023-05-20T01:28:49.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.764+0000 7fdd57b2a700 1 -- 172.21.15.16:0/1962333830 <== osd.1 v2:172.21.15.16:6808/3860264220 30 ==== osd_op_reply(138 100000001f7.00000000 [stat out=16b] v0'0 uv20 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b4b7e58c0 con 0x563b4b7e9000 2023-05-20T01:28:49.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.764+0000 7fdd59b7c7c0 10 datascan.operator(): handling object 1099511628279.0 2023-05-20T01:28:49.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.764+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:139 19.36 19:6fe38dc2:::100000001f7.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b948e00 con 0x563b4b7e9000 2023-05-20T01:28:49.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.765+0000 7fdd57b2a700 1 -- 172.21.15.16:0/1962333830 <== osd.1 v2:172.21.15.16:6808/3860264220 31 ==== osd_op_reply(139 100000001f7.00000000 [call] v138'27 uv27 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563b4b7e58c0 con 0x563b4b7e9000 2023-05-20T01:28:49.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.765+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:140 19.e 19:70000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b949180 con 0x563b4b7e9400 2023-05-20T01:28:49.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.765+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.6 v2:172.21.15.132:6801/727581123 4 ==== osd_op_reply(140 [pgnls start_epoch 138 out=81b] v136'7 uv7 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b7e9400 2023-05-20T01:28:49.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.765+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:141 19.e 19:71c13145:::1000000022f.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b949500 con 0x563b4b7e9400 2023-05-20T01:28:49.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.765+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.6 v2:172.21.15.132:6801/727581123 5 ==== osd_op_reply(141 1000000022f.00000000 [stat out=16b] v0'0 uv7 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b7e9400 2023-05-20T01:28:49.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.765+0000 7fdd59b7c7c0 10 datascan.operator(): handling object 1099511628335.0 2023-05-20T01:28:49.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.765+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:142 19.e 19:71c13145:::1000000022f.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b949880 con 0x563b4b7e9400 2023-05-20T01:28:49.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.767+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.6 v2:172.21.15.132:6801/727581123 6 ==== osd_op_reply(142 1000000022f.00000000 [call] v138'8 uv8 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b7e9400 2023-05-20T01:28:49.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.767+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:143 19.2e 19:74000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b949c00 con 0x563b4b7e8800 2023-05-20T01:28:49.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.767+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.3 v2:172.21.15.16:6800/3669781890 40 ==== osd_op_reply(143 [pgnls start_epoch 138 out=101b] v136'16 uv16 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b7e8800 2023-05-20T01:28:49.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.767+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:144 19.2e 19:74d13cfc:::1000000021c.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b948000 con 0x563b4b7e8800 2023-05-20T01:28:49.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.767+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.3 v2:172.21.15.16:6800/3669781890 41 ==== osd_op_reply(144 1000000021c.00000000 [stat out=16b] v0'0 uv14 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b7e8800 2023-05-20T01:28:49.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.767+0000 7fdd59b7c7c0 10 datascan.operator(): handling object 1099511628316.0 2023-05-20T01:28:49.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.767+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:145 19.2e 19:74d13cfc:::1000000021c.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b948380 con 0x563b4b7e8800 2023-05-20T01:28:49.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.769+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.3 v2:172.21.15.16:6800/3669781890 42 ==== osd_op_reply(145 1000000021c.00000000 [call] v138'17 uv17 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b7e8800 2023-05-20T01:28:49.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.769+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:146 19.2e 19:7528f44f::::head [pgnls start_epoch 138 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b948700 con 0x563b4b7e8800 2023-05-20T01:28:49.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.769+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.3 v2:172.21.15.16:6800/3669781890 43 ==== osd_op_reply(146 [pgnls start_epoch 138 out=101b] v138'17 uv17 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b7e8800 2023-05-20T01:28:49.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.769+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:147 19.2e 19:7528f44f:::10000000242.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4c33f180 con 0x563b4b7e8800 2023-05-20T01:28:49.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.769+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.3 v2:172.21.15.16:6800/3669781890 44 ==== osd_op_reply(147 10000000242.00000000 [stat out=16b] v0'0 uv16 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b7e8800 2023-05-20T01:28:49.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.769+0000 7fdd59b7c7c0 10 datascan.operator(): handling object 1099511628354.0 2023-05-20T01:28:49.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.769+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:148 19.2e 19:7528f44f:::10000000242.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4c33e700 con 0x563b4b7e8800 2023-05-20T01:28:49.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.770+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.3 v2:172.21.15.16:6800/3669781890 45 ==== osd_op_reply(148 10000000242.00000000 [call] v138'18 uv18 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b7e8800 2023-05-20T01:28:49.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.770+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:149 19.2e 19:7611a124::::head [pgnls start_epoch 138 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4c33f880 con 0x563b4b7e8800 2023-05-20T01:28:49.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.770+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.3 v2:172.21.15.16:6800/3669781890 46 ==== osd_op_reply(149 [pgnls start_epoch 138 out=81b] v138'18 uv18 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b7e8800 2023-05-20T01:28:49.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.771+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:150 19.2e 19:7611a124:::1000000021e.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4c33ee00 con 0x563b4b7e8800 2023-05-20T01:28:49.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.771+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.3 v2:172.21.15.16:6800/3669781890 47 ==== osd_op_reply(150 1000000021e.00000000 [stat out=16b] v0'0 uv15 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b7e8800 2023-05-20T01:28:49.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.771+0000 7fdd59b7c7c0 10 datascan.operator(): handling object 1099511628318.0 2023-05-20T01:28:49.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.771+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:151 19.2e 19:7611a124:::1000000021e.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4c33f500 con 0x563b4b7e8800 2023-05-20T01:28:49.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.772+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.3 v2:172.21.15.16:6800/3669781890 48 ==== osd_op_reply(151 1000000021e.00000000 [call] v138'19 uv19 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b7e8800 2023-05-20T01:28:49.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.772+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:152 19.1e 19:78000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91e000 con 0x563b4b7e8800 2023-05-20T01:28:49.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.772+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.3 v2:172.21.15.16:6800/3669781890 49 ==== osd_op_reply(152 [pgnls start_epoch 138 out=101b] v136'11 uv11 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b7e8800 2023-05-20T01:28:49.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.772+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:153 19.1e 19:789b801b:::1000000020c.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91e380 con 0x563b4b7e8800 2023-05-20T01:28:49.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.772+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.3 v2:172.21.15.16:6800/3669781890 50 ==== osd_op_reply(153 1000000020c.00000000 [stat out=16b] v0'0 uv9 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b7e8800 2023-05-20T01:28:49.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.772+0000 7fdd59b7c7c0 10 datascan.operator(): handling object 1099511628300.0 2023-05-20T01:28:49.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.772+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:154 19.1e 19:789b801b:::1000000020c.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91e700 con 0x563b4b7e8800 2023-05-20T01:28:49.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.773+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.3 v2:172.21.15.16:6800/3669781890 51 ==== osd_op_reply(154 1000000020c.00000000 [call] v138'12 uv12 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b7e8800 2023-05-20T01:28:49.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.774+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:155 19.1e 19:7a9d9b47::::head [pgnls start_epoch 138 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91ea80 con 0x563b4b7e8800 2023-05-20T01:28:49.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.774+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.3 v2:172.21.15.16:6800/3669781890 52 ==== osd_op_reply(155 [pgnls start_epoch 138 out=101b] v138'12 uv12 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b7e8800 2023-05-20T01:28:49.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.774+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:156 19.1e 19:7a9d9b47:::1000000022b.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91ee00 con 0x563b4b7e8800 2023-05-20T01:28:49.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.774+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.3 v2:172.21.15.16:6800/3669781890 53 ==== osd_op_reply(156 1000000022b.00000000 [stat out=16b] v0'0 uv10 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b7e8800 2023-05-20T01:28:49.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.774+0000 7fdd59b7c7c0 10 datascan.operator(): handling object 1099511628331.0 2023-05-20T01:28:49.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.774+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:157 19.1e 19:7a9d9b47:::1000000022b.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91f180 con 0x563b4b7e8800 2023-05-20T01:28:49.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.778+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.3 v2:172.21.15.16:6800/3669781890 54 ==== osd_op_reply(157 1000000022b.00000000 [call] v138'13 uv13 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b7e8800 2023-05-20T01:28:49.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.778+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:158 19.1e 19:7ade2fd8::::head [pgnls start_epoch 138 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91f500 con 0x563b4b7e8800 2023-05-20T01:28:49.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.778+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.3 v2:172.21.15.16:6800/3669781890 55 ==== osd_op_reply(158 [pgnls start_epoch 138 out=81b] v138'13 uv13 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b7e8800 2023-05-20T01:28:49.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.778+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:159 19.1e 19:7ade2fd8:::1000000024e.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91f880 con 0x563b4b7e8800 2023-05-20T01:28:49.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.779+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.3 v2:172.21.15.16:6800/3669781890 56 ==== osd_op_reply(159 1000000024e.00000000 [stat out=16b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b7e8800 2023-05-20T01:28:49.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.779+0000 7fdd59b7c7c0 10 datascan.operator(): handling object 1099511628366.0 2023-05-20T01:28:49.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.779+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:160 19.1e 19:7ade2fd8:::1000000024e.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91fc00 con 0x563b4b7e8800 2023-05-20T01:28:49.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.780+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.3 v2:172.21.15.16:6800/3669781890 57 ==== osd_op_reply(160 1000000024e.00000000 [call] v138'14 uv14 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b7e8800 2023-05-20T01:28:49.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.780+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:161 19.3e 19:7c000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91e000 con 0x563b4b7e9000 2023-05-20T01:28:49.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.780+0000 7fdd57b2a700 1 -- 172.21.15.16:0/1962333830 <== osd.1 v2:172.21.15.16:6808/3860264220 32 ==== osd_op_reply(161 [pgnls start_epoch 138 out=81b] v136'6 uv6 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x563b4b7e58c0 con 0x563b4b7e9000 2023-05-20T01:28:49.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.780+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:162 19.3e 19:7f918703:::10000000229.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91e380 con 0x563b4b7e9000 2023-05-20T01:28:49.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.781+0000 7fdd57b2a700 1 -- 172.21.15.16:0/1962333830 <== osd.1 v2:172.21.15.16:6808/3860264220 33 ==== osd_op_reply(162 10000000229.00000000 [stat out=16b] v0'0 uv6 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b4b7e58c0 con 0x563b4b7e9000 2023-05-20T01:28:49.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.781+0000 7fdd59b7c7c0 10 datascan.operator(): handling object 1099511628329.0 2023-05-20T01:28:49.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.781+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:163 19.3e 19:7f918703:::10000000229.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91e700 con 0x563b4b7e9000 2023-05-20T01:28:49.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.782+0000 7fdd57b2a700 1 -- 172.21.15.16:0/1962333830 <== osd.1 v2:172.21.15.16:6808/3860264220 34 ==== osd_op_reply(163 10000000229.00000000 [call] v138'7 uv7 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563b4b7e58c0 con 0x563b4b7e9000 2023-05-20T01:28:49.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.782+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:164 19.1 19:80000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91ea80 con 0x563b4b7e9000 2023-05-20T01:28:49.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.782+0000 7fdd57b2a700 1 -- 172.21.15.16:0/1962333830 <== osd.1 v2:172.21.15.16:6808/3860264220 35 ==== osd_op_reply(164 [pgnls start_epoch 138 out=81b] v136'7 uv7 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x563b4b7e58c0 con 0x563b4b7e9000 2023-05-20T01:28:49.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.782+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:165 19.1 19:805bc955:::10000000224.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4c33f180 con 0x563b4b7e9000 2023-05-20T01:28:49.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.782+0000 7fdd57b2a700 1 -- 172.21.15.16:0/1962333830 <== osd.1 v2:172.21.15.16:6808/3860264220 36 ==== osd_op_reply(165 10000000224.00000000 [stat out=16b] v0'0 uv7 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b4b7e58c0 con 0x563b4b7e9000 2023-05-20T01:28:49.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.782+0000 7fdd59b7c7c0 10 datascan.operator(): handling object 1099511628324.0 2023-05-20T01:28:49.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.782+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:166 19.1 19:805bc955:::10000000224.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4c33e700 con 0x563b4b7e9000 2023-05-20T01:28:49.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.783+0000 7fdd57b2a700 1 -- 172.21.15.16:0/1962333830 <== osd.1 v2:172.21.15.16:6808/3860264220 37 ==== osd_op_reply(166 10000000224.00000000 [call] v138'8 uv8 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563b4b7e58c0 con 0x563b4b7e9000 2023-05-20T01:28:49.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.784+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:167 19.21 19:84000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4c33f880 con 0x563b4b7e9c00 2023-05-20T01:28:49.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.784+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.7 v2:172.21.15.132:6800/466001149 13 ==== osd_op_reply(167 [pgnls start_epoch 138 out=49b] v136'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b7e9c00 2023-05-20T01:28:49.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.784+0000 7fdd52b20700 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:168 19.11 19:88000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4c33ea80 con 0x563b4b7e9400 2023-05-20T01:28:49.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.784+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.6 v2:172.21.15.132:6801/727581123 7 ==== osd_op_reply(168 [pgnls start_epoch 138 out=49b] v136'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b7e9400 2023-05-20T01:28:49.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.784+0000 7fdd53321700 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:169 19.31 19:8c000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b96c000 con 0x563b4b80ac00 2023-05-20T01:28:49.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.785+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.2 v2:172.21.15.16:6816/293187030 18 ==== osd_op_reply(169 [pgnls start_epoch 138 out=101b] v136'11 uv11 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b80ac00 2023-05-20T01:28:49.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.785+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:170 19.31 19:8c51d9b0:::10000000216.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4c33ee00 con 0x563b4b80ac00 2023-05-20T01:28:49.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.785+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.2 v2:172.21.15.16:6816/293187030 19 ==== osd_op_reply(170 10000000216.00000000 [stat out=16b] v0'0 uv10 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b80ac00 2023-05-20T01:28:49.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.785+0000 7fdd59b7c7c0 10 datascan.operator(): handling object 1099511628310.0 2023-05-20T01:28:49.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.785+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:171 19.31 19:8c51d9b0:::10000000216.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4c33f500 con 0x563b4b80ac00 2023-05-20T01:28:49.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.786+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.2 v2:172.21.15.16:6816/293187030 20 ==== osd_op_reply(171 10000000216.00000000 [call] v138'12 uv12 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b80ac00 2023-05-20T01:28:49.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.786+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:172 19.31 19:8fb7f7b7::::head [pgnls start_epoch 138 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b949c00 con 0x563b4b80ac00 2023-05-20T01:28:49.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.786+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.2 v2:172.21.15.16:6816/293187030 21 ==== osd_op_reply(172 [pgnls start_epoch 138 out=81b] v138'12 uv12 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b80ac00 2023-05-20T01:28:49.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.786+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:173 19.31 19:8fb7f7b7:::10000000251.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b948000 con 0x563b4b80ac00 2023-05-20T01:28:49.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.787+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.2 v2:172.21.15.16:6816/293187030 22 ==== osd_op_reply(173 10000000251.00000000 [stat out=16b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b80ac00 2023-05-20T01:28:49.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.787+0000 7fdd59b7c7c0 10 datascan.operator(): handling object 1099511628369.0 2023-05-20T01:28:49.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.787+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:174 19.31 19:8fb7f7b7:::10000000251.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b948380 con 0x563b4b80ac00 2023-05-20T01:28:49.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.788+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.2 v2:172.21.15.16:6816/293187030 23 ==== osd_op_reply(174 10000000251.00000000 [call] v138'13 uv13 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b80ac00 2023-05-20T01:28:49.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.788+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:175 19.9 19:90000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b948700 con 0x563b4b7e9c00 2023-05-20T01:28:49.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.788+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.7 v2:172.21.15.132:6800/466001149 14 ==== osd_op_reply(175 [pgnls start_epoch 138 out=101b] v136'6 uv6 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b7e9c00 2023-05-20T01:28:49.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.788+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:176 19.9 19:915bff82:::1000000024d.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b96c000 con 0x563b4b7e9c00 2023-05-20T01:28:49.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.789+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.7 v2:172.21.15.132:6800/466001149 15 ==== osd_op_reply(176 1000000024d.00000000 [stat out=16b] v0'0 uv6 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b7e9c00 2023-05-20T01:28:49.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.789+0000 7fdd59b7c7c0 10 datascan.operator(): handling object 1099511628365.0 2023-05-20T01:28:49.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.789+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:177 19.9 19:915bff82:::1000000024d.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b96ce00 con 0x563b4b7e9c00 2023-05-20T01:28:49.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.790+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.7 v2:172.21.15.132:6800/466001149 16 ==== osd_op_reply(177 1000000024d.00000000 [call] v138'7 uv7 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b7e9c00 2023-05-20T01:28:49.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.790+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:178 19.9 19:91e366d7::::head [pgnls start_epoch 138 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b96d180 con 0x563b4b7e9c00 2023-05-20T01:28:49.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.790+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.7 v2:172.21.15.132:6800/466001149 17 ==== osd_op_reply(178 [pgnls start_epoch 138 out=81b] v138'7 uv7 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b7e9c00 2023-05-20T01:28:49.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.790+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:179 19.9 19:91e366d7:::10000000230.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b96d500 con 0x563b4b7e9c00 2023-05-20T01:28:49.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.790+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.7 v2:172.21.15.132:6800/466001149 18 ==== osd_op_reply(179 10000000230.00000000 [stat out=16b] v0'0 uv5 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b7e9c00 2023-05-20T01:28:49.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.790+0000 7fdd59b7c7c0 10 datascan.operator(): handling object 1099511628336.0 2023-05-20T01:28:49.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.790+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:180 19.9 19:91e366d7:::10000000230.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b96d880 con 0x563b4b7e9c00 2023-05-20T01:28:49.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.791+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.7 v2:172.21.15.132:6800/466001149 19 ==== osd_op_reply(180 10000000230.00000000 [call] v138'8 uv8 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b7e9c00 2023-05-20T01:28:49.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.791+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:181 19.29 19:94000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b96dc00 con 0x563b4b7e9400 2023-05-20T01:28:49.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.792+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.6 v2:172.21.15.132:6801/727581123 8 ==== osd_op_reply(181 [pgnls start_epoch 138 out=101b] v136'14 uv14 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b7e9400 2023-05-20T01:28:49.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.792+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:182 19.29 19:94c02207:::1000000024a.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b949180 con 0x563b4b7e9400 2023-05-20T01:28:49.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.792+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.6 v2:172.21.15.132:6801/727581123 9 ==== osd_op_reply(182 1000000024a.00000000 [stat out=16b] v0'0 uv13 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b7e9400 2023-05-20T01:28:49.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.792+0000 7fdd59b7c7c0 10 datascan.operator(): handling object 1099511628362.0 2023-05-20T01:28:49.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.792+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:183 19.29 19:94c02207:::1000000024a.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b949500 con 0x563b4b7e9400 2023-05-20T01:28:49.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.793+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.6 v2:172.21.15.132:6801/727581123 10 ==== osd_op_reply(183 1000000024a.00000000 [call] v138'15 uv15 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b7e9400 2023-05-20T01:28:49.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.794+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:184 19.29 19:95e8e66d::::head [pgnls start_epoch 138 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b949880 con 0x563b4b7e9400 2023-05-20T01:28:49.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.794+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.6 v2:172.21.15.132:6801/727581123 11 ==== osd_op_reply(184 [pgnls start_epoch 138 out=101b] v138'15 uv15 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b7e9400 2023-05-20T01:28:49.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.794+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:185 19.29 19:95e8e66d:::10000000256.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b949c00 con 0x563b4b7e9400 2023-05-20T01:28:49.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.794+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.6 v2:172.21.15.132:6801/727581123 12 ==== osd_op_reply(185 10000000256.00000000 [stat out=16b] v0'0 uv14 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b7e9400 2023-05-20T01:28:49.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.794+0000 7fdd59b7c7c0 10 datascan.operator(): handling object 1099511628374.0 2023-05-20T01:28:49.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.794+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:186 19.29 19:95e8e66d:::10000000256.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4c33ea80 con 0x563b4b7e9400 2023-05-20T01:28:49.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.796+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.6 v2:172.21.15.132:6801/727581123 13 ==== osd_op_reply(186 10000000256.00000000 [call] v138'16 uv16 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b7e9400 2023-05-20T01:28:49.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.796+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:187 19.29 19:97401066::::head [pgnls start_epoch 138 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4c33ee00 con 0x563b4b7e9400 2023-05-20T01:28:49.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.796+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.6 v2:172.21.15.132:6801/727581123 14 ==== osd_op_reply(187 [pgnls start_epoch 138 out=81b] v138'16 uv16 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b7e9400 2023-05-20T01:28:49.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.796+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:188 19.29 19:97401066:::10000000247.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4c33f500 con 0x563b4b7e9400 2023-05-20T01:28:49.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.796+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.6 v2:172.21.15.132:6801/727581123 15 ==== osd_op_reply(188 10000000247.00000000 [stat out=16b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b7e9400 2023-05-20T01:28:49.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.796+0000 7fdd59b7c7c0 10 datascan.operator(): handling object 1099511628359.0 2023-05-20T01:28:49.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.796+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:189 19.29 19:97401066:::10000000247.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4c33ea80 con 0x563b4b7e9400 2023-05-20T01:28:49.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.797+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.6 v2:172.21.15.132:6801/727581123 16 ==== osd_op_reply(189 10000000247.00000000 [call] v138'17 uv17 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b7e9400 2023-05-20T01:28:49.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.798+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:190 19.19 19:98000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4c33ee00 con 0x563b4b7e9000 2023-05-20T01:28:49.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.798+0000 7fdd57b2a700 1 -- 172.21.15.16:0/1962333830 <== osd.1 v2:172.21.15.16:6808/3860264220 38 ==== osd_op_reply(190 [pgnls start_epoch 138 out=101b] v136'14 uv14 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x563b4b7e58c0 con 0x563b4b7e9000 2023-05-20T01:28:49.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.798+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:191 19.19 19:992a6246:::10000000200.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b948000 con 0x563b4b7e9000 2023-05-20T01:28:49.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.798+0000 7fdd57b2a700 1 -- 172.21.15.16:0/1962333830 <== osd.1 v2:172.21.15.16:6808/3860264220 39 ==== osd_op_reply(191 10000000200.00000000 [stat out=16b] v0'0 uv14 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b4b7e58c0 con 0x563b4b7e9000 2023-05-20T01:28:49.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.798+0000 7fdd59b7c7c0 10 datascan.operator(): handling object 1099511628288.0 2023-05-20T01:28:49.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.798+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:192 19.19 19:992a6246:::10000000200.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b948380 con 0x563b4b7e9000 2023-05-20T01:28:49.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.799+0000 7fdd57b2a700 1 -- 172.21.15.16:0/1962333830 <== osd.1 v2:172.21.15.16:6808/3860264220 40 ==== osd_op_reply(192 10000000200.00000000 [call] v138'15 uv15 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563b4b7e58c0 con 0x563b4b7e9000 2023-05-20T01:28:49.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.799+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:193 19.19 19:997b7129::::head [pgnls start_epoch 138 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b949180 con 0x563b4b7e9000 2023-05-20T01:28:49.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.800+0000 7fdd57b2a700 1 -- 172.21.15.16:0/1962333830 <== osd.1 v2:172.21.15.16:6808/3860264220 41 ==== osd_op_reply(193 [pgnls start_epoch 138 out=81b] v138'15 uv15 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x563b4b7e58c0 con 0x563b4b7e9000 2023-05-20T01:28:49.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.800+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:194 19.19 19:997b7129:::100000001f8.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b949500 con 0x563b4b7e9000 2023-05-20T01:28:49.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.800+0000 7fdd57b2a700 1 -- 172.21.15.16:0/1962333830 <== osd.1 v2:172.21.15.16:6808/3860264220 42 ==== osd_op_reply(194 100000001f8.00000000 [stat out=16b] v0'0 uv13 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b4b7e58c0 con 0x563b4b7e9000 2023-05-20T01:28:49.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.800+0000 7fdd59b7c7c0 10 datascan.operator(): handling object 1099511628280.0 2023-05-20T01:28:49.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.800+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:195 19.19 19:997b7129:::100000001f8.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b949880 con 0x563b4b7e9000 2023-05-20T01:28:49.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.801+0000 7fdd57b2a700 1 -- 172.21.15.16:0/1962333830 <== osd.1 v2:172.21.15.16:6808/3860264220 43 ==== osd_op_reply(195 100000001f8.00000000 [call] v138'16 uv16 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563b4b7e58c0 con 0x563b4b7e9000 2023-05-20T01:28:49.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.801+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:196 19.39 19:9c000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b949c00 con 0x563b4b7e8c00 2023-05-20T01:28:49.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.802+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.4 v2:172.21.15.132:6817/506317223 8 ==== osd_op_reply(196 [pgnls start_epoch 138 out=81b] v136'7 uv7 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b7e8c00 2023-05-20T01:28:49.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.802+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:197 19.39 19:9f1351c3:::10000000222.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b96dc00 con 0x563b4b7e8c00 2023-05-20T01:28:49.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.802+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.4 v2:172.21.15.132:6817/506317223 9 ==== osd_op_reply(197 10000000222.00000000 [stat out=16b] v0'0 uv7 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b7e8c00 2023-05-20T01:28:49.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.802+0000 7fdd59b7c7c0 10 datascan.operator(): handling object 1099511628322.0 2023-05-20T01:28:49.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.802+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:198 19.39 19:9f1351c3:::10000000222.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4c33f180 con 0x563b4b7e8c00 2023-05-20T01:28:49.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.803+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.4 v2:172.21.15.132:6817/506317223 10 ==== osd_op_reply(198 10000000222.00000000 [call] v138'8 uv8 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b7e8c00 2023-05-20T01:28:49.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.803+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:199 19.5 19:a0000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4c33e700 con 0x563b4b7e8c00 2023-05-20T01:28:49.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.804+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.4 v2:172.21.15.132:6817/506317223 11 ==== osd_op_reply(199 [pgnls start_epoch 138 out=101b] v136'9 uv9 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b7e8c00 2023-05-20T01:28:49.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.804+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:200 19.5 19:a0beda95:::10000000204.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4c33ee00 con 0x563b4b7e8c00 2023-05-20T01:28:49.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.804+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.4 v2:172.21.15.132:6817/506317223 12 ==== osd_op_reply(200 10000000204.00000000 [stat out=16b] v0'0 uv8 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b7e8c00 2023-05-20T01:28:49.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.804+0000 7fdd59b7c7c0 10 datascan.operator(): handling object 1099511628292.0 2023-05-20T01:28:49.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.804+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:201 19.5 19:a0beda95:::10000000204.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91e000 con 0x563b4b7e8c00 2023-05-20T01:28:49.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.805+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.4 v2:172.21.15.132:6817/506317223 13 ==== osd_op_reply(201 10000000204.00000000 [call] v138'10 uv10 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b7e8c00 2023-05-20T01:28:49.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.805+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:202 19.5 19:a3a50212::::head [pgnls start_epoch 138 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91e380 con 0x563b4b7e8c00 2023-05-20T01:28:49.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.806+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.4 v2:172.21.15.132:6817/506317223 14 ==== osd_op_reply(202 [pgnls start_epoch 138 out=81b] v138'10 uv10 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b7e8c00 2023-05-20T01:28:49.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.806+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:203 19.5 19:a3a50212:::10000000254.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91e700 con 0x563b4b7e8c00 2023-05-20T01:28:49.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.806+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.4 v2:172.21.15.132:6817/506317223 15 ==== osd_op_reply(203 10000000254.00000000 [stat out=16b] v0'0 uv9 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b7e8c00 2023-05-20T01:28:49.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.806+0000 7fdd59b7c7c0 10 datascan.operator(): handling object 1099511628372.0 2023-05-20T01:28:49.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.806+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:204 19.5 19:a3a50212:::10000000254.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91ea80 con 0x563b4b7e8c00 2023-05-20T01:28:49.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.807+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.4 v2:172.21.15.132:6817/506317223 16 ==== osd_op_reply(204 10000000254.00000000 [call] v138'11 uv11 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b7e8c00 2023-05-20T01:28:49.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.807+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:205 19.25 19:a4000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b948a80 con 0x563b4b80ac00 2023-05-20T01:28:49.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.807+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.2 v2:172.21.15.16:6816/293187030 24 ==== osd_op_reply(205 [pgnls start_epoch 138 out=101b] v136'10 uv10 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b80ac00 2023-05-20T01:28:49.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.807+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:206 19.25 19:a53878a9:::10000000237.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b948e00 con 0x563b4b80ac00 2023-05-20T01:28:49.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.808+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.2 v2:172.21.15.16:6816/293187030 25 ==== osd_op_reply(206 10000000237.00000000 [stat out=16b] v0'0 uv9 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b80ac00 2023-05-20T01:28:49.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.808+0000 7fdd59b7c7c0 10 datascan.operator(): handling object 1099511628343.0 2023-05-20T01:28:49.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.808+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:207 19.25 19:a53878a9:::10000000237.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91e000 con 0x563b4b80ac00 2023-05-20T01:28:49.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.809+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.2 v2:172.21.15.16:6816/293187030 26 ==== osd_op_reply(207 10000000237.00000000 [call] v138'11 uv11 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b80ac00 2023-05-20T01:28:49.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.809+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:208 19.25 19:a7ae8a0f::::head [pgnls start_epoch 138 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91e380 con 0x563b4b80ac00 2023-05-20T01:28:49.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.809+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.2 v2:172.21.15.16:6816/293187030 27 ==== osd_op_reply(208 [pgnls start_epoch 138 out=81b] v138'11 uv11 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b80ac00 2023-05-20T01:28:49.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.809+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:209 19.25 19:a7ae8a0f:::10000000243.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91e700 con 0x563b4b80ac00 2023-05-20T01:28:49.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.809+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.2 v2:172.21.15.16:6816/293187030 28 ==== osd_op_reply(209 10000000243.00000000 [stat out=16b] v0'0 uv10 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b80ac00 2023-05-20T01:28:49.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.809+0000 7fdd59b7c7c0 10 datascan.operator(): handling object 1099511628355.0 2023-05-20T01:28:49.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.809+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:210 19.25 19:a7ae8a0f:::10000000243.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4c33f500 con 0x563b4b80ac00 2023-05-20T01:28:49.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.811+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.2 v2:172.21.15.16:6816/293187030 29 ==== osd_op_reply(210 10000000243.00000000 [call] v138'12 uv12 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b80ac00 2023-05-20T01:28:49.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.811+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:211 19.15 19:a8000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4c33ea80 con 0x563b4b7e9c00 2023-05-20T01:28:49.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.811+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.7 v2:172.21.15.132:6800/466001149 20 ==== osd_op_reply(211 [pgnls start_epoch 138 out=81b] v136'8 uv8 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b7e9c00 2023-05-20T01:28:49.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.811+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:212 19.15 19:a8e69ea8:::1000000020d.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4c33f180 con 0x563b4b7e9c00 2023-05-20T01:28:49.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.811+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.7 v2:172.21.15.132:6800/466001149 21 ==== osd_op_reply(212 1000000020d.00000000 [stat out=16b] v0'0 uv8 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b7e9c00 2023-05-20T01:28:49.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.811+0000 7fdd59b7c7c0 10 datascan.operator(): handling object 1099511628301.0 2023-05-20T01:28:49.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.811+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:213 19.15 19:a8e69ea8:::1000000020d.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4c33e700 con 0x563b4b7e9c00 2023-05-20T01:28:49.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.812+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.7 v2:172.21.15.132:6800/466001149 22 ==== osd_op_reply(213 1000000020d.00000000 [call] v138'9 uv9 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b7e9c00 2023-05-20T01:28:49.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.813+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:214 19.35 19:ac000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4c33ee00 con 0x563b4b80a800 2023-05-20T01:28:49.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.813+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.0 v2:172.21.15.16:6824/3923634179 25 ==== osd_op_reply(214 [pgnls start_epoch 138 out=49b] v136'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b80a800 2023-05-20T01:28:49.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.813+0000 7fdd53321700 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:215 19.d 19:b0000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b96c380 con 0x563b4b7e9000 2023-05-20T01:28:49.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.813+0000 7fdd57b2a700 1 -- 172.21.15.16:0/1962333830 <== osd.1 v2:172.21.15.16:6808/3860264220 44 ==== osd_op_reply(215 [pgnls start_epoch 138 out=49b] v136'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563b4b7e58c0 con 0x563b4b7e9000 2023-05-20T01:28:49.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.813+0000 7fdd52b20700 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:216 19.2d 19:b4000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4c33fc00 con 0x563b4b80a800 2023-05-20T01:28:49.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.814+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.0 v2:172.21.15.16:6824/3923634179 26 ==== osd_op_reply(216 [pgnls start_epoch 138 out=81b] v136'9 uv9 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b80a800 2023-05-20T01:28:49.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.814+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:217 19.2d 19:b79a54f6:::10000000218.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b96dc00 con 0x563b4b80a800 2023-05-20T01:28:49.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.814+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.0 v2:172.21.15.16:6824/3923634179 27 ==== osd_op_reply(217 10000000218.00000000 [stat out=16b] v0'0 uv9 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b80a800 2023-05-20T01:28:49.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.814+0000 7fdd59b7c7c0 10 datascan.operator(): handling object 1099511628312.0 2023-05-20T01:28:49.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.814+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:218 19.2d 19:b79a54f6:::10000000218.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b949c00 con 0x563b4b80a800 2023-05-20T01:28:49.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.815+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.0 v2:172.21.15.16:6824/3923634179 28 ==== osd_op_reply(218 10000000218.00000000 [call] v138'10 uv10 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b80a800 2023-05-20T01:28:49.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.815+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:219 19.1d 19:b8000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4c33f880 con 0x563b4b7e9c00 2023-05-20T01:28:49.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.816+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.7 v2:172.21.15.132:6800/466001149 23 ==== osd_op_reply(219 [pgnls start_epoch 138 out=81b] v136'17 uv17 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b7e9c00 2023-05-20T01:28:49.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.816+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:220 19.1d 19:bab97ecf:::10000000246.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4c33ea80 con 0x563b4b7e9c00 2023-05-20T01:28:49.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.816+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.7 v2:172.21.15.132:6800/466001149 24 ==== osd_op_reply(220 10000000246.00000000 [stat out=16b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b7e9c00 2023-05-20T01:28:49.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.816+0000 7fdd59b7c7c0 10 datascan.operator(): handling object 1099511628358.0 2023-05-20T01:28:49.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.816+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:221 19.1d 19:bab97ecf:::10000000246.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4c33f180 con 0x563b4b7e9c00 2023-05-20T01:28:49.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.817+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.7 v2:172.21.15.132:6800/466001149 25 ==== osd_op_reply(221 10000000246.00000000 [call] v138'18 uv18 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b7e9c00 2023-05-20T01:28:49.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.817+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:222 19.3d 19:bc000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b96c000 con 0x563b4b7e9400 2023-05-20T01:28:49.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.818+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.6 v2:172.21.15.132:6801/727581123 17 ==== osd_op_reply(222 [pgnls start_epoch 138 out=101b] v136'11 uv11 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b7e9400 2023-05-20T01:28:49.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.818+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:223 19.3d 19:bc0c3701:::10000000219.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b96ce00 con 0x563b4b7e9400 2023-05-20T01:28:49.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.818+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.6 v2:172.21.15.132:6801/727581123 18 ==== osd_op_reply(223 10000000219.00000000 [stat out=16b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b7e9400 2023-05-20T01:28:49.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.818+0000 7fdd59b7c7c0 10 datascan.operator(): handling object 1099511628313.0 2023-05-20T01:28:49.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.818+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:224 19.3d 19:bc0c3701:::10000000219.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b96d180 con 0x563b4b7e9400 2023-05-20T01:28:49.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.819+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.6 v2:172.21.15.132:6801/727581123 19 ==== osd_op_reply(224 10000000219.00000000 [call] v138'12 uv12 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b7e9400 2023-05-20T01:28:49.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.819+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:225 19.3d 19:bfcd3380::::head [pgnls start_epoch 138 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b96d500 con 0x563b4b7e9400 2023-05-20T01:28:49.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.819+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.6 v2:172.21.15.132:6801/727581123 20 ==== osd_op_reply(225 [pgnls start_epoch 138 out=81b] v138'12 uv12 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b7e9400 2023-05-20T01:28:49.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.819+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:226 19.3d 19:bfcd3380:::10000000205.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b96d880 con 0x563b4b7e9400 2023-05-20T01:28:49.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.819+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.6 v2:172.21.15.132:6801/727581123 21 ==== osd_op_reply(226 10000000205.00000000 [stat out=16b] v0'0 uv10 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b7e9400 2023-05-20T01:28:49.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.819+0000 7fdd59b7c7c0 10 datascan.operator(): handling object 1099511628293.0 2023-05-20T01:28:49.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.819+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:227 19.3d 19:bfcd3380:::10000000205.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b948700 con 0x563b4b7e9400 2023-05-20T01:28:49.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.820+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.6 v2:172.21.15.132:6801/727581123 22 ==== osd_op_reply(227 10000000205.00000000 [call] v138'13 uv13 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b7e9400 2023-05-20T01:28:49.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.821+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:228 19.3 19:c0000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91ee00 con 0x563b4b7e9800 2023-05-20T01:28:49.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.821+0000 7fdd57b2a700 1 -- 172.21.15.16:0/1962333830 <== osd.5 v2:172.21.15.132:6816/1844689995 7 ==== osd_op_reply(228 [pgnls start_epoch 138 out=49b] v136'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563b4b7e58c0 con 0x563b4b7e9800 2023-05-20T01:28:49.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.821+0000 7fdd53321700 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:229 19.23 19:c4000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b96c700 con 0x563b4b80ac00 2023-05-20T01:28:49.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.821+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.2 v2:172.21.15.16:6816/293187030 30 ==== osd_op_reply(229 [pgnls start_epoch 138 out=49b] v136'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b80ac00 2023-05-20T01:28:49.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.821+0000 7fdd52b20700 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:230 19.13 19:c8000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91ea80 con 0x563b4b80ac00 2023-05-20T01:28:49.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.821+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.2 v2:172.21.15.16:6816/293187030 31 ==== osd_op_reply(230 [pgnls start_epoch 138 out=101b] v136'25 uv25 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b80ac00 2023-05-20T01:28:49.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.821+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:231 19.13 19:c8893608:::1000000023c.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91f180 con 0x563b4b80ac00 2023-05-20T01:28:49.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.822+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.2 v2:172.21.15.16:6816/293187030 32 ==== osd_op_reply(231 1000000023c.00000000 [stat out=16b] v0'0 uv24 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b80ac00 2023-05-20T01:28:49.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.822+0000 7fdd59b7c7c0 10 datascan.operator(): handling object 1099511628348.0 2023-05-20T01:28:49.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.822+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:232 19.13 19:c8893608:::1000000023c.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91f500 con 0x563b4b80ac00 2023-05-20T01:28:49.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.823+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.2 v2:172.21.15.16:6816/293187030 33 ==== osd_op_reply(232 1000000023c.00000000 [call] v138'26 uv26 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b80ac00 2023-05-20T01:28:49.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.823+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:233 19.13 19:c933d8fa::::head [pgnls start_epoch 138 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91f880 con 0x563b4b80ac00 2023-05-20T01:28:49.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.823+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.2 v2:172.21.15.16:6816/293187030 34 ==== osd_op_reply(233 [pgnls start_epoch 138 out=101b] v138'26 uv26 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b80ac00 2023-05-20T01:28:49.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.823+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:234 19.13 19:c933d8fa:::10000000241.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91fc00 con 0x563b4b80ac00 2023-05-20T01:28:49.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.823+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.2 v2:172.21.15.16:6816/293187030 35 ==== osd_op_reply(234 10000000241.00000000 [stat out=16b] v0'0 uv25 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b80ac00 2023-05-20T01:28:49.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.823+0000 7fdd59b7c7c0 10 datascan.operator(): handling object 1099511628353.0 2023-05-20T01:28:49.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.823+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:235 19.13 19:c933d8fa:::10000000241.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b96c000 con 0x563b4b80ac00 2023-05-20T01:28:49.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.824+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.2 v2:172.21.15.16:6816/293187030 36 ==== osd_op_reply(235 10000000241.00000000 [call] v138'27 uv27 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b80ac00 2023-05-20T01:28:49.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.824+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:236 19.13 19:c9855970::::head [pgnls start_epoch 138 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b96ce00 con 0x563b4b80ac00 2023-05-20T01:28:49.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.825+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.2 v2:172.21.15.16:6816/293187030 37 ==== osd_op_reply(236 [pgnls start_epoch 138 out=101b] v138'27 uv27 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b80ac00 2023-05-20T01:28:49.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.825+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:237 19.13 19:c9855970:::10000000239.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b96d180 con 0x563b4b80ac00 2023-05-20T01:28:49.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.825+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.2 v2:172.21.15.16:6816/293187030 38 ==== osd_op_reply(237 10000000239.00000000 [stat out=16b] v0'0 uv23 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b80ac00 2023-05-20T01:28:49.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.825+0000 7fdd59b7c7c0 10 datascan.operator(): handling object 1099511628345.0 2023-05-20T01:28:49.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.825+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:238 19.13 19:c9855970:::10000000239.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b96d500 con 0x563b4b80ac00 2023-05-20T01:28:49.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.826+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.2 v2:172.21.15.16:6816/293187030 39 ==== osd_op_reply(238 10000000239.00000000 [call] v138'28 uv28 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b80ac00 2023-05-20T01:28:49.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.826+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:239 19.13 19:ca8fdbba::::head [pgnls start_epoch 138 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4c33f500 con 0x563b4b80ac00 2023-05-20T01:28:49.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.826+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.2 v2:172.21.15.16:6816/293187030 40 ==== osd_op_reply(239 [pgnls start_epoch 138 out=101b] v138'28 uv28 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b80ac00 2023-05-20T01:28:49.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.826+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:240 19.13 19:ca8fdbba:::100000001fb.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91e380 con 0x563b4b80ac00 2023-05-20T01:28:49.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.826+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.2 v2:172.21.15.16:6816/293187030 41 ==== osd_op_reply(240 100000001fb.00000000 [stat out=16b] v0'0 uv21 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b80ac00 2023-05-20T01:28:49.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.826+0000 7fdd59b7c7c0 10 datascan.operator(): handling object 1099511628283.0 2023-05-20T01:28:49.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.826+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:241 19.13 19:ca8fdbba:::100000001fb.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91e700 con 0x563b4b80ac00 2023-05-20T01:28:49.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.827+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.2 v2:172.21.15.16:6816/293187030 42 ==== osd_op_reply(241 100000001fb.00000000 [call] v138'29 uv29 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b80ac00 2023-05-20T01:28:49.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.827+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:242 19.13 19:cbcb6224::::head [pgnls start_epoch 138 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b96d880 con 0x563b4b80ac00 2023-05-20T01:28:49.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.828+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.2 v2:172.21.15.16:6816/293187030 43 ==== osd_op_reply(242 [pgnls start_epoch 138 out=81b] v138'29 uv29 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b80ac00 2023-05-20T01:28:49.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.828+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:243 19.13 19:cbcb6224:::10000000212.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b96c700 con 0x563b4b80ac00 2023-05-20T01:28:49.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.828+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.2 v2:172.21.15.16:6816/293187030 44 ==== osd_op_reply(243 10000000212.00000000 [stat out=16b] v0'0 uv22 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b80ac00 2023-05-20T01:28:49.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.828+0000 7fdd59b7c7c0 10 datascan.operator(): handling object 1099511628306.0 2023-05-20T01:28:49.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.828+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:244 19.13 19:cbcb6224:::10000000212.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b96c000 con 0x563b4b80ac00 2023-05-20T01:28:49.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.829+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.2 v2:172.21.15.16:6816/293187030 45 ==== osd_op_reply(244 10000000212.00000000 [call] v138'30 uv30 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b80ac00 2023-05-20T01:28:49.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.829+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:245 19.33 19:cc000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b96ce00 con 0x563b4b80ac00 2023-05-20T01:28:49.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.829+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.2 v2:172.21.15.16:6816/293187030 46 ==== osd_op_reply(245 [pgnls start_epoch 138 out=81b] v136'12 uv12 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b80ac00 2023-05-20T01:28:49.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.829+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:246 19.33 19:cf9e7a4f:::10000000248.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b96d180 con 0x563b4b80ac00 2023-05-20T01:28:49.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.830+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.2 v2:172.21.15.16:6816/293187030 47 ==== osd_op_reply(246 10000000248.00000000 [stat out=16b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b80ac00 2023-05-20T01:28:49.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.830+0000 7fdd59b7c7c0 10 datascan.operator(): handling object 1099511628360.0 2023-05-20T01:28:49.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.830+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:247 19.33 19:cf9e7a4f:::10000000248.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b96d500 con 0x563b4b80ac00 2023-05-20T01:28:49.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.831+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.2 v2:172.21.15.16:6816/293187030 48 ==== osd_op_reply(247 10000000248.00000000 [call] v138'13 uv13 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b80ac00 2023-05-20T01:28:49.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.831+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:248 19.b 19:d0000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91ea80 con 0x563b4b7e8c00 2023-05-20T01:28:49.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.831+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.4 v2:172.21.15.132:6817/506317223 17 ==== osd_op_reply(248 [pgnls start_epoch 138 out=101b] v136'14 uv14 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b7e8c00 2023-05-20T01:28:49.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.831+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:249 19.b 19:d0d26e50:::10000000225.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91f180 con 0x563b4b7e8c00 2023-05-20T01:28:49.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.832+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.4 v2:172.21.15.132:6817/506317223 18 ==== osd_op_reply(249 10000000225.00000000 [stat out=16b] v0'0 uv14 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b7e8c00 2023-05-20T01:28:49.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.832+0000 7fdd59b7c7c0 10 datascan.operator(): handling object 1099511628325.0 2023-05-20T01:28:49.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.832+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:250 19.b 19:d0d26e50:::10000000225.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91f500 con 0x563b4b7e8c00 2023-05-20T01:28:49.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.833+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.4 v2:172.21.15.132:6817/506317223 19 ==== osd_op_reply(250 10000000225.00000000 [call] v138'15 uv15 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b7e8c00 2023-05-20T01:28:49.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.833+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:251 19.b 19:d2dbd4ac::::head [pgnls start_epoch 138 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91f880 con 0x563b4b7e8c00 2023-05-20T01:28:49.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.833+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.4 v2:172.21.15.132:6817/506317223 20 ==== osd_op_reply(251 [pgnls start_epoch 138 out=101b] v138'15 uv15 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b7e8c00 2023-05-20T01:28:49.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.833+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:252 19.b 19:d2dbd4ac:::10000000217.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91fc00 con 0x563b4b7e8c00 2023-05-20T01:28:49.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.834+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.4 v2:172.21.15.132:6817/506317223 21 ==== osd_op_reply(252 10000000217.00000000 [stat out=16b] v0'0 uv13 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b7e8c00 2023-05-20T01:28:49.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.834+0000 7fdd59b7c7c0 10 datascan.operator(): handling object 1099511628311.0 2023-05-20T01:28:49.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.834+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:253 19.b 19:d2dbd4ac:::10000000217.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b948700 con 0x563b4b7e8c00 2023-05-20T01:28:49.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.835+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.4 v2:172.21.15.132:6817/506317223 22 ==== osd_op_reply(253 10000000217.00000000 [call] v138'16 uv16 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b7e8c00 2023-05-20T01:28:49.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.835+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:254 19.b 19:d3eda40d::::head [pgnls start_epoch 138 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91e380 con 0x563b4b7e8c00 2023-05-20T01:28:49.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.835+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.4 v2:172.21.15.132:6817/506317223 23 ==== osd_op_reply(254 [pgnls start_epoch 138 out=81b] v138'16 uv16 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b7e8c00 2023-05-20T01:28:49.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.836+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:255 19.b 19:d3eda40d:::1000000020f.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91e700 con 0x563b4b7e8c00 2023-05-20T01:28:49.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.836+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.4 v2:172.21.15.132:6817/506317223 24 ==== osd_op_reply(255 1000000020f.00000000 [stat out=16b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b7e8c00 2023-05-20T01:28:49.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.836+0000 7fdd59b7c7c0 10 datascan.operator(): handling object 1099511628303.0 2023-05-20T01:28:49.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.836+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:256 19.b 19:d3eda40d:::1000000020f.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91ea80 con 0x563b4b7e8c00 2023-05-20T01:28:49.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.837+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.4 v2:172.21.15.132:6817/506317223 25 ==== osd_op_reply(256 1000000020f.00000000 [call] v138'17 uv17 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b7e8c00 2023-05-20T01:28:49.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.837+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:257 19.2b 19:d4000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91f180 con 0x563b4b7e9c00 2023-05-20T01:28:49.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.838+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.7 v2:172.21.15.132:6800/466001149 26 ==== osd_op_reply(257 [pgnls start_epoch 138 out=101b] v136'15 uv15 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b7e9c00 2023-05-20T01:28:49.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.838+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:258 19.2b 19:d4a58307:::10000000203.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91f500 con 0x563b4b7e9c00 2023-05-20T01:28:49.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.838+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.7 v2:172.21.15.132:6800/466001149 27 ==== osd_op_reply(258 10000000203.00000000 [stat out=16b] v0'0 uv14 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b7e9c00 2023-05-20T01:28:49.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.838+0000 7fdd59b7c7c0 10 datascan.operator(): handling object 1099511628291.0 2023-05-20T01:28:49.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.838+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:259 19.2b 19:d4a58307:::10000000203.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91f880 con 0x563b4b7e9c00 2023-05-20T01:28:49.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.839+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.7 v2:172.21.15.132:6800/466001149 28 ==== osd_op_reply(259 10000000203.00000000 [call] v138'16 uv16 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b7e9c00 2023-05-20T01:28:49.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.839+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:260 19.2b 19:d5ee56ee::::head [pgnls start_epoch 138 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b96d880 con 0x563b4b7e9c00 2023-05-20T01:28:49.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.840+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.7 v2:172.21.15.132:6800/466001149 29 ==== osd_op_reply(260 [pgnls start_epoch 138 out=81b] v138'16 uv16 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b7e9c00 2023-05-20T01:28:49.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.840+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:261 19.2b 19:d5ee56ee:::1000000022d.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b96c700 con 0x563b4b7e9c00 2023-05-20T01:28:49.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.840+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.7 v2:172.21.15.132:6800/466001149 30 ==== osd_op_reply(261 1000000022d.00000000 [stat out=16b] v0'0 uv15 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b7e9c00 2023-05-20T01:28:49.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.840+0000 7fdd59b7c7c0 10 datascan.operator(): handling object 1099511628333.0 2023-05-20T01:28:49.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.840+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:262 19.2b 19:d5ee56ee:::1000000022d.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b96c000 con 0x563b4b7e9c00 2023-05-20T01:28:49.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.841+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.7 v2:172.21.15.132:6800/466001149 31 ==== osd_op_reply(262 1000000022d.00000000 [call] v138'17 uv17 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b7e9c00 2023-05-20T01:28:49.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.841+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:263 19.1b 19:d8000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b96ce00 con 0x563b4b7e8c00 2023-05-20T01:28:49.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.841+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.4 v2:172.21.15.132:6817/506317223 26 ==== osd_op_reply(263 [pgnls start_epoch 138 out=49b] v136'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b7e8c00 2023-05-20T01:28:49.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.841+0000 7fdd52b20700 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:264 19.3b 19:dc000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91e000 con 0x563b4b7e9c00 2023-05-20T01:28:49.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.842+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.7 v2:172.21.15.132:6800/466001149 32 ==== osd_op_reply(264 [pgnls start_epoch 138 out=101b] v136'10 uv10 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b7e9c00 2023-05-20T01:28:49.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.842+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:265 19.3b 19:dd50df43:::10000000249.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b96d180 con 0x563b4b7e9c00 2023-05-20T01:28:49.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.842+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.7 v2:172.21.15.132:6800/466001149 33 ==== osd_op_reply(265 10000000249.00000000 [stat out=16b] v0'0 uv9 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b7e9c00 2023-05-20T01:28:49.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.842+0000 7fdd59b7c7c0 10 datascan.operator(): handling object 1099511628361.0 2023-05-20T01:28:49.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.842+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:266 19.3b 19:dd50df43:::10000000249.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b96d500 con 0x563b4b7e9c00 2023-05-20T01:28:49.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.843+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.7 v2:172.21.15.132:6800/466001149 34 ==== osd_op_reply(266 10000000249.00000000 [call] v138'11 uv11 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b7e9c00 2023-05-20T01:28:49.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.844+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:267 19.3b 19:deccbd37::::head [pgnls start_epoch 138 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4c33f500 con 0x563b4b7e9c00 2023-05-20T01:28:49.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.844+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.7 v2:172.21.15.132:6800/466001149 35 ==== osd_op_reply(267 [pgnls start_epoch 138 out=101b] v138'11 uv11 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b7e9c00 2023-05-20T01:28:49.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.844+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:268 19.3b 19:deccbd37:::100000001fe.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91f180 con 0x563b4b7e9c00 2023-05-20T01:28:49.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.844+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.7 v2:172.21.15.132:6800/466001149 36 ==== osd_op_reply(268 100000001fe.00000000 [stat out=16b] v0'0 uv8 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b7e9c00 2023-05-20T01:28:49.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.844+0000 7fdd59b7c7c0 10 datascan.operator(): handling object 1099511628286.0 2023-05-20T01:28:49.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.844+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:269 19.3b 19:deccbd37:::100000001fe.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91f500 con 0x563b4b7e9c00 2023-05-20T01:28:49.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.845+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.7 v2:172.21.15.132:6800/466001149 37 ==== osd_op_reply(269 100000001fe.00000000 [call] v138'12 uv12 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b7e9c00 2023-05-20T01:28:49.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.845+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:270 19.3b 19:ded14b84::::head [pgnls start_epoch 138 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91f880 con 0x563b4b7e9c00 2023-05-20T01:28:49.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.846+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.7 v2:172.21.15.132:6800/466001149 38 ==== osd_op_reply(270 [pgnls start_epoch 138 out=81b] v138'12 uv12 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b7e9c00 2023-05-20T01:28:49.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.846+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:271 19.3b 19:ded14b84:::1000000024b.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91e000 con 0x563b4b7e9c00 2023-05-20T01:28:49.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.846+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.7 v2:172.21.15.132:6800/466001149 39 ==== osd_op_reply(271 1000000024b.00000000 [stat out=16b] v0'0 uv10 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b7e9c00 2023-05-20T01:28:49.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.846+0000 7fdd59b7c7c0 10 datascan.operator(): handling object 1099511628363.0 2023-05-20T01:28:49.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.846+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:272 19.3b 19:ded14b84:::1000000024b.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b96dc00 con 0x563b4b7e9c00 2023-05-20T01:28:49.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.847+0000 7fdd57329700 1 -- 172.21.15.16:0/1962333830 <== osd.7 v2:172.21.15.132:6800/466001149 40 ==== osd_op_reply(272 1000000024b.00000000 [call] v138'13 uv13 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563b4b7e5440 con 0x563b4b7e9c00 2023-05-20T01:28:49.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.847+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:273 19.7 19:e0000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b96d880 con 0x563b4b80ac00 2023-05-20T01:28:49.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.847+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.2 v2:172.21.15.16:6816/293187030 49 ==== osd_op_reply(273 [pgnls start_epoch 138 out=101b] v136'12 uv12 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b80ac00 2023-05-20T01:28:49.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.847+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:274 19.7 19:e15a914f:::10000000207.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b96c700 con 0x563b4b80ac00 2023-05-20T01:28:49.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.848+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.2 v2:172.21.15.16:6816/293187030 50 ==== osd_op_reply(274 10000000207.00000000 [stat out=16b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b80ac00 2023-05-20T01:28:49.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.848+0000 7fdd59b7c7c0 10 datascan.operator(): handling object 1099511628295.0 2023-05-20T01:28:49.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.848+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:275 19.7 19:e15a914f:::10000000207.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b96c000 con 0x563b4b80ac00 2023-05-20T01:28:49.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.849+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.2 v2:172.21.15.16:6816/293187030 51 ==== osd_op_reply(275 10000000207.00000000 [call] v138'13 uv13 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b80ac00 2023-05-20T01:28:49.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.849+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:276 19.7 19:e2e07930::::head [pgnls start_epoch 138 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4c33e700 con 0x563b4b80ac00 2023-05-20T01:28:49.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.849+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.2 v2:172.21.15.16:6816/293187030 52 ==== osd_op_reply(276 [pgnls start_epoch 138 out=101b] v138'13 uv13 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b80ac00 2023-05-20T01:28:49.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.849+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:277 19.7 19:e2e07930:::10000000001.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4c33ee00 con 0x563b4b80ac00 2023-05-20T01:28:49.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.849+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.2 v2:172.21.15.16:6816/293187030 53 ==== osd_op_reply(277 10000000001.00000000 [stat out=16b] v0'0 uv10 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b80ac00 2023-05-20T01:28:49.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.849+0000 7fdd59b7c7c0 10 datascan.operator(): handling object 1099511627777.0 2023-05-20T01:28:49.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.849+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:278 19.7 19:e2e07930:::10000000001.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4c33fc00 con 0x563b4b80ac00 2023-05-20T01:28:49.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.850+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.2 v2:172.21.15.16:6816/293187030 54 ==== osd_op_reply(278 10000000001.00000000 [call] v138'14 uv14 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b80ac00 2023-05-20T01:28:49.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.850+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:279 19.7 19:e37752f2::::head [pgnls start_epoch 138 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4c33f880 con 0x563b4b80ac00 2023-05-20T01:28:49.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.850+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.2 v2:172.21.15.16:6816/293187030 55 ==== osd_op_reply(279 [pgnls start_epoch 138 out=81b] v138'14 uv14 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b80ac00 2023-05-20T01:28:49.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.850+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:280 19.7 19:e37752f2:::1000000021f.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4c33ea80 con 0x563b4b80ac00 2023-05-20T01:28:49.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.851+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.2 v2:172.21.15.16:6816/293187030 56 ==== osd_op_reply(280 1000000021f.00000000 [stat out=16b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b80ac00 2023-05-20T01:28:49.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.851+0000 7fdd59b7c7c0 10 datascan.operator(): handling object 1099511628319.0 2023-05-20T01:28:49.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.851+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:281 19.7 19:e37752f2:::1000000021f.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4c33f180 con 0x563b4b80ac00 2023-05-20T01:28:49.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.852+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.2 v2:172.21.15.16:6816/293187030 57 ==== osd_op_reply(281 1000000021f.00000000 [call] v138'15 uv15 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b80ac00 2023-05-20T01:28:49.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.852+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:282 19.27 19:e4000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b949c00 con 0x563b4b7e9000 2023-05-20T01:28:49.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.852+0000 7fdd57b2a700 1 -- 172.21.15.16:0/1962333830 <== osd.1 v2:172.21.15.16:6808/3860264220 45 ==== osd_op_reply(282 [pgnls start_epoch 138 out=101b] v136'12 uv12 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x563b4b7e58c0 con 0x563b4b7e9000 2023-05-20T01:28:49.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.852+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:283 19.27 19:e40c9c52:::10000000210.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4c33e700 con 0x563b4b7e9000 2023-05-20T01:28:49.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.853+0000 7fdd57b2a700 1 -- 172.21.15.16:0/1962333830 <== osd.1 v2:172.21.15.16:6808/3860264220 46 ==== osd_op_reply(283 10000000210.00000000 [stat out=16b] v0'0 uv9 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b4b7e58c0 con 0x563b4b7e9000 2023-05-20T01:28:49.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.853+0000 7fdd59b7c7c0 10 datascan.operator(): handling object 1099511628304.0 2023-05-20T01:28:49.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.853+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:284 19.27 19:e40c9c52:::10000000210.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4c33ee00 con 0x563b4b7e9000 2023-05-20T01:28:49.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.854+0000 7fdd57b2a700 1 -- 172.21.15.16:0/1962333830 <== osd.1 v2:172.21.15.16:6808/3860264220 47 ==== osd_op_reply(284 10000000210.00000000 [call] v138'13 uv13 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563b4b7e58c0 con 0x563b4b7e9000 2023-05-20T01:28:49.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.854+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:285 19.27 19:e5671758::::head [pgnls start_epoch 138 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4c33fc00 con 0x563b4b7e9000 2023-05-20T01:28:49.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.854+0000 7fdd57b2a700 1 -- 172.21.15.16:0/1962333830 <== osd.1 v2:172.21.15.16:6808/3860264220 48 ==== osd_op_reply(285 [pgnls start_epoch 138 out=101b] v138'13 uv13 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x563b4b7e58c0 con 0x563b4b7e9000 2023-05-20T01:28:49.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.854+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:286 19.27 19:e5671758:::1000000024f.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4c33f880 con 0x563b4b7e9000 2023-05-20T01:28:49.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.854+0000 7fdd57b2a700 1 -- 172.21.15.16:0/1962333830 <== osd.1 v2:172.21.15.16:6808/3860264220 49 ==== osd_op_reply(286 1000000024f.00000000 [stat out=16b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b4b7e58c0 con 0x563b4b7e9000 2023-05-20T01:28:49.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.854+0000 7fdd59b7c7c0 10 datascan.operator(): handling object 1099511628367.0 2023-05-20T01:28:49.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.854+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:287 19.27 19:e5671758:::1000000024f.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4c33ea80 con 0x563b4b7e9000 2023-05-20T01:28:49.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.855+0000 7fdd57b2a700 1 -- 172.21.15.16:0/1962333830 <== osd.1 v2:172.21.15.16:6808/3860264220 50 ==== osd_op_reply(287 1000000024f.00000000 [call] v138'14 uv14 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563b4b7e58c0 con 0x563b4b7e9000 2023-05-20T01:28:49.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.855+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:288 19.27 19:e5993619::::head [pgnls start_epoch 138 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b96ce00 con 0x563b4b7e9000 2023-05-20T01:28:49.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.855+0000 7fdd57b2a700 1 -- 172.21.15.16:0/1962333830 <== osd.1 v2:172.21.15.16:6808/3860264220 51 ==== osd_op_reply(288 [pgnls start_epoch 138 out=101b] v138'14 uv14 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x563b4b7e58c0 con 0x563b4b7e9000 2023-05-20T01:28:49.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.855+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:289 19.27 19:e5993619:::10000000208.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b96d880 con 0x563b4b7e9000 2023-05-20T01:28:49.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.856+0000 7fdd57b2a700 1 -- 172.21.15.16:0/1962333830 <== osd.1 v2:172.21.15.16:6808/3860264220 52 ==== osd_op_reply(289 10000000208.00000000 [stat out=16b] v0'0 uv8 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b4b7e58c0 con 0x563b4b7e9000 2023-05-20T01:28:49.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.856+0000 7fdd59b7c7c0 10 datascan.operator(): handling object 1099511628296.0 2023-05-20T01:28:49.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.856+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:290 19.27 19:e5993619:::10000000208.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b96c700 con 0x563b4b7e9000 2023-05-20T01:28:49.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.857+0000 7fdd57b2a700 1 -- 172.21.15.16:0/1962333830 <== osd.1 v2:172.21.15.16:6808/3860264220 53 ==== osd_op_reply(290 10000000208.00000000 [call] v138'15 uv15 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563b4b7e58c0 con 0x563b4b7e9000 2023-05-20T01:28:49.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.857+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:291 19.27 19:e5a5749e::::head [pgnls start_epoch 138 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b96c000 con 0x563b4b7e9000 2023-05-20T01:28:49.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.857+0000 7fdd57b2a700 1 -- 172.21.15.16:0/1962333830 <== osd.1 v2:172.21.15.16:6808/3860264220 54 ==== osd_op_reply(291 [pgnls start_epoch 138 out=101b] v138'15 uv15 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x563b4b7e58c0 con 0x563b4b7e9000 2023-05-20T01:28:49.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.857+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:292 19.27 19:e5a5749e:::1000000021a.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91fc00 con 0x563b4b7e9000 2023-05-20T01:28:49.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.857+0000 7fdd57b2a700 1 -- 172.21.15.16:0/1962333830 <== osd.1 v2:172.21.15.16:6808/3860264220 55 ==== osd_op_reply(292 1000000021a.00000000 [stat out=16b] v0'0 uv10 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b4b7e58c0 con 0x563b4b7e9000 2023-05-20T01:28:49.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.857+0000 7fdd59b7c7c0 10 datascan.operator(): handling object 1099511628314.0 2023-05-20T01:28:49.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.857+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:293 19.27 19:e5a5749e:::1000000021a.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91e380 con 0x563b4b7e9000 2023-05-20T01:28:49.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.858+0000 7fdd57b2a700 1 -- 172.21.15.16:0/1962333830 <== osd.1 v2:172.21.15.16:6808/3860264220 56 ==== osd_op_reply(293 1000000021a.00000000 [call] v138'16 uv16 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563b4b7e58c0 con 0x563b4b7e9000 2023-05-20T01:28:49.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.858+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:294 19.27 19:e692fc64::::head [pgnls start_epoch 138 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91e700 con 0x563b4b7e9000 2023-05-20T01:28:49.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.859+0000 7fdd57b2a700 1 -- 172.21.15.16:0/1962333830 <== osd.1 v2:172.21.15.16:6808/3860264220 57 ==== osd_op_reply(294 [pgnls start_epoch 138 out=81b] v138'16 uv16 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x563b4b7e58c0 con 0x563b4b7e9000 2023-05-20T01:28:49.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.859+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:295 19.27 19:e692fc64:::1000000021b.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91ea80 con 0x563b4b7e9000 2023-05-20T01:28:49.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.859+0000 7fdd57b2a700 1 -- 172.21.15.16:0/1962333830 <== osd.1 v2:172.21.15.16:6808/3860264220 58 ==== osd_op_reply(295 1000000021b.00000000 [stat out=16b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b4b7e58c0 con 0x563b4b7e9000 2023-05-20T01:28:49.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.859+0000 7fdd59b7c7c0 10 datascan.operator(): handling object 1099511628315.0 2023-05-20T01:28:49.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.859+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:296 19.27 19:e692fc64:::1000000021b.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b948700 con 0x563b4b7e9000 2023-05-20T01:28:49.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.860+0000 7fdd57b2a700 1 -- 172.21.15.16:0/1962333830 <== osd.1 v2:172.21.15.16:6808/3860264220 59 ==== osd_op_reply(296 1000000021b.00000000 [call] v138'17 uv17 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563b4b7e58c0 con 0x563b4b7e9000 2023-05-20T01:28:49.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.860+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:297 19.17 19:e8000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4c33fc00 con 0x563b4b7e9800 2023-05-20T01:28:49.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.860+0000 7fdd57b2a700 1 -- 172.21.15.16:0/1962333830 <== osd.5 v2:172.21.15.132:6816/1844689995 8 ==== osd_op_reply(297 [pgnls start_epoch 138 out=101b] v136'13 uv13 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x563b4b7e58c0 con 0x563b4b7e9800 2023-05-20T01:28:49.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.860+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:298 19.17 19:eace0b58:::10000000215.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4c33f880 con 0x563b4b7e9800 2023-05-20T01:28:49.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.860+0000 7fdd57b2a700 1 -- 172.21.15.16:0/1962333830 <== osd.5 v2:172.21.15.132:6816/1844689995 9 ==== osd_op_reply(298 10000000215.00000000 [stat out=16b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b4b7e58c0 con 0x563b4b7e9800 2023-05-20T01:28:49.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.860+0000 7fdd59b7c7c0 10 datascan.operator(): handling object 1099511628309.0 2023-05-20T01:28:49.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.861+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:299 19.17 19:eace0b58:::10000000215.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4c33ea80 con 0x563b4b7e9800 2023-05-20T01:28:49.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.862+0000 7fdd57b2a700 1 -- 172.21.15.16:0/1962333830 <== osd.5 v2:172.21.15.132:6816/1844689995 10 ==== osd_op_reply(299 10000000215.00000000 [call] v138'14 uv14 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563b4b7e58c0 con 0x563b4b7e9800 2023-05-20T01:28:49.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.862+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:300 19.17 19:eb1b33a4::::head [pgnls start_epoch 138 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4c33e700 con 0x563b4b7e9800 2023-05-20T01:28:49.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.862+0000 7fdd57b2a700 1 -- 172.21.15.16:0/1962333830 <== osd.5 v2:172.21.15.132:6816/1844689995 11 ==== osd_op_reply(300 [pgnls start_epoch 138 out=81b] v138'14 uv14 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x563b4b7e58c0 con 0x563b4b7e9800 2023-05-20T01:28:49.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.862+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:301 19.17 19:eb1b33a4:::10000000253.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4c33ee00 con 0x563b4b7e9800 2023-05-20T01:28:49.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.862+0000 7fdd57b2a700 1 -- 172.21.15.16:0/1962333830 <== osd.5 v2:172.21.15.132:6816/1844689995 12 ==== osd_op_reply(301 10000000253.00000000 [stat out=16b] v0'0 uv13 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b4b7e58c0 con 0x563b4b7e9800 2023-05-20T01:28:49.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.862+0000 7fdd59b7c7c0 10 datascan.operator(): handling object 1099511628371.0 2023-05-20T01:28:49.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.862+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:302 19.17 19:eb1b33a4:::10000000253.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b948000 con 0x563b4b7e9800 2023-05-20T01:28:49.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.863+0000 7fdd57b2a700 1 -- 172.21.15.16:0/1962333830 <== osd.5 v2:172.21.15.132:6816/1844689995 13 ==== osd_op_reply(302 10000000253.00000000 [call] v138'15 uv15 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563b4b7e58c0 con 0x563b4b7e9800 2023-05-20T01:28:49.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.863+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:303 19.37 19:ec000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b948380 con 0x563b4b7e9400 2023-05-20T01:28:49.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.863+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.6 v2:172.21.15.132:6801/727581123 23 ==== osd_op_reply(303 [pgnls start_epoch 138 out=49b] v136'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b7e9400 2023-05-20T01:28:49.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.864+0000 7fdd52b20700 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:304 19.f 19:f0000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b948a80 con 0x563b4b7e9800 2023-05-20T01:28:49.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.864+0000 7fdd57b2a700 1 -- 172.21.15.16:0/1962333830 <== osd.5 v2:172.21.15.132:6816/1844689995 14 ==== osd_op_reply(304 [pgnls start_epoch 138 out=81b] v136'11 uv11 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x563b4b7e58c0 con 0x563b4b7e9800 2023-05-20T01:28:49.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.864+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:305 19.f 19:f25dd5df:::1000000022e.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b949180 con 0x563b4b7e9800 2023-05-20T01:28:49.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.864+0000 7fdd57b2a700 1 -- 172.21.15.16:0/1962333830 <== osd.5 v2:172.21.15.132:6816/1844689995 15 ==== osd_op_reply(305 1000000022e.00000000 [stat out=16b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b4b7e58c0 con 0x563b4b7e9800 2023-05-20T01:28:49.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.864+0000 7fdd59b7c7c0 10 datascan.operator(): handling object 1099511628334.0 2023-05-20T01:28:49.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.864+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:306 19.f 19:f25dd5df:::1000000022e.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b949500 con 0x563b4b7e9800 2023-05-20T01:28:49.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.865+0000 7fdd57b2a700 1 -- 172.21.15.16:0/1962333830 <== osd.5 v2:172.21.15.132:6816/1844689995 16 ==== osd_op_reply(306 1000000022e.00000000 [call] v138'12 uv12 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563b4b7e58c0 con 0x563b4b7e9800 2023-05-20T01:28:49.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.865+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:307 19.2f 19:f4000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b949880 con 0x563b4b7e8c00 2023-05-20T01:28:49.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.866+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.4 v2:172.21.15.132:6817/506317223 27 ==== osd_op_reply(307 [pgnls start_epoch 138 out=49b] v136'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b7e8c00 2023-05-20T01:28:49.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.866+0000 7fdd52b20700 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:308 19.1f 19:f8000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b948e00 con 0x563b4b7e9400 2023-05-20T01:28:49.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.866+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.6 v2:172.21.15.132:6801/727581123 24 ==== osd_op_reply(308 [pgnls start_epoch 138 out=101b] v136'10 uv10 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b7e9400 2023-05-20T01:28:49.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.866+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:309 19.1f 19:f87ea8f1:::10000000238.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b949c00 con 0x563b4b7e9400 2023-05-20T01:28:49.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.866+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.6 v2:172.21.15.132:6801/727581123 25 ==== osd_op_reply(309 10000000238.00000000 [stat out=16b] v0'0 uv10 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b7e9400 2023-05-20T01:28:49.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.866+0000 7fdd59b7c7c0 10 datascan.operator(): handling object 1099511628344.0 2023-05-20T01:28:49.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.866+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:310 19.1f 19:f87ea8f1:::10000000238.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91fc00 con 0x563b4b7e9400 2023-05-20T01:28:49.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.867+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.6 v2:172.21.15.132:6801/727581123 26 ==== osd_op_reply(310 10000000238.00000000 [call] v138'11 uv11 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b7e9400 2023-05-20T01:28:49.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.867+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:311 19.1f 19:f8e2d5cc::::head [pgnls start_epoch 138 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91e380 con 0x563b4b7e9400 2023-05-20T01:28:49.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.868+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.6 v2:172.21.15.132:6801/727581123 27 ==== osd_op_reply(311 [pgnls start_epoch 138 out=81b] v138'11 uv11 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b7e9400 2023-05-20T01:28:49.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.868+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:312 19.1f 19:f8e2d5cc:::10000000214.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91e700 con 0x563b4b7e9400 2023-05-20T01:28:49.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.868+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.6 v2:172.21.15.132:6801/727581123 28 ==== osd_op_reply(312 10000000214.00000000 [stat out=16b] v0'0 uv9 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b7e9400 2023-05-20T01:28:49.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.868+0000 7fdd59b7c7c0 10 datascan.operator(): handling object 1099511628308.0 2023-05-20T01:28:49.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.868+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:313 19.1f 19:f8e2d5cc:::10000000214.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91ea80 con 0x563b4b7e9400 2023-05-20T01:28:49.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.869+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.6 v2:172.21.15.132:6801/727581123 29 ==== osd_op_reply(313 10000000214.00000000 [call] v138'12 uv12 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b7e9400 2023-05-20T01:28:49.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.869+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:314 19.3f 19:fc000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b91ee00 con 0x563b4b80ac00 2023-05-20T01:28:49.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.869+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.2 v2:172.21.15.16:6816/293187030 58 ==== osd_op_reply(314 [pgnls start_epoch 138 out=101b] v136'12 uv12 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b80ac00 2023-05-20T01:28:49.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.869+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:315 19.3f 19:fd83cc12:::10000000232.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b96ce00 con 0x563b4b80ac00 2023-05-20T01:28:49.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.869+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.2 v2:172.21.15.16:6816/293187030 59 ==== osd_op_reply(315 10000000232.00000000 [stat out=16b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b80ac00 2023-05-20T01:28:49.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.869+0000 7fdd59b7c7c0 10 datascan.operator(): handling object 1099511628338.0 2023-05-20T01:28:49.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.869+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:316 19.3f 19:fd83cc12:::10000000232.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b96d880 con 0x563b4b80ac00 2023-05-20T01:28:49.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.871+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.2 v2:172.21.15.16:6816/293187030 60 ==== osd_op_reply(316 10000000232.00000000 [call] v138'13 uv13 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b80ac00 2023-05-20T01:28:49.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.871+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:317 19.3f 19:feb1c803::::head [pgnls start_epoch 138 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b96c700 con 0x563b4b80ac00 2023-05-20T01:28:49.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.871+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.2 v2:172.21.15.16:6816/293187030 61 ==== osd_op_reply(317 [pgnls start_epoch 138 out=81b] v138'13 uv13 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b80ac00 2023-05-20T01:28:49.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.871+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:318 19.3f 19:feb1c803:::100000001fa.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b96c000 con 0x563b4b80ac00 2023-05-20T01:28:49.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.872+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.2 v2:172.21.15.16:6816/293187030 62 ==== osd_op_reply(318 100000001fa.00000000 [stat out=16b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b80ac00 2023-05-20T01:28:49.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.872+0000 7fdd59b7c7c0 10 datascan.operator(): handling object 1099511628282.0 2023-05-20T01:28:49.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.872+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:319 19.3f 19:feb1c803:::100000001fa.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) v8 -- 0x563b4b96c380 con 0x563b4b80ac00 2023-05-20T01:28:49.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.873+0000 7fdd5832b700 1 -- 172.21.15.16:0/1962333830 <== osd.2 v2:172.21.15.16:6816/293187030 63 ==== osd_op_reply(319 100000001fa.00000000 [call] v138'14 uv14 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563b4b7e5680 con 0x563b4b80ac00 2023-05-20T01:28:49.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.873+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x563b4b80a800 msgr2=0x563b4b924000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:49.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.873+0000 7fdd59b7c7c0 1 --2- 172.21.15.16:0/1962333830 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x563b4b80a800 0x563b4b924000 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:49.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.873+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x563b4b7e9000 msgr2=0x563b4b55f600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:49.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.873+0000 7fdd59b7c7c0 1 --2- 172.21.15.16:0/1962333830 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x563b4b7e9000 0x563b4b55f600 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:49.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.873+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x563b4b80ac00 msgr2=0x563b4b926c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:49.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.873+0000 7fdd59b7c7c0 1 --2- 172.21.15.16:0/1962333830 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x563b4b80ac00 0x563b4b926c00 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:49.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.873+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x563b4b7e8800 msgr2=0x563b4b560100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:49.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.873+0000 7fdd59b7c7c0 1 --2- 172.21.15.16:0/1962333830 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x563b4b7e8800 0x563b4b560100 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:49.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.873+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x563b4b7e8c00 msgr2=0x563b4b55fb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:49.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.873+0000 7fdd59b7c7c0 1 --2- 172.21.15.16:0/1962333830 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x563b4b7e8c00 0x563b4b55fb80 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:49.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.873+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x563b4b7e9800 msgr2=0x563b4b924b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:49.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.873+0000 7fdd59b7c7c0 1 --2- 172.21.15.16:0/1962333830 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x563b4b7e9800 0x563b4b924b00 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:49.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.873+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x563b4b7e9400 msgr2=0x563b4b924580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:49.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.873+0000 7fdd59b7c7c0 1 --2- 172.21.15.16:0/1962333830 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x563b4b7e9400 0x563b4b924580 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:49.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.873+0000 7fdd57b2a700 1 -- 172.21.15.16:0/1962333830 reap_dead start 2023-05-20T01:28:49.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.873+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x563b4b7e9c00 msgr2=0x563b4b925080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:49.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.873+0000 7fdd59b7c7c0 1 --2- 172.21.15.16:0/1962333830 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x563b4b7e9c00 0x563b4b925080 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:49.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.873+0000 7fdd57b2a700 1 -- 172.21.15.16:0/1962333830 reap_dead start 2023-05-20T01:28:49.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.873+0000 7fdd57b2a700 1 -- 172.21.15.16:0/1962333830 reap_dead start 2023-05-20T01:28:49.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.873+0000 7fdd57b2a700 1 -- 172.21.15.16:0/1962333830 reap_dead start 2023-05-20T01:28:49.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.873+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x563b4b80a400 msgr2=0x563b4b561700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:49.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.873+0000 7fdd59b7c7c0 1 --2- 172.21.15.16:0/1962333830 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x563b4b80a400 0x563b4b561700 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x563b4b7f00f0 tx=0x563b4c35ec00 comp rx=0 tx=0).stop 2023-05-20T01:28:49.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.873+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b4b4c5800 msgr2=0x563b4b561180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:49.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.873+0000 7fdd59b7c7c0 1 --2- 172.21.15.16:0/1962333830 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b4b4c5800 0x563b4b561180 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x563b4b7f13e0 tx=0x563b4c29a000 comp rx=0 tx=0).stop 2023-05-20T01:28:49.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.873+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 shutdown_connections 2023-05-20T01:28:49.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.873+0000 7fdd59b7c7c0 1 --2- 172.21.15.16:0/1962333830 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x563b4b7e9c00 0x563b4b925080 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:49.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.873+0000 7fdd59b7c7c0 1 --2- 172.21.15.16:0/1962333830 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x563b4b80a400 0x563b4b561700 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:49.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.873+0000 7fdd59b7c7c0 1 --2- 172.21.15.16:0/1962333830 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b4b4c5800 0x563b4b561180 unknown :-1 s=CLOSED pgs=1717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:49.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.873+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 >> 172.21.15.16:0/1962333830 conn(0x563b4b4c5c00 msgr2=0x563b4b5a1800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:49.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.873+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 shutdown_connections 2023-05-20T01:28:49.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.873+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1962333830 wait complete. 2023-05-20T01:28:49.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.873+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1784838100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b4b4c4c00 msgr2=0x563b4b55eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:49.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.873+0000 7fdd59b7c7c0 1 --2- 172.21.15.16:0/1784838100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b4b4c4c00 0x563b4b55eb00 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x563b4c290e70 tx=0x563b4c29a3c0 comp rx=0 tx=0).stop 2023-05-20T01:28:49.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.873+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1784838100 shutdown_connections 2023-05-20T01:28:49.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.873+0000 7fdd59b7c7c0 1 --2- 172.21.15.16:0/1784838100 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563b4b4c5000 0x563b4b55f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:49.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.873+0000 7fdd59b7c7c0 1 --2- 172.21.15.16:0/1784838100 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563b4b4c4800 0x563b4b55e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:49.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.873+0000 7fdd59b7c7c0 1 --2- 172.21.15.16:0/1784838100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b4b4c4c00 0x563b4b55eb00 unknown :-1 s=CLOSED pgs=1715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:49.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.873+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1784838100 >> 172.21.15.16:0/1784838100 conn(0x563b4b4c4400 msgr2=0x563b4b5a1000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:49.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.874+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1784838100 shutdown_connections 2023-05-20T01:28:49.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.874+0000 7fdd59b7c7c0 1 -- 172.21.15.16:0/1784838100 wait complete. 2023-05-20T01:28:49.884 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_extents'] in time 0:00:00.260562, result: 2023-05-20T01:28:49.885 DEBUG:teuthology.orchestra.run.smithi016:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_inodes 2023-05-20T01:28:49.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.960+0000 7fe45dd097c0 1 -- start start 2023-05-20T01:28:49.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.960+0000 7fe45dd097c0 1 Processor -- start 2023-05-20T01:28:49.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.961+0000 7fe45dd097c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5594efa88800 0x5594efb23b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:49.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.961+0000 7fe45dd097c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5594efa88c00 0x5594efb23600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:49.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.961+0000 7fe45dd097c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594efa89000 0x5594efb23080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:49.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.961+0000 7fe45dd097c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594efac4a00 con 0x5594efa89000 2023-05-20T01:28:49.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.961+0000 7fe45dd097c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594efac4b40 con 0x5594efa88800 2023-05-20T01:28:49.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.961+0000 7fe45dd097c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x5594efac4c80 con 0x5594efa88c00 2023-05-20T01:28:49.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.961+0000 7fe45c4b8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594efa89000 0x5594efb23080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:49.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.961+0000 7fe45c4b8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594efa89000 0x5594efb23080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40752/0 (socket says 172.21.15.16:40752) 2023-05-20T01:28:49.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.961+0000 7fe45c4b8700 1 -- 172.21.15.16:0/495675089 learned_addr learned my addr 172.21.15.16:0/495675089 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:49.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.961+0000 7fe45bcb7700 1 --2- 172.21.15.16:0/495675089 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5594efa88800 0x5594efb23b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:49.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.961+0000 7fe45b4b6700 1 --2- 172.21.15.16:0/495675089 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5594efa88c00 0x5594efb23600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:49.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.962+0000 7fe45c4b8700 1 -- 172.21.15.16:0/495675089 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5594efa88c00 msgr2=0x5594efb23600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:49.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.962+0000 7fe45c4b8700 1 --2- 172.21.15.16:0/495675089 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5594efa88c00 0x5594efb23600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:49.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.962+0000 7fe45c4b8700 1 -- 172.21.15.16:0/495675089 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5594efa88800 msgr2=0x5594efb23b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:49.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.962+0000 7fe45c4b8700 1 --2- 172.21.15.16:0/495675089 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5594efa88800 0x5594efb23b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:49.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.962+0000 7fe45c4b8700 1 -- 172.21.15.16:0/495675089 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5594efaacd00 con 0x5594efa89000 2023-05-20T01:28:49.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.962+0000 7fe45c4b8700 1 --2- 172.21.15.16:0/495675089 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594efa89000 0x5594efb23080 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x5594f0854e10 tx=0x5594f085e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76b5d8a45e6e8f10 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:49.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.962+0000 7fe4594b2700 1 -- 172.21.15.16:0/495675089 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5594efabd080 con 0x5594efa89000 2023-05-20T01:28:49.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.962+0000 7fe4594b2700 1 -- 172.21.15.16:0/495675089 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5594efadb800 con 0x5594efa89000 2023-05-20T01:28:49.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.962+0000 7fe45dd097c0 1 -- 172.21.15.16:0/495675089 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5594efaad6c0 con 0x5594efa89000 2023-05-20T01:28:49.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.962+0000 7fe45dd097c0 1 -- 172.21.15.16:0/495675089 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x5594efaad520 con 0x5594efa89000 2023-05-20T01:28:49.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.962+0000 7fe45dd097c0 4 waiting for MDS map... 2023-05-20T01:28:49.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.962+0000 7fe4594b2700 1 -- 172.21.15.16:0/495675089 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5594efabd600 con 0x5594efa89000 2023-05-20T01:28:49.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.962+0000 7fe4594b2700 1 -- 172.21.15.16:0/495675089 <== mon.0 v2:172.21.15.16:3300/0 4 ==== osd_map(138..138 src has 1..138) v4 ==== 6554+0+0 (secure 0 0 0) 0x5594efacac40 con 0x5594efa89000 2023-05-20T01:28:49.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.962+0000 7fe4594b2700 1 -- 172.21.15.16:0/495675089 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 106) v1 ==== 2573+0+0 (secure 0 0 0) 0x5594efacef00 con 0x5594efa89000 2023-05-20T01:28:49.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.962+0000 7fe45dd097c0 4 Got MDS map 106 2023-05-20T01:28:49.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.962+0000 7fe45dd097c0 4 datascan.main: Using metadata pool output 2023-05-20T01:28:49.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.962+0000 7fe45dd097c0 4 datascan.main: connecting to RADOS... 2023-05-20T01:28:49.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.963+0000 7fe45dd097c0 1 Processor -- start 2023-05-20T01:28:49.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.963+0000 7fe45dd097c0 1 -- start start 2023-05-20T01:28:49.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.963+0000 7fe45dd097c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5594efa89c00 0x5594efb22b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:49.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.963+0000 7fe45dd097c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5594efda8000 0x5594efb22580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:49.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.963+0000 7fe45dd097c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594efda8400 0x5594efb24100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:49.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.963+0000 7fe45dd097c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594efac4c80 con 0x5594efda8400 2023-05-20T01:28:49.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.963+0000 7fe45dd097c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594efac4b40 con 0x5594efda8000 2023-05-20T01:28:49.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.963+0000 7fe45dd097c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x5594efac5e00 con 0x5594efa89c00 2023-05-20T01:28:49.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.964+0000 7fe45b4b6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594efda8400 0x5594efb24100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:49.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.964+0000 7fe45b4b6700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5594efa89c00 0x5594efb22b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:49.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.964+0000 7fe45bcb7700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5594efda8000 0x5594efb22580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:49.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.964+0000 7fe45b4b6700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5594efa89c00 0x5594efb22b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.16:40792/0 (socket says 172.21.15.16:40792) 2023-05-20T01:28:49.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.964+0000 7fe45b4b6700 1 -- 172.21.15.16:0/3853154406 learned_addr learned my addr 172.21.15.16:0/3853154406 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:49.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.964+0000 7fe45bcb7700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5594efda8000 0x5594efb22580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.16:37778/0 (socket says 172.21.15.16:37778) 2023-05-20T01:28:49.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.964+0000 7fe45b4b6700 1 -- 172.21.15.16:0/3853154406 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5594efa89c00 msgr2=0x5594efb22b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:49.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.964+0000 7fe45b4b6700 1 --2- 172.21.15.16:0/3853154406 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5594efa89c00 0x5594efb22b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:49.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.964+0000 7fe45b4b6700 1 -- 172.21.15.16:0/3853154406 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5594efda8000 msgr2=0x5594efb22580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:49.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.964+0000 7fe45b4b6700 1 --2- 172.21.15.16:0/3853154406 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5594efda8000 0x5594efb22580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:49.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.964+0000 7fe45b4b6700 1 -- 172.21.15.16:0/3853154406 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5594efaad380 con 0x5594efda8400 2023-05-20T01:28:49.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.964+0000 7fe45bcb7700 1 --2- 172.21.15.16:0/3853154406 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5594efda8000 0x5594efb22580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-05-20T01:28:49.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.964+0000 7fe45b4b6700 1 --2- 172.21.15.16:0/3853154406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594efda8400 0x5594efb24100 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x5594efdbc0f0 tx=0x5594f0922c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83ef36610bfbadea server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:49.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.964+0000 7fe4574ae700 1 -- 172.21.15.16:0/3853154406 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5594efabd080 con 0x5594efda8400 2023-05-20T01:28:49.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.964+0000 7fe4574ae700 1 -- 172.21.15.16:0/3853154406 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5594f0920300 con 0x5594efda8400 2023-05-20T01:28:49.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.964+0000 7fe4574ae700 1 -- 172.21.15.16:0/3853154406 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5594efabd600 con 0x5594efda8400 2023-05-20T01:28:49.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.964+0000 7fe45dd097c0 1 -- 172.21.15.16:0/3853154406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594efda8400 msgr2=0x5594efb24100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:49.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.965+0000 7fe45dd097c0 1 --2- 172.21.15.16:0/3853154406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594efda8400 0x5594efb24100 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x5594efdbc0f0 tx=0x5594f0922c60 comp rx=0 tx=0).stop 2023-05-20T01:28:49.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.965+0000 7fe45dd097c0 1 -- 172.21.15.16:0/3853154406 shutdown_connections 2023-05-20T01:28:49.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.965+0000 7fe45dd097c0 1 --2- 172.21.15.16:0/3853154406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594efda8400 0x5594efb24100 unknown :-1 s=CLOSED pgs=1720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:49.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.965+0000 7fe45dd097c0 1 --2- 172.21.15.16:0/3853154406 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5594efa89c00 0x5594efb22b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:49.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.965+0000 7fe45dd097c0 1 --2- 172.21.15.16:0/3853154406 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5594efda8000 0x5594efb22580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:49.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.965+0000 7fe45dd097c0 1 -- 172.21.15.16:0/3853154406 >> 172.21.15.16:0/3853154406 conn(0x5594efa89800 msgr2=0x5594efb65800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:49.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.965+0000 7fe45dd097c0 1 -- 172.21.15.16:0/3853154406 shutdown_connections 2023-05-20T01:28:49.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.965+0000 7fe45dd097c0 1 -- 172.21.15.16:0/3853154406 wait complete. 2023-05-20T01:28:49.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.965+0000 7fe45dd097c0 1 Processor -- start 2023-05-20T01:28:49.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.965+0000 7fe45dd097c0 1 -- start start 2023-05-20T01:28:49.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.966+0000 7fe45dd097c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5594efa89800 0x5594efb22b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:49.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.966+0000 7fe45dd097c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5594efda8000 0x5594efb22580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:49.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.966+0000 7fe45dd097c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594efda8400 0x5594efb24100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:49.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.966+0000 7fe45dd097c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594efac5cc0 con 0x5594efda8400 2023-05-20T01:28:49.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.966+0000 7fe45dd097c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594efac4000 con 0x5594efda8000 2023-05-20T01:28:49.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.966+0000 7fe45dd097c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x5594efb16f00 con 0x5594efa89800 2023-05-20T01:28:49.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.966+0000 7fe45b4b6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594efda8400 0x5594efb24100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:49.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.966+0000 7fe45b4b6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594efda8400 0x5594efb24100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40764/0 (socket says 172.21.15.16:40764) 2023-05-20T01:28:49.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.966+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 learned_addr learned my addr 172.21.15.16:0/1888659255 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:49.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.966+0000 7fe45b4b6700 1 --2- 172.21.15.16:0/1888659255 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5594efa89800 0x5594efb22b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:49.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.966+0000 7fe45bcb7700 1 --2- 172.21.15.16:0/1888659255 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5594efda8000 0x5594efb22580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:49.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.967+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5594efa89800 msgr2=0x5594efb22b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:49.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.967+0000 7fe45b4b6700 1 --2- 172.21.15.16:0/1888659255 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5594efa89800 0x5594efb22b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:49.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.967+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5594efda8000 msgr2=0x5594efb22580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:49.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.967+0000 7fe45b4b6700 1 --2- 172.21.15.16:0/1888659255 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5594efda8000 0x5594efb22580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:49.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.967+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5594efdb41a0 con 0x5594efda8400 2023-05-20T01:28:49.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.967+0000 7fe45bcb7700 1 --2- 172.21.15.16:0/1888659255 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5594efda8000 0x5594efb22580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-05-20T01:28:49.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.967+0000 7fe45b4b6700 1 --2- 172.21.15.16:0/1888659255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594efda8400 0x5594efb24100 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x5594efdbd3b0 tx=0x5594f085e2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:49.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.967+0000 7fe455cab700 1 -- 172.21.15.16:0/1888659255 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5594efabcdc0 con 0x5594efda8400 2023-05-20T01:28:49.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.967+0000 7fe455cab700 1 -- 172.21.15.16:0/1888659255 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5594efada600 con 0x5594efda8400 2023-05-20T01:28:49.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.967+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5594efdb4000 con 0x5594efda8400 2023-05-20T01:28:49.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.967+0000 7fe455cab700 1 -- 172.21.15.16:0/1888659255 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5594efabd080 con 0x5594efda8400 2023-05-20T01:28:49.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.967+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5594efdb4340 con 0x5594efda8400 2023-05-20T01:28:49.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.967+0000 7fe45dd097c0 4 datascan.init: resolving metadata pool 18 2023-05-20T01:28:49.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.968+0000 7fe455cab700 1 -- 172.21.15.16:0/1888659255 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x5594efaea840 con 0x5594efda8400 2023-05-20T01:28:49.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.969+0000 7fe455cab700 1 --2- 172.21.15.16:0/1888659255 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x5594efdd8800 0x5594efec6000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:49.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.969+0000 7fe455cab700 1 -- 172.21.15.16:0/1888659255 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6554+0+0 (secure 0 0 0) 0x5594efacaa80 con 0x5594efda8400 2023-05-20T01:28:49.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.969+0000 7fe45dd097c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-05-20T01:28:49.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.969+0000 7fe45dd097c0 4 datascan.main: data pool 'cephfs_data' has ID 19 2023-05-20T01:28:49.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.969+0000 7fe45dd097c0 4 datascan.main: opening data pool 'cephfs_data' 2023-05-20T01:28:49.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.969+0000 7fe45bcb7700 1 --2- 172.21.15.16:0/1888659255 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x5594efdd8800 0x5594efec6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:49.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.969+0000 7fe45dd097c0 1 --2- 172.21.15.16:0/1888659255 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x5594efda8800 0x5594efb24680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:49.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.969+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:1 18.14 18:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091e700 con 0x5594efda8800 2023-05-20T01:28:49.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.969+0000 7fe45bcb7700 1 --2- 172.21.15.16:0/1888659255 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x5594efdd8800 0x5594efec6000 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x5594f08540f0 tx=0x5594f0923ec0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:49.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.969+0000 7fe45b4b6700 1 --2- 172.21.15.16:0/1888659255 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x5594efda8800 0x5594efb24680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:49.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.969+0000 7fe45b4b6700 1 --2- 172.21.15.16:0/1888659255 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x5594efda8800 0x5594efb24680 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:49.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.970+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 1 ==== osd_op_reply(1 1.00000000.inode [stat out=16b] v0'0 uv17 ondisk = 0) v8 ==== 160+0+16 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:49.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.970+0000 7fe45dd097c0 1 --2- 172.21.15.16:0/1888659255 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x5594efda8c00 0x5594efb24c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:49.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.970+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:2 18.1f 18:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091ea80 con 0x5594efda8c00 2023-05-20T01:28:49.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.970+0000 7fe45c4b8700 1 --2- 172.21.15.16:0/1888659255 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x5594efda8c00 0x5594efb24c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:49.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.970+0000 7fe45c4b8700 1 --2- 172.21.15.16:0/1888659255 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x5594efda8c00 0x5594efb24c00 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:49.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.970+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 1 ==== osd_op_reply(2 100.00000000.inode [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 162+0+16 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:49.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.971+0000 7fe45dd097c0 1 --2- 172.21.15.16:0/1888659255 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x5594efda9000 0x5594efb25180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:49.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.971+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:3 19.0 19:00000000::::head [pgnls-filter start_epoch 138 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091ee00 con 0x5594efda9000 2023-05-20T01:28:49.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.971+0000 7fe45bcb7700 1 --2- 172.21.15.16:0/1888659255 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x5594efda9000 0x5594efb25180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:50.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.971+0000 7fe45bcb7700 1 --2- 172.21.15.16:0/1888659255 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x5594efda9000 0x5594efb25180 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:50.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.971+0000 7fe45bcb7700 1 -- 172.21.15.16:0/1888659255 <== osd.3 v2:172.21.15.16:6800/3669781890 1 ==== osd_op_reply(3 [pgnls-filter start_epoch 138 out=81b] v138'11 uv11 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5594efda18c0 con 0x5594efda9000 2023-05-20T01:28:50.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.971+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:4 19.0 19:00000000::::head [pgnls-filter start_epoch 138 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091f180 con 0x5594efda9000 2023-05-20T01:28:50.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.972+0000 7fe45bcb7700 1 -- 172.21.15.16:0/1888659255 <== osd.3 v2:172.21.15.16:6800/3669781890 2 ==== osd_op_reply(4 [pgnls-filter start_epoch 138 out=81b] v138'11 uv11 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5594efda18c0 con 0x5594efda9000 2023-05-20T01:28:50.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.972+0000 7fe45dd097c0 20 datascan.forall_objects: OSD matched oid 10000000206.00000000 2023-05-20T01:28:50.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.972+0000 7fe45dd097c0 10 datascan.operator(): handling object 10000000206.0 2023-05-20T01:28:50.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.972+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:5 19.0 19:012cdbc7:::10000000206.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091f500 con 0x5594efda9000 2023-05-20T01:28:50.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.972+0000 7fe45bcb7700 1 -- 172.21.15.16:0/1888659255 <== osd.3 v2:172.21.15.16:6800/3669781890 3 ==== osd_op_reply(5 10000000206.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv11 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x5594efda18c0 con 0x5594efda9000 2023-05-20T01:28:50.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.972+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000206 2023-05-20T01:28:50.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.972+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/17 2023-05-20T01:28:50.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.972+0000 7fe45dd097c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=17 2023-05-20T01:28:50.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.972+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:6 18.10 18:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091fc00 con 0x5594efda8800 2023-05-20T01:28:50.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.972+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 2 ==== osd_op_reply(6 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.972+0000 7fe45dd097c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-05-20T01:28:50.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.972+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:7 18.1f 18:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091e700 con 0x5594efda8c00 2023-05-20T01:28:50.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.973+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 2 ==== osd_op_reply(7 1.00000000 [omap-get-vals-by-keys out=589b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+589 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.973+0000 7fe45dd097c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-05-20T01:28:50.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.973+0000 7fe45dd097c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-05-20T01:28:50.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.973+0000 7fe45dd097c0 20 datascan.get_frag_of: frag is 1* 2023-05-20T01:28:50.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.973+0000 7fe45dd097c0 1 --2- 172.21.15.16:0/1888659255 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x5594efda9800 0x5594efb25700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:50.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.973+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:8 18.1b 18:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091fc00 con 0x5594efda9800 2023-05-20T01:28:50.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.973+0000 7fe45b4b6700 1 --2- 172.21.15.16:0/1888659255 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x5594efda9800 0x5594efb25700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:50.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.973+0000 7fe45b4b6700 1 --2- 172.21.15.16:0/1888659255 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x5594efda9800 0x5594efb25700 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:50.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.973+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 1 ==== osd_op_reply(8 10000000000.01800000 [omap-get-header out=274b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.973+0000 7fe45dd097c0 10 datascan.find_or_create_dirfrag: read_version = 3 2023-05-20T01:28:50.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.973+0000 7fe45dd097c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-05-20T01:28:50.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.973+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:9 18.1b 18:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091ee00 con 0x5594efda9800 2023-05-20T01:28:50.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.974+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 2 ==== osd_op_reply(9 10000000000.01800000 [omap-get-vals-by-keys out=577b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+577 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.974+0000 7fe45dd097c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=17 frag=1*, r=0 2023-05-20T01:28:50.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.974+0000 7fe45dd097c0 20 datascan.inject_with_backtrace: Dentry 0x0x10000000000/17 already exists and points to me 2023-05-20T01:28:50.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.974+0000 7fe45dd097c0 1 --2- 172.21.15.16:0/1888659255 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x5594efda9c00 0x5594efec6580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:50.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.974+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:10 19.20 19:04000000::::head [pgnls-filter start_epoch 138 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091f180 con 0x5594efda9c00 2023-05-20T01:28:50.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.974+0000 7fe45c4b8700 1 --2- 172.21.15.16:0/1888659255 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x5594efda9c00 0x5594efec6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:50.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.974+0000 7fe45c4b8700 1 --2- 172.21.15.16:0/1888659255 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x5594efda9c00 0x5594efec6580 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:50.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.975+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.4 v2:172.21.15.132:6817/506317223 1 ==== osd_op_reply(10 [pgnls-filter start_epoch 138 out=81b] v138'12 uv12 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5594efda1680 con 0x5594efda9c00 2023-05-20T01:28:50.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.975+0000 7fe45dd097c0 20 datascan.forall_objects: OSD matched oid 1000000020b.00000000 2023-05-20T01:28:50.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.975+0000 7fe45dd097c0 10 datascan.operator(): handling object 1000000020b.0 2023-05-20T01:28:50.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.975+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:11 19.20 19:05552577:::1000000020b.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091ea80 con 0x5594efda9c00 2023-05-20T01:28:50.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.975+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.4 v2:172.21.15.132:6817/506317223 2 ==== osd_op_reply(11 1000000020b.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv12 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x5594efda1680 con 0x5594efda9c00 2023-05-20T01:28:50.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.975+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: inode: 0x0x1000000020b 2023-05-20T01:28:50.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.975+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/22 2023-05-20T01:28:50.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.975+0000 7fe45dd097c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=22 2023-05-20T01:28:50.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.975+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:12 18.10 18:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091f180 con 0x5594efda8800 2023-05-20T01:28:50.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.975+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 3 ==== osd_op_reply(12 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.975+0000 7fe45dd097c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-05-20T01:28:50.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.975+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:13 18.1f 18:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091fc00 con 0x5594efda8c00 2023-05-20T01:28:50.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.976+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 3 ==== osd_op_reply(13 1.00000000 [omap-get-vals-by-keys out=589b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+589 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.976+0000 7fe45dd097c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-05-20T01:28:50.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.976+0000 7fe45dd097c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-05-20T01:28:50.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.976+0000 7fe45dd097c0 20 datascan.get_frag_of: frag is 1* 2023-05-20T01:28:50.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.976+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:14 18.1b 18:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091ee00 con 0x5594efda9800 2023-05-20T01:28:50.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.976+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 3 ==== osd_op_reply(14 10000000000.01800000 [omap-get-header out=274b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.976+0000 7fe45dd097c0 10 datascan.find_or_create_dirfrag: read_version = 3 2023-05-20T01:28:50.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.976+0000 7fe45dd097c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-05-20T01:28:50.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.976+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:15 18.1b 18:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091f180 con 0x5594efda9800 2023-05-20T01:28:50.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.976+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 4 ==== osd_op_reply(15 10000000000.01800000 [omap-get-vals-by-keys out=577b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+577 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.976+0000 7fe45dd097c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=22 frag=1*, r=0 2023-05-20T01:28:50.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.976+0000 7fe45dd097c0 20 datascan.inject_with_backtrace: Dentry 0x0x10000000000/22 already exists and points to me 2023-05-20T01:28:50.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.976+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:16 19.10 19:08000000::::head [pgnls-filter start_epoch 138 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efeea000 con 0x5594efda9800 2023-05-20T01:28:50.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.976+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 5 ==== osd_op_reply(16 [pgnls-filter start_epoch 138 out=101b] v138'18 uv18 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.976+0000 7fe45dd097c0 20 datascan.forall_objects: OSD matched oid 1000000022a.00000000 2023-05-20T01:28:50.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.976+0000 7fe45dd097c0 10 datascan.operator(): handling object 1000000022a.0 2023-05-20T01:28:50.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.976+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:17 19.10 19:083d5826:::1000000022a.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efeea380 con 0x5594efda9800 2023-05-20T01:28:50.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.977+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 6 ==== osd_op_reply(17 1000000022a.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv15 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.977+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: inode: 0x0x1000000022a 2023-05-20T01:28:50.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.977+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/53 2023-05-20T01:28:50.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.977+0000 7fe45dd097c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=53 2023-05-20T01:28:50.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.977+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:18 18.10 18:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efeeaa80 con 0x5594efda8800 2023-05-20T01:28:50.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.977+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 4 ==== osd_op_reply(18 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.977+0000 7fe45dd097c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-05-20T01:28:50.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.977+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:19 18.1f 18:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efeeae00 con 0x5594efda8c00 2023-05-20T01:28:50.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.977+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 4 ==== osd_op_reply(19 1.00000000 [omap-get-vals-by-keys out=589b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+589 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.977+0000 7fe45dd097c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-05-20T01:28:50.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.977+0000 7fe45dd097c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-05-20T01:28:50.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.977+0000 7fe45dd097c0 20 datascan.get_frag_of: frag is 0* 2023-05-20T01:28:50.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.977+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:20 18.5 18:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efeeb180 con 0x5594efda9800 2023-05-20T01:28:50.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.978+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 7 ==== osd_op_reply(20 10000000000.01000000 [omap-get-header out=274b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.978+0000 7fe45dd097c0 10 datascan.find_or_create_dirfrag: read_version = 3 2023-05-20T01:28:50.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.978+0000 7fe45dd097c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-05-20T01:28:50.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.978+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:21 18.5 18:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efeeb500 con 0x5594efda9800 2023-05-20T01:28:50.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.978+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 8 ==== osd_op_reply(21 10000000000.01000000 [omap-get-vals-by-keys out=577b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+577 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.978+0000 7fe45dd097c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=53 frag=0*, r=0 2023-05-20T01:28:50.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.978+0000 7fe45dd097c0 20 datascan.inject_with_backtrace: Dentry 0x0x10000000000/53 already exists and points to me 2023-05-20T01:28:50.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.978+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:22 19.10 19:08500202::::head [pgnls-filter start_epoch 138 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efeeb880 con 0x5594efda9800 2023-05-20T01:28:50.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.978+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 9 ==== osd_op_reply(22 [pgnls-filter start_epoch 138 out=101b] v138'18 uv18 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.978+0000 7fe45dd097c0 20 datascan.forall_objects: OSD matched oid 1000000023e.00000000 2023-05-20T01:28:50.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.978+0000 7fe45dd097c0 10 datascan.operator(): handling object 1000000023e.0 2023-05-20T01:28:50.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.978+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:23 19.10 19:08500202:::1000000023e.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091e700 con 0x5594efda9800 2023-05-20T01:28:50.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.978+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 10 ==== osd_op_reply(23 1000000023e.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv16 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.979+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: inode: 0x0x1000000023e 2023-05-20T01:28:50.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.979+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/73 2023-05-20T01:28:50.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.979+0000 7fe45dd097c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=73 2023-05-20T01:28:50.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.979+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:24 18.10 18:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091fc00 con 0x5594efda8800 2023-05-20T01:28:50.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.979+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 5 ==== osd_op_reply(24 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.979+0000 7fe45dd097c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-05-20T01:28:50.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.979+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:25 18.1f 18:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091ee00 con 0x5594efda8c00 2023-05-20T01:28:50.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.979+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 5 ==== osd_op_reply(25 1.00000000 [omap-get-vals-by-keys out=589b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+589 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.979+0000 7fe45dd097c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-05-20T01:28:50.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.979+0000 7fe45dd097c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-05-20T01:28:50.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.979+0000 7fe45dd097c0 20 datascan.get_frag_of: frag is 0* 2023-05-20T01:28:50.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.979+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:26 18.5 18:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091f180 con 0x5594efda9800 2023-05-20T01:28:50.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.979+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 11 ==== osd_op_reply(26 10000000000.01000000 [omap-get-header out=274b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.979+0000 7fe45dd097c0 10 datascan.find_or_create_dirfrag: read_version = 3 2023-05-20T01:28:50.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.979+0000 7fe45dd097c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-05-20T01:28:50.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.979+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:27 18.5 18:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efeea000 con 0x5594efda9800 2023-05-20T01:28:50.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.980+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 12 ==== osd_op_reply(27 10000000000.01000000 [omap-get-vals-by-keys out=577b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+577 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.980+0000 7fe45dd097c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=73 frag=0*, r=0 2023-05-20T01:28:50.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.980+0000 7fe45dd097c0 20 datascan.inject_with_backtrace: Dentry 0x0x10000000000/73 already exists and points to me 2023-05-20T01:28:50.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.980+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:28 19.10 19:08b99fea::::head [pgnls-filter start_epoch 138 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efeea380 con 0x5594efda9800 2023-05-20T01:28:50.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.980+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 13 ==== osd_op_reply(28 [pgnls-filter start_epoch 138 out=101b] v138'18 uv18 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.980+0000 7fe45dd097c0 20 datascan.forall_objects: OSD matched oid 100000001fd.00000000 2023-05-20T01:28:50.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.980+0000 7fe45dd097c0 10 datascan.operator(): handling object 100000001fd.0 2023-05-20T01:28:50.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.980+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:29 19.10 19:08b99fea:::100000001fd.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efeea700 con 0x5594efda9800 2023-05-20T01:28:50.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.980+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 14 ==== osd_op_reply(29 100000001fd.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (89) out=89b,getxattr (30) out=30b,getxattr] v0'0 uv17 ondisk = 0) v8 ==== 416+0+157 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.980+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: inode: 0x0x100000001fd 2023-05-20T01:28:50.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.980+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/8 2023-05-20T01:28:50.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.980+0000 7fe45dd097c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=8 2023-05-20T01:28:50.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.980+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:30 18.10 18:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efeeae00 con 0x5594efda8800 2023-05-20T01:28:50.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.981+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 6 ==== osd_op_reply(30 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.981+0000 7fe45dd097c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-05-20T01:28:50.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.981+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:31 18.1f 18:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efeeb180 con 0x5594efda8c00 2023-05-20T01:28:50.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.981+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 6 ==== osd_op_reply(31 1.00000000 [omap-get-vals-by-keys out=589b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+589 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.981+0000 7fe45dd097c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-05-20T01:28:50.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.981+0000 7fe45dd097c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-05-20T01:28:50.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.981+0000 7fe45dd097c0 20 datascan.get_frag_of: frag is 0* 2023-05-20T01:28:50.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.981+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:32 18.5 18:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efeeb500 con 0x5594efda9800 2023-05-20T01:28:50.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.981+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 15 ==== osd_op_reply(32 10000000000.01000000 [omap-get-header out=274b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.981+0000 7fe45dd097c0 10 datascan.find_or_create_dirfrag: read_version = 3 2023-05-20T01:28:50.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.981+0000 7fe45dd097c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-05-20T01:28:50.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.981+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:33 18.5 18:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efeeb880 con 0x5594efda9800 2023-05-20T01:28:50.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.981+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 16 ==== osd_op_reply(33 10000000000.01000000 [omap-get-vals-by-keys out=576b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+576 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.981+0000 7fe45dd097c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=8 frag=0*, r=0 2023-05-20T01:28:50.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.981+0000 7fe45dd097c0 20 datascan.inject_with_backtrace: Dentry 0x0x10000000000/8 already exists and points to me 2023-05-20T01:28:50.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.981+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:34 19.10 19:08e6f8ca::::head [pgnls-filter start_epoch 138 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efeebc00 con 0x5594efda9800 2023-05-20T01:28:50.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.982+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 17 ==== osd_op_reply(34 [pgnls-filter start_epoch 138 out=81b] v138'18 uv18 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.982+0000 7fe45dd097c0 20 datascan.forall_objects: OSD matched oid 10000000257.00000000 2023-05-20T01:28:50.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.982+0000 7fe45dd097c0 10 datascan.operator(): handling object 10000000257.0 2023-05-20T01:28:50.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.982+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:35 19.10 19:08e6f8ca:::10000000257.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efeea000 con 0x5594efda9800 2023-05-20T01:28:50.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.982+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 18 ==== osd_op_reply(35 10000000257.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv18 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.982+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000257 2023-05-20T01:28:50.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.982+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/98 2023-05-20T01:28:50.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.982+0000 7fe45dd097c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=98 2023-05-20T01:28:50.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.982+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:36 18.10 18:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091ea80 con 0x5594efda8800 2023-05-20T01:28:50.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.982+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 7 ==== osd_op_reply(36 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.982+0000 7fe45dd097c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-05-20T01:28:50.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.982+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:37 18.1f 18:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091e700 con 0x5594efda8c00 2023-05-20T01:28:50.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.983+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 7 ==== osd_op_reply(37 1.00000000 [omap-get-vals-by-keys out=589b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+589 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.983+0000 7fe45dd097c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-05-20T01:28:50.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.983+0000 7fe45dd097c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-05-20T01:28:50.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.983+0000 7fe45dd097c0 20 datascan.get_frag_of: frag is 1* 2023-05-20T01:28:50.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.983+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:38 18.1b 18:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091fc00 con 0x5594efda9800 2023-05-20T01:28:50.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.983+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 19 ==== osd_op_reply(38 10000000000.01800000 [omap-get-header out=274b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.983+0000 7fe45dd097c0 10 datascan.find_or_create_dirfrag: read_version = 3 2023-05-20T01:28:50.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.983+0000 7fe45dd097c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-05-20T01:28:50.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.983+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:39 18.1b 18:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091f180 con 0x5594efda9800 2023-05-20T01:28:50.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.983+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 20 ==== osd_op_reply(39 10000000000.01800000 [omap-get-vals-by-keys out=577b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+577 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.983+0000 7fe45dd097c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=98 frag=1*, r=0 2023-05-20T01:28:50.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.983+0000 7fe45dd097c0 20 datascan.inject_with_backtrace: Dentry 0x0x10000000000/98 already exists and points to me 2023-05-20T01:28:50.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.983+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:40 19.30 19:0c000000::::head [pgnls-filter start_epoch 138 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efeeaa80 con 0x5594efda9000 2023-05-20T01:28:50.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.983+0000 7fe45bcb7700 1 -- 172.21.15.16:0/1888659255 <== osd.3 v2:172.21.15.16:6800/3669781890 4 ==== osd_op_reply(40 [pgnls-filter start_epoch 138 out=101b] v138'21 uv21 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x5594efda18c0 con 0x5594efda9000 2023-05-20T01:28:50.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.983+0000 7fe45dd097c0 20 datascan.forall_objects: OSD matched oid 1000000021d.00000000 2023-05-20T01:28:50.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.983+0000 7fe45dd097c0 10 datascan.operator(): handling object 1000000021d.0 2023-05-20T01:28:50.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.983+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:41 19.30 19:0c00cb85:::1000000021d.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efeea700 con 0x5594efda9000 2023-05-20T01:28:50.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.984+0000 7fe45bcb7700 1 -- 172.21.15.16:0/1888659255 <== osd.3 v2:172.21.15.16:6800/3669781890 5 ==== osd_op_reply(41 1000000021d.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv18 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x5594efda18c0 con 0x5594efda9000 2023-05-20T01:28:50.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.984+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: inode: 0x0x1000000021d 2023-05-20T01:28:50.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.984+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/40 2023-05-20T01:28:50.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.984+0000 7fe45dd097c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=40 2023-05-20T01:28:50.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.984+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:42 18.10 18:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efeeb500 con 0x5594efda8800 2023-05-20T01:28:50.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.984+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 8 ==== osd_op_reply(42 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.984+0000 7fe45dd097c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-05-20T01:28:50.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.984+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:43 18.1f 18:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efeeb880 con 0x5594efda8c00 2023-05-20T01:28:50.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.985+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 8 ==== osd_op_reply(43 1.00000000 [omap-get-vals-by-keys out=589b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+589 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.985+0000 7fe45dd097c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-05-20T01:28:50.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.985+0000 7fe45dd097c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-05-20T01:28:50.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.985+0000 7fe45dd097c0 20 datascan.get_frag_of: frag is 1* 2023-05-20T01:28:50.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.985+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:44 18.1b 18:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efeebc00 con 0x5594efda9800 2023-05-20T01:28:50.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.985+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 21 ==== osd_op_reply(44 10000000000.01800000 [omap-get-header out=274b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.985+0000 7fe45dd097c0 10 datascan.find_or_create_dirfrag: read_version = 3 2023-05-20T01:28:50.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.985+0000 7fe45dd097c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-05-20T01:28:50.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.985+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:45 18.1b 18:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efeea380 con 0x5594efda9800 2023-05-20T01:28:50.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.985+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 22 ==== osd_op_reply(45 10000000000.01800000 [omap-get-vals-by-keys out=577b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+577 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.985+0000 7fe45dd097c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=40 frag=1*, r=0 2023-05-20T01:28:50.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.985+0000 7fe45dd097c0 20 datascan.inject_with_backtrace: Dentry 0x0x10000000000/40 already exists and points to me 2023-05-20T01:28:50.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.985+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:46 19.30 19:0c28cce1::::head [pgnls-filter start_epoch 138 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efeeaa80 con 0x5594efda9000 2023-05-20T01:28:50.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.985+0000 7fe45bcb7700 1 -- 172.21.15.16:0/1888659255 <== osd.3 v2:172.21.15.16:6800/3669781890 6 ==== osd_op_reply(46 [pgnls-filter start_epoch 138 out=101b] v138'21 uv21 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x5594efda18c0 con 0x5594efda9000 2023-05-20T01:28:50.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.985+0000 7fe45dd097c0 20 datascan.forall_objects: OSD matched oid 10000000244.00000000 2023-05-20T01:28:50.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.985+0000 7fe45dd097c0 10 datascan.operator(): handling object 10000000244.0 2023-05-20T01:28:50.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.985+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:47 19.30 19:0c28cce1:::10000000244.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091f880 con 0x5594efda9000 2023-05-20T01:28:50.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.986+0000 7fe45bcb7700 1 -- 172.21.15.16:0/1888659255 <== osd.3 v2:172.21.15.16:6800/3669781890 7 ==== osd_op_reply(47 10000000244.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv19 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x5594efda18c0 con 0x5594efda9000 2023-05-20T01:28:50.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.986+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000244 2023-05-20T01:28:50.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.986+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/79 2023-05-20T01:28:50.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.986+0000 7fe45dd097c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=79 2023-05-20T01:28:50.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.986+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:48 18.10 18:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091f500 con 0x5594efda8800 2023-05-20T01:28:50.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.986+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 9 ==== osd_op_reply(48 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.986+0000 7fe45dd097c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-05-20T01:28:50.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.986+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:49 18.1f 18:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efeeae00 con 0x5594efda8c00 2023-05-20T01:28:50.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.986+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 9 ==== osd_op_reply(49 1.00000000 [omap-get-vals-by-keys out=589b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+589 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.986+0000 7fe45dd097c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-05-20T01:28:50.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.986+0000 7fe45dd097c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-05-20T01:28:50.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.986+0000 7fe45dd097c0 20 datascan.get_frag_of: frag is 1* 2023-05-20T01:28:50.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.986+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:50 18.1b 18:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efeea700 con 0x5594efda9800 2023-05-20T01:28:50.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.987+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 23 ==== osd_op_reply(50 10000000000.01800000 [omap-get-header out=274b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.987+0000 7fe45dd097c0 10 datascan.find_or_create_dirfrag: read_version = 3 2023-05-20T01:28:50.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.987+0000 7fe45dd097c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-05-20T01:28:50.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.987+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:51 18.1b 18:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efeeaa80 con 0x5594efda9800 2023-05-20T01:28:50.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.987+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 24 ==== osd_op_reply(51 10000000000.01800000 [omap-get-vals-by-keys out=577b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+577 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.987+0000 7fe45dd097c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=79 frag=1*, r=0 2023-05-20T01:28:50.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.987+0000 7fe45dd097c0 20 datascan.inject_with_backtrace: Dentry 0x0x10000000000/79 already exists and points to me 2023-05-20T01:28:50.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.987+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:52 19.30 19:0cc9e906::::head [pgnls-filter start_epoch 138 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091ee00 con 0x5594efda9000 2023-05-20T01:28:50.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.987+0000 7fe45bcb7700 1 -- 172.21.15.16:0/1888659255 <== osd.3 v2:172.21.15.16:6800/3669781890 8 ==== osd_op_reply(52 [pgnls-filter start_epoch 138 out=101b] v138'21 uv21 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x5594efda18c0 con 0x5594efda9000 2023-05-20T01:28:50.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.987+0000 7fe45dd097c0 20 datascan.forall_objects: OSD matched oid 10000000227.00000000 2023-05-20T01:28:50.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.987+0000 7fe45dd097c0 10 datascan.operator(): handling object 10000000227.0 2023-05-20T01:28:50.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.987+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:53 19.30 19:0cc9e906:::10000000227.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091e700 con 0x5594efda9000 2023-05-20T01:28:50.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.987+0000 7fe45bcb7700 1 -- 172.21.15.16:0/1888659255 <== osd.3 v2:172.21.15.16:6800/3669781890 9 ==== osd_op_reply(53 10000000227.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv20 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x5594efda18c0 con 0x5594efda9000 2023-05-20T01:28:50.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.987+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000227 2023-05-20T01:28:50.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.987+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/50 2023-05-20T01:28:50.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.987+0000 7fe45dd097c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=50 2023-05-20T01:28:50.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.987+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:54 18.10 18:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091fc00 con 0x5594efda8800 2023-05-20T01:28:50.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.988+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 10 ==== osd_op_reply(54 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.988+0000 7fe45dd097c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-05-20T01:28:50.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.988+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:55 18.1f 18:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091f180 con 0x5594efda8c00 2023-05-20T01:28:50.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.988+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 10 ==== osd_op_reply(55 1.00000000 [omap-get-vals-by-keys out=589b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+589 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.988+0000 7fe45dd097c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-05-20T01:28:50.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.988+0000 7fe45dd097c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-05-20T01:28:50.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.988+0000 7fe45dd097c0 20 datascan.get_frag_of: frag is 0* 2023-05-20T01:28:50.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.988+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:56 18.5 18:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091f500 con 0x5594efda9800 2023-05-20T01:28:50.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.988+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 25 ==== osd_op_reply(56 10000000000.01000000 [omap-get-header out=274b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.988+0000 7fe45dd097c0 10 datascan.find_or_create_dirfrag: read_version = 3 2023-05-20T01:28:50.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.988+0000 7fe45dd097c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-05-20T01:28:50.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.988+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:57 18.5 18:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efeeb180 con 0x5594efda9800 2023-05-20T01:28:50.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.989+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 26 ==== osd_op_reply(57 10000000000.01000000 [omap-get-vals-by-keys out=577b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+577 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.989+0000 7fe45dd097c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=50 frag=0*, r=0 2023-05-20T01:28:50.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.989+0000 7fe45dd097c0 20 datascan.inject_with_backtrace: Dentry 0x0x10000000000/50 already exists and points to me 2023-05-20T01:28:50.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.989+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:58 19.30 19:0e3d5a27::::head [pgnls-filter start_epoch 138 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efeeb880 con 0x5594efda9000 2023-05-20T01:28:50.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.989+0000 7fe45bcb7700 1 -- 172.21.15.16:0/1888659255 <== osd.3 v2:172.21.15.16:6800/3669781890 10 ==== osd_op_reply(58 [pgnls-filter start_epoch 138 out=81b] v138'21 uv21 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5594efda18c0 con 0x5594efda9000 2023-05-20T01:28:50.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.989+0000 7fe45dd097c0 20 datascan.forall_objects: OSD matched oid 10000000250.00000000 2023-05-20T01:28:50.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.989+0000 7fe45dd097c0 10 datascan.operator(): handling object 10000000250.0 2023-05-20T01:28:50.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.989+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:59 19.30 19:0e3d5a27:::10000000250.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efeeae00 con 0x5594efda9000 2023-05-20T01:28:50.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.989+0000 7fe45bcb7700 1 -- 172.21.15.16:0/1888659255 <== osd.3 v2:172.21.15.16:6800/3669781890 11 ==== osd_op_reply(59 10000000250.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv21 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x5594efda18c0 con 0x5594efda9000 2023-05-20T01:28:50.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.989+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000250 2023-05-20T01:28:50.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.989+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/91 2023-05-20T01:28:50.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.989+0000 7fe45dd097c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=91 2023-05-20T01:28:50.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.989+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:60 18.10 18:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efeeb500 con 0x5594efda8800 2023-05-20T01:28:50.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.989+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 11 ==== osd_op_reply(60 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.989+0000 7fe45dd097c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-05-20T01:28:50.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.989+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:61 18.1f 18:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efeebc00 con 0x5594efda8c00 2023-05-20T01:28:50.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.990+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 11 ==== osd_op_reply(61 1.00000000 [omap-get-vals-by-keys out=589b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+589 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.990+0000 7fe45dd097c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-05-20T01:28:50.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.990+0000 7fe45dd097c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-05-20T01:28:50.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.990+0000 7fe45dd097c0 20 datascan.get_frag_of: frag is 1* 2023-05-20T01:28:50.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.990+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:62 18.1b 18:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efeea380 con 0x5594efda9800 2023-05-20T01:28:50.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.990+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 27 ==== osd_op_reply(62 10000000000.01800000 [omap-get-header out=274b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.990+0000 7fe45dd097c0 10 datascan.find_or_create_dirfrag: read_version = 3 2023-05-20T01:28:50.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.990+0000 7fe45dd097c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-05-20T01:28:50.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.990+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:63 18.1b 18:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efeeb880 con 0x5594efda9800 2023-05-20T01:28:50.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.990+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 28 ==== osd_op_reply(63 10000000000.01800000 [omap-get-vals-by-keys out=577b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+577 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.990+0000 7fe45dd097c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=91 frag=1*, r=0 2023-05-20T01:28:50.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.990+0000 7fe45dd097c0 20 datascan.inject_with_backtrace: Dentry 0x0x10000000000/91 already exists and points to me 2023-05-20T01:28:50.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.990+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:64 19.8 19:10000000::::head [pgnls-filter start_epoch 138 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091f880 con 0x5594efda9800 2023-05-20T01:28:50.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.990+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 29 ==== osd_op_reply(64 [pgnls-filter start_epoch 138 out=49b] v136'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.991+0000 7fe4574ae700 1 --2- 172.21.15.16:0/1888659255 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x5594eff00000 0x5594efec8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:50.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.991+0000 7fe4574ae700 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:65 19.28 19:14000000::::head [pgnls-filter start_epoch 138 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efeea700 con 0x5594eff00000 2023-05-20T01:28:50.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.991+0000 7fe45bcb7700 1 --2- 172.21.15.16:0/1888659255 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x5594eff00000 0x5594efec8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:50.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.991+0000 7fe45bcb7700 1 --2- 172.21.15.16:0/1888659255 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x5594eff00000 0x5594efec8c00 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:50.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.991+0000 7fe45bcb7700 1 -- 172.21.15.16:0/1888659255 <== osd.0 v2:172.21.15.16:6824/3923634179 1 ==== osd_op_reply(65 [pgnls-filter start_epoch 138 out=101b] v138'18 uv18 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x5594efda18c0 con 0x5594eff00000 2023-05-20T01:28:50.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.991+0000 7fe45dd097c0 20 datascan.forall_objects: OSD matched oid 10000000233.00000000 2023-05-20T01:28:50.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.991+0000 7fe45dd097c0 10 datascan.operator(): handling object 10000000233.0 2023-05-20T01:28:50.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.991+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:66 19.28 19:14ce3b86:::10000000233.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091ee00 con 0x5594eff00000 2023-05-20T01:28:50.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.992+0000 7fe45bcb7700 1 -- 172.21.15.16:0/1888659255 <== osd.0 v2:172.21.15.16:6824/3923634179 2 ==== osd_op_reply(66 10000000233.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv15 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x5594efda18c0 con 0x5594eff00000 2023-05-20T01:28:50.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.992+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000233 2023-05-20T01:28:50.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.992+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/62 2023-05-20T01:28:50.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.992+0000 7fe45dd097c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=62 2023-05-20T01:28:50.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.992+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:67 18.10 18:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091e700 con 0x5594efda8800 2023-05-20T01:28:50.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.992+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 12 ==== osd_op_reply(67 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.992+0000 7fe45dd097c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-05-20T01:28:50.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.992+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:68 18.1f 18:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091fc00 con 0x5594efda8c00 2023-05-20T01:28:50.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.992+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 12 ==== osd_op_reply(68 1.00000000 [omap-get-vals-by-keys out=589b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+589 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.992+0000 7fe45dd097c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-05-20T01:28:50.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.992+0000 7fe45dd097c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-05-20T01:28:50.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.992+0000 7fe45dd097c0 20 datascan.get_frag_of: frag is 1* 2023-05-20T01:28:50.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.992+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:69 18.1b 18:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091f500 con 0x5594efda9800 2023-05-20T01:28:50.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.992+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 30 ==== osd_op_reply(69 10000000000.01800000 [omap-get-header out=274b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.992+0000 7fe45dd097c0 10 datascan.find_or_create_dirfrag: read_version = 3 2023-05-20T01:28:50.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.992+0000 7fe45dd097c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-05-20T01:28:50.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.992+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:70 18.1b 18:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091f880 con 0x5594efda9800 2023-05-20T01:28:50.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.993+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 31 ==== osd_op_reply(70 10000000000.01800000 [omap-get-vals-by-keys out=577b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+577 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.993+0000 7fe45dd097c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=62 frag=1*, r=0 2023-05-20T01:28:50.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.993+0000 7fe45dd097c0 20 datascan.inject_with_backtrace: Dentry 0x0x10000000000/62 already exists and points to me 2023-05-20T01:28:50.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.993+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:71 19.28 19:15246602::::head [pgnls-filter start_epoch 138 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efeeaa80 con 0x5594eff00000 2023-05-20T01:28:50.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.993+0000 7fe45bcb7700 1 -- 172.21.15.16:0/1888659255 <== osd.0 v2:172.21.15.16:6824/3923634179 3 ==== osd_op_reply(71 [pgnls-filter start_epoch 138 out=101b] v138'18 uv18 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x5594efda18c0 con 0x5594eff00000 2023-05-20T01:28:50.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.993+0000 7fe45dd097c0 20 datascan.forall_objects: OSD matched oid 10000000202.00000000 2023-05-20T01:28:50.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.993+0000 7fe45dd097c0 10 datascan.operator(): handling object 10000000202.0 2023-05-20T01:28:50.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.993+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:72 19.28 19:15246602:::10000000202.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efeeb180 con 0x5594eff00000 2023-05-20T01:28:50.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.993+0000 7fe45bcb7700 1 -- 172.21.15.16:0/1888659255 <== osd.0 v2:172.21.15.16:6824/3923634179 4 ==== osd_op_reply(72 10000000202.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv16 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x5594efda18c0 con 0x5594eff00000 2023-05-20T01:28:50.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.993+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000202 2023-05-20T01:28:50.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.993+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/13 2023-05-20T01:28:50.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.993+0000 7fe45dd097c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=13 2023-05-20T01:28:50.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.993+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:73 18.10 18:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efeea380 con 0x5594efda8800 2023-05-20T01:28:50.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.994+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 13 ==== osd_op_reply(73 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.994+0000 7fe45dd097c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-05-20T01:28:50.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.994+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:74 18.1f 18:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efeeb880 con 0x5594efda8c00 2023-05-20T01:28:50.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.994+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 13 ==== osd_op_reply(74 1.00000000 [omap-get-vals-by-keys out=589b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+589 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.994+0000 7fe45dd097c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-05-20T01:28:50.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.994+0000 7fe45dd097c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-05-20T01:28:50.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.994+0000 7fe45dd097c0 20 datascan.get_frag_of: frag is 1* 2023-05-20T01:28:50.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.994+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:75 18.1b 18:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efeea000 con 0x5594efda9800 2023-05-20T01:28:50.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.994+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 32 ==== osd_op_reply(75 10000000000.01800000 [omap-get-header out=274b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.994+0000 7fe45dd097c0 10 datascan.find_or_create_dirfrag: read_version = 3 2023-05-20T01:28:50.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.994+0000 7fe45dd097c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-05-20T01:28:50.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.994+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:76 18.1b 18:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efeeae00 con 0x5594efda9800 2023-05-20T01:28:50.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.994+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 33 ==== osd_op_reply(76 10000000000.01800000 [omap-get-vals-by-keys out=577b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+577 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.994+0000 7fe45dd097c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=13 frag=1*, r=0 2023-05-20T01:28:50.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.994+0000 7fe45dd097c0 20 datascan.inject_with_backtrace: Dentry 0x0x10000000000/13 already exists and points to me 2023-05-20T01:28:50.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.994+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:77 19.28 19:1537fa38::::head [pgnls-filter start_epoch 138 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efeea700 con 0x5594eff00000 2023-05-20T01:28:50.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.995+0000 7fe45bcb7700 1 -- 172.21.15.16:0/1888659255 <== osd.0 v2:172.21.15.16:6824/3923634179 5 ==== osd_op_reply(77 [pgnls-filter start_epoch 138 out=101b] v138'18 uv18 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x5594efda18c0 con 0x5594eff00000 2023-05-20T01:28:50.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.995+0000 7fe45dd097c0 20 datascan.forall_objects: OSD matched oid 10000000236.00000000 2023-05-20T01:28:50.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.995+0000 7fe45dd097c0 10 datascan.operator(): handling object 10000000236.0 2023-05-20T01:28:50.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.995+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:78 19.28 19:1537fa38:::10000000236.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efeeaa80 con 0x5594eff00000 2023-05-20T01:28:50.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.995+0000 7fe45bcb7700 1 -- 172.21.15.16:0/1888659255 <== osd.0 v2:172.21.15.16:6824/3923634179 6 ==== osd_op_reply(78 10000000236.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv17 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x5594efda18c0 con 0x5594eff00000 2023-05-20T01:28:50.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.995+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000236 2023-05-20T01:28:50.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.995+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/65 2023-05-20T01:28:50.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.995+0000 7fe45dd097c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=65 2023-05-20T01:28:50.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.995+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:79 18.10 18:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091ee00 con 0x5594efda8800 2023-05-20T01:28:50.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.995+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 14 ==== osd_op_reply(79 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.995+0000 7fe45dd097c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-05-20T01:28:50.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.995+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:80 18.1f 18:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef4000 con 0x5594efda8c00 2023-05-20T01:28:50.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.995+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 14 ==== osd_op_reply(80 1.00000000 [omap-get-vals-by-keys out=589b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+589 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.996+0000 7fe45dd097c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-05-20T01:28:50.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.996+0000 7fe45dd097c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-05-20T01:28:50.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.996+0000 7fe45dd097c0 20 datascan.get_frag_of: frag is 0* 2023-05-20T01:28:50.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.996+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:81 18.5 18:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef4380 con 0x5594efda9800 2023-05-20T01:28:50.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.996+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 34 ==== osd_op_reply(81 10000000000.01000000 [omap-get-header out=274b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.996+0000 7fe45dd097c0 10 datascan.find_or_create_dirfrag: read_version = 3 2023-05-20T01:28:50.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.996+0000 7fe45dd097c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-05-20T01:28:50.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.996+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:82 18.5 18:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef4700 con 0x5594efda9800 2023-05-20T01:28:50.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.996+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 35 ==== osd_op_reply(82 10000000000.01000000 [omap-get-vals-by-keys out=577b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+577 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.996+0000 7fe45dd097c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=65 frag=0*, r=0 2023-05-20T01:28:50.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.996+0000 7fe45dd097c0 20 datascan.inject_with_backtrace: Dentry 0x0x10000000000/65 already exists and points to me 2023-05-20T01:28:50.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.996+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:83 19.28 19:177a762d::::head [pgnls-filter start_epoch 138 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef4a80 con 0x5594eff00000 2023-05-20T01:28:50.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.996+0000 7fe45bcb7700 1 -- 172.21.15.16:0/1888659255 <== osd.0 v2:172.21.15.16:6824/3923634179 7 ==== osd_op_reply(83 [pgnls-filter start_epoch 138 out=81b] v138'18 uv18 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5594efda18c0 con 0x5594eff00000 2023-05-20T01:28:50.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.996+0000 7fe45dd097c0 20 datascan.forall_objects: OSD matched oid 10000000201.00000000 2023-05-20T01:28:50.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.996+0000 7fe45dd097c0 10 datascan.operator(): handling object 10000000201.0 2023-05-20T01:28:50.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.996+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:84 19.28 19:177a762d:::10000000201.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef4e00 con 0x5594eff00000 2023-05-20T01:28:50.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.997+0000 7fe45bcb7700 1 -- 172.21.15.16:0/1888659255 <== osd.0 v2:172.21.15.16:6824/3923634179 8 ==== osd_op_reply(84 10000000201.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv18 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x5594efda18c0 con 0x5594eff00000 2023-05-20T01:28:50.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.997+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000201 2023-05-20T01:28:50.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.997+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/12 2023-05-20T01:28:50.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.997+0000 7fe45dd097c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=12 2023-05-20T01:28:50.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.997+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:85 18.10 18:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef5500 con 0x5594efda8800 2023-05-20T01:28:50.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.997+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 15 ==== osd_op_reply(85 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.997+0000 7fe45dd097c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-05-20T01:28:50.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.997+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:86 18.1f 18:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef5880 con 0x5594efda8c00 2023-05-20T01:28:50.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.997+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 15 ==== osd_op_reply(86 1.00000000 [omap-get-vals-by-keys out=589b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+589 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.997+0000 7fe45dd097c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-05-20T01:28:50.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.997+0000 7fe45dd097c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-05-20T01:28:50.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.997+0000 7fe45dd097c0 20 datascan.get_frag_of: frag is 0* 2023-05-20T01:28:50.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.997+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:87 18.5 18:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef5c00 con 0x5594efda9800 2023-05-20T01:28:50.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.998+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 36 ==== osd_op_reply(87 10000000000.01000000 [omap-get-header out=274b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.998+0000 7fe45dd097c0 10 datascan.find_or_create_dirfrag: read_version = 3 2023-05-20T01:28:50.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.998+0000 7fe45dd097c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-05-20T01:28:50.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.998+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:88 18.5 18:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef4000 con 0x5594efda9800 2023-05-20T01:28:50.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.998+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 37 ==== osd_op_reply(88 10000000000.01000000 [omap-get-vals-by-keys out=577b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+577 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.998+0000 7fe45dd097c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=12 frag=0*, r=0 2023-05-20T01:28:50.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.998+0000 7fe45dd097c0 20 datascan.inject_with_backtrace: Dentry 0x0x10000000000/12 already exists and points to me 2023-05-20T01:28:50.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.998+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:89 19.18 19:18000000::::head [pgnls-filter start_epoch 138 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef5880 con 0x5594efda8800 2023-05-20T01:28:50.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.998+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 16 ==== osd_op_reply(89 [pgnls-filter start_epoch 138 out=49b] v136'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.998+0000 7fe456cad700 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:90 19.38 19:1c000000::::head [pgnls-filter start_epoch 138 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef6000 con 0x5594efda8800 2023-05-20T01:28:50.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.999+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 17 ==== osd_op_reply(90 [pgnls-filter start_epoch 138 out=49b] v136'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.999+0000 7fe4574ae700 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:91 19.4 19:20000000::::head [pgnls-filter start_epoch 138 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef6380 con 0x5594efda9800 2023-05-20T01:28:50.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.999+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 38 ==== osd_op_reply(91 [pgnls-filter start_epoch 138 out=81b] v138'12 uv12 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.999+0000 7fe45dd097c0 20 datascan.forall_objects: OSD matched oid 10000000220.00000000 2023-05-20T01:28:50.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.999+0000 7fe45dd097c0 10 datascan.operator(): handling object 10000000220.0 2023-05-20T01:28:50.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.999+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:92 19.4 19:2231e764:::10000000220.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef4380 con 0x5594efda9800 2023-05-20T01:28:50.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.999+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 39 ==== osd_op_reply(92 10000000220.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv12 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.999+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000220 2023-05-20T01:28:50.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.999+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/43 2023-05-20T01:28:50.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.999+0000 7fe45dd097c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=43 2023-05-20T01:28:50.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:49.999+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:93 18.10 18:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef5500 con 0x5594efda8800 2023-05-20T01:28:50.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.000+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 18 ==== osd_op_reply(93 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.000+0000 7fe45dd097c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-05-20T01:28:50.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.000+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:94 18.1f 18:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef4a80 con 0x5594efda8c00 2023-05-20T01:28:50.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.000+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 16 ==== osd_op_reply(94 1.00000000 [omap-get-vals-by-keys out=589b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+589 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.000+0000 7fe45dd097c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-05-20T01:28:50.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.000+0000 7fe45dd097c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-05-20T01:28:50.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.000+0000 7fe45dd097c0 20 datascan.get_frag_of: frag is 0* 2023-05-20T01:28:50.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.000+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:95 18.5 18:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef5180 con 0x5594efda9800 2023-05-20T01:28:50.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.000+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 40 ==== osd_op_reply(95 10000000000.01000000 [omap-get-header out=274b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.000+0000 7fe45dd097c0 10 datascan.find_or_create_dirfrag: read_version = 3 2023-05-20T01:28:50.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.000+0000 7fe45dd097c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-05-20T01:28:50.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.000+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:96 18.5 18:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091f180 con 0x5594efda9800 2023-05-20T01:28:50.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.000+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 41 ==== osd_op_reply(96 10000000000.01000000 [omap-get-vals-by-keys out=577b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+577 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.000+0000 7fe45dd097c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=43 frag=0*, r=0 2023-05-20T01:28:50.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.000+0000 7fe45dd097c0 20 datascan.inject_with_backtrace: Dentry 0x0x10000000000/43 already exists and points to me 2023-05-20T01:28:50.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.000+0000 7fe45dd097c0 1 --2- 172.21.15.16:0/1888659255 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x5594eff01000 0x5594efec6b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:50.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.000+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:97 19.24 19:24000000::::head [pgnls-filter start_epoch 138 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091fc00 con 0x5594eff01000 2023-05-20T01:28:50.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.001+0000 7fe45b4b6700 1 --2- 172.21.15.16:0/1888659255 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x5594eff01000 0x5594efec6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:50.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.001+0000 7fe45b4b6700 1 --2- 172.21.15.16:0/1888659255 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x5594eff01000 0x5594efec6b00 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:50.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.001+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.5 v2:172.21.15.132:6816/1844689995 1 ==== osd_op_reply(97 [pgnls-filter start_epoch 138 out=101b] v138'10 uv10 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x5594efda1440 con 0x5594eff01000 2023-05-20T01:28:50.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.001+0000 7fe45dd097c0 20 datascan.forall_objects: OSD matched oid 10000000234.00000000 2023-05-20T01:28:50.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.001+0000 7fe45dd097c0 10 datascan.operator(): handling object 10000000234.0 2023-05-20T01:28:50.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.001+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:98 19.24 19:242ba651:::10000000234.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091e700 con 0x5594eff01000 2023-05-20T01:28:50.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.002+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.5 v2:172.21.15.132:6816/1844689995 2 ==== osd_op_reply(98 10000000234.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv9 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x5594efda1440 con 0x5594eff01000 2023-05-20T01:28:50.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.002+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000234 2023-05-20T01:28:50.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.002+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/63 2023-05-20T01:28:50.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.002+0000 7fe45dd097c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=63 2023-05-20T01:28:50.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.002+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:99 18.10 18:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091f880 con 0x5594efda8800 2023-05-20T01:28:50.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.002+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 19 ==== osd_op_reply(99 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.002+0000 7fe45dd097c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-05-20T01:28:50.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.002+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:100 18.1f 18:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091ee00 con 0x5594efda8c00 2023-05-20T01:28:50.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.002+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 17 ==== osd_op_reply(100 1.00000000 [omap-get-vals-by-keys out=589b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+589 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.002+0000 7fe45dd097c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-05-20T01:28:50.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.002+0000 7fe45dd097c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-05-20T01:28:50.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.002+0000 7fe45dd097c0 20 datascan.get_frag_of: frag is 0* 2023-05-20T01:28:50.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.002+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:101 18.5 18:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091ea80 con 0x5594efda9800 2023-05-20T01:28:50.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.003+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 42 ==== osd_op_reply(101 10000000000.01000000 [omap-get-header out=274b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.003+0000 7fe45dd097c0 10 datascan.find_or_create_dirfrag: read_version = 3 2023-05-20T01:28:50.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.003+0000 7fe45dd097c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-05-20T01:28:50.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.003+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:102 18.5 18:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef6000 con 0x5594efda9800 2023-05-20T01:28:50.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.003+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 43 ==== osd_op_reply(102 10000000000.01000000 [omap-get-vals-by-keys out=577b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+577 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.003+0000 7fe45dd097c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=63 frag=0*, r=0 2023-05-20T01:28:50.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.003+0000 7fe45dd097c0 20 datascan.inject_with_backtrace: Dentry 0x0x10000000000/63 already exists and points to me 2023-05-20T01:28:50.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.003+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:103 19.24 19:25b5fce8::::head [pgnls-filter start_epoch 138 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef6380 con 0x5594eff01000 2023-05-20T01:28:50.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.003+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.5 v2:172.21.15.132:6816/1844689995 3 ==== osd_op_reply(103 [pgnls-filter start_epoch 138 out=81b] v138'10 uv10 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5594efda1440 con 0x5594eff01000 2023-05-20T01:28:50.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.003+0000 7fe45dd097c0 20 datascan.forall_objects: OSD matched oid 10000000231.00000000 2023-05-20T01:28:50.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.003+0000 7fe45dd097c0 10 datascan.operator(): handling object 10000000231.0 2023-05-20T01:28:50.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.004+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:104 19.24 19:25b5fce8:::10000000231.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef6a80 con 0x5594eff01000 2023-05-20T01:28:50.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.004+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.5 v2:172.21.15.132:6816/1844689995 4 ==== osd_op_reply(104 10000000231.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv10 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x5594efda1440 con 0x5594eff01000 2023-05-20T01:28:50.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.004+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000231 2023-05-20T01:28:50.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.004+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/60 2023-05-20T01:28:50.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.004+0000 7fe45dd097c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=60 2023-05-20T01:28:50.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.004+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:105 18.10 18:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef7180 con 0x5594efda8800 2023-05-20T01:28:50.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.004+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 20 ==== osd_op_reply(105 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.004+0000 7fe45dd097c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-05-20T01:28:50.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.004+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:106 18.1f 18:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef7500 con 0x5594efda8c00 2023-05-20T01:28:50.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.004+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 18 ==== osd_op_reply(106 1.00000000 [omap-get-vals-by-keys out=589b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+589 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.004+0000 7fe45dd097c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-05-20T01:28:50.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.004+0000 7fe45dd097c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-05-20T01:28:50.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.004+0000 7fe45dd097c0 20 datascan.get_frag_of: frag is 1* 2023-05-20T01:28:50.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.004+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:107 18.1b 18:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef7880 con 0x5594efda9800 2023-05-20T01:28:50.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.005+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 44 ==== osd_op_reply(107 10000000000.01800000 [omap-get-header out=274b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.005+0000 7fe45dd097c0 10 datascan.find_or_create_dirfrag: read_version = 3 2023-05-20T01:28:50.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.005+0000 7fe45dd097c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-05-20T01:28:50.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.005+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:108 18.1b 18:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef7c00 con 0x5594efda9800 2023-05-20T01:28:50.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.005+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 45 ==== osd_op_reply(108 10000000000.01800000 [omap-get-vals-by-keys out=577b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+577 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.005+0000 7fe45dd097c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=60 frag=1*, r=0 2023-05-20T01:28:50.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.005+0000 7fe45dd097c0 20 datascan.inject_with_backtrace: Dentry 0x0x10000000000/60 already exists and points to me 2023-05-20T01:28:50.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.005+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:109 19.14 19:28000000::::head [pgnls-filter start_epoch 138 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091f180 con 0x5594efda8c00 2023-05-20T01:28:50.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.006+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 19 ==== osd_op_reply(109 [pgnls-filter start_epoch 138 out=101b] v138'20 uv20 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.006+0000 7fe45dd097c0 20 datascan.forall_objects: OSD matched oid 1000000020e.00000000 2023-05-20T01:28:50.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.006+0000 7fe45dd097c0 10 datascan.operator(): handling object 1000000020e.0 2023-05-20T01:28:50.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.006+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:110 19.14 19:29971f97:::1000000020e.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091fc00 con 0x5594efda8c00 2023-05-20T01:28:50.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.006+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 20 ==== osd_op_reply(110 1000000020e.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv18 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.006+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: inode: 0x0x1000000020e 2023-05-20T01:28:50.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.006+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/25 2023-05-20T01:28:50.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.006+0000 7fe45dd097c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=25 2023-05-20T01:28:50.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.006+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:111 18.10 18:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef4000 con 0x5594efda8800 2023-05-20T01:28:50.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.006+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 21 ==== osd_op_reply(111 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.006+0000 7fe45dd097c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-05-20T01:28:50.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.006+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:112 18.1f 18:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef5880 con 0x5594efda8c00 2023-05-20T01:28:50.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.006+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 21 ==== osd_op_reply(112 1.00000000 [omap-get-vals-by-keys out=589b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+589 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.006+0000 7fe45dd097c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-05-20T01:28:50.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.006+0000 7fe45dd097c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-05-20T01:28:50.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.006+0000 7fe45dd097c0 20 datascan.get_frag_of: frag is 1* 2023-05-20T01:28:50.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.006+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:113 18.1b 18:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef4700 con 0x5594efda9800 2023-05-20T01:28:50.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.007+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 46 ==== osd_op_reply(113 10000000000.01800000 [omap-get-header out=274b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.007+0000 7fe45dd097c0 10 datascan.find_or_create_dirfrag: read_version = 3 2023-05-20T01:28:50.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.007+0000 7fe45dd097c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-05-20T01:28:50.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.007+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:114 18.1b 18:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef4380 con 0x5594efda9800 2023-05-20T01:28:50.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.007+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 47 ==== osd_op_reply(114 10000000000.01800000 [omap-get-vals-by-keys out=577b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+577 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.007+0000 7fe45dd097c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=25 frag=1*, r=0 2023-05-20T01:28:50.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.007+0000 7fe45dd097c0 20 datascan.inject_with_backtrace: Dentry 0x0x10000000000/25 already exists and points to me 2023-05-20T01:28:50.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.007+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:115 19.14 19:2ab28be7::::head [pgnls-filter start_epoch 138 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef5500 con 0x5594efda8c00 2023-05-20T01:28:50.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.007+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 22 ==== osd_op_reply(115 [pgnls-filter start_epoch 138 out=101b] v138'20 uv20 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.007+0000 7fe45dd097c0 20 datascan.forall_objects: OSD matched oid 10000000211.00000000 2023-05-20T01:28:50.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.007+0000 7fe45dd097c0 10 datascan.operator(): handling object 10000000211.0 2023-05-20T01:28:50.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.007+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:116 19.14 19:2ab28be7:::10000000211.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef5180 con 0x5594efda8c00 2023-05-20T01:28:50.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.008+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 23 ==== osd_op_reply(116 10000000211.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv19 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.008+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000211 2023-05-20T01:28:50.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.008+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/28 2023-05-20T01:28:50.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.008+0000 7fe45dd097c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=28 2023-05-20T01:28:50.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.008+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:117 18.10 18:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef5c00 con 0x5594efda8800 2023-05-20T01:28:50.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.008+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 22 ==== osd_op_reply(117 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.008+0000 7fe45dd097c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-05-20T01:28:50.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.008+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:118 18.1f 18:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef5880 con 0x5594efda8c00 2023-05-20T01:28:50.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.008+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 24 ==== osd_op_reply(118 1.00000000 [omap-get-vals-by-keys out=589b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+589 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.008+0000 7fe45dd097c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-05-20T01:28:50.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.008+0000 7fe45dd097c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-05-20T01:28:50.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.008+0000 7fe45dd097c0 20 datascan.get_frag_of: frag is 0* 2023-05-20T01:28:50.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.008+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:119 18.5 18:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef4000 con 0x5594efda9800 2023-05-20T01:28:50.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.008+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 48 ==== osd_op_reply(119 10000000000.01000000 [omap-get-header out=274b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.008+0000 7fe45dd097c0 10 datascan.find_or_create_dirfrag: read_version = 3 2023-05-20T01:28:50.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.008+0000 7fe45dd097c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-05-20T01:28:50.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.008+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:120 18.5 18:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef7500 con 0x5594efda9800 2023-05-20T01:28:50.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.009+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 49 ==== osd_op_reply(120 10000000000.01000000 [omap-get-vals-by-keys out=577b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+577 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.009+0000 7fe45dd097c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=28 frag=0*, r=0 2023-05-20T01:28:50.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.009+0000 7fe45dd097c0 20 datascan.inject_with_backtrace: Dentry 0x0x10000000000/28 already exists and points to me 2023-05-20T01:28:50.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.009+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:121 19.14 19:2b89f3ba::::head [pgnls-filter start_epoch 138 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef6000 con 0x5594efda8c00 2023-05-20T01:28:50.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.009+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 25 ==== osd_op_reply(121 [pgnls-filter start_epoch 138 out=81b] v138'20 uv20 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.009+0000 7fe45dd097c0 20 datascan.forall_objects: OSD matched oid 10000000221.00000000 2023-05-20T01:28:50.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.009+0000 7fe45dd097c0 10 datascan.operator(): handling object 10000000221.0 2023-05-20T01:28:50.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.009+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:122 19.14 19:2b89f3ba:::10000000221.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef6380 con 0x5594efda8c00 2023-05-20T01:28:50.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.009+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 26 ==== osd_op_reply(122 10000000221.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv20 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.009+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000221 2023-05-20T01:28:50.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.009+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/44 2023-05-20T01:28:50.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.009+0000 7fe45dd097c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=44 2023-05-20T01:28:50.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.009+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:123 18.10 18:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef6a80 con 0x5594efda8800 2023-05-20T01:28:50.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.010+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 23 ==== osd_op_reply(123 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.010+0000 7fe45dd097c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-05-20T01:28:50.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.010+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:124 18.1f 18:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef7180 con 0x5594efda8c00 2023-05-20T01:28:50.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.010+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 27 ==== osd_op_reply(124 1.00000000 [omap-get-vals-by-keys out=589b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+589 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.010+0000 7fe45dd097c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-05-20T01:28:50.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.010+0000 7fe45dd097c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-05-20T01:28:50.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.010+0000 7fe45dd097c0 20 datascan.get_frag_of: frag is 1* 2023-05-20T01:28:50.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.010+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:125 18.1b 18:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef7880 con 0x5594efda9800 2023-05-20T01:28:50.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.010+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 50 ==== osd_op_reply(125 10000000000.01800000 [omap-get-header out=274b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.010+0000 7fe45dd097c0 10 datascan.find_or_create_dirfrag: read_version = 3 2023-05-20T01:28:50.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.010+0000 7fe45dd097c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-05-20T01:28:50.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.010+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:126 18.1b 18:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef7c00 con 0x5594efda9800 2023-05-20T01:28:50.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.010+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 51 ==== osd_op_reply(126 10000000000.01800000 [omap-get-vals-by-keys out=577b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+577 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.010+0000 7fe45dd097c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=44 frag=1*, r=0 2023-05-20T01:28:50.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.010+0000 7fe45dd097c0 20 datascan.inject_with_backtrace: Dentry 0x0x10000000000/44 already exists and points to me 2023-05-20T01:28:50.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.010+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:127 19.34 19:2c000000::::head [pgnls-filter start_epoch 138 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091ee00 con 0x5594efda8800 2023-05-20T01:28:50.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.011+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 24 ==== osd_op_reply(127 [pgnls-filter start_epoch 138 out=49b] v136'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.011+0000 7fe456cad700 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:128 19.c 19:30000000::::head [pgnls-filter start_epoch 138 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef5500 con 0x5594efda9000 2023-05-20T01:28:50.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.011+0000 7fe45bcb7700 1 -- 172.21.15.16:0/1888659255 <== osd.3 v2:172.21.15.16:6800/3669781890 12 ==== osd_op_reply(128 [pgnls-filter start_epoch 138 out=81b] v138'9 uv9 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5594efda18c0 con 0x5594efda9000 2023-05-20T01:28:50.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.011+0000 7fe45dd097c0 20 datascan.forall_objects: OSD matched oid 100000001fc.00000000 2023-05-20T01:28:50.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.011+0000 7fe45dd097c0 10 datascan.operator(): handling object 100000001fc.0 2023-05-20T01:28:50.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.011+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:129 19.c 19:30604137:::100000001fc.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091f180 con 0x5594efda9000 2023-05-20T01:28:50.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.011+0000 7fe45bcb7700 1 -- 172.21.15.16:0/1888659255 <== osd.3 v2:172.21.15.16:6800/3669781890 13 ==== osd_op_reply(129 100000001fc.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (89) out=89b,getxattr (30) out=30b,getxattr] v0'0 uv9 ondisk = 0) v8 ==== 416+0+157 (crc 0 0 0) 0x5594efda18c0 con 0x5594efda9000 2023-05-20T01:28:50.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.011+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: inode: 0x0x100000001fc 2023-05-20T01:28:50.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.011+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/7 2023-05-20T01:28:50.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.011+0000 7fe45dd097c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=7 2023-05-20T01:28:50.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.011+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:130 18.10 18:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091f500 con 0x5594efda8800 2023-05-20T01:28:50.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.012+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 25 ==== osd_op_reply(130 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.012+0000 7fe45dd097c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-05-20T01:28:50.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.012+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:131 18.1f 18:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091e700 con 0x5594efda8c00 2023-05-20T01:28:50.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.012+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 28 ==== osd_op_reply(131 1.00000000 [omap-get-vals-by-keys out=589b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+589 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.012+0000 7fe45dd097c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-05-20T01:28:50.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.012+0000 7fe45dd097c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-05-20T01:28:50.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.012+0000 7fe45dd097c0 20 datascan.get_frag_of: frag is 1* 2023-05-20T01:28:50.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.012+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:132 18.1b 18:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091f880 con 0x5594efda9800 2023-05-20T01:28:50.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.012+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 52 ==== osd_op_reply(132 10000000000.01800000 [omap-get-header out=274b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.012+0000 7fe45dd097c0 10 datascan.find_or_create_dirfrag: read_version = 3 2023-05-20T01:28:50.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.012+0000 7fe45dd097c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-05-20T01:28:50.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.012+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:133 18.1b 18:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091ea80 con 0x5594efda9800 2023-05-20T01:28:50.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.012+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 53 ==== osd_op_reply(133 10000000000.01800000 [omap-get-vals-by-keys out=576b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+576 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.012+0000 7fe45dd097c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=7 frag=1*, r=0 2023-05-20T01:28:50.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.012+0000 7fe45dd097c0 20 datascan.inject_with_backtrace: Dentry 0x0x10000000000/7 already exists and points to me 2023-05-20T01:28:50.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.012+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:134 19.2c 19:34000000::::head [pgnls-filter start_epoch 138 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef6000 con 0x5594efda9000 2023-05-20T01:28:50.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.013+0000 7fe45bcb7700 1 -- 172.21.15.16:0/1888659255 <== osd.3 v2:172.21.15.16:6800/3669781890 14 ==== osd_op_reply(134 [pgnls-filter start_epoch 138 out=101b] v138'14 uv14 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x5594efda18c0 con 0x5594efda9000 2023-05-20T01:28:50.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.013+0000 7fe45dd097c0 20 datascan.forall_objects: OSD matched oid 10000000223.00000000 2023-05-20T01:28:50.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.013+0000 7fe45dd097c0 10 datascan.operator(): handling object 10000000223.0 2023-05-20T01:28:50.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.013+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:135 19.2c 19:35964134:::10000000223.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef6e00 con 0x5594efda9000 2023-05-20T01:28:50.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.013+0000 7fe45bcb7700 1 -- 172.21.15.16:0/1888659255 <== osd.3 v2:172.21.15.16:6800/3669781890 15 ==== osd_op_reply(135 10000000223.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv13 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x5594efda18c0 con 0x5594efda9000 2023-05-20T01:28:50.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.013+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000223 2023-05-20T01:28:50.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.013+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/46 2023-05-20T01:28:50.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.013+0000 7fe45dd097c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=46 2023-05-20T01:28:50.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.013+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:136 18.10 18:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef7180 con 0x5594efda8800 2023-05-20T01:28:50.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.013+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 26 ==== osd_op_reply(136 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.014+0000 7fe45dd097c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-05-20T01:28:50.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.014+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:137 18.1f 18:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef5180 con 0x5594efda8c00 2023-05-20T01:28:50.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.014+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 29 ==== osd_op_reply(137 1.00000000 [omap-get-vals-by-keys out=589b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+589 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.014+0000 7fe45dd097c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-05-20T01:28:50.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.014+0000 7fe45dd097c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-05-20T01:28:50.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.014+0000 7fe45dd097c0 20 datascan.get_frag_of: frag is 0* 2023-05-20T01:28:50.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.014+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:138 18.5 18:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef5880 con 0x5594efda9800 2023-05-20T01:28:50.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.014+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 54 ==== osd_op_reply(138 10000000000.01000000 [omap-get-header out=274b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.014+0000 7fe45dd097c0 10 datascan.find_or_create_dirfrag: read_version = 3 2023-05-20T01:28:50.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.014+0000 7fe45dd097c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-05-20T01:28:50.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.014+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:139 18.5 18:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef7500 con 0x5594efda9800 2023-05-20T01:28:50.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.014+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 55 ==== osd_op_reply(139 10000000000.01000000 [omap-get-vals-by-keys out=577b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+577 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.014+0000 7fe45dd097c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=46 frag=0*, r=0 2023-05-20T01:28:50.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.014+0000 7fe45dd097c0 20 datascan.inject_with_backtrace: Dentry 0x0x10000000000/46 already exists and points to me 2023-05-20T01:28:50.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.014+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:140 19.2c 19:37389ade::::head [pgnls-filter start_epoch 138 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef6a80 con 0x5594efda9000 2023-05-20T01:28:50.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.015+0000 7fe45bcb7700 1 -- 172.21.15.16:0/1888659255 <== osd.3 v2:172.21.15.16:6800/3669781890 16 ==== osd_op_reply(140 [pgnls-filter start_epoch 138 out=81b] v138'14 uv14 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5594efda18c0 con 0x5594efda9000 2023-05-20T01:28:50.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.015+0000 7fe45dd097c0 20 datascan.forall_objects: OSD matched oid 10000000209.00000000 2023-05-20T01:28:50.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.015+0000 7fe45dd097c0 10 datascan.operator(): handling object 10000000209.0 2023-05-20T01:28:50.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.015+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:141 19.2c 19:37389ade:::10000000209.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef7880 con 0x5594efda9000 2023-05-20T01:28:50.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.015+0000 7fe45bcb7700 1 -- 172.21.15.16:0/1888659255 <== osd.3 v2:172.21.15.16:6800/3669781890 17 ==== osd_op_reply(141 10000000209.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv14 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x5594efda18c0 con 0x5594efda9000 2023-05-20T01:28:50.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.015+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000209 2023-05-20T01:28:50.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.015+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/20 2023-05-20T01:28:50.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.015+0000 7fe45dd097c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=20 2023-05-20T01:28:50.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.015+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:142 18.10 18:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef7180 con 0x5594efda8800 2023-05-20T01:28:50.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.015+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 27 ==== osd_op_reply(142 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.015+0000 7fe45dd097c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-05-20T01:28:50.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.015+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:143 18.1f 18:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091ee00 con 0x5594efda8c00 2023-05-20T01:28:50.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.016+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 30 ==== osd_op_reply(143 1.00000000 [omap-get-vals-by-keys out=589b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+589 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.016+0000 7fe45dd097c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-05-20T01:28:50.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.016+0000 7fe45dd097c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-05-20T01:28:50.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.016+0000 7fe45dd097c0 20 datascan.get_frag_of: frag is 0* 2023-05-20T01:28:50.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.016+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:144 18.5 18:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091f500 con 0x5594efda9800 2023-05-20T01:28:50.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.016+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 56 ==== osd_op_reply(144 10000000000.01000000 [omap-get-header out=274b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.016+0000 7fe45dd097c0 10 datascan.find_or_create_dirfrag: read_version = 3 2023-05-20T01:28:50.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.016+0000 7fe45dd097c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-05-20T01:28:50.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.016+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:145 18.5 18:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091f880 con 0x5594efda9800 2023-05-20T01:28:50.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.016+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 57 ==== osd_op_reply(145 10000000000.01000000 [omap-get-vals-by-keys out=577b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+577 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.016+0000 7fe45dd097c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=20 frag=0*, r=0 2023-05-20T01:28:50.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.016+0000 7fe45dd097c0 20 datascan.inject_with_backtrace: Dentry 0x0x10000000000/20 already exists and points to me 2023-05-20T01:28:50.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.016+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:146 19.1c 19:38000000::::head [pgnls-filter start_epoch 138 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091ea80 con 0x5594efda8c00 2023-05-20T01:28:50.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.016+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 31 ==== osd_op_reply(146 [pgnls-filter start_epoch 138 out=49b] v136'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.016+0000 7fe456cad700 1 --2- 172.21.15.16:0/1888659255 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x5594efdd9000 0x5594efec9180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:50.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.016+0000 7fe456cad700 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:147 19.3c 19:3c000000::::head [pgnls-filter start_epoch 138 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef4a80 con 0x5594efdd9000 2023-05-20T01:28:50.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.017+0000 7fe45c4b8700 1 --2- 172.21.15.16:0/1888659255 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x5594efdd9000 0x5594efec9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:50.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.017+0000 7fe45c4b8700 1 --2- 172.21.15.16:0/1888659255 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x5594efdd9000 0x5594efec9180 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:50.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.017+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.2 v2:172.21.15.16:6816/293187030 1 ==== osd_op_reply(147 [pgnls-filter start_epoch 138 out=49b] v136'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5594efda1680 con 0x5594efdd9000 2023-05-20T01:28:50.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.017+0000 7fe4574ae700 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:148 19.2 19:40000000::::head [pgnls-filter start_epoch 138 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef6700 con 0x5594efda8c00 2023-05-20T01:28:50.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.017+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 32 ==== osd_op_reply(148 [pgnls-filter start_epoch 138 out=49b] v136'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.017+0000 7fe456cad700 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:149 19.22 19:44000000::::head [pgnls-filter start_epoch 138 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091fc00 con 0x5594efdd9000 2023-05-20T01:28:50.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.018+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.2 v2:172.21.15.16:6816/293187030 2 ==== osd_op_reply(149 [pgnls-filter start_epoch 138 out=101b] v138'17 uv17 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x5594efda1680 con 0x5594efdd9000 2023-05-20T01:28:50.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.018+0000 7fe45dd097c0 20 datascan.forall_objects: OSD matched oid 1000000024c.00000000 2023-05-20T01:28:50.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.018+0000 7fe45dd097c0 10 datascan.operator(): handling object 1000000024c.0 2023-05-20T01:28:50.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.018+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:150 19.22 19:450d4ca5:::1000000024c.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef4700 con 0x5594efdd9000 2023-05-20T01:28:50.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.018+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.2 v2:172.21.15.16:6816/293187030 3 ==== osd_op_reply(150 1000000024c.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv15 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x5594efda1680 con 0x5594efdd9000 2023-05-20T01:28:50.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.018+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: inode: 0x0x1000000024c 2023-05-20T01:28:50.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.018+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/87 2023-05-20T01:28:50.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.018+0000 7fe45dd097c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=87 2023-05-20T01:28:50.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.018+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:151 18.10 18:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef5c00 con 0x5594efda8800 2023-05-20T01:28:50.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.018+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 28 ==== osd_op_reply(151 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.018+0000 7fe45dd097c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-05-20T01:28:50.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.018+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:152 18.1f 18:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef4000 con 0x5594efda8c00 2023-05-20T01:28:50.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.019+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 33 ==== osd_op_reply(152 1.00000000 [omap-get-vals-by-keys out=589b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+589 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.019+0000 7fe45dd097c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-05-20T01:28:50.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.019+0000 7fe45dd097c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-05-20T01:28:50.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.019+0000 7fe45dd097c0 20 datascan.get_frag_of: frag is 0* 2023-05-20T01:28:50.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.019+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:153 18.5 18:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef6700 con 0x5594efda9800 2023-05-20T01:28:50.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.019+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 58 ==== osd_op_reply(153 10000000000.01000000 [omap-get-header out=274b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.019+0000 7fe45dd097c0 10 datascan.find_or_create_dirfrag: read_version = 3 2023-05-20T01:28:50.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.019+0000 7fe45dd097c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-05-20T01:28:50.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.019+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:154 18.5 18:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef6000 con 0x5594efda9800 2023-05-20T01:28:50.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.019+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 59 ==== osd_op_reply(154 10000000000.01000000 [omap-get-vals-by-keys out=577b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+577 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.019+0000 7fe45dd097c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=87 frag=0*, r=0 2023-05-20T01:28:50.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.019+0000 7fe45dd097c0 20 datascan.inject_with_backtrace: Dentry 0x0x10000000000/87 already exists and points to me 2023-05-20T01:28:50.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.019+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:155 19.22 19:45b8cac8::::head [pgnls-filter start_epoch 138 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef6380 con 0x5594efdd9000 2023-05-20T01:28:50.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.019+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.2 v2:172.21.15.16:6816/293187030 4 ==== osd_op_reply(155 [pgnls-filter start_epoch 138 out=101b] v138'17 uv17 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x5594efda1680 con 0x5594efdd9000 2023-05-20T01:28:50.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.019+0000 7fe45dd097c0 20 datascan.forall_objects: OSD matched oid 1000000023d.00000000 2023-05-20T01:28:50.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.019+0000 7fe45dd097c0 10 datascan.operator(): handling object 1000000023d.0 2023-05-20T01:28:50.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.019+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:156 19.22 19:45b8cac8:::1000000023d.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef6e00 con 0x5594efdd9000 2023-05-20T01:28:50.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.020+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.2 v2:172.21.15.16:6816/293187030 5 ==== osd_op_reply(156 1000000023d.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv16 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x5594efda1680 con 0x5594efdd9000 2023-05-20T01:28:50.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.020+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: inode: 0x0x1000000023d 2023-05-20T01:28:50.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.020+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/72 2023-05-20T01:28:50.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.020+0000 7fe45dd097c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=72 2023-05-20T01:28:50.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.020+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:157 18.10 18:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef6380 con 0x5594efda8800 2023-05-20T01:28:50.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.020+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 29 ==== osd_op_reply(157 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.020+0000 7fe45dd097c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-05-20T01:28:50.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.020+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:158 18.1f 18:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef4700 con 0x5594efda8c00 2023-05-20T01:28:50.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.020+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 34 ==== osd_op_reply(158 1.00000000 [omap-get-vals-by-keys out=589b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+589 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.020+0000 7fe45dd097c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-05-20T01:28:50.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.020+0000 7fe45dd097c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-05-20T01:28:50.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.020+0000 7fe45dd097c0 20 datascan.get_frag_of: frag is 1* 2023-05-20T01:28:50.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.020+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:159 18.1b 18:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef4380 con 0x5594efda9800 2023-05-20T01:28:50.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.021+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 60 ==== osd_op_reply(159 10000000000.01800000 [omap-get-header out=274b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.021+0000 7fe45dd097c0 10 datascan.find_or_create_dirfrag: read_version = 3 2023-05-20T01:28:50.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.021+0000 7fe45dd097c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-05-20T01:28:50.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.021+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:160 18.1b 18:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef4a80 con 0x5594efda9800 2023-05-20T01:28:50.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.021+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 61 ==== osd_op_reply(160 10000000000.01800000 [omap-get-vals-by-keys out=577b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+577 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.021+0000 7fe45dd097c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=72 frag=1*, r=0 2023-05-20T01:28:50.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.021+0000 7fe45dd097c0 20 datascan.inject_with_backtrace: Dentry 0x0x10000000000/72 already exists and points to me 2023-05-20T01:28:50.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.021+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:161 19.22 19:47efb22f::::head [pgnls-filter start_epoch 138 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef5180 con 0x5594efdd9000 2023-05-20T01:28:50.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.021+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.2 v2:172.21.15.16:6816/293187030 6 ==== osd_op_reply(161 [pgnls-filter start_epoch 138 out=81b] v138'17 uv17 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5594efda1680 con 0x5594efdd9000 2023-05-20T01:28:50.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.021+0000 7fe45dd097c0 20 datascan.forall_objects: OSD matched oid 10000000235.00000000 2023-05-20T01:28:50.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.021+0000 7fe45dd097c0 10 datascan.operator(): handling object 10000000235.0 2023-05-20T01:28:50.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.021+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:162 19.22 19:47efb22f:::10000000235.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef4000 con 0x5594efdd9000 2023-05-20T01:28:50.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.021+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.2 v2:172.21.15.16:6816/293187030 7 ==== osd_op_reply(162 10000000235.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv17 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x5594efda1680 con 0x5594efdd9000 2023-05-20T01:28:50.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.021+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000235 2023-05-20T01:28:50.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.021+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/64 2023-05-20T01:28:50.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.021+0000 7fe45dd097c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=64 2023-05-20T01:28:50.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.021+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:163 18.10 18:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091fc00 con 0x5594efda8800 2023-05-20T01:28:50.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.022+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 30 ==== osd_op_reply(163 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.022+0000 7fe45dd097c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-05-20T01:28:50.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.022+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:164 18.1f 18:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091ea80 con 0x5594efda8c00 2023-05-20T01:28:50.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.022+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 35 ==== osd_op_reply(164 1.00000000 [omap-get-vals-by-keys out=589b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+589 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.022+0000 7fe45dd097c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-05-20T01:28:50.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.022+0000 7fe45dd097c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-05-20T01:28:50.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.022+0000 7fe45dd097c0 20 datascan.get_frag_of: frag is 0* 2023-05-20T01:28:50.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.022+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:165 18.5 18:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091ee00 con 0x5594efda9800 2023-05-20T01:28:50.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.022+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 62 ==== osd_op_reply(165 10000000000.01000000 [omap-get-header out=274b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.022+0000 7fe45dd097c0 10 datascan.find_or_create_dirfrag: read_version = 3 2023-05-20T01:28:50.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.022+0000 7fe45dd097c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-05-20T01:28:50.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.022+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:166 18.5 18:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091e700 con 0x5594efda9800 2023-05-20T01:28:50.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.023+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 63 ==== osd_op_reply(166 10000000000.01000000 [omap-get-vals-by-keys out=577b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+577 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.023+0000 7fe45dd097c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=64 frag=0*, r=0 2023-05-20T01:28:50.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.023+0000 7fe45dd097c0 20 datascan.inject_with_backtrace: Dentry 0x0x10000000000/64 already exists and points to me 2023-05-20T01:28:50.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.023+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:167 19.12 19:48000000::::head [pgnls-filter start_epoch 138 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091f500 con 0x5594efda8c00 2023-05-20T01:28:50.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.023+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 36 ==== osd_op_reply(167 [pgnls-filter start_epoch 138 out=49b] v136'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.023+0000 7fe456cad700 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:168 19.32 19:4c000000::::head [pgnls-filter start_epoch 138 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef4e00 con 0x5594efda9800 2023-05-20T01:28:50.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.023+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 64 ==== osd_op_reply(168 [pgnls-filter start_epoch 138 out=81b] v138'11 uv11 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.023+0000 7fe45dd097c0 20 datascan.forall_objects: OSD matched oid 10000000252.00000000 2023-05-20T01:28:50.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.023+0000 7fe45dd097c0 10 datascan.operator(): handling object 10000000252.0 2023-05-20T01:28:50.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.023+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:169 19.32 19:4e8c6251:::10000000252.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091f880 con 0x5594efda9800 2023-05-20T01:28:50.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.024+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 65 ==== osd_op_reply(169 10000000252.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv11 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.024+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000252 2023-05-20T01:28:50.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.024+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/93 2023-05-20T01:28:50.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.024+0000 7fe45dd097c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=93 2023-05-20T01:28:50.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.024+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:170 18.10 18:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091ee00 con 0x5594efda8800 2023-05-20T01:28:50.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.024+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 31 ==== osd_op_reply(170 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.024+0000 7fe45dd097c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-05-20T01:28:50.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.024+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:171 18.1f 18:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091e700 con 0x5594efda8c00 2023-05-20T01:28:50.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.024+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 37 ==== osd_op_reply(171 1.00000000 [omap-get-vals-by-keys out=589b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+589 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.024+0000 7fe45dd097c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-05-20T01:28:50.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.024+0000 7fe45dd097c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-05-20T01:28:50.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.024+0000 7fe45dd097c0 20 datascan.get_frag_of: frag is 1* 2023-05-20T01:28:50.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.024+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:172 18.1b 18:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef5880 con 0x5594efda9800 2023-05-20T01:28:50.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.024+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 66 ==== osd_op_reply(172 10000000000.01800000 [omap-get-header out=274b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.025+0000 7fe45dd097c0 10 datascan.find_or_create_dirfrag: read_version = 3 2023-05-20T01:28:50.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.025+0000 7fe45dd097c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-05-20T01:28:50.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.025+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:173 18.1b 18:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef5c00 con 0x5594efda9800 2023-05-20T01:28:50.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.025+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 67 ==== osd_op_reply(173 10000000000.01800000 [omap-get-vals-by-keys out=577b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+577 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.025+0000 7fe45dd097c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=93 frag=1*, r=0 2023-05-20T01:28:50.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.025+0000 7fe45dd097c0 20 datascan.inject_with_backtrace: Dentry 0x0x10000000000/93 already exists and points to me 2023-05-20T01:28:50.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.025+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:174 19.a 19:50000000::::head [pgnls-filter start_epoch 138 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef4380 con 0x5594efdd9000 2023-05-20T01:28:50.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.025+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.2 v2:172.21.15.16:6816/293187030 8 ==== osd_op_reply(174 [pgnls-filter start_epoch 138 out=101b] v138'11 uv11 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x5594efda1680 con 0x5594efdd9000 2023-05-20T01:28:50.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.025+0000 7fe45dd097c0 20 datascan.forall_objects: OSD matched oid 1000000023f.00000000 2023-05-20T01:28:50.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.025+0000 7fe45dd097c0 10 datascan.operator(): handling object 1000000023f.0 2023-05-20T01:28:50.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.025+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:175 19.a 19:52dcfedc:::1000000023f.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef4a80 con 0x5594efdd9000 2023-05-20T01:28:50.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.025+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.2 v2:172.21.15.16:6816/293187030 9 ==== osd_op_reply(175 1000000023f.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv10 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x5594efda1680 con 0x5594efdd9000 2023-05-20T01:28:50.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.025+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: inode: 0x0x1000000023f 2023-05-20T01:28:50.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.025+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/74 2023-05-20T01:28:50.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.025+0000 7fe45dd097c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=74 2023-05-20T01:28:50.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.025+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:176 18.10 18:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef7180 con 0x5594efda8800 2023-05-20T01:28:50.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.026+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 32 ==== osd_op_reply(176 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.026+0000 7fe45dd097c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-05-20T01:28:50.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.026+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:177 18.1f 18:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef6700 con 0x5594efda8c00 2023-05-20T01:28:50.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.026+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 38 ==== osd_op_reply(177 1.00000000 [omap-get-vals-by-keys out=589b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+589 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.026+0000 7fe45dd097c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-05-20T01:28:50.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.026+0000 7fe45dd097c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-05-20T01:28:50.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.026+0000 7fe45dd097c0 20 datascan.get_frag_of: frag is 0* 2023-05-20T01:28:50.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.026+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:178 18.5 18:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef6000 con 0x5594efda9800 2023-05-20T01:28:50.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.027+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 68 ==== osd_op_reply(178 10000000000.01000000 [omap-get-header out=274b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.027+0000 7fe45dd097c0 10 datascan.find_or_create_dirfrag: read_version = 3 2023-05-20T01:28:50.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.027+0000 7fe45dd097c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-05-20T01:28:50.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.027+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:179 18.5 18:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef6380 con 0x5594efda9800 2023-05-20T01:28:50.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.027+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 69 ==== osd_op_reply(179 10000000000.01000000 [omap-get-vals-by-keys out=577b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+577 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.027+0000 7fe45dd097c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=74 frag=0*, r=0 2023-05-20T01:28:50.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.027+0000 7fe45dd097c0 20 datascan.inject_with_backtrace: Dentry 0x0x10000000000/74 already exists and points to me 2023-05-20T01:28:50.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.027+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:180 19.a 19:53d16f76::::head [pgnls-filter start_epoch 138 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff36000 con 0x5594efdd9000 2023-05-20T01:28:50.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.027+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.2 v2:172.21.15.16:6816/293187030 10 ==== osd_op_reply(180 [pgnls-filter start_epoch 138 out=81b] v138'11 uv11 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5594efda1680 con 0x5594efdd9000 2023-05-20T01:28:50.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.027+0000 7fe45dd097c0 20 datascan.forall_objects: OSD matched oid 1000000023b.00000000 2023-05-20T01:28:50.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.027+0000 7fe45dd097c0 10 datascan.operator(): handling object 1000000023b.0 2023-05-20T01:28:50.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.027+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:181 19.a 19:53d16f76:::1000000023b.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff36380 con 0x5594efdd9000 2023-05-20T01:28:50.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.027+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.2 v2:172.21.15.16:6816/293187030 11 ==== osd_op_reply(181 1000000023b.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv11 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x5594efda1680 con 0x5594efdd9000 2023-05-20T01:28:50.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.027+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: inode: 0x0x1000000023b 2023-05-20T01:28:50.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.027+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/70 2023-05-20T01:28:50.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.027+0000 7fe45dd097c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=70 2023-05-20T01:28:50.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.027+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:182 18.10 18:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff36a80 con 0x5594efda8800 2023-05-20T01:28:50.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.028+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 33 ==== osd_op_reply(182 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.028+0000 7fe45dd097c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-05-20T01:28:50.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.028+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:183 18.1f 18:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff36e00 con 0x5594efda8c00 2023-05-20T01:28:50.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.028+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 39 ==== osd_op_reply(183 1.00000000 [omap-get-vals-by-keys out=589b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+589 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.028+0000 7fe45dd097c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-05-20T01:28:50.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.028+0000 7fe45dd097c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-05-20T01:28:50.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.028+0000 7fe45dd097c0 20 datascan.get_frag_of: frag is 0* 2023-05-20T01:28:50.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.028+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:184 18.5 18:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff37180 con 0x5594efda9800 2023-05-20T01:28:50.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.028+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 70 ==== osd_op_reply(184 10000000000.01000000 [omap-get-header out=274b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.028+0000 7fe45dd097c0 10 datascan.find_or_create_dirfrag: read_version = 3 2023-05-20T01:28:50.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.028+0000 7fe45dd097c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-05-20T01:28:50.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.028+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:185 18.5 18:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff37500 con 0x5594efda9800 2023-05-20T01:28:50.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.029+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 71 ==== osd_op_reply(185 10000000000.01000000 [omap-get-vals-by-keys out=577b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+577 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.029+0000 7fe45dd097c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=70 frag=0*, r=0 2023-05-20T01:28:50.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.029+0000 7fe45dd097c0 20 datascan.inject_with_backtrace: Dentry 0x0x10000000000/70 already exists and points to me 2023-05-20T01:28:50.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.029+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:186 19.2a 19:54000000::::head [pgnls-filter start_epoch 138 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff37880 con 0x5594eff00000 2023-05-20T01:28:50.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.029+0000 7fe45bcb7700 1 -- 172.21.15.16:0/1888659255 <== osd.0 v2:172.21.15.16:6824/3923634179 9 ==== osd_op_reply(186 [pgnls-filter start_epoch 138 out=101b] v138'21 uv21 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x5594efda18c0 con 0x5594eff00000 2023-05-20T01:28:50.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.029+0000 7fe45dd097c0 20 datascan.forall_objects: OSD matched oid 1000000020a.00000000 2023-05-20T01:28:50.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.029+0000 7fe45dd097c0 10 datascan.operator(): handling object 1000000020a.0 2023-05-20T01:28:50.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.029+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:187 19.2a 19:55a6cd83:::1000000020a.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff37c00 con 0x5594eff00000 2023-05-20T01:28:50.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.029+0000 7fe45bcb7700 1 -- 172.21.15.16:0/1888659255 <== osd.0 v2:172.21.15.16:6824/3923634179 10 ==== osd_op_reply(187 1000000020a.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv18 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x5594efda18c0 con 0x5594eff00000 2023-05-20T01:28:50.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.029+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: inode: 0x0x1000000020a 2023-05-20T01:28:50.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.029+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/21 2023-05-20T01:28:50.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.029+0000 7fe45dd097c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=21 2023-05-20T01:28:50.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.029+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:188 18.10 18:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef7500 con 0x5594efda8800 2023-05-20T01:28:50.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.030+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 34 ==== osd_op_reply(188 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.030+0000 7fe45dd097c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-05-20T01:28:50.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.030+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:189 18.1f 18:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef6e00 con 0x5594efda8c00 2023-05-20T01:28:50.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.030+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 40 ==== osd_op_reply(189 1.00000000 [omap-get-vals-by-keys out=589b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+589 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.030+0000 7fe45dd097c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-05-20T01:28:50.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.030+0000 7fe45dd097c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-05-20T01:28:50.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.030+0000 7fe45dd097c0 20 datascan.get_frag_of: frag is 0* 2023-05-20T01:28:50.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.030+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:190 18.5 18:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef6a80 con 0x5594efda9800 2023-05-20T01:28:50.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.030+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 72 ==== osd_op_reply(190 10000000000.01000000 [omap-get-header out=274b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.030+0000 7fe45dd097c0 10 datascan.find_or_create_dirfrag: read_version = 3 2023-05-20T01:28:50.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.030+0000 7fe45dd097c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-05-20T01:28:50.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.030+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:191 18.5 18:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef6700 con 0x5594efda9800 2023-05-20T01:28:50.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.031+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 73 ==== osd_op_reply(191 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.031+0000 7fe45dd097c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=21 frag=0*, r=0 2023-05-20T01:28:50.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.031+0000 7fe45dd097c0 20 datascan.read_dentry: 21_head not found in result 2023-05-20T01:28:50.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.031+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: Linking inode 0x0x1000000020a at 0x0x10000000000/21 with size=2 bytes 2023-05-20T01:28:50.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.031+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:192 18.5 18:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef4a80 con 0x5594efda9800 2023-05-20T01:28:50.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.032+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 74 ==== osd_op_reply(192 10000000000.01000000 [omap-set-vals] v138'4 uv4 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.032+0000 7fe45dd097c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/21 pointing to 0x0x1000000020a 2023-05-20T01:28:50.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.032+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:193 19.2a 19:56889a45::::head [pgnls-filter start_epoch 138 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef4380 con 0x5594eff00000 2023-05-20T01:28:50.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.032+0000 7fe45bcb7700 1 -- 172.21.15.16:0/1888659255 <== osd.0 v2:172.21.15.16:6824/3923634179 11 ==== osd_op_reply(193 [pgnls-filter start_epoch 138 out=101b] v138'21 uv21 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x5594efda18c0 con 0x5594eff00000 2023-05-20T01:28:50.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.032+0000 7fe45dd097c0 20 datascan.forall_objects: OSD matched oid 10000000213.00000000 2023-05-20T01:28:50.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.032+0000 7fe45dd097c0 10 datascan.operator(): handling object 10000000213.0 2023-05-20T01:28:50.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.032+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:194 19.2a 19:56889a45:::10000000213.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef4000 con 0x5594eff00000 2023-05-20T01:28:50.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.032+0000 7fe45bcb7700 1 -- 172.21.15.16:0/1888659255 <== osd.0 v2:172.21.15.16:6824/3923634179 12 ==== osd_op_reply(194 10000000213.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv19 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x5594efda18c0 con 0x5594eff00000 2023-05-20T01:28:50.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.032+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000213 2023-05-20T01:28:50.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.032+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/30 2023-05-20T01:28:50.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.032+0000 7fe45dd097c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=30 2023-05-20T01:28:50.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.032+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:195 18.10 18:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef4700 con 0x5594efda8800 2023-05-20T01:28:50.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.033+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 35 ==== osd_op_reply(195 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.033+0000 7fe45dd097c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-05-20T01:28:50.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.033+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:196 18.1f 18:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091e700 con 0x5594efda8c00 2023-05-20T01:28:50.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.033+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 41 ==== osd_op_reply(196 1.00000000 [omap-get-vals-by-keys out=589b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+589 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.033+0000 7fe45dd097c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-05-20T01:28:50.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.033+0000 7fe45dd097c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-05-20T01:28:50.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.033+0000 7fe45dd097c0 20 datascan.get_frag_of: frag is 1* 2023-05-20T01:28:50.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.033+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:197 18.1b 18:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091f500 con 0x5594efda9800 2023-05-20T01:28:50.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.033+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 75 ==== osd_op_reply(197 10000000000.01800000 [omap-get-header out=274b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.033+0000 7fe45dd097c0 10 datascan.find_or_create_dirfrag: read_version = 3 2023-05-20T01:28:50.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.033+0000 7fe45dd097c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-05-20T01:28:50.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.033+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:198 18.1b 18:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091ea80 con 0x5594efda9800 2023-05-20T01:28:50.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.034+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 76 ==== osd_op_reply(198 10000000000.01800000 [omap-get-vals-by-keys out=577b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+577 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.034+0000 7fe45dd097c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=30 frag=1*, r=0 2023-05-20T01:28:50.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.034+0000 7fe45dd097c0 20 datascan.inject_with_backtrace: Dentry 0x0x10000000000/30 already exists and points to me 2023-05-20T01:28:50.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.034+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:199 19.2a 19:576ad4f4::::head [pgnls-filter start_epoch 138 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091f180 con 0x5594eff00000 2023-05-20T01:28:50.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.034+0000 7fe45bcb7700 1 -- 172.21.15.16:0/1888659255 <== osd.0 v2:172.21.15.16:6824/3923634179 13 ==== osd_op_reply(199 [pgnls-filter start_epoch 138 out=101b] v138'21 uv21 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x5594efda18c0 con 0x5594eff00000 2023-05-20T01:28:50.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.034+0000 7fe45dd097c0 20 datascan.forall_objects: OSD matched oid 10000000245.00000000 2023-05-20T01:28:50.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.034+0000 7fe45dd097c0 10 datascan.operator(): handling object 10000000245.0 2023-05-20T01:28:50.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.034+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:200 19.2a 19:576ad4f4:::10000000245.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef7c00 con 0x5594eff00000 2023-05-20T01:28:50.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.034+0000 7fe45bcb7700 1 -- 172.21.15.16:0/1888659255 <== osd.0 v2:172.21.15.16:6824/3923634179 14 ==== osd_op_reply(200 10000000245.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv20 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x5594efda18c0 con 0x5594eff00000 2023-05-20T01:28:50.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.034+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000245 2023-05-20T01:28:50.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.034+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/80 2023-05-20T01:28:50.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.034+0000 7fe45dd097c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=80 2023-05-20T01:28:50.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.034+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:201 18.10 18:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef7180 con 0x5594efda8800 2023-05-20T01:28:50.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.035+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 36 ==== osd_op_reply(201 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.035+0000 7fe45dd097c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-05-20T01:28:50.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.035+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:202 18.1f 18:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef6000 con 0x5594efda8c00 2023-05-20T01:28:50.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.035+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 42 ==== osd_op_reply(202 1.00000000 [omap-get-vals-by-keys out=589b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+589 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.035+0000 7fe45dd097c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-05-20T01:28:50.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.035+0000 7fe45dd097c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-05-20T01:28:50.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.035+0000 7fe45dd097c0 20 datascan.get_frag_of: frag is 0* 2023-05-20T01:28:50.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.035+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:203 18.5 18:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef6380 con 0x5594efda9800 2023-05-20T01:28:50.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.035+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 77 ==== osd_op_reply(203 10000000000.01000000 [omap-get-header out=274b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.035+0000 7fe45dd097c0 10 datascan.find_or_create_dirfrag: read_version = 4 2023-05-20T01:28:50.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.035+0000 7fe45dd097c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-05-20T01:28:50.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.035+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:204 18.5 18:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef7500 con 0x5594efda9800 2023-05-20T01:28:50.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.036+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 78 ==== osd_op_reply(204 10000000000.01000000 [omap-get-vals-by-keys out=577b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+577 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.036+0000 7fe45dd097c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=80 frag=0*, r=0 2023-05-20T01:28:50.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.036+0000 7fe45dd097c0 20 datascan.inject_with_backtrace: Dentry 0x0x10000000000/80 already exists and points to me 2023-05-20T01:28:50.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.036+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:205 19.2a 19:57fe7535::::head [pgnls-filter start_epoch 138 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef6a80 con 0x5594eff00000 2023-05-20T01:28:50.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.036+0000 7fe45bcb7700 1 -- 172.21.15.16:0/1888659255 <== osd.0 v2:172.21.15.16:6824/3923634179 15 ==== osd_op_reply(205 [pgnls-filter start_epoch 138 out=81b] v138'21 uv21 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5594efda18c0 con 0x5594eff00000 2023-05-20T01:28:50.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.036+0000 7fe45dd097c0 20 datascan.forall_objects: OSD matched oid 100000001f6.00000000 2023-05-20T01:28:50.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.036+0000 7fe45dd097c0 10 datascan.operator(): handling object 100000001f6.0 2023-05-20T01:28:50.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.036+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:206 19.2a 19:57fe7535:::100000001f6.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef6700 con 0x5594eff00000 2023-05-20T01:28:50.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.036+0000 7fe45bcb7700 1 -- 172.21.15.16:0/1888659255 <== osd.0 v2:172.21.15.16:6824/3923634179 16 ==== osd_op_reply(206 100000001f6.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (89) out=89b,getxattr (30) out=30b,getxattr] v0'0 uv21 ondisk = 0) v8 ==== 416+0+157 (crc 0 0 0) 0x5594efda18c0 con 0x5594eff00000 2023-05-20T01:28:50.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.036+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: inode: 0x0x100000001f6 2023-05-20T01:28:50.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.036+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/1 2023-05-20T01:28:50.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.036+0000 7fe45dd097c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=1 2023-05-20T01:28:50.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.036+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:207 18.10 18:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff36000 con 0x5594efda8800 2023-05-20T01:28:50.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.036+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 37 ==== osd_op_reply(207 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.037+0000 7fe45dd097c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-05-20T01:28:50.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.037+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:208 18.1f 18:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff37c00 con 0x5594efda8c00 2023-05-20T01:28:50.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.037+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 43 ==== osd_op_reply(208 1.00000000 [omap-get-vals-by-keys out=589b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+589 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.037+0000 7fe45dd097c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-05-20T01:28:50.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.037+0000 7fe45dd097c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-05-20T01:28:50.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.037+0000 7fe45dd097c0 20 datascan.get_frag_of: frag is 0* 2023-05-20T01:28:50.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.037+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:209 18.5 18:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff36a80 con 0x5594efda9800 2023-05-20T01:28:50.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.037+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 79 ==== osd_op_reply(209 10000000000.01000000 [omap-get-header out=274b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.037+0000 7fe45dd097c0 10 datascan.find_or_create_dirfrag: read_version = 4 2023-05-20T01:28:50.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.037+0000 7fe45dd097c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-05-20T01:28:50.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.037+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:210 18.5 18:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff37180 con 0x5594efda9800 2023-05-20T01:28:50.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.037+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 80 ==== osd_op_reply(210 10000000000.01000000 [omap-get-vals-by-keys out=576b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+576 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.037+0000 7fe45dd097c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=1 frag=0*, r=0 2023-05-20T01:28:50.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.037+0000 7fe45dd097c0 20 datascan.inject_with_backtrace: Dentry 0x0x10000000000/1 already exists and points to me 2023-05-20T01:28:50.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.037+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:211 19.1a 19:58000000::::head [pgnls-filter start_epoch 138 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff37500 con 0x5594efda9000 2023-05-20T01:28:50.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.038+0000 7fe45bcb7700 1 -- 172.21.15.16:0/1888659255 <== osd.3 v2:172.21.15.16:6800/3669781890 18 ==== osd_op_reply(211 [pgnls-filter start_epoch 138 out=101b] v138'13 uv13 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x5594efda18c0 con 0x5594efda9000 2023-05-20T01:28:50.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.038+0000 7fe45dd097c0 20 datascan.forall_objects: OSD matched oid 10000000226.00000000 2023-05-20T01:28:50.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.038+0000 7fe45dd097c0 10 datascan.operator(): handling object 10000000226.0 2023-05-20T01:28:50.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.038+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:212 19.1a 19:582f7b2d:::10000000226.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef4380 con 0x5594efda9000 2023-05-20T01:28:50.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.038+0000 7fe45bcb7700 1 -- 172.21.15.16:0/1888659255 <== osd.3 v2:172.21.15.16:6800/3669781890 19 ==== osd_op_reply(212 10000000226.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv12 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x5594efda18c0 con 0x5594efda9000 2023-05-20T01:28:50.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.038+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000226 2023-05-20T01:28:50.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.038+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/49 2023-05-20T01:28:50.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.038+0000 7fe45dd097c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=49 2023-05-20T01:28:50.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.038+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:213 18.10 18:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef4000 con 0x5594efda8800 2023-05-20T01:28:50.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.038+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 38 ==== osd_op_reply(213 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.038+0000 7fe45dd097c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-05-20T01:28:50.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.038+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:214 18.1f 18:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef4e00 con 0x5594efda8c00 2023-05-20T01:28:50.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.039+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 44 ==== osd_op_reply(214 1.00000000 [omap-get-vals-by-keys out=589b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+589 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.039+0000 7fe45dd097c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-05-20T01:28:50.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.039+0000 7fe45dd097c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-05-20T01:28:50.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.039+0000 7fe45dd097c0 20 datascan.get_frag_of: frag is 1* 2023-05-20T01:28:50.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.039+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:215 18.1b 18:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef5880 con 0x5594efda9800 2023-05-20T01:28:50.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.039+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 81 ==== osd_op_reply(215 10000000000.01800000 [omap-get-header out=274b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.039+0000 7fe45dd097c0 10 datascan.find_or_create_dirfrag: read_version = 3 2023-05-20T01:28:50.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.039+0000 7fe45dd097c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-05-20T01:28:50.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.039+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:216 18.1b 18:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef5c00 con 0x5594efda9800 2023-05-20T01:28:50.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.039+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 82 ==== osd_op_reply(216 10000000000.01800000 [omap-get-vals-by-keys out=577b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+577 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.039+0000 7fe45dd097c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=49 frag=1*, r=0 2023-05-20T01:28:50.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.039+0000 7fe45dd097c0 20 datascan.inject_with_backtrace: Dentry 0x0x10000000000/49 already exists and points to me 2023-05-20T01:28:50.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.039+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:217 19.1a 19:5a2a1294::::head [pgnls-filter start_epoch 138 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091f180 con 0x5594efda9000 2023-05-20T01:28:50.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.040+0000 7fe45bcb7700 1 -- 172.21.15.16:0/1888659255 <== osd.3 v2:172.21.15.16:6800/3669781890 20 ==== osd_op_reply(217 [pgnls-filter start_epoch 138 out=81b] v138'13 uv13 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5594efda18c0 con 0x5594efda9000 2023-05-20T01:28:50.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.040+0000 7fe45dd097c0 20 datascan.forall_objects: OSD matched oid 1000000023a.00000000 2023-05-20T01:28:50.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.040+0000 7fe45dd097c0 10 datascan.operator(): handling object 1000000023a.0 2023-05-20T01:28:50.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.040+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:218 19.1a 19:5a2a1294:::1000000023a.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091fc00 con 0x5594efda9000 2023-05-20T01:28:50.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.040+0000 7fe45bcb7700 1 -- 172.21.15.16:0/1888659255 <== osd.3 v2:172.21.15.16:6800/3669781890 21 ==== osd_op_reply(218 1000000023a.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv13 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x5594efda18c0 con 0x5594efda9000 2023-05-20T01:28:50.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.040+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: inode: 0x0x1000000023a 2023-05-20T01:28:50.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.040+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/69 2023-05-20T01:28:50.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.040+0000 7fe45dd097c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=69 2023-05-20T01:28:50.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.040+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:219 18.10 18:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091ee00 con 0x5594efda8800 2023-05-20T01:28:50.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.040+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 39 ==== osd_op_reply(219 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.040+0000 7fe45dd097c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-05-20T01:28:50.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.040+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:220 18.1f 18:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091f180 con 0x5594efda8c00 2023-05-20T01:28:50.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.041+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 45 ==== osd_op_reply(220 1.00000000 [omap-get-vals-by-keys out=589b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+589 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.041+0000 7fe45dd097c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-05-20T01:28:50.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.041+0000 7fe45dd097c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-05-20T01:28:50.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.041+0000 7fe45dd097c0 20 datascan.get_frag_of: frag is 0* 2023-05-20T01:28:50.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.041+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:221 18.5 18:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091f880 con 0x5594efda9800 2023-05-20T01:28:50.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.041+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 83 ==== osd_op_reply(221 10000000000.01000000 [omap-get-header out=274b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.041+0000 7fe45dd097c0 10 datascan.find_or_create_dirfrag: read_version = 4 2023-05-20T01:28:50.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.041+0000 7fe45dd097c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-05-20T01:28:50.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.041+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:222 18.5 18:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef5180 con 0x5594efda9800 2023-05-20T01:28:50.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.041+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 84 ==== osd_op_reply(222 10000000000.01000000 [omap-get-vals-by-keys out=577b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+577 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.041+0000 7fe45dd097c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=69 frag=0*, r=0 2023-05-20T01:28:50.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.041+0000 7fe45dd097c0 20 datascan.inject_with_backtrace: Dentry 0x0x10000000000/69 already exists and points to me 2023-05-20T01:28:50.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.041+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:223 19.3a 19:5c000000::::head [pgnls-filter start_epoch 138 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef4380 con 0x5594efda9c00 2023-05-20T01:28:50.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.042+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.4 v2:172.21.15.132:6817/506317223 3 ==== osd_op_reply(223 [pgnls-filter start_epoch 138 out=49b] v136'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5594efda1680 con 0x5594efda9c00 2023-05-20T01:28:50.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.042+0000 7fe456cad700 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:224 19.6 19:60000000::::head [pgnls-filter start_epoch 138 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef5500 con 0x5594efda9c00 2023-05-20T01:28:50.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.042+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.4 v2:172.21.15.132:6817/506317223 4 ==== osd_op_reply(224 [pgnls-filter start_epoch 138 out=81b] v138'10 uv10 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5594efda1680 con 0x5594efda9c00 2023-05-20T01:28:50.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.042+0000 7fe45dd097c0 20 datascan.forall_objects: OSD matched oid 10000000228.00000000 2023-05-20T01:28:50.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.042+0000 7fe45dd097c0 10 datascan.operator(): handling object 10000000228.0 2023-05-20T01:28:50.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.042+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:225 19.6 19:614797eb:::10000000228.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff37880 con 0x5594efda9c00 2023-05-20T01:28:50.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.042+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.4 v2:172.21.15.132:6817/506317223 5 ==== osd_op_reply(225 10000000228.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv10 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x5594efda1680 con 0x5594efda9c00 2023-05-20T01:28:50.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.042+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000228 2023-05-20T01:28:50.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.042+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/51 2023-05-20T01:28:50.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.042+0000 7fe45dd097c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=51 2023-05-20T01:28:50.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.042+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:226 18.10 18:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef7880 con 0x5594efda8800 2023-05-20T01:28:50.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.043+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 40 ==== osd_op_reply(226 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.043+0000 7fe45dd097c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-05-20T01:28:50.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.043+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:227 18.1f 18:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef7c00 con 0x5594efda8c00 2023-05-20T01:28:50.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.043+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 46 ==== osd_op_reply(227 1.00000000 [omap-get-vals-by-keys out=589b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+589 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.043+0000 7fe45dd097c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-05-20T01:28:50.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.043+0000 7fe45dd097c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-05-20T01:28:50.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.043+0000 7fe45dd097c0 20 datascan.get_frag_of: frag is 1* 2023-05-20T01:28:50.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.043+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:228 18.1b 18:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef6a80 con 0x5594efda9800 2023-05-20T01:28:50.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.043+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 85 ==== osd_op_reply(228 10000000000.01800000 [omap-get-header out=274b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.043+0000 7fe45dd097c0 10 datascan.find_or_create_dirfrag: read_version = 3 2023-05-20T01:28:50.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.043+0000 7fe45dd097c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-05-20T01:28:50.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.043+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:229 18.1b 18:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef6700 con 0x5594efda9800 2023-05-20T01:28:50.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.043+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 86 ==== osd_op_reply(229 10000000000.01800000 [omap-get-vals-by-keys out=577b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+577 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.043+0000 7fe45dd097c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=51 frag=1*, r=0 2023-05-20T01:28:50.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.043+0000 7fe45dd097c0 20 datascan.inject_with_backtrace: Dentry 0x0x10000000000/51 already exists and points to me 2023-05-20T01:28:50.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.043+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:230 19.26 19:64000000::::head [pgnls-filter start_epoch 138 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef4a80 con 0x5594efda9000 2023-05-20T01:28:50.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.044+0000 7fe45bcb7700 1 -- 172.21.15.16:0/1888659255 <== osd.3 v2:172.21.15.16:6800/3669781890 22 ==== osd_op_reply(230 [pgnls-filter start_epoch 138 out=101b] v138'24 uv24 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x5594efda18c0 con 0x5594efda9000 2023-05-20T01:28:50.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.044+0000 7fe45dd097c0 20 datascan.forall_objects: OSD matched oid 100000001ff.00000000 2023-05-20T01:28:50.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.044+0000 7fe45dd097c0 10 datascan.operator(): handling object 100000001ff.0 2023-05-20T01:28:50.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.044+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:231 19.26 19:658f2455:::100000001ff.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef4700 con 0x5594efda9000 2023-05-20T01:28:50.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.044+0000 7fe45bcb7700 1 -- 172.21.15.16:0/1888659255 <== osd.3 v2:172.21.15.16:6800/3669781890 23 ==== osd_op_reply(231 100000001ff.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv22 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x5594efda18c0 con 0x5594efda9000 2023-05-20T01:28:50.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.044+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: inode: 0x0x100000001ff 2023-05-20T01:28:50.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.044+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/10 2023-05-20T01:28:50.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.044+0000 7fe45dd097c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=10 2023-05-20T01:28:50.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.044+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:232 18.10 18:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef5880 con 0x5594efda8800 2023-05-20T01:28:50.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.044+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 41 ==== osd_op_reply(232 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.044+0000 7fe45dd097c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-05-20T01:28:50.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.044+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:233 18.1f 18:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef5c00 con 0x5594efda8c00 2023-05-20T01:28:50.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.045+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 47 ==== osd_op_reply(233 1.00000000 [omap-get-vals-by-keys out=589b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+589 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.045+0000 7fe45dd097c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-05-20T01:28:50.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.045+0000 7fe45dd097c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-05-20T01:28:50.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.045+0000 7fe45dd097c0 20 datascan.get_frag_of: frag is 0* 2023-05-20T01:28:50.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.045+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:234 18.5 18:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef5180 con 0x5594efda9800 2023-05-20T01:28:50.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.045+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 87 ==== osd_op_reply(234 10000000000.01000000 [omap-get-header out=274b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.045+0000 7fe45dd097c0 10 datascan.find_or_create_dirfrag: read_version = 4 2023-05-20T01:28:50.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.045+0000 7fe45dd097c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-05-20T01:28:50.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.045+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:235 18.5 18:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091f500 con 0x5594efda9800 2023-05-20T01:28:50.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.045+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 88 ==== osd_op_reply(235 10000000000.01000000 [omap-get-vals-by-keys out=577b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+577 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.045+0000 7fe45dd097c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=10 frag=0*, r=0 2023-05-20T01:28:50.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.045+0000 7fe45dd097c0 20 datascan.inject_with_backtrace: Dentry 0x0x10000000000/10 already exists and points to me 2023-05-20T01:28:50.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.045+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:236 19.26 19:65d05a5c::::head [pgnls-filter start_epoch 138 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091ea80 con 0x5594efda9000 2023-05-20T01:28:50.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.045+0000 7fe45bcb7700 1 -- 172.21.15.16:0/1888659255 <== osd.3 v2:172.21.15.16:6800/3669781890 24 ==== osd_op_reply(236 [pgnls-filter start_epoch 138 out=101b] v138'24 uv24 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x5594efda18c0 con 0x5594efda9000 2023-05-20T01:28:50.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.045+0000 7fe45dd097c0 20 datascan.forall_objects: OSD matched oid 10000000258.00000000 2023-05-20T01:28:50.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.045+0000 7fe45dd097c0 10 datascan.operator(): handling object 10000000258.0 2023-05-20T01:28:50.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.045+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:237 19.26 19:65d05a5c:::10000000258.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091ee00 con 0x5594efda9000 2023-05-20T01:28:50.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.046+0000 7fe45bcb7700 1 -- 172.21.15.16:0/1888659255 <== osd.3 v2:172.21.15.16:6800/3669781890 25 ==== osd_op_reply(237 10000000258.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv23 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x5594efda18c0 con 0x5594efda9000 2023-05-20T01:28:50.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.046+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000258 2023-05-20T01:28:50.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.046+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/99 2023-05-20T01:28:50.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.046+0000 7fe45dd097c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=99 2023-05-20T01:28:50.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.046+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:238 18.10 18:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff36000 con 0x5594efda8800 2023-05-20T01:28:50.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.046+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 42 ==== osd_op_reply(238 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.046+0000 7fe45dd097c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-05-20T01:28:50.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.046+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:239 18.1f 18:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff36a80 con 0x5594efda8c00 2023-05-20T01:28:50.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.046+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 48 ==== osd_op_reply(239 1.00000000 [omap-get-vals-by-keys out=589b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+589 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.046+0000 7fe45dd097c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-05-20T01:28:50.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.046+0000 7fe45dd097c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-05-20T01:28:50.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.046+0000 7fe45dd097c0 20 datascan.get_frag_of: frag is 1* 2023-05-20T01:28:50.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.046+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:240 18.1b 18:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff37180 con 0x5594efda9800 2023-05-20T01:28:50.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.047+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 89 ==== osd_op_reply(240 10000000000.01800000 [omap-get-header out=274b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.047+0000 7fe45dd097c0 10 datascan.find_or_create_dirfrag: read_version = 3 2023-05-20T01:28:50.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.047+0000 7fe45dd097c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-05-20T01:28:50.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.047+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:241 18.1b 18:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef7180 con 0x5594efda9800 2023-05-20T01:28:50.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.047+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 90 ==== osd_op_reply(241 10000000000.01800000 [omap-get-vals-by-keys out=577b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+577 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.047+0000 7fe45dd097c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=99 frag=1*, r=0 2023-05-20T01:28:50.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.047+0000 7fe45dd097c0 20 datascan.inject_with_backtrace: Dentry 0x0x10000000000/99 already exists and points to me 2023-05-20T01:28:50.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.047+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:242 19.26 19:65e03fee::::head [pgnls-filter start_epoch 138 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef6380 con 0x5594efda9000 2023-05-20T01:28:50.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.047+0000 7fe45bcb7700 1 -- 172.21.15.16:0/1888659255 <== osd.3 v2:172.21.15.16:6800/3669781890 26 ==== osd_op_reply(242 [pgnls-filter start_epoch 138 out=81b] v138'24 uv24 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5594efda18c0 con 0x5594efda9000 2023-05-20T01:28:50.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.047+0000 7fe45dd097c0 20 datascan.forall_objects: OSD matched oid 10000000240.00000000 2023-05-20T01:28:50.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.047+0000 7fe45dd097c0 10 datascan.operator(): handling object 10000000240.0 2023-05-20T01:28:50.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.047+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:243 19.26 19:65e03fee:::10000000240.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef7500 con 0x5594efda9000 2023-05-20T01:28:50.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.047+0000 7fe45bcb7700 1 -- 172.21.15.16:0/1888659255 <== osd.3 v2:172.21.15.16:6800/3669781890 27 ==== osd_op_reply(243 10000000240.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv24 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x5594efda18c0 con 0x5594efda9000 2023-05-20T01:28:50.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.047+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000240 2023-05-20T01:28:50.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.047+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/75 2023-05-20T01:28:50.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.047+0000 7fe45dd097c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=75 2023-05-20T01:28:50.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.047+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:244 18.10 18:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef4380 con 0x5594efda8800 2023-05-20T01:28:50.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.048+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 43 ==== osd_op_reply(244 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.048+0000 7fe45dd097c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-05-20T01:28:50.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.048+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:245 18.1f 18:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef5500 con 0x5594efda8c00 2023-05-20T01:28:50.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.048+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 49 ==== osd_op_reply(245 1.00000000 [omap-get-vals-by-keys out=589b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+589 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.048+0000 7fe45dd097c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-05-20T01:28:50.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.048+0000 7fe45dd097c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-05-20T01:28:50.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.048+0000 7fe45dd097c0 20 datascan.get_frag_of: frag is 0* 2023-05-20T01:28:50.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.048+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:246 18.5 18:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef5c00 con 0x5594efda9800 2023-05-20T01:28:50.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.048+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 91 ==== osd_op_reply(246 10000000000.01000000 [omap-get-header out=274b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.048+0000 7fe45dd097c0 10 datascan.find_or_create_dirfrag: read_version = 4 2023-05-20T01:28:50.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.048+0000 7fe45dd097c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-05-20T01:28:50.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.048+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:247 18.5 18:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef6e00 con 0x5594efda9800 2023-05-20T01:28:50.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.048+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 92 ==== osd_op_reply(247 10000000000.01000000 [omap-get-vals-by-keys out=577b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+577 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.048+0000 7fe45dd097c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=75 frag=0*, r=0 2023-05-20T01:28:50.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.048+0000 7fe45dd097c0 20 datascan.inject_with_backtrace: Dentry 0x0x10000000000/75 already exists and points to me 2023-05-20T01:28:50.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.048+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:248 19.16 19:68000000::::head [pgnls-filter start_epoch 138 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef6000 con 0x5594efdd9000 2023-05-20T01:28:50.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.049+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.2 v2:172.21.15.16:6816/293187030 12 ==== osd_op_reply(248 [pgnls-filter start_epoch 138 out=49b] v136'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5594efda1680 con 0x5594efdd9000 2023-05-20T01:28:50.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.049+0000 7fe4574ae700 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:249 19.36 19:6c000000::::head [pgnls-filter start_epoch 138 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff40000 con 0x5594efda9800 2023-05-20T01:28:50.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.049+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 93 ==== osd_op_reply(249 [pgnls-filter start_epoch 138 out=101b] v138'27 uv27 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.049+0000 7fe45dd097c0 20 datascan.forall_objects: OSD matched oid 10000000255.00000000 2023-05-20T01:28:50.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.049+0000 7fe45dd097c0 10 datascan.operator(): handling object 10000000255.0 2023-05-20T01:28:50.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.049+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:250 19.36 19:6cdfec98:::10000000255.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef7c00 con 0x5594efda9800 2023-05-20T01:28:50.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.049+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 94 ==== osd_op_reply(250 10000000255.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv24 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.049+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000255 2023-05-20T01:28:50.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.049+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/96 2023-05-20T01:28:50.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.049+0000 7fe45dd097c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=96 2023-05-20T01:28:50.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.049+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:251 18.10 18:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff36380 con 0x5594efda8800 2023-05-20T01:28:50.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.050+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 44 ==== osd_op_reply(251 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.050+0000 7fe45dd097c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-05-20T01:28:50.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.050+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:252 18.1f 18:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff36e00 con 0x5594efda8c00 2023-05-20T01:28:50.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.050+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 50 ==== osd_op_reply(252 1.00000000 [omap-get-vals-by-keys out=589b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+589 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.050+0000 7fe45dd097c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-05-20T01:28:50.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.050+0000 7fe45dd097c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-05-20T01:28:50.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.050+0000 7fe45dd097c0 20 datascan.get_frag_of: frag is 0* 2023-05-20T01:28:50.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.050+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:253 18.5 18:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff37c00 con 0x5594efda9800 2023-05-20T01:28:50.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.050+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 95 ==== osd_op_reply(253 10000000000.01000000 [omap-get-header out=274b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.050+0000 7fe45dd097c0 10 datascan.find_or_create_dirfrag: read_version = 4 2023-05-20T01:28:50.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.050+0000 7fe45dd097c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-05-20T01:28:50.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.050+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:254 18.5 18:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff37500 con 0x5594efda9800 2023-05-20T01:28:50.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.050+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 96 ==== osd_op_reply(254 10000000000.01000000 [omap-get-vals-by-keys out=577b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+577 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.050+0000 7fe45dd097c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=96 frag=0*, r=0 2023-05-20T01:28:50.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.050+0000 7fe45dd097c0 20 datascan.inject_with_backtrace: Dentry 0x0x10000000000/96 already exists and points to me 2023-05-20T01:28:50.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.051+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:255 19.36 19:6ee5e2c4::::head [pgnls-filter start_epoch 138 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff37880 con 0x5594efda9800 2023-05-20T01:28:50.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.051+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 97 ==== osd_op_reply(255 [pgnls-filter start_epoch 138 out=101b] v138'27 uv27 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.051+0000 7fe45dd097c0 20 datascan.forall_objects: OSD matched oid 100000001f9.00000000 2023-05-20T01:28:50.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.051+0000 7fe45dd097c0 10 datascan.operator(): handling object 100000001f9.0 2023-05-20T01:28:50.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.051+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:256 19.36 19:6ee5e2c4:::100000001f9.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091e700 con 0x5594efda9800 2023-05-20T01:28:50.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.051+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 98 ==== osd_op_reply(256 100000001f9.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (89) out=89b,getxattr (30) out=30b,getxattr] v0'0 uv25 ondisk = 0) v8 ==== 416+0+157 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.051+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: inode: 0x0x100000001f9 2023-05-20T01:28:50.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.051+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/4 2023-05-20T01:28:50.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.051+0000 7fe45dd097c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=4 2023-05-20T01:28:50.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.051+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:257 18.10 18:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff40000 con 0x5594efda8800 2023-05-20T01:28:50.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.051+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 45 ==== osd_op_reply(257 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.051+0000 7fe45dd097c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-05-20T01:28:50.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.051+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:258 18.1f 18:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff40a80 con 0x5594efda8c00 2023-05-20T01:28:50.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.052+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 51 ==== osd_op_reply(258 1.00000000 [omap-get-vals-by-keys out=589b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+589 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.052+0000 7fe45dd097c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-05-20T01:28:50.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.052+0000 7fe45dd097c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-05-20T01:28:50.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.052+0000 7fe45dd097c0 20 datascan.get_frag_of: frag is 0* 2023-05-20T01:28:50.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.052+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:259 18.5 18:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff40e00 con 0x5594efda9800 2023-05-20T01:28:50.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.052+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 99 ==== osd_op_reply(259 10000000000.01000000 [omap-get-header out=274b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.052+0000 7fe45dd097c0 10 datascan.find_or_create_dirfrag: read_version = 4 2023-05-20T01:28:50.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.052+0000 7fe45dd097c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-05-20T01:28:50.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.052+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:260 18.5 18:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff41180 con 0x5594efda9800 2023-05-20T01:28:50.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.052+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 100 ==== osd_op_reply(260 10000000000.01000000 [omap-get-vals-by-keys out=576b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+576 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.052+0000 7fe45dd097c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=4 frag=0*, r=0 2023-05-20T01:28:50.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.052+0000 7fe45dd097c0 20 datascan.inject_with_backtrace: Dentry 0x0x10000000000/4 already exists and points to me 2023-05-20T01:28:50.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.052+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:261 19.36 19:6f746908::::head [pgnls-filter start_epoch 138 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff41500 con 0x5594efda9800 2023-05-20T01:28:50.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.052+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 101 ==== osd_op_reply(261 [pgnls-filter start_epoch 138 out=101b] v138'27 uv27 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.052+0000 7fe45dd097c0 20 datascan.forall_objects: OSD matched oid 1000000022c.00000000 2023-05-20T01:28:50.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.052+0000 7fe45dd097c0 10 datascan.operator(): handling object 1000000022c.0 2023-05-20T01:28:50.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.052+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:262 19.36 19:6f746908:::1000000022c.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff41880 con 0x5594efda9800 2023-05-20T01:28:50.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.053+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 102 ==== osd_op_reply(262 1000000022c.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv26 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.053+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: inode: 0x0x1000000022c 2023-05-20T01:28:50.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.053+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/55 2023-05-20T01:28:50.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.053+0000 7fe45dd097c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=55 2023-05-20T01:28:50.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.053+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:263 18.10 18:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff36000 con 0x5594efda8800 2023-05-20T01:28:50.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.053+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 46 ==== osd_op_reply(263 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.053+0000 7fe45dd097c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-05-20T01:28:50.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.053+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:264 18.1f 18:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff37180 con 0x5594efda8c00 2023-05-20T01:28:50.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.053+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 52 ==== osd_op_reply(264 1.00000000 [omap-get-vals-by-keys out=589b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+589 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.053+0000 7fe45dd097c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-05-20T01:28:50.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.053+0000 7fe45dd097c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-05-20T01:28:50.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.053+0000 7fe45dd097c0 20 datascan.get_frag_of: frag is 0* 2023-05-20T01:28:50.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.053+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:265 18.5 18:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff36700 con 0x5594efda9800 2023-05-20T01:28:50.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.053+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 103 ==== osd_op_reply(265 10000000000.01000000 [omap-get-header out=274b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.053+0000 7fe45dd097c0 10 datascan.find_or_create_dirfrag: read_version = 4 2023-05-20T01:28:50.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.053+0000 7fe45dd097c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-05-20T01:28:50.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.053+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:266 18.5 18:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff36380 con 0x5594efda9800 2023-05-20T01:28:50.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.054+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 104 ==== osd_op_reply(266 10000000000.01000000 [omap-get-vals-by-keys out=577b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+577 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.054+0000 7fe45dd097c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=55 frag=0*, r=0 2023-05-20T01:28:50.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.054+0000 7fe45dd097c0 20 datascan.inject_with_backtrace: Dentry 0x0x10000000000/55 already exists and points to me 2023-05-20T01:28:50.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.054+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:267 19.36 19:6fe38dc2::::head [pgnls-filter start_epoch 138 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff37c00 con 0x5594efda9800 2023-05-20T01:28:50.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.054+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 105 ==== osd_op_reply(267 [pgnls-filter start_epoch 138 out=81b] v138'27 uv27 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.054+0000 7fe45dd097c0 20 datascan.forall_objects: OSD matched oid 100000001f7.00000000 2023-05-20T01:28:50.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.054+0000 7fe45dd097c0 10 datascan.operator(): handling object 100000001f7.0 2023-05-20T01:28:50.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.054+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:268 19.36 19:6fe38dc2:::100000001f7.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef7880 con 0x5594efda9800 2023-05-20T01:28:50.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.054+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 106 ==== osd_op_reply(268 100000001f7.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (89) out=89b,getxattr (30) out=30b,getxattr] v0'0 uv27 ondisk = 0) v8 ==== 416+0+157 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.054+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: inode: 0x0x100000001f7 2023-05-20T01:28:50.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.054+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/2 2023-05-20T01:28:50.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.054+0000 7fe45dd097c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=2 2023-05-20T01:28:50.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.054+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:269 18.10 18:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef6700 con 0x5594efda8800 2023-05-20T01:28:50.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.054+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 47 ==== osd_op_reply(269 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.055+0000 7fe45dd097c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-05-20T01:28:50.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.055+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:270 18.1f 18:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef7180 con 0x5594efda8c00 2023-05-20T01:28:50.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.055+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 53 ==== osd_op_reply(270 1.00000000 [omap-get-vals-by-keys out=589b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+589 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.055+0000 7fe45dd097c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-05-20T01:28:50.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.055+0000 7fe45dd097c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-05-20T01:28:50.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.055+0000 7fe45dd097c0 20 datascan.get_frag_of: frag is 0* 2023-05-20T01:28:50.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.055+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:271 18.5 18:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef6e00 con 0x5594efda9800 2023-05-20T01:28:50.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.055+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 107 ==== osd_op_reply(271 10000000000.01000000 [omap-get-header out=274b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.055+0000 7fe45dd097c0 10 datascan.find_or_create_dirfrag: read_version = 4 2023-05-20T01:28:50.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.055+0000 7fe45dd097c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-05-20T01:28:50.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.055+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:272 18.5 18:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef7c00 con 0x5594efda9800 2023-05-20T01:28:50.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.055+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 108 ==== osd_op_reply(272 10000000000.01000000 [omap-get-vals-by-keys out=576b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+576 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.055+0000 7fe45dd097c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=2 frag=0*, r=0 2023-05-20T01:28:50.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.055+0000 7fe45dd097c0 20 datascan.inject_with_backtrace: Dentry 0x0x10000000000/2 already exists and points to me 2023-05-20T01:28:50.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.055+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:273 19.e 19:70000000::::head [pgnls-filter start_epoch 138 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef5880 con 0x5594efda8800 2023-05-20T01:28:50.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.056+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 48 ==== osd_op_reply(273 [pgnls-filter start_epoch 138 out=81b] v138'8 uv8 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.056+0000 7fe45dd097c0 20 datascan.forall_objects: OSD matched oid 1000000022f.00000000 2023-05-20T01:28:50.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.056+0000 7fe45dd097c0 10 datascan.operator(): handling object 1000000022f.0 2023-05-20T01:28:50.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.056+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:274 19.e 19:71c13145:::1000000022f.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef5180 con 0x5594efda8800 2023-05-20T01:28:50.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.056+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 49 ==== osd_op_reply(274 1000000022f.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv8 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.056+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: inode: 0x0x1000000022f 2023-05-20T01:28:50.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.056+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/58 2023-05-20T01:28:50.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.056+0000 7fe45dd097c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=58 2023-05-20T01:28:50.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.056+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:275 18.10 18:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef5c00 con 0x5594efda8800 2023-05-20T01:28:50.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.056+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 50 ==== osd_op_reply(275 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.056+0000 7fe45dd097c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-05-20T01:28:50.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.056+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:276 18.1f 18:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091f500 con 0x5594efda8c00 2023-05-20T01:28:50.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.057+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 54 ==== osd_op_reply(276 1.00000000 [omap-get-vals-by-keys out=589b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+589 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.057+0000 7fe45dd097c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-05-20T01:28:50.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.057+0000 7fe45dd097c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-05-20T01:28:50.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.057+0000 7fe45dd097c0 20 datascan.get_frag_of: frag is 1* 2023-05-20T01:28:50.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.057+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:277 18.1b 18:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef6a80 con 0x5594efda9800 2023-05-20T01:28:50.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.057+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 109 ==== osd_op_reply(277 10000000000.01800000 [omap-get-header out=274b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.057+0000 7fe45dd097c0 10 datascan.find_or_create_dirfrag: read_version = 3 2023-05-20T01:28:50.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.057+0000 7fe45dd097c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-05-20T01:28:50.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.057+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:278 18.1b 18:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef7880 con 0x5594efda9800 2023-05-20T01:28:50.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.057+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 110 ==== osd_op_reply(278 10000000000.01800000 [omap-get-vals-by-keys out=577b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+577 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.057+0000 7fe45dd097c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=58 frag=1*, r=0 2023-05-20T01:28:50.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.057+0000 7fe45dd097c0 20 datascan.inject_with_backtrace: Dentry 0x0x10000000000/58 already exists and points to me 2023-05-20T01:28:50.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.057+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:279 19.2e 19:74000000::::head [pgnls-filter start_epoch 138 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef6700 con 0x5594efda9000 2023-05-20T01:28:50.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.057+0000 7fe45bcb7700 1 -- 172.21.15.16:0/1888659255 <== osd.3 v2:172.21.15.16:6800/3669781890 28 ==== osd_op_reply(279 [pgnls-filter start_epoch 138 out=101b] v138'19 uv19 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x5594efda18c0 con 0x5594efda9000 2023-05-20T01:28:50.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.057+0000 7fe45dd097c0 20 datascan.forall_objects: OSD matched oid 1000000021c.00000000 2023-05-20T01:28:50.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.057+0000 7fe45dd097c0 10 datascan.operator(): handling object 1000000021c.0 2023-05-20T01:28:50.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.057+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:280 19.2e 19:74d13cfc:::1000000021c.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef6e00 con 0x5594efda9000 2023-05-20T01:28:50.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.058+0000 7fe45bcb7700 1 -- 172.21.15.16:0/1888659255 <== osd.3 v2:172.21.15.16:6800/3669781890 29 ==== osd_op_reply(280 1000000021c.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv17 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x5594efda18c0 con 0x5594efda9000 2023-05-20T01:28:50.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.058+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: inode: 0x0x1000000021c 2023-05-20T01:28:50.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.058+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/39 2023-05-20T01:28:50.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.058+0000 7fe45dd097c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=39 2023-05-20T01:28:50.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.058+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:281 18.10 18:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff37880 con 0x5594efda8800 2023-05-20T01:28:50.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.058+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 51 ==== osd_op_reply(281 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.058+0000 7fe45dd097c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-05-20T01:28:50.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.058+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:282 18.1f 18:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff36000 con 0x5594efda8c00 2023-05-20T01:28:50.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.058+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 55 ==== osd_op_reply(282 1.00000000 [omap-get-vals-by-keys out=589b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+589 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.058+0000 7fe45dd097c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-05-20T01:28:50.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.058+0000 7fe45dd097c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-05-20T01:28:50.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.058+0000 7fe45dd097c0 20 datascan.get_frag_of: frag is 0* 2023-05-20T01:28:50.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.058+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:283 18.5 18:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff36700 con 0x5594efda9800 2023-05-20T01:28:50.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.059+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 111 ==== osd_op_reply(283 10000000000.01000000 [omap-get-header out=274b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.059+0000 7fe45dd097c0 10 datascan.find_or_create_dirfrag: read_version = 4 2023-05-20T01:28:50.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.059+0000 7fe45dd097c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-05-20T01:28:50.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.059+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:284 18.5 18:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff36380 con 0x5594efda9800 2023-05-20T01:28:50.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.059+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 112 ==== osd_op_reply(284 10000000000.01000000 [omap-get-vals-by-keys out=577b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+577 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.059+0000 7fe45dd097c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=39 frag=0*, r=0 2023-05-20T01:28:50.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.059+0000 7fe45dd097c0 20 datascan.inject_with_backtrace: Dentry 0x0x10000000000/39 already exists and points to me 2023-05-20T01:28:50.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.059+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:285 19.2e 19:7528f44f::::head [pgnls-filter start_epoch 138 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff37c00 con 0x5594efda9000 2023-05-20T01:28:50.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.059+0000 7fe45bcb7700 1 -- 172.21.15.16:0/1888659255 <== osd.3 v2:172.21.15.16:6800/3669781890 30 ==== osd_op_reply(285 [pgnls-filter start_epoch 138 out=101b] v138'19 uv19 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x5594efda18c0 con 0x5594efda9000 2023-05-20T01:28:50.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.059+0000 7fe45dd097c0 20 datascan.forall_objects: OSD matched oid 10000000242.00000000 2023-05-20T01:28:50.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.059+0000 7fe45dd097c0 10 datascan.operator(): handling object 10000000242.0 2023-05-20T01:28:50.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.059+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:286 19.2e 19:7528f44f:::10000000242.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff40000 con 0x5594efda9000 2023-05-20T01:28:50.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.060+0000 7fe45bcb7700 1 -- 172.21.15.16:0/1888659255 <== osd.3 v2:172.21.15.16:6800/3669781890 31 ==== osd_op_reply(286 10000000242.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv18 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x5594efda18c0 con 0x5594efda9000 2023-05-20T01:28:50.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.060+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000242 2023-05-20T01:28:50.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.060+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/77 2023-05-20T01:28:50.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.060+0000 7fe45dd097c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=77 2023-05-20T01:28:50.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.060+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:287 18.10 18:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff41180 con 0x5594efda8800 2023-05-20T01:28:50.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.060+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 52 ==== osd_op_reply(287 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.060+0000 7fe45dd097c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-05-20T01:28:50.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.060+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:288 18.1f 18:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff41500 con 0x5594efda8c00 2023-05-20T01:28:50.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.060+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 56 ==== osd_op_reply(288 1.00000000 [omap-get-vals-by-keys out=589b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+589 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.060+0000 7fe45dd097c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-05-20T01:28:50.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.060+0000 7fe45dd097c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-05-20T01:28:50.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.060+0000 7fe45dd097c0 20 datascan.get_frag_of: frag is 0* 2023-05-20T01:28:50.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.060+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:289 18.5 18:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff41c00 con 0x5594efda9800 2023-05-20T01:28:50.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.060+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 113 ==== osd_op_reply(289 10000000000.01000000 [omap-get-header out=274b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.061+0000 7fe45dd097c0 10 datascan.find_or_create_dirfrag: read_version = 4 2023-05-20T01:28:50.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.061+0000 7fe45dd097c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-05-20T01:28:50.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.061+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:290 18.5 18:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff41880 con 0x5594efda9800 2023-05-20T01:28:50.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.061+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 114 ==== osd_op_reply(290 10000000000.01000000 [omap-get-vals-by-keys out=577b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+577 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.061+0000 7fe45dd097c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=77 frag=0*, r=0 2023-05-20T01:28:50.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.061+0000 7fe45dd097c0 20 datascan.inject_with_backtrace: Dentry 0x0x10000000000/77 already exists and points to me 2023-05-20T01:28:50.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.061+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:291 19.2e 19:7611a124::::head [pgnls-filter start_epoch 138 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091f180 con 0x5594efda9000 2023-05-20T01:28:50.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.061+0000 7fe45bcb7700 1 -- 172.21.15.16:0/1888659255 <== osd.3 v2:172.21.15.16:6800/3669781890 32 ==== osd_op_reply(291 [pgnls-filter start_epoch 138 out=81b] v138'19 uv19 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5594efda18c0 con 0x5594efda9000 2023-05-20T01:28:50.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.061+0000 7fe45dd097c0 20 datascan.forall_objects: OSD matched oid 1000000021e.00000000 2023-05-20T01:28:50.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.061+0000 7fe45dd097c0 10 datascan.operator(): handling object 1000000021e.0 2023-05-20T01:28:50.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.061+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:292 19.2e 19:7611a124:::1000000021e.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091e700 con 0x5594efda9000 2023-05-20T01:28:50.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.061+0000 7fe45bcb7700 1 -- 172.21.15.16:0/1888659255 <== osd.3 v2:172.21.15.16:6800/3669781890 33 ==== osd_op_reply(292 1000000021e.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv19 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x5594efda18c0 con 0x5594efda9000 2023-05-20T01:28:50.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.061+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: inode: 0x0x1000000021e 2023-05-20T01:28:50.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.061+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/41 2023-05-20T01:28:50.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.061+0000 7fe45dd097c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=41 2023-05-20T01:28:50.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.061+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:293 18.10 18:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef7500 con 0x5594efda8800 2023-05-20T01:28:50.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.062+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 53 ==== osd_op_reply(293 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.062+0000 7fe45dd097c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-05-20T01:28:50.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.062+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:294 18.1f 18:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef6700 con 0x5594efda8c00 2023-05-20T01:28:50.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.062+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 57 ==== osd_op_reply(294 1.00000000 [omap-get-vals-by-keys out=589b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+589 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.062+0000 7fe45dd097c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-05-20T01:28:50.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.062+0000 7fe45dd097c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-05-20T01:28:50.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.062+0000 7fe45dd097c0 20 datascan.get_frag_of: frag is 1* 2023-05-20T01:28:50.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.062+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:295 18.1b 18:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef4a80 con 0x5594efda9800 2023-05-20T01:28:50.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.062+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 115 ==== osd_op_reply(295 10000000000.01800000 [omap-get-header out=274b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.062+0000 7fe45dd097c0 10 datascan.find_or_create_dirfrag: read_version = 3 2023-05-20T01:28:50.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.062+0000 7fe45dd097c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-05-20T01:28:50.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.062+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:296 18.1b 18:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef4000 con 0x5594efda9800 2023-05-20T01:28:50.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.062+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 116 ==== osd_op_reply(296 10000000000.01800000 [omap-get-vals-by-keys out=577b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+577 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.062+0000 7fe45dd097c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=41 frag=1*, r=0 2023-05-20T01:28:50.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.062+0000 7fe45dd097c0 20 datascan.inject_with_backtrace: Dentry 0x0x10000000000/41 already exists and points to me 2023-05-20T01:28:50.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.063+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:297 19.1e 19:78000000::::head [pgnls-filter start_epoch 138 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef4700 con 0x5594efda9000 2023-05-20T01:28:50.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.063+0000 7fe45bcb7700 1 -- 172.21.15.16:0/1888659255 <== osd.3 v2:172.21.15.16:6800/3669781890 34 ==== osd_op_reply(297 [pgnls-filter start_epoch 138 out=101b] v138'14 uv14 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x5594efda18c0 con 0x5594efda9000 2023-05-20T01:28:50.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.063+0000 7fe45dd097c0 20 datascan.forall_objects: OSD matched oid 1000000020c.00000000 2023-05-20T01:28:50.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.063+0000 7fe45dd097c0 10 datascan.operator(): handling object 1000000020c.0 2023-05-20T01:28:50.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.063+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:298 19.1e 19:789b801b:::1000000020c.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef4e00 con 0x5594efda9000 2023-05-20T01:28:50.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.063+0000 7fe45bcb7700 1 -- 172.21.15.16:0/1888659255 <== osd.3 v2:172.21.15.16:6800/3669781890 35 ==== osd_op_reply(298 1000000020c.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv12 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x5594efda18c0 con 0x5594efda9000 2023-05-20T01:28:50.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.063+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: inode: 0x0x1000000020c 2023-05-20T01:28:50.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.063+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/23 2023-05-20T01:28:50.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.063+0000 7fe45dd097c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=23 2023-05-20T01:28:50.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.063+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:299 18.10 18:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091ea80 con 0x5594efda8800 2023-05-20T01:28:50.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.063+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 54 ==== osd_op_reply(299 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.063+0000 7fe45dd097c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-05-20T01:28:50.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.063+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:300 18.1f 18:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091f880 con 0x5594efda8c00 2023-05-20T01:28:50.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.064+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 58 ==== osd_op_reply(300 1.00000000 [omap-get-vals-by-keys out=589b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+589 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.064+0000 7fe45dd097c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-05-20T01:28:50.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.064+0000 7fe45dd097c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-05-20T01:28:50.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.064+0000 7fe45dd097c0 20 datascan.get_frag_of: frag is 1* 2023-05-20T01:28:50.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.064+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:301 18.1b 18:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091ee00 con 0x5594efda9800 2023-05-20T01:28:50.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.064+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 117 ==== osd_op_reply(301 10000000000.01800000 [omap-get-header out=274b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.064+0000 7fe45dd097c0 10 datascan.find_or_create_dirfrag: read_version = 3 2023-05-20T01:28:50.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.064+0000 7fe45dd097c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-05-20T01:28:50.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.064+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:302 18.1b 18:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff48000 con 0x5594efda9800 2023-05-20T01:28:50.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.064+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 118 ==== osd_op_reply(302 10000000000.01800000 [omap-get-vals-by-keys out=577b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+577 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.064+0000 7fe45dd097c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=23 frag=1*, r=0 2023-05-20T01:28:50.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.064+0000 7fe45dd097c0 20 datascan.inject_with_backtrace: Dentry 0x0x10000000000/23 already exists and points to me 2023-05-20T01:28:50.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.064+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:303 19.1e 19:7a9d9b47::::head [pgnls-filter start_epoch 138 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff48380 con 0x5594efda9000 2023-05-20T01:28:50.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.064+0000 7fe45bcb7700 1 -- 172.21.15.16:0/1888659255 <== osd.3 v2:172.21.15.16:6800/3669781890 36 ==== osd_op_reply(303 [pgnls-filter start_epoch 138 out=101b] v138'14 uv14 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x5594efda18c0 con 0x5594efda9000 2023-05-20T01:28:50.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.065+0000 7fe45dd097c0 20 datascan.forall_objects: OSD matched oid 1000000022b.00000000 2023-05-20T01:28:50.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.065+0000 7fe45dd097c0 10 datascan.operator(): handling object 1000000022b.0 2023-05-20T01:28:50.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.065+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:304 19.1e 19:7a9d9b47:::1000000022b.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff48700 con 0x5594efda9000 2023-05-20T01:28:50.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.065+0000 7fe45bcb7700 1 -- 172.21.15.16:0/1888659255 <== osd.3 v2:172.21.15.16:6800/3669781890 37 ==== osd_op_reply(304 1000000022b.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv13 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x5594efda18c0 con 0x5594efda9000 2023-05-20T01:28:50.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.065+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: inode: 0x0x1000000022b 2023-05-20T01:28:50.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.065+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/54 2023-05-20T01:28:50.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.065+0000 7fe45dd097c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=54 2023-05-20T01:28:50.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.065+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:305 18.10 18:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff48e00 con 0x5594efda8800 2023-05-20T01:28:50.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.065+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 55 ==== osd_op_reply(305 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.065+0000 7fe45dd097c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-05-20T01:28:50.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.065+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:306 18.1f 18:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff49180 con 0x5594efda8c00 2023-05-20T01:28:50.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.066+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 59 ==== osd_op_reply(306 1.00000000 [omap-get-vals-by-keys out=589b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+589 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.066+0000 7fe45dd097c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-05-20T01:28:50.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.066+0000 7fe45dd097c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-05-20T01:28:50.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.066+0000 7fe45dd097c0 20 datascan.get_frag_of: frag is 1* 2023-05-20T01:28:50.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.066+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:307 18.1b 18:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff49500 con 0x5594efda9800 2023-05-20T01:28:50.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.066+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 119 ==== osd_op_reply(307 10000000000.01800000 [omap-get-header out=274b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.066+0000 7fe45dd097c0 10 datascan.find_or_create_dirfrag: read_version = 3 2023-05-20T01:28:50.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.066+0000 7fe45dd097c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-05-20T01:28:50.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.066+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:308 18.1b 18:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff49880 con 0x5594efda9800 2023-05-20T01:28:50.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.066+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 120 ==== osd_op_reply(308 10000000000.01800000 [omap-get-vals-by-keys out=577b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+577 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.066+0000 7fe45dd097c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=54 frag=1*, r=0 2023-05-20T01:28:50.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.066+0000 7fe45dd097c0 20 datascan.inject_with_backtrace: Dentry 0x0x10000000000/54 already exists and points to me 2023-05-20T01:28:50.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.066+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:309 19.1e 19:7ade2fd8::::head [pgnls-filter start_epoch 138 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff49c00 con 0x5594efda9000 2023-05-20T01:28:50.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.067+0000 7fe45bcb7700 1 -- 172.21.15.16:0/1888659255 <== osd.3 v2:172.21.15.16:6800/3669781890 38 ==== osd_op_reply(309 [pgnls-filter start_epoch 138 out=81b] v138'14 uv14 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5594efda18c0 con 0x5594efda9000 2023-05-20T01:28:50.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.067+0000 7fe45dd097c0 20 datascan.forall_objects: OSD matched oid 1000000024e.00000000 2023-05-20T01:28:50.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.067+0000 7fe45dd097c0 10 datascan.operator(): handling object 1000000024e.0 2023-05-20T01:28:50.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.067+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:310 19.1e 19:7ade2fd8:::1000000024e.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff48380 con 0x5594efda9000 2023-05-20T01:28:50.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.067+0000 7fe45bcb7700 1 -- 172.21.15.16:0/1888659255 <== osd.3 v2:172.21.15.16:6800/3669781890 39 ==== osd_op_reply(310 1000000024e.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv14 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x5594efda18c0 con 0x5594efda9000 2023-05-20T01:28:50.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.067+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: inode: 0x0x1000000024e 2023-05-20T01:28:50.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.067+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/89 2023-05-20T01:28:50.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.067+0000 7fe45dd097c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=89 2023-05-20T01:28:50.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.067+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:311 18.10 18:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091f180 con 0x5594efda8800 2023-05-20T01:28:50.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.067+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 56 ==== osd_op_reply(311 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.067+0000 7fe45dd097c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-05-20T01:28:50.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.067+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:312 18.1f 18:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091e700 con 0x5594efda8c00 2023-05-20T01:28:50.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.068+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 60 ==== osd_op_reply(312 1.00000000 [omap-get-vals-by-keys out=589b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+589 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.068+0000 7fe45dd097c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-05-20T01:28:50.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.068+0000 7fe45dd097c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-05-20T01:28:50.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.068+0000 7fe45dd097c0 20 datascan.get_frag_of: frag is 1* 2023-05-20T01:28:50.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.068+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:313 18.1b 18:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091fc00 con 0x5594efda9800 2023-05-20T01:28:50.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.068+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 121 ==== osd_op_reply(313 10000000000.01800000 [omap-get-header out=274b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.068+0000 7fe45dd097c0 10 datascan.find_or_create_dirfrag: read_version = 3 2023-05-20T01:28:50.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.068+0000 7fe45dd097c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-05-20T01:28:50.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.068+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:314 18.1b 18:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091ea80 con 0x5594efda9800 2023-05-20T01:28:50.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.068+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 122 ==== osd_op_reply(314 10000000000.01800000 [omap-get-vals-by-keys out=577b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+577 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.068+0000 7fe45dd097c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=89 frag=1*, r=0 2023-05-20T01:28:50.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.068+0000 7fe45dd097c0 20 datascan.inject_with_backtrace: Dentry 0x0x10000000000/89 already exists and points to me 2023-05-20T01:28:50.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.068+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:315 19.3e 19:7c000000::::head [pgnls-filter start_epoch 138 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091ee00 con 0x5594efda9800 2023-05-20T01:28:50.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.068+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 123 ==== osd_op_reply(315 [pgnls-filter start_epoch 138 out=81b] v138'7 uv7 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.068+0000 7fe45dd097c0 20 datascan.forall_objects: OSD matched oid 10000000229.00000000 2023-05-20T01:28:50.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.068+0000 7fe45dd097c0 10 datascan.operator(): handling object 10000000229.0 2023-05-20T01:28:50.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.068+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:316 19.3e 19:7f918703:::10000000229.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef4700 con 0x5594efda9800 2023-05-20T01:28:50.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.069+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 124 ==== osd_op_reply(316 10000000229.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv7 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.069+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000229 2023-05-20T01:28:50.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.069+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/52 2023-05-20T01:28:50.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.069+0000 7fe45dd097c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=52 2023-05-20T01:28:50.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.069+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:317 18.10 18:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef5880 con 0x5594efda8800 2023-05-20T01:28:50.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.069+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 57 ==== osd_op_reply(317 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.069+0000 7fe45dd097c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-05-20T01:28:50.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.069+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:318 18.1f 18:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef4380 con 0x5594efda8c00 2023-05-20T01:28:50.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.069+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 61 ==== osd_op_reply(318 1.00000000 [omap-get-vals-by-keys out=589b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+589 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.069+0000 7fe45dd097c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-05-20T01:28:50.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.069+0000 7fe45dd097c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-05-20T01:28:50.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.069+0000 7fe45dd097c0 20 datascan.get_frag_of: frag is 0* 2023-05-20T01:28:50.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.069+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:319 18.5 18:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef5180 con 0x5594efda9800 2023-05-20T01:28:50.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.070+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 125 ==== osd_op_reply(319 10000000000.01000000 [omap-get-header out=274b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.070+0000 7fe45dd097c0 10 datascan.find_or_create_dirfrag: read_version = 4 2023-05-20T01:28:50.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.070+0000 7fe45dd097c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-05-20T01:28:50.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.070+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:320 18.5 18:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef5c00 con 0x5594efda9800 2023-05-20T01:28:50.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.070+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 126 ==== osd_op_reply(320 10000000000.01000000 [omap-get-vals-by-keys out=577b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+577 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.070+0000 7fe45dd097c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=52 frag=0*, r=0 2023-05-20T01:28:50.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.070+0000 7fe45dd097c0 20 datascan.inject_with_backtrace: Dentry 0x0x10000000000/52 already exists and points to me 2023-05-20T01:28:50.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.070+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:321 19.1 19:80000000::::head [pgnls-filter start_epoch 138 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef4a80 con 0x5594efda9800 2023-05-20T01:28:50.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.070+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 127 ==== osd_op_reply(321 [pgnls-filter start_epoch 138 out=81b] v138'8 uv8 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.070+0000 7fe45dd097c0 20 datascan.forall_objects: OSD matched oid 10000000224.00000000 2023-05-20T01:28:50.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.070+0000 7fe45dd097c0 10 datascan.operator(): handling object 10000000224.0 2023-05-20T01:28:50.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.070+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:322 19.1 19:805bc955:::10000000224.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef4000 con 0x5594efda9800 2023-05-20T01:28:50.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.070+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 128 ==== osd_op_reply(322 10000000224.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv8 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.070+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000224 2023-05-20T01:28:50.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.070+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/47 2023-05-20T01:28:50.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.070+0000 7fe45dd097c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=47 2023-05-20T01:28:50.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.070+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:323 18.10 18:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef6380 con 0x5594efda8800 2023-05-20T01:28:50.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.071+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 58 ==== osd_op_reply(323 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.071+0000 7fe45dd097c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-05-20T01:28:50.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.071+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:324 18.1f 18:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef7c00 con 0x5594efda8c00 2023-05-20T01:28:50.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.071+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 62 ==== osd_op_reply(324 1.00000000 [omap-get-vals-by-keys out=589b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+589 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.071+0000 7fe45dd097c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-05-20T01:28:50.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.071+0000 7fe45dd097c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-05-20T01:28:50.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.071+0000 7fe45dd097c0 20 datascan.get_frag_of: frag is 1* 2023-05-20T01:28:50.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.071+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:325 18.1b 18:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef6a80 con 0x5594efda9800 2023-05-20T01:28:50.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.071+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 129 ==== osd_op_reply(325 10000000000.01800000 [omap-get-header out=274b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.071+0000 7fe45dd097c0 10 datascan.find_or_create_dirfrag: read_version = 3 2023-05-20T01:28:50.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.071+0000 7fe45dd097c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-05-20T01:28:50.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.071+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:326 18.1b 18:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef7880 con 0x5594efda9800 2023-05-20T01:28:50.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.072+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 130 ==== osd_op_reply(326 10000000000.01800000 [omap-get-vals-by-keys out=577b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+577 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.072+0000 7fe45dd097c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=47 frag=1*, r=0 2023-05-20T01:28:50.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.072+0000 7fe45dd097c0 20 datascan.inject_with_backtrace: Dentry 0x0x10000000000/47 already exists and points to me 2023-05-20T01:28:50.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.072+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:327 19.21 19:84000000::::head [pgnls-filter start_epoch 138 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef7500 con 0x5594efda8c00 2023-05-20T01:28:50.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.072+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 63 ==== osd_op_reply(327 [pgnls-filter start_epoch 138 out=49b] v136'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.072+0000 7fe4574ae700 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:328 19.11 19:88000000::::head [pgnls-filter start_epoch 138 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff40380 con 0x5594efda8800 2023-05-20T01:28:50.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.072+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 59 ==== osd_op_reply(328 [pgnls-filter start_epoch 138 out=49b] v136'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.072+0000 7fe456cad700 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:329 19.31 19:8c000000::::head [pgnls-filter start_epoch 138 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff36a80 con 0x5594efdd9000 2023-05-20T01:28:50.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.073+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.2 v2:172.21.15.16:6816/293187030 13 ==== osd_op_reply(329 [pgnls-filter start_epoch 138 out=101b] v138'13 uv13 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x5594efda1680 con 0x5594efdd9000 2023-05-20T01:28:50.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.073+0000 7fe45dd097c0 20 datascan.forall_objects: OSD matched oid 10000000216.00000000 2023-05-20T01:28:50.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.073+0000 7fe45dd097c0 10 datascan.operator(): handling object 10000000216.0 2023-05-20T01:28:50.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.073+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:330 19.31 19:8c51d9b0:::10000000216.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff40e00 con 0x5594efdd9000 2023-05-20T01:28:50.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.073+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.2 v2:172.21.15.16:6816/293187030 14 ==== osd_op_reply(330 10000000216.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv12 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x5594efda1680 con 0x5594efdd9000 2023-05-20T01:28:50.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.073+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000216 2023-05-20T01:28:50.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.073+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/33 2023-05-20T01:28:50.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.073+0000 7fe45dd097c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=33 2023-05-20T01:28:50.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.073+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:331 18.10 18:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff41180 con 0x5594efda8800 2023-05-20T01:28:50.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.073+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 60 ==== osd_op_reply(331 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.073+0000 7fe45dd097c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-05-20T01:28:50.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.073+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:332 18.1f 18:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff41c00 con 0x5594efda8c00 2023-05-20T01:28:50.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.074+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 64 ==== osd_op_reply(332 1.00000000 [omap-get-vals-by-keys out=589b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+589 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.074+0000 7fe45dd097c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-05-20T01:28:50.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.074+0000 7fe45dd097c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-05-20T01:28:50.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.074+0000 7fe45dd097c0 20 datascan.get_frag_of: frag is 1* 2023-05-20T01:28:50.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.074+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:333 18.1b 18:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff41880 con 0x5594efda9800 2023-05-20T01:28:50.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.074+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 131 ==== osd_op_reply(333 10000000000.01800000 [omap-get-header out=274b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.074+0000 7fe45dd097c0 10 datascan.find_or_create_dirfrag: read_version = 3 2023-05-20T01:28:50.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.074+0000 7fe45dd097c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-05-20T01:28:50.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.074+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:334 18.1b 18:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff37500 con 0x5594efda9800 2023-05-20T01:28:50.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.074+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 132 ==== osd_op_reply(334 10000000000.01800000 [omap-get-vals-by-keys out=577b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+577 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.074+0000 7fe45dd097c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=33 frag=1*, r=0 2023-05-20T01:28:50.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.074+0000 7fe45dd097c0 20 datascan.inject_with_backtrace: Dentry 0x0x10000000000/33 already exists and points to me 2023-05-20T01:28:50.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.074+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:335 19.31 19:8fb7f7b7::::head [pgnls-filter start_epoch 138 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff37c00 con 0x5594efdd9000 2023-05-20T01:28:50.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.074+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.2 v2:172.21.15.16:6816/293187030 15 ==== osd_op_reply(335 [pgnls-filter start_epoch 138 out=81b] v138'13 uv13 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5594efda1680 con 0x5594efdd9000 2023-05-20T01:28:50.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.074+0000 7fe45dd097c0 20 datascan.forall_objects: OSD matched oid 10000000251.00000000 2023-05-20T01:28:50.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.074+0000 7fe45dd097c0 10 datascan.operator(): handling object 10000000251.0 2023-05-20T01:28:50.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.074+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:336 19.31 19:8fb7f7b7:::10000000251.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff37880 con 0x5594efdd9000 2023-05-20T01:28:50.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.075+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.2 v2:172.21.15.16:6816/293187030 16 ==== osd_op_reply(336 10000000251.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv13 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x5594efda1680 con 0x5594efdd9000 2023-05-20T01:28:50.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.075+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000251 2023-05-20T01:28:50.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.075+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/92 2023-05-20T01:28:50.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.075+0000 7fe45dd097c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=92 2023-05-20T01:28:50.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.075+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:337 18.10 18:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff36380 con 0x5594efda8800 2023-05-20T01:28:50.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.075+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 61 ==== osd_op_reply(337 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.075+0000 7fe45dd097c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-05-20T01:28:50.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.075+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:338 18.1f 18:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef6e00 con 0x5594efda8c00 2023-05-20T01:28:50.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.075+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 65 ==== osd_op_reply(338 1.00000000 [omap-get-vals-by-keys out=589b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+589 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.075+0000 7fe45dd097c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-05-20T01:28:50.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.075+0000 7fe45dd097c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-05-20T01:28:50.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.075+0000 7fe45dd097c0 20 datascan.get_frag_of: frag is 0* 2023-05-20T01:28:50.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.075+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:339 18.5 18:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef6380 con 0x5594efda9800 2023-05-20T01:28:50.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.076+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 133 ==== osd_op_reply(339 10000000000.01000000 [omap-get-header out=274b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.076+0000 7fe45dd097c0 10 datascan.find_or_create_dirfrag: read_version = 4 2023-05-20T01:28:50.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.076+0000 7fe45dd097c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-05-20T01:28:50.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.076+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:340 18.5 18:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef6a80 con 0x5594efda9800 2023-05-20T01:28:50.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.076+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 134 ==== osd_op_reply(340 10000000000.01000000 [omap-get-vals-by-keys out=577b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+577 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.076+0000 7fe45dd097c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=92 frag=0*, r=0 2023-05-20T01:28:50.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.076+0000 7fe45dd097c0 20 datascan.inject_with_backtrace: Dentry 0x0x10000000000/92 already exists and points to me 2023-05-20T01:28:50.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.076+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:341 19.9 19:90000000::::head [pgnls-filter start_epoch 138 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef7880 con 0x5594efda8c00 2023-05-20T01:28:50.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.076+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 66 ==== osd_op_reply(341 [pgnls-filter start_epoch 138 out=101b] v138'8 uv8 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.076+0000 7fe45dd097c0 20 datascan.forall_objects: OSD matched oid 1000000024d.00000000 2023-05-20T01:28:50.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.076+0000 7fe45dd097c0 10 datascan.operator(): handling object 1000000024d.0 2023-05-20T01:28:50.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.076+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:342 19.9 19:915bff82:::1000000024d.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef6000 con 0x5594efda8c00 2023-05-20T01:28:50.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.077+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 67 ==== osd_op_reply(342 1000000024d.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv7 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.077+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: inode: 0x0x1000000024d 2023-05-20T01:28:50.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.077+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/88 2023-05-20T01:28:50.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.077+0000 7fe45dd097c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=88 2023-05-20T01:28:50.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.077+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:343 18.10 18:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef6700 con 0x5594efda8800 2023-05-20T01:28:50.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.077+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 62 ==== osd_op_reply(343 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.077+0000 7fe45dd097c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-05-20T01:28:50.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.077+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:344 18.1f 18:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef7c00 con 0x5594efda8c00 2023-05-20T01:28:50.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.078+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 68 ==== osd_op_reply(344 1.00000000 [omap-get-vals-by-keys out=589b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+589 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.078+0000 7fe45dd097c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-05-20T01:28:50.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.078+0000 7fe45dd097c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-05-20T01:28:50.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.078+0000 7fe45dd097c0 20 datascan.get_frag_of: frag is 0* 2023-05-20T01:28:50.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.078+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:345 18.5 18:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091f180 con 0x5594efda9800 2023-05-20T01:28:50.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.078+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 135 ==== osd_op_reply(345 10000000000.01000000 [omap-get-header out=274b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.078+0000 7fe45dd097c0 10 datascan.find_or_create_dirfrag: read_version = 4 2023-05-20T01:28:50.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.078+0000 7fe45dd097c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-05-20T01:28:50.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.078+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:346 18.5 18:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091fc00 con 0x5594efda9800 2023-05-20T01:28:50.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.078+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 136 ==== osd_op_reply(346 10000000000.01000000 [omap-get-vals-by-keys out=577b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+577 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.078+0000 7fe45dd097c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=88 frag=0*, r=0 2023-05-20T01:28:50.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.078+0000 7fe45dd097c0 20 datascan.inject_with_backtrace: Dentry 0x0x10000000000/88 already exists and points to me 2023-05-20T01:28:50.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.078+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:347 19.9 19:91e366d7::::head [pgnls-filter start_epoch 138 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091ea80 con 0x5594efda8c00 2023-05-20T01:28:50.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.079+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 69 ==== osd_op_reply(347 [pgnls-filter start_epoch 138 out=81b] v138'8 uv8 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.079+0000 7fe45dd097c0 20 datascan.forall_objects: OSD matched oid 10000000230.00000000 2023-05-20T01:28:50.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.079+0000 7fe45dd097c0 10 datascan.operator(): handling object 10000000230.0 2023-05-20T01:28:50.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.079+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:348 19.9 19:91e366d7:::10000000230.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091ee00 con 0x5594efda8c00 2023-05-20T01:28:50.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.079+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 70 ==== osd_op_reply(348 10000000230.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv8 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.079+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000230 2023-05-20T01:28:50.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.079+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/59 2023-05-20T01:28:50.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.079+0000 7fe45dd097c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=59 2023-05-20T01:28:50.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.079+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:349 18.10 18:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091f880 con 0x5594efda8800 2023-05-20T01:28:50.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.079+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 63 ==== osd_op_reply(349 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.079+0000 7fe45dd097c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-05-20T01:28:50.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.079+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:350 18.1f 18:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091e700 con 0x5594efda8c00 2023-05-20T01:28:50.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.080+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 71 ==== osd_op_reply(350 1.00000000 [omap-get-vals-by-keys out=589b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+589 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.080+0000 7fe45dd097c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-05-20T01:28:50.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.080+0000 7fe45dd097c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-05-20T01:28:50.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.080+0000 7fe45dd097c0 20 datascan.get_frag_of: frag is 0* 2023-05-20T01:28:50.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.080+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:351 18.5 18:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff48e00 con 0x5594efda9800 2023-05-20T01:28:50.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.080+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 137 ==== osd_op_reply(351 10000000000.01000000 [omap-get-header out=274b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.080+0000 7fe45dd097c0 10 datascan.find_or_create_dirfrag: read_version = 4 2023-05-20T01:28:50.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.080+0000 7fe45dd097c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-05-20T01:28:50.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.080+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:352 18.5 18:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff49500 con 0x5594efda9800 2023-05-20T01:28:50.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.080+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 138 ==== osd_op_reply(352 10000000000.01000000 [omap-get-vals-by-keys out=577b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+577 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.080+0000 7fe45dd097c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=59 frag=0*, r=0 2023-05-20T01:28:50.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.080+0000 7fe45dd097c0 20 datascan.inject_with_backtrace: Dentry 0x0x10000000000/59 already exists and points to me 2023-05-20T01:28:50.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.080+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:353 19.29 19:94000000::::head [pgnls-filter start_epoch 138 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff49880 con 0x5594efda8800 2023-05-20T01:28:50.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.081+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 64 ==== osd_op_reply(353 [pgnls-filter start_epoch 138 out=101b] v138'17 uv17 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.081+0000 7fe45dd097c0 20 datascan.forall_objects: OSD matched oid 1000000024a.00000000 2023-05-20T01:28:50.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.081+0000 7fe45dd097c0 10 datascan.operator(): handling object 1000000024a.0 2023-05-20T01:28:50.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.081+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:354 19.29 19:94c02207:::1000000024a.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff49180 con 0x5594efda8800 2023-05-20T01:28:50.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.081+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 65 ==== osd_op_reply(354 1000000024a.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv15 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.081+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: inode: 0x0x1000000024a 2023-05-20T01:28:50.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.081+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/85 2023-05-20T01:28:50.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.081+0000 7fe45dd097c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=85 2023-05-20T01:28:50.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.081+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:355 18.10 18:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff41180 con 0x5594efda8800 2023-05-20T01:28:50.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.082+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 66 ==== osd_op_reply(355 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.082+0000 7fe45dd097c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-05-20T01:28:50.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.082+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:356 18.1f 18:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff40a80 con 0x5594efda8c00 2023-05-20T01:28:50.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.082+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 72 ==== osd_op_reply(356 1.00000000 [omap-get-vals-by-keys out=589b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+589 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.082+0000 7fe45dd097c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-05-20T01:28:50.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.082+0000 7fe45dd097c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-05-20T01:28:50.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.082+0000 7fe45dd097c0 20 datascan.get_frag_of: frag is 0* 2023-05-20T01:28:50.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.082+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:357 18.5 18:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff41500 con 0x5594efda9800 2023-05-20T01:28:50.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.082+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 139 ==== osd_op_reply(357 10000000000.01000000 [omap-get-header out=274b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.082+0000 7fe45dd097c0 10 datascan.find_or_create_dirfrag: read_version = 4 2023-05-20T01:28:50.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.082+0000 7fe45dd097c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-05-20T01:28:50.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.082+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:358 18.5 18:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091ea80 con 0x5594efda9800 2023-05-20T01:28:50.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.082+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 140 ==== osd_op_reply(358 10000000000.01000000 [omap-get-vals-by-keys out=577b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+577 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.082+0000 7fe45dd097c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=85 frag=0*, r=0 2023-05-20T01:28:50.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.082+0000 7fe45dd097c0 20 datascan.inject_with_backtrace: Dentry 0x0x10000000000/85 already exists and points to me 2023-05-20T01:28:50.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.083+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:359 19.29 19:95e8e66d::::head [pgnls-filter start_epoch 138 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091f500 con 0x5594efda8800 2023-05-20T01:28:50.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.083+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 67 ==== osd_op_reply(359 [pgnls-filter start_epoch 138 out=101b] v138'17 uv17 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.083+0000 7fe45dd097c0 20 datascan.forall_objects: OSD matched oid 10000000256.00000000 2023-05-20T01:28:50.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.083+0000 7fe45dd097c0 10 datascan.operator(): handling object 10000000256.0 2023-05-20T01:28:50.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.083+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:360 19.29 19:95e8e66d:::10000000256.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091ee00 con 0x5594efda8800 2023-05-20T01:28:50.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.083+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 68 ==== osd_op_reply(360 10000000256.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv16 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.083+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000256 2023-05-20T01:28:50.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.083+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/97 2023-05-20T01:28:50.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.083+0000 7fe45dd097c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=97 2023-05-20T01:28:50.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.083+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:361 18.10 18:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef7500 con 0x5594efda8800 2023-05-20T01:28:50.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.083+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 69 ==== osd_op_reply(361 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.084+0000 7fe45dd097c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-05-20T01:28:50.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.084+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:362 18.1f 18:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef6e00 con 0x5594efda8c00 2023-05-20T01:28:50.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.084+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 73 ==== osd_op_reply(362 1.00000000 [omap-get-vals-by-keys out=589b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+589 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.084+0000 7fe45dd097c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-05-20T01:28:50.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.084+0000 7fe45dd097c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-05-20T01:28:50.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.084+0000 7fe45dd097c0 20 datascan.get_frag_of: frag is 0* 2023-05-20T01:28:50.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.084+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:363 18.5 18:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef7880 con 0x5594efda9800 2023-05-20T01:28:50.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.084+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 141 ==== osd_op_reply(363 10000000000.01000000 [omap-get-header out=274b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.084+0000 7fe45dd097c0 10 datascan.find_or_create_dirfrag: read_version = 4 2023-05-20T01:28:50.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.084+0000 7fe45dd097c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-05-20T01:28:50.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.084+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:364 18.5 18:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef7180 con 0x5594efda9800 2023-05-20T01:28:50.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.084+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 142 ==== osd_op_reply(364 10000000000.01000000 [omap-get-vals-by-keys out=577b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+577 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.084+0000 7fe45dd097c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=97 frag=0*, r=0 2023-05-20T01:28:50.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.084+0000 7fe45dd097c0 20 datascan.inject_with_backtrace: Dentry 0x0x10000000000/97 already exists and points to me 2023-05-20T01:28:50.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.084+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:365 19.29 19:97401066::::head [pgnls-filter start_epoch 138 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef6000 con 0x5594efda8800 2023-05-20T01:28:50.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.085+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 70 ==== osd_op_reply(365 [pgnls-filter start_epoch 138 out=81b] v138'17 uv17 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.085+0000 7fe45dd097c0 20 datascan.forall_objects: OSD matched oid 10000000247.00000000 2023-05-20T01:28:50.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.085+0000 7fe45dd097c0 10 datascan.operator(): handling object 10000000247.0 2023-05-20T01:28:50.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.085+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:366 19.29 19:97401066:::10000000247.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef7c00 con 0x5594efda8800 2023-05-20T01:28:50.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.085+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 71 ==== osd_op_reply(366 10000000247.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv17 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.085+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000247 2023-05-20T01:28:50.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.085+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/82 2023-05-20T01:28:50.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.085+0000 7fe45dd097c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=82 2023-05-20T01:28:50.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.085+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:367 18.10 18:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff37c00 con 0x5594efda8800 2023-05-20T01:28:50.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.085+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 72 ==== osd_op_reply(367 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.085+0000 7fe45dd097c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-05-20T01:28:50.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.085+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:368 18.1f 18:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff36700 con 0x5594efda8c00 2023-05-20T01:28:50.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.086+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 74 ==== osd_op_reply(368 1.00000000 [omap-get-vals-by-keys out=589b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+589 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.086+0000 7fe45dd097c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-05-20T01:28:50.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.086+0000 7fe45dd097c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-05-20T01:28:50.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.086+0000 7fe45dd097c0 20 datascan.get_frag_of: frag is 1* 2023-05-20T01:28:50.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.086+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:369 18.1b 18:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff37880 con 0x5594efda9800 2023-05-20T01:28:50.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.086+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 143 ==== osd_op_reply(369 10000000000.01800000 [omap-get-header out=274b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.086+0000 7fe45dd097c0 10 datascan.find_or_create_dirfrag: read_version = 3 2023-05-20T01:28:50.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.086+0000 7fe45dd097c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-05-20T01:28:50.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.086+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:370 18.1b 18:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff40000 con 0x5594efda9800 2023-05-20T01:28:50.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.086+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 144 ==== osd_op_reply(370 10000000000.01800000 [omap-get-vals-by-keys out=577b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+577 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.086+0000 7fe45dd097c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=82 frag=1*, r=0 2023-05-20T01:28:50.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.086+0000 7fe45dd097c0 20 datascan.inject_with_backtrace: Dentry 0x0x10000000000/82 already exists and points to me 2023-05-20T01:28:50.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.086+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:371 19.19 19:98000000::::head [pgnls-filter start_epoch 138 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff40e00 con 0x5594efda9800 2023-05-20T01:28:50.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.087+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 145 ==== osd_op_reply(371 [pgnls-filter start_epoch 138 out=101b] v138'16 uv16 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.087+0000 7fe45dd097c0 20 datascan.forall_objects: OSD matched oid 10000000200.00000000 2023-05-20T01:28:50.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.087+0000 7fe45dd097c0 10 datascan.operator(): handling object 10000000200.0 2023-05-20T01:28:50.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.087+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:372 19.19 19:992a6246:::10000000200.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff41c00 con 0x5594efda9800 2023-05-20T01:28:50.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.087+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 146 ==== osd_op_reply(372 10000000200.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv15 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.087+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000200 2023-05-20T01:28:50.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.087+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/11 2023-05-20T01:28:50.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.087+0000 7fe45dd097c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=11 2023-05-20T01:28:50.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.087+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:373 18.10 18:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef6a80 con 0x5594efda8800 2023-05-20T01:28:50.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.087+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 73 ==== osd_op_reply(373 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.087+0000 7fe45dd097c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-05-20T01:28:50.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.087+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:374 18.1f 18:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef6700 con 0x5594efda8c00 2023-05-20T01:28:50.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.087+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 75 ==== osd_op_reply(374 1.00000000 [omap-get-vals-by-keys out=589b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+589 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.087+0000 7fe45dd097c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-05-20T01:28:50.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.087+0000 7fe45dd097c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-05-20T01:28:50.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.087+0000 7fe45dd097c0 20 datascan.get_frag_of: frag is 0* 2023-05-20T01:28:50.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.088+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:375 18.5 18:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef7500 con 0x5594efda9800 2023-05-20T01:28:50.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.088+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 147 ==== osd_op_reply(375 10000000000.01000000 [omap-get-header out=274b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.088+0000 7fe45dd097c0 10 datascan.find_or_create_dirfrag: read_version = 4 2023-05-20T01:28:50.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.088+0000 7fe45dd097c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-05-20T01:28:50.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.088+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:376 18.5 18:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091f180 con 0x5594efda9800 2023-05-20T01:28:50.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.088+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 148 ==== osd_op_reply(376 10000000000.01000000 [omap-get-vals-by-keys out=577b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+577 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.088+0000 7fe45dd097c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=11 frag=0*, r=0 2023-05-20T01:28:50.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.088+0000 7fe45dd097c0 20 datascan.inject_with_backtrace: Dentry 0x0x10000000000/11 already exists and points to me 2023-05-20T01:28:50.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.088+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:377 19.19 19:997b7129::::head [pgnls-filter start_epoch 138 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091fc00 con 0x5594efda9800 2023-05-20T01:28:50.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.088+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 149 ==== osd_op_reply(377 [pgnls-filter start_epoch 138 out=81b] v138'16 uv16 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.088+0000 7fe45dd097c0 20 datascan.forall_objects: OSD matched oid 100000001f8.00000000 2023-05-20T01:28:50.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.088+0000 7fe45dd097c0 10 datascan.operator(): handling object 100000001f8.0 2023-05-20T01:28:50.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.088+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:378 19.19 19:997b7129:::100000001f8.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091f880 con 0x5594efda9800 2023-05-20T01:28:50.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.089+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 150 ==== osd_op_reply(378 100000001f8.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (89) out=89b,getxattr (30) out=30b,getxattr] v0'0 uv16 ondisk = 0) v8 ==== 416+0+157 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.089+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: inode: 0x0x100000001f8 2023-05-20T01:28:50.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.089+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/3 2023-05-20T01:28:50.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.089+0000 7fe45dd097c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=3 2023-05-20T01:28:50.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.089+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:379 18.10 18:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091f500 con 0x5594efda8800 2023-05-20T01:28:50.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.089+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 74 ==== osd_op_reply(379 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.089+0000 7fe45dd097c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-05-20T01:28:50.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.089+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:380 18.1f 18:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091e700 con 0x5594efda8c00 2023-05-20T01:28:50.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.089+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 76 ==== osd_op_reply(380 1.00000000 [omap-get-vals-by-keys out=589b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+589 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.089+0000 7fe45dd097c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-05-20T01:28:50.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.089+0000 7fe45dd097c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-05-20T01:28:50.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.089+0000 7fe45dd097c0 20 datascan.get_frag_of: frag is 0* 2023-05-20T01:28:50.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.089+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:381 18.5 18:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091ee00 con 0x5594efda9800 2023-05-20T01:28:50.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.089+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 151 ==== osd_op_reply(381 10000000000.01000000 [omap-get-header out=274b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.089+0000 7fe45dd097c0 10 datascan.find_or_create_dirfrag: read_version = 4 2023-05-20T01:28:50.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.089+0000 7fe45dd097c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-05-20T01:28:50.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.089+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:382 18.5 18:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff41880 con 0x5594efda9800 2023-05-20T01:28:50.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.090+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 152 ==== osd_op_reply(382 10000000000.01000000 [omap-get-vals-by-keys out=576b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+576 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.090+0000 7fe45dd097c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=3 frag=0*, r=0 2023-05-20T01:28:50.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.090+0000 7fe45dd097c0 20 datascan.inject_with_backtrace: Dentry 0x0x10000000000/3 already exists and points to me 2023-05-20T01:28:50.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.090+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:383 19.39 19:9c000000::::head [pgnls-filter start_epoch 138 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff40380 con 0x5594efda9c00 2023-05-20T01:28:50.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.090+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.4 v2:172.21.15.132:6817/506317223 6 ==== osd_op_reply(383 [pgnls-filter start_epoch 138 out=81b] v138'8 uv8 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5594efda1680 con 0x5594efda9c00 2023-05-20T01:28:50.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.090+0000 7fe45dd097c0 20 datascan.forall_objects: OSD matched oid 10000000222.00000000 2023-05-20T01:28:50.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.090+0000 7fe45dd097c0 10 datascan.operator(): handling object 10000000222.0 2023-05-20T01:28:50.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.090+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:384 19.39 19:9f1351c3:::10000000222.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff41180 con 0x5594efda9c00 2023-05-20T01:28:50.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.091+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.4 v2:172.21.15.132:6817/506317223 7 ==== osd_op_reply(384 10000000222.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv8 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x5594efda1680 con 0x5594efda9c00 2023-05-20T01:28:50.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.091+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000222 2023-05-20T01:28:50.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.091+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/45 2023-05-20T01:28:50.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.091+0000 7fe45dd097c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=45 2023-05-20T01:28:50.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.091+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:385 18.10 18:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff48e00 con 0x5594efda8800 2023-05-20T01:28:50.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.091+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 75 ==== osd_op_reply(385 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.091+0000 7fe45dd097c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-05-20T01:28:50.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.091+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:386 18.1f 18:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff49500 con 0x5594efda8c00 2023-05-20T01:28:50.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.091+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 77 ==== osd_op_reply(386 1.00000000 [omap-get-vals-by-keys out=589b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+589 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.091+0000 7fe45dd097c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-05-20T01:28:50.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.091+0000 7fe45dd097c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-05-20T01:28:50.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.091+0000 7fe45dd097c0 20 datascan.get_frag_of: frag is 1* 2023-05-20T01:28:50.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.091+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:387 18.1b 18:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff49880 con 0x5594efda9800 2023-05-20T01:28:50.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.091+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 153 ==== osd_op_reply(387 10000000000.01800000 [omap-get-header out=274b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.091+0000 7fe45dd097c0 10 datascan.find_or_create_dirfrag: read_version = 3 2023-05-20T01:28:50.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.091+0000 7fe45dd097c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-05-20T01:28:50.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.091+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:388 18.1b 18:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff49180 con 0x5594efda9800 2023-05-20T01:28:50.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.092+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 154 ==== osd_op_reply(388 10000000000.01800000 [omap-get-vals-by-keys out=577b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+577 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.092+0000 7fe45dd097c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=45 frag=1*, r=0 2023-05-20T01:28:50.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.092+0000 7fe45dd097c0 20 datascan.inject_with_backtrace: Dentry 0x0x10000000000/45 already exists and points to me 2023-05-20T01:28:50.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.092+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:389 19.5 19:a0000000::::head [pgnls-filter start_epoch 138 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff37500 con 0x5594efda9c00 2023-05-20T01:28:50.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.092+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.4 v2:172.21.15.132:6817/506317223 8 ==== osd_op_reply(389 [pgnls-filter start_epoch 138 out=101b] v138'11 uv11 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x5594efda1680 con 0x5594efda9c00 2023-05-20T01:28:50.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.092+0000 7fe45dd097c0 20 datascan.forall_objects: OSD matched oid 10000000204.00000000 2023-05-20T01:28:50.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.092+0000 7fe45dd097c0 10 datascan.operator(): handling object 10000000204.0 2023-05-20T01:28:50.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.092+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:390 19.5 19:a0beda95:::10000000204.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff36380 con 0x5594efda9c00 2023-05-20T01:28:50.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.092+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.4 v2:172.21.15.132:6817/506317223 9 ==== osd_op_reply(390 10000000204.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv10 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x5594efda1680 con 0x5594efda9c00 2023-05-20T01:28:50.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.093+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000204 2023-05-20T01:28:50.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.093+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/15 2023-05-20T01:28:50.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.093+0000 7fe45dd097c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=15 2023-05-20T01:28:50.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.093+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:391 18.10 18:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff49880 con 0x5594efda8800 2023-05-20T01:28:50.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.093+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 76 ==== osd_op_reply(391 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.093+0000 7fe45dd097c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-05-20T01:28:50.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.093+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:392 18.1f 18:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff40000 con 0x5594efda8c00 2023-05-20T01:28:50.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.093+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 78 ==== osd_op_reply(392 1.00000000 [omap-get-vals-by-keys out=589b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+589 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.093+0000 7fe45dd097c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-05-20T01:28:50.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.093+0000 7fe45dd097c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-05-20T01:28:50.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.093+0000 7fe45dd097c0 20 datascan.get_frag_of: frag is 0* 2023-05-20T01:28:50.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.093+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:393 18.5 18:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff40e00 con 0x5594efda9800 2023-05-20T01:28:50.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.093+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 155 ==== osd_op_reply(393 10000000000.01000000 [omap-get-header out=274b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.093+0000 7fe45dd097c0 10 datascan.find_or_create_dirfrag: read_version = 4 2023-05-20T01:28:50.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.093+0000 7fe45dd097c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-05-20T01:28:50.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.093+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:394 18.5 18:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff41c00 con 0x5594efda9800 2023-05-20T01:28:50.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.094+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 156 ==== osd_op_reply(394 10000000000.01000000 [omap-get-vals-by-keys out=577b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+577 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.094+0000 7fe45dd097c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=15 frag=0*, r=0 2023-05-20T01:28:50.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.094+0000 7fe45dd097c0 20 datascan.inject_with_backtrace: Dentry 0x0x10000000000/15 already exists and points to me 2023-05-20T01:28:50.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.094+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:395 19.5 19:a3a50212::::head [pgnls-filter start_epoch 138 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff41880 con 0x5594efda9c00 2023-05-20T01:28:50.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.094+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.4 v2:172.21.15.132:6817/506317223 10 ==== osd_op_reply(395 [pgnls-filter start_epoch 138 out=81b] v138'11 uv11 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5594efda1680 con 0x5594efda9c00 2023-05-20T01:28:50.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.094+0000 7fe45dd097c0 20 datascan.forall_objects: OSD matched oid 10000000254.00000000 2023-05-20T01:28:50.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.094+0000 7fe45dd097c0 10 datascan.operator(): handling object 10000000254.0 2023-05-20T01:28:50.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.094+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:396 19.5 19:a3a50212:::10000000254.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091f180 con 0x5594efda9c00 2023-05-20T01:28:50.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.094+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.4 v2:172.21.15.132:6817/506317223 11 ==== osd_op_reply(396 10000000254.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv11 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x5594efda1680 con 0x5594efda9c00 2023-05-20T01:28:50.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.094+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000254 2023-05-20T01:28:50.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.094+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/95 2023-05-20T01:28:50.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.094+0000 7fe45dd097c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=95 2023-05-20T01:28:50.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.094+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:397 18.10 18:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091ea80 con 0x5594efda8800 2023-05-20T01:28:50.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.095+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 77 ==== osd_op_reply(397 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.095+0000 7fe45dd097c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-05-20T01:28:50.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.095+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:398 18.1f 18:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091f880 con 0x5594efda8c00 2023-05-20T01:28:50.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.095+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 79 ==== osd_op_reply(398 1.00000000 [omap-get-vals-by-keys out=589b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+589 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.095+0000 7fe45dd097c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-05-20T01:28:50.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.095+0000 7fe45dd097c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-05-20T01:28:50.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.095+0000 7fe45dd097c0 20 datascan.get_frag_of: frag is 0* 2023-05-20T01:28:50.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.095+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:399 18.5 18:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091f500 con 0x5594efda9800 2023-05-20T01:28:50.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.095+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 157 ==== osd_op_reply(399 10000000000.01000000 [omap-get-header out=274b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.095+0000 7fe45dd097c0 10 datascan.find_or_create_dirfrag: read_version = 4 2023-05-20T01:28:50.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.095+0000 7fe45dd097c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-05-20T01:28:50.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.095+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:400 18.5 18:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091ee00 con 0x5594efda9800 2023-05-20T01:28:50.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.095+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 158 ==== osd_op_reply(400 10000000000.01000000 [omap-get-vals-by-keys out=577b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+577 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.095+0000 7fe45dd097c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=95 frag=0*, r=0 2023-05-20T01:28:50.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.096+0000 7fe45dd097c0 20 datascan.inject_with_backtrace: Dentry 0x0x10000000000/95 already exists and points to me 2023-05-20T01:28:50.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.096+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:401 19.25 19:a4000000::::head [pgnls-filter start_epoch 138 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef7880 con 0x5594efdd9000 2023-05-20T01:28:50.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.096+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.2 v2:172.21.15.16:6816/293187030 17 ==== osd_op_reply(401 [pgnls-filter start_epoch 138 out=101b] v138'12 uv12 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x5594efda1680 con 0x5594efdd9000 2023-05-20T01:28:50.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.096+0000 7fe45dd097c0 20 datascan.forall_objects: OSD matched oid 10000000237.00000000 2023-05-20T01:28:50.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.096+0000 7fe45dd097c0 10 datascan.operator(): handling object 10000000237.0 2023-05-20T01:28:50.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.096+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:402 19.25 19:a53878a9:::10000000237.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef7180 con 0x5594efdd9000 2023-05-20T01:28:50.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.096+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.2 v2:172.21.15.16:6816/293187030 18 ==== osd_op_reply(402 10000000237.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv11 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x5594efda1680 con 0x5594efdd9000 2023-05-20T01:28:50.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.096+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000237 2023-05-20T01:28:50.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.096+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/66 2023-05-20T01:28:50.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.096+0000 7fe45dd097c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=66 2023-05-20T01:28:50.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.096+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:403 18.10 18:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef7c00 con 0x5594efda8800 2023-05-20T01:28:50.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.097+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 78 ==== osd_op_reply(403 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.097+0000 7fe45dd097c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-05-20T01:28:50.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.097+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:404 18.1f 18:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef6380 con 0x5594efda8c00 2023-05-20T01:28:50.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.097+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 80 ==== osd_op_reply(404 1.00000000 [omap-get-vals-by-keys out=589b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+589 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.097+0000 7fe45dd097c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-05-20T01:28:50.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.097+0000 7fe45dd097c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-05-20T01:28:50.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.097+0000 7fe45dd097c0 20 datascan.get_frag_of: frag is 0* 2023-05-20T01:28:50.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.097+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:405 18.5 18:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef6a80 con 0x5594efda9800 2023-05-20T01:28:50.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.097+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 159 ==== osd_op_reply(405 10000000000.01000000 [omap-get-header out=274b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.097+0000 7fe45dd097c0 10 datascan.find_or_create_dirfrag: read_version = 4 2023-05-20T01:28:50.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.097+0000 7fe45dd097c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-05-20T01:28:50.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.097+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:406 18.5 18:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef7500 con 0x5594efda9800 2023-05-20T01:28:50.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.097+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 160 ==== osd_op_reply(406 10000000000.01000000 [omap-get-vals-by-keys out=577b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+577 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.097+0000 7fe45dd097c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=66 frag=0*, r=0 2023-05-20T01:28:50.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.097+0000 7fe45dd097c0 20 datascan.inject_with_backtrace: Dentry 0x0x10000000000/66 already exists and points to me 2023-05-20T01:28:50.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.097+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:407 19.25 19:a7ae8a0f::::head [pgnls-filter start_epoch 138 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff36a80 con 0x5594efdd9000 2023-05-20T01:28:50.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.098+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.2 v2:172.21.15.16:6816/293187030 19 ==== osd_op_reply(407 [pgnls-filter start_epoch 138 out=81b] v138'12 uv12 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5594efda1680 con 0x5594efdd9000 2023-05-20T01:28:50.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.098+0000 7fe45dd097c0 20 datascan.forall_objects: OSD matched oid 10000000243.00000000 2023-05-20T01:28:50.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.098+0000 7fe45dd097c0 10 datascan.operator(): handling object 10000000243.0 2023-05-20T01:28:50.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.098+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:408 19.25 19:a7ae8a0f:::10000000243.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff37c00 con 0x5594efdd9000 2023-05-20T01:28:50.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.098+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.2 v2:172.21.15.16:6816/293187030 20 ==== osd_op_reply(408 10000000243.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv12 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x5594efda1680 con 0x5594efdd9000 2023-05-20T01:28:50.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.098+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000243 2023-05-20T01:28:50.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.098+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/78 2023-05-20T01:28:50.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.098+0000 7fe45dd097c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=78 2023-05-20T01:28:50.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.098+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:409 18.10 18:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef6e00 con 0x5594efda8800 2023-05-20T01:28:50.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.098+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 79 ==== osd_op_reply(409 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.098+0000 7fe45dd097c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-05-20T01:28:50.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.098+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:410 18.1f 18:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef6700 con 0x5594efda8c00 2023-05-20T01:28:50.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.099+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 81 ==== osd_op_reply(410 1.00000000 [omap-get-vals-by-keys out=589b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+589 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.099+0000 7fe45dd097c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-05-20T01:28:50.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.099+0000 7fe45dd097c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-05-20T01:28:50.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.099+0000 7fe45dd097c0 20 datascan.get_frag_of: frag is 1* 2023-05-20T01:28:50.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.099+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:411 18.1b 18:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef7880 con 0x5594efda9800 2023-05-20T01:28:50.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.099+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 161 ==== osd_op_reply(411 10000000000.01800000 [omap-get-header out=274b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.099+0000 7fe45dd097c0 10 datascan.find_or_create_dirfrag: read_version = 3 2023-05-20T01:28:50.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.099+0000 7fe45dd097c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-05-20T01:28:50.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.099+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:412 18.1b 18:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091e700 con 0x5594efda9800 2023-05-20T01:28:50.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.099+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 162 ==== osd_op_reply(412 10000000000.01800000 [omap-get-vals-by-keys out=577b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+577 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.099+0000 7fe45dd097c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=78 frag=1*, r=0 2023-05-20T01:28:50.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.099+0000 7fe45dd097c0 20 datascan.inject_with_backtrace: Dentry 0x0x10000000000/78 already exists and points to me 2023-05-20T01:28:50.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.099+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:413 19.15 19:a8000000::::head [pgnls-filter start_epoch 138 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091fc00 con 0x5594efda8c00 2023-05-20T01:28:50.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.100+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 82 ==== osd_op_reply(413 [pgnls-filter start_epoch 138 out=81b] v138'9 uv9 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.100+0000 7fe45dd097c0 20 datascan.forall_objects: OSD matched oid 1000000020d.00000000 2023-05-20T01:28:50.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.100+0000 7fe45dd097c0 10 datascan.operator(): handling object 1000000020d.0 2023-05-20T01:28:50.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.100+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:414 19.15 19:a8e69ea8:::1000000020d.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091f180 con 0x5594efda8c00 2023-05-20T01:28:50.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.100+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 83 ==== osd_op_reply(414 1000000020d.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv9 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.100+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: inode: 0x0x1000000020d 2023-05-20T01:28:50.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.100+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/24 2023-05-20T01:28:50.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.100+0000 7fe45dd097c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=24 2023-05-20T01:28:50.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.100+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:415 18.10 18:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff40a80 con 0x5594efda8800 2023-05-20T01:28:50.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.100+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 80 ==== osd_op_reply(415 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.100+0000 7fe45dd097c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-05-20T01:28:50.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.100+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:416 18.1f 18:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff40380 con 0x5594efda8c00 2023-05-20T01:28:50.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.101+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 84 ==== osd_op_reply(416 1.00000000 [omap-get-vals-by-keys out=589b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+589 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.101+0000 7fe45dd097c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-05-20T01:28:50.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.101+0000 7fe45dd097c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-05-20T01:28:50.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.101+0000 7fe45dd097c0 20 datascan.get_frag_of: frag is 1* 2023-05-20T01:28:50.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.101+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:417 18.1b 18:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff41500 con 0x5594efda9800 2023-05-20T01:28:50.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.101+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 163 ==== osd_op_reply(417 10000000000.01800000 [omap-get-header out=274b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.101+0000 7fe45dd097c0 10 datascan.find_or_create_dirfrag: read_version = 3 2023-05-20T01:28:50.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.101+0000 7fe45dd097c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-05-20T01:28:50.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.101+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:418 18.1b 18:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff41180 con 0x5594efda9800 2023-05-20T01:28:50.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.101+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 164 ==== osd_op_reply(418 10000000000.01800000 [omap-get-vals-by-keys out=577b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+577 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.101+0000 7fe45dd097c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=24 frag=1*, r=0 2023-05-20T01:28:50.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.101+0000 7fe45dd097c0 20 datascan.inject_with_backtrace: Dentry 0x0x10000000000/24 already exists and points to me 2023-05-20T01:28:50.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.101+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:419 19.35 19:ac000000::::head [pgnls-filter start_epoch 138 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff40000 con 0x5594eff00000 2023-05-20T01:28:50.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.101+0000 7fe45bcb7700 1 -- 172.21.15.16:0/1888659255 <== osd.0 v2:172.21.15.16:6824/3923634179 17 ==== osd_op_reply(419 [pgnls-filter start_epoch 138 out=49b] v136'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5594efda18c0 con 0x5594eff00000 2023-05-20T01:28:50.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.101+0000 7fe456cad700 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:420 19.d 19:b0000000::::head [pgnls-filter start_epoch 138 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff36e00 con 0x5594efda9800 2023-05-20T01:28:50.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.102+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 165 ==== osd_op_reply(420 [pgnls-filter start_epoch 138 out=49b] v136'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.102+0000 7fe4574ae700 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:421 19.2d 19:b4000000::::head [pgnls-filter start_epoch 138 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff40700 con 0x5594eff00000 2023-05-20T01:28:50.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.102+0000 7fe45bcb7700 1 -- 172.21.15.16:0/1888659255 <== osd.0 v2:172.21.15.16:6824/3923634179 18 ==== osd_op_reply(421 [pgnls-filter start_epoch 138 out=81b] v138'10 uv10 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5594efda18c0 con 0x5594eff00000 2023-05-20T01:28:50.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.102+0000 7fe45dd097c0 20 datascan.forall_objects: OSD matched oid 10000000218.00000000 2023-05-20T01:28:50.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.102+0000 7fe45dd097c0 10 datascan.operator(): handling object 10000000218.0 2023-05-20T01:28:50.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.102+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:422 19.2d 19:b79a54f6:::10000000218.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff41880 con 0x5594eff00000 2023-05-20T01:28:50.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.102+0000 7fe45bcb7700 1 -- 172.21.15.16:0/1888659255 <== osd.0 v2:172.21.15.16:6824/3923634179 19 ==== osd_op_reply(422 10000000218.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv10 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x5594efda18c0 con 0x5594eff00000 2023-05-20T01:28:50.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.102+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000218 2023-05-20T01:28:50.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.102+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/35 2023-05-20T01:28:50.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.102+0000 7fe45dd097c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=35 2023-05-20T01:28:50.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.102+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:423 18.10 18:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff37500 con 0x5594efda8800 2023-05-20T01:28:50.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.103+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 81 ==== osd_op_reply(423 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.103+0000 7fe45dd097c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-05-20T01:28:50.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.103+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:424 18.1f 18:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff36380 con 0x5594efda8c00 2023-05-20T01:28:50.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.103+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 85 ==== osd_op_reply(424 1.00000000 [omap-get-vals-by-keys out=589b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+589 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.103+0000 7fe45dd097c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-05-20T01:28:50.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.103+0000 7fe45dd097c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-05-20T01:28:50.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.103+0000 7fe45dd097c0 20 datascan.get_frag_of: frag is 1* 2023-05-20T01:28:50.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.103+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:425 18.1b 18:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff49500 con 0x5594efda9800 2023-05-20T01:28:50.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.104+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 166 ==== osd_op_reply(425 10000000000.01800000 [omap-get-header out=274b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.104+0000 7fe45dd097c0 10 datascan.find_or_create_dirfrag: read_version = 3 2023-05-20T01:28:50.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.104+0000 7fe45dd097c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-05-20T01:28:50.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.104+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:426 18.1b 18:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff48e00 con 0x5594efda9800 2023-05-20T01:28:50.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.104+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 167 ==== osd_op_reply(426 10000000000.01800000 [omap-get-vals-by-keys out=577b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+577 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.104+0000 7fe45dd097c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=35 frag=1*, r=0 2023-05-20T01:28:50.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.104+0000 7fe45dd097c0 20 datascan.inject_with_backtrace: Dentry 0x0x10000000000/35 already exists and points to me 2023-05-20T01:28:50.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.104+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:427 19.1d 19:b8000000::::head [pgnls-filter start_epoch 138 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff5a000 con 0x5594efda8c00 2023-05-20T01:28:50.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.105+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 86 ==== osd_op_reply(427 [pgnls-filter start_epoch 138 out=81b] v138'18 uv18 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.105+0000 7fe45dd097c0 20 datascan.forall_objects: OSD matched oid 10000000246.00000000 2023-05-20T01:28:50.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.105+0000 7fe45dd097c0 10 datascan.operator(): handling object 10000000246.0 2023-05-20T01:28:50.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.105+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:428 19.1d 19:bab97ecf:::10000000246.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff5a380 con 0x5594efda8c00 2023-05-20T01:28:50.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.105+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 87 ==== osd_op_reply(428 10000000246.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv18 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.105+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000246 2023-05-20T01:28:50.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.105+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/81 2023-05-20T01:28:50.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.105+0000 7fe45dd097c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=81 2023-05-20T01:28:50.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.105+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:429 18.10 18:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff5aa80 con 0x5594efda8800 2023-05-20T01:28:50.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.106+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 82 ==== osd_op_reply(429 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.106+0000 7fe45dd097c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-05-20T01:28:50.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.106+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:430 18.1f 18:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff5ae00 con 0x5594efda8c00 2023-05-20T01:28:50.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.106+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 88 ==== osd_op_reply(430 1.00000000 [omap-get-vals-by-keys out=589b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+589 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.106+0000 7fe45dd097c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-05-20T01:28:50.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.106+0000 7fe45dd097c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-05-20T01:28:50.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.106+0000 7fe45dd097c0 20 datascan.get_frag_of: frag is 1* 2023-05-20T01:28:50.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.106+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:431 18.1b 18:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff5b180 con 0x5594efda9800 2023-05-20T01:28:50.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.106+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 168 ==== osd_op_reply(431 10000000000.01800000 [omap-get-header out=274b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.106+0000 7fe45dd097c0 10 datascan.find_or_create_dirfrag: read_version = 3 2023-05-20T01:28:50.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.106+0000 7fe45dd097c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-05-20T01:28:50.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.106+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:432 18.1b 18:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff5b500 con 0x5594efda9800 2023-05-20T01:28:50.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.106+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 169 ==== osd_op_reply(432 10000000000.01800000 [omap-get-vals-by-keys out=577b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+577 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.106+0000 7fe45dd097c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=81 frag=1*, r=0 2023-05-20T01:28:50.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.106+0000 7fe45dd097c0 20 datascan.inject_with_backtrace: Dentry 0x0x10000000000/81 already exists and points to me 2023-05-20T01:28:50.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.106+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:433 19.3d 19:bc000000::::head [pgnls-filter start_epoch 138 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff5b880 con 0x5594efda8800 2023-05-20T01:28:50.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.107+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 83 ==== osd_op_reply(433 [pgnls-filter start_epoch 138 out=101b] v138'13 uv13 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.107+0000 7fe45dd097c0 20 datascan.forall_objects: OSD matched oid 10000000219.00000000 2023-05-20T01:28:50.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.107+0000 7fe45dd097c0 10 datascan.operator(): handling object 10000000219.0 2023-05-20T01:28:50.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.107+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:434 19.3d 19:bc0c3701:::10000000219.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff5bc00 con 0x5594efda8800 2023-05-20T01:28:50.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.107+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 84 ==== osd_op_reply(434 10000000219.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv12 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.107+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000219 2023-05-20T01:28:50.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.107+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/36 2023-05-20T01:28:50.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.107+0000 7fe45dd097c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=36 2023-05-20T01:28:50.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.107+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:435 18.10 18:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff5b180 con 0x5594efda8800 2023-05-20T01:28:50.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.107+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 85 ==== osd_op_reply(435 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.107+0000 7fe45dd097c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-05-20T01:28:50.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.107+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:436 18.1f 18:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff5b500 con 0x5594efda8c00 2023-05-20T01:28:50.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.108+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 89 ==== osd_op_reply(436 1.00000000 [omap-get-vals-by-keys out=589b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+589 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.108+0000 7fe45dd097c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-05-20T01:28:50.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.108+0000 7fe45dd097c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-05-20T01:28:50.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.108+0000 7fe45dd097c0 20 datascan.get_frag_of: frag is 0* 2023-05-20T01:28:50.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.108+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:437 18.5 18:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff49180 con 0x5594efda9800 2023-05-20T01:28:50.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.108+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 170 ==== osd_op_reply(437 10000000000.01000000 [omap-get-header out=274b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.108+0000 7fe45dd097c0 10 datascan.find_or_create_dirfrag: read_version = 4 2023-05-20T01:28:50.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.108+0000 7fe45dd097c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-05-20T01:28:50.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.108+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:438 18.5 18:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff49880 con 0x5594efda9800 2023-05-20T01:28:50.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.108+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 171 ==== osd_op_reply(438 10000000000.01000000 [omap-get-vals-by-keys out=577b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+577 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.108+0000 7fe45dd097c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=36 frag=0*, r=0 2023-05-20T01:28:50.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.108+0000 7fe45dd097c0 20 datascan.inject_with_backtrace: Dentry 0x0x10000000000/36 already exists and points to me 2023-05-20T01:28:50.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.108+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:439 19.3d 19:bfcd3380::::head [pgnls-filter start_epoch 138 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff49500 con 0x5594efda8800 2023-05-20T01:28:50.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.108+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 86 ==== osd_op_reply(439 [pgnls-filter start_epoch 138 out=81b] v138'13 uv13 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.108+0000 7fe45dd097c0 20 datascan.forall_objects: OSD matched oid 10000000205.00000000 2023-05-20T01:28:50.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.108+0000 7fe45dd097c0 10 datascan.operator(): handling object 10000000205.0 2023-05-20T01:28:50.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.108+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:440 19.3d 19:bfcd3380:::10000000205.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff48e00 con 0x5594efda8800 2023-05-20T01:28:50.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.109+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 87 ==== osd_op_reply(440 10000000205.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv13 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.109+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000205 2023-05-20T01:28:50.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.109+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/16 2023-05-20T01:28:50.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.109+0000 7fe45dd097c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=16 2023-05-20T01:28:50.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.109+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:441 18.10 18:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff37500 con 0x5594efda8800 2023-05-20T01:28:50.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.109+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 88 ==== osd_op_reply(441 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.109+0000 7fe45dd097c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-05-20T01:28:50.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.109+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:442 18.1f 18:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff40e00 con 0x5594efda8c00 2023-05-20T01:28:50.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.109+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 90 ==== osd_op_reply(442 1.00000000 [omap-get-vals-by-keys out=589b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+589 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.109+0000 7fe45dd097c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-05-20T01:28:50.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.109+0000 7fe45dd097c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-05-20T01:28:50.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.109+0000 7fe45dd097c0 20 datascan.get_frag_of: frag is 1* 2023-05-20T01:28:50.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.109+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:443 18.1b 18:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff41c00 con 0x5594efda9800 2023-05-20T01:28:50.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.110+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 172 ==== osd_op_reply(443 10000000000.01800000 [omap-get-header out=274b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.110+0000 7fe45dd097c0 10 datascan.find_or_create_dirfrag: read_version = 3 2023-05-20T01:28:50.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.110+0000 7fe45dd097c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-05-20T01:28:50.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.110+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:444 18.1b 18:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff40a80 con 0x5594efda9800 2023-05-20T01:28:50.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.110+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 173 ==== osd_op_reply(444 10000000000.01800000 [omap-get-vals-by-keys out=577b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+577 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.110+0000 7fe45dd097c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=16 frag=1*, r=0 2023-05-20T01:28:50.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.110+0000 7fe45dd097c0 20 datascan.inject_with_backtrace: Dentry 0x0x10000000000/16 already exists and points to me 2023-05-20T01:28:50.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.110+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:445 19.3 19:c0000000::::head [pgnls-filter start_epoch 138 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff41500 con 0x5594eff01000 2023-05-20T01:28:50.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.110+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.5 v2:172.21.15.132:6816/1844689995 5 ==== osd_op_reply(445 [pgnls-filter start_epoch 138 out=49b] v136'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5594efda1440 con 0x5594eff01000 2023-05-20T01:28:50.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.111+0000 7fe456cad700 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:446 19.23 19:c4000000::::head [pgnls-filter start_epoch 138 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff37180 con 0x5594efdd9000 2023-05-20T01:28:50.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.111+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.2 v2:172.21.15.16:6816/293187030 21 ==== osd_op_reply(446 [pgnls-filter start_epoch 138 out=49b] v136'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5594efda1680 con 0x5594efdd9000 2023-05-20T01:28:50.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.111+0000 7fe4574ae700 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:447 19.13 19:c8000000::::head [pgnls-filter start_epoch 138 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef6000 con 0x5594efdd9000 2023-05-20T01:28:50.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.111+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.2 v2:172.21.15.16:6816/293187030 22 ==== osd_op_reply(447 [pgnls-filter start_epoch 138 out=101b] v138'30 uv30 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x5594efda1680 con 0x5594efdd9000 2023-05-20T01:28:50.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.111+0000 7fe45dd097c0 20 datascan.forall_objects: OSD matched oid 1000000023c.00000000 2023-05-20T01:28:50.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.111+0000 7fe45dd097c0 10 datascan.operator(): handling object 1000000023c.0 2023-05-20T01:28:50.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.111+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:448 19.13 19:c8893608:::1000000023c.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff41180 con 0x5594efdd9000 2023-05-20T01:28:50.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.111+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.2 v2:172.21.15.16:6816/293187030 23 ==== osd_op_reply(448 1000000023c.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv26 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x5594efda1680 con 0x5594efdd9000 2023-05-20T01:28:50.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.111+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: inode: 0x0x1000000023c 2023-05-20T01:28:50.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.111+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/71 2023-05-20T01:28:50.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.111+0000 7fe45dd097c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=71 2023-05-20T01:28:50.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.111+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:449 18.10 18:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091f500 con 0x5594efda8800 2023-05-20T01:28:50.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.112+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 89 ==== osd_op_reply(449 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.112+0000 7fe45dd097c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-05-20T01:28:50.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.112+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:450 18.1f 18:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091ee00 con 0x5594efda8c00 2023-05-20T01:28:50.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.112+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 91 ==== osd_op_reply(450 1.00000000 [omap-get-vals-by-keys out=589b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+589 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.112+0000 7fe45dd097c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-05-20T01:28:50.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.112+0000 7fe45dd097c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-05-20T01:28:50.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.112+0000 7fe45dd097c0 20 datascan.get_frag_of: frag is 1* 2023-05-20T01:28:50.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.112+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:451 18.1b 18:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091e700 con 0x5594efda9800 2023-05-20T01:28:50.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.112+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 174 ==== osd_op_reply(451 10000000000.01800000 [omap-get-header out=274b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.112+0000 7fe45dd097c0 10 datascan.find_or_create_dirfrag: read_version = 3 2023-05-20T01:28:50.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.112+0000 7fe45dd097c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-05-20T01:28:50.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.112+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:452 18.1b 18:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef7c00 con 0x5594efda9800 2023-05-20T01:28:50.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.113+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 175 ==== osd_op_reply(452 10000000000.01800000 [omap-get-vals-by-keys out=577b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+577 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.113+0000 7fe45dd097c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=71 frag=1*, r=0 2023-05-20T01:28:50.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.113+0000 7fe45dd097c0 20 datascan.inject_with_backtrace: Dentry 0x0x10000000000/71 already exists and points to me 2023-05-20T01:28:50.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.113+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:453 19.13 19:c933d8fa::::head [pgnls-filter start_epoch 138 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef6a80 con 0x5594efdd9000 2023-05-20T01:28:50.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.113+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.2 v2:172.21.15.16:6816/293187030 24 ==== osd_op_reply(453 [pgnls-filter start_epoch 138 out=101b] v138'30 uv30 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x5594efda1680 con 0x5594efdd9000 2023-05-20T01:28:50.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.113+0000 7fe45dd097c0 20 datascan.forall_objects: OSD matched oid 10000000241.00000000 2023-05-20T01:28:50.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.113+0000 7fe45dd097c0 10 datascan.operator(): handling object 10000000241.0 2023-05-20T01:28:50.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.113+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:454 19.13 19:c933d8fa:::10000000241.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef7500 con 0x5594efdd9000 2023-05-20T01:28:50.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.113+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.2 v2:172.21.15.16:6816/293187030 25 ==== osd_op_reply(454 10000000241.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv27 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x5594efda1680 con 0x5594efdd9000 2023-05-20T01:28:50.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.113+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000241 2023-05-20T01:28:50.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.113+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/76 2023-05-20T01:28:50.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.113+0000 7fe45dd097c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=76 2023-05-20T01:28:50.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.113+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:455 18.10 18:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef7880 con 0x5594efda8800 2023-05-20T01:28:50.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.114+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 90 ==== osd_op_reply(455 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.114+0000 7fe45dd097c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-05-20T01:28:50.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.114+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:456 18.1f 18:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff36a80 con 0x5594efda8c00 2023-05-20T01:28:50.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.114+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 92 ==== osd_op_reply(456 1.00000000 [omap-get-vals-by-keys out=589b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+589 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.114+0000 7fe45dd097c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-05-20T01:28:50.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.114+0000 7fe45dd097c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-05-20T01:28:50.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.114+0000 7fe45dd097c0 20 datascan.get_frag_of: frag is 1* 2023-05-20T01:28:50.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.114+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:457 18.1b 18:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff37880 con 0x5594efda9800 2023-05-20T01:28:50.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.114+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 176 ==== osd_op_reply(457 10000000000.01800000 [omap-get-header out=274b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.114+0000 7fe45dd097c0 10 datascan.find_or_create_dirfrag: read_version = 3 2023-05-20T01:28:50.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.114+0000 7fe45dd097c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-05-20T01:28:50.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.114+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:458 18.1b 18:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff37c00 con 0x5594efda9800 2023-05-20T01:28:50.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.114+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 177 ==== osd_op_reply(458 10000000000.01800000 [omap-get-vals-by-keys out=577b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+577 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.114+0000 7fe45dd097c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=76 frag=1*, r=0 2023-05-20T01:28:50.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.114+0000 7fe45dd097c0 20 datascan.inject_with_backtrace: Dentry 0x0x10000000000/76 already exists and points to me 2023-05-20T01:28:50.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.114+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:459 19.13 19:c9855970::::head [pgnls-filter start_epoch 138 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff36380 con 0x5594efdd9000 2023-05-20T01:28:50.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.115+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.2 v2:172.21.15.16:6816/293187030 26 ==== osd_op_reply(459 [pgnls-filter start_epoch 138 out=101b] v138'30 uv30 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x5594efda1680 con 0x5594efdd9000 2023-05-20T01:28:50.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.115+0000 7fe45dd097c0 20 datascan.forall_objects: OSD matched oid 10000000239.00000000 2023-05-20T01:28:50.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.115+0000 7fe45dd097c0 10 datascan.operator(): handling object 10000000239.0 2023-05-20T01:28:50.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.115+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:460 19.13 19:c9855970:::10000000239.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff37180 con 0x5594efdd9000 2023-05-20T01:28:50.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.115+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.2 v2:172.21.15.16:6816/293187030 27 ==== osd_op_reply(460 10000000239.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv28 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x5594efda1680 con 0x5594efdd9000 2023-05-20T01:28:50.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.115+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000239 2023-05-20T01:28:50.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.115+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/68 2023-05-20T01:28:50.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.115+0000 7fe45dd097c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=68 2023-05-20T01:28:50.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.115+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:461 18.10 18:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff40e00 con 0x5594efda8800 2023-05-20T01:28:50.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.115+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 91 ==== osd_op_reply(461 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.115+0000 7fe45dd097c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-05-20T01:28:50.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.115+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:462 18.1f 18:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff5a000 con 0x5594efda8c00 2023-05-20T01:28:50.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.116+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 93 ==== osd_op_reply(462 1.00000000 [omap-get-vals-by-keys out=589b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+589 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.116+0000 7fe45dd097c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-05-20T01:28:50.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.116+0000 7fe45dd097c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-05-20T01:28:50.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.116+0000 7fe45dd097c0 20 datascan.get_frag_of: frag is 1* 2023-05-20T01:28:50.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.116+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:463 18.1b 18:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff5a700 con 0x5594efda9800 2023-05-20T01:28:50.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.116+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 178 ==== osd_op_reply(463 10000000000.01800000 [omap-get-header out=274b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.116+0000 7fe45dd097c0 10 datascan.find_or_create_dirfrag: read_version = 3 2023-05-20T01:28:50.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.116+0000 7fe45dd097c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-05-20T01:28:50.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.116+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:464 18.1b 18:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff5a380 con 0x5594efda9800 2023-05-20T01:28:50.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.116+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 179 ==== osd_op_reply(464 10000000000.01800000 [omap-get-vals-by-keys out=577b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+577 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.116+0000 7fe45dd097c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=68 frag=1*, r=0 2023-05-20T01:28:50.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.116+0000 7fe45dd097c0 20 datascan.inject_with_backtrace: Dentry 0x0x10000000000/68 already exists and points to me 2023-05-20T01:28:50.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.116+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:465 19.13 19:ca8fdbba::::head [pgnls-filter start_epoch 138 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff5ae00 con 0x5594efdd9000 2023-05-20T01:28:50.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.116+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.2 v2:172.21.15.16:6816/293187030 28 ==== osd_op_reply(465 [pgnls-filter start_epoch 138 out=101b] v138'30 uv30 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x5594efda1680 con 0x5594efdd9000 2023-05-20T01:28:50.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.116+0000 7fe45dd097c0 20 datascan.forall_objects: OSD matched oid 100000001fb.00000000 2023-05-20T01:28:50.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.116+0000 7fe45dd097c0 10 datascan.operator(): handling object 100000001fb.0 2023-05-20T01:28:50.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.116+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:466 19.13 19:ca8fdbba:::100000001fb.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff5b500 con 0x5594efdd9000 2023-05-20T01:28:50.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.117+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.2 v2:172.21.15.16:6816/293187030 29 ==== osd_op_reply(466 100000001fb.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (89) out=89b,getxattr (30) out=30b,getxattr] v0'0 uv29 ondisk = 0) v8 ==== 416+0+157 (crc 0 0 0) 0x5594efda1680 con 0x5594efdd9000 2023-05-20T01:28:50.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.117+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: inode: 0x0x100000001fb 2023-05-20T01:28:50.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.117+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/6 2023-05-20T01:28:50.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.117+0000 7fe45dd097c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=6 2023-05-20T01:28:50.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.117+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:467 18.10 18:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091f880 con 0x5594efda8800 2023-05-20T01:28:50.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.117+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 92 ==== osd_op_reply(467 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.117+0000 7fe45dd097c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-05-20T01:28:50.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.117+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:468 18.1f 18:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091f180 con 0x5594efda8c00 2023-05-20T01:28:50.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.117+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 94 ==== osd_op_reply(468 1.00000000 [omap-get-vals-by-keys out=589b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+589 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.117+0000 7fe45dd097c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-05-20T01:28:50.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.117+0000 7fe45dd097c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-05-20T01:28:50.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.117+0000 7fe45dd097c0 20 datascan.get_frag_of: frag is 1* 2023-05-20T01:28:50.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.117+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:469 18.1b 18:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff60000 con 0x5594efda9800 2023-05-20T01:28:50.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.118+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 180 ==== osd_op_reply(469 10000000000.01800000 [omap-get-header out=274b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.118+0000 7fe45dd097c0 10 datascan.find_or_create_dirfrag: read_version = 3 2023-05-20T01:28:50.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.118+0000 7fe45dd097c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-05-20T01:28:50.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.118+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:470 18.1b 18:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff60380 con 0x5594efda9800 2023-05-20T01:28:50.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.118+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 181 ==== osd_op_reply(470 10000000000.01800000 [omap-get-vals-by-keys out=576b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+576 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.118+0000 7fe45dd097c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=6 frag=1*, r=0 2023-05-20T01:28:50.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.118+0000 7fe45dd097c0 20 datascan.inject_with_backtrace: Dentry 0x0x10000000000/6 already exists and points to me 2023-05-20T01:28:50.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.118+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:471 19.13 19:cbcb6224::::head [pgnls-filter start_epoch 138 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff60700 con 0x5594efdd9000 2023-05-20T01:28:50.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.118+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.2 v2:172.21.15.16:6816/293187030 30 ==== osd_op_reply(471 [pgnls-filter start_epoch 138 out=81b] v138'30 uv30 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5594efda1680 con 0x5594efdd9000 2023-05-20T01:28:50.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.118+0000 7fe45dd097c0 20 datascan.forall_objects: OSD matched oid 10000000212.00000000 2023-05-20T01:28:50.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.118+0000 7fe45dd097c0 10 datascan.operator(): handling object 10000000212.0 2023-05-20T01:28:50.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.118+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:472 19.13 19:cbcb6224:::10000000212.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff60a80 con 0x5594efdd9000 2023-05-20T01:28:50.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.118+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.2 v2:172.21.15.16:6816/293187030 31 ==== osd_op_reply(472 10000000212.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv30 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x5594efda1680 con 0x5594efdd9000 2023-05-20T01:28:50.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.118+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000212 2023-05-20T01:28:50.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.118+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/29 2023-05-20T01:28:50.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.118+0000 7fe45dd097c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=29 2023-05-20T01:28:50.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.118+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:473 18.10 18:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff61180 con 0x5594efda8800 2023-05-20T01:28:50.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.119+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 93 ==== osd_op_reply(473 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.119+0000 7fe45dd097c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-05-20T01:28:50.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.119+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:474 18.1f 18:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff61500 con 0x5594efda8c00 2023-05-20T01:28:50.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.119+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 95 ==== osd_op_reply(474 1.00000000 [omap-get-vals-by-keys out=589b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+589 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.119+0000 7fe45dd097c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-05-20T01:28:50.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.119+0000 7fe45dd097c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-05-20T01:28:50.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.119+0000 7fe45dd097c0 20 datascan.get_frag_of: frag is 1* 2023-05-20T01:28:50.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.119+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:475 18.1b 18:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff61880 con 0x5594efda9800 2023-05-20T01:28:50.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.119+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 182 ==== osd_op_reply(475 10000000000.01800000 [omap-get-header out=274b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.119+0000 7fe45dd097c0 10 datascan.find_or_create_dirfrag: read_version = 3 2023-05-20T01:28:50.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.119+0000 7fe45dd097c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-05-20T01:28:50.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.119+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:476 18.1b 18:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff61c00 con 0x5594efda9800 2023-05-20T01:28:50.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.120+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 183 ==== osd_op_reply(476 10000000000.01800000 [omap-get-vals-by-keys out=577b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+577 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.120+0000 7fe45dd097c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=29 frag=1*, r=0 2023-05-20T01:28:50.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.120+0000 7fe45dd097c0 20 datascan.inject_with_backtrace: Dentry 0x0x10000000000/29 already exists and points to me 2023-05-20T01:28:50.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.120+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:477 19.33 19:cc000000::::head [pgnls-filter start_epoch 138 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff60700 con 0x5594efdd9000 2023-05-20T01:28:50.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.120+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.2 v2:172.21.15.16:6816/293187030 32 ==== osd_op_reply(477 [pgnls-filter start_epoch 138 out=81b] v138'13 uv13 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5594efda1680 con 0x5594efdd9000 2023-05-20T01:28:50.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.120+0000 7fe45dd097c0 20 datascan.forall_objects: OSD matched oid 10000000248.00000000 2023-05-20T01:28:50.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.120+0000 7fe45dd097c0 10 datascan.operator(): handling object 10000000248.0 2023-05-20T01:28:50.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.120+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:478 19.33 19:cf9e7a4f:::10000000248.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff60e00 con 0x5594efdd9000 2023-05-20T01:28:50.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.120+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.2 v2:172.21.15.16:6816/293187030 33 ==== osd_op_reply(478 10000000248.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv13 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x5594efda1680 con 0x5594efdd9000 2023-05-20T01:28:50.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.120+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000248 2023-05-20T01:28:50.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.120+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/83 2023-05-20T01:28:50.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.120+0000 7fe45dd097c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=83 2023-05-20T01:28:50.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.120+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:479 18.10 18:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091ee00 con 0x5594efda8800 2023-05-20T01:28:50.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.120+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 94 ==== osd_op_reply(479 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.120+0000 7fe45dd097c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-05-20T01:28:50.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.120+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:480 18.1f 18:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091fc00 con 0x5594efda8c00 2023-05-20T01:28:50.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.121+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 96 ==== osd_op_reply(480 1.00000000 [omap-get-vals-by-keys out=589b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+589 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.121+0000 7fe45dd097c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-05-20T01:28:50.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.121+0000 7fe45dd097c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-05-20T01:28:50.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.121+0000 7fe45dd097c0 20 datascan.get_frag_of: frag is 0* 2023-05-20T01:28:50.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.121+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:481 18.5 18:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091f180 con 0x5594efda9800 2023-05-20T01:28:50.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.121+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 184 ==== osd_op_reply(481 10000000000.01000000 [omap-get-header out=274b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.121+0000 7fe45dd097c0 10 datascan.find_or_create_dirfrag: read_version = 4 2023-05-20T01:28:50.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.121+0000 7fe45dd097c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-05-20T01:28:50.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.121+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:482 18.5 18:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091f500 con 0x5594efda9800 2023-05-20T01:28:50.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.121+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 185 ==== osd_op_reply(482 10000000000.01000000 [omap-get-vals-by-keys out=577b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+577 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.121+0000 7fe45dd097c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=83 frag=0*, r=0 2023-05-20T01:28:50.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.121+0000 7fe45dd097c0 20 datascan.inject_with_backtrace: Dentry 0x0x10000000000/83 already exists and points to me 2023-05-20T01:28:50.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.121+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:483 19.b 19:d0000000::::head [pgnls-filter start_epoch 138 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091e700 con 0x5594efda9c00 2023-05-20T01:28:50.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.122+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.4 v2:172.21.15.132:6817/506317223 12 ==== osd_op_reply(483 [pgnls-filter start_epoch 138 out=101b] v138'17 uv17 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x5594efda1680 con 0x5594efda9c00 2023-05-20T01:28:50.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.122+0000 7fe45dd097c0 20 datascan.forall_objects: OSD matched oid 10000000225.00000000 2023-05-20T01:28:50.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.122+0000 7fe45dd097c0 10 datascan.operator(): handling object 10000000225.0 2023-05-20T01:28:50.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.122+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:484 19.b 19:d0d26e50:::10000000225.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091f880 con 0x5594efda9c00 2023-05-20T01:28:50.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.122+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.4 v2:172.21.15.132:6817/506317223 13 ==== osd_op_reply(484 10000000225.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv15 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x5594efda1680 con 0x5594efda9c00 2023-05-20T01:28:50.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.122+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000225 2023-05-20T01:28:50.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.122+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/48 2023-05-20T01:28:50.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.122+0000 7fe45dd097c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=48 2023-05-20T01:28:50.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.122+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:485 18.10 18:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff40380 con 0x5594efda8800 2023-05-20T01:28:50.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.122+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 95 ==== osd_op_reply(485 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.122+0000 7fe45dd097c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-05-20T01:28:50.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.122+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:486 18.1f 18:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff41c00 con 0x5594efda8c00 2023-05-20T01:28:50.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.123+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 97 ==== osd_op_reply(486 1.00000000 [omap-get-vals-by-keys out=589b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+589 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.123+0000 7fe45dd097c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-05-20T01:28:50.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.123+0000 7fe45dd097c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-05-20T01:28:50.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.123+0000 7fe45dd097c0 20 datascan.get_frag_of: frag is 1* 2023-05-20T01:28:50.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.123+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:487 18.1b 18:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff40a80 con 0x5594efda9800 2023-05-20T01:28:50.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.123+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 186 ==== osd_op_reply(487 10000000000.01800000 [omap-get-header out=274b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.123+0000 7fe45dd097c0 10 datascan.find_or_create_dirfrag: read_version = 3 2023-05-20T01:28:50.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.123+0000 7fe45dd097c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-05-20T01:28:50.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.123+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:488 18.1b 18:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff41500 con 0x5594efda9800 2023-05-20T01:28:50.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.123+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 187 ==== osd_op_reply(488 10000000000.01800000 [omap-get-vals-by-keys out=577b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+577 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.123+0000 7fe45dd097c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=48 frag=1*, r=0 2023-05-20T01:28:50.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.123+0000 7fe45dd097c0 20 datascan.inject_with_backtrace: Dentry 0x0x10000000000/48 already exists and points to me 2023-05-20T01:28:50.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.123+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:489 19.b 19:d2dbd4ac::::head [pgnls-filter start_epoch 138 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff40e00 con 0x5594efda9c00 2023-05-20T01:28:50.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.123+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.4 v2:172.21.15.132:6817/506317223 14 ==== osd_op_reply(489 [pgnls-filter start_epoch 138 out=101b] v138'17 uv17 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x5594efda1680 con 0x5594efda9c00 2023-05-20T01:28:50.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.124+0000 7fe45dd097c0 20 datascan.forall_objects: OSD matched oid 10000000217.00000000 2023-05-20T01:28:50.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.124+0000 7fe45dd097c0 10 datascan.operator(): handling object 10000000217.0 2023-05-20T01:28:50.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.124+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:490 19.b 19:d2dbd4ac:::10000000217.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff36a80 con 0x5594efda9c00 2023-05-20T01:28:50.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.124+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.4 v2:172.21.15.132:6817/506317223 15 ==== osd_op_reply(490 10000000217.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv16 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x5594efda1680 con 0x5594efda9c00 2023-05-20T01:28:50.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.124+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000217 2023-05-20T01:28:50.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.124+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/34 2023-05-20T01:28:50.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.124+0000 7fe45dd097c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=34 2023-05-20T01:28:50.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.124+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:491 18.10 18:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff37180 con 0x5594efda8800 2023-05-20T01:28:50.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.124+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 96 ==== osd_op_reply(491 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.124+0000 7fe45dd097c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-05-20T01:28:50.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.124+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:492 18.1f 18:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff36e00 con 0x5594efda8c00 2023-05-20T01:28:50.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.124+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 98 ==== osd_op_reply(492 1.00000000 [omap-get-vals-by-keys out=589b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+589 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.124+0000 7fe45dd097c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-05-20T01:28:50.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.124+0000 7fe45dd097c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-05-20T01:28:50.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.124+0000 7fe45dd097c0 20 datascan.get_frag_of: frag is 0* 2023-05-20T01:28:50.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.125+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:493 18.5 18:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff37500 con 0x5594efda9800 2023-05-20T01:28:50.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.125+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 188 ==== osd_op_reply(493 10000000000.01000000 [omap-get-header out=274b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.125+0000 7fe45dd097c0 10 datascan.find_or_create_dirfrag: read_version = 4 2023-05-20T01:28:50.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.125+0000 7fe45dd097c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-05-20T01:28:50.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.125+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:494 18.5 18:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff37880 con 0x5594efda9800 2023-05-20T01:28:50.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.125+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 189 ==== osd_op_reply(494 10000000000.01000000 [omap-get-vals-by-keys out=577b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+577 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.125+0000 7fe45dd097c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=34 frag=0*, r=0 2023-05-20T01:28:50.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.125+0000 7fe45dd097c0 20 datascan.inject_with_backtrace: Dentry 0x0x10000000000/34 already exists and points to me 2023-05-20T01:28:50.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.125+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:495 19.b 19:d3eda40d::::head [pgnls-filter start_epoch 138 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff37c00 con 0x5594efda9c00 2023-05-20T01:28:50.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.126+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.4 v2:172.21.15.132:6817/506317223 16 ==== osd_op_reply(495 [pgnls-filter start_epoch 138 out=81b] v138'17 uv17 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5594efda1680 con 0x5594efda9c00 2023-05-20T01:28:50.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.126+0000 7fe45dd097c0 20 datascan.forall_objects: OSD matched oid 1000000020f.00000000 2023-05-20T01:28:50.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.126+0000 7fe45dd097c0 10 datascan.operator(): handling object 1000000020f.0 2023-05-20T01:28:50.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.126+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:496 19.b 19:d3eda40d:::1000000020f.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef6000 con 0x5594efda9c00 2023-05-20T01:28:50.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.126+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.4 v2:172.21.15.132:6817/506317223 17 ==== osd_op_reply(496 1000000020f.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv17 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x5594efda1680 con 0x5594efda9c00 2023-05-20T01:28:50.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.126+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: inode: 0x0x1000000020f 2023-05-20T01:28:50.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.126+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/26 2023-05-20T01:28:50.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.126+0000 7fe45dd097c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=26 2023-05-20T01:28:50.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.126+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:497 18.10 18:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef6e00 con 0x5594efda8800 2023-05-20T01:28:50.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.126+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 97 ==== osd_op_reply(497 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.126+0000 7fe45dd097c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-05-20T01:28:50.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.126+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:498 18.1f 18:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef7500 con 0x5594efda8c00 2023-05-20T01:28:50.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.127+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 99 ==== osd_op_reply(498 1.00000000 [omap-get-vals-by-keys out=589b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+589 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.127+0000 7fe45dd097c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-05-20T01:28:50.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.127+0000 7fe45dd097c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-05-20T01:28:50.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.127+0000 7fe45dd097c0 20 datascan.get_frag_of: frag is 1* 2023-05-20T01:28:50.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.127+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:499 18.1b 18:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef7c00 con 0x5594efda9800 2023-05-20T01:28:50.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.127+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 190 ==== osd_op_reply(499 10000000000.01800000 [omap-get-header out=274b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.127+0000 7fe45dd097c0 10 datascan.find_or_create_dirfrag: read_version = 3 2023-05-20T01:28:50.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.127+0000 7fe45dd097c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-05-20T01:28:50.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.127+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:500 18.1b 18:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef7880 con 0x5594efda9800 2023-05-20T01:28:50.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.127+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 191 ==== osd_op_reply(500 10000000000.01800000 [omap-get-vals-by-keys out=577b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+577 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.127+0000 7fe45dd097c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=26 frag=1*, r=0 2023-05-20T01:28:50.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.127+0000 7fe45dd097c0 20 datascan.inject_with_backtrace: Dentry 0x0x10000000000/26 already exists and points to me 2023-05-20T01:28:50.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.127+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:501 19.2b 19:d4000000::::head [pgnls-filter start_epoch 138 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff49180 con 0x5594efda8c00 2023-05-20T01:28:50.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.128+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 100 ==== osd_op_reply(501 [pgnls-filter start_epoch 138 out=101b] v138'17 uv17 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.128+0000 7fe45dd097c0 20 datascan.forall_objects: OSD matched oid 10000000203.00000000 2023-05-20T01:28:50.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.128+0000 7fe45dd097c0 10 datascan.operator(): handling object 10000000203.0 2023-05-20T01:28:50.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.128+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:502 19.2b 19:d4a58307:::10000000203.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff49880 con 0x5594efda8c00 2023-05-20T01:28:50.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.128+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 101 ==== osd_op_reply(502 10000000203.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv16 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.128+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000203 2023-05-20T01:28:50.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.128+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/14 2023-05-20T01:28:50.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.128+0000 7fe45dd097c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=14 2023-05-20T01:28:50.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.128+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:503 18.10 18:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff48e00 con 0x5594efda8800 2023-05-20T01:28:50.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.128+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 98 ==== osd_op_reply(503 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.128+0000 7fe45dd097c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-05-20T01:28:50.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.128+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:504 18.1f 18:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff60a80 con 0x5594efda8c00 2023-05-20T01:28:50.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.128+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 102 ==== osd_op_reply(504 1.00000000 [omap-get-vals-by-keys out=589b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+589 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.128+0000 7fe45dd097c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-05-20T01:28:50.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.128+0000 7fe45dd097c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-05-20T01:28:50.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.128+0000 7fe45dd097c0 20 datascan.get_frag_of: frag is 1* 2023-05-20T01:28:50.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.128+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:505 18.1b 18:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff61500 con 0x5594efda9800 2023-05-20T01:28:50.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.129+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 192 ==== osd_op_reply(505 10000000000.01800000 [omap-get-header out=274b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.129+0000 7fe45dd097c0 10 datascan.find_or_create_dirfrag: read_version = 3 2023-05-20T01:28:50.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.129+0000 7fe45dd097c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-05-20T01:28:50.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.129+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:506 18.1b 18:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff60000 con 0x5594efda9800 2023-05-20T01:28:50.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.129+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 193 ==== osd_op_reply(506 10000000000.01800000 [omap-get-vals-by-keys out=577b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+577 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.129+0000 7fe45dd097c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=14 frag=1*, r=0 2023-05-20T01:28:50.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.129+0000 7fe45dd097c0 20 datascan.inject_with_backtrace: Dentry 0x0x10000000000/14 already exists and points to me 2023-05-20T01:28:50.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.129+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:507 19.2b 19:d5ee56ee::::head [pgnls-filter start_epoch 138 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff5a000 con 0x5594efda8c00 2023-05-20T01:28:50.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.129+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 103 ==== osd_op_reply(507 [pgnls-filter start_epoch 138 out=81b] v138'17 uv17 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.129+0000 7fe45dd097c0 20 datascan.forall_objects: OSD matched oid 1000000022d.00000000 2023-05-20T01:28:50.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.129+0000 7fe45dd097c0 10 datascan.operator(): handling object 1000000022d.0 2023-05-20T01:28:50.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.129+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:508 19.2b 19:d5ee56ee:::1000000022d.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff5a380 con 0x5594efda8c00 2023-05-20T01:28:50.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.130+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 104 ==== osd_op_reply(508 1000000022d.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv17 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.130+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: inode: 0x0x1000000022d 2023-05-20T01:28:50.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.130+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/56 2023-05-20T01:28:50.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.130+0000 7fe45dd097c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=56 2023-05-20T01:28:50.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.130+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:509 18.10 18:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff5aa80 con 0x5594efda8800 2023-05-20T01:28:50.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.130+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 99 ==== osd_op_reply(509 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.130+0000 7fe45dd097c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-05-20T01:28:50.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.130+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:510 18.1f 18:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff5ae00 con 0x5594efda8c00 2023-05-20T01:28:50.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.130+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 105 ==== osd_op_reply(510 1.00000000 [omap-get-vals-by-keys out=589b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+589 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.131+0000 7fe45dd097c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-05-20T01:28:50.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.131+0000 7fe45dd097c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-05-20T01:28:50.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.131+0000 7fe45dd097c0 20 datascan.get_frag_of: frag is 1* 2023-05-20T01:28:50.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.131+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:511 18.1b 18:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff5b180 con 0x5594efda9800 2023-05-20T01:28:50.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.131+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 194 ==== osd_op_reply(511 10000000000.01800000 [omap-get-header out=274b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.131+0000 7fe45dd097c0 10 datascan.find_or_create_dirfrag: read_version = 3 2023-05-20T01:28:50.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.131+0000 7fe45dd097c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-05-20T01:28:50.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.131+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:512 18.1b 18:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff5b500 con 0x5594efda9800 2023-05-20T01:28:50.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.131+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 195 ==== osd_op_reply(512 10000000000.01800000 [omap-get-vals-by-keys out=577b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+577 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.131+0000 7fe45dd097c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=56 frag=1*, r=0 2023-05-20T01:28:50.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.131+0000 7fe45dd097c0 20 datascan.inject_with_backtrace: Dentry 0x0x10000000000/56 already exists and points to me 2023-05-20T01:28:50.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.131+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:513 19.1b 19:d8000000::::head [pgnls-filter start_epoch 138 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff5b880 con 0x5594efda9c00 2023-05-20T01:28:50.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.132+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.4 v2:172.21.15.132:6817/506317223 18 ==== osd_op_reply(513 [pgnls-filter start_epoch 138 out=49b] v136'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5594efda1680 con 0x5594efda9c00 2023-05-20T01:28:50.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.132+0000 7fe4574ae700 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:514 19.3b 19:dc000000::::head [pgnls-filter start_epoch 138 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef7180 con 0x5594efda8c00 2023-05-20T01:28:50.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.132+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 106 ==== osd_op_reply(514 [pgnls-filter start_epoch 138 out=101b] v138'13 uv13 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.132+0000 7fe45dd097c0 20 datascan.forall_objects: OSD matched oid 10000000249.00000000 2023-05-20T01:28:50.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.132+0000 7fe45dd097c0 10 datascan.operator(): handling object 10000000249.0 2023-05-20T01:28:50.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.132+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:515 19.3b 19:dd50df43:::10000000249.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff5bc00 con 0x5594efda8c00 2023-05-20T01:28:50.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.132+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 107 ==== osd_op_reply(515 10000000249.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv11 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.132+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000249 2023-05-20T01:28:50.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.132+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/84 2023-05-20T01:28:50.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.132+0000 7fe45dd097c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=84 2023-05-20T01:28:50.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.132+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:516 18.10 18:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff60700 con 0x5594efda8800 2023-05-20T01:28:50.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.132+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 100 ==== osd_op_reply(516 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.132+0000 7fe45dd097c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-05-20T01:28:50.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.132+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:517 18.1f 18:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff60e00 con 0x5594efda8c00 2023-05-20T01:28:50.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.133+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 108 ==== osd_op_reply(517 1.00000000 [omap-get-vals-by-keys out=589b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+589 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.133+0000 7fe45dd097c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-05-20T01:28:50.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.133+0000 7fe45dd097c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-05-20T01:28:50.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.133+0000 7fe45dd097c0 20 datascan.get_frag_of: frag is 0* 2023-05-20T01:28:50.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.133+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:518 18.5 18:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff60a80 con 0x5594efda9800 2023-05-20T01:28:50.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.133+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 196 ==== osd_op_reply(518 10000000000.01000000 [omap-get-header out=274b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.133+0000 7fe45dd097c0 10 datascan.find_or_create_dirfrag: read_version = 4 2023-05-20T01:28:50.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.133+0000 7fe45dd097c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-05-20T01:28:50.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.133+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:519 18.5 18:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff49180 con 0x5594efda9800 2023-05-20T01:28:50.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.133+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 197 ==== osd_op_reply(519 10000000000.01000000 [omap-get-vals-by-keys out=577b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+577 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.133+0000 7fe45dd097c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=84 frag=0*, r=0 2023-05-20T01:28:50.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.133+0000 7fe45dd097c0 20 datascan.inject_with_backtrace: Dentry 0x0x10000000000/84 already exists and points to me 2023-05-20T01:28:50.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.133+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:520 19.3b 19:deccbd37::::head [pgnls-filter start_epoch 138 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff49500 con 0x5594efda8c00 2023-05-20T01:28:50.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.134+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 109 ==== osd_op_reply(520 [pgnls-filter start_epoch 138 out=101b] v138'13 uv13 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.134+0000 7fe45dd097c0 20 datascan.forall_objects: OSD matched oid 100000001fe.00000000 2023-05-20T01:28:50.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.134+0000 7fe45dd097c0 10 datascan.operator(): handling object 100000001fe.0 2023-05-20T01:28:50.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.134+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:521 19.3b 19:deccbd37:::100000001fe.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff49880 con 0x5594efda8c00 2023-05-20T01:28:50.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.134+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 110 ==== osd_op_reply(521 100000001fe.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (89) out=89b,getxattr (30) out=30b,getxattr] v0'0 uv12 ondisk = 0) v8 ==== 416+0+157 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.134+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: inode: 0x0x100000001fe 2023-05-20T01:28:50.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.134+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/9 2023-05-20T01:28:50.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.134+0000 7fe45dd097c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=9 2023-05-20T01:28:50.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.134+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:522 18.10 18:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef6000 con 0x5594efda8800 2023-05-20T01:28:50.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.134+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 101 ==== osd_op_reply(522 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.134+0000 7fe45dd097c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-05-20T01:28:50.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.134+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:523 18.1f 18:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef7500 con 0x5594efda8c00 2023-05-20T01:28:50.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.134+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 111 ==== osd_op_reply(523 1.00000000 [omap-get-vals-by-keys out=589b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+589 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.134+0000 7fe45dd097c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-05-20T01:28:50.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.134+0000 7fe45dd097c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-05-20T01:28:50.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.134+0000 7fe45dd097c0 20 datascan.get_frag_of: frag is 0* 2023-05-20T01:28:50.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.134+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:524 18.5 18:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff36380 con 0x5594efda9800 2023-05-20T01:28:50.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.135+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 198 ==== osd_op_reply(524 10000000000.01000000 [omap-get-header out=274b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.135+0000 7fe45dd097c0 10 datascan.find_or_create_dirfrag: read_version = 4 2023-05-20T01:28:50.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.135+0000 7fe45dd097c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-05-20T01:28:50.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.135+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:525 18.5 18:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff36a80 con 0x5594efda9800 2023-05-20T01:28:50.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.135+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 199 ==== osd_op_reply(525 10000000000.01000000 [omap-get-vals-by-keys out=576b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+576 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.135+0000 7fe45dd097c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=9 frag=0*, r=0 2023-05-20T01:28:50.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.135+0000 7fe45dd097c0 20 datascan.inject_with_backtrace: Dentry 0x0x10000000000/9 already exists and points to me 2023-05-20T01:28:50.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.135+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:526 19.3b 19:ded14b84::::head [pgnls-filter start_epoch 138 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff36e00 con 0x5594efda8c00 2023-05-20T01:28:50.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.135+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 112 ==== osd_op_reply(526 [pgnls-filter start_epoch 138 out=81b] v138'13 uv13 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.135+0000 7fe45dd097c0 20 datascan.forall_objects: OSD matched oid 1000000024b.00000000 2023-05-20T01:28:50.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.135+0000 7fe45dd097c0 10 datascan.operator(): handling object 1000000024b.0 2023-05-20T01:28:50.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.135+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:527 19.3b 19:ded14b84:::1000000024b.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff37c00 con 0x5594efda8c00 2023-05-20T01:28:50.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.136+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 113 ==== osd_op_reply(527 1000000024b.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv13 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.136+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: inode: 0x0x1000000024b 2023-05-20T01:28:50.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.136+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/86 2023-05-20T01:28:50.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.136+0000 7fe45dd097c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=86 2023-05-20T01:28:50.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.136+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:528 18.10 18:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff41c00 con 0x5594efda8800 2023-05-20T01:28:50.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.136+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 102 ==== osd_op_reply(528 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.136+0000 7fe45dd097c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-05-20T01:28:50.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.136+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:529 18.1f 18:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff40e00 con 0x5594efda8c00 2023-05-20T01:28:50.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.136+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 114 ==== osd_op_reply(529 1.00000000 [omap-get-vals-by-keys out=589b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+589 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.136+0000 7fe45dd097c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-05-20T01:28:50.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.136+0000 7fe45dd097c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-05-20T01:28:50.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.136+0000 7fe45dd097c0 20 datascan.get_frag_of: frag is 0* 2023-05-20T01:28:50.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.136+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:530 18.5 18:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091ea80 con 0x5594efda9800 2023-05-20T01:28:50.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.136+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 200 ==== osd_op_reply(530 10000000000.01000000 [omap-get-header out=274b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.136+0000 7fe45dd097c0 10 datascan.find_or_create_dirfrag: read_version = 4 2023-05-20T01:28:50.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.136+0000 7fe45dd097c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-05-20T01:28:50.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.136+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:531 18.5 18:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091fc00 con 0x5594efda9800 2023-05-20T01:28:50.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.137+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 201 ==== osd_op_reply(531 10000000000.01000000 [omap-get-vals-by-keys out=577b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+577 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.137+0000 7fe45dd097c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=86 frag=0*, r=0 2023-05-20T01:28:50.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.137+0000 7fe45dd097c0 20 datascan.inject_with_backtrace: Dentry 0x0x10000000000/86 already exists and points to me 2023-05-20T01:28:50.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.137+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:532 19.7 19:e0000000::::head [pgnls-filter start_epoch 138 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091e700 con 0x5594efdd9000 2023-05-20T01:28:50.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.137+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.2 v2:172.21.15.16:6816/293187030 34 ==== osd_op_reply(532 [pgnls-filter start_epoch 138 out=101b] v138'15 uv15 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x5594efda1680 con 0x5594efdd9000 2023-05-20T01:28:50.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.137+0000 7fe45dd097c0 20 datascan.forall_objects: OSD matched oid 10000000207.00000000 2023-05-20T01:28:50.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.137+0000 7fe45dd097c0 10 datascan.operator(): handling object 10000000207.0 2023-05-20T01:28:50.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.137+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:533 19.7 19:e15a914f:::10000000207.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091f880 con 0x5594efdd9000 2023-05-20T01:28:50.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.137+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.2 v2:172.21.15.16:6816/293187030 35 ==== osd_op_reply(533 10000000207.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv13 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x5594efda1680 con 0x5594efdd9000 2023-05-20T01:28:50.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.137+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000207 2023-05-20T01:28:50.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.137+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/18 2023-05-20T01:28:50.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.137+0000 7fe45dd097c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=18 2023-05-20T01:28:50.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.137+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:534 18.10 18:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff49180 con 0x5594efda8800 2023-05-20T01:28:50.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.138+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 103 ==== osd_op_reply(534 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.138+0000 7fe45dd097c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-05-20T01:28:50.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.138+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:535 18.1f 18:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff60380 con 0x5594efda8c00 2023-05-20T01:28:50.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.138+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 115 ==== osd_op_reply(535 1.00000000 [omap-get-vals-by-keys out=589b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+589 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.138+0000 7fe45dd097c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-05-20T01:28:50.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.138+0000 7fe45dd097c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-05-20T01:28:50.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.138+0000 7fe45dd097c0 20 datascan.get_frag_of: frag is 1* 2023-05-20T01:28:50.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.138+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:536 18.1b 18:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff61180 con 0x5594efda9800 2023-05-20T01:28:50.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.138+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 202 ==== osd_op_reply(536 10000000000.01800000 [omap-get-header out=274b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.138+0000 7fe45dd097c0 10 datascan.find_or_create_dirfrag: read_version = 3 2023-05-20T01:28:50.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.138+0000 7fe45dd097c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-05-20T01:28:50.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.138+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:537 18.1b 18:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff61880 con 0x5594efda9800 2023-05-20T01:28:50.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.138+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 203 ==== osd_op_reply(537 10000000000.01800000 [omap-get-vals-by-keys out=577b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+577 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.138+0000 7fe45dd097c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=18 frag=1*, r=0 2023-05-20T01:28:50.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.138+0000 7fe45dd097c0 20 datascan.inject_with_backtrace: Dentry 0x0x10000000000/18 already exists and points to me 2023-05-20T01:28:50.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.138+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:538 19.7 19:e2e07930::::head [pgnls-filter start_epoch 138 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff61c00 con 0x5594efdd9000 2023-05-20T01:28:50.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.139+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.2 v2:172.21.15.16:6816/293187030 36 ==== osd_op_reply(538 [pgnls-filter start_epoch 138 out=101b] v138'15 uv15 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x5594efda1680 con 0x5594efdd9000 2023-05-20T01:28:50.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.139+0000 7fe45dd097c0 20 datascan.forall_objects: OSD matched oid 10000000001.00000000 2023-05-20T01:28:50.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.139+0000 7fe45dd097c0 10 datascan.operator(): handling object 10000000001.0 2023-05-20T01:28:50.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.139+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:539 19.7 19:e2e07930:::10000000001.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff61500 con 0x5594efdd9000 2023-05-20T01:28:50.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.139+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.2 v2:172.21.15.16:6816/293187030 37 ==== osd_op_reply(539 10000000001.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (89) out=89b,getxattr (30) out=30b,getxattr] v0'0 uv14 ondisk = 0) v8 ==== 416+0+157 (crc 0 0 0) 0x5594efda1680 con 0x5594efdd9000 2023-05-20T01:28:50.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.139+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000001 2023-05-20T01:28:50.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.139+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/0 2023-05-20T01:28:50.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.139+0000 7fe45dd097c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=0 2023-05-20T01:28:50.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.139+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:540 18.10 18:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff60700 con 0x5594efda8800 2023-05-20T01:28:50.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.139+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 104 ==== osd_op_reply(540 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.139+0000 7fe45dd097c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-05-20T01:28:50.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.139+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:541 18.1f 18:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff60a80 con 0x5594efda8c00 2023-05-20T01:28:50.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.140+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 116 ==== osd_op_reply(541 1.00000000 [omap-get-vals-by-keys out=589b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+589 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.140+0000 7fe45dd097c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-05-20T01:28:50.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.140+0000 7fe45dd097c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-05-20T01:28:50.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.140+0000 7fe45dd097c0 20 datascan.get_frag_of: frag is 1* 2023-05-20T01:28:50.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.140+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:542 18.1b 18:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff5aa80 con 0x5594efda9800 2023-05-20T01:28:50.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.140+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 204 ==== osd_op_reply(542 10000000000.01800000 [omap-get-header out=274b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.140+0000 7fe45dd097c0 10 datascan.find_or_create_dirfrag: read_version = 3 2023-05-20T01:28:50.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.140+0000 7fe45dd097c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-05-20T01:28:50.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.140+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:543 18.1b 18:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff5b180 con 0x5594efda9800 2023-05-20T01:28:50.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.140+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 205 ==== osd_op_reply(543 10000000000.01800000 [omap-get-vals-by-keys out=576b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+576 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.140+0000 7fe45dd097c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=0 frag=1*, r=0 2023-05-20T01:28:50.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.140+0000 7fe45dd097c0 20 datascan.inject_with_backtrace: Dentry 0x0x10000000000/0 already exists and points to me 2023-05-20T01:28:50.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.140+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:544 19.7 19:e37752f2::::head [pgnls-filter start_epoch 138 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff5b500 con 0x5594efdd9000 2023-05-20T01:28:50.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.140+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.2 v2:172.21.15.16:6816/293187030 38 ==== osd_op_reply(544 [pgnls-filter start_epoch 138 out=81b] v138'15 uv15 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5594efda1680 con 0x5594efdd9000 2023-05-20T01:28:50.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.140+0000 7fe45dd097c0 20 datascan.forall_objects: OSD matched oid 1000000021f.00000000 2023-05-20T01:28:50.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.140+0000 7fe45dd097c0 10 datascan.operator(): handling object 1000000021f.0 2023-05-20T01:28:50.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.140+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:545 19.7 19:e37752f2:::1000000021f.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef6e00 con 0x5594efdd9000 2023-05-20T01:28:50.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.141+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.2 v2:172.21.15.16:6816/293187030 39 ==== osd_op_reply(545 1000000021f.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv15 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x5594efda1680 con 0x5594efdd9000 2023-05-20T01:28:50.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.141+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: inode: 0x0x1000000021f 2023-05-20T01:28:50.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.141+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/42 2023-05-20T01:28:50.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.141+0000 7fe45dd097c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=42 2023-05-20T01:28:50.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.141+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:546 18.10 18:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef7880 con 0x5594efda8800 2023-05-20T01:28:50.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.141+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 105 ==== osd_op_reply(546 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.141+0000 7fe45dd097c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-05-20T01:28:50.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.141+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:547 18.1f 18:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff37180 con 0x5594efda8c00 2023-05-20T01:28:50.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.141+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 117 ==== osd_op_reply(547 1.00000000 [omap-get-vals-by-keys out=589b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+589 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.141+0000 7fe45dd097c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-05-20T01:28:50.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.141+0000 7fe45dd097c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-05-20T01:28:50.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.141+0000 7fe45dd097c0 20 datascan.get_frag_of: frag is 0* 2023-05-20T01:28:50.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.141+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:548 18.5 18:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff37500 con 0x5594efda9800 2023-05-20T01:28:50.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.141+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 206 ==== osd_op_reply(548 10000000000.01000000 [omap-get-header out=274b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.141+0000 7fe45dd097c0 10 datascan.find_or_create_dirfrag: read_version = 4 2023-05-20T01:28:50.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.141+0000 7fe45dd097c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-05-20T01:28:50.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.141+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:549 18.5 18:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff37880 con 0x5594efda9800 2023-05-20T01:28:50.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.142+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 207 ==== osd_op_reply(549 10000000000.01000000 [omap-get-vals-by-keys out=577b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+577 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.142+0000 7fe45dd097c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=42 frag=0*, r=0 2023-05-20T01:28:50.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.142+0000 7fe45dd097c0 20 datascan.inject_with_backtrace: Dentry 0x0x10000000000/42 already exists and points to me 2023-05-20T01:28:50.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.142+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:550 19.27 19:e4000000::::head [pgnls-filter start_epoch 138 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff40380 con 0x5594efda9800 2023-05-20T01:28:50.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.142+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 208 ==== osd_op_reply(550 [pgnls-filter start_epoch 138 out=101b] v138'17 uv17 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.142+0000 7fe45dd097c0 20 datascan.forall_objects: OSD matched oid 10000000210.00000000 2023-05-20T01:28:50.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.142+0000 7fe45dd097c0 10 datascan.operator(): handling object 10000000210.0 2023-05-20T01:28:50.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.142+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:551 19.27 19:e40c9c52:::10000000210.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff40a80 con 0x5594efda9800 2023-05-20T01:28:50.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.142+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 209 ==== osd_op_reply(551 10000000210.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv13 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.143+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000210 2023-05-20T01:28:50.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.143+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/27 2023-05-20T01:28:50.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.143+0000 7fe45dd097c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=27 2023-05-20T01:28:50.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.143+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:552 18.10 18:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091ee00 con 0x5594efda8800 2023-05-20T01:28:50.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.143+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 106 ==== osd_op_reply(552 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.143+0000 7fe45dd097c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-05-20T01:28:50.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.143+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:553 18.1f 18:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091f180 con 0x5594efda8c00 2023-05-20T01:28:50.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.143+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 118 ==== osd_op_reply(553 1.00000000 [omap-get-vals-by-keys out=589b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+589 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.143+0000 7fe45dd097c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-05-20T01:28:50.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.143+0000 7fe45dd097c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-05-20T01:28:50.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.143+0000 7fe45dd097c0 20 datascan.get_frag_of: frag is 0* 2023-05-20T01:28:50.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.143+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:554 18.5 18:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091f500 con 0x5594efda9800 2023-05-20T01:28:50.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.143+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 210 ==== osd_op_reply(554 10000000000.01000000 [omap-get-header out=274b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.143+0000 7fe45dd097c0 10 datascan.find_or_create_dirfrag: read_version = 4 2023-05-20T01:28:50.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.143+0000 7fe45dd097c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-05-20T01:28:50.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.143+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:555 18.5 18:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff60e00 con 0x5594efda9800 2023-05-20T01:28:50.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.144+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 211 ==== osd_op_reply(555 10000000000.01000000 [omap-get-vals-by-keys out=577b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+577 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.144+0000 7fe45dd097c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=27 frag=0*, r=0 2023-05-20T01:28:50.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.144+0000 7fe45dd097c0 20 datascan.inject_with_backtrace: Dentry 0x0x10000000000/27 already exists and points to me 2023-05-20T01:28:50.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.144+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:556 19.27 19:e5671758::::head [pgnls-filter start_epoch 138 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff60380 con 0x5594efda9800 2023-05-20T01:28:50.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.144+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 212 ==== osd_op_reply(556 [pgnls-filter start_epoch 138 out=101b] v138'17 uv17 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.144+0000 7fe45dd097c0 20 datascan.forall_objects: OSD matched oid 1000000024f.00000000 2023-05-20T01:28:50.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.144+0000 7fe45dd097c0 10 datascan.operator(): handling object 1000000024f.0 2023-05-20T01:28:50.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.144+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:557 19.27 19:e5671758:::1000000024f.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff61c00 con 0x5594efda9800 2023-05-20T01:28:50.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.144+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 213 ==== osd_op_reply(557 1000000024f.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv14 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.144+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: inode: 0x0x1000000024f 2023-05-20T01:28:50.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.144+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/90 2023-05-20T01:28:50.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.144+0000 7fe45dd097c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=90 2023-05-20T01:28:50.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.144+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:558 18.10 18:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091e700 con 0x5594efda8800 2023-05-20T01:28:50.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.145+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 107 ==== osd_op_reply(558 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.145+0000 7fe45dd097c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-05-20T01:28:50.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.145+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:559 18.1f 18:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091f880 con 0x5594efda8c00 2023-05-20T01:28:50.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.145+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 119 ==== osd_op_reply(559 1.00000000 [omap-get-vals-by-keys out=589b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+589 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.145+0000 7fe45dd097c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-05-20T01:28:50.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.145+0000 7fe45dd097c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-05-20T01:28:50.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.145+0000 7fe45dd097c0 20 datascan.get_frag_of: frag is 0* 2023-05-20T01:28:50.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.145+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:560 18.5 18:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff49500 con 0x5594efda9800 2023-05-20T01:28:50.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.145+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 214 ==== osd_op_reply(560 10000000000.01000000 [omap-get-header out=274b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.145+0000 7fe45dd097c0 10 datascan.find_or_create_dirfrag: read_version = 4 2023-05-20T01:28:50.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.145+0000 7fe45dd097c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-05-20T01:28:50.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.145+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:561 18.5 18:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff49880 con 0x5594efda9800 2023-05-20T01:28:50.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.146+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 215 ==== osd_op_reply(561 10000000000.01000000 [omap-get-vals-by-keys out=577b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+577 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.146+0000 7fe45dd097c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=90 frag=0*, r=0 2023-05-20T01:28:50.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.146+0000 7fe45dd097c0 20 datascan.inject_with_backtrace: Dentry 0x0x10000000000/90 already exists and points to me 2023-05-20T01:28:50.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.146+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:562 19.27 19:e5993619::::head [pgnls-filter start_epoch 138 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff48e00 con 0x5594efda9800 2023-05-20T01:28:50.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.146+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 216 ==== osd_op_reply(562 [pgnls-filter start_epoch 138 out=101b] v138'17 uv17 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.146+0000 7fe45dd097c0 20 datascan.forall_objects: OSD matched oid 10000000208.00000000 2023-05-20T01:28:50.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.146+0000 7fe45dd097c0 10 datascan.operator(): handling object 10000000208.0 2023-05-20T01:28:50.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.146+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:563 19.27 19:e5993619:::10000000208.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff41180 con 0x5594efda9800 2023-05-20T01:28:50.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.146+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 217 ==== osd_op_reply(563 10000000208.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv15 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.146+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000208 2023-05-20T01:28:50.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.146+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/19 2023-05-20T01:28:50.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.146+0000 7fe45dd097c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=19 2023-05-20T01:28:50.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.146+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:564 18.10 18:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff36e00 con 0x5594efda8800 2023-05-20T01:28:50.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.147+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 108 ==== osd_op_reply(564 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.147+0000 7fe45dd097c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-05-20T01:28:50.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.147+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:565 18.1f 18:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff37c00 con 0x5594efda8c00 2023-05-20T01:28:50.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.147+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 120 ==== osd_op_reply(565 1.00000000 [omap-get-vals-by-keys out=589b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+589 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.147+0000 7fe45dd097c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-05-20T01:28:50.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.147+0000 7fe45dd097c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-05-20T01:28:50.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.147+0000 7fe45dd097c0 20 datascan.get_frag_of: frag is 0* 2023-05-20T01:28:50.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.147+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:566 18.5 18:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef7180 con 0x5594efda9800 2023-05-20T01:28:50.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.147+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 218 ==== osd_op_reply(566 10000000000.01000000 [omap-get-header out=274b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.147+0000 7fe45dd097c0 10 datascan.find_or_create_dirfrag: read_version = 4 2023-05-20T01:28:50.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.147+0000 7fe45dd097c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-05-20T01:28:50.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.147+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:567 18.5 18:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef6a80 con 0x5594efda9800 2023-05-20T01:28:50.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.148+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 219 ==== osd_op_reply(567 10000000000.01000000 [omap-get-vals-by-keys out=577b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+577 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.148+0000 7fe45dd097c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=19 frag=0*, r=0 2023-05-20T01:28:50.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.148+0000 7fe45dd097c0 20 datascan.inject_with_backtrace: Dentry 0x0x10000000000/19 already exists and points to me 2023-05-20T01:28:50.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.148+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:568 19.27 19:e5a5749e::::head [pgnls-filter start_epoch 138 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef7500 con 0x5594efda9800 2023-05-20T01:28:50.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.148+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 220 ==== osd_op_reply(568 [pgnls-filter start_epoch 138 out=101b] v138'17 uv17 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.148+0000 7fe45dd097c0 20 datascan.forall_objects: OSD matched oid 1000000021a.00000000 2023-05-20T01:28:50.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.148+0000 7fe45dd097c0 10 datascan.operator(): handling object 1000000021a.0 2023-05-20T01:28:50.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.148+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:569 19.27 19:e5a5749e:::1000000021a.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff5a700 con 0x5594efda9800 2023-05-20T01:28:50.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.148+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 221 ==== osd_op_reply(569 1000000021a.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv16 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.148+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: inode: 0x0x1000000021a 2023-05-20T01:28:50.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.148+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/37 2023-05-20T01:28:50.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.148+0000 7fe45dd097c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=37 2023-05-20T01:28:50.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.148+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:570 18.10 18:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff5ae00 con 0x5594efda8800 2023-05-20T01:28:50.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.148+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 109 ==== osd_op_reply(570 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.148+0000 7fe45dd097c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-05-20T01:28:50.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.148+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:571 18.1f 18:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff5b880 con 0x5594efda8c00 2023-05-20T01:28:50.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.149+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 121 ==== osd_op_reply(571 1.00000000 [omap-get-vals-by-keys out=589b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+589 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.149+0000 7fe45dd097c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-05-20T01:28:50.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.149+0000 7fe45dd097c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-05-20T01:28:50.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.149+0000 7fe45dd097c0 20 datascan.get_frag_of: frag is 1* 2023-05-20T01:28:50.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.149+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:572 18.1b 18:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff5a000 con 0x5594efda9800 2023-05-20T01:28:50.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.149+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 222 ==== osd_op_reply(572 10000000000.01800000 [omap-get-header out=274b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.149+0000 7fe45dd097c0 10 datascan.find_or_create_dirfrag: read_version = 3 2023-05-20T01:28:50.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.149+0000 7fe45dd097c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-05-20T01:28:50.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.149+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:573 18.1b 18:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff5bc00 con 0x5594efda9800 2023-05-20T01:28:50.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.149+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 223 ==== osd_op_reply(573 10000000000.01800000 [omap-get-vals-by-keys out=577b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+577 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.149+0000 7fe45dd097c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=37 frag=1*, r=0 2023-05-20T01:28:50.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.149+0000 7fe45dd097c0 20 datascan.inject_with_backtrace: Dentry 0x0x10000000000/37 already exists and points to me 2023-05-20T01:28:50.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.149+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:574 19.27 19:e692fc64::::head [pgnls-filter start_epoch 138 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff49180 con 0x5594efda9800 2023-05-20T01:28:50.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.149+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 224 ==== osd_op_reply(574 [pgnls-filter start_epoch 138 out=81b] v138'17 uv17 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.149+0000 7fe45dd097c0 20 datascan.forall_objects: OSD matched oid 1000000021b.00000000 2023-05-20T01:28:50.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.149+0000 7fe45dd097c0 10 datascan.operator(): handling object 1000000021b.0 2023-05-20T01:28:50.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.149+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:575 19.27 19:e692fc64:::1000000021b.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff49500 con 0x5594efda9800 2023-05-20T01:28:50.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.150+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 225 ==== osd_op_reply(575 1000000021b.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv17 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.150+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: inode: 0x0x1000000021b 2023-05-20T01:28:50.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.150+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/38 2023-05-20T01:28:50.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.150+0000 7fe45dd097c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=38 2023-05-20T01:28:50.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.150+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:576 18.10 18:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091fc00 con 0x5594efda8800 2023-05-20T01:28:50.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.150+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 110 ==== osd_op_reply(576 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.150+0000 7fe45dd097c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-05-20T01:28:50.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.150+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:577 18.1f 18:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091ee00 con 0x5594efda8c00 2023-05-20T01:28:50.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.150+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 122 ==== osd_op_reply(577 1.00000000 [omap-get-vals-by-keys out=589b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+589 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.150+0000 7fe45dd097c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-05-20T01:28:50.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.150+0000 7fe45dd097c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-05-20T01:28:50.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.150+0000 7fe45dd097c0 20 datascan.get_frag_of: frag is 0* 2023-05-20T01:28:50.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.150+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:578 18.5 18:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091f500 con 0x5594efda9800 2023-05-20T01:28:50.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.151+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 226 ==== osd_op_reply(578 10000000000.01000000 [omap-get-header out=274b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.151+0000 7fe45dd097c0 10 datascan.find_or_create_dirfrag: read_version = 4 2023-05-20T01:28:50.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.151+0000 7fe45dd097c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-05-20T01:28:50.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.151+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:579 18.5 18:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091e700 con 0x5594efda9800 2023-05-20T01:28:50.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.151+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 227 ==== osd_op_reply(579 10000000000.01000000 [omap-get-vals-by-keys out=577b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+577 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.151+0000 7fe45dd097c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=38 frag=0*, r=0 2023-05-20T01:28:50.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.151+0000 7fe45dd097c0 20 datascan.inject_with_backtrace: Dentry 0x0x10000000000/38 already exists and points to me 2023-05-20T01:28:50.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.151+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:580 19.17 19:e8000000::::head [pgnls-filter start_epoch 138 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff61180 con 0x5594eff01000 2023-05-20T01:28:50.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.151+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.5 v2:172.21.15.132:6816/1844689995 6 ==== osd_op_reply(580 [pgnls-filter start_epoch 138 out=101b] v138'15 uv15 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x5594efda1440 con 0x5594eff01000 2023-05-20T01:28:50.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.151+0000 7fe45dd097c0 20 datascan.forall_objects: OSD matched oid 10000000215.00000000 2023-05-20T01:28:50.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.151+0000 7fe45dd097c0 10 datascan.operator(): handling object 10000000215.0 2023-05-20T01:28:50.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.151+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:581 19.17 19:eace0b58:::10000000215.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff61880 con 0x5594eff01000 2023-05-20T01:28:50.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.152+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.5 v2:172.21.15.132:6816/1844689995 7 ==== osd_op_reply(581 10000000215.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv14 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x5594efda1440 con 0x5594eff01000 2023-05-20T01:28:50.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.152+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000215 2023-05-20T01:28:50.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.152+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/32 2023-05-20T01:28:50.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.152+0000 7fe45dd097c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=32 2023-05-20T01:28:50.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.152+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:582 18.10 18:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff60e00 con 0x5594efda8800 2023-05-20T01:28:50.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.152+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 111 ==== osd_op_reply(582 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.152+0000 7fe45dd097c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-05-20T01:28:50.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.152+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:583 18.1f 18:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff60380 con 0x5594efda8c00 2023-05-20T01:28:50.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.152+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 123 ==== osd_op_reply(583 1.00000000 [omap-get-vals-by-keys out=589b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+589 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.152+0000 7fe45dd097c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-05-20T01:28:50.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.152+0000 7fe45dd097c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-05-20T01:28:50.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.152+0000 7fe45dd097c0 20 datascan.get_frag_of: frag is 1* 2023-05-20T01:28:50.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.152+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:584 18.1b 18:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff60000 con 0x5594efda9800 2023-05-20T01:28:50.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.153+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 228 ==== osd_op_reply(584 10000000000.01800000 [omap-get-header out=274b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.153+0000 7fe45dd097c0 10 datascan.find_or_create_dirfrag: read_version = 3 2023-05-20T01:28:50.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.153+0000 7fe45dd097c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-05-20T01:28:50.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.153+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:585 18.1b 18:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff61c00 con 0x5594efda9800 2023-05-20T01:28:50.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.153+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 229 ==== osd_op_reply(585 10000000000.01800000 [omap-get-vals-by-keys out=577b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+577 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.153+0000 7fe45dd097c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=32 frag=1*, r=0 2023-05-20T01:28:50.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.153+0000 7fe45dd097c0 20 datascan.inject_with_backtrace: Dentry 0x0x10000000000/32 already exists and points to me 2023-05-20T01:28:50.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.153+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:586 19.17 19:eb1b33a4::::head [pgnls-filter start_epoch 138 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff41c00 con 0x5594eff01000 2023-05-20T01:28:50.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.154+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.5 v2:172.21.15.132:6816/1844689995 8 ==== osd_op_reply(586 [pgnls-filter start_epoch 138 out=81b] v138'15 uv15 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5594efda1440 con 0x5594eff01000 2023-05-20T01:28:50.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.154+0000 7fe45dd097c0 20 datascan.forall_objects: OSD matched oid 10000000253.00000000 2023-05-20T01:28:50.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.154+0000 7fe45dd097c0 10 datascan.operator(): handling object 10000000253.0 2023-05-20T01:28:50.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.154+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:587 19.17 19:eb1b33a4:::10000000253.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff40380 con 0x5594eff01000 2023-05-20T01:28:50.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.154+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.5 v2:172.21.15.132:6816/1844689995 9 ==== osd_op_reply(587 10000000253.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv15 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x5594efda1440 con 0x5594eff01000 2023-05-20T01:28:50.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.154+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000253 2023-05-20T01:28:50.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.154+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/94 2023-05-20T01:28:50.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.154+0000 7fe45dd097c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=94 2023-05-20T01:28:50.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.154+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:588 18.10 18:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff40a80 con 0x5594efda8800 2023-05-20T01:28:50.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.155+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 112 ==== osd_op_reply(588 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.155+0000 7fe45dd097c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-05-20T01:28:50.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.155+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:589 18.1f 18:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff36380 con 0x5594efda8c00 2023-05-20T01:28:50.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.155+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 124 ==== osd_op_reply(589 1.00000000 [omap-get-vals-by-keys out=589b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+589 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.155+0000 7fe45dd097c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-05-20T01:28:50.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.155+0000 7fe45dd097c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-05-20T01:28:50.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.155+0000 7fe45dd097c0 20 datascan.get_frag_of: frag is 1* 2023-05-20T01:28:50.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.155+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:590 18.1b 18:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff36a80 con 0x5594efda9800 2023-05-20T01:28:50.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.155+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 230 ==== osd_op_reply(590 10000000000.01800000 [omap-get-header out=274b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.155+0000 7fe45dd097c0 10 datascan.find_or_create_dirfrag: read_version = 3 2023-05-20T01:28:50.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.155+0000 7fe45dd097c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-05-20T01:28:50.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.155+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:591 18.1b 18:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff37500 con 0x5594efda9800 2023-05-20T01:28:50.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.155+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 231 ==== osd_op_reply(591 10000000000.01800000 [omap-get-vals-by-keys out=577b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+577 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.155+0000 7fe45dd097c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=94 frag=1*, r=0 2023-05-20T01:28:50.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.155+0000 7fe45dd097c0 20 datascan.inject_with_backtrace: Dentry 0x0x10000000000/94 already exists and points to me 2023-05-20T01:28:50.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.155+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:592 19.37 19:ec000000::::head [pgnls-filter start_epoch 138 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff37880 con 0x5594efda8800 2023-05-20T01:28:50.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.156+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 113 ==== osd_op_reply(592 [pgnls-filter start_epoch 138 out=49b] v136'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.156+0000 7fe4574ae700 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:593 19.f 19:f0000000::::head [pgnls-filter start_epoch 138 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef6380 con 0x5594eff01000 2023-05-20T01:28:50.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.156+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.5 v2:172.21.15.132:6816/1844689995 10 ==== osd_op_reply(593 [pgnls-filter start_epoch 138 out=81b] v138'12 uv12 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5594efda1440 con 0x5594eff01000 2023-05-20T01:28:50.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.156+0000 7fe45dd097c0 20 datascan.forall_objects: OSD matched oid 1000000022e.00000000 2023-05-20T01:28:50.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.156+0000 7fe45dd097c0 10 datascan.operator(): handling object 1000000022e.0 2023-05-20T01:28:50.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.156+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:594 19.f 19:f25dd5df:::1000000022e.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef6000 con 0x5594eff01000 2023-05-20T01:28:50.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.157+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.5 v2:172.21.15.132:6816/1844689995 11 ==== osd_op_reply(594 1000000022e.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv12 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x5594efda1440 con 0x5594eff01000 2023-05-20T01:28:50.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.157+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: inode: 0x0x1000000022e 2023-05-20T01:28:50.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.157+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/57 2023-05-20T01:28:50.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.157+0000 7fe45dd097c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=57 2023-05-20T01:28:50.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.157+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:595 18.10 18:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff5aa80 con 0x5594efda8800 2023-05-20T01:28:50.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.157+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 114 ==== osd_op_reply(595 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.157+0000 7fe45dd097c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-05-20T01:28:50.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.157+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:596 18.1f 18:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff5b180 con 0x5594efda8c00 2023-05-20T01:28:50.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.158+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 125 ==== osd_op_reply(596 1.00000000 [omap-get-vals-by-keys out=589b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+589 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.158+0000 7fe45dd097c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-05-20T01:28:50.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.158+0000 7fe45dd097c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-05-20T01:28:50.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.158+0000 7fe45dd097c0 20 datascan.get_frag_of: frag is 1* 2023-05-20T01:28:50.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.158+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:597 18.1b 18:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff40e00 con 0x5594efda9800 2023-05-20T01:28:50.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.158+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 232 ==== osd_op_reply(597 10000000000.01800000 [omap-get-header out=274b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.158+0000 7fe45dd097c0 10 datascan.find_or_create_dirfrag: read_version = 3 2023-05-20T01:28:50.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.158+0000 7fe45dd097c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-05-20T01:28:50.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.158+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:598 18.1b 18:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff41180 con 0x5594efda9800 2023-05-20T01:28:50.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.158+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 233 ==== osd_op_reply(598 10000000000.01800000 [omap-get-vals-by-keys out=577b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+577 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.158+0000 7fe45dd097c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=57 frag=1*, r=0 2023-05-20T01:28:50.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.158+0000 7fe45dd097c0 20 datascan.inject_with_backtrace: Dentry 0x0x10000000000/57 already exists and points to me 2023-05-20T01:28:50.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.158+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:599 19.2f 19:f4000000::::head [pgnls-filter start_epoch 138 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff41c00 con 0x5594efda9c00 2023-05-20T01:28:50.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.159+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.4 v2:172.21.15.132:6817/506317223 19 ==== osd_op_reply(599 [pgnls-filter start_epoch 138 out=49b] v136'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5594efda1680 con 0x5594efda9c00 2023-05-20T01:28:50.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.159+0000 7fe4574ae700 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:600 19.1f 19:f8000000::::head [pgnls-filter start_epoch 138 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef6700 con 0x5594efda8800 2023-05-20T01:28:50.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.159+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 115 ==== osd_op_reply(600 [pgnls-filter start_epoch 138 out=101b] v138'12 uv12 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.159+0000 7fe45dd097c0 20 datascan.forall_objects: OSD matched oid 10000000238.00000000 2023-05-20T01:28:50.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.159+0000 7fe45dd097c0 10 datascan.operator(): handling object 10000000238.0 2023-05-20T01:28:50.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.159+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:601 19.1f 19:f87ea8f1:::10000000238.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff41500 con 0x5594efda8800 2023-05-20T01:28:50.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.159+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 116 ==== osd_op_reply(601 10000000238.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv11 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.159+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000238 2023-05-20T01:28:50.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.159+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/67 2023-05-20T01:28:50.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.159+0000 7fe45dd097c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=67 2023-05-20T01:28:50.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.159+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:602 18.10 18:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff60700 con 0x5594efda8800 2023-05-20T01:28:50.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.160+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 117 ==== osd_op_reply(602 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.160+0000 7fe45dd097c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-05-20T01:28:50.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.160+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:603 18.1f 18:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff61880 con 0x5594efda8c00 2023-05-20T01:28:50.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.160+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 126 ==== osd_op_reply(603 1.00000000 [omap-get-vals-by-keys out=589b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+589 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.160+0000 7fe45dd097c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-05-20T01:28:50.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.160+0000 7fe45dd097c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-05-20T01:28:50.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.160+0000 7fe45dd097c0 20 datascan.get_frag_of: frag is 1* 2023-05-20T01:28:50.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.160+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:604 18.1b 18:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff60e00 con 0x5594efda9800 2023-05-20T01:28:50.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.160+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 234 ==== osd_op_reply(604 10000000000.01800000 [omap-get-header out=274b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.160+0000 7fe45dd097c0 10 datascan.find_or_create_dirfrag: read_version = 3 2023-05-20T01:28:50.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.160+0000 7fe45dd097c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-05-20T01:28:50.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.160+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:605 18.1b 18:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff60000 con 0x5594efda9800 2023-05-20T01:28:50.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.160+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 235 ==== osd_op_reply(605 10000000000.01800000 [omap-get-vals-by-keys out=577b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+577 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.160+0000 7fe45dd097c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=67 frag=1*, r=0 2023-05-20T01:28:50.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.160+0000 7fe45dd097c0 20 datascan.inject_with_backtrace: Dentry 0x0x10000000000/67 already exists and points to me 2023-05-20T01:28:50.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.160+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:606 19.1f 19:f8e2d5cc::::head [pgnls-filter start_epoch 138 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff61c00 con 0x5594efda8800 2023-05-20T01:28:50.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.161+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 118 ==== osd_op_reply(606 [pgnls-filter start_epoch 138 out=81b] v138'12 uv12 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.161+0000 7fe45dd097c0 20 datascan.forall_objects: OSD matched oid 10000000214.00000000 2023-05-20T01:28:50.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.161+0000 7fe45dd097c0 10 datascan.operator(): handling object 10000000214.0 2023-05-20T01:28:50.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.161+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:607 19.1f 19:f8e2d5cc:::10000000214.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091ea80 con 0x5594efda8800 2023-05-20T01:28:50.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.161+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 119 ==== osd_op_reply(607 10000000214.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv12 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.161+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000214 2023-05-20T01:28:50.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.161+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/31 2023-05-20T01:28:50.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.161+0000 7fe45dd097c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=31 2023-05-20T01:28:50.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.161+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:608 18.10 18:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091f500 con 0x5594efda8800 2023-05-20T01:28:50.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.161+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 120 ==== osd_op_reply(608 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.161+0000 7fe45dd097c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-05-20T01:28:50.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.161+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:609 18.1f 18:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594f091e700 con 0x5594efda8c00 2023-05-20T01:28:50.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.162+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 127 ==== osd_op_reply(609 1.00000000 [omap-get-vals-by-keys out=589b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+589 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.162+0000 7fe45dd097c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-05-20T01:28:50.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.162+0000 7fe45dd097c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-05-20T01:28:50.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.162+0000 7fe45dd097c0 20 datascan.get_frag_of: frag is 0* 2023-05-20T01:28:50.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.162+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:610 18.5 18:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff49880 con 0x5594efda9800 2023-05-20T01:28:50.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.162+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 236 ==== osd_op_reply(610 10000000000.01000000 [omap-get-header out=274b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.162+0000 7fe45dd097c0 10 datascan.find_or_create_dirfrag: read_version = 4 2023-05-20T01:28:50.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.162+0000 7fe45dd097c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-05-20T01:28:50.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.162+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:611 18.5 18:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff48e00 con 0x5594efda9800 2023-05-20T01:28:50.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.162+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 237 ==== osd_op_reply(611 10000000000.01000000 [omap-get-vals-by-keys out=577b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+577 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.162+0000 7fe45dd097c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=31 frag=0*, r=0 2023-05-20T01:28:50.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.162+0000 7fe45dd097c0 20 datascan.inject_with_backtrace: Dentry 0x0x10000000000/31 already exists and points to me 2023-05-20T01:28:50.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.162+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:612 19.3f 19:fc000000::::head [pgnls-filter start_epoch 138 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff49180 con 0x5594efdd9000 2023-05-20T01:28:50.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.162+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.2 v2:172.21.15.16:6816/293187030 40 ==== osd_op_reply(612 [pgnls-filter start_epoch 138 out=101b] v138'14 uv14 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x5594efda1680 con 0x5594efdd9000 2023-05-20T01:28:50.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.162+0000 7fe45dd097c0 20 datascan.forall_objects: OSD matched oid 10000000232.00000000 2023-05-20T01:28:50.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.162+0000 7fe45dd097c0 10 datascan.operator(): handling object 10000000232.0 2023-05-20T01:28:50.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.162+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:613 19.3f 19:fd83cc12:::10000000232.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff49500 con 0x5594efdd9000 2023-05-20T01:28:50.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.163+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.2 v2:172.21.15.16:6816/293187030 41 ==== osd_op_reply(613 10000000232.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv13 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x5594efda1680 con 0x5594efdd9000 2023-05-20T01:28:50.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.163+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000232 2023-05-20T01:28:50.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.163+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/61 2023-05-20T01:28:50.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.163+0000 7fe45dd097c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=61 2023-05-20T01:28:50.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.163+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:614 18.10 18:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff5a700 con 0x5594efda8800 2023-05-20T01:28:50.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.163+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 121 ==== osd_op_reply(614 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.163+0000 7fe45dd097c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-05-20T01:28:50.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.163+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:615 18.1f 18:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff5ae00 con 0x5594efda8c00 2023-05-20T01:28:50.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.163+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 128 ==== osd_op_reply(615 1.00000000 [omap-get-vals-by-keys out=589b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+589 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.163+0000 7fe45dd097c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-05-20T01:28:50.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.163+0000 7fe45dd097c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-05-20T01:28:50.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.163+0000 7fe45dd097c0 20 datascan.get_frag_of: frag is 0* 2023-05-20T01:28:50.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.163+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:616 18.5 18:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff5a000 con 0x5594efda9800 2023-05-20T01:28:50.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.164+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 238 ==== osd_op_reply(616 10000000000.01000000 [omap-get-header out=274b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.164+0000 7fe45dd097c0 10 datascan.find_or_create_dirfrag: read_version = 4 2023-05-20T01:28:50.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.164+0000 7fe45dd097c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-05-20T01:28:50.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.164+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:617 18.5 18:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff5bc00 con 0x5594efda9800 2023-05-20T01:28:50.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.164+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 239 ==== osd_op_reply(617 10000000000.01000000 [omap-get-vals-by-keys out=577b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+577 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.164+0000 7fe45dd097c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=61 frag=0*, r=0 2023-05-20T01:28:50.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.164+0000 7fe45dd097c0 20 datascan.inject_with_backtrace: Dentry 0x0x10000000000/61 already exists and points to me 2023-05-20T01:28:50.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.164+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:618 19.3f 19:feb1c803::::head [pgnls-filter start_epoch 138 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef7180 con 0x5594efdd9000 2023-05-20T01:28:50.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.164+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.2 v2:172.21.15.16:6816/293187030 42 ==== osd_op_reply(618 [pgnls-filter start_epoch 138 out=81b] v138'14 uv14 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5594efda1680 con 0x5594efdd9000 2023-05-20T01:28:50.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.164+0000 7fe45dd097c0 20 datascan.forall_objects: OSD matched oid 100000001fa.00000000 2023-05-20T01:28:50.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.164+0000 7fe45dd097c0 10 datascan.operator(): handling object 100000001fa.0 2023-05-20T01:28:50.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.164+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:619 19.3f 19:feb1c803:::100000001fa.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef6a80 con 0x5594efdd9000 2023-05-20T01:28:50.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.165+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.2 v2:172.21.15.16:6816/293187030 43 ==== osd_op_reply(619 100000001fa.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (89) out=89b,getxattr (30) out=30b,getxattr] v0'0 uv14 ondisk = 0) v8 ==== 416+0+157 (crc 0 0 0) 0x5594efda1680 con 0x5594efdd9000 2023-05-20T01:28:50.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.165+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: inode: 0x0x100000001fa 2023-05-20T01:28:50.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.165+0000 7fe45dd097c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/5 2023-05-20T01:28:50.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.165+0000 7fe45dd097c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=5 2023-05-20T01:28:50.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.165+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:620 18.10 18:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594eff36e00 con 0x5594efda8800 2023-05-20T01:28:50.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.165+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.6 v2:172.21.15.132:6801/727581123 122 ==== osd_op_reply(620 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x5594efda1440 con 0x5594efda8800 2023-05-20T01:28:50.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.165+0000 7fe45dd097c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-05-20T01:28:50.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.165+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:621 18.1f 18:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef7c00 con 0x5594efda8c00 2023-05-20T01:28:50.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.165+0000 7fe45c4b8700 1 -- 172.21.15.16:0/1888659255 <== osd.7 v2:172.21.15.132:6800/466001149 129 ==== osd_op_reply(621 1.00000000 [omap-get-vals-by-keys out=589b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+589 (crc 0 0 0) 0x5594efda1680 con 0x5594efda8c00 2023-05-20T01:28:50.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.165+0000 7fe45dd097c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-05-20T01:28:50.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.165+0000 7fe45dd097c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-05-20T01:28:50.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.165+0000 7fe45dd097c0 20 datascan.get_frag_of: frag is 1* 2023-05-20T01:28:50.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.165+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:622 18.1b 18:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef6e00 con 0x5594efda9800 2023-05-20T01:28:50.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.165+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 240 ==== osd_op_reply(622 10000000000.01800000 [omap-get-header out=274b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.165+0000 7fe45dd097c0 10 datascan.find_or_create_dirfrag: read_version = 3 2023-05-20T01:28:50.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.165+0000 7fe45dd097c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-05-20T01:28:50.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.165+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:623 18.1b 18:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x5594efef7180 con 0x5594efda9800 2023-05-20T01:28:50.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.166+0000 7fe45b4b6700 1 -- 172.21.15.16:0/1888659255 <== osd.1 v2:172.21.15.16:6808/3860264220 241 ==== osd_op_reply(623 10000000000.01800000 [omap-get-vals-by-keys out=576b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+576 (crc 0 0 0) 0x5594efda1440 con 0x5594efda9800 2023-05-20T01:28:50.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.166+0000 7fe45dd097c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=5 frag=1*, r=0 2023-05-20T01:28:50.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.166+0000 7fe45dd097c0 20 datascan.inject_with_backtrace: Dentry 0x0x10000000000/5 already exists and points to me 2023-05-20T01:28:50.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.166+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x5594eff00000 msgr2=0x5594efec8c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:50.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.166+0000 7fe45dd097c0 1 --2- 172.21.15.16:0/1888659255 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x5594eff00000 0x5594efec8c00 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:50.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.166+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x5594efda9800 msgr2=0x5594efb25700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:50.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.166+0000 7fe45dd097c0 1 --2- 172.21.15.16:0/1888659255 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x5594efda9800 0x5594efb25700 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:50.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.166+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x5594efdd9000 msgr2=0x5594efec9180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:50.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.166+0000 7fe45dd097c0 1 --2- 172.21.15.16:0/1888659255 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x5594efdd9000 0x5594efec9180 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:50.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.166+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x5594efda9000 msgr2=0x5594efb25180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:50.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.166+0000 7fe45dd097c0 1 --2- 172.21.15.16:0/1888659255 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x5594efda9000 0x5594efb25180 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:50.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.166+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x5594efda9c00 msgr2=0x5594efec6580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:50.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.166+0000 7fe45dd097c0 1 --2- 172.21.15.16:0/1888659255 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x5594efda9c00 0x5594efec6580 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:50.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.166+0000 7fe45bcb7700 1 -- 172.21.15.16:0/1888659255 reap_dead start 2023-05-20T01:28:50.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.166+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x5594eff01000 msgr2=0x5594efec6b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:50.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.166+0000 7fe45dd097c0 1 --2- 172.21.15.16:0/1888659255 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x5594eff01000 0x5594efec6b00 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:50.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.166+0000 7fe45bcb7700 1 -- 172.21.15.16:0/1888659255 reap_dead start 2023-05-20T01:28:50.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.166+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x5594efda8800 msgr2=0x5594efb24680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:50.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.166+0000 7fe45dd097c0 1 --2- 172.21.15.16:0/1888659255 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x5594efda8800 0x5594efb24680 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:50.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.166+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x5594efda8c00 msgr2=0x5594efb24c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:50.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.166+0000 7fe45dd097c0 1 --2- 172.21.15.16:0/1888659255 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x5594efda8c00 0x5594efb24c00 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:50.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.166+0000 7fe45bcb7700 1 -- 172.21.15.16:0/1888659255 reap_dead start 2023-05-20T01:28:50.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.166+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x5594efdd8800 msgr2=0x5594efec6000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:50.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.166+0000 7fe45dd097c0 1 --2- 172.21.15.16:0/1888659255 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x5594efdd8800 0x5594efec6000 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x5594f08540f0 tx=0x5594f0923ec0 comp rx=0 tx=0).stop 2023-05-20T01:28:50.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.166+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594efda8400 msgr2=0x5594efb24100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:50.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.166+0000 7fe45dd097c0 1 --2- 172.21.15.16:0/1888659255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594efda8400 0x5594efb24100 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x5594efdbd3b0 tx=0x5594f085e2a0 comp rx=0 tx=0).stop 2023-05-20T01:28:50.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.166+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 shutdown_connections 2023-05-20T01:28:50.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.166+0000 7fe45dd097c0 1 --2- 172.21.15.16:0/1888659255 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x5594efdd8800 0x5594efec6000 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:50.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.166+0000 7fe45dd097c0 1 --2- 172.21.15.16:0/1888659255 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x5594efda8c00 0x5594efb24c00 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:50.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.166+0000 7fe45dd097c0 1 --2- 172.21.15.16:0/1888659255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594efda8400 0x5594efb24100 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:50.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.166+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 >> 172.21.15.16:0/1888659255 conn(0x5594efa89c00 msgr2=0x5594efb65800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:50.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.166+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 shutdown_connections 2023-05-20T01:28:50.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.166+0000 7fe45dd097c0 1 -- 172.21.15.16:0/1888659255 wait complete. 2023-05-20T01:28:50.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.166+0000 7fe45dd097c0 1 -- 172.21.15.16:0/495675089 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594efa89000 msgr2=0x5594efb23080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:50.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.166+0000 7fe45dd097c0 1 --2- 172.21.15.16:0/495675089 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594efa89000 0x5594efb23080 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x5594f0854e10 tx=0x5594f085e000 comp rx=0 tx=0).stop 2023-05-20T01:28:50.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.167+0000 7fe45dd097c0 1 -- 172.21.15.16:0/495675089 shutdown_connections 2023-05-20T01:28:50.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.167+0000 7fe45dd097c0 1 --2- 172.21.15.16:0/495675089 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594efa89000 0x5594efb23080 unknown :-1 s=CLOSED pgs=1719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:50.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.167+0000 7fe45dd097c0 1 --2- 172.21.15.16:0/495675089 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5594efa88800 0x5594efb23b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:50.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.167+0000 7fe45dd097c0 1 --2- 172.21.15.16:0/495675089 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5594efa88c00 0x5594efb23600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:50.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.167+0000 7fe45dd097c0 1 -- 172.21.15.16:0/495675089 >> 172.21.15.16:0/495675089 conn(0x5594efa88400 msgr2=0x5594efb65000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:50.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.167+0000 7fe45dd097c0 1 -- 172.21.15.16:0/495675089 shutdown_connections 2023-05-20T01:28:50.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.167+0000 7fe45dd097c0 1 -- 172.21.15.16:0/495675089 wait complete. 2023-05-20T01:28:50.448 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_inodes'] in time 0:00:00.563122, result: 2023-05-20T01:28:50.448 DEBUG:teuthology.orchestra.run.smithi016:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_links 2023-05-20T01:28:50.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.487+0000 7f6d6a7be7c0 1 -- start start 2023-05-20T01:28:50.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.487+0000 7f6d6a7be7c0 1 Processor -- start 2023-05-20T01:28:50.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.487+0000 7f6d6a7be7c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5636225d5000 0x56362266e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:50.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.487+0000 7f6d6a7be7c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5636225d4c00 0x56362266eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:50.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.487+0000 7f6d6a7be7c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636225d4800 0x56362266f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:50.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.487+0000 7f6d6a7be7c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x563622611cc0 con 0x5636225d4800 2023-05-20T01:28:50.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.487+0000 7f6d6a7be7c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x563622611400 con 0x5636225d4c00 2023-05-20T01:28:50.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.487+0000 7f6d6a7be7c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x563622611e00 con 0x5636225d5000 2023-05-20T01:28:50.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.487+0000 7f6d68f6d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636225d4800 0x56362266f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:50.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.487+0000 7f6d68f6d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636225d4800 0x56362266f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40792/0 (socket says 172.21.15.16:40792) 2023-05-20T01:28:50.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.487+0000 7f6d68f6d700 1 -- 172.21.15.16:0/1797047265 learned_addr learned my addr 172.21.15.16:0/1797047265 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:50.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.488+0000 7f6d67f6b700 1 --2- 172.21.15.16:0/1797047265 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5636225d4c00 0x56362266eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:50.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.488+0000 7f6d6876c700 1 --2- 172.21.15.16:0/1797047265 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5636225d5000 0x56362266e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:50.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.488+0000 7f6d68f6d700 1 -- 172.21.15.16:0/1797047265 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5636225d5000 msgr2=0x56362266e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:50.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.488+0000 7f6d68f6d700 1 --2- 172.21.15.16:0/1797047265 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5636225d5000 0x56362266e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:50.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.488+0000 7f6d68f6d700 1 -- 172.21.15.16:0/1797047265 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5636225d4c00 msgr2=0x56362266eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:50.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.488+0000 7f6d68f6d700 1 --2- 172.21.15.16:0/1797047265 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5636225d4c00 0x56362266eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:50.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.488+0000 7f6d68f6d700 1 -- 172.21.15.16:0/1797047265 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5636225f84e0 con 0x5636225d4800 2023-05-20T01:28:50.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.488+0000 7f6d68f6d700 1 --2- 172.21.15.16:0/1797047265 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636225d4800 0x56362266f080 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x563622734de0 tx=0x56362273ef00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77de34795b1929 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:50.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.488+0000 7f6d65f67700 1 -- 172.21.15.16:0/1797047265 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x563622609600 con 0x5636225d4800 2023-05-20T01:28:50.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.488+0000 7f6d65f67700 1 -- 172.21.15.16:0/1797047265 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x563622626c00 con 0x5636225d4800 2023-05-20T01:28:50.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.488+0000 7f6d65f67700 1 -- 172.21.15.16:0/1797047265 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5636226091e0 con 0x5636225d4800 2023-05-20T01:28:50.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.488+0000 7f6d6a7be7c0 1 -- 172.21.15.16:0/1797047265 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5636225f9520 con 0x5636225d4800 2023-05-20T01:28:50.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.488+0000 7f6d6a7be7c0 1 -- 172.21.15.16:0/1797047265 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x5636225f9380 con 0x5636225d4800 2023-05-20T01:28:50.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.488+0000 7f6d6a7be7c0 4 waiting for MDS map... 2023-05-20T01:28:50.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.488+0000 7f6d65f67700 1 -- 172.21.15.16:0/1797047265 <== mon.0 v2:172.21.15.16:3300/0 4 ==== osd_map(138..138 src has 1..138) v4 ==== 6554+0+0 (secure 0 0 0) 0x563622616c40 con 0x5636225d4800 2023-05-20T01:28:50.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.488+0000 7f6d65f67700 1 -- 172.21.15.16:0/1797047265 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 106) v1 ==== 2573+0+0 (secure 0 0 0) 0x563623450300 con 0x5636225d4800 2023-05-20T01:28:50.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.488+0000 7f6d6a7be7c0 4 Got MDS map 106 2023-05-20T01:28:50.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.489+0000 7f6d6a7be7c0 4 datascan.main: Using metadata pool output 2023-05-20T01:28:50.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.489+0000 7f6d6a7be7c0 4 datascan.main: connecting to RADOS... 2023-05-20T01:28:50.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.489+0000 7f6d6a7be7c0 1 Processor -- start 2023-05-20T01:28:50.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.489+0000 7f6d6a7be7c0 1 -- start start 2023-05-20T01:28:50.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.490+0000 7f6d6a7be7c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5636225d5c00 0x56362266f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:50.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.490+0000 7f6d6a7be7c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5636228e8000 0x56362266fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:50.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.490+0000 7f6d6a7be7c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636228e8400 0x563622670100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:50.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.490+0000 7f6d6a7be7c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x563622610a00 con 0x5636228e8400 2023-05-20T01:28:50.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.490+0000 7f6d6a7be7c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x563622611400 con 0x5636225d5c00 2023-05-20T01:28:50.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.490+0000 7f6d6a7be7c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x563622610b40 con 0x5636228e8000 2023-05-20T01:28:50.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.490+0000 7f6d67f6b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636228e8400 0x563622670100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:50.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.490+0000 7f6d67f6b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636228e8400 0x563622670100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40798/0 (socket says 172.21.15.16:40798) 2023-05-20T01:28:50.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.490+0000 7f6d67f6b700 1 -- 172.21.15.16:0/3072556868 learned_addr learned my addr 172.21.15.16:0/3072556868 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:50.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.490+0000 7f6d6876c700 1 --2- 172.21.15.16:0/3072556868 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5636228e8000 0x56362266fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:50.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.490+0000 7f6d67f6b700 1 --2- 172.21.15.16:0/3072556868 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5636225d5c00 0x56362266f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:50.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.490+0000 7f6d67f6b700 1 -- 172.21.15.16:0/3072556868 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5636228e8000 msgr2=0x56362266fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:50.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.490+0000 7f6d67f6b700 1 --2- 172.21.15.16:0/3072556868 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5636228e8000 0x56362266fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:50.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.490+0000 7f6d67f6b700 1 -- 172.21.15.16:0/3072556868 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5636225d5c00 msgr2=0x56362266f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:50.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.490+0000 7f6d67f6b700 1 --2- 172.21.15.16:0/3072556868 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5636225d5c00 0x56362266f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:50.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.490+0000 7f6d67f6b700 1 -- 172.21.15.16:0/3072556868 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5636225f9d40 con 0x5636228e8400 2023-05-20T01:28:50.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.490+0000 7f6d67f6b700 1 --2- 172.21.15.16:0/3072556868 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636228e8400 0x563622670100 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x5636228fa060 tx=0x56362336d500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbdd8e40cdd981bf server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:50.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.490+0000 7f6d63f63700 1 -- 172.21.15.16:0/3072556868 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x563622609080 con 0x5636228e8400 2023-05-20T01:28:50.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.490+0000 7f6d63f63700 1 -- 172.21.15.16:0/3072556868 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56362345e480 con 0x5636228e8400 2023-05-20T01:28:50.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.490+0000 7f6d63f63700 1 -- 172.21.15.16:0/3072556868 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x563622609600 con 0x5636228e8400 2023-05-20T01:28:50.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.491+0000 7f6d6a7be7c0 1 -- 172.21.15.16:0/3072556868 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636228e8400 msgr2=0x563622670100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:50.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.491+0000 7f6d6a7be7c0 1 --2- 172.21.15.16:0/3072556868 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636228e8400 0x563622670100 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x5636228fa060 tx=0x56362336d500 comp rx=0 tx=0).stop 2023-05-20T01:28:50.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.491+0000 7f6d6a7be7c0 1 -- 172.21.15.16:0/3072556868 shutdown_connections 2023-05-20T01:28:50.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.491+0000 7f6d6a7be7c0 1 --2- 172.21.15.16:0/3072556868 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636228e8400 0x563622670100 unknown :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:50.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.491+0000 7f6d6a7be7c0 1 --2- 172.21.15.16:0/3072556868 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5636225d5c00 0x56362266f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:50.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.491+0000 7f6d6a7be7c0 1 --2- 172.21.15.16:0/3072556868 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5636228e8000 0x56362266fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:50.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.491+0000 7f6d6a7be7c0 1 -- 172.21.15.16:0/3072556868 >> 172.21.15.16:0/3072556868 conn(0x5636225d4400 msgr2=0x5636226b1800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:50.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.491+0000 7f6d6a7be7c0 1 -- 172.21.15.16:0/3072556868 shutdown_connections 2023-05-20T01:28:50.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.491+0000 7f6d6a7be7c0 1 -- 172.21.15.16:0/3072556868 wait complete. 2023-05-20T01:28:50.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.491+0000 7f6d6a7be7c0 1 Processor -- start 2023-05-20T01:28:50.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.492+0000 7f6d6a7be7c0 1 -- start start 2023-05-20T01:28:50.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.492+0000 7f6d6a7be7c0 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5636225d4400 0x563622671180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:50.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.492+0000 7f6d6a7be7c0 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563622912000 0x563622670c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:50.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.492+0000 7f6d6a7be7c0 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563622912400 0x563622670680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:50.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.492+0000 7f6d6a7be7c0 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x563622610c80 con 0x563622912000 2023-05-20T01:28:50.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.492+0000 7f6d6a7be7c0 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x563622610000 con 0x5636225d4400 2023-05-20T01:28:50.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.492+0000 7f6d6a7be7c0 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x563622611180 con 0x563622912400 2023-05-20T01:28:50.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.492+0000 7f6d6876c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563622912000 0x563622670c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:50.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.492+0000 7f6d6876c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563622912000 0x563622670c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40802/0 (socket says 172.21.15.16:40802) 2023-05-20T01:28:50.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.492+0000 7f6d6876c700 1 -- 172.21.15.16:0/4119872052 learned_addr learned my addr 172.21.15.16:0/4119872052 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:50.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.492+0000 7f6d67f6b700 1 --2- 172.21.15.16:0/4119872052 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5636225d4400 0x563622671180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:50.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.492+0000 7f6d67f6b700 1 --2- 172.21.15.16:0/4119872052 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563622912400 0x563622670680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:50.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.493+0000 7f6d6876c700 1 -- 172.21.15.16:0/4119872052 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563622912400 msgr2=0x563622670680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:50.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.493+0000 7f6d6876c700 1 --2- 172.21.15.16:0/4119872052 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563622912400 0x563622670680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:50.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.493+0000 7f6d6876c700 1 -- 172.21.15.16:0/4119872052 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5636225d4400 msgr2=0x563622671180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:50.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.493+0000 7f6d6876c700 1 --2- 172.21.15.16:0/4119872052 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5636225d4400 0x563622671180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:50.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.493+0000 7f6d6876c700 1 -- 172.21.15.16:0/4119872052 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56362273c4e0 con 0x563622912000 2023-05-20T01:28:50.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.493+0000 7f6d6876c700 1 --2- 172.21.15.16:0/4119872052 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563622912000 0x563622670c00 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x5636228fb320 tx=0x563623462c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:50.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.493+0000 7f6d62760700 1 -- 172.21.15.16:0/4119872052 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x563622609080 con 0x563622912000 2023-05-20T01:28:50.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.493+0000 7f6d62760700 1 -- 172.21.15.16:0/4119872052 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56362345e900 con 0x563622912000 2023-05-20T01:28:50.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.493+0000 7f6d6a7be7c0 1 -- 172.21.15.16:0/4119872052 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56362273c340 con 0x563622912000 2023-05-20T01:28:50.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.493+0000 7f6d62760700 1 -- 172.21.15.16:0/4119872052 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x563622609600 con 0x563622912000 2023-05-20T01:28:50.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.493+0000 7f6d6a7be7c0 1 -- 172.21.15.16:0/4119872052 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5636225f91e0 con 0x563622912000 2023-05-20T01:28:50.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.493+0000 7f6d6a7be7c0 4 datascan.init: resolving metadata pool 18 2023-05-20T01:28:50.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.494+0000 7f6d62760700 1 -- 172.21.15.16:0/4119872052 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x563622636840 con 0x563622912000 2023-05-20T01:28:50.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.494+0000 7f6d62760700 1 --2- 172.21.15.16:0/4119872052 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x563622914400 0x563622671700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:50.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.495+0000 7f6d62760700 1 -- 172.21.15.16:0/4119872052 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6554+0+0 (secure 0 0 0) 0x563622616a80 con 0x563622912000 2023-05-20T01:28:50.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.495+0000 7f6d6a7be7c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-05-20T01:28:50.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.495+0000 7f6d6a7be7c0 4 datascan.main: resolving metadata pool 18 2023-05-20T01:28:50.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.495+0000 7f6d67f6b700 1 --2- 172.21.15.16:0/4119872052 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x563622914400 0x563622671700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:50.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.495+0000 7f6d6a7be7c0 1 --2- 172.21.15.16:0/4119872052 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x563622912800 0x563622670100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:50.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.495+0000 7f6d6a7be7c0 1 -- 172.21.15.16:0/4119872052 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:1 18.0 18:00000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563623454700 con 0x563622912800 2023-05-20T01:28:50.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.495+0000 7f6d67f6b700 1 --2- 172.21.15.16:0/4119872052 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x563622912800 0x563622670100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:50.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.495+0000 7f6d67f6b700 1 --2- 172.21.15.16:0/4119872052 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x563622914400 0x563622671700 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x5636228fa1b0 tx=0x56362336d380 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:50.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.495+0000 7f6d67f6b700 1 --2- 172.21.15.16:0/4119872052 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x563622912800 0x563622670100 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:50.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.496+0000 7f6d67f6b700 1 -- 172.21.15.16:0/4119872052 <== osd.0 v2:172.21.15.16:6824/3923634179 1 ==== osd_op_reply(1 [pgnls start_epoch 138 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636228e5440 con 0x563622912800 2023-05-20T01:28:50.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.496+0000 7f6d67f6b700 1 --2- 172.21.15.16:0/4119872052 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x563622914800 0x563622a14000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:50.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.496+0000 7f6d67f6b700 1 -- 172.21.15.16:0/4119872052 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:2 18.10 18:08000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563623454700 con 0x563622914800 2023-05-20T01:28:50.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.496+0000 7f6d68f6d700 1 --2- 172.21.15.16:0/4119872052 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x563622914800 0x563622a14000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:50.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.496+0000 7f6d68f6d700 1 --2- 172.21.15.16:0/4119872052 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x563622914800 0x563622a14000 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:50.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.497+0000 7f6d68f6d700 1 -- 172.21.15.16:0/4119872052 <== osd.6 v2:172.21.15.132:6801/727581123 1 ==== osd_op_reply(2 [pgnls start_epoch 138 out=177b] v136'11 uv11 ondisk = 1) v8 ==== 144+0+177 (crc 0 0 0) 0x5636228e5680 con 0x563622914800 2023-05-20T01:28:50.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.497+0000 7f6d68f6d700 1 -- 172.21.15.16:0/4119872052 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:3 18.8 18:10000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563623454700 con 0x563622912800 2023-05-20T01:28:50.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.497+0000 7f6d67f6b700 1 -- 172.21.15.16:0/4119872052 <== osd.0 v2:172.21.15.16:6824/3923634179 2 ==== osd_op_reply(3 [pgnls start_epoch 138 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636228e5440 con 0x563622912800 2023-05-20T01:28:50.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.497+0000 7f6d67f6b700 1 --2- 172.21.15.16:0/4119872052 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x563622914c00 0x563622a14580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:50.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.497+0000 7f6d67f6b700 1 -- 172.21.15.16:0/4119872052 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:4 18.18 18:18000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563623454700 con 0x563622914c00 2023-05-20T01:28:50.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.497+0000 7f6d6876c700 1 --2- 172.21.15.16:0/4119872052 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x563622914c00 0x563622a14580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:50.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.497+0000 7f6d6876c700 1 --2- 172.21.15.16:0/4119872052 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x563622914c00 0x563622a14580 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:50.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.498+0000 7f6d6876c700 1 -- 172.21.15.16:0/4119872052 <== osd.1 v2:172.21.15.16:6808/3860264220 1 ==== osd_op_reply(4 [pgnls start_epoch 138 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636228e58c0 con 0x563622914c00 2023-05-20T01:28:50.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.498+0000 7f6d6876c700 1 -- 172.21.15.16:0/4119872052 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:5 18.4 18:20000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563623454700 con 0x563622912800 2023-05-20T01:28:50.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.498+0000 7f6d67f6b700 1 -- 172.21.15.16:0/4119872052 <== osd.0 v2:172.21.15.16:6824/3923634179 3 ==== osd_op_reply(5 [pgnls start_epoch 138 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636228e5440 con 0x563622912800 2023-05-20T01:28:50.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.498+0000 7f6d67f6b700 1 -- 172.21.15.16:0/4119872052 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:6 18.14 18:28000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563623454700 con 0x563622914800 2023-05-20T01:28:50.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.498+0000 7f6d68f6d700 1 -- 172.21.15.16:0/4119872052 <== osd.6 v2:172.21.15.132:6801/727581123 2 ==== osd_op_reply(6 [pgnls start_epoch 138 out=149b] v137'24 uv24 ondisk = 1) v8 ==== 144+0+149 (crc 0 0 0) 0x5636228e5680 con 0x563622914800 2023-05-20T01:28:50.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.498+0000 7f6d68f6d700 1 --2- 172.21.15.16:0/4119872052 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x563622915400 0x563622a15080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:50.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.498+0000 7f6d68f6d700 1 -- 172.21.15.16:0/4119872052 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:7 18.c 18:30000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563623454700 con 0x563622915400 2023-05-20T01:28:50.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.499+0000 7f6d67f6b700 1 --2- 172.21.15.16:0/4119872052 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x563622915400 0x563622a15080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:50.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.499+0000 7f6d67f6b700 1 --2- 172.21.15.16:0/4119872052 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x563622915400 0x563622a15080 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:50.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.499+0000 7f6d67f6b700 1 -- 172.21.15.16:0/4119872052 <== osd.5 v2:172.21.15.132:6816/1844689995 1 ==== osd_op_reply(7 [pgnls start_epoch 138 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636228e5440 con 0x563622915400 2023-05-20T01:28:50.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.499+0000 7f6d67f6b700 1 -- 172.21.15.16:0/4119872052 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:8 18.1c 18:38000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563623454700 con 0x563622914c00 2023-05-20T01:28:50.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.500+0000 7f6d6876c700 1 -- 172.21.15.16:0/4119872052 <== osd.1 v2:172.21.15.16:6808/3860264220 2 ==== osd_op_reply(8 [pgnls start_epoch 138 out=73b] v135'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x5636228e58c0 con 0x563622914c00 2023-05-20T01:28:50.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.500+0000 7f6d6876c700 1 --2- 172.21.15.16:0/4119872052 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x563622915c00 0x563622a15600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:50.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.500+0000 7f6d6876c700 1 -- 172.21.15.16:0/4119872052 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:9 18.2 18:40000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563623454700 con 0x563622915c00 2023-05-20T01:28:50.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.500+0000 7f6d68f6d700 1 --2- 172.21.15.16:0/4119872052 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x563622915c00 0x563622a15600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:50.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.500+0000 7f6d68f6d700 1 --2- 172.21.15.16:0/4119872052 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x563622915c00 0x563622a15600 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:50.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.500+0000 7f6d68f6d700 1 -- 172.21.15.16:0/4119872052 <== osd.7 v2:172.21.15.132:6800/466001149 1 ==== osd_op_reply(9 [pgnls start_epoch 138 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636228e5680 con 0x563622915c00 2023-05-20T01:28:50.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.500+0000 7f6d68f6d700 1 --2- 172.21.15.16:0/4119872052 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x563622915800 0x563622a15b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:50.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.500+0000 7f6d68f6d700 1 -- 172.21.15.16:0/4119872052 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:10 18.12 18:48000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563623454700 con 0x563622915800 2023-05-20T01:28:50.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.501+0000 7f6d6876c700 1 --2- 172.21.15.16:0/4119872052 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x563622915800 0x563622a15b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:50.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.501+0000 7f6d6876c700 1 --2- 172.21.15.16:0/4119872052 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x563622915800 0x563622a15b80 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:50.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.501+0000 7f6d6876c700 1 -- 172.21.15.16:0/4119872052 <== osd.2 v2:172.21.15.16:6816/293187030 1 ==== osd_op_reply(10 [pgnls start_epoch 138 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636228e58c0 con 0x563622915800 2023-05-20T01:28:50.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.501+0000 7f6d6876c700 1 --2- 172.21.15.16:0/4119872052 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x563622a30000 0x563622a16680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:50.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.501+0000 7f6d6876c700 1 -- 172.21.15.16:0/4119872052 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:11 18.a 18:50000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563623454700 con 0x563622a30000 2023-05-20T01:28:50.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.501+0000 7f6d67f6b700 1 --2- 172.21.15.16:0/4119872052 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x563622a30000 0x563622a16680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:50.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.501+0000 7f6d67f6b700 1 --2- 172.21.15.16:0/4119872052 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x563622a30000 0x563622a16680 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:50.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.502+0000 7f6d67f6b700 1 -- 172.21.15.16:0/4119872052 <== osd.3 v2:172.21.15.16:6800/3669781890 1 ==== osd_op_reply(11 [pgnls start_epoch 138 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636228e5440 con 0x563622a30000 2023-05-20T01:28:50.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.502+0000 7f6d67f6b700 1 -- 172.21.15.16:0/4119872052 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:12 18.1a 18:58000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563623454700 con 0x563622a30000 2023-05-20T01:28:50.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.502+0000 7f6d67f6b700 1 -- 172.21.15.16:0/4119872052 <== osd.3 v2:172.21.15.16:6800/3669781890 2 ==== osd_op_reply(12 [pgnls start_epoch 138 out=49b] v137'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636228e5440 con 0x563622a30000 2023-05-20T01:28:50.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.502+0000 7f6d67f6b700 1 -- 172.21.15.16:0/4119872052 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:13 18.6 18:60000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563623454700 con 0x563622915c00 2023-05-20T01:28:50.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.502+0000 7f6d68f6d700 1 -- 172.21.15.16:0/4119872052 <== osd.7 v2:172.21.15.132:6800/466001149 2 ==== osd_op_reply(13 [pgnls start_epoch 138 out=77b] v137'4 uv4 ondisk = 1) v8 ==== 144+0+77 (crc 0 0 0) 0x5636228e5680 con 0x563622915c00 2023-05-20T01:28:50.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.502+0000 7f6d68f6d700 1 -- 172.21.15.16:0/4119872052 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:14 18.16 18:68000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563623454700 con 0x563622a30000 2023-05-20T01:28:50.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.503+0000 7f6d67f6b700 1 -- 172.21.15.16:0/4119872052 <== osd.3 v2:172.21.15.16:6800/3669781890 3 ==== osd_op_reply(14 [pgnls start_epoch 138 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636228e5440 con 0x563622a30000 2023-05-20T01:28:50.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.503+0000 7f6d67f6b700 1 -- 172.21.15.16:0/4119872052 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:15 18.e 18:70000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563623454700 con 0x563622912800 2023-05-20T01:28:50.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.503+0000 7f6d67f6b700 1 -- 172.21.15.16:0/4119872052 <== osd.0 v2:172.21.15.16:6824/3923634179 4 ==== osd_op_reply(15 [pgnls start_epoch 138 out=73b] v135'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x5636228e5440 con 0x563622912800 2023-05-20T01:28:50.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.503+0000 7f6d67f6b700 1 -- 172.21.15.16:0/4119872052 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:16 18.1e 18:78000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563623454700 con 0x563622915400 2023-05-20T01:28:50.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.503+0000 7f6d67f6b700 1 -- 172.21.15.16:0/4119872052 <== osd.5 v2:172.21.15.132:6816/1844689995 2 ==== osd_op_reply(16 [pgnls start_epoch 138 out=49b] v137'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636228e5440 con 0x563622915400 2023-05-20T01:28:50.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.503+0000 7f6d67f6b700 1 -- 172.21.15.16:0/4119872052 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:17 18.1 18:80000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563623454700 con 0x563622a30000 2023-05-20T01:28:50.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.504+0000 7f6d67f6b700 1 -- 172.21.15.16:0/4119872052 <== osd.3 v2:172.21.15.16:6800/3669781890 4 ==== osd_op_reply(17 [pgnls start_epoch 138 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636228e5440 con 0x563622a30000 2023-05-20T01:28:50.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.504+0000 7f6d67f6b700 1 -- 172.21.15.16:0/4119872052 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:18 18.11 18:88000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563623454700 con 0x563622914c00 2023-05-20T01:28:50.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.504+0000 7f6d6876c700 1 -- 172.21.15.16:0/4119872052 <== osd.1 v2:172.21.15.16:6808/3860264220 3 ==== osd_op_reply(18 [pgnls start_epoch 138 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636228e58c0 con 0x563622914c00 2023-05-20T01:28:50.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.504+0000 7f6d6876c700 1 -- 172.21.15.16:0/4119872052 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:19 18.9 18:90000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563623454700 con 0x563622a30000 2023-05-20T01:28:50.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.504+0000 7f6d67f6b700 1 -- 172.21.15.16:0/4119872052 <== osd.3 v2:172.21.15.16:6800/3669781890 5 ==== osd_op_reply(19 [pgnls start_epoch 138 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636228e5440 con 0x563622a30000 2023-05-20T01:28:50.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.504+0000 7f6d67f6b700 1 --2- 172.21.15.16:0/4119872052 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x563622915000 0x563622a14b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:50.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.504+0000 7f6d67f6b700 1 -- 172.21.15.16:0/4119872052 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:20 18.19 18:98000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563623454700 con 0x563622915000 2023-05-20T01:28:50.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.504+0000 7f6d68f6d700 1 --2- 172.21.15.16:0/4119872052 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x563622915000 0x563622a14b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:50.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.505+0000 7f6d68f6d700 1 --2- 172.21.15.16:0/4119872052 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x563622915000 0x563622a14b00 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:50.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.505+0000 7f6d68f6d700 1 -- 172.21.15.16:0/4119872052 <== osd.4 v2:172.21.15.132:6817/506317223 1 ==== osd_op_reply(20 [pgnls start_epoch 138 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636228e5680 con 0x563622915000 2023-05-20T01:28:50.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.505+0000 7f6d68f6d700 1 -- 172.21.15.16:0/4119872052 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:21 18.5 18:a0000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563623454700 con 0x563622914c00 2023-05-20T01:28:50.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.505+0000 7f6d6876c700 1 -- 172.21.15.16:0/4119872052 <== osd.1 v2:172.21.15.16:6808/3860264220 4 ==== osd_op_reply(21 [pgnls start_epoch 138 out=81b] v138'4 uv4 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5636228e58c0 con 0x563622914c00 2023-05-20T01:28:50.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.505+0000 7f6d6876c700 1 -- 172.21.15.16:0/4119872052 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:22 18.15 18:a8000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563623454700 con 0x563622915400 2023-05-20T01:28:50.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.506+0000 7f6d67f6b700 1 -- 172.21.15.16:0/4119872052 <== osd.5 v2:172.21.15.132:6816/1844689995 3 ==== osd_op_reply(22 [pgnls start_epoch 138 out=97b] v135'3 uv3 ondisk = 1) v8 ==== 144+0+97 (crc 0 0 0) 0x5636228e5440 con 0x563622915400 2023-05-20T01:28:50.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.506+0000 7f6d67f6b700 1 -- 172.21.15.16:0/4119872052 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:23 18.d 18:b0000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563623454700 con 0x563622915000 2023-05-20T01:28:50.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.506+0000 7f6d68f6d700 1 -- 172.21.15.16:0/4119872052 <== osd.4 v2:172.21.15.132:6817/506317223 2 ==== osd_op_reply(23 [pgnls start_epoch 138 out=74b] v135'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x5636228e5680 con 0x563622915000 2023-05-20T01:28:50.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.506+0000 7f6d68f6d700 1 -- 172.21.15.16:0/4119872052 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:24 18.1d 18:b8000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563623454700 con 0x563622912800 2023-05-20T01:28:50.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.506+0000 7f6d67f6b700 1 -- 172.21.15.16:0/4119872052 <== osd.0 v2:172.21.15.16:6824/3923634179 5 ==== osd_op_reply(24 [pgnls start_epoch 138 out=73b] v135'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x5636228e5440 con 0x563622912800 2023-05-20T01:28:50.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.506+0000 7f6d67f6b700 1 -- 172.21.15.16:0/4119872052 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:25 18.3 18:c0000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563623454700 con 0x563622914c00 2023-05-20T01:28:50.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.506+0000 7f6d6876c700 1 -- 172.21.15.16:0/4119872052 <== osd.1 v2:172.21.15.16:6808/3860264220 5 ==== osd_op_reply(25 [pgnls start_epoch 138 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636228e58c0 con 0x563622914c00 2023-05-20T01:28:50.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.506+0000 7f6d6876c700 1 -- 172.21.15.16:0/4119872052 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:26 18.13 18:c8000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563623454700 con 0x563622915c00 2023-05-20T01:28:50.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.507+0000 7f6d68f6d700 1 -- 172.21.15.16:0/4119872052 <== osd.7 v2:172.21.15.132:6800/466001149 3 ==== osd_op_reply(26 [pgnls start_epoch 138 out=98b] v136'3 uv3 ondisk = 1) v8 ==== 144+0+98 (crc 0 0 0) 0x5636228e5680 con 0x563622915c00 2023-05-20T01:28:50.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.507+0000 7f6d68f6d700 1 -- 172.21.15.16:0/4119872052 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:27 18.b 18:d0000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563623454700 con 0x563622915400 2023-05-20T01:28:50.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.507+0000 7f6d67f6b700 1 -- 172.21.15.16:0/4119872052 <== osd.5 v2:172.21.15.132:6816/1844689995 4 ==== osd_op_reply(27 [pgnls start_epoch 138 out=76b] v137'4 uv3 ondisk = 1) v8 ==== 144+0+76 (crc 0 0 0) 0x5636228e5440 con 0x563622915400 2023-05-20T01:28:50.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.507+0000 7f6d67f6b700 1 -- 172.21.15.16:0/4119872052 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:28 18.1b 18:d8000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563623454700 con 0x563622914c00 2023-05-20T01:28:50.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.507+0000 7f6d6876c700 1 -- 172.21.15.16:0/4119872052 <== osd.1 v2:172.21.15.16:6808/3860264220 6 ==== osd_op_reply(28 [pgnls start_epoch 138 out=81b] v137'4 uv3 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5636228e58c0 con 0x563622914c00 2023-05-20T01:28:50.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.507+0000 7f6d6876c700 1 -- 172.21.15.16:0/4119872052 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:29 18.7 18:e0000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563623454700 con 0x563622915800 2023-05-20T01:28:50.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.508+0000 7f6d6876c700 1 -- 172.21.15.16:0/4119872052 <== osd.2 v2:172.21.15.16:6816/293187030 2 ==== osd_op_reply(29 [pgnls start_epoch 138 out=73b] v135'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x5636228e58c0 con 0x563622915800 2023-05-20T01:28:50.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.508+0000 7f6d6876c700 1 -- 172.21.15.16:0/4119872052 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:30 18.17 18:e8000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563623454700 con 0x563622915400 2023-05-20T01:28:50.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.508+0000 7f6d67f6b700 1 -- 172.21.15.16:0/4119872052 <== osd.5 v2:172.21.15.132:6816/1844689995 5 ==== osd_op_reply(30 [pgnls start_epoch 138 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636228e5440 con 0x563622915400 2023-05-20T01:28:50.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.508+0000 7f6d67f6b700 1 -- 172.21.15.16:0/4119872052 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:31 18.f 18:f0000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563623454700 con 0x563622915400 2023-05-20T01:28:50.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.508+0000 7f6d67f6b700 1 -- 172.21.15.16:0/4119872052 <== osd.5 v2:172.21.15.132:6816/1844689995 6 ==== osd_op_reply(31 [pgnls start_epoch 138 out=73b] v135'1 uv1 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x5636228e5440 con 0x563622915400 2023-05-20T01:28:50.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.508+0000 7f6d67f6b700 1 -- 172.21.15.16:0/4119872052 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:32 18.1f 18:f8000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563623454700 con 0x563622915c00 2023-05-20T01:28:50.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.508+0000 7f6d68f6d700 1 -- 172.21.15.16:0/4119872052 <== osd.7 v2:172.21.15.132:6800/466001149 4 ==== osd_op_reply(32 [pgnls start_epoch 138 out=101b] v137'7 uv6 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x5636228e5680 con 0x563622915c00 2023-05-20T01:28:50.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.509+0000 7f6d6a7be7c0 10 datascan.scan_links: step 1: handling object 601.00000000 2023-05-20T01:28:50.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.509+0000 7f6d6a7be7c0 1 -- 172.21.15.16:0/4119872052 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:33 18.10 18:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563623454a80 con 0x563622914800 2023-05-20T01:28:50.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.509+0000 7f6d68f6d700 1 -- 172.21.15.16:0/4119872052 <== osd.6 v2:172.21.15.132:6801/727581123 3 ==== osd_op_reply(33 601.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5636228e5680 con 0x563622914800 2023-05-20T01:28:50.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.509+0000 7f6d6a7be7c0 10 datascan.scan_links: step 1: handling object 602.00000000 2023-05-20T01:28:50.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.509+0000 7f6d6a7be7c0 1 -- 172.21.15.16:0/4119872052 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:34 18.10 18:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563623454e00 con 0x563622914800 2023-05-20T01:28:50.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.509+0000 7f6d68f6d700 1 -- 172.21.15.16:0/4119872052 <== osd.6 v2:172.21.15.132:6801/727581123 4 ==== osd_op_reply(34 602.00000000 [omap-get-vals out=5b] v0'0 uv6 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5636228e5680 con 0x563622914800 2023-05-20T01:28:50.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.509+0000 7f6d6a7be7c0 10 datascan.scan_links: step 1: handling object 10000000000.00000000 2023-05-20T01:28:50.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.509+0000 7f6d6a7be7c0 1 -- 172.21.15.16:0/4119872052 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:35 18.10 18:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563623454700 con 0x563622914800 2023-05-20T01:28:50.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.510+0000 7f6d68f6d700 1 -- 172.21.15.16:0/4119872052 <== osd.6 v2:172.21.15.132:6801/727581123 5 ==== osd_op_reply(35 10000000000.00000000 [omap-get-vals out=5b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+5 (crc 0 0 0) 0x5636228e5680 con 0x563622914800 2023-05-20T01:28:50.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.510+0000 7f6d6a7be7c0 10 datascan.scan_links: step 1: handling object 600.00000000 2023-05-20T01:28:50.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.510+0000 7f6d6a7be7c0 1 -- 172.21.15.16:0/4119872052 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:36 18.10 18:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563623454a80 con 0x563622914800 2023-05-20T01:28:50.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.510+0000 7f6d68f6d700 1 -- 172.21.15.16:0/4119872052 <== osd.6 v2:172.21.15.132:6801/727581123 6 ==== osd_op_reply(36 600.00000000 [omap-get-vals out=5b] v0'0 uv2 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5636228e5680 con 0x563622914800 2023-05-20T01:28:50.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.510+0000 7f6d6a7be7c0 10 datascan.scan_links: step 1: handling object 603.00000000 2023-05-20T01:28:50.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.510+0000 7f6d6a7be7c0 1 -- 172.21.15.16:0/4119872052 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:37 18.10 18:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563623454e00 con 0x563622914800 2023-05-20T01:28:50.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.510+0000 7f6d68f6d700 1 -- 172.21.15.16:0/4119872052 <== osd.6 v2:172.21.15.132:6801/727581123 7 ==== osd_op_reply(37 603.00000000 [omap-get-vals out=5b] v0'0 uv8 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5636228e5680 con 0x563622914800 2023-05-20T01:28:50.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.510+0000 7f6d6a7be7c0 10 datascan.scan_links: step 1: handling object 1.00000000.inode 2023-05-20T01:28:50.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.510+0000 7f6d6a7be7c0 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2023-05-20T01:28:50.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.510+0000 7f6d6a7be7c0 10 datascan.scan_links: step 1: handling object 200.00000000 2023-05-20T01:28:50.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.510+0000 7f6d6a7be7c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2023-05-20T01:28:50.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.510+0000 7f6d6a7be7c0 10 datascan.scan_links: step 1: handling object 200.00000001 2023-05-20T01:28:50.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.510+0000 7f6d6a7be7c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2023-05-20T01:28:50.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.510+0000 7f6d6a7be7c0 10 datascan.scan_links: step 1: handling object 606.00000000 2023-05-20T01:28:50.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.510+0000 7f6d6a7be7c0 1 -- 172.21.15.16:0/4119872052 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:38 18.14 18:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563623455180 con 0x563622914800 2023-05-20T01:28:50.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.511+0000 7f6d68f6d700 1 -- 172.21.15.16:0/4119872052 <== osd.6 v2:172.21.15.132:6801/727581123 8 ==== osd_op_reply(38 606.00000000 [omap-get-vals out=5b] v0'0 uv2 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5636228e5680 con 0x563622914800 2023-05-20T01:28:50.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.511+0000 7f6d6a7be7c0 10 datascan.scan_links: step 1: handling object 607.00000000 2023-05-20T01:28:50.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.511+0000 7f6d6a7be7c0 1 -- 172.21.15.16:0/4119872052 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:39 18.1c 18:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563623454700 con 0x563622914c00 2023-05-20T01:28:50.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.511+0000 7f6d6876c700 1 -- 172.21.15.16:0/4119872052 <== osd.1 v2:172.21.15.16:6808/3860264220 7 ==== osd_op_reply(39 607.00000000 [omap-get-vals out=5b] v0'0 uv2 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5636228e58c0 con 0x563622914c00 2023-05-20T01:28:50.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.511+0000 7f6d6a7be7c0 10 datascan.scan_links: step 1: handling object mds0_openfiles.0 2023-05-20T01:28:50.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.511+0000 7f6d6a7be7c0 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2023-05-20T01:28:50.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.511+0000 7f6d6a7be7c0 10 datascan.scan_links: step 1: handling object 608.00000000 2023-05-20T01:28:50.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.511+0000 7f6d6a7be7c0 1 -- 172.21.15.16:0/4119872052 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:40 18.e 18:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563623454a80 con 0x563622912800 2023-05-20T01:28:50.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.511+0000 7f6d67f6b700 1 -- 172.21.15.16:0/4119872052 <== osd.0 v2:172.21.15.16:6824/3923634179 6 ==== osd_op_reply(40 608.00000000 [omap-get-vals out=5b] v0'0 uv2 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5636228e5440 con 0x563622912800 2023-05-20T01:28:50.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.511+0000 7f6d6a7be7c0 10 datascan.scan_links: step 1: handling object 10000000000.01000000 2023-05-20T01:28:50.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.511+0000 7f6d6a7be7c0 1 -- 172.21.15.16:0/4119872052 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:41 18.5 18:a40620de:::10000000000.01000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563623454e00 con 0x563622914c00 2023-05-20T01:28:50.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.512+0000 7f6d6876c700 1 -- 172.21.15.16:0/4119872052 <== osd.1 v2:172.21.15.16:6808/3860264220 8 ==== osd_op_reply(41 10000000000.01000000 [omap-get-vals out=29148b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+29148 (crc 0 0 0) 0x5636228e58c0 con 0x563622914c00 2023-05-20T01:28:50.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.512+0000 7f6d6a7be7c0 20 datascan.scan_links: injected ino detected 2023-05-20T01:28:50.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.512+0000 7f6d6a7be7c0 10 datascan.scan_links: step 1: handling object 604.00000000 2023-05-20T01:28:50.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.512+0000 7f6d6a7be7c0 1 -- 172.21.15.16:0/4119872052 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:42 18.15 18:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563623455180 con 0x563622915400 2023-05-20T01:28:50.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.513+0000 7f6d67f6b700 1 -- 172.21.15.16:0/4119872052 <== osd.5 v2:172.21.15.132:6816/1844689995 7 ==== osd_op_reply(42 604.00000000 [omap-get-vals out=5b] v0'0 uv2 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5636228e5440 con 0x563622915400 2023-05-20T01:28:50.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.513+0000 7f6d6a7be7c0 10 datascan.scan_links: step 1: handling object 500.00000000 2023-05-20T01:28:50.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.513+0000 7f6d6a7be7c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2023-05-20T01:28:50.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.513+0000 7f6d6a7be7c0 10 datascan.scan_links: step 1: handling object mds_snaptable 2023-05-20T01:28:50.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.513+0000 7f6d6a7be7c0 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2023-05-20T01:28:50.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.513+0000 7f6d6a7be7c0 10 datascan.scan_links: step 1: handling object 605.00000000 2023-05-20T01:28:50.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.513+0000 7f6d6a7be7c0 1 -- 172.21.15.16:0/4119872052 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:43 18.1d 18:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563623455500 con 0x563622912800 2023-05-20T01:28:50.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.513+0000 7f6d67f6b700 1 -- 172.21.15.16:0/4119872052 <== osd.0 v2:172.21.15.16:6824/3923634179 7 ==== osd_op_reply(43 605.00000000 [omap-get-vals out=5b] v0'0 uv2 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5636228e5440 con 0x563622912800 2023-05-20T01:28:50.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.513+0000 7f6d6a7be7c0 10 datascan.scan_links: step 1: handling object mds0_inotable 2023-05-20T01:28:50.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.513+0000 7f6d6a7be7c0 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2023-05-20T01:28:50.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.513+0000 7f6d6a7be7c0 10 datascan.scan_links: step 1: handling object 100.00000000 2023-05-20T01:28:50.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.513+0000 7f6d6a7be7c0 1 -- 172.21.15.16:0/4119872052 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:44 18.13 18:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563623454a80 con 0x563622915c00 2023-05-20T01:28:50.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.513+0000 7f6d68f6d700 1 -- 172.21.15.16:0/4119872052 <== osd.7 v2:172.21.15.132:6800/466001149 5 ==== osd_op_reply(44 100.00000000 [omap-get-vals out=5195b] v0'0 uv1 ondisk = 0) v8 ==== 156+0+5195 (crc 0 0 0) 0x5636228e5680 con 0x563622915c00 2023-05-20T01:28:50.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.513+0000 7f6d6a7be7c0 10 datascan.scan_links: step 1: handling object mds0_sessionmap 2023-05-20T01:28:50.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.513+0000 7f6d6a7be7c0 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2023-05-20T01:28:50.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.513+0000 7f6d6a7be7c0 10 datascan.scan_links: step 1: handling object 10000000000.01800000 2023-05-20T01:28:50.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.513+0000 7f6d6a7be7c0 1 -- 172.21.15.16:0/4119872052 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:45 18.1b 18:dcc776e0:::10000000000.01800000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563623455180 con 0x563622914c00 2023-05-20T01:28:50.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.514+0000 7f6d6876c700 1 -- 172.21.15.16:0/4119872052 <== osd.1 v2:172.21.15.16:6808/3860264220 9 ==== osd_op_reply(45 10000000000.01800000 [omap-get-vals out=28078b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+28078 (crc 0 0 0) 0x5636228e58c0 con 0x563622914c00 2023-05-20T01:28:50.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.514+0000 7f6d6a7be7c0 10 datascan.scan_links: step 1: handling object 609.00000000 2023-05-20T01:28:50.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.514+0000 7f6d6a7be7c0 1 -- 172.21.15.16:0/4119872052 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:46 18.7 18:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563623454700 con 0x563622915800 2023-05-20T01:28:50.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.514+0000 7f6d6876c700 1 -- 172.21.15.16:0/4119872052 <== osd.2 v2:172.21.15.16:6816/293187030 3 ==== osd_op_reply(46 609.00000000 [omap-get-vals out=5b] v0'0 uv2 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5636228e58c0 con 0x563622915800 2023-05-20T01:28:50.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.514+0000 7f6d6a7be7c0 10 datascan.scan_links: step 1: handling object 400.00000000 2023-05-20T01:28:50.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.514+0000 7f6d6a7be7c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2023-05-20T01:28:50.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.514+0000 7f6d6a7be7c0 10 datascan.scan_links: step 1: handling object 100.00000000.inode 2023-05-20T01:28:50.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.514+0000 7f6d6a7be7c0 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2023-05-20T01:28:50.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.514+0000 7f6d6a7be7c0 10 datascan.scan_links: step 1: handling object 1.00000000 2023-05-20T01:28:50.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.514+0000 7f6d6a7be7c0 1 -- 172.21.15.16:0/4119872052 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:47 18.1f 18:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563623454e00 con 0x563622915c00 2023-05-20T01:28:50.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.515+0000 7f6d68f6d700 1 -- 172.21.15.16:0/4119872052 <== osd.7 v2:172.21.15.132:6800/466001149 6 ==== osd_op_reply(47 1.00000000 [omap-get-vals out=590b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+590 (crc 0 0 0) 0x5636228e5680 con 0x563622915c00 2023-05-20T01:28:50.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.515+0000 7f6d6a7be7c0 1 -- 172.21.15.16:0/4119872052 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:48 18.0 18:00000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563623455880 con 0x563622912800 2023-05-20T01:28:50.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.515+0000 7f6d67f6b700 1 -- 172.21.15.16:0/4119872052 <== osd.0 v2:172.21.15.16:6824/3923634179 8 ==== osd_op_reply(48 [pgnls start_epoch 138 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636228e5440 con 0x563622912800 2023-05-20T01:28:50.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.515+0000 7f6d67f6b700 1 -- 172.21.15.16:0/4119872052 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:49 18.10 18:08000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563623455880 con 0x563622914800 2023-05-20T01:28:50.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.515+0000 7f6d68f6d700 1 -- 172.21.15.16:0/4119872052 <== osd.6 v2:172.21.15.132:6801/727581123 9 ==== osd_op_reply(49 [pgnls start_epoch 138 out=177b] v136'11 uv11 ondisk = 1) v8 ==== 144+0+177 (crc 0 0 0) 0x5636228e5680 con 0x563622914800 2023-05-20T01:28:50.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.515+0000 7f6d68f6d700 1 -- 172.21.15.16:0/4119872052 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:50 18.8 18:10000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563623455880 con 0x563622912800 2023-05-20T01:28:50.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.516+0000 7f6d67f6b700 1 -- 172.21.15.16:0/4119872052 <== osd.0 v2:172.21.15.16:6824/3923634179 9 ==== osd_op_reply(50 [pgnls start_epoch 138 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636228e5440 con 0x563622912800 2023-05-20T01:28:50.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.516+0000 7f6d67f6b700 1 -- 172.21.15.16:0/4119872052 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:51 18.18 18:18000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563623455880 con 0x563622914c00 2023-05-20T01:28:50.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.516+0000 7f6d6876c700 1 -- 172.21.15.16:0/4119872052 <== osd.1 v2:172.21.15.16:6808/3860264220 10 ==== osd_op_reply(51 [pgnls start_epoch 138 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636228e58c0 con 0x563622914c00 2023-05-20T01:28:50.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.516+0000 7f6d6876c700 1 -- 172.21.15.16:0/4119872052 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:52 18.4 18:20000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563623455180 con 0x563622912800 2023-05-20T01:28:50.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.516+0000 7f6d67f6b700 1 -- 172.21.15.16:0/4119872052 <== osd.0 v2:172.21.15.16:6824/3923634179 10 ==== osd_op_reply(52 [pgnls start_epoch 138 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636228e5440 con 0x563622912800 2023-05-20T01:28:50.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.516+0000 7f6d67f6b700 1 -- 172.21.15.16:0/4119872052 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:53 18.14 18:28000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563623455180 con 0x563622914800 2023-05-20T01:28:50.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.517+0000 7f6d68f6d700 1 -- 172.21.15.16:0/4119872052 <== osd.6 v2:172.21.15.132:6801/727581123 10 ==== osd_op_reply(53 [pgnls start_epoch 138 out=149b] v137'24 uv24 ondisk = 1) v8 ==== 144+0+149 (crc 0 0 0) 0x5636228e5680 con 0x563622914800 2023-05-20T01:28:50.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.517+0000 7f6d68f6d700 1 -- 172.21.15.16:0/4119872052 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:54 18.c 18:30000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563623455180 con 0x563622915400 2023-05-20T01:28:50.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.517+0000 7f6d67f6b700 1 -- 172.21.15.16:0/4119872052 <== osd.5 v2:172.21.15.132:6816/1844689995 8 ==== osd_op_reply(54 [pgnls start_epoch 138 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636228e5440 con 0x563622915400 2023-05-20T01:28:50.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.517+0000 7f6d67f6b700 1 -- 172.21.15.16:0/4119872052 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:55 18.1c 18:38000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563623455180 con 0x563622914c00 2023-05-20T01:28:50.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.517+0000 7f6d6876c700 1 -- 172.21.15.16:0/4119872052 <== osd.1 v2:172.21.15.16:6808/3860264220 11 ==== osd_op_reply(55 [pgnls start_epoch 138 out=73b] v135'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x5636228e58c0 con 0x563622914c00 2023-05-20T01:28:50.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.517+0000 7f6d6876c700 1 -- 172.21.15.16:0/4119872052 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:56 18.2 18:40000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563623455180 con 0x563622915c00 2023-05-20T01:28:50.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.517+0000 7f6d68f6d700 1 -- 172.21.15.16:0/4119872052 <== osd.7 v2:172.21.15.132:6800/466001149 7 ==== osd_op_reply(56 [pgnls start_epoch 138 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636228e5680 con 0x563622915c00 2023-05-20T01:28:50.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.517+0000 7f6d68f6d700 1 -- 172.21.15.16:0/4119872052 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:57 18.12 18:48000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563623455180 con 0x563622915800 2023-05-20T01:28:50.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.518+0000 7f6d6876c700 1 -- 172.21.15.16:0/4119872052 <== osd.2 v2:172.21.15.16:6816/293187030 4 ==== osd_op_reply(57 [pgnls start_epoch 138 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636228e58c0 con 0x563622915800 2023-05-20T01:28:50.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.518+0000 7f6d6876c700 1 -- 172.21.15.16:0/4119872052 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:58 18.a 18:50000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563623455180 con 0x563622a30000 2023-05-20T01:28:50.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.518+0000 7f6d67f6b700 1 -- 172.21.15.16:0/4119872052 <== osd.3 v2:172.21.15.16:6800/3669781890 6 ==== osd_op_reply(58 [pgnls start_epoch 138 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636228e5440 con 0x563622a30000 2023-05-20T01:28:50.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.518+0000 7f6d67f6b700 1 -- 172.21.15.16:0/4119872052 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:59 18.1a 18:58000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563623455180 con 0x563622a30000 2023-05-20T01:28:50.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.518+0000 7f6d67f6b700 1 -- 172.21.15.16:0/4119872052 <== osd.3 v2:172.21.15.16:6800/3669781890 7 ==== osd_op_reply(59 [pgnls start_epoch 138 out=49b] v137'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636228e5440 con 0x563622a30000 2023-05-20T01:28:50.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.518+0000 7f6d67f6b700 1 -- 172.21.15.16:0/4119872052 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:60 18.6 18:60000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563623455180 con 0x563622915c00 2023-05-20T01:28:50.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.518+0000 7f6d68f6d700 1 -- 172.21.15.16:0/4119872052 <== osd.7 v2:172.21.15.132:6800/466001149 8 ==== osd_op_reply(60 [pgnls start_epoch 138 out=77b] v137'4 uv4 ondisk = 1) v8 ==== 144+0+77 (crc 0 0 0) 0x5636228e5680 con 0x563622915c00 2023-05-20T01:28:50.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.518+0000 7f6d68f6d700 1 -- 172.21.15.16:0/4119872052 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:61 18.16 18:68000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563623455180 con 0x563622a30000 2023-05-20T01:28:50.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.519+0000 7f6d67f6b700 1 -- 172.21.15.16:0/4119872052 <== osd.3 v2:172.21.15.16:6800/3669781890 8 ==== osd_op_reply(61 [pgnls start_epoch 138 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636228e5440 con 0x563622a30000 2023-05-20T01:28:50.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.519+0000 7f6d67f6b700 1 -- 172.21.15.16:0/4119872052 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:62 18.e 18:70000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563623455180 con 0x563622912800 2023-05-20T01:28:50.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.519+0000 7f6d67f6b700 1 -- 172.21.15.16:0/4119872052 <== osd.0 v2:172.21.15.16:6824/3923634179 11 ==== osd_op_reply(62 [pgnls start_epoch 138 out=73b] v135'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x5636228e5440 con 0x563622912800 2023-05-20T01:28:50.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.519+0000 7f6d67f6b700 1 -- 172.21.15.16:0/4119872052 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:63 18.1e 18:78000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563623455180 con 0x563622915400 2023-05-20T01:28:50.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.519+0000 7f6d67f6b700 1 -- 172.21.15.16:0/4119872052 <== osd.5 v2:172.21.15.132:6816/1844689995 9 ==== osd_op_reply(63 [pgnls start_epoch 138 out=49b] v137'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636228e5440 con 0x563622915400 2023-05-20T01:28:50.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.519+0000 7f6d67f6b700 1 -- 172.21.15.16:0/4119872052 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:64 18.1 18:80000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563623455180 con 0x563622a30000 2023-05-20T01:28:50.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.519+0000 7f6d67f6b700 1 -- 172.21.15.16:0/4119872052 <== osd.3 v2:172.21.15.16:6800/3669781890 9 ==== osd_op_reply(64 [pgnls start_epoch 138 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636228e5440 con 0x563622a30000 2023-05-20T01:28:50.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.519+0000 7f6d67f6b700 1 -- 172.21.15.16:0/4119872052 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:65 18.11 18:88000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563623455180 con 0x563622914c00 2023-05-20T01:28:50.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.520+0000 7f6d6876c700 1 -- 172.21.15.16:0/4119872052 <== osd.1 v2:172.21.15.16:6808/3860264220 12 ==== osd_op_reply(65 [pgnls start_epoch 138 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636228e58c0 con 0x563622914c00 2023-05-20T01:28:50.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.520+0000 7f6d6876c700 1 -- 172.21.15.16:0/4119872052 --> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] -- osd_op(unknown.0.0:66 18.9 18:90000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563623455180 con 0x563622a30000 2023-05-20T01:28:50.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.520+0000 7f6d67f6b700 1 -- 172.21.15.16:0/4119872052 <== osd.3 v2:172.21.15.16:6800/3669781890 10 ==== osd_op_reply(66 [pgnls start_epoch 138 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636228e5440 con 0x563622a30000 2023-05-20T01:28:50.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.520+0000 7f6d67f6b700 1 -- 172.21.15.16:0/4119872052 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:67 18.19 18:98000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563623455180 con 0x563622915000 2023-05-20T01:28:50.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.520+0000 7f6d68f6d700 1 -- 172.21.15.16:0/4119872052 <== osd.4 v2:172.21.15.132:6817/506317223 3 ==== osd_op_reply(67 [pgnls start_epoch 138 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636228e5680 con 0x563622915000 2023-05-20T01:28:50.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.520+0000 7f6d68f6d700 1 -- 172.21.15.16:0/4119872052 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:68 18.5 18:a0000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563623455180 con 0x563622914c00 2023-05-20T01:28:50.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.521+0000 7f6d6876c700 1 -- 172.21.15.16:0/4119872052 <== osd.1 v2:172.21.15.16:6808/3860264220 13 ==== osd_op_reply(68 [pgnls start_epoch 138 out=81b] v138'4 uv4 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5636228e58c0 con 0x563622914c00 2023-05-20T01:28:50.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.521+0000 7f6d6876c700 1 -- 172.21.15.16:0/4119872052 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:69 18.15 18:a8000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563623455180 con 0x563622915400 2023-05-20T01:28:50.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.521+0000 7f6d67f6b700 1 -- 172.21.15.16:0/4119872052 <== osd.5 v2:172.21.15.132:6816/1844689995 10 ==== osd_op_reply(69 [pgnls start_epoch 138 out=97b] v135'3 uv3 ondisk = 1) v8 ==== 144+0+97 (crc 0 0 0) 0x5636228e5440 con 0x563622915400 2023-05-20T01:28:50.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.521+0000 7f6d67f6b700 1 -- 172.21.15.16:0/4119872052 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:70 18.d 18:b0000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563623455180 con 0x563622915000 2023-05-20T01:28:50.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.521+0000 7f6d68f6d700 1 -- 172.21.15.16:0/4119872052 <== osd.4 v2:172.21.15.132:6817/506317223 4 ==== osd_op_reply(70 [pgnls start_epoch 138 out=74b] v135'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x5636228e5680 con 0x563622915000 2023-05-20T01:28:50.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.521+0000 7f6d68f6d700 1 -- 172.21.15.16:0/4119872052 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:71 18.1d 18:b8000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563623455180 con 0x563622912800 2023-05-20T01:28:50.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.522+0000 7f6d67f6b700 1 -- 172.21.15.16:0/4119872052 <== osd.0 v2:172.21.15.16:6824/3923634179 12 ==== osd_op_reply(71 [pgnls start_epoch 138 out=73b] v135'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x5636228e5440 con 0x563622912800 2023-05-20T01:28:50.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.522+0000 7f6d67f6b700 1 -- 172.21.15.16:0/4119872052 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:72 18.3 18:c0000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563623455180 con 0x563622914c00 2023-05-20T01:28:50.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.522+0000 7f6d6876c700 1 -- 172.21.15.16:0/4119872052 <== osd.1 v2:172.21.15.16:6808/3860264220 14 ==== osd_op_reply(72 [pgnls start_epoch 138 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636228e58c0 con 0x563622914c00 2023-05-20T01:28:50.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.522+0000 7f6d6876c700 1 -- 172.21.15.16:0/4119872052 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:73 18.13 18:c8000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563623455180 con 0x563622915c00 2023-05-20T01:28:50.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.522+0000 7f6d68f6d700 1 -- 172.21.15.16:0/4119872052 <== osd.7 v2:172.21.15.132:6800/466001149 9 ==== osd_op_reply(73 [pgnls start_epoch 138 out=98b] v136'3 uv3 ondisk = 1) v8 ==== 144+0+98 (crc 0 0 0) 0x5636228e5680 con 0x563622915c00 2023-05-20T01:28:50.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.522+0000 7f6d68f6d700 1 -- 172.21.15.16:0/4119872052 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:74 18.b 18:d0000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563623455180 con 0x563622915400 2023-05-20T01:28:50.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.523+0000 7f6d67f6b700 1 -- 172.21.15.16:0/4119872052 <== osd.5 v2:172.21.15.132:6816/1844689995 11 ==== osd_op_reply(74 [pgnls start_epoch 138 out=76b] v137'4 uv3 ondisk = 1) v8 ==== 144+0+76 (crc 0 0 0) 0x5636228e5440 con 0x563622915400 2023-05-20T01:28:50.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.523+0000 7f6d67f6b700 1 -- 172.21.15.16:0/4119872052 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:75 18.1b 18:d8000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563623455180 con 0x563622914c00 2023-05-20T01:28:50.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.523+0000 7f6d6876c700 1 -- 172.21.15.16:0/4119872052 <== osd.1 v2:172.21.15.16:6808/3860264220 15 ==== osd_op_reply(75 [pgnls start_epoch 138 out=81b] v137'4 uv3 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5636228e58c0 con 0x563622914c00 2023-05-20T01:28:50.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.523+0000 7f6d6876c700 1 -- 172.21.15.16:0/4119872052 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:76 18.7 18:e0000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563623455180 con 0x563622915800 2023-05-20T01:28:50.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.523+0000 7f6d6876c700 1 -- 172.21.15.16:0/4119872052 <== osd.2 v2:172.21.15.16:6816/293187030 5 ==== osd_op_reply(76 [pgnls start_epoch 138 out=73b] v135'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x5636228e58c0 con 0x563622915800 2023-05-20T01:28:50.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.523+0000 7f6d6876c700 1 -- 172.21.15.16:0/4119872052 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:77 18.17 18:e8000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563623455180 con 0x563622915400 2023-05-20T01:28:50.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.524+0000 7f6d67f6b700 1 -- 172.21.15.16:0/4119872052 <== osd.5 v2:172.21.15.132:6816/1844689995 12 ==== osd_op_reply(77 [pgnls start_epoch 138 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636228e5440 con 0x563622915400 2023-05-20T01:28:50.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.524+0000 7f6d67f6b700 1 -- 172.21.15.16:0/4119872052 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:78 18.f 18:f0000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563623455180 con 0x563622915400 2023-05-20T01:28:50.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.524+0000 7f6d67f6b700 1 -- 172.21.15.16:0/4119872052 <== osd.5 v2:172.21.15.132:6816/1844689995 13 ==== osd_op_reply(78 [pgnls start_epoch 138 out=73b] v135'1 uv1 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x5636228e5440 con 0x563622915400 2023-05-20T01:28:50.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.524+0000 7f6d67f6b700 1 -- 172.21.15.16:0/4119872052 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:79 18.1f 18:f8000000::::head [pgnls start_epoch 138 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e138) v8 -- 0x563623455180 con 0x563622915c00 2023-05-20T01:28:50.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.524+0000 7f6d68f6d700 1 -- 172.21.15.16:0/4119872052 <== osd.7 v2:172.21.15.132:6800/466001149 10 ==== osd_op_reply(79 [pgnls start_epoch 138 out=101b] v137'7 uv6 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x5636228e5680 con 0x563622915c00 2023-05-20T01:28:50.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.524+0000 7f6d6a7be7c0 10 datascan.scan_links: step 2: handling object 601.00000000 2023-05-20T01:28:50.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.524+0000 7f6d6a7be7c0 1 -- 172.21.15.16:0/4119872052 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:80 18.10 18:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563623455c00 con 0x563622914800 2023-05-20T01:28:50.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.525+0000 7f6d68f6d700 1 -- 172.21.15.16:0/4119872052 <== osd.6 v2:172.21.15.132:6801/727581123 11 ==== osd_op_reply(80 601.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5636228e5680 con 0x563622914800 2023-05-20T01:28:50.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.525+0000 7f6d6a7be7c0 10 datascan.scan_links: step 2: handling object 602.00000000 2023-05-20T01:28:50.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.525+0000 7f6d6a7be7c0 1 -- 172.21.15.16:0/4119872052 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:81 18.10 18:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563622a50000 con 0x563622914800 2023-05-20T01:28:50.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.525+0000 7f6d68f6d700 1 -- 172.21.15.16:0/4119872052 <== osd.6 v2:172.21.15.132:6801/727581123 12 ==== osd_op_reply(81 602.00000000 [omap-get-vals out=5b] v0'0 uv6 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5636228e5680 con 0x563622914800 2023-05-20T01:28:50.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.525+0000 7f6d6a7be7c0 10 datascan.scan_links: step 2: handling object 10000000000.00000000 2023-05-20T01:28:50.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.525+0000 7f6d6a7be7c0 1 -- 172.21.15.16:0/4119872052 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:82 18.10 18:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563622a50380 con 0x563622914800 2023-05-20T01:28:50.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.525+0000 7f6d68f6d700 1 -- 172.21.15.16:0/4119872052 <== osd.6 v2:172.21.15.132:6801/727581123 13 ==== osd_op_reply(82 10000000000.00000000 [omap-get-vals out=5b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+5 (crc 0 0 0) 0x5636228e5680 con 0x563622914800 2023-05-20T01:28:50.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.525+0000 7f6d6a7be7c0 10 datascan.scan_links: step 2: handling object 600.00000000 2023-05-20T01:28:50.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.525+0000 7f6d6a7be7c0 1 -- 172.21.15.16:0/4119872052 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:83 18.10 18:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563622a50700 con 0x563622914800 2023-05-20T01:28:50.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.526+0000 7f6d68f6d700 1 -- 172.21.15.16:0/4119872052 <== osd.6 v2:172.21.15.132:6801/727581123 14 ==== osd_op_reply(83 600.00000000 [omap-get-vals out=5b] v0'0 uv2 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5636228e5680 con 0x563622914800 2023-05-20T01:28:50.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.526+0000 7f6d6a7be7c0 10 datascan.scan_links: step 2: handling object 603.00000000 2023-05-20T01:28:50.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.526+0000 7f6d6a7be7c0 1 -- 172.21.15.16:0/4119872052 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:84 18.10 18:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563622a50a80 con 0x563622914800 2023-05-20T01:28:50.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.526+0000 7f6d68f6d700 1 -- 172.21.15.16:0/4119872052 <== osd.6 v2:172.21.15.132:6801/727581123 15 ==== osd_op_reply(84 603.00000000 [omap-get-vals out=5b] v0'0 uv8 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5636228e5680 con 0x563622914800 2023-05-20T01:28:50.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.526+0000 7f6d6a7be7c0 10 datascan.scan_links: step 2: handling object 1.00000000.inode 2023-05-20T01:28:50.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.526+0000 7f6d6a7be7c0 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2023-05-20T01:28:50.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.526+0000 7f6d6a7be7c0 10 datascan.scan_links: step 2: handling object 200.00000000 2023-05-20T01:28:50.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.526+0000 7f6d6a7be7c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2023-05-20T01:28:50.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.526+0000 7f6d6a7be7c0 10 datascan.scan_links: step 2: handling object 200.00000001 2023-05-20T01:28:50.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.526+0000 7f6d6a7be7c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2023-05-20T01:28:50.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.526+0000 7f6d6a7be7c0 10 datascan.scan_links: step 2: handling object 606.00000000 2023-05-20T01:28:50.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.526+0000 7f6d6a7be7c0 1 -- 172.21.15.16:0/4119872052 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:85 18.14 18:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563622a50e00 con 0x563622914800 2023-05-20T01:28:50.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.526+0000 7f6d68f6d700 1 -- 172.21.15.16:0/4119872052 <== osd.6 v2:172.21.15.132:6801/727581123 16 ==== osd_op_reply(85 606.00000000 [omap-get-vals out=5b] v0'0 uv2 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5636228e5680 con 0x563622914800 2023-05-20T01:28:50.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.526+0000 7f6d6a7be7c0 10 datascan.scan_links: step 2: handling object 607.00000000 2023-05-20T01:28:50.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.526+0000 7f6d6a7be7c0 1 -- 172.21.15.16:0/4119872052 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:86 18.1c 18:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563622a51180 con 0x563622914c00 2023-05-20T01:28:50.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.526+0000 7f6d6876c700 1 -- 172.21.15.16:0/4119872052 <== osd.1 v2:172.21.15.16:6808/3860264220 16 ==== osd_op_reply(86 607.00000000 [omap-get-vals out=5b] v0'0 uv2 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5636228e58c0 con 0x563622914c00 2023-05-20T01:28:50.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.526+0000 7f6d6a7be7c0 10 datascan.scan_links: step 2: handling object mds0_openfiles.0 2023-05-20T01:28:50.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.527+0000 7f6d6a7be7c0 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2023-05-20T01:28:50.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.527+0000 7f6d6a7be7c0 10 datascan.scan_links: step 2: handling object 608.00000000 2023-05-20T01:28:50.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.527+0000 7f6d6a7be7c0 1 -- 172.21.15.16:0/4119872052 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:87 18.e 18:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563622a51500 con 0x563622912800 2023-05-20T01:28:50.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.527+0000 7f6d67f6b700 1 -- 172.21.15.16:0/4119872052 <== osd.0 v2:172.21.15.16:6824/3923634179 13 ==== osd_op_reply(87 608.00000000 [omap-get-vals out=5b] v0'0 uv2 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5636228e5440 con 0x563622912800 2023-05-20T01:28:50.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.527+0000 7f6d6a7be7c0 10 datascan.scan_links: step 2: handling object 10000000000.01000000 2023-05-20T01:28:50.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.527+0000 7f6d6a7be7c0 1 -- 172.21.15.16:0/4119872052 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:88 18.5 18:a40620de:::10000000000.01000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563622a50000 con 0x563622914c00 2023-05-20T01:28:50.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.527+0000 7f6d6876c700 1 -- 172.21.15.16:0/4119872052 <== osd.1 v2:172.21.15.16:6808/3860264220 17 ==== osd_op_reply(88 10000000000.01000000 [omap-get-vals out=29148b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+29148 (crc 0 0 0) 0x5636228e58c0 con 0x563622914c00 2023-05-20T01:28:50.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.527+0000 7f6d6a7be7c0 20 datascan.scan_links: injected ino detected 2023-05-20T01:28:50.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.527+0000 7f6d6a7be7c0 20 datascan.scan_links: adding 0x1000000020a for future processing to fix dnfirst 2023-05-20T01:28:50.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.527+0000 7f6d6a7be7c0 10 datascan.scan_links: step 2: handling object 604.00000000 2023-05-20T01:28:50.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.527+0000 7f6d6a7be7c0 1 -- 172.21.15.16:0/4119872052 --> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] -- osd_op(unknown.0.0:89 18.15 18:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563622a51880 con 0x563622915400 2023-05-20T01:28:50.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.528+0000 7f6d67f6b700 1 -- 172.21.15.16:0/4119872052 <== osd.5 v2:172.21.15.132:6816/1844689995 14 ==== osd_op_reply(89 604.00000000 [omap-get-vals out=5b] v0'0 uv2 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5636228e5440 con 0x563622915400 2023-05-20T01:28:50.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.528+0000 7f6d6a7be7c0 10 datascan.scan_links: step 2: handling object 500.00000000 2023-05-20T01:28:50.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.528+0000 7f6d6a7be7c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2023-05-20T01:28:50.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.528+0000 7f6d6a7be7c0 10 datascan.scan_links: step 2: handling object mds_snaptable 2023-05-20T01:28:50.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.528+0000 7f6d6a7be7c0 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2023-05-20T01:28:50.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.528+0000 7f6d6a7be7c0 10 datascan.scan_links: step 2: handling object 605.00000000 2023-05-20T01:28:50.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.528+0000 7f6d6a7be7c0 1 -- 172.21.15.16:0/4119872052 --> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] -- osd_op(unknown.0.0:90 18.1d 18:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563622a51c00 con 0x563622912800 2023-05-20T01:28:50.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.528+0000 7f6d67f6b700 1 -- 172.21.15.16:0/4119872052 <== osd.0 v2:172.21.15.16:6824/3923634179 14 ==== osd_op_reply(90 605.00000000 [omap-get-vals out=5b] v0'0 uv2 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5636228e5440 con 0x563622912800 2023-05-20T01:28:50.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.528+0000 7f6d6a7be7c0 10 datascan.scan_links: step 2: handling object mds0_inotable 2023-05-20T01:28:50.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.528+0000 7f6d6a7be7c0 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2023-05-20T01:28:50.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.528+0000 7f6d6a7be7c0 10 datascan.scan_links: step 2: handling object 100.00000000 2023-05-20T01:28:50.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.528+0000 7f6d6a7be7c0 1 -- 172.21.15.16:0/4119872052 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:91 18.13 18:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563623454a80 con 0x563622915c00 2023-05-20T01:28:50.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.528+0000 7f6d68f6d700 1 -- 172.21.15.16:0/4119872052 <== osd.7 v2:172.21.15.132:6800/466001149 11 ==== osd_op_reply(91 100.00000000 [omap-get-vals out=5195b] v0'0 uv1 ondisk = 0) v8 ==== 156+0+5195 (crc 0 0 0) 0x5636228e5680 con 0x563622915c00 2023-05-20T01:28:50.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.529+0000 7f6d6a7be7c0 10 datascan.scan_links: step 2: handling object mds0_sessionmap 2023-05-20T01:28:50.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.529+0000 7f6d6a7be7c0 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2023-05-20T01:28:50.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.529+0000 7f6d6a7be7c0 10 datascan.scan_links: step 2: handling object 10000000000.01800000 2023-05-20T01:28:50.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.529+0000 7f6d6a7be7c0 1 -- 172.21.15.16:0/4119872052 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:92 18.1b 18:dcc776e0:::10000000000.01800000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563623454e00 con 0x563622914c00 2023-05-20T01:28:50.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.529+0000 7f6d6876c700 1 -- 172.21.15.16:0/4119872052 <== osd.1 v2:172.21.15.16:6808/3860264220 18 ==== osd_op_reply(92 10000000000.01800000 [omap-get-vals out=28078b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+28078 (crc 0 0 0) 0x5636228e58c0 con 0x563622914c00 2023-05-20T01:28:50.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.529+0000 7f6d6a7be7c0 10 datascan.scan_links: step 2: handling object 609.00000000 2023-05-20T01:28:50.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.529+0000 7f6d6a7be7c0 1 -- 172.21.15.16:0/4119872052 --> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] -- osd_op(unknown.0.0:93 18.7 18:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563623455180 con 0x563622915800 2023-05-20T01:28:50.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.529+0000 7f6d6876c700 1 -- 172.21.15.16:0/4119872052 <== osd.2 v2:172.21.15.16:6816/293187030 6 ==== osd_op_reply(93 609.00000000 [omap-get-vals out=5b] v0'0 uv2 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5636228e58c0 con 0x563622915800 2023-05-20T01:28:50.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.529+0000 7f6d6a7be7c0 10 datascan.scan_links: step 2: handling object 400.00000000 2023-05-20T01:28:50.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.529+0000 7f6d6a7be7c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2023-05-20T01:28:50.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.529+0000 7f6d6a7be7c0 10 datascan.scan_links: step 2: handling object 100.00000000.inode 2023-05-20T01:28:50.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.529+0000 7f6d6a7be7c0 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2023-05-20T01:28:50.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.529+0000 7f6d6a7be7c0 10 datascan.scan_links: step 2: handling object 1.00000000 2023-05-20T01:28:50.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.529+0000 7f6d6a7be7c0 1 -- 172.21.15.16:0/4119872052 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:94 18.1f 18:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563623455c00 con 0x563622915c00 2023-05-20T01:28:50.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.530+0000 7f6d68f6d700 1 -- 172.21.15.16:0/4119872052 <== osd.7 v2:172.21.15.132:6800/466001149 12 ==== osd_op_reply(94 1.00000000 [omap-get-vals out=590b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+590 (crc 0 0 0) 0x5636228e5680 con 0x563622915c00 2023-05-20T01:28:50.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.530+0000 7f6d6a7be7c0 10 datascan.scan_links: processing 0 dup_primaries, 0 remote_links 2023-05-20T01:28:50.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.530+0000 7f6d6a7be7c0 10 datascan.scan_links: removing dup dentries from 0 objects 2023-05-20T01:28:50.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.530+0000 7f6d6a7be7c0 10 datascan.scan_links: processing 0 bad_nlink_inos 2023-05-20T01:28:50.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.530+0000 7f6d6a7be7c0 10 datascan.scan_links: processing 1 injected_inos 2023-05-20T01:28:50.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.530+0000 7f6d6a7be7c0 10 datascan.scan_links: handling injected_ino 0x1000000020a 2023-05-20T01:28:50.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.530+0000 7f6d6a7be7c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:21 2023-05-20T01:28:50.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.530+0000 7f6d6a7be7c0 1 -- 172.21.15.16:0/4119872052 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:95 18.5 18:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563622a50380 con 0x563622914c00 2023-05-20T01:28:50.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.530+0000 7f6d6876c700 1 -- 172.21.15.16:0/4119872052 <== osd.1 v2:172.21.15.16:6808/3860264220 19 ==== osd_op_reply(95 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x5636228e58c0 con 0x563622914c00 2023-05-20T01:28:50.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.530+0000 7f6d6a7be7c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=21 frag=0*, r=0 2023-05-20T01:28:50.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.530+0000 7f6d6a7be7c0 20 datascan.scan_links: first is now 2 2023-05-20T01:28:50.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.530+0000 7f6d6a7be7c0 1 -- 172.21.15.16:0/4119872052 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:96 18.5 18:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) v8 -- 0x563622a50700 con 0x563622914c00 2023-05-20T01:28:50.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.531+0000 7f6d6876c700 1 -- 172.21.15.16:0/4119872052 <== osd.1 v2:172.21.15.16:6808/3860264220 20 ==== osd_op_reply(96 10000000000.01000000 [omap-set-vals] v138'5 uv5 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5636228e58c0 con 0x563622914c00 2023-05-20T01:28:50.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.531+0000 7f6d6a7be7c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/21 pointing to 0x0x1000000020a 2023-05-20T01:28:50.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.531+0000 7f6d6a7be7c0 10 datascan.scan_links: updating inotable 2023-05-20T01:28:50.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.531+0000 7f6d6a7be7c0 1 -- 172.21.15.16:0/4119872052 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:97 18.13 18:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563622a50a80 con 0x563622915c00 2023-05-20T01:28:50.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.532+0000 7f6d68f6d700 1 -- 172.21.15.16:0/4119872052 <== osd.7 v2:172.21.15.132:6800/466001149 13 ==== osd_op_reply(97 mds0_inotable [read 0~50 out=50b] v0'0 uv3 ondisk = 0) v8 ==== 157+0+50 (crc 0 0 0) 0x5636228e5680 con 0x563622915c00 2023-05-20T01:28:50.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.532+0000 7f6d6a7be7c0 10 mds.0.inotable: skip_inos was [0x10000000002~0x1f4,0x10000000259~0xfffffffda7] 2023-05-20T01:28:50.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.532+0000 7f6d6a7be7c0 10 mds.0.inotable: skip_inos now [0x10000000259~0xfffffffda7] 2023-05-20T01:28:50.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.532+0000 7f6d6a7be7c0 1 -- 172.21.15.16:0/4119872052 --> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] -- osd_op(unknown.0.0:98 18.13 18:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) v8 -- 0x563623454a80 con 0x563622915c00 2023-05-20T01:28:50.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.533+0000 7f6d68f6d700 1 -- 172.21.15.16:0/4119872052 <== osd.7 v2:172.21.15.132:6800/466001149 14 ==== osd_op_reply(98 mds0_inotable [writefull 0~34] v138'4 uv4 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x5636228e5680 con 0x563622915c00 2023-05-20T01:28:50.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.533+0000 7f6d6a7be7c0 10 datascan.scan_links: updating snaptable 2023-05-20T01:28:50.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.533+0000 7f6d6a7be7c0 1 -- 172.21.15.16:0/4119872052 --> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] -- osd_op(unknown.0.0:99 18.d 18:b50e409b:::mds_snaptable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x563623455c00 con 0x563622915000 2023-05-20T01:28:50.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.534+0000 7f6d68f6d700 1 -- 172.21.15.16:0/4119872052 <== osd.4 v2:172.21.15.132:6817/506317223 5 ==== osd_op_reply(99 mds_snaptable [read 0~70 out=70b] v0'0 uv1 ondisk = 0) v8 ==== 157+0+70 (crc 0 0 0) 0x5636228e5680 con 0x563622915000 2023-05-20T01:28:50.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.534+0000 7f6d6a7be7c0 1 -- 172.21.15.16:0/4119872052 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x563622912800 msgr2=0x563622670100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:50.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.534+0000 7f6d6a7be7c0 1 --2- 172.21.15.16:0/4119872052 >> [v2:172.21.15.16:6824/3923634179,v1:172.21.15.16:6825/3923634179] conn(0x563622912800 0x563622670100 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:50.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.534+0000 7f6d6a7be7c0 1 -- 172.21.15.16:0/4119872052 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x563622914c00 msgr2=0x563622a14580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:50.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.534+0000 7f6d6a7be7c0 1 --2- 172.21.15.16:0/4119872052 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x563622914c00 0x563622a14580 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:50.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.534+0000 7f6d6a7be7c0 1 -- 172.21.15.16:0/4119872052 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x563622915800 msgr2=0x563622a15b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:50.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.534+0000 7f6d6a7be7c0 1 --2- 172.21.15.16:0/4119872052 >> [v2:172.21.15.16:6816/293187030,v1:172.21.15.16:6817/293187030] conn(0x563622915800 0x563622a15b80 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:50.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.534+0000 7f6d6a7be7c0 1 -- 172.21.15.16:0/4119872052 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x563622a30000 msgr2=0x563622a16680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:50.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.534+0000 7f6d6a7be7c0 1 --2- 172.21.15.16:0/4119872052 >> [v2:172.21.15.16:6800/3669781890,v1:172.21.15.16:6801/3669781890] conn(0x563622a30000 0x563622a16680 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:50.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.534+0000 7f6d6a7be7c0 1 -- 172.21.15.16:0/4119872052 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x563622915000 msgr2=0x563622a14b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:50.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.534+0000 7f6d6a7be7c0 1 --2- 172.21.15.16:0/4119872052 >> [v2:172.21.15.132:6817/506317223,v1:172.21.15.132:6819/506317223] conn(0x563622915000 0x563622a14b00 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:50.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.534+0000 7f6d6a7be7c0 1 -- 172.21.15.16:0/4119872052 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x563622915400 msgr2=0x563622a15080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:50.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.534+0000 7f6d6a7be7c0 1 --2- 172.21.15.16:0/4119872052 >> [v2:172.21.15.132:6816/1844689995,v1:172.21.15.132:6818/1844689995] conn(0x563622915400 0x563622a15080 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:50.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.534+0000 7f6d6a7be7c0 1 -- 172.21.15.16:0/4119872052 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x563622914800 msgr2=0x563622a14000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:50.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.534+0000 7f6d6a7be7c0 1 --2- 172.21.15.16:0/4119872052 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x563622914800 0x563622a14000 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:50.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.534+0000 7f6d6876c700 1 -- 172.21.15.16:0/4119872052 reap_dead start 2023-05-20T01:28:50.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.534+0000 7f6d6a7be7c0 1 -- 172.21.15.16:0/4119872052 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x563622915c00 msgr2=0x563622a15600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:50.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.534+0000 7f6d6a7be7c0 1 --2- 172.21.15.16:0/4119872052 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x563622915c00 0x563622a15600 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:50.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.534+0000 7f6d6876c700 1 -- 172.21.15.16:0/4119872052 reap_dead start 2023-05-20T01:28:50.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.534+0000 7f6d6a7be7c0 1 -- 172.21.15.16:0/4119872052 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x563622914400 msgr2=0x563622671700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:50.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.534+0000 7f6d6a7be7c0 1 --2- 172.21.15.16:0/4119872052 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x563622914400 0x563622671700 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x5636228fa1b0 tx=0x56362336d380 comp rx=0 tx=0).stop 2023-05-20T01:28:50.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.534+0000 7f6d6a7be7c0 1 -- 172.21.15.16:0/4119872052 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563622912000 msgr2=0x563622670c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:50.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.534+0000 7f6d6a7be7c0 1 --2- 172.21.15.16:0/4119872052 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563622912000 0x563622670c00 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x5636228fb320 tx=0x563623462c00 comp rx=0 tx=0).stop 2023-05-20T01:28:50.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.534+0000 7f6d6a7be7c0 1 -- 172.21.15.16:0/4119872052 shutdown_connections 2023-05-20T01:28:50.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.534+0000 7f6d6a7be7c0 1 --2- 172.21.15.16:0/4119872052 >> [v2:172.21.15.132:6800/466001149,v1:172.21.15.132:6803/466001149] conn(0x563622915c00 0x563622a15600 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:50.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.534+0000 7f6d6a7be7c0 1 --2- 172.21.15.16:0/4119872052 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x563622914400 0x563622671700 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:50.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.534+0000 7f6d6a7be7c0 1 --2- 172.21.15.16:0/4119872052 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563622912000 0x563622670c00 unknown :-1 s=CLOSED pgs=1726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:50.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.534+0000 7f6d6a7be7c0 1 -- 172.21.15.16:0/4119872052 >> 172.21.15.16:0/4119872052 conn(0x5636225d5c00 msgr2=0x5636226b1800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:50.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.534+0000 7f6d6a7be7c0 1 -- 172.21.15.16:0/4119872052 shutdown_connections 2023-05-20T01:28:50.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.534+0000 7f6d6a7be7c0 1 -- 172.21.15.16:0/4119872052 wait complete. 2023-05-20T01:28:50.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.534+0000 7f6d6a7be7c0 1 -- 172.21.15.16:0/1797047265 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636225d4800 msgr2=0x56362266f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:50.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.534+0000 7f6d6a7be7c0 1 --2- 172.21.15.16:0/1797047265 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636225d4800 0x56362266f080 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x563622734de0 tx=0x56362273ef00 comp rx=0 tx=0).stop 2023-05-20T01:28:50.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.535+0000 7f6d6a7be7c0 1 -- 172.21.15.16:0/1797047265 shutdown_connections 2023-05-20T01:28:50.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.535+0000 7f6d6a7be7c0 1 --2- 172.21.15.16:0/1797047265 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636225d4800 0x56362266f080 unknown :-1 s=CLOSED pgs=1724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:50.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.535+0000 7f6d6a7be7c0 1 --2- 172.21.15.16:0/1797047265 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5636225d5000 0x56362266e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:50.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.535+0000 7f6d6a7be7c0 1 --2- 172.21.15.16:0/1797047265 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5636225d4c00 0x56362266eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:50.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.535+0000 7f6d6a7be7c0 1 -- 172.21.15.16:0/1797047265 >> 172.21.15.16:0/1797047265 conn(0x5636225d5400 msgr2=0x5636226b1000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:50.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.535+0000 7f6d6a7be7c0 1 -- 172.21.15.16:0/1797047265 shutdown_connections 2023-05-20T01:28:50.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.535+0000 7f6d6a7be7c0 1 -- 172.21.15.16:0/1797047265 wait complete. 2023-05-20T01:28:50.572 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_links'] in time 0:00:00.124111, result: 2023-05-20T01:28:50.572 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys 10000000000.01000000 2023-05-20T01:28:50.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.626+0000 7f6596d8b080 1 Processor -- start 2023-05-20T01:28:50.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.626+0000 7f6596d8b080 1 -- start start 2023-05-20T01:28:50.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.627+0000 7f6596d8b080 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b8f843ccd0 0x55b8f8506180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:50.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.627+0000 7f6596d8b080 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55b8f846a590 0x55b8f85139c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:50.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.627+0000 7f6596d8b080 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b8f8510bd0 0x55b8f843c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:50.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.627+0000 7f6596d8b080 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8f83e3a80 con 0x55b8f843ccd0 2023-05-20T01:28:50.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.627+0000 7f6596d8b080 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8f839fe80 con 0x55b8f8510bd0 2023-05-20T01:28:50.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.627+0000 7f6596d8b080 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b8f839fb20 con 0x55b8f846a590 2023-05-20T01:28:50.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.627+0000 7f658ce8b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b8f843ccd0 0x55b8f8506180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:50.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.627+0000 7f658ce8b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b8f843ccd0 0x55b8f8506180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40830/0 (socket says 172.21.15.16:40830) 2023-05-20T01:28:50.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.627+0000 7f658ce8b700 1 -- 172.21.15.16:0/2439023700 learned_addr learned my addr 172.21.15.16:0/2439023700 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:50.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.627+0000 7f658d68c700 1 --2- 172.21.15.16:0/2439023700 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b8f8510bd0 0x55b8f843c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:50.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.627+0000 7f6587fff700 1 --2- 172.21.15.16:0/2439023700 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55b8f846a590 0x55b8f85139c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:50.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.627+0000 7f658ce8b700 1 -- 172.21.15.16:0/2439023700 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55b8f846a590 msgr2=0x55b8f85139c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:50.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.627+0000 7f658ce8b700 1 --2- 172.21.15.16:0/2439023700 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55b8f846a590 0x55b8f85139c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:50.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.628+0000 7f658ce8b700 1 -- 172.21.15.16:0/2439023700 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b8f8510bd0 msgr2=0x55b8f843c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:50.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.628+0000 7f658ce8b700 1 --2- 172.21.15.16:0/2439023700 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b8f8510bd0 0x55b8f843c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:50.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.628+0000 7f658ce8b700 1 -- 172.21.15.16:0/2439023700 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f658800a100 con 0x55b8f843ccd0 2023-05-20T01:28:50.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.628+0000 7f6587fff700 1 --2- 172.21.15.16:0/2439023700 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55b8f846a590 0x55b8f85139c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:28:50.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.628+0000 7f658ce8b700 1 --2- 172.21.15.16:0/2439023700 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b8f843ccd0 0x55b8f8506180 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f6578001cb0 tx=0x7f657800b5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6b392f5941fda3e server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:50.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.628+0000 7f6586ffd700 1 -- 172.21.15.16:0/2439023700 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f657800bd70 con 0x55b8f843ccd0 2023-05-20T01:28:50.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.628+0000 7f6586ffd700 1 -- 172.21.15.16:0/2439023700 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f657800e470 con 0x55b8f843ccd0 2023-05-20T01:28:50.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.628+0000 7f6586ffd700 1 -- 172.21.15.16:0/2439023700 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f657800e820 con 0x55b8f843ccd0 2023-05-20T01:28:50.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.628+0000 7f6596d8b080 1 -- 172.21.15.16:0/2439023700 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b8f843ccd0 msgr2=0x55b8f8506180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:50.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.628+0000 7f6596d8b080 1 --2- 172.21.15.16:0/2439023700 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b8f843ccd0 0x55b8f8506180 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f6578001cb0 tx=0x7f657800b5d0 comp rx=0 tx=0).stop 2023-05-20T01:28:50.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.628+0000 7f6596d8b080 1 -- 172.21.15.16:0/2439023700 shutdown_connections 2023-05-20T01:28:50.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.628+0000 7f6596d8b080 1 --2- 172.21.15.16:0/2439023700 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b8f8510bd0 0x55b8f843c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:50.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.628+0000 7f6596d8b080 1 --2- 172.21.15.16:0/2439023700 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b8f843ccd0 0x55b8f8506180 unknown :-1 s=CLOSED pgs=1728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:50.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.628+0000 7f6596d8b080 1 --2- 172.21.15.16:0/2439023700 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55b8f846a590 0x55b8f85139c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:50.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.628+0000 7f6596d8b080 1 -- 172.21.15.16:0/2439023700 >> 172.21.15.16:0/2439023700 conn(0x55b8f8487580 msgr2=0x55b8f850f240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:50.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.628+0000 7f6596d8b080 1 -- 172.21.15.16:0/2439023700 shutdown_connections 2023-05-20T01:28:50.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.628+0000 7f6596d8b080 1 -- 172.21.15.16:0/2439023700 wait complete. 2023-05-20T01:28:50.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.629+0000 7f6596d8b080 1 Processor -- start 2023-05-20T01:28:50.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.629+0000 7f6596d8b080 1 -- start start 2023-05-20T01:28:50.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.630+0000 7f6596d8b080 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b8f843ccd0 0x55b8f8506320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:50.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.630+0000 7f6596d8b080 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b8f846a590 0x55b8f8506860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:50.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.630+0000 7f6596d8b080 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55b8f8510bd0 0x55b8f85035d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:50.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.630+0000 7f6596d8b080 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8f8375fe0 con 0x55b8f846a590 2023-05-20T01:28:50.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.630+0000 7f6596d8b080 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8f84fce80 con 0x55b8f843ccd0 2023-05-20T01:28:50.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.630+0000 7f6596d8b080 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b8f84f9390 con 0x55b8f8510bd0 2023-05-20T01:28:50.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.630+0000 7f6587fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b8f846a590 0x55b8f8506860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:50.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.630+0000 7f658d68c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55b8f8510bd0 0x55b8f85035d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:50.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.630+0000 7f6587fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b8f846a590 0x55b8f8506860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40840/0 (socket says 172.21.15.16:40840) 2023-05-20T01:28:50.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.630+0000 7f6587fff700 1 -- 172.21.15.16:0/230922356 learned_addr learned my addr 172.21.15.16:0/230922356 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:50.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.630+0000 7f658ce8b700 1 --2- 172.21.15.16:0/230922356 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b8f843ccd0 0x55b8f8506320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:50.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.630+0000 7f6587fff700 1 -- 172.21.15.16:0/230922356 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55b8f8510bd0 msgr2=0x55b8f85035d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:50.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.630+0000 7f6587fff700 1 --2- 172.21.15.16:0/230922356 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55b8f8510bd0 0x55b8f85035d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:50.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.630+0000 7f6587fff700 1 -- 172.21.15.16:0/230922356 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b8f843ccd0 msgr2=0x55b8f8506320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:50.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.630+0000 7f6587fff700 1 --2- 172.21.15.16:0/230922356 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b8f843ccd0 0x55b8f8506320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:50.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.630+0000 7f6587fff700 1 -- 172.21.15.16:0/230922356 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f657c001e10 con 0x55b8f846a590 2023-05-20T01:28:50.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.630+0000 7f658d68c700 1 --2- 172.21.15.16:0/230922356 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55b8f8510bd0 0x55b8f85035d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-05-20T01:28:50.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.630+0000 7f6587fff700 1 --2- 172.21.15.16:0/230922356 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b8f846a590 0x55b8f8506860 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x55b8f8440f70 tx=0x7f6580003e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:50.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.631+0000 7f6585ffb700 1 -- 172.21.15.16:0/230922356 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f658000b990 con 0x55b8f846a590 2023-05-20T01:28:50.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.631+0000 7f6585ffb700 1 -- 172.21.15.16:0/230922356 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6580008040 con 0x55b8f846a590 2023-05-20T01:28:50.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.631+0000 7f6596d8b080 1 -- 172.21.15.16:0/230922356 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55b8f838da80 con 0x55b8f846a590 2023-05-20T01:28:50.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.631+0000 7f6585ffb700 1 -- 172.21.15.16:0/230922356 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f658000bd10 con 0x55b8f846a590 2023-05-20T01:28:50.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.631+0000 7f6596d8b080 1 -- 172.21.15.16:0/230922356 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b8f8381830 con 0x55b8f846a590 2023-05-20T01:28:50.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.632+0000 7f6585ffb700 1 -- 172.21.15.16:0/230922356 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6580011480 con 0x55b8f846a590 2023-05-20T01:28:50.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.632+0000 7f6585ffb700 1 --2- 172.21.15.16:0/230922356 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f6570069600 0x7f657006bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:50.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.632+0000 7f6585ffb700 1 -- 172.21.15.16:0/230922356 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6554+0+0 (secure 0 0 0) 0x7f6580087d90 con 0x55b8f846a590 2023-05-20T01:28:50.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.632+0000 7f658ce8b700 1 --2- 172.21.15.16:0/230922356 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f6570069600 0x7f657006bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:50.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.633+0000 7f6596d8b080 1 --2- 172.21.15.16:0/230922356 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x55b8f8506da0 0x55b8f8511280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:50.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.633+0000 7f6596d8b080 1 -- 172.21.15.16:0/230922356 --> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] -- osd_op(unknown.0.0:1 18.5 18:a40620de:::10000000000.01000000:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x55b8f84fd550 con 0x55b8f8506da0 2023-05-20T01:28:50.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.633+0000 7f658d68c700 1 --2- 172.21.15.16:0/230922356 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x55b8f8506da0 0x55b8f8511280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:50.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.633+0000 7f658ce8b700 1 --2- 172.21.15.16:0/230922356 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f6570069600 0x7f657006bab0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f6578001cb0 tx=0x7f657801b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:50.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.633+0000 7f658d68c700 1 --2- 172.21.15.16:0/230922356 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x55b8f8506da0 0x55b8f8511280 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:50.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.633+0000 7f658d68c700 1 -- 172.21.15.16:0/230922356 <== osd.1 v2:172.21.15.16:6808/3860264220 1 ==== osd_op_reply(1 10000000000.01000000 [omap-get-keys out=560b] v0'0 uv5 ondisk = 0) v8 ==== 164+0+560 (crc 0 0 0) 0x7f658800fb60 con 0x55b8f8506da0 2023-05-20T01:28:50.645 INFO:teuthology.orchestra.run.smithi016.stdout:10_head 2023-05-20T01:28:50.645 INFO:teuthology.orchestra.run.smithi016.stdout:11_head 2023-05-20T01:28:50.645 INFO:teuthology.orchestra.run.smithi016.stdout:12_head 2023-05-20T01:28:50.645 INFO:teuthology.orchestra.run.smithi016.stdout:15_head 2023-05-20T01:28:50.645 INFO:teuthology.orchestra.run.smithi016.stdout:19_head 2023-05-20T01:28:50.645 INFO:teuthology.orchestra.run.smithi016.stdout:1_head 2023-05-20T01:28:50.645 INFO:teuthology.orchestra.run.smithi016.stdout:20_head 2023-05-20T01:28:50.646 INFO:teuthology.orchestra.run.smithi016.stdout:21_head 2023-05-20T01:28:50.646 INFO:teuthology.orchestra.run.smithi016.stdout:27_head 2023-05-20T01:28:50.646 INFO:teuthology.orchestra.run.smithi016.stdout:28_head 2023-05-20T01:28:50.646 INFO:teuthology.orchestra.run.smithi016.stdout:2_head 2023-05-20T01:28:50.646 INFO:teuthology.orchestra.run.smithi016.stdout:31_head 2023-05-20T01:28:50.646 INFO:teuthology.orchestra.run.smithi016.stdout:34_head 2023-05-20T01:28:50.647 INFO:teuthology.orchestra.run.smithi016.stdout:36_head 2023-05-20T01:28:50.647 INFO:teuthology.orchestra.run.smithi016.stdout:38_head 2023-05-20T01:28:50.647 INFO:teuthology.orchestra.run.smithi016.stdout:39_head 2023-05-20T01:28:50.647 INFO:teuthology.orchestra.run.smithi016.stdout:3_head 2023-05-20T01:28:50.648 INFO:teuthology.orchestra.run.smithi016.stdout:42_head 2023-05-20T01:28:50.648 INFO:teuthology.orchestra.run.smithi016.stdout:43_head 2023-05-20T01:28:50.648 INFO:teuthology.orchestra.run.smithi016.stdout:46_head 2023-05-20T01:28:50.648 INFO:teuthology.orchestra.run.smithi016.stdout:4_head 2023-05-20T01:28:50.649 INFO:teuthology.orchestra.run.smithi016.stdout:50_head 2023-05-20T01:28:50.649 INFO:teuthology.orchestra.run.smithi016.stdout:52_head 2023-05-20T01:28:50.649 INFO:teuthology.orchestra.run.smithi016.stdout:53_head 2023-05-20T01:28:50.649 INFO:teuthology.orchestra.run.smithi016.stdout:55_head 2023-05-20T01:28:50.650 INFO:teuthology.orchestra.run.smithi016.stdout:59_head 2023-05-20T01:28:50.650 INFO:teuthology.orchestra.run.smithi016.stdout:61_head 2023-05-20T01:28:50.650 INFO:teuthology.orchestra.run.smithi016.stdout:63_head 2023-05-20T01:28:50.651 INFO:teuthology.orchestra.run.smithi016.stdout:64_head 2023-05-20T01:28:50.651 INFO:teuthology.orchestra.run.smithi016.stdout:65_head 2023-05-20T01:28:50.651 INFO:teuthology.orchestra.run.smithi016.stdout:66_head 2023-05-20T01:28:50.651 INFO:teuthology.orchestra.run.smithi016.stdout:69_head 2023-05-20T01:28:50.651 INFO:teuthology.orchestra.run.smithi016.stdout:70_head 2023-05-20T01:28:50.652 INFO:teuthology.orchestra.run.smithi016.stdout:73_head 2023-05-20T01:28:50.652 INFO:teuthology.orchestra.run.smithi016.stdout:74_head 2023-05-20T01:28:50.652 INFO:teuthology.orchestra.run.smithi016.stdout:75_head 2023-05-20T01:28:50.652 INFO:teuthology.orchestra.run.smithi016.stdout:77_head 2023-05-20T01:28:50.652 INFO:teuthology.orchestra.run.smithi016.stdout:80_head 2023-05-20T01:28:50.652 INFO:teuthology.orchestra.run.smithi016.stdout:83_head 2023-05-20T01:28:50.652 INFO:teuthology.orchestra.run.smithi016.stdout:84_head 2023-05-20T01:28:50.653 INFO:teuthology.orchestra.run.smithi016.stdout:85_head 2023-05-20T01:28:50.653 INFO:teuthology.orchestra.run.smithi016.stdout:86_head 2023-05-20T01:28:50.653 INFO:teuthology.orchestra.run.smithi016.stdout:87_head 2023-05-20T01:28:50.653 INFO:teuthology.orchestra.run.smithi016.stdout:88_head 2023-05-20T01:28:50.653 INFO:teuthology.orchestra.run.smithi016.stdout:8_head 2023-05-20T01:28:50.653 INFO:teuthology.orchestra.run.smithi016.stdout:90_head 2023-05-20T01:28:50.653 INFO:teuthology.orchestra.run.smithi016.stdout:92_head 2023-05-20T01:28:50.654 INFO:teuthology.orchestra.run.smithi016.stdout:95_head 2023-05-20T01:28:50.654 INFO:teuthology.orchestra.run.smithi016.stdout:96_head 2023-05-20T01:28:50.654 INFO:teuthology.orchestra.run.smithi016.stdout:97_head 2023-05-20T01:28:50.654 INFO:teuthology.orchestra.run.smithi016.stdout:9_head 2023-05-20T01:28:50.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.634+0000 7f6596d8b080 1 -- 172.21.15.16:0/230922356 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x55b8f8506da0 msgr2=0x55b8f8511280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:50.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.634+0000 7f6596d8b080 1 --2- 172.21.15.16:0/230922356 >> [v2:172.21.15.16:6808/3860264220,v1:172.21.15.16:6809/3860264220] conn(0x55b8f8506da0 0x55b8f8511280 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:50.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.634+0000 7f6596d8b080 1 -- 172.21.15.16:0/230922356 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f6570069600 msgr2=0x7f657006bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:50.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.634+0000 7f6596d8b080 1 --2- 172.21.15.16:0/230922356 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f6570069600 0x7f657006bab0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f6578001cb0 tx=0x7f657801b040 comp rx=0 tx=0).stop 2023-05-20T01:28:50.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.634+0000 7f6596d8b080 1 -- 172.21.15.16:0/230922356 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b8f846a590 msgr2=0x55b8f8506860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:50.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.634+0000 7f6596d8b080 1 --2- 172.21.15.16:0/230922356 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b8f846a590 0x55b8f8506860 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x55b8f8440f70 tx=0x7f6580003e90 comp rx=0 tx=0).stop 2023-05-20T01:28:50.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.634+0000 7f658d68c700 1 -- 172.21.15.16:0/230922356 reap_dead start 2023-05-20T01:28:50.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.634+0000 7f6596d8b080 1 -- 172.21.15.16:0/230922356 shutdown_connections 2023-05-20T01:28:50.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.634+0000 7f6596d8b080 1 -- 172.21.15.16:0/230922356 >> 172.21.15.16:0/230922356 conn(0x55b8f8487580 msgr2=0x55b8f850f240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:50.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.634+0000 7f6596d8b080 1 -- 172.21.15.16:0/230922356 shutdown_connections 2023-05-20T01:28:50.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.634+0000 7f6596d8b080 1 -- 172.21.15.16:0/230922356 wait complete. 2023-05-20T01:28:50.658 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-05-20T01:28:50.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.784+0000 7fd5827de700 1 Processor -- start 2023-05-20T01:28:50.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.785+0000 7fd5827de700 1 -- start start 2023-05-20T01:28:50.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.785+0000 7fd5827de700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd57c159850 0x7fd57c0ce120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:50.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.785+0000 7fd5827de700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fd57c072480 0x7fd57c0ce660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:50.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.785+0000 7fd5827de700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd57c06de50 con 0x7fd57c06f740 2023-05-20T01:28:50.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.785+0000 7fd5827de700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd57c06e380 con 0x7fd57c072480 2023-05-20T01:28:50.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.785+0000 7fd5827de700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd57c15d450 con 0x7fd57c159850 2023-05-20T01:28:50.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.786+0000 7fd57b7fe700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fd57c072480 0x7fd57c0ce660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:50.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.786+0000 7fd57bfff700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd57c159850 0x7fd57c0ce120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:50.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.786+0000 7fd580d7b700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7fd57c06f740 0x7fd57c071450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.16:33858/0 (socket says 172.21.15.16:33858) 2023-05-20T01:28:50.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.786+0000 7fd580d7b700 1 -- 172.21.15.16:0/1341400402 learned_addr learned my addr 172.21.15.16:0/1341400402 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:28:50.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.786+0000 7fd57b7fe700 1 -- 172.21.15.16:0/1341400402 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd57c159850 msgr2=0x7fd57c0ce120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:50.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.786+0000 7fd57b7fe700 1 --2- 172.21.15.16:0/1341400402 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd57c159850 0x7fd57c0ce120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:50.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.786+0000 7fd57b7fe700 1 -- 172.21.15.16:0/1341400402 >> v1:172.21.15.132:6789/0 conn(0x7fd57c06f740 legacy=0x7fd57c071450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:50.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.786+0000 7fd57affd700 1 -- 172.21.15.16:0/1341400402 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3727543016 0 0) 0x7fd57c06de50 con 0x7fd57c06f740 2023-05-20T01:28:50.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.786+0000 7fd57b7fe700 1 -- 172.21.15.16:0/1341400402 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd57c06e0a0 con 0x7fd57c072480 2023-05-20T01:28:50.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.786+0000 7fd57bfff700 1 --2- 172.21.15.16:0/1341400402 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd57c159850 0x7fd57c0ce120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-05-20T01:28:50.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.786+0000 7fd57b7fe700 1 --2- 172.21.15.16:0/1341400402 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd57c072480 0x7fd57c0ce660 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fd570009840 tx=0x7fd570009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ed67d8f0c42d665 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:50.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.786+0000 7fd57affd700 1 -- 172.21.15.16:0/1341400402 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd570011830 con 0x7fd57c072480 2023-05-20T01:28:50.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.786+0000 7fd57affd700 1 -- 172.21.15.16:0/1341400402 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd570011990 con 0x7fd57c072480 2023-05-20T01:28:50.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.786+0000 7fd57affd700 1 -- 172.21.15.16:0/1341400402 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd570011d20 con 0x7fd57c072480 2023-05-20T01:28:50.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.787+0000 7fd5827de700 1 -- 172.21.15.16:0/1341400402 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd57c072480 msgr2=0x7fd57c0ce660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:50.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.787+0000 7fd5827de700 1 --2- 172.21.15.16:0/1341400402 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd57c072480 0x7fd57c0ce660 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fd570009840 tx=0x7fd570009b50 comp rx=0 tx=0).stop 2023-05-20T01:28:50.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.787+0000 7fd5827de700 1 -- 172.21.15.16:0/1341400402 shutdown_connections 2023-05-20T01:28:50.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.787+0000 7fd5827de700 1 --2- 172.21.15.16:0/1341400402 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd57c159850 0x7fd57c0ce120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:50.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.787+0000 7fd5827de700 1 --2- 172.21.15.16:0/1341400402 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd57c072480 0x7fd57c0ce660 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:50.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.787+0000 7fd5827de700 1 -- 172.21.15.16:0/1341400402 >> 172.21.15.16:0/1341400402 conn(0x7fd57c0b1d90 msgr2=0x7fd57c0b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:50.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.787+0000 7fd5827de700 1 -- 172.21.15.16:0/1341400402 shutdown_connections 2023-05-20T01:28:50.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.787+0000 7fd5827de700 1 -- 172.21.15.16:0/1341400402 wait complete. 2023-05-20T01:28:50.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.788+0000 7fd5827de700 1 Processor -- start 2023-05-20T01:28:50.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.788+0000 7fd5827de700 1 -- start start 2023-05-20T01:28:50.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.789+0000 7fd5827de700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd57c06f740 0x7fd57c0c6530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:50.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.789+0000 7fd5827de700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd57c072480 0x7fd57c159c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:50.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.789+0000 7fd5827de700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd57c159850 0x7fd57c1e4380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:50.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.789+0000 7fd5827de700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd57c15a160 con 0x7fd57c072480 2023-05-20T01:28:50.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.789+0000 7fd5827de700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd57c15a370 con 0x7fd57c159850 2023-05-20T01:28:50.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.789+0000 7fd5827de700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd57c15a650 con 0x7fd57c06f740 2023-05-20T01:28:50.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.789+0000 7fd57b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd57c072480 0x7fd57c159c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:50.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.789+0000 7fd57b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd57c072480 0x7fd57c159c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40854/0 (socket says 172.21.15.16:40854) 2023-05-20T01:28:50.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.789+0000 7fd57b7fe700 1 -- 172.21.15.16:0/933181396 learned_addr learned my addr 172.21.15.16:0/933181396 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:50.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.789+0000 7fd57bfff700 1 --2- 172.21.15.16:0/933181396 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd57c06f740 0x7fd57c0c6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:50.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.789+0000 7fd580d7b700 1 --2- 172.21.15.16:0/933181396 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd57c159850 0x7fd57c1e4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:50.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.790+0000 7fd57b7fe700 1 -- 172.21.15.16:0/933181396 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd57c06f740 msgr2=0x7fd57c0c6530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:50.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.790+0000 7fd57b7fe700 1 --2- 172.21.15.16:0/933181396 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd57c06f740 0x7fd57c0c6530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:50.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.790+0000 7fd57b7fe700 1 -- 172.21.15.16:0/933181396 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd57c159850 msgr2=0x7fd57c1e4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:50.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.790+0000 7fd57b7fe700 1 --2- 172.21.15.16:0/933181396 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd57c159850 0x7fd57c1e4380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:50.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.790+0000 7fd57b7fe700 1 -- 172.21.15.16:0/933181396 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd56c0094a0 con 0x7fd57c072480 2023-05-20T01:28:50.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.790+0000 7fd57b7fe700 1 --2- 172.21.15.16:0/933181396 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd57c072480 0x7fd57c159c20 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7fd570009fd0 tx=0x7fd570004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:50.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.790+0000 7fd578ff9700 1 -- 172.21.15.16:0/933181396 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5700036a0 con 0x7fd57c072480 2023-05-20T01:28:50.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.790+0000 7fd578ff9700 1 -- 172.21.15.16:0/933181396 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd57001f070 con 0x7fd57c072480 2023-05-20T01:28:50.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.790+0000 7fd5827de700 1 -- 172.21.15.16:0/933181396 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5700094a0 con 0x7fd57c072480 2023-05-20T01:28:50.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.790+0000 7fd578ff9700 1 -- 172.21.15.16:0/933181396 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd570003a60 con 0x7fd57c072480 2023-05-20T01:28:50.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.790+0000 7fd5827de700 1 -- 172.21.15.16:0/933181396 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd57c15d980 con 0x7fd57c072480 2023-05-20T01:28:50.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.791+0000 7fd5827de700 1 -- 172.21.15.16:0/933181396 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd57c15d6a0 con 0x7fd57c072480 2023-05-20T01:28:50.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.791+0000 7fd578ff9700 1 -- 172.21.15.16:0/933181396 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd570003bc0 con 0x7fd57c072480 2023-05-20T01:28:50.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.792+0000 7fd578ff9700 1 --2- 172.21.15.16:0/933181396 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd56806ace0 0x7fd56806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:50.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.792+0000 7fd578ff9700 1 -- 172.21.15.16:0/933181396 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6554+0+0 (secure 0 0 0) 0x7fd570021070 con 0x7fd57c072480 2023-05-20T01:28:50.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.792+0000 7fd57bfff700 1 --2- 172.21.15.16:0/933181396 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd56806ace0 0x7fd56806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:50.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.792+0000 7fd57bfff700 1 --2- 172.21.15.16:0/933181396 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd56806ace0 0x7fd56806d190 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fd56c009cd0 tx=0x7fd56c009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:50.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:50.796+0000 7fd578ff9700 1 -- 172.21.15.16:0/933181396 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fd570055d80 con 0x7fd57c072480 2023-05-20T01:28:51.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:51.042+0000 7fd5827de700 1 -- 172.21.15.16:0/933181396 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7fd57c15dc60 con 0x7fd57c072480 2023-05-20T01:28:51.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:51.613+0000 7fd578ff9700 1 -- 172.21.15.16:0/933181396 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v108) v1 ==== 162+0+0 (secure 0 0 0) 0x7fd5700150b0 con 0x7fd57c072480 2023-05-20T01:28:51.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:51.616+0000 7fd5827de700 1 -- 172.21.15.16:0/933181396 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd56806ace0 msgr2=0x7fd56806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:51.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:51.616+0000 7fd5827de700 1 --2- 172.21.15.16:0/933181396 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd56806ace0 0x7fd56806d190 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fd56c009cd0 tx=0x7fd56c009210 comp rx=0 tx=0).stop 2023-05-20T01:28:51.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:51.616+0000 7fd5827de700 1 -- 172.21.15.16:0/933181396 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd57c072480 msgr2=0x7fd57c159c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:51.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:51.616+0000 7fd5827de700 1 --2- 172.21.15.16:0/933181396 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd57c072480 0x7fd57c159c20 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7fd570009fd0 tx=0x7fd570004750 comp rx=0 tx=0).stop 2023-05-20T01:28:51.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:51.616+0000 7fd5827de700 1 -- 172.21.15.16:0/933181396 shutdown_connections 2023-05-20T01:28:51.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:51.616+0000 7fd5827de700 1 --2- 172.21.15.16:0/933181396 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd57c159850 0x7fd57c1e4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:51.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:51.616+0000 7fd5827de700 1 --2- 172.21.15.16:0/933181396 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fd56806ace0 0x7fd56806d190 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:51.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:51.616+0000 7fd5827de700 1 --2- 172.21.15.16:0/933181396 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd57c06f740 0x7fd57c0c6530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:51.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:51.616+0000 7fd5827de700 1 --2- 172.21.15.16:0/933181396 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd57c072480 0x7fd57c159c20 unknown :-1 s=CLOSED pgs=1730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:51.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:51.616+0000 7fd5827de700 1 -- 172.21.15.16:0/933181396 >> 172.21.15.16:0/933181396 conn(0x7fd57c0b1d90 msgr2=0x7fd57c0cddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:51.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:51.616+0000 7fd5827de700 1 -- 172.21.15.16:0/933181396 shutdown_connections 2023-05-20T01:28:51.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:51.616+0000 7fd5827de700 1 -- 172.21.15.16:0/933181396 wait complete. 2023-05-20T01:28:51.622 INFO:teuthology.orchestra.run.smithi016.stderr:cephfs marked joinable; MDS may join as newly active. 2023-05-20T01:28:51.639 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-20T01:28:51.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:51.768+0000 7fecfaec5700 1 Processor -- start 2023-05-20T01:28:51.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:51.768+0000 7fecfaec5700 1 -- start start 2023-05-20T01:28:51.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:51.769+0000 7fecfaec5700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fecf406ab70 0x7fecf406f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:51.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:51.769+0000 7fecfaec5700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecf406fd90 con 0x7fecf4157400 2023-05-20T01:28:51.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:51.769+0000 7fecfaec5700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecf415d8d0 con 0x7fecf415d3d0 2023-05-20T01:28:51.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:51.769+0000 7fecfaec5700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecf415dbb0 con 0x7fecf406ab70 2023-05-20T01:28:51.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:51.769+0000 7fecf8c61700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fecf4157400 0x7fecf4072100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:52400/0 (socket says 172.21.15.16:52400) 2023-05-20T01:28:51.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:51.769+0000 7fecf8c61700 1 -- 172.21.15.16:0/2678673521 learned_addr learned my addr 172.21.15.16:0/2678673521 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:28:51.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:51.769+0000 7fecf9462700 1 --2- 172.21.15.16:0/2678673521 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fecf406ab70 0x7fecf406f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:51.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:51.769+0000 7fecf37fe700 1 -- 172.21.15.16:0/2678673521 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1552714321 0 0) 0x7fecf406fd90 con 0x7fecf4157400 2023-05-20T01:28:51.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:51.769+0000 7fecf37fe700 1 -- 172.21.15.16:0/2678673521 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fece40035b0 con 0x7fecf4157400 2023-05-20T01:28:51.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:51.769+0000 7fecf37fe700 1 -- 172.21.15.16:0/2678673521 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2048984741 0 0) 0x7fecf415d8d0 con 0x7fecf415d3d0 2023-05-20T01:28:51.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:51.769+0000 7fecf37fe700 1 -- 172.21.15.16:0/2678673521 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fecf406fd90 con 0x7fecf415d3d0 2023-05-20T01:28:51.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:51.769+0000 7fecf9462700 1 -- 172.21.15.16:0/2678673521 >> v1:172.21.15.132:6789/0 conn(0x7fecf415d3d0 legacy=0x7fecf406a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:51.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:51.769+0000 7fecf37fe700 1 -- 172.21.15.16:0/2678673521 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2254768242 0 0) 0x7fece40035b0 con 0x7fecf4157400 2023-05-20T01:28:51.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:51.769+0000 7fecf9462700 1 -- 172.21.15.16:0/2678673521 >> v1:172.21.15.16:6789/0 conn(0x7fecf4157400 legacy=0x7fecf4072100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:51.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:51.769+0000 7fecf9462700 1 -- 172.21.15.16:0/2678673521 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecf415de90 con 0x7fecf406ab70 2023-05-20T01:28:51.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:51.770+0000 7fecf9462700 1 --2- 172.21.15.16:0/2678673521 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fecf406ab70 0x7fecf406f850 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fecec009dc0 tx=0x7fecec00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80addf142b6c0751 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:51.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:51.770+0000 7fecf37fe700 1 -- 172.21.15.16:0/2678673521 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecec00e070 con 0x7fecf406ab70 2023-05-20T01:28:51.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:51.770+0000 7fecf37fe700 1 -- 172.21.15.16:0/2678673521 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecec003a20 con 0x7fecf406ab70 2023-05-20T01:28:51.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:51.770+0000 7fecf37fe700 1 -- 172.21.15.16:0/2678673521 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fece8001670 con 0x7fecf406ab70 2023-05-20T01:28:51.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:51.770+0000 7fecfaec5700 1 -- 172.21.15.16:0/2678673521 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fecf406ab70 msgr2=0x7fecf406f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:51.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:51.770+0000 7fecfaec5700 1 --2- 172.21.15.16:0/2678673521 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fecf406ab70 0x7fecf406f850 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fecec009dc0 tx=0x7fecec00b330 comp rx=0 tx=0).stop 2023-05-20T01:28:51.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:51.770+0000 7fecfaec5700 1 -- 172.21.15.16:0/2678673521 shutdown_connections 2023-05-20T01:28:51.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:51.770+0000 7fecfaec5700 1 --2- 172.21.15.16:0/2678673521 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fecf406ab70 0x7fecf406f850 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:51.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:51.770+0000 7fecfaec5700 1 -- 172.21.15.16:0/2678673521 >> 172.21.15.16:0/2678673521 conn(0x7fecf40cd930 msgr2=0x7fecf40d0220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:51.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:51.770+0000 7fecfaec5700 1 -- 172.21.15.16:0/2678673521 shutdown_connections 2023-05-20T01:28:51.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:51.770+0000 7fecfaec5700 1 -- 172.21.15.16:0/2678673521 wait complete. 2023-05-20T01:28:51.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:51.772+0000 7fecfaec5700 1 Processor -- start 2023-05-20T01:28:51.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:51.772+0000 7fecfaec5700 1 -- start start 2023-05-20T01:28:51.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:51.773+0000 7fecfaec5700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fecf406ab70 0x7fecf406f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:51.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:51.773+0000 7fecfaec5700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fecf4157400 0x7fecf4158750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:51.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:51.773+0000 7fecfaec5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fecf415d3d0 0x7fecf41f1f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:51.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:51.773+0000 7fecfaec5700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecf40b3050 con 0x7fecf415d3d0 2023-05-20T01:28:51.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:51.773+0000 7fecfaec5700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecf40b1a80 con 0x7fecf406ab70 2023-05-20T01:28:51.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:51.773+0000 7fecfaec5700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecf40b32a0 con 0x7fecf4157400 2023-05-20T01:28:51.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:51.773+0000 7fecf9462700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fecf415d3d0 0x7fecf41f1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:51.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:51.773+0000 7fecf9462700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fecf415d3d0 0x7fecf41f1f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40882/0 (socket says 172.21.15.16:40882) 2023-05-20T01:28:51.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:51.773+0000 7fecf9462700 1 -- 172.21.15.16:0/718238549 learned_addr learned my addr 172.21.15.16:0/718238549 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:51.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:51.773+0000 7fecf3fff700 1 --2- 172.21.15.16:0/718238549 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fecf4157400 0x7fecf4158750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:51.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:51.773+0000 7fecf8c61700 1 --2- 172.21.15.16:0/718238549 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fecf406ab70 0x7fecf406f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:51.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:51.773+0000 7fecf9462700 1 -- 172.21.15.16:0/718238549 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fecf4157400 msgr2=0x7fecf4158750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:51.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:51.773+0000 7fecf9462700 1 --2- 172.21.15.16:0/718238549 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fecf4157400 0x7fecf4158750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:51.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:51.773+0000 7fecf9462700 1 -- 172.21.15.16:0/718238549 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fecf406ab70 msgr2=0x7fecf406f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:51.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:51.773+0000 7fecf9462700 1 --2- 172.21.15.16:0/718238549 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fecf406ab70 0x7fecf406f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:51.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:51.773+0000 7fecf9462700 1 -- 172.21.15.16:0/718238549 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecec009a70 con 0x7fecf415d3d0 2023-05-20T01:28:51.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:51.773+0000 7fecf9462700 1 --2- 172.21.15.16:0/718238549 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fecf415d3d0 0x7fecf41f1f80 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7fecec009d90 tx=0x7fecec004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:51.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:51.773+0000 7fecf17fa700 1 -- 172.21.15.16:0/718238549 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecec0042c0 con 0x7fecf415d3d0 2023-05-20T01:28:51.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:51.773+0000 7fecf17fa700 1 -- 172.21.15.16:0/718238549 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecec004440 con 0x7fecf415d3d0 2023-05-20T01:28:51.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:51.773+0000 7fecfaec5700 1 -- 172.21.15.16:0/718238549 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecf41577d0 con 0x7fecf415d3d0 2023-05-20T01:28:51.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:51.773+0000 7fecf17fa700 1 -- 172.21.15.16:0/718238549 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecec0047f0 con 0x7fecf415d3d0 2023-05-20T01:28:51.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:51.773+0000 7fecfaec5700 1 -- 172.21.15.16:0/718238549 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecf415e170 con 0x7fecf415d3d0 2023-05-20T01:28:51.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:51.774+0000 7fecfaec5700 1 -- 172.21.15.16:0/718238549 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecf40cef60 con 0x7fecf415d3d0 2023-05-20T01:28:51.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:51.775+0000 7fecf17fa700 1 -- 172.21.15.16:0/718238549 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fecec01c460 con 0x7fecf415d3d0 2023-05-20T01:28:51.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:51.775+0000 7fecf17fa700 1 --2- 172.21.15.16:0/718238549 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fecd806ace0 0x7fecd806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:51.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:51.775+0000 7fecf17fa700 1 -- 172.21.15.16:0/718238549 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6554+0+0 (secure 0 0 0) 0x7fecec08ac80 con 0x7fecf415d3d0 2023-05-20T01:28:51.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:51.775+0000 7fecf8c61700 1 --2- 172.21.15.16:0/718238549 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fecd806ace0 0x7fecd806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:51.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:51.776+0000 7fecf8c61700 1 --2- 172.21.15.16:0/718238549 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fecd806ace0 0x7fecd806d190 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fece8004a60 tx=0x7fece8000bc0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:51.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:51.780+0000 7fecf17fa700 1 -- 172.21.15.16:0/718238549 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fecec055ab0 con 0x7fecf415d3d0 2023-05-20T01:28:52.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:52.023+0000 7fecfaec5700 1 -- 172.21.15.16:0/718238549 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fecf404e8c0 con 0x7fecf415d3d0 2023-05-20T01:28:52.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:52.023+0000 7fecf17fa700 1 -- 172.21.15.16:0/718238549 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 109 v109) v1 ==== 95+0+4684 (secure 0 0 0) 0x7fecec0590d0 con 0x7fecf415d3d0 2023-05-20T01:28:52.029 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:28:52.030 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":109,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7043,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/1859755557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1859755557},{"type":"v1","addr":"172.21.15.16:6836","nonce":1859755557}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94},{"gid":7046,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/2654783598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":2654783598},{"type":"v1","addr":"172.21.15.132:6834","nonce":2654783598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94},{"gid":7049,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2042939723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":2042939723},{"type":"v1","addr":"172.21.15.132:6835","nonce":2042939723}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":109,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-20T01:28:27.093957+0000","modified":"2023-05-20T01:28:51.614521+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":138,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9054},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9054":{"gid":9054,"name":"a","rank":0,"incarnation":109,"state":"up:replay","state_seq":1,"addr":"172.21.15.16:6837/2952207608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":2952207608},{"type":"v1","addr":"172.21.15.16:6837","nonce":2952207608}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-05-20T01:28:52.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:52.025+0000 7fecfaec5700 1 -- 172.21.15.16:0/718238549 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fecd806ace0 msgr2=0x7fecd806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:52.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:52.025+0000 7fecfaec5700 1 --2- 172.21.15.16:0/718238549 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fecd806ace0 0x7fecd806d190 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fece8004a60 tx=0x7fece8000bc0 comp rx=0 tx=0).stop 2023-05-20T01:28:52.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:52.025+0000 7fecfaec5700 1 -- 172.21.15.16:0/718238549 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fecf415d3d0 msgr2=0x7fecf41f1f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:52.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:52.025+0000 7fecfaec5700 1 --2- 172.21.15.16:0/718238549 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fecf415d3d0 0x7fecf41f1f80 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7fecec009d90 tx=0x7fecec004ad0 comp rx=0 tx=0).stop 2023-05-20T01:28:52.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:52.025+0000 7fecfaec5700 1 -- 172.21.15.16:0/718238549 shutdown_connections 2023-05-20T01:28:52.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:52.025+0000 7fecfaec5700 1 --2- 172.21.15.16:0/718238549 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fecf415d3d0 0x7fecf41f1f80 unknown :-1 s=CLOSED pgs=1732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:52.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:52.025+0000 7fecfaec5700 1 --2- 172.21.15.16:0/718238549 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fecf406ab70 0x7fecf406f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:52.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:52.025+0000 7fecfaec5700 1 --2- 172.21.15.16:0/718238549 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fecd806ace0 0x7fecd806d190 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:52.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:52.025+0000 7fecfaec5700 1 --2- 172.21.15.16:0/718238549 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fecf4157400 0x7fecf4158750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:52.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:52.025+0000 7fecfaec5700 1 -- 172.21.15.16:0/718238549 >> 172.21.15.16:0/718238549 conn(0x7fecf40cd930 msgr2=0x7fecf415f360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:52.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:52.025+0000 7fecfaec5700 1 -- 172.21.15.16:0/718238549 shutdown_connections 2023-05-20T01:28:52.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:52.025+0000 7fecfaec5700 1 -- 172.21.15.16:0/718238549 wait complete. 2023-05-20T01:28:52.042 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 109 2023-05-20T01:28:52.446 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 109, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-05-20T01:28:27.093957+0000', 'modified': '2023-05-20T01:28:51.614521+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 138, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9054}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9054': {'gid': 9054, 'name': 'a', 'rank': 0, 'incarnation': 109, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.16:6837/2952207608', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6835', 'nonce': 2952207608}, {'type': 'v1', 'addr': '172.21.15.16:6837', 'nonce': 2952207608}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [19], 'metadata_pool': 18, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-05-20T01:28:52.447 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9054:up:replay 2023-05-20T01:28:53.448 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-20T01:28:53.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:53.583+0000 7faf4f5a6700 1 Processor -- start 2023-05-20T01:28:53.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:53.583+0000 7faf4f5a6700 1 -- start start 2023-05-20T01:28:53.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:53.584+0000 7faf4f5a6700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faf480cdc40 0x7faf480d0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:53.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:53.584+0000 7faf4f5a6700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7faf480cd710 0x7faf480cf8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:53.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:53.584+0000 7faf4f5a6700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7faf480ce010 0x7faf480c4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:53.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:53.584+0000 7faf4f5a6700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf48070e70 con 0x7faf480cd710 2023-05-20T01:28:53.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:53.584+0000 7faf4f5a6700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf480d09b0 con 0x7faf480ce010 2023-05-20T01:28:53.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:53.584+0000 7faf4f5a6700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf480cfe20 con 0x7faf480cdc40 2023-05-20T01:28:53.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:53.584+0000 7faf4cb41700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7faf480cd710 0x7faf480cf8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:53.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:53.584+0000 7faf4cb41700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7faf480cd710 0x7faf480cf8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40888/0 (socket says 172.21.15.16:40888) 2023-05-20T01:28:53.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:53.584+0000 7faf4cb41700 1 -- 172.21.15.16:0/898074598 learned_addr learned my addr 172.21.15.16:0/898074598 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:53.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:53.585+0000 7faf4d342700 1 --2- 172.21.15.16:0/898074598 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faf480cdc40 0x7faf480d0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:53.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:53.585+0000 7faf4db43700 1 --2- 172.21.15.16:0/898074598 >> v2:172.21.15.132:3300/0 conn(0x7faf480ce010 0x7faf480c4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:53.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:53.585+0000 7faf4cb41700 1 -- 172.21.15.16:0/898074598 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faf480cdc40 msgr2=0x7faf480d0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:53.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:53.585+0000 7faf4cb41700 1 --2- 172.21.15.16:0/898074598 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faf480cdc40 0x7faf480d0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:53.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:53.585+0000 7faf4cb41700 1 -- 172.21.15.16:0/898074598 >> v2:172.21.15.132:3300/0 conn(0x7faf480ce010 msgr2=0x7faf480c4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:53.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:53.585+0000 7faf4cb41700 1 --2- 172.21.15.16:0/898074598 >> v2:172.21.15.132:3300/0 conn(0x7faf480ce010 0x7faf480c4d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:53.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:53.585+0000 7faf4cb41700 1 -- 172.21.15.16:0/898074598 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf480d0c10 con 0x7faf480cd710 2023-05-20T01:28:53.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:53.585+0000 7faf4d342700 1 --2- 172.21.15.16:0/898074598 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faf480cdc40 0x7faf480d0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:28:53.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:53.585+0000 7faf4db43700 1 --2- 172.21.15.16:0/898074598 >> v2:172.21.15.132:3300/0 conn(0x7faf480ce010 0x7faf480c4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:28:53.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:53.585+0000 7faf4cb41700 1 --2- 172.21.15.16:0/898074598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf480cd710 0x7faf480cf8e0 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7faf48071fa0 tx=0x7faf44009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93c75c6b3bee1cb4 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:53.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:53.585+0000 7faf3ffff700 1 -- 172.21.15.16:0/898074598 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf4400b870 con 0x7faf480cd710 2023-05-20T01:28:53.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:53.585+0000 7faf3ffff700 1 -- 172.21.15.16:0/898074598 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf4400b9d0 con 0x7faf480cd710 2023-05-20T01:28:53.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:53.585+0000 7faf3ffff700 1 -- 172.21.15.16:0/898074598 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf4400bd60 con 0x7faf480cd710 2023-05-20T01:28:53.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:53.586+0000 7faf4f5a6700 1 -- 172.21.15.16:0/898074598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf480cd710 msgr2=0x7faf480cf8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:53.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:53.586+0000 7faf4f5a6700 1 --2- 172.21.15.16:0/898074598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf480cd710 0x7faf480cf8e0 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7faf48071fa0 tx=0x7faf44009bd0 comp rx=0 tx=0).stop 2023-05-20T01:28:53.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:53.586+0000 7faf4f5a6700 1 -- 172.21.15.16:0/898074598 shutdown_connections 2023-05-20T01:28:53.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:53.586+0000 7faf4f5a6700 1 --2- 172.21.15.16:0/898074598 >> v2:172.21.15.132:3300/0 conn(0x7faf480ce010 0x7faf480c4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:53.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:53.586+0000 7faf4f5a6700 1 --2- 172.21.15.16:0/898074598 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faf480cdc40 0x7faf480d0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:53.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:53.586+0000 7faf4f5a6700 1 --2- 172.21.15.16:0/898074598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf480cd710 0x7faf480cf8e0 unknown :-1 s=CLOSED pgs=1733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:53.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:53.586+0000 7faf4f5a6700 1 -- 172.21.15.16:0/898074598 >> 172.21.15.16:0/898074598 conn(0x7faf4806e4c0 msgr2=0x7faf4806e8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:53.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:53.586+0000 7faf4f5a6700 1 -- 172.21.15.16:0/898074598 shutdown_connections 2023-05-20T01:28:53.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:53.586+0000 7faf4f5a6700 1 -- 172.21.15.16:0/898074598 wait complete. 2023-05-20T01:28:53.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:53.587+0000 7faf4f5a6700 1 Processor -- start 2023-05-20T01:28:53.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:53.587+0000 7faf4f5a6700 1 -- start start 2023-05-20T01:28:53.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:53.588+0000 7faf4f5a6700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faf480cd710 0x7faf481dfe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:53.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:53.588+0000 7faf4f5a6700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faf480cdc40 0x7faf481e6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:53.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:53.588+0000 7faf4f5a6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf480ce010 0x7faf481e65d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:53.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:53.588+0000 7faf4f5a6700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf481e0780 con 0x7faf480ce010 2023-05-20T01:28:53.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:53.588+0000 7faf4f5a6700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf481e0a60 con 0x7faf480cdc40 2023-05-20T01:28:53.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:53.588+0000 7faf4f5a6700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf481e0d40 con 0x7faf480cd710 2023-05-20T01:28:53.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:53.589+0000 7faf4db43700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf480ce010 0x7faf481e65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:53.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:53.589+0000 7faf4db43700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf480ce010 0x7faf481e65d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40896/0 (socket says 172.21.15.16:40896) 2023-05-20T01:28:53.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:53.589+0000 7faf4db43700 1 -- 172.21.15.16:0/475366167 learned_addr learned my addr 172.21.15.16:0/475366167 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:53.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:53.589+0000 7faf4d342700 1 --2- 172.21.15.16:0/475366167 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faf480cd710 0x7faf481dfe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:53.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:53.589+0000 7faf4cb41700 1 --2- 172.21.15.16:0/475366167 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faf480cdc40 0x7faf481e6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:53.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:53.589+0000 7faf4db43700 1 -- 172.21.15.16:0/475366167 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faf480cd710 msgr2=0x7faf481dfe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:53.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:53.589+0000 7faf4db43700 1 --2- 172.21.15.16:0/475366167 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faf480cd710 0x7faf481dfe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:53.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:53.589+0000 7faf4db43700 1 -- 172.21.15.16:0/475366167 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faf480cdc40 msgr2=0x7faf481e6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:53.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:53.589+0000 7faf4db43700 1 --2- 172.21.15.16:0/475366167 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faf480cdc40 0x7faf481e6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:53.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:53.589+0000 7faf4db43700 1 -- 172.21.15.16:0/475366167 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf44009550 con 0x7faf480ce010 2023-05-20T01:28:53.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:53.589+0000 7faf4cb41700 1 --2- 172.21.15.16:0/475366167 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faf480cdc40 0x7faf481e6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:28:53.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:53.589+0000 7faf4db43700 1 --2- 172.21.15.16:0/475366167 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf480ce010 0x7faf481e65d0 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7faf4000cd50 tx=0x7faf4000a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:53.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:53.589+0000 7faf3dffb700 1 -- 172.21.15.16:0/475366167 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf4000f480 con 0x7faf480ce010 2023-05-20T01:28:53.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:53.589+0000 7faf3dffb700 1 -- 172.21.15.16:0/475366167 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf4000f600 con 0x7faf480ce010 2023-05-20T01:28:53.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:53.589+0000 7faf4f5a6700 1 -- 172.21.15.16:0/475366167 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf480cee00 con 0x7faf480ce010 2023-05-20T01:28:53.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:53.589+0000 7faf3dffb700 1 -- 172.21.15.16:0/475366167 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf4000f9b0 con 0x7faf480ce010 2023-05-20T01:28:53.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:53.590+0000 7faf4f5a6700 1 -- 172.21.15.16:0/475366167 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf480d0ef0 con 0x7faf480ce010 2023-05-20T01:28:53.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:53.591+0000 7faf4f5a6700 1 -- 172.21.15.16:0/475366167 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf480cf0e0 con 0x7faf480ce010 2023-05-20T01:28:53.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:53.591+0000 7faf3dffb700 1 -- 172.21.15.16:0/475366167 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7faf4000b040 con 0x7faf480ce010 2023-05-20T01:28:53.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:53.591+0000 7faf3dffb700 1 --2- 172.21.15.16:0/475366167 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7faf2c06ace0 0x7faf2c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:53.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:53.592+0000 7faf4d342700 1 --2- 172.21.15.16:0/475366167 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7faf2c06ace0 0x7faf2c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:53.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:53.592+0000 7faf3dffb700 1 -- 172.21.15.16:0/475366167 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6554+0+0 (secure 0 0 0) 0x7faf40089f00 con 0x7faf480ce010 2023-05-20T01:28:53.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:53.592+0000 7faf4d342700 1 --2- 172.21.15.16:0/475366167 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7faf2c06ace0 0x7faf2c06d190 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7faf3800aa10 tx=0x7faf38008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:53.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:53.596+0000 7faf3dffb700 1 -- 172.21.15.16:0/475366167 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7faf40054dc0 con 0x7faf480ce010 2023-05-20T01:28:53.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:53.843+0000 7faf4f5a6700 1 -- 172.21.15.16:0/475366167 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faf4804e8c0 con 0x7faf480ce010 2023-05-20T01:28:53.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:53.843+0000 7faf3dffb700 1 -- 172.21.15.16:0/475366167 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 111 v111) v1 ==== 95+0+4684 (secure 0 0 0) 0x7faf4001d090 con 0x7faf480ce010 2023-05-20T01:28:53.848 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:28:53.848 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":111,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7043,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/1859755557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1859755557},{"type":"v1","addr":"172.21.15.16:6836","nonce":1859755557}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94},{"gid":7046,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/2654783598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":2654783598},{"type":"v1","addr":"172.21.15.132:6834","nonce":2654783598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94},{"gid":7049,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2042939723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":2042939723},{"type":"v1","addr":"172.21.15.132:6835","nonce":2042939723}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":111,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-20T01:28:27.093957+0000","modified":"2023-05-20T01:28:53.622752+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":138,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9054},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9054":{"gid":9054,"name":"a","rank":0,"incarnation":109,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.16:6837/2952207608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":2952207608},{"type":"v1","addr":"172.21.15.16:6837","nonce":2952207608}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-05-20T01:28:53.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:53.845+0000 7faf4f5a6700 1 -- 172.21.15.16:0/475366167 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7faf2c06ace0 msgr2=0x7faf2c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:53.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:53.845+0000 7faf4f5a6700 1 --2- 172.21.15.16:0/475366167 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7faf2c06ace0 0x7faf2c06d190 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7faf3800aa10 tx=0x7faf38008040 comp rx=0 tx=0).stop 2023-05-20T01:28:53.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:53.846+0000 7faf4f5a6700 1 -- 172.21.15.16:0/475366167 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf480ce010 msgr2=0x7faf481e65d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:53.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:53.846+0000 7faf4f5a6700 1 --2- 172.21.15.16:0/475366167 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf480ce010 0x7faf481e65d0 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7faf4000cd50 tx=0x7faf4000a560 comp rx=0 tx=0).stop 2023-05-20T01:28:53.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:53.846+0000 7faf4f5a6700 1 -- 172.21.15.16:0/475366167 shutdown_connections 2023-05-20T01:28:53.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:53.846+0000 7faf4f5a6700 1 --2- 172.21.15.16:0/475366167 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf480ce010 0x7faf481e65d0 unknown :-1 s=CLOSED pgs=1734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:53.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:53.846+0000 7faf4f5a6700 1 --2- 172.21.15.16:0/475366167 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7faf2c06ace0 0x7faf2c06d190 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:53.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:53.846+0000 7faf4f5a6700 1 --2- 172.21.15.16:0/475366167 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faf480cd710 0x7faf481dfe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:53.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:53.846+0000 7faf4f5a6700 1 --2- 172.21.15.16:0/475366167 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faf480cdc40 0x7faf481e6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:53.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:53.846+0000 7faf4f5a6700 1 -- 172.21.15.16:0/475366167 >> 172.21.15.16:0/475366167 conn(0x7faf4806e4c0 msgr2=0x7faf4806e890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:53.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:53.846+0000 7faf4f5a6700 1 -- 172.21.15.16:0/475366167 shutdown_connections 2023-05-20T01:28:53.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:53.846+0000 7faf4f5a6700 1 -- 172.21.15.16:0/475366167 wait complete. 2023-05-20T01:28:53.852 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 111 2023-05-20T01:28:54.267 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 111, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-05-20T01:28:27.093957+0000', 'modified': '2023-05-20T01:28:53.622752+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 138, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9054}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9054': {'gid': 9054, 'name': 'a', 'rank': 0, 'incarnation': 109, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.16:6837/2952207608', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6835', 'nonce': 2952207608}, {'type': 'v1', 'addr': '172.21.15.16:6837', 'nonce': 2952207608}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [19], 'metadata_pool': 18, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-05-20T01:28:54.268 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9054:up:rejoin 2023-05-20T01:28:55.269 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-20T01:28:55.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:55.404+0000 7f1881529700 1 Processor -- start 2023-05-20T01:28:55.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:55.404+0000 7f1881529700 1 -- start start 2023-05-20T01:28:55.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:55.405+0000 7f1881529700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f187c0d3fe0 0x7f187c0d2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:55.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:55.405+0000 7f1881529700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f187c0b56c0 0x7f187c0cf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:55.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:55.405+0000 7f1881529700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f187c0d2b50 con 0x7f187c15ce80 2023-05-20T01:28:55.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:55.405+0000 7f1881529700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f187c0d4470 con 0x7f187c0d3fe0 2023-05-20T01:28:55.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:55.405+0000 7f1881529700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f187c15d250 con 0x7f187c0b56c0 2023-05-20T01:28:55.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:55.405+0000 7f187affd700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f187c15ce80 0x7f187c0b4fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:52432/0 (socket says 172.21.15.16:52432) 2023-05-20T01:28:55.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:55.405+0000 7f187affd700 1 -- 172.21.15.16:0/1386130617 learned_addr learned my addr 172.21.15.16:0/1386130617 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:28:55.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:55.405+0000 7f1879ffb700 1 -- 172.21.15.16:0/1386130617 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 498463443 0 0) 0x7f187c0d2b50 con 0x7f187c15ce80 2023-05-20T01:28:55.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:55.405+0000 7f187a7fc700 1 --2- 172.21.15.16:0/1386130617 >> v2:172.21.15.132:3300/0 conn(0x7f187c0d3fe0 0x7f187c0d2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:55.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:55.405+0000 7f187b7fe700 1 --2- 172.21.15.16:0/1386130617 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f187c0b56c0 0x7f187c0cf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:55.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:55.405+0000 7f1879ffb700 1 -- 172.21.15.16:0/1386130617 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f18680035b0 con 0x7f187c15ce80 2023-05-20T01:28:55.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:55.405+0000 7f1879ffb700 1 -- 172.21.15.16:0/1386130617 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 722619143 0 0) 0x7f18680035b0 con 0x7f187c15ce80 2023-05-20T01:28:55.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:55.405+0000 7f1879ffb700 1 -- 172.21.15.16:0/1386130617 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f187c0b56c0 msgr2=0x7f187c0cf980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:55.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:55.405+0000 7f1879ffb700 1 --2- 172.21.15.16:0/1386130617 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f187c0b56c0 0x7f187c0cf980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:55.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:55.405+0000 7f1879ffb700 1 -- 172.21.15.16:0/1386130617 >> v2:172.21.15.132:3300/0 conn(0x7f187c0d3fe0 msgr2=0x7f187c0d2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:55.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:55.405+0000 7f1879ffb700 1 --2- 172.21.15.16:0/1386130617 >> v2:172.21.15.132:3300/0 conn(0x7f187c0d3fe0 0x7f187c0d2fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:55.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:55.405+0000 7f1879ffb700 1 -- 172.21.15.16:0/1386130617 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f187c0d2220 con 0x7f187c15ce80 2023-05-20T01:28:55.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:55.405+0000 7f1879ffb700 1 -- 172.21.15.16:0/1386130617 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7f1864001670 con 0x7f187c15ce80 2023-05-20T01:28:55.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:55.406+0000 7f1879ffb700 1 -- 172.21.15.16:0/1386130617 >> v1:172.21.15.16:6789/0 conn(0x7f187c15ce80 legacy=0x7f187c0b4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:55.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:55.406+0000 7f1879ffb700 1 --2- 172.21.15.16:0/1386130617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1868003a00 0x7f1868003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:55.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:55.406+0000 7f1879ffb700 1 --2- 172.21.15.16:0/1386130617 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1868004430 0x7f18680048f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:55.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:55.406+0000 7f1879ffb700 1 --2- 172.21.15.16:0/1386130617 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f187c0b56c0 0x7f1868005060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:55.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:55.406+0000 7f1879ffb700 1 -- 172.21.15.16:0/1386130617 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f187c0d4470 con 0x7f1868003a00 2023-05-20T01:28:55.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:55.406+0000 7f1879ffb700 1 -- 172.21.15.16:0/1386130617 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f187c15d250 con 0x7f1868004430 2023-05-20T01:28:55.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:55.406+0000 7f1879ffb700 1 -- 172.21.15.16:0/1386130617 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f186800ba10 con 0x7f187c0b56c0 2023-05-20T01:28:55.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:55.406+0000 7f187affd700 1 --2- 172.21.15.16:0/1386130617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1868003a00 0x7f1868003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:55.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:55.406+0000 7f187a7fc700 1 --2- 172.21.15.16:0/1386130617 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1868004430 0x7f18680048f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:55.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:55.406+0000 7f187b7fe700 1 --2- 172.21.15.16:0/1386130617 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f187c0b56c0 0x7f1868005060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:55.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:55.406+0000 7f187affd700 1 -- 172.21.15.16:0/1386130617 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f187c0b56c0 msgr2=0x7f1868005060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:55.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:55.406+0000 7f187affd700 1 --2- 172.21.15.16:0/1386130617 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f187c0b56c0 0x7f1868005060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:55.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:55.406+0000 7f187affd700 1 -- 172.21.15.16:0/1386130617 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1868004430 msgr2=0x7f18680048f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:55.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:55.406+0000 7f187affd700 1 --2- 172.21.15.16:0/1386130617 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1868004430 0x7f18680048f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:55.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:55.406+0000 7f187affd700 1 -- 172.21.15.16:0/1386130617 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18680035b0 con 0x7f1868003a00 2023-05-20T01:28:55.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:55.406+0000 7f187affd700 1 --2- 172.21.15.16:0/1386130617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1868003a00 0x7f1868003e10 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f186400c3a0 tx=0x7f1868005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac1a7b94cce77426 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:55.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:55.406+0000 7f1879ffb700 1 -- 172.21.15.16:0/1386130617 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1864007c60 con 0x7f1868003a00 2023-05-20T01:28:55.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:55.406+0000 7f1879ffb700 1 -- 172.21.15.16:0/1386130617 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1864005960 con 0x7f1868003a00 2023-05-20T01:28:55.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:55.406+0000 7f1879ffb700 1 -- 172.21.15.16:0/1386130617 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f186400cea0 con 0x7f1868003a00 2023-05-20T01:28:55.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:55.407+0000 7f1881529700 1 -- 172.21.15.16:0/1386130617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1868003a00 msgr2=0x7f1868003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:55.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:55.407+0000 7f1881529700 1 --2- 172.21.15.16:0/1386130617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1868003a00 0x7f1868003e10 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f186400c3a0 tx=0x7f1868005880 comp rx=0 tx=0).stop 2023-05-20T01:28:55.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:55.407+0000 7f187b7fe700 1 -- 172.21.15.16:0/1386130617 reap_dead start 2023-05-20T01:28:55.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:55.407+0000 7f1881529700 1 -- 172.21.15.16:0/1386130617 shutdown_connections 2023-05-20T01:28:55.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:55.407+0000 7f1881529700 1 -- 172.21.15.16:0/1386130617 >> 172.21.15.16:0/1386130617 conn(0x7f187c09c800 msgr2=0x7f187c0d3530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:55.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:55.407+0000 7f1881529700 1 -- 172.21.15.16:0/1386130617 shutdown_connections 2023-05-20T01:28:55.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:55.407+0000 7f1881529700 1 -- 172.21.15.16:0/1386130617 wait complete. 2023-05-20T01:28:55.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:55.408+0000 7f1881529700 1 Processor -- start 2023-05-20T01:28:55.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:55.408+0000 7f1881529700 1 -- start start 2023-05-20T01:28:55.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:55.409+0000 7f1881529700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1868003a00 0x7f187c0b1db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:55.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:55.409+0000 7f1881529700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f187c150140 0x7f187c0c9e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:55.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:55.409+0000 7f1881529700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f187c0b00f0 0x7f187c14f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:55.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:55.409+0000 7f1881529700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f187c0b22f0 con 0x7f187c150140 2023-05-20T01:28:55.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:55.409+0000 7f1881529700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f187c14dfa0 con 0x7f187c0b00f0 2023-05-20T01:28:55.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:55.409+0000 7f1881529700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f187c0ca350 con 0x7f1868003a00 2023-05-20T01:28:55.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:55.409+0000 7f187a7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f187c150140 0x7f187c0c9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:55.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:55.409+0000 7f187a7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f187c150140 0x7f187c0c9e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40918/0 (socket says 172.21.15.16:40918) 2023-05-20T01:28:55.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:55.409+0000 7f187a7fc700 1 -- 172.21.15.16:0/4190759707 learned_addr learned my addr 172.21.15.16:0/4190759707 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:55.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:55.409+0000 7f187affd700 1 --2- 172.21.15.16:0/4190759707 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1868003a00 0x7f187c0b1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:55.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:55.409+0000 7f187b7fe700 1 --2- 172.21.15.16:0/4190759707 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f187c0b00f0 0x7f187c14f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:55.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:55.409+0000 7f187a7fc700 1 -- 172.21.15.16:0/4190759707 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1868003a00 msgr2=0x7f187c0b1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:55.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:55.409+0000 7f187a7fc700 1 --2- 172.21.15.16:0/4190759707 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1868003a00 0x7f187c0b1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:55.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:55.409+0000 7f187a7fc700 1 -- 172.21.15.16:0/4190759707 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f187c0b00f0 msgr2=0x7f187c14f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:55.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:55.409+0000 7f187a7fc700 1 --2- 172.21.15.16:0/4190759707 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f187c0b00f0 0x7f187c14f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:55.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:55.409+0000 7f187a7fc700 1 -- 172.21.15.16:0/4190759707 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f187c0d27e0 con 0x7f187c150140 2023-05-20T01:28:55.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:55.410+0000 7f187a7fc700 1 --2- 172.21.15.16:0/4190759707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f187c150140 0x7f187c0c9e10 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f186c011aa0 tx=0x7f186c011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:55.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:55.410+0000 7f18737fe700 1 -- 172.21.15.16:0/4190759707 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f186c00f980 con 0x7f187c150140 2023-05-20T01:28:55.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:55.410+0000 7f18737fe700 1 -- 172.21.15.16:0/4190759707 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f186c00fb00 con 0x7f187c150140 2023-05-20T01:28:55.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:55.410+0000 7f1881529700 1 -- 172.21.15.16:0/4190759707 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f187c0d2500 con 0x7f187c150140 2023-05-20T01:28:55.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:55.410+0000 7f18737fe700 1 -- 172.21.15.16:0/4190759707 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f186c00feb0 con 0x7f187c150140 2023-05-20T01:28:55.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:55.410+0000 7f1881529700 1 -- 172.21.15.16:0/4190759707 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f187c0b4a00 con 0x7f187c150140 2023-05-20T01:28:55.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:55.411+0000 7f1881529700 1 -- 172.21.15.16:0/4190759707 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f187c066ce0 con 0x7f187c150140 2023-05-20T01:28:55.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:55.411+0000 7f18737fe700 1 -- 172.21.15.16:0/4190759707 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f186c014440 con 0x7f187c150140 2023-05-20T01:28:55.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:55.412+0000 7f18737fe700 1 --2- 172.21.15.16:0/4190759707 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f185c06ace0 0x7f185c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:55.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:55.412+0000 7f18737fe700 1 -- 172.21.15.16:0/4190759707 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6554+0+0 (secure 0 0 0) 0x7f186c089e20 con 0x7f187c150140 2023-05-20T01:28:55.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:55.412+0000 7f187affd700 1 --2- 172.21.15.16:0/4190759707 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f185c06ace0 0x7f185c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:55.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:55.412+0000 7f187affd700 1 --2- 172.21.15.16:0/4190759707 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f185c06ace0 0x7f185c06d190 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f1864001050 tx=0x7f186400a000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:55.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:55.416+0000 7f18737fe700 1 -- 172.21.15.16:0/4190759707 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f186c054db0 con 0x7f187c150140 2023-05-20T01:28:55.613 INFO:tasks.ceph.mds.a.smithi016.stderr:2023-05-20T01:28:55.609+0000 7f7d31901700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-05-20T01:28:55.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:55.661+0000 7f1881529700 1 -- 172.21.15.16:0/4190759707 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f187c14e110 con 0x7f187c150140 2023-05-20T01:28:55.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:55.661+0000 7f18737fe700 1 -- 172.21.15.16:0/4190759707 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) v1 ==== 95+0+4684 (secure 0 0 0) 0x7f186c022090 con 0x7f187c150140 2023-05-20T01:28:55.666 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:28:55.666 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":112,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7043,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/1859755557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1859755557},{"type":"v1","addr":"172.21.15.16:6836","nonce":1859755557}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94},{"gid":7046,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/2654783598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":2654783598},{"type":"v1","addr":"172.21.15.132:6834","nonce":2654783598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94},{"gid":7049,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2042939723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":2042939723},{"type":"v1","addr":"172.21.15.132:6835","nonce":2042939723}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-20T01:28:27.093957+0000","modified":"2023-05-20T01:28:54.632173+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":138,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9054},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9054":{"gid":9054,"name":"a","rank":0,"incarnation":109,"state":"up:active","state_seq":4,"addr":"172.21.15.16:6837/2952207608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":2952207608},{"type":"v1","addr":"172.21.15.16:6837","nonce":2952207608}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-05-20T01:28:55.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:55.663+0000 7f1881529700 1 -- 172.21.15.16:0/4190759707 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f185c06ace0 msgr2=0x7f185c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:55.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:55.663+0000 7f1881529700 1 --2- 172.21.15.16:0/4190759707 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f185c06ace0 0x7f185c06d190 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f1864001050 tx=0x7f186400a000 comp rx=0 tx=0).stop 2023-05-20T01:28:55.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:55.663+0000 7f1881529700 1 -- 172.21.15.16:0/4190759707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f187c150140 msgr2=0x7f187c0c9e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:55.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:55.663+0000 7f1881529700 1 --2- 172.21.15.16:0/4190759707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f187c150140 0x7f187c0c9e10 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f186c011aa0 tx=0x7f186c011e60 comp rx=0 tx=0).stop 2023-05-20T01:28:55.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:55.663+0000 7f1881529700 1 -- 172.21.15.16:0/4190759707 shutdown_connections 2023-05-20T01:28:55.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:55.663+0000 7f1881529700 1 --2- 172.21.15.16:0/4190759707 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f187c0b00f0 0x7f187c14f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:55.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:55.663+0000 7f1881529700 1 --2- 172.21.15.16:0/4190759707 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f185c06ace0 0x7f185c06d190 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:55.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:55.663+0000 7f1881529700 1 --2- 172.21.15.16:0/4190759707 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1868003a00 0x7f187c0b1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:55.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:55.663+0000 7f1881529700 1 --2- 172.21.15.16:0/4190759707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f187c150140 0x7f187c0c9e10 unknown :-1 s=CLOSED pgs=1737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:55.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:55.663+0000 7f1881529700 1 -- 172.21.15.16:0/4190759707 >> 172.21.15.16:0/4190759707 conn(0x7f187c09c800 msgr2=0x7f187c15a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:55.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:55.663+0000 7f1881529700 1 -- 172.21.15.16:0/4190759707 shutdown_connections 2023-05-20T01:28:55.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:55.663+0000 7f1881529700 1 -- 172.21.15.16:0/4190759707 wait complete. 2023-05-20T01:28:55.669 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 112 2023-05-20T01:28:56.081 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 112, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-05-20T01:28:27.093957+0000', 'modified': '2023-05-20T01:28:54.632173+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 138, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9054}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9054': {'gid': 9054, 'name': 'a', 'rank': 0, 'incarnation': 109, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.16:6837/2952207608', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6835', 'nonce': 2952207608}, {'type': 'v1', 'addr': '172.21.15.16:6837', 'nonce': 2952207608}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [19], 'metadata_pool': 18, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-05-20T01:28:56.082 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-05-20T01:28:56.082 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-05-20T01:28:56.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.208+0000 7f7118048700 1 Processor -- start 2023-05-20T01:28:56.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.208+0000 7f7118048700 1 -- start start 2023-05-20T01:28:56.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.209+0000 7f7118048700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f71100713e0 0x7f71100cf370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:56.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.209+0000 7f7118048700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f71100717b0 0x7f7110160820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:56.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.209+0000 7f7118048700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f711015d7f0 con 0x7f7110072400 2023-05-20T01:28:56.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.209+0000 7f7118048700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71100d0080 con 0x7f71100713e0 2023-05-20T01:28:56.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.209+0000 7f7118048700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71100cf8b0 con 0x7f71100717b0 2023-05-20T01:28:56.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.209+0000 7f71165e5700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f71100717b0 0x7f7110160820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:56.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.209+0000 7f7115de4700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f71100713e0 0x7f71100cf370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:56.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.209+0000 7f71155e3700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f7110072400 0x7f711015dc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.16:33936/0 (socket says 172.21.15.16:33936) 2023-05-20T01:28:56.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.209+0000 7f71155e3700 1 -- 172.21.15.16:0/2655337841 learned_addr learned my addr 172.21.15.16:0/2655337841 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:28:56.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.209+0000 7f7115de4700 1 -- 172.21.15.16:0/2655337841 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f71100717b0 msgr2=0x7f7110160820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:56.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.210+0000 7f7115de4700 1 --2- 172.21.15.16:0/2655337841 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f71100717b0 0x7f7110160820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:56.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.210+0000 7f7115de4700 1 -- 172.21.15.16:0/2655337841 >> v1:172.21.15.132:6789/0 conn(0x7f7110072400 legacy=0x7f711015dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:56.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.210+0000 7f7115de4700 1 -- 172.21.15.16:0/2655337841 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f711015d200 con 0x7f71100713e0 2023-05-20T01:28:56.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.210+0000 7f7114de2700 1 -- 172.21.15.16:0/2655337841 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3702515424 0 0) 0x7f711015d7f0 con 0x7f7110072400 2023-05-20T01:28:56.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.210+0000 7f71165e5700 1 --2- 172.21.15.16:0/2655337841 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f71100717b0 0x7f7110160820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-05-20T01:28:56.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.210+0000 7f7115de4700 1 --2- 172.21.15.16:0/2655337841 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f71100713e0 0x7f71100cf370 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f710c009840 tx=0x7f710c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ea9eb4d708866a9 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:56.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.210+0000 7f7114de2700 1 -- 172.21.15.16:0/2655337841 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f710c011830 con 0x7f71100713e0 2023-05-20T01:28:56.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.210+0000 7f7114de2700 1 -- 172.21.15.16:0/2655337841 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f710c011990 con 0x7f71100713e0 2023-05-20T01:28:56.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.210+0000 7f7114de2700 1 -- 172.21.15.16:0/2655337841 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f710c011d20 con 0x7f71100713e0 2023-05-20T01:28:56.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.210+0000 7f7118048700 1 -- 172.21.15.16:0/2655337841 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f71100713e0 msgr2=0x7f71100cf370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:56.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.210+0000 7f7118048700 1 --2- 172.21.15.16:0/2655337841 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f71100713e0 0x7f71100cf370 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f710c009840 tx=0x7f710c009b50 comp rx=0 tx=0).stop 2023-05-20T01:28:56.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.211+0000 7f7118048700 1 -- 172.21.15.16:0/2655337841 shutdown_connections 2023-05-20T01:28:56.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.211+0000 7f7118048700 1 --2- 172.21.15.16:0/2655337841 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f71100717b0 0x7f7110160820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:56.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.211+0000 7f7118048700 1 --2- 172.21.15.16:0/2655337841 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f71100713e0 0x7f71100cf370 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:56.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.211+0000 7f7118048700 1 -- 172.21.15.16:0/2655337841 >> 172.21.15.16:0/2655337841 conn(0x7f711006e250 msgr2=0x7f71100ccc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:56.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.211+0000 7f7118048700 1 -- 172.21.15.16:0/2655337841 shutdown_connections 2023-05-20T01:28:56.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.211+0000 7f7118048700 1 -- 172.21.15.16:0/2655337841 wait complete. 2023-05-20T01:28:56.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.212+0000 7f7118048700 1 Processor -- start 2023-05-20T01:28:56.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.212+0000 7f7118048700 1 -- start start 2023-05-20T01:28:56.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.213+0000 7f7118048700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f71100713e0 0x7f71101f1f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:56.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.213+0000 7f7118048700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f71100717b0 0x7f71100aec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:56.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.213+0000 7f7118048700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7110072400 0x7f71100b3140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:56.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.213+0000 7f7118048700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71100c4850 con 0x7f7110072400 2023-05-20T01:28:56.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.213+0000 7f7118048700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71100c4b30 con 0x7f71100717b0 2023-05-20T01:28:56.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.213+0000 7f7118048700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71100c4e10 con 0x7f71100713e0 2023-05-20T01:28:56.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.213+0000 7f71165e5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7110072400 0x7f71100b3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:56.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.213+0000 7f7115de4700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f71100713e0 0x7f71101f1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:56.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.213+0000 7f71165e5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7110072400 0x7f71100b3140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40934/0 (socket says 172.21.15.16:40934) 2023-05-20T01:28:56.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.213+0000 7f71155e3700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f71100717b0 0x7f71100aec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:56.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.213+0000 7f71165e5700 1 -- 172.21.15.16:0/2951197163 learned_addr learned my addr 172.21.15.16:0/2951197163 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:56.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.213+0000 7f71165e5700 1 -- 172.21.15.16:0/2951197163 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f71100713e0 msgr2=0x7f71101f1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:56.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.213+0000 7f71165e5700 1 --2- 172.21.15.16:0/2951197163 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f71100713e0 0x7f71101f1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:56.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.213+0000 7f71165e5700 1 -- 172.21.15.16:0/2951197163 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f71100717b0 msgr2=0x7f71100aec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:56.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.213+0000 7f71165e5700 1 --2- 172.21.15.16:0/2951197163 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f71100717b0 0x7f71100aec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:56.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.213+0000 7f71165e5700 1 -- 172.21.15.16:0/2951197163 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7108009a70 con 0x7f7110072400 2023-05-20T01:28:56.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.213+0000 7f71165e5700 1 --2- 172.21.15.16:0/2951197163 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7110072400 0x7f71100b3140 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f710800efc0 tx=0x7f710800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:56.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.213+0000 7f71067fc700 1 -- 172.21.15.16:0/2951197163 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f710800ca90 con 0x7f7110072400 2023-05-20T01:28:56.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.214+0000 7f71067fc700 1 -- 172.21.15.16:0/2951197163 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f710800f040 con 0x7f7110072400 2023-05-20T01:28:56.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.214+0000 7f7118048700 1 -- 172.21.15.16:0/2951197163 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f710c0094a0 con 0x7f7110072400 2023-05-20T01:28:56.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.214+0000 7f71067fc700 1 -- 172.21.15.16:0/2951197163 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f710800ce10 con 0x7f7110072400 2023-05-20T01:28:56.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.214+0000 7f7118048700 1 -- 172.21.15.16:0/2951197163 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71100cfac0 con 0x7f7110072400 2023-05-20T01:28:56.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.215+0000 7f71067fc700 1 -- 172.21.15.16:0/2951197163 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7108014480 con 0x7f7110072400 2023-05-20T01:28:56.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.215+0000 7f71067fc700 1 --2- 172.21.15.16:0/2951197163 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f70f406ace0 0x7f70f406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:56.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.215+0000 7f71067fc700 1 -- 172.21.15.16:0/2951197163 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6554+0+0 (secure 0 0 0) 0x7f7108089d30 con 0x7f7110072400 2023-05-20T01:28:56.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.215+0000 7f7115de4700 1 --2- 172.21.15.16:0/2951197163 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f70f406ace0 0x7f70f406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:56.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.216+0000 7f7115de4700 1 --2- 172.21.15.16:0/2951197163 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f70f406ace0 0x7f70f406d190 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f710c009790 tx=0x7f710c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:56.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.218+0000 7f71241d4b80 1 Processor -- start 2023-05-20T01:28:56.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.218+0000 7f71241d4b80 1 -- start start 2023-05-20T01:28:56.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.218+0000 7f71241d4b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56410b89f400 0x56410b89f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:56.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.218+0000 7f71241d4b80 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56410b89b700 0x56410b874460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:56.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.218+0000 7f71241d4b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56410b872870 0x56410b8749a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:56.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.218+0000 7f71241d4b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x56410b77a010 con 0x56410b872870 2023-05-20T01:28:56.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.218+0000 7f71241d4b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x56410b7cc850 con 0x56410b89f400 2023-05-20T01:28:56.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.218+0000 7f71241d4b80 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x56410b7c5ce0 con 0x56410b89b700 2023-05-20T01:28:56.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.219+0000 7f71165e5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56410b872870 0x56410b8749a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:56.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.219+0000 7f71165e5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56410b872870 0x56410b8749a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40942/0 (socket says 172.21.15.16:40942) 2023-05-20T01:28:56.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.219+0000 7f71165e5700 1 -- 172.21.15.16:0/1922815015 learned_addr learned my addr 172.21.15.16:0/1922815015 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:56.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.219+0000 7f71155e3700 1 --2- 172.21.15.16:0/1922815015 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56410b89b700 0x56410b874460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:56.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.219+0000 7f7115de4700 1 --2- 172.21.15.16:0/1922815015 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56410b89f400 0x56410b89f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:56.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.219+0000 7f71165e5700 1 -- 172.21.15.16:0/1922815015 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56410b89b700 msgr2=0x56410b874460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:56.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.219+0000 7f71165e5700 1 --2- 172.21.15.16:0/1922815015 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56410b89b700 0x56410b874460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:56.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.219+0000 7f71165e5700 1 -- 172.21.15.16:0/1922815015 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56410b89f400 msgr2=0x56410b89f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:56.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.219+0000 7f71165e5700 1 --2- 172.21.15.16:0/1922815015 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56410b89f400 0x56410b89f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:56.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.219+0000 7f71165e5700 1 -- 172.21.15.16:0/1922815015 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56410b6475e0 con 0x56410b872870 2023-05-20T01:28:56.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.219+0000 7f71165e5700 1 --2- 172.21.15.16:0/1922815015 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56410b872870 0x56410b8749a0 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f7108087e70 tx=0x7f710805c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a8271abc9e0f119 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:56.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.219+0000 7f70f37fe700 1 -- 172.21.15.16:0/1922815015 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7108056c40 con 0x56410b872870 2023-05-20T01:28:56.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.219+0000 7f70f37fe700 1 -- 172.21.15.16:0/1922815015 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7108057d90 con 0x56410b872870 2023-05-20T01:28:56.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.219+0000 7f70f37fe700 1 -- 172.21.15.16:0/1922815015 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7108060760 con 0x56410b872870 2023-05-20T01:28:56.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.220+0000 7f71241d4b80 1 -- 172.21.15.16:0/1922815015 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56410b872870 msgr2=0x56410b8749a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:56.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.220+0000 7f71241d4b80 1 --2- 172.21.15.16:0/1922815015 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56410b872870 0x56410b8749a0 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f7108087e70 tx=0x7f710805c8e0 comp rx=0 tx=0).stop 2023-05-20T01:28:56.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.220+0000 7f71241d4b80 1 -- 172.21.15.16:0/1922815015 shutdown_connections 2023-05-20T01:28:56.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.220+0000 7f71241d4b80 1 --2- 172.21.15.16:0/1922815015 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56410b872870 0x56410b8749a0 unknown :-1 s=CLOSED pgs=1739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:56.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.220+0000 7f71241d4b80 1 --2- 172.21.15.16:0/1922815015 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56410b89f400 0x56410b89f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:56.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.220+0000 7f71241d4b80 1 --2- 172.21.15.16:0/1922815015 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56410b89b700 0x56410b874460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:56.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.220+0000 7f71241d4b80 1 -- 172.21.15.16:0/1922815015 >> 172.21.15.16:0/1922815015 conn(0x56410b869f30 msgr2=0x56410b8ba210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:56.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.220+0000 7f71241d4b80 1 -- 172.21.15.16:0/1922815015 shutdown_connections 2023-05-20T01:28:56.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.220+0000 7f71241d4b80 1 -- 172.21.15.16:0/1922815015 wait complete. 2023-05-20T01:28:56.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.220+0000 7f71241d4b80 1 -- start start 2023-05-20T01:28:56.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.221+0000 7f71241d4b80 1 Processor -- start 2023-05-20T01:28:56.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.221+0000 7f71241d4b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56410b872870 0x56410b8bb690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:56.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.221+0000 7f71241d4b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56410b89b700 0x56410b8bbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:56.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.221+0000 7f71241d4b80 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56410b89f400 0x56410b8bc110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:56.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.221+0000 7f71241d4b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x56410b767890 con 0x56410b89b700 2023-05-20T01:28:56.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.222+0000 7f71241d4b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x56410b767530 con 0x56410b872870 2023-05-20T01:28:56.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.222+0000 7f71241d4b80 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x56410b6498d0 con 0x56410b89f400 2023-05-20T01:28:56.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.222+0000 7f71241d4b80 10 client.0 _ll_register_callbacks cb 0x56410b785730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-05-20T01:28:56.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.222+0000 7f71155e3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56410b89b700 0x56410b8bbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:56.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.222+0000 7f71155e3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56410b89b700 0x56410b8bbbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40944/0 (socket says 172.21.15.16:40944) 2023-05-20T01:28:56.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.222+0000 7f71155e3700 1 -- 172.21.15.16:0/1861477617 learned_addr learned my addr 172.21.15.16:0/1861477617 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:56.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.222+0000 7f71165e5700 1 --2- 172.21.15.16:0/1861477617 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56410b89f400 0x56410b8bc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:56.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.222+0000 7f7115de4700 1 --2- 172.21.15.16:0/1861477617 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56410b872870 0x56410b8bb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:56.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.222+0000 7f71155e3700 1 -- 172.21.15.16:0/1861477617 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56410b89f400 msgr2=0x56410b8bc110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:56.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.222+0000 7f71155e3700 1 --2- 172.21.15.16:0/1861477617 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56410b89f400 0x56410b8bc110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:56.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.222+0000 7f71155e3700 1 -- 172.21.15.16:0/1861477617 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56410b872870 msgr2=0x56410b8bb690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:56.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.222+0000 7f71155e3700 1 --2- 172.21.15.16:0/1861477617 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56410b872870 0x56410b8bb690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:56.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.222+0000 7f71155e3700 1 -- 172.21.15.16:0/1861477617 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71100cfac0 con 0x56410b89b700 2023-05-20T01:28:56.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.222+0000 7f71155e3700 1 --2- 172.21.15.16:0/1861477617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56410b89b700 0x56410b8bbbd0 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f7100000f60 tx=0x7f7100001570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e34a2d36a8b1556 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:56.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.222+0000 7f70f0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-05-20T01:28:56.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.222+0000 7f70f0ff9700 1 -- 172.21.15.16:0/1861477617 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7100001cc0 con 0x56410b89b700 2023-05-20T01:28:56.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.222+0000 7f70f0ff9700 1 -- 172.21.15.16:0/1861477617 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(138..138 src has 1..138) v4 ==== 6554+0+0 (secure 0 0 0) 0x7f710000cad0 con 0x56410b89b700 2023-05-20T01:28:56.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.222+0000 7f71241d4b80 1 -- 172.21.15.16:0/1861477617 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56410b8676f0 con 0x56410b89b700 2023-05-20T01:28:56.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.222+0000 7f70f0ff9700 1 -- 172.21.15.16:0/1861477617 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=112) v2 ==== 24+0+0 (secure 0 0 0) 0x7f710000cc90 con 0x56410b89b700 2023-05-20T01:28:56.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.222+0000 7f71241d4b80 10 client.9135 fetch_fsmap learned FSMap version 112 2023-05-20T01:28:56.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.222+0000 7f71241d4b80 1 -- 172.21.15.16:0/1861477617 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=112,monmap=2+}) v3 -- 0x56410b78c140 con 0x56410b89b700 2023-05-20T01:28:56.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.223+0000 7f70f0ff9700 1 -- 172.21.15.16:0/1861477617 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7100023020 con 0x56410b89b700 2023-05-20T01:28:56.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.223+0000 7f70f0ff9700 1 -- 172.21.15.16:0/1861477617 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 112) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f7100012050 con 0x56410b89b700 2023-05-20T01:28:56.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.223+0000 7f71241d4b80 10 client.9135 fetch_fsmap finished waiting for FSMap version 112 2023-05-20T01:28:56.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.223+0000 7f71241d4b80 10 client.9135 resolve_mds: resolved name 'a' to daemon mds.a 2023-05-20T01:28:56.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.223+0000 7f71241d4b80 20 client.9135 populate_metadata read hostname 'smithi016' 2023-05-20T01:28:56.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.223+0000 7f71241d4b80 1 --2- 172.21.15.16:0/1861477617 >> [v2:172.21.15.16:6835/2952207608,v1:172.21.15.16:6837/2952207608] conn(0x56410b8b72e0 0x56410b8b5db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:56.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.223+0000 7f71241d4b80 4 client.9135 mds_command: new command op to 9054 tid=0[{"prefix": "get_command_descriptions"}] 2023-05-20T01:28:56.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.223+0000 7f71241d4b80 1 -- 172.21.15.16:0/1861477617 --> [v2:172.21.15.16:6835/2952207608,v1:172.21.15.16:6837/2952207608] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56410b788050 con 0x56410b8b72e0 2023-05-20T01:28:56.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.223+0000 7f7115de4700 1 --2- 172.21.15.16:0/1861477617 >> [v2:172.21.15.16:6835/2952207608,v1:172.21.15.16:6837/2952207608] conn(0x56410b8b72e0 0x56410b8b5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:56.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.223+0000 7f70f0ff9700 10 client.9135 ms_handle_connect on v2:172.21.15.16:6835/2952207608 2023-05-20T01:28:56.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.223+0000 7f7115de4700 1 --2- 172.21.15.16:0/1861477617 >> [v2:172.21.15.16:6835/2952207608,v1:172.21.15.16:6837/2952207608] conn(0x56410b8b72e0 0x56410b8b5db0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25cce7a9c7c75d91 server_cookie=be78e0b4e9f063e3 in_seq=0 out_seq=0 2023-05-20T01:28:56.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.224+0000 7f70f0ff9700 1 -- 172.21.15.16:0/1861477617 <== mds.0 v2:172.21.15.16:6835/2952207608 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56410b788050 con 0x56410b8b72e0 2023-05-20T01:28:56.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.224+0000 7f70f0ff9700 10 client.9135 handle_command_reply: tid=0 2023-05-20T01:28:56.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.224+0000 7f71241d4b80 1 client.9135 shutdown 2023-05-20T01:28:56.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.224+0000 7f71241d4b80 1 -- 172.21.15.16:0/1861477617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56410b89b700 msgr2=0x56410b8bbbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:56.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.224+0000 7f71241d4b80 1 --2- 172.21.15.16:0/1861477617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56410b89b700 0x56410b8bbbd0 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f7100000f60 tx=0x7f7100001570 comp rx=0 tx=0).stop 2023-05-20T01:28:56.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.224+0000 7f71241d4b80 1 -- 172.21.15.16:0/1861477617 shutdown_connections 2023-05-20T01:28:56.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.224+0000 7f71241d4b80 1 --2- 172.21.15.16:0/1861477617 >> [v2:172.21.15.16:6835/2952207608,v1:172.21.15.16:6837/2952207608] conn(0x56410b8b72e0 0x56410b8b5db0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:56.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.224+0000 7f71241d4b80 1 --2- 172.21.15.16:0/1861477617 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56410b89f400 0x56410b8bc110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:56.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.224+0000 7f71241d4b80 1 --2- 172.21.15.16:0/1861477617 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56410b872870 0x56410b8bb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:56.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.224+0000 7f71241d4b80 1 --2- 172.21.15.16:0/1861477617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56410b89b700 0x56410b8bbbd0 unknown :-1 s=CLOSED pgs=1740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:56.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.224+0000 7f71241d4b80 1 -- 172.21.15.16:0/1861477617 >> 172.21.15.16:0/1861477617 conn(0x56410b869f30 msgr2=0x56410b89bb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:56.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.224+0000 7f70f0ff9700 0 client.9135 ms_handle_reset on v2:172.21.15.16:6835/2952207608 2023-05-20T01:28:56.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.224+0000 7f71241d4b80 1 -- 172.21.15.16:0/1861477617 shutdown_connections 2023-05-20T01:28:56.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.224+0000 7f71241d4b80 1 -- 172.21.15.16:0/1861477617 wait complete. 2023-05-20T01:28:56.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.224+0000 7f71241d4b80 20 client.9135 trim_cache size 0 max 16384 2023-05-20T01:28:56.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.242+0000 7f71241d4b80 1 Processor -- start 2023-05-20T01:28:56.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.242+0000 7f71241d4b80 1 -- start start 2023-05-20T01:28:56.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.243+0000 7f71241d4b80 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56410b872870 0x56410b8bd210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:56.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.243+0000 7f71241d4b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56410b89b700 0x56410b86e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:56.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.243+0000 7f71241d4b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56410b89f400 0x56410b86e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:56.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.243+0000 7f71241d4b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x56410b8a3160 con 0x56410b89f400 2023-05-20T01:28:56.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.243+0000 7f71241d4b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x56410b8bc910 con 0x56410b89b700 2023-05-20T01:28:56.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.243+0000 7f71241d4b80 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x56410b8be2e0 con 0x56410b872870 2023-05-20T01:28:56.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.243+0000 7f71165e5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56410b89f400 0x56410b86e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:56.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.243+0000 7f71165e5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56410b89f400 0x56410b86e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40956/0 (socket says 172.21.15.16:40956) 2023-05-20T01:28:56.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.243+0000 7f71165e5700 1 -- 172.21.15.16:0/2546927715 learned_addr learned my addr 172.21.15.16:0/2546927715 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:56.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.244+0000 7f71155e3700 1 --2- 172.21.15.16:0/2546927715 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56410b89b700 0x56410b86e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:56.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.244+0000 7f7115de4700 1 --2- 172.21.15.16:0/2546927715 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56410b872870 0x56410b8bd210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:56.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.244+0000 7f71165e5700 1 -- 172.21.15.16:0/2546927715 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56410b872870 msgr2=0x56410b8bd210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:56.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.244+0000 7f71165e5700 1 --2- 172.21.15.16:0/2546927715 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56410b872870 0x56410b8bd210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:56.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.244+0000 7f71165e5700 1 -- 172.21.15.16:0/2546927715 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56410b89b700 msgr2=0x56410b86e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:56.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.244+0000 7f71165e5700 1 --2- 172.21.15.16:0/2546927715 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56410b89b700 0x56410b86e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:56.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.244+0000 7f71165e5700 1 -- 172.21.15.16:0/2546927715 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56410b78be90 con 0x56410b89f400 2023-05-20T01:28:56.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.244+0000 7f71165e5700 1 --2- 172.21.15.16:0/2546927715 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56410b89f400 0x56410b86e660 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f7108058830 tx=0x7f7108089000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6504e698b5a1351 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:56.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.244+0000 7f70f3fff700 1 -- 172.21.15.16:0/2546927715 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7108058670 con 0x56410b89f400 2023-05-20T01:28:56.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.244+0000 7f70f3fff700 1 -- 172.21.15.16:0/2546927715 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7108056c40 con 0x56410b89f400 2023-05-20T01:28:56.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.244+0000 7f70f3fff700 1 -- 172.21.15.16:0/2546927715 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7108060760 con 0x56410b89f400 2023-05-20T01:28:56.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.244+0000 7f71241d4b80 1 -- 172.21.15.16:0/2546927715 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56410b89f400 msgr2=0x56410b86e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:56.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.244+0000 7f71241d4b80 1 --2- 172.21.15.16:0/2546927715 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56410b89f400 0x56410b86e660 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f7108058830 tx=0x7f7108089000 comp rx=0 tx=0).stop 2023-05-20T01:28:56.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.245+0000 7f71241d4b80 1 -- 172.21.15.16:0/2546927715 shutdown_connections 2023-05-20T01:28:56.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.245+0000 7f71241d4b80 1 --2- 172.21.15.16:0/2546927715 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56410b89f400 0x56410b86e660 unknown :-1 s=CLOSED pgs=1741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:56.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.245+0000 7f71241d4b80 1 --2- 172.21.15.16:0/2546927715 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56410b872870 0x56410b8bd210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:56.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.245+0000 7f71241d4b80 1 --2- 172.21.15.16:0/2546927715 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56410b89b700 0x56410b86e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:56.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.245+0000 7f71241d4b80 1 -- 172.21.15.16:0/2546927715 >> 172.21.15.16:0/2546927715 conn(0x56410b869f30 msgr2=0x56410b86f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:56.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.245+0000 7f71241d4b80 1 -- 172.21.15.16:0/2546927715 shutdown_connections 2023-05-20T01:28:56.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.245+0000 7f71241d4b80 1 -- 172.21.15.16:0/2546927715 wait complete. 2023-05-20T01:28:56.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.245+0000 7f71241d4b80 1 -- start start 2023-05-20T01:28:56.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.245+0000 7f71241d4b80 1 Processor -- start 2023-05-20T01:28:56.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.246+0000 7f71241d4b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56410b872870 0x56410b8a2320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:56.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.246+0000 7f71241d4b80 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56410b89b700 0x56410b89bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:56.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.246+0000 7f71241d4b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56410b89f400 0x56410b89c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:56.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.246+0000 7f71241d4b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x56410b8a1670 con 0x56410b872870 2023-05-20T01:28:56.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.246+0000 7f71241d4b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x56410b8a1ad0 con 0x56410b89f400 2023-05-20T01:28:56.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.246+0000 7f71241d4b80 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x56410b89ac20 con 0x56410b89b700 2023-05-20T01:28:56.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.246+0000 7f71241d4b80 10 client.0 _ll_register_callbacks cb 0x7f7100007360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-05-20T01:28:56.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.246+0000 7f7115de4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56410b872870 0x56410b8a2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:56.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.246+0000 7f7115de4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56410b872870 0x56410b8a2320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40958/0 (socket says 172.21.15.16:40958) 2023-05-20T01:28:56.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.246+0000 7f7115de4700 1 -- 172.21.15.16:0/1358865156 learned_addr learned my addr 172.21.15.16:0/1358865156 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:56.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.246+0000 7f71155e3700 1 --2- 172.21.15.16:0/1358865156 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56410b89b700 0x56410b89bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:56.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.246+0000 7f71165e5700 1 --2- 172.21.15.16:0/1358865156 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56410b89f400 0x56410b89c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:56.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.247+0000 7f7115de4700 1 -- 172.21.15.16:0/1358865156 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56410b89b700 msgr2=0x56410b89bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:56.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.247+0000 7f7115de4700 1 --2- 172.21.15.16:0/1358865156 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56410b89b700 0x56410b89bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:56.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.247+0000 7f7115de4700 1 -- 172.21.15.16:0/1358865156 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56410b89f400 msgr2=0x56410b89c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:56.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.247+0000 7f7115de4700 1 --2- 172.21.15.16:0/1358865156 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56410b89f400 0x56410b89c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:56.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.247+0000 7f7115de4700 1 -- 172.21.15.16:0/1358865156 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56410b6475e0 con 0x56410b872870 2023-05-20T01:28:56.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.247+0000 7f71165e5700 1 --2- 172.21.15.16:0/1358865156 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56410b89f400 0x56410b89c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:28:56.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.247+0000 7f7115de4700 1 --2- 172.21.15.16:0/1358865156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56410b872870 0x56410b8a2320 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f710c0090b0 tx=0x7f710c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1a207132277777a server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:56.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.247+0000 7f70f0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-05-20T01:28:56.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.247+0000 7f70f0ff9700 1 -- 172.21.15.16:0/1358865156 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f710c020070 con 0x56410b872870 2023-05-20T01:28:56.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.247+0000 7f70f0ff9700 1 -- 172.21.15.16:0/1358865156 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(138..138 src has 1..138) v4 ==== 6554+0+0 (secure 0 0 0) 0x7f710c0354d0 con 0x56410b872870 2023-05-20T01:28:56.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.247+0000 7f71241d4b80 1 -- 172.21.15.16:0/1358865156 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7108057d90 con 0x56410b872870 2023-05-20T01:28:56.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.247+0000 7f70f0ff9700 1 -- 172.21.15.16:0/1358865156 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=112) v2 ==== 24+0+0 (secure 0 0 0) 0x7f710c035a50 con 0x56410b872870 2023-05-20T01:28:56.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.247+0000 7f71241d4b80 10 client.9141 fetch_fsmap learned FSMap version 112 2023-05-20T01:28:56.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.247+0000 7f71241d4b80 1 -- 172.21.15.16:0/1358865156 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=112,monmap=2+}) v3 -- 0x56410b5dcc10 con 0x56410b872870 2023-05-20T01:28:56.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.247+0000 7f70f0ff9700 1 -- 172.21.15.16:0/1358865156 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7108057d90 con 0x56410b872870 2023-05-20T01:28:56.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.247+0000 7f70f0ff9700 1 -- 172.21.15.16:0/1358865156 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 112) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f710c035bb0 con 0x56410b872870 2023-05-20T01:28:56.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.247+0000 7f71241d4b80 10 client.9141 fetch_fsmap finished waiting for FSMap version 112 2023-05-20T01:28:56.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.247+0000 7f71241d4b80 10 client.9141 resolve_mds: resolved name 'a' to daemon mds.a 2023-05-20T01:28:56.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.247+0000 7f71241d4b80 20 client.9141 populate_metadata read hostname 'smithi016' 2023-05-20T01:28:56.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.247+0000 7f71241d4b80 1 --2- 172.21.15.16:0/1358865156 >> [v2:172.21.15.16:6835/2952207608,v1:172.21.15.16:6837/2952207608] conn(0x56410b9208a0 0x56410b920c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:56.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.247+0000 7f71241d4b80 4 client.9141 mds_command: new command op to 9054 tid=0[{"prefix": "status"}] 2023-05-20T01:28:56.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.247+0000 7f71241d4b80 1 -- 172.21.15.16:0/1358865156 --> [v2:172.21.15.16:6835/2952207608,v1:172.21.15.16:6837/2952207608] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56410b794610 con 0x56410b9208a0 2023-05-20T01:28:56.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.247+0000 7f71155e3700 1 --2- 172.21.15.16:0/1358865156 >> [v2:172.21.15.16:6835/2952207608,v1:172.21.15.16:6837/2952207608] conn(0x56410b9208a0 0x56410b920c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:56.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.248+0000 7f70f0ff9700 10 client.9141 ms_handle_connect on v2:172.21.15.16:6835/2952207608 2023-05-20T01:28:56.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.248+0000 7f71155e3700 1 --2- 172.21.15.16:0/1358865156 >> [v2:172.21.15.16:6835/2952207608,v1:172.21.15.16:6837/2952207608] conn(0x56410b9208a0 0x56410b920c70 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f50e47baa4d8f8e9 server_cookie=6ee98faa14093f27 in_seq=0 out_seq=0 2023-05-20T01:28:56.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.248+0000 7f70f0ff9700 1 -- 172.21.15.16:0/1358865156 <== mds.0 v2:172.21.15.16:6835/2952207608 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+334 (crc 0 0 0) 0x56410b794610 con 0x56410b9208a0 2023-05-20T01:28:56.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.248+0000 7f70f0ff9700 10 client.9141 handle_command_reply: tid=0 2023-05-20T01:28:56.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.248+0000 7f71241d4b80 1 client.9141 shutdown 2023-05-20T01:28:56.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.248+0000 7f71241d4b80 1 -- 172.21.15.16:0/1358865156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56410b872870 msgr2=0x56410b8a2320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:56.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.248+0000 7f71241d4b80 1 --2- 172.21.15.16:0/1358865156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56410b872870 0x56410b8a2320 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f710c0090b0 tx=0x7f710c0047c0 comp rx=0 tx=0).stop 2023-05-20T01:28:56.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.248+0000 7f71241d4b80 1 -- 172.21.15.16:0/1358865156 shutdown_connections 2023-05-20T01:28:56.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.248+0000 7f71241d4b80 1 --2- 172.21.15.16:0/1358865156 >> [v2:172.21.15.16:6835/2952207608,v1:172.21.15.16:6837/2952207608] conn(0x56410b9208a0 0x56410b920c70 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:56.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.248+0000 7f71241d4b80 1 --2- 172.21.15.16:0/1358865156 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56410b89f400 0x56410b89c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:56.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.248+0000 7f71241d4b80 1 --2- 172.21.15.16:0/1358865156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56410b872870 0x56410b8a2320 unknown :-1 s=CLOSED pgs=1742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:56.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.248+0000 7f70f0ff9700 0 client.9141 ms_handle_reset on v2:172.21.15.16:6835/2952207608 2023-05-20T01:28:56.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.248+0000 7f71241d4b80 1 --2- 172.21.15.16:0/1358865156 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56410b89b700 0x56410b89bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:56.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.248+0000 7f71241d4b80 1 -- 172.21.15.16:0/1358865156 >> 172.21.15.16:0/1358865156 conn(0x56410b869f30 msgr2=0x56410b8a0a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:56.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.248+0000 7f71241d4b80 1 -- 172.21.15.16:0/1358865156 shutdown_connections 2023-05-20T01:28:56.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.248+0000 7f71241d4b80 1 -- 172.21.15.16:0/1358865156 wait complete. 2023-05-20T01:28:56.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.248+0000 7f71241d4b80 20 client.9141 trim_cache size 0 max 16384 2023-05-20T01:28:56.264 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-05-20T01:28:56.264 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "c2245cc5-ecab-4087-a59b-bc81db901676", 2023-05-20T01:28:56.264 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2023-05-20T01:28:56.264 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 9054, 2023-05-20T01:28:56.264 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2023-05-20T01:28:56.264 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2023-05-20T01:28:56.265 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2023-05-20T01:28:56.265 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 4.6308097669999997, 2023-05-20T01:28:56.265 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 112, 2023-05-20T01:28:56.265 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 138, 2023-05-20T01:28:56.265 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 138, 2023-05-20T01:28:56.265 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 6.6125557170000002 2023-05-20T01:28:56.265 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-05-20T01:28:56.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.249+0000 7f7118048700 1 -- 172.21.15.16:0/2951197163 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f70f406ace0 msgr2=0x7f70f406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:56.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.249+0000 7f7118048700 1 --2- 172.21.15.16:0/2951197163 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f70f406ace0 0x7f70f406d190 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f710c009790 tx=0x7f710c008030 comp rx=0 tx=0).stop 2023-05-20T01:28:56.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.249+0000 7f7118048700 1 -- 172.21.15.16:0/2951197163 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7110072400 msgr2=0x7f71100b3140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:56.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.249+0000 7f7118048700 1 --2- 172.21.15.16:0/2951197163 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7110072400 0x7f71100b3140 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f710800efc0 tx=0x7f710800c3a0 comp rx=0 tx=0).stop 2023-05-20T01:28:56.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.249+0000 7f7118048700 1 -- 172.21.15.16:0/2951197163 shutdown_connections 2023-05-20T01:28:56.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.249+0000 7f7118048700 1 --2- 172.21.15.16:0/2951197163 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7110072400 0x7f71100b3140 unknown :-1 s=CLOSED pgs=1738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:56.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.249+0000 7f7118048700 1 --2- 172.21.15.16:0/2951197163 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f70f406ace0 0x7f70f406d190 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:56.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.249+0000 7f7118048700 1 --2- 172.21.15.16:0/2951197163 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f71100713e0 0x7f71101f1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:56.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.249+0000 7f7118048700 1 --2- 172.21.15.16:0/2951197163 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f71100717b0 0x7f71100aec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:56.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.249+0000 7f7118048700 1 -- 172.21.15.16:0/2951197163 >> 172.21.15.16:0/2951197163 conn(0x7f711006e250 msgr2=0x7f71100cb260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:56.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.249+0000 7f7118048700 1 -- 172.21.15.16:0/2951197163 shutdown_connections 2023-05-20T01:28:56.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.249+0000 7f7118048700 1 -- 172.21.15.16:0/2951197163 wait complete. 2023-05-20T01:28:56.269 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-05-20T01:28:56.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.389+0000 7f53a9207700 1 Processor -- start 2023-05-20T01:28:56.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.389+0000 7f53a9207700 1 -- start start 2023-05-20T01:28:56.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.389+0000 7f53a9207700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f53a406a460 0x7f53a406a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:56.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.390+0000 7f53a9207700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f53a40713c0 0x7f53a4071830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:56.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.390+0000 7f53a9207700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53a406f030 con 0x7f53a415e480 2023-05-20T01:28:56.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.390+0000 7f53a9207700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53a4071d70 con 0x7f53a406a460 2023-05-20T01:28:56.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.390+0000 7f53a9207700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53a40cbc90 con 0x7f53a40713c0 2023-05-20T01:28:56.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.390+0000 7f53a2d9d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f53a406a460 0x7f53a406a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:56.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.390+0000 7f53a359e700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f53a415e480 0x7f53a415e900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:52498/0 (socket says 172.21.15.16:52498) 2023-05-20T01:28:56.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.390+0000 7f53a2d9d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f53a406a460 0x7f53a406a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40966/0 (socket says 172.21.15.16:40966) 2023-05-20T01:28:56.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.390+0000 7f53a359e700 1 -- 172.21.15.16:0/204440973 learned_addr learned my addr 172.21.15.16:0/204440973 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:28:56.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.390+0000 7f53a259c700 1 --2- 172.21.15.16:0/204440973 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f53a40713c0 0x7f53a4071830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:56.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.390+0000 7f53a2d9d700 1 -- 172.21.15.16:0/204440973 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f53a40713c0 msgr2=0x7f53a4071830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:56.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.390+0000 7f53a2d9d700 1 --2- 172.21.15.16:0/204440973 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f53a40713c0 0x7f53a4071830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:56.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.390+0000 7f53a2d9d700 1 -- 172.21.15.16:0/204440973 >> v1:172.21.15.16:6789/0 conn(0x7f53a415e480 legacy=0x7f53a415e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:56.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.390+0000 7f53a1d9b700 1 -- 172.21.15.16:0/204440973 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 370308585 0 0) 0x7f53a406f030 con 0x7f53a415e480 2023-05-20T01:28:56.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.390+0000 7f53a2d9d700 1 -- 172.21.15.16:0/204440973 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53a40cb9d0 con 0x7f53a406a460 2023-05-20T01:28:56.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.390+0000 7f53a2d9d700 1 --2- 172.21.15.16:0/204440973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53a406a460 0x7f53a406a830 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7f5394009840 tx=0x7f5394009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffbcc358048a998b server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:56.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.390+0000 7f53a1d9b700 1 -- 172.21.15.16:0/204440973 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5394011830 con 0x7f53a406a460 2023-05-20T01:28:56.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.390+0000 7f53a1d9b700 1 -- 172.21.15.16:0/204440973 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5394011990 con 0x7f53a406a460 2023-05-20T01:28:56.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.390+0000 7f53a1d9b700 1 -- 172.21.15.16:0/204440973 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5394011d20 con 0x7f53a406a460 2023-05-20T01:28:56.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.391+0000 7f53a9207700 1 -- 172.21.15.16:0/204440973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53a406a460 msgr2=0x7f53a406a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:56.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.391+0000 7f53a9207700 1 --2- 172.21.15.16:0/204440973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53a406a460 0x7f53a406a830 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7f5394009840 tx=0x7f5394009b50 comp rx=0 tx=0).stop 2023-05-20T01:28:56.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.391+0000 7f53a9207700 1 -- 172.21.15.16:0/204440973 shutdown_connections 2023-05-20T01:28:56.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.391+0000 7f53a9207700 1 --2- 172.21.15.16:0/204440973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53a406a460 0x7f53a406a830 unknown :-1 s=CLOSED pgs=1744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:56.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.391+0000 7f53a9207700 1 --2- 172.21.15.16:0/204440973 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f53a40713c0 0x7f53a4071830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:56.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.391+0000 7f53a9207700 1 -- 172.21.15.16:0/204440973 >> 172.21.15.16:0/204440973 conn(0x7f53a40b1c90 msgr2=0x7f53a40b2060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:56.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.391+0000 7f53a9207700 1 -- 172.21.15.16:0/204440973 shutdown_connections 2023-05-20T01:28:56.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.391+0000 7f53a9207700 1 -- 172.21.15.16:0/204440973 wait complete. 2023-05-20T01:28:56.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.392+0000 7f53a9207700 1 Processor -- start 2023-05-20T01:28:56.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.393+0000 7f53a9207700 1 -- start start 2023-05-20T01:28:56.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.393+0000 7f53a9207700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f53a406a460 0x7f53a4151f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:56.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.393+0000 7f53a9207700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53a40713c0 0x7f53a41e42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:56.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.393+0000 7f53a9207700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f53a415e480 0x7f53a41f0a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:56.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.393+0000 7f53a9207700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53a41524a0 con 0x7f53a40713c0 2023-05-20T01:28:56.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.393+0000 7f53a9207700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53a41f19a0 con 0x7f53a415e480 2023-05-20T01:28:56.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.393+0000 7f53a9207700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53a41f13e0 con 0x7f53a406a460 2023-05-20T01:28:56.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.393+0000 7f53a259c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53a40713c0 0x7f53a41e42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:56.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.393+0000 7f53a259c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53a40713c0 0x7f53a41e42c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40974/0 (socket says 172.21.15.16:40974) 2023-05-20T01:28:56.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.393+0000 7f53a259c700 1 -- 172.21.15.16:0/3784554350 learned_addr learned my addr 172.21.15.16:0/3784554350 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:56.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.394+0000 7f53a2d9d700 1 --2- 172.21.15.16:0/3784554350 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f53a406a460 0x7f53a4151f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:56.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.394+0000 7f53a359e700 1 --2- 172.21.15.16:0/3784554350 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f53a415e480 0x7f53a41f0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:56.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.394+0000 7f53a259c700 1 -- 172.21.15.16:0/3784554350 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f53a406a460 msgr2=0x7f53a4151f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:56.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.394+0000 7f53a259c700 1 --2- 172.21.15.16:0/3784554350 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f53a406a460 0x7f53a4151f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:56.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.394+0000 7f53a259c700 1 -- 172.21.15.16:0/3784554350 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f53a415e480 msgr2=0x7f53a41f0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:56.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.394+0000 7f53a259c700 1 --2- 172.21.15.16:0/3784554350 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f53a415e480 0x7f53a41f0a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:56.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.394+0000 7f53a259c700 1 -- 172.21.15.16:0/3784554350 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53940094a0 con 0x7f53a40713c0 2023-05-20T01:28:56.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.394+0000 7f53a259c700 1 --2- 172.21.15.16:0/3784554350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53a40713c0 0x7f53a41e42c0 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f539800e9b0 tx=0x7f539800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:56.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.394+0000 7f538b7fe700 1 -- 172.21.15.16:0/3784554350 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5398009710 con 0x7f53a40713c0 2023-05-20T01:28:56.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.394+0000 7f538b7fe700 1 -- 172.21.15.16:0/3784554350 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f539800c850 con 0x7f53a40713c0 2023-05-20T01:28:56.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.394+0000 7f538b7fe700 1 -- 172.21.15.16:0/3784554350 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f539800cbc0 con 0x7f53a40713c0 2023-05-20T01:28:56.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.394+0000 7f53a9207700 1 -- 172.21.15.16:0/3784554350 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53a406ad70 con 0x7f53a40713c0 2023-05-20T01:28:56.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.394+0000 7f53a9207700 1 -- 172.21.15.16:0/3784554350 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53a4072820 con 0x7f53a40713c0 2023-05-20T01:28:56.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.395+0000 7f53a9207700 1 -- 172.21.15.16:0/3784554350 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53a4071f80 con 0x7f53a40713c0 2023-05-20T01:28:56.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.395+0000 7f538b7fe700 1 -- 172.21.15.16:0/3784554350 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f539800cd20 con 0x7f53a40713c0 2023-05-20T01:28:56.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.396+0000 7f538b7fe700 1 --2- 172.21.15.16:0/3784554350 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f539006ace0 0x7f539006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:56.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.396+0000 7f538b7fe700 1 -- 172.21.15.16:0/3784554350 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6554+0+0 (secure 0 0 0) 0x7f53980896f0 con 0x7f53a40713c0 2023-05-20T01:28:56.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.396+0000 7f53a2d9d700 1 --2- 172.21.15.16:0/3784554350 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f539006ace0 0x7f539006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:56.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.396+0000 7f53a2d9d700 1 --2- 172.21.15.16:0/3784554350 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f539006ace0 0x7f539006d190 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f53940090b0 tx=0x7f5394008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:56.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.400+0000 7f538b7fe700 1 -- 172.21.15.16:0/3784554350 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f53980580d0 con 0x7f53a40713c0 2023-05-20T01:28:56.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.647+0000 7f53a9207700 1 -- 172.21.15.16:0/3784554350 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f53a404e8c0 con 0x7f53a40713c0 2023-05-20T01:28:56.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.647+0000 7f538b7fe700 1 -- 172.21.15.16:0/3784554350 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v112) v1 ==== 53+0+73 (secure 0 0 0) 0x7f5398057c60 con 0x7f53a40713c0 2023-05-20T01:28:56.652 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-05-20T01:28:56.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.649+0000 7f53a9207700 1 -- 172.21.15.16:0/3784554350 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f539006ace0 msgr2=0x7f539006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:56.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.649+0000 7f53a9207700 1 --2- 172.21.15.16:0/3784554350 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f539006ace0 0x7f539006d190 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f53940090b0 tx=0x7f5394008030 comp rx=0 tx=0).stop 2023-05-20T01:28:56.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.649+0000 7f53a9207700 1 -- 172.21.15.16:0/3784554350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53a40713c0 msgr2=0x7f53a41e42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:56.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.649+0000 7f53a9207700 1 --2- 172.21.15.16:0/3784554350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53a40713c0 0x7f53a41e42c0 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f539800e9b0 tx=0x7f539800ed70 comp rx=0 tx=0).stop 2023-05-20T01:28:56.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.649+0000 7f53a9207700 1 -- 172.21.15.16:0/3784554350 shutdown_connections 2023-05-20T01:28:56.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.649+0000 7f53a9207700 1 --2- 172.21.15.16:0/3784554350 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f53a415e480 0x7f53a41f0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:56.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.650+0000 7f53a9207700 1 --2- 172.21.15.16:0/3784554350 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7f539006ace0 0x7f539006d190 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:56.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.650+0000 7f53a9207700 1 --2- 172.21.15.16:0/3784554350 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f53a406a460 0x7f53a4151f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:56.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.650+0000 7f53a9207700 1 --2- 172.21.15.16:0/3784554350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53a40713c0 0x7f53a41e42c0 unknown :-1 s=CLOSED pgs=1745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:56.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.650+0000 7f53a9207700 1 -- 172.21.15.16:0/3784554350 >> 172.21.15.16:0/3784554350 conn(0x7f53a40b1c90 msgr2=0x7f53a40b1460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:56.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.650+0000 7f53a9207700 1 -- 172.21.15.16:0/3784554350 shutdown_connections 2023-05-20T01:28:56.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:56.650+0000 7f53a9207700 1 -- 172.21.15.16:0/3784554350 wait complete. 2023-05-20T01:28:56.666 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-05-20T01:28:56.666 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-05-20T01:28:56.666 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2023-05-20T01:28:56.667 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-05-20T01:28:56.667 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-05-20T01:28:56.667 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-05-20T01:28:56.667 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-05-20T01:28:56.668 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-05-20T01:28:56.668 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-05-20T01:28:56.668 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:28:56.669 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2023-05-20T01:28:56.686 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-05-20T01:28:56.686 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-05-20T01:28:56.686 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2023-05-20T01:28:56.686 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-05-20T01:28:56.686 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2023-05-20T01:28:56.686 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-05-20T01:28:56.687 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-05-20T01:28:56.687 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2023-05-20T01:28:56.687 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-05-20T01:28:56.687 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2023-05-20T01:28:56.687 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-05-20T01:28:56.687 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2023-05-20T01:28:56.687 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-05-20T01:28:56.687 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 41697sec preferred_lft 41697sec 2023-05-20T01:28:56.688 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link 2023-05-20T01:28:56.688 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-05-20T01:28:56.688 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-05-20T01:28:56.688 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2023-05-20T01:28:56.688 INFO:teuthology.orchestra.run.smithi016.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-05-20T01:28:56.688 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0a:2e:0c:6b:5d:42 brd ff:ff:ff:ff:ff:ff 2023-05-20T01:28:56.688 INFO:teuthology.orchestra.run.smithi016.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-05-20T01:28:56.689 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-05-20T01:28:56.689 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::7c00:b2ff:fef4:946/64 scope link 2023-05-20T01:28:56.689 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-05-20T01:28:56.689 INFO:teuthology.orchestra.run.smithi016.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-05-20T01:28:56.689 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether d2:e4:70:a7:b9:4b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-05-20T01:28:56.689 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::d0e4:70ff:fea7:b94b/64 scope link 2023-05-20T01:28:56.689 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-05-20T01:28:56.689 INFO:teuthology.orchestra.run.smithi016.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-05-20T01:28:56.690 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0a:2e:0c:6b:5d:42 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-05-20T01:28:56.690 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::82e:cff:fe6b:5d42/64 scope link 2023-05-20T01:28:56.690 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-05-20T01:28:56.690 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:28:56.691 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-05-20T01:28:56.751 INFO:teuthology.orchestra.run.smithi016.stdout:1 2023-05-20T01:28:56.753 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:28:56.753 DEBUG:teuthology.orchestra.run.smithi016:> route 2023-05-20T01:28:56.811 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2023-05-20T01:28:56.812 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-05-20T01:28:56.812 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-05-20T01:28:56.812 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-05-20T01:28:56.812 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-05-20T01:28:56.812 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:28:56.813 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2023-05-20T01:28:56.813 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2023-05-20T01:28:56.814 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-05-20T01:28:56.814 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-05-20T01:28:56.815 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-05-20T01:28:56.815 DEBUG:teuthology.orchestra.run.smithi016:> ') 2023-05-20T01:28:56.918 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:28:56.918 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2023-05-20T01:28:56.935 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-05-20T01:28:56.935 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-05-20T01:28:56.936 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-05-20T01:28:56.936 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:28:56.936 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.0 up 2023-05-20T01:28:57.006 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-05-20T01:28:57.006 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-05-20T01:28:57.063 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-05-20T01:28:57.063 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-05-20T01:28:57.063 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-05-20T01:28:57.120 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-05-20T01:28:57.120 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:28:57.120 DEBUG:teuthology.orchestra.run.smithi016:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-05-20T01:28:57.206 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-05-20T01:28:57.206 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-05-20T01:28:57.235 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-05-20T01:28:57.235 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-05-20T01:28:57.235 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.132:3300,172.21.15.132:3301" 2023-05-20T01:28:57.235 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: trying mount with new device syntax: 0@c2245cc5-ecab-4087-a59b-bc81db901676.cephfs=/ 2023-05-20T01:28:57.236 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.16:3300/172.21.15.132:3300/172.21.15.132:3301" will pass to kernel 2023-05-20T01:28:57.236 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: v2 mount support check returned 2 2023-05-20T01:28:57.236 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: kernel does not support v2 syntax 2023-05-20T01:28:57.236 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: switching to using v1 address with old syntax 2023-05-20T01:28:57.266 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-05-20T01:28:57.266 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-05-20T01:28:57.266 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.132:3300,172.21.15.132:3301" 2023-05-20T01:28:57.266 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: trying mount with new device syntax: 0@c2245cc5-ecab-4087-a59b-bc81db901676.cephfs=/ 2023-05-20T01:28:57.267 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.16:3300/172.21.15.132:3300/172.21.15.132:3301" will pass to kernel 2023-05-20T01:28:57.267 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: v2 mount support check returned 2 2023-05-20T01:28:57.267 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: kernel does not support v2 syntax 2023-05-20T01:28:57.267 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: switching to using v1 address with old syntax 2023-05-20T01:28:57.267 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:6789,172.21.15.132:6789,172.21.15.132:6790" 2023-05-20T01:28:57.267 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.16:6789,172.21.15.132:6789,172.21.15.132:6790:/ 2023-05-20T01:28:57.267 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=c2245cc5-ecab-4087-a59b-bc81db901676" will pass to kernel 2023-05-20T01:28:57.274 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-05-20T01:28:57.275 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:28:57.275 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-05-20T01:28:57.312 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:28:57.313 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-05-20T01:28:57.384 INFO:teuthology.orchestra.run.smithi016.stdout:client9153 2023-05-20T01:28:57.385 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-05-20T01:28:57.385 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2023-05-20T01:28:57.445 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:28:57.446 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls -l subdir/) 2023-05-20T01:28:57.529 INFO:teuthology.orchestra.run.smithi016.stdout:total 50 2023-05-20T01:28:57.529 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-rw-r--. 1 ubuntu ubuntu 1 May 20 01:28 0 2023-05-20T01:28:57.530 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-rw-r--. 1 ubuntu ubuntu 1 May 20 01:28 1 2023-05-20T01:28:57.530 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-rw-r--. 1 ubuntu ubuntu 2 May 20 01:28 10 2023-05-20T01:28:57.530 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-rw-r--. 1 ubuntu ubuntu 2 May 20 01:28 11 2023-05-20T01:28:57.530 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-rw-r--. 1 ubuntu ubuntu 2 May 20 01:28 12 2023-05-20T01:28:57.530 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-rw-r--. 1 ubuntu ubuntu 2 May 20 01:28 13 2023-05-20T01:28:57.530 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-rw-r--. 1 ubuntu ubuntu 2 May 20 01:28 14 2023-05-20T01:28:57.531 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-rw-r--. 1 ubuntu ubuntu 2 May 20 01:28 15 2023-05-20T01:28:57.531 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-rw-r--. 1 ubuntu ubuntu 2 May 20 01:28 16 2023-05-20T01:28:57.531 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-rw-r--. 1 ubuntu ubuntu 2 May 20 01:28 17 2023-05-20T01:28:57.531 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-rw-r--. 1 ubuntu ubuntu 2 May 20 01:28 18 2023-05-20T01:28:57.531 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-rw-r--. 1 ubuntu ubuntu 2 May 20 01:28 19 2023-05-20T01:28:57.531 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-rw-r--. 1 ubuntu ubuntu 1 May 20 01:28 2 2023-05-20T01:28:57.531 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-rw-r--. 1 ubuntu ubuntu 2 May 20 01:28 20 2023-05-20T01:28:57.532 INFO:teuthology.orchestra.run.smithi016.stdout:-r-x------. 1 root root 2 May 20 01:28 21 2023-05-20T01:28:57.532 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-rw-r--. 1 ubuntu ubuntu 2 May 20 01:28 22 2023-05-20T01:28:57.532 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-rw-r--. 1 ubuntu ubuntu 2 May 20 01:28 23 2023-05-20T01:28:57.532 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-rw-r--. 1 ubuntu ubuntu 2 May 20 01:28 24 2023-05-20T01:28:57.532 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-rw-r--. 1 ubuntu ubuntu 2 May 20 01:28 25 2023-05-20T01:28:57.532 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-rw-r--. 1 ubuntu ubuntu 2 May 20 01:28 26 2023-05-20T01:28:57.532 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-rw-r--. 1 ubuntu ubuntu 2 May 20 01:28 27 2023-05-20T01:28:57.533 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-rw-r--. 1 ubuntu ubuntu 2 May 20 01:28 28 2023-05-20T01:28:57.533 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-rw-r--. 1 ubuntu ubuntu 2 May 20 01:28 29 2023-05-20T01:28:57.533 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-rw-r--. 1 ubuntu ubuntu 1 May 20 01:28 3 2023-05-20T01:28:57.533 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-rw-r--. 1 ubuntu ubuntu 2 May 20 01:28 30 2023-05-20T01:28:57.533 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-rw-r--. 1 ubuntu ubuntu 2 May 20 01:28 31 2023-05-20T01:28:57.534 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-rw-r--. 1 ubuntu ubuntu 2 May 20 01:28 32 2023-05-20T01:28:57.534 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-rw-r--. 1 ubuntu ubuntu 2 May 20 01:28 33 2023-05-20T01:28:57.534 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-rw-r--. 1 ubuntu ubuntu 2 May 20 01:28 34 2023-05-20T01:28:57.534 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-rw-r--. 1 ubuntu ubuntu 2 May 20 01:28 35 2023-05-20T01:28:57.534 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-rw-r--. 1 ubuntu ubuntu 2 May 20 01:28 36 2023-05-20T01:28:57.535 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-rw-r--. 1 ubuntu ubuntu 2 May 20 01:28 37 2023-05-20T01:28:57.535 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-rw-r--. 1 ubuntu ubuntu 2 May 20 01:28 38 2023-05-20T01:28:57.535 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-rw-r--. 1 ubuntu ubuntu 2 May 20 01:28 39 2023-05-20T01:28:57.535 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-rw-r--. 1 ubuntu ubuntu 1 May 20 01:28 4 2023-05-20T01:28:57.535 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-rw-r--. 1 ubuntu ubuntu 2 May 20 01:28 40 2023-05-20T01:28:57.535 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-rw-r--. 1 ubuntu ubuntu 2 May 20 01:28 41 2023-05-20T01:28:57.535 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-rw-r--. 1 ubuntu ubuntu 2 May 20 01:28 42 2023-05-20T01:28:57.535 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-rw-r--. 1 ubuntu ubuntu 2 May 20 01:28 43 2023-05-20T01:28:57.536 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-rw-r--. 1 ubuntu ubuntu 2 May 20 01:28 44 2023-05-20T01:28:57.536 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-rw-r--. 1 ubuntu ubuntu 2 May 20 01:28 45 2023-05-20T01:28:57.536 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-rw-r--. 1 ubuntu ubuntu 2 May 20 01:28 46 2023-05-20T01:28:57.536 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-rw-r--. 1 ubuntu ubuntu 2 May 20 01:28 47 2023-05-20T01:28:57.536 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-rw-r--. 1 ubuntu ubuntu 2 May 20 01:28 48 2023-05-20T01:28:57.536 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-rw-r--. 1 ubuntu ubuntu 2 May 20 01:28 49 2023-05-20T01:28:57.536 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-rw-r--. 1 ubuntu ubuntu 1 May 20 01:28 5 2023-05-20T01:28:57.537 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-rw-r--. 1 ubuntu ubuntu 2 May 20 01:28 50 2023-05-20T01:28:57.537 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-rw-r--. 1 ubuntu ubuntu 2 May 20 01:28 51 2023-05-20T01:28:57.537 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-rw-r--. 1 ubuntu ubuntu 2 May 20 01:28 52 2023-05-20T01:28:57.537 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-rw-r--. 1 ubuntu ubuntu 2 May 20 01:28 53 2023-05-20T01:28:57.537 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-rw-r--. 1 ubuntu ubuntu 2 May 20 01:28 54 2023-05-20T01:28:57.537 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-rw-r--. 1 ubuntu ubuntu 2 May 20 01:28 55 2023-05-20T01:28:57.537 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-rw-r--. 1 ubuntu ubuntu 2 May 20 01:28 56 2023-05-20T01:28:57.537 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-rw-r--. 1 ubuntu ubuntu 2 May 20 01:28 57 2023-05-20T01:28:57.538 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-rw-r--. 1 ubuntu ubuntu 2 May 20 01:28 58 2023-05-20T01:28:57.538 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-rw-r--. 1 ubuntu ubuntu 2 May 20 01:28 59 2023-05-20T01:28:57.538 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-rw-r--. 1 ubuntu ubuntu 1 May 20 01:28 6 2023-05-20T01:28:57.538 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-rw-r--. 1 ubuntu ubuntu 2 May 20 01:28 60 2023-05-20T01:28:57.538 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-rw-r--. 1 ubuntu ubuntu 2 May 20 01:28 61 2023-05-20T01:28:57.538 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-rw-r--. 1 ubuntu ubuntu 2 May 20 01:28 62 2023-05-20T01:28:57.538 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-rw-r--. 1 ubuntu ubuntu 2 May 20 01:28 63 2023-05-20T01:28:57.539 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-rw-r--. 1 ubuntu ubuntu 2 May 20 01:28 64 2023-05-20T01:28:57.539 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-rw-r--. 1 ubuntu ubuntu 2 May 20 01:28 65 2023-05-20T01:28:57.539 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-rw-r--. 1 ubuntu ubuntu 2 May 20 01:28 66 2023-05-20T01:28:57.539 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-rw-r--. 1 ubuntu ubuntu 2 May 20 01:28 67 2023-05-20T01:28:57.540 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-rw-r--. 1 ubuntu ubuntu 2 May 20 01:28 68 2023-05-20T01:28:57.540 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-rw-r--. 1 ubuntu ubuntu 2 May 20 01:28 69 2023-05-20T01:28:57.540 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-rw-r--. 1 ubuntu ubuntu 1 May 20 01:28 7 2023-05-20T01:28:57.540 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-rw-r--. 1 ubuntu ubuntu 2 May 20 01:28 70 2023-05-20T01:28:57.541 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-rw-r--. 1 ubuntu ubuntu 2 May 20 01:28 71 2023-05-20T01:28:57.541 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-rw-r--. 1 ubuntu ubuntu 2 May 20 01:28 72 2023-05-20T01:28:57.541 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-rw-r--. 1 ubuntu ubuntu 2 May 20 01:28 73 2023-05-20T01:28:57.541 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-rw-r--. 1 ubuntu ubuntu 2 May 20 01:28 74 2023-05-20T01:28:57.541 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-rw-r--. 1 ubuntu ubuntu 2 May 20 01:28 75 2023-05-20T01:28:57.541 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-rw-r--. 1 ubuntu ubuntu 2 May 20 01:28 76 2023-05-20T01:28:57.541 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-rw-r--. 1 ubuntu ubuntu 2 May 20 01:28 77 2023-05-20T01:28:57.542 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-rw-r--. 1 ubuntu ubuntu 2 May 20 01:28 78 2023-05-20T01:28:57.542 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-rw-r--. 1 ubuntu ubuntu 2 May 20 01:28 79 2023-05-20T01:28:57.542 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-rw-r--. 1 ubuntu ubuntu 1 May 20 01:28 8 2023-05-20T01:28:57.542 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-rw-r--. 1 ubuntu ubuntu 2 May 20 01:28 80 2023-05-20T01:28:57.542 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-rw-r--. 1 ubuntu ubuntu 2 May 20 01:28 81 2023-05-20T01:28:57.543 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-rw-r--. 1 ubuntu ubuntu 2 May 20 01:28 82 2023-05-20T01:28:57.543 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-rw-r--. 1 ubuntu ubuntu 2 May 20 01:28 83 2023-05-20T01:28:57.543 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-rw-r--. 1 ubuntu ubuntu 2 May 20 01:28 84 2023-05-20T01:28:57.543 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-rw-r--. 1 ubuntu ubuntu 2 May 20 01:28 85 2023-05-20T01:28:57.544 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-rw-r--. 1 ubuntu ubuntu 2 May 20 01:28 86 2023-05-20T01:28:57.544 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-rw-r--. 1 ubuntu ubuntu 2 May 20 01:28 87 2023-05-20T01:28:57.544 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-rw-r--. 1 ubuntu ubuntu 2 May 20 01:28 88 2023-05-20T01:28:57.544 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-rw-r--. 1 ubuntu ubuntu 2 May 20 01:28 89 2023-05-20T01:28:57.544 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-rw-r--. 1 ubuntu ubuntu 1 May 20 01:28 9 2023-05-20T01:28:57.544 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-rw-r--. 1 ubuntu ubuntu 2 May 20 01:28 90 2023-05-20T01:28:57.544 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-rw-r--. 1 ubuntu ubuntu 2 May 20 01:28 91 2023-05-20T01:28:57.545 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-rw-r--. 1 ubuntu ubuntu 2 May 20 01:28 92 2023-05-20T01:28:57.545 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-rw-r--. 1 ubuntu ubuntu 2 May 20 01:28 93 2023-05-20T01:28:57.545 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-rw-r--. 1 ubuntu ubuntu 2 May 20 01:28 94 2023-05-20T01:28:57.545 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-rw-r--. 1 ubuntu ubuntu 2 May 20 01:28 95 2023-05-20T01:28:57.545 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-rw-r--. 1 ubuntu ubuntu 2 May 20 01:28 96 2023-05-20T01:28:57.545 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-rw-r--. 1 ubuntu ubuntu 2 May 20 01:28 97 2023-05-20T01:28:57.546 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-rw-r--. 1 ubuntu ubuntu 2 May 20 01:28 98 2023-05-20T01:28:57.546 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-rw-r--. 1 ubuntu ubuntu 2 May 20 01:28 99 2023-05-20T01:28:57.547 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:28:57.547 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo cat subdir/21') 2023-05-20T01:28:57.592 INFO:teuthology.orchestra.run.smithi016.stdout:21 2023-05-20T01:28:57.592 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-05-20T01:28:57.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:57.752+0000 7fc8c66b3700 1 Processor -- start 2023-05-20T01:28:57.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:57.753+0000 7fc8c66b3700 1 -- start start 2023-05-20T01:28:57.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:57.753+0000 7fc8c66b3700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fc8c00b5e90 0x7fc8c015e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:57.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:57.753+0000 7fc8c66b3700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8c00cbbc0 con 0x7fc8c00b58d0 2023-05-20T01:28:57.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:57.753+0000 7fc8c66b3700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8c015eb20 con 0x7fc8c0071eb0 2023-05-20T01:28:57.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:57.753+0000 7fc8c66b3700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8c00cbdf0 con 0x7fc8c00b5e90 2023-05-20T01:28:57.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:57.753+0000 7fc8c4c50700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fc8c00b58d0 0x7fc8c00b1f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:52516/0 (socket says 172.21.15.16:52516) 2023-05-20T01:28:57.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:57.753+0000 7fc8c4c50700 1 -- 172.21.15.16:0/146563102 learned_addr learned my addr 172.21.15.16:0/146563102 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:28:57.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:57.754+0000 7fc8bf7fe700 1 --2- 172.21.15.16:0/146563102 >> v2:172.21.15.132:3300/0 conn(0x7fc8c00b5e90 0x7fc8c015e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:57.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:57.754+0000 7fc8beffd700 1 -- 172.21.15.16:0/146563102 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2561900523 0 0) 0x7fc8c00cbbc0 con 0x7fc8c00b58d0 2023-05-20T01:28:57.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:57.754+0000 7fc8beffd700 1 -- 172.21.15.16:0/146563102 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc8a80035b0 con 0x7fc8c00b58d0 2023-05-20T01:28:57.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:57.754+0000 7fc8beffd700 1 -- 172.21.15.16:0/146563102 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 285750268 0 0) 0x7fc8a80035b0 con 0x7fc8c00b58d0 2023-05-20T01:28:57.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:57.754+0000 7fc8bf7fe700 1 -- 172.21.15.16:0/146563102 >> v1:172.21.15.132:6789/0 conn(0x7fc8c0071eb0 legacy=0x7fc8c0072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:57.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:57.754+0000 7fc8bf7fe700 1 -- 172.21.15.16:0/146563102 >> v1:172.21.15.16:6789/0 conn(0x7fc8c00b58d0 legacy=0x7fc8c00b1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:57.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:57.754+0000 7fc8bf7fe700 1 -- 172.21.15.16:0/146563102 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8c00cc0d0 con 0x7fc8c00b5e90 2023-05-20T01:28:57.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:57.754+0000 7fc8bf7fe700 1 --2- 172.21.15.16:0/146563102 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc8c00b5e90 0x7fc8c015e5e0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fc8b0009840 tx=0x7fc8b0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a102d3aafd24e62 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:57.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:57.754+0000 7fc8beffd700 1 -- 172.21.15.16:0/146563102 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b80019f0 con 0x7fc8c00b5e90 2023-05-20T01:28:57.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:57.754+0000 7fc8beffd700 1 -- 172.21.15.16:0/146563102 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8b0004030 con 0x7fc8c00b5e90 2023-05-20T01:28:57.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:57.754+0000 7fc8beffd700 1 -- 172.21.15.16:0/146563102 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b00043c0 con 0x7fc8c00b5e90 2023-05-20T01:28:57.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:57.755+0000 7fc8c66b3700 1 -- 172.21.15.16:0/146563102 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc8c00b5e90 msgr2=0x7fc8c015e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:57.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:57.755+0000 7fc8c66b3700 1 --2- 172.21.15.16:0/146563102 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc8c00b5e90 0x7fc8c015e5e0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fc8b0009840 tx=0x7fc8b0009b50 comp rx=0 tx=0).stop 2023-05-20T01:28:57.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:57.755+0000 7fc8c66b3700 1 -- 172.21.15.16:0/146563102 shutdown_connections 2023-05-20T01:28:57.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:57.755+0000 7fc8c66b3700 1 --2- 172.21.15.16:0/146563102 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc8c00b5e90 0x7fc8c015e5e0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:57.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:57.755+0000 7fc8c66b3700 1 -- 172.21.15.16:0/146563102 >> 172.21.15.16:0/146563102 conn(0x7fc8c006e250 msgr2=0x7fc8c00ccc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:57.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:57.755+0000 7fc8c66b3700 1 -- 172.21.15.16:0/146563102 shutdown_connections 2023-05-20T01:28:57.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:57.755+0000 7fc8c66b3700 1 -- 172.21.15.16:0/146563102 wait complete. 2023-05-20T01:28:57.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:57.756+0000 7fc8c66b3700 1 Processor -- start 2023-05-20T01:28:57.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:57.756+0000 7fc8c66b3700 1 -- start start 2023-05-20T01:28:57.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:57.757+0000 7fc8c66b3700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc8c0071eb0 0x7fc8c015c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:57.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:57.757+0000 7fc8c66b3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8c00b58d0 0x7fc8c00b34a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:57.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:57.757+0000 7fc8c66b3700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc8c00b5e90 0x7fc8c00b39e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:57.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:57.757+0000 7fc8c66b3700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8c015d2e0 con 0x7fc8c00b58d0 2023-05-20T01:28:57.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:57.757+0000 7fc8c66b3700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8c00c5c90 con 0x7fc8c0071eb0 2023-05-20T01:28:57.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:57.757+0000 7fc8c66b3700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8c00c5f70 con 0x7fc8c00b5e90 2023-05-20T01:28:57.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:57.757+0000 7fc8bf7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8c00b58d0 0x7fc8c00b34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:57.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:57.757+0000 7fc8bf7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8c00b58d0 0x7fc8c00b34a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40992/0 (socket says 172.21.15.16:40992) 2023-05-20T01:28:57.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:57.757+0000 7fc8bf7fe700 1 -- 172.21.15.16:0/1918913015 learned_addr learned my addr 172.21.15.16:0/1918913015 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:57.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:57.758+0000 7fc8c4c50700 1 --2- 172.21.15.16:0/1918913015 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc8c00b5e90 0x7fc8c00b39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:57.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:57.758+0000 7fc8bffff700 1 --2- 172.21.15.16:0/1918913015 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc8c0071eb0 0x7fc8c015c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:57.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:57.758+0000 7fc8bf7fe700 1 -- 172.21.15.16:0/1918913015 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc8c00b5e90 msgr2=0x7fc8c00b39e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:57.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:57.758+0000 7fc8bf7fe700 1 --2- 172.21.15.16:0/1918913015 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc8c00b5e90 0x7fc8c00b39e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:57.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:57.758+0000 7fc8bf7fe700 1 -- 172.21.15.16:0/1918913015 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc8c0071eb0 msgr2=0x7fc8c015c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:57.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:57.758+0000 7fc8bf7fe700 1 --2- 172.21.15.16:0/1918913015 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc8c0071eb0 0x7fc8c015c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:57.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:57.758+0000 7fc8bf7fe700 1 -- 172.21.15.16:0/1918913015 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8b00094a0 con 0x7fc8c00b58d0 2023-05-20T01:28:57.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:57.758+0000 7fc8bffff700 1 --2- 172.21.15.16:0/1918913015 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc8c0071eb0 0x7fc8c015c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:28:57.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:57.758+0000 7fc8bf7fe700 1 --2- 172.21.15.16:0/1918913015 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8c00b58d0 0x7fc8c00b34a0 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7fc8b0009b90 tx=0x7fc8b0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:57.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:57.758+0000 7fc8bcff9700 1 -- 172.21.15.16:0/1918913015 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b0011830 con 0x7fc8c00b58d0 2023-05-20T01:28:57.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:57.758+0000 7fc8bcff9700 1 -- 172.21.15.16:0/1918913015 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8b001f070 con 0x7fc8c00b58d0 2023-05-20T01:28:57.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:57.758+0000 7fc8c66b3700 1 -- 172.21.15.16:0/1918913015 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8c00cc690 con 0x7fc8c00b58d0 2023-05-20T01:28:57.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:57.758+0000 7fc8bcff9700 1 -- 172.21.15.16:0/1918913015 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b0011c10 con 0x7fc8c00b58d0 2023-05-20T01:28:57.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:57.758+0000 7fc8c66b3700 1 -- 172.21.15.16:0/1918913015 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8c00cc3b0 con 0x7fc8c00b58d0 2023-05-20T01:28:57.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:57.759+0000 7fc8c66b3700 1 -- 172.21.15.16:0/1918913015 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8c00cca00 con 0x7fc8c00b58d0 2023-05-20T01:28:57.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:57.759+0000 7fc8bcff9700 1 -- 172.21.15.16:0/1918913015 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc8b001a440 con 0x7fc8c00b58d0 2023-05-20T01:28:57.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:57.760+0000 7fc8bcff9700 1 --2- 172.21.15.16:0/1918913015 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc8ac06ace0 0x7fc8ac06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:57.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:57.760+0000 7fc8bcff9700 1 -- 172.21.15.16:0/1918913015 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6554+0+0 (secure 0 0 0) 0x7fc8b0021070 con 0x7fc8c00b58d0 2023-05-20T01:28:57.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:57.760+0000 7fc8bffff700 1 --2- 172.21.15.16:0/1918913015 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc8ac06ace0 0x7fc8ac06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:57.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:57.760+0000 7fc8bffff700 1 --2- 172.21.15.16:0/1918913015 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc8ac06ace0 0x7fc8ac06d190 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fc8b4000f60 tx=0x7fc8b4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:57.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:57.764+0000 7fc8bcff9700 1 -- 172.21.15.16:0/1918913015 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fc8b0056000 con 0x7fc8c00b58d0 2023-05-20T01:28:58.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.004+0000 7fc8c66b3700 1 -- 172.21.15.16:0/1918913015 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc8c0066b00 con 0x7fc8c00b58d0 2023-05-20T01:28:58.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.004+0000 7fc8bcff9700 1 -- 172.21.15.16:0/1918913015 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) v1 ==== 95+0+4684 (secure 0 0 0) 0x7fc8b0059620 con 0x7fc8c00b58d0 2023-05-20T01:28:58.009 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-05-20T01:28:58.009 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":112,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7043,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/1859755557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1859755557},{"type":"v1","addr":"172.21.15.16:6836","nonce":1859755557}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94},{"gid":7046,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/2654783598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":2654783598},{"type":"v1","addr":"172.21.15.132:6834","nonce":2654783598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94},{"gid":7049,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2042939723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":2042939723},{"type":"v1","addr":"172.21.15.132:6835","nonce":2042939723}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-05-20T01:28:27.093957+0000","modified":"2023-05-20T01:28:54.632173+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":138,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9054},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9054":{"gid":9054,"name":"a","rank":0,"incarnation":109,"state":"up:active","state_seq":4,"addr":"172.21.15.16:6837/2952207608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":2952207608},{"type":"v1","addr":"172.21.15.16:6837","nonce":2952207608}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-05-20T01:28:58.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.006+0000 7fc8c66b3700 1 -- 172.21.15.16:0/1918913015 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc8ac06ace0 msgr2=0x7fc8ac06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:58.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.006+0000 7fc8c66b3700 1 --2- 172.21.15.16:0/1918913015 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc8ac06ace0 0x7fc8ac06d190 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fc8b4000f60 tx=0x7fc8b4009040 comp rx=0 tx=0).stop 2023-05-20T01:28:58.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.006+0000 7fc8c66b3700 1 -- 172.21.15.16:0/1918913015 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8c00b58d0 msgr2=0x7fc8c00b34a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:58.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.006+0000 7fc8c66b3700 1 --2- 172.21.15.16:0/1918913015 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8c00b58d0 0x7fc8c00b34a0 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7fc8b0009b90 tx=0x7fc8b0004750 comp rx=0 tx=0).stop 2023-05-20T01:28:58.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.006+0000 7fc8c66b3700 1 -- 172.21.15.16:0/1918913015 shutdown_connections 2023-05-20T01:28:58.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.006+0000 7fc8c66b3700 1 --2- 172.21.15.16:0/1918913015 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fc8ac06ace0 0x7fc8ac06d190 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:58.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.006+0000 7fc8c66b3700 1 --2- 172.21.15.16:0/1918913015 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc8c00b5e90 0x7fc8c00b39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:58.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.006+0000 7fc8c66b3700 1 --2- 172.21.15.16:0/1918913015 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc8c0071eb0 0x7fc8c015c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:58.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.006+0000 7fc8c66b3700 1 --2- 172.21.15.16:0/1918913015 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8c00b58d0 0x7fc8c00b34a0 unknown :-1 s=CLOSED pgs=1748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:58.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.006+0000 7fc8c66b3700 1 -- 172.21.15.16:0/1918913015 >> 172.21.15.16:0/1918913015 conn(0x7fc8c006e250 msgr2=0x7fc8c00b2110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:58.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.006+0000 7fc8c66b3700 1 -- 172.21.15.16:0/1918913015 shutdown_connections 2023-05-20T01:28:58.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.006+0000 7fc8c66b3700 1 -- 172.21.15.16:0/1918913015 wait complete. 2023-05-20T01:28:58.013 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 112 2023-05-20T01:28:58.022 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:28:58.023 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls subdir) 2023-05-20T01:28:58.043 INFO:teuthology.orchestra.run.smithi016.stdout:0 2023-05-20T01:28:58.044 INFO:teuthology.orchestra.run.smithi016.stdout:1 2023-05-20T01:28:58.044 INFO:teuthology.orchestra.run.smithi016.stdout:10 2023-05-20T01:28:58.044 INFO:teuthology.orchestra.run.smithi016.stdout:11 2023-05-20T01:28:58.044 INFO:teuthology.orchestra.run.smithi016.stdout:12 2023-05-20T01:28:58.045 INFO:teuthology.orchestra.run.smithi016.stdout:13 2023-05-20T01:28:58.045 INFO:teuthology.orchestra.run.smithi016.stdout:14 2023-05-20T01:28:58.045 INFO:teuthology.orchestra.run.smithi016.stdout:15 2023-05-20T01:28:58.045 INFO:teuthology.orchestra.run.smithi016.stdout:16 2023-05-20T01:28:58.045 INFO:teuthology.orchestra.run.smithi016.stdout:17 2023-05-20T01:28:58.045 INFO:teuthology.orchestra.run.smithi016.stdout:18 2023-05-20T01:28:58.045 INFO:teuthology.orchestra.run.smithi016.stdout:19 2023-05-20T01:28:58.046 INFO:teuthology.orchestra.run.smithi016.stdout:2 2023-05-20T01:28:58.046 INFO:teuthology.orchestra.run.smithi016.stdout:20 2023-05-20T01:28:58.046 INFO:teuthology.orchestra.run.smithi016.stdout:21 2023-05-20T01:28:58.046 INFO:teuthology.orchestra.run.smithi016.stdout:22 2023-05-20T01:28:58.046 INFO:teuthology.orchestra.run.smithi016.stdout:23 2023-05-20T01:28:58.046 INFO:teuthology.orchestra.run.smithi016.stdout:24 2023-05-20T01:28:58.046 INFO:teuthology.orchestra.run.smithi016.stdout:25 2023-05-20T01:28:58.046 INFO:teuthology.orchestra.run.smithi016.stdout:26 2023-05-20T01:28:58.047 INFO:teuthology.orchestra.run.smithi016.stdout:27 2023-05-20T01:28:58.047 INFO:teuthology.orchestra.run.smithi016.stdout:28 2023-05-20T01:28:58.047 INFO:teuthology.orchestra.run.smithi016.stdout:29 2023-05-20T01:28:58.047 INFO:teuthology.orchestra.run.smithi016.stdout:3 2023-05-20T01:28:58.047 INFO:teuthology.orchestra.run.smithi016.stdout:30 2023-05-20T01:28:58.047 INFO:teuthology.orchestra.run.smithi016.stdout:31 2023-05-20T01:28:58.047 INFO:teuthology.orchestra.run.smithi016.stdout:32 2023-05-20T01:28:58.047 INFO:teuthology.orchestra.run.smithi016.stdout:33 2023-05-20T01:28:58.048 INFO:teuthology.orchestra.run.smithi016.stdout:34 2023-05-20T01:28:58.048 INFO:teuthology.orchestra.run.smithi016.stdout:35 2023-05-20T01:28:58.048 INFO:teuthology.orchestra.run.smithi016.stdout:36 2023-05-20T01:28:58.048 INFO:teuthology.orchestra.run.smithi016.stdout:37 2023-05-20T01:28:58.048 INFO:teuthology.orchestra.run.smithi016.stdout:38 2023-05-20T01:28:58.049 INFO:teuthology.orchestra.run.smithi016.stdout:39 2023-05-20T01:28:58.049 INFO:teuthology.orchestra.run.smithi016.stdout:4 2023-05-20T01:28:58.049 INFO:teuthology.orchestra.run.smithi016.stdout:40 2023-05-20T01:28:58.049 INFO:teuthology.orchestra.run.smithi016.stdout:41 2023-05-20T01:28:58.049 INFO:teuthology.orchestra.run.smithi016.stdout:42 2023-05-20T01:28:58.049 INFO:teuthology.orchestra.run.smithi016.stdout:43 2023-05-20T01:28:58.049 INFO:teuthology.orchestra.run.smithi016.stdout:44 2023-05-20T01:28:58.050 INFO:teuthology.orchestra.run.smithi016.stdout:45 2023-05-20T01:28:58.050 INFO:teuthology.orchestra.run.smithi016.stdout:46 2023-05-20T01:28:58.050 INFO:teuthology.orchestra.run.smithi016.stdout:47 2023-05-20T01:28:58.050 INFO:teuthology.orchestra.run.smithi016.stdout:48 2023-05-20T01:28:58.050 INFO:teuthology.orchestra.run.smithi016.stdout:49 2023-05-20T01:28:58.050 INFO:teuthology.orchestra.run.smithi016.stdout:5 2023-05-20T01:28:58.051 INFO:teuthology.orchestra.run.smithi016.stdout:50 2023-05-20T01:28:58.051 INFO:teuthology.orchestra.run.smithi016.stdout:51 2023-05-20T01:28:58.051 INFO:teuthology.orchestra.run.smithi016.stdout:52 2023-05-20T01:28:58.051 INFO:teuthology.orchestra.run.smithi016.stdout:53 2023-05-20T01:28:58.051 INFO:teuthology.orchestra.run.smithi016.stdout:54 2023-05-20T01:28:58.051 INFO:teuthology.orchestra.run.smithi016.stdout:55 2023-05-20T01:28:58.051 INFO:teuthology.orchestra.run.smithi016.stdout:56 2023-05-20T01:28:58.052 INFO:teuthology.orchestra.run.smithi016.stdout:57 2023-05-20T01:28:58.052 INFO:teuthology.orchestra.run.smithi016.stdout:58 2023-05-20T01:28:58.052 INFO:teuthology.orchestra.run.smithi016.stdout:59 2023-05-20T01:28:58.052 INFO:teuthology.orchestra.run.smithi016.stdout:6 2023-05-20T01:28:58.052 INFO:teuthology.orchestra.run.smithi016.stdout:60 2023-05-20T01:28:58.052 INFO:teuthology.orchestra.run.smithi016.stdout:61 2023-05-20T01:28:58.052 INFO:teuthology.orchestra.run.smithi016.stdout:62 2023-05-20T01:28:58.052 INFO:teuthology.orchestra.run.smithi016.stdout:63 2023-05-20T01:28:58.053 INFO:teuthology.orchestra.run.smithi016.stdout:64 2023-05-20T01:28:58.053 INFO:teuthology.orchestra.run.smithi016.stdout:65 2023-05-20T01:28:58.053 INFO:teuthology.orchestra.run.smithi016.stdout:66 2023-05-20T01:28:58.053 INFO:teuthology.orchestra.run.smithi016.stdout:67 2023-05-20T01:28:58.053 INFO:teuthology.orchestra.run.smithi016.stdout:68 2023-05-20T01:28:58.053 INFO:teuthology.orchestra.run.smithi016.stdout:69 2023-05-20T01:28:58.053 INFO:teuthology.orchestra.run.smithi016.stdout:7 2023-05-20T01:28:58.053 INFO:teuthology.orchestra.run.smithi016.stdout:70 2023-05-20T01:28:58.054 INFO:teuthology.orchestra.run.smithi016.stdout:71 2023-05-20T01:28:58.054 INFO:teuthology.orchestra.run.smithi016.stdout:72 2023-05-20T01:28:58.054 INFO:teuthology.orchestra.run.smithi016.stdout:73 2023-05-20T01:28:58.054 INFO:teuthology.orchestra.run.smithi016.stdout:74 2023-05-20T01:28:58.054 INFO:teuthology.orchestra.run.smithi016.stdout:75 2023-05-20T01:28:58.054 INFO:teuthology.orchestra.run.smithi016.stdout:76 2023-05-20T01:28:58.054 INFO:teuthology.orchestra.run.smithi016.stdout:77 2023-05-20T01:28:58.055 INFO:teuthology.orchestra.run.smithi016.stdout:78 2023-05-20T01:28:58.055 INFO:teuthology.orchestra.run.smithi016.stdout:79 2023-05-20T01:28:58.055 INFO:teuthology.orchestra.run.smithi016.stdout:8 2023-05-20T01:28:58.055 INFO:teuthology.orchestra.run.smithi016.stdout:80 2023-05-20T01:28:58.055 INFO:teuthology.orchestra.run.smithi016.stdout:81 2023-05-20T01:28:58.055 INFO:teuthology.orchestra.run.smithi016.stdout:82 2023-05-20T01:28:58.055 INFO:teuthology.orchestra.run.smithi016.stdout:83 2023-05-20T01:28:58.055 INFO:teuthology.orchestra.run.smithi016.stdout:84 2023-05-20T01:28:58.056 INFO:teuthology.orchestra.run.smithi016.stdout:85 2023-05-20T01:28:58.056 INFO:teuthology.orchestra.run.smithi016.stdout:86 2023-05-20T01:28:58.056 INFO:teuthology.orchestra.run.smithi016.stdout:87 2023-05-20T01:28:58.056 INFO:teuthology.orchestra.run.smithi016.stdout:88 2023-05-20T01:28:58.056 INFO:teuthology.orchestra.run.smithi016.stdout:89 2023-05-20T01:28:58.056 INFO:teuthology.orchestra.run.smithi016.stdout:9 2023-05-20T01:28:58.056 INFO:teuthology.orchestra.run.smithi016.stdout:90 2023-05-20T01:28:58.056 INFO:teuthology.orchestra.run.smithi016.stdout:91 2023-05-20T01:28:58.057 INFO:teuthology.orchestra.run.smithi016.stdout:92 2023-05-20T01:28:58.057 INFO:teuthology.orchestra.run.smithi016.stdout:93 2023-05-20T01:28:58.057 INFO:teuthology.orchestra.run.smithi016.stdout:94 2023-05-20T01:28:58.057 INFO:teuthology.orchestra.run.smithi016.stdout:95 2023-05-20T01:28:58.057 INFO:teuthology.orchestra.run.smithi016.stdout:96 2023-05-20T01:28:58.057 INFO:teuthology.orchestra.run.smithi016.stdout:97 2023-05-20T01:28:58.057 INFO:teuthology.orchestra.run.smithi016.stdout:98 2023-05-20T01:28:58.058 INFO:teuthology.orchestra.run.smithi016.stdout:99 2023-05-20T01:28:58.058 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json dirfrag merge /subdir 0/0 2023-05-20T01:28:58.226 DEBUG:tasks.cephfs.filesystem:_json_asok output empty 2023-05-20T01:28:58.226 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2023-05-20T01:28:58.377 INFO:teuthology.orchestra.run.smithi016.stdout:{"message":"","return_code":0} 2023-05-20T01:28:58.377 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2023-05-20T01:28:58.378 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys 10000000000.00000000 2023-05-20T01:28:58.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.416+0000 7fb31bf0f080 1 Processor -- start 2023-05-20T01:28:58.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.416+0000 7fb31bf0f080 1 -- start start 2023-05-20T01:28:58.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.417+0000 7fb31bf0f080 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55564ad2d280 0x55564ade58b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:58.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.417+0000 7fb31bf0f080 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55564ad5ad10 0x55564ad76bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:58.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.417+0000 7fb31bf0f080 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55564ae01350 0x55564ad77130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:58.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.417+0000 7fb31bf0f080 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55564ac90e80 con 0x55564ad5ad10 2023-05-20T01:28:58.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.417+0000 7fb31bf0f080 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55564ac90b20 con 0x55564ad2d280 2023-05-20T01:28:58.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.417+0000 7fb31bf0f080 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55564ac66fe0 con 0x55564ae01350 2023-05-20T01:28:58.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.417+0000 7fb31180e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55564ad5ad10 0x55564ad76bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:58.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.417+0000 7fb31180e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55564ad5ad10 0x55564ad76bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41006/0 (socket says 172.21.15.16:41006) 2023-05-20T01:28:58.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.417+0000 7fb31180e700 1 -- 172.21.15.16:0/1584099097 learned_addr learned my addr 172.21.15.16:0/1584099097 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:58.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.418+0000 7fb31180e700 1 -- 172.21.15.16:0/1584099097 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55564ae01350 msgr2=0x55564ad77130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:58.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.418+0000 7fb31200f700 1 --2- 172.21.15.16:0/1584099097 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55564ad2d280 0x55564ade58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:58.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.418+0000 7fb312810700 1 --2- 172.21.15.16:0/1584099097 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55564ae01350 0x55564ad77130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:58.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.418+0000 7fb31180e700 1 --2- 172.21.15.16:0/1584099097 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55564ae01350 0x55564ad77130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:58.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.418+0000 7fb31180e700 1 -- 172.21.15.16:0/1584099097 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55564ad2d280 msgr2=0x55564ade58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:58.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.418+0000 7fb31180e700 1 --2- 172.21.15.16:0/1584099097 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55564ad2d280 0x55564ade58b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:58.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.418+0000 7fb31180e700 1 -- 172.21.15.16:0/1584099097 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb308005d20 con 0x55564ad5ad10 2023-05-20T01:28:58.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.418+0000 7fb31200f700 1 --2- 172.21.15.16:0/1584099097 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55564ad2d280 0x55564ade58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:28:58.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.418+0000 7fb312810700 1 --2- 172.21.15.16:0/1584099097 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55564ae01350 0x55564ad77130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:28:58.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.418+0000 7fb31180e700 1 --2- 172.21.15.16:0/1584099097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55564ad5ad10 0x55564ad76bf0 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7fb304009790 tx=0x7fb304009aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=669c9d5a4dcc3b83 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:58.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.418+0000 7fb31080c700 1 -- 172.21.15.16:0/1584099097 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb30400e070 con 0x55564ad5ad10 2023-05-20T01:28:58.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.418+0000 7fb31080c700 1 -- 172.21.15.16:0/1584099097 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb304006440 con 0x55564ad5ad10 2023-05-20T01:28:58.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.418+0000 7fb31080c700 1 -- 172.21.15.16:0/1584099097 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3040067d0 con 0x55564ad5ad10 2023-05-20T01:28:58.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.418+0000 7fb31bf0f080 1 -- 172.21.15.16:0/1584099097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55564ad5ad10 msgr2=0x55564ad76bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:58.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.418+0000 7fb31bf0f080 1 --2- 172.21.15.16:0/1584099097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55564ad5ad10 0x55564ad76bf0 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7fb304009790 tx=0x7fb304009aa0 comp rx=0 tx=0).stop 2023-05-20T01:28:58.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.418+0000 7fb31bf0f080 1 -- 172.21.15.16:0/1584099097 shutdown_connections 2023-05-20T01:28:58.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.418+0000 7fb31bf0f080 1 --2- 172.21.15.16:0/1584099097 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55564ae01350 0x55564ad77130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:58.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.418+0000 7fb31bf0f080 1 --2- 172.21.15.16:0/1584099097 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55564ad2d280 0x55564ade58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:58.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.418+0000 7fb31bf0f080 1 --2- 172.21.15.16:0/1584099097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55564ad5ad10 0x55564ad76bf0 unknown :-1 s=CLOSED pgs=1751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:58.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.418+0000 7fb31bf0f080 1 -- 172.21.15.16:0/1584099097 >> 172.21.15.16:0/1584099097 conn(0x55564ad77d00 msgr2=0x55564adfe0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:58.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.418+0000 7fb31bf0f080 1 -- 172.21.15.16:0/1584099097 shutdown_connections 2023-05-20T01:28:58.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.418+0000 7fb31bf0f080 1 -- 172.21.15.16:0/1584099097 wait complete. 2023-05-20T01:28:58.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.419+0000 7fb31bf0f080 1 Processor -- start 2023-05-20T01:28:58.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.419+0000 7fb31bf0f080 1 -- start start 2023-05-20T01:28:58.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.420+0000 7fb31bf0f080 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55564ad2d280 0x55564ade4b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:58.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.420+0000 7fb31bf0f080 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55564ad5ad10 0x55564adff8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:58.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.420+0000 7fb31bf0f080 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55564ae01350 0x55564adffe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:58.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.420+0000 7fb31bf0f080 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55564ade50a0 con 0x55564ad5ad10 2023-05-20T01:28:58.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.420+0000 7fb31bf0f080 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55564ae00360 con 0x55564ad2d280 2023-05-20T01:28:58.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.420+0000 7fb31bf0f080 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55564adfc5d0 con 0x55564ae01350 2023-05-20T01:28:58.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.420+0000 7fb31180e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55564ad5ad10 0x55564adff8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:58.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.420+0000 7fb31180e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55564ad5ad10 0x55564adff8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41012/0 (socket says 172.21.15.16:41012) 2023-05-20T01:28:58.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.420+0000 7fb31180e700 1 -- 172.21.15.16:0/2193639465 learned_addr learned my addr 172.21.15.16:0/2193639465 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:58.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.420+0000 7fb31180e700 1 -- 172.21.15.16:0/2193639465 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55564ae01350 msgr2=0x55564adffe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:58.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.420+0000 7fb31180e700 1 --2- 172.21.15.16:0/2193639465 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55564ae01350 0x55564adffe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:58.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.420+0000 7fb31180e700 1 -- 172.21.15.16:0/2193639465 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55564ad2d280 msgr2=0x55564ade4b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:58.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.420+0000 7fb31180e700 1 --2- 172.21.15.16:0/2193639465 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55564ad2d280 0x55564ade4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:58.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.420+0000 7fb31180e700 1 -- 172.21.15.16:0/2193639465 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb304009410 con 0x55564ad5ad10 2023-05-20T01:28:58.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.420+0000 7fb31180e700 1 --2- 172.21.15.16:0/2193639465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55564ad5ad10 0x55564adff8e0 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7fb304011000 tx=0x7fb304003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:58.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.420+0000 7fb3037fe700 1 -- 172.21.15.16:0/2193639465 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb304005690 con 0x55564ad5ad10 2023-05-20T01:28:58.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.421+0000 7fb3037fe700 1 -- 172.21.15.16:0/2193639465 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb30401c650 con 0x55564ad5ad10 2023-05-20T01:28:58.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.421+0000 7fb31bf0f080 1 -- 172.21.15.16:0/2193639465 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55564ac724a0 con 0x55564ad5ad10 2023-05-20T01:28:58.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.421+0000 7fb3037fe700 1 -- 172.21.15.16:0/2193639465 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb30401c9e0 con 0x55564ad5ad10 2023-05-20T01:28:58.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.421+0000 7fb31bf0f080 1 -- 172.21.15.16:0/2193639465 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55564ac7ea80 con 0x55564ad5ad10 2023-05-20T01:28:58.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.422+0000 7fb3037fe700 1 -- 172.21.15.16:0/2193639465 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb30401cb40 con 0x55564ad5ad10 2023-05-20T01:28:58.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.422+0000 7fb3037fe700 1 --2- 172.21.15.16:0/2193639465 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb2f4069600 0x7fb2f406bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:58.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.422+0000 7fb31200f700 1 --2- 172.21.15.16:0/2193639465 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb2f4069600 0x7fb2f406bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:58.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.422+0000 7fb3037fe700 1 -- 172.21.15.16:0/2193639465 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6554+0+0 (secure 0 0 0) 0x7fb304089660 con 0x55564ad5ad10 2023-05-20T01:28:58.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.423+0000 7fb31bf0f080 1 --2- 172.21.15.16:0/2193639465 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x55564ad1b100 0x55564adfb050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:58.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.423+0000 7fb31bf0f080 1 -- 172.21.15.16:0/2193639465 --> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] -- osd_op(unknown.0.0:1 18.10 18:0cf6ad0f:::10000000000.00000000:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138) v8 -- 0x55564adfb590 con 0x55564ad1b100 2023-05-20T01:28:58.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.423+0000 7fb31200f700 1 --2- 172.21.15.16:0/2193639465 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb2f4069600 0x7fb2f406bab0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fb308015040 tx=0x7fb30800d000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:58.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.423+0000 7fb312810700 1 --2- 172.21.15.16:0/2193639465 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x55564ad1b100 0x55564adfb050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:58.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.423+0000 7fb312810700 1 --2- 172.21.15.16:0/2193639465 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x55564ad1b100 0x55564adfb050 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:58.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.424+0000 7fb312810700 1 -- 172.21.15.16:0/2193639465 <== osd.6 v2:172.21.15.132:6801/727581123 1 ==== osd_op_reply(1 10000000000.00000000 [omap-get-keys out=1095b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+1095 (crc 0 0 0) 0x7fb30c004450 con 0x55564ad1b100 2023-05-20T01:28:58.437 INFO:teuthology.orchestra.run.smithi016.stdout:0_head 2023-05-20T01:28:58.437 INFO:teuthology.orchestra.run.smithi016.stdout:10_head 2023-05-20T01:28:58.438 INFO:teuthology.orchestra.run.smithi016.stdout:11_head 2023-05-20T01:28:58.438 INFO:teuthology.orchestra.run.smithi016.stdout:12_head 2023-05-20T01:28:58.438 INFO:teuthology.orchestra.run.smithi016.stdout:13_head 2023-05-20T01:28:58.438 INFO:teuthology.orchestra.run.smithi016.stdout:14_head 2023-05-20T01:28:58.438 INFO:teuthology.orchestra.run.smithi016.stdout:15_head 2023-05-20T01:28:58.438 INFO:teuthology.orchestra.run.smithi016.stdout:16_head 2023-05-20T01:28:58.439 INFO:teuthology.orchestra.run.smithi016.stdout:17_head 2023-05-20T01:28:58.439 INFO:teuthology.orchestra.run.smithi016.stdout:18_head 2023-05-20T01:28:58.439 INFO:teuthology.orchestra.run.smithi016.stdout:19_head 2023-05-20T01:28:58.439 INFO:teuthology.orchestra.run.smithi016.stdout:1_head 2023-05-20T01:28:58.440 INFO:teuthology.orchestra.run.smithi016.stdout:20_head 2023-05-20T01:28:58.440 INFO:teuthology.orchestra.run.smithi016.stdout:21_head 2023-05-20T01:28:58.440 INFO:teuthology.orchestra.run.smithi016.stdout:22_head 2023-05-20T01:28:58.440 INFO:teuthology.orchestra.run.smithi016.stdout:23_head 2023-05-20T01:28:58.440 INFO:teuthology.orchestra.run.smithi016.stdout:24_head 2023-05-20T01:28:58.440 INFO:teuthology.orchestra.run.smithi016.stdout:25_head 2023-05-20T01:28:58.441 INFO:teuthology.orchestra.run.smithi016.stdout:26_head 2023-05-20T01:28:58.441 INFO:teuthology.orchestra.run.smithi016.stdout:27_head 2023-05-20T01:28:58.441 INFO:teuthology.orchestra.run.smithi016.stdout:28_head 2023-05-20T01:28:58.441 INFO:teuthology.orchestra.run.smithi016.stdout:29_head 2023-05-20T01:28:58.441 INFO:teuthology.orchestra.run.smithi016.stdout:2_head 2023-05-20T01:28:58.441 INFO:teuthology.orchestra.run.smithi016.stdout:30_head 2023-05-20T01:28:58.441 INFO:teuthology.orchestra.run.smithi016.stdout:31_head 2023-05-20T01:28:58.442 INFO:teuthology.orchestra.run.smithi016.stdout:32_head 2023-05-20T01:28:58.442 INFO:teuthology.orchestra.run.smithi016.stdout:33_head 2023-05-20T01:28:58.442 INFO:teuthology.orchestra.run.smithi016.stdout:34_head 2023-05-20T01:28:58.442 INFO:teuthology.orchestra.run.smithi016.stdout:35_head 2023-05-20T01:28:58.442 INFO:teuthology.orchestra.run.smithi016.stdout:36_head 2023-05-20T01:28:58.442 INFO:teuthology.orchestra.run.smithi016.stdout:37_head 2023-05-20T01:28:58.442 INFO:teuthology.orchestra.run.smithi016.stdout:38_head 2023-05-20T01:28:58.442 INFO:teuthology.orchestra.run.smithi016.stdout:39_head 2023-05-20T01:28:58.443 INFO:teuthology.orchestra.run.smithi016.stdout:3_head 2023-05-20T01:28:58.443 INFO:teuthology.orchestra.run.smithi016.stdout:40_head 2023-05-20T01:28:58.443 INFO:teuthology.orchestra.run.smithi016.stdout:41_head 2023-05-20T01:28:58.443 INFO:teuthology.orchestra.run.smithi016.stdout:42_head 2023-05-20T01:28:58.443 INFO:teuthology.orchestra.run.smithi016.stdout:43_head 2023-05-20T01:28:58.443 INFO:teuthology.orchestra.run.smithi016.stdout:44_head 2023-05-20T01:28:58.443 INFO:teuthology.orchestra.run.smithi016.stdout:45_head 2023-05-20T01:28:58.444 INFO:teuthology.orchestra.run.smithi016.stdout:46_head 2023-05-20T01:28:58.444 INFO:teuthology.orchestra.run.smithi016.stdout:47_head 2023-05-20T01:28:58.444 INFO:teuthology.orchestra.run.smithi016.stdout:48_head 2023-05-20T01:28:58.445 INFO:teuthology.orchestra.run.smithi016.stdout:49_head 2023-05-20T01:28:58.445 INFO:teuthology.orchestra.run.smithi016.stdout:4_head 2023-05-20T01:28:58.445 INFO:teuthology.orchestra.run.smithi016.stdout:50_head 2023-05-20T01:28:58.445 INFO:teuthology.orchestra.run.smithi016.stdout:51_head 2023-05-20T01:28:58.445 INFO:teuthology.orchestra.run.smithi016.stdout:52_head 2023-05-20T01:28:58.445 INFO:teuthology.orchestra.run.smithi016.stdout:53_head 2023-05-20T01:28:58.445 INFO:teuthology.orchestra.run.smithi016.stdout:54_head 2023-05-20T01:28:58.445 INFO:teuthology.orchestra.run.smithi016.stdout:55_head 2023-05-20T01:28:58.446 INFO:teuthology.orchestra.run.smithi016.stdout:56_head 2023-05-20T01:28:58.446 INFO:teuthology.orchestra.run.smithi016.stdout:57_head 2023-05-20T01:28:58.446 INFO:teuthology.orchestra.run.smithi016.stdout:58_head 2023-05-20T01:28:58.446 INFO:teuthology.orchestra.run.smithi016.stdout:59_head 2023-05-20T01:28:58.446 INFO:teuthology.orchestra.run.smithi016.stdout:5_head 2023-05-20T01:28:58.446 INFO:teuthology.orchestra.run.smithi016.stdout:60_head 2023-05-20T01:28:58.447 INFO:teuthology.orchestra.run.smithi016.stdout:61_head 2023-05-20T01:28:58.447 INFO:teuthology.orchestra.run.smithi016.stdout:62_head 2023-05-20T01:28:58.447 INFO:teuthology.orchestra.run.smithi016.stdout:63_head 2023-05-20T01:28:58.447 INFO:teuthology.orchestra.run.smithi016.stdout:64_head 2023-05-20T01:28:58.447 INFO:teuthology.orchestra.run.smithi016.stdout:65_head 2023-05-20T01:28:58.447 INFO:teuthology.orchestra.run.smithi016.stdout:66_head 2023-05-20T01:28:58.448 INFO:teuthology.orchestra.run.smithi016.stdout:67_head 2023-05-20T01:28:58.448 INFO:teuthology.orchestra.run.smithi016.stdout:68_head 2023-05-20T01:28:58.448 INFO:teuthology.orchestra.run.smithi016.stdout:69_head 2023-05-20T01:28:58.448 INFO:teuthology.orchestra.run.smithi016.stdout:6_head 2023-05-20T01:28:58.448 INFO:teuthology.orchestra.run.smithi016.stdout:70_head 2023-05-20T01:28:58.448 INFO:teuthology.orchestra.run.smithi016.stdout:71_head 2023-05-20T01:28:58.449 INFO:teuthology.orchestra.run.smithi016.stdout:72_head 2023-05-20T01:28:58.449 INFO:teuthology.orchestra.run.smithi016.stdout:73_head 2023-05-20T01:28:58.449 INFO:teuthology.orchestra.run.smithi016.stdout:74_head 2023-05-20T01:28:58.449 INFO:teuthology.orchestra.run.smithi016.stdout:75_head 2023-05-20T01:28:58.449 INFO:teuthology.orchestra.run.smithi016.stdout:76_head 2023-05-20T01:28:58.449 INFO:teuthology.orchestra.run.smithi016.stdout:77_head 2023-05-20T01:28:58.449 INFO:teuthology.orchestra.run.smithi016.stdout:78_head 2023-05-20T01:28:58.450 INFO:teuthology.orchestra.run.smithi016.stdout:79_head 2023-05-20T01:28:58.450 INFO:teuthology.orchestra.run.smithi016.stdout:7_head 2023-05-20T01:28:58.450 INFO:teuthology.orchestra.run.smithi016.stdout:80_head 2023-05-20T01:28:58.450 INFO:teuthology.orchestra.run.smithi016.stdout:81_head 2023-05-20T01:28:58.450 INFO:teuthology.orchestra.run.smithi016.stdout:82_head 2023-05-20T01:28:58.450 INFO:teuthology.orchestra.run.smithi016.stdout:83_head 2023-05-20T01:28:58.451 INFO:teuthology.orchestra.run.smithi016.stdout:84_head 2023-05-20T01:28:58.451 INFO:teuthology.orchestra.run.smithi016.stdout:85_head 2023-05-20T01:28:58.451 INFO:teuthology.orchestra.run.smithi016.stdout:86_head 2023-05-20T01:28:58.451 INFO:teuthology.orchestra.run.smithi016.stdout:87_head 2023-05-20T01:28:58.452 INFO:teuthology.orchestra.run.smithi016.stdout:88_head 2023-05-20T01:28:58.452 INFO:teuthology.orchestra.run.smithi016.stdout:89_head 2023-05-20T01:28:58.452 INFO:teuthology.orchestra.run.smithi016.stdout:8_head 2023-05-20T01:28:58.452 INFO:teuthology.orchestra.run.smithi016.stdout:90_head 2023-05-20T01:28:58.452 INFO:teuthology.orchestra.run.smithi016.stdout:91_head 2023-05-20T01:28:58.452 INFO:teuthology.orchestra.run.smithi016.stdout:92_head 2023-05-20T01:28:58.452 INFO:teuthology.orchestra.run.smithi016.stdout:93_head 2023-05-20T01:28:58.453 INFO:teuthology.orchestra.run.smithi016.stdout:94_head 2023-05-20T01:28:58.453 INFO:teuthology.orchestra.run.smithi016.stdout:95_head 2023-05-20T01:28:58.453 INFO:teuthology.orchestra.run.smithi016.stdout:96_head 2023-05-20T01:28:58.453 INFO:teuthology.orchestra.run.smithi016.stdout:97_head 2023-05-20T01:28:58.453 INFO:teuthology.orchestra.run.smithi016.stdout:98_head 2023-05-20T01:28:58.453 INFO:teuthology.orchestra.run.smithi016.stdout:99_head 2023-05-20T01:28:58.453 INFO:teuthology.orchestra.run.smithi016.stdout:9_head 2023-05-20T01:28:58.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.424+0000 7fb31bf0f080 1 -- 172.21.15.16:0/2193639465 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x55564ad1b100 msgr2=0x55564adfb050 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:58.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.424+0000 7fb31bf0f080 1 --2- 172.21.15.16:0/2193639465 >> [v2:172.21.15.132:6801/727581123,v1:172.21.15.132:6802/727581123] conn(0x55564ad1b100 0x55564adfb050 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:58.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.424+0000 7fb31bf0f080 1 -- 172.21.15.16:0/2193639465 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb2f4069600 msgr2=0x7fb2f406bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:58.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.424+0000 7fb31bf0f080 1 --2- 172.21.15.16:0/2193639465 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fb2f4069600 0x7fb2f406bab0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fb308015040 tx=0x7fb30800d000 comp rx=0 tx=0).stop 2023-05-20T01:28:58.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.424+0000 7fb31bf0f080 1 -- 172.21.15.16:0/2193639465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55564ad5ad10 msgr2=0x55564adff8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:58.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.424+0000 7fb31bf0f080 1 --2- 172.21.15.16:0/2193639465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55564ad5ad10 0x55564adff8e0 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7fb304011000 tx=0x7fb304003020 comp rx=0 tx=0).stop 2023-05-20T01:28:58.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.424+0000 7fb312810700 1 -- 172.21.15.16:0/2193639465 reap_dead start 2023-05-20T01:28:58.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.424+0000 7fb31bf0f080 1 -- 172.21.15.16:0/2193639465 shutdown_connections 2023-05-20T01:28:58.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.424+0000 7fb31bf0f080 1 -- 172.21.15.16:0/2193639465 >> 172.21.15.16:0/2193639465 conn(0x55564ad77d00 msgr2=0x55564adfd0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:58.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.424+0000 7fb31bf0f080 1 -- 172.21.15.16:0/2193639465 shutdown_connections 2023-05-20T01:28:58.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.424+0000 7fb31bf0f080 1 -- 172.21.15.16:0/2193639465 wait complete. 2023-05-20T01:28:58.457 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.6:0 scrub start /subdir repair,recursive 2023-05-20T01:28:58.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.620+0000 7ff056fce700 1 Processor -- start 2023-05-20T01:28:58.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.620+0000 7ff056fce700 1 -- start start 2023-05-20T01:28:58.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.621+0000 7ff056fce700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff050159b00 0x7ff050072010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:58.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.621+0000 7ff056fce700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff0500b2970 0x7ff05006e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:58.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.621+0000 7ff056fce700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff05015e4b0 con 0x7ff05006eda0 2023-05-20T01:28:58.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.621+0000 7ff056fce700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff050072980 con 0x7ff050159b00 2023-05-20T01:28:58.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.621+0000 7ff056fce700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff050159ed0 con 0x7ff0500b2970 2023-05-20T01:28:58.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.621+0000 7ff054d6a700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff050159b00 0x7ff050072010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:58.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.621+0000 7ff05556b700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ff05006eda0 0x7ff0500ce120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:52548/0 (socket says 172.21.15.16:52548) 2023-05-20T01:28:58.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.621+0000 7ff05556b700 1 -- 172.21.15.16:0/1559374390 learned_addr learned my addr 172.21.15.16:0/1559374390 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:28:58.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.622+0000 7ff04ffff700 1 --2- 172.21.15.16:0/1559374390 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff0500b2970 0x7ff05006e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:58.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.622+0000 7ff054d6a700 1 -- 172.21.15.16:0/1559374390 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff0500b2970 msgr2=0x7ff05006e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:58.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.622+0000 7ff054d6a700 1 --2- 172.21.15.16:0/1559374390 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff0500b2970 0x7ff05006e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:58.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.622+0000 7ff054d6a700 1 -- 172.21.15.16:0/1559374390 >> v1:172.21.15.16:6789/0 conn(0x7ff05006eda0 legacy=0x7ff0500ce120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:58.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.622+0000 7ff054d6a700 1 -- 172.21.15.16:0/1559374390 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff05006f1f0 con 0x7ff050159b00 2023-05-20T01:28:58.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.622+0000 7ff04ffff700 1 --2- 172.21.15.16:0/1559374390 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff0500b2970 0x7ff05006e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:28:58.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.622+0000 7ff054d6a700 1 --2- 172.21.15.16:0/1559374390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff050159b00 0x7ff050072010 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7ff044009870 tx=0x7ff044009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=627ae3dcda21607f server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:58.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.622+0000 7ff04f7fe700 1 -- 172.21.15.16:0/1559374390 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff044011830 con 0x7ff050159b00 2023-05-20T01:28:58.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.622+0000 7ff04f7fe700 1 -- 172.21.15.16:0/1559374390 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff044011990 con 0x7ff050159b00 2023-05-20T01:28:58.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.622+0000 7ff04f7fe700 1 -- 172.21.15.16:0/1559374390 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff044011d20 con 0x7ff050159b00 2023-05-20T01:28:58.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.622+0000 7ff056fce700 1 -- 172.21.15.16:0/1559374390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff050159b00 msgr2=0x7ff050072010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:58.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.622+0000 7ff056fce700 1 --2- 172.21.15.16:0/1559374390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff050159b00 0x7ff050072010 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7ff044009870 tx=0x7ff044009b80 comp rx=0 tx=0).stop 2023-05-20T01:28:58.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.623+0000 7ff056fce700 1 -- 172.21.15.16:0/1559374390 shutdown_connections 2023-05-20T01:28:58.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.623+0000 7ff056fce700 1 --2- 172.21.15.16:0/1559374390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff050159b00 0x7ff050072010 unknown :-1 s=CLOSED pgs=1754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:58.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.623+0000 7ff056fce700 1 --2- 172.21.15.16:0/1559374390 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff0500b2970 0x7ff05006e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:58.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.623+0000 7ff056fce700 1 -- 172.21.15.16:0/1559374390 >> 172.21.15.16:0/1559374390 conn(0x7ff05006d600 msgr2=0x7ff05006d9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:58.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.623+0000 7ff056fce700 1 -- 172.21.15.16:0/1559374390 shutdown_connections 2023-05-20T01:28:58.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.623+0000 7ff056fce700 1 -- 172.21.15.16:0/1559374390 wait complete. 2023-05-20T01:28:58.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.624+0000 7ff056fce700 1 Processor -- start 2023-05-20T01:28:58.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.624+0000 7ff056fce700 1 -- start start 2023-05-20T01:28:58.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.625+0000 7ff056fce700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff05006eda0 0x7ff0500cb630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:58.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.625+0000 7ff056fce700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff0500b2970 0x7ff0501588f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:58.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.625+0000 7ff056fce700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff050159b00 0x7ff050158e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:58.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.625+0000 7ff056fce700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0501534d0 con 0x7ff050159b00 2023-05-20T01:28:58.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.625+0000 7ff056fce700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff050153770 con 0x7ff05006eda0 2023-05-20T01:28:58.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.625+0000 7ff056fce700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff050153a50 con 0x7ff0500b2970 2023-05-20T01:28:58.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.625+0000 7ff05556b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff050159b00 0x7ff050158e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:58.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.625+0000 7ff05556b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff050159b00 0x7ff050158e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41028/0 (socket says 172.21.15.16:41028) 2023-05-20T01:28:58.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.625+0000 7ff05556b700 1 -- 172.21.15.16:0/1862776185 learned_addr learned my addr 172.21.15.16:0/1862776185 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:58.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.625+0000 7ff04ffff700 1 --2- 172.21.15.16:0/1862776185 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff0500b2970 0x7ff0501588f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:58.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.625+0000 7ff054d6a700 1 --2- 172.21.15.16:0/1862776185 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff05006eda0 0x7ff0500cb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:58.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.625+0000 7ff05556b700 1 -- 172.21.15.16:0/1862776185 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff0500b2970 msgr2=0x7ff0501588f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:58.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.625+0000 7ff05556b700 1 --2- 172.21.15.16:0/1862776185 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff0500b2970 0x7ff0501588f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:58.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.625+0000 7ff05556b700 1 -- 172.21.15.16:0/1862776185 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff05006eda0 msgr2=0x7ff0500cb630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:58.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.625+0000 7ff05556b700 1 --2- 172.21.15.16:0/1862776185 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff05006eda0 0x7ff0500cb630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:58.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.625+0000 7ff05556b700 1 -- 172.21.15.16:0/1862776185 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0440094a0 con 0x7ff050159b00 2023-05-20T01:28:58.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.625+0000 7ff05556b700 1 --2- 172.21.15.16:0/1862776185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff050159b00 0x7ff050158e30 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7ff04800cbc0 tx=0x7ff04800ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:58.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.626+0000 7ff04d7fa700 1 -- 172.21.15.16:0/1862776185 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff04800a9e0 con 0x7ff050159b00 2023-05-20T01:28:58.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.626+0000 7ff04d7fa700 1 -- 172.21.15.16:0/1862776185 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff04800ab40 con 0x7ff050159b00 2023-05-20T01:28:58.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.626+0000 7ff056fce700 1 -- 172.21.15.16:0/1862776185 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff05015d9d0 con 0x7ff050159b00 2023-05-20T01:28:58.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.626+0000 7ff04d7fa700 1 -- 172.21.15.16:0/1862776185 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff048012400 con 0x7ff050159b00 2023-05-20T01:28:58.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.626+0000 7ff056fce700 1 -- 172.21.15.16:0/1862776185 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff05015d6f0 con 0x7ff050159b00 2023-05-20T01:28:58.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.627+0000 7ff04d7fa700 1 -- 172.21.15.16:0/1862776185 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff04801b3b0 con 0x7ff050159b00 2023-05-20T01:28:58.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.628+0000 7ff04d7fa700 1 --2- 172.21.15.16:0/1862776185 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff03406ace0 0x7ff03406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:58.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.628+0000 7ff04d7fa700 1 -- 172.21.15.16:0/1862776185 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6554+0+0 (secure 0 0 0) 0x7ff048089180 con 0x7ff050159b00 2023-05-20T01:28:58.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.628+0000 7ff054d6a700 1 --2- 172.21.15.16:0/1862776185 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff03406ace0 0x7ff03406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:58.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.628+0000 7ff054d6a700 1 --2- 172.21.15.16:0/1862776185 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff03406ace0 0x7ff03406d190 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7ff044000c00 tx=0x7ff04401c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:58.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.630+0000 7ff06315ab80 1 Processor -- start 2023-05-20T01:28:58.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.630+0000 7ff06315ab80 1 -- start start 2023-05-20T01:28:58.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.630+0000 7ff06315ab80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557d66b8f4e0 0x557d66b8f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:58.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.630+0000 7ff06315ab80 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x557d66b8b7e0 0x557d66b64540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:58.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.630+0000 7ff06315ab80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557d66b62950 0x557d66b64a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:58.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.630+0000 7ff06315ab80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d66a6a0f0 con 0x557d66b8f4e0 2023-05-20T01:28:58.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.630+0000 7ff06315ab80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d66abc930 con 0x557d66b62950 2023-05-20T01:28:58.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.630+0000 7ff06315ab80 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x557d66ab5dc0 con 0x557d66b8b7e0 2023-05-20T01:28:58.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.630+0000 7ff054d6a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557d66b8f4e0 0x557d66b8f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:58.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.630+0000 7ff054d6a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557d66b8f4e0 0x557d66b8f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41032/0 (socket says 172.21.15.16:41032) 2023-05-20T01:28:58.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.630+0000 7ff054d6a700 1 -- 172.21.15.16:0/1066265603 learned_addr learned my addr 172.21.15.16:0/1066265603 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:58.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.631+0000 7ff054d6a700 1 -- 172.21.15.16:0/1066265603 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x557d66b8b7e0 msgr2=0x557d66b64540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:58.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.631+0000 7ff054d6a700 1 --2- 172.21.15.16:0/1066265603 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x557d66b8b7e0 0x557d66b64540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:58.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.631+0000 7ff054d6a700 1 -- 172.21.15.16:0/1066265603 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557d66b62950 msgr2=0x557d66b64a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:58.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.631+0000 7ff054d6a700 1 --2- 172.21.15.16:0/1066265603 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557d66b62950 0x557d66b64a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:58.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.631+0000 7ff054d6a700 1 -- 172.21.15.16:0/1066265603 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557d669376c0 con 0x557d66b8f4e0 2023-05-20T01:28:58.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.631+0000 7ff054d6a700 1 --2- 172.21.15.16:0/1066265603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557d66b8f4e0 0x557d66b8f8b0 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7ff0500b2e10 tx=0x7ff044010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f923f1fd693f0e0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:58.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.631+0000 7ff03a7fc700 1 -- 172.21.15.16:0/1066265603 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff044003920 con 0x557d66b8f4e0 2023-05-20T01:28:58.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.631+0000 7ff03a7fc700 1 -- 172.21.15.16:0/1066265603 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff04402a040 con 0x557d66b8f4e0 2023-05-20T01:28:58.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.631+0000 7ff03a7fc700 1 -- 172.21.15.16:0/1066265603 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff044003ca0 con 0x557d66b8f4e0 2023-05-20T01:28:58.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.632+0000 7ff06315ab80 1 -- 172.21.15.16:0/1066265603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557d66b8f4e0 msgr2=0x557d66b8f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:58.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.632+0000 7ff06315ab80 1 --2- 172.21.15.16:0/1066265603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557d66b8f4e0 0x557d66b8f8b0 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7ff0500b2e10 tx=0x7ff044010000 comp rx=0 tx=0).stop 2023-05-20T01:28:58.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.632+0000 7ff06315ab80 1 -- 172.21.15.16:0/1066265603 shutdown_connections 2023-05-20T01:28:58.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.632+0000 7ff06315ab80 1 --2- 172.21.15.16:0/1066265603 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557d66b62950 0x557d66b64a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:58.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.632+0000 7ff06315ab80 1 --2- 172.21.15.16:0/1066265603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557d66b8f4e0 0x557d66b8f8b0 unknown :-1 s=CLOSED pgs=1756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:58.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.632+0000 7ff06315ab80 1 --2- 172.21.15.16:0/1066265603 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x557d66b8b7e0 0x557d66b64540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:58.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.632+0000 7ff06315ab80 1 -- 172.21.15.16:0/1066265603 >> 172.21.15.16:0/1066265603 conn(0x557d66b5a010 msgr2=0x557d66baa2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:58.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.632+0000 7ff06315ab80 1 -- 172.21.15.16:0/1066265603 shutdown_connections 2023-05-20T01:28:58.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.632+0000 7ff06315ab80 1 -- 172.21.15.16:0/1066265603 wait complete. 2023-05-20T01:28:58.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.632+0000 7ff06315ab80 1 -- start start 2023-05-20T01:28:58.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.633+0000 7ff06315ab80 1 Processor -- start 2023-05-20T01:28:58.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.634+0000 7ff06315ab80 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x557d66b62950 0x557d66ba9610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:58.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.634+0000 7ff06315ab80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557d66b8b7e0 0x557d66b81e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:58.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.634+0000 7ff06315ab80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557d66b8f4e0 0x557d66b5e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:58.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.634+0000 7ff06315ab80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d66a57970 con 0x557d66b8b7e0 2023-05-20T01:28:58.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.634+0000 7ff06315ab80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d66a57610 con 0x557d66b8f4e0 2023-05-20T01:28:58.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.634+0000 7ff06315ab80 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x557d669399b0 con 0x557d66b62950 2023-05-20T01:28:58.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.634+0000 7ff06315ab80 10 client.0 _ll_register_callbacks cb 0x557d66a75810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-05-20T01:28:58.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.634+0000 7ff04ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557d66b8b7e0 0x557d66b81e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:58.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.634+0000 7ff04ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557d66b8b7e0 0x557d66b81e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41040/0 (socket says 172.21.15.16:41040) 2023-05-20T01:28:58.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.634+0000 7ff04ffff700 1 -- 172.21.15.16:0/1454763846 learned_addr learned my addr 172.21.15.16:0/1454763846 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:58.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.634+0000 7ff05556b700 1 --2- 172.21.15.16:0/1454763846 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557d66b8f4e0 0x557d66b5e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:58.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.634+0000 7ff054d6a700 1 --2- 172.21.15.16:0/1454763846 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x557d66b62950 0x557d66ba9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:58.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.634+0000 7ff04ffff700 1 -- 172.21.15.16:0/1454763846 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x557d66b62950 msgr2=0x557d66ba9610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:58.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.634+0000 7ff04ffff700 1 --2- 172.21.15.16:0/1454763846 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x557d66b62950 0x557d66ba9610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:58.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.634+0000 7ff04ffff700 1 -- 172.21.15.16:0/1454763846 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557d66b8f4e0 msgr2=0x557d66b5e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:58.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.634+0000 7ff04ffff700 1 --2- 172.21.15.16:0/1454763846 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557d66b8f4e0 0x557d66b5e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:58.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.634+0000 7ff04ffff700 1 -- 172.21.15.16:0/1454763846 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff044011d40 con 0x557d66b8b7e0 2023-05-20T01:28:58.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.634+0000 7ff04ffff700 1 --2- 172.21.15.16:0/1454763846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557d66b8b7e0 0x557d66b81e90 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7ff03c006fd0 tx=0x7ff03c00f990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf80173506ea39a7 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:58.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.634+0000 7ff02f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-05-20T01:28:58.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.635+0000 7ff02f7fe700 1 -- 172.21.15.16:0/1454763846 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff03c013850 con 0x557d66b8b7e0 2023-05-20T01:28:58.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.635+0000 7ff02f7fe700 1 -- 172.21.15.16:0/1454763846 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(138..138 src has 1..138) v4 ==== 6554+0+0 (secure 0 0 0) 0x7ff03c01ce00 con 0x557d66b8b7e0 2023-05-20T01:28:58.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.635+0000 7ff06315ab80 1 -- 172.21.15.16:0/1454763846 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557d66b577d0 con 0x557d66b8b7e0 2023-05-20T01:28:58.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.635+0000 7ff02f7fe700 1 -- 172.21.15.16:0/1454763846 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=112) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff03c0139b0 con 0x557d66b8b7e0 2023-05-20T01:28:58.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.635+0000 7ff06315ab80 10 client.9186 fetch_fsmap learned FSMap version 112 2023-05-20T01:28:58.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.635+0000 7ff06315ab80 1 -- 172.21.15.16:0/1454763846 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=112,monmap=2+}) v3 -- 0x557d66a7c220 con 0x557d66b8b7e0 2023-05-20T01:28:58.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.635+0000 7ff02f7fe700 1 -- 172.21.15.16:0/1454763846 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557d66b577d0 con 0x557d66b8b7e0 2023-05-20T01:28:58.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.635+0000 7ff02f7fe700 1 -- 172.21.15.16:0/1454763846 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 112) v1 ==== 3114+0+0 (secure 0 0 0) 0x7ff03c019020 con 0x557d66b8b7e0 2023-05-20T01:28:58.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.635+0000 7ff06315ab80 10 client.9186 fetch_fsmap finished waiting for FSMap version 112 2023-05-20T01:28:58.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.635+0000 7ff06315ab80 10 client.9186 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2023-05-20T01:28:58.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.635+0000 7ff06315ab80 20 client.9186 populate_metadata read hostname 'smithi016' 2023-05-20T01:28:58.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.635+0000 7ff06315ab80 1 --2- 172.21.15.16:0/1454763846 >> [v2:172.21.15.16:6835/2952207608,v1:172.21.15.16:6837/2952207608] conn(0x557d66b82580 0x557d66b868b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:58.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.635+0000 7ff06315ab80 4 client.9186 mds_command: new command op to 9054 tid=0[{"prefix": "get_command_descriptions"}] 2023-05-20T01:28:58.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.635+0000 7ff06315ab80 1 -- 172.21.15.16:0/1454763846 --> [v2:172.21.15.16:6835/2952207608,v1:172.21.15.16:6837/2952207608] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557d66a78130 con 0x557d66b82580 2023-05-20T01:28:58.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.635+0000 7ff054d6a700 1 --2- 172.21.15.16:0/1454763846 >> [v2:172.21.15.16:6835/2952207608,v1:172.21.15.16:6837/2952207608] conn(0x557d66b82580 0x557d66b868b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:58.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.636+0000 7ff054d6a700 1 --2- 172.21.15.16:0/1454763846 >> [v2:172.21.15.16:6835/2952207608,v1:172.21.15.16:6837/2952207608] conn(0x557d66b82580 0x557d66b868b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9160dffd98a1d871 server_cookie=5598b73163e4a735 in_seq=0 out_seq=0 2023-05-20T01:28:58.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.636+0000 7ff02f7fe700 10 client.9186 ms_handle_connect on v2:172.21.15.16:6835/2952207608 2023-05-20T01:28:58.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.636+0000 7ff02f7fe700 1 -- 172.21.15.16:0/1454763846 <== mds.0 v2:172.21.15.16:6835/2952207608 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x557d66a78130 con 0x557d66b82580 2023-05-20T01:28:58.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.636+0000 7ff02f7fe700 10 client.9186 handle_command_reply: tid=0 2023-05-20T01:28:58.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.636+0000 7ff06315ab80 1 client.9186 shutdown 2023-05-20T01:28:58.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.636+0000 7ff06315ab80 1 -- 172.21.15.16:0/1454763846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557d66b8b7e0 msgr2=0x557d66b81e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:58.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.636+0000 7ff06315ab80 1 --2- 172.21.15.16:0/1454763846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557d66b8b7e0 0x557d66b81e90 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7ff03c006fd0 tx=0x7ff03c00f990 comp rx=0 tx=0).stop 2023-05-20T01:28:58.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.636+0000 7ff06315ab80 1 -- 172.21.15.16:0/1454763846 shutdown_connections 2023-05-20T01:28:58.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.636+0000 7ff06315ab80 1 --2- 172.21.15.16:0/1454763846 >> [v2:172.21.15.16:6835/2952207608,v1:172.21.15.16:6837/2952207608] conn(0x557d66b82580 0x557d66b868b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:58.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.636+0000 7ff06315ab80 1 --2- 172.21.15.16:0/1454763846 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557d66b8f4e0 0x557d66b5e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:58.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.636+0000 7ff02f7fe700 0 client.9186 ms_handle_reset on v2:172.21.15.16:6835/2952207608 2023-05-20T01:28:58.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.636+0000 7ff06315ab80 1 --2- 172.21.15.16:0/1454763846 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x557d66b62950 0x557d66ba9610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:58.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.636+0000 7ff06315ab80 1 --2- 172.21.15.16:0/1454763846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557d66b8b7e0 0x557d66b81e90 unknown :-1 s=CLOSED pgs=1757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:58.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.636+0000 7ff06315ab80 1 -- 172.21.15.16:0/1454763846 >> 172.21.15.16:0/1454763846 conn(0x557d66b5a010 msgr2=0x557d66b60780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:58.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.636+0000 7ff06315ab80 1 -- 172.21.15.16:0/1454763846 shutdown_connections 2023-05-20T01:28:58.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.636+0000 7ff06315ab80 1 -- 172.21.15.16:0/1454763846 wait complete. 2023-05-20T01:28:58.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.637+0000 7ff06315ab80 20 client.9186 trim_cache size 0 max 16384 2023-05-20T01:28:58.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.654+0000 7ff06315ab80 1 Processor -- start 2023-05-20T01:28:58.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.654+0000 7ff06315ab80 1 -- start start 2023-05-20T01:28:58.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.655+0000 7ff06315ab80 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x557d66b62950 0x557d66b841d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:58.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.655+0000 7ff06315ab80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557d66b8b7e0 0x557d66b81e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:58.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.655+0000 7ff06315ab80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557d66b8f4e0 0x557d66baa4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:58.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.655+0000 7ff06315ab80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d66c0c890 con 0x557d66b8f4e0 2023-05-20T01:28:58.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.655+0000 7ff06315ab80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d66b5e770 con 0x557d66b8b7e0 2023-05-20T01:28:58.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.655+0000 7ff06315ab80 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x557d66b87a80 con 0x557d66b62950 2023-05-20T01:28:58.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.655+0000 7ff05556b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557d66b8f4e0 0x557d66baa4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:58.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.655+0000 7ff05556b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557d66b8f4e0 0x557d66baa4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41050/0 (socket says 172.21.15.16:41050) 2023-05-20T01:28:58.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.655+0000 7ff05556b700 1 -- 172.21.15.16:0/1850660737 learned_addr learned my addr 172.21.15.16:0/1850660737 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:58.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.655+0000 7ff054d6a700 1 --2- 172.21.15.16:0/1850660737 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x557d66b62950 0x557d66b841d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:58.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.655+0000 7ff04ffff700 1 --2- 172.21.15.16:0/1850660737 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557d66b8b7e0 0x557d66b81e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:58.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.655+0000 7ff05556b700 1 -- 172.21.15.16:0/1850660737 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x557d66b62950 msgr2=0x557d66b841d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:58.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.655+0000 7ff05556b700 1 --2- 172.21.15.16:0/1850660737 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x557d66b62950 0x557d66b841d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:58.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.655+0000 7ff05556b700 1 -- 172.21.15.16:0/1850660737 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557d66b8b7e0 msgr2=0x557d66b81e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:58.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.655+0000 7ff05556b700 1 --2- 172.21.15.16:0/1850660737 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557d66b8b7e0 0x557d66b81e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:58.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.655+0000 7ff05556b700 1 -- 172.21.15.16:0/1850660737 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557d66b5b7b0 con 0x557d66b8f4e0 2023-05-20T01:28:58.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.655+0000 7ff05556b700 1 --2- 172.21.15.16:0/1850660737 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557d66b8f4e0 0x557d66baa4e0 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7ff048057ab0 tx=0x7ff048054220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fc384693b47abd3 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:58.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.655+0000 7ff03affd700 1 -- 172.21.15.16:0/1850660737 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff04805bcc0 con 0x557d66b8f4e0 2023-05-20T01:28:58.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.655+0000 7ff03affd700 1 -- 172.21.15.16:0/1850660737 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff048060050 con 0x557d66b8f4e0 2023-05-20T01:28:58.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.655+0000 7ff03affd700 1 -- 172.21.15.16:0/1850660737 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff04805fbc0 con 0x557d66b8f4e0 2023-05-20T01:28:58.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.656+0000 7ff06315ab80 1 -- 172.21.15.16:0/1850660737 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557d66b8f4e0 msgr2=0x557d66baa4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:58.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.656+0000 7ff06315ab80 1 --2- 172.21.15.16:0/1850660737 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557d66b8f4e0 0x557d66baa4e0 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7ff048057ab0 tx=0x7ff048054220 comp rx=0 tx=0).stop 2023-05-20T01:28:58.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.656+0000 7ff06315ab80 1 -- 172.21.15.16:0/1850660737 shutdown_connections 2023-05-20T01:28:58.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.656+0000 7ff06315ab80 1 --2- 172.21.15.16:0/1850660737 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557d66b8f4e0 0x557d66baa4e0 unknown :-1 s=CLOSED pgs=1758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:58.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.656+0000 7ff06315ab80 1 --2- 172.21.15.16:0/1850660737 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x557d66b62950 0x557d66b841d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:58.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.656+0000 7ff06315ab80 1 --2- 172.21.15.16:0/1850660737 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557d66b8b7e0 0x557d66b81e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:58.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.656+0000 7ff06315ab80 1 -- 172.21.15.16:0/1850660737 >> 172.21.15.16:0/1850660737 conn(0x557d66b5a010 msgr2=0x557d66ba78f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:58.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.656+0000 7ff06315ab80 1 -- 172.21.15.16:0/1850660737 shutdown_connections 2023-05-20T01:28:58.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.656+0000 7ff06315ab80 1 -- 172.21.15.16:0/1850660737 wait complete. 2023-05-20T01:28:58.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.657+0000 7ff06315ab80 1 -- start start 2023-05-20T01:28:58.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.657+0000 7ff06315ab80 1 Processor -- start 2023-05-20T01:28:58.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.658+0000 7ff06315ab80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557d66b62950 0x557d66b83fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:58.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.658+0000 7ff06315ab80 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x557d66b8b7e0 0x557d66b84500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:58.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.658+0000 7ff06315ab80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557d66b8f4e0 0x557d66b8bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:58.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.658+0000 7ff06315ab80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d66b88d90 con 0x557d66b62950 2023-05-20T01:28:58.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.658+0000 7ff06315ab80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d66b80d00 con 0x557d66b8f4e0 2023-05-20T01:28:58.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.658+0000 7ff06315ab80 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x557d66b63930 con 0x557d66b8b7e0 2023-05-20T01:28:58.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.658+0000 7ff06315ab80 10 client.0 _ll_register_callbacks cb 0x7ff03c00ac30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-05-20T01:28:58.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.658+0000 7ff054d6a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557d66b62950 0x557d66b83fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:58.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.658+0000 7ff054d6a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557d66b62950 0x557d66b83fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41052/0 (socket says 172.21.15.16:41052) 2023-05-20T01:28:58.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.658+0000 7ff054d6a700 1 -- 172.21.15.16:0/1977726606 learned_addr learned my addr 172.21.15.16:0/1977726606 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:58.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.658+0000 7ff04ffff700 1 --2- 172.21.15.16:0/1977726606 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x557d66b8b7e0 0x557d66b84500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:58.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.658+0000 7ff05556b700 1 --2- 172.21.15.16:0/1977726606 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557d66b8f4e0 0x557d66b8bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:58.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.658+0000 7ff054d6a700 1 -- 172.21.15.16:0/1977726606 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x557d66b8b7e0 msgr2=0x557d66b84500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:58.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.658+0000 7ff054d6a700 1 --2- 172.21.15.16:0/1977726606 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x557d66b8b7e0 0x557d66b84500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:58.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.658+0000 7ff054d6a700 1 -- 172.21.15.16:0/1977726606 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557d66b8f4e0 msgr2=0x557d66b8bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:58.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.658+0000 7ff054d6a700 1 --2- 172.21.15.16:0/1977726606 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557d66b8f4e0 0x557d66b8bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:58.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.658+0000 7ff054d6a700 1 -- 172.21.15.16:0/1977726606 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff05015d6f0 con 0x557d66b62950 2023-05-20T01:28:58.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.658+0000 7ff05556b700 1 --2- 172.21.15.16:0/1977726606 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557d66b8f4e0 0x557d66b8bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:28:58.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.658+0000 7ff054d6a700 1 --2- 172.21.15.16:0/1977726606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557d66b62950 0x557d66b83fc0 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7ff044011860 tx=0x7ff0440047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13e58180319c86b5 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:58.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.658+0000 7ff02f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-05-20T01:28:58.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.658+0000 7ff02f7fe700 1 -- 172.21.15.16:0/1977726606 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff04403d040 con 0x557d66b62950 2023-05-20T01:28:58.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.658+0000 7ff02f7fe700 1 -- 172.21.15.16:0/1977726606 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(138..138 src has 1..138) v4 ==== 6554+0+0 (secure 0 0 0) 0x7ff04401f070 con 0x557d66b62950 2023-05-20T01:28:58.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.658+0000 7ff06315ab80 1 -- 172.21.15.16:0/1977726606 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff04402a040 con 0x557d66b62950 2023-05-20T01:28:58.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.659+0000 7ff02f7fe700 1 -- 172.21.15.16:0/1977726606 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=112) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff044003ed0 con 0x557d66b62950 2023-05-20T01:28:58.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.659+0000 7ff06315ab80 10 client.9192 fetch_fsmap learned FSMap version 112 2023-05-20T01:28:58.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.659+0000 7ff06315ab80 1 -- 172.21.15.16:0/1977726606 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=112,monmap=2+}) v3 -- 0x557d668e3e70 con 0x557d66b62950 2023-05-20T01:28:58.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.659+0000 7ff02f7fe700 1 -- 172.21.15.16:0/1977726606 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff04402a040 con 0x557d66b62950 2023-05-20T01:28:58.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.659+0000 7ff02f7fe700 1 -- 172.21.15.16:0/1977726606 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 112) v1 ==== 3114+0+0 (secure 0 0 0) 0x7ff04402d070 con 0x557d66b62950 2023-05-20T01:28:58.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.659+0000 7ff06315ab80 10 client.9192 fetch_fsmap finished waiting for FSMap version 112 2023-05-20T01:28:58.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.659+0000 7ff06315ab80 10 client.9192 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2023-05-20T01:28:58.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.659+0000 7ff06315ab80 20 client.9192 populate_metadata read hostname 'smithi016' 2023-05-20T01:28:58.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.659+0000 7ff06315ab80 1 --2- 172.21.15.16:0/1977726606 >> [v2:172.21.15.16:6835/2952207608,v1:172.21.15.16:6837/2952207608] conn(0x557d66bf80a0 0x557d66b8c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:58.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.659+0000 7ff06315ab80 4 client.9192 mds_command: new command op to 9054 tid=0[{"prefix": "scrub start", "path": "/subdir", "scrubops": ["repair", "recursive"]}] 2023-05-20T01:28:58.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.659+0000 7ff06315ab80 1 -- 172.21.15.16:0/1977726606 --> [v2:172.21.15.16:6835/2952207608,v1:172.21.15.16:6837/2952207608] -- command(tid 0: {"prefix": "scrub start", "path": "/subdir", "scrubops": ["repair", "recursive"]}) v1 -- 0x557d66a846f0 con 0x557d66bf80a0 2023-05-20T01:28:58.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.659+0000 7ff04ffff700 1 --2- 172.21.15.16:0/1977726606 >> [v2:172.21.15.16:6835/2952207608,v1:172.21.15.16:6837/2952207608] conn(0x557d66bf80a0 0x557d66b8c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:58.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.659+0000 7ff02f7fe700 10 client.9192 ms_handle_connect on v2:172.21.15.16:6835/2952207608 2023-05-20T01:28:58.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.659+0000 7ff04ffff700 1 --2- 172.21.15.16:0/1977726606 >> [v2:172.21.15.16:6835/2952207608,v1:172.21.15.16:6837/2952207608] conn(0x557d66bf80a0 0x557d66b8c0f0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c2dbc168c78408f server_cookie=7007c1f75a817334 in_seq=0 out_seq=0 2023-05-20T01:28:58.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.661+0000 7ff02f7fe700 1 -- 172.21.15.16:0/1977726606 <== mds.0 v2:172.21.15.16:6835/2952207608 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x557d66a846f0 con 0x557d66bf80a0 2023-05-20T01:28:58.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.661+0000 7ff02f7fe700 10 client.9192 handle_command_reply: tid=0 2023-05-20T01:28:58.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.661+0000 7ff06315ab80 1 client.9192 shutdown 2023-05-20T01:28:58.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.661+0000 7ff06315ab80 1 -- 172.21.15.16:0/1977726606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557d66b62950 msgr2=0x557d66b83fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:58.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.661+0000 7ff06315ab80 1 --2- 172.21.15.16:0/1977726606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557d66b62950 0x557d66b83fc0 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7ff044011860 tx=0x7ff0440047c0 comp rx=0 tx=0).stop 2023-05-20T01:28:58.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.661+0000 7ff06315ab80 1 -- 172.21.15.16:0/1977726606 shutdown_connections 2023-05-20T01:28:58.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.661+0000 7ff06315ab80 1 --2- 172.21.15.16:0/1977726606 >> [v2:172.21.15.16:6835/2952207608,v1:172.21.15.16:6837/2952207608] conn(0x557d66bf80a0 0x557d66b8c0f0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:58.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.661+0000 7ff06315ab80 1 --2- 172.21.15.16:0/1977726606 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557d66b8f4e0 0x557d66b8bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:58.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.661+0000 7ff06315ab80 1 --2- 172.21.15.16:0/1977726606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557d66b62950 0x557d66b83fc0 unknown :-1 s=CLOSED pgs=1759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:58.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.661+0000 7ff02f7fe700 0 client.9192 ms_handle_reset on v2:172.21.15.16:6835/2952207608 2023-05-20T01:28:58.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.661+0000 7ff06315ab80 1 --2- 172.21.15.16:0/1977726606 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x557d66b8b7e0 0x557d66b84500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:58.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.661+0000 7ff06315ab80 1 -- 172.21.15.16:0/1977726606 >> 172.21.15.16:0/1977726606 conn(0x557d66b5a010 msgr2=0x557d66ba78f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:58.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.661+0000 7ff06315ab80 1 -- 172.21.15.16:0/1977726606 shutdown_connections 2023-05-20T01:28:58.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.662+0000 7ff06315ab80 1 -- 172.21.15.16:0/1977726606 wait complete. 2023-05-20T01:28:58.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.662+0000 7ff06315ab80 20 client.9192 trim_cache size 0 max 16384 2023-05-20T01:28:58.685 INFO:tasks.ceph.mds.a.smithi016.stderr:2023-05-20T01:28:58.662+0000 7f7d2e0fa700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000020a [2,head] /subdir/21 auth v1 ap=1 DIRTYPARENT s=2 n() (iversion lock) caps={9153=pAsLsXsFscr/-@5} | request=0 lock=0 caps=1 dirtyparent=1 authpin=1 scrubqueue=0 0x564f866e7180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(19)0x1000000020a:[<0x10000000000/21 v380>,<0x1/subdir v604>]//[]","memoryvalue":"(19)0x1000000020a:[<0x10000000000/21 v1>,<0x1/subdir v609>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-05-20T01:28:58.685 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-05-20T01:28:58.686 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2023-05-20T01:28:58.686 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "2244077a-3cc7-41af-b481-9f4e67cff762", 2023-05-20T01:28:58.686 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2023-05-20T01:28:58.687 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-05-20T01:28:58.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.662+0000 7ff056fce700 1 -- 172.21.15.16:0/1862776185 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff03406ace0 msgr2=0x7ff03406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:58.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.662+0000 7ff056fce700 1 --2- 172.21.15.16:0/1862776185 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff03406ace0 0x7ff03406d190 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7ff044000c00 tx=0x7ff04401c040 comp rx=0 tx=0).stop 2023-05-20T01:28:58.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.662+0000 7ff056fce700 1 -- 172.21.15.16:0/1862776185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff050159b00 msgr2=0x7ff050158e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:58.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.662+0000 7ff056fce700 1 --2- 172.21.15.16:0/1862776185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff050159b00 0x7ff050158e30 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7ff04800cbc0 tx=0x7ff04800ced0 comp rx=0 tx=0).stop 2023-05-20T01:28:58.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.662+0000 7ff056fce700 1 -- 172.21.15.16:0/1862776185 shutdown_connections 2023-05-20T01:28:58.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.662+0000 7ff056fce700 1 --2- 172.21.15.16:0/1862776185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff050159b00 0x7ff050158e30 unknown :-1 s=CLOSED pgs=1755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:58.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.662+0000 7ff056fce700 1 --2- 172.21.15.16:0/1862776185 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff05006eda0 0x7ff0500cb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:58.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.662+0000 7ff056fce700 1 --2- 172.21.15.16:0/1862776185 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff03406ace0 0x7ff03406d190 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:58.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.662+0000 7ff056fce700 1 --2- 172.21.15.16:0/1862776185 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff0500b2970 0x7ff0501588f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:58.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.662+0000 7ff056fce700 1 -- 172.21.15.16:0/1862776185 >> 172.21.15.16:0/1862776185 conn(0x7ff05006d600 msgr2=0x7ff0500cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:58.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.663+0000 7ff056fce700 1 -- 172.21.15.16:0/1862776185 shutdown_connections 2023-05-20T01:28:58.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.663+0000 7ff056fce700 1 -- 172.21.15.16:0/1862776185 wait complete. 2023-05-20T01:28:58.693 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.6:0 scrub status 2023-05-20T01:28:58.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.816+0000 7fda7d562700 1 Processor -- start 2023-05-20T01:28:58.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.817+0000 7fda7d562700 1 -- start start 2023-05-20T01:28:58.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.817+0000 7fda7d562700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fda780ce230 0x7fda780cf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:58.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.817+0000 7fda7d562700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda780b43b0 con 0x7fda780b4620 2023-05-20T01:28:58.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.817+0000 7fda7d562700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda780b40d0 con 0x7fda780b39e0 2023-05-20T01:28:58.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.817+0000 7fda7d562700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda7815d570 con 0x7fda780ce230 2023-05-20T01:28:58.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.817+0000 7fda777fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fda780b4620 0x7fda780d25a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:52588/0 (socket says 172.21.15.16:52588) 2023-05-20T01:28:58.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.817+0000 7fda777fe700 1 -- 172.21.15.16:0/1938639093 learned_addr learned my addr 172.21.15.16:0/1938639093 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:28:58.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.818+0000 7fda75ffb700 1 -- 172.21.15.16:0/1938639093 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4187797834 0 0) 0x7fda780b43b0 con 0x7fda780b4620 2023-05-20T01:28:58.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.818+0000 7fda75ffb700 1 -- 172.21.15.16:0/1938639093 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fda600035b0 con 0x7fda780b4620 2023-05-20T01:28:58.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.818+0000 7fda767fc700 1 --2- 172.21.15.16:0/1938639093 >> v2:172.21.15.132:3300/0 conn(0x7fda780ce230 0x7fda780cf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:58.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.818+0000 7fda75ffb700 1 -- 172.21.15.16:0/1938639093 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1296739274 0 0) 0x7fda600035b0 con 0x7fda780b4620 2023-05-20T01:28:58.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.818+0000 7fda75ffb700 1 -- 172.21.15.16:0/1938639093 >> v2:172.21.15.132:3300/0 conn(0x7fda780ce230 msgr2=0x7fda780cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:58.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.818+0000 7fda75ffb700 1 --2- 172.21.15.16:0/1938639093 >> v2:172.21.15.132:3300/0 conn(0x7fda780ce230 0x7fda780cf9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:58.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.818+0000 7fda75ffb700 1 -- 172.21.15.16:0/1938639093 >> v1:172.21.15.132:6789/0 conn(0x7fda780b39e0 legacy=0x7fda780cdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:58.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.818+0000 7fda75ffb700 1 -- 172.21.15.16:0/1938639093 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda780d1ac0 con 0x7fda780b4620 2023-05-20T01:28:58.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.818+0000 7fda75ffb700 1 -- 172.21.15.16:0/1938639093 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3642658959 0 0) 0x7fda700019f0 con 0x7fda780b4620 2023-05-20T01:28:58.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.818+0000 7fda75ffb700 1 -- 172.21.15.16:0/1938639093 >> v1:172.21.15.16:6789/0 conn(0x7fda780b4620 legacy=0x7fda780d25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:58.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.818+0000 7fda75ffb700 1 --2- 172.21.15.16:0/1938639093 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fda60003a00 0x7fda60003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:58.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.818+0000 7fda75ffb700 1 --2- 172.21.15.16:0/1938639093 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fda60004850 0x7fda60004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:58.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.818+0000 7fda75ffb700 1 --2- 172.21.15.16:0/1938639093 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda60005200 0x7fda6000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:58.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.818+0000 7fda75ffb700 1 -- 172.21.15.16:0/1938639093 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda7815d570 con 0x7fda60005200 2023-05-20T01:28:58.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.818+0000 7fda75ffb700 1 -- 172.21.15.16:0/1938639093 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda6000c060 con 0x7fda60004850 2023-05-20T01:28:58.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.818+0000 7fda75ffb700 1 -- 172.21.15.16:0/1938639093 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda6000c220 con 0x7fda60003a00 2023-05-20T01:28:58.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.818+0000 7fda777fe700 1 --2- 172.21.15.16:0/1938639093 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda60005200 0x7fda6000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:58.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.818+0000 7fda76ffd700 1 --2- 172.21.15.16:0/1938639093 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fda60003a00 0x7fda60003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:58.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.818+0000 7fda767fc700 1 --2- 172.21.15.16:0/1938639093 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fda60004850 0x7fda60004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:58.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.819+0000 7fda777fe700 1 -- 172.21.15.16:0/1938639093 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fda60003a00 msgr2=0x7fda60003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:58.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.819+0000 7fda777fe700 1 --2- 172.21.15.16:0/1938639093 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fda60003a00 0x7fda60003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:58.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.819+0000 7fda777fe700 1 -- 172.21.15.16:0/1938639093 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fda60004850 msgr2=0x7fda60004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:58.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.819+0000 7fda777fe700 1 --2- 172.21.15.16:0/1938639093 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fda60004850 0x7fda60004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:58.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.819+0000 7fda777fe700 1 -- 172.21.15.16:0/1938639093 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda600035b0 con 0x7fda60005200 2023-05-20T01:28:58.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.819+0000 7fda777fe700 1 -- 172.21.15.16:0/1938639093 reap_dead start 2023-05-20T01:28:58.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.819+0000 7fda777fe700 1 --2- 172.21.15.16:0/1938639093 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda60005200 0x7fda6000ba10 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7fda7000a800 tx=0x7fda60005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d21176e028e464b7 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:58.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.819+0000 7fda75ffb700 1 -- 172.21.15.16:0/1938639093 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda7000a2f0 con 0x7fda60005200 2023-05-20T01:28:58.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.819+0000 7fda75ffb700 1 -- 172.21.15.16:0/1938639093 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda70012620 con 0x7fda60005200 2023-05-20T01:28:58.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.819+0000 7fda75ffb700 1 -- 172.21.15.16:0/1938639093 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda700129d0 con 0x7fda60005200 2023-05-20T01:28:58.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.819+0000 7fda7d562700 1 -- 172.21.15.16:0/1938639093 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda60005200 msgr2=0x7fda6000ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:58.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.819+0000 7fda7d562700 1 --2- 172.21.15.16:0/1938639093 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda60005200 0x7fda6000ba10 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7fda7000a800 tx=0x7fda60005880 comp rx=0 tx=0).stop 2023-05-20T01:28:58.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.819+0000 7fda7d562700 1 -- 172.21.15.16:0/1938639093 shutdown_connections 2023-05-20T01:28:58.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.819+0000 7fda7d562700 1 --2- 172.21.15.16:0/1938639093 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda60005200 0x7fda6000ba10 unknown :-1 s=CLOSED pgs=1761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:58.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.819+0000 7fda7d562700 1 -- 172.21.15.16:0/1938639093 >> 172.21.15.16:0/1938639093 conn(0x7fda7809c820 msgr2=0x7fda780d34d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:58.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.819+0000 7fda7d562700 1 -- 172.21.15.16:0/1938639093 shutdown_connections 2023-05-20T01:28:58.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.820+0000 7fda7d562700 1 -- 172.21.15.16:0/1938639093 wait complete. 2023-05-20T01:28:58.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.821+0000 7fda7d562700 1 Processor -- start 2023-05-20T01:28:58.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.821+0000 7fda7d562700 1 -- start start 2023-05-20T01:28:58.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.822+0000 7fda7d562700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fda60005200 0x7fda780cc800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:58.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.822+0000 7fda7d562700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda7815c380 0x7fda780c96f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:58.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.822+0000 7fda7d562700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fda7815b780 0x7fda7815bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:58.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.822+0000 7fda7d562700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda7815d910 con 0x7fda7815c380 2023-05-20T01:28:58.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.822+0000 7fda7d562700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda7815c140 con 0x7fda60005200 2023-05-20T01:28:58.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.822+0000 7fda7d562700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda780cd0c0 con 0x7fda7815b780 2023-05-20T01:28:58.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.822+0000 7fda767fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda7815c380 0x7fda780c96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:58.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.822+0000 7fda767fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda7815c380 0x7fda780c96f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41074/0 (socket says 172.21.15.16:41074) 2023-05-20T01:28:58.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.822+0000 7fda767fc700 1 -- 172.21.15.16:0/4193004888 learned_addr learned my addr 172.21.15.16:0/4193004888 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:58.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.822+0000 7fda76ffd700 1 --2- 172.21.15.16:0/4193004888 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fda60005200 0x7fda780cc800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:58.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.822+0000 7fda777fe700 1 --2- 172.21.15.16:0/4193004888 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fda7815b780 0x7fda7815bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:58.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.822+0000 7fda767fc700 1 -- 172.21.15.16:0/4193004888 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fda7815b780 msgr2=0x7fda7815bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:58.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.822+0000 7fda767fc700 1 --2- 172.21.15.16:0/4193004888 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fda7815b780 0x7fda7815bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:58.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.822+0000 7fda767fc700 1 -- 172.21.15.16:0/4193004888 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fda60005200 msgr2=0x7fda780cc800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:58.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.822+0000 7fda767fc700 1 --2- 172.21.15.16:0/4193004888 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fda60005200 0x7fda780cc800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:58.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.822+0000 7fda767fc700 1 -- 172.21.15.16:0/4193004888 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda780d2080 con 0x7fda7815c380 2023-05-20T01:28:58.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.822+0000 7fda76ffd700 1 --2- 172.21.15.16:0/4193004888 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fda60005200 0x7fda780cc800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:28:58.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.822+0000 7fda777fe700 1 --2- 172.21.15.16:0/4193004888 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fda7815b780 0x7fda7815bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:28:58.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.822+0000 7fda767fc700 1 --2- 172.21.15.16:0/4193004888 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda7815c380 0x7fda780c96f0 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7fda68011970 tx=0x7fda68011d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:58.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.822+0000 7fda5f7fe700 1 -- 172.21.15.16:0/4193004888 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda68004240 con 0x7fda7815c380 2023-05-20T01:28:58.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.822+0000 7fda5f7fe700 1 -- 172.21.15.16:0/4193004888 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda680043a0 con 0x7fda7815c380 2023-05-20T01:28:58.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.822+0000 7fda7d562700 1 -- 172.21.15.16:0/4193004888 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda780d1da0 con 0x7fda7815c380 2023-05-20T01:28:58.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.823+0000 7fda5f7fe700 1 -- 172.21.15.16:0/4193004888 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda6800f950 con 0x7fda7815c380 2023-05-20T01:28:58.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.823+0000 7fda7d562700 1 -- 172.21.15.16:0/4193004888 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda780b3db0 con 0x7fda7815c380 2023-05-20T01:28:58.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.824+0000 7fda5f7fe700 1 -- 172.21.15.16:0/4193004888 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fda6800fab0 con 0x7fda7815c380 2023-05-20T01:28:58.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.824+0000 7fda5f7fe700 1 --2- 172.21.15.16:0/4193004888 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fda6406ad10 0x7fda6406d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:58.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.824+0000 7fda5f7fe700 1 -- 172.21.15.16:0/4193004888 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6554+0+0 (secure 0 0 0) 0x7fda68089af0 con 0x7fda7815c380 2023-05-20T01:28:58.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.824+0000 7fda76ffd700 1 --2- 172.21.15.16:0/4193004888 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fda6406ad10 0x7fda6406d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:58.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.825+0000 7fda76ffd700 1 --2- 172.21.15.16:0/4193004888 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fda6406ad10 0x7fda6406d1c0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fda6c000c00 tx=0x7fda6c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:58.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.826+0000 7fda896eeb80 1 Processor -- start 2023-05-20T01:28:58.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.827+0000 7fda896eeb80 1 -- start start 2023-05-20T01:28:58.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.827+0000 7fda896eeb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b78ef8450 0x561b78ef8820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:58.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.827+0000 7fda896eeb80 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x561b78ef4750 0x561b78ecd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:58.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.827+0000 7fda896eeb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561b78ecb8c0 0x561b78ecd9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:58.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.827+0000 7fda896eeb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b78dd3060 con 0x561b78ef8450 2023-05-20T01:28:58.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.827+0000 7fda896eeb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b78e258a0 con 0x561b78ecb8c0 2023-05-20T01:28:58.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.827+0000 7fda896eeb80 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x561b78e1ed30 con 0x561b78ef4750 2023-05-20T01:28:58.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.828+0000 7fda76ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b78ef8450 0x561b78ef8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:58.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.828+0000 7fda76ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b78ef8450 0x561b78ef8820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41080/0 (socket says 172.21.15.16:41080) 2023-05-20T01:28:58.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.828+0000 7fda76ffd700 1 -- 172.21.15.16:0/1193840819 learned_addr learned my addr 172.21.15.16:0/1193840819 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:58.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.828+0000 7fda777fe700 1 --2- 172.21.15.16:0/1193840819 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561b78ecb8c0 0x561b78ecd9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:58.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.828+0000 7fda767fc700 1 --2- 172.21.15.16:0/1193840819 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x561b78ef4750 0x561b78ecd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:58.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.828+0000 7fda76ffd700 1 -- 172.21.15.16:0/1193840819 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x561b78ef4750 msgr2=0x561b78ecd4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:58.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.828+0000 7fda76ffd700 1 --2- 172.21.15.16:0/1193840819 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x561b78ef4750 0x561b78ecd4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:58.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.828+0000 7fda76ffd700 1 -- 172.21.15.16:0/1193840819 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561b78ecb8c0 msgr2=0x561b78ecd9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:58.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.828+0000 7fda76ffd700 1 --2- 172.21.15.16:0/1193840819 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561b78ecb8c0 0x561b78ecd9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:58.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.828+0000 7fda76ffd700 1 -- 172.21.15.16:0/1193840819 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561b78ca0630 con 0x561b78ef8450 2023-05-20T01:28:58.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.828+0000 7fda767fc700 1 --2- 172.21.15.16:0/1193840819 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x561b78ef4750 0x561b78ecd4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:28:58.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.828+0000 7fda76ffd700 1 --2- 172.21.15.16:0/1193840819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b78ef8450 0x561b78ef8820 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7fda6c015820 tx=0x7fda6c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6eb701a7a6f53b1 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:58.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.828+0000 7fda5cff9700 1 -- 172.21.15.16:0/1193840819 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda6c014660 con 0x561b78ef8450 2023-05-20T01:28:58.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.828+0000 7fda5cff9700 1 -- 172.21.15.16:0/1193840819 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda6c0147c0 con 0x561b78ef8450 2023-05-20T01:28:58.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.828+0000 7fda5cff9700 1 -- 172.21.15.16:0/1193840819 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda6c014b30 con 0x561b78ef8450 2023-05-20T01:28:58.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.829+0000 7fda896eeb80 1 -- 172.21.15.16:0/1193840819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b78ef8450 msgr2=0x561b78ef8820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:58.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.829+0000 7fda896eeb80 1 --2- 172.21.15.16:0/1193840819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b78ef8450 0x561b78ef8820 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7fda6c015820 tx=0x7fda6c014000 comp rx=0 tx=0).stop 2023-05-20T01:28:58.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.829+0000 7fda896eeb80 1 -- 172.21.15.16:0/1193840819 shutdown_connections 2023-05-20T01:28:58.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.829+0000 7fda896eeb80 1 --2- 172.21.15.16:0/1193840819 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561b78ecb8c0 0x561b78ecd9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:58.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.829+0000 7fda896eeb80 1 --2- 172.21.15.16:0/1193840819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b78ef8450 0x561b78ef8820 unknown :-1 s=CLOSED pgs=1763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:58.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.829+0000 7fda896eeb80 1 --2- 172.21.15.16:0/1193840819 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x561b78ef4750 0x561b78ecd4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:58.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.829+0000 7fda896eeb80 1 -- 172.21.15.16:0/1193840819 >> 172.21.15.16:0/1193840819 conn(0x561b78ec2f80 msgr2=0x561b78f13260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:58.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.829+0000 7fda896eeb80 1 -- 172.21.15.16:0/1193840819 shutdown_connections 2023-05-20T01:28:58.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.829+0000 7fda896eeb80 1 -- 172.21.15.16:0/1193840819 wait complete. 2023-05-20T01:28:58.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.829+0000 7fda896eeb80 1 -- start start 2023-05-20T01:28:58.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.830+0000 7fda896eeb80 1 Processor -- start 2023-05-20T01:28:58.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.830+0000 7fda896eeb80 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x561b78ecb8c0 0x561b78ecd030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:58.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.830+0000 7fda896eeb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561b78ef4750 0x561b78ecd570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:58.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.830+0000 7fda896eeb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b78ef8450 0x561b78eec7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:58.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.830+0000 7fda896eeb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b78dc08e0 con 0x561b78ef8450 2023-05-20T01:28:58.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.830+0000 7fda896eeb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b78dc0580 con 0x561b78ef4750 2023-05-20T01:28:58.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.830+0000 7fda896eeb80 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x561b78ca2920 con 0x561b78ecb8c0 2023-05-20T01:28:58.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.830+0000 7fda896eeb80 10 client.0 _ll_register_callbacks cb 0x561b78dde780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-05-20T01:28:58.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.830+0000 7fda777fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b78ef8450 0x561b78eec7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:58.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.831+0000 7fda777fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b78ef8450 0x561b78eec7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41090/0 (socket says 172.21.15.16:41090) 2023-05-20T01:28:58.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.831+0000 7fda777fe700 1 -- 172.21.15.16:0/685349836 learned_addr learned my addr 172.21.15.16:0/685349836 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:58.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.831+0000 7fda76ffd700 1 --2- 172.21.15.16:0/685349836 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x561b78ecb8c0 0x561b78ecd030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:58.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.831+0000 7fda767fc700 1 --2- 172.21.15.16:0/685349836 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561b78ef4750 0x561b78ecd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:58.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.831+0000 7fda777fe700 1 -- 172.21.15.16:0/685349836 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x561b78ecb8c0 msgr2=0x561b78ecd030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:58.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.831+0000 7fda777fe700 1 --2- 172.21.15.16:0/685349836 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x561b78ecb8c0 0x561b78ecd030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:58.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.831+0000 7fda777fe700 1 -- 172.21.15.16:0/685349836 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561b78ef4750 msgr2=0x561b78ecd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:58.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.831+0000 7fda777fe700 1 --2- 172.21.15.16:0/685349836 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561b78ef4750 0x561b78ecd570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:58.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.831+0000 7fda777fe700 1 -- 172.21.15.16:0/685349836 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda780b40d0 con 0x561b78ef8450 2023-05-20T01:28:58.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.831+0000 7fda777fe700 1 --2- 172.21.15.16:0/685349836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b78ef8450 0x561b78eec7f0 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7fda70000f80 tx=0x7fda70009050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c040f6c3d110b4b4 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:58.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.831+0000 7fda55ffb700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-05-20T01:28:58.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.831+0000 7fda55ffb700 1 -- 172.21.15.16:0/685349836 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda70012990 con 0x561b78ef8450 2023-05-20T01:28:58.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.831+0000 7fda55ffb700 1 -- 172.21.15.16:0/685349836 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(138..138 src has 1..138) v4 ==== 6554+0+0 (secure 0 0 0) 0x7fda70012af0 con 0x561b78ef8450 2023-05-20T01:28:58.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.831+0000 7fda896eeb80 1 -- 172.21.15.16:0/685349836 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561b78ec0740 con 0x561b78ef8450 2023-05-20T01:28:58.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.831+0000 7fda55ffb700 1 -- 172.21.15.16:0/685349836 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=112) v2 ==== 24+0+0 (secure 0 0 0) 0x7fda70012cf0 con 0x561b78ef8450 2023-05-20T01:28:58.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.831+0000 7fda896eeb80 10 client.9204 fetch_fsmap learned FSMap version 112 2023-05-20T01:28:58.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.831+0000 7fda896eeb80 1 -- 172.21.15.16:0/685349836 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=112,monmap=2+}) v3 -- 0x561b78de5190 con 0x561b78ef8450 2023-05-20T01:28:58.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.831+0000 7fda55ffb700 1 -- 172.21.15.16:0/685349836 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561b78ec0740 con 0x561b78ef8450 2023-05-20T01:28:58.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.832+0000 7fda55ffb700 1 -- 172.21.15.16:0/685349836 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 112) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fda7000a830 con 0x561b78ef8450 2023-05-20T01:28:58.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.832+0000 7fda896eeb80 10 client.9204 fetch_fsmap finished waiting for FSMap version 112 2023-05-20T01:28:58.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.832+0000 7fda896eeb80 10 client.9204 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2023-05-20T01:28:58.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.832+0000 7fda896eeb80 20 client.9204 populate_metadata read hostname 'smithi016' 2023-05-20T01:28:58.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.832+0000 7fda896eeb80 1 --2- 172.21.15.16:0/685349836 >> [v2:172.21.15.16:6835/2952207608,v1:172.21.15.16:6837/2952207608] conn(0x561b78f16f90 0x561b78eed220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:58.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.832+0000 7fda896eeb80 4 client.9204 mds_command: new command op to 9054 tid=0[{"prefix": "get_command_descriptions"}] 2023-05-20T01:28:58.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.832+0000 7fda896eeb80 1 -- 172.21.15.16:0/685349836 --> [v2:172.21.15.16:6835/2952207608,v1:172.21.15.16:6837/2952207608] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561b78de10a0 con 0x561b78f16f90 2023-05-20T01:28:58.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.832+0000 7fda76ffd700 1 --2- 172.21.15.16:0/685349836 >> [v2:172.21.15.16:6835/2952207608,v1:172.21.15.16:6837/2952207608] conn(0x561b78f16f90 0x561b78eed220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:58.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.832+0000 7fda55ffb700 10 client.9204 ms_handle_connect on v2:172.21.15.16:6835/2952207608 2023-05-20T01:28:58.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.832+0000 7fda76ffd700 1 --2- 172.21.15.16:0/685349836 >> [v2:172.21.15.16:6835/2952207608,v1:172.21.15.16:6837/2952207608] conn(0x561b78f16f90 0x561b78eed220 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9b9b9fd79dfeb99 server_cookie=6b43f8c2c9f325fe in_seq=0 out_seq=0 2023-05-20T01:28:58.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.833+0000 7fda55ffb700 1 -- 172.21.15.16:0/685349836 <== mds.0 v2:172.21.15.16:6835/2952207608 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x561b78de10a0 con 0x561b78f16f90 2023-05-20T01:28:58.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.833+0000 7fda55ffb700 10 client.9204 handle_command_reply: tid=0 2023-05-20T01:28:58.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.833+0000 7fda896eeb80 1 client.9204 shutdown 2023-05-20T01:28:58.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.833+0000 7fda896eeb80 1 -- 172.21.15.16:0/685349836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b78ef8450 msgr2=0x561b78eec7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:58.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.833+0000 7fda896eeb80 1 --2- 172.21.15.16:0/685349836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b78ef8450 0x561b78eec7f0 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7fda70000f80 tx=0x7fda70009050 comp rx=0 tx=0).stop 2023-05-20T01:28:58.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.833+0000 7fda896eeb80 1 -- 172.21.15.16:0/685349836 shutdown_connections 2023-05-20T01:28:58.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.833+0000 7fda896eeb80 1 --2- 172.21.15.16:0/685349836 >> [v2:172.21.15.16:6835/2952207608,v1:172.21.15.16:6837/2952207608] conn(0x561b78f16f90 0x561b78eed220 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:58.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.833+0000 7fda896eeb80 1 --2- 172.21.15.16:0/685349836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b78ef8450 0x561b78eec7f0 unknown :-1 s=CLOSED pgs=1764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:58.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.833+0000 7fda896eeb80 1 --2- 172.21.15.16:0/685349836 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x561b78ecb8c0 0x561b78ecd030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:58.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.833+0000 7fda55ffb700 0 client.9204 ms_handle_reset on v2:172.21.15.16:6835/2952207608 2023-05-20T01:28:58.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.833+0000 7fda896eeb80 1 --2- 172.21.15.16:0/685349836 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561b78ef4750 0x561b78ecd570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:58.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.833+0000 7fda896eeb80 1 -- 172.21.15.16:0/685349836 >> 172.21.15.16:0/685349836 conn(0x561b78ec2f80 msgr2=0x561b78ec97f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:58.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.833+0000 7fda896eeb80 1 -- 172.21.15.16:0/685349836 shutdown_connections 2023-05-20T01:28:58.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.833+0000 7fda896eeb80 1 -- 172.21.15.16:0/685349836 wait complete. 2023-05-20T01:28:58.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.833+0000 7fda896eeb80 20 client.9204 trim_cache size 0 max 16384 2023-05-20T01:28:58.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.850+0000 7fda896eeb80 1 Processor -- start 2023-05-20T01:28:58.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.850+0000 7fda896eeb80 1 -- start start 2023-05-20T01:28:58.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.851+0000 7fda896eeb80 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x561b78ecb8c0 0x561b78f168d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:58.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.851+0000 7fda896eeb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561b78ef4750 0x561b78eed220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:58.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.851+0000 7fda896eeb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b78ef8450 0x561b78eed760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:58.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.851+0000 7fda896eeb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b78f16e10 con 0x561b78ef8450 2023-05-20T01:28:58.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.851+0000 7fda896eeb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b78ef5e20 con 0x561b78ef4750 2023-05-20T01:28:58.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.851+0000 7fda896eeb80 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x561b78ec0320 con 0x561b78ecb8c0 2023-05-20T01:28:58.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.851+0000 7fda777fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b78ef8450 0x561b78eed760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:58.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.851+0000 7fda777fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b78ef8450 0x561b78eed760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41098/0 (socket says 172.21.15.16:41098) 2023-05-20T01:28:58.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.851+0000 7fda777fe700 1 -- 172.21.15.16:0/3917491209 learned_addr learned my addr 172.21.15.16:0/3917491209 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:58.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.851+0000 7fda76ffd700 1 --2- 172.21.15.16:0/3917491209 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x561b78ecb8c0 0x561b78f168d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:58.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.851+0000 7fda767fc700 1 --2- 172.21.15.16:0/3917491209 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561b78ef4750 0x561b78eed220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:58.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.851+0000 7fda777fe700 1 -- 172.21.15.16:0/3917491209 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x561b78ecb8c0 msgr2=0x561b78f168d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:58.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.851+0000 7fda777fe700 1 --2- 172.21.15.16:0/3917491209 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x561b78ecb8c0 0x561b78f168d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:58.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.851+0000 7fda777fe700 1 -- 172.21.15.16:0/3917491209 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561b78ef4750 msgr2=0x561b78eed220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:58.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.851+0000 7fda777fe700 1 --2- 172.21.15.16:0/3917491209 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561b78ef4750 0x561b78eed220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:58.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.851+0000 7fda777fe700 1 -- 172.21.15.16:0/3917491209 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561b78de4ee0 con 0x561b78ef8450 2023-05-20T01:28:58.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.851+0000 7fda777fe700 1 --2- 172.21.15.16:0/3917491209 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b78ef8450 0x561b78eed760 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7fda70012f80 tx=0x7fda70021040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6800107718735a52 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:58.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.851+0000 7fda5d7fa700 1 -- 172.21.15.16:0/3917491209 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda7000a800 con 0x561b78ef8450 2023-05-20T01:28:58.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.851+0000 7fda5d7fa700 1 -- 172.21.15.16:0/3917491209 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda7000a960 con 0x561b78ef8450 2023-05-20T01:28:58.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.851+0000 7fda5d7fa700 1 -- 172.21.15.16:0/3917491209 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda70012790 con 0x561b78ef8450 2023-05-20T01:28:58.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.852+0000 7fda896eeb80 1 -- 172.21.15.16:0/3917491209 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b78ef8450 msgr2=0x561b78eed760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:58.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.852+0000 7fda896eeb80 1 --2- 172.21.15.16:0/3917491209 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b78ef8450 0x561b78eed760 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7fda70012f80 tx=0x7fda70021040 comp rx=0 tx=0).stop 2023-05-20T01:28:58.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.852+0000 7fda896eeb80 1 -- 172.21.15.16:0/3917491209 shutdown_connections 2023-05-20T01:28:58.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.852+0000 7fda896eeb80 1 --2- 172.21.15.16:0/3917491209 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b78ef8450 0x561b78eed760 unknown :-1 s=CLOSED pgs=1765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:58.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.852+0000 7fda896eeb80 1 --2- 172.21.15.16:0/3917491209 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x561b78ecb8c0 0x561b78f168d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:58.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.852+0000 7fda896eeb80 1 --2- 172.21.15.16:0/3917491209 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561b78ef4750 0x561b78eed220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:58.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.852+0000 7fda896eeb80 1 -- 172.21.15.16:0/3917491209 >> 172.21.15.16:0/3917491209 conn(0x561b78ec2f80 msgr2=0x561b78f13260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:58.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.852+0000 7fda896eeb80 1 -- 172.21.15.16:0/3917491209 shutdown_connections 2023-05-20T01:28:58.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.852+0000 7fda896eeb80 1 -- 172.21.15.16:0/3917491209 wait complete. 2023-05-20T01:28:58.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.852+0000 7fda896eeb80 1 -- start start 2023-05-20T01:28:58.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.853+0000 7fda896eeb80 1 Processor -- start 2023-05-20T01:28:58.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.853+0000 7fda896eeb80 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x561b78ecb8c0 0x561b78f7cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:58.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.853+0000 7fda896eeb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561b78ef4750 0x561b78ecd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:58.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.853+0000 7fda896eeb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b78ef8450 0x561b78ecdc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:58.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.853+0000 7fda896eeb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b78f7d660 con 0x561b78ef8450 2023-05-20T01:28:58.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.853+0000 7fda896eeb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b78f0a040 con 0x561b78ef4750 2023-05-20T01:28:58.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.853+0000 7fda896eeb80 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x561b78ecea80 con 0x561b78ecb8c0 2023-05-20T01:28:58.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.853+0000 7fda896eeb80 10 client.0 _ll_register_callbacks cb 0x7fda680090e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-05-20T01:28:58.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.853+0000 7fda777fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b78ef8450 0x561b78ecdc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:58.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.853+0000 7fda777fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b78ef8450 0x561b78ecdc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41104/0 (socket says 172.21.15.16:41104) 2023-05-20T01:28:58.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.853+0000 7fda777fe700 1 -- 172.21.15.16:0/1275993432 learned_addr learned my addr 172.21.15.16:0/1275993432 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:28:58.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.854+0000 7fda76ffd700 1 --2- 172.21.15.16:0/1275993432 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x561b78ecb8c0 0x561b78f7cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:58.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.854+0000 7fda767fc700 1 --2- 172.21.15.16:0/1275993432 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561b78ef4750 0x561b78ecd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:58.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.854+0000 7fda777fe700 1 -- 172.21.15.16:0/1275993432 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x561b78ecb8c0 msgr2=0x561b78f7cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:58.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.854+0000 7fda777fe700 1 --2- 172.21.15.16:0/1275993432 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x561b78ecb8c0 0x561b78f7cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:58.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.854+0000 7fda777fe700 1 -- 172.21.15.16:0/1275993432 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561b78ef4750 msgr2=0x561b78ecd6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:28:58.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.854+0000 7fda777fe700 1 --2- 172.21.15.16:0/1275993432 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561b78ef4750 0x561b78ecd6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:58.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.854+0000 7fda777fe700 1 -- 172.21.15.16:0/1275993432 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561b78de5190 con 0x561b78ef8450 2023-05-20T01:28:58.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.854+0000 7fda777fe700 1 --2- 172.21.15.16:0/1275993432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b78ef8450 0x561b78ecdc10 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7fda7000adf0 tx=0x7fda700021f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11b2eb49ac1ef38d server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:28:58.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.854+0000 7fda55ffb700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-05-20T01:28:58.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.854+0000 7fda55ffb700 1 -- 172.21.15.16:0/1275993432 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda7003d070 con 0x561b78ef8450 2023-05-20T01:28:58.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.854+0000 7fda55ffb700 1 -- 172.21.15.16:0/1275993432 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(138..138 src has 1..138) v4 ==== 6554+0+0 (secure 0 0 0) 0x7fda700129a0 con 0x561b78ef8450 2023-05-20T01:28:58.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.854+0000 7fda896eeb80 1 -- 172.21.15.16:0/1275993432 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fda6c0147c0 con 0x561b78ef8450 2023-05-20T01:28:58.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.854+0000 7fda55ffb700 1 -- 172.21.15.16:0/1275993432 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=112) v2 ==== 24+0+0 (secure 0 0 0) 0x7fda7000a800 con 0x561b78ef8450 2023-05-20T01:28:58.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.854+0000 7fda896eeb80 10 client.9210 fetch_fsmap learned FSMap version 112 2023-05-20T01:28:58.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.854+0000 7fda896eeb80 1 -- 172.21.15.16:0/1275993432 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=112,monmap=2+}) v3 -- 0x561b78c35c60 con 0x561b78ef8450 2023-05-20T01:28:58.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.854+0000 7fda55ffb700 1 -- 172.21.15.16:0/1275993432 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda6c0147c0 con 0x561b78ef8450 2023-05-20T01:28:58.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.854+0000 7fda55ffb700 1 -- 172.21.15.16:0/1275993432 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 112) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fda70040020 con 0x561b78ef8450 2023-05-20T01:28:58.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.854+0000 7fda896eeb80 10 client.9210 fetch_fsmap finished waiting for FSMap version 112 2023-05-20T01:28:58.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.854+0000 7fda896eeb80 10 client.9210 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2023-05-20T01:28:58.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.854+0000 7fda896eeb80 20 client.9210 populate_metadata read hostname 'smithi016' 2023-05-20T01:28:58.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.854+0000 7fda896eeb80 1 --2- 172.21.15.16:0/1275993432 >> [v2:172.21.15.16:6835/2952207608,v1:172.21.15.16:6837/2952207608] conn(0x561b78f7c8f0 0x561b78eed220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:28:58.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.855+0000 7fda896eeb80 4 client.9210 mds_command: new command op to 9054 tid=0[{"prefix": "scrub status"}] 2023-05-20T01:28:58.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.855+0000 7fda896eeb80 1 -- 172.21.15.16:0/1275993432 --> [v2:172.21.15.16:6835/2952207608,v1:172.21.15.16:6837/2952207608] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561b78ded660 con 0x561b78f7c8f0 2023-05-20T01:28:58.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.855+0000 7fda76ffd700 1 --2- 172.21.15.16:0/1275993432 >> [v2:172.21.15.16:6835/2952207608,v1:172.21.15.16:6837/2952207608] conn(0x561b78f7c8f0 0x561b78eed220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:28:58.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.855+0000 7fda55ffb700 10 client.9210 ms_handle_connect on v2:172.21.15.16:6835/2952207608 2023-05-20T01:28:58.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.855+0000 7fda76ffd700 1 --2- 172.21.15.16:0/1275993432 >> [v2:172.21.15.16:6835/2952207608,v1:172.21.15.16:6837/2952207608] conn(0x561b78f7c8f0 0x561b78eed220 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=133abc7d9d6f0a27 server_cookie=f43c0435cf84a645 in_seq=0 out_seq=0 2023-05-20T01:28:58.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.855+0000 7fda55ffb700 1 -- 172.21.15.16:0/1275993432 <== mds.0 v2:172.21.15.16:6835/2952207608 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+266 (crc 0 0 0) 0x561b78ded660 con 0x561b78f7c8f0 2023-05-20T01:28:58.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.855+0000 7fda55ffb700 10 client.9210 handle_command_reply: tid=0 2023-05-20T01:28:58.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.855+0000 7fda896eeb80 1 client.9210 shutdown 2023-05-20T01:28:58.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.855+0000 7fda896eeb80 1 -- 172.21.15.16:0/1275993432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b78ef8450 msgr2=0x561b78ecdc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:58.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.855+0000 7fda896eeb80 1 --2- 172.21.15.16:0/1275993432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b78ef8450 0x561b78ecdc10 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7fda7000adf0 tx=0x7fda700021f0 comp rx=0 tx=0).stop 2023-05-20T01:28:58.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.855+0000 7fda896eeb80 1 -- 172.21.15.16:0/1275993432 shutdown_connections 2023-05-20T01:28:58.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.855+0000 7fda896eeb80 1 --2- 172.21.15.16:0/1275993432 >> [v2:172.21.15.16:6835/2952207608,v1:172.21.15.16:6837/2952207608] conn(0x561b78f7c8f0 0x561b78eed220 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:58.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.855+0000 7fda896eeb80 1 --2- 172.21.15.16:0/1275993432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b78ef8450 0x561b78ecdc10 unknown :-1 s=CLOSED pgs=1766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:58.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.855+0000 7fda55ffb700 0 client.9210 ms_handle_reset on v2:172.21.15.16:6835/2952207608 2023-05-20T01:28:58.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.855+0000 7fda896eeb80 1 --2- 172.21.15.16:0/1275993432 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x561b78ecb8c0 0x561b78f7cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:58.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.855+0000 7fda896eeb80 1 --2- 172.21.15.16:0/1275993432 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561b78ef4750 0x561b78ecd6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:58.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.855+0000 7fda896eeb80 1 -- 172.21.15.16:0/1275993432 >> 172.21.15.16:0/1275993432 conn(0x561b78ec2f80 msgr2=0x561b78f155c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:58.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.855+0000 7fda896eeb80 1 -- 172.21.15.16:0/1275993432 shutdown_connections 2023-05-20T01:28:58.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.855+0000 7fda896eeb80 1 -- 172.21.15.16:0/1275993432 wait complete. 2023-05-20T01:28:58.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.855+0000 7fda896eeb80 20 client.9210 trim_cache size 0 max 16384 2023-05-20T01:28:58.870 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-05-20T01:28:58.870 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2023-05-20T01:28:58.870 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2023-05-20T01:28:58.870 INFO:teuthology.orchestra.run.smithi016.stdout: "2244077a-3cc7-41af-b481-9f4e67cff762": { 2023-05-20T01:28:58.870 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/subdir", 2023-05-20T01:28:58.871 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "2244077a-3cc7-41af-b481-9f4e67cff762", 2023-05-20T01:28:58.871 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,repair" 2023-05-20T01:28:58.871 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:28:58.871 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-05-20T01:28:58.871 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-05-20T01:28:58.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.856+0000 7fda7d562700 1 -- 172.21.15.16:0/4193004888 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fda6406ad10 msgr2=0x7fda6406d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:58.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.856+0000 7fda7d562700 1 --2- 172.21.15.16:0/4193004888 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fda6406ad10 0x7fda6406d1c0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fda6c000c00 tx=0x7fda6c009040 comp rx=0 tx=0).stop 2023-05-20T01:28:58.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.856+0000 7fda7d562700 1 -- 172.21.15.16:0/4193004888 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda7815c380 msgr2=0x7fda780c96f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:28:58.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.856+0000 7fda7d562700 1 --2- 172.21.15.16:0/4193004888 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda7815c380 0x7fda780c96f0 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7fda68011970 tx=0x7fda68011d30 comp rx=0 tx=0).stop 2023-05-20T01:28:58.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.856+0000 7fda7d562700 1 -- 172.21.15.16:0/4193004888 shutdown_connections 2023-05-20T01:28:58.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.856+0000 7fda7d562700 1 --2- 172.21.15.16:0/4193004888 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7fda6406ad10 0x7fda6406d1c0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:58.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.856+0000 7fda7d562700 1 --2- 172.21.15.16:0/4193004888 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fda7815b780 0x7fda7815bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:58.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.856+0000 7fda7d562700 1 --2- 172.21.15.16:0/4193004888 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fda60005200 0x7fda780cc800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:58.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.856+0000 7fda7d562700 1 --2- 172.21.15.16:0/4193004888 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda7815c380 0x7fda780c96f0 unknown :-1 s=CLOSED pgs=1762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:28:58.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.856+0000 7fda7d562700 1 -- 172.21.15.16:0/4193004888 >> 172.21.15.16:0/4193004888 conn(0x7fda7809c820 msgr2=0x7fda7806b780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:28:58.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.856+0000 7fda7d562700 1 -- 172.21.15.16:0/4193004888 shutdown_connections 2023-05-20T01:28:58.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:28:58.856+0000 7fda7d562700 1 -- 172.21.15.16:0/4193004888 wait complete. 2023-05-20T01:28:58.874 INFO:tasks.cephfs.filesystem:scrub status for tag:2244077a-3cc7-41af-b481-9f4e67cff762 - {'path': '/subdir', 'tag': '2244077a-3cc7-41af-b481-9f4e67cff762', 'options': 'recursive,repair'} 2023-05-20T01:29:07.327 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:29:07.331 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:29:28.875 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.6:0 scrub status 2023-05-20T01:29:29.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.011+0000 7ff282256700 1 Processor -- start 2023-05-20T01:29:29.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.011+0000 7ff282256700 1 -- start start 2023-05-20T01:29:29.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.012+0000 7ff282256700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff27c0b4f30 0x7ff27c0b5320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:29:29.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.012+0000 7ff282256700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff27c15aaf0 con 0x7ff27c15db10 2023-05-20T01:29:29.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.012+0000 7ff282256700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff27c0b5860 con 0x7ff27c15a720 2023-05-20T01:29:29.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.012+0000 7ff282256700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff27c15e010 con 0x7ff27c0b4f30 2023-05-20T01:29:29.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.012+0000 7ff27bfff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff27c0b4f30 0x7ff27c0b5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:29:29.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.012+0000 7ff27affd700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ff27c15db10 0x7ff27c0d1080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:52638/0 (socket says 172.21.15.16:52638) 2023-05-20T01:29:29.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.013+0000 7ff27affd700 1 -- 172.21.15.16:0/846419455 learned_addr learned my addr 172.21.15.16:0/846419455 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-05-20T01:29:29.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.013+0000 7ff27bfff700 1 -- 172.21.15.16:0/846419455 >> v1:172.21.15.132:6789/0 conn(0x7ff27c15a720 legacy=0x7ff27c0cf770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:29:29.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.013+0000 7ff27bfff700 1 -- 172.21.15.16:0/846419455 >> v1:172.21.15.16:6789/0 conn(0x7ff27c15db10 legacy=0x7ff27c0d1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:29:29.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.013+0000 7ff27a7fc700 1 -- 172.21.15.16:0/846419455 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3002790145 0 0) 0x7ff27c15aaf0 con 0x7ff27c15db10 2023-05-20T01:29:29.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.013+0000 7ff27bfff700 1 -- 172.21.15.16:0/846419455 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff27c0cdba0 con 0x7ff27c0b4f30 2023-05-20T01:29:29.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.013+0000 7ff27bfff700 1 --2- 172.21.15.16:0/846419455 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff27c0b4f30 0x7ff27c0b5320 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7ff274009d90 tx=0x7ff27400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b85895a69e4ba32 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:29:29.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.013+0000 7ff27a7fc700 1 -- 172.21.15.16:0/846419455 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff27400e070 con 0x7ff27c0b4f30 2023-05-20T01:29:29.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.013+0000 7ff27a7fc700 1 -- 172.21.15.16:0/846419455 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff27400bb90 con 0x7ff27c0b4f30 2023-05-20T01:29:29.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.013+0000 7ff27a7fc700 1 -- 172.21.15.16:0/846419455 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff274003a20 con 0x7ff27c0b4f30 2023-05-20T01:29:29.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.014+0000 7ff282256700 1 -- 172.21.15.16:0/846419455 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff27c0b4f30 msgr2=0x7ff27c0b5320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:29:29.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.014+0000 7ff282256700 1 --2- 172.21.15.16:0/846419455 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff27c0b4f30 0x7ff27c0b5320 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7ff274009d90 tx=0x7ff27400b330 comp rx=0 tx=0).stop 2023-05-20T01:29:29.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.014+0000 7ff282256700 1 -- 172.21.15.16:0/846419455 shutdown_connections 2023-05-20T01:29:29.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.014+0000 7ff282256700 1 --2- 172.21.15.16:0/846419455 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff27c0b4f30 0x7ff27c0b5320 unknown :-1 s=CLOSED pgs=1768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:29:29.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.014+0000 7ff282256700 1 -- 172.21.15.16:0/846419455 >> 172.21.15.16:0/846419455 conn(0x7ff27c0cff30 msgr2=0x7ff27c0d0330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:29:29.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.014+0000 7ff282256700 1 -- 172.21.15.16:0/846419455 shutdown_connections 2023-05-20T01:29:29.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.014+0000 7ff282256700 1 -- 172.21.15.16:0/846419455 wait complete. 2023-05-20T01:29:29.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.015+0000 7ff282256700 1 Processor -- start 2023-05-20T01:29:29.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.015+0000 7ff282256700 1 -- start start 2023-05-20T01:29:29.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.016+0000 7ff282256700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff27c0b4f30 0x7ff27c0b8650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:29:29.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.016+0000 7ff282256700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff27c15a720 0x7ff27c1f59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:29:29.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.016+0000 7ff282256700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff27c15db10 0x7ff27c1f5f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:29:29.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.016+0000 7ff282256700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff27c14a090 con 0x7ff27c15db10 2023-05-20T01:29:29.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.016+0000 7ff282256700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff27c0b40e0 con 0x7ff27c15a720 2023-05-20T01:29:29.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.016+0000 7ff282256700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff27c0b34d0 con 0x7ff27c0b4f30 2023-05-20T01:29:29.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.016+0000 7ff27bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff27c15db10 0x7ff27c1f5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:29:29.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.016+0000 7ff27bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff27c15db10 0x7ff27c1f5f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41116/0 (socket says 172.21.15.16:41116) 2023-05-20T01:29:29.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.016+0000 7ff27bfff700 1 -- 172.21.15.16:0/2591951631 learned_addr learned my addr 172.21.15.16:0/2591951631 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:29:29.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.016+0000 7ff27bfff700 1 -- 172.21.15.16:0/2591951631 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff27c0b4f30 msgr2=0x7ff27c0b8650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:29:29.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.016+0000 7ff27b7fe700 1 --2- 172.21.15.16:0/2591951631 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff27c0b4f30 0x7ff27c0b8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:29:29.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.017+0000 7ff27bfff700 1 --2- 172.21.15.16:0/2591951631 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff27c0b4f30 0x7ff27c0b8650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:29:29.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.017+0000 7ff27bfff700 1 -- 172.21.15.16:0/2591951631 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff27c15a720 msgr2=0x7ff27c1f59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:29:29.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.017+0000 7ff27affd700 1 --2- 172.21.15.16:0/2591951631 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff27c15a720 0x7ff27c1f59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:29:29.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.017+0000 7ff27bfff700 1 --2- 172.21.15.16:0/2591951631 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff27c15a720 0x7ff27c1f59c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:29:29.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.017+0000 7ff27bfff700 1 -- 172.21.15.16:0/2591951631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff274009a70 con 0x7ff27c15db10 2023-05-20T01:29:29.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.017+0000 7ff27affd700 1 --2- 172.21.15.16:0/2591951631 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff27c15a720 0x7ff27c1f59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:29:29.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.017+0000 7ff27bfff700 1 --2- 172.21.15.16:0/2591951631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff27c15db10 0x7ff27c1f5f00 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7ff27c15ac80 tx=0x7ff274004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:29:29.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.017+0000 7ff263fff700 1 -- 172.21.15.16:0/2591951631 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff274012630 con 0x7ff27c15db10 2023-05-20T01:29:29.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.017+0000 7ff263fff700 1 -- 172.21.15.16:0/2591951631 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2740127b0 con 0x7ff27c15db10 2023-05-20T01:29:29.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.017+0000 7ff282256700 1 -- 172.21.15.16:0/2591951631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff27c0ce160 con 0x7ff27c15db10 2023-05-20T01:29:29.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.017+0000 7ff263fff700 1 -- 172.21.15.16:0/2591951631 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff274012b60 con 0x7ff27c15db10 2023-05-20T01:29:29.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.017+0000 7ff282256700 1 -- 172.21.15.16:0/2591951631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff27c0cde80 con 0x7ff27c15db10 2023-05-20T01:29:29.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.018+0000 7ff263fff700 1 -- 172.21.15.16:0/2591951631 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff274012cc0 con 0x7ff27c15db10 2023-05-20T01:29:29.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.019+0000 7ff263fff700 1 --2- 172.21.15.16:0/2591951631 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff26806ace0 0x7ff26806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:29:29.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.019+0000 7ff263fff700 1 -- 172.21.15.16:0/2591951631 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 6612+0+0 (secure 0 0 0) 0x7ff27408b920 con 0x7ff27c15db10 2023-05-20T01:29:29.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.019+0000 7ff27b7fe700 1 --2- 172.21.15.16:0/2591951631 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff26806ace0 0x7ff26806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:29:29.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.019+0000 7ff27b7fe700 1 --2- 172.21.15.16:0/2591951631 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff26806ace0 0x7ff26806d190 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7ff26c003e60 tx=0x7ff26c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:29:29.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.021+0000 7ff28e3e2b80 1 Processor -- start 2023-05-20T01:29:29.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.021+0000 7ff28e3e2b80 1 -- start start 2023-05-20T01:29:29.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.022+0000 7ff28e3e2b80 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55b0ecf19450 0x55b0ecf19820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:29:29.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.022+0000 7ff28e3e2b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b0ecf15750 0x55b0eceee4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:29:29.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.022+0000 7ff28e3e2b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b0eceec8c0 0x55b0eceee9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:29:29.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.022+0000 7ff28e3e2b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0ecdf4060 con 0x55b0eceec8c0 2023-05-20T01:29:29.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.022+0000 7ff28e3e2b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0ece468a0 con 0x55b0ecf15750 2023-05-20T01:29:29.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.022+0000 7ff28e3e2b80 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b0ece3fd30 con 0x55b0ecf19450 2023-05-20T01:29:29.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.022+0000 7ff27bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b0eceec8c0 0x55b0eceee9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:29:29.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.022+0000 7ff27bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b0eceec8c0 0x55b0eceee9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41122/0 (socket says 172.21.15.16:41122) 2023-05-20T01:29:29.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.022+0000 7ff27bfff700 1 -- 172.21.15.16:0/500851638 learned_addr learned my addr 172.21.15.16:0/500851638 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:29:29.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.023+0000 7ff27b7fe700 1 --2- 172.21.15.16:0/500851638 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55b0ecf19450 0x55b0ecf19820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:29:29.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.023+0000 7ff27affd700 1 --2- 172.21.15.16:0/500851638 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b0ecf15750 0x55b0eceee4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:29:29.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.023+0000 7ff27bfff700 1 -- 172.21.15.16:0/500851638 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55b0ecf19450 msgr2=0x55b0ecf19820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:29:29.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.023+0000 7ff27bfff700 1 --2- 172.21.15.16:0/500851638 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55b0ecf19450 0x55b0ecf19820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:29:29.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.023+0000 7ff27bfff700 1 -- 172.21.15.16:0/500851638 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b0ecf15750 msgr2=0x55b0eceee4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:29:29.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.023+0000 7ff27bfff700 1 --2- 172.21.15.16:0/500851638 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b0ecf15750 0x55b0eceee4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:29:29.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.023+0000 7ff27bfff700 1 -- 172.21.15.16:0/500851638 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b0eccc1630 con 0x55b0eceec8c0 2023-05-20T01:29:29.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.023+0000 7ff27b7fe700 1 --2- 172.21.15.16:0/500851638 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55b0ecf19450 0x55b0ecf19820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:29:29.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.023+0000 7ff27affd700 1 --2- 172.21.15.16:0/500851638 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b0ecf15750 0x55b0eceee4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-05-20T01:29:29.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.023+0000 7ff27bfff700 1 --2- 172.21.15.16:0/500851638 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b0eceec8c0 0x55b0eceee9f0 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7ff274089a90 tx=0x7ff27405e580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=127c05d74ed74b68 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:29:29.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.023+0000 7ff2617fa700 1 -- 172.21.15.16:0/500851638 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff27405e160 con 0x55b0eceec8c0 2023-05-20T01:29:29.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.023+0000 7ff2617fa700 1 -- 172.21.15.16:0/500851638 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff27405e2c0 con 0x55b0eceec8c0 2023-05-20T01:29:29.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.023+0000 7ff2617fa700 1 -- 172.21.15.16:0/500851638 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff27401d420 con 0x55b0eceec8c0 2023-05-20T01:29:29.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.024+0000 7ff28e3e2b80 1 -- 172.21.15.16:0/500851638 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b0eceec8c0 msgr2=0x55b0eceee9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:29:29.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.024+0000 7ff28e3e2b80 1 --2- 172.21.15.16:0/500851638 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b0eceec8c0 0x55b0eceee9f0 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7ff274089a90 tx=0x7ff27405e580 comp rx=0 tx=0).stop 2023-05-20T01:29:29.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.024+0000 7ff28e3e2b80 1 -- 172.21.15.16:0/500851638 shutdown_connections 2023-05-20T01:29:29.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.024+0000 7ff28e3e2b80 1 --2- 172.21.15.16:0/500851638 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b0eceec8c0 0x55b0eceee9f0 unknown :-1 s=CLOSED pgs=1770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:29:29.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.024+0000 7ff28e3e2b80 1 --2- 172.21.15.16:0/500851638 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55b0ecf19450 0x55b0ecf19820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:29:29.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.024+0000 7ff28e3e2b80 1 --2- 172.21.15.16:0/500851638 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b0ecf15750 0x55b0eceee4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:29:29.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.024+0000 7ff28e3e2b80 1 -- 172.21.15.16:0/500851638 >> 172.21.15.16:0/500851638 conn(0x55b0ecee3f80 msgr2=0x55b0ecf34260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:29:29.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.024+0000 7ff28e3e2b80 1 -- 172.21.15.16:0/500851638 shutdown_connections 2023-05-20T01:29:29.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.024+0000 7ff28e3e2b80 1 -- 172.21.15.16:0/500851638 wait complete. 2023-05-20T01:29:29.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.024+0000 7ff28e3e2b80 1 -- start start 2023-05-20T01:29:29.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.025+0000 7ff28e3e2b80 1 Processor -- start 2023-05-20T01:29:29.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.025+0000 7ff28e3e2b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b0eceec8c0 0x55b0ecf12540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:29:29.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.025+0000 7ff28e3e2b80 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55b0ecf15750 0x55b0ecf33760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:29:29.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.025+0000 7ff28e3e2b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b0ecf19450 0x55b0eceee790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:29:29.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.025+0000 7ff28e3e2b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0ecde18e0 con 0x55b0eceec8c0 2023-05-20T01:29:29.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.025+0000 7ff28e3e2b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0ecde1580 con 0x55b0ecf19450 2023-05-20T01:29:29.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.025+0000 7ff28e3e2b80 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b0eccc3920 con 0x55b0ecf15750 2023-05-20T01:29:29.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.025+0000 7ff28e3e2b80 10 client.0 _ll_register_callbacks cb 0x55b0ecdff780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-05-20T01:29:29.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.025+0000 7ff27b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b0eceec8c0 0x55b0ecf12540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:29:29.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.026+0000 7ff27b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b0eceec8c0 0x55b0ecf12540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41124/0 (socket says 172.21.15.16:41124) 2023-05-20T01:29:29.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.026+0000 7ff27b7fe700 1 -- 172.21.15.16:0/3322790350 learned_addr learned my addr 172.21.15.16:0/3322790350 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:29:29.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.026+0000 7ff27bfff700 1 --2- 172.21.15.16:0/3322790350 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b0ecf19450 0x55b0eceee790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:29:29.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.026+0000 7ff27affd700 1 --2- 172.21.15.16:0/3322790350 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55b0ecf15750 0x55b0ecf33760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:29:29.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.026+0000 7ff27b7fe700 1 -- 172.21.15.16:0/3322790350 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55b0ecf15750 msgr2=0x55b0ecf33760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:29:29.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.026+0000 7ff27b7fe700 1 --2- 172.21.15.16:0/3322790350 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55b0ecf15750 0x55b0ecf33760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:29:29.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.026+0000 7ff27b7fe700 1 -- 172.21.15.16:0/3322790350 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b0ecf19450 msgr2=0x55b0eceee790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:29:29.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.026+0000 7ff27b7fe700 1 --2- 172.21.15.16:0/3322790350 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b0ecf19450 0x55b0eceee790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:29:29.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.026+0000 7ff27b7fe700 1 -- 172.21.15.16:0/3322790350 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff27c0cde80 con 0x55b0eceec8c0 2023-05-20T01:29:29.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.026+0000 7ff27bfff700 1 --2- 172.21.15.16:0/3322790350 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b0ecf19450 0x55b0eceee790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-05-20T01:29:29.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.026+0000 7ff27b7fe700 1 --2- 172.21.15.16:0/3322790350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b0eceec8c0 0x55b0ecf12540 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7ff26c014e20 tx=0x7ff26c003780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2c0b364e783a0ce server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:29:29.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.026+0000 7ff25a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-05-20T01:29:29.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.026+0000 7ff25a7fc700 1 -- 172.21.15.16:0/3322790350 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff26c012e00 con 0x55b0eceec8c0 2023-05-20T01:29:29.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.026+0000 7ff25a7fc700 1 -- 172.21.15.16:0/3322790350 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(140..140 src has 1..140) v4 ==== 6612+0+0 (secure 0 0 0) 0x7ff26c020420 con 0x55b0eceec8c0 2023-05-20T01:29:29.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.026+0000 7ff28e3e2b80 1 -- 172.21.15.16:0/3322790350 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b0ecee1740 con 0x55b0eceec8c0 2023-05-20T01:29:29.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.026+0000 7ff25a7fc700 1 -- 172.21.15.16:0/3322790350 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=112) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff26c020630 con 0x55b0eceec8c0 2023-05-20T01:29:29.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.026+0000 7ff28e3e2b80 10 client.9225 fetch_fsmap learned FSMap version 112 2023-05-20T01:29:29.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.026+0000 7ff28e3e2b80 1 -- 172.21.15.16:0/3322790350 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=112,monmap=2+}) v3 -- 0x55b0ece06190 con 0x55b0eceec8c0 2023-05-20T01:29:29.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.026+0000 7ff25a7fc700 1 -- 172.21.15.16:0/3322790350 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b0ecee1740 con 0x55b0eceec8c0 2023-05-20T01:29:29.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.026+0000 7ff25a7fc700 1 -- 172.21.15.16:0/3322790350 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 112) v1 ==== 3114+0+0 (secure 0 0 0) 0x7ff26c020810 con 0x55b0eceec8c0 2023-05-20T01:29:29.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.027+0000 7ff28e3e2b80 10 client.9225 fetch_fsmap finished waiting for FSMap version 112 2023-05-20T01:29:29.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.027+0000 7ff28e3e2b80 10 client.9225 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2023-05-20T01:29:29.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.027+0000 7ff28e3e2b80 20 client.9225 populate_metadata read hostname 'smithi016' 2023-05-20T01:29:29.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.027+0000 7ff28e3e2b80 1 --2- 172.21.15.16:0/3322790350 >> [v2:172.21.15.16:6835/2952207608,v1:172.21.15.16:6837/2952207608] conn(0x55b0ecf46bf0 0x55b0ecf46ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:29:29.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.027+0000 7ff28e3e2b80 4 client.9225 mds_command: new command op to 9054 tid=0[{"prefix": "get_command_descriptions"}] 2023-05-20T01:29:29.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.027+0000 7ff28e3e2b80 1 -- 172.21.15.16:0/3322790350 --> [v2:172.21.15.16:6835/2952207608,v1:172.21.15.16:6837/2952207608] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b0ece020a0 con 0x55b0ecf46bf0 2023-05-20T01:29:29.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.027+0000 7ff27affd700 1 --2- 172.21.15.16:0/3322790350 >> [v2:172.21.15.16:6835/2952207608,v1:172.21.15.16:6837/2952207608] conn(0x55b0ecf46bf0 0x55b0ecf46ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:29:29.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.027+0000 7ff27affd700 1 --2- 172.21.15.16:0/3322790350 >> [v2:172.21.15.16:6835/2952207608,v1:172.21.15.16:6837/2952207608] conn(0x55b0ecf46bf0 0x55b0ecf46ff0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8032fa4b24d309b9 server_cookie=e8cb8b32f31516ab in_seq=0 out_seq=0 2023-05-20T01:29:29.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.027+0000 7ff25a7fc700 10 client.9225 ms_handle_connect on v2:172.21.15.16:6835/2952207608 2023-05-20T01:29:29.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.028+0000 7ff25a7fc700 1 -- 172.21.15.16:0/3322790350 <== mds.0 v2:172.21.15.16:6835/2952207608 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55b0ece020a0 con 0x55b0ecf46bf0 2023-05-20T01:29:29.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.028+0000 7ff25a7fc700 10 client.9225 handle_command_reply: tid=0 2023-05-20T01:29:29.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.028+0000 7ff28e3e2b80 1 client.9225 shutdown 2023-05-20T01:29:29.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.028+0000 7ff28e3e2b80 1 -- 172.21.15.16:0/3322790350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b0eceec8c0 msgr2=0x55b0ecf12540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:29:29.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.028+0000 7ff28e3e2b80 1 --2- 172.21.15.16:0/3322790350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b0eceec8c0 0x55b0ecf12540 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7ff26c014e20 tx=0x7ff26c003780 comp rx=0 tx=0).stop 2023-05-20T01:29:29.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.028+0000 7ff28e3e2b80 1 -- 172.21.15.16:0/3322790350 shutdown_connections 2023-05-20T01:29:29.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.028+0000 7ff28e3e2b80 1 --2- 172.21.15.16:0/3322790350 >> [v2:172.21.15.16:6835/2952207608,v1:172.21.15.16:6837/2952207608] conn(0x55b0ecf46bf0 0x55b0ecf46ff0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:29:29.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.028+0000 7ff28e3e2b80 1 --2- 172.21.15.16:0/3322790350 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b0ecf19450 0x55b0eceee790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:29:29.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.028+0000 7ff28e3e2b80 1 --2- 172.21.15.16:0/3322790350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b0eceec8c0 0x55b0ecf12540 unknown :-1 s=CLOSED pgs=1771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:29:29.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.028+0000 7ff28e3e2b80 1 --2- 172.21.15.16:0/3322790350 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55b0ecf15750 0x55b0ecf33760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:29:29.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.028+0000 7ff25a7fc700 0 client.9225 ms_handle_reset on v2:172.21.15.16:6835/2952207608 2023-05-20T01:29:29.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.028+0000 7ff28e3e2b80 1 -- 172.21.15.16:0/3322790350 >> 172.21.15.16:0/3322790350 conn(0x55b0ecee3f80 msgr2=0x55b0eceea6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:29:29.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.028+0000 7ff28e3e2b80 1 -- 172.21.15.16:0/3322790350 shutdown_connections 2023-05-20T01:29:29.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.028+0000 7ff28e3e2b80 1 -- 172.21.15.16:0/3322790350 wait complete. 2023-05-20T01:29:29.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.028+0000 7ff28e3e2b80 20 client.9225 trim_cache size 0 max 16384 2023-05-20T01:29:29.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.045+0000 7ff28e3e2b80 1 Processor -- start 2023-05-20T01:29:29.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.045+0000 7ff28e3e2b80 1 -- start start 2023-05-20T01:29:29.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.046+0000 7ff28e3e2b80 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55b0eceec8c0 0x55b0ecf35ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:29:29.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.046+0000 7ff28e3e2b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b0ecf15750 0x55b0ecee5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:29:29.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.046+0000 7ff28e3e2b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b0ecf19450 0x55b0ecef0840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:29:29.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.046+0000 7ff28e3e2b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0ecf46800 con 0x55b0ecf15750 2023-05-20T01:29:29.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.046+0000 7ff28e3e2b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0ecf31d30 con 0x55b0ecf19450 2023-05-20T01:29:29.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.046+0000 7ff28e3e2b80 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b0ecee5990 con 0x55b0eceec8c0 2023-05-20T01:29:29.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.046+0000 7ff27affd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b0ecf15750 0x55b0ecee5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:29:29.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.046+0000 7ff27affd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b0ecf15750 0x55b0ecee5450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41134/0 (socket says 172.21.15.16:41134) 2023-05-20T01:29:29.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.046+0000 7ff27affd700 1 -- 172.21.15.16:0/1951229569 learned_addr learned my addr 172.21.15.16:0/1951229569 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:29:29.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.046+0000 7ff27affd700 1 -- 172.21.15.16:0/1951229569 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55b0eceec8c0 msgr2=0x55b0ecf35ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:29:29.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.046+0000 7ff27affd700 1 --2- 172.21.15.16:0/1951229569 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55b0eceec8c0 0x55b0ecf35ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:29:29.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.046+0000 7ff27affd700 1 -- 172.21.15.16:0/1951229569 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b0ecf19450 msgr2=0x55b0ecef0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:29:29.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.046+0000 7ff27affd700 1 --2- 172.21.15.16:0/1951229569 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b0ecf19450 0x55b0ecef0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:29:29.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.046+0000 7ff27affd700 1 -- 172.21.15.16:0/1951229569 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b0ece05ee0 con 0x55b0ecf15750 2023-05-20T01:29:29.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.046+0000 7ff27affd700 1 --2- 172.21.15.16:0/1951229569 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b0ecf15750 0x55b0ecee5450 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7ff27c15af20 tx=0x7ff270009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60e36a956148b9b2 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:29:29.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.046+0000 7ff261ffb700 1 -- 172.21.15.16:0/1951229569 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2700070c0 con 0x55b0ecf15750 2023-05-20T01:29:29.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.046+0000 7ff261ffb700 1 -- 172.21.15.16:0/1951229569 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff27000d040 con 0x55b0ecf15750 2023-05-20T01:29:29.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.046+0000 7ff261ffb700 1 -- 172.21.15.16:0/1951229569 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff270007440 con 0x55b0ecf15750 2023-05-20T01:29:29.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.047+0000 7ff28e3e2b80 1 -- 172.21.15.16:0/1951229569 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b0ecf15750 msgr2=0x55b0ecee5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:29:29.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.047+0000 7ff28e3e2b80 1 --2- 172.21.15.16:0/1951229569 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b0ecf15750 0x55b0ecee5450 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7ff27c15af20 tx=0x7ff270009040 comp rx=0 tx=0).stop 2023-05-20T01:29:29.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.047+0000 7ff28e3e2b80 1 -- 172.21.15.16:0/1951229569 shutdown_connections 2023-05-20T01:29:29.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.047+0000 7ff28e3e2b80 1 --2- 172.21.15.16:0/1951229569 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b0ecf19450 0x55b0ecef0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:29:29.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.047+0000 7ff28e3e2b80 1 --2- 172.21.15.16:0/1951229569 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55b0eceec8c0 0x55b0ecf35ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:29:29.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.047+0000 7ff28e3e2b80 1 --2- 172.21.15.16:0/1951229569 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b0ecf15750 0x55b0ecee5450 unknown :-1 s=CLOSED pgs=1772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:29:29.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.047+0000 7ff28e3e2b80 1 -- 172.21.15.16:0/1951229569 >> 172.21.15.16:0/1951229569 conn(0x55b0ecee3f80 msgr2=0x55b0ecf33480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:29:29.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.047+0000 7ff28e3e2b80 1 -- 172.21.15.16:0/1951229569 shutdown_connections 2023-05-20T01:29:29.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.047+0000 7ff28e3e2b80 1 -- 172.21.15.16:0/1951229569 wait complete. 2023-05-20T01:29:29.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.048+0000 7ff28e3e2b80 1 -- start start 2023-05-20T01:29:29.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.048+0000 7ff28e3e2b80 1 Processor -- start 2023-05-20T01:29:29.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.049+0000 7ff28e3e2b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b0eceec8c0 0x55b0ecf34290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:29:29.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.049+0000 7ff28e3e2b80 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55b0ecf15750 0x55b0ecf347d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:29:29.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.049+0000 7ff28e3e2b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b0ecf19450 0x55b0ecf30630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:29:29.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.049+0000 7ff28e3e2b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0ecf168e0 con 0x55b0eceec8c0 2023-05-20T01:29:29.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.049+0000 7ff28e3e2b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0ecf44b60 con 0x55b0ecf19450 2023-05-20T01:29:29.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.049+0000 7ff28e3e2b80 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b0ecf16180 con 0x55b0ecf15750 2023-05-20T01:29:29.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.049+0000 7ff28e3e2b80 10 client.0 _ll_register_callbacks cb 0x7ff26c014710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-05-20T01:29:29.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.049+0000 7ff27b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b0eceec8c0 0x55b0ecf34290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:29:29.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.049+0000 7ff27b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b0eceec8c0 0x55b0ecf34290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41138/0 (socket says 172.21.15.16:41138) 2023-05-20T01:29:29.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.049+0000 7ff27b7fe700 1 -- 172.21.15.16:0/4018980360 learned_addr learned my addr 172.21.15.16:0/4018980360 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-05-20T01:29:29.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.049+0000 7ff27b7fe700 1 -- 172.21.15.16:0/4018980360 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55b0ecf15750 msgr2=0x55b0ecf347d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:29:29.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.049+0000 7ff27b7fe700 1 --2- 172.21.15.16:0/4018980360 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55b0ecf15750 0x55b0ecf347d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:29:29.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.049+0000 7ff27b7fe700 1 -- 172.21.15.16:0/4018980360 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b0ecf19450 msgr2=0x55b0ecf30630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-05-20T01:29:29.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.049+0000 7ff27b7fe700 1 --2- 172.21.15.16:0/4018980360 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b0ecf19450 0x55b0ecf30630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:29:29.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.049+0000 7ff27b7fe700 1 -- 172.21.15.16:0/4018980360 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff27000f040 con 0x55b0eceec8c0 2023-05-20T01:29:29.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.049+0000 7ff27b7fe700 1 --2- 172.21.15.16:0/4018980360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b0eceec8c0 0x55b0ecf34290 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7ff26c007e70 tx=0x7ff26c0037f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e871968d50e36c52 server_cookie=0 in_seq=0 out_seq=0 2023-05-20T01:29:29.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.049+0000 7ff25a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-05-20T01:29:29.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.049+0000 7ff25a7fc700 1 -- 172.21.15.16:0/4018980360 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff26c012e00 con 0x55b0eceec8c0 2023-05-20T01:29:29.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.049+0000 7ff25a7fc700 1 -- 172.21.15.16:0/4018980360 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(140..140 src has 1..140) v4 ==== 6612+0+0 (secure 0 0 0) 0x7ff26c028350 con 0x55b0eceec8c0 2023-05-20T01:29:29.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.049+0000 7ff28e3e2b80 1 -- 172.21.15.16:0/4018980360 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff27405e2c0 con 0x55b0eceec8c0 2023-05-20T01:29:29.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.049+0000 7ff25a7fc700 1 -- 172.21.15.16:0/4018980360 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=112) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff26c028510 con 0x55b0eceec8c0 2023-05-20T01:29:29.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.049+0000 7ff28e3e2b80 10 client.9231 fetch_fsmap learned FSMap version 112 2023-05-20T01:29:29.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.049+0000 7ff28e3e2b80 1 -- 172.21.15.16:0/4018980360 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=112,monmap=2+}) v3 -- 0x55b0ecc56c60 con 0x55b0eceec8c0 2023-05-20T01:29:29.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.050+0000 7ff25a7fc700 1 -- 172.21.15.16:0/4018980360 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff27405e2c0 con 0x55b0eceec8c0 2023-05-20T01:29:29.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.050+0000 7ff25a7fc700 1 -- 172.21.15.16:0/4018980360 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 112) v1 ==== 3114+0+0 (secure 0 0 0) 0x7ff26c037020 con 0x55b0eceec8c0 2023-05-20T01:29:29.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.050+0000 7ff28e3e2b80 10 client.9231 fetch_fsmap finished waiting for FSMap version 112 2023-05-20T01:29:29.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.050+0000 7ff28e3e2b80 10 client.9231 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2023-05-20T01:29:29.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.050+0000 7ff28e3e2b80 20 client.9231 populate_metadata read hostname 'smithi016' 2023-05-20T01:29:29.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.050+0000 7ff28e3e2b80 1 --2- 172.21.15.16:0/4018980360 >> [v2:172.21.15.16:6835/2952207608,v1:172.21.15.16:6837/2952207608] conn(0x55b0ecf4d7f0 0x55b0ecf30b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-05-20T01:29:29.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.050+0000 7ff28e3e2b80 4 client.9231 mds_command: new command op to 9054 tid=0[{"prefix": "scrub status"}] 2023-05-20T01:29:29.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.050+0000 7ff28e3e2b80 1 -- 172.21.15.16:0/4018980360 --> [v2:172.21.15.16:6835/2952207608,v1:172.21.15.16:6837/2952207608] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b0ece0e660 con 0x55b0ecf4d7f0 2023-05-20T01:29:29.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.050+0000 7ff27affd700 1 --2- 172.21.15.16:0/4018980360 >> [v2:172.21.15.16:6835/2952207608,v1:172.21.15.16:6837/2952207608] conn(0x55b0ecf4d7f0 0x55b0ecf30b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-05-20T01:29:29.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.050+0000 7ff25a7fc700 10 client.9231 ms_handle_connect on v2:172.21.15.16:6835/2952207608 2023-05-20T01:29:29.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.050+0000 7ff27affd700 1 --2- 172.21.15.16:0/4018980360 >> [v2:172.21.15.16:6835/2952207608,v1:172.21.15.16:6837/2952207608] conn(0x55b0ecf4d7f0 0x55b0ecf30b70 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7e53c5076236a97 server_cookie=7b5d2ea9d7d241ed in_seq=0 out_seq=0 2023-05-20T01:29:29.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.050+0000 7ff25a7fc700 1 -- 172.21.15.16:0/4018980360 <== mds.0 v2:172.21.15.16:6835/2952207608 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55b0ece0e660 con 0x55b0ecf4d7f0 2023-05-20T01:29:29.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.050+0000 7ff25a7fc700 10 client.9231 handle_command_reply: tid=0 2023-05-20T01:29:29.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.050+0000 7ff28e3e2b80 1 client.9231 shutdown 2023-05-20T01:29:29.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.050+0000 7ff28e3e2b80 1 -- 172.21.15.16:0/4018980360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b0eceec8c0 msgr2=0x55b0ecf34290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:29:29.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.050+0000 7ff28e3e2b80 1 --2- 172.21.15.16:0/4018980360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b0eceec8c0 0x55b0ecf34290 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7ff26c007e70 tx=0x7ff26c0037f0 comp rx=0 tx=0).stop 2023-05-20T01:29:29.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.051+0000 7ff28e3e2b80 1 -- 172.21.15.16:0/4018980360 shutdown_connections 2023-05-20T01:29:29.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.051+0000 7ff28e3e2b80 1 --2- 172.21.15.16:0/4018980360 >> [v2:172.21.15.16:6835/2952207608,v1:172.21.15.16:6837/2952207608] conn(0x55b0ecf4d7f0 0x55b0ecf30b70 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:29:29.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.051+0000 7ff28e3e2b80 1 --2- 172.21.15.16:0/4018980360 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b0ecf19450 0x55b0ecf30630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:29:29.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.051+0000 7ff28e3e2b80 1 --2- 172.21.15.16:0/4018980360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b0eceec8c0 0x55b0ecf34290 unknown :-1 s=CLOSED pgs=1773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:29:29.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.051+0000 7ff25a7fc700 0 client.9231 ms_handle_reset on v2:172.21.15.16:6835/2952207608 2023-05-20T01:29:29.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.051+0000 7ff28e3e2b80 1 --2- 172.21.15.16:0/4018980360 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55b0ecf15750 0x55b0ecf347d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:29:29.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.051+0000 7ff28e3e2b80 1 -- 172.21.15.16:0/4018980360 >> 172.21.15.16:0/4018980360 conn(0x55b0ecee3f80 msgr2=0x55b0ecf33480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:29:29.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.051+0000 7ff28e3e2b80 1 -- 172.21.15.16:0/4018980360 shutdown_connections 2023-05-20T01:29:29.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.051+0000 7ff28e3e2b80 1 -- 172.21.15.16:0/4018980360 wait complete. 2023-05-20T01:29:29.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.051+0000 7ff28e3e2b80 20 client.9231 trim_cache size 0 max 16384 2023-05-20T01:29:29.064 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-05-20T01:29:29.064 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2023-05-20T01:29:29.064 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2023-05-20T01:29:29.065 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-05-20T01:29:29.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.051+0000 7ff282256700 1 -- 172.21.15.16:0/2591951631 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff26806ace0 msgr2=0x7ff26806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:29:29.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.051+0000 7ff282256700 1 --2- 172.21.15.16:0/2591951631 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff26806ace0 0x7ff26806d190 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7ff26c003e60 tx=0x7ff26c009040 comp rx=0 tx=0).stop 2023-05-20T01:29:29.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.051+0000 7ff282256700 1 -- 172.21.15.16:0/2591951631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff27c15db10 msgr2=0x7ff27c1f5f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-05-20T01:29:29.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.051+0000 7ff282256700 1 --2- 172.21.15.16:0/2591951631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff27c15db10 0x7ff27c1f5f00 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7ff27c15ac80 tx=0x7ff274004ad0 comp rx=0 tx=0).stop 2023-05-20T01:29:29.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.051+0000 7ff282256700 1 -- 172.21.15.16:0/2591951631 shutdown_connections 2023-05-20T01:29:29.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.051+0000 7ff282256700 1 --2- 172.21.15.16:0/2591951631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff27c15db10 0x7ff27c1f5f00 unknown :-1 s=CLOSED pgs=1769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:29:29.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.051+0000 7ff282256700 1 --2- 172.21.15.16:0/2591951631 >> [v2:172.21.15.16:6832/3817237998,v1:172.21.15.16:6833/3817237998] conn(0x7ff26806ace0 0x7ff26806d190 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:29:29.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.051+0000 7ff282256700 1 --2- 172.21.15.16:0/2591951631 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff27c0b4f30 0x7ff27c0b8650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:29:29.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.051+0000 7ff282256700 1 --2- 172.21.15.16:0/2591951631 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff27c15a720 0x7ff27c1f59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-05-20T01:29:29.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.051+0000 7ff282256700 1 -- 172.21.15.16:0/2591951631 >> 172.21.15.16:0/2591951631 conn(0x7ff27c0cff30 msgr2=0x7ff27c159930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-05-20T01:29:29.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.051+0000 7ff282256700 1 -- 172.21.15.16:0/2591951631 shutdown_connections 2023-05-20T01:29:29.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-05-20T01:29:29.051+0000 7ff282256700 1 -- 172.21.15.16:0/2591951631 wait complete. 2023-05-20T01:29:29.068 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-05-20T01:29:29.068 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-05-20T01:29:29.068 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -rf subdir/) 2023-05-20T01:29:37.432 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:29:37.435 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:30:07.513 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:30:07.516 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:30:37.585 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:30:37.588 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:31:07.665 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:31:07.669 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:31:37.911 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:31:37.914 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:32:07.995 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:32:07.998 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:32:38.063 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:32:38.066 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:33:08.144 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:33:08.148 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:33:38.216 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:33:38.219 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:34:08.289 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:34:08.292 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:34:29.076 ERROR:teuthology:Uncaught exception (Hub) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_69caab8ec6f44c61c991ae693e8e7b8cd859a896/virtualenv/lib/python3.8/site-packages/paramiko/channel.py", line 747, in recv_stderr out = self.in_stderr_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_69caab8ec6f44c61c991ae693e8e7b8cd859a896/virtualenv/lib/python3.8/site-packages/paramiko/buffered_pipe.py", line 164, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_teuthology_69caab8ec6f44c61c991ae693e8e7b8cd859a896/teuthology/orchestra/run.py", line 323, in copy_file_to copy_to_log(src, logger, capture=stream, quiet=quiet) File "/home/teuthworker/src/git.ceph.com_teuthology_69caab8ec6f44c61c991ae693e8e7b8cd859a896/teuthology/orchestra/run.py", line 276, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_teuthology_69caab8ec6f44c61c991ae693e8e7b8cd859a896/virtualenv/lib/python3.8/site-packages/paramiko/file.py", line 125, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_teuthology_69caab8ec6f44c61c991ae693e8e7b8cd859a896/virtualenv/lib/python3.8/site-packages/paramiko/file.py", line 291, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_teuthology_69caab8ec6f44c61c991ae693e8e7b8cd859a896/virtualenv/lib/python3.8/site-packages/paramiko/channel.py", line 1376, in _read return self.channel.recv_stderr(size) File "/home/teuthworker/src/git.ceph.com_teuthology_69caab8ec6f44c61c991ae693e8e7b8cd859a896/virtualenv/lib/python3.8/site-packages/paramiko/channel.py", line 749, in recv_stderr raise socket.timeout() socket.timeout 2023-05-20T01:34:29.193 ERROR:teuthology:Uncaught exception (Hub) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_69caab8ec6f44c61c991ae693e8e7b8cd859a896/virtualenv/lib/python3.8/site-packages/paramiko/channel.py", line 699, in recv out = self.in_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_69caab8ec6f44c61c991ae693e8e7b8cd859a896/virtualenv/lib/python3.8/site-packages/paramiko/buffered_pipe.py", line 164, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_teuthology_69caab8ec6f44c61c991ae693e8e7b8cd859a896/teuthology/orchestra/run.py", line 323, in copy_file_to copy_to_log(src, logger, capture=stream, quiet=quiet) File "/home/teuthworker/src/git.ceph.com_teuthology_69caab8ec6f44c61c991ae693e8e7b8cd859a896/teuthology/orchestra/run.py", line 276, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_teuthology_69caab8ec6f44c61c991ae693e8e7b8cd859a896/virtualenv/lib/python3.8/site-packages/paramiko/file.py", line 125, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_teuthology_69caab8ec6f44c61c991ae693e8e7b8cd859a896/virtualenv/lib/python3.8/site-packages/paramiko/file.py", line 291, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_teuthology_69caab8ec6f44c61c991ae693e8e7b8cd859a896/virtualenv/lib/python3.8/site-packages/paramiko/channel.py", line 1361, in _read return self.channel.recv(size) File "/home/teuthworker/src/git.ceph.com_teuthology_69caab8ec6f44c61c991ae693e8e7b8cd859a896/virtualenv/lib/python3.8/site-packages/paramiko/channel.py", line 701, in recv raise socket.timeout() socket.timeout 2023-05-20T01:34:38.377 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:34:38.379 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:35:08.450 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:35:08.452 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:35:38.519 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:35:38.522 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:36:08.583 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:36:08.586 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:36:38.689 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:36:38.692 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:37:08.773 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:37:08.777 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:37:38.838 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:37:38.842 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:38:08.913 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:38:08.916 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:38:38.976 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:38:38.979 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:39:09.048 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:39:09.051 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:39:39.128 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:39:39.131 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:40:09.210 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:40:09.213 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:40:39.282 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:40:39.285 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:41:09.352 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:41:09.355 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:41:39.452 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:41:39.455 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:42:09.515 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:42:09.518 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:42:39.595 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:42:39.597 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:43:09.683 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:43:09.686 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:43:39.754 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:43:39.758 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:44:09.826 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:44:09.830 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:44:39.906 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:44:39.909 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:45:09.994 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:45:09.997 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:45:40.067 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:45:40.071 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:46:10.138 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:46:10.141 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:46:40.279 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:46:40.283 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:47:10.409 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:47:10.412 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:47:40.514 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:47:40.517 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:48:10.577 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:48:10.581 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:48:40.651 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:48:40.654 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:49:10.755 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:49:10.758 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:49:40.826 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:49:40.829 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:50:11.011 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:50:11.014 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:50:41.086 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:50:41.090 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:51:11.150 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:51:11.154 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:51:41.245 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:51:41.249 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:52:11.318 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:52:11.322 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:52:41.392 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:52:41.395 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:53:11.462 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:53:11.466 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:53:41.538 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:53:41.542 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:54:11.614 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:54:11.618 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:54:41.681 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:54:41.684 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:55:11.766 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:55:11.769 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:55:41.838 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:55:41.841 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:56:11.960 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:56:11.966 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:56:42.041 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:56:42.044 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:57:12.373 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:57:12.377 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:57:42.434 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:57:42.438 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:58:12.515 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:58:12.518 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:58:42.577 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:58:42.580 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:59:12.708 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:59:12.711 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:59:42.784 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T01:59:42.787 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:00:12.851 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:00:12.856 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:00:42.927 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:00:42.930 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:01:12.994 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:01:12.997 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:01:43.084 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:01:43.087 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:02:13.150 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:02:13.153 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:02:43.222 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:02:43.225 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:03:13.286 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:03:13.289 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:03:43.367 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:03:43.370 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:04:13.456 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:04:13.458 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:04:43.528 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:04:43.531 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:05:13.607 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:05:13.610 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:05:43.788 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:05:43.791 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:06:13.919 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:06:13.922 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:06:44.066 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:06:44.069 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:07:14.157 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:07:14.160 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:07:44.234 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:07:44.237 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:08:14.306 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:08:14.309 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:08:44.377 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:08:44.380 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:09:14.458 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:09:14.461 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:09:44.521 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:09:44.525 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:10:14.584 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:10:14.587 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:10:44.665 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:10:44.668 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:11:14.755 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:11:14.758 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:11:44.834 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:11:44.838 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:12:14.914 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:12:14.918 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:12:44.994 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:12:44.997 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:13:15.158 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:13:15.166 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:13:45.229 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:13:45.233 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:14:15.301 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:14:15.304 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:14:45.374 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:14:45.377 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:15:15.461 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:15:15.464 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:15:45.534 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:15:45.537 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:16:15.611 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:16:15.615 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:16:45.681 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:16:45.684 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:17:16.074 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:17:16.077 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:17:46.195 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:17:46.198 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:18:16.298 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:18:16.302 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:18:46.412 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:18:46.416 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:19:16.477 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:19:16.481 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:19:46.566 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:19:46.568 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:20:16.639 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:20:16.642 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:20:46.712 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:20:46.715 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:21:16.799 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:21:16.802 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:21:47.046 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:21:47.049 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:22:17.109 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:22:17.112 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:22:47.282 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:22:47.284 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:23:17.378 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:23:17.381 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:23:47.541 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:23:47.545 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:24:17.621 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:24:17.624 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:24:47.685 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:24:47.688 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:25:17.841 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:25:17.845 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:25:48.118 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:25:48.122 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:26:18.184 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:26:18.187 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:26:48.273 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:26:48.276 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:27:18.344 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:27:18.347 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:27:48.558 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:27:48.562 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:28:18.739 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:28:18.742 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:28:48.810 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:28:48.814 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:29:18.881 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:29:18.884 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:29:48.957 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:29:48.960 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:30:19.033 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:30:19.037 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:30:49.115 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:30:49.121 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:31:19.189 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:31:19.192 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:31:49.266 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:31:49.269 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:32:19.339 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:32:19.342 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:32:49.585 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:32:49.588 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:33:19.649 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:33:19.652 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:33:49.738 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:33:49.741 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:34:19.808 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:34:19.811 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:34:49.872 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:34:49.876 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:35:19.944 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:35:19.948 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:35:50.033 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:35:50.036 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:36:20.097 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:36:20.100 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:36:50.169 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:36:50.172 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:37:20.231 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:37:20.234 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:37:50.326 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:37:50.329 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:38:20.406 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:38:20.410 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:38:50.489 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:38:50.493 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:39:20.562 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:39:20.565 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:39:50.625 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:39:50.628 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:40:20.698 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:40:20.701 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:40:50.762 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:40:50.765 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:41:20.851 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:41:20.854 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:41:50.930 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:41:50.933 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:42:21.020 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:42:21.023 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:42:51.101 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:42:51.104 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:43:21.173 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:43:21.177 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:43:51.244 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:43:51.247 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:44:21.334 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:44:21.338 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:44:51.404 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:44:51.407 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:45:21.463 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:45:21.466 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:45:51.535 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:45:51.538 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:46:21.629 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:46:21.633 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:46:51.784 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:46:51.787 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:47:21.947 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:47:21.951 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:47:52.012 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:47:52.015 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:48:22.099 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:48:22.102 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:48:52.173 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:48:52.176 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:49:22.303 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:49:22.306 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:49:52.392 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:49:52.395 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:50:22.456 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:50:22.459 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:50:52.529 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:50:52.532 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:51:22.607 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:51:22.611 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:51:52.680 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:51:52.683 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:52:22.768 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:52:22.772 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:52:52.973 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:52:52.976 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:53:23.152 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:53:23.155 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:53:53.242 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:53:53.245 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:54:23.317 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:54:23.321 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:54:53.404 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:54:53.408 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:55:23.475 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:55:23.478 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:55:53.547 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:55:53.551 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:56:23.620 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:56:23.623 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:56:53.708 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:56:53.710 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:57:23.779 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:57:23.782 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:57:53.851 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:57:53.855 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:58:23.922 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:58:23.925 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:58:53.989 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:58:53.993 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:59:24.092 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:59:24.095 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:59:54.285 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T02:59:54.288 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:00:24.357 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:00:24.360 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:00:54.649 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:00:54.653 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:01:24.824 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:01:24.827 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:01:54.893 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:01:54.896 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:02:24.965 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:02:24.968 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:02:55.055 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:02:55.058 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:03:25.127 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:03:25.132 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:03:55.198 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:03:55.201 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:04:25.294 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:04:25.298 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:04:55.379 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:04:55.384 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:05:25.464 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:05:25.468 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:05:55.737 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:05:55.739 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:06:25.816 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:06:25.819 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:06:55.879 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:06:55.882 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:07:25.976 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:07:25.979 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:07:56.065 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:07:56.068 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:08:26.137 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:08:26.139 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:08:56.205 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:08:56.208 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:09:26.281 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:09:26.283 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:09:56.562 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:09:56.565 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:10:26.641 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:10:26.645 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:10:56.714 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:10:56.717 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:11:26.826 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:11:26.830 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:11:56.899 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:11:56.902 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:12:26.988 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:12:26.991 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:12:57.067 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:12:57.069 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:13:27.138 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:13:27.141 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:13:57.215 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:13:57.218 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:14:27.542 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:14:27.545 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:14:57.680 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:14:57.684 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:15:27.802 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:15:27.805 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:15:57.881 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:15:57.884 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:16:27.953 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:16:27.957 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:16:58.184 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:16:58.188 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:17:28.256 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:17:29.164 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:17:59.236 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:17:59.239 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:18:29.299 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:18:29.302 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:18:59.382 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:18:59.385 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:19:29.481 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:19:29.484 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:19:59.553 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:19:59.556 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:20:29.632 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:20:29.635 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:20:59.706 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:20:59.709 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:21:29.793 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:21:29.797 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:21:59.865 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:21:59.868 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:22:29.949 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:22:29.952 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:23:00.042 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:23:00.045 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:23:30.114 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:23:30.119 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:24:00.178 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:24:00.181 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:24:30.258 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:24:30.261 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:25:00.346 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:25:00.349 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:25:30.435 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:25:30.438 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:26:00.511 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:26:00.516 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:26:30.596 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:26:30.600 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:27:00.834 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:27:01.116 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:27:31.180 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:27:31.183 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:28:01.252 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:28:01.255 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:28:31.324 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:28:31.327 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:29:01.622 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:29:01.625 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:29:31.802 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:29:31.805 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:30:01.948 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:30:01.951 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:30:32.122 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:30:32.125 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:31:02.186 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:31:02.188 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:31:32.255 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:31:32.258 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:32:02.327 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:32:02.330 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:32:32.421 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:32:32.423 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:33:02.521 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:33:02.524 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:33:32.618 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:33:32.623 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:34:02.731 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:34:02.736 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:34:32.812 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:34:32.815 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:35:02.893 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:35:02.897 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:35:33.232 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:35:33.237 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:36:03.295 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:36:03.298 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:36:33.435 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:36:33.438 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:37:03.521 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:37:03.524 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:37:33.596 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:37:33.599 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:38:03.683 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:38:03.686 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:38:33.758 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:38:33.761 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:39:03.834 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:39:03.837 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:39:33.919 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:39:33.922 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:40:03.986 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:40:03.989 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:40:34.057 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:40:34.060 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:41:04.146 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:41:04.149 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:41:34.235 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:41:34.246 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:42:04.326 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:42:04.330 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:42:34.402 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:42:34.405 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:43:04.484 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:43:04.487 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:43:34.572 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:43:34.575 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:44:04.878 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:44:04.881 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:44:35.079 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:44:35.083 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:45:05.257 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:45:05.260 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:45:35.336 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:45:35.339 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:46:05.401 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:46:05.405 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:46:35.466 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:46:35.468 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:47:05.551 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:47:05.554 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:47:35.623 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:47:35.626 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:48:05.703 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:48:05.706 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:48:35.776 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:48:35.779 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:49:05.864 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:49:05.867 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:49:35.952 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:49:35.955 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:50:06.032 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:50:06.035 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:50:36.180 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:50:36.182 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:51:06.302 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:51:06.305 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:51:36.373 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:51:36.376 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:52:06.436 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:52:06.439 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:52:36.511 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:52:36.515 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:53:06.585 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:53:06.588 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:53:36.669 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:53:36.673 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:54:06.738 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:54:06.742 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:54:36.822 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:54:36.825 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:55:06.894 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:55:06.898 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:55:36.968 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:55:36.971 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:56:07.058 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:56:07.062 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:56:37.152 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:56:37.155 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:57:07.223 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:57:07.226 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:57:37.289 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:57:37.292 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:58:07.361 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:58:07.365 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:58:37.447 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:58:37.450 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:59:07.519 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:59:07.522 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:59:37.613 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T03:59:37.616 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:00:07.702 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:00:07.706 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:00:37.776 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:00:37.779 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:01:07.848 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:01:07.851 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:01:37.937 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:01:37.940 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:02:08.011 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:02:08.014 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:02:38.100 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:02:38.103 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:03:08.169 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:03:08.172 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:03:38.238 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:03:38.241 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:04:08.330 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:04:08.333 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:04:38.409 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:04:38.412 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:05:08.481 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:05:08.484 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:05:38.563 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:05:38.567 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:06:08.638 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:06:08.640 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:06:38.699 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:06:38.702 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:07:08.773 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:07:08.776 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:07:38.895 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:07:38.976 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:08:09.051 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:08:09.054 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:08:39.114 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:08:39.118 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:09:09.411 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:09:09.414 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:09:39.635 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:09:39.638 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:10:09.705 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:10:09.708 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:10:39.786 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:10:39.789 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:11:09.983 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:11:09.986 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:11:40.063 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:11:40.067 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:12:10.144 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:12:10.146 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:12:40.220 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:12:40.223 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:13:10.296 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:13:10.299 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:13:40.367 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:13:40.370 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:14:10.439 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:14:10.442 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:14:40.520 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:14:40.524 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:15:10.600 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:15:10.602 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:15:40.682 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:15:40.685 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:16:10.756 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:16:10.759 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:16:40.849 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:16:40.852 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:17:10.912 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:17:10.916 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:17:40.989 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:17:40.993 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:18:11.224 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:18:11.227 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:18:41.354 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:18:41.357 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:19:11.483 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:19:11.485 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:19:41.630 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:19:41.634 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:20:11.769 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:20:11.786 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:20:41.916 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:20:41.919 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:21:12.045 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:21:12.048 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:21:42.184 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:21:42.187 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:22:12.297 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:22:12.300 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:22:42.443 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:22:42.446 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:23:12.515 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:23:12.518 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:23:42.586 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:23:42.588 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:24:12.668 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:24:12.675 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:24:42.747 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:24:42.750 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:25:12.819 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:25:12.822 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:25:42.892 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:25:42.899 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:26:12.957 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:26:12.960 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:26:43.040 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:26:43.043 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:27:13.114 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:27:13.117 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:27:43.200 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:27:43.203 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:28:13.273 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:28:13.277 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:28:43.344 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:28:43.347 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:29:13.485 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:29:13.489 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:29:43.565 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:29:43.568 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:30:13.729 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:30:13.733 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:30:43.811 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:30:43.814 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:31:13.881 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:31:13.884 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:31:44.027 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:31:44.030 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:32:14.094 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:32:14.097 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:32:44.245 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:32:44.249 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:33:14.375 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:33:14.379 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:33:44.556 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:33:44.558 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:34:14.693 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:34:14.696 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:34:44.874 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:34:44.878 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:35:14.947 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:35:14.950 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:35:45.136 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:35:45.139 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:36:15.265 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:36:15.268 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:36:45.397 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:36:45.400 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:37:15.508 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:37:15.511 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:37:45.638 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:37:45.641 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:38:15.743 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:38:15.745 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:38:45.889 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:38:45.892 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:39:15.954 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:39:15.956 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:39:46.109 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:39:46.112 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:40:16.239 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:40:16.243 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:40:46.345 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:40:46.348 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:41:16.484 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:41:16.487 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:41:46.589 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:41:46.591 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:42:16.746 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:42:16.749 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:42:46.832 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:42:46.835 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:43:16.954 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:43:16.958 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:43:47.068 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:43:47.071 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:44:17.132 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:44:17.135 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:44:47.270 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:44:47.273 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:45:17.336 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:45:17.339 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:45:47.463 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:45:47.466 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:46:17.593 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:46:17.597 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:46:47.740 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:46:47.744 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:47:17.854 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:47:17.857 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:47:47.994 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:47:47.999 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:48:18.055 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:48:18.058 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:48:48.203 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:48:48.207 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:49:18.290 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:49:18.293 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:49:48.371 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:49:48.374 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:50:18.536 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:50:18.539 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:50:48.664 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:50:48.668 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:51:18.802 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:51:18.805 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:51:48.983 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:51:48.986 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:52:19.062 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:52:19.066 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:52:49.126 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:52:49.129 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:53:19.190 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:53:19.193 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:53:49.263 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:53:49.266 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:54:19.361 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:54:19.364 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:54:49.523 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:54:49.526 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:55:19.598 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:55:19.601 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:55:49.662 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:55:49.665 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:56:19.726 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:56:19.729 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:56:49.791 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:56:49.794 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:57:19.857 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:57:19.861 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:57:49.961 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:57:49.965 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:58:20.034 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:58:20.039 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:58:50.103 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:58:50.106 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:59:20.163 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:59:20.166 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:59:50.234 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T04:59:50.237 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:00:20.297 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:00:20.300 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:00:50.386 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:00:50.389 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:01:20.458 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:01:20.462 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:01:50.521 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:01:50.524 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:02:20.584 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:02:20.587 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:02:50.647 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:02:50.650 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:03:20.710 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:03:20.713 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:03:50.775 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:03:50.778 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:04:20.850 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:04:20.855 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:04:51.010 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:04:51.013 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:05:21.085 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:05:21.088 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:05:51.171 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:05:51.175 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:06:21.242 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:06:21.246 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:06:51.305 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:06:51.309 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:07:21.444 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:07:21.447 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:07:51.523 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:07:51.526 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:08:21.669 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:08:21.673 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:08:51.742 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:08:51.745 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:09:21.882 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:09:21.884 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:09:51.994 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:09:51.997 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:10:22.116 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:10:22.120 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:10:52.297 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:10:52.300 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:11:22.426 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:11:22.429 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:11:52.485 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:11:52.490 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:12:22.552 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:12:22.555 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:12:52.615 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:12:52.618 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:13:22.687 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:13:22.691 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:13:52.835 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:13:52.838 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:14:23.000 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:14:23.003 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:14:53.062 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:14:53.066 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:15:23.208 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:15:23.212 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:15:53.322 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:15:53.325 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:16:23.443 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:16:23.447 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:16:53.557 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:16:53.560 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:17:23.687 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:17:23.690 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:17:53.792 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:17:53.796 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:18:23.905 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:18:23.908 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:18:54.044 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:18:54.047 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:19:24.157 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:19:24.160 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:19:54.317 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:19:54.320 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:20:24.401 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:20:24.404 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:20:54.465 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:20:54.468 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:21:24.603 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:21:24.606 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:21:54.683 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:21:54.686 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:22:24.747 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:22:24.750 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:22:54.811 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:22:54.813 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:23:24.882 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:23:24.885 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:23:54.945 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:23:54.947 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:24:25.002 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:24:25.005 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:24:55.066 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:24:55.070 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:25:25.143 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:25:25.146 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:25:55.198 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:25:55.201 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:26:25.280 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:26:25.284 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:26:55.342 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:26:55.345 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:27:25.397 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:27:25.400 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:27:55.460 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:27:55.463 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:28:25.525 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:28:25.528 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:28:55.672 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:28:55.675 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:29:25.753 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:29:25.756 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:29:55.892 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:29:55.895 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:30:26.013 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:30:26.016 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:30:56.160 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:30:56.164 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:31:26.290 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:31:26.293 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:31:56.428 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:31:56.431 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:32:26.541 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:32:26.545 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:32:56.680 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:32:56.683 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:33:26.785 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:33:26.791 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:33:56.932 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:33:56.935 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:34:27.046 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:34:27.050 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:34:57.193 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:34:57.197 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:35:27.308 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:35:27.311 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:35:57.438 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:35:57.443 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:36:27.577 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:36:27.580 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:36:57.699 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:36:57.702 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:37:27.822 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:37:27.825 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:37:57.975 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:37:57.978 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:38:28.088 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:38:28.091 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:38:58.246 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:38:58.249 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:39:28.315 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:39:28.319 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:39:58.471 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:39:58.475 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:40:28.602 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:40:28.606 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:40:58.773 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:40:58.777 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:41:28.870 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:41:28.873 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:41:59.024 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:41:59.027 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:42:29.173 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:42:29.177 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:42:59.310 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:42:59.313 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:43:29.449 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:43:29.452 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:43:59.586 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:43:59.590 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:44:29.717 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:44:29.720 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:44:59.855 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:44:59.859 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:45:29.978 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:45:29.981 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:46:00.140 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:46:00.143 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:46:30.202 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:46:30.205 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:47:00.342 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:47:00.345 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:47:30.438 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:47:30.441 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:48:00.594 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:48:00.597 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:48:30.724 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:48:30.727 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:49:00.842 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:49:00.845 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:49:30.970 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:49:30.973 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:50:01.048 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:50:01.051 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:50:31.186 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:50:31.189 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:51:01.273 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:51:01.276 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:51:31.338 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:51:31.341 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:52:01.404 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:52:01.409 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:52:31.465 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:52:31.468 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:53:01.611 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:53:01.616 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:53:31.717 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:53:31.719 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:54:01.793 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:54:01.796 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:54:31.859 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:54:31.863 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:55:01.923 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:55:01.928 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:55:31.987 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:55:31.990 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:56:02.117 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:56:02.120 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:56:32.254 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:56:32.258 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:57:02.378 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:57:02.380 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:57:32.533 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:57:32.537 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:58:02.655 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:58:02.659 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:58:32.720 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:58:32.724 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:59:02.841 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:59:02.844 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:59:32.904 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T05:59:32.907 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:00:03.053 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:00:03.056 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:00:33.179 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:00:33.181 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:01:03.296 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:01:03.300 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:01:33.594 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:01:33.598 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:02:03.657 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:02:03.660 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:02:33.822 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:02:33.825 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:03:03.984 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:03:03.987 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:03:34.081 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:03:34.084 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:04:04.203 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:04:04.206 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:04:34.367 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:04:34.370 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:05:04.513 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:05:04.516 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:05:34.602 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:05:34.606 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:06:04.748 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:06:04.752 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:06:34.837 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:06:34.841 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:07:04.896 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:07:04.899 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:07:34.973 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:07:34.975 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:08:05.032 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:08:05.036 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:08:35.091 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:08:35.094 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:09:05.153 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:09:05.156 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:09:35.234 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:09:35.237 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:10:05.301 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:10:05.303 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:10:35.380 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:10:35.383 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:11:05.460 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:11:05.464 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:11:35.524 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:11:35.527 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:12:05.589 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:12:05.592 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:12:35.785 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:12:35.788 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:13:05.850 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:13:05.854 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:13:35.913 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:13:35.917 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:14:05.988 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:14:05.991 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:14:36.051 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:14:36.059 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:15:06.123 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:15:06.126 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:15:36.184 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:15:36.187 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:16:06.260 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:16:06.263 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:16:36.323 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:16:36.326 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:17:06.514 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:17:06.518 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:17:36.577 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:17:36.584 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:18:06.647 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:18:06.650 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:18:36.807 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:18:36.810 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:19:06.870 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:19:06.873 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:19:36.934 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:19:36.938 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:20:06.998 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:20:07.002 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:20:37.071 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:20:37.074 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:21:07.141 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:21:07.145 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:21:37.288 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:21:37.291 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:22:07.360 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:22:07.363 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:22:37.474 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:22:37.477 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:23:07.854 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:23:07.858 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:23:37.917 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:23:37.920 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:24:07.990 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:24:07.993 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:24:38.054 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:24:38.057 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:25:08.142 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:25:08.145 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:25:38.216 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:25:38.219 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:26:08.311 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:26:08.314 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:26:38.386 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:26:38.389 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:27:08.449 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:27:08.452 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:27:38.519 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:27:38.523 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:28:08.603 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:28:08.607 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:28:38.696 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:28:38.700 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:29:08.759 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:29:08.763 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:29:38.823 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:29:38.827 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:30:08.888 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:30:08.891 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:30:38.975 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:30:38.978 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:31:09.049 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:31:09.052 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:31:39.138 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:31:39.141 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:32:09.249 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:32:09.252 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:32:39.317 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:32:39.320 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:33:09.381 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:33:09.385 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:33:39.447 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:33:39.450 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:34:09.517 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:34:09.521 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:34:39.583 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:34:39.586 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:35:09.646 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:35:09.650 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:35:39.710 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:35:39.713 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:36:09.775 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:36:09.778 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:36:39.854 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:36:39.857 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:37:09.927 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:37:09.930 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:37:40.006 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:37:40.009 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:38:10.069 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:38:10.072 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:38:40.150 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:38:40.153 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:39:10.204 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:39:10.207 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:39:40.268 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:39:40.271 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:40:10.332 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:40:10.335 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:40:40.421 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:40:40.424 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:41:10.512 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:41:10.515 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:41:40.597 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:41:40.600 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:42:10.664 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:42:10.667 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:42:40.730 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:42:40.732 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:43:10.793 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:43:10.797 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:43:40.865 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:43:40.870 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:44:10.930 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:44:10.934 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:44:41.012 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:44:41.015 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:45:11.075 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:45:11.081 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:45:41.215 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:45:41.219 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:46:11.310 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:46:11.314 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:46:41.457 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:46:41.460 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:47:11.561 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:47:11.565 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:47:41.700 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:47:41.704 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:48:11.840 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:48:11.844 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:48:41.961 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:48:41.965 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:49:12.108 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:49:12.112 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:49:42.263 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:49:42.267 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:50:12.360 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:50:12.363 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:50:42.516 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:50:42.520 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:51:12.670 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:51:12.674 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:51:42.775 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:51:42.777 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:52:12.896 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:52:12.899 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:52:43.044 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:52:43.047 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:53:13.199 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:53:13.205 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:53:43.279 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:53:43.282 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:54:13.395 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:54:13.398 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:54:43.458 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:54:43.462 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:55:13.597 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:55:13.601 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:55:43.727 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:55:43.731 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:56:13.808 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:56:13.811 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:56:43.887 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:56:43.890 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:57:13.960 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:57:13.963 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:57:44.024 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:57:44.029 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:58:14.086 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:58:14.090 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:58:44.162 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:58:44.165 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:59:14.239 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:59:14.243 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:59:44.302 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T06:59:44.305 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:00:14.374 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:00:14.377 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:00:44.438 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:00:44.441 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:01:14.504 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:01:14.511 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:01:44.572 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:01:44.576 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:02:14.665 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:02:14.670 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:02:44.730 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:02:44.734 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:03:14.805 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:03:14.809 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:03:44.869 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:03:44.873 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:04:14.948 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:04:14.953 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:04:45.013 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:04:45.016 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:05:15.077 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:05:15.080 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:05:45.141 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:05:45.144 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:06:15.212 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:06:15.215 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:06:45.284 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:06:45.287 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:07:15.369 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:07:15.373 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:07:45.435 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:07:45.439 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:08:15.499 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:08:15.503 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:08:45.562 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:08:45.565 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:09:15.625 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:09:15.628 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:09:45.690 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:09:45.693 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:10:15.753 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:10:15.757 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:10:45.818 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:10:45.821 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:11:15.874 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:11:15.877 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:11:45.937 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:11:45.940 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:12:16.006 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:12:16.010 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:12:46.083 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:12:46.087 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:13:16.146 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:13:16.150 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:13:46.215 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:13:46.218 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:14:16.278 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:14:16.281 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:14:46.343 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:14:46.346 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:15:16.405 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:15:16.409 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:15:46.477 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:15:46.481 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:16:16.565 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:16:16.569 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:16:46.638 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:16:46.642 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:17:16.711 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:17:16.714 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:17:46.867 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:17:46.870 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:18:16.940 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:18:16.942 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:18:47.019 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:18:47.021 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:19:17.102 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:19:17.104 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:19:47.162 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:19:47.165 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:20:17.234 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:20:17.241 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:20:47.298 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:20:47.301 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:21:17.370 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:21:17.373 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:21:47.442 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:21:47.445 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:22:17.638 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:22:17.642 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:22:47.702 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:22:47.705 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:23:17.766 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:23:17.769 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:23:47.839 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:23:47.842 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:24:17.896 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:24:17.899 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:24:47.961 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:24:47.964 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:25:18.024 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:25:18.027 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:25:48.089 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:25:48.091 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:26:18.161 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:26:18.164 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:26:48.237 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:26:48.246 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:27:18.317 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:27:18.320 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:27:48.379 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:27:48.382 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:28:18.443 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:28:18.446 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:28:48.507 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:28:48.510 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:29:18.563 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:29:18.566 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:29:48.679 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:29:48.682 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:30:18.741 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:30:18.744 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:30:48.797 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:30:48.801 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:31:18.866 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:31:18.869 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:31:48.926 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:31:48.929 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:32:18.993 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:32:18.996 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:32:49.081 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:32:49.084 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:33:19.151 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:33:19.160 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:33:49.309 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:33:49.312 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:34:19.398 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:34:19.402 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:34:49.544 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:34:49.548 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:35:19.658 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:35:19.663 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:35:49.780 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:35:49.783 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:36:19.902 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:36:19.905 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:36:49.965 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:36:49.968 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:37:20.036 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:37:20.040 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:37:50.120 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:37:50.123 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:38:20.309 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:38:20.312 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:38:50.377 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:38:50.432 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:39:20.491 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:39:20.494 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:39:50.712 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:39:50.716 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:40:20.775 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:40:20.778 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:40:50.857 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:40:50.860 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:41:20.921 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:41:20.924 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:41:50.976 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:41:50.979 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:42:21.040 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:42:21.043 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:42:51.104 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:42:51.107 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:43:21.176 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:43:21.179 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:43:51.237 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:43:51.240 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:44:21.300 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:44:21.303 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:44:51.356 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:44:51.359 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:45:21.424 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:45:21.427 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:45:51.491 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:45:51.494 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:46:21.553 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:46:21.556 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:46:51.625 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:46:51.628 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:47:21.690 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:47:21.693 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:47:51.753 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:47:51.756 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:48:21.842 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:48:21.845 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:48:51.915 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:48:51.919 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:49:21.979 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:49:21.982 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:49:52.058 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:49:52.062 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:50:22.133 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:50:22.136 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:50:52.197 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:50:52.200 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:51:22.265 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:51:22.268 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:51:52.346 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:51:52.349 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:52:22.423 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:52:22.426 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:52:52.511 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:52:52.514 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:53:22.594 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:53:22.597 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:53:52.667 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:53:52.669 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:54:22.721 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:54:22.724 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:54:52.786 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:54:52.790 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:55:22.850 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:55:22.853 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:55:52.937 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:55:52.940 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:56:23.020 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:56:23.023 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:56:53.085 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:56:53.088 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:57:23.148 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:57:23.152 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:57:53.212 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:57:53.216 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:58:23.371 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:58:23.375 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:58:53.492 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:58:53.496 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:59:23.640 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:59:23.643 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:59:53.986 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T07:59:53.993 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:00:24.044 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:00:24.047 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:00:54.189 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:00:54.193 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:01:24.302 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:01:24.305 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:01:54.450 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:01:54.453 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:02:24.580 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:02:24.583 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:02:54.703 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:02:54.706 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:03:24.849 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:03:24.852 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:03:54.988 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:03:54.991 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:04:25.227 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:04:25.230 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:04:55.339 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:04:55.342 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:05:25.512 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:05:25.516 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:05:55.576 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:05:55.579 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:06:25.673 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:06:25.677 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:06:55.820 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:06:55.823 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:07:25.883 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:07:25.886 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:07:56.037 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:07:56.040 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:08:26.118 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:08:26.121 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:08:56.212 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:08:56.216 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:09:26.278 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:09:26.281 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:09:56.342 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:09:56.345 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:10:26.406 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:10:26.409 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:10:56.471 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:10:56.474 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:11:26.575 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:11:26.579 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:11:56.722 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:11:56.725 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:12:26.853 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:12:26.855 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:12:56.924 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:12:56.926 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:13:26.988 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:13:26.991 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:13:57.135 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:13:57.138 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:14:27.307 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:14:27.311 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:14:57.462 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:14:57.465 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:15:27.593 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:15:27.596 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:15:57.739 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:15:57.742 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:16:27.861 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:16:27.865 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:16:57.966 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:16:57.969 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:17:28.113 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:17:28.116 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:17:58.243 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:17:58.246 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:18:28.348 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:18:28.351 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:18:58.497 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:18:58.500 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:19:28.568 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:19:28.571 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:19:58.732 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:19:58.735 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:20:28.879 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:20:28.883 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:20:59.019 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:20:59.022 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:21:29.139 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:21:29.142 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:21:59.287 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:21:59.290 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:22:29.442 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:22:29.445 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:22:59.523 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:22:59.526 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:23:29.652 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:23:29.655 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:23:59.727 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:23:59.730 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:24:29.791 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:24:29.795 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:24:59.873 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:24:59.877 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:25:29.939 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:25:29.943 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:25:59.994 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:25:59.998 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:26:30.058 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:26:30.061 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:27:00.130 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:27:00.133 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:27:30.228 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:27:30.232 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:28:00.292 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:28:00.296 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:28:30.354 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:28:30.358 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:29:00.442 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:29:00.652 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:29:30.727 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:29:30.730 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:30:00.811 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:30:00.818 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:30:30.905 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:30:30.908 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:31:00.977 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:31:00.980 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:31:31.042 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:31:31.070 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:32:01.131 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:32:01.135 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:32:31.202 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:32:31.205 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:33:01.284 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:33:01.287 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:33:31.382 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:33:31.385 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:34:01.454 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:34:01.457 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:34:31.525 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:34:31.529 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:35:01.596 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:35:01.601 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:35:31.662 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:35:31.664 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:36:01.726 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:36:01.729 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:36:31.804 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:36:31.807 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:37:01.868 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:37:01.871 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:37:31.935 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:37:31.939 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:38:02.008 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:38:02.011 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:38:32.080 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:38:32.083 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:39:02.238 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:39:02.241 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:39:32.376 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:39:32.379 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:40:02.473 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:40:02.477 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:40:32.612 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:40:32.616 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:41:02.750 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:41:02.754 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:41:32.888 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:41:32.891 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:42:02.954 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:42:02.956 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:42:33.114 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:42:33.118 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:43:03.204 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:43:03.207 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:43:33.334 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:43:33.337 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:44:03.403 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:44:03.407 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:44:33.500 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:44:33.503 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:45:03.572 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:45:03.576 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:45:33.655 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:45:33.658 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:46:03.734 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:46:03.737 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:46:33.808 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:46:33.811 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:47:03.870 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:47:03.873 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:47:33.943 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:47:33.947 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:48:04.007 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:48:04.010 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:48:34.079 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:48:34.083 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:49:04.169 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:49:04.172 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:49:34.242 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:49:34.245 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:50:04.315 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:50:04.318 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:50:34.378 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:50:34.381 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:51:04.438 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:51:04.441 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:51:34.602 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:51:34.605 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:52:04.748 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:52:04.752 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:52:34.854 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:52:34.856 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:53:04.992 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:53:04.995 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:53:35.097 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:53:35.100 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:54:05.177 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:54:05.181 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:54:35.251 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:54:35.253 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:55:05.371 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:55:05.375 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:55:35.452 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:55:35.455 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:56:05.607 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:56:05.610 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:56:35.754 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:56:35.758 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:57:05.851 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:57:05.854 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:57:36.015 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:57:36.018 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:58:06.152 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:58:06.155 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:58:36.274 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:58:36.277 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:59:06.506 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:59:06.509 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:59:36.670 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T08:59:36.674 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:00:06.792 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:00:06.795 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:00:36.922 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:00:36.926 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:01:06.994 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:01:06.999 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:01:37.059 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:01:37.062 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:02:07.122 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:02:07.125 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:02:37.195 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:02:37.198 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:03:07.258 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:03:07.261 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:03:37.339 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:03:37.342 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:04:07.419 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:04:07.422 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:04:37.510 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:04:37.513 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:05:07.571 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:05:07.575 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:05:37.647 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:05:37.650 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:06:07.715 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:06:07.718 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:06:37.791 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:06:37.794 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:07:07.867 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:07:07.870 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:07:37.962 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:07:37.983 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:08:08.102 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:08:08.105 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:08:38.258 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:08:38.261 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:09:08.379 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:09:08.382 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:09:38.452 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:09:38.455 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:10:08.599 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:10:08.602 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:10:38.905 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:10:38.908 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:11:08.970 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:11:08.973 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:11:39.043 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:11:39.046 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:12:09.106 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:12:09.109 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:12:39.179 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:12:39.182 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:13:09.235 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:13:09.238 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:13:39.307 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:13:39.310 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:14:09.383 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:14:09.386 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:14:39.453 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:14:39.460 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:15:09.511 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:15:09.514 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:15:39.589 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:15:39.592 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:16:09.654 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:16:09.658 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:16:39.743 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:16:39.746 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:17:09.807 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:17:09.811 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:17:39.944 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:17:39.947 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:18:10.094 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:18:10.097 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:18:40.206 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:18:40.215 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:19:10.294 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:19:10.298 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:19:40.452 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:19:40.456 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:20:10.597 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:20:10.600 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:20:40.678 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:20:40.682 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:21:10.804 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:21:11.419 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:21:41.479 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:21:41.482 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:22:11.601 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:22:11.605 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:22:41.739 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:22:41.742 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:23:11.894 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:23:11.897 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:23:41.958 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:23:41.961 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:24:12.121 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:24:12.124 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:24:42.285 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:24:42.288 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:25:12.365 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:25:12.368 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:25:42.504 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:25:42.507 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:26:12.651 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:26:12.654 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:26:42.727 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:26:42.730 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:27:12.887 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:27:12.890 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:27:43.042 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:27:43.045 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:28:13.104 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:28:13.108 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:28:43.169 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:28:43.173 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:29:13.331 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:29:13.334 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:29:43.405 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:29:43.408 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:30:13.542 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:30:13.545 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:30:43.689 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:30:43.693 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:31:13.820 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:31:13.824 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:31:43.950 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:31:43.953 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:32:14.081 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:32:14.084 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:32:44.159 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:32:44.162 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:33:14.224 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:33:14.227 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:33:44.293 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:33:44.296 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:34:14.382 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:34:14.385 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:34:44.463 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:34:44.467 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:35:14.520 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:35:14.523 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:35:44.583 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:35:44.586 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:36:14.647 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:36:14.650 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:36:44.712 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:36:44.715 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:37:14.775 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:37:14.778 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:37:44.847 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:37:44.850 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:38:14.916 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:38:14.919 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:38:44.993 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:38:44.995 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:39:15.076 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:39:15.079 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:39:45.146 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:39:45.149 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:40:15.279 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:40:15.283 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:40:45.426 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:40:45.429 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:41:15.591 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:41:15.594 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:41:45.719 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:41:45.722 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:42:15.859 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:42:15.862 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:42:45.972 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:42:45.975 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:43:16.102 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:43:16.106 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:43:46.233 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:43:46.236 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:44:16.363 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:44:16.366 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:44:46.519 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:44:46.522 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:45:16.585 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:45:16.589 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:45:46.658 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:45:46.661 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:46:16.719 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:46:16.723 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:46:46.819 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:46:46.823 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:47:16.879 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:47:16.903 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:47:46.972 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:47:46.975 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:48:17.044 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:48:17.047 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:48:47.108 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:48:47.111 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:49:17.184 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:49:17.187 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:49:47.256 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:49:47.259 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:50:17.336 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:50:17.339 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:50:47.457 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:50:47.460 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:51:17.597 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:51:17.600 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:51:47.692 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:51:47.695 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:52:17.823 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:52:17.826 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:52:47.920 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:52:47.923 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:53:18.063 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:53:18.066 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:53:48.213 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:53:48.216 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:54:18.293 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:54:18.296 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:54:48.365 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:54:48.368 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:55:18.521 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:55:18.524 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:55:48.608 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:55:48.612 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:56:18.723 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:56:18.726 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:56:48.802 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:56:48.805 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:57:18.882 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:57:18.885 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:57:48.963 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:57:48.966 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:58:19.027 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:58:19.030 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:58:49.099 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:58:49.103 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:59:19.164 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:59:19.168 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:59:49.250 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T09:59:49.253 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:00:19.342 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:00:19.345 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:00:49.419 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:00:49.422 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:01:19.495 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:01:19.499 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:01:49.568 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:01:49.571 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:02:19.635 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:02:19.638 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:02:49.712 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:02:49.715 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:03:19.793 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:03:19.797 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:03:49.883 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:03:49.886 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:04:20.031 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:04:20.035 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:04:50.112 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:04:50.115 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:05:20.183 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:05:20.186 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:05:50.322 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:05:50.325 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:06:20.443 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:06:20.446 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:06:50.574 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:06:50.577 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:07:20.712 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:07:20.715 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:07:50.775 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:07:50.778 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:08:20.836 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:08:20.840 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:08:50.934 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:08:50.937 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:09:21.002 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:09:21.006 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:09:51.071 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:09:51.074 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:10:21.137 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:10:21.141 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:10:51.216 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:10:51.219 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:11:21.286 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:11:21.289 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:11:51.368 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:11:51.371 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:12:21.424 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:12:21.427 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:12:51.484 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:12:51.487 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:13:21.556 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:13:21.560 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:13:51.636 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:13:51.640 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:14:21.699 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:14:21.702 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:14:51.821 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:14:51.823 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:15:21.893 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:15:21.896 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:15:52.015 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:15:52.018 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:16:22.146 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:16:22.149 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:16:52.292 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:16:52.296 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:17:22.357 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:17:22.360 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:17:52.485 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:17:52.488 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:18:22.549 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:18:22.551 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:18:52.679 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:18:52.682 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:19:22.801 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:19:22.804 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:19:52.878 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:19:52.881 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:20:23.020 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:20:23.023 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:20:53.142 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:20:53.145 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:21:23.305 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:21:23.308 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:21:53.377 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:21:53.380 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:22:23.466 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:22:23.469 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:22:53.546 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:22:53.550 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:23:23.602 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:23:23.605 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:23:53.731 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:23:53.734 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:24:23.843 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:24:23.847 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:24:54.003 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:24:54.006 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:25:24.077 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:25:24.080 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:25:54.218 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:25:54.221 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:26:24.365 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:26:24.368 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:26:54.495 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:26:54.499 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:27:24.567 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:27:24.570 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:27:54.631 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:27:54.634 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:28:24.694 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:28:24.697 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:28:54.768 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:28:54.775 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:29:24.837 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:29:24.840 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:29:54.913 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:29:54.915 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:30:25.056 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:30:25.059 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:30:55.211 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:30:55.215 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:31:25.317 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:31:25.320 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:31:55.446 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:31:55.449 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:32:25.569 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:32:25.572 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:32:55.634 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:32:55.637 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:33:25.696 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:33:25.701 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:33:55.783 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:33:55.786 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:34:25.865 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:34:25.868 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:34:55.945 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:34:55.949 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:35:26.009 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:35:26.013 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:35:56.098 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:35:56.102 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:36:26.179 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:36:26.182 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:36:56.251 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:36:56.254 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:37:26.324 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:37:26.328 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:37:56.411 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:37:56.414 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:38:26.469 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:38:26.472 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:38:56.533 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:38:56.536 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:39:26.610 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:39:26.613 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:39:56.686 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:39:56.689 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:40:26.830 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:40:26.833 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:40:56.919 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:40:56.922 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:41:27.082 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:41:27.085 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:41:57.188 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:41:57.191 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:42:27.353 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:42:27.356 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:42:57.465 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:42:57.468 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:43:27.612 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:43:27.617 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:43:57.751 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:43:57.754 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:44:27.815 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:44:27.818 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:44:57.898 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:44:57.901 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:45:28.051 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:45:28.055 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:45:58.181 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:45:58.183 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:46:28.352 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:46:28.355 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:46:58.416 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:46:58.420 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:47:28.483 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:47:28.486 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:47:58.546 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:47:58.550 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:48:28.606 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:48:28.608 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:48:58.675 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:48:58.678 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:49:28.731 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:49:28.735 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:49:58.813 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:49:58.816 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:50:28.902 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:50:28.904 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:50:58.966 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:50:58.969 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:51:29.112 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:51:29.115 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:51:59.199 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:51:59.203 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:52:29.329 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:52:29.332 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:52:59.434 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:52:59.437 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:53:29.581 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:53:29.584 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:53:59.703 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:53:59.707 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:54:29.825 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:54:29.828 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:54:59.888 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:54:59.891 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:55:30.026 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:55:30.029 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:56:00.108 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:56:00.111 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:56:30.196 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:56:30.199 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:57:00.278 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:57:00.281 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:57:30.338 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:57:30.341 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:58:00.406 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:58:00.410 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:58:30.465 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:58:30.469 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:59:00.537 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:59:00.540 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:59:30.600 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T10:59:30.603 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:00:00.666 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:00:00.669 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:00:30.748 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:00:30.751 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:01:00.837 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:01:00.840 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:01:30.909 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:01:30.912 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:02:00.982 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:02:00.985 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:02:31.065 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:02:31.068 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:03:01.249 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:03:01.252 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:03:31.342 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:03:31.345 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:04:01.414 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:04:01.417 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:04:31.470 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:04:31.473 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:05:01.535 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:05:01.538 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:05:31.600 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:05:31.603 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:06:01.743 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:06:01.746 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:06:31.814 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:06:31.817 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:07:01.894 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:07:01.897 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:07:31.982 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:07:31.985 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:08:02.120 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:08:02.123 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:08:32.234 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:08:32.238 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:09:02.389 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:09:02.393 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:09:32.544 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:09:32.548 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:10:02.607 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:10:02.610 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:10:32.755 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:10:32.758 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:11:02.893 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:11:02.896 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:11:33.032 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:11:33.036 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:12:03.204 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:12:03.207 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:12:33.334 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:12:33.337 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:13:03.426 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:13:03.429 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:13:33.490 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:13:33.493 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:14:03.562 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:14:03.565 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:14:33.627 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:14:33.630 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:15:03.710 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:15:03.714 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:15:33.771 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:15:33.776 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:16:03.845 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:16:03.848 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:16:33.911 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:16:33.915 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:17:03.980 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:17:03.983 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:17:34.037 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:17:34.050 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:18:04.151 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:18:04.154 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:18:34.215 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:18:34.219 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:19:04.296 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:19:04.298 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:19:34.358 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:19:34.366 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:20:04.433 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:20:04.436 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:20:34.560 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:20:34.563 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:21:04.723 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:21:04.726 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:21:34.862 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:21:34.865 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:22:04.983 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:22:04.986 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:22:35.114 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:22:35.116 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:23:05.184 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:23:05.187 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:23:35.323 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:23:35.326 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:24:05.428 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:24:05.431 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:24:35.560 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:24:35.563 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:25:05.723 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:25:05.727 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:25:35.861 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:25:35.865 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:26:05.966 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:26:05.969 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:26:36.079 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:26:36.082 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:27:06.147 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:27:06.150 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:27:36.231 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:27:36.233 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:28:06.311 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:28:06.314 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:28:36.374 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:28:36.377 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:29:06.438 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:29:06.441 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:29:36.501 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:29:36.504 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:30:06.574 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:30:06.577 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:30:36.662 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:30:36.666 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:31:06.736 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:31:06.739 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:31:36.846 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:31:36.849 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:32:06.910 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:32:06.913 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:32:36.973 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:32:36.976 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:33:07.036 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:33:07.039 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:33:37.109 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:33:37.112 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:34:07.178 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:34:07.182 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:34:37.242 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:34:37.246 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:35:07.382 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:35:07.384 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:35:37.570 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:35:37.573 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:36:07.718 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:36:07.721 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:36:37.821 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:36:37.825 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:37:07.936 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:37:07.939 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:37:38.065 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:37:38.069 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:38:08.229 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:38:08.232 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:38:38.306 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:38:38.309 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:39:08.379 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:39:08.382 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:39:38.460 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:39:38.464 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:40:08.525 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:40:08.527 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:40:38.589 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:40:38.592 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:41:08.653 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:41:08.656 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:41:38.716 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:41:38.719 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:42:08.798 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:42:08.801 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:42:38.943 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:42:38.945 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:43:09.001 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:43:09.004 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:43:39.073 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:43:39.076 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:44:09.215 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:44:09.218 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:44:39.280 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:44:39.283 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:45:09.350 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:45:09.352 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:45:39.472 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:45:39.475 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:46:09.621 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:46:09.624 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:46:39.684 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:46:39.686 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:47:09.756 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:47:09.759 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:47:39.829 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:47:39.832 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:48:09.900 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:48:09.903 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:48:40.023 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:48:40.026 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:49:10.177 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:49:10.180 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:49:40.308 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:49:40.311 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:50:10.446 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:50:10.449 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:50:40.569 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:50:40.572 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:51:10.674 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:51:10.676 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:51:40.821 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:51:40.824 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:52:10.925 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:52:10.929 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:52:40.986 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:52:40.990 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:53:11.050 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:53:11.053 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:53:41.151 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:53:41.155 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:54:11.330 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:54:11.333 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:54:41.394 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:54:41.397 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:55:11.465 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:55:11.468 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:55:41.530 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:55:41.533 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:56:11.602 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:56:11.605 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:56:41.666 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:56:41.669 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:57:11.747 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:57:11.750 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:57:41.807 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:57:41.810 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:58:11.900 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:58:11.903 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:58:41.965 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:58:41.969 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:59:12.036 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:59:12.039 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:59:42.111 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T11:59:42.114 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:00:12.190 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:00:12.193 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:00:42.256 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:00:42.259 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:01:12.337 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:01:12.342 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:01:42.400 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:01:42.403 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:02:12.538 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:02:12.541 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:02:42.601 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:02:42.604 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:03:12.667 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:03:12.670 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:03:42.731 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:03:42.734 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:04:12.804 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:04:12.807 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:04:42.889 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:04:42.894 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:05:12.977 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:05:12.980 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:05:43.032 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:05:43.035 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:06:13.102 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:06:13.109 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:06:43.242 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:06:43.245 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:07:13.350 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:07:13.353 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:07:43.510 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:07:43.513 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:08:13.574 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:08:13.576 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:08:43.679 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:08:43.682 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:09:13.818 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:09:13.821 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:09:44.039 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:09:44.043 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:10:14.116 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:10:14.119 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:10:44.283 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:10:44.287 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:11:14.430 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:11:14.433 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:11:44.569 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:11:44.572 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:12:14.649 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:12:14.652 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:12:44.831 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:12:44.834 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:13:14.925 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:13:14.928 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:13:45.063 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:13:45.067 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:14:15.203 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:14:15.206 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:14:45.358 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:14:45.362 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:15:15.455 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:15:15.459 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:15:45.593 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:15:45.596 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:16:15.731 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:16:15.734 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:16:45.818 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:16:45.821 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:17:15.883 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:17:15.886 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:17:45.946 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:17:45.949 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:18:16.011 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:18:16.014 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:18:46.074 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:18:46.077 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:19:16.138 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:19:16.141 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:19:46.202 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:19:46.206 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:20:16.288 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:20:16.291 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:20:46.352 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:20:46.355 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:21:16.424 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:21:16.427 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:21:46.504 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:21:46.508 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:22:16.567 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:22:16.570 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:22:46.714 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:22:46.718 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:23:16.836 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:23:16.839 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:23:46.966 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:23:46.969 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:24:17.213 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:24:17.215 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:24:47.309 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:24:47.312 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:25:17.401 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:25:17.403 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:25:47.536 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:25:47.539 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:26:17.700 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:26:17.703 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:26:47.830 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:26:47.833 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:27:17.960 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:27:17.963 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:27:48.082 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:27:48.085 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:28:18.220 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:28:18.224 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:28:48.282 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:28:48.285 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:29:18.422 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:29:18.425 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:29:48.485 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:29:48.488 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:30:18.649 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:30:18.652 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:30:48.804 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:30:48.808 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:31:18.864 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:31:18.868 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:31:49.007 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:31:49.010 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:32:19.162 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:32:19.165 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:32:49.318 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:32:49.322 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:33:19.373 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:33:19.375 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:33:49.428 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:33:49.431 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:34:19.492 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:34:19.496 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:34:49.557 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:34:49.561 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:35:19.646 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:35:19.649 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:35:49.706 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:35:49.710 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:36:19.767 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:36:19.770 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:36:49.825 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:36:49.829 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:37:19.886 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:37:19.890 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:37:49.978 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:37:49.981 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:38:20.057 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:38:20.060 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:38:50.128 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:38:50.132 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:39:20.193 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:39:20.196 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:39:50.257 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:39:50.261 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:40:20.345 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:40:20.348 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:40:50.408 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:40:50.411 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:41:20.542 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:41:20.545 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:41:50.694 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:41:50.698 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:42:20.754 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:42:20.758 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:42:50.839 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:42:50.843 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:43:20.986 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:43:20.988 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:43:51.040 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:43:51.142 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:44:21.238 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:44:21.241 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:44:51.393 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:44:51.396 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:45:21.515 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:45:21.518 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:45:51.578 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:45:51.582 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:46:21.641 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:46:21.645 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:46:51.706 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:46:51.709 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:47:21.776 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:47:21.779 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:47:51.930 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:47:51.932 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:48:21.990 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:48:21.993 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:48:52.066 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:48:52.069 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:49:22.129 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:49:22.132 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:49:52.192 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:49:52.195 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:50:22.265 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:50:22.268 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:50:52.337 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:50:52.340 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:51:22.410 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:51:22.413 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:51:52.474 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:51:52.479 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:52:22.539 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:52:22.542 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:52:52.602 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:52:52.605 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:53:22.662 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:53:22.665 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:53:52.738 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:53:52.743 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:54:22.806 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:54:22.810 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:54:52.870 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:54:52.873 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:55:22.941 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:55:22.947 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:55:53.022 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:55:53.027 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:56:23.085 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:56:23.089 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:56:53.149 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:56:53.152 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:57:23.236 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:57:23.240 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:57:53.311 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:57:53.314 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:58:23.376 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:58:23.379 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:58:53.439 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:58:53.443 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:59:23.505 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:59:23.509 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:59:53.569 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T12:59:53.573 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T13:00:23.655 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T13:00:23.658 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T13:00:53.746 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T13:00:53.750 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T13:01:23.811 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T13:01:23.814 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T13:01:53.874 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T13:01:53.877 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T13:02:23.955 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T13:02:23.958 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T13:02:54.019 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T13:02:54.022 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T13:03:24.096 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T13:03:24.099 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T13:03:54.156 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T13:03:54.159 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T13:04:24.221 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T13:04:24.225 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T13:04:54.286 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T13:04:54.290 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T13:05:24.351 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T13:05:24.354 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-05-20T13:05:45.133 DEBUG:teuthology.exit:Got signal 15; running 1 handler... 2023-05-20T13:05:45.134 DEBUG:teuthology.task.console_log:Killing console logger for smithi016 2023-05-20T13:05:45.134 DEBUG:teuthology.task.console_log:Killing console logger for smithi132 2023-05-20T13:05:45.134 DEBUG:teuthology.exit:Finished running handlers